2024-09-10T23:44:30.501 INFO:root:teuthology version: 1.2.2.dev2+g3752d38 2024-09-10T23:44:30.530 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-10T23:44:30.632 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/hyelloji-2024-09-10_16:59:47-fs-wip-hemanth-testing-2024-09-10-0723-testing-default-smithi/7899811 branch: wip-hemanth-testing-2024-09-10-0723 description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/legacy wsync/no} objectstore-ec/bluestore-ec-root omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/automatic export-check n/5 replication/default} standby-replay tasks/{0-subvolume/{with-namespace-isolated-and-quota} 1-check-counter 2-scrub/no 3-snaps/no 4-flush/yes 5-quiesce/with-quiesce 6-workunit/suites/dbench}} email: hyelloji@redhat.com first_in_suite: false job_id: '7899811' kernel: client: branch: testing flavor: default kdb: 1 sha1: 2ca44a2f35ca8f36983aaaa46d49312239ef519a ktype: upstream last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi name: hyelloji-2024-09-10_16:59:47-fs-wip-hemanth-testing-2024-09-10-0723-testing-default-smithi no_nested_subset: false openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-hemanth-testing-2024-09-10-0723 ceph: cephfs: ec_profile: - m=2 - k=2 - crush-failure-domain=osd max_mds: 5 session_timeout: 300 standby_replay: true cluster-conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 mds: debug mds: 20 debug mds balancer: 20 debug ms: 2 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mon: mon op complaint time: 120 mon warn on pool no app: false osd: osd op complaint time: 180 conf: global: ms die on skipped message: false mgr: cephsqlite lock renewal timeout: 900000 client mount timeout: 30 debug client: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd shutdown pgref assert: true osd_max_omap_entries_per_request: 10000 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - fs.*is degraded - filesystem is degraded - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - filesystem is offline - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - insufficient standby MDS daemons available - MDS_UP_LESS_THAN_MAX - online, but wants - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - do not have an application enabled - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability - Degraded data redundancy sha1: 8f219c476cb4f26fc5719c661c0323719a45cb30 subvols: create: 2 subvol_options: --namespace-isolated --size 25000000000 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs ceph-fuse: client.0: mount_subvol_num: 0 check-counter: counters: mds: - mds.exported - mds.imported install: ceph: flavor: default sha1: 8f219c476cb4f26fc5719c661c0323719a45cb30 kclient: client.0: mount_subvol_num: 1 mntopts: - ms_mode=legacy - nowsync syntax: v1 selinux: allowlist: - scontext=system_u:system_r:logrotate_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-hemanth-testing-2024-09-10-0723 sha1: 8f219c476cb4f26fc5719c661c0323719a45cb30 owner: scheduled_hyelloji@teuthology priority: 100 repo: https://github.com/ceph/ceph-ci.git roles: - - mon.a - mgr.x - mds.a - mds.d - mds.g - mds.j - osd.0 - osd.3 - osd.6 - osd.9 - client.0 - - mon.b - mgr.y - mds.b - mds.e - mds.h - mds.k - osd.1 - osd.4 - osd.7 - osd.10 - - mon.c - mgr.z - mds.c - mds.f - mds.i - mds.l - osd.2 - osd.5 - osd.8 - osd.11 seed: 6190 sha1: 8f219c476cb4f26fc5719c661c0323719a45cb30 sleep_before_teardown: 0 subset: 479/1024 suite: fs suite_branch: wip-hemanth-testing-2024-09-10-0723 suite_path: /home/teuthworker/src/github.com_ceph_ceph-c_8f219c476cb4f26fc5719c661c0323719a45cb30/qa suite_relpath: qa suite_repo: https://github.com/ceph/ceph-ci.git suite_sha1: 8f219c476cb4f26fc5719c661c0323719a45cb30 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi012.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJIQZTQKJL7T8lW9ZtktMpahtpaDhCutWkHzFTrrxziy83UOI+kV48EbLjQ72pe/d5Oqlr+q39U89IjkCyIesK4= smithi165.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBMBrm+zVe7GDMgo2nPF8iqnBNZPLdKDJqVb8tNFq0K9FTc0SveAIkkwinFWLglvEfn7PMIGMyWY1klqzcMuURNM= smithi178.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBHI+b5Vt0jNcSu33iDayYRTfDRqyB11DuLGPZXtQ+8jIgDXUxJeoloZkoNsprQ94ZtkFXCJN4v/AEN62/5qX2nw= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - cephadm: roleless: false - cephadm.shell: mon.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: mon.a: - ceph fs dump - ceph osd dump - fs.ready: timeout: 300 - kclient: null - exec: mon.a: - ceph fs set cephfs balance_automate true - check-counter: {} - background_exec: mon.a: - while sleep 13; do ceph tell mds.cephfs:0 flush journal; done - quiescer: null - workunit: clients: all: - suites/dbench.sh teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: 3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a timestamp: 2024-09-10_16:59:47 tube: smithi user: hyelloji verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.3401856 2024-09-10T23:44:30.632 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_ceph_ceph-c_8f219c476cb4f26fc5719c661c0323719a45cb30/qa; will attempt to use it 2024-09-10T23:44:30.633 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_ceph_ceph-c_8f219c476cb4f26fc5719c661c0323719a45cb30/qa/tasks 2024-09-10T23:44:30.633 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-09-10T23:44:30.635 INFO:teuthology.task.internal:Checking packages... 2024-09-10T23:44:30.655 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '8f219c476cb4f26fc5719c661c0323719a45cb30' 2024-09-10T23:44:30.655 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-09-10T23:44:30.655 INFO:teuthology.packaging:ref: None 2024-09-10T23:44:30.655 INFO:teuthology.packaging:tag: None 2024-09-10T23:44:30.655 INFO:teuthology.packaging:branch: wip-hemanth-testing-2024-09-10-0723 2024-09-10T23:44:30.655 INFO:teuthology.packaging:sha1: 8f219c476cb4f26fc5719c661c0323719a45cb30 2024-09-10T23:44:30.656 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-hemanth-testing-2024-09-10-0723 2024-09-10T23:44:30.805 INFO:teuthology.task.internal:Found packages for ceph version 19.3.0-4854.g8f219c47 2024-09-10T23:44:30.807 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-09-10T23:44:30.813 INFO:teuthology.task.internal:no buildpackages task found 2024-09-10T23:44:30.813 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-09-10T23:44:30.820 INFO:teuthology.task.internal:Saving configuration 2024-09-10T23:44:30.841 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-09-10T23:44:30.848 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-09-10T23:44:30.880 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi012.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/hyelloji-2024-09-10_16:59:47-fs-wip-hemanth-testing-2024-09-10-0723-testing-default-smithi/7899811', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-10 23:39:00.434413', 'locked_by': 'scheduled_hyelloji@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJIQZTQKJL7T8lW9ZtktMpahtpaDhCutWkHzFTrrxziy83UOI+kV48EbLjQ72pe/d5Oqlr+q39U89IjkCyIesK4='} 2024-09-10T23:44:30.911 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi165.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/hyelloji-2024-09-10_16:59:47-fs-wip-hemanth-testing-2024-09-10-0723-testing-default-smithi/7899811', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-10 23:39:00.438890', 'locked_by': 'scheduled_hyelloji@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBMBrm+zVe7GDMgo2nPF8iqnBNZPLdKDJqVb8tNFq0K9FTc0SveAIkkwinFWLglvEfn7PMIGMyWY1klqzcMuURNM='} 2024-09-10T23:44:30.941 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi178.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/hyelloji-2024-09-10_16:59:47-fs-wip-hemanth-testing-2024-09-10-0723-testing-default-smithi/7899811', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-10 23:39:00.435875', 'locked_by': 'scheduled_hyelloji@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBHI+b5Vt0jNcSu33iDayYRTfDRqyB11DuLGPZXtQ+8jIgDXUxJeoloZkoNsprQ94ZtkFXCJN4v/AEN62/5qX2nw='} 2024-09-10T23:44:30.942 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-09-10T23:44:30.948 INFO:teuthology.task.internal:roles: ubuntu@smithi012.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'mds.a', 'mds.d', 'mds.g', 'mds.j', 'osd.0', 'osd.3', 'osd.6', 'osd.9', 'client.0'] 2024-09-10T23:44:30.948 INFO:teuthology.task.internal:roles: ubuntu@smithi165.front.sepia.ceph.com - ['mon.b', 'mgr.y', 'mds.b', 'mds.e', 'mds.h', 'mds.k', 'osd.1', 'osd.4', 'osd.7', 'osd.10'] 2024-09-10T23:44:30.948 INFO:teuthology.task.internal:roles: ubuntu@smithi178.front.sepia.ceph.com - ['mon.c', 'mgr.z', 'mds.c', 'mds.f', 'mds.i', 'mds.l', 'osd.2', 'osd.5', 'osd.8', 'osd.11'] 2024-09-10T23:44:30.948 INFO:teuthology.run_tasks:Running task console_log... 2024-09-10T23:44:31.065 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f71e52e3be0>, signals=[15]) 2024-09-10T23:44:31.065 INFO:teuthology.run_tasks:Running task internal.connect... 2024-09-10T23:44:31.072 INFO:teuthology.task.internal:Opening connections... 2024-09-10T23:44:31.072 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi012.front.sepia.ceph.com 2024-09-10T23:44:31.073 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi012.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T23:44:31.159 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi165.front.sepia.ceph.com 2024-09-10T23:44:31.160 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi165.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T23:44:31.240 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi178.front.sepia.ceph.com 2024-09-10T23:44:31.240 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi178.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T23:44:31.325 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-09-10T23:44:31.331 DEBUG:teuthology.orchestra.run.smithi012:> uname -m 2024-09-10T23:44:31.349 INFO:teuthology.orchestra.run.smithi012.stdout:x86_64 2024-09-10T23:44:31.350 DEBUG:teuthology.orchestra.run.smithi012:> cat /etc/os-release 2024-09-10T23:44:31.407 INFO:teuthology.orchestra.run.smithi012.stdout:NAME="CentOS Stream" 2024-09-10T23:44:31.407 INFO:teuthology.orchestra.run.smithi012.stdout:VERSION="9" 2024-09-10T23:44:31.407 INFO:teuthology.orchestra.run.smithi012.stdout:ID="centos" 2024-09-10T23:44:31.407 INFO:teuthology.orchestra.run.smithi012.stdout:ID_LIKE="rhel fedora" 2024-09-10T23:44:31.407 INFO:teuthology.orchestra.run.smithi012.stdout:VERSION_ID="9" 2024-09-10T23:44:31.407 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM_ID="platform:el9" 2024-09-10T23:44:31.407 INFO:teuthology.orchestra.run.smithi012.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-10T23:44:31.407 INFO:teuthology.orchestra.run.smithi012.stdout:ANSI_COLOR="0;31" 2024-09-10T23:44:31.407 INFO:teuthology.orchestra.run.smithi012.stdout:LOGO="fedora-logo-icon" 2024-09-10T23:44:31.407 INFO:teuthology.orchestra.run.smithi012.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-10T23:44:31.408 INFO:teuthology.orchestra.run.smithi012.stdout:HOME_URL="https://centos.org/" 2024-09-10T23:44:31.408 INFO:teuthology.orchestra.run.smithi012.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-10T23:44:31.408 INFO:teuthology.orchestra.run.smithi012.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-10T23:44:31.408 INFO:teuthology.orchestra.run.smithi012.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-10T23:44:31.408 INFO:teuthology.lock.ops:Updating smithi012.front.sepia.ceph.com on lock server 2024-09-10T23:44:31.435 DEBUG:teuthology.orchestra.run.smithi165:> uname -m 2024-09-10T23:44:31.454 INFO:teuthology.orchestra.run.smithi165.stdout:x86_64 2024-09-10T23:44:31.455 DEBUG:teuthology.orchestra.run.smithi165:> cat /etc/os-release 2024-09-10T23:44:31.512 INFO:teuthology.orchestra.run.smithi165.stdout:NAME="CentOS Stream" 2024-09-10T23:44:31.512 INFO:teuthology.orchestra.run.smithi165.stdout:VERSION="9" 2024-09-10T23:44:31.512 INFO:teuthology.orchestra.run.smithi165.stdout:ID="centos" 2024-09-10T23:44:31.512 INFO:teuthology.orchestra.run.smithi165.stdout:ID_LIKE="rhel fedora" 2024-09-10T23:44:31.512 INFO:teuthology.orchestra.run.smithi165.stdout:VERSION_ID="9" 2024-09-10T23:44:31.512 INFO:teuthology.orchestra.run.smithi165.stdout:PLATFORM_ID="platform:el9" 2024-09-10T23:44:31.512 INFO:teuthology.orchestra.run.smithi165.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-10T23:44:31.512 INFO:teuthology.orchestra.run.smithi165.stdout:ANSI_COLOR="0;31" 2024-09-10T23:44:31.513 INFO:teuthology.orchestra.run.smithi165.stdout:LOGO="fedora-logo-icon" 2024-09-10T23:44:31.513 INFO:teuthology.orchestra.run.smithi165.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-10T23:44:31.513 INFO:teuthology.orchestra.run.smithi165.stdout:HOME_URL="https://centos.org/" 2024-09-10T23:44:31.513 INFO:teuthology.orchestra.run.smithi165.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-10T23:44:31.513 INFO:teuthology.orchestra.run.smithi165.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-10T23:44:31.513 INFO:teuthology.orchestra.run.smithi165.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-10T23:44:31.513 INFO:teuthology.lock.ops:Updating smithi165.front.sepia.ceph.com on lock server 2024-09-10T23:44:31.543 DEBUG:teuthology.orchestra.run.smithi178:> uname -m 2024-09-10T23:44:31.562 INFO:teuthology.orchestra.run.smithi178.stdout:x86_64 2024-09-10T23:44:31.562 DEBUG:teuthology.orchestra.run.smithi178:> cat /etc/os-release 2024-09-10T23:44:31.621 INFO:teuthology.orchestra.run.smithi178.stdout:NAME="CentOS Stream" 2024-09-10T23:44:31.621 INFO:teuthology.orchestra.run.smithi178.stdout:VERSION="9" 2024-09-10T23:44:31.621 INFO:teuthology.orchestra.run.smithi178.stdout:ID="centos" 2024-09-10T23:44:31.621 INFO:teuthology.orchestra.run.smithi178.stdout:ID_LIKE="rhel fedora" 2024-09-10T23:44:31.621 INFO:teuthology.orchestra.run.smithi178.stdout:VERSION_ID="9" 2024-09-10T23:44:31.621 INFO:teuthology.orchestra.run.smithi178.stdout:PLATFORM_ID="platform:el9" 2024-09-10T23:44:31.621 INFO:teuthology.orchestra.run.smithi178.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-10T23:44:31.621 INFO:teuthology.orchestra.run.smithi178.stdout:ANSI_COLOR="0;31" 2024-09-10T23:44:31.621 INFO:teuthology.orchestra.run.smithi178.stdout:LOGO="fedora-logo-icon" 2024-09-10T23:44:31.621 INFO:teuthology.orchestra.run.smithi178.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-10T23:44:31.622 INFO:teuthology.orchestra.run.smithi178.stdout:HOME_URL="https://centos.org/" 2024-09-10T23:44:31.622 INFO:teuthology.orchestra.run.smithi178.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-10T23:44:31.622 INFO:teuthology.orchestra.run.smithi178.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-10T23:44:31.622 INFO:teuthology.orchestra.run.smithi178.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-10T23:44:31.622 INFO:teuthology.lock.ops:Updating smithi178.front.sepia.ceph.com on lock server 2024-09-10T23:44:31.652 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-09-10T23:44:31.663 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-09-10T23:44:31.668 INFO:teuthology.task.internal:Checking for old test directory... 2024-09-10T23:44:31.668 DEBUG:teuthology.orchestra.run.smithi012:> test '!' -e /home/ubuntu/cephtest 2024-09-10T23:44:31.671 DEBUG:teuthology.orchestra.run.smithi165:> test '!' -e /home/ubuntu/cephtest 2024-09-10T23:44:31.673 DEBUG:teuthology.orchestra.run.smithi178:> test '!' -e /home/ubuntu/cephtest 2024-09-10T23:44:31.690 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-09-10T23:44:31.695 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-09-10T23:44:31.696 DEBUG:teuthology.orchestra.run.smithi012:> test -z $(ls -A /var/lib/ceph) 2024-09-10T23:44:31.729 DEBUG:teuthology.orchestra.run.smithi165:> test -z $(ls -A /var/lib/ceph) 2024-09-10T23:44:31.731 DEBUG:teuthology.orchestra.run.smithi178:> test -z $(ls -A /var/lib/ceph) 2024-09-10T23:44:31.839 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-09-10T23:44:31.937 INFO:teuthology.run_tasks:Running task kernel... 2024-09-10T23:44:31.950 INFO:teuthology.task.kernel:normalize config orig: {'client': {'branch': 'testing'}, 'flavor': 'default', 'kdb': 1, 'sha1': '2ca44a2f35ca8f36983aaaa46d49312239ef519a'} 2024-09-10T23:44:31.950 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'branch': 'testing'}} 2024-09-10T23:44:31.951 INFO:teuthology.task.kernel:config {'client.0': {'branch': 'testing'}}, timeout 300 2024-09-10T23:44:31.951 DEBUG:teuthology.orchestra.run.smithi012:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-10T23:44:31.967 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T23:44:31.968 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&ref=testing 2024-09-10T23:44:32.136 DEBUG:teuthology.task.kernel:sha1 for client.0 is 2ca44a2f35ca8f36983aaaa46d49312239ef519a 2024-09-10T23:44:32.136 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "2ca44a2f35ca8f36983aaaa46d49312239ef519a"... 2024-09-10T23:44:32.137 DEBUG:teuthology.orchestra.run.smithi012:> uname -r 2024-09-10T23:44:32.154 INFO:teuthology.orchestra.run.smithi012.stdout:5.14.0-503.el9.x86_64 2024-09-10T23:44:32.154 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-503.el9.x86_64 vs 2ca44a2f35ca8f36983aaaa46d49312239ef519a 2024-09-10T23:44:32.155 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2024-09-10T23:44:32.155 DEBUG:teuthology.orchestra.run.smithi012:> sudo yum upgrade -y linux-firmware 2024-09-10T23:44:34.606 INFO:teuthology.orchestra.run.smithi012.stdout:CentOS Stream 9 - BaseOS 12 MB/s | 8.2 MB 00:00 2024-09-10T23:44:36.832 INFO:teuthology.orchestra.run.smithi012.stdout:CentOS Stream 9 - AppStream 19 MB/s | 20 MB 00:01 2024-09-10T23:44:42.248 INFO:teuthology.orchestra.run.smithi012.stdout:CentOS Stream 9 - CRB 4.5 MB/s | 6.5 MB 00:01 2024-09-10T23:44:43.858 INFO:teuthology.orchestra.run.smithi012.stdout:CentOS Stream 9 - Extras packages 76 kB/s | 19 kB 00:00 2024-09-10T23:44:44.961 INFO:teuthology.orchestra.run.smithi012.stdout:Extra Packages for Enterprise Linux 23 MB/s | 23 MB 00:00 2024-09-10T23:44:50.733 INFO:teuthology.orchestra.run.smithi012.stdout:lab-extras 16 kB/s | 1.7 kB 00:00 2024-09-10T23:44:52.923 INFO:teuthology.orchestra.run.smithi012.stdout:Dependencies resolved. 2024-09-10T23:44:52.924 INFO:teuthology.orchestra.run.smithi012.stdout:Nothing to do. 2024-09-10T23:44:52.924 INFO:teuthology.orchestra.run.smithi012.stdout:Complete! 2024-09-10T23:44:53.039 INFO:teuthology.task.kernel:Downloading kernel 2ca44a2f35ca8f36983aaaa46d49312239ef519a on client.0... 2024-09-10T23:44:53.040 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=2ca44a2f35ca8f36983aaaa46d49312239ef519a 2024-09-10T23:44:53.206 INFO:teuthology.task.kernel:fetching, builder baseurl is https://3.chacra.ceph.com/r/kernel/testing/2ca44a2f35ca8f36983aaaa46d49312239ef519a/centos/9/flavors/default/x86_64/ 2024-09-10T23:44:53.206 DEBUG:teuthology.orchestra.run.smithi012:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.11.0_rc5_g2ca44a2f35ca-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://3.chacra.ceph.com/r/kernel/testing/2ca44a2f35ca8f36983aaaa46d49312239ef519a/centos/9/flavors/default/x86_64/ --input-file=- 2024-09-10T23:44:53.209 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi012.front.sepia.ceph.com to complete... 2024-09-10T23:44:59.530 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10 23:44:59 URL:https://3.chacra.ceph.com/r/kernel/testing/2ca44a2f35ca8f36983aaaa46d49312239ef519a/centos/9/flavors/default/x86_64/kernel-6.11.0_rc5_g2ca44a2f35ca-1.x86_64.rpm [359633503/359633503] -> "/tmp/kernel.x86_64.rpm" [1] 2024-09-10T23:44:59.530 INFO:teuthology.orchestra.run.smithi012.stderr:FINISHED --2024-09-10 23:44:59-- 2024-09-10T23:44:59.530 INFO:teuthology.orchestra.run.smithi012.stderr:Total wall clock time: 6.3s 2024-09-10T23:44:59.530 INFO:teuthology.orchestra.run.smithi012.stderr:Downloaded: 1 files, 343M in 6.0s (57.1 MB/s) 2024-09-10T23:44:59.533 INFO:teuthology.task.kernel:Installing kernel 2ca44a2f35ca8f36983aaaa46d49312239ef519a on client.0... 2024-09-10T23:44:59.533 DEBUG:teuthology.orchestra.run.smithi012:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2024-09-10T23:45:01.751 INFO:teuthology.orchestra.run.smithi012.stdout:Verifying... ######################################## 2024-09-10T23:45:01.916 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing... ######################################## 2024-09-10T23:45:02.021 INFO:teuthology.orchestra.run.smithi012.stdout:Updating / installing... 2024-09-10T23:45:10.419 INFO:teuthology.orchestra.run.smithi012.stdout:kernel-6.11.0_rc5_g2ca44a2f35ca-1 ######################################## 2024-09-10T23:45:50.362 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi012.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2024-09-10T23:45:50.363 DEBUG:teuthology.orchestra.run.smithi012:> rpm -qlp /tmp/kernel.x86_64.rpm 2024-09-10T23:45:50.459 INFO:teuthology.orchestra.run.smithi012.stdout:/boot/System.map-6.11.0-rc5-g2ca44a2f35ca 2024-09-10T23:45:50.459 INFO:teuthology.orchestra.run.smithi012.stdout:/boot/config-6.11.0-rc5-g2ca44a2f35ca 2024-09-10T23:45:50.459 INFO:teuthology.orchestra.run.smithi012.stdout:/boot/vmlinuz-6.11.0-rc5-g2ca44a2f35ca 2024-09-10T23:45:50.459 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca 2024-09-10T23:45:50.459 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/System.map 2024-09-10T23:45:50.460 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/config 2024-09-10T23:45:50.460 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel 2024-09-10T23:45:50.460 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch 2024-09-10T23:45:50.460 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86 2024-09-10T23:45:50.460 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto 2024-09-10T23:45:50.460 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/blowfish-x86_64.ko 2024-09-10T23:45:50.460 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2024-09-10T23:45:50.460 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2024-09-10T23:45:50.460 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/camellia-x86_64.ko 2024-09-10T23:45:50.460 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2024-09-10T23:45:50.460 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2024-09-10T23:45:50.460 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/crc32-pclmul.ko 2024-09-10T23:45:50.460 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/crc32c-intel.ko 2024-09-10T23:45:50.461 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2024-09-10T23:45:50.461 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2024-09-10T23:45:50.461 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2024-09-10T23:45:50.461 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2024-09-10T23:45:50.461 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/serpent-avx2.ko 2024-09-10T23:45:50.461 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2024-09-10T23:45:50.461 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/sha1-ssse3.ko 2024-09-10T23:45:50.461 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/sha256-ssse3.ko 2024-09-10T23:45:50.461 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/sha512-ssse3.ko 2024-09-10T23:45:50.461 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2024-09-10T23:45:50.462 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2024-09-10T23:45:50.462 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/twofish-x86_64.ko 2024-09-10T23:45:50.462 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kernel 2024-09-10T23:45:50.462 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kernel/cpu 2024-09-10T23:45:50.462 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kernel/cpu/mce 2024-09-10T23:45:50.462 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2024-09-10T23:45:50.462 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kvm 2024-09-10T23:45:50.462 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kvm/kvm-amd.ko 2024-09-10T23:45:50.462 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kvm/kvm-intel.ko 2024-09-10T23:45:50.462 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kvm/kvm.ko 2024-09-10T23:45:50.462 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto 2024-09-10T23:45:50.462 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/ansi_cprng.ko 2024-09-10T23:45:50.462 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/anubis.ko 2024-09-10T23:45:50.463 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/arc4.ko 2024-09-10T23:45:50.463 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx 2024-09-10T23:45:50.463 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/async_memcpy.ko 2024-09-10T23:45:50.463 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/async_pq.ko 2024-09-10T23:45:50.463 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/async_raid6_recov.ko 2024-09-10T23:45:50.463 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/async_tx.ko 2024-09-10T23:45:50.463 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/async_xor.ko 2024-09-10T23:45:50.463 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/raid6test.ko 2024-09-10T23:45:50.463 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/authenc.ko 2024-09-10T23:45:50.463 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/authencesn.ko 2024-09-10T23:45:50.463 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/blake2b_generic.ko 2024-09-10T23:45:50.464 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/blowfish_common.ko 2024-09-10T23:45:50.464 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/blowfish_generic.ko 2024-09-10T23:45:50.464 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/camellia_generic.ko 2024-09-10T23:45:50.464 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/cast5_generic.ko 2024-09-10T23:45:50.464 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/cast6_generic.ko 2024-09-10T23:45:50.464 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/cast_common.ko 2024-09-10T23:45:50.464 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/ccm.ko 2024-09-10T23:45:50.464 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/cmac.ko 2024-09-10T23:45:50.464 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/crc32_generic.ko 2024-09-10T23:45:50.464 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/crypto_engine.ko 2024-09-10T23:45:50.464 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/crypto_user.ko 2024-09-10T23:45:50.464 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/cts.ko 2024-09-10T23:45:50.464 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/des_generic.ko 2024-09-10T23:45:50.465 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/dh_generic.ko 2024-09-10T23:45:50.465 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/ecc.ko 2024-09-10T23:45:50.465 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/ecdh_generic.ko 2024-09-10T23:45:50.465 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/echainiv.ko 2024-09-10T23:45:50.465 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/essiv.ko 2024-09-10T23:45:50.465 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/fcrypt.ko 2024-09-10T23:45:50.465 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/khazad.ko 2024-09-10T23:45:50.465 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/lz4.ko 2024-09-10T23:45:50.465 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/lz4hc.ko 2024-09-10T23:45:50.465 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/md4.ko 2024-09-10T23:45:50.465 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/michael_mic.ko 2024-09-10T23:45:50.465 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/pcbc.ko 2024-09-10T23:45:50.465 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/pcrypt.ko 2024-09-10T23:45:50.466 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/rmd160.ko 2024-09-10T23:45:50.466 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/seed.ko 2024-09-10T23:45:50.466 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/serpent_generic.ko 2024-09-10T23:45:50.466 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/tcrypt.ko 2024-09-10T23:45:50.466 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/tea.ko 2024-09-10T23:45:50.466 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/twofish_common.ko 2024-09-10T23:45:50.466 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/twofish_generic.ko 2024-09-10T23:45:50.466 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/vmac.ko 2024-09-10T23:45:50.466 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/wp512.ko 2024-09-10T23:45:50.466 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/xcbc.ko 2024-09-10T23:45:50.466 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/xor.ko 2024-09-10T23:45:50.466 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/xxhash_generic.ko 2024-09-10T23:45:50.466 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers 2024-09-10T23:45:50.466 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi 2024-09-10T23:45:50.467 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/acpi_ipmi.ko 2024-09-10T23:45:50.467 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/acpi_pad.ko 2024-09-10T23:45:50.467 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/ec_sys.ko 2024-09-10T23:45:50.467 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/platform_profile.ko 2024-09-10T23:45:50.467 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/sbs.ko 2024-09-10T23:45:50.467 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/sbshc.ko 2024-09-10T23:45:50.467 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/video.ko 2024-09-10T23:45:50.467 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata 2024-09-10T23:45:50.467 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/acard-ahci.ko 2024-09-10T23:45:50.467 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/ahci_platform.ko 2024-09-10T23:45:50.467 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/ata_generic.ko 2024-09-10T23:45:50.467 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/libahci_platform.ko 2024-09-10T23:45:50.467 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_acpi.ko 2024-09-10T23:45:50.468 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_ali.ko 2024-09-10T23:45:50.468 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_amd.ko 2024-09-10T23:45:50.468 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_artop.ko 2024-09-10T23:45:50.468 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_atiixp.ko 2024-09-10T23:45:50.468 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_atp867x.ko 2024-09-10T23:45:50.468 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_cmd640.ko 2024-09-10T23:45:50.468 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_cmd64x.ko 2024-09-10T23:45:50.468 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_cypress.ko 2024-09-10T23:45:50.469 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_efar.ko 2024-09-10T23:45:50.469 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_hpt366.ko 2024-09-10T23:45:50.469 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_hpt37x.ko 2024-09-10T23:45:50.469 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_hpt3x2n.ko 2024-09-10T23:45:50.469 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_hpt3x3.ko 2024-09-10T23:45:50.469 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_it8213.ko 2024-09-10T23:45:50.469 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_it821x.ko 2024-09-10T23:45:50.469 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_jmicron.ko 2024-09-10T23:45:50.469 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_marvell.ko 2024-09-10T23:45:50.469 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_mpiix.ko 2024-09-10T23:45:50.469 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_netcell.ko 2024-09-10T23:45:50.470 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_ninja32.ko 2024-09-10T23:45:50.470 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_ns87410.ko 2024-09-10T23:45:50.470 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_ns87415.ko 2024-09-10T23:45:50.470 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_oldpiix.ko 2024-09-10T23:45:50.470 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_opti.ko 2024-09-10T23:45:50.470 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_optidma.ko 2024-09-10T23:45:50.470 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_pcmcia.ko 2024-09-10T23:45:50.470 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_pdc2027x.ko 2024-09-10T23:45:50.470 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_pdc202xx_old.ko 2024-09-10T23:45:50.470 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_piccolo.ko 2024-09-10T23:45:50.470 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_rdc.ko 2024-09-10T23:45:50.471 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_sch.ko 2024-09-10T23:45:50.471 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_serverworks.ko 2024-09-10T23:45:50.471 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_sil680.ko 2024-09-10T23:45:50.471 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_sis.ko 2024-09-10T23:45:50.471 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_sl82c105.ko 2024-09-10T23:45:50.471 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_triflex.ko 2024-09-10T23:45:50.471 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_via.ko 2024-09-10T23:45:50.471 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pdc_adma.ko 2024-09-10T23:45:50.472 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_inic162x.ko 2024-09-10T23:45:50.472 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_mv.ko 2024-09-10T23:45:50.472 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_nv.ko 2024-09-10T23:45:50.472 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_promise.ko 2024-09-10T23:45:50.472 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_qstor.ko 2024-09-10T23:45:50.472 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_sil.ko 2024-09-10T23:45:50.472 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_sil24.ko 2024-09-10T23:45:50.473 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_sis.ko 2024-09-10T23:45:50.473 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_svw.ko 2024-09-10T23:45:50.473 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_sx4.ko 2024-09-10T23:45:50.473 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_uli.ko 2024-09-10T23:45:50.473 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_via.ko 2024-09-10T23:45:50.473 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_vsc.ko 2024-09-10T23:45:50.473 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/auxdisplay 2024-09-10T23:45:50.473 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/auxdisplay/ks0108.ko 2024-09-10T23:45:50.474 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/base 2024-09-10T23:45:50.474 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/base/regmap 2024-09-10T23:45:50.474 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/base/regmap/regmap-i2c.ko 2024-09-10T23:45:50.474 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/base/regmap/regmap-mmio.ko 2024-09-10T23:45:50.474 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/bcma 2024-09-10T23:45:50.474 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/bcma/bcma.ko 2024-09-10T23:45:50.474 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block 2024-09-10T23:45:50.474 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/aoe 2024-09-10T23:45:50.475 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/aoe/aoe.ko 2024-09-10T23:45:50.475 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/brd.ko 2024-09-10T23:45:50.475 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/drbd 2024-09-10T23:45:50.475 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/drbd/drbd.ko 2024-09-10T23:45:50.475 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/floppy.ko 2024-09-10T23:45:50.475 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/loop.ko 2024-09-10T23:45:50.475 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/mtip32xx 2024-09-10T23:45:50.476 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/mtip32xx/mtip32xx.ko 2024-09-10T23:45:50.476 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/nbd.ko 2024-09-10T23:45:50.476 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/null_blk 2024-09-10T23:45:50.476 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/null_blk/null_blk.ko 2024-09-10T23:45:50.476 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/pktcdvd.ko 2024-09-10T23:45:50.476 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/rbd.ko 2024-09-10T23:45:50.476 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/virtio_blk.ko 2024-09-10T23:45:50.476 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/xen-blkback 2024-09-10T23:45:50.477 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/xen-blkback/xen-blkback.ko 2024-09-10T23:45:50.477 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/xen-blkfront.ko 2024-09-10T23:45:50.477 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/zram 2024-09-10T23:45:50.477 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/zram/zram.ko 2024-09-10T23:45:50.477 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char 2024-09-10T23:45:50.477 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hangcheck-timer.ko 2024-09-10T23:45:50.477 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random 2024-09-10T23:45:50.478 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random/amd-rng.ko 2024-09-10T23:45:50.478 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random/intel-rng.ko 2024-09-10T23:45:50.478 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random/timeriomem-rng.ko 2024-09-10T23:45:50.478 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random/via-rng.ko 2024-09-10T23:45:50.478 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random/virtio-rng.ko 2024-09-10T23:45:50.478 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi 2024-09-10T23:45:50.478 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_devintf.ko 2024-09-10T23:45:50.478 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2024-09-10T23:45:50.479 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2024-09-10T23:45:50.479 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_si.ko 2024-09-10T23:45:50.479 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_ssif.ko 2024-09-10T23:45:50.479 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2024-09-10T23:45:50.479 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/lp.ko 2024-09-10T23:45:50.479 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/mwave 2024-09-10T23:45:50.479 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/mwave/mwave.ko 2024-09-10T23:45:50.480 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ppdev.ko 2024-09-10T23:45:50.480 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tlclk.ko 2024-09-10T23:45:50.480 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm 2024-09-10T23:45:50.480 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm.ko 2024-09-10T23:45:50.480 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm_atmel.ko 2024-09-10T23:45:50.480 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm_infineon.ko 2024-09-10T23:45:50.480 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm_nsc.ko 2024-09-10T23:45:50.480 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm_tis.ko 2024-09-10T23:45:50.481 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm_tis_core.ko 2024-09-10T23:45:50.481 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/uv_mmtimer.ko 2024-09-10T23:45:50.481 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/virtio_console.ko 2024-09-10T23:45:50.481 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq 2024-09-10T23:45:50.481 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq/acpi-cpufreq.ko 2024-09-10T23:45:50.481 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2024-09-10T23:45:50.481 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq/p4-clockmod.ko 2024-09-10T23:45:50.481 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq/pcc-cpufreq.ko 2024-09-10T23:45:50.482 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq/powernow-k8.ko 2024-09-10T23:45:50.482 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq/speedstep-lib.ko 2024-09-10T23:45:50.482 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto 2024-09-10T23:45:50.482 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/ccp 2024-09-10T23:45:50.482 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/ccp/ccp-crypto.ko 2024-09-10T23:45:50.482 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/ccp/ccp.ko 2024-09-10T23:45:50.482 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel 2024-09-10T23:45:50.482 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel/qat 2024-09-10T23:45:50.483 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel/qat/qat_common 2024-09-10T23:45:50.483 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2024-09-10T23:45:50.483 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2024-09-10T23:45:50.483 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2024-09-10T23:45:50.483 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/padlock-aes.ko 2024-09-10T23:45:50.483 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/padlock-sha.ko 2024-09-10T23:45:50.483 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/virtio 2024-09-10T23:45:50.484 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/virtio/virtio_crypto.ko 2024-09-10T23:45:50.484 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dca 2024-09-10T23:45:50.484 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dca/dca.ko 2024-09-10T23:45:50.484 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/devfreq 2024-09-10T23:45:50.484 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/devfreq/governor_simpleondemand.ko 2024-09-10T23:45:50.484 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dma 2024-09-10T23:45:50.484 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dma/dw 2024-09-10T23:45:50.485 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dma/dw/dw_dmac.ko 2024-09-10T23:45:50.485 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dma/ioat 2024-09-10T23:45:50.485 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dma/ioat/ioatdma.ko 2024-09-10T23:45:50.485 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac 2024-09-10T23:45:50.485 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/amd64_edac.ko 2024-09-10T23:45:50.485 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/e752x_edac.ko 2024-09-10T23:45:50.485 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/edac_mce_amd.ko 2024-09-10T23:45:50.485 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i3000_edac.ko 2024-09-10T23:45:50.485 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i3200_edac.ko 2024-09-10T23:45:50.486 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i5100_edac.ko 2024-09-10T23:45:50.486 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i5400_edac.ko 2024-09-10T23:45:50.486 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i7300_edac.ko 2024-09-10T23:45:50.486 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i7core_edac.ko 2024-09-10T23:45:50.486 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i82975x_edac.ko 2024-09-10T23:45:50.486 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/ie31200_edac.ko 2024-09-10T23:45:50.486 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/sb_edac.ko 2024-09-10T23:45:50.486 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/x38_edac.ko 2024-09-10T23:45:50.486 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/firmware 2024-09-10T23:45:50.486 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/firmware/edd.ko 2024-09-10T23:45:50.486 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/firmware/iscsi_ibft.ko 2024-09-10T23:45:50.486 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio 2024-09-10T23:45:50.487 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-ich.ko 2024-09-10T23:45:50.487 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-viperboard.ko 2024-09-10T23:45:50.487 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid 2024-09-10T23:45:50.487 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-appleir.ko 2024-09-10T23:45:50.487 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-aureal.ko 2024-09-10T23:45:50.487 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-axff.ko 2024-09-10T23:45:50.487 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-dr.ko 2024-09-10T23:45:50.487 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-elecom.ko 2024-09-10T23:45:50.487 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-elo.ko 2024-09-10T23:45:50.487 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-emsff.ko 2024-09-10T23:45:50.487 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-gaff.ko 2024-09-10T23:45:50.487 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-gt683r.ko 2024-09-10T23:45:50.487 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-gyration.ko 2024-09-10T23:45:50.488 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-holtek-kbd.ko 2024-09-10T23:45:50.488 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-holtek-mouse.ko 2024-09-10T23:45:50.488 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-holtekff.ko 2024-09-10T23:45:50.488 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-hyperv.ko 2024-09-10T23:45:50.488 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-icade.ko 2024-09-10T23:45:50.488 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-keytouch.ko 2024-09-10T23:45:50.488 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-kye.ko 2024-09-10T23:45:50.488 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-lcpower.ko 2024-09-10T23:45:50.488 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-led.ko 2024-09-10T23:45:50.488 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-lenovo.ko 2024-09-10T23:45:50.488 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-logitech-dj.ko 2024-09-10T23:45:50.488 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-logitech-hidpp.ko 2024-09-10T23:45:50.489 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-multitouch.ko 2024-09-10T23:45:50.489 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-ortek.ko 2024-09-10T23:45:50.489 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-penmount.ko 2024-09-10T23:45:50.489 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-petalynx.ko 2024-09-10T23:45:50.489 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-picolcd.ko 2024-09-10T23:45:50.489 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-pl.ko 2024-09-10T23:45:50.489 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-plantronics.ko 2024-09-10T23:45:50.489 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-primax.ko 2024-09-10T23:45:50.489 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-rmi.ko 2024-09-10T23:45:50.489 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-arvo.ko 2024-09-10T23:45:50.489 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-common.ko 2024-09-10T23:45:50.489 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-isku.ko 2024-09-10T23:45:50.489 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-kone.ko 2024-09-10T23:45:50.490 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-koneplus.ko 2024-09-10T23:45:50.490 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-konepure.ko 2024-09-10T23:45:50.490 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-kovaplus.ko 2024-09-10T23:45:50.490 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-lua.ko 2024-09-10T23:45:50.490 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-pyra.ko 2024-09-10T23:45:50.490 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-ryos.ko 2024-09-10T23:45:50.490 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-savu.ko 2024-09-10T23:45:50.490 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat.ko 2024-09-10T23:45:50.490 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-saitek.ko 2024-09-10T23:45:50.490 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-samsung.ko 2024-09-10T23:45:50.490 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-sensor-hub.ko 2024-09-10T23:45:50.490 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-sjoy.ko 2024-09-10T23:45:50.490 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-sony.ko 2024-09-10T23:45:50.490 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-speedlink.ko 2024-09-10T23:45:50.491 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-steelseries.ko 2024-09-10T23:45:50.491 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-sunplus.ko 2024-09-10T23:45:50.491 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-thrustmaster.ko 2024-09-10T23:45:50.491 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-tivo.ko 2024-09-10T23:45:50.491 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-tmff.ko 2024-09-10T23:45:50.491 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-topseed.ko 2024-09-10T23:45:50.491 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-twinhan.ko 2024-09-10T23:45:50.491 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-uclogic.ko 2024-09-10T23:45:50.491 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-waltop.ko 2024-09-10T23:45:50.491 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-wiimote.ko 2024-09-10T23:45:50.491 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-xinmo.ko 2024-09-10T23:45:50.491 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-zpff.ko 2024-09-10T23:45:50.491 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-zydacron.ko 2024-09-10T23:45:50.492 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/uhid.ko 2024-09-10T23:45:50.492 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/wacom.ko 2024-09-10T23:45:50.492 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hv 2024-09-10T23:45:50.492 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hv/hv_balloon.ko 2024-09-10T23:45:50.492 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hv/hv_utils.ko 2024-09-10T23:45:50.492 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hv/hv_vmbus.ko 2024-09-10T23:45:50.492 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon 2024-09-10T23:45:50.492 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/abituguru.ko 2024-09-10T23:45:50.492 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/abituguru3.ko 2024-09-10T23:45:50.492 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/acpi_power_meter.ko 2024-09-10T23:45:50.493 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ad7414.ko 2024-09-10T23:45:50.493 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ad7418.ko 2024-09-10T23:45:50.493 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adc128d818.ko 2024-09-10T23:45:50.493 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adm1025.ko 2024-09-10T23:45:50.493 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adm1026.ko 2024-09-10T23:45:50.493 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adm1029.ko 2024-09-10T23:45:50.493 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adm1031.ko 2024-09-10T23:45:50.493 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adm9240.ko 2024-09-10T23:45:50.493 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ads7828.ko 2024-09-10T23:45:50.493 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7410.ko 2024-09-10T23:45:50.493 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7411.ko 2024-09-10T23:45:50.493 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7462.ko 2024-09-10T23:45:50.494 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7470.ko 2024-09-10T23:45:50.494 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7475.ko 2024-09-10T23:45:50.494 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7x10.ko 2024-09-10T23:45:50.494 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/amc6821.ko 2024-09-10T23:45:50.494 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/applesmc.ko 2024-09-10T23:45:50.494 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/asb100.ko 2024-09-10T23:45:50.494 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/asc7621.ko 2024-09-10T23:45:50.494 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/asus_atk0110.ko 2024-09-10T23:45:50.494 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/atxp1.ko 2024-09-10T23:45:50.494 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/coretemp.ko 2024-09-10T23:45:50.494 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/dell-smm-hwmon.ko 2024-09-10T23:45:50.495 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/dme1737.ko 2024-09-10T23:45:50.495 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ds1621.ko 2024-09-10T23:45:50.495 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ds620.ko 2024-09-10T23:45:50.495 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/emc1403.ko 2024-09-10T23:45:50.495 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/emc6w201.ko 2024-09-10T23:45:50.495 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/f71805f.ko 2024-09-10T23:45:50.495 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/f71882fg.ko 2024-09-10T23:45:50.495 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/f75375s.ko 2024-09-10T23:45:50.495 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/fam15h_power.ko 2024-09-10T23:45:50.495 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/fschmd.ko 2024-09-10T23:45:50.495 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/g760a.ko 2024-09-10T23:45:50.495 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/g762.ko 2024-09-10T23:45:50.496 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/gl518sm.ko 2024-09-10T23:45:50.496 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/gl520sm.ko 2024-09-10T23:45:50.496 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/hwmon-vid.ko 2024-09-10T23:45:50.496 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/i5500_temp.ko 2024-09-10T23:45:50.496 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/i5k_amb.ko 2024-09-10T23:45:50.496 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ibmaem.ko 2024-09-10T23:45:50.496 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ibmpex.ko 2024-09-10T23:45:50.496 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ina209.ko 2024-09-10T23:45:50.496 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ina2xx.ko 2024-09-10T23:45:50.496 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/it87.ko 2024-09-10T23:45:50.497 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/k10temp.ko 2024-09-10T23:45:50.497 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/k8temp.ko 2024-09-10T23:45:50.497 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lineage-pem.ko 2024-09-10T23:45:50.497 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm63.ko 2024-09-10T23:45:50.497 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm73.ko 2024-09-10T23:45:50.497 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm75.ko 2024-09-10T23:45:50.497 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm77.ko 2024-09-10T23:45:50.497 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm78.ko 2024-09-10T23:45:50.497 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm80.ko 2024-09-10T23:45:50.497 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm83.ko 2024-09-10T23:45:50.498 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm85.ko 2024-09-10T23:45:50.498 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm87.ko 2024-09-10T23:45:50.498 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm90.ko 2024-09-10T23:45:50.498 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm92.ko 2024-09-10T23:45:50.498 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm93.ko 2024-09-10T23:45:50.498 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm95234.ko 2024-09-10T23:45:50.498 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm95241.ko 2024-09-10T23:45:50.498 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm95245.ko 2024-09-10T23:45:50.498 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc2945.ko 2024-09-10T23:45:50.498 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4151.ko 2024-09-10T23:45:50.498 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4215.ko 2024-09-10T23:45:50.498 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4222.ko 2024-09-10T23:45:50.498 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4245.ko 2024-09-10T23:45:50.499 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4260.ko 2024-09-10T23:45:50.499 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4261.ko 2024-09-10T23:45:50.499 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max16065.ko 2024-09-10T23:45:50.499 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max1619.ko 2024-09-10T23:45:50.499 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max1668.ko 2024-09-10T23:45:50.499 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max197.ko 2024-09-10T23:45:50.499 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max6639.ko 2024-09-10T23:45:50.499 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max6650.ko 2024-09-10T23:45:50.499 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max6697.ko 2024-09-10T23:45:50.499 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/mcp3021.ko 2024-09-10T23:45:50.499 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/nct6683.ko 2024-09-10T23:45:50.499 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/nct6775-core.ko 2024-09-10T23:45:50.500 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/nct6775.ko 2024-09-10T23:45:50.500 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/nct7802.ko 2024-09-10T23:45:50.500 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pc87360.ko 2024-09-10T23:45:50.500 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pc87427.ko 2024-09-10T23:45:50.500 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pcf8591.ko 2024-09-10T23:45:50.500 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus 2024-09-10T23:45:50.500 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/adm1275.ko 2024-09-10T23:45:50.500 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/lm25066.ko 2024-09-10T23:45:50.501 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/ltc2978.ko 2024-09-10T23:45:50.501 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/max16064.ko 2024-09-10T23:45:50.501 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/max34440.ko 2024-09-10T23:45:50.501 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/max8688.ko 2024-09-10T23:45:50.501 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/pmbus.ko 2024-09-10T23:45:50.501 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2024-09-10T23:45:50.501 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/tps40422.ko 2024-09-10T23:45:50.501 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/ucd9000.ko 2024-09-10T23:45:50.501 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/ucd9200.ko 2024-09-10T23:45:50.501 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/zl6100.ko 2024-09-10T23:45:50.501 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/powr1220.ko 2024-09-10T23:45:50.502 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sch5627.ko 2024-09-10T23:45:50.502 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sch5636.ko 2024-09-10T23:45:50.502 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sch56xx-common.ko 2024-09-10T23:45:50.502 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sht15.ko 2024-09-10T23:45:50.502 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sht21.ko 2024-09-10T23:45:50.502 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/shtc1.ko 2024-09-10T23:45:50.502 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sis5595.ko 2024-09-10T23:45:50.502 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/smsc47b397.ko 2024-09-10T23:45:50.502 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/smsc47m1.ko 2024-09-10T23:45:50.502 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/smsc47m192.ko 2024-09-10T23:45:50.502 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/thmc50.ko 2024-09-10T23:45:50.502 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/tmp102.ko 2024-09-10T23:45:50.502 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/tmp103.ko 2024-09-10T23:45:50.502 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/tmp401.ko 2024-09-10T23:45:50.503 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/tmp421.ko 2024-09-10T23:45:50.503 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/via-cputemp.ko 2024-09-10T23:45:50.503 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/via686a.ko 2024-09-10T23:45:50.503 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/vt1211.ko 2024-09-10T23:45:50.503 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/vt8231.ko 2024-09-10T23:45:50.503 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83627ehf.ko 2024-09-10T23:45:50.503 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83627hf.ko 2024-09-10T23:45:50.503 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83781d.ko 2024-09-10T23:45:50.503 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83791d.ko 2024-09-10T23:45:50.503 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83792d.ko 2024-09-10T23:45:50.503 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83793.ko 2024-09-10T23:45:50.503 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83795.ko 2024-09-10T23:45:50.503 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83l785ts.ko 2024-09-10T23:45:50.503 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83l786ng.ko 2024-09-10T23:45:50.504 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c 2024-09-10T23:45:50.504 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/algos 2024-09-10T23:45:50.504 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2024-09-10T23:45:50.504 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2024-09-10T23:45:50.504 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses 2024-09-10T23:45:50.504 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2024-09-10T23:45:50.504 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-amd756.ko 2024-09-10T23:45:50.504 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-amd8111.ko 2024-09-10T23:45:50.504 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2024-09-10T23:45:50.504 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-designware-core.ko 2024-09-10T23:45:50.504 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2024-09-10T23:45:50.504 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2024-09-10T23:45:50.505 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2024-09-10T23:45:50.505 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-i801.ko 2024-09-10T23:45:50.505 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-isch.ko 2024-09-10T23:45:50.505 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-ismt.ko 2024-09-10T23:45:50.505 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2024-09-10T23:45:50.505 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-nforce2.ko 2024-09-10T23:45:50.505 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-parport.ko 2024-09-10T23:45:50.505 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2024-09-10T23:45:50.505 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-piix4.ko 2024-09-10T23:45:50.505 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-scmi.ko 2024-09-10T23:45:50.505 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-simtec.ko 2024-09-10T23:45:50.505 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-sis96x.ko 2024-09-10T23:45:50.506 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2024-09-10T23:45:50.506 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-via.ko 2024-09-10T23:45:50.506 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-viapro.ko 2024-09-10T23:45:50.506 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-viperboard.ko 2024-09-10T23:45:50.506 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/i2c-dev.ko 2024-09-10T23:45:50.506 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/i2c-mux.ko 2024-09-10T23:45:50.506 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/i2c-slave-eeprom.ko 2024-09-10T23:45:50.506 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/i2c-smbus.ko 2024-09-10T23:45:50.506 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/i2c-stub.ko 2024-09-10T23:45:50.506 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband 2024-09-10T23:45:50.506 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core 2024-09-10T23:45:50.506 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/ib_cm.ko 2024-09-10T23:45:50.506 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/ib_core.ko 2024-09-10T23:45:50.507 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/ib_umad.ko 2024-09-10T23:45:50.507 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/ib_uverbs.ko 2024-09-10T23:45:50.507 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/iw_cm.ko 2024-09-10T23:45:50.507 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/rdma_cm.ko 2024-09-10T23:45:50.507 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/rdma_ucm.ko 2024-09-10T23:45:50.507 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw 2024-09-10T23:45:50.507 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/cxgb4 2024-09-10T23:45:50.507 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2024-09-10T23:45:50.507 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mlx4 2024-09-10T23:45:50.507 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2024-09-10T23:45:50.507 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mlx5 2024-09-10T23:45:50.507 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2024-09-10T23:45:50.507 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mthca 2024-09-10T23:45:50.508 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2024-09-10T23:45:50.508 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp 2024-09-10T23:45:50.508 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/ipoib 2024-09-10T23:45:50.508 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2024-09-10T23:45:50.508 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/iser 2024-09-10T23:45:50.508 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2024-09-10T23:45:50.508 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/isert 2024-09-10T23:45:50.508 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2024-09-10T23:45:50.508 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/srp 2024-09-10T23:45:50.508 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2024-09-10T23:45:50.508 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/srpt 2024-09-10T23:45:50.508 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2024-09-10T23:45:50.509 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input 2024-09-10T23:45:50.509 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/gameport 2024-09-10T23:45:50.509 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/gameport/emu10k1-gp.ko 2024-09-10T23:45:50.509 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/gameport/fm801-gp.ko 2024-09-10T23:45:50.509 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/gameport/gameport.ko 2024-09-10T23:45:50.509 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/joydev.ko 2024-09-10T23:45:50.509 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard 2024-09-10T23:45:50.509 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/gpio_keys.ko 2024-09-10T23:45:50.509 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse 2024-09-10T23:45:50.509 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/appletouch.ko 2024-09-10T23:45:50.509 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/bcm5974.ko 2024-09-10T23:45:50.509 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/cyapatp.ko 2024-09-10T23:45:50.509 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/elan_i2c.ko 2024-09-10T23:45:50.509 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/sermouse.ko 2024-09-10T23:45:50.510 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/synaptics_i2c.ko 2024-09-10T23:45:50.510 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/synaptics_usb.ko 2024-09-10T23:45:50.510 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/vsxxxaa.ko 2024-09-10T23:45:50.510 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/rmi4 2024-09-10T23:45:50.510 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/rmi4/rmi_core.ko 2024-09-10T23:45:50.510 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio 2024-09-10T23:45:50.510 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/altera_ps2.ko 2024-09-10T23:45:50.510 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/arc_ps2.ko 2024-09-10T23:45:50.510 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/hyperv-keyboard.ko 2024-09-10T23:45:50.510 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/serio_raw.ko 2024-09-10T23:45:50.510 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/sparse-keymap.ko 2024-09-10T23:45:50.510 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds 2024-09-10T23:45:50.510 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-blinkm.ko 2024-09-10T23:45:50.511 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-lm3530.ko 2024-09-10T23:45:50.511 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-lp3944.ko 2024-09-10T23:45:50.511 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-ss4200.ko 2024-09-10T23:45:50.511 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger 2024-09-10T23:45:50.511 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2024-09-10T23:45:50.511 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-camera.ko 2024-09-10T23:45:50.511 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2024-09-10T23:45:50.512 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-gpio.ko 2024-09-10T23:45:50.512 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2024-09-10T23:45:50.512 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2024-09-10T23:45:50.512 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-timer.ko 2024-09-10T23:45:50.512 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-transient.ko 2024-09-10T23:45:50.512 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md 2024-09-10T23:45:50.512 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/bcache 2024-09-10T23:45:50.512 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/bcache/bcache.ko 2024-09-10T23:45:50.512 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-bio-prison.ko 2024-09-10T23:45:50.512 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-cache-smq.ko 2024-09-10T23:45:50.512 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-cache.ko 2024-09-10T23:45:50.512 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-crypt.ko 2024-09-10T23:45:50.513 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-delay.ko 2024-09-10T23:45:50.513 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-flakey.ko 2024-09-10T23:45:50.513 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-log-userspace.ko 2024-09-10T23:45:50.513 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-multipath.ko 2024-09-10T23:45:50.513 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-queue-length.ko 2024-09-10T23:45:50.513 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-raid.ko 2024-09-10T23:45:50.513 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-round-robin.ko 2024-09-10T23:45:50.513 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-service-time.ko 2024-09-10T23:45:50.513 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-switch.ko 2024-09-10T23:45:50.513 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-thin-pool.ko 2024-09-10T23:45:50.513 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-verity.ko 2024-09-10T23:45:50.513 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/persistent-data 2024-09-10T23:45:50.514 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2024-09-10T23:45:50.514 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/raid0.ko 2024-09-10T23:45:50.514 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/raid1.ko 2024-09-10T23:45:50.514 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/raid10.ko 2024-09-10T23:45:50.514 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/raid456.ko 2024-09-10T23:45:50.514 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message 2024-09-10T23:45:50.514 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion 2024-09-10T23:45:50.514 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptbase.ko 2024-09-10T23:45:50.514 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptctl.ko 2024-09-10T23:45:50.514 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptfc.ko 2024-09-10T23:45:50.514 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptlan.ko 2024-09-10T23:45:50.514 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptsas.ko 2024-09-10T23:45:50.514 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptscsih.ko 2024-09-10T23:45:50.515 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptspi.ko 2024-09-10T23:45:50.515 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd 2024-09-10T23:45:50.515 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/lpc_ich.ko 2024-09-10T23:45:50.515 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/lpc_sch.ko 2024-09-10T23:45:50.515 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/mfd-core.ko 2024-09-10T23:45:50.515 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/sm501.ko 2024-09-10T23:45:50.515 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/viperboard.ko 2024-09-10T23:45:50.515 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/vx855.ko 2024-09-10T23:45:50.515 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/wl1273-core.ko 2024-09-10T23:45:50.515 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc 2024-09-10T23:45:50.515 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/altera-stapl 2024-09-10T23:45:50.515 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2024-09-10T23:45:50.515 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/apds9802als.ko 2024-09-10T23:45:50.516 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/apds990x.ko 2024-09-10T23:45:50.516 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/bh1770glc.ko 2024-09-10T23:45:50.516 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/cb710 2024-09-10T23:45:50.516 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/cb710/cb710.ko 2024-09-10T23:45:50.516 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/eeprom 2024-09-10T23:45:50.516 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/eeprom/at24.ko 2024-09-10T23:45:50.516 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2024-09-10T23:45:50.516 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/eeprom/max6875.ko 2024-09-10T23:45:50.516 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/enclosure.ko 2024-09-10T23:45:50.516 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/hpilo.ko 2024-09-10T23:45:50.517 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/isl29003.ko 2024-09-10T23:45:50.517 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/isl29020.ko 2024-09-10T23:45:50.517 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/lis3lv02d 2024-09-10T23:45:50.517 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2024-09-10T23:45:50.517 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2024-09-10T23:45:50.517 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/mei 2024-09-10T23:45:50.517 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/mei/mei-me.ko 2024-09-10T23:45:50.517 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/mei/mei-txe.ko 2024-09-10T23:45:50.517 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/mei/mei.ko 2024-09-10T23:45:50.517 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/sgi-gru 2024-09-10T23:45:50.517 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/sgi-gru/gru.ko 2024-09-10T23:45:50.517 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/sgi-xp 2024-09-10T23:45:50.517 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/sgi-xp/xp.ko 2024-09-10T23:45:50.518 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/sgi-xp/xpc.ko 2024-09-10T23:45:50.518 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/sgi-xp/xpnet.ko 2024-09-10T23:45:50.518 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/tifm_7xx1.ko 2024-09-10T23:45:50.518 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/tifm_core.ko 2024-09-10T23:45:50.518 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/tsl2550.ko 2024-09-10T23:45:50.518 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/vmw_balloon.ko 2024-09-10T23:45:50.518 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/vmw_vmci 2024-09-10T23:45:50.518 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2024-09-10T23:45:50.518 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net 2024-09-10T23:45:50.518 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/bonding 2024-09-10T23:45:50.518 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/bonding/bonding.ko 2024-09-10T23:45:50.518 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa 2024-09-10T23:45:50.518 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/b53 2024-09-10T23:45:50.519 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/b53/b53_common.ko 2024-09-10T23:45:50.519 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/bcm-sf2.ko 2024-09-10T23:45:50.519 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/mv88e6060.ko 2024-09-10T23:45:50.519 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/mv88e6xxx 2024-09-10T23:45:50.519 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2024-09-10T23:45:50.519 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dummy.ko 2024-09-10T23:45:50.519 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/eql.ko 2024-09-10T23:45:50.519 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet 2024-09-10T23:45:50.519 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/3com 2024-09-10T23:45:50.519 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2024-09-10T23:45:50.519 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2024-09-10T23:45:50.519 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/3com/3c59x.ko 2024-09-10T23:45:50.519 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/3com/typhoon.ko 2024-09-10T23:45:50.519 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/8390 2024-09-10T23:45:50.520 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/8390/8390.ko 2024-09-10T23:45:50.520 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2024-09-10T23:45:50.520 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2024-09-10T23:45:50.520 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2024-09-10T23:45:50.520 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/adaptec 2024-09-10T23:45:50.520 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/adaptec/starfire.ko 2024-09-10T23:45:50.520 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/agere 2024-09-10T23:45:50.520 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/agere/et131x.ko 2024-09-10T23:45:50.520 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/alteon 2024-09-10T23:45:50.520 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/alteon/acenic.ko 2024-09-10T23:45:50.520 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/altera 2024-09-10T23:45:50.520 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/altera/altera_tse.ko 2024-09-10T23:45:50.520 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/amd 2024-09-10T23:45:50.521 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/amd/amd8111e.ko 2024-09-10T23:45:50.521 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2024-09-10T23:45:50.521 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/amd/pcnet32.ko 2024-09-10T23:45:50.521 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros 2024-09-10T23:45:50.521 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/alx 2024-09-10T23:45:50.521 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2024-09-10T23:45:50.521 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atl1c 2024-09-10T23:45:50.521 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2024-09-10T23:45:50.521 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atl1e 2024-09-10T23:45:50.521 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2024-09-10T23:45:50.521 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atlx 2024-09-10T23:45:50.521 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2024-09-10T23:45:50.521 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2024-09-10T23:45:50.522 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom 2024-09-10T23:45:50.522 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/b44.ko 2024-09-10T23:45:50.522 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2024-09-10T23:45:50.522 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/bnx2x 2024-09-10T23:45:50.522 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2024-09-10T23:45:50.522 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/cnic.ko 2024-09-10T23:45:50.523 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/genet 2024-09-10T23:45:50.523 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2024-09-10T23:45:50.523 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/tg3.ko 2024-09-10T23:45:50.523 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/brocade 2024-09-10T23:45:50.523 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/brocade/bna 2024-09-10T23:45:50.523 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2024-09-10T23:45:50.523 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio 2024-09-10T23:45:50.523 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb 2024-09-10T23:45:50.523 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2024-09-10T23:45:50.523 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb3 2024-09-10T23:45:50.523 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2024-09-10T23:45:50.523 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb4 2024-09-10T23:45:50.523 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2024-09-10T23:45:50.523 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2024-09-10T23:45:50.524 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2024-09-10T23:45:50.524 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/libcxgb 2024-09-10T23:45:50.524 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2024-09-10T23:45:50.524 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/cisco 2024-09-10T23:45:50.524 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/cisco/enic 2024-09-10T23:45:50.524 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2024-09-10T23:45:50.524 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec 2024-09-10T23:45:50.524 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip 2024-09-10T23:45:50.524 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2024-09-10T23:45:50.524 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2024-09-10T23:45:50.524 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2024-09-10T23:45:50.524 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2024-09-10T23:45:50.525 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2024-09-10T23:45:50.525 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2024-09-10T23:45:50.525 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dlink 2024-09-10T23:45:50.525 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dlink/dl2k.ko 2024-09-10T23:45:50.525 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dlink/sundance.ko 2024-09-10T23:45:50.525 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dnet.ko 2024-09-10T23:45:50.525 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/emulex 2024-09-10T23:45:50.525 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/emulex/benet 2024-09-10T23:45:50.525 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2024-09-10T23:45:50.525 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/ethoc.ko 2024-09-10T23:45:50.525 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/fealnx.ko 2024-09-10T23:45:50.525 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel 2024-09-10T23:45:50.525 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/e100.ko 2024-09-10T23:45:50.526 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/e1000 2024-09-10T23:45:50.526 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2024-09-10T23:45:50.526 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/e1000e 2024-09-10T23:45:50.526 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2024-09-10T23:45:50.526 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/fm10k 2024-09-10T23:45:50.526 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2024-09-10T23:45:50.526 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/i40e 2024-09-10T23:45:50.526 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2024-09-10T23:45:50.526 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/iavf 2024-09-10T23:45:50.526 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2024-09-10T23:45:50.526 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/igb 2024-09-10T23:45:50.526 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/igb/igb.ko 2024-09-10T23:45:50.526 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/igbvf 2024-09-10T23:45:50.526 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2024-09-10T23:45:50.527 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/ixgbe 2024-09-10T23:45:50.527 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2024-09-10T23:45:50.527 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/ixgbevf 2024-09-10T23:45:50.527 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2024-09-10T23:45:50.527 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/libeth 2024-09-10T23:45:50.527 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/libeth/libeth.ko 2024-09-10T23:45:50.527 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/libie 2024-09-10T23:45:50.527 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/libie/libie.ko 2024-09-10T23:45:50.527 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/jme.ko 2024-09-10T23:45:50.527 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/marvell 2024-09-10T23:45:50.527 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2024-09-10T23:45:50.527 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/marvell/skge.ko 2024-09-10T23:45:50.527 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/marvell/sky2.ko 2024-09-10T23:45:50.527 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox 2024-09-10T23:45:50.528 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx4 2024-09-10T23:45:50.528 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2024-09-10T23:45:50.528 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2024-09-10T23:45:50.528 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx5 2024-09-10T23:45:50.528 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx5/core 2024-09-10T23:45:50.528 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2024-09-10T23:45:50.528 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/micrel 2024-09-10T23:45:50.528 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2024-09-10T23:45:50.528 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/myricom 2024-09-10T23:45:50.528 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/myricom/myri10ge 2024-09-10T23:45:50.528 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2024-09-10T23:45:50.528 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/natsemi 2024-09-10T23:45:50.528 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2024-09-10T23:45:50.529 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2024-09-10T23:45:50.529 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/neterion 2024-09-10T23:45:50.529 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/neterion/s2io.ko 2024-09-10T23:45:50.529 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/nvidia 2024-09-10T23:45:50.529 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2024-09-10T23:45:50.529 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/packetengines 2024-09-10T23:45:50.529 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2024-09-10T23:45:50.529 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2024-09-10T23:45:50.529 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic 2024-09-10T23:45:50.529 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic/netxen 2024-09-10T23:45:50.529 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2024-09-10T23:45:50.529 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2024-09-10T23:45:50.529 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic/qlcnic 2024-09-10T23:45:50.530 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2024-09-10T23:45:50.530 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/rdc 2024-09-10T23:45:50.530 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/rdc/r6040.ko 2024-09-10T23:45:50.530 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/realtek 2024-09-10T23:45:50.530 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/realtek/8139cp.ko 2024-09-10T23:45:50.530 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/realtek/8139too.ko 2024-09-10T23:45:50.530 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/realtek/atp.ko 2024-09-10T23:45:50.530 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/realtek/r8169.ko 2024-09-10T23:45:50.530 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/rocker 2024-09-10T23:45:50.530 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/rocker/rocker.ko 2024-09-10T23:45:50.530 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sfc 2024-09-10T23:45:50.530 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sfc/sfc.ko 2024-09-10T23:45:50.531 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/silan 2024-09-10T23:45:50.531 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/silan/sc92031.ko 2024-09-10T23:45:50.531 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sis 2024-09-10T23:45:50.531 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sis/sis190.ko 2024-09-10T23:45:50.531 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sis/sis900.ko 2024-09-10T23:45:50.531 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/smsc 2024-09-10T23:45:50.531 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/smsc/epic100.ko 2024-09-10T23:45:50.531 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2024-09-10T23:45:50.531 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2024-09-10T23:45:50.531 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2024-09-10T23:45:50.531 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/stmicro 2024-09-10T23:45:50.531 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/stmicro/stmmac 2024-09-10T23:45:50.532 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2024-09-10T23:45:50.532 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2024-09-10T23:45:50.532 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sun 2024-09-10T23:45:50.532 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sun/cassini.ko 2024-09-10T23:45:50.532 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sun/niu.ko 2024-09-10T23:45:50.532 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sun/sungem.ko 2024-09-10T23:45:50.532 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sun/sunhme.ko 2024-09-10T23:45:50.532 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/tehuti 2024-09-10T23:45:50.532 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2024-09-10T23:45:50.532 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/ti 2024-09-10T23:45:50.532 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/ti/tlan.ko 2024-09-10T23:45:50.532 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/via 2024-09-10T23:45:50.533 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/via/via-rhine.ko 2024-09-10T23:45:50.533 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/via/via-velocity.ko 2024-09-10T23:45:50.533 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/wiznet 2024-09-10T23:45:50.533 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/wiznet/w5100.ko 2024-09-10T23:45:50.533 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/wiznet/w5300.ko 2024-09-10T23:45:50.533 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/xircom 2024-09-10T23:45:50.533 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2024-09-10T23:45:50.534 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/geneve.ko 2024-09-10T23:45:50.534 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/hyperv 2024-09-10T23:45:50.534 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/hyperv/hv_netvsc.ko 2024-09-10T23:45:50.534 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ieee802154 2024-09-10T23:45:50.534 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ieee802154/fakelb.ko 2024-09-10T23:45:50.534 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ifb.ko 2024-09-10T23:45:50.534 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ipvlan 2024-09-10T23:45:50.534 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ipvlan/ipvlan.ko 2024-09-10T23:45:50.534 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/macvlan.ko 2024-09-10T23:45:50.534 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/macvtap.ko 2024-09-10T23:45:50.534 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mdio 2024-09-10T23:45:50.534 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mdio.ko 2024-09-10T23:45:50.534 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2024-09-10T23:45:50.535 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mdio/mdio-bitbang.ko 2024-09-10T23:45:50.535 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mdio/mdio-regmap.ko 2024-09-10T23:45:50.535 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mii.ko 2024-09-10T23:45:50.535 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/net_failover.ko 2024-09-10T23:45:50.535 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/netconsole.ko 2024-09-10T23:45:50.535 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/nlmon.ko 2024-09-10T23:45:50.535 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/pcs 2024-09-10T23:45:50.535 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/pcs/pcs-lynx.ko 2024-09-10T23:45:50.535 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/pcs/pcs_xpcs.ko 2024-09-10T23:45:50.535 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy 2024-09-10T23:45:50.535 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/amd.ko 2024-09-10T23:45:50.535 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/bcm-phy-lib.ko 2024-09-10T23:45:50.535 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/bcm-phy-ptp.ko 2024-09-10T23:45:50.536 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/bcm7xxx.ko 2024-09-10T23:45:50.536 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/bcm87xx.ko 2024-09-10T23:45:50.536 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/broadcom.ko 2024-09-10T23:45:50.536 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/cicada.ko 2024-09-10T23:45:50.536 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/davicom.ko 2024-09-10T23:45:50.536 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/dp83640.ko 2024-09-10T23:45:50.536 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/et1011c.ko 2024-09-10T23:45:50.536 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/icplus.ko 2024-09-10T23:45:50.536 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/lxt.ko 2024-09-10T23:45:50.536 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/marvell.ko 2024-09-10T23:45:50.536 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/micrel.ko 2024-09-10T23:45:50.536 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/national.ko 2024-09-10T23:45:50.536 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/phylink.ko 2024-09-10T23:45:50.537 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/qsemi.ko 2024-09-10T23:45:50.537 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/realtek.ko 2024-09-10T23:45:50.537 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/smsc.ko 2024-09-10T23:45:50.537 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/ste10Xp.ko 2024-09-10T23:45:50.537 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/vitesse.ko 2024-09-10T23:45:50.537 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp 2024-09-10T23:45:50.537 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/bsd_comp.ko 2024-09-10T23:45:50.537 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/ppp_async.ko 2024-09-10T23:45:50.537 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/ppp_deflate.ko 2024-09-10T23:45:50.537 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/ppp_generic.ko 2024-09-10T23:45:50.537 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/ppp_mppe.ko 2024-09-10T23:45:50.537 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/ppp_synctty.ko 2024-09-10T23:45:50.537 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/pppoe.ko 2024-09-10T23:45:50.538 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/pppox.ko 2024-09-10T23:45:50.538 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/pptp.ko 2024-09-10T23:45:50.538 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/slip 2024-09-10T23:45:50.538 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/slip/slhc.ko 2024-09-10T23:45:50.538 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/slip/slip.ko 2024-09-10T23:45:50.538 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/sungem_phy.ko 2024-09-10T23:45:50.538 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/tap.ko 2024-09-10T23:45:50.538 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team 2024-09-10T23:45:50.538 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team.ko 2024-09-10T23:45:50.538 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team_mode_activebackup.ko 2024-09-10T23:45:50.538 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team_mode_broadcast.ko 2024-09-10T23:45:50.538 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team_mode_loadbalance.ko 2024-09-10T23:45:50.538 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team_mode_random.ko 2024-09-10T23:45:50.539 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team_mode_roundrobin.ko 2024-09-10T23:45:50.539 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/tun.ko 2024-09-10T23:45:50.539 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/veth.ko 2024-09-10T23:45:50.539 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/virtio_net.ko 2024-09-10T23:45:50.539 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/vmxnet3 2024-09-10T23:45:50.539 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/vmxnet3/vmxnet3.ko 2024-09-10T23:45:50.539 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/vxlan 2024-09-10T23:45:50.539 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/vxlan/vxlan.ko 2024-09-10T23:45:50.539 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/xen-netback 2024-09-10T23:45:50.539 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/xen-netback/xen-netback.ko 2024-09-10T23:45:50.539 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/xen-netfront.ko 2024-09-10T23:45:50.539 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ntb 2024-09-10T23:45:50.539 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ntb/ntb.ko 2024-09-10T23:45:50.540 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/nvme 2024-09-10T23:45:50.540 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/nvme/host 2024-09-10T23:45:50.540 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/nvme/host/nvme-core.ko 2024-09-10T23:45:50.540 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/nvme/host/nvme.ko 2024-09-10T23:45:50.540 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/parport 2024-09-10T23:45:50.540 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/parport/parport.ko 2024-09-10T23:45:50.540 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/parport/parport_cs.ko 2024-09-10T23:45:50.540 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/parport/parport_pc.ko 2024-09-10T23:45:50.540 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/parport/parport_serial.ko 2024-09-10T23:45:50.540 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci 2024-09-10T23:45:50.540 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/hotplug 2024-09-10T23:45:50.540 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2024-09-10T23:45:50.540 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/pcie 2024-09-10T23:45:50.541 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/pcie/aer_inject.ko 2024-09-10T23:45:50.541 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/xen-pcifront.ko 2024-09-10T23:45:50.541 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pcmcia 2024-09-10T23:45:50.541 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pcmcia/i82092.ko 2024-09-10T23:45:50.541 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pcmcia/pd6729.ko 2024-09-10T23:45:50.541 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pcmcia/yenta_socket.ko 2024-09-10T23:45:50.541 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pinctrl 2024-09-10T23:45:50.541 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pinctrl/intel 2024-09-10T23:45:50.541 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2024-09-10T23:45:50.541 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform 2024-09-10T23:45:50.541 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86 2024-09-10T23:45:50.542 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/acer-wmi.ko 2024-09-10T23:45:50.542 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/acerhdf.ko 2024-09-10T23:45:50.542 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/apple-gmux.ko 2024-09-10T23:45:50.542 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/asus-laptop.ko 2024-09-10T23:45:50.542 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/asus-nb-wmi.ko 2024-09-10T23:45:50.542 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/asus-wmi.ko 2024-09-10T23:45:50.542 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/classmate-laptop.ko 2024-09-10T23:45:50.542 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/eeepc-laptop.ko 2024-09-10T23:45:50.542 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/eeepc-wmi.ko 2024-09-10T23:45:50.542 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/fujitsu-laptop.ko 2024-09-10T23:45:50.542 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/fujitsu-tablet.ko 2024-09-10T23:45:50.542 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/hdaps.ko 2024-09-10T23:45:50.542 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/intel 2024-09-10T23:45:50.542 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/intel/intel-rst.ko 2024-09-10T23:45:50.543 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/intel_ips.ko 2024-09-10T23:45:50.543 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/msi-wmi.ko 2024-09-10T23:45:50.543 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/mxm-wmi.ko 2024-09-10T23:45:50.543 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/panasonic-laptop.ko 2024-09-10T23:45:50.543 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/samsung-laptop.ko 2024-09-10T23:45:50.543 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/samsung-q10.ko 2024-09-10T23:45:50.543 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/topstar-laptop.ko 2024-09-10T23:45:50.543 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2024-09-10T23:45:50.543 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/toshiba_haps.ko 2024-09-10T23:45:50.543 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/wmi-bmof.ko 2024-09-10T23:45:50.543 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/wmi.ko 2024-09-10T23:45:50.543 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/powercap 2024-09-10T23:45:50.544 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/powercap/intel_rapl_common.ko 2024-09-10T23:45:50.544 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/powercap/intel_rapl_msr.ko 2024-09-10T23:45:50.544 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps 2024-09-10T23:45:50.544 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps/clients 2024-09-10T23:45:50.544 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps/clients/pps-gpio.ko 2024-09-10T23:45:50.544 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps/clients/pps-ldisc.ko 2024-09-10T23:45:50.544 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps/clients/pps_parport.ko 2024-09-10T23:45:50.545 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps/pps_core.ko 2024-09-10T23:45:50.545 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ptp 2024-09-10T23:45:50.545 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ptp/ptp.ko 2024-09-10T23:45:50.545 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ptp/ptp_kvm.ko 2024-09-10T23:45:50.545 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ras 2024-09-10T23:45:50.545 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ras/amd 2024-09-10T23:45:50.545 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ras/amd/atl 2024-09-10T23:45:50.546 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ras/amd/atl/amd_atl.ko 2024-09-10T23:45:50.546 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ras/amd/fmpm.ko 2024-09-10T23:45:50.546 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc 2024-09-10T23:45:50.546 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-bq32k.ko 2024-09-10T23:45:50.546 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1286.ko 2024-09-10T23:45:50.546 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1307.ko 2024-09-10T23:45:50.546 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1374.ko 2024-09-10T23:45:50.546 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1511.ko 2024-09-10T23:45:50.546 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1553.ko 2024-09-10T23:45:50.547 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1672.ko 2024-09-10T23:45:50.547 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1742.ko 2024-09-10T23:45:50.547 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds2404.ko 2024-09-10T23:45:50.547 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds3232.ko 2024-09-10T23:45:50.547 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-em3027.ko 2024-09-10T23:45:50.547 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-fm3130.ko 2024-09-10T23:45:50.547 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-isl12022.ko 2024-09-10T23:45:50.547 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-isl1208.ko 2024-09-10T23:45:50.547 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-m41t80.ko 2024-09-10T23:45:50.547 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-m48t35.ko 2024-09-10T23:45:50.547 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-m48t59.ko 2024-09-10T23:45:50.547 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-max6900.ko 2024-09-10T23:45:50.547 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-msm6242.ko 2024-09-10T23:45:50.547 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-pcf2127.ko 2024-09-10T23:45:50.547 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-pcf85063.ko 2024-09-10T23:45:50.548 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-pcf8523.ko 2024-09-10T23:45:50.548 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-pcf8563.ko 2024-09-10T23:45:50.548 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-pcf8583.ko 2024-09-10T23:45:50.548 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-rp5c01.ko 2024-09-10T23:45:50.548 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-rs5c372.ko 2024-09-10T23:45:50.548 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-rv3029c2.ko 2024-09-10T23:45:50.548 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-rx8025.ko 2024-09-10T23:45:50.548 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-rx8581.ko 2024-09-10T23:45:50.548 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-stk17ta8.ko 2024-09-10T23:45:50.548 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-x1205.ko 2024-09-10T23:45:50.548 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi 2024-09-10T23:45:50.548 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/3w-9xxx.ko 2024-09-10T23:45:50.548 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/3w-sas.ko 2024-09-10T23:45:50.549 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/3w-xxxx.ko 2024-09-10T23:45:50.549 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/BusLogic.ko 2024-09-10T23:45:50.549 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/a100u2w.ko 2024-09-10T23:45:50.549 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aacraid 2024-09-10T23:45:50.549 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aacraid/aacraid.ko 2024-09-10T23:45:50.549 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/advansys.ko 2024-09-10T23:45:50.549 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aic7xxx 2024-09-10T23:45:50.549 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2024-09-10T23:45:50.549 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2024-09-10T23:45:50.549 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/am53c974.ko 2024-09-10T23:45:50.549 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/arcmsr 2024-09-10T23:45:50.549 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/arcmsr/arcmsr.ko 2024-09-10T23:45:50.550 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/atp870u.ko 2024-09-10T23:45:50.550 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/be2iscsi 2024-09-10T23:45:50.550 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2024-09-10T23:45:50.550 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bfa 2024-09-10T23:45:50.550 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bfa/bfa.ko 2024-09-10T23:45:50.550 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bnx2fc 2024-09-10T23:45:50.550 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2024-09-10T23:45:50.550 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bnx2i 2024-09-10T23:45:50.550 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bnx2i/bnx2i.ko 2024-09-10T23:45:50.550 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/ch.ko 2024-09-10T23:45:50.550 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/csiostor 2024-09-10T23:45:50.550 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/csiostor/csiostor.ko 2024-09-10T23:45:50.551 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi 2024-09-10T23:45:50.551 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi/cxgb3i 2024-09-10T23:45:50.551 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2024-09-10T23:45:50.551 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi/cxgb4i 2024-09-10T23:45:50.551 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2024-09-10T23:45:50.551 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2024-09-10T23:45:50.551 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/dc395x.ko 2024-09-10T23:45:50.551 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/device_handler 2024-09-10T23:45:50.551 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2024-09-10T23:45:50.551 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2024-09-10T23:45:50.552 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2024-09-10T23:45:50.552 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2024-09-10T23:45:50.552 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/dmx3191d.ko 2024-09-10T23:45:50.552 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/esas2r 2024-09-10T23:45:50.552 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/esas2r/esas2r.ko 2024-09-10T23:45:50.552 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/esp_scsi.ko 2024-09-10T23:45:50.552 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/fcoe 2024-09-10T23:45:50.552 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/fcoe/fcoe.ko 2024-09-10T23:45:50.552 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/fcoe/libfcoe.ko 2024-09-10T23:45:50.553 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/fnic 2024-09-10T23:45:50.553 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/fnic/fnic.ko 2024-09-10T23:45:50.553 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/hpsa.ko 2024-09-10T23:45:50.553 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/hptiop.ko 2024-09-10T23:45:50.553 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/hv_storvsc.ko 2024-09-10T23:45:50.553 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/initio.ko 2024-09-10T23:45:50.553 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/ipr.ko 2024-09-10T23:45:50.553 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/ips.ko 2024-09-10T23:45:50.553 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/isci 2024-09-10T23:45:50.553 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/isci/isci.ko 2024-09-10T23:45:50.553 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2024-09-10T23:45:50.553 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/iscsi_tcp.ko 2024-09-10T23:45:50.553 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libfc 2024-09-10T23:45:50.554 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libfc/libfc.ko 2024-09-10T23:45:50.554 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libiscsi.ko 2024-09-10T23:45:50.554 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libiscsi_tcp.ko 2024-09-10T23:45:50.554 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libsas 2024-09-10T23:45:50.554 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libsas/libsas.ko 2024-09-10T23:45:50.554 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/lpfc 2024-09-10T23:45:50.554 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/lpfc/lpfc.ko 2024-09-10T23:45:50.554 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/megaraid 2024-09-10T23:45:50.554 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/megaraid.ko 2024-09-10T23:45:50.554 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2024-09-10T23:45:50.554 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2024-09-10T23:45:50.554 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2024-09-10T23:45:50.555 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/mpt3sas 2024-09-10T23:45:50.555 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2024-09-10T23:45:50.555 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/mvsas 2024-09-10T23:45:50.555 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/mvsas/mvsas.ko 2024-09-10T23:45:50.555 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/mvumi.ko 2024-09-10T23:45:50.555 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/pm8001 2024-09-10T23:45:50.555 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/pm8001/pm80xx.ko 2024-09-10T23:45:50.555 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/pmcraid.ko 2024-09-10T23:45:50.555 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla1280.ko 2024-09-10T23:45:50.556 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla2xxx 2024-09-10T23:45:50.556 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2024-09-10T23:45:50.556 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2024-09-10T23:45:50.556 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla4xxx 2024-09-10T23:45:50.556 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2024-09-10T23:45:50.556 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/raid_class.ko 2024-09-10T23:45:50.556 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_debug.ko 2024-09-10T23:45:50.556 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_transport_fc.ko 2024-09-10T23:45:50.556 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_transport_iscsi.ko 2024-09-10T23:45:50.556 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_transport_sas.ko 2024-09-10T23:45:50.556 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_transport_spi.ko 2024-09-10T23:45:50.556 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_transport_srp.ko 2024-09-10T23:45:50.557 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/ses.ko 2024-09-10T23:45:50.557 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/st.ko 2024-09-10T23:45:50.557 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/stex.ko 2024-09-10T23:45:50.557 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/sym53c8xx_2 2024-09-10T23:45:50.557 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2024-09-10T23:45:50.557 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/virtio_scsi.ko 2024-09-10T23:45:50.557 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/vmw_pvscsi.ko 2024-09-10T23:45:50.557 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/wd719x.ko 2024-09-10T23:45:50.557 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ssb 2024-09-10T23:45:50.557 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ssb/ssb.ko 2024-09-10T23:45:50.557 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target 2024-09-10T23:45:50.557 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/iscsi 2024-09-10T23:45:50.558 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2024-09-10T23:45:50.558 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/loopback 2024-09-10T23:45:50.558 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/loopback/tcm_loop.ko 2024-09-10T23:45:50.558 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/target_core_file.ko 2024-09-10T23:45:50.558 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/target_core_iblock.ko 2024-09-10T23:45:50.558 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/target_core_mod.ko 2024-09-10T23:45:50.558 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/target_core_pscsi.ko 2024-09-10T23:45:50.558 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/target_core_user.ko 2024-09-10T23:45:50.558 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/tcm_fc 2024-09-10T23:45:50.558 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/tcm_fc/tcm_fc.ko 2024-09-10T23:45:50.558 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal 2024-09-10T23:45:50.558 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel 2024-09-10T23:45:50.559 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal 2024-09-10T23:45:50.559 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2024-09-10T23:45:50.559 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2024-09-10T23:45:50.559 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2024-09-10T23:45:50.559 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2024-09-10T23:45:50.559 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2024-09-10T23:45:50.559 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2024-09-10T23:45:50.559 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2024-09-10T23:45:50.559 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2024-09-10T23:45:50.559 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2024-09-10T23:45:50.559 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2024-09-10T23:45:50.559 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_power_floor.ko 2024-09-10T23:45:50.559 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2024-09-10T23:45:50.560 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2024-09-10T23:45:50.560 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_hint.ko 2024-09-10T23:45:50.560 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_req.ko 2024-09-10T23:45:50.560 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2024-09-10T23:45:50.560 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2024-09-10T23:45:50.560 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2024-09-10T23:45:50.560 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty 2024-09-10T23:45:50.560 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/ipwireless 2024-09-10T23:45:50.560 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/ipwireless/ipwireless.ko 2024-09-10T23:45:50.560 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/n_gsm.ko 2024-09-10T23:45:50.560 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/n_hdlc.ko 2024-09-10T23:45:50.560 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/nozomi.ko 2024-09-10T23:45:50.560 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial 2024-09-10T23:45:50.561 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/8250 2024-09-10T23:45:50.561 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/8250/serial_cs.ko 2024-09-10T23:45:50.561 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/arc_uart.ko 2024-09-10T23:45:50.561 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/jsm 2024-09-10T23:45:50.561 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/jsm/jsm.ko 2024-09-10T23:45:50.561 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ufs 2024-09-10T23:45:50.561 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ufs/core 2024-09-10T23:45:50.561 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ufs/core/ufshcd-core.ko 2024-09-10T23:45:50.561 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ufs/host 2024-09-10T23:45:50.561 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ufs/host/ufshcd-pci.ko 2024-09-10T23:45:50.561 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio 2024-09-10T23:45:50.561 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio.ko 2024-09-10T23:45:50.561 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio_aec.ko 2024-09-10T23:45:50.562 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio_cif.ko 2024-09-10T23:45:50.562 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio_pci_generic.ko 2024-09-10T23:45:50.562 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio_sercos3.ko 2024-09-10T23:45:50.562 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb 2024-09-10T23:45:50.562 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/class 2024-09-10T23:45:50.562 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/class/cdc-acm.ko 2024-09-10T23:45:50.562 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/class/cdc-wdm.ko 2024-09-10T23:45:50.562 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/class/usblp.ko 2024-09-10T23:45:50.562 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/class/usbtmc.ko 2024-09-10T23:45:50.562 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/host 2024-09-10T23:45:50.562 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/host/sl811-hcd.ko 2024-09-10T23:45:50.562 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/image 2024-09-10T23:45:50.562 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/image/mdc800.ko 2024-09-10T23:45:50.563 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/image/microtek.ko 2024-09-10T23:45:50.563 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc 2024-09-10T23:45:50.563 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/adutux.ko 2024-09-10T23:45:50.563 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/appledisplay.ko 2024-09-10T23:45:50.563 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/emi26.ko 2024-09-10T23:45:50.563 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/emi62.ko 2024-09-10T23:45:50.563 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/ezusb.ko 2024-09-10T23:45:50.563 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/idmouse.ko 2024-09-10T23:45:50.563 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/iowarrior.ko 2024-09-10T23:45:50.563 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/isight_firmware.ko 2024-09-10T23:45:50.563 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/ldusb.ko 2024-09-10T23:45:50.563 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/legousbtower.ko 2024-09-10T23:45:50.563 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/sisusbvga 2024-09-10T23:45:50.563 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2024-09-10T23:45:50.564 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/trancevibrator.ko 2024-09-10T23:45:50.564 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/usb3503.ko 2024-09-10T23:45:50.564 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/usblcd.ko 2024-09-10T23:45:50.564 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/usbsevseg.ko 2024-09-10T23:45:50.564 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/uss720.ko 2024-09-10T23:45:50.564 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/yurex.ko 2024-09-10T23:45:50.564 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/phy 2024-09-10T23:45:50.564 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/phy/phy-generic.ko 2024-09-10T23:45:50.564 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial 2024-09-10T23:45:50.564 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/aircable.ko 2024-09-10T23:45:50.564 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ark3116.ko 2024-09-10T23:45:50.565 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/belkin_sa.ko 2024-09-10T23:45:50.565 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ch341.ko 2024-09-10T23:45:50.565 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/cp210x.ko 2024-09-10T23:45:50.565 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/cyberjack.ko 2024-09-10T23:45:50.565 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/cypress_m8.ko 2024-09-10T23:45:50.565 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/digi_acceleport.ko 2024-09-10T23:45:50.565 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/empeg.ko 2024-09-10T23:45:50.565 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ftdi_sio.ko 2024-09-10T23:45:50.565 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/garmin_gps.ko 2024-09-10T23:45:50.565 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/io_edgeport.ko 2024-09-10T23:45:50.566 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/io_ti.ko 2024-09-10T23:45:50.566 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ipaq.ko 2024-09-10T23:45:50.566 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ipw.ko 2024-09-10T23:45:50.566 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ir-usb.ko 2024-09-10T23:45:50.566 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/iuu_phoenix.ko 2024-09-10T23:45:50.566 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/keyspan.ko 2024-09-10T23:45:50.566 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/keyspan_pda.ko 2024-09-10T23:45:50.566 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/kl5kusb105.ko 2024-09-10T23:45:50.566 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/kobil_sct.ko 2024-09-10T23:45:50.566 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/mct_u232.ko 2024-09-10T23:45:50.567 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/mos7720.ko 2024-09-10T23:45:50.567 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/mos7840.ko 2024-09-10T23:45:50.567 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/navman.ko 2024-09-10T23:45:50.567 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/omninet.ko 2024-09-10T23:45:50.567 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/opticon.ko 2024-09-10T23:45:50.567 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/option.ko 2024-09-10T23:45:50.567 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/oti6858.ko 2024-09-10T23:45:50.567 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/pl2303.ko 2024-09-10T23:45:50.567 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/qcaux.ko 2024-09-10T23:45:50.567 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/qcserial.ko 2024-09-10T23:45:50.567 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/quatech2.ko 2024-09-10T23:45:50.567 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/safe_serial.ko 2024-09-10T23:45:50.567 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/sierra.ko 2024-09-10T23:45:50.567 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/spcp8x5.ko 2024-09-10T23:45:50.568 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ssu100.ko 2024-09-10T23:45:50.568 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/symbolserial.ko 2024-09-10T23:45:50.568 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2024-09-10T23:45:50.568 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/usb-serial-simple.ko 2024-09-10T23:45:50.568 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/usb_debug.ko 2024-09-10T23:45:50.568 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/usb_wwan.ko 2024-09-10T23:45:50.568 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/visor.ko 2024-09-10T23:45:50.568 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/whiteheat.ko 2024-09-10T23:45:50.568 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/xsens_mt.ko 2024-09-10T23:45:50.568 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage 2024-09-10T23:45:50.568 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/uas.ko 2024-09-10T23:45:50.568 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-alauda.ko 2024-09-10T23:45:50.568 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-cypress.ko 2024-09-10T23:45:50.569 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-datafab.ko 2024-09-10T23:45:50.569 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-eneub6250.ko 2024-09-10T23:45:50.569 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-freecom.ko 2024-09-10T23:45:50.569 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-isd200.ko 2024-09-10T23:45:50.569 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-jumpshot.ko 2024-09-10T23:45:50.569 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-karma.ko 2024-09-10T23:45:50.569 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-onetouch.ko 2024-09-10T23:45:50.569 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-realtek.ko 2024-09-10T23:45:50.569 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-sddr09.ko 2024-09-10T23:45:50.569 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-sddr55.ko 2024-09-10T23:45:50.569 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-usbat.ko 2024-09-10T23:45:50.569 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/usb-storage.ko 2024-09-10T23:45:50.569 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/usbip 2024-09-10T23:45:50.569 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/usbip/usbip-core.ko 2024-09-10T23:45:50.570 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/usbip/usbip-host.ko 2024-09-10T23:45:50.570 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/usbip/vhci-hcd.ko 2024-09-10T23:45:50.570 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio 2024-09-10T23:45:50.570 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio/pci 2024-09-10T23:45:50.570 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio/pci/vfio-pci-core.ko 2024-09-10T23:45:50.570 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio/pci/vfio-pci.ko 2024-09-10T23:45:50.570 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio/vfio.ko 2024-09-10T23:45:50.570 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio/vfio_iommu_type1.ko 2024-09-10T23:45:50.570 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vhost 2024-09-10T23:45:50.570 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vhost/vhost.ko 2024-09-10T23:45:50.570 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vhost/vhost_iotlb.ko 2024-09-10T23:45:50.570 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vhost/vhost_net.ko 2024-09-10T23:45:50.570 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vhost/vhost_scsi.ko 2024-09-10T23:45:50.570 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video 2024-09-10T23:45:50.570 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight 2024-09-10T23:45:50.571 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/apple_bl.ko 2024-09-10T23:45:50.571 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/lcd.ko 2024-09-10T23:45:50.571 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/platform_lcd.ko 2024-09-10T23:45:50.571 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/virtio 2024-09-10T23:45:50.571 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/virtio/virtio.ko 2024-09-10T23:45:50.571 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/virtio/virtio_balloon.ko 2024-09-10T23:45:50.571 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/virtio/virtio_mmio.ko 2024-09-10T23:45:50.571 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/virtio/virtio_pci.ko 2024-09-10T23:45:50.571 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2024-09-10T23:45:50.571 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2024-09-10T23:45:50.571 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/virtio/virtio_ring.ko 2024-09-10T23:45:50.571 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1 2024-09-10T23:45:50.571 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/masters 2024-09-10T23:45:50.571 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/masters/ds2482.ko 2024-09-10T23:45:50.572 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/masters/ds2490.ko 2024-09-10T23:45:50.572 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves 2024-09-10T23:45:50.572 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2406.ko 2024-09-10T23:45:50.572 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2408.ko 2024-09-10T23:45:50.572 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2413.ko 2024-09-10T23:45:50.572 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2423.ko 2024-09-10T23:45:50.572 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2431.ko 2024-09-10T23:45:50.572 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2433.ko 2024-09-10T23:45:50.572 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2780.ko 2024-09-10T23:45:50.572 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2781.ko 2024-09-10T23:45:50.572 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds28e04.ko 2024-09-10T23:45:50.572 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_smem.ko 2024-09-10T23:45:50.572 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_therm.ko 2024-09-10T23:45:50.572 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/wire.ko 2024-09-10T23:45:50.573 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog 2024-09-10T23:45:50.573 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/alim1535_wdt.ko 2024-09-10T23:45:50.573 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/alim7101_wdt.ko 2024-09-10T23:45:50.573 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/f71808e_wdt.ko 2024-09-10T23:45:50.573 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/hpwdt.ko 2024-09-10T23:45:50.573 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/i6300esb.ko 2024-09-10T23:45:50.573 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/iTCO_vendor_support.ko 2024-09-10T23:45:50.573 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/iTCO_wdt.ko 2024-09-10T23:45:50.573 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/ib700wdt.ko 2024-09-10T23:45:50.573 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/ibmasr.ko 2024-09-10T23:45:50.573 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/ie6xx_wdt.ko 2024-09-10T23:45:50.573 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/it8712f_wdt.ko 2024-09-10T23:45:50.573 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/it87_wdt.ko 2024-09-10T23:45:50.574 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/machzwd.ko 2024-09-10T23:45:50.574 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/nv_tco.ko 2024-09-10T23:45:50.574 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/pcwd_pci.ko 2024-09-10T23:45:50.574 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/pcwd_usb.ko 2024-09-10T23:45:50.574 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2024-09-10T23:45:50.574 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/sch311x_wdt.ko 2024-09-10T23:45:50.574 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/softdog.ko 2024-09-10T23:45:50.574 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/sp5100_tco.ko 2024-09-10T23:45:50.574 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/via_wdt.ko 2024-09-10T23:45:50.574 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/w83627hf_wdt.ko 2024-09-10T23:45:50.574 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/w83877f_wdt.ko 2024-09-10T23:45:50.574 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/w83977f_wdt.ko 2024-09-10T23:45:50.574 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/wdt_pci.ko 2024-09-10T23:45:50.574 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/xen_wdt.ko 2024-09-10T23:45:50.574 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen 2024-09-10T23:45:50.575 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-acpi-processor.ko 2024-09-10T23:45:50.575 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-evtchn.ko 2024-09-10T23:45:50.575 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-gntalloc.ko 2024-09-10T23:45:50.575 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-gntdev.ko 2024-09-10T23:45:50.575 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-pciback 2024-09-10T23:45:50.575 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2024-09-10T23:45:50.575 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-privcmd.ko 2024-09-10T23:45:50.575 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xenfs 2024-09-10T23:45:50.575 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xenfs/xenfs.ko 2024-09-10T23:45:50.575 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs 2024-09-10T23:45:50.575 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/9p 2024-09-10T23:45:50.575 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/9p/9p.ko 2024-09-10T23:45:50.575 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/affs 2024-09-10T23:45:50.576 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/affs/affs.ko 2024-09-10T23:45:50.576 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/befs 2024-09-10T23:45:50.576 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/befs/befs.ko 2024-09-10T23:45:50.576 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/binfmt_misc.ko 2024-09-10T23:45:50.576 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/btrfs 2024-09-10T23:45:50.576 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/btrfs/btrfs.ko 2024-09-10T23:45:50.576 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/cachefiles 2024-09-10T23:45:50.576 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/cachefiles/cachefiles.ko 2024-09-10T23:45:50.576 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ceph 2024-09-10T23:45:50.577 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ceph/ceph.ko 2024-09-10T23:45:50.577 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/coda 2024-09-10T23:45:50.577 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/coda/coda.ko 2024-09-10T23:45:50.577 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/cramfs 2024-09-10T23:45:50.577 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/cramfs/cramfs.ko 2024-09-10T23:45:50.577 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/dlm 2024-09-10T23:45:50.577 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/dlm/dlm.ko 2024-09-10T23:45:50.577 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ecryptfs 2024-09-10T23:45:50.577 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ecryptfs/ecryptfs.ko 2024-09-10T23:45:50.577 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/f2fs 2024-09-10T23:45:50.577 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/f2fs/f2fs.ko 2024-09-10T23:45:50.578 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fat 2024-09-10T23:45:50.578 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fat/fat.ko 2024-09-10T23:45:50.578 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fat/msdos.ko 2024-09-10T23:45:50.578 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fat/vfat.ko 2024-09-10T23:45:50.578 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fuse 2024-09-10T23:45:50.578 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fuse/cuse.ko 2024-09-10T23:45:50.578 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fuse/fuse.ko 2024-09-10T23:45:50.578 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/gfs2 2024-09-10T23:45:50.578 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/gfs2/gfs2.ko 2024-09-10T23:45:50.578 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/hfs 2024-09-10T23:45:50.578 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/hfs/hfs.ko 2024-09-10T23:45:50.578 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/hfsplus 2024-09-10T23:45:50.578 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/hfsplus/hfsplus.ko 2024-09-10T23:45:50.578 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/isofs 2024-09-10T23:45:50.579 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/isofs/isofs.ko 2024-09-10T23:45:50.579 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/jfs 2024-09-10T23:45:50.579 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/jfs/jfs.ko 2024-09-10T23:45:50.579 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/lockd 2024-09-10T23:45:50.579 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/lockd/lockd.ko 2024-09-10T23:45:50.579 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/minix 2024-09-10T23:45:50.579 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/minix/minix.ko 2024-09-10T23:45:50.579 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/netfs 2024-09-10T23:45:50.579 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/netfs/netfs.ko 2024-09-10T23:45:50.579 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs 2024-09-10T23:45:50.579 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/blocklayout 2024-09-10T23:45:50.580 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2024-09-10T23:45:50.580 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/filelayout 2024-09-10T23:45:50.580 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2024-09-10T23:45:50.580 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/flexfilelayout 2024-09-10T23:45:50.580 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2024-09-10T23:45:50.580 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/nfs.ko 2024-09-10T23:45:50.580 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/nfsv3.ko 2024-09-10T23:45:50.580 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/nfsv4.ko 2024-09-10T23:45:50.580 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs_common 2024-09-10T23:45:50.580 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs_common/grace.ko 2024-09-10T23:45:50.580 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs_common/nfs_acl.ko 2024-09-10T23:45:50.580 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfsd 2024-09-10T23:45:50.580 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfsd/nfsd.ko 2024-09-10T23:45:50.581 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nilfs2 2024-09-10T23:45:50.581 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nilfs2/nilfs2.ko 2024-09-10T23:45:50.581 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls 2024-09-10T23:45:50.581 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-celtic.ko 2024-09-10T23:45:50.581 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-centeuro.ko 2024-09-10T23:45:50.581 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-croatian.ko 2024-09-10T23:45:50.581 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-cyrillic.ko 2024-09-10T23:45:50.581 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-gaelic.ko 2024-09-10T23:45:50.581 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-greek.ko 2024-09-10T23:45:50.581 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-iceland.ko 2024-09-10T23:45:50.581 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-inuit.ko 2024-09-10T23:45:50.581 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-roman.ko 2024-09-10T23:45:50.581 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-romanian.ko 2024-09-10T23:45:50.581 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-turkish.ko 2024-09-10T23:45:50.582 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp1250.ko 2024-09-10T23:45:50.582 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp1251.ko 2024-09-10T23:45:50.582 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp1255.ko 2024-09-10T23:45:50.582 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp737.ko 2024-09-10T23:45:50.582 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp775.ko 2024-09-10T23:45:50.582 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp850.ko 2024-09-10T23:45:50.582 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp852.ko 2024-09-10T23:45:50.582 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp855.ko 2024-09-10T23:45:50.582 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp857.ko 2024-09-10T23:45:50.582 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp860.ko 2024-09-10T23:45:50.582 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp861.ko 2024-09-10T23:45:50.582 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp862.ko 2024-09-10T23:45:50.582 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp863.ko 2024-09-10T23:45:50.582 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp864.ko 2024-09-10T23:45:50.582 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp865.ko 2024-09-10T23:45:50.583 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp866.ko 2024-09-10T23:45:50.583 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp869.ko 2024-09-10T23:45:50.583 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp874.ko 2024-09-10T23:45:50.583 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp932.ko 2024-09-10T23:45:50.583 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp936.ko 2024-09-10T23:45:50.583 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp949.ko 2024-09-10T23:45:50.583 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp950.ko 2024-09-10T23:45:50.583 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_euc-jp.ko 2024-09-10T23:45:50.583 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-1.ko 2024-09-10T23:45:50.583 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-13.ko 2024-09-10T23:45:50.583 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-14.ko 2024-09-10T23:45:50.583 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-15.ko 2024-09-10T23:45:50.583 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-2.ko 2024-09-10T23:45:50.583 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-3.ko 2024-09-10T23:45:50.584 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-4.ko 2024-09-10T23:45:50.584 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-5.ko 2024-09-10T23:45:50.584 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-6.ko 2024-09-10T23:45:50.584 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-7.ko 2024-09-10T23:45:50.584 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-9.ko 2024-09-10T23:45:50.584 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_koi8-r.ko 2024-09-10T23:45:50.584 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_koi8-ru.ko 2024-09-10T23:45:50.584 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_koi8-u.ko 2024-09-10T23:45:50.584 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_ucs2_utils.ko 2024-09-10T23:45:50.584 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_utf8.ko 2024-09-10T23:45:50.584 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2 2024-09-10T23:45:50.584 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/cluster 2024-09-10T23:45:50.584 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2024-09-10T23:45:50.585 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/dlm 2024-09-10T23:45:50.585 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2024-09-10T23:45:50.585 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/dlmfs 2024-09-10T23:45:50.585 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2024-09-10T23:45:50.585 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/ocfs2.ko 2024-09-10T23:45:50.585 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2024-09-10T23:45:50.585 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/ocfs2_stack_user.ko 2024-09-10T23:45:50.585 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/ocfs2_stackglue.ko 2024-09-10T23:45:50.585 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/overlayfs 2024-09-10T23:45:50.586 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/overlayfs/overlay.ko 2024-09-10T23:45:50.586 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/pstore 2024-09-10T23:45:50.586 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/pstore/ramoops.ko 2024-09-10T23:45:50.586 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/reiserfs 2024-09-10T23:45:50.586 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/reiserfs/reiserfs.ko 2024-09-10T23:45:50.586 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/romfs 2024-09-10T23:45:50.586 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/romfs/romfs.ko 2024-09-10T23:45:50.587 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb 2024-09-10T23:45:50.587 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb/client 2024-09-10T23:45:50.587 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb/client/cifs.ko 2024-09-10T23:45:50.587 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb/common 2024-09-10T23:45:50.587 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb/common/cifs_arc4.ko 2024-09-10T23:45:50.587 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb/common/cifs_md4.ko 2024-09-10T23:45:50.587 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/squashfs 2024-09-10T23:45:50.587 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/squashfs/squashfs.ko 2024-09-10T23:45:50.587 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/sysv 2024-09-10T23:45:50.588 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/sysv/sysv.ko 2024-09-10T23:45:50.588 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/udf 2024-09-10T23:45:50.588 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/udf/udf.ko 2024-09-10T23:45:50.588 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ufs 2024-09-10T23:45:50.588 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ufs/ufs.ko 2024-09-10T23:45:50.588 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/xfs 2024-09-10T23:45:50.588 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/xfs/xfs.ko 2024-09-10T23:45:50.588 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel 2024-09-10T23:45:50.589 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel/rcu 2024-09-10T23:45:50.589 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel/rcu/rcutorture.ko 2024-09-10T23:45:50.589 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel/torture.ko 2024-09-10T23:45:50.589 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel/trace 2024-09-10T23:45:50.589 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel/trace/ring_buffer_benchmark.ko 2024-09-10T23:45:50.589 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib 2024-09-10T23:45:50.589 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/asn1_encoder.ko 2024-09-10T23:45:50.590 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crc-itu-t.ko 2024-09-10T23:45:50.590 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crc8.ko 2024-09-10T23:45:50.590 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crypto 2024-09-10T23:45:50.590 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crypto/libaescfb.ko 2024-09-10T23:45:50.590 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crypto/libarc4.ko 2024-09-10T23:45:50.590 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crypto/libdes.ko 2024-09-10T23:45:50.590 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/libcrc32c.ko 2024-09-10T23:45:50.590 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/lru_cache.ko 2024-09-10T23:45:50.590 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/lz4 2024-09-10T23:45:50.591 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/lz4/lz4_compress.ko 2024-09-10T23:45:50.591 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/lz4/lz4hc_compress.ko 2024-09-10T23:45:50.591 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/math 2024-09-10T23:45:50.591 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/math/cordic.ko 2024-09-10T23:45:50.591 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/raid6 2024-09-10T23:45:50.591 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/raid6/raid6_pq.ko 2024-09-10T23:45:50.591 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/reed_solomon 2024-09-10T23:45:50.591 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/reed_solomon/reed_solomon.ko 2024-09-10T23:45:50.592 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/ts_bm.ko 2024-09-10T23:45:50.592 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/ts_fsm.ko 2024-09-10T23:45:50.592 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/ts_kmp.ko 2024-09-10T23:45:50.592 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/zstd 2024-09-10T23:45:50.592 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/zstd/zstd_compress.ko 2024-09-10T23:45:50.592 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/mm 2024-09-10T23:45:50.592 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/mm/hwpoison-inject.ko 2024-09-10T23:45:50.592 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net 2024-09-10T23:45:50.593 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan 2024-09-10T23:45:50.593 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/6lowpan.ko 2024-09-10T23:45:50.593 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_dest.ko 2024-09-10T23:45:50.593 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_fragment.ko 2024-09-10T23:45:50.593 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_hop.ko 2024-09-10T23:45:50.593 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_ipv6.ko 2024-09-10T23:45:50.593 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_mobility.ko 2024-09-10T23:45:50.593 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_routing.ko 2024-09-10T23:45:50.594 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_udp.ko 2024-09-10T23:45:50.594 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802 2024-09-10T23:45:50.594 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802/garp.ko 2024-09-10T23:45:50.594 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802/mrp.ko 2024-09-10T23:45:50.594 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802/p8022.ko 2024-09-10T23:45:50.594 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802/psnap.ko 2024-09-10T23:45:50.594 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802/stp.ko 2024-09-10T23:45:50.594 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/8021q 2024-09-10T23:45:50.594 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/8021q/8021q.ko 2024-09-10T23:45:50.595 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/9p 2024-09-10T23:45:50.595 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/9p/9pnet.ko 2024-09-10T23:45:50.595 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/9p/9pnet_fd.ko 2024-09-10T23:45:50.595 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/9p/9pnet_rdma.ko 2024-09-10T23:45:50.595 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/9p/9pnet_virtio.ko 2024-09-10T23:45:50.595 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/batman-adv 2024-09-10T23:45:50.595 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/batman-adv/batman-adv.ko 2024-09-10T23:45:50.595 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge 2024-09-10T23:45:50.596 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/br_netfilter.ko 2024-09-10T23:45:50.596 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/bridge.ko 2024-09-10T23:45:50.596 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter 2024-09-10T23:45:50.596 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_802_3.ko 2024-09-10T23:45:50.596 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_among.ko 2024-09-10T23:45:50.596 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_arp.ko 2024-09-10T23:45:50.596 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_arpreply.ko 2024-09-10T23:45:50.596 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_dnat.ko 2024-09-10T23:45:50.597 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_ip.ko 2024-09-10T23:45:50.597 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_ip6.ko 2024-09-10T23:45:50.597 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_limit.ko 2024-09-10T23:45:50.597 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_log.ko 2024-09-10T23:45:50.597 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_mark.ko 2024-09-10T23:45:50.597 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_mark_m.ko 2024-09-10T23:45:50.597 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_nflog.ko 2024-09-10T23:45:50.597 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_pkttype.ko 2024-09-10T23:45:50.598 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_redirect.ko 2024-09-10T23:45:50.598 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_snat.ko 2024-09-10T23:45:50.598 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_stp.ko 2024-09-10T23:45:50.598 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_vlan.ko 2024-09-10T23:45:50.598 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebtable_broute.ko 2024-09-10T23:45:50.598 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebtable_filter.ko 2024-09-10T23:45:50.598 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebtable_nat.ko 2024-09-10T23:45:50.598 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebtables.ko 2024-09-10T23:45:50.599 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2024-09-10T23:45:50.599 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2024-09-10T23:45:50.599 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2024-09-10T23:45:50.599 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ceph 2024-09-10T23:45:50.599 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ceph/libceph.ko 2024-09-10T23:45:50.599 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/core 2024-09-10T23:45:50.599 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/core/failover.ko 2024-09-10T23:45:50.599 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/core/pktgen.ko 2024-09-10T23:45:50.599 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dccp 2024-09-10T23:45:50.599 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dccp/dccp.ko 2024-09-10T23:45:50.599 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dccp/dccp_diag.ko 2024-09-10T23:45:50.599 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dccp/dccp_ipv4.ko 2024-09-10T23:45:50.599 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dccp/dccp_ipv6.ko 2024-09-10T23:45:50.599 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dns_resolver 2024-09-10T23:45:50.600 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dns_resolver/dns_resolver.ko 2024-09-10T23:45:50.600 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa 2024-09-10T23:45:50.600 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa/dsa_core.ko 2024-09-10T23:45:50.600 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa/tag_brcm.ko 2024-09-10T23:45:50.600 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa/tag_dsa.ko 2024-09-10T23:45:50.600 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa/tag_none.ko 2024-09-10T23:45:50.600 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa/tag_trailer.ko 2024-09-10T23:45:50.600 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ieee802154 2024-09-10T23:45:50.600 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ieee802154/6lowpan 2024-09-10T23:45:50.600 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2024-09-10T23:45:50.600 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ieee802154/ieee802154.ko 2024-09-10T23:45:50.600 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ieee802154/ieee802154_socket.ko 2024-09-10T23:45:50.600 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4 2024-09-10T23:45:50.600 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ah4.ko 2024-09-10T23:45:50.601 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/esp4.ko 2024-09-10T23:45:50.601 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/fou.ko 2024-09-10T23:45:50.601 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/gre.ko 2024-09-10T23:45:50.601 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/inet_diag.ko 2024-09-10T23:45:50.601 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ip_gre.ko 2024-09-10T23:45:50.601 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ip_tunnel.ko 2024-09-10T23:45:50.601 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ip_vti.ko 2024-09-10T23:45:50.601 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ipcomp.ko 2024-09-10T23:45:50.601 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ipip.ko 2024-09-10T23:45:50.601 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter 2024-09-10T23:45:50.601 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/arp_tables.ko 2024-09-10T23:45:50.601 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/arpt_mangle.ko 2024-09-10T23:45:50.601 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/arptable_filter.ko 2024-09-10T23:45:50.602 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/ipt_ECN.ko 2024-09-10T23:45:50.602 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2024-09-10T23:45:50.602 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/ipt_ah.ko 2024-09-10T23:45:50.602 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2024-09-10T23:45:50.602 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/iptable_mangle.ko 2024-09-10T23:45:50.602 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/iptable_nat.ko 2024-09-10T23:45:50.602 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/iptable_raw.ko 2024-09-10T23:45:50.602 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/iptable_security.ko 2024-09-10T23:45:50.602 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2024-09-10T23:45:50.602 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2024-09-10T23:45:50.602 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2024-09-10T23:45:50.602 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2024-09-10T23:45:50.602 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2024-09-10T23:45:50.602 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2024-09-10T23:45:50.603 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2024-09-10T23:45:50.603 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2024-09-10T23:45:50.603 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2024-09-10T23:45:50.603 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2024-09-10T23:45:50.603 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_bic.ko 2024-09-10T23:45:50.603 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_dctcp.ko 2024-09-10T23:45:50.603 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_diag.ko 2024-09-10T23:45:50.603 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_highspeed.ko 2024-09-10T23:45:50.603 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_htcp.ko 2024-09-10T23:45:50.603 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_hybla.ko 2024-09-10T23:45:50.603 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_illinois.ko 2024-09-10T23:45:50.603 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_lp.ko 2024-09-10T23:45:50.603 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_scalable.ko 2024-09-10T23:45:50.603 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_vegas.ko 2024-09-10T23:45:50.603 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_veno.ko 2024-09-10T23:45:50.604 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_westwood.ko 2024-09-10T23:45:50.604 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_yeah.ko 2024-09-10T23:45:50.604 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tunnel4.ko 2024-09-10T23:45:50.604 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/udp_diag.ko 2024-09-10T23:45:50.604 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/udp_tunnel.ko 2024-09-10T23:45:50.604 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/xfrm4_tunnel.ko 2024-09-10T23:45:50.604 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6 2024-09-10T23:45:50.604 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ah6.ko 2024-09-10T23:45:50.604 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/esp6.ko 2024-09-10T23:45:50.604 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/fou6.ko 2024-09-10T23:45:50.604 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ip6_tunnel.ko 2024-09-10T23:45:50.604 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ip6_udp_tunnel.ko 2024-09-10T23:45:50.604 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ip6_vti.ko 2024-09-10T23:45:50.605 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ipcomp6.ko 2024-09-10T23:45:50.605 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter 2024-09-10T23:45:50.605 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6_tables.ko 2024-09-10T23:45:50.605 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2024-09-10T23:45:50.605 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2024-09-10T23:45:50.605 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_ah.ko 2024-09-10T23:45:50.605 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2024-09-10T23:45:50.605 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_frag.ko 2024-09-10T23:45:50.605 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2024-09-10T23:45:50.605 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2024-09-10T23:45:50.605 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_mh.ko 2024-09-10T23:45:50.605 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2024-09-10T23:45:50.605 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_rt.ko 2024-09-10T23:45:50.605 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6table_filter.ko 2024-09-10T23:45:50.606 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2024-09-10T23:45:50.606 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6table_nat.ko 2024-09-10T23:45:50.606 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6table_raw.ko 2024-09-10T23:45:50.606 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6table_security.ko 2024-09-10T23:45:50.606 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2024-09-10T23:45:50.606 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2024-09-10T23:45:50.606 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2024-09-10T23:45:50.606 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2024-09-10T23:45:50.606 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2024-09-10T23:45:50.606 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2024-09-10T23:45:50.606 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2024-09-10T23:45:50.606 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2024-09-10T23:45:50.606 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/sit.ko 2024-09-10T23:45:50.606 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/tunnel6.ko 2024-09-10T23:45:50.607 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/xfrm6_tunnel.ko 2024-09-10T23:45:50.607 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/key 2024-09-10T23:45:50.607 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/key/af_key.ko 2024-09-10T23:45:50.607 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp 2024-09-10T23:45:50.607 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_core.ko 2024-09-10T23:45:50.607 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_debugfs.ko 2024-09-10T23:45:50.607 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_eth.ko 2024-09-10T23:45:50.607 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_ip.ko 2024-09-10T23:45:50.607 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_ip6.ko 2024-09-10T23:45:50.607 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_netlink.ko 2024-09-10T23:45:50.607 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_ppp.ko 2024-09-10T23:45:50.607 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/llc 2024-09-10T23:45:50.607 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/llc/llc.ko 2024-09-10T23:45:50.607 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/mac802154 2024-09-10T23:45:50.608 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/mac802154/mac802154.ko 2024-09-10T23:45:50.608 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/mpls 2024-09-10T23:45:50.608 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/mpls/mpls_gso.ko 2024-09-10T23:45:50.608 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter 2024-09-10T23:45:50.608 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset 2024-09-10T23:45:50.608 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set.ko 2024-09-10T23:45:50.608 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2024-09-10T23:45:50.608 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2024-09-10T23:45:50.608 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2024-09-10T23:45:50.608 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2024-09-10T23:45:50.608 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2024-09-10T23:45:50.608 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2024-09-10T23:45:50.608 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2024-09-10T23:45:50.609 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2024-09-10T23:45:50.609 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2024-09-10T23:45:50.609 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2024-09-10T23:45:50.609 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2024-09-10T23:45:50.609 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2024-09-10T23:45:50.609 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2024-09-10T23:45:50.609 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2024-09-10T23:45:50.609 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_list_set.ko 2024-09-10T23:45:50.609 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs 2024-09-10T23:45:50.609 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs.ko 2024-09-10T23:45:50.609 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2024-09-10T23:45:50.609 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2024-09-10T23:45:50.609 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2024-09-10T23:45:50.610 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2024-09-10T23:45:50.610 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2024-09-10T23:45:50.610 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2024-09-10T23:45:50.610 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2024-09-10T23:45:50.610 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2024-09-10T23:45:50.610 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2024-09-10T23:45:50.610 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2024-09-10T23:45:50.610 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2024-09-10T23:45:50.610 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2024-09-10T23:45:50.610 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2024-09-10T23:45:50.610 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conncount.ko 2024-09-10T23:45:50.610 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack.ko 2024-09-10T23:45:50.610 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_amanda.ko 2024-09-10T23:45:50.610 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_broadcast.ko 2024-09-10T23:45:50.610 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_ftp.ko 2024-09-10T23:45:50.611 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_h323.ko 2024-09-10T23:45:50.611 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_irc.ko 2024-09-10T23:45:50.611 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2024-09-10T23:45:50.611 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_netlink.ko 2024-09-10T23:45:50.611 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_pptp.ko 2024-09-10T23:45:50.611 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_sane.ko 2024-09-10T23:45:50.611 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_sip.ko 2024-09-10T23:45:50.611 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_snmp.ko 2024-09-10T23:45:50.611 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_tftp.ko 2024-09-10T23:45:50.611 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_dup_netdev.ko 2024-09-10T23:45:50.611 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_flow_table.ko 2024-09-10T23:45:50.611 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_flow_table_inet.ko 2024-09-10T23:45:50.611 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_log_syslog.ko 2024-09-10T23:45:50.611 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat.ko 2024-09-10T23:45:50.611 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat_amanda.ko 2024-09-10T23:45:50.612 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat_ftp.ko 2024-09-10T23:45:50.612 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat_irc.ko 2024-09-10T23:45:50.612 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat_sip.ko 2024-09-10T23:45:50.612 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat_tftp.ko 2024-09-10T23:45:50.612 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_synproxy_core.ko 2024-09-10T23:45:50.612 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_tables.ko 2024-09-10T23:45:50.612 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nfnetlink.ko 2024-09-10T23:45:50.612 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nfnetlink_acct.ko 2024-09-10T23:45:50.612 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nfnetlink_log.ko 2024-09-10T23:45:50.612 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nfnetlink_osf.ko 2024-09-10T23:45:50.612 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nfnetlink_queue.ko 2024-09-10T23:45:50.612 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_chain_nat.ko 2024-09-10T23:45:50.612 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_compat.ko 2024-09-10T23:45:50.613 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_ct.ko 2024-09-10T23:45:50.613 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_dup_netdev.ko 2024-09-10T23:45:50.613 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_fib.ko 2024-09-10T23:45:50.613 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_fwd_netdev.ko 2024-09-10T23:45:50.613 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_hash.ko 2024-09-10T23:45:50.613 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_limit.ko 2024-09-10T23:45:50.613 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_log.ko 2024-09-10T23:45:50.613 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_masq.ko 2024-09-10T23:45:50.613 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_nat.ko 2024-09-10T23:45:50.613 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_numgen.ko 2024-09-10T23:45:50.613 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_queue.ko 2024-09-10T23:45:50.613 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_quota.ko 2024-09-10T23:45:50.614 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_redir.ko 2024-09-10T23:45:50.614 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_reject.ko 2024-09-10T23:45:50.614 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_reject_inet.ko 2024-09-10T23:45:50.614 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_socket.ko 2024-09-10T23:45:50.614 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_tunnel.ko 2024-09-10T23:45:50.614 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_AUDIT.ko 2024-09-10T23:45:50.614 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_CHECKSUM.ko 2024-09-10T23:45:50.614 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_CLASSIFY.ko 2024-09-10T23:45:50.614 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_CONNSECMARK.ko 2024-09-10T23:45:50.614 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_CT.ko 2024-09-10T23:45:50.614 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_DSCP.ko 2024-09-10T23:45:50.614 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_HL.ko 2024-09-10T23:45:50.614 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_HMARK.ko 2024-09-10T23:45:50.614 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_IDLETIMER.ko 2024-09-10T23:45:50.615 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_LED.ko 2024-09-10T23:45:50.615 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_LOG.ko 2024-09-10T23:45:50.615 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_MASQUERADE.ko 2024-09-10T23:45:50.615 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_NETMAP.ko 2024-09-10T23:45:50.615 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_NFLOG.ko 2024-09-10T23:45:50.615 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_NFQUEUE.ko 2024-09-10T23:45:50.615 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_RATEEST.ko 2024-09-10T23:45:50.615 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_REDIRECT.ko 2024-09-10T23:45:50.615 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_SECMARK.ko 2024-09-10T23:45:50.615 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_TCPMSS.ko 2024-09-10T23:45:50.615 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2024-09-10T23:45:50.615 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_TEE.ko 2024-09-10T23:45:50.615 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_TPROXY.ko 2024-09-10T23:45:50.615 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_TRACE.ko 2024-09-10T23:45:50.615 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_addrtype.ko 2024-09-10T23:45:50.616 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_bpf.ko 2024-09-10T23:45:50.616 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_cgroup.ko 2024-09-10T23:45:50.616 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_cluster.ko 2024-09-10T23:45:50.616 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_comment.ko 2024-09-10T23:45:50.616 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_connbytes.ko 2024-09-10T23:45:50.616 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_connlabel.ko 2024-09-10T23:45:50.616 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_connlimit.ko 2024-09-10T23:45:50.616 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_connmark.ko 2024-09-10T23:45:50.616 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_conntrack.ko 2024-09-10T23:45:50.616 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_cpu.ko 2024-09-10T23:45:50.616 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_dccp.ko 2024-09-10T23:45:50.617 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_devgroup.ko 2024-09-10T23:45:50.617 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_dscp.ko 2024-09-10T23:45:50.617 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_ecn.ko 2024-09-10T23:45:50.617 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_esp.ko 2024-09-10T23:45:50.617 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_hashlimit.ko 2024-09-10T23:45:50.617 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_helper.ko 2024-09-10T23:45:50.617 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_hl.ko 2024-09-10T23:45:50.617 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_ipcomp.ko 2024-09-10T23:45:50.617 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_iprange.ko 2024-09-10T23:45:50.617 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_ipvs.ko 2024-09-10T23:45:50.617 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_l2tp.ko 2024-09-10T23:45:50.617 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_length.ko 2024-09-10T23:45:50.617 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_limit.ko 2024-09-10T23:45:50.618 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_mac.ko 2024-09-10T23:45:50.618 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_mark.ko 2024-09-10T23:45:50.618 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_multiport.ko 2024-09-10T23:45:50.618 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_nat.ko 2024-09-10T23:45:50.618 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_nfacct.ko 2024-09-10T23:45:50.618 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_osf.ko 2024-09-10T23:45:50.618 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_owner.ko 2024-09-10T23:45:50.618 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_physdev.ko 2024-09-10T23:45:50.618 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_pkttype.ko 2024-09-10T23:45:50.618 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_policy.ko 2024-09-10T23:45:50.618 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_quota.ko 2024-09-10T23:45:50.618 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_rateest.ko 2024-09-10T23:45:50.618 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_realm.ko 2024-09-10T23:45:50.618 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_recent.ko 2024-09-10T23:45:50.619 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_sctp.ko 2024-09-10T23:45:50.619 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_set.ko 2024-09-10T23:45:50.619 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_socket.ko 2024-09-10T23:45:50.619 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_state.ko 2024-09-10T23:45:50.619 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_statistic.ko 2024-09-10T23:45:50.619 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_string.ko 2024-09-10T23:45:50.619 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_tcpmss.ko 2024-09-10T23:45:50.619 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_time.ko 2024-09-10T23:45:50.619 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_u32.ko 2024-09-10T23:45:50.619 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netlink 2024-09-10T23:45:50.619 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netlink/netlink_diag.ko 2024-09-10T23:45:50.619 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/nsh 2024-09-10T23:45:50.620 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/nsh/nsh.ko 2024-09-10T23:45:50.620 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/openvswitch 2024-09-10T23:45:50.620 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/openvswitch/openvswitch.ko 2024-09-10T23:45:50.620 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/openvswitch/vport-geneve.ko 2024-09-10T23:45:50.620 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/openvswitch/vport-gre.ko 2024-09-10T23:45:50.620 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/openvswitch/vport-vxlan.ko 2024-09-10T23:45:50.620 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/packet 2024-09-10T23:45:50.620 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/packet/af_packet_diag.ko 2024-09-10T23:45:50.620 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/rds 2024-09-10T23:45:50.620 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/rds/rds.ko 2024-09-10T23:45:50.620 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/rds/rds_rdma.ko 2024-09-10T23:45:50.620 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/rds/rds_tcp.ko 2024-09-10T23:45:50.620 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched 2024-09-10T23:45:50.620 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_csum.ko 2024-09-10T23:45:50.621 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_gact.ko 2024-09-10T23:45:50.621 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_mirred.ko 2024-09-10T23:45:50.621 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_nat.ko 2024-09-10T23:45:50.621 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_pedit.ko 2024-09-10T23:45:50.621 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_police.ko 2024-09-10T23:45:50.621 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_simple.ko 2024-09-10T23:45:50.621 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_skbedit.ko 2024-09-10T23:45:50.621 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_vlan.ko 2024-09-10T23:45:50.621 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_basic.ko 2024-09-10T23:45:50.621 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_bpf.ko 2024-09-10T23:45:50.621 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_flow.ko 2024-09-10T23:45:50.621 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_fw.ko 2024-09-10T23:45:50.621 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_route.ko 2024-09-10T23:45:50.621 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_u32.ko 2024-09-10T23:45:50.622 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_cmp.ko 2024-09-10T23:45:50.622 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_ipset.ko 2024-09-10T23:45:50.622 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_meta.ko 2024-09-10T23:45:50.622 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_nbyte.ko 2024-09-10T23:45:50.622 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_text.ko 2024-09-10T23:45:50.622 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_u32.ko 2024-09-10T23:45:50.622 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_choke.ko 2024-09-10T23:45:50.622 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_codel.ko 2024-09-10T23:45:50.622 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_drr.ko 2024-09-10T23:45:50.622 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_fq.ko 2024-09-10T23:45:50.622 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_gred.ko 2024-09-10T23:45:50.622 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_hfsc.ko 2024-09-10T23:45:50.622 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_hhf.ko 2024-09-10T23:45:50.622 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_htb.ko 2024-09-10T23:45:50.623 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_ingress.ko 2024-09-10T23:45:50.623 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_mqprio.ko 2024-09-10T23:45:50.623 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_mqprio_lib.ko 2024-09-10T23:45:50.623 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_multiq.ko 2024-09-10T23:45:50.623 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_netem.ko 2024-09-10T23:45:50.623 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_pie.ko 2024-09-10T23:45:50.623 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_plug.ko 2024-09-10T23:45:50.623 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_prio.ko 2024-09-10T23:45:50.623 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_qfq.ko 2024-09-10T23:45:50.623 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_red.ko 2024-09-10T23:45:50.623 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_sfb.ko 2024-09-10T23:45:50.623 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_sfq.ko 2024-09-10T23:45:50.623 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_tbf.ko 2024-09-10T23:45:50.623 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_teql.ko 2024-09-10T23:45:50.623 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sctp 2024-09-10T23:45:50.624 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sctp/sctp.ko 2024-09-10T23:45:50.624 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sctp/sctp_diag.ko 2024-09-10T23:45:50.624 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc 2024-09-10T23:45:50.624 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/auth_gss 2024-09-10T23:45:50.624 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2024-09-10T23:45:50.624 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2024-09-10T23:45:50.624 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/sunrpc.ko 2024-09-10T23:45:50.624 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/xprtrdma 2024-09-10T23:45:50.624 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2024-09-10T23:45:50.624 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/tipc 2024-09-10T23:45:50.624 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/tipc/tipc.ko 2024-09-10T23:45:50.624 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/tipc/tipc_diag.ko 2024-09-10T23:45:50.624 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/unix 2024-09-10T23:45:50.624 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/unix/unix_diag.ko 2024-09-10T23:45:50.624 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock 2024-09-10T23:45:50.625 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2024-09-10T23:45:50.625 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2024-09-10T23:45:50.625 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock/vsock.ko 2024-09-10T23:45:50.625 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock/vsock_diag.ko 2024-09-10T23:45:50.625 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock/vsock_loopback.ko 2024-09-10T23:45:50.625 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/xfrm 2024-09-10T23:45:50.625 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/xfrm/xfrm_ipcomp.ko 2024-09-10T23:45:50.625 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/security 2024-09-10T23:45:50.625 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/security/keys 2024-09-10T23:45:50.625 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/security/keys/encrypted-keys 2024-09-10T23:45:50.625 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2024-09-10T23:45:50.625 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/security/keys/trusted-keys 2024-09-10T23:45:50.625 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/security/keys/trusted-keys/trusted.ko 2024-09-10T23:45:50.626 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.alias 2024-09-10T23:45:50.626 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.alias.bin 2024-09-10T23:45:50.626 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.builtin 2024-09-10T23:45:50.626 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.builtin.alias.bin 2024-09-10T23:45:50.626 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.builtin.bin 2024-09-10T23:45:50.626 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.builtin.modinfo 2024-09-10T23:45:50.626 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.dep 2024-09-10T23:45:50.626 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.dep.bin 2024-09-10T23:45:50.626 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.devname 2024-09-10T23:45:50.626 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.order 2024-09-10T23:45:50.626 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.softdep 2024-09-10T23:45:50.626 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.symbols 2024-09-10T23:45:50.626 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.symbols.bin 2024-09-10T23:45:50.626 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.weakdep 2024-09-10T23:45:50.626 INFO:teuthology.orchestra.run.smithi012.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/vmlinuz 2024-09-10T23:45:50.631 DEBUG:teuthology.task.kernel:get_image_version: 6.11.0-rc5-g2ca44a2f35ca 2024-09-10T23:45:50.631 DEBUG:teuthology.orchestra.run.smithi012:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2024-09-10T23:45:50.655 INFO:teuthology.orchestra.run.smithi012.stdout:postinstall scriptlet (using /bin/sh): 2024-09-10T23:45:50.655 INFO:teuthology.orchestra.run.smithi012.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-09-10T23:45:50.655 INFO:teuthology.orchestra.run.smithi012.stdout: /usr/bin/kernel-install add 6.11.0-rc5-g2ca44a2f35ca /lib/modules/6.11.0-rc5-g2ca44a2f35ca/vmlinuz 2024-09-10T23:45:50.656 INFO:teuthology.orchestra.run.smithi012.stdout:fi 2024-09-10T23:45:50.656 INFO:teuthology.orchestra.run.smithi012.stdout:for file in vmlinuz System.map config; do 2024-09-10T23:45:50.656 INFO:teuthology.orchestra.run.smithi012.stdout: if ! cmp --silent "/lib/modules/6.11.0-rc5-g2ca44a2f35ca/${file}" "/boot/${file}-6.11.0-rc5-g2ca44a2f35ca"; then 2024-09-10T23:45:50.656 INFO:teuthology.orchestra.run.smithi012.stdout: cp "/lib/modules/6.11.0-rc5-g2ca44a2f35ca/${file}" "/boot/${file}-6.11.0-rc5-g2ca44a2f35ca" 2024-09-10T23:45:50.656 INFO:teuthology.orchestra.run.smithi012.stdout: fi 2024-09-10T23:45:50.656 INFO:teuthology.orchestra.run.smithi012.stdout:done 2024-09-10T23:45:50.657 INFO:teuthology.orchestra.run.smithi012.stdout:if [ -d "/lib/modules/6.11.0-rc5-g2ca44a2f35ca/dtb" ] && \ 2024-09-10T23:45:50.657 INFO:teuthology.orchestra.run.smithi012.stdout: ! diff -rq "/lib/modules/6.11.0-rc5-g2ca44a2f35ca/dtb" "/boot/dtb-6.11.0-rc5-g2ca44a2f35ca" >/dev/null 2>&1; then 2024-09-10T23:45:50.657 INFO:teuthology.orchestra.run.smithi012.stdout: rm -rf "/boot/dtb-6.11.0-rc5-g2ca44a2f35ca" 2024-09-10T23:45:50.657 INFO:teuthology.orchestra.run.smithi012.stdout: cp -r "/lib/modules/6.11.0-rc5-g2ca44a2f35ca/dtb" "/boot/dtb-6.11.0-rc5-g2ca44a2f35ca" 2024-09-10T23:45:50.657 INFO:teuthology.orchestra.run.smithi012.stdout:fi 2024-09-10T23:45:50.657 INFO:teuthology.orchestra.run.smithi012.stdout:if [ ! -e "/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.dep" ]; then 2024-09-10T23:45:50.658 INFO:teuthology.orchestra.run.smithi012.stdout: /usr/sbin/depmod 6.11.0-rc5-g2ca44a2f35ca 2024-09-10T23:45:50.658 INFO:teuthology.orchestra.run.smithi012.stdout:fi 2024-09-10T23:45:50.658 INFO:teuthology.orchestra.run.smithi012.stdout:preuninstall scriptlet (using /bin/sh): 2024-09-10T23:45:50.658 INFO:teuthology.orchestra.run.smithi012.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-09-10T23:45:50.658 INFO:teuthology.orchestra.run.smithi012.stdout:kernel-install remove 6.11.0-rc5-g2ca44a2f35ca 2024-09-10T23:45:50.658 INFO:teuthology.orchestra.run.smithi012.stdout:fi 2024-09-10T23:45:50.660 DEBUG:teuthology.orchestra.run.smithi012:> sudo rpm -qi grub2-tools 2024-09-10T23:45:50.734 INFO:teuthology.orchestra.run.smithi012.stdout:Name : grub2-tools 2024-09-10T23:45:50.734 INFO:teuthology.orchestra.run.smithi012.stdout:Epoch : 1 2024-09-10T23:45:50.734 INFO:teuthology.orchestra.run.smithi012.stdout:Version : 2.06 2024-09-10T23:45:50.735 INFO:teuthology.orchestra.run.smithi012.stdout:Release : 82.el9 2024-09-10T23:45:50.735 INFO:teuthology.orchestra.run.smithi012.stdout:Architecture: x86_64 2024-09-10T23:45:50.735 INFO:teuthology.orchestra.run.smithi012.stdout:Install Date: Wed 28 Aug 2024 09:06:27 PM UTC 2024-09-10T23:45:50.735 INFO:teuthology.orchestra.run.smithi012.stdout:Group : Unspecified 2024-09-10T23:45:50.735 INFO:teuthology.orchestra.run.smithi012.stdout:Size : 8274670 2024-09-10T23:45:50.735 INFO:teuthology.orchestra.run.smithi012.stdout:License : GPLv3+ 2024-09-10T23:45:50.735 INFO:teuthology.orchestra.run.smithi012.stdout:Signature : RSA/SHA256, Mon 01 Jul 2024 07:46:42 AM UTC, Key ID 05b555b38483c65d 2024-09-10T23:45:50.735 INFO:teuthology.orchestra.run.smithi012.stdout:Source RPM : grub2-2.06-82.el9.src.rpm 2024-09-10T23:45:50.735 INFO:teuthology.orchestra.run.smithi012.stdout:Build Date : Fri 28 Jun 2024 09:24:44 AM UTC 2024-09-10T23:45:50.735 INFO:teuthology.orchestra.run.smithi012.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-09-10T23:45:50.735 INFO:teuthology.orchestra.run.smithi012.stdout:Packager : builder@centos.org 2024-09-10T23:45:50.735 INFO:teuthology.orchestra.run.smithi012.stdout:Vendor : CentOS 2024-09-10T23:45:50.735 INFO:teuthology.orchestra.run.smithi012.stdout:URL : http://www.gnu.org/software/grub/ 2024-09-10T23:45:50.735 INFO:teuthology.orchestra.run.smithi012.stdout:Summary : Support tools for GRUB. 2024-09-10T23:45:50.736 INFO:teuthology.orchestra.run.smithi012.stdout:Description : 2024-09-10T23:45:50.736 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-10T23:45:50.736 INFO:teuthology.orchestra.run.smithi012.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-09-10T23:45:50.736 INFO:teuthology.orchestra.run.smithi012.stdout:customizable bootloader with modular architecture. It supports a rich 2024-09-10T23:45:50.736 INFO:teuthology.orchestra.run.smithi012.stdout:variety of kernel formats, file systems, computer architectures and 2024-09-10T23:45:50.736 INFO:teuthology.orchestra.run.smithi012.stdout:hardware devices. 2024-09-10T23:45:50.736 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-10T23:45:50.736 INFO:teuthology.orchestra.run.smithi012.stdout:This subpackage provides tools for support of all platforms. 2024-09-10T23:45:50.737 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-09-10T23:45:50.737 INFO:teuthology.task.kernel:Updating grub on smithi012 to boot 6.11.0-rc5-g2ca44a2f35ca 2024-09-10T23:45:50.737 DEBUG:teuthology.orchestra.run.smithi012:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-09-10T23:45:51.335 INFO:teuthology.orchestra.run.smithi012.stderr:Generating grub configuration file ... 2024-09-10T23:45:52.751 INFO:teuthology.orchestra.run.smithi012.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-09-10T23:45:52.766 INFO:teuthology.orchestra.run.smithi012.stderr:done 2024-09-10T23:45:52.768 DEBUG:teuthology.orchestra.run.smithi012:> mktemp 2024-09-10T23:45:52.782 INFO:teuthology.orchestra.run.smithi012.stdout:/tmp/tmp.IbGPI8azoc 2024-09-10T23:45:52.783 DEBUG:teuthology.orchestra.run.smithi012:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.IbGPI8azoc 2024-09-10T23:45:52.850 DEBUG:teuthology.orchestra.run.smithi012:> sudo chmod 0666 /tmp/tmp.IbGPI8azoc 2024-09-10T23:45:52.995 DEBUG:teuthology.orchestra.remote:smithi012:/tmp/tmp.IbGPI8azoc is 6KB 2024-09-10T23:45:53.043 DEBUG:teuthology.orchestra.run.smithi012:> rm -fr /tmp/tmp.IbGPI8azoc 2024-09-10T23:45:53.058 DEBUG:teuthology.orchestra.run.smithi012:> sudo /bin/ls /boot/loader/entries || true 2024-09-10T23:45:53.123 INFO:teuthology.orchestra.run.smithi012.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-0-rescue.conf 2024-09-10T23:45:53.123 INFO:teuthology.orchestra.run.smithi012.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-239.el9.x86_64.conf 2024-09-10T23:45:53.123 INFO:teuthology.orchestra.run.smithi012.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-503.el9.x86_64.conf 2024-09-10T23:45:53.123 INFO:teuthology.orchestra.run.smithi012.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-6.11.0-rc5-g2ca44a2f35ca.conf 2024-09-10T23:45:53.124 DEBUG:teuthology.orchestra.run.smithi012:> sudo grub2-set-default fbf6b6fa6cf04654a0e563f30ecbf43a-6.11.0-rc5-g2ca44a2f35ca 2024-09-10T23:45:53.295 DEBUG:teuthology.orchestra.run.smithi012:> sudo shutdown -r now 2024-09-10T23:46:23.298 INFO:teuthology.task.kernel:Checking client client.0 for new kernel version... 2024-09-10T23:46:23.299 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi012.front.sepia.ceph.com' 2024-09-10T23:46:23.300 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi012.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T23:47:23.310 DEBUG:teuthology.orchestra.remote:timed out 2024-09-10T23:47:32.311 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi012.front.sepia.ceph.com' 2024-09-10T23:47:32.312 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi012.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T23:47:32.690 DEBUG:teuthology.orchestra.run.smithi012:> true 2024-09-10T23:47:33.165 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi012.front.sepia.ceph.com' 2024-09-10T23:47:33.165 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "6.11.0-rc5-g2ca44a2f35ca"... 2024-09-10T23:47:33.166 DEBUG:teuthology.orchestra.run.smithi012:> uname -r 2024-09-10T23:47:33.201 INFO:teuthology.orchestra.run.smithi012.stdout:6.11.0-rc5-g2ca44a2f35ca 2024-09-10T23:47:33.201 DEBUG:teuthology.task.kernel:current kernel version is 6.11.0-rc5-g2ca44a2f35ca vs 6.11.0-rc5-g2ca44a2f35ca 2024-09-10T23:47:33.201 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-10T23:47:33.202 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-09-10T23:47:34.205 DEBUG:teuthology.parallel:result is None 2024-09-10T23:47:34.205 INFO:teuthology.run_tasks:Running task internal.base... 2024-09-10T23:47:34.212 INFO:teuthology.task.internal:Creating test directory... 2024-09-10T23:47:34.212 DEBUG:teuthology.orchestra.run.smithi012:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-10T23:47:34.217 DEBUG:teuthology.orchestra.run.smithi165:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-10T23:47:34.220 DEBUG:teuthology.orchestra.run.smithi178:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-10T23:47:34.249 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-09-10T23:47:34.256 INFO:teuthology.run_tasks:Running task internal.archive... 2024-09-10T23:47:34.265 INFO:teuthology.task.internal:Creating archive directory... 2024-09-10T23:47:34.265 DEBUG:teuthology.orchestra.run.smithi012:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-10T23:47:34.297 DEBUG:teuthology.orchestra.run.smithi165:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-10T23:47:34.300 DEBUG:teuthology.orchestra.run.smithi178:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-10T23:47:34.397 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-09-10T23:47:34.404 INFO:teuthology.task.internal:Enabling coredump saving... 2024-09-10T23:47:34.404 DEBUG:teuthology.orchestra.run.smithi165:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-10T23:47:34.421 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T23:47:34.422 DEBUG:teuthology.orchestra.run.smithi178:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-10T23:47:34.439 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T23:47:34.439 DEBUG:teuthology.orchestra.run.smithi012:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-10T23:47:34.443 DEBUG:teuthology.orchestra.run.smithi165:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-10T23:47:34.464 DEBUG:teuthology.orchestra.run.smithi178:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-10T23:47:34.498 INFO:teuthology.orchestra.run.smithi165.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-10T23:47:34.512 INFO:teuthology.orchestra.run.smithi165.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-10T23:47:34.533 INFO:teuthology.orchestra.run.smithi178.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-10T23:47:34.553 INFO:teuthology.orchestra.run.smithi178.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-10T23:47:34.604 INFO:teuthology.orchestra.run.smithi012.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-10T23:47:34.659 INFO:teuthology.orchestra.run.smithi012.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-10T23:47:34.662 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-09-10T23:47:34.669 INFO:teuthology.task.internal:Configuring sudo... 2024-09-10T23:47:34.669 DEBUG:teuthology.orchestra.run.smithi012:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-10T23:47:34.673 DEBUG:teuthology.orchestra.run.smithi165:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-10T23:47:34.675 DEBUG:teuthology.orchestra.run.smithi178:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-10T23:47:34.725 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-09-10T23:47:34.735 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-09-10T23:47:34.735 DEBUG:teuthology.orchestra.run.smithi012:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-10T23:47:34.740 DEBUG:teuthology.orchestra.run.smithi165:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-10T23:47:34.745 DEBUG:teuthology.orchestra.run.smithi178:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-10T23:47:34.771 DEBUG:teuthology.orchestra.run.smithi012:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-10T23:47:34.889 DEBUG:teuthology.orchestra.run.smithi012:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-10T23:47:35.055 DEBUG:teuthology.orchestra.run.smithi012:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-10T23:47:35.142 DEBUG:teuthology.orchestra.run.smithi012:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-10T23:47:35.200 DEBUG:teuthology.orchestra.run.smithi012:> set -ex 2024-09-10T23:47:35.201 DEBUG:teuthology.orchestra.run.smithi012:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-10T23:47:35.317 DEBUG:teuthology.orchestra.run.smithi165:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-10T23:47:35.373 DEBUG:teuthology.orchestra.run.smithi165:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-10T23:47:35.421 DEBUG:teuthology.orchestra.run.smithi165:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-10T23:47:35.516 DEBUG:teuthology.orchestra.run.smithi165:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-10T23:47:35.550 DEBUG:teuthology.orchestra.run.smithi165:> set -ex 2024-09-10T23:47:35.550 DEBUG:teuthology.orchestra.run.smithi165:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-10T23:47:35.617 DEBUG:teuthology.orchestra.run.smithi178:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-10T23:47:35.671 DEBUG:teuthology.orchestra.run.smithi178:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-10T23:47:35.722 DEBUG:teuthology.orchestra.run.smithi178:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-10T23:47:35.807 DEBUG:teuthology.orchestra.run.smithi178:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-10T23:47:35.852 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2024-09-10T23:47:35.853 DEBUG:teuthology.orchestra.run.smithi178:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-10T23:47:35.921 DEBUG:teuthology.orchestra.run.smithi012:> sudo service rsyslog restart 2024-09-10T23:47:35.925 DEBUG:teuthology.orchestra.run.smithi165:> sudo service rsyslog restart 2024-09-10T23:47:35.927 DEBUG:teuthology.orchestra.run.smithi178:> sudo service rsyslog restart 2024-09-10T23:47:35.965 INFO:teuthology.orchestra.run.smithi165.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-10T23:47:36.002 INFO:teuthology.orchestra.run.smithi178.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-10T23:47:36.018 INFO:teuthology.orchestra.run.smithi012.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-10T23:47:36.369 INFO:teuthology.run_tasks:Running task internal.timer... 2024-09-10T23:47:36.376 INFO:teuthology.task.internal:Starting timer... 2024-09-10T23:47:36.376 INFO:teuthology.run_tasks:Running task pcp... 2024-09-10T23:47:36.407 INFO:teuthology.run_tasks:Running task selinux... 2024-09-10T23:47:36.415 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:logrotate_t:s0']} 2024-09-10T23:47:36.415 DEBUG:teuthology.orchestra.run.smithi012:> sudo service auditd rotate 2024-09-10T23:47:36.529 INFO:teuthology.orchestra.run.smithi012.stdout:Rotating logs: 2024-09-10T23:47:36.532 DEBUG:teuthology.orchestra.run.smithi165:> sudo service auditd rotate 2024-09-10T23:47:36.596 INFO:teuthology.orchestra.run.smithi165.stdout:Rotating logs: 2024-09-10T23:47:36.598 DEBUG:teuthology.orchestra.run.smithi178:> sudo service auditd rotate 2024-09-10T23:47:36.646 INFO:teuthology.orchestra.run.smithi178.stdout:Rotating logs: 2024-09-10T23:47:36.648 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-09-10T23:47:36.648 DEBUG:teuthology.orchestra.run.smithi012:> /usr/sbin/getenforce 2024-09-10T23:47:36.687 INFO:teuthology.orchestra.run.smithi012.stdout:Permissive 2024-09-10T23:47:36.687 DEBUG:teuthology.orchestra.run.smithi165:> /usr/sbin/getenforce 2024-09-10T23:47:36.712 INFO:teuthology.orchestra.run.smithi165.stdout:Permissive 2024-09-10T23:47:36.713 DEBUG:teuthology.orchestra.run.smithi178:> /usr/sbin/getenforce 2024-09-10T23:47:36.743 INFO:teuthology.orchestra.run.smithi178.stdout:Permissive 2024-09-10T23:47:36.744 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi012.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi165.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi178.front.sepia.ceph.com': 'permissive'} 2024-09-10T23:47:36.744 DEBUG:teuthology.orchestra.run.smithi012:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 2024-09-10T23:47:36.802 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T23:47:36.803 DEBUG:teuthology.orchestra.run.smithi165:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 2024-09-10T23:47:36.833 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T23:47:36.833 DEBUG:teuthology.orchestra.run.smithi178:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 2024-09-10T23:47:36.862 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T23:47:36.862 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-09-10T23:47:36.862 DEBUG:teuthology.orchestra.run.smithi012:> sudo /usr/sbin/setenforce permissive 2024-09-10T23:47:36.927 DEBUG:teuthology.orchestra.run.smithi165:> sudo /usr/sbin/setenforce permissive 2024-09-10T23:47:36.976 DEBUG:teuthology.orchestra.run.smithi178:> sudo /usr/sbin/setenforce permissive 2024-09-10T23:47:37.007 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-09-10T23:47:37.017 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-09-10T23:47:37.029 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-09-10T23:47:37.042 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi012.front.sepia.ceph.com,smithi165.front.sepia.ceph.com,smithi178.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-09-10T23:52:15.249 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi012.front.sepia.ceph.com'), Remote(name='ubuntu@smithi165.front.sepia.ceph.com'), Remote(name='ubuntu@smithi178.front.sepia.ceph.com')] 2024-09-10T23:52:15.251 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi012.front.sepia.ceph.com' 2024-09-10T23:52:15.252 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi012.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T23:52:15.368 DEBUG:teuthology.orchestra.run.smithi012:> true 2024-09-10T23:52:15.476 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi012.front.sepia.ceph.com' 2024-09-10T23:52:15.476 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi165.front.sepia.ceph.com' 2024-09-10T23:52:15.477 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi165.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T23:52:15.559 DEBUG:teuthology.orchestra.run.smithi165:> true 2024-09-10T23:52:15.636 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi165.front.sepia.ceph.com' 2024-09-10T23:52:15.637 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi178.front.sepia.ceph.com' 2024-09-10T23:52:15.637 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi178.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T23:52:15.737 DEBUG:teuthology.orchestra.run.smithi178:> true 2024-09-10T23:52:15.816 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi178.front.sepia.ceph.com' 2024-09-10T23:52:15.817 INFO:teuthology.run_tasks:Running task clock... 2024-09-10T23:52:15.830 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-09-10T23:52:15.831 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-10T23:52:15.831 DEBUG:teuthology.orchestra.run.smithi012:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-10T23:52:15.835 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-10T23:52:15.835 DEBUG:teuthology.orchestra.run.smithi165:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-10T23:52:15.837 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-10T23:52:15.837 DEBUG:teuthology.orchestra.run.smithi178:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-10T23:52:15.865 INFO:teuthology.orchestra.run.smithi165.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-10T23:52:15.879 INFO:teuthology.orchestra.run.smithi165.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-10T23:52:15.887 INFO:teuthology.orchestra.run.smithi178.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-10T23:52:15.894 INFO:teuthology.orchestra.run.smithi012.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-10T23:52:15.902 INFO:teuthology.orchestra.run.smithi178.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-10T23:52:15.904 INFO:teuthology.orchestra.run.smithi165.stderr:sudo: ntpd: command not found 2024-09-10T23:52:15.915 INFO:teuthology.orchestra.run.smithi165.stdout:506 Cannot talk to daemon 2024-09-10T23:52:15.924 INFO:teuthology.orchestra.run.smithi012.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-10T23:52:15.927 INFO:teuthology.orchestra.run.smithi178.stderr:sudo: ntpd: command not found 2024-09-10T23:52:15.932 INFO:teuthology.orchestra.run.smithi165.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-10T23:52:15.938 INFO:teuthology.orchestra.run.smithi178.stdout:506 Cannot talk to daemon 2024-09-10T23:52:15.944 INFO:teuthology.orchestra.run.smithi165.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-10T23:52:15.952 INFO:teuthology.orchestra.run.smithi178.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-10T23:52:15.966 INFO:teuthology.orchestra.run.smithi178.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-10T23:52:15.975 INFO:teuthology.orchestra.run.smithi012.stderr:sudo: ntpd: command not found 2024-09-10T23:52:15.998 INFO:teuthology.orchestra.run.smithi012.stdout:506 Cannot talk to daemon 2024-09-10T23:52:16.010 INFO:teuthology.orchestra.run.smithi165.stderr:bash: line 1: ntpq: command not found 2024-09-10T23:52:16.014 INFO:teuthology.orchestra.run.smithi165.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-10T23:52:16.014 INFO:teuthology.orchestra.run.smithi165.stdout:=============================================================================== 2024-09-10T23:52:16.014 INFO:teuthology.orchestra.run.smithi165.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-10T23:52:16.014 INFO:teuthology.orchestra.run.smithi165.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-10T23:52:16.014 INFO:teuthology.orchestra.run.smithi165.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-10T23:52:16.014 INFO:teuthology.orchestra.run.smithi165.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-10T23:52:16.020 INFO:teuthology.orchestra.run.smithi178.stderr:bash: line 1: ntpq: command not found 2024-09-10T23:52:16.024 INFO:teuthology.orchestra.run.smithi178.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-10T23:52:16.024 INFO:teuthology.orchestra.run.smithi178.stdout:=============================================================================== 2024-09-10T23:52:16.024 INFO:teuthology.orchestra.run.smithi178.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-10T23:52:16.024 INFO:teuthology.orchestra.run.smithi178.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-10T23:52:16.024 INFO:teuthology.orchestra.run.smithi178.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-10T23:52:16.024 INFO:teuthology.orchestra.run.smithi178.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-10T23:52:16.028 INFO:teuthology.orchestra.run.smithi012.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-10T23:52:16.058 INFO:teuthology.orchestra.run.smithi012.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-10T23:52:16.155 INFO:teuthology.orchestra.run.smithi012.stderr:bash: line 1: ntpq: command not found 2024-09-10T23:52:16.162 INFO:teuthology.orchestra.run.smithi012.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-10T23:52:16.162 INFO:teuthology.orchestra.run.smithi012.stdout:=============================================================================== 2024-09-10T23:52:16.162 INFO:teuthology.orchestra.run.smithi012.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-10T23:52:16.162 INFO:teuthology.orchestra.run.smithi012.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-10T23:52:16.162 INFO:teuthology.orchestra.run.smithi012.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-10T23:52:16.162 INFO:teuthology.orchestra.run.smithi012.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-10T23:52:16.164 INFO:teuthology.run_tasks:Running task install... 2024-09-10T23:52:16.173 DEBUG:teuthology.task.install:project ceph 2024-09-10T23:52:16.173 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '8f219c476cb4f26fc5719c661c0323719a45cb30'}} 2024-09-10T23:52:16.174 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '8f219c476cb4f26fc5719c661c0323719a45cb30'} 2024-09-10T23:52:16.174 INFO:teuthology.task.install:Using flavor: default 2024-09-10T23:52:16.180 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-09-10T23:52:16.180 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-09-10T23:52:16.181 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '8f219c476cb4f26fc5719c661c0323719a45cb30', 'tag': None, 'wait_for_package': False} 2024-09-10T23:52:16.181 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=8f219c476cb4f26fc5719c661c0323719a45cb30 2024-09-10T23:52:16.183 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '8f219c476cb4f26fc5719c661c0323719a45cb30', 'tag': None, 'wait_for_package': False} 2024-09-10T23:52:16.183 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=8f219c476cb4f26fc5719c661c0323719a45cb30 2024-09-10T23:52:16.184 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '8f219c476cb4f26fc5719c661c0323719a45cb30', 'tag': None, 'wait_for_package': False} 2024-09-10T23:52:16.184 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=8f219c476cb4f26fc5719c661c0323719a45cb30 2024-09-10T23:52:16.342 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/centos/9/flavors/default/ 2024-09-10T23:52:16.343 INFO:teuthology.task.install.rpm:Package version is 19.3.0-4854.g8f219c47 2024-09-10T23:52:16.359 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/centos/9/flavors/default/ 2024-09-10T23:52:16.359 INFO:teuthology.task.install.rpm:Package version is 19.3.0-4854.g8f219c47 2024-09-10T23:52:16.360 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/centos/9/flavors/default/ 2024-09-10T23:52:16.361 INFO:teuthology.task.install.rpm:Package version is 19.3.0-4854.g8f219c47 2024-09-10T23:52:16.474 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-10T23:52:16.474 DEBUG:teuthology.orchestra.run.smithi012:> set -ex 2024-09-10T23:52:16.474 DEBUG:teuthology.orchestra.run.smithi012:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-10T23:52:16.481 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-10T23:52:16.481 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2024-09-10T23:52:16.481 DEBUG:teuthology.orchestra.run.smithi178:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-10T23:52:16.490 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-10T23:52:16.490 DEBUG:teuthology.orchestra.run.smithi165:> set -ex 2024-09-10T23:52:16.490 DEBUG:teuthology.orchestra.run.smithi165:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-10T23:52:16.508 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-09-10T23:52:16.508 DEBUG:teuthology.orchestra.run.smithi178:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/8f219c476cb4f26fc5719c661c0323719a45cb30/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-10T23:52:16.518 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-09-10T23:52:16.518 DEBUG:teuthology.orchestra.run.smithi165:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/8f219c476cb4f26fc5719c661c0323719a45cb30/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-10T23:52:16.532 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-09-10T23:52:16.532 DEBUG:teuthology.orchestra.run.smithi012:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/8f219c476cb4f26fc5719c661c0323719a45cb30/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-10T23:52:16.575 DEBUG:teuthology.orchestra.run.smithi178:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-10T23:52:16.585 DEBUG:teuthology.orchestra.run.smithi165:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-10T23:52:16.629 DEBUG:teuthology.orchestra.run.smithi012:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-10T23:52:16.651 DEBUG:teuthology.orchestra.run.smithi178:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-10T23:52:16.661 DEBUG:teuthology.orchestra.run.smithi165:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-10T23:52:16.708 DEBUG:teuthology.orchestra.run.smithi012:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-10T23:52:16.715 INFO:teuthology.orchestra.run.smithi178.stdout:check_obsoletes = 1 2024-09-10T23:52:16.716 DEBUG:teuthology.orchestra.run.smithi178:> sudo yum clean all 2024-09-10T23:52:16.725 INFO:teuthology.orchestra.run.smithi165.stdout:check_obsoletes = 1 2024-09-10T23:52:16.727 DEBUG:teuthology.orchestra.run.smithi165:> sudo yum clean all 2024-09-10T23:52:16.764 INFO:teuthology.orchestra.run.smithi012.stdout:check_obsoletes = 1 2024-09-10T23:52:16.767 DEBUG:teuthology.orchestra.run.smithi012:> sudo yum clean all 2024-09-10T23:52:16.940 INFO:teuthology.orchestra.run.smithi178.stdout:45 files removed 2024-09-10T23:52:16.953 INFO:teuthology.orchestra.run.smithi165.stdout:45 files removed 2024-09-10T23:52:16.967 DEBUG:teuthology.orchestra.run.smithi178:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-09-10T23:52:16.979 DEBUG:teuthology.orchestra.run.smithi165:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-09-10T23:52:17.049 INFO:teuthology.orchestra.run.smithi012.stdout:45 files removed 2024-09-10T23:52:17.083 DEBUG:teuthology.orchestra.run.smithi012:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-09-10T23:52:17.516 INFO:teuthology.orchestra.run.smithi178.stdout:ceph packages for x86_64 309 kB/s | 84 kB 00:00 2024-09-10T23:52:17.534 INFO:teuthology.orchestra.run.smithi165.stdout:ceph packages for x86_64 306 kB/s | 84 kB 00:00 2024-09-10T23:52:17.765 INFO:teuthology.orchestra.run.smithi012.stdout:ceph packages for x86_64 306 kB/s | 84 kB 00:00 2024-09-10T23:52:17.808 INFO:teuthology.orchestra.run.smithi178.stdout:ceph noarch packages 97 kB/s | 24 kB 00:00 2024-09-10T23:52:17.818 INFO:teuthology.orchestra.run.smithi165.stdout:ceph noarch packages 100 kB/s | 24 kB 00:00 2024-09-10T23:52:18.057 INFO:teuthology.orchestra.run.smithi012.stdout:ceph noarch packages 99 kB/s | 24 kB 00:00 2024-09-10T23:52:18.084 INFO:teuthology.orchestra.run.smithi178.stdout:ceph source packages 8.1 kB/s | 1.9 kB 00:00 2024-09-10T23:52:18.102 INFO:teuthology.orchestra.run.smithi165.stdout:ceph source packages 7.8 kB/s | 1.9 kB 00:00 2024-09-10T23:52:18.341 INFO:teuthology.orchestra.run.smithi012.stdout:ceph source packages 8.0 kB/s | 1.9 kB 00:00 2024-09-10T23:52:20.383 INFO:teuthology.orchestra.run.smithi012.stdout:CentOS Stream 9 - BaseOS 4.1 MB/s | 8.2 MB 00:02 2024-09-10T23:52:20.385 INFO:teuthology.orchestra.run.smithi165.stdout:CentOS Stream 9 - BaseOS 3.7 MB/s | 8.2 MB 00:02 2024-09-10T23:52:20.917 INFO:teuthology.orchestra.run.smithi178.stdout:CentOS Stream 9 - BaseOS 2.9 MB/s | 8.2 MB 00:02 2024-09-10T23:52:22.153 INFO:teuthology.orchestra.run.smithi165.stdout:CentOS Stream 9 - AppStream 30 MB/s | 20 MB 00:00 2024-09-10T23:52:24.385 INFO:teuthology.orchestra.run.smithi178.stdout:CentOS Stream 9 - AppStream 8.6 MB/s | 20 MB 00:02 2024-09-10T23:52:24.536 INFO:teuthology.orchestra.run.smithi012.stdout:CentOS Stream 9 - AppStream 6.9 MB/s | 20 MB 00:02 2024-09-10T23:52:27.479 INFO:teuthology.orchestra.run.smithi165.stdout:CentOS Stream 9 - CRB 5.4 MB/s | 6.5 MB 00:01 2024-09-10T23:52:28.635 INFO:teuthology.orchestra.run.smithi178.stdout:CentOS Stream 9 - CRB 14 MB/s | 6.5 MB 00:00 2024-09-10T23:52:28.979 INFO:teuthology.orchestra.run.smithi165.stdout:CentOS Stream 9 - Extras packages 71 kB/s | 19 kB 00:00 2024-09-10T23:52:29.967 INFO:teuthology.orchestra.run.smithi165.stdout:Extra Packages for Enterprise Linux 26 MB/s | 23 MB 00:00 2024-09-10T23:52:30.136 INFO:teuthology.orchestra.run.smithi178.stdout:CentOS Stream 9 - Extras packages 62 kB/s | 19 kB 00:00 2024-09-10T23:52:31.140 INFO:teuthology.orchestra.run.smithi178.stdout:Extra Packages for Enterprise Linux 26 MB/s | 23 MB 00:00 2024-09-10T23:52:35.431 INFO:teuthology.orchestra.run.smithi165.stdout:lab-extras 44 kB/s | 1.7 kB 00:00 2024-09-10T23:52:36.613 INFO:teuthology.orchestra.run.smithi178.stdout:lab-extras 35 kB/s | 1.7 kB 00:00 2024-09-10T23:52:37.123 INFO:teuthology.orchestra.run.smithi012.stdout:CentOS Stream 9 - CRB 793 kB/s | 6.5 MB 00:08 2024-09-10T23:52:37.707 INFO:teuthology.orchestra.run.smithi165.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-10T23:52:37.708 INFO:teuthology.orchestra.run.smithi165.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-10T23:52:37.719 INFO:teuthology.orchestra.run.smithi165.stdout:Package NetworkManager-1:1.48.8-1.el9.x86_64 is already installed. 2024-09-10T23:52:37.720 INFO:teuthology.orchestra.run.smithi165.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-09-10T23:52:37.721 INFO:teuthology.orchestra.run.smithi165.stdout:Package util-linux-2.37.4-20.el9.x86_64 is already installed. 2024-09-10T23:52:37.721 INFO:teuthology.orchestra.run.smithi165.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-09-10T23:52:37.723 INFO:teuthology.orchestra.run.smithi165.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-09-10T23:52:37.725 INFO:teuthology.orchestra.run.smithi165.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-09-10T23:52:37.726 INFO:teuthology.orchestra.run.smithi165.stdout:Package xfsprogs-6.4.0-4.el9.x86_64 is already installed. 2024-09-10T23:52:37.789 INFO:teuthology.orchestra.run.smithi165.stdout:Dependencies resolved. 2024-09-10T23:52:37.809 INFO:teuthology.orchestra.run.smithi165.stdout:======================================================================================= 2024-09-10T23:52:37.809 INFO:teuthology.orchestra.run.smithi165.stdout: Package Arch Version Repository Size 2024-09-10T23:52:37.809 INFO:teuthology.orchestra.run.smithi165.stdout:======================================================================================= 2024-09-10T23:52:37.809 INFO:teuthology.orchestra.run.smithi165.stdout:Installing: 2024-09-10T23:52:37.810 INFO:teuthology.orchestra.run.smithi165.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-09-10T23:52:37.810 INFO:teuthology.orchestra.run.smithi165.stdout: ceph x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 6.6 k 2024-09-10T23:52:37.810 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-base x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 5.5 M 2024-09-10T23:52:37.810 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-fuse x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 900 k 2024-09-10T23:52:37.810 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-immutable-object-cache x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 145 k 2024-09-10T23:52:37.810 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-mgr x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 1.5 M 2024-09-10T23:52:37.810 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-mgr-cephadm noarch 2:19.3.0-4854.g8f219c47.el9 ceph-noarch 154 k 2024-09-10T23:52:37.810 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-mgr-dashboard noarch 2:19.3.0-4854.g8f219c47.el9 ceph-noarch 14 M 2024-09-10T23:52:37.810 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-mgr-diskprediction-local noarch 2:19.3.0-4854.g8f219c47.el9 ceph-noarch 7.4 M 2024-09-10T23:52:37.810 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-mgr-rook noarch 2:19.3.0-4854.g8f219c47.el9 ceph-noarch 49 k 2024-09-10T23:52:37.810 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-radosgw x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 11 M 2024-09-10T23:52:37.810 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-test x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 50 M 2024-09-10T23:52:37.810 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-volume noarch 2:19.3.0-4854.g8f219c47.el9 ceph-noarch 285 k 2024-09-10T23:52:37.811 INFO:teuthology.orchestra.run.smithi165.stdout: cephadm noarch 2:19.3.0-4854.g8f219c47.el9 ceph-noarch 783 k 2024-09-10T23:52:37.811 INFO:teuthology.orchestra.run.smithi165.stdout: cephfs-mirror x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 225 k 2024-09-10T23:52:37.811 INFO:teuthology.orchestra.run.smithi165.stdout: cephfs-shell x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 27 k 2024-09-10T23:52:37.811 INFO:teuthology.orchestra.run.smithi165.stdout: cephfs-top noarch 2:19.3.0-4854.g8f219c47.el9 ceph-noarch 18 k 2024-09-10T23:52:37.811 INFO:teuthology.orchestra.run.smithi165.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-09-10T23:52:37.811 INFO:teuthology.orchestra.run.smithi165.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-09-10T23:52:37.811 INFO:teuthology.orchestra.run.smithi165.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-09-10T23:52:37.811 INFO:teuthology.orchestra.run.smithi165.stdout: libcephfs-devel x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 32 k 2024-09-10T23:52:37.811 INFO:teuthology.orchestra.run.smithi165.stdout: libcephfs2 x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 755 k 2024-09-10T23:52:37.811 INFO:teuthology.orchestra.run.smithi165.stdout: librados-devel x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 127 k 2024-09-10T23:52:37.811 INFO:teuthology.orchestra.run.smithi165.stdout: libtool x86_64 2.4.6-46.el9 appstream 581 k 2024-09-10T23:52:37.811 INFO:teuthology.orchestra.run.smithi165.stdout: libuuid-devel x86_64 2.37.4-20.el9 appstream 24 k 2024-09-10T23:52:37.812 INFO:teuthology.orchestra.run.smithi165.stdout: openssl-devel x86_64 1:3.2.2-6.el9 appstream 4.4 M 2024-09-10T23:52:37.812 INFO:teuthology.orchestra.run.smithi165.stdout: postgresql x86_64 13.16-1.el9 appstream 1.6 M 2024-09-10T23:52:37.812 INFO:teuthology.orchestra.run.smithi165.stdout: postgresql-contrib x86_64 13.16-1.el9 appstream 910 k 2024-09-10T23:52:37.812 INFO:teuthology.orchestra.run.smithi165.stdout: postgresql-server x86_64 13.16-1.el9 appstream 5.8 M 2024-09-10T23:52:37.812 INFO:teuthology.orchestra.run.smithi165.stdout: python3-cephfs x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 161 k 2024-09-10T23:52:37.812 INFO:teuthology.orchestra.run.smithi165.stdout: python3-rados x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 321 k 2024-09-10T23:52:37.812 INFO:teuthology.orchestra.run.smithi165.stdout: python3-rbd x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 306 k 2024-09-10T23:52:37.812 INFO:teuthology.orchestra.run.smithi165.stdout: python3-rgw x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 99 k 2024-09-10T23:52:37.812 INFO:teuthology.orchestra.run.smithi165.stdout: rbd-fuse x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 85 k 2024-09-10T23:52:37.812 INFO:teuthology.orchestra.run.smithi165.stdout: rbd-mirror x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 3.2 M 2024-09-10T23:52:37.812 INFO:teuthology.orchestra.run.smithi165.stdout: rbd-nbd x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 171 k 2024-09-10T23:52:37.812 INFO:teuthology.orchestra.run.smithi165.stdout: xfsprogs-devel x86_64 6.4.0-4.el9 appstream 96 k 2024-09-10T23:52:37.813 INFO:teuthology.orchestra.run.smithi165.stdout:Upgrading: 2024-09-10T23:52:37.813 INFO:teuthology.orchestra.run.smithi165.stdout: NetworkManager x86_64 1:1.49.90-1.el9 baseos 2.3 M 2024-09-10T23:52:37.813 INFO:teuthology.orchestra.run.smithi165.stdout: NetworkManager-libnm x86_64 1:1.49.90-1.el9 baseos 1.8 M 2024-09-10T23:52:37.813 INFO:teuthology.orchestra.run.smithi165.stdout: NetworkManager-team x86_64 1:1.49.90-1.el9 baseos 41 k 2024-09-10T23:52:37.813 INFO:teuthology.orchestra.run.smithi165.stdout: NetworkManager-tui x86_64 1:1.49.90-1.el9 baseos 249 k 2024-09-10T23:52:37.813 INFO:teuthology.orchestra.run.smithi165.stdout: librados2 x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 3.5 M 2024-09-10T23:52:37.813 INFO:teuthology.orchestra.run.smithi165.stdout: librbd1 x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 3.2 M 2024-09-10T23:52:37.813 INFO:teuthology.orchestra.run.smithi165.stdout: openssl x86_64 1:3.2.2-6.el9 baseos 1.4 M 2024-09-10T23:52:37.813 INFO:teuthology.orchestra.run.smithi165.stdout: openssl-libs x86_64 1:3.2.2-6.el9 baseos 2.4 M 2024-09-10T23:52:37.813 INFO:teuthology.orchestra.run.smithi165.stdout:Installing dependencies: 2024-09-10T23:52:37.813 INFO:teuthology.orchestra.run.smithi165.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-10T23:52:37.813 INFO:teuthology.orchestra.run.smithi165.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-09-10T23:52:37.814 INFO:teuthology.orchestra.run.smithi165.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-10T23:52:37.814 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-common x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 22 M 2024-09-10T23:52:37.814 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-grafana-dashboards noarch 2:19.3.0-4854.g8f219c47.el9 ceph-noarch 34 k 2024-09-10T23:52:37.814 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-mds x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 2.4 M 2024-09-10T23:52:37.814 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-mgr-modules-core noarch 2:19.3.0-4854.g8f219c47.el9 ceph-noarch 291 k 2024-09-10T23:52:37.814 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-mon x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 4.8 M 2024-09-10T23:52:37.814 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-osd x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 17 M 2024-09-10T23:52:37.814 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-prometheus-alerts noarch 2:19.3.0-4854.g8f219c47.el9 ceph-noarch 16 k 2024-09-10T23:52:37.814 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-selinux x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 25 k 2024-09-10T23:52:37.814 INFO:teuthology.orchestra.run.smithi165.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-10T23:52:37.814 INFO:teuthology.orchestra.run.smithi165.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-10T23:52:37.814 INFO:teuthology.orchestra.run.smithi165.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-10T23:52:37.815 INFO:teuthology.orchestra.run.smithi165.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-10T23:52:37.815 INFO:teuthology.orchestra.run.smithi165.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-09-10T23:52:37.815 INFO:teuthology.orchestra.run.smithi165.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-10T23:52:37.815 INFO:teuthology.orchestra.run.smithi165.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-10T23:52:37.815 INFO:teuthology.orchestra.run.smithi165.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-10T23:52:37.815 INFO:teuthology.orchestra.run.smithi165.stdout: libcephsqlite x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 162 k 2024-09-10T23:52:37.815 INFO:teuthology.orchestra.run.smithi165.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-10T23:52:37.815 INFO:teuthology.orchestra.run.smithi165.stdout: libnbd x86_64 1.20.2-2.el9 appstream 171 k 2024-09-10T23:52:37.815 INFO:teuthology.orchestra.run.smithi165.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-10T23:52:37.815 INFO:teuthology.orchestra.run.smithi165.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-10T23:52:37.815 INFO:teuthology.orchestra.run.smithi165.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-10T23:52:37.816 INFO:teuthology.orchestra.run.smithi165.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-10T23:52:37.816 INFO:teuthology.orchestra.run.smithi165.stdout: libradosstriper1 x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 507 k 2024-09-10T23:52:37.816 INFO:teuthology.orchestra.run.smithi165.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-10T23:52:37.816 INFO:teuthology.orchestra.run.smithi165.stdout: librgw2 x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 5.5 M 2024-09-10T23:52:37.816 INFO:teuthology.orchestra.run.smithi165.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-10T23:52:37.816 INFO:teuthology.orchestra.run.smithi165.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-09-10T23:52:37.816 INFO:teuthology.orchestra.run.smithi165.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-10T23:52:37.816 INFO:teuthology.orchestra.run.smithi165.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-10T23:52:37.816 INFO:teuthology.orchestra.run.smithi165.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-10T23:52:37.816 INFO:teuthology.orchestra.run.smithi165.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-10T23:52:37.816 INFO:teuthology.orchestra.run.smithi165.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-10T23:52:37.816 INFO:teuthology.orchestra.run.smithi165.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-10T23:52:37.817 INFO:teuthology.orchestra.run.smithi165.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-10T23:52:37.817 INFO:teuthology.orchestra.run.smithi165.stdout: postgresql-private-libs x86_64 13.16-1.el9 appstream 137 k 2024-09-10T23:52:37.817 INFO:teuthology.orchestra.run.smithi165.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-10T23:52:37.817 INFO:teuthology.orchestra.run.smithi165.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-10T23:52:37.817 INFO:teuthology.orchestra.run.smithi165.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-10T23:52:37.817 INFO:teuthology.orchestra.run.smithi165.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-09-10T23:52:37.817 INFO:teuthology.orchestra.run.smithi165.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-10T23:52:37.817 INFO:teuthology.orchestra.run.smithi165.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-10T23:52:37.817 INFO:teuthology.orchestra.run.smithi165.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-10T23:52:37.817 INFO:teuthology.orchestra.run.smithi165.stdout: python3-ceph-argparse x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 45 k 2024-09-10T23:52:37.817 INFO:teuthology.orchestra.run.smithi165.stdout: python3-ceph-common x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 143 k 2024-09-10T23:52:37.817 INFO:teuthology.orchestra.run.smithi165.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-10T23:52:37.818 INFO:teuthology.orchestra.run.smithi165.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-10T23:52:37.818 INFO:teuthology.orchestra.run.smithi165.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-10T23:52:37.818 INFO:teuthology.orchestra.run.smithi165.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-10T23:52:37.818 INFO:teuthology.orchestra.run.smithi165.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-09-10T23:52:37.818 INFO:teuthology.orchestra.run.smithi165.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-09-10T23:52:37.818 INFO:teuthology.orchestra.run.smithi165.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-10T23:52:37.818 INFO:teuthology.orchestra.run.smithi165.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-10T23:52:37.818 INFO:teuthology.orchestra.run.smithi165.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-10T23:52:37.818 INFO:teuthology.orchestra.run.smithi165.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-10T23:52:37.818 INFO:teuthology.orchestra.run.smithi165.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-10T23:52:37.818 INFO:teuthology.orchestra.run.smithi165.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-10T23:52:37.818 INFO:teuthology.orchestra.run.smithi165.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-10T23:52:37.818 INFO:teuthology.orchestra.run.smithi165.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-10T23:52:37.819 INFO:teuthology.orchestra.run.smithi165.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-10T23:52:37.819 INFO:teuthology.orchestra.run.smithi165.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-10T23:52:37.819 INFO:teuthology.orchestra.run.smithi165.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-10T23:52:37.819 INFO:teuthology.orchestra.run.smithi165.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-10T23:52:37.819 INFO:teuthology.orchestra.run.smithi165.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-10T23:52:37.819 INFO:teuthology.orchestra.run.smithi165.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-10T23:52:37.819 INFO:teuthology.orchestra.run.smithi165.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-10T23:52:37.819 INFO:teuthology.orchestra.run.smithi165.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-10T23:52:37.819 INFO:teuthology.orchestra.run.smithi165.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-10T23:52:37.819 INFO:teuthology.orchestra.run.smithi165.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-10T23:52:37.819 INFO:teuthology.orchestra.run.smithi165.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-10T23:52:37.819 INFO:teuthology.orchestra.run.smithi165.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-10T23:52:37.820 INFO:teuthology.orchestra.run.smithi165.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-10T23:52:37.820 INFO:teuthology.orchestra.run.smithi165.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-10T23:52:37.820 INFO:teuthology.orchestra.run.smithi165.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-10T23:52:37.820 INFO:teuthology.orchestra.run.smithi165.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-10T23:52:37.820 INFO:teuthology.orchestra.run.smithi165.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-10T23:52:37.820 INFO:teuthology.orchestra.run.smithi165.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-10T23:52:37.820 INFO:teuthology.orchestra.run.smithi165.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-10T23:52:37.820 INFO:teuthology.orchestra.run.smithi165.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-10T23:52:37.820 INFO:teuthology.orchestra.run.smithi165.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-10T23:52:37.820 INFO:teuthology.orchestra.run.smithi165.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-10T23:52:37.820 INFO:teuthology.orchestra.run.smithi165.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-10T23:52:37.820 INFO:teuthology.orchestra.run.smithi165.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-10T23:52:37.820 INFO:teuthology.orchestra.run.smithi165.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-09-10T23:52:37.821 INFO:teuthology.orchestra.run.smithi165.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-10T23:52:37.821 INFO:teuthology.orchestra.run.smithi165.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-10T23:52:37.821 INFO:teuthology.orchestra.run.smithi165.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-10T23:52:37.821 INFO:teuthology.orchestra.run.smithi165.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-10T23:52:37.821 INFO:teuthology.orchestra.run.smithi165.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-10T23:52:37.821 INFO:teuthology.orchestra.run.smithi165.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-10T23:52:37.821 INFO:teuthology.orchestra.run.smithi165.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-10T23:52:37.821 INFO:teuthology.orchestra.run.smithi165.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-10T23:52:37.821 INFO:teuthology.orchestra.run.smithi165.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-10T23:52:37.821 INFO:teuthology.orchestra.run.smithi165.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-10T23:52:37.821 INFO:teuthology.orchestra.run.smithi165.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-10T23:52:37.821 INFO:teuthology.orchestra.run.smithi165.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-09-10T23:52:37.822 INFO:teuthology.orchestra.run.smithi165.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-10T23:52:37.822 INFO:teuthology.orchestra.run.smithi165.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-10T23:52:37.822 INFO:teuthology.orchestra.run.smithi165.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-10T23:52:37.822 INFO:teuthology.orchestra.run.smithi165.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-10T23:52:37.822 INFO:teuthology.orchestra.run.smithi165.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-10T23:52:37.822 INFO:teuthology.orchestra.run.smithi165.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-10T23:52:37.822 INFO:teuthology.orchestra.run.smithi165.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-10T23:52:37.822 INFO:teuthology.orchestra.run.smithi165.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-10T23:52:37.822 INFO:teuthology.orchestra.run.smithi165.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-10T23:52:37.822 INFO:teuthology.orchestra.run.smithi165.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-09-10T23:52:37.822 INFO:teuthology.orchestra.run.smithi165.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-10T23:52:37.822 INFO:teuthology.orchestra.run.smithi165.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-09-10T23:52:37.822 INFO:teuthology.orchestra.run.smithi165.stdout:Installing weak dependencies: 2024-09-10T23:52:37.823 INFO:teuthology.orchestra.run.smithi165.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-10T23:52:37.823 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-09-10T23:52:37.823 INFO:teuthology.orchestra.run.smithi165.stdout:Transaction Summary 2024-09-10T23:52:37.823 INFO:teuthology.orchestra.run.smithi165.stdout:======================================================================================= 2024-09-10T23:52:37.823 INFO:teuthology.orchestra.run.smithi165.stdout:Install 149 Packages 2024-09-10T23:52:37.823 INFO:teuthology.orchestra.run.smithi165.stdout:Upgrade 8 Packages 2024-09-10T23:52:37.823 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-09-10T23:52:37.824 INFO:teuthology.orchestra.run.smithi165.stdout:Total download size: 240 M 2024-09-10T23:52:37.824 INFO:teuthology.orchestra.run.smithi165.stdout:Downloading Packages: 2024-09-10T23:52:38.898 INFO:teuthology.orchestra.run.smithi178.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-10T23:52:38.898 INFO:teuthology.orchestra.run.smithi178.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-10T23:52:38.910 INFO:teuthology.orchestra.run.smithi178.stdout:Package NetworkManager-1:1.48.8-1.el9.x86_64 is already installed. 2024-09-10T23:52:38.910 INFO:teuthology.orchestra.run.smithi178.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-09-10T23:52:38.911 INFO:teuthology.orchestra.run.smithi178.stdout:Package util-linux-2.37.4-20.el9.x86_64 is already installed. 2024-09-10T23:52:38.912 INFO:teuthology.orchestra.run.smithi178.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-09-10T23:52:38.914 INFO:teuthology.orchestra.run.smithi178.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-09-10T23:52:38.916 INFO:teuthology.orchestra.run.smithi178.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-09-10T23:52:38.917 INFO:teuthology.orchestra.run.smithi178.stdout:Package xfsprogs-6.4.0-4.el9.x86_64 is already installed. 2024-09-10T23:52:38.979 INFO:teuthology.orchestra.run.smithi178.stdout:Dependencies resolved. 2024-09-10T23:52:38.998 INFO:teuthology.orchestra.run.smithi178.stdout:======================================================================================= 2024-09-10T23:52:38.998 INFO:teuthology.orchestra.run.smithi178.stdout: Package Arch Version Repository Size 2024-09-10T23:52:38.998 INFO:teuthology.orchestra.run.smithi178.stdout:======================================================================================= 2024-09-10T23:52:38.999 INFO:teuthology.orchestra.run.smithi178.stdout:Installing: 2024-09-10T23:52:38.999 INFO:teuthology.orchestra.run.smithi178.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-09-10T23:52:38.999 INFO:teuthology.orchestra.run.smithi178.stdout: ceph x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 6.6 k 2024-09-10T23:52:38.999 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-base x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 5.5 M 2024-09-10T23:52:38.999 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-fuse x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 900 k 2024-09-10T23:52:38.999 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-immutable-object-cache x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 145 k 2024-09-10T23:52:38.999 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-mgr x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 1.5 M 2024-09-10T23:52:38.999 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-mgr-cephadm noarch 2:19.3.0-4854.g8f219c47.el9 ceph-noarch 154 k 2024-09-10T23:52:38.999 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-mgr-dashboard noarch 2:19.3.0-4854.g8f219c47.el9 ceph-noarch 14 M 2024-09-10T23:52:38.999 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-mgr-diskprediction-local noarch 2:19.3.0-4854.g8f219c47.el9 ceph-noarch 7.4 M 2024-09-10T23:52:38.999 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-mgr-rook noarch 2:19.3.0-4854.g8f219c47.el9 ceph-noarch 49 k 2024-09-10T23:52:38.999 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-radosgw x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 11 M 2024-09-10T23:52:38.999 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-test x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 50 M 2024-09-10T23:52:38.999 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-volume noarch 2:19.3.0-4854.g8f219c47.el9 ceph-noarch 285 k 2024-09-10T23:52:39.000 INFO:teuthology.orchestra.run.smithi178.stdout: cephadm noarch 2:19.3.0-4854.g8f219c47.el9 ceph-noarch 783 k 2024-09-10T23:52:39.000 INFO:teuthology.orchestra.run.smithi178.stdout: cephfs-mirror x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 225 k 2024-09-10T23:52:39.000 INFO:teuthology.orchestra.run.smithi178.stdout: cephfs-shell x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 27 k 2024-09-10T23:52:39.000 INFO:teuthology.orchestra.run.smithi178.stdout: cephfs-top noarch 2:19.3.0-4854.g8f219c47.el9 ceph-noarch 18 k 2024-09-10T23:52:39.000 INFO:teuthology.orchestra.run.smithi178.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-09-10T23:52:39.000 INFO:teuthology.orchestra.run.smithi178.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-09-10T23:52:39.000 INFO:teuthology.orchestra.run.smithi178.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-09-10T23:52:39.000 INFO:teuthology.orchestra.run.smithi178.stdout: libcephfs-devel x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 32 k 2024-09-10T23:52:39.000 INFO:teuthology.orchestra.run.smithi178.stdout: libcephfs2 x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 755 k 2024-09-10T23:52:39.000 INFO:teuthology.orchestra.run.smithi178.stdout: librados-devel x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 127 k 2024-09-10T23:52:39.000 INFO:teuthology.orchestra.run.smithi178.stdout: libtool x86_64 2.4.6-46.el9 appstream 581 k 2024-09-10T23:52:39.000 INFO:teuthology.orchestra.run.smithi178.stdout: libuuid-devel x86_64 2.37.4-20.el9 appstream 24 k 2024-09-10T23:52:39.001 INFO:teuthology.orchestra.run.smithi178.stdout: openssl-devel x86_64 1:3.2.2-6.el9 appstream 4.4 M 2024-09-10T23:52:39.001 INFO:teuthology.orchestra.run.smithi178.stdout: postgresql x86_64 13.16-1.el9 appstream 1.6 M 2024-09-10T23:52:39.001 INFO:teuthology.orchestra.run.smithi178.stdout: postgresql-contrib x86_64 13.16-1.el9 appstream 910 k 2024-09-10T23:52:39.001 INFO:teuthology.orchestra.run.smithi178.stdout: postgresql-server x86_64 13.16-1.el9 appstream 5.8 M 2024-09-10T23:52:39.001 INFO:teuthology.orchestra.run.smithi178.stdout: python3-cephfs x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 161 k 2024-09-10T23:52:39.001 INFO:teuthology.orchestra.run.smithi178.stdout: python3-rados x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 321 k 2024-09-10T23:52:39.001 INFO:teuthology.orchestra.run.smithi178.stdout: python3-rbd x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 306 k 2024-09-10T23:52:39.001 INFO:teuthology.orchestra.run.smithi178.stdout: python3-rgw x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 99 k 2024-09-10T23:52:39.001 INFO:teuthology.orchestra.run.smithi178.stdout: rbd-fuse x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 85 k 2024-09-10T23:52:39.001 INFO:teuthology.orchestra.run.smithi178.stdout: rbd-mirror x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 3.2 M 2024-09-10T23:52:39.001 INFO:teuthology.orchestra.run.smithi178.stdout: rbd-nbd x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 171 k 2024-09-10T23:52:39.001 INFO:teuthology.orchestra.run.smithi178.stdout: xfsprogs-devel x86_64 6.4.0-4.el9 appstream 96 k 2024-09-10T23:52:39.001 INFO:teuthology.orchestra.run.smithi178.stdout:Upgrading: 2024-09-10T23:52:39.002 INFO:teuthology.orchestra.run.smithi178.stdout: NetworkManager x86_64 1:1.49.90-1.el9 baseos 2.3 M 2024-09-10T23:52:39.002 INFO:teuthology.orchestra.run.smithi178.stdout: NetworkManager-libnm x86_64 1:1.49.90-1.el9 baseos 1.8 M 2024-09-10T23:52:39.002 INFO:teuthology.orchestra.run.smithi178.stdout: NetworkManager-team x86_64 1:1.49.90-1.el9 baseos 41 k 2024-09-10T23:52:39.002 INFO:teuthology.orchestra.run.smithi178.stdout: NetworkManager-tui x86_64 1:1.49.90-1.el9 baseos 249 k 2024-09-10T23:52:39.002 INFO:teuthology.orchestra.run.smithi178.stdout: librados2 x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 3.5 M 2024-09-10T23:52:39.002 INFO:teuthology.orchestra.run.smithi178.stdout: librbd1 x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 3.2 M 2024-09-10T23:52:39.002 INFO:teuthology.orchestra.run.smithi178.stdout: openssl x86_64 1:3.2.2-6.el9 baseos 1.4 M 2024-09-10T23:52:39.002 INFO:teuthology.orchestra.run.smithi178.stdout: openssl-libs x86_64 1:3.2.2-6.el9 baseos 2.4 M 2024-09-10T23:52:39.002 INFO:teuthology.orchestra.run.smithi178.stdout:Installing dependencies: 2024-09-10T23:52:39.002 INFO:teuthology.orchestra.run.smithi178.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-10T23:52:39.002 INFO:teuthology.orchestra.run.smithi178.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-09-10T23:52:39.002 INFO:teuthology.orchestra.run.smithi178.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-10T23:52:39.002 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-common x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 22 M 2024-09-10T23:52:39.003 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-grafana-dashboards noarch 2:19.3.0-4854.g8f219c47.el9 ceph-noarch 34 k 2024-09-10T23:52:39.003 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-mds x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 2.4 M 2024-09-10T23:52:39.003 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-mgr-modules-core noarch 2:19.3.0-4854.g8f219c47.el9 ceph-noarch 291 k 2024-09-10T23:52:39.003 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-mon x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 4.8 M 2024-09-10T23:52:39.003 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-osd x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 17 M 2024-09-10T23:52:39.003 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-prometheus-alerts noarch 2:19.3.0-4854.g8f219c47.el9 ceph-noarch 16 k 2024-09-10T23:52:39.003 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-selinux x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 25 k 2024-09-10T23:52:39.003 INFO:teuthology.orchestra.run.smithi178.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-10T23:52:39.003 INFO:teuthology.orchestra.run.smithi178.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-10T23:52:39.003 INFO:teuthology.orchestra.run.smithi178.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-10T23:52:39.003 INFO:teuthology.orchestra.run.smithi178.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-10T23:52:39.003 INFO:teuthology.orchestra.run.smithi178.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-09-10T23:52:39.004 INFO:teuthology.orchestra.run.smithi178.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-10T23:52:39.004 INFO:teuthology.orchestra.run.smithi178.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-10T23:52:39.004 INFO:teuthology.orchestra.run.smithi178.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-10T23:52:39.004 INFO:teuthology.orchestra.run.smithi178.stdout: libcephsqlite x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 162 k 2024-09-10T23:52:39.004 INFO:teuthology.orchestra.run.smithi178.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-10T23:52:39.004 INFO:teuthology.orchestra.run.smithi178.stdout: libnbd x86_64 1.20.2-2.el9 appstream 171 k 2024-09-10T23:52:39.004 INFO:teuthology.orchestra.run.smithi178.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-10T23:52:39.004 INFO:teuthology.orchestra.run.smithi178.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-10T23:52:39.004 INFO:teuthology.orchestra.run.smithi178.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-10T23:52:39.004 INFO:teuthology.orchestra.run.smithi178.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-10T23:52:39.004 INFO:teuthology.orchestra.run.smithi178.stdout: libradosstriper1 x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 507 k 2024-09-10T23:52:39.004 INFO:teuthology.orchestra.run.smithi178.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-10T23:52:39.004 INFO:teuthology.orchestra.run.smithi178.stdout: librgw2 x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 5.5 M 2024-09-10T23:52:39.004 INFO:teuthology.orchestra.run.smithi178.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-10T23:52:39.005 INFO:teuthology.orchestra.run.smithi178.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-09-10T23:52:39.005 INFO:teuthology.orchestra.run.smithi178.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-10T23:52:39.005 INFO:teuthology.orchestra.run.smithi178.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-10T23:52:39.005 INFO:teuthology.orchestra.run.smithi178.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-10T23:52:39.005 INFO:teuthology.orchestra.run.smithi178.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-10T23:52:39.005 INFO:teuthology.orchestra.run.smithi178.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-10T23:52:39.005 INFO:teuthology.orchestra.run.smithi178.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-10T23:52:39.005 INFO:teuthology.orchestra.run.smithi178.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-10T23:52:39.005 INFO:teuthology.orchestra.run.smithi178.stdout: postgresql-private-libs x86_64 13.16-1.el9 appstream 137 k 2024-09-10T23:52:39.005 INFO:teuthology.orchestra.run.smithi178.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-10T23:52:39.005 INFO:teuthology.orchestra.run.smithi178.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-10T23:52:39.005 INFO:teuthology.orchestra.run.smithi178.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-10T23:52:39.005 INFO:teuthology.orchestra.run.smithi178.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-09-10T23:52:39.006 INFO:teuthology.orchestra.run.smithi178.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-10T23:52:39.006 INFO:teuthology.orchestra.run.smithi178.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-10T23:52:39.006 INFO:teuthology.orchestra.run.smithi178.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-10T23:52:39.006 INFO:teuthology.orchestra.run.smithi178.stdout: python3-ceph-argparse x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 45 k 2024-09-10T23:52:39.006 INFO:teuthology.orchestra.run.smithi178.stdout: python3-ceph-common x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 143 k 2024-09-10T23:52:39.006 INFO:teuthology.orchestra.run.smithi178.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-10T23:52:39.006 INFO:teuthology.orchestra.run.smithi178.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-10T23:52:39.006 INFO:teuthology.orchestra.run.smithi178.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-10T23:52:39.006 INFO:teuthology.orchestra.run.smithi178.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-10T23:52:39.006 INFO:teuthology.orchestra.run.smithi178.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-09-10T23:52:39.006 INFO:teuthology.orchestra.run.smithi178.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-09-10T23:52:39.006 INFO:teuthology.orchestra.run.smithi178.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-10T23:52:39.007 INFO:teuthology.orchestra.run.smithi178.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-10T23:52:39.007 INFO:teuthology.orchestra.run.smithi178.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-10T23:52:39.007 INFO:teuthology.orchestra.run.smithi178.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-10T23:52:39.007 INFO:teuthology.orchestra.run.smithi178.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-10T23:52:39.007 INFO:teuthology.orchestra.run.smithi178.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-10T23:52:39.007 INFO:teuthology.orchestra.run.smithi178.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-10T23:52:39.007 INFO:teuthology.orchestra.run.smithi178.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-10T23:52:39.007 INFO:teuthology.orchestra.run.smithi178.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-10T23:52:39.007 INFO:teuthology.orchestra.run.smithi178.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-10T23:52:39.007 INFO:teuthology.orchestra.run.smithi178.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-10T23:52:39.007 INFO:teuthology.orchestra.run.smithi178.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-10T23:52:39.007 INFO:teuthology.orchestra.run.smithi178.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-10T23:52:39.007 INFO:teuthology.orchestra.run.smithi178.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-10T23:52:39.007 INFO:teuthology.orchestra.run.smithi178.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-10T23:52:39.008 INFO:teuthology.orchestra.run.smithi178.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-10T23:52:39.008 INFO:teuthology.orchestra.run.smithi178.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-10T23:52:39.008 INFO:teuthology.orchestra.run.smithi178.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-10T23:52:39.008 INFO:teuthology.orchestra.run.smithi178.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-10T23:52:39.008 INFO:teuthology.orchestra.run.smithi178.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-10T23:52:39.008 INFO:teuthology.orchestra.run.smithi178.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-10T23:52:39.008 INFO:teuthology.orchestra.run.smithi178.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-10T23:52:39.008 INFO:teuthology.orchestra.run.smithi178.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-10T23:52:39.008 INFO:teuthology.orchestra.run.smithi178.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-10T23:52:39.008 INFO:teuthology.orchestra.run.smithi178.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-10T23:52:39.008 INFO:teuthology.orchestra.run.smithi178.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-10T23:52:39.008 INFO:teuthology.orchestra.run.smithi178.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-10T23:52:39.008 INFO:teuthology.orchestra.run.smithi178.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-10T23:52:39.008 INFO:teuthology.orchestra.run.smithi178.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-10T23:52:39.009 INFO:teuthology.orchestra.run.smithi178.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-10T23:52:39.009 INFO:teuthology.orchestra.run.smithi178.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-10T23:52:39.009 INFO:teuthology.orchestra.run.smithi178.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-10T23:52:39.009 INFO:teuthology.orchestra.run.smithi178.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-09-10T23:52:39.009 INFO:teuthology.orchestra.run.smithi178.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-10T23:52:39.009 INFO:teuthology.orchestra.run.smithi178.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-10T23:52:39.009 INFO:teuthology.orchestra.run.smithi178.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-10T23:52:39.009 INFO:teuthology.orchestra.run.smithi178.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-10T23:52:39.009 INFO:teuthology.orchestra.run.smithi178.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-10T23:52:39.009 INFO:teuthology.orchestra.run.smithi178.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-10T23:52:39.009 INFO:teuthology.orchestra.run.smithi178.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-10T23:52:39.009 INFO:teuthology.orchestra.run.smithi178.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-10T23:52:39.010 INFO:teuthology.orchestra.run.smithi178.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-10T23:52:39.010 INFO:teuthology.orchestra.run.smithi178.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-10T23:52:39.010 INFO:teuthology.orchestra.run.smithi178.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-10T23:52:39.010 INFO:teuthology.orchestra.run.smithi178.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-09-10T23:52:39.010 INFO:teuthology.orchestra.run.smithi178.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-10T23:52:39.010 INFO:teuthology.orchestra.run.smithi178.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-10T23:52:39.010 INFO:teuthology.orchestra.run.smithi178.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-10T23:52:39.010 INFO:teuthology.orchestra.run.smithi178.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-10T23:52:39.010 INFO:teuthology.orchestra.run.smithi178.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-10T23:52:39.010 INFO:teuthology.orchestra.run.smithi178.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-10T23:52:39.010 INFO:teuthology.orchestra.run.smithi178.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-10T23:52:39.010 INFO:teuthology.orchestra.run.smithi178.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-10T23:52:39.010 INFO:teuthology.orchestra.run.smithi178.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-10T23:52:39.011 INFO:teuthology.orchestra.run.smithi178.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-09-10T23:52:39.011 INFO:teuthology.orchestra.run.smithi178.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-10T23:52:39.011 INFO:teuthology.orchestra.run.smithi178.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-09-10T23:52:39.011 INFO:teuthology.orchestra.run.smithi178.stdout:Installing weak dependencies: 2024-09-10T23:52:39.011 INFO:teuthology.orchestra.run.smithi178.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-10T23:52:39.011 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-09-10T23:52:39.011 INFO:teuthology.orchestra.run.smithi178.stdout:Transaction Summary 2024-09-10T23:52:39.011 INFO:teuthology.orchestra.run.smithi178.stdout:======================================================================================= 2024-09-10T23:52:39.011 INFO:teuthology.orchestra.run.smithi178.stdout:Install 149 Packages 2024-09-10T23:52:39.011 INFO:teuthology.orchestra.run.smithi178.stdout:Upgrade 8 Packages 2024-09-10T23:52:39.011 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-09-10T23:52:39.012 INFO:teuthology.orchestra.run.smithi178.stdout:Total download size: 240 M 2024-09-10T23:52:39.012 INFO:teuthology.orchestra.run.smithi178.stdout:Downloading Packages: 2024-09-10T23:52:39.032 INFO:teuthology.orchestra.run.smithi165.stdout:(1/157): ceph-19.3.0-4854.g8f219c47.el9.x86_64. 58 kB/s | 6.6 kB 00:00 2024-09-10T23:52:39.216 INFO:teuthology.orchestra.run.smithi165.stdout:(2/157): ceph-fuse-19.3.0-4854.g8f219c47.el9.x8 4.8 MB/s | 900 kB 00:00 2024-09-10T23:52:39.317 INFO:teuthology.orchestra.run.smithi165.stdout:(3/157): ceph-immutable-object-cache-19.3.0-485 1.4 MB/s | 145 kB 00:00 2024-09-10T23:52:39.408 INFO:teuthology.orchestra.run.smithi165.stdout:(4/157): ceph-base-19.3.0-4854.g8f219c47.el9.x8 11 MB/s | 5.5 MB 00:00 2024-09-10T23:52:39.482 INFO:teuthology.orchestra.run.smithi012.stdout:CentOS Stream 9 - Extras packages 19 kB/s | 19 kB 00:00 2024-09-10T23:52:39.517 INFO:teuthology.orchestra.run.smithi165.stdout:(5/157): ceph-mds-19.3.0-4854.g8f219c47.el9.x86 12 MB/s | 2.4 MB 00:00 2024-09-10T23:52:39.559 INFO:teuthology.orchestra.run.smithi165.stdout:(6/157): ceph-mgr-19.3.0-4854.g8f219c47.el9.x86 10 MB/s | 1.5 MB 00:00 2024-09-10T23:52:39.872 INFO:teuthology.orchestra.run.smithi178.stdout:(1/157): ceph-19.3.0-4854.g8f219c47.el9.x86_64. 23 kB/s | 6.6 kB 00:00 2024-09-10T23:52:39.884 INFO:teuthology.orchestra.run.smithi165.stdout:(7/157): ceph-mon-19.3.0-4854.g8f219c47.el9.x86 13 MB/s | 4.8 MB 00:00 2024-09-10T23:52:40.143 INFO:teuthology.orchestra.run.smithi165.stdout:(8/157): ceph-common-19.3.0-4854.g8f219c47.el9. 18 MB/s | 22 MB 00:01 2024-09-10T23:52:40.193 INFO:teuthology.orchestra.run.smithi165.stdout:(9/157): ceph-selinux-19.3.0-4854.g8f219c47.el9 505 kB/s | 25 kB 00:00 2024-09-10T23:52:40.247 INFO:teuthology.orchestra.run.smithi178.stdout:(2/157): ceph-fuse-19.3.0-4854.g8f219c47.el9.x8 2.3 MB/s | 900 kB 00:00 2024-09-10T23:52:40.381 INFO:teuthology.orchestra.run.smithi178.stdout:(3/157): ceph-immutable-object-cache-19.3.0-485 1.1 MB/s | 145 kB 00:00 2024-09-10T23:52:40.652 INFO:teuthology.orchestra.run.smithi165.stdout:(10/157): ceph-radosgw-19.3.0-4854.g8f219c47.el 14 MB/s | 11 MB 00:00 2024-09-10T23:52:40.827 INFO:teuthology.orchestra.run.smithi012.stdout:Extra Packages for Enterprise Linux 19 MB/s | 23 MB 00:01 2024-09-10T23:52:40.835 INFO:teuthology.orchestra.run.smithi165.stdout:(11/157): ceph-osd-19.3.0-4854.g8f219c47.el9.x8 13 MB/s | 17 MB 00:01 2024-09-10T23:52:40.860 INFO:teuthology.orchestra.run.smithi165.stdout:(12/157): cephfs-mirror-19.3.0-4854.g8f219c47.e 1.1 MB/s | 225 kB 00:00 2024-09-10T23:52:40.873 INFO:teuthology.orchestra.run.smithi178.stdout:(4/157): ceph-base-19.3.0-4854.g8f219c47.el9.x8 4.3 MB/s | 5.5 MB 00:01 2024-09-10T23:52:40.886 INFO:teuthology.orchestra.run.smithi165.stdout:(13/157): cephfs-shell-19.3.0-4854.g8f219c47.el 540 kB/s | 27 kB 00:00 2024-09-10T23:52:40.911 INFO:teuthology.orchestra.run.smithi165.stdout:(14/157): libcephfs-devel-19.3.0-4854.g8f219c47 632 kB/s | 32 kB 00:00 2024-09-10T23:52:40.948 INFO:teuthology.orchestra.run.smithi178.stdout:(5/157): ceph-mds-19.3.0-4854.g8f219c47.el9.x86 4.2 MB/s | 2.4 MB 00:00 2024-09-10T23:52:40.953 INFO:teuthology.orchestra.run.smithi165.stdout:(15/157): libcephfs2-19.3.0-4854.g8f219c47.el9. 11 MB/s | 755 kB 00:00 2024-09-10T23:52:40.978 INFO:teuthology.orchestra.run.smithi165.stdout:(16/157): libcephsqlite-19.3.0-4854.g8f219c47.e 2.4 MB/s | 162 kB 00:00 2024-09-10T23:52:41.004 INFO:teuthology.orchestra.run.smithi165.stdout:(17/157): librados-devel-19.3.0-4854.g8f219c47. 2.5 MB/s | 127 kB 00:00 2024-09-10T23:52:41.037 INFO:teuthology.orchestra.run.smithi165.stdout:(18/157): libradosstriper1-19.3.0-4854.g8f219c4 8.5 MB/s | 507 kB 00:00 2024-09-10T23:52:41.057 INFO:teuthology.orchestra.run.smithi178.stdout:(6/157): ceph-mgr-19.3.0-4854.g8f219c47.el9.x86 8.3 MB/s | 1.5 MB 00:00 2024-09-10T23:52:41.088 INFO:teuthology.orchestra.run.smithi165.stdout:(19/157): python3-ceph-argparse-19.3.0-4854.g8f 905 kB/s | 45 kB 00:00 2024-09-10T23:52:41.179 INFO:teuthology.orchestra.run.smithi165.stdout:(20/157): python3-ceph-common-19.3.0-4854.g8f21 1.5 MB/s | 143 kB 00:00 2024-09-10T23:52:41.271 INFO:teuthology.orchestra.run.smithi165.stdout:(21/157): python3-cephfs-19.3.0-4854.g8f219c47. 1.7 MB/s | 161 kB 00:00 2024-09-10T23:52:41.363 INFO:teuthology.orchestra.run.smithi165.stdout:(22/157): python3-rados-19.3.0-4854.g8f219c47.e 3.4 MB/s | 321 kB 00:00 2024-09-10T23:52:41.472 INFO:teuthology.orchestra.run.smithi165.stdout:(23/157): librgw2-19.3.0-4854.g8f219c47.el9.x86 12 MB/s | 5.5 MB 00:00 2024-09-10T23:52:41.497 INFO:teuthology.orchestra.run.smithi165.stdout:(24/157): python3-rbd-19.3.0-4854.g8f219c47.el9 2.2 MB/s | 306 kB 00:00 2024-09-10T23:52:41.556 INFO:teuthology.orchestra.run.smithi165.stdout:(25/157): python3-rgw-19.3.0-4854.g8f219c47.el9 1.2 MB/s | 99 kB 00:00 2024-09-10T23:52:41.581 INFO:teuthology.orchestra.run.smithi165.stdout:(26/157): rbd-fuse-19.3.0-4854.g8f219c47.el9.x8 1.0 MB/s | 85 kB 00:00 2024-09-10T23:52:41.682 INFO:teuthology.orchestra.run.smithi165.stdout:(27/157): rbd-nbd-19.3.0-4854.g8f219c47.el9.x86 1.7 MB/s | 171 kB 00:00 2024-09-10T23:52:41.774 INFO:teuthology.orchestra.run.smithi165.stdout:(28/157): ceph-grafana-dashboards-19.3.0-4854.g 374 kB/s | 34 kB 00:00 2024-09-10T23:52:41.874 INFO:teuthology.orchestra.run.smithi178.stdout:(7/157): ceph-mon-19.3.0-4854.g8f219c47.el9.x86 5.2 MB/s | 4.8 MB 00:00 2024-09-10T23:52:41.907 INFO:teuthology.orchestra.run.smithi165.stdout:(29/157): rbd-mirror-19.3.0-4854.g8f219c47.el9. 9.1 MB/s | 3.2 MB 00:00 2024-09-10T23:52:41.933 INFO:teuthology.orchestra.run.smithi165.stdout:(30/157): ceph-mgr-cephadm-19.3.0-4854.g8f219c4 974 kB/s | 154 kB 00:00 2024-09-10T23:52:42.166 INFO:teuthology.orchestra.run.smithi178.stdout:(8/157): ceph-common-19.3.0-4854.g8f219c47.el9. 8.4 MB/s | 22 MB 00:02 2024-09-10T23:52:42.266 INFO:teuthology.orchestra.run.smithi178.stdout:(9/157): ceph-selinux-19.3.0-4854.g8f219c47.el9 252 kB/s | 25 kB 00:00 2024-09-10T23:52:43.008 INFO:teuthology.orchestra.run.smithi178.stdout:(10/157): ceph-osd-19.3.0-4854.g8f219c47.el9.x8 8.8 MB/s | 17 MB 00:01 2024-09-10T23:52:43.033 INFO:teuthology.orchestra.run.smithi165.stdout:(31/157): ceph-mgr-diskprediction-local-19.3.0- 6.7 MB/s | 7.4 MB 00:01 2024-09-10T23:52:43.058 INFO:teuthology.orchestra.run.smithi178.stdout:(11/157): cephfs-mirror-19.3.0-4854.g8f219c47.e 4.4 MB/s | 225 kB 00:00 2024-09-10T23:52:43.108 INFO:teuthology.orchestra.run.smithi178.stdout:(12/157): cephfs-shell-19.3.0-4854.g8f219c47.el 544 kB/s | 27 kB 00:00 2024-09-10T23:52:43.158 INFO:teuthology.orchestra.run.smithi178.stdout:(13/157): libcephfs-devel-19.3.0-4854.g8f219c47 631 kB/s | 32 kB 00:00 2024-09-10T23:52:43.191 INFO:teuthology.orchestra.run.smithi165.stdout:(32/157): ceph-mgr-dashboard-19.3.0-4854.g8f219 11 MB/s | 14 MB 00:01 2024-09-10T23:52:43.216 INFO:teuthology.orchestra.run.smithi165.stdout:(33/157): ceph-mgr-modules-core-19.3.0-4854.g8f 1.5 MB/s | 291 kB 00:00 2024-09-10T23:52:43.233 INFO:teuthology.orchestra.run.smithi165.stdout:(34/157): ceph-mgr-rook-19.3.0-4854.g8f219c47.e 1.1 MB/s | 49 kB 00:00 2024-09-10T23:52:43.250 INFO:teuthology.orchestra.run.smithi178.stdout:(14/157): libcephfs2-19.3.0-4854.g8f219c47.el9. 8.1 MB/s | 755 kB 00:00 2024-09-10T23:52:43.392 INFO:teuthology.orchestra.run.smithi178.stdout:(15/157): ceph-radosgw-19.3.0-4854.g8f219c47.el 7.2 MB/s | 11 MB 00:01 2024-09-10T23:52:43.417 INFO:teuthology.orchestra.run.smithi178.stdout:(16/157): libcephsqlite-19.3.0-4854.g8f219c47.e 970 kB/s | 162 kB 00:00 2024-09-10T23:52:43.443 INFO:teuthology.orchestra.run.smithi178.stdout:(17/157): librados-devel-19.3.0-4854.g8f219c47. 2.5 MB/s | 127 kB 00:00 2024-09-10T23:52:43.477 INFO:teuthology.orchestra.run.smithi178.stdout:(18/157): libradosstriper1-19.3.0-4854.g8f219c4 8.5 MB/s | 507 kB 00:00 2024-09-10T23:52:43.527 INFO:teuthology.orchestra.run.smithi178.stdout:(19/157): python3-ceph-argparse-19.3.0-4854.g8f 909 kB/s | 45 kB 00:00 2024-09-10T23:52:43.586 INFO:teuthology.orchestra.run.smithi178.stdout:(20/157): python3-ceph-common-19.3.0-4854.g8f21 2.4 MB/s | 143 kB 00:00 2024-09-10T23:52:43.644 INFO:teuthology.orchestra.run.smithi178.stdout:(21/157): python3-cephfs-19.3.0-4854.g8f219c47. 2.7 MB/s | 161 kB 00:00 2024-09-10T23:52:43.676 INFO:teuthology.orchestra.run.smithi165.stdout:(35/157): ceph-test-19.3.0-4854.g8f219c47.el9.x 14 MB/s | 50 MB 00:03 2024-09-10T23:52:43.711 INFO:teuthology.orchestra.run.smithi178.stdout:(22/157): python3-rados-19.3.0-4854.g8f219c47.e 4.7 MB/s | 321 kB 00:00 2024-09-10T23:52:43.770 INFO:teuthology.orchestra.run.smithi178.stdout:(23/157): python3-rbd-19.3.0-4854.g8f219c47.el9 5.1 MB/s | 306 kB 00:00 2024-09-10T23:52:43.851 INFO:teuthology.orchestra.run.smithi165.stdout:(36/157): ceph-prometheus-alerts-19.3.0-4854.g8 26 kB/s | 16 kB 00:00 2024-09-10T23:52:43.870 INFO:teuthology.orchestra.run.smithi178.stdout:(24/157): librgw2-19.3.0-4854.g8f219c47.el9.x86 13 MB/s | 5.5 MB 00:00 2024-09-10T23:52:43.876 INFO:teuthology.orchestra.run.smithi165.stdout:(37/157): ceph-volume-19.3.0-4854.g8f219c47.el9 444 kB/s | 285 kB 00:00 2024-09-10T23:52:43.895 INFO:teuthology.orchestra.run.smithi178.stdout:(25/157): python3-rgw-19.3.0-4854.g8f219c47.el9 794 kB/s | 99 kB 00:00 2024-09-10T23:52:43.902 INFO:teuthology.orchestra.run.smithi165.stdout:(38/157): cephfs-top-19.3.0-4854.g8f219c47.el9. 363 kB/s | 18 kB 00:00 2024-09-10T23:52:43.921 INFO:teuthology.orchestra.run.smithi178.stdout:(26/157): rbd-fuse-19.3.0-4854.g8f219c47.el9.x8 1.7 MB/s | 85 kB 00:00 2024-09-10T23:52:43.944 INFO:teuthology.orchestra.run.smithi165.stdout:(39/157): cephadm-19.3.0-4854.g8f219c47.el9.noa 2.9 MB/s | 783 kB 00:00 2024-09-10T23:52:43.979 INFO:teuthology.orchestra.run.smithi178.stdout:(27/157): rbd-nbd-19.3.0-4854.g8f219c47.el9.x86 2.9 MB/s | 171 kB 00:00 2024-09-10T23:52:44.038 INFO:teuthology.orchestra.run.smithi178.stdout:(28/157): ceph-grafana-dashboards-19.3.0-4854.g 588 kB/s | 34 kB 00:00 2024-09-10T23:52:44.096 INFO:teuthology.orchestra.run.smithi178.stdout:(29/157): ceph-mgr-cephadm-19.3.0-4854.g8f219c4 2.6 MB/s | 154 kB 00:00 2024-09-10T23:52:44.155 INFO:teuthology.orchestra.run.smithi178.stdout:(30/157): rbd-mirror-19.3.0-4854.g8f219c47.el9. 12 MB/s | 3.2 MB 00:00 2024-09-10T23:52:44.536 INFO:teuthology.orchestra.run.smithi165.stdout:(40/157): fuse-2.9.9-16.el9.x86_64.rpm 121 kB/s | 80 kB 00:00 2024-09-10T23:52:44.680 INFO:teuthology.orchestra.run.smithi178.stdout:(31/157): ceph-mgr-diskprediction-local-19.3.0- 14 MB/s | 7.4 MB 00:00 2024-09-10T23:52:44.727 INFO:teuthology.orchestra.run.smithi165.stdout:(41/157): libquadmath-11.5.0-2.el9.x86_64.rpm 244 kB/s | 191 kB 00:00 2024-09-10T23:52:44.739 INFO:teuthology.orchestra.run.smithi178.stdout:(32/157): ceph-mgr-modules-core-19.3.0-4854.g8f 4.9 MB/s | 291 kB 00:00 2024-09-10T23:52:44.789 INFO:teuthology.orchestra.run.smithi178.stdout:(33/157): ceph-mgr-rook-19.3.0-4854.g8f219c47.e 992 kB/s | 49 kB 00:00 2024-09-10T23:52:44.839 INFO:teuthology.orchestra.run.smithi178.stdout:(34/157): ceph-prometheus-alerts-19.3.0-4854.g8 330 kB/s | 16 kB 00:00 2024-09-10T23:52:44.898 INFO:teuthology.orchestra.run.smithi178.stdout:(35/157): ceph-volume-19.3.0-4854.g8f219c47.el9 4.8 MB/s | 285 kB 00:00 2024-09-10T23:52:44.990 INFO:teuthology.orchestra.run.smithi178.stdout:(36/157): cephadm-19.3.0-4854.g8f219c47.el9.noa 8.4 MB/s | 783 kB 00:00 2024-09-10T23:52:45.132 INFO:teuthology.orchestra.run.smithi178.stdout:(37/157): ceph-mgr-dashboard-19.3.0-4854.g8f219 14 MB/s | 14 MB 00:01 2024-09-10T23:52:45.148 INFO:teuthology.orchestra.run.smithi178.stdout:(38/157): cephfs-top-19.3.0-4854.g8f219c47.el9. 115 kB/s | 18 kB 00:00 2024-09-10T23:52:45.378 INFO:teuthology.orchestra.run.smithi165.stdout:(42/157): libgfortran-11.5.0-2.el9.x86_64.rpm 543 kB/s | 801 kB 00:01 2024-09-10T23:52:45.503 INFO:teuthology.orchestra.run.smithi165.stdout:(43/157): python3-cffi-1.14.5-5.el9.x86_64.rpm 262 kB/s | 253 kB 00:00 2024-09-10T23:52:45.615 INFO:teuthology.orchestra.run.smithi178.stdout:(39/157): ceph-test-19.3.0-4854.g8f219c47.el9.x 15 MB/s | 50 MB 00:03 2024-09-10T23:52:45.641 INFO:teuthology.orchestra.run.smithi178.stdout:(40/157): fuse-2.9.9-16.el9.x86_64.rpm 157 kB/s | 80 kB 00:00 2024-09-10T23:52:45.724 INFO:teuthology.orchestra.run.smithi178.stdout:(41/157): python3-cffi-1.14.5-5.el9.x86_64.rpm 3.0 MB/s | 253 kB 00:00 2024-09-10T23:52:45.758 INFO:teuthology.orchestra.run.smithi178.stdout:(42/157): libgfortran-11.5.0-2.el9.x86_64.rpm 1.3 MB/s | 801 kB 00:00 2024-09-10T23:52:45.762 INFO:teuthology.orchestra.run.smithi165.stdout:(44/157): python3-cryptography-36.0.1-4.el9.x86 1.2 MB/s | 1.2 MB 00:01 2024-09-10T23:52:45.792 INFO:teuthology.orchestra.run.smithi178.stdout:(43/157): libquadmath-11.5.0-2.el9.x86_64.rpm 1.1 MB/s | 191 kB 00:00 2024-09-10T23:52:45.817 INFO:teuthology.orchestra.run.smithi178.stdout:(44/157): python3-ply-3.11-14.el9.noarch.rpm 1.8 MB/s | 106 kB 00:00 2024-09-10T23:52:45.834 INFO:teuthology.orchestra.run.smithi178.stdout:(45/157): python3-pycparser-2.20-6.el9.noarch.r 3.2 MB/s | 135 kB 00:00 2024-09-10T23:52:45.876 INFO:teuthology.orchestra.run.smithi178.stdout:(46/157): python3-requests-2.25.1-8.el9.noarch. 2.1 MB/s | 125 kB 00:00 2024-09-10T23:52:45.918 INFO:teuthology.orchestra.run.smithi178.stdout:(47/157): python3-urllib3-1.26.5-6.el9.noarch.r 2.5 MB/s | 215 kB 00:00 2024-09-10T23:52:46.043 INFO:teuthology.orchestra.run.smithi178.stdout:(48/157): python3-cryptography-36.0.1-4.el9.x86 3.9 MB/s | 1.2 MB 00:00 2024-09-10T23:52:46.103 INFO:teuthology.orchestra.run.smithi165.stdout:(45/157): python3-ply-3.11-14.el9.noarch.rpm 147 kB/s | 106 kB 00:00 2024-09-10T23:52:46.160 INFO:teuthology.orchestra.run.smithi178.stdout:(49/157): automake-1.16.2-8.el9.noarch.rpm 2.4 MB/s | 689 kB 00:00 2024-09-10T23:52:46.194 INFO:teuthology.orchestra.run.smithi178.stdout:(50/157): bison-3.7.4-5.el9.x86_64.rpm 3.3 MB/s | 943 kB 00:00 2024-09-10T23:52:46.219 INFO:teuthology.orchestra.run.smithi178.stdout:(51/157): boost-program-options-1.75.0-8.el9.x8 609 kB/s | 107 kB 00:00 2024-09-10T23:52:46.244 INFO:teuthology.orchestra.run.smithi178.stdout:(52/157): elfutils-libelf-devel-0.191-4.el9.x86 259 kB/s | 22 kB 00:00 2024-09-10T23:52:46.245 INFO:teuthology.orchestra.run.smithi165.stdout:(46/157): python3-pycparser-2.20-6.el9.noarch.r 182 kB/s | 135 kB 00:00 2024-09-10T23:52:46.278 INFO:teuthology.orchestra.run.smithi178.stdout:(53/157): flex-2.6.4-9.el9.x86_64.rpm 3.7 MB/s | 313 kB 00:00 2024-09-10T23:52:46.303 INFO:teuthology.orchestra.run.smithi178.stdout:(54/157): flexiblas-3.0.4-8.el9.x86_64.rpm 559 kB/s | 33 kB 00:00 2024-09-10T23:52:46.328 INFO:teuthology.orchestra.run.smithi178.stdout:(55/157): flexiblas-openblas-openmp-3.0.4-8.el9 363 kB/s | 18 kB 00:00 2024-09-10T23:52:46.353 INFO:teuthology.orchestra.run.smithi178.stdout:(56/157): libaio-devel-0.3.111-13.el9.x86_64.rp 222 kB/s | 11 kB 00:00 2024-09-10T23:52:46.404 INFO:teuthology.orchestra.run.smithi178.stdout:(57/157): libpmemobj-1.12.1-1.el9.x86_64.rpm 3.1 MB/s | 160 kB 00:00 2024-09-10T23:52:46.429 INFO:teuthology.orchestra.run.smithi178.stdout:(58/157): libnbd-1.20.2-2.el9.x86_64.rpm 1.7 MB/s | 171 kB 00:00 2024-09-10T23:52:46.454 INFO:teuthology.orchestra.run.smithi178.stdout:(59/157): librabbitmq-0.11.0-7.el9.x86_64.rpm 903 kB/s | 45 kB 00:00 2024-09-10T23:52:46.464 INFO:teuthology.orchestra.run.smithi165.stdout:(47/157): python3-requests-2.25.1-8.el9.noarch. 178 kB/s | 125 kB 00:00 2024-09-10T23:52:46.596 INFO:teuthology.orchestra.run.smithi178.stdout:(60/157): libtool-2.4.6-46.el9.x86_64.rpm 4.0 MB/s | 581 kB 00:00 2024-09-10T23:52:46.630 INFO:teuthology.orchestra.run.smithi178.stdout:(61/157): librdkafka-1.6.1-102.el9.x86_64.rpm 3.2 MB/s | 662 kB 00:00 2024-09-10T23:52:46.647 INFO:teuthology.orchestra.run.smithi178.stdout:(62/157): libuuid-devel-2.37.4-20.el9.x86_64.rp 486 kB/s | 24 kB 00:00 2024-09-10T23:52:46.680 INFO:teuthology.orchestra.run.smithi178.stdout:(63/157): libzstd-devel-1.5.1-2.el9.x86_64.rpm 947 kB/s | 47 kB 00:00 2024-09-10T23:52:46.756 INFO:teuthology.orchestra.run.smithi178.stdout:(64/157): flexiblas-netlib-3.0.4-8.el9.x86_64.r 5.8 MB/s | 3.0 MB 00:00 2024-09-10T23:52:46.781 INFO:teuthology.orchestra.run.smithi178.stdout:(65/157): lua-5.4.4-4.el9.x86_64.rpm 1.8 MB/s | 188 kB 00:00 2024-09-10T23:52:46.806 INFO:teuthology.orchestra.run.smithi178.stdout:(66/157): lttng-ust-2.12.0-6.el9.x86_64.rpm 1.8 MB/s | 292 kB 00:00 2024-09-10T23:52:46.823 INFO:teuthology.orchestra.run.smithi178.stdout:(67/157): openblas-0.3.26-2.el9.x86_64.rpm 575 kB/s | 38 kB 00:00 2024-09-10T23:52:46.912 INFO:teuthology.orchestra.run.smithi165.stdout:(48/157): python3-urllib3-1.26.5-6.el9.noarch.r 266 kB/s | 215 kB 00:00 2024-09-10T23:52:47.063 INFO:teuthology.orchestra.run.smithi165.stdout:(49/157): automake-1.16.2-8.el9.noarch.rpm 844 kB/s | 689 kB 00:00 2024-09-10T23:52:47.073 INFO:teuthology.orchestra.run.smithi178.stdout:(68/157): postgresql-13.16-1.el9.x86_64.rpm 6.3 MB/s | 1.6 MB 00:00 2024-09-10T23:52:47.179 INFO:teuthology.orchestra.run.smithi165.stdout:(50/157): elfutils-libelf-devel-0.191-4.el9.x86 186 kB/s | 22 kB 00:00 2024-09-10T23:52:47.240 INFO:teuthology.orchestra.run.smithi178.stdout:(69/157): postgresql-contrib-13.16-1.el9.x86_64 5.3 MB/s | 910 kB 00:00 2024-09-10T23:52:47.288 INFO:teuthology.orchestra.run.smithi165.stdout:(51/157): bison-3.7.4-5.el9.x86_64.rpm 1.1 MB/s | 943 kB 00:00 2024-09-10T23:52:47.290 INFO:teuthology.orchestra.run.smithi178.stdout:(70/157): postgresql-private-libs-13.16-1.el9.x 2.7 MB/s | 137 kB 00:00 2024-09-10T23:52:47.313 INFO:teuthology.orchestra.run.smithi165.stdout:(52/157): flex-2.6.4-9.el9.x86_64.rpm 2.3 MB/s | 313 kB 00:00 2024-09-10T23:52:47.372 INFO:teuthology.orchestra.run.smithi165.stdout:(53/157): boost-program-options-1.75.0-8.el9.x8 233 kB/s | 107 kB 00:00 2024-09-10T23:52:47.397 INFO:teuthology.orchestra.run.smithi165.stdout:(54/157): flexiblas-3.0.4-8.el9.x86_64.rpm 300 kB/s | 33 kB 00:00 2024-09-10T23:52:47.514 INFO:teuthology.orchestra.run.smithi165.stdout:(55/157): flexiblas-openblas-openmp-3.0.4-8.el9 128 kB/s | 18 kB 00:00 2024-09-10T23:52:47.516 INFO:teuthology.orchestra.run.smithi012.stdout:lab-extras 30 kB/s | 1.7 kB 00:00 2024-09-10T23:52:47.531 INFO:teuthology.orchestra.run.smithi165.stdout:(56/157): libaio-devel-0.3.111-13.el9.x86_64.rp 83 kB/s | 11 kB 00:00 2024-09-10T23:52:47.656 INFO:teuthology.orchestra.run.smithi165.stdout:(57/157): libnbd-1.20.2-2.el9.x86_64.rpm 1.3 MB/s | 171 kB 00:00 2024-09-10T23:52:47.773 INFO:teuthology.orchestra.run.smithi165.stdout:(58/157): flexiblas-netlib-3.0.4-8.el9.x86_64.r 6.5 MB/s | 3.0 MB 00:00 2024-09-10T23:52:47.790 INFO:teuthology.orchestra.run.smithi165.stdout:(59/157): librabbitmq-0.11.0-7.el9.x86_64.rpm 339 kB/s | 45 kB 00:00 2024-09-10T23:52:47.812 INFO:teuthology.orchestra.run.smithi165.stdout:(60/157): libpmemobj-1.12.1-1.el9.x86_64.rpm 572 kB/s | 160 kB 00:00 2024-09-10T23:52:47.866 INFO:teuthology.orchestra.run.smithi178.stdout:(71/157): openblas-openmp-0.3.26-2.el9.x86_64.r 4.6 MB/s | 4.9 MB 00:01 2024-09-10T23:52:47.904 INFO:teuthology.orchestra.run.smithi165.stdout:(61/157): librdkafka-1.6.1-102.el9.x86_64.rpm 5.0 MB/s | 662 kB 00:00 2024-09-10T23:52:47.929 INFO:teuthology.orchestra.run.smithi165.stdout:(62/157): libuuid-devel-2.37.4-20.el9.x86_64.rp 209 kB/s | 24 kB 00:00 2024-09-10T23:52:48.012 INFO:teuthology.orchestra.run.smithi165.stdout:(63/157): libzstd-devel-1.5.1-2.el9.x86_64.rpm 438 kB/s | 47 kB 00:00 2024-09-10T23:52:48.049 INFO:teuthology.orchestra.run.smithi178.stdout:(72/157): openssl-devel-3.2.2-6.el9.x86_64.rpm 3.6 MB/s | 4.4 MB 00:01 2024-09-10T23:52:48.080 INFO:teuthology.orchestra.run.smithi165.stdout:(64/157): libtool-2.4.6-46.el9.x86_64.rpm 2.0 MB/s | 581 kB 00:00 2024-09-10T23:52:48.130 INFO:teuthology.orchestra.run.smithi165.stdout:(65/157): lua-5.4.4-4.el9.x86_64.rpm 1.6 MB/s | 188 kB 00:00 2024-09-10T23:52:48.141 INFO:teuthology.orchestra.run.smithi178.stdout:(73/157): python3-attrs-20.3.0-7.el9.noarch.rpm 954 kB/s | 87 kB 00:00 2024-09-10T23:52:48.155 INFO:teuthology.orchestra.run.smithi165.stdout:(66/157): lttng-ust-2.12.0-6.el9.x86_64.rpm 1.3 MB/s | 292 kB 00:00 2024-09-10T23:52:48.188 INFO:teuthology.orchestra.run.smithi165.stdout:(67/157): openblas-0.3.26-2.el9.x86_64.rpm 354 kB/s | 38 kB 00:00 2024-09-10T23:52:48.225 INFO:teuthology.orchestra.run.smithi178.stdout:(74/157): protobuf-3.14.0-13.el9.x86_64.rpm 2.8 MB/s | 1.0 MB 00:00 2024-09-10T23:52:48.275 INFO:teuthology.orchestra.run.smithi178.stdout:(75/157): python3-colorama-0.4.6-3.el9.noarch.r 1.2 MB/s | 61 kB 00:00 2024-09-10T23:52:48.384 INFO:teuthology.orchestra.run.smithi178.stdout:(76/157): python3-devel-3.9.19-8.el9.x86_64.rpm 2.2 MB/s | 245 kB 00:00 2024-09-10T23:52:48.493 INFO:teuthology.orchestra.run.smithi178.stdout:(77/157): python3-jinja2-2.11.3-6.el9.noarch.rp 2.2 MB/s | 248 kB 00:00 2024-09-10T23:52:48.498 INFO:teuthology.orchestra.run.smithi165.stdout:(68/157): postgresql-13.16-1.el9.x86_64.rpm 5.1 MB/s | 1.6 MB 00:00 2024-09-10T23:52:48.559 INFO:teuthology.orchestra.run.smithi178.stdout:(78/157): python3-jsonpatch-1.21-16.el9.noarch. 387 kB/s | 26 kB 00:00 2024-09-10T23:52:48.618 INFO:teuthology.orchestra.run.smithi178.stdout:(79/157): python3-jsonpointer-2.0-4.el9.noarch. 328 kB/s | 19 kB 00:00 2024-09-10T23:52:48.647 INFO:teuthology.orchestra.run.smithi165.stdout:(69/157): postgresql-contrib-13.16-1.el9.x86_64 5.9 MB/s | 910 kB 00:00 2024-09-10T23:52:48.694 INFO:teuthology.orchestra.run.smithi178.stdout:(80/157): postgresql-server-13.16-1.el9.x86_64. 4.1 MB/s | 5.8 MB 00:01 2024-09-10T23:52:48.735 INFO:teuthology.orchestra.run.smithi178.stdout:(81/157): python3-mako-1.1.4-6.el9.noarch.rpm 1.4 MB/s | 172 kB 00:00 2024-09-10T23:52:48.760 INFO:teuthology.orchestra.run.smithi178.stdout:(82/157): python3-markupsafe-1.1.1-12.el9.x86_6 521 kB/s | 35 kB 00:00 2024-09-10T23:52:48.764 INFO:teuthology.orchestra.run.smithi165.stdout:(70/157): postgresql-private-libs-13.16-1.el9.x 1.2 MB/s | 137 kB 00:00 2024-09-10T23:52:48.844 INFO:teuthology.orchestra.run.smithi178.stdout:(83/157): python3-numpy-f2py-1.20.1-5.el9.x86_6 2.9 MB/s | 250 kB 00:00 2024-09-10T23:52:48.928 INFO:teuthology.orchestra.run.smithi178.stdout:(84/157): python3-oauthlib-3.1.1-5.el9.noarch.r 2.6 MB/s | 222 kB 00:00 2024-09-10T23:52:48.986 INFO:teuthology.orchestra.run.smithi178.stdout:(85/157): python3-packaging-20.9-5.el9.noarch.r 1.3 MB/s | 77 kB 00:00 2024-09-10T23:52:49.037 INFO:teuthology.orchestra.run.smithi178.stdout:(86/157): python3-prettytable-0.7.2-27.el9.noar 845 kB/s | 42 kB 00:00 2024-09-10T23:52:49.145 INFO:teuthology.orchestra.run.smithi178.stdout:(87/157): python3-protobuf-3.14.0-13.el9.noarch 2.4 MB/s | 269 kB 00:00 2024-09-10T23:52:49.229 INFO:teuthology.orchestra.run.smithi178.stdout:(88/157): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.9 MB/s | 159 kB 00:00 2024-09-10T23:52:49.479 INFO:teuthology.orchestra.run.smithi178.stdout:(89/157): python3-pyasn1-modules-0.4.8-6.el9.no 1.1 MB/s | 279 kB 00:00 2024-09-10T23:52:49.770 INFO:teuthology.orchestra.run.smithi178.stdout:(90/157): python3-pytz-2021.1-5.el9.noarch.rpm 1.0 MB/s | 51 kB 00:00 2024-09-10T23:52:49.770 INFO:teuthology.orchestra.run.smithi178.stdout:(91/157): python3-requests-oauthlib-1.3.0-12.el 1.0 MB/s | 54 kB 00:00 2024-09-10T23:52:50.022 INFO:teuthology.orchestra.run.smithi178.stdout:(92/157): python3-babel-2.9.1-2.el9.noarch.rpm 3.2 MB/s | 6.0 MB 00:01 2024-09-10T23:52:50.072 INFO:teuthology.orchestra.run.smithi178.stdout:(93/157): python3-toml-0.10.2-6.el9.noarch.rpm 837 kB/s | 42 kB 00:00 2024-09-10T23:52:50.113 INFO:teuthology.orchestra.run.smithi012.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-10T23:52:50.114 INFO:teuthology.orchestra.run.smithi012.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-10T23:52:50.127 INFO:teuthology.orchestra.run.smithi012.stdout:Package NetworkManager-1:1.48.8-1.el9.x86_64 is already installed. 2024-09-10T23:52:50.128 INFO:teuthology.orchestra.run.smithi012.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-09-10T23:52:50.129 INFO:teuthology.orchestra.run.smithi012.stdout:Package util-linux-2.37.4-20.el9.x86_64 is already installed. 2024-09-10T23:52:50.130 INFO:teuthology.orchestra.run.smithi012.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-09-10T23:52:50.131 INFO:teuthology.orchestra.run.smithi165.stdout:(71/157): openssl-devel-3.2.2-6.el9.x86_64.rpm 2.2 MB/s | 4.4 MB 00:01 2024-09-10T23:52:50.131 INFO:teuthology.orchestra.run.smithi012.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-09-10T23:52:50.134 INFO:teuthology.orchestra.run.smithi012.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-09-10T23:52:50.135 INFO:teuthology.orchestra.run.smithi012.stdout:Package xfsprogs-6.4.0-4.el9.x86_64 is already installed. 2024-09-10T23:52:50.147 INFO:teuthology.orchestra.run.smithi178.stdout:(94/157): python3-numpy-1.20.1-5.el9.x86_64.rpm 3.6 MB/s | 5.1 MB 00:01 2024-09-10T23:52:50.173 INFO:teuthology.orchestra.run.smithi178.stdout:(95/157): python3-wcwidth-0.2.5-8.el9.noarch.rp 441 kB/s | 44 kB 00:00 2024-09-10T23:52:50.206 INFO:teuthology.orchestra.run.smithi012.stdout:Dependencies resolved. 2024-09-10T23:52:50.223 INFO:teuthology.orchestra.run.smithi178.stdout:(96/157): qatlib-service-24.02.0-1.el9.x86_64.r 720 kB/s | 36 kB 00:00 2024-09-10T23:52:50.228 INFO:teuthology.orchestra.run.smithi012.stdout:======================================================================================= 2024-09-10T23:52:50.228 INFO:teuthology.orchestra.run.smithi012.stdout: Package Arch Version Repository Size 2024-09-10T23:52:50.228 INFO:teuthology.orchestra.run.smithi012.stdout:======================================================================================= 2024-09-10T23:52:50.228 INFO:teuthology.orchestra.run.smithi012.stdout:Installing: 2024-09-10T23:52:50.228 INFO:teuthology.orchestra.run.smithi012.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-09-10T23:52:50.228 INFO:teuthology.orchestra.run.smithi012.stdout: ceph x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 6.6 k 2024-09-10T23:52:50.228 INFO:teuthology.orchestra.run.smithi012.stdout: ceph-base x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 5.5 M 2024-09-10T23:52:50.228 INFO:teuthology.orchestra.run.smithi012.stdout: ceph-fuse x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 900 k 2024-09-10T23:52:50.228 INFO:teuthology.orchestra.run.smithi012.stdout: ceph-immutable-object-cache x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 145 k 2024-09-10T23:52:50.228 INFO:teuthology.orchestra.run.smithi012.stdout: ceph-mgr x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 1.5 M 2024-09-10T23:52:50.228 INFO:teuthology.orchestra.run.smithi012.stdout: ceph-mgr-cephadm noarch 2:19.3.0-4854.g8f219c47.el9 ceph-noarch 154 k 2024-09-10T23:52:50.229 INFO:teuthology.orchestra.run.smithi012.stdout: ceph-mgr-dashboard noarch 2:19.3.0-4854.g8f219c47.el9 ceph-noarch 14 M 2024-09-10T23:52:50.229 INFO:teuthology.orchestra.run.smithi012.stdout: ceph-mgr-diskprediction-local noarch 2:19.3.0-4854.g8f219c47.el9 ceph-noarch 7.4 M 2024-09-10T23:52:50.229 INFO:teuthology.orchestra.run.smithi012.stdout: ceph-mgr-rook noarch 2:19.3.0-4854.g8f219c47.el9 ceph-noarch 49 k 2024-09-10T23:52:50.229 INFO:teuthology.orchestra.run.smithi012.stdout: ceph-radosgw x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 11 M 2024-09-10T23:52:50.229 INFO:teuthology.orchestra.run.smithi012.stdout: ceph-test x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 50 M 2024-09-10T23:52:50.229 INFO:teuthology.orchestra.run.smithi012.stdout: ceph-volume noarch 2:19.3.0-4854.g8f219c47.el9 ceph-noarch 285 k 2024-09-10T23:52:50.229 INFO:teuthology.orchestra.run.smithi012.stdout: cephadm noarch 2:19.3.0-4854.g8f219c47.el9 ceph-noarch 783 k 2024-09-10T23:52:50.229 INFO:teuthology.orchestra.run.smithi012.stdout: cephfs-mirror x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 225 k 2024-09-10T23:52:50.229 INFO:teuthology.orchestra.run.smithi012.stdout: cephfs-shell x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 27 k 2024-09-10T23:52:50.229 INFO:teuthology.orchestra.run.smithi012.stdout: cephfs-top noarch 2:19.3.0-4854.g8f219c47.el9 ceph-noarch 18 k 2024-09-10T23:52:50.229 INFO:teuthology.orchestra.run.smithi012.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-09-10T23:52:50.229 INFO:teuthology.orchestra.run.smithi012.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-09-10T23:52:50.230 INFO:teuthology.orchestra.run.smithi012.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-09-10T23:52:50.230 INFO:teuthology.orchestra.run.smithi012.stdout: libcephfs-devel x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 32 k 2024-09-10T23:52:50.230 INFO:teuthology.orchestra.run.smithi012.stdout: libcephfs2 x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 755 k 2024-09-10T23:52:50.230 INFO:teuthology.orchestra.run.smithi012.stdout: librados-devel x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 127 k 2024-09-10T23:52:50.230 INFO:teuthology.orchestra.run.smithi012.stdout: libtool x86_64 2.4.6-46.el9 appstream 581 k 2024-09-10T23:52:50.230 INFO:teuthology.orchestra.run.smithi012.stdout: libuuid-devel x86_64 2.37.4-20.el9 appstream 24 k 2024-09-10T23:52:50.230 INFO:teuthology.orchestra.run.smithi012.stdout: openssl-devel x86_64 1:3.2.2-6.el9 appstream 4.4 M 2024-09-10T23:52:50.230 INFO:teuthology.orchestra.run.smithi012.stdout: postgresql x86_64 13.16-1.el9 appstream 1.6 M 2024-09-10T23:52:50.230 INFO:teuthology.orchestra.run.smithi012.stdout: postgresql-contrib x86_64 13.16-1.el9 appstream 910 k 2024-09-10T23:52:50.230 INFO:teuthology.orchestra.run.smithi012.stdout: postgresql-server x86_64 13.16-1.el9 appstream 5.8 M 2024-09-10T23:52:50.230 INFO:teuthology.orchestra.run.smithi012.stdout: python3-cephfs x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 161 k 2024-09-10T23:52:50.230 INFO:teuthology.orchestra.run.smithi012.stdout: python3-rados x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 321 k 2024-09-10T23:52:50.230 INFO:teuthology.orchestra.run.smithi012.stdout: python3-rbd x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 306 k 2024-09-10T23:52:50.231 INFO:teuthology.orchestra.run.smithi012.stdout: python3-rgw x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 99 k 2024-09-10T23:52:50.231 INFO:teuthology.orchestra.run.smithi012.stdout: rbd-fuse x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 85 k 2024-09-10T23:52:50.231 INFO:teuthology.orchestra.run.smithi012.stdout: rbd-mirror x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 3.2 M 2024-09-10T23:52:50.231 INFO:teuthology.orchestra.run.smithi012.stdout: rbd-nbd x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 171 k 2024-09-10T23:52:50.231 INFO:teuthology.orchestra.run.smithi012.stdout: xfsprogs-devel x86_64 6.4.0-4.el9 appstream 96 k 2024-09-10T23:52:50.231 INFO:teuthology.orchestra.run.smithi012.stdout:Upgrading: 2024-09-10T23:52:50.231 INFO:teuthology.orchestra.run.smithi012.stdout: NetworkManager x86_64 1:1.49.90-1.el9 baseos 2.3 M 2024-09-10T23:52:50.231 INFO:teuthology.orchestra.run.smithi012.stdout: NetworkManager-libnm x86_64 1:1.49.90-1.el9 baseos 1.8 M 2024-09-10T23:52:50.231 INFO:teuthology.orchestra.run.smithi012.stdout: NetworkManager-team x86_64 1:1.49.90-1.el9 baseos 41 k 2024-09-10T23:52:50.231 INFO:teuthology.orchestra.run.smithi012.stdout: NetworkManager-tui x86_64 1:1.49.90-1.el9 baseos 249 k 2024-09-10T23:52:50.231 INFO:teuthology.orchestra.run.smithi012.stdout: librados2 x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 3.5 M 2024-09-10T23:52:50.231 INFO:teuthology.orchestra.run.smithi012.stdout: librbd1 x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 3.2 M 2024-09-10T23:52:50.232 INFO:teuthology.orchestra.run.smithi012.stdout: openssl x86_64 1:3.2.2-6.el9 baseos 1.4 M 2024-09-10T23:52:50.232 INFO:teuthology.orchestra.run.smithi012.stdout: openssl-libs x86_64 1:3.2.2-6.el9 baseos 2.4 M 2024-09-10T23:52:50.232 INFO:teuthology.orchestra.run.smithi012.stdout:Installing dependencies: 2024-09-10T23:52:50.232 INFO:teuthology.orchestra.run.smithi012.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-10T23:52:50.232 INFO:teuthology.orchestra.run.smithi012.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-09-10T23:52:50.232 INFO:teuthology.orchestra.run.smithi012.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-10T23:52:50.232 INFO:teuthology.orchestra.run.smithi012.stdout: ceph-common x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 22 M 2024-09-10T23:52:50.232 INFO:teuthology.orchestra.run.smithi012.stdout: ceph-grafana-dashboards noarch 2:19.3.0-4854.g8f219c47.el9 ceph-noarch 34 k 2024-09-10T23:52:50.232 INFO:teuthology.orchestra.run.smithi012.stdout: ceph-mds x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 2.4 M 2024-09-10T23:52:50.232 INFO:teuthology.orchestra.run.smithi012.stdout: ceph-mgr-modules-core noarch 2:19.3.0-4854.g8f219c47.el9 ceph-noarch 291 k 2024-09-10T23:52:50.232 INFO:teuthology.orchestra.run.smithi012.stdout: ceph-mon x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 4.8 M 2024-09-10T23:52:50.232 INFO:teuthology.orchestra.run.smithi012.stdout: ceph-osd x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 17 M 2024-09-10T23:52:50.233 INFO:teuthology.orchestra.run.smithi012.stdout: ceph-prometheus-alerts noarch 2:19.3.0-4854.g8f219c47.el9 ceph-noarch 16 k 2024-09-10T23:52:50.233 INFO:teuthology.orchestra.run.smithi012.stdout: ceph-selinux x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 25 k 2024-09-10T23:52:50.233 INFO:teuthology.orchestra.run.smithi012.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-10T23:52:50.233 INFO:teuthology.orchestra.run.smithi012.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-10T23:52:50.233 INFO:teuthology.orchestra.run.smithi012.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-10T23:52:50.233 INFO:teuthology.orchestra.run.smithi012.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-10T23:52:50.233 INFO:teuthology.orchestra.run.smithi012.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-09-10T23:52:50.233 INFO:teuthology.orchestra.run.smithi012.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-10T23:52:50.233 INFO:teuthology.orchestra.run.smithi012.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-10T23:52:50.233 INFO:teuthology.orchestra.run.smithi012.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-10T23:52:50.233 INFO:teuthology.orchestra.run.smithi012.stdout: libcephsqlite x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 162 k 2024-09-10T23:52:50.233 INFO:teuthology.orchestra.run.smithi012.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-10T23:52:50.234 INFO:teuthology.orchestra.run.smithi012.stdout: libnbd x86_64 1.20.2-2.el9 appstream 171 k 2024-09-10T23:52:50.234 INFO:teuthology.orchestra.run.smithi012.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-10T23:52:50.234 INFO:teuthology.orchestra.run.smithi012.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-10T23:52:50.234 INFO:teuthology.orchestra.run.smithi012.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-10T23:52:50.234 INFO:teuthology.orchestra.run.smithi012.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-10T23:52:50.234 INFO:teuthology.orchestra.run.smithi012.stdout: libradosstriper1 x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 507 k 2024-09-10T23:52:50.234 INFO:teuthology.orchestra.run.smithi012.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-10T23:52:50.234 INFO:teuthology.orchestra.run.smithi012.stdout: librgw2 x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 5.5 M 2024-09-10T23:52:50.234 INFO:teuthology.orchestra.run.smithi012.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-10T23:52:50.234 INFO:teuthology.orchestra.run.smithi012.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-09-10T23:52:50.234 INFO:teuthology.orchestra.run.smithi012.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-10T23:52:50.234 INFO:teuthology.orchestra.run.smithi012.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-10T23:52:50.234 INFO:teuthology.orchestra.run.smithi012.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-10T23:52:50.235 INFO:teuthology.orchestra.run.smithi012.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-10T23:52:50.235 INFO:teuthology.orchestra.run.smithi012.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-10T23:52:50.235 INFO:teuthology.orchestra.run.smithi012.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-10T23:52:50.235 INFO:teuthology.orchestra.run.smithi012.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-10T23:52:50.235 INFO:teuthology.orchestra.run.smithi012.stdout: postgresql-private-libs x86_64 13.16-1.el9 appstream 137 k 2024-09-10T23:52:50.235 INFO:teuthology.orchestra.run.smithi012.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-10T23:52:50.235 INFO:teuthology.orchestra.run.smithi012.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-10T23:52:50.235 INFO:teuthology.orchestra.run.smithi012.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-10T23:52:50.235 INFO:teuthology.orchestra.run.smithi012.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-09-10T23:52:50.235 INFO:teuthology.orchestra.run.smithi012.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-10T23:52:50.235 INFO:teuthology.orchestra.run.smithi012.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-10T23:52:50.236 INFO:teuthology.orchestra.run.smithi012.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-10T23:52:50.236 INFO:teuthology.orchestra.run.smithi012.stdout: python3-ceph-argparse x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 45 k 2024-09-10T23:52:50.236 INFO:teuthology.orchestra.run.smithi012.stdout: python3-ceph-common x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 143 k 2024-09-10T23:52:50.236 INFO:teuthology.orchestra.run.smithi012.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-10T23:52:50.236 INFO:teuthology.orchestra.run.smithi012.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-10T23:52:50.236 INFO:teuthology.orchestra.run.smithi012.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-10T23:52:50.236 INFO:teuthology.orchestra.run.smithi012.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-10T23:52:50.236 INFO:teuthology.orchestra.run.smithi012.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-09-10T23:52:50.237 INFO:teuthology.orchestra.run.smithi012.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-09-10T23:52:50.237 INFO:teuthology.orchestra.run.smithi012.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-10T23:52:50.237 INFO:teuthology.orchestra.run.smithi012.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-10T23:52:50.237 INFO:teuthology.orchestra.run.smithi012.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-10T23:52:50.237 INFO:teuthology.orchestra.run.smithi012.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-10T23:52:50.237 INFO:teuthology.orchestra.run.smithi012.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-10T23:52:50.237 INFO:teuthology.orchestra.run.smithi012.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-10T23:52:50.237 INFO:teuthology.orchestra.run.smithi012.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-10T23:52:50.237 INFO:teuthology.orchestra.run.smithi012.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-10T23:52:50.237 INFO:teuthology.orchestra.run.smithi012.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-10T23:52:50.237 INFO:teuthology.orchestra.run.smithi012.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-10T23:52:50.237 INFO:teuthology.orchestra.run.smithi012.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-10T23:52:50.237 INFO:teuthology.orchestra.run.smithi012.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-10T23:52:50.238 INFO:teuthology.orchestra.run.smithi012.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-10T23:52:50.238 INFO:teuthology.orchestra.run.smithi012.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-10T23:52:50.238 INFO:teuthology.orchestra.run.smithi012.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-10T23:52:50.238 INFO:teuthology.orchestra.run.smithi012.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-10T23:52:50.238 INFO:teuthology.orchestra.run.smithi012.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-10T23:52:50.238 INFO:teuthology.orchestra.run.smithi012.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-10T23:52:50.238 INFO:teuthology.orchestra.run.smithi012.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-10T23:52:50.238 INFO:teuthology.orchestra.run.smithi012.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-10T23:52:50.238 INFO:teuthology.orchestra.run.smithi012.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-10T23:52:50.238 INFO:teuthology.orchestra.run.smithi012.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-10T23:52:50.238 INFO:teuthology.orchestra.run.smithi012.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-10T23:52:50.238 INFO:teuthology.orchestra.run.smithi012.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-10T23:52:50.238 INFO:teuthology.orchestra.run.smithi012.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-10T23:52:50.239 INFO:teuthology.orchestra.run.smithi012.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-10T23:52:50.239 INFO:teuthology.orchestra.run.smithi012.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-10T23:52:50.239 INFO:teuthology.orchestra.run.smithi012.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-10T23:52:50.239 INFO:teuthology.orchestra.run.smithi012.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-10T23:52:50.239 INFO:teuthology.orchestra.run.smithi012.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-10T23:52:50.239 INFO:teuthology.orchestra.run.smithi012.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-10T23:52:50.239 INFO:teuthology.orchestra.run.smithi012.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-10T23:52:50.239 INFO:teuthology.orchestra.run.smithi012.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-09-10T23:52:50.239 INFO:teuthology.orchestra.run.smithi012.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-10T23:52:50.239 INFO:teuthology.orchestra.run.smithi012.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-10T23:52:50.239 INFO:teuthology.orchestra.run.smithi012.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-10T23:52:50.239 INFO:teuthology.orchestra.run.smithi012.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-10T23:52:50.239 INFO:teuthology.orchestra.run.smithi012.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-10T23:52:50.240 INFO:teuthology.orchestra.run.smithi012.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-10T23:52:50.240 INFO:teuthology.orchestra.run.smithi012.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-10T23:52:50.240 INFO:teuthology.orchestra.run.smithi012.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-10T23:52:50.240 INFO:teuthology.orchestra.run.smithi012.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-10T23:52:50.240 INFO:teuthology.orchestra.run.smithi012.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-10T23:52:50.240 INFO:teuthology.orchestra.run.smithi012.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-10T23:52:50.240 INFO:teuthology.orchestra.run.smithi012.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-09-10T23:52:50.240 INFO:teuthology.orchestra.run.smithi012.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-10T23:52:50.240 INFO:teuthology.orchestra.run.smithi012.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-10T23:52:50.240 INFO:teuthology.orchestra.run.smithi012.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-10T23:52:50.240 INFO:teuthology.orchestra.run.smithi012.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-10T23:52:50.240 INFO:teuthology.orchestra.run.smithi012.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-10T23:52:50.240 INFO:teuthology.orchestra.run.smithi012.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-10T23:52:50.241 INFO:teuthology.orchestra.run.smithi012.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-10T23:52:50.241 INFO:teuthology.orchestra.run.smithi012.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-10T23:52:50.241 INFO:teuthology.orchestra.run.smithi012.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-10T23:52:50.241 INFO:teuthology.orchestra.run.smithi012.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-09-10T23:52:50.241 INFO:teuthology.orchestra.run.smithi012.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-10T23:52:50.241 INFO:teuthology.orchestra.run.smithi012.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-09-10T23:52:50.241 INFO:teuthology.orchestra.run.smithi012.stdout:Installing weak dependencies: 2024-09-10T23:52:50.241 INFO:teuthology.orchestra.run.smithi012.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-10T23:52:50.241 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-10T23:52:50.241 INFO:teuthology.orchestra.run.smithi012.stdout:Transaction Summary 2024-09-10T23:52:50.241 INFO:teuthology.orchestra.run.smithi012.stdout:======================================================================================= 2024-09-10T23:52:50.241 INFO:teuthology.orchestra.run.smithi012.stdout:Install 149 Packages 2024-09-10T23:52:50.242 INFO:teuthology.orchestra.run.smithi012.stdout:Upgrade 8 Packages 2024-09-10T23:52:50.242 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-10T23:52:50.242 INFO:teuthology.orchestra.run.smithi012.stdout:Total download size: 240 M 2024-09-10T23:52:50.242 INFO:teuthology.orchestra.run.smithi012.stdout:Downloading Packages: 2024-09-10T23:52:50.248 INFO:teuthology.orchestra.run.smithi178.stdout:(97/157): qatlib-24.02.0-1.el9.x86_64.rpm 2.1 MB/s | 221 kB 00:00 2024-09-10T23:52:50.273 INFO:teuthology.orchestra.run.smithi178.stdout:(98/157): qatzip-libs-1.2.0-1.el9.x86_64.rpm 947 kB/s | 47 kB 00:00 2024-09-10T23:52:50.324 INFO:teuthology.orchestra.run.smithi178.stdout:(99/157): uuid-1.6.2-55.el9.x86_64.rpm 1.1 MB/s | 58 kB 00:00 2024-09-10T23:52:50.348 INFO:teuthology.orchestra.run.smithi165.stdout:(72/157): protobuf-3.14.0-13.el9.x86_64.rpm 4.7 MB/s | 1.0 MB 00:00 2024-09-10T23:52:50.357 INFO:teuthology.orchestra.run.smithi178.stdout:(100/157): socat-1.7.4.1-6.el9.x86_64.rpm 2.7 MB/s | 304 kB 00:00 2024-09-10T23:52:50.382 INFO:teuthology.orchestra.run.smithi178.stdout:(101/157): xfsprogs-devel-6.4.0-4.el9.x86_64.rp 1.6 MB/s | 96 kB 00:00 2024-09-10T23:52:50.407 INFO:teuthology.orchestra.run.smithi178.stdout:(102/157): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.2 MB/s | 64 kB 00:00 2024-09-10T23:52:50.433 INFO:teuthology.orchestra.run.smithi178.stdout:(103/157): zlib-devel-1.2.11-41.el9.x86_64.rpm 894 kB/s | 45 kB 00:00 2024-09-10T23:52:50.458 INFO:teuthology.orchestra.run.smithi178.stdout:(104/157): lua-devel-5.4.4-4.el9.x86_64.rpm 444 kB/s | 22 kB 00:00 2024-09-10T23:52:50.465 INFO:teuthology.orchestra.run.smithi165.stdout:(73/157): python3-attrs-20.3.0-7.el9.noarch.rpm 750 kB/s | 87 kB 00:00 2024-09-10T23:52:50.508 INFO:teuthology.orchestra.run.smithi178.stdout:(105/157): abseil-cpp-20211102.0-3.el9.x86_64.r 11 MB/s | 548 kB 00:00 2024-09-10T23:52:50.534 INFO:teuthology.orchestra.run.smithi178.stdout:(106/157): gperftools-libs-2.9.1-2.el9.x86_64.r 12 MB/s | 309 kB 00:00 2024-09-10T23:52:50.559 INFO:teuthology.orchestra.run.smithi178.stdout:(107/157): grpc-data-1.46.7-10.el9.noarch.rpm 775 kB/s | 19 kB 00:00 2024-09-10T23:52:50.684 INFO:teuthology.orchestra.run.smithi178.stdout:(108/157): libarrow-9.0.0-11.el9.x86_64.rpm 35 MB/s | 4.4 MB 00:00 2024-09-10T23:52:50.718 INFO:teuthology.orchestra.run.smithi178.stdout:(109/157): protobuf-compiler-3.14.0-13.el9.x86_ 3.0 MB/s | 863 kB 00:00 2024-09-10T23:52:50.743 INFO:teuthology.orchestra.run.smithi178.stdout:(110/157): libarrow-doc-9.0.0-11.el9.noarch.rpm 444 kB/s | 26 kB 00:00 2024-09-10T23:52:50.777 INFO:teuthology.orchestra.run.smithi178.stdout:(111/157): libunwind-1.6.2-1.el9.x86_64.rpm 2.6 MB/s | 67 kB 00:00 2024-09-10T23:52:50.802 INFO:teuthology.orchestra.run.smithi178.stdout:(112/157): luarocks-3.9.2-5.el9.noarch.rpm 6.4 MB/s | 151 kB 00:00 2024-09-10T23:52:50.827 INFO:teuthology.orchestra.run.smithi178.stdout:(113/157): liboath-2.6.7-2.el9.x86_64.rpm 445 kB/s | 49 kB 00:00 2024-09-10T23:52:50.861 INFO:teuthology.orchestra.run.smithi178.stdout:(114/157): parquet-libs-9.0.0-11.el9.x86_64.rpm 14 MB/s | 839 kB 00:00 2024-09-10T23:52:50.886 INFO:teuthology.orchestra.run.smithi178.stdout:(115/157): python3-bcrypt-3.2.2-1.el9.x86_64.rp 1.7 MB/s | 43 kB 00:00 2024-09-10T23:52:50.911 INFO:teuthology.orchestra.run.smithi178.stdout:(116/157): python3-cachetools-4.2.4-1.el9.noarc 1.3 MB/s | 32 kB 00:00 2024-09-10T23:52:50.937 INFO:teuthology.orchestra.run.smithi178.stdout:(117/157): python3-certifi-2023.05.07-4.el9.noa 563 kB/s | 14 kB 00:00 2024-09-10T23:52:50.962 INFO:teuthology.orchestra.run.smithi178.stdout:(118/157): python3-asyncssh-2.13.2-5.el9.noarch 4.0 MB/s | 548 kB 00:00 2024-09-10T23:52:50.967 INFO:teuthology.orchestra.run.smithi012.stdout:(1/157): ceph-19.3.0-4854.g8f219c47.el9.x86_64. 50 kB/s | 6.6 kB 00:00 2024-09-10T23:52:50.987 INFO:teuthology.orchestra.run.smithi178.stdout:(119/157): python3-cheroot-8.6.0-4.el9.noarch.r 3.3 MB/s | 172 kB 00:00 2024-09-10T23:52:51.021 INFO:teuthology.orchestra.run.smithi178.stdout:(120/157): python3-cmd2-2.4.2-2.el9.noarch.rpm 8.9 MB/s | 303 kB 00:00 2024-09-10T23:52:51.054 INFO:teuthology.orchestra.run.smithi178.stdout:(121/157): python3-cherrypy-18.6.1-2.el9.noarch 3.8 MB/s | 358 kB 00:00 2024-09-10T23:52:51.080 INFO:teuthology.orchestra.run.smithi178.stdout:(122/157): python3-google-auth-2.34.0-1.el9.noa 3.7 MB/s | 220 kB 00:00 2024-09-10T23:52:51.105 INFO:teuthology.orchestra.run.smithi178.stdout:(123/157): python3-grpcio-tools-1.46.7-10.el9.x 5.7 MB/s | 144 kB 00:00 2024-09-10T23:52:51.130 INFO:teuthology.orchestra.run.smithi178.stdout:(124/157): python3-jaraco-8.2.1-3.el9.noarch.rp 429 kB/s | 11 kB 00:00 2024-09-10T23:52:51.134 INFO:teuthology.orchestra.run.smithi012.stdout:(2/157): ceph-fuse-19.3.0-4854.g8f219c47.el9.x8 5.3 MB/s | 900 kB 00:00 2024-09-10T23:52:51.174 INFO:teuthology.orchestra.run.smithi178.stdout:(125/157): python3-jaraco-classes-3.2.1-5.el9.n 415 kB/s | 18 kB 00:00 2024-09-10T23:52:51.206 INFO:teuthology.orchestra.run.smithi178.stdout:(126/157): python3-jaraco-collections-3.0.0-8.e 721 kB/s | 23 kB 00:00 2024-09-10T23:52:51.218 INFO:teuthology.orchestra.run.smithi012.stdout:(3/157): ceph-immutable-object-cache-19.3.0-485 1.7 MB/s | 145 kB 00:00 2024-09-10T23:52:51.231 INFO:teuthology.orchestra.run.smithi178.stdout:(127/157): python3-jaraco-functools-3.5.0-2.el9 779 kB/s | 19 kB 00:00 2024-09-10T23:52:51.256 INFO:teuthology.orchestra.run.smithi178.stdout:(128/157): python3-jaraco-text-3.2.0-6.el9.noar 787 kB/s | 20 kB 00:00 2024-09-10T23:52:51.307 INFO:teuthology.orchestra.run.smithi178.stdout:(129/157): python3-grpcio-1.46.7-10.el9.x86_64. 8.1 MB/s | 2.0 MB 00:00 2024-09-10T23:52:51.326 INFO:teuthology.orchestra.run.smithi012.stdout:(4/157): ceph-base-19.3.0-4854.g8f219c47.el9.x8 11 MB/s | 5.5 MB 00:00 2024-09-10T23:52:51.332 INFO:teuthology.orchestra.run.smithi178.stdout:(130/157): python3-logutils-0.3.5-21.el9.noarch 1.8 MB/s | 46 kB 00:00 2024-09-10T23:52:51.349 INFO:teuthology.orchestra.run.smithi178.stdout:(131/157): python3-more-itertools-8.12.0-2.el9. 4.7 MB/s | 79 kB 00:00 2024-09-10T23:52:51.375 INFO:teuthology.orchestra.run.smithi178.stdout:(132/157): python3-natsort-7.1.1-5.el9.noarch.r 2.3 MB/s | 58 kB 00:00 2024-09-10T23:52:51.408 INFO:teuthology.orchestra.run.smithi178.stdout:(133/157): python3-kubernetes-26.1.0-2.el9.noar 6.8 MB/s | 1.0 MB 00:00 2024-09-10T23:52:51.433 INFO:teuthology.orchestra.run.smithi178.stdout:(134/157): python3-portend-3.1.0-2.el9.noarch.r 658 kB/s | 16 kB 00:00 2024-09-10T23:52:51.452 INFO:teuthology.orchestra.run.smithi012.stdout:(5/157): ceph-mds-19.3.0-4854.g8f219c47.el9.x86 10 MB/s | 2.4 MB 00:00 2024-09-10T23:52:51.458 INFO:teuthology.orchestra.run.smithi178.stdout:(135/157): python3-pecan-1.4.2-3.el9.noarch.rpm 3.2 MB/s | 272 kB 00:00 2024-09-10T23:52:51.484 INFO:teuthology.orchestra.run.smithi178.stdout:(136/157): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.7 MB/s | 90 kB 00:00 2024-09-10T23:52:51.485 INFO:teuthology.orchestra.run.smithi012.stdout:(6/157): ceph-mgr-19.3.0-4854.g8f219c47.el9.x86 9.7 MB/s | 1.5 MB 00:00 2024-09-10T23:52:51.509 INFO:teuthology.orchestra.run.smithi178.stdout:(137/157): python3-pyperclip-1.8.2-2.el9.noarch 560 kB/s | 28 kB 00:00 2024-09-10T23:52:51.535 INFO:teuthology.orchestra.run.smithi178.stdout:(138/157): python3-repoze-lru-0.7-16.el9.noarch 608 kB/s | 31 kB 00:00 2024-09-10T23:52:51.560 INFO:teuthology.orchestra.run.smithi178.stdout:(139/157): python3-rsa-4.9-2.el9.noarch.rpm 2.3 MB/s | 59 kB 00:00 2024-09-10T23:52:51.585 INFO:teuthology.orchestra.run.smithi178.stdout:(140/157): python3-routes-2.5.1-5.el9.noarch.rp 2.4 MB/s | 188 kB 00:00 2024-09-10T23:52:51.610 INFO:teuthology.orchestra.run.smithi178.stdout:(141/157): python3-tempora-5.0.0-2.el9.noarch.r 712 kB/s | 36 kB 00:00 2024-09-10T23:52:51.636 INFO:teuthology.orchestra.run.smithi178.stdout:(142/157): python3-typing-extensions-4.12.2-2.e 1.4 MB/s | 74 kB 00:00 2024-09-10T23:52:51.661 INFO:teuthology.orchestra.run.smithi178.stdout:(143/157): python3-websocket-client-1.2.3-2.el9 3.5 MB/s | 90 kB 00:00 2024-09-10T23:52:51.686 INFO:teuthology.orchestra.run.smithi178.stdout:(144/157): python3-webob-1.8.8-2.el9.noarch.rpm 3.0 MB/s | 230 kB 00:00 2024-09-10T23:52:51.703 INFO:teuthology.orchestra.run.smithi178.stdout:(145/157): python3-zc-lockfile-2.0-10.el9.noarc 1.2 MB/s | 20 kB 00:00 2024-09-10T23:52:51.729 INFO:teuthology.orchestra.run.smithi178.stdout:(146/157): python3-werkzeug-2.0.3-3.el9.1.noarc 6.2 MB/s | 427 kB 00:00 2024-09-10T23:52:51.754 INFO:teuthology.orchestra.run.smithi178.stdout:(147/157): re2-20211101-20.el9.x86_64.rpm 3.7 MB/s | 191 kB 00:00 2024-09-10T23:52:51.812 INFO:teuthology.orchestra.run.smithi178.stdout:(148/157): thrift-0.15.0-2.el9.x86_64.rpm 19 MB/s | 1.6 MB 00:00 2024-09-10T23:52:51.877 INFO:teuthology.orchestra.run.smithi012.stdout:(7/157): ceph-mon-19.3.0-4854.g8f219c47.el9.x86 11 MB/s | 4.8 MB 00:00 2024-09-10T23:52:52.186 INFO:teuthology.orchestra.run.smithi012.stdout:(8/157): ceph-common-19.3.0-4854.g8f219c47.el9. 16 MB/s | 22 MB 00:01 2024-09-10T23:52:52.204 INFO:teuthology.orchestra.run.smithi178.stdout:(149/157): librados2-19.3.0-4854.g8f219c47.el9. 7.7 MB/s | 3.5 MB 00:00 2024-09-10T23:52:52.228 INFO:teuthology.orchestra.run.smithi012.stdout:(9/157): ceph-selinux-19.3.0-4854.g8f219c47.el9 612 kB/s | 25 kB 00:00 2024-09-10T23:52:52.255 INFO:teuthology.orchestra.run.smithi178.stdout:(150/157): librbd1-19.3.0-4854.g8f219c47.el9.x8 7.3 MB/s | 3.2 MB 00:00 2024-09-10T23:52:52.553 INFO:teuthology.orchestra.run.smithi012.stdout:(10/157): ceph-osd-19.3.0-4854.g8f219c47.el9.x8 16 MB/s | 17 MB 00:01 2024-09-10T23:52:52.603 INFO:teuthology.orchestra.run.smithi012.stdout:(11/157): cephfs-mirror-19.3.0-4854.g8f219c47.e 4.4 MB/s | 225 kB 00:00 2024-09-10T23:52:52.680 INFO:teuthology.orchestra.run.smithi178.stdout:(151/157): NetworkManager-libnm-1.49.90-1.el9.x 4.2 MB/s | 1.8 MB 00:00 2024-09-10T23:52:52.722 INFO:teuthology.orchestra.run.smithi178.stdout:(152/157): NetworkManager-team-1.49.90-1.el9.x8 983 kB/s | 41 kB 00:00 2024-09-10T23:52:52.753 INFO:teuthology.orchestra.run.smithi012.stdout:(12/157): ceph-radosgw-19.3.0-4854.g8f219c47.el 12 MB/s | 11 MB 00:00 2024-09-10T23:52:52.807 INFO:teuthology.orchestra.run.smithi165.stdout:(74/157): openblas-openmp-0.3.26-2.el9.x86_64.r 1.1 MB/s | 4.9 MB 00:04 2024-09-10T23:52:52.814 INFO:teuthology.orchestra.run.smithi178.stdout:(153/157): NetworkManager-1.49.90-1.el9.x86_64. 3.8 MB/s | 2.3 MB 00:00 2024-09-10T23:52:52.837 INFO:teuthology.orchestra.run.smithi012.stdout:(13/157): cephfs-shell-19.3.0-4854.g8f219c47.el 116 kB/s | 27 kB 00:00 2024-09-10T23:52:52.847 INFO:teuthology.orchestra.run.smithi178.stdout:(154/157): NetworkManager-tui-1.49.90-1.el9.x86 1.9 MB/s | 249 kB 00:00 2024-09-10T23:52:52.854 INFO:teuthology.orchestra.run.smithi012.stdout:(14/157): libcephfs-devel-19.3.0-4854.g8f219c47 318 kB/s | 32 kB 00:00 2024-09-10T23:52:52.904 INFO:teuthology.orchestra.run.smithi012.stdout:(15/157): libcephfs2-19.3.0-4854.g8f219c47.el9. 11 MB/s | 755 kB 00:00 2024-09-10T23:52:52.915 INFO:teuthology.orchestra.run.smithi165.stdout:(75/157): python3-colorama-0.4.6-3.el9.noarch.r 566 kB/s | 61 kB 00:00 2024-09-10T23:52:52.929 INFO:teuthology.orchestra.run.smithi012.stdout:(16/157): libcephsqlite-19.3.0-4854.g8f219c47.e 2.1 MB/s | 162 kB 00:00 2024-09-10T23:52:52.955 INFO:teuthology.orchestra.run.smithi012.stdout:(17/157): librados-devel-19.3.0-4854.g8f219c47. 2.5 MB/s | 127 kB 00:00 2024-09-10T23:52:52.988 INFO:teuthology.orchestra.run.smithi012.stdout:(18/157): libradosstriper1-19.3.0-4854.g8f219c4 8.5 MB/s | 507 kB 00:00 2024-09-10T23:52:53.030 INFO:teuthology.orchestra.run.smithi012.stdout:(19/157): python3-ceph-argparse-19.3.0-4854.g8f 1.1 MB/s | 45 kB 00:00 2024-09-10T23:52:53.089 INFO:teuthology.orchestra.run.smithi012.stdout:(20/157): python3-ceph-common-19.3.0-4854.g8f21 2.4 MB/s | 143 kB 00:00 2024-09-10T23:52:53.156 INFO:teuthology.orchestra.run.smithi012.stdout:(21/157): python3-cephfs-19.3.0-4854.g8f219c47. 2.4 MB/s | 161 kB 00:00 2024-09-10T23:52:53.214 INFO:teuthology.orchestra.run.smithi178.stdout:(155/157): openssl-3.2.2-6.el9.x86_64.rpm 3.4 MB/s | 1.4 MB 00:00 2024-09-10T23:52:53.265 INFO:teuthology.orchestra.run.smithi012.stdout:(22/157): librgw2-19.3.0-4854.g8f219c47.el9.x86 18 MB/s | 5.5 MB 00:00 2024-09-10T23:52:53.290 INFO:teuthology.orchestra.run.smithi165.stdout:(76/157): python3-devel-3.9.19-8.el9.x86_64.rpm 653 kB/s | 245 kB 00:00 2024-09-10T23:52:53.298 INFO:teuthology.orchestra.run.smithi012.stdout:(23/157): python3-rados-19.3.0-4854.g8f219c47.e 2.2 MB/s | 321 kB 00:00 2024-09-10T23:52:53.323 INFO:teuthology.orchestra.run.smithi012.stdout:(24/157): python3-rbd-19.3.0-4854.g8f219c47.el9 5.1 MB/s | 306 kB 00:00 2024-09-10T23:52:53.349 INFO:teuthology.orchestra.run.smithi012.stdout:(25/157): python3-rgw-19.3.0-4854.g8f219c47.el9 1.9 MB/s | 99 kB 00:00 2024-09-10T23:52:53.376 INFO:teuthology.orchestra.run.smithi012.stdout:(26/157): rbd-fuse-19.3.0-4854.g8f219c47.el9.x8 1.7 MB/s | 85 kB 00:00 2024-09-10T23:52:53.424 INFO:teuthology.orchestra.run.smithi012.stdout:(27/157): rbd-nbd-19.3.0-4854.g8f219c47.el9.x86 3.4 MB/s | 171 kB 00:00 2024-09-10T23:52:53.483 INFO:teuthology.orchestra.run.smithi012.stdout:(28/157): ceph-grafana-dashboards-19.3.0-4854.g 588 kB/s | 34 kB 00:00 2024-09-10T23:52:53.515 INFO:teuthology.orchestra.run.smithi178.stdout:(156/157): openssl-libs-3.2.2-6.el9.x86_64.rpm 3.6 MB/s | 2.4 MB 00:00 2024-09-10T23:52:53.570 INFO:teuthology.orchestra.run.smithi012.stdout:(29/157): rbd-mirror-19.3.0-4854.g8f219c47.el9. 14 MB/s | 3.2 MB 00:00 2024-09-10T23:52:53.592 INFO:teuthology.orchestra.run.smithi012.stdout:(30/157): ceph-mgr-cephadm-19.3.0-4854.g8f219c4 1.4 MB/s | 154 kB 00:00 2024-09-10T23:52:53.666 INFO:teuthology.orchestra.run.smithi165.stdout:(77/157): python3-jinja2-2.11.3-6.el9.noarch.rp 662 kB/s | 248 kB 00:00 2024-09-10T23:52:53.766 INFO:teuthology.orchestra.run.smithi165.stdout:(78/157): python3-jsonpatch-1.21-16.el9.noarch. 256 kB/s | 26 kB 00:00 2024-09-10T23:52:53.891 INFO:teuthology.orchestra.run.smithi165.stdout:(79/157): python3-jsonpointer-2.0-4.el9.noarch. 153 kB/s | 19 kB 00:00 2024-09-10T23:52:53.915 INFO:teuthology.orchestra.run.smithi178.stdout:(157/157): python3-scipy-1.6.2-8.el9.x86_64.rpm 3.7 MB/s | 16 MB 00:04 2024-09-10T23:52:53.920 INFO:teuthology.orchestra.run.smithi178.stdout:-------------------------------------------------------------------------------- 2024-09-10T23:52:53.920 INFO:teuthology.orchestra.run.smithi178.stdout:Total 16 MB/s | 240 MB 00:14 2024-09-10T23:52:54.084 INFO:teuthology.orchestra.run.smithi012.stdout:(31/157): ceph-mgr-diskprediction-local-19.3.0- 15 MB/s | 7.4 MB 00:00 2024-09-10T23:52:54.151 INFO:teuthology.orchestra.run.smithi012.stdout:(32/157): ceph-mgr-modules-core-19.3.0-4854.g8f 4.3 MB/s | 291 kB 00:00 2024-09-10T23:52:54.183 INFO:teuthology.orchestra.run.smithi165.stdout:(80/157): python3-mako-1.1.4-6.el9.noarch.rpm 590 kB/s | 172 kB 00:00 2024-09-10T23:52:54.209 INFO:teuthology.orchestra.run.smithi012.stdout:(33/157): ceph-mgr-rook-19.3.0-4854.g8f219c47.e 848 kB/s | 49 kB 00:00 2024-09-10T23:52:54.268 INFO:teuthology.orchestra.run.smithi012.stdout:(34/157): ceph-prometheus-alerts-19.3.0-4854.g8 282 kB/s | 16 kB 00:00 2024-09-10T23:52:54.292 INFO:teuthology.orchestra.run.smithi165.stdout:(81/157): python3-markupsafe-1.1.1-12.el9.x86_6 320 kB/s | 35 kB 00:00 2024-09-10T23:52:54.465 INFO:teuthology.orchestra.run.smithi012.stdout:(35/157): ceph-mgr-dashboard-19.3.0-4854.g8f219 16 MB/s | 14 MB 00:00 2024-09-10T23:52:54.493 INFO:teuthology.orchestra.run.smithi012.stdout:(36/157): ceph-volume-19.3.0-4854.g8f219c47.el9 1.2 MB/s | 285 kB 00:00 2024-09-10T23:52:54.544 INFO:teuthology.orchestra.run.smithi012.stdout:(37/157): cephadm-19.3.0-4854.g8f219c47.el9.noa 9.7 MB/s | 783 kB 00:00 2024-09-10T23:52:54.561 INFO:teuthology.orchestra.run.smithi012.stdout:(38/157): cephfs-top-19.3.0-4854.g8f219c47.el9. 272 kB/s | 18 kB 00:00 2024-09-10T23:52:54.634 INFO:teuthology.orchestra.run.smithi165.stdout:(82/157): postgresql-server-13.16-1.el9.x86_64. 1.0 MB/s | 5.8 MB 00:05 2024-09-10T23:52:55.017 INFO:teuthology.orchestra.run.smithi165.stdout:(83/157): python3-numpy-f2py-1.20.1-5.el9.x86_6 654 kB/s | 250 kB 00:00 2024-09-10T23:52:55.153 INFO:teuthology.orchestra.run.smithi012.stdout:(39/157): ceph-test-19.3.0-4854.g8f219c47.el9.x 17 MB/s | 50 MB 00:02 2024-09-10T23:52:55.158 INFO:teuthology.orchestra.run.smithi178.stdout:Running transaction check 2024-09-10T23:52:55.310 INFO:teuthology.orchestra.run.smithi178.stdout:Transaction check succeeded. 2024-09-10T23:52:55.310 INFO:teuthology.orchestra.run.smithi178.stdout:Running transaction test 2024-09-10T23:52:55.393 INFO:teuthology.orchestra.run.smithi165.stdout:(84/157): python3-oauthlib-3.1.1-5.el9.noarch.r 592 kB/s | 222 kB 00:00 2024-09-10T23:52:55.610 INFO:teuthology.orchestra.run.smithi165.stdout:(85/157): python3-packaging-20.9-5.el9.noarch.r 356 kB/s | 77 kB 00:00 2024-09-10T23:52:55.645 INFO:teuthology.orchestra.run.smithi012.stdout:(40/157): fuse-2.9.9-16.el9.x86_64.rpm 72 kB/s | 80 kB 00:01 2024-09-10T23:52:55.719 INFO:teuthology.orchestra.run.smithi165.stdout:(86/157): python3-prettytable-0.7.2-27.el9.noar 390 kB/s | 42 kB 00:00 2024-09-10T23:52:55.912 INFO:teuthology.orchestra.run.smithi012.stdout:(41/157): libgfortran-11.5.0-2.el9.x86_64.rpm 593 kB/s | 801 kB 00:01 2024-09-10T23:52:55.937 INFO:teuthology.orchestra.run.smithi012.stdout:(42/157): libquadmath-11.5.0-2.el9.x86_64.rpm 244 kB/s | 191 kB 00:00 2024-09-10T23:52:56.177 INFO:teuthology.orchestra.run.smithi165.stdout:(87/157): python3-protobuf-3.14.0-13.el9.noarch 588 kB/s | 269 kB 00:00 2024-09-10T23:52:56.286 INFO:teuthology.orchestra.run.smithi165.stdout:(88/157): python3-babel-2.9.1-2.el9.noarch.rpm 1.0 MB/s | 6.0 MB 00:05 2024-09-10T23:52:56.486 INFO:teuthology.orchestra.run.smithi165.stdout:(89/157): python3-pyasn1-0.4.8-6.el9.noarch.rpm 514 kB/s | 159 kB 00:00 2024-09-10T23:52:56.495 INFO:teuthology.orchestra.run.smithi012.stdout:(43/157): python3-cffi-1.14.5-5.el9.x86_64.rpm 298 kB/s | 253 kB 00:00 2024-09-10T23:52:56.611 INFO:teuthology.orchestra.run.smithi165.stdout:(90/157): python3-pytz-2021.1-5.el9.noarch.rpm 408 kB/s | 51 kB 00:00 2024-09-10T23:52:56.637 INFO:teuthology.orchestra.run.smithi012.stdout:(44/157): python3-ply-3.11-14.el9.noarch.rpm 152 kB/s | 106 kB 00:00 2024-09-10T23:52:56.720 INFO:teuthology.orchestra.run.smithi165.stdout:(91/157): python3-requests-oauthlib-1.3.0-12.el 495 kB/s | 54 kB 00:00 2024-09-10T23:52:56.745 INFO:teuthology.orchestra.run.smithi165.stdout:(92/157): python3-pyasn1-modules-0.4.8-6.el9.no 607 kB/s | 279 kB 00:00 2024-09-10T23:52:56.853 INFO:teuthology.orchestra.run.smithi165.stdout:(93/157): python3-toml-0.10.2-6.el9.noarch.rpm 384 kB/s | 42 kB 00:00 2024-09-10T23:52:56.962 INFO:teuthology.orchestra.run.smithi165.stdout:(94/157): python3-wcwidth-0.2.5-8.el9.noarch.rp 408 kB/s | 44 kB 00:00 2024-09-10T23:52:57.204 INFO:teuthology.orchestra.run.smithi012.stdout:(45/157): python3-pycparser-2.20-6.el9.noarch.r 191 kB/s | 135 kB 00:00 2024-09-10T23:52:57.255 INFO:teuthology.orchestra.run.smithi178.stdout:Transaction test succeeded. 2024-09-10T23:52:57.256 INFO:teuthology.orchestra.run.smithi178.stdout:Running transaction 2024-09-10T23:52:57.337 INFO:teuthology.orchestra.run.smithi165.stdout:(95/157): qatlib-24.02.0-1.el9.x86_64.rpm 588 kB/s | 221 kB 00:00 2024-09-10T23:52:57.346 INFO:teuthology.orchestra.run.smithi012.stdout:(46/157): python3-requests-2.25.1-8.el9.noarch. 176 kB/s | 125 kB 00:00 2024-09-10T23:52:57.455 INFO:teuthology.orchestra.run.smithi165.stdout:(96/157): qatlib-service-24.02.0-1.el9.x86_64.r 308 kB/s | 36 kB 00:00 2024-09-10T23:52:57.555 INFO:teuthology.orchestra.run.smithi012.stdout:(47/157): automake-1.16.2-8.el9.noarch.rpm 3.2 MB/s | 689 kB 00:00 2024-09-10T23:52:57.638 INFO:teuthology.orchestra.run.smithi012.stdout:(48/157): bison-3.7.4-5.el9.x86_64.rpm 11 MB/s | 943 kB 00:00 2024-09-10T23:52:57.672 INFO:teuthology.orchestra.run.smithi165.stdout:(97/157): qatzip-libs-1.2.0-1.el9.x86_64.rpm 218 kB/s | 47 kB 00:00 2024-09-10T23:52:57.689 INFO:teuthology.orchestra.run.smithi012.stdout:(49/157): boost-program-options-1.75.0-8.el9.x8 2.1 MB/s | 107 kB 00:00 2024-09-10T23:52:57.730 INFO:teuthology.orchestra.run.smithi012.stdout:(50/157): elfutils-libelf-devel-0.191-4.el9.x86 525 kB/s | 22 kB 00:00 2024-09-10T23:52:57.781 INFO:teuthology.orchestra.run.smithi012.stdout:(51/157): flex-2.6.4-9.el9.x86_64.rpm 6.1 MB/s | 313 kB 00:00 2024-09-10T23:52:57.823 INFO:teuthology.orchestra.run.smithi012.stdout:(52/157): flexiblas-3.0.4-8.el9.x86_64.rpm 785 kB/s | 33 kB 00:00 2024-09-10T23:52:57.965 INFO:teuthology.orchestra.run.smithi012.stdout:(53/157): flexiblas-netlib-3.0.4-8.el9.x86_64.r 21 MB/s | 3.0 MB 00:00 2024-09-10T23:52:58.007 INFO:teuthology.orchestra.run.smithi012.stdout:(54/157): python3-cryptography-36.0.1-4.el9.x86 599 kB/s | 1.2 MB 00:02 2024-09-10T23:52:58.051 INFO:teuthology.orchestra.run.smithi012.stdout:(55/157): flexiblas-openblas-openmp-3.0.4-8.el9 212 kB/s | 18 kB 00:00 2024-09-10T23:52:58.099 INFO:teuthology.orchestra.run.smithi012.stdout:(56/157): libaio-devel-0.3.111-13.el9.x86_64.rp 121 kB/s | 11 kB 00:00 2024-09-10T23:52:58.124 INFO:teuthology.orchestra.run.smithi012.stdout:(57/157): python3-urllib3-1.26.5-6.el9.noarch.r 234 kB/s | 215 kB 00:00 2024-09-10T23:52:58.138 INFO:teuthology.orchestra.run.smithi165.stdout:(98/157): socat-1.7.4.1-6.el9.x86_64.rpm 653 kB/s | 304 kB 00:00 2024-09-10T23:52:58.149 INFO:teuthology.orchestra.run.smithi012.stdout:(58/157): libnbd-1.20.2-2.el9.x86_64.rpm 1.7 MB/s | 171 kB 00:00 2024-09-10T23:52:58.200 INFO:teuthology.orchestra.run.smithi012.stdout:(59/157): librdkafka-1.6.1-102.el9.x86_64.rpm 13 MB/s | 662 kB 00:00 2024-09-10T23:52:58.225 INFO:teuthology.orchestra.run.smithi012.stdout:(60/157): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.3 MB/s | 160 kB 00:00 2024-09-10T23:52:58.247 INFO:teuthology.orchestra.run.smithi165.stdout:(99/157): uuid-1.6.2-55.el9.x86_64.rpm 534 kB/s | 58 kB 00:00 2024-09-10T23:52:58.250 INFO:teuthology.orchestra.run.smithi012.stdout:(61/157): librabbitmq-0.11.0-7.el9.x86_64.rpm 360 kB/s | 45 kB 00:00 2024-09-10T23:52:58.276 INFO:teuthology.orchestra.run.smithi012.stdout:(62/157): libuuid-devel-2.37.4-20.el9.x86_64.rp 490 kB/s | 24 kB 00:00 2024-09-10T23:52:58.309 INFO:teuthology.orchestra.run.smithi012.stdout:(63/157): libtool-2.4.6-46.el9.x86_64.rpm 5.2 MB/s | 581 kB 00:00 2024-09-10T23:52:58.334 INFO:teuthology.orchestra.run.smithi012.stdout:(64/157): libzstd-devel-1.5.1-2.el9.x86_64.rpm 568 kB/s | 47 kB 00:00 2024-09-10T23:52:58.360 INFO:teuthology.orchestra.run.smithi012.stdout:(65/157): lua-5.4.4-4.el9.x86_64.rpm 3.7 MB/s | 188 kB 00:00 2024-09-10T23:52:58.385 INFO:teuthology.orchestra.run.smithi012.stdout:(66/157): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.6 MB/s | 292 kB 00:00 2024-09-10T23:52:58.410 INFO:teuthology.orchestra.run.smithi012.stdout:(67/157): openblas-0.3.26-2.el9.x86_64.rpm 511 kB/s | 38 kB 00:00 2024-09-10T23:52:58.447 INFO:teuthology.orchestra.run.smithi165.stdout:(100/157): xfsprogs-devel-6.4.0-4.el9.x86_64.rp 479 kB/s | 96 kB 00:00 2024-09-10T23:52:58.519 INFO:teuthology.orchestra.run.smithi012.stdout:(68/157): openblas-openmp-0.3.26-2.el9.x86_64.r 31 MB/s | 4.9 MB 00:00 2024-09-10T23:52:58.611 INFO:teuthology.orchestra.run.smithi012.stdout:(69/157): openssl-devel-3.2.2-6.el9.x86_64.rpm 20 MB/s | 4.4 MB 00:00 2024-09-10T23:52:58.644 INFO:teuthology.orchestra.run.smithi012.stdout:(70/157): postgresql-contrib-13.16-1.el9.x86_64 7.1 MB/s | 910 kB 00:00 2024-09-10T23:52:58.647 INFO:teuthology.orchestra.run.smithi165.stdout:(101/157): xmlstarlet-1.6.1-20.el9.x86_64.rpm 318 kB/s | 64 kB 00:00 2024-09-10T23:52:58.669 INFO:teuthology.orchestra.run.smithi012.stdout:(71/157): postgresql-private-libs-13.16-1.el9.x 2.3 MB/s | 137 kB 00:00 2024-09-10T23:52:58.711 INFO:teuthology.orchestra.run.smithi012.stdout:(72/157): postgresql-13.16-1.el9.x86_64.rpm 5.3 MB/s | 1.6 MB 00:00 2024-09-10T23:52:58.756 INFO:teuthology.orchestra.run.smithi165.stdout:(102/157): zlib-devel-1.2.11-41.el9.x86_64.rpm 414 kB/s | 45 kB 00:00 2024-09-10T23:52:58.761 INFO:teuthology.orchestra.run.smithi012.stdout:(73/157): protobuf-3.14.0-13.el9.x86_64.rpm 11 MB/s | 1.0 MB 00:00 2024-09-10T23:52:58.787 INFO:teuthology.orchestra.run.smithi012.stdout:(74/157): python3-attrs-20.3.0-7.el9.noarch.rpm 1.1 MB/s | 87 kB 00:00 2024-09-10T23:52:58.829 INFO:teuthology.orchestra.run.smithi012.stdout:(75/157): python3-colorama-0.4.6-3.el9.noarch.r 1.4 MB/s | 61 kB 00:00 2024-09-10T23:52:58.873 INFO:teuthology.orchestra.run.smithi165.stdout:(103/157): lua-devel-5.4.4-4.el9.x86_64.rpm 191 kB/s | 22 kB 00:00 2024-09-10T23:52:58.954 INFO:teuthology.orchestra.run.smithi012.stdout:(76/157): postgresql-server-13.16-1.el9.x86_64. 19 MB/s | 5.8 MB 00:00 2024-09-10T23:52:58.988 INFO:teuthology.orchestra.run.smithi012.stdout:(77/157): python3-devel-3.9.19-8.el9.x86_64.rpm 1.5 MB/s | 245 kB 00:00 2024-09-10T23:52:59.013 INFO:teuthology.orchestra.run.smithi012.stdout:(78/157): python3-jinja2-2.11.3-6.el9.noarch.rp 4.1 MB/s | 248 kB 00:00 2024-09-10T23:52:59.038 INFO:teuthology.orchestra.run.smithi012.stdout:(79/157): python3-jsonpatch-1.21-16.el9.noarch. 512 kB/s | 26 kB 00:00 2024-09-10T23:52:59.063 INFO:teuthology.orchestra.run.smithi012.stdout:(80/157): python3-jsonpointer-2.0-4.el9.noarch. 384 kB/s | 19 kB 00:00 2024-09-10T23:52:59.089 INFO:teuthology.orchestra.run.smithi012.stdout:(81/157): python3-mako-1.1.4-6.el9.noarch.rpm 3.4 MB/s | 172 kB 00:00 2024-09-10T23:52:59.098 INFO:teuthology.orchestra.run.smithi165.stdout:(104/157): protobuf-compiler-3.14.0-13.el9.x86_ 3.7 MB/s | 863 kB 00:00 2024-09-10T23:52:59.114 INFO:teuthology.orchestra.run.smithi012.stdout:(82/157): python3-markupsafe-1.1.1-12.el9.x86_6 695 kB/s | 35 kB 00:00 2024-09-10T23:52:59.149 INFO:teuthology.orchestra.run.smithi165.stdout:(105/157): abseil-cpp-20211102.0-3.el9.x86_64.r 11 MB/s | 548 kB 00:00 2024-09-10T23:52:59.185 INFO:teuthology.orchestra.run.smithi165.stdout:(106/157): gperftools-libs-2.9.1-2.el9.x86_64.r 9.1 MB/s | 309 kB 00:00 2024-09-10T23:52:59.208 INFO:teuthology.orchestra.run.smithi165.stdout:(107/157): grpc-data-1.46.7-10.el9.noarch.rpm 775 kB/s | 19 kB 00:00 2024-09-10T23:52:59.214 INFO:teuthology.orchestra.run.smithi012.stdout:(83/157): python3-babel-2.9.1-2.el9.noarch.rpm 13 MB/s | 6.0 MB 00:00 2024-09-10T23:52:59.239 INFO:teuthology.orchestra.run.smithi012.stdout:(84/157): python3-numpy-f2py-1.20.1-5.el9.x86_6 2.0 MB/s | 250 kB 00:00 2024-09-10T23:52:59.265 INFO:teuthology.orchestra.run.smithi012.stdout:(85/157): python3-oauthlib-3.1.1-5.el9.noarch.r 4.3 MB/s | 222 kB 00:00 2024-09-10T23:52:59.290 INFO:teuthology.orchestra.run.smithi012.stdout:(86/157): python3-packaging-20.9-5.el9.noarch.r 1.5 MB/s | 77 kB 00:00 2024-09-10T23:52:59.315 INFO:teuthology.orchestra.run.smithi012.stdout:(87/157): python3-prettytable-0.7.2-27.el9.noar 845 kB/s | 42 kB 00:00 2024-09-10T23:52:59.316 INFO:teuthology.orchestra.run.smithi165.stdout:(108/157): libarrow-9.0.0-11.el9.x86_64.rpm 41 MB/s | 4.4 MB 00:00 2024-09-10T23:52:59.340 INFO:teuthology.orchestra.run.smithi012.stdout:(88/157): python3-protobuf-3.14.0-13.el9.noarch 5.3 MB/s | 269 kB 00:00 2024-09-10T23:52:59.342 INFO:teuthology.orchestra.run.smithi165.stdout:(109/157): libarrow-doc-9.0.0-11.el9.noarch.rpm 1.0 MB/s | 26 kB 00:00 2024-09-10T23:52:59.359 INFO:teuthology.orchestra.run.smithi165.stdout:(110/157): liboath-2.6.7-2.el9.x86_64.rpm 2.9 MB/s | 49 kB 00:00 2024-09-10T23:52:59.366 INFO:teuthology.orchestra.run.smithi012.stdout:(89/157): python3-pyasn1-0.4.8-6.el9.noarch.rpm 3.1 MB/s | 159 kB 00:00 2024-09-10T23:52:59.384 INFO:teuthology.orchestra.run.smithi165.stdout:(111/157): libunwind-1.6.2-1.el9.x86_64.rpm 2.6 MB/s | 67 kB 00:00 2024-09-10T23:52:59.409 INFO:teuthology.orchestra.run.smithi165.stdout:(112/157): luarocks-3.9.2-5.el9.noarch.rpm 5.9 MB/s | 151 kB 00:00 2024-09-10T23:52:59.416 INFO:teuthology.orchestra.run.smithi012.stdout:(90/157): python3-pyasn1-modules-0.4.8-6.el9.no 3.6 MB/s | 279 kB 00:00 2024-09-10T23:52:59.441 INFO:teuthology.orchestra.run.smithi012.stdout:(91/157): python3-pytz-2021.1-5.el9.noarch.rpm 680 kB/s | 51 kB 00:00 2024-09-10T23:52:59.451 INFO:teuthology.orchestra.run.smithi165.stdout:(113/157): parquet-libs-9.0.0-11.el9.x86_64.rpm 20 MB/s | 839 kB 00:00 2024-09-10T23:52:59.475 INFO:teuthology.orchestra.run.smithi012.stdout:(92/157): python3-requests-oauthlib-1.3.0-12.el 919 kB/s | 54 kB 00:00 2024-09-10T23:52:59.485 INFO:teuthology.orchestra.run.smithi165.stdout:(114/157): python3-asyncssh-2.13.2-5.el9.noarch 16 MB/s | 548 kB 00:00 2024-09-10T23:52:59.502 INFO:teuthology.orchestra.run.smithi165.stdout:(115/157): python3-bcrypt-3.2.2-1.el9.x86_64.rp 2.5 MB/s | 43 kB 00:00 2024-09-10T23:52:59.517 INFO:teuthology.orchestra.run.smithi012.stdout:(93/157): python3-toml-0.10.2-6.el9.noarch.rpm 1.0 MB/s | 42 kB 00:00 2024-09-10T23:52:59.518 INFO:teuthology.orchestra.run.smithi165.stdout:(116/157): python3-cachetools-4.2.4-1.el9.noarc 1.9 MB/s | 32 kB 00:00 2024-09-10T23:52:59.536 INFO:teuthology.orchestra.run.smithi165.stdout:(117/157): python3-certifi-2023.05.07-4.el9.noa 851 kB/s | 14 kB 00:00 2024-09-10T23:52:59.561 INFO:teuthology.orchestra.run.smithi165.stdout:(118/157): python3-cheroot-8.6.0-4.el9.noarch.r 6.7 MB/s | 172 kB 00:00 2024-09-10T23:52:59.592 INFO:teuthology.orchestra.run.smithi012.stdout:(94/157): python3-numpy-1.20.1-5.el9.x86_64.rpm 10 MB/s | 5.1 MB 00:00 2024-09-10T23:52:59.594 INFO:teuthology.orchestra.run.smithi165.stdout:(119/157): python3-cherrypy-18.6.1-2.el9.noarch 11 MB/s | 358 kB 00:00 2024-09-10T23:52:59.617 INFO:teuthology.orchestra.run.smithi012.stdout:(95/157): python3-wcwidth-0.2.5-8.el9.noarch.rp 442 kB/s | 44 kB 00:00 2024-09-10T23:52:59.620 INFO:teuthology.orchestra.run.smithi165.stdout:(120/157): python3-cmd2-2.4.2-2.el9.noarch.rpm 12 MB/s | 303 kB 00:00 2024-09-10T23:52:59.643 INFO:teuthology.orchestra.run.smithi012.stdout:(96/157): qatlib-24.02.0-1.el9.x86_64.rpm 4.3 MB/s | 221 kB 00:00 2024-09-10T23:52:59.645 INFO:teuthology.orchestra.run.smithi165.stdout:(121/157): python3-google-auth-2.34.0-1.el9.noa 8.6 MB/s | 220 kB 00:00 2024-09-10T23:52:59.668 INFO:teuthology.orchestra.run.smithi012.stdout:(97/157): qatlib-service-24.02.0-1.el9.x86_64.r 715 kB/s | 36 kB 00:00 2024-09-10T23:52:59.693 INFO:teuthology.orchestra.run.smithi012.stdout:(98/157): qatzip-libs-1.2.0-1.el9.x86_64.rpm 947 kB/s | 47 kB 00:00 2024-09-10T23:52:59.712 INFO:teuthology.orchestra.run.smithi165.stdout:(122/157): python3-grpcio-1.46.7-10.el9.x86_64. 31 MB/s | 2.0 MB 00:00 2024-09-10T23:52:59.718 INFO:teuthology.orchestra.run.smithi012.stdout:(99/157): socat-1.7.4.1-6.el9.x86_64.rpm 5.9 MB/s | 304 kB 00:00 2024-09-10T23:52:59.739 INFO:teuthology.orchestra.run.smithi165.stdout:(123/157): python3-grpcio-tools-1.46.7-10.el9.x 5.7 MB/s | 144 kB 00:00 2024-09-10T23:52:59.744 INFO:teuthology.orchestra.run.smithi012.stdout:(100/157): uuid-1.6.2-55.el9.x86_64.rpm 1.1 MB/s | 58 kB 00:00 2024-09-10T23:52:59.755 INFO:teuthology.orchestra.run.smithi165.stdout:(124/157): python3-jaraco-8.2.1-3.el9.noarch.rp 646 kB/s | 11 kB 00:00 2024-09-10T23:52:59.770 INFO:teuthology.orchestra.run.smithi012.stdout:(101/157): xfsprogs-devel-6.4.0-4.el9.x86_64.rp 1.9 MB/s | 96 kB 00:00 2024-09-10T23:52:59.772 INFO:teuthology.orchestra.run.smithi165.stdout:(125/157): python3-jaraco-classes-3.2.1-5.el9.n 1.0 MB/s | 18 kB 00:00 2024-09-10T23:52:59.797 INFO:teuthology.orchestra.run.smithi165.stdout:(126/157): python3-jaraco-collections-3.0.0-8.e 928 kB/s | 23 kB 00:00 2024-09-10T23:52:59.797 INFO:teuthology.orchestra.run.smithi012.stdout:(102/157): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.2 MB/s | 64 kB 00:00 2024-09-10T23:52:59.822 INFO:teuthology.orchestra.run.smithi165.stdout:(127/157): python3-jaraco-functools-3.5.0-2.el9 781 kB/s | 19 kB 00:00 2024-09-10T23:52:59.840 INFO:teuthology.orchestra.run.smithi012.stdout:(103/157): zlib-devel-1.2.11-41.el9.x86_64.rpm 633 kB/s | 45 kB 00:00 2024-09-10T23:52:59.847 INFO:teuthology.orchestra.run.smithi165.stdout:(128/157): python3-jaraco-text-3.2.0-6.el9.noar 792 kB/s | 20 kB 00:00 2024-09-10T23:52:59.890 INFO:teuthology.orchestra.run.smithi165.stdout:(129/157): python3-kubernetes-26.1.0-2.el9.noar 24 MB/s | 1.0 MB 00:00 2024-09-10T23:52:59.906 INFO:teuthology.orchestra.run.smithi165.stdout:(130/157): python3-logutils-0.3.5-21.el9.noarch 2.7 MB/s | 46 kB 00:00 2024-09-10T23:52:59.911 INFO:teuthology.orchestra.run.smithi012.stdout:(104/157): lua-devel-5.4.4-4.el9.x86_64.rpm 191 kB/s | 22 kB 00:00 2024-09-10T23:52:59.932 INFO:teuthology.orchestra.run.smithi165.stdout:(131/157): python3-more-itertools-8.12.0-2.el9. 3.1 MB/s | 79 kB 00:00 2024-09-10T23:52:59.957 INFO:teuthology.orchestra.run.smithi165.stdout:(132/157): python3-natsort-7.1.1-5.el9.noarch.r 2.3 MB/s | 58 kB 00:00 2024-09-10T23:52:59.978 INFO:teuthology.orchestra.run.smithi012.stdout:(105/157): abseil-cpp-20211102.0-3.el9.x86_64.r 8.1 MB/s | 548 kB 00:00 2024-09-10T23:52:59.982 INFO:teuthology.orchestra.run.smithi165.stdout:(133/157): python3-pecan-1.4.2-3.el9.noarch.rpm 11 MB/s | 272 kB 00:00 2024-09-10T23:53:00.007 INFO:teuthology.orchestra.run.smithi165.stdout:(134/157): python3-portend-3.1.0-2.el9.noarch.r 654 kB/s | 16 kB 00:00 2024-09-10T23:53:00.012 INFO:teuthology.orchestra.run.smithi012.stdout:(106/157): protobuf-compiler-3.14.0-13.el9.x86_ 4.9 MB/s | 863 kB 00:00 2024-09-10T23:53:00.032 INFO:teuthology.orchestra.run.smithi165.stdout:(135/157): python3-pyOpenSSL-21.0.0-1.el9.noarc 3.5 MB/s | 90 kB 00:00 2024-09-10T23:53:00.054 INFO:teuthology.orchestra.run.smithi012.stdout:(107/157): gperftools-libs-2.9.1-2.el9.x86_64.r 4.0 MB/s | 309 kB 00:00 2024-09-10T23:53:00.058 INFO:teuthology.orchestra.run.smithi165.stdout:(136/157): python3-pyperclip-1.8.2-2.el9.noarch 1.1 MB/s | 28 kB 00:00 2024-09-10T23:53:00.079 INFO:teuthology.orchestra.run.smithi012.stdout:(108/157): grpc-data-1.46.7-10.el9.noarch.rpm 291 kB/s | 19 kB 00:00 2024-09-10T23:53:00.083 INFO:teuthology.orchestra.run.smithi165.stdout:(137/157): python3-repoze-lru-0.7-16.el9.noarch 1.2 MB/s | 31 kB 00:00 2024-09-10T23:53:00.105 INFO:teuthology.orchestra.run.smithi012.stdout:(109/157): libarrow-doc-9.0.0-11.el9.noarch.rpm 1.0 MB/s | 26 kB 00:00 2024-09-10T23:53:00.109 INFO:teuthology.orchestra.run.smithi165.stdout:(138/157): python3-routes-2.5.1-5.el9.noarch.rp 7.4 MB/s | 188 kB 00:00 2024-09-10T23:53:00.130 INFO:teuthology.orchestra.run.smithi012.stdout:(110/157): liboath-2.6.7-2.el9.x86_64.rpm 1.9 MB/s | 49 kB 00:00 2024-09-10T23:53:00.134 INFO:teuthology.orchestra.run.smithi165.stdout:(139/157): python3-rsa-4.9-2.el9.noarch.rpm 2.3 MB/s | 59 kB 00:00 2024-09-10T23:53:00.155 INFO:teuthology.orchestra.run.smithi012.stdout:(111/157): libunwind-1.6.2-1.el9.x86_64.rpm 2.7 MB/s | 67 kB 00:00 2024-09-10T23:53:00.159 INFO:teuthology.orchestra.run.smithi165.stdout:(140/157): python3-tempora-5.0.0-2.el9.noarch.r 1.4 MB/s | 36 kB 00:00 2024-09-10T23:53:00.185 INFO:teuthology.orchestra.run.smithi165.stdout:(141/157): python3-typing-extensions-4.12.2-2.e 2.9 MB/s | 74 kB 00:00 2024-09-10T23:53:00.189 INFO:teuthology.orchestra.run.smithi012.stdout:(112/157): luarocks-3.9.2-5.el9.noarch.rpm 4.5 MB/s | 151 kB 00:00 2024-09-10T23:53:00.210 INFO:teuthology.orchestra.run.smithi165.stdout:(142/157): python3-webob-1.8.8-2.el9.noarch.rpm 9.0 MB/s | 230 kB 00:00 2024-09-10T23:53:00.235 INFO:teuthology.orchestra.run.smithi165.stdout:(143/157): python3-websocket-client-1.2.3-2.el9 3.5 MB/s | 90 kB 00:00 2024-09-10T23:53:00.269 INFO:teuthology.orchestra.run.smithi165.stdout:(144/157): python3-werkzeug-2.0.3-3.el9.1.noarc 13 MB/s | 427 kB 00:00 2024-09-10T23:53:00.294 INFO:teuthology.orchestra.run.smithi165.stdout:(145/157): python3-zc-lockfile-2.0-10.el9.noarc 801 kB/s | 20 kB 00:00 2024-09-10T23:53:00.319 INFO:teuthology.orchestra.run.smithi165.stdout:(146/157): re2-20211101-20.el9.x86_64.rpm 7.5 MB/s | 191 kB 00:00 2024-09-10T23:53:00.364 INFO:teuthology.orchestra.run.smithi012.stdout:(113/157): python3-scipy-1.6.2-8.el9.x86_64.rpm 18 MB/s | 16 MB 00:00 2024-09-10T23:53:00.378 INFO:teuthology.orchestra.run.smithi165.stdout:(147/157): thrift-0.15.0-2.el9.x86_64.rpm 27 MB/s | 1.6 MB 00:00 2024-09-10T23:53:00.431 INFO:teuthology.orchestra.run.smithi012.stdout:(114/157): parquet-libs-9.0.0-11.el9.x86_64.rpm 3.4 MB/s | 839 kB 00:00 2024-09-10T23:53:00.448 INFO:teuthology.orchestra.run.smithi012.stdout:(115/157): python3-bcrypt-3.2.2-1.el9.x86_64.rp 2.6 MB/s | 43 kB 00:00 2024-09-10T23:53:00.465 INFO:teuthology.orchestra.run.smithi012.stdout:(116/157): python3-cachetools-4.2.4-1.el9.noarc 1.9 MB/s | 32 kB 00:00 2024-09-10T23:53:00.482 INFO:teuthology.orchestra.run.smithi012.stdout:(117/157): python3-certifi-2023.05.07-4.el9.noa 854 kB/s | 14 kB 00:00 2024-09-10T23:53:00.516 INFO:teuthology.orchestra.run.smithi012.stdout:(118/157): python3-cheroot-8.6.0-4.el9.noarch.r 5.1 MB/s | 172 kB 00:00 2024-09-10T23:53:00.541 INFO:teuthology.orchestra.run.smithi012.stdout:(119/157): python3-asyncssh-2.13.2-5.el9.noarch 3.0 MB/s | 548 kB 00:00 2024-09-10T23:53:00.575 INFO:teuthology.orchestra.run.smithi012.stdout:(120/157): python3-cherrypy-18.6.1-2.el9.noarch 6.0 MB/s | 358 kB 00:00 2024-09-10T23:53:00.620 INFO:teuthology.orchestra.run.smithi012.stdout:(121/157): python3-cmd2-2.4.2-2.el9.noarch.rpm 3.9 MB/s | 303 kB 00:00 2024-09-10T23:53:00.651 INFO:teuthology.orchestra.run.smithi012.stdout:(122/157): python3-google-auth-2.34.0-1.el9.noa 2.8 MB/s | 220 kB 00:00 2024-09-10T23:53:00.684 INFO:teuthology.orchestra.run.smithi012.stdout:(123/157): python3-grpcio-tools-1.46.7-10.el9.x 4.3 MB/s | 144 kB 00:00 2024-09-10T23:53:00.686 INFO:teuthology.orchestra.run.smithi165.stdout:(148/157): librados2-19.3.0-4854.g8f219c47.el9. 11 MB/s | 3.5 MB 00:00 2024-09-10T23:53:00.735 INFO:teuthology.orchestra.run.smithi012.stdout:(124/157): python3-jaraco-8.2.1-3.el9.noarch.rp 214 kB/s | 11 kB 00:00 2024-09-10T23:53:00.759 INFO:teuthology.orchestra.run.smithi012.stdout:(125/157): python3-jaraco-classes-3.2.1-5.el9.n 720 kB/s | 18 kB 00:00 2024-09-10T23:53:00.785 INFO:teuthology.orchestra.run.smithi012.stdout:(126/157): python3-jaraco-collections-3.0.0-8.e 930 kB/s | 23 kB 00:00 2024-09-10T23:53:00.810 INFO:teuthology.orchestra.run.smithi012.stdout:(127/157): python3-jaraco-functools-3.5.0-2.el9 780 kB/s | 19 kB 00:00 2024-09-10T23:53:00.835 INFO:teuthology.orchestra.run.smithi012.stdout:(128/157): python3-jaraco-text-3.2.0-6.el9.noar 791 kB/s | 20 kB 00:00 2024-09-10T23:53:00.904 INFO:teuthology.orchestra.run.smithi165.stdout:(149/157): librbd1-19.3.0-4854.g8f219c47.el9.x8 15 MB/s | 3.2 MB 00:00 2024-09-10T23:53:00.911 INFO:teuthology.orchestra.run.smithi012.stdout:(129/157): libarrow-9.0.0-11.el9.x86_64.rpm 5.2 MB/s | 4.4 MB 00:00 2024-09-10T23:53:00.936 INFO:teuthology.orchestra.run.smithi012.stdout:(130/157): python3-logutils-0.3.5-21.el9.noarch 1.8 MB/s | 46 kB 00:00 2024-09-10T23:53:00.961 INFO:teuthology.orchestra.run.smithi012.stdout:(131/157): python3-more-itertools-8.12.0-2.el9. 3.1 MB/s | 79 kB 00:00 2024-09-10T23:53:00.987 INFO:teuthology.orchestra.run.smithi012.stdout:(132/157): python3-natsort-7.1.1-5.el9.noarch.r 2.2 MB/s | 58 kB 00:00 2024-09-10T23:53:01.029 INFO:teuthology.orchestra.run.smithi012.stdout:(133/157): python3-kubernetes-26.1.0-2.el9.noar 5.3 MB/s | 1.0 MB 00:00 2024-09-10T23:53:01.062 INFO:teuthology.orchestra.run.smithi012.stdout:(134/157): python3-pecan-1.4.2-3.el9.noarch.rpm 3.5 MB/s | 272 kB 00:00 2024-09-10T23:53:01.088 INFO:teuthology.orchestra.run.smithi012.stdout:(135/157): python3-portend-3.1.0-2.el9.noarch.r 280 kB/s | 16 kB 00:00 2024-09-10T23:53:01.138 INFO:teuthology.orchestra.run.smithi012.stdout:(136/157): python3-grpcio-1.46.7-10.el9.x86_64. 3.9 MB/s | 2.0 MB 00:00 2024-09-10T23:53:01.163 INFO:teuthology.orchestra.run.smithi012.stdout:(137/157): python3-pyperclip-1.8.2-2.el9.noarch 376 kB/s | 28 kB 00:00 2024-09-10T23:53:01.188 INFO:teuthology.orchestra.run.smithi012.stdout:(138/157): python3-pyOpenSSL-21.0.0-1.el9.noarc 715 kB/s | 90 kB 00:00 2024-09-10T23:53:01.213 INFO:teuthology.orchestra.run.smithi012.stdout:(139/157): python3-repoze-lru-0.7-16.el9.noarch 408 kB/s | 31 kB 00:00 2024-09-10T23:53:01.239 INFO:teuthology.orchestra.run.smithi012.stdout:(140/157): python3-rsa-4.9-2.el9.noarch.rpm 1.2 MB/s | 59 kB 00:00 2024-09-10T23:53:01.264 INFO:teuthology.orchestra.run.smithi012.stdout:(141/157): python3-tempora-5.0.0-2.el9.noarch.r 713 kB/s | 36 kB 00:00 2024-09-10T23:53:01.289 INFO:teuthology.orchestra.run.smithi012.stdout:(142/157): python3-typing-extensions-4.12.2-2.e 1.5 MB/s | 74 kB 00:00 2024-09-10T23:53:01.323 INFO:teuthology.orchestra.run.smithi012.stdout:(143/157): python3-routes-2.5.1-5.el9.noarch.rp 1.2 MB/s | 188 kB 00:00 2024-09-10T23:53:01.349 INFO:teuthology.orchestra.run.smithi012.stdout:(144/157): python3-websocket-client-1.2.3-2.el9 1.5 MB/s | 90 kB 00:00 2024-09-10T23:53:01.374 INFO:teuthology.orchestra.run.smithi012.stdout:(145/157): python3-webob-1.8.8-2.el9.noarch.rpm 2.1 MB/s | 230 kB 00:00 2024-09-10T23:53:01.399 INFO:teuthology.orchestra.run.smithi012.stdout:(146/157): python3-zc-lockfile-2.0-10.el9.noarc 399 kB/s | 20 kB 00:00 2024-09-10T23:53:01.433 INFO:teuthology.orchestra.run.smithi012.stdout:(147/157): re2-20211101-20.el9.x86_64.rpm 3.2 MB/s | 191 kB 00:00 2024-09-10T23:53:01.458 INFO:teuthology.orchestra.run.smithi012.stdout:(148/157): python3-werkzeug-2.0.3-3.el9.1.noarc 3.1 MB/s | 427 kB 00:00 2024-09-10T23:53:01.517 INFO:teuthology.orchestra.run.smithi012.stdout:(149/157): thrift-0.15.0-2.el9.x86_64.rpm 14 MB/s | 1.6 MB 00:00 2024-09-10T23:53:01.692 INFO:teuthology.orchestra.run.smithi012.stdout:(150/157): librados2-19.3.0-4854.g8f219c47.el9. 13 MB/s | 3.5 MB 00:00 2024-09-10T23:53:01.759 INFO:teuthology.orchestra.run.smithi012.stdout:(151/157): librbd1-19.3.0-4854.g8f219c47.el9.x8 11 MB/s | 3.2 MB 00:00 2024-09-10T23:53:02.271 INFO:teuthology.orchestra.run.smithi165.stdout:(150/157): python3-numpy-1.20.1-5.el9.x86_64.rp 654 kB/s | 5.1 MB 00:07 2024-09-10T23:53:02.376 INFO:teuthology.orchestra.run.smithi012.stdout:(152/157): NetworkManager-team-1.49.90-1.el9.x8 66 kB/s | 41 kB 00:00 2024-09-10T23:53:02.785 INFO:teuthology.orchestra.run.smithi012.stdout:(153/157): NetworkManager-1.49.90-1.el9.x86_64. 1.8 MB/s | 2.3 MB 00:01 2024-09-10T23:53:02.835 INFO:teuthology.orchestra.run.smithi012.stdout:(154/157): NetworkManager-libnm-1.49.90-1.el9.x 1.6 MB/s | 1.8 MB 00:01 2024-09-10T23:53:03.310 INFO:teuthology.orchestra.run.smithi012.stdout:(155/157): NetworkManager-tui-1.49.90-1.el9.x86 266 kB/s | 249 kB 00:00 2024-09-10T23:53:03.354 INFO:teuthology.orchestra.run.smithi165.stdout:(151/157): NetworkManager-libnm-1.49.90-1.el9.x 1.7 MB/s | 1.8 MB 00:01 2024-09-10T23:53:03.579 INFO:teuthology.orchestra.run.smithi165.stdout:(152/157): NetworkManager-1.49.90-1.el9.x86_64. 888 kB/s | 2.3 MB 00:02 2024-09-10T23:53:03.727 INFO:teuthology.orchestra.run.smithi012.stdout:(156/157): openssl-3.2.2-6.el9.x86_64.rpm 1.4 MB/s | 1.4 MB 00:00 2024-09-10T23:53:03.795 INFO:teuthology.orchestra.run.smithi178.stdout: Preparing : 1/1 2024-09-10T23:53:03.869 INFO:teuthology.orchestra.run.smithi012.stdout:(157/157): openssl-libs-3.2.2-6.el9.x86_64.rpm 2.3 MB/s | 2.4 MB 00:01 2024-09-10T23:53:03.876 INFO:teuthology.orchestra.run.smithi012.stdout:-------------------------------------------------------------------------------- 2024-09-10T23:53:03.877 INFO:teuthology.orchestra.run.smithi012.stdout:Total 18 MB/s | 240 MB 00:13 2024-09-10T23:53:03.918 INFO:teuthology.orchestra.run.smithi178.stdout: Upgrading : openssl-libs-1:3.2.2-6.el9.x86_64 1/165 2024-09-10T23:53:03.938 INFO:teuthology.orchestra.run.smithi165.stdout:(153/157): NetworkManager-team-1.49.90-1.el9.x8 70 kB/s | 41 kB 00:00 2024-09-10T23:53:03.956 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/165 2024-09-10T23:53:04.009 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/165 2024-09-10T23:53:04.052 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/165 2024-09-10T23:53:04.077 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/165 2024-09-10T23:53:04.120 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/165 2024-09-10T23:53:04.338 INFO:teuthology.orchestra.run.smithi165.stdout:(154/157): NetworkManager-tui-1.49.90-1.el9.x86 331 kB/s | 249 kB 00:00 2024-09-10T23:53:04.444 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/165 2024-09-10T23:53:04.562 INFO:teuthology.orchestra.run.smithi178.stdout: Upgrading : librados2-2:19.3.0-4854.g8f219c47.el9.x86_64 8/165 2024-09-10T23:53:04.668 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: librados2-2:19.3.0-4854.g8f219c47.el9.x86_64 8/165 2024-09-10T23:53:04.696 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : libcephfs2-2:19.3.0-4854.g8f219c47.el9.x86_64 9/165 2024-09-10T23:53:04.771 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: libcephfs2-2:19.3.0-4854.g8f219c47.el9.x86_64 9/165 2024-09-10T23:53:04.800 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-rados-2:19.3.0-4854.g8f219c47.el9.x86_64 10/165 2024-09-10T23:53:04.829 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : postgresql-private-libs-13.16-1.el9.x86_64 11/165 2024-09-10T23:53:04.982 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : libnbd-1.20.2-2.el9.x86_64 12/165 2024-09-10T23:53:05.039 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : postgresql-13.16-1.el9.x86_64 13/165 2024-09-10T23:53:05.068 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : libcephsqlite-2:19.3.0-4854.g8f219c47.el9.x86_64 14/165 2024-09-10T23:53:05.113 INFO:teuthology.orchestra.run.smithi165.stdout:(155/157): openssl-3.2.2-6.el9.x86_64.rpm 1.2 MB/s | 1.4 MB 00:01 2024-09-10T23:53:05.172 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: libcephsqlite-2:19.3.0-4854.g8f219c47.el9.x86_64 14/165 2024-09-10T23:53:05.202 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : libradosstriper1-2:19.3.0-4854.g8f219c47.el9.x86 15/165 2024-09-10T23:53:05.265 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: libradosstriper1-2:19.3.0-4854.g8f219c47.el9.x86 15/165 2024-09-10T23:53:05.286 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 16/165 2024-09-10T23:53:05.308 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 16/165 2024-09-10T23:53:05.496 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 17/165 2024-09-10T23:53:05.613 INFO:teuthology.orchestra.run.smithi165.stdout:(156/157): openssl-libs-3.2.2-6.el9.x86_64.rpm 1.9 MB/s | 2.4 MB 00:01 2024-09-10T23:53:05.621 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 17/165 2024-09-10T23:53:05.743 INFO:teuthology.orchestra.run.smithi178.stdout: Upgrading : NetworkManager-libnm-1:1.49.90-1.el9.x86_64 18/165 2024-09-10T23:53:05.843 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: NetworkManager-1:1.49.90-1.el9.x86_64 19/165 2024-09-10T23:53:05.919 INFO:teuthology.orchestra.run.smithi178.stdout: Upgrading : NetworkManager-1:1.49.90-1.el9.x86_64 19/165 2024-09-10T23:53:05.968 INFO:teuthology.orchestra.run.smithi012.stdout:Running transaction check 2024-09-10T23:53:06.128 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: NetworkManager-1:1.49.90-1.el9.x86_64 19/165 2024-09-10T23:53:06.169 INFO:teuthology.orchestra.run.smithi012.stdout:Transaction check succeeded. 2024-09-10T23:53:06.169 INFO:teuthology.orchestra.run.smithi012.stdout:Running transaction test 2024-09-10T23:53:06.232 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : re2-1:20211101-20.el9.x86_64 20/165 2024-09-10T23:53:06.290 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 21/165 2024-09-10T23:53:06.325 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : liboath-2.6.7-2.el9.x86_64 22/165 2024-09-10T23:53:06.406 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 23/165 2024-09-10T23:53:06.450 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 24/165 2024-09-10T23:53:06.482 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-packaging-20.9-5.el9.noarch 25/165 2024-09-10T23:53:06.534 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 26/165 2024-09-10T23:53:06.567 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 27/165 2024-09-10T23:53:06.595 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : lua-5.4.4-4.el9.x86_64 28/165 2024-09-10T23:53:06.664 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 29/165 2024-09-10T23:53:06.725 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 30/165 2024-09-10T23:53:06.756 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 31/165 2024-09-10T23:53:06.792 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 32/165 2024-09-10T23:53:06.870 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 33/165 2024-09-10T23:53:06.903 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-ceph-common-2:19.3.0-4854.g8f219c47.el9. 34/165 2024-09-10T23:53:06.948 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-ceph-argparse-2:19.3.0-4854.g8f219c47.el 35/165 2024-09-10T23:53:06.975 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-cephfs-2:19.3.0-4854.g8f219c47.el9.x86_6 36/165 2024-09-10T23:53:07.046 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 37/165 2024-09-10T23:53:07.090 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : luarocks-3.9.2-5.el9.noarch 38/165 2024-09-10T23:53:07.173 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 39/165 2024-09-10T23:53:07.312 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 40/165 2024-09-10T23:53:07.363 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 41/165 2024-09-10T23:53:07.915 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-rsa-4.9-2.el9.noarch 42/165 2024-09-10T23:53:08.001 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 43/165 2024-09-10T23:53:08.028 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 44/165 2024-09-10T23:53:08.080 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 45/165 2024-09-10T23:53:08.080 INFO:teuthology.orchestra.run.smithi165.stdout:(157/157): python3-scipy-1.6.2-8.el9.x86_64.rpm 1.4 MB/s | 16 MB 00:11 2024-09-10T23:53:08.085 INFO:teuthology.orchestra.run.smithi165.stdout:-------------------------------------------------------------------------------- 2024-09-10T23:53:08.085 INFO:teuthology.orchestra.run.smithi165.stdout:Total 7.9 MB/s | 240 MB 00:30 2024-09-10T23:53:08.118 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : librados-devel-2:19.3.0-4854.g8f219c47.el9.x86_6 46/165 2024-09-10T23:53:08.167 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 47/165 2024-09-10T23:53:08.324 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 48/165 2024-09-10T23:53:08.395 INFO:teuthology.orchestra.run.smithi178.stdout: Upgrading : openssl-1:3.2.2-6.el9.x86_64 49/165 2024-09-10T23:53:08.486 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 50/165 2024-09-10T23:53:08.543 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 51/165 2024-09-10T23:53:08.615 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 52/165 2024-09-10T23:53:08.656 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 53/165 2024-09-10T23:53:08.700 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 54/165 2024-09-10T23:53:08.728 INFO:teuthology.orchestra.run.smithi012.stdout:Transaction test succeeded. 2024-09-10T23:53:08.728 INFO:teuthology.orchestra.run.smithi012.stdout:Running transaction 2024-09-10T23:53:08.755 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 55/165 2024-09-10T23:53:08.791 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 56/165 2024-09-10T23:53:08.834 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 57/165 2024-09-10T23:53:08.876 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 58/165 2024-09-10T23:53:09.011 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 59/165 2024-09-10T23:53:09.061 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 60/165 2024-09-10T23:53:09.102 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 61/165 2024-09-10T23:53:09.214 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 62/165 2024-09-10T23:53:09.253 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 63/165 2024-09-10T23:53:09.293 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 64/165 2024-09-10T23:53:09.319 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 65/165 2024-09-10T23:53:09.341 INFO:teuthology.orchestra.run.smithi165.stdout:Running transaction check 2024-09-10T23:53:09.343 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 66/165 2024-09-10T23:53:09.418 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 67/165 2024-09-10T23:53:09.496 INFO:teuthology.orchestra.run.smithi165.stdout:Transaction check succeeded. 2024-09-10T23:53:09.496 INFO:teuthology.orchestra.run.smithi165.stdout:Running transaction test 2024-09-10T23:53:09.811 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 68/165 2024-09-10T23:53:09.851 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 69/165 2024-09-10T23:53:09.879 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 70/165 2024-09-10T23:53:09.940 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 71/165 2024-09-10T23:53:10.107 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : uuid-1.6.2-55.el9.x86_64 72/165 2024-09-10T23:53:10.261 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 73/165 2024-09-10T23:53:10.340 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 74/165 2024-09-10T23:53:10.374 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 75/165 2024-09-10T23:53:10.408 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 76/165 2024-09-10T23:53:10.467 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 77/165 2024-09-10T23:53:10.500 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 78/165 2024-09-10T23:53:10.592 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 79/165 2024-09-10T23:53:10.647 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 80/165 2024-09-10T23:53:10.678 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 81/165 2024-09-10T23:53:10.783 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 82/165 2024-09-10T23:53:10.837 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 83/165 2024-09-10T23:53:11.393 INFO:teuthology.orchestra.run.smithi165.stdout:Transaction test succeeded. 2024-09-10T23:53:11.394 INFO:teuthology.orchestra.run.smithi165.stdout:Running transaction 2024-09-10T23:53:11.558 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 84/165 2024-09-10T23:53:11.611 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 85/165 2024-09-10T23:53:11.644 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 86/165 2024-09-10T23:53:11.751 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 87/165 2024-09-10T23:53:11.808 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 88/165 2024-09-10T23:53:11.853 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 89/165 2024-09-10T23:53:11.940 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 90/165 2024-09-10T23:53:11.989 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 91/165 2024-09-10T23:53:12.173 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : openblas-0.3.26-2.el9.x86_64 92/165 2024-09-10T23:53:12.203 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 93/165 2024-09-10T23:53:12.280 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 94/165 2024-09-10T23:53:12.923 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 95/165 2024-09-10T23:53:13.044 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 96/165 2024-09-10T23:53:14.267 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 97/165 2024-09-10T23:53:14.357 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 98/165 2024-09-10T23:53:14.392 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 99/165 2024-09-10T23:53:14.426 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : libuuid-devel-2.37.4-20.el9.x86_64 100/165 2024-09-10T23:53:14.744 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 101/165 2024-09-10T23:53:14.789 INFO:teuthology.orchestra.run.smithi178.stdout: Upgrading : librbd1-2:19.3.0-4854.g8f219c47.el9.x86_64 102/165 2024-09-10T23:53:14.871 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: librbd1-2:19.3.0-4854.g8f219c47.el9.x86_64 102/165 2024-09-10T23:53:14.911 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-rbd-2:19.3.0-4854.g8f219c47.el9.x86_64 103/165 2024-09-10T23:53:14.954 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 104/165 2024-09-10T23:53:15.464 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 105/165 2024-09-10T23:53:15.492 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : librgw2-2:19.3.0-4854.g8f219c47.el9.x86_64 106/165 2024-09-10T23:53:15.513 INFO:teuthology.orchestra.run.smithi012.stdout: Preparing : 1/1 2024-09-10T23:53:15.571 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: librgw2-2:19.3.0-4854.g8f219c47.el9.x86_64 106/165 2024-09-10T23:53:15.602 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-rgw-2:19.3.0-4854.g8f219c47.el9.x86_64 107/165 2024-09-10T23:53:15.698 INFO:teuthology.orchestra.run.smithi012.stdout: Upgrading : openssl-libs-1:3.2.2-6.el9.x86_64 1/165 2024-09-10T23:53:15.957 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/165 2024-09-10T23:53:16.101 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/165 2024-09-10T23:53:16.143 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/165 2024-09-10T23:53:16.174 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/165 2024-09-10T23:53:16.232 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/165 2024-09-10T23:53:16.622 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/165 2024-09-10T23:53:16.724 INFO:teuthology.orchestra.run.smithi012.stdout: Upgrading : librados2-2:19.3.0-4854.g8f219c47.el9.x86_64 8/165 2024-09-10T23:53:16.873 INFO:teuthology.orchestra.run.smithi012.stdout: Running scriptlet: librados2-2:19.3.0-4854.g8f219c47.el9.x86_64 8/165 2024-09-10T23:53:16.900 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : libcephfs2-2:19.3.0-4854.g8f219c47.el9.x86_64 9/165 2024-09-10T23:53:17.015 INFO:teuthology.orchestra.run.smithi012.stdout: Running scriptlet: libcephfs2-2:19.3.0-4854.g8f219c47.el9.x86_64 9/165 2024-09-10T23:53:17.052 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-rados-2:19.3.0-4854.g8f219c47.el9.x86_64 10/165 2024-09-10T23:53:17.087 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : postgresql-private-libs-13.16-1.el9.x86_64 11/165 2024-09-10T23:53:17.319 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : libnbd-1.20.2-2.el9.x86_64 12/165 2024-09-10T23:53:17.379 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : postgresql-13.16-1.el9.x86_64 13/165 2024-09-10T23:53:17.405 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : libcephsqlite-2:19.3.0-4854.g8f219c47.el9.x86_64 14/165 2024-09-10T23:53:17.517 INFO:teuthology.orchestra.run.smithi012.stdout: Running scriptlet: libcephsqlite-2:19.3.0-4854.g8f219c47.el9.x86_64 14/165 2024-09-10T23:53:17.556 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : libradosstriper1-2:19.3.0-4854.g8f219c47.el9.x86 15/165 2024-09-10T23:53:17.663 INFO:teuthology.orchestra.run.smithi012.stdout: Running scriptlet: libradosstriper1-2:19.3.0-4854.g8f219c47.el9.x86 15/165 2024-09-10T23:53:17.682 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 16/165 2024-09-10T23:53:17.723 INFO:teuthology.orchestra.run.smithi012.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 16/165 2024-09-10T23:53:17.787 INFO:teuthology.orchestra.run.smithi165.stdout: Preparing : 1/1 2024-09-10T23:53:17.909 INFO:teuthology.orchestra.run.smithi165.stdout: Upgrading : openssl-libs-1:3.2.2-6.el9.x86_64 1/165 2024-09-10T23:53:17.934 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: ceph-common-2:19.3.0-4854.g8f219c47.el9.x86_64 108/165 2024-09-10T23:53:17.947 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/165 2024-09-10T23:53:17.968 INFO:teuthology.orchestra.run.smithi012.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 17/165 2024-09-10T23:53:17.973 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : ceph-common-2:19.3.0-4854.g8f219c47.el9.x86_64 108/165 2024-09-10T23:53:18.001 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/165 2024-09-10T23:53:18.034 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/165 2024-09-10T23:53:18.060 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/165 2024-09-10T23:53:18.102 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/165 2024-09-10T23:53:18.136 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 17/165 2024-09-10T23:53:18.313 INFO:teuthology.orchestra.run.smithi012.stdout: Upgrading : NetworkManager-libnm-1:1.49.90-1.el9.x86_64 18/165 2024-09-10T23:53:18.437 INFO:teuthology.orchestra.run.smithi012.stdout: Running scriptlet: NetworkManager-1:1.49.90-1.el9.x86_64 19/165 2024-09-10T23:53:18.492 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/165 2024-09-10T23:53:18.538 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: ceph-common-2:19.3.0-4854.g8f219c47.el9.x86_64 108/165 2024-09-10T23:53:18.565 INFO:teuthology.orchestra.run.smithi012.stdout: Upgrading : NetworkManager-1:1.49.90-1.el9.x86_64 19/165 2024-09-10T23:53:18.576 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : ceph-base-2:19.3.0-4854.g8f219c47.el9.x86_64 109/165 2024-09-10T23:53:18.672 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: ceph-base-2:19.3.0-4854.g8f219c47.el9.x86_64 109/165 2024-09-10T23:53:18.672 INFO:teuthology.orchestra.run.smithi178.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-10T23:53:18.672 INFO:teuthology.orchestra.run.smithi178.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-10T23:53:18.672 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-09-10T23:53:18.686 INFO:teuthology.orchestra.run.smithi165.stdout: Upgrading : librados2-2:19.3.0-4854.g8f219c47.el9.x86_64 8/165 2024-09-10T23:53:18.702 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : ceph-selinux-2:19.3.0-4854.g8f219c47.el9.x86_64 110/165 2024-09-10T23:53:18.842 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: librados2-2:19.3.0-4854.g8f219c47.el9.x86_64 8/165 2024-09-10T23:53:18.859 INFO:teuthology.orchestra.run.smithi012.stdout: Running scriptlet: NetworkManager-1:1.49.90-1.el9.x86_64 19/165 2024-09-10T23:53:18.879 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : libcephfs2-2:19.3.0-4854.g8f219c47.el9.x86_64 9/165 2024-09-10T23:53:18.971 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: libcephfs2-2:19.3.0-4854.g8f219c47.el9.x86_64 9/165 2024-09-10T23:53:19.000 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-rados-2:19.3.0-4854.g8f219c47.el9.x86_64 10/165 2024-09-10T23:53:19.012 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : re2-1:20211101-20.el9.x86_64 20/165 2024-09-10T23:53:19.029 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : postgresql-private-libs-13.16-1.el9.x86_64 11/165 2024-09-10T23:53:19.046 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 21/165 2024-09-10T23:53:19.091 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : liboath-2.6.7-2.el9.x86_64 22/165 2024-09-10T23:53:19.176 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 23/165 2024-09-10T23:53:19.182 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : libnbd-1.20.2-2.el9.x86_64 12/165 2024-09-10T23:53:19.231 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : postgresql-13.16-1.el9.x86_64 13/165 2024-09-10T23:53:19.236 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 24/165 2024-09-10T23:53:19.259 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : libcephsqlite-2:19.3.0-4854.g8f219c47.el9.x86_64 14/165 2024-09-10T23:53:19.282 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-packaging-20.9-5.el9.noarch 25/165 2024-09-10T23:53:19.337 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 26/165 2024-09-10T23:53:19.363 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: libcephsqlite-2:19.3.0-4854.g8f219c47.el9.x86_64 14/165 2024-09-10T23:53:19.375 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 27/165 2024-09-10T23:53:19.385 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : libradosstriper1-2:19.3.0-4854.g8f219c47.el9.x86 15/165 2024-09-10T23:53:19.408 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : lua-5.4.4-4.el9.x86_64 28/165 2024-09-10T23:53:19.448 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: libradosstriper1-2:19.3.0-4854.g8f219c47.el9.x86 15/165 2024-09-10T23:53:19.470 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 16/165 2024-09-10T23:53:19.490 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 16/165 2024-09-10T23:53:19.500 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 29/165 2024-09-10T23:53:19.596 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 30/165 2024-09-10T23:53:19.637 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 31/165 2024-09-10T23:53:19.671 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 17/165 2024-09-10T23:53:19.684 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 32/165 2024-09-10T23:53:19.778 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 33/165 2024-09-10T23:53:19.796 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 17/165 2024-09-10T23:53:19.816 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-ceph-common-2:19.3.0-4854.g8f219c47.el9. 34/165 2024-09-10T23:53:19.860 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-ceph-argparse-2:19.3.0-4854.g8f219c47.el 35/165 2024-09-10T23:53:19.888 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-cephfs-2:19.3.0-4854.g8f219c47.el9.x86_6 36/165 2024-09-10T23:53:19.902 INFO:teuthology.orchestra.run.smithi165.stdout: Upgrading : NetworkManager-libnm-1:1.49.90-1.el9.x86_64 18/165 2024-09-10T23:53:19.984 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 37/165 2024-09-10T23:53:19.998 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: NetworkManager-1:1.49.90-1.el9.x86_64 19/165 2024-09-10T23:53:20.049 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : luarocks-3.9.2-5.el9.noarch 38/165 2024-09-10T23:53:20.087 INFO:teuthology.orchestra.run.smithi165.stdout: Upgrading : NetworkManager-1:1.49.90-1.el9.x86_64 19/165 2024-09-10T23:53:20.145 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 39/165 2024-09-10T23:53:20.313 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: NetworkManager-1:1.49.90-1.el9.x86_64 19/165 2024-09-10T23:53:20.321 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 40/165 2024-09-10T23:53:20.388 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 41/165 2024-09-10T23:53:20.416 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : re2-1:20211101-20.el9.x86_64 20/165 2024-09-10T23:53:20.449 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 21/165 2024-09-10T23:53:20.484 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : liboath-2.6.7-2.el9.x86_64 22/165 2024-09-10T23:53:20.566 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 23/165 2024-09-10T23:53:20.618 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 24/165 2024-09-10T23:53:20.650 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-packaging-20.9-5.el9.noarch 25/165 2024-09-10T23:53:20.694 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 26/165 2024-09-10T23:53:20.727 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 27/165 2024-09-10T23:53:20.754 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : lua-5.4.4-4.el9.x86_64 28/165 2024-09-10T23:53:20.824 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 29/165 2024-09-10T23:53:20.885 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 30/165 2024-09-10T23:53:20.916 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 31/165 2024-09-10T23:53:20.951 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 32/165 2024-09-10T23:53:21.029 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 33/165 2024-09-10T23:53:21.062 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-ceph-common-2:19.3.0-4854.g8f219c47.el9. 34/165 2024-09-10T23:53:21.076 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-rsa-4.9-2.el9.noarch 42/165 2024-09-10T23:53:21.105 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-ceph-argparse-2:19.3.0-4854.g8f219c47.el 35/165 2024-09-10T23:53:21.138 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-cephfs-2:19.3.0-4854.g8f219c47.el9.x86_6 36/165 2024-09-10T23:53:21.182 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 43/165 2024-09-10T23:53:21.208 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 37/165 2024-09-10T23:53:21.230 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 44/165 2024-09-10T23:53:21.253 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : luarocks-3.9.2-5.el9.noarch 38/165 2024-09-10T23:53:21.293 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 45/165 2024-09-10T23:53:21.328 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 39/165 2024-09-10T23:53:21.347 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : librados-devel-2:19.3.0-4854.g8f219c47.el9.x86_6 46/165 2024-09-10T23:53:21.399 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 47/165 2024-09-10T23:53:21.468 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 40/165 2024-09-10T23:53:21.522 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 41/165 2024-09-10T23:53:21.601 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 48/165 2024-09-10T23:53:21.699 INFO:teuthology.orchestra.run.smithi012.stdout: Upgrading : openssl-1:3.2.2-6.el9.x86_64 49/165 2024-09-10T23:53:21.819 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 50/165 2024-09-10T23:53:21.888 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 51/165 2024-09-10T23:53:21.981 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 52/165 2024-09-10T23:53:22.042 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 53/165 2024-09-10T23:53:22.047 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-rsa-4.9-2.el9.noarch 42/165 2024-09-10T23:53:22.078 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 54/165 2024-09-10T23:53:22.134 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 43/165 2024-09-10T23:53:22.139 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 55/165 2024-09-10T23:53:22.172 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 44/165 2024-09-10T23:53:22.178 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 56/165 2024-09-10T23:53:22.215 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 45/165 2024-09-10T23:53:22.253 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : librados-devel-2:19.3.0-4854.g8f219c47.el9.x86_6 46/165 2024-09-10T23:53:22.261 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 57/165 2024-09-10T23:53:22.295 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 47/165 2024-09-10T23:53:22.316 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 58/165 2024-09-10T23:53:22.452 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 48/165 2024-09-10T23:53:22.498 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 59/165 2024-09-10T23:53:22.506 INFO:teuthology.orchestra.run.smithi165.stdout: Upgrading : openssl-1:3.2.2-6.el9.x86_64 49/165 2024-09-10T23:53:22.548 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 60/165 2024-09-10T23:53:22.592 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 61/165 2024-09-10T23:53:22.598 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 50/165 2024-09-10T23:53:22.662 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 51/165 2024-09-10T23:53:22.719 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 62/165 2024-09-10T23:53:22.742 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 52/165 2024-09-10T23:53:22.768 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 63/165 2024-09-10T23:53:22.775 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 53/165 2024-09-10T23:53:22.813 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 54/165 2024-09-10T23:53:22.813 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 64/165 2024-09-10T23:53:22.848 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 65/165 2024-09-10T23:53:22.857 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 55/165 2024-09-10T23:53:22.873 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 66/165 2024-09-10T23:53:22.894 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 56/165 2024-09-10T23:53:22.936 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 57/165 2024-09-10T23:53:22.979 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 58/165 2024-09-10T23:53:22.994 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 67/165 2024-09-10T23:53:23.121 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 59/165 2024-09-10T23:53:23.158 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 68/165 2024-09-10T23:53:23.170 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 60/165 2024-09-10T23:53:23.205 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 69/165 2024-09-10T23:53:23.211 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 61/165 2024-09-10T23:53:23.246 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 70/165 2024-09-10T23:53:23.291 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 71/165 2024-09-10T23:53:23.313 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 62/165 2024-09-10T23:53:23.337 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : uuid-1.6.2-55.el9.x86_64 72/165 2024-09-10T23:53:23.345 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 63/165 2024-09-10T23:53:23.374 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 73/165 2024-09-10T23:53:23.385 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 64/165 2024-09-10T23:53:23.410 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 74/165 2024-09-10T23:53:23.419 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 65/165 2024-09-10T23:53:23.444 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 66/165 2024-09-10T23:53:23.445 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 75/165 2024-09-10T23:53:23.478 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 76/165 2024-09-10T23:53:23.518 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 67/165 2024-09-10T23:53:23.555 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 77/165 2024-09-10T23:53:23.587 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 78/165 2024-09-10T23:53:23.645 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 68/165 2024-09-10T23:53:23.685 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 69/165 2024-09-10T23:53:23.692 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 79/165 2024-09-10T23:53:23.713 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 70/165 2024-09-10T23:53:23.747 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 80/165 2024-09-10T23:53:23.755 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 71/165 2024-09-10T23:53:23.791 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 81/165 2024-09-10T23:53:23.803 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : uuid-1.6.2-55.el9.x86_64 72/165 2024-09-10T23:53:23.835 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 73/165 2024-09-10T23:53:23.865 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 74/165 2024-09-10T23:53:23.912 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 82/165 2024-09-10T23:53:23.919 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 75/165 2024-09-10T23:53:23.958 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 76/165 2024-09-10T23:53:23.981 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 83/165 2024-09-10T23:53:24.000 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 77/165 2024-09-10T23:53:24.034 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 78/165 2024-09-10T23:53:24.122 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 79/165 2024-09-10T23:53:24.165 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 80/165 2024-09-10T23:53:24.196 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 81/165 2024-09-10T23:53:24.307 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 82/165 2024-09-10T23:53:24.364 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 83/165 2024-09-10T23:53:24.958 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 84/165 2024-09-10T23:53:25.015 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 85/165 2024-09-10T23:53:25.048 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 86/165 2024-09-10T23:53:25.079 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 84/165 2024-09-10T23:53:25.128 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 85/165 2024-09-10T23:53:25.162 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 86/165 2024-09-10T23:53:25.171 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 87/165 2024-09-10T23:53:25.241 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 88/165 2024-09-10T23:53:25.260 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 87/165 2024-09-10T23:53:25.295 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 89/165 2024-09-10T23:53:25.317 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 88/165 2024-09-10T23:53:25.370 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 89/165 2024-09-10T23:53:25.407 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 90/165 2024-09-10T23:53:25.452 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 91/165 2024-09-10T23:53:25.483 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 90/165 2024-09-10T23:53:25.507 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 91/165 2024-09-10T23:53:25.691 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : openblas-0.3.26-2.el9.x86_64 92/165 2024-09-10T23:53:25.701 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : openblas-0.3.26-2.el9.x86_64 92/165 2024-09-10T23:53:25.722 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 93/165 2024-09-10T23:53:25.747 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 93/165 2024-09-10T23:53:25.804 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 94/165 2024-09-10T23:53:25.845 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 94/165 2024-09-10T23:53:26.484 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 95/165 2024-09-10T23:53:26.602 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 96/165 2024-09-10T23:53:26.724 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 95/165 2024-09-10T23:53:26.875 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 96/165 2024-09-10T23:53:27.824 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 97/165 2024-09-10T23:53:27.901 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 98/165 2024-09-10T23:53:27.929 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 99/165 2024-09-10T23:53:27.963 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : libuuid-devel-2.37.4-20.el9.x86_64 100/165 2024-09-10T23:53:28.281 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 101/165 2024-09-10T23:53:28.326 INFO:teuthology.orchestra.run.smithi165.stdout: Upgrading : librbd1-2:19.3.0-4854.g8f219c47.el9.x86_64 102/165 2024-09-10T23:53:28.416 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: librbd1-2:19.3.0-4854.g8f219c47.el9.x86_64 102/165 2024-09-10T23:53:28.447 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-rbd-2:19.3.0-4854.g8f219c47.el9.x86_64 103/165 2024-09-10T23:53:28.483 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 104/165 2024-09-10T23:53:28.568 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 97/165 2024-09-10T23:53:28.661 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 98/165 2024-09-10T23:53:28.698 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 99/165 2024-09-10T23:53:28.731 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : libuuid-devel-2.37.4-20.el9.x86_64 100/165 2024-09-10T23:53:28.973 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 105/165 2024-09-10T23:53:29.004 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : librgw2-2:19.3.0-4854.g8f219c47.el9.x86_64 106/165 2024-09-10T23:53:29.066 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: librgw2-2:19.3.0-4854.g8f219c47.el9.x86_64 106/165 2024-09-10T23:53:29.089 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-rgw-2:19.3.0-4854.g8f219c47.el9.x86_64 107/165 2024-09-10T23:53:29.110 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 101/165 2024-09-10T23:53:29.150 INFO:teuthology.orchestra.run.smithi012.stdout: Upgrading : librbd1-2:19.3.0-4854.g8f219c47.el9.x86_64 102/165 2024-09-10T23:53:29.263 INFO:teuthology.orchestra.run.smithi012.stdout: Running scriptlet: librbd1-2:19.3.0-4854.g8f219c47.el9.x86_64 102/165 2024-09-10T23:53:29.297 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-rbd-2:19.3.0-4854.g8f219c47.el9.x86_64 103/165 2024-09-10T23:53:29.347 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 104/165 2024-09-10T23:53:29.919 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 105/165 2024-09-10T23:53:29.953 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : librgw2-2:19.3.0-4854.g8f219c47.el9.x86_64 106/165 2024-09-10T23:53:30.027 INFO:teuthology.orchestra.run.smithi012.stdout: Running scriptlet: librgw2-2:19.3.0-4854.g8f219c47.el9.x86_64 106/165 2024-09-10T23:53:30.054 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-rgw-2:19.3.0-4854.g8f219c47.el9.x86_64 107/165 2024-09-10T23:53:30.487 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: ceph-selinux-2:19.3.0-4854.g8f219c47.el9.x86_64 110/165 2024-09-10T23:53:30.487 INFO:teuthology.orchestra.run.smithi178.stdout:skipping the directory /sys 2024-09-10T23:53:30.487 INFO:teuthology.orchestra.run.smithi178.stdout:skipping the directory /proc 2024-09-10T23:53:30.488 INFO:teuthology.orchestra.run.smithi178.stdout:skipping the directory /mnt 2024-09-10T23:53:30.488 INFO:teuthology.orchestra.run.smithi178.stdout:skipping the directory /var/tmp 2024-09-10T23:53:30.488 INFO:teuthology.orchestra.run.smithi178.stdout:skipping the directory /home 2024-09-10T23:53:30.488 INFO:teuthology.orchestra.run.smithi178.stdout:skipping the directory /root 2024-09-10T23:53:30.488 INFO:teuthology.orchestra.run.smithi178.stdout:skipping the directory /tmp 2024-09-10T23:53:30.488 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-09-10T23:53:30.662 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : ceph-volume-2:19.3.0-4854.g8f219c47.el9.noarch 111/165 2024-09-10T23:53:30.678 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: ceph-volume-2:19.3.0-4854.g8f219c47.el9.noarch 111/165 2024-09-10T23:53:30.678 INFO:teuthology.orchestra.run.smithi178.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T23:53:30.679 INFO:teuthology.orchestra.run.smithi178.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-10T23:53:30.679 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-09-10T23:53:31.385 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: ceph-common-2:19.3.0-4854.g8f219c47.el9.x86_64 108/165 2024-09-10T23:53:31.427 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : ceph-common-2:19.3.0-4854.g8f219c47.el9.x86_64 108/165 2024-09-10T23:53:31.978 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: ceph-common-2:19.3.0-4854.g8f219c47.el9.x86_64 108/165 2024-09-10T23:53:32.023 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : ceph-base-2:19.3.0-4854.g8f219c47.el9.x86_64 109/165 2024-09-10T23:53:32.112 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: ceph-base-2:19.3.0-4854.g8f219c47.el9.x86_64 109/165 2024-09-10T23:53:32.113 INFO:teuthology.orchestra.run.smithi165.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-10T23:53:32.113 INFO:teuthology.orchestra.run.smithi165.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-10T23:53:32.113 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-09-10T23:53:32.140 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : ceph-selinux-2:19.3.0-4854.g8f219c47.el9.x86_64 110/165 2024-09-10T23:53:32.190 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : ceph-osd-2:19.3.0-4854.g8f219c47.el9.x86_64 112/165 2024-09-10T23:53:32.217 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: ceph-osd-2:19.3.0-4854.g8f219c47.el9.x86_64 112/165 2024-09-10T23:53:32.217 INFO:teuthology.orchestra.run.smithi178.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T23:53:32.217 INFO:teuthology.orchestra.run.smithi178.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-10T23:53:32.217 INFO:teuthology.orchestra.run.smithi178.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-10T23:53:32.218 INFO:teuthology.orchestra.run.smithi178.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-10T23:53:32.218 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-09-10T23:53:32.441 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : ceph-mds-2:19.3.0-4854.g8f219c47.el9.x86_64 113/165 2024-09-10T23:53:32.464 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: ceph-mds-2:19.3.0-4854.g8f219c47.el9.x86_64 113/165 2024-09-10T23:53:32.464 INFO:teuthology.orchestra.run.smithi178.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T23:53:32.464 INFO:teuthology.orchestra.run.smithi178.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-10T23:53:32.464 INFO:teuthology.orchestra.run.smithi178.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-10T23:53:32.464 INFO:teuthology.orchestra.run.smithi178.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-10T23:53:32.465 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-09-10T23:53:32.775 INFO:teuthology.orchestra.run.smithi012.stdout: Running scriptlet: ceph-common-2:19.3.0-4854.g8f219c47.el9.x86_64 108/165 2024-09-10T23:53:32.817 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : ceph-common-2:19.3.0-4854.g8f219c47.el9.x86_64 108/165 2024-09-10T23:53:32.908 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : ceph-mon-2:19.3.0-4854.g8f219c47.el9.x86_64 114/165 2024-09-10T23:53:32.932 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: ceph-mon-2:19.3.0-4854.g8f219c47.el9.x86_64 114/165 2024-09-10T23:53:32.932 INFO:teuthology.orchestra.run.smithi178.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T23:53:32.932 INFO:teuthology.orchestra.run.smithi178.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-10T23:53:32.932 INFO:teuthology.orchestra.run.smithi178.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-10T23:53:32.932 INFO:teuthology.orchestra.run.smithi178.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-10T23:53:32.932 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-09-10T23:53:33.038 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : automake-1.16.2-8.el9.noarch 115/165 2024-09-10T23:53:33.086 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-ply-3.11-14.el9.noarch 116/165 2024-09-10T23:53:33.141 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 117/165 2024-09-10T23:53:33.333 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 118/165 2024-09-10T23:53:33.384 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 119/165 2024-09-10T23:53:33.453 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 120/165 2024-09-10T23:53:33.501 INFO:teuthology.orchestra.run.smithi012.stdout: Running scriptlet: ceph-common-2:19.3.0-4854.g8f219c47.el9.x86_64 108/165 2024-09-10T23:53:33.542 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 121/165 2024-09-10T23:53:33.545 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : ceph-base-2:19.3.0-4854.g8f219c47.el9.x86_64 109/165 2024-09-10T23:53:33.662 INFO:teuthology.orchestra.run.smithi012.stdout: Running scriptlet: ceph-base-2:19.3.0-4854.g8f219c47.el9.x86_64 109/165 2024-09-10T23:53:33.663 INFO:teuthology.orchestra.run.smithi012.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-10T23:53:33.663 INFO:teuthology.orchestra.run.smithi012.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-10T23:53:33.663 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-10T23:53:33.680 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 122/165 2024-09-10T23:53:33.705 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : ceph-selinux-2:19.3.0-4854.g8f219c47.el9.x86_64 110/165 2024-09-10T23:53:33.726 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 123/165 2024-09-10T23:53:33.757 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 124/165 2024-09-10T23:53:33.786 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : fuse-2.9.9-16.el9.x86_64 125/165 2024-09-10T23:53:34.233 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: cephadm-2:19.3.0-4854.g8f219c47.el9.noarch 126/165 2024-09-10T23:53:34.259 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : cephadm-2:19.3.0-4854.g8f219c47.el9.noarch 126/165 2024-09-10T23:53:34.291 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : ceph-prometheus-alerts-2:19.3.0-4854.g8f219c47.e 127/165 2024-09-10T23:53:34.356 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : ceph-grafana-dashboards-2:19.3.0-4854.g8f219c47. 128/165 2024-09-10T23:53:34.382 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : ceph-mgr-cephadm-2:19.3.0-4854.g8f219c47.el9.noa 129/165 2024-09-10T23:53:37.537 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.3.0-4854.g8f219c47.el9.noa 129/165 2024-09-10T23:53:37.597 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : ceph-mgr-dashboard-2:19.3.0-4854.g8f219c47.el9.n 130/165 2024-09-10T23:53:38.554 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.3.0-4854.g8f219c47.el9.n 130/165 2024-09-10T23:53:38.589 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : ceph-mgr-diskprediction-local-2:19.3.0-4854.g8f2 131/165 2024-09-10T23:53:38.700 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.3.0-4854.g8f2 131/165 2024-09-10T23:53:38.866 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : ceph-mgr-modules-core-2:19.3.0-4854.g8f219c47.el 132/165 2024-09-10T23:53:38.890 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : ceph-mgr-2:19.3.0-4854.g8f219c47.el9.x86_64 133/165 2024-09-10T23:53:38.915 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: ceph-mgr-2:19.3.0-4854.g8f219c47.el9.x86_64 133/165 2024-09-10T23:53:38.915 INFO:teuthology.orchestra.run.smithi178.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T23:53:38.915 INFO:teuthology.orchestra.run.smithi178.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-10T23:53:38.915 INFO:teuthology.orchestra.run.smithi178.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-10T23:53:38.916 INFO:teuthology.orchestra.run.smithi178.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-10T23:53:38.916 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-09-10T23:53:38.966 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : ceph-mgr-rook-2:19.3.0-4854.g8f219c47.el9.noarch 134/165 2024-09-10T23:53:38.978 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: ceph-mgr-rook-2:19.3.0-4854.g8f219c47.el9.noarch 134/165 2024-09-10T23:53:39.075 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : ceph-2:19.3.0-4854.g8f219c47.el9.x86_64 135/165 2024-09-10T23:53:39.130 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : ceph-fuse-2:19.3.0-4854.g8f219c47.el9.x86_64 136/165 2024-09-10T23:53:40.082 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : libtool-2.4.6-46.el9.x86_64 137/165 2024-09-10T23:53:40.129 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : ceph-radosgw-2:19.3.0-4854.g8f219c47.el9.x86_64 138/165 2024-09-10T23:53:40.151 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: ceph-radosgw-2:19.3.0-4854.g8f219c47.el9.x86_64 138/165 2024-09-10T23:53:40.151 INFO:teuthology.orchestra.run.smithi178.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T23:53:40.151 INFO:teuthology.orchestra.run.smithi178.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-10T23:53:40.151 INFO:teuthology.orchestra.run.smithi178.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-10T23:53:40.151 INFO:teuthology.orchestra.run.smithi178.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-10T23:53:40.152 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-09-10T23:53:40.194 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : ceph-immutable-object-cache-2:19.3.0-4854.g8f219 139/165 2024-09-10T23:53:40.217 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.3.0-4854.g8f219 139/165 2024-09-10T23:53:40.217 INFO:teuthology.orchestra.run.smithi178.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T23:53:40.217 INFO:teuthology.orchestra.run.smithi178.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-10T23:53:40.217 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-09-10T23:53:40.262 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : cephfs-mirror-2:19.3.0-4854.g8f219c47.el9.x86_64 140/165 2024-09-10T23:53:40.285 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: cephfs-mirror-2:19.3.0-4854.g8f219c47.el9.x86_64 140/165 2024-09-10T23:53:40.285 INFO:teuthology.orchestra.run.smithi178.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T23:53:40.285 INFO:teuthology.orchestra.run.smithi178.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-09-10T23:53:40.285 INFO:teuthology.orchestra.run.smithi178.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-10T23:53:40.286 INFO:teuthology.orchestra.run.smithi178.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-10T23:53:40.286 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-09-10T23:53:40.595 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : rbd-mirror-2:19.3.0-4854.g8f219c47.el9.x86_64 141/165 2024-09-10T23:53:40.619 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: rbd-mirror-2:19.3.0-4854.g8f219c47.el9.x86_64 141/165 2024-09-10T23:53:40.619 INFO:teuthology.orchestra.run.smithi178.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T23:53:40.619 INFO:teuthology.orchestra.run.smithi178.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-10T23:53:40.619 INFO:teuthology.orchestra.run.smithi178.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-10T23:53:40.619 INFO:teuthology.orchestra.run.smithi178.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-10T23:53:40.619 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-09-10T23:53:44.024 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: ceph-selinux-2:19.3.0-4854.g8f219c47.el9.x86_64 110/165 2024-09-10T23:53:44.024 INFO:teuthology.orchestra.run.smithi165.stdout:skipping the directory /sys 2024-09-10T23:53:44.024 INFO:teuthology.orchestra.run.smithi165.stdout:skipping the directory /proc 2024-09-10T23:53:44.025 INFO:teuthology.orchestra.run.smithi165.stdout:skipping the directory /mnt 2024-09-10T23:53:44.025 INFO:teuthology.orchestra.run.smithi165.stdout:skipping the directory /var/tmp 2024-09-10T23:53:44.025 INFO:teuthology.orchestra.run.smithi165.stdout:skipping the directory /home 2024-09-10T23:53:44.025 INFO:teuthology.orchestra.run.smithi165.stdout:skipping the directory /root 2024-09-10T23:53:44.025 INFO:teuthology.orchestra.run.smithi165.stdout:skipping the directory /tmp 2024-09-10T23:53:44.025 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-09-10T23:53:44.205 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : ceph-volume-2:19.3.0-4854.g8f219c47.el9.noarch 111/165 2024-09-10T23:53:44.221 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: ceph-volume-2:19.3.0-4854.g8f219c47.el9.noarch 111/165 2024-09-10T23:53:44.221 INFO:teuthology.orchestra.run.smithi165.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T23:53:44.221 INFO:teuthology.orchestra.run.smithi165.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-10T23:53:44.221 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-09-10T23:53:45.274 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : ceph-test-2:19.3.0-4854.g8f219c47.el9.x86_64 142/165 2024-09-10T23:53:45.315 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : rbd-fuse-2:19.3.0-4854.g8f219c47.el9.x86_64 143/165 2024-09-10T23:53:45.356 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : rbd-nbd-2:19.3.0-4854.g8f219c47.el9.x86_64 144/165 2024-09-10T23:53:45.388 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : xfsprogs-devel-6.4.0-4.el9.x86_64 145/165 2024-09-10T23:53:45.415 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 146/165 2024-09-10T23:53:45.594 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : cephfs-shell-2:19.3.0-4854.g8f219c47.el9.x86_64 147/165 2024-09-10T23:53:45.637 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : postgresql-contrib-13.16-1.el9.x86_64 148/165 2024-09-10T23:53:45.676 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : libcephfs-devel-2:19.3.0-4854.g8f219c47.el9.x86_ 149/165 2024-09-10T23:53:45.705 INFO:teuthology.orchestra.run.smithi178.stdout: Upgrading : NetworkManager-team-1:1.49.90-1.el9.x86_64 150/165 2024-09-10T23:53:45.742 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : ceph-osd-2:19.3.0-4854.g8f219c47.el9.x86_64 112/165 2024-09-10T23:53:45.752 INFO:teuthology.orchestra.run.smithi178.stdout: Upgrading : NetworkManager-tui-1:1.49.90-1.el9.x86_64 151/165 2024-09-10T23:53:45.769 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: ceph-osd-2:19.3.0-4854.g8f219c47.el9.x86_64 112/165 2024-09-10T23:53:45.770 INFO:teuthology.orchestra.run.smithi165.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T23:53:45.770 INFO:teuthology.orchestra.run.smithi165.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-10T23:53:45.770 INFO:teuthology.orchestra.run.smithi165.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-10T23:53:45.770 INFO:teuthology.orchestra.run.smithi165.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-10T23:53:45.770 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-09-10T23:53:46.002 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : ceph-mds-2:19.3.0-4854.g8f219c47.el9.x86_64 113/165 2024-09-10T23:53:46.025 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: ceph-mds-2:19.3.0-4854.g8f219c47.el9.x86_64 113/165 2024-09-10T23:53:46.026 INFO:teuthology.orchestra.run.smithi165.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T23:53:46.026 INFO:teuthology.orchestra.run.smithi165.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-10T23:53:46.026 INFO:teuthology.orchestra.run.smithi165.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-10T23:53:46.026 INFO:teuthology.orchestra.run.smithi165.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-10T23:53:46.026 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-09-10T23:53:46.291 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 152/165 2024-09-10T23:53:46.333 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : postgresql-server-13.16-1.el9.x86_64 152/165 2024-09-10T23:53:46.354 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 152/165 2024-09-10T23:53:46.485 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : ceph-mon-2:19.3.0-4854.g8f219c47.el9.x86_64 114/165 2024-09-10T23:53:46.508 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: ceph-mon-2:19.3.0-4854.g8f219c47.el9.x86_64 114/165 2024-09-10T23:53:46.509 INFO:teuthology.orchestra.run.smithi165.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T23:53:46.509 INFO:teuthology.orchestra.run.smithi165.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-10T23:53:46.509 INFO:teuthology.orchestra.run.smithi165.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-10T23:53:46.509 INFO:teuthology.orchestra.run.smithi165.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-10T23:53:46.509 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-09-10T23:53:46.615 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : automake-1.16.2-8.el9.noarch 115/165 2024-09-10T23:53:46.673 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-ply-3.11-14.el9.noarch 116/165 2024-09-10T23:53:46.728 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 117/165 2024-09-10T23:53:46.913 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 118/165 2024-09-10T23:53:46.962 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 119/165 2024-09-10T23:53:47.031 INFO:teuthology.orchestra.run.smithi012.stdout: Running scriptlet: ceph-selinux-2:19.3.0-4854.g8f219c47.el9.x86_64 110/165 2024-09-10T23:53:47.031 INFO:teuthology.orchestra.run.smithi012.stdout:skipping the directory /sys 2024-09-10T23:53:47.031 INFO:teuthology.orchestra.run.smithi012.stdout:skipping the directory /proc 2024-09-10T23:53:47.031 INFO:teuthology.orchestra.run.smithi012.stdout:skipping the directory /mnt 2024-09-10T23:53:47.031 INFO:teuthology.orchestra.run.smithi012.stdout:skipping the directory /var/tmp 2024-09-10T23:53:47.031 INFO:teuthology.orchestra.run.smithi012.stdout:skipping the directory /home 2024-09-10T23:53:47.032 INFO:teuthology.orchestra.run.smithi012.stdout:skipping the directory /root 2024-09-10T23:53:47.032 INFO:teuthology.orchestra.run.smithi012.stdout:skipping the directory /tmp 2024-09-10T23:53:47.032 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-10T23:53:47.039 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 120/165 2024-09-10T23:53:47.128 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 121/165 2024-09-10T23:53:47.182 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : cephfs-top-2:19.3.0-4854.g8f219c47.el9.noarch 153/165 2024-09-10T23:53:47.257 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : ceph-volume-2:19.3.0-4854.g8f219c47.el9.noarch 111/165 2024-09-10T23:53:47.271 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : openssl-devel-1:3.2.2-6.el9.x86_64 154/165 2024-09-10T23:53:47.275 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 122/165 2024-09-10T23:53:47.296 INFO:teuthology.orchestra.run.smithi012.stdout: Running scriptlet: ceph-volume-2:19.3.0-4854.g8f219c47.el9.noarch 111/165 2024-09-10T23:53:47.296 INFO:teuthology.orchestra.run.smithi012.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T23:53:47.296 INFO:teuthology.orchestra.run.smithi012.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-10T23:53:47.296 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-10T23:53:47.311 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 155/165 2024-09-10T23:53:47.320 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 123/165 2024-09-10T23:53:47.361 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 124/165 2024-09-10T23:53:47.389 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : fuse-2.9.9-16.el9.x86_64 125/165 2024-09-10T23:53:47.401 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : flex-2.6.4-9.el9.x86_64 156/165 2024-09-10T23:53:47.432 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : bison-3.7.4-5.el9.x86_64 157/165 2024-09-10T23:53:47.457 INFO:teuthology.orchestra.run.smithi178.stdout: Cleanup : NetworkManager-tui-1:1.48.8-1.el9.x86_64 158/165 2024-09-10T23:53:47.483 INFO:teuthology.orchestra.run.smithi178.stdout: Cleanup : openssl-1:3.2.2-4.el9.x86_64 159/165 2024-09-10T23:53:47.484 INFO:teuthology.orchestra.run.smithi178.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 160/165 2024-09-10T23:53:47.559 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 160/165 2024-09-10T23:53:47.559 INFO:teuthology.orchestra.run.smithi178.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 161/165 2024-09-10T23:53:47.626 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 161/165 2024-09-10T23:53:47.651 INFO:teuthology.orchestra.run.smithi178.stdout: Cleanup : NetworkManager-team-1:1.48.8-1.el9.x86_64 162/165 2024-09-10T23:53:47.656 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 163/165 2024-09-10T23:53:47.657 INFO:teuthology.orchestra.run.smithi178.stdout: Cleanup : NetworkManager-1:1.48.8-1.el9.x86_64 163/165 2024-09-10T23:53:47.818 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: cephadm-2:19.3.0-4854.g8f219c47.el9.noarch 126/165 2024-09-10T23:53:47.837 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 163/165 2024-09-10T23:53:47.837 INFO:teuthology.orchestra.run.smithi178.stdout: Cleanup : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 164/165 2024-09-10T23:53:47.846 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : cephadm-2:19.3.0-4854.g8f219c47.el9.noarch 126/165 2024-09-10T23:53:47.878 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : ceph-prometheus-alerts-2:19.3.0-4854.g8f219c47.e 127/165 2024-09-10T23:53:47.912 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: NetworkManager-libnm-1:1.48.8-1.el9.x86_64 164/165 2024-09-10T23:53:47.944 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : ceph-grafana-dashboards-2:19.3.0-4854.g8f219c47. 128/165 2024-09-10T23:53:47.977 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : ceph-mgr-cephadm-2:19.3.0-4854.g8f219c47.el9.noa 129/165 2024-09-10T23:53:48.017 INFO:teuthology.orchestra.run.smithi178.stdout: Cleanup : openssl-libs-1:3.2.2-4.el9.x86_64 165/165 2024-09-10T23:53:48.993 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : ceph-osd-2:19.3.0-4854.g8f219c47.el9.x86_64 112/165 2024-09-10T23:53:49.052 INFO:teuthology.orchestra.run.smithi012.stdout: Running scriptlet: ceph-osd-2:19.3.0-4854.g8f219c47.el9.x86_64 112/165 2024-09-10T23:53:49.052 INFO:teuthology.orchestra.run.smithi012.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T23:53:49.052 INFO:teuthology.orchestra.run.smithi012.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-10T23:53:49.052 INFO:teuthology.orchestra.run.smithi012.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-10T23:53:49.052 INFO:teuthology.orchestra.run.smithi012.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-10T23:53:49.052 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-10T23:53:49.310 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : ceph-mds-2:19.3.0-4854.g8f219c47.el9.x86_64 113/165 2024-09-10T23:53:49.362 INFO:teuthology.orchestra.run.smithi012.stdout: Running scriptlet: ceph-mds-2:19.3.0-4854.g8f219c47.el9.x86_64 113/165 2024-09-10T23:53:49.362 INFO:teuthology.orchestra.run.smithi012.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T23:53:49.362 INFO:teuthology.orchestra.run.smithi012.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-10T23:53:49.362 INFO:teuthology.orchestra.run.smithi012.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-10T23:53:49.362 INFO:teuthology.orchestra.run.smithi012.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-10T23:53:49.362 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-10T23:53:49.894 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : ceph-mon-2:19.3.0-4854.g8f219c47.el9.x86_64 114/165 2024-09-10T23:53:49.945 INFO:teuthology.orchestra.run.smithi012.stdout: Running scriptlet: ceph-mon-2:19.3.0-4854.g8f219c47.el9.x86_64 114/165 2024-09-10T23:53:49.946 INFO:teuthology.orchestra.run.smithi012.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T23:53:49.946 INFO:teuthology.orchestra.run.smithi012.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-10T23:53:49.946 INFO:teuthology.orchestra.run.smithi012.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-10T23:53:49.946 INFO:teuthology.orchestra.run.smithi012.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-10T23:53:49.946 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-10T23:53:50.079 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : automake-1.16.2-8.el9.noarch 115/165 2024-09-10T23:53:50.140 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-ply-3.11-14.el9.noarch 116/165 2024-09-10T23:53:50.213 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 117/165 2024-09-10T23:53:50.453 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 118/165 2024-09-10T23:53:50.516 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 119/165 2024-09-10T23:53:50.606 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 120/165 2024-09-10T23:53:50.724 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 121/165 2024-09-10T23:53:50.905 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 122/165 2024-09-10T23:53:51.019 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 123/165 2024-09-10T23:53:51.250 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.3.0-4854.g8f219c47.el9.noa 129/165 2024-09-10T23:53:51.250 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : ceph-mgr-dashboard-2:19.3.0-4854.g8f219c47.el9.n 130/165 2024-09-10T23:53:51.250 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 124/165 2024-09-10T23:53:51.250 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : fuse-2.9.9-16.el9.x86_64 125/165 2024-09-10T23:53:51.608 INFO:teuthology.orchestra.run.smithi012.stdout: Running scriptlet: cephadm-2:19.3.0-4854.g8f219c47.el9.noarch 126/165 2024-09-10T23:53:51.646 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : cephadm-2:19.3.0-4854.g8f219c47.el9.noarch 126/165 2024-09-10T23:53:51.714 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : ceph-prometheus-alerts-2:19.3.0-4854.g8f219c47.e 127/165 2024-09-10T23:53:51.798 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : ceph-grafana-dashboards-2:19.3.0-4854.g8f219c47. 128/165 2024-09-10T23:53:51.835 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : ceph-mgr-cephadm-2:19.3.0-4854.g8f219c47.el9.noa 129/165 2024-09-10T23:53:52.184 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.3.0-4854.g8f219c47.el9.n 130/165 2024-09-10T23:53:52.226 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : ceph-mgr-diskprediction-local-2:19.3.0-4854.g8f2 131/165 2024-09-10T23:53:52.340 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.3.0-4854.g8f2 131/165 2024-09-10T23:53:52.505 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : ceph-mgr-modules-core-2:19.3.0-4854.g8f219c47.el 132/165 2024-09-10T23:53:52.561 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : ceph-mgr-2:19.3.0-4854.g8f219c47.el9.x86_64 133/165 2024-09-10T23:53:52.585 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: ceph-mgr-2:19.3.0-4854.g8f219c47.el9.x86_64 133/165 2024-09-10T23:53:52.585 INFO:teuthology.orchestra.run.smithi165.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T23:53:52.585 INFO:teuthology.orchestra.run.smithi165.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-10T23:53:52.585 INFO:teuthology.orchestra.run.smithi165.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-10T23:53:52.586 INFO:teuthology.orchestra.run.smithi165.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-10T23:53:52.586 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-09-10T23:53:52.637 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : ceph-mgr-rook-2:19.3.0-4854.g8f219c47.el9.noarch 134/165 2024-09-10T23:53:52.649 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: ceph-mgr-rook-2:19.3.0-4854.g8f219c47.el9.noarch 134/165 2024-09-10T23:53:52.747 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : ceph-2:19.3.0-4854.g8f219c47.el9.x86_64 135/165 2024-09-10T23:53:52.794 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : ceph-fuse-2:19.3.0-4854.g8f219c47.el9.x86_64 136/165 2024-09-10T23:53:53.743 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : libtool-2.4.6-46.el9.x86_64 137/165 2024-09-10T23:53:53.773 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : ceph-radosgw-2:19.3.0-4854.g8f219c47.el9.x86_64 138/165 2024-09-10T23:53:53.797 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: ceph-radosgw-2:19.3.0-4854.g8f219c47.el9.x86_64 138/165 2024-09-10T23:53:53.797 INFO:teuthology.orchestra.run.smithi165.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T23:53:53.797 INFO:teuthology.orchestra.run.smithi165.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-10T23:53:53.797 INFO:teuthology.orchestra.run.smithi165.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-10T23:53:53.798 INFO:teuthology.orchestra.run.smithi165.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-10T23:53:53.798 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-09-10T23:53:53.841 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : ceph-immutable-object-cache-2:19.3.0-4854.g8f219 139/165 2024-09-10T23:53:53.864 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.3.0-4854.g8f219 139/165 2024-09-10T23:53:53.864 INFO:teuthology.orchestra.run.smithi165.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T23:53:53.864 INFO:teuthology.orchestra.run.smithi165.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-10T23:53:53.864 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-09-10T23:53:53.925 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : cephfs-mirror-2:19.3.0-4854.g8f219c47.el9.x86_64 140/165 2024-09-10T23:53:53.948 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: cephfs-mirror-2:19.3.0-4854.g8f219c47.el9.x86_64 140/165 2024-09-10T23:53:53.948 INFO:teuthology.orchestra.run.smithi165.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T23:53:53.949 INFO:teuthology.orchestra.run.smithi165.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-09-10T23:53:53.949 INFO:teuthology.orchestra.run.smithi165.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-10T23:53:53.949 INFO:teuthology.orchestra.run.smithi165.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-10T23:53:53.949 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-09-10T23:53:54.309 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : rbd-mirror-2:19.3.0-4854.g8f219c47.el9.x86_64 141/165 2024-09-10T23:53:54.332 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: rbd-mirror-2:19.3.0-4854.g8f219c47.el9.x86_64 141/165 2024-09-10T23:53:54.333 INFO:teuthology.orchestra.run.smithi165.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T23:53:54.333 INFO:teuthology.orchestra.run.smithi165.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-10T23:53:54.333 INFO:teuthology.orchestra.run.smithi165.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-10T23:53:54.333 INFO:teuthology.orchestra.run.smithi165.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-10T23:53:54.333 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-09-10T23:53:54.555 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: openssl-libs-1:3.2.2-4.el9.x86_64 165/165 2024-09-10T23:53:54.555 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : ceph-2:19.3.0-4854.g8f219c47.el9.x86_64 1/165 2024-09-10T23:53:54.555 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : ceph-base-2:19.3.0-4854.g8f219c47.el9.x86_64 2/165 2024-09-10T23:53:54.556 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : ceph-common-2:19.3.0-4854.g8f219c47.el9.x86_64 3/165 2024-09-10T23:53:54.556 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : ceph-fuse-2:19.3.0-4854.g8f219c47.el9.x86_64 4/165 2024-09-10T23:53:54.556 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : ceph-immutable-object-cache-2:19.3.0-4854.g8f219 5/165 2024-09-10T23:53:54.556 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : ceph-mds-2:19.3.0-4854.g8f219c47.el9.x86_64 6/165 2024-09-10T23:53:54.556 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : ceph-mgr-2:19.3.0-4854.g8f219c47.el9.x86_64 7/165 2024-09-10T23:53:54.556 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : ceph-mon-2:19.3.0-4854.g8f219c47.el9.x86_64 8/165 2024-09-10T23:53:54.556 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : ceph-osd-2:19.3.0-4854.g8f219c47.el9.x86_64 9/165 2024-09-10T23:53:54.556 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : ceph-radosgw-2:19.3.0-4854.g8f219c47.el9.x86_64 10/165 2024-09-10T23:53:54.556 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : ceph-selinux-2:19.3.0-4854.g8f219c47.el9.x86_64 11/165 2024-09-10T23:53:54.556 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : ceph-test-2:19.3.0-4854.g8f219c47.el9.x86_64 12/165 2024-09-10T23:53:54.556 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : cephfs-mirror-2:19.3.0-4854.g8f219c47.el9.x86_64 13/165 2024-09-10T23:53:54.556 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : cephfs-shell-2:19.3.0-4854.g8f219c47.el9.x86_64 14/165 2024-09-10T23:53:54.557 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : libcephfs-devel-2:19.3.0-4854.g8f219c47.el9.x86_ 15/165 2024-09-10T23:53:54.557 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : libcephfs2-2:19.3.0-4854.g8f219c47.el9.x86_64 16/165 2024-09-10T23:53:54.557 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : libcephsqlite-2:19.3.0-4854.g8f219c47.el9.x86_64 17/165 2024-09-10T23:53:54.557 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : librados-devel-2:19.3.0-4854.g8f219c47.el9.x86_6 18/165 2024-09-10T23:53:54.557 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : libradosstriper1-2:19.3.0-4854.g8f219c47.el9.x86 19/165 2024-09-10T23:53:54.557 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : librgw2-2:19.3.0-4854.g8f219c47.el9.x86_64 20/165 2024-09-10T23:53:54.557 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-ceph-argparse-2:19.3.0-4854.g8f219c47.el 21/165 2024-09-10T23:53:54.557 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-ceph-common-2:19.3.0-4854.g8f219c47.el9. 22/165 2024-09-10T23:53:54.557 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-cephfs-2:19.3.0-4854.g8f219c47.el9.x86_6 23/165 2024-09-10T23:53:54.557 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-rados-2:19.3.0-4854.g8f219c47.el9.x86_64 24/165 2024-09-10T23:53:54.557 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-rbd-2:19.3.0-4854.g8f219c47.el9.x86_64 25/165 2024-09-10T23:53:54.557 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-rgw-2:19.3.0-4854.g8f219c47.el9.x86_64 26/165 2024-09-10T23:53:54.557 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : rbd-fuse-2:19.3.0-4854.g8f219c47.el9.x86_64 27/165 2024-09-10T23:53:54.558 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : rbd-mirror-2:19.3.0-4854.g8f219c47.el9.x86_64 28/165 2024-09-10T23:53:54.558 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : rbd-nbd-2:19.3.0-4854.g8f219c47.el9.x86_64 29/165 2024-09-10T23:53:54.569 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : ceph-grafana-dashboards-2:19.3.0-4854.g8f219c47. 30/165 2024-09-10T23:53:54.569 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : ceph-mgr-cephadm-2:19.3.0-4854.g8f219c47.el9.noa 31/165 2024-09-10T23:53:54.570 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : ceph-mgr-dashboard-2:19.3.0-4854.g8f219c47.el9.n 32/165 2024-09-10T23:53:54.570 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.3.0-4854.g8f2 33/165 2024-09-10T23:53:54.570 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : ceph-mgr-modules-core-2:19.3.0-4854.g8f219c47.el 34/165 2024-09-10T23:53:54.570 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : ceph-mgr-rook-2:19.3.0-4854.g8f219c47.el9.noarch 35/165 2024-09-10T23:53:54.570 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : ceph-prometheus-alerts-2:19.3.0-4854.g8f219c47.e 36/165 2024-09-10T23:53:54.570 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : ceph-volume-2:19.3.0-4854.g8f219c47.el9.noarch 37/165 2024-09-10T23:53:54.570 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : cephadm-2:19.3.0-4854.g8f219c47.el9.noarch 38/165 2024-09-10T23:53:54.570 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : cephfs-top-2:19.3.0-4854.g8f219c47.el9.noarch 39/165 2024-09-10T23:53:54.570 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 40/165 2024-09-10T23:53:54.570 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 41/165 2024-09-10T23:53:54.570 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 42/165 2024-09-10T23:53:54.571 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/165 2024-09-10T23:53:54.571 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/165 2024-09-10T23:53:54.571 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/165 2024-09-10T23:53:54.571 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/165 2024-09-10T23:53:54.571 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/165 2024-09-10T23:53:54.571 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 48/165 2024-09-10T23:53:54.571 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/165 2024-09-10T23:53:54.571 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/165 2024-09-10T23:53:54.571 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/165 2024-09-10T23:53:54.571 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/165 2024-09-10T23:53:54.571 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/165 2024-09-10T23:53:54.571 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/165 2024-09-10T23:53:54.571 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/165 2024-09-10T23:53:54.572 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/165 2024-09-10T23:53:54.572 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/165 2024-09-10T23:53:54.572 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : libnbd-1.20.2-2.el9.x86_64 58/165 2024-09-10T23:53:54.572 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 59/165 2024-09-10T23:53:54.572 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 60/165 2024-09-10T23:53:54.572 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 61/165 2024-09-10T23:53:54.572 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : libtool-2.4.6-46.el9.x86_64 62/165 2024-09-10T23:53:54.572 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : libuuid-devel-2.37.4-20.el9.x86_64 63/165 2024-09-10T23:53:54.572 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 64/165 2024-09-10T23:53:54.572 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 65/165 2024-09-10T23:53:54.572 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : lua-5.4.4-4.el9.x86_64 66/165 2024-09-10T23:53:54.572 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 67/165 2024-09-10T23:53:54.573 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 68/165 2024-09-10T23:53:54.573 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : openssl-devel-1:3.2.2-6.el9.x86_64 69/165 2024-09-10T23:53:54.573 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : postgresql-13.16-1.el9.x86_64 70/165 2024-09-10T23:53:54.573 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : postgresql-contrib-13.16-1.el9.x86_64 71/165 2024-09-10T23:53:54.573 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : postgresql-private-libs-13.16-1.el9.x86_64 72/165 2024-09-10T23:53:54.573 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : postgresql-server-13.16-1.el9.x86_64 73/165 2024-09-10T23:53:54.573 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 74/165 2024-09-10T23:53:54.573 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 75/165 2024-09-10T23:53:54.573 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 76/165 2024-09-10T23:53:54.573 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 77/165 2024-09-10T23:53:54.573 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 78/165 2024-09-10T23:53:54.573 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 79/165 2024-09-10T23:53:54.574 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 80/165 2024-09-10T23:53:54.574 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 81/165 2024-09-10T23:53:54.574 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 82/165 2024-09-10T23:53:54.574 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 83/165 2024-09-10T23:53:54.574 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 84/165 2024-09-10T23:53:54.574 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 85/165 2024-09-10T23:53:54.574 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 86/165 2024-09-10T23:53:54.574 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 87/165 2024-09-10T23:53:54.574 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 88/165 2024-09-10T23:53:54.574 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 89/165 2024-09-10T23:53:54.574 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 90/165 2024-09-10T23:53:54.574 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 91/165 2024-09-10T23:53:54.574 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 92/165 2024-09-10T23:53:54.574 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 93/165 2024-09-10T23:53:54.575 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 94/165 2024-09-10T23:53:54.575 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 95/165 2024-09-10T23:53:54.575 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 96/165 2024-09-10T23:53:54.575 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 97/165 2024-09-10T23:53:54.575 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 98/165 2024-09-10T23:53:54.575 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 99/165 2024-09-10T23:53:54.575 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 100/165 2024-09-10T23:53:54.575 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 101/165 2024-09-10T23:53:54.575 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : xfsprogs-devel-6.4.0-4.el9.x86_64 102/165 2024-09-10T23:53:54.575 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 103/165 2024-09-10T23:53:54.575 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 104/165 2024-09-10T23:53:54.575 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 105/165 2024-09-10T23:53:54.575 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 106/165 2024-09-10T23:53:54.575 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 107/165 2024-09-10T23:53:54.576 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 108/165 2024-09-10T23:53:54.576 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 109/165 2024-09-10T23:53:54.576 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 110/165 2024-09-10T23:53:54.576 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 111/165 2024-09-10T23:53:54.576 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 112/165 2024-09-10T23:53:54.576 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 113/165 2024-09-10T23:53:54.576 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 114/165 2024-09-10T23:53:54.576 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 115/165 2024-09-10T23:53:54.576 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 116/165 2024-09-10T23:53:54.576 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 117/165 2024-09-10T23:53:54.576 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 118/165 2024-09-10T23:53:54.576 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 119/165 2024-09-10T23:53:54.576 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 120/165 2024-09-10T23:53:54.576 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 121/165 2024-09-10T23:53:54.576 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 122/165 2024-09-10T23:53:54.577 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 123/165 2024-09-10T23:53:54.577 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 124/165 2024-09-10T23:53:54.577 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 125/165 2024-09-10T23:53:54.577 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 126/165 2024-09-10T23:53:54.577 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 127/165 2024-09-10T23:53:54.577 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 128/165 2024-09-10T23:53:54.577 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 129/165 2024-09-10T23:53:54.577 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 130/165 2024-09-10T23:53:54.577 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 131/165 2024-09-10T23:53:54.577 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 132/165 2024-09-10T23:53:54.577 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 133/165 2024-09-10T23:53:54.577 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 134/165 2024-09-10T23:53:54.577 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 135/165 2024-09-10T23:53:54.577 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 136/165 2024-09-10T23:53:54.578 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 137/165 2024-09-10T23:53:54.578 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 138/165 2024-09-10T23:53:54.578 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 139/165 2024-09-10T23:53:54.578 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 140/165 2024-09-10T23:53:54.578 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 141/165 2024-09-10T23:53:54.578 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 142/165 2024-09-10T23:53:54.578 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 143/165 2024-09-10T23:53:54.578 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 144/165 2024-09-10T23:53:54.578 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 145/165 2024-09-10T23:53:54.578 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 146/165 2024-09-10T23:53:54.578 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 147/165 2024-09-10T23:53:54.578 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : re2-1:20211101-20.el9.x86_64 148/165 2024-09-10T23:53:54.578 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 149/165 2024-09-10T23:53:54.578 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : librados2-2:19.3.0-4854.g8f219c47.el9.x86_64 150/165 2024-09-10T23:53:54.578 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 151/165 2024-09-10T23:53:54.579 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : librbd1-2:19.3.0-4854.g8f219c47.el9.x86_64 152/165 2024-09-10T23:53:54.579 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 153/165 2024-09-10T23:53:54.579 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : NetworkManager-1:1.49.90-1.el9.x86_64 154/165 2024-09-10T23:53:54.579 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : NetworkManager-1:1.48.8-1.el9.x86_64 155/165 2024-09-10T23:53:54.579 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : NetworkManager-libnm-1:1.49.90-1.el9.x86_64 156/165 2024-09-10T23:53:54.579 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 157/165 2024-09-10T23:53:54.579 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : NetworkManager-team-1:1.49.90-1.el9.x86_64 158/165 2024-09-10T23:53:54.579 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : NetworkManager-team-1:1.48.8-1.el9.x86_64 159/165 2024-09-10T23:53:54.579 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : NetworkManager-tui-1:1.49.90-1.el9.x86_64 160/165 2024-09-10T23:53:54.579 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : NetworkManager-tui-1:1.48.8-1.el9.x86_64 161/165 2024-09-10T23:53:54.579 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : openssl-1:3.2.2-6.el9.x86_64 162/165 2024-09-10T23:53:54.579 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : openssl-1:3.2.2-4.el9.x86_64 163/165 2024-09-10T23:53:54.579 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : openssl-libs-1:3.2.2-6.el9.x86_64 164/165 2024-09-10T23:53:55.749 INFO:teuthology.orchestra.run.smithi012.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.3.0-4854.g8f219c47.el9.noa 129/165 2024-09-10T23:53:55.816 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : ceph-mgr-dashboard-2:19.3.0-4854.g8f219c47.el9.n 130/165 2024-09-10T23:53:56.980 INFO:teuthology.orchestra.run.smithi012.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.3.0-4854.g8f219c47.el9.n 130/165 2024-09-10T23:53:57.015 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : ceph-mgr-diskprediction-local-2:19.3.0-4854.g8f2 131/165 2024-09-10T23:53:57.177 INFO:teuthology.orchestra.run.smithi012.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.3.0-4854.g8f2 131/165 2024-09-10T23:53:57.193 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : openssl-libs-1:3.2.2-4.el9.x86_64 165/165 2024-09-10T23:53:57.193 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-09-10T23:53:57.193 INFO:teuthology.orchestra.run.smithi178.stdout:Upgraded: 2024-09-10T23:53:57.194 INFO:teuthology.orchestra.run.smithi178.stdout: NetworkManager-1:1.49.90-1.el9.x86_64 2024-09-10T23:53:57.194 INFO:teuthology.orchestra.run.smithi178.stdout: NetworkManager-libnm-1:1.49.90-1.el9.x86_64 2024-09-10T23:53:57.194 INFO:teuthology.orchestra.run.smithi178.stdout: NetworkManager-team-1:1.49.90-1.el9.x86_64 2024-09-10T23:53:57.194 INFO:teuthology.orchestra.run.smithi178.stdout: NetworkManager-tui-1:1.49.90-1.el9.x86_64 2024-09-10T23:53:57.194 INFO:teuthology.orchestra.run.smithi178.stdout: librados2-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T23:53:57.194 INFO:teuthology.orchestra.run.smithi178.stdout: librbd1-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T23:53:57.194 INFO:teuthology.orchestra.run.smithi178.stdout: openssl-1:3.2.2-6.el9.x86_64 2024-09-10T23:53:57.194 INFO:teuthology.orchestra.run.smithi178.stdout: openssl-libs-1:3.2.2-6.el9.x86_64 2024-09-10T23:53:57.194 INFO:teuthology.orchestra.run.smithi178.stdout:Installed: 2024-09-10T23:53:57.194 INFO:teuthology.orchestra.run.smithi178.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-10T23:53:57.194 INFO:teuthology.orchestra.run.smithi178.stdout: automake-1.16.2-8.el9.noarch 2024-09-10T23:53:57.194 INFO:teuthology.orchestra.run.smithi178.stdout: bison-3.7.4-5.el9.x86_64 2024-09-10T23:53:57.194 INFO:teuthology.orchestra.run.smithi178.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-10T23:53:57.195 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T23:53:57.195 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-base-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T23:53:57.195 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-common-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T23:53:57.195 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-fuse-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T23:53:57.195 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-grafana-dashboards-2:19.3.0-4854.g8f219c47.el9.noarch 2024-09-10T23:53:57.195 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-immutable-object-cache-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T23:53:57.195 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-mds-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T23:53:57.195 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-mgr-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T23:53:57.195 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-mgr-cephadm-2:19.3.0-4854.g8f219c47.el9.noarch 2024-09-10T23:53:57.195 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-mgr-dashboard-2:19.3.0-4854.g8f219c47.el9.noarch 2024-09-10T23:53:57.195 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-mgr-diskprediction-local-2:19.3.0-4854.g8f219c47.el9.noarch 2024-09-10T23:53:57.195 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-mgr-modules-core-2:19.3.0-4854.g8f219c47.el9.noarch 2024-09-10T23:53:57.195 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-mgr-rook-2:19.3.0-4854.g8f219c47.el9.noarch 2024-09-10T23:53:57.196 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-mon-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T23:53:57.196 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-osd-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T23:53:57.196 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-prometheus-alerts-2:19.3.0-4854.g8f219c47.el9.noarch 2024-09-10T23:53:57.196 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-radosgw-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T23:53:57.196 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-selinux-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T23:53:57.196 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-test-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T23:53:57.196 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-volume-2:19.3.0-4854.g8f219c47.el9.noarch 2024-09-10T23:53:57.196 INFO:teuthology.orchestra.run.smithi178.stdout: cephadm-2:19.3.0-4854.g8f219c47.el9.noarch 2024-09-10T23:53:57.196 INFO:teuthology.orchestra.run.smithi178.stdout: cephfs-mirror-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T23:53:57.196 INFO:teuthology.orchestra.run.smithi178.stdout: cephfs-shell-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T23:53:57.196 INFO:teuthology.orchestra.run.smithi178.stdout: cephfs-top-2:19.3.0-4854.g8f219c47.el9.noarch 2024-09-10T23:53:57.196 INFO:teuthology.orchestra.run.smithi178.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-09-10T23:53:57.197 INFO:teuthology.orchestra.run.smithi178.stdout: flex-2.6.4-9.el9.x86_64 2024-09-10T23:53:57.197 INFO:teuthology.orchestra.run.smithi178.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-10T23:53:57.197 INFO:teuthology.orchestra.run.smithi178.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-10T23:53:57.197 INFO:teuthology.orchestra.run.smithi178.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-10T23:53:57.197 INFO:teuthology.orchestra.run.smithi178.stdout: fuse-2.9.9-16.el9.x86_64 2024-09-10T23:53:57.197 INFO:teuthology.orchestra.run.smithi178.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-09-10T23:53:57.197 INFO:teuthology.orchestra.run.smithi178.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-10T23:53:57.197 INFO:teuthology.orchestra.run.smithi178.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-09-10T23:53:57.197 INFO:teuthology.orchestra.run.smithi178.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-10T23:53:57.197 INFO:teuthology.orchestra.run.smithi178.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-10T23:53:57.197 INFO:teuthology.orchestra.run.smithi178.stdout: libcephfs-devel-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T23:53:57.198 INFO:teuthology.orchestra.run.smithi178.stdout: libcephfs2-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T23:53:57.198 INFO:teuthology.orchestra.run.smithi178.stdout: libcephsqlite-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T23:53:57.198 INFO:teuthology.orchestra.run.smithi178.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-10T23:53:57.198 INFO:teuthology.orchestra.run.smithi178.stdout: libnbd-1.20.2-2.el9.x86_64 2024-09-10T23:53:57.198 INFO:teuthology.orchestra.run.smithi178.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-10T23:53:57.198 INFO:teuthology.orchestra.run.smithi178.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-10T23:53:57.198 INFO:teuthology.orchestra.run.smithi178.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-10T23:53:57.198 INFO:teuthology.orchestra.run.smithi178.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-10T23:53:57.198 INFO:teuthology.orchestra.run.smithi178.stdout: librados-devel-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T23:53:57.198 INFO:teuthology.orchestra.run.smithi178.stdout: libradosstriper1-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T23:53:57.198 INFO:teuthology.orchestra.run.smithi178.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-10T23:53:57.198 INFO:teuthology.orchestra.run.smithi178.stdout: librgw2-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T23:53:57.198 INFO:teuthology.orchestra.run.smithi178.stdout: libtool-2.4.6-46.el9.x86_64 2024-09-10T23:53:57.199 INFO:teuthology.orchestra.run.smithi178.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-10T23:53:57.199 INFO:teuthology.orchestra.run.smithi178.stdout: libuuid-devel-2.37.4-20.el9.x86_64 2024-09-10T23:53:57.199 INFO:teuthology.orchestra.run.smithi178.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-09-10T23:53:57.199 INFO:teuthology.orchestra.run.smithi178.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-10T23:53:57.199 INFO:teuthology.orchestra.run.smithi178.stdout: lua-5.4.4-4.el9.x86_64 2024-09-10T23:53:57.199 INFO:teuthology.orchestra.run.smithi178.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-10T23:53:57.199 INFO:teuthology.orchestra.run.smithi178.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-10T23:53:57.199 INFO:teuthology.orchestra.run.smithi178.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-10T23:53:57.199 INFO:teuthology.orchestra.run.smithi178.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-10T23:53:57.199 INFO:teuthology.orchestra.run.smithi178.stdout: openssl-devel-1:3.2.2-6.el9.x86_64 2024-09-10T23:53:57.199 INFO:teuthology.orchestra.run.smithi178.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-10T23:53:57.199 INFO:teuthology.orchestra.run.smithi178.stdout: postgresql-13.16-1.el9.x86_64 2024-09-10T23:53:57.200 INFO:teuthology.orchestra.run.smithi178.stdout: postgresql-contrib-13.16-1.el9.x86_64 2024-09-10T23:53:57.200 INFO:teuthology.orchestra.run.smithi178.stdout: postgresql-private-libs-13.16-1.el9.x86_64 2024-09-10T23:53:57.200 INFO:teuthology.orchestra.run.smithi178.stdout: postgresql-server-13.16-1.el9.x86_64 2024-09-10T23:53:57.200 INFO:teuthology.orchestra.run.smithi178.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-10T23:53:57.200 INFO:teuthology.orchestra.run.smithi178.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-10T23:53:57.200 INFO:teuthology.orchestra.run.smithi178.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-10T23:53:57.200 INFO:teuthology.orchestra.run.smithi178.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-09-10T23:53:57.200 INFO:teuthology.orchestra.run.smithi178.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-10T23:53:57.200 INFO:teuthology.orchestra.run.smithi178.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-10T23:53:57.200 INFO:teuthology.orchestra.run.smithi178.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-10T23:53:57.200 INFO:teuthology.orchestra.run.smithi178.stdout: python3-ceph-argparse-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T23:53:57.200 INFO:teuthology.orchestra.run.smithi178.stdout: python3-ceph-common-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T23:53:57.201 INFO:teuthology.orchestra.run.smithi178.stdout: python3-cephfs-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T23:53:57.201 INFO:teuthology.orchestra.run.smithi178.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-10T23:53:57.201 INFO:teuthology.orchestra.run.smithi178.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-10T23:53:57.201 INFO:teuthology.orchestra.run.smithi178.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-10T23:53:57.201 INFO:teuthology.orchestra.run.smithi178.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-10T23:53:57.201 INFO:teuthology.orchestra.run.smithi178.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-09-10T23:53:57.201 INFO:teuthology.orchestra.run.smithi178.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-09-10T23:53:57.201 INFO:teuthology.orchestra.run.smithi178.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-10T23:53:57.201 INFO:teuthology.orchestra.run.smithi178.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-10T23:53:57.201 INFO:teuthology.orchestra.run.smithi178.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-10T23:53:57.201 INFO:teuthology.orchestra.run.smithi178.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-10T23:53:57.201 INFO:teuthology.orchestra.run.smithi178.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-10T23:53:57.202 INFO:teuthology.orchestra.run.smithi178.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-10T23:53:57.202 INFO:teuthology.orchestra.run.smithi178.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-10T23:53:57.202 INFO:teuthology.orchestra.run.smithi178.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-10T23:53:57.202 INFO:teuthology.orchestra.run.smithi178.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-10T23:53:57.202 INFO:teuthology.orchestra.run.smithi178.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-10T23:53:57.202 INFO:teuthology.orchestra.run.smithi178.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-10T23:53:57.202 INFO:teuthology.orchestra.run.smithi178.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-10T23:53:57.202 INFO:teuthology.orchestra.run.smithi178.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-10T23:53:57.202 INFO:teuthology.orchestra.run.smithi178.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-10T23:53:57.202 INFO:teuthology.orchestra.run.smithi178.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-10T23:53:57.202 INFO:teuthology.orchestra.run.smithi178.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-10T23:53:57.203 INFO:teuthology.orchestra.run.smithi178.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-10T23:53:57.203 INFO:teuthology.orchestra.run.smithi178.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-10T23:53:57.203 INFO:teuthology.orchestra.run.smithi178.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-10T23:53:57.203 INFO:teuthology.orchestra.run.smithi178.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-10T23:53:57.203 INFO:teuthology.orchestra.run.smithi178.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-10T23:53:57.203 INFO:teuthology.orchestra.run.smithi178.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-10T23:53:57.203 INFO:teuthology.orchestra.run.smithi178.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-10T23:53:57.203 INFO:teuthology.orchestra.run.smithi178.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-10T23:53:57.203 INFO:teuthology.orchestra.run.smithi178.stdout: python3-ply-3.11-14.el9.noarch 2024-09-10T23:53:57.203 INFO:teuthology.orchestra.run.smithi178.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-10T23:53:57.203 INFO:teuthology.orchestra.run.smithi178.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-10T23:53:57.203 INFO:teuthology.orchestra.run.smithi178.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-10T23:53:57.203 INFO:teuthology.orchestra.run.smithi178.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-10T23:53:57.204 INFO:teuthology.orchestra.run.smithi178.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-10T23:53:57.204 INFO:teuthology.orchestra.run.smithi178.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-10T23:53:57.204 INFO:teuthology.orchestra.run.smithi178.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-10T23:53:57.204 INFO:teuthology.orchestra.run.smithi178.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-09-10T23:53:57.204 INFO:teuthology.orchestra.run.smithi178.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-10T23:53:57.204 INFO:teuthology.orchestra.run.smithi178.stdout: python3-rados-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T23:53:57.204 INFO:teuthology.orchestra.run.smithi178.stdout: python3-rbd-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T23:53:57.204 INFO:teuthology.orchestra.run.smithi178.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-10T23:53:57.204 INFO:teuthology.orchestra.run.smithi178.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-10T23:53:57.204 INFO:teuthology.orchestra.run.smithi178.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-10T23:53:57.204 INFO:teuthology.orchestra.run.smithi178.stdout: python3-rgw-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T23:53:57.205 INFO:teuthology.orchestra.run.smithi178.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-10T23:53:57.205 INFO:teuthology.orchestra.run.smithi178.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-10T23:53:57.205 INFO:teuthology.orchestra.run.smithi178.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-10T23:53:57.205 INFO:teuthology.orchestra.run.smithi178.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-10T23:53:57.205 INFO:teuthology.orchestra.run.smithi178.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-10T23:53:57.205 INFO:teuthology.orchestra.run.smithi178.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-10T23:53:57.205 INFO:teuthology.orchestra.run.smithi178.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-10T23:53:57.205 INFO:teuthology.orchestra.run.smithi178.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-09-10T23:53:57.205 INFO:teuthology.orchestra.run.smithi178.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-10T23:53:57.205 INFO:teuthology.orchestra.run.smithi178.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-10T23:53:57.205 INFO:teuthology.orchestra.run.smithi178.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-10T23:53:57.205 INFO:teuthology.orchestra.run.smithi178.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-10T23:53:57.205 INFO:teuthology.orchestra.run.smithi178.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-10T23:53:57.206 INFO:teuthology.orchestra.run.smithi178.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-10T23:53:57.206 INFO:teuthology.orchestra.run.smithi178.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-10T23:53:57.206 INFO:teuthology.orchestra.run.smithi178.stdout: rbd-fuse-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T23:53:57.206 INFO:teuthology.orchestra.run.smithi178.stdout: rbd-mirror-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T23:53:57.206 INFO:teuthology.orchestra.run.smithi178.stdout: rbd-nbd-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T23:53:57.206 INFO:teuthology.orchestra.run.smithi178.stdout: re2-1:20211101-20.el9.x86_64 2024-09-10T23:53:57.206 INFO:teuthology.orchestra.run.smithi178.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-10T23:53:57.206 INFO:teuthology.orchestra.run.smithi178.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-10T23:53:57.206 INFO:teuthology.orchestra.run.smithi178.stdout: uuid-1.6.2-55.el9.x86_64 2024-09-10T23:53:57.206 INFO:teuthology.orchestra.run.smithi178.stdout: xfsprogs-devel-6.4.0-4.el9.x86_64 2024-09-10T23:53:57.206 INFO:teuthology.orchestra.run.smithi178.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-10T23:53:57.207 INFO:teuthology.orchestra.run.smithi178.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-09-10T23:53:57.207 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-09-10T23:53:57.207 INFO:teuthology.orchestra.run.smithi178.stdout:Complete! 2024-09-10T23:53:57.365 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : ceph-mgr-modules-core-2:19.3.0-4854.g8f219c47.el 132/165 2024-09-10T23:53:57.392 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : ceph-mgr-2:19.3.0-4854.g8f219c47.el9.x86_64 133/165 2024-09-10T23:53:57.445 INFO:teuthology.orchestra.run.smithi012.stdout: Running scriptlet: ceph-mgr-2:19.3.0-4854.g8f219c47.el9.x86_64 133/165 2024-09-10T23:53:57.445 INFO:teuthology.orchestra.run.smithi012.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T23:53:57.445 INFO:teuthology.orchestra.run.smithi012.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-10T23:53:57.445 INFO:teuthology.orchestra.run.smithi012.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-10T23:53:57.445 INFO:teuthology.orchestra.run.smithi012.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-10T23:53:57.445 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-10T23:53:57.501 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : ceph-mgr-rook-2:19.3.0-4854.g8f219c47.el9.noarch 134/165 2024-09-10T23:53:57.528 INFO:teuthology.orchestra.run.smithi012.stdout: Running scriptlet: ceph-mgr-rook-2:19.3.0-4854.g8f219c47.el9.noarch 134/165 2024-09-10T23:53:57.626 DEBUG:teuthology.parallel:result is None 2024-09-10T23:53:57.640 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : ceph-2:19.3.0-4854.g8f219c47.el9.x86_64 135/165 2024-09-10T23:53:57.705 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : ceph-fuse-2:19.3.0-4854.g8f219c47.el9.x86_64 136/165 2024-09-10T23:53:58.804 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : libtool-2.4.6-46.el9.x86_64 137/165 2024-09-10T23:53:58.837 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : ceph-radosgw-2:19.3.0-4854.g8f219c47.el9.x86_64 138/165 2024-09-10T23:53:58.889 INFO:teuthology.orchestra.run.smithi012.stdout: Running scriptlet: ceph-radosgw-2:19.3.0-4854.g8f219c47.el9.x86_64 138/165 2024-09-10T23:53:58.889 INFO:teuthology.orchestra.run.smithi012.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T23:53:58.889 INFO:teuthology.orchestra.run.smithi012.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-10T23:53:58.890 INFO:teuthology.orchestra.run.smithi012.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-10T23:53:58.890 INFO:teuthology.orchestra.run.smithi012.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-10T23:53:58.890 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-10T23:53:58.973 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : ceph-immutable-object-cache-2:19.3.0-4854.g8f219 139/165 2024-09-10T23:53:59.013 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : ceph-test-2:19.3.0-4854.g8f219c47.el9.x86_64 142/165 2024-09-10T23:53:59.022 INFO:teuthology.orchestra.run.smithi012.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.3.0-4854.g8f219 139/165 2024-09-10T23:53:59.022 INFO:teuthology.orchestra.run.smithi012.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T23:53:59.022 INFO:teuthology.orchestra.run.smithi012.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-10T23:53:59.022 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-10T23:53:59.053 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : rbd-fuse-2:19.3.0-4854.g8f219c47.el9.x86_64 143/165 2024-09-10T23:53:59.071 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : cephfs-mirror-2:19.3.0-4854.g8f219c47.el9.x86_64 140/165 2024-09-10T23:53:59.094 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : rbd-nbd-2:19.3.0-4854.g8f219c47.el9.x86_64 144/165 2024-09-10T23:53:59.123 INFO:teuthology.orchestra.run.smithi012.stdout: Running scriptlet: cephfs-mirror-2:19.3.0-4854.g8f219c47.el9.x86_64 140/165 2024-09-10T23:53:59.123 INFO:teuthology.orchestra.run.smithi012.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T23:53:59.124 INFO:teuthology.orchestra.run.smithi012.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-09-10T23:53:59.124 INFO:teuthology.orchestra.run.smithi012.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-10T23:53:59.124 INFO:teuthology.orchestra.run.smithi012.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-10T23:53:59.124 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-10T23:53:59.127 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : xfsprogs-devel-6.4.0-4.el9.x86_64 145/165 2024-09-10T23:53:59.154 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 146/165 2024-09-10T23:53:59.338 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : cephfs-shell-2:19.3.0-4854.g8f219c47.el9.x86_64 147/165 2024-09-10T23:53:59.377 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : postgresql-contrib-13.16-1.el9.x86_64 148/165 2024-09-10T23:53:59.407 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : libcephfs-devel-2:19.3.0-4854.g8f219c47.el9.x86_ 149/165 2024-09-10T23:53:59.445 INFO:teuthology.orchestra.run.smithi165.stdout: Upgrading : NetworkManager-team-1:1.49.90-1.el9.x86_64 150/165 2024-09-10T23:53:59.489 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : rbd-mirror-2:19.3.0-4854.g8f219c47.el9.x86_64 141/165 2024-09-10T23:53:59.491 INFO:teuthology.orchestra.run.smithi165.stdout: Upgrading : NetworkManager-tui-1:1.49.90-1.el9.x86_64 151/165 2024-09-10T23:53:59.540 INFO:teuthology.orchestra.run.smithi012.stdout: Running scriptlet: rbd-mirror-2:19.3.0-4854.g8f219c47.el9.x86_64 141/165 2024-09-10T23:53:59.541 INFO:teuthology.orchestra.run.smithi012.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T23:53:59.541 INFO:teuthology.orchestra.run.smithi012.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-10T23:53:59.541 INFO:teuthology.orchestra.run.smithi012.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-10T23:53:59.541 INFO:teuthology.orchestra.run.smithi012.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-10T23:53:59.541 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-10T23:54:00.090 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 152/165 2024-09-10T23:54:00.131 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : postgresql-server-13.16-1.el9.x86_64 152/165 2024-09-10T23:54:00.152 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 152/165 2024-09-10T23:54:00.990 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : cephfs-top-2:19.3.0-4854.g8f219c47.el9.noarch 153/165 2024-09-10T23:54:01.079 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : openssl-devel-1:3.2.2-6.el9.x86_64 154/165 2024-09-10T23:54:01.128 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 155/165 2024-09-10T23:54:01.216 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : flex-2.6.4-9.el9.x86_64 156/165 2024-09-10T23:54:01.274 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : bison-3.7.4-5.el9.x86_64 157/165 2024-09-10T23:54:01.299 INFO:teuthology.orchestra.run.smithi165.stdout: Cleanup : NetworkManager-tui-1:1.48.8-1.el9.x86_64 158/165 2024-09-10T23:54:01.325 INFO:teuthology.orchestra.run.smithi165.stdout: Cleanup : openssl-1:3.2.2-4.el9.x86_64 159/165 2024-09-10T23:54:01.325 INFO:teuthology.orchestra.run.smithi165.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 160/165 2024-09-10T23:54:01.392 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 160/165 2024-09-10T23:54:01.392 INFO:teuthology.orchestra.run.smithi165.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 161/165 2024-09-10T23:54:01.468 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 161/165 2024-09-10T23:54:01.493 INFO:teuthology.orchestra.run.smithi165.stdout: Cleanup : NetworkManager-team-1:1.48.8-1.el9.x86_64 162/165 2024-09-10T23:54:01.498 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 163/165 2024-09-10T23:54:01.499 INFO:teuthology.orchestra.run.smithi165.stdout: Cleanup : NetworkManager-1:1.48.8-1.el9.x86_64 163/165 2024-09-10T23:54:01.679 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 163/165 2024-09-10T23:54:01.679 INFO:teuthology.orchestra.run.smithi165.stdout: Cleanup : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 164/165 2024-09-10T23:54:01.771 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: NetworkManager-libnm-1:1.48.8-1.el9.x86_64 164/165 2024-09-10T23:54:01.835 INFO:teuthology.orchestra.run.smithi165.stdout: Cleanup : openssl-libs-1:3.2.2-4.el9.x86_64 165/165 2024-09-10T23:54:04.970 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : ceph-test-2:19.3.0-4854.g8f219c47.el9.x86_64 142/165 2024-09-10T23:54:05.028 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : rbd-fuse-2:19.3.0-4854.g8f219c47.el9.x86_64 143/165 2024-09-10T23:54:05.073 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : rbd-nbd-2:19.3.0-4854.g8f219c47.el9.x86_64 144/165 2024-09-10T23:54:05.109 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : xfsprogs-devel-6.4.0-4.el9.x86_64 145/165 2024-09-10T23:54:05.152 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 146/165 2024-09-10T23:54:05.417 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : cephfs-shell-2:19.3.0-4854.g8f219c47.el9.x86_64 147/165 2024-09-10T23:54:05.474 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : postgresql-contrib-13.16-1.el9.x86_64 148/165 2024-09-10T23:54:05.504 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : libcephfs-devel-2:19.3.0-4854.g8f219c47.el9.x86_ 149/165 2024-09-10T23:54:05.545 INFO:teuthology.orchestra.run.smithi012.stdout: Upgrading : NetworkManager-team-1:1.49.90-1.el9.x86_64 150/165 2024-09-10T23:54:05.612 INFO:teuthology.orchestra.run.smithi012.stdout: Upgrading : NetworkManager-tui-1:1.49.90-1.el9.x86_64 151/165 2024-09-10T23:54:06.316 INFO:teuthology.orchestra.run.smithi012.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 152/165 2024-09-10T23:54:06.368 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : postgresql-server-13.16-1.el9.x86_64 152/165 2024-09-10T23:54:06.410 INFO:teuthology.orchestra.run.smithi012.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 152/165 2024-09-10T23:54:07.800 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : cephfs-top-2:19.3.0-4854.g8f219c47.el9.noarch 153/165 2024-09-10T23:54:07.936 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : openssl-devel-1:3.2.2-6.el9.x86_64 154/165 2024-09-10T23:54:07.989 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: openssl-libs-1:3.2.2-4.el9.x86_64 165/165 2024-09-10T23:54:07.989 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : ceph-2:19.3.0-4854.g8f219c47.el9.x86_64 1/165 2024-09-10T23:54:07.989 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : ceph-base-2:19.3.0-4854.g8f219c47.el9.x86_64 2/165 2024-09-10T23:54:07.989 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : ceph-common-2:19.3.0-4854.g8f219c47.el9.x86_64 3/165 2024-09-10T23:54:07.989 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : ceph-fuse-2:19.3.0-4854.g8f219c47.el9.x86_64 4/165 2024-09-10T23:54:07.989 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : ceph-immutable-object-cache-2:19.3.0-4854.g8f219 5/165 2024-09-10T23:54:07.989 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : ceph-mds-2:19.3.0-4854.g8f219c47.el9.x86_64 6/165 2024-09-10T23:54:07.990 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : ceph-mgr-2:19.3.0-4854.g8f219c47.el9.x86_64 7/165 2024-09-10T23:54:07.990 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : ceph-mon-2:19.3.0-4854.g8f219c47.el9.x86_64 8/165 2024-09-10T23:54:07.990 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : ceph-osd-2:19.3.0-4854.g8f219c47.el9.x86_64 9/165 2024-09-10T23:54:07.990 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : ceph-radosgw-2:19.3.0-4854.g8f219c47.el9.x86_64 10/165 2024-09-10T23:54:07.990 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : ceph-selinux-2:19.3.0-4854.g8f219c47.el9.x86_64 11/165 2024-09-10T23:54:07.990 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : ceph-test-2:19.3.0-4854.g8f219c47.el9.x86_64 12/165 2024-09-10T23:54:07.990 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : cephfs-mirror-2:19.3.0-4854.g8f219c47.el9.x86_64 13/165 2024-09-10T23:54:07.990 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : cephfs-shell-2:19.3.0-4854.g8f219c47.el9.x86_64 14/165 2024-09-10T23:54:07.990 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : libcephfs-devel-2:19.3.0-4854.g8f219c47.el9.x86_ 15/165 2024-09-10T23:54:07.990 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : libcephfs2-2:19.3.0-4854.g8f219c47.el9.x86_64 16/165 2024-09-10T23:54:07.990 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : libcephsqlite-2:19.3.0-4854.g8f219c47.el9.x86_64 17/165 2024-09-10T23:54:07.990 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : librados-devel-2:19.3.0-4854.g8f219c47.el9.x86_6 18/165 2024-09-10T23:54:07.990 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : libradosstriper1-2:19.3.0-4854.g8f219c47.el9.x86 19/165 2024-09-10T23:54:07.991 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : librgw2-2:19.3.0-4854.g8f219c47.el9.x86_64 20/165 2024-09-10T23:54:07.991 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-ceph-argparse-2:19.3.0-4854.g8f219c47.el 21/165 2024-09-10T23:54:07.991 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-ceph-common-2:19.3.0-4854.g8f219c47.el9. 22/165 2024-09-10T23:54:07.991 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-cephfs-2:19.3.0-4854.g8f219c47.el9.x86_6 23/165 2024-09-10T23:54:07.991 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-rados-2:19.3.0-4854.g8f219c47.el9.x86_64 24/165 2024-09-10T23:54:07.991 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-rbd-2:19.3.0-4854.g8f219c47.el9.x86_64 25/165 2024-09-10T23:54:07.991 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-rgw-2:19.3.0-4854.g8f219c47.el9.x86_64 26/165 2024-09-10T23:54:07.991 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : rbd-fuse-2:19.3.0-4854.g8f219c47.el9.x86_64 27/165 2024-09-10T23:54:07.991 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : rbd-mirror-2:19.3.0-4854.g8f219c47.el9.x86_64 28/165 2024-09-10T23:54:07.991 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : rbd-nbd-2:19.3.0-4854.g8f219c47.el9.x86_64 29/165 2024-09-10T23:54:07.991 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : ceph-grafana-dashboards-2:19.3.0-4854.g8f219c47. 30/165 2024-09-10T23:54:07.991 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : ceph-mgr-cephadm-2:19.3.0-4854.g8f219c47.el9.noa 31/165 2024-09-10T23:54:07.991 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : ceph-mgr-dashboard-2:19.3.0-4854.g8f219c47.el9.n 32/165 2024-09-10T23:54:07.992 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.3.0-4854.g8f2 33/165 2024-09-10T23:54:07.992 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : ceph-mgr-modules-core-2:19.3.0-4854.g8f219c47.el 34/165 2024-09-10T23:54:07.992 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : ceph-mgr-rook-2:19.3.0-4854.g8f219c47.el9.noarch 35/165 2024-09-10T23:54:07.992 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : ceph-prometheus-alerts-2:19.3.0-4854.g8f219c47.e 36/165 2024-09-10T23:54:07.992 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : ceph-volume-2:19.3.0-4854.g8f219c47.el9.noarch 37/165 2024-09-10T23:54:07.992 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : cephadm-2:19.3.0-4854.g8f219c47.el9.noarch 38/165 2024-09-10T23:54:07.992 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : cephfs-top-2:19.3.0-4854.g8f219c47.el9.noarch 39/165 2024-09-10T23:54:07.992 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 40/165 2024-09-10T23:54:07.992 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 41/165 2024-09-10T23:54:07.992 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 42/165 2024-09-10T23:54:07.992 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/165 2024-09-10T23:54:07.992 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/165 2024-09-10T23:54:07.992 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/165 2024-09-10T23:54:07.993 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/165 2024-09-10T23:54:07.993 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/165 2024-09-10T23:54:07.993 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 48/165 2024-09-10T23:54:07.993 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/165 2024-09-10T23:54:07.993 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/165 2024-09-10T23:54:07.993 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/165 2024-09-10T23:54:07.993 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/165 2024-09-10T23:54:07.993 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/165 2024-09-10T23:54:07.993 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/165 2024-09-10T23:54:07.993 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/165 2024-09-10T23:54:07.993 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/165 2024-09-10T23:54:07.993 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/165 2024-09-10T23:54:07.994 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : libnbd-1.20.2-2.el9.x86_64 58/165 2024-09-10T23:54:07.994 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 59/165 2024-09-10T23:54:07.994 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 60/165 2024-09-10T23:54:07.994 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 61/165 2024-09-10T23:54:07.994 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : libtool-2.4.6-46.el9.x86_64 62/165 2024-09-10T23:54:07.994 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : libuuid-devel-2.37.4-20.el9.x86_64 63/165 2024-09-10T23:54:07.994 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 64/165 2024-09-10T23:54:07.994 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 65/165 2024-09-10T23:54:07.994 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : lua-5.4.4-4.el9.x86_64 66/165 2024-09-10T23:54:07.994 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 67/165 2024-09-10T23:54:07.994 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 68/165 2024-09-10T23:54:07.994 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : openssl-devel-1:3.2.2-6.el9.x86_64 69/165 2024-09-10T23:54:07.994 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : postgresql-13.16-1.el9.x86_64 70/165 2024-09-10T23:54:07.995 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : postgresql-contrib-13.16-1.el9.x86_64 71/165 2024-09-10T23:54:07.995 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : postgresql-private-libs-13.16-1.el9.x86_64 72/165 2024-09-10T23:54:07.995 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : postgresql-server-13.16-1.el9.x86_64 73/165 2024-09-10T23:54:07.995 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 74/165 2024-09-10T23:54:07.995 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 75/165 2024-09-10T23:54:07.995 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 76/165 2024-09-10T23:54:07.995 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 77/165 2024-09-10T23:54:07.995 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 78/165 2024-09-10T23:54:07.995 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 79/165 2024-09-10T23:54:07.995 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 80/165 2024-09-10T23:54:07.995 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 81/165 2024-09-10T23:54:07.995 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 82/165 2024-09-10T23:54:07.995 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 83/165 2024-09-10T23:54:07.996 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 84/165 2024-09-10T23:54:07.996 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 85/165 2024-09-10T23:54:07.996 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 86/165 2024-09-10T23:54:07.996 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 87/165 2024-09-10T23:54:07.996 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 88/165 2024-09-10T23:54:07.996 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 89/165 2024-09-10T23:54:07.996 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 90/165 2024-09-10T23:54:07.996 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 91/165 2024-09-10T23:54:07.996 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 92/165 2024-09-10T23:54:07.996 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 93/165 2024-09-10T23:54:07.996 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 94/165 2024-09-10T23:54:07.996 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 95/165 2024-09-10T23:54:07.996 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 96/165 2024-09-10T23:54:07.997 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 97/165 2024-09-10T23:54:07.997 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 98/165 2024-09-10T23:54:07.997 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 99/165 2024-09-10T23:54:07.997 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 100/165 2024-09-10T23:54:07.997 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 101/165 2024-09-10T23:54:07.997 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : xfsprogs-devel-6.4.0-4.el9.x86_64 102/165 2024-09-10T23:54:07.997 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 103/165 2024-09-10T23:54:07.997 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 104/165 2024-09-10T23:54:07.997 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 105/165 2024-09-10T23:54:07.997 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 106/165 2024-09-10T23:54:07.997 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 107/165 2024-09-10T23:54:07.997 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 108/165 2024-09-10T23:54:07.998 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 109/165 2024-09-10T23:54:07.998 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 110/165 2024-09-10T23:54:07.998 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 111/165 2024-09-10T23:54:07.998 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 112/165 2024-09-10T23:54:07.998 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 113/165 2024-09-10T23:54:07.998 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 114/165 2024-09-10T23:54:07.998 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 115/165 2024-09-10T23:54:07.998 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 116/165 2024-09-10T23:54:07.998 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 117/165 2024-09-10T23:54:07.998 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 118/165 2024-09-10T23:54:07.998 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 119/165 2024-09-10T23:54:07.998 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 120/165 2024-09-10T23:54:07.998 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 121/165 2024-09-10T23:54:07.999 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 122/165 2024-09-10T23:54:07.999 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 123/165 2024-09-10T23:54:07.999 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 124/165 2024-09-10T23:54:07.999 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 125/165 2024-09-10T23:54:07.999 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 126/165 2024-09-10T23:54:07.999 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 127/165 2024-09-10T23:54:07.999 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 128/165 2024-09-10T23:54:07.999 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 129/165 2024-09-10T23:54:07.999 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 130/165 2024-09-10T23:54:07.999 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 131/165 2024-09-10T23:54:07.999 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 132/165 2024-09-10T23:54:07.999 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 133/165 2024-09-10T23:54:07.999 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 134/165 2024-09-10T23:54:08.000 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 135/165 2024-09-10T23:54:08.000 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 136/165 2024-09-10T23:54:08.000 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 137/165 2024-09-10T23:54:08.000 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 138/165 2024-09-10T23:54:08.000 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 139/165 2024-09-10T23:54:08.000 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 140/165 2024-09-10T23:54:08.000 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 141/165 2024-09-10T23:54:08.000 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 142/165 2024-09-10T23:54:08.000 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 143/165 2024-09-10T23:54:08.000 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 144/165 2024-09-10T23:54:08.000 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 145/165 2024-09-10T23:54:08.000 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 146/165 2024-09-10T23:54:08.000 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 147/165 2024-09-10T23:54:08.001 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : re2-1:20211101-20.el9.x86_64 148/165 2024-09-10T23:54:08.001 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 149/165 2024-09-10T23:54:08.001 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : librados2-2:19.3.0-4854.g8f219c47.el9.x86_64 150/165 2024-09-10T23:54:08.001 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 151/165 2024-09-10T23:54:08.001 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : librbd1-2:19.3.0-4854.g8f219c47.el9.x86_64 152/165 2024-09-10T23:54:08.001 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 153/165 2024-09-10T23:54:08.001 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : NetworkManager-1:1.49.90-1.el9.x86_64 154/165 2024-09-10T23:54:08.001 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : NetworkManager-1:1.48.8-1.el9.x86_64 155/165 2024-09-10T23:54:08.001 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : NetworkManager-libnm-1:1.49.90-1.el9.x86_64 156/165 2024-09-10T23:54:08.001 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 157/165 2024-09-10T23:54:08.001 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : NetworkManager-team-1:1.49.90-1.el9.x86_64 158/165 2024-09-10T23:54:08.001 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : NetworkManager-team-1:1.48.8-1.el9.x86_64 159/165 2024-09-10T23:54:08.001 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : NetworkManager-tui-1:1.49.90-1.el9.x86_64 160/165 2024-09-10T23:54:08.002 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : NetworkManager-tui-1:1.48.8-1.el9.x86_64 161/165 2024-09-10T23:54:08.002 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : openssl-1:3.2.2-6.el9.x86_64 162/165 2024-09-10T23:54:08.002 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : openssl-1:3.2.2-4.el9.x86_64 163/165 2024-09-10T23:54:08.002 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : openssl-libs-1:3.2.2-6.el9.x86_64 164/165 2024-09-10T23:54:08.002 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 155/165 2024-09-10T23:54:08.116 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : flex-2.6.4-9.el9.x86_64 156/165 2024-09-10T23:54:08.151 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : bison-3.7.4-5.el9.x86_64 157/165 2024-09-10T23:54:08.184 INFO:teuthology.orchestra.run.smithi012.stdout: Cleanup : NetworkManager-tui-1:1.48.8-1.el9.x86_64 158/165 2024-09-10T23:54:08.221 INFO:teuthology.orchestra.run.smithi012.stdout: Cleanup : openssl-1:3.2.2-4.el9.x86_64 159/165 2024-09-10T23:54:08.221 INFO:teuthology.orchestra.run.smithi012.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 160/165 2024-09-10T23:54:08.336 INFO:teuthology.orchestra.run.smithi012.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 160/165 2024-09-10T23:54:08.337 INFO:teuthology.orchestra.run.smithi012.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 161/165 2024-09-10T23:54:08.454 INFO:teuthology.orchestra.run.smithi012.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 161/165 2024-09-10T23:54:08.478 INFO:teuthology.orchestra.run.smithi012.stdout: Cleanup : NetworkManager-team-1:1.48.8-1.el9.x86_64 162/165 2024-09-10T23:54:08.494 INFO:teuthology.orchestra.run.smithi012.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 163/165 2024-09-10T23:54:08.494 INFO:teuthology.orchestra.run.smithi012.stdout: Cleanup : NetworkManager-1:1.48.8-1.el9.x86_64 163/165 2024-09-10T23:54:08.739 INFO:teuthology.orchestra.run.smithi012.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 163/165 2024-09-10T23:54:08.739 INFO:teuthology.orchestra.run.smithi012.stdout: Cleanup : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 164/165 2024-09-10T23:54:08.882 INFO:teuthology.orchestra.run.smithi012.stdout: Running scriptlet: NetworkManager-libnm-1:1.48.8-1.el9.x86_64 164/165 2024-09-10T23:54:08.959 INFO:teuthology.orchestra.run.smithi012.stdout: Cleanup : openssl-libs-1:3.2.2-4.el9.x86_64 165/165 2024-09-10T23:54:09.151 INFO:teuthology.orchestra.run.smithi012.stdout: Running scriptlet: openssl-libs-1:3.2.2-4.el9.x86_64 165/165 2024-09-10T23:54:09.151 INFO:teuthology.orchestra.run.smithi012.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2024-09-10T23:54:09.151 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-10T23:54:10.390 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : openssl-libs-1:3.2.2-4.el9.x86_64 165/165 2024-09-10T23:54:10.390 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-09-10T23:54:10.390 INFO:teuthology.orchestra.run.smithi165.stdout:Upgraded: 2024-09-10T23:54:10.390 INFO:teuthology.orchestra.run.smithi165.stdout: NetworkManager-1:1.49.90-1.el9.x86_64 2024-09-10T23:54:10.390 INFO:teuthology.orchestra.run.smithi165.stdout: NetworkManager-libnm-1:1.49.90-1.el9.x86_64 2024-09-10T23:54:10.390 INFO:teuthology.orchestra.run.smithi165.stdout: NetworkManager-team-1:1.49.90-1.el9.x86_64 2024-09-10T23:54:10.390 INFO:teuthology.orchestra.run.smithi165.stdout: NetworkManager-tui-1:1.49.90-1.el9.x86_64 2024-09-10T23:54:10.390 INFO:teuthology.orchestra.run.smithi165.stdout: librados2-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T23:54:10.390 INFO:teuthology.orchestra.run.smithi165.stdout: librbd1-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T23:54:10.391 INFO:teuthology.orchestra.run.smithi165.stdout: openssl-1:3.2.2-6.el9.x86_64 2024-09-10T23:54:10.391 INFO:teuthology.orchestra.run.smithi165.stdout: openssl-libs-1:3.2.2-6.el9.x86_64 2024-09-10T23:54:10.391 INFO:teuthology.orchestra.run.smithi165.stdout:Installed: 2024-09-10T23:54:10.391 INFO:teuthology.orchestra.run.smithi165.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-10T23:54:10.391 INFO:teuthology.orchestra.run.smithi165.stdout: automake-1.16.2-8.el9.noarch 2024-09-10T23:54:10.391 INFO:teuthology.orchestra.run.smithi165.stdout: bison-3.7.4-5.el9.x86_64 2024-09-10T23:54:10.391 INFO:teuthology.orchestra.run.smithi165.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-10T23:54:10.391 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T23:54:10.391 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-base-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T23:54:10.391 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-common-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T23:54:10.391 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-fuse-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T23:54:10.391 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-grafana-dashboards-2:19.3.0-4854.g8f219c47.el9.noarch 2024-09-10T23:54:10.391 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-immutable-object-cache-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T23:54:10.391 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-mds-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T23:54:10.392 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-mgr-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T23:54:10.392 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-mgr-cephadm-2:19.3.0-4854.g8f219c47.el9.noarch 2024-09-10T23:54:10.392 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-mgr-dashboard-2:19.3.0-4854.g8f219c47.el9.noarch 2024-09-10T23:54:10.392 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-mgr-diskprediction-local-2:19.3.0-4854.g8f219c47.el9.noarch 2024-09-10T23:54:10.392 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-mgr-modules-core-2:19.3.0-4854.g8f219c47.el9.noarch 2024-09-10T23:54:10.392 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-mgr-rook-2:19.3.0-4854.g8f219c47.el9.noarch 2024-09-10T23:54:10.392 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-mon-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T23:54:10.392 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-osd-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T23:54:10.392 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-prometheus-alerts-2:19.3.0-4854.g8f219c47.el9.noarch 2024-09-10T23:54:10.392 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-radosgw-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T23:54:10.393 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-selinux-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T23:54:10.393 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-test-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T23:54:10.393 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-volume-2:19.3.0-4854.g8f219c47.el9.noarch 2024-09-10T23:54:10.393 INFO:teuthology.orchestra.run.smithi165.stdout: cephadm-2:19.3.0-4854.g8f219c47.el9.noarch 2024-09-10T23:54:10.393 INFO:teuthology.orchestra.run.smithi165.stdout: cephfs-mirror-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T23:54:10.393 INFO:teuthology.orchestra.run.smithi165.stdout: cephfs-shell-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T23:54:10.393 INFO:teuthology.orchestra.run.smithi165.stdout: cephfs-top-2:19.3.0-4854.g8f219c47.el9.noarch 2024-09-10T23:54:10.394 INFO:teuthology.orchestra.run.smithi165.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-09-10T23:54:10.394 INFO:teuthology.orchestra.run.smithi165.stdout: flex-2.6.4-9.el9.x86_64 2024-09-10T23:54:10.394 INFO:teuthology.orchestra.run.smithi165.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-10T23:54:10.394 INFO:teuthology.orchestra.run.smithi165.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-10T23:54:10.394 INFO:teuthology.orchestra.run.smithi165.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-10T23:54:10.394 INFO:teuthology.orchestra.run.smithi165.stdout: fuse-2.9.9-16.el9.x86_64 2024-09-10T23:54:10.394 INFO:teuthology.orchestra.run.smithi165.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-09-10T23:54:10.394 INFO:teuthology.orchestra.run.smithi165.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-10T23:54:10.394 INFO:teuthology.orchestra.run.smithi165.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-09-10T23:54:10.394 INFO:teuthology.orchestra.run.smithi165.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-10T23:54:10.395 INFO:teuthology.orchestra.run.smithi165.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-10T23:54:10.395 INFO:teuthology.orchestra.run.smithi165.stdout: libcephfs-devel-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T23:54:10.395 INFO:teuthology.orchestra.run.smithi165.stdout: libcephfs2-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T23:54:10.395 INFO:teuthology.orchestra.run.smithi165.stdout: libcephsqlite-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T23:54:10.395 INFO:teuthology.orchestra.run.smithi165.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-10T23:54:10.395 INFO:teuthology.orchestra.run.smithi165.stdout: libnbd-1.20.2-2.el9.x86_64 2024-09-10T23:54:10.395 INFO:teuthology.orchestra.run.smithi165.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-10T23:54:10.395 INFO:teuthology.orchestra.run.smithi165.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-10T23:54:10.395 INFO:teuthology.orchestra.run.smithi165.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-10T23:54:10.395 INFO:teuthology.orchestra.run.smithi165.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-10T23:54:10.395 INFO:teuthology.orchestra.run.smithi165.stdout: librados-devel-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T23:54:10.395 INFO:teuthology.orchestra.run.smithi165.stdout: libradosstriper1-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T23:54:10.395 INFO:teuthology.orchestra.run.smithi165.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-10T23:54:10.395 INFO:teuthology.orchestra.run.smithi165.stdout: librgw2-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T23:54:10.396 INFO:teuthology.orchestra.run.smithi165.stdout: libtool-2.4.6-46.el9.x86_64 2024-09-10T23:54:10.396 INFO:teuthology.orchestra.run.smithi165.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-10T23:54:10.396 INFO:teuthology.orchestra.run.smithi165.stdout: libuuid-devel-2.37.4-20.el9.x86_64 2024-09-10T23:54:10.396 INFO:teuthology.orchestra.run.smithi165.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-09-10T23:54:10.396 INFO:teuthology.orchestra.run.smithi165.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-10T23:54:10.396 INFO:teuthology.orchestra.run.smithi165.stdout: lua-5.4.4-4.el9.x86_64 2024-09-10T23:54:10.396 INFO:teuthology.orchestra.run.smithi165.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-10T23:54:10.396 INFO:teuthology.orchestra.run.smithi165.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-10T23:54:10.396 INFO:teuthology.orchestra.run.smithi165.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-10T23:54:10.396 INFO:teuthology.orchestra.run.smithi165.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-10T23:54:10.396 INFO:teuthology.orchestra.run.smithi165.stdout: openssl-devel-1:3.2.2-6.el9.x86_64 2024-09-10T23:54:10.396 INFO:teuthology.orchestra.run.smithi165.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-10T23:54:10.396 INFO:teuthology.orchestra.run.smithi165.stdout: postgresql-13.16-1.el9.x86_64 2024-09-10T23:54:10.396 INFO:teuthology.orchestra.run.smithi165.stdout: postgresql-contrib-13.16-1.el9.x86_64 2024-09-10T23:54:10.397 INFO:teuthology.orchestra.run.smithi165.stdout: postgresql-private-libs-13.16-1.el9.x86_64 2024-09-10T23:54:10.397 INFO:teuthology.orchestra.run.smithi165.stdout: postgresql-server-13.16-1.el9.x86_64 2024-09-10T23:54:10.397 INFO:teuthology.orchestra.run.smithi165.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-10T23:54:10.397 INFO:teuthology.orchestra.run.smithi165.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-10T23:54:10.397 INFO:teuthology.orchestra.run.smithi165.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-10T23:54:10.397 INFO:teuthology.orchestra.run.smithi165.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-09-10T23:54:10.397 INFO:teuthology.orchestra.run.smithi165.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-10T23:54:10.397 INFO:teuthology.orchestra.run.smithi165.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-10T23:54:10.397 INFO:teuthology.orchestra.run.smithi165.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-10T23:54:10.397 INFO:teuthology.orchestra.run.smithi165.stdout: python3-ceph-argparse-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T23:54:10.397 INFO:teuthology.orchestra.run.smithi165.stdout: python3-ceph-common-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T23:54:10.397 INFO:teuthology.orchestra.run.smithi165.stdout: python3-cephfs-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T23:54:10.397 INFO:teuthology.orchestra.run.smithi165.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-10T23:54:10.398 INFO:teuthology.orchestra.run.smithi165.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-10T23:54:10.398 INFO:teuthology.orchestra.run.smithi165.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-10T23:54:10.398 INFO:teuthology.orchestra.run.smithi165.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-10T23:54:10.398 INFO:teuthology.orchestra.run.smithi165.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-09-10T23:54:10.398 INFO:teuthology.orchestra.run.smithi165.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-09-10T23:54:10.398 INFO:teuthology.orchestra.run.smithi165.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-10T23:54:10.398 INFO:teuthology.orchestra.run.smithi165.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-10T23:54:10.398 INFO:teuthology.orchestra.run.smithi165.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-10T23:54:10.398 INFO:teuthology.orchestra.run.smithi165.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-10T23:54:10.398 INFO:teuthology.orchestra.run.smithi165.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-10T23:54:10.398 INFO:teuthology.orchestra.run.smithi165.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-10T23:54:10.398 INFO:teuthology.orchestra.run.smithi165.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-10T23:54:10.398 INFO:teuthology.orchestra.run.smithi165.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-10T23:54:10.398 INFO:teuthology.orchestra.run.smithi165.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-10T23:54:10.399 INFO:teuthology.orchestra.run.smithi165.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-10T23:54:10.399 INFO:teuthology.orchestra.run.smithi165.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-10T23:54:10.399 INFO:teuthology.orchestra.run.smithi165.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-10T23:54:10.399 INFO:teuthology.orchestra.run.smithi165.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-10T23:54:10.399 INFO:teuthology.orchestra.run.smithi165.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-10T23:54:10.399 INFO:teuthology.orchestra.run.smithi165.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-10T23:54:10.399 INFO:teuthology.orchestra.run.smithi165.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-10T23:54:10.399 INFO:teuthology.orchestra.run.smithi165.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-10T23:54:10.399 INFO:teuthology.orchestra.run.smithi165.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-10T23:54:10.399 INFO:teuthology.orchestra.run.smithi165.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-10T23:54:10.399 INFO:teuthology.orchestra.run.smithi165.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-10T23:54:10.399 INFO:teuthology.orchestra.run.smithi165.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-10T23:54:10.399 INFO:teuthology.orchestra.run.smithi165.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-10T23:54:10.400 INFO:teuthology.orchestra.run.smithi165.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-10T23:54:10.400 INFO:teuthology.orchestra.run.smithi165.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-10T23:54:10.400 INFO:teuthology.orchestra.run.smithi165.stdout: python3-ply-3.11-14.el9.noarch 2024-09-10T23:54:10.400 INFO:teuthology.orchestra.run.smithi165.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-10T23:54:10.400 INFO:teuthology.orchestra.run.smithi165.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-10T23:54:10.400 INFO:teuthology.orchestra.run.smithi165.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-10T23:54:10.400 INFO:teuthology.orchestra.run.smithi165.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-10T23:54:10.400 INFO:teuthology.orchestra.run.smithi165.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-10T23:54:10.400 INFO:teuthology.orchestra.run.smithi165.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-10T23:54:10.400 INFO:teuthology.orchestra.run.smithi165.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-10T23:54:10.400 INFO:teuthology.orchestra.run.smithi165.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-09-10T23:54:10.400 INFO:teuthology.orchestra.run.smithi165.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-10T23:54:10.400 INFO:teuthology.orchestra.run.smithi165.stdout: python3-rados-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T23:54:10.400 INFO:teuthology.orchestra.run.smithi165.stdout: python3-rbd-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T23:54:10.401 INFO:teuthology.orchestra.run.smithi165.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-10T23:54:10.401 INFO:teuthology.orchestra.run.smithi165.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-10T23:54:10.401 INFO:teuthology.orchestra.run.smithi165.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-10T23:54:10.401 INFO:teuthology.orchestra.run.smithi165.stdout: python3-rgw-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T23:54:10.401 INFO:teuthology.orchestra.run.smithi165.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-10T23:54:10.401 INFO:teuthology.orchestra.run.smithi165.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-10T23:54:10.401 INFO:teuthology.orchestra.run.smithi165.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-10T23:54:10.401 INFO:teuthology.orchestra.run.smithi165.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-10T23:54:10.401 INFO:teuthology.orchestra.run.smithi165.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-10T23:54:10.401 INFO:teuthology.orchestra.run.smithi165.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-10T23:54:10.401 INFO:teuthology.orchestra.run.smithi165.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-10T23:54:10.401 INFO:teuthology.orchestra.run.smithi165.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-09-10T23:54:10.401 INFO:teuthology.orchestra.run.smithi165.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-10T23:54:10.401 INFO:teuthology.orchestra.run.smithi165.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-10T23:54:10.401 INFO:teuthology.orchestra.run.smithi165.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-10T23:54:10.402 INFO:teuthology.orchestra.run.smithi165.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-10T23:54:10.402 INFO:teuthology.orchestra.run.smithi165.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-10T23:54:10.402 INFO:teuthology.orchestra.run.smithi165.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-10T23:54:10.402 INFO:teuthology.orchestra.run.smithi165.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-10T23:54:10.402 INFO:teuthology.orchestra.run.smithi165.stdout: rbd-fuse-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T23:54:10.402 INFO:teuthology.orchestra.run.smithi165.stdout: rbd-mirror-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T23:54:10.402 INFO:teuthology.orchestra.run.smithi165.stdout: rbd-nbd-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T23:54:10.402 INFO:teuthology.orchestra.run.smithi165.stdout: re2-1:20211101-20.el9.x86_64 2024-09-10T23:54:10.402 INFO:teuthology.orchestra.run.smithi165.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-10T23:54:10.402 INFO:teuthology.orchestra.run.smithi165.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-10T23:54:10.402 INFO:teuthology.orchestra.run.smithi165.stdout: uuid-1.6.2-55.el9.x86_64 2024-09-10T23:54:10.402 INFO:teuthology.orchestra.run.smithi165.stdout: xfsprogs-devel-6.4.0-4.el9.x86_64 2024-09-10T23:54:10.402 INFO:teuthology.orchestra.run.smithi165.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-10T23:54:10.403 INFO:teuthology.orchestra.run.smithi165.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-09-10T23:54:10.403 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-09-10T23:54:10.403 INFO:teuthology.orchestra.run.smithi165.stdout:Complete! 2024-09-10T23:54:10.825 DEBUG:teuthology.parallel:result is None 2024-09-10T23:54:15.432 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : ceph-2:19.3.0-4854.g8f219c47.el9.x86_64 1/165 2024-09-10T23:54:15.433 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : ceph-base-2:19.3.0-4854.g8f219c47.el9.x86_64 2/165 2024-09-10T23:54:15.433 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : ceph-common-2:19.3.0-4854.g8f219c47.el9.x86_64 3/165 2024-09-10T23:54:15.433 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : ceph-fuse-2:19.3.0-4854.g8f219c47.el9.x86_64 4/165 2024-09-10T23:54:15.433 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : ceph-immutable-object-cache-2:19.3.0-4854.g8f219 5/165 2024-09-10T23:54:15.433 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : ceph-mds-2:19.3.0-4854.g8f219c47.el9.x86_64 6/165 2024-09-10T23:54:15.433 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : ceph-mgr-2:19.3.0-4854.g8f219c47.el9.x86_64 7/165 2024-09-10T23:54:15.433 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : ceph-mon-2:19.3.0-4854.g8f219c47.el9.x86_64 8/165 2024-09-10T23:54:15.433 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : ceph-osd-2:19.3.0-4854.g8f219c47.el9.x86_64 9/165 2024-09-10T23:54:15.433 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : ceph-radosgw-2:19.3.0-4854.g8f219c47.el9.x86_64 10/165 2024-09-10T23:54:15.433 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : ceph-selinux-2:19.3.0-4854.g8f219c47.el9.x86_64 11/165 2024-09-10T23:54:15.433 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : ceph-test-2:19.3.0-4854.g8f219c47.el9.x86_64 12/165 2024-09-10T23:54:15.433 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : cephfs-mirror-2:19.3.0-4854.g8f219c47.el9.x86_64 13/165 2024-09-10T23:54:15.434 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : cephfs-shell-2:19.3.0-4854.g8f219c47.el9.x86_64 14/165 2024-09-10T23:54:15.434 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : libcephfs-devel-2:19.3.0-4854.g8f219c47.el9.x86_ 15/165 2024-09-10T23:54:15.434 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : libcephfs2-2:19.3.0-4854.g8f219c47.el9.x86_64 16/165 2024-09-10T23:54:15.434 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : libcephsqlite-2:19.3.0-4854.g8f219c47.el9.x86_64 17/165 2024-09-10T23:54:15.434 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : librados-devel-2:19.3.0-4854.g8f219c47.el9.x86_6 18/165 2024-09-10T23:54:15.434 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : libradosstriper1-2:19.3.0-4854.g8f219c47.el9.x86 19/165 2024-09-10T23:54:15.434 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : librgw2-2:19.3.0-4854.g8f219c47.el9.x86_64 20/165 2024-09-10T23:54:15.434 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-ceph-argparse-2:19.3.0-4854.g8f219c47.el 21/165 2024-09-10T23:54:15.434 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-ceph-common-2:19.3.0-4854.g8f219c47.el9. 22/165 2024-09-10T23:54:15.434 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-cephfs-2:19.3.0-4854.g8f219c47.el9.x86_6 23/165 2024-09-10T23:54:15.434 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-rados-2:19.3.0-4854.g8f219c47.el9.x86_64 24/165 2024-09-10T23:54:15.434 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-rbd-2:19.3.0-4854.g8f219c47.el9.x86_64 25/165 2024-09-10T23:54:15.434 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-rgw-2:19.3.0-4854.g8f219c47.el9.x86_64 26/165 2024-09-10T23:54:15.434 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : rbd-fuse-2:19.3.0-4854.g8f219c47.el9.x86_64 27/165 2024-09-10T23:54:15.435 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : rbd-mirror-2:19.3.0-4854.g8f219c47.el9.x86_64 28/165 2024-09-10T23:54:15.435 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : rbd-nbd-2:19.3.0-4854.g8f219c47.el9.x86_64 29/165 2024-09-10T23:54:15.435 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : ceph-grafana-dashboards-2:19.3.0-4854.g8f219c47. 30/165 2024-09-10T23:54:15.435 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : ceph-mgr-cephadm-2:19.3.0-4854.g8f219c47.el9.noa 31/165 2024-09-10T23:54:15.435 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : ceph-mgr-dashboard-2:19.3.0-4854.g8f219c47.el9.n 32/165 2024-09-10T23:54:15.435 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.3.0-4854.g8f2 33/165 2024-09-10T23:54:15.435 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : ceph-mgr-modules-core-2:19.3.0-4854.g8f219c47.el 34/165 2024-09-10T23:54:15.435 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : ceph-mgr-rook-2:19.3.0-4854.g8f219c47.el9.noarch 35/165 2024-09-10T23:54:15.435 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : ceph-prometheus-alerts-2:19.3.0-4854.g8f219c47.e 36/165 2024-09-10T23:54:15.435 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : ceph-volume-2:19.3.0-4854.g8f219c47.el9.noarch 37/165 2024-09-10T23:54:15.435 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : cephadm-2:19.3.0-4854.g8f219c47.el9.noarch 38/165 2024-09-10T23:54:15.435 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : cephfs-top-2:19.3.0-4854.g8f219c47.el9.noarch 39/165 2024-09-10T23:54:15.435 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 40/165 2024-09-10T23:54:15.436 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 41/165 2024-09-10T23:54:15.436 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 42/165 2024-09-10T23:54:15.436 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/165 2024-09-10T23:54:15.436 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/165 2024-09-10T23:54:15.436 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/165 2024-09-10T23:54:15.436 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/165 2024-09-10T23:54:15.436 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/165 2024-09-10T23:54:15.436 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 48/165 2024-09-10T23:54:15.436 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/165 2024-09-10T23:54:15.436 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/165 2024-09-10T23:54:15.436 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/165 2024-09-10T23:54:15.436 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/165 2024-09-10T23:54:15.436 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/165 2024-09-10T23:54:15.437 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/165 2024-09-10T23:54:15.437 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/165 2024-09-10T23:54:15.437 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/165 2024-09-10T23:54:15.437 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/165 2024-09-10T23:54:15.437 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : libnbd-1.20.2-2.el9.x86_64 58/165 2024-09-10T23:54:15.437 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 59/165 2024-09-10T23:54:15.437 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 60/165 2024-09-10T23:54:15.437 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 61/165 2024-09-10T23:54:15.437 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : libtool-2.4.6-46.el9.x86_64 62/165 2024-09-10T23:54:15.437 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : libuuid-devel-2.37.4-20.el9.x86_64 63/165 2024-09-10T23:54:15.437 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 64/165 2024-09-10T23:54:15.437 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 65/165 2024-09-10T23:54:15.437 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : lua-5.4.4-4.el9.x86_64 66/165 2024-09-10T23:54:15.438 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 67/165 2024-09-10T23:54:15.438 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 68/165 2024-09-10T23:54:15.438 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : openssl-devel-1:3.2.2-6.el9.x86_64 69/165 2024-09-10T23:54:15.438 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : postgresql-13.16-1.el9.x86_64 70/165 2024-09-10T23:54:15.438 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : postgresql-contrib-13.16-1.el9.x86_64 71/165 2024-09-10T23:54:15.438 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : postgresql-private-libs-13.16-1.el9.x86_64 72/165 2024-09-10T23:54:15.438 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : postgresql-server-13.16-1.el9.x86_64 73/165 2024-09-10T23:54:15.438 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 74/165 2024-09-10T23:54:15.438 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 75/165 2024-09-10T23:54:15.438 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 76/165 2024-09-10T23:54:15.438 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 77/165 2024-09-10T23:54:15.438 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 78/165 2024-09-10T23:54:15.438 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 79/165 2024-09-10T23:54:15.438 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 80/165 2024-09-10T23:54:15.439 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 81/165 2024-09-10T23:54:15.439 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 82/165 2024-09-10T23:54:15.439 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 83/165 2024-09-10T23:54:15.439 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 84/165 2024-09-10T23:54:15.439 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 85/165 2024-09-10T23:54:15.439 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 86/165 2024-09-10T23:54:15.439 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 87/165 2024-09-10T23:54:15.439 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 88/165 2024-09-10T23:54:15.439 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 89/165 2024-09-10T23:54:15.439 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 90/165 2024-09-10T23:54:15.439 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 91/165 2024-09-10T23:54:15.439 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 92/165 2024-09-10T23:54:15.439 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 93/165 2024-09-10T23:54:15.439 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 94/165 2024-09-10T23:54:15.440 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 95/165 2024-09-10T23:54:15.440 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 96/165 2024-09-10T23:54:15.440 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 97/165 2024-09-10T23:54:15.440 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 98/165 2024-09-10T23:54:15.440 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 99/165 2024-09-10T23:54:15.440 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 100/165 2024-09-10T23:54:15.440 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 101/165 2024-09-10T23:54:15.440 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : xfsprogs-devel-6.4.0-4.el9.x86_64 102/165 2024-09-10T23:54:15.440 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 103/165 2024-09-10T23:54:15.440 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 104/165 2024-09-10T23:54:15.440 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 105/165 2024-09-10T23:54:15.440 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 106/165 2024-09-10T23:54:15.441 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 107/165 2024-09-10T23:54:15.441 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 108/165 2024-09-10T23:54:15.441 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 109/165 2024-09-10T23:54:15.441 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 110/165 2024-09-10T23:54:15.441 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 111/165 2024-09-10T23:54:15.441 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 112/165 2024-09-10T23:54:15.441 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 113/165 2024-09-10T23:54:15.441 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 114/165 2024-09-10T23:54:15.441 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 115/165 2024-09-10T23:54:15.441 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 116/165 2024-09-10T23:54:15.441 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 117/165 2024-09-10T23:54:15.441 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 118/165 2024-09-10T23:54:15.441 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 119/165 2024-09-10T23:54:15.442 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 120/165 2024-09-10T23:54:15.442 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 121/165 2024-09-10T23:54:15.442 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 122/165 2024-09-10T23:54:15.442 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 123/165 2024-09-10T23:54:15.442 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 124/165 2024-09-10T23:54:15.442 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 125/165 2024-09-10T23:54:15.442 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 126/165 2024-09-10T23:54:15.442 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 127/165 2024-09-10T23:54:15.442 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 128/165 2024-09-10T23:54:15.442 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 129/165 2024-09-10T23:54:15.442 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 130/165 2024-09-10T23:54:15.442 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 131/165 2024-09-10T23:54:15.442 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 132/165 2024-09-10T23:54:15.442 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 133/165 2024-09-10T23:54:15.443 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 134/165 2024-09-10T23:54:15.443 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 135/165 2024-09-10T23:54:15.443 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 136/165 2024-09-10T23:54:15.443 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 137/165 2024-09-10T23:54:15.443 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 138/165 2024-09-10T23:54:15.443 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 139/165 2024-09-10T23:54:15.443 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 140/165 2024-09-10T23:54:15.443 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 141/165 2024-09-10T23:54:15.443 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 142/165 2024-09-10T23:54:15.443 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 143/165 2024-09-10T23:54:15.443 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 144/165 2024-09-10T23:54:15.443 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 145/165 2024-09-10T23:54:15.443 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 146/165 2024-09-10T23:54:15.444 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 147/165 2024-09-10T23:54:15.444 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : re2-1:20211101-20.el9.x86_64 148/165 2024-09-10T23:54:15.444 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 149/165 2024-09-10T23:54:15.444 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : librados2-2:19.3.0-4854.g8f219c47.el9.x86_64 150/165 2024-09-10T23:54:15.444 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 151/165 2024-09-10T23:54:15.444 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : librbd1-2:19.3.0-4854.g8f219c47.el9.x86_64 152/165 2024-09-10T23:54:15.444 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 153/165 2024-09-10T23:54:15.444 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : NetworkManager-1:1.49.90-1.el9.x86_64 154/165 2024-09-10T23:54:15.444 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : NetworkManager-1:1.48.8-1.el9.x86_64 155/165 2024-09-10T23:54:15.444 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : NetworkManager-libnm-1:1.49.90-1.el9.x86_64 156/165 2024-09-10T23:54:15.444 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 157/165 2024-09-10T23:54:15.444 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : NetworkManager-team-1:1.49.90-1.el9.x86_64 158/165 2024-09-10T23:54:15.444 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : NetworkManager-team-1:1.48.8-1.el9.x86_64 159/165 2024-09-10T23:54:15.445 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : NetworkManager-tui-1:1.49.90-1.el9.x86_64 160/165 2024-09-10T23:54:15.445 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : NetworkManager-tui-1:1.48.8-1.el9.x86_64 161/165 2024-09-10T23:54:15.445 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : openssl-1:3.2.2-6.el9.x86_64 162/165 2024-09-10T23:54:15.445 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : openssl-1:3.2.2-4.el9.x86_64 163/165 2024-09-10T23:54:15.445 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : openssl-libs-1:3.2.2-6.el9.x86_64 164/165 2024-09-10T23:54:17.664 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : openssl-libs-1:3.2.2-4.el9.x86_64 165/165 2024-09-10T23:54:17.664 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-10T23:54:17.664 INFO:teuthology.orchestra.run.smithi012.stdout:Upgraded: 2024-09-10T23:54:17.664 INFO:teuthology.orchestra.run.smithi012.stdout: NetworkManager-1:1.49.90-1.el9.x86_64 2024-09-10T23:54:17.664 INFO:teuthology.orchestra.run.smithi012.stdout: NetworkManager-libnm-1:1.49.90-1.el9.x86_64 2024-09-10T23:54:17.664 INFO:teuthology.orchestra.run.smithi012.stdout: NetworkManager-team-1:1.49.90-1.el9.x86_64 2024-09-10T23:54:17.664 INFO:teuthology.orchestra.run.smithi012.stdout: NetworkManager-tui-1:1.49.90-1.el9.x86_64 2024-09-10T23:54:17.664 INFO:teuthology.orchestra.run.smithi012.stdout: librados2-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T23:54:17.664 INFO:teuthology.orchestra.run.smithi012.stdout: librbd1-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T23:54:17.665 INFO:teuthology.orchestra.run.smithi012.stdout: openssl-1:3.2.2-6.el9.x86_64 2024-09-10T23:54:17.665 INFO:teuthology.orchestra.run.smithi012.stdout: openssl-libs-1:3.2.2-6.el9.x86_64 2024-09-10T23:54:17.665 INFO:teuthology.orchestra.run.smithi012.stdout:Installed: 2024-09-10T23:54:17.665 INFO:teuthology.orchestra.run.smithi012.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-10T23:54:17.665 INFO:teuthology.orchestra.run.smithi012.stdout: automake-1.16.2-8.el9.noarch 2024-09-10T23:54:17.665 INFO:teuthology.orchestra.run.smithi012.stdout: bison-3.7.4-5.el9.x86_64 2024-09-10T23:54:17.665 INFO:teuthology.orchestra.run.smithi012.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-10T23:54:17.665 INFO:teuthology.orchestra.run.smithi012.stdout: ceph-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T23:54:17.665 INFO:teuthology.orchestra.run.smithi012.stdout: ceph-base-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T23:54:17.665 INFO:teuthology.orchestra.run.smithi012.stdout: ceph-common-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T23:54:17.665 INFO:teuthology.orchestra.run.smithi012.stdout: ceph-fuse-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T23:54:17.665 INFO:teuthology.orchestra.run.smithi012.stdout: ceph-grafana-dashboards-2:19.3.0-4854.g8f219c47.el9.noarch 2024-09-10T23:54:17.666 INFO:teuthology.orchestra.run.smithi012.stdout: ceph-immutable-object-cache-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T23:54:17.666 INFO:teuthology.orchestra.run.smithi012.stdout: ceph-mds-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T23:54:17.666 INFO:teuthology.orchestra.run.smithi012.stdout: ceph-mgr-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T23:54:17.666 INFO:teuthology.orchestra.run.smithi012.stdout: ceph-mgr-cephadm-2:19.3.0-4854.g8f219c47.el9.noarch 2024-09-10T23:54:17.666 INFO:teuthology.orchestra.run.smithi012.stdout: ceph-mgr-dashboard-2:19.3.0-4854.g8f219c47.el9.noarch 2024-09-10T23:54:17.666 INFO:teuthology.orchestra.run.smithi012.stdout: ceph-mgr-diskprediction-local-2:19.3.0-4854.g8f219c47.el9.noarch 2024-09-10T23:54:17.666 INFO:teuthology.orchestra.run.smithi012.stdout: ceph-mgr-modules-core-2:19.3.0-4854.g8f219c47.el9.noarch 2024-09-10T23:54:17.666 INFO:teuthology.orchestra.run.smithi012.stdout: ceph-mgr-rook-2:19.3.0-4854.g8f219c47.el9.noarch 2024-09-10T23:54:17.666 INFO:teuthology.orchestra.run.smithi012.stdout: ceph-mon-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T23:54:17.666 INFO:teuthology.orchestra.run.smithi012.stdout: ceph-osd-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T23:54:17.666 INFO:teuthology.orchestra.run.smithi012.stdout: ceph-prometheus-alerts-2:19.3.0-4854.g8f219c47.el9.noarch 2024-09-10T23:54:17.666 INFO:teuthology.orchestra.run.smithi012.stdout: ceph-radosgw-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T23:54:17.666 INFO:teuthology.orchestra.run.smithi012.stdout: ceph-selinux-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T23:54:17.667 INFO:teuthology.orchestra.run.smithi012.stdout: ceph-test-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T23:54:17.667 INFO:teuthology.orchestra.run.smithi012.stdout: ceph-volume-2:19.3.0-4854.g8f219c47.el9.noarch 2024-09-10T23:54:17.667 INFO:teuthology.orchestra.run.smithi012.stdout: cephadm-2:19.3.0-4854.g8f219c47.el9.noarch 2024-09-10T23:54:17.667 INFO:teuthology.orchestra.run.smithi012.stdout: cephfs-mirror-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T23:54:17.667 INFO:teuthology.orchestra.run.smithi012.stdout: cephfs-shell-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T23:54:17.667 INFO:teuthology.orchestra.run.smithi012.stdout: cephfs-top-2:19.3.0-4854.g8f219c47.el9.noarch 2024-09-10T23:54:17.667 INFO:teuthology.orchestra.run.smithi012.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-09-10T23:54:17.667 INFO:teuthology.orchestra.run.smithi012.stdout: flex-2.6.4-9.el9.x86_64 2024-09-10T23:54:17.667 INFO:teuthology.orchestra.run.smithi012.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-10T23:54:17.667 INFO:teuthology.orchestra.run.smithi012.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-10T23:54:17.667 INFO:teuthology.orchestra.run.smithi012.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-10T23:54:17.667 INFO:teuthology.orchestra.run.smithi012.stdout: fuse-2.9.9-16.el9.x86_64 2024-09-10T23:54:17.667 INFO:teuthology.orchestra.run.smithi012.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-09-10T23:54:17.667 INFO:teuthology.orchestra.run.smithi012.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-10T23:54:17.668 INFO:teuthology.orchestra.run.smithi012.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-09-10T23:54:17.668 INFO:teuthology.orchestra.run.smithi012.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-10T23:54:17.668 INFO:teuthology.orchestra.run.smithi012.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-10T23:54:17.668 INFO:teuthology.orchestra.run.smithi012.stdout: libcephfs-devel-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T23:54:17.668 INFO:teuthology.orchestra.run.smithi012.stdout: libcephfs2-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T23:54:17.668 INFO:teuthology.orchestra.run.smithi012.stdout: libcephsqlite-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T23:54:17.668 INFO:teuthology.orchestra.run.smithi012.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-10T23:54:17.668 INFO:teuthology.orchestra.run.smithi012.stdout: libnbd-1.20.2-2.el9.x86_64 2024-09-10T23:54:17.668 INFO:teuthology.orchestra.run.smithi012.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-10T23:54:17.668 INFO:teuthology.orchestra.run.smithi012.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-10T23:54:17.668 INFO:teuthology.orchestra.run.smithi012.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-10T23:54:17.668 INFO:teuthology.orchestra.run.smithi012.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-10T23:54:17.668 INFO:teuthology.orchestra.run.smithi012.stdout: librados-devel-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T23:54:17.669 INFO:teuthology.orchestra.run.smithi012.stdout: libradosstriper1-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T23:54:17.669 INFO:teuthology.orchestra.run.smithi012.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-10T23:54:17.669 INFO:teuthology.orchestra.run.smithi012.stdout: librgw2-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T23:54:17.669 INFO:teuthology.orchestra.run.smithi012.stdout: libtool-2.4.6-46.el9.x86_64 2024-09-10T23:54:17.669 INFO:teuthology.orchestra.run.smithi012.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-10T23:54:17.669 INFO:teuthology.orchestra.run.smithi012.stdout: libuuid-devel-2.37.4-20.el9.x86_64 2024-09-10T23:54:17.669 INFO:teuthology.orchestra.run.smithi012.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-09-10T23:54:17.669 INFO:teuthology.orchestra.run.smithi012.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-10T23:54:17.669 INFO:teuthology.orchestra.run.smithi012.stdout: lua-5.4.4-4.el9.x86_64 2024-09-10T23:54:17.669 INFO:teuthology.orchestra.run.smithi012.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-10T23:54:17.669 INFO:teuthology.orchestra.run.smithi012.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-10T23:54:17.669 INFO:teuthology.orchestra.run.smithi012.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-10T23:54:17.670 INFO:teuthology.orchestra.run.smithi012.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-10T23:54:17.670 INFO:teuthology.orchestra.run.smithi012.stdout: openssl-devel-1:3.2.2-6.el9.x86_64 2024-09-10T23:54:17.670 INFO:teuthology.orchestra.run.smithi012.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-10T23:54:17.670 INFO:teuthology.orchestra.run.smithi012.stdout: postgresql-13.16-1.el9.x86_64 2024-09-10T23:54:17.670 INFO:teuthology.orchestra.run.smithi012.stdout: postgresql-contrib-13.16-1.el9.x86_64 2024-09-10T23:54:17.670 INFO:teuthology.orchestra.run.smithi012.stdout: postgresql-private-libs-13.16-1.el9.x86_64 2024-09-10T23:54:17.670 INFO:teuthology.orchestra.run.smithi012.stdout: postgresql-server-13.16-1.el9.x86_64 2024-09-10T23:54:17.670 INFO:teuthology.orchestra.run.smithi012.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-10T23:54:17.671 INFO:teuthology.orchestra.run.smithi012.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-10T23:54:17.671 INFO:teuthology.orchestra.run.smithi012.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-10T23:54:17.671 INFO:teuthology.orchestra.run.smithi012.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-09-10T23:54:17.671 INFO:teuthology.orchestra.run.smithi012.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-10T23:54:17.671 INFO:teuthology.orchestra.run.smithi012.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-10T23:54:17.671 INFO:teuthology.orchestra.run.smithi012.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-10T23:54:17.671 INFO:teuthology.orchestra.run.smithi012.stdout: python3-ceph-argparse-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T23:54:17.671 INFO:teuthology.orchestra.run.smithi012.stdout: python3-ceph-common-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T23:54:17.671 INFO:teuthology.orchestra.run.smithi012.stdout: python3-cephfs-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T23:54:17.671 INFO:teuthology.orchestra.run.smithi012.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-10T23:54:17.671 INFO:teuthology.orchestra.run.smithi012.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-10T23:54:17.672 INFO:teuthology.orchestra.run.smithi012.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-10T23:54:17.672 INFO:teuthology.orchestra.run.smithi012.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-10T23:54:17.672 INFO:teuthology.orchestra.run.smithi012.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-09-10T23:54:17.672 INFO:teuthology.orchestra.run.smithi012.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-09-10T23:54:17.672 INFO:teuthology.orchestra.run.smithi012.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-10T23:54:17.672 INFO:teuthology.orchestra.run.smithi012.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-10T23:54:17.672 INFO:teuthology.orchestra.run.smithi012.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-10T23:54:17.672 INFO:teuthology.orchestra.run.smithi012.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-10T23:54:17.672 INFO:teuthology.orchestra.run.smithi012.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-10T23:54:17.672 INFO:teuthology.orchestra.run.smithi012.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-10T23:54:17.672 INFO:teuthology.orchestra.run.smithi012.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-10T23:54:17.672 INFO:teuthology.orchestra.run.smithi012.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-10T23:54:17.672 INFO:teuthology.orchestra.run.smithi012.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-10T23:54:17.673 INFO:teuthology.orchestra.run.smithi012.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-10T23:54:17.673 INFO:teuthology.orchestra.run.smithi012.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-10T23:54:17.673 INFO:teuthology.orchestra.run.smithi012.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-10T23:54:17.673 INFO:teuthology.orchestra.run.smithi012.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-10T23:54:17.673 INFO:teuthology.orchestra.run.smithi012.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-10T23:54:17.673 INFO:teuthology.orchestra.run.smithi012.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-10T23:54:17.673 INFO:teuthology.orchestra.run.smithi012.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-10T23:54:17.673 INFO:teuthology.orchestra.run.smithi012.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-10T23:54:17.673 INFO:teuthology.orchestra.run.smithi012.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-10T23:54:17.673 INFO:teuthology.orchestra.run.smithi012.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-10T23:54:17.673 INFO:teuthology.orchestra.run.smithi012.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-10T23:54:17.673 INFO:teuthology.orchestra.run.smithi012.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-10T23:54:17.674 INFO:teuthology.orchestra.run.smithi012.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-10T23:54:17.674 INFO:teuthology.orchestra.run.smithi012.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-10T23:54:17.674 INFO:teuthology.orchestra.run.smithi012.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-10T23:54:17.674 INFO:teuthology.orchestra.run.smithi012.stdout: python3-ply-3.11-14.el9.noarch 2024-09-10T23:54:17.674 INFO:teuthology.orchestra.run.smithi012.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-10T23:54:17.674 INFO:teuthology.orchestra.run.smithi012.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-10T23:54:17.674 INFO:teuthology.orchestra.run.smithi012.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-10T23:54:17.674 INFO:teuthology.orchestra.run.smithi012.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-10T23:54:17.674 INFO:teuthology.orchestra.run.smithi012.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-10T23:54:17.674 INFO:teuthology.orchestra.run.smithi012.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-10T23:54:17.674 INFO:teuthology.orchestra.run.smithi012.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-10T23:54:17.674 INFO:teuthology.orchestra.run.smithi012.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-09-10T23:54:17.675 INFO:teuthology.orchestra.run.smithi012.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-10T23:54:17.675 INFO:teuthology.orchestra.run.smithi012.stdout: python3-rados-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T23:54:17.675 INFO:teuthology.orchestra.run.smithi012.stdout: python3-rbd-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T23:54:17.675 INFO:teuthology.orchestra.run.smithi012.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-10T23:54:17.675 INFO:teuthology.orchestra.run.smithi012.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-10T23:54:17.675 INFO:teuthology.orchestra.run.smithi012.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-10T23:54:17.675 INFO:teuthology.orchestra.run.smithi012.stdout: python3-rgw-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T23:54:17.675 INFO:teuthology.orchestra.run.smithi012.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-10T23:54:17.675 INFO:teuthology.orchestra.run.smithi012.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-10T23:54:17.676 INFO:teuthology.orchestra.run.smithi012.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-10T23:54:17.676 INFO:teuthology.orchestra.run.smithi012.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-10T23:54:17.676 INFO:teuthology.orchestra.run.smithi012.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-10T23:54:17.676 INFO:teuthology.orchestra.run.smithi012.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-10T23:54:17.676 INFO:teuthology.orchestra.run.smithi012.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-10T23:54:17.676 INFO:teuthology.orchestra.run.smithi012.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-09-10T23:54:17.676 INFO:teuthology.orchestra.run.smithi012.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-10T23:54:17.676 INFO:teuthology.orchestra.run.smithi012.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-10T23:54:17.677 INFO:teuthology.orchestra.run.smithi012.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-10T23:54:17.677 INFO:teuthology.orchestra.run.smithi012.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-10T23:54:17.677 INFO:teuthology.orchestra.run.smithi012.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-10T23:54:17.677 INFO:teuthology.orchestra.run.smithi012.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-10T23:54:17.677 INFO:teuthology.orchestra.run.smithi012.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-10T23:54:17.677 INFO:teuthology.orchestra.run.smithi012.stdout: rbd-fuse-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T23:54:17.677 INFO:teuthology.orchestra.run.smithi012.stdout: rbd-mirror-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T23:54:17.677 INFO:teuthology.orchestra.run.smithi012.stdout: rbd-nbd-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T23:54:17.677 INFO:teuthology.orchestra.run.smithi012.stdout: re2-1:20211101-20.el9.x86_64 2024-09-10T23:54:17.677 INFO:teuthology.orchestra.run.smithi012.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-10T23:54:17.677 INFO:teuthology.orchestra.run.smithi012.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-10T23:54:17.677 INFO:teuthology.orchestra.run.smithi012.stdout: uuid-1.6.2-55.el9.x86_64 2024-09-10T23:54:17.677 INFO:teuthology.orchestra.run.smithi012.stdout: xfsprogs-devel-6.4.0-4.el9.x86_64 2024-09-10T23:54:17.678 INFO:teuthology.orchestra.run.smithi012.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-10T23:54:17.678 INFO:teuthology.orchestra.run.smithi012.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-09-10T23:54:17.678 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-10T23:54:17.678 INFO:teuthology.orchestra.run.smithi012.stdout:Complete! 2024-09-10T23:54:18.421 DEBUG:teuthology.parallel:result is None 2024-09-10T23:54:18.422 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=8f219c476cb4f26fc5719c661c0323719a45cb30 2024-09-10T23:54:18.577 DEBUG:teuthology.orchestra.run.smithi012:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-10T23:54:18.623 INFO:teuthology.orchestra.run.smithi012.stdout:19.3.0-4854.g8f219c47.el9 2024-09-10T23:54:18.624 INFO:teuthology.packaging:The installed version of ceph is 19.3.0-4854.g8f219c47.el9 2024-09-10T23:54:18.624 INFO:teuthology.task.install:The correct ceph version 19.3.0-4854.g8f219c47 is installed. 2024-09-10T23:54:18.626 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=8f219c476cb4f26fc5719c661c0323719a45cb30 2024-09-10T23:54:18.772 DEBUG:teuthology.orchestra.run.smithi165:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-10T23:54:18.799 INFO:teuthology.orchestra.run.smithi165.stdout:19.3.0-4854.g8f219c47.el9 2024-09-10T23:54:18.800 INFO:teuthology.packaging:The installed version of ceph is 19.3.0-4854.g8f219c47.el9 2024-09-10T23:54:18.800 INFO:teuthology.task.install:The correct ceph version 19.3.0-4854.g8f219c47 is installed. 2024-09-10T23:54:18.802 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=8f219c476cb4f26fc5719c661c0323719a45cb30 2024-09-10T23:54:18.948 DEBUG:teuthology.orchestra.run.smithi178:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-10T23:54:18.975 INFO:teuthology.orchestra.run.smithi178.stdout:19.3.0-4854.g8f219c47.el9 2024-09-10T23:54:18.975 INFO:teuthology.packaging:The installed version of ceph is 19.3.0-4854.g8f219c47.el9 2024-09-10T23:54:18.975 INFO:teuthology.task.install:The correct ceph version 19.3.0-4854.g8f219c47 is installed. 2024-09-10T23:54:18.977 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-09-10T23:54:18.977 DEBUG:teuthology.orchestra.run.smithi012:> set -ex 2024-09-10T23:54:18.977 DEBUG:teuthology.orchestra.run.smithi012:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-10T23:54:19.041 DEBUG:teuthology.orchestra.run.smithi165:> set -ex 2024-09-10T23:54:19.041 DEBUG:teuthology.orchestra.run.smithi165:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-10T23:54:19.074 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2024-09-10T23:54:19.074 DEBUG:teuthology.orchestra.run.smithi178:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-10T23:54:19.107 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-09-10T23:54:19.108 DEBUG:teuthology.orchestra.run.smithi012:> set -ex 2024-09-10T23:54:19.108 DEBUG:teuthology.orchestra.run.smithi012:> sudo dd of=/usr/bin/daemon-helper 2024-09-10T23:54:19.169 DEBUG:teuthology.orchestra.run.smithi012:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-10T23:54:19.268 DEBUG:teuthology.orchestra.run.smithi165:> set -ex 2024-09-10T23:54:19.268 DEBUG:teuthology.orchestra.run.smithi165:> sudo dd of=/usr/bin/daemon-helper 2024-09-10T23:54:19.301 DEBUG:teuthology.orchestra.run.smithi165:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-10T23:54:19.370 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2024-09-10T23:54:19.370 DEBUG:teuthology.orchestra.run.smithi178:> sudo dd of=/usr/bin/daemon-helper 2024-09-10T23:54:19.401 DEBUG:teuthology.orchestra.run.smithi178:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-10T23:54:19.470 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-09-10T23:54:19.471 DEBUG:teuthology.orchestra.run.smithi012:> set -ex 2024-09-10T23:54:19.471 DEBUG:teuthology.orchestra.run.smithi012:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-10T23:54:19.532 DEBUG:teuthology.orchestra.run.smithi012:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-10T23:54:19.636 DEBUG:teuthology.orchestra.run.smithi165:> set -ex 2024-09-10T23:54:19.636 DEBUG:teuthology.orchestra.run.smithi165:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-10T23:54:19.663 DEBUG:teuthology.orchestra.run.smithi165:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-10T23:54:19.728 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2024-09-10T23:54:19.728 DEBUG:teuthology.orchestra.run.smithi178:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-10T23:54:19.753 DEBUG:teuthology.orchestra.run.smithi178:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-10T23:54:19.819 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-09-10T23:54:19.819 DEBUG:teuthology.orchestra.run.smithi012:> set -ex 2024-09-10T23:54:19.819 DEBUG:teuthology.orchestra.run.smithi012:> sudo dd of=/usr/bin/stdin-killer 2024-09-10T23:54:19.879 DEBUG:teuthology.orchestra.run.smithi012:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-10T23:54:19.985 DEBUG:teuthology.orchestra.run.smithi165:> set -ex 2024-09-10T23:54:19.985 DEBUG:teuthology.orchestra.run.smithi165:> sudo dd of=/usr/bin/stdin-killer 2024-09-10T23:54:20.010 DEBUG:teuthology.orchestra.run.smithi165:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-10T23:54:20.076 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2024-09-10T23:54:20.076 DEBUG:teuthology.orchestra.run.smithi178:> sudo dd of=/usr/bin/stdin-killer 2024-09-10T23:54:20.101 DEBUG:teuthology.orchestra.run.smithi178:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-10T23:54:20.166 INFO:teuthology.run_tasks:Running task cephadm... 2024-09-10T23:54:20.272 INFO:tasks.cephadm:Config: {'roleless': False, 'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd'], 'max_mds': 5, 'session_timeout': 300, 'standby_replay': True}, 'cluster-conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 2, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mon': {'mon op complaint time': 120, 'mon warn on pool no app': False}, 'osd': {'osd op complaint time': 180}}, 'conf': {'global': {'ms die on skipped message': False}, 'mgr': {'cephsqlite lock renewal timeout': 900000, 'client mount timeout': 30, 'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd shutdown pgref assert': True, 'osd_max_omap_entries_per_request': 10000}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'fs.*is degraded', 'filesystem is degraded', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'filesystem is offline', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'insufficient standby MDS daemons available', 'MDS_UP_LESS_THAN_MAX', 'online, but wants', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'Degraded data redundancy'], 'sha1': '8f219c476cb4f26fc5719c661c0323719a45cb30', 'subvols': {'create': 2, 'subvol_options': '--namespace-isolated --size 25000000000'}} 2024-09-10T23:54:20.272 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 2024-09-10T23:54:20.273 INFO:tasks.cephadm:Cluster fsid is ff3ad74a-6fcf-11ef-bcea-c7b262605968 2024-09-10T23:54:20.273 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-09-10T23:54:20.273 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.12', 'mon.b': '172.21.15.165', 'mon.c': '172.21.15.178'} 2024-09-10T23:54:20.274 INFO:tasks.cephadm:First mon is mon.a on smithi012 2024-09-10T23:54:20.274 INFO:tasks.cephadm:First mgr is x 2024-09-10T23:54:20.274 INFO:tasks.cephadm:Normalizing hostnames... 2024-09-10T23:54:20.274 DEBUG:teuthology.orchestra.run.smithi012:> sudo hostname $(hostname -s) 2024-09-10T23:54:20.337 DEBUG:teuthology.orchestra.run.smithi165:> sudo hostname $(hostname -s) 2024-09-10T23:54:20.362 DEBUG:teuthology.orchestra.run.smithi178:> sudo hostname $(hostname -s) 2024-09-10T23:54:20.387 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2024-09-10T23:54:20.388 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=8f219c476cb4f26fc5719c661c0323719a45cb30 2024-09-10T23:54:20.532 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': '8f219c476cb4f26fc5719c661c0323719a45cb30', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/82938/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.3.0-4854-g8f219c47', 'node_name': '172.21.2.8+braggi08', 'job_name': 'ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.3.0-4854.g8f219c47'}, 'url': 'https://3.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-09-10 09:27:13.215997', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'wip-hemanth-testing-2024-09-10-0723', 'chacra_url': 'https://3.chacra.ceph.com/repos/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/centos/9/flavors/default/', 'archs': ['source', 'x86_64'], 'distro': 'centos'}] 2024-09-10T23:54:20.675 INFO:tasks.util.chacra:got chacra host 3.chacra.ceph.com, ref wip-hemanth-testing-2024-09-10-0723, sha1 8f219c476cb4f26fc5719c661c0323719a45cb30 from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F9%2Fx86_64&flavor=default&sha1=8f219c476cb4f26fc5719c661c0323719a45cb30 2024-09-10T23:54:20.677 INFO:tasks.cephadm:Discovered cachra url: https://3.chacra.ceph.com/binaries/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/centos/9/x86_64/flavors/default/cephadm 2024-09-10T23:54:20.677 INFO:tasks.cephadm:Downloading cephadm from url: https://3.chacra.ceph.com/binaries/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/centos/9/x86_64/flavors/default/cephadm 2024-09-10T23:54:20.677 DEBUG:teuthology.orchestra.run.smithi012:> curl --silent -L https://3.chacra.ceph.com/binaries/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-09-10T23:54:21.082 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r--. 1 ubuntu ubuntu 803561 Sep 10 23:54 /home/ubuntu/cephtest/cephadm 2024-09-10T23:54:21.082 DEBUG:teuthology.orchestra.run.smithi165:> curl --silent -L https://3.chacra.ceph.com/binaries/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-09-10T23:54:21.460 INFO:teuthology.orchestra.run.smithi165.stdout:-rw-r--r--. 1 ubuntu ubuntu 803561 Sep 10 23:54 /home/ubuntu/cephtest/cephadm 2024-09-10T23:54:21.460 DEBUG:teuthology.orchestra.run.smithi178:> curl --silent -L https://3.chacra.ceph.com/binaries/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-09-10T23:54:21.852 INFO:teuthology.orchestra.run.smithi178.stdout:-rw-r--r--. 1 ubuntu ubuntu 803561 Sep 10 23:54 /home/ubuntu/cephtest/cephadm 2024-09-10T23:54:21.853 DEBUG:teuthology.orchestra.run.smithi012:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-09-10T23:54:21.893 DEBUG:teuthology.orchestra.run.smithi165:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-09-10T23:54:21.914 DEBUG:teuthology.orchestra.run.smithi178:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-09-10T23:54:21.950 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 on all hosts... 2024-09-10T23:54:21.950 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 pull 2024-09-10T23:54:21.955 DEBUG:teuthology.orchestra.run.smithi165:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 pull 2024-09-10T23:54:21.958 DEBUG:teuthology.orchestra.run.smithi178:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 pull 2024-09-10T23:54:23.566 INFO:teuthology.orchestra.run.smithi165.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30... 2024-09-10T23:54:23.621 INFO:teuthology.orchestra.run.smithi178.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30... 2024-09-10T23:54:23.995 INFO:teuthology.orchestra.run.smithi012.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30... 2024-09-10T23:55:01.965 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-09-10T23:55:01.966 INFO:teuthology.orchestra.run.smithi165.stdout: "ceph_version": "ceph version 19.3.0-4854-g8f219c47 (8f219c476cb4f26fc5719c661c0323719a45cb30) squid (dev)", 2024-09-10T23:55:01.966 INFO:teuthology.orchestra.run.smithi165.stdout: "image_id": "589dd18248e494ba411dc838aa4ae0e5e569213f08fcd7a2cf22cd3ef7e3c480", 2024-09-10T23:55:01.966 INFO:teuthology.orchestra.run.smithi165.stdout: "repo_digests": [ 2024-09-10T23:55:01.966 INFO:teuthology.orchestra.run.smithi165.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e" 2024-09-10T23:55:01.966 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-10T23:55:01.966 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-09-10T23:55:03.955 INFO:teuthology.orchestra.run.smithi178.stdout:{ 2024-09-10T23:55:03.955 INFO:teuthology.orchestra.run.smithi178.stdout: "ceph_version": "ceph version 19.3.0-4854-g8f219c47 (8f219c476cb4f26fc5719c661c0323719a45cb30) squid (dev)", 2024-09-10T23:55:03.955 INFO:teuthology.orchestra.run.smithi178.stdout: "image_id": "589dd18248e494ba411dc838aa4ae0e5e569213f08fcd7a2cf22cd3ef7e3c480", 2024-09-10T23:55:03.955 INFO:teuthology.orchestra.run.smithi178.stdout: "repo_digests": [ 2024-09-10T23:55:03.955 INFO:teuthology.orchestra.run.smithi178.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e" 2024-09-10T23:55:03.955 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-10T23:55:03.955 INFO:teuthology.orchestra.run.smithi178.stdout:} 2024-09-10T23:55:05.703 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-09-10T23:55:05.703 INFO:teuthology.orchestra.run.smithi012.stdout: "ceph_version": "ceph version 19.3.0-4854-g8f219c47 (8f219c476cb4f26fc5719c661c0323719a45cb30) squid (dev)", 2024-09-10T23:55:05.704 INFO:teuthology.orchestra.run.smithi012.stdout: "image_id": "589dd18248e494ba411dc838aa4ae0e5e569213f08fcd7a2cf22cd3ef7e3c480", 2024-09-10T23:55:05.704 INFO:teuthology.orchestra.run.smithi012.stdout: "repo_digests": [ 2024-09-10T23:55:05.704 INFO:teuthology.orchestra.run.smithi012.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e" 2024-09-10T23:55:05.704 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-10T23:55:05.704 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-09-10T23:55:05.735 DEBUG:teuthology.orchestra.run.smithi012:> sudo mkdir -p /etc/ceph 2024-09-10T23:55:05.793 DEBUG:teuthology.orchestra.run.smithi165:> sudo mkdir -p /etc/ceph 2024-09-10T23:55:05.827 DEBUG:teuthology.orchestra.run.smithi178:> sudo mkdir -p /etc/ceph 2024-09-10T23:55:05.860 DEBUG:teuthology.orchestra.run.smithi012:> sudo chmod 777 /etc/ceph 2024-09-10T23:55:05.916 DEBUG:teuthology.orchestra.run.smithi165:> sudo chmod 777 /etc/ceph 2024-09-10T23:55:05.945 DEBUG:teuthology.orchestra.run.smithi178:> sudo chmod 777 /etc/ceph 2024-09-10T23:55:05.974 INFO:tasks.cephadm:Writing seed config... 2024-09-10T23:55:05.975 INFO:tasks.cephadm: override: [global] ms die on skipped message = False 2024-09-10T23:55:05.975 INFO:tasks.cephadm: override: [mgr] cephsqlite lock renewal timeout = 900000 2024-09-10T23:55:05.975 INFO:tasks.cephadm: override: [mgr] client mount timeout = 30 2024-09-10T23:55:05.975 INFO:tasks.cephadm: override: [mgr] debug client = 20 2024-09-10T23:55:05.975 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-09-10T23:55:05.976 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-09-10T23:55:05.976 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-09-10T23:55:05.976 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-09-10T23:55:05.976 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-09-10T23:55:05.976 INFO:tasks.cephadm: override: [mon] mon down mkfs grace = 300 2024-09-10T23:55:05.976 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2024-09-10T23:55:05.976 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2024-09-10T23:55:05.976 INFO:tasks.cephadm: override: [osd] debug bluefs = 20 2024-09-10T23:55:05.976 INFO:tasks.cephadm: override: [osd] debug bluestore = 20 2024-09-10T23:55:05.976 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-09-10T23:55:05.976 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-09-10T23:55:05.976 INFO:tasks.cephadm: override: [osd] debug rocksdb = 10 2024-09-10T23:55:05.976 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2024-09-10T23:55:05.976 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2024-09-10T23:55:05.977 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2024-09-10T23:55:05.977 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2024-09-10T23:55:05.977 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2024-09-10T23:55:05.977 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2024-09-10T23:55:05.977 INFO:tasks.cephadm: override: [osd] osd_max_omap_entries_per_request = 10000 2024-09-10T23:55:05.978 DEBUG:teuthology.orchestra.run.smithi012:> set -ex 2024-09-10T23:55:05.978 DEBUG:teuthology.orchestra.run.smithi012:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-09-10T23:55:06.013 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log to file = true mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = ff3ad74a-6fcf-11ef-bcea-c7b262605968 ms die on skipped message = False [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true bluestore block size = 96636764160 bluestore fsck on mount = True debug bluefs = 20 debug bluestore = 20 debug ms = 1 debug osd = 20 debug rocksdb = 10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd_max_omap_entries_per_request = 10000 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false cephsqlite lock renewal timeout = 900000 client mount timeout = 30 debug client = 20 debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon down mkfs grace = 300 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true 2024-09-10T23:55:06.013 DEBUG:teuthology.orchestra.run.smithi012:mon.a> sudo journalctl -f -n 0 -u ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968@mon.a.service 2024-09-10T23:55:06.058 DEBUG:teuthology.orchestra.run.smithi012:mgr.x> sudo journalctl -f -n 0 -u ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968@mgr.x.service 2024-09-10T23:55:06.102 INFO:tasks.cephadm:Bootstrapping... 2024-09-10T23:55:06.102 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 -v bootstrap --fsid ff3ad74a-6fcf-11ef-bcea-c7b262605968 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id x --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.12 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-09-10T23:55:06.367 INFO:teuthology.orchestra.run.smithi012.stdout:-------------------------------------------------------------------------------- 2024-09-10T23:55:06.368 INFO:teuthology.orchestra.run.smithi012.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30', '-v', 'bootstrap', '--fsid', 'ff3ad74a-6fcf-11ef-bcea-c7b262605968', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'x', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.12', '--skip-admin-label'] 2024-09-10T23:55:06.368 INFO:teuthology.orchestra.run.smithi012.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-09-10T23:55:06.368 INFO:teuthology.orchestra.run.smithi012.stdout:Verifying podman|docker is present... 2024-09-10T23:55:06.421 INFO:teuthology.orchestra.run.smithi012.stdout:/bin/podman: stdout 5.2.2 2024-09-10T23:55:06.421 INFO:teuthology.orchestra.run.smithi012.stdout:Verifying lvm2 is present... 2024-09-10T23:55:06.421 INFO:teuthology.orchestra.run.smithi012.stdout:Verifying time synchronization is in place... 2024-09-10T23:55:06.434 INFO:teuthology.orchestra.run.smithi012.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-09-10T23:55:06.434 INFO:teuthology.orchestra.run.smithi012.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-09-10T23:55:06.447 INFO:teuthology.orchestra.run.smithi012.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-09-10T23:55:06.447 INFO:teuthology.orchestra.run.smithi012.stdout:systemctl: stdout inactive 2024-09-10T23:55:06.461 INFO:teuthology.orchestra.run.smithi012.stdout:systemctl: stdout enabled 2024-09-10T23:55:06.473 INFO:teuthology.orchestra.run.smithi012.stdout:systemctl: stdout active 2024-09-10T23:55:06.473 INFO:teuthology.orchestra.run.smithi012.stdout:Unit chronyd.service is enabled and running 2024-09-10T23:55:06.473 INFO:teuthology.orchestra.run.smithi012.stdout:Repeating the final host check... 2024-09-10T23:55:06.516 INFO:teuthology.orchestra.run.smithi012.stdout:/bin/podman: stdout 5.2.2 2024-09-10T23:55:06.517 INFO:teuthology.orchestra.run.smithi012.stdout:podman (/bin/podman) version 5.2.2 is present 2024-09-10T23:55:06.517 INFO:teuthology.orchestra.run.smithi012.stdout:systemctl is present 2024-09-10T23:55:06.517 INFO:teuthology.orchestra.run.smithi012.stdout:lvcreate is present 2024-09-10T23:55:06.529 INFO:teuthology.orchestra.run.smithi012.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-09-10T23:55:06.530 INFO:teuthology.orchestra.run.smithi012.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-09-10T23:55:06.542 INFO:teuthology.orchestra.run.smithi012.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-09-10T23:55:06.542 INFO:teuthology.orchestra.run.smithi012.stdout:systemctl: stdout inactive 2024-09-10T23:55:06.555 INFO:teuthology.orchestra.run.smithi012.stdout:systemctl: stdout enabled 2024-09-10T23:55:06.568 INFO:teuthology.orchestra.run.smithi012.stdout:systemctl: stdout active 2024-09-10T23:55:06.568 INFO:teuthology.orchestra.run.smithi012.stdout:Unit chronyd.service is enabled and running 2024-09-10T23:55:06.568 INFO:teuthology.orchestra.run.smithi012.stdout:Host looks OK 2024-09-10T23:55:06.568 INFO:teuthology.orchestra.run.smithi012.stdout:Cluster fsid: ff3ad74a-6fcf-11ef-bcea-c7b262605968 2024-09-10T23:55:06.569 INFO:teuthology.orchestra.run.smithi012.stdout:Acquiring lock 139769484780400 on /run/cephadm/ff3ad74a-6fcf-11ef-bcea-c7b262605968.lock 2024-09-10T23:55:06.569 INFO:teuthology.orchestra.run.smithi012.stdout:Lock 139769484780400 acquired on /run/cephadm/ff3ad74a-6fcf-11ef-bcea-c7b262605968.lock 2024-09-10T23:55:06.569 INFO:teuthology.orchestra.run.smithi012.stdout:Verifying IP 172.21.15.12 port 3300 ... 2024-09-10T23:55:06.569 INFO:teuthology.orchestra.run.smithi012.stdout:Verifying IP 172.21.15.12 port 6789 ... 2024-09-10T23:55:06.570 INFO:teuthology.orchestra.run.smithi012.stdout:Base mon IP(s) is [172.21.15.12:3300, 172.21.15.12:6789], mon addrv is [v2:172.21.15.12:3300,v1:172.21.15.12:6789] 2024-09-10T23:55:06.576 INFO:teuthology.orchestra.run.smithi012.stdout:/sbin/ip: stdout default via 172.21.15.254 dev ens1f0 proto dhcp src 172.21.15.12 metric 100 2024-09-10T23:55:06.576 INFO:teuthology.orchestra.run.smithi012.stdout:/sbin/ip: stdout 172.21.0.0/20 dev ens1f0 proto kernel scope link src 172.21.15.12 metric 100 2024-09-10T23:55:06.582 INFO:teuthology.orchestra.run.smithi012.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-09-10T23:55:06.582 INFO:teuthology.orchestra.run.smithi012.stdout:/sbin/ip: stdout fe80::/64 dev ens1f0 proto kernel metric 256 pref medium 2024-09-10T23:55:06.582 INFO:teuthology.orchestra.run.smithi012.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev ens1f0 proto ra metric 1024 expires 1799sec hoplimit 64 pref medium 2024-09-10T23:55:06.588 INFO:teuthology.orchestra.run.smithi012.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-09-10T23:55:06.588 INFO:teuthology.orchestra.run.smithi012.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-09-10T23:55:06.588 INFO:teuthology.orchestra.run.smithi012.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-09-10T23:55:06.588 INFO:teuthology.orchestra.run.smithi012.stdout:/sbin/ip: stdout 4: ens1f0: mtu 1500 state UP qlen 1000 2024-09-10T23:55:06.588 INFO:teuthology.orchestra.run.smithi012.stdout:/sbin/ip: stdout inet6 fe80::ae1f:6bff:fef8:25ea/64 scope link 2024-09-10T23:55:06.589 INFO:teuthology.orchestra.run.smithi012.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-09-10T23:55:06.589 INFO:teuthology.orchestra.run.smithi012.stdout:Mon IP `172.21.15.12` is in CIDR network `172.21.0.0/20` 2024-09-10T23:55:06.589 INFO:teuthology.orchestra.run.smithi012.stdout:Mon IP `172.21.15.12` is in CIDR network `172.21.0.0/20` 2024-09-10T23:55:06.589 INFO:teuthology.orchestra.run.smithi012.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-09-10T23:55:06.590 INFO:teuthology.orchestra.run.smithi012.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-09-10T23:55:06.591 INFO:teuthology.orchestra.run.smithi012.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30... 2024-09-10T23:55:07.090 INFO:teuthology.orchestra.run.smithi012.stdout:/bin/podman: stdout 589dd18248e494ba411dc838aa4ae0e5e569213f08fcd7a2cf22cd3ef7e3c480 2024-09-10T23:55:07.090 INFO:teuthology.orchestra.run.smithi012.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30... 2024-09-10T23:55:07.090 INFO:teuthology.orchestra.run.smithi012.stdout:/bin/podman: stderr Getting image source signatures 2024-09-10T23:55:07.090 INFO:teuthology.orchestra.run.smithi012.stdout:/bin/podman: stderr Copying blob sha256:edd8cbe4deb828f809e8d76f308301dd93e971948e909d25cc03015fed1fd796 2024-09-10T23:55:07.090 INFO:teuthology.orchestra.run.smithi012.stdout:/bin/podman: stderr Copying blob sha256:e8b54c863393b7a8216a71737771b73b16a8e43ec7acf927c7faa175c255e551 2024-09-10T23:55:07.091 INFO:teuthology.orchestra.run.smithi012.stdout:/bin/podman: stderr Copying config sha256:589dd18248e494ba411dc838aa4ae0e5e569213f08fcd7a2cf22cd3ef7e3c480 2024-09-10T23:55:07.091 INFO:teuthology.orchestra.run.smithi012.stdout:/bin/podman: stderr Writing manifest to image destination 2024-09-10T23:55:08.167 INFO:teuthology.orchestra.run.smithi012.stdout:ceph: stdout ceph version 19.3.0-4854-g8f219c47 (8f219c476cb4f26fc5719c661c0323719a45cb30) squid (dev) 2024-09-10T23:55:08.167 INFO:teuthology.orchestra.run.smithi012.stdout:Ceph version: ceph version 19.3.0-4854-g8f219c47 (8f219c476cb4f26fc5719c661c0323719a45cb30) squid (dev) 2024-09-10T23:55:08.167 INFO:teuthology.orchestra.run.smithi012.stdout:Extracting ceph user uid/gid from container image... 2024-09-10T23:55:09.155 INFO:teuthology.orchestra.run.smithi012.stdout:stat: stdout 167 167 2024-09-10T23:55:09.156 INFO:teuthology.orchestra.run.smithi012.stdout:Creating initial keys... 2024-09-10T23:55:10.145 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph-authtool: stdout AQBd3OBmEnUVHxAABRc4G8oOJaG8phcTdbBsOQ== 2024-09-10T23:55:11.195 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph-authtool: stdout AQBe3OBmXCJPIRAAZaguKlhueyw9zcjxcwW7Hw== 2024-09-10T23:55:12.233 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph-authtool: stdout AQBf3OBmFec9IxAAicozZPU9Y2GZe9sXmHgGzA== 2024-09-10T23:55:12.234 INFO:teuthology.orchestra.run.smithi012.stdout:Creating initial monmap... 2024-09-10T23:55:13.248 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-09-10T23:55:13.248 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = quincy 2024-09-10T23:55:13.248 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to ff3ad74a-6fcf-11ef-bcea-c7b262605968 2024-09-10T23:55:13.248 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-09-10T23:55:13.248 INFO:teuthology.orchestra.run.smithi012.stdout:monmaptool for a [v2:172.21.15.12:3300,v1:172.21.15.12:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-09-10T23:55:13.248 INFO:teuthology.orchestra.run.smithi012.stdout:setting min_mon_release = quincy 2024-09-10T23:55:13.249 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/monmaptool: set fsid to ff3ad74a-6fcf-11ef-bcea-c7b262605968 2024-09-10T23:55:13.249 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-09-10T23:55:13.249 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-10T23:55:13.249 INFO:teuthology.orchestra.run.smithi012.stdout:Creating mon... 2024-09-10T23:55:14.228 INFO:teuthology.orchestra.run.smithi012.stdout:create mon.a on 2024-09-10T23:55:14.529 INFO:teuthology.orchestra.run.smithi012.stdout:systemctl: stderr Removed "/etc/systemd/system/multi-user.target.wants/ceph.target". 2024-09-10T23:55:14.756 INFO:teuthology.orchestra.run.smithi012.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-09-10T23:55:15.019 INFO:teuthology.orchestra.run.smithi012.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968.target → /etc/systemd/system/ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968.target. 2024-09-10T23:55:15.019 INFO:teuthology.orchestra.run.smithi012.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968.target → /etc/systemd/system/ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968.target. 2024-09-10T23:55:15.380 INFO:teuthology.orchestra.run.smithi012.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968@mon.a 2024-09-10T23:55:15.380 INFO:teuthology.orchestra.run.smithi012.stdout:systemctl: stderr Failed to reset failed state of unit ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968@mon.a.service: Unit ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968@mon.a.service not loaded. 2024-09-10T23:55:15.651 INFO:teuthology.orchestra.run.smithi012.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968.target.wants/ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968@mon.a.service → /etc/systemd/system/ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968@.service. 2024-09-10T23:55:16.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:55:16 smithi012 podman[29950]: 2024-09-10 23:55:16.257890632 +0000 UTC m=+0.347214111 container start 1100b48335d9b35274008fb8b92cef127dba9f8f77a5de12f3649555d34f7b6b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30, name=ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-mon-a, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-10T23:55:16.378 INFO:teuthology.orchestra.run.smithi012.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-10T23:55:16.378 INFO:teuthology.orchestra.run.smithi012.stdout:systemctl: stdout disabled 2024-09-10T23:55:16.391 INFO:teuthology.orchestra.run.smithi012.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-10T23:55:16.392 INFO:teuthology.orchestra.run.smithi012.stdout:systemctl: stdout inactive 2024-09-10T23:55:16.392 INFO:teuthology.orchestra.run.smithi012.stdout:firewalld.service is not enabled 2024-09-10T23:55:16.392 INFO:teuthology.orchestra.run.smithi012.stdout:Not possible to enable service . firewalld.service is not available 2024-09-10T23:55:16.392 INFO:teuthology.orchestra.run.smithi012.stdout:Waiting for mon to start... 2024-09-10T23:55:16.392 INFO:teuthology.orchestra.run.smithi012.stdout:Waiting for mon... 2024-09-10T23:55:16.638 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:55:16 smithi012 ceph-mon[29968]: mkfs ff3ad74a-6fcf-11ef-bcea-c7b262605968 2024-09-10T23:55:16.638 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:55:16 smithi012 ceph-mon[29968]: mon.a is new leader, mons a in quorum (ranks 0) 2024-09-10T23:55:16.638 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:55:16 smithi012 bash[29950]: 1100b48335d9b35274008fb8b92cef127dba9f8f77a5de12f3649555d34f7b6b 2024-09-10T23:55:16.638 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:55:16 smithi012 systemd[1]: Started Ceph mon.a for ff3ad74a-6fcf-11ef-bcea-c7b262605968. 2024-09-10T23:55:17.808 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:55:17 smithi012 ceph-mon[29968]: mon.a is new leader, mons a in quorum (ranks 0) 2024-09-10T23:55:17.808 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:55:17 smithi012 ceph-mon[29968]: monmap epoch 1 2024-09-10T23:55:17.809 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:55:17 smithi012 ceph-mon[29968]: fsid ff3ad74a-6fcf-11ef-bcea-c7b262605968 2024-09-10T23:55:17.809 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:55:17 smithi012 ceph-mon[29968]: last_changed 2024-09-10T23:55:12.644680+0000 2024-09-10T23:55:17.809 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:55:17 smithi012 ceph-mon[29968]: created 2024-09-10T23:55:12.644680+0000 2024-09-10T23:55:17.809 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:55:17 smithi012 ceph-mon[29968]: min_mon_release 19 (squid) 2024-09-10T23:55:17.809 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:55:17 smithi012 ceph-mon[29968]: election_strategy: 1 2024-09-10T23:55:17.809 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:55:17 smithi012 ceph-mon[29968]: 0: [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] mon.a 2024-09-10T23:55:17.809 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:55:17 smithi012 ceph-mon[29968]: fsmap 2024-09-10T23:55:17.809 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:55:17 smithi012 ceph-mon[29968]: osdmap e1: 0 total, 0 up, 0 in 2024-09-10T23:55:17.809 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:55:17 smithi012 ceph-mon[29968]: mgrmap e1: no daemons active 2024-09-10T23:55:17.809 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:55:17 smithi012 ceph-mon[29968]: from='client.? 172.21.15.12:0/869165125' entity='client.admin' cmd={"prefix": "status"} : dispatch 2024-09-10T23:55:17.813 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout cluster: 2024-09-10T23:55:17.815 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout id: ff3ad74a-6fcf-11ef-bcea-c7b262605968 2024-09-10T23:55:17.815 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-09-10T23:55:17.815 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout 2024-09-10T23:55:17.816 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout services: 2024-09-10T23:55:17.816 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.617171s) 2024-09-10T23:55:17.816 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-09-10T23:55:17.816 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-09-10T23:55:17.816 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout 2024-09-10T23:55:17.816 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout data: 2024-09-10T23:55:17.816 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-09-10T23:55:17.816 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-09-10T23:55:17.816 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-09-10T23:55:17.816 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout pgs: 2024-09-10T23:55:17.816 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout 2024-09-10T23:55:17.816 INFO:teuthology.orchestra.run.smithi012.stdout:mon is available 2024-09-10T23:55:17.817 INFO:teuthology.orchestra.run.smithi012.stdout:Assimilating anything we can from ceph.conf... 2024-09-10T23:55:18.686 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:55:18 smithi012 ceph-mon[29968]: from='client.? 172.21.15.12:0/3742423575' entity='client.admin' cmd={"prefix": "config assimilate-conf"} : dispatch 2024-09-10T23:55:18.687 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:55:18 smithi012 ceph-mon[29968]: from='client.? 172.21.15.12:0/3742423575' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-09-10T23:55:18.955 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout 2024-09-10T23:55:18.955 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout [global] 2024-09-10T23:55:18.955 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout fsid = ff3ad74a-6fcf-11ef-bcea-c7b262605968 2024-09-10T23:55:18.955 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout mon_cluster_log_file_level = debug 2024-09-10T23:55:18.955 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.12:3300,v1:172.21.15.12:6789] 2024-09-10T23:55:18.955 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-09-10T23:55:18.955 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-09-10T23:55:18.955 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-09-10T23:55:18.955 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-09-10T23:55:18.955 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout 2024-09-10T23:55:18.955 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout [mgr] 2024-09-10T23:55:18.956 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-09-10T23:55:18.956 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout 2024-09-10T23:55:18.956 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout [osd] 2024-09-10T23:55:18.956 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-09-10T23:55:18.956 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-09-10T23:55:18.956 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-09-10T23:55:18.956 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-09-10T23:55:18.956 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-09-10T23:55:18.956 INFO:teuthology.orchestra.run.smithi012.stdout:Generating new minimal ceph.conf... 2024-09-10T23:55:19.813 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:55:19 smithi012 ceph-mon[29968]: from='client.? 172.21.15.12:0/2642039808' entity='client.admin' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T23:55:20.093 INFO:teuthology.orchestra.run.smithi012.stdout:Restarting the monitor... 2024-09-10T23:55:20.563 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:55:20 smithi012 systemd[1]: Stopping Ceph mon.a for ff3ad74a-6fcf-11ef-bcea-c7b262605968... 2024-09-10T23:55:20.976 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:55:20 smithi012 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-mon-a[29964]: 2024-09-10T23:55:20.600+0000 7fc850a00640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T23:55:20.976 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:55:20 smithi012 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-mon-a[29964]: 2024-09-10T23:55:20.600+0000 7fc850a00640 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-09-10T23:55:20.976 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:55:20 smithi012 podman[30208]: 2024-09-10 23:55:20.725187381 +0000 UTC m=+0.372122026 container died 1100b48335d9b35274008fb8b92cef127dba9f8f77a5de12f3649555d34f7b6b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30, name=ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-mon-a, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS) 2024-09-10T23:55:21.241 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:55:21 smithi012 podman[30208]: 2024-09-10 23:55:21.081341494 +0000 UTC m=+0.728276134 container cleanup 1100b48335d9b35274008fb8b92cef127dba9f8f77a5de12f3649555d34f7b6b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30, name=ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-mon-a, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , ceph=True) 2024-09-10T23:55:21.241 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:55:21 smithi012 bash[30208]: ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-mon-a 2024-09-10T23:55:21.491 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:55:21 smithi012 podman[30221]: 2024-09-10 23:55:21.24044911 +0000 UTC m=+0.510128637 container remove 1100b48335d9b35274008fb8b92cef127dba9f8f77a5de12f3649555d34f7b6b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30, name=ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-mon-a, GIT_BRANCH=HEAD, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20240731) 2024-09-10T23:55:21.800 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:55:21 smithi012 systemd[1]: ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968@mon.a.service: Deactivated successfully. 2024-09-10T23:55:21.800 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:55:21 smithi012 systemd[1]: Stopped Ceph mon.a for ff3ad74a-6fcf-11ef-bcea-c7b262605968. 2024-09-10T23:55:21.800 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:55:21 smithi012 systemd[1]: ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968@mon.a.service: Consumed 1.213s CPU time. 2024-09-10T23:55:21.801 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:55:21 smithi012 systemd[1]: Starting Ceph mon.a for ff3ad74a-6fcf-11ef-bcea-c7b262605968... 2024-09-10T23:55:22.053 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:55:21 smithi012 podman[30314]: 2024-09-10 23:55:21.799813433 +0000 UTC m=+0.116050489 container create 08abb0cfbc0b538576931bc46d941dcfd693e99b9d1541a83e889de3fdae4390 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30, name=ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-mon-a, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True) 2024-09-10T23:55:22.053 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:55:21 smithi012 podman[30314]: 2024-09-10 23:55:21.711585455 +0000 UTC m=+0.027822512 image pull 589dd18248e494ba411dc838aa4ae0e5e569213f08fcd7a2cf22cd3ef7e3c480 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 2024-09-10T23:55:22.053 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:55:22 smithi012 podman[30314]: 2024-09-10 23:55:22.010780142 +0000 UTC m=+0.327017199 container init 08abb0cfbc0b538576931bc46d941dcfd693e99b9d1541a83e889de3fdae4390 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30, name=ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-mon-a, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, ceph=True, GIT_BRANCH=HEAD, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.schema-version=1.0) 2024-09-10T23:55:22.053 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:55:22 smithi012 podman[30314]: 2024-09-10 23:55:22.017521962 +0000 UTC m=+0.333759020 container start 08abb0cfbc0b538576931bc46d941dcfd693e99b9d1541a83e889de3fdae4390 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30, name=ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-mon-a, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-09-10T23:55:22.087 INFO:teuthology.orchestra.run.smithi012.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-09-10T23:55:22.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:55:22 smithi012 ceph-mon[30333]: set uid:gid to 167:167 (ceph:ceph) 2024-09-10T23:55:22.314 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:55:22 smithi012 ceph-mon[30333]: ceph version 19.3.0-4854-g8f219c47 (8f219c476cb4f26fc5719c661c0323719a45cb30) squid (dev), process ceph-mon, pid 2 2024-09-10T23:55:22.314 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:55:22 smithi012 ceph-mon[30333]: pidfile_write: ignore empty --pid-file 2024-09-10T23:55:22.314 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:55:22 smithi012 ceph-mon[30333]: load: jerasure load: lrc 2024-09-10T23:55:22.314 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:55:22 smithi012 ceph-mon[30333]: rocksdb: RocksDB version: 7.9.2 2024-09-10T23:55:22.314 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:55:22 smithi012 ceph-mon[30333]: rocksdb: Git sha 0 2024-09-10T23:55:22.315 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:55:22 smithi012 ceph-mon[30333]: rocksdb: Compile date 2024-09-10 08:02:51 2024-09-10T23:55:22.315 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:55:22 smithi012 ceph-mon[30333]: rocksdb: DB SUMMARY 2024-09-10T23:55:22.315 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:55:22 smithi012 ceph-mon[30333]: rocksdb: DB Session ID: AX7XWL7QK8NHRIIVE2VS 2024-09-10T23:55:22.315 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:55:22 smithi012 ceph-mon[30333]: rocksdb: CURRENT file: CURRENT 2024-09-10T23:55:22.315 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:55:22 smithi012 ceph-mon[30333]: rocksdb: IDENTITY file: IDENTITY 2024-09-10T23:55:22.315 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:55:22 smithi012 ceph-mon[30333]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2024-09-10T23:55:22.316 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:55:22 smithi012 ceph-mon[30333]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2024-09-10T23:55:22.316 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:55:22 smithi012 ceph-mon[30333]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 94588 ; 2024-09-10T23:55:22.316 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:55:22 smithi012 ceph-mon[30333]: rocksdb: Options.error_if_exists: 0 2024-09-10T23:55:22.316 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:55:22 smithi012 ceph-mon[30333]: rocksdb: Options.create_if_missing: 0 2024-09-10T23:55:22.316 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:55:22 smithi012 ceph-mon[30333]: rocksdb: Options.paranoid_checks: 1 2024-09-10T23:55:22.316 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:55:22 smithi012 ceph-mon[30333]: rocksdb: Options.flush_verify_memtable_count: 1 2024-09-10T23:55:22.317 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:55:22 smithi012 ceph-mon[30333]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-09-10T23:55:22.317 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:55:22 smithi012 ceph-mon[30333]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-09-10T23:55:22.317 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:55:22 smithi012 ceph-mon[30333]: rocksdb: Options.env: 0x5602683ba180 2024-09-10T23:55:22.317 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:55:22 smithi012 ceph-mon[30333]: rocksdb: Options.fs: PosixFileSystem 2024-09-10T23:55:22.317 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:55:22 smithi012 ceph-mon[30333]: rocksdb: Options.info_log: 0x56028c6996e0 2024-09-10T23:55:22.318 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:55:22 smithi012 ceph-mon[30333]: rocksdb: Options.max_file_opening_threads: 16 2024-09-10T23:55:22.318 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:55:22 smithi012 ceph-mon[30333]: rocksdb: Options.statistics: (nil) 2024-09-10T23:55:22.318 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:55:22 smithi012 ceph-mon[30333]: rocksdb: Options.use_fsync: 0 2024-09-10T23:55:22.318 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:55:22 smithi012 ceph-mon[30333]: rocksdb: Options.max_log_file_size: 0 2024-09-10T23:55:22.318 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:55:22 smithi012 ceph-mon[30333]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-09-10T23:55:22.319 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:55:22 smithi012 ceph-mon[30333]: rocksdb: Options.log_file_time_to_roll: 0 2024-09-10T23:55:22.319 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:55:22 smithi012 ceph-mon[30333]: rocksdb: Options.keep_log_file_num: 1000 2024-09-10T23:55:22.319 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:55:22 smithi012 ceph-mon[30333]: rocksdb: Options.recycle_log_file_num: 0 2024-09-10T23:55:22.319 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:55:22 smithi012 ceph-mon[30333]: rocksdb: Options.allow_fallocate: 1 2024-09-10T23:55:22.319 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:55:22 smithi012 ceph-mon[30333]: rocksdb: Options.allow_mmap_reads: 0 2024-09-10T23:55:22.320 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:55:22 smithi012 ceph-mon[30333]: rocksdb: Options.allow_mmap_writes: 0 2024-09-10T23:55:22.320 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:55:22 smithi012 ceph-mon[30333]: rocksdb: Options.use_direct_reads: 0 2024-09-10T23:55:22.320 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:55:22 smithi012 ceph-mon[30333]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-09-10T23:55:22.320 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:55:22 smithi012 ceph-mon[30333]: rocksdb: Options.create_missing_column_families: 0 2024-09-10T23:55:22.320 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:55:22 smithi012 ceph-mon[30333]: rocksdb: Options.db_log_dir: 2024-09-10T23:55:22.320 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:55:22 smithi012 ceph-mon[30333]: rocksdb: Options.wal_dir: 2024-09-10T23:55:22.321 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:55:22 smithi012 ceph-mon[30333]: rocksdb: Options.table_cache_numshardbits: 6 2024-09-10T23:55:22.321 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:55:22 smithi012 ceph-mon[30333]: rocksdb: Options.WAL_ttl_seconds: 0 2024-09-10T23:55:22.321 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:55:22 smithi012 ceph-mon[30333]: rocksdb: Options.WAL_size_limit_MB: 0 2024-09-10T23:55:22.321 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:55:22 smithi012 ceph-mon[30333]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-09-10T23:55:22.321 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:55:22 smithi012 ceph-mon[30333]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-09-10T23:55:22.321 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:55:22 smithi012 ceph-mon[30333]: rocksdb: Options.is_fd_close_on_exec: 1 2024-09-10T23:55:22.321 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:55:22 smithi012 ceph-mon[30333]: rocksdb: Options.advise_random_on_open: 1 2024-09-10T23:55:22.321 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:55:22 smithi012 ceph-mon[30333]: rocksdb: Options.db_write_buffer_size: 0 2024-09-10T23:55:22.321 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:55:22 smithi012 ceph-mon[30333]: rocksdb: Options.write_buffer_manager: 0x56028c69db80 2024-09-10T23:55:22.321 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:55:22 smithi012 ceph-mon[30333]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-09-10T23:55:22.321 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:55:22 smithi012 ceph-mon[30333]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-09-10T23:55:22.321 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:55:22 smithi012 ceph-mon[30333]: rocksdb: Options.use_adaptive_mutex: 0 2024-09-10T23:55:22.321 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:55:22 smithi012 ceph-mon[30333]: rocksdb: Options.rate_limiter: (nil) 2024-09-10T23:55:22.322 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:55:22 smithi012 ceph-mon[30333]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-09-10T23:55:22.322 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:55:22 smithi012 ceph-mon[30333]: rocksdb: Options.wal_recovery_mode: 2 2024-09-10T23:55:22.322 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:55:22 smithi012 ceph-mon[30333]: rocksdb: Options.enable_thread_tracking: 0 2024-09-10T23:55:22.322 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:55:22 smithi012 ceph-mon[30333]: rocksdb: Options.enable_pipelined_write: 0 2024-09-10T23:55:22.322 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:55:22 smithi012 ceph-mon[30333]: rocksdb: Options.unordered_write: 0 2024-09-10T23:55:22.322 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:55:22 smithi012 ceph-mon[30333]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-09-10T23:55:22.322 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:55:22 smithi012 ceph-mon[30333]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-09-10T23:55:22.322 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:55:22 smithi012 ceph-mon[30333]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-09-10T23:55:22.322 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:55:22 smithi012 ceph-mon[30333]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-09-10T23:55:22.322 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:55:22 smithi012 ceph-mon[30333]: rocksdb: Options.row_cache: None 2024-09-10T23:55:22.322 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:55:22 smithi012 ceph-mon[30333]: rocksdb: Options.wal_filter: None 2024-09-10T23:55:22.322 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:55:22 smithi012 ceph-mon[30333]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-09-10T23:55:22.322 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:55:22 smithi012 ceph-mon[30333]: rocksdb: Options.allow_ingest_behind: 0 2024-09-10T23:55:22.323 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:55:22 smithi012 ceph-mon[30333]: rocksdb: Options.two_write_queues: 0 2024-09-10T23:55:22.323 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:55:22 smithi012 ceph-mon[30333]: rocksdb: Options.manual_wal_flush: 0 2024-09-10T23:55:22.323 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:55:22 smithi012 ceph-mon[30333]: rocksdb: Options.wal_compression: 0 2024-09-10T23:55:22.323 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:55:22 smithi012 ceph-mon[30333]: rocksdb: Options.atomic_flush: 0 2024-09-10T23:55:22.323 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:55:22 smithi012 ceph-mon[30333]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-09-10T23:55:22.323 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:55:22 smithi012 ceph-mon[30333]: rocksdb: Options.persist_stats_to_disk: 0 2024-09-10T23:55:22.323 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:55:22 smithi012 ceph-mon[30333]: rocksdb: Options.write_dbid_to_manifest: 0 2024-09-10T23:55:22.323 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:55:22 smithi012 ceph-mon[30333]: rocksdb: Options.log_readahead_size: 0 2024-09-10T23:55:22.323 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:55:22 smithi012 ceph-mon[30333]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-09-10T23:55:22.323 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:55:22 smithi012 ceph-mon[30333]: rocksdb: Options.best_efforts_recovery: 0 2024-09-10T23:55:22.323 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:55:22 smithi012 ceph-mon[30333]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-09-10T23:55:22.323 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:55:22 smithi012 ceph-mon[30333]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-09-10T23:55:22.323 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:55:22 smithi012 ceph-mon[30333]: rocksdb: Options.allow_data_in_errors: 0 2024-09-10T23:55:22.324 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:55:22 smithi012 ceph-mon[30333]: rocksdb: Options.db_host_id: __hostname__ 2024-09-10T23:55:22.324 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:55:22 smithi012 ceph-mon[30333]: rocksdb: Options.enforce_single_del_contracts: true 2024-09-10T23:55:22.325 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:55:22 smithi012 ceph-mon[30333]: rocksdb: Options.max_background_jobs: 2 2024-09-10T23:55:22.325 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:55:22 smithi012 ceph-mon[30333]: rocksdb: Options.max_background_compactions: -1 2024-09-10T23:55:22.325 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:55:22 smithi012 ceph-mon[30333]: rocksdb: Options.max_subcompactions: 1 2024-09-10T23:55:22.325 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:55:22 smithi012 ceph-mon[30333]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-09-10T23:55:22.325 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:55:22 smithi012 ceph-mon[30333]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-09-10T23:55:22.325 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:55:22 smithi012 ceph-mon[30333]: rocksdb: Options.delayed_write_rate : 16777216 2024-09-10T23:55:22.325 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:55:22 smithi012 ceph-mon[30333]: rocksdb: Options.max_total_wal_size: 0 2024-09-10T23:55:22.326 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:55:22 smithi012 ceph-mon[30333]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-09-10T23:55:22.326 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:55:22 smithi012 ceph-mon[30333]: rocksdb: Options.stats_dump_period_sec: 600 2024-09-10T23:55:22.326 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:55:22 smithi012 ceph-mon[30333]: rocksdb: Options.stats_persist_period_sec: 600 2024-09-10T23:55:22.326 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:55:22 smithi012 ceph-mon[30333]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-09-10T23:55:22.326 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:55:22 smithi012 ceph-mon[30333]: rocksdb: Options.max_open_files: -1 2024-09-10T23:55:22.326 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:55:22 smithi012 ceph-mon[30333]: rocksdb: Options.bytes_per_sync: 0 2024-09-10T23:55:22.326 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:55:22 smithi012 ceph-mon[30333]: rocksdb: Options.wal_bytes_per_sync: 0 2024-09-10T23:55:22.326 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:55:22 smithi012 ceph-mon[30333]: rocksdb: Options.strict_bytes_per_sync: 0 2024-09-10T23:55:22.326 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:55:22 smithi012 ceph-mon[30333]: rocksdb: Options.compaction_readahead_size: 0 2024-09-10T23:55:22.326 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:55:22 smithi012 ceph-mon[30333]: rocksdb: Options.max_background_flushes: -1 2024-09-10T23:55:22.326 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:55:22 smithi012 ceph-mon[30333]: rocksdb: Compression algorithms supported: 2024-09-10T23:55:22.326 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:55:22 smithi012 ceph-mon[30333]: rocksdb: kZSTD supported: 0 2024-09-10T23:55:22.327 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:55:22 smithi012 ceph-mon[30333]: rocksdb: kXpressCompression supported: 0 2024-09-10T23:55:22.327 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:55:22 smithi012 ceph-mon[30333]: rocksdb: kBZip2Compression supported: 0 2024-09-10T23:55:22.327 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:55:22 smithi012 ceph-mon[30333]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-09-10T23:55:22.327 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:55:22 smithi012 ceph-mon[30333]: rocksdb: kLZ4Compression supported: 1 2024-09-10T23:55:22.327 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:55:22 smithi012 ceph-mon[30333]: rocksdb: kZlibCompression supported: 1 2024-09-10T23:55:22.327 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:55:22 smithi012 ceph-mon[30333]: rocksdb: kLZ4HCCompression supported: 1 2024-09-10T23:55:22.327 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:55:22 smithi012 ceph-mon[30333]: rocksdb: kSnappyCompression supported: 1 2024-09-10T23:55:22.327 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:55:22 smithi012 ceph-mon[30333]: rocksdb: Fast CRC32 supported: Supported on x86 2024-09-10T23:55:22.327 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:55:22 smithi012 ceph-mon[30333]: rocksdb: DMutex implementation: pthread_mutex_t 2024-09-10T23:55:22.327 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:55:22 smithi012 ceph-mon[30333]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-09-10T23:55:22.327 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:55:22 smithi012 ceph-mon[30333]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-09-10T23:55:22.327 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:55:22 smithi012 ceph-mon[30333]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-09-10T23:55:22.327 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:55:22 smithi012 ceph-mon[30333]: rocksdb: Options.merge_operator: 2024-09-10T23:55:22.328 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:55:22 smithi012 ceph-mon[30333]: rocksdb: Options.compaction_filter: None 2024-09-10T23:55:22.328 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:55:22 smithi012 ceph-mon[30333]: rocksdb: Options.compaction_filter_factory: None 2024-09-10T23:55:22.328 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:55:22 smithi012 ceph-mon[30333]: rocksdb: Options.sst_partitioner_factory: None 2024-09-10T23:55:22.328 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:55:22 smithi012 ceph-mon[30333]: rocksdb: Options.memtable_factory: SkipListFactory 2024-09-10T23:55:22.328 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:55:22 smithi012 ceph-mon[30333]: rocksdb: Options.table_factory: BlockBasedTable 2024-09-10T23:55:22.328 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:55:22 smithi012 ceph-mon[30333]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x56028c699300) 2024-09-10T23:55:22.328 INFO:journalctl@ceph.mon.a.smithi012.stdout: cache_index_and_filter_blocks: 1 2024-09-10T23:55:22.328 INFO:journalctl@ceph.mon.a.smithi012.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-09-10T23:55:22.328 INFO:journalctl@ceph.mon.a.smithi012.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-09-10T23:55:22.328 INFO:journalctl@ceph.mon.a.smithi012.stdout: pin_top_level_index_and_filter: 1 2024-09-10T23:55:22.328 INFO:journalctl@ceph.mon.a.smithi012.stdout: index_type: 0 2024-09-10T23:55:22.328 INFO:journalctl@ceph.mon.a.smithi012.stdout: data_block_index_type: 0 2024-09-10T23:55:22.328 INFO:journalctl@ceph.mon.a.smithi012.stdout: index_shortening: 1 2024-09-10T23:55:22.328 INFO:journalctl@ceph.mon.a.smithi012.stdout: data_block_hash_table_util_ratio: 0.750000 2024-09-10T23:55:22.329 INFO:journalctl@ceph.mon.a.smithi012.stdout: checksum: 4 2024-09-10T23:55:22.329 INFO:journalctl@ceph.mon.a.smithi012.stdout: no_block_cache: 0 2024-09-10T23:55:22.329 INFO:journalctl@ceph.mon.a.smithi012.stdout: block_cache: 0x56028c68d610 2024-09-10T23:55:22.329 INFO:journalctl@ceph.mon.a.smithi012.stdout: block_cache_name: BinnedLRUCache 2024-09-10T23:55:22.329 INFO:journalctl@ceph.mon.a.smithi012.stdout: block_cache_options: 2024-09-10T23:55:22.329 INFO:journalctl@ceph.mon.a.smithi012.stdout: capacity : 536870912 2024-09-10T23:55:22.329 INFO:journalctl@ceph.mon.a.smithi012.stdout: num_shard_bits : 4 2024-09-10T23:55:22.329 INFO:journalctl@ceph.mon.a.smithi012.stdout: strict_capacity_limit : 0 2024-09-10T23:55:22.329 INFO:journalctl@ceph.mon.a.smithi012.stdout: high_pri_pool_ratio: 0.000 2024-09-10T23:55:22.329 INFO:journalctl@ceph.mon.a.smithi012.stdout: block_cache_compressed: (nil) 2024-09-10T23:55:22.329 INFO:journalctl@ceph.mon.a.smithi012.stdout: persistent_cache: (nil) 2024-09-10T23:55:22.329 INFO:journalctl@ceph.mon.a.smithi012.stdout: block_size: 4096 2024-09-10T23:55:22.329 INFO:journalctl@ceph.mon.a.smithi012.stdout: block_size_deviation: 10 2024-09-10T23:55:22.330 INFO:journalctl@ceph.mon.a.smithi012.stdout: block_restart_interval: 16 2024-09-10T23:55:22.330 INFO:journalctl@ceph.mon.a.smithi012.stdout: index_block_restart_interval: 1 2024-09-10T23:55:22.330 INFO:journalctl@ceph.mon.a.smithi012.stdout: metadata_block_size: 4096 2024-09-10T23:55:22.330 INFO:journalctl@ceph.mon.a.smithi012.stdout: partition_filters: 0 2024-09-10T23:55:22.330 INFO:journalctl@ceph.mon.a.smithi012.stdout: use_delta_encoding: 1 2024-09-10T23:55:22.330 INFO:journalctl@ceph.mon.a.smithi012.stdout: filter_policy: bloomfilter 2024-09-10T23:55:22.330 INFO:journalctl@ceph.mon.a.smithi012.stdout: whole_key_filtering: 1 2024-09-10T23:55:22.330 INFO:journalctl@ceph.mon.a.smithi012.stdout: verify_compression: 0 2024-09-10T23:55:22.330 INFO:journalctl@ceph.mon.a.smithi012.stdout: read_amp_bytes_per_bit: 0 2024-09-10T23:55:22.330 INFO:journalctl@ceph.mon.a.smithi012.stdout: format_version: 5 2024-09-10T23:55:22.330 INFO:journalctl@ceph.mon.a.smithi012.stdout: enable_index_compression: 1 2024-09-10T23:55:22.330 INFO:journalctl@ceph.mon.a.smithi012.stdout: block_align: 0 2024-09-10T23:55:22.331 INFO:journalctl@ceph.mon.a.smithi012.stdout: max_auto_readahead_size: 262144 2024-09-10T23:55:22.331 INFO:journalctl@ceph.mon.a.smithi012.stdout: prepopulate_block_cache: 0 2024-09-10T23:55:22.331 INFO:journalctl@ceph.mon.a.smithi012.stdout: initial_auto_readahead_size: 8192 2024-09-10T23:55:22.331 INFO:journalctl@ceph.mon.a.smithi012.stdout: num_file_reads_for_auto_readahead: 2 2024-09-10T23:55:22.331 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:55:22 smithi012 ceph-mon[30333]: rocksdb: Options.write_buffer_size: 33554432 2024-09-10T23:55:22.331 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:55:22 smithi012 ceph-mon[30333]: rocksdb: Options.max_write_buffer_number: 2 2024-09-10T23:55:22.331 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:55:22 smithi012 ceph-mon[30333]: rocksdb: Options.compression: NoCompression 2024-09-10T23:55:22.331 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:55:22 smithi012 ceph-mon[30333]: rocksdb: Options.bottommost_compression: Disabled 2024-09-10T23:55:22.331 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:55:22 smithi012 ceph-mon[30333]: rocksdb: Options.prefix_extractor: nullptr 2024-09-10T23:55:22.331 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:55:22 smithi012 ceph-mon[30333]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-09-10T23:55:22.331 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:55:22 smithi012 ceph-mon[30333]: rocksdb: Options.num_levels: 7 2024-09-10T23:55:22.331 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:55:22 smithi012 ceph-mon[30333]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-09-10T23:55:22.331 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:55:22 smithi012 ceph-mon[30333]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-09-10T23:55:22.332 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:55:22 smithi012 ceph-mon[30333]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-09-10T23:55:22.332 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:55:22 smithi012 ceph-mon[30333]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-09-10T23:55:22.332 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:55:22 smithi012 ceph-mon[30333]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-09-10T23:55:22.332 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:55:22 smithi012 ceph-mon[30333]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-09-10T23:55:22.332 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:55:22 smithi012 ceph-mon[30333]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-09-10T23:55:22.332 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:55:22 smithi012 ceph-mon[30333]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-09-10T23:55:22.332 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:55:22 smithi012 ceph-mon[30333]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-09-10T23:55:22.332 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:55:22 smithi012 ceph-mon[30333]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-09-10T23:55:22.332 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:55:22 smithi012 ceph-mon[30333]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-09-10T23:55:22.332 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:55:22 smithi012 ceph-mon[30333]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-09-10T23:55:22.332 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:55:22 smithi012 ceph-mon[30333]: rocksdb: Options.compression_opts.window_bits: -14 2024-09-10T23:55:22.332 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:55:22 smithi012 ceph-mon[30333]: rocksdb: Options.compression_opts.level: 32767 2024-09-10T23:55:22.333 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:55:22 smithi012 ceph-mon[30333]: rocksdb: Options.compression_opts.strategy: 0 2024-09-10T23:55:22.333 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:55:22 smithi012 ceph-mon[30333]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-09-10T23:55:22.333 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:55:22 smithi012 ceph-mon[30333]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-09-10T23:55:22.333 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:55:22 smithi012 ceph-mon[30333]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-09-10T23:55:22.333 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:55:22 smithi012 ceph-mon[30333]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-09-10T23:55:22.333 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:55:22 smithi012 ceph-mon[30333]: rocksdb: Options.compression_opts.enabled: false 2024-09-10T23:55:22.333 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:55:22 smithi012 ceph-mon[30333]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-09-10T23:55:22.333 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:55:22 smithi012 ceph-mon[30333]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-09-10T23:55:22.333 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:55:22 smithi012 ceph-mon[30333]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-09-10T23:55:22.333 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:55:22 smithi012 ceph-mon[30333]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-09-10T23:55:22.333 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:55:22 smithi012 ceph-mon[30333]: rocksdb: Options.target_file_size_base: 67108864 2024-09-10T23:55:22.333 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:55:22 smithi012 ceph-mon[30333]: rocksdb: Options.target_file_size_multiplier: 1 2024-09-10T23:55:22.333 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:55:22 smithi012 ceph-mon[30333]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-09-10T23:55:22.334 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:55:22 smithi012 ceph-mon[30333]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-09-10T23:55:22.334 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:55:22 smithi012 ceph-mon[30333]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-09-10T23:55:22.334 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:55:22 smithi012 ceph-mon[30333]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-09-10T23:55:22.334 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:55:22 smithi012 ceph-mon[30333]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-09-10T23:55:22.334 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:55:22 smithi012 ceph-mon[30333]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-09-10T23:55:22.334 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:55:22 smithi012 ceph-mon[30333]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-09-10T23:55:22.334 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:55:22 smithi012 ceph-mon[30333]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-09-10T23:55:22.334 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:55:22 smithi012 ceph-mon[30333]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-09-10T23:55:22.334 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:55:22 smithi012 ceph-mon[30333]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-09-10T23:55:22.334 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:55:22 smithi012 ceph-mon[30333]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-09-10T23:55:22.334 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:55:22 smithi012 ceph-mon[30333]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-09-10T23:55:22.334 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:55:22 smithi012 ceph-mon[30333]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-09-10T23:55:22.334 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:55:22 smithi012 ceph-mon[30333]: rocksdb: Options.arena_block_size: 1048576 2024-09-10T23:55:22.335 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:55:22 smithi012 ceph-mon[30333]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-09-10T23:55:22.335 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:55:22 smithi012 ceph-mon[30333]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-09-10T23:55:22.335 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:55:22 smithi012 ceph-mon[30333]: rocksdb: Options.disable_auto_compactions: 0 2024-09-10T23:55:22.335 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:55:22 smithi012 ceph-mon[30333]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-09-10T23:55:22.335 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:55:22 smithi012 ceph-mon[30333]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-09-10T23:55:22.335 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:55:22 smithi012 ceph-mon[30333]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-09-10T23:55:22.335 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:55:22 smithi012 ceph-mon[30333]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-09-10T23:55:22.335 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:55:22 smithi012 ceph-mon[30333]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-09-10T23:55:22.335 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:55:22 smithi012 ceph-mon[30333]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-09-10T23:55:22.335 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:55:22 smithi012 ceph-mon[30333]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-09-10T23:55:22.335 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:55:22 smithi012 ceph-mon[30333]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-09-10T23:55:22.335 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:55:22 smithi012 ceph-mon[30333]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-09-10T23:55:22.335 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:55:22 smithi012 ceph-mon[30333]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-09-10T23:55:22.335 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:55:22 smithi012 ceph-mon[30333]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-09-10T23:55:22.336 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:55:22 smithi012 ceph-mon[30333]: rocksdb: Options.inplace_update_support: 0 2024-09-10T23:55:22.336 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:55:22 smithi012 ceph-mon[30333]: rocksdb: Options.inplace_update_num_locks: 10000 2024-09-10T23:55:22.336 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:55:22 smithi012 ceph-mon[30333]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-09-10T23:55:22.336 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:55:22 smithi012 ceph-mon[30333]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-09-10T23:55:22.336 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:55:22 smithi012 ceph-mon[30333]: rocksdb: Options.memtable_huge_page_size: 0 2024-09-10T23:55:22.336 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:55:22 smithi012 ceph-mon[30333]: rocksdb: Options.bloom_locality: 0 2024-09-10T23:55:22.336 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:55:22 smithi012 ceph-mon[30333]: rocksdb: Options.max_successive_merges: 0 2024-09-10T23:55:22.336 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:55:22 smithi012 ceph-mon[30333]: rocksdb: Options.optimize_filters_for_hits: 0 2024-09-10T23:55:22.336 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:55:22 smithi012 ceph-mon[30333]: rocksdb: Options.paranoid_file_checks: 0 2024-09-10T23:55:22.336 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:55:22 smithi012 ceph-mon[30333]: rocksdb: Options.force_consistency_checks: 1 2024-09-10T23:55:22.336 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:55:22 smithi012 ceph-mon[30333]: rocksdb: Options.report_bg_io_stats: 0 2024-09-10T23:55:22.336 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:55:22 smithi012 ceph-mon[30333]: rocksdb: Options.ttl: 2592000 2024-09-10T23:55:22.337 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:55:22 smithi012 ceph-mon[30333]: rocksdb: Options.periodic_compaction_seconds: 0 2024-09-10T23:55:22.337 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:55:22 smithi012 ceph-mon[30333]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-09-10T23:55:22.337 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:55:22 smithi012 ceph-mon[30333]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-09-10T23:55:22.337 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:55:22 smithi012 ceph-mon[30333]: rocksdb: Options.enable_blob_files: false 2024-09-10T23:55:22.337 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:55:22 smithi012 ceph-mon[30333]: rocksdb: Options.min_blob_size: 0 2024-09-10T23:55:22.337 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:55:22 smithi012 ceph-mon[30333]: rocksdb: Options.blob_file_size: 268435456 2024-09-10T23:55:22.337 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:55:22 smithi012 ceph-mon[30333]: rocksdb: Options.blob_compression_type: NoCompression 2024-09-10T23:55:22.337 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:55:22 smithi012 ceph-mon[30333]: rocksdb: Options.enable_blob_garbage_collection: false 2024-09-10T23:55:22.337 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:55:22 smithi012 ceph-mon[30333]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-09-10T23:55:22.337 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:55:22 smithi012 ceph-mon[30333]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-09-10T23:55:22.337 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:55:22 smithi012 ceph-mon[30333]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-09-10T23:55:22.337 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:55:22 smithi012 ceph-mon[30333]: rocksdb: Options.blob_file_starting_level: 0 2024-09-10T23:55:22.337 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:55:22 smithi012 ceph-mon[30333]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-09-10T23:55:22.337 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:55:22 smithi012 ceph-mon[30333]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2024-09-10T23:55:22.338 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:55:22 smithi012 ceph-mon[30333]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2024-09-10T23:55:22.338 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:55:22 smithi012 ceph-mon[30333]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: a67689b3-977a-443f-8747-1f3c02f593a7 2024-09-10T23:55:22.338 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:55:22 smithi012 ceph-mon[30333]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726012522062103, "job": 1, "event": "recovery_started", "wal_files": [9]} 2024-09-10T23:55:22.338 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:55:22 smithi012 ceph-mon[30333]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2024-09-10T23:55:22.338 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:55:22 smithi012 ceph-mon[30333]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726012522064650, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 90820, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 283, "table_properties": {"data_size": 88861, "index_size": 279, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 645, "raw_key_size": 11555, "raw_average_key_size": 47, "raw_value_size": 82330, "raw_average_value_size": 340, "num_data_blocks": 12, "num_entries": 242, "num_filter_entries": 242, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1726012522, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "a67689b3-977a-443f-8747-1f3c02f593a7", "db_session_id": "AX7XWL7QK8NHRIIVE2VS", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2024-09-10T23:55:22.338 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:55:22 smithi012 ceph-mon[30333]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726012522064922, "job": 1, "event": "recovery_finished"} 2024-09-10T23:55:22.339 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:55:22 smithi012 ceph-mon[30333]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2024-09-10T23:55:22.339 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:55:22 smithi012 ceph-mon[30333]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-09-10T23:55:22.339 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:55:22 smithi012 ceph-mon[30333]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x56028c6bee00 2024-09-10T23:55:22.340 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:55:22 smithi012 ceph-mon[30333]: rocksdb: DB pointer 0x56028c7be000 2024-09-10T23:55:22.340 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:55:22 smithi012 ceph-mon[30333]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-09-10T23:55:22.340 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:55:22 smithi012 ceph-mon[30333]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-09-10T23:55:22.340 INFO:journalctl@ceph.mon.a.smithi012.stdout: ** DB Stats ** 2024-09-10T23:55:22.340 INFO:journalctl@ceph.mon.a.smithi012.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-10T23:55:22.340 INFO:journalctl@ceph.mon.a.smithi012.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-09-10T23:55:22.340 INFO:journalctl@ceph.mon.a.smithi012.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-10T23:55:22.340 INFO:journalctl@ceph.mon.a.smithi012.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-10T23:55:22.340 INFO:journalctl@ceph.mon.a.smithi012.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-09-10T23:55:22.340 INFO:journalctl@ceph.mon.a.smithi012.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-10T23:55:22.340 INFO:journalctl@ceph.mon.a.smithi012.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-10T23:55:22.340 INFO:journalctl@ceph.mon.a.smithi012.stdout: 2024-09-10T23:55:22.340 INFO:journalctl@ceph.mon.a.smithi012.stdout: ** Compaction Stats [default] ** 2024-09-10T23:55:22.341 INFO:journalctl@ceph.mon.a.smithi012.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-10T23:55:22.341 INFO:journalctl@ceph.mon.a.smithi012.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-09-10T23:55:22.341 INFO:journalctl@ceph.mon.a.smithi012.stdout: L0 2/0 90.55 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 54.1 0.00 0.00 1 0.002 0 0 0.0 0.0 2024-09-10T23:55:22.341 INFO:journalctl@ceph.mon.a.smithi012.stdout: Sum 2/0 90.55 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 54.1 0.00 0.00 1 0.002 0 0 0.0 0.0 2024-09-10T23:55:22.341 INFO:journalctl@ceph.mon.a.smithi012.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 54.1 0.00 0.00 1 0.002 0 0 0.0 0.0 2024-09-10T23:55:22.341 INFO:journalctl@ceph.mon.a.smithi012.stdout: 2024-09-10T23:55:22.341 INFO:journalctl@ceph.mon.a.smithi012.stdout: ** Compaction Stats [default] ** 2024-09-10T23:55:22.341 INFO:journalctl@ceph.mon.a.smithi012.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-10T23:55:22.341 INFO:journalctl@ceph.mon.a.smithi012.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-10T23:55:22.341 INFO:journalctl@ceph.mon.a.smithi012.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 54.1 0.00 0.00 1 0.002 0 0 0.0 0.0 2024-09-10T23:55:22.341 INFO:journalctl@ceph.mon.a.smithi012.stdout: 2024-09-10T23:55:22.341 INFO:journalctl@ceph.mon.a.smithi012.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-09-10T23:55:22.341 INFO:journalctl@ceph.mon.a.smithi012.stdout: 2024-09-10T23:55:22.342 INFO:journalctl@ceph.mon.a.smithi012.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-10T23:55:22.342 INFO:journalctl@ceph.mon.a.smithi012.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-09-10T23:55:22.342 INFO:journalctl@ceph.mon.a.smithi012.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-09-10T23:55:22.342 INFO:journalctl@ceph.mon.a.smithi012.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-09-10T23:55:22.342 INFO:journalctl@ceph.mon.a.smithi012.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-09-10T23:55:22.342 INFO:journalctl@ceph.mon.a.smithi012.stdout: AddFile(Keys): cumulative 0, interval 0 2024-09-10T23:55:22.342 INFO:journalctl@ceph.mon.a.smithi012.stdout: Cumulative compaction: 0.00 GB write, 9.83 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-10T23:55:22.342 INFO:journalctl@ceph.mon.a.smithi012.stdout: Interval compaction: 0.00 GB write, 9.83 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-10T23:55:22.342 INFO:journalctl@ceph.mon.a.smithi012.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-09-10T23:55:22.342 INFO:journalctl@ceph.mon.a.smithi012.stdout: Block cache BinnedLRUCache@0x56028c68d610#2 capacity: 512.00 MB usage: 1.30 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 9e-06 secs_since: 0 2024-09-10T23:55:22.342 INFO:journalctl@ceph.mon.a.smithi012.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,0.83 KB,0.000157952%) IndexBlock(2,0.47 KB,8.9407e-05%) Misc(1,0.00 KB,0%) 2024-09-10T23:55:22.342 INFO:journalctl@ceph.mon.a.smithi012.stdout: 2024-09-10T23:55:22.342 INFO:journalctl@ceph.mon.a.smithi012.stdout: ** File Read Latency Histogram By Level [default] ** 2024-09-10T23:55:22.343 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:55:22 smithi012 ceph-mon[30333]: starting mon.a rank 0 at public addrs [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] at bind addrs [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid ff3ad74a-6fcf-11ef-bcea-c7b262605968 2024-09-10T23:55:22.343 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:55:22 smithi012 ceph-mon[30333]: mon.a@-1(???) e1 preinit fsid ff3ad74a-6fcf-11ef-bcea-c7b262605968 2024-09-10T23:55:22.343 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:55:22 smithi012 ceph-mon[30333]: mon.a@-1(???).mds e1 new map 2024-09-10T23:55:22.343 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:55:22 smithi012 ceph-mon[30333]: mon.a@-1(???).mds e1 print_map 2024-09-10T23:55:22.343 INFO:journalctl@ceph.mon.a.smithi012.stdout: e1 2024-09-10T23:55:22.343 INFO:journalctl@ceph.mon.a.smithi012.stdout: btime 2024-09-10T23:55:16:317386+0000 2024-09-10T23:55:22.343 INFO:journalctl@ceph.mon.a.smithi012.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-09-10T23:55:22.343 INFO:journalctl@ceph.mon.a.smithi012.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-10T23:55:22.343 INFO:journalctl@ceph.mon.a.smithi012.stdout: legacy client fscid: -1 2024-09-10T23:55:22.343 INFO:journalctl@ceph.mon.a.smithi012.stdout: 2024-09-10T23:55:22.344 INFO:journalctl@ceph.mon.a.smithi012.stdout: No filesystems configured 2024-09-10T23:55:22.344 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:55:22 smithi012 ceph-mon[30333]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-09-10T23:55:22.344 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:55:22 smithi012 ceph-mon[30333]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-09-10T23:55:22.344 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:55:22 smithi012 ceph-mon[30333]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-09-10T23:55:22.344 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:55:22 smithi012 ceph-mon[30333]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-09-10T23:55:22.344 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:55:22 smithi012 ceph-mon[30333]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-09-10T23:55:22.344 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:55:22 smithi012 bash[30314]: 08abb0cfbc0b538576931bc46d941dcfd693e99b9d1541a83e889de3fdae4390 2024-09-10T23:55:22.344 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:55:22 smithi012 systemd[1]: Started Ceph mon.a for ff3ad74a-6fcf-11ef-bcea-c7b262605968. 2024-09-10T23:55:22.344 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:55:22 smithi012 ceph-mon[30333]: mon.a is new leader, mons a in quorum (ranks 0) 2024-09-10T23:55:22.345 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:55:22 smithi012 ceph-mon[30333]: monmap epoch 1 2024-09-10T23:55:22.345 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:55:22 smithi012 ceph-mon[30333]: fsid ff3ad74a-6fcf-11ef-bcea-c7b262605968 2024-09-10T23:55:22.345 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:55:22 smithi012 ceph-mon[30333]: last_changed 2024-09-10T23:55:12.644680+0000 2024-09-10T23:55:22.345 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:55:22 smithi012 ceph-mon[30333]: created 2024-09-10T23:55:12.644680+0000 2024-09-10T23:55:22.345 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:55:22 smithi012 ceph-mon[30333]: min_mon_release 19 (squid) 2024-09-10T23:55:22.345 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:55:22 smithi012 ceph-mon[30333]: election_strategy: 1 2024-09-10T23:55:22.345 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:55:22 smithi012 ceph-mon[30333]: 0: [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] mon.a 2024-09-10T23:55:22.345 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:55:22 smithi012 ceph-mon[30333]: fsmap 2024-09-10T23:55:22.345 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:55:22 smithi012 ceph-mon[30333]: osdmap e1: 0 total, 0 up, 0 in 2024-09-10T23:55:22.345 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:55:22 smithi012 ceph-mon[30333]: mgrmap e1: no daemons active 2024-09-10T23:55:28.563 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:55:28 smithi012 ceph-mon[30333]: from='client.? 172.21.15.12:0/1061403619' entity='client.admin' 2024-09-10T23:55:32.153 INFO:teuthology.orchestra.run.smithi012.stdout:Wrote config to /etc/ceph/ceph.conf 2024-09-10T23:55:32.176 INFO:teuthology.orchestra.run.smithi012.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-09-10T23:55:32.176 INFO:teuthology.orchestra.run.smithi012.stdout:Creating mgr... 2024-09-10T23:55:32.177 INFO:teuthology.orchestra.run.smithi012.stdout:Verifying port 0.0.0.0:9283 ... 2024-09-10T23:55:32.177 INFO:teuthology.orchestra.run.smithi012.stdout:Verifying port 0.0.0.0:8765 ... 2024-09-10T23:55:32.529 INFO:teuthology.orchestra.run.smithi012.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968@mgr.x 2024-09-10T23:55:32.529 INFO:teuthology.orchestra.run.smithi012.stdout:systemctl: stderr Failed to reset failed state of unit ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968@mgr.x.service: Unit ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968@mgr.x.service not loaded. 2024-09-10T23:55:32.761 INFO:teuthology.orchestra.run.smithi012.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968.target.wants/ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968@mgr.x.service → /etc/systemd/system/ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968@.service. 2024-09-10T23:55:32.795 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Sep 10 23:55:32 smithi012 systemd[1]: Starting Ceph mgr.x for ff3ad74a-6fcf-11ef-bcea-c7b262605968... 2024-09-10T23:55:33.381 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Sep 10 23:55:33 smithi012 podman[30545]: 2024-09-10 23:55:33.006838096 +0000 UTC m=+0.027441506 image pull 589dd18248e494ba411dc838aa4ae0e5e569213f08fcd7a2cf22cd3ef7e3c480 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 2024-09-10T23:55:33.382 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Sep 10 23:55:33 smithi012 podman[30545]: 2024-09-10 23:55:33.102756299 +0000 UTC m=+0.123359708 container create a392e948a60bd82b4a684df9916521a914799a568af52c75b7f200fd373df4f5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30, name=ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-mgr-x, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_BRANCH=HEAD, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-10T23:55:33.382 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Sep 10 23:55:33 smithi012 podman[30545]: 2024-09-10 23:55:33.297113606 +0000 UTC m=+0.317717022 container init a392e948a60bd82b4a684df9916521a914799a568af52c75b7f200fd373df4f5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30, name=ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-mgr-x, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , ceph=True, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, GIT_CLEAN=True) 2024-09-10T23:55:33.382 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Sep 10 23:55:33 smithi012 podman[30545]: 2024-09-10 23:55:33.303485467 +0000 UTC m=+0.324088878 container start a392e948a60bd82b4a684df9916521a914799a568af52c75b7f200fd373df4f5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30, name=ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-mgr-x, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , ceph=True, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True) 2024-09-10T23:55:33.382 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Sep 10 23:55:33 smithi012 bash[30545]: a392e948a60bd82b4a684df9916521a914799a568af52c75b7f200fd373df4f5 2024-09-10T23:55:33.408 INFO:teuthology.orchestra.run.smithi012.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-10T23:55:33.409 INFO:teuthology.orchestra.run.smithi012.stdout:systemctl: stdout disabled 2024-09-10T23:55:33.422 INFO:teuthology.orchestra.run.smithi012.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-10T23:55:33.423 INFO:teuthology.orchestra.run.smithi012.stdout:systemctl: stdout inactive 2024-09-10T23:55:33.423 INFO:teuthology.orchestra.run.smithi012.stdout:firewalld.service is not enabled 2024-09-10T23:55:33.423 INFO:teuthology.orchestra.run.smithi012.stdout:Not possible to enable service . firewalld.service is not available 2024-09-10T23:55:33.440 INFO:teuthology.orchestra.run.smithi012.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-10T23:55:33.440 INFO:teuthology.orchestra.run.smithi012.stdout:systemctl: stdout disabled 2024-09-10T23:55:33.454 INFO:teuthology.orchestra.run.smithi012.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-10T23:55:33.454 INFO:teuthology.orchestra.run.smithi012.stdout:systemctl: stdout inactive 2024-09-10T23:55:33.454 INFO:teuthology.orchestra.run.smithi012.stdout:firewalld.service is not enabled 2024-09-10T23:55:33.454 INFO:teuthology.orchestra.run.smithi012.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2024-09-10T23:55:33.454 INFO:teuthology.orchestra.run.smithi012.stdout:Waiting for mgr to start... 2024-09-10T23:55:33.454 INFO:teuthology.orchestra.run.smithi012.stdout:Waiting for mgr... 2024-09-10T23:55:33.665 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Sep 10 23:55:33 smithi012 systemd[1]: Started Ceph mgr.x for ff3ad74a-6fcf-11ef-bcea-c7b262605968. 2024-09-10T23:55:34.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:55:34 smithi012 ceph-mon[30333]: from='client.? 172.21.15.12:0/2200345876' entity='client.admin' cmd={"prefix": "status", "format": "json-pretty"} : dispatch 2024-09-10T23:55:34.718 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout 2024-09-10T23:55:34.718 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout { 2024-09-10T23:55:34.719 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "fsid": "ff3ad74a-6fcf-11ef-bcea-c7b262605968", 2024-09-10T23:55:34.719 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "health": { 2024-09-10T23:55:34.719 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-09-10T23:55:34.719 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-09-10T23:55:34.719 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-09-10T23:55:34.719 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout }, 2024-09-10T23:55:34.719 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-09-10T23:55:34.719 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-09-10T23:55:34.719 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout 0 2024-09-10T23:55:34.719 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout ], 2024-09-10T23:55:34.719 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-09-10T23:55:34.720 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "a" 2024-09-10T23:55:34.720 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout ], 2024-09-10T23:55:34.720 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "quorum_age": 12, 2024-09-10T23:55:34.720 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "monmap": { 2024-09-10T23:55:34.720 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-10T23:55:34.720 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-09-10T23:55:34.720 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-09-10T23:55:34.720 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout }, 2024-09-10T23:55:34.721 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-09-10T23:55:34.721 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-10T23:55:34.721 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-09-10T23:55:34.721 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-09-10T23:55:34.721 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-09-10T23:55:34.721 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-09-10T23:55:34.721 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-09-10T23:55:34.721 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-09-10T23:55:34.721 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout }, 2024-09-10T23:55:34.721 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-09-10T23:55:34.721 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-09-10T23:55:34.721 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-09-10T23:55:34.722 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-09-10T23:55:34.722 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-09-10T23:55:34.722 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-09-10T23:55:34.722 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-09-10T23:55:34.722 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-09-10T23:55:34.722 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-09-10T23:55:34.722 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout }, 2024-09-10T23:55:34.722 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-09-10T23:55:34.722 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-10T23:55:34.722 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "btime": "2024-09-10T23:55:16:317386+0000", 2024-09-10T23:55:34.722 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-09-10T23:55:34.722 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-09-10T23:55:34.722 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout }, 2024-09-10T23:55:34.722 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-09-10T23:55:34.723 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "available": false, 2024-09-10T23:55:34.723 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-09-10T23:55:34.723 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "modules": [ 2024-09-10T23:55:34.723 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "iostat", 2024-09-10T23:55:34.723 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "nfs", 2024-09-10T23:55:34.723 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "restful" 2024-09-10T23:55:34.723 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout ], 2024-09-10T23:55:34.723 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-10T23:55:34.723 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout }, 2024-09-10T23:55:34.723 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-09-10T23:55:34.723 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-10T23:55:34.723 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "modified": "2024-09-10T23:55:16.318147+0000", 2024-09-10T23:55:34.723 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-10T23:55:34.724 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout }, 2024-09-10T23:55:34.724 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-09-10T23:55:34.724 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout } 2024-09-10T23:55:34.724 INFO:teuthology.orchestra.run.smithi012.stdout:mgr not available, waiting (1/15)... 2024-09-10T23:55:37.813 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:55:37 smithi012 ceph-mon[30333]: from='client.? 172.21.15.12:0/3441753587' entity='client.admin' cmd={"prefix": "status", "format": "json-pretty"} : dispatch 2024-09-10T23:55:38.394 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout 2024-09-10T23:55:38.394 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout { 2024-09-10T23:55:38.394 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "fsid": "ff3ad74a-6fcf-11ef-bcea-c7b262605968", 2024-09-10T23:55:38.395 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "health": { 2024-09-10T23:55:38.395 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-09-10T23:55:38.395 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-09-10T23:55:38.395 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-09-10T23:55:38.395 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout }, 2024-09-10T23:55:38.395 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-09-10T23:55:38.395 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-09-10T23:55:38.395 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout 0 2024-09-10T23:55:38.395 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout ], 2024-09-10T23:55:38.395 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-09-10T23:55:38.395 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "a" 2024-09-10T23:55:38.395 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout ], 2024-09-10T23:55:38.395 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "quorum_age": 15, 2024-09-10T23:55:38.396 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "monmap": { 2024-09-10T23:55:38.396 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-10T23:55:38.396 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-09-10T23:55:38.396 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-09-10T23:55:38.396 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout }, 2024-09-10T23:55:38.396 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-09-10T23:55:38.396 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-10T23:55:38.396 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-09-10T23:55:38.396 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-09-10T23:55:38.396 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-09-10T23:55:38.396 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-09-10T23:55:38.396 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-09-10T23:55:38.397 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-09-10T23:55:38.400 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout }, 2024-09-10T23:55:38.400 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-09-10T23:55:38.400 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-09-10T23:55:38.400 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-09-10T23:55:38.400 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-09-10T23:55:38.400 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-09-10T23:55:38.400 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-09-10T23:55:38.400 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-09-10T23:55:38.401 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-09-10T23:55:38.401 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-09-10T23:55:38.401 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout }, 2024-09-10T23:55:38.401 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-09-10T23:55:38.401 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-10T23:55:38.401 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "btime": "2024-09-10T23:55:16:317386+0000", 2024-09-10T23:55:38.401 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-09-10T23:55:38.401 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-09-10T23:55:38.401 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout }, 2024-09-10T23:55:38.401 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-09-10T23:55:38.401 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "available": false, 2024-09-10T23:55:38.401 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-09-10T23:55:38.401 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "modules": [ 2024-09-10T23:55:38.401 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "iostat", 2024-09-10T23:55:38.402 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "nfs", 2024-09-10T23:55:38.402 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "restful" 2024-09-10T23:55:38.402 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout ], 2024-09-10T23:55:38.402 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-10T23:55:38.402 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout }, 2024-09-10T23:55:38.402 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-09-10T23:55:38.402 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-10T23:55:38.402 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "modified": "2024-09-10T23:55:16.318147+0000", 2024-09-10T23:55:38.402 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-10T23:55:38.402 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout }, 2024-09-10T23:55:38.402 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-09-10T23:55:38.402 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout } 2024-09-10T23:55:38.402 INFO:teuthology.orchestra.run.smithi012.stdout:mgr not available, waiting (2/15)... 2024-09-10T23:55:40.063 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:55:39 smithi012 ceph-mon[30333]: Activating manager daemon x 2024-09-10T23:55:40.063 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:55:39 smithi012 ceph-mon[30333]: mgrmap e2: x(active, starting, since 0.00487215s) 2024-09-10T23:55:40.063 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:55:39 smithi012 ceph-mon[30333]: from='mgr.14100 172.21.15.12:0/4044522740' entity='mgr.x' cmd={"prefix": "mds metadata"} : dispatch 2024-09-10T23:55:40.063 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:55:39 smithi012 ceph-mon[30333]: from='mgr.14100 172.21.15.12:0/4044522740' entity='mgr.x' cmd={"prefix": "osd metadata"} : dispatch 2024-09-10T23:55:40.063 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:55:39 smithi012 ceph-mon[30333]: from='mgr.14100 172.21.15.12:0/4044522740' entity='mgr.x' cmd={"prefix": "mon metadata"} : dispatch 2024-09-10T23:55:40.063 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:55:39 smithi012 ceph-mon[30333]: from='mgr.14100 172.21.15.12:0/4044522740' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-10T23:55:40.063 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:55:39 smithi012 ceph-mon[30333]: from='mgr.14100 172.21.15.12:0/4044522740' entity='mgr.x' cmd={"prefix": "mgr metadata", "who": "x", "id": "x"} : dispatch 2024-09-10T23:55:40.063 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:55:39 smithi012 ceph-mon[30333]: Manager daemon x is now available 2024-09-10T23:55:40.064 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:55:39 smithi012 ceph-mon[30333]: from='mgr.14100 172.21.15.12:0/4044522740' entity='mgr.x' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"} : dispatch 2024-09-10T23:55:40.064 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:55:39 smithi012 ceph-mon[30333]: from='mgr.14100 172.21.15.12:0/4044522740' entity='mgr.x' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"} : dispatch 2024-09-10T23:55:40.064 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:55:39 smithi012 ceph-mon[30333]: from='mgr.14100 172.21.15.12:0/4044522740' entity='mgr.x' 2024-09-10T23:55:40.064 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:55:39 smithi012 ceph-mon[30333]: from='mgr.14100 172.21.15.12:0/4044522740' entity='mgr.x' 2024-09-10T23:55:40.064 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:55:39 smithi012 ceph-mon[30333]: from='mgr.14100 172.21.15.12:0/4044522740' entity='mgr.x' 2024-09-10T23:55:41.766 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout 2024-09-10T23:55:41.766 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout { 2024-09-10T23:55:41.766 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "fsid": "ff3ad74a-6fcf-11ef-bcea-c7b262605968", 2024-09-10T23:55:41.766 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "health": { 2024-09-10T23:55:41.766 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-09-10T23:55:41.766 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-09-10T23:55:41.766 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-09-10T23:55:41.767 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout }, 2024-09-10T23:55:41.767 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-09-10T23:55:41.767 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-09-10T23:55:41.767 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout 0 2024-09-10T23:55:41.767 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout ], 2024-09-10T23:55:41.767 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-09-10T23:55:41.767 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "a" 2024-09-10T23:55:41.767 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout ], 2024-09-10T23:55:41.767 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "quorum_age": 19, 2024-09-10T23:55:41.767 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "monmap": { 2024-09-10T23:55:41.767 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-10T23:55:41.767 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-09-10T23:55:41.767 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-09-10T23:55:41.768 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout }, 2024-09-10T23:55:41.768 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-09-10T23:55:41.768 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-10T23:55:41.768 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-09-10T23:55:41.768 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-09-10T23:55:41.768 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-09-10T23:55:41.768 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-09-10T23:55:41.768 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-09-10T23:55:41.768 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-09-10T23:55:41.768 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout }, 2024-09-10T23:55:41.768 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-09-10T23:55:41.768 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-09-10T23:55:41.768 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-09-10T23:55:41.769 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-09-10T23:55:41.769 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-09-10T23:55:41.769 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-09-10T23:55:41.769 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-09-10T23:55:41.769 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-09-10T23:55:41.769 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-09-10T23:55:41.769 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout }, 2024-09-10T23:55:41.769 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-09-10T23:55:41.769 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-10T23:55:41.769 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "btime": "2024-09-10T23:55:16:317386+0000", 2024-09-10T23:55:41.769 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-09-10T23:55:41.769 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-09-10T23:55:41.769 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout }, 2024-09-10T23:55:41.769 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-09-10T23:55:41.769 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "available": true, 2024-09-10T23:55:41.770 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-09-10T23:55:41.770 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "modules": [ 2024-09-10T23:55:41.770 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "iostat", 2024-09-10T23:55:41.770 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "nfs", 2024-09-10T23:55:41.770 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "restful" 2024-09-10T23:55:41.770 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout ], 2024-09-10T23:55:41.770 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-10T23:55:41.770 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout }, 2024-09-10T23:55:41.770 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-09-10T23:55:41.770 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-10T23:55:41.770 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "modified": "2024-09-10T23:55:16.318147+0000", 2024-09-10T23:55:41.770 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-10T23:55:41.770 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout }, 2024-09-10T23:55:41.771 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-09-10T23:55:41.771 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout } 2024-09-10T23:55:41.771 INFO:teuthology.orchestra.run.smithi012.stdout:mgr is available 2024-09-10T23:55:42.008 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:55:41 smithi012 ceph-mon[30333]: mgrmap e3: x(active, since 1.00974s) 2024-09-10T23:55:42.008 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:55:41 smithi012 ceph-mon[30333]: from='client.? 172.21.15.12:0/2051523713' entity='client.admin' cmd={"prefix": "status", "format": "json-pretty"} : dispatch 2024-09-10T23:55:43.063 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:55:42 smithi012 ceph-mon[30333]: mgrmap e4: x(active, since 2s) 2024-09-10T23:55:43.063 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:55:42 smithi012 ceph-mon[30333]: from='client.? 172.21.15.12:0/3918259778' entity='client.admin' cmd={"prefix": "config assimilate-conf"} : dispatch 2024-09-10T23:55:43.174 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout 2024-09-10T23:55:43.174 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout [global] 2024-09-10T23:55:43.174 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout fsid = ff3ad74a-6fcf-11ef-bcea-c7b262605968 2024-09-10T23:55:43.174 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout mon_cluster_log_file_level = debug 2024-09-10T23:55:43.174 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.12:3300,v1:172.21.15.12:6789] 2024-09-10T23:55:43.175 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-09-10T23:55:43.175 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-09-10T23:55:43.175 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-09-10T23:55:43.175 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-09-10T23:55:43.175 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout 2024-09-10T23:55:43.175 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout [mgr] 2024-09-10T23:55:43.175 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-09-10T23:55:43.175 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout 2024-09-10T23:55:43.175 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout [osd] 2024-09-10T23:55:43.175 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-09-10T23:55:43.175 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-09-10T23:55:43.175 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-09-10T23:55:43.175 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-09-10T23:55:43.176 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-09-10T23:55:43.176 INFO:teuthology.orchestra.run.smithi012.stdout:Enabling cephadm module... 2024-09-10T23:55:44.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:55:43 smithi012 ceph-mon[30333]: from='client.? 172.21.15.12:0/3067688388' entity='client.admin' cmd={"prefix": "mgr module enable", "module": "cephadm"} : dispatch 2024-09-10T23:55:44.313 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Sep 10 23:55:44 smithi012 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-mgr-x[30560]: ignoring --setuser ceph since I am not root 2024-09-10T23:55:44.313 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Sep 10 23:55:44 smithi012 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-mgr-x[30560]: ignoring --setgroup ceph since I am not root 2024-09-10T23:55:45.312 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:55:44 smithi012 ceph-mon[30333]: from='client.? 172.21.15.12:0/3067688388' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-09-10T23:55:45.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:55:44 smithi012 ceph-mon[30333]: mgrmap e5: x(active, since 4s) 2024-09-10T23:55:46.056 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:55:45 smithi012 ceph-mon[30333]: from='client.? 172.21.15.12:0/1878335411' entity='client.admin' cmd={"prefix": "mgr stat"} : dispatch 2024-09-10T23:55:46.061 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout { 2024-09-10T23:55:46.061 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-09-10T23:55:46.061 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "available": true, 2024-09-10T23:55:46.061 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-09-10T23:55:46.061 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-09-10T23:55:46.061 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout } 2024-09-10T23:55:46.062 INFO:teuthology.orchestra.run.smithi012.stdout:Waiting for the mgr to restart... 2024-09-10T23:55:46.062 INFO:teuthology.orchestra.run.smithi012.stdout:Waiting for mgr epoch 5... 2024-09-10T23:55:50.812 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:55:50 smithi012 ceph-mon[30333]: Active manager daemon x restarted 2024-09-10T23:55:50.813 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:55:50 smithi012 ceph-mon[30333]: Activating manager daemon x 2024-09-10T23:55:50.813 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:55:50 smithi012 ceph-mon[30333]: osdmap e2: 0 total, 0 up, 0 in 2024-09-10T23:55:50.813 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:55:50 smithi012 ceph-mon[30333]: mgrmap e6: x(active, starting, since 0.00366577s) 2024-09-10T23:55:50.813 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:55:50 smithi012 ceph-mon[30333]: from='mgr.14118 172.21.15.12:0/3636166767' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-10T23:55:50.813 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:55:50 smithi012 ceph-mon[30333]: from='mgr.14118 172.21.15.12:0/3636166767' entity='mgr.x' cmd={"prefix": "mgr metadata", "who": "x", "id": "x"} : dispatch 2024-09-10T23:55:50.813 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:55:50 smithi012 ceph-mon[30333]: from='mgr.14118 172.21.15.12:0/3636166767' entity='mgr.x' cmd={"prefix": "mds metadata"} : dispatch 2024-09-10T23:55:50.813 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:55:50 smithi012 ceph-mon[30333]: from='mgr.14118 172.21.15.12:0/3636166767' entity='mgr.x' cmd={"prefix": "osd metadata"} : dispatch 2024-09-10T23:55:50.814 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:55:50 smithi012 ceph-mon[30333]: from='mgr.14118 172.21.15.12:0/3636166767' entity='mgr.x' cmd={"prefix": "mon metadata"} : dispatch 2024-09-10T23:55:50.814 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:55:50 smithi012 ceph-mon[30333]: Manager daemon x is now available 2024-09-10T23:55:51.965 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout { 2024-09-10T23:55:51.966 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-09-10T23:55:51.966 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "initialized": true 2024-09-10T23:55:51.966 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout } 2024-09-10T23:55:51.966 INFO:teuthology.orchestra.run.smithi012.stdout:mgr epoch 5 is available 2024-09-10T23:55:51.966 INFO:teuthology.orchestra.run.smithi012.stdout:Setting orchestrator backend to cephadm... 2024-09-10T23:55:52.488 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:55:52 smithi012 ceph-mon[30333]: mgrmap e7: x(active, since 1.00648s) 2024-09-10T23:55:52.488 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:55:52 smithi012 ceph-mon[30333]: from='mgr.14118 172.21.15.12:0/3636166767' entity='mgr.x' 2024-09-10T23:55:52.488 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:55:52 smithi012 ceph-mon[30333]: from='mgr.14118 172.21.15.12:0/3636166767' entity='mgr.x' 2024-09-10T23:55:52.488 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:55:52 smithi012 ceph-mon[30333]: Found migration_current of "None". Setting to last migration. 2024-09-10T23:55:52.488 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:55:52 smithi012 ceph-mon[30333]: from='mgr.14118 172.21.15.12:0/3636166767' entity='mgr.x' 2024-09-10T23:55:52.489 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:55:52 smithi012 ceph-mon[30333]: from='mgr.14118 172.21.15.12:0/3636166767' entity='mgr.x' 2024-09-10T23:55:52.489 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:55:52 smithi012 ceph-mon[30333]: from='mgr.14118 172.21.15.12:0/3636166767' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T23:55:52.489 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:55:52 smithi012 ceph-mon[30333]: from='mgr.14118 172.21.15.12:0/3636166767' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T23:55:52.489 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:55:52 smithi012 ceph-mon[30333]: from='mgr.14118 172.21.15.12:0/3636166767' entity='mgr.x' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"} : dispatch 2024-09-10T23:55:52.489 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:55:52 smithi012 ceph-mon[30333]: from='mgr.14118 172.21.15.12:0/3636166767' entity='mgr.x' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"} : dispatch 2024-09-10T23:55:53.063 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Sep 10 23:55:52 smithi012 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-mgr-x[30560]: 2024-09-10T23:55:52.789+0000 7fc61ea00640 -1 log_channel(cephadm) log [ERR] : [10/Sep/2024:23:55:52] ENGINE Error in HTTPServer.serve 2024-09-10T23:55:53.063 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Sep 10 23:55:52 smithi012 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-mgr-x[30560]: Traceback (most recent call last): 2024-09-10T23:55:53.063 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Sep 10 23:55:52 smithi012 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-mgr-x[30560]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-10T23:55:53.063 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Sep 10 23:55:52 smithi012 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-mgr-x[30560]: self._connections.run(self.expiration_interval) 2024-09-10T23:55:53.063 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Sep 10 23:55:52 smithi012 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-mgr-x[30560]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-10T23:55:53.064 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Sep 10 23:55:52 smithi012 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-mgr-x[30560]: self._run(expiration_interval) 2024-09-10T23:55:53.064 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Sep 10 23:55:52 smithi012 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-mgr-x[30560]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-10T23:55:53.064 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Sep 10 23:55:52 smithi012 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-mgr-x[30560]: new_conn = self._from_server_socket(self.server.socket) 2024-09-10T23:55:53.064 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Sep 10 23:55:52 smithi012 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-mgr-x[30560]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-10T23:55:53.064 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Sep 10 23:55:52 smithi012 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-mgr-x[30560]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-10T23:55:53.064 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Sep 10 23:55:52 smithi012 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-mgr-x[30560]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-10T23:55:53.064 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Sep 10 23:55:52 smithi012 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-mgr-x[30560]: s = self.context.wrap_socket( 2024-09-10T23:55:53.064 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Sep 10 23:55:52 smithi012 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-mgr-x[30560]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-10T23:55:53.064 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Sep 10 23:55:52 smithi012 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-mgr-x[30560]: return self.sslsocket_class._create( 2024-09-10T23:55:53.064 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Sep 10 23:55:52 smithi012 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-mgr-x[30560]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-10T23:55:53.064 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Sep 10 23:55:52 smithi012 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-mgr-x[30560]: self.do_handshake() 2024-09-10T23:55:53.064 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Sep 10 23:55:52 smithi012 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-mgr-x[30560]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-10T23:55:53.065 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Sep 10 23:55:52 smithi012 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-mgr-x[30560]: self._sslobj.do_handshake() 2024-09-10T23:55:53.065 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Sep 10 23:55:52 smithi012 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-mgr-x[30560]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-10T23:55:53.065 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Sep 10 23:55:52 smithi012 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-mgr-x[30560]: 2024-09-10T23:55:53.626 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:55:53 smithi012 ceph-mon[30333]: [10/Sep/2024:23:55:52] ENGINE Bus STARTING 2024-09-10T23:55:53.626 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:55:53 smithi012 ceph-mon[30333]: mgrmap e8: x(active, since 2s) 2024-09-10T23:55:53.626 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:55:53 smithi012 ceph-mon[30333]: [10/Sep/2024:23:55:52] ENGINE Serving on http://172.21.15.12:8765 2024-09-10T23:55:53.627 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:55:53 smithi012 ceph-mon[30333]: from='client.14130 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T23:55:53.627 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:55:53 smithi012 ceph-mon[30333]: from='mgr.14118 172.21.15.12:0/3636166767' entity='mgr.x' 2024-09-10T23:55:53.627 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:55:53 smithi012 ceph-mon[30333]: from='mgr.14118 172.21.15.12:0/3636166767' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T23:55:53.627 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:55:53 smithi012 ceph-mon[30333]: [10/Sep/2024:23:55:52] ENGINE Serving on https://172.21.15.12:7150 2024-09-10T23:55:53.627 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:55:53 smithi012 ceph-mon[30333]: [10/Sep/2024:23:55:52] ENGINE Bus STARTED 2024-09-10T23:55:53.627 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:55:53 smithi012 ceph-mon[30333]: from='mgr.14118 172.21.15.12:0/3636166767' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T23:55:53.627 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:55:53 smithi012 ceph-mon[30333]: [10/Sep/2024:23:55:52] ENGINE Error in HTTPServer.serve 2024-09-10T23:55:53.627 INFO:journalctl@ceph.mon.a.smithi012.stdout: Traceback (most recent call last): 2024-09-10T23:55:53.627 INFO:journalctl@ceph.mon.a.smithi012.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-10T23:55:53.627 INFO:journalctl@ceph.mon.a.smithi012.stdout: self._connections.run(self.expiration_interval) 2024-09-10T23:55:53.627 INFO:journalctl@ceph.mon.a.smithi012.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-10T23:55:53.627 INFO:journalctl@ceph.mon.a.smithi012.stdout: self._run(expiration_interval) 2024-09-10T23:55:53.627 INFO:journalctl@ceph.mon.a.smithi012.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-10T23:55:53.628 INFO:journalctl@ceph.mon.a.smithi012.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-10T23:55:53.628 INFO:journalctl@ceph.mon.a.smithi012.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-10T23:55:53.628 INFO:journalctl@ceph.mon.a.smithi012.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-10T23:55:53.628 INFO:journalctl@ceph.mon.a.smithi012.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-10T23:55:53.628 INFO:journalctl@ceph.mon.a.smithi012.stdout: s = self.context.wrap_socket( 2024-09-10T23:55:53.628 INFO:journalctl@ceph.mon.a.smithi012.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-10T23:55:53.628 INFO:journalctl@ceph.mon.a.smithi012.stdout: return self.sslsocket_class._create( 2024-09-10T23:55:53.628 INFO:journalctl@ceph.mon.a.smithi012.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-10T23:55:53.628 INFO:journalctl@ceph.mon.a.smithi012.stdout: self.do_handshake() 2024-09-10T23:55:53.628 INFO:journalctl@ceph.mon.a.smithi012.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-10T23:55:53.628 INFO:journalctl@ceph.mon.a.smithi012.stdout: self._sslobj.do_handshake() 2024-09-10T23:55:53.628 INFO:journalctl@ceph.mon.a.smithi012.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-10T23:55:54.689 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:55:54 smithi012 ceph-mon[30333]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T23:55:54.701 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout value unchanged 2024-09-10T23:55:54.701 INFO:teuthology.orchestra.run.smithi012.stdout:Generating ssh key... 2024-09-10T23:55:56.046 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Sep 10 23:55:55 smithi012 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-mgr-x[30560]: Generating public/private rsa key pair. 2024-09-10T23:55:56.046 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Sep 10 23:55:55 smithi012 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-mgr-x[30560]: Your identification has been saved in /tmp/tmpk8y38lfi/key 2024-09-10T23:55:56.046 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Sep 10 23:55:55 smithi012 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-mgr-x[30560]: Your public key has been saved in /tmp/tmpk8y38lfi/key.pub 2024-09-10T23:55:56.046 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Sep 10 23:55:55 smithi012 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-mgr-x[30560]: The key fingerprint is: 2024-09-10T23:55:56.046 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Sep 10 23:55:55 smithi012 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-mgr-x[30560]: SHA256:/VGrSjSIEb1dWzNM8QwH8SeAskIbsTgPulRyslaVWD4 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968 2024-09-10T23:55:56.046 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Sep 10 23:55:55 smithi012 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-mgr-x[30560]: The key's randomart image is: 2024-09-10T23:55:56.046 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Sep 10 23:55:55 smithi012 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-mgr-x[30560]: +---[RSA 3072]----+ 2024-09-10T23:55:56.047 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Sep 10 23:55:55 smithi012 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-mgr-x[30560]: | o=+ ..o*+.| 2024-09-10T23:55:56.047 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Sep 10 23:55:55 smithi012 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-mgr-x[30560]: | .++oo . ..=* | 2024-09-10T23:55:56.047 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Sep 10 23:55:55 smithi012 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-mgr-x[30560]: | o B.Eo = . ooo=| 2024-09-10T23:55:56.047 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Sep 10 23:55:55 smithi012 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-mgr-x[30560]: | O +o+oo. .. o.| 2024-09-10T23:55:56.047 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Sep 10 23:55:55 smithi012 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-mgr-x[30560]: | = o.S + . . | 2024-09-10T23:55:56.047 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Sep 10 23:55:55 smithi012 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-mgr-x[30560]: | o . . o o | 2024-09-10T23:55:56.047 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Sep 10 23:55:55 smithi012 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-mgr-x[30560]: | . . o | 2024-09-10T23:55:56.047 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Sep 10 23:55:55 smithi012 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-mgr-x[30560]: | . . | 2024-09-10T23:55:56.047 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Sep 10 23:55:55 smithi012 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-mgr-x[30560]: | . | 2024-09-10T23:55:56.047 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Sep 10 23:55:55 smithi012 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-mgr-x[30560]: +----[SHA256]-----+ 2024-09-10T23:55:57.024 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:55:56 smithi012 ceph-mon[30333]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T23:55:57.025 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:55:56 smithi012 ceph-mon[30333]: Generating ssh key... 2024-09-10T23:55:57.025 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:55:56 smithi012 ceph-mon[30333]: from='mgr.14118 172.21.15.12:0/3636166767' entity='mgr.x' 2024-09-10T23:55:57.025 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:55:56 smithi012 ceph-mon[30333]: from='mgr.14118 172.21.15.12:0/3636166767' entity='mgr.x' 2024-09-10T23:55:57.633 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCbvjVzhDtfoV2p8UgOiWADhAz0pJBdJ69V1saVJETlY7TvbfPCGtv4Yjx8BXSfaRpB/WiTriP6oPY1GrXD7SRr/MPhVO1HSygajA1/uouG4J0nHae/ZFPMNNOR7NJVfF3GNwFjhr5zFyzq6vvr5aTGnCOCZdkj3/ym9gm/4mZ70IvHUTqzgMRDshVRfIObO6p/TAufBLmbp4oVy5F2PYDEEVYY3tB6179osKnq+Pc3didzK6qKAD/csL2sSWQJt1GkimHUiJjfCqmgWYbkFfLstA+BQuUwDluJ127bV/Pxz/HUgEwlf3X8KTkwbmlgptBnrcxTKI188BqLLw7Uiy6F4FX35h1CVE9UH1syk+uVoyw2NZ0ed65FpuOcUHsMw0n+Xh8Pakr/H8kMKhiVUQeDZFYOOo1NV1FhT3lylr2VOJarlujQONjBJozIAqGr36l2nxLCKgB+ndwzRp1OYRoU6uonFcBjLbbYIRRzKiMKXlBl/38lm/21G43MOqAMMI0= ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968 2024-09-10T23:55:57.633 INFO:teuthology.orchestra.run.smithi012.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-09-10T23:55:57.633 INFO:teuthology.orchestra.run.smithi012.stdout:Adding key to root@localhost authorized_keys... 2024-09-10T23:55:57.644 INFO:teuthology.orchestra.run.smithi012.stdout:Adding host smithi012... 2024-09-10T23:55:57.922 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:55:57 smithi012 ceph-mon[30333]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T23:55:59.505 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:55:59 smithi012 ceph-mon[30333]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi012", "addr": "172.21.15.12", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T23:56:00.562 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:56:00 smithi012 ceph-mon[30333]: Deploying cephadm binary to smithi012 2024-09-10T23:56:03.516 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout Added host 'smithi012' with addr '172.21.15.12' 2024-09-10T23:56:03.516 INFO:teuthology.orchestra.run.smithi012.stdout:Deploying unmanaged mon service... 2024-09-10T23:56:04.062 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:56:03 smithi012 ceph-mon[30333]: from='mgr.14118 172.21.15.12:0/3636166767' entity='mgr.x' 2024-09-10T23:56:04.063 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:56:03 smithi012 ceph-mon[30333]: Added host smithi012 2024-09-10T23:56:04.063 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:56:03 smithi012 ceph-mon[30333]: from='mgr.14118 172.21.15.12:0/3636166767' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T23:56:05.117 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-09-10T23:56:05.117 INFO:teuthology.orchestra.run.smithi012.stdout:Deploying unmanaged mgr service... 2024-09-10T23:56:05.602 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:56:05 smithi012 ceph-mon[30333]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T23:56:05.602 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:56:05 smithi012 ceph-mon[30333]: Saving service mon spec with placement count:5 2024-09-10T23:56:05.603 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:56:05 smithi012 ceph-mon[30333]: from='mgr.14118 172.21.15.12:0/3636166767' entity='mgr.x' 2024-09-10T23:56:07.096 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-09-10T23:56:07.350 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:56:07 smithi012 ceph-mon[30333]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T23:56:07.350 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:56:07 smithi012 ceph-mon[30333]: Saving service mgr spec with placement count:2 2024-09-10T23:56:07.350 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:56:07 smithi012 ceph-mon[30333]: from='mgr.14118 172.21.15.12:0/3636166767' entity='mgr.x' 2024-09-10T23:56:07.350 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:56:07 smithi012 ceph-mon[30333]: from='mgr.14118 172.21.15.12:0/3636166767' entity='mgr.x' 2024-09-10T23:56:07.350 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:56:07 smithi012 ceph-mon[30333]: from='mgr.14118 172.21.15.12:0/3636166767' entity='mgr.x' 2024-09-10T23:56:09.025 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:56:08 smithi012 ceph-mon[30333]: from='client.? 172.21.15.12:0/695443506' entity='client.admin' 2024-09-10T23:56:09.025 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:56:08 smithi012 ceph-mon[30333]: from='mgr.14118 172.21.15.12:0/3636166767' entity='mgr.x' 2024-09-10T23:56:10.405 INFO:teuthology.orchestra.run.smithi012.stdout:Enabling the dashboard module... 2024-09-10T23:56:11.051 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:56:10 smithi012 ceph-mon[30333]: from='client.? 172.21.15.12:0/1664915659' entity='client.admin' 2024-09-10T23:56:11.051 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:56:10 smithi012 ceph-mon[30333]: from='mgr.14118 172.21.15.12:0/3636166767' entity='mgr.x' 2024-09-10T23:56:11.957 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:56:11 smithi012 ceph-mon[30333]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T23:56:11.957 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:56:11 smithi012 ceph-mon[30333]: from='client.? 172.21.15.12:0/3817788050' entity='client.admin' cmd={"prefix": "mgr module enable", "module": "dashboard"} : dispatch 2024-09-10T23:56:11.957 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Sep 10 23:56:11 smithi012 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-mgr-x[30560]: ignoring --setuser ceph since I am not root 2024-09-10T23:56:11.957 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Sep 10 23:56:11 smithi012 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-mgr-x[30560]: ignoring --setgroup ceph since I am not root 2024-09-10T23:56:12.858 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:56:12 smithi012 ceph-mon[30333]: from='client.? 172.21.15.12:0/3817788050' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-09-10T23:56:12.858 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:56:12 smithi012 ceph-mon[30333]: mgrmap e9: x(active, since 21s) 2024-09-10T23:56:13.991 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:56:13 smithi012 ceph-mon[30333]: from='client.? 172.21.15.12:0/1896691356' entity='client.admin' cmd={"prefix": "mgr stat"} : dispatch 2024-09-10T23:56:13.997 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout { 2024-09-10T23:56:13.997 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-09-10T23:56:13.997 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "available": true, 2024-09-10T23:56:13.998 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-09-10T23:56:13.998 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-09-10T23:56:13.998 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout } 2024-09-10T23:56:13.998 INFO:teuthology.orchestra.run.smithi012.stdout:Waiting for the mgr to restart... 2024-09-10T23:56:13.998 INFO:teuthology.orchestra.run.smithi012.stdout:Waiting for mgr epoch 9... 2024-09-10T23:56:18.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:56:18 smithi012 ceph-mon[30333]: Active manager daemon x restarted 2024-09-10T23:56:18.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:56:18 smithi012 ceph-mon[30333]: Activating manager daemon x 2024-09-10T23:56:18.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:56:18 smithi012 ceph-mon[30333]: osdmap e3: 0 total, 0 up, 0 in 2024-09-10T23:56:18.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:56:18 smithi012 ceph-mon[30333]: mgrmap e10: x(active, starting, since 0.00421712s) 2024-09-10T23:56:18.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:56:18 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-10T23:56:18.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:56:18 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "mgr metadata", "who": "x", "id": "x"} : dispatch 2024-09-10T23:56:18.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:56:18 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "mds metadata"} : dispatch 2024-09-10T23:56:18.314 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:56:18 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "osd metadata"} : dispatch 2024-09-10T23:56:18.314 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:56:18 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "mon metadata"} : dispatch 2024-09-10T23:56:18.314 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:56:18 smithi012 ceph-mon[30333]: Manager daemon x is now available 2024-09-10T23:56:19.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:56:19 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T23:56:19.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:56:19 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"} : dispatch 2024-09-10T23:56:19.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:56:19 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"} : dispatch 2024-09-10T23:56:19.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:56:19 smithi012 ceph-mon[30333]: mgrmap e11: x(active, since 1.00962s) 2024-09-10T23:56:19.718 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout { 2024-09-10T23:56:19.719 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-09-10T23:56:19.719 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "initialized": true 2024-09-10T23:56:19.719 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout } 2024-09-10T23:56:19.719 INFO:teuthology.orchestra.run.smithi012.stdout:mgr epoch 9 is available 2024-09-10T23:56:19.719 INFO:teuthology.orchestra.run.smithi012.stdout:Using certmgr to generate dashboard self-signed certificate... 2024-09-10T23:56:19.968 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Sep 10 23:56:19 smithi012 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-mgr-x[30560]: 2024-09-10T23:56:19.766+0000 7f9fc7200640 -1 log_channel(cephadm) log [ERR] : [10/Sep/2024:23:56:19] ENGINE Error in HTTPServer.serve 2024-09-10T23:56:19.969 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Sep 10 23:56:19 smithi012 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-mgr-x[30560]: Traceback (most recent call last): 2024-09-10T23:56:19.969 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Sep 10 23:56:19 smithi012 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-mgr-x[30560]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-10T23:56:19.969 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Sep 10 23:56:19 smithi012 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-mgr-x[30560]: self._connections.run(self.expiration_interval) 2024-09-10T23:56:19.969 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Sep 10 23:56:19 smithi012 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-mgr-x[30560]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-10T23:56:19.969 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Sep 10 23:56:19 smithi012 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-mgr-x[30560]: self._run(expiration_interval) 2024-09-10T23:56:19.969 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Sep 10 23:56:19 smithi012 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-mgr-x[30560]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-10T23:56:19.969 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Sep 10 23:56:19 smithi012 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-mgr-x[30560]: new_conn = self._from_server_socket(self.server.socket) 2024-09-10T23:56:19.969 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Sep 10 23:56:19 smithi012 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-mgr-x[30560]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-10T23:56:19.969 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Sep 10 23:56:19 smithi012 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-mgr-x[30560]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-10T23:56:19.969 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Sep 10 23:56:19 smithi012 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-mgr-x[30560]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-10T23:56:19.969 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Sep 10 23:56:19 smithi012 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-mgr-x[30560]: s = self.context.wrap_socket( 2024-09-10T23:56:19.970 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Sep 10 23:56:19 smithi012 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-mgr-x[30560]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-10T23:56:19.970 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Sep 10 23:56:19 smithi012 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-mgr-x[30560]: return self.sslsocket_class._create( 2024-09-10T23:56:19.970 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Sep 10 23:56:19 smithi012 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-mgr-x[30560]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-10T23:56:19.970 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Sep 10 23:56:19 smithi012 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-mgr-x[30560]: self.do_handshake() 2024-09-10T23:56:19.970 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Sep 10 23:56:19 smithi012 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-mgr-x[30560]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-10T23:56:19.970 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Sep 10 23:56:19 smithi012 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-mgr-x[30560]: self._sslobj.do_handshake() 2024-09-10T23:56:19.970 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Sep 10 23:56:19 smithi012 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-mgr-x[30560]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-10T23:56:19.970 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Sep 10 23:56:19 smithi012 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-mgr-x[30560]: 2024-09-10T23:56:20.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:56:20 smithi012 ceph-mon[30333]: [10/Sep/2024:23:56:19] ENGINE Bus STARTING 2024-09-10T23:56:20.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:56:20 smithi012 ceph-mon[30333]: [10/Sep/2024:23:56:19] ENGINE Serving on http://172.21.15.12:8765 2024-09-10T23:56:20.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:56:20 smithi012 ceph-mon[30333]: [10/Sep/2024:23:56:19] ENGINE Serving on https://172.21.15.12:7150 2024-09-10T23:56:20.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:56:20 smithi012 ceph-mon[30333]: [10/Sep/2024:23:56:19] ENGINE Bus STARTED 2024-09-10T23:56:20.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:56:20 smithi012 ceph-mon[30333]: [10/Sep/2024:23:56:19] ENGINE Error in HTTPServer.serve 2024-09-10T23:56:20.313 INFO:journalctl@ceph.mon.a.smithi012.stdout: Traceback (most recent call last): 2024-09-10T23:56:20.313 INFO:journalctl@ceph.mon.a.smithi012.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-10T23:56:20.313 INFO:journalctl@ceph.mon.a.smithi012.stdout: self._connections.run(self.expiration_interval) 2024-09-10T23:56:20.314 INFO:journalctl@ceph.mon.a.smithi012.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-10T23:56:20.314 INFO:journalctl@ceph.mon.a.smithi012.stdout: self._run(expiration_interval) 2024-09-10T23:56:20.314 INFO:journalctl@ceph.mon.a.smithi012.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-10T23:56:20.314 INFO:journalctl@ceph.mon.a.smithi012.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-10T23:56:20.314 INFO:journalctl@ceph.mon.a.smithi012.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-10T23:56:20.314 INFO:journalctl@ceph.mon.a.smithi012.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-10T23:56:20.314 INFO:journalctl@ceph.mon.a.smithi012.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-10T23:56:20.314 INFO:journalctl@ceph.mon.a.smithi012.stdout: s = self.context.wrap_socket( 2024-09-10T23:56:20.314 INFO:journalctl@ceph.mon.a.smithi012.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-10T23:56:20.314 INFO:journalctl@ceph.mon.a.smithi012.stdout: return self.sslsocket_class._create( 2024-09-10T23:56:20.314 INFO:journalctl@ceph.mon.a.smithi012.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-10T23:56:20.314 INFO:journalctl@ceph.mon.a.smithi012.stdout: self.do_handshake() 2024-09-10T23:56:20.314 INFO:journalctl@ceph.mon.a.smithi012.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-10T23:56:20.314 INFO:journalctl@ceph.mon.a.smithi012.stdout: self._sslobj.do_handshake() 2024-09-10T23:56:20.315 INFO:journalctl@ceph.mon.a.smithi012.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-10T23:56:21.312 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:56:21 smithi012 ceph-mon[30333]: mgrmap e12: x(active, since 2s) 2024-09-10T23:56:21.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:56:21 smithi012 ceph-mon[30333]: from='client.14162 -' entity='client.admin' cmd=[{"prefix": "orch certmgr generate-certificates", "module_name": "dashboard", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T23:56:21.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:56:21 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-10T23:56:22.094 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout {"cert": "-----BEGIN CERTIFICATE-----\nMIIE+zCCAuOgAwIBAgIUV72gWzGyURlpbNGOqr2KszPEMg4wDQYJKoZIhvcNAQEL\nBQAwFzEVMBMGA1UEAwwMY2VwaGFkbS1yb290MB4XDTI0MDkxMDIzNTYyMVoXDTM0\nMDkxMTIzNTYyMVowFzEVMBMGA1UEAwwMMTcyLjIxLjE1LjEyMIICIjANBgkqhkiG\n9w0BAQEFAAOCAg8AMIICCgKCAgEA26namrcPKDUwrD6bTAMrZS/5Hgkmyc6LkmZI\nMDJd1e+wCUabIjOw/fTs1R3/SgPmDuEQNaGIK/kSbc/oTtQUCLKBwVi9MhJ2BMML\nCYXpFkFIAeSAqQDO+v5kL6kfNOdj4V22spXdj4MH4sdLsbsI6eJnGFMeBDF/bXJG\nskF6QW7Ixkc7S96zyQ6AR/uP8RDy2sVUHQUYWoZMDz6eofYCp6pk0hUL4T7dRc8v\nYge3+9/QhG5IAhFSAAIn9uzSBprzXV9zUq9J9v3bQrKMdpZCyxSUPNtPMbLxiBah\nlYpodAPQ/c3nOJHrXdMuIWlBnsA537nBumcJqbeqHByPo94gd8SuGSLOn+EB19tN\n+a3KzTVOc9csdKjWHFpcjUce2dkf7awpSiJnoULe8MIpAP+AyvDLwdNehFLR/Qnd\nF7WQlmvrMwc/e/iutAqC1+gkGw17EDPHZ9Nm7js59a/fnyoMpxpZkQUaz4qiNLkY\nm6WVH4QTRyzGL4eCjkKHPBifeMrfseHuWj5gcVjRINQPeTMj1hgu6fasGIYWcoLg\nQVZE+YitDOZgnIODnqsYreYoA2i1mh+7gKJZ324HemeyaYt11ssjICy84RPvdyWl\nyu25rlsyrfq3S7SkTDvL1WQuI1rwT0kgMFMfVS2m4o2FO57AolRKBQgfcl98UnaR\ndsLNaekCAwEAAaM/MD0wLQYDVR0RBCYwJIIJc21pdGhpMDEyghFkYXNoYm9hcmRf\nc2VydmVyc4cErBUPDDAMBgNVHRMBAf8EAjAAMA0GCSqGSIb3DQEBCwUAA4ICAQC8\njeCkhV9fxaZP9RUG8yR5lxF28AvKhDCnZDnBB6Opg1/kbWb2oKP1jI3icrPSGVHf\nq9D2c6zs5Va4yIkWJRXSo5FRKBBsbuQGTkWk0cPdlrxt4zIabCOjHr5cafGKHvGU\npnX4WCpYaGJoN03R4JJMxe4Ci6M6e3UlPJjEPrAyBJ6VK9OJvw3pTrhnDXFs8L+o\nXjEkQ0+3hU16bmpeTrpbqrRigWsjU+HNDwd725umF9lAK/gclU83R44Xi9//NQQg\nZc1VV70SFT6cOEkBP6cj4eLbzAbk9VzRJ1rJrg6OEHcFrCr0OCsYV3dVkc5c4Eet\n89+CVklJ3b86EPDlVuu8Q9tgz38+16Fl2OXnPHzYDpMf7h5xNHQXKb9X5fydGUXJ\n6dY3zdADqglntJ+8cJCcoqwtt6pwmoJs9Z2RufQSkg50cCIb51RUar0HRf6oKTh9\nbAMEgbrrmcJdhen6K+xDaeeh+8o38ZW0s84Q1XEZVFBCwJVVdAJA38o4F7v4Znq5\nzkMEzznM1ilEK3zc0kEZwup6+pICw8BJ2hbvMs6L6AFFF8SahaesX+nUri4/fwmL\nK7AdLNQgSbVBl8kG7hSZ1rMnPgLYrMS9fRf1KU3nlTTosHLaygx8tsiI7zwF6H8y\nvsLrZ9h2sfhgqmZ10y9qOutqgjQ8Q4MoQt5lN20saw==\n-----END CERTIFICATE-----\n", "key": "-----BEGIN RSA PRIVATE KEY-----\nMIIJKAIBAAKCAgEA26namrcPKDUwrD6bTAMrZS/5Hgkmyc6LkmZIMDJd1e+wCUab\nIjOw/fTs1R3/SgPmDuEQNaGIK/kSbc/oTtQUCLKBwVi9MhJ2BMMLCYXpFkFIAeSA\nqQDO+v5kL6kfNOdj4V22spXdj4MH4sdLsbsI6eJnGFMeBDF/bXJGskF6QW7Ixkc7\nS96zyQ6AR/uP8RDy2sVUHQUYWoZMDz6eofYCp6pk0hUL4T7dRc8vYge3+9/QhG5I\nAhFSAAIn9uzSBprzXV9zUq9J9v3bQrKMdpZCyxSUPNtPMbLxiBahlYpodAPQ/c3n\nOJHrXdMuIWlBnsA537nBumcJqbeqHByPo94gd8SuGSLOn+EB19tN+a3KzTVOc9cs\ndKjWHFpcjUce2dkf7awpSiJnoULe8MIpAP+AyvDLwdNehFLR/QndF7WQlmvrMwc/\ne/iutAqC1+gkGw17EDPHZ9Nm7js59a/fnyoMpxpZkQUaz4qiNLkYm6WVH4QTRyzG\nL4eCjkKHPBifeMrfseHuWj5gcVjRINQPeTMj1hgu6fasGIYWcoLgQVZE+YitDOZg\nnIODnqsYreYoA2i1mh+7gKJZ324HemeyaYt11ssjICy84RPvdyWlyu25rlsyrfq3\nS7SkTDvL1WQuI1rwT0kgMFMfVS2m4o2FO57AolRKBQgfcl98UnaRdsLNaekCAwEA\nAQKCAgAPFAiZEmwITedkiWsiU3ze9iUu6ECo/WQ0BVCa+1gY23o9VsMHHN5/Pjjk\nE6oul9gqyQzya39ECnW3AzyebZ0e8DbrMVbSWTVP9B39tjsW354MYDWrNC6hCMWS\nhnTJjkEbSxIjjdk+tMc9rEHARlcRK+z4/NqcfdPSDNQ4v/kfBPZj8ueJeJTlfhj0\n5/ocWXYcKYuxrsP6UcnmiX0835zUysPfLlnpiSXJ+RP28hHxaI6BCqtnde4nqT46\nS00K7ke6jvbk8/qbKVeLy2MMMFoPI99dko3L+u0ssuX+VHkBTrJBu8ith8ms3/ZE\njZu3JJmMZ406aQB/LYfwJE9BV7npci1Z/YCPAU0rzRDLKJBwXLIqbpzrm6GE9HBs\nLDPBewO3PjJ37HxObUufQr8VEI6ke6p9ZeXiLYwvl1NcK82USKdN2kiTI8OuWx0+\nWZjY+xBAw6leaQf7jG7wBtuX1b3EZ4kZ2nM8vvi3AaQ2z+HvcotDmCLS5nnVZo41\ncdg0SlJJLUIjlB0i5UkL/U4FxCS8VXh5h3fwnojLHfP+KiznPEenXXoLf0KY6d7V\n3rEBZlQlyy8b5+p7aLip67K7Nlo5L8L0T2iwZvbq7gg0LUpFZwazvYL3OpItt3N7\nRwDMxLkwBUcknB9KvDm7fMfIn/EEQnAL/8egXb9sEBYyBJhzIQKCAQEA9BrVEw3d\nwS11bD5dHZ0rtcz6Bs8xrCJncNUItOV7Ao7ajzduBCYxBFHCJg5XZUJOY7MuX2PQ\nUcfoG2qfw1ls1QMBBrqSyc7AetkKhL4NQygD7VznbxVf/aNeahFCUVUSRrqz79qq\nYf3K5HpAXWZ7ydPQGYvdANsnaNgjU54bquu+WneneFkfWW+H3hwZPvfNqTZoX2qy\nrdQYVSOpQrz+1I7ZqK8qlzLTo4Qs73BRpXBUsFdVOEMf5VRgQoR6JgzLMgQRNSu3\nzQ1eVyaoZ5HFkjQQBSmEoFT6VJpvPzFFolqfJcjm5OTwStq3zYG1olzwnPy54O/t\nDFJ1LcxOol+FoQKCAQEA5l4evxe3wQcpEOIArwQSdq01SGxxbUUDEXhnIu02XMau\nERMr7LvrgwOZ0ICe8FpTXKYiQjSzVt5w5C2noaaAc2ZMnW2a4dHWw5d7AuN6XY5c\nd82XtTITJmSCL7LEMqFdigiH2UX3hbNRJgR/tfC4khTVn5sldeOJkcilzTZzHErY\n5tBBuqRo18GjwL/vQ0dbOjySwFYgyIlkIV4YLyB4zCcnltBBDw8BtEMirPW7NFVj\n3HGHw8ePD0HqIZ04cAxg2glH1qPCTah+D1rlH11iFdVdvEQT1F+xTJE8zY3RwySx\noG7YZk1orSMCEYYQjdkaDJZFuKUouzUV8Ona92/vSQKCAQAfkCCg+BgGVHRsbJvp\nqlruhPSHUN8hXMl3Q3R+v3VqXUJRZl9pVe7cmh26gfw28Z3lMlP6rZNQH8MZWG4T\n+koiOOM3FYTJZ1xoBgxgIZrhZhkr5ele6aqhxy/8EwM6NP/J+XYuu11wkjpb1Jrh\n+NBkxmiQBQ+sfqnX2qqmszjsntLcK8CRxSRSkiNFSujQxYXoLUrNm6RC6bvDtnq6\nUGm63kWylAOzzTSkXgDZbPKBq8anjoNMZRGIu6I+t5Z7uQfFDdWTxHVdpC7gofUA\nf07Gmsi/giDOAm7j5Zwg1aQlXAZcAmAolJmldCZ0AiOUPOQ3yE3Yke1avIa+Rz6b\nrDJBAoIBAAS+z5dUH77kLN5iKW1xggfbzYsuYK/Fiiij+Evp4wLKPO+xG2iG/nT2\ncAaxf5Sn56uy6GXAvCj/8AdjQV69FUxLQSzFAzm/ntLsneYTVYPz2MfG/mcHHwnd\nbCkbX5tDY6/iORoqrMyeQv+rfA39ZW/s6fGduonRJBZSscemZXyhthIbHB+CH37C\nSGNCEOWr5J8wfD1mCMrYua1OSOU/n/1kmRYsOArdXkWeVGqHrVGIqMPzur43sCS6\nUN5cZLs6JgyXwFkVin0bCW27i5i3zbtxZ7rnFBl91yMiosQBNd71NIt/fQmucoeO\nfw5pWpHxdO3pcoo5LLueflM2HgKW1bECggEBAIj4rVNc49wnZilSBylN+eyiY+QM\nFe9bVPSakFeCoEyWHi/+oIjHGnPOMUFfgfb9isLJcThi/uDNu9axa2kQwkTdy4GX\nx1m8q9xmzYsE69WC33rdQKL8+Hpk4x4NLBJVdVwHnIlJdmmPWFbCJ2OpIUg2AFay\nP4Hq4NgPvXyA70FnKIeZ8yb3M9lyUg4R9PpYDewKWVPXGleLyRxCKZWswrHPXpwy\ndolPmNnGWiod0lejccfy7hr/Ks0Xdaxm4HBkOhIeSyQT5MGCzzYom+YX0avipFM1\nWKeAErqwJwQpQbkqmQBTxgBv9FdhZvU5v9AIE/gsxj+1CcQVehAeZ5csJQw=\n-----END RSA PRIVATE KEY-----\n"} 2024-09-10T23:56:23.426 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout SSL certificate updated 2024-09-10T23:56:23.669 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:56:23 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-10T23:56:23.669 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:56:23 smithi012 ceph-mon[30333]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "dashboard set-ssl-certificate", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T23:56:23.669 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:56:23 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-10T23:56:25.205 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:56:25 smithi012 ceph-mon[30333]: from='client.14166 -' entity='client.admin' cmd=[{"prefix": "dashboard set-ssl-certificate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T23:56:25.206 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:56:25 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-10T23:56:25.210 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout SSL certificate key updated 2024-09-10T23:56:25.210 INFO:teuthology.orchestra.run.smithi012.stdout:Creating initial admin user... 2024-09-10T23:56:26.812 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$4cQwDdZSKfFJoxIGnQqW3e8rhu6wwCJwTTQkpkwy5VIivGBmuHzFK", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1726012586, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-09-10T23:56:26.812 INFO:teuthology.orchestra.run.smithi012.stdout:Fetching dashboard port number... 2024-09-10T23:56:27.489 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:56:27 smithi012 ceph-mon[30333]: from='client.14168 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T23:56:27.489 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:56:27 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-10T23:56:28.100 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout 8443 2024-09-10T23:56:28.118 INFO:teuthology.orchestra.run.smithi012.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-10T23:56:28.118 INFO:teuthology.orchestra.run.smithi012.stdout:systemctl: stdout disabled 2024-09-10T23:56:28.131 INFO:teuthology.orchestra.run.smithi012.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-10T23:56:28.131 INFO:teuthology.orchestra.run.smithi012.stdout:systemctl: stdout inactive 2024-09-10T23:56:28.131 INFO:teuthology.orchestra.run.smithi012.stdout:firewalld.service is not enabled 2024-09-10T23:56:28.131 INFO:teuthology.orchestra.run.smithi012.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-09-10T23:56:28.133 INFO:teuthology.orchestra.run.smithi012.stdout:Ceph Dashboard is now available at: 2024-09-10T23:56:28.133 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-10T23:56:28.133 INFO:teuthology.orchestra.run.smithi012.stdout: URL: https://smithi012.front.sepia.ceph.com:8443/ 2024-09-10T23:56:28.134 INFO:teuthology.orchestra.run.smithi012.stdout: User: admin 2024-09-10T23:56:28.134 INFO:teuthology.orchestra.run.smithi012.stdout: Password: 7jat0tlpn4 2024-09-10T23:56:28.134 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-10T23:56:28.134 INFO:teuthology.orchestra.run.smithi012.stdout:Saving cluster configuration to /var/lib/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/config directory 2024-09-10T23:56:28.396 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:56:28 smithi012 ceph-mon[30333]: from='client.? 172.21.15.12:0/4235908634' entity='client.admin' cmd={"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} : dispatch 2024-09-10T23:56:30.025 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-09-10T23:56:30.025 INFO:teuthology.orchestra.run.smithi012.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-09-10T23:56:30.025 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-10T23:56:30.025 INFO:teuthology.orchestra.run.smithi012.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid ff3ad74a-6fcf-11ef-bcea-c7b262605968 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-09-10T23:56:30.026 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-10T23:56:30.026 INFO:teuthology.orchestra.run.smithi012.stdout:Or, if you are only running a single cluster on this host: 2024-09-10T23:56:30.026 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-10T23:56:30.026 INFO:teuthology.orchestra.run.smithi012.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-09-10T23:56:30.026 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-10T23:56:30.026 INFO:teuthology.orchestra.run.smithi012.stdout:Please consider enabling telemetry to help improve Ceph: 2024-09-10T23:56:30.026 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-10T23:56:30.026 INFO:teuthology.orchestra.run.smithi012.stdout: ceph telemetry on 2024-09-10T23:56:30.026 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-10T23:56:30.026 INFO:teuthology.orchestra.run.smithi012.stdout:For more information see: 2024-09-10T23:56:30.026 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-10T23:56:30.026 INFO:teuthology.orchestra.run.smithi012.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2024-09-10T23:56:30.026 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-10T23:56:30.027 INFO:teuthology.orchestra.run.smithi012.stdout:Bootstrap complete. 2024-09-10T23:56:30.087 INFO:tasks.cephadm:Fetching config... 2024-09-10T23:56:30.087 DEBUG:teuthology.orchestra.run.smithi012:> set -ex 2024-09-10T23:56:30.088 DEBUG:teuthology.orchestra.run.smithi012:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-09-10T23:56:30.123 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-09-10T23:56:30.124 DEBUG:teuthology.orchestra.run.smithi012:> set -ex 2024-09-10T23:56:30.124 DEBUG:teuthology.orchestra.run.smithi012:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-09-10T23:56:30.202 INFO:tasks.cephadm:Fetching mon keyring... 2024-09-10T23:56:30.202 DEBUG:teuthology.orchestra.run.smithi012:> set -ex 2024-09-10T23:56:30.202 DEBUG:teuthology.orchestra.run.smithi012:> sudo dd if=/var/lib/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/mon.a/keyring of=/dev/stdout 2024-09-10T23:56:30.282 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:56:30 smithi012 ceph-mon[30333]: from='client.? 172.21.15.12:0/2033014689' entity='client.admin' 2024-09-10T23:56:30.300 INFO:tasks.cephadm:Fetching pub ssh key... 2024-09-10T23:56:30.300 DEBUG:teuthology.orchestra.run.smithi012:> set -ex 2024-09-10T23:56:30.301 DEBUG:teuthology.orchestra.run.smithi012:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-09-10T23:56:30.376 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-09-10T23:56:30.376 DEBUG:teuthology.orchestra.run.smithi012:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCbvjVzhDtfoV2p8UgOiWADhAz0pJBdJ69V1saVJETlY7TvbfPCGtv4Yjx8BXSfaRpB/WiTriP6oPY1GrXD7SRr/MPhVO1HSygajA1/uouG4J0nHae/ZFPMNNOR7NJVfF3GNwFjhr5zFyzq6vvr5aTGnCOCZdkj3/ym9gm/4mZ70IvHUTqzgMRDshVRfIObO6p/TAufBLmbp4oVy5F2PYDEEVYY3tB6179osKnq+Pc3didzK6qKAD/csL2sSWQJt1GkimHUiJjfCqmgWYbkFfLstA+BQuUwDluJ127bV/Pxz/HUgEwlf3X8KTkwbmlgptBnrcxTKI188BqLLw7Uiy6F4FX35h1CVE9UH1syk+uVoyw2NZ0ed65FpuOcUHsMw0n+Xh8Pakr/H8kMKhiVUQeDZFYOOo1NV1FhT3lylr2VOJarlujQONjBJozIAqGr36l2nxLCKgB+ndwzRp1OYRoU6uonFcBjLbbYIRRzKiMKXlBl/38lm/21G43MOqAMMI0= ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-09-10T23:56:30.496 INFO:teuthology.orchestra.run.smithi012.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCbvjVzhDtfoV2p8UgOiWADhAz0pJBdJ69V1saVJETlY7TvbfPCGtv4Yjx8BXSfaRpB/WiTriP6oPY1GrXD7SRr/MPhVO1HSygajA1/uouG4J0nHae/ZFPMNNOR7NJVfF3GNwFjhr5zFyzq6vvr5aTGnCOCZdkj3/ym9gm/4mZ70IvHUTqzgMRDshVRfIObO6p/TAufBLmbp4oVy5F2PYDEEVYY3tB6179osKnq+Pc3didzK6qKAD/csL2sSWQJt1GkimHUiJjfCqmgWYbkFfLstA+BQuUwDluJ127bV/Pxz/HUgEwlf3X8KTkwbmlgptBnrcxTKI188BqLLw7Uiy6F4FX35h1CVE9UH1syk+uVoyw2NZ0ed65FpuOcUHsMw0n+Xh8Pakr/H8kMKhiVUQeDZFYOOo1NV1FhT3lylr2VOJarlujQONjBJozIAqGr36l2nxLCKgB+ndwzRp1OYRoU6uonFcBjLbbYIRRzKiMKXlBl/38lm/21G43MOqAMMI0= ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968 2024-09-10T23:56:30.521 DEBUG:teuthology.orchestra.run.smithi165:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCbvjVzhDtfoV2p8UgOiWADhAz0pJBdJ69V1saVJETlY7TvbfPCGtv4Yjx8BXSfaRpB/WiTriP6oPY1GrXD7SRr/MPhVO1HSygajA1/uouG4J0nHae/ZFPMNNOR7NJVfF3GNwFjhr5zFyzq6vvr5aTGnCOCZdkj3/ym9gm/4mZ70IvHUTqzgMRDshVRfIObO6p/TAufBLmbp4oVy5F2PYDEEVYY3tB6179osKnq+Pc3didzK6qKAD/csL2sSWQJt1GkimHUiJjfCqmgWYbkFfLstA+BQuUwDluJ127bV/Pxz/HUgEwlf3X8KTkwbmlgptBnrcxTKI188BqLLw7Uiy6F4FX35h1CVE9UH1syk+uVoyw2NZ0ed65FpuOcUHsMw0n+Xh8Pakr/H8kMKhiVUQeDZFYOOo1NV1FhT3lylr2VOJarlujQONjBJozIAqGr36l2nxLCKgB+ndwzRp1OYRoU6uonFcBjLbbYIRRzKiMKXlBl/38lm/21G43MOqAMMI0= ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-09-10T23:56:30.562 INFO:teuthology.orchestra.run.smithi165.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCbvjVzhDtfoV2p8UgOiWADhAz0pJBdJ69V1saVJETlY7TvbfPCGtv4Yjx8BXSfaRpB/WiTriP6oPY1GrXD7SRr/MPhVO1HSygajA1/uouG4J0nHae/ZFPMNNOR7NJVfF3GNwFjhr5zFyzq6vvr5aTGnCOCZdkj3/ym9gm/4mZ70IvHUTqzgMRDshVRfIObO6p/TAufBLmbp4oVy5F2PYDEEVYY3tB6179osKnq+Pc3didzK6qKAD/csL2sSWQJt1GkimHUiJjfCqmgWYbkFfLstA+BQuUwDluJ127bV/Pxz/HUgEwlf3X8KTkwbmlgptBnrcxTKI188BqLLw7Uiy6F4FX35h1CVE9UH1syk+uVoyw2NZ0ed65FpuOcUHsMw0n+Xh8Pakr/H8kMKhiVUQeDZFYOOo1NV1FhT3lylr2VOJarlujQONjBJozIAqGr36l2nxLCKgB+ndwzRp1OYRoU6uonFcBjLbbYIRRzKiMKXlBl/38lm/21G43MOqAMMI0= ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968 2024-09-10T23:56:30.575 DEBUG:teuthology.orchestra.run.smithi178:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCbvjVzhDtfoV2p8UgOiWADhAz0pJBdJ69V1saVJETlY7TvbfPCGtv4Yjx8BXSfaRpB/WiTriP6oPY1GrXD7SRr/MPhVO1HSygajA1/uouG4J0nHae/ZFPMNNOR7NJVfF3GNwFjhr5zFyzq6vvr5aTGnCOCZdkj3/ym9gm/4mZ70IvHUTqzgMRDshVRfIObO6p/TAufBLmbp4oVy5F2PYDEEVYY3tB6179osKnq+Pc3didzK6qKAD/csL2sSWQJt1GkimHUiJjfCqmgWYbkFfLstA+BQuUwDluJ127bV/Pxz/HUgEwlf3X8KTkwbmlgptBnrcxTKI188BqLLw7Uiy6F4FX35h1CVE9UH1syk+uVoyw2NZ0ed65FpuOcUHsMw0n+Xh8Pakr/H8kMKhiVUQeDZFYOOo1NV1FhT3lylr2VOJarlujQONjBJozIAqGr36l2nxLCKgB+ndwzRp1OYRoU6uonFcBjLbbYIRRzKiMKXlBl/38lm/21G43MOqAMMI0= ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-09-10T23:56:30.616 INFO:teuthology.orchestra.run.smithi178.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCbvjVzhDtfoV2p8UgOiWADhAz0pJBdJ69V1saVJETlY7TvbfPCGtv4Yjx8BXSfaRpB/WiTriP6oPY1GrXD7SRr/MPhVO1HSygajA1/uouG4J0nHae/ZFPMNNOR7NJVfF3GNwFjhr5zFyzq6vvr5aTGnCOCZdkj3/ym9gm/4mZ70IvHUTqzgMRDshVRfIObO6p/TAufBLmbp4oVy5F2PYDEEVYY3tB6179osKnq+Pc3didzK6qKAD/csL2sSWQJt1GkimHUiJjfCqmgWYbkFfLstA+BQuUwDluJ127bV/Pxz/HUgEwlf3X8KTkwbmlgptBnrcxTKI188BqLLw7Uiy6F4FX35h1CVE9UH1syk+uVoyw2NZ0ed65FpuOcUHsMw0n+Xh8Pakr/H8kMKhiVUQeDZFYOOo1NV1FhT3lylr2VOJarlujQONjBJozIAqGr36l2nxLCKgB+ndwzRp1OYRoU6uonFcBjLbbYIRRzKiMKXlBl/38lm/21G43MOqAMMI0= ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968 2024-09-10T23:56:30.629 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ff3ad74a-6fcf-11ef-bcea-c7b262605968 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-09-10T23:56:31.015 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T23:56:31.433 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:56:31 smithi012 ceph-mon[30333]: mgrmap e13: x(active, since 12s) 2024-09-10T23:56:32.625 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-09-10T23:56:32.625 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ff3ad74a-6fcf-11ef-bcea-c7b262605968 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-09-10T23:56:33.063 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:56:32 smithi012 ceph-mon[30333]: from='client.? 172.21.15.12:0/2443824289' entity='client.admin' 2024-09-10T23:56:33.232 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T23:56:34.816 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi165 2024-09-10T23:56:34.816 DEBUG:teuthology.orchestra.run.smithi165:> set -ex 2024-09-10T23:56:34.816 DEBUG:teuthology.orchestra.run.smithi165:> dd of=/etc/ceph/ceph.conf 2024-09-10T23:56:34.836 DEBUG:teuthology.orchestra.run.smithi165:> set -ex 2024-09-10T23:56:34.836 DEBUG:teuthology.orchestra.run.smithi165:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-10T23:56:34.894 INFO:tasks.cephadm:Adding host smithi165 to orchestrator... 2024-09-10T23:56:34.895 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ff3ad74a-6fcf-11ef-bcea-c7b262605968 -- ceph orch host add smithi165 2024-09-10T23:56:35.229 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:56:35 smithi012 ceph-mon[30333]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T23:56:35.229 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:56:35 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-10T23:56:35.700 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T23:56:37.062 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:56:36 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-10T23:56:37.063 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:56:36 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-10T23:56:37.063 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:56:36 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd/host:smithi012", "name": "osd_memory_target"} : dispatch 2024-09-10T23:56:37.063 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:56:36 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T23:56:37.063 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:56:36 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T23:56:37.063 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:56:36 smithi012 ceph-mon[30333]: Updating smithi012:/etc/ceph/ceph.conf 2024-09-10T23:56:37.938 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:56:37 smithi012 ceph-mon[30333]: Updating smithi012:/var/lib/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T23:56:37.938 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:56:37 smithi012 ceph-mon[30333]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi165", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T23:56:37.938 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:56:37 smithi012 ceph-mon[30333]: Updating smithi012:/etc/ceph/ceph.client.admin.keyring 2024-09-10T23:56:39.063 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:56:38 smithi012 ceph-mon[30333]: Updating smithi012:/var/lib/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-10T23:56:39.063 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:56:38 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-10T23:56:39.063 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:56:38 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-10T23:56:39.063 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:56:38 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-10T23:56:39.063 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:56:38 smithi012 ceph-mon[30333]: Deploying cephadm binary to smithi165 2024-09-10T23:56:39.063 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:56:38 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T23:56:39.063 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:56:38 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T23:56:39.063 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:56:38 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T23:56:39.063 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:56:38 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-10T23:56:40.062 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:56:39 smithi012 ceph-mon[30333]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T23:56:40.419 INFO:teuthology.orchestra.run.smithi012.stdout:Added host 'smithi165' with addr '172.21.15.165' 2024-09-10T23:56:41.037 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ff3ad74a-6fcf-11ef-bcea-c7b262605968 -- ceph orch host ls --format=json 2024-09-10T23:56:41.405 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T23:56:41.675 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:56:41 smithi012 ceph-mon[30333]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T23:56:41.675 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:56:41 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-10T23:56:41.675 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:56:41 smithi012 ceph-mon[30333]: Added host smithi165 2024-09-10T23:56:41.675 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:56:41 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T23:56:41.675 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:56:41 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-10T23:56:41.676 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:56:41 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-10T23:56:42.091 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-10T23:56:42.092 INFO:teuthology.orchestra.run.smithi012.stdout:[{"addr": "172.21.15.12", "hostname": "smithi012", "labels": [], "status": ""}, {"addr": "172.21.15.165", "hostname": "smithi165", "labels": [], "status": ""}] 2024-09-10T23:56:42.765 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi178 2024-09-10T23:56:42.765 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2024-09-10T23:56:42.765 DEBUG:teuthology.orchestra.run.smithi178:> dd of=/etc/ceph/ceph.conf 2024-09-10T23:56:42.785 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2024-09-10T23:56:42.785 DEBUG:teuthology.orchestra.run.smithi178:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-10T23:56:42.843 INFO:tasks.cephadm:Adding host smithi178 to orchestrator... 2024-09-10T23:56:42.844 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ff3ad74a-6fcf-11ef-bcea-c7b262605968 -- ceph orch host add smithi178 2024-09-10T23:56:43.062 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:56:42 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-10T23:56:43.210 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T23:56:44.062 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:56:43 smithi012 ceph-mon[30333]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T23:56:44.063 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:56:43 smithi012 ceph-mon[30333]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T23:56:45.813 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:56:45 smithi012 ceph-mon[30333]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi178", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T23:56:45.813 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:56:45 smithi012 ceph-mon[30333]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T23:56:45.813 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:56:45 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-10T23:56:45.813 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:56:45 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-10T23:56:45.813 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:56:45 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-10T23:56:45.813 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:56:45 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-10T23:56:45.813 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:56:45 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd/host:smithi165", "name": "osd_memory_target"} : dispatch 2024-09-10T23:56:45.813 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:56:45 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T23:56:45.813 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:56:45 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T23:56:45.814 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:56:45 smithi012 ceph-mon[30333]: Updating smithi165:/etc/ceph/ceph.conf 2024-09-10T23:56:45.814 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:56:45 smithi012 ceph-mon[30333]: Updating smithi165:/var/lib/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T23:56:45.814 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:56:45 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-10T23:56:45.814 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:56:45 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-10T23:56:45.814 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:56:45 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-10T23:56:46.813 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:56:46 smithi012 ceph-mon[30333]: Updating smithi165:/etc/ceph/ceph.client.admin.keyring 2024-09-10T23:56:46.813 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:56:46 smithi012 ceph-mon[30333]: Deploying cephadm binary to smithi178 2024-09-10T23:56:46.813 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:56:46 smithi012 ceph-mon[30333]: Updating smithi165:/var/lib/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-10T23:56:47.812 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:56:47 smithi012 ceph-mon[30333]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T23:56:47.839 INFO:teuthology.orchestra.run.smithi012.stdout:Added host 'smithi178' with addr '172.21.15.178' 2024-09-10T23:56:48.503 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ff3ad74a-6fcf-11ef-bcea-c7b262605968 -- ceph orch host ls --format=json 2024-09-10T23:56:48.871 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T23:56:49.145 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:56:48 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-10T23:56:49.145 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:56:48 smithi012 ceph-mon[30333]: Added host smithi178 2024-09-10T23:56:49.145 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:56:48 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T23:56:49.145 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:56:48 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-10T23:56:49.145 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:56:48 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-10T23:56:49.589 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-10T23:56:49.589 INFO:teuthology.orchestra.run.smithi012.stdout:[{"addr": "172.21.15.12", "hostname": "smithi012", "labels": [], "status": ""}, {"addr": "172.21.15.165", "hostname": "smithi165", "labels": [], "status": ""}, {"addr": "172.21.15.178", "hostname": "smithi178", "labels": [], "status": ""}] 2024-09-10T23:56:50.153 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:56:49 smithi012 ceph-mon[30333]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T23:56:50.153 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:56:49 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-10T23:56:50.217 INFO:tasks.cephadm:Setting crush tunables to default 2024-09-10T23:56:50.218 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ff3ad74a-6fcf-11ef-bcea-c7b262605968 -- ceph osd crush tunables default 2024-09-10T23:56:50.582 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T23:56:51.034 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:56:50 smithi012 ceph-mon[30333]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T23:56:51.850 INFO:teuthology.orchestra.run.smithi012.stderr:adjusted tunables profile to default 2024-09-10T23:56:52.113 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:56:51 smithi012 ceph-mon[30333]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T23:56:52.113 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:56:51 smithi012 ceph-mon[30333]: from='client.? 172.21.15.12:0/361548279' entity='client.admin' cmd={"prefix": "osd crush tunables", "profile": "default"} : dispatch 2024-09-10T23:56:52.468 INFO:tasks.cephadm:Adding mon.a on smithi012 2024-09-10T23:56:52.468 INFO:tasks.cephadm:Adding mon.b on smithi165 2024-09-10T23:56:52.468 INFO:tasks.cephadm:Adding mon.c on smithi178 2024-09-10T23:56:52.468 DEBUG:teuthology.orchestra.run.smithi178:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ff3ad74a-6fcf-11ef-bcea-c7b262605968 -- ceph orch apply mon '3;smithi012:172.21.15.12=a;smithi165:172.21.15.165=b;smithi178:172.21.15.178=c' 2024-09-10T23:56:52.713 INFO:teuthology.orchestra.run.smithi178.stderr:Inferring config /var/lib/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T23:56:53.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:56:52 smithi012 ceph-mon[30333]: from='client.? 172.21.15.12:0/361548279' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-09-10T23:56:53.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:56:52 smithi012 ceph-mon[30333]: osdmap e4: 0 total, 0 up, 0 in 2024-09-10T23:56:53.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:56:52 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-10T23:56:53.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:56:52 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-10T23:56:53.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:56:52 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-10T23:56:53.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:56:52 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-10T23:56:53.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:56:52 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd/host:smithi178", "name": "osd_memory_target"} : dispatch 2024-09-10T23:56:53.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:56:52 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T23:56:53.314 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:56:52 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T23:56:53.314 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:56:52 smithi012 ceph-mon[30333]: Updating smithi178:/etc/ceph/ceph.conf 2024-09-10T23:56:53.314 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:56:52 smithi012 ceph-mon[30333]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T23:56:53.314 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:56:52 smithi012 ceph-mon[30333]: Updating smithi178:/var/lib/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T23:56:53.314 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:56:52 smithi012 ceph-mon[30333]: Updating smithi178:/etc/ceph/ceph.client.admin.keyring 2024-09-10T23:56:53.314 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:56:52 smithi012 ceph-mon[30333]: Updating smithi178:/var/lib/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-10T23:56:53.314 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:56:52 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-10T23:56:53.314 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:56:52 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-10T23:56:53.314 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:56:52 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-10T23:56:53.377 INFO:teuthology.orchestra.run.smithi178.stdout:Scheduled mon update... 2024-09-10T23:56:53.909 DEBUG:teuthology.orchestra.run.smithi165:mon.b> sudo journalctl -f -n 0 -u ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968@mon.b.service 2024-09-10T23:56:53.912 DEBUG:teuthology.orchestra.run.smithi178:mon.c> sudo journalctl -f -n 0 -u ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968@mon.c.service 2024-09-10T23:56:53.914 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-10T23:56:53.915 DEBUG:teuthology.orchestra.run.smithi178:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ff3ad74a-6fcf-11ef-bcea-c7b262605968 -- ceph mon dump -f json 2024-09-10T23:56:54.813 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:56:54 smithi012 ceph-mon[30333]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi012:172.21.15.12=a;smithi165:172.21.15.165=b;smithi178:172.21.15.178=c", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T23:56:54.813 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:56:54 smithi012 ceph-mon[30333]: Saving service mon spec with placement smithi012:172.21.15.12=a;smithi165:172.21.15.165=b;smithi178:172.21.15.178=c;count:3 2024-09-10T23:56:54.813 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:56:54 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-10T23:56:54.813 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:56:54 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T23:56:54.813 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:56:54 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T23:56:54.813 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:56:54 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T23:56:54.813 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:56:54 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-10T23:56:54.813 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:56:54 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-10T23:56:54.813 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:56:54 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T23:56:54.813 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:56:54 smithi012 ceph-mon[30333]: Deploying daemon mon.c on smithi178 2024-09-10T23:56:55.114 INFO:teuthology.orchestra.run.smithi178.stderr:Inferring config /var/lib/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T23:56:55.813 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:56:55 smithi012 ceph-mon[30333]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T23:56:56.767 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-09-10T23:56:56.768 INFO:teuthology.orchestra.run.smithi178.stdout:{"epoch":1,"fsid":"ff3ad74a-6fcf-11ef-bcea-c7b262605968","modified":"2024-09-10T23:55:12.644680Z","created":"2024-09-10T23:55:12.644680Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:3300","nonce":0},{"type":"v1","addr":"172.21.15.12:6789","nonce":0}]},"addr":"172.21.15.12:6789/0","public_addr":"172.21.15.12:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-09-10T23:56:56.768 INFO:teuthology.orchestra.run.smithi178.stderr:dumped monmap epoch 1 2024-09-10T23:56:57.685 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:56:57 smithi178 systemd[1]: Starting Ceph mon.c for ff3ad74a-6fcf-11ef-bcea-c7b262605968... 2024-09-10T23:56:57.813 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:56:57 smithi012 ceph-mon[30333]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T23:56:57.813 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:56:57 smithi012 ceph-mon[30333]: from='client.? 172.21.15.178:0/730918050' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-09-10T23:56:57.985 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:56:57 smithi178 podman[35901]: 2024-09-10 23:56:57.892906181 +0000 UTC m=+0.112930947 container create 23ccd19ad543b365a550274fa9c35d98232a7fed1e1f94e1bf2a1838729ae31e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-mon-c, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-09-10T23:56:57.985 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:56:57 smithi178 podman[35901]: 2024-09-10 23:56:57.800558153 +0000 UTC m=+0.020582931 image pull 589dd18248e494ba411dc838aa4ae0e5e569213f08fcd7a2cf22cd3ef7e3c480 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e 2024-09-10T23:56:58.581 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:56:58 smithi178 podman[35901]: 2024-09-10 23:56:58.096162599 +0000 UTC m=+0.316187367 container init 23ccd19ad543b365a550274fa9c35d98232a7fed1e1f94e1bf2a1838729ae31e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-mon-c, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.license=GPLv2, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-10T23:56:58.581 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:56:58 smithi178 podman[35901]: 2024-09-10 23:56:58.099425296 +0000 UTC m=+0.319450058 container start 23ccd19ad543b365a550274fa9c35d98232a7fed1e1f94e1bf2a1838729ae31e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-mon-c, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0) 2024-09-10T23:56:58.581 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:56:58 smithi178 ceph-mon[35919]: set uid:gid to 167:167 (ceph:ceph) 2024-09-10T23:56:58.581 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:56:58 smithi178 ceph-mon[35919]: ceph version 19.3.0-4854-g8f219c47 (8f219c476cb4f26fc5719c661c0323719a45cb30) squid (dev), process ceph-mon, pid 2 2024-09-10T23:56:58.581 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:56:58 smithi178 ceph-mon[35919]: pidfile_write: ignore empty --pid-file 2024-09-10T23:56:58.581 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:56:58 smithi178 ceph-mon[35919]: load: jerasure load: lrc 2024-09-10T23:56:58.581 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:56:58 smithi178 ceph-mon[35919]: rocksdb: RocksDB version: 7.9.2 2024-09-10T23:56:58.582 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:56:58 smithi178 ceph-mon[35919]: rocksdb: Git sha 0 2024-09-10T23:56:58.582 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:56:58 smithi178 ceph-mon[35919]: rocksdb: Compile date 2024-09-10 08:02:51 2024-09-10T23:56:58.582 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:56:58 smithi178 ceph-mon[35919]: rocksdb: DB SUMMARY 2024-09-10T23:56:58.582 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:56:58 smithi178 ceph-mon[35919]: rocksdb: DB Session ID: 8JH8ZAMGT3ZT5AJRIMUO 2024-09-10T23:56:58.582 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:56:58 smithi178 ceph-mon[35919]: rocksdb: CURRENT file: CURRENT 2024-09-10T23:56:58.582 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:56:58 smithi178 ceph-mon[35919]: rocksdb: IDENTITY file: IDENTITY 2024-09-10T23:56:58.582 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:56:58 smithi178 ceph-mon[35919]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2024-09-10T23:56:58.582 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:56:58 smithi178 ceph-mon[35919]: rocksdb: SST files in /var/lib/ceph/mon/ceph-c/store.db dir, Total Num: 0, files: 2024-09-10T23:56:58.582 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:56:58 smithi178 ceph-mon[35919]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-c/store.db: 000004.log size: 511 ; 2024-09-10T23:56:58.582 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:56:58 smithi178 ceph-mon[35919]: rocksdb: Options.error_if_exists: 0 2024-09-10T23:56:58.582 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:56:58 smithi178 ceph-mon[35919]: rocksdb: Options.create_if_missing: 0 2024-09-10T23:56:58.582 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:56:58 smithi178 ceph-mon[35919]: rocksdb: Options.paranoid_checks: 1 2024-09-10T23:56:58.582 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:56:58 smithi178 ceph-mon[35919]: rocksdb: Options.flush_verify_memtable_count: 1 2024-09-10T23:56:58.583 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:56:58 smithi178 ceph-mon[35919]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-09-10T23:56:58.583 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:56:58 smithi178 ceph-mon[35919]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-09-10T23:56:58.583 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:56:58 smithi178 ceph-mon[35919]: rocksdb: Options.env: 0x556cb86bf180 2024-09-10T23:56:58.583 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:56:58 smithi178 ceph-mon[35919]: rocksdb: Options.fs: PosixFileSystem 2024-09-10T23:56:58.583 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:56:58 smithi178 ceph-mon[35919]: rocksdb: Options.info_log: 0x556cba3e3700 2024-09-10T23:56:58.583 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:56:58 smithi178 ceph-mon[35919]: rocksdb: Options.max_file_opening_threads: 16 2024-09-10T23:56:58.584 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:56:58 smithi178 ceph-mon[35919]: rocksdb: Options.statistics: (nil) 2024-09-10T23:56:58.585 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:56:58 smithi178 ceph-mon[35919]: rocksdb: Options.use_fsync: 0 2024-09-10T23:56:58.585 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:56:58 smithi178 ceph-mon[35919]: rocksdb: Options.max_log_file_size: 0 2024-09-10T23:56:58.585 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:56:58 smithi178 ceph-mon[35919]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-09-10T23:56:58.585 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:56:58 smithi178 ceph-mon[35919]: rocksdb: Options.log_file_time_to_roll: 0 2024-09-10T23:56:58.585 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:56:58 smithi178 ceph-mon[35919]: rocksdb: Options.keep_log_file_num: 1000 2024-09-10T23:56:58.585 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:56:58 smithi178 ceph-mon[35919]: rocksdb: Options.recycle_log_file_num: 0 2024-09-10T23:56:58.585 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:56:58 smithi178 ceph-mon[35919]: rocksdb: Options.allow_fallocate: 1 2024-09-10T23:56:58.585 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:56:58 smithi178 ceph-mon[35919]: rocksdb: Options.allow_mmap_reads: 0 2024-09-10T23:56:58.585 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:56:58 smithi178 ceph-mon[35919]: rocksdb: Options.allow_mmap_writes: 0 2024-09-10T23:56:58.585 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:56:58 smithi178 ceph-mon[35919]: rocksdb: Options.use_direct_reads: 0 2024-09-10T23:56:58.585 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:56:58 smithi178 ceph-mon[35919]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-09-10T23:56:58.586 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:56:58 smithi178 ceph-mon[35919]: rocksdb: Options.create_missing_column_families: 0 2024-09-10T23:56:58.586 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:56:58 smithi178 ceph-mon[35919]: rocksdb: Options.db_log_dir: 2024-09-10T23:56:58.586 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:56:58 smithi178 ceph-mon[35919]: rocksdb: Options.wal_dir: 2024-09-10T23:56:58.586 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:56:58 smithi178 ceph-mon[35919]: rocksdb: Options.table_cache_numshardbits: 6 2024-09-10T23:56:58.586 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:56:58 smithi178 ceph-mon[35919]: rocksdb: Options.WAL_ttl_seconds: 0 2024-09-10T23:56:58.586 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:56:58 smithi178 ceph-mon[35919]: rocksdb: Options.WAL_size_limit_MB: 0 2024-09-10T23:56:58.586 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:56:58 smithi178 ceph-mon[35919]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-09-10T23:56:58.586 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:56:58 smithi178 ceph-mon[35919]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-09-10T23:56:58.586 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:56:58 smithi178 ceph-mon[35919]: rocksdb: Options.is_fd_close_on_exec: 1 2024-09-10T23:56:58.586 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:56:58 smithi178 ceph-mon[35919]: rocksdb: Options.advise_random_on_open: 1 2024-09-10T23:56:58.586 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:56:58 smithi178 ceph-mon[35919]: rocksdb: Options.db_write_buffer_size: 0 2024-09-10T23:56:58.586 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:56:58 smithi178 ceph-mon[35919]: rocksdb: Options.write_buffer_manager: 0x556cba3e7b80 2024-09-10T23:56:58.586 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:56:58 smithi178 ceph-mon[35919]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-09-10T23:56:58.587 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:56:58 smithi178 ceph-mon[35919]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-09-10T23:56:58.587 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:56:58 smithi178 ceph-mon[35919]: rocksdb: Options.use_adaptive_mutex: 0 2024-09-10T23:56:58.587 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:56:58 smithi178 ceph-mon[35919]: rocksdb: Options.rate_limiter: (nil) 2024-09-10T23:56:58.587 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:56:58 smithi178 ceph-mon[35919]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-09-10T23:56:58.587 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:56:58 smithi178 ceph-mon[35919]: rocksdb: Options.wal_recovery_mode: 2 2024-09-10T23:56:58.587 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:56:58 smithi178 ceph-mon[35919]: rocksdb: Options.enable_thread_tracking: 0 2024-09-10T23:56:58.587 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:56:58 smithi178 ceph-mon[35919]: rocksdb: Options.enable_pipelined_write: 0 2024-09-10T23:56:58.587 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:56:58 smithi178 ceph-mon[35919]: rocksdb: Options.unordered_write: 0 2024-09-10T23:56:58.587 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:56:58 smithi178 ceph-mon[35919]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-09-10T23:56:58.587 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:56:58 smithi178 ceph-mon[35919]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-09-10T23:56:58.587 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:56:58 smithi178 ceph-mon[35919]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-09-10T23:56:58.587 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:56:58 smithi178 ceph-mon[35919]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-09-10T23:56:58.588 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:56:58 smithi178 ceph-mon[35919]: rocksdb: Options.row_cache: None 2024-09-10T23:56:58.588 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:56:58 smithi178 ceph-mon[35919]: rocksdb: Options.wal_filter: None 2024-09-10T23:56:58.588 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:56:58 smithi178 ceph-mon[35919]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-09-10T23:56:58.588 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:56:58 smithi178 ceph-mon[35919]: rocksdb: Options.allow_ingest_behind: 0 2024-09-10T23:56:58.588 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:56:58 smithi178 ceph-mon[35919]: rocksdb: Options.two_write_queues: 0 2024-09-10T23:56:58.588 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:56:58 smithi178 ceph-mon[35919]: rocksdb: Options.manual_wal_flush: 0 2024-09-10T23:56:58.588 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:56:58 smithi178 ceph-mon[35919]: rocksdb: Options.wal_compression: 0 2024-09-10T23:56:58.588 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:56:58 smithi178 ceph-mon[35919]: rocksdb: Options.atomic_flush: 0 2024-09-10T23:56:58.588 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:56:58 smithi178 ceph-mon[35919]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-09-10T23:56:58.588 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:56:58 smithi178 ceph-mon[35919]: rocksdb: Options.persist_stats_to_disk: 0 2024-09-10T23:56:58.588 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:56:58 smithi178 ceph-mon[35919]: rocksdb: Options.write_dbid_to_manifest: 0 2024-09-10T23:56:58.588 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:56:58 smithi178 ceph-mon[35919]: rocksdb: Options.log_readahead_size: 0 2024-09-10T23:56:58.589 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:56:58 smithi178 ceph-mon[35919]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-09-10T23:56:58.589 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:56:58 smithi178 ceph-mon[35919]: rocksdb: Options.best_efforts_recovery: 0 2024-09-10T23:56:58.589 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:56:58 smithi178 ceph-mon[35919]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-09-10T23:56:58.589 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:56:58 smithi178 ceph-mon[35919]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-09-10T23:56:58.589 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:56:58 smithi178 ceph-mon[35919]: rocksdb: Options.allow_data_in_errors: 0 2024-09-10T23:56:58.589 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:56:58 smithi178 ceph-mon[35919]: rocksdb: Options.db_host_id: __hostname__ 2024-09-10T23:56:58.589 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:56:58 smithi178 ceph-mon[35919]: rocksdb: Options.enforce_single_del_contracts: true 2024-09-10T23:56:58.589 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:56:58 smithi178 ceph-mon[35919]: rocksdb: Options.max_background_jobs: 2 2024-09-10T23:56:58.589 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:56:58 smithi178 ceph-mon[35919]: rocksdb: Options.max_background_compactions: -1 2024-09-10T23:56:58.589 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:56:58 smithi178 ceph-mon[35919]: rocksdb: Options.max_subcompactions: 1 2024-09-10T23:56:58.589 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:56:58 smithi178 ceph-mon[35919]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-09-10T23:56:58.589 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:56:58 smithi178 ceph-mon[35919]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-09-10T23:56:58.589 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:56:58 smithi178 ceph-mon[35919]: rocksdb: Options.delayed_write_rate : 16777216 2024-09-10T23:56:58.590 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:56:58 smithi178 ceph-mon[35919]: rocksdb: Options.max_total_wal_size: 0 2024-09-10T23:56:58.590 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:56:58 smithi178 ceph-mon[35919]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-09-10T23:56:58.590 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:56:58 smithi178 ceph-mon[35919]: rocksdb: Options.stats_dump_period_sec: 600 2024-09-10T23:56:58.590 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:56:58 smithi178 ceph-mon[35919]: rocksdb: Options.stats_persist_period_sec: 600 2024-09-10T23:56:58.590 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:56:58 smithi178 ceph-mon[35919]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-09-10T23:56:58.590 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:56:58 smithi178 ceph-mon[35919]: rocksdb: Options.max_open_files: -1 2024-09-10T23:56:58.590 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:56:58 smithi178 ceph-mon[35919]: rocksdb: Options.bytes_per_sync: 0 2024-09-10T23:56:58.590 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:56:58 smithi178 ceph-mon[35919]: rocksdb: Options.wal_bytes_per_sync: 0 2024-09-10T23:56:58.590 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:56:58 smithi178 ceph-mon[35919]: rocksdb: Options.strict_bytes_per_sync: 0 2024-09-10T23:56:58.590 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:56:58 smithi178 ceph-mon[35919]: rocksdb: Options.compaction_readahead_size: 0 2024-09-10T23:56:58.590 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:56:58 smithi178 ceph-mon[35919]: rocksdb: Options.max_background_flushes: -1 2024-09-10T23:56:58.590 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:56:58 smithi178 ceph-mon[35919]: rocksdb: Compression algorithms supported: 2024-09-10T23:56:58.590 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:56:58 smithi178 ceph-mon[35919]: rocksdb: kZSTD supported: 0 2024-09-10T23:56:58.590 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:56:58 smithi178 ceph-mon[35919]: rocksdb: kXpressCompression supported: 0 2024-09-10T23:56:58.591 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:56:58 smithi178 ceph-mon[35919]: rocksdb: kBZip2Compression supported: 0 2024-09-10T23:56:58.591 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:56:58 smithi178 ceph-mon[35919]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-09-10T23:56:58.591 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:56:58 smithi178 ceph-mon[35919]: rocksdb: kLZ4Compression supported: 1 2024-09-10T23:56:58.591 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:56:58 smithi178 ceph-mon[35919]: rocksdb: kZlibCompression supported: 1 2024-09-10T23:56:58.591 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:56:58 smithi178 ceph-mon[35919]: rocksdb: kLZ4HCCompression supported: 1 2024-09-10T23:56:58.591 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:56:58 smithi178 ceph-mon[35919]: rocksdb: kSnappyCompression supported: 1 2024-09-10T23:56:58.591 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:56:58 smithi178 ceph-mon[35919]: rocksdb: Fast CRC32 supported: Supported on x86 2024-09-10T23:56:58.591 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:56:58 smithi178 ceph-mon[35919]: rocksdb: DMutex implementation: pthread_mutex_t 2024-09-10T23:56:58.591 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:56:58 smithi178 ceph-mon[35919]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 2024-09-10T23:56:58.591 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:56:58 smithi178 ceph-mon[35919]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-09-10T23:56:58.591 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:56:58 smithi178 ceph-mon[35919]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-09-10T23:56:58.591 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:56:58 smithi178 ceph-mon[35919]: rocksdb: Options.merge_operator: 2024-09-10T23:56:58.591 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:56:58 smithi178 ceph-mon[35919]: rocksdb: Options.compaction_filter: None 2024-09-10T23:56:58.592 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:56:58 smithi178 ceph-mon[35919]: rocksdb: Options.compaction_filter_factory: None 2024-09-10T23:56:58.592 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:56:58 smithi178 ceph-mon[35919]: rocksdb: Options.sst_partitioner_factory: None 2024-09-10T23:56:58.592 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:56:58 smithi178 ceph-mon[35919]: rocksdb: Options.memtable_factory: SkipListFactory 2024-09-10T23:56:58.592 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:56:58 smithi178 ceph-mon[35919]: rocksdb: Options.table_factory: BlockBasedTable 2024-09-10T23:56:58.592 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:56:58 smithi178 ceph-mon[35919]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x556cba3e3320) 2024-09-10T23:56:58.592 INFO:journalctl@ceph.mon.c.smithi178.stdout: cache_index_and_filter_blocks: 1 2024-09-10T23:56:58.592 INFO:journalctl@ceph.mon.c.smithi178.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-09-10T23:56:58.592 INFO:journalctl@ceph.mon.c.smithi178.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-09-10T23:56:58.592 INFO:journalctl@ceph.mon.c.smithi178.stdout: pin_top_level_index_and_filter: 1 2024-09-10T23:56:58.592 INFO:journalctl@ceph.mon.c.smithi178.stdout: index_type: 0 2024-09-10T23:56:58.592 INFO:journalctl@ceph.mon.c.smithi178.stdout: data_block_index_type: 0 2024-09-10T23:56:58.592 INFO:journalctl@ceph.mon.c.smithi178.stdout: index_shortening: 1 2024-09-10T23:56:58.592 INFO:journalctl@ceph.mon.c.smithi178.stdout: data_block_hash_table_util_ratio: 0.750000 2024-09-10T23:56:58.593 INFO:journalctl@ceph.mon.c.smithi178.stdout: checksum: 4 2024-09-10T23:56:58.593 INFO:journalctl@ceph.mon.c.smithi178.stdout: no_block_cache: 0 2024-09-10T23:56:58.593 INFO:journalctl@ceph.mon.c.smithi178.stdout: block_cache: 0x556cba3d7610 2024-09-10T23:56:58.593 INFO:journalctl@ceph.mon.c.smithi178.stdout: block_cache_name: BinnedLRUCache 2024-09-10T23:56:58.593 INFO:journalctl@ceph.mon.c.smithi178.stdout: block_cache_options: 2024-09-10T23:56:58.593 INFO:journalctl@ceph.mon.c.smithi178.stdout: capacity : 536870912 2024-09-10T23:56:58.593 INFO:journalctl@ceph.mon.c.smithi178.stdout: num_shard_bits : 4 2024-09-10T23:56:58.593 INFO:journalctl@ceph.mon.c.smithi178.stdout: strict_capacity_limit : 0 2024-09-10T23:56:58.593 INFO:journalctl@ceph.mon.c.smithi178.stdout: high_pri_pool_ratio: 0.000 2024-09-10T23:56:58.593 INFO:journalctl@ceph.mon.c.smithi178.stdout: block_cache_compressed: (nil) 2024-09-10T23:56:58.593 INFO:journalctl@ceph.mon.c.smithi178.stdout: persistent_cache: (nil) 2024-09-10T23:56:58.593 INFO:journalctl@ceph.mon.c.smithi178.stdout: block_size: 4096 2024-09-10T23:56:58.594 INFO:journalctl@ceph.mon.c.smithi178.stdout: block_size_deviation: 10 2024-09-10T23:56:58.594 INFO:journalctl@ceph.mon.c.smithi178.stdout: block_restart_interval: 16 2024-09-10T23:56:58.594 INFO:journalctl@ceph.mon.c.smithi178.stdout: index_block_restart_interval: 1 2024-09-10T23:56:58.594 INFO:journalctl@ceph.mon.c.smithi178.stdout: metadata_block_size: 4096 2024-09-10T23:56:58.594 INFO:journalctl@ceph.mon.c.smithi178.stdout: partition_filters: 0 2024-09-10T23:56:58.594 INFO:journalctl@ceph.mon.c.smithi178.stdout: use_delta_encoding: 1 2024-09-10T23:56:58.594 INFO:journalctl@ceph.mon.c.smithi178.stdout: filter_policy: bloomfilter 2024-09-10T23:56:58.594 INFO:journalctl@ceph.mon.c.smithi178.stdout: whole_key_filtering: 1 2024-09-10T23:56:58.594 INFO:journalctl@ceph.mon.c.smithi178.stdout: verify_compression: 0 2024-09-10T23:56:58.594 INFO:journalctl@ceph.mon.c.smithi178.stdout: read_amp_bytes_per_bit: 0 2024-09-10T23:56:58.594 INFO:journalctl@ceph.mon.c.smithi178.stdout: format_version: 5 2024-09-10T23:56:58.594 INFO:journalctl@ceph.mon.c.smithi178.stdout: enable_index_compression: 1 2024-09-10T23:56:58.594 INFO:journalctl@ceph.mon.c.smithi178.stdout: block_align: 0 2024-09-10T23:56:58.594 INFO:journalctl@ceph.mon.c.smithi178.stdout: max_auto_readahead_size: 262144 2024-09-10T23:56:58.595 INFO:journalctl@ceph.mon.c.smithi178.stdout: prepopulate_block_cache: 0 2024-09-10T23:56:58.595 INFO:journalctl@ceph.mon.c.smithi178.stdout: initial_auto_readahead_size: 8192 2024-09-10T23:56:58.595 INFO:journalctl@ceph.mon.c.smithi178.stdout: num_file_reads_for_auto_readahead: 2 2024-09-10T23:56:58.595 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:56:58 smithi178 ceph-mon[35919]: rocksdb: Options.write_buffer_size: 33554432 2024-09-10T23:56:58.595 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:56:58 smithi178 ceph-mon[35919]: rocksdb: Options.max_write_buffer_number: 2 2024-09-10T23:56:58.595 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:56:58 smithi178 ceph-mon[35919]: rocksdb: Options.compression: NoCompression 2024-09-10T23:56:58.595 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:56:58 smithi178 ceph-mon[35919]: rocksdb: Options.bottommost_compression: Disabled 2024-09-10T23:56:58.595 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:56:58 smithi178 ceph-mon[35919]: rocksdb: Options.prefix_extractor: nullptr 2024-09-10T23:56:58.595 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:56:58 smithi178 ceph-mon[35919]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-09-10T23:56:58.595 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:56:58 smithi178 ceph-mon[35919]: rocksdb: Options.num_levels: 7 2024-09-10T23:56:58.595 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:56:58 smithi178 ceph-mon[35919]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-09-10T23:56:58.596 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:56:58 smithi178 ceph-mon[35919]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-09-10T23:56:58.596 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:56:58 smithi178 ceph-mon[35919]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-09-10T23:56:58.596 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:56:58 smithi178 ceph-mon[35919]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-09-10T23:56:58.596 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:56:58 smithi178 ceph-mon[35919]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-09-10T23:56:58.596 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:56:58 smithi178 ceph-mon[35919]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-09-10T23:56:58.596 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:56:58 smithi178 ceph-mon[35919]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-09-10T23:56:58.596 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:56:58 smithi178 ceph-mon[35919]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-09-10T23:56:58.596 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:56:58 smithi178 ceph-mon[35919]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-09-10T23:56:58.596 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:56:58 smithi178 ceph-mon[35919]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-09-10T23:56:58.596 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:56:58 smithi178 ceph-mon[35919]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-09-10T23:56:58.596 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:56:58 smithi178 ceph-mon[35919]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-09-10T23:56:58.597 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:56:58 smithi178 ceph-mon[35919]: rocksdb: Options.compression_opts.window_bits: -14 2024-09-10T23:56:58.597 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:56:58 smithi178 ceph-mon[35919]: rocksdb: Options.compression_opts.level: 32767 2024-09-10T23:56:58.597 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:56:58 smithi178 ceph-mon[35919]: rocksdb: Options.compression_opts.strategy: 0 2024-09-10T23:56:58.597 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:56:58 smithi178 ceph-mon[35919]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-09-10T23:56:58.597 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:56:58 smithi178 ceph-mon[35919]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-09-10T23:56:58.597 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:56:58 smithi178 ceph-mon[35919]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-09-10T23:56:58.597 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:56:58 smithi178 ceph-mon[35919]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-09-10T23:56:58.597 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:56:58 smithi178 ceph-mon[35919]: rocksdb: Options.compression_opts.enabled: false 2024-09-10T23:56:58.597 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:56:58 smithi178 ceph-mon[35919]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-09-10T23:56:58.597 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:56:58 smithi178 ceph-mon[35919]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-09-10T23:56:58.597 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:56:58 smithi178 ceph-mon[35919]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-09-10T23:56:58.597 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:56:58 smithi178 ceph-mon[35919]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-09-10T23:56:58.597 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:56:58 smithi178 ceph-mon[35919]: rocksdb: Options.target_file_size_base: 67108864 2024-09-10T23:56:58.598 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:56:58 smithi178 ceph-mon[35919]: rocksdb: Options.target_file_size_multiplier: 1 2024-09-10T23:56:58.598 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:56:58 smithi178 ceph-mon[35919]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-09-10T23:56:58.598 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:56:58 smithi178 ceph-mon[35919]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-09-10T23:56:58.598 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:56:58 smithi178 ceph-mon[35919]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-09-10T23:56:58.598 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:56:58 smithi178 ceph-mon[35919]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-09-10T23:56:58.598 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:56:58 smithi178 ceph-mon[35919]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-09-10T23:56:58.598 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:56:58 smithi178 ceph-mon[35919]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-09-10T23:56:58.598 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:56:58 smithi178 ceph-mon[35919]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-09-10T23:56:58.598 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:56:58 smithi178 ceph-mon[35919]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-09-10T23:56:58.598 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:56:58 smithi178 ceph-mon[35919]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-09-10T23:56:58.598 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:56:58 smithi178 ceph-mon[35919]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-09-10T23:56:58.598 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:56:58 smithi178 ceph-mon[35919]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-09-10T23:56:58.598 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:56:58 smithi178 ceph-mon[35919]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-09-10T23:56:58.599 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:56:58 smithi178 ceph-mon[35919]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-09-10T23:56:58.599 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:56:58 smithi178 ceph-mon[35919]: rocksdb: Options.arena_block_size: 1048576 2024-09-10T23:56:58.599 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:56:58 smithi178 ceph-mon[35919]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-09-10T23:56:58.599 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:56:58 smithi178 ceph-mon[35919]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-09-10T23:56:58.599 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:56:58 smithi178 ceph-mon[35919]: rocksdb: Options.disable_auto_compactions: 0 2024-09-10T23:56:58.599 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:56:58 smithi178 ceph-mon[35919]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-09-10T23:56:58.599 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:56:58 smithi178 ceph-mon[35919]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-09-10T23:56:58.599 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:56:58 smithi178 ceph-mon[35919]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-09-10T23:56:58.599 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:56:58 smithi178 ceph-mon[35919]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-09-10T23:56:58.599 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:56:58 smithi178 ceph-mon[35919]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-09-10T23:56:58.599 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:56:58 smithi178 ceph-mon[35919]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-09-10T23:56:58.599 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:56:58 smithi178 ceph-mon[35919]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-09-10T23:56:58.599 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:56:58 smithi178 ceph-mon[35919]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-09-10T23:56:58.600 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:56:58 smithi178 ceph-mon[35919]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-09-10T23:56:58.600 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:56:58 smithi178 ceph-mon[35919]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-09-10T23:56:58.600 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:56:58 smithi178 ceph-mon[35919]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-09-10T23:56:58.600 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:56:58 smithi178 ceph-mon[35919]: rocksdb: Options.inplace_update_support: 0 2024-09-10T23:56:58.600 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:56:58 smithi178 ceph-mon[35919]: rocksdb: Options.inplace_update_num_locks: 10000 2024-09-10T23:56:58.600 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:56:58 smithi178 ceph-mon[35919]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-09-10T23:56:58.600 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:56:58 smithi178 ceph-mon[35919]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-09-10T23:56:58.600 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:56:58 smithi178 ceph-mon[35919]: rocksdb: Options.memtable_huge_page_size: 0 2024-09-10T23:56:58.600 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:56:58 smithi178 ceph-mon[35919]: rocksdb: Options.bloom_locality: 0 2024-09-10T23:56:58.600 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:56:58 smithi178 ceph-mon[35919]: rocksdb: Options.max_successive_merges: 0 2024-09-10T23:56:58.600 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:56:58 smithi178 ceph-mon[35919]: rocksdb: Options.optimize_filters_for_hits: 0 2024-09-10T23:56:58.600 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:56:58 smithi178 ceph-mon[35919]: rocksdb: Options.paranoid_file_checks: 0 2024-09-10T23:56:58.600 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:56:58 smithi178 ceph-mon[35919]: rocksdb: Options.force_consistency_checks: 1 2024-09-10T23:56:58.600 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:56:58 smithi178 ceph-mon[35919]: rocksdb: Options.report_bg_io_stats: 0 2024-09-10T23:56:58.601 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:56:58 smithi178 ceph-mon[35919]: rocksdb: Options.ttl: 2592000 2024-09-10T23:56:58.601 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:56:58 smithi178 ceph-mon[35919]: rocksdb: Options.periodic_compaction_seconds: 0 2024-09-10T23:56:58.601 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:56:58 smithi178 ceph-mon[35919]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-09-10T23:56:58.601 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:56:58 smithi178 ceph-mon[35919]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-09-10T23:56:58.601 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:56:58 smithi178 ceph-mon[35919]: rocksdb: Options.enable_blob_files: false 2024-09-10T23:56:58.601 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:56:58 smithi178 ceph-mon[35919]: rocksdb: Options.min_blob_size: 0 2024-09-10T23:56:58.601 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:56:58 smithi178 ceph-mon[35919]: rocksdb: Options.blob_file_size: 268435456 2024-09-10T23:56:58.601 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:56:58 smithi178 ceph-mon[35919]: rocksdb: Options.blob_compression_type: NoCompression 2024-09-10T23:56:58.601 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:56:58 smithi178 ceph-mon[35919]: rocksdb: Options.enable_blob_garbage_collection: false 2024-09-10T23:56:58.601 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:56:58 smithi178 ceph-mon[35919]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-09-10T23:56:58.601 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:56:58 smithi178 ceph-mon[35919]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-09-10T23:56:58.601 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:56:58 smithi178 ceph-mon[35919]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-09-10T23:56:58.601 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:56:58 smithi178 ceph-mon[35919]: rocksdb: Options.blob_file_starting_level: 0 2024-09-10T23:56:58.602 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:56:58 smithi178 ceph-mon[35919]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-09-10T23:56:58.602 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:56:58 smithi178 ceph-mon[35919]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-09-10T23:56:58.602 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:56:58 smithi178 ceph-mon[35919]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2024-09-10T23:56:58.602 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:56:58 smithi178 ceph-mon[35919]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 4e3e47a0-710e-423a-8612-6980d7a10647 2024-09-10T23:56:58.602 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:56:58 smithi178 ceph-mon[35919]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726012618126466, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-09-10T23:56:58.602 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:56:58 smithi178 ceph-mon[35919]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2024-09-10T23:56:58.602 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:56:58 smithi178 ceph-mon[35919]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726012618126863, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1648, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 523, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 401, "raw_average_value_size": 80, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1726012618, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "4e3e47a0-710e-423a-8612-6980d7a10647", "db_session_id": "8JH8ZAMGT3ZT5AJRIMUO", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2024-09-10T23:56:58.602 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:56:58 smithi178 ceph-mon[35919]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726012618126955, "job": 1, "event": "recovery_finished"} 2024-09-10T23:56:58.602 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:56:58 smithi178 ceph-mon[35919]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2024-09-10T23:56:58.602 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:56:58 smithi178 ceph-mon[35919]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-c/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-09-10T23:56:58.602 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:56:58 smithi178 ceph-mon[35919]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x556cba406e00 2024-09-10T23:56:58.602 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:56:58 smithi178 ceph-mon[35919]: rocksdb: DB pointer 0x556cba508000 2024-09-10T23:56:58.602 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:56:58 smithi178 ceph-mon[35919]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-09-10T23:56:58.603 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:56:58 smithi178 ceph-mon[35919]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-09-10T23:56:58.603 INFO:journalctl@ceph.mon.c.smithi178.stdout: ** DB Stats ** 2024-09-10T23:56:58.603 INFO:journalctl@ceph.mon.c.smithi178.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-10T23:56:58.603 INFO:journalctl@ceph.mon.c.smithi178.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-09-10T23:56:58.603 INFO:journalctl@ceph.mon.c.smithi178.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-10T23:56:58.603 INFO:journalctl@ceph.mon.c.smithi178.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-10T23:56:58.603 INFO:journalctl@ceph.mon.c.smithi178.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-09-10T23:56:58.603 INFO:journalctl@ceph.mon.c.smithi178.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-10T23:56:58.603 INFO:journalctl@ceph.mon.c.smithi178.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-10T23:56:58.603 INFO:journalctl@ceph.mon.c.smithi178.stdout: 2024-09-10T23:56:58.603 INFO:journalctl@ceph.mon.c.smithi178.stdout: ** Compaction Stats [default] ** 2024-09-10T23:56:58.603 INFO:journalctl@ceph.mon.c.smithi178.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-10T23:56:58.603 INFO:journalctl@ceph.mon.c.smithi178.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-09-10T23:56:58.604 INFO:journalctl@ceph.mon.c.smithi178.stdout: L0 1/0 1.61 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.1 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-09-10T23:56:58.604 INFO:journalctl@ceph.mon.c.smithi178.stdout: Sum 1/0 1.61 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.1 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-09-10T23:56:58.604 INFO:journalctl@ceph.mon.c.smithi178.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.1 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-09-10T23:56:58.604 INFO:journalctl@ceph.mon.c.smithi178.stdout: 2024-09-10T23:56:58.604 INFO:journalctl@ceph.mon.c.smithi178.stdout: ** Compaction Stats [default] ** 2024-09-10T23:56:58.604 INFO:journalctl@ceph.mon.c.smithi178.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-10T23:56:58.604 INFO:journalctl@ceph.mon.c.smithi178.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-10T23:56:58.604 INFO:journalctl@ceph.mon.c.smithi178.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 4.1 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-09-10T23:56:58.604 INFO:journalctl@ceph.mon.c.smithi178.stdout: 2024-09-10T23:56:58.604 INFO:journalctl@ceph.mon.c.smithi178.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-09-10T23:56:58.604 INFO:journalctl@ceph.mon.c.smithi178.stdout: 2024-09-10T23:56:58.604 INFO:journalctl@ceph.mon.c.smithi178.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-10T23:56:58.604 INFO:journalctl@ceph.mon.c.smithi178.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-09-10T23:56:58.605 INFO:journalctl@ceph.mon.c.smithi178.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-09-10T23:56:58.605 INFO:journalctl@ceph.mon.c.smithi178.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-09-10T23:56:58.605 INFO:journalctl@ceph.mon.c.smithi178.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-09-10T23:56:58.605 INFO:journalctl@ceph.mon.c.smithi178.stdout: AddFile(Keys): cumulative 0, interval 0 2024-09-10T23:56:58.605 INFO:journalctl@ceph.mon.c.smithi178.stdout: Cumulative compaction: 0.00 GB write, 0.63 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-10T23:56:58.605 INFO:journalctl@ceph.mon.c.smithi178.stdout: Interval compaction: 0.00 GB write, 0.63 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-10T23:56:58.605 INFO:journalctl@ceph.mon.c.smithi178.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-09-10T23:56:58.605 INFO:journalctl@ceph.mon.c.smithi178.stdout: Block cache BinnedLRUCache@0x556cba3d7610#2 capacity: 512.00 MB usage: 0.22 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 9e-06 secs_since: 0 2024-09-10T23:56:58.605 INFO:journalctl@ceph.mon.c.smithi178.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2024-09-10T23:56:58.605 INFO:journalctl@ceph.mon.c.smithi178.stdout: 2024-09-10T23:56:58.605 INFO:journalctl@ceph.mon.c.smithi178.stdout: ** File Read Latency Histogram By Level [default] ** 2024-09-10T23:56:58.605 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:56:58 smithi178 ceph-mon[35919]: mon.c does not exist in monmap, will attempt to join an existing cluster 2024-09-10T23:56:58.605 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:56:58 smithi178 ceph-mon[35919]: using public_addr v2:172.21.15.178:0/0 -> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] 2024-09-10T23:56:58.606 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:56:58 smithi178 ceph-mon[35919]: starting mon.c rank -1 at public addrs [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] at bind addrs [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon_data /var/lib/ceph/mon/ceph-c fsid ff3ad74a-6fcf-11ef-bcea-c7b262605968 2024-09-10T23:56:58.606 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:56:58 smithi178 ceph-mon[35919]: mon.c@-1(???) e0 preinit fsid ff3ad74a-6fcf-11ef-bcea-c7b262605968 2024-09-10T23:56:58.606 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:56:58 smithi178 ceph-mon[35919]: mon.c@-1(synchronizing).mds e1 new map 2024-09-10T23:56:58.606 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:56:58 smithi178 ceph-mon[35919]: mon.c@-1(synchronizing).mds e1 print_map 2024-09-10T23:56:58.606 INFO:journalctl@ceph.mon.c.smithi178.stdout: e1 2024-09-10T23:56:58.606 INFO:journalctl@ceph.mon.c.smithi178.stdout: btime 2024-09-10T23:55:16:317386+0000 2024-09-10T23:56:58.606 INFO:journalctl@ceph.mon.c.smithi178.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-09-10T23:56:58.606 INFO:journalctl@ceph.mon.c.smithi178.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-10T23:56:58.606 INFO:journalctl@ceph.mon.c.smithi178.stdout: legacy client fscid: -1 2024-09-10T23:56:58.606 INFO:journalctl@ceph.mon.c.smithi178.stdout: 2024-09-10T23:56:58.606 INFO:journalctl@ceph.mon.c.smithi178.stdout: No filesystems configured 2024-09-10T23:56:58.606 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:56:58 smithi178 ceph-mon[35919]: mon.c@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-09-10T23:56:58.606 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:56:58 smithi178 ceph-mon[35919]: mon.c@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-09-10T23:56:58.607 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:56:58 smithi178 ceph-mon[35919]: mon.c@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-09-10T23:56:58.607 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:56:58 smithi178 ceph-mon[35919]: mon.c@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-09-10T23:56:58.607 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:56:58 smithi178 ceph-mon[35919]: mon.c@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-09-10T23:56:58.607 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:56:58 smithi178 ceph-mon[35919]: mon.c@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-09-10T23:56:58.607 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:56:58 smithi178 ceph-mon[35919]: mon.c@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-09-10T23:56:58.607 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:56:58 smithi178 ceph-mon[35919]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-10T23:56:58.607 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:56:58 smithi178 ceph-mon[35919]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-10T23:56:58.607 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:56:58 smithi178 ceph-mon[35919]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-10T23:56:58.607 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:56:58 smithi178 ceph-mon[35919]: Updating smithi165:/etc/ceph/ceph.client.admin.keyring 2024-09-10T23:56:58.607 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:56:58 smithi178 ceph-mon[35919]: Deploying cephadm binary to smithi178 2024-09-10T23:56:58.607 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:56:58 smithi178 ceph-mon[35919]: Updating smithi165:/var/lib/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-10T23:56:58.607 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:56:58 smithi178 ceph-mon[35919]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T23:56:58.607 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:56:58 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-10T23:56:58.608 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:56:58 smithi178 ceph-mon[35919]: Added host smithi178 2024-09-10T23:56:58.608 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:56:58 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T23:56:58.608 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:56:58 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-10T23:56:58.608 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:56:58 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-10T23:56:58.608 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:56:58 smithi178 ceph-mon[35919]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T23:56:58.608 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:56:58 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-10T23:56:58.608 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:56:58 smithi178 ceph-mon[35919]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T23:56:58.608 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:56:58 smithi178 ceph-mon[35919]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T23:56:58.608 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:56:58 smithi178 ceph-mon[35919]: from='client.? 172.21.15.12:0/361548279' entity='client.admin' cmd={"prefix": "osd crush tunables", "profile": "default"} : dispatch 2024-09-10T23:56:58.608 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:56:58 smithi178 ceph-mon[35919]: from='client.? 172.21.15.12:0/361548279' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-09-10T23:56:58.608 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:56:58 smithi178 ceph-mon[35919]: osdmap e4: 0 total, 0 up, 0 in 2024-09-10T23:56:58.608 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:56:58 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-10T23:56:58.608 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:56:58 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-10T23:56:58.609 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:56:58 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-10T23:56:58.609 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:56:58 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-10T23:56:58.609 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:56:58 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd/host:smithi178", "name": "osd_memory_target"} : dispatch 2024-09-10T23:56:58.609 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:56:58 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T23:56:58.609 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:56:58 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T23:56:58.609 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:56:58 smithi178 ceph-mon[35919]: Updating smithi178:/etc/ceph/ceph.conf 2024-09-10T23:56:58.609 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:56:58 smithi178 ceph-mon[35919]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T23:56:58.609 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:56:58 smithi178 ceph-mon[35919]: Updating smithi178:/var/lib/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T23:56:58.609 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:56:58 smithi178 ceph-mon[35919]: Updating smithi178:/etc/ceph/ceph.client.admin.keyring 2024-09-10T23:56:58.609 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:56:58 smithi178 ceph-mon[35919]: Updating smithi178:/var/lib/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-10T23:56:58.609 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:56:58 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-10T23:56:58.609 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:56:58 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-10T23:56:58.609 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:56:58 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-10T23:56:58.610 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:56:58 smithi178 ceph-mon[35919]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi012:172.21.15.12=a;smithi165:172.21.15.165=b;smithi178:172.21.15.178=c", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T23:56:58.610 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:56:58 smithi178 ceph-mon[35919]: Saving service mon spec with placement smithi012:172.21.15.12=a;smithi165:172.21.15.165=b;smithi178:172.21.15.178=c;count:3 2024-09-10T23:56:58.610 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:56:58 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-10T23:56:58.610 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:56:58 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T23:56:58.610 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:56:58 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T23:56:58.610 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:56:58 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T23:56:58.610 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:56:58 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-10T23:56:58.610 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:56:58 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-10T23:56:58.610 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:56:58 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T23:56:58.610 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:56:58 smithi178 ceph-mon[35919]: Deploying daemon mon.c on smithi178 2024-09-10T23:56:58.610 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:56:58 smithi178 ceph-mon[35919]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T23:56:58.610 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:56:58 smithi178 ceph-mon[35919]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T23:56:58.610 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:56:58 smithi178 ceph-mon[35919]: from='client.? 172.21.15.178:0/730918050' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-09-10T23:56:58.610 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:56:58 smithi178 ceph-mon[35919]: mon.c@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-09-10T23:56:58.611 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:56:58 smithi178 bash[35901]: 23ccd19ad543b365a550274fa9c35d98232a7fed1e1f94e1bf2a1838729ae31e 2024-09-10T23:56:58.611 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:56:58 smithi178 systemd[1]: Started Ceph mon.c for ff3ad74a-6fcf-11ef-bcea-c7b262605968. 2024-09-10T23:56:58.718 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-10T23:56:58.718 DEBUG:teuthology.orchestra.run.smithi178:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ff3ad74a-6fcf-11ef-bcea-c7b262605968 -- ceph mon dump -f json 2024-09-10T23:56:58.973 INFO:teuthology.orchestra.run.smithi178.stderr:Inferring config /var/lib/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/mon.c/config 2024-09-10T23:57:03.563 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:57:03 smithi012 ceph-mon[30333]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T23:57:03.563 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:57:03 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-10T23:57:03.564 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:57:03 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-10T23:57:03.564 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:57:03 smithi012 ceph-mon[30333]: mon.a calling monitor election 2024-09-10T23:57:03.564 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:57:03 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-10T23:57:03.564 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:57:03 smithi012 ceph-mon[30333]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T23:57:03.564 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:57:03 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-10T23:57:03.564 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:57:03 smithi012 ceph-mon[30333]: mon.c calling monitor election 2024-09-10T23:57:03.564 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:57:03 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-10T23:57:03.564 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:57:03 smithi012 ceph-mon[30333]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T23:57:03.564 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:57:03 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-10T23:57:03.564 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:57:03 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-10T23:57:03.564 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:57:03 smithi012 ceph-mon[30333]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-10T23:57:03.565 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:57:03 smithi012 ceph-mon[30333]: monmap epoch 2 2024-09-10T23:57:03.565 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:57:03 smithi012 ceph-mon[30333]: fsid ff3ad74a-6fcf-11ef-bcea-c7b262605968 2024-09-10T23:57:03.565 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:57:03 smithi012 ceph-mon[30333]: last_changed 2024-09-10T23:56:58.153268+0000 2024-09-10T23:57:03.565 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:57:03 smithi012 ceph-mon[30333]: created 2024-09-10T23:55:12.644680+0000 2024-09-10T23:57:03.565 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:57:03 smithi012 ceph-mon[30333]: min_mon_release 19 (squid) 2024-09-10T23:57:03.565 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:57:03 smithi012 ceph-mon[30333]: election_strategy: 1 2024-09-10T23:57:03.565 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:57:03 smithi012 ceph-mon[30333]: 0: [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] mon.a 2024-09-10T23:57:03.565 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:57:03 smithi012 ceph-mon[30333]: 1: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.c 2024-09-10T23:57:03.565 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:57:03 smithi012 ceph-mon[30333]: fsmap 2024-09-10T23:57:03.565 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:57:03 smithi012 ceph-mon[30333]: osdmap e4: 0 total, 0 up, 0 in 2024-09-10T23:57:03.565 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:57:03 smithi012 ceph-mon[30333]: mgrmap e13: x(active, since 45s) 2024-09-10T23:57:03.565 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:57:03 smithi012 ceph-mon[30333]: overall HEALTH_OK 2024-09-10T23:57:03.565 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:57:03 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-10T23:57:03.565 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:57:03 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-10T23:57:03.566 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:57:03 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-10T23:57:03.566 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:57:03 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-10T23:57:03.566 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:57:03 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T23:57:03.581 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:57:03 smithi178 ceph-mon[35919]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T23:57:03.581 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:57:03 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-10T23:57:03.581 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:57:03 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-10T23:57:03.581 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:57:03 smithi178 ceph-mon[35919]: mon.a calling monitor election 2024-09-10T23:57:03.581 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:57:03 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-10T23:57:03.581 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:57:03 smithi178 ceph-mon[35919]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T23:57:03.582 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:57:03 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-10T23:57:03.582 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:57:03 smithi178 ceph-mon[35919]: mon.c calling monitor election 2024-09-10T23:57:03.582 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:57:03 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-10T23:57:03.582 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:57:03 smithi178 ceph-mon[35919]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T23:57:03.582 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:57:03 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-10T23:57:03.582 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:57:03 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-10T23:57:03.582 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:57:03 smithi178 ceph-mon[35919]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-10T23:57:03.582 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:57:03 smithi178 ceph-mon[35919]: monmap epoch 2 2024-09-10T23:57:03.582 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:57:03 smithi178 ceph-mon[35919]: fsid ff3ad74a-6fcf-11ef-bcea-c7b262605968 2024-09-10T23:57:03.582 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:57:03 smithi178 ceph-mon[35919]: last_changed 2024-09-10T23:56:58.153268+0000 2024-09-10T23:57:03.582 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:57:03 smithi178 ceph-mon[35919]: created 2024-09-10T23:55:12.644680+0000 2024-09-10T23:57:03.582 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:57:03 smithi178 ceph-mon[35919]: min_mon_release 19 (squid) 2024-09-10T23:57:03.582 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:57:03 smithi178 ceph-mon[35919]: election_strategy: 1 2024-09-10T23:57:03.582 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:57:03 smithi178 ceph-mon[35919]: 0: [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] mon.a 2024-09-10T23:57:03.583 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:57:03 smithi178 ceph-mon[35919]: 1: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.c 2024-09-10T23:57:03.583 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:57:03 smithi178 ceph-mon[35919]: fsmap 2024-09-10T23:57:03.583 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:57:03 smithi178 ceph-mon[35919]: osdmap e4: 0 total, 0 up, 0 in 2024-09-10T23:57:03.583 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:57:03 smithi178 ceph-mon[35919]: mgrmap e13: x(active, since 45s) 2024-09-10T23:57:03.583 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:57:03 smithi178 ceph-mon[35919]: overall HEALTH_OK 2024-09-10T23:57:03.583 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:57:03 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-10T23:57:03.583 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:57:03 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-10T23:57:03.583 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:57:03 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-10T23:57:03.583 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:57:03 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-10T23:57:03.583 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:57:03 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T23:57:04.070 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-09-10T23:57:04.071 INFO:teuthology.orchestra.run.smithi178.stdout:{"epoch":2,"fsid":"ff3ad74a-6fcf-11ef-bcea-c7b262605968","modified":"2024-09-10T23:56:58.153268Z","created":"2024-09-10T23:55:12.644680Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:3300","nonce":0},{"type":"v1","addr":"172.21.15.12:6789","nonce":0}]},"addr":"172.21.15.12:6789/0","public_addr":"172.21.15.12:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:3300","nonce":0},{"type":"v1","addr":"172.21.15.178:6789","nonce":0}]},"addr":"172.21.15.178:6789/0","public_addr":"172.21.15.178:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-09-10T23:57:04.071 INFO:teuthology.orchestra.run.smithi178.stderr:dumped monmap epoch 2 2024-09-10T23:57:04.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:57:04 smithi178 ceph-mon[35919]: Deploying daemon mon.b on smithi165 2024-09-10T23:57:04.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:57:04 smithi178 ceph-mon[35919]: from='client.? 172.21.15.178:0/903600878' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-09-10T23:57:04.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:57:04 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-10T23:57:04.484 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:57:04 smithi012 ceph-mon[30333]: Deploying daemon mon.b on smithi165 2024-09-10T23:57:04.485 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:57:04 smithi012 ceph-mon[30333]: from='client.? 172.21.15.178:0/903600878' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-09-10T23:57:04.485 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:57:04 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-10T23:57:05.563 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:57:05 smithi012 ceph-mon[30333]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T23:57:05.563 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Sep 10 23:57:05 smithi012 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-mgr-x[30560]: 2024-09-10T23:57:05.152+0000 7fa00e600640 -1 mgr.server handle_report got status from non-daemon mon.c 2024-09-10T23:57:05.580 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:57:05 smithi178 ceph-mon[35919]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T23:57:05.615 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-10T23:57:05.615 DEBUG:teuthology.orchestra.run.smithi178:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ff3ad74a-6fcf-11ef-bcea-c7b262605968 -- ceph mon dump -f json 2024-09-10T23:57:05.869 INFO:teuthology.orchestra.run.smithi178.stderr:Inferring config /var/lib/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/mon.c/config 2024-09-10T23:57:06.521 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-09-10T23:57:06.521 INFO:teuthology.orchestra.run.smithi178.stdout:{"epoch":2,"fsid":"ff3ad74a-6fcf-11ef-bcea-c7b262605968","modified":"2024-09-10T23:56:58.153268Z","created":"2024-09-10T23:55:12.644680Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:3300","nonce":0},{"type":"v1","addr":"172.21.15.12:6789","nonce":0}]},"addr":"172.21.15.12:6789/0","public_addr":"172.21.15.12:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:3300","nonce":0},{"type":"v1","addr":"172.21.15.178:6789","nonce":0}]},"addr":"172.21.15.178:6789/0","public_addr":"172.21.15.178:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-09-10T23:57:06.522 INFO:teuthology.orchestra.run.smithi178.stderr:dumped monmap epoch 2 2024-09-10T23:57:06.782 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:06 smithi165 podman[35336]: 2024-09-10 23:57:06.752801013 +0000 UTC m=+0.119214067 container create 77449392964c91580fa9aa94f7de799550d86a3308e285bee7b6cc19999096b7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-mon-b, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=) 2024-09-10T23:57:07.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:06 smithi165 podman[35336]: 2024-09-10 23:57:06.955066618 +0000 UTC m=+0.321479686 container init 77449392964c91580fa9aa94f7de799550d86a3308e285bee7b6cc19999096b7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-mon-b, GIT_CLEAN=True, CEPH_POINT_RELEASE=, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-09-10T23:57:07.292 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:06 smithi165 podman[35336]: 2024-09-10 23:57:06.958495207 +0000 UTC m=+0.324908264 container start 77449392964c91580fa9aa94f7de799550d86a3308e285bee7b6cc19999096b7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-mon-b, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, ceph=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD) 2024-09-10T23:57:07.293 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:06 smithi165 ceph-mon[35356]: set uid:gid to 167:167 (ceph:ceph) 2024-09-10T23:57:07.293 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:06 smithi165 ceph-mon[35356]: ceph version 19.3.0-4854-g8f219c47 (8f219c476cb4f26fc5719c661c0323719a45cb30) squid (dev), process ceph-mon, pid 2 2024-09-10T23:57:07.293 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:06 smithi165 ceph-mon[35356]: pidfile_write: ignore empty --pid-file 2024-09-10T23:57:07.293 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:06 smithi165 ceph-mon[35356]: load: jerasure load: lrc 2024-09-10T23:57:07.293 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:06 smithi165 ceph-mon[35356]: rocksdb: RocksDB version: 7.9.2 2024-09-10T23:57:07.293 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:06 smithi165 ceph-mon[35356]: rocksdb: Git sha 0 2024-09-10T23:57:07.293 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:06 smithi165 ceph-mon[35356]: rocksdb: Compile date 2024-09-10 08:02:51 2024-09-10T23:57:07.293 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:06 smithi165 ceph-mon[35356]: rocksdb: DB SUMMARY 2024-09-10T23:57:07.293 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:06 smithi165 ceph-mon[35356]: rocksdb: DB Session ID: 082CDN2WSZDA70NBP9L2 2024-09-10T23:57:07.293 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:06 smithi165 ceph-mon[35356]: rocksdb: CURRENT file: CURRENT 2024-09-10T23:57:07.293 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:06 smithi165 ceph-mon[35356]: rocksdb: IDENTITY file: IDENTITY 2024-09-10T23:57:07.293 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:06 smithi165 ceph-mon[35356]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2024-09-10T23:57:07.293 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:06 smithi165 ceph-mon[35356]: rocksdb: SST files in /var/lib/ceph/mon/ceph-b/store.db dir, Total Num: 0, files: 2024-09-10T23:57:07.294 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:06 smithi165 ceph-mon[35356]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-b/store.db: 000004.log size: 636 ; 2024-09-10T23:57:07.294 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:06 smithi165 ceph-mon[35356]: rocksdb: Options.error_if_exists: 0 2024-09-10T23:57:07.294 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:06 smithi165 ceph-mon[35356]: rocksdb: Options.create_if_missing: 0 2024-09-10T23:57:07.294 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:06 smithi165 ceph-mon[35356]: rocksdb: Options.paranoid_checks: 1 2024-09-10T23:57:07.294 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:06 smithi165 ceph-mon[35356]: rocksdb: Options.flush_verify_memtable_count: 1 2024-09-10T23:57:07.294 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:06 smithi165 ceph-mon[35356]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-09-10T23:57:07.294 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:06 smithi165 ceph-mon[35356]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-09-10T23:57:07.294 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:06 smithi165 ceph-mon[35356]: rocksdb: Options.env: 0x556f7e7ed180 2024-09-10T23:57:07.294 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:06 smithi165 ceph-mon[35356]: rocksdb: Options.fs: PosixFileSystem 2024-09-10T23:57:07.294 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:06 smithi165 ceph-mon[35356]: rocksdb: Options.info_log: 0x556f805d7700 2024-09-10T23:57:07.294 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:06 smithi165 ceph-mon[35356]: rocksdb: Options.max_file_opening_threads: 16 2024-09-10T23:57:07.296 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:06 smithi165 ceph-mon[35356]: rocksdb: Options.statistics: (nil) 2024-09-10T23:57:07.296 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:06 smithi165 ceph-mon[35356]: rocksdb: Options.use_fsync: 0 2024-09-10T23:57:07.296 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:06 smithi165 ceph-mon[35356]: rocksdb: Options.max_log_file_size: 0 2024-09-10T23:57:07.296 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:06 smithi165 ceph-mon[35356]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-09-10T23:57:07.296 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:06 smithi165 ceph-mon[35356]: rocksdb: Options.log_file_time_to_roll: 0 2024-09-10T23:57:07.296 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:06 smithi165 ceph-mon[35356]: rocksdb: Options.keep_log_file_num: 1000 2024-09-10T23:57:07.296 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:06 smithi165 ceph-mon[35356]: rocksdb: Options.recycle_log_file_num: 0 2024-09-10T23:57:07.296 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:06 smithi165 ceph-mon[35356]: rocksdb: Options.allow_fallocate: 1 2024-09-10T23:57:07.296 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:06 smithi165 ceph-mon[35356]: rocksdb: Options.allow_mmap_reads: 0 2024-09-10T23:57:07.296 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:06 smithi165 ceph-mon[35356]: rocksdb: Options.allow_mmap_writes: 0 2024-09-10T23:57:07.297 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:06 smithi165 ceph-mon[35356]: rocksdb: Options.use_direct_reads: 0 2024-09-10T23:57:07.297 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:06 smithi165 ceph-mon[35356]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-09-10T23:57:07.297 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:06 smithi165 ceph-mon[35356]: rocksdb: Options.create_missing_column_families: 0 2024-09-10T23:57:07.297 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:06 smithi165 ceph-mon[35356]: rocksdb: Options.db_log_dir: 2024-09-10T23:57:07.297 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:06 smithi165 ceph-mon[35356]: rocksdb: Options.wal_dir: 2024-09-10T23:57:07.297 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:06 smithi165 ceph-mon[35356]: rocksdb: Options.table_cache_numshardbits: 6 2024-09-10T23:57:07.297 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:06 smithi165 ceph-mon[35356]: rocksdb: Options.WAL_ttl_seconds: 0 2024-09-10T23:57:07.297 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:06 smithi165 ceph-mon[35356]: rocksdb: Options.WAL_size_limit_MB: 0 2024-09-10T23:57:07.297 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:06 smithi165 ceph-mon[35356]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-09-10T23:57:07.297 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:06 smithi165 ceph-mon[35356]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-09-10T23:57:07.298 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:06 smithi165 ceph-mon[35356]: rocksdb: Options.is_fd_close_on_exec: 1 2024-09-10T23:57:07.298 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:06 smithi165 ceph-mon[35356]: rocksdb: Options.advise_random_on_open: 1 2024-09-10T23:57:07.298 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:06 smithi165 ceph-mon[35356]: rocksdb: Options.db_write_buffer_size: 0 2024-09-10T23:57:07.298 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:06 smithi165 ceph-mon[35356]: rocksdb: Options.write_buffer_manager: 0x556f805dbb80 2024-09-10T23:57:07.298 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:06 smithi165 ceph-mon[35356]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-09-10T23:57:07.298 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:06 smithi165 ceph-mon[35356]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-09-10T23:57:07.298 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:06 smithi165 ceph-mon[35356]: rocksdb: Options.use_adaptive_mutex: 0 2024-09-10T23:57:07.298 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:06 smithi165 ceph-mon[35356]: rocksdb: Options.rate_limiter: (nil) 2024-09-10T23:57:07.298 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:06 smithi165 ceph-mon[35356]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-09-10T23:57:07.298 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:06 smithi165 ceph-mon[35356]: rocksdb: Options.wal_recovery_mode: 2 2024-09-10T23:57:07.298 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:06 smithi165 ceph-mon[35356]: rocksdb: Options.enable_thread_tracking: 0 2024-09-10T23:57:07.298 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:06 smithi165 ceph-mon[35356]: rocksdb: Options.enable_pipelined_write: 0 2024-09-10T23:57:07.299 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:06 smithi165 ceph-mon[35356]: rocksdb: Options.unordered_write: 0 2024-09-10T23:57:07.299 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:06 smithi165 ceph-mon[35356]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-09-10T23:57:07.299 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:06 smithi165 ceph-mon[35356]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-09-10T23:57:07.299 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:06 smithi165 ceph-mon[35356]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-09-10T23:57:07.299 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:06 smithi165 ceph-mon[35356]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-09-10T23:57:07.299 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:06 smithi165 ceph-mon[35356]: rocksdb: Options.row_cache: None 2024-09-10T23:57:07.299 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:06 smithi165 ceph-mon[35356]: rocksdb: Options.wal_filter: None 2024-09-10T23:57:07.299 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:06 smithi165 ceph-mon[35356]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-09-10T23:57:07.299 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:06 smithi165 ceph-mon[35356]: rocksdb: Options.allow_ingest_behind: 0 2024-09-10T23:57:07.299 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:06 smithi165 ceph-mon[35356]: rocksdb: Options.two_write_queues: 0 2024-09-10T23:57:07.300 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:06 smithi165 ceph-mon[35356]: rocksdb: Options.manual_wal_flush: 0 2024-09-10T23:57:07.300 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:06 smithi165 ceph-mon[35356]: rocksdb: Options.wal_compression: 0 2024-09-10T23:57:07.300 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:06 smithi165 ceph-mon[35356]: rocksdb: Options.atomic_flush: 0 2024-09-10T23:57:07.300 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:06 smithi165 ceph-mon[35356]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-09-10T23:57:07.300 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:06 smithi165 ceph-mon[35356]: rocksdb: Options.persist_stats_to_disk: 0 2024-09-10T23:57:07.300 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:06 smithi165 ceph-mon[35356]: rocksdb: Options.write_dbid_to_manifest: 0 2024-09-10T23:57:07.300 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:06 smithi165 ceph-mon[35356]: rocksdb: Options.log_readahead_size: 0 2024-09-10T23:57:07.300 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:06 smithi165 ceph-mon[35356]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-09-10T23:57:07.300 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:06 smithi165 ceph-mon[35356]: rocksdb: Options.best_efforts_recovery: 0 2024-09-10T23:57:07.300 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:06 smithi165 ceph-mon[35356]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-09-10T23:57:07.300 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:06 smithi165 ceph-mon[35356]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-09-10T23:57:07.300 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:06 smithi165 ceph-mon[35356]: rocksdb: Options.allow_data_in_errors: 0 2024-09-10T23:57:07.300 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:06 smithi165 ceph-mon[35356]: rocksdb: Options.db_host_id: __hostname__ 2024-09-10T23:57:07.301 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:06 smithi165 ceph-mon[35356]: rocksdb: Options.enforce_single_del_contracts: true 2024-09-10T23:57:07.301 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:06 smithi165 ceph-mon[35356]: rocksdb: Options.max_background_jobs: 2 2024-09-10T23:57:07.301 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:06 smithi165 ceph-mon[35356]: rocksdb: Options.max_background_compactions: -1 2024-09-10T23:57:07.301 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:06 smithi165 ceph-mon[35356]: rocksdb: Options.max_subcompactions: 1 2024-09-10T23:57:07.301 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:06 smithi165 ceph-mon[35356]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-09-10T23:57:07.301 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:06 smithi165 ceph-mon[35356]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-09-10T23:57:07.301 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:06 smithi165 ceph-mon[35356]: rocksdb: Options.delayed_write_rate : 16777216 2024-09-10T23:57:07.301 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:06 smithi165 ceph-mon[35356]: rocksdb: Options.max_total_wal_size: 0 2024-09-10T23:57:07.301 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:06 smithi165 ceph-mon[35356]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-09-10T23:57:07.301 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:06 smithi165 ceph-mon[35356]: rocksdb: Options.stats_dump_period_sec: 600 2024-09-10T23:57:07.301 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:06 smithi165 ceph-mon[35356]: rocksdb: Options.stats_persist_period_sec: 600 2024-09-10T23:57:07.301 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:06 smithi165 ceph-mon[35356]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-09-10T23:57:07.301 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:06 smithi165 ceph-mon[35356]: rocksdb: Options.max_open_files: -1 2024-09-10T23:57:07.302 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:06 smithi165 ceph-mon[35356]: rocksdb: Options.bytes_per_sync: 0 2024-09-10T23:57:07.302 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:06 smithi165 ceph-mon[35356]: rocksdb: Options.wal_bytes_per_sync: 0 2024-09-10T23:57:07.302 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:06 smithi165 ceph-mon[35356]: rocksdb: Options.strict_bytes_per_sync: 0 2024-09-10T23:57:07.302 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:06 smithi165 ceph-mon[35356]: rocksdb: Options.compaction_readahead_size: 0 2024-09-10T23:57:07.302 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:06 smithi165 ceph-mon[35356]: rocksdb: Options.max_background_flushes: -1 2024-09-10T23:57:07.302 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:06 smithi165 ceph-mon[35356]: rocksdb: Compression algorithms supported: 2024-09-10T23:57:07.302 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:06 smithi165 ceph-mon[35356]: rocksdb: kZSTD supported: 0 2024-09-10T23:57:07.302 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:06 smithi165 ceph-mon[35356]: rocksdb: kXpressCompression supported: 0 2024-09-10T23:57:07.302 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:06 smithi165 ceph-mon[35356]: rocksdb: kBZip2Compression supported: 0 2024-09-10T23:57:07.302 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:06 smithi165 ceph-mon[35356]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-09-10T23:57:07.302 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:06 smithi165 ceph-mon[35356]: rocksdb: kLZ4Compression supported: 1 2024-09-10T23:57:07.302 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:06 smithi165 ceph-mon[35356]: rocksdb: kZlibCompression supported: 1 2024-09-10T23:57:07.302 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:06 smithi165 ceph-mon[35356]: rocksdb: kLZ4HCCompression supported: 1 2024-09-10T23:57:07.303 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:06 smithi165 ceph-mon[35356]: rocksdb: kSnappyCompression supported: 1 2024-09-10T23:57:07.303 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:06 smithi165 ceph-mon[35356]: rocksdb: Fast CRC32 supported: Supported on x86 2024-09-10T23:57:07.303 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:06 smithi165 ceph-mon[35356]: rocksdb: DMutex implementation: pthread_mutex_t 2024-09-10T23:57:07.303 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:06 smithi165 ceph-mon[35356]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 2024-09-10T23:57:07.303 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:06 smithi165 ceph-mon[35356]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-09-10T23:57:07.303 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:06 smithi165 ceph-mon[35356]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-09-10T23:57:07.303 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:06 smithi165 ceph-mon[35356]: rocksdb: Options.merge_operator: 2024-09-10T23:57:07.303 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:06 smithi165 ceph-mon[35356]: rocksdb: Options.compaction_filter: None 2024-09-10T23:57:07.303 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:06 smithi165 ceph-mon[35356]: rocksdb: Options.compaction_filter_factory: None 2024-09-10T23:57:07.303 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:06 smithi165 ceph-mon[35356]: rocksdb: Options.sst_partitioner_factory: None 2024-09-10T23:57:07.303 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:06 smithi165 ceph-mon[35356]: rocksdb: Options.memtable_factory: SkipListFactory 2024-09-10T23:57:07.303 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:06 smithi165 ceph-mon[35356]: rocksdb: Options.table_factory: BlockBasedTable 2024-09-10T23:57:07.303 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:06 smithi165 ceph-mon[35356]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x556f805d7320) 2024-09-10T23:57:07.303 INFO:journalctl@ceph.mon.b.smithi165.stdout: cache_index_and_filter_blocks: 1 2024-09-10T23:57:07.304 INFO:journalctl@ceph.mon.b.smithi165.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-09-10T23:57:07.304 INFO:journalctl@ceph.mon.b.smithi165.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-09-10T23:57:07.304 INFO:journalctl@ceph.mon.b.smithi165.stdout: pin_top_level_index_and_filter: 1 2024-09-10T23:57:07.304 INFO:journalctl@ceph.mon.b.smithi165.stdout: index_type: 0 2024-09-10T23:57:07.304 INFO:journalctl@ceph.mon.b.smithi165.stdout: data_block_index_type: 0 2024-09-10T23:57:07.304 INFO:journalctl@ceph.mon.b.smithi165.stdout: index_shortening: 1 2024-09-10T23:57:07.304 INFO:journalctl@ceph.mon.b.smithi165.stdout: data_block_hash_table_util_ratio: 0.750000 2024-09-10T23:57:07.304 INFO:journalctl@ceph.mon.b.smithi165.stdout: checksum: 4 2024-09-10T23:57:07.304 INFO:journalctl@ceph.mon.b.smithi165.stdout: no_block_cache: 0 2024-09-10T23:57:07.304 INFO:journalctl@ceph.mon.b.smithi165.stdout: block_cache: 0x556f805cb610 2024-09-10T23:57:07.304 INFO:journalctl@ceph.mon.b.smithi165.stdout: block_cache_name: BinnedLRUCache 2024-09-10T23:57:07.304 INFO:journalctl@ceph.mon.b.smithi165.stdout: block_cache_options: 2024-09-10T23:57:07.304 INFO:journalctl@ceph.mon.b.smithi165.stdout: capacity : 536870912 2024-09-10T23:57:07.305 INFO:journalctl@ceph.mon.b.smithi165.stdout: num_shard_bits : 4 2024-09-10T23:57:07.305 INFO:journalctl@ceph.mon.b.smithi165.stdout: strict_capacity_limit : 0 2024-09-10T23:57:07.305 INFO:journalctl@ceph.mon.b.smithi165.stdout: high_pri_pool_ratio: 0.000 2024-09-10T23:57:07.305 INFO:journalctl@ceph.mon.b.smithi165.stdout: block_cache_compressed: (nil) 2024-09-10T23:57:07.305 INFO:journalctl@ceph.mon.b.smithi165.stdout: persistent_cache: (nil) 2024-09-10T23:57:07.305 INFO:journalctl@ceph.mon.b.smithi165.stdout: block_size: 4096 2024-09-10T23:57:07.305 INFO:journalctl@ceph.mon.b.smithi165.stdout: block_size_deviation: 10 2024-09-10T23:57:07.305 INFO:journalctl@ceph.mon.b.smithi165.stdout: block_restart_interval: 16 2024-09-10T23:57:07.305 INFO:journalctl@ceph.mon.b.smithi165.stdout: index_block_restart_interval: 1 2024-09-10T23:57:07.305 INFO:journalctl@ceph.mon.b.smithi165.stdout: metadata_block_size: 4096 2024-09-10T23:57:07.305 INFO:journalctl@ceph.mon.b.smithi165.stdout: partition_filters: 0 2024-09-10T23:57:07.305 INFO:journalctl@ceph.mon.b.smithi165.stdout: use_delta_encoding: 1 2024-09-10T23:57:07.305 INFO:journalctl@ceph.mon.b.smithi165.stdout: filter_policy: bloomfilter 2024-09-10T23:57:07.306 INFO:journalctl@ceph.mon.b.smithi165.stdout: whole_key_filtering: 1 2024-09-10T23:57:07.306 INFO:journalctl@ceph.mon.b.smithi165.stdout: verify_compression: 0 2024-09-10T23:57:07.306 INFO:journalctl@ceph.mon.b.smithi165.stdout: read_amp_bytes_per_bit: 0 2024-09-10T23:57:07.306 INFO:journalctl@ceph.mon.b.smithi165.stdout: format_version: 5 2024-09-10T23:57:07.306 INFO:journalctl@ceph.mon.b.smithi165.stdout: enable_index_compression: 1 2024-09-10T23:57:07.306 INFO:journalctl@ceph.mon.b.smithi165.stdout: block_align: 0 2024-09-10T23:57:07.306 INFO:journalctl@ceph.mon.b.smithi165.stdout: max_auto_readahead_size: 262144 2024-09-10T23:57:07.306 INFO:journalctl@ceph.mon.b.smithi165.stdout: prepopulate_block_cache: 0 2024-09-10T23:57:07.306 INFO:journalctl@ceph.mon.b.smithi165.stdout: initial_auto_readahead_size: 8192 2024-09-10T23:57:07.306 INFO:journalctl@ceph.mon.b.smithi165.stdout: num_file_reads_for_auto_readahead: 2 2024-09-10T23:57:07.306 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:06 smithi165 ceph-mon[35356]: rocksdb: Options.write_buffer_size: 33554432 2024-09-10T23:57:07.306 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:06 smithi165 ceph-mon[35356]: rocksdb: Options.max_write_buffer_number: 2 2024-09-10T23:57:07.306 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:06 smithi165 ceph-mon[35356]: rocksdb: Options.compression: NoCompression 2024-09-10T23:57:07.307 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:06 smithi165 ceph-mon[35356]: rocksdb: Options.bottommost_compression: Disabled 2024-09-10T23:57:07.307 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:06 smithi165 ceph-mon[35356]: rocksdb: Options.prefix_extractor: nullptr 2024-09-10T23:57:07.307 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:06 smithi165 ceph-mon[35356]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-09-10T23:57:07.307 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:06 smithi165 ceph-mon[35356]: rocksdb: Options.num_levels: 7 2024-09-10T23:57:07.307 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:06 smithi165 ceph-mon[35356]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-09-10T23:57:07.307 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:06 smithi165 ceph-mon[35356]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-09-10T23:57:07.307 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:06 smithi165 ceph-mon[35356]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-09-10T23:57:07.307 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:06 smithi165 ceph-mon[35356]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-09-10T23:57:07.307 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:06 smithi165 ceph-mon[35356]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-09-10T23:57:07.307 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:06 smithi165 ceph-mon[35356]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-09-10T23:57:07.307 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:06 smithi165 ceph-mon[35356]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-09-10T23:57:07.307 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:06 smithi165 ceph-mon[35356]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-09-10T23:57:07.307 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:06 smithi165 ceph-mon[35356]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-09-10T23:57:07.307 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:06 smithi165 ceph-mon[35356]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-09-10T23:57:07.308 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:06 smithi165 ceph-mon[35356]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-09-10T23:57:07.308 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:06 smithi165 ceph-mon[35356]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-09-10T23:57:07.308 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:06 smithi165 ceph-mon[35356]: rocksdb: Options.compression_opts.window_bits: -14 2024-09-10T23:57:07.308 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:06 smithi165 ceph-mon[35356]: rocksdb: Options.compression_opts.level: 32767 2024-09-10T23:57:07.308 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:06 smithi165 ceph-mon[35356]: rocksdb: Options.compression_opts.strategy: 0 2024-09-10T23:57:07.308 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:06 smithi165 ceph-mon[35356]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-09-10T23:57:07.308 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:06 smithi165 ceph-mon[35356]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-09-10T23:57:07.308 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:06 smithi165 ceph-mon[35356]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-09-10T23:57:07.308 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:06 smithi165 ceph-mon[35356]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-09-10T23:57:07.308 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:06 smithi165 ceph-mon[35356]: rocksdb: Options.compression_opts.enabled: false 2024-09-10T23:57:07.308 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:06 smithi165 ceph-mon[35356]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-09-10T23:57:07.308 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:06 smithi165 ceph-mon[35356]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-09-10T23:57:07.308 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:06 smithi165 ceph-mon[35356]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-09-10T23:57:07.309 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:06 smithi165 ceph-mon[35356]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-09-10T23:57:07.309 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:06 smithi165 ceph-mon[35356]: rocksdb: Options.target_file_size_base: 67108864 2024-09-10T23:57:07.309 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:06 smithi165 ceph-mon[35356]: rocksdb: Options.target_file_size_multiplier: 1 2024-09-10T23:57:07.309 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:06 smithi165 ceph-mon[35356]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-09-10T23:57:07.309 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:06 smithi165 ceph-mon[35356]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-09-10T23:57:07.309 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:06 smithi165 ceph-mon[35356]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-09-10T23:57:07.309 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:06 smithi165 ceph-mon[35356]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-09-10T23:57:07.309 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:06 smithi165 ceph-mon[35356]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-09-10T23:57:07.309 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:06 smithi165 ceph-mon[35356]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-09-10T23:57:07.309 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:06 smithi165 ceph-mon[35356]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-09-10T23:57:07.309 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:06 smithi165 ceph-mon[35356]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-09-10T23:57:07.309 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:06 smithi165 ceph-mon[35356]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-09-10T23:57:07.309 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:06 smithi165 ceph-mon[35356]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-09-10T23:57:07.309 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:06 smithi165 ceph-mon[35356]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-09-10T23:57:07.310 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:06 smithi165 ceph-mon[35356]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-09-10T23:57:07.310 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:06 smithi165 ceph-mon[35356]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-09-10T23:57:07.310 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:06 smithi165 ceph-mon[35356]: rocksdb: Options.arena_block_size: 1048576 2024-09-10T23:57:07.310 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:06 smithi165 ceph-mon[35356]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-09-10T23:57:07.310 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:06 smithi165 ceph-mon[35356]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-09-10T23:57:07.310 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:06 smithi165 ceph-mon[35356]: rocksdb: Options.disable_auto_compactions: 0 2024-09-10T23:57:07.310 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:06 smithi165 ceph-mon[35356]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-09-10T23:57:07.310 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:06 smithi165 ceph-mon[35356]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-09-10T23:57:07.310 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:06 smithi165 ceph-mon[35356]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-09-10T23:57:07.310 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:06 smithi165 ceph-mon[35356]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-09-10T23:57:07.311 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:06 smithi165 ceph-mon[35356]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-09-10T23:57:07.311 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:06 smithi165 ceph-mon[35356]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-09-10T23:57:07.311 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:06 smithi165 ceph-mon[35356]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-09-10T23:57:07.311 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:06 smithi165 ceph-mon[35356]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-09-10T23:57:07.311 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:06 smithi165 ceph-mon[35356]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-09-10T23:57:07.311 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:06 smithi165 ceph-mon[35356]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-09-10T23:57:07.311 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:06 smithi165 ceph-mon[35356]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-09-10T23:57:07.311 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:06 smithi165 ceph-mon[35356]: rocksdb: Options.inplace_update_support: 0 2024-09-10T23:57:07.311 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:06 smithi165 ceph-mon[35356]: rocksdb: Options.inplace_update_num_locks: 10000 2024-09-10T23:57:07.311 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:06 smithi165 ceph-mon[35356]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-09-10T23:57:07.311 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:06 smithi165 ceph-mon[35356]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-09-10T23:57:07.311 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:06 smithi165 ceph-mon[35356]: rocksdb: Options.memtable_huge_page_size: 0 2024-09-10T23:57:07.311 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:06 smithi165 ceph-mon[35356]: rocksdb: Options.bloom_locality: 0 2024-09-10T23:57:07.312 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:06 smithi165 ceph-mon[35356]: rocksdb: Options.max_successive_merges: 0 2024-09-10T23:57:07.312 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:06 smithi165 ceph-mon[35356]: rocksdb: Options.optimize_filters_for_hits: 0 2024-09-10T23:57:07.312 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:06 smithi165 ceph-mon[35356]: rocksdb: Options.paranoid_file_checks: 0 2024-09-10T23:57:07.312 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:06 smithi165 ceph-mon[35356]: rocksdb: Options.force_consistency_checks: 1 2024-09-10T23:57:07.312 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:06 smithi165 ceph-mon[35356]: rocksdb: Options.report_bg_io_stats: 0 2024-09-10T23:57:07.312 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:06 smithi165 ceph-mon[35356]: rocksdb: Options.ttl: 2592000 2024-09-10T23:57:07.312 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:06 smithi165 ceph-mon[35356]: rocksdb: Options.periodic_compaction_seconds: 0 2024-09-10T23:57:07.312 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:06 smithi165 ceph-mon[35356]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-09-10T23:57:07.312 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:06 smithi165 ceph-mon[35356]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-09-10T23:57:07.312 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:06 smithi165 ceph-mon[35356]: rocksdb: Options.enable_blob_files: false 2024-09-10T23:57:07.312 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:06 smithi165 ceph-mon[35356]: rocksdb: Options.min_blob_size: 0 2024-09-10T23:57:07.312 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:06 smithi165 ceph-mon[35356]: rocksdb: Options.blob_file_size: 268435456 2024-09-10T23:57:07.312 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:06 smithi165 ceph-mon[35356]: rocksdb: Options.blob_compression_type: NoCompression 2024-09-10T23:57:07.313 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:06 smithi165 ceph-mon[35356]: rocksdb: Options.enable_blob_garbage_collection: false 2024-09-10T23:57:07.313 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:06 smithi165 ceph-mon[35356]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-09-10T23:57:07.313 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:06 smithi165 ceph-mon[35356]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-09-10T23:57:07.313 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:06 smithi165 ceph-mon[35356]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-09-10T23:57:07.313 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:06 smithi165 ceph-mon[35356]: rocksdb: Options.blob_file_starting_level: 0 2024-09-10T23:57:07.313 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:06 smithi165 ceph-mon[35356]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-09-10T23:57:07.313 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:06 smithi165 ceph-mon[35356]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-09-10T23:57:07.313 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:06 smithi165 ceph-mon[35356]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2024-09-10T23:57:07.314 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:06 smithi165 ceph-mon[35356]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: af348881-4f5c-4932-932a-b2bc394fe032 2024-09-10T23:57:07.314 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:06 smithi165 ceph-mon[35356]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726012626985259, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-09-10T23:57:07.314 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:06 smithi165 ceph-mon[35356]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2024-09-10T23:57:07.314 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:06 smithi165 ceph-mon[35356]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726012626985662, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1773, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 648, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 526, "raw_average_value_size": 105, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1726012626, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "af348881-4f5c-4932-932a-b2bc394fe032", "db_session_id": "082CDN2WSZDA70NBP9L2", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2024-09-10T23:57:07.314 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:06 smithi165 ceph-mon[35356]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726012626985759, "job": 1, "event": "recovery_finished"} 2024-09-10T23:57:07.314 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:06 smithi165 ceph-mon[35356]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2024-09-10T23:57:07.314 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:06 smithi165 ceph-mon[35356]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-b/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-09-10T23:57:07.314 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:06 smithi165 ceph-mon[35356]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x556f805fae00 2024-09-10T23:57:07.314 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:06 smithi165 ceph-mon[35356]: rocksdb: DB pointer 0x556f806fc000 2024-09-10T23:57:07.314 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:06 smithi165 ceph-mon[35356]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-09-10T23:57:07.314 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:06 smithi165 ceph-mon[35356]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-09-10T23:57:07.314 INFO:journalctl@ceph.mon.b.smithi165.stdout: ** DB Stats ** 2024-09-10T23:57:07.314 INFO:journalctl@ceph.mon.b.smithi165.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-10T23:57:07.314 INFO:journalctl@ceph.mon.b.smithi165.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-09-10T23:57:07.315 INFO:journalctl@ceph.mon.b.smithi165.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-10T23:57:07.315 INFO:journalctl@ceph.mon.b.smithi165.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-10T23:57:07.315 INFO:journalctl@ceph.mon.b.smithi165.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-09-10T23:57:07.315 INFO:journalctl@ceph.mon.b.smithi165.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-10T23:57:07.315 INFO:journalctl@ceph.mon.b.smithi165.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-10T23:57:07.315 INFO:journalctl@ceph.mon.b.smithi165.stdout: 2024-09-10T23:57:07.315 INFO:journalctl@ceph.mon.b.smithi165.stdout: ** Compaction Stats [default] ** 2024-09-10T23:57:07.315 INFO:journalctl@ceph.mon.b.smithi165.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-10T23:57:07.315 INFO:journalctl@ceph.mon.b.smithi165.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-09-10T23:57:07.315 INFO:journalctl@ceph.mon.b.smithi165.stdout: L0 1/0 1.73 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.4 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-09-10T23:57:07.315 INFO:journalctl@ceph.mon.b.smithi165.stdout: Sum 1/0 1.73 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.4 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-09-10T23:57:07.315 INFO:journalctl@ceph.mon.b.smithi165.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.4 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-09-10T23:57:07.315 INFO:journalctl@ceph.mon.b.smithi165.stdout: 2024-09-10T23:57:07.316 INFO:journalctl@ceph.mon.b.smithi165.stdout: ** Compaction Stats [default] ** 2024-09-10T23:57:07.316 INFO:journalctl@ceph.mon.b.smithi165.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-10T23:57:07.316 INFO:journalctl@ceph.mon.b.smithi165.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-10T23:57:07.316 INFO:journalctl@ceph.mon.b.smithi165.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 4.4 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-09-10T23:57:07.316 INFO:journalctl@ceph.mon.b.smithi165.stdout: 2024-09-10T23:57:07.316 INFO:journalctl@ceph.mon.b.smithi165.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-09-10T23:57:07.316 INFO:journalctl@ceph.mon.b.smithi165.stdout: 2024-09-10T23:57:07.316 INFO:journalctl@ceph.mon.b.smithi165.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-10T23:57:07.316 INFO:journalctl@ceph.mon.b.smithi165.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-09-10T23:57:07.316 INFO:journalctl@ceph.mon.b.smithi165.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-09-10T23:57:07.317 INFO:journalctl@ceph.mon.b.smithi165.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-09-10T23:57:07.317 INFO:journalctl@ceph.mon.b.smithi165.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-09-10T23:57:07.317 INFO:journalctl@ceph.mon.b.smithi165.stdout: AddFile(Keys): cumulative 0, interval 0 2024-09-10T23:57:07.317 INFO:journalctl@ceph.mon.b.smithi165.stdout: Cumulative compaction: 0.00 GB write, 0.67 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-10T23:57:07.317 INFO:journalctl@ceph.mon.b.smithi165.stdout: Interval compaction: 0.00 GB write, 0.67 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-10T23:57:07.317 INFO:journalctl@ceph.mon.b.smithi165.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-09-10T23:57:07.317 INFO:journalctl@ceph.mon.b.smithi165.stdout: Block cache BinnedLRUCache@0x556f805cb610#2 capacity: 512.00 MB usage: 0.22 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1e-05 secs_since: 0 2024-09-10T23:57:07.317 INFO:journalctl@ceph.mon.b.smithi165.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2024-09-10T23:57:07.317 INFO:journalctl@ceph.mon.b.smithi165.stdout: 2024-09-10T23:57:07.317 INFO:journalctl@ceph.mon.b.smithi165.stdout: ** File Read Latency Histogram By Level [default] ** 2024-09-10T23:57:07.317 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:06 smithi165 ceph-mon[35356]: mon.b does not exist in monmap, will attempt to join an existing cluster 2024-09-10T23:57:07.317 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:06 smithi165 ceph-mon[35356]: using public_addr v2:172.21.15.165:0/0 -> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] 2024-09-10T23:57:07.317 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:06 smithi165 ceph-mon[35356]: starting mon.b rank -1 at public addrs [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] at bind addrs [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] mon_data /var/lib/ceph/mon/ceph-b fsid ff3ad74a-6fcf-11ef-bcea-c7b262605968 2024-09-10T23:57:07.318 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:06 smithi165 ceph-mon[35356]: mon.b@-1(???) e0 preinit fsid ff3ad74a-6fcf-11ef-bcea-c7b262605968 2024-09-10T23:57:07.318 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:07 smithi165 ceph-mon[35356]: mon.b@-1(synchronizing).mds e1 new map 2024-09-10T23:57:07.318 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:07 smithi165 ceph-mon[35356]: mon.b@-1(synchronizing).mds e1 print_map 2024-09-10T23:57:07.318 INFO:journalctl@ceph.mon.b.smithi165.stdout: e1 2024-09-10T23:57:07.318 INFO:journalctl@ceph.mon.b.smithi165.stdout: btime 2024-09-10T23:55:16:317386+0000 2024-09-10T23:57:07.318 INFO:journalctl@ceph.mon.b.smithi165.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-09-10T23:57:07.318 INFO:journalctl@ceph.mon.b.smithi165.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-10T23:57:07.318 INFO:journalctl@ceph.mon.b.smithi165.stdout: legacy client fscid: -1 2024-09-10T23:57:07.318 INFO:journalctl@ceph.mon.b.smithi165.stdout: 2024-09-10T23:57:07.318 INFO:journalctl@ceph.mon.b.smithi165.stdout: No filesystems configured 2024-09-10T23:57:07.318 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:07 smithi165 ceph-mon[35356]: mon.b@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-09-10T23:57:07.319 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:07 smithi165 ceph-mon[35356]: mon.b@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-09-10T23:57:07.319 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:07 smithi165 ceph-mon[35356]: mon.b@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-09-10T23:57:07.319 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:07 smithi165 ceph-mon[35356]: mon.b@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-09-10T23:57:07.319 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:07 smithi165 ceph-mon[35356]: mon.b@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-09-10T23:57:07.319 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:07 smithi165 ceph-mon[35356]: mon.b@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-09-10T23:57:07.319 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:07 smithi165 ceph-mon[35356]: mon.b@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-09-10T23:57:07.319 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:07 smithi165 ceph-mon[35356]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-10T23:57:07.319 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:07 smithi165 ceph-mon[35356]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-10T23:57:07.319 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:07 smithi165 ceph-mon[35356]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-10T23:57:07.319 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:07 smithi165 ceph-mon[35356]: Updating smithi165:/etc/ceph/ceph.client.admin.keyring 2024-09-10T23:57:07.319 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:07 smithi165 ceph-mon[35356]: Deploying cephadm binary to smithi178 2024-09-10T23:57:07.319 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:07 smithi165 ceph-mon[35356]: Updating smithi165:/var/lib/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-10T23:57:07.320 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:07 smithi165 ceph-mon[35356]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T23:57:07.320 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:07 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-10T23:57:07.320 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:07 smithi165 ceph-mon[35356]: Added host smithi178 2024-09-10T23:57:07.320 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:07 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T23:57:07.320 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:07 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-10T23:57:07.320 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:07 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-10T23:57:07.320 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:07 smithi165 ceph-mon[35356]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T23:57:07.320 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:07 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-10T23:57:07.320 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:07 smithi165 ceph-mon[35356]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T23:57:07.320 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:07 smithi165 ceph-mon[35356]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T23:57:07.320 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:07 smithi165 ceph-mon[35356]: from='client.? 172.21.15.12:0/361548279' entity='client.admin' cmd={"prefix": "osd crush tunables", "profile": "default"} : dispatch 2024-09-10T23:57:07.320 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:07 smithi165 ceph-mon[35356]: from='client.? 172.21.15.12:0/361548279' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-09-10T23:57:07.321 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:07 smithi165 ceph-mon[35356]: osdmap e4: 0 total, 0 up, 0 in 2024-09-10T23:57:07.321 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:07 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-10T23:57:07.321 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:07 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-10T23:57:07.321 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:07 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-10T23:57:07.321 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:07 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-10T23:57:07.321 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:07 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd/host:smithi178", "name": "osd_memory_target"} : dispatch 2024-09-10T23:57:07.321 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:07 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T23:57:07.321 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:07 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T23:57:07.321 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:07 smithi165 ceph-mon[35356]: Updating smithi178:/etc/ceph/ceph.conf 2024-09-10T23:57:07.321 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:07 smithi165 ceph-mon[35356]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T23:57:07.321 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:07 smithi165 ceph-mon[35356]: Updating smithi178:/var/lib/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T23:57:07.321 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:07 smithi165 ceph-mon[35356]: Updating smithi178:/etc/ceph/ceph.client.admin.keyring 2024-09-10T23:57:07.321 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:07 smithi165 ceph-mon[35356]: Updating smithi178:/var/lib/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-10T23:57:07.322 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:07 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-10T23:57:07.322 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:07 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-10T23:57:07.322 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:07 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-10T23:57:07.322 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:07 smithi165 ceph-mon[35356]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi012:172.21.15.12=a;smithi165:172.21.15.165=b;smithi178:172.21.15.178=c", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T23:57:07.322 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:07 smithi165 ceph-mon[35356]: Saving service mon spec with placement smithi012:172.21.15.12=a;smithi165:172.21.15.165=b;smithi178:172.21.15.178=c;count:3 2024-09-10T23:57:07.322 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:07 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-10T23:57:07.322 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:07 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T23:57:07.322 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:07 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T23:57:07.322 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:07 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T23:57:07.322 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:07 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-10T23:57:07.322 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:07 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-10T23:57:07.322 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:07 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T23:57:07.322 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:07 smithi165 ceph-mon[35356]: Deploying daemon mon.c on smithi178 2024-09-10T23:57:07.323 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:07 smithi165 ceph-mon[35356]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T23:57:07.323 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:07 smithi165 ceph-mon[35356]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T23:57:07.323 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:07 smithi165 ceph-mon[35356]: from='client.? 172.21.15.178:0/730918050' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-09-10T23:57:07.323 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:07 smithi165 ceph-mon[35356]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T23:57:07.323 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:07 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-10T23:57:07.323 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:07 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-10T23:57:07.323 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:07 smithi165 ceph-mon[35356]: mon.a calling monitor election 2024-09-10T23:57:07.323 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:07 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-10T23:57:07.323 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:07 smithi165 ceph-mon[35356]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T23:57:07.323 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:07 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-10T23:57:07.323 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:07 smithi165 ceph-mon[35356]: mon.c calling monitor election 2024-09-10T23:57:07.323 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:07 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-10T23:57:07.323 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:07 smithi165 ceph-mon[35356]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T23:57:07.324 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:07 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-10T23:57:07.324 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:07 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-10T23:57:07.324 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:07 smithi165 ceph-mon[35356]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-10T23:57:07.324 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:07 smithi165 ceph-mon[35356]: monmap epoch 2 2024-09-10T23:57:07.324 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:07 smithi165 ceph-mon[35356]: fsid ff3ad74a-6fcf-11ef-bcea-c7b262605968 2024-09-10T23:57:07.324 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:07 smithi165 ceph-mon[35356]: last_changed 2024-09-10T23:56:58.153268+0000 2024-09-10T23:57:07.324 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:07 smithi165 ceph-mon[35356]: created 2024-09-10T23:55:12.644680+0000 2024-09-10T23:57:07.324 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:07 smithi165 ceph-mon[35356]: min_mon_release 19 (squid) 2024-09-10T23:57:07.324 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:07 smithi165 ceph-mon[35356]: election_strategy: 1 2024-09-10T23:57:07.324 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:07 smithi165 ceph-mon[35356]: 0: [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] mon.a 2024-09-10T23:57:07.324 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:07 smithi165 ceph-mon[35356]: 1: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.c 2024-09-10T23:57:07.324 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:07 smithi165 ceph-mon[35356]: fsmap 2024-09-10T23:57:07.324 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:07 smithi165 ceph-mon[35356]: osdmap e4: 0 total, 0 up, 0 in 2024-09-10T23:57:07.325 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:07 smithi165 ceph-mon[35356]: mgrmap e13: x(active, since 45s) 2024-09-10T23:57:07.325 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:07 smithi165 ceph-mon[35356]: overall HEALTH_OK 2024-09-10T23:57:07.325 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:07 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-10T23:57:07.325 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:07 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-10T23:57:07.325 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:07 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-10T23:57:07.325 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:07 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-10T23:57:07.325 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:07 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T23:57:07.325 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:07 smithi165 ceph-mon[35356]: Deploying daemon mon.b on smithi165 2024-09-10T23:57:07.325 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:07 smithi165 ceph-mon[35356]: from='client.? 172.21.15.178:0/903600878' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-09-10T23:57:07.325 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:07 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-10T23:57:07.325 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:07 smithi165 ceph-mon[35356]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T23:57:07.325 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:07 smithi165 ceph-mon[35356]: mon.b@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-09-10T23:57:07.325 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:07 smithi165 bash[35336]: 77449392964c91580fa9aa94f7de799550d86a3308e285bee7b6cc19999096b7 2024-09-10T23:57:07.326 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:07 smithi165 systemd[1]: Started Ceph mon.b for ff3ad74a-6fcf-11ef-bcea-c7b262605968. 2024-09-10T23:57:08.055 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-10T23:57:08.055 DEBUG:teuthology.orchestra.run.smithi178:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ff3ad74a-6fcf-11ef-bcea-c7b262605968 -- ceph mon dump -f json 2024-09-10T23:57:08.302 INFO:teuthology.orchestra.run.smithi178.stderr:Inferring config /var/lib/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/mon.c/config 2024-09-10T23:57:12.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:57:12 smithi178 ceph-mon[35919]: mon.c calling monitor election 2024-09-10T23:57:12.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:57:12 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-10T23:57:12.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:57:12 smithi178 ceph-mon[35919]: mon.a calling monitor election 2024-09-10T23:57:12.332 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:57:12 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-10T23:57:12.332 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:57:12 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-10T23:57:12.332 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:57:12 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-10T23:57:12.332 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:57:12 smithi178 ceph-mon[35919]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T23:57:12.332 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:57:12 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-10T23:57:12.332 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:57:12 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-10T23:57:12.332 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:57:12 smithi178 ceph-mon[35919]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T23:57:12.332 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:57:12 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-10T23:57:12.332 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:57:12 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-10T23:57:12.332 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:57:12 smithi178 ceph-mon[35919]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-10T23:57:12.332 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:57:12 smithi178 ceph-mon[35919]: monmap epoch 3 2024-09-10T23:57:12.332 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:57:12 smithi178 ceph-mon[35919]: fsid ff3ad74a-6fcf-11ef-bcea-c7b262605968 2024-09-10T23:57:12.332 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:57:12 smithi178 ceph-mon[35919]: last_changed 2024-09-10T23:57:07.011167+0000 2024-09-10T23:57:12.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:57:12 smithi178 ceph-mon[35919]: created 2024-09-10T23:55:12.644680+0000 2024-09-10T23:57:12.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:57:12 smithi178 ceph-mon[35919]: min_mon_release 19 (squid) 2024-09-10T23:57:12.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:57:12 smithi178 ceph-mon[35919]: election_strategy: 1 2024-09-10T23:57:12.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:57:12 smithi178 ceph-mon[35919]: 0: [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] mon.a 2024-09-10T23:57:12.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:57:12 smithi178 ceph-mon[35919]: 1: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.c 2024-09-10T23:57:12.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:57:12 smithi178 ceph-mon[35919]: 2: [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] mon.b 2024-09-10T23:57:12.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:57:12 smithi178 ceph-mon[35919]: fsmap 2024-09-10T23:57:12.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:57:12 smithi178 ceph-mon[35919]: osdmap e4: 0 total, 0 up, 0 in 2024-09-10T23:57:12.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:57:12 smithi178 ceph-mon[35919]: mgrmap e13: x(active, since 54s) 2024-09-10T23:57:12.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:57:12 smithi178 ceph-mon[35919]: overall HEALTH_OK 2024-09-10T23:57:12.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:57:12 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-10T23:57:12.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:57:12 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-10T23:57:12.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:57:12 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-10T23:57:12.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:57:12 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-10T23:57:12.563 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:57:12 smithi012 ceph-mon[30333]: mon.c calling monitor election 2024-09-10T23:57:12.563 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:57:12 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-10T23:57:12.563 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:57:12 smithi012 ceph-mon[30333]: mon.a calling monitor election 2024-09-10T23:57:12.563 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:57:12 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-10T23:57:12.563 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:57:12 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-10T23:57:12.563 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:57:12 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-10T23:57:12.563 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:57:12 smithi012 ceph-mon[30333]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T23:57:12.563 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:57:12 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-10T23:57:12.564 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:57:12 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-10T23:57:12.564 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:57:12 smithi012 ceph-mon[30333]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T23:57:12.564 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:57:12 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-10T23:57:12.564 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:57:12 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-10T23:57:12.564 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:57:12 smithi012 ceph-mon[30333]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-10T23:57:12.564 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:57:12 smithi012 ceph-mon[30333]: monmap epoch 3 2024-09-10T23:57:12.564 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:57:12 smithi012 ceph-mon[30333]: fsid ff3ad74a-6fcf-11ef-bcea-c7b262605968 2024-09-10T23:57:12.564 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:57:12 smithi012 ceph-mon[30333]: last_changed 2024-09-10T23:57:07.011167+0000 2024-09-10T23:57:12.564 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:57:12 smithi012 ceph-mon[30333]: created 2024-09-10T23:55:12.644680+0000 2024-09-10T23:57:12.564 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:57:12 smithi012 ceph-mon[30333]: min_mon_release 19 (squid) 2024-09-10T23:57:12.564 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:57:12 smithi012 ceph-mon[30333]: election_strategy: 1 2024-09-10T23:57:12.564 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:57:12 smithi012 ceph-mon[30333]: 0: [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] mon.a 2024-09-10T23:57:12.564 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:57:12 smithi012 ceph-mon[30333]: 1: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.c 2024-09-10T23:57:12.565 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:57:12 smithi012 ceph-mon[30333]: 2: [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] mon.b 2024-09-10T23:57:12.565 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:57:12 smithi012 ceph-mon[30333]: fsmap 2024-09-10T23:57:12.565 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:57:12 smithi012 ceph-mon[30333]: osdmap e4: 0 total, 0 up, 0 in 2024-09-10T23:57:12.565 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:57:12 smithi012 ceph-mon[30333]: mgrmap e13: x(active, since 54s) 2024-09-10T23:57:12.565 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:57:12 smithi012 ceph-mon[30333]: overall HEALTH_OK 2024-09-10T23:57:12.565 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:57:12 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-10T23:57:12.565 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:57:12 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-10T23:57:12.565 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:57:12 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-10T23:57:12.565 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:57:12 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-10T23:57:12.580 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-09-10T23:57:12.580 INFO:teuthology.orchestra.run.smithi178.stdout:{"epoch":3,"fsid":"ff3ad74a-6fcf-11ef-bcea-c7b262605968","modified":"2024-09-10T23:57:07.011167Z","created":"2024-09-10T23:55:12.644680Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:3300","nonce":0},{"type":"v1","addr":"172.21.15.12:6789","nonce":0}]},"addr":"172.21.15.12:6789/0","public_addr":"172.21.15.12:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:3300","nonce":0},{"type":"v1","addr":"172.21.15.178:6789","nonce":0}]},"addr":"172.21.15.178:6789/0","public_addr":"172.21.15.178:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:3300","nonce":0},{"type":"v1","addr":"172.21.15.165:6789","nonce":0}]},"addr":"172.21.15.165:6789/0","public_addr":"172.21.15.165:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-09-10T23:57:12.580 INFO:teuthology.orchestra.run.smithi178.stderr:dumped monmap epoch 3 2024-09-10T23:57:13.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:57:13 smithi178 ceph-mon[35919]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T23:57:13.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:57:13 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T23:57:13.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:57:13 smithi178 ceph-mon[35919]: from='client.? 172.21.15.178:0/4232467133' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-09-10T23:57:13.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:57:13 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-10T23:57:13.562 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:57:13 smithi012 ceph-mon[30333]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T23:57:13.563 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:57:13 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T23:57:13.563 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:57:13 smithi012 ceph-mon[30333]: from='client.? 172.21.15.178:0/4232467133' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-09-10T23:57:13.563 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:57:13 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-10T23:57:13.892 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-09-10T23:57:13.892 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ff3ad74a-6fcf-11ef-bcea-c7b262605968 -- ceph config generate-minimal-conf 2024-09-10T23:57:14.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:14 smithi165 ceph-mon[35356]: mon.c calling monitor election 2024-09-10T23:57:14.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:14 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-10T23:57:14.283 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:14 smithi165 ceph-mon[35356]: mon.a calling monitor election 2024-09-10T23:57:14.283 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:14 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-10T23:57:14.283 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:14 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-10T23:57:14.283 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:14 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-10T23:57:14.283 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:14 smithi165 ceph-mon[35356]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T23:57:14.283 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:14 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-10T23:57:14.283 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:14 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-10T23:57:14.283 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:14 smithi165 ceph-mon[35356]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T23:57:14.283 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:14 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-10T23:57:14.283 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:14 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-10T23:57:14.283 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:14 smithi165 ceph-mon[35356]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-10T23:57:14.283 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:14 smithi165 ceph-mon[35356]: monmap epoch 3 2024-09-10T23:57:14.283 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:14 smithi165 ceph-mon[35356]: fsid ff3ad74a-6fcf-11ef-bcea-c7b262605968 2024-09-10T23:57:14.284 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:14 smithi165 ceph-mon[35356]: last_changed 2024-09-10T23:57:07.011167+0000 2024-09-10T23:57:14.284 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:14 smithi165 ceph-mon[35356]: created 2024-09-10T23:55:12.644680+0000 2024-09-10T23:57:14.284 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:14 smithi165 ceph-mon[35356]: min_mon_release 19 (squid) 2024-09-10T23:57:14.284 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:14 smithi165 ceph-mon[35356]: election_strategy: 1 2024-09-10T23:57:14.284 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:14 smithi165 ceph-mon[35356]: 0: [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] mon.a 2024-09-10T23:57:14.284 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:14 smithi165 ceph-mon[35356]: 1: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.c 2024-09-10T23:57:14.284 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:14 smithi165 ceph-mon[35356]: 2: [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] mon.b 2024-09-10T23:57:14.284 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:14 smithi165 ceph-mon[35356]: fsmap 2024-09-10T23:57:14.284 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:14 smithi165 ceph-mon[35356]: osdmap e4: 0 total, 0 up, 0 in 2024-09-10T23:57:14.284 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:14 smithi165 ceph-mon[35356]: mgrmap e13: x(active, since 54s) 2024-09-10T23:57:14.284 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:14 smithi165 ceph-mon[35356]: overall HEALTH_OK 2024-09-10T23:57:14.284 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:14 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-10T23:57:14.284 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:14 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-10T23:57:14.285 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:14 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-10T23:57:14.285 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:14 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-10T23:57:14.285 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:14 smithi165 ceph-mon[35356]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T23:57:14.285 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:14 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T23:57:14.285 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:14 smithi165 ceph-mon[35356]: from='client.? 172.21.15.178:0/4232467133' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-09-10T23:57:14.285 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:14 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-10T23:57:14.286 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T23:57:15.044 INFO:teuthology.orchestra.run.smithi012.stdout:# minimal ceph.conf for ff3ad74a-6fcf-11ef-bcea-c7b262605968 2024-09-10T23:57:15.044 INFO:teuthology.orchestra.run.smithi012.stdout:[global] 2024-09-10T23:57:15.044 INFO:teuthology.orchestra.run.smithi012.stdout: fsid = ff3ad74a-6fcf-11ef-bcea-c7b262605968 2024-09-10T23:57:15.044 INFO:teuthology.orchestra.run.smithi012.stdout: mon_host = [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] 2024-09-10T23:57:15.283 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:14 smithi165 ceph-mon[35356]: mon.b calling monitor election 2024-09-10T23:57:15.283 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:14 smithi165 ceph-mon[35356]: Updating smithi012:/var/lib/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T23:57:15.283 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:14 smithi165 ceph-mon[35356]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T23:57:15.283 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:14 smithi165 ceph-mon[35356]: mon.b calling monitor election 2024-09-10T23:57:15.283 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:14 smithi165 ceph-mon[35356]: mon.c calling monitor election 2024-09-10T23:57:15.283 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:14 smithi165 ceph-mon[35356]: mon.a calling monitor election 2024-09-10T23:57:15.283 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:14 smithi165 ceph-mon[35356]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-09-10T23:57:15.283 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:14 smithi165 ceph-mon[35356]: monmap epoch 3 2024-09-10T23:57:15.284 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:14 smithi165 ceph-mon[35356]: fsid ff3ad74a-6fcf-11ef-bcea-c7b262605968 2024-09-10T23:57:15.284 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:14 smithi165 ceph-mon[35356]: last_changed 2024-09-10T23:57:07.011167+0000 2024-09-10T23:57:15.284 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:14 smithi165 ceph-mon[35356]: created 2024-09-10T23:55:12.644680+0000 2024-09-10T23:57:15.284 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:14 smithi165 ceph-mon[35356]: min_mon_release 19 (squid) 2024-09-10T23:57:15.284 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:14 smithi165 ceph-mon[35356]: election_strategy: 1 2024-09-10T23:57:15.284 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:14 smithi165 ceph-mon[35356]: 0: [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] mon.a 2024-09-10T23:57:15.284 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:14 smithi165 ceph-mon[35356]: 1: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.c 2024-09-10T23:57:15.284 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:14 smithi165 ceph-mon[35356]: 2: [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] mon.b 2024-09-10T23:57:15.284 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:14 smithi165 ceph-mon[35356]: fsmap 2024-09-10T23:57:15.284 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:14 smithi165 ceph-mon[35356]: osdmap e4: 0 total, 0 up, 0 in 2024-09-10T23:57:15.284 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:14 smithi165 ceph-mon[35356]: mgrmap e13: x(active, since 56s) 2024-09-10T23:57:15.285 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:14 smithi165 ceph-mon[35356]: overall HEALTH_OK 2024-09-10T23:57:15.285 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:14 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-10T23:57:15.285 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:14 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-10T23:57:15.285 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:14 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-10T23:57:15.285 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:14 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-10T23:57:15.285 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:14 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-10T23:57:15.285 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:14 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-10T23:57:15.285 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:14 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-10T23:57:15.285 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:14 smithi165 ceph-mon[35356]: Reconfiguring mon.a (unknown last config time)... 2024-09-10T23:57:15.285 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:14 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-10T23:57:15.285 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:14 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-09-10T23:57:15.285 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:14 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T23:57:15.286 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:14 smithi165 ceph-mon[35356]: Reconfiguring daemon mon.a on smithi012 2024-09-10T23:57:15.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:57:14 smithi012 ceph-mon[30333]: mon.b calling monitor election 2024-09-10T23:57:15.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:57:14 smithi012 ceph-mon[30333]: Updating smithi012:/var/lib/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T23:57:15.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:57:14 smithi012 ceph-mon[30333]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T23:57:15.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:57:14 smithi012 ceph-mon[30333]: mon.b calling monitor election 2024-09-10T23:57:15.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:57:14 smithi012 ceph-mon[30333]: mon.c calling monitor election 2024-09-10T23:57:15.314 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:57:14 smithi012 ceph-mon[30333]: mon.a calling monitor election 2024-09-10T23:57:15.314 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:57:14 smithi012 ceph-mon[30333]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-09-10T23:57:15.314 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:57:14 smithi012 ceph-mon[30333]: monmap epoch 3 2024-09-10T23:57:15.314 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:57:14 smithi012 ceph-mon[30333]: fsid ff3ad74a-6fcf-11ef-bcea-c7b262605968 2024-09-10T23:57:15.314 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:57:14 smithi012 ceph-mon[30333]: last_changed 2024-09-10T23:57:07.011167+0000 2024-09-10T23:57:15.314 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:57:14 smithi012 ceph-mon[30333]: created 2024-09-10T23:55:12.644680+0000 2024-09-10T23:57:15.314 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:57:14 smithi012 ceph-mon[30333]: min_mon_release 19 (squid) 2024-09-10T23:57:15.314 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:57:14 smithi012 ceph-mon[30333]: election_strategy: 1 2024-09-10T23:57:15.314 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:57:14 smithi012 ceph-mon[30333]: 0: [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] mon.a 2024-09-10T23:57:15.314 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:57:14 smithi012 ceph-mon[30333]: 1: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.c 2024-09-10T23:57:15.314 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:57:14 smithi012 ceph-mon[30333]: 2: [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] mon.b 2024-09-10T23:57:15.314 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:57:14 smithi012 ceph-mon[30333]: fsmap 2024-09-10T23:57:15.315 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:57:14 smithi012 ceph-mon[30333]: osdmap e4: 0 total, 0 up, 0 in 2024-09-10T23:57:15.315 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:57:14 smithi012 ceph-mon[30333]: mgrmap e13: x(active, since 56s) 2024-09-10T23:57:15.315 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:57:14 smithi012 ceph-mon[30333]: overall HEALTH_OK 2024-09-10T23:57:15.315 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:57:14 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-10T23:57:15.315 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:57:14 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-10T23:57:15.315 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:57:14 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-10T23:57:15.315 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:57:14 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-10T23:57:15.315 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:57:14 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-10T23:57:15.315 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:57:14 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-10T23:57:15.315 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:57:14 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-10T23:57:15.315 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:57:14 smithi012 ceph-mon[30333]: Reconfiguring mon.a (unknown last config time)... 2024-09-10T23:57:15.316 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:57:14 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-10T23:57:15.316 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:57:14 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-09-10T23:57:15.316 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:57:14 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T23:57:15.316 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:57:14 smithi012 ceph-mon[30333]: Reconfiguring daemon mon.a on smithi012 2024-09-10T23:57:15.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:57:14 smithi178 ceph-mon[35919]: mon.b calling monitor election 2024-09-10T23:57:15.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:57:14 smithi178 ceph-mon[35919]: Updating smithi012:/var/lib/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T23:57:15.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:57:14 smithi178 ceph-mon[35919]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T23:57:15.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:57:14 smithi178 ceph-mon[35919]: mon.b calling monitor election 2024-09-10T23:57:15.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:57:14 smithi178 ceph-mon[35919]: mon.c calling monitor election 2024-09-10T23:57:15.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:57:14 smithi178 ceph-mon[35919]: mon.a calling monitor election 2024-09-10T23:57:15.332 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:57:14 smithi178 ceph-mon[35919]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-09-10T23:57:15.332 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:57:14 smithi178 ceph-mon[35919]: monmap epoch 3 2024-09-10T23:57:15.332 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:57:14 smithi178 ceph-mon[35919]: fsid ff3ad74a-6fcf-11ef-bcea-c7b262605968 2024-09-10T23:57:15.332 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:57:14 smithi178 ceph-mon[35919]: last_changed 2024-09-10T23:57:07.011167+0000 2024-09-10T23:57:15.332 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:57:14 smithi178 ceph-mon[35919]: created 2024-09-10T23:55:12.644680+0000 2024-09-10T23:57:15.332 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:57:14 smithi178 ceph-mon[35919]: min_mon_release 19 (squid) 2024-09-10T23:57:15.332 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:57:14 smithi178 ceph-mon[35919]: election_strategy: 1 2024-09-10T23:57:15.332 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:57:14 smithi178 ceph-mon[35919]: 0: [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] mon.a 2024-09-10T23:57:15.332 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:57:14 smithi178 ceph-mon[35919]: 1: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.c 2024-09-10T23:57:15.332 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:57:14 smithi178 ceph-mon[35919]: 2: [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] mon.b 2024-09-10T23:57:15.332 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:57:14 smithi178 ceph-mon[35919]: fsmap 2024-09-10T23:57:15.332 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:57:14 smithi178 ceph-mon[35919]: osdmap e4: 0 total, 0 up, 0 in 2024-09-10T23:57:15.332 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:57:14 smithi178 ceph-mon[35919]: mgrmap e13: x(active, since 56s) 2024-09-10T23:57:15.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:57:14 smithi178 ceph-mon[35919]: overall HEALTH_OK 2024-09-10T23:57:15.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:57:14 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-10T23:57:15.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:57:14 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-10T23:57:15.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:57:14 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-10T23:57:15.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:57:14 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-10T23:57:15.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:57:14 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-10T23:57:15.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:57:14 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-10T23:57:15.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:57:14 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-10T23:57:15.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:57:14 smithi178 ceph-mon[35919]: Reconfiguring mon.a (unknown last config time)... 2024-09-10T23:57:15.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:57:14 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-10T23:57:15.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:57:14 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-09-10T23:57:15.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:57:14 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T23:57:15.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:57:14 smithi178 ceph-mon[35919]: Reconfiguring daemon mon.a on smithi012 2024-09-10T23:57:15.949 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-09-10T23:57:15.949 DEBUG:teuthology.orchestra.run.smithi012:> set -ex 2024-09-10T23:57:15.949 DEBUG:teuthology.orchestra.run.smithi012:> sudo dd of=/etc/ceph/ceph.conf 2024-09-10T23:57:16.008 DEBUG:teuthology.orchestra.run.smithi012:> set -ex 2024-09-10T23:57:16.008 DEBUG:teuthology.orchestra.run.smithi012:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-10T23:57:16.106 DEBUG:teuthology.orchestra.run.smithi165:> set -ex 2024-09-10T23:57:16.106 DEBUG:teuthology.orchestra.run.smithi165:> sudo dd of=/etc/ceph/ceph.conf 2024-09-10T23:57:16.129 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:15 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-10T23:57:16.129 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:15 smithi165 ceph-mon[35356]: from='client.? 172.21.15.12:0/2779616409' entity='client.admin' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T23:57:16.134 DEBUG:teuthology.orchestra.run.smithi165:> set -ex 2024-09-10T23:57:16.134 DEBUG:teuthology.orchestra.run.smithi165:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-10T23:57:16.199 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2024-09-10T23:57:16.199 DEBUG:teuthology.orchestra.run.smithi178:> sudo dd of=/etc/ceph/ceph.conf 2024-09-10T23:57:16.231 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:57:15 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-10T23:57:16.231 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:57:15 smithi178 ceph-mon[35919]: from='client.? 172.21.15.12:0/2779616409' entity='client.admin' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T23:57:16.232 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2024-09-10T23:57:16.232 DEBUG:teuthology.orchestra.run.smithi178:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-10T23:57:16.302 INFO:tasks.cephadm:Adding mgr.x on smithi012 2024-09-10T23:57:16.302 INFO:tasks.cephadm:Adding mgr.y on smithi165 2024-09-10T23:57:16.302 INFO:tasks.cephadm:Adding mgr.z on smithi178 2024-09-10T23:57:16.302 DEBUG:teuthology.orchestra.run.smithi178:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ff3ad74a-6fcf-11ef-bcea-c7b262605968 -- ceph orch apply mgr '3;smithi012=x;smithi165=y;smithi178=z' 2024-09-10T23:57:16.312 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Sep 10 23:57:16 smithi012 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-mgr-x[30560]: 2024-09-10T23:57:16.010+0000 7fa00e600640 -1 mgr.server handle_report got status from non-daemon mon.b 2024-09-10T23:57:16.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:57:15 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-10T23:57:16.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:57:15 smithi012 ceph-mon[30333]: from='client.? 172.21.15.12:0/2779616409' entity='client.admin' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T23:57:16.584 INFO:teuthology.orchestra.run.smithi178.stderr:Inferring config /var/lib/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/mon.c/config 2024-09-10T23:57:17.285 INFO:teuthology.orchestra.run.smithi178.stdout:Scheduled mgr update... 2024-09-10T23:57:17.812 DEBUG:teuthology.orchestra.run.smithi165:mgr.y> sudo journalctl -f -n 0 -u ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968@mgr.y.service 2024-09-10T23:57:17.815 DEBUG:teuthology.orchestra.run.smithi178:mgr.z> sudo journalctl -f -n 0 -u ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968@mgr.z.service 2024-09-10T23:57:17.817 INFO:tasks.cephadm:setting config [client] client_mount_timeout = 600 2024-09-10T23:57:17.818 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ff3ad74a-6fcf-11ef-bcea-c7b262605968 -- ceph config set client client_mount_timeout 600 2024-09-10T23:57:17.822 INFO:tasks.cephadm:setting config [client] debug_client = 20 2024-09-10T23:57:17.822 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ff3ad74a-6fcf-11ef-bcea-c7b262605968 -- ceph config set client debug_client 20 2024-09-10T23:57:17.830 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:57:17 smithi178 ceph-mon[35919]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T23:57:17.831 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:57:17 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-10T23:57:17.831 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:57:17 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-10T23:57:17.831 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:57:17 smithi178 ceph-mon[35919]: Reconfiguring mon.b (monmap changed)... 2024-09-10T23:57:17.831 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:57:17 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-10T23:57:17.831 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:57:17 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-09-10T23:57:17.831 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:57:17 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T23:57:17.831 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:57:17 smithi178 ceph-mon[35919]: Reconfiguring daemon mon.b on smithi165 2024-09-10T23:57:17.831 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:57:17 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-10T23:57:17.866 INFO:tasks.cephadm:setting config [client] debug_ms = 1 2024-09-10T23:57:17.866 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ff3ad74a-6fcf-11ef-bcea-c7b262605968 -- ceph config set client debug_ms 1 2024-09-10T23:57:17.912 INFO:tasks.cephadm:setting config [client] rados_mon_op_timeout = 900 2024-09-10T23:57:17.913 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ff3ad74a-6fcf-11ef-bcea-c7b262605968 -- ceph config set client rados_mon_op_timeout 900 2024-09-10T23:57:17.959 INFO:tasks.cephadm:setting config [client] rados_osd_op_timeout = 900 2024-09-10T23:57:17.959 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ff3ad74a-6fcf-11ef-bcea-c7b262605968 -- ceph config set client rados_osd_op_timeout 900 2024-09-10T23:57:18.005 INFO:tasks.cephadm:setting config [mds] debug_mds = 20 2024-09-10T23:57:18.005 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ff3ad74a-6fcf-11ef-bcea-c7b262605968 -- ceph config set mds debug_mds 20 2024-09-10T23:57:18.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:17 smithi165 ceph-mon[35356]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T23:57:18.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:17 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-10T23:57:18.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:17 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-10T23:57:18.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:17 smithi165 ceph-mon[35356]: Reconfiguring mon.b (monmap changed)... 2024-09-10T23:57:18.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:17 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-10T23:57:18.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:17 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-09-10T23:57:18.033 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:17 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T23:57:18.033 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:17 smithi165 ceph-mon[35356]: Reconfiguring daemon mon.b on smithi165 2024-09-10T23:57:18.033 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:17 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-10T23:57:18.052 INFO:tasks.cephadm:setting config [mds] debug_mds_balancer = 20 2024-09-10T23:57:18.052 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ff3ad74a-6fcf-11ef-bcea-c7b262605968 -- ceph config set mds debug_mds_balancer 20 2024-09-10T23:57:18.063 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:57:17 smithi012 ceph-mon[30333]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T23:57:18.063 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:57:17 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-10T23:57:18.063 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:57:17 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-10T23:57:18.063 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:57:17 smithi012 ceph-mon[30333]: Reconfiguring mon.b (monmap changed)... 2024-09-10T23:57:18.063 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:57:17 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-10T23:57:18.063 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:57:17 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-09-10T23:57:18.063 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:57:17 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T23:57:18.063 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:57:17 smithi012 ceph-mon[30333]: Reconfiguring daemon mon.b on smithi165 2024-09-10T23:57:18.064 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:57:17 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-10T23:57:18.070 INFO:tasks.cephadm:setting config [mds] debug_ms = 2 2024-09-10T23:57:18.070 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ff3ad74a-6fcf-11ef-bcea-c7b262605968 -- ceph config set mds debug_ms 2 2024-09-10T23:57:18.122 INFO:tasks.cephadm:setting config [mds] mds_bal_fragment_size_max = 10000 2024-09-10T23:57:18.122 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ff3ad74a-6fcf-11ef-bcea-c7b262605968 -- ceph config set mds mds_bal_fragment_size_max 10000 2024-09-10T23:57:18.173 INFO:tasks.cephadm:setting config [mds] mds_bal_merge_size = 5 2024-09-10T23:57:18.173 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ff3ad74a-6fcf-11ef-bcea-c7b262605968 -- ceph config set mds mds_bal_merge_size 5 2024-09-10T23:57:18.225 INFO:tasks.cephadm:setting config [mds] mds_bal_split_bits = 3 2024-09-10T23:57:18.225 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ff3ad74a-6fcf-11ef-bcea-c7b262605968 -- ceph config set mds mds_bal_split_bits 3 2024-09-10T23:57:18.283 INFO:tasks.cephadm:setting config [mds] mds_bal_split_size = 100 2024-09-10T23:57:18.283 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ff3ad74a-6fcf-11ef-bcea-c7b262605968 -- ceph config set mds mds_bal_split_size 100 2024-09-10T23:57:18.338 INFO:tasks.cephadm:setting config [mds] mds_debug_frag = True 2024-09-10T23:57:18.338 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ff3ad74a-6fcf-11ef-bcea-c7b262605968 -- ceph config set mds mds_debug_frag True 2024-09-10T23:57:18.407 INFO:tasks.cephadm:setting config [mds] mds_debug_scatterstat = True 2024-09-10T23:57:18.407 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ff3ad74a-6fcf-11ef-bcea-c7b262605968 -- ceph config set mds mds_debug_scatterstat True 2024-09-10T23:57:18.468 INFO:tasks.cephadm:setting config [mds] mds_op_complaint_time = 180 2024-09-10T23:57:18.468 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ff3ad74a-6fcf-11ef-bcea-c7b262605968 -- ceph config set mds mds_op_complaint_time 180 2024-09-10T23:57:18.530 INFO:tasks.cephadm:setting config [mds] mds_verify_scatter = True 2024-09-10T23:57:18.531 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ff3ad74a-6fcf-11ef-bcea-c7b262605968 -- ceph config set mds mds_verify_scatter True 2024-09-10T23:57:18.580 INFO:tasks.cephadm:setting config [mds] osd_op_complaint_time = 180 2024-09-10T23:57:18.580 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ff3ad74a-6fcf-11ef-bcea-c7b262605968 -- ceph config set mds osd_op_complaint_time 180 2024-09-10T23:57:18.638 INFO:tasks.cephadm:setting config [mds] rados_mon_op_timeout = 900 2024-09-10T23:57:18.638 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ff3ad74a-6fcf-11ef-bcea-c7b262605968 -- ceph config set mds rados_mon_op_timeout 900 2024-09-10T23:57:18.642 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T23:57:18.642 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T23:57:18.704 INFO:tasks.cephadm:setting config [mds] rados_osd_op_timeout = 900 2024-09-10T23:57:18.704 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ff3ad74a-6fcf-11ef-bcea-c7b262605968 -- ceph config set mds rados_osd_op_timeout 900 2024-09-10T23:57:18.761 INFO:tasks.cephadm:setting config [mon] mon_op_complaint_time = 120 2024-09-10T23:57:18.761 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ff3ad74a-6fcf-11ef-bcea-c7b262605968 -- ceph config set mon mon_op_complaint_time 120 2024-09-10T23:57:18.829 INFO:tasks.cephadm:setting config [mon] mon_warn_on_pool_no_app = False 2024-09-10T23:57:18.829 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ff3ad74a-6fcf-11ef-bcea-c7b262605968 -- ceph config set mon mon_warn_on_pool_no_app False 2024-09-10T23:57:18.864 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T23:57:18.873 INFO:tasks.cephadm:setting config [osd] osd_op_complaint_time = 180 2024-09-10T23:57:18.873 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ff3ad74a-6fcf-11ef-bcea-c7b262605968 -- ceph config set osd osd_op_complaint_time 180 2024-09-10T23:57:18.905 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:57:18 smithi012 ceph-mon[30333]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi012=x;smithi165=y;smithi178=z", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T23:57:18.905 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:57:18 smithi012 ceph-mon[30333]: Saving service mgr spec with placement smithi012=x;smithi165=y;smithi178=z;count:3 2024-09-10T23:57:18.905 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:57:18 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-10T23:57:18.905 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:57:18 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-10T23:57:18.905 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:57:18 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-10T23:57:18.905 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:57:18 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-09-10T23:57:18.905 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:57:18 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T23:57:18.917 DEBUG:tasks.cephadm:set 22 configs 2024-09-10T23:57:18.917 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'ff3ad74a-6fcf-11ef-bcea-c7b262605968', '--', 'ceph', 'config', 'set', 'client', 'client_mount_timeout', '600'], check_status=True, hostname='smithi012') 2024-09-10T23:57:19.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:57:18 smithi178 ceph-mon[35919]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi012=x;smithi165=y;smithi178=z", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T23:57:19.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:57:18 smithi178 ceph-mon[35919]: Saving service mgr spec with placement smithi012=x;smithi165=y;smithi178=z;count:3 2024-09-10T23:57:19.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:57:18 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-10T23:57:19.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:57:18 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-10T23:57:19.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:57:18 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-10T23:57:19.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:57:18 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-09-10T23:57:19.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:57:18 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T23:57:19.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:18 smithi165 ceph-mon[35356]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi012=x;smithi165=y;smithi178=z", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T23:57:19.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:18 smithi165 ceph-mon[35356]: Saving service mgr spec with placement smithi012=x;smithi165=y;smithi178=z;count:3 2024-09-10T23:57:19.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:18 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-10T23:57:19.283 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:18 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-10T23:57:19.283 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:18 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-10T23:57:19.283 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:18 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-09-10T23:57:19.283 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:18 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T23:57:19.852 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T23:57:20.013 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:57:19 smithi012 ceph-mon[30333]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T23:57:20.013 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:57:19 smithi012 ceph-mon[30333]: Reconfiguring mon.c (monmap changed)... 2024-09-10T23:57:20.013 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:57:19 smithi012 ceph-mon[30333]: Reconfiguring daemon mon.c on smithi178 2024-09-10T23:57:20.013 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:57:19 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-10T23:57:20.013 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:57:19 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-10T23:57:20.014 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:57:19 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T23:57:20.014 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:57:19 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T23:57:20.014 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:57:19 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T23:57:20.014 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:57:19 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-10T23:57:20.014 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:57:19 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]} : dispatch 2024-09-10T23:57:20.014 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:57:19 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-10T23:57:20.014 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:57:19 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "mgr services"} : dispatch 2024-09-10T23:57:20.014 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:57:19 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T23:57:20.067 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:57:19 smithi178 ceph-mon[35919]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T23:57:20.068 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:57:19 smithi178 ceph-mon[35919]: Reconfiguring mon.c (monmap changed)... 2024-09-10T23:57:20.068 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:57:19 smithi178 ceph-mon[35919]: Reconfiguring daemon mon.c on smithi178 2024-09-10T23:57:20.068 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:57:19 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-10T23:57:20.068 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:57:19 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-10T23:57:20.068 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:57:19 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T23:57:20.068 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:57:19 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T23:57:20.068 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:57:19 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T23:57:20.068 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:57:19 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-10T23:57:20.068 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:57:19 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]} : dispatch 2024-09-10T23:57:20.068 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:57:19 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-10T23:57:20.068 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:57:19 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "mgr services"} : dispatch 2024-09-10T23:57:20.068 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:57:19 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T23:57:20.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:19 smithi165 ceph-mon[35356]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T23:57:20.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:19 smithi165 ceph-mon[35356]: Reconfiguring mon.c (monmap changed)... 2024-09-10T23:57:20.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:19 smithi165 ceph-mon[35356]: Reconfiguring daemon mon.c on smithi178 2024-09-10T23:57:20.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:19 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-10T23:57:20.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:19 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-10T23:57:20.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:19 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T23:57:20.283 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:19 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T23:57:20.283 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:19 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T23:57:20.283 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:19 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-10T23:57:20.283 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:19 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]} : dispatch 2024-09-10T23:57:20.283 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:19 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-10T23:57:20.283 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:19 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "mgr services"} : dispatch 2024-09-10T23:57:20.283 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:19 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T23:57:20.899 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:57:20 smithi178 ceph-mon[35919]: Deploying daemon mgr.z on smithi178 2024-09-10T23:57:20.899 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:57:20 smithi178 ceph-mon[35919]: from='client.? ' entity='client.admin' 2024-09-10T23:57:20.959 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:57:20 smithi012 ceph-mon[30333]: Deploying daemon mgr.z on smithi178 2024-09-10T23:57:20.960 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:57:20 smithi012 ceph-mon[30333]: from='client.? ' entity='client.admin' 2024-09-10T23:57:21.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:20 smithi165 ceph-mon[35356]: Deploying daemon mgr.z on smithi178 2024-09-10T23:57:21.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:20 smithi165 ceph-mon[35356]: from='client.? ' entity='client.admin' 2024-09-10T23:57:21.296 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:21.293+0000 7f70f9800640 1 -- 172.21.15.12:0/2197280803 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f70d803f510 msgr2=0x7f70d80419d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:57:21.296 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:21.293+0000 7f70f9800640 1 --2- 172.21.15.12:0/2197280803 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f70d803f510 0x7f70d80419d0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f70e40095f0 tx=0x7f70e4005f90 comp rx=0 tx=0).stop 2024-09-10T23:57:21.296 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:21.293+0000 7f70f9800640 1 -- 172.21.15.12:0/2197280803 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f70f41a0350 msgr2=0x7f70f41a0800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:57:21.296 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:21.293+0000 7f70f9800640 1 --2- 172.21.15.12:0/2197280803 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f70f41a0350 0x7f70f41a0800 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f70ec009fd0 tx=0x7f70ec00ef10 comp rx=0 tx=0).stop 2024-09-10T23:57:21.296 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:21.293+0000 7f70f9800640 1 -- 172.21.15.12:0/2197280803 shutdown_connections 2024-09-10T23:57:21.296 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:21.293+0000 7f70f9800640 1 --2- 172.21.15.12:0/2197280803 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f70d803f510 0x7f70d80419d0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:57:21.296 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:21.293+0000 7f70f9800640 1 --2- 172.21.15.12:0/2197280803 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f70f41a0350 0x7f70f41a0800 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:57:21.296 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:21.293+0000 7f70f9800640 1 --2- 172.21.15.12:0/2197280803 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f70f4076f80 0x7f70f419f710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:57:21.297 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:21.293+0000 7f70f9800640 1 --2- 172.21.15.12:0/2197280803 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f70f4075b20 0x7f70f419f0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:57:21.297 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:21.293+0000 7f70f9800640 1 -- 172.21.15.12:0/2197280803 >> 172.21.15.12:0/2197280803 conn(0x7f70f40fe320 msgr2=0x7f70f40fea30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:57:21.297 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:21.293+0000 7f70f9800640 1 -- 172.21.15.12:0/2197280803 shutdown_connections 2024-09-10T23:57:21.297 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:21.293+0000 7f70f9800640 1 -- 172.21.15.12:0/2197280803 wait complete. 2024-09-10T23:57:21.496 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:21.494+0000 7fb0b1a00640 1 -- 172.21.15.12:0/710098661 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command([{prefix=config set, name=debug_client}] v 0) -- 0x7fb0ac109bb0 con 0x7fb0ac102890 2024-09-10T23:57:21.501 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:21.499+0000 7fb0a8c00640 1 -- 172.21.15.12:0/710098661 <== mon.1 v2:172.21.15.178:3300/0 8 ==== config(24 keys) ==== 1039+0+0 (secure 0 0 0) 0x7fb09c00fcc0 con 0x7fb0ac102890 2024-09-10T23:57:21.501 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:21.499+0000 7fb0a8c00640 1 -- 172.21.15.12:0/710098661 <== mon.1 v2:172.21.15.178:3300/0 9 ==== mon_command_ack([{prefix=config set, name=debug_client}]=0 v12)=0 v12) ==== 114+0+0 (secure 0 0 0) 0x7fb09c003830 con 0x7fb0ac102890 2024-09-10T23:57:21.503 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:21.501+0000 7fb0b1a00640 1 -- 172.21.15.12:0/710098661 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fb09003f5c0 msgr2=0x7fb090041a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:57:21.503 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:21.501+0000 7fb0b1a00640 1 --2- 172.21.15.12:0/710098661 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fb09003f5c0 0x7fb090041a80 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fb098006010 tx=0x7fb098005a20 comp rx=0 tx=0).stop 2024-09-10T23:57:21.503 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:21.501+0000 7fb0b1a00640 1 -- 172.21.15.12:0/710098661 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb0ac102890 msgr2=0x7fb0ac076cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:57:21.504 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:21.501+0000 7fb0b1a00640 1 --2- 172.21.15.12:0/710098661 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb0ac102890 0x7fb0ac076cd0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fb09c00ea60 tx=0x7fb09c00ef20 comp rx=0 tx=0).stop 2024-09-10T23:57:21.504 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:21.501+0000 7fb0b1a00640 1 -- 172.21.15.12:0/710098661 shutdown_connections 2024-09-10T23:57:21.504 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:21.501+0000 7fb0b1a00640 1 --2- 172.21.15.12:0/710098661 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fb09003f5c0 0x7fb090041a80 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:57:21.504 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:21.501+0000 7fb0b1a00640 1 --2- 172.21.15.12:0/710098661 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb0ac1043b0 0x7fb0ac075a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:57:21.504 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:21.501+0000 7fb0b1a00640 1 --2- 172.21.15.12:0/710098661 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb0ac1039f0 0x7fb0ac075540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:57:21.504 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:21.501+0000 7fb0b1a00640 1 --2- 172.21.15.12:0/710098661 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb0ac102890 0x7fb0ac076cd0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:57:21.504 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:21.501+0000 7fb0b1a00640 1 -- 172.21.15.12:0/710098661 >> 172.21.15.12:0/710098661 conn(0x7fb0ac0fe180 msgr2=0x7fb0ac106d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:57:21.504 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:21.501+0000 7fb0b1a00640 1 -- 172.21.15.12:0/710098661 shutdown_connections 2024-09-10T23:57:21.504 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:21.502+0000 7fb0b1a00640 1 -- 172.21.15.12:0/710098661 wait complete. 2024-09-10T23:57:21.569 INFO:journalctl@ceph.mgr.z.smithi178.stdout:Sep 10 23:57:21 smithi178 systemd[1]: Starting Ceph mgr.z for ff3ad74a-6fcf-11ef-bcea-c7b262605968... 2024-09-10T23:57:21.823 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:57:21 smithi178 ceph-mon[35919]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T23:57:21.823 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:57:21 smithi178 ceph-mon[35919]: from='client.? 172.21.15.12:0/2197280803' entity='client.admin' 2024-09-10T23:57:21.823 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:57:21 smithi178 ceph-mon[35919]: from='client.? ' entity='client.admin' 2024-09-10T23:57:21.824 INFO:journalctl@ceph.mgr.z.smithi178.stdout:Sep 10 23:57:21 smithi178 podman[37491]: 2024-09-10 23:57:21.470883545 +0000 UTC m=+0.015653419 image pull 589dd18248e494ba411dc838aa4ae0e5e569213f08fcd7a2cf22cd3ef7e3c480 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e 2024-09-10T23:57:21.824 INFO:journalctl@ceph.mgr.z.smithi178.stdout:Sep 10 23:57:21 smithi178 podman[37491]: 2024-09-10 23:57:21.609411383 +0000 UTC m=+0.154181248 container create 14fc6152c232862818efb28ca50714ad767f41bc99ea943105835cf378a96bae (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-mgr-z, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=) 2024-09-10T23:57:21.824 INFO:journalctl@ceph.mgr.z.smithi178.stdout:Sep 10 23:57:21 smithi178 podman[37491]: 2024-09-10 23:57:21.795222771 +0000 UTC m=+0.339992637 container init 14fc6152c232862818efb28ca50714ad767f41bc99ea943105835cf378a96bae (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-mgr-z, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-10T23:57:21.824 INFO:journalctl@ceph.mgr.z.smithi178.stdout:Sep 10 23:57:21 smithi178 podman[37491]: 2024-09-10 23:57:21.798113346 +0000 UTC m=+0.342883224 container start 14fc6152c232862818efb28ca50714ad767f41bc99ea943105835cf378a96bae (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-mgr-z, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-10T23:57:21.824 INFO:journalctl@ceph.mgr.z.smithi178.stdout:Sep 10 23:57:21 smithi178 ceph-mgr[37511]: -- 172.21.15.178:0/2108101369 <== mon.1 v2:172.21.15.178:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x556073ba90e0 con 0x556073b87000 2024-09-10T23:57:21.948 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:57:21 smithi012 ceph-mon[30333]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T23:57:21.948 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:57:21 smithi012 ceph-mon[30333]: from='client.? 172.21.15.12:0/2197280803' entity='client.admin' 2024-09-10T23:57:21.948 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:57:21 smithi012 ceph-mon[30333]: from='client.? ' entity='client.admin' 2024-09-10T23:57:22.081 INFO:journalctl@ceph.mgr.z.smithi178.stdout:Sep 10 23:57:21 smithi178 bash[37491]: 14fc6152c232862818efb28ca50714ad767f41bc99ea943105835cf378a96bae 2024-09-10T23:57:22.081 INFO:journalctl@ceph.mgr.z.smithi178.stdout:Sep 10 23:57:21 smithi178 systemd[1]: Started Ceph mgr.z for ff3ad74a-6fcf-11ef-bcea-c7b262605968. 2024-09-10T23:57:22.267 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:22.055+0000 7fa130200640 1 -- 172.21.15.12:0/424167844 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa128075aa0 msgr2=0x7fa128075f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:57:22.267 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:22.055+0000 7fa130200640 1 --2- 172.21.15.12:0/424167844 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa128075aa0 0x7fa128075f20 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fa114009860 tx=0x7fa11402a340 comp rx=0 tx=0).stop 2024-09-10T23:57:22.267 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:22.056+0000 7fa130200640 1 -- 172.21.15.12:0/424167844 shutdown_connections 2024-09-10T23:57:22.267 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:22.056+0000 7fa130200640 1 --2- 172.21.15.12:0/424167844 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa12810a9d0 0x7fa12810ce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:57:22.267 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:22.056+0000 7fa130200640 1 --2- 172.21.15.12:0/424167844 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa128075aa0 0x7fa128075f20 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:57:22.267 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:22.056+0000 7fa130200640 1 --2- 172.21.15.12:0/424167844 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa128076f00 0x7fa128075560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:57:22.267 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:22.056+0000 7fa130200640 1 -- 172.21.15.12:0/424167844 >> 172.21.15.12:0/424167844 conn(0x7fa1280fe2f0 msgr2=0x7fa128100730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:57:22.268 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:22.056+0000 7fa130200640 1 -- 172.21.15.12:0/424167844 shutdown_connections 2024-09-10T23:57:22.268 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:22.056+0000 7fa130200640 1 -- 172.21.15.12:0/424167844 wait complete. 2024-09-10T23:57:22.268 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:22.057+0000 7fa130200640 1 Processor -- start 2024-09-10T23:57:22.268 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:22.057+0000 7fa130200640 1 -- start start 2024-09-10T23:57:22.268 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:22.057+0000 7fa130200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa128075aa0 0x7fa12819ae40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:57:22.268 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:22.057+0000 7fa130200640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa128076f00 0x7fa12819b490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:57:22.268 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:22.057+0000 7fa130200640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa12810a9d0 0x7fa12819c0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:57:22.268 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:22.057+0000 7fa130200640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fa128110630 con 0x7fa12810a9d0 2024-09-10T23:57:22.268 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:22.057+0000 7fa130200640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fa1281104b0 con 0x7fa128075aa0 2024-09-10T23:57:22.268 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:22.057+0000 7fa130200640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fa1281107b0 con 0x7fa128076f00 2024-09-10T23:57:22.268 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:22.057+0000 7fa12d800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa128075aa0 0x7fa12819ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:57:22.268 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:22.057+0000 7fa12d800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa128075aa0 0x7fa12819ae40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.12:59214/0 (socket says 172.21.15.12:59214) 2024-09-10T23:57:22.269 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:22.057+0000 7fa12d800640 1 -- 172.21.15.12:0/1973928206 learned_addr learned my addr 172.21.15.12:0/1973928206 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-10T23:57:22.269 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:22.057+0000 7fa12ce00640 1 --2- 172.21.15.12:0/1973928206 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa128076f00 0x7fa12819b490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:57:22.269 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:22.057+0000 7fa12e200640 1 --2- 172.21.15.12:0/1973928206 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa12810a9d0 0x7fa12819c0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:57:22.269 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:22.058+0000 7fa12d800640 1 -- 172.21.15.12:0/1973928206 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa128076f00 msgr2=0x7fa12819b490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:57:22.269 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:22.058+0000 7fa12d800640 1 --2- 172.21.15.12:0/1973928206 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa128076f00 0x7fa12819b490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:57:22.269 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:22.058+0000 7fa12d800640 1 -- 172.21.15.12:0/1973928206 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa12810a9d0 msgr2=0x7fa12819c0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:57:22.269 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:22.058+0000 7fa12d800640 1 --2- 172.21.15.12:0/1973928206 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa12810a9d0 0x7fa12819c0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:57:22.269 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:22.058+0000 7fa12d800640 1 -- 172.21.15.12:0/1973928206 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa12819c970 con 0x7fa128075aa0 2024-09-10T23:57:22.269 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:22.058+0000 7fa12ce00640 1 --2- 172.21.15.12:0/1973928206 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa128076f00 0x7fa12819b490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:57:22.269 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:22.058+0000 7fa12d800640 1 --2- 172.21.15.12:0/1973928206 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa128075aa0 0x7fa12819ae40 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fa11c00e9c0 tx=0x7fa11c00ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:57:22.269 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:22.058+0000 7fa126000640 1 -- 172.21.15.12:0/1973928206 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa11c00ccc0 con 0x7fa128075aa0 2024-09-10T23:57:22.269 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:22.058+0000 7fa130200640 1 -- 172.21.15.12:0/1973928206 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa12819cc60 con 0x7fa128075aa0 2024-09-10T23:57:22.269 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:22.058+0000 7fa126000640 1 -- 172.21.15.12:0/1973928206 <== mon.2 v2:172.21.15.165:3300/0 2 ==== config(24 keys) ==== 1039+0+0 (secure 0 0 0) 0x7fa11c005490 con 0x7fa128075aa0 2024-09-10T23:57:22.269 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:22.058+0000 7fa126000640 1 -- 172.21.15.12:0/1973928206 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa11c004e60 con 0x7fa128075aa0 2024-09-10T23:57:22.270 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:22.058+0000 7fa130200640 1 -- 172.21.15.12:0/1973928206 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1281a3340 con 0x7fa128075aa0 2024-09-10T23:57:22.270 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:22.060+0000 7fa130200640 1 -- 172.21.15.12:0/1973928206 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa128075f20 con 0x7fa128075aa0 2024-09-10T23:57:22.270 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:22.060+0000 7fa126000640 1 -- 172.21.15.12:0/1973928206 <== mon.2 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 13) ==== 51480+0+0 (secure 0 0 0) 0x7fa11c005020 con 0x7fa128075aa0 2024-09-10T23:57:22.270 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:22.060+0000 7fa126000640 1 --2- 172.21.15.12:0/1973928206 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fa11003f510 0x7fa1100419d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:57:22.270 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:22.060+0000 7fa126000640 1 -- 172.21.15.12:0/1973928206 <== mon.2 v2:172.21.15.165:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fa11c0540a0 con 0x7fa128075aa0 2024-09-10T23:57:22.270 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:22.060+0000 7fa12ce00640 1 --2- 172.21.15.12:0/1973928206 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fa11003f510 0x7fa1100419d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:57:22.270 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:22.061+0000 7fa12ce00640 1 --2- 172.21.15.12:0/1973928206 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fa11003f510 0x7fa1100419d0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fa114009830 tx=0x7fa1140058b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:57:22.270 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:22.065+0000 7fa126000640 1 -- 172.21.15.12:0/1973928206 <== mon.2 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7fa11c01fd20 con 0x7fa128075aa0 2024-09-10T23:57:22.270 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:22.264+0000 7fa130200640 1 -- 172.21.15.12:0/1973928206 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command([{prefix=config set, name=rados_mon_op_timeout}] v 0) -- 0x7fa12802e5c0 con 0x7fa128075aa0 2024-09-10T23:57:22.272 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:22.270+0000 7fa126000640 1 -- 172.21.15.12:0/1973928206 <== mon.2 v2:172.21.15.165:3300/0 7 ==== config(25 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fa11c011cf0 con 0x7fa128075aa0 2024-09-10T23:57:22.272 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:22.270+0000 7fa126000640 1 -- 172.21.15.12:0/1973928206 <== mon.2 v2:172.21.15.165:3300/0 8 ==== mon_command_ack([{prefix=config set, name=rados_mon_op_timeout}]=0 v13)=0 v13) ==== 123+0+0 (secure 0 0 0) 0x7fa11c020ca0 con 0x7fa128075aa0 2024-09-10T23:57:22.275 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:22.273+0000 7fa130200640 1 -- 172.21.15.12:0/1973928206 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fa11003f510 msgr2=0x7fa1100419d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:57:22.275 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:22.273+0000 7fa130200640 1 --2- 172.21.15.12:0/1973928206 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fa11003f510 0x7fa1100419d0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fa114009830 tx=0x7fa1140058b0 comp rx=0 tx=0).stop 2024-09-10T23:57:22.275 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:22.273+0000 7fa130200640 1 -- 172.21.15.12:0/1973928206 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa128075aa0 msgr2=0x7fa12819ae40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:57:22.275 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:22.273+0000 7fa130200640 1 --2- 172.21.15.12:0/1973928206 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa128075aa0 0x7fa12819ae40 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fa11c00e9c0 tx=0x7fa11c00ee80 comp rx=0 tx=0).stop 2024-09-10T23:57:22.275 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:22.273+0000 7fa130200640 1 -- 172.21.15.12:0/1973928206 shutdown_connections 2024-09-10T23:57:22.276 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:22.273+0000 7fa130200640 1 --2- 172.21.15.12:0/1973928206 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fa11003f510 0x7fa1100419d0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:57:22.276 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:22.273+0000 7fa130200640 1 --2- 172.21.15.12:0/1973928206 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa12810a9d0 0x7fa12819c0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:57:22.276 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:22.273+0000 7fa130200640 1 --2- 172.21.15.12:0/1973928206 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa128076f00 0x7fa12819b490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:57:22.276 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:22.273+0000 7fa130200640 1 --2- 172.21.15.12:0/1973928206 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa128075aa0 0x7fa12819ae40 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:57:22.276 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:22.273+0000 7fa130200640 1 -- 172.21.15.12:0/1973928206 >> 172.21.15.12:0/1973928206 conn(0x7fa1280fe2f0 msgr2=0x7fa1281006e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:57:22.276 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:22.273+0000 7fa130200640 1 -- 172.21.15.12:0/1973928206 shutdown_connections 2024-09-10T23:57:22.276 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:22.273+0000 7fa130200640 1 -- 172.21.15.12:0/1973928206 wait complete. 2024-09-10T23:57:22.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:21 smithi165 ceph-mon[35356]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T23:57:22.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:21 smithi165 ceph-mon[35356]: from='client.? 172.21.15.12:0/2197280803' entity='client.admin' 2024-09-10T23:57:22.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:21 smithi165 ceph-mon[35356]: from='client.? ' entity='client.admin' 2024-09-10T23:57:22.858 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'ff3ad74a-6fcf-11ef-bcea-c7b262605968', '--', 'ceph', 'config', 'set', 'client', 'debug_client', '20'], check_status=True, hostname='smithi012') 2024-09-10T23:57:23.137 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:57:22 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-10T23:57:23.137 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:57:22 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-10T23:57:23.137 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:57:22 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-10T23:57:23.137 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:57:22 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]} : dispatch 2024-09-10T23:57:23.137 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:57:22 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-10T23:57:23.137 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:57:22 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "mgr services"} : dispatch 2024-09-10T23:57:23.137 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:57:22 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T23:57:23.137 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:57:22 smithi012 ceph-mon[30333]: Deploying daemon mgr.y on smithi165 2024-09-10T23:57:23.138 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:57:22 smithi012 ceph-mon[30333]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T23:57:23.138 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:57:22 smithi012 ceph-mon[30333]: from='client.? ' entity='client.admin' 2024-09-10T23:57:23.175 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'ff3ad74a-6fcf-11ef-bcea-c7b262605968', '--', 'ceph', 'config', 'set', 'client', 'debug_ms', '1'], check_status=True, hostname='smithi012') 2024-09-10T23:57:23.283 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:22 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-10T23:57:23.283 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:22 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-10T23:57:23.283 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:22 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-10T23:57:23.283 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:22 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]} : dispatch 2024-09-10T23:57:23.283 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:22 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-10T23:57:23.284 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:22 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "mgr services"} : dispatch 2024-09-10T23:57:23.284 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:22 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T23:57:23.284 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:22 smithi165 ceph-mon[35356]: Deploying daemon mgr.y on smithi165 2024-09-10T23:57:23.284 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:22 smithi165 ceph-mon[35356]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T23:57:23.284 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:22 smithi165 ceph-mon[35356]: from='client.? ' entity='client.admin' 2024-09-10T23:57:23.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:57:22 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-10T23:57:23.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:57:22 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-10T23:57:23.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:57:22 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-10T23:57:23.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:57:22 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]} : dispatch 2024-09-10T23:57:23.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:57:22 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-10T23:57:23.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:57:22 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "mgr services"} : dispatch 2024-09-10T23:57:23.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:57:22 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T23:57:23.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:57:22 smithi178 ceph-mon[35919]: Deploying daemon mgr.y on smithi165 2024-09-10T23:57:23.332 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:57:22 smithi178 ceph-mon[35919]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T23:57:23.332 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:57:22 smithi178 ceph-mon[35919]: from='client.? ' entity='client.admin' 2024-09-10T23:57:23.622 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T23:57:23.945 INFO:journalctl@ceph.mgr.y.smithi165.stdout:Sep 10 23:57:23 smithi165 systemd[1]: Starting Ceph mgr.y for ff3ad74a-6fcf-11ef-bcea-c7b262605968... 2024-09-10T23:57:24.022 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T23:57:24.031 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T23:57:24.041 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T23:57:24.042 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T23:57:24.057 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T23:57:24.058 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T23:57:24.078 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T23:57:24.085 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T23:57:24.159 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'ff3ad74a-6fcf-11ef-bcea-c7b262605968', '--', 'ceph', 'config', 'set', 'client', 'rados_mon_op_timeout', '900'], check_status=True, hostname='smithi012') 2024-09-10T23:57:24.159 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'ff3ad74a-6fcf-11ef-bcea-c7b262605968', '--', 'ceph', 'config', 'set', 'client', 'rados_osd_op_timeout', '900'], check_status=True, hostname='smithi012') 2024-09-10T23:57:24.175 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T23:57:24.239 INFO:journalctl@ceph.mgr.y.smithi165.stdout:Sep 10 23:57:23 smithi165 podman[36475]: 2024-09-10 23:57:23.846876423 +0000 UTC m=+0.015894371 image pull 589dd18248e494ba411dc838aa4ae0e5e569213f08fcd7a2cf22cd3ef7e3c480 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e 2024-09-10T23:57:24.240 INFO:journalctl@ceph.mgr.y.smithi165.stdout:Sep 10 23:57:23 smithi165 podman[36475]: 2024-09-10 23:57:23.945119688 +0000 UTC m=+0.114137627 container create 7a55726343a6b90f231188e07f209ca366831f3becc0a11087ea828d41b1dbf9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-mgr-y, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, CEPH_POINT_RELEASE=) 2024-09-10T23:57:24.240 INFO:journalctl@ceph.mgr.y.smithi165.stdout:Sep 10 23:57:24 smithi165 podman[36475]: 2024-09-10 23:57:24.164181268 +0000 UTC m=+0.333199207 container init 7a55726343a6b90f231188e07f209ca366831f3becc0a11087ea828d41b1dbf9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-mgr-y, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0) 2024-09-10T23:57:24.240 INFO:journalctl@ceph.mgr.y.smithi165.stdout:Sep 10 23:57:24 smithi165 podman[36475]: 2024-09-10 23:57:24.167050488 +0000 UTC m=+0.336068428 container start 7a55726343a6b90f231188e07f209ca366831f3becc0a11087ea828d41b1dbf9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-mgr-y, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , ceph=True, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-10T23:57:24.240 INFO:journalctl@ceph.mgr.y.smithi165.stdout:Sep 10 23:57:24 smithi165 ceph-mgr[36494]: -- 172.21.15.165:0/3252661617 <== mon.2 v2:172.21.15.165:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55d9b66df0e0 con 0x55d9b66bd000 2024-09-10T23:57:24.240 INFO:journalctl@ceph.mgr.y.smithi165.stdout:Sep 10 23:57:24 smithi165 bash[36475]: 7a55726343a6b90f231188e07f209ca366831f3becc0a11087ea828d41b1dbf9 2024-09-10T23:57:24.532 INFO:journalctl@ceph.mgr.y.smithi165.stdout:Sep 10 23:57:24 smithi165 systemd[1]: Started Ceph mgr.y for ff3ad74a-6fcf-11ef-bcea-c7b262605968. 2024-09-10T23:57:24.851 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:24.847+0000 7ff653a00640 1 -- 172.21.15.12:0/1643096280 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff64c106760 msgr2=0x7ff64c10d0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:57:24.851 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:24.847+0000 7ff653a00640 1 --2- 172.21.15.12:0/1643096280 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff64c106760 0x7ff64c10d0e0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7ff644009cd0 tx=0x7ff64402a8e0 comp rx=0 tx=0).stop 2024-09-10T23:57:24.851 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:24.847+0000 7ff653a00640 1 -- 172.21.15.12:0/1643096280 shutdown_connections 2024-09-10T23:57:24.852 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:24.847+0000 7ff653a00640 1 --2- 172.21.15.12:0/1643096280 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff64c106760 0x7ff64c10d0e0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:57:24.852 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:24.847+0000 7ff653a00640 1 --2- 172.21.15.12:0/1643096280 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff64c105da0 0x7ff64c106220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:57:24.852 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:24.847+0000 7ff653a00640 1 --2- 172.21.15.12:0/1643096280 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff64c104c40 0x7ff64c105040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:57:24.852 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:24.847+0000 7ff653a00640 1 -- 172.21.15.12:0/1643096280 >> 172.21.15.12:0/1643096280 conn(0x7ff64c100510 msgr2=0x7ff64c102950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:57:24.852 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:24.847+0000 7ff653a00640 1 -- 172.21.15.12:0/1643096280 shutdown_connections 2024-09-10T23:57:24.852 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:24.847+0000 7ff653a00640 1 -- 172.21.15.12:0/1643096280 wait complete. 2024-09-10T23:57:24.852 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:24.848+0000 7ff653a00640 1 Processor -- start 2024-09-10T23:57:24.852 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:24.848+0000 7ff653a00640 1 -- start start 2024-09-10T23:57:24.852 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:24.848+0000 7ff653a00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff64c104c40 0x7ff64c19aec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:57:24.852 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:24.848+0000 7ff653a00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff64c105da0 0x7ff64c19b510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:57:24.852 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:24.848+0000 7ff653a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff64c106760 0x7ff64c19c150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:57:24.853 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:24.848+0000 7ff653a00640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7ff64c1107d0 con 0x7ff64c105da0 2024-09-10T23:57:24.853 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:24.848+0000 7ff653a00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7ff64c110650 con 0x7ff64c106760 2024-09-10T23:57:24.853 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:24.848+0000 7ff653a00640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7ff64c110950 con 0x7ff64c104c40 2024-09-10T23:57:24.853 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:24.848+0000 7ff64be00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff64c105da0 0x7ff64c19b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:57:24.853 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:24.849+0000 7ff651a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff64c106760 0x7ff64c19c150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:57:24.853 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:24.849+0000 7ff651000640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff64c104c40 0x7ff64c19aec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:57:24.853 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:24.849+0000 7ff651a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff64c106760 0x7ff64c19c150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.12:59220/0 (socket says 172.21.15.12:59220) 2024-09-10T23:57:24.853 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:24.849+0000 7ff651a00640 1 -- 172.21.15.12:0/1150299372 learned_addr learned my addr 172.21.15.12:0/1150299372 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-10T23:57:24.853 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:24.849+0000 7ff651000640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff64c104c40 0x7ff64c19aec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.12:43422/0 (socket says 172.21.15.12:43422) 2024-09-10T23:57:24.853 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:24.849+0000 7ff64be00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff64c105da0 0x7ff64c19b510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:54692/0 (socket says 172.21.15.12:54692) 2024-09-10T23:57:24.853 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:24.849+0000 7ff651a00640 1 -- 172.21.15.12:0/1150299372 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff64c104c40 msgr2=0x7ff64c19aec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:57:24.854 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:24.849+0000 7ff651a00640 1 --2- 172.21.15.12:0/1150299372 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff64c104c40 0x7ff64c19aec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:57:24.854 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:24.849+0000 7ff651a00640 1 -- 172.21.15.12:0/1150299372 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff64c105da0 msgr2=0x7ff64c19b510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:57:24.854 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:24.849+0000 7ff651a00640 1 --2- 172.21.15.12:0/1150299372 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff64c105da0 0x7ff64c19b510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:57:24.854 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:24.849+0000 7ff651a00640 1 -- 172.21.15.12:0/1150299372 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff64c19c9f0 con 0x7ff64c106760 2024-09-10T23:57:24.854 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:24.849+0000 7ff64be00640 1 --2- 172.21.15.12:0/1150299372 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff64c105da0 0x7ff64c19b510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:57:24.854 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:24.849+0000 7ff651a00640 1 --2- 172.21.15.12:0/1150299372 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff64c106760 0x7ff64c19c150 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7ff644031040 tx=0x7ff644003ba0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:57:24.855 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:24.850+0000 7ff649600640 1 -- 172.21.15.12:0/1150299372 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff644004240 con 0x7ff64c106760 2024-09-10T23:57:24.855 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:24.850+0000 7ff649600640 1 -- 172.21.15.12:0/1150299372 <== mon.2 v2:172.21.15.165:3300/0 2 ==== config(25 keys) ==== 1070+0+0 (secure 0 0 0) 0x7ff6440305a0 con 0x7ff64c106760 2024-09-10T23:57:24.855 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:24.850+0000 7ff653a00640 1 -- 172.21.15.12:0/1150299372 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff64c19cc80 con 0x7ff64c106760 2024-09-10T23:57:24.855 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:24.850+0000 7ff649600640 1 -- 172.21.15.12:0/1150299372 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6440466e0 con 0x7ff64c106760 2024-09-10T23:57:24.855 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:24.850+0000 7ff653a00640 1 -- 172.21.15.12:0/1150299372 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff64c1a33c0 con 0x7ff64c106760 2024-09-10T23:57:24.855 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:24.851+0000 7ff649600640 1 -- 172.21.15.12:0/1150299372 <== mon.2 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 13) ==== 51480+0+0 (secure 0 0 0) 0x7ff64404e070 con 0x7ff64c106760 2024-09-10T23:57:24.855 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:24.851+0000 7ff653a00640 1 -- 172.21.15.12:0/1150299372 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff64c105040 con 0x7ff64c106760 2024-09-10T23:57:24.856 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:24.852+0000 7ff649600640 1 --2- 172.21.15.12:0/1150299372 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7ff63403f510 0x7ff6340419d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:57:24.856 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:24.852+0000 7ff649600640 1 -- 172.21.15.12:0/1150299372 <== mon.2 v2:172.21.15.165:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7ff64402e030 con 0x7ff64c106760 2024-09-10T23:57:24.856 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:24.852+0000 7ff651000640 1 --2- 172.21.15.12:0/1150299372 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7ff63403f510 0x7ff6340419d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:57:24.856 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:24.853+0000 7ff651000640 1 --2- 172.21.15.12:0/1150299372 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7ff63403f510 0x7ff6340419d0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7ff640005fd0 tx=0x7ff64000b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:57:24.862 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:24.860+0000 7ff649600640 1 -- 172.21.15.12:0/1150299372 <== mon.2 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7ff64404b060 con 0x7ff64c106760 2024-09-10T23:57:25.076 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:25.074+0000 7ff653a00640 1 -- 172.21.15.12:0/1150299372 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command([{prefix=config set, name=mds_debug_frag}] v 0) -- 0x7ff64c063d50 con 0x7ff64c106760 2024-09-10T23:57:25.082 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:25.079+0000 7ff649600640 1 -- 172.21.15.12:0/1150299372 <== mon.2 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_debug_frag}]=0 v14)=0 v14) ==== 115+0+0 (secure 0 0 0) 0x7ff64404f3e0 con 0x7ff64c106760 2024-09-10T23:57:25.085 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:25.082+0000 7ff653a00640 1 -- 172.21.15.12:0/1150299372 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7ff63403f510 msgr2=0x7ff6340419d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:57:25.085 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:25.082+0000 7ff653a00640 1 --2- 172.21.15.12:0/1150299372 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7ff63403f510 0x7ff6340419d0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7ff640005fd0 tx=0x7ff64000b390 comp rx=0 tx=0).stop 2024-09-10T23:57:25.085 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:25.082+0000 7ff653a00640 1 -- 172.21.15.12:0/1150299372 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff64c106760 msgr2=0x7ff64c19c150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:57:25.085 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:25.082+0000 7ff653a00640 1 --2- 172.21.15.12:0/1150299372 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff64c106760 0x7ff64c19c150 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7ff644031040 tx=0x7ff644003ba0 comp rx=0 tx=0).stop 2024-09-10T23:57:25.085 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:25.082+0000 7ff653a00640 1 -- 172.21.15.12:0/1150299372 shutdown_connections 2024-09-10T23:57:25.085 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:25.082+0000 7ff653a00640 1 --2- 172.21.15.12:0/1150299372 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7ff63403f510 0x7ff6340419d0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:57:25.085 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:25.082+0000 7ff653a00640 1 --2- 172.21.15.12:0/1150299372 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff64c106760 0x7ff64c19c150 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:57:25.085 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:25.082+0000 7ff653a00640 1 --2- 172.21.15.12:0/1150299372 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff64c105da0 0x7ff64c19b510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:57:25.085 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:25.082+0000 7ff653a00640 1 --2- 172.21.15.12:0/1150299372 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff64c104c40 0x7ff64c19aec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:57:25.085 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:25.082+0000 7ff653a00640 1 -- 172.21.15.12:0/1150299372 >> 172.21.15.12:0/1150299372 conn(0x7ff64c100510 msgr2=0x7ff64c109190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:57:25.085 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:25.083+0000 7ff653a00640 1 -- 172.21.15.12:0/1150299372 shutdown_connections 2024-09-10T23:57:25.085 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:25.083+0000 7ff653a00640 1 -- 172.21.15.12:0/1150299372 wait complete. 2024-09-10T23:57:25.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:57:25 smithi178 ceph-mon[35919]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T23:57:25.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:57:25 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-10T23:57:25.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:57:25 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-10T23:57:25.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:57:25 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-10T23:57:25.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:57:25 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-10T23:57:25.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:57:25 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T23:57:25.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:57:25 smithi178 ceph-mon[35919]: from='client.? ' entity='client.admin' 2024-09-10T23:57:25.402 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:57:25 smithi012 ceph-mon[30333]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T23:57:25.402 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:57:25 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-10T23:57:25.402 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:57:25 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-10T23:57:25.402 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:57:25 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-10T23:57:25.402 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:57:25 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-10T23:57:25.402 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:57:25 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T23:57:25.402 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:57:25 smithi012 ceph-mon[30333]: from='client.? ' entity='client.admin' 2024-09-10T23:57:25.782 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:25 smithi165 ceph-mon[35356]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T23:57:25.782 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:25 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-10T23:57:25.782 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:25 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-10T23:57:25.782 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:25 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-10T23:57:25.782 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:25 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-10T23:57:25.782 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:25 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T23:57:25.783 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:25 smithi165 ceph-mon[35356]: from='client.? ' entity='client.admin' 2024-09-10T23:57:26.494 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:26.476+0000 7f483b200640 1 -- 172.21.15.12:0/3343366618 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f483c1065c0 msgr2=0x7f483c10cf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:57:26.494 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:26.476+0000 7f483b200640 1 --2- 172.21.15.12:0/3343366618 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f483c1065c0 0x7f483c10cf40 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f4834009c10 tx=0x7f483402a6f0 comp rx=0 tx=0).stop 2024-09-10T23:57:26.494 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:26.476+0000 7f483b200640 1 -- 172.21.15.12:0/3343366618 shutdown_connections 2024-09-10T23:57:26.494 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:26.476+0000 7f483b200640 1 --2- 172.21.15.12:0/3343366618 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f483c1065c0 0x7f483c10cf40 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:57:26.494 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:26.476+0000 7f483b200640 1 --2- 172.21.15.12:0/3343366618 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f483c105c00 0x7f483c106080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:57:26.495 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:26.476+0000 7f483b200640 1 --2- 172.21.15.12:0/3343366618 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f483c104aa0 0x7f483c104ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:57:26.495 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:26.476+0000 7f483b200640 1 -- 172.21.15.12:0/3343366618 >> 172.21.15.12:0/3343366618 conn(0x7f483c100370 msgr2=0x7f483c1027b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:57:26.495 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:26.476+0000 7f483b200640 1 -- 172.21.15.12:0/3343366618 shutdown_connections 2024-09-10T23:57:26.495 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:26.476+0000 7f483b200640 1 -- 172.21.15.12:0/3343366618 wait complete. 2024-09-10T23:57:26.495 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:26.477+0000 7f483b200640 1 Processor -- start 2024-09-10T23:57:26.495 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:26.477+0000 7f483b200640 1 -- start start 2024-09-10T23:57:26.495 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:26.477+0000 7f483b200640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f483c104aa0 0x7f483c19ac50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:57:26.495 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:26.477+0000 7f483b200640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f483c105c00 0x7f483c19b2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:57:26.495 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:26.477+0000 7f483b200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f483c1065c0 0x7f483c19bee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:57:26.495 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:26.477+0000 7f483b200640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f483c1105c0 con 0x7f483c105c00 2024-09-10T23:57:26.495 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:26.477+0000 7f483b200640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f483c110440 con 0x7f483c1065c0 2024-09-10T23:57:26.495 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:26.477+0000 7f483b200640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f483c110740 con 0x7f483c104aa0 2024-09-10T23:57:26.495 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:26.477+0000 7f4839400640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f483c105c00 0x7f483c19b2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:57:26.496 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:26.477+0000 7f483a800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f483c1065c0 0x7f483c19bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:57:26.496 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:26.477+0000 7f4839e00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f483c104aa0 0x7f483c19ac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:57:26.496 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:26.477+0000 7f483a800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f483c1065c0 0x7f483c19bee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.12:59264/0 (socket says 172.21.15.12:59264) 2024-09-10T23:57:26.496 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:26.477+0000 7f483a800640 1 -- 172.21.15.12:0/529214429 learned_addr learned my addr 172.21.15.12:0/529214429 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-10T23:57:26.496 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:26.478+0000 7f483a800640 1 -- 172.21.15.12:0/529214429 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f483c104aa0 msgr2=0x7f483c19ac50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:57:26.496 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:26.478+0000 7f483a800640 1 --2- 172.21.15.12:0/529214429 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f483c104aa0 0x7f483c19ac50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:57:26.496 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:26.478+0000 7f483a800640 1 -- 172.21.15.12:0/529214429 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f483c105c00 msgr2=0x7f483c19b2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:57:26.496 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:26.478+0000 7f483a800640 1 --2- 172.21.15.12:0/529214429 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f483c105c00 0x7f483c19b2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:57:26.496 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:26.478+0000 7f483a800640 1 -- 172.21.15.12:0/529214429 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f483c19c780 con 0x7f483c1065c0 2024-09-10T23:57:26.496 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:26.478+0000 7f483a800640 1 --2- 172.21.15.12:0/529214429 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f483c1065c0 0x7f483c19bee0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f483402ac10 tx=0x7f48340305d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:57:26.496 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:26.478+0000 7f4832a00640 1 -- 172.21.15.12:0/529214429 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4834041020 con 0x7f483c1065c0 2024-09-10T23:57:26.496 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:26.479+0000 7f4832a00640 1 -- 172.21.15.12:0/529214429 <== mon.2 v2:172.21.15.165:3300/0 2 ==== config(25 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f4834037040 con 0x7f483c1065c0 2024-09-10T23:57:26.496 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:26.479+0000 7f483b200640 1 -- 172.21.15.12:0/529214429 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f483c19ca10 con 0x7f483c1065c0 2024-09-10T23:57:26.497 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:26.479+0000 7f4832a00640 1 -- 172.21.15.12:0/529214429 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4834036070 con 0x7f483c1065c0 2024-09-10T23:57:26.497 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:26.479+0000 7f483b200640 1 -- 172.21.15.12:0/529214429 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f483c1a3150 con 0x7f483c1065c0 2024-09-10T23:57:26.497 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:26.480+0000 7f4832a00640 1 -- 172.21.15.12:0/529214429 <== mon.2 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 13) ==== 51480+0+0 (secure 0 0 0) 0x7f4834004560 con 0x7f483c1065c0 2024-09-10T23:57:26.497 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:26.480+0000 7f483b200640 1 -- 172.21.15.12:0/529214429 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f483c104ea0 con 0x7f483c1065c0 2024-09-10T23:57:26.497 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:26.480+0000 7f4832a00640 1 --2- 172.21.15.12:0/529214429 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f482003f5c0 0x7f4820041a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:57:26.497 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:26.480+0000 7f4832a00640 1 -- 172.21.15.12:0/529214429 <== mon.2 v2:172.21.15.165:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f4834074950 con 0x7f483c1065c0 2024-09-10T23:57:26.497 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:26.480+0000 7f4839e00640 1 --2- 172.21.15.12:0/529214429 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f482003f5c0 0x7f4820041a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:57:26.497 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:26.481+0000 7f4839e00640 1 --2- 172.21.15.12:0/529214429 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f482003f5c0 0x7f4820041a80 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f482c005fd0 tx=0x7f482c009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:57:26.497 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:26.485+0000 7f4832a00640 1 -- 172.21.15.12:0/529214429 <== mon.2 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f483404b020 con 0x7f483c1065c0 2024-09-10T23:57:26.609 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:26.383+0000 7fb399200640 1 -- 172.21.15.12:0/1091625856 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb394103b80 msgr2=0x7fb394104000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:57:26.609 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:26.383+0000 7fb399200640 1 --2- 172.21.15.12:0/1091625856 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb394103b80 0x7fb394104000 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fb380009890 tx=0x7fb38002a360 comp rx=0 tx=0).stop 2024-09-10T23:57:26.609 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:26.383+0000 7fb399200640 1 -- 172.21.15.12:0/1091625856 shutdown_connections 2024-09-10T23:57:26.609 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:26.383+0000 7fb399200640 1 --2- 172.21.15.12:0/1091625856 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb394104540 0x7fb39410aec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:57:26.610 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:26.383+0000 7fb399200640 1 --2- 172.21.15.12:0/1091625856 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb394103b80 0x7fb394104000 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:57:26.610 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:26.383+0000 7fb399200640 1 --2- 172.21.15.12:0/1091625856 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb394102a20 0x7fb394102e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:57:26.610 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:26.383+0000 7fb399200640 1 -- 172.21.15.12:0/1091625856 >> 172.21.15.12:0/1091625856 conn(0x7fb3940fe2f0 msgr2=0x7fb394100730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:57:26.610 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:26.383+0000 7fb399200640 1 -- 172.21.15.12:0/1091625856 shutdown_connections 2024-09-10T23:57:26.610 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:26.383+0000 7fb399200640 1 -- 172.21.15.12:0/1091625856 wait complete. 2024-09-10T23:57:26.610 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:26.384+0000 7fb399200640 1 Processor -- start 2024-09-10T23:57:26.610 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:26.384+0000 7fb399200640 1 -- start start 2024-09-10T23:57:26.610 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:26.384+0000 7fb399200640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb394102a20 0x7fb394076c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:57:26.610 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:26.384+0000 7fb399200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb394103b80 0x7fb394075540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:57:26.610 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:26.384+0000 7fb399200640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb394104540 0x7fb394075a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:57:26.610 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:26.384+0000 7fb399200640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fb39410e560 con 0x7fb394104540 2024-09-10T23:57:26.610 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:26.384+0000 7fb399200640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fb39410e3e0 con 0x7fb394103b80 2024-09-10T23:57:26.610 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:26.384+0000 7fb399200640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fb39410e6e0 con 0x7fb394102a20 2024-09-10T23:57:26.611 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:26.385+0000 7fb391e00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb394102a20 0x7fb394076c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:57:26.611 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:26.385+0000 7fb391e00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb394102a20 0x7fb394076c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.12:43438/0 (socket says 172.21.15.12:43438) 2024-09-10T23:57:26.611 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:26.385+0000 7fb391e00640 1 -- 172.21.15.12:0/275089016 learned_addr learned my addr 172.21.15.12:0/275089016 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-10T23:57:26.611 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:26.385+0000 7fb391400640 1 --2- 172.21.15.12:0/275089016 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb394103b80 0x7fb394075540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:57:26.611 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:26.385+0000 7fb392800640 1 --2- 172.21.15.12:0/275089016 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb394104540 0x7fb394075a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:57:26.611 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:26.385+0000 7fb391e00640 1 -- 172.21.15.12:0/275089016 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb394103b80 msgr2=0x7fb394075540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:57:26.611 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:26.385+0000 7fb391e00640 1 --2- 172.21.15.12:0/275089016 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb394103b80 0x7fb394075540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:57:26.611 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:26.385+0000 7fb391e00640 1 -- 172.21.15.12:0/275089016 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb394104540 msgr2=0x7fb394075a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:57:26.611 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:26.385+0000 7fb391e00640 1 --2- 172.21.15.12:0/275089016 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb394104540 0x7fb394075a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:57:26.611 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:26.385+0000 7fb391e00640 1 -- 172.21.15.12:0/275089016 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb394076280 con 0x7fb394102a20 2024-09-10T23:57:26.611 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:26.385+0000 7fb392800640 1 --2- 172.21.15.12:0/275089016 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb394104540 0x7fb394075a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:57:26.611 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:26.385+0000 7fb391400640 1 --2- 172.21.15.12:0/275089016 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb394103b80 0x7fb394075540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:57:26.611 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:26.385+0000 7fb391e00640 1 --2- 172.21.15.12:0/275089016 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb394102a20 0x7fb394076c60 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fb38400e9f0 tx=0x7fb38400edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:57:26.611 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:26.386+0000 7fb38aa00640 1 -- 172.21.15.12:0/275089016 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb38400cd80 con 0x7fb394102a20 2024-09-10T23:57:26.612 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:26.386+0000 7fb38aa00640 1 -- 172.21.15.12:0/275089016 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(25 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fb3840054b0 con 0x7fb394102a20 2024-09-10T23:57:26.612 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:26.386+0000 7fb399200640 1 -- 172.21.15.12:0/275089016 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb3941a2e60 con 0x7fb394102a20 2024-09-10T23:57:26.612 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:26.386+0000 7fb38aa00640 1 -- 172.21.15.12:0/275089016 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb384010430 con 0x7fb394102a20 2024-09-10T23:57:26.612 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:26.386+0000 7fb399200640 1 -- 172.21.15.12:0/275089016 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb3941a32b0 con 0x7fb394102a20 2024-09-10T23:57:26.612 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:26.387+0000 7fb38aa00640 1 -- 172.21.15.12:0/275089016 <== mon.1 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 13) ==== 51480+0+0 (secure 0 0 0) 0x7fb384005020 con 0x7fb394102a20 2024-09-10T23:57:26.612 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:26.387+0000 7fb399200640 1 -- 172.21.15.12:0/275089016 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb394109e20 con 0x7fb394102a20 2024-09-10T23:57:26.612 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:26.387+0000 7fb38aa00640 1 --2- 172.21.15.12:0/275089016 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fb37803f510 0x7fb3780419d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:57:26.612 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:26.388+0000 7fb38aa00640 1 -- 172.21.15.12:0/275089016 <== mon.1 v2:172.21.15.178:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fb3840541d0 con 0x7fb394102a20 2024-09-10T23:57:26.612 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:26.388+0000 7fb391400640 1 --2- 172.21.15.12:0/275089016 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fb37803f510 0x7fb3780419d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:57:26.612 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:26.388+0000 7fb391400640 1 --2- 172.21.15.12:0/275089016 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fb37803f510 0x7fb3780419d0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fb38002a950 tx=0x7fb380000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:57:26.612 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:26.392+0000 7fb38aa00640 1 -- 172.21.15.12:0/275089016 <== mon.1 v2:172.21.15.178:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7fb384004770 con 0x7fb394102a20 2024-09-10T23:57:26.612 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:26.596+0000 7fb399200640 1 -- 172.21.15.12:0/275089016 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command([{prefix=config set, name=mds_op_complaint_time}] v 0) -- 0x7fb394063df0 con 0x7fb394102a20 2024-09-10T23:57:26.612 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:26.600+0000 7fb38aa00640 1 -- 172.21.15.12:0/275089016 <== mon.1 v2:172.21.15.178:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_op_complaint_time}]=0 v15)=0 v15) ==== 121+0+0 (secure 0 0 0) 0x7fb384021440 con 0x7fb394102a20 2024-09-10T23:57:26.613 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:26.603+0000 7fb399200640 1 -- 172.21.15.12:0/275089016 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fb37803f510 msgr2=0x7fb3780419d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:57:26.613 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:26.603+0000 7fb399200640 1 --2- 172.21.15.12:0/275089016 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fb37803f510 0x7fb3780419d0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fb38002a950 tx=0x7fb380000bc0 comp rx=0 tx=0).stop 2024-09-10T23:57:26.613 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:26.603+0000 7fb399200640 1 -- 172.21.15.12:0/275089016 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb394102a20 msgr2=0x7fb394076c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:57:26.613 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:26.603+0000 7fb399200640 1 --2- 172.21.15.12:0/275089016 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb394102a20 0x7fb394076c60 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fb38400e9f0 tx=0x7fb38400edf0 comp rx=0 tx=0).stop 2024-09-10T23:57:26.613 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:26.603+0000 7fb399200640 1 -- 172.21.15.12:0/275089016 shutdown_connections 2024-09-10T23:57:26.613 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:26.603+0000 7fb399200640 1 --2- 172.21.15.12:0/275089016 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fb37803f510 0x7fb3780419d0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:57:26.613 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:26.603+0000 7fb399200640 1 --2- 172.21.15.12:0/275089016 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb394104540 0x7fb394075a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:57:26.613 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:26.603+0000 7fb399200640 1 --2- 172.21.15.12:0/275089016 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb394103b80 0x7fb394075540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:57:26.613 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:26.603+0000 7fb399200640 1 --2- 172.21.15.12:0/275089016 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb394102a20 0x7fb394076c60 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:57:26.613 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:26.603+0000 7fb399200640 1 -- 172.21.15.12:0/275089016 >> 172.21.15.12:0/275089016 conn(0x7fb3940fe2f0 msgr2=0x7fb394106ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:57:26.613 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:26.603+0000 7fb399200640 1 -- 172.21.15.12:0/275089016 shutdown_connections 2024-09-10T23:57:26.613 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:26.603+0000 7fb399200640 1 -- 172.21.15.12:0/275089016 wait complete. 2024-09-10T23:57:26.688 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:26.686+0000 7f483b200640 1 -- 172.21.15.12:0/529214429 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command([{prefix=config set, name=mon_op_complaint_time}] v 0) -- 0x7f483c063d50 con 0x7f483c1065c0 2024-09-10T23:57:26.695 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:26.693+0000 7f4832a00640 1 -- 172.21.15.12:0/529214429 <== mon.2 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_op_complaint_time}]=0 v16)=0 v16) ==== 121+0+0 (secure 0 0 0) 0x7f4834003d80 con 0x7f483c1065c0 2024-09-10T23:57:26.698 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:26.695+0000 7f483b200640 1 -- 172.21.15.12:0/529214429 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f482003f5c0 msgr2=0x7f4820041a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:57:26.698 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:26.695+0000 7f483b200640 1 --2- 172.21.15.12:0/529214429 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f482003f5c0 0x7f4820041a80 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f482c005fd0 tx=0x7f482c009260 comp rx=0 tx=0).stop 2024-09-10T23:57:26.698 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:26.695+0000 7f483b200640 1 -- 172.21.15.12:0/529214429 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f483c1065c0 msgr2=0x7f483c19bee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:57:26.698 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:26.695+0000 7f483b200640 1 --2- 172.21.15.12:0/529214429 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f483c1065c0 0x7f483c19bee0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f483402ac10 tx=0x7f48340305d0 comp rx=0 tx=0).stop 2024-09-10T23:57:26.698 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:26.695+0000 7f483b200640 1 -- 172.21.15.12:0/529214429 shutdown_connections 2024-09-10T23:57:26.698 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:26.696+0000 7f483b200640 1 --2- 172.21.15.12:0/529214429 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f482003f5c0 0x7f4820041a80 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:57:26.698 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:26.696+0000 7f483b200640 1 --2- 172.21.15.12:0/529214429 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f483c1065c0 0x7f483c19bee0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:57:26.698 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:26.696+0000 7f483b200640 1 --2- 172.21.15.12:0/529214429 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f483c105c00 0x7f483c19b2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:57:26.698 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:26.696+0000 7f483b200640 1 --2- 172.21.15.12:0/529214429 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f483c104aa0 0x7f483c19ac50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:57:26.698 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:26.696+0000 7f483b200640 1 -- 172.21.15.12:0/529214429 >> 172.21.15.12:0/529214429 conn(0x7f483c100370 msgr2=0x7f483c108f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:57:26.698 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:26.696+0000 7f483b200640 1 -- 172.21.15.12:0/529214429 shutdown_connections 2024-09-10T23:57:26.698 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:26.696+0000 7f483b200640 1 -- 172.21.15.12:0/529214429 wait complete. 2024-09-10T23:57:26.712 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:57:26 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-10T23:57:26.712 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:57:26 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-10T23:57:26.712 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:57:26 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-10T23:57:26.712 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:57:26 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-10T23:57:26.712 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:57:26 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T23:57:26.713 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:57:26 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T23:57:26.713 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:57:26 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-10T23:57:26.713 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:57:26 smithi012 ceph-mon[30333]: Reconfiguring mgr.x (unknown last config time)... 2024-09-10T23:57:26.713 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:57:26 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]} : dispatch 2024-09-10T23:57:26.713 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:57:26 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "mgr services"} : dispatch 2024-09-10T23:57:26.713 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:57:26 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T23:57:26.713 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:57:26 smithi012 ceph-mon[30333]: Reconfiguring daemon mgr.x on smithi012 2024-09-10T23:57:26.831 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:57:26 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-10T23:57:26.831 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:57:26 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-10T23:57:26.831 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:57:26 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-10T23:57:26.831 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:57:26 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-10T23:57:26.831 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:57:26 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T23:57:26.831 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:57:26 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T23:57:26.831 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:57:26 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-10T23:57:26.831 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:57:26 smithi178 ceph-mon[35919]: Reconfiguring mgr.x (unknown last config time)... 2024-09-10T23:57:26.831 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:57:26 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]} : dispatch 2024-09-10T23:57:26.831 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:57:26 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "mgr services"} : dispatch 2024-09-10T23:57:26.831 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:57:26 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T23:57:26.832 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:57:26 smithi178 ceph-mon[35919]: Reconfiguring daemon mgr.x on smithi012 2024-09-10T23:57:27.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:26 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-10T23:57:27.033 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:26 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-10T23:57:27.033 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:26 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-10T23:57:27.033 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:26 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-10T23:57:27.033 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:26 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T23:57:27.033 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:26 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T23:57:27.033 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:26 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-10T23:57:27.033 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:26 smithi165 ceph-mon[35356]: Reconfiguring mgr.x (unknown last config time)... 2024-09-10T23:57:27.033 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:26 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]} : dispatch 2024-09-10T23:57:27.033 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:26 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "mgr services"} : dispatch 2024-09-10T23:57:27.033 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:26 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T23:57:27.034 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:26 smithi165 ceph-mon[35356]: Reconfiguring daemon mgr.x on smithi012 2024-09-10T23:57:27.423 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:27.420+0000 7fee68000640 1 -- 172.21.15.12:0/61710646 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fee60075b20 msgr2=0x7fee60075fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:57:27.423 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:27.420+0000 7fee68000640 1 --2- 172.21.15.12:0/61710646 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fee60075b20 0x7fee60075fa0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fee4c009860 tx=0x7fee4c02a340 comp rx=0 tx=0).stop 2024-09-10T23:57:27.423 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:27.421+0000 7fee68000640 1 -- 172.21.15.12:0/61710646 shutdown_connections 2024-09-10T23:57:27.423 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:27.421+0000 7fee68000640 1 --2- 172.21.15.12:0/61710646 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fee60106540 0x7fee60111200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:57:27.423 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:27.421+0000 7fee68000640 1 --2- 172.21.15.12:0/61710646 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fee60075b20 0x7fee60075fa0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:57:27.424 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:27.421+0000 7fee68000640 1 --2- 172.21.15.12:0/61710646 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fee60076f80 0x7fee600755e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:57:27.424 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:27.421+0000 7fee68000640 1 -- 172.21.15.12:0/61710646 >> 172.21.15.12:0/61710646 conn(0x7fee600fe2f0 msgr2=0x7fee60100730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:57:27.424 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:27.421+0000 7fee68000640 1 -- 172.21.15.12:0/61710646 shutdown_connections 2024-09-10T23:57:27.424 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:27.421+0000 7fee68000640 1 -- 172.21.15.12:0/61710646 wait complete. 2024-09-10T23:57:27.424 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:27.422+0000 7fee68000640 1 Processor -- start 2024-09-10T23:57:27.424 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:27.423+0000 7fee68000640 1 -- start start 2024-09-10T23:57:27.425 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:27.423+0000 7fee68000640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fee60075b20 0x7fee6019f330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:57:27.425 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:27.423+0000 7fee68000640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fee60076f80 0x7fee6019f980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:57:27.425 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:27.423+0000 7fee68000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fee601a05c0 0x7fee601a0a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:57:27.425 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:27.423+0000 7fee68000640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fee60114b90 con 0x7fee60075b20 2024-09-10T23:57:27.425 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:27.423+0000 7fee68000640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fee60114a10 con 0x7fee601a05c0 2024-09-10T23:57:27.425 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:27.423+0000 7fee68000640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fee60114d10 con 0x7fee60076f80 2024-09-10T23:57:27.426 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:27.423+0000 7fee64c00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fee60076f80 0x7fee6019f980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:57:27.426 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:27.423+0000 7fee65600640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fee60075b20 0x7fee6019f330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:57:27.426 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:27.423+0000 7fee64c00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fee60076f80 0x7fee6019f980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.12:43456/0 (socket says 172.21.15.12:43456) 2024-09-10T23:57:27.426 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:27.423+0000 7fee64c00640 1 -- 172.21.15.12:0/4241750791 learned_addr learned my addr 172.21.15.12:0/4241750791 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-10T23:57:27.426 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:27.423+0000 7fee66000640 1 --2- 172.21.15.12:0/4241750791 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fee601a05c0 0x7fee601a0a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:57:27.427 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:27.424+0000 7fee66000640 1 -- 172.21.15.12:0/4241750791 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fee60076f80 msgr2=0x7fee6019f980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:57:27.427 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:27.424+0000 7fee66000640 1 --2- 172.21.15.12:0/4241750791 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fee60076f80 0x7fee6019f980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:57:27.427 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:27.424+0000 7fee66000640 1 -- 172.21.15.12:0/4241750791 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fee60075b20 msgr2=0x7fee6019f330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:57:27.427 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:27.424+0000 7fee66000640 1 --2- 172.21.15.12:0/4241750791 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fee60075b20 0x7fee6019f330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:57:27.427 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:27.424+0000 7fee66000640 1 -- 172.21.15.12:0/4241750791 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee601a1240 con 0x7fee601a05c0 2024-09-10T23:57:27.427 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:27.424+0000 7fee64c00640 1 --2- 172.21.15.12:0/4241750791 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fee60076f80 0x7fee6019f980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:57:27.427 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:27.424+0000 7fee66000640 1 --2- 172.21.15.12:0/4241750791 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fee601a05c0 0x7fee601a0a70 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fee5800e940 tx=0x7fee5800ee00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:57:27.427 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:27.424+0000 7fee5e000640 1 -- 172.21.15.12:0/4241750791 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee58004d20 con 0x7fee601a05c0 2024-09-10T23:57:27.427 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:27.424+0000 7fee5e000640 1 -- 172.21.15.12:0/4241750791 <== mon.2 v2:172.21.15.165:3300/0 2 ==== config(25 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fee58005020 con 0x7fee601a05c0 2024-09-10T23:57:27.427 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:27.424+0000 7fee68000640 1 -- 172.21.15.12:0/4241750791 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fee601a7660 con 0x7fee601a05c0 2024-09-10T23:57:27.427 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:27.424+0000 7fee5e000640 1 -- 172.21.15.12:0/4241750791 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee580116d0 con 0x7fee601a05c0 2024-09-10T23:57:27.427 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:27.424+0000 7fee68000640 1 -- 172.21.15.12:0/4241750791 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fee601a7b10 con 0x7fee601a05c0 2024-09-10T23:57:27.428 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:27.425+0000 7fee5e000640 1 -- 172.21.15.12:0/4241750791 <== mon.2 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 13) ==== 51480+0+0 (secure 0 0 0) 0x7fee5800f040 con 0x7fee601a05c0 2024-09-10T23:57:27.428 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:27.425+0000 7fee68000640 1 -- 172.21.15.12:0/4241750791 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fee60104c00 con 0x7fee601a05c0 2024-09-10T23:57:27.428 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:27.426+0000 7fee5e000640 1 --2- 172.21.15.12:0/4241750791 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fee4803f510 0x7fee480419d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:57:27.428 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:27.426+0000 7fee5e000640 1 -- 172.21.15.12:0/4241750791 <== mon.2 v2:172.21.15.165:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fee58053bb0 con 0x7fee601a05c0 2024-09-10T23:57:27.428 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:27.426+0000 7fee65600640 1 --2- 172.21.15.12:0/4241750791 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fee4803f510 0x7fee480419d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:57:27.429 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:27.427+0000 7fee65600640 1 --2- 172.21.15.12:0/4241750791 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fee4803f510 0x7fee480419d0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fee54009530 tx=0x7fee54009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:57:27.433 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:27.431+0000 7fee5e000640 1 -- 172.21.15.12:0/4241750791 <== mon.2 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7fee58016030 con 0x7fee601a05c0 2024-09-10T23:57:27.449 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T23:57:27.450 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T23:57:27.644 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:27.642+0000 7fee68000640 1 -- 172.21.15.12:0/4241750791 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command([{prefix=config set, name=debug_mds_balancer}] v 0) -- 0x7fee60106540 con 0x7fee601a05c0 2024-09-10T23:57:27.650 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:27.648+0000 7fee5e000640 1 -- 172.21.15.12:0/4241750791 <== mon.2 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{prefix=config set, name=debug_mds_balancer}]=0 v17)=0 v17) ==== 117+0+0 (secure 0 0 0) 0x7fee580051d0 con 0x7fee601a05c0 2024-09-10T23:57:27.653 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:27.651+0000 7fee68000640 1 -- 172.21.15.12:0/4241750791 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fee4803f510 msgr2=0x7fee480419d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:57:27.653 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:27.651+0000 7fee68000640 1 --2- 172.21.15.12:0/4241750791 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fee4803f510 0x7fee480419d0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fee54009530 tx=0x7fee54009260 comp rx=0 tx=0).stop 2024-09-10T23:57:27.653 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:27.651+0000 7fee68000640 1 -- 172.21.15.12:0/4241750791 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fee601a05c0 msgr2=0x7fee601a0a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:57:27.653 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:27.651+0000 7fee68000640 1 --2- 172.21.15.12:0/4241750791 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fee601a05c0 0x7fee601a0a70 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fee5800e940 tx=0x7fee5800ee00 comp rx=0 tx=0).stop 2024-09-10T23:57:27.653 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:27.651+0000 7fee68000640 1 -- 172.21.15.12:0/4241750791 shutdown_connections 2024-09-10T23:57:27.653 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:27.651+0000 7fee68000640 1 --2- 172.21.15.12:0/4241750791 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fee4803f510 0x7fee480419d0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:57:27.653 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:27.651+0000 7fee68000640 1 --2- 172.21.15.12:0/4241750791 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fee601a05c0 0x7fee601a0a70 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:57:27.654 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:27.651+0000 7fee68000640 1 --2- 172.21.15.12:0/4241750791 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fee60076f80 0x7fee6019f980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:57:27.654 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:27.651+0000 7fee68000640 1 --2- 172.21.15.12:0/4241750791 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fee60075b20 0x7fee6019f330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:57:27.654 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:27.651+0000 7fee68000640 1 -- 172.21.15.12:0/4241750791 >> 172.21.15.12:0/4241750791 conn(0x7fee600fe2f0 msgr2=0x7fee60100710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:57:27.654 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:27.651+0000 7fee68000640 1 -- 172.21.15.12:0/4241750791 shutdown_connections 2024-09-10T23:57:27.654 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:27.651+0000 7fee68000640 1 -- 172.21.15.12:0/4241750791 wait complete. 2024-09-10T23:57:27.745 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:57:27 smithi012 ceph-mon[30333]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T23:57:27.745 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:57:27 smithi012 ceph-mon[30333]: from='client.? ' entity='client.admin' 2024-09-10T23:57:27.745 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:57:27 smithi012 ceph-mon[30333]: from='client.? ' entity='client.admin' 2024-09-10T23:57:27.745 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:57:27 smithi012 ceph-mon[30333]: Standby manager daemon z started 2024-09-10T23:57:27.745 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:57:27 smithi012 ceph-mon[30333]: from='mgr.? 172.21.15.178:0/1260535139' entity='mgr.z' cmd={"prefix": "config-key get", "key": "mgr/dashboard/z/crt"} : dispatch 2024-09-10T23:57:27.745 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:57:27 smithi012 ceph-mon[30333]: from='mgr.? 172.21.15.178:0/1260535139' entity='mgr.z' cmd={"prefix": "config-key get", "key": "mgr/dashboard/crt"} : dispatch 2024-09-10T23:57:27.745 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:57:27 smithi012 ceph-mon[30333]: from='mgr.? 172.21.15.178:0/1260535139' entity='mgr.z' cmd={"prefix": "config-key get", "key": "mgr/dashboard/z/key"} : dispatch 2024-09-10T23:57:27.745 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:57:27 smithi012 ceph-mon[30333]: from='mgr.? 172.21.15.178:0/1260535139' entity='mgr.z' cmd={"prefix": "config-key get", "key": "mgr/dashboard/key"} : dispatch 2024-09-10T23:57:27.748 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T23:57:28.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:27 smithi165 ceph-mon[35356]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T23:57:28.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:27 smithi165 ceph-mon[35356]: from='client.? ' entity='client.admin' 2024-09-10T23:57:28.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:27 smithi165 ceph-mon[35356]: from='client.? ' entity='client.admin' 2024-09-10T23:57:28.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:27 smithi165 ceph-mon[35356]: Standby manager daemon z started 2024-09-10T23:57:28.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:27 smithi165 ceph-mon[35356]: from='mgr.? 172.21.15.178:0/1260535139' entity='mgr.z' cmd={"prefix": "config-key get", "key": "mgr/dashboard/z/crt"} : dispatch 2024-09-10T23:57:28.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:27 smithi165 ceph-mon[35356]: from='mgr.? 172.21.15.178:0/1260535139' entity='mgr.z' cmd={"prefix": "config-key get", "key": "mgr/dashboard/crt"} : dispatch 2024-09-10T23:57:28.033 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:27 smithi165 ceph-mon[35356]: from='mgr.? 172.21.15.178:0/1260535139' entity='mgr.z' cmd={"prefix": "config-key get", "key": "mgr/dashboard/z/key"} : dispatch 2024-09-10T23:57:28.033 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:27 smithi165 ceph-mon[35356]: from='mgr.? 172.21.15.178:0/1260535139' entity='mgr.z' cmd={"prefix": "config-key get", "key": "mgr/dashboard/key"} : dispatch 2024-09-10T23:57:28.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:57:27 smithi178 ceph-mon[35919]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T23:57:28.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:57:27 smithi178 ceph-mon[35919]: from='client.? ' entity='client.admin' 2024-09-10T23:57:28.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:57:27 smithi178 ceph-mon[35919]: from='client.? ' entity='client.admin' 2024-09-10T23:57:28.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:57:27 smithi178 ceph-mon[35919]: Standby manager daemon z started 2024-09-10T23:57:28.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:57:27 smithi178 ceph-mon[35919]: from='mgr.? 172.21.15.178:0/1260535139' entity='mgr.z' cmd={"prefix": "config-key get", "key": "mgr/dashboard/z/crt"} : dispatch 2024-09-10T23:57:28.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:57:27 smithi178 ceph-mon[35919]: from='mgr.? 172.21.15.178:0/1260535139' entity='mgr.z' cmd={"prefix": "config-key get", "key": "mgr/dashboard/crt"} : dispatch 2024-09-10T23:57:28.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:57:27 smithi178 ceph-mon[35919]: from='mgr.? 172.21.15.178:0/1260535139' entity='mgr.z' cmd={"prefix": "config-key get", "key": "mgr/dashboard/z/key"} : dispatch 2024-09-10T23:57:28.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:57:27 smithi178 ceph-mon[35919]: from='mgr.? 172.21.15.178:0/1260535139' entity='mgr.z' cmd={"prefix": "config-key get", "key": "mgr/dashboard/key"} : dispatch 2024-09-10T23:57:28.178 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:28.176+0000 7fa41e400640 1 -- 172.21.15.12:0/4026422930 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa418105c00 msgr2=0x7fa418106080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:57:28.178 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:28.176+0000 7fa41e400640 1 --2- 172.21.15.12:0/4026422930 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa418105c00 0x7fa418106080 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fa404009930 tx=0x7fa40402a420 comp rx=0 tx=0).stop 2024-09-10T23:57:28.178 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:28.176+0000 7fa41e400640 1 -- 172.21.15.12:0/4026422930 shutdown_connections 2024-09-10T23:57:28.179 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:28.176+0000 7fa41e400640 1 --2- 172.21.15.12:0/4026422930 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa4181065c0 0x7fa41810cf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:57:28.179 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:28.176+0000 7fa41e400640 1 --2- 172.21.15.12:0/4026422930 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa418105c00 0x7fa418106080 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:57:28.179 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:28.176+0000 7fa41e400640 1 --2- 172.21.15.12:0/4026422930 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa418104aa0 0x7fa418104ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:57:28.179 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:28.176+0000 7fa41e400640 1 -- 172.21.15.12:0/4026422930 >> 172.21.15.12:0/4026422930 conn(0x7fa418100370 msgr2=0x7fa4181027b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:57:28.179 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:28.176+0000 7fa41e400640 1 -- 172.21.15.12:0/4026422930 shutdown_connections 2024-09-10T23:57:28.179 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:28.176+0000 7fa41e400640 1 -- 172.21.15.12:0/4026422930 wait complete. 2024-09-10T23:57:28.180 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:28.177+0000 7fa41e400640 1 Processor -- start 2024-09-10T23:57:28.180 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:28.178+0000 7fa41e400640 1 -- start start 2024-09-10T23:57:28.180 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:28.178+0000 7fa41e400640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa418104aa0 0x7fa41819ad00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:57:28.180 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:28.178+0000 7fa41e400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa418105c00 0x7fa41819b350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:57:28.180 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:28.178+0000 7fa41e400640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa4181065c0 0x7fa41819bf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:57:28.181 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:28.178+0000 7fa41e400640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fa418110610 con 0x7fa418104aa0 2024-09-10T23:57:28.181 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:28.178+0000 7fa41e400640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fa418110490 con 0x7fa418105c00 2024-09-10T23:57:28.181 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:28.178+0000 7fa41e400640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fa418110790 con 0x7fa4181065c0 2024-09-10T23:57:28.181 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:28.178+0000 7fa417e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa418105c00 0x7fa41819b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:57:28.181 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:28.178+0000 7fa417e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa418105c00 0x7fa41819b350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.12:59310/0 (socket says 172.21.15.12:59310) 2024-09-10T23:57:28.181 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:28.178+0000 7fa417e00640 1 -- 172.21.15.12:0/2157063947 learned_addr learned my addr 172.21.15.12:0/2157063947 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-10T23:57:28.181 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:28.178+0000 7fa41d000640 1 --2- 172.21.15.12:0/2157063947 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa418104aa0 0x7fa41819ad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:57:28.181 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:28.178+0000 7fa41da00640 1 --2- 172.21.15.12:0/2157063947 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa4181065c0 0x7fa41819bf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:57:28.182 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:28.179+0000 7fa417e00640 1 -- 172.21.15.12:0/2157063947 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa4181065c0 msgr2=0x7fa41819bf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:57:28.182 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:28.179+0000 7fa417e00640 1 --2- 172.21.15.12:0/2157063947 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa4181065c0 0x7fa41819bf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:57:28.182 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:28.179+0000 7fa417e00640 1 -- 172.21.15.12:0/2157063947 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa418104aa0 msgr2=0x7fa41819ad00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:57:28.182 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:28.179+0000 7fa417e00640 1 --2- 172.21.15.12:0/2157063947 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa418104aa0 0x7fa41819ad00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:57:28.182 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:28.179+0000 7fa417e00640 1 -- 172.21.15.12:0/2157063947 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa41819c830 con 0x7fa418105c00 2024-09-10T23:57:28.182 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:28.179+0000 7fa417e00640 1 --2- 172.21.15.12:0/2157063947 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa418105c00 0x7fa41819b350 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fa404005ab0 tx=0x7fa404003c80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:57:28.182 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:28.179+0000 7fa415600640 1 -- 172.21.15.12:0/2157063947 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa40403f070 con 0x7fa418105c00 2024-09-10T23:57:28.183 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:28.179+0000 7fa415600640 1 -- 172.21.15.12:0/2157063947 <== mon.2 v2:172.21.15.165:3300/0 2 ==== config(25 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fa404031d10 con 0x7fa418105c00 2024-09-10T23:57:28.183 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:28.179+0000 7fa415600640 1 -- 172.21.15.12:0/2157063947 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa404035030 con 0x7fa418105c00 2024-09-10T23:57:28.183 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:28.179+0000 7fa41e400640 1 -- 172.21.15.12:0/2157063947 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa41819cac0 con 0x7fa418105c00 2024-09-10T23:57:28.183 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:28.180+0000 7fa41e400640 1 -- 172.21.15.12:0/2157063947 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4181a3200 con 0x7fa418105c00 2024-09-10T23:57:28.183 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:28.181+0000 7fa41e400640 1 -- 172.21.15.12:0/2157063947 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa418104ea0 con 0x7fa418105c00 2024-09-10T23:57:28.183 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:28.182+0000 7fa415600640 1 -- 172.21.15.12:0/2157063947 <== mon.2 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 14) ==== 102337+0+0 (secure 0 0 0) 0x7fa40403b460 con 0x7fa418105c00 2024-09-10T23:57:28.185 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:28.183+0000 7fa415600640 1 --2- 172.21.15.12:0/2157063947 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fa40007a450 0x7fa40007c910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:57:28.185 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:28.183+0000 7fa415600640 1 -- 172.21.15.12:0/2157063947 <== mon.2 v2:172.21.15.165:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fa4040bc9b0 con 0x7fa418105c00 2024-09-10T23:57:28.185 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:28.183+0000 7fa41d000640 1 --2- 172.21.15.12:0/2157063947 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fa40007a450 0x7fa40007c910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:57:28.185 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:28.184+0000 7fa41d000640 1 --2- 172.21.15.12:0/2157063947 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fa40007a450 0x7fa40007c910 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fa40c00bbe0 tx=0x7fa40c00b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:57:28.189 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:28.187+0000 7fa415600640 1 -- 172.21.15.12:0/2157063947 <== mon.2 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7fa404084ff0 con 0x7fa418105c00 2024-09-10T23:57:28.394 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:28.392+0000 7fa41e400640 1 -- 172.21.15.12:0/2157063947 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command([{prefix=config set, name=mon_warn_on_pool_no_app}] v 0) -- 0x7fa418063d50 con 0x7fa418105c00 2024-09-10T23:57:28.403 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:28.400+0000 7fa415600640 1 -- 172.21.15.12:0/2157063947 <== mon.2 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_warn_on_pool_no_app}]=0 v18)=0 v18) ==== 125+0+0 (secure 0 0 0) 0x7fa404089a20 con 0x7fa418105c00 2024-09-10T23:57:28.408 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:28.404+0000 7fa41e400640 1 -- 172.21.15.12:0/2157063947 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fa40007a450 msgr2=0x7fa40007c910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:57:28.409 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:28.404+0000 7fa41e400640 1 --2- 172.21.15.12:0/2157063947 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fa40007a450 0x7fa40007c910 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fa40c00bbe0 tx=0x7fa40c00b390 comp rx=0 tx=0).stop 2024-09-10T23:57:28.409 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:28.404+0000 7fa41e400640 1 -- 172.21.15.12:0/2157063947 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa418105c00 msgr2=0x7fa41819b350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:57:28.409 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:28.404+0000 7fa41e400640 1 --2- 172.21.15.12:0/2157063947 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa418105c00 0x7fa41819b350 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fa404005ab0 tx=0x7fa404003c80 comp rx=0 tx=0).stop 2024-09-10T23:57:28.409 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:28.404+0000 7fa41e400640 1 -- 172.21.15.12:0/2157063947 shutdown_connections 2024-09-10T23:57:28.409 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:28.404+0000 7fa41e400640 1 --2- 172.21.15.12:0/2157063947 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fa40007a450 0x7fa40007c910 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:57:28.409 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:28.404+0000 7fa41e400640 1 --2- 172.21.15.12:0/2157063947 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa4181065c0 0x7fa41819bf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:57:28.409 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:28.404+0000 7fa41e400640 1 --2- 172.21.15.12:0/2157063947 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa418105c00 0x7fa41819b350 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:57:28.409 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:28.404+0000 7fa41e400640 1 --2- 172.21.15.12:0/2157063947 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa418104aa0 0x7fa41819ad00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:57:28.409 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:28.404+0000 7fa41e400640 1 -- 172.21.15.12:0/2157063947 >> 172.21.15.12:0/2157063947 conn(0x7fa418100370 msgr2=0x7fa418108f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:57:28.409 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:28.404+0000 7fa41e400640 1 -- 172.21.15.12:0/2157063947 shutdown_connections 2024-09-10T23:57:28.409 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:28.405+0000 7fa41e400640 1 -- 172.21.15.12:0/2157063947 wait complete. 2024-09-10T23:57:28.917 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:57:28 smithi012 ceph-mon[30333]: from='client.? ' entity='client.admin' 2024-09-10T23:57:28.917 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:57:28 smithi012 ceph-mon[30333]: mgrmap e14: x(active, since 69s), standbys: z 2024-09-10T23:57:28.917 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:57:28 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "mgr metadata", "who": "z", "id": "z"} : dispatch 2024-09-10T23:57:28.917 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:57:28 smithi012 ceph-mon[30333]: from='client.? ' entity='client.admin' 2024-09-10T23:57:28.917 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:57:28 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-10T23:57:28.918 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:57:28 smithi012 ceph-mon[30333]: from='client.? ' entity='client.admin' 2024-09-10T23:57:29.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:28 smithi165 ceph-mon[35356]: from='client.? ' entity='client.admin' 2024-09-10T23:57:29.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:28 smithi165 ceph-mon[35356]: mgrmap e14: x(active, since 69s), standbys: z 2024-09-10T23:57:29.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:28 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "mgr metadata", "who": "z", "id": "z"} : dispatch 2024-09-10T23:57:29.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:28 smithi165 ceph-mon[35356]: from='client.? ' entity='client.admin' 2024-09-10T23:57:29.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:28 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-10T23:57:29.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:28 smithi165 ceph-mon[35356]: from='client.? ' entity='client.admin' 2024-09-10T23:57:29.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:57:28 smithi178 ceph-mon[35919]: from='client.? ' entity='client.admin' 2024-09-10T23:57:29.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:57:28 smithi178 ceph-mon[35919]: mgrmap e14: x(active, since 69s), standbys: z 2024-09-10T23:57:29.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:57:28 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "mgr metadata", "who": "z", "id": "z"} : dispatch 2024-09-10T23:57:29.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:57:28 smithi178 ceph-mon[35919]: from='client.? ' entity='client.admin' 2024-09-10T23:57:29.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:57:28 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-10T23:57:29.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:57:28 smithi178 ceph-mon[35919]: from='client.? ' entity='client.admin' 2024-09-10T23:57:29.203 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:28.394+0000 7fd71fc00640 1 -- 172.21.15.12:0/1823063344 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd718102a30 msgr2=0x7fd718102e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:57:29.204 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:28.394+0000 7fd71fc00640 1 --2- 172.21.15.12:0/1823063344 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd718102a30 0x7fd718102e30 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fd70c009930 tx=0x7fd70c02a420 comp rx=0 tx=0).stop 2024-09-10T23:57:29.204 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:28.394+0000 7fd71fc00640 1 -- 172.21.15.12:0/1823063344 shutdown_connections 2024-09-10T23:57:29.204 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:28.394+0000 7fd71fc00640 1 --2- 172.21.15.12:0/1823063344 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd718104550 0x7fd71810aed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:57:29.204 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:28.394+0000 7fd71fc00640 1 --2- 172.21.15.12:0/1823063344 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd718103b90 0x7fd718104010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:57:29.204 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:28.394+0000 7fd71fc00640 1 --2- 172.21.15.12:0/1823063344 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd718102a30 0x7fd718102e30 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:57:29.204 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:28.394+0000 7fd71fc00640 1 -- 172.21.15.12:0/1823063344 >> 172.21.15.12:0/1823063344 conn(0x7fd7180fe320 msgr2=0x7fd718100740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:57:29.204 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:28.394+0000 7fd71fc00640 1 -- 172.21.15.12:0/1823063344 shutdown_connections 2024-09-10T23:57:29.204 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:28.394+0000 7fd71fc00640 1 -- 172.21.15.12:0/1823063344 wait complete. 2024-09-10T23:57:29.204 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:28.395+0000 7fd71fc00640 1 Processor -- start 2024-09-10T23:57:29.204 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:28.395+0000 7fd71fc00640 1 -- start start 2024-09-10T23:57:29.204 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:28.395+0000 7fd71fc00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd718102a30 0x7fd718076cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:57:29.205 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:28.395+0000 7fd71fc00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd718103b90 0x7fd718075540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:57:29.205 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:28.395+0000 7fd71fc00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd718104550 0x7fd718075a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:57:29.205 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:28.395+0000 7fd71fc00640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fd71810e5a0 con 0x7fd718103b90 2024-09-10T23:57:29.205 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:28.395+0000 7fd71fc00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fd71810e420 con 0x7fd718102a30 2024-09-10T23:57:29.205 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:28.395+0000 7fd71fc00640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fd71810e720 con 0x7fd718104550 2024-09-10T23:57:29.205 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:28.396+0000 7fd71d200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd718102a30 0x7fd718076cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:57:29.205 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:28.396+0000 7fd71d200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd718102a30 0x7fd718076cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.12:59318/0 (socket says 172.21.15.12:59318) 2024-09-10T23:57:29.205 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:28.396+0000 7fd71d200640 1 -- 172.21.15.12:0/1962508111 learned_addr learned my addr 172.21.15.12:0/1962508111 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-10T23:57:29.205 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:28.396+0000 7fd717e00640 1 --2- 172.21.15.12:0/1962508111 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd718103b90 0x7fd718075540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:57:29.205 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:28.396+0000 7fd71dc00640 1 --2- 172.21.15.12:0/1962508111 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd718104550 0x7fd718075a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:57:29.205 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:28.396+0000 7fd71d200640 1 -- 172.21.15.12:0/1962508111 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd718104550 msgr2=0x7fd718075a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:57:29.205 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:28.396+0000 7fd71d200640 1 --2- 172.21.15.12:0/1962508111 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd718104550 0x7fd718075a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:57:29.205 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:28.396+0000 7fd71d200640 1 -- 172.21.15.12:0/1962508111 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd718103b90 msgr2=0x7fd718075540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:57:29.206 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:28.396+0000 7fd71d200640 1 --2- 172.21.15.12:0/1962508111 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd718103b90 0x7fd718075540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:57:29.206 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:28.396+0000 7fd71d200640 1 -- 172.21.15.12:0/1962508111 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd718076340 con 0x7fd718102a30 2024-09-10T23:57:29.206 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:28.396+0000 7fd717e00640 1 --2- 172.21.15.12:0/1962508111 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd718103b90 0x7fd718075540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:57:29.206 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:28.396+0000 7fd71dc00640 1 --2- 172.21.15.12:0/1962508111 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd718104550 0x7fd718075a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:57:29.206 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:28.396+0000 7fd71d200640 1 --2- 172.21.15.12:0/1962508111 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd718102a30 0x7fd718076cd0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fd70c005ab0 tx=0x7fd70c003b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:57:29.206 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:28.399+0000 7fd715600640 1 -- 172.21.15.12:0/1962508111 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd70c03f070 con 0x7fd718102a30 2024-09-10T23:57:29.206 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:28.399+0000 7fd715600640 1 -- 172.21.15.12:0/1962508111 <== mon.2 v2:172.21.15.165:3300/0 2 ==== config(25 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fd70c031b00 con 0x7fd718102a30 2024-09-10T23:57:29.206 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:28.399+0000 7fd715600640 1 -- 172.21.15.12:0/1962508111 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd70c035030 con 0x7fd718102a30 2024-09-10T23:57:29.206 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:28.400+0000 7fd71fc00640 1 -- 172.21.15.12:0/1962508111 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd7181a2db0 con 0x7fd718102a30 2024-09-10T23:57:29.206 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:28.400+0000 7fd71fc00640 1 -- 172.21.15.12:0/1962508111 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd7181a31b0 con 0x7fd718102a30 2024-09-10T23:57:29.206 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:28.401+0000 7fd71fc00640 1 -- 172.21.15.12:0/1962508111 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd718109db0 con 0x7fd718102a30 2024-09-10T23:57:29.206 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:28.404+0000 7fd715600640 1 -- 172.21.15.12:0/1962508111 <== mon.2 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 14) ==== 102337+0+0 (secure 0 0 0) 0x7fd70c031670 con 0x7fd718102a30 2024-09-10T23:57:29.206 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:28.405+0000 7fd715600640 1 --2- 172.21.15.12:0/1962508111 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fd70007a3b0 0x7fd70007c870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:57:29.207 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:28.405+0000 7fd715600640 1 -- 172.21.15.12:0/1962508111 <== mon.2 v2:172.21.15.165:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fd70c0bc8e0 con 0x7fd718102a30 2024-09-10T23:57:29.207 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:28.405+0000 7fd717e00640 1 --2- 172.21.15.12:0/1962508111 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fd70007a3b0 0x7fd70007c870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:57:29.207 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:28.406+0000 7fd717e00640 1 --2- 172.21.15.12:0/1962508111 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fd70007a3b0 0x7fd70007c870 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fd704009c20 tx=0x7fd704005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:57:29.207 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:28.409+0000 7fd715600640 1 -- 172.21.15.12:0/1962508111 <== mon.2 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7fd70c085610 con 0x7fd718102a30 2024-09-10T23:57:29.207 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:28.619+0000 7fd71fc00640 1 -- 172.21.15.12:0/1962508111 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command([{prefix=config set, name=rados_mon_op_timeout}] v 0) -- 0x7fd718063df0 con 0x7fd718102a30 2024-09-10T23:57:29.207 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:28.625+0000 7fd715600640 1 -- 172.21.15.12:0/1962508111 <== mon.2 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{prefix=config set, name=rados_mon_op_timeout}]=0 v19)=0 v19) ==== 120+0+0 (secure 0 0 0) 0x7fd70c084f10 con 0x7fd718102a30 2024-09-10T23:57:29.207 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:28.627+0000 7fd71fc00640 1 -- 172.21.15.12:0/1962508111 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fd70007a3b0 msgr2=0x7fd70007c870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:57:29.207 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:28.627+0000 7fd71fc00640 1 --2- 172.21.15.12:0/1962508111 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fd70007a3b0 0x7fd70007c870 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fd704009c20 tx=0x7fd704005f90 comp rx=0 tx=0).stop 2024-09-10T23:57:29.207 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:28.627+0000 7fd71fc00640 1 -- 172.21.15.12:0/1962508111 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd718102a30 msgr2=0x7fd718076cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:57:29.207 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:28.627+0000 7fd71fc00640 1 --2- 172.21.15.12:0/1962508111 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd718102a30 0x7fd718076cd0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fd70c005ab0 tx=0x7fd70c003b90 comp rx=0 tx=0).stop 2024-09-10T23:57:29.207 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:28.627+0000 7fd71fc00640 1 -- 172.21.15.12:0/1962508111 shutdown_connections 2024-09-10T23:57:29.207 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:28.627+0000 7fd71fc00640 1 --2- 172.21.15.12:0/1962508111 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fd70007a3b0 0x7fd70007c870 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:57:29.207 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:28.627+0000 7fd71fc00640 1 --2- 172.21.15.12:0/1962508111 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd718104550 0x7fd718075a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:57:29.208 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:28.627+0000 7fd71fc00640 1 --2- 172.21.15.12:0/1962508111 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd718103b90 0x7fd718075540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:57:29.208 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:28.627+0000 7fd71fc00640 1 --2- 172.21.15.12:0/1962508111 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd718102a30 0x7fd718076cd0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:57:29.208 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:28.627+0000 7fd71fc00640 1 -- 172.21.15.12:0/1962508111 >> 172.21.15.12:0/1962508111 conn(0x7fd7180fe320 msgr2=0x7fd718106f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:57:29.208 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:28.627+0000 7fd71fc00640 1 -- 172.21.15.12:0/1962508111 shutdown_connections 2024-09-10T23:57:29.208 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:28.627+0000 7fd71fc00640 1 -- 172.21.15.12:0/1962508111 wait complete. 2024-09-10T23:57:29.435 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T23:57:29.436 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T23:57:29.808 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:29.805+0000 7f7fa0a00640 1 -- 172.21.15.12:0/1239364248 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7f9c102a30 msgr2=0x7f7f9c102e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:57:29.808 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:29.805+0000 7f7fa0a00640 1 --2- 172.21.15.12:0/1239364248 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7f9c102a30 0x7f7f9c102e30 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f7f8c009930 tx=0x7f7f8c02a420 comp rx=0 tx=0).stop 2024-09-10T23:57:29.808 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:29.805+0000 7f7fa0a00640 1 -- 172.21.15.12:0/1239364248 shutdown_connections 2024-09-10T23:57:29.808 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:29.805+0000 7f7fa0a00640 1 --2- 172.21.15.12:0/1239364248 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7f9c104550 0x7f7f9c10aed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:57:29.808 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:29.805+0000 7f7fa0a00640 1 --2- 172.21.15.12:0/1239364248 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7f9c103b90 0x7f7f9c104010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:57:29.808 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:29.805+0000 7f7fa0a00640 1 --2- 172.21.15.12:0/1239364248 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7f9c102a30 0x7f7f9c102e30 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:57:29.808 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:29.805+0000 7f7fa0a00640 1 -- 172.21.15.12:0/1239364248 >> 172.21.15.12:0/1239364248 conn(0x7f7f9c0fe320 msgr2=0x7f7f9c100740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:57:29.808 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:29.806+0000 7f7fa0a00640 1 -- 172.21.15.12:0/1239364248 shutdown_connections 2024-09-10T23:57:29.808 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:29.806+0000 7f7fa0a00640 1 -- 172.21.15.12:0/1239364248 wait complete. 2024-09-10T23:57:29.810 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:29.806+0000 7f7fa0a00640 1 Processor -- start 2024-09-10T23:57:29.810 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:29.807+0000 7f7fa0a00640 1 -- start start 2024-09-10T23:57:29.810 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:29.807+0000 7f7fa0a00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7f9c102a30 0x7f7f9c198d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:57:29.810 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:29.807+0000 7f7fa0a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7f9c103b90 0x7f7f9c199390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:57:29.810 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:29.807+0000 7f7fa0a00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7f9c104550 0x7f7f9c199fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:57:29.810 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:29.807+0000 7f7fa0a00640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f7f9c10e570 con 0x7f7f9c102a30 2024-09-10T23:57:29.810 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:29.807+0000 7f7fa0a00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f7f9c10e3f0 con 0x7f7f9c103b90 2024-09-10T23:57:29.810 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:29.807+0000 7f7fa0a00640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f7f9c10e6f0 con 0x7f7f9c104550 2024-09-10T23:57:29.810 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:29.807+0000 7f7f93e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7f9c103b90 0x7f7f9c199390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:57:29.810 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:29.807+0000 7f7f99e00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7f9c102a30 0x7f7f9c198d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:57:29.811 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:29.807+0000 7f7f93e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7f9c103b90 0x7f7f9c199390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.12:33866/0 (socket says 172.21.15.12:33866) 2024-09-10T23:57:29.811 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:29.807+0000 7f7f93e00640 1 -- 172.21.15.12:0/748549578 learned_addr learned my addr 172.21.15.12:0/748549578 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-10T23:57:29.811 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:29.807+0000 7f7f9a800640 1 --2- 172.21.15.12:0/748549578 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7f9c104550 0x7f7f9c199fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:57:29.811 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:29.808+0000 7f7f93e00640 1 -- 172.21.15.12:0/748549578 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7f9c104550 msgr2=0x7f7f9c199fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:57:29.811 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:29.808+0000 7f7f93e00640 1 --2- 172.21.15.12:0/748549578 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7f9c104550 0x7f7f9c199fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:57:29.811 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:29.808+0000 7f7f93e00640 1 -- 172.21.15.12:0/748549578 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7f9c102a30 msgr2=0x7f7f9c198d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:57:29.811 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:29.808+0000 7f7f93e00640 1 --2- 172.21.15.12:0/748549578 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7f9c102a30 0x7f7f9c198d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:57:29.811 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:29.808+0000 7f7f93e00640 1 -- 172.21.15.12:0/748549578 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f9c19a870 con 0x7f7f9c103b90 2024-09-10T23:57:29.811 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:29.808+0000 7f7f93e00640 1 --2- 172.21.15.12:0/748549578 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7f9c103b90 0x7f7f9c199390 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f7f8400d950 tx=0x7f7f8400de10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:57:29.812 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:29.808+0000 7f7f92a00640 1 -- 172.21.15.12:0/748549578 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f84017070 con 0x7f7f9c103b90 2024-09-10T23:57:29.812 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:29.808+0000 7f7f92a00640 1 -- 172.21.15.12:0/748549578 <== mon.2 v2:172.21.15.165:3300/0 2 ==== config(25 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f7f84009c30 con 0x7f7f9c103b90 2024-09-10T23:57:29.812 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:29.808+0000 7f7f92a00640 1 -- 172.21.15.12:0/748549578 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f8400f620 con 0x7f7f9c103b90 2024-09-10T23:57:29.812 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:29.808+0000 7f7fa0a00640 1 -- 172.21.15.12:0/748549578 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7f9c19ab60 con 0x7f7f9c103b90 2024-09-10T23:57:29.812 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:29.808+0000 7f7fa0a00640 1 -- 172.21.15.12:0/748549578 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7f9c075550 con 0x7f7f9c103b90 2024-09-10T23:57:29.812 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:29.810+0000 7f7fa0a00640 1 -- 172.21.15.12:0/748549578 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7f9c19adc0 con 0x7f7f9c103b90 2024-09-10T23:57:29.813 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:29.811+0000 7f7f92a00640 1 -- 172.21.15.12:0/748549578 <== mon.2 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 14) ==== 102337+0+0 (secure 0 0 0) 0x7f7f84010460 con 0x7f7f9c103b90 2024-09-10T23:57:29.813 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:29.812+0000 7f7f92a00640 1 --2- 172.21.15.12:0/748549578 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f7f8007a3b0 0x7f7f8007c870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:57:29.814 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:29.812+0000 7f7f92a00640 1 -- 172.21.15.12:0/748549578 <== mon.2 v2:172.21.15.165:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f7f8409ba00 con 0x7f7f9c103b90 2024-09-10T23:57:29.814 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:29.812+0000 7f7f99e00640 1 --2- 172.21.15.12:0/748549578 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f7f8007a3b0 0x7f7f8007c870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:57:29.814 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:29.812+0000 7f7f99e00640 1 --2- 172.21.15.12:0/748549578 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f7f8007a3b0 0x7f7f8007c870 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f7f8c009900 tx=0x7f7f8c005720 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:57:29.818 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:29.816+0000 7f7f92a00640 1 -- 172.21.15.12:0/748549578 <== mon.2 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f7f84009de0 con 0x7f7f9c103b90 2024-09-10T23:57:29.846 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:29.843+0000 7f7f92a00640 1 -- 172.21.15.12:0/748549578 <== mon.2 v2:172.21.15.165:3300/0 7 ==== mgrmap(e 15) ==== 153194+0+0 (secure 0 0 0) 0x7f7f840602b0 con 0x7f7f9c103b90 2024-09-10T23:57:30.009 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:57:29 smithi012 ceph-mon[30333]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T23:57:30.009 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:57:29 smithi012 ceph-mon[30333]: from='client.? ' entity='client.admin' 2024-09-10T23:57:30.009 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:57:29 smithi012 ceph-mon[30333]: Standby manager daemon y started 2024-09-10T23:57:30.009 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:57:29 smithi012 ceph-mon[30333]: from='mgr.? 172.21.15.165:0/2502005638' entity='mgr.y' cmd={"prefix": "config-key get", "key": "mgr/dashboard/y/crt"} : dispatch 2024-09-10T23:57:30.009 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:57:29 smithi012 ceph-mon[30333]: from='mgr.? 172.21.15.165:0/2502005638' entity='mgr.y' cmd={"prefix": "config-key get", "key": "mgr/dashboard/crt"} : dispatch 2024-09-10T23:57:30.009 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:57:29 smithi012 ceph-mon[30333]: from='mgr.? 172.21.15.165:0/2502005638' entity='mgr.y' cmd={"prefix": "config-key get", "key": "mgr/dashboard/y/key"} : dispatch 2024-09-10T23:57:30.010 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:57:29 smithi012 ceph-mon[30333]: from='mgr.? 172.21.15.165:0/2502005638' entity='mgr.y' cmd={"prefix": "config-key get", "key": "mgr/dashboard/key"} : dispatch 2024-09-10T23:57:30.028 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:30.026+0000 7f7fa0a00640 1 -- 172.21.15.12:0/748549578 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command([{prefix=config set, name=mds_bal_merge_size}] v 0) -- 0x7f7f9c0757b0 con 0x7f7f9c103b90 2024-09-10T23:57:30.034 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:30.032+0000 7f7f92a00640 1 -- 172.21.15.12:0/748549578 <== mon.2 v2:172.21.15.165:3300/0 8 ==== mon_command_ack([{prefix=config set, name=mds_bal_merge_size}]=0 v21)=0 v21) ==== 116+0+0 (secure 0 0 0) 0x7f7f840b58b0 con 0x7f7f9c103b90 2024-09-10T23:57:30.040 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:30.035+0000 7f7fa0a00640 1 -- 172.21.15.12:0/748549578 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f7f8007a3b0 msgr2=0x7f7f8007c870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:57:30.041 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:30.035+0000 7f7fa0a00640 1 --2- 172.21.15.12:0/748549578 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f7f8007a3b0 0x7f7f8007c870 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f7f8c009900 tx=0x7f7f8c005720 comp rx=0 tx=0).stop 2024-09-10T23:57:30.041 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:30.035+0000 7f7fa0a00640 1 -- 172.21.15.12:0/748549578 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7f9c103b90 msgr2=0x7f7f9c199390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:57:30.041 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:30.035+0000 7f7fa0a00640 1 --2- 172.21.15.12:0/748549578 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7f9c103b90 0x7f7f9c199390 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f7f8400d950 tx=0x7f7f8400de10 comp rx=0 tx=0).stop 2024-09-10T23:57:30.041 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:30.035+0000 7f7fa0a00640 1 -- 172.21.15.12:0/748549578 shutdown_connections 2024-09-10T23:57:30.041 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:30.035+0000 7f7fa0a00640 1 --2- 172.21.15.12:0/748549578 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f7f8007a3b0 0x7f7f8007c870 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:57:30.041 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:30.035+0000 7f7fa0a00640 1 --2- 172.21.15.12:0/748549578 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7f9c104550 0x7f7f9c199fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:57:30.041 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:30.036+0000 7f7fa0a00640 1 --2- 172.21.15.12:0/748549578 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7f9c103b90 0x7f7f9c199390 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:57:30.041 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:30.036+0000 7f7fa0a00640 1 --2- 172.21.15.12:0/748549578 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7f9c102a30 0x7f7f9c198d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:57:30.041 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:30.036+0000 7f7fa0a00640 1 -- 172.21.15.12:0/748549578 >> 172.21.15.12:0/748549578 conn(0x7f7f9c0fe320 msgr2=0x7f7f9c104f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:57:30.041 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:30.036+0000 7f7fa0a00640 1 -- 172.21.15.12:0/748549578 shutdown_connections 2024-09-10T23:57:30.041 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:30.036+0000 7f7fa0a00640 1 -- 172.21.15.12:0/748549578 wait complete. 2024-09-10T23:57:30.143 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:28.598+0000 7f8f3e600640 1 -- 172.21.15.12:0/2046940413 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8f38102890 msgr2=0x7f8f38102c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:57:30.143 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:28.598+0000 7f8f3e600640 1 --2- 172.21.15.12:0/2046940413 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8f38102890 0x7f8f38102c90 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f8f28009930 tx=0x7f8f2802a420 comp rx=0 tx=0).stop 2024-09-10T23:57:30.143 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:28.598+0000 7f8f3e600640 1 -- 172.21.15.12:0/2046940413 shutdown_connections 2024-09-10T23:57:30.143 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:28.598+0000 7f8f3e600640 1 --2- 172.21.15.12:0/2046940413 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8f381043b0 0x7f8f3810ad30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:57:30.143 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:28.598+0000 7f8f3e600640 1 --2- 172.21.15.12:0/2046940413 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8f381039f0 0x7f8f38103e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:57:30.143 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:28.598+0000 7f8f3e600640 1 --2- 172.21.15.12:0/2046940413 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8f38102890 0x7f8f38102c90 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:57:30.143 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:28.598+0000 7f8f3e600640 1 -- 172.21.15.12:0/2046940413 >> 172.21.15.12:0/2046940413 conn(0x7f8f380fe180 msgr2=0x7f8f381005a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:57:30.144 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:28.599+0000 7f8f3e600640 1 -- 172.21.15.12:0/2046940413 shutdown_connections 2024-09-10T23:57:30.144 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:28.599+0000 7f8f3e600640 1 -- 172.21.15.12:0/2046940413 wait complete. 2024-09-10T23:57:30.144 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:28.599+0000 7f8f3e600640 1 Processor -- start 2024-09-10T23:57:30.144 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:28.600+0000 7f8f3e600640 1 -- start start 2024-09-10T23:57:30.144 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:28.600+0000 7f8f3e600640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8f38102890 0x7f8f3819ad00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:57:30.144 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:28.600+0000 7f8f3e600640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8f381039f0 0x7f8f3819b350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:57:30.144 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:28.600+0000 7f8f3e600640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8f381043b0 0x7f8f3819bf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:57:30.144 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:28.600+0000 7f8f3e600640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f8f3810e400 con 0x7f8f381043b0 2024-09-10T23:57:30.144 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:28.600+0000 7f8f3e600640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f8f3810e280 con 0x7f8f381039f0 2024-09-10T23:57:30.144 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:28.600+0000 7f8f3e600640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f8f3810e580 con 0x7f8f38102890 2024-09-10T23:57:30.144 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:28.600+0000 7f8f3d200640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8f38102890 0x7f8f3819ad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:57:30.144 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:28.600+0000 7f8f2fe00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8f381039f0 0x7f8f3819b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:57:30.145 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:28.600+0000 7f8f3d200640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8f38102890 0x7f8f3819ad00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.12:43510/0 (socket says 172.21.15.12:43510) 2024-09-10T23:57:30.145 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:28.600+0000 7f8f3d200640 1 -- 172.21.15.12:0/3747933307 learned_addr learned my addr 172.21.15.12:0/3747933307 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-10T23:57:30.145 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:28.600+0000 7f8f3dc00640 1 --2- 172.21.15.12:0/3747933307 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8f381043b0 0x7f8f3819bf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:57:30.145 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:28.601+0000 7f8f2fe00640 1 -- 172.21.15.12:0/3747933307 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8f38102890 msgr2=0x7f8f3819ad00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:57:30.145 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:28.601+0000 7f8f2fe00640 1 --2- 172.21.15.12:0/3747933307 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8f38102890 0x7f8f3819ad00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:57:30.145 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:28.601+0000 7f8f2fe00640 1 -- 172.21.15.12:0/3747933307 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8f381043b0 msgr2=0x7f8f3819bf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:57:30.145 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:28.601+0000 7f8f2fe00640 1 --2- 172.21.15.12:0/3747933307 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8f381043b0 0x7f8f3819bf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:57:30.145 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:28.601+0000 7f8f2fe00640 1 -- 172.21.15.12:0/3747933307 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f3819c830 con 0x7f8f381039f0 2024-09-10T23:57:30.145 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:28.601+0000 7f8f2fe00640 1 --2- 172.21.15.12:0/3747933307 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8f381039f0 0x7f8f3819b350 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f8f2000e9f0 tx=0x7f8f2000eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:57:30.145 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:28.601+0000 7f8f36000640 1 -- 172.21.15.12:0/3747933307 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f2000ccc0 con 0x7f8f381039f0 2024-09-10T23:57:30.145 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:28.601+0000 7f8f3e600640 1 -- 172.21.15.12:0/3747933307 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f3819cb20 con 0x7f8f381039f0 2024-09-10T23:57:30.145 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:28.601+0000 7f8f36000640 1 -- 172.21.15.12:0/3747933307 <== mon.2 v2:172.21.15.165:3300/0 2 ==== config(25 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f8f200108c0 con 0x7f8f381039f0 2024-09-10T23:57:30.146 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:28.601+0000 7f8f36000640 1 -- 172.21.15.12:0/3747933307 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f200187c0 con 0x7f8f381039f0 2024-09-10T23:57:30.146 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:28.601+0000 7f8f3e600640 1 -- 172.21.15.12:0/3747933307 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f381a3200 con 0x7f8f381039f0 2024-09-10T23:57:30.146 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:28.602+0000 7f8f3e600640 1 -- 172.21.15.12:0/3747933307 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8f381a33d0 con 0x7f8f381039f0 2024-09-10T23:57:30.146 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:28.604+0000 7f8f36000640 1 -- 172.21.15.12:0/3747933307 <== mon.2 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 14) ==== 102337+0+0 (secure 0 0 0) 0x7f8f20010430 con 0x7f8f381039f0 2024-09-10T23:57:30.146 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:28.605+0000 7f8f36000640 1 --2- 172.21.15.12:0/3747933307 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f8f1c07a450 0x7f8f1c07c910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:57:30.146 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:28.605+0000 7f8f36000640 1 -- 172.21.15.12:0/3747933307 <== mon.2 v2:172.21.15.165:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f8f2009c190 con 0x7f8f381039f0 2024-09-10T23:57:30.146 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:28.605+0000 7f8f3d200640 1 --2- 172.21.15.12:0/3747933307 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f8f1c07a450 0x7f8f1c07c910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:57:30.146 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:28.606+0000 7f8f3d200640 1 --2- 172.21.15.12:0/3747933307 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f8f1c07a450 0x7f8f1c07c910 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f8f28005ec0 tx=0x7f8f28005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:57:30.146 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:28.609+0000 7f8f36000640 1 -- 172.21.15.12:0/3747933307 <== mon.2 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f8f20064750 con 0x7f8f381039f0 2024-09-10T23:57:30.146 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:28.828+0000 7f8f3e600640 1 -- 172.21.15.12:0/3747933307 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command([{prefix=config set, name=rados_osd_op_timeout}] v 0) -- 0x7f8f381a35a0 con 0x7f8f381039f0 2024-09-10T23:57:30.146 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:28.833+0000 7f8f36000640 1 -- 172.21.15.12:0/3747933307 <== mon.2 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{prefix=config set, name=rados_osd_op_timeout}]=0 v20)=0 v20) ==== 120+0+0 (secure 0 0 0) 0x7f8f20069180 con 0x7f8f381039f0 2024-09-10T23:57:30.147 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:28.835+0000 7f8f3e600640 1 -- 172.21.15.12:0/3747933307 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f8f1c07a450 msgr2=0x7f8f1c07c910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:57:30.147 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:28.835+0000 7f8f3e600640 1 --2- 172.21.15.12:0/3747933307 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f8f1c07a450 0x7f8f1c07c910 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f8f28005ec0 tx=0x7f8f28005a20 comp rx=0 tx=0).stop 2024-09-10T23:57:30.147 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:28.835+0000 7f8f3e600640 1 -- 172.21.15.12:0/3747933307 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8f381039f0 msgr2=0x7f8f3819b350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:57:30.147 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:28.835+0000 7f8f3e600640 1 --2- 172.21.15.12:0/3747933307 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8f381039f0 0x7f8f3819b350 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f8f2000e9f0 tx=0x7f8f2000eeb0 comp rx=0 tx=0).stop 2024-09-10T23:57:30.147 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:28.835+0000 7f8f3e600640 1 -- 172.21.15.12:0/3747933307 shutdown_connections 2024-09-10T23:57:30.147 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:28.835+0000 7f8f3e600640 1 --2- 172.21.15.12:0/3747933307 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f8f1c07a450 0x7f8f1c07c910 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:57:30.147 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:28.835+0000 7f8f3e600640 1 --2- 172.21.15.12:0/3747933307 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8f381043b0 0x7f8f3819bf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:57:30.147 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:28.835+0000 7f8f3e600640 1 --2- 172.21.15.12:0/3747933307 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8f381039f0 0x7f8f3819b350 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:57:30.147 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:28.835+0000 7f8f3e600640 1 --2- 172.21.15.12:0/3747933307 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8f38102890 0x7f8f3819ad00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:57:30.147 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:28.835+0000 7f8f3e600640 1 -- 172.21.15.12:0/3747933307 >> 172.21.15.12:0/3747933307 conn(0x7f8f380fe180 msgr2=0x7f8f38075380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:57:30.147 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:28.835+0000 7f8f3e600640 1 -- 172.21.15.12:0/3747933307 shutdown_connections 2024-09-10T23:57:30.148 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:28.835+0000 7f8f3e600640 1 -- 172.21.15.12:0/3747933307 wait complete. 2024-09-10T23:57:30.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:29 smithi165 ceph-mon[35356]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T23:57:30.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:29 smithi165 ceph-mon[35356]: from='client.? ' entity='client.admin' 2024-09-10T23:57:30.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:29 smithi165 ceph-mon[35356]: Standby manager daemon y started 2024-09-10T23:57:30.283 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:29 smithi165 ceph-mon[35356]: from='mgr.? 172.21.15.165:0/2502005638' entity='mgr.y' cmd={"prefix": "config-key get", "key": "mgr/dashboard/y/crt"} : dispatch 2024-09-10T23:57:30.283 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:29 smithi165 ceph-mon[35356]: from='mgr.? 172.21.15.165:0/2502005638' entity='mgr.y' cmd={"prefix": "config-key get", "key": "mgr/dashboard/crt"} : dispatch 2024-09-10T23:57:30.283 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:29 smithi165 ceph-mon[35356]: from='mgr.? 172.21.15.165:0/2502005638' entity='mgr.y' cmd={"prefix": "config-key get", "key": "mgr/dashboard/y/key"} : dispatch 2024-09-10T23:57:30.283 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:29 smithi165 ceph-mon[35356]: from='mgr.? 172.21.15.165:0/2502005638' entity='mgr.y' cmd={"prefix": "config-key get", "key": "mgr/dashboard/key"} : dispatch 2024-09-10T23:57:30.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:57:29 smithi178 ceph-mon[35919]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T23:57:30.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:57:29 smithi178 ceph-mon[35919]: from='client.? ' entity='client.admin' 2024-09-10T23:57:30.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:57:29 smithi178 ceph-mon[35919]: Standby manager daemon y started 2024-09-10T23:57:30.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:57:29 smithi178 ceph-mon[35919]: from='mgr.? 172.21.15.165:0/2502005638' entity='mgr.y' cmd={"prefix": "config-key get", "key": "mgr/dashboard/y/crt"} : dispatch 2024-09-10T23:57:30.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:57:29 smithi178 ceph-mon[35919]: from='mgr.? 172.21.15.165:0/2502005638' entity='mgr.y' cmd={"prefix": "config-key get", "key": "mgr/dashboard/crt"} : dispatch 2024-09-10T23:57:30.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:57:29 smithi178 ceph-mon[35919]: from='mgr.? 172.21.15.165:0/2502005638' entity='mgr.y' cmd={"prefix": "config-key get", "key": "mgr/dashboard/y/key"} : dispatch 2024-09-10T23:57:30.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:57:29 smithi178 ceph-mon[35919]: from='mgr.? 172.21.15.165:0/2502005638' entity='mgr.y' cmd={"prefix": "config-key get", "key": "mgr/dashboard/key"} : dispatch 2024-09-10T23:57:30.528 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:30.525+0000 7fdc32400640 1 -- 172.21.15.12:0/3207665411 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fdc2c104aa0 msgr2=0x7fdc2c104ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:57:30.528 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:30.525+0000 7fdc32400640 1 --2- 172.21.15.12:0/3207665411 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fdc2c104aa0 0x7fdc2c104ea0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fdc20009930 tx=0x7fdc2002a420 comp rx=0 tx=0).stop 2024-09-10T23:57:30.528 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:30.526+0000 7fdc32400640 1 -- 172.21.15.12:0/3207665411 shutdown_connections 2024-09-10T23:57:30.528 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:30.526+0000 7fdc32400640 1 --2- 172.21.15.12:0/3207665411 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdc2c1065c0 0x7fdc2c10cf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:57:30.528 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:30.526+0000 7fdc32400640 1 --2- 172.21.15.12:0/3207665411 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdc2c105c00 0x7fdc2c106080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:57:30.529 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:30.526+0000 7fdc32400640 1 --2- 172.21.15.12:0/3207665411 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fdc2c104aa0 0x7fdc2c104ea0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:57:30.529 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:30.526+0000 7fdc32400640 1 -- 172.21.15.12:0/3207665411 >> 172.21.15.12:0/3207665411 conn(0x7fdc2c100370 msgr2=0x7fdc2c1027b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:57:30.529 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:30.526+0000 7fdc32400640 1 -- 172.21.15.12:0/3207665411 shutdown_connections 2024-09-10T23:57:30.529 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:30.526+0000 7fdc32400640 1 -- 172.21.15.12:0/3207665411 wait complete. 2024-09-10T23:57:30.530 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:30.527+0000 7fdc32400640 1 Processor -- start 2024-09-10T23:57:30.530 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:30.527+0000 7fdc32400640 1 -- start start 2024-09-10T23:57:30.530 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:30.528+0000 7fdc32400640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdc2c104aa0 0x7fdc2c19aca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:57:30.530 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:30.528+0000 7fdc32400640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdc2c105c00 0x7fdc2c19b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:57:30.530 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:30.528+0000 7fdc32400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fdc2c1065c0 0x7fdc2c19bf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:57:30.530 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:30.528+0000 7fdc32400640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fdc2c110630 con 0x7fdc2c104aa0 2024-09-10T23:57:30.530 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:30.528+0000 7fdc32400640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fdc2c1104b0 con 0x7fdc2c1065c0 2024-09-10T23:57:30.530 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:30.528+0000 7fdc32400640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fdc2c1107b0 con 0x7fdc2c105c00 2024-09-10T23:57:30.530 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:30.528+0000 7fdc31000640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdc2c104aa0 0x7fdc2c19aca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:57:30.530 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:30.528+0000 7fdc31000640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdc2c104aa0 0x7fdc2c19aca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:55680/0 (socket says 172.21.15.12:55680) 2024-09-10T23:57:30.531 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:30.528+0000 7fdc31000640 1 -- 172.21.15.12:0/3900829867 learned_addr learned my addr 172.21.15.12:0/3900829867 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-10T23:57:30.531 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:30.528+0000 7fdc2be00640 1 --2- 172.21.15.12:0/3900829867 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdc2c105c00 0x7fdc2c19b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:57:30.531 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:30.528+0000 7fdc31a00640 1 --2- 172.21.15.12:0/3900829867 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fdc2c1065c0 0x7fdc2c19bf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:57:30.532 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:30.528+0000 7fdc31000640 1 -- 172.21.15.12:0/3900829867 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdc2c105c00 msgr2=0x7fdc2c19b2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:57:30.532 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:30.528+0000 7fdc31000640 1 --2- 172.21.15.12:0/3900829867 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdc2c105c00 0x7fdc2c19b2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:57:30.532 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:30.528+0000 7fdc31000640 1 -- 172.21.15.12:0/3900829867 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fdc2c1065c0 msgr2=0x7fdc2c19bf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:57:30.532 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:30.528+0000 7fdc31000640 1 --2- 172.21.15.12:0/3900829867 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fdc2c1065c0 0x7fdc2c19bf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:57:30.532 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:30.528+0000 7fdc31000640 1 -- 172.21.15.12:0/3900829867 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc2c19c7d0 con 0x7fdc2c104aa0 2024-09-10T23:57:30.532 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:30.529+0000 7fdc31000640 1 --2- 172.21.15.12:0/3900829867 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdc2c104aa0 0x7fdc2c19aca0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fdc20005ab0 tx=0x7fdc20003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:57:30.532 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:30.529+0000 7fdc29600640 1 -- 172.21.15.12:0/3900829867 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc2003f070 con 0x7fdc2c104aa0 2024-09-10T23:57:30.532 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:30.529+0000 7fdc29600640 1 -- 172.21.15.12:0/3900829867 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(25 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fdc20031b10 con 0x7fdc2c104aa0 2024-09-10T23:57:30.532 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:30.529+0000 7fdc29600640 1 -- 172.21.15.12:0/3900829867 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc20035030 con 0x7fdc2c104aa0 2024-09-10T23:57:30.533 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:30.529+0000 7fdc32400640 1 -- 172.21.15.12:0/3900829867 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdc2c19ca60 con 0x7fdc2c104aa0 2024-09-10T23:57:30.533 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:30.529+0000 7fdc32400640 1 -- 172.21.15.12:0/3900829867 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdc2c1a2fd0 con 0x7fdc2c104aa0 2024-09-10T23:57:30.533 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:30.530+0000 7fdc32400640 1 -- 172.21.15.12:0/3900829867 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdc2c10bd50 con 0x7fdc2c104aa0 2024-09-10T23:57:30.534 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:30.532+0000 7fdc29600640 1 -- 172.21.15.12:0/3900829867 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 15) ==== 153194+0+0 (secure 0 0 0) 0x7fdc20031680 con 0x7fdc2c104aa0 2024-09-10T23:57:30.536 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:30.534+0000 7fdc29600640 1 --2- 172.21.15.12:0/3900829867 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fdc140b53f0 0x7fdc140b78b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:57:30.536 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:30.534+0000 7fdc29600640 1 -- 172.21.15.12:0/3900829867 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fdc20104900 con 0x7fdc2c104aa0 2024-09-10T23:57:30.536 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:30.534+0000 7fdc2be00640 1 --2- 172.21.15.12:0/3900829867 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fdc140b53f0 0x7fdc140b78b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:57:30.537 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:30.535+0000 7fdc2be00640 1 --2- 172.21.15.12:0/3900829867 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fdc140b53f0 0x7fdc140b78b0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fdc180095f0 tx=0x7fdc18005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:57:30.540 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:30.538+0000 7fdc29600640 1 -- 172.21.15.12:0/3900829867 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7fdc200ccda0 con 0x7fdc2c104aa0 2024-09-10T23:57:30.729 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:30.727+0000 7f5345a00640 1 -- 172.21.15.12:0/249863910 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5340104aa0 msgr2=0x7f5340104ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:57:30.729 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:30.727+0000 7f5345a00640 1 --2- 172.21.15.12:0/249863910 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5340104aa0 0x7f5340104ea0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f5330009960 tx=0x7f533002a460 comp rx=0 tx=0).stop 2024-09-10T23:57:30.729 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:30.727+0000 7f5345a00640 1 -- 172.21.15.12:0/249863910 shutdown_connections 2024-09-10T23:57:30.729 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:30.727+0000 7f5345a00640 1 --2- 172.21.15.12:0/249863910 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f53401065c0 0x7f534010cf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:57:30.729 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:30.727+0000 7f5345a00640 1 --2- 172.21.15.12:0/249863910 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5340105c00 0x7f5340106080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:57:30.729 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:30.727+0000 7f5345a00640 1 --2- 172.21.15.12:0/249863910 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5340104aa0 0x7f5340104ea0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:57:30.730 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:30.727+0000 7f5345a00640 1 -- 172.21.15.12:0/249863910 >> 172.21.15.12:0/249863910 conn(0x7f5340100370 msgr2=0x7f53401027b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:57:30.730 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:30.727+0000 7f5345a00640 1 -- 172.21.15.12:0/249863910 shutdown_connections 2024-09-10T23:57:30.730 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:30.727+0000 7f5345a00640 1 -- 172.21.15.12:0/249863910 wait complete. 2024-09-10T23:57:30.731 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:30.728+0000 7f5345a00640 1 Processor -- start 2024-09-10T23:57:30.731 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:30.728+0000 7f5345a00640 1 -- start start 2024-09-10T23:57:30.731 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:30.728+0000 7f5345a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5340104aa0 0x7f534019ad80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:57:30.731 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:30.728+0000 7f5345a00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5340105c00 0x7f534019b3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:57:30.731 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:30.728+0000 7f5345a00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f53401065c0 0x7f534019c010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:57:30.731 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:30.728+0000 7f5345a00640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f5340110630 con 0x7f5340105c00 2024-09-10T23:57:30.731 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:30.728+0000 7f5345a00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f53401104b0 con 0x7f5340104aa0 2024-09-10T23:57:30.731 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:30.728+0000 7f5345a00640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f53401107b0 con 0x7f53401065c0 2024-09-10T23:57:30.731 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:30.728+0000 7f533f400640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5340105c00 0x7f534019b3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:57:30.731 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:30.729+0000 7f5345000640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f53401065c0 0x7f534019c010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:57:30.731 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:30.729+0000 7f533fe00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5340104aa0 0x7f534019ad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:57:30.731 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:30.729+0000 7f5345000640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f53401065c0 0x7f534019c010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.12:33946/0 (socket says 172.21.15.12:33946) 2024-09-10T23:57:30.732 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:30.729+0000 7f5345000640 1 -- 172.21.15.12:0/1426804390 learned_addr learned my addr 172.21.15.12:0/1426804390 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-10T23:57:30.732 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:30.729+0000 7f5345000640 1 -- 172.21.15.12:0/1426804390 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5340104aa0 msgr2=0x7f534019ad80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:57:30.733 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:30.729+0000 7f5345000640 1 --2- 172.21.15.12:0/1426804390 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5340104aa0 0x7f534019ad80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:57:30.733 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:30.729+0000 7f5345000640 1 -- 172.21.15.12:0/1426804390 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5340105c00 msgr2=0x7f534019b3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:57:30.733 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:30.729+0000 7f5345000640 1 --2- 172.21.15.12:0/1426804390 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5340105c00 0x7f534019b3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:57:30.733 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:30.729+0000 7f5345000640 1 -- 172.21.15.12:0/1426804390 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f534019c8b0 con 0x7f53401065c0 2024-09-10T23:57:30.733 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:30.729+0000 7f533fe00640 1 --2- 172.21.15.12:0/1426804390 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5340104aa0 0x7f534019ad80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:57:30.733 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:30.729+0000 7f533f400640 1 --2- 172.21.15.12:0/1426804390 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5340105c00 0x7f534019b3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:57:30.733 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:30.729+0000 7f5345000640 1 --2- 172.21.15.12:0/1426804390 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f53401065c0 0x7f534019c010 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f533800eb10 tx=0x7f533800c2d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:57:30.733 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:30.730+0000 7f533cc00640 1 -- 172.21.15.12:0/1426804390 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5338004b20 con 0x7f53401065c0 2024-09-10T23:57:30.733 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:30.730+0000 7f533cc00640 1 -- 172.21.15.12:0/1426804390 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(25 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f5338004cc0 con 0x7f53401065c0 2024-09-10T23:57:30.733 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:30.730+0000 7f5345a00640 1 -- 172.21.15.12:0/1426804390 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f534019cba0 con 0x7f53401065c0 2024-09-10T23:57:30.733 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:30.730+0000 7f533cc00640 1 -- 172.21.15.12:0/1426804390 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5338010690 con 0x7f53401065c0 2024-09-10T23:57:30.733 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:30.730+0000 7f5345a00640 1 -- 172.21.15.12:0/1426804390 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53401a3280 con 0x7f53401065c0 2024-09-10T23:57:30.734 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:30.731+0000 7f5345a00640 1 -- 172.21.15.12:0/1426804390 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f534010bdc0 con 0x7f53401065c0 2024-09-10T23:57:30.735 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:30.733+0000 7f533cc00640 1 -- 172.21.15.12:0/1426804390 <== mon.1 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 15) ==== 153194+0+0 (secure 0 0 0) 0x7f5338005000 con 0x7f53401065c0 2024-09-10T23:57:30.736 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:30.734+0000 7f533cc00640 1 --2- 172.21.15.12:0/1426804390 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f53240b53f0 0x7f53240b78b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:57:30.737 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:30.735+0000 7f533cc00640 1 -- 172.21.15.12:0/1426804390 <== mon.1 v2:172.21.15.178:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f53380e4490 con 0x7f53401065c0 2024-09-10T23:57:30.737 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:30.735+0000 7f533fe00640 1 --2- 172.21.15.12:0/1426804390 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f53240b53f0 0x7f53240b78b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:57:30.737 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:30.735+0000 7f533fe00640 1 --2- 172.21.15.12:0/1426804390 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f53240b53f0 0x7f53240b78b0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f5330000c00 tx=0x7f5330005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:57:30.740 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:30.738+0000 7f533cc00640 1 -- 172.21.15.12:0/1426804390 <== mon.1 v2:172.21.15.178:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f53380acf80 con 0x7f53401065c0 2024-09-10T23:57:30.749 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:30.747+0000 7fdc32400640 1 -- 172.21.15.12:0/3900829867 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command([{prefix=config set, name=mds_bal_fragment_size_max}] v 0) -- 0x7fdc2c02e5c0 con 0x7fdc2c104aa0 2024-09-10T23:57:30.753 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:30.751+0000 7fdc29600640 1 -- 172.21.15.12:0/3900829867 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_bal_fragment_size_max}]=0 v22)=0 v22) ==== 127+0+0 (secure 0 0 0) 0x7fdc200d17d0 con 0x7fdc2c104aa0 2024-09-10T23:57:30.756 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:30.754+0000 7fdc32400640 1 -- 172.21.15.12:0/3900829867 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fdc140b53f0 msgr2=0x7fdc140b78b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:57:30.756 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:30.754+0000 7fdc32400640 1 --2- 172.21.15.12:0/3900829867 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fdc140b53f0 0x7fdc140b78b0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fdc180095f0 tx=0x7fdc18005f90 comp rx=0 tx=0).stop 2024-09-10T23:57:30.756 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:30.754+0000 7fdc32400640 1 -- 172.21.15.12:0/3900829867 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdc2c104aa0 msgr2=0x7fdc2c19aca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:57:30.756 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:30.754+0000 7fdc32400640 1 --2- 172.21.15.12:0/3900829867 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdc2c104aa0 0x7fdc2c19aca0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fdc20005ab0 tx=0x7fdc20003c10 comp rx=0 tx=0).stop 2024-09-10T23:57:30.756 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:30.754+0000 7fdc32400640 1 -- 172.21.15.12:0/3900829867 shutdown_connections 2024-09-10T23:57:30.757 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:30.754+0000 7fdc32400640 1 --2- 172.21.15.12:0/3900829867 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fdc140b53f0 0x7fdc140b78b0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:57:30.757 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:30.754+0000 7fdc32400640 1 --2- 172.21.15.12:0/3900829867 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fdc2c1065c0 0x7fdc2c19bf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:57:30.757 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:30.754+0000 7fdc32400640 1 --2- 172.21.15.12:0/3900829867 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdc2c105c00 0x7fdc2c19b2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:57:30.757 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:30.754+0000 7fdc32400640 1 --2- 172.21.15.12:0/3900829867 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdc2c104aa0 0x7fdc2c19aca0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:57:30.757 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:30.754+0000 7fdc32400640 1 -- 172.21.15.12:0/3900829867 >> 172.21.15.12:0/3900829867 conn(0x7fdc2c100370 msgr2=0x7fdc2c108f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:57:30.757 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:30.754+0000 7fdc32400640 1 -- 172.21.15.12:0/3900829867 shutdown_connections 2024-09-10T23:57:30.757 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:30.754+0000 7fdc32400640 1 -- 172.21.15.12:0/3900829867 wait complete. 2024-09-10T23:57:30.953 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:30.951+0000 7f5345a00640 1 -- 172.21.15.12:0/1426804390 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command([{prefix=config set, name=debug_mds}] v 0) -- 0x7f534002e5c0 con 0x7f53401065c0 2024-09-10T23:57:30.953 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:57:30 smithi012 ceph-mon[30333]: mgrmap e15: x(active, since 71s), standbys: z, y 2024-09-10T23:57:30.954 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:57:30 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "mgr metadata", "who": "y", "id": "y"} : dispatch 2024-09-10T23:57:30.954 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:57:30 smithi012 ceph-mon[30333]: from='client.? ' entity='client.admin' 2024-09-10T23:57:30.954 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:57:30 smithi012 ceph-mon[30333]: from='client.? 172.21.15.12:0/3900829867' entity='client.admin' 2024-09-10T23:57:30.960 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:30.958+0000 7f533cc00640 1 -- 172.21.15.12:0/1426804390 <== mon.1 v2:172.21.15.178:3300/0 7 ==== mon_command_ack([{prefix=config set, name=debug_mds}]=0 v23)=0 v23) ==== 108+0+0 (secure 0 0 0) 0x7f53380ac880 con 0x7f53401065c0 2024-09-10T23:57:30.963 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:30.960+0000 7f5345a00640 1 -- 172.21.15.12:0/1426804390 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f53240b53f0 msgr2=0x7f53240b78b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:57:30.963 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:30.960+0000 7f5345a00640 1 --2- 172.21.15.12:0/1426804390 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f53240b53f0 0x7f53240b78b0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f5330000c00 tx=0x7f5330005a20 comp rx=0 tx=0).stop 2024-09-10T23:57:30.963 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:30.960+0000 7f5345a00640 1 -- 172.21.15.12:0/1426804390 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f53401065c0 msgr2=0x7f534019c010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:57:30.963 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:30.960+0000 7f5345a00640 1 --2- 172.21.15.12:0/1426804390 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f53401065c0 0x7f534019c010 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f533800eb10 tx=0x7f533800c2d0 comp rx=0 tx=0).stop 2024-09-10T23:57:30.963 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:30.960+0000 7f5345a00640 1 -- 172.21.15.12:0/1426804390 shutdown_connections 2024-09-10T23:57:30.963 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:30.960+0000 7f5345a00640 1 --2- 172.21.15.12:0/1426804390 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f53240b53f0 0x7f53240b78b0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:57:30.963 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:30.960+0000 7f5345a00640 1 --2- 172.21.15.12:0/1426804390 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f53401065c0 0x7f534019c010 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:57:30.963 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:30.960+0000 7f5345a00640 1 --2- 172.21.15.12:0/1426804390 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5340105c00 0x7f534019b3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:57:30.963 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:30.960+0000 7f5345a00640 1 --2- 172.21.15.12:0/1426804390 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5340104aa0 0x7f534019ad80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:57:30.963 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:30.960+0000 7f5345a00640 1 -- 172.21.15.12:0/1426804390 >> 172.21.15.12:0/1426804390 conn(0x7f5340100370 msgr2=0x7f5340108f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:57:30.963 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:30.961+0000 7f5345a00640 1 -- 172.21.15.12:0/1426804390 shutdown_connections 2024-09-10T23:57:30.964 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:30.961+0000 7f5345a00640 1 -- 172.21.15.12:0/1426804390 wait complete. 2024-09-10T23:57:31.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:30 smithi165 ceph-mon[35356]: mgrmap e15: x(active, since 71s), standbys: z, y 2024-09-10T23:57:31.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:30 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "mgr metadata", "who": "y", "id": "y"} : dispatch 2024-09-10T23:57:31.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:30 smithi165 ceph-mon[35356]: from='client.? ' entity='client.admin' 2024-09-10T23:57:31.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:30 smithi165 ceph-mon[35356]: from='client.? 172.21.15.12:0/3900829867' entity='client.admin' 2024-09-10T23:57:31.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:57:30 smithi178 ceph-mon[35919]: mgrmap e15: x(active, since 71s), standbys: z, y 2024-09-10T23:57:31.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:57:30 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "mgr metadata", "who": "y", "id": "y"} : dispatch 2024-09-10T23:57:31.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:57:30 smithi178 ceph-mon[35919]: from='client.? ' entity='client.admin' 2024-09-10T23:57:31.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:57:30 smithi178 ceph-mon[35919]: from='client.? 172.21.15.12:0/3900829867' entity='client.admin' 2024-09-10T23:57:32.062 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:57:31 smithi012 ceph-mon[30333]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T23:57:32.063 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:57:31 smithi012 ceph-mon[30333]: from='client.? ' entity='client.admin' 2024-09-10T23:57:32.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:31 smithi165 ceph-mon[35356]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T23:57:32.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:31 smithi165 ceph-mon[35356]: from='client.? ' entity='client.admin' 2024-09-10T23:57:32.330 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:57:31 smithi178 ceph-mon[35919]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T23:57:32.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:57:31 smithi178 ceph-mon[35919]: from='client.? ' entity='client.admin' 2024-09-10T23:57:33.813 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:57:33 smithi012 ceph-mon[30333]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T23:57:33.814 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:57:33 smithi012 ceph-mon[30333]: from='client.? 172.21.15.12:0/1066251683' entity='client.admin' 2024-09-10T23:57:33.814 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:57:33 smithi012 ceph-mon[30333]: from='client.? 172.21.15.12:0/3542586147' entity='client.admin' 2024-09-10T23:57:34.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:33 smithi165 ceph-mon[35356]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T23:57:34.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:33 smithi165 ceph-mon[35356]: from='client.? 172.21.15.12:0/1066251683' entity='client.admin' 2024-09-10T23:57:34.033 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:33 smithi165 ceph-mon[35356]: from='client.? 172.21.15.12:0/3542586147' entity='client.admin' 2024-09-10T23:57:34.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:57:33 smithi178 ceph-mon[35919]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T23:57:34.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:57:33 smithi178 ceph-mon[35919]: from='client.? 172.21.15.12:0/1066251683' entity='client.admin' 2024-09-10T23:57:34.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:57:33 smithi178 ceph-mon[35919]: from='client.? 172.21.15.12:0/3542586147' entity='client.admin' 2024-09-10T23:57:34.409 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:34.406+0000 7f7f4e600640 1 -- 172.21.15.12:0/248610896 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7f48105da0 msgr2=0x7f7f48106220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:57:34.409 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:34.406+0000 7f7f4e600640 1 --2- 172.21.15.12:0/248610896 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7f48105da0 0x7f7f48106220 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f7f30009930 tx=0x7f7f3002a420 comp rx=0 tx=0).stop 2024-09-10T23:57:34.409 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:34.407+0000 7f7f4e600640 1 -- 172.21.15.12:0/248610896 shutdown_connections 2024-09-10T23:57:34.409 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:34.407+0000 7f7f4e600640 1 --2- 172.21.15.12:0/248610896 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7f48106760 0x7f7f4810d0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:57:34.409 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:34.407+0000 7f7f4e600640 1 --2- 172.21.15.12:0/248610896 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7f48105da0 0x7f7f48106220 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:57:34.409 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:34.407+0000 7f7f4e600640 1 --2- 172.21.15.12:0/248610896 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7f48104c40 0x7f7f48105040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:57:34.410 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:34.407+0000 7f7f4e600640 1 -- 172.21.15.12:0/248610896 >> 172.21.15.12:0/248610896 conn(0x7f7f48100510 msgr2=0x7f7f48102950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:57:34.410 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:34.407+0000 7f7f4e600640 1 -- 172.21.15.12:0/248610896 shutdown_connections 2024-09-10T23:57:34.410 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:34.407+0000 7f7f4e600640 1 -- 172.21.15.12:0/248610896 wait complete. 2024-09-10T23:57:34.411 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:34.408+0000 7f7f4e600640 1 Processor -- start 2024-09-10T23:57:34.411 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:34.408+0000 7f7f4e600640 1 -- start start 2024-09-10T23:57:34.411 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:34.408+0000 7f7f4e600640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7f48104c40 0x7f7f4819afe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:57:34.411 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:34.408+0000 7f7f4e600640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7f48105da0 0x7f7f4819b630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:57:34.412 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:34.408+0000 7f7f4e600640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7f48106760 0x7f7f4819c270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:57:34.412 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:34.408+0000 7f7f4e600640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f7f48110910 con 0x7f7f48106760 2024-09-10T23:57:34.412 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:34.408+0000 7f7f4e600640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f7f48110790 con 0x7f7f48105da0 2024-09-10T23:57:34.412 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:34.408+0000 7f7f4e600640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f7f48110a90 con 0x7f7f48104c40 2024-09-10T23:57:34.412 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:34.408+0000 7f7f47400640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7f48104c40 0x7f7f4819afe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:57:34.412 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:34.408+0000 7f7f3ea00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7f48105da0 0x7f7f4819b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:57:34.412 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:34.408+0000 7f7f47400640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7f48104c40 0x7f7f4819afe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.12:34006/0 (socket says 172.21.15.12:34006) 2024-09-10T23:57:34.412 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:34.408+0000 7f7f47400640 1 -- 172.21.15.12:0/4133501745 learned_addr learned my addr 172.21.15.12:0/4133501745 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-10T23:57:34.412 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:34.408+0000 7f7f47e00640 1 --2- 172.21.15.12:0/4133501745 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7f48106760 0x7f7f4819c270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:57:34.412 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:34.409+0000 7f7f47400640 1 -- 172.21.15.12:0/4133501745 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7f48105da0 msgr2=0x7f7f4819b630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:57:34.412 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:34.409+0000 7f7f47400640 1 --2- 172.21.15.12:0/4133501745 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7f48105da0 0x7f7f4819b630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:57:34.413 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:34.409+0000 7f7f47400640 1 -- 172.21.15.12:0/4133501745 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7f48106760 msgr2=0x7f7f4819c270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:57:34.413 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:34.409+0000 7f7f47400640 1 --2- 172.21.15.12:0/4133501745 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7f48106760 0x7f7f4819c270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:57:34.413 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:34.409+0000 7f7f47400640 1 -- 172.21.15.12:0/4133501745 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f4819cb10 con 0x7f7f48104c40 2024-09-10T23:57:34.413 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:34.409+0000 7f7f47400640 1 --2- 172.21.15.12:0/4133501745 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7f48104c40 0x7f7f4819afe0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f7f3800e9f0 tx=0x7f7f3800edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:57:34.413 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:34.409+0000 7f7f44c00640 1 -- 172.21.15.12:0/4133501745 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f3800cd80 con 0x7f7f48104c40 2024-09-10T23:57:34.413 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:34.409+0000 7f7f44c00640 1 -- 172.21.15.12:0/4133501745 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7f380054d0 con 0x7f7f48104c40 2024-09-10T23:57:34.413 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:34.409+0000 7f7f4e600640 1 -- 172.21.15.12:0/4133501745 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7f4819ce00 con 0x7f7f48104c40 2024-09-10T23:57:34.413 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:34.409+0000 7f7f44c00640 1 -- 172.21.15.12:0/4133501745 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f38010690 con 0x7f7f48104c40 2024-09-10T23:57:34.414 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:34.409+0000 7f7f4e600640 1 -- 172.21.15.12:0/4133501745 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7f481a34e0 con 0x7f7f48104c40 2024-09-10T23:57:34.414 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:34.410+0000 7f7f4e600640 1 -- 172.21.15.12:0/4133501745 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7f4810bfa0 con 0x7f7f48104c40 2024-09-10T23:57:34.414 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:34.413+0000 7f7f44c00640 1 -- 172.21.15.12:0/4133501745 <== mon.1 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 15) ==== 153194+0+0 (secure 0 0 0) 0x7f7f38004e30 con 0x7f7f48104c40 2024-09-10T23:57:34.416 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:34.414+0000 7f7f44c00640 1 --2- 172.21.15.12:0/4133501745 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f7f2c0b52d0 0x7f7f2c0b7790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:57:34.416 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:34.414+0000 7f7f44c00640 1 -- 172.21.15.12:0/4133501745 <== mon.1 v2:172.21.15.178:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f7f380e4500 con 0x7f7f48104c40 2024-09-10T23:57:34.416 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:34.414+0000 7f7f3ea00640 1 --2- 172.21.15.12:0/4133501745 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f7f2c0b52d0 0x7f7f2c0b7790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:57:34.417 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:34.415+0000 7f7f3ea00640 1 --2- 172.21.15.12:0/4133501745 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f7f2c0b52d0 0x7f7f2c0b7790 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f7f30009900 tx=0x7f7f30031450 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:57:34.417 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:34.416+0000 7f7f44c00640 1 -- 172.21.15.12:0/4133501745 <== mon.1 v2:172.21.15.178:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f7f38056220 con 0x7f7f48104c40 2024-09-10T23:57:34.467 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:33.054+0000 7f7b26400640 1 -- 172.21.15.12:0/2848645124 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7b20105c00 msgr2=0x7f7b20106080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:57:34.468 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:33.054+0000 7f7b26400640 1 --2- 172.21.15.12:0/2848645124 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7b20105c00 0x7f7b20106080 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f7b0c009930 tx=0x7f7b0c02a420 comp rx=0 tx=0).stop 2024-09-10T23:57:34.468 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:33.054+0000 7f7b26400640 1 -- 172.21.15.12:0/2848645124 shutdown_connections 2024-09-10T23:57:34.468 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:33.054+0000 7f7b26400640 1 --2- 172.21.15.12:0/2848645124 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7b201065c0 0x7f7b2010cf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:57:34.468 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:33.054+0000 7f7b26400640 1 --2- 172.21.15.12:0/2848645124 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7b20105c00 0x7f7b20106080 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:57:34.468 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:33.054+0000 7f7b26400640 1 --2- 172.21.15.12:0/2848645124 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7b20104aa0 0x7f7b20104ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:57:34.468 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:33.054+0000 7f7b26400640 1 -- 172.21.15.12:0/2848645124 >> 172.21.15.12:0/2848645124 conn(0x7f7b20100370 msgr2=0x7f7b201027b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:57:34.468 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:33.054+0000 7f7b26400640 1 -- 172.21.15.12:0/2848645124 shutdown_connections 2024-09-10T23:57:34.468 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:33.054+0000 7f7b26400640 1 -- 172.21.15.12:0/2848645124 wait complete. 2024-09-10T23:57:34.468 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:33.055+0000 7f7b26400640 1 Processor -- start 2024-09-10T23:57:34.468 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:33.055+0000 7f7b26400640 1 -- start start 2024-09-10T23:57:34.468 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:33.055+0000 7f7b26400640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7b20104aa0 0x7f7b2019ad50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:57:34.469 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:33.055+0000 7f7b26400640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7b20105c00 0x7f7b2019b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:57:34.469 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:33.055+0000 7f7b26400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7b201065c0 0x7f7b2019bfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:57:34.469 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:33.055+0000 7f7b26400640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f7b20110770 con 0x7f7b20104aa0 2024-09-10T23:57:34.469 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:33.055+0000 7f7b26400640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f7b201105f0 con 0x7f7b201065c0 2024-09-10T23:57:34.469 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:33.055+0000 7f7b26400640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f7b201108f0 con 0x7f7b20105c00 2024-09-10T23:57:34.469 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:33.055+0000 7f7b25000640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7b20104aa0 0x7f7b2019ad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:57:34.469 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:33.055+0000 7f7b25000640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7b20104aa0 0x7f7b2019ad50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:55728/0 (socket says 172.21.15.12:55728) 2024-09-10T23:57:34.469 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:33.055+0000 7f7b25000640 1 -- 172.21.15.12:0/3542586147 learned_addr learned my addr 172.21.15.12:0/3542586147 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-10T23:57:34.469 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:33.055+0000 7f7b1fe00640 1 --2- 172.21.15.12:0/3542586147 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7b20105c00 0x7f7b2019b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:57:34.469 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:33.055+0000 7f7b25a00640 1 --2- 172.21.15.12:0/3542586147 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7b201065c0 0x7f7b2019bfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:57:34.469 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:33.056+0000 7f7b25000640 1 -- 172.21.15.12:0/3542586147 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7b20105c00 msgr2=0x7f7b2019b3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:57:34.469 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:33.056+0000 7f7b25000640 1 --2- 172.21.15.12:0/3542586147 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7b20105c00 0x7f7b2019b3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:57:34.469 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:33.056+0000 7f7b25000640 1 -- 172.21.15.12:0/3542586147 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7b201065c0 msgr2=0x7f7b2019bfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:57:34.470 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:33.056+0000 7f7b25000640 1 --2- 172.21.15.12:0/3542586147 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7b201065c0 0x7f7b2019bfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:57:34.470 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:33.056+0000 7f7b25000640 1 -- 172.21.15.12:0/3542586147 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b2019c880 con 0x7f7b20104aa0 2024-09-10T23:57:34.470 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:33.056+0000 7f7b25000640 1 --2- 172.21.15.12:0/3542586147 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7b20104aa0 0x7f7b2019ad50 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f7b1400ea60 tx=0x7f7b1400ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:57:34.470 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:33.056+0000 7f7b1d600640 1 -- 172.21.15.12:0/3542586147 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b1400cde0 con 0x7f7b20104aa0 2024-09-10T23:57:34.470 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:33.056+0000 7f7b1d600640 1 -- 172.21.15.12:0/3542586147 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7b140035f0 con 0x7f7b20104aa0 2024-09-10T23:57:34.470 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:33.056+0000 7f7b1d600640 1 -- 172.21.15.12:0/3542586147 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b14010690 con 0x7f7b20104aa0 2024-09-10T23:57:34.470 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:33.056+0000 7f7b26400640 1 -- 172.21.15.12:0/3542586147 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7b2019cb70 con 0x7f7b20104aa0 2024-09-10T23:57:34.470 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:33.056+0000 7f7b26400640 1 -- 172.21.15.12:0/3542586147 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7b201a3250 con 0x7f7b20104aa0 2024-09-10T23:57:34.470 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:33.058+0000 7f7b12a00640 1 -- 172.21.15.12:0/3542586147 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7b20063d50 con 0x7f7b20104aa0 2024-09-10T23:57:34.470 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:33.060+0000 7f7b1d600640 1 -- 172.21.15.12:0/3542586147 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 15) ==== 153194+0+0 (secure 0 0 0) 0x7f7b14004000 con 0x7f7b20104aa0 2024-09-10T23:57:34.470 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:33.061+0000 7f7b1d600640 1 --2- 172.21.15.12:0/3542586147 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f7b080b5370 0x7f7b080b7830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:57:34.470 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:33.061+0000 7f7b1d600640 1 -- 172.21.15.12:0/3542586147 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f7b140e41a0 con 0x7f7b20104aa0 2024-09-10T23:57:34.471 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:33.061+0000 7f7b1fe00640 1 --2- 172.21.15.12:0/3542586147 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f7b080b5370 0x7f7b080b7830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:57:34.471 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:33.062+0000 7f7b1fe00640 1 --2- 172.21.15.12:0/3542586147 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f7b080b5370 0x7f7b080b7830 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f7b0c009900 tx=0x7f7b0c031450 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:57:34.471 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:33.065+0000 7f7b1d600640 1 -- 172.21.15.12:0/3542586147 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f7b140acc90 con 0x7f7b20104aa0 2024-09-10T23:57:34.471 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:33.270+0000 7f7b12a00640 1 -- 172.21.15.12:0/3542586147 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command([{prefix=config set, name=mds_verify_scatter}] v 0) -- 0x7f7b20063f20 con 0x7f7b20104aa0 2024-09-10T23:57:34.471 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:33.276+0000 7f7b1d600640 1 -- 172.21.15.12:0/3542586147 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_verify_scatter}]=0 v25)=0 v25) ==== 119+0+0 (secure 0 0 0) 0x7f7b140ac590 con 0x7f7b20104aa0 2024-09-10T23:57:34.471 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:33.279+0000 7f7b12a00640 1 -- 172.21.15.12:0/3542586147 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f7b080b5370 msgr2=0x7f7b080b7830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:57:34.471 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:33.279+0000 7f7b12a00640 1 --2- 172.21.15.12:0/3542586147 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f7b080b5370 0x7f7b080b7830 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f7b0c009900 tx=0x7f7b0c031450 comp rx=0 tx=0).stop 2024-09-10T23:57:34.471 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:33.279+0000 7f7b12a00640 1 -- 172.21.15.12:0/3542586147 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7b20104aa0 msgr2=0x7f7b2019ad50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:57:34.471 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:33.279+0000 7f7b12a00640 1 --2- 172.21.15.12:0/3542586147 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7b20104aa0 0x7f7b2019ad50 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f7b1400ea60 tx=0x7f7b1400ef20 comp rx=0 tx=0).stop 2024-09-10T23:57:34.471 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:33.279+0000 7f7b12a00640 1 -- 172.21.15.12:0/3542586147 shutdown_connections 2024-09-10T23:57:34.471 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:33.279+0000 7f7b12a00640 1 --2- 172.21.15.12:0/3542586147 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f7b080b5370 0x7f7b080b7830 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:57:34.471 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:33.279+0000 7f7b12a00640 1 --2- 172.21.15.12:0/3542586147 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7b201065c0 0x7f7b2019bfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:57:34.471 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:33.279+0000 7f7b12a00640 1 --2- 172.21.15.12:0/3542586147 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7b20105c00 0x7f7b2019b3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:57:34.472 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:33.279+0000 7f7b12a00640 1 --2- 172.21.15.12:0/3542586147 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7b20104aa0 0x7f7b2019ad50 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:57:34.472 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:33.279+0000 7f7b12a00640 1 -- 172.21.15.12:0/3542586147 >> 172.21.15.12:0/3542586147 conn(0x7f7b20100370 msgr2=0x7f7b20108f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:57:34.472 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:33.279+0000 7f7b12a00640 1 -- 172.21.15.12:0/3542586147 shutdown_connections 2024-09-10T23:57:34.472 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:33.279+0000 7f7b12a00640 1 -- 172.21.15.12:0/3542586147 wait complete. 2024-09-10T23:57:34.620 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:34.618+0000 7f7f4e600640 1 -- 172.21.15.12:0/4133501745 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command([{prefix=config set, name=mds_bal_split_bits}] v 0) -- 0x7f7f4802e5c0 con 0x7f7f48104c40 2024-09-10T23:57:34.629 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:34.627+0000 7f7f44c00640 1 -- 172.21.15.12:0/4133501745 <== mon.1 v2:172.21.15.178:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_bal_split_bits}]=0 v27)=0 v27) ==== 116+0+0 (secure 0 0 0) 0x7f7f380acff0 con 0x7f7f48104c40 2024-09-10T23:57:34.631 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:34.629+0000 7f7f4e600640 1 -- 172.21.15.12:0/4133501745 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f7f2c0b52d0 msgr2=0x7f7f2c0b7790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:57:34.631 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:34.629+0000 7f7f4e600640 1 --2- 172.21.15.12:0/4133501745 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f7f2c0b52d0 0x7f7f2c0b7790 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f7f30009900 tx=0x7f7f30031450 comp rx=0 tx=0).stop 2024-09-10T23:57:34.631 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:34.629+0000 7f7f4e600640 1 -- 172.21.15.12:0/4133501745 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7f48104c40 msgr2=0x7f7f4819afe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:57:34.632 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:34.629+0000 7f7f4e600640 1 --2- 172.21.15.12:0/4133501745 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7f48104c40 0x7f7f4819afe0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f7f3800e9f0 tx=0x7f7f3800edf0 comp rx=0 tx=0).stop 2024-09-10T23:57:34.632 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:34.629+0000 7f7f4e600640 1 -- 172.21.15.12:0/4133501745 shutdown_connections 2024-09-10T23:57:34.632 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:34.629+0000 7f7f4e600640 1 --2- 172.21.15.12:0/4133501745 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f7f2c0b52d0 0x7f7f2c0b7790 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:57:34.632 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:34.629+0000 7f7f4e600640 1 --2- 172.21.15.12:0/4133501745 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7f48106760 0x7f7f4819c270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:57:34.632 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:34.629+0000 7f7f4e600640 1 --2- 172.21.15.12:0/4133501745 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7f48105da0 0x7f7f4819b630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:57:34.632 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:34.629+0000 7f7f4e600640 1 --2- 172.21.15.12:0/4133501745 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7f48104c40 0x7f7f4819afe0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:57:34.632 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:34.629+0000 7f7f4e600640 1 -- 172.21.15.12:0/4133501745 >> 172.21.15.12:0/4133501745 conn(0x7f7f48100510 msgr2=0x7f7f48109170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:57:34.632 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:34.629+0000 7f7f4e600640 1 -- 172.21.15.12:0/4133501745 shutdown_connections 2024-09-10T23:57:34.632 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:34.629+0000 7f7f4e600640 1 -- 172.21.15.12:0/4133501745 wait complete. 2024-09-10T23:57:34.774 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:57:34 smithi012 ceph-mon[30333]: from='client.? ' entity='client.admin' 2024-09-10T23:57:34.775 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:57:34 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-10T23:57:34.775 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:57:34 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-10T23:57:34.775 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:57:34 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T23:57:34.775 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:57:34 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T23:57:34.775 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:57:34 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T23:57:34.775 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:57:34 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-10T23:57:34.775 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:57:34 smithi012 ceph-mon[30333]: from='client.? ' entity='client.admin' 2024-09-10T23:57:35.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:34 smithi165 ceph-mon[35356]: from='client.? ' entity='client.admin' 2024-09-10T23:57:35.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:34 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-10T23:57:35.033 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:34 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-10T23:57:35.033 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:34 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T23:57:35.033 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:34 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T23:57:35.033 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:34 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T23:57:35.033 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:34 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-10T23:57:35.033 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:34 smithi165 ceph-mon[35356]: from='client.? ' entity='client.admin' 2024-09-10T23:57:35.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:57:34 smithi178 ceph-mon[35919]: from='client.? ' entity='client.admin' 2024-09-10T23:57:35.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:57:34 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-10T23:57:35.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:57:34 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-10T23:57:35.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:57:34 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T23:57:35.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:57:34 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T23:57:35.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:57:34 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T23:57:35.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:57:34 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-10T23:57:35.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:57:34 smithi178 ceph-mon[35919]: from='client.? ' entity='client.admin' 2024-09-10T23:57:35.498 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:32.364+0000 7f99d1e00640 1 -- 172.21.15.12:0/1172941631 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f99cc075ac0 msgr2=0x7f99cc075f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:57:35.498 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:32.364+0000 7f99d1e00640 1 --2- 172.21.15.12:0/1172941631 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f99cc075ac0 0x7f99cc075f40 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f99b4009900 tx=0x7f99b402a3f0 comp rx=0 tx=0).stop 2024-09-10T23:57:35.498 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:32.365+0000 7f99d1e00640 1 -- 172.21.15.12:0/1172941631 shutdown_connections 2024-09-10T23:57:35.498 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:32.365+0000 7f99d1e00640 1 --2- 172.21.15.12:0/1172941631 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f99cc10a9b0 0x7f99cc10ce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:57:35.498 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:32.365+0000 7f99d1e00640 1 --2- 172.21.15.12:0/1172941631 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f99cc075ac0 0x7f99cc075f40 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:57:35.498 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:32.365+0000 7f99d1e00640 1 --2- 172.21.15.12:0/1172941631 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f99cc076f20 0x7f99cc075580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:57:35.498 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:32.365+0000 7f99d1e00640 1 -- 172.21.15.12:0/1172941631 >> 172.21.15.12:0/1172941631 conn(0x7f99cc0fe2f0 msgr2=0x7f99cc100730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:57:35.499 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:32.365+0000 7f99d1e00640 1 -- 172.21.15.12:0/1172941631 shutdown_connections 2024-09-10T23:57:35.499 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:32.365+0000 7f99d1e00640 1 -- 172.21.15.12:0/1172941631 wait complete. 2024-09-10T23:57:35.499 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:32.366+0000 7f99d1e00640 1 Processor -- start 2024-09-10T23:57:35.499 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:32.366+0000 7f99d1e00640 1 -- start start 2024-09-10T23:57:35.499 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:32.366+0000 7f99d1e00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f99cc075ac0 0x7f99cc19af20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:57:35.499 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:32.366+0000 7f99d1e00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f99cc076f20 0x7f99cc19b570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:57:35.499 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:32.366+0000 7f99d1e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f99cc10a9b0 0x7f99cc19c1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:57:35.499 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:32.366+0000 7f99d1e00640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f99cc1107d0 con 0x7f99cc075ac0 2024-09-10T23:57:35.499 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:32.366+0000 7f99d1e00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f99cc110650 con 0x7f99cc10a9b0 2024-09-10T23:57:35.499 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:32.366+0000 7f99d1e00640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f99cc110950 con 0x7f99cc076f20 2024-09-10T23:57:35.499 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:32.366+0000 7f99caa00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f99cc075ac0 0x7f99cc19af20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:57:35.499 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:32.366+0000 7f99caa00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f99cc075ac0 0x7f99cc19af20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:55724/0 (socket says 172.21.15.12:55724) 2024-09-10T23:57:35.499 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:32.366+0000 7f99caa00640 1 -- 172.21.15.12:0/1066251683 learned_addr learned my addr 172.21.15.12:0/1066251683 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-10T23:57:35.500 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:32.366+0000 7f99c3e00640 1 --2- 172.21.15.12:0/1066251683 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f99cc076f20 0x7f99cc19b570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:57:35.500 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:32.367+0000 7f99cb400640 1 --2- 172.21.15.12:0/1066251683 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f99cc10a9b0 0x7f99cc19c1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:57:35.500 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:32.367+0000 7f99caa00640 1 -- 172.21.15.12:0/1066251683 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f99cc076f20 msgr2=0x7f99cc19b570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:57:35.500 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:32.367+0000 7f99caa00640 1 --2- 172.21.15.12:0/1066251683 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f99cc076f20 0x7f99cc19b570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:57:35.500 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:32.367+0000 7f99caa00640 1 -- 172.21.15.12:0/1066251683 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f99cc10a9b0 msgr2=0x7f99cc19c1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:57:35.500 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:32.367+0000 7f99caa00640 1 --2- 172.21.15.12:0/1066251683 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f99cc10a9b0 0x7f99cc19c1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:57:35.500 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:32.367+0000 7f99caa00640 1 -- 172.21.15.12:0/1066251683 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99cc19ca50 con 0x7f99cc075ac0 2024-09-10T23:57:35.500 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:32.367+0000 7f99cb400640 1 --2- 172.21.15.12:0/1066251683 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f99cc10a9b0 0x7f99cc19c1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T23:57:35.500 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:32.367+0000 7f99caa00640 1 --2- 172.21.15.12:0/1066251683 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f99cc075ac0 0x7f99cc19af20 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f99bc00e9f0 tx=0x7f99bc00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:57:35.500 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:32.367+0000 7f99d0a00640 1 -- 172.21.15.12:0/1066251683 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99bc00cd80 con 0x7f99cc075ac0 2024-09-10T23:57:35.500 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:32.367+0000 7f99d0a00640 1 -- 172.21.15.12:0/1066251683 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(25 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f99bc0108c0 con 0x7f99cc075ac0 2024-09-10T23:57:35.500 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:32.367+0000 7f99d1e00640 1 -- 172.21.15.12:0/1066251683 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f99cc19cd40 con 0x7f99cc075ac0 2024-09-10T23:57:35.501 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:32.368+0000 7f99d0a00640 1 -- 172.21.15.12:0/1066251683 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99bc0187d0 con 0x7f99cc075ac0 2024-09-10T23:57:35.501 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:32.368+0000 7f99d1e00640 1 -- 172.21.15.12:0/1066251683 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99cc1a3420 con 0x7f99cc075ac0 2024-09-10T23:57:35.501 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:32.369+0000 7f99d1e00640 1 -- 172.21.15.12:0/1066251683 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f99a80052c0 con 0x7f99cc075ac0 2024-09-10T23:57:35.501 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:32.371+0000 7f99d0a00640 1 -- 172.21.15.12:0/1066251683 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 15) ==== 153194+0+0 (secure 0 0 0) 0x7f99bc010430 con 0x7f99cc075ac0 2024-09-10T23:57:35.501 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:32.374+0000 7f99d0a00640 1 --2- 172.21.15.12:0/1066251683 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f99b00b5350 0x7f99b00b7810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:57:35.501 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:32.374+0000 7f99d0a00640 1 -- 172.21.15.12:0/1066251683 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f99bc0e4bc0 con 0x7f99cc075ac0 2024-09-10T23:57:35.501 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:32.374+0000 7f99c3e00640 1 --2- 172.21.15.12:0/1066251683 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f99b00b5350 0x7f99b00b7810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:57:35.501 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:32.375+0000 7f99c3e00640 1 --2- 172.21.15.12:0/1066251683 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f99b00b5350 0x7f99b00b7810 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f99b4006010 tx=0x7f99b4000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:57:35.501 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:32.375+0000 7f99d0a00640 1 -- 172.21.15.12:0/1066251683 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f99bc026070 con 0x7f99cc075ac0 2024-09-10T23:57:35.501 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:32.579+0000 7f99d1e00640 1 -- 172.21.15.12:0/1066251683 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command([{prefix=config set, name=rados_osd_op_timeout}] v 0) -- 0x7f99a80055b0 con 0x7f99cc075ac0 2024-09-10T23:57:35.501 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:32.583+0000 7f99d0a00640 1 -- 172.21.15.12:0/1066251683 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{prefix=config set, name=rados_osd_op_timeout}]=0 v24)=0 v24) ==== 123+0+0 (secure 0 0 0) 0x7f99bc0ad6b0 con 0x7f99cc075ac0 2024-09-10T23:57:35.501 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:32.583+0000 7f99d0a00640 1 -- 172.21.15.12:0/1066251683 <== mon.0 v2:172.21.15.12:3300/0 8 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f99bc0acac0 con 0x7f99cc075ac0 2024-09-10T23:57:35.502 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:32.585+0000 7f99d1e00640 1 -- 172.21.15.12:0/1066251683 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f99b00b5350 msgr2=0x7f99b00b7810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:57:35.502 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:32.585+0000 7f99d1e00640 1 --2- 172.21.15.12:0/1066251683 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f99b00b5350 0x7f99b00b7810 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f99b4006010 tx=0x7f99b4000bc0 comp rx=0 tx=0).stop 2024-09-10T23:57:35.502 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:32.585+0000 7f99d1e00640 1 -- 172.21.15.12:0/1066251683 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f99cc075ac0 msgr2=0x7f99cc19af20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:57:35.502 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:32.585+0000 7f99d1e00640 1 --2- 172.21.15.12:0/1066251683 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f99cc075ac0 0x7f99cc19af20 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f99bc00e9f0 tx=0x7f99bc00eeb0 comp rx=0 tx=0).stop 2024-09-10T23:57:35.502 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:32.585+0000 7f99d1e00640 1 -- 172.21.15.12:0/1066251683 shutdown_connections 2024-09-10T23:57:35.502 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:32.585+0000 7f99d1e00640 1 --2- 172.21.15.12:0/1066251683 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f99b00b5350 0x7f99b00b7810 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:57:35.502 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:32.585+0000 7f99d1e00640 1 --2- 172.21.15.12:0/1066251683 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f99cc10a9b0 0x7f99cc19c1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:57:35.502 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:32.585+0000 7f99d1e00640 1 --2- 172.21.15.12:0/1066251683 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f99cc076f20 0x7f99cc19b570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:57:35.502 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:32.585+0000 7f99d1e00640 1 --2- 172.21.15.12:0/1066251683 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f99cc075ac0 0x7f99cc19af20 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:57:35.502 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:32.585+0000 7f99d1e00640 1 -- 172.21.15.12:0/1066251683 >> 172.21.15.12:0/1066251683 conn(0x7f99cc0fe2f0 msgr2=0x7f99cc0fe910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:57:35.502 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:32.585+0000 7f99d1e00640 1 -- 172.21.15.12:0/1066251683 shutdown_connections 2024-09-10T23:57:35.502 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:32.586+0000 7f99d1e00640 1 -- 172.21.15.12:0/1066251683 wait complete. 2024-09-10T23:57:36.006 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:57:35 smithi012 ceph-mon[30333]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T23:57:36.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:35 smithi165 ceph-mon[35356]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T23:57:36.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:57:35 smithi178 ceph-mon[35919]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T23:57:36.228 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T23:57:37.166 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:33.440+0000 7ffadc200640 1 -- 172.21.15.12:0/837729200 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ffad4103b90 msgr2=0x7ffad4104010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:57:37.166 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:33.440+0000 7ffadc200640 1 --2- 172.21.15.12:0/837729200 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ffad4103b90 0x7ffad4104010 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7ffac0009930 tx=0x7ffac002a420 comp rx=0 tx=0).stop 2024-09-10T23:57:37.166 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:33.441+0000 7ffadc200640 1 -- 172.21.15.12:0/837729200 shutdown_connections 2024-09-10T23:57:37.166 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:33.441+0000 7ffadc200640 1 --2- 172.21.15.12:0/837729200 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ffad4104550 0x7ffad410aed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:57:37.166 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:33.441+0000 7ffadc200640 1 --2- 172.21.15.12:0/837729200 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ffad4103b90 0x7ffad4104010 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:57:37.166 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:33.441+0000 7ffadc200640 1 --2- 172.21.15.12:0/837729200 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ffad4102a30 0x7ffad4102e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:57:37.166 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:33.441+0000 7ffadc200640 1 -- 172.21.15.12:0/837729200 >> 172.21.15.12:0/837729200 conn(0x7ffad40fe320 msgr2=0x7ffad4100740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:57:37.166 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:33.441+0000 7ffadc200640 1 -- 172.21.15.12:0/837729200 shutdown_connections 2024-09-10T23:57:37.167 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:33.441+0000 7ffadc200640 1 -- 172.21.15.12:0/837729200 wait complete. 2024-09-10T23:57:37.167 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:33.442+0000 7ffadc200640 1 Processor -- start 2024-09-10T23:57:37.167 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:33.442+0000 7ffadc200640 1 -- start start 2024-09-10T23:57:37.167 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:33.442+0000 7ffadc200640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ffad4102a30 0x7ffad419b000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:57:37.167 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:33.442+0000 7ffadc200640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ffad4103b90 0x7ffad419b650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:57:37.167 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:33.442+0000 7ffadc200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ffad4104550 0x7ffad419c290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:57:37.167 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:33.442+0000 7ffadc200640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7ffad410e700 con 0x7ffad4103b90 2024-09-10T23:57:37.167 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:33.442+0000 7ffadc200640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7ffad410e580 con 0x7ffad4104550 2024-09-10T23:57:37.167 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:33.442+0000 7ffadc200640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7ffad410e880 con 0x7ffad4102a30 2024-09-10T23:57:37.167 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:33.442+0000 7ffad8e00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ffad4103b90 0x7ffad419b650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:57:37.167 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:33.442+0000 7ffad9800640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ffad4102a30 0x7ffad419b000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:57:37.167 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:33.442+0000 7ffad9800640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ffad4102a30 0x7ffad419b000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.12:33982/0 (socket says 172.21.15.12:33982) 2024-09-10T23:57:37.167 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:33.442+0000 7ffad8e00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ffad4103b90 0x7ffad419b650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:55752/0 (socket says 172.21.15.12:55752) 2024-09-10T23:57:37.168 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:33.442+0000 7ffad9800640 1 -- 172.21.15.12:0/644590464 learned_addr learned my addr 172.21.15.12:0/644590464 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-10T23:57:37.168 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:33.443+0000 7ffada200640 1 --2- 172.21.15.12:0/644590464 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ffad4104550 0x7ffad419c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:57:37.168 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:33.443+0000 7ffad9800640 1 -- 172.21.15.12:0/644590464 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ffad4104550 msgr2=0x7ffad419c290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:57:37.168 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:33.443+0000 7ffad9800640 1 --2- 172.21.15.12:0/644590464 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ffad4104550 0x7ffad419c290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:57:37.168 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:33.443+0000 7ffad9800640 1 -- 172.21.15.12:0/644590464 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ffad4103b90 msgr2=0x7ffad419b650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:57:37.168 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:33.443+0000 7ffad9800640 1 --2- 172.21.15.12:0/644590464 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ffad4103b90 0x7ffad419b650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:57:37.168 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:33.443+0000 7ffad9800640 1 -- 172.21.15.12:0/644590464 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffad419cb30 con 0x7ffad4102a30 2024-09-10T23:57:37.168 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:33.443+0000 7ffad8e00640 1 --2- 172.21.15.12:0/644590464 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ffad4103b90 0x7ffad419b650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:57:37.168 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:33.443+0000 7ffad9800640 1 --2- 172.21.15.12:0/644590464 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ffad4102a30 0x7ffad419b000 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7ffac800e9f0 tx=0x7ffac800edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:57:37.168 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:33.443+0000 7ffad2000640 1 -- 172.21.15.12:0/644590464 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffac800cd80 con 0x7ffad4102a30 2024-09-10T23:57:37.168 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:33.443+0000 7ffad2000640 1 -- 172.21.15.12:0/644590464 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffac80054d0 con 0x7ffad4102a30 2024-09-10T23:57:37.168 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:33.443+0000 7ffad2000640 1 -- 172.21.15.12:0/644590464 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffac8010690 con 0x7ffad4102a30 2024-09-10T23:57:37.169 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:33.443+0000 7ffadc200640 1 -- 172.21.15.12:0/644590464 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffad419ce20 con 0x7ffad4102a30 2024-09-10T23:57:37.169 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:33.444+0000 7ffadc200640 1 -- 172.21.15.12:0/644590464 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffad41a3500 con 0x7ffad4102a30 2024-09-10T23:57:37.169 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:33.445+0000 7ffadc200640 1 -- 172.21.15.12:0/644590464 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffad4102e30 con 0x7ffad4102a30 2024-09-10T23:57:37.169 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:33.447+0000 7ffad2000640 1 -- 172.21.15.12:0/644590464 <== mon.1 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 15) ==== 153194+0+0 (secure 0 0 0) 0x7ffac8004e90 con 0x7ffad4102a30 2024-09-10T23:57:37.169 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:33.448+0000 7ffad2000640 1 --2- 172.21.15.12:0/644590464 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7ffabc0b52d0 0x7ffabc0b7790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:57:37.169 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:33.448+0000 7ffad2000640 1 -- 172.21.15.12:0/644590464 <== mon.1 v2:172.21.15.178:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7ffac80e43f0 con 0x7ffad4102a30 2024-09-10T23:57:37.169 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:33.448+0000 7ffad8e00640 1 --2- 172.21.15.12:0/644590464 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7ffabc0b52d0 0x7ffabc0b7790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:57:37.169 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:33.449+0000 7ffad8e00640 1 --2- 172.21.15.12:0/644590464 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7ffabc0b52d0 0x7ffabc0b7790 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7ffac0009900 tx=0x7ffac0031450 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:57:37.169 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:33.452+0000 7ffad2000640 1 -- 172.21.15.12:0/644590464 <== mon.1 v2:172.21.15.178:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7ffac80acee0 con 0x7ffad4102a30 2024-09-10T23:57:37.169 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:33.653+0000 7ffadc200640 1 -- 172.21.15.12:0/644590464 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command([{prefix=config set, name=debug_ms}] v 0) -- 0x7ffad41a36d0 con 0x7ffad4102a30 2024-09-10T23:57:37.169 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:33.661+0000 7ffad2000640 1 -- 172.21.15.12:0/644590464 <== mon.1 v2:172.21.15.178:3300/0 7 ==== mon_command_ack([{prefix=config set, name=debug_ms}]=0 v26)=0 v26) ==== 106+0+0 (secure 0 0 0) 0x7ffac80ac7e0 con 0x7ffad4102a30 2024-09-10T23:57:37.169 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:33.663+0000 7ffadc200640 1 -- 172.21.15.12:0/644590464 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7ffabc0b52d0 msgr2=0x7ffabc0b7790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:57:37.169 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:33.663+0000 7ffadc200640 1 --2- 172.21.15.12:0/644590464 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7ffabc0b52d0 0x7ffabc0b7790 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7ffac0009900 tx=0x7ffac0031450 comp rx=0 tx=0).stop 2024-09-10T23:57:37.170 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:33.663+0000 7ffadc200640 1 -- 172.21.15.12:0/644590464 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ffad4102a30 msgr2=0x7ffad419b000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:57:37.170 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:33.663+0000 7ffadc200640 1 --2- 172.21.15.12:0/644590464 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ffad4102a30 0x7ffad419b000 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7ffac800e9f0 tx=0x7ffac800edf0 comp rx=0 tx=0).stop 2024-09-10T23:57:37.170 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:33.663+0000 7ffadc200640 1 -- 172.21.15.12:0/644590464 shutdown_connections 2024-09-10T23:57:37.170 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:33.663+0000 7ffadc200640 1 --2- 172.21.15.12:0/644590464 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7ffabc0b52d0 0x7ffabc0b7790 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:57:37.170 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:33.663+0000 7ffadc200640 1 --2- 172.21.15.12:0/644590464 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ffad4104550 0x7ffad419c290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:57:37.170 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:33.663+0000 7ffadc200640 1 --2- 172.21.15.12:0/644590464 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ffad4103b90 0x7ffad419b650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:57:37.170 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:33.663+0000 7ffadc200640 1 --2- 172.21.15.12:0/644590464 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ffad4102a30 0x7ffad419b000 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:57:37.170 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:33.663+0000 7ffadc200640 1 -- 172.21.15.12:0/644590464 >> 172.21.15.12:0/644590464 conn(0x7ffad40fe320 msgr2=0x7ffad4109290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:57:37.170 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:33.663+0000 7ffadc200640 1 -- 172.21.15.12:0/644590464 shutdown_connections 2024-09-10T23:57:37.170 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:33.663+0000 7ffadc200640 1 -- 172.21.15.12:0/644590464 wait complete. 2024-09-10T23:57:37.418 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:37.416+0000 7f0fc3000640 1 -- 172.21.15.12:0/1422941508 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0fbc077a10 msgr2=0x7f0fbc077e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:57:37.419 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:37.416+0000 7f0fc3000640 1 --2- 172.21.15.12:0/1422941508 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0fbc077a10 0x7f0fbc077e10 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f0fac009900 tx=0x7f0fac02a3f0 comp rx=0 tx=0).stop 2024-09-10T23:57:37.419 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:37.416+0000 7f0fc3000640 1 -- 172.21.15.12:0/1422941508 shutdown_connections 2024-09-10T23:57:37.419 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:37.416+0000 7f0fc3000640 1 --2- 172.21.15.12:0/1422941508 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0fbc079530 0x7f0fbc079e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:57:37.419 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:37.416+0000 7f0fc3000640 1 --2- 172.21.15.12:0/1422941508 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0fbc078b70 0x7f0fbc078ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:57:37.419 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:37.416+0000 7f0fc3000640 1 --2- 172.21.15.12:0/1422941508 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0fbc077a10 0x7f0fbc077e10 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:57:37.419 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:37.416+0000 7f0fc3000640 1 -- 172.21.15.12:0/1422941508 >> 172.21.15.12:0/1422941508 conn(0x7f0fbc100510 msgr2=0x7f0fbc102950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:57:37.419 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:37.416+0000 7f0fc3000640 1 -- 172.21.15.12:0/1422941508 shutdown_connections 2024-09-10T23:57:37.419 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:37.416+0000 7f0fc3000640 1 -- 172.21.15.12:0/1422941508 wait complete. 2024-09-10T23:57:37.421 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:37.417+0000 7f0fc3000640 1 Processor -- start 2024-09-10T23:57:37.421 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:37.417+0000 7f0fc3000640 1 -- start start 2024-09-10T23:57:37.421 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:37.418+0000 7f0fc3000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0fbc077a10 0x7f0fbc19f410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:57:37.421 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:37.418+0000 7f0fc3000640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0fbc078b70 0x7f0fbc19fa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:57:37.421 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:37.418+0000 7f0fc3000640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0fbc079530 0x7f0fbc1a06a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:57:37.421 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:37.418+0000 7f0fc3000640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f0fbc114d20 con 0x7f0fbc078b70 2024-09-10T23:57:37.421 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:37.418+0000 7f0fc3000640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f0fbc114ba0 con 0x7f0fbc077a10 2024-09-10T23:57:37.421 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:37.418+0000 7f0fc3000640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f0fbc114ea0 con 0x7f0fbc079530 2024-09-10T23:57:37.422 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:37.418+0000 7f0fbbe00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0fbc077a10 0x7f0fbc19f410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:57:37.422 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:37.418+0000 7f0fbb400640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0fbc078b70 0x7f0fbc19fa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:57:37.422 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:37.418+0000 7f0fbbe00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0fbc077a10 0x7f0fbc19f410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.12:34002/0 (socket says 172.21.15.12:34002) 2024-09-10T23:57:37.422 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:37.418+0000 7f0fbbe00640 1 -- 172.21.15.12:0/1705439739 learned_addr learned my addr 172.21.15.12:0/1705439739 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-10T23:57:37.422 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:37.418+0000 7f0fc1000640 1 --2- 172.21.15.12:0/1705439739 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0fbc079530 0x7f0fbc1a06a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:57:37.422 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:37.418+0000 7f0fbbe00640 1 -- 172.21.15.12:0/1705439739 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0fbc079530 msgr2=0x7f0fbc1a06a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:57:37.422 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:37.418+0000 7f0fbbe00640 1 --2- 172.21.15.12:0/1705439739 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0fbc079530 0x7f0fbc1a06a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:57:37.422 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:37.418+0000 7f0fbbe00640 1 -- 172.21.15.12:0/1705439739 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0fbc078b70 msgr2=0x7f0fbc19fa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:57:37.422 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:37.418+0000 7f0fbbe00640 1 --2- 172.21.15.12:0/1705439739 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0fbc078b70 0x7f0fbc19fa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:57:37.422 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:37.419+0000 7f0fbbe00640 1 -- 172.21.15.12:0/1705439739 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0fbc1a0f40 con 0x7f0fbc077a10 2024-09-10T23:57:37.422 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:37.419+0000 7f0fbb400640 1 --2- 172.21.15.12:0/1705439739 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0fbc078b70 0x7f0fbc19fa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:57:37.422 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:37.419+0000 7f0fbbe00640 1 --2- 172.21.15.12:0/1705439739 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0fbc077a10 0x7f0fbc19f410 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f0fac005ab0 tx=0x7f0fac003b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:57:37.422 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:37.419+0000 7f0fb8c00640 1 -- 172.21.15.12:0/1705439739 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fac03f070 con 0x7f0fbc077a10 2024-09-10T23:57:37.423 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:37.419+0000 7f0fc3000640 1 -- 172.21.15.12:0/1705439739 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0fbc1a11d0 con 0x7f0fbc077a10 2024-09-10T23:57:37.423 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:37.419+0000 7f0fb8c00640 1 -- 172.21.15.12:0/1705439739 <== mon.2 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0fac031c50 con 0x7f0fbc077a10 2024-09-10T23:57:37.423 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:37.419+0000 7f0fb8c00640 1 -- 172.21.15.12:0/1705439739 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fac035030 con 0x7f0fbc077a10 2024-09-10T23:57:37.423 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:37.419+0000 7f0fc3000640 1 -- 172.21.15.12:0/1705439739 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0fbc1a7740 con 0x7f0fbc077a10 2024-09-10T23:57:37.423 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:37.420+0000 7f0fc3000640 1 -- 172.21.15.12:0/1705439739 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0fbc111700 con 0x7f0fbc077a10 2024-09-10T23:57:37.424 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:37.423+0000 7f0fb8c00640 1 -- 172.21.15.12:0/1705439739 <== mon.2 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 15) ==== 153194+0+0 (secure 0 0 0) 0x7f0fac03b460 con 0x7f0fbc077a10 2024-09-10T23:57:37.426 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:37.424+0000 7f0fb8c00640 1 --2- 172.21.15.12:0/1705439739 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f0fa00b5250 0x7f0fa00b7710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:57:37.426 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:37.424+0000 7f0fb8c00640 1 -- 172.21.15.12:0/1705439739 <== mon.2 v2:172.21.15.165:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f0fac104c70 con 0x7f0fbc077a10 2024-09-10T23:57:37.426 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:37.424+0000 7f0fbb400640 1 --2- 172.21.15.12:0/1705439739 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f0fa00b5250 0x7f0fa00b7710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:57:37.427 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:37.425+0000 7f0fbb400640 1 --2- 172.21.15.12:0/1705439739 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f0fa00b5250 0x7f0fa00b7710 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f0fa8009fc0 tx=0x7f0fa8005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:57:37.428 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:37.426+0000 7f0fb8c00640 1 -- 172.21.15.12:0/1705439739 <== mon.2 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f0fac0cd110 con 0x7f0fbc077a10 2024-09-10T23:57:37.628 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:37.626+0000 7f0fc3000640 1 -- 172.21.15.12:0/1705439739 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command([{prefix=config set, name=osd_op_complaint_time}] v 0) -- 0x7f0fbc02e5c0 con 0x7f0fbc077a10 2024-09-10T23:57:37.633 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:37.632+0000 7f0fb8c00640 1 -- 172.21.15.12:0/1705439739 <== mon.2 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_op_complaint_time}]=0 v28)=0 v28) ==== 121+0+0 (secure 0 0 0) 0x7f0fac0d1b40 con 0x7f0fbc077a10 2024-09-10T23:57:37.636 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:37.634+0000 7f0fc3000640 1 -- 172.21.15.12:0/1705439739 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f0fa00b5250 msgr2=0x7f0fa00b7710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:57:37.636 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:37.634+0000 7f0fc3000640 1 --2- 172.21.15.12:0/1705439739 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f0fa00b5250 0x7f0fa00b7710 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f0fa8009fc0 tx=0x7f0fa8005f90 comp rx=0 tx=0).stop 2024-09-10T23:57:37.636 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:37.634+0000 7f0fc3000640 1 -- 172.21.15.12:0/1705439739 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0fbc077a10 msgr2=0x7f0fbc19f410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:57:37.636 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:37.634+0000 7f0fc3000640 1 --2- 172.21.15.12:0/1705439739 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0fbc077a10 0x7f0fbc19f410 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f0fac005ab0 tx=0x7f0fac003b90 comp rx=0 tx=0).stop 2024-09-10T23:57:37.637 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:37.634+0000 7f0fc3000640 1 -- 172.21.15.12:0/1705439739 shutdown_connections 2024-09-10T23:57:37.637 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:37.634+0000 7f0fc3000640 1 --2- 172.21.15.12:0/1705439739 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f0fa00b5250 0x7f0fa00b7710 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:57:37.637 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:37.634+0000 7f0fc3000640 1 --2- 172.21.15.12:0/1705439739 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0fbc079530 0x7f0fbc1a06a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:57:37.637 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:37.634+0000 7f0fc3000640 1 --2- 172.21.15.12:0/1705439739 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0fbc078b70 0x7f0fbc19fa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:57:37.637 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:37.634+0000 7f0fc3000640 1 --2- 172.21.15.12:0/1705439739 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0fbc077a10 0x7f0fbc19f410 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:57:37.637 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:37.634+0000 7f0fc3000640 1 -- 172.21.15.12:0/1705439739 >> 172.21.15.12:0/1705439739 conn(0x7f0fbc100510 msgr2=0x7f0fbc1020f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:57:37.637 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:37.634+0000 7f0fc3000640 1 -- 172.21.15.12:0/1705439739 shutdown_connections 2024-09-10T23:57:37.637 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:37.634+0000 7f0fc3000640 1 -- 172.21.15.12:0/1705439739 wait complete. 2024-09-10T23:57:38.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:37 smithi165 ceph-mon[35356]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T23:57:38.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:37 smithi165 ceph-mon[35356]: from='client.? ' entity='client.admin' 2024-09-10T23:57:38.062 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:57:37 smithi012 ceph-mon[30333]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T23:57:38.063 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:57:37 smithi012 ceph-mon[30333]: from='client.? ' entity='client.admin' 2024-09-10T23:57:38.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:57:37 smithi178 ceph-mon[35919]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T23:57:38.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:57:37 smithi178 ceph-mon[35919]: from='client.? ' entity='client.admin' 2024-09-10T23:57:38.660 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T23:57:38.660 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T23:57:39.611 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'ff3ad74a-6fcf-11ef-bcea-c7b262605968', '--', 'ceph', 'config', 'set', 'mds', 'debug_mds', '20'], check_status=True, hostname='smithi012') 2024-09-10T23:57:39.611 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'ff3ad74a-6fcf-11ef-bcea-c7b262605968', '--', 'ceph', 'config', 'set', 'mds', 'debug_mds_balancer', '20'], check_status=True, hostname='smithi012') 2024-09-10T23:57:39.612 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'ff3ad74a-6fcf-11ef-bcea-c7b262605968', '--', 'ceph', 'config', 'set', 'mds', 'debug_ms', '2'], check_status=True, hostname='smithi012') 2024-09-10T23:57:39.812 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:57:39 smithi012 ceph-mon[30333]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T23:57:40.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:39 smithi165 ceph-mon[35356]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T23:57:40.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:57:39 smithi178 ceph-mon[35919]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T23:57:40.677 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'ff3ad74a-6fcf-11ef-bcea-c7b262605968', '--', 'ceph', 'config', 'set', 'mds', 'mds_bal_fragment_size_max', '10000'], check_status=True, hostname='smithi012') 2024-09-10T23:57:40.677 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'ff3ad74a-6fcf-11ef-bcea-c7b262605968', '--', 'ceph', 'config', 'set', 'mds', 'mds_bal_merge_size', '5'], check_status=True, hostname='smithi012') 2024-09-10T23:57:40.678 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'ff3ad74a-6fcf-11ef-bcea-c7b262605968', '--', 'ceph', 'config', 'set', 'mds', 'mds_bal_split_bits', '3'], check_status=True, hostname='smithi012') 2024-09-10T23:57:40.678 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'ff3ad74a-6fcf-11ef-bcea-c7b262605968', '--', 'ceph', 'config', 'set', 'mds', 'mds_bal_split_size', '100'], check_status=True, hostname='smithi012') 2024-09-10T23:57:40.990 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:40.988+0000 7fcb54000640 1 -- 172.21.15.12:0/2532513457 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcb4c106780 msgr2=0x7fcb4c10d100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:57:40.990 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:40.988+0000 7fcb54000640 1 --2- 172.21.15.12:0/2532513457 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcb4c106780 0x7fcb4c10d100 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fcb44009930 tx=0x7fcb4402a420 comp rx=0 tx=0).stop 2024-09-10T23:57:40.991 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:40.988+0000 7fcb54000640 1 -- 172.21.15.12:0/2532513457 shutdown_connections 2024-09-10T23:57:40.991 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:40.988+0000 7fcb54000640 1 --2- 172.21.15.12:0/2532513457 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcb4c106780 0x7fcb4c10d100 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:57:40.991 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:40.988+0000 7fcb54000640 1 --2- 172.21.15.12:0/2532513457 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcb4c105dc0 0x7fcb4c106240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:57:40.991 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:40.988+0000 7fcb54000640 1 --2- 172.21.15.12:0/2532513457 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcb4c104c60 0x7fcb4c105060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:57:40.991 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:40.988+0000 7fcb54000640 1 -- 172.21.15.12:0/2532513457 >> 172.21.15.12:0/2532513457 conn(0x7fcb4c100510 msgr2=0x7fcb4c102970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:57:40.991 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:40.988+0000 7fcb54000640 1 -- 172.21.15.12:0/2532513457 shutdown_connections 2024-09-10T23:57:40.991 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:40.988+0000 7fcb54000640 1 -- 172.21.15.12:0/2532513457 wait complete. 2024-09-10T23:57:40.993 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:40.989+0000 7fcb54000640 1 Processor -- start 2024-09-10T23:57:40.993 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:40.989+0000 7fcb54000640 1 -- start start 2024-09-10T23:57:40.993 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:40.989+0000 7fcb54000640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcb4c104c60 0x7fcb4c19af90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:57:40.993 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:40.989+0000 7fcb54000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcb4c105dc0 0x7fcb4c19b5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:57:40.994 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:40.989+0000 7fcb54000640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcb4c106780 0x7fcb4c19c220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:57:40.994 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:40.989+0000 7fcb54000640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fcb4c110930 con 0x7fcb4c106780 2024-09-10T23:57:40.994 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:40.989+0000 7fcb54000640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fcb4c1107b0 con 0x7fcb4c105dc0 2024-09-10T23:57:40.994 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:40.989+0000 7fcb54000640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fcb4c110ab0 con 0x7fcb4c104c60 2024-09-10T23:57:40.994 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:40.990+0000 7fcb51600640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcb4c104c60 0x7fcb4c19af90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:57:40.994 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:40.990+0000 7fcb51600640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcb4c104c60 0x7fcb4c19af90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.12:49530/0 (socket says 172.21.15.12:49530) 2024-09-10T23:57:40.994 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:40.990+0000 7fcb52000640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcb4c106780 0x7fcb4c19c220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:57:40.994 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:40.990+0000 7fcb51600640 1 -- 172.21.15.12:0/3356096026 learned_addr learned my addr 172.21.15.12:0/3356096026 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-10T23:57:40.994 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:40.990+0000 7fcb50c00640 1 --2- 172.21.15.12:0/3356096026 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcb4c105dc0 0x7fcb4c19b5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:57:40.994 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:40.990+0000 7fcb51600640 1 -- 172.21.15.12:0/3356096026 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcb4c105dc0 msgr2=0x7fcb4c19b5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:57:40.995 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:40.990+0000 7fcb51600640 1 --2- 172.21.15.12:0/3356096026 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcb4c105dc0 0x7fcb4c19b5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:57:40.995 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:40.990+0000 7fcb51600640 1 -- 172.21.15.12:0/3356096026 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcb4c106780 msgr2=0x7fcb4c19c220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:57:40.995 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:40.990+0000 7fcb51600640 1 --2- 172.21.15.12:0/3356096026 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcb4c106780 0x7fcb4c19c220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:57:40.995 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:40.990+0000 7fcb51600640 1 -- 172.21.15.12:0/3356096026 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb4c19cac0 con 0x7fcb4c104c60 2024-09-10T23:57:40.995 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:40.990+0000 7fcb51600640 1 --2- 172.21.15.12:0/3356096026 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcb4c104c60 0x7fcb4c19af90 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fcb4000ebc0 tx=0x7fcb4000c6a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:57:40.995 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:40.991+0000 7fcb4a000640 1 -- 172.21.15.12:0/3356096026 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb40010630 con 0x7fcb4c104c60 2024-09-10T23:57:40.995 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:40.991+0000 7fcb4a000640 1 -- 172.21.15.12:0/3356096026 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcb40010c80 con 0x7fcb4c104c60 2024-09-10T23:57:40.995 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:40.991+0000 7fcb4a000640 1 -- 172.21.15.12:0/3356096026 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb40018b60 con 0x7fcb4c104c60 2024-09-10T23:57:40.995 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:40.991+0000 7fcb54000640 1 -- 172.21.15.12:0/3356096026 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcb4c19cdb0 con 0x7fcb4c104c60 2024-09-10T23:57:40.995 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:40.991+0000 7fcb54000640 1 -- 172.21.15.12:0/3356096026 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcb4c1a3490 con 0x7fcb4c104c60 2024-09-10T23:57:40.996 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:40.992+0000 7fcb54000640 1 -- 172.21.15.12:0/3356096026 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcb4c10bf80 con 0x7fcb4c104c60 2024-09-10T23:57:40.996 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:40.994+0000 7fcb4a000640 1 -- 172.21.15.12:0/3356096026 <== mon.1 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 15) ==== 153194+0+0 (secure 0 0 0) 0x7fcb400107d0 con 0x7fcb4c104c60 2024-09-10T23:57:40.998 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:40.996+0000 7fcb4a000640 1 --2- 172.21.15.12:0/3356096026 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fcb340b52d0 0x7fcb340b7790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:57:40.998 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:40.996+0000 7fcb4a000640 1 -- 172.21.15.12:0/3356096026 <== mon.1 v2:172.21.15.178:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fcb400e4500 con 0x7fcb4c104c60 2024-09-10T23:57:40.998 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:40.996+0000 7fcb50c00640 1 --2- 172.21.15.12:0/3356096026 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fcb340b52d0 0x7fcb340b7790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:57:40.998 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:40.997+0000 7fcb50c00640 1 --2- 172.21.15.12:0/3356096026 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fcb340b52d0 0x7fcb340b7790 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fcb38009bd0 tx=0x7fcb38005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:57:40.999 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:40.998+0000 7fcb4a000640 1 -- 172.21.15.12:0/3356096026 <== mon.1 v2:172.21.15.178:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7fcb400ac9a0 con 0x7fcb4c104c60 2024-09-10T23:57:41.199 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:41.197+0000 7fcb54000640 1 -- 172.21.15.12:0/3356096026 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command([{prefix=config set, name=mds_debug_scatterstat}] v 0) -- 0x7fcb4c02e5c0 con 0x7fcb4c104c60 2024-09-10T23:57:41.204 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:41.202+0000 7fcb4a000640 1 -- 172.21.15.12:0/3356096026 <== mon.1 v2:172.21.15.178:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_debug_scatterstat}]=0 v29)=0 v29) ==== 122+0+0 (secure 0 0 0) 0x7fcb400b13d0 con 0x7fcb4c104c60 2024-09-10T23:57:41.207 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:41.205+0000 7fcb54000640 1 -- 172.21.15.12:0/3356096026 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fcb340b52d0 msgr2=0x7fcb340b7790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:57:41.207 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:41.205+0000 7fcb54000640 1 --2- 172.21.15.12:0/3356096026 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fcb340b52d0 0x7fcb340b7790 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fcb38009bd0 tx=0x7fcb38005f90 comp rx=0 tx=0).stop 2024-09-10T23:57:41.207 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:41.205+0000 7fcb54000640 1 -- 172.21.15.12:0/3356096026 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcb4c104c60 msgr2=0x7fcb4c19af90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:57:41.207 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:41.205+0000 7fcb54000640 1 --2- 172.21.15.12:0/3356096026 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcb4c104c60 0x7fcb4c19af90 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fcb4000ebc0 tx=0x7fcb4000c6a0 comp rx=0 tx=0).stop 2024-09-10T23:57:41.208 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:41.205+0000 7fcb54000640 1 -- 172.21.15.12:0/3356096026 shutdown_connections 2024-09-10T23:57:41.208 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:41.205+0000 7fcb54000640 1 --2- 172.21.15.12:0/3356096026 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fcb340b52d0 0x7fcb340b7790 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:57:41.208 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:41.205+0000 7fcb54000640 1 --2- 172.21.15.12:0/3356096026 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcb4c106780 0x7fcb4c19c220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:57:41.208 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:41.205+0000 7fcb54000640 1 --2- 172.21.15.12:0/3356096026 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcb4c105dc0 0x7fcb4c19b5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:57:41.208 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:41.205+0000 7fcb54000640 1 --2- 172.21.15.12:0/3356096026 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcb4c104c60 0x7fcb4c19af90 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:57:41.208 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:41.205+0000 7fcb54000640 1 -- 172.21.15.12:0/3356096026 >> 172.21.15.12:0/3356096026 conn(0x7fcb4c100510 msgr2=0x7fcb4c109150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:57:41.208 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:41.206+0000 7fcb54000640 1 -- 172.21.15.12:0/3356096026 shutdown_connections 2024-09-10T23:57:41.208 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:41.206+0000 7fcb54000640 1 -- 172.21.15.12:0/3356096026 wait complete. 2024-09-10T23:57:41.361 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:41.236+0000 7f924ee00640 1 -- 172.21.15.12:0/1700226017 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9248076f80 msgr2=0x7f92480755e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:57:41.361 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:41.236+0000 7f924ee00640 1 --2- 172.21.15.12:0/1700226017 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9248076f80 0x7f92480755e0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f9238009900 tx=0x7f923802a410 comp rx=0 tx=0).stop 2024-09-10T23:57:41.361 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:41.236+0000 7f924ee00640 1 -- 172.21.15.12:0/1700226017 shutdown_connections 2024-09-10T23:57:41.361 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:41.236+0000 7f924ee00640 1 --2- 172.21.15.12:0/1700226017 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f924810a960 0x7f924810ce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:57:41.361 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:41.236+0000 7f924ee00640 1 --2- 172.21.15.12:0/1700226017 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9248075b20 0x7f9248075fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:57:41.361 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:41.236+0000 7f924ee00640 1 --2- 172.21.15.12:0/1700226017 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9248076f80 0x7f92480755e0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:57:41.361 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:41.236+0000 7f924ee00640 1 -- 172.21.15.12:0/1700226017 >> 172.21.15.12:0/1700226017 conn(0x7f92480fe320 msgr2=0x7f9248100740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:57:41.361 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:41.236+0000 7f924ee00640 1 -- 172.21.15.12:0/1700226017 shutdown_connections 2024-09-10T23:57:41.361 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:41.236+0000 7f924ee00640 1 -- 172.21.15.12:0/1700226017 wait complete. 2024-09-10T23:57:41.362 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:41.236+0000 7f924ee00640 1 Processor -- start 2024-09-10T23:57:41.362 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:41.237+0000 7f924ee00640 1 -- start start 2024-09-10T23:57:41.362 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:41.237+0000 7f924ee00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9248075b20 0x7f924819aff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:57:41.362 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:41.237+0000 7f924ee00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9248076f80 0x7f924819b640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:57:41.362 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:41.237+0000 7f924ee00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f924810a960 0x7f924819c280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:57:41.362 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:41.237+0000 7f924ee00640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f92481108c0 con 0x7f9248076f80 2024-09-10T23:57:41.362 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:41.237+0000 7f924ee00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f9248110740 con 0x7f924810a960 2024-09-10T23:57:41.362 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:41.237+0000 7f924ee00640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f9248110a40 con 0x7f9248075b20 2024-09-10T23:57:41.362 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:41.237+0000 7f9247400640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9248076f80 0x7f924819b640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:57:41.362 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:41.237+0000 7f9247400640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9248076f80 0x7f924819b640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:44354/0 (socket says 172.21.15.12:44354) 2024-09-10T23:57:41.362 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:41.237+0000 7f9247400640 1 -- 172.21.15.12:0/4012470975 learned_addr learned my addr 172.21.15.12:0/4012470975 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-10T23:57:41.362 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:41.237+0000 7f924ce00640 1 --2- 172.21.15.12:0/4012470975 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f924810a960 0x7f924819c280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:57:41.362 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:41.237+0000 7f9247e00640 1 --2- 172.21.15.12:0/4012470975 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9248075b20 0x7f924819aff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:57:41.363 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:41.238+0000 7f9247400640 1 -- 172.21.15.12:0/4012470975 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9248075b20 msgr2=0x7f924819aff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:57:41.363 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:41.238+0000 7f9247400640 1 --2- 172.21.15.12:0/4012470975 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9248075b20 0x7f924819aff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:57:41.363 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:41.238+0000 7f9247400640 1 -- 172.21.15.12:0/4012470975 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f924810a960 msgr2=0x7f924819c280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:57:41.363 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:41.238+0000 7f9247400640 1 --2- 172.21.15.12:0/4012470975 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f924810a960 0x7f924819c280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:57:41.363 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:41.238+0000 7f9247400640 1 -- 172.21.15.12:0/4012470975 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f924819cb20 con 0x7f9248076f80 2024-09-10T23:57:41.363 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:41.238+0000 7f924ce00640 1 --2- 172.21.15.12:0/4012470975 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f924810a960 0x7f924819c280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:57:41.363 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:41.238+0000 7f9247e00640 1 --2- 172.21.15.12:0/4012470975 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9248075b20 0x7f924819aff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T23:57:41.363 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:41.238+0000 7f9247400640 1 --2- 172.21.15.12:0/4012470975 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9248076f80 0x7f924819b640 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f923400d950 tx=0x7f923400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:57:41.363 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:41.239+0000 7f9244c00640 1 -- 172.21.15.12:0/4012470975 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9234017030 con 0x7f9248076f80 2024-09-10T23:57:41.363 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:41.239+0000 7f924ee00640 1 -- 172.21.15.12:0/4012470975 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f924819ce10 con 0x7f9248076f80 2024-09-10T23:57:41.363 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:41.239+0000 7f924ee00640 1 -- 172.21.15.12:0/4012470975 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92481a34f0 con 0x7f9248076f80 2024-09-10T23:57:41.363 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:41.239+0000 7f9244c00640 1 -- 172.21.15.12:0/4012470975 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9234004020 con 0x7f9248076f80 2024-09-10T23:57:41.363 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:41.239+0000 7f9244c00640 1 -- 172.21.15.12:0/4012470975 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9234005650 con 0x7f9248076f80 2024-09-10T23:57:41.364 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:41.242+0000 7f924ee00640 1 -- 172.21.15.12:0/4012470975 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f92280052c0 con 0x7f9248076f80 2024-09-10T23:57:41.364 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:41.244+0000 7f9244c00640 1 -- 172.21.15.12:0/4012470975 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 15) ==== 153194+0+0 (secure 0 0 0) 0x7f92340041d0 con 0x7f9248076f80 2024-09-10T23:57:41.364 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:41.245+0000 7f9244c00640 1 --2- 172.21.15.12:0/4012470975 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f922c0b52d0 0x7f922c0b7790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:57:41.364 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:41.245+0000 7f9244c00640 1 -- 172.21.15.12:0/4012470975 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f92340e37e0 con 0x7f9248076f80 2024-09-10T23:57:41.364 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:41.245+0000 7f9247e00640 1 --2- 172.21.15.12:0/4012470975 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f922c0b52d0 0x7f922c0b7790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:57:41.364 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:41.246+0000 7f9247e00640 1 --2- 172.21.15.12:0/4012470975 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f922c0b52d0 0x7f922c0b7790 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f9238031500 tx=0x7f9238005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:57:41.364 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:41.249+0000 7f9244c00640 1 -- 172.21.15.12:0/4012470975 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f92340ac2d0 con 0x7f9248076f80 2024-09-10T23:57:41.461 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:41.365+0000 7f47f1400640 1 -- 172.21.15.12:0/2088488560 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f47ec105da0 msgr2=0x7f47ec106220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:57:41.462 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:41.365+0000 7f47f1400640 1 --2- 172.21.15.12:0/2088488560 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f47ec105da0 0x7f47ec106220 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f47d8009930 tx=0x7f47d802a420 comp rx=0 tx=0).stop 2024-09-10T23:57:41.462 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:41.365+0000 7f47f1400640 1 -- 172.21.15.12:0/2088488560 shutdown_connections 2024-09-10T23:57:41.462 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:41.365+0000 7f47f1400640 1 --2- 172.21.15.12:0/2088488560 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f47ec106760 0x7f47ec10d0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:57:41.462 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:41.365+0000 7f47f1400640 1 --2- 172.21.15.12:0/2088488560 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f47ec105da0 0x7f47ec106220 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:57:41.462 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:41.365+0000 7f47f1400640 1 --2- 172.21.15.12:0/2088488560 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f47ec104c40 0x7f47ec105040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:57:41.462 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:41.365+0000 7f47f1400640 1 -- 172.21.15.12:0/2088488560 >> 172.21.15.12:0/2088488560 conn(0x7f47ec100510 msgr2=0x7f47ec102950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:57:41.462 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:41.365+0000 7f47f1400640 1 -- 172.21.15.12:0/2088488560 shutdown_connections 2024-09-10T23:57:41.462 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:41.365+0000 7f47f1400640 1 -- 172.21.15.12:0/2088488560 wait complete. 2024-09-10T23:57:41.462 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:41.367+0000 7f47f1400640 1 Processor -- start 2024-09-10T23:57:41.462 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:41.367+0000 7f47f1400640 1 -- start start 2024-09-10T23:57:41.462 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:41.367+0000 7f47f1400640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f47ec104c40 0x7f47ec19b060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:57:41.462 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:41.367+0000 7f47f1400640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f47ec105da0 0x7f47ec19b6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:57:41.462 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:41.367+0000 7f47f1400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f47ec106760 0x7f47ec19c2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:57:41.462 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:41.367+0000 7f47f1400640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f47ec110910 con 0x7f47ec105da0 2024-09-10T23:57:41.463 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:41.367+0000 7f47f1400640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f47ec110790 con 0x7f47ec106760 2024-09-10T23:57:41.463 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:41.367+0000 7f47f1400640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f47ec110a90 con 0x7f47ec104c40 2024-09-10T23:57:41.463 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:41.368+0000 7f47e9e00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f47ec105da0 0x7f47ec19b6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:57:41.463 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:41.368+0000 7f47e9e00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f47ec105da0 0x7f47ec19b6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:44372/0 (socket says 172.21.15.12:44372) 2024-09-10T23:57:41.463 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:41.368+0000 7f47e9e00640 1 -- 172.21.15.12:0/3192472517 learned_addr learned my addr 172.21.15.12:0/3192472517 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-10T23:57:41.463 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:41.368+0000 7f47ea800640 1 --2- 172.21.15.12:0/3192472517 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f47ec104c40 0x7f47ec19b060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:57:41.463 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:41.368+0000 7f47eb200640 1 --2- 172.21.15.12:0/3192472517 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f47ec106760 0x7f47ec19c2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:57:41.463 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:41.368+0000 7f47e9e00640 1 -- 172.21.15.12:0/3192472517 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f47ec104c40 msgr2=0x7f47ec19b060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:57:41.463 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:41.368+0000 7f47e9e00640 1 --2- 172.21.15.12:0/3192472517 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f47ec104c40 0x7f47ec19b060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:57:41.463 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:41.368+0000 7f47e9e00640 1 -- 172.21.15.12:0/3192472517 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f47ec106760 msgr2=0x7f47ec19c2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:57:41.463 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:41.368+0000 7f47e9e00640 1 --2- 172.21.15.12:0/3192472517 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f47ec106760 0x7f47ec19c2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:57:41.463 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:41.368+0000 7f47e9e00640 1 -- 172.21.15.12:0/3192472517 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47ec19cb90 con 0x7f47ec105da0 2024-09-10T23:57:41.463 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:41.368+0000 7f47e9e00640 1 --2- 172.21.15.12:0/3192472517 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f47ec105da0 0x7f47ec19b6b0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f47d8031490 tx=0x7f47d8003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:57:41.463 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:41.369+0000 7f47e3400640 1 -- 172.21.15.12:0/3192472517 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47d803f070 con 0x7f47ec105da0 2024-09-10T23:57:41.463 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:41.369+0000 7f47e3400640 1 -- 172.21.15.12:0/3192472517 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f47d8031bf0 con 0x7f47ec105da0 2024-09-10T23:57:41.464 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:41.369+0000 7f47f1400640 1 -- 172.21.15.12:0/3192472517 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f47ec19ce20 con 0x7f47ec105da0 2024-09-10T23:57:41.464 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:41.369+0000 7f47e3400640 1 -- 172.21.15.12:0/3192472517 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47d8035030 con 0x7f47ec105da0 2024-09-10T23:57:41.464 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:41.369+0000 7f47f1400640 1 -- 172.21.15.12:0/3192472517 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47ec1a3390 con 0x7f47ec105da0 2024-09-10T23:57:41.464 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:41.370+0000 7f47f1400640 1 -- 172.21.15.12:0/3192472517 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f47ec10bfc0 con 0x7f47ec105da0 2024-09-10T23:57:41.464 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:41.374+0000 7f47e3400640 1 -- 172.21.15.12:0/3192472517 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 15) ==== 153194+0+0 (secure 0 0 0) 0x7f47d8031740 con 0x7f47ec105da0 2024-09-10T23:57:41.464 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:41.375+0000 7f47e3400640 1 --2- 172.21.15.12:0/3192472517 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f47d00b5250 0x7f47d00b7710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:57:41.464 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:41.375+0000 7f47e3400640 1 -- 172.21.15.12:0/3192472517 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f47d81048f0 con 0x7f47ec105da0 2024-09-10T23:57:41.464 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:41.375+0000 7f47ea800640 1 --2- 172.21.15.12:0/3192472517 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f47d00b5250 0x7f47d00b7710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:57:41.464 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:41.376+0000 7f47ea800640 1 --2- 172.21.15.12:0/3192472517 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f47d00b5250 0x7f47d00b7710 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f47dc009bd0 tx=0x7f47dc005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:57:41.464 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:41.379+0000 7f47e3400640 1 -- 172.21.15.12:0/3192472517 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f47d80ccd90 con 0x7f47ec105da0 2024-09-10T23:57:41.476 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:41.475+0000 7f924ee00640 1 -- 172.21.15.12:0/4012470975 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command([{prefix=config set, name=osd_op_complaint_time}] v 0) -- 0x7f92280055b0 con 0x7f9248076f80 2024-09-10T23:57:41.481 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:41.480+0000 7f9244c00640 1 -- 172.21.15.12:0/4012470975 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_op_complaint_time}]=0 v30)=0 v30) ==== 121+0+0 (secure 0 0 0) 0x7f92340abbd0 con 0x7f9248076f80 2024-09-10T23:57:41.484 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:41.482+0000 7f924ee00640 1 -- 172.21.15.12:0/4012470975 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f922c0b52d0 msgr2=0x7f922c0b7790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:57:41.484 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:41.482+0000 7f924ee00640 1 --2- 172.21.15.12:0/4012470975 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f922c0b52d0 0x7f922c0b7790 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f9238031500 tx=0x7f9238005a20 comp rx=0 tx=0).stop 2024-09-10T23:57:41.484 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:41.483+0000 7f924ee00640 1 -- 172.21.15.12:0/4012470975 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9248076f80 msgr2=0x7f924819b640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:57:41.484 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:41.483+0000 7f924ee00640 1 --2- 172.21.15.12:0/4012470975 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9248076f80 0x7f924819b640 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f923400d950 tx=0x7f923400de10 comp rx=0 tx=0).stop 2024-09-10T23:57:41.484 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:41.483+0000 7f924ee00640 1 -- 172.21.15.12:0/4012470975 shutdown_connections 2024-09-10T23:57:41.484 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:41.483+0000 7f924ee00640 1 --2- 172.21.15.12:0/4012470975 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f922c0b52d0 0x7f922c0b7790 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:57:41.484 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:41.483+0000 7f924ee00640 1 --2- 172.21.15.12:0/4012470975 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f924810a960 0x7f924819c280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:57:41.484 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:41.483+0000 7f924ee00640 1 --2- 172.21.15.12:0/4012470975 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9248076f80 0x7f924819b640 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:57:41.484 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:41.483+0000 7f924ee00640 1 --2- 172.21.15.12:0/4012470975 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9248075b20 0x7f924819aff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:57:41.484 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:41.483+0000 7f924ee00640 1 -- 172.21.15.12:0/4012470975 >> 172.21.15.12:0/4012470975 conn(0x7f92480fe320 msgr2=0x7f92480fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:57:41.484 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:41.483+0000 7f924ee00640 1 -- 172.21.15.12:0/4012470975 shutdown_connections 2024-09-10T23:57:41.485 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:41.483+0000 7f924ee00640 1 -- 172.21.15.12:0/4012470975 wait complete. 2024-09-10T23:57:41.586 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:41.585+0000 7f47f1400640 1 -- 172.21.15.12:0/3192472517 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command([{prefix=config set, name=mds_bal_split_size}] v 0) -- 0x7f47ec063d50 con 0x7f47ec105da0 2024-09-10T23:57:41.590 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:41.589+0000 7f47e3400640 1 -- 172.21.15.12:0/3192472517 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_bal_split_size}]=0 v31)=0 v31) ==== 118+0+0 (secure 0 0 0) 0x7f47d80d17c0 con 0x7f47ec105da0 2024-09-10T23:57:41.593 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:41.592+0000 7f47f1400640 1 -- 172.21.15.12:0/3192472517 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f47d00b5250 msgr2=0x7f47d00b7710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:57:41.593 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:41.592+0000 7f47f1400640 1 --2- 172.21.15.12:0/3192472517 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f47d00b5250 0x7f47d00b7710 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f47dc009bd0 tx=0x7f47dc005f90 comp rx=0 tx=0).stop 2024-09-10T23:57:41.594 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:41.592+0000 7f47f1400640 1 -- 172.21.15.12:0/3192472517 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f47ec105da0 msgr2=0x7f47ec19b6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:57:41.594 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:41.592+0000 7f47f1400640 1 --2- 172.21.15.12:0/3192472517 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f47ec105da0 0x7f47ec19b6b0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f47d8031490 tx=0x7f47d8003c10 comp rx=0 tx=0).stop 2024-09-10T23:57:41.594 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:41.592+0000 7f47f1400640 1 -- 172.21.15.12:0/3192472517 shutdown_connections 2024-09-10T23:57:41.594 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:41.592+0000 7f47f1400640 1 --2- 172.21.15.12:0/3192472517 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f47d00b5250 0x7f47d00b7710 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:57:41.594 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:41.592+0000 7f47f1400640 1 --2- 172.21.15.12:0/3192472517 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f47ec106760 0x7f47ec19c2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:57:41.594 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:41.592+0000 7f47f1400640 1 --2- 172.21.15.12:0/3192472517 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f47ec105da0 0x7f47ec19b6b0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:57:41.594 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:41.592+0000 7f47f1400640 1 --2- 172.21.15.12:0/3192472517 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f47ec104c40 0x7f47ec19b060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:57:41.594 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:41.592+0000 7f47f1400640 1 -- 172.21.15.12:0/3192472517 >> 172.21.15.12:0/3192472517 conn(0x7f47ec100510 msgr2=0x7f47ec109190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:57:41.594 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:41.593+0000 7f47f1400640 1 -- 172.21.15.12:0/3192472517 shutdown_connections 2024-09-10T23:57:41.594 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:41.593+0000 7f47f1400640 1 -- 172.21.15.12:0/3192472517 wait complete. 2024-09-10T23:57:41.806 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:57:41 smithi012 ceph-mon[30333]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T23:57:41.806 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:57:41 smithi012 ceph-mon[30333]: from='client.? ' entity='client.admin' 2024-09-10T23:57:41.806 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:57:41 smithi012 ceph-mon[30333]: from='client.? 172.21.15.12:0/4012470975' entity='client.admin' 2024-09-10T23:57:41.807 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:57:41 smithi012 ceph-mon[30333]: from='client.? 172.21.15.12:0/3192472517' entity='client.admin' 2024-09-10T23:57:42.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:41 smithi165 ceph-mon[35356]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T23:57:42.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:41 smithi165 ceph-mon[35356]: from='client.? ' entity='client.admin' 2024-09-10T23:57:42.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:41 smithi165 ceph-mon[35356]: from='client.? 172.21.15.12:0/4012470975' entity='client.admin' 2024-09-10T23:57:42.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:41 smithi165 ceph-mon[35356]: from='client.? 172.21.15.12:0/3192472517' entity='client.admin' 2024-09-10T23:57:42.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:57:41 smithi178 ceph-mon[35919]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T23:57:42.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:57:41 smithi178 ceph-mon[35919]: from='client.? ' entity='client.admin' 2024-09-10T23:57:42.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:57:41 smithi178 ceph-mon[35919]: from='client.? 172.21.15.12:0/4012470975' entity='client.admin' 2024-09-10T23:57:42.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:57:41 smithi178 ceph-mon[35919]: from='client.? 172.21.15.12:0/3192472517' entity='client.admin' 2024-09-10T23:57:43.329 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'ff3ad74a-6fcf-11ef-bcea-c7b262605968', '--', 'ceph', 'config', 'set', 'mds', 'mds_debug_frag', 'True'], check_status=True, hostname='smithi012') 2024-09-10T23:57:43.329 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'ff3ad74a-6fcf-11ef-bcea-c7b262605968', '--', 'ceph', 'config', 'set', 'mds', 'mds_debug_scatterstat', 'True'], check_status=True, hostname='smithi012') 2024-09-10T23:57:43.329 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'ff3ad74a-6fcf-11ef-bcea-c7b262605968', '--', 'ceph', 'config', 'set', 'mds', 'mds_op_complaint_time', '180'], check_status=True, hostname='smithi012') 2024-09-10T23:57:43.329 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'ff3ad74a-6fcf-11ef-bcea-c7b262605968', '--', 'ceph', 'config', 'set', 'mds', 'mds_verify_scatter', 'True'], check_status=True, hostname='smithi012') 2024-09-10T23:57:43.329 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'ff3ad74a-6fcf-11ef-bcea-c7b262605968', '--', 'ceph', 'config', 'set', 'mds', 'osd_op_complaint_time', '180'], check_status=True, hostname='smithi012') 2024-09-10T23:57:43.329 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'ff3ad74a-6fcf-11ef-bcea-c7b262605968', '--', 'ceph', 'config', 'set', 'mds', 'rados_mon_op_timeout', '900'], check_status=True, hostname='smithi012') 2024-09-10T23:57:43.329 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'ff3ad74a-6fcf-11ef-bcea-c7b262605968', '--', 'ceph', 'config', 'set', 'mds', 'rados_osd_op_timeout', '900'], check_status=True, hostname='smithi012') 2024-09-10T23:57:43.330 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'ff3ad74a-6fcf-11ef-bcea-c7b262605968', '--', 'ceph', 'config', 'set', 'mon', 'mon_op_complaint_time', '120'], check_status=True, hostname='smithi012') 2024-09-10T23:57:43.330 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'ff3ad74a-6fcf-11ef-bcea-c7b262605968', '--', 'ceph', 'config', 'set', 'mon', 'mon_warn_on_pool_no_app', 'False'], check_status=True, hostname='smithi012') 2024-09-10T23:57:43.330 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'ff3ad74a-6fcf-11ef-bcea-c7b262605968', '--', 'ceph', 'config', 'set', 'osd', 'osd_op_complaint_time', '180'], check_status=True, hostname='smithi012') 2024-09-10T23:57:43.330 INFO:tasks.cephadm:Deploying OSDs... 2024-09-10T23:57:43.330 DEBUG:teuthology.orchestra.run.smithi012:> set -ex 2024-09-10T23:57:43.330 DEBUG:teuthology.orchestra.run.smithi012:> dd if=/scratch_devs of=/dev/stdout 2024-09-10T23:57:43.367 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-10T23:57:43.367 DEBUG:teuthology.orchestra.run.smithi012:> stat /dev/vg_nvme/lv_1 2024-09-10T23:57:43.443 INFO:teuthology.orchestra.run.smithi012.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-10T23:57:43.443 INFO:teuthology.orchestra.run.smithi012.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-10T23:57:43.443 INFO:teuthology.orchestra.run.smithi012.stdout:Device: 6h/6d Inode: 634 Links: 1 2024-09-10T23:57:43.443 INFO:teuthology.orchestra.run.smithi012.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-10T23:57:43.443 INFO:teuthology.orchestra.run.smithi012.stdout:Context: system_u:object_r:device_t:s0 2024-09-10T23:57:43.443 INFO:teuthology.orchestra.run.smithi012.stdout:Access: 2024-09-10 23:56:34.587983069 +0000 2024-09-10T23:57:43.444 INFO:teuthology.orchestra.run.smithi012.stdout:Modify: 2024-09-10 23:51:52.056558878 +0000 2024-09-10T23:57:43.444 INFO:teuthology.orchestra.run.smithi012.stdout:Change: 2024-09-10 23:51:52.056558878 +0000 2024-09-10T23:57:43.444 INFO:teuthology.orchestra.run.smithi012.stdout: Birth: 2024-09-10 23:51:52.056558878 +0000 2024-09-10T23:57:43.444 DEBUG:teuthology.orchestra.run.smithi012:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-10T23:57:43.538 INFO:teuthology.orchestra.run.smithi012.stderr:1+0 records in 2024-09-10T23:57:43.538 INFO:teuthology.orchestra.run.smithi012.stderr:1+0 records out 2024-09-10T23:57:43.538 INFO:teuthology.orchestra.run.smithi012.stderr:512 bytes copied, 0.00033901 s, 1.5 MB/s 2024-09-10T23:57:43.541 DEBUG:teuthology.orchestra.run.smithi012:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-10T23:57:43.577 DEBUG:teuthology.orchestra.run.smithi012:> stat /dev/vg_nvme/lv_2 2024-09-10T23:57:43.652 INFO:teuthology.orchestra.run.smithi012.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-10T23:57:43.653 INFO:teuthology.orchestra.run.smithi012.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-10T23:57:43.653 INFO:teuthology.orchestra.run.smithi012.stdout:Device: 6h/6d Inode: 653 Links: 1 2024-09-10T23:57:43.653 INFO:teuthology.orchestra.run.smithi012.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-10T23:57:43.653 INFO:teuthology.orchestra.run.smithi012.stdout:Context: system_u:object_r:device_t:s0 2024-09-10T23:57:43.653 INFO:teuthology.orchestra.run.smithi012.stdout:Access: 2024-09-10 23:56:34.628982726 +0000 2024-09-10T23:57:43.653 INFO:teuthology.orchestra.run.smithi012.stdout:Modify: 2024-09-10 23:51:52.503555068 +0000 2024-09-10T23:57:43.653 INFO:teuthology.orchestra.run.smithi012.stdout:Change: 2024-09-10 23:51:52.503555068 +0000 2024-09-10T23:57:43.653 INFO:teuthology.orchestra.run.smithi012.stdout: Birth: 2024-09-10 23:51:52.503555068 +0000 2024-09-10T23:57:43.653 DEBUG:teuthology.orchestra.run.smithi012:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-10T23:57:43.746 INFO:teuthology.orchestra.run.smithi012.stderr:1+0 records in 2024-09-10T23:57:43.746 INFO:teuthology.orchestra.run.smithi012.stderr:1+0 records out 2024-09-10T23:57:43.746 INFO:teuthology.orchestra.run.smithi012.stderr:512 bytes copied, 0.000307018 s, 1.7 MB/s 2024-09-10T23:57:43.748 DEBUG:teuthology.orchestra.run.smithi012:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-10T23:57:43.785 DEBUG:teuthology.orchestra.run.smithi012:> stat /dev/vg_nvme/lv_3 2024-09-10T23:57:43.813 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:57:43 smithi012 ceph-mon[30333]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T23:57:43.847 INFO:teuthology.orchestra.run.smithi012.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-10T23:57:43.847 INFO:teuthology.orchestra.run.smithi012.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-10T23:57:43.847 INFO:teuthology.orchestra.run.smithi012.stdout:Device: 6h/6d Inode: 672 Links: 1 2024-09-10T23:57:43.847 INFO:teuthology.orchestra.run.smithi012.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-10T23:57:43.847 INFO:teuthology.orchestra.run.smithi012.stdout:Context: system_u:object_r:device_t:s0 2024-09-10T23:57:43.847 INFO:teuthology.orchestra.run.smithi012.stdout:Access: 2024-09-10 23:56:34.669982382 +0000 2024-09-10T23:57:43.848 INFO:teuthology.orchestra.run.smithi012.stdout:Modify: 2024-09-10 23:51:52.972551071 +0000 2024-09-10T23:57:43.848 INFO:teuthology.orchestra.run.smithi012.stdout:Change: 2024-09-10 23:51:52.972551071 +0000 2024-09-10T23:57:43.848 INFO:teuthology.orchestra.run.smithi012.stdout: Birth: 2024-09-10 23:51:52.972551071 +0000 2024-09-10T23:57:43.848 DEBUG:teuthology.orchestra.run.smithi012:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-10T23:57:43.940 INFO:teuthology.orchestra.run.smithi012.stderr:1+0 records in 2024-09-10T23:57:43.941 INFO:teuthology.orchestra.run.smithi012.stderr:1+0 records out 2024-09-10T23:57:43.941 INFO:teuthology.orchestra.run.smithi012.stderr:512 bytes copied, 0.000263848 s, 1.9 MB/s 2024-09-10T23:57:43.943 DEBUG:teuthology.orchestra.run.smithi012:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-10T23:57:43.980 DEBUG:teuthology.orchestra.run.smithi012:> stat /dev/vg_nvme/lv_4 2024-09-10T23:57:44.033 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:43 smithi165 ceph-mon[35356]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T23:57:44.068 INFO:teuthology.orchestra.run.smithi012.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-10T23:57:44.068 INFO:teuthology.orchestra.run.smithi012.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-10T23:57:44.068 INFO:teuthology.orchestra.run.smithi012.stdout:Device: 6h/6d Inode: 688 Links: 1 2024-09-10T23:57:44.068 INFO:teuthology.orchestra.run.smithi012.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-10T23:57:44.068 INFO:teuthology.orchestra.run.smithi012.stdout:Context: system_u:object_r:device_t:s0 2024-09-10T23:57:44.069 INFO:teuthology.orchestra.run.smithi012.stdout:Access: 2024-09-10 23:56:34.709982047 +0000 2024-09-10T23:57:44.069 INFO:teuthology.orchestra.run.smithi012.stdout:Modify: 2024-09-10 23:51:53.436547117 +0000 2024-09-10T23:57:44.069 INFO:teuthology.orchestra.run.smithi012.stdout:Change: 2024-09-10 23:51:53.436547117 +0000 2024-09-10T23:57:44.069 INFO:teuthology.orchestra.run.smithi012.stdout: Birth: 2024-09-10 23:51:53.436547117 +0000 2024-09-10T23:57:44.069 DEBUG:teuthology.orchestra.run.smithi012:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-10T23:57:44.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:57:43 smithi178 ceph-mon[35919]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T23:57:44.167 INFO:teuthology.orchestra.run.smithi012.stderr:1+0 records in 2024-09-10T23:57:44.168 INFO:teuthology.orchestra.run.smithi012.stderr:1+0 records out 2024-09-10T23:57:44.168 INFO:teuthology.orchestra.run.smithi012.stderr:512 bytes copied, 0.000265324 s, 1.9 MB/s 2024-09-10T23:57:44.170 DEBUG:teuthology.orchestra.run.smithi012:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-10T23:57:44.207 DEBUG:teuthology.orchestra.run.smithi165:> set -ex 2024-09-10T23:57:44.207 DEBUG:teuthology.orchestra.run.smithi165:> dd if=/scratch_devs of=/dev/stdout 2024-09-10T23:57:44.228 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-10T23:57:44.229 DEBUG:teuthology.orchestra.run.smithi165:> stat /dev/vg_nvme/lv_1 2024-09-10T23:57:44.289 INFO:teuthology.orchestra.run.smithi165.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-10T23:57:44.289 INFO:teuthology.orchestra.run.smithi165.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-10T23:57:44.290 INFO:teuthology.orchestra.run.smithi165.stdout:Device: 5h/5d Inode: 664 Links: 1 2024-09-10T23:57:44.290 INFO:teuthology.orchestra.run.smithi165.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-10T23:57:44.290 INFO:teuthology.orchestra.run.smithi165.stdout:Context: system_u:object_r:device_t:s0 2024-09-10T23:57:44.290 INFO:teuthology.orchestra.run.smithi165.stdout:Access: 2024-09-10 23:57:23.671579202 +0000 2024-09-10T23:57:44.290 INFO:teuthology.orchestra.run.smithi165.stdout:Modify: 2024-09-10 23:51:20.865680369 +0000 2024-09-10T23:57:44.290 INFO:teuthology.orchestra.run.smithi165.stdout:Change: 2024-09-10 23:51:20.865680369 +0000 2024-09-10T23:57:44.290 INFO:teuthology.orchestra.run.smithi165.stdout: Birth: 2024-09-10 23:51:20.865680369 +0000 2024-09-10T23:57:44.290 DEBUG:teuthology.orchestra.run.smithi165:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-10T23:57:44.362 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-09-10T23:57:44.363 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-09-10T23:57:44.363 INFO:teuthology.orchestra.run.smithi165.stderr:512 bytes copied, 0.000236116 s, 2.2 MB/s 2024-09-10T23:57:44.364 DEBUG:teuthology.orchestra.run.smithi165:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-10T23:57:44.423 DEBUG:teuthology.orchestra.run.smithi165:> stat /dev/vg_nvme/lv_2 2024-09-10T23:57:44.482 INFO:teuthology.orchestra.run.smithi165.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-10T23:57:44.482 INFO:teuthology.orchestra.run.smithi165.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-10T23:57:44.483 INFO:teuthology.orchestra.run.smithi165.stdout:Device: 5h/5d Inode: 683 Links: 1 2024-09-10T23:57:44.483 INFO:teuthology.orchestra.run.smithi165.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-10T23:57:44.483 INFO:teuthology.orchestra.run.smithi165.stdout:Context: system_u:object_r:device_t:s0 2024-09-10T23:57:44.483 INFO:teuthology.orchestra.run.smithi165.stdout:Access: 2024-09-10 23:57:23.671579202 +0000 2024-09-10T23:57:44.483 INFO:teuthology.orchestra.run.smithi165.stdout:Modify: 2024-09-10 23:51:21.208675171 +0000 2024-09-10T23:57:44.483 INFO:teuthology.orchestra.run.smithi165.stdout:Change: 2024-09-10 23:51:21.208675171 +0000 2024-09-10T23:57:44.483 INFO:teuthology.orchestra.run.smithi165.stdout: Birth: 2024-09-10 23:51:21.208675171 +0000 2024-09-10T23:57:44.484 DEBUG:teuthology.orchestra.run.smithi165:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-10T23:57:44.550 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-09-10T23:57:44.551 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-09-10T23:57:44.551 INFO:teuthology.orchestra.run.smithi165.stderr:512 bytes copied, 0.000234527 s, 2.2 MB/s 2024-09-10T23:57:44.552 DEBUG:teuthology.orchestra.run.smithi165:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-10T23:57:44.612 DEBUG:teuthology.orchestra.run.smithi165:> stat /dev/vg_nvme/lv_3 2024-09-10T23:57:44.670 INFO:teuthology.orchestra.run.smithi165.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-10T23:57:44.670 INFO:teuthology.orchestra.run.smithi165.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-10T23:57:44.670 INFO:teuthology.orchestra.run.smithi165.stdout:Device: 5h/5d Inode: 703 Links: 1 2024-09-10T23:57:44.670 INFO:teuthology.orchestra.run.smithi165.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-10T23:57:44.670 INFO:teuthology.orchestra.run.smithi165.stdout:Context: system_u:object_r:device_t:s0 2024-09-10T23:57:44.671 INFO:teuthology.orchestra.run.smithi165.stdout:Access: 2024-09-10 23:57:23.672579187 +0000 2024-09-10T23:57:44.671 INFO:teuthology.orchestra.run.smithi165.stdout:Modify: 2024-09-10 23:51:21.532670261 +0000 2024-09-10T23:57:44.671 INFO:teuthology.orchestra.run.smithi165.stdout:Change: 2024-09-10 23:51:21.532670261 +0000 2024-09-10T23:57:44.671 INFO:teuthology.orchestra.run.smithi165.stdout: Birth: 2024-09-10 23:51:21.532670261 +0000 2024-09-10T23:57:44.671 DEBUG:teuthology.orchestra.run.smithi165:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-10T23:57:44.737 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-09-10T23:57:44.737 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-09-10T23:57:44.737 INFO:teuthology.orchestra.run.smithi165.stderr:512 bytes copied, 0.000225535 s, 2.3 MB/s 2024-09-10T23:57:44.739 DEBUG:teuthology.orchestra.run.smithi165:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-10T23:57:44.797 DEBUG:teuthology.orchestra.run.smithi165:> stat /dev/vg_nvme/lv_4 2024-09-10T23:57:44.854 INFO:teuthology.orchestra.run.smithi165.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-10T23:57:44.855 INFO:teuthology.orchestra.run.smithi165.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-10T23:57:44.855 INFO:teuthology.orchestra.run.smithi165.stdout:Device: 5h/5d Inode: 721 Links: 1 2024-09-10T23:57:44.855 INFO:teuthology.orchestra.run.smithi165.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-10T23:57:44.855 INFO:teuthology.orchestra.run.smithi165.stdout:Context: system_u:object_r:device_t:s0 2024-09-10T23:57:44.855 INFO:teuthology.orchestra.run.smithi165.stdout:Access: 2024-09-10 23:57:23.672579187 +0000 2024-09-10T23:57:44.855 INFO:teuthology.orchestra.run.smithi165.stdout:Modify: 2024-09-10 23:51:21.863665245 +0000 2024-09-10T23:57:44.855 INFO:teuthology.orchestra.run.smithi165.stdout:Change: 2024-09-10 23:51:21.863665245 +0000 2024-09-10T23:57:44.855 INFO:teuthology.orchestra.run.smithi165.stdout: Birth: 2024-09-10 23:51:21.863665245 +0000 2024-09-10T23:57:44.855 DEBUG:teuthology.orchestra.run.smithi165:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-10T23:57:44.919 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-09-10T23:57:44.919 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-09-10T23:57:44.919 INFO:teuthology.orchestra.run.smithi165.stderr:512 bytes copied, 0.000211397 s, 2.4 MB/s 2024-09-10T23:57:44.920 DEBUG:teuthology.orchestra.run.smithi165:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-10T23:57:44.978 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2024-09-10T23:57:44.978 DEBUG:teuthology.orchestra.run.smithi178:> dd if=/scratch_devs of=/dev/stdout 2024-09-10T23:57:44.998 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-10T23:57:44.998 DEBUG:teuthology.orchestra.run.smithi178:> stat /dev/vg_nvme/lv_1 2024-09-10T23:57:45.056 INFO:teuthology.orchestra.run.smithi178.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-10T23:57:45.056 INFO:teuthology.orchestra.run.smithi178.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-10T23:57:45.057 INFO:teuthology.orchestra.run.smithi178.stdout:Device: 5h/5d Inode: 664 Links: 1 2024-09-10T23:57:45.057 INFO:teuthology.orchestra.run.smithi178.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-10T23:57:45.057 INFO:teuthology.orchestra.run.smithi178.stdout:Context: system_u:object_r:device_t:s0 2024-09-10T23:57:45.057 INFO:teuthology.orchestra.run.smithi178.stdout:Access: 2024-09-10 23:57:21.293132376 +0000 2024-09-10T23:57:45.057 INFO:teuthology.orchestra.run.smithi178.stdout:Modify: 2024-09-10 23:51:20.848319048 +0000 2024-09-10T23:57:45.057 INFO:teuthology.orchestra.run.smithi178.stdout:Change: 2024-09-10 23:51:20.848319048 +0000 2024-09-10T23:57:45.057 INFO:teuthology.orchestra.run.smithi178.stdout: Birth: 2024-09-10 23:51:20.848319048 +0000 2024-09-10T23:57:45.058 DEBUG:teuthology.orchestra.run.smithi178:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-10T23:57:45.126 INFO:teuthology.orchestra.run.smithi178.stderr:1+0 records in 2024-09-10T23:57:45.127 INFO:teuthology.orchestra.run.smithi178.stderr:1+0 records out 2024-09-10T23:57:45.127 INFO:teuthology.orchestra.run.smithi178.stderr:512 bytes copied, 0.000132301 s, 3.9 MB/s 2024-09-10T23:57:45.128 DEBUG:teuthology.orchestra.run.smithi178:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-10T23:57:45.189 DEBUG:teuthology.orchestra.run.smithi178:> stat /dev/vg_nvme/lv_2 2024-09-10T23:57:45.248 INFO:teuthology.orchestra.run.smithi178.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-10T23:57:45.248 INFO:teuthology.orchestra.run.smithi178.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-10T23:57:45.248 INFO:teuthology.orchestra.run.smithi178.stdout:Device: 5h/5d Inode: 686 Links: 1 2024-09-10T23:57:45.248 INFO:teuthology.orchestra.run.smithi178.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-10T23:57:45.248 INFO:teuthology.orchestra.run.smithi178.stdout:Context: system_u:object_r:device_t:s0 2024-09-10T23:57:45.248 INFO:teuthology.orchestra.run.smithi178.stdout:Access: 2024-09-10 23:57:21.294132357 +0000 2024-09-10T23:57:45.248 INFO:teuthology.orchestra.run.smithi178.stdout:Modify: 2024-09-10 23:51:21.175312627 +0000 2024-09-10T23:57:45.248 INFO:teuthology.orchestra.run.smithi178.stdout:Change: 2024-09-10 23:51:21.175312627 +0000 2024-09-10T23:57:45.248 INFO:teuthology.orchestra.run.smithi178.stdout: Birth: 2024-09-10 23:51:21.175312627 +0000 2024-09-10T23:57:45.249 DEBUG:teuthology.orchestra.run.smithi178:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-10T23:57:45.314 INFO:teuthology.orchestra.run.smithi178.stderr:1+0 records in 2024-09-10T23:57:45.314 INFO:teuthology.orchestra.run.smithi178.stderr:1+0 records out 2024-09-10T23:57:45.314 INFO:teuthology.orchestra.run.smithi178.stderr:512 bytes copied, 0.000100205 s, 5.1 MB/s 2024-09-10T23:57:45.315 DEBUG:teuthology.orchestra.run.smithi178:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-10T23:57:45.374 DEBUG:teuthology.orchestra.run.smithi178:> stat /dev/vg_nvme/lv_3 2024-09-10T23:57:45.430 INFO:teuthology.orchestra.run.smithi178.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-10T23:57:45.430 INFO:teuthology.orchestra.run.smithi178.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-10T23:57:45.430 INFO:teuthology.orchestra.run.smithi178.stdout:Device: 5h/5d Inode: 703 Links: 1 2024-09-10T23:57:45.430 INFO:teuthology.orchestra.run.smithi178.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-10T23:57:45.430 INFO:teuthology.orchestra.run.smithi178.stdout:Context: system_u:object_r:device_t:s0 2024-09-10T23:57:45.430 INFO:teuthology.orchestra.run.smithi178.stdout:Access: 2024-09-10 23:57:21.294132357 +0000 2024-09-10T23:57:45.430 INFO:teuthology.orchestra.run.smithi178.stdout:Modify: 2024-09-10 23:51:21.478306676 +0000 2024-09-10T23:57:45.430 INFO:teuthology.orchestra.run.smithi178.stdout:Change: 2024-09-10 23:51:21.478306676 +0000 2024-09-10T23:57:45.430 INFO:teuthology.orchestra.run.smithi178.stdout: Birth: 2024-09-10 23:51:21.478306676 +0000 2024-09-10T23:57:45.430 DEBUG:teuthology.orchestra.run.smithi178:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-10T23:57:45.494 INFO:teuthology.orchestra.run.smithi178.stderr:1+0 records in 2024-09-10T23:57:45.494 INFO:teuthology.orchestra.run.smithi178.stderr:1+0 records out 2024-09-10T23:57:45.494 INFO:teuthology.orchestra.run.smithi178.stderr:512 bytes copied, 9.7539e-05 s, 5.2 MB/s 2024-09-10T23:57:45.495 DEBUG:teuthology.orchestra.run.smithi178:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-10T23:57:45.551 DEBUG:teuthology.orchestra.run.smithi178:> stat /dev/vg_nvme/lv_4 2024-09-10T23:57:45.607 INFO:teuthology.orchestra.run.smithi178.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-10T23:57:45.607 INFO:teuthology.orchestra.run.smithi178.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-10T23:57:45.608 INFO:teuthology.orchestra.run.smithi178.stdout:Device: 5h/5d Inode: 724 Links: 1 2024-09-10T23:57:45.608 INFO:teuthology.orchestra.run.smithi178.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-10T23:57:45.608 INFO:teuthology.orchestra.run.smithi178.stdout:Context: system_u:object_r:device_t:s0 2024-09-10T23:57:45.608 INFO:teuthology.orchestra.run.smithi178.stdout:Access: 2024-09-10 23:57:21.295132337 +0000 2024-09-10T23:57:45.608 INFO:teuthology.orchestra.run.smithi178.stdout:Modify: 2024-09-10 23:51:21.793300490 +0000 2024-09-10T23:57:45.608 INFO:teuthology.orchestra.run.smithi178.stdout:Change: 2024-09-10 23:51:21.793300490 +0000 2024-09-10T23:57:45.608 INFO:teuthology.orchestra.run.smithi178.stdout: Birth: 2024-09-10 23:51:21.793300490 +0000 2024-09-10T23:57:45.608 DEBUG:teuthology.orchestra.run.smithi178:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-10T23:57:45.671 INFO:teuthology.orchestra.run.smithi178.stderr:1+0 records in 2024-09-10T23:57:45.671 INFO:teuthology.orchestra.run.smithi178.stderr:1+0 records out 2024-09-10T23:57:45.671 INFO:teuthology.orchestra.run.smithi178.stderr:512 bytes copied, 0.000110149 s, 4.6 MB/s 2024-09-10T23:57:45.673 DEBUG:teuthology.orchestra.run.smithi178:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-10T23:57:45.730 INFO:tasks.cephadm:Deploying osd.0 on smithi012 with /dev/vg_nvme/lv_4... 2024-09-10T23:57:45.730 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ff3ad74a-6fcf-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_4 2024-09-10T23:57:45.831 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:57:45 smithi178 ceph-mon[35919]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T23:57:46.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:45 smithi165 ceph-mon[35356]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T23:57:46.062 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:57:45 smithi012 ceph-mon[30333]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T23:57:46.105 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T23:57:47.805 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:57:47 smithi012 ceph-mon[30333]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T23:57:48.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:47 smithi165 ceph-mon[35356]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T23:57:48.080 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:57:47 smithi178 ceph-mon[35919]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T23:57:48.942 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-10T23:57:48.976 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ff3ad74a-6fcf-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi012:vg_nvme/lv_4 2024-09-10T23:57:49.358 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T23:57:49.788 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:57:49 smithi012 ceph-mon[30333]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T23:57:49.916 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:49.914+0000 7f77f6600640 1 -- 172.21.15.12:0/4269096535 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f77f0105da0 msgr2=0x7f77f0106220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:57:49.916 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:49.914+0000 7f77f6600640 1 --2- 172.21.15.12:0/4269096535 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f77f0105da0 0x7f77f0106220 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f77dc009930 tx=0x7f77dc02a420 comp rx=0 tx=0).stop 2024-09-10T23:57:49.916 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:49.914+0000 7f77f6600640 1 -- 172.21.15.12:0/4269096535 shutdown_connections 2024-09-10T23:57:49.916 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:49.914+0000 7f77f6600640 1 --2- 172.21.15.12:0/4269096535 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f77f0106760 0x7f77f010d0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:57:49.916 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:49.914+0000 7f77f6600640 1 --2- 172.21.15.12:0/4269096535 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f77f0105da0 0x7f77f0106220 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:57:49.916 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:49.914+0000 7f77f6600640 1 --2- 172.21.15.12:0/4269096535 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f77f0104c40 0x7f77f0105040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:57:49.917 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:49.914+0000 7f77f6600640 1 -- 172.21.15.12:0/4269096535 >> 172.21.15.12:0/4269096535 conn(0x7f77f0100510 msgr2=0x7f77f0102950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:57:49.917 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:49.914+0000 7f77f6600640 1 -- 172.21.15.12:0/4269096535 shutdown_connections 2024-09-10T23:57:49.917 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:49.914+0000 7f77f6600640 1 -- 172.21.15.12:0/4269096535 wait complete. 2024-09-10T23:57:49.917 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:49.915+0000 7f77f6600640 1 Processor -- start 2024-09-10T23:57:49.917 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:49.915+0000 7f77f6600640 1 -- start start 2024-09-10T23:57:49.919 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:49.916+0000 7f77f6600640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f77f0104c40 0x7f77f019b020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:57:49.919 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:49.916+0000 7f77f6600640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f77f0105da0 0x7f77f019b670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:57:49.919 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:49.916+0000 7f77f6600640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f77f0106760 0x7f77f019c2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:57:49.919 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:49.916+0000 7f77f6600640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f77f01108f0 con 0x7f77f0104c40 2024-09-10T23:57:49.919 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:49.916+0000 7f77f6600640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f77f0110770 con 0x7f77f0106760 2024-09-10T23:57:49.919 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:49.916+0000 7f77f6600640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f77f0110a70 con 0x7f77f0105da0 2024-09-10T23:57:49.919 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:49.916+0000 7f77ef400640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f77f0104c40 0x7f77f019b020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:57:49.919 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:49.916+0000 7f77ef400640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f77f0104c40 0x7f77f019b020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:51972/0 (socket says 172.21.15.12:51972) 2024-09-10T23:57:49.919 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:49.916+0000 7f77ef400640 1 -- 172.21.15.12:0/1446412250 learned_addr learned my addr 172.21.15.12:0/1446412250 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-10T23:57:49.919 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:49.916+0000 7f77efe00640 1 --2- 172.21.15.12:0/1446412250 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f77f0106760 0x7f77f019c2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:57:49.919 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:49.916+0000 7f77eea00640 1 --2- 172.21.15.12:0/1446412250 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f77f0105da0 0x7f77f019b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:57:49.919 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:49.916+0000 7f77ef400640 1 -- 172.21.15.12:0/1446412250 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f77f0105da0 msgr2=0x7f77f019b670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:57:49.919 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:49.916+0000 7f77ef400640 1 --2- 172.21.15.12:0/1446412250 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f77f0105da0 0x7f77f019b670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:57:49.920 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:49.916+0000 7f77ef400640 1 -- 172.21.15.12:0/1446412250 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f77f0106760 msgr2=0x7f77f019c2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:57:49.920 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:49.916+0000 7f77ef400640 1 --2- 172.21.15.12:0/1446412250 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f77f0106760 0x7f77f019c2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:57:49.920 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:49.916+0000 7f77ef400640 1 -- 172.21.15.12:0/1446412250 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77f019cb50 con 0x7f77f0104c40 2024-09-10T23:57:49.920 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:49.917+0000 7f77ef400640 1 --2- 172.21.15.12:0/1446412250 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f77f0104c40 0x7f77f019b020 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f77e000e9f0 tx=0x7f77e000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:57:49.920 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:49.917+0000 7f77e7e00640 1 -- 172.21.15.12:0/1446412250 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77e000cd80 con 0x7f77f0104c40 2024-09-10T23:57:49.920 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:49.917+0000 7f77f6600640 1 -- 172.21.15.12:0/1446412250 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f77f019ce40 con 0x7f77f0104c40 2024-09-10T23:57:49.920 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:49.917+0000 7f77e7e00640 1 -- 172.21.15.12:0/1446412250 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f77e00108e0 con 0x7f77f0104c40 2024-09-10T23:57:49.920 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:49.917+0000 7f77e7e00640 1 -- 172.21.15.12:0/1446412250 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77e0018980 con 0x7f77f0104c40 2024-09-10T23:57:49.920 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:49.917+0000 7f77f6600640 1 -- 172.21.15.12:0/1446412250 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77f01a3520 con 0x7f77f0104c40 2024-09-10T23:57:49.921 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:49.919+0000 7f77f6600640 1 -- 172.21.15.12:0/1446412250 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f77f010bfc0 con 0x7f77f0104c40 2024-09-10T23:57:49.921 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:49.920+0000 7f77e7e00640 1 -- 172.21.15.12:0/1446412250 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 15) ==== 153194+0+0 (secure 0 0 0) 0x7f77e0010430 con 0x7f77f0104c40 2024-09-10T23:57:49.923 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:49.922+0000 7f77e7e00640 1 --2- 172.21.15.12:0/1446412250 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f77d40b52d0 0x7f77d40b7790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:57:49.923 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:49.922+0000 7f77e7e00640 1 -- 172.21.15.12:0/1446412250 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f77e00e4250 con 0x7f77f0104c40 2024-09-10T23:57:49.923 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:49.922+0000 7f77eea00640 1 --2- 172.21.15.12:0/1446412250 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f77d40b52d0 0x7f77d40b7790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:57:49.924 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:49.923+0000 7f77eea00640 1 --2- 172.21.15.12:0/1446412250 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f77d40b52d0 0x7f77d40b7790 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f77dc009900 tx=0x7f77dc031450 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:57:49.927 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:49.926+0000 7f77e7e00640 1 -- 172.21.15.12:0/1446412250 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f77e00ac6f0 con 0x7f77f0104c40 2024-09-10T23:57:50.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:49 smithi165 ceph-mon[35356]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T23:57:50.080 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:57:49 smithi178 ceph-mon[35919]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T23:57:50.133 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:57:50.132+0000 7f77f6600640 1 -- 172.21.15.12:0/1446412250 --> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi012:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f77f0063590 con 0x7f77d40b52d0 2024-09-10T23:57:50.956 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:57:50 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-10T23:57:50.956 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:57:50 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-10T23:57:50.956 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:57:50 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T23:57:51.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:50 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-10T23:57:51.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:50 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-10T23:57:51.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:50 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T23:57:51.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:57:50 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-10T23:57:51.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:57:50 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-10T23:57:51.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:57:50 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T23:57:51.956 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:57:51 smithi012 ceph-mon[30333]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T23:57:51.956 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:57:51 smithi012 ceph-mon[30333]: from='client.14352 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi012:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T23:57:52.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:51 smithi165 ceph-mon[35356]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T23:57:52.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:51 smithi165 ceph-mon[35356]: from='client.14352 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi012:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T23:57:52.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:57:51 smithi178 ceph-mon[35919]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T23:57:52.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:57:51 smithi178 ceph-mon[35919]: from='client.14352 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi012:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T23:57:54.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:53 smithi165 ceph-mon[35356]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T23:57:54.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:53 smithi165 ceph-mon[35356]: from='client.? 172.21.15.12:0/111977585' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "6382c200-a912-42a1-958b-79191b21249b"} : dispatch 2024-09-10T23:57:54.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:53 smithi165 ceph-mon[35356]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "6382c200-a912-42a1-958b-79191b21249b"} : dispatch 2024-09-10T23:57:54.033 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:53 smithi165 ceph-mon[35356]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6382c200-a912-42a1-958b-79191b21249b"}]': finished 2024-09-10T23:57:54.033 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:53 smithi165 ceph-mon[35356]: osdmap e5: 1 total, 0 up, 1 in 2024-09-10T23:57:54.033 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:53 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-10T23:57:54.033 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:53 smithi165 ceph-mon[35356]: from='client.? 172.21.15.12:0/394766917' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-10T23:57:54.063 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:57:53 smithi012 ceph-mon[30333]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T23:57:54.063 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:57:53 smithi012 ceph-mon[30333]: from='client.? 172.21.15.12:0/111977585' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "6382c200-a912-42a1-958b-79191b21249b"} : dispatch 2024-09-10T23:57:54.063 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:57:53 smithi012 ceph-mon[30333]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "6382c200-a912-42a1-958b-79191b21249b"} : dispatch 2024-09-10T23:57:54.063 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:57:53 smithi012 ceph-mon[30333]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6382c200-a912-42a1-958b-79191b21249b"}]': finished 2024-09-10T23:57:54.063 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:57:53 smithi012 ceph-mon[30333]: osdmap e5: 1 total, 0 up, 1 in 2024-09-10T23:57:54.063 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:57:53 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-10T23:57:54.063 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:57:53 smithi012 ceph-mon[30333]: from='client.? 172.21.15.12:0/394766917' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-10T23:57:54.080 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:57:53 smithi178 ceph-mon[35919]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T23:57:54.080 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:57:53 smithi178 ceph-mon[35919]: from='client.? 172.21.15.12:0/111977585' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "6382c200-a912-42a1-958b-79191b21249b"} : dispatch 2024-09-10T23:57:54.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:57:53 smithi178 ceph-mon[35919]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "6382c200-a912-42a1-958b-79191b21249b"} : dispatch 2024-09-10T23:57:54.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:57:53 smithi178 ceph-mon[35919]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6382c200-a912-42a1-958b-79191b21249b"}]': finished 2024-09-10T23:57:54.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:57:53 smithi178 ceph-mon[35919]: osdmap e5: 1 total, 0 up, 1 in 2024-09-10T23:57:54.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:57:53 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-10T23:57:54.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:57:53 smithi178 ceph-mon[35919]: from='client.? 172.21.15.12:0/394766917' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-10T23:57:56.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:55 smithi165 ceph-mon[35356]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T23:57:56.062 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:57:55 smithi012 ceph-mon[30333]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T23:57:56.080 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:57:55 smithi178 ceph-mon[35919]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T23:57:58.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:57 smithi165 ceph-mon[35356]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T23:57:58.063 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:57:57 smithi012 ceph-mon[30333]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T23:57:58.080 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:57:57 smithi178 ceph-mon[35919]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T23:57:59.984 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:57:59 smithi012 ceph-mon[30333]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T23:58:00.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:57:59 smithi165 ceph-mon[35356]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T23:58:00.080 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:57:59 smithi178 ceph-mon[35919]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T23:58:02.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:58:01 smithi165 ceph-mon[35356]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T23:58:02.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:58:01 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.0"} : dispatch 2024-09-10T23:58:02.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:58:01 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T23:58:02.062 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:58:01 smithi012 ceph-mon[30333]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T23:58:02.062 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:58:01 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.0"} : dispatch 2024-09-10T23:58:02.062 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:58:01 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T23:58:02.080 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:58:01 smithi178 ceph-mon[35919]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T23:58:02.080 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:58:01 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.0"} : dispatch 2024-09-10T23:58:02.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:58:01 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T23:58:02.813 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:58:02 smithi012 ceph-mon[30333]: Deploying daemon osd.0 on smithi012 2024-09-10T23:58:03.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:58:02 smithi165 ceph-mon[35356]: Deploying daemon osd.0 on smithi012 2024-09-10T23:58:03.080 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:58:02 smithi178 ceph-mon[35919]: Deploying daemon osd.0 on smithi012 2024-09-10T23:58:04.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:58:03 smithi165 ceph-mon[35356]: pgmap v47: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T23:58:04.062 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:58:03 smithi012 ceph-mon[30333]: pgmap v47: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T23:58:04.080 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:58:03 smithi178 ceph-mon[35919]: pgmap v47: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T23:58:05.813 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:58:05 smithi012 ceph-mon[30333]: pgmap v48: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T23:58:06.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:58:05 smithi165 ceph-mon[35356]: pgmap v48: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T23:58:06.080 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:58:05 smithi178 ceph-mon[35919]: pgmap v48: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T23:58:08.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:58:07 smithi165 ceph-mon[35356]: pgmap v49: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T23:58:08.063 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:58:07 smithi012 ceph-mon[30333]: pgmap v49: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T23:58:08.080 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:58:07 smithi178 ceph-mon[35919]: pgmap v49: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T23:58:09.993 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:58:09 smithi012 ceph-mon[30333]: pgmap v50: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T23:58:09.993 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:58:09 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T23:58:09.993 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:58:09 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-10T23:58:09.993 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:58:09 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-10T23:58:10.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:58:09 smithi165 ceph-mon[35356]: pgmap v50: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T23:58:10.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:58:09 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T23:58:10.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:58:09 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-10T23:58:10.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:58:09 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-10T23:58:10.080 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:58:09 smithi178 ceph-mon[35919]: pgmap v50: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T23:58:10.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:58:09 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T23:58:10.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:58:09 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-10T23:58:10.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:58:09 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-10T23:58:12.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:58:11 smithi165 ceph-mon[35356]: pgmap v51: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T23:58:12.062 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:58:11 smithi012 ceph-mon[30333]: pgmap v51: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T23:58:12.080 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:58:11 smithi178 ceph-mon[35919]: pgmap v51: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T23:58:13.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:58:12 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-10T23:58:13.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:58:12 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-10T23:58:13.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:58:12 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T23:58:13.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:58:12 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T23:58:13.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:58:12 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-10T23:58:13.283 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:58:12 smithi165 ceph-mon[35356]: pgmap v52: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T23:58:13.298 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:58:12 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-10T23:58:13.298 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:58:12 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-10T23:58:13.299 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:58:12 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T23:58:13.299 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:58:12 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T23:58:13.299 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:58:12 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-10T23:58:13.299 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:58:12 smithi012 ceph-mon[30333]: pgmap v52: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T23:58:13.330 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:58:12 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-10T23:58:13.330 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:58:12 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-10T23:58:13.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:58:12 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T23:58:13.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:58:12 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T23:58:13.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:58:12 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-10T23:58:13.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:58:12 smithi178 ceph-mon[35919]: pgmap v52: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T23:58:13.664 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:58:13.663+0000 7f77e7e00640 1 -- 172.21.15.12:0/1446412250 <== mgr.14150 v2:172.21.15.12:6800/3442939137 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f77f0063590 con 0x7f77d40b52d0 2024-09-10T23:58:13.664 INFO:teuthology.orchestra.run.smithi012.stdout:Created osd(s) 0 on host 'smithi012' 2024-09-10T23:58:13.666 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:58:13.665+0000 7f77f6600640 1 -- 172.21.15.12:0/1446412250 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f77d40b52d0 msgr2=0x7f77d40b7790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:58:13.666 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:58:13.665+0000 7f77f6600640 1 --2- 172.21.15.12:0/1446412250 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f77d40b52d0 0x7f77d40b7790 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f77dc009900 tx=0x7f77dc031450 comp rx=0 tx=0).stop 2024-09-10T23:58:13.666 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:58:13.665+0000 7f77f6600640 1 -- 172.21.15.12:0/1446412250 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f77f0104c40 msgr2=0x7f77f019b020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:58:13.666 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:58:13.665+0000 7f77f6600640 1 --2- 172.21.15.12:0/1446412250 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f77f0104c40 0x7f77f019b020 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f77e000e9f0 tx=0x7f77e000eeb0 comp rx=0 tx=0).stop 2024-09-10T23:58:13.667 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:58:13.665+0000 7f77f6600640 1 -- 172.21.15.12:0/1446412250 shutdown_connections 2024-09-10T23:58:13.667 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:58:13.666+0000 7f77f6600640 1 --2- 172.21.15.12:0/1446412250 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f77d40b52d0 0x7f77d40b7790 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:58:13.667 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:58:13.666+0000 7f77f6600640 1 --2- 172.21.15.12:0/1446412250 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f77f0106760 0x7f77f019c2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:58:13.667 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:58:13.666+0000 7f77f6600640 1 --2- 172.21.15.12:0/1446412250 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f77f0105da0 0x7f77f019b670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:58:13.667 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:58:13.666+0000 7f77f6600640 1 --2- 172.21.15.12:0/1446412250 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f77f0104c40 0x7f77f019b020 secure :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f77e000e9f0 tx=0x7f77e000eeb0 comp rx=0 tx=0).stop 2024-09-10T23:58:13.667 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:58:13.666+0000 7f77f6600640 1 -- 172.21.15.12:0/1446412250 >> 172.21.15.12:0/1446412250 conn(0x7f77f0100510 msgr2=0x7f77f0109190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:58:13.668 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:58:13.666+0000 7f77f6600640 1 -- 172.21.15.12:0/1446412250 shutdown_connections 2024-09-10T23:58:13.668 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:58:13.667+0000 7f77f6600640 1 -- 172.21.15.12:0/1446412250 wait complete. 2024-09-10T23:58:14.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:58:13 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T23:58:14.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:58:13 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-10T23:58:14.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:58:13 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-10T23:58:14.287 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:58:13 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T23:58:14.287 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:58:13 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-10T23:58:14.287 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:58:13 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-10T23:58:14.323 DEBUG:teuthology.orchestra.run.smithi012:osd.0> sudo journalctl -f -n 0 -u ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968@osd.0.service 2024-09-10T23:58:14.327 INFO:tasks.cephadm:Deploying osd.1 on smithi165 with /dev/vg_nvme/lv_4... 2024-09-10T23:58:14.327 DEBUG:teuthology.orchestra.run.smithi165:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ff3ad74a-6fcf-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_4 2024-09-10T23:58:14.330 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:58:13 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T23:58:14.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:58:13 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-10T23:58:14.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:58:13 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-10T23:58:14.576 INFO:teuthology.orchestra.run.smithi165.stderr:Inferring config /var/lib/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T23:58:15.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:58:14 smithi165 ceph-mon[35356]: pgmap v53: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T23:58:15.306 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:58:14 smithi012 ceph-mon[30333]: pgmap v53: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T23:58:15.306 INFO:journalctl@ceph.osd.0.smithi012.stdout:Sep 10 23:58:15 smithi012 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-0[46630]: 2024-09-10T23:58:15.052+0000 7f71a6c69640 -1 osd.0 0 log_to_monitors true 2024-09-10T23:58:15.330 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:58:14 smithi178 ceph-mon[35919]: pgmap v53: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T23:58:16.096 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:58:15 smithi012 ceph-mon[30333]: from='osd.0 [v2:172.21.15.12:6802/1237101991,v1:172.21.15.12:6803/1237101991]' entity='osd.0' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]} : dispatch 2024-09-10T23:58:16.215 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:58:15 smithi165 ceph-mon[35356]: from='osd.0 [v2:172.21.15.12:6802/1237101991,v1:172.21.15.12:6803/1237101991]' entity='osd.0' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]} : dispatch 2024-09-10T23:58:16.330 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:58:15 smithi178 ceph-mon[35919]: from='osd.0 [v2:172.21.15.12:6802/1237101991,v1:172.21.15.12:6803/1237101991]' entity='osd.0' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]} : dispatch 2024-09-10T23:58:16.888 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-09-10T23:58:16.916 DEBUG:teuthology.orchestra.run.smithi165:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ff3ad74a-6fcf-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi165:vg_nvme/lv_4 2024-09-10T23:58:17.165 INFO:teuthology.orchestra.run.smithi165.stderr:Inferring config /var/lib/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T23:58:17.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:58:16 smithi165 ceph-mon[35356]: from='osd.0 [v2:172.21.15.12:6802/1237101991,v1:172.21.15.12:6803/1237101991]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-10T23:58:17.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:58:16 smithi165 ceph-mon[35356]: osdmap e6: 1 total, 0 up, 1 in 2024-09-10T23:58:17.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:58:16 smithi165 ceph-mon[35356]: from='osd.0 [v2:172.21.15.12:6802/1237101991,v1:172.21.15.12:6803/1237101991]' entity='osd.0' cmd={"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi012", "root=default"]} : dispatch 2024-09-10T23:58:17.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:58:16 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-10T23:58:17.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:58:16 smithi165 ceph-mon[35356]: pgmap v55: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T23:58:17.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:58:16 smithi012 ceph-mon[30333]: from='osd.0 [v2:172.21.15.12:6802/1237101991,v1:172.21.15.12:6803/1237101991]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-10T23:58:17.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:58:16 smithi012 ceph-mon[30333]: osdmap e6: 1 total, 0 up, 1 in 2024-09-10T23:58:17.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:58:16 smithi012 ceph-mon[30333]: from='osd.0 [v2:172.21.15.12:6802/1237101991,v1:172.21.15.12:6803/1237101991]' entity='osd.0' cmd={"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi012", "root=default"]} : dispatch 2024-09-10T23:58:17.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:58:16 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-10T23:58:17.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:58:16 smithi012 ceph-mon[30333]: pgmap v55: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T23:58:17.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:58:16 smithi178 ceph-mon[35919]: from='osd.0 [v2:172.21.15.12:6802/1237101991,v1:172.21.15.12:6803/1237101991]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-10T23:58:17.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:58:16 smithi178 ceph-mon[35919]: osdmap e6: 1 total, 0 up, 1 in 2024-09-10T23:58:17.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:58:16 smithi178 ceph-mon[35919]: from='osd.0 [v2:172.21.15.12:6802/1237101991,v1:172.21.15.12:6803/1237101991]' entity='osd.0' cmd={"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi012", "root=default"]} : dispatch 2024-09-10T23:58:17.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:58:16 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-10T23:58:17.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:58:16 smithi178 ceph-mon[35919]: pgmap v55: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T23:58:17.615 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-10T23:58:17.612+0000 7f6692a9b640 1 -- 172.21.15.165:0/1571149520 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f668c101930 msgr2=0x7f668c101d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:58:17.615 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-10T23:58:17.612+0000 7f6692a9b640 1 --2- 172.21.15.165:0/1571149520 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f668c101930 0x7f668c101d30 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f667c009930 tx=0x7f667c02a420 comp rx=0 tx=0).stop 2024-09-10T23:58:17.615 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-10T23:58:17.612+0000 7f6692a9b640 1 -- 172.21.15.165:0/1571149520 shutdown_connections 2024-09-10T23:58:17.616 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-10T23:58:17.612+0000 7f6692a9b640 1 --2- 172.21.15.165:0/1571149520 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f668c103450 0x7f668c109dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:58:17.616 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-10T23:58:17.612+0000 7f6692a9b640 1 --2- 172.21.15.165:0/1571149520 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f668c102a90 0x7f668c102f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:58:17.616 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-10T23:58:17.612+0000 7f6692a9b640 1 --2- 172.21.15.165:0/1571149520 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f668c101930 0x7f668c101d30 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:58:17.616 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-10T23:58:17.612+0000 7f6692a9b640 1 -- 172.21.15.165:0/1571149520 >> 172.21.15.165:0/1571149520 conn(0x7f668c0fd200 msgr2=0x7f668c0ff640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:58:17.616 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-10T23:58:17.612+0000 7f6692a9b640 1 -- 172.21.15.165:0/1571149520 shutdown_connections 2024-09-10T23:58:17.616 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-10T23:58:17.612+0000 7f6692a9b640 1 -- 172.21.15.165:0/1571149520 wait complete. 2024-09-10T23:58:17.616 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-10T23:58:17.613+0000 7f6692a9b640 1 Processor -- start 2024-09-10T23:58:17.616 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-10T23:58:17.613+0000 7f6692a9b640 1 -- start start 2024-09-10T23:58:17.616 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-10T23:58:17.613+0000 7f6692a9b640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f668c101930 0x7f668c197d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:58:17.616 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-10T23:58:17.613+0000 7f6692a9b640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f668c102a90 0x7f668c1983e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:58:17.616 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-10T23:58:17.613+0000 7f6692a9b640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f668c103450 0x7f668c199020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:58:17.617 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-10T23:58:17.613+0000 7f6692a9b640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f668c10d590 con 0x7f668c102a90 2024-09-10T23:58:17.617 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-10T23:58:17.613+0000 7f6692a9b640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f668c10d410 con 0x7f668c101930 2024-09-10T23:58:17.617 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-10T23:58:17.613+0000 7f6692a9b640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f668c10d710 con 0x7f668c103450 2024-09-10T23:58:17.617 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-10T23:58:17.613+0000 7f6690837640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f668c101930 0x7f668c197d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:58:17.617 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-10T23:58:17.613+0000 7f6690837640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f668c101930 0x7f668c197d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:49314/0 (socket says 172.21.15.165:49314) 2024-09-10T23:58:17.617 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-10T23:58:17.613+0000 7f6690837640 1 -- 172.21.15.165:0/1575295235 learned_addr learned my addr 172.21.15.165:0/1575295235 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-09-10T23:58:17.617 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-10T23:58:17.613+0000 7f6683fff640 1 --2- 172.21.15.165:0/1575295235 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f668c102a90 0x7f668c1983e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:58:17.617 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-10T23:58:17.613+0000 7f6691038640 1 --2- 172.21.15.165:0/1575295235 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f668c103450 0x7f668c199020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:58:17.617 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-10T23:58:17.613+0000 7f6690837640 1 -- 172.21.15.165:0/1575295235 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f668c103450 msgr2=0x7f668c199020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:58:17.617 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-10T23:58:17.613+0000 7f6690837640 1 --2- 172.21.15.165:0/1575295235 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f668c103450 0x7f668c199020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:58:17.617 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-10T23:58:17.613+0000 7f6690837640 1 -- 172.21.15.165:0/1575295235 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f668c102a90 msgr2=0x7f668c1983e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:58:17.617 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-10T23:58:17.613+0000 7f6690837640 1 --2- 172.21.15.165:0/1575295235 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f668c102a90 0x7f668c1983e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:58:17.618 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-10T23:58:17.613+0000 7f6690837640 1 -- 172.21.15.165:0/1575295235 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f668c1998c0 con 0x7f668c101930 2024-09-10T23:58:17.618 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-10T23:58:17.613+0000 7f6683fff640 1 --2- 172.21.15.165:0/1575295235 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f668c102a90 0x7f668c1983e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T23:58:17.618 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-10T23:58:17.613+0000 7f6690837640 1 --2- 172.21.15.165:0/1575295235 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f668c101930 0x7f668c197d90 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f667c005ab0 tx=0x7f667c0042f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:58:17.618 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-10T23:58:17.614+0000 7f6681ffb640 1 -- 172.21.15.165:0/1575295235 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f667c03f030 con 0x7f668c101930 2024-09-10T23:58:17.618 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-10T23:58:17.614+0000 7f6681ffb640 1 -- 172.21.15.165:0/1575295235 <== mon.2 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f667c003cb0 con 0x7f668c101930 2024-09-10T23:58:17.618 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-10T23:58:17.614+0000 7f6692a9b640 1 -- 172.21.15.165:0/1575295235 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f668c199b50 con 0x7f668c101930 2024-09-10T23:58:17.618 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-10T23:58:17.614+0000 7f6681ffb640 1 -- 172.21.15.165:0/1575295235 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f667c035070 con 0x7f668c101930 2024-09-10T23:58:17.618 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-10T23:58:17.614+0000 7f6692a9b640 1 -- 172.21.15.165:0/1575295235 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f668c1a0290 con 0x7f668c101930 2024-09-10T23:58:17.619 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-10T23:58:17.615+0000 7f6692a9b640 1 -- 172.21.15.165:0/1575295235 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f668c101d30 con 0x7f668c101930 2024-09-10T23:58:17.619 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-10T23:58:17.616+0000 7f6681ffb640 1 -- 172.21.15.165:0/1575295235 <== mon.2 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 15) ==== 153194+0+0 (secure 0 0 0) 0x7f667c03bb50 con 0x7f668c101930 2024-09-10T23:58:17.619 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-10T23:58:17.617+0000 7f6681ffb640 1 --2- 172.21.15.165:0/1575295235 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f66740b52d0 0x7f66740b7790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:58:17.620 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-10T23:58:17.617+0000 7f6681ffb640 1 -- 172.21.15.165:0/1575295235 <== mon.2 v2:172.21.15.165:3300/0 5 ==== osd_map(7..7 src has 1..7) ==== 1487+0+0 (secure 0 0 0) 0x7f667c104cb0 con 0x7f668c101930 2024-09-10T23:58:17.620 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-10T23:58:17.617+0000 7f6683fff640 1 --2- 172.21.15.165:0/1575295235 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f66740b52d0 0x7f66740b7790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:58:17.620 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-10T23:58:17.618+0000 7f6683fff640 1 --2- 172.21.15.165:0/1575295235 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f66740b52d0 0x7f66740b7790 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f66880058c0 tx=0x7f6688005ec0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:58:17.621 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-10T23:58:17.620+0000 7f6681ffb640 1 -- 172.21.15.165:0/1575295235 <== mon.2 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f667c0cd700 con 0x7f668c101930 2024-09-10T23:58:17.808 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-10T23:58:17.807+0000 7f6692a9b640 1 -- 172.21.15.165:0/1575295235 --> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi165:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f668c063650 con 0x7f66740b52d0 2024-09-10T23:58:17.973 INFO:journalctl@ceph.osd.0.smithi012.stdout:Sep 10 23:58:17 smithi012 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-0[46630]: 2024-09-10T23:58:17.676+0000 7f71a2200640 -1 osd.0 0 waiting for initial osdmap 2024-09-10T23:58:17.973 INFO:journalctl@ceph.osd.0.smithi012.stdout:Sep 10 23:58:17 smithi012 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-0[46630]: 2024-09-10T23:58:17.688+0000 7f7197800640 -1 osd.0 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-10T23:58:18.239 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:58:17 smithi165 ceph-mon[35356]: from='osd.0 [v2:172.21.15.12:6802/1237101991,v1:172.21.15.12:6803/1237101991]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi012", "root=default"]}]': finished 2024-09-10T23:58:18.239 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:58:17 smithi165 ceph-mon[35356]: osdmap e7: 1 total, 0 up, 1 in 2024-09-10T23:58:18.239 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:58:17 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-10T23:58:18.239 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:58:17 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-10T23:58:18.239 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:58:17 smithi165 ceph-mon[35356]: from='osd.0 [v2:172.21.15.12:6802/1237101991,v1:172.21.15.12:6803/1237101991]' entity='osd.0' 2024-09-10T23:58:18.239 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:58:17 smithi165 ceph-mon[35356]: from='client.24266 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi165:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T23:58:18.239 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:58:17 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-10T23:58:18.239 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:58:17 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-10T23:58:18.239 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:58:17 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T23:58:18.239 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:58:17 smithi165 ceph-mon[35356]: Detected new or changed devices on smithi012 2024-09-10T23:58:18.240 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:58:17 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-10T23:58:18.240 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:58:17 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-10T23:58:18.240 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:58:17 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-10T23:58:18.240 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:58:17 smithi165 ceph-mon[35356]: Adjusting osd_memory_target on smithi012 to 17094M 2024-09-10T23:58:18.240 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:58:17 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-10T23:58:18.240 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:58:17 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T23:58:18.240 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:58:17 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T23:58:18.240 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:58:17 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-10T23:58:18.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:58:17 smithi012 ceph-mon[30333]: from='osd.0 [v2:172.21.15.12:6802/1237101991,v1:172.21.15.12:6803/1237101991]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi012", "root=default"]}]': finished 2024-09-10T23:58:18.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:58:17 smithi012 ceph-mon[30333]: osdmap e7: 1 total, 0 up, 1 in 2024-09-10T23:58:18.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:58:17 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-10T23:58:18.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:58:17 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-10T23:58:18.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:58:17 smithi012 ceph-mon[30333]: from='osd.0 [v2:172.21.15.12:6802/1237101991,v1:172.21.15.12:6803/1237101991]' entity='osd.0' 2024-09-10T23:58:18.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:58:17 smithi012 ceph-mon[30333]: from='client.24266 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi165:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T23:58:18.314 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:58:17 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-10T23:58:18.314 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:58:17 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-10T23:58:18.314 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:58:17 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T23:58:18.314 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:58:17 smithi012 ceph-mon[30333]: Detected new or changed devices on smithi012 2024-09-10T23:58:18.314 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:58:17 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-10T23:58:18.314 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:58:17 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-10T23:58:18.314 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:58:17 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-10T23:58:18.314 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:58:17 smithi012 ceph-mon[30333]: Adjusting osd_memory_target on smithi012 to 17094M 2024-09-10T23:58:18.314 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:58:17 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-10T23:58:18.314 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:58:17 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T23:58:18.314 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:58:17 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T23:58:18.314 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:58:17 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-10T23:58:18.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:58:17 smithi178 ceph-mon[35919]: from='osd.0 [v2:172.21.15.12:6802/1237101991,v1:172.21.15.12:6803/1237101991]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi012", "root=default"]}]': finished 2024-09-10T23:58:18.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:58:17 smithi178 ceph-mon[35919]: osdmap e7: 1 total, 0 up, 1 in 2024-09-10T23:58:18.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:58:17 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-10T23:58:18.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:58:17 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-10T23:58:18.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:58:17 smithi178 ceph-mon[35919]: from='osd.0 [v2:172.21.15.12:6802/1237101991,v1:172.21.15.12:6803/1237101991]' entity='osd.0' 2024-09-10T23:58:18.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:58:17 smithi178 ceph-mon[35919]: from='client.24266 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi165:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T23:58:18.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:58:17 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-10T23:58:18.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:58:17 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-10T23:58:18.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:58:17 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T23:58:18.332 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:58:17 smithi178 ceph-mon[35919]: Detected new or changed devices on smithi012 2024-09-10T23:58:18.332 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:58:17 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-10T23:58:18.332 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:58:17 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-10T23:58:18.332 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:58:17 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-10T23:58:18.332 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:58:17 smithi178 ceph-mon[35919]: Adjusting osd_memory_target on smithi012 to 17094M 2024-09-10T23:58:18.332 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:58:17 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-10T23:58:18.332 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:58:17 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T23:58:18.332 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:58:17 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T23:58:18.332 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:58:17 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-10T23:58:19.236 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:58:18 smithi165 ceph-mon[35356]: purged_snaps scrub starts 2024-09-10T23:58:19.236 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:58:18 smithi165 ceph-mon[35356]: purged_snaps scrub ok 2024-09-10T23:58:19.236 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:58:18 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-10T23:58:19.236 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:58:18 smithi165 ceph-mon[35356]: pgmap v57: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T23:58:19.236 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:58:18 smithi165 ceph-mon[35356]: osd.0 [v2:172.21.15.12:6802/1237101991,v1:172.21.15.12:6803/1237101991] boot 2024-09-10T23:58:19.236 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:58:18 smithi165 ceph-mon[35356]: osdmap e8: 1 total, 1 up, 1 in 2024-09-10T23:58:19.236 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:58:18 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-10T23:58:19.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:58:18 smithi012 ceph-mon[30333]: purged_snaps scrub starts 2024-09-10T23:58:19.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:58:18 smithi012 ceph-mon[30333]: purged_snaps scrub ok 2024-09-10T23:58:19.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:58:18 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-10T23:58:19.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:58:18 smithi012 ceph-mon[30333]: pgmap v57: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T23:58:19.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:58:18 smithi012 ceph-mon[30333]: osd.0 [v2:172.21.15.12:6802/1237101991,v1:172.21.15.12:6803/1237101991] boot 2024-09-10T23:58:19.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:58:18 smithi012 ceph-mon[30333]: osdmap e8: 1 total, 1 up, 1 in 2024-09-10T23:58:19.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:58:18 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-10T23:58:19.330 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:58:18 smithi178 ceph-mon[35919]: purged_snaps scrub starts 2024-09-10T23:58:19.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:58:18 smithi178 ceph-mon[35919]: purged_snaps scrub ok 2024-09-10T23:58:19.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:58:18 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-10T23:58:19.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:58:18 smithi178 ceph-mon[35919]: pgmap v57: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T23:58:19.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:58:18 smithi178 ceph-mon[35919]: osd.0 [v2:172.21.15.12:6802/1237101991,v1:172.21.15.12:6803/1237101991] boot 2024-09-10T23:58:19.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:58:18 smithi178 ceph-mon[35919]: osdmap e8: 1 total, 1 up, 1 in 2024-09-10T23:58:19.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:58:18 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-10T23:58:21.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:58:20 smithi165 ceph-mon[35356]: osdmap e9: 1 total, 1 up, 1 in 2024-09-10T23:58:21.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:58:20 smithi165 ceph-mon[35356]: pgmap v60: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-10T23:58:21.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:58:20 smithi165 ceph-mon[35356]: from='client.? 172.21.15.165:0/1735122774' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "fe0e1d0a-bdc5-4859-aa84-9038059a45f6"} : dispatch 2024-09-10T23:58:21.283 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:58:20 smithi165 ceph-mon[35356]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "fe0e1d0a-bdc5-4859-aa84-9038059a45f6"} : dispatch 2024-09-10T23:58:21.283 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:58:20 smithi165 ceph-mon[35356]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "fe0e1d0a-bdc5-4859-aa84-9038059a45f6"}]': finished 2024-09-10T23:58:21.283 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:58:20 smithi165 ceph-mon[35356]: osdmap e10: 2 total, 1 up, 2 in 2024-09-10T23:58:21.283 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:58:20 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-10T23:58:21.283 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:58:20 smithi165 ceph-mon[35356]: from='client.? 172.21.15.165:0/1464140037' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-10T23:58:21.312 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:58:20 smithi012 ceph-mon[30333]: osdmap e9: 1 total, 1 up, 1 in 2024-09-10T23:58:21.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:58:20 smithi012 ceph-mon[30333]: pgmap v60: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-10T23:58:21.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:58:20 smithi012 ceph-mon[30333]: from='client.? 172.21.15.165:0/1735122774' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "fe0e1d0a-bdc5-4859-aa84-9038059a45f6"} : dispatch 2024-09-10T23:58:21.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:58:20 smithi012 ceph-mon[30333]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "fe0e1d0a-bdc5-4859-aa84-9038059a45f6"} : dispatch 2024-09-10T23:58:21.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:58:20 smithi012 ceph-mon[30333]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "fe0e1d0a-bdc5-4859-aa84-9038059a45f6"}]': finished 2024-09-10T23:58:21.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:58:20 smithi012 ceph-mon[30333]: osdmap e10: 2 total, 1 up, 2 in 2024-09-10T23:58:21.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:58:20 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-10T23:58:21.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:58:20 smithi012 ceph-mon[30333]: from='client.? 172.21.15.165:0/1464140037' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-10T23:58:21.330 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:58:20 smithi178 ceph-mon[35919]: osdmap e9: 1 total, 1 up, 1 in 2024-09-10T23:58:21.330 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:58:20 smithi178 ceph-mon[35919]: pgmap v60: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-10T23:58:21.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:58:20 smithi178 ceph-mon[35919]: from='client.? 172.21.15.165:0/1735122774' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "fe0e1d0a-bdc5-4859-aa84-9038059a45f6"} : dispatch 2024-09-10T23:58:21.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:58:20 smithi178 ceph-mon[35919]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "fe0e1d0a-bdc5-4859-aa84-9038059a45f6"} : dispatch 2024-09-10T23:58:21.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:58:20 smithi178 ceph-mon[35919]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "fe0e1d0a-bdc5-4859-aa84-9038059a45f6"}]': finished 2024-09-10T23:58:21.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:58:20 smithi178 ceph-mon[35919]: osdmap e10: 2 total, 1 up, 2 in 2024-09-10T23:58:21.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:58:20 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-10T23:58:21.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:58:20 smithi178 ceph-mon[35919]: from='client.? 172.21.15.165:0/1464140037' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-10T23:58:23.330 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:58:23 smithi178 ceph-mon[35919]: pgmap v62: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-10T23:58:23.532 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:58:23 smithi165 ceph-mon[35356]: pgmap v62: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-10T23:58:23.563 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:58:23 smithi012 ceph-mon[30333]: pgmap v62: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-10T23:58:25.501 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:58:25 smithi165 ceph-mon[35356]: pgmap v63: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-10T23:58:25.563 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:58:25 smithi012 ceph-mon[30333]: pgmap v63: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-10T23:58:25.583 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:58:25 smithi178 ceph-mon[35919]: pgmap v63: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-10T23:58:27.214 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:58:27 smithi165 ceph-mon[35356]: pgmap v64: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-10T23:58:27.563 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:58:27 smithi012 ceph-mon[30333]: pgmap v64: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-10T23:58:27.580 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:58:27 smithi178 ceph-mon[35919]: pgmap v64: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-10T23:58:29.139 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:58:29 smithi165 ceph-mon[35356]: pgmap v65: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-10T23:58:29.139 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:58:29 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.1"} : dispatch 2024-09-10T23:58:29.139 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:58:29 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T23:58:29.140 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:58:29 smithi165 ceph-mon[35356]: Deploying daemon osd.1 on smithi165 2024-09-10T23:58:29.563 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:58:29 smithi012 ceph-mon[30333]: pgmap v65: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-10T23:58:29.563 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:58:29 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.1"} : dispatch 2024-09-10T23:58:29.563 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:58:29 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T23:58:29.563 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:58:29 smithi012 ceph-mon[30333]: Deploying daemon osd.1 on smithi165 2024-09-10T23:58:29.581 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:58:29 smithi178 ceph-mon[35919]: pgmap v65: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-10T23:58:29.581 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:58:29 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.1"} : dispatch 2024-09-10T23:58:29.581 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:58:29 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T23:58:29.581 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:58:29 smithi178 ceph-mon[35919]: Deploying daemon osd.1 on smithi165 2024-09-10T23:58:31.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:58:31 smithi012 ceph-mon[30333]: pgmap v66: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-10T23:58:31.379 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:58:31 smithi165 ceph-mon[35356]: pgmap v66: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-10T23:58:31.581 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:58:31 smithi178 ceph-mon[35919]: pgmap v66: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-10T23:58:33.452 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:58:33 smithi165 ceph-mon[35356]: pgmap v67: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-10T23:58:33.562 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:58:33 smithi012 ceph-mon[30333]: pgmap v67: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-10T23:58:33.580 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:58:33 smithi178 ceph-mon[35919]: pgmap v67: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-10T23:58:35.319 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:58:35 smithi165 ceph-mon[35356]: pgmap v68: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-10T23:58:35.319 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:58:35 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T23:58:35.319 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:58:35 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-10T23:58:35.319 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:58:35 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-10T23:58:35.365 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:58:35 smithi012 ceph-mon[30333]: pgmap v68: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-10T23:58:35.365 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:58:35 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T23:58:35.365 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:58:35 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-10T23:58:35.365 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:58:35 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-10T23:58:35.383 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:58:35 smithi178 ceph-mon[35919]: pgmap v68: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-10T23:58:35.383 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:58:35 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T23:58:35.383 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:58:35 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-10T23:58:35.383 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:58:35 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-10T23:58:37.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:58:37 smithi165 ceph-mon[35356]: pgmap v69: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-10T23:58:37.563 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:58:37 smithi012 ceph-mon[30333]: pgmap v69: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-10T23:58:37.580 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:58:37 smithi178 ceph-mon[35919]: pgmap v69: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-10T23:58:38.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:58:38 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-10T23:58:38.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:58:38 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-10T23:58:38.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:58:38 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T23:58:38.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:58:38 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T23:58:38.283 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:58:38 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-10T23:58:38.563 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:58:38 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-10T23:58:38.563 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:58:38 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-10T23:58:38.563 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:58:38 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T23:58:38.563 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:58:38 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T23:58:38.563 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:58:38 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-10T23:58:38.581 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:58:38 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-10T23:58:38.581 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:58:38 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-10T23:58:38.581 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:58:38 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T23:58:38.581 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:58:38 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T23:58:38.581 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:58:38 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-10T23:58:38.722 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-10T23:58:38.720+0000 7f6681ffb640 1 -- 172.21.15.165:0/1575295235 <== mgr.14150 v2:172.21.15.12:6800/3442939137 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f668c063650 con 0x7f66740b52d0 2024-09-10T23:58:38.722 INFO:teuthology.orchestra.run.smithi165.stdout:Created osd(s) 1 on host 'smithi165' 2024-09-10T23:58:38.725 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-10T23:58:38.723+0000 7f6692a9b640 1 -- 172.21.15.165:0/1575295235 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f66740b52d0 msgr2=0x7f66740b7790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:58:38.725 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-10T23:58:38.723+0000 7f6692a9b640 1 --2- 172.21.15.165:0/1575295235 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f66740b52d0 0x7f66740b7790 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f66880058c0 tx=0x7f6688005ec0 comp rx=0 tx=0).stop 2024-09-10T23:58:38.725 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-10T23:58:38.723+0000 7f6692a9b640 1 -- 172.21.15.165:0/1575295235 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f668c101930 msgr2=0x7f668c197d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:58:38.725 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-10T23:58:38.723+0000 7f6692a9b640 1 --2- 172.21.15.165:0/1575295235 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f668c101930 0x7f668c197d90 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f667c005ab0 tx=0x7f667c0042f0 comp rx=0 tx=0).stop 2024-09-10T23:58:38.725 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-10T23:58:38.723+0000 7f6692a9b640 1 -- 172.21.15.165:0/1575295235 shutdown_connections 2024-09-10T23:58:38.725 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-10T23:58:38.723+0000 7f6692a9b640 1 --2- 172.21.15.165:0/1575295235 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f66740b52d0 0x7f66740b7790 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:58:38.725 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-10T23:58:38.723+0000 7f6692a9b640 1 --2- 172.21.15.165:0/1575295235 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f668c103450 0x7f668c199020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:58:38.725 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-10T23:58:38.723+0000 7f6692a9b640 1 --2- 172.21.15.165:0/1575295235 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f668c102a90 0x7f668c1983e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:58:38.726 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-10T23:58:38.723+0000 7f6692a9b640 1 --2- 172.21.15.165:0/1575295235 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f668c101930 0x7f668c197d90 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:58:38.726 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-10T23:58:38.723+0000 7f6692a9b640 1 -- 172.21.15.165:0/1575295235 >> 172.21.15.165:0/1575295235 conn(0x7f668c0fd200 msgr2=0x7f668c105f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:58:38.726 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-10T23:58:38.724+0000 7f6692a9b640 1 -- 172.21.15.165:0/1575295235 shutdown_connections 2024-09-10T23:58:38.726 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-10T23:58:38.724+0000 7f6692a9b640 1 -- 172.21.15.165:0/1575295235 wait complete. 2024-09-10T23:58:39.385 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:58:39 smithi165 ceph-mon[35356]: pgmap v70: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-10T23:58:39.385 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:58:39 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T23:58:39.385 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:58:39 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-10T23:58:39.385 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:58:39 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-10T23:58:39.413 DEBUG:teuthology.orchestra.run.smithi165:osd.1> sudo journalctl -f -n 0 -u ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968@osd.1.service 2024-09-10T23:58:39.415 INFO:tasks.cephadm:Deploying osd.2 on smithi178 with /dev/vg_nvme/lv_4... 2024-09-10T23:58:39.415 DEBUG:teuthology.orchestra.run.smithi178:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ff3ad74a-6fcf-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_4 2024-09-10T23:58:39.442 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:58:39 smithi178 ceph-mon[35919]: pgmap v70: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-10T23:58:39.443 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:58:39 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T23:58:39.443 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:58:39 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-10T23:58:39.443 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:58:39 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-10T23:58:39.563 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:58:39 smithi012 ceph-mon[30333]: pgmap v70: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-10T23:58:39.563 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:58:39 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T23:58:39.563 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:58:39 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-10T23:58:39.563 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:58:39 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-10T23:58:39.672 INFO:teuthology.orchestra.run.smithi178.stderr:Inferring config /var/lib/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/mon.c/config 2024-09-10T23:58:40.737 INFO:journalctl@ceph.osd.1.smithi165.stdout:Sep 10 23:58:40 smithi165 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-1[39824]: 2024-09-10T23:58:40.478+0000 7f7480d8e640 -1 osd.1 0 log_to_monitors true 2024-09-10T23:58:41.203 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:58:41 smithi178 ceph-mon[35919]: pgmap v71: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-10T23:58:41.203 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:58:41 smithi178 ceph-mon[35919]: from='osd.1 [v2:172.21.15.165:6800/545848456,v1:172.21.15.165:6801/545848456]' entity='osd.1' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]} : dispatch 2024-09-10T23:58:41.203 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:58:41 smithi178 ceph-mon[35919]: from='osd.1 ' entity='osd.1' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]} : dispatch 2024-09-10T23:58:41.456 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:58:41 smithi165 ceph-mon[35356]: pgmap v71: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-10T23:58:41.456 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:58:41 smithi165 ceph-mon[35356]: from='osd.1 [v2:172.21.15.165:6800/545848456,v1:172.21.15.165:6801/545848456]' entity='osd.1' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]} : dispatch 2024-09-10T23:58:41.456 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:58:41 smithi165 ceph-mon[35356]: from='osd.1 ' entity='osd.1' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]} : dispatch 2024-09-10T23:58:41.562 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:58:41 smithi012 ceph-mon[30333]: pgmap v71: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-10T23:58:41.563 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:58:41 smithi012 ceph-mon[30333]: from='osd.1 [v2:172.21.15.165:6800/545848456,v1:172.21.15.165:6801/545848456]' entity='osd.1' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]} : dispatch 2024-09-10T23:58:41.563 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:58:41 smithi012 ceph-mon[30333]: from='osd.1 ' entity='osd.1' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]} : dispatch 2024-09-10T23:58:42.307 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:58:42 smithi178 ceph-mon[35919]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-10T23:58:42.308 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:58:42 smithi178 ceph-mon[35919]: osdmap e11: 2 total, 1 up, 2 in 2024-09-10T23:58:42.308 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:58:42 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-10T23:58:42.308 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:58:42 smithi178 ceph-mon[35919]: from='osd.1 [v2:172.21.15.165:6800/545848456,v1:172.21.15.165:6801/545848456]' entity='osd.1' cmd={"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi165", "root=default"]} : dispatch 2024-09-10T23:58:42.308 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:58:42 smithi178 ceph-mon[35919]: from='osd.1 ' entity='osd.1' cmd={"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi165", "root=default"]} : dispatch 2024-09-10T23:58:42.310 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-09-10T23:58:42.341 DEBUG:teuthology.orchestra.run.smithi178:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ff3ad74a-6fcf-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi178:vg_nvme/lv_4 2024-09-10T23:58:42.532 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:58:42 smithi165 ceph-mon[35356]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-10T23:58:42.532 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:58:42 smithi165 ceph-mon[35356]: osdmap e11: 2 total, 1 up, 2 in 2024-09-10T23:58:42.532 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:58:42 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-10T23:58:42.532 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:58:42 smithi165 ceph-mon[35356]: from='osd.1 [v2:172.21.15.165:6800/545848456,v1:172.21.15.165:6801/545848456]' entity='osd.1' cmd={"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi165", "root=default"]} : dispatch 2024-09-10T23:58:42.532 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:58:42 smithi165 ceph-mon[35356]: from='osd.1 ' entity='osd.1' cmd={"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi165", "root=default"]} : dispatch 2024-09-10T23:58:42.562 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:58:42 smithi012 ceph-mon[30333]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-10T23:58:42.563 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:58:42 smithi012 ceph-mon[30333]: osdmap e11: 2 total, 1 up, 2 in 2024-09-10T23:58:42.563 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:58:42 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-10T23:58:42.563 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:58:42 smithi012 ceph-mon[30333]: from='osd.1 [v2:172.21.15.165:6800/545848456,v1:172.21.15.165:6801/545848456]' entity='osd.1' cmd={"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi165", "root=default"]} : dispatch 2024-09-10T23:58:42.563 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:58:42 smithi012 ceph-mon[30333]: from='osd.1 ' entity='osd.1' cmd={"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi165", "root=default"]} : dispatch 2024-09-10T23:58:42.638 INFO:teuthology.orchestra.run.smithi178.stderr:Inferring config /var/lib/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/mon.c/config 2024-09-10T23:58:43.115 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T23:58:43.112+0000 7f9695f92640 1 -- 172.21.15.178:0/2757342558 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9690101910 msgr2=0x7f9690101d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:58:43.115 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T23:58:43.112+0000 7f9695f92640 1 --2- 172.21.15.178:0/2757342558 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9690101910 0x7f9690101d10 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f9684009960 tx=0x7f968402a460 comp rx=0 tx=0).stop 2024-09-10T23:58:43.115 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T23:58:43.112+0000 7f9695f92640 1 -- 172.21.15.178:0/2757342558 shutdown_connections 2024-09-10T23:58:43.115 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T23:58:43.112+0000 7f9695f92640 1 --2- 172.21.15.178:0/2757342558 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9690103430 0x7f9690109db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:58:43.115 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T23:58:43.112+0000 7f9695f92640 1 --2- 172.21.15.178:0/2757342558 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9690102a70 0x7f9690102ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:58:43.115 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T23:58:43.112+0000 7f9695f92640 1 --2- 172.21.15.178:0/2757342558 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9690101910 0x7f9690101d10 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:58:43.115 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T23:58:43.112+0000 7f9695f92640 1 -- 172.21.15.178:0/2757342558 >> 172.21.15.178:0/2757342558 conn(0x7f96900fd1e0 msgr2=0x7f96900ff620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:58:43.115 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T23:58:43.112+0000 7f9695f92640 1 -- 172.21.15.178:0/2757342558 shutdown_connections 2024-09-10T23:58:43.116 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T23:58:43.112+0000 7f9695f92640 1 -- 172.21.15.178:0/2757342558 wait complete. 2024-09-10T23:58:43.116 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T23:58:43.112+0000 7f9695f92640 1 Processor -- start 2024-09-10T23:58:43.116 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T23:58:43.112+0000 7f9695f92640 1 -- start start 2024-09-10T23:58:43.116 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T23:58:43.113+0000 7f9695f92640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9690101910 0x7f96901056d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:58:43.116 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T23:58:43.113+0000 7f9695f92640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9690102a70 0x7f9690103f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:58:43.116 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T23:58:43.113+0000 7f9695f92640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9690103430 0x7f9690104480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:58:43.116 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T23:58:43.113+0000 7f9695f92640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f969010d5f0 con 0x7f9690102a70 2024-09-10T23:58:43.116 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T23:58:43.113+0000 7f9695f92640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f969010d470 con 0x7f9690101910 2024-09-10T23:58:43.116 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T23:58:43.113+0000 7f9695f92640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f969010d770 con 0x7f9690103430 2024-09-10T23:58:43.116 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T23:58:43.113+0000 7f968ffff640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9690103430 0x7f9690104480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:58:43.116 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T23:58:43.113+0000 7f968ffff640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9690103430 0x7f9690104480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:60782/0 (socket says 172.21.15.178:60782) 2024-09-10T23:58:43.116 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T23:58:43.113+0000 7f968ffff640 1 -- 172.21.15.178:0/3592420451 learned_addr learned my addr 172.21.15.178:0/3592420451 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-10T23:58:43.117 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T23:58:43.113+0000 7f968effd640 1 --2- 172.21.15.178:0/3592420451 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9690102a70 0x7f9690103f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:58:43.117 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T23:58:43.113+0000 7f968f7fe640 1 --2- 172.21.15.178:0/3592420451 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9690101910 0x7f96901056d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:58:43.117 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T23:58:43.113+0000 7f968ffff640 1 -- 172.21.15.178:0/3592420451 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9690101910 msgr2=0x7f96901056d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:58:43.117 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T23:58:43.113+0000 7f968ffff640 1 --2- 172.21.15.178:0/3592420451 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9690101910 0x7f96901056d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:58:43.117 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T23:58:43.113+0000 7f968ffff640 1 -- 172.21.15.178:0/3592420451 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9690102a70 msgr2=0x7f9690103f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:58:43.117 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T23:58:43.113+0000 7f968ffff640 1 --2- 172.21.15.178:0/3592420451 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9690102a70 0x7f9690103f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:58:43.117 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T23:58:43.113+0000 7f968ffff640 1 -- 172.21.15.178:0/3592420451 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9690104d40 con 0x7f9690103430 2024-09-10T23:58:43.117 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T23:58:43.113+0000 7f968ffff640 1 --2- 172.21.15.178:0/3592420451 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9690103430 0x7f9690104480 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f968800b970 tx=0x7f968800bd70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:58:43.117 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T23:58:43.114+0000 7f968cff9640 1 -- 172.21.15.178:0/3592420451 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9688019070 con 0x7f9690103430 2024-09-10T23:58:43.117 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T23:58:43.114+0000 7f968cff9640 1 -- 172.21.15.178:0/3592420451 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9688014440 con 0x7f9690103430 2024-09-10T23:58:43.117 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T23:58:43.114+0000 7f9695f92640 1 -- 172.21.15.178:0/3592420451 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96901a2f20 con 0x7f9690103430 2024-09-10T23:58:43.118 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T23:58:43.114+0000 7f968cff9640 1 -- 172.21.15.178:0/3592420451 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96880075d0 con 0x7f9690103430 2024-09-10T23:58:43.118 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T23:58:43.114+0000 7f9695f92640 1 -- 172.21.15.178:0/3592420451 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96901a3400 con 0x7f9690103430 2024-09-10T23:58:43.118 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T23:58:43.115+0000 7f9695f92640 1 -- 172.21.15.178:0/3592420451 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f96901a36f0 con 0x7f9690103430 2024-09-10T23:58:43.118 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T23:58:43.116+0000 7f968cff9640 1 -- 172.21.15.178:0/3592420451 <== mon.1 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 15) ==== 153194+0+0 (secure 0 0 0) 0x7f96880145f0 con 0x7f9690103430 2024-09-10T23:58:43.119 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T23:58:43.117+0000 7f968cff9640 1 --2- 172.21.15.178:0/3592420451 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f96780b5250 0x7f96780b7710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:58:43.119 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T23:58:43.117+0000 7f968cff9640 1 -- 172.21.15.178:0/3592420451 <== mon.1 v2:172.21.15.178:3300/0 5 ==== osd_map(12..12 src has 1..12) ==== 2029+0+0 (secure 0 0 0) 0x7f96880e3900 con 0x7f9690103430 2024-09-10T23:58:43.119 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T23:58:43.117+0000 7f968f7fe640 1 --2- 172.21.15.178:0/3592420451 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f96780b5250 0x7f96780b7710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:58:43.119 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T23:58:43.118+0000 7f968f7fe640 1 --2- 172.21.15.178:0/3592420451 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f96780b5250 0x7f96780b7710 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f9684006010 tx=0x7f9684005d50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:58:43.122 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T23:58:43.121+0000 7f968cff9640 1 -- 172.21.15.178:0/3592420451 <== mon.1 v2:172.21.15.178:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f96880abae0 con 0x7f9690103430 2024-09-10T23:58:43.142 INFO:journalctl@ceph.osd.1.smithi165.stdout:Sep 10 23:58:42 smithi165 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-1[39824]: 2024-09-10T23:58:42.814+0000 7f747cd11640 -1 osd.1 0 waiting for initial osdmap 2024-09-10T23:58:43.142 INFO:journalctl@ceph.osd.1.smithi165.stdout:Sep 10 23:58:42 smithi165 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-1[39824]: 2024-09-10T23:58:42.818+0000 7f7474b30640 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-10T23:58:43.314 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T23:58:43.313+0000 7f9695f92640 1 -- 172.21.15.178:0/3592420451 --> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi178:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f96900ff340 con 0x7f96780b5250 2024-09-10T23:58:43.315 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:58:43 smithi178 ceph-mon[35919]: pgmap v73: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-10T23:58:43.315 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:58:43 smithi178 ceph-mon[35919]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi165", "root=default"]}]': finished 2024-09-10T23:58:43.315 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:58:43 smithi178 ceph-mon[35919]: osdmap e12: 2 total, 1 up, 2 in 2024-09-10T23:58:43.315 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:58:43 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-10T23:58:43.315 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:58:43 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-10T23:58:43.316 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:58:43 smithi178 ceph-mon[35919]: Detected new or changed devices on smithi165 2024-09-10T23:58:43.316 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:58:43 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-10T23:58:43.316 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:58:43 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-10T23:58:43.316 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:58:43 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-10T23:58:43.316 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:58:43 smithi178 ceph-mon[35919]: Adjusting osd_memory_target on smithi165 to 17088M 2024-09-10T23:58:43.316 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:58:43 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-10T23:58:43.316 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:58:43 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T23:58:43.316 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:58:43 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T23:58:43.316 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:58:43 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-10T23:58:43.316 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:58:43 smithi178 ceph-mon[35919]: from='osd.1 ' entity='osd.1' 2024-09-10T23:58:43.316 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:58:43 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-10T23:58:43.532 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:58:43 smithi165 ceph-mon[35356]: pgmap v73: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-10T23:58:43.532 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:58:43 smithi165 ceph-mon[35356]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi165", "root=default"]}]': finished 2024-09-10T23:58:43.533 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:58:43 smithi165 ceph-mon[35356]: osdmap e12: 2 total, 1 up, 2 in 2024-09-10T23:58:43.533 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:58:43 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-10T23:58:43.533 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:58:43 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-10T23:58:43.533 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:58:43 smithi165 ceph-mon[35356]: Detected new or changed devices on smithi165 2024-09-10T23:58:43.533 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:58:43 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-10T23:58:43.533 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:58:43 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-10T23:58:43.533 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:58:43 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-10T23:58:43.533 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:58:43 smithi165 ceph-mon[35356]: Adjusting osd_memory_target on smithi165 to 17088M 2024-09-10T23:58:43.533 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:58:43 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-10T23:58:43.533 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:58:43 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T23:58:43.533 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:58:43 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T23:58:43.533 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:58:43 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-10T23:58:43.534 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:58:43 smithi165 ceph-mon[35356]: from='osd.1 ' entity='osd.1' 2024-09-10T23:58:43.534 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:58:43 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-10T23:58:43.563 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:58:43 smithi012 ceph-mon[30333]: pgmap v73: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-10T23:58:43.563 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:58:43 smithi012 ceph-mon[30333]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi165", "root=default"]}]': finished 2024-09-10T23:58:43.563 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:58:43 smithi012 ceph-mon[30333]: osdmap e12: 2 total, 1 up, 2 in 2024-09-10T23:58:43.563 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:58:43 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-10T23:58:43.563 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:58:43 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-10T23:58:43.563 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:58:43 smithi012 ceph-mon[30333]: Detected new or changed devices on smithi165 2024-09-10T23:58:43.564 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:58:43 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-10T23:58:43.564 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:58:43 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-10T23:58:43.564 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:58:43 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-10T23:58:43.564 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:58:43 smithi012 ceph-mon[30333]: Adjusting osd_memory_target on smithi165 to 17088M 2024-09-10T23:58:43.564 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:58:43 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-10T23:58:43.564 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:58:43 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T23:58:43.564 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:58:43 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T23:58:43.564 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:58:43 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-10T23:58:43.564 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:58:43 smithi012 ceph-mon[30333]: from='osd.1 ' entity='osd.1' 2024-09-10T23:58:43.564 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:58:43 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-10T23:58:44.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:58:44 smithi178 ceph-mon[35919]: purged_snaps scrub starts 2024-09-10T23:58:44.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:58:44 smithi178 ceph-mon[35919]: purged_snaps scrub ok 2024-09-10T23:58:44.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:58:44 smithi178 ceph-mon[35919]: from='client.24286 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi178:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T23:58:44.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:58:44 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-10T23:58:44.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:58:44 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-10T23:58:44.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:58:44 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T23:58:44.332 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:58:44 smithi178 ceph-mon[35919]: osd.1 [v2:172.21.15.165:6800/545848456,v1:172.21.15.165:6801/545848456] boot 2024-09-10T23:58:44.332 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:58:44 smithi178 ceph-mon[35919]: osdmap e13: 2 total, 2 up, 2 in 2024-09-10T23:58:44.332 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:58:44 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-10T23:58:44.532 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:58:44 smithi165 ceph-mon[35356]: purged_snaps scrub starts 2024-09-10T23:58:44.532 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:58:44 smithi165 ceph-mon[35356]: purged_snaps scrub ok 2024-09-10T23:58:44.532 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:58:44 smithi165 ceph-mon[35356]: from='client.24286 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi178:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T23:58:44.532 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:58:44 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-10T23:58:44.532 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:58:44 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-10T23:58:44.533 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:58:44 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T23:58:44.533 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:58:44 smithi165 ceph-mon[35356]: osd.1 [v2:172.21.15.165:6800/545848456,v1:172.21.15.165:6801/545848456] boot 2024-09-10T23:58:44.533 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:58:44 smithi165 ceph-mon[35356]: osdmap e13: 2 total, 2 up, 2 in 2024-09-10T23:58:44.533 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:58:44 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-10T23:58:44.562 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:58:44 smithi012 ceph-mon[30333]: purged_snaps scrub starts 2024-09-10T23:58:44.563 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:58:44 smithi012 ceph-mon[30333]: purged_snaps scrub ok 2024-09-10T23:58:44.563 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:58:44 smithi012 ceph-mon[30333]: from='client.24286 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi178:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T23:58:44.563 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:58:44 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-10T23:58:44.563 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:58:44 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-10T23:58:44.563 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:58:44 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T23:58:44.563 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:58:44 smithi012 ceph-mon[30333]: osd.1 [v2:172.21.15.165:6800/545848456,v1:172.21.15.165:6801/545848456] boot 2024-09-10T23:58:44.563 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:58:44 smithi012 ceph-mon[30333]: osdmap e13: 2 total, 2 up, 2 in 2024-09-10T23:58:44.563 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:58:44 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-10T23:58:46.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:58:45 smithi178 ceph-mon[35919]: pgmap v76: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-10T23:58:46.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:58:45 smithi178 ceph-mon[35919]: osdmap e14: 2 total, 2 up, 2 in 2024-09-10T23:58:46.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:58:45 smithi178 ceph-mon[35919]: from='client.? 172.21.15.178:0/3840400457' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "94f640dc-5f78-4e2c-87d9-03e96a6e7ba0"} : dispatch 2024-09-10T23:58:46.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:58:45 smithi178 ceph-mon[35919]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "94f640dc-5f78-4e2c-87d9-03e96a6e7ba0"} : dispatch 2024-09-10T23:58:46.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:58:45 smithi178 ceph-mon[35919]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "94f640dc-5f78-4e2c-87d9-03e96a6e7ba0"}]': finished 2024-09-10T23:58:46.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:58:45 smithi178 ceph-mon[35919]: osdmap e15: 3 total, 2 up, 3 in 2024-09-10T23:58:46.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:58:45 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-10T23:58:46.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:58:45 smithi165 ceph-mon[35356]: pgmap v76: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-10T23:58:46.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:58:45 smithi165 ceph-mon[35356]: osdmap e14: 2 total, 2 up, 2 in 2024-09-10T23:58:46.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:58:45 smithi165 ceph-mon[35356]: from='client.? 172.21.15.178:0/3840400457' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "94f640dc-5f78-4e2c-87d9-03e96a6e7ba0"} : dispatch 2024-09-10T23:58:46.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:58:45 smithi165 ceph-mon[35356]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "94f640dc-5f78-4e2c-87d9-03e96a6e7ba0"} : dispatch 2024-09-10T23:58:46.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:58:45 smithi165 ceph-mon[35356]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "94f640dc-5f78-4e2c-87d9-03e96a6e7ba0"}]': finished 2024-09-10T23:58:46.283 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:58:45 smithi165 ceph-mon[35356]: osdmap e15: 3 total, 2 up, 3 in 2024-09-10T23:58:46.283 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:58:45 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-10T23:58:46.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:58:45 smithi012 ceph-mon[30333]: pgmap v76: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-10T23:58:46.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:58:45 smithi012 ceph-mon[30333]: osdmap e14: 2 total, 2 up, 2 in 2024-09-10T23:58:46.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:58:45 smithi012 ceph-mon[30333]: from='client.? 172.21.15.178:0/3840400457' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "94f640dc-5f78-4e2c-87d9-03e96a6e7ba0"} : dispatch 2024-09-10T23:58:46.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:58:45 smithi012 ceph-mon[30333]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "94f640dc-5f78-4e2c-87d9-03e96a6e7ba0"} : dispatch 2024-09-10T23:58:46.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:58:45 smithi012 ceph-mon[30333]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "94f640dc-5f78-4e2c-87d9-03e96a6e7ba0"}]': finished 2024-09-10T23:58:46.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:58:45 smithi012 ceph-mon[30333]: osdmap e15: 3 total, 2 up, 3 in 2024-09-10T23:58:46.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:58:45 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-10T23:58:47.080 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:58:46 smithi178 ceph-mon[35919]: from='client.? 172.21.15.178:0/199055106' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-10T23:58:47.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:58:46 smithi165 ceph-mon[35356]: from='client.? 172.21.15.178:0/199055106' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-10T23:58:47.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:58:46 smithi012 ceph-mon[30333]: from='client.? 172.21.15.178:0/199055106' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-10T23:58:48.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:58:47 smithi165 ceph-mon[35356]: pgmap v79: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-10T23:58:48.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:58:47 smithi012 ceph-mon[30333]: pgmap v79: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-10T23:58:48.330 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:58:47 smithi178 ceph-mon[35919]: pgmap v79: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-10T23:58:50.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:58:49 smithi165 ceph-mon[35356]: pgmap v80: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-10T23:58:50.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:58:49 smithi012 ceph-mon[30333]: pgmap v80: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-10T23:58:50.330 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:58:49 smithi178 ceph-mon[35919]: pgmap v80: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-10T23:58:52.059 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:58:51 smithi178 ceph-mon[35919]: pgmap v81: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-10T23:58:52.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:58:51 smithi165 ceph-mon[35356]: pgmap v81: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-10T23:58:52.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:58:51 smithi012 ceph-mon[30333]: pgmap v81: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-10T23:58:54.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:58:53 smithi165 ceph-mon[35356]: pgmap v82: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-10T23:58:54.294 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:58:53 smithi178 ceph-mon[35919]: pgmap v82: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-10T23:58:54.312 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:58:53 smithi012 ceph-mon[30333]: pgmap v82: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-10T23:58:55.233 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:58:54 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.2"} : dispatch 2024-09-10T23:58:55.234 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:58:54 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T23:58:55.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:58:54 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.2"} : dispatch 2024-09-10T23:58:55.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:58:54 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T23:58:55.312 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:58:54 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.2"} : dispatch 2024-09-10T23:58:55.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:58:54 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T23:58:56.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:58:55 smithi178 ceph-mon[35919]: Deploying daemon osd.2 on smithi178 2024-09-10T23:58:56.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:58:55 smithi178 ceph-mon[35919]: pgmap v83: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-10T23:58:56.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:58:55 smithi165 ceph-mon[35356]: Deploying daemon osd.2 on smithi178 2024-09-10T23:58:56.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:58:55 smithi165 ceph-mon[35356]: pgmap v83: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-10T23:58:56.312 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:58:55 smithi012 ceph-mon[30333]: Deploying daemon osd.2 on smithi178 2024-09-10T23:58:56.312 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:58:55 smithi012 ceph-mon[30333]: pgmap v83: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-10T23:58:58.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:58:57 smithi165 ceph-mon[35356]: pgmap v84: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-10T23:58:58.312 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:58:57 smithi012 ceph-mon[30333]: pgmap v84: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-10T23:58:58.330 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:58:57 smithi178 ceph-mon[35919]: pgmap v84: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-10T23:59:00.053 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:58:59 smithi178 ceph-mon[35919]: pgmap v85: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-10T23:59:00.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:58:59 smithi165 ceph-mon[35356]: pgmap v85: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-10T23:59:00.312 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:58:59 smithi012 ceph-mon[30333]: pgmap v85: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-10T23:59:01.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:59:00 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T23:59:01.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:59:00 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-10T23:59:01.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:59:00 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-10T23:59:01.312 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:59:00 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T23:59:01.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:59:00 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-10T23:59:01.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:59:00 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-10T23:59:01.330 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:59:00 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T23:59:01.330 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:59:00 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-10T23:59:01.330 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:59:00 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-10T23:59:02.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:59:01 smithi165 ceph-mon[35356]: pgmap v86: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-10T23:59:02.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:59:01 smithi012 ceph-mon[30333]: pgmap v86: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-10T23:59:02.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:59:01 smithi178 ceph-mon[35919]: pgmap v86: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-10T23:59:03.831 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:59:03 smithi178 ceph-mon[35919]: pgmap v87: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-10T23:59:03.831 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:59:03 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-10T23:59:03.831 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:59:03 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-10T23:59:03.831 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:59:03 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T23:59:03.831 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:59:03 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T23:59:03.831 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:59:03 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-10T23:59:04.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:59:03 smithi165 ceph-mon[35356]: pgmap v87: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-10T23:59:04.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:59:03 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-10T23:59:04.033 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:59:03 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-10T23:59:04.033 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:59:03 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T23:59:04.033 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:59:03 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T23:59:04.033 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:59:03 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-10T23:59:04.062 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:59:03 smithi012 ceph-mon[30333]: pgmap v87: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-10T23:59:04.063 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:59:03 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-10T23:59:04.063 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:59:03 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-10T23:59:04.063 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:59:03 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T23:59:04.063 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:59:03 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T23:59:04.063 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:59:03 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-10T23:59:04.175 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T23:59:04.173+0000 7f968cff9640 1 -- 172.21.15.178:0/3592420451 <== mgr.14150 v2:172.21.15.12:6800/3442939137 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f96900ff340 con 0x7f96780b5250 2024-09-10T23:59:04.175 INFO:teuthology.orchestra.run.smithi178.stdout:Created osd(s) 2 on host 'smithi178' 2024-09-10T23:59:04.177 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T23:59:04.176+0000 7f9695f92640 1 -- 172.21.15.178:0/3592420451 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f96780b5250 msgr2=0x7f96780b7710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:59:04.177 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T23:59:04.176+0000 7f9695f92640 1 --2- 172.21.15.178:0/3592420451 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f96780b5250 0x7f96780b7710 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f9684006010 tx=0x7f9684005d50 comp rx=0 tx=0).stop 2024-09-10T23:59:04.177 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T23:59:04.176+0000 7f9695f92640 1 -- 172.21.15.178:0/3592420451 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9690103430 msgr2=0x7f9690104480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:59:04.178 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T23:59:04.176+0000 7f9695f92640 1 --2- 172.21.15.178:0/3592420451 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9690103430 0x7f9690104480 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f968800b970 tx=0x7f968800bd70 comp rx=0 tx=0).stop 2024-09-10T23:59:04.178 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T23:59:04.176+0000 7f9695f92640 1 -- 172.21.15.178:0/3592420451 shutdown_connections 2024-09-10T23:59:04.178 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T23:59:04.176+0000 7f9695f92640 1 --2- 172.21.15.178:0/3592420451 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f96780b5250 0x7f96780b7710 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:59:04.178 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T23:59:04.176+0000 7f9695f92640 1 --2- 172.21.15.178:0/3592420451 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9690103430 0x7f9690104480 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:59:04.178 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T23:59:04.176+0000 7f9695f92640 1 --2- 172.21.15.178:0/3592420451 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9690102a70 0x7f9690103f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:59:04.178 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T23:59:04.176+0000 7f9695f92640 1 --2- 172.21.15.178:0/3592420451 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9690101910 0x7f96901056d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:59:04.178 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T23:59:04.176+0000 7f9695f92640 1 -- 172.21.15.178:0/3592420451 >> 172.21.15.178:0/3592420451 conn(0x7f96900fd1e0 msgr2=0x7f96900fec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:59:04.178 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T23:59:04.176+0000 7f9695f92640 1 -- 172.21.15.178:0/3592420451 shutdown_connections 2024-09-10T23:59:04.178 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T23:59:04.176+0000 7f9695f92640 1 -- 172.21.15.178:0/3592420451 wait complete. 2024-09-10T23:59:04.718 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:59:04 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T23:59:04.718 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:59:04 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-10T23:59:04.718 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:59:04 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-10T23:59:04.745 DEBUG:teuthology.orchestra.run.smithi178:osd.2> sudo journalctl -f -n 0 -u ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968@osd.2.service 2024-09-10T23:59:04.747 INFO:tasks.cephadm:Deploying osd.3 on smithi012 with /dev/vg_nvme/lv_3... 2024-09-10T23:59:04.747 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ff3ad74a-6fcf-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_3 2024-09-10T23:59:05.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:59:04 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T23:59:05.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:59:04 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-10T23:59:05.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:59:04 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-10T23:59:05.062 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:59:04 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T23:59:05.062 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:59:04 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-10T23:59:05.063 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:59:04 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-10T23:59:05.128 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T23:59:05.920 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:59:05 smithi178 ceph-mon[35919]: pgmap v88: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-10T23:59:05.921 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:59:05 smithi012 ceph-mon[30333]: pgmap v88: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-10T23:59:05.933 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:59:05 smithi165 ceph-mon[35356]: pgmap v88: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-10T23:59:06.177 INFO:journalctl@ceph.osd.2.smithi178.stdout:Sep 10 23:59:06 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-2[40889]: 2024-09-10T23:59:06.137+0000 7f311a10b640 -1 osd.2 0 log_to_monitors true 2024-09-10T23:59:06.831 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:59:06 smithi178 ceph-mon[35919]: from='osd.2 [v2:172.21.15.178:6800/4131162695,v1:172.21.15.178:6801/4131162695]' entity='osd.2' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]} : dispatch 2024-09-10T23:59:06.831 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:59:06 smithi178 ceph-mon[35919]: from='osd.2 ' entity='osd.2' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]} : dispatch 2024-09-10T23:59:06.898 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:59:06 smithi012 ceph-mon[30333]: from='osd.2 [v2:172.21.15.178:6800/4131162695,v1:172.21.15.178:6801/4131162695]' entity='osd.2' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]} : dispatch 2024-09-10T23:59:06.898 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:59:06 smithi012 ceph-mon[30333]: from='osd.2 ' entity='osd.2' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]} : dispatch 2024-09-10T23:59:07.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:59:06 smithi165 ceph-mon[35356]: from='osd.2 [v2:172.21.15.178:6800/4131162695,v1:172.21.15.178:6801/4131162695]' entity='osd.2' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]} : dispatch 2024-09-10T23:59:07.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:59:06 smithi165 ceph-mon[35356]: from='osd.2 ' entity='osd.2' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]} : dispatch 2024-09-10T23:59:07.831 INFO:journalctl@ceph.osd.2.smithi178.stdout:Sep 10 23:59:07 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-2[40889]: 2024-09-10T23:59:07.627+0000 7f311608e640 -1 osd.2 0 waiting for initial osdmap 2024-09-10T23:59:07.831 INFO:journalctl@ceph.osd.2.smithi178.stdout:Sep 10 23:59:07 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-2[40889]: 2024-09-10T23:59:07.630+0000 7f3111eb5640 -1 osd.2 17 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-10T23:59:07.831 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:59:07 smithi178 ceph-mon[35919]: pgmap v89: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-10T23:59:07.831 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:59:07 smithi178 ceph-mon[35919]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-10T23:59:07.831 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:59:07 smithi178 ceph-mon[35919]: osdmap e16: 3 total, 2 up, 3 in 2024-09-10T23:59:07.831 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:59:07 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-10T23:59:07.831 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:59:07 smithi178 ceph-mon[35919]: from='osd.2 [v2:172.21.15.178:6800/4131162695,v1:172.21.15.178:6801/4131162695]' entity='osd.2' cmd={"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi178", "root=default"]} : dispatch 2024-09-10T23:59:07.832 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:59:07 smithi178 ceph-mon[35919]: from='osd.2 ' entity='osd.2' cmd={"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi178", "root=default"]} : dispatch 2024-09-10T23:59:07.832 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:59:07 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-10T23:59:07.832 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:59:07 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-10T23:59:07.832 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:59:07 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-10T23:59:07.832 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:59:07 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-10T23:59:07.832 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:59:07 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T23:59:07.832 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:59:07 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T23:59:07.832 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:59:07 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-10T23:59:07.843 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:59:07 smithi012 ceph-mon[30333]: pgmap v89: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-10T23:59:07.844 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:59:07 smithi012 ceph-mon[30333]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-10T23:59:07.844 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:59:07 smithi012 ceph-mon[30333]: osdmap e16: 3 total, 2 up, 3 in 2024-09-10T23:59:07.844 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:59:07 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-10T23:59:07.844 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:59:07 smithi012 ceph-mon[30333]: from='osd.2 [v2:172.21.15.178:6800/4131162695,v1:172.21.15.178:6801/4131162695]' entity='osd.2' cmd={"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi178", "root=default"]} : dispatch 2024-09-10T23:59:07.844 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:59:07 smithi012 ceph-mon[30333]: from='osd.2 ' entity='osd.2' cmd={"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi178", "root=default"]} : dispatch 2024-09-10T23:59:07.844 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:59:07 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-10T23:59:07.844 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:59:07 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-10T23:59:07.844 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:59:07 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-10T23:59:07.844 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:59:07 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-10T23:59:07.844 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:59:07 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T23:59:07.845 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:59:07 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T23:59:07.845 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:59:07 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-10T23:59:08.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:59:07 smithi165 ceph-mon[35356]: pgmap v89: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-10T23:59:08.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:59:07 smithi165 ceph-mon[35356]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-10T23:59:08.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:59:07 smithi165 ceph-mon[35356]: osdmap e16: 3 total, 2 up, 3 in 2024-09-10T23:59:08.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:59:07 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-10T23:59:08.033 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:59:07 smithi165 ceph-mon[35356]: from='osd.2 [v2:172.21.15.178:6800/4131162695,v1:172.21.15.178:6801/4131162695]' entity='osd.2' cmd={"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi178", "root=default"]} : dispatch 2024-09-10T23:59:08.033 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:59:07 smithi165 ceph-mon[35356]: from='osd.2 ' entity='osd.2' cmd={"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi178", "root=default"]} : dispatch 2024-09-10T23:59:08.033 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:59:07 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-10T23:59:08.033 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:59:07 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-10T23:59:08.033 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:59:07 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-10T23:59:08.033 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:59:07 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-10T23:59:08.033 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:59:07 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T23:59:08.033 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:59:07 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T23:59:08.033 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:59:07 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-10T23:59:08.158 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-10T23:59:08.189 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ff3ad74a-6fcf-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi012:vg_nvme/lv_3 2024-09-10T23:59:08.558 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T23:59:08.955 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:59:08 smithi012 ceph-mon[30333]: Detected new or changed devices on smithi178 2024-09-10T23:59:08.956 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:59:08 smithi012 ceph-mon[30333]: Adjusting osd_memory_target on smithi178 to 17088M 2024-09-10T23:59:08.956 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:59:08 smithi012 ceph-mon[30333]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi178", "root=default"]}]': finished 2024-09-10T23:59:08.956 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:59:08 smithi012 ceph-mon[30333]: osdmap e17: 3 total, 2 up, 3 in 2024-09-10T23:59:08.956 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:59:08 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-10T23:59:08.956 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:59:08 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-10T23:59:09.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:59:08 smithi165 ceph-mon[35356]: Detected new or changed devices on smithi178 2024-09-10T23:59:09.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:59:08 smithi165 ceph-mon[35356]: Adjusting osd_memory_target on smithi178 to 17088M 2024-09-10T23:59:09.033 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:59:08 smithi165 ceph-mon[35356]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi178", "root=default"]}]': finished 2024-09-10T23:59:09.033 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:59:08 smithi165 ceph-mon[35356]: osdmap e17: 3 total, 2 up, 3 in 2024-09-10T23:59:09.033 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:59:08 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-10T23:59:09.033 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:59:08 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-10T23:59:09.073 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:59:09.070+0000 7f49aa400640 1 -- 172.21.15.12:0/4064984303 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f49a4104aa0 msgr2=0x7f49a4104ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:59:09.073 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:59:09.070+0000 7f49aa400640 1 --2- 172.21.15.12:0/4064984303 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f49a4104aa0 0x7f49a4104ea0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f4998009930 tx=0x7f499802a420 comp rx=0 tx=0).stop 2024-09-10T23:59:09.073 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:59:09.070+0000 7f49aa400640 1 -- 172.21.15.12:0/4064984303 shutdown_connections 2024-09-10T23:59:09.074 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:59:09.070+0000 7f49aa400640 1 --2- 172.21.15.12:0/4064984303 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f49a41065c0 0x7f49a410cf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:59:09.074 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:59:09.070+0000 7f49aa400640 1 --2- 172.21.15.12:0/4064984303 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f49a4105c00 0x7f49a4106080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:59:09.074 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:59:09.070+0000 7f49aa400640 1 --2- 172.21.15.12:0/4064984303 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f49a4104aa0 0x7f49a4104ea0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:59:09.074 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:59:09.070+0000 7f49aa400640 1 -- 172.21.15.12:0/4064984303 >> 172.21.15.12:0/4064984303 conn(0x7f49a4100370 msgr2=0x7f49a41027b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:59:09.074 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:59:09.070+0000 7f49aa400640 1 -- 172.21.15.12:0/4064984303 shutdown_connections 2024-09-10T23:59:09.074 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:59:09.070+0000 7f49aa400640 1 -- 172.21.15.12:0/4064984303 wait complete. 2024-09-10T23:59:09.075 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:59:09.071+0000 7f49aa400640 1 Processor -- start 2024-09-10T23:59:09.075 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:59:09.071+0000 7f49aa400640 1 -- start start 2024-09-10T23:59:09.078 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:59:09.071+0000 7f49aa400640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f49a4104aa0 0x7f49a419adf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:59:09.078 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:59:09.071+0000 7f49aa400640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f49a4105c00 0x7f49a419b440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:59:09.078 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:59:09.072+0000 7f49aa400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f49a41065c0 0x7f49a419c080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:59:09.078 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:59:09.072+0000 7f49aa400640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f49a4110750 con 0x7f49a4104aa0 2024-09-10T23:59:09.078 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:59:09.072+0000 7f49aa400640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f49a41105d0 con 0x7f49a41065c0 2024-09-10T23:59:09.079 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:59:09.072+0000 7f49aa400640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f49a41108d0 con 0x7f49a4105c00 2024-09-10T23:59:09.079 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:59:09.072+0000 7f49a9000640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f49a4104aa0 0x7f49a419adf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:59:09.079 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:59:09.072+0000 7f49a9000640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f49a4104aa0 0x7f49a419adf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:54842/0 (socket says 172.21.15.12:54842) 2024-09-10T23:59:09.079 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:59:09.072+0000 7f49a9000640 1 -- 172.21.15.12:0/1973115948 learned_addr learned my addr 172.21.15.12:0/1973115948 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-10T23:59:09.079 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:59:09.072+0000 7f49a3e00640 1 --2- 172.21.15.12:0/1973115948 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f49a4105c00 0x7f49a419b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:59:09.079 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:59:09.072+0000 7f49a9a00640 1 --2- 172.21.15.12:0/1973115948 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f49a41065c0 0x7f49a419c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:59:09.080 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:59:09.072+0000 7f49a9000640 1 -- 172.21.15.12:0/1973115948 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f49a4105c00 msgr2=0x7f49a419b440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:59:09.080 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:59:09.072+0000 7f49a9000640 1 --2- 172.21.15.12:0/1973115948 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f49a4105c00 0x7f49a419b440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:59:09.080 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:59:09.072+0000 7f49a9000640 1 -- 172.21.15.12:0/1973115948 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f49a41065c0 msgr2=0x7f49a419c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:59:09.080 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:59:09.072+0000 7f49a9000640 1 --2- 172.21.15.12:0/1973115948 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f49a41065c0 0x7f49a419c080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:59:09.080 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:59:09.072+0000 7f49a9000640 1 -- 172.21.15.12:0/1973115948 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49a419c920 con 0x7f49a4104aa0 2024-09-10T23:59:09.080 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:59:09.073+0000 7f49a9000640 1 --2- 172.21.15.12:0/1973115948 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f49a4104aa0 0x7f49a419adf0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f4998031490 tx=0x7f4998003c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:59:09.080 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:59:09.073+0000 7f49a1600640 1 -- 172.21.15.12:0/1973115948 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f499803f070 con 0x7f49a4104aa0 2024-09-10T23:59:09.081 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:59:09.073+0000 7f49a1600640 1 -- 172.21.15.12:0/1973115948 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4998031df0 con 0x7f49a4104aa0 2024-09-10T23:59:09.081 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:59:09.073+0000 7f49a1600640 1 -- 172.21.15.12:0/1973115948 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4998035030 con 0x7f49a4104aa0 2024-09-10T23:59:09.081 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:59:09.073+0000 7f49aa400640 1 -- 172.21.15.12:0/1973115948 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49a419cbb0 con 0x7f49a4104aa0 2024-09-10T23:59:09.081 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:59:09.073+0000 7f49aa400640 1 -- 172.21.15.12:0/1973115948 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49a41a3120 con 0x7f49a4104aa0 2024-09-10T23:59:09.081 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:59:09.074+0000 7f49aa400640 1 -- 172.21.15.12:0/1973115948 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f49a410bdc0 con 0x7f49a4104aa0 2024-09-10T23:59:09.082 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:59:08 smithi178 ceph-mon[35919]: Detected new or changed devices on smithi178 2024-09-10T23:59:09.082 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:59:08 smithi178 ceph-mon[35919]: Adjusting osd_memory_target on smithi178 to 17088M 2024-09-10T23:59:09.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:59:08 smithi178 ceph-mon[35919]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi178", "root=default"]}]': finished 2024-09-10T23:59:09.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:59:08 smithi178 ceph-mon[35919]: osdmap e17: 3 total, 2 up, 3 in 2024-09-10T23:59:09.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:59:08 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-10T23:59:09.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:59:08 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-10T23:59:09.084 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:59:09.076+0000 7f49a1600640 1 -- 172.21.15.12:0/1973115948 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 15) ==== 153194+0+0 (secure 0 0 0) 0x7f499803b560 con 0x7f49a4104aa0 2024-09-10T23:59:09.084 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:59:09.078+0000 7f49a1600640 1 --2- 172.21.15.12:0/1973115948 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f498c0b53f0 0x7f498c0b78b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:59:09.084 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:59:09.078+0000 7f49a1600640 1 -- 172.21.15.12:0/1973115948 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(18..18 src has 1..18) ==== 2851+0+0 (secure 0 0 0) 0x7f49981055c0 con 0x7f49a4104aa0 2024-09-10T23:59:09.084 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:59:09.078+0000 7f49a3e00640 1 --2- 172.21.15.12:0/1973115948 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f498c0b53f0 0x7f498c0b78b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:59:09.085 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:59:09.079+0000 7f49a3e00640 1 --2- 172.21.15.12:0/1973115948 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f498c0b53f0 0x7f498c0b78b0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f4990009bd0 tx=0x7f4990005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:59:09.085 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:59:09.080+0000 7f49a1600640 1 -- 172.21.15.12:0/1973115948 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f49980cd3c0 con 0x7f49a4104aa0 2024-09-10T23:59:09.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:59:09.287+0000 7f49aa400640 1 -- 172.21.15.12:0/1973115948 --> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi012:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f49a4063590 con 0x7f498c0b53f0 2024-09-10T23:59:09.884 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:59:09 smithi012 ceph-mon[30333]: purged_snaps scrub starts 2024-09-10T23:59:09.884 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:59:09 smithi012 ceph-mon[30333]: purged_snaps scrub ok 2024-09-10T23:59:09.885 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:59:09 smithi012 ceph-mon[30333]: pgmap v92: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-10T23:59:09.885 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:59:09 smithi012 ceph-mon[30333]: osd.2 [v2:172.21.15.178:6800/4131162695,v1:172.21.15.178:6801/4131162695] boot 2024-09-10T23:59:09.885 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:59:09 smithi012 ceph-mon[30333]: osdmap e18: 3 total, 3 up, 3 in 2024-09-10T23:59:09.885 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:59:09 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-10T23:59:09.885 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:59:09 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-10T23:59:09.885 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:59:09 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-10T23:59:09.885 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:59:09 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T23:59:10.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:59:09 smithi165 ceph-mon[35356]: purged_snaps scrub starts 2024-09-10T23:59:10.033 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:59:09 smithi165 ceph-mon[35356]: purged_snaps scrub ok 2024-09-10T23:59:10.033 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:59:09 smithi165 ceph-mon[35356]: pgmap v92: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-10T23:59:10.033 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:59:09 smithi165 ceph-mon[35356]: osd.2 [v2:172.21.15.178:6800/4131162695,v1:172.21.15.178:6801/4131162695] boot 2024-09-10T23:59:10.033 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:59:09 smithi165 ceph-mon[35356]: osdmap e18: 3 total, 3 up, 3 in 2024-09-10T23:59:10.034 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:59:09 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-10T23:59:10.034 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:59:09 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-10T23:59:10.034 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:59:09 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-10T23:59:10.034 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:59:09 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T23:59:10.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:59:09 smithi178 ceph-mon[35919]: purged_snaps scrub starts 2024-09-10T23:59:10.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:59:09 smithi178 ceph-mon[35919]: purged_snaps scrub ok 2024-09-10T23:59:10.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:59:09 smithi178 ceph-mon[35919]: pgmap v92: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-10T23:59:10.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:59:09 smithi178 ceph-mon[35919]: osd.2 [v2:172.21.15.178:6800/4131162695,v1:172.21.15.178:6801/4131162695] boot 2024-09-10T23:59:10.082 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:59:09 smithi178 ceph-mon[35919]: osdmap e18: 3 total, 3 up, 3 in 2024-09-10T23:59:10.082 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:59:09 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-10T23:59:10.082 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:59:09 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-10T23:59:10.082 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:59:09 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-10T23:59:10.082 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:59:09 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T23:59:10.885 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:59:10 smithi012 ceph-mon[30333]: from='client.14400 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi012:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T23:59:10.885 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:59:10 smithi012 ceph-mon[30333]: osdmap e19: 3 total, 3 up, 3 in 2024-09-10T23:59:10.885 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:59:10 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true} : dispatch 2024-09-10T23:59:11.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:59:10 smithi165 ceph-mon[35356]: from='client.14400 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi012:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T23:59:11.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:59:10 smithi165 ceph-mon[35356]: osdmap e19: 3 total, 3 up, 3 in 2024-09-10T23:59:11.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:59:10 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true} : dispatch 2024-09-10T23:59:11.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:59:10 smithi178 ceph-mon[35919]: from='client.14400 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi012:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T23:59:11.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:59:10 smithi178 ceph-mon[35919]: osdmap e19: 3 total, 3 up, 3 in 2024-09-10T23:59:11.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:59:10 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true} : dispatch 2024-09-10T23:59:11.728 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:59:11 smithi012 ceph-mon[30333]: pgmap v95: 0 pgs: ; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-10T23:59:11.952 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:59:11 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-09-10T23:59:11.953 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:59:11 smithi012 ceph-mon[30333]: osdmap e20: 3 total, 3 up, 3 in 2024-09-10T23:59:11.953 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:59:11 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true} : dispatch 2024-09-10T23:59:11.954 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:59:11 smithi178 ceph-mon[35919]: pgmap v95: 0 pgs: ; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-10T23:59:11.954 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:59:11 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-09-10T23:59:11.954 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:59:11 smithi178 ceph-mon[35919]: osdmap e20: 3 total, 3 up, 3 in 2024-09-10T23:59:11.954 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:59:11 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true} : dispatch 2024-09-10T23:59:11.954 INFO:journalctl@ceph.osd.2.smithi178.stdout:Sep 10 23:59:11 smithi178 sudo[44508]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-10T23:59:11.954 INFO:journalctl@ceph.osd.2.smithi178.stdout:Sep 10 23:59:11 smithi178 sudo[44508]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-10T23:59:11.954 INFO:journalctl@ceph.osd.2.smithi178.stdout:Sep 10 23:59:11 smithi178 sudo[44508]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-10T23:59:11.954 INFO:journalctl@ceph.osd.2.smithi178.stdout:Sep 10 23:59:11 smithi178 sudo[44508]: pam_unix(sudo:session): session closed for user root 2024-09-10T23:59:11.989 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:59:11 smithi012 sudo[51198]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-10T23:59:11.989 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:59:11 smithi012 sudo[51198]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-10T23:59:11.989 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:59:11 smithi012 sudo[51198]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-10T23:59:11.989 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:59:11 smithi012 sudo[51198]: pam_unix(sudo:session): session closed for user root 2024-09-10T23:59:11.989 INFO:journalctl@ceph.osd.0.smithi012.stdout:Sep 10 23:59:11 smithi012 sudo[51179]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-10T23:59:11.989 INFO:journalctl@ceph.osd.0.smithi012.stdout:Sep 10 23:59:11 smithi012 sudo[51179]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-10T23:59:11.989 INFO:journalctl@ceph.osd.0.smithi012.stdout:Sep 10 23:59:11 smithi012 sudo[51179]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-10T23:59:11.989 INFO:journalctl@ceph.osd.0.smithi012.stdout:Sep 10 23:59:11 smithi012 sudo[51179]: pam_unix(sudo:session): session closed for user root 2024-09-10T23:59:11.990 INFO:journalctl@ceph.osd.0.smithi012.stdout:Sep 10 23:59:11 smithi012 sudo[51185]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-10T23:59:11.990 INFO:journalctl@ceph.osd.0.smithi012.stdout:Sep 10 23:59:11 smithi012 sudo[51185]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-10T23:59:11.990 INFO:journalctl@ceph.osd.0.smithi012.stdout:Sep 10 23:59:11 smithi012 sudo[51185]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-10T23:59:11.990 INFO:journalctl@ceph.osd.0.smithi012.stdout:Sep 10 23:59:11 smithi012 sudo[51185]: pam_unix(sudo:session): session closed for user root 2024-09-10T23:59:12.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:59:11 smithi165 ceph-mon[35356]: pgmap v95: 0 pgs: ; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-10T23:59:12.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:59:11 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-09-10T23:59:12.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:59:11 smithi165 ceph-mon[35356]: osdmap e20: 3 total, 3 up, 3 in 2024-09-10T23:59:12.033 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:59:11 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true} : dispatch 2024-09-10T23:59:12.033 INFO:journalctl@ceph.osd.1.smithi165.stdout:Sep 10 23:59:11 smithi165 sudo[43661]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-10T23:59:12.033 INFO:journalctl@ceph.osd.1.smithi165.stdout:Sep 10 23:59:11 smithi165 sudo[43661]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-10T23:59:12.033 INFO:journalctl@ceph.osd.1.smithi165.stdout:Sep 10 23:59:11 smithi165 sudo[43661]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-10T23:59:12.033 INFO:journalctl@ceph.osd.1.smithi165.stdout:Sep 10 23:59:11 smithi165 sudo[43661]: pam_unix(sudo:session): session closed for user root 2024-09-10T23:59:12.033 INFO:journalctl@ceph.osd.1.smithi165.stdout:Sep 10 23:59:11 smithi165 sudo[43664]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-10T23:59:12.033 INFO:journalctl@ceph.osd.1.smithi165.stdout:Sep 10 23:59:11 smithi165 sudo[43664]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-10T23:59:12.033 INFO:journalctl@ceph.osd.1.smithi165.stdout:Sep 10 23:59:11 smithi165 sudo[43664]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-10T23:59:12.033 INFO:journalctl@ceph.osd.1.smithi165.stdout:Sep 10 23:59:11 smithi165 sudo[43664]: pam_unix(sudo:session): session closed for user root 2024-09-10T23:59:12.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:59:12 smithi012 sudo[51220]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-10T23:59:12.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:59:12 smithi012 sudo[51220]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-10T23:59:12.317 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:59:12 smithi012 sudo[51220]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-10T23:59:12.317 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:59:12 smithi012 sudo[51220]: pam_unix(sudo:session): session closed for user root 2024-09-10T23:59:12.330 INFO:journalctl@ceph.osd.2.smithi178.stdout:Sep 10 23:59:11 smithi178 sudo[44511]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-10T23:59:12.331 INFO:journalctl@ceph.osd.2.smithi178.stdout:Sep 10 23:59:11 smithi178 sudo[44511]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-10T23:59:12.331 INFO:journalctl@ceph.osd.2.smithi178.stdout:Sep 10 23:59:11 smithi178 sudo[44511]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-10T23:59:12.331 INFO:journalctl@ceph.osd.2.smithi178.stdout:Sep 10 23:59:11 smithi178 sudo[44511]: pam_unix(sudo:session): session closed for user root 2024-09-10T23:59:12.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:59:12 smithi178 sudo[44514]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-10T23:59:12.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:59:12 smithi178 sudo[44514]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-10T23:59:12.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:59:12 smithi178 sudo[44514]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-10T23:59:12.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:59:12 smithi178 sudo[44514]: pam_unix(sudo:session): session closed for user root 2024-09-10T23:59:12.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:59:12 smithi178 sudo[44517]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-10T23:59:12.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:59:12 smithi178 sudo[44517]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-10T23:59:12.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:59:12 smithi178 sudo[44517]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-10T23:59:12.332 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:59:12 smithi178 sudo[44517]: pam_unix(sudo:session): session closed for user root 2024-09-10T23:59:12.532 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:59:12 smithi165 sudo[43667]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-10T23:59:12.532 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:59:12 smithi165 sudo[43667]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-10T23:59:12.532 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:59:12 smithi165 sudo[43667]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-10T23:59:12.532 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:59:12 smithi165 sudo[43667]: pam_unix(sudo:session): session closed for user root 2024-09-10T23:59:12.532 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:59:12 smithi165 sudo[43670]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-10T23:59:12.533 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:59:12 smithi165 sudo[43670]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-10T23:59:12.533 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:59:12 smithi165 sudo[43670]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-10T23:59:12.533 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:59:12 smithi165 sudo[43670]: pam_unix(sudo:session): session closed for user root 2024-09-10T23:59:12.813 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:59:12 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-09-10T23:59:12.813 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:59:12 smithi012 ceph-mon[30333]: osdmap e21: 3 total, 3 up, 3 in 2024-09-10T23:59:12.813 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:59:12 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-10T23:59:12.813 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:59:12 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-10T23:59:12.813 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:59:12 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-10T23:59:12.813 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:59:12 smithi012 ceph-mon[30333]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-10T23:59:12.813 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:59:12 smithi012 ceph-mon[30333]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-10T23:59:12.814 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:59:12 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-10T23:59:12.814 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:59:12 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-10T23:59:12.814 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:59:12 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-10T23:59:12.814 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:59:12 smithi012 ceph-mon[30333]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-10T23:59:12.814 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:59:12 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-10T23:59:12.814 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:59:12 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-10T23:59:12.814 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:59:12 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-10T23:59:12.814 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:59:12 smithi012 ceph-mon[30333]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-10T23:59:12.814 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:59:12 smithi012 ceph-mon[30333]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-10T23:59:12.814 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:59:12 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-10T23:59:12.814 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:59:12 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-10T23:59:12.814 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:59:12 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-10T23:59:12.814 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:59:12 smithi012 ceph-mon[30333]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-10T23:59:12.815 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:59:12 smithi012 ceph-mon[30333]: from='client.? 172.21.15.12:0/3724929072' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "af2f64d6-c9be-44a9-b041-8d599a7c080d"} : dispatch 2024-09-10T23:59:12.815 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:59:12 smithi012 ceph-mon[30333]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "af2f64d6-c9be-44a9-b041-8d599a7c080d"} : dispatch 2024-09-10T23:59:12.815 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:59:12 smithi012 ceph-mon[30333]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "af2f64d6-c9be-44a9-b041-8d599a7c080d"}]': finished 2024-09-10T23:59:12.815 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:59:12 smithi012 ceph-mon[30333]: osdmap e22: 4 total, 3 up, 4 in 2024-09-10T23:59:12.815 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:59:12 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-10T23:59:12.965 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:59:12.963+0000 7f49a1600640 1 -- 172.21.15.12:0/1973115948 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mgrmap(e 16) ==== 153250+0+0 (secure 0 0 0) 0x7f49980c9630 con 0x7f49a4104aa0 2024-09-10T23:59:13.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:59:12 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-09-10T23:59:13.033 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:59:12 smithi165 ceph-mon[35356]: osdmap e21: 3 total, 3 up, 3 in 2024-09-10T23:59:13.033 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:59:12 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-10T23:59:13.033 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:59:12 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-10T23:59:13.033 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:59:12 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-10T23:59:13.033 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:59:12 smithi165 ceph-mon[35356]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-10T23:59:13.033 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:59:12 smithi165 ceph-mon[35356]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-10T23:59:13.033 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:59:12 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-10T23:59:13.033 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:59:12 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-10T23:59:13.033 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:59:12 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-10T23:59:13.033 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:59:12 smithi165 ceph-mon[35356]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-10T23:59:13.033 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:59:12 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-10T23:59:13.034 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:59:12 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-10T23:59:13.034 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:59:12 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-10T23:59:13.034 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:59:12 smithi165 ceph-mon[35356]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-10T23:59:13.034 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:59:12 smithi165 ceph-mon[35356]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-10T23:59:13.034 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:59:12 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-10T23:59:13.034 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:59:12 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-10T23:59:13.034 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:59:12 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-10T23:59:13.035 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:59:12 smithi165 ceph-mon[35356]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-10T23:59:13.035 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:59:12 smithi165 ceph-mon[35356]: from='client.? 172.21.15.12:0/3724929072' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "af2f64d6-c9be-44a9-b041-8d599a7c080d"} : dispatch 2024-09-10T23:59:13.035 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:59:12 smithi165 ceph-mon[35356]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "af2f64d6-c9be-44a9-b041-8d599a7c080d"} : dispatch 2024-09-10T23:59:13.035 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:59:12 smithi165 ceph-mon[35356]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "af2f64d6-c9be-44a9-b041-8d599a7c080d"}]': finished 2024-09-10T23:59:13.035 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:59:12 smithi165 ceph-mon[35356]: osdmap e22: 4 total, 3 up, 4 in 2024-09-10T23:59:13.035 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:59:12 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-10T23:59:13.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:59:12 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-09-10T23:59:13.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:59:12 smithi178 ceph-mon[35919]: osdmap e21: 3 total, 3 up, 3 in 2024-09-10T23:59:13.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:59:12 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-10T23:59:13.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:59:12 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-10T23:59:13.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:59:12 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-10T23:59:13.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:59:12 smithi178 ceph-mon[35919]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-10T23:59:13.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:59:12 smithi178 ceph-mon[35919]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-10T23:59:13.082 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:59:12 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-10T23:59:13.082 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:59:12 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-10T23:59:13.082 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:59:12 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-10T23:59:13.082 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:59:12 smithi178 ceph-mon[35919]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-10T23:59:13.082 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:59:12 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-10T23:59:13.082 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:59:12 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-10T23:59:13.082 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:59:12 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-10T23:59:13.082 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:59:12 smithi178 ceph-mon[35919]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-10T23:59:13.082 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:59:12 smithi178 ceph-mon[35919]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-10T23:59:13.082 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:59:12 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-10T23:59:13.082 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:59:12 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-10T23:59:13.082 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:59:12 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-10T23:59:13.082 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:59:12 smithi178 ceph-mon[35919]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-10T23:59:13.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:59:12 smithi178 ceph-mon[35919]: from='client.? 172.21.15.12:0/3724929072' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "af2f64d6-c9be-44a9-b041-8d599a7c080d"} : dispatch 2024-09-10T23:59:13.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:59:12 smithi178 ceph-mon[35919]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "af2f64d6-c9be-44a9-b041-8d599a7c080d"} : dispatch 2024-09-10T23:59:13.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:59:12 smithi178 ceph-mon[35919]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "af2f64d6-c9be-44a9-b041-8d599a7c080d"}]': finished 2024-09-10T23:59:13.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:59:12 smithi178 ceph-mon[35919]: osdmap e22: 4 total, 3 up, 4 in 2024-09-10T23:59:13.083 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:59:12 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-10T23:59:13.782 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:59:13 smithi165 ceph-mon[35356]: pgmap v98: 1 pgs: 1 unknown; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-10T23:59:13.782 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:59:13 smithi165 ceph-mon[35356]: from='client.? 172.21.15.12:0/655240045' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-10T23:59:13.782 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:59:13 smithi165 ceph-mon[35356]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-09-10T23:59:14.062 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:59:13 smithi012 ceph-mon[30333]: pgmap v98: 1 pgs: 1 unknown; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-10T23:59:14.063 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:59:13 smithi012 ceph-mon[30333]: from='client.? 172.21.15.12:0/655240045' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-10T23:59:14.063 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:59:13 smithi012 ceph-mon[30333]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-09-10T23:59:14.080 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:59:13 smithi178 ceph-mon[35919]: pgmap v98: 1 pgs: 1 unknown; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-10T23:59:14.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:59:13 smithi178 ceph-mon[35919]: from='client.? 172.21.15.12:0/655240045' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-10T23:59:14.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:59:13 smithi178 ceph-mon[35919]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-09-10T23:59:16.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:59:15 smithi165 ceph-mon[35356]: pgmap v100: 1 pgs: 1 unknown; 0 B data, 89 MiB used, 268 GiB / 268 GiB avail 2024-09-10T23:59:16.062 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:59:15 smithi012 ceph-mon[30333]: pgmap v100: 1 pgs: 1 unknown; 0 B data, 89 MiB used, 268 GiB / 268 GiB avail 2024-09-10T23:59:16.080 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:59:15 smithi178 ceph-mon[35919]: pgmap v100: 1 pgs: 1 unknown; 0 B data, 89 MiB used, 268 GiB / 268 GiB avail 2024-09-10T23:59:18.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:59:17 smithi165 ceph-mon[35356]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-09-10T23:59:18.062 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:59:17 smithi012 ceph-mon[30333]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-09-10T23:59:18.080 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:59:17 smithi178 ceph-mon[35919]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-09-10T23:59:19.997 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:59:19 smithi012 ceph-mon[30333]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-09-10T23:59:20.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:59:19 smithi165 ceph-mon[35356]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-09-10T23:59:20.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:59:19 smithi178 ceph-mon[35919]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-09-10T23:59:21.812 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:59:21 smithi012 ceph-mon[30333]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-09-10T23:59:21.813 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:59:21 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.3"} : dispatch 2024-09-10T23:59:21.813 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:59:21 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T23:59:22.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:59:21 smithi165 ceph-mon[35356]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-09-10T23:59:22.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:59:21 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.3"} : dispatch 2024-09-10T23:59:22.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:59:21 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T23:59:22.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:59:21 smithi178 ceph-mon[35919]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-09-10T23:59:22.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:59:21 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.3"} : dispatch 2024-09-10T23:59:22.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:59:21 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T23:59:22.813 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:59:22 smithi012 ceph-mon[30333]: Deploying daemon osd.3 on smithi012 2024-09-10T23:59:23.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:59:22 smithi165 ceph-mon[35356]: Deploying daemon osd.3 on smithi012 2024-09-10T23:59:23.080 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:59:22 smithi178 ceph-mon[35919]: Deploying daemon osd.3 on smithi012 2024-09-10T23:59:23.927 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:59:23 smithi012 ceph-mon[30333]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-09-10T23:59:24.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:59:23 smithi165 ceph-mon[35356]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-09-10T23:59:24.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:59:23 smithi178 ceph-mon[35919]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-09-10T23:59:25.813 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:59:25 smithi012 ceph-mon[30333]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-09-10T23:59:26.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:59:25 smithi165 ceph-mon[35356]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-09-10T23:59:26.080 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:59:25 smithi178 ceph-mon[35919]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-09-10T23:59:27.813 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:59:27 smithi012 ceph-mon[30333]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-09-10T23:59:28.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:59:27 smithi165 ceph-mon[35356]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-09-10T23:59:28.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:59:27 smithi178 ceph-mon[35919]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-09-10T23:59:28.812 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:59:28 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T23:59:28.813 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:59:28 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-10T23:59:29.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:59:28 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T23:59:29.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:59:28 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-10T23:59:29.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:59:28 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T23:59:29.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:59:28 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-10T23:59:30.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:59:29 smithi165 ceph-mon[35356]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-09-10T23:59:30.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:59:29 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-10T23:59:30.058 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:59:29 smithi012 ceph-mon[30333]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-09-10T23:59:30.058 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:59:29 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-10T23:59:30.080 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:59:29 smithi178 ceph-mon[35919]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-09-10T23:59:30.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:59:29 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-10T23:59:32.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:59:31 smithi165 ceph-mon[35356]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-09-10T23:59:32.062 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:59:31 smithi012 ceph-mon[30333]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-09-10T23:59:32.080 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:59:31 smithi178 ceph-mon[35919]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-09-10T23:59:33.063 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:59:32 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-10T23:59:33.063 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:59:32 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-10T23:59:33.063 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:59:32 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T23:59:33.063 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:59:32 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T23:59:33.063 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:59:32 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-10T23:59:33.063 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:59:32 smithi012 ceph-mon[30333]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-09-10T23:59:33.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:59:32 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-10T23:59:33.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:59:32 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-10T23:59:33.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:59:32 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T23:59:33.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:59:32 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T23:59:33.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:59:32 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-10T23:59:33.283 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:59:32 smithi165 ceph-mon[35356]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-09-10T23:59:33.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:59:32 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-10T23:59:33.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:59:32 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-10T23:59:33.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:59:32 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T23:59:33.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:59:32 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T23:59:33.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:59:32 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-10T23:59:33.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:59:32 smithi178 ceph-mon[35919]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-09-10T23:59:33.611 INFO:teuthology.orchestra.run.smithi012.stdout:Created osd(s) 3 on host 'smithi012' 2024-09-10T23:59:33.612 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:59:33.609+0000 7f49a1600640 1 -- 172.21.15.12:0/1973115948 <== mgr.14150 v2:172.21.15.12:6800/3442939137 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f49a4063590 con 0x7f498c0b53f0 2024-09-10T23:59:33.614 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:59:33.612+0000 7f49aa400640 1 -- 172.21.15.12:0/1973115948 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f498c0b53f0 msgr2=0x7f498c0b78b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:59:33.614 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:59:33.612+0000 7f49aa400640 1 --2- 172.21.15.12:0/1973115948 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f498c0b53f0 0x7f498c0b78b0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f4990009bd0 tx=0x7f4990005f90 comp rx=0 tx=0).stop 2024-09-10T23:59:33.614 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:59:33.612+0000 7f49aa400640 1 -- 172.21.15.12:0/1973115948 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f49a4104aa0 msgr2=0x7f49a419adf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:59:33.614 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:59:33.612+0000 7f49aa400640 1 --2- 172.21.15.12:0/1973115948 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f49a4104aa0 0x7f49a419adf0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f4998031490 tx=0x7f4998003c80 comp rx=0 tx=0).stop 2024-09-10T23:59:33.614 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:59:33.612+0000 7f49aa400640 1 -- 172.21.15.12:0/1973115948 shutdown_connections 2024-09-10T23:59:33.614 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:59:33.612+0000 7f49aa400640 1 --2- 172.21.15.12:0/1973115948 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f498c0b53f0 0x7f498c0b78b0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:59:33.614 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:59:33.612+0000 7f49aa400640 1 --2- 172.21.15.12:0/1973115948 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f49a41065c0 0x7f49a419c080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:59:33.614 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:59:33.612+0000 7f49aa400640 1 --2- 172.21.15.12:0/1973115948 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f49a4105c00 0x7f49a419b440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:59:33.615 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:59:33.612+0000 7f49aa400640 1 --2- 172.21.15.12:0/1973115948 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f49a4104aa0 0x7f49a419adf0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:59:33.615 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:59:33.612+0000 7f49aa400640 1 -- 172.21.15.12:0/1973115948 >> 172.21.15.12:0/1973115948 conn(0x7f49a4100370 msgr2=0x7f49a4108f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:59:33.615 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:59:33.612+0000 7f49aa400640 1 -- 172.21.15.12:0/1973115948 shutdown_connections 2024-09-10T23:59:33.615 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-10T23:59:33.612+0000 7f49aa400640 1 -- 172.21.15.12:0/1973115948 wait complete. 2024-09-10T23:59:34.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:59:33 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T23:59:34.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:59:33 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-10T23:59:34.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:59:33 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-10T23:59:34.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:59:33 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T23:59:34.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:59:33 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-10T23:59:34.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:59:33 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-10T23:59:34.330 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:59:33 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T23:59:34.330 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:59:33 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-10T23:59:34.330 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:59:33 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-10T23:59:34.382 DEBUG:teuthology.orchestra.run.smithi012:osd.3> sudo journalctl -f -n 0 -u ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968@osd.3.service 2024-09-10T23:59:34.386 INFO:tasks.cephadm:Deploying osd.4 on smithi165 with /dev/vg_nvme/lv_3... 2024-09-10T23:59:34.387 DEBUG:teuthology.orchestra.run.smithi165:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ff3ad74a-6fcf-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_3 2024-09-10T23:59:34.641 INFO:teuthology.orchestra.run.smithi165.stderr:Inferring config /var/lib/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T23:59:34.904 INFO:journalctl@ceph.osd.3.smithi012.stdout:Sep 10 23:59:34 smithi012 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-3[53245]: 2024-09-10T23:59:34.701+0000 7f974e497640 -1 osd.3 0 log_to_monitors true 2024-09-10T23:59:35.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:59:34 smithi165 ceph-mon[35356]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-09-10T23:59:35.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:59:34 smithi165 ceph-mon[35356]: from='osd.3 [v2:172.21.15.12:6810/452464245,v1:172.21.15.12:6811/452464245]' entity='osd.3' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]} : dispatch 2024-09-10T23:59:35.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:59:34 smithi165 ceph-mon[35356]: from='osd.3 ' entity='osd.3' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]} : dispatch 2024-09-10T23:59:35.232 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:59:34 smithi178 ceph-mon[35919]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-09-10T23:59:35.233 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:59:34 smithi178 ceph-mon[35919]: from='osd.3 [v2:172.21.15.12:6810/452464245,v1:172.21.15.12:6811/452464245]' entity='osd.3' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]} : dispatch 2024-09-10T23:59:35.233 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:59:34 smithi178 ceph-mon[35919]: from='osd.3 ' entity='osd.3' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]} : dispatch 2024-09-10T23:59:35.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:59:34 smithi012 ceph-mon[30333]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-09-10T23:59:35.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:59:34 smithi012 ceph-mon[30333]: from='osd.3 [v2:172.21.15.12:6810/452464245,v1:172.21.15.12:6811/452464245]' entity='osd.3' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]} : dispatch 2024-09-10T23:59:35.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:59:34 smithi012 ceph-mon[30333]: from='osd.3 ' entity='osd.3' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]} : dispatch 2024-09-10T23:59:36.019 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:59:35 smithi165 ceph-mon[35356]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-09-10T23:59:36.019 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:59:35 smithi165 ceph-mon[35356]: from='osd.3 [v2:172.21.15.12:6810/452464245,v1:172.21.15.12:6811/452464245]' entity='osd.3' cmd={"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi012", "root=default"]} : dispatch 2024-09-10T23:59:36.019 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:59:35 smithi165 ceph-mon[35356]: osdmap e23: 4 total, 3 up, 4 in 2024-09-10T23:59:36.019 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:59:35 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-10T23:59:36.020 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:59:35 smithi165 ceph-mon[35356]: from='osd.3 ' entity='osd.3' cmd={"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi012", "root=default"]} : dispatch 2024-09-10T23:59:36.246 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:59:35 smithi178 ceph-mon[35919]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-09-10T23:59:36.246 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:59:35 smithi178 ceph-mon[35919]: from='osd.3 [v2:172.21.15.12:6810/452464245,v1:172.21.15.12:6811/452464245]' entity='osd.3' cmd={"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi012", "root=default"]} : dispatch 2024-09-10T23:59:36.246 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:59:35 smithi178 ceph-mon[35919]: osdmap e23: 4 total, 3 up, 4 in 2024-09-10T23:59:36.246 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:59:35 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-10T23:59:36.246 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:59:35 smithi178 ceph-mon[35919]: from='osd.3 ' entity='osd.3' cmd={"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi012", "root=default"]} : dispatch 2024-09-10T23:59:36.285 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:59:35 smithi012 ceph-mon[30333]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-09-10T23:59:36.285 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:59:35 smithi012 ceph-mon[30333]: from='osd.3 [v2:172.21.15.12:6810/452464245,v1:172.21.15.12:6811/452464245]' entity='osd.3' cmd={"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi012", "root=default"]} : dispatch 2024-09-10T23:59:36.285 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:59:35 smithi012 ceph-mon[30333]: osdmap e23: 4 total, 3 up, 4 in 2024-09-10T23:59:36.285 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:59:35 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-10T23:59:36.285 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:59:35 smithi012 ceph-mon[30333]: from='osd.3 ' entity='osd.3' cmd={"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi012", "root=default"]} : dispatch 2024-09-10T23:59:37.062 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:59:36 smithi012 ceph-mon[30333]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi012", "root=default"]}]': finished 2024-09-10T23:59:37.063 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:59:36 smithi012 ceph-mon[30333]: osdmap e24: 4 total, 3 up, 4 in 2024-09-10T23:59:37.063 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:59:36 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-10T23:59:37.063 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:59:36 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-10T23:59:37.063 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:59:36 smithi012 ceph-mon[30333]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-09-10T23:59:37.063 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:59:36 smithi012 ceph-mon[30333]: from='osd.3 ' entity='osd.3' 2024-09-10T23:59:37.063 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:59:36 smithi012 ceph-mon[30333]: osd.3 [v2:172.21.15.12:6810/452464245,v1:172.21.15.12:6811/452464245] boot 2024-09-10T23:59:37.063 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:59:36 smithi012 ceph-mon[30333]: osdmap e25: 4 total, 4 up, 4 in 2024-09-10T23:59:37.063 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:59:36 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-10T23:59:37.064 INFO:journalctl@ceph.osd.3.smithi012.stdout:Sep 10 23:59:36 smithi012 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-3[53245]: 2024-09-10T23:59:36.736+0000 7f9749400640 -1 osd.3 0 waiting for initial osdmap 2024-09-10T23:59:37.064 INFO:journalctl@ceph.osd.3.smithi012.stdout:Sep 10 23:59:36 smithi012 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-3[53245]: 2024-09-10T23:59:36.758+0000 7f973f400640 -1 osd.3 24 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-10T23:59:37.222 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:59:36 smithi165 ceph-mon[35356]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi012", "root=default"]}]': finished 2024-09-10T23:59:37.222 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:59:36 smithi165 ceph-mon[35356]: osdmap e24: 4 total, 3 up, 4 in 2024-09-10T23:59:37.222 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:59:36 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-10T23:59:37.222 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:59:36 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-10T23:59:37.222 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:59:36 smithi165 ceph-mon[35356]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-09-10T23:59:37.223 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:59:36 smithi165 ceph-mon[35356]: from='osd.3 ' entity='osd.3' 2024-09-10T23:59:37.223 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:59:36 smithi165 ceph-mon[35356]: osd.3 [v2:172.21.15.12:6810/452464245,v1:172.21.15.12:6811/452464245] boot 2024-09-10T23:59:37.223 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:59:36 smithi165 ceph-mon[35356]: osdmap e25: 4 total, 4 up, 4 in 2024-09-10T23:59:37.223 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:59:36 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-10T23:59:37.225 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-09-10T23:59:37.252 DEBUG:teuthology.orchestra.run.smithi165:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ff3ad74a-6fcf-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi165:vg_nvme/lv_3 2024-09-10T23:59:37.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:59:36 smithi178 ceph-mon[35919]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi012", "root=default"]}]': finished 2024-09-10T23:59:37.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:59:36 smithi178 ceph-mon[35919]: osdmap e24: 4 total, 3 up, 4 in 2024-09-10T23:59:37.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:59:36 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-10T23:59:37.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:59:36 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-10T23:59:37.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:59:36 smithi178 ceph-mon[35919]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-09-10T23:59:37.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:59:36 smithi178 ceph-mon[35919]: from='osd.3 ' entity='osd.3' 2024-09-10T23:59:37.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:59:36 smithi178 ceph-mon[35919]: osd.3 [v2:172.21.15.12:6810/452464245,v1:172.21.15.12:6811/452464245] boot 2024-09-10T23:59:37.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:59:36 smithi178 ceph-mon[35919]: osdmap e25: 4 total, 4 up, 4 in 2024-09-10T23:59:37.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:59:36 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-10T23:59:37.509 INFO:teuthology.orchestra.run.smithi165.stderr:Inferring config /var/lib/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T23:59:37.958 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-10T23:59:37.935+0000 7ff2bfde1640 1 -- 172.21.15.165:0/2592733549 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff2b8100e10 msgr2=0x7ff2b81011f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:59:37.958 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-10T23:59:37.935+0000 7ff2bfde1640 1 --2- 172.21.15.165:0/2592733549 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff2b8100e10 0x7ff2b81011f0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7ff2b4009900 tx=0x7ff2b402a410 comp rx=0 tx=0).stop 2024-09-10T23:59:37.958 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-10T23:59:37.935+0000 7ff2bfde1640 1 -- 172.21.15.165:0/2592733549 shutdown_connections 2024-09-10T23:59:37.958 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-10T23:59:37.935+0000 7ff2bfde1640 1 --2- 172.21.15.165:0/2592733549 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff2b8108810 0x7ff2b810ac00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:59:37.959 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-10T23:59:37.935+0000 7ff2bfde1640 1 --2- 172.21.15.165:0/2592733549 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff2b8105ee0 0x7ff2b81082d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:59:37.959 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-10T23:59:37.935+0000 7ff2bfde1640 1 --2- 172.21.15.165:0/2592733549 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff2b8100e10 0x7ff2b81011f0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:59:37.959 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-10T23:59:37.935+0000 7ff2bfde1640 1 -- 172.21.15.165:0/2592733549 >> 172.21.15.165:0/2592733549 conn(0x7ff2b80fc960 msgr2=0x7ff2b80feda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:59:37.959 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-10T23:59:37.935+0000 7ff2bfde1640 1 -- 172.21.15.165:0/2592733549 shutdown_connections 2024-09-10T23:59:37.959 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-10T23:59:37.935+0000 7ff2bfde1640 1 -- 172.21.15.165:0/2592733549 wait complete. 2024-09-10T23:59:37.959 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-10T23:59:37.935+0000 7ff2bfde1640 1 Processor -- start 2024-09-10T23:59:37.959 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-10T23:59:37.936+0000 7ff2bfde1640 1 -- start start 2024-09-10T23:59:37.959 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-10T23:59:37.936+0000 7ff2bfde1640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff2b8100e10 0x7ff2b8198d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:59:37.959 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-10T23:59:37.936+0000 7ff2bfde1640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff2b8105ee0 0x7ff2b81993c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:59:37.959 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-10T23:59:37.936+0000 7ff2bfde1640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff2b8108810 0x7ff2b819a000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:59:37.960 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-10T23:59:37.936+0000 7ff2bfde1640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7ff2b810e670 con 0x7ff2b8108810 2024-09-10T23:59:37.960 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-10T23:59:37.936+0000 7ff2bfde1640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7ff2b810e4f0 con 0x7ff2b8100e10 2024-09-10T23:59:37.960 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-10T23:59:37.936+0000 7ff2bfde1640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7ff2b810e7f0 con 0x7ff2b8105ee0 2024-09-10T23:59:37.960 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-10T23:59:37.936+0000 7ff2bdb7d640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff2b8100e10 0x7ff2b8198d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:59:37.960 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-10T23:59:37.936+0000 7ff2bdb7d640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff2b8100e10 0x7ff2b8198d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:36814/0 (socket says 172.21.15.165:36814) 2024-09-10T23:59:37.960 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-10T23:59:37.936+0000 7ff2bdb7d640 1 -- 172.21.15.165:0/2939432058 learned_addr learned my addr 172.21.15.165:0/2939432058 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-09-10T23:59:37.960 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-10T23:59:37.936+0000 7ff2bd37c640 1 --2- 172.21.15.165:0/2939432058 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff2b8105ee0 0x7ff2b81993c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:59:37.960 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-10T23:59:37.936+0000 7ff2be37e640 1 --2- 172.21.15.165:0/2939432058 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff2b8108810 0x7ff2b819a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:59:37.960 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-10T23:59:37.936+0000 7ff2bdb7d640 1 -- 172.21.15.165:0/2939432058 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff2b8105ee0 msgr2=0x7ff2b81993c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:59:37.960 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-10T23:59:37.936+0000 7ff2bdb7d640 1 --2- 172.21.15.165:0/2939432058 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff2b8105ee0 0x7ff2b81993c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:59:37.960 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-10T23:59:37.936+0000 7ff2bdb7d640 1 -- 172.21.15.165:0/2939432058 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff2b8108810 msgr2=0x7ff2b819a000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:59:37.961 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-10T23:59:37.936+0000 7ff2bdb7d640 1 --2- 172.21.15.165:0/2939432058 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff2b8108810 0x7ff2b819a000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:59:37.961 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-10T23:59:37.936+0000 7ff2bdb7d640 1 -- 172.21.15.165:0/2939432058 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2b819a8a0 con 0x7ff2b8100e10 2024-09-10T23:59:37.961 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-10T23:59:37.936+0000 7ff2bdb7d640 1 --2- 172.21.15.165:0/2939432058 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff2b8100e10 0x7ff2b8198d70 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7ff2b4005940 tx=0x7ff2b4004430 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:59:37.961 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-10T23:59:37.937+0000 7ff2aeffd640 1 -- 172.21.15.165:0/2939432058 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2b403f030 con 0x7ff2b8100e10 2024-09-10T23:59:37.961 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-10T23:59:37.937+0000 7ff2aeffd640 1 -- 172.21.15.165:0/2939432058 <== mon.2 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff2b4003cb0 con 0x7ff2b8100e10 2024-09-10T23:59:37.961 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-10T23:59:37.937+0000 7ff2bfde1640 1 -- 172.21.15.165:0/2939432058 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff2b819ab30 con 0x7ff2b8100e10 2024-09-10T23:59:37.961 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-10T23:59:37.937+0000 7ff2aeffd640 1 -- 172.21.15.165:0/2939432058 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2b4035070 con 0x7ff2b8100e10 2024-09-10T23:59:37.961 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-10T23:59:37.937+0000 7ff2bfde1640 1 -- 172.21.15.165:0/2939432058 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2b81a10a0 con 0x7ff2b8100e10 2024-09-10T23:59:37.961 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-10T23:59:37.938+0000 7ff2bfde1640 1 -- 172.21.15.165:0/2939432058 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff2b80fe450 con 0x7ff2b8100e10 2024-09-10T23:59:37.961 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-10T23:59:37.939+0000 7ff2aeffd640 1 -- 172.21.15.165:0/2939432058 <== mon.2 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 16) ==== 153250+0+0 (secure 0 0 0) 0x7ff2b403b460 con 0x7ff2b8100e10 2024-09-10T23:59:37.961 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-10T23:59:37.940+0000 7ff2aeffd640 1 --2- 172.21.15.165:0/2939432058 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7ff2a40b52e0 0x7ff2a40b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T23:59:37.961 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-10T23:59:37.940+0000 7ff2aeffd640 1 -- 172.21.15.165:0/2939432058 <== mon.2 v2:172.21.15.165:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 3669+0+0 (secure 0 0 0) 0x7ff2b4105860 con 0x7ff2b8100e10 2024-09-10T23:59:37.962 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-10T23:59:37.940+0000 7ff2bd37c640 1 --2- 172.21.15.165:0/2939432058 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7ff2a40b52e0 0x7ff2a40b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T23:59:37.962 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-10T23:59:37.940+0000 7ff2bd37c640 1 --2- 172.21.15.165:0/2939432058 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7ff2a40b52e0 0x7ff2a40b77a0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7ff2a80058c0 tx=0x7ff2a800a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T23:59:37.962 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-10T23:59:37.943+0000 7ff2aeffd640 1 -- 172.21.15.165:0/2939432058 <== mon.2 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7ff2b40cd940 con 0x7ff2b8100e10 2024-09-10T23:59:38.138 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-10T23:59:38.137+0000 7ff2bfde1640 1 -- 172.21.15.165:0/2939432058 --> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi165:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7ff2b8063590 con 0x7ff2a40b52e0 2024-09-10T23:59:38.532 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:59:38 smithi165 ceph-mon[35356]: purged_snaps scrub starts 2024-09-10T23:59:38.532 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:59:38 smithi165 ceph-mon[35356]: purged_snaps scrub ok 2024-09-10T23:59:38.532 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:59:38 smithi165 ceph-mon[35356]: osdmap e26: 4 total, 4 up, 4 in 2024-09-10T23:59:38.532 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:59:38 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-10T23:59:38.532 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:59:38 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-10T23:59:38.533 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:59:38 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-10T23:59:38.533 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:59:38 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-10T23:59:38.533 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:59:38 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-10T23:59:38.533 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:59:38 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T23:59:38.533 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:59:38 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T23:59:38.533 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:59:38 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-10T23:59:38.563 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:59:38 smithi012 ceph-mon[30333]: purged_snaps scrub starts 2024-09-10T23:59:38.563 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:59:38 smithi012 ceph-mon[30333]: purged_snaps scrub ok 2024-09-10T23:59:38.563 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:59:38 smithi012 ceph-mon[30333]: osdmap e26: 4 total, 4 up, 4 in 2024-09-10T23:59:38.563 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:59:38 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-10T23:59:38.563 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:59:38 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-10T23:59:38.563 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:59:38 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-10T23:59:38.563 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:59:38 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-10T23:59:38.563 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:59:38 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-10T23:59:38.563 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:59:38 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T23:59:38.564 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:59:38 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T23:59:38.564 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:59:38 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-10T23:59:38.580 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:59:38 smithi178 ceph-mon[35919]: purged_snaps scrub starts 2024-09-10T23:59:38.580 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:59:38 smithi178 ceph-mon[35919]: purged_snaps scrub ok 2024-09-10T23:59:38.581 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:59:38 smithi178 ceph-mon[35919]: osdmap e26: 4 total, 4 up, 4 in 2024-09-10T23:59:38.581 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:59:38 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-10T23:59:38.581 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:59:38 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-10T23:59:38.581 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:59:38 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-10T23:59:38.581 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:59:38 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-10T23:59:38.581 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:59:38 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-10T23:59:38.581 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:59:38 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T23:59:38.581 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:59:38 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T23:59:38.581 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:59:38 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-10T23:59:39.530 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:59:39 smithi165 ceph-mon[35356]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 521 MiB used, 357 GiB / 358 GiB avail 2024-09-10T23:59:39.531 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:59:39 smithi165 ceph-mon[35356]: Detected new or changed devices on smithi012 2024-09-10T23:59:39.531 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:59:39 smithi165 ceph-mon[35356]: Adjusting osd_memory_target on smithi012 to 8547M 2024-09-10T23:59:39.531 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:59:39 smithi165 ceph-mon[35356]: osdmap e27: 4 total, 4 up, 4 in 2024-09-10T23:59:39.531 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:59:39 smithi165 ceph-mon[35356]: from='client.24347 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi165:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T23:59:39.531 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:59:39 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-10T23:59:39.531 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:59:39 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-10T23:59:39.531 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:59:39 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T23:59:39.563 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:59:39 smithi012 ceph-mon[30333]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 521 MiB used, 357 GiB / 358 GiB avail 2024-09-10T23:59:39.563 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:59:39 smithi012 ceph-mon[30333]: Detected new or changed devices on smithi012 2024-09-10T23:59:39.563 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:59:39 smithi012 ceph-mon[30333]: Adjusting osd_memory_target on smithi012 to 8547M 2024-09-10T23:59:39.563 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:59:39 smithi012 ceph-mon[30333]: osdmap e27: 4 total, 4 up, 4 in 2024-09-10T23:59:39.563 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:59:39 smithi012 ceph-mon[30333]: from='client.24347 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi165:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T23:59:39.563 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:59:39 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-10T23:59:39.563 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:59:39 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-10T23:59:39.563 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:59:39 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T23:59:39.581 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:59:39 smithi178 ceph-mon[35919]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 521 MiB used, 357 GiB / 358 GiB avail 2024-09-10T23:59:39.581 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:59:39 smithi178 ceph-mon[35919]: Detected new or changed devices on smithi012 2024-09-10T23:59:39.581 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:59:39 smithi178 ceph-mon[35919]: Adjusting osd_memory_target on smithi012 to 8547M 2024-09-10T23:59:39.581 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:59:39 smithi178 ceph-mon[35919]: osdmap e27: 4 total, 4 up, 4 in 2024-09-10T23:59:39.581 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:59:39 smithi178 ceph-mon[35919]: from='client.24347 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi165:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T23:59:39.581 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:59:39 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-10T23:59:39.581 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:59:39 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-10T23:59:39.581 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:59:39 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T23:59:41.532 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:59:41 smithi165 ceph-mon[35356]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 520 MiB used, 357 GiB / 358 GiB avail; 142 KiB/s, 0 objects/s recovering 2024-09-10T23:59:41.532 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:59:41 smithi165 ceph-mon[35356]: from='client.? 172.21.15.165:0/2386016444' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "ad76ac70-20c0-44ae-9866-5f9cfce2334c"} : dispatch 2024-09-10T23:59:41.532 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:59:41 smithi165 ceph-mon[35356]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "ad76ac70-20c0-44ae-9866-5f9cfce2334c"} : dispatch 2024-09-10T23:59:41.532 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:59:41 smithi165 ceph-mon[35356]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ad76ac70-20c0-44ae-9866-5f9cfce2334c"}]': finished 2024-09-10T23:59:41.532 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:59:41 smithi165 ceph-mon[35356]: osdmap e28: 5 total, 4 up, 5 in 2024-09-10T23:59:41.532 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:59:41 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-10T23:59:41.533 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:59:41 smithi165 ceph-mon[35356]: from='client.? 172.21.15.165:0/1779326484' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-10T23:59:41.563 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:59:41 smithi012 ceph-mon[30333]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 520 MiB used, 357 GiB / 358 GiB avail; 142 KiB/s, 0 objects/s recovering 2024-09-10T23:59:41.563 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:59:41 smithi012 ceph-mon[30333]: from='client.? 172.21.15.165:0/2386016444' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "ad76ac70-20c0-44ae-9866-5f9cfce2334c"} : dispatch 2024-09-10T23:59:41.563 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:59:41 smithi012 ceph-mon[30333]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "ad76ac70-20c0-44ae-9866-5f9cfce2334c"} : dispatch 2024-09-10T23:59:41.563 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:59:41 smithi012 ceph-mon[30333]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ad76ac70-20c0-44ae-9866-5f9cfce2334c"}]': finished 2024-09-10T23:59:41.563 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:59:41 smithi012 ceph-mon[30333]: osdmap e28: 5 total, 4 up, 5 in 2024-09-10T23:59:41.563 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:59:41 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-10T23:59:41.563 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:59:41 smithi012 ceph-mon[30333]: from='client.? 172.21.15.165:0/1779326484' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-10T23:59:41.580 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:59:41 smithi178 ceph-mon[35919]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 520 MiB used, 357 GiB / 358 GiB avail; 142 KiB/s, 0 objects/s recovering 2024-09-10T23:59:41.580 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:59:41 smithi178 ceph-mon[35919]: from='client.? 172.21.15.165:0/2386016444' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "ad76ac70-20c0-44ae-9866-5f9cfce2334c"} : dispatch 2024-09-10T23:59:41.581 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:59:41 smithi178 ceph-mon[35919]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "ad76ac70-20c0-44ae-9866-5f9cfce2334c"} : dispatch 2024-09-10T23:59:41.581 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:59:41 smithi178 ceph-mon[35919]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ad76ac70-20c0-44ae-9866-5f9cfce2334c"}]': finished 2024-09-10T23:59:41.581 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:59:41 smithi178 ceph-mon[35919]: osdmap e28: 5 total, 4 up, 5 in 2024-09-10T23:59:41.581 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:59:41 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-10T23:59:41.581 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:59:41 smithi178 ceph-mon[35919]: from='client.? 172.21.15.165:0/1779326484' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-10T23:59:43.532 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:59:43 smithi165 ceph-mon[35356]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 114 KiB/s, 0 objects/s recovering 2024-09-10T23:59:43.563 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:59:43 smithi012 ceph-mon[30333]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 114 KiB/s, 0 objects/s recovering 2024-09-10T23:59:43.580 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:59:43 smithi178 ceph-mon[35919]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 114 KiB/s, 0 objects/s recovering 2024-09-10T23:59:45.532 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:59:45 smithi165 ceph-mon[35356]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 83 KiB/s, 0 objects/s recovering 2024-09-10T23:59:45.563 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:59:45 smithi012 ceph-mon[30333]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 83 KiB/s, 0 objects/s recovering 2024-09-10T23:59:45.580 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:59:45 smithi178 ceph-mon[35919]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 83 KiB/s, 0 objects/s recovering 2024-09-10T23:59:47.330 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:59:47 smithi178 ceph-mon[35919]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-10T23:59:47.452 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:59:47 smithi165 ceph-mon[35356]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-10T23:59:47.562 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:59:47 smithi012 ceph-mon[30333]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-10T23:59:49.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:59:49 smithi165 ceph-mon[35356]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-10T23:59:49.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:59:49 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.4"} : dispatch 2024-09-10T23:59:49.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:59:49 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T23:59:49.562 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:59:49 smithi012 ceph-mon[30333]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-10T23:59:49.563 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:59:49 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.4"} : dispatch 2024-09-10T23:59:49.563 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:59:49 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T23:59:49.580 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:59:49 smithi178 ceph-mon[35919]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-10T23:59:49.580 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:59:49 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.4"} : dispatch 2024-09-10T23:59:49.581 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:59:49 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T23:59:50.512 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:59:50 smithi165 ceph-mon[35356]: Deploying daemon osd.4 on smithi165 2024-09-10T23:59:50.562 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:59:50 smithi012 ceph-mon[30333]: Deploying daemon osd.4 on smithi165 2024-09-10T23:59:50.581 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:59:50 smithi178 ceph-mon[35919]: Deploying daemon osd.4 on smithi165 2024-09-10T23:59:51.512 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:59:51 smithi165 ceph-mon[35356]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-09-10T23:59:51.562 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:59:51 smithi012 ceph-mon[30333]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-09-10T23:59:51.580 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:59:51 smithi178 ceph-mon[35919]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-09-10T23:59:53.532 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:59:53 smithi165 ceph-mon[35356]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-09-10T23:59:53.563 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:59:53 smithi012 ceph-mon[30333]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-09-10T23:59:53.580 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:59:53 smithi178 ceph-mon[35919]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-09-10T23:59:55.532 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:59:55 smithi165 ceph-mon[35356]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-09-10T23:59:55.562 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:59:55 smithi012 ceph-mon[30333]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-09-10T23:59:55.580 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:59:55 smithi178 ceph-mon[35919]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-09-10T23:59:56.437 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:59:56 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T23:59:56.437 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:59:56 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-10T23:59:56.438 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:59:56 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-10T23:59:56.563 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:59:56 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T23:59:56.563 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:59:56 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-10T23:59:56.563 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:59:56 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-10T23:59:56.580 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:59:56 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T23:59:56.580 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:59:56 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-10T23:59:56.581 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:59:56 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-10T23:59:57.562 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:59:57 smithi012 ceph-mon[30333]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-09-10T23:59:57.580 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:59:57 smithi178 ceph-mon[35919]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-09-10T23:59:57.659 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:59:57 smithi165 ceph-mon[35356]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-09-10T23:59:59.273 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:59:58 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-10T23:59:59.273 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:59:58 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-10T23:59:59.273 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:59:58 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T23:59:59.273 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:59:58 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T23:59:59.273 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:59:58 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-10T23:59:59.273 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 10 23:59:58 smithi165 ceph-mon[35356]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-09-10T23:59:59.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:59:58 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-10T23:59:59.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:59:58 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-10T23:59:59.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:59:58 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T23:59:59.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:59:58 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T23:59:59.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:59:58 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-10T23:59:59.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 10 23:59:58 smithi012 ceph-mon[30333]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-09-10T23:59:59.320 INFO:teuthology.orchestra.run.smithi165.stdout:Created osd(s) 4 on host 'smithi165' 2024-09-10T23:59:59.320 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-10T23:59:59.318+0000 7ff2aeffd640 1 -- 172.21.15.165:0/2939432058 <== mgr.14150 v2:172.21.15.12:6800/3442939137 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7ff2b8063590 con 0x7ff2a40b52e0 2024-09-10T23:59:59.322 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-10T23:59:59.321+0000 7ff2bfde1640 1 -- 172.21.15.165:0/2939432058 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7ff2a40b52e0 msgr2=0x7ff2a40b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:59:59.322 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-10T23:59:59.321+0000 7ff2bfde1640 1 --2- 172.21.15.165:0/2939432058 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7ff2a40b52e0 0x7ff2a40b77a0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7ff2a80058c0 tx=0x7ff2a800a2d0 comp rx=0 tx=0).stop 2024-09-10T23:59:59.323 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-10T23:59:59.321+0000 7ff2bfde1640 1 -- 172.21.15.165:0/2939432058 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff2b8100e10 msgr2=0x7ff2b8198d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T23:59:59.323 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-10T23:59:59.321+0000 7ff2bfde1640 1 --2- 172.21.15.165:0/2939432058 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff2b8100e10 0x7ff2b8198d70 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7ff2b4005940 tx=0x7ff2b4004430 comp rx=0 tx=0).stop 2024-09-10T23:59:59.323 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-10T23:59:59.321+0000 7ff2bfde1640 1 -- 172.21.15.165:0/2939432058 shutdown_connections 2024-09-10T23:59:59.323 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-10T23:59:59.321+0000 7ff2bfde1640 1 --2- 172.21.15.165:0/2939432058 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7ff2a40b52e0 0x7ff2a40b77a0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:59:59.323 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-10T23:59:59.321+0000 7ff2bfde1640 1 --2- 172.21.15.165:0/2939432058 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff2b8108810 0x7ff2b819a000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:59:59.323 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-10T23:59:59.321+0000 7ff2bfde1640 1 --2- 172.21.15.165:0/2939432058 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff2b8105ee0 0x7ff2b81993c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:59:59.323 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-10T23:59:59.321+0000 7ff2bfde1640 1 --2- 172.21.15.165:0/2939432058 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff2b8100e10 0x7ff2b8198d70 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T23:59:59.323 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-10T23:59:59.321+0000 7ff2bfde1640 1 -- 172.21.15.165:0/2939432058 >> 172.21.15.165:0/2939432058 conn(0x7ff2b80fc960 msgr2=0x7ff2b8106ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T23:59:59.323 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-10T23:59:59.321+0000 7ff2bfde1640 1 -- 172.21.15.165:0/2939432058 shutdown_connections 2024-09-10T23:59:59.323 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-10T23:59:59.321+0000 7ff2bfde1640 1 -- 172.21.15.165:0/2939432058 wait complete. 2024-09-10T23:59:59.330 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:59:58 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-10T23:59:59.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:59:58 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-10T23:59:59.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:59:58 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T23:59:59.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:59:58 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T23:59:59.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:59:58 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-10T23:59:59.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 10 23:59:58 smithi178 ceph-mon[35919]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-09-11T00:00:00.018 DEBUG:teuthology.orchestra.run.smithi165:osd.4> sudo journalctl -f -n 0 -u ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968@osd.4.service 2024-09-11T00:00:00.020 INFO:tasks.cephadm:Deploying osd.5 on smithi178 with /dev/vg_nvme/lv_3... 2024-09-11T00:00:00.020 DEBUG:teuthology.orchestra.run.smithi178:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ff3ad74a-6fcf-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_3 2024-09-11T00:00:00.254 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:00:00 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T00:00:00.254 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:00:00 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:00:00.254 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:00:00 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:00:00.254 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:00:00 smithi178 ceph-mon[35919]: overall HEALTH_OK 2024-09-11T00:00:00.257 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:00:00 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T00:00:00.257 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:00:00 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:00:00.257 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:00:00 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:00:00.257 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:00:00 smithi165 ceph-mon[35356]: overall HEALTH_OK 2024-09-11T00:00:00.286 INFO:teuthology.orchestra.run.smithi178.stderr:Inferring config /var/lib/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/mon.c/config 2024-09-11T00:00:00.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:00:00 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T00:00:00.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:00:00 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:00:00.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:00:00 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:00:00.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:00:00 smithi012 ceph-mon[30333]: overall HEALTH_OK 2024-09-11T00:00:01.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:00:01 smithi165 ceph-mon[35356]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-09-11T00:00:01.312 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:00:01 smithi012 ceph-mon[30333]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-09-11T00:00:01.330 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:00:01 smithi178 ceph-mon[35919]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-09-11T00:00:01.863 INFO:journalctl@ceph.osd.4.smithi165.stdout:Sep 11 00:00:01 smithi165 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-4[46433]: 2024-09-11T00:00:01.592+0000 7f8666e79640 -1 osd.4 0 log_to_monitors true 2024-09-11T00:00:02.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:00:02 smithi165 ceph-mon[35356]: from='osd.4 [v2:172.21.15.165:6808/1899879280,v1:172.21.15.165:6809/1899879280]' entity='osd.4' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]} : dispatch 2024-09-11T00:00:02.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:00:02 smithi165 ceph-mon[35356]: from='osd.4 ' entity='osd.4' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]} : dispatch 2024-09-11T00:00:02.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:00:02 smithi012 ceph-mon[30333]: from='osd.4 [v2:172.21.15.165:6808/1899879280,v1:172.21.15.165:6809/1899879280]' entity='osd.4' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]} : dispatch 2024-09-11T00:00:02.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:00:02 smithi012 ceph-mon[30333]: from='osd.4 ' entity='osd.4' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]} : dispatch 2024-09-11T00:00:02.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:00:02 smithi178 ceph-mon[35919]: from='osd.4 [v2:172.21.15.165:6808/1899879280,v1:172.21.15.165:6809/1899879280]' entity='osd.4' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]} : dispatch 2024-09-11T00:00:02.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:00:02 smithi178 ceph-mon[35919]: from='osd.4 ' entity='osd.4' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]} : dispatch 2024-09-11T00:00:02.744 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-09-11T00:00:02.770 DEBUG:teuthology.orchestra.run.smithi178:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ff3ad74a-6fcf-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi178:vg_nvme/lv_3 2024-09-11T00:00:03.013 INFO:teuthology.orchestra.run.smithi178.stderr:Inferring config /var/lib/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/mon.c/config 2024-09-11T00:00:03.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:00:03 smithi165 ceph-mon[35356]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-11T00:00:03.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:00:03 smithi165 ceph-mon[35356]: osdmap e29: 5 total, 4 up, 5 in 2024-09-11T00:00:03.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:00:03 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-11T00:00:03.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:00:03 smithi165 ceph-mon[35356]: from='osd.4 [v2:172.21.15.165:6808/1899879280,v1:172.21.15.165:6809/1899879280]' entity='osd.4' cmd={"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi165", "root=default"]} : dispatch 2024-09-11T00:00:03.283 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:00:03 smithi165 ceph-mon[35356]: from='osd.4 ' entity='osd.4' cmd={"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi165", "root=default"]} : dispatch 2024-09-11T00:00:03.283 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:00:03 smithi165 ceph-mon[35356]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-09-11T00:00:03.283 INFO:journalctl@ceph.osd.4.smithi165.stdout:Sep 11 00:00:03 smithi165 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-4[46433]: 2024-09-11T00:00:03.032+0000 7f8662dfc640 -1 osd.4 0 waiting for initial osdmap 2024-09-11T00:00:03.283 INFO:journalctl@ceph.osd.4.smithi165.stdout:Sep 11 00:00:03 smithi165 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-4[46433]: 2024-09-11T00:00:03.040+0000 7f865ec23640 -1 osd.4 30 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-11T00:00:03.295 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:00:03 smithi178 ceph-mon[35919]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-11T00:00:03.295 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:00:03 smithi178 ceph-mon[35919]: osdmap e29: 5 total, 4 up, 5 in 2024-09-11T00:00:03.295 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:00:03 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-11T00:00:03.295 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:00:03 smithi178 ceph-mon[35919]: from='osd.4 [v2:172.21.15.165:6808/1899879280,v1:172.21.15.165:6809/1899879280]' entity='osd.4' cmd={"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi165", "root=default"]} : dispatch 2024-09-11T00:00:03.295 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:00:03 smithi178 ceph-mon[35919]: from='osd.4 ' entity='osd.4' cmd={"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi165", "root=default"]} : dispatch 2024-09-11T00:00:03.295 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:00:03 smithi178 ceph-mon[35919]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-09-11T00:00:03.312 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:00:03 smithi012 ceph-mon[30333]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-11T00:00:03.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:00:03 smithi012 ceph-mon[30333]: osdmap e29: 5 total, 4 up, 5 in 2024-09-11T00:00:03.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:00:03 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-11T00:00:03.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:00:03 smithi012 ceph-mon[30333]: from='osd.4 [v2:172.21.15.165:6808/1899879280,v1:172.21.15.165:6809/1899879280]' entity='osd.4' cmd={"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi165", "root=default"]} : dispatch 2024-09-11T00:00:03.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:00:03 smithi012 ceph-mon[30333]: from='osd.4 ' entity='osd.4' cmd={"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi165", "root=default"]} : dispatch 2024-09-11T00:00:03.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:00:03 smithi012 ceph-mon[30333]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-09-11T00:00:03.519 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T00:00:03.499+0000 7f1ff28a4640 1 -- 172.21.15.178:0/3891183281 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1fec06c160 msgr2=0x7f1fec105600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:00:03.519 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T00:00:03.499+0000 7f1ff28a4640 1 --2- 172.21.15.178:0/3891183281 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1fec06c160 0x7f1fec105600 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f1fdc009900 tx=0x7f1fdc02a410 comp rx=0 tx=0).stop 2024-09-11T00:00:03.520 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T00:00:03.499+0000 7f1ff28a4640 1 -- 172.21.15.178:0/3891183281 shutdown_connections 2024-09-11T00:00:03.520 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T00:00:03.499+0000 7f1ff28a4640 1 --2- 172.21.15.178:0/3891183281 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1fec108740 0x7f1fec10ac00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:00:03.520 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T00:00:03.499+0000 7f1ff28a4640 1 --2- 172.21.15.178:0/3891183281 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1fec105b90 0x7f1fec108000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:00:03.520 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T00:00:03.499+0000 7f1ff28a4640 1 --2- 172.21.15.178:0/3891183281 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1fec06c160 0x7f1fec105600 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:00:03.520 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T00:00:03.499+0000 7f1ff28a4640 1 -- 172.21.15.178:0/3891183281 >> 172.21.15.178:0/3891183281 conn(0x7f1fec0fd1f0 msgr2=0x7f1fec0ff630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:00:03.520 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T00:00:03.499+0000 7f1ff28a4640 1 -- 172.21.15.178:0/3891183281 shutdown_connections 2024-09-11T00:00:03.520 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T00:00:03.499+0000 7f1ff28a4640 1 -- 172.21.15.178:0/3891183281 wait complete. 2024-09-11T00:00:03.520 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T00:00:03.500+0000 7f1ff28a4640 1 Processor -- start 2024-09-11T00:00:03.520 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T00:00:03.500+0000 7f1ff28a4640 1 -- start start 2024-09-11T00:00:03.520 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T00:00:03.500+0000 7f1ff28a4640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1fec06c160 0x7f1fec102d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:00:03.520 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T00:00:03.500+0000 7f1ff28a4640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1fec105b90 0x7f1fec1015a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:00:03.520 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T00:00:03.500+0000 7f1ff28a4640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1fec108740 0x7f1fec101ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:00:03.521 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T00:00:03.500+0000 7f1ff28a4640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f1fec10e690 con 0x7f1fec105b90 2024-09-11T00:00:03.521 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T00:00:03.500+0000 7f1ff28a4640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f1fec10e510 con 0x7f1fec06c160 2024-09-11T00:00:03.521 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T00:00:03.500+0000 7f1ff28a4640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f1fec10e810 con 0x7f1fec108740 2024-09-11T00:00:03.521 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T00:00:03.500+0000 7f1ff0e41640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1fec108740 0x7f1fec101ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:00:03.521 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T00:00:03.500+0000 7f1ff0e41640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1fec108740 0x7f1fec101ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:53480/0 (socket says 172.21.15.178:53480) 2024-09-11T00:00:03.521 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T00:00:03.500+0000 7f1ff0e41640 1 -- 172.21.15.178:0/2633458313 learned_addr learned my addr 172.21.15.178:0/2633458313 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-11T00:00:03.521 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T00:00:03.500+0000 7f1febfff640 1 --2- 172.21.15.178:0/2633458313 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1fec06c160 0x7f1fec102d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:00:03.521 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T00:00:03.501+0000 7f1ff0e41640 1 -- 172.21.15.178:0/2633458313 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1fec06c160 msgr2=0x7f1fec102d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:00:03.521 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T00:00:03.501+0000 7f1ff0e41640 1 --2- 172.21.15.178:0/2633458313 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1fec06c160 0x7f1fec102d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:00:03.522 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T00:00:03.501+0000 7f1ff0e41640 1 -- 172.21.15.178:0/2633458313 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1fec105b90 msgr2=0x7f1fec1015a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:00:03.522 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T00:00:03.501+0000 7f1ff0e41640 1 --2- 172.21.15.178:0/2633458313 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1fec105b90 0x7f1fec1015a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:00:03.522 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T00:00:03.501+0000 7f1ff0e41640 1 -- 172.21.15.178:0/2633458313 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1fec1023a0 con 0x7f1fec108740 2024-09-11T00:00:03.522 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T00:00:03.501+0000 7f1ff0e41640 1 --2- 172.21.15.178:0/2633458313 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1fec108740 0x7f1fec101ae0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f1fe400eaa0 tx=0x7f1fe400ef60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:00:03.522 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T00:00:03.501+0000 7f1fe97fa640 1 -- 172.21.15.178:0/2633458313 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fe400cdb0 con 0x7f1fec108740 2024-09-11T00:00:03.522 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T00:00:03.501+0000 7f1fe97fa640 1 -- 172.21.15.178:0/2633458313 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1fe4005020 con 0x7f1fec108740 2024-09-11T00:00:03.522 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T00:00:03.501+0000 7f1ff28a4640 1 -- 172.21.15.178:0/2633458313 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1fec1a2f50 con 0x7f1fec108740 2024-09-11T00:00:03.522 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T00:00:03.501+0000 7f1fe97fa640 1 -- 172.21.15.178:0/2633458313 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fe40116d0 con 0x7f1fec108740 2024-09-11T00:00:03.522 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T00:00:03.501+0000 7f1ff28a4640 1 -- 172.21.15.178:0/2633458313 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1fec1a33b0 con 0x7f1fec108740 2024-09-11T00:00:03.522 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T00:00:03.502+0000 7f1ff28a4640 1 -- 172.21.15.178:0/2633458313 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1fec0fece0 con 0x7f1fec108740 2024-09-11T00:00:03.522 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T00:00:03.503+0000 7f1fe97fa640 1 -- 172.21.15.178:0/2633458313 <== mon.1 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 16) ==== 153250+0+0 (secure 0 0 0) 0x7f1fe400f040 con 0x7f1fec108740 2024-09-11T00:00:03.523 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T00:00:03.504+0000 7f1fe97fa640 1 --2- 172.21.15.178:0/2633458313 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f1fd40b52e0 0x7f1fd40b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:00:03.523 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T00:00:03.505+0000 7f1fe97fa640 1 -- 172.21.15.178:0/2633458313 <== mon.1 v2:172.21.15.178:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 3821+0+0 (secure 0 0 0) 0x7f1fe40e4860 con 0x7f1fec108740 2024-09-11T00:00:03.523 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T00:00:03.505+0000 7f1febfff640 1 --2- 172.21.15.178:0/2633458313 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f1fd40b52e0 0x7f1fd40b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:00:03.523 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T00:00:03.505+0000 7f1febfff640 1 --2- 172.21.15.178:0/2633458313 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f1fd40b52e0 0x7f1fd40b77a0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f1fdc000c00 tx=0x7f1fdc0314a0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:00:03.523 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T00:00:03.509+0000 7f1fe97fa640 1 -- 172.21.15.178:0/2633458313 <== mon.1 v2:172.21.15.178:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f1fe4016030 con 0x7f1fec108740 2024-09-11T00:00:03.696 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T00:00:03.694+0000 7f1ff28a4640 1 -- 172.21.15.178:0/2633458313 --> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi178:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f1fec063590 con 0x7f1fd40b52e0 2024-09-11T00:00:04.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:00:04 smithi012 ceph-mon[30333]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi165", "root=default"]}]': finished 2024-09-11T00:00:04.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:00:04 smithi012 ceph-mon[30333]: osdmap e30: 5 total, 4 up, 5 in 2024-09-11T00:00:04.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:00:04 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-11T00:00:04.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:00:04 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-11T00:00:04.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:00:04 smithi012 ceph-mon[30333]: Detected new or changed devices on smithi165 2024-09-11T00:00:04.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:00:04 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:00:04.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:00:04 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:00:04.314 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:00:04 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-11T00:00:04.314 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:00:04 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-11T00:00:04.314 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:00:04 smithi012 ceph-mon[30333]: Adjusting osd_memory_target on smithi165 to 8544M 2024-09-11T00:00:04.314 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:00:04 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:00:04.314 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:00:04 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:00:04.314 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:00:04 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T00:00:04.314 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:00:04 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:00:04.314 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:00:04 smithi012 ceph-mon[30333]: from='client.24370 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi178:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T00:00:04.314 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:00:04 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-11T00:00:04.314 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:00:04 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-11T00:00:04.314 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:00:04 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:00:04.321 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:00:04 smithi178 ceph-mon[35919]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi165", "root=default"]}]': finished 2024-09-11T00:00:04.322 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:00:04 smithi178 ceph-mon[35919]: osdmap e30: 5 total, 4 up, 5 in 2024-09-11T00:00:04.322 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:00:04 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-11T00:00:04.322 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:00:04 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-11T00:00:04.322 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:00:04 smithi178 ceph-mon[35919]: Detected new or changed devices on smithi165 2024-09-11T00:00:04.322 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:00:04 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:00:04.322 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:00:04 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:00:04.322 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:00:04 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-11T00:00:04.322 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:00:04 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-11T00:00:04.322 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:00:04 smithi178 ceph-mon[35919]: Adjusting osd_memory_target on smithi165 to 8544M 2024-09-11T00:00:04.322 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:00:04 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:00:04.322 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:00:04 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:00:04.322 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:00:04 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T00:00:04.322 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:00:04 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:00:04.323 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:00:04 smithi178 ceph-mon[35919]: from='client.24370 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi178:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T00:00:04.323 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:00:04 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-11T00:00:04.323 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:00:04 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-11T00:00:04.323 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:00:04 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:00:04.532 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:00:04 smithi165 ceph-mon[35356]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi165", "root=default"]}]': finished 2024-09-11T00:00:04.533 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:00:04 smithi165 ceph-mon[35356]: osdmap e30: 5 total, 4 up, 5 in 2024-09-11T00:00:04.533 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:00:04 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-11T00:00:04.533 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:00:04 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-11T00:00:04.533 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:00:04 smithi165 ceph-mon[35356]: Detected new or changed devices on smithi165 2024-09-11T00:00:04.533 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:00:04 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:00:04.533 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:00:04 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:00:04.533 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:00:04 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-11T00:00:04.533 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:00:04 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-11T00:00:04.533 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:00:04 smithi165 ceph-mon[35356]: Adjusting osd_memory_target on smithi165 to 8544M 2024-09-11T00:00:04.533 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:00:04 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:00:04.533 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:00:04 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:00:04.533 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:00:04 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T00:00:04.533 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:00:04 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:00:04.534 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:00:04 smithi165 ceph-mon[35356]: from='client.24370 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi178:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T00:00:04.534 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:00:04 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-11T00:00:04.534 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:00:04 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-11T00:00:04.534 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:00:04 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:00:05.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:00:05 smithi012 ceph-mon[30333]: purged_snaps scrub starts 2024-09-11T00:00:05.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:00:05 smithi012 ceph-mon[30333]: purged_snaps scrub ok 2024-09-11T00:00:05.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:00:05 smithi012 ceph-mon[30333]: osd.4 [v2:172.21.15.165:6808/1899879280,v1:172.21.15.165:6809/1899879280] boot 2024-09-11T00:00:05.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:00:05 smithi012 ceph-mon[30333]: osdmap e31: 5 total, 5 up, 5 in 2024-09-11T00:00:05.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:00:05 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-11T00:00:05.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:00:05 smithi012 ceph-mon[30333]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-09-11T00:00:05.330 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:00:05 smithi178 ceph-mon[35919]: purged_snaps scrub starts 2024-09-11T00:00:05.330 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:00:05 smithi178 ceph-mon[35919]: purged_snaps scrub ok 2024-09-11T00:00:05.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:00:05 smithi178 ceph-mon[35919]: osd.4 [v2:172.21.15.165:6808/1899879280,v1:172.21.15.165:6809/1899879280] boot 2024-09-11T00:00:05.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:00:05 smithi178 ceph-mon[35919]: osdmap e31: 5 total, 5 up, 5 in 2024-09-11T00:00:05.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:00:05 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-11T00:00:05.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:00:05 smithi178 ceph-mon[35919]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-09-11T00:00:05.433 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:00:05 smithi165 ceph-mon[35356]: purged_snaps scrub starts 2024-09-11T00:00:05.434 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:00:05 smithi165 ceph-mon[35356]: purged_snaps scrub ok 2024-09-11T00:00:05.434 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:00:05 smithi165 ceph-mon[35356]: osd.4 [v2:172.21.15.165:6808/1899879280,v1:172.21.15.165:6809/1899879280] boot 2024-09-11T00:00:05.434 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:00:05 smithi165 ceph-mon[35356]: osdmap e31: 5 total, 5 up, 5 in 2024-09-11T00:00:05.434 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:00:05 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-11T00:00:05.434 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:00:05 smithi165 ceph-mon[35356]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-09-11T00:00:06.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:00:06 smithi012 ceph-mon[30333]: osdmap e32: 5 total, 5 up, 5 in 2024-09-11T00:00:06.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:00:06 smithi178 ceph-mon[35919]: osdmap e32: 5 total, 5 up, 5 in 2024-09-11T00:00:06.532 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:00:06 smithi165 ceph-mon[35356]: osdmap e32: 5 total, 5 up, 5 in 2024-09-11T00:00:07.080 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:00:07 smithi178 ceph-mon[35919]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-09-11T00:00:07.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:00:07 smithi178 ceph-mon[35919]: from='client.? 172.21.15.178:0/3545393247' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "aafda2cc-62fe-4fcb-9a96-8732e0b501d5"} : dispatch 2024-09-11T00:00:07.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:00:07 smithi178 ceph-mon[35919]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "aafda2cc-62fe-4fcb-9a96-8732e0b501d5"} : dispatch 2024-09-11T00:00:07.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:00:07 smithi178 ceph-mon[35919]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "aafda2cc-62fe-4fcb-9a96-8732e0b501d5"}]': finished 2024-09-11T00:00:07.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:00:07 smithi178 ceph-mon[35919]: osdmap e33: 6 total, 5 up, 6 in 2024-09-11T00:00:07.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:00:07 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-11T00:00:07.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:00:07 smithi178 ceph-mon[35919]: from='client.? 172.21.15.178:0/2543299938' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-11T00:00:07.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:00:07 smithi012 ceph-mon[30333]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-09-11T00:00:07.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:00:07 smithi012 ceph-mon[30333]: from='client.? 172.21.15.178:0/3545393247' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "aafda2cc-62fe-4fcb-9a96-8732e0b501d5"} : dispatch 2024-09-11T00:00:07.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:00:07 smithi012 ceph-mon[30333]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "aafda2cc-62fe-4fcb-9a96-8732e0b501d5"} : dispatch 2024-09-11T00:00:07.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:00:07 smithi012 ceph-mon[30333]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "aafda2cc-62fe-4fcb-9a96-8732e0b501d5"}]': finished 2024-09-11T00:00:07.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:00:07 smithi012 ceph-mon[30333]: osdmap e33: 6 total, 5 up, 6 in 2024-09-11T00:00:07.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:00:07 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-11T00:00:07.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:00:07 smithi012 ceph-mon[30333]: from='client.? 172.21.15.178:0/2543299938' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-11T00:00:07.532 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:00:07 smithi165 ceph-mon[35356]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-09-11T00:00:07.532 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:00:07 smithi165 ceph-mon[35356]: from='client.? 172.21.15.178:0/3545393247' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "aafda2cc-62fe-4fcb-9a96-8732e0b501d5"} : dispatch 2024-09-11T00:00:07.532 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:00:07 smithi165 ceph-mon[35356]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "aafda2cc-62fe-4fcb-9a96-8732e0b501d5"} : dispatch 2024-09-11T00:00:07.532 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:00:07 smithi165 ceph-mon[35356]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "aafda2cc-62fe-4fcb-9a96-8732e0b501d5"}]': finished 2024-09-11T00:00:07.532 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:00:07 smithi165 ceph-mon[35356]: osdmap e33: 6 total, 5 up, 6 in 2024-09-11T00:00:07.533 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:00:07 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-11T00:00:07.533 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:00:07 smithi165 ceph-mon[35356]: from='client.? 172.21.15.178:0/2543299938' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-11T00:00:09.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:00:09 smithi012 ceph-mon[30333]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-09-11T00:00:09.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:00:09 smithi178 ceph-mon[35919]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-09-11T00:00:09.532 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:00:09 smithi165 ceph-mon[35356]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-09-11T00:00:11.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:00:11 smithi012 ceph-mon[30333]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-09-11T00:00:11.330 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:00:11 smithi178 ceph-mon[35919]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-09-11T00:00:11.532 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:00:11 smithi165 ceph-mon[35356]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-09-11T00:00:13.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:00:13 smithi012 ceph-mon[30333]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-09-11T00:00:13.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:00:13 smithi178 ceph-mon[35919]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-09-11T00:00:13.532 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:00:13 smithi165 ceph-mon[35356]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-09-11T00:00:15.329 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:00:15 smithi178 ceph-mon[35919]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-09-11T00:00:15.329 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:00:15 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.5"} : dispatch 2024-09-11T00:00:15.330 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:00:15 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:00:15.330 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:00:15 smithi178 ceph-mon[35919]: Deploying daemon osd.5 on smithi178 2024-09-11T00:00:15.532 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:00:15 smithi165 ceph-mon[35356]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-09-11T00:00:15.532 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:00:15 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.5"} : dispatch 2024-09-11T00:00:15.532 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:00:15 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:00:15.532 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:00:15 smithi165 ceph-mon[35356]: Deploying daemon osd.5 on smithi178 2024-09-11T00:00:15.562 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:00:15 smithi012 ceph-mon[30333]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-09-11T00:00:15.563 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:00:15 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.5"} : dispatch 2024-09-11T00:00:15.563 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:00:15 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:00:15.563 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:00:15 smithi012 ceph-mon[30333]: Deploying daemon osd.5 on smithi178 2024-09-11T00:00:17.320 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:00:17 smithi178 ceph-mon[35919]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-09-11T00:00:17.532 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:00:17 smithi165 ceph-mon[35356]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-09-11T00:00:17.562 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:00:17 smithi012 ceph-mon[30333]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-09-11T00:00:19.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:00:19 smithi178 ceph-mon[35919]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-09-11T00:00:19.532 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:00:19 smithi165 ceph-mon[35356]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-09-11T00:00:19.563 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:00:19 smithi012 ceph-mon[30333]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-09-11T00:00:21.330 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:00:21 smithi178 ceph-mon[35919]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-09-11T00:00:21.532 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:00:21 smithi165 ceph-mon[35356]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-09-11T00:00:21.563 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:00:21 smithi012 ceph-mon[30333]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-09-11T00:00:22.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:00:22 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T00:00:22.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:00:22 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:00:22.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:00:22 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:00:22.532 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:00:22 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T00:00:22.532 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:00:22 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:00:22.532 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:00:22 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:00:22.562 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:00:22 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T00:00:22.563 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:00:22 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:00:22.563 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:00:22 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:00:23.420 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:00:23 smithi178 ceph-mon[35919]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-09-11T00:00:23.532 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:00:23 smithi165 ceph-mon[35356]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-09-11T00:00:23.563 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:00:23 smithi012 ceph-mon[30333]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-09-11T00:00:25.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:00:24 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:00:25.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:00:24 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:00:25.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:00:24 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:00:25.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:00:24 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T00:00:25.033 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:00:24 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:00:25.039 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:00:24 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:00:25.039 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:00:24 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:00:25.039 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:00:24 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:00:25.039 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:00:24 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T00:00:25.039 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:00:24 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:00:25.063 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:00:24 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:00:25.063 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:00:24 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:00:25.063 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:00:24 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:00:25.063 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:00:24 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T00:00:25.063 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:00:24 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:00:25.087 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T00:00:25.086+0000 7f1fe97fa640 1 -- 172.21.15.178:0/2633458313 <== mgr.14150 v2:172.21.15.12:6800/3442939137 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f1fec063590 con 0x7f1fd40b52e0 2024-09-11T00:00:25.087 INFO:teuthology.orchestra.run.smithi178.stdout:Created osd(s) 5 on host 'smithi178' 2024-09-11T00:00:25.090 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T00:00:25.088+0000 7f1ff28a4640 1 -- 172.21.15.178:0/2633458313 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f1fd40b52e0 msgr2=0x7f1fd40b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:00:25.090 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T00:00:25.088+0000 7f1ff28a4640 1 --2- 172.21.15.178:0/2633458313 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f1fd40b52e0 0x7f1fd40b77a0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f1fdc000c00 tx=0x7f1fdc0314a0 comp rx=0 tx=0).stop 2024-09-11T00:00:25.090 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T00:00:25.088+0000 7f1ff28a4640 1 -- 172.21.15.178:0/2633458313 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1fec108740 msgr2=0x7f1fec101ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:00:25.090 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T00:00:25.088+0000 7f1ff28a4640 1 --2- 172.21.15.178:0/2633458313 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1fec108740 0x7f1fec101ae0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f1fe400eaa0 tx=0x7f1fe400ef60 comp rx=0 tx=0).stop 2024-09-11T00:00:25.090 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T00:00:25.088+0000 7f1ff28a4640 1 -- 172.21.15.178:0/2633458313 shutdown_connections 2024-09-11T00:00:25.090 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T00:00:25.088+0000 7f1ff28a4640 1 --2- 172.21.15.178:0/2633458313 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f1fd40b52e0 0x7f1fd40b77a0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:00:25.090 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T00:00:25.088+0000 7f1ff28a4640 1 --2- 172.21.15.178:0/2633458313 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1fec108740 0x7f1fec101ae0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:00:25.090 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T00:00:25.088+0000 7f1ff28a4640 1 --2- 172.21.15.178:0/2633458313 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1fec105b90 0x7f1fec1015a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:00:25.090 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T00:00:25.088+0000 7f1ff28a4640 1 --2- 172.21.15.178:0/2633458313 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1fec06c160 0x7f1fec102d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:00:25.090 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T00:00:25.088+0000 7f1ff28a4640 1 -- 172.21.15.178:0/2633458313 >> 172.21.15.178:0/2633458313 conn(0x7f1fec0fd1f0 msgr2=0x7f1fec103f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:00:25.091 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T00:00:25.088+0000 7f1ff28a4640 1 -- 172.21.15.178:0/2633458313 shutdown_connections 2024-09-11T00:00:25.091 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T00:00:25.088+0000 7f1ff28a4640 1 -- 172.21.15.178:0/2633458313 wait complete. 2024-09-11T00:00:25.755 DEBUG:teuthology.orchestra.run.smithi178:osd.5> sudo journalctl -f -n 0 -u ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968@osd.5.service 2024-09-11T00:00:25.758 INFO:tasks.cephadm:Deploying osd.6 on smithi012 with /dev/vg_nvme/lv_2... 2024-09-11T00:00:25.758 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ff3ad74a-6fcf-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_2 2024-09-11T00:00:25.977 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:00:25 smithi178 ceph-mon[35919]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-09-11T00:00:25.977 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:00:25 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T00:00:25.977 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:00:25 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:00:25.977 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:00:25 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:00:26.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:00:25 smithi165 ceph-mon[35356]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-09-11T00:00:26.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:00:25 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T00:00:26.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:00:25 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:00:26.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:00:25 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:00:26.062 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:00:25 smithi012 ceph-mon[30333]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-09-11T00:00:26.062 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:00:25 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T00:00:26.062 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:00:25 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:00:26.063 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:00:25 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:00:26.143 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:00:27.261 INFO:journalctl@ceph.osd.5.smithi178.stdout:Sep 11 00:00:27 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-5[47367]: 2024-09-11T00:00:27.104+0000 7fc6ea43d640 -1 osd.5 0 log_to_monitors true 2024-09-11T00:00:27.994 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:00:27 smithi012 ceph-mon[30333]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-09-11T00:00:27.994 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:00:27 smithi012 ceph-mon[30333]: from='osd.5 [v2:172.21.15.178:6808/3042228555,v1:172.21.15.178:6809/3042228555]' entity='osd.5' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]} : dispatch 2024-09-11T00:00:27.994 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:00:27 smithi012 ceph-mon[30333]: from='osd.5 ' entity='osd.5' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]} : dispatch 2024-09-11T00:00:28.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:00:27 smithi165 ceph-mon[35356]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-09-11T00:00:28.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:00:27 smithi165 ceph-mon[35356]: from='osd.5 [v2:172.21.15.178:6808/3042228555,v1:172.21.15.178:6809/3042228555]' entity='osd.5' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]} : dispatch 2024-09-11T00:00:28.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:00:27 smithi165 ceph-mon[35356]: from='osd.5 ' entity='osd.5' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]} : dispatch 2024-09-11T00:00:28.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:00:27 smithi178 ceph-mon[35919]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-09-11T00:00:28.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:00:27 smithi178 ceph-mon[35919]: from='osd.5 [v2:172.21.15.178:6808/3042228555,v1:172.21.15.178:6809/3042228555]' entity='osd.5' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]} : dispatch 2024-09-11T00:00:28.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:00:27 smithi178 ceph-mon[35919]: from='osd.5 ' entity='osd.5' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]} : dispatch 2024-09-11T00:00:28.831 INFO:journalctl@ceph.osd.5.smithi178.stdout:Sep 11 00:00:28 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-5[47367]: 2024-09-11T00:00:28.768+0000 7fc6e63c0640 -1 osd.5 0 waiting for initial osdmap 2024-09-11T00:00:28.831 INFO:journalctl@ceph.osd.5.smithi178.stdout:Sep 11 00:00:28 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-5[47367]: 2024-09-11T00:00:28.775+0000 7fc6e21e7640 -1 osd.5 35 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-11T00:00:28.831 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:00:28 smithi178 ceph-mon[35919]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-09-11T00:00:28.831 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:00:28 smithi178 ceph-mon[35919]: osdmap e34: 6 total, 5 up, 6 in 2024-09-11T00:00:28.831 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:00:28 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-11T00:00:28.831 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:00:28 smithi178 ceph-mon[35919]: from='osd.5 [v2:172.21.15.178:6808/3042228555,v1:172.21.15.178:6809/3042228555]' entity='osd.5' cmd={"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi178", "root=default"]} : dispatch 2024-09-11T00:00:28.832 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:00:28 smithi178 ceph-mon[35919]: from='osd.5 ' entity='osd.5' cmd={"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi178", "root=default"]} : dispatch 2024-09-11T00:00:28.832 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:00:28 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:00:28.832 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:00:28 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:00:28.832 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:00:28 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-11T00:00:28.832 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:00:28 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-11T00:00:28.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:00:28 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:00:28.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:00:28 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:00:28.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:00:28 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T00:00:28.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:00:28 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:00:29.033 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:00:28 smithi165 ceph-mon[35356]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-09-11T00:00:29.033 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:00:28 smithi165 ceph-mon[35356]: osdmap e34: 6 total, 5 up, 6 in 2024-09-11T00:00:29.033 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:00:28 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-11T00:00:29.034 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:00:28 smithi165 ceph-mon[35356]: from='osd.5 [v2:172.21.15.178:6808/3042228555,v1:172.21.15.178:6809/3042228555]' entity='osd.5' cmd={"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi178", "root=default"]} : dispatch 2024-09-11T00:00:29.034 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:00:28 smithi165 ceph-mon[35356]: from='osd.5 ' entity='osd.5' cmd={"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi178", "root=default"]} : dispatch 2024-09-11T00:00:29.034 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:00:28 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:00:29.034 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:00:28 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:00:29.034 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:00:28 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-11T00:00:29.035 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:00:28 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-11T00:00:29.035 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:00:28 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:00:29.035 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:00:28 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:00:29.035 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:00:28 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T00:00:29.035 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:00:28 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:00:29.063 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:00:28 smithi012 ceph-mon[30333]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-09-11T00:00:29.063 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:00:28 smithi012 ceph-mon[30333]: osdmap e34: 6 total, 5 up, 6 in 2024-09-11T00:00:29.063 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:00:28 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-11T00:00:29.064 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:00:28 smithi012 ceph-mon[30333]: from='osd.5 [v2:172.21.15.178:6808/3042228555,v1:172.21.15.178:6809/3042228555]' entity='osd.5' cmd={"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi178", "root=default"]} : dispatch 2024-09-11T00:00:29.064 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:00:28 smithi012 ceph-mon[30333]: from='osd.5 ' entity='osd.5' cmd={"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi178", "root=default"]} : dispatch 2024-09-11T00:00:29.064 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:00:28 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:00:29.064 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:00:28 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:00:29.064 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:00:28 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-11T00:00:29.065 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:00:28 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-11T00:00:29.065 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:00:28 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:00:29.065 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:00:28 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:00:29.065 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:00:28 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T00:00:29.066 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:00:28 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:00:29.265 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-11T00:00:29.299 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ff3ad74a-6fcf-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi012:vg_nvme/lv_2 2024-09-11T00:00:29.678 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:00:30.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:00:29 smithi165 ceph-mon[35356]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-09-11T00:00:30.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:00:29 smithi165 ceph-mon[35356]: Detected new or changed devices on smithi178 2024-09-11T00:00:30.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:00:29 smithi165 ceph-mon[35356]: Adjusting osd_memory_target on smithi178 to 8544M 2024-09-11T00:00:30.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:00:29 smithi165 ceph-mon[35356]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi178", "root=default"]}]': finished 2024-09-11T00:00:30.033 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:00:29 smithi165 ceph-mon[35356]: osdmap e35: 6 total, 5 up, 6 in 2024-09-11T00:00:30.033 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:00:29 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-11T00:00:30.033 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:00:29 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-11T00:00:30.063 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:00:29 smithi012 ceph-mon[30333]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-09-11T00:00:30.063 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:00:29 smithi012 ceph-mon[30333]: Detected new or changed devices on smithi178 2024-09-11T00:00:30.063 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:00:29 smithi012 ceph-mon[30333]: Adjusting osd_memory_target on smithi178 to 8544M 2024-09-11T00:00:30.063 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:00:29 smithi012 ceph-mon[30333]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi178", "root=default"]}]': finished 2024-09-11T00:00:30.063 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:00:29 smithi012 ceph-mon[30333]: osdmap e35: 6 total, 5 up, 6 in 2024-09-11T00:00:30.063 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:00:29 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-11T00:00:30.063 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:00:29 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-11T00:00:30.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:00:29 smithi178 ceph-mon[35919]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-09-11T00:00:30.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:00:29 smithi178 ceph-mon[35919]: Detected new or changed devices on smithi178 2024-09-11T00:00:30.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:00:29 smithi178 ceph-mon[35919]: Adjusting osd_memory_target on smithi178 to 8544M 2024-09-11T00:00:30.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:00:29 smithi178 ceph-mon[35919]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi178", "root=default"]}]': finished 2024-09-11T00:00:30.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:00:29 smithi178 ceph-mon[35919]: osdmap e35: 6 total, 5 up, 6 in 2024-09-11T00:00:30.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:00:29 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-11T00:00:30.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:00:29 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-11T00:00:30.335 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:00:30.316+0000 7f2b43800640 1 -- 172.21.15.12:0/2702107538 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2b3c077a10 msgr2=0x7f2b3c077e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:00:30.336 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:00:30.316+0000 7f2b43800640 1 --2- 172.21.15.12:0/2702107538 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2b3c077a10 0x7f2b3c077e10 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f2b2c009900 tx=0x7f2b2c02a3f0 comp rx=0 tx=0).stop 2024-09-11T00:00:30.336 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:00:30.317+0000 7f2b43800640 1 -- 172.21.15.12:0/2702107538 shutdown_connections 2024-09-11T00:00:30.336 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:00:30.317+0000 7f2b43800640 1 --2- 172.21.15.12:0/2702107538 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2b3c079530 0x7f2b3c079e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:00:30.336 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:00:30.317+0000 7f2b43800640 1 --2- 172.21.15.12:0/2702107538 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2b3c078b70 0x7f2b3c078ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:00:30.336 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:00:30.317+0000 7f2b43800640 1 --2- 172.21.15.12:0/2702107538 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2b3c077a10 0x7f2b3c077e10 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:00:30.336 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:00:30.317+0000 7f2b43800640 1 -- 172.21.15.12:0/2702107538 >> 172.21.15.12:0/2702107538 conn(0x7f2b3c100510 msgr2=0x7f2b3c102950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:00:30.336 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:00:30.317+0000 7f2b43800640 1 -- 172.21.15.12:0/2702107538 shutdown_connections 2024-09-11T00:00:30.336 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:00:30.317+0000 7f2b43800640 1 -- 172.21.15.12:0/2702107538 wait complete. 2024-09-11T00:00:30.336 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:00:30.318+0000 7f2b43800640 1 Processor -- start 2024-09-11T00:00:30.337 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:00:30.318+0000 7f2b43800640 1 -- start start 2024-09-11T00:00:30.337 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:00:30.318+0000 7f2b43800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2b3c077a10 0x7f2b3c19f420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:00:30.337 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:00:30.318+0000 7f2b43800640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2b3c078b70 0x7f2b3c19fa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:00:30.337 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:00:30.319+0000 7f2b43800640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2b3c079530 0x7f2b3c1a06b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:00:30.337 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:00:30.319+0000 7f2b43800640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f2b3c114d20 con 0x7f2b3c078b70 2024-09-11T00:00:30.337 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:00:30.319+0000 7f2b43800640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f2b3c114ba0 con 0x7f2b3c077a10 2024-09-11T00:00:30.337 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:00:30.319+0000 7f2b43800640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f2b3c114ea0 con 0x7f2b3c079530 2024-09-11T00:00:30.337 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:00:30.319+0000 7f2b33e00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2b3c078b70 0x7f2b3c19fa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:00:30.337 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:00:30.319+0000 7f2b33e00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2b3c078b70 0x7f2b3c19fa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:49094/0 (socket says 172.21.15.12:49094) 2024-09-11T00:00:30.338 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:00:30.319+0000 7f2b33e00640 1 -- 172.21.15.12:0/1694051336 learned_addr learned my addr 172.21.15.12:0/1694051336 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:00:30.338 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:00:30.319+0000 7f2b40e00640 1 --2- 172.21.15.12:0/1694051336 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2b3c077a10 0x7f2b3c19f420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:00:30.338 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:00:30.319+0000 7f2b41800640 1 --2- 172.21.15.12:0/1694051336 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2b3c079530 0x7f2b3c1a06b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:00:30.338 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:00:30.319+0000 7f2b33e00640 1 -- 172.21.15.12:0/1694051336 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2b3c079530 msgr2=0x7f2b3c1a06b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:00:30.338 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:00:30.319+0000 7f2b33e00640 1 --2- 172.21.15.12:0/1694051336 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2b3c079530 0x7f2b3c1a06b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:00:30.338 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:00:30.319+0000 7f2b33e00640 1 -- 172.21.15.12:0/1694051336 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2b3c077a10 msgr2=0x7f2b3c19f420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:00:30.338 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:00:30.319+0000 7f2b33e00640 1 --2- 172.21.15.12:0/1694051336 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2b3c077a10 0x7f2b3c19f420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:00:30.338 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:00:30.319+0000 7f2b33e00640 1 -- 172.21.15.12:0/1694051336 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b3c1a0f50 con 0x7f2b3c078b70 2024-09-11T00:00:30.338 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:00:30.320+0000 7f2b33e00640 1 --2- 172.21.15.12:0/1694051336 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2b3c078b70 0x7f2b3c19fa70 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f2b2400e9f0 tx=0x7f2b2400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:00:30.338 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:00:30.320+0000 7f2b3a000640 1 -- 172.21.15.12:0/1694051336 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b2400cd80 con 0x7f2b3c078b70 2024-09-11T00:00:30.338 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:00:30.320+0000 7f2b43800640 1 -- 172.21.15.12:0/1694051336 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2b3c1a1240 con 0x7f2b3c078b70 2024-09-11T00:00:30.338 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:00:30.320+0000 7f2b3a000640 1 -- 172.21.15.12:0/1694051336 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2b240054d0 con 0x7f2b3c078b70 2024-09-11T00:00:30.338 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:00:30.320+0000 7f2b3a000640 1 -- 172.21.15.12:0/1694051336 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b24018630 con 0x7f2b3c078b70 2024-09-11T00:00:30.339 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:00:30.320+0000 7f2b43800640 1 -- 172.21.15.12:0/1694051336 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2b3c1a7920 con 0x7f2b3c078b70 2024-09-11T00:00:30.339 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:00:30.321+0000 7f2b43800640 1 -- 172.21.15.12:0/1694051336 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2b3c111700 con 0x7f2b3c078b70 2024-09-11T00:00:30.339 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:00:30.323+0000 7f2b3a000640 1 -- 172.21.15.12:0/1694051336 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 16) ==== 153250+0+0 (secure 0 0 0) 0x7f2b24004e70 con 0x7f2b3c078b70 2024-09-11T00:00:30.339 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:00:30.325+0000 7f2b3a000640 1 --2- 172.21.15.12:0/1694051336 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f2b200b52e0 0x7f2b200b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:00:30.339 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:00:30.325+0000 7f2b3a000640 1 -- 172.21.15.12:0/1694051336 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 4566+0+0 (secure 0 0 0) 0x7f2b240e52d0 con 0x7f2b3c078b70 2024-09-11T00:00:30.339 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:00:30.325+0000 7f2b40e00640 1 --2- 172.21.15.12:0/1694051336 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f2b200b52e0 0x7f2b200b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:00:30.339 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:00:30.326+0000 7f2b40e00640 1 --2- 172.21.15.12:0/1694051336 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f2b200b52e0 0x7f2b200b77a0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f2b2c031500 tx=0x7f2b2c005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:00:30.339 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:00:30.329+0000 7f2b3a000640 1 -- 172.21.15.12:0/1694051336 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f2b240acf90 con 0x7f2b3c078b70 2024-09-11T00:00:30.541 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:00:30.539+0000 7f2b43800640 1 -- 172.21.15.12:0/1694051336 --> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi012:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f2b3c063590 con 0x7f2b200b52e0 2024-09-11T00:00:31.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:00:30 smithi165 ceph-mon[35356]: purged_snaps scrub starts 2024-09-11T00:00:31.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:00:30 smithi165 ceph-mon[35356]: purged_snaps scrub ok 2024-09-11T00:00:31.033 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:00:30 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-11T00:00:31.034 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:00:30 smithi165 ceph-mon[35356]: osd.5 [v2:172.21.15.178:6808/3042228555,v1:172.21.15.178:6809/3042228555] boot 2024-09-11T00:00:31.034 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:00:30 smithi165 ceph-mon[35356]: osdmap e36: 6 total, 6 up, 6 in 2024-09-11T00:00:31.034 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:00:30 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-11T00:00:31.034 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:00:30 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-11T00:00:31.034 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:00:30 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-11T00:00:31.034 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:00:30 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:00:31.063 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:00:30 smithi012 ceph-mon[30333]: purged_snaps scrub starts 2024-09-11T00:00:31.063 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:00:30 smithi012 ceph-mon[30333]: purged_snaps scrub ok 2024-09-11T00:00:31.063 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:00:30 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-11T00:00:31.063 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:00:30 smithi012 ceph-mon[30333]: osd.5 [v2:172.21.15.178:6808/3042228555,v1:172.21.15.178:6809/3042228555] boot 2024-09-11T00:00:31.063 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:00:30 smithi012 ceph-mon[30333]: osdmap e36: 6 total, 6 up, 6 in 2024-09-11T00:00:31.063 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:00:30 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-11T00:00:31.063 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:00:30 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-11T00:00:31.063 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:00:30 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-11T00:00:31.064 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:00:30 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:00:31.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:00:30 smithi178 ceph-mon[35919]: purged_snaps scrub starts 2024-09-11T00:00:31.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:00:30 smithi178 ceph-mon[35919]: purged_snaps scrub ok 2024-09-11T00:00:31.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:00:30 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-11T00:00:31.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:00:30 smithi178 ceph-mon[35919]: osd.5 [v2:172.21.15.178:6808/3042228555,v1:172.21.15.178:6809/3042228555] boot 2024-09-11T00:00:31.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:00:30 smithi178 ceph-mon[35919]: osdmap e36: 6 total, 6 up, 6 in 2024-09-11T00:00:31.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:00:30 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-11T00:00:31.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:00:30 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-11T00:00:31.082 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:00:30 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-11T00:00:31.082 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:00:30 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:00:32.063 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:00:31 smithi012 ceph-mon[30333]: pgmap v152: 1 pgs: 1 peering; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-09-11T00:00:32.063 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:00:31 smithi012 ceph-mon[30333]: from='client.14445 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi012:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T00:00:32.063 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:00:31 smithi012 ceph-mon[30333]: osdmap e37: 6 total, 6 up, 6 in 2024-09-11T00:00:32.080 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:00:31 smithi178 ceph-mon[35919]: pgmap v152: 1 pgs: 1 peering; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-09-11T00:00:32.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:00:31 smithi178 ceph-mon[35919]: from='client.14445 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi012:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T00:00:32.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:00:31 smithi178 ceph-mon[35919]: osdmap e37: 6 total, 6 up, 6 in 2024-09-11T00:00:32.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:00:31 smithi165 ceph-mon[35356]: pgmap v152: 1 pgs: 1 peering; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-09-11T00:00:32.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:00:31 smithi165 ceph-mon[35356]: from='client.14445 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi012:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T00:00:32.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:00:31 smithi165 ceph-mon[35356]: osdmap e37: 6 total, 6 up, 6 in 2024-09-11T00:00:33.046 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:00:32 smithi012 ceph-mon[30333]: osdmap e38: 6 total, 6 up, 6 in 2024-09-11T00:00:33.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:00:32 smithi178 ceph-mon[35919]: osdmap e38: 6 total, 6 up, 6 in 2024-09-11T00:00:33.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:00:32 smithi165 ceph-mon[35356]: osdmap e38: 6 total, 6 up, 6 in 2024-09-11T00:00:34.049 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:00:33 smithi012 ceph-mon[30333]: pgmap v155: 1 pgs: 1 peering; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-09-11T00:00:34.049 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:00:33 smithi012 ceph-mon[30333]: from='client.? 172.21.15.12:0/3721611422' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "375eca52-efe3-42a5-bbb1-d7d392d71f15"} : dispatch 2024-09-11T00:00:34.049 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:00:33 smithi012 ceph-mon[30333]: from='client.? 172.21.15.12:0/3721611422' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "375eca52-efe3-42a5-bbb1-d7d392d71f15"}]': finished 2024-09-11T00:00:34.049 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:00:33 smithi012 ceph-mon[30333]: osdmap e39: 7 total, 6 up, 7 in 2024-09-11T00:00:34.049 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:00:33 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-11T00:00:34.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:00:33 smithi178 ceph-mon[35919]: pgmap v155: 1 pgs: 1 peering; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-09-11T00:00:34.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:00:33 smithi178 ceph-mon[35919]: from='client.? 172.21.15.12:0/3721611422' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "375eca52-efe3-42a5-bbb1-d7d392d71f15"} : dispatch 2024-09-11T00:00:34.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:00:33 smithi178 ceph-mon[35919]: from='client.? 172.21.15.12:0/3721611422' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "375eca52-efe3-42a5-bbb1-d7d392d71f15"}]': finished 2024-09-11T00:00:34.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:00:33 smithi178 ceph-mon[35919]: osdmap e39: 7 total, 6 up, 7 in 2024-09-11T00:00:34.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:00:33 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-11T00:00:34.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:00:33 smithi165 ceph-mon[35356]: pgmap v155: 1 pgs: 1 peering; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-09-11T00:00:34.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:00:33 smithi165 ceph-mon[35356]: from='client.? 172.21.15.12:0/3721611422' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "375eca52-efe3-42a5-bbb1-d7d392d71f15"} : dispatch 2024-09-11T00:00:34.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:00:33 smithi165 ceph-mon[35356]: from='client.? 172.21.15.12:0/3721611422' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "375eca52-efe3-42a5-bbb1-d7d392d71f15"}]': finished 2024-09-11T00:00:34.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:00:33 smithi165 ceph-mon[35356]: osdmap e39: 7 total, 6 up, 7 in 2024-09-11T00:00:34.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:00:33 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-11T00:00:35.062 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:00:34 smithi012 ceph-mon[30333]: from='client.? 172.21.15.12:0/3233020988' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-11T00:00:35.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:00:34 smithi178 ceph-mon[35919]: from='client.? 172.21.15.12:0/3233020988' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-11T00:00:35.183 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:00:34 smithi165 ceph-mon[35356]: from='client.? 172.21.15.12:0/3233020988' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-11T00:00:36.062 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:00:35 smithi012 ceph-mon[30333]: pgmap v157: 1 pgs: 1 peering; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-09-11T00:00:36.080 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:00:35 smithi178 ceph-mon[35919]: pgmap v157: 1 pgs: 1 peering; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-09-11T00:00:36.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:00:35 smithi165 ceph-mon[35356]: pgmap v157: 1 pgs: 1 peering; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-09-11T00:00:38.063 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:00:37 smithi012 ceph-mon[30333]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 175 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-11T00:00:38.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:00:37 smithi178 ceph-mon[35919]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 175 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-11T00:00:38.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:00:37 smithi165 ceph-mon[35356]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 175 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-11T00:00:40.063 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:00:39 smithi012 ceph-mon[30333]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 79 KiB/s, 0 objects/s recovering 2024-09-11T00:00:40.080 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:00:39 smithi178 ceph-mon[35919]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 79 KiB/s, 0 objects/s recovering 2024-09-11T00:00:40.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:00:39 smithi165 ceph-mon[35356]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 79 KiB/s, 0 objects/s recovering 2024-09-11T00:00:41.946 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:00:41 smithi012 ceph-mon[30333]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 70 KiB/s, 0 objects/s recovering 2024-09-11T00:00:42.080 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:00:41 smithi178 ceph-mon[35919]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 70 KiB/s, 0 objects/s recovering 2024-09-11T00:00:42.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:00:41 smithi165 ceph-mon[35356]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 70 KiB/s, 0 objects/s recovering 2024-09-11T00:00:43.869 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:00:43 smithi012 ceph-mon[30333]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-11T00:00:43.869 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:00:43 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.6"} : dispatch 2024-09-11T00:00:43.869 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:00:43 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:00:44.080 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:00:43 smithi178 ceph-mon[35919]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-11T00:00:44.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:00:43 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.6"} : dispatch 2024-09-11T00:00:44.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:00:43 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:00:44.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:00:43 smithi165 ceph-mon[35356]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-11T00:00:44.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:00:43 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.6"} : dispatch 2024-09-11T00:00:44.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:00:43 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:00:45.063 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:00:44 smithi012 ceph-mon[30333]: Deploying daemon osd.6 on smithi012 2024-09-11T00:00:45.080 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:00:44 smithi178 ceph-mon[35919]: Deploying daemon osd.6 on smithi012 2024-09-11T00:00:45.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:00:44 smithi165 ceph-mon[35356]: Deploying daemon osd.6 on smithi012 2024-09-11T00:00:46.080 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:00:45 smithi178 ceph-mon[35919]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 56 KiB/s, 0 objects/s recovering 2024-09-11T00:00:46.222 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:00:45 smithi012 ceph-mon[30333]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 56 KiB/s, 0 objects/s recovering 2024-09-11T00:00:46.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:00:45 smithi165 ceph-mon[35356]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 56 KiB/s, 0 objects/s recovering 2024-09-11T00:00:48.063 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:00:47 smithi012 ceph-mon[30333]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-11T00:00:48.080 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:00:47 smithi178 ceph-mon[35919]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-11T00:00:48.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:00:47 smithi165 ceph-mon[35356]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-11T00:00:50.040 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:00:49 smithi012 ceph-mon[30333]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-09-11T00:00:50.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:00:49 smithi165 ceph-mon[35356]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-09-11T00:00:50.330 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:00:49 smithi178 ceph-mon[35919]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-09-11T00:00:51.120 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:00:50 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T00:00:51.120 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:00:50 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:00:51.120 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:00:50 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:00:51.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:00:50 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T00:00:51.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:00:50 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:00:51.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:00:50 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:00:51.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:00:50 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T00:00:51.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:00:50 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:00:51.333 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:00:50 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:00:52.140 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:00:51 smithi012 ceph-mon[30333]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-09-11T00:00:52.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:00:51 smithi165 ceph-mon[35356]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-09-11T00:00:52.330 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:00:51 smithi178 ceph-mon[35919]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-09-11T00:00:54.063 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:00:53 smithi012 ceph-mon[30333]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-09-11T00:00:54.063 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:00:53 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:00:54.063 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:00:53 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:00:54.063 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:00:53 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:00:54.063 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:00:53 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T00:00:54.063 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:00:53 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:00:54.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:00:53 smithi165 ceph-mon[35356]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-09-11T00:00:54.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:00:53 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:00:54.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:00:53 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:00:54.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:00:53 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:00:54.283 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:00:53 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T00:00:54.283 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:00:53 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:00:54.330 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:00:53 smithi178 ceph-mon[35919]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-09-11T00:00:54.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:00:53 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:00:54.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:00:53 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:00:54.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:00:53 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:00:54.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:00:53 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T00:00:54.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:00:53 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:00:55.109 INFO:teuthology.orchestra.run.smithi012.stdout:Created osd(s) 6 on host 'smithi012' 2024-09-11T00:00:55.109 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:00:55.106+0000 7f2b3a000640 1 -- 172.21.15.12:0/1694051336 <== mgr.14150 v2:172.21.15.12:6800/3442939137 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f2b3c063590 con 0x7f2b200b52e0 2024-09-11T00:00:55.112 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:00:55.109+0000 7f2b43800640 1 -- 172.21.15.12:0/1694051336 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f2b200b52e0 msgr2=0x7f2b200b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:00:55.112 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:00:55.110+0000 7f2b43800640 1 --2- 172.21.15.12:0/1694051336 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f2b200b52e0 0x7f2b200b77a0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f2b2c031500 tx=0x7f2b2c005a20 comp rx=0 tx=0).stop 2024-09-11T00:00:55.112 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:00:55.110+0000 7f2b43800640 1 -- 172.21.15.12:0/1694051336 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2b3c078b70 msgr2=0x7f2b3c19fa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:00:55.112 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:00:55.110+0000 7f2b43800640 1 --2- 172.21.15.12:0/1694051336 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2b3c078b70 0x7f2b3c19fa70 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f2b2400e9f0 tx=0x7f2b2400edf0 comp rx=0 tx=0).stop 2024-09-11T00:00:55.112 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:00:55.110+0000 7f2b43800640 1 -- 172.21.15.12:0/1694051336 shutdown_connections 2024-09-11T00:00:55.112 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:00:55.110+0000 7f2b43800640 1 --2- 172.21.15.12:0/1694051336 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f2b200b52e0 0x7f2b200b77a0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:00:55.112 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:00:55.110+0000 7f2b43800640 1 --2- 172.21.15.12:0/1694051336 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2b3c079530 0x7f2b3c1a06b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:00:55.112 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:00:55.110+0000 7f2b43800640 1 --2- 172.21.15.12:0/1694051336 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2b3c078b70 0x7f2b3c19fa70 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:00:55.112 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:00:55.110+0000 7f2b43800640 1 --2- 172.21.15.12:0/1694051336 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2b3c077a10 0x7f2b3c19f420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:00:55.113 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:00:55.110+0000 7f2b43800640 1 -- 172.21.15.12:0/1694051336 >> 172.21.15.12:0/1694051336 conn(0x7f2b3c100510 msgr2=0x7f2b3c1020e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:00:55.113 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:00:55.110+0000 7f2b43800640 1 -- 172.21.15.12:0/1694051336 shutdown_connections 2024-09-11T00:00:55.113 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:00:55.110+0000 7f2b43800640 1 -- 172.21.15.12:0/1694051336 wait complete. 2024-09-11T00:00:55.848 DEBUG:teuthology.orchestra.run.smithi012:osd.6> sudo journalctl -f -n 0 -u ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968@osd.6.service 2024-09-11T00:00:55.852 INFO:tasks.cephadm:Deploying osd.7 on smithi165 with /dev/vg_nvme/lv_2... 2024-09-11T00:00:55.852 DEBUG:teuthology.orchestra.run.smithi165:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ff3ad74a-6fcf-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_2 2024-09-11T00:00:56.062 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:00:55 smithi012 ceph-mon[30333]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-09-11T00:00:56.063 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:00:55 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T00:00:56.063 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:00:55 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:00:56.063 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:00:55 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:00:56.108 INFO:teuthology.orchestra.run.smithi165.stderr:Inferring config /var/lib/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T00:00:56.228 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:00:55 smithi165 ceph-mon[35356]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-09-11T00:00:56.229 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:00:55 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T00:00:56.229 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:00:55 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:00:56.229 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:00:55 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:00:56.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:00:55 smithi178 ceph-mon[35919]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-09-11T00:00:56.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:00:55 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T00:00:56.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:00:55 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:00:56.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:00:55 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:00:56.773 INFO:journalctl@ceph.osd.6.smithi012.stdout:Sep 11 00:00:56 smithi012 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-6[59757]: 2024-09-11T00:00:56.394+0000 7f2e7e634640 -1 osd.6 0 log_to_monitors true 2024-09-11T00:00:57.062 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:00:56 smithi012 ceph-mon[30333]: from='osd.6 [v2:172.21.15.12:6818/3868322467,v1:172.21.15.12:6819/3868322467]' entity='osd.6' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]} : dispatch 2024-09-11T00:00:57.176 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:00:56 smithi165 ceph-mon[35356]: from='osd.6 [v2:172.21.15.12:6818/3868322467,v1:172.21.15.12:6819/3868322467]' entity='osd.6' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]} : dispatch 2024-09-11T00:00:57.330 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:00:56 smithi178 ceph-mon[35919]: from='osd.6 [v2:172.21.15.12:6818/3868322467,v1:172.21.15.12:6819/3868322467]' entity='osd.6' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]} : dispatch 2024-09-11T00:00:58.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:00:57 smithi165 ceph-mon[35356]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-09-11T00:00:58.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:00:57 smithi165 ceph-mon[35356]: from='osd.6 [v2:172.21.15.12:6818/3868322467,v1:172.21.15.12:6819/3868322467]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-09-11T00:00:58.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:00:57 smithi165 ceph-mon[35356]: osdmap e40: 7 total, 6 up, 7 in 2024-09-11T00:00:58.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:00:57 smithi165 ceph-mon[35356]: from='osd.6 [v2:172.21.15.12:6818/3868322467,v1:172.21.15.12:6819/3868322467]' entity='osd.6' cmd={"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi012", "root=default"]} : dispatch 2024-09-11T00:00:58.283 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:00:57 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-11T00:00:58.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:00:57 smithi012 ceph-mon[30333]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-09-11T00:00:58.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:00:57 smithi012 ceph-mon[30333]: from='osd.6 [v2:172.21.15.12:6818/3868322467,v1:172.21.15.12:6819/3868322467]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-09-11T00:00:58.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:00:57 smithi012 ceph-mon[30333]: osdmap e40: 7 total, 6 up, 7 in 2024-09-11T00:00:58.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:00:57 smithi012 ceph-mon[30333]: from='osd.6 [v2:172.21.15.12:6818/3868322467,v1:172.21.15.12:6819/3868322467]' entity='osd.6' cmd={"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi012", "root=default"]} : dispatch 2024-09-11T00:00:58.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:00:57 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-11T00:00:58.330 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:00:57 smithi178 ceph-mon[35919]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-09-11T00:00:58.330 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:00:57 smithi178 ceph-mon[35919]: from='osd.6 [v2:172.21.15.12:6818/3868322467,v1:172.21.15.12:6819/3868322467]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-09-11T00:00:58.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:00:57 smithi178 ceph-mon[35919]: osdmap e40: 7 total, 6 up, 7 in 2024-09-11T00:00:58.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:00:57 smithi178 ceph-mon[35919]: from='osd.6 [v2:172.21.15.12:6818/3868322467,v1:172.21.15.12:6819/3868322467]' entity='osd.6' cmd={"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi012", "root=default"]} : dispatch 2024-09-11T00:00:58.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:00:57 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-11T00:00:58.679 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-09-11T00:00:58.705 DEBUG:teuthology.orchestra.run.smithi165:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ff3ad74a-6fcf-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi165:vg_nvme/lv_2 2024-09-11T00:00:58.958 INFO:teuthology.orchestra.run.smithi165.stderr:Inferring config /var/lib/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T00:00:58.979 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:00:58 smithi165 ceph-mon[35356]: from='osd.6 [v2:172.21.15.12:6818/3868322467,v1:172.21.15.12:6819/3868322467]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi012", "root=default"]}]': finished 2024-09-11T00:00:58.979 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:00:58 smithi165 ceph-mon[35356]: osdmap e41: 7 total, 6 up, 7 in 2024-09-11T00:00:58.979 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:00:58 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-11T00:00:58.979 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:00:58 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-11T00:00:58.979 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:00:58 smithi165 ceph-mon[35356]: from='osd.6 [v2:172.21.15.12:6818/3868322467,v1:172.21.15.12:6819/3868322467]' entity='osd.6' 2024-09-11T00:00:59.028 INFO:journalctl@ceph.osd.6.smithi012.stdout:Sep 11 00:00:58 smithi012 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-6[59757]: 2024-09-11T00:00:58.689+0000 7f2e79200640 -1 osd.6 0 waiting for initial osdmap 2024-09-11T00:00:59.029 INFO:journalctl@ceph.osd.6.smithi012.stdout:Sep 11 00:00:58 smithi012 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-6[59757]: 2024-09-11T00:00:58.710+0000 7f2e6f200640 -1 osd.6 41 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-11T00:00:59.029 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:00:58 smithi012 ceph-mon[30333]: from='osd.6 [v2:172.21.15.12:6818/3868322467,v1:172.21.15.12:6819/3868322467]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi012", "root=default"]}]': finished 2024-09-11T00:00:59.029 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:00:58 smithi012 ceph-mon[30333]: osdmap e41: 7 total, 6 up, 7 in 2024-09-11T00:00:59.029 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:00:58 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-11T00:00:59.029 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:00:58 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-11T00:00:59.029 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:00:58 smithi012 ceph-mon[30333]: from='osd.6 [v2:172.21.15.12:6818/3868322467,v1:172.21.15.12:6819/3868322467]' entity='osd.6' 2024-09-11T00:00:59.330 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:00:58 smithi178 ceph-mon[35919]: from='osd.6 [v2:172.21.15.12:6818/3868322467,v1:172.21.15.12:6819/3868322467]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi012", "root=default"]}]': finished 2024-09-11T00:00:59.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:00:58 smithi178 ceph-mon[35919]: osdmap e41: 7 total, 6 up, 7 in 2024-09-11T00:00:59.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:00:58 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-11T00:00:59.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:00:58 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-11T00:00:59.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:00:58 smithi178 ceph-mon[35919]: from='osd.6 [v2:172.21.15.12:6818/3868322467,v1:172.21.15.12:6819/3868322467]' entity='osd.6' 2024-09-11T00:00:59.499 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-11T00:00:59.497+0000 7f5c26a94640 1 -- 172.21.15.165:0/791630121 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5c20101930 msgr2=0x7f5c20101d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:00:59.499 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-11T00:00:59.497+0000 7f5c26a94640 1 --2- 172.21.15.165:0/791630121 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5c20101930 0x7f5c20101d30 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f5c14009930 tx=0x7f5c1402a420 comp rx=0 tx=0).stop 2024-09-11T00:00:59.499 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-11T00:00:59.497+0000 7f5c26a94640 1 -- 172.21.15.165:0/791630121 shutdown_connections 2024-09-11T00:00:59.499 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-11T00:00:59.497+0000 7f5c26a94640 1 --2- 172.21.15.165:0/791630121 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5c20103450 0x7f5c20109dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:00:59.499 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-11T00:00:59.497+0000 7f5c26a94640 1 --2- 172.21.15.165:0/791630121 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5c20102a90 0x7f5c20102f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:00:59.499 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-11T00:00:59.497+0000 7f5c26a94640 1 --2- 172.21.15.165:0/791630121 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5c20101930 0x7f5c20101d30 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:00:59.499 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-11T00:00:59.497+0000 7f5c26a94640 1 -- 172.21.15.165:0/791630121 >> 172.21.15.165:0/791630121 conn(0x7f5c200fd220 msgr2=0x7f5c200ff640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:00:59.499 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-11T00:00:59.497+0000 7f5c26a94640 1 -- 172.21.15.165:0/791630121 shutdown_connections 2024-09-11T00:00:59.499 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-11T00:00:59.497+0000 7f5c26a94640 1 -- 172.21.15.165:0/791630121 wait complete. 2024-09-11T00:00:59.499 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-11T00:00:59.498+0000 7f5c26a94640 1 Processor -- start 2024-09-11T00:00:59.499 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-11T00:00:59.498+0000 7f5c26a94640 1 -- start start 2024-09-11T00:00:59.501 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-11T00:00:59.498+0000 7f5c26a94640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5c20101930 0x7f5c2006bd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:00:59.501 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-11T00:00:59.498+0000 7f5c26a94640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5c20102a90 0x7f5c2019dbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:00:59.501 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-11T00:00:59.498+0000 7f5c26a94640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5c20103450 0x7f5c201a01a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:00:59.502 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-11T00:00:59.498+0000 7f5c26a94640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f5c2010d610 con 0x7f5c20102a90 2024-09-11T00:00:59.502 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-11T00:00:59.498+0000 7f5c26a94640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f5c2010d490 con 0x7f5c20103450 2024-09-11T00:00:59.502 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-11T00:00:59.498+0000 7f5c26a94640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f5c2010d790 con 0x7f5c20101930 2024-09-11T00:00:59.502 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-11T00:00:59.498+0000 7f5c25031640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5c20103450 0x7f5c201a01a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:00:59.502 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-11T00:00:59.498+0000 7f5c1ffff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5c20102a90 0x7f5c2019dbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:00:59.502 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-11T00:00:59.498+0000 7f5c25031640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5c20103450 0x7f5c201a01a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:41756/0 (socket says 172.21.15.165:41756) 2024-09-11T00:00:59.502 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-11T00:00:59.498+0000 7f5c25031640 1 -- 172.21.15.165:0/1797551703 learned_addr learned my addr 172.21.15.165:0/1797551703 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-09-11T00:00:59.502 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-11T00:00:59.498+0000 7f5c24830640 1 --2- 172.21.15.165:0/1797551703 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5c20101930 0x7f5c2006bd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:00:59.503 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-11T00:00:59.498+0000 7f5c25031640 1 -- 172.21.15.165:0/1797551703 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5c20101930 msgr2=0x7f5c2006bd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:00:59.503 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-11T00:00:59.498+0000 7f5c25031640 1 --2- 172.21.15.165:0/1797551703 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5c20101930 0x7f5c2006bd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:00:59.503 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-11T00:00:59.498+0000 7f5c25031640 1 -- 172.21.15.165:0/1797551703 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5c20102a90 msgr2=0x7f5c2019dbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:00:59.503 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-11T00:00:59.498+0000 7f5c25031640 1 --2- 172.21.15.165:0/1797551703 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5c20102a90 0x7f5c2019dbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:00:59.503 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-11T00:00:59.498+0000 7f5c25031640 1 -- 172.21.15.165:0/1797551703 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c201a0a60 con 0x7f5c20103450 2024-09-11T00:00:59.503 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-11T00:00:59.499+0000 7f5c25031640 1 --2- 172.21.15.165:0/1797551703 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5c20103450 0x7f5c201a01a0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f5c1800b970 tx=0x7f5c1800be30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:00:59.503 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-11T00:00:59.499+0000 7f5c1dffb640 1 -- 172.21.15.165:0/1797551703 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c18019070 con 0x7f5c20103450 2024-09-11T00:00:59.503 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-11T00:00:59.499+0000 7f5c1dffb640 1 -- 172.21.15.165:0/1797551703 <== mon.2 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5c18014440 con 0x7f5c20103450 2024-09-11T00:00:59.503 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-11T00:00:59.499+0000 7f5c1dffb640 1 -- 172.21.15.165:0/1797551703 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c180075d0 con 0x7f5c20103450 2024-09-11T00:00:59.503 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-11T00:00:59.499+0000 7f5c26a94640 1 -- 172.21.15.165:0/1797551703 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5c201a0d50 con 0x7f5c20103450 2024-09-11T00:00:59.503 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-11T00:00:59.499+0000 7f5c26a94640 1 -- 172.21.15.165:0/1797551703 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5c201a1230 con 0x7f5c20103450 2024-09-11T00:00:59.503 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-11T00:00:59.500+0000 7f5c26a94640 1 -- 172.21.15.165:0/1797551703 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5c20108cb0 con 0x7f5c20103450 2024-09-11T00:00:59.504 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-11T00:00:59.501+0000 7f5c1dffb640 1 -- 172.21.15.165:0/1797551703 <== mon.2 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 16) ==== 153250+0+0 (secure 0 0 0) 0x7f5c1800cb40 con 0x7f5c20103450 2024-09-11T00:00:59.504 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-11T00:00:59.502+0000 7f5c1dffb640 1 --2- 172.21.15.165:0/1797551703 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f5c100b5360 0x7f5c100b7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:00:59.504 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-11T00:00:59.502+0000 7f5c1dffb640 1 -- 172.21.15.165:0/1797551703 <== mon.2 v2:172.21.15.165:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 4685+0+0 (secure 0 0 0) 0x7f5c180e44e0 con 0x7f5c20103450 2024-09-11T00:00:59.504 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-11T00:00:59.502+0000 7f5c24830640 1 --2- 172.21.15.165:0/1797551703 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f5c100b5360 0x7f5c100b7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:00:59.505 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-11T00:00:59.503+0000 7f5c24830640 1 --2- 172.21.15.165:0/1797551703 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f5c100b5360 0x7f5c100b7820 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f5c14006010 tx=0x7f5c14005e50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:00:59.507 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-11T00:00:59.506+0000 7f5c1dffb640 1 -- 172.21.15.165:0/1797551703 <== mon.2 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f5c180abb80 con 0x7f5c20103450 2024-09-11T00:00:59.702 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-11T00:00:59.701+0000 7f5c26a94640 1 -- 172.21.15.165:0/1797551703 --> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi165:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f5c20063590 con 0x7f5c100b5360 2024-09-11T00:01:00.122 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:00:59 smithi165 ceph-mon[35356]: purged_snaps scrub starts 2024-09-11T00:01:00.122 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:00:59 smithi165 ceph-mon[35356]: purged_snaps scrub ok 2024-09-11T00:01:00.122 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:00:59 smithi165 ceph-mon[35356]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-09-11T00:01:00.122 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:00:59 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-11T00:01:00.122 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:00:59 smithi165 ceph-mon[35356]: osd.6 [v2:172.21.15.12:6818/3868322467,v1:172.21.15.12:6819/3868322467] boot 2024-09-11T00:01:00.122 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:00:59 smithi165 ceph-mon[35356]: osdmap e42: 7 total, 7 up, 7 in 2024-09-11T00:01:00.122 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:00:59 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-11T00:01:00.122 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:00:59 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-11T00:01:00.122 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:00:59 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-11T00:01:00.122 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:00:59 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:01:00.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:00:59 smithi012 ceph-mon[30333]: purged_snaps scrub starts 2024-09-11T00:01:00.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:00:59 smithi012 ceph-mon[30333]: purged_snaps scrub ok 2024-09-11T00:01:00.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:00:59 smithi012 ceph-mon[30333]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-09-11T00:01:00.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:00:59 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-11T00:01:00.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:00:59 smithi012 ceph-mon[30333]: osd.6 [v2:172.21.15.12:6818/3868322467,v1:172.21.15.12:6819/3868322467] boot 2024-09-11T00:01:00.314 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:00:59 smithi012 ceph-mon[30333]: osdmap e42: 7 total, 7 up, 7 in 2024-09-11T00:01:00.314 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:00:59 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-11T00:01:00.314 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:00:59 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-11T00:01:00.314 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:00:59 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-11T00:01:00.314 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:00:59 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:01:00.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:00:59 smithi178 ceph-mon[35919]: purged_snaps scrub starts 2024-09-11T00:01:00.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:00:59 smithi178 ceph-mon[35919]: purged_snaps scrub ok 2024-09-11T00:01:00.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:00:59 smithi178 ceph-mon[35919]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-09-11T00:01:00.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:00:59 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-11T00:01:00.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:00:59 smithi178 ceph-mon[35919]: osd.6 [v2:172.21.15.12:6818/3868322467,v1:172.21.15.12:6819/3868322467] boot 2024-09-11T00:01:00.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:00:59 smithi178 ceph-mon[35919]: osdmap e42: 7 total, 7 up, 7 in 2024-09-11T00:01:00.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:00:59 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-11T00:01:00.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:00:59 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-11T00:01:00.332 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:00:59 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-11T00:01:00.332 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:00:59 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:01:01.086 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:01:00 smithi165 ceph-mon[35356]: from='client.24428 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi165:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T00:01:01.086 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:01:00 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:01:01.086 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:01:00 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:01:01.087 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:01:00 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-11T00:01:01.087 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:01:00 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-11T00:01:01.087 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:01:00 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-09-11T00:01:01.087 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:01:00 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:01:01.087 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:01:00 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:01:01.087 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:01:00 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T00:01:01.087 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:01:00 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:01:01.127 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:01:00 smithi178 ceph-mon[35919]: from='client.24428 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi165:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T00:01:01.127 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:01:00 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:01:01.127 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:01:00 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:01:01.127 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:01:00 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-11T00:01:01.127 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:01:00 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-11T00:01:01.127 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:01:00 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-09-11T00:01:01.128 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:01:00 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:01:01.128 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:01:00 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:01:01.128 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:01:00 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T00:01:01.128 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:01:00 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:01:01.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:01:00 smithi012 ceph-mon[30333]: from='client.24428 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi165:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T00:01:01.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:01:00 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:01:01.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:01:00 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:01:01.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:01:00 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-11T00:01:01.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:01:00 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-11T00:01:01.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:01:00 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-09-11T00:01:01.314 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:01:00 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:01:01.314 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:01:00 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:01:01.314 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:01:00 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T00:01:01.314 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:01:00 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:01:02.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:01:01 smithi165 ceph-mon[35356]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 205 MiB used, 626 GiB / 626 GiB avail 2024-09-11T00:01:02.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:01:01 smithi165 ceph-mon[35356]: Detected new or changed devices on smithi012 2024-09-11T00:01:02.033 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:01:01 smithi165 ceph-mon[35356]: Adjusting osd_memory_target on smithi012 to 5698M 2024-09-11T00:01:02.033 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:01:01 smithi165 ceph-mon[35356]: osdmap e43: 7 total, 7 up, 7 in 2024-09-11T00:01:02.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:01:01 smithi012 ceph-mon[30333]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 205 MiB used, 626 GiB / 626 GiB avail 2024-09-11T00:01:02.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:01:01 smithi012 ceph-mon[30333]: Detected new or changed devices on smithi012 2024-09-11T00:01:02.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:01:01 smithi012 ceph-mon[30333]: Adjusting osd_memory_target on smithi012 to 5698M 2024-09-11T00:01:02.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:01:01 smithi012 ceph-mon[30333]: osdmap e43: 7 total, 7 up, 7 in 2024-09-11T00:01:02.330 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:01:01 smithi178 ceph-mon[35919]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 205 MiB used, 626 GiB / 626 GiB avail 2024-09-11T00:01:02.330 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:01:01 smithi178 ceph-mon[35919]: Detected new or changed devices on smithi012 2024-09-11T00:01:02.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:01:01 smithi178 ceph-mon[35919]: Adjusting osd_memory_target on smithi012 to 5698M 2024-09-11T00:01:02.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:01:01 smithi178 ceph-mon[35919]: osdmap e43: 7 total, 7 up, 7 in 2024-09-11T00:01:03.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:01:02 smithi165 ceph-mon[35356]: from='client.? 172.21.15.165:0/295444650' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "0fd619f5-44c3-4cf7-ae7a-388dd798de36"} : dispatch 2024-09-11T00:01:03.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:01:02 smithi165 ceph-mon[35356]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "0fd619f5-44c3-4cf7-ae7a-388dd798de36"} : dispatch 2024-09-11T00:01:03.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:01:02 smithi165 ceph-mon[35356]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0fd619f5-44c3-4cf7-ae7a-388dd798de36"}]': finished 2024-09-11T00:01:03.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:01:02 smithi165 ceph-mon[35356]: osdmap e44: 8 total, 7 up, 8 in 2024-09-11T00:01:03.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:01:02 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-11T00:01:03.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:01:02 smithi165 ceph-mon[35356]: from='client.? 172.21.15.165:0/3375929381' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-11T00:01:03.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:01:02 smithi012 ceph-mon[30333]: from='client.? 172.21.15.165:0/295444650' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "0fd619f5-44c3-4cf7-ae7a-388dd798de36"} : dispatch 2024-09-11T00:01:03.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:01:02 smithi012 ceph-mon[30333]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "0fd619f5-44c3-4cf7-ae7a-388dd798de36"} : dispatch 2024-09-11T00:01:03.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:01:02 smithi012 ceph-mon[30333]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0fd619f5-44c3-4cf7-ae7a-388dd798de36"}]': finished 2024-09-11T00:01:03.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:01:02 smithi012 ceph-mon[30333]: osdmap e44: 8 total, 7 up, 8 in 2024-09-11T00:01:03.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:01:02 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-11T00:01:03.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:01:02 smithi012 ceph-mon[30333]: from='client.? 172.21.15.165:0/3375929381' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-11T00:01:03.330 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:01:02 smithi178 ceph-mon[35919]: from='client.? 172.21.15.165:0/295444650' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "0fd619f5-44c3-4cf7-ae7a-388dd798de36"} : dispatch 2024-09-11T00:01:03.330 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:01:02 smithi178 ceph-mon[35919]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "0fd619f5-44c3-4cf7-ae7a-388dd798de36"} : dispatch 2024-09-11T00:01:03.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:01:02 smithi178 ceph-mon[35919]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0fd619f5-44c3-4cf7-ae7a-388dd798de36"}]': finished 2024-09-11T00:01:03.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:01:02 smithi178 ceph-mon[35919]: osdmap e44: 8 total, 7 up, 8 in 2024-09-11T00:01:03.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:01:02 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-11T00:01:03.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:01:02 smithi178 ceph-mon[35919]: from='client.? 172.21.15.165:0/3375929381' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-11T00:01:04.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:01:03 smithi165 ceph-mon[35356]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 205 MiB used, 626 GiB / 626 GiB avail 2024-09-11T00:01:04.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:01:03 smithi012 ceph-mon[30333]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 205 MiB used, 626 GiB / 626 GiB avail 2024-09-11T00:01:04.330 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:01:03 smithi178 ceph-mon[35919]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 205 MiB used, 626 GiB / 626 GiB avail 2024-09-11T00:01:06.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:01:05 smithi165 ceph-mon[35356]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 205 MiB used, 626 GiB / 626 GiB avail 2024-09-11T00:01:06.312 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:01:05 smithi012 ceph-mon[30333]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 205 MiB used, 626 GiB / 626 GiB avail 2024-09-11T00:01:06.330 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:01:05 smithi178 ceph-mon[35919]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 205 MiB used, 626 GiB / 626 GiB avail 2024-09-11T00:01:08.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:01:07 smithi165 ceph-mon[35356]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 205 MiB used, 626 GiB / 626 GiB avail 2024-09-11T00:01:08.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:01:07 smithi012 ceph-mon[30333]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 205 MiB used, 626 GiB / 626 GiB avail 2024-09-11T00:01:08.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:01:07 smithi178 ceph-mon[35919]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 205 MiB used, 626 GiB / 626 GiB avail 2024-09-11T00:01:09.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:01:08 smithi165 ceph-mon[35356]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-09-11T00:01:09.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:01:08 smithi012 ceph-mon[30333]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-09-11T00:01:09.330 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:01:08 smithi178 ceph-mon[35919]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-09-11T00:01:11.404 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:01:11 smithi165 ceph-mon[35356]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-09-11T00:01:11.405 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:01:11 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.7"} : dispatch 2024-09-11T00:01:11.405 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:01:11 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:01:11.405 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:01:11 smithi165 ceph-mon[35356]: Deploying daemon osd.7 on smithi165 2024-09-11T00:01:11.563 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:01:11 smithi012 ceph-mon[30333]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-09-11T00:01:11.563 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:01:11 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.7"} : dispatch 2024-09-11T00:01:11.563 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:01:11 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:01:11.563 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:01:11 smithi012 ceph-mon[30333]: Deploying daemon osd.7 on smithi165 2024-09-11T00:01:11.584 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:01:11 smithi178 ceph-mon[35919]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-09-11T00:01:11.584 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:01:11 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.7"} : dispatch 2024-09-11T00:01:11.585 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:01:11 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:01:11.585 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:01:11 smithi178 ceph-mon[35919]: Deploying daemon osd.7 on smithi165 2024-09-11T00:01:13.398 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:01:13 smithi165 ceph-mon[35356]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-09-11T00:01:13.563 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:01:13 smithi012 ceph-mon[30333]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-09-11T00:01:13.580 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:01:13 smithi178 ceph-mon[35919]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-09-11T00:01:15.532 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:01:15 smithi165 ceph-mon[35356]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-09-11T00:01:15.563 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:01:15 smithi012 ceph-mon[30333]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-09-11T00:01:15.580 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:01:15 smithi178 ceph-mon[35919]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-09-11T00:01:17.386 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:01:17 smithi165 ceph-mon[35356]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-09-11T00:01:17.386 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:01:17 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T00:01:17.386 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:01:17 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:01:17.386 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:01:17 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:01:17.563 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:01:17 smithi012 ceph-mon[30333]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-09-11T00:01:17.563 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:01:17 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T00:01:17.563 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:01:17 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:01:17.563 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:01:17 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:01:17.580 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:01:17 smithi178 ceph-mon[35919]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-09-11T00:01:17.581 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:01:17 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T00:01:17.581 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:01:17 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:01:17.581 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:01:17 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:01:19.180 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:01:19 smithi165 ceph-mon[35356]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-09-11T00:01:19.563 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:01:19 smithi012 ceph-mon[30333]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-09-11T00:01:19.580 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:01:19 smithi178 ceph-mon[35919]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-09-11T00:01:20.723 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-11T00:01:20.721+0000 7f5c1dffb640 1 -- 172.21.15.165:0/1797551703 <== mgr.14150 v2:172.21.15.12:6800/3442939137 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f5c20063590 con 0x7f5c100b5360 2024-09-11T00:01:20.723 INFO:teuthology.orchestra.run.smithi165.stdout:Created osd(s) 7 on host 'smithi165' 2024-09-11T00:01:20.726 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-11T00:01:20.724+0000 7f5c26a94640 1 -- 172.21.15.165:0/1797551703 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f5c100b5360 msgr2=0x7f5c100b7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:01:20.726 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-11T00:01:20.724+0000 7f5c26a94640 1 --2- 172.21.15.165:0/1797551703 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f5c100b5360 0x7f5c100b7820 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f5c14006010 tx=0x7f5c14005e50 comp rx=0 tx=0).stop 2024-09-11T00:01:20.726 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-11T00:01:20.724+0000 7f5c26a94640 1 -- 172.21.15.165:0/1797551703 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5c20103450 msgr2=0x7f5c201a01a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:01:20.726 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-11T00:01:20.724+0000 7f5c26a94640 1 --2- 172.21.15.165:0/1797551703 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5c20103450 0x7f5c201a01a0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f5c1800b970 tx=0x7f5c1800be30 comp rx=0 tx=0).stop 2024-09-11T00:01:20.726 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-11T00:01:20.724+0000 7f5c26a94640 1 -- 172.21.15.165:0/1797551703 shutdown_connections 2024-09-11T00:01:20.726 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-11T00:01:20.724+0000 7f5c26a94640 1 --2- 172.21.15.165:0/1797551703 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f5c100b5360 0x7f5c100b7820 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:01:20.726 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-11T00:01:20.724+0000 7f5c26a94640 1 --2- 172.21.15.165:0/1797551703 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5c20103450 0x7f5c201a01a0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:01:20.726 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-11T00:01:20.724+0000 7f5c26a94640 1 --2- 172.21.15.165:0/1797551703 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5c20102a90 0x7f5c2019dbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:01:20.726 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-11T00:01:20.724+0000 7f5c26a94640 1 --2- 172.21.15.165:0/1797551703 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5c20101930 0x7f5c2006bd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:01:20.726 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-11T00:01:20.724+0000 7f5c26a94640 1 -- 172.21.15.165:0/1797551703 >> 172.21.15.165:0/1797551703 conn(0x7f5c200fd220 msgr2=0x7f5c20105e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:01:20.726 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-11T00:01:20.725+0000 7f5c26a94640 1 -- 172.21.15.165:0/1797551703 shutdown_connections 2024-09-11T00:01:20.727 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-11T00:01:20.725+0000 7f5c26a94640 1 -- 172.21.15.165:0/1797551703 wait complete. 2024-09-11T00:01:21.023 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:01:20 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:01:21.023 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:01:20 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:01:21.023 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:01:20 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:01:21.023 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:01:20 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T00:01:21.023 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:01:20 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:01:21.062 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:01:20 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:01:21.063 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:01:20 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:01:21.063 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:01:20 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:01:21.063 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:01:20 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T00:01:21.063 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:01:20 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:01:21.080 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:01:20 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:01:21.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:01:20 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:01:21.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:01:20 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:01:21.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:01:20 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T00:01:21.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:01:20 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:01:21.330 DEBUG:teuthology.orchestra.run.smithi165:osd.7> sudo journalctl -f -n 0 -u ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968@osd.7.service 2024-09-11T00:01:21.332 INFO:tasks.cephadm:Deploying osd.8 on smithi178 with /dev/vg_nvme/lv_2... 2024-09-11T00:01:21.332 DEBUG:teuthology.orchestra.run.smithi178:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ff3ad74a-6fcf-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_2 2024-09-11T00:01:21.587 INFO:teuthology.orchestra.run.smithi178.stderr:Inferring config /var/lib/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/mon.c/config 2024-09-11T00:01:21.893 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:01:21 smithi165 ceph-mon[35356]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-09-11T00:01:21.893 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:01:21 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T00:01:21.893 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:01:21 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:01:21.893 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:01:21 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:01:21.979 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:01:21 smithi178 ceph-mon[35919]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-09-11T00:01:21.980 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:01:21 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T00:01:21.980 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:01:21 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:01:21.980 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:01:21 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:01:22.063 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:01:21 smithi012 ceph-mon[30333]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-09-11T00:01:22.063 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:01:21 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T00:01:22.063 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:01:21 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:01:22.063 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:01:21 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:01:23.532 INFO:journalctl@ceph.osd.7.smithi165.stdout:Sep 11 00:01:23 smithi165 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-7[52941]: 2024-09-11T00:01:23.179+0000 7ffbe794b640 -1 osd.7 0 log_to_monitors true 2024-09-11T00:01:23.945 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:01:23 smithi165 ceph-mon[35356]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-09-11T00:01:23.945 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:01:23 smithi165 ceph-mon[35356]: from='osd.7 [v2:172.21.15.165:6816/1143773735,v1:172.21.15.165:6817/1143773735]' entity='osd.7' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]} : dispatch 2024-09-11T00:01:23.945 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:01:23 smithi165 ceph-mon[35356]: from='osd.7 ' entity='osd.7' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]} : dispatch 2024-09-11T00:01:24.058 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:01:23 smithi178 ceph-mon[35919]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-09-11T00:01:24.059 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:01:23 smithi178 ceph-mon[35919]: from='osd.7 [v2:172.21.15.165:6816/1143773735,v1:172.21.15.165:6817/1143773735]' entity='osd.7' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]} : dispatch 2024-09-11T00:01:24.059 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:01:23 smithi178 ceph-mon[35919]: from='osd.7 ' entity='osd.7' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]} : dispatch 2024-09-11T00:01:24.063 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-09-11T00:01:24.064 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:01:23 smithi012 ceph-mon[30333]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-09-11T00:01:24.064 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:01:23 smithi012 ceph-mon[30333]: from='osd.7 [v2:172.21.15.165:6816/1143773735,v1:172.21.15.165:6817/1143773735]' entity='osd.7' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]} : dispatch 2024-09-11T00:01:24.064 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:01:23 smithi012 ceph-mon[30333]: from='osd.7 ' entity='osd.7' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]} : dispatch 2024-09-11T00:01:24.087 DEBUG:teuthology.orchestra.run.smithi178:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ff3ad74a-6fcf-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi178:vg_nvme/lv_2 2024-09-11T00:01:24.346 INFO:teuthology.orchestra.run.smithi178.stderr:Inferring config /var/lib/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/mon.c/config 2024-09-11T00:01:24.754 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:01:24 smithi178 ceph-mon[35919]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-09-11T00:01:24.754 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:01:24 smithi178 ceph-mon[35919]: osdmap e45: 8 total, 7 up, 8 in 2024-09-11T00:01:24.754 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:01:24 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-11T00:01:24.754 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:01:24 smithi178 ceph-mon[35919]: from='osd.7 [v2:172.21.15.165:6816/1143773735,v1:172.21.15.165:6817/1143773735]' entity='osd.7' cmd={"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi165", "root=default"]} : dispatch 2024-09-11T00:01:24.754 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:01:24 smithi178 ceph-mon[35919]: from='osd.7 ' entity='osd.7' cmd={"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi165", "root=default"]} : dispatch 2024-09-11T00:01:24.754 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:01:24 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:01:24.755 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:01:24 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:01:24.755 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:01:24 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-11T00:01:24.755 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:01:24 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-11T00:01:24.755 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:01:24 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-09-11T00:01:24.755 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:01:24 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:01:24.755 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:01:24 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:01:24.755 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:01:24 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T00:01:24.755 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:01:24 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:01:24.838 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T00:01:24.835+0000 7f4588a51640 1 -- 172.21.15.178:0/3968169928 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4584101950 msgr2=0x7f4584101d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:01:24.839 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T00:01:24.835+0000 7f4588a51640 1 --2- 172.21.15.178:0/3968169928 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4584101950 0x7f4584101d50 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f4574009960 tx=0x7f457402a480 comp rx=0 tx=0).stop 2024-09-11T00:01:24.839 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T00:01:24.835+0000 7f4588a51640 1 -- 172.21.15.178:0/3968169928 shutdown_connections 2024-09-11T00:01:24.839 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T00:01:24.835+0000 7f4588a51640 1 --2- 172.21.15.178:0/3968169928 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4584103470 0x7f4584109df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:01:24.839 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T00:01:24.835+0000 7f4588a51640 1 --2- 172.21.15.178:0/3968169928 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4584102ab0 0x7f4584102f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:01:24.839 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T00:01:24.835+0000 7f4588a51640 1 --2- 172.21.15.178:0/3968169928 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4584101950 0x7f4584101d50 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:01:24.839 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T00:01:24.835+0000 7f4588a51640 1 -- 172.21.15.178:0/3968169928 >> 172.21.15.178:0/3968169928 conn(0x7f45840fd200 msgr2=0x7f45840ff660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:01:24.839 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T00:01:24.835+0000 7f4588a51640 1 -- 172.21.15.178:0/3968169928 shutdown_connections 2024-09-11T00:01:24.839 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T00:01:24.835+0000 7f4588a51640 1 -- 172.21.15.178:0/3968169928 wait complete. 2024-09-11T00:01:24.840 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T00:01:24.836+0000 7f4588a51640 1 Processor -- start 2024-09-11T00:01:24.840 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T00:01:24.836+0000 7f4588a51640 1 -- start start 2024-09-11T00:01:24.840 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T00:01:24.836+0000 7f4588a51640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4584101950 0x7f4584105710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:01:24.840 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T00:01:24.836+0000 7f4588a51640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4584102ab0 0x7f4584103f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:01:24.840 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T00:01:24.836+0000 7f4588a51640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4584103470 0x7f45841044c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:01:24.840 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T00:01:24.836+0000 7f4588a51640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f458410d650 con 0x7f4584102ab0 2024-09-11T00:01:24.840 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T00:01:24.836+0000 7f4588a51640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f458410d4d0 con 0x7f4584101950 2024-09-11T00:01:24.840 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T00:01:24.836+0000 7f4588a51640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f458410d7d0 con 0x7f4584103470 2024-09-11T00:01:24.840 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T00:01:24.836+0000 7f4582d9d640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4584103470 0x7f45841044c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:01:24.840 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T00:01:24.837+0000 7f4582d9d640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4584103470 0x7f45841044c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:37008/0 (socket says 172.21.15.178:37008) 2024-09-11T00:01:24.841 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T00:01:24.837+0000 7f4582d9d640 1 -- 172.21.15.178:0/3659883022 learned_addr learned my addr 172.21.15.178:0/3659883022 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-11T00:01:24.841 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T00:01:24.837+0000 7f458259c640 1 --2- 172.21.15.178:0/3659883022 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4584101950 0x7f4584105710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:01:24.841 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T00:01:24.837+0000 7f4581d9b640 1 --2- 172.21.15.178:0/3659883022 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4584102ab0 0x7f4584103f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:01:24.841 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T00:01:24.837+0000 7f4582d9d640 1 -- 172.21.15.178:0/3659883022 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4584101950 msgr2=0x7f4584105710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:01:24.841 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T00:01:24.837+0000 7f4582d9d640 1 --2- 172.21.15.178:0/3659883022 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4584101950 0x7f4584105710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:01:24.841 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T00:01:24.837+0000 7f4582d9d640 1 -- 172.21.15.178:0/3659883022 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4584102ab0 msgr2=0x7f4584103f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:01:24.841 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T00:01:24.837+0000 7f4582d9d640 1 --2- 172.21.15.178:0/3659883022 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4584102ab0 0x7f4584103f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:01:24.841 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T00:01:24.837+0000 7f4582d9d640 1 -- 172.21.15.178:0/3659883022 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4584104d80 con 0x7f4584103470 2024-09-11T00:01:24.841 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T00:01:24.837+0000 7f4582d9d640 1 --2- 172.21.15.178:0/3659883022 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4584103470 0x7f45841044c0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f457c00eb10 tx=0x7f457c00c2d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:01:24.841 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T00:01:24.837+0000 7f45737fe640 1 -- 172.21.15.178:0/3659883022 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f457c004b20 con 0x7f4584103470 2024-09-11T00:01:24.842 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T00:01:24.837+0000 7f45737fe640 1 -- 172.21.15.178:0/3659883022 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f457c004cc0 con 0x7f4584103470 2024-09-11T00:01:24.842 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T00:01:24.837+0000 7f4588a51640 1 -- 172.21.15.178:0/3659883022 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f45841a2fe0 con 0x7f4584103470 2024-09-11T00:01:24.842 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T00:01:24.837+0000 7f45737fe640 1 -- 172.21.15.178:0/3659883022 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f457c010670 con 0x7f4584103470 2024-09-11T00:01:24.842 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T00:01:24.837+0000 7f4588a51640 1 -- 172.21.15.178:0/3659883022 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45841a3510 con 0x7f4584103470 2024-09-11T00:01:24.843 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T00:01:24.838+0000 7f4588a51640 1 -- 172.21.15.178:0/3659883022 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f45841a37a0 con 0x7f4584103470 2024-09-11T00:01:24.843 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T00:01:24.840+0000 7f45737fe640 1 -- 172.21.15.178:0/3659883022 <== mon.1 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 16) ==== 153250+0+0 (secure 0 0 0) 0x7f457c005000 con 0x7f4584103470 2024-09-11T00:01:24.843 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T00:01:24.841+0000 7f45737fe640 1 --2- 172.21.15.178:0/3659883022 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f456c0b5360 0x7f456c0b7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:01:24.843 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T00:01:24.841+0000 7f45737fe640 1 -- 172.21.15.178:0/3659883022 <== mon.1 v2:172.21.15.178:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5117+0+0 (secure 0 0 0) 0x7f457c0e56a0 con 0x7f4584103470 2024-09-11T00:01:24.843 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T00:01:24.841+0000 7f458259c640 1 --2- 172.21.15.178:0/3659883022 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f456c0b5360 0x7f456c0b7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:01:24.843 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T00:01:24.842+0000 7f458259c640 1 --2- 172.21.15.178:0/3659883022 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f456c0b5360 0x7f456c0b7820 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f45740096c0 tx=0x7f4574005d50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:01:24.845 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T00:01:24.844+0000 7f45737fe640 1 -- 172.21.15.178:0/3659883022 <== mon.1 v2:172.21.15.178:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f457c056220 con 0x7f4584103470 2024-09-11T00:01:25.029 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T00:01:25.027+0000 7f4588a51640 1 -- 172.21.15.178:0/3659883022 --> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi178:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f45840ff400 con 0x7f456c0b5360 2024-09-11T00:01:25.032 INFO:journalctl@ceph.osd.7.smithi165.stdout:Sep 11 00:01:24 smithi165 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-7[52941]: 2024-09-11T00:01:24.703+0000 7ffbe38ce640 -1 osd.7 0 waiting for initial osdmap 2024-09-11T00:01:25.032 INFO:journalctl@ceph.osd.7.smithi165.stdout:Sep 11 00:01:24 smithi165 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-7[52941]: 2024-09-11T00:01:24.712+0000 7ffbdf6f5640 -1 osd.7 46 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-11T00:01:25.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:01:24 smithi165 ceph-mon[35356]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-09-11T00:01:25.033 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:01:24 smithi165 ceph-mon[35356]: osdmap e45: 8 total, 7 up, 8 in 2024-09-11T00:01:25.033 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:01:24 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-11T00:01:25.033 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:01:24 smithi165 ceph-mon[35356]: from='osd.7 [v2:172.21.15.165:6816/1143773735,v1:172.21.15.165:6817/1143773735]' entity='osd.7' cmd={"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi165", "root=default"]} : dispatch 2024-09-11T00:01:25.033 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:01:24 smithi165 ceph-mon[35356]: from='osd.7 ' entity='osd.7' cmd={"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi165", "root=default"]} : dispatch 2024-09-11T00:01:25.033 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:01:24 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:01:25.033 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:01:24 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:01:25.033 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:01:24 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-11T00:01:25.033 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:01:24 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-11T00:01:25.033 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:01:24 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-09-11T00:01:25.033 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:01:24 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:01:25.033 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:01:24 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:01:25.033 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:01:24 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T00:01:25.033 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:01:24 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:01:25.063 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:01:24 smithi012 ceph-mon[30333]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-09-11T00:01:25.063 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:01:24 smithi012 ceph-mon[30333]: osdmap e45: 8 total, 7 up, 8 in 2024-09-11T00:01:25.063 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:01:24 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-11T00:01:25.063 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:01:24 smithi012 ceph-mon[30333]: from='osd.7 [v2:172.21.15.165:6816/1143773735,v1:172.21.15.165:6817/1143773735]' entity='osd.7' cmd={"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi165", "root=default"]} : dispatch 2024-09-11T00:01:25.063 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:01:24 smithi012 ceph-mon[30333]: from='osd.7 ' entity='osd.7' cmd={"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi165", "root=default"]} : dispatch 2024-09-11T00:01:25.063 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:01:24 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:01:25.064 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:01:24 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:01:25.064 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:01:24 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-11T00:01:25.064 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:01:24 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-11T00:01:25.064 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:01:24 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-09-11T00:01:25.064 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:01:24 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:01:25.064 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:01:24 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:01:25.064 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:01:24 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T00:01:25.064 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:01:24 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:01:26.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:01:25 smithi165 ceph-mon[35356]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-09-11T00:01:26.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:01:25 smithi165 ceph-mon[35356]: Detected new or changed devices on smithi165 2024-09-11T00:01:26.033 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:01:25 smithi165 ceph-mon[35356]: Adjusting osd_memory_target on smithi165 to 5696M 2024-09-11T00:01:26.033 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:01:25 smithi165 ceph-mon[35356]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi165", "root=default"]}]': finished 2024-09-11T00:01:26.033 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:01:25 smithi165 ceph-mon[35356]: osdmap e46: 8 total, 7 up, 8 in 2024-09-11T00:01:26.033 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:01:25 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-11T00:01:26.033 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:01:25 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-11T00:01:26.033 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:01:25 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-11T00:01:26.033 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:01:25 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-11T00:01:26.033 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:01:25 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:01:26.033 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:01:25 smithi165 ceph-mon[35356]: osd.7 [v2:172.21.15.165:6816/1143773735,v1:172.21.15.165:6817/1143773735] boot 2024-09-11T00:01:26.033 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:01:25 smithi165 ceph-mon[35356]: osdmap e47: 8 total, 8 up, 8 in 2024-09-11T00:01:26.033 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:01:25 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-11T00:01:26.063 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:01:25 smithi012 ceph-mon[30333]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-09-11T00:01:26.063 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:01:25 smithi012 ceph-mon[30333]: Detected new or changed devices on smithi165 2024-09-11T00:01:26.063 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:01:25 smithi012 ceph-mon[30333]: Adjusting osd_memory_target on smithi165 to 5696M 2024-09-11T00:01:26.063 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:01:25 smithi012 ceph-mon[30333]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi165", "root=default"]}]': finished 2024-09-11T00:01:26.063 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:01:25 smithi012 ceph-mon[30333]: osdmap e46: 8 total, 7 up, 8 in 2024-09-11T00:01:26.063 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:01:25 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-11T00:01:26.063 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:01:25 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-11T00:01:26.063 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:01:25 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-11T00:01:26.063 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:01:25 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-11T00:01:26.063 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:01:25 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:01:26.064 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:01:25 smithi012 ceph-mon[30333]: osd.7 [v2:172.21.15.165:6816/1143773735,v1:172.21.15.165:6817/1143773735] boot 2024-09-11T00:01:26.064 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:01:25 smithi012 ceph-mon[30333]: osdmap e47: 8 total, 8 up, 8 in 2024-09-11T00:01:26.064 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:01:25 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-11T00:01:26.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:01:25 smithi178 ceph-mon[35919]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-09-11T00:01:26.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:01:25 smithi178 ceph-mon[35919]: Detected new or changed devices on smithi165 2024-09-11T00:01:26.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:01:25 smithi178 ceph-mon[35919]: Adjusting osd_memory_target on smithi165 to 5696M 2024-09-11T00:01:26.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:01:25 smithi178 ceph-mon[35919]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi165", "root=default"]}]': finished 2024-09-11T00:01:26.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:01:25 smithi178 ceph-mon[35919]: osdmap e46: 8 total, 7 up, 8 in 2024-09-11T00:01:26.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:01:25 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-11T00:01:26.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:01:25 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-11T00:01:26.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:01:25 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-11T00:01:26.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:01:25 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-11T00:01:26.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:01:25 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:01:26.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:01:25 smithi178 ceph-mon[35919]: osd.7 [v2:172.21.15.165:6816/1143773735,v1:172.21.15.165:6817/1143773735] boot 2024-09-11T00:01:26.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:01:25 smithi178 ceph-mon[35919]: osdmap e47: 8 total, 8 up, 8 in 2024-09-11T00:01:26.082 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:01:25 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-11T00:01:26.956 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:01:26 smithi178 ceph-mon[35919]: purged_snaps scrub starts 2024-09-11T00:01:26.956 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:01:26 smithi178 ceph-mon[35919]: purged_snaps scrub ok 2024-09-11T00:01:26.956 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:01:26 smithi178 ceph-mon[35919]: from='client.24433 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi178:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T00:01:26.957 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:01:26 smithi178 ceph-mon[35919]: osdmap e48: 8 total, 8 up, 8 in 2024-09-11T00:01:27.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:01:26 smithi165 ceph-mon[35356]: purged_snaps scrub starts 2024-09-11T00:01:27.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:01:26 smithi165 ceph-mon[35356]: purged_snaps scrub ok 2024-09-11T00:01:27.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:01:26 smithi165 ceph-mon[35356]: from='client.24433 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi178:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T00:01:27.033 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:01:26 smithi165 ceph-mon[35356]: osdmap e48: 8 total, 8 up, 8 in 2024-09-11T00:01:27.062 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:01:26 smithi012 ceph-mon[30333]: purged_snaps scrub starts 2024-09-11T00:01:27.063 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:01:26 smithi012 ceph-mon[30333]: purged_snaps scrub ok 2024-09-11T00:01:27.063 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:01:26 smithi012 ceph-mon[30333]: from='client.24433 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi178:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T00:01:27.063 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:01:26 smithi012 ceph-mon[30333]: osdmap e48: 8 total, 8 up, 8 in 2024-09-11T00:01:28.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:01:27 smithi165 ceph-mon[35356]: pgmap v191: 1 pgs: 1 remapped; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-09-11T00:01:28.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:01:27 smithi165 ceph-mon[35356]: from='client.? 172.21.15.178:0/2534405461' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "b03d14ce-8427-462a-9bcc-27683c549f53"} : dispatch 2024-09-11T00:01:28.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:01:27 smithi165 ceph-mon[35356]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "b03d14ce-8427-462a-9bcc-27683c549f53"} : dispatch 2024-09-11T00:01:28.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:01:27 smithi165 ceph-mon[35356]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b03d14ce-8427-462a-9bcc-27683c549f53"}]': finished 2024-09-11T00:01:28.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:01:27 smithi165 ceph-mon[35356]: osdmap e49: 9 total, 8 up, 9 in 2024-09-11T00:01:28.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:01:27 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-11T00:01:28.037 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:01:27 smithi178 ceph-mon[35919]: pgmap v191: 1 pgs: 1 remapped; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-09-11T00:01:28.037 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:01:27 smithi178 ceph-mon[35919]: from='client.? 172.21.15.178:0/2534405461' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "b03d14ce-8427-462a-9bcc-27683c549f53"} : dispatch 2024-09-11T00:01:28.037 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:01:27 smithi178 ceph-mon[35919]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "b03d14ce-8427-462a-9bcc-27683c549f53"} : dispatch 2024-09-11T00:01:28.037 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:01:27 smithi178 ceph-mon[35919]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b03d14ce-8427-462a-9bcc-27683c549f53"}]': finished 2024-09-11T00:01:28.037 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:01:27 smithi178 ceph-mon[35919]: osdmap e49: 9 total, 8 up, 9 in 2024-09-11T00:01:28.037 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:01:27 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-11T00:01:28.063 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:01:27 smithi012 ceph-mon[30333]: pgmap v191: 1 pgs: 1 remapped; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-09-11T00:01:28.063 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:01:27 smithi012 ceph-mon[30333]: from='client.? 172.21.15.178:0/2534405461' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "b03d14ce-8427-462a-9bcc-27683c549f53"} : dispatch 2024-09-11T00:01:28.063 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:01:27 smithi012 ceph-mon[30333]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "b03d14ce-8427-462a-9bcc-27683c549f53"} : dispatch 2024-09-11T00:01:28.063 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:01:27 smithi012 ceph-mon[30333]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b03d14ce-8427-462a-9bcc-27683c549f53"}]': finished 2024-09-11T00:01:28.063 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:01:27 smithi012 ceph-mon[30333]: osdmap e49: 9 total, 8 up, 9 in 2024-09-11T00:01:28.063 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:01:27 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-11T00:01:29.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:01:28 smithi165 ceph-mon[35356]: from='client.? 172.21.15.178:0/1156540062' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-11T00:01:29.062 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:01:28 smithi012 ceph-mon[30333]: from='client.? 172.21.15.178:0/1156540062' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-11T00:01:29.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:01:28 smithi178 ceph-mon[35919]: from='client.? 172.21.15.178:0/1156540062' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-11T00:01:30.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:01:29 smithi165 ceph-mon[35356]: pgmap v194: 1 pgs: 1 remapped; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-09-11T00:01:30.062 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:01:29 smithi012 ceph-mon[30333]: pgmap v194: 1 pgs: 1 remapped; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-09-11T00:01:30.080 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:01:29 smithi178 ceph-mon[35919]: pgmap v194: 1 pgs: 1 remapped; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-09-11T00:01:31.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:01:30 smithi165 ceph-mon[35356]: Health check failed: Reduced data availability: 1 pg inactive (PG_AVAILABILITY) 2024-09-11T00:01:31.062 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:01:30 smithi012 ceph-mon[30333]: Health check failed: Reduced data availability: 1 pg inactive (PG_AVAILABILITY) 2024-09-11T00:01:31.080 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:01:30 smithi178 ceph-mon[35919]: Health check failed: Reduced data availability: 1 pg inactive (PG_AVAILABILITY) 2024-09-11T00:01:32.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:01:31 smithi165 ceph-mon[35356]: pgmap v195: 1 pgs: 1 remapped; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-09-11T00:01:32.064 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:01:31 smithi012 ceph-mon[30333]: pgmap v195: 1 pgs: 1 remapped; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-09-11T00:01:32.080 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:01:31 smithi178 ceph-mon[35919]: pgmap v195: 1 pgs: 1 remapped; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-09-11T00:01:33.044 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:01:32 smithi178 ceph-mon[35919]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg inactive) 2024-09-11T00:01:33.044 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:01:32 smithi178 ceph-mon[35919]: Cluster is now healthy 2024-09-11T00:01:33.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:01:32 smithi165 ceph-mon[35356]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg inactive) 2024-09-11T00:01:33.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:01:32 smithi165 ceph-mon[35356]: Cluster is now healthy 2024-09-11T00:01:33.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:01:32 smithi012 ceph-mon[30333]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg inactive) 2024-09-11T00:01:33.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:01:32 smithi012 ceph-mon[30333]: Cluster is now healthy 2024-09-11T00:01:34.165 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:01:33 smithi178 ceph-mon[35919]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail; 91 KiB/s, 0 objects/s recovering 2024-09-11T00:01:34.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:01:33 smithi165 ceph-mon[35356]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail; 91 KiB/s, 0 objects/s recovering 2024-09-11T00:01:34.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:01:33 smithi012 ceph-mon[30333]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail; 91 KiB/s, 0 objects/s recovering 2024-09-11T00:01:36.097 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:01:35 smithi165 ceph-mon[35356]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-09-11T00:01:36.115 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:01:35 smithi012 ceph-mon[30333]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-09-11T00:01:36.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:01:35 smithi178 ceph-mon[35919]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-09-11T00:01:37.080 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:01:36 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.8"} : dispatch 2024-09-11T00:01:37.080 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:01:36 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:01:37.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:01:36 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.8"} : dispatch 2024-09-11T00:01:37.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:01:36 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:01:37.312 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:01:36 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.8"} : dispatch 2024-09-11T00:01:37.312 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:01:36 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:01:37.896 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:01:37 smithi178 ceph-mon[35919]: Deploying daemon osd.8 on smithi178 2024-09-11T00:01:37.896 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:01:37 smithi178 ceph-mon[35919]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail; 123 KiB/s, 0 objects/s recovering 2024-09-11T00:01:38.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:01:37 smithi165 ceph-mon[35356]: Deploying daemon osd.8 on smithi178 2024-09-11T00:01:38.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:01:37 smithi165 ceph-mon[35356]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail; 123 KiB/s, 0 objects/s recovering 2024-09-11T00:01:38.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:01:37 smithi012 ceph-mon[30333]: Deploying daemon osd.8 on smithi178 2024-09-11T00:01:38.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:01:37 smithi012 ceph-mon[30333]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail; 123 KiB/s, 0 objects/s recovering 2024-09-11T00:01:40.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:01:39 smithi178 ceph-mon[35919]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail; 108 KiB/s, 0 objects/s recovering 2024-09-11T00:01:40.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:01:39 smithi165 ceph-mon[35356]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail; 108 KiB/s, 0 objects/s recovering 2024-09-11T00:01:40.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:01:39 smithi012 ceph-mon[30333]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail; 108 KiB/s, 0 objects/s recovering 2024-09-11T00:01:41.940 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:01:41 smithi178 ceph-mon[35919]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-11T00:01:42.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:01:41 smithi165 ceph-mon[35356]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-11T00:01:42.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:01:41 smithi012 ceph-mon[30333]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-11T00:01:42.873 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:01:42 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T00:01:42.874 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:01:42 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:01:42.874 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:01:42 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:01:43.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:01:42 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T00:01:43.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:01:42 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:01:43.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:01:42 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:01:43.312 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:01:42 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T00:01:43.312 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:01:42 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:01:43.312 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:01:42 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:01:44.013 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:01:43 smithi178 ceph-mon[35919]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-11T00:01:44.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:01:43 smithi165 ceph-mon[35356]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-11T00:01:44.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:01:43 smithi012 ceph-mon[30333]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-11T00:01:45.503 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T00:01:45.502+0000 7f45737fe640 1 -- 172.21.15.178:0/3659883022 <== mgr.14150 v2:172.21.15.12:6800/3442939137 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f45840ff400 con 0x7f456c0b5360 2024-09-11T00:01:45.503 INFO:teuthology.orchestra.run.smithi178.stdout:Created osd(s) 8 on host 'smithi178' 2024-09-11T00:01:45.505 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T00:01:45.504+0000 7f4588a51640 1 -- 172.21.15.178:0/3659883022 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f456c0b5360 msgr2=0x7f456c0b7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:01:45.505 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T00:01:45.504+0000 7f4588a51640 1 --2- 172.21.15.178:0/3659883022 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f456c0b5360 0x7f456c0b7820 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f45740096c0 tx=0x7f4574005d50 comp rx=0 tx=0).stop 2024-09-11T00:01:45.505 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T00:01:45.504+0000 7f4588a51640 1 -- 172.21.15.178:0/3659883022 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4584103470 msgr2=0x7f45841044c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:01:45.505 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T00:01:45.504+0000 7f4588a51640 1 --2- 172.21.15.178:0/3659883022 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4584103470 0x7f45841044c0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f457c00eb10 tx=0x7f457c00c2d0 comp rx=0 tx=0).stop 2024-09-11T00:01:45.505 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T00:01:45.504+0000 7f4588a51640 1 -- 172.21.15.178:0/3659883022 shutdown_connections 2024-09-11T00:01:45.506 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T00:01:45.504+0000 7f4588a51640 1 --2- 172.21.15.178:0/3659883022 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f456c0b5360 0x7f456c0b7820 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:01:45.506 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T00:01:45.504+0000 7f4588a51640 1 --2- 172.21.15.178:0/3659883022 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4584103470 0x7f45841044c0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:01:45.506 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T00:01:45.504+0000 7f4588a51640 1 --2- 172.21.15.178:0/3659883022 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4584102ab0 0x7f4584103f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:01:45.506 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T00:01:45.504+0000 7f4588a51640 1 --2- 172.21.15.178:0/3659883022 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4584101950 0x7f4584105710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:01:45.506 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T00:01:45.504+0000 7f4588a51640 1 -- 172.21.15.178:0/3659883022 >> 172.21.15.178:0/3659883022 conn(0x7f45840fd200 msgr2=0x7f45840fecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:01:45.506 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T00:01:45.504+0000 7f4588a51640 1 -- 172.21.15.178:0/3659883022 shutdown_connections 2024-09-11T00:01:45.506 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T00:01:45.504+0000 7f4588a51640 1 -- 172.21.15.178:0/3659883022 wait complete. 2024-09-11T00:01:46.167 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:01:45 smithi178 ceph-mon[35919]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-09-11T00:01:46.167 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:01:45 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:01:46.167 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:01:45 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:01:46.194 DEBUG:teuthology.orchestra.run.smithi178:osd.8> sudo journalctl -f -n 0 -u ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968@osd.8.service 2024-09-11T00:01:46.196 INFO:tasks.cephadm:Deploying osd.9 on smithi012 with /dev/vg_nvme/lv_1... 2024-09-11T00:01:46.196 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ff3ad74a-6fcf-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_1 2024-09-11T00:01:46.251 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:01:45 smithi012 ceph-mon[30333]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-09-11T00:01:46.252 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:01:45 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:01:46.252 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:01:45 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:01:46.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:01:45 smithi165 ceph-mon[35356]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-09-11T00:01:46.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:01:45 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:01:46.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:01:45 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:01:46.583 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:01:47.532 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:01:47 smithi165 ceph-mon[35356]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-09-11T00:01:47.532 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:01:47 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:01:47.532 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:01:47 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:01:47.562 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:01:47 smithi012 ceph-mon[30333]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-09-11T00:01:47.563 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:01:47 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:01:47.563 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:01:47 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:01:47.580 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:01:47 smithi178 ceph-mon[35919]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-09-11T00:01:47.581 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:01:47 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:01:47.581 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:01:47 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:01:47.924 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 11 00:01:47 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-8[54042]: 2024-09-11T00:01:47.773+0000 7f2312e31640 -1 osd.8 0 log_to_monitors true 2024-09-11T00:01:48.378 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:01:48 smithi012 ceph-mon[30333]: from='osd.8 [v2:172.21.15.178:6816/2891445058,v1:172.21.15.178:6817/2891445058]' entity='osd.8' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]} : dispatch 2024-09-11T00:01:48.378 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:01:48 smithi012 ceph-mon[30333]: from='osd.8 ' entity='osd.8' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]} : dispatch 2024-09-11T00:01:48.580 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:01:48 smithi178 ceph-mon[35919]: from='osd.8 [v2:172.21.15.178:6816/2891445058,v1:172.21.15.178:6817/2891445058]' entity='osd.8' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]} : dispatch 2024-09-11T00:01:48.580 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:01:48 smithi178 ceph-mon[35919]: from='osd.8 ' entity='osd.8' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]} : dispatch 2024-09-11T00:01:48.782 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:01:48 smithi165 ceph-mon[35356]: from='osd.8 [v2:172.21.15.178:6816/2891445058,v1:172.21.15.178:6817/2891445058]' entity='osd.8' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]} : dispatch 2024-09-11T00:01:48.782 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:01:48 smithi165 ceph-mon[35356]: from='osd.8 ' entity='osd.8' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]} : dispatch 2024-09-11T00:01:49.563 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:01:49 smithi012 ceph-mon[30333]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-09-11T00:01:49.563 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:01:49 smithi012 ceph-mon[30333]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-09-11T00:01:49.563 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:01:49 smithi012 ceph-mon[30333]: from='osd.8 [v2:172.21.15.178:6816/2891445058,v1:172.21.15.178:6817/2891445058]' entity='osd.8' cmd={"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi178", "root=default"]} : dispatch 2024-09-11T00:01:49.563 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:01:49 smithi012 ceph-mon[30333]: osdmap e50: 9 total, 8 up, 9 in 2024-09-11T00:01:49.563 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:01:49 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-11T00:01:49.563 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:01:49 smithi012 ceph-mon[30333]: from='osd.8 ' entity='osd.8' cmd={"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi178", "root=default"]} : dispatch 2024-09-11T00:01:49.580 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:01:49 smithi178 ceph-mon[35919]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-09-11T00:01:49.580 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:01:49 smithi178 ceph-mon[35919]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-09-11T00:01:49.581 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:01:49 smithi178 ceph-mon[35919]: from='osd.8 [v2:172.21.15.178:6816/2891445058,v1:172.21.15.178:6817/2891445058]' entity='osd.8' cmd={"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi178", "root=default"]} : dispatch 2024-09-11T00:01:49.581 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:01:49 smithi178 ceph-mon[35919]: osdmap e50: 9 total, 8 up, 9 in 2024-09-11T00:01:49.581 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:01:49 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-11T00:01:49.581 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:01:49 smithi178 ceph-mon[35919]: from='osd.8 ' entity='osd.8' cmd={"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi178", "root=default"]} : dispatch 2024-09-11T00:01:49.751 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-11T00:01:49.782 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:01:49 smithi165 ceph-mon[35356]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-09-11T00:01:49.782 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:01:49 smithi165 ceph-mon[35356]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-09-11T00:01:49.782 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:01:49 smithi165 ceph-mon[35356]: from='osd.8 [v2:172.21.15.178:6816/2891445058,v1:172.21.15.178:6817/2891445058]' entity='osd.8' cmd={"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi178", "root=default"]} : dispatch 2024-09-11T00:01:49.782 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:01:49 smithi165 ceph-mon[35356]: osdmap e50: 9 total, 8 up, 9 in 2024-09-11T00:01:49.782 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:01:49 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-11T00:01:49.783 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:01:49 smithi165 ceph-mon[35356]: from='osd.8 ' entity='osd.8' cmd={"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi178", "root=default"]} : dispatch 2024-09-11T00:01:49.786 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ff3ad74a-6fcf-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi012:vg_nvme/lv_1 2024-09-11T00:01:50.080 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 11 00:01:49 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-8[54042]: 2024-09-11T00:01:49.809+0000 7f230edb4640 -1 osd.8 0 waiting for initial osdmap 2024-09-11T00:01:50.081 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 11 00:01:49 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-8[54042]: 2024-09-11T00:01:49.818+0000 7f2306bd3640 -1 osd.8 51 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-11T00:01:50.183 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:01:50.563 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:01:50 smithi012 ceph-mon[30333]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi178", "root=default"]}]': finished 2024-09-11T00:01:50.563 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:01:50 smithi012 ceph-mon[30333]: osdmap e51: 9 total, 8 up, 9 in 2024-09-11T00:01:50.563 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:01:50 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-11T00:01:50.563 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:01:50 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-11T00:01:50.564 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:01:50 smithi012 ceph-mon[30333]: Detected new or changed devices on smithi178 2024-09-11T00:01:50.564 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:01:50 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:01:50.564 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:01:50 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:01:50.564 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:01:50 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-11T00:01:50.564 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:01:50 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-11T00:01:50.565 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:01:50 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"} : dispatch 2024-09-11T00:01:50.565 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:01:50 smithi012 ceph-mon[30333]: Adjusting osd_memory_target on smithi178 to 5696M 2024-09-11T00:01:50.565 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:01:50 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:01:50.565 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:01:50 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:01:50.565 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:01:50 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T00:01:50.565 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:01:50 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:01:50.565 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:01:50 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T00:01:50.565 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:01:50 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:01:50.565 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:01:50 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T00:01:50.566 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:01:50 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:01:50.566 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:01:50 smithi012 ceph-mon[30333]: from='osd.8 ' entity='osd.8' 2024-09-11T00:01:50.581 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:01:50 smithi178 ceph-mon[35919]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi178", "root=default"]}]': finished 2024-09-11T00:01:50.581 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:01:50 smithi178 ceph-mon[35919]: osdmap e51: 9 total, 8 up, 9 in 2024-09-11T00:01:50.581 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:01:50 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-11T00:01:50.581 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:01:50 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-11T00:01:50.581 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:01:50 smithi178 ceph-mon[35919]: Detected new or changed devices on smithi178 2024-09-11T00:01:50.581 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:01:50 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:01:50.581 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:01:50 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:01:50.581 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:01:50 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-11T00:01:50.581 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:01:50 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-11T00:01:50.581 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:01:50 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"} : dispatch 2024-09-11T00:01:50.582 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:01:50 smithi178 ceph-mon[35919]: Adjusting osd_memory_target on smithi178 to 5696M 2024-09-11T00:01:50.582 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:01:50 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:01:50.582 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:01:50 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:01:50.582 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:01:50 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T00:01:50.582 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:01:50 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:01:50.582 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:01:50 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T00:01:50.582 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:01:50 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:01:50.582 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:01:50 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T00:01:50.582 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:01:50 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:01:50.582 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:01:50 smithi178 ceph-mon[35919]: from='osd.8 ' entity='osd.8' 2024-09-11T00:01:50.716 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:01:50.714+0000 7f749de00640 1 -- 172.21.15.12:0/3554118559 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7498105da0 msgr2=0x7f7498106220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:01:50.716 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:01:50.714+0000 7f749de00640 1 --2- 172.21.15.12:0/3554118559 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7498105da0 0x7f7498106220 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f7480009930 tx=0x7f748002a420 comp rx=0 tx=0).stop 2024-09-11T00:01:50.716 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:01:50.714+0000 7f749de00640 1 -- 172.21.15.12:0/3554118559 shutdown_connections 2024-09-11T00:01:50.716 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:01:50.714+0000 7f749de00640 1 --2- 172.21.15.12:0/3554118559 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7498106760 0x7f749810d0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:01:50.716 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:01:50.714+0000 7f749de00640 1 --2- 172.21.15.12:0/3554118559 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7498105da0 0x7f7498106220 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:01:50.717 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:01:50.714+0000 7f749de00640 1 --2- 172.21.15.12:0/3554118559 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7498104c40 0x7f7498105040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:01:50.717 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:01:50.714+0000 7f749de00640 1 -- 172.21.15.12:0/3554118559 >> 172.21.15.12:0/3554118559 conn(0x7f7498100510 msgr2=0x7f7498102950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:01:50.717 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:01:50.715+0000 7f749de00640 1 -- 172.21.15.12:0/3554118559 shutdown_connections 2024-09-11T00:01:50.717 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:01:50.715+0000 7f749de00640 1 -- 172.21.15.12:0/3554118559 wait complete. 2024-09-11T00:01:50.718 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:01:50.716+0000 7f749de00640 1 Processor -- start 2024-09-11T00:01:50.718 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:01:50.716+0000 7f749de00640 1 -- start start 2024-09-11T00:01:50.718 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:01:50.716+0000 7f749de00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7498104c40 0x7f749819aff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:01:50.718 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:01:50.716+0000 7f749de00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7498105da0 0x7f749819b640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:01:50.718 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:01:50.716+0000 7f749de00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7498106760 0x7f749819c280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:01:50.718 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:01:50.716+0000 7f749de00640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f7498110910 con 0x7f7498105da0 2024-09-11T00:01:50.718 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:01:50.716+0000 7f749de00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f7498110790 con 0x7f7498104c40 2024-09-11T00:01:50.718 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:01:50.716+0000 7f749de00640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f7498110a90 con 0x7f7498106760 2024-09-11T00:01:50.719 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:01:50.716+0000 7f748fe00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7498105da0 0x7f749819b640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:01:50.719 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:01:50.717+0000 7f748fe00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7498105da0 0x7f749819b640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:47382/0 (socket says 172.21.15.12:47382) 2024-09-11T00:01:50.719 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:01:50.717+0000 7f7497400640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7498106760 0x7f749819c280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:01:50.719 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:01:50.717+0000 7f748fe00640 1 -- 172.21.15.12:0/1106916257 learned_addr learned my addr 172.21.15.12:0/1106916257 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:01:50.719 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:01:50.717+0000 7f7496a00640 1 --2- 172.21.15.12:0/1106916257 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7498104c40 0x7f749819aff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:01:50.719 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:01:50.717+0000 7f7497400640 1 -- 172.21.15.12:0/1106916257 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7498104c40 msgr2=0x7f749819aff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:01:50.719 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:01:50.717+0000 7f7497400640 1 --2- 172.21.15.12:0/1106916257 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7498104c40 0x7f749819aff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:01:50.719 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:01:50.717+0000 7f7497400640 1 -- 172.21.15.12:0/1106916257 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7498105da0 msgr2=0x7f749819b640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:01:50.719 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:01:50.717+0000 7f7497400640 1 --2- 172.21.15.12:0/1106916257 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7498105da0 0x7f749819b640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:01:50.719 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:01:50.717+0000 7f7497400640 1 -- 172.21.15.12:0/1106916257 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f749819cb20 con 0x7f7498106760 2024-09-11T00:01:50.719 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:01:50.717+0000 7f748fe00640 1 --2- 172.21.15.12:0/1106916257 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7498105da0 0x7f749819b640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:01:50.719 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:01:50.717+0000 7f7497400640 1 --2- 172.21.15.12:0/1106916257 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7498106760 0x7f749819c280 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f7490009fd0 tx=0x7f749000ef10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:01:50.720 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:01:50.718+0000 7f749ca00640 1 -- 172.21.15.12:0/1106916257 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f749000ce20 con 0x7f7498106760 2024-09-11T00:01:50.720 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:01:50.718+0000 7f749ca00640 1 -- 172.21.15.12:0/1106916257 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7490005020 con 0x7f7498106760 2024-09-11T00:01:50.721 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:01:50.718+0000 7f749de00640 1 -- 172.21.15.12:0/1106916257 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f749819ce10 con 0x7f7498106760 2024-09-11T00:01:50.721 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:01:50.718+0000 7f749ca00640 1 -- 172.21.15.12:0/1106916257 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7490010690 con 0x7f7498106760 2024-09-11T00:01:50.721 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:01:50.718+0000 7f749de00640 1 -- 172.21.15.12:0/1106916257 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74981a34f0 con 0x7f7498106760 2024-09-11T00:01:50.721 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:01:50.719+0000 7f749de00640 1 -- 172.21.15.12:0/1106916257 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f749810bf60 con 0x7f7498106760 2024-09-11T00:01:50.722 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:01:50.721+0000 7f749ca00640 1 -- 172.21.15.12:0/1106916257 <== mon.1 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 16) ==== 153250+0+0 (secure 0 0 0) 0x7f74900054d0 con 0x7f7498106760 2024-09-11T00:01:50.723 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:01:50.722+0000 7f749ca00640 1 --2- 172.21.15.12:0/1106916257 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f747c0b5360 0x7f747c0b7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:01:50.723 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:01:50.723+0000 7f749ca00640 1 -- 172.21.15.12:0/1106916257 <== mon.1 v2:172.21.15.178:3300/0 5 ==== osd_map(52..52 src has 1..52) ==== 5862+0+0 (secure 0 0 0) 0x7f74900e5920 con 0x7f7498106760 2024-09-11T00:01:50.724 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:01:50.723+0000 7f7496a00640 1 --2- 172.21.15.12:0/1106916257 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f747c0b5360 0x7f747c0b7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:01:50.724 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:01:50.723+0000 7f7496a00640 1 --2- 172.21.15.12:0/1106916257 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f747c0b5360 0x7f747c0b7820 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f7488009bd0 tx=0x7f7488005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:01:50.727 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:01:50.726+0000 7f749ca00640 1 -- 172.21.15.12:0/1106916257 <== mon.1 v2:172.21.15.178:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f74900ad0d0 con 0x7f7498106760 2024-09-11T00:01:50.782 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:01:50 smithi165 ceph-mon[35356]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi178", "root=default"]}]': finished 2024-09-11T00:01:50.782 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:01:50 smithi165 ceph-mon[35356]: osdmap e51: 9 total, 8 up, 9 in 2024-09-11T00:01:50.783 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:01:50 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-11T00:01:50.783 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:01:50 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-11T00:01:50.783 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:01:50 smithi165 ceph-mon[35356]: Detected new or changed devices on smithi178 2024-09-11T00:01:50.783 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:01:50 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:01:50.783 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:01:50 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:01:50.783 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:01:50 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-11T00:01:50.783 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:01:50 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-11T00:01:50.783 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:01:50 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"} : dispatch 2024-09-11T00:01:50.783 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:01:50 smithi165 ceph-mon[35356]: Adjusting osd_memory_target on smithi178 to 5696M 2024-09-11T00:01:50.783 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:01:50 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:01:50.783 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:01:50 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:01:50.783 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:01:50 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T00:01:50.783 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:01:50 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:01:50.784 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:01:50 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T00:01:50.784 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:01:50 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:01:50.784 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:01:50 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T00:01:50.784 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:01:50 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:01:50.784 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:01:50 smithi165 ceph-mon[35356]: from='osd.8 ' entity='osd.8' 2024-09-11T00:01:50.931 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:01:50.930+0000 7f749de00640 1 -- 172.21.15.12:0/1106916257 --> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi012:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f7498063590 con 0x7f747c0b5360 2024-09-11T00:01:51.563 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:01:51 smithi012 ceph-mon[30333]: purged_snaps scrub starts 2024-09-11T00:01:51.563 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:01:51 smithi012 ceph-mon[30333]: purged_snaps scrub ok 2024-09-11T00:01:51.563 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:01:51 smithi012 ceph-mon[30333]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-09-11T00:01:51.563 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:01:51 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-11T00:01:51.563 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:01:51 smithi012 ceph-mon[30333]: osd.8 [v2:172.21.15.178:6816/2891445058,v1:172.21.15.178:6817/2891445058] boot 2024-09-11T00:01:51.563 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:01:51 smithi012 ceph-mon[30333]: osdmap e52: 9 total, 9 up, 9 in 2024-09-11T00:01:51.563 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:01:51 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-11T00:01:51.563 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:01:51 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-11T00:01:51.563 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:01:51 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-11T00:01:51.563 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:01:51 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:01:51.580 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:01:51 smithi178 ceph-mon[35919]: purged_snaps scrub starts 2024-09-11T00:01:51.581 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:01:51 smithi178 ceph-mon[35919]: purged_snaps scrub ok 2024-09-11T00:01:51.581 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:01:51 smithi178 ceph-mon[35919]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-09-11T00:01:51.581 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:01:51 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-11T00:01:51.581 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:01:51 smithi178 ceph-mon[35919]: osd.8 [v2:172.21.15.178:6816/2891445058,v1:172.21.15.178:6817/2891445058] boot 2024-09-11T00:01:51.581 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:01:51 smithi178 ceph-mon[35919]: osdmap e52: 9 total, 9 up, 9 in 2024-09-11T00:01:51.581 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:01:51 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-11T00:01:51.581 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:01:51 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-11T00:01:51.581 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:01:51 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-11T00:01:51.581 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:01:51 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:01:51.782 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:01:51 smithi165 ceph-mon[35356]: purged_snaps scrub starts 2024-09-11T00:01:51.782 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:01:51 smithi165 ceph-mon[35356]: purged_snaps scrub ok 2024-09-11T00:01:51.782 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:01:51 smithi165 ceph-mon[35356]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-09-11T00:01:51.782 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:01:51 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-11T00:01:51.782 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:01:51 smithi165 ceph-mon[35356]: osd.8 [v2:172.21.15.178:6816/2891445058,v1:172.21.15.178:6817/2891445058] boot 2024-09-11T00:01:51.783 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:01:51 smithi165 ceph-mon[35356]: osdmap e52: 9 total, 9 up, 9 in 2024-09-11T00:01:51.783 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:01:51 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-11T00:01:51.783 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:01:51 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-11T00:01:51.783 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:01:51 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-11T00:01:51.783 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:01:51 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:01:52.399 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:01:52 smithi012 ceph-mon[30333]: from='client.24460 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi012:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T00:01:52.399 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:01:52 smithi012 ceph-mon[30333]: osdmap e53: 9 total, 9 up, 9 in 2024-09-11T00:01:52.580 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:01:52 smithi178 ceph-mon[35919]: from='client.24460 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi012:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T00:01:52.581 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:01:52 smithi178 ceph-mon[35919]: osdmap e53: 9 total, 9 up, 9 in 2024-09-11T00:01:52.782 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:01:52 smithi165 ceph-mon[35356]: from='client.24460 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi012:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T00:01:52.782 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:01:52 smithi165 ceph-mon[35356]: osdmap e53: 9 total, 9 up, 9 in 2024-09-11T00:01:53.562 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:01:53 smithi012 ceph-mon[30333]: pgmap v210: 1 pgs: 1 remapped+peering; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-09-11T00:01:53.563 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:01:53 smithi012 ceph-mon[30333]: osdmap e54: 9 total, 9 up, 9 in 2024-09-11T00:01:53.580 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:01:53 smithi178 ceph-mon[35919]: pgmap v210: 1 pgs: 1 remapped+peering; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-09-11T00:01:53.580 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:01:53 smithi178 ceph-mon[35919]: osdmap e54: 9 total, 9 up, 9 in 2024-09-11T00:01:53.782 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:01:53 smithi165 ceph-mon[35356]: pgmap v210: 1 pgs: 1 remapped+peering; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-09-11T00:01:53.782 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:01:53 smithi165 ceph-mon[35356]: osdmap e54: 9 total, 9 up, 9 in 2024-09-11T00:01:54.580 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:01:54 smithi178 ceph-mon[35919]: from='client.? 172.21.15.12:0/2349523118' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "81a6cf69-2a24-4a6c-9adf-6b7df38cfacb"} : dispatch 2024-09-11T00:01:54.581 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:01:54 smithi178 ceph-mon[35919]: from='client.? 172.21.15.12:0/2349523118' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "81a6cf69-2a24-4a6c-9adf-6b7df38cfacb"}]': finished 2024-09-11T00:01:54.581 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:01:54 smithi178 ceph-mon[35919]: osdmap e55: 10 total, 9 up, 10 in 2024-09-11T00:01:54.581 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:01:54 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-11T00:01:54.782 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:01:54 smithi165 ceph-mon[35356]: from='client.? 172.21.15.12:0/2349523118' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "81a6cf69-2a24-4a6c-9adf-6b7df38cfacb"} : dispatch 2024-09-11T00:01:54.782 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:01:54 smithi165 ceph-mon[35356]: from='client.? 172.21.15.12:0/2349523118' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "81a6cf69-2a24-4a6c-9adf-6b7df38cfacb"}]': finished 2024-09-11T00:01:54.782 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:01:54 smithi165 ceph-mon[35356]: osdmap e55: 10 total, 9 up, 10 in 2024-09-11T00:01:54.782 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:01:54 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-11T00:01:54.812 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:01:54 smithi012 ceph-mon[30333]: from='client.? 172.21.15.12:0/2349523118' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "81a6cf69-2a24-4a6c-9adf-6b7df38cfacb"} : dispatch 2024-09-11T00:01:54.812 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:01:54 smithi012 ceph-mon[30333]: from='client.? 172.21.15.12:0/2349523118' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "81a6cf69-2a24-4a6c-9adf-6b7df38cfacb"}]': finished 2024-09-11T00:01:54.812 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:01:54 smithi012 ceph-mon[30333]: osdmap e55: 10 total, 9 up, 10 in 2024-09-11T00:01:54.813 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:01:54 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-11T00:01:55.580 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:01:55 smithi178 ceph-mon[35919]: pgmap v212: 1 pgs: 1 remapped+peering; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-09-11T00:01:55.581 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:01:55 smithi178 ceph-mon[35919]: from='client.? 172.21.15.12:0/2795153591' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-11T00:01:55.782 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:01:55 smithi165 ceph-mon[35356]: pgmap v212: 1 pgs: 1 remapped+peering; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-09-11T00:01:55.782 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:01:55 smithi165 ceph-mon[35356]: from='client.? 172.21.15.12:0/2795153591' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-11T00:01:55.812 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:01:55 smithi012 ceph-mon[30333]: pgmap v212: 1 pgs: 1 remapped+peering; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-09-11T00:01:55.812 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:01:55 smithi012 ceph-mon[30333]: from='client.? 172.21.15.12:0/2795153591' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-11T00:01:57.581 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:01:57 smithi178 ceph-mon[35919]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail; 100 KiB/s, 0 objects/s recovering 2024-09-11T00:01:57.782 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:01:57 smithi165 ceph-mon[35356]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail; 100 KiB/s, 0 objects/s recovering 2024-09-11T00:01:57.813 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:01:57 smithi012 ceph-mon[30333]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail; 100 KiB/s, 0 objects/s recovering 2024-09-11T00:01:59.521 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:01:59 smithi012 ceph-mon[30333]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail; 85 KiB/s, 0 objects/s recovering 2024-09-11T00:01:59.581 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:01:59 smithi178 ceph-mon[35919]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail; 85 KiB/s, 0 objects/s recovering 2024-09-11T00:01:59.782 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:01:59 smithi165 ceph-mon[35356]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail; 85 KiB/s, 0 objects/s recovering 2024-09-11T00:02:01.502 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:02:01 smithi012 ceph-mon[30333]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-11T00:02:01.581 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:02:01 smithi178 ceph-mon[35919]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-11T00:02:01.782 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:02:01 smithi165 ceph-mon[35356]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-11T00:02:03.630 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:02:03 smithi012 ceph-mon[30333]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail; 59 KiB/s, 0 objects/s recovering 2024-09-11T00:02:03.630 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:02:03 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.9"} : dispatch 2024-09-11T00:02:03.630 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:02:03 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:02:03.782 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:02:03 smithi165 ceph-mon[35356]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail; 59 KiB/s, 0 objects/s recovering 2024-09-11T00:02:03.782 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:02:03 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.9"} : dispatch 2024-09-11T00:02:03.783 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:02:03 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:02:03.831 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:02:03 smithi178 ceph-mon[35919]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail; 59 KiB/s, 0 objects/s recovering 2024-09-11T00:02:03.831 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:02:03 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.9"} : dispatch 2024-09-11T00:02:03.831 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:02:03 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:02:04.656 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:02:04 smithi012 ceph-mon[30333]: Deploying daemon osd.9 on smithi012 2024-09-11T00:02:04.782 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:02:04 smithi165 ceph-mon[35356]: Deploying daemon osd.9 on smithi012 2024-09-11T00:02:04.830 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:02:04 smithi178 ceph-mon[35919]: Deploying daemon osd.9 on smithi012 2024-09-11T00:02:05.562 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:02:05 smithi012 ceph-mon[30333]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-11T00:02:05.782 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:02:05 smithi165 ceph-mon[35356]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-11T00:02:05.831 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:02:05 smithi178 ceph-mon[35919]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-11T00:02:07.452 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:02:07 smithi012 ceph-mon[30333]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-11T00:02:07.633 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:02:07 smithi178 ceph-mon[35919]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-11T00:02:07.782 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:02:07 smithi165 ceph-mon[35356]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-11T00:02:09.624 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:02:09 smithi012 ceph-mon[30333]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-09-11T00:02:09.782 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:02:09 smithi165 ceph-mon[35356]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-09-11T00:02:09.831 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:02:09 smithi178 ceph-mon[35919]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-09-11T00:02:10.701 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:02:10 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T00:02:10.701 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:02:10 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:02:10.701 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:02:10 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:02:10.782 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:02:10 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T00:02:10.782 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:02:10 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:02:10.782 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:02:10 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:02:10.830 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:02:10 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T00:02:10.830 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:02:10 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:02:10.831 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:02:10 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:02:11.669 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:02:11 smithi012 ceph-mon[30333]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-09-11T00:02:11.782 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:02:11 smithi165 ceph-mon[35356]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-09-11T00:02:11.830 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:02:11 smithi178 ceph-mon[35919]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-09-11T00:02:13.782 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:02:13 smithi012 ceph-mon[30333]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-09-11T00:02:13.782 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:02:13 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:02:13.782 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:02:13 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:02:13.782 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:02:13 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:02:13.782 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:02:13 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T00:02:13.783 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:02:13 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:02:13.783 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:02:13 smithi165 ceph-mon[35356]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-09-11T00:02:13.783 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:02:13 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:02:13.783 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:02:13 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:02:13.783 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:02:13 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:02:13.783 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:02:13 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T00:02:13.783 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:02:13 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:02:13.831 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:02:13 smithi178 ceph-mon[35919]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-09-11T00:02:13.831 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:02:13 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:02:13.831 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:02:13 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:02:13.831 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:02:13 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:02:13.831 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:02:13 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T00:02:13.831 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:02:13 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:02:14.132 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:02:14.131+0000 7f749ca00640 1 -- 172.21.15.12:0/1106916257 <== mgr.14150 v2:172.21.15.12:6800/3442939137 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f7498063590 con 0x7f747c0b5360 2024-09-11T00:02:14.133 INFO:teuthology.orchestra.run.smithi012.stdout:Created osd(s) 9 on host 'smithi012' 2024-09-11T00:02:14.135 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:02:14.134+0000 7f749de00640 1 -- 172.21.15.12:0/1106916257 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f747c0b5360 msgr2=0x7f747c0b7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:02:14.135 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:02:14.134+0000 7f749de00640 1 --2- 172.21.15.12:0/1106916257 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f747c0b5360 0x7f747c0b7820 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f7488009bd0 tx=0x7f7488005f90 comp rx=0 tx=0).stop 2024-09-11T00:02:14.135 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:02:14.134+0000 7f749de00640 1 -- 172.21.15.12:0/1106916257 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7498106760 msgr2=0x7f749819c280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:02:14.136 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:02:14.134+0000 7f749de00640 1 --2- 172.21.15.12:0/1106916257 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7498106760 0x7f749819c280 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f7490009fd0 tx=0x7f749000ef10 comp rx=0 tx=0).stop 2024-09-11T00:02:14.136 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:02:14.134+0000 7f749de00640 1 -- 172.21.15.12:0/1106916257 shutdown_connections 2024-09-11T00:02:14.136 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:02:14.134+0000 7f749de00640 1 --2- 172.21.15.12:0/1106916257 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f747c0b5360 0x7f747c0b7820 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:02:14.136 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:02:14.134+0000 7f749de00640 1 --2- 172.21.15.12:0/1106916257 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7498106760 0x7f749819c280 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:02:14.136 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:02:14.134+0000 7f749de00640 1 --2- 172.21.15.12:0/1106916257 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7498105da0 0x7f749819b640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:02:14.136 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:02:14.134+0000 7f749de00640 1 --2- 172.21.15.12:0/1106916257 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7498104c40 0x7f749819aff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:02:14.136 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:02:14.134+0000 7f749de00640 1 -- 172.21.15.12:0/1106916257 >> 172.21.15.12:0/1106916257 conn(0x7f7498100510 msgr2=0x7f7498109130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:02:14.136 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:02:14.134+0000 7f749de00640 1 -- 172.21.15.12:0/1106916257 shutdown_connections 2024-09-11T00:02:14.136 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:02:14.134+0000 7f749de00640 1 -- 172.21.15.12:0/1106916257 wait complete. 2024-09-11T00:02:14.761 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:02:14 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T00:02:14.761 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:02:14 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:02:14.761 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:02:14 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:02:14.782 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:02:14 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T00:02:14.782 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:02:14 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:02:14.782 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:02:14 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:02:14.807 DEBUG:teuthology.orchestra.run.smithi012:osd.9> sudo journalctl -f -n 0 -u ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968@osd.9.service 2024-09-11T00:02:14.811 INFO:tasks.cephadm:Deploying osd.10 on smithi165 with /dev/vg_nvme/lv_1... 2024-09-11T00:02:14.812 DEBUG:teuthology.orchestra.run.smithi165:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ff3ad74a-6fcf-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_1 2024-09-11T00:02:14.830 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:02:14 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T00:02:14.830 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:02:14 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:02:14.831 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:02:14 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:02:15.068 INFO:teuthology.orchestra.run.smithi165.stderr:Inferring config /var/lib/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T00:02:15.511 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:02:15 smithi165 ceph-mon[35356]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-09-11T00:02:15.743 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:02:15 smithi012 ceph-mon[30333]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-09-11T00:02:15.830 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:02:15 smithi178 ceph-mon[35919]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-09-11T00:02:16.063 INFO:journalctl@ceph.osd.9.smithi012.stdout:Sep 11 00:02:15 smithi012 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-9[66733]: 2024-09-11T00:02:15.845+0000 7f5f5be35640 -1 osd.9 0 log_to_monitors true 2024-09-11T00:02:16.469 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:02:16 smithi012 ceph-mon[30333]: from='osd.9 [v2:172.21.15.12:6826/3359634903,v1:172.21.15.12:6827/3359634903]' entity='osd.9' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]} : dispatch 2024-09-11T00:02:16.469 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:02:16 smithi012 ceph-mon[30333]: from='osd.9 ' entity='osd.9' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]} : dispatch 2024-09-11T00:02:16.657 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:02:16 smithi165 ceph-mon[35356]: from='osd.9 [v2:172.21.15.12:6826/3359634903,v1:172.21.15.12:6827/3359634903]' entity='osd.9' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]} : dispatch 2024-09-11T00:02:16.657 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:02:16 smithi165 ceph-mon[35356]: from='osd.9 ' entity='osd.9' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]} : dispatch 2024-09-11T00:02:16.830 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:02:16 smithi178 ceph-mon[35919]: from='osd.9 [v2:172.21.15.12:6826/3359634903,v1:172.21.15.12:6827/3359634903]' entity='osd.9' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]} : dispatch 2024-09-11T00:02:16.831 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:02:16 smithi178 ceph-mon[35919]: from='osd.9 ' entity='osd.9' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]} : dispatch 2024-09-11T00:02:17.532 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:02:17 smithi165 ceph-mon[35356]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-09-11T00:02:17.532 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:02:17 smithi165 ceph-mon[35356]: from='osd.9 ' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-09-11T00:02:17.532 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:02:17 smithi165 ceph-mon[35356]: osdmap e56: 10 total, 9 up, 10 in 2024-09-11T00:02:17.533 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:02:17 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-11T00:02:17.533 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:02:17 smithi165 ceph-mon[35356]: from='osd.9 [v2:172.21.15.12:6826/3359634903,v1:172.21.15.12:6827/3359634903]' entity='osd.9' cmd={"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi012", "root=default"]} : dispatch 2024-09-11T00:02:17.533 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:02:17 smithi165 ceph-mon[35356]: from='osd.9 ' entity='osd.9' cmd={"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi012", "root=default"]} : dispatch 2024-09-11T00:02:17.723 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:02:17 smithi012 ceph-mon[30333]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-09-11T00:02:17.723 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:02:17 smithi012 ceph-mon[30333]: from='osd.9 ' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-09-11T00:02:17.723 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:02:17 smithi012 ceph-mon[30333]: osdmap e56: 10 total, 9 up, 10 in 2024-09-11T00:02:17.723 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:02:17 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-11T00:02:17.723 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:02:17 smithi012 ceph-mon[30333]: from='osd.9 [v2:172.21.15.12:6826/3359634903,v1:172.21.15.12:6827/3359634903]' entity='osd.9' cmd={"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi012", "root=default"]} : dispatch 2024-09-11T00:02:17.723 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:02:17 smithi012 ceph-mon[30333]: from='osd.9 ' entity='osd.9' cmd={"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi012", "root=default"]} : dispatch 2024-09-11T00:02:17.830 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:02:17 smithi178 ceph-mon[35919]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-09-11T00:02:17.831 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:02:17 smithi178 ceph-mon[35919]: from='osd.9 ' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-09-11T00:02:17.831 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:02:17 smithi178 ceph-mon[35919]: osdmap e56: 10 total, 9 up, 10 in 2024-09-11T00:02:17.831 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:02:17 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-11T00:02:17.831 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:02:17 smithi178 ceph-mon[35919]: from='osd.9 [v2:172.21.15.12:6826/3359634903,v1:172.21.15.12:6827/3359634903]' entity='osd.9' cmd={"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi012", "root=default"]} : dispatch 2024-09-11T00:02:17.831 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:02:17 smithi178 ceph-mon[35919]: from='osd.9 ' entity='osd.9' cmd={"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi012", "root=default"]} : dispatch 2024-09-11T00:02:17.986 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-09-11T00:02:18.014 DEBUG:teuthology.orchestra.run.smithi165:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ff3ad74a-6fcf-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi165:vg_nvme/lv_1 2024-09-11T00:02:18.269 INFO:teuthology.orchestra.run.smithi165.stderr:Inferring config /var/lib/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T00:02:18.373 INFO:journalctl@ceph.osd.9.smithi012.stdout:Sep 11 00:02:18 smithi012 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-9[66733]: 2024-09-11T00:02:18.210+0000 7f5f56a00640 -1 osd.9 0 waiting for initial osdmap 2024-09-11T00:02:18.374 INFO:journalctl@ceph.osd.9.smithi012.stdout:Sep 11 00:02:18 smithi012 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-9[66733]: 2024-09-11T00:02:18.235+0000 7f5f4ca00640 -1 osd.9 57 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-11T00:02:18.720 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:02:18 smithi165 ceph-mon[35356]: from='osd.9 ' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi012", "root=default"]}]': finished 2024-09-11T00:02:18.720 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:02:18 smithi165 ceph-mon[35356]: osdmap e57: 10 total, 9 up, 10 in 2024-09-11T00:02:18.720 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:02:18 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-11T00:02:18.720 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:02:18 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-11T00:02:18.720 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:02:18 smithi165 ceph-mon[35356]: from='osd.9 ' entity='osd.9' 2024-09-11T00:02:18.812 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:02:18 smithi012 ceph-mon[30333]: from='osd.9 ' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi012", "root=default"]}]': finished 2024-09-11T00:02:18.813 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:02:18 smithi012 ceph-mon[30333]: osdmap e57: 10 total, 9 up, 10 in 2024-09-11T00:02:18.813 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:02:18 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-11T00:02:18.813 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:02:18 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-11T00:02:18.813 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:02:18 smithi012 ceph-mon[30333]: from='osd.9 ' entity='osd.9' 2024-09-11T00:02:18.820 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-11T00:02:18.803+0000 7f7f091d9640 1 -- 172.21.15.165:0/3859957376 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7f04103b90 msgr2=0x7f7f04104010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:02:18.820 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-11T00:02:18.803+0000 7f7f091d9640 1 --2- 172.21.15.165:0/3859957376 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7f04103b90 0x7f7f04104010 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f7eec009930 tx=0x7f7eec02a420 comp rx=0 tx=0).stop 2024-09-11T00:02:18.821 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-11T00:02:18.803+0000 7f7f091d9640 1 -- 172.21.15.165:0/3859957376 shutdown_connections 2024-09-11T00:02:18.821 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-11T00:02:18.803+0000 7f7f091d9640 1 --2- 172.21.15.165:0/3859957376 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7f04104550 0x7f7f0410aed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:02:18.821 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-11T00:02:18.803+0000 7f7f091d9640 1 --2- 172.21.15.165:0/3859957376 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7f04103b90 0x7f7f04104010 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:02:18.821 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-11T00:02:18.803+0000 7f7f091d9640 1 --2- 172.21.15.165:0/3859957376 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7f04102a30 0x7f7f04102e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:02:18.821 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-11T00:02:18.803+0000 7f7f091d9640 1 -- 172.21.15.165:0/3859957376 >> 172.21.15.165:0/3859957376 conn(0x7f7f040fe320 msgr2=0x7f7f04100740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:02:18.821 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-11T00:02:18.803+0000 7f7f091d9640 1 -- 172.21.15.165:0/3859957376 shutdown_connections 2024-09-11T00:02:18.821 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-11T00:02:18.803+0000 7f7f091d9640 1 -- 172.21.15.165:0/3859957376 wait complete. 2024-09-11T00:02:18.821 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-11T00:02:18.804+0000 7f7f091d9640 1 Processor -- start 2024-09-11T00:02:18.821 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-11T00:02:18.804+0000 7f7f091d9640 1 -- start start 2024-09-11T00:02:18.821 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-11T00:02:18.804+0000 7f7f091d9640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7f04102a30 0x7f7f04198ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:02:18.821 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-11T00:02:18.804+0000 7f7f091d9640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7f04103b90 0x7f7f041994f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:02:18.822 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-11T00:02:18.804+0000 7f7f091d9640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7f04104550 0x7f7f0419a130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:02:18.822 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-11T00:02:18.804+0000 7f7f091d9640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f7f0410e710 con 0x7f7f04104550 2024-09-11T00:02:18.822 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-11T00:02:18.804+0000 7f7f091d9640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f7f0410e590 con 0x7f7f04103b90 2024-09-11T00:02:18.822 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-11T00:02:18.804+0000 7f7f091d9640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f7f0410e890 con 0x7f7f04102a30 2024-09-11T00:02:18.822 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-11T00:02:18.804+0000 7f7efbfff640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7f04103b90 0x7f7f041994f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:02:18.822 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-11T00:02:18.804+0000 7f7efbfff640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7f04103b90 0x7f7f041994f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:36900/0 (socket says 172.21.15.165:36900) 2024-09-11T00:02:18.822 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-11T00:02:18.804+0000 7f7efbfff640 1 -- 172.21.15.165:0/304822470 learned_addr learned my addr 172.21.15.165:0/304822470 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-09-11T00:02:18.822 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-11T00:02:18.804+0000 7f7f0359e640 1 --2- 172.21.15.165:0/304822470 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7f04104550 0x7f7f0419a130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:02:18.822 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-11T00:02:18.805+0000 7f7f02d9d640 1 --2- 172.21.15.165:0/304822470 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7f04102a30 0x7f7f04198ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:02:18.822 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-11T00:02:18.805+0000 7f7efbfff640 1 -- 172.21.15.165:0/304822470 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7f04102a30 msgr2=0x7f7f04198ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:02:18.822 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-11T00:02:18.805+0000 7f7efbfff640 1 --2- 172.21.15.165:0/304822470 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7f04102a30 0x7f7f04198ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:02:18.822 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-11T00:02:18.805+0000 7f7efbfff640 1 -- 172.21.15.165:0/304822470 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7f04104550 msgr2=0x7f7f0419a130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:02:18.823 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-11T00:02:18.805+0000 7f7efbfff640 1 --2- 172.21.15.165:0/304822470 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7f04104550 0x7f7f0419a130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:02:18.823 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-11T00:02:18.805+0000 7f7efbfff640 1 -- 172.21.15.165:0/304822470 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f0419a9d0 con 0x7f7f04103b90 2024-09-11T00:02:18.823 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-11T00:02:18.805+0000 7f7efbfff640 1 --2- 172.21.15.165:0/304822470 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7f04103b90 0x7f7f041994f0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f7eec009230 tx=0x7f7eec0042f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:02:18.823 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-11T00:02:18.805+0000 7f7f00d99640 1 -- 172.21.15.165:0/304822470 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7eec03f030 con 0x7f7f04103b90 2024-09-11T00:02:18.823 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-11T00:02:18.805+0000 7f7f00d99640 1 -- 172.21.15.165:0/304822470 <== mon.2 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7eec003cb0 con 0x7f7f04103b90 2024-09-11T00:02:18.823 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-11T00:02:18.805+0000 7f7f091d9640 1 -- 172.21.15.165:0/304822470 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7f0419ac60 con 0x7f7f04103b90 2024-09-11T00:02:18.823 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-11T00:02:18.805+0000 7f7f00d99640 1 -- 172.21.15.165:0/304822470 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7eec035070 con 0x7f7f04103b90 2024-09-11T00:02:18.823 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-11T00:02:18.805+0000 7f7f091d9640 1 -- 172.21.15.165:0/304822470 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7f041a11d0 con 0x7f7f04103b90 2024-09-11T00:02:18.823 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-11T00:02:18.806+0000 7f7f091d9640 1 -- 172.21.15.165:0/304822470 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7f04109d50 con 0x7f7f04103b90 2024-09-11T00:02:18.823 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-11T00:02:18.807+0000 7f7f00d99640 1 -- 172.21.15.165:0/304822470 <== mon.2 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 16) ==== 153250+0+0 (secure 0 0 0) 0x7f7eec03bc90 con 0x7f7f04103b90 2024-09-11T00:02:18.823 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-11T00:02:18.808+0000 7f7f00d99640 1 --2- 172.21.15.165:0/304822470 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f7ef00b52e0 0x7f7ef00b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:02:18.823 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-11T00:02:18.809+0000 7f7f00d99640 1 -- 172.21.15.165:0/304822470 <== mon.2 v2:172.21.15.165:3300/0 5 ==== osd_map(57..57 src has 1..57) ==== 5981+0+0 (secure 0 0 0) 0x7f7eec1060e0 con 0x7f7f04103b90 2024-09-11T00:02:18.824 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-11T00:02:18.809+0000 7f7f02d9d640 1 --2- 172.21.15.165:0/304822470 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f7ef00b52e0 0x7f7ef00b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:02:18.824 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-11T00:02:18.810+0000 7f7f02d9d640 1 --2- 172.21.15.165:0/304822470 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f7ef00b52e0 0x7f7ef00b77a0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f7ef4005fd0 tx=0x7f7ef4005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:02:18.824 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-11T00:02:18.812+0000 7f7f00d99640 1 -- 172.21.15.165:0/304822470 <== mon.2 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f7eec0cd8c0 con 0x7f7f04103b90 2024-09-11T00:02:18.830 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:02:18 smithi178 ceph-mon[35919]: from='osd.9 ' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi012", "root=default"]}]': finished 2024-09-11T00:02:18.830 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:02:18 smithi178 ceph-mon[35919]: osdmap e57: 10 total, 9 up, 10 in 2024-09-11T00:02:18.831 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:02:18 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-11T00:02:18.831 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:02:18 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-11T00:02:18.831 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:02:18 smithi178 ceph-mon[35919]: from='osd.9 ' entity='osd.9' 2024-09-11T00:02:19.000 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-11T00:02:18.998+0000 7f7f091d9640 1 -- 172.21.15.165:0/304822470 --> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi165:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f7f04063590 con 0x7f7ef00b52e0 2024-09-11T00:02:19.581 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:02:19 smithi178 ceph-mon[35919]: purged_snaps scrub starts 2024-09-11T00:02:19.581 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:02:19 smithi178 ceph-mon[35919]: purged_snaps scrub ok 2024-09-11T00:02:19.581 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:02:19 smithi178 ceph-mon[35919]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-09-11T00:02:19.581 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:02:19 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-11T00:02:19.582 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:02:19 smithi178 ceph-mon[35919]: Detected new or changed devices on smithi012 2024-09-11T00:02:19.582 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:02:19 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:02:19.582 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:02:19 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:02:19.582 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:02:19 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-11T00:02:19.582 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:02:19 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-11T00:02:19.582 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:02:19 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-09-11T00:02:19.582 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:02:19 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"} : dispatch 2024-09-11T00:02:19.582 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:02:19 smithi178 ceph-mon[35919]: Adjusting osd_memory_target on smithi012 to 4273M 2024-09-11T00:02:19.582 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:02:19 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:02:19.582 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:02:19 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:02:19.582 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:02:19 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T00:02:19.582 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:02:19 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:02:19.582 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:02:19 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-11T00:02:19.583 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:02:19 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-11T00:02:19.583 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:02:19 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:02:19.583 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:02:19 smithi178 ceph-mon[35919]: osd.9 [v2:172.21.15.12:6826/3359634903,v1:172.21.15.12:6827/3359634903] boot 2024-09-11T00:02:19.583 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:02:19 smithi178 ceph-mon[35919]: osdmap e58: 10 total, 10 up, 10 in 2024-09-11T00:02:19.583 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:02:19 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-11T00:02:19.673 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:02:19 smithi165 ceph-mon[35356]: purged_snaps scrub starts 2024-09-11T00:02:19.673 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:02:19 smithi165 ceph-mon[35356]: purged_snaps scrub ok 2024-09-11T00:02:19.673 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:02:19 smithi165 ceph-mon[35356]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-09-11T00:02:19.674 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:02:19 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-11T00:02:19.674 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:02:19 smithi165 ceph-mon[35356]: Detected new or changed devices on smithi012 2024-09-11T00:02:19.674 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:02:19 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:02:19.674 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:02:19 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:02:19.675 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:02:19 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-11T00:02:19.675 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:02:19 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-11T00:02:19.675 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:02:19 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-09-11T00:02:19.675 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:02:19 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"} : dispatch 2024-09-11T00:02:19.675 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:02:19 smithi165 ceph-mon[35356]: Adjusting osd_memory_target on smithi012 to 4273M 2024-09-11T00:02:19.675 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:02:19 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:02:19.675 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:02:19 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:02:19.675 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:02:19 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T00:02:19.676 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:02:19 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:02:19.676 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:02:19 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-11T00:02:19.676 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:02:19 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-11T00:02:19.676 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:02:19 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:02:19.676 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:02:19 smithi165 ceph-mon[35356]: osd.9 [v2:172.21.15.12:6826/3359634903,v1:172.21.15.12:6827/3359634903] boot 2024-09-11T00:02:19.676 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:02:19 smithi165 ceph-mon[35356]: osdmap e58: 10 total, 10 up, 10 in 2024-09-11T00:02:19.676 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:02:19 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-11T00:02:19.813 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:02:19 smithi012 ceph-mon[30333]: purged_snaps scrub starts 2024-09-11T00:02:19.813 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:02:19 smithi012 ceph-mon[30333]: purged_snaps scrub ok 2024-09-11T00:02:19.813 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:02:19 smithi012 ceph-mon[30333]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-09-11T00:02:19.813 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:02:19 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-11T00:02:19.813 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:02:19 smithi012 ceph-mon[30333]: Detected new or changed devices on smithi012 2024-09-11T00:02:19.813 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:02:19 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:02:19.813 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:02:19 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:02:19.813 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:02:19 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-11T00:02:19.813 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:02:19 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-11T00:02:19.814 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:02:19 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-09-11T00:02:19.814 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:02:19 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"} : dispatch 2024-09-11T00:02:19.814 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:02:19 smithi012 ceph-mon[30333]: Adjusting osd_memory_target on smithi012 to 4273M 2024-09-11T00:02:19.814 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:02:19 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:02:19.814 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:02:19 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:02:19.814 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:02:19 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T00:02:19.814 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:02:19 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:02:19.814 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:02:19 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-11T00:02:19.814 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:02:19 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-11T00:02:19.814 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:02:19 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:02:19.814 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:02:19 smithi012 ceph-mon[30333]: osd.9 [v2:172.21.15.12:6826/3359634903,v1:172.21.15.12:6827/3359634903] boot 2024-09-11T00:02:19.814 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:02:19 smithi012 ceph-mon[30333]: osdmap e58: 10 total, 10 up, 10 in 2024-09-11T00:02:19.814 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:02:19 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-11T00:02:20.562 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:02:20 smithi012 ceph-mon[30333]: from='client.24497 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi165:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T00:02:20.644 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:02:20 smithi165 ceph-mon[35356]: from='client.24497 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi165:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T00:02:20.830 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:02:20 smithi178 ceph-mon[35919]: from='client.24497 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi165:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T00:02:21.707 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:02:21 smithi165 ceph-mon[35356]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-09-11T00:02:21.707 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:02:21 smithi165 ceph-mon[35356]: osdmap e59: 10 total, 10 up, 10 in 2024-09-11T00:02:21.813 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:02:21 smithi012 ceph-mon[30333]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-09-11T00:02:21.813 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:02:21 smithi012 ceph-mon[30333]: osdmap e59: 10 total, 10 up, 10 in 2024-09-11T00:02:21.830 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:02:21 smithi178 ceph-mon[35919]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-09-11T00:02:21.830 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:02:21 smithi178 ceph-mon[35919]: osdmap e59: 10 total, 10 up, 10 in 2024-09-11T00:02:22.782 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:02:22 smithi165 ceph-mon[35356]: from='client.? 172.21.15.165:0/1605629256' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "c1ad1f0a-33f4-4c32-91f6-4d030a29f1dd"} : dispatch 2024-09-11T00:02:22.782 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:02:22 smithi165 ceph-mon[35356]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "c1ad1f0a-33f4-4c32-91f6-4d030a29f1dd"} : dispatch 2024-09-11T00:02:22.782 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:02:22 smithi165 ceph-mon[35356]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c1ad1f0a-33f4-4c32-91f6-4d030a29f1dd"}]': finished 2024-09-11T00:02:22.783 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:02:22 smithi165 ceph-mon[35356]: osdmap e60: 11 total, 10 up, 11 in 2024-09-11T00:02:22.783 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:02:22 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-11T00:02:22.783 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:02:22 smithi165 ceph-mon[35356]: from='client.? 172.21.15.165:0/2490352067' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-11T00:02:22.813 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:02:22 smithi012 ceph-mon[30333]: from='client.? 172.21.15.165:0/1605629256' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "c1ad1f0a-33f4-4c32-91f6-4d030a29f1dd"} : dispatch 2024-09-11T00:02:22.813 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:02:22 smithi012 ceph-mon[30333]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "c1ad1f0a-33f4-4c32-91f6-4d030a29f1dd"} : dispatch 2024-09-11T00:02:22.813 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:02:22 smithi012 ceph-mon[30333]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c1ad1f0a-33f4-4c32-91f6-4d030a29f1dd"}]': finished 2024-09-11T00:02:22.813 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:02:22 smithi012 ceph-mon[30333]: osdmap e60: 11 total, 10 up, 11 in 2024-09-11T00:02:22.813 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:02:22 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-11T00:02:22.813 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:02:22 smithi012 ceph-mon[30333]: from='client.? 172.21.15.165:0/2490352067' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-11T00:02:22.830 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:02:22 smithi178 ceph-mon[35919]: from='client.? 172.21.15.165:0/1605629256' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "c1ad1f0a-33f4-4c32-91f6-4d030a29f1dd"} : dispatch 2024-09-11T00:02:22.830 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:02:22 smithi178 ceph-mon[35919]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "c1ad1f0a-33f4-4c32-91f6-4d030a29f1dd"} : dispatch 2024-09-11T00:02:22.830 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:02:22 smithi178 ceph-mon[35919]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c1ad1f0a-33f4-4c32-91f6-4d030a29f1dd"}]': finished 2024-09-11T00:02:22.831 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:02:22 smithi178 ceph-mon[35919]: osdmap e60: 11 total, 10 up, 11 in 2024-09-11T00:02:22.831 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:02:22 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-11T00:02:22.831 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:02:22 smithi178 ceph-mon[35919]: from='client.? 172.21.15.165:0/2490352067' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-11T00:02:23.782 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:02:23 smithi165 ceph-mon[35356]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-09-11T00:02:23.813 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:02:23 smithi012 ceph-mon[30333]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-09-11T00:02:23.830 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:02:23 smithi178 ceph-mon[35919]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-09-11T00:02:25.782 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:02:25 smithi165 ceph-mon[35356]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-09-11T00:02:25.812 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:02:25 smithi012 ceph-mon[30333]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-09-11T00:02:25.830 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:02:25 smithi178 ceph-mon[35919]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-09-11T00:02:27.776 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:02:27 smithi165 ceph-mon[35356]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-09-11T00:02:27.813 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:02:27 smithi012 ceph-mon[30333]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-09-11T00:02:27.830 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:02:27 smithi178 ceph-mon[35919]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-09-11T00:02:29.532 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:02:29 smithi165 ceph-mon[35356]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-09-11T00:02:29.812 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:02:29 smithi012 ceph-mon[30333]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-09-11T00:02:29.830 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:02:29 smithi178 ceph-mon[35919]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-09-11T00:02:30.782 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:02:30 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.10"} : dispatch 2024-09-11T00:02:30.782 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:02:30 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:02:30.812 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:02:30 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.10"} : dispatch 2024-09-11T00:02:30.813 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:02:30 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:02:30.830 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:02:30 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.10"} : dispatch 2024-09-11T00:02:30.831 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:02:30 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:02:31.782 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:02:31 smithi165 ceph-mon[35356]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-09-11T00:02:31.782 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:02:31 smithi165 ceph-mon[35356]: Deploying daemon osd.10 on smithi165 2024-09-11T00:02:31.812 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:02:31 smithi012 ceph-mon[30333]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-09-11T00:02:31.813 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:02:31 smithi012 ceph-mon[30333]: Deploying daemon osd.10 on smithi165 2024-09-11T00:02:31.831 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:02:31 smithi178 ceph-mon[35919]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-09-11T00:02:31.831 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:02:31 smithi178 ceph-mon[35919]: Deploying daemon osd.10 on smithi165 2024-09-11T00:02:33.782 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:02:33 smithi165 ceph-mon[35356]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-09-11T00:02:33.812 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:02:33 smithi012 ceph-mon[30333]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-09-11T00:02:33.830 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:02:33 smithi178 ceph-mon[35919]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-09-11T00:02:35.779 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:02:35 smithi165 ceph-mon[35356]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-09-11T00:02:35.812 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:02:35 smithi012 ceph-mon[30333]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-09-11T00:02:35.830 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:02:35 smithi178 ceph-mon[35919]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-09-11T00:02:36.782 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:02:36 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T00:02:36.782 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:02:36 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:02:36.782 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:02:36 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:02:36.812 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:02:36 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T00:02:36.813 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:02:36 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:02:36.813 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:02:36 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:02:36.830 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:02:36 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T00:02:36.831 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:02:36 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:02:36.831 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:02:36 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:02:37.651 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:02:37 smithi165 ceph-mon[35356]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-09-11T00:02:37.813 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:02:37 smithi012 ceph-mon[30333]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-09-11T00:02:37.830 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:02:37 smithi178 ceph-mon[35919]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-09-11T00:02:39.782 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:02:39 smithi165 ceph-mon[35356]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-09-11T00:02:39.782 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:02:39 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:02:39.782 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:02:39 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:02:39.782 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:02:39 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:02:39.783 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:02:39 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T00:02:39.783 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:02:39 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:02:39.812 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:02:39 smithi012 ceph-mon[30333]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-09-11T00:02:39.813 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:02:39 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:02:39.813 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:02:39 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:02:39.813 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:02:39 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:02:39.813 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:02:39 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T00:02:39.813 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:02:39 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:02:39.830 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:02:39 smithi178 ceph-mon[35919]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-09-11T00:02:39.830 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:02:39 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:02:39.831 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:02:39 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:02:39.831 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:02:39 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:02:39.831 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:02:39 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T00:02:39.831 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:02:39 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:02:39.873 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-11T00:02:39.870+0000 7f7f00d99640 1 -- 172.21.15.165:0/304822470 <== mgr.14150 v2:172.21.15.12:6800/3442939137 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7f7f04063590 con 0x7f7ef00b52e0 2024-09-11T00:02:39.874 INFO:teuthology.orchestra.run.smithi165.stdout:Created osd(s) 10 on host 'smithi165' 2024-09-11T00:02:39.875 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-11T00:02:39.873+0000 7f7f091d9640 1 -- 172.21.15.165:0/304822470 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f7ef00b52e0 msgr2=0x7f7ef00b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:02:39.875 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-11T00:02:39.873+0000 7f7f091d9640 1 --2- 172.21.15.165:0/304822470 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f7ef00b52e0 0x7f7ef00b77a0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f7ef4005fd0 tx=0x7f7ef4005850 comp rx=0 tx=0).stop 2024-09-11T00:02:39.875 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-11T00:02:39.873+0000 7f7f091d9640 1 -- 172.21.15.165:0/304822470 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7f04103b90 msgr2=0x7f7f041994f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:02:39.875 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-11T00:02:39.873+0000 7f7f091d9640 1 --2- 172.21.15.165:0/304822470 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7f04103b90 0x7f7f041994f0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f7eec009230 tx=0x7f7eec0042f0 comp rx=0 tx=0).stop 2024-09-11T00:02:39.876 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-11T00:02:39.873+0000 7f7f091d9640 1 -- 172.21.15.165:0/304822470 shutdown_connections 2024-09-11T00:02:39.876 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-11T00:02:39.873+0000 7f7f091d9640 1 --2- 172.21.15.165:0/304822470 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f7ef00b52e0 0x7f7ef00b77a0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:02:39.876 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-11T00:02:39.873+0000 7f7f091d9640 1 --2- 172.21.15.165:0/304822470 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7f04104550 0x7f7f0419a130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:02:39.876 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-11T00:02:39.873+0000 7f7f091d9640 1 --2- 172.21.15.165:0/304822470 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7f04103b90 0x7f7f041994f0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:02:39.876 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-11T00:02:39.873+0000 7f7f091d9640 1 --2- 172.21.15.165:0/304822470 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7f04102a30 0x7f7f04198ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:02:39.876 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-11T00:02:39.873+0000 7f7f091d9640 1 -- 172.21.15.165:0/304822470 >> 172.21.15.165:0/304822470 conn(0x7f7f040fe320 msgr2=0x7f7f04106f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:02:39.876 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-11T00:02:39.873+0000 7f7f091d9640 1 -- 172.21.15.165:0/304822470 shutdown_connections 2024-09-11T00:02:39.876 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-11T00:02:39.873+0000 7f7f091d9640 1 -- 172.21.15.165:0/304822470 wait complete. 2024-09-11T00:02:40.603 DEBUG:teuthology.orchestra.run.smithi165:osd.10> sudo journalctl -f -n 0 -u ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968@osd.10.service 2024-09-11T00:02:40.605 INFO:tasks.cephadm:Deploying osd.11 on smithi178 with /dev/vg_nvme/lv_1... 2024-09-11T00:02:40.605 DEBUG:teuthology.orchestra.run.smithi178:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ff3ad74a-6fcf-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_1 2024-09-11T00:02:40.782 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:02:40 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T00:02:40.782 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:02:40 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:02:40.782 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:02:40 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:02:40.812 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:02:40 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T00:02:40.813 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:02:40 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:02:40.813 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:02:40 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:02:40.830 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:02:40 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T00:02:40.830 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:02:40 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:02:40.831 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:02:40 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:02:40.863 INFO:teuthology.orchestra.run.smithi178.stderr:Inferring config /var/lib/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/mon.c/config 2024-09-11T00:02:42.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:02:41 smithi165 ceph-mon[35356]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-09-11T00:02:42.128 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:02:41 smithi178 ceph-mon[35919]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-09-11T00:02:42.312 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:02:41 smithi012 ceph-mon[30333]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-09-11T00:02:42.532 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 11 00:02:42 smithi165 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-10[59927]: 2024-09-11T00:02:42.137+0000 7f2d05e70640 -1 osd.10 0 log_to_monitors true 2024-09-11T00:02:43.043 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:02:42 smithi178 ceph-mon[35919]: from='osd.10 [v2:172.21.15.165:6824/3642325778,v1:172.21.15.165:6825/3642325778]' entity='osd.10' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]} : dispatch 2024-09-11T00:02:43.043 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:02:42 smithi178 ceph-mon[35919]: from='osd.10 ' entity='osd.10' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]} : dispatch 2024-09-11T00:02:43.200 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:02:42 smithi165 ceph-mon[35356]: from='osd.10 [v2:172.21.15.165:6824/3642325778,v1:172.21.15.165:6825/3642325778]' entity='osd.10' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]} : dispatch 2024-09-11T00:02:43.200 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:02:42 smithi165 ceph-mon[35356]: from='osd.10 ' entity='osd.10' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]} : dispatch 2024-09-11T00:02:43.312 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:02:42 smithi012 ceph-mon[30333]: from='osd.10 [v2:172.21.15.165:6824/3642325778,v1:172.21.15.165:6825/3642325778]' entity='osd.10' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]} : dispatch 2024-09-11T00:02:43.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:02:42 smithi012 ceph-mon[30333]: from='osd.10 ' entity='osd.10' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]} : dispatch 2024-09-11T00:02:43.330 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-09-11T00:02:43.355 DEBUG:teuthology.orchestra.run.smithi178:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ff3ad74a-6fcf-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi178:vg_nvme/lv_1 2024-09-11T00:02:43.605 INFO:teuthology.orchestra.run.smithi178.stderr:Inferring config /var/lib/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/mon.c/config 2024-09-11T00:02:44.106 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T00:02:44.090+0000 7f3827a49640 1 -- 172.21.15.178:0/2568287634 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f382006b680 msgr2=0x7f3820105650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:02:44.106 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T00:02:44.090+0000 7f3827a49640 1 --2- 172.21.15.178:0/2568287634 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f382006b680 0x7f3820105650 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f381c009900 tx=0x7f381c02a410 comp rx=0 tx=0).stop 2024-09-11T00:02:44.106 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T00:02:44.090+0000 7f3827a49640 1 -- 172.21.15.178:0/2568287634 shutdown_connections 2024-09-11T00:02:44.106 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T00:02:44.090+0000 7f3827a49640 1 --2- 172.21.15.178:0/2568287634 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3820108720 0x7f382010abe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:02:44.106 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T00:02:44.090+0000 7f3827a49640 1 --2- 172.21.15.178:0/2568287634 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3820105b90 0x7f3820107fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:02:44.107 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T00:02:44.090+0000 7f3827a49640 1 --2- 172.21.15.178:0/2568287634 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f382006b680 0x7f3820105650 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:02:44.107 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T00:02:44.090+0000 7f3827a49640 1 -- 172.21.15.178:0/2568287634 >> 172.21.15.178:0/2568287634 conn(0x7f38200fd1e0 msgr2=0x7f38200ff620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:02:44.107 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T00:02:44.090+0000 7f3827a49640 1 -- 172.21.15.178:0/2568287634 shutdown_connections 2024-09-11T00:02:44.107 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T00:02:44.090+0000 7f3827a49640 1 -- 172.21.15.178:0/2568287634 wait complete. 2024-09-11T00:02:44.107 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T00:02:44.090+0000 7f3827a49640 1 Processor -- start 2024-09-11T00:02:44.107 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T00:02:44.091+0000 7f3827a49640 1 -- start start 2024-09-11T00:02:44.107 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T00:02:44.091+0000 7f3827a49640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f382006b680 0x7f3820198e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:02:44.107 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T00:02:44.091+0000 7f3827a49640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3820105b90 0x7f3820199450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:02:44.107 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T00:02:44.091+0000 7f3827a49640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3820108720 0x7f382019a090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:02:44.107 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T00:02:44.091+0000 7f3827a49640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f382010e680 con 0x7f3820105b90 2024-09-11T00:02:44.107 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T00:02:44.091+0000 7f3827a49640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f382010e500 con 0x7f382006b680 2024-09-11T00:02:44.107 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T00:02:44.091+0000 7f3827a49640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f382010e800 con 0x7f3820108720 2024-09-11T00:02:44.108 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T00:02:44.091+0000 7f3825fe6640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3820108720 0x7f382019a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:02:44.108 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T00:02:44.091+0000 7f3825fe6640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3820108720 0x7f382019a090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:43874/0 (socket says 172.21.15.178:43874) 2024-09-11T00:02:44.108 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T00:02:44.091+0000 7f3825fe6640 1 -- 172.21.15.178:0/4104428819 learned_addr learned my addr 172.21.15.178:0/4104428819 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-11T00:02:44.108 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T00:02:44.091+0000 7f38257e5640 1 --2- 172.21.15.178:0/4104428819 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f382006b680 0x7f3820198e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:02:44.108 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T00:02:44.091+0000 7f3825fe6640 1 -- 172.21.15.178:0/4104428819 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f382006b680 msgr2=0x7f3820198e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:02:44.108 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T00:02:44.091+0000 7f3825fe6640 1 --2- 172.21.15.178:0/4104428819 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f382006b680 0x7f3820198e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:02:44.108 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T00:02:44.091+0000 7f3824fe4640 1 --2- 172.21.15.178:0/4104428819 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3820105b90 0x7f3820199450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:02:44.108 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T00:02:44.091+0000 7f3825fe6640 1 -- 172.21.15.178:0/4104428819 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3820105b90 msgr2=0x7f3820199450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:02:44.108 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T00:02:44.091+0000 7f3825fe6640 1 --2- 172.21.15.178:0/4104428819 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3820105b90 0x7f3820199450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:02:44.108 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T00:02:44.091+0000 7f3825fe6640 1 -- 172.21.15.178:0/4104428819 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f382019a930 con 0x7f3820108720 2024-09-11T00:02:44.108 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T00:02:44.091+0000 7f3824fe4640 1 --2- 172.21.15.178:0/4104428819 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3820105b90 0x7f3820199450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T00:02:44.108 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T00:02:44.091+0000 7f3825fe6640 1 --2- 172.21.15.178:0/4104428819 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3820108720 0x7f382019a090 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f381800eaa0 tx=0x7f381800ef60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:02:44.109 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T00:02:44.091+0000 7f38127fc640 1 -- 172.21.15.178:0/4104428819 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f381800cdb0 con 0x7f3820108720 2024-09-11T00:02:44.109 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T00:02:44.092+0000 7f38127fc640 1 -- 172.21.15.178:0/4104428819 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3818005020 con 0x7f3820108720 2024-09-11T00:02:44.109 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T00:02:44.092+0000 7f3827a49640 1 -- 172.21.15.178:0/4104428819 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f382019ac20 con 0x7f3820108720 2024-09-11T00:02:44.109 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T00:02:44.092+0000 7f38127fc640 1 -- 172.21.15.178:0/4104428819 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38180116d0 con 0x7f3820108720 2024-09-11T00:02:44.109 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T00:02:44.092+0000 7f3827a49640 1 -- 172.21.15.178:0/4104428819 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38201a1300 con 0x7f3820108720 2024-09-11T00:02:44.109 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T00:02:44.093+0000 7f3827a49640 1 -- 172.21.15.178:0/4104428819 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f38200fecd0 con 0x7f3820108720 2024-09-11T00:02:44.109 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T00:02:44.094+0000 7f38127fc640 1 -- 172.21.15.178:0/4104428819 <== mon.1 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 16) ==== 153250+0+0 (secure 0 0 0) 0x7f381800f040 con 0x7f3820108720 2024-09-11T00:02:44.109 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T00:02:44.095+0000 7f38127fc640 1 --2- 172.21.15.178:0/4104428819 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f380c0b52e0 0x7f380c0b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:02:44.109 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T00:02:44.095+0000 7f38127fc640 1 -- 172.21.15.178:0/4104428819 <== mon.1 v2:172.21.15.178:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 6414+0+0 (secure 0 0 0) 0x7f38180e5700 con 0x7f3820108720 2024-09-11T00:02:44.109 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T00:02:44.095+0000 7f38257e5640 1 --2- 172.21.15.178:0/4104428819 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f380c0b52e0 0x7f380c0b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:02:44.109 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T00:02:44.095+0000 7f38257e5640 1 --2- 172.21.15.178:0/4104428819 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f380c0b52e0 0x7f380c0b77a0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f381c000c00 tx=0x7f381c0314a0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:02:44.109 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T00:02:44.101+0000 7f38127fc640 1 -- 172.21.15.178:0/4104428819 <== mon.1 v2:172.21.15.178:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f3818016030 con 0x7f3820108720 2024-09-11T00:02:44.282 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 11 00:02:43 smithi165 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-10[59927]: 2024-09-11T00:02:43.889+0000 7f2d01df3640 -1 osd.10 0 waiting for initial osdmap 2024-09-11T00:02:44.282 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 11 00:02:43 smithi165 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-10[59927]: 2024-09-11T00:02:43.898+0000 7f2cfdc1a640 -1 osd.10 62 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-11T00:02:44.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:02:43 smithi165 ceph-mon[35356]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-09-11T00:02:44.283 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:02:43 smithi165 ceph-mon[35356]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-09-11T00:02:44.283 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:02:43 smithi165 ceph-mon[35356]: osdmap e61: 11 total, 10 up, 11 in 2024-09-11T00:02:44.283 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:02:43 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-11T00:02:44.283 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:02:43 smithi165 ceph-mon[35356]: from='osd.10 [v2:172.21.15.165:6824/3642325778,v1:172.21.15.165:6825/3642325778]' entity='osd.10' cmd={"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi165", "root=default"]} : dispatch 2024-09-11T00:02:44.283 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:02:43 smithi165 ceph-mon[35356]: from='osd.10 ' entity='osd.10' cmd={"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi165", "root=default"]} : dispatch 2024-09-11T00:02:44.283 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:02:43 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:02:44.283 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:02:43 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:02:44.283 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:02:43 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-11T00:02:44.283 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:02:43 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-11T00:02:44.283 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:02:43 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-09-11T00:02:44.283 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:02:43 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"} : dispatch 2024-09-11T00:02:44.283 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:02:43 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:02:44.283 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:02:43 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:02:44.284 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:02:43 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T00:02:44.284 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:02:43 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:02:44.295 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T00:02:44.293+0000 7f3827a49640 1 -- 172.21.15.178:0/4104428819 --> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi178:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f3820063590 con 0x7f380c0b52e0 2024-09-11T00:02:44.296 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:02:43 smithi178 ceph-mon[35919]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-09-11T00:02:44.296 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:02:43 smithi178 ceph-mon[35919]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-09-11T00:02:44.296 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:02:43 smithi178 ceph-mon[35919]: osdmap e61: 11 total, 10 up, 11 in 2024-09-11T00:02:44.296 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:02:43 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-11T00:02:44.296 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:02:43 smithi178 ceph-mon[35919]: from='osd.10 [v2:172.21.15.165:6824/3642325778,v1:172.21.15.165:6825/3642325778]' entity='osd.10' cmd={"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi165", "root=default"]} : dispatch 2024-09-11T00:02:44.296 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:02:43 smithi178 ceph-mon[35919]: from='osd.10 ' entity='osd.10' cmd={"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi165", "root=default"]} : dispatch 2024-09-11T00:02:44.296 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:02:43 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:02:44.296 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:02:43 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:02:44.296 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:02:43 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-11T00:02:44.296 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:02:43 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-11T00:02:44.297 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:02:43 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-09-11T00:02:44.297 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:02:43 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"} : dispatch 2024-09-11T00:02:44.297 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:02:43 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:02:44.297 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:02:43 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:02:44.297 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:02:43 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T00:02:44.297 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:02:43 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:02:44.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:02:43 smithi012 ceph-mon[30333]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-09-11T00:02:44.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:02:43 smithi012 ceph-mon[30333]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-09-11T00:02:44.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:02:43 smithi012 ceph-mon[30333]: osdmap e61: 11 total, 10 up, 11 in 2024-09-11T00:02:44.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:02:43 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-11T00:02:44.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:02:43 smithi012 ceph-mon[30333]: from='osd.10 [v2:172.21.15.165:6824/3642325778,v1:172.21.15.165:6825/3642325778]' entity='osd.10' cmd={"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi165", "root=default"]} : dispatch 2024-09-11T00:02:44.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:02:43 smithi012 ceph-mon[30333]: from='osd.10 ' entity='osd.10' cmd={"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi165", "root=default"]} : dispatch 2024-09-11T00:02:44.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:02:43 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:02:44.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:02:43 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:02:44.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:02:43 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-11T00:02:44.314 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:02:43 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-11T00:02:44.314 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:02:43 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-09-11T00:02:44.314 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:02:43 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"} : dispatch 2024-09-11T00:02:44.314 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:02:43 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:02:44.314 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:02:43 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:02:44.314 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:02:43 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T00:02:44.314 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:02:43 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:02:44.893 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:02:44 smithi178 ceph-mon[35919]: Detected new or changed devices on smithi165 2024-09-11T00:02:44.893 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:02:44 smithi178 ceph-mon[35919]: Adjusting osd_memory_target on smithi165 to 4272M 2024-09-11T00:02:44.893 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:02:44 smithi178 ceph-mon[35919]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi165", "root=default"]}]': finished 2024-09-11T00:02:44.893 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:02:44 smithi178 ceph-mon[35919]: osdmap e62: 11 total, 10 up, 11 in 2024-09-11T00:02:44.893 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:02:44 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-11T00:02:44.893 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:02:44 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-11T00:02:44.894 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:02:44 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-11T00:02:44.894 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:02:44 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-11T00:02:44.894 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:02:44 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:02:45.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:02:44 smithi165 ceph-mon[35356]: Detected new or changed devices on smithi165 2024-09-11T00:02:45.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:02:44 smithi165 ceph-mon[35356]: Adjusting osd_memory_target on smithi165 to 4272M 2024-09-11T00:02:45.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:02:44 smithi165 ceph-mon[35356]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi165", "root=default"]}]': finished 2024-09-11T00:02:45.283 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:02:44 smithi165 ceph-mon[35356]: osdmap e62: 11 total, 10 up, 11 in 2024-09-11T00:02:45.283 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:02:44 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-11T00:02:45.283 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:02:44 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-11T00:02:45.283 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:02:44 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-11T00:02:45.283 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:02:44 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-11T00:02:45.283 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:02:44 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:02:45.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:02:44 smithi012 ceph-mon[30333]: Detected new or changed devices on smithi165 2024-09-11T00:02:45.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:02:44 smithi012 ceph-mon[30333]: Adjusting osd_memory_target on smithi165 to 4272M 2024-09-11T00:02:45.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:02:44 smithi012 ceph-mon[30333]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi165", "root=default"]}]': finished 2024-09-11T00:02:45.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:02:44 smithi012 ceph-mon[30333]: osdmap e62: 11 total, 10 up, 11 in 2024-09-11T00:02:45.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:02:44 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-11T00:02:45.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:02:44 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-11T00:02:45.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:02:44 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-11T00:02:45.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:02:44 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-11T00:02:45.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:02:44 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:02:46.226 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:02:45 smithi178 ceph-mon[35919]: purged_snaps scrub starts 2024-09-11T00:02:46.226 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:02:45 smithi178 ceph-mon[35919]: purged_snaps scrub ok 2024-09-11T00:02:46.226 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:02:45 smithi178 ceph-mon[35919]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-09-11T00:02:46.226 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:02:45 smithi178 ceph-mon[35919]: from='client.24499 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi178:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T00:02:46.226 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:02:45 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-11T00:02:46.226 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:02:45 smithi178 ceph-mon[35919]: osd.10 [v2:172.21.15.165:6824/3642325778,v1:172.21.15.165:6825/3642325778] boot 2024-09-11T00:02:46.227 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:02:45 smithi178 ceph-mon[35919]: osdmap e63: 11 total, 11 up, 11 in 2024-09-11T00:02:46.227 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:02:45 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-11T00:02:46.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:02:45 smithi165 ceph-mon[35356]: purged_snaps scrub starts 2024-09-11T00:02:46.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:02:45 smithi165 ceph-mon[35356]: purged_snaps scrub ok 2024-09-11T00:02:46.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:02:45 smithi165 ceph-mon[35356]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-09-11T00:02:46.283 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:02:45 smithi165 ceph-mon[35356]: from='client.24499 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi178:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T00:02:46.283 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:02:45 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-11T00:02:46.283 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:02:45 smithi165 ceph-mon[35356]: osd.10 [v2:172.21.15.165:6824/3642325778,v1:172.21.15.165:6825/3642325778] boot 2024-09-11T00:02:46.283 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:02:45 smithi165 ceph-mon[35356]: osdmap e63: 11 total, 11 up, 11 in 2024-09-11T00:02:46.283 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:02:45 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-11T00:02:46.312 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:02:45 smithi012 ceph-mon[30333]: purged_snaps scrub starts 2024-09-11T00:02:46.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:02:45 smithi012 ceph-mon[30333]: purged_snaps scrub ok 2024-09-11T00:02:46.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:02:45 smithi012 ceph-mon[30333]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-09-11T00:02:46.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:02:45 smithi012 ceph-mon[30333]: from='client.24499 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi178:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T00:02:46.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:02:45 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-11T00:02:46.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:02:45 smithi012 ceph-mon[30333]: osd.10 [v2:172.21.15.165:6824/3642325778,v1:172.21.15.165:6825/3642325778] boot 2024-09-11T00:02:46.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:02:45 smithi012 ceph-mon[30333]: osdmap e63: 11 total, 11 up, 11 in 2024-09-11T00:02:46.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:02:45 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-11T00:02:47.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:02:46 smithi178 ceph-mon[35919]: osdmap e64: 11 total, 11 up, 11 in 2024-09-11T00:02:47.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:02:46 smithi178 ceph-mon[35919]: from='client.? 172.21.15.178:0/3473095766' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "8413c9b0-680b-4d02-aeab-b3b3ff16c3d6"} : dispatch 2024-09-11T00:02:47.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:02:46 smithi178 ceph-mon[35919]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "8413c9b0-680b-4d02-aeab-b3b3ff16c3d6"} : dispatch 2024-09-11T00:02:47.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:02:46 smithi178 ceph-mon[35919]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8413c9b0-680b-4d02-aeab-b3b3ff16c3d6"}]': finished 2024-09-11T00:02:47.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:02:46 smithi178 ceph-mon[35919]: osdmap e65: 12 total, 11 up, 12 in 2024-09-11T00:02:47.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:02:46 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-11T00:02:47.285 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:02:46 smithi165 ceph-mon[35356]: osdmap e64: 11 total, 11 up, 11 in 2024-09-11T00:02:47.285 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:02:46 smithi165 ceph-mon[35356]: from='client.? 172.21.15.178:0/3473095766' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "8413c9b0-680b-4d02-aeab-b3b3ff16c3d6"} : dispatch 2024-09-11T00:02:47.285 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:02:46 smithi165 ceph-mon[35356]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "8413c9b0-680b-4d02-aeab-b3b3ff16c3d6"} : dispatch 2024-09-11T00:02:47.285 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:02:46 smithi165 ceph-mon[35356]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8413c9b0-680b-4d02-aeab-b3b3ff16c3d6"}]': finished 2024-09-11T00:02:47.285 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:02:46 smithi165 ceph-mon[35356]: osdmap e65: 12 total, 11 up, 12 in 2024-09-11T00:02:47.285 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:02:46 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-11T00:02:47.312 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:02:46 smithi012 ceph-mon[30333]: osdmap e64: 11 total, 11 up, 11 in 2024-09-11T00:02:47.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:02:46 smithi012 ceph-mon[30333]: from='client.? 172.21.15.178:0/3473095766' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "8413c9b0-680b-4d02-aeab-b3b3ff16c3d6"} : dispatch 2024-09-11T00:02:47.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:02:46 smithi012 ceph-mon[30333]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "8413c9b0-680b-4d02-aeab-b3b3ff16c3d6"} : dispatch 2024-09-11T00:02:47.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:02:46 smithi012 ceph-mon[30333]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8413c9b0-680b-4d02-aeab-b3b3ff16c3d6"}]': finished 2024-09-11T00:02:47.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:02:46 smithi012 ceph-mon[30333]: osdmap e65: 12 total, 11 up, 12 in 2024-09-11T00:02:47.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:02:46 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-11T00:02:48.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:02:47 smithi165 ceph-mon[35356]: pgmap v248: 1 pgs: 1 remapped+peering; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail 2024-09-11T00:02:48.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:02:47 smithi165 ceph-mon[35356]: from='client.? 172.21.15.178:0/3246237614' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-11T00:02:48.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:02:47 smithi012 ceph-mon[30333]: pgmap v248: 1 pgs: 1 remapped+peering; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail 2024-09-11T00:02:48.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:02:47 smithi012 ceph-mon[30333]: from='client.? 172.21.15.178:0/3246237614' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-11T00:02:48.330 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:02:47 smithi178 ceph-mon[35919]: pgmap v248: 1 pgs: 1 remapped+peering; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail 2024-09-11T00:02:48.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:02:47 smithi178 ceph-mon[35919]: from='client.? 172.21.15.178:0/3246237614' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-11T00:02:49.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:02:48 smithi165 ceph-mon[35356]: pgmap v250: 1 pgs: 1 remapped+peering; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail 2024-09-11T00:02:49.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:02:48 smithi012 ceph-mon[30333]: pgmap v250: 1 pgs: 1 remapped+peering; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail 2024-09-11T00:02:49.330 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:02:48 smithi178 ceph-mon[35919]: pgmap v250: 1 pgs: 1 remapped+peering; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail 2024-09-11T00:02:51.492 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:02:51 smithi178 ceph-mon[35919]: pgmap v251: 1 pgs: 1 remapped+peering; 577 KiB data, 332 MiB used, 983 GiB / 983 GiB avail 2024-09-11T00:02:51.532 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:02:51 smithi165 ceph-mon[35356]: pgmap v251: 1 pgs: 1 remapped+peering; 577 KiB data, 332 MiB used, 983 GiB / 983 GiB avail 2024-09-11T00:02:51.562 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:02:51 smithi012 ceph-mon[30333]: pgmap v251: 1 pgs: 1 remapped+peering; 577 KiB data, 332 MiB used, 983 GiB / 983 GiB avail 2024-09-11T00:02:53.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:02:53 smithi178 ceph-mon[35919]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 332 MiB used, 983 GiB / 983 GiB avail; 80 KiB/s, 0 objects/s recovering 2024-09-11T00:02:53.532 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:02:53 smithi165 ceph-mon[35356]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 332 MiB used, 983 GiB / 983 GiB avail; 80 KiB/s, 0 objects/s recovering 2024-09-11T00:02:53.562 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:02:53 smithi012 ceph-mon[30333]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 332 MiB used, 983 GiB / 983 GiB avail; 80 KiB/s, 0 objects/s recovering 2024-09-11T00:02:55.330 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:02:55 smithi178 ceph-mon[35919]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 332 MiB used, 983 GiB / 983 GiB avail; 71 KiB/s, 0 objects/s recovering 2024-09-11T00:02:55.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:02:55 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.11"} : dispatch 2024-09-11T00:02:55.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:02:55 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:02:55.532 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:02:55 smithi165 ceph-mon[35356]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 332 MiB used, 983 GiB / 983 GiB avail; 71 KiB/s, 0 objects/s recovering 2024-09-11T00:02:55.533 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:02:55 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.11"} : dispatch 2024-09-11T00:02:55.533 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:02:55 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:02:55.562 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:02:55 smithi012 ceph-mon[30333]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 332 MiB used, 983 GiB / 983 GiB avail; 71 KiB/s, 0 objects/s recovering 2024-09-11T00:02:55.563 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:02:55 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.11"} : dispatch 2024-09-11T00:02:55.563 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:02:55 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:02:56.511 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:02:56 smithi178 ceph-mon[35919]: Deploying daemon osd.11 on smithi178 2024-09-11T00:02:56.532 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:02:56 smithi165 ceph-mon[35356]: Deploying daemon osd.11 on smithi178 2024-09-11T00:02:56.563 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:02:56 smithi012 ceph-mon[30333]: Deploying daemon osd.11 on smithi178 2024-09-11T00:02:57.394 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:02:57 smithi178 ceph-mon[35919]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 332 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-11T00:02:57.532 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:02:57 smithi165 ceph-mon[35356]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 332 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-11T00:02:57.562 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:02:57 smithi012 ceph-mon[30333]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 332 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-11T00:02:59.532 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:02:59 smithi165 ceph-mon[35356]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 332 MiB used, 983 GiB / 983 GiB avail; 50 KiB/s, 0 objects/s recovering 2024-09-11T00:02:59.538 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:02:59 smithi178 ceph-mon[35919]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 332 MiB used, 983 GiB / 983 GiB avail; 50 KiB/s, 0 objects/s recovering 2024-09-11T00:02:59.563 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:02:59 smithi012 ceph-mon[30333]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 332 MiB used, 983 GiB / 983 GiB avail; 50 KiB/s, 0 objects/s recovering 2024-09-11T00:03:01.452 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:03:01 smithi178 ceph-mon[35919]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 332 MiB used, 983 GiB / 983 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-11T00:03:01.452 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:03:01 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T00:03:01.452 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:03:01 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:03:01.452 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:03:01 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:03:01.532 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:03:01 smithi165 ceph-mon[35356]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 332 MiB used, 983 GiB / 983 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-11T00:03:01.532 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:03:01 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T00:03:01.532 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:03:01 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:03:01.532 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:03:01 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:03:01.563 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:03:01 smithi012 ceph-mon[30333]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 332 MiB used, 983 GiB / 983 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-11T00:03:01.563 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:03:01 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T00:03:01.563 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:03:01 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:03:01.563 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:03:01 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:03:03.532 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:03:03 smithi178 ceph-mon[35919]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 332 MiB used, 983 GiB / 983 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-11T00:03:03.532 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:03:03 smithi165 ceph-mon[35356]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 332 MiB used, 983 GiB / 983 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-11T00:03:03.562 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:03:03 smithi012 ceph-mon[30333]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 332 MiB used, 983 GiB / 983 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-11T00:03:04.104 INFO:teuthology.orchestra.run.smithi178.stdout:Created osd(s) 11 on host 'smithi178' 2024-09-11T00:03:04.105 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T00:03:04.103+0000 7f38127fc640 1 -- 172.21.15.178:0/4104428819 <== mgr.14150 v2:172.21.15.12:6800/3442939137 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7f3820063590 con 0x7f380c0b52e0 2024-09-11T00:03:04.107 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T00:03:04.106+0000 7f3827a49640 1 -- 172.21.15.178:0/4104428819 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f380c0b52e0 msgr2=0x7f380c0b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:03:04.107 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T00:03:04.106+0000 7f3827a49640 1 --2- 172.21.15.178:0/4104428819 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f380c0b52e0 0x7f380c0b77a0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f381c000c00 tx=0x7f381c0314a0 comp rx=0 tx=0).stop 2024-09-11T00:03:04.107 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T00:03:04.106+0000 7f3827a49640 1 -- 172.21.15.178:0/4104428819 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3820108720 msgr2=0x7f382019a090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:03:04.107 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T00:03:04.106+0000 7f3827a49640 1 --2- 172.21.15.178:0/4104428819 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3820108720 0x7f382019a090 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f381800eaa0 tx=0x7f381800ef60 comp rx=0 tx=0).stop 2024-09-11T00:03:04.107 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T00:03:04.106+0000 7f3827a49640 1 -- 172.21.15.178:0/4104428819 shutdown_connections 2024-09-11T00:03:04.107 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T00:03:04.106+0000 7f3827a49640 1 --2- 172.21.15.178:0/4104428819 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f380c0b52e0 0x7f380c0b77a0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:03:04.107 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T00:03:04.106+0000 7f3827a49640 1 --2- 172.21.15.178:0/4104428819 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3820108720 0x7f382019a090 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:03:04.108 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T00:03:04.106+0000 7f3827a49640 1 --2- 172.21.15.178:0/4104428819 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3820105b90 0x7f3820199450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:03:04.108 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T00:03:04.106+0000 7f3827a49640 1 --2- 172.21.15.178:0/4104428819 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f382006b680 0x7f3820198e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:03:04.108 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T00:03:04.106+0000 7f3827a49640 1 -- 172.21.15.178:0/4104428819 >> 172.21.15.178:0/4104428819 conn(0x7f38200fd1e0 msgr2=0x7f3820103e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:03:04.108 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T00:03:04.106+0000 7f3827a49640 1 -- 172.21.15.178:0/4104428819 shutdown_connections 2024-09-11T00:03:04.108 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T00:03:04.106+0000 7f3827a49640 1 -- 172.21.15.178:0/4104428819 wait complete. 2024-09-11T00:03:04.782 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:03:04 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:03:04.782 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:03:04 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:03:04.782 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:03:04 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:03:04.782 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:03:04 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T00:03:04.782 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:03:04 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:03:04.783 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:03:04 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T00:03:04.783 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:03:04 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:03:04.783 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:03:04 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:03:04.812 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:03:04 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:03:04.813 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:03:04 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:03:04.813 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:03:04 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:03:04.813 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:03:04 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T00:03:04.813 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:03:04 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:03:04.813 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:03:04 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T00:03:04.813 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:03:04 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:03:04.813 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:03:04 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:03:04.813 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:03:04 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:03:04.814 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:03:04 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:03:04.814 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:03:04 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:03:04.814 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:03:04 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T00:03:04.814 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:03:04 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:03:04.814 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:03:04 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T00:03:04.814 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:03:04 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:03:04.814 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:03:04 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:03:04.842 DEBUG:teuthology.orchestra.run.smithi178:osd.11> sudo journalctl -f -n 0 -u ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968@osd.11.service 2024-09-11T00:03:04.845 INFO:tasks.cephadm:Waiting for 12 OSDs to come up... 2024-09-11T00:03:04.846 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ff3ad74a-6fcf-11ef-bcea-c7b262605968 -- ceph osd stat -f json 2024-09-11T00:03:05.229 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:03:05.631 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:03:05 smithi012 ceph-mon[30333]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 332 MiB used, 983 GiB / 983 GiB avail 2024-09-11T00:03:05.721 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:03:05 smithi178 ceph-mon[35919]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 332 MiB used, 983 GiB / 983 GiB avail 2024-09-11T00:03:05.782 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:03:05 smithi165 ceph-mon[35356]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 332 MiB used, 983 GiB / 983 GiB avail 2024-09-11T00:03:05.899 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:05.896+0000 7fd80cc00640 1 -- 172.21.15.12:0/2385773027 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd808076f80 msgr2=0x7fd8080755e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:03:05.899 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:05.896+0000 7fd80cc00640 1 --2- 172.21.15.12:0/2385773027 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd808076f80 0x7fd8080755e0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fd7f8009900 tx=0x7fd7f802a3f0 comp rx=0 tx=0).stop 2024-09-11T00:03:05.900 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:05.897+0000 7fd80cc00640 1 -- 172.21.15.12:0/2385773027 shutdown_connections 2024-09-11T00:03:05.900 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:05.897+0000 7fd80cc00640 1 --2- 172.21.15.12:0/2385773027 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd808106550 0x7fd808111210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:03:05.900 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:05.897+0000 7fd80cc00640 1 --2- 172.21.15.12:0/2385773027 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd808075b20 0x7fd808075fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:03:05.900 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:05.897+0000 7fd80cc00640 1 --2- 172.21.15.12:0/2385773027 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd808076f80 0x7fd8080755e0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:03:05.900 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:05.897+0000 7fd80cc00640 1 -- 172.21.15.12:0/2385773027 >> 172.21.15.12:0/2385773027 conn(0x7fd8080fe320 msgr2=0x7fd808100740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:03:05.900 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:05.897+0000 7fd80cc00640 1 -- 172.21.15.12:0/2385773027 shutdown_connections 2024-09-11T00:03:05.900 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:05.897+0000 7fd80cc00640 1 -- 172.21.15.12:0/2385773027 wait complete. 2024-09-11T00:03:05.900 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:05.898+0000 7fd80cc00640 1 Processor -- start 2024-09-11T00:03:05.901 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:05.898+0000 7fd80cc00640 1 -- start start 2024-09-11T00:03:05.901 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:05.898+0000 7fd80cc00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd808075b20 0x7fd80819f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:03:05.901 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:05.898+0000 7fd80cc00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd808076f80 0x7fd80819fa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:03:05.901 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:05.898+0000 7fd80cc00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd8081a0680 0x7fd8081a0b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:03:05.901 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:05.898+0000 7fd80cc00640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fd808114cc0 con 0x7fd808076f80 2024-09-11T00:03:05.902 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:05.898+0000 7fd80cc00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fd808114b40 con 0x7fd808075b20 2024-09-11T00:03:05.902 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:05.898+0000 7fd80cc00640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fd808114e40 con 0x7fd8081a0680 2024-09-11T00:03:05.902 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:05.898+0000 7fd7ffe00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd808076f80 0x7fd80819fa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:03:05.902 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:05.898+0000 7fd7ffe00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd808076f80 0x7fd80819fa40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:39684/0 (socket says 172.21.15.12:39684) 2024-09-11T00:03:05.902 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:05.898+0000 7fd7ffe00640 1 -- 172.21.15.12:0/2661471119 learned_addr learned my addr 172.21.15.12:0/2661471119 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:03:05.902 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:05.899+0000 7fd805e00640 1 --2- 172.21.15.12:0/2661471119 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd808075b20 0x7fd80819f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:03:05.902 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:05.899+0000 7fd806800640 1 --2- 172.21.15.12:0/2661471119 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd8081a0680 0x7fd8081a0b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:03:05.902 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:05.899+0000 7fd7ffe00640 1 -- 172.21.15.12:0/2661471119 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd8081a0680 msgr2=0x7fd8081a0b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:03:05.902 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:05.899+0000 7fd7ffe00640 1 --2- 172.21.15.12:0/2661471119 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd8081a0680 0x7fd8081a0b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:03:05.903 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:05.899+0000 7fd7ffe00640 1 -- 172.21.15.12:0/2661471119 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd808075b20 msgr2=0x7fd80819f3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:03:05.903 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:05.899+0000 7fd7ffe00640 1 --2- 172.21.15.12:0/2661471119 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd808075b20 0x7fd80819f3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:03:05.903 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:05.899+0000 7fd7ffe00640 1 -- 172.21.15.12:0/2661471119 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8081a1300 con 0x7fd808076f80 2024-09-11T00:03:05.903 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:05.899+0000 7fd7ffe00640 1 --2- 172.21.15.12:0/2661471119 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd808076f80 0x7fd80819fa40 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fd7f000e9f0 tx=0x7fd7f000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:03:05.904 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:05.900+0000 7fd7fea00640 1 -- 172.21.15.12:0/2661471119 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7f000cd80 con 0x7fd808076f80 2024-09-11T00:03:05.904 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:05.900+0000 7fd7fea00640 1 -- 172.21.15.12:0/2661471119 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd7f00108e0 con 0x7fd808076f80 2024-09-11T00:03:05.904 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:05.900+0000 7fd80cc00640 1 -- 172.21.15.12:0/2661471119 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd8081a7720 con 0x7fd808076f80 2024-09-11T00:03:05.904 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:05.900+0000 7fd7fea00640 1 -- 172.21.15.12:0/2661471119 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7f0018980 con 0x7fd808076f80 2024-09-11T00:03:05.904 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:05.900+0000 7fd80cc00640 1 -- 172.21.15.12:0/2661471119 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd8081a7c20 con 0x7fd808076f80 2024-09-11T00:03:05.904 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:05.901+0000 7fd80cc00640 1 -- 172.21.15.12:0/2661471119 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd808104ad0 con 0x7fd808076f80 2024-09-11T00:03:05.905 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:05.903+0000 7fd7fea00640 1 -- 172.21.15.12:0/2661471119 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 16) ==== 153250+0+0 (secure 0 0 0) 0x7fd7f0010430 con 0x7fd808076f80 2024-09-11T00:03:05.907 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:05.905+0000 7fd7fea00640 1 --2- 172.21.15.12:0/2661471119 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fd7ec0b5360 0x7fd7ec0b7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:03:05.907 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:05.906+0000 7fd7fea00640 1 -- 172.21.15.12:0/2661471119 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(65..65 src has 1..65) ==== 6809+0+0 (secure 0 0 0) 0x7fd7f00e5e00 con 0x7fd808076f80 2024-09-11T00:03:05.907 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:05.906+0000 7fd805e00640 1 --2- 172.21.15.12:0/2661471119 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fd7ec0b5360 0x7fd7ec0b7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:03:05.908 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:05.906+0000 7fd805e00640 1 --2- 172.21.15.12:0/2661471119 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fd7ec0b5360 0x7fd7ec0b7820 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fd7f8000c00 tx=0x7fd7f80314a0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:03:05.908 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:05.907+0000 7fd7fea00640 1 -- 172.21.15.12:0/2661471119 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7fd7f00acbb0 con 0x7fd808076f80 2024-09-11T00:03:06.105 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:06.103+0000 7fd80cc00640 1 -- 172.21.15.12:0/2661471119 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7fd80802e700 con 0x7fd808076f80 2024-09-11T00:03:06.105 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:06.104+0000 7fd7fea00640 1 -- 172.21.15.12:0/2661471119 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v65) ==== 74+0+133 (secure 0 0 0) 0x7fd7f00b15e0 con 0x7fd808076f80 2024-09-11T00:03:06.106 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-11T00:03:06.108 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:06.106+0000 7fd80cc00640 1 -- 172.21.15.12:0/2661471119 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fd7ec0b5360 msgr2=0x7fd7ec0b7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:03:06.108 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:06.106+0000 7fd80cc00640 1 --2- 172.21.15.12:0/2661471119 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fd7ec0b5360 0x7fd7ec0b7820 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fd7f8000c00 tx=0x7fd7f80314a0 comp rx=0 tx=0).stop 2024-09-11T00:03:06.108 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:06.106+0000 7fd80cc00640 1 -- 172.21.15.12:0/2661471119 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd808076f80 msgr2=0x7fd80819fa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:03:06.108 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:06.106+0000 7fd80cc00640 1 --2- 172.21.15.12:0/2661471119 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd808076f80 0x7fd80819fa40 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fd7f000e9f0 tx=0x7fd7f000eeb0 comp rx=0 tx=0).stop 2024-09-11T00:03:06.108 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:06.106+0000 7fd80cc00640 1 -- 172.21.15.12:0/2661471119 shutdown_connections 2024-09-11T00:03:06.108 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:06.106+0000 7fd80cc00640 1 --2- 172.21.15.12:0/2661471119 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fd7ec0b5360 0x7fd7ec0b7820 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:03:06.108 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:06.106+0000 7fd80cc00640 1 --2- 172.21.15.12:0/2661471119 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd8081a0680 0x7fd8081a0b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:03:06.108 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:06.106+0000 7fd80cc00640 1 --2- 172.21.15.12:0/2661471119 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd808076f80 0x7fd80819fa40 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:03:06.109 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:06.106+0000 7fd80cc00640 1 --2- 172.21.15.12:0/2661471119 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd808075b20 0x7fd80819f3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:03:06.109 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:06.106+0000 7fd80cc00640 1 -- 172.21.15.12:0/2661471119 >> 172.21.15.12:0/2661471119 conn(0x7fd8080fe320 msgr2=0x7fd808100710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:03:06.109 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:06.106+0000 7fd80cc00640 1 -- 172.21.15.12:0/2661471119 shutdown_connections 2024-09-11T00:03:06.109 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:06.106+0000 7fd80cc00640 1 -- 172.21.15.12:0/2661471119 wait complete. 2024-09-11T00:03:06.581 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:03:06 smithi178 ceph-mon[35919]: from='client.? 172.21.15.12:0/2661471119' entity='client.admin' cmd={"prefix": "osd stat", "format": "json"} : dispatch 2024-09-11T00:03:06.581 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:03:06 smithi178 ceph-mon[35919]: from='osd.11 [v2:172.21.15.178:6824/2937059756,v1:172.21.15.178:6825/2937059756]' entity='osd.11' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]} : dispatch 2024-09-11T00:03:06.581 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:03:06 smithi178 ceph-mon[35919]: from='osd.11 ' entity='osd.11' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]} : dispatch 2024-09-11T00:03:06.581 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 00:03:06 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-11[60852]: 2024-09-11T00:03:06.376+0000 7fd75f566640 -1 osd.11 0 log_to_monitors true 2024-09-11T00:03:06.782 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:03:06 smithi165 ceph-mon[35356]: from='client.? 172.21.15.12:0/2661471119' entity='client.admin' cmd={"prefix": "osd stat", "format": "json"} : dispatch 2024-09-11T00:03:06.782 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:03:06 smithi165 ceph-mon[35356]: from='osd.11 [v2:172.21.15.178:6824/2937059756,v1:172.21.15.178:6825/2937059756]' entity='osd.11' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]} : dispatch 2024-09-11T00:03:06.782 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:03:06 smithi165 ceph-mon[35356]: from='osd.11 ' entity='osd.11' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]} : dispatch 2024-09-11T00:03:06.812 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:03:06 smithi012 ceph-mon[30333]: from='client.? 172.21.15.12:0/2661471119' entity='client.admin' cmd={"prefix": "osd stat", "format": "json"} : dispatch 2024-09-11T00:03:06.812 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:03:06 smithi012 ceph-mon[30333]: from='osd.11 [v2:172.21.15.178:6824/2937059756,v1:172.21.15.178:6825/2937059756]' entity='osd.11' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]} : dispatch 2024-09-11T00:03:06.813 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:03:06 smithi012 ceph-mon[30333]: from='osd.11 ' entity='osd.11' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]} : dispatch 2024-09-11T00:03:06.967 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":65,"num_osds":12,"num_up_osds":11,"osd_up_since":1726012964,"num_in_osds":12,"osd_in_since":1726012966,"num_remapped_pgs":0} 2024-09-11T00:03:07.697 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:03:07 smithi178 ceph-mon[35919]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 332 MiB used, 983 GiB / 983 GiB avail 2024-09-11T00:03:07.697 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:03:07 smithi178 ceph-mon[35919]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-09-11T00:03:07.697 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:03:07 smithi178 ceph-mon[35919]: from='osd.11 [v2:172.21.15.178:6824/2937059756,v1:172.21.15.178:6825/2937059756]' entity='osd.11' cmd={"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi178", "root=default"]} : dispatch 2024-09-11T00:03:07.697 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:03:07 smithi178 ceph-mon[35919]: osdmap e66: 12 total, 11 up, 12 in 2024-09-11T00:03:07.697 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:03:07 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-11T00:03:07.697 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:03:07 smithi178 ceph-mon[35919]: from='osd.11 ' entity='osd.11' cmd={"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi178", "root=default"]} : dispatch 2024-09-11T00:03:07.782 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:03:07 smithi165 ceph-mon[35356]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 332 MiB used, 983 GiB / 983 GiB avail 2024-09-11T00:03:07.782 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:03:07 smithi165 ceph-mon[35356]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-09-11T00:03:07.783 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:03:07 smithi165 ceph-mon[35356]: from='osd.11 [v2:172.21.15.178:6824/2937059756,v1:172.21.15.178:6825/2937059756]' entity='osd.11' cmd={"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi178", "root=default"]} : dispatch 2024-09-11T00:03:07.783 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:03:07 smithi165 ceph-mon[35356]: osdmap e66: 12 total, 11 up, 12 in 2024-09-11T00:03:07.783 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:03:07 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-11T00:03:07.783 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:03:07 smithi165 ceph-mon[35356]: from='osd.11 ' entity='osd.11' cmd={"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi178", "root=default"]} : dispatch 2024-09-11T00:03:07.812 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:03:07 smithi012 ceph-mon[30333]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 332 MiB used, 983 GiB / 983 GiB avail 2024-09-11T00:03:07.812 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:03:07 smithi012 ceph-mon[30333]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-09-11T00:03:07.813 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:03:07 smithi012 ceph-mon[30333]: from='osd.11 [v2:172.21.15.178:6824/2937059756,v1:172.21.15.178:6825/2937059756]' entity='osd.11' cmd={"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi178", "root=default"]} : dispatch 2024-09-11T00:03:07.813 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:03:07 smithi012 ceph-mon[30333]: osdmap e66: 12 total, 11 up, 12 in 2024-09-11T00:03:07.813 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:03:07 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-11T00:03:07.813 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:03:07 smithi012 ceph-mon[30333]: from='osd.11 ' entity='osd.11' cmd={"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi178", "root=default"]} : dispatch 2024-09-11T00:03:07.968 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ff3ad74a-6fcf-11ef-bcea-c7b262605968 -- ceph osd stat -f json 2024-09-11T00:03:08.330 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 00:03:07 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-11[60852]: 2024-09-11T00:03:07.955+0000 7fd75b4e9640 -1 osd.11 0 waiting for initial osdmap 2024-09-11T00:03:08.331 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 00:03:07 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-11[60852]: 2024-09-11T00:03:07.966+0000 7fd753308640 -1 osd.11 67 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-11T00:03:08.358 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:03:08.617 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:03:08 smithi012 ceph-mon[30333]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi178", "root=default"]}]': finished 2024-09-11T00:03:08.618 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:03:08 smithi012 ceph-mon[30333]: osdmap e67: 12 total, 11 up, 12 in 2024-09-11T00:03:08.618 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:03:08 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-11T00:03:08.618 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:03:08 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-11T00:03:08.618 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:03:08 smithi012 ceph-mon[30333]: Detected new or changed devices on smithi178 2024-09-11T00:03:08.618 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:03:08 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:03:08.618 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:03:08 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:03:08.618 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:03:08 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-11T00:03:08.618 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:03:08 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-11T00:03:08.618 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:03:08 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"} : dispatch 2024-09-11T00:03:08.618 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:03:08 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"} : dispatch 2024-09-11T00:03:08.618 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:03:08 smithi012 ceph-mon[30333]: Adjusting osd_memory_target on smithi178 to 4272M 2024-09-11T00:03:08.618 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:03:08 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:03:08.618 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:03:08 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:03:08.619 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:03:08 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T00:03:08.619 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:03:08 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:03:08.619 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:03:08 smithi012 ceph-mon[30333]: from='osd.11 ' entity='osd.11' 2024-09-11T00:03:08.619 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:03:08 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-11T00:03:08.782 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:03:08 smithi165 ceph-mon[35356]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi178", "root=default"]}]': finished 2024-09-11T00:03:08.782 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:03:08 smithi165 ceph-mon[35356]: osdmap e67: 12 total, 11 up, 12 in 2024-09-11T00:03:08.782 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:03:08 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-11T00:03:08.782 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:03:08 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-11T00:03:08.782 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:03:08 smithi165 ceph-mon[35356]: Detected new or changed devices on smithi178 2024-09-11T00:03:08.782 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:03:08 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:03:08.783 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:03:08 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:03:08.783 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:03:08 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-11T00:03:08.783 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:03:08 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-11T00:03:08.783 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:03:08 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"} : dispatch 2024-09-11T00:03:08.783 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:03:08 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"} : dispatch 2024-09-11T00:03:08.783 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:03:08 smithi165 ceph-mon[35356]: Adjusting osd_memory_target on smithi178 to 4272M 2024-09-11T00:03:08.783 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:03:08 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:03:08.783 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:03:08 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:03:08.783 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:03:08 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T00:03:08.783 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:03:08 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:03:08.783 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:03:08 smithi165 ceph-mon[35356]: from='osd.11 ' entity='osd.11' 2024-09-11T00:03:08.783 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:03:08 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-11T00:03:08.831 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:03:08 smithi178 ceph-mon[35919]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi178", "root=default"]}]': finished 2024-09-11T00:03:08.831 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:03:08 smithi178 ceph-mon[35919]: osdmap e67: 12 total, 11 up, 12 in 2024-09-11T00:03:08.831 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:03:08 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-11T00:03:08.831 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:03:08 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-11T00:03:08.831 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:03:08 smithi178 ceph-mon[35919]: Detected new or changed devices on smithi178 2024-09-11T00:03:08.831 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:03:08 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:03:08.831 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:03:08 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:03:08.831 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:03:08 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-11T00:03:08.832 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:03:08 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-11T00:03:08.832 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:03:08 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"} : dispatch 2024-09-11T00:03:08.832 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:03:08 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"} : dispatch 2024-09-11T00:03:08.832 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:03:08 smithi178 ceph-mon[35919]: Adjusting osd_memory_target on smithi178 to 4272M 2024-09-11T00:03:08.832 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:03:08 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:03:08.832 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:03:08 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:03:08.832 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:03:08 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T00:03:08.832 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:03:08 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:03:08.832 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:03:08 smithi178 ceph-mon[35919]: from='osd.11 ' entity='osd.11' 2024-09-11T00:03:08.832 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:03:08 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-11T00:03:08.870 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:08.867+0000 7f34a7c00640 1 -- 172.21.15.12:0/449910020 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f34a0078b70 msgr2=0x7f34a0078ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:03:08.870 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:08.867+0000 7f34a7c00640 1 --2- 172.21.15.12:0/449910020 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f34a0078b70 0x7f34a0078ff0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f348c009860 tx=0x7f348c02a340 comp rx=0 tx=0).stop 2024-09-11T00:03:08.870 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:08.868+0000 7f34a7c00640 1 -- 172.21.15.12:0/449910020 shutdown_connections 2024-09-11T00:03:08.870 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:08.868+0000 7f34a7c00640 1 --2- 172.21.15.12:0/449910020 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f34a0079530 0x7f34a0079e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:03:08.870 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:08.868+0000 7f34a7c00640 1 --2- 172.21.15.12:0/449910020 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f34a0078b70 0x7f34a0078ff0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:03:08.870 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:08.868+0000 7f34a7c00640 1 --2- 172.21.15.12:0/449910020 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f34a0077a10 0x7f34a0077e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:03:08.871 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:08.868+0000 7f34a7c00640 1 -- 172.21.15.12:0/449910020 >> 172.21.15.12:0/449910020 conn(0x7f34a0100510 msgr2=0x7f34a0102970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:03:08.871 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:08.868+0000 7f34a7c00640 1 -- 172.21.15.12:0/449910020 shutdown_connections 2024-09-11T00:03:08.871 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:08.868+0000 7f34a7c00640 1 -- 172.21.15.12:0/449910020 wait complete. 2024-09-11T00:03:08.871 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:08.869+0000 7f34a7c00640 1 Processor -- start 2024-09-11T00:03:08.871 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:08.869+0000 7f34a7c00640 1 -- start start 2024-09-11T00:03:08.872 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:08.870+0000 7f34a7c00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f34a0077a10 0x7f34a019f400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:03:08.872 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:08.870+0000 7f34a7c00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f34a0078b70 0x7f34a019fa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:03:08.872 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:08.870+0000 7f34a7c00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f34a0079530 0x7f34a01a0690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:03:08.873 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:08.870+0000 7f34a7c00640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f34a0114cf0 con 0x7f34a0079530 2024-09-11T00:03:08.873 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:08.870+0000 7f34a7c00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f34a0114b70 con 0x7f34a0077a10 2024-09-11T00:03:08.873 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:08.870+0000 7f34a7c00640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f34a0114e70 con 0x7f34a0078b70 2024-09-11T00:03:08.873 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:08.870+0000 7f34a5200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f34a0077a10 0x7f34a019f400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:03:08.873 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:08.870+0000 7f34a5200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f34a0077a10 0x7f34a019f400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.12:46358/0 (socket says 172.21.15.12:46358) 2024-09-11T00:03:08.873 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:08.870+0000 7f34a5200640 1 -- 172.21.15.12:0/1858150002 learned_addr learned my addr 172.21.15.12:0/1858150002 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:03:08.873 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:08.870+0000 7f349fe00640 1 --2- 172.21.15.12:0/1858150002 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f34a0078b70 0x7f34a019fa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:03:08.873 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:08.870+0000 7f34a5c00640 1 --2- 172.21.15.12:0/1858150002 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f34a0079530 0x7f34a01a0690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:03:08.873 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:08.870+0000 7f34a5200640 1 -- 172.21.15.12:0/1858150002 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f34a0078b70 msgr2=0x7f34a019fa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:03:08.873 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:08.870+0000 7f34a5200640 1 --2- 172.21.15.12:0/1858150002 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f34a0078b70 0x7f34a019fa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:03:08.873 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:08.870+0000 7f34a5200640 1 -- 172.21.15.12:0/1858150002 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f34a0079530 msgr2=0x7f34a01a0690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:03:08.873 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:08.870+0000 7f34a5200640 1 --2- 172.21.15.12:0/1858150002 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f34a0079530 0x7f34a01a0690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:03:08.874 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:08.870+0000 7f34a5200640 1 -- 172.21.15.12:0/1858150002 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34a01a0f30 con 0x7f34a0077a10 2024-09-11T00:03:08.874 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:08.870+0000 7f34a5c00640 1 --2- 172.21.15.12:0/1858150002 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f34a0079530 0x7f34a01a0690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T00:03:08.874 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:08.871+0000 7f34a5200640 1 --2- 172.21.15.12:0/1858150002 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f34a0077a10 0x7f34a019f400 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f349400e9c0 tx=0x7f349400ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:03:08.874 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:08.871+0000 7f349d600640 1 -- 172.21.15.12:0/1858150002 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f349400ccc0 con 0x7f34a0077a10 2024-09-11T00:03:08.874 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:08.871+0000 7f349d600640 1 -- 172.21.15.12:0/1858150002 <== mon.2 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f34940054d0 con 0x7f34a0077a10 2024-09-11T00:03:08.874 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:08.871+0000 7f349d600640 1 -- 172.21.15.12:0/1858150002 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3494010430 con 0x7f34a0077a10 2024-09-11T00:03:08.875 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:08.871+0000 7f34a7c00640 1 -- 172.21.15.12:0/1858150002 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f34a01a1220 con 0x7f34a0077a10 2024-09-11T00:03:08.875 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:08.871+0000 7f34a7c00640 1 -- 172.21.15.12:0/1858150002 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34a01a7900 con 0x7f34a0077a10 2024-09-11T00:03:08.875 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:08.872+0000 7f34a7c00640 1 -- 172.21.15.12:0/1858150002 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f34a0078ff0 con 0x7f34a0077a10 2024-09-11T00:03:08.876 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:08.874+0000 7f349d600640 1 -- 172.21.15.12:0/1858150002 <== mon.2 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 16) ==== 153250+0+0 (secure 0 0 0) 0x7f3494004f80 con 0x7f34a0077a10 2024-09-11T00:03:08.877 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:08.876+0000 7f349d600640 1 --2- 172.21.15.12:0/1858150002 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f34880b5360 0x7f34880b7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:03:08.878 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:08.876+0000 7f349d600640 1 -- 172.21.15.12:0/1858150002 <== mon.2 v2:172.21.15.165:3300/0 5 ==== osd_map(67..67 src has 1..67) ==== 6847+0+0 (secure 0 0 0) 0x7f34940e5af0 con 0x7f34a0077a10 2024-09-11T00:03:08.878 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:08.876+0000 7f349fe00640 1 --2- 172.21.15.12:0/1858150002 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f34880b5360 0x7f34880b7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:03:08.878 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:08.877+0000 7f349fe00640 1 --2- 172.21.15.12:0/1858150002 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f34880b5360 0x7f34880b7820 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f348c000c00 tx=0x7f348c030470 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:03:08.879 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:08.877+0000 7f349d600640 1 -- 172.21.15.12:0/1858150002 <== mon.2 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f34940aced0 con 0x7f34a0077a10 2024-09-11T00:03:09.082 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:09.080+0000 7f34a7c00640 1 -- 172.21.15.12:0/1858150002 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7f34a002e640 con 0x7f34a0077a10 2024-09-11T00:03:09.083 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:09.081+0000 7f349d600640 1 -- 172.21.15.12:0/1858150002 <== mon.2 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v68) ==== 74+0+133 (secure 0 0 0) 0x7f34940cf020 con 0x7f34a0077a10 2024-09-11T00:03:09.083 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-11T00:03:09.085 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:09.083+0000 7f34a7c00640 1 -- 172.21.15.12:0/1858150002 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f34880b5360 msgr2=0x7f34880b7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:03:09.085 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:09.083+0000 7f34a7c00640 1 --2- 172.21.15.12:0/1858150002 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f34880b5360 0x7f34880b7820 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f348c000c00 tx=0x7f348c030470 comp rx=0 tx=0).stop 2024-09-11T00:03:09.085 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:09.083+0000 7f34a7c00640 1 -- 172.21.15.12:0/1858150002 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f34a0077a10 msgr2=0x7f34a019f400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:03:09.085 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:09.083+0000 7f34a7c00640 1 --2- 172.21.15.12:0/1858150002 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f34a0077a10 0x7f34a019f400 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f349400e9c0 tx=0x7f349400ee80 comp rx=0 tx=0).stop 2024-09-11T00:03:09.085 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:09.083+0000 7f34a7c00640 1 -- 172.21.15.12:0/1858150002 shutdown_connections 2024-09-11T00:03:09.086 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:09.083+0000 7f34a7c00640 1 --2- 172.21.15.12:0/1858150002 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f34880b5360 0x7f34880b7820 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:03:09.086 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:09.083+0000 7f34a7c00640 1 --2- 172.21.15.12:0/1858150002 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f34a0079530 0x7f34a01a0690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:03:09.086 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:09.083+0000 7f34a7c00640 1 --2- 172.21.15.12:0/1858150002 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f34a0078b70 0x7f34a019fa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:03:09.086 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:09.083+0000 7f34a7c00640 1 --2- 172.21.15.12:0/1858150002 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f34a0077a10 0x7f34a019f400 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:03:09.086 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:09.083+0000 7f34a7c00640 1 -- 172.21.15.12:0/1858150002 >> 172.21.15.12:0/1858150002 conn(0x7f34a0100510 msgr2=0x7f34a01020d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:03:09.086 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:09.083+0000 7f34a7c00640 1 -- 172.21.15.12:0/1858150002 shutdown_connections 2024-09-11T00:03:09.086 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:09.083+0000 7f34a7c00640 1 -- 172.21.15.12:0/1858150002 wait complete. 2024-09-11T00:03:09.782 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:03:09 smithi165 ceph-mon[35356]: purged_snaps scrub starts 2024-09-11T00:03:09.782 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:03:09 smithi165 ceph-mon[35356]: purged_snaps scrub ok 2024-09-11T00:03:09.782 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:03:09 smithi165 ceph-mon[35356]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 332 MiB used, 983 GiB / 983 GiB avail 2024-09-11T00:03:09.782 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:03:09 smithi165 ceph-mon[35356]: osd.11 [v2:172.21.15.178:6824/2937059756,v1:172.21.15.178:6825/2937059756] boot 2024-09-11T00:03:09.783 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:03:09 smithi165 ceph-mon[35356]: osdmap e68: 12 total, 12 up, 12 in 2024-09-11T00:03:09.783 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:03:09 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-11T00:03:09.783 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:03:09 smithi165 ceph-mon[35356]: from='client.? 172.21.15.12:0/1858150002' entity='client.admin' cmd={"prefix": "osd stat", "format": "json"} : dispatch 2024-09-11T00:03:09.794 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:03:09 smithi012 ceph-mon[30333]: purged_snaps scrub starts 2024-09-11T00:03:09.794 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:03:09 smithi012 ceph-mon[30333]: purged_snaps scrub ok 2024-09-11T00:03:09.794 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:03:09 smithi012 ceph-mon[30333]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 332 MiB used, 983 GiB / 983 GiB avail 2024-09-11T00:03:09.794 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:03:09 smithi012 ceph-mon[30333]: osd.11 [v2:172.21.15.178:6824/2937059756,v1:172.21.15.178:6825/2937059756] boot 2024-09-11T00:03:09.794 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:03:09 smithi012 ceph-mon[30333]: osdmap e68: 12 total, 12 up, 12 in 2024-09-11T00:03:09.794 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:03:09 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-11T00:03:09.794 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:03:09 smithi012 ceph-mon[30333]: from='client.? 172.21.15.12:0/1858150002' entity='client.admin' cmd={"prefix": "osd stat", "format": "json"} : dispatch 2024-09-11T00:03:09.830 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":68,"num_osds":12,"num_up_osds":12,"osd_up_since":1726012988,"num_in_osds":12,"osd_in_since":1726012966,"num_remapped_pgs":0} 2024-09-11T00:03:09.831 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid ff3ad74a-6fcf-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-11T00:03:09.831 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:03:09 smithi178 ceph-mon[35919]: purged_snaps scrub starts 2024-09-11T00:03:09.832 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:03:09 smithi178 ceph-mon[35919]: purged_snaps scrub ok 2024-09-11T00:03:09.832 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:03:09 smithi178 ceph-mon[35919]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 332 MiB used, 983 GiB / 983 GiB avail 2024-09-11T00:03:09.832 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:03:09 smithi178 ceph-mon[35919]: osd.11 [v2:172.21.15.178:6824/2937059756,v1:172.21.15.178:6825/2937059756] boot 2024-09-11T00:03:09.832 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:03:09 smithi178 ceph-mon[35919]: osdmap e68: 12 total, 12 up, 12 in 2024-09-11T00:03:09.832 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:03:09 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-11T00:03:09.832 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:03:09 smithi178 ceph-mon[35919]: from='client.? 172.21.15.12:0/1858150002' entity='client.admin' cmd={"prefix": "osd stat", "format": "json"} : dispatch 2024-09-11T00:03:10.207 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:03:10.738 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:10.732+0000 7f36f3a00640 1 -- 172.21.15.12:0/3206729333 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f36ec102da0 msgr2=0x7f36ec103180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:03:10.738 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:10.732+0000 7f36f3a00640 1 --2- 172.21.15.12:0/3206729333 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f36ec102da0 0x7f36ec103180 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f36e0009900 tx=0x7f36e002a410 comp rx=0 tx=0).stop 2024-09-11T00:03:10.738 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:10.733+0000 7f36f3a00640 1 -- 172.21.15.12:0/3206729333 shutdown_connections 2024-09-11T00:03:10.738 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:10.733+0000 7f36f3a00640 1 --2- 172.21.15.12:0/3206729333 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f36ec109130 0x7f36ec10f930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:03:10.739 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:10.733+0000 7f36f3a00640 1 --2- 172.21.15.12:0/3206729333 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f36ec1087b0 0x7f36ec108bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:03:10.739 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:10.733+0000 7f36f3a00640 1 --2- 172.21.15.12:0/3206729333 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f36ec102da0 0x7f36ec103180 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:03:10.739 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:10.733+0000 7f36f3a00640 1 -- 172.21.15.12:0/3206729333 >> 172.21.15.12:0/3206729333 conn(0x7f36ec0fe810 msgr2=0x7f36ec100c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:03:10.739 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:10.733+0000 7f36f3a00640 1 -- 172.21.15.12:0/3206729333 shutdown_connections 2024-09-11T00:03:10.739 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:10.733+0000 7f36f3a00640 1 -- 172.21.15.12:0/3206729333 wait complete. 2024-09-11T00:03:10.739 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:10.734+0000 7f36f3a00640 1 Processor -- start 2024-09-11T00:03:10.739 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:10.734+0000 7f36f3a00640 1 -- start start 2024-09-11T00:03:10.739 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:10.734+0000 7f36f3a00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f36ec102da0 0x7f36ec1992e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:03:10.739 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:10.734+0000 7f36f3a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f36ec1087b0 0x7f36ec199820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:03:10.739 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:10.734+0000 7f36f3a00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f36ec109130 0x7f36ec19ebc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:03:10.739 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:10.734+0000 7f36f3a00640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f36ec112df0 con 0x7f36ec109130 2024-09-11T00:03:10.739 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:10.734+0000 7f36f3a00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f36ec112c70 con 0x7f36ec1087b0 2024-09-11T00:03:10.740 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:10.734+0000 7f36f3a00640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f36ec112f70 con 0x7f36ec102da0 2024-09-11T00:03:10.740 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:10.734+0000 7f36f1000640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f36ec102da0 0x7f36ec1992e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:03:10.740 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:10.734+0000 7f36f1000640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f36ec102da0 0x7f36ec1992e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.12:58592/0 (socket says 172.21.15.12:58592) 2024-09-11T00:03:10.740 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:10.734+0000 7f36f1000640 1 -- 172.21.15.12:0/3788255306 learned_addr learned my addr 172.21.15.12:0/3788255306 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:03:10.740 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:10.734+0000 7f36f1a00640 1 --2- 172.21.15.12:0/3788255306 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f36ec109130 0x7f36ec19ebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:03:10.740 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:10.735+0000 7f36ebe00640 1 --2- 172.21.15.12:0/3788255306 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f36ec1087b0 0x7f36ec199820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:03:10.740 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:10.735+0000 7f36f1000640 1 -- 172.21.15.12:0/3788255306 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f36ec1087b0 msgr2=0x7f36ec199820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:03:10.740 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:10.735+0000 7f36f1000640 1 --2- 172.21.15.12:0/3788255306 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f36ec1087b0 0x7f36ec199820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:03:10.740 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:10.735+0000 7f36f1000640 1 -- 172.21.15.12:0/3788255306 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f36ec109130 msgr2=0x7f36ec19ebc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:03:10.740 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:10.735+0000 7f36f1000640 1 --2- 172.21.15.12:0/3788255306 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f36ec109130 0x7f36ec19ebc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:03:10.740 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:10.735+0000 7f36f1000640 1 -- 172.21.15.12:0/3788255306 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36ec19f390 con 0x7f36ec102da0 2024-09-11T00:03:10.740 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:10.735+0000 7f36f1000640 1 --2- 172.21.15.12:0/3788255306 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f36ec102da0 0x7f36ec1992e0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f36e0005940 tx=0x7f36e00040c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:03:10.740 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:10.735+0000 7f36e9600640 1 -- 172.21.15.12:0/3788255306 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36e003f030 con 0x7f36ec102da0 2024-09-11T00:03:10.741 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:10.736+0000 7f36e9600640 1 -- 172.21.15.12:0/3788255306 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f36e00044c0 con 0x7f36ec102da0 2024-09-11T00:03:10.741 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:10.736+0000 7f36e9600640 1 -- 172.21.15.12:0/3788255306 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36e0035070 con 0x7f36ec102da0 2024-09-11T00:03:10.741 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:10.736+0000 7f36f3a00640 1 -- 172.21.15.12:0/3788255306 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f36ec19f620 con 0x7f36ec102da0 2024-09-11T00:03:10.741 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:10.736+0000 7f36f3a00640 1 -- 172.21.15.12:0/3788255306 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f36ec1a5d40 con 0x7f36ec102da0 2024-09-11T00:03:10.741 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:10.737+0000 7f36f3a00640 1 -- 172.21.15.12:0/3788255306 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f36ec103180 con 0x7f36ec102da0 2024-09-11T00:03:10.742 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:10.740+0000 7f36e9600640 1 -- 172.21.15.12:0/3788255306 <== mon.1 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 16) ==== 153250+0+0 (secure 0 0 0) 0x7f36e003ba20 con 0x7f36ec102da0 2024-09-11T00:03:10.743 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:10.741+0000 7f36e9600640 1 --2- 172.21.15.12:0/3788255306 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f36d40b5360 0x7f36d40b7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:03:10.744 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:10.742+0000 7f36e9600640 1 -- 172.21.15.12:0/3788255306 <== mon.1 v2:172.21.15.178:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 7127+0+0 (secure 0 0 0) 0x7f36e0105fa0 con 0x7f36ec102da0 2024-09-11T00:03:10.744 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:10.742+0000 7f36ebe00640 1 --2- 172.21.15.12:0/3788255306 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f36d40b5360 0x7f36d40b7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:03:10.744 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:10.742+0000 7f36ebe00640 1 --2- 172.21.15.12:0/3788255306 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f36d40b5360 0x7f36d40b7820 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f36ec0ffba0 tx=0x7f36d8005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:03:10.744 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:10.743+0000 7f36e9600640 1 -- 172.21.15.12:0/3788255306 <== mon.1 v2:172.21.15.178:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f36e00ccc10 con 0x7f36ec102da0 2024-09-11T00:03:10.940 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:10.938+0000 7f36f3a00640 1 -- 172.21.15.12:0/3788255306 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f36ec02e5c0 con 0x7f36ec102da0 2024-09-11T00:03:10.941 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:10.939+0000 7f36e9600640 1 -- 172.21.15.12:0/3788255306 <== mon.1 v2:172.21.15.178:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v69) ==== 74+0+18492 (secure 0 0 0) 0x7f36e00d1640 con 0x7f36ec102da0 2024-09-11T00:03:10.941 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-11T00:03:10.941 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":69,"fsid":"ff3ad74a-6fcf-11ef-bcea-c7b262605968","created":"2024-09-10T23:55:16.317643+0000","modified":"2024-09-11T00:03:09.964037+0000","last_up_change":"2024-09-11T00:03:08.959256+0000","last_in_change":"2024-09-11T00:02:46.564631+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T23:59:10.488728+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6382c200-a912-42a1-958b-79191b21249b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6802","nonce":1237101991},{"type":"v1","addr":"172.21.15.12:6803","nonce":1237101991}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6804","nonce":1237101991},{"type":"v1","addr":"172.21.15.12:6805","nonce":1237101991}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6808","nonce":1237101991},{"type":"v1","addr":"172.21.15.12:6809","nonce":1237101991}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6806","nonce":1237101991},{"type":"v1","addr":"172.21.15.12:6807","nonce":1237101991}]},"public_addr":"172.21.15.12:6803/1237101991","cluster_addr":"172.21.15.12:6805/1237101991","heartbeat_back_addr":"172.21.15.12:6809/1237101991","heartbeat_front_addr":"172.21.15.12:6807/1237101991","state":["exists","up"]},{"osd":1,"uuid":"fe0e1d0a-bdc5-4859-aa84-9038059a45f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6800","nonce":545848456},{"type":"v1","addr":"172.21.15.165:6801","nonce":545848456}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6802","nonce":545848456},{"type":"v1","addr":"172.21.15.165:6803","nonce":545848456}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6806","nonce":545848456},{"type":"v1","addr":"172.21.15.165:6807","nonce":545848456}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6804","nonce":545848456},{"type":"v1","addr":"172.21.15.165:6805","nonce":545848456}]},"public_addr":"172.21.15.165:6801/545848456","cluster_addr":"172.21.15.165:6803/545848456","heartbeat_back_addr":"172.21.15.165:6807/545848456","heartbeat_front_addr":"172.21.15.165:6805/545848456","state":["exists","up"]},{"osd":2,"uuid":"94f640dc-5f78-4e2c-87d9-03e96a6e7ba0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6800","nonce":4131162695},{"type":"v1","addr":"172.21.15.178:6801","nonce":4131162695}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":4131162695},{"type":"v1","addr":"172.21.15.178:6803","nonce":4131162695}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":4131162695},{"type":"v1","addr":"172.21.15.178:6807","nonce":4131162695}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":4131162695},{"type":"v1","addr":"172.21.15.178:6805","nonce":4131162695}]},"public_addr":"172.21.15.178:6801/4131162695","cluster_addr":"172.21.15.178:6803/4131162695","heartbeat_back_addr":"172.21.15.178:6807/4131162695","heartbeat_front_addr":"172.21.15.178:6805/4131162695","state":["exists","up"]},{"osd":3,"uuid":"af2f64d6-c9be-44a9-b041-8d599a7c080d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6810","nonce":452464245},{"type":"v1","addr":"172.21.15.12:6811","nonce":452464245}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6812","nonce":452464245},{"type":"v1","addr":"172.21.15.12:6813","nonce":452464245}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6816","nonce":452464245},{"type":"v1","addr":"172.21.15.12:6817","nonce":452464245}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6814","nonce":452464245},{"type":"v1","addr":"172.21.15.12:6815","nonce":452464245}]},"public_addr":"172.21.15.12:6811/452464245","cluster_addr":"172.21.15.12:6813/452464245","heartbeat_back_addr":"172.21.15.12:6817/452464245","heartbeat_front_addr":"172.21.15.12:6815/452464245","state":["exists","up"]},{"osd":4,"uuid":"ad76ac70-20c0-44ae-9866-5f9cfce2334c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6808","nonce":1899879280},{"type":"v1","addr":"172.21.15.165:6809","nonce":1899879280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6810","nonce":1899879280},{"type":"v1","addr":"172.21.15.165:6811","nonce":1899879280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6814","nonce":1899879280},{"type":"v1","addr":"172.21.15.165:6815","nonce":1899879280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6812","nonce":1899879280},{"type":"v1","addr":"172.21.15.165:6813","nonce":1899879280}]},"public_addr":"172.21.15.165:6809/1899879280","cluster_addr":"172.21.15.165:6811/1899879280","heartbeat_back_addr":"172.21.15.165:6815/1899879280","heartbeat_front_addr":"172.21.15.165:6813/1899879280","state":["exists","up"]},{"osd":5,"uuid":"aafda2cc-62fe-4fcb-9a96-8732e0b501d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":3042228555},{"type":"v1","addr":"172.21.15.178:6809","nonce":3042228555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":3042228555},{"type":"v1","addr":"172.21.15.178:6811","nonce":3042228555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6814","nonce":3042228555},{"type":"v1","addr":"172.21.15.178:6815","nonce":3042228555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":3042228555},{"type":"v1","addr":"172.21.15.178:6813","nonce":3042228555}]},"public_addr":"172.21.15.178:6809/3042228555","cluster_addr":"172.21.15.178:6811/3042228555","heartbeat_back_addr":"172.21.15.178:6815/3042228555","heartbeat_front_addr":"172.21.15.178:6813/3042228555","state":["exists","up"]},{"osd":6,"uuid":"375eca52-efe3-42a5-bbb1-d7d392d71f15","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6818","nonce":3868322467},{"type":"v1","addr":"172.21.15.12:6819","nonce":3868322467}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6820","nonce":3868322467},{"type":"v1","addr":"172.21.15.12:6821","nonce":3868322467}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6824","nonce":3868322467},{"type":"v1","addr":"172.21.15.12:6825","nonce":3868322467}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6822","nonce":3868322467},{"type":"v1","addr":"172.21.15.12:6823","nonce":3868322467}]},"public_addr":"172.21.15.12:6819/3868322467","cluster_addr":"172.21.15.12:6821/3868322467","heartbeat_back_addr":"172.21.15.12:6825/3868322467","heartbeat_front_addr":"172.21.15.12:6823/3868322467","state":["exists","up"]},{"osd":7,"uuid":"0fd619f5-44c3-4cf7-ae7a-388dd798de36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6816","nonce":1143773735},{"type":"v1","addr":"172.21.15.165:6817","nonce":1143773735}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6818","nonce":1143773735},{"type":"v1","addr":"172.21.15.165:6819","nonce":1143773735}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6822","nonce":1143773735},{"type":"v1","addr":"172.21.15.165:6823","nonce":1143773735}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6820","nonce":1143773735},{"type":"v1","addr":"172.21.15.165:6821","nonce":1143773735}]},"public_addr":"172.21.15.165:6817/1143773735","cluster_addr":"172.21.15.165:6819/1143773735","heartbeat_back_addr":"172.21.15.165:6823/1143773735","heartbeat_front_addr":"172.21.15.165:6821/1143773735","state":["exists","up"]},{"osd":8,"uuid":"b03d14ce-8427-462a-9bcc-27683c549f53","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":2891445058},{"type":"v1","addr":"172.21.15.178:6817","nonce":2891445058}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6818","nonce":2891445058},{"type":"v1","addr":"172.21.15.178:6819","nonce":2891445058}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6822","nonce":2891445058},{"type":"v1","addr":"172.21.15.178:6823","nonce":2891445058}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":2891445058},{"type":"v1","addr":"172.21.15.178:6821","nonce":2891445058}]},"public_addr":"172.21.15.178:6817/2891445058","cluster_addr":"172.21.15.178:6819/2891445058","heartbeat_back_addr":"172.21.15.178:6823/2891445058","heartbeat_front_addr":"172.21.15.178:6821/2891445058","state":["exists","up"]},{"osd":9,"uuid":"81a6cf69-2a24-4a6c-9adf-6b7df38cfacb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6826","nonce":3359634903},{"type":"v1","addr":"172.21.15.12:6827","nonce":3359634903}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6828","nonce":3359634903},{"type":"v1","addr":"172.21.15.12:6829","nonce":3359634903}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6832","nonce":3359634903},{"type":"v1","addr":"172.21.15.12:6833","nonce":3359634903}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6830","nonce":3359634903},{"type":"v1","addr":"172.21.15.12:6831","nonce":3359634903}]},"public_addr":"172.21.15.12:6827/3359634903","cluster_addr":"172.21.15.12:6829/3359634903","heartbeat_back_addr":"172.21.15.12:6833/3359634903","heartbeat_front_addr":"172.21.15.12:6831/3359634903","state":["exists","up"]},{"osd":10,"uuid":"c1ad1f0a-33f4-4c32-91f6-4d030a29f1dd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6824","nonce":3642325778},{"type":"v1","addr":"172.21.15.165:6825","nonce":3642325778}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6826","nonce":3642325778},{"type":"v1","addr":"172.21.15.165:6827","nonce":3642325778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6830","nonce":3642325778},{"type":"v1","addr":"172.21.15.165:6831","nonce":3642325778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6828","nonce":3642325778},{"type":"v1","addr":"172.21.15.165:6829","nonce":3642325778}]},"public_addr":"172.21.15.165:6825/3642325778","cluster_addr":"172.21.15.165:6827/3642325778","heartbeat_back_addr":"172.21.15.165:6831/3642325778","heartbeat_front_addr":"172.21.15.165:6829/3642325778","state":["exists","up"]},{"osd":11,"uuid":"8413c9b0-680b-4d02-aeab-b3b3ff16c3d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":2937059756},{"type":"v1","addr":"172.21.15.178:6825","nonce":2937059756}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6826","nonce":2937059756},{"type":"v1","addr":"172.21.15.178:6827","nonce":2937059756}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":2937059756},{"type":"v1","addr":"172.21.15.178:6831","nonce":2937059756}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":2937059756},{"type":"v1","addr":"172.21.15.178:6829","nonce":2937059756}]},"public_addr":"172.21.15.178:6825/2937059756","cluster_addr":"172.21.15.178:6827/2937059756","heartbeat_back_addr":"172.21.15.178:6831/2937059756","heartbeat_front_addr":"172.21.15.178:6829/2937059756","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:58:16.036348+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:58:41.483150+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:59:07.107283+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:59:35.706574+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T00:00:02.555720+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T00:00:28.092847+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T00:00:57.396495+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T00:01:24.216775+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T00:01:48.807850+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T00:02:16.838565+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T00:02:43.131736+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T00:03:07.353155+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.12:6801/4177251511":"2024-09-11T23:56:17.999820+0000","172.21.15.12:6800/4177251511":"2024-09-11T23:56:17.999820+0000","172.21.15.12:0/3273692817":"2024-09-11T23:56:17.999820+0000","172.21.15.12:0/4017778158":"2024-09-11T23:56:17.999820+0000","172.21.15.12:0/2499921051":"2024-09-11T23:55:50.302721+0000","172.21.15.12:6801/760140910":"2024-09-11T23:55:50.302721+0000","172.21.15.12:6800/760140910":"2024-09-11T23:55:50.302721+0000","172.21.15.12:0/2810703302":"2024-09-11T23:56:17.999820+0000","172.21.15.12:0/2325436065":"2024-09-11T23:55:50.302721+0000","172.21.15.12:0/3137053479":"2024-09-11T23:55:50.302721+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-11T00:03:10.943 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:10.941+0000 7f36f3a00640 1 -- 172.21.15.12:0/3788255306 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f36d40b5360 msgr2=0x7f36d40b7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:03:10.944 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:10.941+0000 7f36f3a00640 1 --2- 172.21.15.12:0/3788255306 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f36d40b5360 0x7f36d40b7820 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f36ec0ffba0 tx=0x7f36d8005f90 comp rx=0 tx=0).stop 2024-09-11T00:03:10.944 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:10.941+0000 7f36f3a00640 1 -- 172.21.15.12:0/3788255306 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f36ec102da0 msgr2=0x7f36ec1992e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:03:10.944 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:10.941+0000 7f36f3a00640 1 --2- 172.21.15.12:0/3788255306 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f36ec102da0 0x7f36ec1992e0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f36e0005940 tx=0x7f36e00040c0 comp rx=0 tx=0).stop 2024-09-11T00:03:10.944 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:10.942+0000 7f36f3a00640 1 -- 172.21.15.12:0/3788255306 shutdown_connections 2024-09-11T00:03:10.944 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:10.942+0000 7f36f3a00640 1 --2- 172.21.15.12:0/3788255306 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f36d40b5360 0x7f36d40b7820 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:03:10.944 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:10.942+0000 7f36f3a00640 1 --2- 172.21.15.12:0/3788255306 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f36ec109130 0x7f36ec19ebc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:03:10.944 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:10.942+0000 7f36f3a00640 1 --2- 172.21.15.12:0/3788255306 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f36ec1087b0 0x7f36ec199820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:03:10.944 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:10.942+0000 7f36f3a00640 1 --2- 172.21.15.12:0/3788255306 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f36ec102da0 0x7f36ec1992e0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:03:10.944 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:10.942+0000 7f36f3a00640 1 -- 172.21.15.12:0/3788255306 >> 172.21.15.12:0/3788255306 conn(0x7f36ec0fe810 msgr2=0x7f36ec106a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:03:10.944 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:10.942+0000 7f36f3a00640 1 -- 172.21.15.12:0/3788255306 shutdown_connections 2024-09-11T00:03:10.944 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:10.942+0000 7f36f3a00640 1 -- 172.21.15.12:0/3788255306 wait complete. 2024-09-11T00:03:11.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:03:10 smithi165 ceph-mon[35356]: osdmap e69: 12 total, 12 up, 12 in 2024-09-11T00:03:11.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:03:10 smithi165 ceph-mon[35356]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T00:03:11.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:03:10 smithi165 ceph-mon[35356]: from='client.? 172.21.15.12:0/3788255306' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-11T00:03:11.303 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:03:10 smithi012 ceph-mon[30333]: osdmap e69: 12 total, 12 up, 12 in 2024-09-11T00:03:11.303 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:03:10 smithi012 ceph-mon[30333]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T00:03:11.303 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:03:10 smithi012 ceph-mon[30333]: from='client.? 172.21.15.12:0/3788255306' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-11T00:03:11.330 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:03:10 smithi178 ceph-mon[35919]: osdmap e69: 12 total, 12 up, 12 in 2024-09-11T00:03:11.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:03:10 smithi178 ceph-mon[35919]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T00:03:11.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:03:10 smithi178 ceph-mon[35919]: from='client.? 172.21.15.12:0/3788255306' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-11T00:03:11.798 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-10T23:59:10.488728+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-11T00:03:11.799 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid ff3ad74a-6fcf-11ef-bcea-c7b262605968 -- ceph osd pool get .mgr pg_num 2024-09-11T00:03:12.173 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:03:12.695 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:12.692+0000 7fb552a00640 1 -- 172.21.15.12:0/3775657571 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb54c1087d0 msgr2=0x7fb54c108c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:03:12.695 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:12.692+0000 7fb552a00640 1 --2- 172.21.15.12:0/3775657571 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb54c1087d0 0x7fb54c108c10 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fb534009900 tx=0x7fb53402a410 comp rx=0 tx=0).stop 2024-09-11T00:03:12.695 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:12.692+0000 7fb552a00640 1 -- 172.21.15.12:0/3775657571 shutdown_connections 2024-09-11T00:03:12.695 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:12.692+0000 7fb552a00640 1 --2- 172.21.15.12:0/3775657571 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb54c109150 0x7fb54c10f950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:03:12.695 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:12.692+0000 7fb552a00640 1 --2- 172.21.15.12:0/3775657571 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb54c1087d0 0x7fb54c108c10 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:03:12.695 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:12.692+0000 7fb552a00640 1 --2- 172.21.15.12:0/3775657571 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb54c102dc0 0x7fb54c1031a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:03:12.695 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:12.692+0000 7fb552a00640 1 -- 172.21.15.12:0/3775657571 >> 172.21.15.12:0/3775657571 conn(0x7fb54c0fe830 msgr2=0x7fb54c100c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:03:12.696 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:12.692+0000 7fb552a00640 1 -- 172.21.15.12:0/3775657571 shutdown_connections 2024-09-11T00:03:12.696 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:12.692+0000 7fb552a00640 1 -- 172.21.15.12:0/3775657571 wait complete. 2024-09-11T00:03:12.696 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:12.693+0000 7fb552a00640 1 Processor -- start 2024-09-11T00:03:12.697 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:12.694+0000 7fb552a00640 1 -- start start 2024-09-11T00:03:12.697 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:12.694+0000 7fb552a00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb54c102dc0 0x7fb54c19b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:03:12.697 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:12.694+0000 7fb552a00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb54c1087d0 0x7fb54c19ba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:03:12.697 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:12.694+0000 7fb552a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb54c109150 0x7fb54c1a0dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:03:12.697 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:12.694+0000 7fb552a00640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fb54c112e20 con 0x7fb54c102dc0 2024-09-11T00:03:12.697 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:12.694+0000 7fb552a00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fb54c112ca0 con 0x7fb54c109150 2024-09-11T00:03:12.697 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:12.694+0000 7fb552a00640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fb54c112fa0 con 0x7fb54c1087d0 2024-09-11T00:03:12.697 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:12.694+0000 7fb54be00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb54c102dc0 0x7fb54c19b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:03:12.697 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:12.694+0000 7fb543400640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb54c1087d0 0x7fb54c19ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:03:12.698 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:12.694+0000 7fb550a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb54c109150 0x7fb54c1a0dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:03:12.698 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:12.694+0000 7fb543400640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb54c1087d0 0x7fb54c19ba20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.12:58598/0 (socket says 172.21.15.12:58598) 2024-09-11T00:03:12.698 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:12.694+0000 7fb550a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb54c109150 0x7fb54c1a0dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.12:60406/0 (socket says 172.21.15.12:60406) 2024-09-11T00:03:12.698 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:12.694+0000 7fb54be00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb54c102dc0 0x7fb54c19b4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:58596/0 (socket says 172.21.15.12:58596) 2024-09-11T00:03:12.698 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:12.694+0000 7fb543400640 1 -- 172.21.15.12:0/309955523 learned_addr learned my addr 172.21.15.12:0/309955523 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:03:12.698 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:12.695+0000 7fb543400640 1 -- 172.21.15.12:0/309955523 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb54c109150 msgr2=0x7fb54c1a0dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:03:12.698 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:12.695+0000 7fb543400640 1 --2- 172.21.15.12:0/309955523 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb54c109150 0x7fb54c1a0dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:03:12.698 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:12.695+0000 7fb543400640 1 -- 172.21.15.12:0/309955523 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb54c102dc0 msgr2=0x7fb54c19b4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:03:12.698 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:12.695+0000 7fb543400640 1 --2- 172.21.15.12:0/309955523 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb54c102dc0 0x7fb54c19b4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:03:12.698 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:12.695+0000 7fb543400640 1 -- 172.21.15.12:0/309955523 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb54c1a1590 con 0x7fb54c1087d0 2024-09-11T00:03:12.698 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:12.695+0000 7fb54be00640 1 --2- 172.21.15.12:0/309955523 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb54c102dc0 0x7fb54c19b4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:03:12.698 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:12.695+0000 7fb543400640 1 --2- 172.21.15.12:0/309955523 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb54c1087d0 0x7fb54c19ba20 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fb534005ab0 tx=0x7fb534003c10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:03:12.699 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:12.695+0000 7fb549600640 1 -- 172.21.15.12:0/309955523 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb53403f070 con 0x7fb54c1087d0 2024-09-11T00:03:12.699 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:12.695+0000 7fb552a00640 1 -- 172.21.15.12:0/309955523 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb54c1a1820 con 0x7fb54c1087d0 2024-09-11T00:03:12.699 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:12.696+0000 7fb549600640 1 -- 172.21.15.12:0/309955523 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb534031b60 con 0x7fb54c1087d0 2024-09-11T00:03:12.699 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:12.696+0000 7fb552a00640 1 -- 172.21.15.12:0/309955523 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb54c1a7f40 con 0x7fb54c1087d0 2024-09-11T00:03:12.699 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:12.696+0000 7fb549600640 1 -- 172.21.15.12:0/309955523 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb534035030 con 0x7fb54c1087d0 2024-09-11T00:03:12.699 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:12.697+0000 7fb552a00640 1 -- 172.21.15.12:0/309955523 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb54c1031a0 con 0x7fb54c1087d0 2024-09-11T00:03:12.701 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:12.699+0000 7fb549600640 1 -- 172.21.15.12:0/309955523 <== mon.1 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 16) ==== 153250+0+0 (secure 0 0 0) 0x7fb53404c060 con 0x7fb54c1087d0 2024-09-11T00:03:12.702 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:12.701+0000 7fb549600640 1 --2- 172.21.15.12:0/309955523 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fb5300b52e0 0x7fb5300b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:03:12.703 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:12.701+0000 7fb549600640 1 -- 172.21.15.12:0/309955523 <== mon.1 v2:172.21.15.178:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 7127+0+0 (secure 0 0 0) 0x7fb5341060f0 con 0x7fb54c1087d0 2024-09-11T00:03:12.703 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:12.701+0000 7fb54be00640 1 --2- 172.21.15.12:0/309955523 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fb5300b52e0 0x7fb5300b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:03:12.703 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:12.701+0000 7fb54be00640 1 --2- 172.21.15.12:0/309955523 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fb5300b52e0 0x7fb5300b77a0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fb53c009fc0 tx=0x7fb53c005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:03:12.704 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:12.702+0000 7fb549600640 1 -- 172.21.15.12:0/309955523 <== mon.1 v2:172.21.15.178:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7fb5340ccd60 con 0x7fb54c1087d0 2024-09-11T00:03:12.898 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:12.896+0000 7fb552a00640 1 -- 172.21.15.12:0/309955523 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fb54c10edb0 con 0x7fb54c1087d0 2024-09-11T00:03:12.899 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:12.897+0000 7fb549600640 1 -- 172.21.15.12:0/309955523 <== mon.1 v2:172.21.15.178:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v69) ==== 93+0+10 (secure 0 0 0) 0x7fb5340d1790 con 0x7fb54c1087d0 2024-09-11T00:03:12.899 INFO:teuthology.orchestra.run.smithi012.stdout:pg_num: 1 2024-09-11T00:03:12.902 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:12.900+0000 7fb552a00640 1 -- 172.21.15.12:0/309955523 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fb5300b52e0 msgr2=0x7fb5300b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:03:12.902 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:12.900+0000 7fb552a00640 1 --2- 172.21.15.12:0/309955523 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fb5300b52e0 0x7fb5300b77a0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fb53c009fc0 tx=0x7fb53c005f90 comp rx=0 tx=0).stop 2024-09-11T00:03:12.903 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:12.900+0000 7fb552a00640 1 -- 172.21.15.12:0/309955523 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb54c1087d0 msgr2=0x7fb54c19ba20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:03:12.903 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:12.900+0000 7fb552a00640 1 --2- 172.21.15.12:0/309955523 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb54c1087d0 0x7fb54c19ba20 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fb534005ab0 tx=0x7fb534003c10 comp rx=0 tx=0).stop 2024-09-11T00:03:12.903 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:12.900+0000 7fb552a00640 1 -- 172.21.15.12:0/309955523 shutdown_connections 2024-09-11T00:03:12.903 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:12.900+0000 7fb552a00640 1 --2- 172.21.15.12:0/309955523 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fb5300b52e0 0x7fb5300b77a0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:03:12.903 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:12.900+0000 7fb552a00640 1 --2- 172.21.15.12:0/309955523 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb54c109150 0x7fb54c1a0dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:03:12.903 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:12.900+0000 7fb552a00640 1 --2- 172.21.15.12:0/309955523 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb54c1087d0 0x7fb54c19ba20 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:03:12.903 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:12.900+0000 7fb552a00640 1 --2- 172.21.15.12:0/309955523 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb54c102dc0 0x7fb54c19b4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:03:12.903 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:12.900+0000 7fb552a00640 1 -- 172.21.15.12:0/309955523 >> 172.21.15.12:0/309955523 conn(0x7fb54c0fe830 msgr2=0x7fb54c105db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:03:12.903 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:12.901+0000 7fb552a00640 1 -- 172.21.15.12:0/309955523 shutdown_connections 2024-09-11T00:03:12.903 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:12.901+0000 7fb552a00640 1 -- 172.21.15.12:0/309955523 wait complete. 2024-09-11T00:03:13.254 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:03:13 smithi012 ceph-mon[30333]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T00:03:13.254 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:03:13 smithi012 ceph-mon[30333]: from='client.? 172.21.15.12:0/309955523' entity='client.admin' cmd={"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} : dispatch 2024-09-11T00:03:13.532 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:03:13 smithi165 ceph-mon[35356]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T00:03:13.532 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:03:13 smithi165 ceph-mon[35356]: from='client.? 172.21.15.12:0/309955523' entity='client.admin' cmd={"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} : dispatch 2024-09-11T00:03:13.580 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:03:13 smithi178 ceph-mon[35919]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T00:03:13.581 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:03:13 smithi178 ceph-mon[35919]: from='client.? 172.21.15.12:0/309955523' entity='client.admin' cmd={"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} : dispatch 2024-09-11T00:03:13.848 INFO:tasks.cephadm:Adding mds.a on smithi012 2024-09-11T00:03:13.848 INFO:tasks.cephadm:Adding mds.d on smithi012 2024-09-11T00:03:13.848 INFO:tasks.cephadm:Adding mds.g on smithi012 2024-09-11T00:03:13.848 INFO:tasks.cephadm:Adding mds.j on smithi012 2024-09-11T00:03:13.848 INFO:tasks.cephadm:Adding mds.b on smithi165 2024-09-11T00:03:13.848 INFO:tasks.cephadm:Adding mds.e on smithi165 2024-09-11T00:03:13.848 INFO:tasks.cephadm:Adding mds.h on smithi165 2024-09-11T00:03:13.848 INFO:tasks.cephadm:Adding mds.k on smithi165 2024-09-11T00:03:13.849 INFO:tasks.cephadm:Adding mds.c on smithi178 2024-09-11T00:03:13.849 INFO:tasks.cephadm:Adding mds.f on smithi178 2024-09-11T00:03:13.849 INFO:tasks.cephadm:Adding mds.i on smithi178 2024-09-11T00:03:13.849 INFO:tasks.cephadm:Adding mds.l on smithi178 2024-09-11T00:03:13.849 DEBUG:teuthology.orchestra.run.smithi178:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ff3ad74a-6fcf-11ef-bcea-c7b262605968 -- ceph orch apply mds all '12;smithi012=a;smithi012=d;smithi012=g;smithi012=j;smithi165=b;smithi165=e;smithi165=h;smithi165=k;smithi178=c;smithi178=f;smithi178=i;smithi178=l' 2024-09-11T00:03:14.108 INFO:teuthology.orchestra.run.smithi178.stderr:Inferring config /var/lib/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/mon.c/config 2024-09-11T00:03:14.633 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T00:03:14.608+0000 7f32d8199640 1 -- 172.21.15.178:0/2186784262 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f32d0103b90 msgr2=0x7f32d0104010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:03:14.634 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T00:03:14.608+0000 7f32d8199640 1 --2- 172.21.15.178:0/2186784262 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f32d0103b90 0x7f32d0104010 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f32c0009960 tx=0x7f32c002a460 comp rx=0 tx=0).stop 2024-09-11T00:03:14.634 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T00:03:14.608+0000 7f32d8199640 1 -- 172.21.15.178:0/2186784262 shutdown_connections 2024-09-11T00:03:14.634 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T00:03:14.608+0000 7f32d8199640 1 --2- 172.21.15.178:0/2186784262 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f32d0104550 0x7f32d010aed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:03:14.634 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T00:03:14.608+0000 7f32d8199640 1 --2- 172.21.15.178:0/2186784262 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f32d0103b90 0x7f32d0104010 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:03:14.634 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T00:03:14.608+0000 7f32d8199640 1 --2- 172.21.15.178:0/2186784262 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f32d0102a30 0x7f32d0102e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:03:14.634 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T00:03:14.608+0000 7f32d8199640 1 -- 172.21.15.178:0/2186784262 >> 172.21.15.178:0/2186784262 conn(0x7f32d00fe320 msgr2=0x7f32d0100740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:03:14.634 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T00:03:14.608+0000 7f32d8199640 1 -- 172.21.15.178:0/2186784262 shutdown_connections 2024-09-11T00:03:14.634 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T00:03:14.608+0000 7f32d8199640 1 -- 172.21.15.178:0/2186784262 wait complete. 2024-09-11T00:03:14.634 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T00:03:14.609+0000 7f32d8199640 1 Processor -- start 2024-09-11T00:03:14.634 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T00:03:14.609+0000 7f32d8199640 1 -- start start 2024-09-11T00:03:14.634 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T00:03:14.609+0000 7f32d8199640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f32d0102a30 0x7f32d0076cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:03:14.634 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T00:03:14.609+0000 7f32d8199640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f32d0103b90 0x7f32d0075540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:03:14.635 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T00:03:14.609+0000 7f32d8199640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f32d0104550 0x7f32d0075a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:03:14.635 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T00:03:14.609+0000 7f32d8199640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f32d010e6e0 con 0x7f32d0102a30 2024-09-11T00:03:14.635 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T00:03:14.609+0000 7f32d8199640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f32d010e560 con 0x7f32d0104550 2024-09-11T00:03:14.635 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T00:03:14.609+0000 7f32d8199640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f32d010e860 con 0x7f32d0103b90 2024-09-11T00:03:14.635 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T00:03:14.609+0000 7f32d5734640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f32d0103b90 0x7f32d0075540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:03:14.635 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T00:03:14.609+0000 7f32d5734640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f32d0103b90 0x7f32d0075540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:43436/0 (socket says 172.21.15.178:43436) 2024-09-11T00:03:14.635 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T00:03:14.609+0000 7f32d5734640 1 -- 172.21.15.178:0/603579166 learned_addr learned my addr 172.21.15.178:0/603579166 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-11T00:03:14.635 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T00:03:14.609+0000 7f32d6736640 1 --2- 172.21.15.178:0/603579166 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f32d0104550 0x7f32d0075a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:03:14.635 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T00:03:14.609+0000 7f32d5f35640 1 --2- 172.21.15.178:0/603579166 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f32d0102a30 0x7f32d0076cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:03:14.635 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T00:03:14.609+0000 7f32d5734640 1 -- 172.21.15.178:0/603579166 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f32d0104550 msgr2=0x7f32d0075a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:03:14.635 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T00:03:14.609+0000 7f32d5734640 1 --2- 172.21.15.178:0/603579166 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f32d0104550 0x7f32d0075a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:03:14.635 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T00:03:14.609+0000 7f32d5734640 1 -- 172.21.15.178:0/603579166 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f32d0102a30 msgr2=0x7f32d0076cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:03:14.636 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T00:03:14.609+0000 7f32d5734640 1 --2- 172.21.15.178:0/603579166 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f32d0102a30 0x7f32d0076cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:03:14.636 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T00:03:14.609+0000 7f32d5734640 1 -- 172.21.15.178:0/603579166 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32d0076340 con 0x7f32d0103b90 2024-09-11T00:03:14.636 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T00:03:14.610+0000 7f32d5734640 1 --2- 172.21.15.178:0/603579166 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f32d0103b90 0x7f32d0075540 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f32c0031460 tx=0x7f32c0003b90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:03:14.636 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T00:03:14.610+0000 7f32c6ffd640 1 -- 172.21.15.178:0/603579166 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32c003f070 con 0x7f32d0103b90 2024-09-11T00:03:14.636 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T00:03:14.610+0000 7f32c6ffd640 1 -- 172.21.15.178:0/603579166 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f32c0031bb0 con 0x7f32d0103b90 2024-09-11T00:03:14.636 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T00:03:14.610+0000 7f32d8199640 1 -- 172.21.15.178:0/603579166 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f32d01a2ed0 con 0x7f32d0103b90 2024-09-11T00:03:14.636 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T00:03:14.610+0000 7f32c6ffd640 1 -- 172.21.15.178:0/603579166 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32c0035030 con 0x7f32d0103b90 2024-09-11T00:03:14.636 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T00:03:14.610+0000 7f32d8199640 1 -- 172.21.15.178:0/603579166 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32d01a3330 con 0x7f32d0103b90 2024-09-11T00:03:14.636 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T00:03:14.611+0000 7f32d8199640 1 -- 172.21.15.178:0/603579166 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f32d0109db0 con 0x7f32d0103b90 2024-09-11T00:03:14.636 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T00:03:14.612+0000 7f32c6ffd640 1 -- 172.21.15.178:0/603579166 <== mon.1 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 16) ==== 153250+0+0 (secure 0 0 0) 0x7f32c003b460 con 0x7f32d0103b90 2024-09-11T00:03:14.636 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T00:03:14.613+0000 7f32c6ffd640 1 --2- 172.21.15.178:0/603579166 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f32bc0b5360 0x7f32bc0b7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:03:14.636 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T00:03:14.613+0000 7f32c6ffd640 1 -- 172.21.15.178:0/603579166 <== mon.1 v2:172.21.15.178:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 7127+0+0 (secure 0 0 0) 0x7f32c0106790 con 0x7f32d0103b90 2024-09-11T00:03:14.637 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T00:03:14.613+0000 7f32d5f35640 1 --2- 172.21.15.178:0/603579166 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f32bc0b5360 0x7f32bc0b7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:03:14.637 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T00:03:14.614+0000 7f32d5f35640 1 --2- 172.21.15.178:0/603579166 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f32bc0b5360 0x7f32bc0b7820 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f32cc0058c0 tx=0x7f32cc00a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:03:14.637 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T00:03:14.616+0000 7f32c6ffd640 1 -- 172.21.15.178:0/603579166 <== mon.1 v2:172.21.15.178:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f32c00cd4a0 con 0x7f32d0103b90 2024-09-11T00:03:14.803 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T00:03:14.802+0000 7f32d8199640 1 -- 172.21.15.178:0/603579166 --> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] -- mgr_command(tid 0: {"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi012=a;smithi012=d;smithi012=g;smithi012=j;smithi165=b;smithi165=e;smithi165=h;smithi165=k;smithi178=c;smithi178=f;smithi178=i;smithi178=l", "target": ["mon-mgr", ""]}) -- 0x7f32d0063750 con 0x7f32bc0b5360 2024-09-11T00:03:14.812 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T00:03:14.811+0000 7f32c6ffd640 1 -- 172.21.15.178:0/603579166 <== mgr.14150 v2:172.21.15.12:6800/3442939137 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+28 (secure 0 0 0) 0x7f32d0063750 con 0x7f32bc0b5360 2024-09-11T00:03:14.812 INFO:teuthology.orchestra.run.smithi178.stdout:Scheduled mds.all update... 2024-09-11T00:03:14.814 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T00:03:14.813+0000 7f32d8199640 1 -- 172.21.15.178:0/603579166 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f32bc0b5360 msgr2=0x7f32bc0b7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:03:14.814 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T00:03:14.813+0000 7f32d8199640 1 --2- 172.21.15.178:0/603579166 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f32bc0b5360 0x7f32bc0b7820 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f32cc0058c0 tx=0x7f32cc00a2d0 comp rx=0 tx=0).stop 2024-09-11T00:03:14.815 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T00:03:14.813+0000 7f32d8199640 1 -- 172.21.15.178:0/603579166 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f32d0103b90 msgr2=0x7f32d0075540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:03:14.815 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T00:03:14.813+0000 7f32d8199640 1 --2- 172.21.15.178:0/603579166 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f32d0103b90 0x7f32d0075540 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f32c0031460 tx=0x7f32c0003b90 comp rx=0 tx=0).stop 2024-09-11T00:03:14.815 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T00:03:14.813+0000 7f32d8199640 1 -- 172.21.15.178:0/603579166 shutdown_connections 2024-09-11T00:03:14.815 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T00:03:14.813+0000 7f32d8199640 1 --2- 172.21.15.178:0/603579166 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f32bc0b5360 0x7f32bc0b7820 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:03:14.815 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T00:03:14.813+0000 7f32d8199640 1 --2- 172.21.15.178:0/603579166 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f32d0104550 0x7f32d0075a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:03:14.815 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T00:03:14.813+0000 7f32d8199640 1 --2- 172.21.15.178:0/603579166 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f32d0103b90 0x7f32d0075540 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:03:14.815 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T00:03:14.813+0000 7f32d8199640 1 --2- 172.21.15.178:0/603579166 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f32d0102a30 0x7f32d0076cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:03:14.815 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T00:03:14.813+0000 7f32d8199640 1 -- 172.21.15.178:0/603579166 >> 172.21.15.178:0/603579166 conn(0x7f32d00fe320 msgr2=0x7f32d0106f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:03:14.815 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T00:03:14.813+0000 7f32d8199640 1 -- 172.21.15.178:0/603579166 shutdown_connections 2024-09-11T00:03:14.815 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-11T00:03:14.813+0000 7f32d8199640 1 -- 172.21.15.178:0/603579166 wait complete. 2024-09-11T00:03:15.457 DEBUG:teuthology.orchestra.run.smithi012:mds.a> sudo journalctl -f -n 0 -u ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968@mds.a.service 2024-09-11T00:03:15.463 DEBUG:teuthology.orchestra.run.smithi012:mds.d> sudo journalctl -f -n 0 -u ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968@mds.d.service 2024-09-11T00:03:15.507 DEBUG:teuthology.orchestra.run.smithi012:mds.g> sudo journalctl -f -n 0 -u ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968@mds.g.service 2024-09-11T00:03:15.554 DEBUG:teuthology.orchestra.run.smithi012:mds.j> sudo journalctl -f -n 0 -u ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968@mds.j.service 2024-09-11T00:03:15.598 DEBUG:teuthology.orchestra.run.smithi165:mds.b> sudo journalctl -f -n 0 -u ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968@mds.b.service 2024-09-11T00:03:15.602 DEBUG:teuthology.orchestra.run.smithi165:mds.e> sudo journalctl -f -n 0 -u ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968@mds.e.service 2024-09-11T00:03:15.645 DEBUG:teuthology.orchestra.run.smithi165:mds.h> sudo journalctl -f -n 0 -u ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968@mds.h.service 2024-09-11T00:03:15.689 DEBUG:teuthology.orchestra.run.smithi165:mds.k> sudo journalctl -f -n 0 -u ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968@mds.k.service 2024-09-11T00:03:15.732 DEBUG:teuthology.orchestra.run.smithi178:mds.c> sudo journalctl -f -n 0 -u ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968@mds.c.service 2024-09-11T00:03:15.734 DEBUG:teuthology.orchestra.run.smithi178:mds.f> sudo journalctl -f -n 0 -u ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968@mds.f.service 2024-09-11T00:03:15.777 DEBUG:teuthology.orchestra.run.smithi178:mds.i> sudo journalctl -f -n 0 -u ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968@mds.i.service 2024-09-11T00:03:15.820 DEBUG:teuthology.orchestra.run.smithi178:mds.l> sudo journalctl -f -n 0 -u ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968@mds.l.service 2024-09-11T00:03:15.863 INFO:tasks.cephadm:Setting up CephFS filesystem(s)... 2024-09-11T00:03:15.863 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid ff3ad74a-6fcf-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T00:03:16.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:03:15 smithi165 ceph-mon[35356]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T00:03:16.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:03:15 smithi165 ceph-mon[35356]: from='client.24550 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi012=a;smithi012=d;smithi012=g;smithi012=j;smithi165=b;smithi165=e;smithi165=h;smithi165=k;smithi178=c;smithi178=f;smithi178=i;smithi178=l", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T00:03:16.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:03:15 smithi165 ceph-mon[35356]: Saving service mds.all spec with placement smithi012=a;smithi012=d;smithi012=g;smithi012=j;smithi165=b;smithi165=e;smithi165=h;smithi165=k;smithi178=c;smithi178=f;smithi178=i;smithi178=l;count:12 2024-09-11T00:03:16.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:03:15 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:03:16.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:03:15 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T00:03:16.033 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:03:15 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:03:16.033 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:03:15 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T00:03:16.033 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:03:15 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:03:16.033 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:03:15 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-11T00:03:16.033 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:03:15 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-11T00:03:16.033 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:03:15 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:03:16.033 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:03:15 smithi178 ceph-mon[35919]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T00:03:16.033 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:03:15 smithi178 ceph-mon[35919]: from='client.24550 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi012=a;smithi012=d;smithi012=g;smithi012=j;smithi165=b;smithi165=e;smithi165=h;smithi165=k;smithi178=c;smithi178=f;smithi178=i;smithi178=l", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T00:03:16.033 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:03:15 smithi178 ceph-mon[35919]: Saving service mds.all spec with placement smithi012=a;smithi012=d;smithi012=g;smithi012=j;smithi165=b;smithi165=e;smithi165=h;smithi165=k;smithi178=c;smithi178=f;smithi178=i;smithi178=l;count:12 2024-09-11T00:03:16.034 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:03:15 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:03:16.034 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:03:15 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T00:03:16.034 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:03:15 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:03:16.034 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:03:15 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T00:03:16.034 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:03:15 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:03:16.034 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:03:15 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-11T00:03:16.034 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:03:15 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-11T00:03:16.034 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:03:15 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:03:16.063 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:03:15 smithi012 ceph-mon[30333]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T00:03:16.063 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:03:15 smithi012 ceph-mon[30333]: from='client.24550 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi012=a;smithi012=d;smithi012=g;smithi012=j;smithi165=b;smithi165=e;smithi165=h;smithi165=k;smithi178=c;smithi178=f;smithi178=i;smithi178=l", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T00:03:16.063 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:03:15 smithi012 ceph-mon[30333]: Saving service mds.all spec with placement smithi012=a;smithi012=d;smithi012=g;smithi012=j;smithi165=b;smithi165=e;smithi165=h;smithi165=k;smithi178=c;smithi178=f;smithi178=i;smithi178=l;count:12 2024-09-11T00:03:16.063 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:03:15 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:03:16.063 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:03:15 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T00:03:16.063 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:03:15 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:03:16.063 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:03:15 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T00:03:16.063 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:03:15 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:03:16.063 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:03:15 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-11T00:03:16.063 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:03:15 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-11T00:03:16.063 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:03:15 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:03:16.234 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:03:16.941 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:03:16 smithi012 ceph-mon[30333]: Deploying daemon mds.l on smithi178 2024-09-11T00:03:16.944 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:16.938+0000 7fa48aa00640 1 -- 172.21.15.12:0/2083442012 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa484075540 msgr2=0x7fa484075920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:03:16.944 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:16.938+0000 7fa48aa00640 1 --2- 172.21.15.12:0/2083442012 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa484075540 0x7fa484075920 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fa474009900 tx=0x7fa47402a3f0 comp rx=0 tx=0).stop 2024-09-11T00:03:16.944 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:16.938+0000 7fa48aa00640 1 -- 172.21.15.12:0/2083442012 shutdown_connections 2024-09-11T00:03:16.944 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:16.938+0000 7fa48aa00640 1 --2- 172.21.15.12:0/2083442012 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa484113b00 0x7fa484115ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:03:16.945 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:16.938+0000 7fa48aa00640 1 --2- 172.21.15.12:0/2083442012 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa484075e60 0x7fa4841135c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:03:16.945 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:16.938+0000 7fa48aa00640 1 --2- 172.21.15.12:0/2083442012 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa484075540 0x7fa484075920 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:03:16.945 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:16.938+0000 7fa48aa00640 1 -- 172.21.15.12:0/2083442012 >> 172.21.15.12:0/2083442012 conn(0x7fa4840fe810 msgr2=0x7fa484100c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:03:16.945 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:16.938+0000 7fa48aa00640 1 -- 172.21.15.12:0/2083442012 shutdown_connections 2024-09-11T00:03:16.945 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:16.939+0000 7fa48aa00640 1 -- 172.21.15.12:0/2083442012 wait complete. 2024-09-11T00:03:16.945 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:16.940+0000 7fa48aa00640 1 Processor -- start 2024-09-11T00:03:16.945 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:16.940+0000 7fa48aa00640 1 -- start start 2024-09-11T00:03:16.945 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:16.940+0000 7fa48aa00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa484075540 0x7fa48419f8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:03:16.945 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:16.940+0000 7fa48aa00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa484075e60 0x7fa48419fe30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:03:16.945 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:16.940+0000 7fa48aa00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa484113b00 0x7fa4841a51d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:03:16.945 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:16.940+0000 7fa48aa00640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fa484119380 con 0x7fa484113b00 2024-09-11T00:03:16.945 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:16.940+0000 7fa48aa00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fa484119200 con 0x7fa484075e60 2024-09-11T00:03:16.946 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:16.940+0000 7fa48aa00640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fa484119500 con 0x7fa484075540 2024-09-11T00:03:16.946 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:16.941+0000 7fa483e00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa484075540 0x7fa48419f8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:03:16.946 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:16.941+0000 7fa488a00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa484113b00 0x7fa4841a51d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:03:16.946 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:16.941+0000 7fa47be00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa484075e60 0x7fa48419fe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:03:16.946 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:16.941+0000 7fa483e00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa484075540 0x7fa48419f8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.12:58622/0 (socket says 172.21.15.12:58622) 2024-09-11T00:03:16.946 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:16.941+0000 7fa488a00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa484113b00 0x7fa4841a51d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:58622/0 (socket says 172.21.15.12:58622) 2024-09-11T00:03:16.946 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:16.941+0000 7fa47be00640 1 --2- 172.21.15.12:0/2519972802 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa484075e60 0x7fa48419fe30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.12:60426/0 (socket says 172.21.15.12:60426) 2024-09-11T00:03:16.946 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:16.941+0000 7fa483e00640 1 -- 172.21.15.12:0/2519972802 learned_addr learned my addr 172.21.15.12:0/2519972802 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:03:16.946 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:16.941+0000 7fa488a00640 1 -- 172.21.15.12:0/2519972802 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa484075540 msgr2=0x7fa48419f8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:03:16.946 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:16.941+0000 7fa488a00640 1 --2- 172.21.15.12:0/2519972802 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa484075540 0x7fa48419f8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:03:16.946 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:16.941+0000 7fa488a00640 1 -- 172.21.15.12:0/2519972802 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa484075e60 msgr2=0x7fa48419fe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:03:16.946 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:16.941+0000 7fa488a00640 1 --2- 172.21.15.12:0/2519972802 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa484075e60 0x7fa48419fe30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:03:16.946 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:16.941+0000 7fa488a00640 1 -- 172.21.15.12:0/2519972802 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4841a59a0 con 0x7fa484113b00 2024-09-11T00:03:16.947 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:16.941+0000 7fa47be00640 1 --2- 172.21.15.12:0/2519972802 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa484075e60 0x7fa48419fe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:03:16.947 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:16.942+0000 7fa488a00640 1 --2- 172.21.15.12:0/2519972802 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa484113b00 0x7fa4841a51d0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fa47c005fd0 tx=0x7fa47c00ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:03:16.947 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:16.942+0000 7fa481600640 1 -- 172.21.15.12:0/2519972802 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa47c00cdf0 con 0x7fa484113b00 2024-09-11T00:03:16.947 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:16.942+0000 7fa481600640 1 -- 172.21.15.12:0/2519972802 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa47c005020 con 0x7fa484113b00 2024-09-11T00:03:16.947 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:16.942+0000 7fa48aa00640 1 -- 172.21.15.12:0/2519972802 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa4841a5c90 con 0x7fa484113b00 2024-09-11T00:03:16.947 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:16.942+0000 7fa481600640 1 -- 172.21.15.12:0/2519972802 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa47c010690 con 0x7fa484113b00 2024-09-11T00:03:16.947 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:16.942+0000 7fa48aa00640 1 -- 172.21.15.12:0/2519972802 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4841ac350 con 0x7fa484113b00 2024-09-11T00:03:16.947 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:16.943+0000 7fa48aa00640 1 -- 172.21.15.12:0/2519972802 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa484075920 con 0x7fa484113b00 2024-09-11T00:03:16.948 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:16.945+0000 7fa481600640 1 -- 172.21.15.12:0/2519972802 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 16) ==== 153250+0+0 (secure 0 0 0) 0x7fa47c005460 con 0x7fa484113b00 2024-09-11T00:03:16.948 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:16.947+0000 7fa481600640 1 --2- 172.21.15.12:0/2519972802 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fa4680b52e0 0x7fa4680b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:03:16.949 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:16.947+0000 7fa481600640 1 -- 172.21.15.12:0/2519972802 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 7127+0+0 (secure 0 0 0) 0x7fa47c0e5e90 con 0x7fa484113b00 2024-09-11T00:03:16.949 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:16.947+0000 7fa483e00640 1 --2- 172.21.15.12:0/2519972802 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fa4680b52e0 0x7fa4680b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:03:16.949 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:16.948+0000 7fa483e00640 1 --2- 172.21.15.12:0/2519972802 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fa4680b52e0 0x7fa4680b77a0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fa474031440 tx=0x7fa4740058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:03:16.950 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:16.949+0000 7fa481600640 1 -- 172.21.15.12:0/2519972802 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7fa47c0ad150 con 0x7fa484113b00 2024-09-11T00:03:17.070 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:03:16 smithi178 ceph-mon[35919]: Deploying daemon mds.l on smithi178 2024-09-11T00:03:17.183 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:17.182+0000 7fa48aa00640 1 -- 172.21.15.12:0/2519972802 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa4841a0a20 con 0x7fa484113b00 2024-09-11T00:03:17.184 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:17.182+0000 7fa481600640 1 -- 172.21.15.12:0/2519972802 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 1 v1) ==== 93+0+611 (secure 0 0 0) 0x7fa4841a0a20 con 0x7fa484113b00 2024-09-11T00:03:17.184 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-11T00:03:17.184 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":1,"btime":"2024-09-10T23:55:16:317386+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[]} 2024-09-11T00:03:17.185 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 1 2024-09-11T00:03:17.187 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:17.184+0000 7fa48aa00640 1 -- 172.21.15.12:0/2519972802 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fa4680b52e0 msgr2=0x7fa4680b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:03:17.187 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:17.184+0000 7fa48aa00640 1 --2- 172.21.15.12:0/2519972802 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fa4680b52e0 0x7fa4680b77a0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fa474031440 tx=0x7fa4740058d0 comp rx=0 tx=0).stop 2024-09-11T00:03:17.187 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:17.184+0000 7fa48aa00640 1 -- 172.21.15.12:0/2519972802 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa484113b00 msgr2=0x7fa4841a51d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:03:17.187 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:17.184+0000 7fa48aa00640 1 --2- 172.21.15.12:0/2519972802 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa484113b00 0x7fa4841a51d0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fa47c005fd0 tx=0x7fa47c00ef10 comp rx=0 tx=0).stop 2024-09-11T00:03:17.187 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:17.185+0000 7fa48aa00640 1 -- 172.21.15.12:0/2519972802 shutdown_connections 2024-09-11T00:03:17.187 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:17.185+0000 7fa48aa00640 1 --2- 172.21.15.12:0/2519972802 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fa4680b52e0 0x7fa4680b77a0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:03:17.187 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:17.185+0000 7fa48aa00640 1 --2- 172.21.15.12:0/2519972802 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa484113b00 0x7fa4841a51d0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:03:17.187 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:17.185+0000 7fa48aa00640 1 --2- 172.21.15.12:0/2519972802 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa484075e60 0x7fa48419fe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:03:17.187 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:17.185+0000 7fa48aa00640 1 --2- 172.21.15.12:0/2519972802 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa484075540 0x7fa48419f8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:03:17.187 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:17.185+0000 7fa48aa00640 1 -- 172.21.15.12:0/2519972802 >> 172.21.15.12:0/2519972802 conn(0x7fa4840fe810 msgr2=0x7fa4840ff470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:03:17.187 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:17.185+0000 7fa48aa00640 1 -- 172.21.15.12:0/2519972802 shutdown_connections 2024-09-11T00:03:17.187 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:17.185+0000 7fa48aa00640 1 -- 172.21.15.12:0/2519972802 wait complete. 2024-09-11T00:03:17.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:03:16 smithi165 ceph-mon[35356]: Deploying daemon mds.l on smithi178 2024-09-11T00:03:18.016 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:03:17 smithi012 ceph-mon[30333]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T00:03:18.016 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:03:17 smithi012 ceph-mon[30333]: from='client.? 172.21.15.12:0/2519972802' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T00:03:18.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:03:17 smithi178 ceph-mon[35919]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T00:03:18.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:03:17 smithi178 ceph-mon[35919]: from='client.? 172.21.15.12:0/2519972802' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T00:03:18.081 INFO:journalctl@ceph.mds.l.smithi178.stdout:Sep 11 00:03:17 smithi178 podman[65460]: 2024-09-11 00:03:17.955478833 +0000 UTC m=+0.136951323 container create f02db3543a9a1bb4f605b09d1bce6c8b7983e10355c399b367353636cae4b226 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-mds-l, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, io.buildah.version=1.37.2, ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-09-11T00:03:18.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:03:17 smithi165 ceph-mon[35356]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T00:03:18.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:03:17 smithi165 ceph-mon[35356]: from='client.? 172.21.15.12:0/2519972802' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T00:03:18.831 INFO:journalctl@ceph.mds.l.smithi178.stdout:Sep 11 00:03:18 smithi178 podman[65460]: 2024-09-11 00:03:18.333118286 +0000 UTC m=+0.514590776 container init f02db3543a9a1bb4f605b09d1bce6c8b7983e10355c399b367353636cae4b226 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-mds-l, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-09-11T00:03:18.831 INFO:journalctl@ceph.mds.l.smithi178.stdout:Sep 11 00:03:18 smithi178 podman[65460]: 2024-09-11 00:03:18.336270449 +0000 UTC m=+0.517742939 container start f02db3543a9a1bb4f605b09d1bce6c8b7983e10355c399b367353636cae4b226 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-mds-l, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-11T00:03:18.831 INFO:journalctl@ceph.mds.l.smithi178.stdout:Sep 11 00:03:18 smithi178 ceph-mds[65480]: -- 172.21.15.178:0/1527359706 <== mon.1 v2:172.21.15.178:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55b5c8765860 con 0x55b5c8743000 2024-09-11T00:03:18.831 INFO:journalctl@ceph.mds.l.smithi178.stdout:Sep 11 00:03:18 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-mds-l[65476]: starting mds.l at 2024-09-11T00:03:18.831 INFO:journalctl@ceph.mds.l.smithi178.stdout:Sep 11 00:03:18 smithi178 bash[65460]: f02db3543a9a1bb4f605b09d1bce6c8b7983e10355c399b367353636cae4b226 2024-09-11T00:03:18.831 INFO:journalctl@ceph.mds.l.smithi178.stdout:Sep 11 00:03:18 smithi178 systemd[1]: Started Ceph mds.l for ff3ad74a-6fcf-11ef-bcea-c7b262605968. 2024-09-11T00:03:19.532 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:03:19 smithi165 ceph-mon[35356]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T00:03:19.532 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:03:19 smithi165 ceph-mon[35356]: mds.? [v2:172.21.15.178:6832/1141636112,v1:172.21.15.178:6833/1141636112] up:boot 2024-09-11T00:03:19.532 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:03:19 smithi165 ceph-mon[35356]: fsmap 1 up:standby 2024-09-11T00:03:19.532 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:03:19 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "l"} : dispatch 2024-09-11T00:03:19.532 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:03:19 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:03:19.533 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:03:19 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:03:19.533 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:03:19 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:03:19.533 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:03:19 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-11T00:03:19.533 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:03:19 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-11T00:03:19.533 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:03:19 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:03:19.533 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:03:19 smithi165 ceph-mon[35356]: Deploying daemon mds.k on smithi165 2024-09-11T00:03:19.674 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:03:19 smithi012 ceph-mon[30333]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T00:03:19.674 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:03:19 smithi012 ceph-mon[30333]: mds.? [v2:172.21.15.178:6832/1141636112,v1:172.21.15.178:6833/1141636112] up:boot 2024-09-11T00:03:19.674 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:03:19 smithi012 ceph-mon[30333]: fsmap 1 up:standby 2024-09-11T00:03:19.674 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:03:19 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "l"} : dispatch 2024-09-11T00:03:19.674 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:03:19 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:03:19.674 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:03:19 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:03:19.675 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:03:19 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:03:19.675 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:03:19 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-11T00:03:19.675 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:03:19 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-11T00:03:19.675 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:03:19 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:03:19.675 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:03:19 smithi012 ceph-mon[30333]: Deploying daemon mds.k on smithi165 2024-09-11T00:03:19.831 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:03:19 smithi178 ceph-mon[35919]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T00:03:19.831 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:03:19 smithi178 ceph-mon[35919]: mds.? [v2:172.21.15.178:6832/1141636112,v1:172.21.15.178:6833/1141636112] up:boot 2024-09-11T00:03:19.831 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:03:19 smithi178 ceph-mon[35919]: fsmap 1 up:standby 2024-09-11T00:03:19.831 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:03:19 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "l"} : dispatch 2024-09-11T00:03:19.831 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:03:19 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:03:19.831 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:03:19 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:03:19.831 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:03:19 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:03:19.831 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:03:19 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-11T00:03:19.831 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:03:19 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-11T00:03:19.831 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:03:19 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:03:19.832 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:03:19 smithi178 ceph-mon[35919]: Deploying daemon mds.k on smithi165 2024-09-11T00:03:20.055 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid ff3ad74a-6fcf-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T00:03:20.441 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:03:21.115 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:21.112+0000 7f5a66600640 1 -- 172.21.15.12:0/3167049286 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5a6010b330 msgr2=0x7f5a60111b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:03:21.115 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:21.112+0000 7f5a66600640 1 --2- 172.21.15.12:0/3167049286 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5a6010b330 0x7f5a60111b30 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f5a58009ca0 tx=0x7f5a5802a8c0 comp rx=0 tx=0).stop 2024-09-11T00:03:21.115 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:21.112+0000 7f5a66600640 1 -- 172.21.15.12:0/3167049286 shutdown_connections 2024-09-11T00:03:21.115 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:21.112+0000 7f5a66600640 1 --2- 172.21.15.12:0/3167049286 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5a6010b330 0x7f5a60111b30 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:03:21.115 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:21.112+0000 7f5a66600640 1 --2- 172.21.15.12:0/3167049286 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5a6010a9b0 0x7f5a6010adf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:03:21.115 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:21.112+0000 7f5a66600640 1 --2- 172.21.15.12:0/3167049286 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5a60104fa0 0x7f5a60105380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:03:21.115 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:21.112+0000 7f5a66600640 1 -- 172.21.15.12:0/3167049286 >> 172.21.15.12:0/3167049286 conn(0x7f5a60100a50 msgr2=0x7f5a60102e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:03:21.115 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:21.112+0000 7f5a66600640 1 -- 172.21.15.12:0/3167049286 shutdown_connections 2024-09-11T00:03:21.115 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:21.113+0000 7f5a66600640 1 -- 172.21.15.12:0/3167049286 wait complete. 2024-09-11T00:03:21.116 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:21.113+0000 7f5a66600640 1 Processor -- start 2024-09-11T00:03:21.116 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:21.114+0000 7f5a66600640 1 -- start start 2024-09-11T00:03:21.116 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:21.114+0000 7f5a66600640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5a60104fa0 0x7f5a6019b4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:03:21.117 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:21.114+0000 7f5a66600640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5a6010a9b0 0x7f5a6019b9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:03:21.117 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:21.114+0000 7f5a66600640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5a6010b330 0x7f5a601a0d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:03:21.117 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:21.114+0000 7f5a66600640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f5a60114fb0 con 0x7f5a6010a9b0 2024-09-11T00:03:21.117 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:21.114+0000 7f5a66600640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f5a60114e30 con 0x7f5a6010b330 2024-09-11T00:03:21.117 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:21.114+0000 7f5a66600640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f5a60115130 con 0x7f5a60104fa0 2024-09-11T00:03:21.117 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:21.114+0000 7f5a5f400640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5a60104fa0 0x7f5a6019b4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:03:21.117 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:21.114+0000 7f5a5f400640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5a60104fa0 0x7f5a6019b4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.12:46584/0 (socket says 172.21.15.12:46584) 2024-09-11T00:03:21.117 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:21.114+0000 7f5a5f400640 1 -- 172.21.15.12:0/3386808123 learned_addr learned my addr 172.21.15.12:0/3386808123 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:03:21.117 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:21.114+0000 7f5a57e00640 1 --2- 172.21.15.12:0/3386808123 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5a6010a9b0 0x7f5a6019b9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:03:21.117 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:21.114+0000 7f5a5fe00640 1 --2- 172.21.15.12:0/3386808123 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5a6010b330 0x7f5a601a0d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:03:21.118 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:21.115+0000 7f5a5f400640 1 -- 172.21.15.12:0/3386808123 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5a6010b330 msgr2=0x7f5a601a0d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:03:21.118 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:21.115+0000 7f5a5f400640 1 --2- 172.21.15.12:0/3386808123 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5a6010b330 0x7f5a601a0d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:03:21.118 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:21.115+0000 7f5a5f400640 1 -- 172.21.15.12:0/3386808123 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5a6010a9b0 msgr2=0x7f5a6019b9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:03:21.118 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:21.115+0000 7f5a5f400640 1 --2- 172.21.15.12:0/3386808123 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5a6010a9b0 0x7f5a6019b9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:03:21.119 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:21.115+0000 7f5a5f400640 1 -- 172.21.15.12:0/3386808123 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a601a1550 con 0x7f5a60104fa0 2024-09-11T00:03:21.119 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:21.115+0000 7f5a57e00640 1 --2- 172.21.15.12:0/3386808123 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5a6010a9b0 0x7f5a6019b9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:03:21.119 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:21.115+0000 7f5a5fe00640 1 --2- 172.21.15.12:0/3386808123 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5a6010b330 0x7f5a601a0d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:03:21.119 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:21.115+0000 7f5a5f400640 1 --2- 172.21.15.12:0/3386808123 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5a60104fa0 0x7f5a6019b4a0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f5a5000e9f0 tx=0x7f5a5000edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:03:21.119 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:21.116+0000 7f5a5cc00640 1 -- 172.21.15.12:0/3386808123 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a5000cd80 con 0x7f5a60104fa0 2024-09-11T00:03:21.119 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:21.116+0000 7f5a66600640 1 -- 172.21.15.12:0/3386808123 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5a601a1840 con 0x7f5a60104fa0 2024-09-11T00:03:21.119 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:21.116+0000 7f5a5cc00640 1 -- 172.21.15.12:0/3386808123 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5a500054d0 con 0x7f5a60104fa0 2024-09-11T00:03:21.119 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:21.116+0000 7f5a5cc00640 1 -- 172.21.15.12:0/3386808123 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a50010690 con 0x7f5a60104fa0 2024-09-11T00:03:21.119 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:21.116+0000 7f5a66600640 1 -- 172.21.15.12:0/3386808123 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5a601a7f00 con 0x7f5a60104fa0 2024-09-11T00:03:21.120 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:21.117+0000 7f5a66600640 1 -- 172.21.15.12:0/3386808123 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5a60105380 con 0x7f5a60104fa0 2024-09-11T00:03:21.121 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:21.119+0000 7f5a5cc00640 1 -- 172.21.15.12:0/3386808123 <== mon.1 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 16) ==== 153250+0+0 (secure 0 0 0) 0x7f5a50004e90 con 0x7f5a60104fa0 2024-09-11T00:03:21.122 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:21.120+0000 7f5a5cc00640 1 --2- 172.21.15.12:0/3386808123 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f5a440b5360 0x7f5a440b7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:03:21.123 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:21.121+0000 7f5a5cc00640 1 -- 172.21.15.12:0/3386808123 <== mon.1 v2:172.21.15.178:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 7127+0+0 (secure 0 0 0) 0x7f5a500e5b00 con 0x7f5a60104fa0 2024-09-11T00:03:21.123 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:21.121+0000 7f5a57e00640 1 --2- 172.21.15.12:0/3386808123 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f5a440b5360 0x7f5a440b7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:03:21.123 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:21.121+0000 7f5a57e00640 1 --2- 172.21.15.12:0/3386808123 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f5a440b5360 0x7f5a440b7820 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f5a601033e0 tx=0x7f5a48005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:03:21.124 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:21.122+0000 7f5a5cc00640 1 -- 172.21.15.12:0/3386808123 <== mon.1 v2:172.21.15.178:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f5a500acdc0 con 0x7f5a60104fa0 2024-09-11T00:03:21.282 INFO:journalctl@ceph.mds.k.smithi165.stdout:Sep 11 00:03:21 smithi165 systemd[1]: Started Ceph mds.k for ff3ad74a-6fcf-11ef-bcea-c7b262605968. 2024-09-11T00:03:21.359 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:21.357+0000 7f5a66600640 1 -- 172.21.15.12:0/3386808123 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5a6019c5d0 con 0x7f5a60104fa0 2024-09-11T00:03:21.359 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:21.357+0000 7f5a5cc00640 1 -- 172.21.15.12:0/3386808123 <== mon.1 v2:172.21.15.178:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) ==== 93+0+1438 (secure 0 0 0) 0x7f5a500e3e20 con 0x7f5a60104fa0 2024-09-11T00:03:21.360 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-11T00:03:21.360 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":2,"btime":"2024-09-11T00:03:18:364764+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24562,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6833/1141636112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":1141636112},{"type":"v1","addr":"172.21.15.178:6833","nonce":1141636112}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-09-11T00:03:21.360 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 2 2024-09-11T00:03:21.362 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:21.360+0000 7f5a66600640 1 -- 172.21.15.12:0/3386808123 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f5a440b5360 msgr2=0x7f5a440b7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:03:21.362 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:21.360+0000 7f5a66600640 1 --2- 172.21.15.12:0/3386808123 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f5a440b5360 0x7f5a440b7820 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f5a601033e0 tx=0x7f5a48005f90 comp rx=0 tx=0).stop 2024-09-11T00:03:21.362 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:21.360+0000 7f5a66600640 1 -- 172.21.15.12:0/3386808123 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5a60104fa0 msgr2=0x7f5a6019b4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:03:21.362 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:21.360+0000 7f5a66600640 1 --2- 172.21.15.12:0/3386808123 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5a60104fa0 0x7f5a6019b4a0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f5a5000e9f0 tx=0x7f5a5000edf0 comp rx=0 tx=0).stop 2024-09-11T00:03:21.362 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:21.360+0000 7f5a66600640 1 -- 172.21.15.12:0/3386808123 shutdown_connections 2024-09-11T00:03:21.362 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:21.360+0000 7f5a66600640 1 --2- 172.21.15.12:0/3386808123 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f5a440b5360 0x7f5a440b7820 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:03:21.362 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:21.360+0000 7f5a66600640 1 --2- 172.21.15.12:0/3386808123 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5a6010b330 0x7f5a601a0d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:03:21.363 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:21.360+0000 7f5a66600640 1 --2- 172.21.15.12:0/3386808123 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5a6010a9b0 0x7f5a6019b9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:03:21.363 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:21.360+0000 7f5a66600640 1 --2- 172.21.15.12:0/3386808123 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5a60104fa0 0x7f5a6019b4a0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:03:21.363 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:21.360+0000 7f5a66600640 1 -- 172.21.15.12:0/3386808123 >> 172.21.15.12:0/3386808123 conn(0x7f5a60100a50 msgr2=0x7f5a601012d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:03:21.363 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:21.360+0000 7f5a66600640 1 -- 172.21.15.12:0/3386808123 shutdown_connections 2024-09-11T00:03:21.363 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:21.360+0000 7f5a66600640 1 -- 172.21.15.12:0/3386808123 wait complete. 2024-09-11T00:03:21.757 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:03:21 smithi012 ceph-mon[30333]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T00:03:21.757 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:03:21 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:03:21.757 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:03:21 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:03:21.757 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:03:21 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:03:21.757 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:03:21 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-11T00:03:21.758 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:03:21 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-11T00:03:21.758 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:03:21 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:03:21.758 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:03:21 smithi012 ceph-mon[30333]: from='client.? 172.21.15.12:0/3386808123' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T00:03:21.782 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:03:21 smithi165 ceph-mon[35356]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T00:03:21.782 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:03:21 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:03:21.782 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:03:21 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:03:21.782 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:03:21 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:03:21.782 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:03:21 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-11T00:03:21.782 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:03:21 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-11T00:03:21.782 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:03:21 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:03:21.782 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:03:21 smithi165 ceph-mon[35356]: from='client.? 172.21.15.12:0/3386808123' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T00:03:21.831 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:03:21 smithi178 ceph-mon[35919]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T00:03:21.831 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:03:21 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:03:21.831 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:03:21 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:03:21.831 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:03:21 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:03:21.831 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:03:21 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-11T00:03:21.831 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:03:21 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-11T00:03:21.831 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:03:21 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:03:21.831 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:03:21 smithi178 ceph-mon[35919]: from='client.? 172.21.15.12:0/3386808123' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T00:03:22.532 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:03:22 smithi165 ceph-mon[35356]: Deploying daemon mds.j on smithi012 2024-09-11T00:03:22.532 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:03:22 smithi165 ceph-mon[35356]: mds.? [v2:172.21.15.165:6832/3111980970,v1:172.21.15.165:6833/3111980970] up:boot 2024-09-11T00:03:22.532 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:03:22 smithi165 ceph-mon[35356]: fsmap 2 up:standby 2024-09-11T00:03:22.532 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:03:22 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "k"} : dispatch 2024-09-11T00:03:22.638 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:03:22 smithi012 ceph-mon[30333]: Deploying daemon mds.j on smithi012 2024-09-11T00:03:22.638 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:03:22 smithi012 ceph-mon[30333]: mds.? [v2:172.21.15.165:6832/3111980970,v1:172.21.15.165:6833/3111980970] up:boot 2024-09-11T00:03:22.638 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:03:22 smithi012 ceph-mon[30333]: fsmap 2 up:standby 2024-09-11T00:03:22.638 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:03:22 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "k"} : dispatch 2024-09-11T00:03:22.830 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:03:22 smithi178 ceph-mon[35919]: Deploying daemon mds.j on smithi012 2024-09-11T00:03:22.831 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:03:22 smithi178 ceph-mon[35919]: mds.? [v2:172.21.15.165:6832/3111980970,v1:172.21.15.165:6833/3111980970] up:boot 2024-09-11T00:03:22.831 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:03:22 smithi178 ceph-mon[35919]: fsmap 2 up:standby 2024-09-11T00:03:22.831 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:03:22 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "k"} : dispatch 2024-09-11T00:03:23.546 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:03:23 smithi012 ceph-mon[30333]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T00:03:23.782 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:03:23 smithi165 ceph-mon[35356]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T00:03:23.830 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:03:23 smithi178 ceph-mon[35919]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T00:03:24.135 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid ff3ad74a-6fcf-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T00:03:24.562 INFO:journalctl@ceph.mds.j.smithi012.stdout:Sep 11 00:03:24 smithi012 systemd[1]: Starting Ceph mds.j for ff3ad74a-6fcf-11ef-bcea-c7b262605968... 2024-09-11T00:03:24.936 INFO:journalctl@ceph.mds.j.smithi012.stdout:Sep 11 00:03:24 smithi012 podman[71766]: 2024-09-11 00:03:24.567266778 +0000 UTC m=+0.030033842 image pull 589dd18248e494ba411dc838aa4ae0e5e569213f08fcd7a2cf22cd3ef7e3c480 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e 2024-09-11T00:03:24.936 INFO:journalctl@ceph.mds.j.smithi012.stdout:Sep 11 00:03:24 smithi012 podman[71766]: 2024-09-11 00:03:24.689100974 +0000 UTC m=+0.151868014 container create 03bb22653329948ee894ab0727d5f9e6238c8f1d82179e4c84115d5893132343 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-mds-j, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-09-11T00:03:24.936 INFO:journalctl@ceph.mds.j.smithi012.stdout:Sep 11 00:03:24 smithi012 podman[71766]: 2024-09-11 00:03:24.933696166 +0000 UTC m=+0.396463221 container init 03bb22653329948ee894ab0727d5f9e6238c8f1d82179e4c84115d5893132343 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-mds-j, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, ceph=True, org.label-schema.schema-version=1.0) 2024-09-11T00:03:25.106 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:03:25.250 INFO:journalctl@ceph.mds.j.smithi012.stdout:Sep 11 00:03:24 smithi012 podman[71766]: 2024-09-11 00:03:24.940262469 +0000 UTC m=+0.403029513 container start 03bb22653329948ee894ab0727d5f9e6238c8f1d82179e4c84115d5893132343 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-mds-j, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD) 2024-09-11T00:03:25.250 INFO:journalctl@ceph.mds.j.smithi012.stdout:Sep 11 00:03:24 smithi012 ceph-mds[71799]: -- 172.21.15.12:0/2055058682 <== mon.0 v2:172.21.15.12:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x561f94de3c20 con 0x561f94dc1400 2024-09-11T00:03:25.251 INFO:journalctl@ceph.mds.j.smithi012.stdout:Sep 11 00:03:24 smithi012 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-mds-j[71795]: starting mds.j at 2024-09-11T00:03:25.251 INFO:journalctl@ceph.mds.j.smithi012.stdout:Sep 11 00:03:25 smithi012 bash[71766]: 03bb22653329948ee894ab0727d5f9e6238c8f1d82179e4c84115d5893132343 2024-09-11T00:03:25.251 INFO:journalctl@ceph.mds.j.smithi012.stdout:Sep 11 00:03:25 smithi012 systemd[1]: Started Ceph mds.j for ff3ad74a-6fcf-11ef-bcea-c7b262605968. 2024-09-11T00:03:25.538 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:03:25 smithi012 ceph-mon[30333]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T00:03:25.538 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:03:25 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:03:25.538 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:03:25 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:03:25.538 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:03:25 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:03:25.538 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:03:25 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-11T00:03:25.538 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:03:25 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-11T00:03:25.538 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:03:25 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:03:25.662 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:25.660+0000 7f6066800640 1 -- 172.21.15.12:0/3182461291 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6060075540 msgr2=0x7f6060075920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:03:25.662 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:25.660+0000 7f6066800640 1 --2- 172.21.15.12:0/3182461291 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6060075540 0x7f6060075920 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f6050009900 tx=0x7f605002a410 comp rx=0 tx=0).stop 2024-09-11T00:03:25.663 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:25.660+0000 7f6066800640 1 -- 172.21.15.12:0/3182461291 shutdown_connections 2024-09-11T00:03:25.663 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:25.660+0000 7f6066800640 1 --2- 172.21.15.12:0/3182461291 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6060113b20 0x7f6060115f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:03:25.663 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:25.660+0000 7f6066800640 1 --2- 172.21.15.12:0/3182461291 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6060075e60 0x7f60601135e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:03:25.663 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:25.660+0000 7f6066800640 1 --2- 172.21.15.12:0/3182461291 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6060075540 0x7f6060075920 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:03:25.663 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:25.660+0000 7f6066800640 1 -- 172.21.15.12:0/3182461291 >> 172.21.15.12:0/3182461291 conn(0x7f60600fe830 msgr2=0x7f6060100c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:03:25.663 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:25.660+0000 7f6066800640 1 -- 172.21.15.12:0/3182461291 shutdown_connections 2024-09-11T00:03:25.663 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:25.660+0000 7f6066800640 1 -- 172.21.15.12:0/3182461291 wait complete. 2024-09-11T00:03:25.665 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:25.661+0000 7f6066800640 1 Processor -- start 2024-09-11T00:03:25.665 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:25.661+0000 7f6066800640 1 -- start start 2024-09-11T00:03:25.666 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:25.662+0000 7f6066800640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6060075540 0x7f606019f900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:03:25.666 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:25.662+0000 7f6066800640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6060075e60 0x7f606019fe40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:03:25.666 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:25.662+0000 7f6066800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6060113b20 0x7f60601a51e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:03:25.666 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:25.662+0000 7f6066800640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f60601193b0 con 0x7f6060075540 2024-09-11T00:03:25.666 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:25.662+0000 7f6066800640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f6060119230 con 0x7f6060113b20 2024-09-11T00:03:25.666 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:25.662+0000 7f6066800640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f6060119530 con 0x7f6060075e60 2024-09-11T00:03:25.666 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:25.662+0000 7f605f400640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6060075540 0x7f606019f900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:03:25.666 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:25.662+0000 7f605f400640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6060075540 0x7f606019f900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:37286/0 (socket says 172.21.15.12:37286) 2024-09-11T00:03:25.666 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:25.662+0000 7f605f400640 1 -- 172.21.15.12:0/3442895557 learned_addr learned my addr 172.21.15.12:0/3442895557 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:03:25.667 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:25.662+0000 7f6057e00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6060075e60 0x7f606019fe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:03:25.667 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:25.662+0000 7f605fe00640 1 --2- 172.21.15.12:0/3442895557 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6060113b20 0x7f60601a51e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:03:25.667 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:25.663+0000 7f6057e00640 1 -- 172.21.15.12:0/3442895557 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6060113b20 msgr2=0x7f60601a51e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:03:25.667 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:25.663+0000 7f6057e00640 1 --2- 172.21.15.12:0/3442895557 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6060113b20 0x7f60601a51e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:03:25.667 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:25.663+0000 7f6057e00640 1 -- 172.21.15.12:0/3442895557 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6060075540 msgr2=0x7f606019f900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:03:25.667 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:25.663+0000 7f6057e00640 1 --2- 172.21.15.12:0/3442895557 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6060075540 0x7f606019f900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:03:25.667 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:25.663+0000 7f6057e00640 1 -- 172.21.15.12:0/3442895557 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60601a59b0 con 0x7f6060075e60 2024-09-11T00:03:25.667 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:25.663+0000 7f6057e00640 1 --2- 172.21.15.12:0/3442895557 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6060075e60 0x7f606019fe40 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f604800e9f0 tx=0x7f604800eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:03:25.667 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:25.663+0000 7f605cc00640 1 -- 172.21.15.12:0/3442895557 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f604800ccc0 con 0x7f6060075e60 2024-09-11T00:03:25.667 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:25.663+0000 7f605cc00640 1 -- 172.21.15.12:0/3442895557 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f60480108e0 con 0x7f6060075e60 2024-09-11T00:03:25.668 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:25.663+0000 7f6066800640 1 -- 172.21.15.12:0/3442895557 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f60601a5ca0 con 0x7f6060075e60 2024-09-11T00:03:25.668 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:25.663+0000 7f605cc00640 1 -- 172.21.15.12:0/3442895557 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6048018880 con 0x7f6060075e60 2024-09-11T00:03:25.668 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:25.664+0000 7f6066800640 1 -- 172.21.15.12:0/3442895557 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60601ac360 con 0x7f6060075e60 2024-09-11T00:03:25.668 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:25.665+0000 7f6066800640 1 -- 172.21.15.12:0/3442895557 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6060075920 con 0x7f6060075e60 2024-09-11T00:03:25.670 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:25.668+0000 7f605cc00640 1 -- 172.21.15.12:0/3442895557 <== mon.1 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 16) ==== 153250+0+0 (secure 0 0 0) 0x7f6048010430 con 0x7f6060075e60 2024-09-11T00:03:25.672 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:25.670+0000 7f605cc00640 1 --2- 172.21.15.12:0/3442895557 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f60440b52e0 0x7f60440b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:03:25.672 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:25.670+0000 7f605cc00640 1 -- 172.21.15.12:0/3442895557 <== mon.1 v2:172.21.15.178:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 7127+0+0 (secure 0 0 0) 0x7f60480e5af0 con 0x7f6060075e60 2024-09-11T00:03:25.672 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:25.671+0000 7f605f400640 1 --2- 172.21.15.12:0/3442895557 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f60440b52e0 0x7f60440b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:03:25.673 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:25.671+0000 7f605f400640 1 --2- 172.21.15.12:0/3442895557 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f60440b52e0 0x7f60440b77a0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f6050031440 tx=0x7f60500058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:03:25.675 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:25.673+0000 7f605cc00640 1 -- 172.21.15.12:0/3442895557 <== mon.1 v2:172.21.15.178:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f60480ac760 con 0x7f6060075e60 2024-09-11T00:03:25.782 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:03:25 smithi165 ceph-mon[35356]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T00:03:25.782 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:03:25 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:03:25.782 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:03:25 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:03:25.783 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:03:25 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:03:25.783 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:03:25 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-11T00:03:25.783 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:03:25 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-11T00:03:25.783 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:03:25 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:03:25.831 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:03:25 smithi178 ceph-mon[35919]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T00:03:25.831 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:03:25 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:03:25.831 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:03:25 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:03:25.831 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:03:25 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:03:25.831 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:03:25 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-11T00:03:25.831 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:03:25 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-11T00:03:25.831 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:03:25 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:03:25.915 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:25.913+0000 7f6066800640 1 -- 172.21.15.12:0/3442895557 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f60601ac6d0 con 0x7f6060075e60 2024-09-11T00:03:25.915 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:25.914+0000 7f605cc00640 1 -- 172.21.15.12:0/3442895557 <== mon.1 v2:172.21.15.178:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 4 v4) ==== 93+0+3088 (secure 0 0 0) 0x7f60480b1190 con 0x7f6060075e60 2024-09-11T00:03:25.916 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-11T00:03:25.916 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":4,"btime":"2024-09-11T00:03:25:384035+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14577,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6835/447171591","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":447171591},{"type":"v1","addr":"172.21.15.12:6835","nonce":447171591}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24562,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6833/1141636112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":1141636112},{"type":"v1","addr":"172.21.15.178:6833","nonce":1141636112}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24581,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6833/3111980970","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6832","nonce":3111980970},{"type":"v1","addr":"172.21.15.165:6833","nonce":3111980970}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-09-11T00:03:25.916 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 4 2024-09-11T00:03:25.918 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:25.916+0000 7f6066800640 1 -- 172.21.15.12:0/3442895557 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f60440b52e0 msgr2=0x7f60440b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:03:25.918 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:25.916+0000 7f6066800640 1 --2- 172.21.15.12:0/3442895557 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f60440b52e0 0x7f60440b77a0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f6050031440 tx=0x7f60500058d0 comp rx=0 tx=0).stop 2024-09-11T00:03:25.918 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:25.916+0000 7f6066800640 1 -- 172.21.15.12:0/3442895557 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6060075e60 msgr2=0x7f606019fe40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:03:25.918 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:25.916+0000 7f6066800640 1 --2- 172.21.15.12:0/3442895557 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6060075e60 0x7f606019fe40 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f604800e9f0 tx=0x7f604800eeb0 comp rx=0 tx=0).stop 2024-09-11T00:03:25.918 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:25.916+0000 7f6066800640 1 -- 172.21.15.12:0/3442895557 shutdown_connections 2024-09-11T00:03:25.919 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:25.916+0000 7f6066800640 1 --2- 172.21.15.12:0/3442895557 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f60440b52e0 0x7f60440b77a0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:03:25.919 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:25.916+0000 7f6066800640 1 --2- 172.21.15.12:0/3442895557 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6060113b20 0x7f60601a51e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:03:25.919 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:25.916+0000 7f6066800640 1 --2- 172.21.15.12:0/3442895557 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6060075e60 0x7f606019fe40 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:03:25.919 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:25.916+0000 7f6066800640 1 --2- 172.21.15.12:0/3442895557 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6060075540 0x7f606019f900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:03:25.919 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:25.916+0000 7f6066800640 1 -- 172.21.15.12:0/3442895557 >> 172.21.15.12:0/3442895557 conn(0x7f60600fe830 msgr2=0x7f60600ff430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:03:25.919 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:25.916+0000 7f6066800640 1 -- 172.21.15.12:0/3442895557 shutdown_connections 2024-09-11T00:03:25.919 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:25.916+0000 7f6066800640 1 -- 172.21.15.12:0/3442895557 wait complete. 2024-09-11T00:03:26.563 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:03:26 smithi012 ceph-mon[30333]: Deploying daemon mds.d on smithi012 2024-09-11T00:03:26.563 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:03:26 smithi012 ceph-mon[30333]: mds.? [v2:172.21.15.12:6834/447171591,v1:172.21.15.12:6835/447171591] up:boot 2024-09-11T00:03:26.563 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:03:26 smithi012 ceph-mon[30333]: fsmap 3 up:standby 2024-09-11T00:03:26.563 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:03:26 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "j"} : dispatch 2024-09-11T00:03:26.563 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:03:26 smithi012 ceph-mon[30333]: from='client.? 172.21.15.12:0/3442895557' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T00:03:26.782 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:03:26 smithi165 ceph-mon[35356]: Deploying daemon mds.d on smithi012 2024-09-11T00:03:26.782 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:03:26 smithi165 ceph-mon[35356]: mds.? [v2:172.21.15.12:6834/447171591,v1:172.21.15.12:6835/447171591] up:boot 2024-09-11T00:03:26.782 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:03:26 smithi165 ceph-mon[35356]: fsmap 3 up:standby 2024-09-11T00:03:26.782 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:03:26 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "j"} : dispatch 2024-09-11T00:03:26.782 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:03:26 smithi165 ceph-mon[35356]: from='client.? 172.21.15.12:0/3442895557' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T00:03:26.830 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:03:26 smithi178 ceph-mon[35919]: Deploying daemon mds.d on smithi012 2024-09-11T00:03:26.831 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:03:26 smithi178 ceph-mon[35919]: mds.? [v2:172.21.15.12:6834/447171591,v1:172.21.15.12:6835/447171591] up:boot 2024-09-11T00:03:26.831 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:03:26 smithi178 ceph-mon[35919]: fsmap 3 up:standby 2024-09-11T00:03:26.831 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:03:26 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "j"} : dispatch 2024-09-11T00:03:26.831 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:03:26 smithi178 ceph-mon[35919]: from='client.? 172.21.15.12:0/3442895557' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T00:03:27.782 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:03:27 smithi165 ceph-mon[35356]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T00:03:27.812 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:03:27 smithi012 ceph-mon[30333]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T00:03:27.830 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:03:27 smithi178 ceph-mon[35919]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T00:03:28.535 INFO:journalctl@ceph.mds.d.smithi012.stdout:Sep 11 00:03:28 smithi012 systemd[1]: Starting Ceph mds.d for ff3ad74a-6fcf-11ef-bcea-c7b262605968... 2024-09-11T00:03:28.813 INFO:journalctl@ceph.mds.d.smithi012.stdout:Sep 11 00:03:28 smithi012 podman[72144]: 2024-09-11 00:03:28.438636376 +0000 UTC m=+0.027890564 image pull 589dd18248e494ba411dc838aa4ae0e5e569213f08fcd7a2cf22cd3ef7e3c480 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e 2024-09-11T00:03:28.813 INFO:journalctl@ceph.mds.d.smithi012.stdout:Sep 11 00:03:28 smithi012 podman[72144]: 2024-09-11 00:03:28.599772449 +0000 UTC m=+0.189026636 container create 2726b503ecafd1e80d4a0f2918b3ca8dd51ed74472effa1c1f7f5d85c4e86948 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-mds-d, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_CLEAN=True, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS) 2024-09-11T00:03:29.073 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid ff3ad74a-6fcf-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T00:03:29.124 INFO:journalctl@ceph.mds.d.smithi012.stdout:Sep 11 00:03:28 smithi012 podman[72144]: 2024-09-11 00:03:28.869118712 +0000 UTC m=+0.458372899 container init 2726b503ecafd1e80d4a0f2918b3ca8dd51ed74472effa1c1f7f5d85c4e86948 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-mds-d, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, io.buildah.version=1.37.2) 2024-09-11T00:03:29.125 INFO:journalctl@ceph.mds.d.smithi012.stdout:Sep 11 00:03:28 smithi012 podman[72144]: 2024-09-11 00:03:28.875624471 +0000 UTC m=+0.464878658 container start 2726b503ecafd1e80d4a0f2918b3ca8dd51ed74472effa1c1f7f5d85c4e86948 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-mds-d, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, ceph=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_CLEAN=True) 2024-09-11T00:03:29.125 INFO:journalctl@ceph.mds.d.smithi012.stdout:Sep 11 00:03:28 smithi012 ceph-mds[72163]: -- 172.21.15.12:0/587611356 <== mon.0 v2:172.21.15.12:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55ab30ba7860 con 0x55ab30b85000 2024-09-11T00:03:29.125 INFO:journalctl@ceph.mds.d.smithi012.stdout:Sep 11 00:03:28 smithi012 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-mds-d[72159]: starting mds.d at 2024-09-11T00:03:29.125 INFO:journalctl@ceph.mds.d.smithi012.stdout:Sep 11 00:03:28 smithi012 bash[72144]: 2726b503ecafd1e80d4a0f2918b3ca8dd51ed74472effa1c1f7f5d85c4e86948 2024-09-11T00:03:29.125 INFO:journalctl@ceph.mds.d.smithi012.stdout:Sep 11 00:03:28 smithi012 systemd[1]: Started Ceph mds.d for ff3ad74a-6fcf-11ef-bcea-c7b262605968. 2024-09-11T00:03:29.457 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:03:29.740 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:03:29 smithi165 ceph-mon[35356]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T00:03:29.740 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:03:29 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:03:29.741 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:03:29 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:03:29.741 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:03:29 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:03:29.741 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:03:29 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-11T00:03:29.741 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:03:29 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-11T00:03:29.741 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:03:29 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:03:29.762 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:03:29 smithi012 ceph-mon[30333]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T00:03:29.762 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:03:29 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:03:29.763 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:03:29 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:03:29.763 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:03:29 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:03:29.763 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:03:29 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-11T00:03:29.763 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:03:29 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-11T00:03:29.763 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:03:29 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:03:29.830 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:03:29 smithi178 ceph-mon[35919]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T00:03:29.831 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:03:29 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:03:29.831 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:03:29 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:03:29.831 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:03:29 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:03:29.831 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:03:29 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-11T00:03:29.831 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:03:29 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-11T00:03:29.831 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:03:29 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:03:30.003 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:30.001+0000 7f0e96400640 1 -- 172.21.15.12:0/3720556719 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0e90108620 msgr2=0x7f0e90108a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:03:30.003 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:30.001+0000 7f0e96400640 1 --2- 172.21.15.12:0/3720556719 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0e90108620 0x7f0e90108a60 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f0e7c009900 tx=0x7f0e7c02a410 comp rx=0 tx=0).stop 2024-09-11T00:03:30.003 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:30.001+0000 7f0e96400640 1 -- 172.21.15.12:0/3720556719 shutdown_connections 2024-09-11T00:03:30.004 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:30.001+0000 7f0e96400640 1 --2- 172.21.15.12:0/3720556719 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0e90108fa0 0x7f0e9010f7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:03:30.004 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:30.001+0000 7f0e96400640 1 --2- 172.21.15.12:0/3720556719 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0e90108620 0x7f0e90108a60 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:03:30.004 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:30.001+0000 7f0e96400640 1 --2- 172.21.15.12:0/3720556719 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0e90102c10 0x7f0e90102ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:03:30.004 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:30.001+0000 7f0e96400640 1 -- 172.21.15.12:0/3720556719 >> 172.21.15.12:0/3720556719 conn(0x7f0e900fe660 msgr2=0x7f0e90100a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:03:30.004 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:30.001+0000 7f0e96400640 1 -- 172.21.15.12:0/3720556719 shutdown_connections 2024-09-11T00:03:30.004 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:30.001+0000 7f0e96400640 1 -- 172.21.15.12:0/3720556719 wait complete. 2024-09-11T00:03:30.005 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:30.002+0000 7f0e96400640 1 Processor -- start 2024-09-11T00:03:30.005 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:30.002+0000 7f0e96400640 1 -- start start 2024-09-11T00:03:30.005 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:30.003+0000 7f0e96400640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0e90102c10 0x7f0e9019b250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:03:30.005 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:30.003+0000 7f0e96400640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0e90108620 0x7f0e9019b790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:03:30.006 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:30.003+0000 7f0e96400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0e90108fa0 0x7f0e901a0b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:03:30.006 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:30.003+0000 7f0e96400640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f0e90112cb0 con 0x7f0e90102c10 2024-09-11T00:03:30.006 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:30.003+0000 7f0e96400640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f0e90112b30 con 0x7f0e90108fa0 2024-09-11T00:03:30.006 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:30.003+0000 7f0e96400640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f0e90112e30 con 0x7f0e90108620 2024-09-11T00:03:30.006 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:30.003+0000 7f0e95000640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0e90102c10 0x7f0e9019b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:03:30.006 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:30.003+0000 7f0e8fe00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0e90108620 0x7f0e9019b790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:03:30.006 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:30.003+0000 7f0e95000640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0e90102c10 0x7f0e9019b250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:35364/0 (socket says 172.21.15.12:35364) 2024-09-11T00:03:30.006 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:30.003+0000 7f0e95000640 1 -- 172.21.15.12:0/296513250 learned_addr learned my addr 172.21.15.12:0/296513250 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:03:30.006 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:30.003+0000 7f0e95a00640 1 --2- 172.21.15.12:0/296513250 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0e90108fa0 0x7f0e901a0b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:03:30.007 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:30.004+0000 7f0e95000640 1 -- 172.21.15.12:0/296513250 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0e90108620 msgr2=0x7f0e9019b790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:03:30.007 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:30.004+0000 7f0e95000640 1 --2- 172.21.15.12:0/296513250 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0e90108620 0x7f0e9019b790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:03:30.007 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:30.004+0000 7f0e95000640 1 -- 172.21.15.12:0/296513250 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0e90108fa0 msgr2=0x7f0e901a0b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:03:30.007 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:30.004+0000 7f0e95000640 1 --2- 172.21.15.12:0/296513250 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0e90108fa0 0x7f0e901a0b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:03:30.007 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:30.004+0000 7f0e95000640 1 -- 172.21.15.12:0/296513250 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e901a1300 con 0x7f0e90102c10 2024-09-11T00:03:30.007 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:30.004+0000 7f0e95000640 1 --2- 172.21.15.12:0/296513250 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0e90102c10 0x7f0e9019b250 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f0e8400e9f0 tx=0x7f0e8400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:03:30.007 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:30.004+0000 7f0e8d600640 1 -- 172.21.15.12:0/296513250 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e8400cd80 con 0x7f0e90102c10 2024-09-11T00:03:30.008 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:30.004+0000 7f0e8d600640 1 -- 172.21.15.12:0/296513250 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0e840108e0 con 0x7f0e90102c10 2024-09-11T00:03:30.008 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:30.004+0000 7f0e8d600640 1 -- 172.21.15.12:0/296513250 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e84018980 con 0x7f0e90102c10 2024-09-11T00:03:30.008 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:30.004+0000 7f0e96400640 1 -- 172.21.15.12:0/296513250 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0e901a15f0 con 0x7f0e90102c10 2024-09-11T00:03:30.008 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:30.005+0000 7f0e96400640 1 -- 172.21.15.12:0/296513250 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0e901a7cb0 con 0x7f0e90102c10 2024-09-11T00:03:30.008 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:30.006+0000 7f0e96400640 1 -- 172.21.15.12:0/296513250 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0e90102ff0 con 0x7f0e90102c10 2024-09-11T00:03:30.010 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:30.008+0000 7f0e8d600640 1 -- 172.21.15.12:0/296513250 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 16) ==== 153250+0+0 (secure 0 0 0) 0x7f0e84010430 con 0x7f0e90102c10 2024-09-11T00:03:30.011 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:30.009+0000 7f0e8d600640 1 --2- 172.21.15.12:0/296513250 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f0e780b5410 0x7f0e780b78d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:03:30.011 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:30.010+0000 7f0e8d600640 1 -- 172.21.15.12:0/296513250 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 7127+0+0 (secure 0 0 0) 0x7f0e840e5f20 con 0x7f0e90102c10 2024-09-11T00:03:30.012 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:30.010+0000 7f0e8fe00640 1 --2- 172.21.15.12:0/296513250 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f0e780b5410 0x7f0e780b78d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:03:30.012 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:30.010+0000 7f0e8fe00640 1 --2- 172.21.15.12:0/296513250 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f0e780b5410 0x7f0e780b78d0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f0e7c005ab0 tx=0x7f0e7c005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:03:30.013 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:30.011+0000 7f0e8d600640 1 -- 172.21.15.12:0/296513250 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f0e840acb90 con 0x7f0e90102c10 2024-09-11T00:03:30.248 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:30.246+0000 7f0e96400640 1 -- 172.21.15.12:0/296513250 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0e9002e5c0 con 0x7f0e90102c10 2024-09-11T00:03:30.249 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:30.247+0000 7f0e8d600640 1 -- 172.21.15.12:0/296513250 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) ==== 93+0+3910 (secure 0 0 0) 0x7f0e840b15c0 con 0x7f0e90102c10 2024-09-11T00:03:30.250 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-11T00:03:30.250 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":5,"btime":"2024-09-11T00:03:29:402386+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14577,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6835/447171591","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":447171591},{"type":"v1","addr":"172.21.15.12:6835","nonce":447171591}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14589,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/901486040","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":901486040},{"type":"v1","addr":"172.21.15.12:6837","nonce":901486040}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24562,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6833/1141636112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":1141636112},{"type":"v1","addr":"172.21.15.178:6833","nonce":1141636112}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24581,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6833/3111980970","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6832","nonce":3111980970},{"type":"v1","addr":"172.21.15.165:6833","nonce":3111980970}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-09-11T00:03:30.250 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 5 2024-09-11T00:03:30.251 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:30.249+0000 7f0e96400640 1 -- 172.21.15.12:0/296513250 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f0e780b5410 msgr2=0x7f0e780b78d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:03:30.251 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:30.249+0000 7f0e96400640 1 --2- 172.21.15.12:0/296513250 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f0e780b5410 0x7f0e780b78d0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f0e7c005ab0 tx=0x7f0e7c005a20 comp rx=0 tx=0).stop 2024-09-11T00:03:30.251 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:30.249+0000 7f0e96400640 1 -- 172.21.15.12:0/296513250 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0e90102c10 msgr2=0x7f0e9019b250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:03:30.251 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:30.249+0000 7f0e96400640 1 --2- 172.21.15.12:0/296513250 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0e90102c10 0x7f0e9019b250 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f0e8400e9f0 tx=0x7f0e8400eeb0 comp rx=0 tx=0).stop 2024-09-11T00:03:30.252 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:30.249+0000 7f0e96400640 1 -- 172.21.15.12:0/296513250 shutdown_connections 2024-09-11T00:03:30.252 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:30.249+0000 7f0e96400640 1 --2- 172.21.15.12:0/296513250 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f0e780b5410 0x7f0e780b78d0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:03:30.252 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:30.249+0000 7f0e96400640 1 --2- 172.21.15.12:0/296513250 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0e90108fa0 0x7f0e901a0b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:03:30.252 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:30.249+0000 7f0e96400640 1 --2- 172.21.15.12:0/296513250 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0e90108620 0x7f0e9019b790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:03:30.252 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:30.249+0000 7f0e96400640 1 --2- 172.21.15.12:0/296513250 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0e90102c10 0x7f0e9019b250 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:03:30.252 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:30.249+0000 7f0e96400640 1 -- 172.21.15.12:0/296513250 >> 172.21.15.12:0/296513250 conn(0x7f0e900fe660 msgr2=0x7f0e90104bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:03:30.252 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:30.250+0000 7f0e96400640 1 -- 172.21.15.12:0/296513250 shutdown_connections 2024-09-11T00:03:30.252 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:30.250+0000 7f0e96400640 1 -- 172.21.15.12:0/296513250 wait complete. 2024-09-11T00:03:30.562 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:03:30 smithi012 ceph-mon[30333]: Deploying daemon mds.e on smithi165 2024-09-11T00:03:30.563 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:03:30 smithi012 ceph-mon[30333]: mds.? [v2:172.21.15.12:6836/901486040,v1:172.21.15.12:6837/901486040] up:boot 2024-09-11T00:03:30.563 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:03:30 smithi012 ceph-mon[30333]: fsmap 4 up:standby 2024-09-11T00:03:30.563 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:03:30 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "d"} : dispatch 2024-09-11T00:03:30.563 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:03:30 smithi012 ceph-mon[30333]: from='client.? 172.21.15.12:0/296513250' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T00:03:30.712 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:03:30 smithi165 ceph-mon[35356]: Deploying daemon mds.e on smithi165 2024-09-11T00:03:30.712 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:03:30 smithi165 ceph-mon[35356]: mds.? [v2:172.21.15.12:6836/901486040,v1:172.21.15.12:6837/901486040] up:boot 2024-09-11T00:03:30.712 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:03:30 smithi165 ceph-mon[35356]: fsmap 4 up:standby 2024-09-11T00:03:30.712 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:03:30 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "d"} : dispatch 2024-09-11T00:03:30.712 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:03:30 smithi165 ceph-mon[35356]: from='client.? 172.21.15.12:0/296513250' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T00:03:30.830 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:03:30 smithi178 ceph-mon[35919]: Deploying daemon mds.e on smithi165 2024-09-11T00:03:30.830 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:03:30 smithi178 ceph-mon[35919]: mds.? [v2:172.21.15.12:6836/901486040,v1:172.21.15.12:6837/901486040] up:boot 2024-09-11T00:03:30.831 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:03:30 smithi178 ceph-mon[35919]: fsmap 4 up:standby 2024-09-11T00:03:30.831 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:03:30 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "d"} : dispatch 2024-09-11T00:03:30.831 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:03:30 smithi178 ceph-mon[35919]: from='client.? 172.21.15.12:0/296513250' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T00:03:31.420 INFO:journalctl@ceph.mds.e.smithi165.stdout:Sep 11 00:03:31 smithi165 podman[64954]: 2024-09-11 00:03:31.313006108 +0000 UTC m=+0.115937330 container create ab3021c1e997422c7427fda2d2de6609cab63f982c382ea1744265f51e86fcae (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-mds-e, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-11T00:03:31.693 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:03:31 smithi165 ceph-mon[35356]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T00:03:31.694 INFO:journalctl@ceph.mds.e.smithi165.stdout:Sep 11 00:03:31 smithi165 podman[64954]: 2024-09-11 00:03:31.666132179 +0000 UTC m=+0.469063359 container init ab3021c1e997422c7427fda2d2de6609cab63f982c382ea1744265f51e86fcae (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-mds-e, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, ceph=True, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True) 2024-09-11T00:03:31.694 INFO:journalctl@ceph.mds.e.smithi165.stdout:Sep 11 00:03:31 smithi165 podman[64954]: 2024-09-11 00:03:31.669076198 +0000 UTC m=+0.472007382 container start ab3021c1e997422c7427fda2d2de6609cab63f982c382ea1744265f51e86fcae (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-mds-e, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True) 2024-09-11T00:03:31.813 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:03:31 smithi012 ceph-mon[30333]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T00:03:31.830 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:03:31 smithi178 ceph-mon[35919]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T00:03:32.032 INFO:journalctl@ceph.mds.e.smithi165.stdout:Sep 11 00:03:31 smithi165 ceph-mds[64973]: -- 172.21.15.165:0/3516909195 <== mon.2 v2:172.21.15.165:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55da887f7860 con 0x55da887d5400 2024-09-11T00:03:32.032 INFO:journalctl@ceph.mds.e.smithi165.stdout:Sep 11 00:03:31 smithi165 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-mds-e[64969]: starting mds.e at 2024-09-11T00:03:32.032 INFO:journalctl@ceph.mds.e.smithi165.stdout:Sep 11 00:03:31 smithi165 bash[64954]: ab3021c1e997422c7427fda2d2de6609cab63f982c382ea1744265f51e86fcae 2024-09-11T00:03:32.032 INFO:journalctl@ceph.mds.e.smithi165.stdout:Sep 11 00:03:31 smithi165 systemd[1]: Started Ceph mds.e for ff3ad74a-6fcf-11ef-bcea-c7b262605968. 2024-09-11T00:03:32.990 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid ff3ad74a-6fcf-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T00:03:33.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:03:32 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:03:33.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:03:32 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:03:33.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:03:32 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:03:33.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:03:32 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-11T00:03:33.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:03:32 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-11T00:03:33.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:03:32 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:03:33.082 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:03:32 smithi178 ceph-mon[35919]: Deploying daemon mds.f on smithi178 2024-09-11T00:03:33.082 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:03:32 smithi178 ceph-mon[35919]: mds.? [v2:172.21.15.165:6834/2425395274,v1:172.21.15.165:6835/2425395274] up:boot 2024-09-11T00:03:33.082 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:03:32 smithi178 ceph-mon[35919]: fsmap 5 up:standby 2024-09-11T00:03:33.082 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:03:32 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "e"} : dispatch 2024-09-11T00:03:33.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:03:32 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:03:33.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:03:32 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:03:33.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:03:32 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:03:33.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:03:32 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-11T00:03:33.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:03:32 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-11T00:03:33.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:03:32 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:03:33.283 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:03:32 smithi165 ceph-mon[35356]: Deploying daemon mds.f on smithi178 2024-09-11T00:03:33.283 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:03:32 smithi165 ceph-mon[35356]: mds.? [v2:172.21.15.165:6834/2425395274,v1:172.21.15.165:6835/2425395274] up:boot 2024-09-11T00:03:33.283 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:03:32 smithi165 ceph-mon[35356]: fsmap 5 up:standby 2024-09-11T00:03:33.283 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:03:32 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "e"} : dispatch 2024-09-11T00:03:33.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:03:32 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:03:33.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:03:32 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:03:33.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:03:32 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:03:33.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:03:32 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-11T00:03:33.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:03:32 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-11T00:03:33.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:03:32 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:03:33.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:03:32 smithi012 ceph-mon[30333]: Deploying daemon mds.f on smithi178 2024-09-11T00:03:33.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:03:32 smithi012 ceph-mon[30333]: mds.? [v2:172.21.15.165:6834/2425395274,v1:172.21.15.165:6835/2425395274] up:boot 2024-09-11T00:03:33.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:03:32 smithi012 ceph-mon[30333]: fsmap 5 up:standby 2024-09-11T00:03:33.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:03:32 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "e"} : dispatch 2024-09-11T00:03:33.379 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:03:33.831 INFO:journalctl@ceph.mds.f.smithi178.stdout:Sep 11 00:03:33 smithi178 systemd[1]: Starting Ceph mds.f for ff3ad74a-6fcf-11ef-bcea-c7b262605968... 2024-09-11T00:03:33.922 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:33.915+0000 7fc138200640 1 -- 172.21.15.12:0/2397513017 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc1301087d0 msgr2=0x7fc130108c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:03:33.922 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:33.915+0000 7fc138200640 1 --2- 172.21.15.12:0/2397513017 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc1301087d0 0x7fc130108c10 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fc11c009900 tx=0x7fc11c02a410 comp rx=0 tx=0).stop 2024-09-11T00:03:33.922 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:33.915+0000 7fc138200640 1 -- 172.21.15.12:0/2397513017 shutdown_connections 2024-09-11T00:03:33.922 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:33.915+0000 7fc138200640 1 --2- 172.21.15.12:0/2397513017 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc130109150 0x7fc13010f950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:03:33.922 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:33.915+0000 7fc138200640 1 --2- 172.21.15.12:0/2397513017 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc1301087d0 0x7fc130108c10 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:03:33.922 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:33.915+0000 7fc138200640 1 --2- 172.21.15.12:0/2397513017 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc130102dc0 0x7fc1301031a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:03:33.922 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:33.915+0000 7fc138200640 1 -- 172.21.15.12:0/2397513017 >> 172.21.15.12:0/2397513017 conn(0x7fc1300fe830 msgr2=0x7fc130100c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:03:33.922 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:33.915+0000 7fc138200640 1 -- 172.21.15.12:0/2397513017 shutdown_connections 2024-09-11T00:03:33.922 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:33.915+0000 7fc138200640 1 -- 172.21.15.12:0/2397513017 wait complete. 2024-09-11T00:03:33.923 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:33.917+0000 7fc138200640 1 Processor -- start 2024-09-11T00:03:33.923 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:33.917+0000 7fc138200640 1 -- start start 2024-09-11T00:03:33.923 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:33.917+0000 7fc138200640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc130102dc0 0x7fc130199320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:03:33.923 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:33.917+0000 7fc138200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc1301087d0 0x7fc130199860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:03:33.923 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:33.917+0000 7fc138200640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc130109150 0x7fc13019ec00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:03:33.923 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:33.917+0000 7fc138200640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fc130112dd0 con 0x7fc130109150 2024-09-11T00:03:33.923 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:33.917+0000 7fc138200640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fc130112c50 con 0x7fc1301087d0 2024-09-11T00:03:33.923 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:33.917+0000 7fc138200640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fc130112f50 con 0x7fc130102dc0 2024-09-11T00:03:33.923 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:33.918+0000 7fc136200640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc130109150 0x7fc13019ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:03:33.923 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:33.918+0000 7fc135800640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc130102dc0 0x7fc130199320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:03:33.923 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:33.918+0000 7fc135800640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc130102dc0 0x7fc130199320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.12:52636/0 (socket says 172.21.15.12:52636) 2024-09-11T00:03:33.924 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:33.918+0000 7fc134e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc1301087d0 0x7fc130199860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:03:33.924 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:33.918+0000 7fc135800640 1 -- 172.21.15.12:0/881016352 learned_addr learned my addr 172.21.15.12:0/881016352 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:03:33.924 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:33.918+0000 7fc135800640 1 -- 172.21.15.12:0/881016352 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc1301087d0 msgr2=0x7fc130199860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:03:33.924 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:33.918+0000 7fc135800640 1 --2- 172.21.15.12:0/881016352 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc1301087d0 0x7fc130199860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:03:33.924 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:33.918+0000 7fc135800640 1 -- 172.21.15.12:0/881016352 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc130109150 msgr2=0x7fc13019ec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:03:33.924 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:33.918+0000 7fc135800640 1 --2- 172.21.15.12:0/881016352 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc130109150 0x7fc13019ec00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:03:33.924 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:33.918+0000 7fc135800640 1 -- 172.21.15.12:0/881016352 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc13019f3d0 con 0x7fc130102dc0 2024-09-11T00:03:33.924 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:33.918+0000 7fc136200640 1 --2- 172.21.15.12:0/881016352 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc130109150 0x7fc13019ec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:03:33.924 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:33.918+0000 7fc135800640 1 --2- 172.21.15.12:0/881016352 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc130102dc0 0x7fc130199320 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fc12400d950 tx=0x7fc12400de10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:03:33.924 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:33.919+0000 7fc12e000640 1 -- 172.21.15.12:0/881016352 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc124017030 con 0x7fc130102dc0 2024-09-11T00:03:33.924 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:33.919+0000 7fc138200640 1 -- 172.21.15.12:0/881016352 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc13019f6c0 con 0x7fc130102dc0 2024-09-11T00:03:33.924 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:33.919+0000 7fc12e000640 1 -- 172.21.15.12:0/881016352 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc124004020 con 0x7fc130102dc0 2024-09-11T00:03:33.924 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:33.919+0000 7fc12e000640 1 -- 172.21.15.12:0/881016352 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1240047a0 con 0x7fc130102dc0 2024-09-11T00:03:33.924 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:33.919+0000 7fc138200640 1 -- 172.21.15.12:0/881016352 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1301a5d80 con 0x7fc130102dc0 2024-09-11T00:03:33.925 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:33.920+0000 7fc138200640 1 -- 172.21.15.12:0/881016352 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc1301031a0 con 0x7fc130102dc0 2024-09-11T00:03:33.925 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:33.922+0000 7fc12e000640 1 -- 172.21.15.12:0/881016352 <== mon.1 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 16) ==== 153250+0+0 (secure 0 0 0) 0x7fc124009a30 con 0x7fc130102dc0 2024-09-11T00:03:33.925 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:33.923+0000 7fc12e000640 1 --2- 172.21.15.12:0/881016352 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fc1180b52e0 0x7fc1180b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:03:33.925 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:33.923+0000 7fc12e000640 1 -- 172.21.15.12:0/881016352 <== mon.1 v2:172.21.15.178:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 7127+0+0 (secure 0 0 0) 0x7fc1240e5160 con 0x7fc130102dc0 2024-09-11T00:03:33.926 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:33.923+0000 7fc134e00640 1 --2- 172.21.15.12:0/881016352 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fc1180b52e0 0x7fc1180b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:03:33.926 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:33.924+0000 7fc134e00640 1 --2- 172.21.15.12:0/881016352 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fc1180b52e0 0x7fc1180b77a0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fc11c031440 tx=0x7fc11c005b50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:03:33.926 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:33.925+0000 7fc12e000640 1 -- 172.21.15.12:0/881016352 <== mon.1 v2:172.21.15.178:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7fc1240ac420 con 0x7fc130102dc0 2024-09-11T00:03:34.062 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:03:33 smithi012 ceph-mon[30333]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T00:03:34.089 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:03:33 smithi178 ceph-mon[35919]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T00:03:34.090 INFO:journalctl@ceph.mds.f.smithi178.stdout:Sep 11 00:03:33 smithi178 podman[65761]: 2024-09-11 00:03:33.942801667 +0000 UTC m=+0.090912839 container create d1d64697e6990eaeca3e9a3fcdbc1b04051498931e300f9cd173b967494d28ab (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-mds-f, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS) 2024-09-11T00:03:34.090 INFO:journalctl@ceph.mds.f.smithi178.stdout:Sep 11 00:03:33 smithi178 podman[65761]: 2024-09-11 00:03:33.867372978 +0000 UTC m=+0.015484163 image pull 589dd18248e494ba411dc838aa4ae0e5e569213f08fcd7a2cf22cd3ef7e3c480 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e 2024-09-11T00:03:34.161 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:34.159+0000 7fc138200640 1 -- 172.21.15.12:0/881016352 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc13002e5c0 con 0x7fc130102dc0 2024-09-11T00:03:34.162 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:34.160+0000 7fc12e000640 1 -- 172.21.15.12:0/881016352 <== mon.1 v2:172.21.15.178:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) ==== 93+0+4738 (secure 0 0 0) 0x7fc1240abd20 con 0x7fc130102dc0 2024-09-11T00:03:34.163 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-11T00:03:34.163 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":6,"btime":"2024-09-11T00:03:32:419229+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14577,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6835/447171591","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":447171591},{"type":"v1","addr":"172.21.15.12:6835","nonce":447171591}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14589,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/901486040","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":901486040},{"type":"v1","addr":"172.21.15.12:6837","nonce":901486040}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24562,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6833/1141636112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":1141636112},{"type":"v1","addr":"172.21.15.178:6833","nonce":1141636112}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24581,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6833/3111980970","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6832","nonce":3111980970},{"type":"v1","addr":"172.21.15.165:6833","nonce":3111980970}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24614,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6835/2425395274","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":2425395274},{"type":"v1","addr":"172.21.15.165:6835","nonce":2425395274}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[]} 2024-09-11T00:03:34.163 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 6 2024-09-11T00:03:34.165 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:34.163+0000 7fc138200640 1 -- 172.21.15.12:0/881016352 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fc1180b52e0 msgr2=0x7fc1180b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:03:34.165 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:34.163+0000 7fc138200640 1 --2- 172.21.15.12:0/881016352 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fc1180b52e0 0x7fc1180b77a0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fc11c031440 tx=0x7fc11c005b50 comp rx=0 tx=0).stop 2024-09-11T00:03:34.165 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:34.163+0000 7fc138200640 1 -- 172.21.15.12:0/881016352 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc130102dc0 msgr2=0x7fc130199320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:03:34.165 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:34.163+0000 7fc138200640 1 --2- 172.21.15.12:0/881016352 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc130102dc0 0x7fc130199320 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fc12400d950 tx=0x7fc12400de10 comp rx=0 tx=0).stop 2024-09-11T00:03:34.165 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:34.163+0000 7fc138200640 1 -- 172.21.15.12:0/881016352 shutdown_connections 2024-09-11T00:03:34.165 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:34.163+0000 7fc138200640 1 --2- 172.21.15.12:0/881016352 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fc1180b52e0 0x7fc1180b77a0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:03:34.165 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:34.163+0000 7fc138200640 1 --2- 172.21.15.12:0/881016352 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc130109150 0x7fc13019ec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:03:34.165 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:34.163+0000 7fc138200640 1 --2- 172.21.15.12:0/881016352 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc1301087d0 0x7fc130199860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:03:34.165 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:34.163+0000 7fc138200640 1 --2- 172.21.15.12:0/881016352 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc130102dc0 0x7fc130199320 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:03:34.165 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:34.163+0000 7fc138200640 1 -- 172.21.15.12:0/881016352 >> 172.21.15.12:0/881016352 conn(0x7fc1300fe830 msgr2=0x7fc130100c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:03:34.166 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:34.163+0000 7fc138200640 1 -- 172.21.15.12:0/881016352 shutdown_connections 2024-09-11T00:03:34.166 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:34.163+0000 7fc138200640 1 -- 172.21.15.12:0/881016352 wait complete. 2024-09-11T00:03:34.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:03:33 smithi165 ceph-mon[35356]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T00:03:34.581 INFO:journalctl@ceph.mds.f.smithi178.stdout:Sep 11 00:03:34 smithi178 podman[65761]: 2024-09-11 00:03:34.195657825 +0000 UTC m=+0.343769003 container init d1d64697e6990eaeca3e9a3fcdbc1b04051498931e300f9cd173b967494d28ab (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-mds-f, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-09-11T00:03:34.581 INFO:journalctl@ceph.mds.f.smithi178.stdout:Sep 11 00:03:34 smithi178 podman[65761]: 2024-09-11 00:03:34.198380986 +0000 UTC m=+0.346492156 container start d1d64697e6990eaeca3e9a3fcdbc1b04051498931e300f9cd173b967494d28ab (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-mds-f, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240731, GIT_CLEAN=True) 2024-09-11T00:03:34.581 INFO:journalctl@ceph.mds.f.smithi178.stdout:Sep 11 00:03:34 smithi178 ceph-mds[65780]: -- 172.21.15.178:0/2137432121 <== mon.1 v2:172.21.15.178:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x56358fab7860 con 0x56358fa95400 2024-09-11T00:03:34.581 INFO:journalctl@ceph.mds.f.smithi178.stdout:Sep 11 00:03:34 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-mds-f[65776]: starting mds.f at 2024-09-11T00:03:34.581 INFO:journalctl@ceph.mds.f.smithi178.stdout:Sep 11 00:03:34 smithi178 bash[65761]: d1d64697e6990eaeca3e9a3fcdbc1b04051498931e300f9cd173b967494d28ab 2024-09-11T00:03:34.581 INFO:journalctl@ceph.mds.f.smithi178.stdout:Sep 11 00:03:34 smithi178 systemd[1]: Started Ceph mds.f for ff3ad74a-6fcf-11ef-bcea-c7b262605968. 2024-09-11T00:03:35.033 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:03:34 smithi178 ceph-mon[35919]: from='client.? 172.21.15.12:0/881016352' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T00:03:35.033 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:03:34 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:03:35.033 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:03:34 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:03:35.033 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:03:34 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:03:35.033 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:03:34 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-11T00:03:35.034 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:03:34 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-11T00:03:35.034 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:03:34 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:03:35.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:03:34 smithi165 ceph-mon[35356]: from='client.? 172.21.15.12:0/881016352' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T00:03:35.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:03:34 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:03:35.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:03:34 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:03:35.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:03:34 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:03:35.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:03:34 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-11T00:03:35.283 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:03:34 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-11T00:03:35.283 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:03:34 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:03:35.312 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:03:34 smithi012 ceph-mon[30333]: from='client.? 172.21.15.12:0/881016352' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T00:03:35.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:03:34 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:03:35.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:03:34 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:03:35.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:03:34 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:03:35.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:03:34 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-11T00:03:35.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:03:34 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-11T00:03:35.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:03:34 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:03:36.080 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:03:35 smithi178 ceph-mon[35919]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T00:03:36.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:03:35 smithi178 ceph-mon[35919]: Deploying daemon mds.c on smithi178 2024-09-11T00:03:36.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:03:35 smithi178 ceph-mon[35919]: mds.? [v2:172.21.15.178:6834/4084868287,v1:172.21.15.178:6835/4084868287] up:boot 2024-09-11T00:03:36.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:03:35 smithi178 ceph-mon[35919]: fsmap 6 up:standby 2024-09-11T00:03:36.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:03:35 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "f"} : dispatch 2024-09-11T00:03:36.215 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:03:35 smithi012 ceph-mon[30333]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T00:03:36.215 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:03:35 smithi012 ceph-mon[30333]: Deploying daemon mds.c on smithi178 2024-09-11T00:03:36.215 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:03:35 smithi012 ceph-mon[30333]: mds.? [v2:172.21.15.178:6834/4084868287,v1:172.21.15.178:6835/4084868287] up:boot 2024-09-11T00:03:36.216 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:03:35 smithi012 ceph-mon[30333]: fsmap 6 up:standby 2024-09-11T00:03:36.216 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:03:35 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "f"} : dispatch 2024-09-11T00:03:36.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:03:35 smithi165 ceph-mon[35356]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T00:03:36.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:03:35 smithi165 ceph-mon[35356]: Deploying daemon mds.c on smithi178 2024-09-11T00:03:36.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:03:35 smithi165 ceph-mon[35356]: mds.? [v2:172.21.15.178:6834/4084868287,v1:172.21.15.178:6835/4084868287] up:boot 2024-09-11T00:03:36.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:03:35 smithi165 ceph-mon[35356]: fsmap 6 up:standby 2024-09-11T00:03:36.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:03:35 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "f"} : dispatch 2024-09-11T00:03:36.491 INFO:journalctl@ceph.mds.c.smithi178.stdout:Sep 11 00:03:36 smithi178 systemd[1]: Starting Ceph mds.c for ff3ad74a-6fcf-11ef-bcea-c7b262605968... 2024-09-11T00:03:36.770 INFO:journalctl@ceph.mds.c.smithi178.stdout:Sep 11 00:03:36 smithi178 podman[66059]: 2024-09-11 00:03:36.393017011 +0000 UTC m=+0.015778014 image pull 589dd18248e494ba411dc838aa4ae0e5e569213f08fcd7a2cf22cd3ef7e3c480 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e 2024-09-11T00:03:36.770 INFO:journalctl@ceph.mds.c.smithi178.stdout:Sep 11 00:03:36 smithi178 podman[66059]: 2024-09-11 00:03:36.506260112 +0000 UTC m=+0.129021152 container create 93a61ce72254fe725a8e5520275388e03a48417cad7a691c76a277ab16ff0dc6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-mds-c, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, maintainer=Guillaume Abrioux , RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-11T00:03:36.770 INFO:journalctl@ceph.mds.c.smithi178.stdout:Sep 11 00:03:36 smithi178 podman[66059]: 2024-09-11 00:03:36.769574054 +0000 UTC m=+0.392335052 container init 93a61ce72254fe725a8e5520275388e03a48417cad7a691c76a277ab16ff0dc6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-mds-c, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, GIT_CLEAN=True, ceph=True, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-11T00:03:36.882 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid ff3ad74a-6fcf-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T00:03:37.081 INFO:journalctl@ceph.mds.c.smithi178.stdout:Sep 11 00:03:36 smithi178 podman[66059]: 2024-09-11 00:03:36.77238146 +0000 UTC m=+0.395142467 container start 93a61ce72254fe725a8e5520275388e03a48417cad7a691c76a277ab16ff0dc6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-mds-c, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0) 2024-09-11T00:03:37.081 INFO:journalctl@ceph.mds.c.smithi178.stdout:Sep 11 00:03:36 smithi178 ceph-mds[66078]: -- 172.21.15.178:0/3643227290 <== mon.1 v2:172.21.15.178:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5589cabdb860 con 0x5589cabb9400 2024-09-11T00:03:37.081 INFO:journalctl@ceph.mds.c.smithi178.stdout:Sep 11 00:03:36 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-mds-c[66074]: starting mds.c at 2024-09-11T00:03:37.081 INFO:journalctl@ceph.mds.c.smithi178.stdout:Sep 11 00:03:36 smithi178 bash[66059]: 93a61ce72254fe725a8e5520275388e03a48417cad7a691c76a277ab16ff0dc6 2024-09-11T00:03:37.081 INFO:journalctl@ceph.mds.c.smithi178.stdout:Sep 11 00:03:36 smithi178 systemd[1]: Started Ceph mds.c for ff3ad74a-6fcf-11ef-bcea-c7b262605968. 2024-09-11T00:03:37.283 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:03:37.878 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:37.851+0000 7f43a2600640 1 -- 172.21.15.12:0/1181264918 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f439c10d120 msgr2=0x7f439c10f580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:03:37.878 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:37.851+0000 7f43a2600640 1 --2- 172.21.15.12:0/1181264918 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f439c10d120 0x7f439c10f580 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f4394009ca0 tx=0x7f439402a8c0 comp rx=0 tx=0).stop 2024-09-11T00:03:37.878 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:37.852+0000 7f43a2600640 1 -- 172.21.15.12:0/1181264918 shutdown_connections 2024-09-11T00:03:37.878 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:37.852+0000 7f43a2600640 1 --2- 172.21.15.12:0/1181264918 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f439c10d120 0x7f439c10f580 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:03:37.878 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:37.852+0000 7f43a2600640 1 --2- 172.21.15.12:0/1181264918 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f439c1045e0 0x7f439c10cad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:03:37.878 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:37.852+0000 7f43a2600640 1 --2- 172.21.15.12:0/1181264918 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f439c103cc0 0x7f439c1040a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:03:37.879 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:37.852+0000 7f43a2600640 1 -- 172.21.15.12:0/1181264918 >> 172.21.15.12:0/1181264918 conn(0x7f439c0fd700 msgr2=0x7f439c0ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:03:37.879 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:37.852+0000 7f43a2600640 1 -- 172.21.15.12:0/1181264918 shutdown_connections 2024-09-11T00:03:37.879 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:37.852+0000 7f43a2600640 1 -- 172.21.15.12:0/1181264918 wait complete. 2024-09-11T00:03:37.879 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:37.853+0000 7f43a2600640 1 Processor -- start 2024-09-11T00:03:37.879 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:37.853+0000 7f43a2600640 1 -- start start 2024-09-11T00:03:37.879 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:37.854+0000 7f43a2600640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f439c103cc0 0x7f439c1992f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:03:37.879 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:37.854+0000 7f43a2600640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f439c1045e0 0x7f439c199830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:03:37.879 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:37.854+0000 7f43a2600640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f439c10d120 0x7f439c19ebd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:03:37.879 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:37.854+0000 7f43a2600640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f439c112de0 con 0x7f439c10d120 2024-09-11T00:03:37.879 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:37.854+0000 7f43a2600640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f439c112c60 con 0x7f439c103cc0 2024-09-11T00:03:37.879 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:37.854+0000 7f43a2600640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f439c112f60 con 0x7f439c1045e0 2024-09-11T00:03:37.879 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:37.854+0000 7f439b400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f439c103cc0 0x7f439c1992f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:03:37.880 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:37.854+0000 7f439b400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f439c103cc0 0x7f439c1992f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.12:52540/0 (socket says 172.21.15.12:52540) 2024-09-11T00:03:37.880 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:37.854+0000 7f439b400640 1 -- 172.21.15.12:0/1757004945 learned_addr learned my addr 172.21.15.12:0/1757004945 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:03:37.880 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:37.854+0000 7f439be00640 1 --2- 172.21.15.12:0/1757004945 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f439c10d120 0x7f439c19ebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:03:37.880 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:37.854+0000 7f4393e00640 1 --2- 172.21.15.12:0/1757004945 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f439c1045e0 0x7f439c199830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:03:37.880 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:37.854+0000 7f439b400640 1 -- 172.21.15.12:0/1757004945 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f439c1045e0 msgr2=0x7f439c199830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:03:37.880 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:37.854+0000 7f439b400640 1 --2- 172.21.15.12:0/1757004945 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f439c1045e0 0x7f439c199830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:03:37.880 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:37.854+0000 7f439b400640 1 -- 172.21.15.12:0/1757004945 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f439c10d120 msgr2=0x7f439c19ebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:03:37.880 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:37.854+0000 7f439b400640 1 --2- 172.21.15.12:0/1757004945 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f439c10d120 0x7f439c19ebd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:03:37.880 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:37.854+0000 7f439b400640 1 -- 172.21.15.12:0/1757004945 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f439c19f3a0 con 0x7f439c103cc0 2024-09-11T00:03:37.880 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:37.855+0000 7f439b400640 1 --2- 172.21.15.12:0/1757004945 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f439c103cc0 0x7f439c1992f0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f438c00e9f0 tx=0x7f438c00edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:03:37.880 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:37.855+0000 7f4398c00640 1 -- 172.21.15.12:0/1757004945 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f438c00cd80 con 0x7f439c103cc0 2024-09-11T00:03:37.880 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:37.855+0000 7f4398c00640 1 -- 172.21.15.12:0/1757004945 <== mon.2 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f438c0054d0 con 0x7f439c103cc0 2024-09-11T00:03:37.880 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:37.855+0000 7f43a2600640 1 -- 172.21.15.12:0/1757004945 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f439c19f690 con 0x7f439c103cc0 2024-09-11T00:03:37.880 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:37.855+0000 7f4398c00640 1 -- 172.21.15.12:0/1757004945 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f438c010690 con 0x7f439c103cc0 2024-09-11T00:03:37.881 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:37.855+0000 7f43a2600640 1 -- 172.21.15.12:0/1757004945 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f439c1a5d50 con 0x7f439c103cc0 2024-09-11T00:03:37.881 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:37.857+0000 7f43a2600640 1 -- 172.21.15.12:0/1757004945 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f439c1040a0 con 0x7f439c103cc0 2024-09-11T00:03:37.881 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:37.858+0000 7f4398c00640 1 -- 172.21.15.12:0/1757004945 <== mon.2 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 16) ==== 153250+0+0 (secure 0 0 0) 0x7f438c004e90 con 0x7f439c103cc0 2024-09-11T00:03:37.881 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:37.860+0000 7f4398c00640 1 --2- 172.21.15.12:0/1757004945 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f43800b5360 0x7f43800b7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:03:37.881 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:37.860+0000 7f4398c00640 1 -- 172.21.15.12:0/1757004945 <== mon.2 v2:172.21.15.165:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 7127+0+0 (secure 0 0 0) 0x7f438c0e5ce0 con 0x7f439c103cc0 2024-09-11T00:03:37.881 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:37.860+0000 7f4393e00640 1 --2- 172.21.15.12:0/1757004945 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f43800b5360 0x7f43800b7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:03:37.881 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:37.861+0000 7f4393e00640 1 --2- 172.21.15.12:0/1757004945 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f43800b5360 0x7f43800b7820 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f439c0ffe00 tx=0x7f4384005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:03:37.881 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:37.863+0000 7f4398c00640 1 -- 172.21.15.12:0/1757004945 <== mon.2 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f438c0acfa0 con 0x7f439c103cc0 2024-09-11T00:03:37.891 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:03:37 smithi165 ceph-mon[35356]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T00:03:37.891 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:03:37 smithi165 ceph-mon[35356]: mds.? [v2:172.21.15.178:6836/526874763,v1:172.21.15.178:6837/526874763] up:boot 2024-09-11T00:03:37.891 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:03:37 smithi165 ceph-mon[35356]: fsmap 7 up:standby 2024-09-11T00:03:37.891 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:03:37 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "c"} : dispatch 2024-09-11T00:03:37.891 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:03:37 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:03:37.891 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:03:37 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:03:37.891 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:03:37 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:03:37.891 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:03:37 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-11T00:03:37.891 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:03:37 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-11T00:03:37.892 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:03:37 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:03:38.100 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:38.098+0000 7f43a2600640 1 -- 172.21.15.12:0/1757004945 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f439c1a6120 con 0x7f439c103cc0 2024-09-11T00:03:38.100 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:38.099+0000 7f4398c00640 1 -- 172.21.15.12:0/1757004945 <== mon.2 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+6391 (secure 0 0 0) 0x7f438c0ac8a0 con 0x7f439c103cc0 2024-09-11T00:03:38.100 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-11T00:03:38.101 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":8,"btime":"2024-09-11T00:03:36:844626+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14577,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6835/447171591","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":447171591},{"type":"v1","addr":"172.21.15.12:6835","nonce":447171591}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14589,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/901486040","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":901486040},{"type":"v1","addr":"172.21.15.12:6837","nonce":901486040}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24562,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6833/1141636112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":1141636112},{"type":"v1","addr":"172.21.15.178:6833","nonce":1141636112}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24581,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6833/3111980970","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6832","nonce":3111980970},{"type":"v1","addr":"172.21.15.165:6833","nonce":3111980970}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24607,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6835/4084868287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":4084868287},{"type":"v1","addr":"172.21.15.178:6835","nonce":4084868287}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24613,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6837/526874763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":526874763},{"type":"v1","addr":"172.21.15.178:6837","nonce":526874763}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24614,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6835/2425395274","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":2425395274},{"type":"v1","addr":"172.21.15.165:6835","nonce":2425395274}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[]} 2024-09-11T00:03:38.101 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 8 2024-09-11T00:03:38.103 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:38.101+0000 7f43a2600640 1 -- 172.21.15.12:0/1757004945 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f43800b5360 msgr2=0x7f43800b7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:03:38.103 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:38.101+0000 7f43a2600640 1 --2- 172.21.15.12:0/1757004945 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f43800b5360 0x7f43800b7820 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f439c0ffe00 tx=0x7f4384005f90 comp rx=0 tx=0).stop 2024-09-11T00:03:38.103 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:38.102+0000 7f43a2600640 1 -- 172.21.15.12:0/1757004945 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f439c103cc0 msgr2=0x7f439c1992f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:03:38.103 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:38.102+0000 7f43a2600640 1 --2- 172.21.15.12:0/1757004945 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f439c103cc0 0x7f439c1992f0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f438c00e9f0 tx=0x7f438c00edf0 comp rx=0 tx=0).stop 2024-09-11T00:03:38.103 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:38.102+0000 7f43a2600640 1 -- 172.21.15.12:0/1757004945 shutdown_connections 2024-09-11T00:03:38.103 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:38.102+0000 7f43a2600640 1 --2- 172.21.15.12:0/1757004945 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f43800b5360 0x7f43800b7820 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:03:38.103 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:38.102+0000 7f43a2600640 1 --2- 172.21.15.12:0/1757004945 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f439c10d120 0x7f439c19ebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:03:38.103 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:38.102+0000 7f43a2600640 1 --2- 172.21.15.12:0/1757004945 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f439c1045e0 0x7f439c199830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:03:38.103 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:38.102+0000 7f43a2600640 1 --2- 172.21.15.12:0/1757004945 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f439c103cc0 0x7f439c1992f0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:03:38.103 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:38.102+0000 7f43a2600640 1 -- 172.21.15.12:0/1757004945 >> 172.21.15.12:0/1757004945 conn(0x7f439c0fd700 msgr2=0x7f439c10f330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:03:38.103 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:38.102+0000 7f43a2600640 1 -- 172.21.15.12:0/1757004945 shutdown_connections 2024-09-11T00:03:38.103 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:38.102+0000 7f43a2600640 1 -- 172.21.15.12:0/1757004945 wait complete. 2024-09-11T00:03:38.104 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:03:37 smithi012 ceph-mon[30333]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T00:03:38.104 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:03:37 smithi012 ceph-mon[30333]: mds.? [v2:172.21.15.178:6836/526874763,v1:172.21.15.178:6837/526874763] up:boot 2024-09-11T00:03:38.104 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:03:37 smithi012 ceph-mon[30333]: fsmap 7 up:standby 2024-09-11T00:03:38.104 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:03:37 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "c"} : dispatch 2024-09-11T00:03:38.104 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:03:37 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:03:38.104 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:03:37 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:03:38.104 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:03:37 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:03:38.104 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:03:37 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-11T00:03:38.104 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:03:37 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-11T00:03:38.105 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:03:37 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:03:38.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:03:37 smithi178 ceph-mon[35919]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T00:03:38.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:03:37 smithi178 ceph-mon[35919]: mds.? [v2:172.21.15.178:6836/526874763,v1:172.21.15.178:6837/526874763] up:boot 2024-09-11T00:03:38.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:03:37 smithi178 ceph-mon[35919]: fsmap 7 up:standby 2024-09-11T00:03:38.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:03:37 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "c"} : dispatch 2024-09-11T00:03:38.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:03:37 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:03:38.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:03:37 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:03:38.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:03:37 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:03:38.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:03:37 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-11T00:03:38.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:03:37 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-11T00:03:38.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:03:37 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:03:39.201 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:03:38 smithi165 ceph-mon[35356]: Deploying daemon mds.h on smithi165 2024-09-11T00:03:39.201 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:03:38 smithi165 ceph-mon[35356]: from='client.? 172.21.15.12:0/1757004945' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T00:03:39.202 INFO:journalctl@ceph.mds.h.smithi165.stdout:Sep 11 00:03:38 smithi165 systemd[1]: Starting Ceph mds.h for ff3ad74a-6fcf-11ef-bcea-c7b262605968... 2024-09-11T00:03:39.306 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:03:38 smithi178 ceph-mon[35919]: Deploying daemon mds.h on smithi165 2024-09-11T00:03:39.306 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:03:38 smithi178 ceph-mon[35919]: from='client.? 172.21.15.12:0/1757004945' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T00:03:39.312 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:03:38 smithi012 ceph-mon[30333]: Deploying daemon mds.h on smithi165 2024-09-11T00:03:39.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:03:38 smithi012 ceph-mon[30333]: from='client.? 172.21.15.12:0/1757004945' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T00:03:39.487 INFO:journalctl@ceph.mds.h.smithi165.stdout:Sep 11 00:03:39 smithi165 podman[65268]: 2024-09-11 00:03:39.103238984 +0000 UTC m=+0.016060654 image pull 589dd18248e494ba411dc838aa4ae0e5e569213f08fcd7a2cf22cd3ef7e3c480 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e 2024-09-11T00:03:39.488 INFO:journalctl@ceph.mds.h.smithi165.stdout:Sep 11 00:03:39 smithi165 podman[65268]: 2024-09-11 00:03:39.242633197 +0000 UTC m=+0.155454852 container create e73b97112e11ff1d50109bd64bd016310f71e41a86e4dd4dc1c4dcf117726462 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-mds-h, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-11T00:03:39.488 INFO:journalctl@ceph.mds.h.smithi165.stdout:Sep 11 00:03:39 smithi165 podman[65268]: 2024-09-11 00:03:39.48687683 +0000 UTC m=+0.399698489 container init e73b97112e11ff1d50109bd64bd016310f71e41a86e4dd4dc1c4dcf117726462 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-mds-h, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, ceph=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.37.2) 2024-09-11T00:03:39.782 INFO:journalctl@ceph.mds.h.smithi165.stdout:Sep 11 00:03:39 smithi165 podman[65268]: 2024-09-11 00:03:39.489733784 +0000 UTC m=+0.402555451 container start e73b97112e11ff1d50109bd64bd016310f71e41a86e4dd4dc1c4dcf117726462 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-mds-h, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-11T00:03:39.783 INFO:journalctl@ceph.mds.h.smithi165.stdout:Sep 11 00:03:39 smithi165 ceph-mds[65287]: -- 172.21.15.165:0/2062076654 <== mon.2 v2:172.21.15.165:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55bd30b8f860 con 0x55bd30b6d400 2024-09-11T00:03:39.783 INFO:journalctl@ceph.mds.h.smithi165.stdout:Sep 11 00:03:39 smithi165 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-mds-h[65283]: starting mds.h at 2024-09-11T00:03:39.783 INFO:journalctl@ceph.mds.h.smithi165.stdout:Sep 11 00:03:39 smithi165 bash[65268]: e73b97112e11ff1d50109bd64bd016310f71e41a86e4dd4dc1c4dcf117726462 2024-09-11T00:03:39.783 INFO:journalctl@ceph.mds.h.smithi165.stdout:Sep 11 00:03:39 smithi165 systemd[1]: Started Ceph mds.h for ff3ad74a-6fcf-11ef-bcea-c7b262605968. 2024-09-11T00:03:40.161 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:03:39 smithi178 ceph-mon[35919]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T00:03:40.161 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:03:39 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:03:40.161 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:03:39 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:03:40.161 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:03:39 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:03:40.161 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:03:39 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-11T00:03:40.162 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:03:39 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-11T00:03:40.162 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:03:39 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:03:40.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:03:39 smithi165 ceph-mon[35356]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T00:03:40.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:03:39 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:03:40.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:03:39 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:03:40.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:03:39 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:03:40.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:03:39 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-11T00:03:40.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:03:39 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-11T00:03:40.283 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:03:39 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:03:40.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:03:39 smithi012 ceph-mon[30333]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T00:03:40.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:03:39 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:03:40.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:03:39 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:03:40.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:03:39 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:03:40.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:03:39 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-11T00:03:40.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:03:39 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-11T00:03:40.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:03:39 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:03:40.864 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid ff3ad74a-6fcf-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T00:03:41.138 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:03:40 smithi178 ceph-mon[35919]: Deploying daemon mds.i on smithi178 2024-09-11T00:03:41.138 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:03:40 smithi178 ceph-mon[35919]: mds.? [v2:172.21.15.165:6836/6215088,v1:172.21.15.165:6837/6215088] up:boot 2024-09-11T00:03:41.138 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:03:40 smithi178 ceph-mon[35919]: fsmap 8 up:standby 2024-09-11T00:03:41.138 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:03:40 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "h"} : dispatch 2024-09-11T00:03:41.257 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:03:41.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:03:40 smithi165 ceph-mon[35356]: Deploying daemon mds.i on smithi178 2024-09-11T00:03:41.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:03:40 smithi165 ceph-mon[35356]: mds.? [v2:172.21.15.165:6836/6215088,v1:172.21.15.165:6837/6215088] up:boot 2024-09-11T00:03:41.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:03:40 smithi165 ceph-mon[35356]: fsmap 8 up:standby 2024-09-11T00:03:41.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:03:40 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "h"} : dispatch 2024-09-11T00:03:41.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:03:40 smithi012 ceph-mon[30333]: Deploying daemon mds.i on smithi178 2024-09-11T00:03:41.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:03:40 smithi012 ceph-mon[30333]: mds.? [v2:172.21.15.165:6836/6215088,v1:172.21.15.165:6837/6215088] up:boot 2024-09-11T00:03:41.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:03:40 smithi012 ceph-mon[30333]: fsmap 8 up:standby 2024-09-11T00:03:41.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:03:40 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "h"} : dispatch 2024-09-11T00:03:41.831 INFO:journalctl@ceph.mds.i.smithi178.stdout:Sep 11 00:03:41 smithi178 systemd[1]: Starting Ceph mds.i for ff3ad74a-6fcf-11ef-bcea-c7b262605968... 2024-09-11T00:03:41.943 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:03:41 smithi012 ceph-mon[30333]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T00:03:41.958 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:41.957+0000 7f51bb800640 1 -- 172.21.15.12:0/2497331156 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f51b41087b0 msgr2=0x7f51b4108bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:03:41.958 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:41.957+0000 7f51bb800640 1 --2- 172.21.15.12:0/2497331156 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f51b41087b0 0x7f51b4108bf0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f51a0009900 tx=0x7f51a002a410 comp rx=0 tx=0).stop 2024-09-11T00:03:41.958 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:41.957+0000 7f51bb800640 1 -- 172.21.15.12:0/2497331156 shutdown_connections 2024-09-11T00:03:41.959 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:41.957+0000 7f51bb800640 1 --2- 172.21.15.12:0/2497331156 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f51b4109130 0x7f51b410f930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:03:41.959 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:41.957+0000 7f51bb800640 1 --2- 172.21.15.12:0/2497331156 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f51b41087b0 0x7f51b4108bf0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:03:41.959 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:41.957+0000 7f51bb800640 1 --2- 172.21.15.12:0/2497331156 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f51b4102da0 0x7f51b4103180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:03:41.959 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:41.957+0000 7f51bb800640 1 -- 172.21.15.12:0/2497331156 >> 172.21.15.12:0/2497331156 conn(0x7f51b40fe810 msgr2=0x7f51b4100c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:03:41.959 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:41.957+0000 7f51bb800640 1 -- 172.21.15.12:0/2497331156 shutdown_connections 2024-09-11T00:03:41.959 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:41.957+0000 7f51bb800640 1 -- 172.21.15.12:0/2497331156 wait complete. 2024-09-11T00:03:41.960 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:41.958+0000 7f51bb800640 1 Processor -- start 2024-09-11T00:03:41.960 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:41.958+0000 7f51bb800640 1 -- start start 2024-09-11T00:03:41.960 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:41.958+0000 7f51bb800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f51b4102da0 0x7f51b419b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:03:41.961 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:41.958+0000 7f51bb800640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f51b41087b0 0x7f51b419ba00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:03:41.961 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:41.958+0000 7f51bb800640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f51b4109130 0x7f51b41a0da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:03:41.961 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:41.958+0000 7f51bb800640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f51b4112dd0 con 0x7f51b41087b0 2024-09-11T00:03:41.961 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:41.958+0000 7f51bb800640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f51b4112c50 con 0x7f51b4102da0 2024-09-11T00:03:41.961 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:41.958+0000 7f51bb800640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f51b4112f50 con 0x7f51b4109130 2024-09-11T00:03:41.961 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:41.959+0000 7f51b3e00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f51b41087b0 0x7f51b419ba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:03:41.961 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:41.959+0000 7f51b3e00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f51b41087b0 0x7f51b419ba00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:55530/0 (socket says 172.21.15.12:55530) 2024-09-11T00:03:41.961 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:41.959+0000 7f51b3e00640 1 -- 172.21.15.12:0/479839880 learned_addr learned my addr 172.21.15.12:0/479839880 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:03:41.961 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:41.959+0000 7f51b8e00640 1 --2- 172.21.15.12:0/479839880 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f51b4102da0 0x7f51b419b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:03:41.961 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:41.959+0000 7f51b9800640 1 --2- 172.21.15.12:0/479839880 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f51b4109130 0x7f51b41a0da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:03:41.961 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:41.959+0000 7f51b3e00640 1 -- 172.21.15.12:0/479839880 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f51b4109130 msgr2=0x7f51b41a0da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:03:41.962 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:41.959+0000 7f51b3e00640 1 --2- 172.21.15.12:0/479839880 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f51b4109130 0x7f51b41a0da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:03:41.962 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:41.959+0000 7f51b3e00640 1 -- 172.21.15.12:0/479839880 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f51b4102da0 msgr2=0x7f51b419b4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:03:41.962 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:41.959+0000 7f51b3e00640 1 --2- 172.21.15.12:0/479839880 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f51b4102da0 0x7f51b419b4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:03:41.962 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:41.959+0000 7f51b3e00640 1 -- 172.21.15.12:0/479839880 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51b41a1570 con 0x7f51b41087b0 2024-09-11T00:03:41.962 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:41.959+0000 7f51b8e00640 1 --2- 172.21.15.12:0/479839880 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f51b4102da0 0x7f51b419b4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:03:41.962 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:41.960+0000 7f51b3e00640 1 --2- 172.21.15.12:0/479839880 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f51b41087b0 0x7f51b419ba00 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f51a0005ab0 tx=0x7f51a0003c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:03:41.962 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:41.960+0000 7f51b1600640 1 -- 172.21.15.12:0/479839880 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51a003f070 con 0x7f51b41087b0 2024-09-11T00:03:41.963 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:41.960+0000 7f51b1600640 1 -- 172.21.15.12:0/479839880 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f51a0031d60 con 0x7f51b41087b0 2024-09-11T00:03:41.963 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:41.960+0000 7f51b1600640 1 -- 172.21.15.12:0/479839880 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51a0035030 con 0x7f51b41087b0 2024-09-11T00:03:41.963 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:41.960+0000 7f51bb800640 1 -- 172.21.15.12:0/479839880 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f51b41a1800 con 0x7f51b41087b0 2024-09-11T00:03:41.963 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:41.960+0000 7f51bb800640 1 -- 172.21.15.12:0/479839880 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51b41a7f20 con 0x7f51b41087b0 2024-09-11T00:03:41.963 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:41.961+0000 7f51bb800640 1 -- 172.21.15.12:0/479839880 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f51b4104050 con 0x7f51b41087b0 2024-09-11T00:03:41.964 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:41.964+0000 7f51b1600640 1 -- 172.21.15.12:0/479839880 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 16) ==== 153250+0+0 (secure 0 0 0) 0x7f51a004c060 con 0x7f51b41087b0 2024-09-11T00:03:41.966 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:41.965+0000 7f51b1600640 1 --2- 172.21.15.12:0/479839880 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f51980b5360 0x7f51980b7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:03:41.966 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:41.965+0000 7f51b1600640 1 -- 172.21.15.12:0/479839880 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 7127+0+0 (secure 0 0 0) 0x7f51a01060d0 con 0x7f51b41087b0 2024-09-11T00:03:41.966 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:41.966+0000 7f51b8e00640 1 --2- 172.21.15.12:0/479839880 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f51980b5360 0x7f51980b7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:03:41.967 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:41.966+0000 7f51b8e00640 1 --2- 172.21.15.12:0/479839880 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f51980b5360 0x7f51980b7820 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f51a4005d00 tx=0x7f51a400b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:03:41.968 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:41.967+0000 7f51b1600640 1 -- 172.21.15.12:0/479839880 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f51a00ccd40 con 0x7f51b41087b0 2024-09-11T00:03:42.178 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:03:41 smithi178 ceph-mon[35919]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T00:03:42.178 INFO:journalctl@ceph.mds.i.smithi178.stdout:Sep 11 00:03:41 smithi178 podman[66372]: 2024-09-11 00:03:41.746445122 +0000 UTC m=+0.016064930 image pull 589dd18248e494ba411dc838aa4ae0e5e569213f08fcd7a2cf22cd3ef7e3c480 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e 2024-09-11T00:03:42.178 INFO:journalctl@ceph.mds.i.smithi178.stdout:Sep 11 00:03:41 smithi178 podman[66372]: 2024-09-11 00:03:41.873849711 +0000 UTC m=+0.143469505 container create 99c4f702fdec80f2d032dd2d254d6ce9eaf6c09e0099319e32a4b82697446762 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-mds-i, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, ceph=True) 2024-09-11T00:03:42.206 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:42.205+0000 7f51bb800640 1 -- 172.21.15.12:0/479839880 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f51b4103180 con 0x7f51b41087b0 2024-09-11T00:03:42.207 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:42.206+0000 7f51b1600640 1 -- 172.21.15.12:0/479839880 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+7210 (secure 0 0 0) 0x7f51b4103180 con 0x7f51b41087b0 2024-09-11T00:03:42.207 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-11T00:03:42.207 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":9,"btime":"2024-09-11T00:03:39:857177+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14577,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6835/447171591","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":447171591},{"type":"v1","addr":"172.21.15.12:6835","nonce":447171591}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14589,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/901486040","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":901486040},{"type":"v1","addr":"172.21.15.12:6837","nonce":901486040}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24562,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6833/1141636112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":1141636112},{"type":"v1","addr":"172.21.15.178:6833","nonce":1141636112}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24581,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6833/3111980970","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6832","nonce":3111980970},{"type":"v1","addr":"172.21.15.165:6833","nonce":3111980970}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24607,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6835/4084868287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":4084868287},{"type":"v1","addr":"172.21.15.178:6835","nonce":4084868287}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24613,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6837/526874763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":526874763},{"type":"v1","addr":"172.21.15.178:6837","nonce":526874763}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24614,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6835/2425395274","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":2425395274},{"type":"v1","addr":"172.21.15.165:6835","nonce":2425395274}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24638,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6837/6215088","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6836","nonce":6215088},{"type":"v1","addr":"172.21.15.165:6837","nonce":6215088}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[]} 2024-09-11T00:03:42.207 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 9 2024-09-11T00:03:42.210 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:42.209+0000 7f51bb800640 1 -- 172.21.15.12:0/479839880 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f51980b5360 msgr2=0x7f51980b7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:03:42.210 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:42.209+0000 7f51bb800640 1 --2- 172.21.15.12:0/479839880 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f51980b5360 0x7f51980b7820 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f51a4005d00 tx=0x7f51a400b410 comp rx=0 tx=0).stop 2024-09-11T00:03:42.210 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:42.209+0000 7f51bb800640 1 -- 172.21.15.12:0/479839880 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f51b41087b0 msgr2=0x7f51b419ba00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:03:42.210 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:42.209+0000 7f51bb800640 1 --2- 172.21.15.12:0/479839880 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f51b41087b0 0x7f51b419ba00 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f51a0005ab0 tx=0x7f51a0003c80 comp rx=0 tx=0).stop 2024-09-11T00:03:42.210 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:42.209+0000 7f51bb800640 1 -- 172.21.15.12:0/479839880 shutdown_connections 2024-09-11T00:03:42.210 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:42.209+0000 7f51bb800640 1 --2- 172.21.15.12:0/479839880 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f51980b5360 0x7f51980b7820 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:03:42.210 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:42.209+0000 7f51bb800640 1 --2- 172.21.15.12:0/479839880 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f51b4109130 0x7f51b41a0da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:03:42.210 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:42.209+0000 7f51bb800640 1 --2- 172.21.15.12:0/479839880 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f51b41087b0 0x7f51b419ba00 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:03:42.211 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:42.209+0000 7f51bb800640 1 --2- 172.21.15.12:0/479839880 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f51b4102da0 0x7f51b419b4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:03:42.211 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:42.209+0000 7f51bb800640 1 -- 172.21.15.12:0/479839880 >> 172.21.15.12:0/479839880 conn(0x7f51b40fe810 msgr2=0x7f51b4105d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:03:42.211 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:42.209+0000 7f51bb800640 1 -- 172.21.15.12:0/479839880 shutdown_connections 2024-09-11T00:03:42.211 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:42.209+0000 7f51bb800640 1 -- 172.21.15.12:0/479839880 wait complete. 2024-09-11T00:03:42.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:03:41 smithi165 ceph-mon[35356]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T00:03:42.581 INFO:journalctl@ceph.mds.i.smithi178.stdout:Sep 11 00:03:42 smithi178 podman[66372]: 2024-09-11 00:03:42.177474698 +0000 UTC m=+0.447094493 container init 99c4f702fdec80f2d032dd2d254d6ce9eaf6c09e0099319e32a4b82697446762 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-mds-i, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0) 2024-09-11T00:03:42.581 INFO:journalctl@ceph.mds.i.smithi178.stdout:Sep 11 00:03:42 smithi178 podman[66372]: 2024-09-11 00:03:42.18041455 +0000 UTC m=+0.450034345 container start 99c4f702fdec80f2d032dd2d254d6ce9eaf6c09e0099319e32a4b82697446762 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-mds-i, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240731, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS) 2024-09-11T00:03:42.581 INFO:journalctl@ceph.mds.i.smithi178.stdout:Sep 11 00:03:42 smithi178 ceph-mds[66391]: -- 172.21.15.178:0/1541550070 <== mon.1 v2:172.21.15.178:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x563674ddb860 con 0x563674db8800 2024-09-11T00:03:42.581 INFO:journalctl@ceph.mds.i.smithi178.stdout:Sep 11 00:03:42 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-mds-i[66387]: starting mds.i at 2024-09-11T00:03:42.581 INFO:journalctl@ceph.mds.i.smithi178.stdout:Sep 11 00:03:42 smithi178 bash[66372]: 99c4f702fdec80f2d032dd2d254d6ce9eaf6c09e0099319e32a4b82697446762 2024-09-11T00:03:42.581 INFO:journalctl@ceph.mds.i.smithi178.stdout:Sep 11 00:03:42 smithi178 systemd[1]: Started Ceph mds.i for ff3ad74a-6fcf-11ef-bcea-c7b262605968. 2024-09-11T00:03:43.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:03:42 smithi165 ceph-mon[35356]: from='client.? 172.21.15.12:0/479839880' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T00:03:43.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:03:42 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:03:43.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:03:42 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:03:43.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:03:42 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:03:43.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:03:42 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-11T00:03:43.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:03:42 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-11T00:03:43.283 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:03:42 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:03:43.297 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:03:42 smithi012 ceph-mon[30333]: from='client.? 172.21.15.12:0/479839880' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T00:03:43.298 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:03:42 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:03:43.298 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:03:42 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:03:43.298 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:03:42 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:03:43.298 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:03:42 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-11T00:03:43.298 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:03:42 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-11T00:03:43.298 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:03:42 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:03:43.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:03:42 smithi178 ceph-mon[35919]: from='client.? 172.21.15.12:0/479839880' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T00:03:43.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:03:42 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:03:43.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:03:42 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:03:43.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:03:42 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:03:43.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:03:42 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-11T00:03:43.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:03:42 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-11T00:03:43.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:03:42 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:03:44.270 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:03:43 smithi012 ceph-mon[30333]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T00:03:44.274 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:03:43 smithi012 ceph-mon[30333]: Deploying daemon mds.g on smithi012 2024-09-11T00:03:44.274 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:03:43 smithi012 ceph-mon[30333]: mds.? [v2:172.21.15.178:6838/70628097,v1:172.21.15.178:6839/70628097] up:boot 2024-09-11T00:03:44.274 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:03:43 smithi012 ceph-mon[30333]: fsmap 9 up:standby 2024-09-11T00:03:44.274 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:03:43 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "i"} : dispatch 2024-09-11T00:03:44.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:03:43 smithi165 ceph-mon[35356]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T00:03:44.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:03:43 smithi165 ceph-mon[35356]: Deploying daemon mds.g on smithi012 2024-09-11T00:03:44.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:03:43 smithi165 ceph-mon[35356]: mds.? [v2:172.21.15.178:6838/70628097,v1:172.21.15.178:6839/70628097] up:boot 2024-09-11T00:03:44.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:03:43 smithi165 ceph-mon[35356]: fsmap 9 up:standby 2024-09-11T00:03:44.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:03:43 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "i"} : dispatch 2024-09-11T00:03:44.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:03:43 smithi178 ceph-mon[35919]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T00:03:44.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:03:43 smithi178 ceph-mon[35919]: Deploying daemon mds.g on smithi012 2024-09-11T00:03:44.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:03:43 smithi178 ceph-mon[35919]: mds.? [v2:172.21.15.178:6838/70628097,v1:172.21.15.178:6839/70628097] up:boot 2024-09-11T00:03:44.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:03:43 smithi178 ceph-mon[35919]: fsmap 9 up:standby 2024-09-11T00:03:44.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:03:43 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "i"} : dispatch 2024-09-11T00:03:44.976 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid ff3ad74a-6fcf-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T00:03:45.563 INFO:journalctl@ceph.mds.g.smithi012.stdout:Sep 11 00:03:45 smithi012 podman[72923]: 2024-09-11 00:03:45.524132644 +0000 UTC m=+0.378771653 container start 04ca535a4928c5b8c2fa25f67ceabd5d3b96f951e1826eefefc064e42fbeb938 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-mds-g, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.build-date=20240731) 2024-09-11T00:03:45.744 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:03:45.885 INFO:journalctl@ceph.mds.g.smithi012.stdout:Sep 11 00:03:45 smithi012 ceph-mds[72966]: -- 172.21.15.12:0/3715224396 <== mon.0 v2:172.21.15.12:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x560d1042b860 con 0x560d10409000 2024-09-11T00:03:45.885 INFO:journalctl@ceph.mds.g.smithi012.stdout:Sep 11 00:03:45 smithi012 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-mds-g[72952]: starting mds.g at 2024-09-11T00:03:45.885 INFO:journalctl@ceph.mds.g.smithi012.stdout:Sep 11 00:03:45 smithi012 bash[72923]: 04ca535a4928c5b8c2fa25f67ceabd5d3b96f951e1826eefefc064e42fbeb938 2024-09-11T00:03:45.885 INFO:journalctl@ceph.mds.g.smithi012.stdout:Sep 11 00:03:45 smithi012 systemd[1]: Started Ceph mds.g for ff3ad74a-6fcf-11ef-bcea-c7b262605968. 2024-09-11T00:03:46.195 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:03:45 smithi012 ceph-mon[30333]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T00:03:46.195 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:03:45 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:03:46.195 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:03:45 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:03:46.195 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:03:45 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:03:46.195 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:03:45 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-11T00:03:46.195 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:03:45 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-11T00:03:46.195 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:03:45 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:03:46.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:03:45 smithi165 ceph-mon[35356]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T00:03:46.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:03:45 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:03:46.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:03:45 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:03:46.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:03:45 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:03:46.283 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:03:45 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-11T00:03:46.283 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:03:45 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-11T00:03:46.283 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:03:45 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:03:46.320 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:46.312+0000 7f9d09c00640 1 -- 172.21.15.12:0/4263114515 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9d04102c10 msgr2=0x7f9d04102ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:03:46.320 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:46.312+0000 7f9d09c00640 1 --2- 172.21.15.12:0/4263114515 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9d04102c10 0x7f9d04102ff0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f9cf4009900 tx=0x7f9cf402a3f0 comp rx=0 tx=0).stop 2024-09-11T00:03:46.320 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:46.312+0000 7f9d09c00640 1 -- 172.21.15.12:0/4263114515 shutdown_connections 2024-09-11T00:03:46.320 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:46.312+0000 7f9d09c00640 1 --2- 172.21.15.12:0/4263114515 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9d04108fa0 0x7f9d0410f7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:03:46.320 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:46.312+0000 7f9d09c00640 1 --2- 172.21.15.12:0/4263114515 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9d04108620 0x7f9d04108a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:03:46.321 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:46.312+0000 7f9d09c00640 1 --2- 172.21.15.12:0/4263114515 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9d04102c10 0x7f9d04102ff0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:03:46.321 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:46.312+0000 7f9d09c00640 1 -- 172.21.15.12:0/4263114515 >> 172.21.15.12:0/4263114515 conn(0x7f9d040fe660 msgr2=0x7f9d04100a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:03:46.321 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:46.312+0000 7f9d09c00640 1 -- 172.21.15.12:0/4263114515 shutdown_connections 2024-09-11T00:03:46.321 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:46.312+0000 7f9d09c00640 1 -- 172.21.15.12:0/4263114515 wait complete. 2024-09-11T00:03:46.321 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:46.313+0000 7f9d09c00640 1 Processor -- start 2024-09-11T00:03:46.321 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:46.313+0000 7f9d09c00640 1 -- start start 2024-09-11T00:03:46.321 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:46.314+0000 7f9d09c00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9d04102c10 0x7f9d0419b230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:03:46.321 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:46.314+0000 7f9d09c00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9d04108620 0x7f9d0419b770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:03:46.321 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:46.314+0000 7f9d09c00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9d04108fa0 0x7f9d041a0b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:03:46.321 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:46.314+0000 7f9d09c00640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f9d04112c30 con 0x7f9d04108620 2024-09-11T00:03:46.321 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:46.314+0000 7f9d09c00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f9d04112ab0 con 0x7f9d04108fa0 2024-09-11T00:03:46.321 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:46.314+0000 7f9d09c00640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f9d04112db0 con 0x7f9d04102c10 2024-09-11T00:03:46.322 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:46.314+0000 7f9cfbe00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9d04108620 0x7f9d0419b770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:03:46.322 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:46.314+0000 7f9d09200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9d04108fa0 0x7f9d041a0b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:03:46.322 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:46.314+0000 7f9d03e00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9d04102c10 0x7f9d0419b230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:03:46.322 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:46.314+0000 7f9cfbe00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9d04108620 0x7f9d0419b770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:55550/0 (socket says 172.21.15.12:55550) 2024-09-11T00:03:46.322 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:46.314+0000 7f9d09200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9d04108fa0 0x7f9d041a0b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.12:44488/0 (socket says 172.21.15.12:44488) 2024-09-11T00:03:46.322 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:46.314+0000 7f9d03e00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9d04102c10 0x7f9d0419b230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.12:36122/0 (socket says 172.21.15.12:36122) 2024-09-11T00:03:46.322 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:46.314+0000 7f9cfbe00640 1 -- 172.21.15.12:0/4159162764 learned_addr learned my addr 172.21.15.12:0/4159162764 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:03:46.322 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:46.315+0000 7f9d09200640 1 -- 172.21.15.12:0/4159162764 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9d04102c10 msgr2=0x7f9d0419b230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:03:46.322 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:46.315+0000 7f9d09200640 1 --2- 172.21.15.12:0/4159162764 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9d04102c10 0x7f9d0419b230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:03:46.322 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:46.315+0000 7f9d09200640 1 -- 172.21.15.12:0/4159162764 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9d04108620 msgr2=0x7f9d0419b770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:03:46.322 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:46.315+0000 7f9d09200640 1 --2- 172.21.15.12:0/4159162764 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9d04108620 0x7f9d0419b770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:03:46.322 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:46.315+0000 7f9d09200640 1 -- 172.21.15.12:0/4159162764 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d041a12e0 con 0x7f9d04108fa0 2024-09-11T00:03:46.322 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:46.315+0000 7f9d09200640 1 --2- 172.21.15.12:0/4159162764 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9d04108fa0 0x7f9d041a0b10 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f9cfc00e940 tx=0x7f9cfc00ee00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:03:46.323 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:46.315+0000 7f9d01600640 1 -- 172.21.15.12:0/4159162764 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cfc004d20 con 0x7f9d04108fa0 2024-09-11T00:03:46.323 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:46.315+0000 7f9d09c00640 1 -- 172.21.15.12:0/4159162764 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9d041a15d0 con 0x7f9d04108fa0 2024-09-11T00:03:46.323 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:46.315+0000 7f9d01600640 1 -- 172.21.15.12:0/4159162764 <== mon.2 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9cfc005020 con 0x7f9d04108fa0 2024-09-11T00:03:46.323 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:46.315+0000 7f9d01600640 1 -- 172.21.15.12:0/4159162764 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cfc0116d0 con 0x7f9d04108fa0 2024-09-11T00:03:46.323 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:46.315+0000 7f9d09c00640 1 -- 172.21.15.12:0/4159162764 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9d041a7c90 con 0x7f9d04108fa0 2024-09-11T00:03:46.323 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:46.316+0000 7f9d09c00640 1 -- 172.21.15.12:0/4159162764 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9d04102ff0 con 0x7f9d04108fa0 2024-09-11T00:03:46.323 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:46.319+0000 7f9d01600640 1 -- 172.21.15.12:0/4159162764 <== mon.2 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 16) ==== 153250+0+0 (secure 0 0 0) 0x7f9cfc00f040 con 0x7f9d04108fa0 2024-09-11T00:03:46.324 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:46.320+0000 7f9d01600640 1 --2- 172.21.15.12:0/4159162764 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f9ce80b5410 0x7f9ce80b78d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:03:46.324 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:46.320+0000 7f9d01600640 1 -- 172.21.15.12:0/4159162764 <== mon.2 v2:172.21.15.165:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 7127+0+0 (secure 0 0 0) 0x7f9cfc0e5470 con 0x7f9d04108fa0 2024-09-11T00:03:46.324 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:46.321+0000 7f9d03e00640 1 --2- 172.21.15.12:0/4159162764 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f9ce80b5410 0x7f9ce80b78d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:03:46.324 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:46.321+0000 7f9d03e00640 1 --2- 172.21.15.12:0/4159162764 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f9ce80b5410 0x7f9ce80b78d0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f9cf4031440 tx=0x7f9cf4005b50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:03:46.324 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:46.322+0000 7f9d01600640 1 -- 172.21.15.12:0/4159162764 <== mon.2 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f9cfc016030 con 0x7f9d04108fa0 2024-09-11T00:03:46.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:03:45 smithi178 ceph-mon[35919]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T00:03:46.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:03:45 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:03:46.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:03:45 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:03:46.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:03:45 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:03:46.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:03:45 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-11T00:03:46.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:03:45 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-11T00:03:46.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:03:45 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:03:46.565 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:46.564+0000 7f9d09c00640 1 -- 172.21.15.12:0/4159162764 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9d0419c360 con 0x7f9d04108fa0 2024-09-11T00:03:46.565 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:46.565+0000 7f9d01600640 1 -- 172.21.15.12:0/4159162764 <== mon.2 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) ==== 94+0+8860 (secure 0 0 0) 0x7f9cfc0ac160 con 0x7f9d04108fa0 2024-09-11T00:03:46.566 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-11T00:03:46.566 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":11,"btime":"2024-09-11T00:03:45:883096+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14577,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6835/447171591","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":447171591},{"type":"v1","addr":"172.21.15.12:6835","nonce":447171591}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14589,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/901486040","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":901486040},{"type":"v1","addr":"172.21.15.12:6837","nonce":901486040}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24562,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6833/1141636112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":1141636112},{"type":"v1","addr":"172.21.15.178:6833","nonce":1141636112}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24581,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6833/3111980970","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6832","nonce":3111980970},{"type":"v1","addr":"172.21.15.165:6833","nonce":3111980970}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24607,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6835/4084868287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":4084868287},{"type":"v1","addr":"172.21.15.178:6835","nonce":4084868287}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24613,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6837/526874763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":526874763},{"type":"v1","addr":"172.21.15.178:6837","nonce":526874763}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24614,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6835/2425395274","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":2425395274},{"type":"v1","addr":"172.21.15.165:6835","nonce":2425395274}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24637,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6839/70628097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6838","nonce":70628097},{"type":"v1","addr":"172.21.15.178:6839","nonce":70628097}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24638,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6837/6215088","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6836","nonce":6215088},{"type":"v1","addr":"172.21.15.165:6837","nonce":6215088}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24643,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6839/3124093009","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6838","nonce":3124093009},{"type":"v1","addr":"172.21.15.12:6839","nonce":3124093009}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[]} 2024-09-11T00:03:46.566 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 11 2024-09-11T00:03:46.568 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:46.567+0000 7f9d09c00640 1 -- 172.21.15.12:0/4159162764 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f9ce80b5410 msgr2=0x7f9ce80b78d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:03:46.568 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:46.567+0000 7f9d09c00640 1 --2- 172.21.15.12:0/4159162764 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f9ce80b5410 0x7f9ce80b78d0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f9cf4031440 tx=0x7f9cf4005b50 comp rx=0 tx=0).stop 2024-09-11T00:03:46.569 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:46.567+0000 7f9d09c00640 1 -- 172.21.15.12:0/4159162764 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9d04108fa0 msgr2=0x7f9d041a0b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:03:46.569 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:46.567+0000 7f9d09c00640 1 --2- 172.21.15.12:0/4159162764 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9d04108fa0 0x7f9d041a0b10 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f9cfc00e940 tx=0x7f9cfc00ee00 comp rx=0 tx=0).stop 2024-09-11T00:03:46.569 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:46.567+0000 7f9d09c00640 1 -- 172.21.15.12:0/4159162764 shutdown_connections 2024-09-11T00:03:46.569 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:46.567+0000 7f9d09c00640 1 --2- 172.21.15.12:0/4159162764 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f9ce80b5410 0x7f9ce80b78d0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:03:46.569 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:46.567+0000 7f9d09c00640 1 --2- 172.21.15.12:0/4159162764 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9d04108fa0 0x7f9d041a0b10 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:03:46.570 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:46.567+0000 7f9d09c00640 1 --2- 172.21.15.12:0/4159162764 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9d04108620 0x7f9d0419b770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:03:46.570 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:46.567+0000 7f9d09c00640 1 --2- 172.21.15.12:0/4159162764 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9d04102c10 0x7f9d0419b230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:03:46.570 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:46.567+0000 7f9d09c00640 1 -- 172.21.15.12:0/4159162764 >> 172.21.15.12:0/4159162764 conn(0x7f9d040fe660 msgr2=0x7f9d04104bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:03:46.570 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:46.568+0000 7f9d09c00640 1 -- 172.21.15.12:0/4159162764 shutdown_connections 2024-09-11T00:03:46.570 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:46.568+0000 7f9d09c00640 1 -- 172.21.15.12:0/4159162764 wait complete. 2024-09-11T00:03:47.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:03:46 smithi165 ceph-mon[35356]: Deploying daemon mds.a on smithi012 2024-09-11T00:03:47.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:03:46 smithi165 ceph-mon[35356]: mds.? [v2:172.21.15.12:6838/3124093009,v1:172.21.15.12:6839/3124093009] up:boot 2024-09-11T00:03:47.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:03:46 smithi165 ceph-mon[35356]: fsmap 10 up:standby 2024-09-11T00:03:47.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:03:46 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "g"} : dispatch 2024-09-11T00:03:47.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:03:46 smithi165 ceph-mon[35356]: from='client.? 172.21.15.12:0/4159162764' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T00:03:47.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:03:46 smithi012 ceph-mon[30333]: Deploying daemon mds.a on smithi012 2024-09-11T00:03:47.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:03:46 smithi012 ceph-mon[30333]: mds.? [v2:172.21.15.12:6838/3124093009,v1:172.21.15.12:6839/3124093009] up:boot 2024-09-11T00:03:47.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:03:46 smithi012 ceph-mon[30333]: fsmap 10 up:standby 2024-09-11T00:03:47.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:03:46 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "g"} : dispatch 2024-09-11T00:03:47.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:03:46 smithi012 ceph-mon[30333]: from='client.? 172.21.15.12:0/4159162764' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T00:03:47.330 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:03:46 smithi178 ceph-mon[35919]: Deploying daemon mds.a on smithi012 2024-09-11T00:03:47.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:03:46 smithi178 ceph-mon[35919]: mds.? [v2:172.21.15.12:6838/3124093009,v1:172.21.15.12:6839/3124093009] up:boot 2024-09-11T00:03:47.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:03:46 smithi178 ceph-mon[35919]: fsmap 10 up:standby 2024-09-11T00:03:47.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:03:46 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "g"} : dispatch 2024-09-11T00:03:47.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:03:46 smithi178 ceph-mon[35919]: from='client.? 172.21.15.12:0/4159162764' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T00:03:48.062 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:03:47 smithi012 ceph-mon[30333]: pgmap v283: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T00:03:48.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:03:47 smithi165 ceph-mon[35356]: pgmap v283: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T00:03:48.330 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:03:47 smithi178 ceph-mon[35919]: pgmap v283: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T00:03:48.813 INFO:journalctl@ceph.mds.a.smithi012.stdout:Sep 11 00:03:48 smithi012 systemd[1]: Starting Ceph mds.a for ff3ad74a-6fcf-11ef-bcea-c7b262605968... 2024-09-11T00:03:49.080 INFO:journalctl@ceph.mds.a.smithi012.stdout:Sep 11 00:03:48 smithi012 podman[73320]: 2024-09-11 00:03:48.719755829 +0000 UTC m=+0.027067532 image pull 589dd18248e494ba411dc838aa4ae0e5e569213f08fcd7a2cf22cd3ef7e3c480 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e 2024-09-11T00:03:49.080 INFO:journalctl@ceph.mds.a.smithi012.stdout:Sep 11 00:03:48 smithi012 podman[73320]: 2024-09-11 00:03:48.84310635 +0000 UTC m=+0.150418038 container create 8e26bf6033aa9b348aab11dc982055ef7790ff8d53bfd6ae3ab93e12da1f6f63 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-mds-a, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-11T00:03:49.080 INFO:journalctl@ceph.mds.a.smithi012.stdout:Sep 11 00:03:49 smithi012 podman[73320]: 2024-09-11 00:03:49.079059617 +0000 UTC m=+0.386371315 container init 8e26bf6033aa9b348aab11dc982055ef7790ff8d53bfd6ae3ab93e12da1f6f63 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-mds-a, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, ceph=True) 2024-09-11T00:03:49.563 INFO:journalctl@ceph.mds.a.smithi012.stdout:Sep 11 00:03:49 smithi012 podman[73320]: 2024-09-11 00:03:49.086327312 +0000 UTC m=+0.393638993 container start 8e26bf6033aa9b348aab11dc982055ef7790ff8d53bfd6ae3ab93e12da1f6f63 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-mds-a, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.vendor=CentOS, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, ceph=True, org.label-schema.schema-version=1.0) 2024-09-11T00:03:49.563 INFO:journalctl@ceph.mds.a.smithi012.stdout:Sep 11 00:03:49 smithi012 ceph-mds[73339]: -- 172.21.15.12:0/3819640562 <== mon.0 v2:172.21.15.12:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x558657a8b860 con 0x558657a69400 2024-09-11T00:03:49.563 INFO:journalctl@ceph.mds.a.smithi012.stdout:Sep 11 00:03:49 smithi012 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-mds-a[73335]: starting mds.a at 2024-09-11T00:03:49.563 INFO:journalctl@ceph.mds.a.smithi012.stdout:Sep 11 00:03:49 smithi012 bash[73320]: 8e26bf6033aa9b348aab11dc982055ef7790ff8d53bfd6ae3ab93e12da1f6f63 2024-09-11T00:03:49.563 INFO:journalctl@ceph.mds.a.smithi012.stdout:Sep 11 00:03:49 smithi012 systemd[1]: Started Ceph mds.a for ff3ad74a-6fcf-11ef-bcea-c7b262605968. 2024-09-11T00:03:50.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:03:49 smithi165 ceph-mon[35356]: pgmap v284: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T00:03:50.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:03:49 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:03:50.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:03:49 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:03:50.033 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:03:49 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:03:50.033 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:03:49 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-11T00:03:50.033 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:03:49 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-11T00:03:50.033 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:03:49 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:03:50.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:03:49 smithi012 ceph-mon[30333]: pgmap v284: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T00:03:50.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:03:49 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:03:50.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:03:49 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:03:50.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:03:49 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:03:50.314 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:03:49 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-11T00:03:50.314 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:03:49 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-11T00:03:50.314 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:03:49 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:03:50.329 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid ff3ad74a-6fcf-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T00:03:50.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:03:49 smithi178 ceph-mon[35919]: pgmap v284: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T00:03:50.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:03:49 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:03:50.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:03:49 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:03:50.332 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:03:49 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:03:50.332 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:03:49 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-11T00:03:50.332 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:03:49 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-11T00:03:50.332 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:03:49 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:03:50.726 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:03:51.021 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:03:50 smithi165 ceph-mon[35356]: Deploying daemon mds.b on smithi165 2024-09-11T00:03:51.021 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:03:50 smithi165 ceph-mon[35356]: mds.? [v2:172.21.15.12:6840/3069919437,v1:172.21.15.12:6841/3069919437] up:boot 2024-09-11T00:03:51.022 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:03:50 smithi165 ceph-mon[35356]: fsmap 11 up:standby 2024-09-11T00:03:51.022 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:03:50 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "a"} : dispatch 2024-09-11T00:03:51.133 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:03:50 smithi012 ceph-mon[30333]: Deploying daemon mds.b on smithi165 2024-09-11T00:03:51.133 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:03:50 smithi012 ceph-mon[30333]: mds.? [v2:172.21.15.12:6840/3069919437,v1:172.21.15.12:6841/3069919437] up:boot 2024-09-11T00:03:51.133 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:03:50 smithi012 ceph-mon[30333]: fsmap 11 up:standby 2024-09-11T00:03:51.133 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:03:50 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "a"} : dispatch 2024-09-11T00:03:51.266 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:51.245+0000 7ff104400640 1 -- 172.21.15.12:0/4046049070 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff0fc113b20 msgr2=0x7ff0fc115f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:03:51.266 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:51.245+0000 7ff104400640 1 --2- 172.21.15.12:0/4046049070 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff0fc113b20 0x7ff0fc115f10 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7ff0f4009ca0 tx=0x7ff0f402a8c0 comp rx=0 tx=0).stop 2024-09-11T00:03:51.266 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:51.245+0000 7ff104400640 1 -- 172.21.15.12:0/4046049070 shutdown_connections 2024-09-11T00:03:51.267 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:51.245+0000 7ff104400640 1 --2- 172.21.15.12:0/4046049070 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff0fc113b20 0x7ff0fc115f10 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:03:51.267 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:51.245+0000 7ff104400640 1 --2- 172.21.15.12:0/4046049070 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff0fc075e60 0x7ff0fc1135e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:03:51.267 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:51.245+0000 7ff104400640 1 --2- 172.21.15.12:0/4046049070 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff0fc075540 0x7ff0fc075920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:03:51.267 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:51.245+0000 7ff104400640 1 -- 172.21.15.12:0/4046049070 >> 172.21.15.12:0/4046049070 conn(0x7ff0fc0fe830 msgr2=0x7ff0fc100c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:03:51.267 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:51.245+0000 7ff104400640 1 -- 172.21.15.12:0/4046049070 shutdown_connections 2024-09-11T00:03:51.267 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:51.245+0000 7ff104400640 1 -- 172.21.15.12:0/4046049070 wait complete. 2024-09-11T00:03:51.267 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:51.246+0000 7ff104400640 1 Processor -- start 2024-09-11T00:03:51.267 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:51.246+0000 7ff104400640 1 -- start start 2024-09-11T00:03:51.267 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:51.246+0000 7ff104400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff0fc075540 0x7ff0fc19f920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:03:51.267 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:51.246+0000 7ff104400640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff0fc075e60 0x7ff0fc19fe60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:03:51.267 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:51.246+0000 7ff104400640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff0fc113b20 0x7ff0fc1a5200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:03:51.267 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:51.246+0000 7ff104400640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7ff0fc119390 con 0x7ff0fc075e60 2024-09-11T00:03:51.267 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:51.246+0000 7ff104400640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7ff0fc119210 con 0x7ff0fc075540 2024-09-11T00:03:51.268 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:51.246+0000 7ff104400640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7ff0fc119510 con 0x7ff0fc113b20 2024-09-11T00:03:51.268 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:51.247+0000 7ff101a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff0fc075540 0x7ff0fc19f920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:03:51.268 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:51.247+0000 7ff101a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff0fc075540 0x7ff0fc19f920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.12:34926/0 (socket says 172.21.15.12:34926) 2024-09-11T00:03:51.268 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:51.247+0000 7ff101a00640 1 -- 172.21.15.12:0/1761333277 learned_addr learned my addr 172.21.15.12:0/1761333277 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:03:51.268 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:51.247+0000 7ff101000640 1 --2- 172.21.15.12:0/1761333277 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff0fc075e60 0x7ff0fc19fe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:03:51.268 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:51.247+0000 7ff102400640 1 --2- 172.21.15.12:0/1761333277 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff0fc113b20 0x7ff0fc1a5200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:03:51.268 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:51.247+0000 7ff101a00640 1 -- 172.21.15.12:0/1761333277 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff0fc113b20 msgr2=0x7ff0fc1a5200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:03:51.268 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:51.247+0000 7ff101a00640 1 --2- 172.21.15.12:0/1761333277 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff0fc113b20 0x7ff0fc1a5200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:03:51.268 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:51.247+0000 7ff101a00640 1 -- 172.21.15.12:0/1761333277 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff0fc075e60 msgr2=0x7ff0fc19fe60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:03:51.268 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:51.247+0000 7ff101a00640 1 --2- 172.21.15.12:0/1761333277 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff0fc075e60 0x7ff0fc19fe60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:03:51.268 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:51.247+0000 7ff101a00640 1 -- 172.21.15.12:0/1761333277 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0fc1a59d0 con 0x7ff0fc075540 2024-09-11T00:03:51.268 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:51.247+0000 7ff101a00640 1 --2- 172.21.15.12:0/1761333277 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff0fc075540 0x7ff0fc19f920 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7ff0f000e9f0 tx=0x7ff0f000edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:03:51.269 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:51.247+0000 7ff0fa000640 1 -- 172.21.15.12:0/1761333277 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0f000cd80 con 0x7ff0fc075540 2024-09-11T00:03:51.269 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:51.247+0000 7ff0fa000640 1 -- 172.21.15.12:0/1761333277 <== mon.2 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff0f00054d0 con 0x7ff0fc075540 2024-09-11T00:03:51.269 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:51.247+0000 7ff104400640 1 -- 172.21.15.12:0/1761333277 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff0fc1a5cc0 con 0x7ff0fc075540 2024-09-11T00:03:51.269 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:51.248+0000 7ff0fa000640 1 -- 172.21.15.12:0/1761333277 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0f0010690 con 0x7ff0fc075540 2024-09-11T00:03:51.269 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:51.248+0000 7ff104400640 1 -- 172.21.15.12:0/1761333277 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0fc1ac380 con 0x7ff0fc075540 2024-09-11T00:03:51.269 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:51.249+0000 7ff104400640 1 -- 172.21.15.12:0/1761333277 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff0fc075920 con 0x7ff0fc075540 2024-09-11T00:03:51.269 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:51.251+0000 7ff0fa000640 1 -- 172.21.15.12:0/1761333277 <== mon.2 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 16) ==== 153250+0+0 (secure 0 0 0) 0x7ff0f0004e30 con 0x7ff0fc075540 2024-09-11T00:03:51.269 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:51.252+0000 7ff0fa000640 1 --2- 172.21.15.12:0/1761333277 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7ff0e40b52e0 0x7ff0e40b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:03:51.269 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:51.253+0000 7ff0fa000640 1 -- 172.21.15.12:0/1761333277 <== mon.2 v2:172.21.15.165:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 7127+0+0 (secure 0 0 0) 0x7ff0f00e5e60 con 0x7ff0fc075540 2024-09-11T00:03:51.269 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:51.253+0000 7ff101000640 1 --2- 172.21.15.12:0/1761333277 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7ff0e40b52e0 0x7ff0e40b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:03:51.269 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:51.253+0000 7ff101000640 1 --2- 172.21.15.12:0/1761333277 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7ff0e40b52e0 0x7ff0e40b77a0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7ff0e80095f0 tx=0x7ff0e8005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:03:51.269 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:51.255+0000 7ff0fa000640 1 -- 172.21.15.12:0/1761333277 <== mon.2 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7ff0f0056220 con 0x7ff0fc075540 2024-09-11T00:03:51.330 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:03:50 smithi178 ceph-mon[35919]: Deploying daemon mds.b on smithi165 2024-09-11T00:03:51.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:03:50 smithi178 ceph-mon[35919]: mds.? [v2:172.21.15.12:6840/3069919437,v1:172.21.15.12:6841/3069919437] up:boot 2024-09-11T00:03:51.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:03:50 smithi178 ceph-mon[35919]: fsmap 11 up:standby 2024-09-11T00:03:51.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:03:50 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "a"} : dispatch 2024-09-11T00:03:51.490 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:51.489+0000 7ff104400640 1 -- 172.21.15.12:0/1761333277 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff0fc1ac6e0 con 0x7ff0fc075540 2024-09-11T00:03:51.491 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:51.490+0000 7ff0fa000640 1 -- 172.21.15.12:0/1761333277 <== mon.2 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) ==== 94+0+9686 (secure 0 0 0) 0x7ff0f00ad120 con 0x7ff0fc075540 2024-09-11T00:03:51.491 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-11T00:03:51.491 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":12,"btime":"2024-09-11T00:03:49:902149+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14577,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6835/447171591","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":447171591},{"type":"v1","addr":"172.21.15.12:6835","nonce":447171591}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14589,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/901486040","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":901486040},{"type":"v1","addr":"172.21.15.12:6837","nonce":901486040}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14631,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6841/3069919437","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6840","nonce":3069919437},{"type":"v1","addr":"172.21.15.12:6841","nonce":3069919437}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24562,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6833/1141636112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":1141636112},{"type":"v1","addr":"172.21.15.178:6833","nonce":1141636112}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24581,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6833/3111980970","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6832","nonce":3111980970},{"type":"v1","addr":"172.21.15.165:6833","nonce":3111980970}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24607,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6835/4084868287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":4084868287},{"type":"v1","addr":"172.21.15.178:6835","nonce":4084868287}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24613,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6837/526874763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":526874763},{"type":"v1","addr":"172.21.15.178:6837","nonce":526874763}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24614,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6835/2425395274","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":2425395274},{"type":"v1","addr":"172.21.15.165:6835","nonce":2425395274}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24637,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6839/70628097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6838","nonce":70628097},{"type":"v1","addr":"172.21.15.178:6839","nonce":70628097}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24638,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6837/6215088","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6836","nonce":6215088},{"type":"v1","addr":"172.21.15.165:6837","nonce":6215088}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24643,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6839/3124093009","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6838","nonce":3124093009},{"type":"v1","addr":"172.21.15.12:6839","nonce":3124093009}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[]} 2024-09-11T00:03:51.491 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 12 2024-09-11T00:03:51.493 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:51.492+0000 7ff104400640 1 -- 172.21.15.12:0/1761333277 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7ff0e40b52e0 msgr2=0x7ff0e40b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:03:51.493 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:51.492+0000 7ff104400640 1 --2- 172.21.15.12:0/1761333277 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7ff0e40b52e0 0x7ff0e40b77a0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7ff0e80095f0 tx=0x7ff0e8005f90 comp rx=0 tx=0).stop 2024-09-11T00:03:51.494 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:51.492+0000 7ff104400640 1 -- 172.21.15.12:0/1761333277 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff0fc075540 msgr2=0x7ff0fc19f920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:03:51.494 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:51.492+0000 7ff104400640 1 --2- 172.21.15.12:0/1761333277 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff0fc075540 0x7ff0fc19f920 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7ff0f000e9f0 tx=0x7ff0f000edf0 comp rx=0 tx=0).stop 2024-09-11T00:03:51.494 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:51.492+0000 7ff104400640 1 -- 172.21.15.12:0/1761333277 shutdown_connections 2024-09-11T00:03:51.494 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:51.492+0000 7ff104400640 1 --2- 172.21.15.12:0/1761333277 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7ff0e40b52e0 0x7ff0e40b77a0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:03:51.494 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:51.492+0000 7ff104400640 1 --2- 172.21.15.12:0/1761333277 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff0fc113b20 0x7ff0fc1a5200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:03:51.494 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:51.492+0000 7ff104400640 1 --2- 172.21.15.12:0/1761333277 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff0fc075e60 0x7ff0fc19fe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:03:51.494 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:51.492+0000 7ff104400640 1 --2- 172.21.15.12:0/1761333277 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff0fc075540 0x7ff0fc19f920 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:03:51.494 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:51.492+0000 7ff104400640 1 -- 172.21.15.12:0/1761333277 >> 172.21.15.12:0/1761333277 conn(0x7ff0fc0fe830 msgr2=0x7ff0fc0ff430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:03:51.494 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:51.492+0000 7ff104400640 1 -- 172.21.15.12:0/1761333277 shutdown_connections 2024-09-11T00:03:51.494 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:51.493+0000 7ff104400640 1 -- 172.21.15.12:0/1761333277 wait complete. 2024-09-11T00:03:52.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:03:51 smithi165 ceph-mon[35356]: pgmap v285: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T00:03:52.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:03:51 smithi165 ceph-mon[35356]: from='client.? 172.21.15.12:0/1761333277' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T00:03:52.033 INFO:journalctl@ceph.mds.b.smithi165.stdout:Sep 11 00:03:52 smithi165 systemd[1]: Started Ceph mds.b for ff3ad74a-6fcf-11ef-bcea-c7b262605968. 2024-09-11T00:03:52.062 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:03:51 smithi012 ceph-mon[30333]: pgmap v285: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T00:03:52.063 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:03:51 smithi012 ceph-mon[30333]: from='client.? 172.21.15.12:0/1761333277' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T00:03:52.330 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:03:51 smithi178 ceph-mon[35919]: pgmap v285: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T00:03:52.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:03:51 smithi178 ceph-mon[35919]: from='client.? 172.21.15.12:0/1761333277' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T00:03:53.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:03:53 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:03:53.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:03:53 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:03:53.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:03:53 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:03:53.082 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:03:53 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:03:53.082 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:03:53 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:03:53.082 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:03:53 smithi178 ceph-mon[35919]: pgmap v286: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T00:03:53.082 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:03:53 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T00:03:53.082 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:03:53 smithi178 ceph-mon[35919]: mds.? [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] up:boot 2024-09-11T00:03:53.082 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:03:53 smithi178 ceph-mon[35919]: fsmap 12 up:standby 2024-09-11T00:03:53.082 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:03:53 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "b"} : dispatch 2024-09-11T00:03:53.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:03:53 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:03:53.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:03:53 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:03:53.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:03:53 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:03:53.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:03:53 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:03:53.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:03:53 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:03:53.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:03:53 smithi165 ceph-mon[35356]: pgmap v286: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T00:03:53.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:03:53 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T00:03:53.283 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:03:53 smithi165 ceph-mon[35356]: mds.? [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] up:boot 2024-09-11T00:03:53.283 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:03:53 smithi165 ceph-mon[35356]: fsmap 12 up:standby 2024-09-11T00:03:53.283 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:03:53 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "b"} : dispatch 2024-09-11T00:03:53.312 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:03:53 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:03:53.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:03:53 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:03:53.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:03:53 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:03:53.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:03:53 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:03:53.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:03:53 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:03:53.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:03:53 smithi012 ceph-mon[30333]: pgmap v286: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T00:03:53.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:03:53 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T00:03:53.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:03:53 smithi012 ceph-mon[30333]: mds.? [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] up:boot 2024-09-11T00:03:53.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:03:53 smithi012 ceph-mon[30333]: fsmap 12 up:standby 2024-09-11T00:03:53.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:03:53 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "b"} : dispatch 2024-09-11T00:03:54.125 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid ff3ad74a-6fcf-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T00:03:54.532 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:03:54.784 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:03:54 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:03:54.785 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:03:54 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:03:54.785 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:03:54 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:03:54.785 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:03:54 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:03:54.785 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:03:54 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:03:54.785 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:03:54 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:03:54.785 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:03:54 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:03:54.812 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:03:54 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:03:54.813 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:03:54 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:03:54.813 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:03:54 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:03:54.813 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:03:54 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:03:54.813 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:03:54 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:03:54.813 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:03:54 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:03:54.813 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:03:54 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:03:54.831 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:03:54 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:03:54.831 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:03:54 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:03:54.831 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:03:54 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:03:54.831 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:03:54 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:03:54.831 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:03:54 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:03:54.831 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:03:54 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:03:54.831 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:03:54 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:03:55.084 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:55.083+0000 7fac99a00640 1 -- 172.21.15.12:0/2162121413 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fac94102da0 msgr2=0x7fac94103180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:03:55.084 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:55.083+0000 7fac99a00640 1 --2- 172.21.15.12:0/2162121413 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fac94102da0 0x7fac94103180 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fac84009900 tx=0x7fac8402a410 comp rx=0 tx=0).stop 2024-09-11T00:03:55.085 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:55.083+0000 7fac99a00640 1 -- 172.21.15.12:0/2162121413 shutdown_connections 2024-09-11T00:03:55.085 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:55.083+0000 7fac99a00640 1 --2- 172.21.15.12:0/2162121413 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fac94109130 0x7fac9410f930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:03:55.085 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:55.083+0000 7fac99a00640 1 --2- 172.21.15.12:0/2162121413 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fac941087b0 0x7fac94108bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:03:55.085 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:55.083+0000 7fac99a00640 1 --2- 172.21.15.12:0/2162121413 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fac94102da0 0x7fac94103180 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:03:55.085 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:55.083+0000 7fac99a00640 1 -- 172.21.15.12:0/2162121413 >> 172.21.15.12:0/2162121413 conn(0x7fac940fe810 msgr2=0x7fac94100c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:03:55.085 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:55.083+0000 7fac99a00640 1 -- 172.21.15.12:0/2162121413 shutdown_connections 2024-09-11T00:03:55.085 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:55.083+0000 7fac99a00640 1 -- 172.21.15.12:0/2162121413 wait complete. 2024-09-11T00:03:55.086 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:55.084+0000 7fac99a00640 1 Processor -- start 2024-09-11T00:03:55.086 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:55.084+0000 7fac99a00640 1 -- start start 2024-09-11T00:03:55.086 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:55.084+0000 7fac99a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fac94102da0 0x7fac9419b470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:03:55.086 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:55.084+0000 7fac99a00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fac941087b0 0x7fac9419b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:03:55.087 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:55.084+0000 7fac99a00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fac94109130 0x7fac941a0d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:03:55.087 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:55.085+0000 7fac99a00640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fac94112e40 con 0x7fac94109130 2024-09-11T00:03:55.087 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:55.085+0000 7fac99a00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fac94112cc0 con 0x7fac94102da0 2024-09-11T00:03:55.087 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:55.085+0000 7fac99a00640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fac94112fc0 con 0x7fac941087b0 2024-09-11T00:03:55.087 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:55.085+0000 7fac92a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fac94102da0 0x7fac9419b470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:03:55.087 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:55.085+0000 7fac92a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fac94102da0 0x7fac9419b470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.12:34932/0 (socket says 172.21.15.12:34932) 2024-09-11T00:03:55.087 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:55.085+0000 7fac92a00640 1 -- 172.21.15.12:0/1215941065 learned_addr learned my addr 172.21.15.12:0/1215941065 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:03:55.087 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:55.085+0000 7fac93400640 1 --2- 172.21.15.12:0/1215941065 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fac94109130 0x7fac941a0d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:03:55.087 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:55.085+0000 7fac92000640 1 --2- 172.21.15.12:0/1215941065 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fac941087b0 0x7fac9419b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:03:55.087 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:55.085+0000 7fac92a00640 1 -- 172.21.15.12:0/1215941065 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fac941087b0 msgr2=0x7fac9419b9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:03:55.087 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:55.085+0000 7fac92a00640 1 --2- 172.21.15.12:0/1215941065 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fac941087b0 0x7fac9419b9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:03:55.087 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:55.085+0000 7fac92a00640 1 -- 172.21.15.12:0/1215941065 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fac94109130 msgr2=0x7fac941a0d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:03:55.087 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:55.085+0000 7fac92a00640 1 --2- 172.21.15.12:0/1215941065 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fac94109130 0x7fac941a0d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:03:55.088 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:55.085+0000 7fac92a00640 1 -- 172.21.15.12:0/1215941065 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac941a1520 con 0x7fac94102da0 2024-09-11T00:03:55.088 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:55.086+0000 7fac92a00640 1 --2- 172.21.15.12:0/1215941065 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fac94102da0 0x7fac9419b470 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fac84005ab0 tx=0x7fac84003c80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:03:55.088 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:55.086+0000 7fac8b400640 1 -- 172.21.15.12:0/1215941065 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac8403f070 con 0x7fac94102da0 2024-09-11T00:03:55.089 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:55.086+0000 7fac8b400640 1 -- 172.21.15.12:0/1215941065 <== mon.2 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fac84031d60 con 0x7fac94102da0 2024-09-11T00:03:55.089 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:55.086+0000 7fac99a00640 1 -- 172.21.15.12:0/1215941065 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fac941a17b0 con 0x7fac94102da0 2024-09-11T00:03:55.089 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:55.086+0000 7fac8b400640 1 -- 172.21.15.12:0/1215941065 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac84035030 con 0x7fac94102da0 2024-09-11T00:03:55.089 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:55.086+0000 7fac99a00640 1 -- 172.21.15.12:0/1215941065 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fac941a7ed0 con 0x7fac94102da0 2024-09-11T00:03:55.089 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:55.087+0000 7fac99a00640 1 -- 172.21.15.12:0/1215941065 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fac94103180 con 0x7fac94102da0 2024-09-11T00:03:55.090 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:55.089+0000 7fac8b400640 1 -- 172.21.15.12:0/1215941065 <== mon.2 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 16) ==== 153250+0+0 (secure 0 0 0) 0x7fac8403b460 con 0x7fac94102da0 2024-09-11T00:03:55.092 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:55.091+0000 7fac8b400640 1 --2- 172.21.15.12:0/1215941065 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fac780b5360 0x7fac780b7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:03:55.092 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:55.091+0000 7fac8b400640 1 -- 172.21.15.12:0/1215941065 <== mon.2 v2:172.21.15.165:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 7127+0+0 (secure 0 0 0) 0x7fac84106200 con 0x7fac94102da0 2024-09-11T00:03:55.092 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:55.091+0000 7fac92000640 1 --2- 172.21.15.12:0/1215941065 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fac780b5360 0x7fac780b7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:03:55.092 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:55.092+0000 7fac92000640 1 --2- 172.21.15.12:0/1215941065 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fac780b5360 0x7fac780b7820 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fac800095f0 tx=0x7fac80005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:03:55.095 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:55.094+0000 7fac8b400640 1 -- 172.21.15.12:0/1215941065 <== mon.2 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7fac840cce70 con 0x7fac94102da0 2024-09-11T00:03:55.391 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:55.390+0000 7fac99a00640 1 -- 172.21.15.12:0/1215941065 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fac941a8240 con 0x7fac94102da0 2024-09-11T00:03:55.392 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:55.391+0000 7fac8b400640 1 -- 172.21.15.12:0/1215941065 <== mon.2 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) ==== 94+0+10512 (secure 0 0 0) 0x7fac840d18a0 con 0x7fac94102da0 2024-09-11T00:03:55.392 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-11T00:03:55.392 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":13,"btime":"2024-09-11T00:03:52:916233+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14577,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6835/447171591","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":447171591},{"type":"v1","addr":"172.21.15.12:6835","nonce":447171591}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14589,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/901486040","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":901486040},{"type":"v1","addr":"172.21.15.12:6837","nonce":901486040}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14631,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6841/3069919437","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6840","nonce":3069919437},{"type":"v1","addr":"172.21.15.12:6841","nonce":3069919437}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24562,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6833/1141636112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":1141636112},{"type":"v1","addr":"172.21.15.178:6833","nonce":1141636112}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24581,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6833/3111980970","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6832","nonce":3111980970},{"type":"v1","addr":"172.21.15.165:6833","nonce":3111980970}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24607,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6835/4084868287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":4084868287},{"type":"v1","addr":"172.21.15.178:6835","nonce":4084868287}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24613,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6837/526874763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":526874763},{"type":"v1","addr":"172.21.15.178:6837","nonce":526874763}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24614,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6835/2425395274","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":2425395274},{"type":"v1","addr":"172.21.15.165:6835","nonce":2425395274}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24637,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6839/70628097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6838","nonce":70628097},{"type":"v1","addr":"172.21.15.178:6839","nonce":70628097}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24638,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6837/6215088","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6836","nonce":6215088},{"type":"v1","addr":"172.21.15.165:6837","nonce":6215088}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24643,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6839/3124093009","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6838","nonce":3124093009},{"type":"v1","addr":"172.21.15.12:6839","nonce":3124093009}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24674,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6839/501312984","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6838","nonce":501312984},{"type":"v1","addr":"172.21.15.165:6839","nonce":501312984}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[]} 2024-09-11T00:03:55.393 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 13 2024-09-11T00:03:55.395 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:55.394+0000 7fac99a00640 1 -- 172.21.15.12:0/1215941065 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fac780b5360 msgr2=0x7fac780b7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:03:55.395 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:55.394+0000 7fac99a00640 1 --2- 172.21.15.12:0/1215941065 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fac780b5360 0x7fac780b7820 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fac800095f0 tx=0x7fac80005f90 comp rx=0 tx=0).stop 2024-09-11T00:03:55.395 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:55.394+0000 7fac99a00640 1 -- 172.21.15.12:0/1215941065 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fac94102da0 msgr2=0x7fac9419b470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:03:55.395 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:55.394+0000 7fac99a00640 1 --2- 172.21.15.12:0/1215941065 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fac94102da0 0x7fac9419b470 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fac84005ab0 tx=0x7fac84003c80 comp rx=0 tx=0).stop 2024-09-11T00:03:55.395 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:55.394+0000 7fac99a00640 1 -- 172.21.15.12:0/1215941065 shutdown_connections 2024-09-11T00:03:55.395 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:55.394+0000 7fac99a00640 1 --2- 172.21.15.12:0/1215941065 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fac780b5360 0x7fac780b7820 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:03:55.395 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:55.394+0000 7fac99a00640 1 --2- 172.21.15.12:0/1215941065 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fac94109130 0x7fac941a0d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:03:55.396 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:55.394+0000 7fac99a00640 1 --2- 172.21.15.12:0/1215941065 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fac941087b0 0x7fac9419b9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:03:55.396 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:55.394+0000 7fac99a00640 1 --2- 172.21.15.12:0/1215941065 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fac94102da0 0x7fac9419b470 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:03:55.396 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:55.394+0000 7fac99a00640 1 -- 172.21.15.12:0/1215941065 >> 172.21.15.12:0/1215941065 conn(0x7fac940fe810 msgr2=0x7fac940ffe90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:03:55.396 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:55.395+0000 7fac99a00640 1 -- 172.21.15.12:0/1215941065 shutdown_connections 2024-09-11T00:03:55.396 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:55.395+0000 7fac99a00640 1 -- 172.21.15.12:0/1215941065 wait complete. 2024-09-11T00:03:55.992 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:03:55 smithi165 ceph-mon[35356]: pgmap v287: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T00:03:55.992 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:03:55 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:03:55.992 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:03:55 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T00:03:55.993 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:03:55 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:03:55.993 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:03:55 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T00:03:55.993 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:03:55 smithi165 ceph-mon[35356]: from='client.? 172.21.15.12:0/1215941065' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T00:03:56.062 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:03:55 smithi012 ceph-mon[30333]: pgmap v287: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T00:03:56.063 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:03:55 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:03:56.063 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:03:55 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T00:03:56.063 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:03:55 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:03:56.063 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:03:55 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T00:03:56.063 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:03:55 smithi012 ceph-mon[30333]: from='client.? 172.21.15.12:0/1215941065' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T00:03:56.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:03:55 smithi178 ceph-mon[35919]: pgmap v287: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T00:03:56.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:03:55 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:03:56.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:03:55 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T00:03:56.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:03:55 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:03:56.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:03:55 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T00:03:56.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:03:55 smithi178 ceph-mon[35919]: from='client.? 172.21.15.12:0/1215941065' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T00:03:56.161 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid ff3ad74a-6fcf-11ef-bcea-c7b262605968 -- ceph --format=json-pretty osd lspools 2024-09-11T00:03:57.221 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:03:57.952 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:03:57 smithi012 ceph-mon[30333]: pgmap v288: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T00:03:57.952 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:03:57 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:03:57.952 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:03:57 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:03:57.952 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:03:57 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:03:57.952 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:03:57 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:03:57.965 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:57.950+0000 7f46abe00640 1 -- 172.21.15.12:0/3753457598 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f46a4109150 msgr2=0x7f46a410f950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:03:57.965 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:57.950+0000 7f46abe00640 1 --2- 172.21.15.12:0/3753457598 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f46a4109150 0x7f46a410f950 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f469c009ca0 tx=0x7f469c02a8c0 comp rx=0 tx=0).stop 2024-09-11T00:03:57.965 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:57.950+0000 7f46abe00640 1 -- 172.21.15.12:0/3753457598 shutdown_connections 2024-09-11T00:03:57.966 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:57.950+0000 7f46abe00640 1 --2- 172.21.15.12:0/3753457598 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f46a4109150 0x7f46a410f950 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:03:57.966 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:57.950+0000 7f46abe00640 1 --2- 172.21.15.12:0/3753457598 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f46a41087d0 0x7f46a4108c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:03:57.966 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:57.950+0000 7f46abe00640 1 --2- 172.21.15.12:0/3753457598 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f46a4102dc0 0x7f46a41031a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:03:57.966 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:57.950+0000 7f46abe00640 1 -- 172.21.15.12:0/3753457598 >> 172.21.15.12:0/3753457598 conn(0x7f46a40fe830 msgr2=0x7f46a4100c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:03:57.966 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:57.950+0000 7f46abe00640 1 -- 172.21.15.12:0/3753457598 shutdown_connections 2024-09-11T00:03:57.966 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:57.950+0000 7f46abe00640 1 -- 172.21.15.12:0/3753457598 wait complete. 2024-09-11T00:03:57.966 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:57.951+0000 7f46abe00640 1 Processor -- start 2024-09-11T00:03:57.966 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:57.951+0000 7f46abe00640 1 -- start start 2024-09-11T00:03:57.966 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:57.952+0000 7f46abe00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f46a4102dc0 0x7f46a419b4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:03:57.966 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:57.952+0000 7f46abe00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f46a41087d0 0x7f46a419ba10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:03:57.966 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:57.952+0000 7f46abe00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f46a4109150 0x7f46a41a0db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:03:57.966 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:57.952+0000 7f46abe00640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f46a4112e60 con 0x7f46a4102dc0 2024-09-11T00:03:57.966 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:57.952+0000 7f46abe00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f46a4112ce0 con 0x7f46a4109150 2024-09-11T00:03:57.967 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:57.952+0000 7f46abe00640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f46a4112fe0 con 0x7f46a41087d0 2024-09-11T00:03:57.967 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:57.952+0000 7f46a9400640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f46a4102dc0 0x7f46a419b4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:03:57.967 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:57.952+0000 7f46a9400640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f46a4102dc0 0x7f46a419b4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:53070/0 (socket says 172.21.15.12:53070) 2024-09-11T00:03:57.967 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:57.952+0000 7f46a9400640 1 -- 172.21.15.12:0/3468017823 learned_addr learned my addr 172.21.15.12:0/3468017823 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:03:57.967 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:57.952+0000 7f46a8a00640 1 --2- 172.21.15.12:0/3468017823 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f46a41087d0 0x7f46a419ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:03:57.967 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:57.952+0000 7f46a9e00640 1 --2- 172.21.15.12:0/3468017823 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f46a4109150 0x7f46a41a0db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:03:57.967 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:57.952+0000 7f46a9400640 1 -- 172.21.15.12:0/3468017823 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f46a41087d0 msgr2=0x7f46a419ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:03:57.967 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:57.952+0000 7f46a9400640 1 --2- 172.21.15.12:0/3468017823 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f46a41087d0 0x7f46a419ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:03:57.967 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:57.953+0000 7f46a9400640 1 -- 172.21.15.12:0/3468017823 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f46a4109150 msgr2=0x7f46a41a0db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:03:57.967 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:57.953+0000 7f46a9400640 1 --2- 172.21.15.12:0/3468017823 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f46a4109150 0x7f46a41a0db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:03:57.967 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:57.953+0000 7f46a9400640 1 -- 172.21.15.12:0/3468017823 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46a41a1580 con 0x7f46a4102dc0 2024-09-11T00:03:57.967 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:57.953+0000 7f46a9400640 1 --2- 172.21.15.12:0/3468017823 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f46a4102dc0 0x7f46a419b4d0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f469800e9f0 tx=0x7f469800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:03:57.968 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:57.953+0000 7f46a2000640 1 -- 172.21.15.12:0/3468017823 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f469800cd80 con 0x7f46a4102dc0 2024-09-11T00:03:57.968 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:57.953+0000 7f46a2000640 1 -- 172.21.15.12:0/3468017823 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f46980054d0 con 0x7f46a4102dc0 2024-09-11T00:03:57.968 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:57.953+0000 7f46abe00640 1 -- 172.21.15.12:0/3468017823 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46a41a1870 con 0x7f46a4102dc0 2024-09-11T00:03:57.968 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:57.953+0000 7f46a2000640 1 -- 172.21.15.12:0/3468017823 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4698010690 con 0x7f46a4102dc0 2024-09-11T00:03:57.968 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:57.953+0000 7f46abe00640 1 -- 172.21.15.12:0/3468017823 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46a41a7f30 con 0x7f46a4102dc0 2024-09-11T00:03:57.968 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:57.955+0000 7f46abe00640 1 -- 172.21.15.12:0/3468017823 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f46a41031a0 con 0x7f46a4102dc0 2024-09-11T00:03:57.968 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:57.957+0000 7f46a2000640 1 -- 172.21.15.12:0/3468017823 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 16) ==== 153250+0+0 (secure 0 0 0) 0x7f4698004e30 con 0x7f46a4102dc0 2024-09-11T00:03:57.968 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:57.958+0000 7f46a2000640 1 --2- 172.21.15.12:0/3468017823 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f468c0b5360 0x7f468c0b7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:03:57.968 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:57.958+0000 7f46a2000640 1 -- 172.21.15.12:0/3468017823 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 7127+0+0 (secure 0 0 0) 0x7f46980e5d30 con 0x7f46a4102dc0 2024-09-11T00:03:57.968 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:57.959+0000 7f46a8a00640 1 --2- 172.21.15.12:0/3468017823 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f468c0b5360 0x7f468c0b7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:03:57.968 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:57.959+0000 7f46a8a00640 1 --2- 172.21.15.12:0/3468017823 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f468c0b5360 0x7f468c0b7820 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f46a41080c0 tx=0x7f469000b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:03:57.968 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:57.960+0000 7f46a2000640 1 -- 172.21.15.12:0/3468017823 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f4698056220 con 0x7f46a4102dc0 2024-09-11T00:03:58.080 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:03:57 smithi178 ceph-mon[35919]: pgmap v288: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T00:03:58.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:03:57 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:03:58.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:03:57 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:03:58.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:03:57 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:03:58.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:03:57 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:03:58.164 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:58.163+0000 7f46abe00640 1 -- 172.21.15.12:0/3468017823 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f46a402e5c0 con 0x7f46a4102dc0 2024-09-11T00:03:58.165 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:58.164+0000 7f46a2000640 1 -- 172.21.15.12:0/3468017823 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v69) ==== 84+0+65 (secure 0 0 0) 0x7f46980acff0 con 0x7f46a4102dc0 2024-09-11T00:03:58.165 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-11T00:03:58.165 INFO:teuthology.orchestra.run.smithi012.stdout:[ 2024-09-11T00:03:58.165 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-11T00:03:58.165 INFO:teuthology.orchestra.run.smithi012.stdout: "poolnum": 1, 2024-09-11T00:03:58.165 INFO:teuthology.orchestra.run.smithi012.stdout: "poolname": ".mgr" 2024-09-11T00:03:58.166 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:03:58.166 INFO:teuthology.orchestra.run.smithi012.stdout:] 2024-09-11T00:03:58.168 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:58.167+0000 7f46abe00640 1 -- 172.21.15.12:0/3468017823 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f468c0b5360 msgr2=0x7f468c0b7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:03:58.168 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:58.167+0000 7f46abe00640 1 --2- 172.21.15.12:0/3468017823 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f468c0b5360 0x7f468c0b7820 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f46a41080c0 tx=0x7f469000b410 comp rx=0 tx=0).stop 2024-09-11T00:03:58.168 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:58.167+0000 7f46abe00640 1 -- 172.21.15.12:0/3468017823 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f46a4102dc0 msgr2=0x7f46a419b4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:03:58.168 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:58.167+0000 7f46abe00640 1 --2- 172.21.15.12:0/3468017823 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f46a4102dc0 0x7f46a419b4d0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f469800e9f0 tx=0x7f469800edf0 comp rx=0 tx=0).stop 2024-09-11T00:03:58.168 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:58.167+0000 7f46abe00640 1 -- 172.21.15.12:0/3468017823 shutdown_connections 2024-09-11T00:03:58.168 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:58.167+0000 7f46abe00640 1 --2- 172.21.15.12:0/3468017823 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f468c0b5360 0x7f468c0b7820 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:03:58.168 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:58.167+0000 7f46abe00640 1 --2- 172.21.15.12:0/3468017823 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f46a4109150 0x7f46a41a0db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:03:58.168 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:58.167+0000 7f46abe00640 1 --2- 172.21.15.12:0/3468017823 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f46a41087d0 0x7f46a419ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:03:58.168 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:58.167+0000 7f46abe00640 1 --2- 172.21.15.12:0/3468017823 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f46a4102dc0 0x7f46a419b4d0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:03:58.168 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:58.167+0000 7f46abe00640 1 -- 172.21.15.12:0/3468017823 >> 172.21.15.12:0/3468017823 conn(0x7f46a40fe830 msgr2=0x7f46a4075ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:03:58.168 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:58.167+0000 7f46abe00640 1 -- 172.21.15.12:0/3468017823 shutdown_connections 2024-09-11T00:03:58.169 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:58.167+0000 7f46abe00640 1 -- 172.21.15.12:0/3468017823 wait complete. 2024-09-11T00:03:58.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:03:57 smithi165 ceph-mon[35356]: pgmap v288: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T00:03:58.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:03:57 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:03:58.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:03:57 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:03:58.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:03:57 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:03:58.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:03:57 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:03:58.886 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-11T00:03:58.887 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid ff3ad74a-6fcf-11ef-bcea-c7b262605968 -- ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-11T00:03:59.062 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:03:58 smithi012 ceph-mon[30333]: from='client.? 172.21.15.12:0/3468017823' entity='client.admin' cmd={"prefix": "osd lspools", "format": "json-pretty"} : dispatch 2024-09-11T00:03:59.080 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:03:58 smithi178 ceph-mon[35919]: from='client.? 172.21.15.12:0/3468017823' entity='client.admin' cmd={"prefix": "osd lspools", "format": "json-pretty"} : dispatch 2024-09-11T00:03:59.284 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:03:58 smithi165 ceph-mon[35356]: from='client.? 172.21.15.12:0/3468017823' entity='client.admin' cmd={"prefix": "osd lspools", "format": "json-pretty"} : dispatch 2024-09-11T00:03:59.289 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:03:59.847 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:59.846+0000 7f318b200640 1 -- 172.21.15.12:0/2917098911 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f31841087d0 msgr2=0x7f3184108c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:03:59.848 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:59.846+0000 7f318b200640 1 --2- 172.21.15.12:0/2917098911 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f31841087d0 0x7f3184108c10 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f3170009900 tx=0x7f317002a3f0 comp rx=0 tx=0).stop 2024-09-11T00:03:59.848 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:59.846+0000 7f318b200640 1 -- 172.21.15.12:0/2917098911 shutdown_connections 2024-09-11T00:03:59.848 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:59.846+0000 7f318b200640 1 --2- 172.21.15.12:0/2917098911 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3184109150 0x7f318410f950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:03:59.848 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:59.846+0000 7f318b200640 1 --2- 172.21.15.12:0/2917098911 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f31841087d0 0x7f3184108c10 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:03:59.848 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:59.846+0000 7f318b200640 1 --2- 172.21.15.12:0/2917098911 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3184102dc0 0x7f31841031a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:03:59.848 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:59.846+0000 7f318b200640 1 -- 172.21.15.12:0/2917098911 >> 172.21.15.12:0/2917098911 conn(0x7f31840fe830 msgr2=0x7f3184100c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:03:59.848 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:59.846+0000 7f318b200640 1 -- 172.21.15.12:0/2917098911 shutdown_connections 2024-09-11T00:03:59.848 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:59.846+0000 7f318b200640 1 -- 172.21.15.12:0/2917098911 wait complete. 2024-09-11T00:03:59.849 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:59.847+0000 7f318b200640 1 Processor -- start 2024-09-11T00:03:59.849 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:59.848+0000 7f318b200640 1 -- start start 2024-09-11T00:03:59.849 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:59.848+0000 7f318b200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3184102dc0 0x7f318419b450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:03:59.849 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:59.848+0000 7f318b200640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f31841087d0 0x7f318419b990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:03:59.849 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:59.848+0000 7f318b200640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3184109150 0x7f31841a0d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:03:59.849 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:59.848+0000 7f318b200640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f3184112da0 con 0x7f3184109150 2024-09-11T00:03:59.850 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:59.848+0000 7f318b200640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f3184112c20 con 0x7f3184102dc0 2024-09-11T00:03:59.850 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:59.848+0000 7f318b200640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f3184112f20 con 0x7f31841087d0 2024-09-11T00:03:59.850 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:59.848+0000 7f3183e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3184102dc0 0x7f318419b450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:03:59.850 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:59.848+0000 7f3183e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3184102dc0 0x7f318419b450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.12:46390/0 (socket says 172.21.15.12:46390) 2024-09-11T00:03:59.850 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:59.848+0000 7f3183e00640 1 -- 172.21.15.12:0/4171403346 learned_addr learned my addr 172.21.15.12:0/4171403346 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:03:59.850 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:59.848+0000 7f3189200640 1 --2- 172.21.15.12:0/4171403346 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3184109150 0x7f31841a0d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:03:59.850 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:59.848+0000 7f3183400640 1 --2- 172.21.15.12:0/4171403346 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f31841087d0 0x7f318419b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:03:59.851 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:59.849+0000 7f3183e00640 1 -- 172.21.15.12:0/4171403346 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f31841087d0 msgr2=0x7f318419b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:03:59.851 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:59.849+0000 7f3183e00640 1 --2- 172.21.15.12:0/4171403346 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f31841087d0 0x7f318419b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:03:59.851 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:59.849+0000 7f3183e00640 1 -- 172.21.15.12:0/4171403346 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3184109150 msgr2=0x7f31841a0d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:03:59.851 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:59.849+0000 7f3183e00640 1 --2- 172.21.15.12:0/4171403346 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3184109150 0x7f31841a0d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:03:59.851 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:59.849+0000 7f3183e00640 1 -- 172.21.15.12:0/4171403346 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31841a1500 con 0x7f3184102dc0 2024-09-11T00:03:59.852 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:59.849+0000 7f3183e00640 1 --2- 172.21.15.12:0/4171403346 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3184102dc0 0x7f318419b450 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f317400d950 tx=0x7f317400de10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:03:59.852 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:59.849+0000 7f3180c00640 1 -- 172.21.15.12:0/4171403346 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3174017070 con 0x7f3184102dc0 2024-09-11T00:03:59.852 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:59.849+0000 7f3180c00640 1 -- 172.21.15.12:0/4171403346 <== mon.2 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3174009c30 con 0x7f3184102dc0 2024-09-11T00:03:59.852 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:59.849+0000 7f3180c00640 1 -- 172.21.15.12:0/4171403346 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f317400f620 con 0x7f3184102dc0 2024-09-11T00:03:59.852 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:59.849+0000 7f318b200640 1 -- 172.21.15.12:0/4171403346 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f31841a17f0 con 0x7f3184102dc0 2024-09-11T00:03:59.852 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:59.849+0000 7f318b200640 1 -- 172.21.15.12:0/4171403346 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f31841a8080 con 0x7f3184102dc0 2024-09-11T00:03:59.852 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:59.850+0000 7f318b200640 1 -- 172.21.15.12:0/4171403346 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3184104070 con 0x7f3184102dc0 2024-09-11T00:03:59.853 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:59.853+0000 7f3180c00640 1 -- 172.21.15.12:0/4171403346 <== mon.2 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 16) ==== 153250+0+0 (secure 0 0 0) 0x7f3174010460 con 0x7f3184102dc0 2024-09-11T00:03:59.855 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:59.854+0000 7f3180c00640 1 --2- 172.21.15.12:0/4171403346 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f31680b5360 0x7f31680b7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:03:59.855 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:59.854+0000 7f3180c00640 1 -- 172.21.15.12:0/4171403346 <== mon.2 v2:172.21.15.165:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 7127+0+0 (secure 0 0 0) 0x7f31740e5320 con 0x7f3184102dc0 2024-09-11T00:03:59.855 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:59.854+0000 7f3183400640 1 --2- 172.21.15.12:0/4171403346 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f31680b5360 0x7f31680b7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:03:59.855 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:59.855+0000 7f3183400640 1 --2- 172.21.15.12:0/4171403346 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f31680b5360 0x7f31680b7820 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f3170031440 tx=0x7f3170005b50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:03:59.858 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:03:59.858+0000 7f3180c00640 1 -- 172.21.15.12:0/4171403346 <== mon.2 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f3174009de0 con 0x7f3184102dc0 2024-09-11T00:04:00.055 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:00.054+0000 7f318b200640 1 -- 172.21.15.12:0/4171403346 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f31841031a0 con 0x7f3184102dc0 2024-09-11T00:04:00.055 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:03:59 smithi012 ceph-mon[30333]: pgmap v289: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T00:04:00.055 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:03:59 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:04:00.055 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:03:59 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:04:00.055 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:03:59 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:04:00.056 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:03:59 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T00:04:00.056 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:03:59 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:04:00.080 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:03:59 smithi178 ceph-mon[35919]: pgmap v289: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T00:04:00.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:03:59 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:04:00.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:03:59 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:04:00.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:03:59 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:04:00.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:03:59 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T00:04:00.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:03:59 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:04:00.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:03:59 smithi165 ceph-mon[35356]: pgmap v289: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T00:04:00.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:03:59 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:04:00.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:03:59 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:04:00.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:03:59 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:04:00.283 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:03:59 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T00:04:00.283 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:03:59 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:04:00.810 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:00.807+0000 7f3180c00640 1 -- 172.21.15.12:0/4171403346 <== mon.2 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v70) ==== 138+0+0 (secure 0 0 0) 0x7f31841031a0 con 0x7f3184102dc0 2024-09-11T00:04:00.810 INFO:teuthology.orchestra.run.smithi012.stderr:pool 'cephfs_metadata' created 2024-09-11T00:04:00.814 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:00.813+0000 7f318b200640 1 -- 172.21.15.12:0/4171403346 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f31680b5360 msgr2=0x7f31680b7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:04:00.815 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:00.813+0000 7f318b200640 1 --2- 172.21.15.12:0/4171403346 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f31680b5360 0x7f31680b7820 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f3170031440 tx=0x7f3170005b50 comp rx=0 tx=0).stop 2024-09-11T00:04:00.815 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:00.813+0000 7f318b200640 1 -- 172.21.15.12:0/4171403346 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3184102dc0 msgr2=0x7f318419b450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:04:00.815 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:00.813+0000 7f318b200640 1 --2- 172.21.15.12:0/4171403346 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3184102dc0 0x7f318419b450 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f317400d950 tx=0x7f317400de10 comp rx=0 tx=0).stop 2024-09-11T00:04:00.815 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:00.814+0000 7f318b200640 1 -- 172.21.15.12:0/4171403346 shutdown_connections 2024-09-11T00:04:00.815 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:00.814+0000 7f318b200640 1 --2- 172.21.15.12:0/4171403346 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f31680b5360 0x7f31680b7820 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:04:00.815 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:00.814+0000 7f318b200640 1 --2- 172.21.15.12:0/4171403346 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3184109150 0x7f31841a0d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:04:00.815 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:00.814+0000 7f318b200640 1 --2- 172.21.15.12:0/4171403346 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f31841087d0 0x7f318419b990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:04:00.815 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:00.814+0000 7f318b200640 1 --2- 172.21.15.12:0/4171403346 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3184102dc0 0x7f318419b450 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:04:00.815 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:00.814+0000 7f318b200640 1 -- 172.21.15.12:0/4171403346 >> 172.21.15.12:0/4171403346 conn(0x7f31840fe830 msgr2=0x7f3184105db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:04:00.815 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:00.814+0000 7f318b200640 1 -- 172.21.15.12:0/4171403346 shutdown_connections 2024-09-11T00:04:00.815 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:00.814+0000 7f318b200640 1 -- 172.21.15.12:0/4171403346 wait complete. 2024-09-11T00:04:01.062 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:04:00 smithi012 ceph-mon[30333]: from='client.? 172.21.15.12:0/4171403346' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} : dispatch 2024-09-11T00:04:01.063 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:04:00 smithi012 ceph-mon[30333]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} : dispatch 2024-09-11T00:04:01.080 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:04:00 smithi178 ceph-mon[35919]: from='client.? 172.21.15.12:0/4171403346' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} : dispatch 2024-09-11T00:04:01.080 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:04:00 smithi178 ceph-mon[35919]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} : dispatch 2024-09-11T00:04:01.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:04:00 smithi165 ceph-mon[35356]: from='client.? 172.21.15.12:0/4171403346' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} : dispatch 2024-09-11T00:04:01.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:04:00 smithi165 ceph-mon[35356]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} : dispatch 2024-09-11T00:04:01.535 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid ff3ad74a-6fcf-11ef-bcea-c7b262605968 -- ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.05 2024-09-11T00:04:01.815 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:04:01 smithi012 ceph-mon[30333]: pgmap v290: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T00:04:01.815 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:04:01 smithi012 ceph-mon[30333]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-09-11T00:04:01.815 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:04:01 smithi012 ceph-mon[30333]: osdmap e70: 12 total, 12 up, 12 in 2024-09-11T00:04:01.925 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:04:02.080 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:04:01 smithi178 ceph-mon[35919]: pgmap v290: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T00:04:02.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:04:01 smithi178 ceph-mon[35919]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-09-11T00:04:02.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:04:01 smithi178 ceph-mon[35919]: osdmap e70: 12 total, 12 up, 12 in 2024-09-11T00:04:02.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:04:01 smithi165 ceph-mon[35356]: pgmap v290: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T00:04:02.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:04:01 smithi165 ceph-mon[35356]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-09-11T00:04:02.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:04:01 smithi165 ceph-mon[35356]: osdmap e70: 12 total, 12 up, 12 in 2024-09-11T00:04:02.472 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:02.471+0000 7fee56a00640 1 -- 172.21.15.12:0/635877591 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fee50104fa0 msgr2=0x7fee50105380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:04:02.473 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:02.471+0000 7fee56a00640 1 --2- 172.21.15.12:0/635877591 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fee50104fa0 0x7fee50105380 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fee40009ca0 tx=0x7fee4002a8c0 comp rx=0 tx=0).stop 2024-09-11T00:04:02.473 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:02.471+0000 7fee56a00640 1 -- 172.21.15.12:0/635877591 shutdown_connections 2024-09-11T00:04:02.473 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:02.471+0000 7fee56a00640 1 --2- 172.21.15.12:0/635877591 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fee5010b330 0x7fee50111b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:04:02.473 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:02.471+0000 7fee56a00640 1 --2- 172.21.15.12:0/635877591 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fee5010a9b0 0x7fee5010adf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:04:02.473 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:02.471+0000 7fee56a00640 1 --2- 172.21.15.12:0/635877591 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fee50104fa0 0x7fee50105380 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:04:02.473 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:02.471+0000 7fee56a00640 1 -- 172.21.15.12:0/635877591 >> 172.21.15.12:0/635877591 conn(0x7fee50100a50 msgr2=0x7fee50102e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:04:02.473 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:02.471+0000 7fee56a00640 1 -- 172.21.15.12:0/635877591 shutdown_connections 2024-09-11T00:04:02.473 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:02.471+0000 7fee56a00640 1 -- 172.21.15.12:0/635877591 wait complete. 2024-09-11T00:04:02.475 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:02.472+0000 7fee56a00640 1 Processor -- start 2024-09-11T00:04:02.475 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:02.472+0000 7fee56a00640 1 -- start start 2024-09-11T00:04:02.475 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:02.473+0000 7fee56a00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fee50104fa0 0x7fee5019b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:04:02.475 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:02.473+0000 7fee56a00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fee5010a9b0 0x7fee5019ba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:04:02.475 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:02.473+0000 7fee56a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fee5010b330 0x7fee501a0dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:04:02.475 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:02.473+0000 7fee56a00640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fee50114fb0 con 0x7fee50104fa0 2024-09-11T00:04:02.476 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:02.473+0000 7fee56a00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fee50114e30 con 0x7fee5010b330 2024-09-11T00:04:02.476 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:02.473+0000 7fee56a00640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fee50115130 con 0x7fee5010a9b0 2024-09-11T00:04:02.476 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:02.473+0000 7fee4fe00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fee50104fa0 0x7fee5019b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:04:02.476 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:02.473+0000 7fee4fe00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fee50104fa0 0x7fee5019b4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:59048/0 (socket says 172.21.15.12:59048) 2024-09-11T00:04:02.476 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:02.473+0000 7fee4fe00640 1 -- 172.21.15.12:0/1005244275 learned_addr learned my addr 172.21.15.12:0/1005244275 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:04:02.476 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:02.473+0000 7fee47400640 1 --2- 172.21.15.12:0/1005244275 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fee5010a9b0 0x7fee5019ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:04:02.476 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:02.473+0000 7fee54a00640 1 --2- 172.21.15.12:0/1005244275 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fee5010b330 0x7fee501a0dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:04:02.476 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:02.474+0000 7fee47400640 1 -- 172.21.15.12:0/1005244275 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fee5010b330 msgr2=0x7fee501a0dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:04:02.476 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:02.474+0000 7fee47400640 1 --2- 172.21.15.12:0/1005244275 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fee5010b330 0x7fee501a0dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:04:02.476 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:02.474+0000 7fee47400640 1 -- 172.21.15.12:0/1005244275 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fee50104fa0 msgr2=0x7fee5019b4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:04:02.476 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:02.474+0000 7fee47400640 1 --2- 172.21.15.12:0/1005244275 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fee50104fa0 0x7fee5019b4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:04:02.476 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:02.474+0000 7fee47400640 1 -- 172.21.15.12:0/1005244275 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee501a1590 con 0x7fee5010a9b0 2024-09-11T00:04:02.477 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:02.474+0000 7fee47400640 1 --2- 172.21.15.12:0/1005244275 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fee5010a9b0 0x7fee5019ba20 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fee3800e9f0 tx=0x7fee3800eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:04:02.477 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:02.474+0000 7fee4d600640 1 -- 172.21.15.12:0/1005244275 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee3800ccc0 con 0x7fee5010a9b0 2024-09-11T00:04:02.477 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:02.474+0000 7fee56a00640 1 -- 172.21.15.12:0/1005244275 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fee501a1880 con 0x7fee5010a9b0 2024-09-11T00:04:02.477 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:02.474+0000 7fee4d600640 1 -- 172.21.15.12:0/1005244275 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fee38005020 con 0x7fee5010a9b0 2024-09-11T00:04:02.477 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:02.474+0000 7fee4d600640 1 -- 172.21.15.12:0/1005244275 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee38004a00 con 0x7fee5010a9b0 2024-09-11T00:04:02.477 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:02.474+0000 7fee56a00640 1 -- 172.21.15.12:0/1005244275 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fee501a7f40 con 0x7fee5010a9b0 2024-09-11T00:04:02.477 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:02.475+0000 7fee56a00640 1 -- 172.21.15.12:0/1005244275 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fee50105380 con 0x7fee5010a9b0 2024-09-11T00:04:02.478 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:02.478+0000 7fee4d600640 1 -- 172.21.15.12:0/1005244275 <== mon.1 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 16) ==== 153250+0+0 (secure 0 0 0) 0x7fee38004ba0 con 0x7fee5010a9b0 2024-09-11T00:04:02.481 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:02.480+0000 7fee4d600640 1 --2- 172.21.15.12:0/1005244275 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fee340b52e0 0x7fee340b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:04:02.481 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:02.480+0000 7fee4d600640 1 -- 172.21.15.12:0/1005244275 <== mon.1 v2:172.21.15.178:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 7497+0+0 (secure 0 0 0) 0x7fee380e5970 con 0x7fee5010a9b0 2024-09-11T00:04:02.481 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:02.480+0000 7fee4fe00640 1 --2- 172.21.15.12:0/1005244275 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fee340b52e0 0x7fee340b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:04:02.481 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:02.481+0000 7fee4d600640 1 -- 172.21.15.12:0/1005244275 <== mon.1 v2:172.21.15.178:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7fee380acb50 con 0x7fee5010a9b0 2024-09-11T00:04:02.481 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:02.481+0000 7fee4fe00640 1 --2- 172.21.15.12:0/1005244275 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fee340b52e0 0x7fee340b77a0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fee40009a60 tx=0x7fee40005d80 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:04:02.681 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:02.680+0000 7fee56a00640 1 -- 172.21.15.12:0/1005244275 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05} v 0) -- 0x7fee5002e5c0 con 0x7fee5010a9b0 2024-09-11T00:04:02.814 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:02.812+0000 7fee4d600640 1 -- 172.21.15.12:0/1005244275 <== mon.1 v2:172.21.15.178:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]=0 pool 'cephfs_data' created v72) ==== 171+0+0 (secure 0 0 0) 0x7fee380ac450 con 0x7fee5010a9b0 2024-09-11T00:04:02.814 INFO:teuthology.orchestra.run.smithi012.stderr:pool 'cephfs_data' created 2024-09-11T00:04:02.820 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:02.817+0000 7fee56a00640 1 -- 172.21.15.12:0/1005244275 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fee340b52e0 msgr2=0x7fee340b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:04:02.820 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:02.817+0000 7fee56a00640 1 --2- 172.21.15.12:0/1005244275 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fee340b52e0 0x7fee340b77a0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fee40009a60 tx=0x7fee40005d80 comp rx=0 tx=0).stop 2024-09-11T00:04:02.820 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:02.817+0000 7fee56a00640 1 -- 172.21.15.12:0/1005244275 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fee5010a9b0 msgr2=0x7fee5019ba20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:04:02.820 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:02.817+0000 7fee56a00640 1 --2- 172.21.15.12:0/1005244275 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fee5010a9b0 0x7fee5019ba20 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fee3800e9f0 tx=0x7fee3800eeb0 comp rx=0 tx=0).stop 2024-09-11T00:04:02.820 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:02.818+0000 7fee56a00640 1 -- 172.21.15.12:0/1005244275 shutdown_connections 2024-09-11T00:04:02.820 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:02.818+0000 7fee56a00640 1 --2- 172.21.15.12:0/1005244275 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fee340b52e0 0x7fee340b77a0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:04:02.820 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:02.818+0000 7fee56a00640 1 --2- 172.21.15.12:0/1005244275 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fee5010b330 0x7fee501a0dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:04:02.820 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:02.818+0000 7fee56a00640 1 --2- 172.21.15.12:0/1005244275 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fee5010a9b0 0x7fee5019ba20 secure :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fee3800e9f0 tx=0x7fee3800eeb0 comp rx=0 tx=0).stop 2024-09-11T00:04:02.820 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:02.818+0000 7fee56a00640 1 --2- 172.21.15.12:0/1005244275 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fee50104fa0 0x7fee5019b4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:04:02.820 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:02.818+0000 7fee56a00640 1 -- 172.21.15.12:0/1005244275 >> 172.21.15.12:0/1005244275 conn(0x7fee50100a50 msgr2=0x7fee50102e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:04:02.821 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:02.820+0000 7fee56a00640 1 -- 172.21.15.12:0/1005244275 shutdown_connections 2024-09-11T00:04:02.821 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:02.820+0000 7fee56a00640 1 -- 172.21.15.12:0/1005244275 wait complete. 2024-09-11T00:04:03.063 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:04:02 smithi012 ceph-mon[30333]: osdmap e71: 12 total, 12 up, 12 in 2024-09-11T00:04:03.063 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:04:02 smithi012 ceph-mon[30333]: from='client.? 172.21.15.12:0/1005244275' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05} : dispatch 2024-09-11T00:04:03.063 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:04:02 smithi012 ceph-mon[30333]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05} : dispatch 2024-09-11T00:04:03.063 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:04:02 smithi012 ceph-mon[30333]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-09-11T00:04:03.063 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:04:02 smithi012 ceph-mon[30333]: osdmap e72: 12 total, 12 up, 12 in 2024-09-11T00:04:03.080 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:04:02 smithi178 ceph-mon[35919]: osdmap e71: 12 total, 12 up, 12 in 2024-09-11T00:04:03.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:04:02 smithi178 ceph-mon[35919]: from='client.? 172.21.15.12:0/1005244275' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05} : dispatch 2024-09-11T00:04:03.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:04:02 smithi178 ceph-mon[35919]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05} : dispatch 2024-09-11T00:04:03.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:04:02 smithi178 ceph-mon[35919]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-09-11T00:04:03.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:04:02 smithi178 ceph-mon[35919]: osdmap e72: 12 total, 12 up, 12 in 2024-09-11T00:04:03.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:04:02 smithi165 ceph-mon[35356]: osdmap e71: 12 total, 12 up, 12 in 2024-09-11T00:04:03.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:04:02 smithi165 ceph-mon[35356]: from='client.? 172.21.15.12:0/1005244275' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05} : dispatch 2024-09-11T00:04:03.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:04:02 smithi165 ceph-mon[35356]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05} : dispatch 2024-09-11T00:04:03.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:04:02 smithi165 ceph-mon[35356]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-09-11T00:04:03.283 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:04:02 smithi165 ceph-mon[35356]: osdmap e72: 12 total, 12 up, 12 in 2024-09-11T00:04:03.577 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid ff3ad74a-6fcf-11ef-bcea-c7b262605968 -- ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-11T00:04:03.975 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:04:04.089 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:04:03 smithi165 ceph-mon[35356]: pgmap v293: 33 pgs: 9 creating+peering, 21 unknown, 3 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T00:04:04.089 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:04:03 smithi165 ceph-mon[35356]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T00:04:04.089 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:04:03 smithi165 ceph-mon[35356]: osdmap e73: 12 total, 12 up, 12 in 2024-09-11T00:04:04.113 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:04:03 smithi012 ceph-mon[30333]: pgmap v293: 33 pgs: 9 creating+peering, 21 unknown, 3 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T00:04:04.114 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:04:03 smithi012 ceph-mon[30333]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T00:04:04.114 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:04:03 smithi012 ceph-mon[30333]: osdmap e73: 12 total, 12 up, 12 in 2024-09-11T00:04:04.330 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:04:03 smithi178 ceph-mon[35919]: pgmap v293: 33 pgs: 9 creating+peering, 21 unknown, 3 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T00:04:04.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:04:03 smithi178 ceph-mon[35919]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T00:04:04.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:04:03 smithi178 ceph-mon[35919]: osdmap e73: 12 total, 12 up, 12 in 2024-09-11T00:04:04.491 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:04.490+0000 7fad53e00640 1 -- 172.21.15.12:0/2173087336 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fad5410a800 msgr2=0x7fad5410ac40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:04:04.491 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:04.490+0000 7fad53e00640 1 --2- 172.21.15.12:0/2173087336 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fad5410a800 0x7fad5410ac40 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fad40009900 tx=0x7fad4002a3f0 comp rx=0 tx=0).stop 2024-09-11T00:04:04.491 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:04.490+0000 7fad53e00640 1 -- 172.21.15.12:0/2173087336 shutdown_connections 2024-09-11T00:04:04.492 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:04.490+0000 7fad53e00640 1 --2- 172.21.15.12:0/2173087336 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fad5410b180 0x7fad54111980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:04:04.492 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:04.490+0000 7fad53e00640 1 --2- 172.21.15.12:0/2173087336 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fad5410a800 0x7fad5410ac40 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:04:04.492 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:04.490+0000 7fad53e00640 1 --2- 172.21.15.12:0/2173087336 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fad54104df0 0x7fad541051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:04:04.492 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:04.490+0000 7fad53e00640 1 -- 172.21.15.12:0/2173087336 >> 172.21.15.12:0/2173087336 conn(0x7fad54100840 msgr2=0x7fad54102c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:04:04.492 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:04.490+0000 7fad53e00640 1 -- 172.21.15.12:0/2173087336 shutdown_connections 2024-09-11T00:04:04.492 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:04.490+0000 7fad53e00640 1 -- 172.21.15.12:0/2173087336 wait complete. 2024-09-11T00:04:04.494 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:04.491+0000 7fad53e00640 1 Processor -- start 2024-09-11T00:04:04.494 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:04.491+0000 7fad53e00640 1 -- start start 2024-09-11T00:04:04.494 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:04.491+0000 7fad53e00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fad54104df0 0x7fad5419b210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:04:04.494 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:04.491+0000 7fad53e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fad5410a800 0x7fad5419b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:04:04.494 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:04.491+0000 7fad53e00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fad5410b180 0x7fad541a0af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:04:04.494 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:04.491+0000 7fad53e00640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fad54114e00 con 0x7fad5410b180 2024-09-11T00:04:04.494 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:04.491+0000 7fad53e00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fad54114c80 con 0x7fad5410a800 2024-09-11T00:04:04.494 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:04.491+0000 7fad53e00640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fad54114f80 con 0x7fad54104df0 2024-09-11T00:04:04.494 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:04.492+0000 7fad53400640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fad5410b180 0x7fad541a0af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:04:04.494 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:04.492+0000 7fad53400640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fad5410b180 0x7fad541a0af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:59060/0 (socket says 172.21.15.12:59060) 2024-09-11T00:04:04.495 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:04.492+0000 7fad53400640 1 -- 172.21.15.12:0/2405553644 learned_addr learned my addr 172.21.15.12:0/2405553644 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:04:04.495 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:04.492+0000 7fad52a00640 1 --2- 172.21.15.12:0/2405553644 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fad54104df0 0x7fad5419b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:04:04.495 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:04.492+0000 7fad52000640 1 --2- 172.21.15.12:0/2405553644 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fad5410a800 0x7fad5419b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:04:04.495 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:04.492+0000 7fad53400640 1 -- 172.21.15.12:0/2405553644 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fad54104df0 msgr2=0x7fad5419b210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:04:04.495 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:04.492+0000 7fad53400640 1 --2- 172.21.15.12:0/2405553644 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fad54104df0 0x7fad5419b210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:04:04.495 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:04.492+0000 7fad53400640 1 -- 172.21.15.12:0/2405553644 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fad5410a800 msgr2=0x7fad5419b750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:04:04.495 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:04.492+0000 7fad53400640 1 --2- 172.21.15.12:0/2405553644 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fad5410a800 0x7fad5419b750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:04:04.495 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:04.492+0000 7fad53400640 1 -- 172.21.15.12:0/2405553644 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad541a12c0 con 0x7fad5410b180 2024-09-11T00:04:04.495 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:04.492+0000 7fad52000640 1 --2- 172.21.15.12:0/2405553644 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fad5410a800 0x7fad5419b750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:04:04.495 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:04.493+0000 7fad53400640 1 --2- 172.21.15.12:0/2405553644 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fad5410b180 0x7fad541a0af0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fad4c00e970 tx=0x7fad4c00ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:04:04.495 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:04.493+0000 7fad4b400640 1 -- 172.21.15.12:0/2405553644 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad4c00ccc0 con 0x7fad5410b180 2024-09-11T00:04:04.495 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:04.493+0000 7fad4b400640 1 -- 172.21.15.12:0/2405553644 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fad4c00ce60 con 0x7fad5410b180 2024-09-11T00:04:04.495 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:04.493+0000 7fad53e00640 1 -- 172.21.15.12:0/2405553644 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fad541a15b0 con 0x7fad5410b180 2024-09-11T00:04:04.496 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:04.493+0000 7fad4b400640 1 -- 172.21.15.12:0/2405553644 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad4c010430 con 0x7fad5410b180 2024-09-11T00:04:04.496 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:04.493+0000 7fad53e00640 1 -- 172.21.15.12:0/2405553644 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fad541a7e40 con 0x7fad5410b180 2024-09-11T00:04:04.496 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:04.494+0000 7fad53e00640 1 -- 172.21.15.12:0/2405553644 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fad541051d0 con 0x7fad5410b180 2024-09-11T00:04:04.497 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:04.497+0000 7fad4b400640 1 -- 172.21.15.12:0/2405553644 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 16) ==== 153250+0+0 (secure 0 0 0) 0x7fad4c004f70 con 0x7fad5410b180 2024-09-11T00:04:04.499 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:04.498+0000 7fad4b400640 1 --2- 172.21.15.12:0/2405553644 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fad380b5410 0x7fad380b78d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:04:04.499 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:04.498+0000 7fad4b400640 1 -- 172.21.15.12:0/2405553644 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 7879+0+0 (secure 0 0 0) 0x7fad4c0e5ff0 con 0x7fad5410b180 2024-09-11T00:04:04.499 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:04.498+0000 7fad52a00640 1 --2- 172.21.15.12:0/2405553644 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fad380b5410 0x7fad380b78d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:04:04.500 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:04.499+0000 7fad52a00640 1 --2- 172.21.15.12:0/2405553644 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fad380b5410 0x7fad380b78d0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fad44005fd0 tx=0x7fad44009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:04:04.500 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:04.500+0000 7fad4b400640 1 -- 172.21.15.12:0/2405553644 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7fad4c056220 con 0x7fad5410b180 2024-09-11T00:04:04.739 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:04.739+0000 7fad53e00640 1 -- 172.21.15.12:0/2405553644 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7fad5419c340 con 0x7fad5410b180 2024-09-11T00:04:04.838 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:04.837+0000 7fad4b400640 1 -- 172.21.15.12:0/2405553644 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v14) ==== 172+0+0 (secure 0 0 0) 0x7fad5419c340 con 0x7fad5410b180 2024-09-11T00:04:04.838 INFO:teuthology.orchestra.run.smithi012.stderr:new fs with metadata pool 2 and data pool 3 2024-09-11T00:04:04.840 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:04.839+0000 7fad53e00640 1 -- 172.21.15.12:0/2405553644 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fad380b5410 msgr2=0x7fad380b78d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:04:04.840 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:04.839+0000 7fad53e00640 1 --2- 172.21.15.12:0/2405553644 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fad380b5410 0x7fad380b78d0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fad44005fd0 tx=0x7fad44009500 comp rx=0 tx=0).stop 2024-09-11T00:04:04.840 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:04.839+0000 7fad53e00640 1 -- 172.21.15.12:0/2405553644 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fad5410b180 msgr2=0x7fad541a0af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:04:04.840 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:04.839+0000 7fad53e00640 1 --2- 172.21.15.12:0/2405553644 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fad5410b180 0x7fad541a0af0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fad4c00e970 tx=0x7fad4c00ed70 comp rx=0 tx=0).stop 2024-09-11T00:04:04.841 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:04.839+0000 7fad53e00640 1 -- 172.21.15.12:0/2405553644 shutdown_connections 2024-09-11T00:04:04.841 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:04.839+0000 7fad53e00640 1 --2- 172.21.15.12:0/2405553644 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fad380b5410 0x7fad380b78d0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:04:04.841 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:04.839+0000 7fad53e00640 1 --2- 172.21.15.12:0/2405553644 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fad5410b180 0x7fad541a0af0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:04:04.841 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:04.839+0000 7fad53e00640 1 --2- 172.21.15.12:0/2405553644 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fad5410a800 0x7fad5419b750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:04:04.841 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:04.839+0000 7fad53e00640 1 --2- 172.21.15.12:0/2405553644 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fad54104df0 0x7fad5419b210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:04:04.841 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:04.839+0000 7fad53e00640 1 -- 172.21.15.12:0/2405553644 >> 172.21.15.12:0/2405553644 conn(0x7fad54100840 msgr2=0x7fad54107de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:04:04.841 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:04.839+0000 7fad53e00640 1 -- 172.21.15.12:0/2405553644 shutdown_connections 2024-09-11T00:04:04.842 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:04.840+0000 7fad53e00640 1 -- 172.21.15.12:0/2405553644 wait complete. 2024-09-11T00:04:05.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:04:04 smithi165 ceph-mon[35356]: from='client.? 172.21.15.12:0/2405553644' entity='client.admin' cmd={"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} : dispatch 2024-09-11T00:04:05.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:04:04 smithi165 ceph-mon[35356]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-09-11T00:04:05.063 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:04:04 smithi012 ceph-mon[30333]: from='client.? 172.21.15.12:0/2405553644' entity='client.admin' cmd={"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} : dispatch 2024-09-11T00:04:05.063 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:04:04 smithi012 ceph-mon[30333]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-09-11T00:04:05.330 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:04:04 smithi178 ceph-mon[35919]: from='client.? 172.21.15.12:0/2405553644' entity='client.admin' cmd={"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} : dispatch 2024-09-11T00:04:05.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:04:04 smithi178 ceph-mon[35919]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-09-11T00:04:05.654 DEBUG:tasks.cephfs.filesystem:EC profile is ['m=2', 'k=2', 'crush-failure-domain=osd'] 2024-09-11T00:04:05.655 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid ff3ad74a-6fcf-11ef-bcea-c7b262605968 -- ceph osd erasure-code-profile set cephfs_data_ec m=2 k=2 crush-failure-domain=osd 2024-09-11T00:04:06.037 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:04:06.062 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:04:05 smithi012 ceph-mon[30333]: pgmap v296: 97 pgs: 9 creating+peering, 77 unknown, 11 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T00:04:06.063 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:04:05 smithi012 ceph-mon[30333]: osdmap e74: 12 total, 12 up, 12 in 2024-09-11T00:04:06.063 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:04:05 smithi012 ceph-mon[30333]: from='client.? 172.21.15.12:0/2405553644' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-09-11T00:04:06.063 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:04:05 smithi012 ceph-mon[30333]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-09-11T00:04:06.063 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:04:05 smithi012 ceph-mon[30333]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-09-11T00:04:06.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:04:05 smithi165 ceph-mon[35356]: pgmap v296: 97 pgs: 9 creating+peering, 77 unknown, 11 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T00:04:06.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:04:05 smithi165 ceph-mon[35356]: osdmap e74: 12 total, 12 up, 12 in 2024-09-11T00:04:06.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:04:05 smithi165 ceph-mon[35356]: from='client.? 172.21.15.12:0/2405553644' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-09-11T00:04:06.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:04:05 smithi165 ceph-mon[35356]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-09-11T00:04:06.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:04:05 smithi165 ceph-mon[35356]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-09-11T00:04:06.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:04:05 smithi178 ceph-mon[35919]: pgmap v296: 97 pgs: 9 creating+peering, 77 unknown, 11 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T00:04:06.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:04:05 smithi178 ceph-mon[35919]: osdmap e74: 12 total, 12 up, 12 in 2024-09-11T00:04:06.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:04:05 smithi178 ceph-mon[35919]: from='client.? 172.21.15.12:0/2405553644' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-09-11T00:04:06.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:04:05 smithi178 ceph-mon[35919]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-09-11T00:04:06.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:04:05 smithi178 ceph-mon[35919]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-09-11T00:04:06.614 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:06.612+0000 7f869c800640 1 -- 172.21.15.12:0/2767191348 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f869410b330 msgr2=0x7f8694111b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:04:06.615 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:06.612+0000 7f869c800640 1 --2- 172.21.15.12:0/2767191348 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f869410b330 0x7f8694111b30 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f8690009ca0 tx=0x7f869002a8c0 comp rx=0 tx=0).stop 2024-09-11T00:04:06.615 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:06.613+0000 7f869c800640 1 -- 172.21.15.12:0/2767191348 shutdown_connections 2024-09-11T00:04:06.615 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:06.613+0000 7f869c800640 1 --2- 172.21.15.12:0/2767191348 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f869410b330 0x7f8694111b30 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:04:06.615 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:06.613+0000 7f869c800640 1 --2- 172.21.15.12:0/2767191348 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f869410a9b0 0x7f869410adf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:04:06.615 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:06.613+0000 7f869c800640 1 --2- 172.21.15.12:0/2767191348 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8694104fa0 0x7f8694105380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:04:06.615 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:06.613+0000 7f869c800640 1 -- 172.21.15.12:0/2767191348 >> 172.21.15.12:0/2767191348 conn(0x7f8694100a50 msgr2=0x7f8694102e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:04:06.615 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:06.613+0000 7f869c800640 1 -- 172.21.15.12:0/2767191348 shutdown_connections 2024-09-11T00:04:06.615 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:06.613+0000 7f869c800640 1 -- 172.21.15.12:0/2767191348 wait complete. 2024-09-11T00:04:06.615 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:06.614+0000 7f869c800640 1 Processor -- start 2024-09-11T00:04:06.615 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:06.614+0000 7f869c800640 1 -- start start 2024-09-11T00:04:06.617 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:06.614+0000 7f869c800640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8694104fa0 0x7f869419b440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:04:06.617 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:06.614+0000 7f869c800640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f869410a9b0 0x7f869419b980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:04:06.617 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:06.614+0000 7f869c800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f869410b330 0x7f86941a0d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:04:06.617 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:06.614+0000 7f869c800640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f8694114f60 con 0x7f869410a9b0 2024-09-11T00:04:06.617 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:06.614+0000 7f869c800640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f8694114de0 con 0x7f869410b330 2024-09-11T00:04:06.617 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:06.614+0000 7f869c800640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f86941150e0 con 0x7f8694104fa0 2024-09-11T00:04:06.618 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:06.614+0000 7f8699400640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f869410a9b0 0x7f869419b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:04:06.618 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:06.615+0000 7f8699e00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8694104fa0 0x7f869419b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:04:06.618 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:06.615+0000 7f8699400640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f869410a9b0 0x7f869419b980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:59086/0 (socket says 172.21.15.12:59086) 2024-09-11T00:04:06.618 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:06.615+0000 7f8699400640 1 -- 172.21.15.12:0/3751156754 learned_addr learned my addr 172.21.15.12:0/3751156754 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:04:06.618 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:06.615+0000 7f869a800640 1 --2- 172.21.15.12:0/3751156754 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f869410b330 0x7f86941a0d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:04:06.618 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:06.615+0000 7f8699400640 1 -- 172.21.15.12:0/3751156754 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8694104fa0 msgr2=0x7f869419b440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:04:06.618 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:06.615+0000 7f8699400640 1 --2- 172.21.15.12:0/3751156754 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8694104fa0 0x7f869419b440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:04:06.618 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:06.615+0000 7f8699400640 1 -- 172.21.15.12:0/3751156754 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f869410b330 msgr2=0x7f86941a0d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:04:06.618 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:06.615+0000 7f8699400640 1 --2- 172.21.15.12:0/3751156754 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f869410b330 0x7f86941a0d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:04:06.618 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:06.615+0000 7f8699400640 1 -- 172.21.15.12:0/3751156754 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86941a14f0 con 0x7f869410a9b0 2024-09-11T00:04:06.618 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:06.615+0000 7f8699e00640 1 --2- 172.21.15.12:0/3751156754 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8694104fa0 0x7f869419b440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:04:06.618 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:06.615+0000 7f869a800640 1 --2- 172.21.15.12:0/3751156754 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f869410b330 0x7f86941a0d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T00:04:06.619 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:06.615+0000 7f8699400640 1 --2- 172.21.15.12:0/3751156754 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f869410a9b0 0x7f869419b980 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f868400e9f0 tx=0x7f868400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:04:06.619 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:06.616+0000 7f868ea00640 1 -- 172.21.15.12:0/3751156754 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f868400cd80 con 0x7f869410a9b0 2024-09-11T00:04:06.619 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:06.616+0000 7f868ea00640 1 -- 172.21.15.12:0/3751156754 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f86840108e0 con 0x7f869410a9b0 2024-09-11T00:04:06.619 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:06.616+0000 7f869c800640 1 -- 172.21.15.12:0/3751156754 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f86941a17e0 con 0x7f869410a9b0 2024-09-11T00:04:06.619 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:06.616+0000 7f868ea00640 1 -- 172.21.15.12:0/3751156754 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8684018980 con 0x7f869410a9b0 2024-09-11T00:04:06.619 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:06.616+0000 7f869c800640 1 -- 172.21.15.12:0/3751156754 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86941a7ea0 con 0x7f869410a9b0 2024-09-11T00:04:06.619 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:06.617+0000 7f869c800640 1 -- 172.21.15.12:0/3751156754 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8694105380 con 0x7f869410a9b0 2024-09-11T00:04:06.620 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:06.619+0000 7f868ea00640 1 -- 172.21.15.12:0/3751156754 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 16) ==== 153250+0+0 (secure 0 0 0) 0x7f8684010430 con 0x7f869410a9b0 2024-09-11T00:04:06.622 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:06.622+0000 7f868ea00640 1 --2- 172.21.15.12:0/3751156754 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f867c0b5360 0x7f867c0b7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:04:06.623 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:06.622+0000 7f868ea00640 1 -- 172.21.15.12:0/3751156754 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 7979+0+0 (secure 0 0 0) 0x7f86840e5ea0 con 0x7f869410a9b0 2024-09-11T00:04:06.623 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:06.622+0000 7f8699e00640 1 --2- 172.21.15.12:0/3751156754 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f867c0b5360 0x7f867c0b7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:04:06.623 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:06.623+0000 7f8699e00640 1 --2- 172.21.15.12:0/3751156754 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f867c0b5360 0x7f867c0b7820 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f8688009fc0 tx=0x7f8688005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:04:06.625 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:06.624+0000 7f868ea00640 1 -- 172.21.15.12:0/3751156754 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f86840ace10 con 0x7f869410a9b0 2024-09-11T00:04:06.833 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:06.832+0000 7f869c800640 1 -- 172.21.15.12:0/3751156754 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]} v 0) -- 0x7f869402e5c0 con 0x7f869410a9b0 2024-09-11T00:04:06.848 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:06.848+0000 7f868ea00640 1 -- 172.21.15.12:0/3751156754 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]=0 v75) ==== 157+0+0 (secure 0 0 0) 0x7f86840ac710 con 0x7f869410a9b0 2024-09-11T00:04:06.852 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:06.851+0000 7f869c800640 1 -- 172.21.15.12:0/3751156754 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f867c0b5360 msgr2=0x7f867c0b7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:04:06.852 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:06.851+0000 7f869c800640 1 --2- 172.21.15.12:0/3751156754 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f867c0b5360 0x7f867c0b7820 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f8688009fc0 tx=0x7f8688005f90 comp rx=0 tx=0).stop 2024-09-11T00:04:06.852 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:06.851+0000 7f869c800640 1 -- 172.21.15.12:0/3751156754 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f869410a9b0 msgr2=0x7f869419b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:04:06.853 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:06.851+0000 7f869c800640 1 --2- 172.21.15.12:0/3751156754 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f869410a9b0 0x7f869419b980 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f868400e9f0 tx=0x7f868400eeb0 comp rx=0 tx=0).stop 2024-09-11T00:04:06.853 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:06.851+0000 7f869c800640 1 -- 172.21.15.12:0/3751156754 shutdown_connections 2024-09-11T00:04:06.853 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:06.851+0000 7f869c800640 1 --2- 172.21.15.12:0/3751156754 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f867c0b5360 0x7f867c0b7820 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:04:06.853 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:06.851+0000 7f869c800640 1 --2- 172.21.15.12:0/3751156754 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f869410b330 0x7f86941a0d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:04:06.853 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:06.851+0000 7f869c800640 1 --2- 172.21.15.12:0/3751156754 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f869410a9b0 0x7f869419b980 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:04:06.853 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:06.851+0000 7f869c800640 1 --2- 172.21.15.12:0/3751156754 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8694104fa0 0x7f869419b440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:04:06.853 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:06.851+0000 7f869c800640 1 -- 172.21.15.12:0/3751156754 >> 172.21.15.12:0/3751156754 conn(0x7f8694100a50 msgr2=0x7f8694102e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:04:06.853 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:06.851+0000 7f869c800640 1 -- 172.21.15.12:0/3751156754 shutdown_connections 2024-09-11T00:04:06.853 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:06.852+0000 7f869c800640 1 -- 172.21.15.12:0/3751156754 wait complete. 2024-09-11T00:04:06.854 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:04:06 smithi012 ceph-mon[30333]: mds.? [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] up:active 2024-09-11T00:04:06.854 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:04:06 smithi012 ceph-mon[30333]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-11T00:04:06.854 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:04:06 smithi012 ceph-mon[30333]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-11T00:04:06.854 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:04:06 smithi012 ceph-mon[30333]: from='client.? 172.21.15.12:0/3751156754' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]} : dispatch 2024-09-11T00:04:07.183 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:04:06 smithi165 ceph-mon[35356]: mds.? [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] up:active 2024-09-11T00:04:07.184 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:04:06 smithi165 ceph-mon[35356]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-11T00:04:07.184 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:04:06 smithi165 ceph-mon[35356]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-11T00:04:07.184 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:04:06 smithi165 ceph-mon[35356]: from='client.? 172.21.15.12:0/3751156754' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]} : dispatch 2024-09-11T00:04:07.232 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:04:06 smithi178 ceph-mon[35919]: mds.? [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] up:active 2024-09-11T00:04:07.232 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:04:06 smithi178 ceph-mon[35919]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-11T00:04:07.232 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:04:06 smithi178 ceph-mon[35919]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-11T00:04:07.232 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:04:06 smithi178 ceph-mon[35919]: from='client.? 172.21.15.12:0/3751156754' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]} : dispatch 2024-09-11T00:04:07.694 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid ff3ad74a-6fcf-11ef-bcea-c7b262605968 -- ceph osd pool create cephfs_data_ec erasure cephfs_data_ec --pg_num_min 64 --target_size_ratio 0.9 2024-09-11T00:04:08.063 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:04:07 smithi012 ceph-mon[30333]: pgmap v298: 97 pgs: 6 creating+peering, 12 unknown, 79 active+clean; 578 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s wr, 3 op/s 2024-09-11T00:04:08.063 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:04:07 smithi012 ceph-mon[30333]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-09-11T00:04:08.063 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:04:07 smithi012 ceph-mon[30333]: Cluster is now healthy 2024-09-11T00:04:08.063 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:04:07 smithi012 ceph-mon[30333]: from='client.? 172.21.15.12:0/3751156754' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-09-11T00:04:08.063 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:04:07 smithi012 ceph-mon[30333]: osdmap e75: 12 total, 12 up, 12 in 2024-09-11T00:04:08.079 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:04:08.249 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:04:07 smithi178 ceph-mon[35919]: pgmap v298: 97 pgs: 6 creating+peering, 12 unknown, 79 active+clean; 578 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s wr, 3 op/s 2024-09-11T00:04:08.249 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:04:07 smithi178 ceph-mon[35919]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-09-11T00:04:08.249 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:04:07 smithi178 ceph-mon[35919]: Cluster is now healthy 2024-09-11T00:04:08.250 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:04:07 smithi178 ceph-mon[35919]: from='client.? 172.21.15.12:0/3751156754' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-09-11T00:04:08.250 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:04:07 smithi178 ceph-mon[35919]: osdmap e75: 12 total, 12 up, 12 in 2024-09-11T00:04:08.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:04:07 smithi165 ceph-mon[35356]: pgmap v298: 97 pgs: 6 creating+peering, 12 unknown, 79 active+clean; 578 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s wr, 3 op/s 2024-09-11T00:04:08.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:04:07 smithi165 ceph-mon[35356]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-09-11T00:04:08.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:04:07 smithi165 ceph-mon[35356]: Cluster is now healthy 2024-09-11T00:04:08.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:04:07 smithi165 ceph-mon[35356]: from='client.? 172.21.15.12:0/3751156754' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-09-11T00:04:08.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:04:07 smithi165 ceph-mon[35356]: osdmap e75: 12 total, 12 up, 12 in 2024-09-11T00:04:08.283 INFO:journalctl@ceph.mds.b.smithi165.stdout:Sep 11 00:04:07 smithi165 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-mds-b[65581]: 2024-09-11T00:04:07.924+0000 7fcd51d71640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-11T00:04:08.657 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:08.652+0000 7ff13e400640 1 -- 172.21.15.12:0/1086191626 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff1381087b0 msgr2=0x7ff138108bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:04:08.657 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:08.652+0000 7ff13e400640 1 --2- 172.21.15.12:0/1086191626 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff1381087b0 0x7ff138108bf0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7ff120009900 tx=0x7ff12002a3f0 comp rx=0 tx=0).stop 2024-09-11T00:04:08.657 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:08.652+0000 7ff13e400640 1 -- 172.21.15.12:0/1086191626 shutdown_connections 2024-09-11T00:04:08.657 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:08.652+0000 7ff13e400640 1 --2- 172.21.15.12:0/1086191626 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff138109130 0x7ff13810f930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:04:08.657 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:08.652+0000 7ff13e400640 1 --2- 172.21.15.12:0/1086191626 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff1381087b0 0x7ff138108bf0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:04:08.657 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:08.652+0000 7ff13e400640 1 --2- 172.21.15.12:0/1086191626 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff138102da0 0x7ff138103180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:04:08.657 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:08.652+0000 7ff13e400640 1 -- 172.21.15.12:0/1086191626 >> 172.21.15.12:0/1086191626 conn(0x7ff1380fe810 msgr2=0x7ff138100c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:04:08.657 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:08.652+0000 7ff13e400640 1 -- 172.21.15.12:0/1086191626 shutdown_connections 2024-09-11T00:04:08.657 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:08.652+0000 7ff13e400640 1 -- 172.21.15.12:0/1086191626 wait complete. 2024-09-11T00:04:08.658 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:08.653+0000 7ff13e400640 1 Processor -- start 2024-09-11T00:04:08.658 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:08.654+0000 7ff13e400640 1 -- start start 2024-09-11T00:04:08.658 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:08.654+0000 7ff13e400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff138102da0 0x7ff138199280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:04:08.658 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:08.654+0000 7ff13e400640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff1381087b0 0x7ff1381997c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:04:08.658 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:08.654+0000 7ff13e400640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff138109130 0x7ff13819eb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:04:08.658 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:08.654+0000 7ff13e400640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7ff138112dd0 con 0x7ff138109130 2024-09-11T00:04:08.658 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:08.654+0000 7ff13e400640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7ff138112c50 con 0x7ff138102da0 2024-09-11T00:04:08.658 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:08.654+0000 7ff13e400640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7ff138112f50 con 0x7ff1381087b0 2024-09-11T00:04:08.658 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:08.654+0000 7ff137400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff138102da0 0x7ff138199280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:04:08.658 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:08.654+0000 7ff12ea00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff1381087b0 0x7ff1381997c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:04:08.659 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:08.654+0000 7ff137400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff138102da0 0x7ff138199280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.12:46464/0 (socket says 172.21.15.12:46464) 2024-09-11T00:04:08.659 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:08.654+0000 7ff12ea00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff1381087b0 0x7ff1381997c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.12:43248/0 (socket says 172.21.15.12:43248) 2024-09-11T00:04:08.659 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:08.654+0000 7ff137400640 1 -- 172.21.15.12:0/2089982224 learned_addr learned my addr 172.21.15.12:0/2089982224 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:04:08.659 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:08.654+0000 7ff137e00640 1 --2- 172.21.15.12:0/2089982224 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff138109130 0x7ff13819eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:04:08.659 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:08.655+0000 7ff12ea00640 1 -- 172.21.15.12:0/2089982224 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff138102da0 msgr2=0x7ff138199280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:04:08.659 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:08.655+0000 7ff12ea00640 1 --2- 172.21.15.12:0/2089982224 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff138102da0 0x7ff138199280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:04:08.659 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:08.655+0000 7ff12ea00640 1 -- 172.21.15.12:0/2089982224 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff138109130 msgr2=0x7ff13819eb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:04:08.659 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:08.655+0000 7ff12ea00640 1 --2- 172.21.15.12:0/2089982224 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff138109130 0x7ff13819eb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:04:08.659 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:08.655+0000 7ff12ea00640 1 -- 172.21.15.12:0/2089982224 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff13819f330 con 0x7ff1381087b0 2024-09-11T00:04:08.659 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:08.655+0000 7ff137e00640 1 --2- 172.21.15.12:0/2089982224 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff138109130 0x7ff13819eb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:04:08.659 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:08.655+0000 7ff137400640 1 --2- 172.21.15.12:0/2089982224 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff138102da0 0x7ff138199280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:04:08.659 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:08.655+0000 7ff12ea00640 1 --2- 172.21.15.12:0/2089982224 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff1381087b0 0x7ff1381997c0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7ff12002a9e0 tx=0x7ff12003ac10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:04:08.659 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:08.655+0000 7ff134c00640 1 -- 172.21.15.12:0/2089982224 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff120035070 con 0x7ff1381087b0 2024-09-11T00:04:08.660 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:08.655+0000 7ff134c00640 1 -- 172.21.15.12:0/2089982224 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff120003cb0 con 0x7ff1381087b0 2024-09-11T00:04:08.660 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:08.655+0000 7ff134c00640 1 -- 172.21.15.12:0/2089982224 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff120031d80 con 0x7ff1381087b0 2024-09-11T00:04:08.660 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:08.655+0000 7ff13e400640 1 -- 172.21.15.12:0/2089982224 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff13819f5c0 con 0x7ff1381087b0 2024-09-11T00:04:08.660 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:08.656+0000 7ff13e400640 1 -- 172.21.15.12:0/2089982224 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff138075380 con 0x7ff1381087b0 2024-09-11T00:04:08.660 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:08.657+0000 7ff13e400640 1 -- 172.21.15.12:0/2089982224 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff138104050 con 0x7ff1381087b0 2024-09-11T00:04:08.660 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:08.659+0000 7ff134c00640 1 -- 172.21.15.12:0/2089982224 <== mon.1 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 16) ==== 153250+0+0 (secure 0 0 0) 0x7ff1200314a0 con 0x7ff1381087b0 2024-09-11T00:04:08.661 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:08.660+0000 7ff134c00640 1 --2- 172.21.15.12:0/2089982224 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7ff11c0b5360 0x7ff11c0b7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:04:08.662 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:08.660+0000 7ff134c00640 1 -- 172.21.15.12:0/2089982224 <== mon.1 v2:172.21.15.178:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 8277+0+0 (secure 0 0 0) 0x7ff120109ff0 con 0x7ff1381087b0 2024-09-11T00:04:08.662 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:08.660+0000 7ff137400640 1 --2- 172.21.15.12:0/2089982224 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7ff11c0b5360 0x7ff11c0b7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:04:08.662 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:08.661+0000 7ff137400640 1 --2- 172.21.15.12:0/2089982224 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7ff11c0b5360 0x7ff11c0b7820 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7ff1280095f0 tx=0x7ff128005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:04:08.663 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:08.662+0000 7ff134c00640 1 -- 172.21.15.12:0/2089982224 <== mon.1 v2:172.21.15.178:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7ff1200d0e40 con 0x7ff1381087b0 2024-09-11T00:04:08.858 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:08.857+0000 7ff13e400640 1 -- 172.21.15.12:0/2089982224 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7ff13810af70 con 0x7ff1381087b0 2024-09-11T00:04:10.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:04:09 smithi165 ceph-mon[35356]: pgmap v300: 97 pgs: 97 active+clean; 578 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s wr, 7 op/s 2024-09-11T00:04:10.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:04:09 smithi165 ceph-mon[35356]: from='client.? 172.21.15.12:0/2089982224' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9} : dispatch 2024-09-11T00:04:10.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:04:09 smithi165 ceph-mon[35356]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9} : dispatch 2024-09-11T00:04:10.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:04:09 smithi012 ceph-mon[30333]: pgmap v300: 97 pgs: 97 active+clean; 578 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s wr, 7 op/s 2024-09-11T00:04:10.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:04:09 smithi012 ceph-mon[30333]: from='client.? 172.21.15.12:0/2089982224' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9} : dispatch 2024-09-11T00:04:10.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:04:09 smithi012 ceph-mon[30333]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9} : dispatch 2024-09-11T00:04:10.330 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:04:09 smithi178 ceph-mon[35919]: pgmap v300: 97 pgs: 97 active+clean; 578 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s wr, 7 op/s 2024-09-11T00:04:10.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:04:09 smithi178 ceph-mon[35919]: from='client.? 172.21.15.12:0/2089982224' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9} : dispatch 2024-09-11T00:04:10.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:04:09 smithi178 ceph-mon[35919]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9} : dispatch 2024-09-11T00:04:10.856 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:10.852+0000 7ff134c00640 1 -- 172.21.15.12:0/2089982224 <== mon.1 v2:172.21.15.178:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data_ec' created v77) ==== 228+0+0 (secure 0 0 0) 0x7ff1200d0740 con 0x7ff1381087b0 2024-09-11T00:04:10.858 INFO:teuthology.orchestra.run.smithi012.stderr:pool 'cephfs_data_ec' created 2024-09-11T00:04:10.861 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:10.860+0000 7ff13e400640 1 -- 172.21.15.12:0/2089982224 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7ff11c0b5360 msgr2=0x7ff11c0b7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:04:10.862 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:10.860+0000 7ff13e400640 1 --2- 172.21.15.12:0/2089982224 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7ff11c0b5360 0x7ff11c0b7820 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7ff1280095f0 tx=0x7ff128005f90 comp rx=0 tx=0).stop 2024-09-11T00:04:10.862 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:10.860+0000 7ff13e400640 1 -- 172.21.15.12:0/2089982224 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff1381087b0 msgr2=0x7ff1381997c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:04:10.863 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:10.860+0000 7ff13e400640 1 --2- 172.21.15.12:0/2089982224 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff1381087b0 0x7ff1381997c0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7ff12002a9e0 tx=0x7ff12003ac10 comp rx=0 tx=0).stop 2024-09-11T00:04:10.863 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:10.861+0000 7ff13e400640 1 -- 172.21.15.12:0/2089982224 shutdown_connections 2024-09-11T00:04:10.863 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:10.861+0000 7ff13e400640 1 --2- 172.21.15.12:0/2089982224 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7ff11c0b5360 0x7ff11c0b7820 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:04:10.864 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:10.861+0000 7ff13e400640 1 --2- 172.21.15.12:0/2089982224 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff138109130 0x7ff13819eb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:04:10.864 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:10.861+0000 7ff13e400640 1 --2- 172.21.15.12:0/2089982224 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff1381087b0 0x7ff1381997c0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:04:10.864 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:10.861+0000 7ff13e400640 1 --2- 172.21.15.12:0/2089982224 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff138102da0 0x7ff138199280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:04:10.864 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:10.861+0000 7ff13e400640 1 -- 172.21.15.12:0/2089982224 >> 172.21.15.12:0/2089982224 conn(0x7ff1380fe810 msgr2=0x7ff138100c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:04:10.864 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:10.861+0000 7ff13e400640 1 -- 172.21.15.12:0/2089982224 shutdown_connections 2024-09-11T00:04:10.864 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:10.861+0000 7ff13e400640 1 -- 172.21.15.12:0/2089982224 wait complete. 2024-09-11T00:04:11.238 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:04:10 smithi012 ceph-mon[30333]: osdmap e76: 12 total, 12 up, 12 in 2024-09-11T00:04:11.238 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:04:10 smithi012 ceph-mon[30333]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-09-11T00:04:11.238 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:04:10 smithi012 ceph-mon[30333]: osdmap e77: 12 total, 12 up, 12 in 2024-09-11T00:04:11.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:04:10 smithi165 ceph-mon[35356]: osdmap e76: 12 total, 12 up, 12 in 2024-09-11T00:04:11.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:04:10 smithi165 ceph-mon[35356]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-09-11T00:04:11.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:04:10 smithi165 ceph-mon[35356]: osdmap e77: 12 total, 12 up, 12 in 2024-09-11T00:04:11.330 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:04:10 smithi178 ceph-mon[35919]: osdmap e76: 12 total, 12 up, 12 in 2024-09-11T00:04:11.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:04:10 smithi178 ceph-mon[35919]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-09-11T00:04:11.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:04:10 smithi178 ceph-mon[35919]: osdmap e77: 12 total, 12 up, 12 in 2024-09-11T00:04:11.714 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid ff3ad74a-6fcf-11ef-bcea-c7b262605968 -- ceph osd pool set cephfs_data_ec allow_ec_overwrites true 2024-09-11T00:04:12.120 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:04:12.258 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:04:11 smithi012 ceph-mon[30333]: pgmap v302: 97 pgs: 97 active+clean; 579 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s wr, 7 op/s 2024-09-11T00:04:12.258 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:04:11 smithi012 ceph-mon[30333]: osdmap e78: 12 total, 12 up, 12 in 2024-09-11T00:04:12.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:04:11 smithi165 ceph-mon[35356]: pgmap v302: 97 pgs: 97 active+clean; 579 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s wr, 7 op/s 2024-09-11T00:04:12.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:04:11 smithi165 ceph-mon[35356]: osdmap e78: 12 total, 12 up, 12 in 2024-09-11T00:04:12.330 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:04:11 smithi178 ceph-mon[35919]: pgmap v302: 97 pgs: 97 active+clean; 579 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s wr, 7 op/s 2024-09-11T00:04:12.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:04:11 smithi178 ceph-mon[35919]: osdmap e78: 12 total, 12 up, 12 in 2024-09-11T00:04:12.656 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:12.655+0000 7fb688a00640 1 -- 172.21.15.12:0/1811081564 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb684075540 msgr2=0x7fb684075920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:04:12.657 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:12.655+0000 7fb688a00640 1 --2- 172.21.15.12:0/1811081564 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb684075540 0x7fb684075920 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fb674009ca0 tx=0x7fb67402a8c0 comp rx=0 tx=0).stop 2024-09-11T00:04:12.657 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:12.655+0000 7fb688a00640 1 -- 172.21.15.12:0/1811081564 shutdown_connections 2024-09-11T00:04:12.657 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:12.655+0000 7fb688a00640 1 --2- 172.21.15.12:0/1811081564 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb684113b20 0x7fb684115f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:04:12.657 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:12.655+0000 7fb688a00640 1 --2- 172.21.15.12:0/1811081564 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb684075e60 0x7fb6841135e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:04:12.657 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:12.655+0000 7fb688a00640 1 --2- 172.21.15.12:0/1811081564 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb684075540 0x7fb684075920 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:04:12.657 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:12.655+0000 7fb688a00640 1 -- 172.21.15.12:0/1811081564 >> 172.21.15.12:0/1811081564 conn(0x7fb6840fe830 msgr2=0x7fb684100c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:04:12.657 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:12.655+0000 7fb688a00640 1 -- 172.21.15.12:0/1811081564 shutdown_connections 2024-09-11T00:04:12.657 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:12.655+0000 7fb688a00640 1 -- 172.21.15.12:0/1811081564 wait complete. 2024-09-11T00:04:12.659 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:12.656+0000 7fb688a00640 1 Processor -- start 2024-09-11T00:04:12.659 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:12.656+0000 7fb688a00640 1 -- start start 2024-09-11T00:04:12.659 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:12.657+0000 7fb688a00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb684075540 0x7fb68419f900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:04:12.659 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:12.657+0000 7fb688a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb684075e60 0x7fb68419fe40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:04:12.659 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:12.657+0000 7fb688a00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb684113b20 0x7fb6841a51e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:04:12.659 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:12.657+0000 7fb688a00640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fb684119410 con 0x7fb684113b20 2024-09-11T00:04:12.659 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:12.657+0000 7fb688a00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fb684119290 con 0x7fb684075e60 2024-09-11T00:04:12.659 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:12.657+0000 7fb688a00640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fb684119590 con 0x7fb684075540 2024-09-11T00:04:12.659 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:12.657+0000 7fb681e00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb684075540 0x7fb68419f900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:04:12.659 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:12.657+0000 7fb681e00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb684075540 0x7fb68419f900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.12:54188/0 (socket says 172.21.15.12:54188) 2024-09-11T00:04:12.659 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:12.657+0000 7fb681e00640 1 -- 172.21.15.12:0/759403607 learned_addr learned my addr 172.21.15.12:0/759403607 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:04:12.660 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:12.657+0000 7fb682800640 1 --2- 172.21.15.12:0/759403607 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb684113b20 0x7fb6841a51e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:04:12.660 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:12.657+0000 7fb679400640 1 --2- 172.21.15.12:0/759403607 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb684075e60 0x7fb68419fe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:04:12.660 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:12.658+0000 7fb681e00640 1 -- 172.21.15.12:0/759403607 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb684075e60 msgr2=0x7fb68419fe40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:04:12.660 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:12.658+0000 7fb681e00640 1 --2- 172.21.15.12:0/759403607 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb684075e60 0x7fb68419fe40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:04:12.660 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:12.658+0000 7fb681e00640 1 -- 172.21.15.12:0/759403607 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb684113b20 msgr2=0x7fb6841a51e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:04:12.660 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:12.658+0000 7fb681e00640 1 --2- 172.21.15.12:0/759403607 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb684113b20 0x7fb6841a51e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:04:12.660 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:12.658+0000 7fb681e00640 1 -- 172.21.15.12:0/759403607 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6841a59b0 con 0x7fb684075540 2024-09-11T00:04:12.660 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:12.658+0000 7fb681e00640 1 --2- 172.21.15.12:0/759403607 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb684075540 0x7fb68419f900 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fb674005e30 tx=0x7fb674031840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:04:12.661 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:12.658+0000 7fb67b400640 1 -- 172.21.15.12:0/759403607 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb674003b70 con 0x7fb684075540 2024-09-11T00:04:12.661 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:12.658+0000 7fb67b400640 1 -- 172.21.15.12:0/759403607 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb674003d10 con 0x7fb684075540 2024-09-11T00:04:12.661 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:12.658+0000 7fb688a00640 1 -- 172.21.15.12:0/759403607 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb6841a5c40 con 0x7fb684075540 2024-09-11T00:04:12.661 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:12.658+0000 7fb67b400640 1 -- 172.21.15.12:0/759403607 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb67402fa30 con 0x7fb684075540 2024-09-11T00:04:12.661 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:12.658+0000 7fb688a00640 1 -- 172.21.15.12:0/759403607 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6841ac360 con 0x7fb684075540 2024-09-11T00:04:12.662 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:12.659+0000 7fb688a00640 1 -- 172.21.15.12:0/759403607 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb684075920 con 0x7fb684075540 2024-09-11T00:04:12.663 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:12.662+0000 7fb67b400640 1 -- 172.21.15.12:0/759403607 <== mon.1 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 16) ==== 153250+0+0 (secure 0 0 0) 0x7fb67402fbd0 con 0x7fb684075540 2024-09-11T00:04:12.664 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:12.663+0000 7fb67b400640 1 --2- 172.21.15.12:0/759403607 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fb6680b5360 0x7fb6680b7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:04:12.665 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:12.663+0000 7fb67b400640 1 -- 172.21.15.12:0/759403607 <== mon.1 v2:172.21.15.178:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8770+0+0 (secure 0 0 0) 0x7fb674106fb0 con 0x7fb684075540 2024-09-11T00:04:12.665 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:12.664+0000 7fb679400640 1 --2- 172.21.15.12:0/759403607 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fb6680b5360 0x7fb6680b7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:04:12.665 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:12.664+0000 7fb679400640 1 --2- 172.21.15.12:0/759403607 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fb6680b5360 0x7fb6680b7820 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fb6841160b0 tx=0x7fb66c005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:04:12.666 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:12.665+0000 7fb67b400640 1 -- 172.21.15.12:0/759403607 <== mon.1 v2:172.21.15.178:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7fb6740cdbf0 con 0x7fb684075540 2024-09-11T00:04:12.865 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:12.864+0000 7fb688a00640 1 -- 172.21.15.12:0/759403607 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"} v 0) -- 0x7fb6841a0a30 con 0x7fb684075540 2024-09-11T00:04:13.859 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:13.857+0000 7fb67b400640 1 -- 172.21.15.12:0/759403607 <== mon.1 v2:172.21.15.178:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]=0 set pool 4 allow_ec_overwrites to true v80) ==== 169+0+0 (secure 0 0 0) 0x7fb6740cd4f0 con 0x7fb684075540 2024-09-11T00:04:13.859 INFO:teuthology.orchestra.run.smithi012.stderr:set pool 4 allow_ec_overwrites to true 2024-09-11T00:04:13.862 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:13.861+0000 7fb688a00640 1 -- 172.21.15.12:0/759403607 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fb6680b5360 msgr2=0x7fb6680b7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:04:13.862 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:13.861+0000 7fb688a00640 1 --2- 172.21.15.12:0/759403607 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fb6680b5360 0x7fb6680b7820 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fb6841160b0 tx=0x7fb66c005f90 comp rx=0 tx=0).stop 2024-09-11T00:04:13.862 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:13.861+0000 7fb688a00640 1 -- 172.21.15.12:0/759403607 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb684075540 msgr2=0x7fb68419f900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:04:13.862 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:13.861+0000 7fb688a00640 1 --2- 172.21.15.12:0/759403607 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb684075540 0x7fb68419f900 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fb674005e30 tx=0x7fb674031840 comp rx=0 tx=0).stop 2024-09-11T00:04:13.862 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:13.861+0000 7fb688a00640 1 -- 172.21.15.12:0/759403607 shutdown_connections 2024-09-11T00:04:13.862 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:13.861+0000 7fb688a00640 1 --2- 172.21.15.12:0/759403607 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fb6680b5360 0x7fb6680b7820 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:04:13.862 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:13.861+0000 7fb688a00640 1 --2- 172.21.15.12:0/759403607 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb684113b20 0x7fb6841a51e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:04:13.863 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:13.861+0000 7fb688a00640 1 --2- 172.21.15.12:0/759403607 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb684075e60 0x7fb68419fe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:04:13.863 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:13.861+0000 7fb688a00640 1 --2- 172.21.15.12:0/759403607 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb684075540 0x7fb68419f900 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:04:13.863 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:13.861+0000 7fb688a00640 1 -- 172.21.15.12:0/759403607 >> 172.21.15.12:0/759403607 conn(0x7fb6840fe830 msgr2=0x7fb6840ff430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:04:13.863 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:13.861+0000 7fb688a00640 1 -- 172.21.15.12:0/759403607 shutdown_connections 2024-09-11T00:04:13.863 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:13.861+0000 7fb688a00640 1 -- 172.21.15.12:0/759403607 wait complete. 2024-09-11T00:04:14.209 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:04:13 smithi012 ceph-mon[30333]: pgmap v305: 129 pgs: 1 creating+activating, 1 creating+peering, 29 unknown, 98 active+clean; 579 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s wr, 1 op/s 2024-09-11T00:04:14.209 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:04:13 smithi012 ceph-mon[30333]: osdmap e79: 12 total, 12 up, 12 in 2024-09-11T00:04:14.209 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:04:13 smithi012 ceph-mon[30333]: from='client.? 172.21.15.12:0/759403607' entity='client.admin' cmd={"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"} : dispatch 2024-09-11T00:04:14.209 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:04:13 smithi012 ceph-mon[30333]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"} : dispatch 2024-09-11T00:04:14.209 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:04:13 smithi012 ceph-mon[30333]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T00:04:14.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:04:13 smithi165 ceph-mon[35356]: pgmap v305: 129 pgs: 1 creating+activating, 1 creating+peering, 29 unknown, 98 active+clean; 579 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s wr, 1 op/s 2024-09-11T00:04:14.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:04:13 smithi165 ceph-mon[35356]: osdmap e79: 12 total, 12 up, 12 in 2024-09-11T00:04:14.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:04:13 smithi165 ceph-mon[35356]: from='client.? 172.21.15.12:0/759403607' entity='client.admin' cmd={"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"} : dispatch 2024-09-11T00:04:14.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:04:13 smithi165 ceph-mon[35356]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"} : dispatch 2024-09-11T00:04:14.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:04:13 smithi165 ceph-mon[35356]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T00:04:14.330 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:04:13 smithi178 ceph-mon[35919]: pgmap v305: 129 pgs: 1 creating+activating, 1 creating+peering, 29 unknown, 98 active+clean; 579 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s wr, 1 op/s 2024-09-11T00:04:14.330 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:04:13 smithi178 ceph-mon[35919]: osdmap e79: 12 total, 12 up, 12 in 2024-09-11T00:04:14.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:04:13 smithi178 ceph-mon[35919]: from='client.? 172.21.15.12:0/759403607' entity='client.admin' cmd={"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"} : dispatch 2024-09-11T00:04:14.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:04:13 smithi178 ceph-mon[35919]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"} : dispatch 2024-09-11T00:04:14.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:04:13 smithi178 ceph-mon[35919]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T00:04:14.625 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid ff3ad74a-6fcf-11ef-bcea-c7b262605968 -- ceph fs add_data_pool cephfs cephfs_data_ec 2024-09-11T00:04:15.009 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:04:15.149 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:04:14 smithi012 ceph-mon[30333]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-09-11T00:04:15.149 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:04:14 smithi012 ceph-mon[30333]: osdmap e80: 12 total, 12 up, 12 in 2024-09-11T00:04:15.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:04:14 smithi165 ceph-mon[35356]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-09-11T00:04:15.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:04:14 smithi165 ceph-mon[35356]: osdmap e80: 12 total, 12 up, 12 in 2024-09-11T00:04:15.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:04:14 smithi178 ceph-mon[35919]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-09-11T00:04:15.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:04:14 smithi178 ceph-mon[35919]: osdmap e80: 12 total, 12 up, 12 in 2024-09-11T00:04:15.551 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:15.549+0000 7f1cb9400640 1 -- 172.21.15.12:0/678241611 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1cb4109150 msgr2=0x7f1cb410f950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:04:15.551 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:15.549+0000 7f1cb9400640 1 --2- 172.21.15.12:0/678241611 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1cb4109150 0x7f1cb410f950 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f1cac009ca0 tx=0x7f1cac02a8c0 comp rx=0 tx=0).stop 2024-09-11T00:04:15.552 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:15.549+0000 7f1cb9400640 1 -- 172.21.15.12:0/678241611 shutdown_connections 2024-09-11T00:04:15.552 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:15.549+0000 7f1cb9400640 1 --2- 172.21.15.12:0/678241611 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1cb4109150 0x7f1cb410f950 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:04:15.552 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:15.549+0000 7f1cb9400640 1 --2- 172.21.15.12:0/678241611 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1cb41087d0 0x7f1cb4108c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:04:15.552 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:15.549+0000 7f1cb9400640 1 --2- 172.21.15.12:0/678241611 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1cb4102dc0 0x7f1cb41031a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:04:15.552 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:15.549+0000 7f1cb9400640 1 -- 172.21.15.12:0/678241611 >> 172.21.15.12:0/678241611 conn(0x7f1cb40fe830 msgr2=0x7f1cb4100c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:04:15.552 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:15.550+0000 7f1cb9400640 1 -- 172.21.15.12:0/678241611 shutdown_connections 2024-09-11T00:04:15.552 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:15.550+0000 7f1cb9400640 1 -- 172.21.15.12:0/678241611 wait complete. 2024-09-11T00:04:15.552 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:15.550+0000 7f1cb9400640 1 Processor -- start 2024-09-11T00:04:15.552 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:15.551+0000 7f1cb9400640 1 -- start start 2024-09-11T00:04:15.553 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:15.551+0000 7f1cb9400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1cb4102dc0 0x7f1cb4199240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:04:15.553 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:15.551+0000 7f1cb9400640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1cb41087d0 0x7f1cb4199780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:04:15.554 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:15.551+0000 7f1cb9400640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1cb4109150 0x7f1cb419eb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:04:15.554 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:15.551+0000 7f1cb9400640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f1cb4112dd0 con 0x7f1cb41087d0 2024-09-11T00:04:15.554 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:15.551+0000 7f1cb9400640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f1cb4112c50 con 0x7f1cb4102dc0 2024-09-11T00:04:15.554 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:15.551+0000 7f1cb9400640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f1cb4112f50 con 0x7f1cb4109150 2024-09-11T00:04:15.554 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:15.551+0000 7f1cb1e00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1cb41087d0 0x7f1cb4199780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:04:15.554 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:15.551+0000 7f1cb2800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1cb4102dc0 0x7f1cb4199240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:04:15.554 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:15.551+0000 7f1cb2800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1cb4102dc0 0x7f1cb4199240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.12:34840/0 (socket says 172.21.15.12:34840) 2024-09-11T00:04:15.554 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:15.551+0000 7f1cb3200640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1cb4109150 0x7f1cb419eb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:04:15.554 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:15.551+0000 7f1cb2800640 1 -- 172.21.15.12:0/2588330900 learned_addr learned my addr 172.21.15.12:0/2588330900 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:04:15.554 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:15.551+0000 7f1cb1e00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1cb41087d0 0x7f1cb4199780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:51126/0 (socket says 172.21.15.12:51126) 2024-09-11T00:04:15.554 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:15.552+0000 7f1cb2800640 1 -- 172.21.15.12:0/2588330900 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1cb4109150 msgr2=0x7f1cb419eb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:04:15.554 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:15.552+0000 7f1cb2800640 1 --2- 172.21.15.12:0/2588330900 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1cb4109150 0x7f1cb419eb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:04:15.554 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:15.552+0000 7f1cb2800640 1 -- 172.21.15.12:0/2588330900 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1cb41087d0 msgr2=0x7f1cb4199780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:04:15.555 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:15.552+0000 7f1cb2800640 1 --2- 172.21.15.12:0/2588330900 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1cb41087d0 0x7f1cb4199780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:04:15.555 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:15.552+0000 7f1cb2800640 1 -- 172.21.15.12:0/2588330900 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1cb419f2f0 con 0x7f1cb4102dc0 2024-09-11T00:04:15.555 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:15.552+0000 7f1cb1e00640 1 --2- 172.21.15.12:0/2588330900 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1cb41087d0 0x7f1cb4199780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:04:15.555 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:15.552+0000 7f1cb2800640 1 --2- 172.21.15.12:0/2588330900 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1cb4102dc0 0x7f1cb4199240 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f1ca400d950 tx=0x7f1ca400de10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:04:15.555 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:15.552+0000 7f1cab400640 1 -- 172.21.15.12:0/2588330900 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ca4017030 con 0x7f1cb4102dc0 2024-09-11T00:04:15.555 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:15.552+0000 7f1cb9400640 1 -- 172.21.15.12:0/2588330900 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1cb419f5e0 con 0x7f1cb4102dc0 2024-09-11T00:04:15.555 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:15.553+0000 7f1cab400640 1 -- 172.21.15.12:0/2588330900 <== mon.2 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1ca4004020 con 0x7f1cb4102dc0 2024-09-11T00:04:15.555 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:15.553+0000 7f1cab400640 1 -- 172.21.15.12:0/2588330900 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ca40047a0 con 0x7f1cb4102dc0 2024-09-11T00:04:15.555 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:15.553+0000 7f1cb9400640 1 -- 172.21.15.12:0/2588330900 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1cb41a5e70 con 0x7f1cb4102dc0 2024-09-11T00:04:15.556 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:15.554+0000 7f1cb9400640 1 -- 172.21.15.12:0/2588330900 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1cb41031a0 con 0x7f1cb4102dc0 2024-09-11T00:04:15.557 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:15.556+0000 7f1cab400640 1 -- 172.21.15.12:0/2588330900 <== mon.2 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 16) ==== 153250+0+0 (secure 0 0 0) 0x7f1ca4009a30 con 0x7f1cb4102dc0 2024-09-11T00:04:15.558 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:15.557+0000 7f1cab400640 1 --2- 172.21.15.12:0/2588330900 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f1c980b52e0 0x7f1c980b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:04:15.558 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:15.557+0000 7f1cab400640 1 -- 172.21.15.12:0/2588330900 <== mon.2 v2:172.21.15.165:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8770+0+0 (secure 0 0 0) 0x7f1ca40e5e50 con 0x7f1cb4102dc0 2024-09-11T00:04:15.558 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:15.558+0000 7f1cb1e00640 1 --2- 172.21.15.12:0/2588330900 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f1c980b52e0 0x7f1c980b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:04:15.559 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:15.558+0000 7f1cb1e00640 1 --2- 172.21.15.12:0/2588330900 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f1c980b52e0 0x7f1c980b77a0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f1cb4104ea0 tx=0x7f1ca0005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:04:15.560 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:15.559+0000 7f1cab400640 1 -- 172.21.15.12:0/2588330900 <== mon.2 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f1ca40acaf0 con 0x7f1cb4102dc0 2024-09-11T00:04:15.794 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:15.794+0000 7f1cb9400640 1 -- 172.21.15.12:0/2588330900 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"} v 0) -- 0x7f1cb419a370 con 0x7f1cb4102dc0 2024-09-11T00:04:15.872 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:15.871+0000 7f1cab400640 1 -- 172.21.15.12:0/2588330900 <== mon.2 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]=0 added data pool 4 to fsmap v17) ==== 137+0+0 (secure 0 0 0) 0x7f1ca40ac3f0 con 0x7f1cb4102dc0 2024-09-11T00:04:15.872 INFO:teuthology.orchestra.run.smithi012.stderr:added data pool 4 to fsmap 2024-09-11T00:04:15.874 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:15.873+0000 7f1cb9400640 1 -- 172.21.15.12:0/2588330900 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f1c980b52e0 msgr2=0x7f1c980b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:04:15.874 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:15.873+0000 7f1cb9400640 1 --2- 172.21.15.12:0/2588330900 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f1c980b52e0 0x7f1c980b77a0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f1cb4104ea0 tx=0x7f1ca0005f90 comp rx=0 tx=0).stop 2024-09-11T00:04:15.874 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:15.873+0000 7f1cb9400640 1 -- 172.21.15.12:0/2588330900 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1cb4102dc0 msgr2=0x7f1cb4199240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:04:15.874 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:15.873+0000 7f1cb9400640 1 --2- 172.21.15.12:0/2588330900 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1cb4102dc0 0x7f1cb4199240 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f1ca400d950 tx=0x7f1ca400de10 comp rx=0 tx=0).stop 2024-09-11T00:04:15.874 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:15.873+0000 7f1cb9400640 1 -- 172.21.15.12:0/2588330900 shutdown_connections 2024-09-11T00:04:15.874 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:15.873+0000 7f1cb9400640 1 --2- 172.21.15.12:0/2588330900 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f1c980b52e0 0x7f1c980b77a0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:04:15.875 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:15.873+0000 7f1cb9400640 1 --2- 172.21.15.12:0/2588330900 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1cb4109150 0x7f1cb419eb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:04:15.875 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:15.873+0000 7f1cb9400640 1 --2- 172.21.15.12:0/2588330900 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1cb41087d0 0x7f1cb4199780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:04:15.875 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:15.873+0000 7f1cb9400640 1 --2- 172.21.15.12:0/2588330900 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1cb4102dc0 0x7f1cb4199240 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:04:15.875 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:15.873+0000 7f1cb9400640 1 -- 172.21.15.12:0/2588330900 >> 172.21.15.12:0/2588330900 conn(0x7f1cb40fe830 msgr2=0x7f1cb40ff120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:04:15.875 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:15.873+0000 7f1cb9400640 1 -- 172.21.15.12:0/2588330900 shutdown_connections 2024-09-11T00:04:15.875 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:15.873+0000 7f1cb9400640 1 -- 172.21.15.12:0/2588330900 wait complete. 2024-09-11T00:04:16.063 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:04:15 smithi012 ceph-mon[30333]: pgmap v308: 129 pgs: 1 creating+activating, 1 creating+peering, 27 unknown, 100 active+clean; 579 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T00:04:16.063 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:04:15 smithi012 ceph-mon[30333]: from='client.? 172.21.15.12:0/2588330900' entity='client.admin' cmd={"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"} : dispatch 2024-09-11T00:04:16.063 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:04:15 smithi012 ceph-mon[30333]: from='client.? ' entity='client.admin' cmd={"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"} : dispatch 2024-09-11T00:04:16.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:04:15 smithi165 ceph-mon[35356]: pgmap v308: 129 pgs: 1 creating+activating, 1 creating+peering, 27 unknown, 100 active+clean; 579 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T00:04:16.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:04:15 smithi165 ceph-mon[35356]: from='client.? 172.21.15.12:0/2588330900' entity='client.admin' cmd={"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"} : dispatch 2024-09-11T00:04:16.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:04:15 smithi165 ceph-mon[35356]: from='client.? ' entity='client.admin' cmd={"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"} : dispatch 2024-09-11T00:04:16.330 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:04:15 smithi178 ceph-mon[35919]: pgmap v308: 129 pgs: 1 creating+activating, 1 creating+peering, 27 unknown, 100 active+clean; 579 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T00:04:16.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:04:15 smithi178 ceph-mon[35919]: from='client.? 172.21.15.12:0/2588330900' entity='client.admin' cmd={"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"} : dispatch 2024-09-11T00:04:16.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:04:15 smithi178 ceph-mon[35919]: from='client.? ' entity='client.admin' cmd={"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"} : dispatch 2024-09-11T00:04:16.606 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid ff3ad74a-6fcf-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T00:04:16.987 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:04:17.149 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:04:16 smithi012 ceph-mon[30333]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-09-11T00:04:17.149 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:04:16 smithi012 ceph-mon[30333]: osdmap e81: 12 total, 12 up, 12 in 2024-09-11T00:04:17.149 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:04:16 smithi012 ceph-mon[30333]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-11T00:04:17.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:04:16 smithi165 ceph-mon[35356]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-09-11T00:04:17.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:04:16 smithi165 ceph-mon[35356]: osdmap e81: 12 total, 12 up, 12 in 2024-09-11T00:04:17.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:04:16 smithi165 ceph-mon[35356]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-11T00:04:17.332 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:04:16 smithi178 ceph-mon[35919]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-09-11T00:04:17.332 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:04:16 smithi178 ceph-mon[35919]: osdmap e81: 12 total, 12 up, 12 in 2024-09-11T00:04:17.332 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:04:16 smithi178 ceph-mon[35919]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-11T00:04:17.557 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:17.555+0000 7f5972a00640 1 -- 172.21.15.12:0/2350677573 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f596c103cc0 msgr2=0x7f596c1040a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:04:17.557 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:17.555+0000 7f5972a00640 1 --2- 172.21.15.12:0/2350677573 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f596c103cc0 0x7f596c1040a0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f595c009900 tx=0x7f595c02a410 comp rx=0 tx=0).stop 2024-09-11T00:04:17.557 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:17.555+0000 7f5972a00640 1 -- 172.21.15.12:0/2350677573 shutdown_connections 2024-09-11T00:04:17.557 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:17.555+0000 7f5972a00640 1 --2- 172.21.15.12:0/2350677573 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f596c10d120 0x7f596c10f580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:04:17.557 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:17.555+0000 7f5972a00640 1 --2- 172.21.15.12:0/2350677573 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f596c1045e0 0x7f596c10cad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:04:17.557 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:17.555+0000 7f5972a00640 1 --2- 172.21.15.12:0/2350677573 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f596c103cc0 0x7f596c1040a0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:04:17.557 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:17.555+0000 7f5972a00640 1 -- 172.21.15.12:0/2350677573 >> 172.21.15.12:0/2350677573 conn(0x7f596c0fd700 msgr2=0x7f596c0ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:04:17.557 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:17.556+0000 7f5972a00640 1 -- 172.21.15.12:0/2350677573 shutdown_connections 2024-09-11T00:04:17.557 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:17.556+0000 7f5972a00640 1 -- 172.21.15.12:0/2350677573 wait complete. 2024-09-11T00:04:17.559 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:17.556+0000 7f5972a00640 1 Processor -- start 2024-09-11T00:04:17.559 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:17.557+0000 7f5972a00640 1 -- start start 2024-09-11T00:04:17.559 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:17.557+0000 7f5972a00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f596c103cc0 0x7f596c19b470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:04:17.559 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:17.557+0000 7f5972a00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f596c1045e0 0x7f596c19b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:04:17.559 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:17.557+0000 7f5972a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f596c10d120 0x7f596c1a0d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:04:17.559 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:17.557+0000 7f5972a00640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f596c112d90 con 0x7f596c103cc0 2024-09-11T00:04:17.559 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:17.557+0000 7f5972a00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f596c112c10 con 0x7f596c10d120 2024-09-11T00:04:17.559 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:17.557+0000 7f5972a00640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f596c112f10 con 0x7f596c1045e0 2024-09-11T00:04:17.560 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:17.557+0000 7f596be00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f596c103cc0 0x7f596c19b470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:04:17.560 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:17.557+0000 7f5970a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f596c10d120 0x7f596c1a0d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:04:17.560 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:17.557+0000 7f596be00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f596c103cc0 0x7f596c19b470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:51152/0 (socket says 172.21.15.12:51152) 2024-09-11T00:04:17.560 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:17.557+0000 7f596be00640 1 -- 172.21.15.12:0/1409796771 learned_addr learned my addr 172.21.15.12:0/1409796771 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:04:17.560 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:17.558+0000 7f596b400640 1 --2- 172.21.15.12:0/1409796771 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f596c1045e0 0x7f596c19b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:04:17.560 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:17.558+0000 7f5970a00640 1 -- 172.21.15.12:0/1409796771 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f596c1045e0 msgr2=0x7f596c19b9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:04:17.560 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:17.558+0000 7f5970a00640 1 --2- 172.21.15.12:0/1409796771 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f596c1045e0 0x7f596c19b9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:04:17.560 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:17.558+0000 7f5970a00640 1 -- 172.21.15.12:0/1409796771 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f596c103cc0 msgr2=0x7f596c19b470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:04:17.560 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:17.558+0000 7f5970a00640 1 --2- 172.21.15.12:0/1409796771 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f596c103cc0 0x7f596c19b470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:04:17.560 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:17.558+0000 7f5970a00640 1 -- 172.21.15.12:0/1409796771 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f596c1a1520 con 0x7f596c10d120 2024-09-11T00:04:17.560 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:17.558+0000 7f596b400640 1 --2- 172.21.15.12:0/1409796771 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f596c1045e0 0x7f596c19b9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T00:04:17.561 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:17.558+0000 7f5970a00640 1 --2- 172.21.15.12:0/1409796771 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f596c10d120 0x7f596c1a0d50 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f5964009fd0 tx=0x7f596400ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:04:17.561 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:17.558+0000 7f5968c00640 1 -- 172.21.15.12:0/1409796771 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f596400ce20 con 0x7f596c10d120 2024-09-11T00:04:17.561 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:17.558+0000 7f5972a00640 1 -- 172.21.15.12:0/1409796771 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f596c1a1810 con 0x7f596c10d120 2024-09-11T00:04:17.562 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:17.559+0000 7f5968c00640 1 -- 172.21.15.12:0/1409796771 <== mon.2 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5964005020 con 0x7f596c10d120 2024-09-11T00:04:17.562 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:17.559+0000 7f5968c00640 1 -- 172.21.15.12:0/1409796771 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5964010690 con 0x7f596c10d120 2024-09-11T00:04:17.562 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:17.559+0000 7f5972a00640 1 -- 172.21.15.12:0/1409796771 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f596c1a7ed0 con 0x7f596c10d120 2024-09-11T00:04:17.562 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:17.560+0000 7f5972a00640 1 -- 172.21.15.12:0/1409796771 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f596c1040a0 con 0x7f596c10d120 2024-09-11T00:04:17.563 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:17.562+0000 7f5968c00640 1 -- 172.21.15.12:0/1409796771 <== mon.2 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 16) ==== 153250+0+0 (secure 0 0 0) 0x7f59640054d0 con 0x7f596c10d120 2024-09-11T00:04:17.564 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:17.563+0000 7f5968c00640 1 --2- 172.21.15.12:0/1409796771 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f59500b52e0 0x7f59500b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:04:17.564 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:17.564+0000 7f5968c00640 1 -- 172.21.15.12:0/1409796771 <== mon.2 v2:172.21.15.165:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8802+0+0 (secure 0 0 0) 0x7f59640e6570 con 0x7f596c10d120 2024-09-11T00:04:17.565 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:17.564+0000 7f596be00640 1 --2- 172.21.15.12:0/1409796771 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f59500b52e0 0x7f59500b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:04:17.565 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:17.564+0000 7f596be00640 1 --2- 172.21.15.12:0/1409796771 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f59500b52e0 0x7f59500b77a0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f595c031440 tx=0x7f595c0058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:04:17.566 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:17.565+0000 7f5968c00640 1 -- 172.21.15.12:0/1409796771 <== mon.2 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f59640ad190 con 0x7f596c10d120 2024-09-11T00:04:17.798 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:17.797+0000 7f5972a00640 1 -- 172.21.15.12:0/1409796771 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f596c1a8300 con 0x7f596c10d120 2024-09-11T00:04:17.798 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:17.798+0000 7f5968c00640 1 -- 172.21.15.12:0/1409796771 <== mon.2 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+11830 (secure 0 0 0) 0x7f59640aca90 con 0x7f596c10d120 2024-09-11T00:04:17.799 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-11T00:04:17.799 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":17,"btime":"2024-09-11T00:04:15:865268+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14577,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6835/447171591","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":447171591},{"type":"v1","addr":"172.21.15.12:6835","nonce":447171591}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14589,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/901486040","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":901486040},{"type":"v1","addr":"172.21.15.12:6837","nonce":901486040}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14631,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6841/3069919437","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6840","nonce":3069919437},{"type":"v1","addr":"172.21.15.12:6841","nonce":3069919437}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24562,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6833/1141636112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":1141636112},{"type":"v1","addr":"172.21.15.178:6833","nonce":1141636112}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24581,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6833/3111980970","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6832","nonce":3111980970},{"type":"v1","addr":"172.21.15.165:6833","nonce":3111980970}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24607,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6835/4084868287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":4084868287},{"type":"v1","addr":"172.21.15.178:6835","nonce":4084868287}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24613,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6837/526874763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":526874763},{"type":"v1","addr":"172.21.15.178:6837","nonce":526874763}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24614,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6835/2425395274","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":2425395274},{"type":"v1","addr":"172.21.15.165:6835","nonce":2425395274}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24637,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6839/70628097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6838","nonce":70628097},{"type":"v1","addr":"172.21.15.178:6839","nonce":70628097}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24638,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6837/6215088","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6836","nonce":6215088},{"type":"v1","addr":"172.21.15.165:6837","nonce":6215088}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24643,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6839/3124093009","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6838","nonce":3124093009},{"type":"v1","addr":"172.21.15.12:6839","nonce":3124093009}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-11T00:04:04.739881+0000","modified":"2024-09-11T00:04:15.795786+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24674},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24674":{"gid":24674,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.165:6839/501312984","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6838","nonce":501312984},{"type":"v1","addr":"172.21.15.165:6839","nonce":501312984}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":24674,"qdb_cluster":[24674]},"id":1}]} 2024-09-11T00:04:17.800 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 17 2024-09-11T00:04:17.802 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:17.801+0000 7f5972a00640 1 -- 172.21.15.12:0/1409796771 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f59500b52e0 msgr2=0x7f59500b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:04:17.802 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:17.801+0000 7f5972a00640 1 --2- 172.21.15.12:0/1409796771 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f59500b52e0 0x7f59500b77a0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f595c031440 tx=0x7f595c0058d0 comp rx=0 tx=0).stop 2024-09-11T00:04:17.802 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:17.801+0000 7f5972a00640 1 -- 172.21.15.12:0/1409796771 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f596c10d120 msgr2=0x7f596c1a0d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:04:17.802 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:17.801+0000 7f5972a00640 1 --2- 172.21.15.12:0/1409796771 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f596c10d120 0x7f596c1a0d50 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f5964009fd0 tx=0x7f596400ef10 comp rx=0 tx=0).stop 2024-09-11T00:04:17.802 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:17.801+0000 7f5972a00640 1 -- 172.21.15.12:0/1409796771 shutdown_connections 2024-09-11T00:04:17.802 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:17.801+0000 7f5972a00640 1 --2- 172.21.15.12:0/1409796771 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f59500b52e0 0x7f59500b77a0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:04:17.802 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:17.801+0000 7f5972a00640 1 --2- 172.21.15.12:0/1409796771 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f596c10d120 0x7f596c1a0d50 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:04:17.802 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:17.801+0000 7f5972a00640 1 --2- 172.21.15.12:0/1409796771 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f596c1045e0 0x7f596c19b9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:04:17.802 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:17.801+0000 7f5972a00640 1 --2- 172.21.15.12:0/1409796771 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f596c103cc0 0x7f596c19b470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:04:17.803 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:17.801+0000 7f5972a00640 1 -- 172.21.15.12:0/1409796771 >> 172.21.15.12:0/1409796771 conn(0x7f596c0fd700 msgr2=0x7f596c10f370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:04:17.803 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:17.801+0000 7f5972a00640 1 -- 172.21.15.12:0/1409796771 shutdown_connections 2024-09-11T00:04:17.803 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:17.801+0000 7f5972a00640 1 -- 172.21.15.12:0/1409796771 wait complete. 2024-09-11T00:04:18.062 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:04:17 smithi012 ceph-mon[30333]: pgmap v310: 129 pgs: 1 creating+activating, 1 creating+peering, 127 active+clean; 579 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T00:04:18.063 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:04:17 smithi012 ceph-mon[30333]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-09-11T00:04:18.063 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:04:17 smithi012 ceph-mon[30333]: Cluster is now healthy 2024-09-11T00:04:18.063 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:04:17 smithi012 ceph-mon[30333]: from='client.? 172.21.15.12:0/1409796771' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T00:04:18.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:04:17 smithi165 ceph-mon[35356]: pgmap v310: 129 pgs: 1 creating+activating, 1 creating+peering, 127 active+clean; 579 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T00:04:18.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:04:17 smithi165 ceph-mon[35356]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-09-11T00:04:18.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:04:17 smithi165 ceph-mon[35356]: Cluster is now healthy 2024-09-11T00:04:18.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:04:17 smithi165 ceph-mon[35356]: from='client.? 172.21.15.12:0/1409796771' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T00:04:18.330 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:04:17 smithi178 ceph-mon[35919]: pgmap v310: 129 pgs: 1 creating+activating, 1 creating+peering, 127 active+clean; 579 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T00:04:18.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:04:17 smithi178 ceph-mon[35919]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-09-11T00:04:18.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:04:17 smithi178 ceph-mon[35919]: Cluster is now healthy 2024-09-11T00:04:18.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:04:17 smithi178 ceph-mon[35919]: from='client.? 172.21.15.12:0/1409796771' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T00:04:18.687 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid ff3ad74a-6fcf-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-11T00:04:19.068 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:04:19.619 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:19.618+0000 7ffa75600640 1 -- 172.21.15.12:0/1587760130 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ffa70104e10 msgr2=0x7ffa701051f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:04:19.619 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:19.618+0000 7ffa75600640 1 --2- 172.21.15.12:0/1587760130 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ffa70104e10 0x7ffa701051f0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7ffa60009900 tx=0x7ffa6002a410 comp rx=0 tx=0).stop 2024-09-11T00:04:19.620 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:19.618+0000 7ffa75600640 1 -- 172.21.15.12:0/1587760130 shutdown_connections 2024-09-11T00:04:19.620 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:19.618+0000 7ffa75600640 1 --2- 172.21.15.12:0/1587760130 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ffa7010b1a0 0x7ffa701119a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:04:19.620 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:19.618+0000 7ffa75600640 1 --2- 172.21.15.12:0/1587760130 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ffa7010a820 0x7ffa7010ac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:04:19.620 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:19.618+0000 7ffa75600640 1 --2- 172.21.15.12:0/1587760130 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ffa70104e10 0x7ffa701051f0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:04:19.620 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:19.618+0000 7ffa75600640 1 -- 172.21.15.12:0/1587760130 >> 172.21.15.12:0/1587760130 conn(0x7ffa70100840 msgr2=0x7ffa70102c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:04:19.620 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:19.618+0000 7ffa75600640 1 -- 172.21.15.12:0/1587760130 shutdown_connections 2024-09-11T00:04:19.620 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:19.618+0000 7ffa75600640 1 -- 172.21.15.12:0/1587760130 wait complete. 2024-09-11T00:04:19.621 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:19.619+0000 7ffa75600640 1 Processor -- start 2024-09-11T00:04:19.621 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:19.619+0000 7ffa75600640 1 -- start start 2024-09-11T00:04:19.621 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:19.620+0000 7ffa75600640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ffa70104e10 0x7ffa7019b240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:04:19.621 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:19.620+0000 7ffa75600640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ffa7010a820 0x7ffa7019b780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:04:19.622 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:19.620+0000 7ffa75600640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ffa7010b1a0 0x7ffa701a0b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:04:19.622 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:19.620+0000 7ffa75600640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7ffa70114eb0 con 0x7ffa70104e10 2024-09-11T00:04:19.622 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:19.620+0000 7ffa75600640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7ffa70114d30 con 0x7ffa7010b1a0 2024-09-11T00:04:19.622 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:19.620+0000 7ffa75600640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7ffa70115030 con 0x7ffa7010a820 2024-09-11T00:04:19.622 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:19.620+0000 7ffa6fe00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ffa70104e10 0x7ffa7019b240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:04:19.622 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:19.620+0000 7ffa6f400640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ffa7010a820 0x7ffa7019b780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:04:19.622 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:19.620+0000 7ffa6f400640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ffa7010a820 0x7ffa7019b780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.12:43112/0 (socket says 172.21.15.12:43112) 2024-09-11T00:04:19.622 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:19.620+0000 7ffa6fe00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ffa70104e10 0x7ffa7019b240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:50550/0 (socket says 172.21.15.12:50550) 2024-09-11T00:04:19.622 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:19.620+0000 7ffa6f400640 1 -- 172.21.15.12:0/3954532130 learned_addr learned my addr 172.21.15.12:0/3954532130 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:04:19.622 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:19.620+0000 7ffa74c00640 1 --2- 172.21.15.12:0/3954532130 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ffa7010b1a0 0x7ffa701a0b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:04:19.622 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:19.620+0000 7ffa6f400640 1 -- 172.21.15.12:0/3954532130 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ffa7010b1a0 msgr2=0x7ffa701a0b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:04:19.622 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:19.620+0000 7ffa6f400640 1 --2- 172.21.15.12:0/3954532130 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ffa7010b1a0 0x7ffa701a0b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:04:19.622 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:19.620+0000 7ffa6f400640 1 -- 172.21.15.12:0/3954532130 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ffa70104e10 msgr2=0x7ffa7019b240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:04:19.623 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:19.620+0000 7ffa6f400640 1 --2- 172.21.15.12:0/3954532130 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ffa70104e10 0x7ffa7019b240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:04:19.623 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:19.620+0000 7ffa6f400640 1 -- 172.21.15.12:0/3954532130 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa701a12f0 con 0x7ffa7010a820 2024-09-11T00:04:19.624 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:19.621+0000 7ffa6f400640 1 --2- 172.21.15.12:0/3954532130 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ffa7010a820 0x7ffa7019b780 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7ffa5c00ea60 tx=0x7ffa5c00ef20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:04:19.624 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:19.621+0000 7ffa6cc00640 1 -- 172.21.15.12:0/3954532130 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa5c00cde0 con 0x7ffa7010a820 2024-09-11T00:04:19.624 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:19.621+0000 7ffa75600640 1 -- 172.21.15.12:0/3954532130 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa701a15e0 con 0x7ffa7010a820 2024-09-11T00:04:19.624 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:19.621+0000 7ffa6cc00640 1 -- 172.21.15.12:0/3954532130 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffa5c0035f0 con 0x7ffa7010a820 2024-09-11T00:04:19.624 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:19.621+0000 7ffa6cc00640 1 -- 172.21.15.12:0/3954532130 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa5c010690 con 0x7ffa7010a820 2024-09-11T00:04:19.624 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:19.621+0000 7ffa75600640 1 -- 172.21.15.12:0/3954532130 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa701a7e70 con 0x7ffa7010a820 2024-09-11T00:04:19.624 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:19.622+0000 7ffa75600640 1 -- 172.21.15.12:0/3954532130 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffa701051f0 con 0x7ffa7010a820 2024-09-11T00:04:19.625 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:19.624+0000 7ffa6cc00640 1 -- 172.21.15.12:0/3954532130 <== mon.1 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 16) ==== 153250+0+0 (secure 0 0 0) 0x7ffa5c004000 con 0x7ffa7010a820 2024-09-11T00:04:19.627 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:19.626+0000 7ffa6cc00640 1 --2- 172.21.15.12:0/3954532130 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7ffa540b5410 0x7ffa540b78d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:04:19.627 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:19.626+0000 7ffa6cc00640 1 -- 172.21.15.12:0/3954532130 <== mon.1 v2:172.21.15.178:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8802+0+0 (secure 0 0 0) 0x7ffa5c0e6220 con 0x7ffa7010a820 2024-09-11T00:04:19.627 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:19.626+0000 7ffa6fe00640 1 --2- 172.21.15.12:0/3954532130 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7ffa540b5410 0x7ffa540b78d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:04:19.627 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:19.627+0000 7ffa6fe00640 1 --2- 172.21.15.12:0/3954532130 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7ffa540b5410 0x7ffa540b78d0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7ffa60005ab0 tx=0x7ffa60005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:04:19.628 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:19.627+0000 7ffa6cc00640 1 -- 172.21.15.12:0/3954532130 <== mon.1 v2:172.21.15.178:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7ffa5c0ace40 con 0x7ffa7010a820 2024-09-11T00:04:19.824 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:19.823+0000 7ffa75600640 1 -- 172.21.15.12:0/3954532130 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ffa701a80a0 con 0x7ffa7010a820 2024-09-11T00:04:19.826 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:19.825+0000 7ffa6cc00640 1 -- 172.21.15.12:0/3954532130 <== mon.1 v2:172.21.15.178:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) ==== 74+0+24236 (secure 0 0 0) 0x7ffa5c0ac740 con 0x7ffa7010a820 2024-09-11T00:04:19.826 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-11T00:04:19.826 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":81,"fsid":"ff3ad74a-6fcf-11ef-bcea-c7b262605968","created":"2024-09-10T23:55:16.317643+0000","modified":"2024-09-11T00:04:15.795652+0000","last_up_change":"2024-09-11T00:03:08.959256+0000","last_in_change":"2024-09-11T00:02:46.564631+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T23:59:10.488728+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-11T00:04:00.065618+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"74","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-11T00:04:02.689888+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"74","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-11T00:04:09.862120+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"6382c200-a912-42a1-958b-79191b21249b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6802","nonce":1237101991},{"type":"v1","addr":"172.21.15.12:6803","nonce":1237101991}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6804","nonce":1237101991},{"type":"v1","addr":"172.21.15.12:6805","nonce":1237101991}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6808","nonce":1237101991},{"type":"v1","addr":"172.21.15.12:6809","nonce":1237101991}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6806","nonce":1237101991},{"type":"v1","addr":"172.21.15.12:6807","nonce":1237101991}]},"public_addr":"172.21.15.12:6803/1237101991","cluster_addr":"172.21.15.12:6805/1237101991","heartbeat_back_addr":"172.21.15.12:6809/1237101991","heartbeat_front_addr":"172.21.15.12:6807/1237101991","state":["exists","up"]},{"osd":1,"uuid":"fe0e1d0a-bdc5-4859-aa84-9038059a45f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6800","nonce":545848456},{"type":"v1","addr":"172.21.15.165:6801","nonce":545848456}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6802","nonce":545848456},{"type":"v1","addr":"172.21.15.165:6803","nonce":545848456}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6806","nonce":545848456},{"type":"v1","addr":"172.21.15.165:6807","nonce":545848456}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6804","nonce":545848456},{"type":"v1","addr":"172.21.15.165:6805","nonce":545848456}]},"public_addr":"172.21.15.165:6801/545848456","cluster_addr":"172.21.15.165:6803/545848456","heartbeat_back_addr":"172.21.15.165:6807/545848456","heartbeat_front_addr":"172.21.15.165:6805/545848456","state":["exists","up"]},{"osd":2,"uuid":"94f640dc-5f78-4e2c-87d9-03e96a6e7ba0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6800","nonce":4131162695},{"type":"v1","addr":"172.21.15.178:6801","nonce":4131162695}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":4131162695},{"type":"v1","addr":"172.21.15.178:6803","nonce":4131162695}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":4131162695},{"type":"v1","addr":"172.21.15.178:6807","nonce":4131162695}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":4131162695},{"type":"v1","addr":"172.21.15.178:6805","nonce":4131162695}]},"public_addr":"172.21.15.178:6801/4131162695","cluster_addr":"172.21.15.178:6803/4131162695","heartbeat_back_addr":"172.21.15.178:6807/4131162695","heartbeat_front_addr":"172.21.15.178:6805/4131162695","state":["exists","up"]},{"osd":3,"uuid":"af2f64d6-c9be-44a9-b041-8d599a7c080d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6810","nonce":452464245},{"type":"v1","addr":"172.21.15.12:6811","nonce":452464245}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6812","nonce":452464245},{"type":"v1","addr":"172.21.15.12:6813","nonce":452464245}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6816","nonce":452464245},{"type":"v1","addr":"172.21.15.12:6817","nonce":452464245}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6814","nonce":452464245},{"type":"v1","addr":"172.21.15.12:6815","nonce":452464245}]},"public_addr":"172.21.15.12:6811/452464245","cluster_addr":"172.21.15.12:6813/452464245","heartbeat_back_addr":"172.21.15.12:6817/452464245","heartbeat_front_addr":"172.21.15.12:6815/452464245","state":["exists","up"]},{"osd":4,"uuid":"ad76ac70-20c0-44ae-9866-5f9cfce2334c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6808","nonce":1899879280},{"type":"v1","addr":"172.21.15.165:6809","nonce":1899879280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6810","nonce":1899879280},{"type":"v1","addr":"172.21.15.165:6811","nonce":1899879280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6814","nonce":1899879280},{"type":"v1","addr":"172.21.15.165:6815","nonce":1899879280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6812","nonce":1899879280},{"type":"v1","addr":"172.21.15.165:6813","nonce":1899879280}]},"public_addr":"172.21.15.165:6809/1899879280","cluster_addr":"172.21.15.165:6811/1899879280","heartbeat_back_addr":"172.21.15.165:6815/1899879280","heartbeat_front_addr":"172.21.15.165:6813/1899879280","state":["exists","up"]},{"osd":5,"uuid":"aafda2cc-62fe-4fcb-9a96-8732e0b501d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":3042228555},{"type":"v1","addr":"172.21.15.178:6809","nonce":3042228555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":3042228555},{"type":"v1","addr":"172.21.15.178:6811","nonce":3042228555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6814","nonce":3042228555},{"type":"v1","addr":"172.21.15.178:6815","nonce":3042228555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":3042228555},{"type":"v1","addr":"172.21.15.178:6813","nonce":3042228555}]},"public_addr":"172.21.15.178:6809/3042228555","cluster_addr":"172.21.15.178:6811/3042228555","heartbeat_back_addr":"172.21.15.178:6815/3042228555","heartbeat_front_addr":"172.21.15.178:6813/3042228555","state":["exists","up"]},{"osd":6,"uuid":"375eca52-efe3-42a5-bbb1-d7d392d71f15","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6818","nonce":3868322467},{"type":"v1","addr":"172.21.15.12:6819","nonce":3868322467}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6820","nonce":3868322467},{"type":"v1","addr":"172.21.15.12:6821","nonce":3868322467}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6824","nonce":3868322467},{"type":"v1","addr":"172.21.15.12:6825","nonce":3868322467}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6822","nonce":3868322467},{"type":"v1","addr":"172.21.15.12:6823","nonce":3868322467}]},"public_addr":"172.21.15.12:6819/3868322467","cluster_addr":"172.21.15.12:6821/3868322467","heartbeat_back_addr":"172.21.15.12:6825/3868322467","heartbeat_front_addr":"172.21.15.12:6823/3868322467","state":["exists","up"]},{"osd":7,"uuid":"0fd619f5-44c3-4cf7-ae7a-388dd798de36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6816","nonce":1143773735},{"type":"v1","addr":"172.21.15.165:6817","nonce":1143773735}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6818","nonce":1143773735},{"type":"v1","addr":"172.21.15.165:6819","nonce":1143773735}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6822","nonce":1143773735},{"type":"v1","addr":"172.21.15.165:6823","nonce":1143773735}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6820","nonce":1143773735},{"type":"v1","addr":"172.21.15.165:6821","nonce":1143773735}]},"public_addr":"172.21.15.165:6817/1143773735","cluster_addr":"172.21.15.165:6819/1143773735","heartbeat_back_addr":"172.21.15.165:6823/1143773735","heartbeat_front_addr":"172.21.15.165:6821/1143773735","state":["exists","up"]},{"osd":8,"uuid":"b03d14ce-8427-462a-9bcc-27683c549f53","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":2891445058},{"type":"v1","addr":"172.21.15.178:6817","nonce":2891445058}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6818","nonce":2891445058},{"type":"v1","addr":"172.21.15.178:6819","nonce":2891445058}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6822","nonce":2891445058},{"type":"v1","addr":"172.21.15.178:6823","nonce":2891445058}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":2891445058},{"type":"v1","addr":"172.21.15.178:6821","nonce":2891445058}]},"public_addr":"172.21.15.178:6817/2891445058","cluster_addr":"172.21.15.178:6819/2891445058","heartbeat_back_addr":"172.21.15.178:6823/2891445058","heartbeat_front_addr":"172.21.15.178:6821/2891445058","state":["exists","up"]},{"osd":9,"uuid":"81a6cf69-2a24-4a6c-9adf-6b7df38cfacb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6826","nonce":3359634903},{"type":"v1","addr":"172.21.15.12:6827","nonce":3359634903}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6828","nonce":3359634903},{"type":"v1","addr":"172.21.15.12:6829","nonce":3359634903}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6832","nonce":3359634903},{"type":"v1","addr":"172.21.15.12:6833","nonce":3359634903}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6830","nonce":3359634903},{"type":"v1","addr":"172.21.15.12:6831","nonce":3359634903}]},"public_addr":"172.21.15.12:6827/3359634903","cluster_addr":"172.21.15.12:6829/3359634903","heartbeat_back_addr":"172.21.15.12:6833/3359634903","heartbeat_front_addr":"172.21.15.12:6831/3359634903","state":["exists","up"]},{"osd":10,"uuid":"c1ad1f0a-33f4-4c32-91f6-4d030a29f1dd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6824","nonce":3642325778},{"type":"v1","addr":"172.21.15.165:6825","nonce":3642325778}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6826","nonce":3642325778},{"type":"v1","addr":"172.21.15.165:6827","nonce":3642325778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6830","nonce":3642325778},{"type":"v1","addr":"172.21.15.165:6831","nonce":3642325778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6828","nonce":3642325778},{"type":"v1","addr":"172.21.15.165:6829","nonce":3642325778}]},"public_addr":"172.21.15.165:6825/3642325778","cluster_addr":"172.21.15.165:6827/3642325778","heartbeat_back_addr":"172.21.15.165:6831/3642325778","heartbeat_front_addr":"172.21.15.165:6829/3642325778","state":["exists","up"]},{"osd":11,"uuid":"8413c9b0-680b-4d02-aeab-b3b3ff16c3d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":2937059756},{"type":"v1","addr":"172.21.15.178:6825","nonce":2937059756}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6826","nonce":2937059756},{"type":"v1","addr":"172.21.15.178:6827","nonce":2937059756}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":2937059756},{"type":"v1","addr":"172.21.15.178:6831","nonce":2937059756}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":2937059756},{"type":"v1","addr":"172.21.15.178:6829","nonce":2937059756}]},"public_addr":"172.21.15.178:6825/2937059756","cluster_addr":"172.21.15.178:6827/2937059756","heartbeat_back_addr":"172.21.15.178:6831/2937059756","heartbeat_front_addr":"172.21.15.178:6829/2937059756","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:58:16.036348+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:58:41.483150+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:59:07.107283+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:59:35.706574+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T00:00:02.555720+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T00:00:28.092847+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T00:00:57.396495+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T00:01:24.216775+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T00:01:48.807850+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T00:02:16.838565+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T00:02:43.131736+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T00:03:07.353155+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.12:6801/4177251511":"2024-09-11T23:56:17.999820+0000","172.21.15.12:6800/4177251511":"2024-09-11T23:56:17.999820+0000","172.21.15.12:0/3273692817":"2024-09-11T23:56:17.999820+0000","172.21.15.12:0/4017778158":"2024-09-11T23:56:17.999820+0000","172.21.15.12:0/2499921051":"2024-09-11T23:55:50.302721+0000","172.21.15.12:6801/760140910":"2024-09-11T23:55:50.302721+0000","172.21.15.12:6800/760140910":"2024-09-11T23:55:50.302721+0000","172.21.15.12:0/2810703302":"2024-09-11T23:56:17.999820+0000","172.21.15.12:0/2325436065":"2024-09-11T23:55:50.302721+0000","172.21.15.12:0/3137053479":"2024-09-11T23:55:50.302721+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-11T00:04:19.828 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:19.827+0000 7ffa75600640 1 -- 172.21.15.12:0/3954532130 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7ffa540b5410 msgr2=0x7ffa540b78d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:04:19.828 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:19.827+0000 7ffa75600640 1 --2- 172.21.15.12:0/3954532130 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7ffa540b5410 0x7ffa540b78d0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7ffa60005ab0 tx=0x7ffa60005a20 comp rx=0 tx=0).stop 2024-09-11T00:04:19.828 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:19.827+0000 7ffa75600640 1 -- 172.21.15.12:0/3954532130 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ffa7010a820 msgr2=0x7ffa7019b780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:04:19.828 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:19.827+0000 7ffa75600640 1 --2- 172.21.15.12:0/3954532130 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ffa7010a820 0x7ffa7019b780 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7ffa5c00ea60 tx=0x7ffa5c00ef20 comp rx=0 tx=0).stop 2024-09-11T00:04:19.829 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:19.827+0000 7ffa75600640 1 -- 172.21.15.12:0/3954532130 shutdown_connections 2024-09-11T00:04:19.829 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:19.827+0000 7ffa75600640 1 --2- 172.21.15.12:0/3954532130 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7ffa540b5410 0x7ffa540b78d0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:04:19.829 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:19.827+0000 7ffa75600640 1 --2- 172.21.15.12:0/3954532130 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ffa7010b1a0 0x7ffa701a0b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:04:19.829 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:19.827+0000 7ffa75600640 1 --2- 172.21.15.12:0/3954532130 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ffa7010a820 0x7ffa7019b780 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:04:19.829 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:19.827+0000 7ffa75600640 1 --2- 172.21.15.12:0/3954532130 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ffa70104e10 0x7ffa7019b240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:04:19.829 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:19.827+0000 7ffa75600640 1 -- 172.21.15.12:0/3954532130 >> 172.21.15.12:0/3954532130 conn(0x7ffa70100840 msgr2=0x7ffa70102c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:04:19.829 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:19.828+0000 7ffa75600640 1 -- 172.21.15.12:0/3954532130 shutdown_connections 2024-09-11T00:04:19.829 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:19.828+0000 7ffa75600640 1 -- 172.21.15.12:0/3954532130 wait complete. 2024-09-11T00:04:20.233 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:04:19 smithi012 ceph-mon[30333]: pgmap v311: 129 pgs: 129 active+clean; 579 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T00:04:20.233 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:04:19 smithi012 ceph-mon[30333]: from='client.? 172.21.15.12:0/3954532130' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-11T00:04:20.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:04:19 smithi165 ceph-mon[35356]: pgmap v311: 129 pgs: 129 active+clean; 579 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T00:04:20.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:04:19 smithi165 ceph-mon[35356]: from='client.? 172.21.15.12:0/3954532130' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-11T00:04:20.330 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:04:19 smithi178 ceph-mon[35919]: pgmap v311: 129 pgs: 129 active+clean; 579 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T00:04:20.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:04:19 smithi178 ceph-mon[35919]: from='client.? 172.21.15.12:0/3954532130' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-11T00:04:20.683 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid ff3ad74a-6fcf-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-11T00:04:21.061 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:04:21.578 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:21.576+0000 7f6736200640 1 -- 172.21.15.12:0/3564797206 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6730077ff0 msgr2=0x7f67301132a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:04:21.578 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:21.576+0000 7f6736200640 1 --2- 172.21.15.12:0/3564797206 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6730077ff0 0x7f67301132a0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f6718009900 tx=0x7f671802a3f0 comp rx=0 tx=0).stop 2024-09-11T00:04:21.578 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:21.576+0000 7f6736200640 1 -- 172.21.15.12:0/3564797206 shutdown_connections 2024-09-11T00:04:21.578 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:21.576+0000 7f6736200640 1 --2- 172.21.15.12:0/3564797206 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f67301137e0 0x7f6730115bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:04:21.578 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:21.576+0000 7f6736200640 1 --2- 172.21.15.12:0/3564797206 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6730077ff0 0x7f67301132a0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:04:21.578 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:21.576+0000 7f6736200640 1 --2- 172.21.15.12:0/3564797206 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f67300776d0 0x7f6730077ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:04:21.578 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:21.576+0000 7f6736200640 1 -- 172.21.15.12:0/3564797206 >> 172.21.15.12:0/3564797206 conn(0x7f6730100840 msgr2=0x7f6730102c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:04:21.578 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:21.576+0000 7f6736200640 1 -- 172.21.15.12:0/3564797206 shutdown_connections 2024-09-11T00:04:21.578 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:21.577+0000 7f6736200640 1 -- 172.21.15.12:0/3564797206 wait complete. 2024-09-11T00:04:21.579 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:21.577+0000 7f6736200640 1 Processor -- start 2024-09-11T00:04:21.580 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:21.578+0000 7f6736200640 1 -- start start 2024-09-11T00:04:21.580 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:21.578+0000 7f6736200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f67300776d0 0x7f673019f5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:04:21.580 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:21.578+0000 7f6736200640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6730077ff0 0x7f673019fb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:04:21.580 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:21.578+0000 7f6736200640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f67301137e0 0x7f67301a4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:04:21.580 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:21.578+0000 7f6736200640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f67301191e0 con 0x7f6730077ff0 2024-09-11T00:04:21.580 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:21.578+0000 7f6736200640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f6730119060 con 0x7f67300776d0 2024-09-11T00:04:21.580 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:21.578+0000 7f6736200640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f6730119360 con 0x7f67301137e0 2024-09-11T00:04:21.580 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:21.578+0000 7f6727e00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6730077ff0 0x7f673019fb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:04:21.580 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:21.578+0000 7f6734e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f67300776d0 0x7f673019f5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:04:21.580 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:21.578+0000 7f6734e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f67300776d0 0x7f673019f5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.12:52934/0 (socket says 172.21.15.12:52934) 2024-09-11T00:04:21.580 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:21.578+0000 7f6734e00640 1 -- 172.21.15.12:0/713652564 learned_addr learned my addr 172.21.15.12:0/713652564 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:04:21.580 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:21.579+0000 7f6735800640 1 --2- 172.21.15.12:0/713652564 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f67301137e0 0x7f67301a4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:04:21.581 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:21.579+0000 7f6727e00640 1 -- 172.21.15.12:0/713652564 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f67301137e0 msgr2=0x7f67301a4ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:04:21.581 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:21.579+0000 7f6727e00640 1 --2- 172.21.15.12:0/713652564 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f67301137e0 0x7f67301a4ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:04:21.581 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:21.579+0000 7f6727e00640 1 -- 172.21.15.12:0/713652564 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f67300776d0 msgr2=0x7f673019f5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:04:21.581 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:21.579+0000 7f6727e00640 1 --2- 172.21.15.12:0/713652564 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f67300776d0 0x7f673019f5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:04:21.581 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:21.579+0000 7f6727e00640 1 -- 172.21.15.12:0/713652564 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67301a56a0 con 0x7f6730077ff0 2024-09-11T00:04:21.582 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:21.579+0000 7f6727e00640 1 --2- 172.21.15.12:0/713652564 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6730077ff0 0x7f673019fb30 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f6718006010 tx=0x7f6718004060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:04:21.582 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:21.580+0000 7f672e000640 1 -- 172.21.15.12:0/713652564 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f671803f070 con 0x7f6730077ff0 2024-09-11T00:04:21.582 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:21.580+0000 7f672e000640 1 -- 172.21.15.12:0/713652564 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6718003ca0 con 0x7f6730077ff0 2024-09-11T00:04:21.582 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:21.580+0000 7f672e000640 1 -- 172.21.15.12:0/713652564 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6718035070 con 0x7f6730077ff0 2024-09-11T00:04:21.582 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:21.580+0000 7f6736200640 1 -- 172.21.15.12:0/713652564 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f67301a5930 con 0x7f6730077ff0 2024-09-11T00:04:21.582 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:21.580+0000 7f6736200640 1 -- 172.21.15.12:0/713652564 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67301ac050 con 0x7f6730077ff0 2024-09-11T00:04:21.582 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:21.581+0000 7f6736200640 1 -- 172.21.15.12:0/713652564 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6730077ab0 con 0x7f6730077ff0 2024-09-11T00:04:21.586 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:21.585+0000 7f672e000640 1 -- 172.21.15.12:0/713652564 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 16) ==== 153250+0+0 (secure 0 0 0) 0x7f671804b060 con 0x7f6730077ff0 2024-09-11T00:04:21.587 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:21.586+0000 7f672e000640 1 --2- 172.21.15.12:0/713652564 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f67140b5410 0x7f67140b78d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:04:21.587 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:21.587+0000 7f672e000640 1 -- 172.21.15.12:0/713652564 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8802+0+0 (secure 0 0 0) 0x7f67181061e0 con 0x7f6730077ff0 2024-09-11T00:04:21.588 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:21.587+0000 7f6734e00640 1 --2- 172.21.15.12:0/713652564 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f67140b5410 0x7f67140b78d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:04:21.588 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:21.587+0000 7f6734e00640 1 --2- 172.21.15.12:0/713652564 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f67140b5410 0x7f67140b78d0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f672000bca0 tx=0x7f672000b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:04:21.590 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:21.589+0000 7f672e000640 1 -- 172.21.15.12:0/713652564 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f67180cc7b0 con 0x7f6730077ff0 2024-09-11T00:04:21.785 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:21.784+0000 7f6736200640 1 -- 172.21.15.12:0/713652564 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f67301ac350 con 0x7f6730077ff0 2024-09-11T00:04:21.786 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:21.786+0000 7f672e000640 1 -- 172.21.15.12:0/713652564 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) ==== 74+0+24236 (secure 0 0 0) 0x7f67180d11e0 con 0x7f6730077ff0 2024-09-11T00:04:21.787 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-11T00:04:21.787 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":81,"fsid":"ff3ad74a-6fcf-11ef-bcea-c7b262605968","created":"2024-09-10T23:55:16.317643+0000","modified":"2024-09-11T00:04:15.795652+0000","last_up_change":"2024-09-11T00:03:08.959256+0000","last_in_change":"2024-09-11T00:02:46.564631+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T23:59:10.488728+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-11T00:04:00.065618+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"74","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-11T00:04:02.689888+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"74","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-11T00:04:09.862120+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"6382c200-a912-42a1-958b-79191b21249b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6802","nonce":1237101991},{"type":"v1","addr":"172.21.15.12:6803","nonce":1237101991}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6804","nonce":1237101991},{"type":"v1","addr":"172.21.15.12:6805","nonce":1237101991}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6808","nonce":1237101991},{"type":"v1","addr":"172.21.15.12:6809","nonce":1237101991}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6806","nonce":1237101991},{"type":"v1","addr":"172.21.15.12:6807","nonce":1237101991}]},"public_addr":"172.21.15.12:6803/1237101991","cluster_addr":"172.21.15.12:6805/1237101991","heartbeat_back_addr":"172.21.15.12:6809/1237101991","heartbeat_front_addr":"172.21.15.12:6807/1237101991","state":["exists","up"]},{"osd":1,"uuid":"fe0e1d0a-bdc5-4859-aa84-9038059a45f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6800","nonce":545848456},{"type":"v1","addr":"172.21.15.165:6801","nonce":545848456}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6802","nonce":545848456},{"type":"v1","addr":"172.21.15.165:6803","nonce":545848456}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6806","nonce":545848456},{"type":"v1","addr":"172.21.15.165:6807","nonce":545848456}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6804","nonce":545848456},{"type":"v1","addr":"172.21.15.165:6805","nonce":545848456}]},"public_addr":"172.21.15.165:6801/545848456","cluster_addr":"172.21.15.165:6803/545848456","heartbeat_back_addr":"172.21.15.165:6807/545848456","heartbeat_front_addr":"172.21.15.165:6805/545848456","state":["exists","up"]},{"osd":2,"uuid":"94f640dc-5f78-4e2c-87d9-03e96a6e7ba0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6800","nonce":4131162695},{"type":"v1","addr":"172.21.15.178:6801","nonce":4131162695}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":4131162695},{"type":"v1","addr":"172.21.15.178:6803","nonce":4131162695}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":4131162695},{"type":"v1","addr":"172.21.15.178:6807","nonce":4131162695}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":4131162695},{"type":"v1","addr":"172.21.15.178:6805","nonce":4131162695}]},"public_addr":"172.21.15.178:6801/4131162695","cluster_addr":"172.21.15.178:6803/4131162695","heartbeat_back_addr":"172.21.15.178:6807/4131162695","heartbeat_front_addr":"172.21.15.178:6805/4131162695","state":["exists","up"]},{"osd":3,"uuid":"af2f64d6-c9be-44a9-b041-8d599a7c080d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6810","nonce":452464245},{"type":"v1","addr":"172.21.15.12:6811","nonce":452464245}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6812","nonce":452464245},{"type":"v1","addr":"172.21.15.12:6813","nonce":452464245}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6816","nonce":452464245},{"type":"v1","addr":"172.21.15.12:6817","nonce":452464245}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6814","nonce":452464245},{"type":"v1","addr":"172.21.15.12:6815","nonce":452464245}]},"public_addr":"172.21.15.12:6811/452464245","cluster_addr":"172.21.15.12:6813/452464245","heartbeat_back_addr":"172.21.15.12:6817/452464245","heartbeat_front_addr":"172.21.15.12:6815/452464245","state":["exists","up"]},{"osd":4,"uuid":"ad76ac70-20c0-44ae-9866-5f9cfce2334c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6808","nonce":1899879280},{"type":"v1","addr":"172.21.15.165:6809","nonce":1899879280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6810","nonce":1899879280},{"type":"v1","addr":"172.21.15.165:6811","nonce":1899879280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6814","nonce":1899879280},{"type":"v1","addr":"172.21.15.165:6815","nonce":1899879280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6812","nonce":1899879280},{"type":"v1","addr":"172.21.15.165:6813","nonce":1899879280}]},"public_addr":"172.21.15.165:6809/1899879280","cluster_addr":"172.21.15.165:6811/1899879280","heartbeat_back_addr":"172.21.15.165:6815/1899879280","heartbeat_front_addr":"172.21.15.165:6813/1899879280","state":["exists","up"]},{"osd":5,"uuid":"aafda2cc-62fe-4fcb-9a96-8732e0b501d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":3042228555},{"type":"v1","addr":"172.21.15.178:6809","nonce":3042228555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":3042228555},{"type":"v1","addr":"172.21.15.178:6811","nonce":3042228555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6814","nonce":3042228555},{"type":"v1","addr":"172.21.15.178:6815","nonce":3042228555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":3042228555},{"type":"v1","addr":"172.21.15.178:6813","nonce":3042228555}]},"public_addr":"172.21.15.178:6809/3042228555","cluster_addr":"172.21.15.178:6811/3042228555","heartbeat_back_addr":"172.21.15.178:6815/3042228555","heartbeat_front_addr":"172.21.15.178:6813/3042228555","state":["exists","up"]},{"osd":6,"uuid":"375eca52-efe3-42a5-bbb1-d7d392d71f15","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6818","nonce":3868322467},{"type":"v1","addr":"172.21.15.12:6819","nonce":3868322467}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6820","nonce":3868322467},{"type":"v1","addr":"172.21.15.12:6821","nonce":3868322467}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6824","nonce":3868322467},{"type":"v1","addr":"172.21.15.12:6825","nonce":3868322467}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6822","nonce":3868322467},{"type":"v1","addr":"172.21.15.12:6823","nonce":3868322467}]},"public_addr":"172.21.15.12:6819/3868322467","cluster_addr":"172.21.15.12:6821/3868322467","heartbeat_back_addr":"172.21.15.12:6825/3868322467","heartbeat_front_addr":"172.21.15.12:6823/3868322467","state":["exists","up"]},{"osd":7,"uuid":"0fd619f5-44c3-4cf7-ae7a-388dd798de36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6816","nonce":1143773735},{"type":"v1","addr":"172.21.15.165:6817","nonce":1143773735}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6818","nonce":1143773735},{"type":"v1","addr":"172.21.15.165:6819","nonce":1143773735}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6822","nonce":1143773735},{"type":"v1","addr":"172.21.15.165:6823","nonce":1143773735}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6820","nonce":1143773735},{"type":"v1","addr":"172.21.15.165:6821","nonce":1143773735}]},"public_addr":"172.21.15.165:6817/1143773735","cluster_addr":"172.21.15.165:6819/1143773735","heartbeat_back_addr":"172.21.15.165:6823/1143773735","heartbeat_front_addr":"172.21.15.165:6821/1143773735","state":["exists","up"]},{"osd":8,"uuid":"b03d14ce-8427-462a-9bcc-27683c549f53","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":2891445058},{"type":"v1","addr":"172.21.15.178:6817","nonce":2891445058}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6818","nonce":2891445058},{"type":"v1","addr":"172.21.15.178:6819","nonce":2891445058}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6822","nonce":2891445058},{"type":"v1","addr":"172.21.15.178:6823","nonce":2891445058}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":2891445058},{"type":"v1","addr":"172.21.15.178:6821","nonce":2891445058}]},"public_addr":"172.21.15.178:6817/2891445058","cluster_addr":"172.21.15.178:6819/2891445058","heartbeat_back_addr":"172.21.15.178:6823/2891445058","heartbeat_front_addr":"172.21.15.178:6821/2891445058","state":["exists","up"]},{"osd":9,"uuid":"81a6cf69-2a24-4a6c-9adf-6b7df38cfacb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6826","nonce":3359634903},{"type":"v1","addr":"172.21.15.12:6827","nonce":3359634903}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6828","nonce":3359634903},{"type":"v1","addr":"172.21.15.12:6829","nonce":3359634903}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6832","nonce":3359634903},{"type":"v1","addr":"172.21.15.12:6833","nonce":3359634903}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6830","nonce":3359634903},{"type":"v1","addr":"172.21.15.12:6831","nonce":3359634903}]},"public_addr":"172.21.15.12:6827/3359634903","cluster_addr":"172.21.15.12:6829/3359634903","heartbeat_back_addr":"172.21.15.12:6833/3359634903","heartbeat_front_addr":"172.21.15.12:6831/3359634903","state":["exists","up"]},{"osd":10,"uuid":"c1ad1f0a-33f4-4c32-91f6-4d030a29f1dd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6824","nonce":3642325778},{"type":"v1","addr":"172.21.15.165:6825","nonce":3642325778}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6826","nonce":3642325778},{"type":"v1","addr":"172.21.15.165:6827","nonce":3642325778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6830","nonce":3642325778},{"type":"v1","addr":"172.21.15.165:6831","nonce":3642325778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6828","nonce":3642325778},{"type":"v1","addr":"172.21.15.165:6829","nonce":3642325778}]},"public_addr":"172.21.15.165:6825/3642325778","cluster_addr":"172.21.15.165:6827/3642325778","heartbeat_back_addr":"172.21.15.165:6831/3642325778","heartbeat_front_addr":"172.21.15.165:6829/3642325778","state":["exists","up"]},{"osd":11,"uuid":"8413c9b0-680b-4d02-aeab-b3b3ff16c3d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":2937059756},{"type":"v1","addr":"172.21.15.178:6825","nonce":2937059756}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6826","nonce":2937059756},{"type":"v1","addr":"172.21.15.178:6827","nonce":2937059756}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":2937059756},{"type":"v1","addr":"172.21.15.178:6831","nonce":2937059756}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":2937059756},{"type":"v1","addr":"172.21.15.178:6829","nonce":2937059756}]},"public_addr":"172.21.15.178:6825/2937059756","cluster_addr":"172.21.15.178:6827/2937059756","heartbeat_back_addr":"172.21.15.178:6831/2937059756","heartbeat_front_addr":"172.21.15.178:6829/2937059756","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:58:16.036348+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:58:41.483150+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:59:07.107283+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:59:35.706574+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T00:00:02.555720+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T00:00:28.092847+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T00:00:57.396495+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T00:01:24.216775+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T00:01:48.807850+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T00:02:16.838565+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T00:02:43.131736+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T00:03:07.353155+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.12:6801/4177251511":"2024-09-11T23:56:17.999820+0000","172.21.15.12:6800/4177251511":"2024-09-11T23:56:17.999820+0000","172.21.15.12:0/3273692817":"2024-09-11T23:56:17.999820+0000","172.21.15.12:0/4017778158":"2024-09-11T23:56:17.999820+0000","172.21.15.12:0/2499921051":"2024-09-11T23:55:50.302721+0000","172.21.15.12:6801/760140910":"2024-09-11T23:55:50.302721+0000","172.21.15.12:6800/760140910":"2024-09-11T23:55:50.302721+0000","172.21.15.12:0/2810703302":"2024-09-11T23:56:17.999820+0000","172.21.15.12:0/2325436065":"2024-09-11T23:55:50.302721+0000","172.21.15.12:0/3137053479":"2024-09-11T23:55:50.302721+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-11T00:04:21.790 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:21.789+0000 7f6736200640 1 -- 172.21.15.12:0/713652564 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f67140b5410 msgr2=0x7f67140b78d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:04:21.790 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:21.789+0000 7f6736200640 1 --2- 172.21.15.12:0/713652564 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f67140b5410 0x7f67140b78d0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f672000bca0 tx=0x7f672000b410 comp rx=0 tx=0).stop 2024-09-11T00:04:21.790 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:21.789+0000 7f6736200640 1 -- 172.21.15.12:0/713652564 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6730077ff0 msgr2=0x7f673019fb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:04:21.790 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:21.789+0000 7f6736200640 1 --2- 172.21.15.12:0/713652564 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6730077ff0 0x7f673019fb30 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f6718006010 tx=0x7f6718004060 comp rx=0 tx=0).stop 2024-09-11T00:04:21.790 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:21.789+0000 7f6736200640 1 -- 172.21.15.12:0/713652564 shutdown_connections 2024-09-11T00:04:21.790 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:21.789+0000 7f6736200640 1 --2- 172.21.15.12:0/713652564 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f67140b5410 0x7f67140b78d0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:04:21.790 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:21.789+0000 7f6736200640 1 --2- 172.21.15.12:0/713652564 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f67301137e0 0x7f67301a4ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:04:21.790 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:21.789+0000 7f6736200640 1 --2- 172.21.15.12:0/713652564 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6730077ff0 0x7f673019fb30 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:04:21.791 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:21.789+0000 7f6736200640 1 --2- 172.21.15.12:0/713652564 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f67300776d0 0x7f673019f5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:04:21.791 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:21.789+0000 7f6736200640 1 -- 172.21.15.12:0/713652564 >> 172.21.15.12:0/713652564 conn(0x7f6730100840 msgr2=0x7f6730102c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:04:21.791 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:21.789+0000 7f6736200640 1 -- 172.21.15.12:0/713652564 shutdown_connections 2024-09-11T00:04:21.791 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:21.789+0000 7f6736200640 1 -- 172.21.15.12:0/713652564 wait complete. 2024-09-11T00:04:22.063 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:04:21 smithi012 ceph-mon[30333]: pgmap v312: 129 pgs: 129 active+clean; 579 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T00:04:22.063 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:04:21 smithi012 ceph-mon[30333]: from='client.? 172.21.15.12:0/713652564' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-11T00:04:22.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:04:21 smithi165 ceph-mon[35356]: pgmap v312: 129 pgs: 129 active+clean; 579 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T00:04:22.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:04:21 smithi165 ceph-mon[35356]: from='client.? 172.21.15.12:0/713652564' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-11T00:04:22.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:04:21 smithi178 ceph-mon[35919]: pgmap v312: 129 pgs: 129 active+clean; 579 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T00:04:22.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:04:21 smithi178 ceph-mon[35919]: from='client.? 172.21.15.12:0/713652564' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-11T00:04:22.673 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid ff3ad74a-6fcf-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T00:04:23.050 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:04:23.583 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:23.553+0000 7ff67e600640 1 -- 172.21.15.12:0/4021649882 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff678075e00 msgr2=0x7ff67810ecc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:04:23.583 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:23.553+0000 7ff67e600640 1 --2- 172.21.15.12:0/4021649882 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff678075e00 0x7ff67810ecc0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7ff664009900 tx=0x7ff66402a3f0 comp rx=0 tx=0).stop 2024-09-11T00:04:23.583 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:23.553+0000 7ff67e600640 1 -- 172.21.15.12:0/4021649882 shutdown_connections 2024-09-11T00:04:23.583 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:23.553+0000 7ff67e600640 1 --2- 172.21.15.12:0/4021649882 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff67810f200 0x7ff6781115f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:04:23.583 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:23.553+0000 7ff67e600640 1 --2- 172.21.15.12:0/4021649882 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff678075e00 0x7ff67810ecc0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:04:23.583 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:23.553+0000 7ff67e600640 1 --2- 172.21.15.12:0/4021649882 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff6780754e0 0x7ff6780758c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:04:23.584 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:23.553+0000 7ff67e600640 1 -- 172.21.15.12:0/4021649882 >> 172.21.15.12:0/4021649882 conn(0x7ff6780fe660 msgr2=0x7ff678100a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:04:23.584 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:23.553+0000 7ff67e600640 1 -- 172.21.15.12:0/4021649882 shutdown_connections 2024-09-11T00:04:23.584 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:23.553+0000 7ff67e600640 1 -- 172.21.15.12:0/4021649882 wait complete. 2024-09-11T00:04:23.584 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:23.554+0000 7ff67e600640 1 Processor -- start 2024-09-11T00:04:23.584 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:23.554+0000 7ff67e600640 1 -- start start 2024-09-11T00:04:23.584 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:23.554+0000 7ff67e600640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff6780754e0 0x7ff67819b250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:04:23.584 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:23.554+0000 7ff67e600640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff678075e00 0x7ff67819b790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:04:23.584 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:23.554+0000 7ff67e600640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff67810f200 0x7ff6781a0b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:04:23.584 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:23.554+0000 7ff67e600640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7ff678114da0 con 0x7ff6780754e0 2024-09-11T00:04:23.584 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:23.554+0000 7ff67e600640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7ff678114c20 con 0x7ff678075e00 2024-09-11T00:04:23.584 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:23.554+0000 7ff67e600640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7ff678114f20 con 0x7ff67810f200 2024-09-11T00:04:23.584 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:23.555+0000 7ff67d200640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff6780754e0 0x7ff67819b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:04:23.585 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:23.555+0000 7ff67d200640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff6780754e0 0x7ff67819b250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:50574/0 (socket says 172.21.15.12:50574) 2024-09-11T00:04:23.585 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:23.555+0000 7ff67d200640 1 -- 172.21.15.12:0/2766332908 learned_addr learned my addr 172.21.15.12:0/2766332908 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:04:23.585 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:23.555+0000 7ff677e00640 1 --2- 172.21.15.12:0/2766332908 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff678075e00 0x7ff67819b790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:04:23.585 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:23.555+0000 7ff67dc00640 1 --2- 172.21.15.12:0/2766332908 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff67810f200 0x7ff6781a0b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:04:23.585 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:23.555+0000 7ff67d200640 1 -- 172.21.15.12:0/2766332908 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff67810f200 msgr2=0x7ff6781a0b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:04:23.585 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:23.555+0000 7ff67d200640 1 --2- 172.21.15.12:0/2766332908 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff67810f200 0x7ff6781a0b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:04:23.585 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:23.555+0000 7ff67d200640 1 -- 172.21.15.12:0/2766332908 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff678075e00 msgr2=0x7ff67819b790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:04:23.585 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:23.555+0000 7ff67d200640 1 --2- 172.21.15.12:0/2766332908 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff678075e00 0x7ff67819b790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:04:23.585 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:23.555+0000 7ff67d200640 1 -- 172.21.15.12:0/2766332908 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6781a1300 con 0x7ff6780754e0 2024-09-11T00:04:23.586 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:23.555+0000 7ff67d200640 1 --2- 172.21.15.12:0/2766332908 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff6780754e0 0x7ff67819b250 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7ff668009d50 tx=0x7ff66800c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:04:23.586 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:23.556+0000 7ff675600640 1 -- 172.21.15.12:0/2766332908 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff668003b70 con 0x7ff6780754e0 2024-09-11T00:04:23.586 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:23.556+0000 7ff675600640 1 -- 172.21.15.12:0/2766332908 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff668003d10 con 0x7ff6780754e0 2024-09-11T00:04:23.586 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:23.556+0000 7ff675600640 1 -- 172.21.15.12:0/2766332908 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6680108a0 con 0x7ff6780754e0 2024-09-11T00:04:23.586 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:23.556+0000 7ff67e600640 1 -- 172.21.15.12:0/2766332908 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff6781a15f0 con 0x7ff6780754e0 2024-09-11T00:04:23.586 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:23.556+0000 7ff67e600640 1 -- 172.21.15.12:0/2766332908 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff6781a7cb0 con 0x7ff6780754e0 2024-09-11T00:04:23.586 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:23.557+0000 7ff67e600640 1 -- 172.21.15.12:0/2766332908 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff6780758c0 con 0x7ff6780754e0 2024-09-11T00:04:23.586 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:23.559+0000 7ff675600640 1 -- 172.21.15.12:0/2766332908 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 16) ==== 153250+0+0 (secure 0 0 0) 0x7ff668004000 con 0x7ff6780754e0 2024-09-11T00:04:23.586 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:23.561+0000 7ff675600640 1 --2- 172.21.15.12:0/2766332908 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7ff65c0b5390 0x7ff65c0b7850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:04:23.586 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:23.561+0000 7ff675600640 1 -- 172.21.15.12:0/2766332908 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8802+0+0 (secure 0 0 0) 0x7ff6680e6540 con 0x7ff6780754e0 2024-09-11T00:04:23.586 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:23.561+0000 7ff677e00640 1 --2- 172.21.15.12:0/2766332908 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7ff65c0b5390 0x7ff65c0b7850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:04:23.586 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:23.562+0000 7ff677e00640 1 --2- 172.21.15.12:0/2766332908 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7ff65c0b5390 0x7ff65c0b7850 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7ff664031440 tx=0x7ff664005b50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:04:23.586 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:23.565+0000 7ff675600640 1 -- 172.21.15.12:0/2766332908 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7ff6680ad160 con 0x7ff6780754e0 2024-09-11T00:04:23.799 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:23.798+0000 7ff67e600640 1 -- 172.21.15.12:0/2766332908 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff6781a8060 con 0x7ff6780754e0 2024-09-11T00:04:23.799 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:23.799+0000 7ff675600640 1 -- 172.21.15.12:0/2766332908 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+11830 (secure 0 0 0) 0x7ff6680aca60 con 0x7ff6780754e0 2024-09-11T00:04:23.800 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-11T00:04:23.800 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":17,"btime":"2024-09-11T00:04:15:865268+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14577,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6835/447171591","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":447171591},{"type":"v1","addr":"172.21.15.12:6835","nonce":447171591}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14589,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/901486040","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":901486040},{"type":"v1","addr":"172.21.15.12:6837","nonce":901486040}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14631,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6841/3069919437","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6840","nonce":3069919437},{"type":"v1","addr":"172.21.15.12:6841","nonce":3069919437}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24562,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6833/1141636112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":1141636112},{"type":"v1","addr":"172.21.15.178:6833","nonce":1141636112}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24581,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6833/3111980970","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6832","nonce":3111980970},{"type":"v1","addr":"172.21.15.165:6833","nonce":3111980970}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24607,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6835/4084868287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":4084868287},{"type":"v1","addr":"172.21.15.178:6835","nonce":4084868287}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24613,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6837/526874763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":526874763},{"type":"v1","addr":"172.21.15.178:6837","nonce":526874763}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24614,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6835/2425395274","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":2425395274},{"type":"v1","addr":"172.21.15.165:6835","nonce":2425395274}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24637,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6839/70628097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6838","nonce":70628097},{"type":"v1","addr":"172.21.15.178:6839","nonce":70628097}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24638,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6837/6215088","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6836","nonce":6215088},{"type":"v1","addr":"172.21.15.165:6837","nonce":6215088}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24643,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6839/3124093009","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6838","nonce":3124093009},{"type":"v1","addr":"172.21.15.12:6839","nonce":3124093009}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-11T00:04:04.739881+0000","modified":"2024-09-11T00:04:15.795786+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24674},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24674":{"gid":24674,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.165:6839/501312984","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6838","nonce":501312984},{"type":"v1","addr":"172.21.15.165:6839","nonce":501312984}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":24674,"qdb_cluster":[24674]},"id":1}]} 2024-09-11T00:04:23.800 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 17 2024-09-11T00:04:23.803 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:23.801+0000 7ff67e600640 1 -- 172.21.15.12:0/2766332908 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7ff65c0b5390 msgr2=0x7ff65c0b7850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:04:23.803 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:23.801+0000 7ff67e600640 1 --2- 172.21.15.12:0/2766332908 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7ff65c0b5390 0x7ff65c0b7850 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7ff664031440 tx=0x7ff664005b50 comp rx=0 tx=0).stop 2024-09-11T00:04:23.803 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:23.801+0000 7ff67e600640 1 -- 172.21.15.12:0/2766332908 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff6780754e0 msgr2=0x7ff67819b250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:04:23.803 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:23.801+0000 7ff67e600640 1 --2- 172.21.15.12:0/2766332908 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff6780754e0 0x7ff67819b250 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7ff668009d50 tx=0x7ff66800c6a0 comp rx=0 tx=0).stop 2024-09-11T00:04:23.803 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:23.802+0000 7ff67e600640 1 -- 172.21.15.12:0/2766332908 shutdown_connections 2024-09-11T00:04:23.803 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:23.802+0000 7ff67e600640 1 --2- 172.21.15.12:0/2766332908 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7ff65c0b5390 0x7ff65c0b7850 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:04:23.803 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:23.802+0000 7ff67e600640 1 --2- 172.21.15.12:0/2766332908 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff67810f200 0x7ff6781a0b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:04:23.803 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:23.802+0000 7ff67e600640 1 --2- 172.21.15.12:0/2766332908 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff678075e00 0x7ff67819b790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:04:23.803 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:23.802+0000 7ff67e600640 1 --2- 172.21.15.12:0/2766332908 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff6780754e0 0x7ff67819b250 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:04:23.803 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:23.802+0000 7ff67e600640 1 -- 172.21.15.12:0/2766332908 >> 172.21.15.12:0/2766332908 conn(0x7ff6780fe660 msgr2=0x7ff6781113d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:04:23.803 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:23.802+0000 7ff67e600640 1 -- 172.21.15.12:0/2766332908 shutdown_connections 2024-09-11T00:04:23.804 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:23.802+0000 7ff67e600640 1 -- 172.21.15.12:0/2766332908 wait complete. 2024-09-11T00:04:24.063 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:04:23 smithi012 ceph-mon[30333]: pgmap v313: 129 pgs: 129 active+clean; 579 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T00:04:24.063 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:04:23 smithi012 ceph-mon[30333]: from='client.? 172.21.15.12:0/2766332908' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T00:04:24.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:04:23 smithi165 ceph-mon[35356]: pgmap v313: 129 pgs: 129 active+clean; 579 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T00:04:24.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:04:23 smithi165 ceph-mon[35356]: from='client.? 172.21.15.12:0/2766332908' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T00:04:24.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:04:23 smithi178 ceph-mon[35919]: pgmap v313: 129 pgs: 129 active+clean; 579 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T00:04:24.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:04:23 smithi178 ceph-mon[35919]: from='client.? 172.21.15.12:0/2766332908' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T00:04:24.610 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid ff3ad74a-6fcf-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-11T00:04:24.992 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:04:25.536 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:25.535+0000 7f47e3200640 1 -- 172.21.15.12:0/1330040326 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f47e40776d0 msgr2=0x7f47e4077ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:04:25.536 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:25.535+0000 7f47e3200640 1 --2- 172.21.15.12:0/1330040326 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f47e40776d0 0x7f47e4077ab0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f47d4009900 tx=0x7f47d402a3f0 comp rx=0 tx=0).stop 2024-09-11T00:04:25.536 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:25.535+0000 7f47e3200640 1 -- 172.21.15.12:0/1330040326 shutdown_connections 2024-09-11T00:04:25.536 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:25.535+0000 7f47e3200640 1 --2- 172.21.15.12:0/1330040326 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f47e41137e0 0x7f47e4115bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:04:25.537 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:25.535+0000 7f47e3200640 1 --2- 172.21.15.12:0/1330040326 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f47e4077ff0 0x7f47e41132a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:04:25.537 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:25.535+0000 7f47e3200640 1 --2- 172.21.15.12:0/1330040326 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f47e40776d0 0x7f47e4077ab0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:04:25.537 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:25.535+0000 7f47e3200640 1 -- 172.21.15.12:0/1330040326 >> 172.21.15.12:0/1330040326 conn(0x7f47e4100840 msgr2=0x7f47e4102c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:04:25.537 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:25.535+0000 7f47e3200640 1 -- 172.21.15.12:0/1330040326 shutdown_connections 2024-09-11T00:04:25.537 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:25.535+0000 7f47e3200640 1 -- 172.21.15.12:0/1330040326 wait complete. 2024-09-11T00:04:25.538 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:25.536+0000 7f47e3200640 1 Processor -- start 2024-09-11T00:04:25.538 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:25.536+0000 7f47e3200640 1 -- start start 2024-09-11T00:04:25.538 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:25.536+0000 7f47e3200640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f47e40776d0 0x7f47e419f610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:04:25.538 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:25.536+0000 7f47e3200640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f47e4077ff0 0x7f47e419fb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:04:25.539 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:25.536+0000 7f47e3200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f47e41137e0 0x7f47e41a4ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:04:25.539 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:25.536+0000 7f47e3200640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f47e4119240 con 0x7f47e4077ff0 2024-09-11T00:04:25.539 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:25.536+0000 7f47e3200640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f47e41190c0 con 0x7f47e41137e0 2024-09-11T00:04:25.539 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:25.536+0000 7f47e3200640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f47e41193c0 con 0x7f47e40776d0 2024-09-11T00:04:25.539 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:25.537+0000 7f47e1400640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f47e4077ff0 0x7f47e419fb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:04:25.539 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:25.537+0000 7f47e1e00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f47e40776d0 0x7f47e419f610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:04:25.539 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:25.537+0000 7f47e1400640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f47e4077ff0 0x7f47e419fb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:50598/0 (socket says 172.21.15.12:50598) 2024-09-11T00:04:25.539 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:25.537+0000 7f47e1400640 1 -- 172.21.15.12:0/1081395051 learned_addr learned my addr 172.21.15.12:0/1081395051 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:04:25.539 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:25.537+0000 7f47e2800640 1 --2- 172.21.15.12:0/1081395051 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f47e41137e0 0x7f47e41a4ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:04:25.539 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:25.537+0000 7f47e1e00640 1 -- 172.21.15.12:0/1081395051 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f47e41137e0 msgr2=0x7f47e41a4ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:04:25.539 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:25.537+0000 7f47e1e00640 1 --2- 172.21.15.12:0/1081395051 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f47e41137e0 0x7f47e41a4ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:04:25.539 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:25.537+0000 7f47e1e00640 1 -- 172.21.15.12:0/1081395051 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f47e4077ff0 msgr2=0x7f47e419fb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:04:25.540 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:25.537+0000 7f47e1e00640 1 --2- 172.21.15.12:0/1081395051 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f47e4077ff0 0x7f47e419fb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:04:25.540 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:25.537+0000 7f47e1e00640 1 -- 172.21.15.12:0/1081395051 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47e41a56c0 con 0x7f47e40776d0 2024-09-11T00:04:25.540 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:25.538+0000 7f47e1e00640 1 --2- 172.21.15.12:0/1081395051 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f47e40776d0 0x7f47e419f610 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f47d402a9e0 tx=0x7f47d4003b90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:04:25.541 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:25.538+0000 7f47daa00640 1 -- 172.21.15.12:0/1081395051 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47d403f070 con 0x7f47e40776d0 2024-09-11T00:04:25.541 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:25.538+0000 7f47daa00640 1 -- 172.21.15.12:0/1081395051 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f47d4031b50 con 0x7f47e40776d0 2024-09-11T00:04:25.541 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:25.538+0000 7f47daa00640 1 -- 172.21.15.12:0/1081395051 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47d4035030 con 0x7f47e40776d0 2024-09-11T00:04:25.541 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:25.538+0000 7f47e3200640 1 -- 172.21.15.12:0/1081395051 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f47e41a5950 con 0x7f47e40776d0 2024-09-11T00:04:25.541 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:25.538+0000 7f47e3200640 1 -- 172.21.15.12:0/1081395051 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47e41ac070 con 0x7f47e40776d0 2024-09-11T00:04:25.541 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:25.539+0000 7f47e3200640 1 -- 172.21.15.12:0/1081395051 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f47e4077ab0 con 0x7f47e40776d0 2024-09-11T00:04:25.542 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:25.542+0000 7f47daa00640 1 -- 172.21.15.12:0/1081395051 <== mon.1 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 16) ==== 153250+0+0 (secure 0 0 0) 0x7f47d40316a0 con 0x7f47e40776d0 2024-09-11T00:04:25.545 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:25.543+0000 7f47daa00640 1 --2- 172.21.15.12:0/1081395051 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f47c80b5410 0x7f47c80b78d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:04:25.545 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:25.543+0000 7f47daa00640 1 -- 172.21.15.12:0/1081395051 <== mon.1 v2:172.21.15.178:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8802+0+0 (secure 0 0 0) 0x7f47d41067f0 con 0x7f47e40776d0 2024-09-11T00:04:25.545 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:25.543+0000 7f47e1400640 1 --2- 172.21.15.12:0/1081395051 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f47c80b5410 0x7f47c80b78d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:04:25.545 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:25.544+0000 7f47e1400640 1 --2- 172.21.15.12:0/1081395051 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f47c80b5410 0x7f47c80b78d0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f47e407a0f0 tx=0x7f47d0005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:04:25.545 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:25.545+0000 7f47daa00640 1 -- 172.21.15.12:0/1081395051 <== mon.1 v2:172.21.15.178:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f47d4031d00 con 0x7f47e40776d0 2024-09-11T00:04:25.742 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:25.741+0000 7f47e3200640 1 -- 172.21.15.12:0/1081395051 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f47e41ac350 con 0x7f47e40776d0 2024-09-11T00:04:25.743 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:25.742+0000 7f47daa00640 1 -- 172.21.15.12:0/1081395051 <== mon.1 v2:172.21.15.178:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) ==== 74+0+24236 (secure 0 0 0) 0x7f47d40ccdc0 con 0x7f47e40776d0 2024-09-11T00:04:25.744 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-11T00:04:25.744 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":81,"fsid":"ff3ad74a-6fcf-11ef-bcea-c7b262605968","created":"2024-09-10T23:55:16.317643+0000","modified":"2024-09-11T00:04:15.795652+0000","last_up_change":"2024-09-11T00:03:08.959256+0000","last_in_change":"2024-09-11T00:02:46.564631+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T23:59:10.488728+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-11T00:04:00.065618+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"74","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-11T00:04:02.689888+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"74","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-11T00:04:09.862120+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"6382c200-a912-42a1-958b-79191b21249b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6802","nonce":1237101991},{"type":"v1","addr":"172.21.15.12:6803","nonce":1237101991}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6804","nonce":1237101991},{"type":"v1","addr":"172.21.15.12:6805","nonce":1237101991}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6808","nonce":1237101991},{"type":"v1","addr":"172.21.15.12:6809","nonce":1237101991}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6806","nonce":1237101991},{"type":"v1","addr":"172.21.15.12:6807","nonce":1237101991}]},"public_addr":"172.21.15.12:6803/1237101991","cluster_addr":"172.21.15.12:6805/1237101991","heartbeat_back_addr":"172.21.15.12:6809/1237101991","heartbeat_front_addr":"172.21.15.12:6807/1237101991","state":["exists","up"]},{"osd":1,"uuid":"fe0e1d0a-bdc5-4859-aa84-9038059a45f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6800","nonce":545848456},{"type":"v1","addr":"172.21.15.165:6801","nonce":545848456}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6802","nonce":545848456},{"type":"v1","addr":"172.21.15.165:6803","nonce":545848456}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6806","nonce":545848456},{"type":"v1","addr":"172.21.15.165:6807","nonce":545848456}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6804","nonce":545848456},{"type":"v1","addr":"172.21.15.165:6805","nonce":545848456}]},"public_addr":"172.21.15.165:6801/545848456","cluster_addr":"172.21.15.165:6803/545848456","heartbeat_back_addr":"172.21.15.165:6807/545848456","heartbeat_front_addr":"172.21.15.165:6805/545848456","state":["exists","up"]},{"osd":2,"uuid":"94f640dc-5f78-4e2c-87d9-03e96a6e7ba0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6800","nonce":4131162695},{"type":"v1","addr":"172.21.15.178:6801","nonce":4131162695}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":4131162695},{"type":"v1","addr":"172.21.15.178:6803","nonce":4131162695}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":4131162695},{"type":"v1","addr":"172.21.15.178:6807","nonce":4131162695}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":4131162695},{"type":"v1","addr":"172.21.15.178:6805","nonce":4131162695}]},"public_addr":"172.21.15.178:6801/4131162695","cluster_addr":"172.21.15.178:6803/4131162695","heartbeat_back_addr":"172.21.15.178:6807/4131162695","heartbeat_front_addr":"172.21.15.178:6805/4131162695","state":["exists","up"]},{"osd":3,"uuid":"af2f64d6-c9be-44a9-b041-8d599a7c080d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6810","nonce":452464245},{"type":"v1","addr":"172.21.15.12:6811","nonce":452464245}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6812","nonce":452464245},{"type":"v1","addr":"172.21.15.12:6813","nonce":452464245}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6816","nonce":452464245},{"type":"v1","addr":"172.21.15.12:6817","nonce":452464245}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6814","nonce":452464245},{"type":"v1","addr":"172.21.15.12:6815","nonce":452464245}]},"public_addr":"172.21.15.12:6811/452464245","cluster_addr":"172.21.15.12:6813/452464245","heartbeat_back_addr":"172.21.15.12:6817/452464245","heartbeat_front_addr":"172.21.15.12:6815/452464245","state":["exists","up"]},{"osd":4,"uuid":"ad76ac70-20c0-44ae-9866-5f9cfce2334c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6808","nonce":1899879280},{"type":"v1","addr":"172.21.15.165:6809","nonce":1899879280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6810","nonce":1899879280},{"type":"v1","addr":"172.21.15.165:6811","nonce":1899879280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6814","nonce":1899879280},{"type":"v1","addr":"172.21.15.165:6815","nonce":1899879280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6812","nonce":1899879280},{"type":"v1","addr":"172.21.15.165:6813","nonce":1899879280}]},"public_addr":"172.21.15.165:6809/1899879280","cluster_addr":"172.21.15.165:6811/1899879280","heartbeat_back_addr":"172.21.15.165:6815/1899879280","heartbeat_front_addr":"172.21.15.165:6813/1899879280","state":["exists","up"]},{"osd":5,"uuid":"aafda2cc-62fe-4fcb-9a96-8732e0b501d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":3042228555},{"type":"v1","addr":"172.21.15.178:6809","nonce":3042228555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":3042228555},{"type":"v1","addr":"172.21.15.178:6811","nonce":3042228555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6814","nonce":3042228555},{"type":"v1","addr":"172.21.15.178:6815","nonce":3042228555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":3042228555},{"type":"v1","addr":"172.21.15.178:6813","nonce":3042228555}]},"public_addr":"172.21.15.178:6809/3042228555","cluster_addr":"172.21.15.178:6811/3042228555","heartbeat_back_addr":"172.21.15.178:6815/3042228555","heartbeat_front_addr":"172.21.15.178:6813/3042228555","state":["exists","up"]},{"osd":6,"uuid":"375eca52-efe3-42a5-bbb1-d7d392d71f15","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6818","nonce":3868322467},{"type":"v1","addr":"172.21.15.12:6819","nonce":3868322467}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6820","nonce":3868322467},{"type":"v1","addr":"172.21.15.12:6821","nonce":3868322467}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6824","nonce":3868322467},{"type":"v1","addr":"172.21.15.12:6825","nonce":3868322467}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6822","nonce":3868322467},{"type":"v1","addr":"172.21.15.12:6823","nonce":3868322467}]},"public_addr":"172.21.15.12:6819/3868322467","cluster_addr":"172.21.15.12:6821/3868322467","heartbeat_back_addr":"172.21.15.12:6825/3868322467","heartbeat_front_addr":"172.21.15.12:6823/3868322467","state":["exists","up"]},{"osd":7,"uuid":"0fd619f5-44c3-4cf7-ae7a-388dd798de36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6816","nonce":1143773735},{"type":"v1","addr":"172.21.15.165:6817","nonce":1143773735}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6818","nonce":1143773735},{"type":"v1","addr":"172.21.15.165:6819","nonce":1143773735}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6822","nonce":1143773735},{"type":"v1","addr":"172.21.15.165:6823","nonce":1143773735}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6820","nonce":1143773735},{"type":"v1","addr":"172.21.15.165:6821","nonce":1143773735}]},"public_addr":"172.21.15.165:6817/1143773735","cluster_addr":"172.21.15.165:6819/1143773735","heartbeat_back_addr":"172.21.15.165:6823/1143773735","heartbeat_front_addr":"172.21.15.165:6821/1143773735","state":["exists","up"]},{"osd":8,"uuid":"b03d14ce-8427-462a-9bcc-27683c549f53","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":2891445058},{"type":"v1","addr":"172.21.15.178:6817","nonce":2891445058}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6818","nonce":2891445058},{"type":"v1","addr":"172.21.15.178:6819","nonce":2891445058}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6822","nonce":2891445058},{"type":"v1","addr":"172.21.15.178:6823","nonce":2891445058}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":2891445058},{"type":"v1","addr":"172.21.15.178:6821","nonce":2891445058}]},"public_addr":"172.21.15.178:6817/2891445058","cluster_addr":"172.21.15.178:6819/2891445058","heartbeat_back_addr":"172.21.15.178:6823/2891445058","heartbeat_front_addr":"172.21.15.178:6821/2891445058","state":["exists","up"]},{"osd":9,"uuid":"81a6cf69-2a24-4a6c-9adf-6b7df38cfacb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6826","nonce":3359634903},{"type":"v1","addr":"172.21.15.12:6827","nonce":3359634903}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6828","nonce":3359634903},{"type":"v1","addr":"172.21.15.12:6829","nonce":3359634903}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6832","nonce":3359634903},{"type":"v1","addr":"172.21.15.12:6833","nonce":3359634903}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6830","nonce":3359634903},{"type":"v1","addr":"172.21.15.12:6831","nonce":3359634903}]},"public_addr":"172.21.15.12:6827/3359634903","cluster_addr":"172.21.15.12:6829/3359634903","heartbeat_back_addr":"172.21.15.12:6833/3359634903","heartbeat_front_addr":"172.21.15.12:6831/3359634903","state":["exists","up"]},{"osd":10,"uuid":"c1ad1f0a-33f4-4c32-91f6-4d030a29f1dd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6824","nonce":3642325778},{"type":"v1","addr":"172.21.15.165:6825","nonce":3642325778}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6826","nonce":3642325778},{"type":"v1","addr":"172.21.15.165:6827","nonce":3642325778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6830","nonce":3642325778},{"type":"v1","addr":"172.21.15.165:6831","nonce":3642325778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6828","nonce":3642325778},{"type":"v1","addr":"172.21.15.165:6829","nonce":3642325778}]},"public_addr":"172.21.15.165:6825/3642325778","cluster_addr":"172.21.15.165:6827/3642325778","heartbeat_back_addr":"172.21.15.165:6831/3642325778","heartbeat_front_addr":"172.21.15.165:6829/3642325778","state":["exists","up"]},{"osd":11,"uuid":"8413c9b0-680b-4d02-aeab-b3b3ff16c3d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":2937059756},{"type":"v1","addr":"172.21.15.178:6825","nonce":2937059756}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6826","nonce":2937059756},{"type":"v1","addr":"172.21.15.178:6827","nonce":2937059756}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":2937059756},{"type":"v1","addr":"172.21.15.178:6831","nonce":2937059756}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":2937059756},{"type":"v1","addr":"172.21.15.178:6829","nonce":2937059756}]},"public_addr":"172.21.15.178:6825/2937059756","cluster_addr":"172.21.15.178:6827/2937059756","heartbeat_back_addr":"172.21.15.178:6831/2937059756","heartbeat_front_addr":"172.21.15.178:6829/2937059756","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:58:16.036348+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:58:41.483150+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:59:07.107283+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:59:35.706574+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T00:00:02.555720+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T00:00:28.092847+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T00:00:57.396495+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T00:01:24.216775+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T00:01:48.807850+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T00:02:16.838565+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T00:02:43.131736+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T00:03:07.353155+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.12:6801/4177251511":"2024-09-11T23:56:17.999820+0000","172.21.15.12:6800/4177251511":"2024-09-11T23:56:17.999820+0000","172.21.15.12:0/3273692817":"2024-09-11T23:56:17.999820+0000","172.21.15.12:0/4017778158":"2024-09-11T23:56:17.999820+0000","172.21.15.12:0/2499921051":"2024-09-11T23:55:50.302721+0000","172.21.15.12:6801/760140910":"2024-09-11T23:55:50.302721+0000","172.21.15.12:6800/760140910":"2024-09-11T23:55:50.302721+0000","172.21.15.12:0/2810703302":"2024-09-11T23:56:17.999820+0000","172.21.15.12:0/2325436065":"2024-09-11T23:55:50.302721+0000","172.21.15.12:0/3137053479":"2024-09-11T23:55:50.302721+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-11T00:04:25.746 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:25.745+0000 7f47e3200640 1 -- 172.21.15.12:0/1081395051 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f47c80b5410 msgr2=0x7f47c80b78d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:04:25.746 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:25.745+0000 7f47e3200640 1 --2- 172.21.15.12:0/1081395051 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f47c80b5410 0x7f47c80b78d0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f47e407a0f0 tx=0x7f47d0005f90 comp rx=0 tx=0).stop 2024-09-11T00:04:25.746 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:25.745+0000 7f47e3200640 1 -- 172.21.15.12:0/1081395051 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f47e40776d0 msgr2=0x7f47e419f610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:04:25.746 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:25.745+0000 7f47e3200640 1 --2- 172.21.15.12:0/1081395051 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f47e40776d0 0x7f47e419f610 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f47d402a9e0 tx=0x7f47d4003b90 comp rx=0 tx=0).stop 2024-09-11T00:04:25.746 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:25.745+0000 7f47e3200640 1 -- 172.21.15.12:0/1081395051 shutdown_connections 2024-09-11T00:04:25.747 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:25.745+0000 7f47e3200640 1 --2- 172.21.15.12:0/1081395051 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f47c80b5410 0x7f47c80b78d0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:04:25.747 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:25.745+0000 7f47e3200640 1 --2- 172.21.15.12:0/1081395051 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f47e41137e0 0x7f47e41a4ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:04:25.747 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:25.745+0000 7f47e3200640 1 --2- 172.21.15.12:0/1081395051 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f47e4077ff0 0x7f47e419fb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:04:25.747 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:25.745+0000 7f47e3200640 1 --2- 172.21.15.12:0/1081395051 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f47e40776d0 0x7f47e419f610 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:04:25.747 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:25.745+0000 7f47e3200640 1 -- 172.21.15.12:0/1081395051 >> 172.21.15.12:0/1081395051 conn(0x7f47e4100840 msgr2=0x7f47e4102c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:04:25.747 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:25.745+0000 7f47e3200640 1 -- 172.21.15.12:0/1081395051 shutdown_connections 2024-09-11T00:04:25.747 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:25.745+0000 7f47e3200640 1 -- 172.21.15.12:0/1081395051 wait complete. 2024-09-11T00:04:26.063 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:04:25 smithi012 ceph-mon[30333]: pgmap v314: 129 pgs: 129 active+clean; 579 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T00:04:26.063 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:04:25 smithi012 ceph-mon[30333]: from='client.? 172.21.15.12:0/1081395051' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-11T00:04:26.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:04:25 smithi165 ceph-mon[35356]: pgmap v314: 129 pgs: 129 active+clean; 579 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T00:04:26.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:04:25 smithi165 ceph-mon[35356]: from='client.? 172.21.15.12:0/1081395051' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-11T00:04:26.330 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:04:25 smithi178 ceph-mon[35919]: pgmap v314: 129 pgs: 129 active+clean; 579 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T00:04:26.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:04:25 smithi178 ceph-mon[35919]: from='client.? 172.21.15.12:0/1081395051' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-11T00:04:26.492 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 17, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-11T00:04:04.739881+0000', 'modified': '2024-09-11T00:04:15.795786+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 24674}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24674': {'gid': 24674, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.165:6839/501312984', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.165:6838', 'nonce': 501312984}, {'type': 'v1', 'addr': '172.21.15.165:6839', 'nonce': 501312984}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 1, 'qdb_leader': 24674, 'qdb_cluster': [24674]} 2024-09-11T00:04:26.492 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-11T00:04:26.492 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid ff3ad74a-6fcf-11ef-bcea-c7b262605968 -- ceph tell mds.b status 2024-09-11T00:04:26.870 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:04:27.412 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:27.410+0000 7f45b3200640 1 -- 172.21.15.12:0/2036914295 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f45ac104df0 msgr2=0x7f45ac1051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:04:27.412 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:27.410+0000 7f45b3200640 1 --2- 172.21.15.12:0/2036914295 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f45ac104df0 0x7f45ac1051d0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f45a0009860 tx=0x7f45a002a340 comp rx=0 tx=0).stop 2024-09-11T00:04:27.412 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:27.410+0000 7f45b3200640 1 -- 172.21.15.12:0/2036914295 shutdown_connections 2024-09-11T00:04:27.413 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:27.410+0000 7f45b3200640 1 --2- 172.21.15.12:0/2036914295 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f45ac10b180 0x7f45ac111980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:04:27.413 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:27.410+0000 7f45b3200640 1 --2- 172.21.15.12:0/2036914295 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f45ac10a800 0x7f45ac10ac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:04:27.413 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:27.410+0000 7f45b3200640 1 --2- 172.21.15.12:0/2036914295 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f45ac104df0 0x7f45ac1051d0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:04:27.413 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:27.410+0000 7f45b3200640 1 -- 172.21.15.12:0/2036914295 >> 172.21.15.12:0/2036914295 conn(0x7f45ac100840 msgr2=0x7f45ac102c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:04:27.413 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:27.410+0000 7f45b3200640 1 -- 172.21.15.12:0/2036914295 shutdown_connections 2024-09-11T00:04:27.413 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:27.410+0000 7f45b3200640 1 -- 172.21.15.12:0/2036914295 wait complete. 2024-09-11T00:04:27.414 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:27.411+0000 7f45b3200640 1 Processor -- start 2024-09-11T00:04:27.415 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:27.411+0000 7f45b3200640 1 -- start start 2024-09-11T00:04:27.415 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:27.411+0000 7f45b3200640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f45ac104df0 0x7f45ac19b1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:04:27.415 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:27.411+0000 7f45b3200640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f45ac10a800 0x7f45ac19b6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:04:27.415 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:27.411+0000 7f45b3200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f45ac10b180 0x7f45ac1a0a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:04:27.415 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:27.411+0000 7f45b3200640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f45ac114e40 con 0x7f45ac10a800 2024-09-11T00:04:27.415 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:27.411+0000 7f45b3200640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f45ac114cc0 con 0x7f45ac10b180 2024-09-11T00:04:27.415 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:27.411+0000 7f45b3200640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f45ac114fc0 con 0x7f45ac104df0 2024-09-11T00:04:27.415 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:27.412+0000 7f45b1400640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f45ac10a800 0x7f45ac19b6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:04:27.415 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:27.412+0000 7f45b1400640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f45ac10a800 0x7f45ac19b6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:50628/0 (socket says 172.21.15.12:50628) 2024-09-11T00:04:27.415 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:27.412+0000 7f45b1400640 1 -- 172.21.15.12:0/1815666111 learned_addr learned my addr 172.21.15.12:0/1815666111 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:04:27.415 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:27.412+0000 7f45b2800640 1 --2- 172.21.15.12:0/1815666111 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f45ac10b180 0x7f45ac1a0a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:04:27.415 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:27.412+0000 7f45b1e00640 1 --2- 172.21.15.12:0/1815666111 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f45ac104df0 0x7f45ac19b1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:04:27.416 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:27.412+0000 7f45b1400640 1 -- 172.21.15.12:0/1815666111 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f45ac104df0 msgr2=0x7f45ac19b1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:04:27.416 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:27.412+0000 7f45b1400640 1 --2- 172.21.15.12:0/1815666111 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f45ac104df0 0x7f45ac19b1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:04:27.416 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:27.412+0000 7f45b1400640 1 -- 172.21.15.12:0/1815666111 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f45ac10b180 msgr2=0x7f45ac1a0a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:04:27.416 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:27.412+0000 7f45b1400640 1 --2- 172.21.15.12:0/1815666111 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f45ac10b180 0x7f45ac1a0a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:04:27.416 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:27.412+0000 7f45b1400640 1 -- 172.21.15.12:0/1815666111 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45ac1a1260 con 0x7f45ac10a800 2024-09-11T00:04:27.416 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:27.412+0000 7f45b2800640 1 --2- 172.21.15.12:0/1815666111 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f45ac10b180 0x7f45ac1a0a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:04:27.416 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:27.413+0000 7f45b1400640 1 --2- 172.21.15.12:0/1815666111 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f45ac10a800 0x7f45ac19b6f0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f459c00e9c0 tx=0x7f459c00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:04:27.417 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:27.413+0000 7f45a6a00640 1 -- 172.21.15.12:0/1815666111 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f459c00ccc0 con 0x7f45ac10a800 2024-09-11T00:04:27.417 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:27.413+0000 7f45b3200640 1 -- 172.21.15.12:0/1815666111 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f45ac1a1550 con 0x7f45ac10a800 2024-09-11T00:04:27.417 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:27.413+0000 7f45a6a00640 1 -- 172.21.15.12:0/1815666111 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f459c0054d0 con 0x7f45ac10a800 2024-09-11T00:04:27.417 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:27.413+0000 7f45a6a00640 1 -- 172.21.15.12:0/1815666111 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f459c010430 con 0x7f45ac10a800 2024-09-11T00:04:27.417 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:27.413+0000 7f45b3200640 1 -- 172.21.15.12:0/1815666111 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45ac1a7de0 con 0x7f45ac10a800 2024-09-11T00:04:27.418 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:27.416+0000 7f45b71cb740 1 Processor -- start 2024-09-11T00:04:27.418 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:27.416+0000 7f45b71cb740 1 -- start start 2024-09-11T00:04:27.418 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:27.417+0000 7f45a6a00640 1 -- 172.21.15.12:0/1815666111 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 16) ==== 153250+0+0 (secure 0 0 0) 0x7f459c004f80 con 0x7f45ac10a800 2024-09-11T00:04:27.420 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:27.417+0000 7f45b71cb740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f272dcc400 0x55f272dcc7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:04:27.420 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:27.417+0000 7f45b71cb740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f272dccd20 0x55f272dd8770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:04:27.420 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:27.417+0000 7f45b71cb740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f272dd8cb0 0x55f272ddb0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:04:27.420 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:27.417+0000 7f45b71cb740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55f272d43d50 con 0x55f272dccd20 2024-09-11T00:04:27.420 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:27.417+0000 7f45b71cb740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55f272d33180 con 0x55f272dd8cb0 2024-09-11T00:04:27.420 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:27.417+0000 7f45b71cb740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55f272d36f90 con 0x55f272dcc400 2024-09-11T00:04:27.420 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:27.417+0000 7f45b1e00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f272dccd20 0x55f272dd8770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:04:27.420 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:27.417+0000 7f45b2800640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f272dcc400 0x55f272dcc7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:04:27.420 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:27.417+0000 7f45b2800640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f272dcc400 0x55f272dcc7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.12:43210/0 (socket says 172.21.15.12:43210) 2024-09-11T00:04:27.420 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:27.417+0000 7f45b2800640 1 -- 172.21.15.12:0/2312712350 learned_addr learned my addr 172.21.15.12:0/2312712350 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:04:27.421 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:27.417+0000 7f45b1400640 1 --2- 172.21.15.12:0/2312712350 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f272dd8cb0 0x55f272ddb0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:04:27.421 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:27.418+0000 7f45b1e00640 1 -- 172.21.15.12:0/2312712350 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f272dcc400 msgr2=0x55f272dcc7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:04:27.421 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:27.418+0000 7f45b1e00640 1 --2- 172.21.15.12:0/2312712350 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f272dcc400 0x55f272dcc7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:04:27.421 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:27.418+0000 7f45b1e00640 1 -- 172.21.15.12:0/2312712350 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f272dd8cb0 msgr2=0x55f272ddb0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:04:27.421 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:27.418+0000 7f45b1e00640 1 --2- 172.21.15.12:0/2312712350 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f272dd8cb0 0x55f272ddb0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:04:27.421 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:27.418+0000 7f45b1e00640 1 -- 172.21.15.12:0/2312712350 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f272cfda70 con 0x55f272dccd20 2024-09-11T00:04:27.421 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:27.418+0000 7f45b1400640 1 --2- 172.21.15.12:0/2312712350 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f272dd8cb0 0x55f272ddb0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:04:27.421 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:27.418+0000 7f45b1e00640 1 --2- 172.21.15.12:0/2312712350 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f272dccd20 0x55f272dd8770 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f45a002a860 tx=0x7f45a0005d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=391e5c818c1dbfd2 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:04:27.423 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:27.418+0000 7f459b000640 1 -- 172.21.15.12:0/2312712350 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45a0037040 con 0x55f272dccd20 2024-09-11T00:04:27.423 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:27.418+0000 7f459b000640 1 -- 172.21.15.12:0/2312712350 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f45a0042070 con 0x55f272dccd20 2024-09-11T00:04:27.423 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:27.418+0000 7f459b000640 1 -- 172.21.15.12:0/2312712350 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45a0003830 con 0x55f272dccd20 2024-09-11T00:04:27.423 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:27.419+0000 7f45a6a00640 1 --2- 172.21.15.12:0/1815666111 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f45940b5390 0x7f45940b7850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:04:27.423 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:27.419+0000 7f45b71cb740 1 -- 172.21.15.12:0/2312712350 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f272dccd20 msgr2=0x55f272dd8770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:04:27.424 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:27.419+0000 7f45b71cb740 1 --2- 172.21.15.12:0/2312712350 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f272dccd20 0x55f272dd8770 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f45a002a860 tx=0x7f45a0005d90 comp rx=0 tx=0).stop 2024-09-11T00:04:27.424 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:27.419+0000 7f45a6a00640 1 -- 172.21.15.12:0/1815666111 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8802+0+0 (secure 0 0 0) 0x7f459c0e6280 con 0x7f45ac10a800 2024-09-11T00:04:27.424 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:27.419+0000 7f45b2800640 1 --2- 172.21.15.12:0/1815666111 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f45940b5390 0x7f45940b7850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:04:27.424 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:27.419+0000 7f45b71cb740 1 -- 172.21.15.12:0/2312712350 shutdown_connections 2024-09-11T00:04:27.424 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:27.419+0000 7f45b71cb740 1 --2- 172.21.15.12:0/2312712350 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f272dd8cb0 0x55f272ddb0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:04:27.424 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:27.419+0000 7f45b71cb740 1 --2- 172.21.15.12:0/2312712350 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f272dccd20 0x55f272dd8770 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:04:27.424 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:27.419+0000 7f45b71cb740 1 --2- 172.21.15.12:0/2312712350 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f272dcc400 0x55f272dcc7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:04:27.424 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:27.419+0000 7f45b71cb740 1 -- 172.21.15.12:0/2312712350 >> 172.21.15.12:0/2312712350 conn(0x55f272dca230 msgr2=0x55f272dcb3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:04:27.424 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:27.420+0000 7f45b71cb740 1 -- 172.21.15.12:0/2312712350 shutdown_connections 2024-09-11T00:04:27.424 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:27.420+0000 7f45b71cb740 1 -- 172.21.15.12:0/2312712350 wait complete. 2024-09-11T00:04:27.424 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:27.420+0000 7f45b2800640 1 --2- 172.21.15.12:0/1815666111 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f45940b5390 0x7f45940b7850 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f45a8009fd0 tx=0x7f45a8009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:04:27.424 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:27.420+0000 7f45b71cb740 1 -- start start 2024-09-11T00:04:27.425 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:27.420+0000 7f45b71cb740 1 Processor -- start 2024-09-11T00:04:27.425 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:27.421+0000 7f45b71cb740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f272dcc400 0x55f272dd5cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:04:27.425 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:27.421+0000 7f45b71cb740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f272dccd20 0x55f272dd6210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:04:27.425 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:27.421+0000 7f45b71cb740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f272dd8cb0 0x55f272dd6750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:04:27.425 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:27.421+0000 7f45b71cb740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55f272b945f0 con 0x55f272dd8cb0 2024-09-11T00:04:27.425 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:27.421+0000 7f45b71cb740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55f272cce460 con 0x55f272dccd20 2024-09-11T00:04:27.425 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:27.421+0000 7f45b71cb740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55f272ddb9a0 con 0x55f272dcc400 2024-09-11T00:04:27.425 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:27.421+0000 7f45b71cb740 10 client.0 _ll_register_callbacks cb 0x55f272cde210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:04:27.425 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:27.421+0000 7f45b2800640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f272dd8cb0 0x55f272dd6750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:04:27.425 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:27.421+0000 7f45b1400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f272dccd20 0x55f272dd6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:04:27.425 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:27.421+0000 7f45b1e00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f272dcc400 0x55f272dd5cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:04:27.425 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:27.421+0000 7f45b1400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f272dccd20 0x55f272dd6210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.12:53014/0 (socket says 172.21.15.12:53014) 2024-09-11T00:04:27.425 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:27.421+0000 7f45b1400640 1 -- 172.21.15.12:0/3887222938 learned_addr learned my addr 172.21.15.12:0/3887222938 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:04:27.426 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:27.421+0000 7f45b1400640 1 -- 172.21.15.12:0/3887222938 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f272dcc400 msgr2=0x55f272dd5cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:04:27.426 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:27.421+0000 7f45b1400640 1 --2- 172.21.15.12:0/3887222938 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f272dcc400 0x55f272dd5cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:04:27.426 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:27.421+0000 7f45b1400640 1 -- 172.21.15.12:0/3887222938 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f272dd8cb0 msgr2=0x55f272dd6750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:04:27.426 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:27.421+0000 7f45b1400640 1 --2- 172.21.15.12:0/3887222938 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f272dd8cb0 0x55f272dd6750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:04:27.426 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:27.421+0000 7f45b1400640 1 -- 172.21.15.12:0/3887222938 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f272a91540 con 0x55f272dccd20 2024-09-11T00:04:27.428 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:27.422+0000 7f45b1400640 1 --2- 172.21.15.12:0/3887222938 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f272dccd20 0x55f272dd6210 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f459c0a8d70 tx=0x7f459c004a20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3c21acbe1d222b8c server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:04:27.428 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:27.422+0000 7f458f400640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-09-11T00:04:27.428 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:27.422+0000 7f458f400640 1 -- 172.21.15.12:0/3887222938 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f459c0ac570 con 0x55f272dccd20 2024-09-11T00:04:27.428 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:27.422+0000 7f45b71cb740 1 -- 172.21.15.12:0/3887222938 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55f272cd0a70 con 0x55f272dccd20 2024-09-11T00:04:27.428 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:27.422+0000 7f458f400640 1 -- 172.21.15.12:0/3887222938 <== mon.2 v2:172.21.15.165:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8802+0+0 (secure 0 0 0) 0x7f459c0e9dd0 con 0x55f272dccd20 2024-09-11T00:04:27.428 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:27.422+0000 7f458f400640 1 -- 172.21.15.12:0/3887222938 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) ==== 24+0+0 (secure 0 0 0) 0x7f459c0abf10 con 0x55f272dccd20 2024-09-11T00:04:27.429 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:27.422+0000 7f45b71cb740 10 client.24770 fetch_fsmap learned FSMap version 17 2024-09-11T00:04:27.429 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:27.422+0000 7f45b71cb740 1 -- 172.21.15.12:0/3887222938 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=4+}) -- 0x55f272ceb6f0 con 0x55f272dccd20 2024-09-11T00:04:27.429 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:27.422+0000 7f458f400640 1 -- 172.21.15.12:0/3887222938 <== mon.2 v2:172.21.15.165:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55f272cd0a70 con 0x55f272dccd20 2024-09-11T00:04:27.429 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:27.423+0000 7f458f400640 1 -- 172.21.15.12:0/3887222938 <== mon.2 v2:172.21.15.165:3300/0 5 ==== fsmap(e 17) ==== 8350+0+0 (secure 0 0 0) 0x7f459c0cfaf0 con 0x55f272dccd20 2024-09-11T00:04:27.429 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:27.423+0000 7f45b71cb740 10 client.24770 fetch_fsmap finished waiting for FSMap version 17 2024-09-11T00:04:27.429 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:27.423+0000 7f45b71cb740 10 client.24770 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-11T00:04:27.429 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:27.423+0000 7f45b71cb740 20 client.24770 populate_metadata read hostname 'smithi012' 2024-09-11T00:04:27.429 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:27.423+0000 7f45b71cb740 1 --2- 172.21.15.12:0/3887222938 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55f272de99c0 0x55f272deae30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:04:27.429 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:27.423+0000 7f45b71cb740 4 client.24770 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T00:04:27.429 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:27.423+0000 7f45b71cb740 1 -- 172.21.15.12:0/3887222938 --> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f272d45ca0 con 0x55f272de99c0 2024-09-11T00:04:27.429 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:27.423+0000 7f45b1e00640 1 --2- 172.21.15.12:0/3887222938 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55f272de99c0 0x55f272deae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:04:27.429 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:27.424+0000 7f45b1e00640 1 --2- 172.21.15.12:0/3887222938 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55f272de99c0 0x55f272deae30 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f8101a36721547d7 server_cookie=24a7a0acdfad6042 in_seq=0 out_seq=0 2024-09-11T00:04:27.429 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:27.424+0000 7f458f400640 10 client.24770 ms_handle_connect on v2:172.21.15.165:6838/501312984 2024-09-11T00:04:27.430 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:27.425+0000 7f458f400640 1 -- 172.21.15.12:0/3887222938 <== mds.0 v2:172.21.15.165:6838/501312984 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55f272d45ca0 con 0x55f272de99c0 2024-09-11T00:04:27.430 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:27.425+0000 7f458f400640 10 client.24770 handle_command_reply: tid=0 2024-09-11T00:04:27.430 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:27.425+0000 7f45b71cb740 1 client.24770 shutdown 2024-09-11T00:04:27.430 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:27.425+0000 7f45b71cb740 1 -- 172.21.15.12:0/3887222938 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f272dccd20 msgr2=0x55f272dd6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:04:27.430 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:27.425+0000 7f45b71cb740 1 --2- 172.21.15.12:0/3887222938 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f272dccd20 0x55f272dd6210 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f459c0a8d70 tx=0x7f459c004a20 comp rx=0 tx=0).stop 2024-09-11T00:04:27.430 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:27.425+0000 7f45b71cb740 1 -- 172.21.15.12:0/3887222938 shutdown_connections 2024-09-11T00:04:27.430 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:27.425+0000 7f45b71cb740 1 --2- 172.21.15.12:0/3887222938 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55f272de99c0 0x55f272deae30 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:04:27.430 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:27.425+0000 7f45b71cb740 1 --2- 172.21.15.12:0/3887222938 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f272dd8cb0 0x55f272dd6750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:04:27.430 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:27.425+0000 7f45b71cb740 1 --2- 172.21.15.12:0/3887222938 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f272dccd20 0x55f272dd6210 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:04:27.430 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:27.425+0000 7f45b71cb740 1 --2- 172.21.15.12:0/3887222938 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f272dcc400 0x55f272dd5cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:04:27.430 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:27.425+0000 7f45b71cb740 1 -- 172.21.15.12:0/3887222938 >> 172.21.15.12:0/3887222938 conn(0x55f272dca230 msgr2=0x55f272dcfdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:04:27.430 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:27.425+0000 7f458f400640 0 client.24770 ms_handle_reset on v2:172.21.15.165:6838/501312984 2024-09-11T00:04:27.430 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:27.425+0000 7f45b71cb740 1 -- 172.21.15.12:0/3887222938 shutdown_connections 2024-09-11T00:04:27.431 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:27.425+0000 7f45b71cb740 1 -- 172.21.15.12:0/3887222938 wait complete. 2024-09-11T00:04:27.431 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:27.425+0000 7f45b71cb740 20 client.24770 trim_cache size 0 max 16384 2024-09-11T00:04:27.446 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:27.445+0000 7f45b71cb740 1 Processor -- start 2024-09-11T00:04:27.447 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:27.445+0000 7f45b71cb740 1 -- start start 2024-09-11T00:04:27.447 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:27.445+0000 7f45b71cb740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f272dcc400 0x55f272dd05c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:04:27.447 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:27.445+0000 7f45b71cb740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f272dccd20 0x55f272dd6a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:04:27.448 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:27.445+0000 7f45b71cb740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f272dd8cb0 0x55f272dd0fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:04:27.448 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:27.445+0000 7f45b71cb740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55f272de9170 con 0x55f272dd8cb0 2024-09-11T00:04:27.448 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:27.445+0000 7f45b71cb740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55f272de9840 con 0x55f272dcc400 2024-09-11T00:04:27.448 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:27.445+0000 7f45b71cb740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55f272de9f10 con 0x55f272dccd20 2024-09-11T00:04:27.448 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:27.445+0000 7f45b2800640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f272dd8cb0 0x55f272dd0fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:04:27.448 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:27.446+0000 7f45b1400640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f272dccd20 0x55f272dd6a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:04:27.448 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:27.446+0000 7f45b2800640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f272dd8cb0 0x55f272dd0fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:50656/0 (socket says 172.21.15.12:50656) 2024-09-11T00:04:27.448 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:27.446+0000 7f45b1e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f272dcc400 0x55f272dd05c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:04:27.448 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:27.446+0000 7f45b2800640 1 -- 172.21.15.12:0/380821518 learned_addr learned my addr 172.21.15.12:0/380821518 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:04:27.449 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:27.446+0000 7f45b1400640 1 -- 172.21.15.12:0/380821518 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f272dcc400 msgr2=0x55f272dd05c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:04:27.450 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:27.446+0000 7f45b1400640 1 --2- 172.21.15.12:0/380821518 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f272dcc400 0x55f272dd05c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:04:27.450 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:27.446+0000 7f45b1400640 1 -- 172.21.15.12:0/380821518 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f272dd8cb0 msgr2=0x55f272dd0fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:04:27.450 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:27.446+0000 7f45b1400640 1 --2- 172.21.15.12:0/380821518 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f272dd8cb0 0x55f272dd0fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:04:27.450 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:27.446+0000 7f45b1400640 1 -- 172.21.15.12:0/380821518 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f272dea3c0 con 0x55f272dccd20 2024-09-11T00:04:27.450 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:27.446+0000 7f45b1400640 1 --2- 172.21.15.12:0/380821518 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f272dccd20 0x55f272dd6a90 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f459c0ac4a0 tx=0x7f459c0094c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f3119eb9c754db3b server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:04:27.450 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:27.446+0000 7f459a600640 1 -- 172.21.15.12:0/380821518 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f459c0ac0f0 con 0x55f272dccd20 2024-09-11T00:04:27.450 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:27.447+0000 7f459a600640 1 -- 172.21.15.12:0/380821518 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f459c0cf040 con 0x55f272dccd20 2024-09-11T00:04:27.450 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:27.447+0000 7f459a600640 1 -- 172.21.15.12:0/380821518 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f459c0ac570 con 0x55f272dccd20 2024-09-11T00:04:27.450 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:27.447+0000 7f45b71cb740 1 -- 172.21.15.12:0/380821518 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f272dccd20 msgr2=0x55f272dd6a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:04:27.450 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:27.447+0000 7f45b71cb740 1 --2- 172.21.15.12:0/380821518 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f272dccd20 0x55f272dd6a90 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f459c0ac4a0 tx=0x7f459c0094c0 comp rx=0 tx=0).stop 2024-09-11T00:04:27.450 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:27.447+0000 7f45b71cb740 1 -- 172.21.15.12:0/380821518 shutdown_connections 2024-09-11T00:04:27.450 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:27.447+0000 7f45b71cb740 1 --2- 172.21.15.12:0/380821518 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f272dd8cb0 0x55f272dd0fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:04:27.451 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:27.447+0000 7f45b71cb740 1 --2- 172.21.15.12:0/380821518 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f272dccd20 0x55f272dd6a90 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:04:27.451 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:27.447+0000 7f45b71cb740 1 --2- 172.21.15.12:0/380821518 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f272dcc400 0x55f272dd05c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:04:27.451 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:27.447+0000 7f45b71cb740 1 -- 172.21.15.12:0/380821518 >> 172.21.15.12:0/380821518 conn(0x55f272dca230 msgr2=0x55f272de88a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:04:27.451 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:27.447+0000 7f45b71cb740 1 -- 172.21.15.12:0/380821518 shutdown_connections 2024-09-11T00:04:27.451 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:27.447+0000 7f45b71cb740 1 -- 172.21.15.12:0/380821518 wait complete. 2024-09-11T00:04:27.451 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:27.448+0000 7f45b71cb740 1 -- start start 2024-09-11T00:04:27.451 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:27.448+0000 7f45b71cb740 1 Processor -- start 2024-09-11T00:04:27.454 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:27.448+0000 7f45b71cb740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f272dcc400 0x55f272df3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:04:27.454 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:27.448+0000 7f45b71cb740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f272dccd20 0x55f272decf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:04:27.454 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:27.448+0000 7f45b71cb740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f272dd8cb0 0x55f272ded6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:04:27.454 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:27.449+0000 7f45b71cb740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55f272deacb0 con 0x55f272dccd20 2024-09-11T00:04:27.454 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:27.449+0000 7f45b71cb740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55f272dea5e0 con 0x55f272dcc400 2024-09-11T00:04:27.454 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:27.449+0000 7f45b71cb740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55f272deb380 con 0x55f272dd8cb0 2024-09-11T00:04:27.454 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:27.449+0000 7f45b71cb740 10 client.0 _ll_register_callbacks cb 0x7f459c005680 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:04:27.455 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:27.449+0000 7f45b1400640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f272dccd20 0x55f272decf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:04:27.455 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:27.449+0000 7f45b1400640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f272dccd20 0x55f272decf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:50660/0 (socket says 172.21.15.12:50660) 2024-09-11T00:04:27.455 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:27.449+0000 7f45b2800640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f272dd8cb0 0x55f272ded6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:04:27.455 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:27.449+0000 7f45b1400640 1 -- 172.21.15.12:0/635614369 learned_addr learned my addr 172.21.15.12:0/635614369 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:04:27.455 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:27.449+0000 7f45b1e00640 1 --2- 172.21.15.12:0/635614369 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f272dcc400 0x55f272df3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:04:27.455 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:27.449+0000 7f45b1400640 1 -- 172.21.15.12:0/635614369 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f272dd8cb0 msgr2=0x55f272ded6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:04:27.455 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:27.449+0000 7f45b1400640 1 --2- 172.21.15.12:0/635614369 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f272dd8cb0 0x55f272ded6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:04:27.455 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:27.449+0000 7f45b1400640 1 -- 172.21.15.12:0/635614369 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f272dcc400 msgr2=0x55f272df3ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:04:27.455 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:27.449+0000 7f45b1400640 1 --2- 172.21.15.12:0/635614369 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f272dcc400 0x55f272df3ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:04:27.455 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:27.449+0000 7f45b1400640 1 -- 172.21.15.12:0/635614369 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f272df2710 con 0x55f272dccd20 2024-09-11T00:04:27.455 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:27.449+0000 7f45b1400640 1 --2- 172.21.15.12:0/635614369 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f272dccd20 0x55f272decf80 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f459c009530 tx=0x7f459c0cfd20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a8f79f638e3971 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:04:27.455 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:27.449+0000 7f458f400640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-09-11T00:04:27.456 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:27.450+0000 7f458f400640 1 -- 172.21.15.12:0/635614369 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f459c0ac570 con 0x55f272dccd20 2024-09-11T00:04:27.456 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:27.450+0000 7f458f400640 1 -- 172.21.15.12:0/635614369 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8802+0+0 (secure 0 0 0) 0x7f459c0a80f0 con 0x55f272dccd20 2024-09-11T00:04:27.456 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:27.450+0000 7f45b71cb740 1 -- 172.21.15.12:0/635614369 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f459c0cf040 con 0x55f272dccd20 2024-09-11T00:04:27.456 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:27.450+0000 7f458f400640 1 -- 172.21.15.12:0/635614369 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) ==== 24+0+0 (secure 0 0 0) 0x7f459c0ac0f0 con 0x55f272dccd20 2024-09-11T00:04:27.456 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:27.450+0000 7f45b71cb740 10 client.14739 fetch_fsmap learned FSMap version 17 2024-09-11T00:04:27.456 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:27.450+0000 7f45b71cb740 1 -- 172.21.15.12:0/635614369 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=4+}) -- 0x55f272e56580 con 0x55f272dccd20 2024-09-11T00:04:27.456 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:27.450+0000 7f458f400640 1 -- 172.21.15.12:0/635614369 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f459c0cf040 con 0x55f272dccd20 2024-09-11T00:04:27.456 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:27.450+0000 7f458f400640 1 -- 172.21.15.12:0/635614369 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 17) ==== 8350+0+0 (secure 0 0 0) 0x7f459c0a82f0 con 0x55f272dccd20 2024-09-11T00:04:27.456 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:27.451+0000 7f45b71cb740 10 client.14739 fetch_fsmap finished waiting for FSMap version 17 2024-09-11T00:04:27.456 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:27.451+0000 7f45b71cb740 10 client.14739 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-11T00:04:27.456 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:27.451+0000 7f45b71cb740 20 client.14739 populate_metadata read hostname 'smithi012' 2024-09-11T00:04:27.457 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:27.451+0000 7f45b71cb740 1 --2- 172.21.15.12:0/635614369 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55f272e5ace0 0x55f272e599f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:04:27.457 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:27.451+0000 7f45b71cb740 4 client.14739 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-11T00:04:27.457 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:27.451+0000 7f45b71cb740 1 -- 172.21.15.12:0/635614369 --> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] -- command(tid 0: {"prefix": "status"}) -- 0x55f272d41750 con 0x55f272e5ace0 2024-09-11T00:04:27.457 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:27.451+0000 7f45b1e00640 1 --2- 172.21.15.12:0/635614369 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55f272e5ace0 0x55f272e599f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:04:27.457 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:27.451+0000 7f45b1e00640 1 --2- 172.21.15.12:0/635614369 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55f272e5ace0 0x55f272e599f0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7894462c3ac6d24b server_cookie=f9c496b34bf53cb9 in_seq=0 out_seq=0 2024-09-11T00:04:27.457 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:27.451+0000 7f458f400640 10 client.14739 ms_handle_connect on v2:172.21.15.165:6838/501312984 2024-09-11T00:04:27.457 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:27.452+0000 7f458f400640 1 -- 172.21.15.12:0/635614369 <== mds.0 v2:172.21.15.165:6838/501312984 1 ==== command_reply(tid 0: 0 ) ==== 8+0+326 (crc 0 0 0) 0x55f272d41750 con 0x55f272e5ace0 2024-09-11T00:04:27.457 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:27.452+0000 7f458f400640 10 client.14739 handle_command_reply: tid=0 2024-09-11T00:04:27.457 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:27.452+0000 7f45b71cb740 1 client.14739 shutdown 2024-09-11T00:04:27.457 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:27.452+0000 7f45b71cb740 1 -- 172.21.15.12:0/635614369 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f272dccd20 msgr2=0x55f272decf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:04:27.457 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:27.452+0000 7f45b71cb740 1 --2- 172.21.15.12:0/635614369 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f272dccd20 0x55f272decf80 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f459c009530 tx=0x7f459c0cfd20 comp rx=0 tx=0).stop 2024-09-11T00:04:27.457 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:27.452+0000 7f45b71cb740 1 -- 172.21.15.12:0/635614369 shutdown_connections 2024-09-11T00:04:27.457 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:27.452+0000 7f45b71cb740 1 --2- 172.21.15.12:0/635614369 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55f272e5ace0 0x55f272e599f0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:04:27.458 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:27.452+0000 7f45b71cb740 1 --2- 172.21.15.12:0/635614369 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f272dd8cb0 0x55f272ded6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:04:27.458 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:27.452+0000 7f45b71cb740 1 --2- 172.21.15.12:0/635614369 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f272dccd20 0x55f272decf80 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:04:27.458 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:27.452+0000 7f45b71cb740 1 --2- 172.21.15.12:0/635614369 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f272dcc400 0x55f272df3ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:04:27.458 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:27.452+0000 7f458f400640 0 client.14739 ms_handle_reset on v2:172.21.15.165:6838/501312984 2024-09-11T00:04:27.458 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:27.452+0000 7f45b71cb740 1 -- 172.21.15.12:0/635614369 >> 172.21.15.12:0/635614369 conn(0x55f272dca230 msgr2=0x55f272dd0fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:04:27.458 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:27.452+0000 7f45b71cb740 1 -- 172.21.15.12:0/635614369 shutdown_connections 2024-09-11T00:04:27.458 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:27.452+0000 7f45b71cb740 1 -- 172.21.15.12:0/635614369 wait complete. 2024-09-11T00:04:27.458 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:27.452+0000 7f45b71cb740 20 client.14739 trim_cache size 0 max 16384 2024-09-11T00:04:27.459 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-09-11T00:04:27.459 INFO:teuthology.orchestra.run.smithi012.stdout: "cluster_fsid": "ff3ad74a-6fcf-11ef-bcea-c7b262605968", 2024-09-11T00:04:27.459 INFO:teuthology.orchestra.run.smithi012.stdout: "whoami": 0, 2024-09-11T00:04:27.459 INFO:teuthology.orchestra.run.smithi012.stdout: "id": 24674, 2024-09-11T00:04:27.459 INFO:teuthology.orchestra.run.smithi012.stdout: "want_state": "up:active", 2024-09-11T00:04:27.459 INFO:teuthology.orchestra.run.smithi012.stdout: "state": "up:active", 2024-09-11T00:04:27.459 INFO:teuthology.orchestra.run.smithi012.stdout: "fs_name": "cephfs", 2024-09-11T00:04:27.459 INFO:teuthology.orchestra.run.smithi012.stdout: "rank_uptime": 22.614597383, 2024-09-11T00:04:27.459 INFO:teuthology.orchestra.run.smithi012.stdout: "mdsmap_epoch": 17, 2024-09-11T00:04:27.459 INFO:teuthology.orchestra.run.smithi012.stdout: "osdmap_epoch": 81, 2024-09-11T00:04:27.459 INFO:teuthology.orchestra.run.smithi012.stdout: "osdmap_epoch_barrier": 80, 2024-09-11T00:04:27.460 INFO:teuthology.orchestra.run.smithi012.stdout: "uptime": 35.523645784000003 2024-09-11T00:04:27.460 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-09-11T00:04:27.460 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:27.453+0000 7f45b3200640 1 -- 172.21.15.12:0/1815666111 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f45940b5390 msgr2=0x7f45940b7850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:04:27.460 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:27.453+0000 7f45b3200640 1 --2- 172.21.15.12:0/1815666111 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f45940b5390 0x7f45940b7850 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f45a8009fd0 tx=0x7f45a8009500 comp rx=0 tx=0).stop 2024-09-11T00:04:27.460 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:27.453+0000 7f45b3200640 1 -- 172.21.15.12:0/1815666111 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f45ac10a800 msgr2=0x7f45ac19b6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:04:27.460 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:27.453+0000 7f45b3200640 1 --2- 172.21.15.12:0/1815666111 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f45ac10a800 0x7f45ac19b6f0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f459c00e9c0 tx=0x7f459c00ee80 comp rx=0 tx=0).stop 2024-09-11T00:04:27.461 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:27.453+0000 7f45b3200640 1 -- 172.21.15.12:0/1815666111 shutdown_connections 2024-09-11T00:04:27.461 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:27.453+0000 7f45b3200640 1 --2- 172.21.15.12:0/1815666111 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f45940b5390 0x7f45940b7850 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:04:27.461 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:27.453+0000 7f45b3200640 1 --2- 172.21.15.12:0/1815666111 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f45ac10b180 0x7f45ac1a0a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:04:27.461 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:27.453+0000 7f45b3200640 1 --2- 172.21.15.12:0/1815666111 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f45ac10a800 0x7f45ac19b6f0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:04:27.461 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:27.453+0000 7f45b3200640 1 --2- 172.21.15.12:0/1815666111 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f45ac104df0 0x7f45ac19b1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:04:27.461 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:27.453+0000 7f45b3200640 1 -- 172.21.15.12:0/1815666111 >> 172.21.15.12:0/1815666111 conn(0x7f45ac100840 msgr2=0x7f45ac103d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:04:27.461 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:27.453+0000 7f45b3200640 1 -- 172.21.15.12:0/1815666111 shutdown_connections 2024-09-11T00:04:27.461 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:27.453+0000 7f45b3200640 1 -- 172.21.15.12:0/1815666111 wait complete. 2024-09-11T00:04:28.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:04:27 smithi165 ceph-mon[35356]: pgmap v315: 129 pgs: 129 active+clean; 579 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T00:04:28.304 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:04:27 smithi012 ceph-mon[30333]: pgmap v315: 129 pgs: 129 active+clean; 579 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T00:04:28.330 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:04:27 smithi178 ceph-mon[35919]: pgmap v315: 129 pgs: 129 active+clean; 579 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T00:04:28.346 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-09-11T00:04:28.346 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-09-11T00:04:28.346 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid ff3ad74a-6fcf-11ef-bcea-c7b262605968 -- ceph fs ls 2024-09-11T00:04:28.725 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:04:29.405 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:29.403+0000 7f12cf400640 1 -- 172.21.15.12:0/3516100616 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f12c8113b20 msgr2=0x7f12c8115f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:04:29.405 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:29.403+0000 7f12cf400640 1 --2- 172.21.15.12:0/3516100616 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f12c8113b20 0x7f12c8115f10 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f12c0009ca0 tx=0x7f12c002a8c0 comp rx=0 tx=0).stop 2024-09-11T00:04:29.405 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:29.403+0000 7f12cf400640 1 -- 172.21.15.12:0/3516100616 shutdown_connections 2024-09-11T00:04:29.405 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:29.403+0000 7f12cf400640 1 --2- 172.21.15.12:0/3516100616 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f12c8113b20 0x7f12c8115f10 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:04:29.405 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:29.403+0000 7f12cf400640 1 --2- 172.21.15.12:0/3516100616 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f12c8075e60 0x7f12c81135e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:04:29.406 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:29.403+0000 7f12cf400640 1 --2- 172.21.15.12:0/3516100616 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f12c8075540 0x7f12c8075920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:04:29.406 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:29.403+0000 7f12cf400640 1 -- 172.21.15.12:0/3516100616 >> 172.21.15.12:0/3516100616 conn(0x7f12c80fe830 msgr2=0x7f12c8100c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:04:29.406 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:29.403+0000 7f12cf400640 1 -- 172.21.15.12:0/3516100616 shutdown_connections 2024-09-11T00:04:29.406 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:29.403+0000 7f12cf400640 1 -- 172.21.15.12:0/3516100616 wait complete. 2024-09-11T00:04:29.407 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:29.404+0000 7f12cf400640 1 Processor -- start 2024-09-11T00:04:29.407 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:29.404+0000 7f12cf400640 1 -- start start 2024-09-11T00:04:29.407 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:29.404+0000 7f12cf400640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f12c8075540 0x7f12c819f910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:04:29.407 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:29.404+0000 7f12cf400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f12c8075e60 0x7f12c819fe50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:04:29.407 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:29.405+0000 7f12cf400640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f12c8113b20 0x7f12c81a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:04:29.407 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:29.405+0000 7f12cf400640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f12c8119380 con 0x7f12c8113b20 2024-09-11T00:04:29.407 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:29.405+0000 7f12cf400640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f12c8119200 con 0x7f12c8075e60 2024-09-11T00:04:29.407 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:29.405+0000 7f12cf400640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f12c8119500 con 0x7f12c8075540 2024-09-11T00:04:29.407 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:29.405+0000 7f12cd400640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f12c8113b20 0x7f12c81a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:04:29.407 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:29.405+0000 7f12cca00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f12c8075540 0x7f12c819f910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:04:29.408 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:29.405+0000 7f12cd400640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f12c8113b20 0x7f12c81a51f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:34296/0 (socket says 172.21.15.12:34296) 2024-09-11T00:04:29.408 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:29.405+0000 7f12cca00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f12c8075540 0x7f12c819f910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.12:53180/0 (socket says 172.21.15.12:53180) 2024-09-11T00:04:29.408 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:29.405+0000 7f12cd400640 1 -- 172.21.15.12:0/3560609243 learned_addr learned my addr 172.21.15.12:0/3560609243 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:04:29.408 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:29.405+0000 7f12bfe00640 1 --2- 172.21.15.12:0/3560609243 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f12c8075e60 0x7f12c819fe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:04:29.409 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:29.405+0000 7f12cd400640 1 -- 172.21.15.12:0/3560609243 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f12c8075540 msgr2=0x7f12c819f910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:04:29.409 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:29.405+0000 7f12cd400640 1 --2- 172.21.15.12:0/3560609243 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f12c8075540 0x7f12c819f910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:04:29.409 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:29.405+0000 7f12cd400640 1 -- 172.21.15.12:0/3560609243 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f12c8075e60 msgr2=0x7f12c819fe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:04:29.409 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:29.405+0000 7f12cd400640 1 --2- 172.21.15.12:0/3560609243 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f12c8075e60 0x7f12c819fe50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:04:29.409 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:29.405+0000 7f12cd400640 1 -- 172.21.15.12:0/3560609243 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12c81a59c0 con 0x7f12c8113b20 2024-09-11T00:04:29.409 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:29.406+0000 7f12cd400640 1 --2- 172.21.15.12:0/3560609243 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f12c8113b20 0x7f12c81a51f0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f12c0009430 tx=0x7f12c00316f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:04:29.409 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:29.406+0000 7f12c6000640 1 -- 172.21.15.12:0/3560609243 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12c0003b70 con 0x7f12c8113b20 2024-09-11T00:04:29.409 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:29.406+0000 7f12c6000640 1 -- 172.21.15.12:0/3560609243 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f12c0003d10 con 0x7f12c8113b20 2024-09-11T00:04:29.410 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:29.406+0000 7f12cf400640 1 -- 172.21.15.12:0/3560609243 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f12c81a5c50 con 0x7f12c8113b20 2024-09-11T00:04:29.410 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:29.406+0000 7f12c6000640 1 -- 172.21.15.12:0/3560609243 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12c002fb20 con 0x7f12c8113b20 2024-09-11T00:04:29.410 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:29.406+0000 7f12cf400640 1 -- 172.21.15.12:0/3560609243 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12c81ac370 con 0x7f12c8113b20 2024-09-11T00:04:29.410 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:29.407+0000 7f12cf400640 1 -- 172.21.15.12:0/3560609243 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f12c8075920 con 0x7f12c8113b20 2024-09-11T00:04:29.411 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:29.409+0000 7f12c6000640 1 -- 172.21.15.12:0/3560609243 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 16) ==== 153250+0+0 (secure 0 0 0) 0x7f12c0004580 con 0x7f12c8113b20 2024-09-11T00:04:29.412 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:29.411+0000 7f12c6000640 1 --2- 172.21.15.12:0/3560609243 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f12a80b52e0 0x7f12a80b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:04:29.413 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:29.411+0000 7f12c6000640 1 -- 172.21.15.12:0/3560609243 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8802+0+0 (secure 0 0 0) 0x7f12c0107090 con 0x7f12c8113b20 2024-09-11T00:04:29.413 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:29.411+0000 7f12cca00640 1 --2- 172.21.15.12:0/3560609243 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f12a80b52e0 0x7f12a80b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:04:29.413 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:29.412+0000 7f12cca00640 1 --2- 172.21.15.12:0/3560609243 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f12a80b52e0 0x7f12a80b77a0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f12b8009bd0 tx=0x7f12b8005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:04:29.416 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:29.413+0000 7f12c6000640 1 -- 172.21.15.12:0/3560609243 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f12c00cdd10 con 0x7f12c8113b20 2024-09-11T00:04:29.656 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:29.654+0000 7f12cf400640 1 -- 172.21.15.12:0/3560609243 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f12c81ac660 con 0x7f12c8113b20 2024-09-11T00:04:29.657 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:29.655+0000 7f12c6000640 1 -- 172.21.15.12:0/3560609243 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v17) ==== 53+0+88 (secure 0 0 0) 0x7f12c00cd610 con 0x7f12c8113b20 2024-09-11T00:04:29.657 INFO:teuthology.orchestra.run.smithi012.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-09-11T00:04:29.659 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:29.657+0000 7f12cf400640 1 -- 172.21.15.12:0/3560609243 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f12a80b52e0 msgr2=0x7f12a80b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:04:29.659 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:29.657+0000 7f12cf400640 1 --2- 172.21.15.12:0/3560609243 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f12a80b52e0 0x7f12a80b77a0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f12b8009bd0 tx=0x7f12b8005f90 comp rx=0 tx=0).stop 2024-09-11T00:04:29.659 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:29.657+0000 7f12cf400640 1 -- 172.21.15.12:0/3560609243 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f12c8113b20 msgr2=0x7f12c81a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:04:29.659 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:29.657+0000 7f12cf400640 1 --2- 172.21.15.12:0/3560609243 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f12c8113b20 0x7f12c81a51f0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f12c0009430 tx=0x7f12c00316f0 comp rx=0 tx=0).stop 2024-09-11T00:04:29.659 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:29.657+0000 7f12cf400640 1 -- 172.21.15.12:0/3560609243 shutdown_connections 2024-09-11T00:04:29.660 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:29.657+0000 7f12cf400640 1 --2- 172.21.15.12:0/3560609243 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f12a80b52e0 0x7f12a80b77a0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:04:29.660 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:29.657+0000 7f12cf400640 1 --2- 172.21.15.12:0/3560609243 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f12c8113b20 0x7f12c81a51f0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:04:29.660 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:29.657+0000 7f12cf400640 1 --2- 172.21.15.12:0/3560609243 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f12c8075e60 0x7f12c819fe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:04:29.660 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:29.657+0000 7f12cf400640 1 --2- 172.21.15.12:0/3560609243 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f12c8075540 0x7f12c819f910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:04:29.660 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:29.657+0000 7f12cf400640 1 -- 172.21.15.12:0/3560609243 >> 172.21.15.12:0/3560609243 conn(0x7f12c80fe830 msgr2=0x7f12c80ff430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:04:29.660 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:29.657+0000 7f12cf400640 1 -- 172.21.15.12:0/3560609243 shutdown_connections 2024-09-11T00:04:29.660 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:29.657+0000 7f12cf400640 1 -- 172.21.15.12:0/3560609243 wait complete. 2024-09-11T00:04:30.045 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:04:29 smithi012 ceph-mon[30333]: pgmap v316: 129 pgs: 129 active+clean; 579 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T00:04:30.045 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:04:29 smithi012 ceph-mon[30333]: from='client.? 172.21.15.12:0/3560609243' entity='client.admin' cmd={"prefix": "fs ls"} : dispatch 2024-09-11T00:04:30.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:04:29 smithi165 ceph-mon[35356]: pgmap v316: 129 pgs: 129 active+clean; 579 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T00:04:30.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:04:29 smithi165 ceph-mon[35356]: from='client.? 172.21.15.12:0/3560609243' entity='client.admin' cmd={"prefix": "fs ls"} : dispatch 2024-09-11T00:04:30.330 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:04:29 smithi178 ceph-mon[35919]: pgmap v316: 129 pgs: 129 active+clean; 579 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T00:04:30.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:04:29 smithi178 ceph-mon[35919]: from='client.? 172.21.15.12:0/3560609243' entity='client.admin' cmd={"prefix": "fs ls"} : dispatch 2024-09-11T00:04:30.541 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-11T00:04:30.541 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-11T00:04:30.541 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi012.front.sepia.ceph.com 2024-09-11T00:04:30.541 INFO:tasks.cephfs.mount:self.client.name = client.admin 2024-09-11T00:04:30.541 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-09-11T00:04:30.541 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-11T00:04:30.541 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-11T00:04:30.541 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-11T00:04:30.542 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.admin' 2024-09-11T00:04:30.542 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T00:04:30.542 DEBUG:teuthology.orchestra.run.smithi012:> ip addr 2024-09-11T00:04:30.578 INFO:teuthology.orchestra.run.smithi012.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-11T00:04:30.578 INFO:teuthology.orchestra.run.smithi012.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-11T00:04:30.578 INFO:teuthology.orchestra.run.smithi012.stdout: inet 127.0.0.1/8 scope host lo 2024-09-11T00:04:30.578 INFO:teuthology.orchestra.run.smithi012.stdout: valid_lft forever preferred_lft forever 2024-09-11T00:04:30.578 INFO:teuthology.orchestra.run.smithi012.stdout: inet6 ::1/128 scope host 2024-09-11T00:04:30.578 INFO:teuthology.orchestra.run.smithi012.stdout: valid_lft forever preferred_lft forever 2024-09-11T00:04:30.578 INFO:teuthology.orchestra.run.smithi012.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-11T00:04:30.579 INFO:teuthology.orchestra.run.smithi012.stdout: link/ether 0c:c4:7a:6b:fd:a4 brd ff:ff:ff:ff:ff:ff 2024-09-11T00:04:30.579 INFO:teuthology.orchestra.run.smithi012.stdout: altname enp7s0f0 2024-09-11T00:04:30.579 INFO:teuthology.orchestra.run.smithi012.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-11T00:04:30.579 INFO:teuthology.orchestra.run.smithi012.stdout: link/ether 0c:c4:7a:6b:fd:a5 brd ff:ff:ff:ff:ff:ff 2024-09-11T00:04:30.579 INFO:teuthology.orchestra.run.smithi012.stdout: altname enp7s0f1 2024-09-11T00:04:30.579 INFO:teuthology.orchestra.run.smithi012.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-11T00:04:30.579 INFO:teuthology.orchestra.run.smithi012.stdout: link/ether ac:1f:6b:f8:25:ea brd ff:ff:ff:ff:ff:ff 2024-09-11T00:04:30.579 INFO:teuthology.orchestra.run.smithi012.stdout: altname enp1s0f0 2024-09-11T00:04:30.579 INFO:teuthology.orchestra.run.smithi012.stdout: inet 172.21.15.12/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-09-11T00:04:30.579 INFO:teuthology.orchestra.run.smithi012.stdout: valid_lft 42180sec preferred_lft 42180sec 2024-09-11T00:04:30.579 INFO:teuthology.orchestra.run.smithi012.stdout: inet6 fe80::ae1f:6bff:fef8:25ea/64 scope link 2024-09-11T00:04:30.579 INFO:teuthology.orchestra.run.smithi012.stdout: valid_lft forever preferred_lft forever 2024-09-11T00:04:30.579 INFO:teuthology.orchestra.run.smithi012.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-11T00:04:30.580 INFO:teuthology.orchestra.run.smithi012.stdout: link/ether ac:1f:6b:f8:25:eb brd ff:ff:ff:ff:ff:ff 2024-09-11T00:04:30.580 INFO:teuthology.orchestra.run.smithi012.stdout: altname enp1s0f1 2024-09-11T00:04:30.581 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-09-11T00:04:30.581 DEBUG:teuthology.orchestra.run.smithi012:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-11T00:04:30.581 DEBUG:teuthology.orchestra.run.smithi012:> set -e 2024-09-11T00:04:30.581 DEBUG:teuthology.orchestra.run.smithi012:> sudo ip link add name ceph-brx type bridge 2024-09-11T00:04:30.582 DEBUG:teuthology.orchestra.run.smithi012:> sudo ip addr flush dev ceph-brx 2024-09-11T00:04:30.582 DEBUG:teuthology.orchestra.run.smithi012:> sudo ip link set ceph-brx up 2024-09-11T00:04:30.582 DEBUG:teuthology.orchestra.run.smithi012:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-09-11T00:04:30.582 DEBUG:teuthology.orchestra.run.smithi012:> ') 2024-09-11T00:04:30.699 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:30 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-11T00:04:31.015 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:31 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-11T00:04:31.028 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T00:04:31.028 DEBUG:teuthology.orchestra.run.smithi012:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-11T00:04:31.084 INFO:teuthology.orchestra.run.smithi012.stdout:1 2024-09-11T00:04:31.087 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T00:04:31.087 DEBUG:teuthology.orchestra.run.smithi012:> route 2024-09-11T00:04:31.140 INFO:teuthology.orchestra.run.smithi012.stdout:Kernel IP routing table 2024-09-11T00:04:31.140 INFO:teuthology.orchestra.run.smithi012.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-11T00:04:31.140 INFO:teuthology.orchestra.run.smithi012.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-09-11T00:04:31.140 INFO:teuthology.orchestra.run.smithi012.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-09-11T00:04:31.140 INFO:teuthology.orchestra.run.smithi012.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-11T00:04:31.141 DEBUG:teuthology.orchestra.run.smithi012:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-11T00:04:31.141 DEBUG:teuthology.orchestra.run.smithi012:> set -e 2024-09-11T00:04:31.141 DEBUG:teuthology.orchestra.run.smithi012:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-09-11T00:04:31.141 DEBUG:teuthology.orchestra.run.smithi012:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-09-11T00:04:31.141 DEBUG:teuthology.orchestra.run.smithi012:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-09-11T00:04:31.141 DEBUG:teuthology.orchestra.run.smithi012:> ') 2024-09-11T00:04:31.220 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:31 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-11T00:04:31.754 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:31 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-11T00:04:31.762 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T00:04:31.763 DEBUG:teuthology.orchestra.run.smithi012:> ip netns list 2024-09-11T00:04:31.799 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T00:04:31.799 DEBUG:teuthology.orchestra.run.smithi012:> ip netns list-id 2024-09-11T00:04:31.875 DEBUG:teuthology.orchestra.run.smithi012:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-11T00:04:31.875 DEBUG:teuthology.orchestra.run.smithi012:> set -e 2024-09-11T00:04:31.875 DEBUG:teuthology.orchestra.run.smithi012:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-09-11T00:04:31.875 DEBUG:teuthology.orchestra.run.smithi012:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.admin 0 2024-09-11T00:04:31.875 DEBUG:teuthology.orchestra.run.smithi012:> ') 2024-09-11T00:04:31.993 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:31 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-11T00:04:32.010 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:04:31 smithi012 ceph-mon[30333]: pgmap v317: 129 pgs: 129 active+clean; 579 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T00:04:32.047 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:32 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-11T00:04:32.055 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.admin' with 192.168.0.1/16 2024-09-11T00:04:32.056 DEBUG:teuthology.orchestra.run.smithi012:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-11T00:04:32.056 DEBUG:teuthology.orchestra.run.smithi012:> set -e 2024-09-11T00:04:32.056 DEBUG:teuthology.orchestra.run.smithi012:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.admin type veth peer name brx.0 2024-09-11T00:04:32.056 DEBUG:teuthology.orchestra.run.smithi012:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-09-11T00:04:32.056 DEBUG:teuthology.orchestra.run.smithi012:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set veth0 up 2024-09-11T00:04:32.056 DEBUG:teuthology.orchestra.run.smithi012:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set lo up 2024-09-11T00:04:32.056 DEBUG:teuthology.orchestra.run.smithi012:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip route add default via 192.168.255.254 2024-09-11T00:04:32.056 DEBUG:teuthology.orchestra.run.smithi012:> ') 2024-09-11T00:04:32.134 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:32 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-11T00:04:32.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:04:31 smithi165 ceph-mon[35356]: pgmap v317: 129 pgs: 129 active+clean; 579 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T00:04:32.299 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:32 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-11T00:04:32.307 DEBUG:teuthology.orchestra.run.smithi012:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-11T00:04:32.307 DEBUG:teuthology.orchestra.run.smithi012:> set -e 2024-09-11T00:04:32.308 DEBUG:teuthology.orchestra.run.smithi012:> sudo ip link set brx.0 up 2024-09-11T00:04:32.308 DEBUG:teuthology.orchestra.run.smithi012:> sudo ip link set dev brx.0 master ceph-brx 2024-09-11T00:04:32.308 DEBUG:teuthology.orchestra.run.smithi012:> ') 2024-09-11T00:04:32.330 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:04:31 smithi178 ceph-mon[35919]: pgmap v317: 129 pgs: 129 active+clean; 579 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T00:04:32.387 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:32 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-11T00:04:32.468 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:32 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-11T00:04:32.476 INFO:tasks.cephfs.fuse_mount:Client client.admin config is {} 2024-09-11T00:04:32.476 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-11T00:04:32.476 DEBUG:teuthology.orchestra.run.smithi012:> mkdir -p -v /home/ubuntu/cephtest/mnt.admin 2024-09-11T00:04:32.511 INFO:teuthology.orchestra.run.smithi012.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.admin' 2024-09-11T00:04:32.512 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-11T00:04:32.512 DEBUG:teuthology.orchestra.run.smithi012:> chmod 0000 /home/ubuntu/cephtest/mnt.admin 2024-09-11T00:04:32.586 DEBUG:teuthology.orchestra.run.smithi012:> sudo modprobe fuse 2024-09-11T00:04:32.685 DEBUG:teuthology.orchestra.run.smithi012:> cat /proc/self/mounts | awk '{print $2}' 2024-09-11T00:04:32.721 INFO:teuthology.orchestra.run.smithi012.stdout:/proc 2024-09-11T00:04:32.721 INFO:teuthology.orchestra.run.smithi012.stdout:/sys 2024-09-11T00:04:32.721 INFO:teuthology.orchestra.run.smithi012.stdout:/dev 2024-09-11T00:04:32.721 INFO:teuthology.orchestra.run.smithi012.stdout:/sys/kernel/security 2024-09-11T00:04:32.721 INFO:teuthology.orchestra.run.smithi012.stdout:/dev/shm 2024-09-11T00:04:32.721 INFO:teuthology.orchestra.run.smithi012.stdout:/dev/pts 2024-09-11T00:04:32.721 INFO:teuthology.orchestra.run.smithi012.stdout:/run 2024-09-11T00:04:32.721 INFO:teuthology.orchestra.run.smithi012.stdout:/sys/fs/cgroup 2024-09-11T00:04:32.721 INFO:teuthology.orchestra.run.smithi012.stdout:/sys/fs/pstore 2024-09-11T00:04:32.722 INFO:teuthology.orchestra.run.smithi012.stdout:/sys/fs/bpf 2024-09-11T00:04:32.722 INFO:teuthology.orchestra.run.smithi012.stdout:/ 2024-09-11T00:04:32.722 INFO:teuthology.orchestra.run.smithi012.stdout:/sys/fs/selinux 2024-09-11T00:04:32.722 INFO:teuthology.orchestra.run.smithi012.stdout:/proc/sys/fs/binfmt_misc 2024-09-11T00:04:32.722 INFO:teuthology.orchestra.run.smithi012.stdout:/dev/hugepages 2024-09-11T00:04:32.722 INFO:teuthology.orchestra.run.smithi012.stdout:/dev/mqueue 2024-09-11T00:04:32.722 INFO:teuthology.orchestra.run.smithi012.stdout:/sys/kernel/debug 2024-09-11T00:04:32.722 INFO:teuthology.orchestra.run.smithi012.stdout:/sys/kernel/tracing 2024-09-11T00:04:32.722 INFO:teuthology.orchestra.run.smithi012.stdout:/sys/fs/fuse/connections 2024-09-11T00:04:32.723 INFO:teuthology.orchestra.run.smithi012.stdout:/sys/kernel/config 2024-09-11T00:04:32.723 INFO:teuthology.orchestra.run.smithi012.stdout:/run/credentials/systemd-sysctl.service 2024-09-11T00:04:32.723 INFO:teuthology.orchestra.run.smithi012.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-09-11T00:04:32.723 INFO:teuthology.orchestra.run.smithi012.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-09-11T00:04:32.723 INFO:teuthology.orchestra.run.smithi012.stdout:/var/lib/nfs/rpc_pipefs 2024-09-11T00:04:32.723 INFO:teuthology.orchestra.run.smithi012.stdout:/run/user/1000 2024-09-11T00:04:32.723 INFO:teuthology.orchestra.run.smithi012.stdout:/var/lib/ceph 2024-09-11T00:04:32.723 INFO:teuthology.orchestra.run.smithi012.stdout:/var/lib/containers/storage/overlay 2024-09-11T00:04:32.723 INFO:teuthology.orchestra.run.smithi012.stdout:/var/lib/containers/storage/overlay/0ae1cab31459129d0e9ca9e5b545aabfff9cca564e337e53730104e7684288db/merged 2024-09-11T00:04:32.723 INFO:teuthology.orchestra.run.smithi012.stdout:/var/lib/containers/storage/overlay/0cfe38d93503f03fe729a3dedbe4444afd71f5dd80fe6d15d355894c318ed60e/merged 2024-09-11T00:04:32.723 INFO:teuthology.orchestra.run.smithi012.stdout:/run/user/0 2024-09-11T00:04:32.723 INFO:teuthology.orchestra.run.smithi012.stdout:/proc/sys/fs/binfmt_misc 2024-09-11T00:04:32.723 INFO:teuthology.orchestra.run.smithi012.stdout:/var/lib/containers/storage/overlay/b3cbb8aabd51d08ba59af7cbf720c53c037ea39c9671a0bb4f455a894b8dbce5/merged 2024-09-11T00:04:32.724 INFO:teuthology.orchestra.run.smithi012.stdout:/var/lib/containers/storage/overlay/8ceb54a2543e78a98825bdf92388c8dd29b57ab845988078ddf4d2562289a2fe/merged 2024-09-11T00:04:32.724 INFO:teuthology.orchestra.run.smithi012.stdout:/var/lib/containers/storage/overlay/6bf098d282da9d0a91f0a823db4ae6655f4ebe16f76172851d1cb82fc3ea35db/merged 2024-09-11T00:04:32.724 INFO:teuthology.orchestra.run.smithi012.stdout:/var/lib/containers/storage/overlay/d09882e1c3a08d867e0f320a3c862f4e45f4f78bd7b989277ce621187d504b78/merged 2024-09-11T00:04:32.724 INFO:teuthology.orchestra.run.smithi012.stdout:/var/lib/containers/storage/overlay/22c29987480c9d9be358f08403ebef23d2659245730285d43b993a932d4c2311/merged 2024-09-11T00:04:32.724 INFO:teuthology.orchestra.run.smithi012.stdout:/var/lib/containers/storage/overlay/3808fe41a9d3266aa88dc684842143a3fbcfe53b5ed595478da663bcbf063567/merged 2024-09-11T00:04:32.724 INFO:teuthology.orchestra.run.smithi012.stdout:/var/lib/containers/storage/overlay/e679ebcdb2910eb019306972ceb8ecfee0e66a67ebc231ea91bfdb9a65a144bc/merged 2024-09-11T00:04:32.725 INFO:teuthology.orchestra.run.smithi012.stdout:/var/lib/containers/storage/overlay/27faeff14467aecb446f91446033c80feed6c9bd2958e6ef85fb23ea0ac434d7/merged 2024-09-11T00:04:32.725 INFO:teuthology.orchestra.run.smithi012.stdout:/run/netns 2024-09-11T00:04:32.725 INFO:teuthology.orchestra.run.smithi012.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-09-11T00:04:32.725 INFO:teuthology.orchestra.run.smithi012.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-09-11T00:04:32.726 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T00:04:32.726 DEBUG:teuthology.orchestra.run.smithi012:> ls /sys/fs/fuse/connections 2024-09-11T00:04:32.802 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-09-11T00:04:32.802 DEBUG:teuthology.orchestra.run.smithi012:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.admin --id admin --client_fs=cephfs) 2024-09-11T00:04:32.846 DEBUG:teuthology.orchestra.run.smithi012:> sudo modprobe fuse 2024-09-11T00:04:32.904 DEBUG:teuthology.orchestra.run.smithi012:> cat /proc/self/mounts | awk '{print $2}' 2024-09-11T00:04:32.941 INFO:teuthology.orchestra.run.smithi012.stdout:/proc 2024-09-11T00:04:32.941 INFO:teuthology.orchestra.run.smithi012.stdout:/sys 2024-09-11T00:04:32.941 INFO:teuthology.orchestra.run.smithi012.stdout:/dev 2024-09-11T00:04:32.941 INFO:teuthology.orchestra.run.smithi012.stdout:/sys/kernel/security 2024-09-11T00:04:32.942 INFO:teuthology.orchestra.run.smithi012.stdout:/dev/shm 2024-09-11T00:04:32.942 INFO:teuthology.orchestra.run.smithi012.stdout:/dev/pts 2024-09-11T00:04:32.942 INFO:teuthology.orchestra.run.smithi012.stdout:/run 2024-09-11T00:04:32.942 INFO:teuthology.orchestra.run.smithi012.stdout:/sys/fs/cgroup 2024-09-11T00:04:32.942 INFO:teuthology.orchestra.run.smithi012.stdout:/sys/fs/pstore 2024-09-11T00:04:32.942 INFO:teuthology.orchestra.run.smithi012.stdout:/sys/fs/bpf 2024-09-11T00:04:32.942 INFO:teuthology.orchestra.run.smithi012.stdout:/ 2024-09-11T00:04:32.942 INFO:teuthology.orchestra.run.smithi012.stdout:/sys/fs/selinux 2024-09-11T00:04:32.942 INFO:teuthology.orchestra.run.smithi012.stdout:/proc/sys/fs/binfmt_misc 2024-09-11T00:04:32.942 INFO:teuthology.orchestra.run.smithi012.stdout:/dev/hugepages 2024-09-11T00:04:32.942 INFO:teuthology.orchestra.run.smithi012.stdout:/dev/mqueue 2024-09-11T00:04:32.942 INFO:teuthology.orchestra.run.smithi012.stdout:/sys/kernel/debug 2024-09-11T00:04:32.943 INFO:teuthology.orchestra.run.smithi012.stdout:/sys/kernel/tracing 2024-09-11T00:04:32.943 INFO:teuthology.orchestra.run.smithi012.stdout:/sys/fs/fuse/connections 2024-09-11T00:04:32.943 INFO:teuthology.orchestra.run.smithi012.stdout:/sys/kernel/config 2024-09-11T00:04:32.943 INFO:teuthology.orchestra.run.smithi012.stdout:/run/credentials/systemd-sysctl.service 2024-09-11T00:04:32.943 INFO:teuthology.orchestra.run.smithi012.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-09-11T00:04:32.943 INFO:teuthology.orchestra.run.smithi012.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-09-11T00:04:32.943 INFO:teuthology.orchestra.run.smithi012.stdout:/var/lib/nfs/rpc_pipefs 2024-09-11T00:04:32.943 INFO:teuthology.orchestra.run.smithi012.stdout:/run/user/1000 2024-09-11T00:04:32.943 INFO:teuthology.orchestra.run.smithi012.stdout:/var/lib/ceph 2024-09-11T00:04:32.943 INFO:teuthology.orchestra.run.smithi012.stdout:/var/lib/containers/storage/overlay 2024-09-11T00:04:32.943 INFO:teuthology.orchestra.run.smithi012.stdout:/var/lib/containers/storage/overlay/0ae1cab31459129d0e9ca9e5b545aabfff9cca564e337e53730104e7684288db/merged 2024-09-11T00:04:32.943 INFO:teuthology.orchestra.run.smithi012.stdout:/var/lib/containers/storage/overlay/0cfe38d93503f03fe729a3dedbe4444afd71f5dd80fe6d15d355894c318ed60e/merged 2024-09-11T00:04:32.943 INFO:teuthology.orchestra.run.smithi012.stdout:/run/user/0 2024-09-11T00:04:32.944 INFO:teuthology.orchestra.run.smithi012.stdout:/proc/sys/fs/binfmt_misc 2024-09-11T00:04:32.944 INFO:teuthology.orchestra.run.smithi012.stdout:/var/lib/containers/storage/overlay/b3cbb8aabd51d08ba59af7cbf720c53c037ea39c9671a0bb4f455a894b8dbce5/merged 2024-09-11T00:04:32.944 INFO:teuthology.orchestra.run.smithi012.stdout:/var/lib/containers/storage/overlay/8ceb54a2543e78a98825bdf92388c8dd29b57ab845988078ddf4d2562289a2fe/merged 2024-09-11T00:04:32.944 INFO:teuthology.orchestra.run.smithi012.stdout:/var/lib/containers/storage/overlay/6bf098d282da9d0a91f0a823db4ae6655f4ebe16f76172851d1cb82fc3ea35db/merged 2024-09-11T00:04:32.944 INFO:teuthology.orchestra.run.smithi012.stdout:/var/lib/containers/storage/overlay/d09882e1c3a08d867e0f320a3c862f4e45f4f78bd7b989277ce621187d504b78/merged 2024-09-11T00:04:32.944 INFO:teuthology.orchestra.run.smithi012.stdout:/var/lib/containers/storage/overlay/22c29987480c9d9be358f08403ebef23d2659245730285d43b993a932d4c2311/merged 2024-09-11T00:04:32.944 INFO:teuthology.orchestra.run.smithi012.stdout:/var/lib/containers/storage/overlay/3808fe41a9d3266aa88dc684842143a3fbcfe53b5ed595478da663bcbf063567/merged 2024-09-11T00:04:32.944 INFO:teuthology.orchestra.run.smithi012.stdout:/var/lib/containers/storage/overlay/e679ebcdb2910eb019306972ceb8ecfee0e66a67ebc231ea91bfdb9a65a144bc/merged 2024-09-11T00:04:32.944 INFO:teuthology.orchestra.run.smithi012.stdout:/var/lib/containers/storage/overlay/27faeff14467aecb446f91446033c80feed6c9bd2958e6ef85fb23ea0ac434d7/merged 2024-09-11T00:04:32.944 INFO:teuthology.orchestra.run.smithi012.stdout:/run/netns 2024-09-11T00:04:32.944 INFO:teuthology.orchestra.run.smithi012.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-09-11T00:04:32.944 INFO:teuthology.orchestra.run.smithi012.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-09-11T00:04:32.945 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T00:04:32.945 DEBUG:teuthology.orchestra.run.smithi012:> ls /sys/fs/fuse/connections 2024-09-11T00:04:33.028 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi012.stderr:2024-09-11T00:04:33.026+0000 7fcd414d1480 -1 init, newargv = 0x5584c3e19390 newargc=15 2024-09-11T00:04:33.029 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi012.stderr:ceph-fuse[76586]: starting ceph client 2024-09-11T00:04:33.038 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi012.stderr:ceph-fuse[76586]: starting fuse 2024-09-11T00:04:34.122 DEBUG:teuthology.orchestra.run.smithi012:> sudo modprobe fuse 2024-09-11T00:04:34.176 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:04:33 smithi012 ceph-mon[30333]: pgmap v318: 129 pgs: 129 active+clean; 579 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T00:04:34.187 DEBUG:teuthology.orchestra.run.smithi012:> cat /proc/self/mounts | awk '{print $2}' 2024-09-11T00:04:34.225 INFO:teuthology.orchestra.run.smithi012.stdout:/proc 2024-09-11T00:04:34.226 INFO:teuthology.orchestra.run.smithi012.stdout:/sys 2024-09-11T00:04:34.226 INFO:teuthology.orchestra.run.smithi012.stdout:/dev 2024-09-11T00:04:34.226 INFO:teuthology.orchestra.run.smithi012.stdout:/sys/kernel/security 2024-09-11T00:04:34.226 INFO:teuthology.orchestra.run.smithi012.stdout:/dev/shm 2024-09-11T00:04:34.226 INFO:teuthology.orchestra.run.smithi012.stdout:/dev/pts 2024-09-11T00:04:34.226 INFO:teuthology.orchestra.run.smithi012.stdout:/run 2024-09-11T00:04:34.226 INFO:teuthology.orchestra.run.smithi012.stdout:/sys/fs/cgroup 2024-09-11T00:04:34.226 INFO:teuthology.orchestra.run.smithi012.stdout:/sys/fs/pstore 2024-09-11T00:04:34.226 INFO:teuthology.orchestra.run.smithi012.stdout:/sys/fs/bpf 2024-09-11T00:04:34.226 INFO:teuthology.orchestra.run.smithi012.stdout:/ 2024-09-11T00:04:34.226 INFO:teuthology.orchestra.run.smithi012.stdout:/sys/fs/selinux 2024-09-11T00:04:34.226 INFO:teuthology.orchestra.run.smithi012.stdout:/proc/sys/fs/binfmt_misc 2024-09-11T00:04:34.227 INFO:teuthology.orchestra.run.smithi012.stdout:/dev/hugepages 2024-09-11T00:04:34.227 INFO:teuthology.orchestra.run.smithi012.stdout:/dev/mqueue 2024-09-11T00:04:34.227 INFO:teuthology.orchestra.run.smithi012.stdout:/sys/kernel/debug 2024-09-11T00:04:34.227 INFO:teuthology.orchestra.run.smithi012.stdout:/sys/kernel/tracing 2024-09-11T00:04:34.227 INFO:teuthology.orchestra.run.smithi012.stdout:/sys/fs/fuse/connections 2024-09-11T00:04:34.227 INFO:teuthology.orchestra.run.smithi012.stdout:/sys/kernel/config 2024-09-11T00:04:34.227 INFO:teuthology.orchestra.run.smithi012.stdout:/run/credentials/systemd-sysctl.service 2024-09-11T00:04:34.227 INFO:teuthology.orchestra.run.smithi012.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-09-11T00:04:34.227 INFO:teuthology.orchestra.run.smithi012.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-09-11T00:04:34.227 INFO:teuthology.orchestra.run.smithi012.stdout:/var/lib/nfs/rpc_pipefs 2024-09-11T00:04:34.227 INFO:teuthology.orchestra.run.smithi012.stdout:/run/user/1000 2024-09-11T00:04:34.227 INFO:teuthology.orchestra.run.smithi012.stdout:/var/lib/ceph 2024-09-11T00:04:34.228 INFO:teuthology.orchestra.run.smithi012.stdout:/var/lib/containers/storage/overlay 2024-09-11T00:04:34.228 INFO:teuthology.orchestra.run.smithi012.stdout:/var/lib/containers/storage/overlay/0ae1cab31459129d0e9ca9e5b545aabfff9cca564e337e53730104e7684288db/merged 2024-09-11T00:04:34.228 INFO:teuthology.orchestra.run.smithi012.stdout:/var/lib/containers/storage/overlay/0cfe38d93503f03fe729a3dedbe4444afd71f5dd80fe6d15d355894c318ed60e/merged 2024-09-11T00:04:34.228 INFO:teuthology.orchestra.run.smithi012.stdout:/run/user/0 2024-09-11T00:04:34.228 INFO:teuthology.orchestra.run.smithi012.stdout:/proc/sys/fs/binfmt_misc 2024-09-11T00:04:34.228 INFO:teuthology.orchestra.run.smithi012.stdout:/var/lib/containers/storage/overlay/b3cbb8aabd51d08ba59af7cbf720c53c037ea39c9671a0bb4f455a894b8dbce5/merged 2024-09-11T00:04:34.228 INFO:teuthology.orchestra.run.smithi012.stdout:/var/lib/containers/storage/overlay/8ceb54a2543e78a98825bdf92388c8dd29b57ab845988078ddf4d2562289a2fe/merged 2024-09-11T00:04:34.228 INFO:teuthology.orchestra.run.smithi012.stdout:/var/lib/containers/storage/overlay/6bf098d282da9d0a91f0a823db4ae6655f4ebe16f76172851d1cb82fc3ea35db/merged 2024-09-11T00:04:34.228 INFO:teuthology.orchestra.run.smithi012.stdout:/var/lib/containers/storage/overlay/d09882e1c3a08d867e0f320a3c862f4e45f4f78bd7b989277ce621187d504b78/merged 2024-09-11T00:04:34.228 INFO:teuthology.orchestra.run.smithi012.stdout:/var/lib/containers/storage/overlay/22c29987480c9d9be358f08403ebef23d2659245730285d43b993a932d4c2311/merged 2024-09-11T00:04:34.228 INFO:teuthology.orchestra.run.smithi012.stdout:/var/lib/containers/storage/overlay/3808fe41a9d3266aa88dc684842143a3fbcfe53b5ed595478da663bcbf063567/merged 2024-09-11T00:04:34.229 INFO:teuthology.orchestra.run.smithi012.stdout:/var/lib/containers/storage/overlay/e679ebcdb2910eb019306972ceb8ecfee0e66a67ebc231ea91bfdb9a65a144bc/merged 2024-09-11T00:04:34.229 INFO:teuthology.orchestra.run.smithi012.stdout:/var/lib/containers/storage/overlay/27faeff14467aecb446f91446033c80feed6c9bd2958e6ef85fb23ea0ac434d7/merged 2024-09-11T00:04:34.229 INFO:teuthology.orchestra.run.smithi012.stdout:/run/netns 2024-09-11T00:04:34.229 INFO:teuthology.orchestra.run.smithi012.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-09-11T00:04:34.229 INFO:teuthology.orchestra.run.smithi012.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-09-11T00:04:34.229 INFO:teuthology.orchestra.run.smithi012.stdout:/home/ubuntu/cephtest/mnt.admin 2024-09-11T00:04:34.230 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T00:04:34.230 DEBUG:teuthology.orchestra.run.smithi012:> ls /sys/fs/fuse/connections 2024-09-11T00:04:34.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:04:33 smithi165 ceph-mon[35356]: pgmap v318: 129 pgs: 129 active+clean; 579 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T00:04:34.307 INFO:teuthology.orchestra.run.smithi012.stdout:120 2024-09-11T00:04:34.307 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [120] 2024-09-11T00:04:34.307 DEBUG:teuthology.orchestra.run.smithi012:> sudo stdin-killer -- python3 -c ' 2024-09-11T00:04:34.308 DEBUG:teuthology.orchestra.run.smithi012:> import glob 2024-09-11T00:04:34.308 DEBUG:teuthology.orchestra.run.smithi012:> import re 2024-09-11T00:04:34.308 DEBUG:teuthology.orchestra.run.smithi012:> import os 2024-09-11T00:04:34.308 DEBUG:teuthology.orchestra.run.smithi012:> import subprocess 2024-09-11T00:04:34.308 DEBUG:teuthology.orchestra.run.smithi012:> 2024-09-11T00:04:34.308 DEBUG:teuthology.orchestra.run.smithi012:> def _find_admin_socket(client_name): 2024-09-11T00:04:34.308 DEBUG:teuthology.orchestra.run.smithi012:> asok_path = "/var/run/ceph/ceph-client.admin.*.asok" 2024-09-11T00:04:34.308 DEBUG:teuthology.orchestra.run.smithi012:> files = glob.glob(asok_path) 2024-09-11T00:04:34.308 DEBUG:teuthology.orchestra.run.smithi012:> mountpoint = "/home/ubuntu/cephtest/mnt.admin" 2024-09-11T00:04:34.308 DEBUG:teuthology.orchestra.run.smithi012:> 2024-09-11T00:04:34.309 DEBUG:teuthology.orchestra.run.smithi012:> # Given a non-glob path, it better be there 2024-09-11T00:04:34.309 DEBUG:teuthology.orchestra.run.smithi012:> if "*" not in asok_path: 2024-09-11T00:04:34.309 DEBUG:teuthology.orchestra.run.smithi012:> assert(len(files) == 1) 2024-09-11T00:04:34.309 DEBUG:teuthology.orchestra.run.smithi012:> return files[0] 2024-09-11T00:04:34.309 DEBUG:teuthology.orchestra.run.smithi012:> 2024-09-11T00:04:34.309 DEBUG:teuthology.orchestra.run.smithi012:> for f in files: 2024-09-11T00:04:34.309 DEBUG:teuthology.orchestra.run.smithi012:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-09-11T00:04:34.309 DEBUG:teuthology.orchestra.run.smithi012:> if os.path.exists("/proc/{0}".format(pid)): 2024-09-11T00:04:34.309 DEBUG:teuthology.orchestra.run.smithi012:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-09-11T00:04:34.309 DEBUG:teuthology.orchestra.run.smithi012:> contents = proc_f.read() 2024-09-11T00:04:34.309 DEBUG:teuthology.orchestra.run.smithi012:> if mountpoint in contents: 2024-09-11T00:04:34.309 DEBUG:teuthology.orchestra.run.smithi012:> return f 2024-09-11T00:04:34.309 DEBUG:teuthology.orchestra.run.smithi012:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-09-11T00:04:34.309 DEBUG:teuthology.orchestra.run.smithi012:> 2024-09-11T00:04:34.309 DEBUG:teuthology.orchestra.run.smithi012:> print(_find_admin_socket("client.admin")) 2024-09-11T00:04:34.310 DEBUG:teuthology.orchestra.run.smithi012:> ' 2024-09-11T00:04:34.330 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:04:33 smithi178 ceph-mon[35919]: pgmap v318: 129 pgs: 129 active+clean; 579 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T00:04:34.474 INFO:teuthology.orchestra.run.smithi012.stdout:/var/run/ceph/ceph-client.admin.76586.asok 2024-09-11T00:04:34.478 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:34 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-11T00:04:34.488 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.admin.76586.asok 2024-09-11T00:04:34.488 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T00:04:34.489 DEBUG:teuthology.orchestra.run.smithi012:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.admin.76586.asok status 2024-09-11T00:04:34.675 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-09-11T00:04:34.675 INFO:teuthology.orchestra.run.smithi012.stdout: "metadata": { 2024-09-11T00:04:34.675 INFO:teuthology.orchestra.run.smithi012.stdout: "ceph_sha1": "8f219c476cb4f26fc5719c661c0323719a45cb30", 2024-09-11T00:04:34.676 INFO:teuthology.orchestra.run.smithi012.stdout: "ceph_version": "ceph version 19.3.0-4854-g8f219c47 (8f219c476cb4f26fc5719c661c0323719a45cb30) squid (dev)", 2024-09-11T00:04:34.676 INFO:teuthology.orchestra.run.smithi012.stdout: "entity_id": "admin", 2024-09-11T00:04:34.676 INFO:teuthology.orchestra.run.smithi012.stdout: "hostname": "smithi012", 2024-09-11T00:04:34.676 INFO:teuthology.orchestra.run.smithi012.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.admin", 2024-09-11T00:04:34.676 INFO:teuthology.orchestra.run.smithi012.stdout: "pid": "76586", 2024-09-11T00:04:34.676 INFO:teuthology.orchestra.run.smithi012.stdout: "root": "/" 2024-09-11T00:04:34.676 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:04:34.676 INFO:teuthology.orchestra.run.smithi012.stdout: "dentry_count": 0, 2024-09-11T00:04:34.676 INFO:teuthology.orchestra.run.smithi012.stdout: "dentry_pinned_count": 0, 2024-09-11T00:04:34.676 INFO:teuthology.orchestra.run.smithi012.stdout: "id": 14751, 2024-09-11T00:04:34.676 INFO:teuthology.orchestra.run.smithi012.stdout: "inst": { 2024-09-11T00:04:34.676 INFO:teuthology.orchestra.run.smithi012.stdout: "name": { 2024-09-11T00:04:34.677 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "client", 2024-09-11T00:04:34.677 INFO:teuthology.orchestra.run.smithi012.stdout: "num": 14751 2024-09-11T00:04:34.677 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:04:34.677 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": { 2024-09-11T00:04:34.677 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v1", 2024-09-11T00:04:34.677 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "192.168.0.1:0", 2024-09-11T00:04:34.677 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 1044209762 2024-09-11T00:04:34.677 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:04:34.677 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:04:34.677 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": { 2024-09-11T00:04:34.677 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v1", 2024-09-11T00:04:34.677 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "192.168.0.1:0", 2024-09-11T00:04:34.678 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 1044209762 2024-09-11T00:04:34.678 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:04:34.678 INFO:teuthology.orchestra.run.smithi012.stdout: "inst_str": "client.14751 192.168.0.1:0/1044209762", 2024-09-11T00:04:34.678 INFO:teuthology.orchestra.run.smithi012.stdout: "addr_str": "192.168.0.1:0/1044209762", 2024-09-11T00:04:34.678 INFO:teuthology.orchestra.run.smithi012.stdout: "inode_count": 1, 2024-09-11T00:04:34.678 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_epoch": 17, 2024-09-11T00:04:34.678 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_epoch": 81, 2024-09-11T00:04:34.678 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_epoch_barrier": 0, 2024-09-11T00:04:34.678 INFO:teuthology.orchestra.run.smithi012.stdout: "blocklisted": false, 2024-09-11T00:04:34.678 INFO:teuthology.orchestra.run.smithi012.stdout: "fs_name": "cephfs" 2024-09-11T00:04:34.678 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-09-11T00:04:34.687 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T00:04:34.688 DEBUG:teuthology.orchestra.run.smithi012:> stat --file-system '--printf=%T 2024-09-11T00:04:34.688 DEBUG:teuthology.orchestra.run.smithi012:> ' -- /home/ubuntu/cephtest/mnt.admin 2024-09-11T00:04:34.723 INFO:teuthology.orchestra.run.smithi012.stdout:fuseblk 2024-09-11T00:04:34.724 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.admin 2024-09-11T00:04:34.724 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T00:04:34.724 DEBUG:teuthology.orchestra.run.smithi012:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.admin 2024-09-11T00:04:34.821 DEBUG:teuthology.orchestra.run.smithi012:> (cd /home/ubuntu/cephtest/mnt.admin && exec stdin-killer --timeout=900 -- bash -c 'setfattr -n ceph.dir.layout.pool -v cephfs_data_ec . && getfattr -n ceph.dir.layout .') 2024-09-11T00:04:34.904 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:34 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-11T00:04:34.926 INFO:teuthology.orchestra.run.smithi012.stdout:# file: . 2024-09-11T00:04:34.927 INFO:teuthology.orchestra.run.smithi012.stdout:ceph.dir.layout="stripe_unit=4194304 stripe_count=1 object_size=4194304 pool=cephfs_data_ec" 2024-09-11T00:04:34.927 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-11T00:04:34.927 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:34 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-11T00:04:34.935 DEBUG:teuthology.orchestra.run.smithi012:> set -ex 2024-09-11T00:04:34.935 DEBUG:teuthology.orchestra.run.smithi012:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-11T00:04:34.972 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid ff3ad74a-6fcf-11ef-bcea-c7b262605968 -- ceph osd blocklist ls 2024-09-11T00:04:35.391 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:04:35.979 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:35.976+0000 7fd283000640 1 -- 172.21.15.12:0/2155187532 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd27c0776d0 msgr2=0x7fd27c077ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:04:35.979 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:35.976+0000 7fd283000640 1 --2- 172.21.15.12:0/2155187532 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd27c0776d0 0x7fd27c077ab0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fd26c009860 tx=0x7fd26c02a340 comp rx=0 tx=0).stop 2024-09-11T00:04:35.979 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:35.976+0000 7fd283000640 1 -- 172.21.15.12:0/2155187532 shutdown_connections 2024-09-11T00:04:35.979 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:35.976+0000 7fd283000640 1 --2- 172.21.15.12:0/2155187532 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd27c113990 0x7fd27c115d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:04:35.979 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:35.976+0000 7fd283000640 1 --2- 172.21.15.12:0/2155187532 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd27c077ff0 0x7fd27c113450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:04:35.979 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:35.976+0000 7fd283000640 1 --2- 172.21.15.12:0/2155187532 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd27c0776d0 0x7fd27c077ab0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:04:35.979 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:35.976+0000 7fd283000640 1 -- 172.21.15.12:0/2155187532 >> 172.21.15.12:0/2155187532 conn(0x7fd27c100a50 msgr2=0x7fd27c102e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:04:35.979 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:35.977+0000 7fd283000640 1 -- 172.21.15.12:0/2155187532 shutdown_connections 2024-09-11T00:04:35.980 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:35.977+0000 7fd283000640 1 -- 172.21.15.12:0/2155187532 wait complete. 2024-09-11T00:04:35.981 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:35.977+0000 7fd283000640 1 Processor -- start 2024-09-11T00:04:35.981 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:35.978+0000 7fd283000640 1 -- start start 2024-09-11T00:04:35.981 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:35.978+0000 7fd283000640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd27c0776d0 0x7fd27c19f860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:04:35.981 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:35.978+0000 7fd283000640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd27c077ff0 0x7fd27c19fda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:04:35.981 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:35.978+0000 7fd283000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd27c113990 0x7fd27c1a5140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:04:35.981 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:35.978+0000 7fd283000640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fd27c119390 con 0x7fd27c0776d0 2024-09-11T00:04:35.981 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:35.978+0000 7fd283000640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fd27c119210 con 0x7fd27c113990 2024-09-11T00:04:35.981 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:35.978+0000 7fd283000640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fd27c119510 con 0x7fd27c077ff0 2024-09-11T00:04:35.981 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:35.978+0000 7fd27be00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd27c0776d0 0x7fd27c19f860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:04:35.981 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:35.978+0000 7fd27be00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd27c0776d0 0x7fd27c19f860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:34314/0 (socket says 172.21.15.12:34314) 2024-09-11T00:04:35.982 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:35.978+0000 7fd27be00640 1 -- 172.21.15.12:0/4142719245 learned_addr learned my addr 172.21.15.12:0/4142719245 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:04:35.982 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:35.978+0000 7fd281000640 1 --2- 172.21.15.12:0/4142719245 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd27c113990 0x7fd27c1a5140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:04:35.982 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:35.978+0000 7fd27b400640 1 --2- 172.21.15.12:0/4142719245 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd27c077ff0 0x7fd27c19fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:04:35.982 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:35.979+0000 7fd27be00640 1 -- 172.21.15.12:0/4142719245 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd27c077ff0 msgr2=0x7fd27c19fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:04:35.982 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:35.979+0000 7fd27be00640 1 --2- 172.21.15.12:0/4142719245 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd27c077ff0 0x7fd27c19fda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:04:35.982 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:35.979+0000 7fd27be00640 1 -- 172.21.15.12:0/4142719245 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd27c113990 msgr2=0x7fd27c1a5140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:04:35.982 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:35.979+0000 7fd27be00640 1 --2- 172.21.15.12:0/4142719245 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd27c113990 0x7fd27c1a5140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:04:35.982 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:35.979+0000 7fd27be00640 1 -- 172.21.15.12:0/4142719245 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd27c1a5910 con 0x7fd27c0776d0 2024-09-11T00:04:35.982 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:35.979+0000 7fd27be00640 1 --2- 172.21.15.12:0/4142719245 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd27c0776d0 0x7fd27c19f860 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fd26c035040 tx=0x7fd26c032d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:04:35.983 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:35.979+0000 7fd278c00640 1 -- 172.21.15.12:0/4142719245 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd26c037040 con 0x7fd27c0776d0 2024-09-11T00:04:35.983 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:35.979+0000 7fd278c00640 1 -- 172.21.15.12:0/4142719245 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd26c041070 con 0x7fd27c0776d0 2024-09-11T00:04:35.983 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:35.980+0000 7fd283000640 1 -- 172.21.15.12:0/4142719245 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd27c1a5ba0 con 0x7fd27c0776d0 2024-09-11T00:04:35.983 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:35.980+0000 7fd278c00640 1 -- 172.21.15.12:0/4142719245 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd26c003850 con 0x7fd27c0776d0 2024-09-11T00:04:35.983 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:35.980+0000 7fd283000640 1 -- 172.21.15.12:0/4142719245 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd27c1ac2c0 con 0x7fd27c0776d0 2024-09-11T00:04:35.983 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:35.981+0000 7fd283000640 1 -- 172.21.15.12:0/4142719245 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd27c077ab0 con 0x7fd27c0776d0 2024-09-11T00:04:35.985 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:35.983+0000 7fd278c00640 1 -- 172.21.15.12:0/4142719245 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 16) ==== 153250+0+0 (secure 0 0 0) 0x7fd26c004050 con 0x7fd27c0776d0 2024-09-11T00:04:35.988 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:35.986+0000 7fd278c00640 1 --2- 172.21.15.12:0/4142719245 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fd2600b52e0 0x7fd2600b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:04:35.988 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:35.986+0000 7fd278c00640 1 -- 172.21.15.12:0/4142719245 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8802+0+0 (secure 0 0 0) 0x7fd26c1061a0 con 0x7fd27c0776d0 2024-09-11T00:04:35.988 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:35.986+0000 7fd27b400640 1 --2- 172.21.15.12:0/4142719245 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fd2600b52e0 0x7fd2600b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:04:35.988 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:35.987+0000 7fd27b400640 1 --2- 172.21.15.12:0/4142719245 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fd2600b52e0 0x7fd2600b77a0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fd27c115e60 tx=0x7fd268009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:04:35.990 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:35.988+0000 7fd278c00640 1 -- 172.21.15.12:0/4142719245 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7fd26c04c060 con 0x7fd27c0776d0 2024-09-11T00:04:36.201 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:36.199+0000 7fd283000640 1 -- 172.21.15.12:0/4142719245 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fd27c1a0990 con 0x7fd27c0776d0 2024-09-11T00:04:36.201 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:04:35 smithi012 ceph-mon[30333]: pgmap v319: 129 pgs: 129 active+clean; 579 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T00:04:36.202 INFO:teuthology.orchestra.run.smithi012.stdout:172.21.15.12:6801/4177251511 2024-09-11T23:56:17.999820+0000 2024-09-11T00:04:36.202 INFO:teuthology.orchestra.run.smithi012.stdout:172.21.15.12:6800/4177251511 2024-09-11T23:56:17.999820+0000 2024-09-11T00:04:36.202 INFO:teuthology.orchestra.run.smithi012.stdout:172.21.15.12:0/3273692817 2024-09-11T23:56:17.999820+0000 2024-09-11T00:04:36.202 INFO:teuthology.orchestra.run.smithi012.stdout:172.21.15.12:0/4017778158 2024-09-11T23:56:17.999820+0000 2024-09-11T00:04:36.202 INFO:teuthology.orchestra.run.smithi012.stdout:172.21.15.12:0/2499921051 2024-09-11T23:55:50.302721+0000 2024-09-11T00:04:36.202 INFO:teuthology.orchestra.run.smithi012.stdout:172.21.15.12:6801/760140910 2024-09-11T23:55:50.302721+0000 2024-09-11T00:04:36.202 INFO:teuthology.orchestra.run.smithi012.stdout:172.21.15.12:6800/760140910 2024-09-11T23:55:50.302721+0000 2024-09-11T00:04:36.203 INFO:teuthology.orchestra.run.smithi012.stdout:172.21.15.12:0/2810703302 2024-09-11T23:56:17.999820+0000 2024-09-11T00:04:36.203 INFO:teuthology.orchestra.run.smithi012.stdout:172.21.15.12:0/2325436065 2024-09-11T23:55:50.302721+0000 2024-09-11T00:04:36.203 INFO:teuthology.orchestra.run.smithi012.stdout:172.21.15.12:0/3137053479 2024-09-11T23:55:50.302721+0000 2024-09-11T00:04:36.203 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:36.199+0000 7fd278c00640 1 -- 172.21.15.12:0/4142719245 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v81) ==== 81+0+590 (secure 0 0 0) 0x7fd26c0ccdc0 con 0x7fd27c0776d0 2024-09-11T00:04:36.203 INFO:teuthology.orchestra.run.smithi012.stderr:listed 10 entries 2024-09-11T00:04:36.205 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:36.203+0000 7fd283000640 1 -- 172.21.15.12:0/4142719245 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fd2600b52e0 msgr2=0x7fd2600b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:04:36.205 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:36.203+0000 7fd283000640 1 --2- 172.21.15.12:0/4142719245 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fd2600b52e0 0x7fd2600b77a0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fd27c115e60 tx=0x7fd268009260 comp rx=0 tx=0).stop 2024-09-11T00:04:36.205 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:36.203+0000 7fd283000640 1 -- 172.21.15.12:0/4142719245 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd27c0776d0 msgr2=0x7fd27c19f860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:04:36.205 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:36.203+0000 7fd283000640 1 --2- 172.21.15.12:0/4142719245 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd27c0776d0 0x7fd27c19f860 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fd26c035040 tx=0x7fd26c032d50 comp rx=0 tx=0).stop 2024-09-11T00:04:36.205 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:36.203+0000 7fd283000640 1 -- 172.21.15.12:0/4142719245 shutdown_connections 2024-09-11T00:04:36.205 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:36.203+0000 7fd283000640 1 --2- 172.21.15.12:0/4142719245 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fd2600b52e0 0x7fd2600b77a0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:04:36.205 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:36.203+0000 7fd283000640 1 --2- 172.21.15.12:0/4142719245 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd27c113990 0x7fd27c1a5140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:04:36.205 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:36.203+0000 7fd283000640 1 --2- 172.21.15.12:0/4142719245 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd27c077ff0 0x7fd27c19fda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:04:36.206 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:36.203+0000 7fd283000640 1 --2- 172.21.15.12:0/4142719245 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd27c0776d0 0x7fd27c19f860 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:04:36.206 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:36.203+0000 7fd283000640 1 -- 172.21.15.12:0/4142719245 >> 172.21.15.12:0/4142719245 conn(0x7fd27c100a50 msgr2=0x7fd27c102e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:04:36.206 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:36.203+0000 7fd283000640 1 -- 172.21.15.12:0/4142719245 shutdown_connections 2024-09-11T00:04:36.206 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:36.203+0000 7fd283000640 1 -- 172.21.15.12:0/4142719245 wait complete. 2024-09-11T00:04:36.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:04:35 smithi165 ceph-mon[35356]: pgmap v319: 129 pgs: 129 active+clean; 579 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T00:04:36.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:04:35 smithi178 ceph-mon[35919]: pgmap v319: 129 pgs: 129 active+clean; 579 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T00:04:37.007 DEBUG:teuthology.orchestra.run.smithi012:> set -ex 2024-09-11T00:04:37.007 DEBUG:teuthology.orchestra.run.smithi012:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-11T00:04:37.044 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid ff3ad74a-6fcf-11ef-bcea-c7b262605968 -- ceph osd blocklist ls 2024-09-11T00:04:37.183 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:04:36 smithi165 ceph-mon[35356]: from='client.? 172.21.15.12:0/4142719245' entity='client.admin' cmd={"prefix": "osd blocklist ls"} : dispatch 2024-09-11T00:04:37.232 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:04:36 smithi178 ceph-mon[35919]: from='client.? 172.21.15.12:0/4142719245' entity='client.admin' cmd={"prefix": "osd blocklist ls"} : dispatch 2024-09-11T00:04:37.312 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:04:36 smithi012 ceph-mon[30333]: from='client.? 172.21.15.12:0/4142719245' entity='client.admin' cmd={"prefix": "osd blocklist ls"} : dispatch 2024-09-11T00:04:37.467 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:04:38.060 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:38.057+0000 7fb2fc000640 1 -- 172.21.15.12:0/2637573643 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb2f4075f40 msgr2=0x7fb2f410ee10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:04:38.060 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:38.057+0000 7fb2fc000640 1 --2- 172.21.15.12:0/2637573643 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb2f4075f40 0x7fb2f410ee10 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fb2e0009900 tx=0x7fb2e002a410 comp rx=0 tx=0).stop 2024-09-11T00:04:38.060 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:38.057+0000 7fb2fc000640 1 -- 172.21.15.12:0/2637573643 shutdown_connections 2024-09-11T00:04:38.060 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:38.057+0000 7fb2fc000640 1 --2- 172.21.15.12:0/2637573643 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb2f410f3e0 0x7fb2f41117d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:04:38.060 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:38.057+0000 7fb2fc000640 1 --2- 172.21.15.12:0/2637573643 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb2f4075f40 0x7fb2f410ee10 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:04:38.060 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:38.057+0000 7fb2fc000640 1 --2- 172.21.15.12:0/2637573643 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb2f4075540 0x7fb2f4075920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:04:38.060 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:38.057+0000 7fb2fc000640 1 -- 172.21.15.12:0/2637573643 >> 172.21.15.12:0/2637573643 conn(0x7fb2f40fe830 msgr2=0x7fb2f4100c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:04:38.060 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:38.057+0000 7fb2fc000640 1 -- 172.21.15.12:0/2637573643 shutdown_connections 2024-09-11T00:04:38.061 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:38.057+0000 7fb2fc000640 1 -- 172.21.15.12:0/2637573643 wait complete. 2024-09-11T00:04:38.061 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:38.058+0000 7fb2fc000640 1 Processor -- start 2024-09-11T00:04:38.061 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:38.058+0000 7fb2fc000640 1 -- start start 2024-09-11T00:04:38.062 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:38.058+0000 7fb2fc000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb2f4075540 0x7fb2f419b4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:04:38.062 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:38.058+0000 7fb2fc000640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb2f4075f40 0x7fb2f419b9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:04:38.062 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:38.058+0000 7fb2fc000640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb2f410f3e0 0x7fb2f41a0d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:04:38.062 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:38.058+0000 7fb2fc000640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fb2f4114fa0 con 0x7fb2f410f3e0 2024-09-11T00:04:38.062 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:38.058+0000 7fb2fc000640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fb2f4114e20 con 0x7fb2f4075540 2024-09-11T00:04:38.062 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:38.058+0000 7fb2fc000640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fb2f4115120 con 0x7fb2f4075f40 2024-09-11T00:04:38.063 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:38.059+0000 7fb2fa000640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb2f410f3e0 0x7fb2f41a0d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:04:38.063 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:38.059+0000 7fb2f9600640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb2f4075540 0x7fb2f419b4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:04:38.063 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:38.059+0000 7fb2fa000640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb2f410f3e0 0x7fb2f41a0d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:34328/0 (socket says 172.21.15.12:34328) 2024-09-11T00:04:38.063 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:38.059+0000 7fb2fa000640 1 -- 172.21.15.12:0/3814122785 learned_addr learned my addr 172.21.15.12:0/3814122785 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:04:38.063 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:38.059+0000 7fb2f8c00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb2f4075f40 0x7fb2f419b9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:04:38.063 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:38.059+0000 7fb2f9600640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb2f4075540 0x7fb2f419b4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.12:55426/0 (socket says 172.21.15.12:55426) 2024-09-11T00:04:38.063 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:38.059+0000 7fb2f9600640 1 -- 172.21.15.12:0/3814122785 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb2f4075f40 msgr2=0x7fb2f419b9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:04:38.063 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:38.059+0000 7fb2f9600640 1 --2- 172.21.15.12:0/3814122785 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb2f4075f40 0x7fb2f419b9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:04:38.063 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:38.059+0000 7fb2f9600640 1 -- 172.21.15.12:0/3814122785 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb2f410f3e0 msgr2=0x7fb2f41a0d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:04:38.063 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:38.059+0000 7fb2f9600640 1 --2- 172.21.15.12:0/3814122785 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb2f410f3e0 0x7fb2f41a0d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:04:38.063 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:38.059+0000 7fb2f9600640 1 -- 172.21.15.12:0/3814122785 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2f41a1560 con 0x7fb2f4075540 2024-09-11T00:04:38.063 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:38.059+0000 7fb2fa000640 1 --2- 172.21.15.12:0/3814122785 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb2f410f3e0 0x7fb2f41a0d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:04:38.064 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:38.060+0000 7fb2f9600640 1 --2- 172.21.15.12:0/3814122785 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb2f4075540 0x7fb2f419b4b0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fb2e800e9f0 tx=0x7fb2e800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:04:38.064 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:38.060+0000 7fb2f2000640 1 -- 172.21.15.12:0/3814122785 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2e800ccc0 con 0x7fb2f4075540 2024-09-11T00:04:38.064 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:38.060+0000 7fb2fc000640 1 -- 172.21.15.12:0/3814122785 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb2f41a1850 con 0x7fb2f4075540 2024-09-11T00:04:38.064 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:38.060+0000 7fb2f2000640 1 -- 172.21.15.12:0/3814122785 <== mon.2 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb2e80108e0 con 0x7fb2f4075540 2024-09-11T00:04:38.064 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:38.060+0000 7fb2f2000640 1 -- 172.21.15.12:0/3814122785 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2e8018880 con 0x7fb2f4075540 2024-09-11T00:04:38.064 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:38.060+0000 7fb2fc000640 1 -- 172.21.15.12:0/3814122785 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2f41a80e0 con 0x7fb2f4075540 2024-09-11T00:04:38.064 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:38.061+0000 7fb2fc000640 1 -- 172.21.15.12:0/3814122785 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb2f4076a50 con 0x7fb2f4075540 2024-09-11T00:04:38.065 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:38.063+0000 7fb2f2000640 1 -- 172.21.15.12:0/3814122785 <== mon.2 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 16) ==== 153250+0+0 (secure 0 0 0) 0x7fb2e8010430 con 0x7fb2f4075540 2024-09-11T00:04:38.067 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:38.065+0000 7fb2f2000640 1 --2- 172.21.15.12:0/3814122785 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fb2dc0b5360 0x7fb2dc0b7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:04:38.067 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:38.065+0000 7fb2f2000640 1 -- 172.21.15.12:0/3814122785 <== mon.2 v2:172.21.15.165:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8802+0+0 (secure 0 0 0) 0x7fb2e80e6100 con 0x7fb2f4075540 2024-09-11T00:04:38.067 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:38.065+0000 7fb2f8c00640 1 --2- 172.21.15.12:0/3814122785 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fb2dc0b5360 0x7fb2dc0b7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:04:38.067 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:38.066+0000 7fb2f8c00640 1 --2- 172.21.15.12:0/3814122785 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fb2dc0b5360 0x7fb2dc0b7820 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fb2e0031440 tx=0x7fb2e0005b50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:04:38.068 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:38.067+0000 7fb2f2000640 1 -- 172.21.15.12:0/3814122785 <== mon.2 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7fb2e80ac6d0 con 0x7fb2f4075540 2024-09-11T00:04:38.199 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:04:37 smithi165 ceph-mon[35356]: pgmap v320: 129 pgs: 129 active+clean; 581 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-09-11T00:04:38.276 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:38.274+0000 7fb2fc000640 1 -- 172.21.15.12:0/3814122785 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fb2f4075920 con 0x7fb2f4075540 2024-09-11T00:04:38.276 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:04:37 smithi012 ceph-mon[30333]: pgmap v320: 129 pgs: 129 active+clean; 581 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-09-11T00:04:38.277 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:38.275+0000 7fb2f2000640 1 -- 172.21.15.12:0/3814122785 <== mon.2 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v81) ==== 81+0+590 (secure 0 0 0) 0x7fb2e80e3e20 con 0x7fb2f4075540 2024-09-11T00:04:38.277 INFO:teuthology.orchestra.run.smithi012.stdout:172.21.15.12:6801/4177251511 2024-09-11T23:56:17.999820+0000 2024-09-11T00:04:38.277 INFO:teuthology.orchestra.run.smithi012.stdout:172.21.15.12:6800/4177251511 2024-09-11T23:56:17.999820+0000 2024-09-11T00:04:38.277 INFO:teuthology.orchestra.run.smithi012.stdout:172.21.15.12:0/3273692817 2024-09-11T23:56:17.999820+0000 2024-09-11T00:04:38.277 INFO:teuthology.orchestra.run.smithi012.stdout:172.21.15.12:0/4017778158 2024-09-11T23:56:17.999820+0000 2024-09-11T00:04:38.277 INFO:teuthology.orchestra.run.smithi012.stdout:172.21.15.12:0/2499921051 2024-09-11T23:55:50.302721+0000 2024-09-11T00:04:38.278 INFO:teuthology.orchestra.run.smithi012.stdout:172.21.15.12:6801/760140910 2024-09-11T23:55:50.302721+0000 2024-09-11T00:04:38.278 INFO:teuthology.orchestra.run.smithi012.stdout:172.21.15.12:6800/760140910 2024-09-11T23:55:50.302721+0000 2024-09-11T00:04:38.278 INFO:teuthology.orchestra.run.smithi012.stdout:172.21.15.12:0/2810703302 2024-09-11T23:56:17.999820+0000 2024-09-11T00:04:38.278 INFO:teuthology.orchestra.run.smithi012.stdout:172.21.15.12:0/2325436065 2024-09-11T23:55:50.302721+0000 2024-09-11T00:04:38.278 INFO:teuthology.orchestra.run.smithi012.stdout:172.21.15.12:0/3137053479 2024-09-11T23:55:50.302721+0000 2024-09-11T00:04:38.278 INFO:teuthology.orchestra.run.smithi012.stderr:listed 10 entries 2024-09-11T00:04:38.280 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:38.277+0000 7fb2fc000640 1 -- 172.21.15.12:0/3814122785 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fb2dc0b5360 msgr2=0x7fb2dc0b7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:04:38.280 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:38.277+0000 7fb2fc000640 1 --2- 172.21.15.12:0/3814122785 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fb2dc0b5360 0x7fb2dc0b7820 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fb2e0031440 tx=0x7fb2e0005b50 comp rx=0 tx=0).stop 2024-09-11T00:04:38.280 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:38.277+0000 7fb2fc000640 1 -- 172.21.15.12:0/3814122785 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb2f4075540 msgr2=0x7fb2f419b4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:04:38.280 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:38.277+0000 7fb2fc000640 1 --2- 172.21.15.12:0/3814122785 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb2f4075540 0x7fb2f419b4b0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fb2e800e9f0 tx=0x7fb2e800eeb0 comp rx=0 tx=0).stop 2024-09-11T00:04:38.280 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:38.278+0000 7fb2fc000640 1 -- 172.21.15.12:0/3814122785 shutdown_connections 2024-09-11T00:04:38.280 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:38.278+0000 7fb2fc000640 1 --2- 172.21.15.12:0/3814122785 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fb2dc0b5360 0x7fb2dc0b7820 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:04:38.280 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:38.278+0000 7fb2fc000640 1 --2- 172.21.15.12:0/3814122785 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb2f410f3e0 0x7fb2f41a0d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:04:38.280 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:38.278+0000 7fb2fc000640 1 --2- 172.21.15.12:0/3814122785 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb2f4075f40 0x7fb2f419b9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:04:38.280 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:38.278+0000 7fb2fc000640 1 --2- 172.21.15.12:0/3814122785 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb2f4075540 0x7fb2f419b4b0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:04:38.280 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:38.278+0000 7fb2fc000640 1 -- 172.21.15.12:0/3814122785 >> 172.21.15.12:0/3814122785 conn(0x7fb2f40fe830 msgr2=0x7fb2f41115c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:04:38.280 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:38.278+0000 7fb2fc000640 1 -- 172.21.15.12:0/3814122785 shutdown_connections 2024-09-11T00:04:38.281 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:38.278+0000 7fb2fc000640 1 -- 172.21.15.12:0/3814122785 wait complete. 2024-09-11T00:04:38.330 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:04:37 smithi178 ceph-mon[35919]: pgmap v320: 129 pgs: 129 active+clean; 581 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-09-11T00:04:39.071 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi012.front.sepia.ceph.com... 2024-09-11T00:04:39.071 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T00:04:39.071 DEBUG:teuthology.orchestra.run.smithi012:> sudo fusermount -u /home/ubuntu/cephtest/mnt.admin 2024-09-11T00:04:39.127 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi012.stderr:ceph-fuse[76586]: fuse finished with error 0 and tester_r 0 2024-09-11T00:04:39.132 INFO:teuthology.orchestra.run:waiting for 300 2024-09-11T00:04:39.205 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:04:38 smithi012 ceph-mon[30333]: from='client.? 172.21.15.12:0/3814122785' entity='client.admin' cmd={"prefix": "osd blocklist ls"} : dispatch 2024-09-11T00:04:39.210 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi012.front.sepia.ceph.com 2024-09-11T00:04:39.210 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T00:04:39.211 DEBUG:teuthology.orchestra.run.smithi012:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.admin) 2024-09-11T00:04:39.257 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid ff3ad74a-6fcf-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-11T00:04:39.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:04:38 smithi165 ceph-mon[35356]: from='client.? 172.21.15.12:0/3814122785' entity='client.admin' cmd={"prefix": "osd blocklist ls"} : dispatch 2024-09-11T00:04:39.330 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:04:38 smithi178 ceph-mon[35919]: from='client.? 172.21.15.12:0/3814122785' entity='client.admin' cmd={"prefix": "osd blocklist ls"} : dispatch 2024-09-11T00:04:39.634 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:04:40.026 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:04:39 smithi012 ceph-mon[30333]: pgmap v321: 129 pgs: 129 active+clean; 581 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-09-11T00:04:40.131 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:40.129+0000 7face9c00640 1 -- 172.21.15.12:0/292296494 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7face410d120 msgr2=0x7face410f580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:04:40.132 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:40.129+0000 7face9c00640 1 --2- 172.21.15.12:0/292296494 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7face410d120 0x7face410f580 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7facdc009ca0 tx=0x7facdc02a8c0 comp rx=0 tx=0).stop 2024-09-11T00:04:40.132 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:40.129+0000 7face9c00640 1 -- 172.21.15.12:0/292296494 shutdown_connections 2024-09-11T00:04:40.132 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:40.129+0000 7face9c00640 1 --2- 172.21.15.12:0/292296494 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7face410d120 0x7face410f580 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:04:40.132 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:40.129+0000 7face9c00640 1 --2- 172.21.15.12:0/292296494 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7face41045e0 0x7face410cad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:04:40.132 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:40.129+0000 7face9c00640 1 --2- 172.21.15.12:0/292296494 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7face4103cc0 0x7face41040a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:04:40.132 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:40.129+0000 7face9c00640 1 -- 172.21.15.12:0/292296494 >> 172.21.15.12:0/292296494 conn(0x7face40fd700 msgr2=0x7face40ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:04:40.132 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:40.129+0000 7face9c00640 1 -- 172.21.15.12:0/292296494 shutdown_connections 2024-09-11T00:04:40.133 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:40.129+0000 7face9c00640 1 -- 172.21.15.12:0/292296494 wait complete. 2024-09-11T00:04:40.135 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:40.130+0000 7face9c00640 1 Processor -- start 2024-09-11T00:04:40.135 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:40.130+0000 7face9c00640 1 -- start start 2024-09-11T00:04:40.135 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:40.131+0000 7face9c00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7face4103cc0 0x7face419b490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:04:40.136 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:40.131+0000 7face9c00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7face41045e0 0x7face419b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:04:40.136 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:40.131+0000 7face9c00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7face410d120 0x7face41a0d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:04:40.136 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:40.131+0000 7face9c00640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7face4112de0 con 0x7face41045e0 2024-09-11T00:04:40.136 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:40.131+0000 7face9c00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7face4112c60 con 0x7face4103cc0 2024-09-11T00:04:40.136 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:40.131+0000 7face9c00640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7face4112f60 con 0x7face410d120 2024-09-11T00:04:40.136 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:40.131+0000 7face2a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7face4103cc0 0x7face419b490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:04:40.137 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:40.131+0000 7facdbe00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7face41045e0 0x7face419b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:04:40.137 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:40.131+0000 7face2a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7face4103cc0 0x7face419b490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.12:56880/0 (socket says 172.21.15.12:56880) 2024-09-11T00:04:40.137 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:40.131+0000 7face2a00640 1 -- 172.21.15.12:0/3752345394 learned_addr learned my addr 172.21.15.12:0/3752345394 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:04:40.137 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:40.131+0000 7face3400640 1 --2- 172.21.15.12:0/3752345394 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7face410d120 0x7face41a0d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:04:40.137 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:40.131+0000 7face2a00640 1 -- 172.21.15.12:0/3752345394 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7face410d120 msgr2=0x7face41a0d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:04:40.137 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:40.131+0000 7face2a00640 1 --2- 172.21.15.12:0/3752345394 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7face410d120 0x7face41a0d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:04:40.137 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:40.132+0000 7face2a00640 1 -- 172.21.15.12:0/3752345394 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7face41045e0 msgr2=0x7face419b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:04:40.137 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:40.132+0000 7face2a00640 1 --2- 172.21.15.12:0/3752345394 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7face41045e0 0x7face419b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:04:40.137 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:40.132+0000 7face2a00640 1 -- 172.21.15.12:0/3752345394 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7face41a1540 con 0x7face4103cc0 2024-09-11T00:04:40.137 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:40.132+0000 7facdbe00640 1 --2- 172.21.15.12:0/3752345394 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7face41045e0 0x7face419b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:04:40.138 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:40.132+0000 7face2a00640 1 --2- 172.21.15.12:0/3752345394 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7face4103cc0 0x7face419b490 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7facd400d950 tx=0x7facd400de10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:04:40.138 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:40.132+0000 7facdb400640 1 -- 172.21.15.12:0/3752345394 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facd4017070 con 0x7face4103cc0 2024-09-11T00:04:40.138 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:40.132+0000 7face9c00640 1 -- 172.21.15.12:0/3752345394 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7face41a1830 con 0x7face4103cc0 2024-09-11T00:04:40.138 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:40.132+0000 7facdb400640 1 -- 172.21.15.12:0/3752345394 <== mon.2 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7facd4009c30 con 0x7face4103cc0 2024-09-11T00:04:40.138 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:40.132+0000 7facdb400640 1 -- 172.21.15.12:0/3752345394 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facd400f620 con 0x7face4103cc0 2024-09-11T00:04:40.138 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:40.132+0000 7face9c00640 1 -- 172.21.15.12:0/3752345394 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7face41a80c0 con 0x7face4103cc0 2024-09-11T00:04:40.139 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:40.133+0000 7face9c00640 1 -- 172.21.15.12:0/3752345394 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7face406bc10 con 0x7face4103cc0 2024-09-11T00:04:40.139 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:40.136+0000 7facdb400640 1 -- 172.21.15.12:0/3752345394 <== mon.2 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 16) ==== 153250+0+0 (secure 0 0 0) 0x7facd4005030 con 0x7face4103cc0 2024-09-11T00:04:40.139 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:40.137+0000 7facdb400640 1 --2- 172.21.15.12:0/3752345394 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7facc80b52e0 0x7facc80b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:04:40.139 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:40.137+0000 7facdb400640 1 -- 172.21.15.12:0/3752345394 <== mon.2 v2:172.21.15.165:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8802+0+0 (secure 0 0 0) 0x7facd40e59e0 con 0x7face4103cc0 2024-09-11T00:04:40.140 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:40.137+0000 7facdbe00640 1 --2- 172.21.15.12:0/3752345394 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7facc80b52e0 0x7facc80b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:04:40.140 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:40.138+0000 7facdbe00640 1 --2- 172.21.15.12:0/3752345394 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7facc80b52e0 0x7facc80b77a0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7face410e7f0 tx=0x7faccc005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:04:40.140 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:40.138+0000 7facdb400640 1 -- 172.21.15.12:0/3752345394 <== mon.2 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7facd4009de0 con 0x7face4103cc0 2024-09-11T00:04:40.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:04:39 smithi165 ceph-mon[35356]: pgmap v321: 129 pgs: 129 active+clean; 581 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-09-11T00:04:40.330 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:04:39 smithi178 ceph-mon[35919]: pgmap v321: 129 pgs: 129 active+clean; 581 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-09-11T00:04:40.341 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:40.339+0000 7face9c00640 1 -- 172.21.15.12:0/3752345394 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7face41a8330 con 0x7face4103cc0 2024-09-11T00:04:40.344 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:40.341+0000 7facdb400640 1 -- 172.21.15.12:0/3752345394 <== mon.2 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) ==== 74+0+24236 (secure 0 0 0) 0x7facd40abfb0 con 0x7face4103cc0 2024-09-11T00:04:40.344 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-11T00:04:40.345 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":81,"fsid":"ff3ad74a-6fcf-11ef-bcea-c7b262605968","created":"2024-09-10T23:55:16.317643+0000","modified":"2024-09-11T00:04:15.795652+0000","last_up_change":"2024-09-11T00:03:08.959256+0000","last_in_change":"2024-09-11T00:02:46.564631+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T23:59:10.488728+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-11T00:04:00.065618+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"74","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-11T00:04:02.689888+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"74","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-11T00:04:09.862120+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"6382c200-a912-42a1-958b-79191b21249b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6802","nonce":1237101991},{"type":"v1","addr":"172.21.15.12:6803","nonce":1237101991}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6804","nonce":1237101991},{"type":"v1","addr":"172.21.15.12:6805","nonce":1237101991}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6808","nonce":1237101991},{"type":"v1","addr":"172.21.15.12:6809","nonce":1237101991}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6806","nonce":1237101991},{"type":"v1","addr":"172.21.15.12:6807","nonce":1237101991}]},"public_addr":"172.21.15.12:6803/1237101991","cluster_addr":"172.21.15.12:6805/1237101991","heartbeat_back_addr":"172.21.15.12:6809/1237101991","heartbeat_front_addr":"172.21.15.12:6807/1237101991","state":["exists","up"]},{"osd":1,"uuid":"fe0e1d0a-bdc5-4859-aa84-9038059a45f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6800","nonce":545848456},{"type":"v1","addr":"172.21.15.165:6801","nonce":545848456}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6802","nonce":545848456},{"type":"v1","addr":"172.21.15.165:6803","nonce":545848456}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6806","nonce":545848456},{"type":"v1","addr":"172.21.15.165:6807","nonce":545848456}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6804","nonce":545848456},{"type":"v1","addr":"172.21.15.165:6805","nonce":545848456}]},"public_addr":"172.21.15.165:6801/545848456","cluster_addr":"172.21.15.165:6803/545848456","heartbeat_back_addr":"172.21.15.165:6807/545848456","heartbeat_front_addr":"172.21.15.165:6805/545848456","state":["exists","up"]},{"osd":2,"uuid":"94f640dc-5f78-4e2c-87d9-03e96a6e7ba0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6800","nonce":4131162695},{"type":"v1","addr":"172.21.15.178:6801","nonce":4131162695}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":4131162695},{"type":"v1","addr":"172.21.15.178:6803","nonce":4131162695}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":4131162695},{"type":"v1","addr":"172.21.15.178:6807","nonce":4131162695}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":4131162695},{"type":"v1","addr":"172.21.15.178:6805","nonce":4131162695}]},"public_addr":"172.21.15.178:6801/4131162695","cluster_addr":"172.21.15.178:6803/4131162695","heartbeat_back_addr":"172.21.15.178:6807/4131162695","heartbeat_front_addr":"172.21.15.178:6805/4131162695","state":["exists","up"]},{"osd":3,"uuid":"af2f64d6-c9be-44a9-b041-8d599a7c080d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6810","nonce":452464245},{"type":"v1","addr":"172.21.15.12:6811","nonce":452464245}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6812","nonce":452464245},{"type":"v1","addr":"172.21.15.12:6813","nonce":452464245}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6816","nonce":452464245},{"type":"v1","addr":"172.21.15.12:6817","nonce":452464245}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6814","nonce":452464245},{"type":"v1","addr":"172.21.15.12:6815","nonce":452464245}]},"public_addr":"172.21.15.12:6811/452464245","cluster_addr":"172.21.15.12:6813/452464245","heartbeat_back_addr":"172.21.15.12:6817/452464245","heartbeat_front_addr":"172.21.15.12:6815/452464245","state":["exists","up"]},{"osd":4,"uuid":"ad76ac70-20c0-44ae-9866-5f9cfce2334c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6808","nonce":1899879280},{"type":"v1","addr":"172.21.15.165:6809","nonce":1899879280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6810","nonce":1899879280},{"type":"v1","addr":"172.21.15.165:6811","nonce":1899879280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6814","nonce":1899879280},{"type":"v1","addr":"172.21.15.165:6815","nonce":1899879280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6812","nonce":1899879280},{"type":"v1","addr":"172.21.15.165:6813","nonce":1899879280}]},"public_addr":"172.21.15.165:6809/1899879280","cluster_addr":"172.21.15.165:6811/1899879280","heartbeat_back_addr":"172.21.15.165:6815/1899879280","heartbeat_front_addr":"172.21.15.165:6813/1899879280","state":["exists","up"]},{"osd":5,"uuid":"aafda2cc-62fe-4fcb-9a96-8732e0b501d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":3042228555},{"type":"v1","addr":"172.21.15.178:6809","nonce":3042228555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":3042228555},{"type":"v1","addr":"172.21.15.178:6811","nonce":3042228555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6814","nonce":3042228555},{"type":"v1","addr":"172.21.15.178:6815","nonce":3042228555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":3042228555},{"type":"v1","addr":"172.21.15.178:6813","nonce":3042228555}]},"public_addr":"172.21.15.178:6809/3042228555","cluster_addr":"172.21.15.178:6811/3042228555","heartbeat_back_addr":"172.21.15.178:6815/3042228555","heartbeat_front_addr":"172.21.15.178:6813/3042228555","state":["exists","up"]},{"osd":6,"uuid":"375eca52-efe3-42a5-bbb1-d7d392d71f15","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6818","nonce":3868322467},{"type":"v1","addr":"172.21.15.12:6819","nonce":3868322467}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6820","nonce":3868322467},{"type":"v1","addr":"172.21.15.12:6821","nonce":3868322467}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6824","nonce":3868322467},{"type":"v1","addr":"172.21.15.12:6825","nonce":3868322467}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6822","nonce":3868322467},{"type":"v1","addr":"172.21.15.12:6823","nonce":3868322467}]},"public_addr":"172.21.15.12:6819/3868322467","cluster_addr":"172.21.15.12:6821/3868322467","heartbeat_back_addr":"172.21.15.12:6825/3868322467","heartbeat_front_addr":"172.21.15.12:6823/3868322467","state":["exists","up"]},{"osd":7,"uuid":"0fd619f5-44c3-4cf7-ae7a-388dd798de36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6816","nonce":1143773735},{"type":"v1","addr":"172.21.15.165:6817","nonce":1143773735}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6818","nonce":1143773735},{"type":"v1","addr":"172.21.15.165:6819","nonce":1143773735}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6822","nonce":1143773735},{"type":"v1","addr":"172.21.15.165:6823","nonce":1143773735}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6820","nonce":1143773735},{"type":"v1","addr":"172.21.15.165:6821","nonce":1143773735}]},"public_addr":"172.21.15.165:6817/1143773735","cluster_addr":"172.21.15.165:6819/1143773735","heartbeat_back_addr":"172.21.15.165:6823/1143773735","heartbeat_front_addr":"172.21.15.165:6821/1143773735","state":["exists","up"]},{"osd":8,"uuid":"b03d14ce-8427-462a-9bcc-27683c549f53","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":2891445058},{"type":"v1","addr":"172.21.15.178:6817","nonce":2891445058}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6818","nonce":2891445058},{"type":"v1","addr":"172.21.15.178:6819","nonce":2891445058}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6822","nonce":2891445058},{"type":"v1","addr":"172.21.15.178:6823","nonce":2891445058}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":2891445058},{"type":"v1","addr":"172.21.15.178:6821","nonce":2891445058}]},"public_addr":"172.21.15.178:6817/2891445058","cluster_addr":"172.21.15.178:6819/2891445058","heartbeat_back_addr":"172.21.15.178:6823/2891445058","heartbeat_front_addr":"172.21.15.178:6821/2891445058","state":["exists","up"]},{"osd":9,"uuid":"81a6cf69-2a24-4a6c-9adf-6b7df38cfacb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6826","nonce":3359634903},{"type":"v1","addr":"172.21.15.12:6827","nonce":3359634903}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6828","nonce":3359634903},{"type":"v1","addr":"172.21.15.12:6829","nonce":3359634903}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6832","nonce":3359634903},{"type":"v1","addr":"172.21.15.12:6833","nonce":3359634903}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6830","nonce":3359634903},{"type":"v1","addr":"172.21.15.12:6831","nonce":3359634903}]},"public_addr":"172.21.15.12:6827/3359634903","cluster_addr":"172.21.15.12:6829/3359634903","heartbeat_back_addr":"172.21.15.12:6833/3359634903","heartbeat_front_addr":"172.21.15.12:6831/3359634903","state":["exists","up"]},{"osd":10,"uuid":"c1ad1f0a-33f4-4c32-91f6-4d030a29f1dd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6824","nonce":3642325778},{"type":"v1","addr":"172.21.15.165:6825","nonce":3642325778}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6826","nonce":3642325778},{"type":"v1","addr":"172.21.15.165:6827","nonce":3642325778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6830","nonce":3642325778},{"type":"v1","addr":"172.21.15.165:6831","nonce":3642325778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6828","nonce":3642325778},{"type":"v1","addr":"172.21.15.165:6829","nonce":3642325778}]},"public_addr":"172.21.15.165:6825/3642325778","cluster_addr":"172.21.15.165:6827/3642325778","heartbeat_back_addr":"172.21.15.165:6831/3642325778","heartbeat_front_addr":"172.21.15.165:6829/3642325778","state":["exists","up"]},{"osd":11,"uuid":"8413c9b0-680b-4d02-aeab-b3b3ff16c3d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":2937059756},{"type":"v1","addr":"172.21.15.178:6825","nonce":2937059756}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6826","nonce":2937059756},{"type":"v1","addr":"172.21.15.178:6827","nonce":2937059756}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":2937059756},{"type":"v1","addr":"172.21.15.178:6831","nonce":2937059756}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":2937059756},{"type":"v1","addr":"172.21.15.178:6829","nonce":2937059756}]},"public_addr":"172.21.15.178:6825/2937059756","cluster_addr":"172.21.15.178:6827/2937059756","heartbeat_back_addr":"172.21.15.178:6831/2937059756","heartbeat_front_addr":"172.21.15.178:6829/2937059756","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:58:16.036348+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:58:41.483150+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:59:07.107283+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:59:35.706574+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T00:00:02.555720+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T00:00:28.092847+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T00:00:57.396495+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T00:01:24.216775+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T00:01:48.807850+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T00:02:16.838565+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T00:02:43.131736+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T00:03:07.353155+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.12:6801/4177251511":"2024-09-11T23:56:17.999820+0000","172.21.15.12:6800/4177251511":"2024-09-11T23:56:17.999820+0000","172.21.15.12:0/3273692817":"2024-09-11T23:56:17.999820+0000","172.21.15.12:0/4017778158":"2024-09-11T23:56:17.999820+0000","172.21.15.12:0/2499921051":"2024-09-11T23:55:50.302721+0000","172.21.15.12:6801/760140910":"2024-09-11T23:55:50.302721+0000","172.21.15.12:6800/760140910":"2024-09-11T23:55:50.302721+0000","172.21.15.12:0/2810703302":"2024-09-11T23:56:17.999820+0000","172.21.15.12:0/2325436065":"2024-09-11T23:55:50.302721+0000","172.21.15.12:0/3137053479":"2024-09-11T23:55:50.302721+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-11T00:04:40.346 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:40.344+0000 7face9c00640 1 -- 172.21.15.12:0/3752345394 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7facc80b52e0 msgr2=0x7facc80b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:04:40.346 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:40.344+0000 7face9c00640 1 --2- 172.21.15.12:0/3752345394 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7facc80b52e0 0x7facc80b77a0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7face410e7f0 tx=0x7faccc005f90 comp rx=0 tx=0).stop 2024-09-11T00:04:40.346 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:40.344+0000 7face9c00640 1 -- 172.21.15.12:0/3752345394 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7face4103cc0 msgr2=0x7face419b490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:04:40.346 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:40.344+0000 7face9c00640 1 --2- 172.21.15.12:0/3752345394 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7face4103cc0 0x7face419b490 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7facd400d950 tx=0x7facd400de10 comp rx=0 tx=0).stop 2024-09-11T00:04:40.346 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:40.344+0000 7face9c00640 1 -- 172.21.15.12:0/3752345394 shutdown_connections 2024-09-11T00:04:40.346 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:40.344+0000 7face9c00640 1 --2- 172.21.15.12:0/3752345394 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7facc80b52e0 0x7facc80b77a0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:04:40.346 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:40.344+0000 7face9c00640 1 --2- 172.21.15.12:0/3752345394 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7face410d120 0x7face41a0d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:04:40.346 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:40.344+0000 7face9c00640 1 --2- 172.21.15.12:0/3752345394 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7face41045e0 0x7face419b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:04:40.346 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:40.344+0000 7face9c00640 1 --2- 172.21.15.12:0/3752345394 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7face4103cc0 0x7face419b490 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:04:40.346 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:40.344+0000 7face9c00640 1 -- 172.21.15.12:0/3752345394 >> 172.21.15.12:0/3752345394 conn(0x7face40fd700 msgr2=0x7face410f2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:04:40.347 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:40.344+0000 7face9c00640 1 -- 172.21.15.12:0/3752345394 shutdown_connections 2024-09-11T00:04:40.347 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:40.344+0000 7face9c00640 1 -- 172.21.15.12:0/3752345394 wait complete. 2024-09-11T00:04:40.980 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:04:40 smithi012 ceph-mon[30333]: from='client.? 172.21.15.12:0/3752345394' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-11T00:04:41.017 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid ff3ad74a-6fcf-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-11T00:04:41.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:04:40 smithi165 ceph-mon[35356]: from='client.? 172.21.15.12:0/3752345394' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-11T00:04:41.330 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:04:40 smithi178 ceph-mon[35919]: from='client.? 172.21.15.12:0/3752345394' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-11T00:04:41.395 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:04:41.948 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:41.945+0000 7f93aa000640 1 -- 172.21.15.12:0/1063267891 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f93a4075de0 msgr2=0x7f93a410ee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:04:41.948 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:41.945+0000 7f93aa000640 1 --2- 172.21.15.12:0/1063267891 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f93a4075de0 0x7f93a410ee70 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f9390009860 tx=0x7f939002a340 comp rx=0 tx=0).stop 2024-09-11T00:04:41.948 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:41.945+0000 7f93aa000640 1 -- 172.21.15.12:0/1063267891 shutdown_connections 2024-09-11T00:04:41.948 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:41.945+0000 7f93aa000640 1 --2- 172.21.15.12:0/1063267891 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f93a410f3b0 0x7f93a41117a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:04:41.948 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:41.945+0000 7f93aa000640 1 --2- 172.21.15.12:0/1063267891 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f93a4075de0 0x7f93a410ee70 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:04:41.948 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:41.945+0000 7f93aa000640 1 --2- 172.21.15.12:0/1063267891 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f93a40754c0 0x7f93a40758a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:04:41.948 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:41.945+0000 7f93aa000640 1 -- 172.21.15.12:0/1063267891 >> 172.21.15.12:0/1063267891 conn(0x7f93a40fe810 msgr2=0x7f93a4100c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:04:41.948 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:41.945+0000 7f93aa000640 1 -- 172.21.15.12:0/1063267891 shutdown_connections 2024-09-11T00:04:41.948 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:41.945+0000 7f93aa000640 1 -- 172.21.15.12:0/1063267891 wait complete. 2024-09-11T00:04:41.949 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:41.946+0000 7f93aa000640 1 Processor -- start 2024-09-11T00:04:41.949 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:41.946+0000 7f93aa000640 1 -- start start 2024-09-11T00:04:41.950 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:41.946+0000 7f93aa000640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f93a40754c0 0x7f93a419b510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:04:41.950 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:41.946+0000 7f93aa000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f93a4075de0 0x7f93a419ba50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:04:41.950 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:41.947+0000 7f93aa000640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f93a410f3b0 0x7f93a41a0df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:04:41.950 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:41.947+0000 7f93aa000640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f93a4114fa0 con 0x7f93a40754c0 2024-09-11T00:04:41.950 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:41.947+0000 7f93aa000640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f93a4114e20 con 0x7f93a4075de0 2024-09-11T00:04:41.950 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:41.947+0000 7f93aa000640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f93a4115120 con 0x7f93a410f3b0 2024-09-11T00:04:41.950 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:41.947+0000 7f93a2a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f93a4075de0 0x7f93a419ba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:04:41.950 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:41.947+0000 7f93a3e00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f93a410f3b0 0x7f93a41a0df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:04:41.951 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:41.947+0000 7f93a2a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f93a4075de0 0x7f93a419ba50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.12:56890/0 (socket says 172.21.15.12:56890) 2024-09-11T00:04:41.951 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:41.947+0000 7f93a2a00640 1 -- 172.21.15.12:0/3413341715 learned_addr learned my addr 172.21.15.12:0/3413341715 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:04:41.951 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:41.947+0000 7f93a3400640 1 --2- 172.21.15.12:0/3413341715 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f93a40754c0 0x7f93a419b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:04:41.951 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:41.947+0000 7f93a3e00640 1 -- 172.21.15.12:0/3413341715 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f93a4075de0 msgr2=0x7f93a419ba50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:04:41.951 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:41.947+0000 7f93a3e00640 1 --2- 172.21.15.12:0/3413341715 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f93a4075de0 0x7f93a419ba50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:04:41.951 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:41.947+0000 7f93a3e00640 1 -- 172.21.15.12:0/3413341715 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f93a40754c0 msgr2=0x7f93a419b510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:04:41.951 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:41.947+0000 7f93a3e00640 1 --2- 172.21.15.12:0/3413341715 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f93a40754c0 0x7f93a419b510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:04:41.951 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:41.947+0000 7f93a3e00640 1 -- 172.21.15.12:0/3413341715 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93a41a15c0 con 0x7f93a410f3b0 2024-09-11T00:04:41.951 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:41.947+0000 7f93a2a00640 1 --2- 172.21.15.12:0/3413341715 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f93a4075de0 0x7f93a419ba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:04:41.951 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:41.948+0000 7f93a3e00640 1 --2- 172.21.15.12:0/3413341715 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f93a410f3b0 0x7f93a41a0df0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f93a406c0d0 tx=0x7f939c00ee50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:04:41.951 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:41.948+0000 7f939be00640 1 -- 172.21.15.12:0/3413341715 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f939c00cd30 con 0x7f93a410f3b0 2024-09-11T00:04:41.951 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:41.948+0000 7f93aa000640 1 -- 172.21.15.12:0/3413341715 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f93a41a18b0 con 0x7f93a410f3b0 2024-09-11T00:04:41.951 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:41.948+0000 7f939be00640 1 -- 172.21.15.12:0/3413341715 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f939c005020 con 0x7f93a410f3b0 2024-09-11T00:04:41.952 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:41.948+0000 7f939be00640 1 -- 172.21.15.12:0/3413341715 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f939c010690 con 0x7f93a410f3b0 2024-09-11T00:04:41.952 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:41.948+0000 7f93aa000640 1 -- 172.21.15.12:0/3413341715 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f93a41a7f70 con 0x7f93a410f3b0 2024-09-11T00:04:41.952 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:41.949+0000 7f93aa000640 1 -- 172.21.15.12:0/3413341715 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f93a40758a0 con 0x7f93a410f3b0 2024-09-11T00:04:41.953 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:41.951+0000 7f939be00640 1 -- 172.21.15.12:0/3413341715 <== mon.1 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 16) ==== 153250+0+0 (secure 0 0 0) 0x7f939c0054d0 con 0x7f93a410f3b0 2024-09-11T00:04:41.954 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:41.953+0000 7f939be00640 1 --2- 172.21.15.12:0/3413341715 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f93880b52e0 0x7f93880b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:04:41.955 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:41.953+0000 7f939be00640 1 -- 172.21.15.12:0/3413341715 <== mon.1 v2:172.21.15.178:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8802+0+0 (secure 0 0 0) 0x7f939c0e6560 con 0x7f93a410f3b0 2024-09-11T00:04:41.955 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:41.953+0000 7f93a3400640 1 --2- 172.21.15.12:0/3413341715 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f93880b52e0 0x7f93880b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:04:41.955 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:41.953+0000 7f93a3400640 1 --2- 172.21.15.12:0/3413341715 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f93880b52e0 0x7f93880b77a0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f9394005fd0 tx=0x7f9394009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:04:41.958 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:41.956+0000 7f939be00640 1 -- 172.21.15.12:0/3413341715 <== mon.1 v2:172.21.15.178:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f939c0ad180 con 0x7f93a410f3b0 2024-09-11T00:04:42.155 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:42.153+0000 7f93aa000640 1 -- 172.21.15.12:0/3413341715 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f93a410a530 con 0x7f93a410f3b0 2024-09-11T00:04:42.156 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:04:41 smithi012 ceph-mon[30333]: pgmap v322: 129 pgs: 129 active+clean; 581 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-09-11T00:04:42.157 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:42.155+0000 7f939be00640 1 -- 172.21.15.12:0/3413341715 <== mon.1 v2:172.21.15.178:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) ==== 74+0+24236 (secure 0 0 0) 0x7f939c0aca80 con 0x7f93a410f3b0 2024-09-11T00:04:42.157 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-11T00:04:42.158 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":81,"fsid":"ff3ad74a-6fcf-11ef-bcea-c7b262605968","created":"2024-09-10T23:55:16.317643+0000","modified":"2024-09-11T00:04:15.795652+0000","last_up_change":"2024-09-11T00:03:08.959256+0000","last_in_change":"2024-09-11T00:02:46.564631+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T23:59:10.488728+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-11T00:04:00.065618+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"74","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-11T00:04:02.689888+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"74","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-11T00:04:09.862120+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"6382c200-a912-42a1-958b-79191b21249b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6802","nonce":1237101991},{"type":"v1","addr":"172.21.15.12:6803","nonce":1237101991}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6804","nonce":1237101991},{"type":"v1","addr":"172.21.15.12:6805","nonce":1237101991}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6808","nonce":1237101991},{"type":"v1","addr":"172.21.15.12:6809","nonce":1237101991}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6806","nonce":1237101991},{"type":"v1","addr":"172.21.15.12:6807","nonce":1237101991}]},"public_addr":"172.21.15.12:6803/1237101991","cluster_addr":"172.21.15.12:6805/1237101991","heartbeat_back_addr":"172.21.15.12:6809/1237101991","heartbeat_front_addr":"172.21.15.12:6807/1237101991","state":["exists","up"]},{"osd":1,"uuid":"fe0e1d0a-bdc5-4859-aa84-9038059a45f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6800","nonce":545848456},{"type":"v1","addr":"172.21.15.165:6801","nonce":545848456}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6802","nonce":545848456},{"type":"v1","addr":"172.21.15.165:6803","nonce":545848456}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6806","nonce":545848456},{"type":"v1","addr":"172.21.15.165:6807","nonce":545848456}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6804","nonce":545848456},{"type":"v1","addr":"172.21.15.165:6805","nonce":545848456}]},"public_addr":"172.21.15.165:6801/545848456","cluster_addr":"172.21.15.165:6803/545848456","heartbeat_back_addr":"172.21.15.165:6807/545848456","heartbeat_front_addr":"172.21.15.165:6805/545848456","state":["exists","up"]},{"osd":2,"uuid":"94f640dc-5f78-4e2c-87d9-03e96a6e7ba0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6800","nonce":4131162695},{"type":"v1","addr":"172.21.15.178:6801","nonce":4131162695}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":4131162695},{"type":"v1","addr":"172.21.15.178:6803","nonce":4131162695}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":4131162695},{"type":"v1","addr":"172.21.15.178:6807","nonce":4131162695}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":4131162695},{"type":"v1","addr":"172.21.15.178:6805","nonce":4131162695}]},"public_addr":"172.21.15.178:6801/4131162695","cluster_addr":"172.21.15.178:6803/4131162695","heartbeat_back_addr":"172.21.15.178:6807/4131162695","heartbeat_front_addr":"172.21.15.178:6805/4131162695","state":["exists","up"]},{"osd":3,"uuid":"af2f64d6-c9be-44a9-b041-8d599a7c080d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6810","nonce":452464245},{"type":"v1","addr":"172.21.15.12:6811","nonce":452464245}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6812","nonce":452464245},{"type":"v1","addr":"172.21.15.12:6813","nonce":452464245}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6816","nonce":452464245},{"type":"v1","addr":"172.21.15.12:6817","nonce":452464245}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6814","nonce":452464245},{"type":"v1","addr":"172.21.15.12:6815","nonce":452464245}]},"public_addr":"172.21.15.12:6811/452464245","cluster_addr":"172.21.15.12:6813/452464245","heartbeat_back_addr":"172.21.15.12:6817/452464245","heartbeat_front_addr":"172.21.15.12:6815/452464245","state":["exists","up"]},{"osd":4,"uuid":"ad76ac70-20c0-44ae-9866-5f9cfce2334c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6808","nonce":1899879280},{"type":"v1","addr":"172.21.15.165:6809","nonce":1899879280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6810","nonce":1899879280},{"type":"v1","addr":"172.21.15.165:6811","nonce":1899879280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6814","nonce":1899879280},{"type":"v1","addr":"172.21.15.165:6815","nonce":1899879280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6812","nonce":1899879280},{"type":"v1","addr":"172.21.15.165:6813","nonce":1899879280}]},"public_addr":"172.21.15.165:6809/1899879280","cluster_addr":"172.21.15.165:6811/1899879280","heartbeat_back_addr":"172.21.15.165:6815/1899879280","heartbeat_front_addr":"172.21.15.165:6813/1899879280","state":["exists","up"]},{"osd":5,"uuid":"aafda2cc-62fe-4fcb-9a96-8732e0b501d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":3042228555},{"type":"v1","addr":"172.21.15.178:6809","nonce":3042228555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":3042228555},{"type":"v1","addr":"172.21.15.178:6811","nonce":3042228555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6814","nonce":3042228555},{"type":"v1","addr":"172.21.15.178:6815","nonce":3042228555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":3042228555},{"type":"v1","addr":"172.21.15.178:6813","nonce":3042228555}]},"public_addr":"172.21.15.178:6809/3042228555","cluster_addr":"172.21.15.178:6811/3042228555","heartbeat_back_addr":"172.21.15.178:6815/3042228555","heartbeat_front_addr":"172.21.15.178:6813/3042228555","state":["exists","up"]},{"osd":6,"uuid":"375eca52-efe3-42a5-bbb1-d7d392d71f15","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6818","nonce":3868322467},{"type":"v1","addr":"172.21.15.12:6819","nonce":3868322467}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6820","nonce":3868322467},{"type":"v1","addr":"172.21.15.12:6821","nonce":3868322467}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6824","nonce":3868322467},{"type":"v1","addr":"172.21.15.12:6825","nonce":3868322467}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6822","nonce":3868322467},{"type":"v1","addr":"172.21.15.12:6823","nonce":3868322467}]},"public_addr":"172.21.15.12:6819/3868322467","cluster_addr":"172.21.15.12:6821/3868322467","heartbeat_back_addr":"172.21.15.12:6825/3868322467","heartbeat_front_addr":"172.21.15.12:6823/3868322467","state":["exists","up"]},{"osd":7,"uuid":"0fd619f5-44c3-4cf7-ae7a-388dd798de36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6816","nonce":1143773735},{"type":"v1","addr":"172.21.15.165:6817","nonce":1143773735}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6818","nonce":1143773735},{"type":"v1","addr":"172.21.15.165:6819","nonce":1143773735}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6822","nonce":1143773735},{"type":"v1","addr":"172.21.15.165:6823","nonce":1143773735}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6820","nonce":1143773735},{"type":"v1","addr":"172.21.15.165:6821","nonce":1143773735}]},"public_addr":"172.21.15.165:6817/1143773735","cluster_addr":"172.21.15.165:6819/1143773735","heartbeat_back_addr":"172.21.15.165:6823/1143773735","heartbeat_front_addr":"172.21.15.165:6821/1143773735","state":["exists","up"]},{"osd":8,"uuid":"b03d14ce-8427-462a-9bcc-27683c549f53","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":2891445058},{"type":"v1","addr":"172.21.15.178:6817","nonce":2891445058}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6818","nonce":2891445058},{"type":"v1","addr":"172.21.15.178:6819","nonce":2891445058}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6822","nonce":2891445058},{"type":"v1","addr":"172.21.15.178:6823","nonce":2891445058}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":2891445058},{"type":"v1","addr":"172.21.15.178:6821","nonce":2891445058}]},"public_addr":"172.21.15.178:6817/2891445058","cluster_addr":"172.21.15.178:6819/2891445058","heartbeat_back_addr":"172.21.15.178:6823/2891445058","heartbeat_front_addr":"172.21.15.178:6821/2891445058","state":["exists","up"]},{"osd":9,"uuid":"81a6cf69-2a24-4a6c-9adf-6b7df38cfacb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6826","nonce":3359634903},{"type":"v1","addr":"172.21.15.12:6827","nonce":3359634903}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6828","nonce":3359634903},{"type":"v1","addr":"172.21.15.12:6829","nonce":3359634903}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6832","nonce":3359634903},{"type":"v1","addr":"172.21.15.12:6833","nonce":3359634903}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6830","nonce":3359634903},{"type":"v1","addr":"172.21.15.12:6831","nonce":3359634903}]},"public_addr":"172.21.15.12:6827/3359634903","cluster_addr":"172.21.15.12:6829/3359634903","heartbeat_back_addr":"172.21.15.12:6833/3359634903","heartbeat_front_addr":"172.21.15.12:6831/3359634903","state":["exists","up"]},{"osd":10,"uuid":"c1ad1f0a-33f4-4c32-91f6-4d030a29f1dd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6824","nonce":3642325778},{"type":"v1","addr":"172.21.15.165:6825","nonce":3642325778}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6826","nonce":3642325778},{"type":"v1","addr":"172.21.15.165:6827","nonce":3642325778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6830","nonce":3642325778},{"type":"v1","addr":"172.21.15.165:6831","nonce":3642325778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6828","nonce":3642325778},{"type":"v1","addr":"172.21.15.165:6829","nonce":3642325778}]},"public_addr":"172.21.15.165:6825/3642325778","cluster_addr":"172.21.15.165:6827/3642325778","heartbeat_back_addr":"172.21.15.165:6831/3642325778","heartbeat_front_addr":"172.21.15.165:6829/3642325778","state":["exists","up"]},{"osd":11,"uuid":"8413c9b0-680b-4d02-aeab-b3b3ff16c3d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":2937059756},{"type":"v1","addr":"172.21.15.178:6825","nonce":2937059756}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6826","nonce":2937059756},{"type":"v1","addr":"172.21.15.178:6827","nonce":2937059756}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":2937059756},{"type":"v1","addr":"172.21.15.178:6831","nonce":2937059756}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":2937059756},{"type":"v1","addr":"172.21.15.178:6829","nonce":2937059756}]},"public_addr":"172.21.15.178:6825/2937059756","cluster_addr":"172.21.15.178:6827/2937059756","heartbeat_back_addr":"172.21.15.178:6831/2937059756","heartbeat_front_addr":"172.21.15.178:6829/2937059756","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:58:16.036348+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:58:41.483150+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:59:07.107283+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:59:35.706574+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T00:00:02.555720+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T00:00:28.092847+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T00:00:57.396495+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T00:01:24.216775+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T00:01:48.807850+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T00:02:16.838565+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T00:02:43.131736+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T00:03:07.353155+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.12:6801/4177251511":"2024-09-11T23:56:17.999820+0000","172.21.15.12:6800/4177251511":"2024-09-11T23:56:17.999820+0000","172.21.15.12:0/3273692817":"2024-09-11T23:56:17.999820+0000","172.21.15.12:0/4017778158":"2024-09-11T23:56:17.999820+0000","172.21.15.12:0/2499921051":"2024-09-11T23:55:50.302721+0000","172.21.15.12:6801/760140910":"2024-09-11T23:55:50.302721+0000","172.21.15.12:6800/760140910":"2024-09-11T23:55:50.302721+0000","172.21.15.12:0/2810703302":"2024-09-11T23:56:17.999820+0000","172.21.15.12:0/2325436065":"2024-09-11T23:55:50.302721+0000","172.21.15.12:0/3137053479":"2024-09-11T23:55:50.302721+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-11T00:04:42.160 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:42.158+0000 7f93aa000640 1 -- 172.21.15.12:0/3413341715 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f93880b52e0 msgr2=0x7f93880b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:04:42.160 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:42.158+0000 7f93aa000640 1 --2- 172.21.15.12:0/3413341715 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f93880b52e0 0x7f93880b77a0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f9394005fd0 tx=0x7f9394009260 comp rx=0 tx=0).stop 2024-09-11T00:04:42.160 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:42.158+0000 7f93aa000640 1 -- 172.21.15.12:0/3413341715 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f93a410f3b0 msgr2=0x7f93a41a0df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:04:42.160 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:42.158+0000 7f93aa000640 1 --2- 172.21.15.12:0/3413341715 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f93a410f3b0 0x7f93a41a0df0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f93a406c0d0 tx=0x7f939c00ee50 comp rx=0 tx=0).stop 2024-09-11T00:04:42.160 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:42.158+0000 7f93aa000640 1 -- 172.21.15.12:0/3413341715 shutdown_connections 2024-09-11T00:04:42.160 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:42.158+0000 7f93aa000640 1 --2- 172.21.15.12:0/3413341715 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f93880b52e0 0x7f93880b77a0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:04:42.161 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:42.158+0000 7f93aa000640 1 --2- 172.21.15.12:0/3413341715 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f93a410f3b0 0x7f93a41a0df0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:04:42.161 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:42.158+0000 7f93aa000640 1 --2- 172.21.15.12:0/3413341715 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f93a4075de0 0x7f93a419ba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:04:42.161 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:42.158+0000 7f93aa000640 1 --2- 172.21.15.12:0/3413341715 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f93a40754c0 0x7f93a419b510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:04:42.161 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:42.158+0000 7f93aa000640 1 -- 172.21.15.12:0/3413341715 >> 172.21.15.12:0/3413341715 conn(0x7f93a40fe810 msgr2=0x7f93a40ff410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:04:42.161 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:42.158+0000 7f93aa000640 1 -- 172.21.15.12:0/3413341715 shutdown_connections 2024-09-11T00:04:42.161 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:42.158+0000 7f93aa000640 1 -- 172.21.15.12:0/3413341715 wait complete. 2024-09-11T00:04:42.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:04:41 smithi165 ceph-mon[35356]: pgmap v322: 129 pgs: 129 active+clean; 581 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-09-11T00:04:42.330 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:04:41 smithi178 ceph-mon[35919]: pgmap v322: 129 pgs: 129 active+clean; 581 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-09-11T00:04:42.812 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid ff3ad74a-6fcf-11ef-bcea-c7b262605968 -- ceph fs set cephfs standby_count_wanted 0 2024-09-11T00:04:43.063 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:04:42 smithi012 ceph-mon[30333]: pgmap v323: 129 pgs: 129 active+clean; 581 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-09-11T00:04:43.063 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:04:42 smithi012 ceph-mon[30333]: from='client.? 172.21.15.12:0/3413341715' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-11T00:04:43.192 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:04:43.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:04:42 smithi165 ceph-mon[35356]: pgmap v323: 129 pgs: 129 active+clean; 581 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-09-11T00:04:43.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:04:42 smithi165 ceph-mon[35356]: from='client.? 172.21.15.12:0/3413341715' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-11T00:04:43.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:04:42 smithi178 ceph-mon[35919]: pgmap v323: 129 pgs: 129 active+clean; 581 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-09-11T00:04:43.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:04:42 smithi178 ceph-mon[35919]: from='client.? 172.21.15.12:0/3413341715' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-11T00:04:43.901 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:43.899+0000 7fdfad400640 1 -- 172.21.15.12:0/1077672519 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdfa8075540 msgr2=0x7fdfa8075920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:04:43.901 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:43.899+0000 7fdfad400640 1 --2- 172.21.15.12:0/1077672519 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdfa8075540 0x7fdfa8075920 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fdf98009900 tx=0x7fdf9802a3f0 comp rx=0 tx=0).stop 2024-09-11T00:04:43.902 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:43.899+0000 7fdfad400640 1 -- 172.21.15.12:0/1077672519 shutdown_connections 2024-09-11T00:04:43.902 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:43.899+0000 7fdfad400640 1 --2- 172.21.15.12:0/1077672519 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdfa810f3e0 0x7fdfa81117d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:04:43.902 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:43.899+0000 7fdfad400640 1 --2- 172.21.15.12:0/1077672519 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fdfa8075f40 0x7fdfa810ee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:04:43.902 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:43.899+0000 7fdfad400640 1 --2- 172.21.15.12:0/1077672519 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdfa8075540 0x7fdfa8075920 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:04:43.902 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:43.899+0000 7fdfad400640 1 -- 172.21.15.12:0/1077672519 >> 172.21.15.12:0/1077672519 conn(0x7fdfa80fe830 msgr2=0x7fdfa8100c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:04:43.902 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:43.899+0000 7fdfad400640 1 -- 172.21.15.12:0/1077672519 shutdown_connections 2024-09-11T00:04:43.902 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:43.899+0000 7fdfad400640 1 -- 172.21.15.12:0/1077672519 wait complete. 2024-09-11T00:04:43.903 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:43.900+0000 7fdfad400640 1 Processor -- start 2024-09-11T00:04:43.903 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:43.900+0000 7fdfad400640 1 -- start start 2024-09-11T00:04:43.904 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:43.900+0000 7fdfad400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fdfa8075540 0x7fdfa819b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:04:43.904 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:43.900+0000 7fdfad400640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdfa8075f40 0x7fdfa819ba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:04:43.904 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:43.900+0000 7fdfad400640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdfa810f3e0 0x7fdfa81a0dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:04:43.904 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:43.900+0000 7fdfad400640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fdfa8114f90 con 0x7fdfa8075f40 2024-09-11T00:04:43.904 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:43.900+0000 7fdfad400640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fdfa8114e10 con 0x7fdfa8075540 2024-09-11T00:04:43.904 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:43.900+0000 7fdfad400640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fdfa8115110 con 0x7fdfa810f3e0 2024-09-11T00:04:43.904 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:43.901+0000 7fdfa6800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fdfa8075540 0x7fdfa819b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:04:43.904 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:43.901+0000 7fdfa6800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fdfa8075540 0x7fdfa819b4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.12:56908/0 (socket says 172.21.15.12:56908) 2024-09-11T00:04:43.904 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:43.901+0000 7fdfa6800640 1 -- 172.21.15.12:0/395400598 learned_addr learned my addr 172.21.15.12:0/395400598 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:04:43.904 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:43.901+0000 7fdf9de00640 1 --2- 172.21.15.12:0/395400598 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdfa8075f40 0x7fdfa819ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:04:43.904 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:43.901+0000 7fdfa7200640 1 --2- 172.21.15.12:0/395400598 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdfa810f3e0 0x7fdfa81a0dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:04:43.904 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:43.901+0000 7fdfa6800640 1 -- 172.21.15.12:0/395400598 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdfa810f3e0 msgr2=0x7fdfa81a0dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:04:43.904 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:43.901+0000 7fdfa6800640 1 --2- 172.21.15.12:0/395400598 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdfa810f3e0 0x7fdfa81a0dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:04:43.905 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:43.901+0000 7fdfa6800640 1 -- 172.21.15.12:0/395400598 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdfa8075f40 msgr2=0x7fdfa819ba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:04:43.905 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:43.901+0000 7fdfa6800640 1 --2- 172.21.15.12:0/395400598 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdfa8075f40 0x7fdfa819ba20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:04:43.905 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:43.901+0000 7fdfa6800640 1 -- 172.21.15.12:0/395400598 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdfa81a1590 con 0x7fdfa8075540 2024-09-11T00:04:43.905 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:43.901+0000 7fdfa7200640 1 --2- 172.21.15.12:0/395400598 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdfa810f3e0 0x7fdfa81a0dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:04:43.905 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:43.902+0000 7fdfa6800640 1 --2- 172.21.15.12:0/395400598 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fdfa8075540 0x7fdfa819b4e0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fdf98005940 tx=0x7fdf980040c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:04:43.906 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:43.902+0000 7fdf9fe00640 1 -- 172.21.15.12:0/395400598 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf9803f030 con 0x7fdfa8075540 2024-09-11T00:04:43.906 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:43.902+0000 7fdfad400640 1 -- 172.21.15.12:0/395400598 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdfa81a1820 con 0x7fdfa8075540 2024-09-11T00:04:43.906 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:43.902+0000 7fdf9fe00640 1 -- 172.21.15.12:0/395400598 <== mon.2 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdf980044c0 con 0x7fdfa8075540 2024-09-11T00:04:43.906 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:43.902+0000 7fdf9fe00640 1 -- 172.21.15.12:0/395400598 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf98035070 con 0x7fdfa8075540 2024-09-11T00:04:43.906 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:43.902+0000 7fdfad400640 1 -- 172.21.15.12:0/395400598 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdfa81a7f40 con 0x7fdfa8075540 2024-09-11T00:04:43.906 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:43.903+0000 7fdfad400640 1 -- 172.21.15.12:0/395400598 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdfa8075920 con 0x7fdfa8075540 2024-09-11T00:04:43.907 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:43.906+0000 7fdf9fe00640 1 -- 172.21.15.12:0/395400598 <== mon.2 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 16) ==== 153250+0+0 (secure 0 0 0) 0x7fdf9803b910 con 0x7fdfa8075540 2024-09-11T00:04:43.909 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:43.907+0000 7fdf9fe00640 1 --2- 172.21.15.12:0/395400598 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fdf8c0b52e0 0x7fdf8c0b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:04:43.909 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:43.907+0000 7fdf9fe00640 1 -- 172.21.15.12:0/395400598 <== mon.2 v2:172.21.15.165:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8802+0+0 (secure 0 0 0) 0x7fdf981066a0 con 0x7fdfa8075540 2024-09-11T00:04:43.909 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:43.907+0000 7fdf9de00640 1 --2- 172.21.15.12:0/395400598 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fdf8c0b52e0 0x7fdf8c0b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:04:43.910 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:43.908+0000 7fdf9de00640 1 --2- 172.21.15.12:0/395400598 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fdf8c0b52e0 0x7fdf8c0b77a0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fdfa8111a50 tx=0x7fdf90005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:04:43.911 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:43.909+0000 7fdf9fe00640 1 -- 172.21.15.12:0/395400598 <== mon.2 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7fdf980cd2c0 con 0x7fdfa8075540 2024-09-11T00:04:44.144 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:44.142+0000 7fdfad400640 1 -- 172.21.15.12:0/395400598 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7fdfa81a8280 con 0x7fdfa8075540 2024-09-11T00:04:44.204 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:44.202+0000 7fdf9fe00640 1 -- 172.21.15.12:0/395400598 <== mon.2 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v18) ==== 118+0+0 (secure 0 0 0) 0x7fdf980ccbc0 con 0x7fdfa8075540 2024-09-11T00:04:44.206 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:44.204+0000 7fdfad400640 1 -- 172.21.15.12:0/395400598 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fdf8c0b52e0 msgr2=0x7fdf8c0b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:04:44.206 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:44.204+0000 7fdfad400640 1 --2- 172.21.15.12:0/395400598 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fdf8c0b52e0 0x7fdf8c0b77a0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fdfa8111a50 tx=0x7fdf90005f90 comp rx=0 tx=0).stop 2024-09-11T00:04:44.206 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:44.204+0000 7fdfad400640 1 -- 172.21.15.12:0/395400598 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fdfa8075540 msgr2=0x7fdfa819b4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:04:44.206 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:44.204+0000 7fdfad400640 1 --2- 172.21.15.12:0/395400598 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fdfa8075540 0x7fdfa819b4e0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fdf98005940 tx=0x7fdf980040c0 comp rx=0 tx=0).stop 2024-09-11T00:04:44.206 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:44.204+0000 7fdfad400640 1 -- 172.21.15.12:0/395400598 shutdown_connections 2024-09-11T00:04:44.207 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:44.204+0000 7fdfad400640 1 --2- 172.21.15.12:0/395400598 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fdf8c0b52e0 0x7fdf8c0b77a0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:04:44.207 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:44.204+0000 7fdfad400640 1 --2- 172.21.15.12:0/395400598 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdfa810f3e0 0x7fdfa81a0dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:04:44.207 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:44.204+0000 7fdfad400640 1 --2- 172.21.15.12:0/395400598 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdfa8075f40 0x7fdfa819ba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:04:44.207 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:44.204+0000 7fdfad400640 1 --2- 172.21.15.12:0/395400598 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fdfa8075540 0x7fdfa819b4e0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:04:44.207 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:44.204+0000 7fdfad400640 1 -- 172.21.15.12:0/395400598 >> 172.21.15.12:0/395400598 conn(0x7fdfa80fe830 msgr2=0x7fdfa80ff430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:04:44.207 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:44.204+0000 7fdfad400640 1 -- 172.21.15.12:0/395400598 shutdown_connections 2024-09-11T00:04:44.207 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:44.204+0000 7fdfad400640 1 -- 172.21.15.12:0/395400598 wait complete. 2024-09-11T00:04:44.532 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:04:44 smithi165 ceph-mon[35356]: from='client.? 172.21.15.12:0/395400598' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} : dispatch 2024-09-11T00:04:44.532 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:04:44 smithi165 ceph-mon[35356]: from='client.? ' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} : dispatch 2024-09-11T00:04:44.562 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:04:44 smithi012 ceph-mon[30333]: from='client.? 172.21.15.12:0/395400598' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} : dispatch 2024-09-11T00:04:44.563 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:04:44 smithi012 ceph-mon[30333]: from='client.? ' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} : dispatch 2024-09-11T00:04:44.580 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:04:44 smithi178 ceph-mon[35919]: from='client.? 172.21.15.12:0/395400598' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} : dispatch 2024-09-11T00:04:44.580 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:04:44 smithi178 ceph-mon[35919]: from='client.? ' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} : dispatch 2024-09-11T00:04:44.962 DEBUG:tasks.cephfs.filesystem:fs_config: {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd'], 'max_mds': 5, 'session_timeout': 300, 'standby_replay': True, 'subvols': {'create': 2, 'subvol_options': '--namespace-isolated --size 25000000000'}} 2024-09-11T00:04:44.962 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid ff3ad74a-6fcf-11ef-bcea-c7b262605968 -- ceph fs set cephfs max_mds 5 2024-09-11T00:04:45.339 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:04:45.479 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:04:45 smithi012 ceph-mon[30333]: pgmap v324: 129 pgs: 129 active+clean; 581 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-09-11T00:04:45.479 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:04:45 smithi012 ceph-mon[30333]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-09-11T00:04:45.479 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:04:45 smithi012 ceph-mon[30333]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-11T00:04:45.532 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:04:45 smithi165 ceph-mon[35356]: pgmap v324: 129 pgs: 129 active+clean; 581 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-09-11T00:04:45.532 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:04:45 smithi165 ceph-mon[35356]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-09-11T00:04:45.532 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:04:45 smithi165 ceph-mon[35356]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-11T00:04:45.580 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:04:45 smithi178 ceph-mon[35919]: pgmap v324: 129 pgs: 129 active+clean; 581 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-09-11T00:04:45.581 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:04:45 smithi178 ceph-mon[35919]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-09-11T00:04:45.581 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:04:45 smithi178 ceph-mon[35919]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-11T00:04:45.895 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:45.883+0000 7f22b3200640 1 -- 172.21.15.12:0/45045223 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f22b4108620 msgr2=0x7f22b4108a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:04:45.895 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:45.883+0000 7f22b3200640 1 --2- 172.21.15.12:0/45045223 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f22b4108620 0x7f22b4108a60 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f229c009900 tx=0x7f229c02a410 comp rx=0 tx=0).stop 2024-09-11T00:04:45.895 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:45.883+0000 7f22b3200640 1 -- 172.21.15.12:0/45045223 shutdown_connections 2024-09-11T00:04:45.895 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:45.883+0000 7f22b3200640 1 --2- 172.21.15.12:0/45045223 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f22b4108fa0 0x7f22b410f7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:04:45.895 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:45.883+0000 7f22b3200640 1 --2- 172.21.15.12:0/45045223 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f22b4108620 0x7f22b4108a60 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:04:45.895 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:45.883+0000 7f22b3200640 1 --2- 172.21.15.12:0/45045223 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f22b4102c10 0x7f22b4102ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:04:45.895 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:45.883+0000 7f22b3200640 1 -- 172.21.15.12:0/45045223 >> 172.21.15.12:0/45045223 conn(0x7f22b40fe660 msgr2=0x7f22b4100a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:04:45.895 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:45.883+0000 7f22b3200640 1 -- 172.21.15.12:0/45045223 shutdown_connections 2024-09-11T00:04:45.895 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:45.884+0000 7f22b3200640 1 -- 172.21.15.12:0/45045223 wait complete. 2024-09-11T00:04:45.895 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:45.884+0000 7f22b3200640 1 Processor -- start 2024-09-11T00:04:45.896 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:45.885+0000 7f22b3200640 1 -- start start 2024-09-11T00:04:45.896 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:45.885+0000 7f22b3200640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f22b4102c10 0x7f22b4199060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:04:45.896 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:45.885+0000 7f22b3200640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f22b4108620 0x7f22b41995a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:04:45.896 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:45.885+0000 7f22b3200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f22b4108fa0 0x7f22b419e940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:04:45.896 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:45.885+0000 7f22b3200640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f22b4112c70 con 0x7f22b4108620 2024-09-11T00:04:45.896 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:45.885+0000 7f22b3200640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f22b4112af0 con 0x7f22b4108fa0 2024-09-11T00:04:45.896 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:45.885+0000 7f22b3200640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f22b4112df0 con 0x7f22b4102c10 2024-09-11T00:04:45.896 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:45.885+0000 7f22abe00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f22b4108620 0x7f22b41995a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:04:45.896 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:45.885+0000 7f22abe00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f22b4108620 0x7f22b41995a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:43404/0 (socket says 172.21.15.12:43404) 2024-09-11T00:04:45.896 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:45.885+0000 7f22abe00640 1 -- 172.21.15.12:0/1197402060 learned_addr learned my addr 172.21.15.12:0/1197402060 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:04:45.896 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:45.885+0000 7f22b2800640 1 --2- 172.21.15.12:0/1197402060 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f22b4108fa0 0x7f22b419e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:04:45.896 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:45.885+0000 7f22b1e00640 1 --2- 172.21.15.12:0/1197402060 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f22b4102c10 0x7f22b4199060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:04:45.896 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:45.886+0000 7f22abe00640 1 -- 172.21.15.12:0/1197402060 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f22b4102c10 msgr2=0x7f22b4199060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:04:45.897 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:45.886+0000 7f22abe00640 1 --2- 172.21.15.12:0/1197402060 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f22b4102c10 0x7f22b4199060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:04:45.897 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:45.886+0000 7f22abe00640 1 -- 172.21.15.12:0/1197402060 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f22b4108fa0 msgr2=0x7f22b419e940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:04:45.897 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:45.886+0000 7f22abe00640 1 --2- 172.21.15.12:0/1197402060 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f22b4108fa0 0x7f22b419e940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:04:45.897 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:45.886+0000 7f22abe00640 1 -- 172.21.15.12:0/1197402060 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22b419f110 con 0x7f22b4108620 2024-09-11T00:04:45.897 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:45.886+0000 7f22abe00640 1 --2- 172.21.15.12:0/1197402060 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f22b4108620 0x7f22b41995a0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f229c005ab0 tx=0x7f229c003c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:04:45.897 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:45.886+0000 7f22aaa00640 1 -- 172.21.15.12:0/1197402060 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f229c03f070 con 0x7f22b4108620 2024-09-11T00:04:45.897 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:45.886+0000 7f22aaa00640 1 -- 172.21.15.12:0/1197402060 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f229c031d60 con 0x7f22b4108620 2024-09-11T00:04:45.897 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:45.886+0000 7f22aaa00640 1 -- 172.21.15.12:0/1197402060 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f229c035030 con 0x7f22b4108620 2024-09-11T00:04:45.897 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:45.886+0000 7f22b3200640 1 -- 172.21.15.12:0/1197402060 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22b419f3a0 con 0x7f22b4108620 2024-09-11T00:04:45.897 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:45.887+0000 7f22b3200640 1 -- 172.21.15.12:0/1197402060 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22b41a5ac0 con 0x7f22b4108620 2024-09-11T00:04:45.897 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:45.888+0000 7f22b3200640 1 -- 172.21.15.12:0/1197402060 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f22b4102ff0 con 0x7f22b4108620 2024-09-11T00:04:45.897 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:45.890+0000 7f22aaa00640 1 -- 172.21.15.12:0/1197402060 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 16) ==== 153250+0+0 (secure 0 0 0) 0x7f229c03b4f0 con 0x7f22b4108620 2024-09-11T00:04:45.897 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:45.892+0000 7f22aaa00640 1 --2- 172.21.15.12:0/1197402060 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f22980b5410 0x7f22980b78d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:04:45.898 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:45.892+0000 7f22aaa00640 1 -- 172.21.15.12:0/1197402060 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8802+0+0 (secure 0 0 0) 0x7f229c106e90 con 0x7f22b4108620 2024-09-11T00:04:45.898 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:45.892+0000 7f22b1e00640 1 --2- 172.21.15.12:0/1197402060 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f22980b5410 0x7f22980b78d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:04:45.898 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:45.893+0000 7f22b1e00640 1 --2- 172.21.15.12:0/1197402060 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f22980b5410 0x7f22980b78d0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f22a4005d00 tx=0x7f22a400b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:04:45.898 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:45.894+0000 7f22aaa00640 1 -- 172.21.15.12:0/1197402060 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f229c0cd460 con 0x7f22b4108620 2024-09-11T00:04:46.131 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:46.130+0000 7f22b3200640 1 -- 172.21.15.12:0/1197402060 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"} v 0) -- 0x7f22b41a5d20 con 0x7f22b4108620 2024-09-11T00:04:46.213 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:46.211+0000 7f22aaa00640 1 -- 172.21.15.12:0/1197402060 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]=0 v19) ==== 105+0+0 (secure 0 0 0) 0x7f229c0d1e90 con 0x7f22b4108620 2024-09-11T00:04:46.216 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:46.214+0000 7f22b3200640 1 -- 172.21.15.12:0/1197402060 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f22980b5410 msgr2=0x7f22980b78d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:04:46.216 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:46.214+0000 7f22b3200640 1 --2- 172.21.15.12:0/1197402060 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f22980b5410 0x7f22980b78d0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f22a4005d00 tx=0x7f22a400b410 comp rx=0 tx=0).stop 2024-09-11T00:04:46.216 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:46.214+0000 7f22b3200640 1 -- 172.21.15.12:0/1197402060 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f22b4108620 msgr2=0x7f22b41995a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:04:46.216 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:46.214+0000 7f22b3200640 1 --2- 172.21.15.12:0/1197402060 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f22b4108620 0x7f22b41995a0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f229c005ab0 tx=0x7f229c003c80 comp rx=0 tx=0).stop 2024-09-11T00:04:46.217 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:46.214+0000 7f22b3200640 1 -- 172.21.15.12:0/1197402060 shutdown_connections 2024-09-11T00:04:46.217 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:46.214+0000 7f22b3200640 1 --2- 172.21.15.12:0/1197402060 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f22980b5410 0x7f22980b78d0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:04:46.217 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:46.214+0000 7f22b3200640 1 --2- 172.21.15.12:0/1197402060 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f22b4108fa0 0x7f22b419e940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:04:46.217 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:46.214+0000 7f22b3200640 1 --2- 172.21.15.12:0/1197402060 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f22b4108620 0x7f22b41995a0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:04:46.217 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:46.214+0000 7f22b3200640 1 --2- 172.21.15.12:0/1197402060 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f22b4102c10 0x7f22b4199060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:04:46.217 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:46.214+0000 7f22b3200640 1 -- 172.21.15.12:0/1197402060 >> 172.21.15.12:0/1197402060 conn(0x7f22b40fe660 msgr2=0x7f22b4100a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:04:46.217 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:46.214+0000 7f22b3200640 1 -- 172.21.15.12:0/1197402060 shutdown_connections 2024-09-11T00:04:46.217 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:46.215+0000 7f22b3200640 1 -- 172.21.15.12:0/1197402060 wait complete. 2024-09-11T00:04:46.532 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:04:46 smithi165 ceph-mon[35356]: from='client.? 172.21.15.12:0/1197402060' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"} : dispatch 2024-09-11T00:04:46.562 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:04:46 smithi012 ceph-mon[30333]: from='client.? 172.21.15.12:0/1197402060' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"} : dispatch 2024-09-11T00:04:46.580 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:04:46 smithi178 ceph-mon[35919]: from='client.? 172.21.15.12:0/1197402060' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"} : dispatch 2024-09-11T00:04:46.946 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid ff3ad74a-6fcf-11ef-bcea-c7b262605968 -- ceph fs set cephfs allow_standby_replay true 2024-09-11T00:04:47.333 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:04:47.472 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:04:47 smithi012 ceph-mon[30333]: pgmap v325: 129 pgs: 129 active+clean; 581 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-09-11T00:04:47.472 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:04:47 smithi012 ceph-mon[30333]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-09-11T00:04:47.472 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:04:47 smithi012 ceph-mon[30333]: from='client.? 172.21.15.12:0/1197402060' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-09-11T00:04:47.472 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:04:47 smithi012 ceph-mon[30333]: daemon mds.g assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-09-11T00:04:47.472 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:04:47 smithi012 ceph-mon[30333]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-11T00:04:47.472 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:04:47 smithi012 ceph-mon[30333]: fsmap cephfs:2 {0=b=up:active,1=g=up:creating} 10 up:standby 2024-09-11T00:04:47.472 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:04:47 smithi012 ceph-mon[30333]: daemon mds.g is now active in filesystem cephfs as rank 1 2024-09-11T00:04:47.532 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:04:47 smithi165 ceph-mon[35356]: pgmap v325: 129 pgs: 129 active+clean; 581 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-09-11T00:04:47.532 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:04:47 smithi165 ceph-mon[35356]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-09-11T00:04:47.533 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:04:47 smithi165 ceph-mon[35356]: from='client.? 172.21.15.12:0/1197402060' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-09-11T00:04:47.533 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:04:47 smithi165 ceph-mon[35356]: daemon mds.g assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-09-11T00:04:47.533 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:04:47 smithi165 ceph-mon[35356]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-11T00:04:47.533 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:04:47 smithi165 ceph-mon[35356]: fsmap cephfs:2 {0=b=up:active,1=g=up:creating} 10 up:standby 2024-09-11T00:04:47.533 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:04:47 smithi165 ceph-mon[35356]: daemon mds.g is now active in filesystem cephfs as rank 1 2024-09-11T00:04:47.581 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:04:47 smithi178 ceph-mon[35919]: pgmap v325: 129 pgs: 129 active+clean; 581 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-09-11T00:04:47.581 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:04:47 smithi178 ceph-mon[35919]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-09-11T00:04:47.581 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:04:47 smithi178 ceph-mon[35919]: from='client.? 172.21.15.12:0/1197402060' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-09-11T00:04:47.581 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:04:47 smithi178 ceph-mon[35919]: daemon mds.g assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-09-11T00:04:47.581 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:04:47 smithi178 ceph-mon[35919]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-11T00:04:47.581 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:04:47 smithi178 ceph-mon[35919]: fsmap cephfs:2 {0=b=up:active,1=g=up:creating} 10 up:standby 2024-09-11T00:04:47.581 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:04:47 smithi178 ceph-mon[35919]: daemon mds.g is now active in filesystem cephfs as rank 1 2024-09-11T00:04:47.868 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:47.865+0000 7ff907400640 1 -- 172.21.15.12:0/1516292108 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff9001087d0 msgr2=0x7ff900108c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:04:47.868 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:47.865+0000 7ff907400640 1 --2- 172.21.15.12:0/1516292108 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff9001087d0 0x7ff900108c10 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7ff8e8009900 tx=0x7ff8e802a3f0 comp rx=0 tx=0).stop 2024-09-11T00:04:47.868 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:47.865+0000 7ff907400640 1 -- 172.21.15.12:0/1516292108 shutdown_connections 2024-09-11T00:04:47.868 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:47.865+0000 7ff907400640 1 --2- 172.21.15.12:0/1516292108 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff900109150 0x7ff90010f950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:04:47.868 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:47.865+0000 7ff907400640 1 --2- 172.21.15.12:0/1516292108 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff9001087d0 0x7ff900108c10 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:04:47.868 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:47.865+0000 7ff907400640 1 --2- 172.21.15.12:0/1516292108 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff900102dc0 0x7ff9001031a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:04:47.868 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:47.865+0000 7ff907400640 1 -- 172.21.15.12:0/1516292108 >> 172.21.15.12:0/1516292108 conn(0x7ff9000fe830 msgr2=0x7ff900100c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:04:47.868 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:47.865+0000 7ff907400640 1 -- 172.21.15.12:0/1516292108 shutdown_connections 2024-09-11T00:04:47.869 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:47.866+0000 7ff907400640 1 -- 172.21.15.12:0/1516292108 wait complete. 2024-09-11T00:04:47.874 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:47.866+0000 7ff907400640 1 Processor -- start 2024-09-11T00:04:47.874 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:47.867+0000 7ff907400640 1 -- start start 2024-09-11T00:04:47.874 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:47.867+0000 7ff907400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff900102dc0 0x7ff90019b490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:04:47.874 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:47.867+0000 7ff907400640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff9001087d0 0x7ff90019b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:04:47.874 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:47.867+0000 7ff907400640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff900109150 0x7ff9001a0d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:04:47.874 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:47.867+0000 7ff907400640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7ff900112da0 con 0x7ff9001087d0 2024-09-11T00:04:47.874 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:47.867+0000 7ff907400640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7ff900112c20 con 0x7ff900102dc0 2024-09-11T00:04:47.875 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:47.867+0000 7ff907400640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7ff900112f20 con 0x7ff900109150 2024-09-11T00:04:47.875 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:47.867+0000 7ff8f7e00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff9001087d0 0x7ff90019b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:04:47.875 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:47.867+0000 7ff904a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff900102dc0 0x7ff90019b490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:04:47.875 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:47.867+0000 7ff8f7e00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff9001087d0 0x7ff90019b9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:43422/0 (socket says 172.21.15.12:43422) 2024-09-11T00:04:47.875 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:47.867+0000 7ff8f7e00640 1 -- 172.21.15.12:0/3215912542 learned_addr learned my addr 172.21.15.12:0/3215912542 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:04:47.875 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:47.867+0000 7ff904a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff900102dc0 0x7ff90019b490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.12:56946/0 (socket says 172.21.15.12:56946) 2024-09-11T00:04:47.875 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:47.867+0000 7ff905400640 1 --2- 172.21.15.12:0/3215912542 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff900109150 0x7ff9001a0d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:04:47.875 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:47.868+0000 7ff8f7e00640 1 -- 172.21.15.12:0/3215912542 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff900109150 msgr2=0x7ff9001a0d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:04:47.875 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:47.868+0000 7ff8f7e00640 1 --2- 172.21.15.12:0/3215912542 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff900109150 0x7ff9001a0d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:04:47.875 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:47.868+0000 7ff8f7e00640 1 -- 172.21.15.12:0/3215912542 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff900102dc0 msgr2=0x7ff90019b490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:04:47.875 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:47.868+0000 7ff8f7e00640 1 --2- 172.21.15.12:0/3215912542 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff900102dc0 0x7ff90019b490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:04:47.875 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:47.868+0000 7ff8f7e00640 1 -- 172.21.15.12:0/3215912542 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9001a1540 con 0x7ff9001087d0 2024-09-11T00:04:47.875 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:47.868+0000 7ff904a00640 1 --2- 172.21.15.12:0/3215912542 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff900102dc0 0x7ff90019b490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:04:47.876 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:47.868+0000 7ff8f7e00640 1 --2- 172.21.15.12:0/3215912542 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff9001087d0 0x7ff90019b9d0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7ff8e8005940 tx=0x7ff8e8004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:04:47.876 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:47.869+0000 7ff8fe000640 1 -- 172.21.15.12:0/3215912542 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8e803f030 con 0x7ff9001087d0 2024-09-11T00:04:47.876 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:47.869+0000 7ff8fe000640 1 -- 172.21.15.12:0/3215912542 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff8e8003aa0 con 0x7ff9001087d0 2024-09-11T00:04:47.876 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:47.869+0000 7ff907400640 1 -- 172.21.15.12:0/3215912542 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff9001a17d0 con 0x7ff9001087d0 2024-09-11T00:04:47.876 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:47.869+0000 7ff8fe000640 1 -- 172.21.15.12:0/3215912542 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8e8035070 con 0x7ff9001087d0 2024-09-11T00:04:47.876 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:47.869+0000 7ff907400640 1 -- 172.21.15.12:0/3215912542 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9001a7ef0 con 0x7ff9001087d0 2024-09-11T00:04:47.876 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:47.871+0000 7ff907400640 1 -- 172.21.15.12:0/3215912542 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff9001031a0 con 0x7ff9001087d0 2024-09-11T00:04:47.877 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:47.873+0000 7ff8fe000640 1 -- 172.21.15.12:0/3215912542 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 16) ==== 153250+0+0 (secure 0 0 0) 0x7ff8e8003c50 con 0x7ff9001087d0 2024-09-11T00:04:47.877 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:47.874+0000 7ff8fe000640 1 --2- 172.21.15.12:0/3215912542 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7ff8e40b52e0 0x7ff8e40b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:04:47.877 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:47.874+0000 7ff8fe000640 1 -- 172.21.15.12:0/3215912542 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8802+0+0 (secure 0 0 0) 0x7ff8e81062e0 con 0x7ff9001087d0 2024-09-11T00:04:47.877 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:47.874+0000 7ff904a00640 1 --2- 172.21.15.12:0/3215912542 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7ff8e40b52e0 0x7ff8e40b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:04:47.877 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:47.875+0000 7ff904a00640 1 --2- 172.21.15.12:0/3215912542 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7ff8e40b52e0 0x7ff8e40b77a0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7ff8f0005d00 tx=0x7ff8f000b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:04:47.880 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:47.878+0000 7ff8fe000640 1 -- 172.21.15.12:0/3215912542 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7ff8e80cc910 con 0x7ff9001087d0 2024-09-11T00:04:48.117 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:48.116+0000 7ff907400640 1 -- 172.21.15.12:0/3215912542 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) -- 0x7ff90010f340 con 0x7ff9001087d0 2024-09-11T00:04:48.224 INFO:journalctl@ceph.mds.b.smithi165.stdout:Sep 11 00:04:47 smithi165 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-mds-b[65581]: 2024-09-11T00:04:47.926+0000 7fcd51d71640 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2024-09-11T00:04:48.228 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:48.226+0000 7ff8fe000640 1 -- 172.21.15.12:0/3215912542 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v23) ==== 121+0+0 (secure 0 0 0) 0x7ff8e80d1340 con 0x7ff9001087d0 2024-09-11T00:04:48.231 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:48.229+0000 7ff907400640 1 -- 172.21.15.12:0/3215912542 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7ff8e40b52e0 msgr2=0x7ff8e40b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:04:48.231 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:48.229+0000 7ff907400640 1 --2- 172.21.15.12:0/3215912542 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7ff8e40b52e0 0x7ff8e40b77a0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7ff8f0005d00 tx=0x7ff8f000b410 comp rx=0 tx=0).stop 2024-09-11T00:04:48.231 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:48.229+0000 7ff907400640 1 -- 172.21.15.12:0/3215912542 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff9001087d0 msgr2=0x7ff90019b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:04:48.231 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:48.229+0000 7ff907400640 1 --2- 172.21.15.12:0/3215912542 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff9001087d0 0x7ff90019b9d0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7ff8e8005940 tx=0x7ff8e8004270 comp rx=0 tx=0).stop 2024-09-11T00:04:48.232 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:48.229+0000 7ff907400640 1 -- 172.21.15.12:0/3215912542 shutdown_connections 2024-09-11T00:04:48.232 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:48.229+0000 7ff907400640 1 --2- 172.21.15.12:0/3215912542 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7ff8e40b52e0 0x7ff8e40b77a0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:04:48.232 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:48.229+0000 7ff907400640 1 --2- 172.21.15.12:0/3215912542 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff900109150 0x7ff9001a0d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:04:48.232 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:48.229+0000 7ff907400640 1 --2- 172.21.15.12:0/3215912542 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff9001087d0 0x7ff90019b9d0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:04:48.232 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:48.229+0000 7ff907400640 1 --2- 172.21.15.12:0/3215912542 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff900102dc0 0x7ff90019b490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:04:48.232 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:48.229+0000 7ff907400640 1 -- 172.21.15.12:0/3215912542 >> 172.21.15.12:0/3215912542 conn(0x7ff9000fe830 msgr2=0x7ff900105db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:04:48.232 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:48.229+0000 7ff907400640 1 -- 172.21.15.12:0/3215912542 shutdown_connections 2024-09-11T00:04:48.232 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:48.230+0000 7ff907400640 1 -- 172.21.15.12:0/3215912542 wait complete. 2024-09-11T00:04:48.532 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:04:48 smithi165 ceph-mon[35356]: mds.? [v2:172.21.15.12:6838/3124093009,v1:172.21.15.12:6839/3124093009] up:active 2024-09-11T00:04:48.532 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:04:48 smithi165 ceph-mon[35356]: daemon mds.h assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-09-11T00:04:48.533 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:04:48 smithi165 ceph-mon[35356]: fsmap cephfs:2 {0=b=up:active,1=g=up:active} 10 up:standby 2024-09-11T00:04:48.533 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:04:48 smithi165 ceph-mon[35356]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=h=up:creating} 9 up:standby 2024-09-11T00:04:48.533 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:04:48 smithi165 ceph-mon[35356]: daemon mds.h is now active in filesystem cephfs as rank 2 2024-09-11T00:04:48.533 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:04:48 smithi165 ceph-mon[35356]: from='client.? 172.21.15.12:0/3215912542' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} : dispatch 2024-09-11T00:04:48.562 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:04:48 smithi012 ceph-mon[30333]: mds.? [v2:172.21.15.12:6838/3124093009,v1:172.21.15.12:6839/3124093009] up:active 2024-09-11T00:04:48.563 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:04:48 smithi012 ceph-mon[30333]: daemon mds.h assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-09-11T00:04:48.563 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:04:48 smithi012 ceph-mon[30333]: fsmap cephfs:2 {0=b=up:active,1=g=up:active} 10 up:standby 2024-09-11T00:04:48.563 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:04:48 smithi012 ceph-mon[30333]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=h=up:creating} 9 up:standby 2024-09-11T00:04:48.563 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:04:48 smithi012 ceph-mon[30333]: daemon mds.h is now active in filesystem cephfs as rank 2 2024-09-11T00:04:48.563 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:04:48 smithi012 ceph-mon[30333]: from='client.? 172.21.15.12:0/3215912542' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} : dispatch 2024-09-11T00:04:48.580 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:04:48 smithi178 ceph-mon[35919]: mds.? [v2:172.21.15.12:6838/3124093009,v1:172.21.15.12:6839/3124093009] up:active 2024-09-11T00:04:48.581 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:04:48 smithi178 ceph-mon[35919]: daemon mds.h assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-09-11T00:04:48.581 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:04:48 smithi178 ceph-mon[35919]: fsmap cephfs:2 {0=b=up:active,1=g=up:active} 10 up:standby 2024-09-11T00:04:48.581 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:04:48 smithi178 ceph-mon[35919]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=h=up:creating} 9 up:standby 2024-09-11T00:04:48.581 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:04:48 smithi178 ceph-mon[35919]: daemon mds.h is now active in filesystem cephfs as rank 2 2024-09-11T00:04:48.581 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:04:48 smithi178 ceph-mon[35919]: from='client.? 172.21.15.12:0/3215912542' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} : dispatch 2024-09-11T00:04:48.987 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid ff3ad74a-6fcf-11ef-bcea-c7b262605968 -- ceph fs set cephfs session_timeout 300 2024-09-11T00:04:49.374 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:04:49.499 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:04:49 smithi165 ceph-mon[35356]: pgmap v326: 129 pgs: 129 active+clean; 581 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 0 op/s 2024-09-11T00:04:49.499 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:04:49 smithi165 ceph-mon[35356]: from='client.? 172.21.15.12:0/3215912542' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-09-11T00:04:49.500 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:04:49 smithi165 ceph-mon[35356]: mds.? [v2:172.21.15.165:6836/6215088,v1:172.21.15.165:6837/6215088] up:active 2024-09-11T00:04:49.500 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:04:49 smithi165 ceph-mon[35356]: daemon mds.i assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-09-11T00:04:49.500 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:04:49 smithi165 ceph-mon[35356]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=h=up:active} 9 up:standby 2024-09-11T00:04:49.500 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:04:49 smithi165 ceph-mon[35356]: fsmap cephfs:4 {0=b=up:active,1=g=up:active,2=h=up:active,3=i=up:creating} 8 up:standby 2024-09-11T00:04:49.500 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:04:49 smithi165 ceph-mon[35356]: daemon mds.i is now active in filesystem cephfs as rank 3 2024-09-11T00:04:49.513 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:04:49 smithi012 ceph-mon[30333]: pgmap v326: 129 pgs: 129 active+clean; 581 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 0 op/s 2024-09-11T00:04:49.516 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:04:49 smithi012 ceph-mon[30333]: from='client.? 172.21.15.12:0/3215912542' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-09-11T00:04:49.516 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:04:49 smithi012 ceph-mon[30333]: mds.? [v2:172.21.15.165:6836/6215088,v1:172.21.15.165:6837/6215088] up:active 2024-09-11T00:04:49.516 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:04:49 smithi012 ceph-mon[30333]: daemon mds.i assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-09-11T00:04:49.516 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:04:49 smithi012 ceph-mon[30333]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=h=up:active} 9 up:standby 2024-09-11T00:04:49.516 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:04:49 smithi012 ceph-mon[30333]: fsmap cephfs:4 {0=b=up:active,1=g=up:active,2=h=up:active,3=i=up:creating} 8 up:standby 2024-09-11T00:04:49.517 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:04:49 smithi012 ceph-mon[30333]: daemon mds.i is now active in filesystem cephfs as rank 3 2024-09-11T00:04:49.580 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:04:49 smithi178 ceph-mon[35919]: pgmap v326: 129 pgs: 129 active+clean; 581 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 0 op/s 2024-09-11T00:04:49.581 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:04:49 smithi178 ceph-mon[35919]: from='client.? 172.21.15.12:0/3215912542' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-09-11T00:04:49.581 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:04:49 smithi178 ceph-mon[35919]: mds.? [v2:172.21.15.165:6836/6215088,v1:172.21.15.165:6837/6215088] up:active 2024-09-11T00:04:49.581 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:04:49 smithi178 ceph-mon[35919]: daemon mds.i assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-09-11T00:04:49.581 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:04:49 smithi178 ceph-mon[35919]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=h=up:active} 9 up:standby 2024-09-11T00:04:49.581 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:04:49 smithi178 ceph-mon[35919]: fsmap cephfs:4 {0=b=up:active,1=g=up:active,2=h=up:active,3=i=up:creating} 8 up:standby 2024-09-11T00:04:49.581 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:04:49 smithi178 ceph-mon[35919]: daemon mds.i is now active in filesystem cephfs as rank 3 2024-09-11T00:04:49.953 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:49.950+0000 7f4c3ac00640 1 -- 172.21.15.12:0/4164793955 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4c34102c10 msgr2=0x7f4c34102ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:04:49.953 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:49.950+0000 7f4c3ac00640 1 --2- 172.21.15.12:0/4164793955 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4c34102c10 0x7f4c34102ff0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f4c28009900 tx=0x7f4c2802a3f0 comp rx=0 tx=0).stop 2024-09-11T00:04:49.953 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:49.950+0000 7f4c3ac00640 1 -- 172.21.15.12:0/4164793955 shutdown_connections 2024-09-11T00:04:49.953 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:49.950+0000 7f4c3ac00640 1 --2- 172.21.15.12:0/4164793955 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4c34108fa0 0x7f4c3410f7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:04:49.953 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:49.950+0000 7f4c3ac00640 1 --2- 172.21.15.12:0/4164793955 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4c34108620 0x7f4c34108a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:04:49.953 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:49.950+0000 7f4c3ac00640 1 --2- 172.21.15.12:0/4164793955 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4c34102c10 0x7f4c34102ff0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:04:49.953 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:49.950+0000 7f4c3ac00640 1 -- 172.21.15.12:0/4164793955 >> 172.21.15.12:0/4164793955 conn(0x7f4c340fe660 msgr2=0x7f4c34100a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:04:49.954 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:49.950+0000 7f4c3ac00640 1 -- 172.21.15.12:0/4164793955 shutdown_connections 2024-09-11T00:04:49.954 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:49.950+0000 7f4c3ac00640 1 -- 172.21.15.12:0/4164793955 wait complete. 2024-09-11T00:04:49.956 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:49.951+0000 7f4c3ac00640 1 Processor -- start 2024-09-11T00:04:49.956 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:49.951+0000 7f4c3ac00640 1 -- start start 2024-09-11T00:04:49.956 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:49.952+0000 7f4c3ac00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4c34102c10 0x7f4c340754a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:04:49.956 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:49.952+0000 7f4c3ac00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4c34108620 0x7f4c340759e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:04:49.956 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:49.952+0000 7f4c3ac00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4c34108fa0 0x7f4c34075f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:04:49.956 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:49.952+0000 7f4c3ac00640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f4c34112c60 con 0x7f4c34102c10 2024-09-11T00:04:49.957 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:49.952+0000 7f4c3ac00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f4c34112ae0 con 0x7f4c34108620 2024-09-11T00:04:49.957 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:49.952+0000 7f4c3ac00640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f4c34112de0 con 0x7f4c34108fa0 2024-09-11T00:04:49.957 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:49.952+0000 7f4c39800640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4c34102c10 0x7f4c340754a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:04:49.957 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:49.952+0000 7f4c38e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4c34108620 0x7f4c340759e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:04:49.957 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:49.952+0000 7f4c39800640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4c34102c10 0x7f4c340754a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:40550/0 (socket says 172.21.15.12:40550) 2024-09-11T00:04:49.957 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:49.952+0000 7f4c39800640 1 -- 172.21.15.12:0/2049696826 learned_addr learned my addr 172.21.15.12:0/2049696826 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:04:49.957 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:49.952+0000 7f4c3a200640 1 --2- 172.21.15.12:0/2049696826 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4c34108fa0 0x7f4c34075f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:04:49.957 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:49.952+0000 7f4c39800640 1 -- 172.21.15.12:0/2049696826 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4c34108fa0 msgr2=0x7f4c34075f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:04:49.958 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:49.952+0000 7f4c39800640 1 --2- 172.21.15.12:0/2049696826 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4c34108fa0 0x7f4c34075f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:04:49.958 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:49.952+0000 7f4c39800640 1 -- 172.21.15.12:0/2049696826 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4c34108620 msgr2=0x7f4c340759e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:04:49.958 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:49.952+0000 7f4c39800640 1 --2- 172.21.15.12:0/2049696826 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4c34108620 0x7f4c340759e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:04:49.958 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:49.952+0000 7f4c39800640 1 -- 172.21.15.12:0/2049696826 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c341a74a0 con 0x7f4c34102c10 2024-09-11T00:04:49.958 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:49.953+0000 7f4c39800640 1 --2- 172.21.15.12:0/2049696826 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4c34102c10 0x7f4c340754a0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f4c28005940 tx=0x7f4c280040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:04:49.958 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:49.953+0000 7f4c32000640 1 -- 172.21.15.12:0/2049696826 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c2803f030 con 0x7f4c34102c10 2024-09-11T00:04:49.958 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:49.953+0000 7f4c32000640 1 -- 172.21.15.12:0/2049696826 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4c280044c0 con 0x7f4c34102c10 2024-09-11T00:04:49.959 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:49.953+0000 7f4c32000640 1 -- 172.21.15.12:0/2049696826 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c28035070 con 0x7f4c34102c10 2024-09-11T00:04:49.959 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:49.953+0000 7f4c3ac00640 1 -- 172.21.15.12:0/2049696826 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4c341a7730 con 0x7f4c34102c10 2024-09-11T00:04:49.959 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:49.953+0000 7f4c3ac00640 1 -- 172.21.15.12:0/2049696826 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4c341a7bc0 con 0x7f4c34102c10 2024-09-11T00:04:49.959 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:49.955+0000 7f4c3ac00640 1 -- 172.21.15.12:0/2049696826 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4c34102ff0 con 0x7f4c34102c10 2024-09-11T00:04:49.960 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:49.957+0000 7f4c32000640 1 -- 172.21.15.12:0/2049696826 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 16) ==== 153250+0+0 (secure 0 0 0) 0x7f4c2803ba20 con 0x7f4c34102c10 2024-09-11T00:04:49.960 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:49.958+0000 7f4c32000640 1 --2- 172.21.15.12:0/2049696826 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f4c1c0b5410 0x7f4c1c0b78d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:04:49.961 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:49.959+0000 7f4c32000640 1 -- 172.21.15.12:0/2049696826 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8802+0+0 (secure 0 0 0) 0x7f4c28106640 con 0x7f4c34102c10 2024-09-11T00:04:49.961 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:49.959+0000 7f4c38e00640 1 --2- 172.21.15.12:0/2049696826 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f4c1c0b5410 0x7f4c1c0b78d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:04:49.961 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:49.959+0000 7f4c38e00640 1 --2- 172.21.15.12:0/2049696826 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f4c1c0b5410 0x7f4c1c0b78d0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f4c34076f90 tx=0x7f4c20005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:04:49.962 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:49.960+0000 7f4c32000640 1 -- 172.21.15.12:0/2049696826 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f4c280ccc10 con 0x7f4c34102c10 2024-09-11T00:04:50.198 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:50.196+0000 7f4c3ac00640 1 -- 172.21.15.12:0/2049696826 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) -- 0x7f4c3402e5c0 con 0x7f4c34102c10 2024-09-11T00:04:50.245 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:50.243+0000 7f4c32000640 1 -- 172.21.15.12:0/2049696826 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v27) ==== 115+0+0 (secure 0 0 0) 0x7f4c280d1640 con 0x7f4c34102c10 2024-09-11T00:04:50.247 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:50.245+0000 7f4c3ac00640 1 -- 172.21.15.12:0/2049696826 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f4c1c0b5410 msgr2=0x7f4c1c0b78d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:04:50.248 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:50.245+0000 7f4c3ac00640 1 --2- 172.21.15.12:0/2049696826 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f4c1c0b5410 0x7f4c1c0b78d0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f4c34076f90 tx=0x7f4c20005f90 comp rx=0 tx=0).stop 2024-09-11T00:04:50.248 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:50.245+0000 7f4c3ac00640 1 -- 172.21.15.12:0/2049696826 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4c34102c10 msgr2=0x7f4c340754a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:04:50.248 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:50.245+0000 7f4c3ac00640 1 --2- 172.21.15.12:0/2049696826 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4c34102c10 0x7f4c340754a0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f4c28005940 tx=0x7f4c280040c0 comp rx=0 tx=0).stop 2024-09-11T00:04:50.248 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:50.246+0000 7f4c3ac00640 1 -- 172.21.15.12:0/2049696826 shutdown_connections 2024-09-11T00:04:50.248 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:50.246+0000 7f4c3ac00640 1 --2- 172.21.15.12:0/2049696826 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f4c1c0b5410 0x7f4c1c0b78d0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:04:50.248 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:50.246+0000 7f4c3ac00640 1 --2- 172.21.15.12:0/2049696826 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4c34108fa0 0x7f4c34075f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:04:50.248 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:50.246+0000 7f4c3ac00640 1 --2- 172.21.15.12:0/2049696826 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4c34108620 0x7f4c340759e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:04:50.248 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:50.246+0000 7f4c3ac00640 1 --2- 172.21.15.12:0/2049696826 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4c34102c10 0x7f4c340754a0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:04:50.248 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:50.246+0000 7f4c3ac00640 1 -- 172.21.15.12:0/2049696826 >> 172.21.15.12:0/2049696826 conn(0x7f4c340fe660 msgr2=0x7f4c34100a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:04:50.249 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:50.246+0000 7f4c3ac00640 1 -- 172.21.15.12:0/2049696826 shutdown_connections 2024-09-11T00:04:50.249 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:50.246+0000 7f4c3ac00640 1 -- 172.21.15.12:0/2049696826 wait complete. 2024-09-11T00:04:50.532 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:04:50 smithi165 ceph-mon[35356]: mds.? [v2:172.21.15.178:6838/70628097,v1:172.21.15.178:6839/70628097] up:active 2024-09-11T00:04:50.532 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:04:50 smithi165 ceph-mon[35356]: daemon mds.e assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-09-11T00:04:50.532 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:04:50 smithi165 ceph-mon[35356]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-09-11T00:04:50.532 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:04:50 smithi165 ceph-mon[35356]: Cluster is now healthy 2024-09-11T00:04:50.533 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:04:50 smithi165 ceph-mon[35356]: fsmap cephfs:4 {0=b=up:active,1=g=up:active,2=h=up:active,3=i=up:active} 8 up:standby 2024-09-11T00:04:50.533 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:04:50 smithi165 ceph-mon[35356]: fsmap cephfs:5 1 up:creating 7 up:standby 4 up:active 2024-09-11T00:04:50.533 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:04:50 smithi165 ceph-mon[35356]: daemon mds.e is now active in filesystem cephfs as rank 4 2024-09-11T00:04:50.533 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:04:50 smithi165 ceph-mon[35356]: from='client.? 172.21.15.12:0/2049696826' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} : dispatch 2024-09-11T00:04:50.563 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:04:50 smithi012 ceph-mon[30333]: mds.? [v2:172.21.15.178:6838/70628097,v1:172.21.15.178:6839/70628097] up:active 2024-09-11T00:04:50.563 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:04:50 smithi012 ceph-mon[30333]: daemon mds.e assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-09-11T00:04:50.563 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:04:50 smithi012 ceph-mon[30333]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-09-11T00:04:50.563 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:04:50 smithi012 ceph-mon[30333]: Cluster is now healthy 2024-09-11T00:04:50.563 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:04:50 smithi012 ceph-mon[30333]: fsmap cephfs:4 {0=b=up:active,1=g=up:active,2=h=up:active,3=i=up:active} 8 up:standby 2024-09-11T00:04:50.563 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:04:50 smithi012 ceph-mon[30333]: fsmap cephfs:5 1 up:creating 7 up:standby 4 up:active 2024-09-11T00:04:50.563 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:04:50 smithi012 ceph-mon[30333]: daemon mds.e is now active in filesystem cephfs as rank 4 2024-09-11T00:04:50.563 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:04:50 smithi012 ceph-mon[30333]: from='client.? 172.21.15.12:0/2049696826' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} : dispatch 2024-09-11T00:04:50.581 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:04:50 smithi178 ceph-mon[35919]: mds.? [v2:172.21.15.178:6838/70628097,v1:172.21.15.178:6839/70628097] up:active 2024-09-11T00:04:50.581 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:04:50 smithi178 ceph-mon[35919]: daemon mds.e assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-09-11T00:04:50.581 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:04:50 smithi178 ceph-mon[35919]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-09-11T00:04:50.581 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:04:50 smithi178 ceph-mon[35919]: Cluster is now healthy 2024-09-11T00:04:50.581 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:04:50 smithi178 ceph-mon[35919]: fsmap cephfs:4 {0=b=up:active,1=g=up:active,2=h=up:active,3=i=up:active} 8 up:standby 2024-09-11T00:04:50.581 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:04:50 smithi178 ceph-mon[35919]: fsmap cephfs:5 1 up:creating 7 up:standby 4 up:active 2024-09-11T00:04:50.581 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:04:50 smithi178 ceph-mon[35919]: daemon mds.e is now active in filesystem cephfs as rank 4 2024-09-11T00:04:50.581 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:04:50 smithi178 ceph-mon[35919]: from='client.? 172.21.15.12:0/2049696826' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} : dispatch 2024-09-11T00:04:50.951 DEBUG:tasks.cephfs.filesystem:Creating {'create': 2, 'subvol_options': '--namespace-isolated --size 25000000000'} subvols for filesystem 'cephfs' 2024-09-11T00:04:50.952 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid ff3ad74a-6fcf-11ef-bcea-c7b262605968 -- ceph fs subvolumegroup create cephfs qa 2024-09-11T00:04:51.329 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:04:51.532 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:04:51 smithi165 ceph-mon[35356]: pgmap v327: 129 pgs: 129 active+clean; 583 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s wr, 6 op/s 2024-09-11T00:04:51.532 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:04:51 smithi165 ceph-mon[35356]: from='client.? 172.21.15.12:0/2049696826' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-09-11T00:04:51.532 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:04:51 smithi165 ceph-mon[35356]: mds.? [v2:172.21.15.165:6834/2425395274,v1:172.21.15.165:6835/2425395274] up:active 2024-09-11T00:04:51.532 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:04:51 smithi165 ceph-mon[35356]: fsmap cephfs:5 7 up:standby 5 up:active 2024-09-11T00:04:51.532 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:04:51 smithi165 ceph-mon[35356]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T00:04:51.537 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:04:51 smithi012 ceph-mon[30333]: pgmap v327: 129 pgs: 129 active+clean; 583 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s wr, 6 op/s 2024-09-11T00:04:51.537 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:04:51 smithi012 ceph-mon[30333]: from='client.? 172.21.15.12:0/2049696826' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-09-11T00:04:51.537 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:04:51 smithi012 ceph-mon[30333]: mds.? [v2:172.21.15.165:6834/2425395274,v1:172.21.15.165:6835/2425395274] up:active 2024-09-11T00:04:51.537 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:04:51 smithi012 ceph-mon[30333]: fsmap cephfs:5 7 up:standby 5 up:active 2024-09-11T00:04:51.537 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:04:51 smithi012 ceph-mon[30333]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T00:04:51.581 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:04:51 smithi178 ceph-mon[35919]: pgmap v327: 129 pgs: 129 active+clean; 583 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s wr, 6 op/s 2024-09-11T00:04:51.581 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:04:51 smithi178 ceph-mon[35919]: from='client.? 172.21.15.12:0/2049696826' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-09-11T00:04:51.581 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:04:51 smithi178 ceph-mon[35919]: mds.? [v2:172.21.15.165:6834/2425395274,v1:172.21.15.165:6835/2425395274] up:active 2024-09-11T00:04:51.581 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:04:51 smithi178 ceph-mon[35919]: fsmap cephfs:5 7 up:standby 5 up:active 2024-09-11T00:04:51.581 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:04:51 smithi178 ceph-mon[35919]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T00:04:51.774 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:51.772+0000 7f2315e00640 1 -- 172.21.15.12:0/2938800010 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2310075e60 msgr2=0x7f23101135e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:04:51.775 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:51.772+0000 7f2315e00640 1 --2- 172.21.15.12:0/2938800010 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2310075e60 0x7f23101135e0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f22f8009900 tx=0x7f22f802a410 comp rx=0 tx=0).stop 2024-09-11T00:04:51.775 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:51.772+0000 7f2315e00640 1 -- 172.21.15.12:0/2938800010 shutdown_connections 2024-09-11T00:04:51.775 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:51.772+0000 7f2315e00640 1 --2- 172.21.15.12:0/2938800010 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2310113b20 0x7f2310115f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:04:51.775 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:51.772+0000 7f2315e00640 1 --2- 172.21.15.12:0/2938800010 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2310075e60 0x7f23101135e0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:04:51.775 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:51.772+0000 7f2315e00640 1 --2- 172.21.15.12:0/2938800010 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2310075540 0x7f2310075920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:04:51.776 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:51.772+0000 7f2315e00640 1 -- 172.21.15.12:0/2938800010 >> 172.21.15.12:0/2938800010 conn(0x7f23100fe830 msgr2=0x7f2310100c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:04:51.776 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:51.772+0000 7f2315e00640 1 -- 172.21.15.12:0/2938800010 shutdown_connections 2024-09-11T00:04:51.776 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:51.772+0000 7f2315e00640 1 -- 172.21.15.12:0/2938800010 wait complete. 2024-09-11T00:04:51.779 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:51.773+0000 7f2315e00640 1 Processor -- start 2024-09-11T00:04:51.779 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:51.773+0000 7f2315e00640 1 -- start start 2024-09-11T00:04:51.779 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:51.774+0000 7f2315e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2310075540 0x7f231019f8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:04:51.780 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:51.774+0000 7f2315e00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2310075e60 0x7f231019fe30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:04:51.780 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:51.774+0000 7f2315e00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2310113b20 0x7f23101a51d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:04:51.780 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:51.774+0000 7f2315e00640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f2310119400 con 0x7f2310113b20 2024-09-11T00:04:51.780 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:51.774+0000 7f2315e00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f2310119280 con 0x7f2310075540 2024-09-11T00:04:51.780 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:51.774+0000 7f2315e00640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f2310119580 con 0x7f2310075e60 2024-09-11T00:04:51.780 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:51.774+0000 7f230ea00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2310075540 0x7f231019f8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:04:51.781 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:51.774+0000 7f2306000640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2310075e60 0x7f231019fe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:04:51.781 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:51.774+0000 7f230ea00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2310075540 0x7f231019f8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.12:40620/0 (socket says 172.21.15.12:40620) 2024-09-11T00:04:51.781 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:51.774+0000 7f230ea00640 1 -- 172.21.15.12:0/2434343695 learned_addr learned my addr 172.21.15.12:0/2434343695 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:04:51.781 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:51.774+0000 7f230f400640 1 --2- 172.21.15.12:0/2434343695 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2310113b20 0x7f23101a51d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:04:51.781 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:51.775+0000 7f230ea00640 1 -- 172.21.15.12:0/2434343695 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2310075e60 msgr2=0x7f231019fe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:04:51.781 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:51.775+0000 7f230ea00640 1 --2- 172.21.15.12:0/2434343695 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2310075e60 0x7f231019fe30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:04:51.782 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:51.775+0000 7f230ea00640 1 -- 172.21.15.12:0/2434343695 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2310113b20 msgr2=0x7f23101a51d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:04:51.782 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:51.775+0000 7f230ea00640 1 --2- 172.21.15.12:0/2434343695 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2310113b20 0x7f23101a51d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:04:51.782 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:51.775+0000 7f230ea00640 1 -- 172.21.15.12:0/2434343695 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23101a59a0 con 0x7f2310075540 2024-09-11T00:04:51.782 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:51.775+0000 7f230ea00640 1 --2- 172.21.15.12:0/2434343695 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2310075540 0x7f231019f8f0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f230000e9f0 tx=0x7f230000eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:04:51.782 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:51.775+0000 7f2314a00640 1 -- 172.21.15.12:0/2434343695 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f230000cd80 con 0x7f2310075540 2024-09-11T00:04:51.782 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:51.775+0000 7f2315e00640 1 -- 172.21.15.12:0/2434343695 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f23101a5c90 con 0x7f2310075540 2024-09-11T00:04:51.782 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:51.775+0000 7f2314a00640 1 -- 172.21.15.12:0/2434343695 <== mon.2 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f23000108e0 con 0x7f2310075540 2024-09-11T00:04:51.783 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:51.775+0000 7f2314a00640 1 -- 172.21.15.12:0/2434343695 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2300018980 con 0x7f2310075540 2024-09-11T00:04:51.783 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:51.775+0000 7f2315e00640 1 -- 172.21.15.12:0/2434343695 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23101ac520 con 0x7f2310075540 2024-09-11T00:04:51.783 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:51.776+0000 7f2315e00640 1 -- 172.21.15.12:0/2434343695 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2310076a50 con 0x7f2310075540 2024-09-11T00:04:51.785 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:51.779+0000 7f2314a00640 1 -- 172.21.15.12:0/2434343695 <== mon.2 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 16) ==== 153250+0+0 (secure 0 0 0) 0x7f2300010430 con 0x7f2310075540 2024-09-11T00:04:51.785 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:51.780+0000 7f2314a00640 1 --2- 172.21.15.12:0/2434343695 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f22f40b52e0 0x7f22f40b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:04:51.785 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:51.780+0000 7f2314a00640 1 -- 172.21.15.12:0/2434343695 <== mon.2 v2:172.21.15.165:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8802+0+0 (secure 0 0 0) 0x7f23000e6670 con 0x7f2310075540 2024-09-11T00:04:51.785 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:51.781+0000 7f2306000640 1 --2- 172.21.15.12:0/2434343695 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f22f40b52e0 0x7f22f40b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:04:51.785 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:51.781+0000 7f2306000640 1 --2- 172.21.15.12:0/2434343695 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f22f40b52e0 0x7f22f40b77a0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f22f8005ab0 tx=0x7f22f8005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:04:51.785 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:51.782+0000 7f2314a00640 1 -- 172.21.15.12:0/2434343695 <== mon.2 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f23000acc40 con 0x7f2310075540 2024-09-11T00:04:52.019 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:52.017+0000 7f2315e00640 1 -- 172.21.15.12:0/2434343695 --> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f2310100b50 con 0x7f22f40b52e0 2024-09-11T00:04:52.052 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:52.050+0000 7f2314a00640 1 -- 172.21.15.12:0/2434343695 <== mgr.14150 v2:172.21.15.12:6800/3442939137 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f2310100b50 con 0x7f22f40b52e0 2024-09-11T00:04:52.055 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:52.052+0000 7f2315e00640 1 -- 172.21.15.12:0/2434343695 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f22f40b52e0 msgr2=0x7f22f40b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:04:52.055 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:52.052+0000 7f2315e00640 1 --2- 172.21.15.12:0/2434343695 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f22f40b52e0 0x7f22f40b77a0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f22f8005ab0 tx=0x7f22f8005a20 comp rx=0 tx=0).stop 2024-09-11T00:04:52.055 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:52.052+0000 7f2315e00640 1 -- 172.21.15.12:0/2434343695 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2310075540 msgr2=0x7f231019f8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:04:52.056 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:52.052+0000 7f2315e00640 1 --2- 172.21.15.12:0/2434343695 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2310075540 0x7f231019f8f0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f230000e9f0 tx=0x7f230000eeb0 comp rx=0 tx=0).stop 2024-09-11T00:04:52.056 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:52.052+0000 7f2315e00640 1 -- 172.21.15.12:0/2434343695 shutdown_connections 2024-09-11T00:04:52.056 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:52.052+0000 7f2315e00640 1 --2- 172.21.15.12:0/2434343695 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f22f40b52e0 0x7f22f40b77a0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:04:52.056 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:52.052+0000 7f2315e00640 1 --2- 172.21.15.12:0/2434343695 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2310113b20 0x7f23101a51d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:04:52.056 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:52.052+0000 7f2315e00640 1 --2- 172.21.15.12:0/2434343695 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2310075e60 0x7f231019fe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:04:52.056 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:52.052+0000 7f2315e00640 1 --2- 172.21.15.12:0/2434343695 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2310075540 0x7f231019f8f0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:04:52.056 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:52.052+0000 7f2315e00640 1 -- 172.21.15.12:0/2434343695 >> 172.21.15.12:0/2434343695 conn(0x7f23100fe830 msgr2=0x7f23100ff430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:04:52.056 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:52.053+0000 7f2315e00640 1 -- 172.21.15.12:0/2434343695 shutdown_connections 2024-09-11T00:04:52.056 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:52.053+0000 7f2315e00640 1 -- 172.21.15.12:0/2434343695 wait complete. 2024-09-11T00:04:52.313 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Sep 11 00:04:52 smithi012 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-mgr-x[30560]: 2024-09-11T00:04:52.023+0000 7f9fcea00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-11T00:04:52.313 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Sep 11 00:04:52 smithi012 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-mgr-x[30560]: 2024-09-11T00:04:52.023+0000 7f9fcea00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-11T00:04:52.313 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Sep 11 00:04:52 smithi012 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-mgr-x[30560]: 2024-09-11T00:04:52.023+0000 7f9fcea00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-11T00:04:52.313 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Sep 11 00:04:52 smithi012 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-mgr-x[30560]: 2024-09-11T00:04:52.023+0000 7f9fcea00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-11T00:04:52.314 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Sep 11 00:04:52 smithi012 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-mgr-x[30560]: 2024-09-11T00:04:52.023+0000 7f9fcea00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-11T00:04:52.748 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid ff3ad74a-6fcf-11ef-bcea-c7b262605968 -- ceph fs subvolume create cephfs sv_0 --group_name qa 2024-09-11T00:04:53.128 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:04:53.524 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:04:53 smithi012 ceph-mon[30333]: from='client.24842 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T00:04:53.524 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:04:53 smithi012 ceph-mon[30333]: pgmap v328: 129 pgs: 129 active+clean; 587 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 4.1 KiB/s wr, 15 op/s 2024-09-11T00:04:53.525 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:04:53 smithi012 ceph-mon[30333]: mgrmap e17: x(active, since 8m), standbys: z, y 2024-09-11T00:04:53.532 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:04:53 smithi165 ceph-mon[35356]: from='client.24842 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T00:04:53.532 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:04:53 smithi165 ceph-mon[35356]: pgmap v328: 129 pgs: 129 active+clean; 587 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 4.1 KiB/s wr, 15 op/s 2024-09-11T00:04:53.532 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:04:53 smithi165 ceph-mon[35356]: mgrmap e17: x(active, since 8m), standbys: z, y 2024-09-11T00:04:53.581 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:04:53 smithi178 ceph-mon[35919]: from='client.24842 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T00:04:53.581 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:04:53 smithi178 ceph-mon[35919]: pgmap v328: 129 pgs: 129 active+clean; 587 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 4.1 KiB/s wr, 15 op/s 2024-09-11T00:04:53.581 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:04:53 smithi178 ceph-mon[35919]: mgrmap e17: x(active, since 8m), standbys: z, y 2024-09-11T00:04:53.643 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:53.640+0000 7f1cc0c00640 1 -- 172.21.15.12:0/995412453 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1cbc10b330 msgr2=0x7f1cbc111b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:04:53.643 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:53.640+0000 7f1cc0c00640 1 --2- 172.21.15.12:0/995412453 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1cbc10b330 0x7f1cbc111b30 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f1cb4009be0 tx=0x7f1cb402a6d0 comp rx=0 tx=0).stop 2024-09-11T00:04:53.643 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:53.640+0000 7f1cc0c00640 1 -- 172.21.15.12:0/995412453 shutdown_connections 2024-09-11T00:04:53.643 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:53.640+0000 7f1cc0c00640 1 --2- 172.21.15.12:0/995412453 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1cbc10b330 0x7f1cbc111b30 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:04:53.643 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:53.640+0000 7f1cc0c00640 1 --2- 172.21.15.12:0/995412453 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1cbc10a9b0 0x7f1cbc10adf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:04:53.643 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:53.640+0000 7f1cc0c00640 1 --2- 172.21.15.12:0/995412453 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1cbc104fa0 0x7f1cbc105380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:04:53.644 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:53.640+0000 7f1cc0c00640 1 -- 172.21.15.12:0/995412453 >> 172.21.15.12:0/995412453 conn(0x7f1cbc100a50 msgr2=0x7f1cbc102e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:04:53.644 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:53.640+0000 7f1cc0c00640 1 -- 172.21.15.12:0/995412453 shutdown_connections 2024-09-11T00:04:53.644 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:53.640+0000 7f1cc0c00640 1 -- 172.21.15.12:0/995412453 wait complete. 2024-09-11T00:04:53.644 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:53.641+0000 7f1cc0c00640 1 Processor -- start 2024-09-11T00:04:53.645 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:53.641+0000 7f1cc0c00640 1 -- start start 2024-09-11T00:04:53.645 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:53.642+0000 7f1cc0c00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1cbc104fa0 0x7f1cbc19b490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:04:53.645 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:53.642+0000 7f1cc0c00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1cbc10a9b0 0x7f1cbc19b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:04:53.646 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:53.642+0000 7f1cc0c00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1cbc10b330 0x7f1cbc1a0d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:04:53.646 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:53.642+0000 7f1cc0c00640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f1cbc114f60 con 0x7f1cbc10b330 2024-09-11T00:04:53.646 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:53.642+0000 7f1cc0c00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f1cbc114de0 con 0x7f1cbc10a9b0 2024-09-11T00:04:53.646 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:53.642+0000 7f1cc0c00640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f1cbc1150e0 con 0x7f1cbc104fa0 2024-09-11T00:04:53.646 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:53.642+0000 7f1cb9e00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1cbc104fa0 0x7f1cbc19b490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:04:53.646 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:53.642+0000 7f1cb9e00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1cbc104fa0 0x7f1cbc19b490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.12:42558/0 (socket says 172.21.15.12:42558) 2024-09-11T00:04:53.646 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:53.642+0000 7f1cb9e00640 1 -- 172.21.15.12:0/3720260811 learned_addr learned my addr 172.21.15.12:0/3720260811 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:04:53.646 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:53.642+0000 7f1cb9400640 1 --2- 172.21.15.12:0/3720260811 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1cbc10a9b0 0x7f1cbc19b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:04:53.646 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:53.642+0000 7f1cba800640 1 --2- 172.21.15.12:0/3720260811 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1cbc10b330 0x7f1cbc1a0d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:04:53.646 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:53.643+0000 7f1cb9e00640 1 -- 172.21.15.12:0/3720260811 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1cbc10a9b0 msgr2=0x7f1cbc19b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:04:53.646 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:53.643+0000 7f1cb9e00640 1 --2- 172.21.15.12:0/3720260811 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1cbc10a9b0 0x7f1cbc19b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:04:53.646 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:53.643+0000 7f1cb9e00640 1 -- 172.21.15.12:0/3720260811 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1cbc10b330 msgr2=0x7f1cbc1a0d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:04:53.647 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:53.643+0000 7f1cb9e00640 1 --2- 172.21.15.12:0/3720260811 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1cbc10b330 0x7f1cbc1a0d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:04:53.647 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:53.643+0000 7f1cb9e00640 1 -- 172.21.15.12:0/3720260811 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1cbc1a1540 con 0x7f1cbc104fa0 2024-09-11T00:04:53.647 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:53.643+0000 7f1cb9400640 1 --2- 172.21.15.12:0/3720260811 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1cbc10a9b0 0x7f1cbc19b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:04:53.647 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:53.643+0000 7f1cb9e00640 1 --2- 172.21.15.12:0/3720260811 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1cbc104fa0 0x7f1cbc19b490 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f1cac00e9c0 tx=0x7f1cac00ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:04:53.647 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:53.643+0000 7f1cb2a00640 1 -- 172.21.15.12:0/3720260811 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cac00cde0 con 0x7f1cbc104fa0 2024-09-11T00:04:53.647 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:53.643+0000 7f1cb2a00640 1 -- 172.21.15.12:0/3720260811 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1cac0035f0 con 0x7f1cbc104fa0 2024-09-11T00:04:53.647 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:53.643+0000 7f1cb2a00640 1 -- 172.21.15.12:0/3720260811 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cac010690 con 0x7f1cbc104fa0 2024-09-11T00:04:53.647 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:53.643+0000 7f1cc0c00640 1 -- 172.21.15.12:0/3720260811 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1cbc1a1830 con 0x7f1cbc104fa0 2024-09-11T00:04:53.647 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:53.644+0000 7f1cc0c00640 1 -- 172.21.15.12:0/3720260811 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1cbc1a7ef0 con 0x7f1cbc104fa0 2024-09-11T00:04:53.647 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:53.645+0000 7f1cc0c00640 1 -- 172.21.15.12:0/3720260811 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1cbc105380 con 0x7f1cbc104fa0 2024-09-11T00:04:53.649 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:53.647+0000 7f1cb2a00640 1 -- 172.21.15.12:0/3720260811 <== mon.1 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 17) ==== 153301+0+0 (secure 0 0 0) 0x7f1cac004000 con 0x7f1cbc104fa0 2024-09-11T00:04:53.650 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:53.648+0000 7f1cb2a00640 1 --2- 172.21.15.12:0/3720260811 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f1ca00b53f0 0x7f1ca00b78b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:04:53.651 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:53.649+0000 7f1cb2a00640 1 -- 172.21.15.12:0/3720260811 <== mon.1 v2:172.21.15.178:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8802+0+0 (secure 0 0 0) 0x7f1cac0e6310 con 0x7f1cbc104fa0 2024-09-11T00:04:53.651 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:53.649+0000 7f1cb9400640 1 --2- 172.21.15.12:0/3720260811 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f1ca00b53f0 0x7f1ca00b78b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:04:53.652 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:53.649+0000 7f1cb9400640 1 --2- 172.21.15.12:0/3720260811 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f1ca00b53f0 0x7f1ca00b78b0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f1cbc1042b0 tx=0x7f1ca8005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:04:53.652 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:53.650+0000 7f1cb2a00640 1 -- 172.21.15.12:0/3720260811 <== mon.1 v2:172.21.15.178:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f1cac0ace60 con 0x7f1cbc104fa0 2024-09-11T00:04:53.872 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:53.870+0000 7f1cc0c00640 1 -- 172.21.15.12:0/3720260811 --> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f1cbc063590 con 0x7f1ca00b53f0 2024-09-11T00:04:53.899 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:53.897+0000 7f1cb2a00640 1 -- 172.21.15.12:0/3720260811 <== mgr.14150 v2:172.21.15.12:6800/3442939137 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f1cbc063590 con 0x7f1ca00b53f0 2024-09-11T00:04:53.902 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:53.900+0000 7f1cc0c00640 1 -- 172.21.15.12:0/3720260811 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f1ca00b53f0 msgr2=0x7f1ca00b78b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:04:53.902 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:53.900+0000 7f1cc0c00640 1 --2- 172.21.15.12:0/3720260811 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f1ca00b53f0 0x7f1ca00b78b0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f1cbc1042b0 tx=0x7f1ca8005f90 comp rx=0 tx=0).stop 2024-09-11T00:04:53.902 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:53.900+0000 7f1cc0c00640 1 -- 172.21.15.12:0/3720260811 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1cbc104fa0 msgr2=0x7f1cbc19b490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:04:53.902 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:53.900+0000 7f1cc0c00640 1 --2- 172.21.15.12:0/3720260811 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1cbc104fa0 0x7f1cbc19b490 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f1cac00e9c0 tx=0x7f1cac00ee80 comp rx=0 tx=0).stop 2024-09-11T00:04:53.902 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:53.900+0000 7f1cc0c00640 1 -- 172.21.15.12:0/3720260811 shutdown_connections 2024-09-11T00:04:53.902 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:53.900+0000 7f1cc0c00640 1 --2- 172.21.15.12:0/3720260811 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f1ca00b53f0 0x7f1ca00b78b0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:04:53.902 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:53.900+0000 7f1cc0c00640 1 --2- 172.21.15.12:0/3720260811 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1cbc10b330 0x7f1cbc1a0d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:04:53.903 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:53.900+0000 7f1cc0c00640 1 --2- 172.21.15.12:0/3720260811 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1cbc10a9b0 0x7f1cbc19b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:04:53.903 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:53.900+0000 7f1cc0c00640 1 --2- 172.21.15.12:0/3720260811 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1cbc104fa0 0x7f1cbc19b490 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:04:53.903 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:53.900+0000 7f1cc0c00640 1 -- 172.21.15.12:0/3720260811 >> 172.21.15.12:0/3720260811 conn(0x7f1cbc100a50 msgr2=0x7f1cbc102e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:04:53.903 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:53.900+0000 7f1cc0c00640 1 -- 172.21.15.12:0/3720260811 shutdown_connections 2024-09-11T00:04:53.903 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:53.900+0000 7f1cc0c00640 1 -- 172.21.15.12:0/3720260811 wait complete. 2024-09-11T00:04:54.512 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:04:54 smithi012 ceph-mon[30333]: from='client.24841 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T00:04:54.532 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:04:54 smithi165 ceph-mon[35356]: from='client.24841 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T00:04:54.549 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid ff3ad74a-6fcf-11ef-bcea-c7b262605968 -- ceph fs subvolume getpath cephfs --group_name qa sv_0 2024-09-11T00:04:54.580 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:04:54 smithi178 ceph-mon[35919]: from='client.24841 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T00:04:54.928 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:04:55.341 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:04:55 smithi012 ceph-mon[30333]: pgmap v329: 129 pgs: 129 active+clean; 587 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 4.0 KiB/s wr, 15 op/s 2024-09-11T00:04:55.460 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:55.458+0000 7f8f6d800640 1 -- 172.21.15.12:0/3081765841 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8f6810a9b0 msgr2=0x7f8f6810adf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:04:55.461 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:55.458+0000 7f8f6d800640 1 --2- 172.21.15.12:0/3081765841 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8f6810a9b0 0x7f8f6810adf0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f8f50009900 tx=0x7f8f5002a3f0 comp rx=0 tx=0).stop 2024-09-11T00:04:55.461 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:55.458+0000 7f8f6d800640 1 -- 172.21.15.12:0/3081765841 shutdown_connections 2024-09-11T00:04:55.461 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:55.458+0000 7f8f6d800640 1 --2- 172.21.15.12:0/3081765841 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8f6810b330 0x7f8f68111b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:04:55.461 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:55.458+0000 7f8f6d800640 1 --2- 172.21.15.12:0/3081765841 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8f6810a9b0 0x7f8f6810adf0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:04:55.461 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:55.458+0000 7f8f6d800640 1 --2- 172.21.15.12:0/3081765841 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8f68104fa0 0x7f8f68105380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:04:55.461 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:55.458+0000 7f8f6d800640 1 -- 172.21.15.12:0/3081765841 >> 172.21.15.12:0/3081765841 conn(0x7f8f681009f0 msgr2=0x7f8f68102e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:04:55.461 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:55.458+0000 7f8f6d800640 1 -- 172.21.15.12:0/3081765841 shutdown_connections 2024-09-11T00:04:55.462 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:55.458+0000 7f8f6d800640 1 -- 172.21.15.12:0/3081765841 wait complete. 2024-09-11T00:04:55.464 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:55.459+0000 7f8f6d800640 1 Processor -- start 2024-09-11T00:04:55.464 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:55.459+0000 7f8f6d800640 1 -- start start 2024-09-11T00:04:55.464 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:55.459+0000 7f8f6d800640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8f68104fa0 0x7f8f6819b460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:04:55.465 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:55.459+0000 7f8f6d800640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8f6810a9b0 0x7f8f6819b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:04:55.465 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:55.459+0000 7f8f6d800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8f6810b330 0x7f8f681a0d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:04:55.465 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:55.459+0000 7f8f6d800640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f8f68114fb0 con 0x7f8f68104fa0 2024-09-11T00:04:55.465 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:55.459+0000 7f8f6d800640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f8f68114e30 con 0x7f8f6810b330 2024-09-11T00:04:55.465 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:55.459+0000 7f8f6d800640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f8f68115130 con 0x7f8f6810a9b0 2024-09-11T00:04:55.465 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:55.460+0000 7f8f66a00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8f68104fa0 0x7f8f6819b460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:04:55.465 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:55.460+0000 7f8f66a00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8f68104fa0 0x7f8f6819b460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:40630/0 (socket says 172.21.15.12:40630) 2024-09-11T00:04:55.465 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:55.460+0000 7f8f66a00640 1 -- 172.21.15.12:0/1995474637 learned_addr learned my addr 172.21.15.12:0/1995474637 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:04:55.465 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:55.460+0000 7f8f67400640 1 --2- 172.21.15.12:0/1995474637 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8f6810b330 0x7f8f681a0d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:04:55.465 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:55.460+0000 7f8f5e000640 1 --2- 172.21.15.12:0/1995474637 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8f6810a9b0 0x7f8f6819b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:04:55.466 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:55.461+0000 7f8f67400640 1 -- 172.21.15.12:0/1995474637 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8f6810a9b0 msgr2=0x7f8f6819b9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:04:55.466 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:55.461+0000 7f8f67400640 1 --2- 172.21.15.12:0/1995474637 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8f6810a9b0 0x7f8f6819b9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:04:55.466 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:55.461+0000 7f8f67400640 1 -- 172.21.15.12:0/1995474637 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8f68104fa0 msgr2=0x7f8f6819b460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:04:55.466 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:55.461+0000 7f8f67400640 1 --2- 172.21.15.12:0/1995474637 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8f68104fa0 0x7f8f6819b460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:04:55.467 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:55.461+0000 7f8f67400640 1 -- 172.21.15.12:0/1995474637 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f681a1510 con 0x7f8f6810b330 2024-09-11T00:04:55.467 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:55.461+0000 7f8f66a00640 1 --2- 172.21.15.12:0/1995474637 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8f68104fa0 0x7f8f6819b460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:04:55.467 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:55.461+0000 7f8f67400640 1 --2- 172.21.15.12:0/1995474637 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8f6810b330 0x7f8f681a0d40 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f8f60009fd0 tx=0x7f8f6000ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:04:55.467 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:55.461+0000 7f8f5fe00640 1 -- 172.21.15.12:0/1995474637 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f6000ce20 con 0x7f8f6810b330 2024-09-11T00:04:55.467 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:55.461+0000 7f8f5fe00640 1 -- 172.21.15.12:0/1995474637 <== mon.2 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8f60005020 con 0x7f8f6810b330 2024-09-11T00:04:55.467 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:55.461+0000 7f8f5fe00640 1 -- 172.21.15.12:0/1995474637 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f60010690 con 0x7f8f6810b330 2024-09-11T00:04:55.467 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:55.461+0000 7f8f6d800640 1 -- 172.21.15.12:0/1995474637 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f681a1800 con 0x7f8f6810b330 2024-09-11T00:04:55.468 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:55.461+0000 7f8f6d800640 1 -- 172.21.15.12:0/1995474637 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f681a7ec0 con 0x7f8f6810b330 2024-09-11T00:04:55.469 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:55.462+0000 7f8f6d800640 1 -- 172.21.15.12:0/1995474637 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8f68105380 con 0x7f8f6810b330 2024-09-11T00:04:55.469 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:55.465+0000 7f8f5fe00640 1 -- 172.21.15.12:0/1995474637 <== mon.2 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 17) ==== 153301+0+0 (secure 0 0 0) 0x7f8f600054d0 con 0x7f8f6810b330 2024-09-11T00:04:55.470 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:55.466+0000 7f8f5fe00640 1 --2- 172.21.15.12:0/1995474637 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f8f4c0b53f0 0x7f8f4c0b78b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:04:55.470 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:55.466+0000 7f8f5fe00640 1 -- 172.21.15.12:0/1995474637 <== mon.2 v2:172.21.15.165:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8802+0+0 (secure 0 0 0) 0x7f8f600e66c0 con 0x7f8f6810b330 2024-09-11T00:04:55.470 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:55.466+0000 7f8f66a00640 1 --2- 172.21.15.12:0/1995474637 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f8f4c0b53f0 0x7f8f4c0b78b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:04:55.470 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:55.467+0000 7f8f66a00640 1 --2- 172.21.15.12:0/1995474637 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f8f4c0b53f0 0x7f8f4c0b78b0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f8f580095f0 tx=0x7f8f58005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:04:55.470 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:55.468+0000 7f8f5fe00640 1 -- 172.21.15.12:0/1995474637 <== mon.2 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f8f600ad180 con 0x7f8f6810b330 2024-09-11T00:04:55.532 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:04:55 smithi165 ceph-mon[35356]: pgmap v329: 129 pgs: 129 active+clean; 587 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 4.0 KiB/s wr, 15 op/s 2024-09-11T00:04:55.580 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:04:55 smithi178 ceph-mon[35919]: pgmap v329: 129 pgs: 129 active+clean; 587 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 4.0 KiB/s wr, 15 op/s 2024-09-11T00:04:55.694 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:55.692+0000 7f8f6d800640 1 -- 172.21.15.12:0/1995474637 --> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}) -- 0x7f8f68063590 con 0x7f8f4c0b53f0 2024-09-11T00:04:55.700 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:55.698+0000 7f8f5fe00640 1 -- 172.21.15.12:0/1995474637 <== mgr.14150 v2:172.21.15.12:6800/3442939137 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7f8f68063590 con 0x7f8f4c0b53f0 2024-09-11T00:04:55.700 INFO:teuthology.orchestra.run.smithi012.stdout:/volumes/qa/sv_0/9bd8d047-3363-4faa-8edb-ae61701a95ae 2024-09-11T00:04:55.703 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:55.700+0000 7f8f6d800640 1 -- 172.21.15.12:0/1995474637 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f8f4c0b53f0 msgr2=0x7f8f4c0b78b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:04:55.703 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:55.700+0000 7f8f6d800640 1 --2- 172.21.15.12:0/1995474637 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f8f4c0b53f0 0x7f8f4c0b78b0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f8f580095f0 tx=0x7f8f58005f90 comp rx=0 tx=0).stop 2024-09-11T00:04:55.703 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:55.700+0000 7f8f6d800640 1 -- 172.21.15.12:0/1995474637 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8f6810b330 msgr2=0x7f8f681a0d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:04:55.703 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:55.700+0000 7f8f6d800640 1 --2- 172.21.15.12:0/1995474637 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8f6810b330 0x7f8f681a0d40 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f8f60009fd0 tx=0x7f8f6000ef10 comp rx=0 tx=0).stop 2024-09-11T00:04:55.703 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:55.700+0000 7f8f6d800640 1 -- 172.21.15.12:0/1995474637 shutdown_connections 2024-09-11T00:04:55.703 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:55.700+0000 7f8f6d800640 1 --2- 172.21.15.12:0/1995474637 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f8f4c0b53f0 0x7f8f4c0b78b0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:04:55.703 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:55.700+0000 7f8f6d800640 1 --2- 172.21.15.12:0/1995474637 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8f6810b330 0x7f8f681a0d40 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:04:55.703 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:55.700+0000 7f8f6d800640 1 --2- 172.21.15.12:0/1995474637 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8f6810a9b0 0x7f8f6819b9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:04:55.703 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:55.700+0000 7f8f6d800640 1 --2- 172.21.15.12:0/1995474637 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8f68104fa0 0x7f8f6819b460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:04:55.703 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:55.701+0000 7f8f6d800640 1 -- 172.21.15.12:0/1995474637 >> 172.21.15.12:0/1995474637 conn(0x7f8f681009f0 msgr2=0x7f8f68102de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:04:55.704 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:55.701+0000 7f8f6d800640 1 -- 172.21.15.12:0/1995474637 shutdown_connections 2024-09-11T00:04:55.704 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:55.701+0000 7f8f6d800640 1 -- 172.21.15.12:0/1995474637 wait complete. 2024-09-11T00:04:56.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:04:56 smithi012 ceph-mon[30333]: from='client.24860 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T00:04:56.408 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid ff3ad74a-6fcf-11ef-bcea-c7b262605968 -- ceph fs subvolume create cephfs sv_1 --group_name qa 2024-09-11T00:04:56.532 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:04:56 smithi165 ceph-mon[35356]: from='client.24860 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T00:04:56.580 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:04:56 smithi178 ceph-mon[35919]: from='client.24860 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T00:04:56.787 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:04:57.348 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:57.326+0000 7fdd01c00640 1 -- 172.21.15.12:0/557789981 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdcfc10b330 msgr2=0x7fdcfc111b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:04:57.348 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:57.326+0000 7fdd01c00640 1 --2- 172.21.15.12:0/557789981 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdcfc10b330 0x7fdcfc111b30 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fdcf4009ca0 tx=0x7fdcf402a8c0 comp rx=0 tx=0).stop 2024-09-11T00:04:57.349 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:57.326+0000 7fdd01c00640 1 -- 172.21.15.12:0/557789981 shutdown_connections 2024-09-11T00:04:57.349 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:57.326+0000 7fdd01c00640 1 --2- 172.21.15.12:0/557789981 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdcfc10b330 0x7fdcfc111b30 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:04:57.349 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:57.326+0000 7fdd01c00640 1 --2- 172.21.15.12:0/557789981 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fdcfc10a9b0 0x7fdcfc10adf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:04:57.349 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:57.326+0000 7fdd01c00640 1 --2- 172.21.15.12:0/557789981 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdcfc104fa0 0x7fdcfc105380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:04:57.349 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:57.326+0000 7fdd01c00640 1 -- 172.21.15.12:0/557789981 >> 172.21.15.12:0/557789981 conn(0x7fdcfc100a50 msgr2=0x7fdcfc102e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:04:57.349 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:57.326+0000 7fdd01c00640 1 -- 172.21.15.12:0/557789981 shutdown_connections 2024-09-11T00:04:57.349 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:57.326+0000 7fdd01c00640 1 -- 172.21.15.12:0/557789981 wait complete. 2024-09-11T00:04:57.349 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:57.327+0000 7fdd01c00640 1 Processor -- start 2024-09-11T00:04:57.349 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:57.328+0000 7fdd01c00640 1 -- start start 2024-09-11T00:04:57.350 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:57.328+0000 7fdd01c00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fdcfc104fa0 0x7fdcfc19b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:04:57.350 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:57.328+0000 7fdd01c00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdcfc10a9b0 0x7fdcfc19b970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:04:57.350 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:57.328+0000 7fdd01c00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdcfc10b330 0x7fdcfc1a0d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:04:57.350 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:57.328+0000 7fdd01c00640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fdcfc114f60 con 0x7fdcfc10a9b0 2024-09-11T00:04:57.350 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:57.328+0000 7fdd01c00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fdcfc114de0 con 0x7fdcfc104fa0 2024-09-11T00:04:57.350 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:57.328+0000 7fdd01c00640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fdcfc1150e0 con 0x7fdcfc10b330 2024-09-11T00:04:57.350 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:57.328+0000 7fdcf2000640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdcfc10a9b0 0x7fdcfc19b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:04:57.350 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:57.328+0000 7fdcf2000640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdcfc10a9b0 0x7fdcfc19b970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:40662/0 (socket says 172.21.15.12:40662) 2024-09-11T00:04:57.350 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:57.328+0000 7fdcf2000640 1 -- 172.21.15.12:0/1095686033 learned_addr learned my addr 172.21.15.12:0/1095686033 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:04:57.350 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:57.328+0000 7fdcfaa00640 1 --2- 172.21.15.12:0/1095686033 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fdcfc104fa0 0x7fdcfc19b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:04:57.350 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:57.328+0000 7fdcfb400640 1 --2- 172.21.15.12:0/1095686033 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdcfc10b330 0x7fdcfc1a0d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:04:57.350 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:57.329+0000 7fdcf2000640 1 -- 172.21.15.12:0/1095686033 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdcfc10b330 msgr2=0x7fdcfc1a0d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:04:57.351 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:57.329+0000 7fdcf2000640 1 --2- 172.21.15.12:0/1095686033 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdcfc10b330 0x7fdcfc1a0d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:04:57.351 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:57.329+0000 7fdcf2000640 1 -- 172.21.15.12:0/1095686033 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fdcfc104fa0 msgr2=0x7fdcfc19b430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:04:57.351 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:57.329+0000 7fdcf2000640 1 --2- 172.21.15.12:0/1095686033 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fdcfc104fa0 0x7fdcfc19b430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:04:57.351 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:57.329+0000 7fdcf2000640 1 -- 172.21.15.12:0/1095686033 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdcfc1a14e0 con 0x7fdcfc10a9b0 2024-09-11T00:04:57.351 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:57.329+0000 7fdcfaa00640 1 --2- 172.21.15.12:0/1095686033 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fdcfc104fa0 0x7fdcfc19b430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:04:57.351 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:57.329+0000 7fdcf2000640 1 --2- 172.21.15.12:0/1095686033 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdcfc10a9b0 0x7fdcfc19b970 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fdce400e9f0 tx=0x7fdce400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:04:57.351 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:57.329+0000 7fdcf3e00640 1 -- 172.21.15.12:0/1095686033 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdce400cd80 con 0x7fdcfc10a9b0 2024-09-11T00:04:57.351 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:57.329+0000 7fdcf3e00640 1 -- 172.21.15.12:0/1095686033 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdce40108e0 con 0x7fdcfc10a9b0 2024-09-11T00:04:57.351 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:57.329+0000 7fdd01c00640 1 -- 172.21.15.12:0/1095686033 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdcfc1a17d0 con 0x7fdcfc10a9b0 2024-09-11T00:04:57.351 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:57.329+0000 7fdcf3e00640 1 -- 172.21.15.12:0/1095686033 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdce4018980 con 0x7fdcfc10a9b0 2024-09-11T00:04:57.351 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:57.330+0000 7fdd01c00640 1 -- 172.21.15.12:0/1095686033 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdcfc1a7e90 con 0x7fdcfc10a9b0 2024-09-11T00:04:57.352 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:57.331+0000 7fdd01c00640 1 -- 172.21.15.12:0/1095686033 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdcfc105380 con 0x7fdcfc10a9b0 2024-09-11T00:04:57.352 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:57.333+0000 7fdcf3e00640 1 -- 172.21.15.12:0/1095686033 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 17) ==== 153301+0+0 (secure 0 0 0) 0x7fdce4010430 con 0x7fdcfc10a9b0 2024-09-11T00:04:57.352 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:57.335+0000 7fdcf3e00640 1 --2- 172.21.15.12:0/1095686033 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fdce00b53f0 0x7fdce00b78b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:04:57.352 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:57.335+0000 7fdcf3e00640 1 -- 172.21.15.12:0/1095686033 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8802+0+0 (secure 0 0 0) 0x7fdce40e6240 con 0x7fdcfc10a9b0 2024-09-11T00:04:57.352 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:57.335+0000 7fdcfaa00640 1 --2- 172.21.15.12:0/1095686033 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fdce00b53f0 0x7fdce00b78b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:04:57.353 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:57.336+0000 7fdcfaa00640 1 --2- 172.21.15.12:0/1095686033 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fdce00b53f0 0x7fdce00b78b0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fdcec005d00 tx=0x7fdcec00b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:04:57.353 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:57.339+0000 7fdcf3e00640 1 -- 172.21.15.12:0/1095686033 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7fdce40ac6b0 con 0x7fdcfc10a9b0 2024-09-11T00:04:57.532 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:04:57 smithi165 ceph-mon[35356]: pgmap v330: 129 pgs: 129 active+clean; 608 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 6.3 KiB/s wr, 22 op/s 2024-09-11T00:04:57.562 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:04:57 smithi012 ceph-mon[30333]: pgmap v330: 129 pgs: 129 active+clean; 608 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 6.3 KiB/s wr, 22 op/s 2024-09-11T00:04:57.563 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:57.561+0000 7fdd01c00640 1 -- 172.21.15.12:0/1095686033 --> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7fdcfc02e640 con 0x7fdce00b53f0 2024-09-11T00:04:57.580 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:04:57 smithi178 ceph-mon[35919]: pgmap v330: 129 pgs: 129 active+clean; 608 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 6.3 KiB/s wr, 22 op/s 2024-09-11T00:04:57.590 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:57.588+0000 7fdcf3e00640 1 -- 172.21.15.12:0/1095686033 <== mgr.14150 v2:172.21.15.12:6800/3442939137 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fdcfc02e640 con 0x7fdce00b53f0 2024-09-11T00:04:57.592 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:57.590+0000 7fdd01c00640 1 -- 172.21.15.12:0/1095686033 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fdce00b53f0 msgr2=0x7fdce00b78b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:04:57.592 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:57.590+0000 7fdd01c00640 1 --2- 172.21.15.12:0/1095686033 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fdce00b53f0 0x7fdce00b78b0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fdcec005d00 tx=0x7fdcec00b410 comp rx=0 tx=0).stop 2024-09-11T00:04:57.592 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:57.590+0000 7fdd01c00640 1 -- 172.21.15.12:0/1095686033 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdcfc10a9b0 msgr2=0x7fdcfc19b970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:04:57.593 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:57.590+0000 7fdd01c00640 1 --2- 172.21.15.12:0/1095686033 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdcfc10a9b0 0x7fdcfc19b970 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fdce400e9f0 tx=0x7fdce400eeb0 comp rx=0 tx=0).stop 2024-09-11T00:04:57.593 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:57.590+0000 7fdd01c00640 1 -- 172.21.15.12:0/1095686033 shutdown_connections 2024-09-11T00:04:57.593 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:57.590+0000 7fdd01c00640 1 --2- 172.21.15.12:0/1095686033 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fdce00b53f0 0x7fdce00b78b0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:04:57.593 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:57.590+0000 7fdd01c00640 1 --2- 172.21.15.12:0/1095686033 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdcfc10b330 0x7fdcfc1a0d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:04:57.593 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:57.590+0000 7fdd01c00640 1 --2- 172.21.15.12:0/1095686033 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdcfc10a9b0 0x7fdcfc19b970 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:04:57.593 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:57.590+0000 7fdd01c00640 1 --2- 172.21.15.12:0/1095686033 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fdcfc104fa0 0x7fdcfc19b430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:04:57.593 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:57.590+0000 7fdd01c00640 1 -- 172.21.15.12:0/1095686033 >> 172.21.15.12:0/1095686033 conn(0x7fdcfc100a50 msgr2=0x7fdcfc102e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:04:57.593 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:57.591+0000 7fdd01c00640 1 -- 172.21.15.12:0/1095686033 shutdown_connections 2024-09-11T00:04:57.593 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:57.591+0000 7fdd01c00640 1 -- 172.21.15.12:0/1095686033 wait complete. 2024-09-11T00:04:58.240 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid ff3ad74a-6fcf-11ef-bcea-c7b262605968 -- ceph fs subvolume getpath cephfs --group_name qa sv_1 2024-09-11T00:04:58.532 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:04:58 smithi165 ceph-mon[35356]: from='client.14829 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T00:04:58.562 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:04:58 smithi012 ceph-mon[30333]: from='client.14829 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T00:04:58.580 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:04:58 smithi178 ceph-mon[35919]: from='client.14829 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T00:04:58.619 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:04:59.156 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:59.154+0000 7f9d3aa00640 1 -- 172.21.15.12:0/267573036 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9d341087d0 msgr2=0x7f9d34108c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:04:59.156 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:59.154+0000 7f9d3aa00640 1 --2- 172.21.15.12:0/267573036 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9d341087d0 0x7f9d34108c10 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f9d1c009900 tx=0x7f9d1c02a410 comp rx=0 tx=0).stop 2024-09-11T00:04:59.156 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:59.154+0000 7f9d3aa00640 1 -- 172.21.15.12:0/267573036 shutdown_connections 2024-09-11T00:04:59.157 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:59.154+0000 7f9d3aa00640 1 --2- 172.21.15.12:0/267573036 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9d34109150 0x7f9d3410f950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:04:59.157 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:59.154+0000 7f9d3aa00640 1 --2- 172.21.15.12:0/267573036 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9d341087d0 0x7f9d34108c10 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:04:59.157 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:59.154+0000 7f9d3aa00640 1 --2- 172.21.15.12:0/267573036 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9d34102dc0 0x7f9d341031a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:04:59.157 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:59.154+0000 7f9d3aa00640 1 -- 172.21.15.12:0/267573036 >> 172.21.15.12:0/267573036 conn(0x7f9d340fe830 msgr2=0x7f9d34100c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:04:59.157 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:59.154+0000 7f9d3aa00640 1 -- 172.21.15.12:0/267573036 shutdown_connections 2024-09-11T00:04:59.157 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:59.154+0000 7f9d3aa00640 1 -- 172.21.15.12:0/267573036 wait complete. 2024-09-11T00:04:59.159 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:59.155+0000 7f9d3aa00640 1 Processor -- start 2024-09-11T00:04:59.159 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:59.155+0000 7f9d3aa00640 1 -- start start 2024-09-11T00:04:59.160 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:59.155+0000 7f9d3aa00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9d34102dc0 0x7f9d34199280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:04:59.160 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:59.155+0000 7f9d3aa00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9d341087d0 0x7f9d341997c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:04:59.160 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:59.155+0000 7f9d3aa00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9d34109150 0x7f9d3419eb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:04:59.160 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:59.156+0000 7f9d3aa00640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f9d34112e10 con 0x7f9d34109150 2024-09-11T00:04:59.160 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:59.156+0000 7f9d3aa00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f9d34112c90 con 0x7f9d34102dc0 2024-09-11T00:04:59.160 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:59.156+0000 7f9d3aa00640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f9d34112f90 con 0x7f9d341087d0 2024-09-11T00:04:59.160 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:59.156+0000 7f9d38a00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9d34109150 0x7f9d3419eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:04:59.160 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:59.156+0000 7f9d33e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9d34102dc0 0x7f9d34199280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:04:59.160 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:59.156+0000 7f9d38a00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9d34109150 0x7f9d3419eb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:60260/0 (socket says 172.21.15.12:60260) 2024-09-11T00:04:59.160 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:59.156+0000 7f9d38a00640 1 -- 172.21.15.12:0/1864089959 learned_addr learned my addr 172.21.15.12:0/1864089959 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:04:59.161 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:59.156+0000 7f9d2b400640 1 --2- 172.21.15.12:0/1864089959 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9d341087d0 0x7f9d341997c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:04:59.161 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:59.156+0000 7f9d38a00640 1 -- 172.21.15.12:0/1864089959 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9d341087d0 msgr2=0x7f9d341997c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:04:59.161 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:59.156+0000 7f9d38a00640 1 --2- 172.21.15.12:0/1864089959 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9d341087d0 0x7f9d341997c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:04:59.161 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:59.156+0000 7f9d38a00640 1 -- 172.21.15.12:0/1864089959 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9d34102dc0 msgr2=0x7f9d34199280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:04:59.161 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:59.156+0000 7f9d38a00640 1 --2- 172.21.15.12:0/1864089959 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9d34102dc0 0x7f9d34199280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:04:59.161 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:59.156+0000 7f9d38a00640 1 -- 172.21.15.12:0/1864089959 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d3419f330 con 0x7f9d34109150 2024-09-11T00:04:59.161 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:59.157+0000 7f9d38a00640 1 --2- 172.21.15.12:0/1864089959 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9d34109150 0x7f9d3419eb60 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f9d2c00eb10 tx=0x7f9d2c00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:04:59.161 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:59.157+0000 7f9d31600640 1 -- 172.21.15.12:0/1864089959 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d2c004b20 con 0x7f9d34109150 2024-09-11T00:04:59.161 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:59.157+0000 7f9d31600640 1 -- 172.21.15.12:0/1864089959 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9d2c004cc0 con 0x7f9d34109150 2024-09-11T00:04:59.161 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:59.157+0000 7f9d31600640 1 -- 172.21.15.12:0/1864089959 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d2c010670 con 0x7f9d34109150 2024-09-11T00:04:59.161 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:59.157+0000 7f9d3aa00640 1 -- 172.21.15.12:0/1864089959 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9d3419f620 con 0x7f9d34109150 2024-09-11T00:04:59.162 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:59.157+0000 7f9d3aa00640 1 -- 172.21.15.12:0/1864089959 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9d341a5ce0 con 0x7f9d34109150 2024-09-11T00:04:59.162 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:59.158+0000 7f9d3aa00640 1 -- 172.21.15.12:0/1864089959 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9d341031a0 con 0x7f9d34109150 2024-09-11T00:04:59.162 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:59.160+0000 7f9d31600640 1 -- 172.21.15.12:0/1864089959 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 17) ==== 153301+0+0 (secure 0 0 0) 0x7f9d2c005020 con 0x7f9d34109150 2024-09-11T00:04:59.164 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:59.162+0000 7f9d31600640 1 --2- 172.21.15.12:0/1864089959 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f9d180b53f0 0x7f9d180b78b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:04:59.164 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:59.162+0000 7f9d31600640 1 -- 172.21.15.12:0/1864089959 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8802+0+0 (secure 0 0 0) 0x7f9d2c0e64e0 con 0x7f9d34109150 2024-09-11T00:04:59.164 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:59.162+0000 7f9d33e00640 1 --2- 172.21.15.12:0/1864089959 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f9d180b53f0 0x7f9d180b78b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:04:59.164 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:59.163+0000 7f9d33e00640 1 --2- 172.21.15.12:0/1864089959 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f9d180b53f0 0x7f9d180b78b0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f9d24009bd0 tx=0x7f9d24005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:04:59.168 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:59.166+0000 7f9d31600640 1 -- 172.21.15.12:0/1864089959 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f9d2c0ad000 con 0x7f9d34109150 2024-09-11T00:04:59.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:04:59 smithi012 ceph-mon[30333]: pgmap v331: 129 pgs: 129 active+clean; 608 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 6.3 KiB/s wr, 22 op/s 2024-09-11T00:04:59.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:04:59 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T00:04:59.397 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:59.395+0000 7f9d3aa00640 1 -- 172.21.15.12:0/1864089959 --> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}) -- 0x7f9d34063590 con 0x7f9d180b53f0 2024-09-11T00:04:59.403 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:59.401+0000 7f9d31600640 1 -- 172.21.15.12:0/1864089959 <== mgr.14150 v2:172.21.15.12:6800/3442939137 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7f9d34063590 con 0x7f9d180b53f0 2024-09-11T00:04:59.403 INFO:teuthology.orchestra.run.smithi012.stdout:/volumes/qa/sv_1/40f120f8-5df9-4be4-beff-2e141f21ed64 2024-09-11T00:04:59.406 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:59.404+0000 7f9d3aa00640 1 -- 172.21.15.12:0/1864089959 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f9d180b53f0 msgr2=0x7f9d180b78b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:04:59.406 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:59.404+0000 7f9d3aa00640 1 --2- 172.21.15.12:0/1864089959 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f9d180b53f0 0x7f9d180b78b0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f9d24009bd0 tx=0x7f9d24005f90 comp rx=0 tx=0).stop 2024-09-11T00:04:59.406 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:59.404+0000 7f9d3aa00640 1 -- 172.21.15.12:0/1864089959 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9d34109150 msgr2=0x7f9d3419eb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:04:59.406 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:59.404+0000 7f9d3aa00640 1 --2- 172.21.15.12:0/1864089959 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9d34109150 0x7f9d3419eb60 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f9d2c00eb10 tx=0x7f9d2c00c2d0 comp rx=0 tx=0).stop 2024-09-11T00:04:59.406 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:59.404+0000 7f9d3aa00640 1 -- 172.21.15.12:0/1864089959 shutdown_connections 2024-09-11T00:04:59.406 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:59.404+0000 7f9d3aa00640 1 --2- 172.21.15.12:0/1864089959 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f9d180b53f0 0x7f9d180b78b0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:04:59.407 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:59.404+0000 7f9d3aa00640 1 --2- 172.21.15.12:0/1864089959 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9d34109150 0x7f9d3419eb60 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:04:59.407 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:59.404+0000 7f9d3aa00640 1 --2- 172.21.15.12:0/1864089959 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9d341087d0 0x7f9d341997c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:04:59.407 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:59.404+0000 7f9d3aa00640 1 --2- 172.21.15.12:0/1864089959 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9d34102dc0 0x7f9d34199280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:04:59.407 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:59.404+0000 7f9d3aa00640 1 -- 172.21.15.12:0/1864089959 >> 172.21.15.12:0/1864089959 conn(0x7f9d340fe830 msgr2=0x7f9d34100c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:04:59.407 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:59.404+0000 7f9d3aa00640 1 -- 172.21.15.12:0/1864089959 shutdown_connections 2024-09-11T00:04:59.407 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:04:59.404+0000 7f9d3aa00640 1 -- 172.21.15.12:0/1864089959 wait complete. 2024-09-11T00:04:59.532 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:04:59 smithi165 ceph-mon[35356]: pgmap v331: 129 pgs: 129 active+clean; 608 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 6.3 KiB/s wr, 22 op/s 2024-09-11T00:04:59.532 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:04:59 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T00:04:59.580 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:04:59 smithi178 ceph-mon[35919]: pgmap v331: 129 pgs: 129 active+clean; 608 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 6.3 KiB/s wr, 22 op/s 2024-09-11T00:04:59.580 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:04:59 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T00:05:00.065 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid ff3ad74a-6fcf-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T00:05:00.555 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:05:00 smithi012 ceph-mon[30333]: from='client.14835 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T00:05:00.581 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:05:00 smithi178 ceph-mon[35919]: from='client.14835 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T00:05:00.782 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:05:00 smithi165 ceph-mon[35356]: from='client.14835 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T00:05:01.101 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:05:01.656 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:01.653+0000 7f9823200640 1 -- 172.21.15.12:0/1979793741 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f981c10a800 msgr2=0x7f981c10ac40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:05:01.656 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:01.653+0000 7f9823200640 1 --2- 172.21.15.12:0/1979793741 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f981c10a800 0x7f981c10ac40 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f980c009900 tx=0x7f980c02a3f0 comp rx=0 tx=0).stop 2024-09-11T00:05:01.656 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:01.653+0000 7f9823200640 1 -- 172.21.15.12:0/1979793741 shutdown_connections 2024-09-11T00:05:01.656 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:01.653+0000 7f9823200640 1 --2- 172.21.15.12:0/1979793741 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f981c10b180 0x7f981c111980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:05:01.656 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:01.653+0000 7f9823200640 1 --2- 172.21.15.12:0/1979793741 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f981c10a800 0x7f981c10ac40 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:05:01.656 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:01.653+0000 7f9823200640 1 --2- 172.21.15.12:0/1979793741 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f981c104df0 0x7f981c1051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:05:01.656 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:01.653+0000 7f9823200640 1 -- 172.21.15.12:0/1979793741 >> 172.21.15.12:0/1979793741 conn(0x7f981c100840 msgr2=0x7f981c102c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:05:01.656 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:01.653+0000 7f9823200640 1 -- 172.21.15.12:0/1979793741 shutdown_connections 2024-09-11T00:05:01.656 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:01.653+0000 7f9823200640 1 -- 172.21.15.12:0/1979793741 wait complete. 2024-09-11T00:05:01.658 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:01.654+0000 7f9823200640 1 Processor -- start 2024-09-11T00:05:01.659 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:01.654+0000 7f9823200640 1 -- start start 2024-09-11T00:05:01.659 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:01.655+0000 7f9823200640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f981c104df0 0x7f981c19b1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:05:01.659 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:01.655+0000 7f9823200640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f981c10a800 0x7f981c19b6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:05:01.659 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:01.655+0000 7f9823200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f981c10b180 0x7f981c1a0a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:05:01.659 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:01.655+0000 7f9823200640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f981c114e40 con 0x7f981c104df0 2024-09-11T00:05:01.659 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:01.655+0000 7f9823200640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f981c114cc0 con 0x7f981c10b180 2024-09-11T00:05:01.659 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:01.655+0000 7f9823200640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f981c114fc0 con 0x7f981c10a800 2024-09-11T00:05:01.659 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:01.655+0000 7f9821e00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f981c104df0 0x7f981c19b1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:05:01.659 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:01.655+0000 7f9821e00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f981c104df0 0x7f981c19b1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:60272/0 (socket says 172.21.15.12:60272) 2024-09-11T00:05:01.659 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:01.655+0000 7f9821e00640 1 -- 172.21.15.12:0/2130222053 learned_addr learned my addr 172.21.15.12:0/2130222053 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:05:01.659 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:01.655+0000 7f9821400640 1 --2- 172.21.15.12:0/2130222053 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f981c10a800 0x7f981c19b6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:05:01.660 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:01.655+0000 7f9822800640 1 --2- 172.21.15.12:0/2130222053 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f981c10b180 0x7f981c1a0a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:05:01.660 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:01.655+0000 7f9821e00640 1 -- 172.21.15.12:0/2130222053 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f981c10a800 msgr2=0x7f981c19b6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:05:01.660 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:01.655+0000 7f9821e00640 1 --2- 172.21.15.12:0/2130222053 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f981c10a800 0x7f981c19b6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:05:01.660 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:01.655+0000 7f9821e00640 1 -- 172.21.15.12:0/2130222053 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f981c10b180 msgr2=0x7f981c1a0a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:05:01.660 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:01.655+0000 7f9821e00640 1 --2- 172.21.15.12:0/2130222053 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f981c10b180 0x7f981c1a0a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:05:01.660 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:01.655+0000 7f9821e00640 1 -- 172.21.15.12:0/2130222053 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f981c1a1260 con 0x7f981c104df0 2024-09-11T00:05:01.660 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:01.656+0000 7f9821e00640 1 --2- 172.21.15.12:0/2130222053 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f981c104df0 0x7f981c19b1b0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f981000e9f0 tx=0x7f981000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:05:01.660 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:01.656+0000 7f9816a00640 1 -- 172.21.15.12:0/2130222053 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f981000cd80 con 0x7f981c104df0 2024-09-11T00:05:01.660 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:01.656+0000 7f9816a00640 1 -- 172.21.15.12:0/2130222053 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f98100054d0 con 0x7f981c104df0 2024-09-11T00:05:01.661 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:01.656+0000 7f9823200640 1 -- 172.21.15.12:0/2130222053 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f981c1a1550 con 0x7f981c104df0 2024-09-11T00:05:01.661 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:01.656+0000 7f9816a00640 1 -- 172.21.15.12:0/2130222053 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9810010690 con 0x7f981c104df0 2024-09-11T00:05:01.661 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:01.656+0000 7f9823200640 1 -- 172.21.15.12:0/2130222053 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f981c1a7c10 con 0x7f981c104df0 2024-09-11T00:05:01.661 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:01.657+0000 7f9823200640 1 -- 172.21.15.12:0/2130222053 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f981c1051d0 con 0x7f981c104df0 2024-09-11T00:05:01.661 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:01.660+0000 7f9816a00640 1 -- 172.21.15.12:0/2130222053 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 17) ==== 153301+0+0 (secure 0 0 0) 0x7f9810004e30 con 0x7f981c104df0 2024-09-11T00:05:01.663 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:01.661+0000 7f9816a00640 1 --2- 172.21.15.12:0/2130222053 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f98040b5490 0x7f98040b7950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:05:01.663 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:01.661+0000 7f9816a00640 1 -- 172.21.15.12:0/2130222053 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8802+0+0 (secure 0 0 0) 0x7f98100e6560 con 0x7f981c104df0 2024-09-11T00:05:01.663 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:01.661+0000 7f9821400640 1 --2- 172.21.15.12:0/2130222053 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f98040b5490 0x7f98040b7950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:05:01.664 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:01.662+0000 7f9821400640 1 --2- 172.21.15.12:0/2130222053 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f98040b5490 0x7f98040b7950 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f980c031440 tx=0x7f980c005b50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:05:01.665 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:01.663+0000 7f9816a00640 1 -- 172.21.15.12:0/2130222053 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f9810056220 con 0x7f981c104df0 2024-09-11T00:05:01.812 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:05:01 smithi012 ceph-mon[30333]: pgmap v332: 129 pgs: 129 active+clean; 608 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 6.4 KiB/s wr, 23 op/s 2024-09-11T00:05:01.813 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:05:01 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:05:01.813 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:05:01 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:05:01.813 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:05:01 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:05:01.813 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:05:01 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:05:01.910 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:01.908+0000 7f9823200640 1 -- 172.21.15.12:0/2130222053 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f981c1a7f70 con 0x7f981c104df0 2024-09-11T00:05:01.910 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:01.909+0000 7f9816a00640 1 -- 172.21.15.12:0/2130222053 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11965 (secure 0 0 0) 0x7f98100ad020 con 0x7f981c104df0 2024-09-11T00:05:01.911 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-11T00:05:01.911 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":28,"btime":"2024-09-11T00:04:50:247323+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14577,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6835/447171591","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":447171591},{"type":"v1","addr":"172.21.15.12:6835","nonce":447171591}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14589,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/901486040","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":901486040},{"type":"v1","addr":"172.21.15.12:6837","nonce":901486040}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T00:04:04.739881+0000","modified":"2024-09-11T00:04:50.247301+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24674,"mds_1":24643,"mds_2":24638,"mds_3":24637,"mds_4":24614},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14631":{"gid":14631,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.12:6841/3069919437","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6840","nonce":3069919437},{"type":"v1","addr":"172.21.15.12:6841","nonce":3069919437}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24562":{"gid":24562,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6833/1141636112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":1141636112},{"type":"v1","addr":"172.21.15.178:6833","nonce":1141636112}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24581":{"gid":24581,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6833/3111980970","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6832","nonce":3111980970},{"type":"v1","addr":"172.21.15.165:6833","nonce":3111980970}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24607":{"gid":24607,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6835/4084868287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":4084868287},{"type":"v1","addr":"172.21.15.178:6835","nonce":4084868287}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24613":{"gid":24613,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6837/526874763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":526874763},{"type":"v1","addr":"172.21.15.178:6837","nonce":526874763}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24614":{"gid":24614,"name":"e","rank":4,"incarnation":26,"state":"up:active","state_seq":21,"addr":"172.21.15.165:6835/2425395274","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":2425395274},{"type":"v1","addr":"172.21.15.165:6835","nonce":2425395274}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24637":{"gid":24637,"name":"i","rank":3,"incarnation":24,"state":"up:active","state_seq":18,"addr":"172.21.15.178:6839/70628097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6838","nonce":70628097},{"type":"v1","addr":"172.21.15.178:6839","nonce":70628097}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24638":{"gid":24638,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":18,"addr":"172.21.15.165:6837/6215088","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6836","nonce":6215088},{"type":"v1","addr":"172.21.15.165:6837","nonce":6215088}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24643":{"gid":24643,"name":"g","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.12:6839/3124093009","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6838","nonce":3124093009},{"type":"v1","addr":"172.21.15.12:6839","nonce":3124093009}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24674":{"gid":24674,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.165:6839/501312984","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6838","nonce":501312984},{"type":"v1","addr":"172.21.15.165:6839","nonce":501312984}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24674,"qdb_cluster":[24674,24643,24638,24637,24614]},"id":1}]} 2024-09-11T00:05:01.911 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 28 2024-09-11T00:05:01.914 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:01.911+0000 7f9823200640 1 -- 172.21.15.12:0/2130222053 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f98040b5490 msgr2=0x7f98040b7950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:05:01.914 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:01.911+0000 7f9823200640 1 --2- 172.21.15.12:0/2130222053 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f98040b5490 0x7f98040b7950 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f980c031440 tx=0x7f980c005b50 comp rx=0 tx=0).stop 2024-09-11T00:05:01.914 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:01.911+0000 7f9823200640 1 -- 172.21.15.12:0/2130222053 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f981c104df0 msgr2=0x7f981c19b1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:05:01.914 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:01.911+0000 7f9823200640 1 --2- 172.21.15.12:0/2130222053 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f981c104df0 0x7f981c19b1b0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f981000e9f0 tx=0x7f981000edf0 comp rx=0 tx=0).stop 2024-09-11T00:05:01.914 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:01.912+0000 7f9823200640 1 -- 172.21.15.12:0/2130222053 shutdown_connections 2024-09-11T00:05:01.914 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:01.912+0000 7f9823200640 1 --2- 172.21.15.12:0/2130222053 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f98040b5490 0x7f98040b7950 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:05:01.914 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:01.912+0000 7f9823200640 1 --2- 172.21.15.12:0/2130222053 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f981c10b180 0x7f981c1a0a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:05:01.914 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:01.912+0000 7f9823200640 1 --2- 172.21.15.12:0/2130222053 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f981c10a800 0x7f981c19b6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:05:01.914 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:01.912+0000 7f9823200640 1 --2- 172.21.15.12:0/2130222053 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f981c104df0 0x7f981c19b1b0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:05:01.914 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:01.912+0000 7f9823200640 1 -- 172.21.15.12:0/2130222053 >> 172.21.15.12:0/2130222053 conn(0x7f981c100840 msgr2=0x7f981c103d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:05:01.914 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:01.912+0000 7f9823200640 1 -- 172.21.15.12:0/2130222053 shutdown_connections 2024-09-11T00:05:01.915 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:01.912+0000 7f9823200640 1 -- 172.21.15.12:0/2130222053 wait complete. 2024-09-11T00:05:02.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:05:01 smithi165 ceph-mon[35356]: pgmap v332: 129 pgs: 129 active+clean; 608 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 6.4 KiB/s wr, 23 op/s 2024-09-11T00:05:02.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:05:01 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:05:02.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:05:01 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:05:02.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:05:01 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:05:02.033 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:05:01 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:05:02.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:05:01 smithi178 ceph-mon[35919]: pgmap v332: 129 pgs: 129 active+clean; 608 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 6.4 KiB/s wr, 23 op/s 2024-09-11T00:05:02.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:05:01 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:05:02.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:05:01 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:05:02.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:05:01 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:05:02.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:05:01 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:05:02.671 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid ff3ad74a-6fcf-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-11T00:05:03.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:05:02 smithi165 ceph-mon[35356]: from='client.? 172.21.15.12:0/2130222053' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T00:05:03.062 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:05:02 smithi012 ceph-mon[30333]: from='client.? 172.21.15.12:0/2130222053' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T00:05:03.063 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:05:03.080 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:05:02 smithi178 ceph-mon[35919]: from='client.? 172.21.15.12:0/2130222053' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T00:05:03.600 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:03.597+0000 7fd1f8c00640 1 -- 172.21.15.12:0/1659373410 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd1f4108fa0 msgr2=0x7fd1f410f7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:05:03.600 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:03.597+0000 7fd1f8c00640 1 --2- 172.21.15.12:0/1659373410 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd1f4108fa0 0x7fd1f410f7a0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fd1ec009be0 tx=0x7fd1ec02a6d0 comp rx=0 tx=0).stop 2024-09-11T00:05:03.600 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:03.597+0000 7fd1f8c00640 1 -- 172.21.15.12:0/1659373410 shutdown_connections 2024-09-11T00:05:03.600 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:03.597+0000 7fd1f8c00640 1 --2- 172.21.15.12:0/1659373410 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd1f4108fa0 0x7fd1f410f7a0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:05:03.600 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:03.597+0000 7fd1f8c00640 1 --2- 172.21.15.12:0/1659373410 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd1f4108620 0x7fd1f4108a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:05:03.600 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:03.597+0000 7fd1f8c00640 1 --2- 172.21.15.12:0/1659373410 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd1f4102c10 0x7fd1f4102ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:05:03.600 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:03.597+0000 7fd1f8c00640 1 -- 172.21.15.12:0/1659373410 >> 172.21.15.12:0/1659373410 conn(0x7fd1f40fe660 msgr2=0x7fd1f4100a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:05:03.600 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:03.597+0000 7fd1f8c00640 1 -- 172.21.15.12:0/1659373410 shutdown_connections 2024-09-11T00:05:03.601 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:03.597+0000 7fd1f8c00640 1 -- 172.21.15.12:0/1659373410 wait complete. 2024-09-11T00:05:03.603 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:03.598+0000 7fd1f8c00640 1 Processor -- start 2024-09-11T00:05:03.603 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:03.599+0000 7fd1f8c00640 1 -- start start 2024-09-11T00:05:03.603 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:03.599+0000 7fd1f8c00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd1f4102c10 0x7fd1f419b290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:05:03.603 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:03.599+0000 7fd1f8c00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd1f4108620 0x7fd1f419b7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:05:03.603 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:03.599+0000 7fd1f8c00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd1f4108fa0 0x7fd1f41a0b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:05:03.603 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:03.599+0000 7fd1f8c00640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fd1f4112c60 con 0x7fd1f4102c10 2024-09-11T00:05:03.603 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:03.599+0000 7fd1f8c00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fd1f4112ae0 con 0x7fd1f4108620 2024-09-11T00:05:03.603 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:03.599+0000 7fd1f8c00640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fd1f4112de0 con 0x7fd1f4108fa0 2024-09-11T00:05:03.603 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:03.599+0000 7fd1f3400640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd1f4102c10 0x7fd1f419b290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:05:03.603 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:03.599+0000 7fd1f3400640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd1f4102c10 0x7fd1f419b290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:60296/0 (socket says 172.21.15.12:60296) 2024-09-11T00:05:03.604 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:03.599+0000 7fd1f3400640 1 -- 172.21.15.12:0/2897633301 learned_addr learned my addr 172.21.15.12:0/2897633301 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:05:03.604 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:03.600+0000 7fd1ebe00640 1 --2- 172.21.15.12:0/2897633301 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd1f4108620 0x7fd1f419b7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:05:03.604 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:03.600+0000 7fd1f3e00640 1 --2- 172.21.15.12:0/2897633301 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd1f4108fa0 0x7fd1f41a0b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:05:03.604 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:03.600+0000 7fd1f3400640 1 -- 172.21.15.12:0/2897633301 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd1f4108fa0 msgr2=0x7fd1f41a0b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:05:03.604 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:03.600+0000 7fd1f3400640 1 --2- 172.21.15.12:0/2897633301 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd1f4108fa0 0x7fd1f41a0b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:05:03.604 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:03.600+0000 7fd1f3400640 1 -- 172.21.15.12:0/2897633301 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd1f4108620 msgr2=0x7fd1f419b7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:05:03.604 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:03.600+0000 7fd1f3400640 1 --2- 172.21.15.12:0/2897633301 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd1f4108620 0x7fd1f419b7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:05:03.604 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:03.600+0000 7fd1f3400640 1 -- 172.21.15.12:0/2897633301 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1f41a1340 con 0x7fd1f4102c10 2024-09-11T00:05:03.605 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:03.600+0000 7fd1f3e00640 1 --2- 172.21.15.12:0/2897633301 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd1f4108fa0 0x7fd1f41a0b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:05:03.605 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:03.600+0000 7fd1f3400640 1 --2- 172.21.15.12:0/2897633301 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd1f4102c10 0x7fd1f419b290 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fd1e400ea60 tx=0x7fd1e400ee60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:05:03.605 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:03.601+0000 7fd1f0c00640 1 -- 172.21.15.12:0/2897633301 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1e400cd20 con 0x7fd1f4102c10 2024-09-11T00:05:03.605 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:03.601+0000 7fd1f8c00640 1 -- 172.21.15.12:0/2897633301 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd1f41a1630 con 0x7fd1f4102c10 2024-09-11T00:05:03.605 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:03.601+0000 7fd1f0c00640 1 -- 172.21.15.12:0/2897633301 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd1e40035f0 con 0x7fd1f4102c10 2024-09-11T00:05:03.605 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:03.601+0000 7fd1f0c00640 1 -- 172.21.15.12:0/2897633301 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1e4010690 con 0x7fd1f4102c10 2024-09-11T00:05:03.605 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:03.601+0000 7fd1f8c00640 1 -- 172.21.15.12:0/2897633301 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1f41a7ec0 con 0x7fd1f4102c10 2024-09-11T00:05:03.606 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:03.602+0000 7fd1f8c00640 1 -- 172.21.15.12:0/2897633301 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd1f4102ff0 con 0x7fd1f4102c10 2024-09-11T00:05:03.606 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:03.604+0000 7fd1f0c00640 1 -- 172.21.15.12:0/2897633301 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 17) ==== 153301+0+0 (secure 0 0 0) 0x7fd1e4004000 con 0x7fd1f4102c10 2024-09-11T00:05:03.608 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:03.606+0000 7fd1f0c00640 1 --2- 172.21.15.12:0/2897633301 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fd1d80b5490 0x7fd1d80b7950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:05:03.608 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:03.606+0000 7fd1f0c00640 1 -- 172.21.15.12:0/2897633301 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8802+0+0 (secure 0 0 0) 0x7fd1e40e64a0 con 0x7fd1f4102c10 2024-09-11T00:05:03.608 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:03.606+0000 7fd1ebe00640 1 --2- 172.21.15.12:0/2897633301 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fd1d80b5490 0x7fd1d80b7950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:05:03.609 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:03.607+0000 7fd1ebe00640 1 --2- 172.21.15.12:0/2897633301 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fd1d80b5490 0x7fd1d80b7950 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fd1f4107e90 tx=0x7fd1dc009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:05:03.613 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:03.611+0000 7fd1f0c00640 1 -- 172.21.15.12:0/2897633301 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7fd1e4056220 con 0x7fd1f4102c10 2024-09-11T00:05:03.812 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:03.810+0000 7fd1f8c00640 1 -- 172.21.15.12:0/2897633301 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd1f419c3c0 con 0x7fd1f4102c10 2024-09-11T00:05:03.814 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:03.812+0000 7fd1f0c00640 1 -- 172.21.15.12:0/2897633301 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) ==== 74+0+24236 (secure 0 0 0) 0x7fd1e40acf60 con 0x7fd1f4102c10 2024-09-11T00:05:03.814 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-11T00:05:03.814 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":81,"fsid":"ff3ad74a-6fcf-11ef-bcea-c7b262605968","created":"2024-09-10T23:55:16.317643+0000","modified":"2024-09-11T00:04:15.795652+0000","last_up_change":"2024-09-11T00:03:08.959256+0000","last_in_change":"2024-09-11T00:02:46.564631+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T23:59:10.488728+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-11T00:04:00.065618+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"74","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-11T00:04:02.689888+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"74","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-11T00:04:09.862120+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"6382c200-a912-42a1-958b-79191b21249b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6802","nonce":1237101991},{"type":"v1","addr":"172.21.15.12:6803","nonce":1237101991}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6804","nonce":1237101991},{"type":"v1","addr":"172.21.15.12:6805","nonce":1237101991}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6808","nonce":1237101991},{"type":"v1","addr":"172.21.15.12:6809","nonce":1237101991}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6806","nonce":1237101991},{"type":"v1","addr":"172.21.15.12:6807","nonce":1237101991}]},"public_addr":"172.21.15.12:6803/1237101991","cluster_addr":"172.21.15.12:6805/1237101991","heartbeat_back_addr":"172.21.15.12:6809/1237101991","heartbeat_front_addr":"172.21.15.12:6807/1237101991","state":["exists","up"]},{"osd":1,"uuid":"fe0e1d0a-bdc5-4859-aa84-9038059a45f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6800","nonce":545848456},{"type":"v1","addr":"172.21.15.165:6801","nonce":545848456}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6802","nonce":545848456},{"type":"v1","addr":"172.21.15.165:6803","nonce":545848456}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6806","nonce":545848456},{"type":"v1","addr":"172.21.15.165:6807","nonce":545848456}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6804","nonce":545848456},{"type":"v1","addr":"172.21.15.165:6805","nonce":545848456}]},"public_addr":"172.21.15.165:6801/545848456","cluster_addr":"172.21.15.165:6803/545848456","heartbeat_back_addr":"172.21.15.165:6807/545848456","heartbeat_front_addr":"172.21.15.165:6805/545848456","state":["exists","up"]},{"osd":2,"uuid":"94f640dc-5f78-4e2c-87d9-03e96a6e7ba0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6800","nonce":4131162695},{"type":"v1","addr":"172.21.15.178:6801","nonce":4131162695}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":4131162695},{"type":"v1","addr":"172.21.15.178:6803","nonce":4131162695}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":4131162695},{"type":"v1","addr":"172.21.15.178:6807","nonce":4131162695}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":4131162695},{"type":"v1","addr":"172.21.15.178:6805","nonce":4131162695}]},"public_addr":"172.21.15.178:6801/4131162695","cluster_addr":"172.21.15.178:6803/4131162695","heartbeat_back_addr":"172.21.15.178:6807/4131162695","heartbeat_front_addr":"172.21.15.178:6805/4131162695","state":["exists","up"]},{"osd":3,"uuid":"af2f64d6-c9be-44a9-b041-8d599a7c080d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6810","nonce":452464245},{"type":"v1","addr":"172.21.15.12:6811","nonce":452464245}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6812","nonce":452464245},{"type":"v1","addr":"172.21.15.12:6813","nonce":452464245}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6816","nonce":452464245},{"type":"v1","addr":"172.21.15.12:6817","nonce":452464245}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6814","nonce":452464245},{"type":"v1","addr":"172.21.15.12:6815","nonce":452464245}]},"public_addr":"172.21.15.12:6811/452464245","cluster_addr":"172.21.15.12:6813/452464245","heartbeat_back_addr":"172.21.15.12:6817/452464245","heartbeat_front_addr":"172.21.15.12:6815/452464245","state":["exists","up"]},{"osd":4,"uuid":"ad76ac70-20c0-44ae-9866-5f9cfce2334c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6808","nonce":1899879280},{"type":"v1","addr":"172.21.15.165:6809","nonce":1899879280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6810","nonce":1899879280},{"type":"v1","addr":"172.21.15.165:6811","nonce":1899879280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6814","nonce":1899879280},{"type":"v1","addr":"172.21.15.165:6815","nonce":1899879280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6812","nonce":1899879280},{"type":"v1","addr":"172.21.15.165:6813","nonce":1899879280}]},"public_addr":"172.21.15.165:6809/1899879280","cluster_addr":"172.21.15.165:6811/1899879280","heartbeat_back_addr":"172.21.15.165:6815/1899879280","heartbeat_front_addr":"172.21.15.165:6813/1899879280","state":["exists","up"]},{"osd":5,"uuid":"aafda2cc-62fe-4fcb-9a96-8732e0b501d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":3042228555},{"type":"v1","addr":"172.21.15.178:6809","nonce":3042228555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":3042228555},{"type":"v1","addr":"172.21.15.178:6811","nonce":3042228555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6814","nonce":3042228555},{"type":"v1","addr":"172.21.15.178:6815","nonce":3042228555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":3042228555},{"type":"v1","addr":"172.21.15.178:6813","nonce":3042228555}]},"public_addr":"172.21.15.178:6809/3042228555","cluster_addr":"172.21.15.178:6811/3042228555","heartbeat_back_addr":"172.21.15.178:6815/3042228555","heartbeat_front_addr":"172.21.15.178:6813/3042228555","state":["exists","up"]},{"osd":6,"uuid":"375eca52-efe3-42a5-bbb1-d7d392d71f15","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6818","nonce":3868322467},{"type":"v1","addr":"172.21.15.12:6819","nonce":3868322467}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6820","nonce":3868322467},{"type":"v1","addr":"172.21.15.12:6821","nonce":3868322467}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6824","nonce":3868322467},{"type":"v1","addr":"172.21.15.12:6825","nonce":3868322467}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6822","nonce":3868322467},{"type":"v1","addr":"172.21.15.12:6823","nonce":3868322467}]},"public_addr":"172.21.15.12:6819/3868322467","cluster_addr":"172.21.15.12:6821/3868322467","heartbeat_back_addr":"172.21.15.12:6825/3868322467","heartbeat_front_addr":"172.21.15.12:6823/3868322467","state":["exists","up"]},{"osd":7,"uuid":"0fd619f5-44c3-4cf7-ae7a-388dd798de36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6816","nonce":1143773735},{"type":"v1","addr":"172.21.15.165:6817","nonce":1143773735}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6818","nonce":1143773735},{"type":"v1","addr":"172.21.15.165:6819","nonce":1143773735}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6822","nonce":1143773735},{"type":"v1","addr":"172.21.15.165:6823","nonce":1143773735}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6820","nonce":1143773735},{"type":"v1","addr":"172.21.15.165:6821","nonce":1143773735}]},"public_addr":"172.21.15.165:6817/1143773735","cluster_addr":"172.21.15.165:6819/1143773735","heartbeat_back_addr":"172.21.15.165:6823/1143773735","heartbeat_front_addr":"172.21.15.165:6821/1143773735","state":["exists","up"]},{"osd":8,"uuid":"b03d14ce-8427-462a-9bcc-27683c549f53","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":2891445058},{"type":"v1","addr":"172.21.15.178:6817","nonce":2891445058}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6818","nonce":2891445058},{"type":"v1","addr":"172.21.15.178:6819","nonce":2891445058}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6822","nonce":2891445058},{"type":"v1","addr":"172.21.15.178:6823","nonce":2891445058}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":2891445058},{"type":"v1","addr":"172.21.15.178:6821","nonce":2891445058}]},"public_addr":"172.21.15.178:6817/2891445058","cluster_addr":"172.21.15.178:6819/2891445058","heartbeat_back_addr":"172.21.15.178:6823/2891445058","heartbeat_front_addr":"172.21.15.178:6821/2891445058","state":["exists","up"]},{"osd":9,"uuid":"81a6cf69-2a24-4a6c-9adf-6b7df38cfacb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6826","nonce":3359634903},{"type":"v1","addr":"172.21.15.12:6827","nonce":3359634903}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6828","nonce":3359634903},{"type":"v1","addr":"172.21.15.12:6829","nonce":3359634903}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6832","nonce":3359634903},{"type":"v1","addr":"172.21.15.12:6833","nonce":3359634903}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6830","nonce":3359634903},{"type":"v1","addr":"172.21.15.12:6831","nonce":3359634903}]},"public_addr":"172.21.15.12:6827/3359634903","cluster_addr":"172.21.15.12:6829/3359634903","heartbeat_back_addr":"172.21.15.12:6833/3359634903","heartbeat_front_addr":"172.21.15.12:6831/3359634903","state":["exists","up"]},{"osd":10,"uuid":"c1ad1f0a-33f4-4c32-91f6-4d030a29f1dd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6824","nonce":3642325778},{"type":"v1","addr":"172.21.15.165:6825","nonce":3642325778}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6826","nonce":3642325778},{"type":"v1","addr":"172.21.15.165:6827","nonce":3642325778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6830","nonce":3642325778},{"type":"v1","addr":"172.21.15.165:6831","nonce":3642325778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6828","nonce":3642325778},{"type":"v1","addr":"172.21.15.165:6829","nonce":3642325778}]},"public_addr":"172.21.15.165:6825/3642325778","cluster_addr":"172.21.15.165:6827/3642325778","heartbeat_back_addr":"172.21.15.165:6831/3642325778","heartbeat_front_addr":"172.21.15.165:6829/3642325778","state":["exists","up"]},{"osd":11,"uuid":"8413c9b0-680b-4d02-aeab-b3b3ff16c3d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":2937059756},{"type":"v1","addr":"172.21.15.178:6825","nonce":2937059756}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6826","nonce":2937059756},{"type":"v1","addr":"172.21.15.178:6827","nonce":2937059756}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":2937059756},{"type":"v1","addr":"172.21.15.178:6831","nonce":2937059756}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":2937059756},{"type":"v1","addr":"172.21.15.178:6829","nonce":2937059756}]},"public_addr":"172.21.15.178:6825/2937059756","cluster_addr":"172.21.15.178:6827/2937059756","heartbeat_back_addr":"172.21.15.178:6831/2937059756","heartbeat_front_addr":"172.21.15.178:6829/2937059756","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:58:16.036348+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:58:41.483150+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:59:07.107283+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:59:35.706574+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T00:00:02.555720+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T00:00:28.092847+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T00:00:57.396495+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T00:01:24.216775+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T00:01:48.807850+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T00:02:16.838565+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T00:02:43.131736+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T00:03:07.353155+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.12:6801/4177251511":"2024-09-11T23:56:17.999820+0000","172.21.15.12:6800/4177251511":"2024-09-11T23:56:17.999820+0000","172.21.15.12:0/3273692817":"2024-09-11T23:56:17.999820+0000","172.21.15.12:0/4017778158":"2024-09-11T23:56:17.999820+0000","172.21.15.12:0/2499921051":"2024-09-11T23:55:50.302721+0000","172.21.15.12:6801/760140910":"2024-09-11T23:55:50.302721+0000","172.21.15.12:6800/760140910":"2024-09-11T23:55:50.302721+0000","172.21.15.12:0/2810703302":"2024-09-11T23:56:17.999820+0000","172.21.15.12:0/2325436065":"2024-09-11T23:55:50.302721+0000","172.21.15.12:0/3137053479":"2024-09-11T23:55:50.302721+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-11T00:05:03.816 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:03.814+0000 7fd1f8c00640 1 -- 172.21.15.12:0/2897633301 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fd1d80b5490 msgr2=0x7fd1d80b7950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:05:03.817 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:03.814+0000 7fd1f8c00640 1 --2- 172.21.15.12:0/2897633301 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fd1d80b5490 0x7fd1d80b7950 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fd1f4107e90 tx=0x7fd1dc009260 comp rx=0 tx=0).stop 2024-09-11T00:05:03.817 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:03.814+0000 7fd1f8c00640 1 -- 172.21.15.12:0/2897633301 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd1f4102c10 msgr2=0x7fd1f419b290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:05:03.817 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:03.814+0000 7fd1f8c00640 1 --2- 172.21.15.12:0/2897633301 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd1f4102c10 0x7fd1f419b290 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fd1e400ea60 tx=0x7fd1e400ee60 comp rx=0 tx=0).stop 2024-09-11T00:05:03.817 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:03.815+0000 7fd1f8c00640 1 -- 172.21.15.12:0/2897633301 shutdown_connections 2024-09-11T00:05:03.817 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:03.815+0000 7fd1f8c00640 1 --2- 172.21.15.12:0/2897633301 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fd1d80b5490 0x7fd1d80b7950 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:05:03.817 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:03.815+0000 7fd1f8c00640 1 --2- 172.21.15.12:0/2897633301 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd1f4108fa0 0x7fd1f41a0b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:05:03.817 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:03.815+0000 7fd1f8c00640 1 --2- 172.21.15.12:0/2897633301 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd1f4108620 0x7fd1f419b7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:05:03.817 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:03.815+0000 7fd1f8c00640 1 --2- 172.21.15.12:0/2897633301 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd1f4102c10 0x7fd1f419b290 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:05:03.817 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:03.815+0000 7fd1f8c00640 1 -- 172.21.15.12:0/2897633301 >> 172.21.15.12:0/2897633301 conn(0x7fd1f40fe660 msgr2=0x7fd1f4104bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:05:03.817 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:03.815+0000 7fd1f8c00640 1 -- 172.21.15.12:0/2897633301 shutdown_connections 2024-09-11T00:05:03.817 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:03.815+0000 7fd1f8c00640 1 -- 172.21.15.12:0/2897633301 wait complete. 2024-09-11T00:05:04.063 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:05:03 smithi012 ceph-mon[30333]: pgmap v333: 129 pgs: 129 active+clean; 626 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.2 KiB/s wr, 20 op/s 2024-09-11T00:05:04.063 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:05:03 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:05:04.063 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:05:03 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:05:04.063 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:05:03 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:05:04.063 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:05:03 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T00:05:04.063 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:05:03 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:05:04.080 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:05:03 smithi178 ceph-mon[35919]: pgmap v333: 129 pgs: 129 active+clean; 626 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.2 KiB/s wr, 20 op/s 2024-09-11T00:05:04.080 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:05:03 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:05:04.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:05:03 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:05:04.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:05:03 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:05:04.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:05:03 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T00:05:04.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:05:03 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:05:04.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:05:03 smithi165 ceph-mon[35356]: pgmap v333: 129 pgs: 129 active+clean; 626 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.2 KiB/s wr, 20 op/s 2024-09-11T00:05:04.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:05:03 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:05:04.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:05:03 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:05:04.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:05:03 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:05:04.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:05:03 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T00:05:04.283 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:05:03 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:05:04.543 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-09-11T00:05:04.543 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid ff3ad74a-6fcf-11ef-bcea-c7b262605968 -- ceph pg dump --format=json 2024-09-11T00:05:04.925 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:05:05.063 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:05:04 smithi012 ceph-mon[30333]: from='client.? 172.21.15.12:0/2897633301' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-11T00:05:05.080 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:05:04 smithi178 ceph-mon[35919]: from='client.? 172.21.15.12:0/2897633301' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-11T00:05:05.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:05:04 smithi165 ceph-mon[35356]: from='client.? 172.21.15.12:0/2897633301' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-11T00:05:05.438 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:05.434+0000 7fac33a00640 1 -- 172.21.15.12:0/3925963983 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fac2c113b00 msgr2=0x7fac2c115ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:05:05.438 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:05.434+0000 7fac33a00640 1 --2- 172.21.15.12:0/3925963983 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fac2c113b00 0x7fac2c115ef0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fac24009ca0 tx=0x7fac2402a8c0 comp rx=0 tx=0).stop 2024-09-11T00:05:05.438 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:05.435+0000 7fac33a00640 1 -- 172.21.15.12:0/3925963983 shutdown_connections 2024-09-11T00:05:05.438 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:05.435+0000 7fac33a00640 1 --2- 172.21.15.12:0/3925963983 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fac2c113b00 0x7fac2c115ef0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:05:05.438 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:05.435+0000 7fac33a00640 1 --2- 172.21.15.12:0/3925963983 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fac2c075e60 0x7fac2c1135c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:05:05.438 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:05.435+0000 7fac33a00640 1 --2- 172.21.15.12:0/3925963983 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fac2c075540 0x7fac2c075920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:05:05.439 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:05.435+0000 7fac33a00640 1 -- 172.21.15.12:0/3925963983 >> 172.21.15.12:0/3925963983 conn(0x7fac2c0fe810 msgr2=0x7fac2c100c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:05:05.439 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:05.435+0000 7fac33a00640 1 -- 172.21.15.12:0/3925963983 shutdown_connections 2024-09-11T00:05:05.439 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:05.435+0000 7fac33a00640 1 -- 172.21.15.12:0/3925963983 wait complete. 2024-09-11T00:05:05.439 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:05.436+0000 7fac33a00640 1 Processor -- start 2024-09-11T00:05:05.439 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:05.436+0000 7fac33a00640 1 -- start start 2024-09-11T00:05:05.440 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:05.436+0000 7fac33a00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fac2c075540 0x7fac2c19f8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:05:05.440 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:05.436+0000 7fac33a00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fac2c075e60 0x7fac2c19fe20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:05:05.442 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:05.436+0000 7fac33a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fac2c113b00 0x7fac2c1a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:05:05.442 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:05.436+0000 7fac33a00640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fac2c1193b0 con 0x7fac2c075e60 2024-09-11T00:05:05.442 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:05.436+0000 7fac33a00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fac2c119230 con 0x7fac2c113b00 2024-09-11T00:05:05.442 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:05.436+0000 7fac33a00640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fac2c119530 con 0x7fac2c075540 2024-09-11T00:05:05.442 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:05.437+0000 7fac31000640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fac2c075540 0x7fac2c19f8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:05:05.442 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:05.437+0000 7fac23e00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fac2c075e60 0x7fac2c19fe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:05:05.442 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:05.437+0000 7fac31000640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fac2c075540 0x7fac2c19f8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.12:48132/0 (socket says 172.21.15.12:48132) 2024-09-11T00:05:05.442 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:05.437+0000 7fac31000640 1 -- 172.21.15.12:0/597837249 learned_addr learned my addr 172.21.15.12:0/597837249 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:05:05.442 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:05.437+0000 7fac31a00640 1 --2- 172.21.15.12:0/597837249 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fac2c113b00 0x7fac2c1a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:05:05.443 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:05.437+0000 7fac31000640 1 -- 172.21.15.12:0/597837249 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fac2c113b00 msgr2=0x7fac2c1a51c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:05:05.443 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:05.437+0000 7fac31000640 1 --2- 172.21.15.12:0/597837249 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fac2c113b00 0x7fac2c1a51c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:05:05.443 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:05.437+0000 7fac31000640 1 -- 172.21.15.12:0/597837249 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fac2c075e60 msgr2=0x7fac2c19fe20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:05:05.443 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:05.437+0000 7fac31000640 1 --2- 172.21.15.12:0/597837249 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fac2c075e60 0x7fac2c19fe20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:05:05.443 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:05.437+0000 7fac31000640 1 -- 172.21.15.12:0/597837249 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac2c1a5990 con 0x7fac2c075540 2024-09-11T00:05:05.443 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:05.437+0000 7fac31a00640 1 --2- 172.21.15.12:0/597837249 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fac2c113b00 0x7fac2c1a51c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T00:05:05.443 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:05.438+0000 7fac31000640 1 --2- 172.21.15.12:0/597837249 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fac2c075540 0x7fac2c19f8e0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fac1c00e9f0 tx=0x7fac1c00eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:05:05.443 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:05.438+0000 7fac2a000640 1 -- 172.21.15.12:0/597837249 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac1c00cd80 con 0x7fac2c075540 2024-09-11T00:05:05.443 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:05.438+0000 7fac2a000640 1 -- 172.21.15.12:0/597837249 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fac1c0108e0 con 0x7fac2c075540 2024-09-11T00:05:05.443 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:05.438+0000 7fac2a000640 1 -- 172.21.15.12:0/597837249 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac1c018980 con 0x7fac2c075540 2024-09-11T00:05:05.443 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:05.438+0000 7fac33a00640 1 -- 172.21.15.12:0/597837249 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fac2c1a5c80 con 0x7fac2c075540 2024-09-11T00:05:05.443 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:05.438+0000 7fac33a00640 1 -- 172.21.15.12:0/597837249 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fac2c1ac510 con 0x7fac2c075540 2024-09-11T00:05:05.444 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:05.439+0000 7fac33a00640 1 -- 172.21.15.12:0/597837249 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fac2c075920 con 0x7fac2c075540 2024-09-11T00:05:05.444 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:05.442+0000 7fac2a000640 1 -- 172.21.15.12:0/597837249 <== mon.1 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 17) ==== 153301+0+0 (secure 0 0 0) 0x7fac1c010430 con 0x7fac2c075540 2024-09-11T00:05:05.445 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:05.443+0000 7fac2a000640 1 --2- 172.21.15.12:0/597837249 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fac100b53f0 0x7fac100b78b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:05:05.445 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:05.443+0000 7fac2a000640 1 -- 172.21.15.12:0/597837249 <== mon.1 v2:172.21.15.178:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8802+0+0 (secure 0 0 0) 0x7fac1c0e6800 con 0x7fac2c075540 2024-09-11T00:05:05.445 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:05.443+0000 7fac23e00640 1 --2- 172.21.15.12:0/597837249 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fac100b53f0 0x7fac100b78b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:05:05.446 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:05.444+0000 7fac23e00640 1 --2- 172.21.15.12:0/597837249 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fac100b53f0 0x7fac100b78b0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fac2c1162d0 tx=0x7fac14005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:05:05.446 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:05.445+0000 7fac2a000640 1 -- 172.21.15.12:0/597837249 <== mon.1 v2:172.21.15.178:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7fac1c0ad2c0 con 0x7fac2c075540 2024-09-11T00:05:05.639 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:05.637+0000 7fac33a00640 1 -- 172.21.15.12:0/597837249 --> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fac2c100b30 con 0x7fac100b53f0 2024-09-11T00:05:05.646 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:05.645+0000 7fac2a000640 1 -- 172.21.15.12:0/597837249 <== mgr.14150 v2:172.21.15.12:6800/3442939137 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340372 (secure 0 0 0) 0x7fac2c100b30 con 0x7fac100b53f0 2024-09-11T00:05:05.652 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-11T00:05:05.654 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-11T00:05:05.655 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:05.650+0000 7fac33a00640 1 -- 172.21.15.12:0/597837249 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fac100b53f0 msgr2=0x7fac100b78b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:05:05.655 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:05.650+0000 7fac33a00640 1 --2- 172.21.15.12:0/597837249 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fac100b53f0 0x7fac100b78b0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fac2c1162d0 tx=0x7fac14005f90 comp rx=0 tx=0).stop 2024-09-11T00:05:05.656 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:05.650+0000 7fac33a00640 1 -- 172.21.15.12:0/597837249 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fac2c075540 msgr2=0x7fac2c19f8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:05:05.656 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:05.650+0000 7fac33a00640 1 --2- 172.21.15.12:0/597837249 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fac2c075540 0x7fac2c19f8e0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fac1c00e9f0 tx=0x7fac1c00eeb0 comp rx=0 tx=0).stop 2024-09-11T00:05:05.656 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:05.650+0000 7fac33a00640 1 -- 172.21.15.12:0/597837249 shutdown_connections 2024-09-11T00:05:05.656 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:05.650+0000 7fac33a00640 1 --2- 172.21.15.12:0/597837249 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fac100b53f0 0x7fac100b78b0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:05:05.656 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:05.650+0000 7fac33a00640 1 --2- 172.21.15.12:0/597837249 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fac2c113b00 0x7fac2c1a51c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:05:05.656 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:05.650+0000 7fac33a00640 1 --2- 172.21.15.12:0/597837249 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fac2c075e60 0x7fac2c19fe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:05:05.656 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:05.650+0000 7fac33a00640 1 --2- 172.21.15.12:0/597837249 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fac2c075540 0x7fac2c19f8e0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:05:05.656 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:05.650+0000 7fac33a00640 1 -- 172.21.15.12:0/597837249 >> 172.21.15.12:0/597837249 conn(0x7fac2c0fe810 msgr2=0x7fac2c0ff410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:05:05.656 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:05.650+0000 7fac33a00640 1 -- 172.21.15.12:0/597837249 shutdown_connections 2024-09-11T00:05:05.656 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:05.650+0000 7fac33a00640 1 -- 172.21.15.12:0/597837249 wait complete. 2024-09-11T00:05:06.005 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:05:05 smithi012 ceph-mon[30333]: pgmap v334: 129 pgs: 129 active+clean; 626 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 4.2 KiB/s wr, 11 op/s 2024-09-11T00:05:06.113 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:05:05 smithi178 ceph-mon[35919]: pgmap v334: 129 pgs: 129 active+clean; 626 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 4.2 KiB/s wr, 11 op/s 2024-09-11T00:05:06.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:05:05 smithi165 ceph-mon[35356]: pgmap v334: 129 pgs: 129 active+clean; 626 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 4.2 KiB/s wr, 11 op/s 2024-09-11T00:05:06.371 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":334,"stamp":"2024-09-11T00:05:04.104331+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":641147,"num_objects":102,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":220,"num_read_kb":194,"num_write":355,"num_write_kb":2268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":295,"ondisk_log_size":295,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":378724,"kb_used_data":12836,"kb_used_omap":18,"kb_used_meta":365677,"kb_avail":1124563100,"statfs":{"total":1151940427776,"available":1151552614400,"internally_reserved":0,"allocated":13144064,"data_stored":7278947,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19142,"internal_metadata":374453562},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":40515,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":120,"num_read_kb":106,"num_write":23,"num_write_kb":51,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003588"},"pg_stats":[{"pgid":"2.1f","version":"81'13","reported_seq":57,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T00:04:58.270062+0000","last_change":"2024-09-11T00:04:01.820872+0000","last_active":"2024-09-11T00:04:58.270062+0000","last_peered":"2024-09-11T00:04:58.270062+0000","last_clean":"2024-09-11T00:04:58.270062+0000","last_became_active":"2024-09-11T00:04:01.820404+0000","last_became_peered":"2024-09-11T00:04:01.820404+0000","last_unstale":"2024-09-11T00:04:58.270062+0000","last_undegraded":"2024-09-11T00:04:58.270062+0000","last_fullsized":"2024-09-11T00:04:58.270062+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_clean_scrub_stamp":"2024-09-11T00:04:00.794959+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:15:38.648289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":11,"num_read_kb":11,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T00:04:16.752091+0000","last_change":"2024-09-11T00:04:03.842131+0000","last_active":"2024-09-11T00:04:16.752091+0000","last_peered":"2024-09-11T00:04:16.752091+0000","last_clean":"2024-09-11T00:04:16.752091+0000","last_became_active":"2024-09-11T00:04:03.841555+0000","last_became_peered":"2024-09-11T00:04:03.841555+0000","last_unstale":"2024-09-11T00:04:16.752091+0000","last_undegraded":"2024-09-11T00:04:16.752091+0000","last_fullsized":"2024-09-11T00:04:16.752091+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:39:28.301596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":21,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T00:04:16.882034+0000","last_change":"2024-09-11T00:04:11.872109+0000","last_active":"2024-09-11T00:04:16.882034+0000","last_peered":"2024-09-11T00:04:16.882034+0000","last_clean":"2024-09-11T00:04:16.882034+0000","last_became_active":"2024-09-11T00:04:11.871279+0000","last_became_peered":"2024-09-11T00:04:11.871279+0000","last_unstale":"2024-09-11T00:04:16.882034+0000","last_undegraded":"2024-09-11T00:04:16.882034+0000","last_fullsized":"2024-09-11T00:04:16.882034+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_clean_scrub_stamp":"2024-09-11T00:04:10.845136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:13:32.837555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"81'2","reported_seq":35,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T00:04:16.752110+0000","last_change":"2024-09-11T00:04:01.942288+0000","last_active":"2024-09-11T00:04:16.752110+0000","last_peered":"2024-09-11T00:04:16.752110+0000","last_clean":"2024-09-11T00:04:16.752110+0000","last_became_active":"2024-09-11T00:04:01.941838+0000","last_became_peered":"2024-09-11T00:04:01.941838+0000","last_unstale":"2024-09-11T00:04:16.752110+0000","last_undegraded":"2024-09-11T00:04:16.752110+0000","last_fullsized":"2024-09-11T00:04:16.752110+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_clean_scrub_stamp":"2024-09-11T00:04:00.794959+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:48:44.867267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T00:04:17.305155+0000","last_change":"2024-09-11T00:04:03.839951+0000","last_active":"2024-09-11T00:04:17.305155+0000","last_peered":"2024-09-11T00:04:17.305155+0000","last_clean":"2024-09-11T00:04:17.305155+0000","last_became_active":"2024-09-11T00:04:03.839591+0000","last_became_peered":"2024-09-11T00:04:03.839591+0000","last_unstale":"2024-09-11T00:04:17.305155+0000","last_undegraded":"2024-09-11T00:04:17.305155+0000","last_fullsized":"2024-09-11T00:04:17.305155+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:12:49.603310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":21,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T00:04:17.435436+0000","last_change":"2024-09-11T00:04:11.872026+0000","last_active":"2024-09-11T00:04:17.435436+0000","last_peered":"2024-09-11T00:04:17.435436+0000","last_clean":"2024-09-11T00:04:17.435436+0000","last_became_active":"2024-09-11T00:04:11.871699+0000","last_became_peered":"2024-09-11T00:04:11.871699+0000","last_unstale":"2024-09-11T00:04:17.435436+0000","last_undegraded":"2024-09-11T00:04:17.435436+0000","last_fullsized":"2024-09-11T00:04:17.435436+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_clean_scrub_stamp":"2024-09-11T00:04:10.845136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:07:47.757223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"81'12","reported_seq":46,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T00:04:50.254784+0000","last_change":"2024-09-11T00:04:01.819942+0000","last_active":"2024-09-11T00:04:50.254784+0000","last_peered":"2024-09-11T00:04:50.254784+0000","last_clean":"2024-09-11T00:04:50.254784+0000","last_became_active":"2024-09-11T00:04:01.819337+0000","last_became_peered":"2024-09-11T00:04:01.819337+0000","last_unstale":"2024-09-11T00:04:50.254784+0000","last_undegraded":"2024-09-11T00:04:50.254784+0000","last_fullsized":"2024-09-11T00:04:50.254784+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_clean_scrub_stamp":"2024-09-11T00:04:00.794959+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:22:26.947067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T00:04:17.435640+0000","last_change":"2024-09-11T00:04:03.839276+0000","last_active":"2024-09-11T00:04:17.435640+0000","last_peered":"2024-09-11T00:04:17.435640+0000","last_clean":"2024-09-11T00:04:17.435640+0000","last_became_active":"2024-09-11T00:04:03.838843+0000","last_became_peered":"2024-09-11T00:04:03.838843+0000","last_unstale":"2024-09-11T00:04:17.435640+0000","last_undegraded":"2024-09-11T00:04:17.435640+0000","last_fullsized":"2024-09-11T00:04:17.435640+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:26:42.640628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":21,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T00:04:17.436523+0000","last_change":"2024-09-11T00:04:11.872345+0000","last_active":"2024-09-11T00:04:17.436523+0000","last_peered":"2024-09-11T00:04:17.436523+0000","last_clean":"2024-09-11T00:04:17.436523+0000","last_became_active":"2024-09-11T00:04:11.871994+0000","last_became_peered":"2024-09-11T00:04:11.871994+0000","last_unstale":"2024-09-11T00:04:17.436523+0000","last_undegraded":"2024-09-11T00:04:17.436523+0000","last_fullsized":"2024-09-11T00:04:17.436523+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_clean_scrub_stamp":"2024-09-11T00:04:10.845136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:14:23.906778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"81'5","reported_seq":39,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T00:04:49.248998+0000","last_change":"2024-09-11T00:04:01.819581+0000","last_active":"2024-09-11T00:04:49.248998+0000","last_peered":"2024-09-11T00:04:49.248998+0000","last_clean":"2024-09-11T00:04:49.248998+0000","last_became_active":"2024-09-11T00:04:01.818699+0000","last_became_peered":"2024-09-11T00:04:01.818699+0000","last_unstale":"2024-09-11T00:04:49.248998+0000","last_undegraded":"2024-09-11T00:04:49.248998+0000","last_fullsized":"2024-09-11T00:04:49.248998+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_clean_scrub_stamp":"2024-09-11T00:04:00.794959+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:07:24.540825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T00:04:17.436276+0000","last_change":"2024-09-11T00:04:03.835107+0000","last_active":"2024-09-11T00:04:17.436276+0000","last_peered":"2024-09-11T00:04:17.436276+0000","last_clean":"2024-09-11T00:04:17.436276+0000","last_became_active":"2024-09-11T00:04:03.834782+0000","last_became_peered":"2024-09-11T00:04:03.834782+0000","last_unstale":"2024-09-11T00:04:17.436276+0000","last_undegraded":"2024-09-11T00:04:17.436276+0000","last_fullsized":"2024-09-11T00:04:17.436276+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:46:54.472270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":21,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T00:04:17.435721+0000","last_change":"2024-09-11T00:04:11.874168+0000","last_active":"2024-09-11T00:04:17.435721+0000","last_peered":"2024-09-11T00:04:17.435721+0000","last_clean":"2024-09-11T00:04:17.435721+0000","last_became_active":"2024-09-11T00:04:11.873810+0000","last_became_peered":"2024-09-11T00:04:11.873810+0000","last_unstale":"2024-09-11T00:04:17.435721+0000","last_undegraded":"2024-09-11T00:04:17.435721+0000","last_fullsized":"2024-09-11T00:04:17.435721+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_clean_scrub_stamp":"2024-09-11T00:04:10.845136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:06:05.130815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T00:04:15.912270+0000","last_change":"2024-09-11T00:04:03.834016+0000","last_active":"2024-09-11T00:04:15.912270+0000","last_peered":"2024-09-11T00:04:15.912270+0000","last_clean":"2024-09-11T00:04:15.912270+0000","last_became_active":"2024-09-11T00:04:03.833374+0000","last_became_peered":"2024-09-11T00:04:03.833374+0000","last_unstale":"2024-09-11T00:04:15.912270+0000","last_undegraded":"2024-09-11T00:04:15.912270+0000","last_fullsized":"2024-09-11T00:04:15.912270+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:13:34.068594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"81'4","reported_seq":38,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T00:04:46.232389+0000","last_change":"2024-09-11T00:04:01.821173+0000","last_active":"2024-09-11T00:04:46.232389+0000","last_peered":"2024-09-11T00:04:46.232389+0000","last_clean":"2024-09-11T00:04:46.232389+0000","last_became_active":"2024-09-11T00:04:01.820615+0000","last_became_peered":"2024-09-11T00:04:01.820615+0000","last_unstale":"2024-09-11T00:04:46.232389+0000","last_undegraded":"2024-09-11T00:04:46.232389+0000","last_fullsized":"2024-09-11T00:04:46.232389+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_clean_scrub_stamp":"2024-09-11T00:04:00.794959+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:22:11.842837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":21,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T00:04:17.436132+0000","last_change":"2024-09-11T00:04:12.077637+0000","last_active":"2024-09-11T00:04:17.436132+0000","last_peered":"2024-09-11T00:04:17.436132+0000","last_clean":"2024-09-11T00:04:17.436132+0000","last_became_active":"2024-09-11T00:04:12.076450+0000","last_became_peered":"2024-09-11T00:04:12.076450+0000","last_unstale":"2024-09-11T00:04:17.436132+0000","last_undegraded":"2024-09-11T00:04:17.436132+0000","last_fullsized":"2024-09-11T00:04:17.436132+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_clean_scrub_stamp":"2024-09-11T00:04:10.845136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:50:54.870468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T00:04:16.751592+0000","last_change":"2024-09-11T00:04:03.846398+0000","last_active":"2024-09-11T00:04:16.751592+0000","last_peered":"2024-09-11T00:04:16.751592+0000","last_clean":"2024-09-11T00:04:16.751592+0000","last_became_active":"2024-09-11T00:04:03.846051+0000","last_became_peered":"2024-09-11T00:04:03.846051+0000","last_unstale":"2024-09-11T00:04:16.751592+0000","last_undegraded":"2024-09-11T00:04:16.751592+0000","last_fullsized":"2024-09-11T00:04:16.751592+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:00:38.506122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"81'3","reported_seq":36,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T00:04:48.244416+0000","last_change":"2024-09-11T00:04:01.818083+0000","last_active":"2024-09-11T00:04:48.244416+0000","last_peered":"2024-09-11T00:04:48.244416+0000","last_clean":"2024-09-11T00:04:48.244416+0000","last_became_active":"2024-09-11T00:04:01.817253+0000","last_became_peered":"2024-09-11T00:04:01.817253+0000","last_unstale":"2024-09-11T00:04:48.244416+0000","last_undegraded":"2024-09-11T00:04:48.244416+0000","last_fullsized":"2024-09-11T00:04:48.244416+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_clean_scrub_stamp":"2024-09-11T00:04:00.794959+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:35:23.429074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"81'1","reported_seq":22,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T00:04:17.436272+0000","last_change":"2024-09-11T00:04:11.879582+0000","last_active":"2024-09-11T00:04:17.436272+0000","last_peered":"2024-09-11T00:04:17.436272+0000","last_clean":"2024-09-11T00:04:17.436272+0000","last_became_active":"2024-09-11T00:04:11.879098+0000","last_became_peered":"2024-09-11T00:04:11.879098+0000","last_unstale":"2024-09-11T00:04:17.436272+0000","last_undegraded":"2024-09-11T00:04:17.436272+0000","last_fullsized":"2024-09-11T00:04:17.436272+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_clean_scrub_stamp":"2024-09-11T00:04:10.845136+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:06.292841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T00:04:15.869547+0000","last_change":"2024-09-11T00:04:03.834615+0000","last_active":"2024-09-11T00:04:15.869547+0000","last_peered":"2024-09-11T00:04:15.869547+0000","last_clean":"2024-09-11T00:04:15.869547+0000","last_became_active":"2024-09-11T00:04:03.834284+0000","last_became_peered":"2024-09-11T00:04:03.834284+0000","last_unstale":"2024-09-11T00:04:15.869547+0000","last_undegraded":"2024-09-11T00:04:15.869547+0000","last_fullsized":"2024-09-11T00:04:15.869547+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:24:09.542774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"81'7","reported_seq":40,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T00:04:49.250027+0000","last_change":"2024-09-11T00:04:01.818156+0000","last_active":"2024-09-11T00:04:49.250027+0000","last_peered":"2024-09-11T00:04:49.250027+0000","last_clean":"2024-09-11T00:04:49.250027+0000","last_became_active":"2024-09-11T00:04:01.817820+0000","last_became_peered":"2024-09-11T00:04:01.817820+0000","last_unstale":"2024-09-11T00:04:49.250027+0000","last_undegraded":"2024-09-11T00:04:49.250027+0000","last_fullsized":"2024-09-11T00:04:49.250027+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_clean_scrub_stamp":"2024-09-11T00:04:00.794959+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:15:46.304276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":21,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T00:04:17.435835+0000","last_change":"2024-09-11T00:04:11.877704+0000","last_active":"2024-09-11T00:04:17.435835+0000","last_peered":"2024-09-11T00:04:17.435835+0000","last_clean":"2024-09-11T00:04:17.435835+0000","last_became_active":"2024-09-11T00:04:11.877320+0000","last_became_peered":"2024-09-11T00:04:11.877320+0000","last_unstale":"2024-09-11T00:04:17.435835+0000","last_undegraded":"2024-09-11T00:04:17.435835+0000","last_fullsized":"2024-09-11T00:04:17.435835+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_clean_scrub_stamp":"2024-09-11T00:04:10.845136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:31:35.046012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T00:04:17.435622+0000","last_change":"2024-09-11T00:04:03.828240+0000","last_active":"2024-09-11T00:04:17.435622+0000","last_peered":"2024-09-11T00:04:17.435622+0000","last_clean":"2024-09-11T00:04:17.435622+0000","last_became_active":"2024-09-11T00:04:03.827890+0000","last_became_peered":"2024-09-11T00:04:03.827890+0000","last_unstale":"2024-09-11T00:04:17.435622+0000","last_undegraded":"2024-09-11T00:04:17.435622+0000","last_fullsized":"2024-09-11T00:04:17.435622+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:43:12.146717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"81'2","reported_seq":35,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T00:04:46.232647+0000","last_change":"2024-09-11T00:04:01.942805+0000","last_active":"2024-09-11T00:04:46.232647+0000","last_peered":"2024-09-11T00:04:46.232647+0000","last_clean":"2024-09-11T00:04:46.232647+0000","last_became_active":"2024-09-11T00:04:01.942429+0000","last_became_peered":"2024-09-11T00:04:01.942429+0000","last_unstale":"2024-09-11T00:04:46.232647+0000","last_undegraded":"2024-09-11T00:04:46.232647+0000","last_fullsized":"2024-09-11T00:04:46.232647+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_clean_scrub_stamp":"2024-09-11T00:04:00.794959+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:51:22.403825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":21,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T00:04:17.436458+0000","last_change":"2024-09-11T00:04:12.077246+0000","last_active":"2024-09-11T00:04:17.436458+0000","last_peered":"2024-09-11T00:04:17.436458+0000","last_clean":"2024-09-11T00:04:17.436458+0000","last_became_active":"2024-09-11T00:04:12.076481+0000","last_became_peered":"2024-09-11T00:04:12.076481+0000","last_unstale":"2024-09-11T00:04:17.436458+0000","last_undegraded":"2024-09-11T00:04:17.436458+0000","last_fullsized":"2024-09-11T00:04:17.436458+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_clean_scrub_stamp":"2024-09-11T00:04:10.845136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:55:54.347334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T00:04:17.304877+0000","last_change":"2024-09-11T00:04:03.838431+0000","last_active":"2024-09-11T00:04:17.304877+0000","last_peered":"2024-09-11T00:04:17.304877+0000","last_clean":"2024-09-11T00:04:17.304877+0000","last_became_active":"2024-09-11T00:04:03.834451+0000","last_became_peered":"2024-09-11T00:04:03.834451+0000","last_unstale":"2024-09-11T00:04:17.304877+0000","last_undegraded":"2024-09-11T00:04:17.304877+0000","last_fullsized":"2024-09-11T00:04:17.304877+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:23:01.594299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"81'7","reported_seq":41,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T00:04:49.250641+0000","last_change":"2024-09-11T00:04:02.588834+0000","last_active":"2024-09-11T00:04:49.250641+0000","last_peered":"2024-09-11T00:04:49.250641+0000","last_clean":"2024-09-11T00:04:49.250641+0000","last_became_active":"2024-09-11T00:04:02.588477+0000","last_became_peered":"2024-09-11T00:04:02.588477+0000","last_unstale":"2024-09-11T00:04:49.250641+0000","last_undegraded":"2024-09-11T00:04:49.250641+0000","last_fullsized":"2024-09-11T00:04:49.250641+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_clean_scrub_stamp":"2024-09-11T00:04:00.794959+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:31:21.805591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":21,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T00:04:16.882086+0000","last_change":"2024-09-11T00:04:11.863381+0000","last_active":"2024-09-11T00:04:16.882086+0000","last_peered":"2024-09-11T00:04:16.882086+0000","last_clean":"2024-09-11T00:04:16.882086+0000","last_became_active":"2024-09-11T00:04:11.862944+0000","last_became_peered":"2024-09-11T00:04:11.862944+0000","last_unstale":"2024-09-11T00:04:16.882086+0000","last_undegraded":"2024-09-11T00:04:16.882086+0000","last_fullsized":"2024-09-11T00:04:16.882086+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_clean_scrub_stamp":"2024-09-11T00:04:10.845136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:00:14.185992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T00:04:16.751871+0000","last_change":"2024-09-11T00:04:03.829750+0000","last_active":"2024-09-11T00:04:16.751871+0000","last_peered":"2024-09-11T00:04:16.751871+0000","last_clean":"2024-09-11T00:04:16.751871+0000","last_became_active":"2024-09-11T00:04:03.828166+0000","last_became_peered":"2024-09-11T00:04:03.828166+0000","last_unstale":"2024-09-11T00:04:16.751871+0000","last_undegraded":"2024-09-11T00:04:16.751871+0000","last_fullsized":"2024-09-11T00:04:16.751871+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:55:01.667075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"81'10","reported_seq":43,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T00:04:54.850758+0000","last_change":"2024-09-11T00:04:01.818083+0000","last_active":"2024-09-11T00:04:54.850758+0000","last_peered":"2024-09-11T00:04:54.850758+0000","last_clean":"2024-09-11T00:04:54.850758+0000","last_became_active":"2024-09-11T00:04:01.817784+0000","last_became_peered":"2024-09-11T00:04:01.817784+0000","last_unstale":"2024-09-11T00:04:54.850758+0000","last_undegraded":"2024-09-11T00:04:54.850758+0000","last_fullsized":"2024-09-11T00:04:54.850758+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_clean_scrub_stamp":"2024-09-11T00:04:00.794959+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:38:55.466913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":21,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T00:04:16.751305+0000","last_change":"2024-09-11T00:04:12.077053+0000","last_active":"2024-09-11T00:04:16.751305+0000","last_peered":"2024-09-11T00:04:16.751305+0000","last_clean":"2024-09-11T00:04:16.751305+0000","last_became_active":"2024-09-11T00:04:12.076632+0000","last_became_peered":"2024-09-11T00:04:12.076632+0000","last_unstale":"2024-09-11T00:04:16.751305+0000","last_undegraded":"2024-09-11T00:04:16.751305+0000","last_fullsized":"2024-09-11T00:04:16.751305+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_clean_scrub_stamp":"2024-09-11T00:04:10.845136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:19:38.321429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T00:04:15.869722+0000","last_change":"2024-09-11T00:04:03.830316+0000","last_active":"2024-09-11T00:04:15.869722+0000","last_peered":"2024-09-11T00:04:15.869722+0000","last_clean":"2024-09-11T00:04:15.869722+0000","last_became_active":"2024-09-11T00:04:03.829932+0000","last_became_peered":"2024-09-11T00:04:03.829932+0000","last_unstale":"2024-09-11T00:04:15.869722+0000","last_undegraded":"2024-09-11T00:04:15.869722+0000","last_fullsized":"2024-09-11T00:04:15.869722+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:06:14.546651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"81'4","reported_seq":38,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T00:04:49.250496+0000","last_change":"2024-09-11T00:04:01.943004+0000","last_active":"2024-09-11T00:04:49.250496+0000","last_peered":"2024-09-11T00:04:49.250496+0000","last_clean":"2024-09-11T00:04:49.250496+0000","last_became_active":"2024-09-11T00:04:01.942432+0000","last_became_peered":"2024-09-11T00:04:01.942432+0000","last_unstale":"2024-09-11T00:04:49.250496+0000","last_undegraded":"2024-09-11T00:04:49.250496+0000","last_fullsized":"2024-09-11T00:04:49.250496+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_clean_scrub_stamp":"2024-09-11T00:04:00.794959+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:52:50.472596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":21,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T00:04:15.869457+0000","last_change":"2024-09-11T00:04:11.866778+0000","last_active":"2024-09-11T00:04:15.869457+0000","last_peered":"2024-09-11T00:04:15.869457+0000","last_clean":"2024-09-11T00:04:15.869457+0000","last_became_active":"2024-09-11T00:04:11.866333+0000","last_became_peered":"2024-09-11T00:04:11.866333+0000","last_unstale":"2024-09-11T00:04:15.869457+0000","last_undegraded":"2024-09-11T00:04:15.869457+0000","last_fullsized":"2024-09-11T00:04:15.869457+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_clean_scrub_stamp":"2024-09-11T00:04:10.845136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:33.680119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T00:04:15.869851+0000","last_change":"2024-09-11T00:04:03.832481+0000","last_active":"2024-09-11T00:04:15.869851+0000","last_peered":"2024-09-11T00:04:15.869851+0000","last_clean":"2024-09-11T00:04:15.869851+0000","last_became_active":"2024-09-11T00:04:03.832162+0000","last_became_peered":"2024-09-11T00:04:03.832162+0000","last_unstale":"2024-09-11T00:04:15.869851+0000","last_undegraded":"2024-09-11T00:04:15.869851+0000","last_fullsized":"2024-09-11T00:04:15.869851+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:32:52.550128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"81'1","reported_seq":34,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T00:04:17.435694+0000","last_change":"2024-09-11T00:04:01.943647+0000","last_active":"2024-09-11T00:04:17.435694+0000","last_peered":"2024-09-11T00:04:17.435694+0000","last_clean":"2024-09-11T00:04:17.435694+0000","last_became_active":"2024-09-11T00:04:01.942714+0000","last_became_peered":"2024-09-11T00:04:01.942714+0000","last_unstale":"2024-09-11T00:04:17.435694+0000","last_undegraded":"2024-09-11T00:04:17.435694+0000","last_fullsized":"2024-09-11T00:04:17.435694+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_clean_scrub_stamp":"2024-09-11T00:04:00.794959+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:22:23.687268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":21,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T00:04:17.435685+0000","last_change":"2024-09-11T00:04:12.077133+0000","last_active":"2024-09-11T00:04:17.435685+0000","last_peered":"2024-09-11T00:04:17.435685+0000","last_clean":"2024-09-11T00:04:17.435685+0000","last_became_active":"2024-09-11T00:04:12.076657+0000","last_became_peered":"2024-09-11T00:04:12.076657+0000","last_unstale":"2024-09-11T00:04:17.435685+0000","last_undegraded":"2024-09-11T00:04:17.435685+0000","last_fullsized":"2024-09-11T00:04:17.435685+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_clean_scrub_stamp":"2024-09-11T00:04:10.845136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:05:40.643405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T00:04:15.870093+0000","last_change":"2024-09-11T00:04:03.835454+0000","last_active":"2024-09-11T00:04:15.870093+0000","last_peered":"2024-09-11T00:04:15.870093+0000","last_clean":"2024-09-11T00:04:15.870093+0000","last_became_active":"2024-09-11T00:04:03.835122+0000","last_became_peered":"2024-09-11T00:04:03.835122+0000","last_unstale":"2024-09-11T00:04:15.870093+0000","last_undegraded":"2024-09-11T00:04:15.870093+0000","last_fullsized":"2024-09-11T00:04:15.870093+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:25:49.672766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"81'6","reported_seq":52,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T00:04:59.272075+0000","last_change":"2024-09-11T00:04:01.823738+0000","last_active":"2024-09-11T00:04:59.272075+0000","last_peered":"2024-09-11T00:04:59.272075+0000","last_clean":"2024-09-11T00:04:59.272075+0000","last_became_active":"2024-09-11T00:04:01.823320+0000","last_became_peered":"2024-09-11T00:04:01.823320+0000","last_unstale":"2024-09-11T00:04:59.272075+0000","last_undegraded":"2024-09-11T00:04:59.272075+0000","last_fullsized":"2024-09-11T00:04:59.272075+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_clean_scrub_stamp":"2024-09-11T00:04:00.794959+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:34:30.088454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":13,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":21,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T00:04:15.869640+0000","last_change":"2024-09-11T00:04:11.863906+0000","last_active":"2024-09-11T00:04:15.869640+0000","last_peered":"2024-09-11T00:04:15.869640+0000","last_clean":"2024-09-11T00:04:15.869640+0000","last_became_active":"2024-09-11T00:04:11.863571+0000","last_became_peered":"2024-09-11T00:04:11.863571+0000","last_unstale":"2024-09-11T00:04:15.869640+0000","last_undegraded":"2024-09-11T00:04:15.869640+0000","last_fullsized":"2024-09-11T00:04:15.869640+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_clean_scrub_stamp":"2024-09-11T00:04:10.845136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:04:40.288886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T00:04:17.436310+0000","last_change":"2024-09-11T00:04:03.826623+0000","last_active":"2024-09-11T00:04:17.436310+0000","last_peered":"2024-09-11T00:04:17.436310+0000","last_clean":"2024-09-11T00:04:17.436310+0000","last_became_active":"2024-09-11T00:04:03.826202+0000","last_became_peered":"2024-09-11T00:04:03.826202+0000","last_unstale":"2024-09-11T00:04:17.436310+0000","last_undegraded":"2024-09-11T00:04:17.436310+0000","last_fullsized":"2024-09-11T00:04:17.436310+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:23:13.222654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"81'5","reported_seq":49,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T00:04:58.270161+0000","last_change":"2024-09-11T00:04:01.942857+0000","last_active":"2024-09-11T00:04:58.270161+0000","last_peered":"2024-09-11T00:04:58.270161+0000","last_clean":"2024-09-11T00:04:58.270161+0000","last_became_active":"2024-09-11T00:04:01.942494+0000","last_became_peered":"2024-09-11T00:04:01.942494+0000","last_unstale":"2024-09-11T00:04:58.270161+0000","last_undegraded":"2024-09-11T00:04:58.270161+0000","last_fullsized":"2024-09-11T00:04:58.270161+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_clean_scrub_stamp":"2024-09-11T00:04:00.794959+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:40:38.587178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":11,"num_read_kb":11,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":21,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T00:04:15.869817+0000","last_change":"2024-09-11T00:04:12.077771+0000","last_active":"2024-09-11T00:04:15.869817+0000","last_peered":"2024-09-11T00:04:15.869817+0000","last_clean":"2024-09-11T00:04:15.869817+0000","last_became_active":"2024-09-11T00:04:12.077366+0000","last_became_peered":"2024-09-11T00:04:12.077366+0000","last_unstale":"2024-09-11T00:04:15.869817+0000","last_undegraded":"2024-09-11T00:04:15.869817+0000","last_fullsized":"2024-09-11T00:04:15.869817+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_clean_scrub_stamp":"2024-09-11T00:04:10.845136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:04:26.808201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T00:04:15.869166+0000","last_change":"2024-09-11T00:04:03.827872+0000","last_active":"2024-09-11T00:04:15.869166+0000","last_peered":"2024-09-11T00:04:15.869166+0000","last_clean":"2024-09-11T00:04:15.869166+0000","last_became_active":"2024-09-11T00:04:03.827577+0000","last_became_peered":"2024-09-11T00:04:03.827577+0000","last_unstale":"2024-09-11T00:04:15.869166+0000","last_undegraded":"2024-09-11T00:04:15.869166+0000","last_fullsized":"2024-09-11T00:04:15.869166+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:37:19.205218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"81'4","reported_seq":37,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T00:04:47.236745+0000","last_change":"2024-09-11T00:04:01.819014+0000","last_active":"2024-09-11T00:04:47.236745+0000","last_peered":"2024-09-11T00:04:47.236745+0000","last_clean":"2024-09-11T00:04:47.236745+0000","last_became_active":"2024-09-11T00:04:01.818671+0000","last_became_peered":"2024-09-11T00:04:01.818671+0000","last_unstale":"2024-09-11T00:04:47.236745+0000","last_undegraded":"2024-09-11T00:04:47.236745+0000","last_fullsized":"2024-09-11T00:04:47.236745+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_clean_scrub_stamp":"2024-09-11T00:04:00.794959+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:42:55.226613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":21,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T00:04:17.436640+0000","last_change":"2024-09-11T00:04:12.077554+0000","last_active":"2024-09-11T00:04:17.436640+0000","last_peered":"2024-09-11T00:04:17.436640+0000","last_clean":"2024-09-11T00:04:17.436640+0000","last_became_active":"2024-09-11T00:04:12.076619+0000","last_became_peered":"2024-09-11T00:04:12.076619+0000","last_unstale":"2024-09-11T00:04:17.436640+0000","last_undegraded":"2024-09-11T00:04:17.436640+0000","last_fullsized":"2024-09-11T00:04:17.436640+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_clean_scrub_stamp":"2024-09-11T00:04:10.845136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:30:25.505200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T00:04:16.751606+0000","last_change":"2024-09-11T00:04:03.839615+0000","last_active":"2024-09-11T00:04:16.751606+0000","last_peered":"2024-09-11T00:04:16.751606+0000","last_clean":"2024-09-11T00:04:16.751606+0000","last_became_active":"2024-09-11T00:04:03.832175+0000","last_became_peered":"2024-09-11T00:04:03.832175+0000","last_unstale":"2024-09-11T00:04:16.751606+0000","last_undegraded":"2024-09-11T00:04:16.751606+0000","last_fullsized":"2024-09-11T00:04:16.751606+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:17:09.975345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"81'6","reported_seq":42,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T00:04:50.254721+0000","last_change":"2024-09-11T00:04:01.817248+0000","last_active":"2024-09-11T00:04:50.254721+0000","last_peered":"2024-09-11T00:04:50.254721+0000","last_clean":"2024-09-11T00:04:50.254721+0000","last_became_active":"2024-09-11T00:04:01.816896+0000","last_became_peered":"2024-09-11T00:04:01.816896+0000","last_unstale":"2024-09-11T00:04:50.254721+0000","last_undegraded":"2024-09-11T00:04:50.254721+0000","last_fullsized":"2024-09-11T00:04:50.254721+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_clean_scrub_stamp":"2024-09-11T00:04:00.794959+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:46:51.641881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'92","reported_seq":105,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T00:04:15.912257+0000","last_change":"2024-09-11T00:02:47.245193+0000","last_active":"2024-09-11T00:04:15.912257+0000","last_peered":"2024-09-11T00:04:15.912257+0000","last_clean":"2024-09-11T00:04:15.912257+0000","last_became_active":"2024-09-11T00:02:47.239773+0000","last_became_peered":"2024-09-11T00:02:47.239773+0000","last_unstale":"2024-09-11T00:04:15.912257+0000","last_undegraded":"2024-09-11T00:04:15.912257+0000","last_fullsized":"2024-09-11T00:04:15.912257+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:59:10.637655+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:59:10.637655+0000","last_clean_scrub_stamp":"2024-09-10T23:59:10.637655+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:51:52.914831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"81'1","reported_seq":22,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T00:04:15.912653+0000","last_change":"2024-09-11T00:04:11.867764+0000","last_active":"2024-09-11T00:04:15.912653+0000","last_peered":"2024-09-11T00:04:15.912653+0000","last_clean":"2024-09-11T00:04:15.912653+0000","last_became_active":"2024-09-11T00:04:11.867210+0000","last_became_peered":"2024-09-11T00:04:11.867210+0000","last_unstale":"2024-09-11T00:04:15.912653+0000","last_undegraded":"2024-09-11T00:04:15.912653+0000","last_fullsized":"2024-09-11T00:04:15.912653+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_clean_scrub_stamp":"2024-09-11T00:04:10.845136+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:04:15.439956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T00:04:15.912653+0000","last_change":"2024-09-11T00:04:03.834755+0000","last_active":"2024-09-11T00:04:15.912653+0000","last_peered":"2024-09-11T00:04:15.912653+0000","last_clean":"2024-09-11T00:04:15.912653+0000","last_became_active":"2024-09-11T00:04:03.833955+0000","last_became_peered":"2024-09-11T00:04:03.833955+0000","last_unstale":"2024-09-11T00:04:15.912653+0000","last_undegraded":"2024-09-11T00:04:15.912653+0000","last_fullsized":"2024-09-11T00:04:15.912653+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:47:13.863481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"81'6","reported_seq":52,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T00:04:58.270540+0000","last_change":"2024-09-11T00:04:02.589042+0000","last_active":"2024-09-11T00:04:58.270540+0000","last_peered":"2024-09-11T00:04:58.270540+0000","last_clean":"2024-09-11T00:04:58.270540+0000","last_became_active":"2024-09-11T00:04:02.588611+0000","last_became_peered":"2024-09-11T00:04:02.588611+0000","last_unstale":"2024-09-11T00:04:58.270540+0000","last_undegraded":"2024-09-11T00:04:58.270540+0000","last_fullsized":"2024-09-11T00:04:58.270540+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_clean_scrub_stamp":"2024-09-11T00:04:00.794959+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:41:08.257605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1561,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":13,"num_read_kb":3,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":21,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T00:04:17.435605+0000","last_change":"2024-09-11T00:04:11.860715+0000","last_active":"2024-09-11T00:04:17.435605+0000","last_peered":"2024-09-11T00:04:17.435605+0000","last_clean":"2024-09-11T00:04:17.435605+0000","last_became_active":"2024-09-11T00:04:11.860094+0000","last_became_peered":"2024-09-11T00:04:11.860094+0000","last_unstale":"2024-09-11T00:04:17.435605+0000","last_undegraded":"2024-09-11T00:04:17.435605+0000","last_fullsized":"2024-09-11T00:04:17.435605+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_clean_scrub_stamp":"2024-09-11T00:04:10.845136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:25:55.279079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T00:04:15.912543+0000","last_change":"2024-09-11T00:04:03.822665+0000","last_active":"2024-09-11T00:04:15.912543+0000","last_peered":"2024-09-11T00:04:15.912543+0000","last_clean":"2024-09-11T00:04:15.912543+0000","last_became_active":"2024-09-11T00:04:03.822333+0000","last_became_peered":"2024-09-11T00:04:03.822333+0000","last_unstale":"2024-09-11T00:04:15.912543+0000","last_undegraded":"2024-09-11T00:04:15.912543+0000","last_fullsized":"2024-09-11T00:04:15.912543+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:02:07.465335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"81'3","reported_seq":37,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T00:04:49.248660+0000","last_change":"2024-09-11T00:04:01.817244+0000","last_active":"2024-09-11T00:04:49.248660+0000","last_peered":"2024-09-11T00:04:49.248660+0000","last_clean":"2024-09-11T00:04:49.248660+0000","last_became_active":"2024-09-11T00:04:01.816846+0000","last_became_peered":"2024-09-11T00:04:01.816846+0000","last_unstale":"2024-09-11T00:04:49.248660+0000","last_undegraded":"2024-09-11T00:04:49.248660+0000","last_fullsized":"2024-09-11T00:04:49.248660+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_clean_scrub_stamp":"2024-09-11T00:04:00.794959+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:09:46.232332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":21,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T00:04:17.305061+0000","last_change":"2024-09-11T00:04:11.872439+0000","last_active":"2024-09-11T00:04:17.305061+0000","last_peered":"2024-09-11T00:04:17.305061+0000","last_clean":"2024-09-11T00:04:17.305061+0000","last_became_active":"2024-09-11T00:04:11.872064+0000","last_became_peered":"2024-09-11T00:04:11.872064+0000","last_unstale":"2024-09-11T00:04:17.305061+0000","last_undegraded":"2024-09-11T00:04:17.305061+0000","last_fullsized":"2024-09-11T00:04:17.305061+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_clean_scrub_stamp":"2024-09-11T00:04:10.845136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:15:47.902247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T00:04:15.912593+0000","last_change":"2024-09-11T00:04:03.829600+0000","last_active":"2024-09-11T00:04:15.912593+0000","last_peered":"2024-09-11T00:04:15.912593+0000","last_clean":"2024-09-11T00:04:15.912593+0000","last_became_active":"2024-09-11T00:04:03.829118+0000","last_became_peered":"2024-09-11T00:04:03.829118+0000","last_unstale":"2024-09-11T00:04:15.912593+0000","last_undegraded":"2024-09-11T00:04:15.912593+0000","last_fullsized":"2024-09-11T00:04:15.912593+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:41:07.069816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"81'5","reported_seq":38,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T00:04:49.250507+0000","last_change":"2024-09-11T00:04:01.820205+0000","last_active":"2024-09-11T00:04:49.250507+0000","last_peered":"2024-09-11T00:04:49.250507+0000","last_clean":"2024-09-11T00:04:49.250507+0000","last_became_active":"2024-09-11T00:04:01.819889+0000","last_became_peered":"2024-09-11T00:04:01.819889+0000","last_unstale":"2024-09-11T00:04:49.250507+0000","last_undegraded":"2024-09-11T00:04:49.250507+0000","last_fullsized":"2024-09-11T00:04:49.250507+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_clean_scrub_stamp":"2024-09-11T00:04:00.794959+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:05:05.259474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":21,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T00:04:17.435423+0000","last_change":"2024-09-11T00:04:11.863812+0000","last_active":"2024-09-11T00:04:17.435423+0000","last_peered":"2024-09-11T00:04:17.435423+0000","last_clean":"2024-09-11T00:04:17.435423+0000","last_became_active":"2024-09-11T00:04:11.863412+0000","last_became_peered":"2024-09-11T00:04:11.863412+0000","last_unstale":"2024-09-11T00:04:17.435423+0000","last_undegraded":"2024-09-11T00:04:17.435423+0000","last_fullsized":"2024-09-11T00:04:17.435423+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_clean_scrub_stamp":"2024-09-11T00:04:10.845136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:12:04.140034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T00:04:15.869604+0000","last_change":"2024-09-11T00:04:03.831235+0000","last_active":"2024-09-11T00:04:15.869604+0000","last_peered":"2024-09-11T00:04:15.869604+0000","last_clean":"2024-09-11T00:04:15.869604+0000","last_became_active":"2024-09-11T00:04:03.830054+0000","last_became_peered":"2024-09-11T00:04:03.830054+0000","last_unstale":"2024-09-11T00:04:15.869604+0000","last_undegraded":"2024-09-11T00:04:15.869604+0000","last_fullsized":"2024-09-11T00:04:15.869604+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:28:55.997099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"81'2","reported_seq":36,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T00:04:48.232268+0000","last_change":"2024-09-11T00:04:01.814191+0000","last_active":"2024-09-11T00:04:48.232268+0000","last_peered":"2024-09-11T00:04:48.232268+0000","last_clean":"2024-09-11T00:04:48.232268+0000","last_became_active":"2024-09-11T00:04:01.813778+0000","last_became_peered":"2024-09-11T00:04:01.813778+0000","last_unstale":"2024-09-11T00:04:48.232268+0000","last_undegraded":"2024-09-11T00:04:48.232268+0000","last_fullsized":"2024-09-11T00:04:48.232268+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_clean_scrub_stamp":"2024-09-11T00:04:00.794959+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:01:49.044436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":21,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T00:04:17.435473+0000","last_change":"2024-09-11T00:04:11.870281+0000","last_active":"2024-09-11T00:04:17.435473+0000","last_peered":"2024-09-11T00:04:17.435473+0000","last_clean":"2024-09-11T00:04:17.435473+0000","last_became_active":"2024-09-11T00:04:11.869738+0000","last_became_peered":"2024-09-11T00:04:11.869738+0000","last_unstale":"2024-09-11T00:04:17.435473+0000","last_undegraded":"2024-09-11T00:04:17.435473+0000","last_fullsized":"2024-09-11T00:04:17.435473+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_clean_scrub_stamp":"2024-09-11T00:04:10.845136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:04:08.380199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T00:04:17.435646+0000","last_change":"2024-09-11T00:04:03.835240+0000","last_active":"2024-09-11T00:04:17.435646+0000","last_peered":"2024-09-11T00:04:17.435646+0000","last_clean":"2024-09-11T00:04:17.435646+0000","last_became_active":"2024-09-11T00:04:03.834910+0000","last_became_peered":"2024-09-11T00:04:03.834910+0000","last_unstale":"2024-09-11T00:04:17.435646+0000","last_undegraded":"2024-09-11T00:04:17.435646+0000","last_fullsized":"2024-09-11T00:04:17.435646+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:30:33.996409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.10","version":"81'17","reported_seq":51,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T00:04:49.249166+0000","last_change":"2024-09-11T00:04:01.941860+0000","last_active":"2024-09-11T00:04:49.249166+0000","last_peered":"2024-09-11T00:04:49.249166+0000","last_clean":"2024-09-11T00:04:49.249166+0000","last_became_active":"2024-09-11T00:04:01.941484+0000","last_became_peered":"2024-09-11T00:04:01.941484+0000","last_unstale":"2024-09-11T00:04:49.249166+0000","last_undegraded":"2024-09-11T00:04:49.249166+0000","last_fullsized":"2024-09-11T00:04:49.249166+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_clean_scrub_stamp":"2024-09-11T00:04:00.794959+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:22:16.610237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":21,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T00:04:15.869896+0000","last_change":"2024-09-11T00:04:11.874306+0000","last_active":"2024-09-11T00:04:15.869896+0000","last_peered":"2024-09-11T00:04:15.869896+0000","last_clean":"2024-09-11T00:04:15.869896+0000","last_became_active":"2024-09-11T00:04:11.873934+0000","last_became_peered":"2024-09-11T00:04:11.873934+0000","last_unstale":"2024-09-11T00:04:15.869896+0000","last_undegraded":"2024-09-11T00:04:15.869896+0000","last_fullsized":"2024-09-11T00:04:15.869896+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_clean_scrub_stamp":"2024-09-11T00:04:10.845136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:19:22.584367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T00:04:15.912016+0000","last_change":"2024-09-11T00:04:03.824219+0000","last_active":"2024-09-11T00:04:15.912016+0000","last_peered":"2024-09-11T00:04:15.912016+0000","last_clean":"2024-09-11T00:04:15.912016+0000","last_became_active":"2024-09-11T00:04:03.823877+0000","last_became_peered":"2024-09-11T00:04:03.823877+0000","last_unstale":"2024-09-11T00:04:15.912016+0000","last_undegraded":"2024-09-11T00:04:15.912016+0000","last_fullsized":"2024-09-11T00:04:15.912016+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:04:05.924113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"81'5","reported_seq":50,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T00:04:58.270723+0000","last_change":"2024-09-11T00:04:01.818055+0000","last_active":"2024-09-11T00:04:58.270723+0000","last_peered":"2024-09-11T00:04:58.270723+0000","last_clean":"2024-09-11T00:04:58.270723+0000","last_became_active":"2024-09-11T00:04:01.817483+0000","last_became_peered":"2024-09-11T00:04:01.817483+0000","last_unstale":"2024-09-11T00:04:58.270723+0000","last_undegraded":"2024-09-11T00:04:58.270723+0000","last_fullsized":"2024-09-11T00:04:58.270723+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_clean_scrub_stamp":"2024-09-11T00:04:00.794959+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:16:55.218673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":13,"num_read_kb":3,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":21,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T00:04:17.435840+0000","last_change":"2024-09-11T00:04:11.875918+0000","last_active":"2024-09-11T00:04:17.435840+0000","last_peered":"2024-09-11T00:04:17.435840+0000","last_clean":"2024-09-11T00:04:17.435840+0000","last_became_active":"2024-09-11T00:04:11.875549+0000","last_became_peered":"2024-09-11T00:04:11.875549+0000","last_unstale":"2024-09-11T00:04:17.435840+0000","last_undegraded":"2024-09-11T00:04:17.435840+0000","last_fullsized":"2024-09-11T00:04:17.435840+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_clean_scrub_stamp":"2024-09-11T00:04:10.845136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:16:48.685899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T00:04:15.912429+0000","last_change":"2024-09-11T00:04:03.827671+0000","last_active":"2024-09-11T00:04:15.912429+0000","last_peered":"2024-09-11T00:04:15.912429+0000","last_clean":"2024-09-11T00:04:15.912429+0000","last_became_active":"2024-09-11T00:04:03.827341+0000","last_became_peered":"2024-09-11T00:04:03.827341+0000","last_unstale":"2024-09-11T00:04:15.912429+0000","last_undegraded":"2024-09-11T00:04:15.912429+0000","last_fullsized":"2024-09-11T00:04:15.912429+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:56:16.614035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"81'5","reported_seq":39,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T00:04:49.249302+0000","last_change":"2024-09-11T00:04:01.820632+0000","last_active":"2024-09-11T00:04:49.249302+0000","last_peered":"2024-09-11T00:04:49.249302+0000","last_clean":"2024-09-11T00:04:49.249302+0000","last_became_active":"2024-09-11T00:04:01.820273+0000","last_became_peered":"2024-09-11T00:04:01.820273+0000","last_unstale":"2024-09-11T00:04:49.249302+0000","last_undegraded":"2024-09-11T00:04:49.249302+0000","last_fullsized":"2024-09-11T00:04:49.249302+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_clean_scrub_stamp":"2024-09-11T00:04:00.794959+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:34:53.444515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":21,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T00:04:15.869485+0000","last_change":"2024-09-11T00:04:11.881197+0000","last_active":"2024-09-11T00:04:15.869485+0000","last_peered":"2024-09-11T00:04:15.869485+0000","last_clean":"2024-09-11T00:04:15.869485+0000","last_became_active":"2024-09-11T00:04:11.880763+0000","last_became_peered":"2024-09-11T00:04:11.880763+0000","last_unstale":"2024-09-11T00:04:15.869485+0000","last_undegraded":"2024-09-11T00:04:15.869485+0000","last_fullsized":"2024-09-11T00:04:15.869485+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_clean_scrub_stamp":"2024-09-11T00:04:10.845136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:14:01.616095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T00:04:15.869315+0000","last_change":"2024-09-11T00:04:03.833470+0000","last_active":"2024-09-11T00:04:15.869315+0000","last_peered":"2024-09-11T00:04:15.869315+0000","last_clean":"2024-09-11T00:04:15.869315+0000","last_became_active":"2024-09-11T00:04:03.833120+0000","last_became_peered":"2024-09-11T00:04:03.833120+0000","last_unstale":"2024-09-11T00:04:15.869315+0000","last_undegraded":"2024-09-11T00:04:15.869315+0000","last_fullsized":"2024-09-11T00:04:15.869315+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:29:53.210230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"81'9","reported_seq":44,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T00:04:50.254977+0000","last_change":"2024-09-11T00:04:01.818081+0000","last_active":"2024-09-11T00:04:50.254977+0000","last_peered":"2024-09-11T00:04:50.254977+0000","last_clean":"2024-09-11T00:04:50.254977+0000","last_became_active":"2024-09-11T00:04:01.817716+0000","last_became_peered":"2024-09-11T00:04:01.817716+0000","last_unstale":"2024-09-11T00:04:50.254977+0000","last_undegraded":"2024-09-11T00:04:50.254977+0000","last_fullsized":"2024-09-11T00:04:50.254977+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_clean_scrub_stamp":"2024-09-11T00:04:00.794959+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:01:32.736319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":21,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T00:04:17.436755+0000","last_change":"2024-09-11T00:04:11.870182+0000","last_active":"2024-09-11T00:04:17.436755+0000","last_peered":"2024-09-11T00:04:17.436755+0000","last_clean":"2024-09-11T00:04:17.436755+0000","last_became_active":"2024-09-11T00:04:11.869744+0000","last_became_peered":"2024-09-11T00:04:11.869744+0000","last_unstale":"2024-09-11T00:04:17.436755+0000","last_undegraded":"2024-09-11T00:04:17.436755+0000","last_fullsized":"2024-09-11T00:04:17.436755+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_clean_scrub_stamp":"2024-09-11T00:04:10.845136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:50:24.070404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T00:04:17.436191+0000","last_change":"2024-09-11T00:04:03.828246+0000","last_active":"2024-09-11T00:04:17.436191+0000","last_peered":"2024-09-11T00:04:17.436191+0000","last_clean":"2024-09-11T00:04:17.436191+0000","last_became_active":"2024-09-11T00:04:03.827217+0000","last_became_peered":"2024-09-11T00:04:03.827217+0000","last_unstale":"2024-09-11T00:04:17.436191+0000","last_undegraded":"2024-09-11T00:04:17.436191+0000","last_fullsized":"2024-09-11T00:04:17.436191+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:51:46.018759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"81'27","reported_seq":88,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T00:05:00.274639+0000","last_change":"2024-09-11T00:04:01.819553+0000","last_active":"2024-09-11T00:05:00.274639+0000","last_peered":"2024-09-11T00:05:00.274639+0000","last_clean":"2024-09-11T00:05:00.274639+0000","last_became_active":"2024-09-11T00:04:01.819213+0000","last_became_peered":"2024-09-11T00:04:01.819213+0000","last_unstale":"2024-09-11T00:05:00.274639+0000","last_undegraded":"2024-09-11T00:05:00.274639+0000","last_fullsized":"2024-09-11T00:05:00.274639+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_clean_scrub_stamp":"2024-09-11T00:04:00.794959+0000","objects_scrubbed":0,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:44:11.039982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":43341,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":28,"num_read_kb":56,"num_write":29,"num_write_kb":52,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":21,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T00:04:17.435474+0000","last_change":"2024-09-11T00:04:11.871730+0000","last_active":"2024-09-11T00:04:17.435474+0000","last_peered":"2024-09-11T00:04:17.435474+0000","last_clean":"2024-09-11T00:04:17.435474+0000","last_became_active":"2024-09-11T00:04:11.870659+0000","last_became_peered":"2024-09-11T00:04:11.870659+0000","last_unstale":"2024-09-11T00:04:17.435474+0000","last_undegraded":"2024-09-11T00:04:17.435474+0000","last_fullsized":"2024-09-11T00:04:17.435474+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_clean_scrub_stamp":"2024-09-11T00:04:10.845136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:48:41.835602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T00:04:17.435768+0000","last_change":"2024-09-11T00:04:03.838763+0000","last_active":"2024-09-11T00:04:17.435768+0000","last_peered":"2024-09-11T00:04:17.435768+0000","last_clean":"2024-09-11T00:04:17.435768+0000","last_became_active":"2024-09-11T00:04:03.838409+0000","last_became_peered":"2024-09-11T00:04:03.838409+0000","last_unstale":"2024-09-11T00:04:17.435768+0000","last_undegraded":"2024-09-11T00:04:17.435768+0000","last_fullsized":"2024-09-11T00:04:17.435768+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:57:38.647351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"81'5","reported_seq":39,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T00:04:48.243571+0000","last_change":"2024-09-11T00:04:01.943027+0000","last_active":"2024-09-11T00:04:48.243571+0000","last_peered":"2024-09-11T00:04:48.243571+0000","last_clean":"2024-09-11T00:04:48.243571+0000","last_became_active":"2024-09-11T00:04:01.942558+0000","last_became_peered":"2024-09-11T00:04:01.942558+0000","last_unstale":"2024-09-11T00:04:48.243571+0000","last_undegraded":"2024-09-11T00:04:48.243571+0000","last_fullsized":"2024-09-11T00:04:48.243571+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_clean_scrub_stamp":"2024-09-11T00:04:00.794959+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:48:03.776476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":21,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T00:04:17.436777+0000","last_change":"2024-09-11T00:04:11.868069+0000","last_active":"2024-09-11T00:04:17.436777+0000","last_peered":"2024-09-11T00:04:17.436777+0000","last_clean":"2024-09-11T00:04:17.436777+0000","last_became_active":"2024-09-11T00:04:11.867544+0000","last_became_peered":"2024-09-11T00:04:11.867544+0000","last_unstale":"2024-09-11T00:04:17.436777+0000","last_undegraded":"2024-09-11T00:04:17.436777+0000","last_fullsized":"2024-09-11T00:04:17.436777+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_clean_scrub_stamp":"2024-09-11T00:04:10.845136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:09:20.455807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T00:04:17.305180+0000","last_change":"2024-09-11T00:04:03.836735+0000","last_active":"2024-09-11T00:04:17.305180+0000","last_peered":"2024-09-11T00:04:17.305180+0000","last_clean":"2024-09-11T00:04:17.305180+0000","last_became_active":"2024-09-11T00:04:03.833299+0000","last_became_peered":"2024-09-11T00:04:03.833299+0000","last_unstale":"2024-09-11T00:04:17.305180+0000","last_undegraded":"2024-09-11T00:04:17.305180+0000","last_fullsized":"2024-09-11T00:04:17.305180+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:32:15.924058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"81'4","reported_seq":37,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T00:04:49.249301+0000","last_change":"2024-09-11T00:04:01.823777+0000","last_active":"2024-09-11T00:04:49.249301+0000","last_peered":"2024-09-11T00:04:49.249301+0000","last_clean":"2024-09-11T00:04:49.249301+0000","last_became_active":"2024-09-11T00:04:01.823368+0000","last_became_peered":"2024-09-11T00:04:01.823368+0000","last_unstale":"2024-09-11T00:04:49.249301+0000","last_undegraded":"2024-09-11T00:04:49.249301+0000","last_fullsized":"2024-09-11T00:04:49.249301+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_clean_scrub_stamp":"2024-09-11T00:04:00.794959+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:05:12.584867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":21,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T00:04:17.435215+0000","last_change":"2024-09-11T00:04:11.868303+0000","last_active":"2024-09-11T00:04:17.435215+0000","last_peered":"2024-09-11T00:04:17.435215+0000","last_clean":"2024-09-11T00:04:17.435215+0000","last_became_active":"2024-09-11T00:04:11.867916+0000","last_became_peered":"2024-09-11T00:04:11.867916+0000","last_unstale":"2024-09-11T00:04:17.435215+0000","last_undegraded":"2024-09-11T00:04:17.435215+0000","last_fullsized":"2024-09-11T00:04:17.435215+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_clean_scrub_stamp":"2024-09-11T00:04:10.845136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:08:29.711763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T00:04:17.436694+0000","last_change":"2024-09-11T00:04:03.832712+0000","last_active":"2024-09-11T00:04:17.436694+0000","last_peered":"2024-09-11T00:04:17.436694+0000","last_clean":"2024-09-11T00:04:17.436694+0000","last_became_active":"2024-09-11T00:04:03.832295+0000","last_became_peered":"2024-09-11T00:04:03.832295+0000","last_unstale":"2024-09-11T00:04:17.436694+0000","last_undegraded":"2024-09-11T00:04:17.436694+0000","last_fullsized":"2024-09-11T00:04:17.436694+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:40:12.271930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"81'4","reported_seq":59,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T00:05:00.272802+0000","last_change":"2024-09-11T00:04:01.820891+0000","last_active":"2024-09-11T00:05:00.272802+0000","last_peered":"2024-09-11T00:05:00.272802+0000","last_clean":"2024-09-11T00:05:00.272802+0000","last_became_active":"2024-09-11T00:04:01.820528+0000","last_became_peered":"2024-09-11T00:04:01.820528+0000","last_unstale":"2024-09-11T00:05:00.272802+0000","last_undegraded":"2024-09-11T00:05:00.272802+0000","last_fullsized":"2024-09-11T00:05:00.272802+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_clean_scrub_stamp":"2024-09-11T00:04:00.794959+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:23:38.196618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":22,"num_read_kb":22,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":21,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T00:04:16.752010+0000","last_change":"2024-09-11T00:04:11.878864+0000","last_active":"2024-09-11T00:04:16.752010+0000","last_peered":"2024-09-11T00:04:16.752010+0000","last_clean":"2024-09-11T00:04:16.752010+0000","last_became_active":"2024-09-11T00:04:11.876701+0000","last_became_peered":"2024-09-11T00:04:11.876701+0000","last_unstale":"2024-09-11T00:04:16.752010+0000","last_undegraded":"2024-09-11T00:04:16.752010+0000","last_fullsized":"2024-09-11T00:04:16.752010+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_clean_scrub_stamp":"2024-09-11T00:04:10.845136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:26:05.650618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":21,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T00:04:17.304780+0000","last_change":"2024-09-11T00:04:11.868140+0000","last_active":"2024-09-11T00:04:17.304780+0000","last_peered":"2024-09-11T00:04:17.304780+0000","last_clean":"2024-09-11T00:04:17.304780+0000","last_became_active":"2024-09-11T00:04:11.866459+0000","last_became_peered":"2024-09-11T00:04:11.866459+0000","last_unstale":"2024-09-11T00:04:17.304780+0000","last_undegraded":"2024-09-11T00:04:17.304780+0000","last_fullsized":"2024-09-11T00:04:17.304780+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_clean_scrub_stamp":"2024-09-11T00:04:10.845136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:53:25.106039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T00:04:17.435283+0000","last_change":"2024-09-11T00:04:03.829839+0000","last_active":"2024-09-11T00:04:17.435283+0000","last_peered":"2024-09-11T00:04:17.435283+0000","last_clean":"2024-09-11T00:04:17.435283+0000","last_became_active":"2024-09-11T00:04:03.829553+0000","last_became_peered":"2024-09-11T00:04:03.829553+0000","last_unstale":"2024-09-11T00:04:17.435283+0000","last_undegraded":"2024-09-11T00:04:17.435283+0000","last_fullsized":"2024-09-11T00:04:17.435283+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:45:58.561389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"81'3","reported_seq":37,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T00:04:49.249068+0000","last_change":"2024-09-11T00:04:01.820204+0000","last_active":"2024-09-11T00:04:49.249068+0000","last_peered":"2024-09-11T00:04:49.249068+0000","last_clean":"2024-09-11T00:04:49.249068+0000","last_became_active":"2024-09-11T00:04:01.819797+0000","last_became_peered":"2024-09-11T00:04:01.819797+0000","last_unstale":"2024-09-11T00:04:49.249068+0000","last_undegraded":"2024-09-11T00:04:49.249068+0000","last_fullsized":"2024-09-11T00:04:49.249068+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_clean_scrub_stamp":"2024-09-11T00:04:00.794959+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:56:10.530023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":21,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T00:04:15.869760+0000","last_change":"2024-09-11T00:04:11.879430+0000","last_active":"2024-09-11T00:04:15.869760+0000","last_peered":"2024-09-11T00:04:15.869760+0000","last_clean":"2024-09-11T00:04:15.869760+0000","last_became_active":"2024-09-11T00:04:11.879010+0000","last_became_peered":"2024-09-11T00:04:11.879010+0000","last_unstale":"2024-09-11T00:04:15.869760+0000","last_undegraded":"2024-09-11T00:04:15.869760+0000","last_fullsized":"2024-09-11T00:04:15.869760+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_clean_scrub_stamp":"2024-09-11T00:04:10.845136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:57.737125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T00:04:16.751460+0000","last_change":"2024-09-11T00:04:03.843918+0000","last_active":"2024-09-11T00:04:16.751460+0000","last_peered":"2024-09-11T00:04:16.751460+0000","last_clean":"2024-09-11T00:04:16.751460+0000","last_became_active":"2024-09-11T00:04:03.843591+0000","last_became_peered":"2024-09-11T00:04:03.843591+0000","last_unstale":"2024-09-11T00:04:16.751460+0000","last_undegraded":"2024-09-11T00:04:16.751460+0000","last_fullsized":"2024-09-11T00:04:16.751460+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:26:41.228802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"81'3","reported_seq":36,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T00:04:47.239179+0000","last_change":"2024-09-11T00:04:01.813173+0000","last_active":"2024-09-11T00:04:47.239179+0000","last_peered":"2024-09-11T00:04:47.239179+0000","last_clean":"2024-09-11T00:04:47.239179+0000","last_became_active":"2024-09-11T00:04:01.811799+0000","last_became_peered":"2024-09-11T00:04:01.811799+0000","last_unstale":"2024-09-11T00:04:47.239179+0000","last_undegraded":"2024-09-11T00:04:47.239179+0000","last_fullsized":"2024-09-11T00:04:47.239179+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_clean_scrub_stamp":"2024-09-11T00:04:00.794959+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:16:47.304371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"81'7","reported_seq":40,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T00:04:48.242866+0000","last_change":"2024-09-11T00:04:01.818649+0000","last_active":"2024-09-11T00:04:48.242866+0000","last_peered":"2024-09-11T00:04:48.242866+0000","last_clean":"2024-09-11T00:04:48.242866+0000","last_became_active":"2024-09-11T00:04:01.818287+0000","last_became_peered":"2024-09-11T00:04:01.818287+0000","last_unstale":"2024-09-11T00:04:48.242866+0000","last_undegraded":"2024-09-11T00:04:48.242866+0000","last_fullsized":"2024-09-11T00:04:48.242866+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_clean_scrub_stamp":"2024-09-11T00:04:00.794959+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:42:00.764648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T00:04:17.435994+0000","last_change":"2024-09-11T00:04:03.832829+0000","last_active":"2024-09-11T00:04:17.435994+0000","last_peered":"2024-09-11T00:04:17.435994+0000","last_clean":"2024-09-11T00:04:17.435994+0000","last_became_active":"2024-09-11T00:04:03.832514+0000","last_became_peered":"2024-09-11T00:04:03.832514+0000","last_unstale":"2024-09-11T00:04:17.435994+0000","last_undegraded":"2024-09-11T00:04:17.435994+0000","last_fullsized":"2024-09-11T00:04:17.435994+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:36.643727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":21,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T00:04:16.882276+0000","last_change":"2024-09-11T00:04:11.877596+0000","last_active":"2024-09-11T00:04:16.882276+0000","last_peered":"2024-09-11T00:04:16.882276+0000","last_clean":"2024-09-11T00:04:16.882276+0000","last_became_active":"2024-09-11T00:04:11.877113+0000","last_became_peered":"2024-09-11T00:04:11.877113+0000","last_unstale":"2024-09-11T00:04:16.882276+0000","last_undegraded":"2024-09-11T00:04:16.882276+0000","last_fullsized":"2024-09-11T00:04:16.882276+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_clean_scrub_stamp":"2024-09-11T00:04:10.845136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:47:11.759687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"81'5","reported_seq":51,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T00:04:59.271962+0000","last_change":"2024-09-11T00:04:01.821519+0000","last_active":"2024-09-11T00:04:59.271962+0000","last_peered":"2024-09-11T00:04:59.271962+0000","last_clean":"2024-09-11T00:04:59.271962+0000","last_became_active":"2024-09-11T00:04:01.821207+0000","last_became_peered":"2024-09-11T00:04:01.821207+0000","last_unstale":"2024-09-11T00:04:59.271962+0000","last_undegraded":"2024-09-11T00:04:59.271962+0000","last_fullsized":"2024-09-11T00:04:59.271962+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_clean_scrub_stamp":"2024-09-11T00:04:00.794959+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:16:34.435040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":13,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T00:04:17.435813+0000","last_change":"2024-09-11T00:04:03.832691+0000","last_active":"2024-09-11T00:04:17.435813+0000","last_peered":"2024-09-11T00:04:17.435813+0000","last_clean":"2024-09-11T00:04:17.435813+0000","last_became_active":"2024-09-11T00:04:03.832250+0000","last_became_peered":"2024-09-11T00:04:03.832250+0000","last_unstale":"2024-09-11T00:04:17.435813+0000","last_undegraded":"2024-09-11T00:04:17.435813+0000","last_fullsized":"2024-09-11T00:04:17.435813+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:54.279820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":21,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T00:04:16.882547+0000","last_change":"2024-09-11T00:04:11.864995+0000","last_active":"2024-09-11T00:04:16.882547+0000","last_peered":"2024-09-11T00:04:16.882547+0000","last_clean":"2024-09-11T00:04:16.882547+0000","last_became_active":"2024-09-11T00:04:11.864501+0000","last_became_peered":"2024-09-11T00:04:11.864501+0000","last_unstale":"2024-09-11T00:04:16.882547+0000","last_undegraded":"2024-09-11T00:04:16.882547+0000","last_fullsized":"2024-09-11T00:04:16.882547+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_clean_scrub_stamp":"2024-09-11T00:04:10.845136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:08:08.637714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T00:04:15.869280+0000","last_change":"2024-09-11T00:04:03.829310+0000","last_active":"2024-09-11T00:04:15.869280+0000","last_peered":"2024-09-11T00:04:15.869280+0000","last_clean":"2024-09-11T00:04:15.869280+0000","last_became_active":"2024-09-11T00:04:03.828937+0000","last_became_peered":"2024-09-11T00:04:03.828937+0000","last_unstale":"2024-09-11T00:04:15.869280+0000","last_undegraded":"2024-09-11T00:04:15.869280+0000","last_fullsized":"2024-09-11T00:04:15.869280+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:32:58.263067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T00:04:16.882379+0000","last_change":"2024-09-11T00:04:03.828055+0000","last_active":"2024-09-11T00:04:16.882379+0000","last_peered":"2024-09-11T00:04:16.882379+0000","last_clean":"2024-09-11T00:04:16.882379+0000","last_became_active":"2024-09-11T00:04:03.827382+0000","last_became_peered":"2024-09-11T00:04:03.827382+0000","last_unstale":"2024-09-11T00:04:16.882379+0000","last_undegraded":"2024-09-11T00:04:16.882379+0000","last_fullsized":"2024-09-11T00:04:16.882379+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:02:47.107689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T00:04:16.882015+0000","last_change":"2024-09-11T00:04:03.843549+0000","last_active":"2024-09-11T00:04:16.882015+0000","last_peered":"2024-09-11T00:04:16.882015+0000","last_clean":"2024-09-11T00:04:16.882015+0000","last_became_active":"2024-09-11T00:04:03.843149+0000","last_became_peered":"2024-09-11T00:04:03.843149+0000","last_unstale":"2024-09-11T00:04:16.882015+0000","last_undegraded":"2024-09-11T00:04:16.882015+0000","last_fullsized":"2024-09-11T00:04:16.882015+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:47.288319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T00:04:17.304581+0000","last_change":"2024-09-11T00:04:03.838631+0000","last_active":"2024-09-11T00:04:17.304581+0000","last_peered":"2024-09-11T00:04:17.304581+0000","last_clean":"2024-09-11T00:04:17.304581+0000","last_became_active":"2024-09-11T00:04:03.836382+0000","last_became_peered":"2024-09-11T00:04:03.836382+0000","last_unstale":"2024-09-11T00:04:17.304581+0000","last_undegraded":"2024-09-11T00:04:17.304581+0000","last_fullsized":"2024-09-11T00:04:17.304581+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:36:20.341469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T00:04:15.869636+0000","last_change":"2024-09-11T00:04:03.835068+0000","last_active":"2024-09-11T00:04:15.869636+0000","last_peered":"2024-09-11T00:04:15.869636+0000","last_clean":"2024-09-11T00:04:15.869636+0000","last_became_active":"2024-09-11T00:04:03.833166+0000","last_became_peered":"2024-09-11T00:04:03.833166+0000","last_unstale":"2024-09-11T00:04:15.869636+0000","last_undegraded":"2024-09-11T00:04:15.869636+0000","last_fullsized":"2024-09-11T00:04:15.869636+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:07:06.357597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T00:04:15.869678+0000","last_change":"2024-09-11T00:04:03.827310+0000","last_active":"2024-09-11T00:04:15.869678+0000","last_peered":"2024-09-11T00:04:15.869678+0000","last_clean":"2024-09-11T00:04:15.869678+0000","last_became_active":"2024-09-11T00:04:03.826748+0000","last_became_peered":"2024-09-11T00:04:03.826748+0000","last_unstale":"2024-09-11T00:04:15.869678+0000","last_undegraded":"2024-09-11T00:04:15.869678+0000","last_fullsized":"2024-09-11T00:04:15.869678+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:53:15.656708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T00:04:16.882618+0000","last_change":"2024-09-11T00:04:03.835900+0000","last_active":"2024-09-11T00:04:16.882618+0000","last_peered":"2024-09-11T00:04:16.882618+0000","last_clean":"2024-09-11T00:04:16.882618+0000","last_became_active":"2024-09-11T00:04:03.835137+0000","last_became_peered":"2024-09-11T00:04:03.835137+0000","last_unstale":"2024-09-11T00:04:16.882618+0000","last_undegraded":"2024-09-11T00:04:16.882618+0000","last_fullsized":"2024-09-11T00:04:16.882618+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:23:31.184833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T00:04:16.751865+0000","last_change":"2024-09-11T00:04:03.833028+0000","last_active":"2024-09-11T00:04:16.751865+0000","last_peered":"2024-09-11T00:04:16.751865+0000","last_clean":"2024-09-11T00:04:16.751865+0000","last_became_active":"2024-09-11T00:04:03.832703+0000","last_became_peered":"2024-09-11T00:04:03.832703+0000","last_unstale":"2024-09-11T00:04:16.751865+0000","last_undegraded":"2024-09-11T00:04:16.751865+0000","last_fullsized":"2024-09-11T00:04:16.751865+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:38.115627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T00:04:17.435508+0000","last_change":"2024-09-11T00:04:03.832423+0000","last_active":"2024-09-11T00:04:17.435508+0000","last_peered":"2024-09-11T00:04:17.435508+0000","last_clean":"2024-09-11T00:04:17.435508+0000","last_became_active":"2024-09-11T00:04:03.832090+0000","last_became_peered":"2024-09-11T00:04:03.832090+0000","last_unstale":"2024-09-11T00:04:17.435508+0000","last_undegraded":"2024-09-11T00:04:17.435508+0000","last_fullsized":"2024-09-11T00:04:17.435508+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:01:27.258506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T00:04:16.751836+0000","last_change":"2024-09-11T00:04:03.842854+0000","last_active":"2024-09-11T00:04:16.751836+0000","last_peered":"2024-09-11T00:04:16.751836+0000","last_clean":"2024-09-11T00:04:16.751836+0000","last_became_active":"2024-09-11T00:04:03.838666+0000","last_became_peered":"2024-09-11T00:04:03.838666+0000","last_unstale":"2024-09-11T00:04:16.751836+0000","last_undegraded":"2024-09-11T00:04:16.751836+0000","last_fullsized":"2024-09-11T00:04:16.751836+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:45:48.575830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T00:04:17.436008+0000","last_change":"2024-09-11T00:04:03.834175+0000","last_active":"2024-09-11T00:04:17.436008+0000","last_peered":"2024-09-11T00:04:17.436008+0000","last_clean":"2024-09-11T00:04:17.436008+0000","last_became_active":"2024-09-11T00:04:03.833826+0000","last_became_peered":"2024-09-11T00:04:03.833826+0000","last_unstale":"2024-09-11T00:04:17.436008+0000","last_undegraded":"2024-09-11T00:04:17.436008+0000","last_fullsized":"2024-09-11T00:04:17.436008+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:45:57.323216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T00:04:15.869567+0000","last_change":"2024-09-11T00:04:03.836751+0000","last_active":"2024-09-11T00:04:15.869567+0000","last_peered":"2024-09-11T00:04:15.869567+0000","last_clean":"2024-09-11T00:04:15.869567+0000","last_became_active":"2024-09-11T00:04:03.834064+0000","last_became_peered":"2024-09-11T00:04:03.834064+0000","last_unstale":"2024-09-11T00:04:15.869567+0000","last_undegraded":"2024-09-11T00:04:15.869567+0000","last_fullsized":"2024-09-11T00:04:15.869567+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:06:23.059770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T00:04:15.869456+0000","last_change":"2024-09-11T00:04:03.830196+0000","last_active":"2024-09-11T00:04:15.869456+0000","last_peered":"2024-09-11T00:04:15.869456+0000","last_clean":"2024-09-11T00:04:15.869456+0000","last_became_active":"2024-09-11T00:04:03.829856+0000","last_became_peered":"2024-09-11T00:04:03.829856+0000","last_unstale":"2024-09-11T00:04:15.869456+0000","last_undegraded":"2024-09-11T00:04:15.869456+0000","last_fullsized":"2024-09-11T00:04:15.869456+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:34:47.489316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T00:04:16.882542+0000","last_change":"2024-09-11T00:04:03.835053+0000","last_active":"2024-09-11T00:04:16.882542+0000","last_peered":"2024-09-11T00:04:16.882542+0000","last_clean":"2024-09-11T00:04:16.882542+0000","last_became_active":"2024-09-11T00:04:03.834736+0000","last_became_peered":"2024-09-11T00:04:03.834736+0000","last_unstale":"2024-09-11T00:04:16.882542+0000","last_undegraded":"2024-09-11T00:04:16.882542+0000","last_fullsized":"2024-09-11T00:04:16.882542+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:53:40.253775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T00:04:17.435772+0000","last_change":"2024-09-11T00:04:03.834514+0000","last_active":"2024-09-11T00:04:17.435772+0000","last_peered":"2024-09-11T00:04:17.435772+0000","last_clean":"2024-09-11T00:04:17.435772+0000","last_became_active":"2024-09-11T00:04:03.834185+0000","last_became_peered":"2024-09-11T00:04:03.834185+0000","last_unstale":"2024-09-11T00:04:17.435772+0000","last_undegraded":"2024-09-11T00:04:17.435772+0000","last_fullsized":"2024-09-11T00:04:17.435772+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:33:52.949710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T00:04:15.869944+0000","last_change":"2024-09-11T00:04:03.840371+0000","last_active":"2024-09-11T00:04:15.869944+0000","last_peered":"2024-09-11T00:04:15.869944+0000","last_clean":"2024-09-11T00:04:15.869944+0000","last_became_active":"2024-09-11T00:04:03.839933+0000","last_became_peered":"2024-09-11T00:04:03.839933+0000","last_unstale":"2024-09-11T00:04:15.869944+0000","last_undegraded":"2024-09-11T00:04:15.869944+0000","last_fullsized":"2024-09-11T00:04:15.869944+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:24:53.890837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T00:04:17.435348+0000","last_change":"2024-09-11T00:04:03.838386+0000","last_active":"2024-09-11T00:04:17.435348+0000","last_peered":"2024-09-11T00:04:17.435348+0000","last_clean":"2024-09-11T00:04:17.435348+0000","last_became_active":"2024-09-11T00:04:03.837511+0000","last_became_peered":"2024-09-11T00:04:03.837511+0000","last_unstale":"2024-09-11T00:04:17.435348+0000","last_undegraded":"2024-09-11T00:04:17.435348+0000","last_fullsized":"2024-09-11T00:04:17.435348+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:27.399727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T00:04:17.304478+0000","last_change":"2024-09-11T00:04:03.831442+0000","last_active":"2024-09-11T00:04:17.304478+0000","last_peered":"2024-09-11T00:04:17.304478+0000","last_clean":"2024-09-11T00:04:17.304478+0000","last_became_active":"2024-09-11T00:04:03.831115+0000","last_became_peered":"2024-09-11T00:04:03.831115+0000","last_unstale":"2024-09-11T00:04:17.304478+0000","last_undegraded":"2024-09-11T00:04:17.304478+0000","last_fullsized":"2024-09-11T00:04:17.304478+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:53:13.475484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T00:04:17.304492+0000","last_change":"2024-09-11T00:04:03.832618+0000","last_active":"2024-09-11T00:04:17.304492+0000","last_peered":"2024-09-11T00:04:17.304492+0000","last_clean":"2024-09-11T00:04:17.304492+0000","last_became_active":"2024-09-11T00:04:03.831821+0000","last_became_peered":"2024-09-11T00:04:03.831821+0000","last_unstale":"2024-09-11T00:04:17.304492+0000","last_undegraded":"2024-09-11T00:04:17.304492+0000","last_fullsized":"2024-09-11T00:04:17.304492+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:40:40.402422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T00:04:15.869546+0000","last_change":"2024-09-11T00:04:03.825348+0000","last_active":"2024-09-11T00:04:15.869546+0000","last_peered":"2024-09-11T00:04:15.869546+0000","last_clean":"2024-09-11T00:04:15.869546+0000","last_became_active":"2024-09-11T00:04:03.824977+0000","last_became_peered":"2024-09-11T00:04:03.824977+0000","last_unstale":"2024-09-11T00:04:15.869546+0000","last_undegraded":"2024-09-11T00:04:15.869546+0000","last_fullsized":"2024-09-11T00:04:15.869546+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:19:53.412969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T00:04:17.436524+0000","last_change":"2024-09-11T00:04:03.837471+0000","last_active":"2024-09-11T00:04:17.436524+0000","last_peered":"2024-09-11T00:04:17.436524+0000","last_clean":"2024-09-11T00:04:17.436524+0000","last_became_active":"2024-09-11T00:04:03.836651+0000","last_became_peered":"2024-09-11T00:04:03.836651+0000","last_unstale":"2024-09-11T00:04:17.436524+0000","last_undegraded":"2024-09-11T00:04:17.436524+0000","last_fullsized":"2024-09-11T00:04:17.436524+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:34:19.902779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T00:04:15.912587+0000","last_change":"2024-09-11T00:04:03.840068+0000","last_active":"2024-09-11T00:04:15.912587+0000","last_peered":"2024-09-11T00:04:15.912587+0000","last_clean":"2024-09-11T00:04:15.912587+0000","last_became_active":"2024-09-11T00:04:03.835185+0000","last_became_peered":"2024-09-11T00:04:03.835185+0000","last_unstale":"2024-09-11T00:04:15.912587+0000","last_undegraded":"2024-09-11T00:04:15.912587+0000","last_fullsized":"2024-09-11T00:04:15.912587+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:19:10.699038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T00:04:15.870175+0000","last_change":"2024-09-11T00:04:03.828576+0000","last_active":"2024-09-11T00:04:15.870175+0000","last_peered":"2024-09-11T00:04:15.870175+0000","last_clean":"2024-09-11T00:04:15.870175+0000","last_became_active":"2024-09-11T00:04:03.828255+0000","last_became_peered":"2024-09-11T00:04:03.828255+0000","last_unstale":"2024-09-11T00:04:15.870175+0000","last_undegraded":"2024-09-11T00:04:15.870175+0000","last_fullsized":"2024-09-11T00:04:15.870175+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:29:54.164145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T00:04:17.436235+0000","last_change":"2024-09-11T00:04:03.841452+0000","last_active":"2024-09-11T00:04:17.436235+0000","last_peered":"2024-09-11T00:04:17.436235+0000","last_clean":"2024-09-11T00:04:17.436235+0000","last_became_active":"2024-09-11T00:04:03.840911+0000","last_became_peered":"2024-09-11T00:04:03.840911+0000","last_unstale":"2024-09-11T00:04:17.436235+0000","last_undegraded":"2024-09-11T00:04:17.436235+0000","last_fullsized":"2024-09-11T00:04:17.436235+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:56:40.121922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T00:04:16.751450+0000","last_change":"2024-09-11T00:04:03.842009+0000","last_active":"2024-09-11T00:04:16.751450+0000","last_peered":"2024-09-11T00:04:16.751450+0000","last_clean":"2024-09-11T00:04:16.751450+0000","last_became_active":"2024-09-11T00:04:03.841625+0000","last_became_peered":"2024-09-11T00:04:03.841625+0000","last_unstale":"2024-09-11T00:04:16.751450+0000","last_undegraded":"2024-09-11T00:04:16.751450+0000","last_fullsized":"2024-09-11T00:04:16.751450+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:15:11.855308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T00:04:15.912122+0000","last_change":"2024-09-11T00:04:03.829998+0000","last_active":"2024-09-11T00:04:15.912122+0000","last_peered":"2024-09-11T00:04:15.912122+0000","last_clean":"2024-09-11T00:04:15.912122+0000","last_became_active":"2024-09-11T00:04:03.829637+0000","last_became_peered":"2024-09-11T00:04:03.829637+0000","last_unstale":"2024-09-11T00:04:15.912122+0000","last_undegraded":"2024-09-11T00:04:15.912122+0000","last_fullsized":"2024-09-11T00:04:15.912122+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:05:10.144719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T00:04:15.869720+0000","last_change":"2024-09-11T00:04:03.834540+0000","last_active":"2024-09-11T00:04:15.869720+0000","last_peered":"2024-09-11T00:04:15.869720+0000","last_clean":"2024-09-11T00:04:15.869720+0000","last_became_active":"2024-09-11T00:04:03.834230+0000","last_became_peered":"2024-09-11T00:04:03.834230+0000","last_unstale":"2024-09-11T00:04:15.869720+0000","last_undegraded":"2024-09-11T00:04:15.869720+0000","last_fullsized":"2024-09-11T00:04:15.869720+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:14:26.125003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T00:04:17.435560+0000","last_change":"2024-09-11T00:04:03.836177+0000","last_active":"2024-09-11T00:04:17.435560+0000","last_peered":"2024-09-11T00:04:17.435560+0000","last_clean":"2024-09-11T00:04:17.435560+0000","last_became_active":"2024-09-11T00:04:03.835848+0000","last_became_peered":"2024-09-11T00:04:03.835848+0000","last_unstale":"2024-09-11T00:04:17.435560+0000","last_undegraded":"2024-09-11T00:04:17.435560+0000","last_fullsized":"2024-09-11T00:04:17.435560+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:57:53.854738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T00:04:17.436393+0000","last_change":"2024-09-11T00:04:03.837614+0000","last_active":"2024-09-11T00:04:17.436393+0000","last_peered":"2024-09-11T00:04:17.436393+0000","last_clean":"2024-09-11T00:04:17.436393+0000","last_became_active":"2024-09-11T00:04:03.837164+0000","last_became_peered":"2024-09-11T00:04:03.837164+0000","last_unstale":"2024-09-11T00:04:17.436393+0000","last_undegraded":"2024-09-11T00:04:17.436393+0000","last_fullsized":"2024-09-11T00:04:17.436393+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:10:38.467648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T00:04:17.304716+0000","last_change":"2024-09-11T00:04:03.833999+0000","last_active":"2024-09-11T00:04:17.304716+0000","last_peered":"2024-09-11T00:04:17.304716+0000","last_clean":"2024-09-11T00:04:17.304716+0000","last_became_active":"2024-09-11T00:04:03.833580+0000","last_became_peered":"2024-09-11T00:04:03.833580+0000","last_unstale":"2024-09-11T00:04:17.304716+0000","last_undegraded":"2024-09-11T00:04:17.304716+0000","last_fullsized":"2024-09-11T00:04:17.304716+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:47:40.902423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T00:04:17.304868+0000","last_change":"2024-09-11T00:04:03.832478+0000","last_active":"2024-09-11T00:04:17.304868+0000","last_peered":"2024-09-11T00:04:17.304868+0000","last_clean":"2024-09-11T00:04:17.304868+0000","last_became_active":"2024-09-11T00:04:03.831811+0000","last_became_peered":"2024-09-11T00:04:03.831811+0000","last_unstale":"2024-09-11T00:04:17.304868+0000","last_undegraded":"2024-09-11T00:04:17.304868+0000","last_fullsized":"2024-09-11T00:04:17.304868+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:53:05.839006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T00:04:15.869905+0000","last_change":"2024-09-11T00:04:03.837077+0000","last_active":"2024-09-11T00:04:15.869905+0000","last_peered":"2024-09-11T00:04:15.869905+0000","last_clean":"2024-09-11T00:04:15.869905+0000","last_became_active":"2024-09-11T00:04:03.835718+0000","last_became_peered":"2024-09-11T00:04:03.835718+0000","last_unstale":"2024-09-11T00:04:15.869905+0000","last_undegraded":"2024-09-11T00:04:15.869905+0000","last_fullsized":"2024-09-11T00:04:15.869905+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:17:00.633257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":50545,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":144,"num_read_kb":130,"num_write":230,"num_write_kb":116,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":516096,"data_stored":151635,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":201,"ondisk_log_size":201,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":68,"seq":292057776153,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36240,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708912,"statfs":{"total":95995035648,"available":95957925888,"internally_reserved":0,"allocated":917504,"data_stored":449299,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":36174276},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":63,"seq":270582939678,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28580,"kb_used_data":1556,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716572,"statfs":{"total":95995035648,"available":95965769728,"internally_reserved":0,"allocated":1593344,"data_stored":1079065,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":58,"seq":249108103201,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36268,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708884,"statfs":{"total":95995035648,"available":95957897216,"internally_reserved":0,"allocated":946176,"data_stored":452934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":52,"seq":223338299433,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36260,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708892,"statfs":{"total":95995035648,"available":95957905408,"internally_reserved":0,"allocated":937984,"data_stored":449216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":47,"seq":201863462956,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28524,"kb_used_data":1500,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716628,"statfs":{"total":95995035648,"available":95965827072,"internally_reserved":0,"allocated":1536000,"data_stored":1045713,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":42,"seq":180388626483,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32080,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713072,"statfs":{"total":95995035648,"available":95962185728,"internally_reserved":0,"allocated":917504,"data_stored":444733,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":36,"seq":154618822712,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28532,"kb_used_data":1508,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716620,"statfs":{"total":95995035648,"available":95965818880,"internally_reserved":0,"allocated":1544192,"data_stored":1040904,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986237,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27928,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717224,"statfs":{"total":95995035648,"available":95966437376,"internally_reserved":0,"allocated":925696,"data_stored":448416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182467,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32092,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713060,"statfs":{"total":95995035648,"available":95962173440,"internally_reserved":0,"allocated":929792,"data_stored":444050,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31914440},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411401,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27988,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717164,"statfs":{"total":95995035648,"available":95966375936,"internally_reserved":0,"allocated":987136,"data_stored":489324,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574925,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32128,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713024,"statfs":{"total":95995035648,"available":95962136576,"internally_reserved":0,"allocated":966656,"data_stored":489104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738452,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32104,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713048,"statfs":{"total":95995035648,"available":95962161152,"internally_reserved":0,"allocated":942080,"data_stored":446189,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2569,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":45484,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":45704,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":700,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":2820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3533,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":45077,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1583,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T00:05:06.377 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid ff3ad74a-6fcf-11ef-bcea-c7b262605968 -- ceph pg dump --format=json 2024-09-11T00:05:06.753 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:05:07.063 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:05:06 smithi012 ceph-mon[30333]: from='client.24877 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T00:05:07.080 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:05:06 smithi178 ceph-mon[35919]: from='client.24877 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T00:05:07.183 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:05:06 smithi165 ceph-mon[35356]: from='client.24877 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T00:05:07.307 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:07.304+0000 7fd086e00640 1 -- 172.21.15.12:0/3755449858 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd080108620 msgr2=0x7fd080108a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:05:07.307 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:07.304+0000 7fd086e00640 1 --2- 172.21.15.12:0/3755449858 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd080108620 0x7fd080108a60 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fd06c009860 tx=0x7fd06c02a340 comp rx=0 tx=0).stop 2024-09-11T00:05:07.307 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:07.304+0000 7fd086e00640 1 -- 172.21.15.12:0/3755449858 shutdown_connections 2024-09-11T00:05:07.307 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:07.304+0000 7fd086e00640 1 --2- 172.21.15.12:0/3755449858 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd080108fa0 0x7fd08010f7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:05:07.307 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:07.304+0000 7fd086e00640 1 --2- 172.21.15.12:0/3755449858 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd080108620 0x7fd080108a60 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:05:07.307 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:07.304+0000 7fd086e00640 1 --2- 172.21.15.12:0/3755449858 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd080102c10 0x7fd080102ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:05:07.307 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:07.304+0000 7fd086e00640 1 -- 172.21.15.12:0/3755449858 >> 172.21.15.12:0/3755449858 conn(0x7fd0800fe660 msgr2=0x7fd080100a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:05:07.307 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:07.304+0000 7fd086e00640 1 -- 172.21.15.12:0/3755449858 shutdown_connections 2024-09-11T00:05:07.307 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:07.304+0000 7fd086e00640 1 -- 172.21.15.12:0/3755449858 wait complete. 2024-09-11T00:05:07.308 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:07.305+0000 7fd086e00640 1 Processor -- start 2024-09-11T00:05:07.308 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:07.306+0000 7fd086e00640 1 -- start start 2024-09-11T00:05:07.308 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:07.306+0000 7fd086e00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd080102c10 0x7fd080199030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:05:07.308 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:07.306+0000 7fd086e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd080108620 0x7fd080199570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:05:07.308 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:07.306+0000 7fd086e00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd080108fa0 0x7fd08019e910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:05:07.309 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:07.306+0000 7fd086e00640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fd080112c60 con 0x7fd080108fa0 2024-09-11T00:05:07.309 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:07.306+0000 7fd086e00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fd080112ae0 con 0x7fd080108620 2024-09-11T00:05:07.309 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:07.306+0000 7fd086e00640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fd080112de0 con 0x7fd080102c10 2024-09-11T00:05:07.309 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:07.306+0000 7fd085a00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd080102c10 0x7fd080199030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:05:07.309 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:07.306+0000 7fd085000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd080108620 0x7fd080199570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:05:07.309 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:07.306+0000 7fd085a00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd080102c10 0x7fd080199030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.12:48146/0 (socket says 172.21.15.12:48146) 2024-09-11T00:05:07.309 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:07.306+0000 7fd085a00640 1 -- 172.21.15.12:0/827486785 learned_addr learned my addr 172.21.15.12:0/827486785 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:05:07.309 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:07.306+0000 7fd086400640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd080108fa0 0x7fd08019e910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:05:07.310 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:07.307+0000 7fd085000640 1 -- 172.21.15.12:0/827486785 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd080102c10 msgr2=0x7fd080199030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:05:07.310 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:07.307+0000 7fd085000640 1 --2- 172.21.15.12:0/827486785 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd080102c10 0x7fd080199030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:05:07.310 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:07.307+0000 7fd085000640 1 -- 172.21.15.12:0/827486785 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd080108fa0 msgr2=0x7fd08019e910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:05:07.310 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:07.307+0000 7fd085000640 1 --2- 172.21.15.12:0/827486785 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd080108fa0 0x7fd08019e910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:05:07.310 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:07.307+0000 7fd085000640 1 -- 172.21.15.12:0/827486785 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd08019f0e0 con 0x7fd080108620 2024-09-11T00:05:07.310 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:07.307+0000 7fd085000640 1 --2- 172.21.15.12:0/827486785 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd080108620 0x7fd080199570 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fd06c035040 tx=0x7fd06c032d50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:05:07.310 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:07.307+0000 7fd07e000640 1 -- 172.21.15.12:0/827486785 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd06c037040 con 0x7fd080108620 2024-09-11T00:05:07.310 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:07.307+0000 7fd07e000640 1 -- 172.21.15.12:0/827486785 <== mon.2 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd06c030c10 con 0x7fd080108620 2024-09-11T00:05:07.310 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:07.307+0000 7fd07e000640 1 -- 172.21.15.12:0/827486785 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd06c0035f0 con 0x7fd080108620 2024-09-11T00:05:07.311 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:07.308+0000 7fd086e00640 1 -- 172.21.15.12:0/827486785 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd08019f370 con 0x7fd080108620 2024-09-11T00:05:07.311 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:07.308+0000 7fd086e00640 1 -- 172.21.15.12:0/827486785 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0801a5a90 con 0x7fd080108620 2024-09-11T00:05:07.311 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:07.309+0000 7fd086e00640 1 -- 172.21.15.12:0/827486785 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd080102ff0 con 0x7fd080108620 2024-09-11T00:05:07.313 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:07.311+0000 7fd07e000640 1 -- 172.21.15.12:0/827486785 <== mon.2 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 17) ==== 153301+0+0 (secure 0 0 0) 0x7fd06c004050 con 0x7fd080108620 2024-09-11T00:05:07.315 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:07.313+0000 7fd07e000640 1 --2- 172.21.15.12:0/827486785 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fd0680b5410 0x7fd0680b78d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:05:07.315 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:07.313+0000 7fd07e000640 1 -- 172.21.15.12:0/827486785 <== mon.2 v2:172.21.15.165:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8802+0+0 (secure 0 0 0) 0x7fd06c1054f0 con 0x7fd080108620 2024-09-11T00:05:07.315 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:07.313+0000 7fd085a00640 1 --2- 172.21.15.12:0/827486785 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fd0680b5410 0x7fd0680b78d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:05:07.316 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:07.314+0000 7fd085a00640 1 --2- 172.21.15.12:0/827486785 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fd0680b5410 0x7fd0680b78d0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fd074009cd0 tx=0x7fd074005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:05:07.316 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:07.315+0000 7fd07e000640 1 -- 172.21.15.12:0/827486785 <== mon.2 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7fd06c04a080 con 0x7fd080108620 2024-09-11T00:05:07.510 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:07.508+0000 7fd086e00640 1 -- 172.21.15.12:0/827486785 --> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fd080105be0 con 0x7fd0680b5410 2024-09-11T00:05:07.517 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:07.515+0000 7fd07e000640 1 -- 172.21.15.12:0/827486785 <== mgr.14150 v2:172.21.15.12:6800/3442939137 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340376 (secure 0 0 0) 0x7fd080105be0 con 0x7fd0680b5410 2024-09-11T00:05:07.522 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-11T00:05:07.524 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-11T00:05:07.527 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:07.520+0000 7fd086e00640 1 -- 172.21.15.12:0/827486785 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fd0680b5410 msgr2=0x7fd0680b78d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:05:07.528 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:07.520+0000 7fd086e00640 1 --2- 172.21.15.12:0/827486785 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fd0680b5410 0x7fd0680b78d0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fd074009cd0 tx=0x7fd074005f90 comp rx=0 tx=0).stop 2024-09-11T00:05:07.528 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:07.520+0000 7fd086e00640 1 -- 172.21.15.12:0/827486785 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd080108620 msgr2=0x7fd080199570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:05:07.528 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:07.520+0000 7fd086e00640 1 --2- 172.21.15.12:0/827486785 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd080108620 0x7fd080199570 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fd06c035040 tx=0x7fd06c032d50 comp rx=0 tx=0).stop 2024-09-11T00:05:07.528 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:07.520+0000 7fd086e00640 1 -- 172.21.15.12:0/827486785 shutdown_connections 2024-09-11T00:05:07.528 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:07.520+0000 7fd086e00640 1 --2- 172.21.15.12:0/827486785 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fd0680b5410 0x7fd0680b78d0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:05:07.528 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:07.520+0000 7fd086e00640 1 --2- 172.21.15.12:0/827486785 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd080108fa0 0x7fd08019e910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:05:07.528 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:07.520+0000 7fd086e00640 1 --2- 172.21.15.12:0/827486785 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd080108620 0x7fd080199570 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:05:07.528 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:07.520+0000 7fd086e00640 1 --2- 172.21.15.12:0/827486785 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd080102c10 0x7fd080199030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:05:07.528 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:07.520+0000 7fd086e00640 1 -- 172.21.15.12:0/827486785 >> 172.21.15.12:0/827486785 conn(0x7fd0800fe660 msgr2=0x7fd080101b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:05:07.528 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:07.520+0000 7fd086e00640 1 -- 172.21.15.12:0/827486785 shutdown_connections 2024-09-11T00:05:07.528 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:07.520+0000 7fd086e00640 1 -- 172.21.15.12:0/827486785 wait complete. 2024-09-11T00:05:08.063 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:05:07 smithi012 ceph-mon[30333]: pgmap v335: 129 pgs: 129 active+clean; 639 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 5.6 KiB/s wr, 17 op/s 2024-09-11T00:05:08.080 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:05:07 smithi178 ceph-mon[35919]: pgmap v335: 129 pgs: 129 active+clean; 639 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 5.6 KiB/s wr, 17 op/s 2024-09-11T00:05:08.258 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":335,"stamp":"2024-09-11T00:05:06.104769+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":653966,"num_objects":102,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":271,"num_read_kb":232,"num_write":367,"num_write_kb":2284,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":303,"ondisk_log_size":303,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":378760,"kb_used_data":12872,"kb_used_omap":18,"kb_used_meta":365677,"kb_avail":1124563064,"statfs":{"total":1151940427776,"available":1151552577536,"internally_reserved":0,"allocated":13180928,"data_stored":7317404,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19142,"internal_metadata":374453562},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":53334,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":171,"num_read_kb":144,"num_write":35,"num_write_kb":67,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003468"},"pg_stats":[{"pgid":"2.1f","version":"81'13","reported_seq":62,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T00:05:03.283134+0000","last_change":"2024-09-11T00:04:01.820872+0000","last_active":"2024-09-11T00:05:03.283134+0000","last_peered":"2024-09-11T00:05:03.283134+0000","last_clean":"2024-09-11T00:05:03.283134+0000","last_became_active":"2024-09-11T00:04:01.820404+0000","last_became_peered":"2024-09-11T00:04:01.820404+0000","last_unstale":"2024-09-11T00:05:03.283134+0000","last_undegraded":"2024-09-11T00:05:03.283134+0000","last_fullsized":"2024-09-11T00:05:03.283134+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_clean_scrub_stamp":"2024-09-11T00:04:00.794959+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:15:38.648289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":16,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T00:04:16.752091+0000","last_change":"2024-09-11T00:04:03.842131+0000","last_active":"2024-09-11T00:04:16.752091+0000","last_peered":"2024-09-11T00:04:16.752091+0000","last_clean":"2024-09-11T00:04:16.752091+0000","last_became_active":"2024-09-11T00:04:03.841555+0000","last_became_peered":"2024-09-11T00:04:03.841555+0000","last_unstale":"2024-09-11T00:04:16.752091+0000","last_undegraded":"2024-09-11T00:04:16.752091+0000","last_fullsized":"2024-09-11T00:04:16.752091+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:39:28.301596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":21,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T00:04:16.882034+0000","last_change":"2024-09-11T00:04:11.872109+0000","last_active":"2024-09-11T00:04:16.882034+0000","last_peered":"2024-09-11T00:04:16.882034+0000","last_clean":"2024-09-11T00:04:16.882034+0000","last_became_active":"2024-09-11T00:04:11.871279+0000","last_became_peered":"2024-09-11T00:04:11.871279+0000","last_unstale":"2024-09-11T00:04:16.882034+0000","last_undegraded":"2024-09-11T00:04:16.882034+0000","last_fullsized":"2024-09-11T00:04:16.882034+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_clean_scrub_stamp":"2024-09-11T00:04:10.845136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:13:32.837555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"81'2","reported_seq":35,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T00:04:16.752110+0000","last_change":"2024-09-11T00:04:01.942288+0000","last_active":"2024-09-11T00:04:16.752110+0000","last_peered":"2024-09-11T00:04:16.752110+0000","last_clean":"2024-09-11T00:04:16.752110+0000","last_became_active":"2024-09-11T00:04:01.941838+0000","last_became_peered":"2024-09-11T00:04:01.941838+0000","last_unstale":"2024-09-11T00:04:16.752110+0000","last_undegraded":"2024-09-11T00:04:16.752110+0000","last_fullsized":"2024-09-11T00:04:16.752110+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_clean_scrub_stamp":"2024-09-11T00:04:00.794959+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:48:44.867267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T00:04:17.305155+0000","last_change":"2024-09-11T00:04:03.839951+0000","last_active":"2024-09-11T00:04:17.305155+0000","last_peered":"2024-09-11T00:04:17.305155+0000","last_clean":"2024-09-11T00:04:17.305155+0000","last_became_active":"2024-09-11T00:04:03.839591+0000","last_became_peered":"2024-09-11T00:04:03.839591+0000","last_unstale":"2024-09-11T00:04:17.305155+0000","last_undegraded":"2024-09-11T00:04:17.305155+0000","last_fullsized":"2024-09-11T00:04:17.305155+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:12:49.603310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":21,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T00:04:17.435436+0000","last_change":"2024-09-11T00:04:11.872026+0000","last_active":"2024-09-11T00:04:17.435436+0000","last_peered":"2024-09-11T00:04:17.435436+0000","last_clean":"2024-09-11T00:04:17.435436+0000","last_became_active":"2024-09-11T00:04:11.871699+0000","last_became_peered":"2024-09-11T00:04:11.871699+0000","last_unstale":"2024-09-11T00:04:17.435436+0000","last_undegraded":"2024-09-11T00:04:17.435436+0000","last_fullsized":"2024-09-11T00:04:17.435436+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_clean_scrub_stamp":"2024-09-11T00:04:10.845136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:07:47.757223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"81'12","reported_seq":46,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T00:04:50.254784+0000","last_change":"2024-09-11T00:04:01.819942+0000","last_active":"2024-09-11T00:04:50.254784+0000","last_peered":"2024-09-11T00:04:50.254784+0000","last_clean":"2024-09-11T00:04:50.254784+0000","last_became_active":"2024-09-11T00:04:01.819337+0000","last_became_peered":"2024-09-11T00:04:01.819337+0000","last_unstale":"2024-09-11T00:04:50.254784+0000","last_undegraded":"2024-09-11T00:04:50.254784+0000","last_fullsized":"2024-09-11T00:04:50.254784+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_clean_scrub_stamp":"2024-09-11T00:04:00.794959+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:22:26.947067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T00:04:17.435640+0000","last_change":"2024-09-11T00:04:03.839276+0000","last_active":"2024-09-11T00:04:17.435640+0000","last_peered":"2024-09-11T00:04:17.435640+0000","last_clean":"2024-09-11T00:04:17.435640+0000","last_became_active":"2024-09-11T00:04:03.838843+0000","last_became_peered":"2024-09-11T00:04:03.838843+0000","last_unstale":"2024-09-11T00:04:17.435640+0000","last_undegraded":"2024-09-11T00:04:17.435640+0000","last_fullsized":"2024-09-11T00:04:17.435640+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:26:42.640628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":21,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T00:04:17.436523+0000","last_change":"2024-09-11T00:04:11.872345+0000","last_active":"2024-09-11T00:04:17.436523+0000","last_peered":"2024-09-11T00:04:17.436523+0000","last_clean":"2024-09-11T00:04:17.436523+0000","last_became_active":"2024-09-11T00:04:11.871994+0000","last_became_peered":"2024-09-11T00:04:11.871994+0000","last_unstale":"2024-09-11T00:04:17.436523+0000","last_undegraded":"2024-09-11T00:04:17.436523+0000","last_fullsized":"2024-09-11T00:04:17.436523+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_clean_scrub_stamp":"2024-09-11T00:04:10.845136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:14:23.906778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"81'5","reported_seq":39,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T00:04:49.248998+0000","last_change":"2024-09-11T00:04:01.819581+0000","last_active":"2024-09-11T00:04:49.248998+0000","last_peered":"2024-09-11T00:04:49.248998+0000","last_clean":"2024-09-11T00:04:49.248998+0000","last_became_active":"2024-09-11T00:04:01.818699+0000","last_became_peered":"2024-09-11T00:04:01.818699+0000","last_unstale":"2024-09-11T00:04:49.248998+0000","last_undegraded":"2024-09-11T00:04:49.248998+0000","last_fullsized":"2024-09-11T00:04:49.248998+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_clean_scrub_stamp":"2024-09-11T00:04:00.794959+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:07:24.540825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T00:04:17.436276+0000","last_change":"2024-09-11T00:04:03.835107+0000","last_active":"2024-09-11T00:04:17.436276+0000","last_peered":"2024-09-11T00:04:17.436276+0000","last_clean":"2024-09-11T00:04:17.436276+0000","last_became_active":"2024-09-11T00:04:03.834782+0000","last_became_peered":"2024-09-11T00:04:03.834782+0000","last_unstale":"2024-09-11T00:04:17.436276+0000","last_undegraded":"2024-09-11T00:04:17.436276+0000","last_fullsized":"2024-09-11T00:04:17.436276+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:46:54.472270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":21,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T00:04:17.435721+0000","last_change":"2024-09-11T00:04:11.874168+0000","last_active":"2024-09-11T00:04:17.435721+0000","last_peered":"2024-09-11T00:04:17.435721+0000","last_clean":"2024-09-11T00:04:17.435721+0000","last_became_active":"2024-09-11T00:04:11.873810+0000","last_became_peered":"2024-09-11T00:04:11.873810+0000","last_unstale":"2024-09-11T00:04:17.435721+0000","last_undegraded":"2024-09-11T00:04:17.435721+0000","last_fullsized":"2024-09-11T00:04:17.435721+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_clean_scrub_stamp":"2024-09-11T00:04:10.845136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:06:05.130815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T00:04:15.912270+0000","last_change":"2024-09-11T00:04:03.834016+0000","last_active":"2024-09-11T00:04:15.912270+0000","last_peered":"2024-09-11T00:04:15.912270+0000","last_clean":"2024-09-11T00:04:15.912270+0000","last_became_active":"2024-09-11T00:04:03.833374+0000","last_became_peered":"2024-09-11T00:04:03.833374+0000","last_unstale":"2024-09-11T00:04:15.912270+0000","last_undegraded":"2024-09-11T00:04:15.912270+0000","last_fullsized":"2024-09-11T00:04:15.912270+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:13:34.068594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"81'4","reported_seq":38,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T00:04:46.232389+0000","last_change":"2024-09-11T00:04:01.821173+0000","last_active":"2024-09-11T00:04:46.232389+0000","last_peered":"2024-09-11T00:04:46.232389+0000","last_clean":"2024-09-11T00:04:46.232389+0000","last_became_active":"2024-09-11T00:04:01.820615+0000","last_became_peered":"2024-09-11T00:04:01.820615+0000","last_unstale":"2024-09-11T00:04:46.232389+0000","last_undegraded":"2024-09-11T00:04:46.232389+0000","last_fullsized":"2024-09-11T00:04:46.232389+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_clean_scrub_stamp":"2024-09-11T00:04:00.794959+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:22:11.842837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":21,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T00:04:17.436132+0000","last_change":"2024-09-11T00:04:12.077637+0000","last_active":"2024-09-11T00:04:17.436132+0000","last_peered":"2024-09-11T00:04:17.436132+0000","last_clean":"2024-09-11T00:04:17.436132+0000","last_became_active":"2024-09-11T00:04:12.076450+0000","last_became_peered":"2024-09-11T00:04:12.076450+0000","last_unstale":"2024-09-11T00:04:17.436132+0000","last_undegraded":"2024-09-11T00:04:17.436132+0000","last_fullsized":"2024-09-11T00:04:17.436132+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_clean_scrub_stamp":"2024-09-11T00:04:10.845136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:50:54.870468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T00:04:16.751592+0000","last_change":"2024-09-11T00:04:03.846398+0000","last_active":"2024-09-11T00:04:16.751592+0000","last_peered":"2024-09-11T00:04:16.751592+0000","last_clean":"2024-09-11T00:04:16.751592+0000","last_became_active":"2024-09-11T00:04:03.846051+0000","last_became_peered":"2024-09-11T00:04:03.846051+0000","last_unstale":"2024-09-11T00:04:16.751592+0000","last_undegraded":"2024-09-11T00:04:16.751592+0000","last_fullsized":"2024-09-11T00:04:16.751592+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:00:38.506122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"81'3","reported_seq":36,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T00:04:48.244416+0000","last_change":"2024-09-11T00:04:01.818083+0000","last_active":"2024-09-11T00:04:48.244416+0000","last_peered":"2024-09-11T00:04:48.244416+0000","last_clean":"2024-09-11T00:04:48.244416+0000","last_became_active":"2024-09-11T00:04:01.817253+0000","last_became_peered":"2024-09-11T00:04:01.817253+0000","last_unstale":"2024-09-11T00:04:48.244416+0000","last_undegraded":"2024-09-11T00:04:48.244416+0000","last_fullsized":"2024-09-11T00:04:48.244416+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_clean_scrub_stamp":"2024-09-11T00:04:00.794959+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:35:23.429074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"81'1","reported_seq":22,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T00:04:17.436272+0000","last_change":"2024-09-11T00:04:11.879582+0000","last_active":"2024-09-11T00:04:17.436272+0000","last_peered":"2024-09-11T00:04:17.436272+0000","last_clean":"2024-09-11T00:04:17.436272+0000","last_became_active":"2024-09-11T00:04:11.879098+0000","last_became_peered":"2024-09-11T00:04:11.879098+0000","last_unstale":"2024-09-11T00:04:17.436272+0000","last_undegraded":"2024-09-11T00:04:17.436272+0000","last_fullsized":"2024-09-11T00:04:17.436272+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_clean_scrub_stamp":"2024-09-11T00:04:10.845136+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:06.292841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T00:04:15.869547+0000","last_change":"2024-09-11T00:04:03.834615+0000","last_active":"2024-09-11T00:04:15.869547+0000","last_peered":"2024-09-11T00:04:15.869547+0000","last_clean":"2024-09-11T00:04:15.869547+0000","last_became_active":"2024-09-11T00:04:03.834284+0000","last_became_peered":"2024-09-11T00:04:03.834284+0000","last_unstale":"2024-09-11T00:04:15.869547+0000","last_undegraded":"2024-09-11T00:04:15.869547+0000","last_fullsized":"2024-09-11T00:04:15.869547+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:24:09.542774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"81'7","reported_seq":40,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T00:04:49.250027+0000","last_change":"2024-09-11T00:04:01.818156+0000","last_active":"2024-09-11T00:04:49.250027+0000","last_peered":"2024-09-11T00:04:49.250027+0000","last_clean":"2024-09-11T00:04:49.250027+0000","last_became_active":"2024-09-11T00:04:01.817820+0000","last_became_peered":"2024-09-11T00:04:01.817820+0000","last_unstale":"2024-09-11T00:04:49.250027+0000","last_undegraded":"2024-09-11T00:04:49.250027+0000","last_fullsized":"2024-09-11T00:04:49.250027+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_clean_scrub_stamp":"2024-09-11T00:04:00.794959+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:15:46.304276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":21,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T00:04:17.435835+0000","last_change":"2024-09-11T00:04:11.877704+0000","last_active":"2024-09-11T00:04:17.435835+0000","last_peered":"2024-09-11T00:04:17.435835+0000","last_clean":"2024-09-11T00:04:17.435835+0000","last_became_active":"2024-09-11T00:04:11.877320+0000","last_became_peered":"2024-09-11T00:04:11.877320+0000","last_unstale":"2024-09-11T00:04:17.435835+0000","last_undegraded":"2024-09-11T00:04:17.435835+0000","last_fullsized":"2024-09-11T00:04:17.435835+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_clean_scrub_stamp":"2024-09-11T00:04:10.845136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:31:35.046012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T00:04:17.435622+0000","last_change":"2024-09-11T00:04:03.828240+0000","last_active":"2024-09-11T00:04:17.435622+0000","last_peered":"2024-09-11T00:04:17.435622+0000","last_clean":"2024-09-11T00:04:17.435622+0000","last_became_active":"2024-09-11T00:04:03.827890+0000","last_became_peered":"2024-09-11T00:04:03.827890+0000","last_unstale":"2024-09-11T00:04:17.435622+0000","last_undegraded":"2024-09-11T00:04:17.435622+0000","last_fullsized":"2024-09-11T00:04:17.435622+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:43:12.146717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"81'2","reported_seq":35,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T00:04:46.232647+0000","last_change":"2024-09-11T00:04:01.942805+0000","last_active":"2024-09-11T00:04:46.232647+0000","last_peered":"2024-09-11T00:04:46.232647+0000","last_clean":"2024-09-11T00:04:46.232647+0000","last_became_active":"2024-09-11T00:04:01.942429+0000","last_became_peered":"2024-09-11T00:04:01.942429+0000","last_unstale":"2024-09-11T00:04:46.232647+0000","last_undegraded":"2024-09-11T00:04:46.232647+0000","last_fullsized":"2024-09-11T00:04:46.232647+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_clean_scrub_stamp":"2024-09-11T00:04:00.794959+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:51:22.403825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":21,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T00:04:17.436458+0000","last_change":"2024-09-11T00:04:12.077246+0000","last_active":"2024-09-11T00:04:17.436458+0000","last_peered":"2024-09-11T00:04:17.436458+0000","last_clean":"2024-09-11T00:04:17.436458+0000","last_became_active":"2024-09-11T00:04:12.076481+0000","last_became_peered":"2024-09-11T00:04:12.076481+0000","last_unstale":"2024-09-11T00:04:17.436458+0000","last_undegraded":"2024-09-11T00:04:17.436458+0000","last_fullsized":"2024-09-11T00:04:17.436458+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_clean_scrub_stamp":"2024-09-11T00:04:10.845136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:55:54.347334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T00:04:17.304877+0000","last_change":"2024-09-11T00:04:03.838431+0000","last_active":"2024-09-11T00:04:17.304877+0000","last_peered":"2024-09-11T00:04:17.304877+0000","last_clean":"2024-09-11T00:04:17.304877+0000","last_became_active":"2024-09-11T00:04:03.834451+0000","last_became_peered":"2024-09-11T00:04:03.834451+0000","last_unstale":"2024-09-11T00:04:17.304877+0000","last_undegraded":"2024-09-11T00:04:17.304877+0000","last_fullsized":"2024-09-11T00:04:17.304877+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:23:01.594299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"81'7","reported_seq":41,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T00:04:49.250641+0000","last_change":"2024-09-11T00:04:02.588834+0000","last_active":"2024-09-11T00:04:49.250641+0000","last_peered":"2024-09-11T00:04:49.250641+0000","last_clean":"2024-09-11T00:04:49.250641+0000","last_became_active":"2024-09-11T00:04:02.588477+0000","last_became_peered":"2024-09-11T00:04:02.588477+0000","last_unstale":"2024-09-11T00:04:49.250641+0000","last_undegraded":"2024-09-11T00:04:49.250641+0000","last_fullsized":"2024-09-11T00:04:49.250641+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_clean_scrub_stamp":"2024-09-11T00:04:00.794959+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:31:21.805591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":21,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T00:04:16.882086+0000","last_change":"2024-09-11T00:04:11.863381+0000","last_active":"2024-09-11T00:04:16.882086+0000","last_peered":"2024-09-11T00:04:16.882086+0000","last_clean":"2024-09-11T00:04:16.882086+0000","last_became_active":"2024-09-11T00:04:11.862944+0000","last_became_peered":"2024-09-11T00:04:11.862944+0000","last_unstale":"2024-09-11T00:04:16.882086+0000","last_undegraded":"2024-09-11T00:04:16.882086+0000","last_fullsized":"2024-09-11T00:04:16.882086+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_clean_scrub_stamp":"2024-09-11T00:04:10.845136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:00:14.185992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T00:04:16.751871+0000","last_change":"2024-09-11T00:04:03.829750+0000","last_active":"2024-09-11T00:04:16.751871+0000","last_peered":"2024-09-11T00:04:16.751871+0000","last_clean":"2024-09-11T00:04:16.751871+0000","last_became_active":"2024-09-11T00:04:03.828166+0000","last_became_peered":"2024-09-11T00:04:03.828166+0000","last_unstale":"2024-09-11T00:04:16.751871+0000","last_undegraded":"2024-09-11T00:04:16.751871+0000","last_fullsized":"2024-09-11T00:04:16.751871+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:55:01.667075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"81'14","reported_seq":47,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T00:05:02.852237+0000","last_change":"2024-09-11T00:04:01.818083+0000","last_active":"2024-09-11T00:05:02.852237+0000","last_peered":"2024-09-11T00:05:02.852237+0000","last_clean":"2024-09-11T00:05:02.852237+0000","last_became_active":"2024-09-11T00:04:01.817784+0000","last_became_peered":"2024-09-11T00:04:01.817784+0000","last_unstale":"2024-09-11T00:05:02.852237+0000","last_undegraded":"2024-09-11T00:05:02.852237+0000","last_fullsized":"2024-09-11T00:05:02.852237+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_clean_scrub_stamp":"2024-09-11T00:04:00.794959+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:38:55.466913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":21,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T00:04:16.751305+0000","last_change":"2024-09-11T00:04:12.077053+0000","last_active":"2024-09-11T00:04:16.751305+0000","last_peered":"2024-09-11T00:04:16.751305+0000","last_clean":"2024-09-11T00:04:16.751305+0000","last_became_active":"2024-09-11T00:04:12.076632+0000","last_became_peered":"2024-09-11T00:04:12.076632+0000","last_unstale":"2024-09-11T00:04:16.751305+0000","last_undegraded":"2024-09-11T00:04:16.751305+0000","last_fullsized":"2024-09-11T00:04:16.751305+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_clean_scrub_stamp":"2024-09-11T00:04:10.845136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:19:38.321429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T00:04:15.869722+0000","last_change":"2024-09-11T00:04:03.830316+0000","last_active":"2024-09-11T00:04:15.869722+0000","last_peered":"2024-09-11T00:04:15.869722+0000","last_clean":"2024-09-11T00:04:15.869722+0000","last_became_active":"2024-09-11T00:04:03.829932+0000","last_became_peered":"2024-09-11T00:04:03.829932+0000","last_unstale":"2024-09-11T00:04:15.869722+0000","last_undegraded":"2024-09-11T00:04:15.869722+0000","last_fullsized":"2024-09-11T00:04:15.869722+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:06:14.546651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"81'4","reported_seq":38,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T00:04:49.250496+0000","last_change":"2024-09-11T00:04:01.943004+0000","last_active":"2024-09-11T00:04:49.250496+0000","last_peered":"2024-09-11T00:04:49.250496+0000","last_clean":"2024-09-11T00:04:49.250496+0000","last_became_active":"2024-09-11T00:04:01.942432+0000","last_became_peered":"2024-09-11T00:04:01.942432+0000","last_unstale":"2024-09-11T00:04:49.250496+0000","last_undegraded":"2024-09-11T00:04:49.250496+0000","last_fullsized":"2024-09-11T00:04:49.250496+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_clean_scrub_stamp":"2024-09-11T00:04:00.794959+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:52:50.472596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":21,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T00:04:15.869457+0000","last_change":"2024-09-11T00:04:11.866778+0000","last_active":"2024-09-11T00:04:15.869457+0000","last_peered":"2024-09-11T00:04:15.869457+0000","last_clean":"2024-09-11T00:04:15.869457+0000","last_became_active":"2024-09-11T00:04:11.866333+0000","last_became_peered":"2024-09-11T00:04:11.866333+0000","last_unstale":"2024-09-11T00:04:15.869457+0000","last_undegraded":"2024-09-11T00:04:15.869457+0000","last_fullsized":"2024-09-11T00:04:15.869457+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_clean_scrub_stamp":"2024-09-11T00:04:10.845136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:33.680119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T00:04:15.869851+0000","last_change":"2024-09-11T00:04:03.832481+0000","last_active":"2024-09-11T00:04:15.869851+0000","last_peered":"2024-09-11T00:04:15.869851+0000","last_clean":"2024-09-11T00:04:15.869851+0000","last_became_active":"2024-09-11T00:04:03.832162+0000","last_became_peered":"2024-09-11T00:04:03.832162+0000","last_unstale":"2024-09-11T00:04:15.869851+0000","last_undegraded":"2024-09-11T00:04:15.869851+0000","last_fullsized":"2024-09-11T00:04:15.869851+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:32:52.550128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"81'1","reported_seq":34,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T00:04:17.435694+0000","last_change":"2024-09-11T00:04:01.943647+0000","last_active":"2024-09-11T00:04:17.435694+0000","last_peered":"2024-09-11T00:04:17.435694+0000","last_clean":"2024-09-11T00:04:17.435694+0000","last_became_active":"2024-09-11T00:04:01.942714+0000","last_became_peered":"2024-09-11T00:04:01.942714+0000","last_unstale":"2024-09-11T00:04:17.435694+0000","last_undegraded":"2024-09-11T00:04:17.435694+0000","last_fullsized":"2024-09-11T00:04:17.435694+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_clean_scrub_stamp":"2024-09-11T00:04:00.794959+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:22:23.687268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":21,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T00:04:17.435685+0000","last_change":"2024-09-11T00:04:12.077133+0000","last_active":"2024-09-11T00:04:17.435685+0000","last_peered":"2024-09-11T00:04:17.435685+0000","last_clean":"2024-09-11T00:04:17.435685+0000","last_became_active":"2024-09-11T00:04:12.076657+0000","last_became_peered":"2024-09-11T00:04:12.076657+0000","last_unstale":"2024-09-11T00:04:17.435685+0000","last_undegraded":"2024-09-11T00:04:17.435685+0000","last_fullsized":"2024-09-11T00:04:17.435685+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_clean_scrub_stamp":"2024-09-11T00:04:10.845136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:05:40.643405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T00:04:15.870093+0000","last_change":"2024-09-11T00:04:03.835454+0000","last_active":"2024-09-11T00:04:15.870093+0000","last_peered":"2024-09-11T00:04:15.870093+0000","last_clean":"2024-09-11T00:04:15.870093+0000","last_became_active":"2024-09-11T00:04:03.835122+0000","last_became_peered":"2024-09-11T00:04:03.835122+0000","last_unstale":"2024-09-11T00:04:15.870093+0000","last_undegraded":"2024-09-11T00:04:15.870093+0000","last_fullsized":"2024-09-11T00:04:15.870093+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:25:49.672766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"81'6","reported_seq":57,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T00:05:04.285126+0000","last_change":"2024-09-11T00:04:01.823738+0000","last_active":"2024-09-11T00:05:04.285126+0000","last_peered":"2024-09-11T00:05:04.285126+0000","last_clean":"2024-09-11T00:05:04.285126+0000","last_became_active":"2024-09-11T00:04:01.823320+0000","last_became_peered":"2024-09-11T00:04:01.823320+0000","last_unstale":"2024-09-11T00:05:04.285126+0000","last_undegraded":"2024-09-11T00:05:04.285126+0000","last_fullsized":"2024-09-11T00:05:04.285126+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_clean_scrub_stamp":"2024-09-11T00:04:00.794959+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:34:30.088454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":18,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":21,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T00:04:15.869640+0000","last_change":"2024-09-11T00:04:11.863906+0000","last_active":"2024-09-11T00:04:15.869640+0000","last_peered":"2024-09-11T00:04:15.869640+0000","last_clean":"2024-09-11T00:04:15.869640+0000","last_became_active":"2024-09-11T00:04:11.863571+0000","last_became_peered":"2024-09-11T00:04:11.863571+0000","last_unstale":"2024-09-11T00:04:15.869640+0000","last_undegraded":"2024-09-11T00:04:15.869640+0000","last_fullsized":"2024-09-11T00:04:15.869640+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_clean_scrub_stamp":"2024-09-11T00:04:10.845136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:04:40.288886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T00:04:17.436310+0000","last_change":"2024-09-11T00:04:03.826623+0000","last_active":"2024-09-11T00:04:17.436310+0000","last_peered":"2024-09-11T00:04:17.436310+0000","last_clean":"2024-09-11T00:04:17.436310+0000","last_became_active":"2024-09-11T00:04:03.826202+0000","last_became_peered":"2024-09-11T00:04:03.826202+0000","last_unstale":"2024-09-11T00:04:17.436310+0000","last_undegraded":"2024-09-11T00:04:17.436310+0000","last_fullsized":"2024-09-11T00:04:17.436310+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:23:13.222654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"81'5","reported_seq":54,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T00:05:03.276816+0000","last_change":"2024-09-11T00:04:01.942857+0000","last_active":"2024-09-11T00:05:03.276816+0000","last_peered":"2024-09-11T00:05:03.276816+0000","last_clean":"2024-09-11T00:05:03.276816+0000","last_became_active":"2024-09-11T00:04:01.942494+0000","last_became_peered":"2024-09-11T00:04:01.942494+0000","last_unstale":"2024-09-11T00:05:03.276816+0000","last_undegraded":"2024-09-11T00:05:03.276816+0000","last_fullsized":"2024-09-11T00:05:03.276816+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_clean_scrub_stamp":"2024-09-11T00:04:00.794959+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:40:38.587178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":16,"num_read_kb":16,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":21,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T00:04:15.869817+0000","last_change":"2024-09-11T00:04:12.077771+0000","last_active":"2024-09-11T00:04:15.869817+0000","last_peered":"2024-09-11T00:04:15.869817+0000","last_clean":"2024-09-11T00:04:15.869817+0000","last_became_active":"2024-09-11T00:04:12.077366+0000","last_became_peered":"2024-09-11T00:04:12.077366+0000","last_unstale":"2024-09-11T00:04:15.869817+0000","last_undegraded":"2024-09-11T00:04:15.869817+0000","last_fullsized":"2024-09-11T00:04:15.869817+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_clean_scrub_stamp":"2024-09-11T00:04:10.845136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:04:26.808201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T00:04:15.869166+0000","last_change":"2024-09-11T00:04:03.827872+0000","last_active":"2024-09-11T00:04:15.869166+0000","last_peered":"2024-09-11T00:04:15.869166+0000","last_clean":"2024-09-11T00:04:15.869166+0000","last_became_active":"2024-09-11T00:04:03.827577+0000","last_became_peered":"2024-09-11T00:04:03.827577+0000","last_unstale":"2024-09-11T00:04:15.869166+0000","last_undegraded":"2024-09-11T00:04:15.869166+0000","last_fullsized":"2024-09-11T00:04:15.869166+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:37:19.205218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"81'4","reported_seq":37,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T00:04:47.236745+0000","last_change":"2024-09-11T00:04:01.819014+0000","last_active":"2024-09-11T00:04:47.236745+0000","last_peered":"2024-09-11T00:04:47.236745+0000","last_clean":"2024-09-11T00:04:47.236745+0000","last_became_active":"2024-09-11T00:04:01.818671+0000","last_became_peered":"2024-09-11T00:04:01.818671+0000","last_unstale":"2024-09-11T00:04:47.236745+0000","last_undegraded":"2024-09-11T00:04:47.236745+0000","last_fullsized":"2024-09-11T00:04:47.236745+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_clean_scrub_stamp":"2024-09-11T00:04:00.794959+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:42:55.226613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":21,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T00:04:17.436640+0000","last_change":"2024-09-11T00:04:12.077554+0000","last_active":"2024-09-11T00:04:17.436640+0000","last_peered":"2024-09-11T00:04:17.436640+0000","last_clean":"2024-09-11T00:04:17.436640+0000","last_became_active":"2024-09-11T00:04:12.076619+0000","last_became_peered":"2024-09-11T00:04:12.076619+0000","last_unstale":"2024-09-11T00:04:17.436640+0000","last_undegraded":"2024-09-11T00:04:17.436640+0000","last_fullsized":"2024-09-11T00:04:17.436640+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_clean_scrub_stamp":"2024-09-11T00:04:10.845136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:30:25.505200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T00:04:16.751606+0000","last_change":"2024-09-11T00:04:03.839615+0000","last_active":"2024-09-11T00:04:16.751606+0000","last_peered":"2024-09-11T00:04:16.751606+0000","last_clean":"2024-09-11T00:04:16.751606+0000","last_became_active":"2024-09-11T00:04:03.832175+0000","last_became_peered":"2024-09-11T00:04:03.832175+0000","last_unstale":"2024-09-11T00:04:16.751606+0000","last_undegraded":"2024-09-11T00:04:16.751606+0000","last_fullsized":"2024-09-11T00:04:16.751606+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:17:09.975345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"81'6","reported_seq":42,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T00:04:50.254721+0000","last_change":"2024-09-11T00:04:01.817248+0000","last_active":"2024-09-11T00:04:50.254721+0000","last_peered":"2024-09-11T00:04:50.254721+0000","last_clean":"2024-09-11T00:04:50.254721+0000","last_became_active":"2024-09-11T00:04:01.816896+0000","last_became_peered":"2024-09-11T00:04:01.816896+0000","last_unstale":"2024-09-11T00:04:50.254721+0000","last_undegraded":"2024-09-11T00:04:50.254721+0000","last_fullsized":"2024-09-11T00:04:50.254721+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_clean_scrub_stamp":"2024-09-11T00:04:00.794959+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:46:51.641881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'92","reported_seq":105,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T00:04:15.912257+0000","last_change":"2024-09-11T00:02:47.245193+0000","last_active":"2024-09-11T00:04:15.912257+0000","last_peered":"2024-09-11T00:04:15.912257+0000","last_clean":"2024-09-11T00:04:15.912257+0000","last_became_active":"2024-09-11T00:02:47.239773+0000","last_became_peered":"2024-09-11T00:02:47.239773+0000","last_unstale":"2024-09-11T00:04:15.912257+0000","last_undegraded":"2024-09-11T00:04:15.912257+0000","last_fullsized":"2024-09-11T00:04:15.912257+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:59:10.637655+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:59:10.637655+0000","last_clean_scrub_stamp":"2024-09-10T23:59:10.637655+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:51:52.914831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"81'1","reported_seq":22,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T00:04:15.912653+0000","last_change":"2024-09-11T00:04:11.867764+0000","last_active":"2024-09-11T00:04:15.912653+0000","last_peered":"2024-09-11T00:04:15.912653+0000","last_clean":"2024-09-11T00:04:15.912653+0000","last_became_active":"2024-09-11T00:04:11.867210+0000","last_became_peered":"2024-09-11T00:04:11.867210+0000","last_unstale":"2024-09-11T00:04:15.912653+0000","last_undegraded":"2024-09-11T00:04:15.912653+0000","last_fullsized":"2024-09-11T00:04:15.912653+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_clean_scrub_stamp":"2024-09-11T00:04:10.845136+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:04:15.439956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T00:04:15.912653+0000","last_change":"2024-09-11T00:04:03.834755+0000","last_active":"2024-09-11T00:04:15.912653+0000","last_peered":"2024-09-11T00:04:15.912653+0000","last_clean":"2024-09-11T00:04:15.912653+0000","last_became_active":"2024-09-11T00:04:03.833955+0000","last_became_peered":"2024-09-11T00:04:03.833955+0000","last_unstale":"2024-09-11T00:04:15.912653+0000","last_undegraded":"2024-09-11T00:04:15.912653+0000","last_fullsized":"2024-09-11T00:04:15.912653+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:47:13.863481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"81'6","reported_seq":57,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T00:05:03.277968+0000","last_change":"2024-09-11T00:04:02.589042+0000","last_active":"2024-09-11T00:05:03.277968+0000","last_peered":"2024-09-11T00:05:03.277968+0000","last_clean":"2024-09-11T00:05:03.277968+0000","last_became_active":"2024-09-11T00:04:02.588611+0000","last_became_peered":"2024-09-11T00:04:02.588611+0000","last_unstale":"2024-09-11T00:05:03.277968+0000","last_undegraded":"2024-09-11T00:05:03.277968+0000","last_fullsized":"2024-09-11T00:05:03.277968+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_clean_scrub_stamp":"2024-09-11T00:04:00.794959+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:41:08.257605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1561,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":18,"num_read_kb":3,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":21,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T00:04:17.435605+0000","last_change":"2024-09-11T00:04:11.860715+0000","last_active":"2024-09-11T00:04:17.435605+0000","last_peered":"2024-09-11T00:04:17.435605+0000","last_clean":"2024-09-11T00:04:17.435605+0000","last_became_active":"2024-09-11T00:04:11.860094+0000","last_became_peered":"2024-09-11T00:04:11.860094+0000","last_unstale":"2024-09-11T00:04:17.435605+0000","last_undegraded":"2024-09-11T00:04:17.435605+0000","last_fullsized":"2024-09-11T00:04:17.435605+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_clean_scrub_stamp":"2024-09-11T00:04:10.845136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:25:55.279079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T00:04:15.912543+0000","last_change":"2024-09-11T00:04:03.822665+0000","last_active":"2024-09-11T00:04:15.912543+0000","last_peered":"2024-09-11T00:04:15.912543+0000","last_clean":"2024-09-11T00:04:15.912543+0000","last_became_active":"2024-09-11T00:04:03.822333+0000","last_became_peered":"2024-09-11T00:04:03.822333+0000","last_unstale":"2024-09-11T00:04:15.912543+0000","last_undegraded":"2024-09-11T00:04:15.912543+0000","last_fullsized":"2024-09-11T00:04:15.912543+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:02:07.465335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"81'3","reported_seq":37,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T00:04:49.248660+0000","last_change":"2024-09-11T00:04:01.817244+0000","last_active":"2024-09-11T00:04:49.248660+0000","last_peered":"2024-09-11T00:04:49.248660+0000","last_clean":"2024-09-11T00:04:49.248660+0000","last_became_active":"2024-09-11T00:04:01.816846+0000","last_became_peered":"2024-09-11T00:04:01.816846+0000","last_unstale":"2024-09-11T00:04:49.248660+0000","last_undegraded":"2024-09-11T00:04:49.248660+0000","last_fullsized":"2024-09-11T00:04:49.248660+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_clean_scrub_stamp":"2024-09-11T00:04:00.794959+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:09:46.232332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":21,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T00:04:17.305061+0000","last_change":"2024-09-11T00:04:11.872439+0000","last_active":"2024-09-11T00:04:17.305061+0000","last_peered":"2024-09-11T00:04:17.305061+0000","last_clean":"2024-09-11T00:04:17.305061+0000","last_became_active":"2024-09-11T00:04:11.872064+0000","last_became_peered":"2024-09-11T00:04:11.872064+0000","last_unstale":"2024-09-11T00:04:17.305061+0000","last_undegraded":"2024-09-11T00:04:17.305061+0000","last_fullsized":"2024-09-11T00:04:17.305061+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_clean_scrub_stamp":"2024-09-11T00:04:10.845136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:15:47.902247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T00:04:15.912593+0000","last_change":"2024-09-11T00:04:03.829600+0000","last_active":"2024-09-11T00:04:15.912593+0000","last_peered":"2024-09-11T00:04:15.912593+0000","last_clean":"2024-09-11T00:04:15.912593+0000","last_became_active":"2024-09-11T00:04:03.829118+0000","last_became_peered":"2024-09-11T00:04:03.829118+0000","last_unstale":"2024-09-11T00:04:15.912593+0000","last_undegraded":"2024-09-11T00:04:15.912593+0000","last_fullsized":"2024-09-11T00:04:15.912593+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:41:07.069816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"81'5","reported_seq":38,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T00:04:49.250507+0000","last_change":"2024-09-11T00:04:01.820205+0000","last_active":"2024-09-11T00:04:49.250507+0000","last_peered":"2024-09-11T00:04:49.250507+0000","last_clean":"2024-09-11T00:04:49.250507+0000","last_became_active":"2024-09-11T00:04:01.819889+0000","last_became_peered":"2024-09-11T00:04:01.819889+0000","last_unstale":"2024-09-11T00:04:49.250507+0000","last_undegraded":"2024-09-11T00:04:49.250507+0000","last_fullsized":"2024-09-11T00:04:49.250507+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_clean_scrub_stamp":"2024-09-11T00:04:00.794959+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:05:05.259474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":21,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T00:04:17.435423+0000","last_change":"2024-09-11T00:04:11.863812+0000","last_active":"2024-09-11T00:04:17.435423+0000","last_peered":"2024-09-11T00:04:17.435423+0000","last_clean":"2024-09-11T00:04:17.435423+0000","last_became_active":"2024-09-11T00:04:11.863412+0000","last_became_peered":"2024-09-11T00:04:11.863412+0000","last_unstale":"2024-09-11T00:04:17.435423+0000","last_undegraded":"2024-09-11T00:04:17.435423+0000","last_fullsized":"2024-09-11T00:04:17.435423+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_clean_scrub_stamp":"2024-09-11T00:04:10.845136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:12:04.140034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T00:04:15.869604+0000","last_change":"2024-09-11T00:04:03.831235+0000","last_active":"2024-09-11T00:04:15.869604+0000","last_peered":"2024-09-11T00:04:15.869604+0000","last_clean":"2024-09-11T00:04:15.869604+0000","last_became_active":"2024-09-11T00:04:03.830054+0000","last_became_peered":"2024-09-11T00:04:03.830054+0000","last_unstale":"2024-09-11T00:04:15.869604+0000","last_undegraded":"2024-09-11T00:04:15.869604+0000","last_fullsized":"2024-09-11T00:04:15.869604+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:28:55.997099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"81'2","reported_seq":36,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T00:04:48.232268+0000","last_change":"2024-09-11T00:04:01.814191+0000","last_active":"2024-09-11T00:04:48.232268+0000","last_peered":"2024-09-11T00:04:48.232268+0000","last_clean":"2024-09-11T00:04:48.232268+0000","last_became_active":"2024-09-11T00:04:01.813778+0000","last_became_peered":"2024-09-11T00:04:01.813778+0000","last_unstale":"2024-09-11T00:04:48.232268+0000","last_undegraded":"2024-09-11T00:04:48.232268+0000","last_fullsized":"2024-09-11T00:04:48.232268+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_clean_scrub_stamp":"2024-09-11T00:04:00.794959+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:01:49.044436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":21,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T00:04:17.435473+0000","last_change":"2024-09-11T00:04:11.870281+0000","last_active":"2024-09-11T00:04:17.435473+0000","last_peered":"2024-09-11T00:04:17.435473+0000","last_clean":"2024-09-11T00:04:17.435473+0000","last_became_active":"2024-09-11T00:04:11.869738+0000","last_became_peered":"2024-09-11T00:04:11.869738+0000","last_unstale":"2024-09-11T00:04:17.435473+0000","last_undegraded":"2024-09-11T00:04:17.435473+0000","last_fullsized":"2024-09-11T00:04:17.435473+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_clean_scrub_stamp":"2024-09-11T00:04:10.845136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:04:08.380199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T00:04:17.435646+0000","last_change":"2024-09-11T00:04:03.835240+0000","last_active":"2024-09-11T00:04:17.435646+0000","last_peered":"2024-09-11T00:04:17.435646+0000","last_clean":"2024-09-11T00:04:17.435646+0000","last_became_active":"2024-09-11T00:04:03.834910+0000","last_became_peered":"2024-09-11T00:04:03.834910+0000","last_unstale":"2024-09-11T00:04:17.435646+0000","last_undegraded":"2024-09-11T00:04:17.435646+0000","last_fullsized":"2024-09-11T00:04:17.435646+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:30:33.996409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.10","version":"81'17","reported_seq":51,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T00:04:49.249166+0000","last_change":"2024-09-11T00:04:01.941860+0000","last_active":"2024-09-11T00:04:49.249166+0000","last_peered":"2024-09-11T00:04:49.249166+0000","last_clean":"2024-09-11T00:04:49.249166+0000","last_became_active":"2024-09-11T00:04:01.941484+0000","last_became_peered":"2024-09-11T00:04:01.941484+0000","last_unstale":"2024-09-11T00:04:49.249166+0000","last_undegraded":"2024-09-11T00:04:49.249166+0000","last_fullsized":"2024-09-11T00:04:49.249166+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_clean_scrub_stamp":"2024-09-11T00:04:00.794959+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:22:16.610237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":21,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T00:04:15.869896+0000","last_change":"2024-09-11T00:04:11.874306+0000","last_active":"2024-09-11T00:04:15.869896+0000","last_peered":"2024-09-11T00:04:15.869896+0000","last_clean":"2024-09-11T00:04:15.869896+0000","last_became_active":"2024-09-11T00:04:11.873934+0000","last_became_peered":"2024-09-11T00:04:11.873934+0000","last_unstale":"2024-09-11T00:04:15.869896+0000","last_undegraded":"2024-09-11T00:04:15.869896+0000","last_fullsized":"2024-09-11T00:04:15.869896+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_clean_scrub_stamp":"2024-09-11T00:04:10.845136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:19:22.584367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T00:04:15.912016+0000","last_change":"2024-09-11T00:04:03.824219+0000","last_active":"2024-09-11T00:04:15.912016+0000","last_peered":"2024-09-11T00:04:15.912016+0000","last_clean":"2024-09-11T00:04:15.912016+0000","last_became_active":"2024-09-11T00:04:03.823877+0000","last_became_peered":"2024-09-11T00:04:03.823877+0000","last_unstale":"2024-09-11T00:04:15.912016+0000","last_undegraded":"2024-09-11T00:04:15.912016+0000","last_fullsized":"2024-09-11T00:04:15.912016+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:04:05.924113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"81'5","reported_seq":55,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T00:05:03.277287+0000","last_change":"2024-09-11T00:04:01.818055+0000","last_active":"2024-09-11T00:05:03.277287+0000","last_peered":"2024-09-11T00:05:03.277287+0000","last_clean":"2024-09-11T00:05:03.277287+0000","last_became_active":"2024-09-11T00:04:01.817483+0000","last_became_peered":"2024-09-11T00:04:01.817483+0000","last_unstale":"2024-09-11T00:05:03.277287+0000","last_undegraded":"2024-09-11T00:05:03.277287+0000","last_fullsized":"2024-09-11T00:05:03.277287+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_clean_scrub_stamp":"2024-09-11T00:04:00.794959+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:16:55.218673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":18,"num_read_kb":3,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":21,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T00:04:17.435840+0000","last_change":"2024-09-11T00:04:11.875918+0000","last_active":"2024-09-11T00:04:17.435840+0000","last_peered":"2024-09-11T00:04:17.435840+0000","last_clean":"2024-09-11T00:04:17.435840+0000","last_became_active":"2024-09-11T00:04:11.875549+0000","last_became_peered":"2024-09-11T00:04:11.875549+0000","last_unstale":"2024-09-11T00:04:17.435840+0000","last_undegraded":"2024-09-11T00:04:17.435840+0000","last_fullsized":"2024-09-11T00:04:17.435840+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_clean_scrub_stamp":"2024-09-11T00:04:10.845136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:16:48.685899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T00:04:15.912429+0000","last_change":"2024-09-11T00:04:03.827671+0000","last_active":"2024-09-11T00:04:15.912429+0000","last_peered":"2024-09-11T00:04:15.912429+0000","last_clean":"2024-09-11T00:04:15.912429+0000","last_became_active":"2024-09-11T00:04:03.827341+0000","last_became_peered":"2024-09-11T00:04:03.827341+0000","last_unstale":"2024-09-11T00:04:15.912429+0000","last_undegraded":"2024-09-11T00:04:15.912429+0000","last_fullsized":"2024-09-11T00:04:15.912429+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:56:16.614035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"81'5","reported_seq":39,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T00:04:49.249302+0000","last_change":"2024-09-11T00:04:01.820632+0000","last_active":"2024-09-11T00:04:49.249302+0000","last_peered":"2024-09-11T00:04:49.249302+0000","last_clean":"2024-09-11T00:04:49.249302+0000","last_became_active":"2024-09-11T00:04:01.820273+0000","last_became_peered":"2024-09-11T00:04:01.820273+0000","last_unstale":"2024-09-11T00:04:49.249302+0000","last_undegraded":"2024-09-11T00:04:49.249302+0000","last_fullsized":"2024-09-11T00:04:49.249302+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_clean_scrub_stamp":"2024-09-11T00:04:00.794959+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:34:53.444515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":21,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T00:04:15.869485+0000","last_change":"2024-09-11T00:04:11.881197+0000","last_active":"2024-09-11T00:04:15.869485+0000","last_peered":"2024-09-11T00:04:15.869485+0000","last_clean":"2024-09-11T00:04:15.869485+0000","last_became_active":"2024-09-11T00:04:11.880763+0000","last_became_peered":"2024-09-11T00:04:11.880763+0000","last_unstale":"2024-09-11T00:04:15.869485+0000","last_undegraded":"2024-09-11T00:04:15.869485+0000","last_fullsized":"2024-09-11T00:04:15.869485+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_clean_scrub_stamp":"2024-09-11T00:04:10.845136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:14:01.616095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T00:04:15.869315+0000","last_change":"2024-09-11T00:04:03.833470+0000","last_active":"2024-09-11T00:04:15.869315+0000","last_peered":"2024-09-11T00:04:15.869315+0000","last_clean":"2024-09-11T00:04:15.869315+0000","last_became_active":"2024-09-11T00:04:03.833120+0000","last_became_peered":"2024-09-11T00:04:03.833120+0000","last_unstale":"2024-09-11T00:04:15.869315+0000","last_undegraded":"2024-09-11T00:04:15.869315+0000","last_fullsized":"2024-09-11T00:04:15.869315+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:29:53.210230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"81'9","reported_seq":44,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T00:04:50.254977+0000","last_change":"2024-09-11T00:04:01.818081+0000","last_active":"2024-09-11T00:04:50.254977+0000","last_peered":"2024-09-11T00:04:50.254977+0000","last_clean":"2024-09-11T00:04:50.254977+0000","last_became_active":"2024-09-11T00:04:01.817716+0000","last_became_peered":"2024-09-11T00:04:01.817716+0000","last_unstale":"2024-09-11T00:04:50.254977+0000","last_undegraded":"2024-09-11T00:04:50.254977+0000","last_fullsized":"2024-09-11T00:04:50.254977+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_clean_scrub_stamp":"2024-09-11T00:04:00.794959+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:01:32.736319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":21,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T00:04:17.436755+0000","last_change":"2024-09-11T00:04:11.870182+0000","last_active":"2024-09-11T00:04:17.436755+0000","last_peered":"2024-09-11T00:04:17.436755+0000","last_clean":"2024-09-11T00:04:17.436755+0000","last_became_active":"2024-09-11T00:04:11.869744+0000","last_became_peered":"2024-09-11T00:04:11.869744+0000","last_unstale":"2024-09-11T00:04:17.436755+0000","last_undegraded":"2024-09-11T00:04:17.436755+0000","last_fullsized":"2024-09-11T00:04:17.436755+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_clean_scrub_stamp":"2024-09-11T00:04:10.845136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:50:24.070404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T00:04:17.436191+0000","last_change":"2024-09-11T00:04:03.828246+0000","last_active":"2024-09-11T00:04:17.436191+0000","last_peered":"2024-09-11T00:04:17.436191+0000","last_clean":"2024-09-11T00:04:17.436191+0000","last_became_active":"2024-09-11T00:04:03.827217+0000","last_became_peered":"2024-09-11T00:04:03.827217+0000","last_unstale":"2024-09-11T00:04:17.436191+0000","last_undegraded":"2024-09-11T00:04:17.436191+0000","last_fullsized":"2024-09-11T00:04:17.436191+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:51:46.018759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"81'31","reported_seq":103,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T00:05:05.280676+0000","last_change":"2024-09-11T00:04:01.819553+0000","last_active":"2024-09-11T00:05:05.280676+0000","last_peered":"2024-09-11T00:05:05.280676+0000","last_clean":"2024-09-11T00:05:05.280676+0000","last_became_active":"2024-09-11T00:04:01.819213+0000","last_became_peered":"2024-09-11T00:04:01.819213+0000","last_unstale":"2024-09-11T00:05:05.280676+0000","last_undegraded":"2024-09-11T00:05:05.280676+0000","last_fullsized":"2024-09-11T00:05:05.280676+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_clean_scrub_stamp":"2024-09-11T00:04:00.794959+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:44:11.039982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":56160,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":39,"num_read_kb":74,"num_write":33,"num_write_kb":67,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":21,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T00:04:17.435474+0000","last_change":"2024-09-11T00:04:11.871730+0000","last_active":"2024-09-11T00:04:17.435474+0000","last_peered":"2024-09-11T00:04:17.435474+0000","last_clean":"2024-09-11T00:04:17.435474+0000","last_became_active":"2024-09-11T00:04:11.870659+0000","last_became_peered":"2024-09-11T00:04:11.870659+0000","last_unstale":"2024-09-11T00:04:17.435474+0000","last_undegraded":"2024-09-11T00:04:17.435474+0000","last_fullsized":"2024-09-11T00:04:17.435474+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_clean_scrub_stamp":"2024-09-11T00:04:10.845136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:48:41.835602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T00:04:17.435768+0000","last_change":"2024-09-11T00:04:03.838763+0000","last_active":"2024-09-11T00:04:17.435768+0000","last_peered":"2024-09-11T00:04:17.435768+0000","last_clean":"2024-09-11T00:04:17.435768+0000","last_became_active":"2024-09-11T00:04:03.838409+0000","last_became_peered":"2024-09-11T00:04:03.838409+0000","last_unstale":"2024-09-11T00:04:17.435768+0000","last_undegraded":"2024-09-11T00:04:17.435768+0000","last_fullsized":"2024-09-11T00:04:17.435768+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:57:38.647351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"81'5","reported_seq":39,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T00:04:48.243571+0000","last_change":"2024-09-11T00:04:01.943027+0000","last_active":"2024-09-11T00:04:48.243571+0000","last_peered":"2024-09-11T00:04:48.243571+0000","last_clean":"2024-09-11T00:04:48.243571+0000","last_became_active":"2024-09-11T00:04:01.942558+0000","last_became_peered":"2024-09-11T00:04:01.942558+0000","last_unstale":"2024-09-11T00:04:48.243571+0000","last_undegraded":"2024-09-11T00:04:48.243571+0000","last_fullsized":"2024-09-11T00:04:48.243571+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_clean_scrub_stamp":"2024-09-11T00:04:00.794959+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:48:03.776476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":21,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T00:04:17.436777+0000","last_change":"2024-09-11T00:04:11.868069+0000","last_active":"2024-09-11T00:04:17.436777+0000","last_peered":"2024-09-11T00:04:17.436777+0000","last_clean":"2024-09-11T00:04:17.436777+0000","last_became_active":"2024-09-11T00:04:11.867544+0000","last_became_peered":"2024-09-11T00:04:11.867544+0000","last_unstale":"2024-09-11T00:04:17.436777+0000","last_undegraded":"2024-09-11T00:04:17.436777+0000","last_fullsized":"2024-09-11T00:04:17.436777+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_clean_scrub_stamp":"2024-09-11T00:04:10.845136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:09:20.455807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T00:04:17.305180+0000","last_change":"2024-09-11T00:04:03.836735+0000","last_active":"2024-09-11T00:04:17.305180+0000","last_peered":"2024-09-11T00:04:17.305180+0000","last_clean":"2024-09-11T00:04:17.305180+0000","last_became_active":"2024-09-11T00:04:03.833299+0000","last_became_peered":"2024-09-11T00:04:03.833299+0000","last_unstale":"2024-09-11T00:04:17.305180+0000","last_undegraded":"2024-09-11T00:04:17.305180+0000","last_fullsized":"2024-09-11T00:04:17.305180+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:32:15.924058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"81'4","reported_seq":37,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T00:04:49.249301+0000","last_change":"2024-09-11T00:04:01.823777+0000","last_active":"2024-09-11T00:04:49.249301+0000","last_peered":"2024-09-11T00:04:49.249301+0000","last_clean":"2024-09-11T00:04:49.249301+0000","last_became_active":"2024-09-11T00:04:01.823368+0000","last_became_peered":"2024-09-11T00:04:01.823368+0000","last_unstale":"2024-09-11T00:04:49.249301+0000","last_undegraded":"2024-09-11T00:04:49.249301+0000","last_fullsized":"2024-09-11T00:04:49.249301+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_clean_scrub_stamp":"2024-09-11T00:04:00.794959+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:05:12.584867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":21,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T00:04:17.435215+0000","last_change":"2024-09-11T00:04:11.868303+0000","last_active":"2024-09-11T00:04:17.435215+0000","last_peered":"2024-09-11T00:04:17.435215+0000","last_clean":"2024-09-11T00:04:17.435215+0000","last_became_active":"2024-09-11T00:04:11.867916+0000","last_became_peered":"2024-09-11T00:04:11.867916+0000","last_unstale":"2024-09-11T00:04:17.435215+0000","last_undegraded":"2024-09-11T00:04:17.435215+0000","last_fullsized":"2024-09-11T00:04:17.435215+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_clean_scrub_stamp":"2024-09-11T00:04:10.845136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:08:29.711763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T00:04:17.436694+0000","last_change":"2024-09-11T00:04:03.832712+0000","last_active":"2024-09-11T00:04:17.436694+0000","last_peered":"2024-09-11T00:04:17.436694+0000","last_clean":"2024-09-11T00:04:17.436694+0000","last_became_active":"2024-09-11T00:04:03.832295+0000","last_became_peered":"2024-09-11T00:04:03.832295+0000","last_unstale":"2024-09-11T00:04:17.436694+0000","last_undegraded":"2024-09-11T00:04:17.436694+0000","last_fullsized":"2024-09-11T00:04:17.436694+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:40:12.271930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"81'4","reported_seq":69,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T00:05:05.280670+0000","last_change":"2024-09-11T00:04:01.820891+0000","last_active":"2024-09-11T00:05:05.280670+0000","last_peered":"2024-09-11T00:05:05.280670+0000","last_clean":"2024-09-11T00:05:05.280670+0000","last_became_active":"2024-09-11T00:04:01.820528+0000","last_became_peered":"2024-09-11T00:04:01.820528+0000","last_unstale":"2024-09-11T00:05:05.280670+0000","last_undegraded":"2024-09-11T00:05:05.280670+0000","last_fullsized":"2024-09-11T00:05:05.280670+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_clean_scrub_stamp":"2024-09-11T00:04:00.794959+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:23:38.196618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":32,"num_read_kb":32,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":21,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T00:04:16.752010+0000","last_change":"2024-09-11T00:04:11.878864+0000","last_active":"2024-09-11T00:04:16.752010+0000","last_peered":"2024-09-11T00:04:16.752010+0000","last_clean":"2024-09-11T00:04:16.752010+0000","last_became_active":"2024-09-11T00:04:11.876701+0000","last_became_peered":"2024-09-11T00:04:11.876701+0000","last_unstale":"2024-09-11T00:04:16.752010+0000","last_undegraded":"2024-09-11T00:04:16.752010+0000","last_fullsized":"2024-09-11T00:04:16.752010+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_clean_scrub_stamp":"2024-09-11T00:04:10.845136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:26:05.650618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":21,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T00:04:17.304780+0000","last_change":"2024-09-11T00:04:11.868140+0000","last_active":"2024-09-11T00:04:17.304780+0000","last_peered":"2024-09-11T00:04:17.304780+0000","last_clean":"2024-09-11T00:04:17.304780+0000","last_became_active":"2024-09-11T00:04:11.866459+0000","last_became_peered":"2024-09-11T00:04:11.866459+0000","last_unstale":"2024-09-11T00:04:17.304780+0000","last_undegraded":"2024-09-11T00:04:17.304780+0000","last_fullsized":"2024-09-11T00:04:17.304780+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_clean_scrub_stamp":"2024-09-11T00:04:10.845136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:53:25.106039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T00:04:17.435283+0000","last_change":"2024-09-11T00:04:03.829839+0000","last_active":"2024-09-11T00:04:17.435283+0000","last_peered":"2024-09-11T00:04:17.435283+0000","last_clean":"2024-09-11T00:04:17.435283+0000","last_became_active":"2024-09-11T00:04:03.829553+0000","last_became_peered":"2024-09-11T00:04:03.829553+0000","last_unstale":"2024-09-11T00:04:17.435283+0000","last_undegraded":"2024-09-11T00:04:17.435283+0000","last_fullsized":"2024-09-11T00:04:17.435283+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:45:58.561389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"81'3","reported_seq":37,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T00:04:49.249068+0000","last_change":"2024-09-11T00:04:01.820204+0000","last_active":"2024-09-11T00:04:49.249068+0000","last_peered":"2024-09-11T00:04:49.249068+0000","last_clean":"2024-09-11T00:04:49.249068+0000","last_became_active":"2024-09-11T00:04:01.819797+0000","last_became_peered":"2024-09-11T00:04:01.819797+0000","last_unstale":"2024-09-11T00:04:49.249068+0000","last_undegraded":"2024-09-11T00:04:49.249068+0000","last_fullsized":"2024-09-11T00:04:49.249068+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_clean_scrub_stamp":"2024-09-11T00:04:00.794959+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:56:10.530023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":21,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T00:04:15.869760+0000","last_change":"2024-09-11T00:04:11.879430+0000","last_active":"2024-09-11T00:04:15.869760+0000","last_peered":"2024-09-11T00:04:15.869760+0000","last_clean":"2024-09-11T00:04:15.869760+0000","last_became_active":"2024-09-11T00:04:11.879010+0000","last_became_peered":"2024-09-11T00:04:11.879010+0000","last_unstale":"2024-09-11T00:04:15.869760+0000","last_undegraded":"2024-09-11T00:04:15.869760+0000","last_fullsized":"2024-09-11T00:04:15.869760+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_clean_scrub_stamp":"2024-09-11T00:04:10.845136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:57.737125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T00:04:16.751460+0000","last_change":"2024-09-11T00:04:03.843918+0000","last_active":"2024-09-11T00:04:16.751460+0000","last_peered":"2024-09-11T00:04:16.751460+0000","last_clean":"2024-09-11T00:04:16.751460+0000","last_became_active":"2024-09-11T00:04:03.843591+0000","last_became_peered":"2024-09-11T00:04:03.843591+0000","last_unstale":"2024-09-11T00:04:16.751460+0000","last_undegraded":"2024-09-11T00:04:16.751460+0000","last_fullsized":"2024-09-11T00:04:16.751460+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:26:41.228802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"81'3","reported_seq":36,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T00:04:47.239179+0000","last_change":"2024-09-11T00:04:01.813173+0000","last_active":"2024-09-11T00:04:47.239179+0000","last_peered":"2024-09-11T00:04:47.239179+0000","last_clean":"2024-09-11T00:04:47.239179+0000","last_became_active":"2024-09-11T00:04:01.811799+0000","last_became_peered":"2024-09-11T00:04:01.811799+0000","last_unstale":"2024-09-11T00:04:47.239179+0000","last_undegraded":"2024-09-11T00:04:47.239179+0000","last_fullsized":"2024-09-11T00:04:47.239179+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_clean_scrub_stamp":"2024-09-11T00:04:00.794959+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:16:47.304371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"81'7","reported_seq":40,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T00:04:48.242866+0000","last_change":"2024-09-11T00:04:01.818649+0000","last_active":"2024-09-11T00:04:48.242866+0000","last_peered":"2024-09-11T00:04:48.242866+0000","last_clean":"2024-09-11T00:04:48.242866+0000","last_became_active":"2024-09-11T00:04:01.818287+0000","last_became_peered":"2024-09-11T00:04:01.818287+0000","last_unstale":"2024-09-11T00:04:48.242866+0000","last_undegraded":"2024-09-11T00:04:48.242866+0000","last_fullsized":"2024-09-11T00:04:48.242866+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_clean_scrub_stamp":"2024-09-11T00:04:00.794959+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:42:00.764648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T00:04:17.435994+0000","last_change":"2024-09-11T00:04:03.832829+0000","last_active":"2024-09-11T00:04:17.435994+0000","last_peered":"2024-09-11T00:04:17.435994+0000","last_clean":"2024-09-11T00:04:17.435994+0000","last_became_active":"2024-09-11T00:04:03.832514+0000","last_became_peered":"2024-09-11T00:04:03.832514+0000","last_unstale":"2024-09-11T00:04:17.435994+0000","last_undegraded":"2024-09-11T00:04:17.435994+0000","last_fullsized":"2024-09-11T00:04:17.435994+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:36.643727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":21,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T00:04:16.882276+0000","last_change":"2024-09-11T00:04:11.877596+0000","last_active":"2024-09-11T00:04:16.882276+0000","last_peered":"2024-09-11T00:04:16.882276+0000","last_clean":"2024-09-11T00:04:16.882276+0000","last_became_active":"2024-09-11T00:04:11.877113+0000","last_became_peered":"2024-09-11T00:04:11.877113+0000","last_unstale":"2024-09-11T00:04:16.882276+0000","last_undegraded":"2024-09-11T00:04:16.882276+0000","last_fullsized":"2024-09-11T00:04:16.882276+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_clean_scrub_stamp":"2024-09-11T00:04:10.845136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:47:11.759687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"81'5","reported_seq":56,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T00:05:04.279784+0000","last_change":"2024-09-11T00:04:01.821519+0000","last_active":"2024-09-11T00:05:04.279784+0000","last_peered":"2024-09-11T00:05:04.279784+0000","last_clean":"2024-09-11T00:05:04.279784+0000","last_became_active":"2024-09-11T00:04:01.821207+0000","last_became_peered":"2024-09-11T00:04:01.821207+0000","last_unstale":"2024-09-11T00:05:04.279784+0000","last_undegraded":"2024-09-11T00:05:04.279784+0000","last_fullsized":"2024-09-11T00:05:04.279784+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_clean_scrub_stamp":"2024-09-11T00:04:00.794959+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:16:34.435040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":18,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T00:04:17.435813+0000","last_change":"2024-09-11T00:04:03.832691+0000","last_active":"2024-09-11T00:04:17.435813+0000","last_peered":"2024-09-11T00:04:17.435813+0000","last_clean":"2024-09-11T00:04:17.435813+0000","last_became_active":"2024-09-11T00:04:03.832250+0000","last_became_peered":"2024-09-11T00:04:03.832250+0000","last_unstale":"2024-09-11T00:04:17.435813+0000","last_undegraded":"2024-09-11T00:04:17.435813+0000","last_fullsized":"2024-09-11T00:04:17.435813+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:54.279820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":21,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T00:04:16.882547+0000","last_change":"2024-09-11T00:04:11.864995+0000","last_active":"2024-09-11T00:04:16.882547+0000","last_peered":"2024-09-11T00:04:16.882547+0000","last_clean":"2024-09-11T00:04:16.882547+0000","last_became_active":"2024-09-11T00:04:11.864501+0000","last_became_peered":"2024-09-11T00:04:11.864501+0000","last_unstale":"2024-09-11T00:04:16.882547+0000","last_undegraded":"2024-09-11T00:04:16.882547+0000","last_fullsized":"2024-09-11T00:04:16.882547+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_clean_scrub_stamp":"2024-09-11T00:04:10.845136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:08:08.637714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T00:04:15.869280+0000","last_change":"2024-09-11T00:04:03.829310+0000","last_active":"2024-09-11T00:04:15.869280+0000","last_peered":"2024-09-11T00:04:15.869280+0000","last_clean":"2024-09-11T00:04:15.869280+0000","last_became_active":"2024-09-11T00:04:03.828937+0000","last_became_peered":"2024-09-11T00:04:03.828937+0000","last_unstale":"2024-09-11T00:04:15.869280+0000","last_undegraded":"2024-09-11T00:04:15.869280+0000","last_fullsized":"2024-09-11T00:04:15.869280+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:32:58.263067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T00:04:16.882379+0000","last_change":"2024-09-11T00:04:03.828055+0000","last_active":"2024-09-11T00:04:16.882379+0000","last_peered":"2024-09-11T00:04:16.882379+0000","last_clean":"2024-09-11T00:04:16.882379+0000","last_became_active":"2024-09-11T00:04:03.827382+0000","last_became_peered":"2024-09-11T00:04:03.827382+0000","last_unstale":"2024-09-11T00:04:16.882379+0000","last_undegraded":"2024-09-11T00:04:16.882379+0000","last_fullsized":"2024-09-11T00:04:16.882379+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:02:47.107689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T00:04:16.882015+0000","last_change":"2024-09-11T00:04:03.843549+0000","last_active":"2024-09-11T00:04:16.882015+0000","last_peered":"2024-09-11T00:04:16.882015+0000","last_clean":"2024-09-11T00:04:16.882015+0000","last_became_active":"2024-09-11T00:04:03.843149+0000","last_became_peered":"2024-09-11T00:04:03.843149+0000","last_unstale":"2024-09-11T00:04:16.882015+0000","last_undegraded":"2024-09-11T00:04:16.882015+0000","last_fullsized":"2024-09-11T00:04:16.882015+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:47.288319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T00:04:17.304581+0000","last_change":"2024-09-11T00:04:03.838631+0000","last_active":"2024-09-11T00:04:17.304581+0000","last_peered":"2024-09-11T00:04:17.304581+0000","last_clean":"2024-09-11T00:04:17.304581+0000","last_became_active":"2024-09-11T00:04:03.836382+0000","last_became_peered":"2024-09-11T00:04:03.836382+0000","last_unstale":"2024-09-11T00:04:17.304581+0000","last_undegraded":"2024-09-11T00:04:17.304581+0000","last_fullsized":"2024-09-11T00:04:17.304581+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:36:20.341469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T00:04:15.869636+0000","last_change":"2024-09-11T00:04:03.835068+0000","last_active":"2024-09-11T00:04:15.869636+0000","last_peered":"2024-09-11T00:04:15.869636+0000","last_clean":"2024-09-11T00:04:15.869636+0000","last_became_active":"2024-09-11T00:04:03.833166+0000","last_became_peered":"2024-09-11T00:04:03.833166+0000","last_unstale":"2024-09-11T00:04:15.869636+0000","last_undegraded":"2024-09-11T00:04:15.869636+0000","last_fullsized":"2024-09-11T00:04:15.869636+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:07:06.357597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T00:04:15.869678+0000","last_change":"2024-09-11T00:04:03.827310+0000","last_active":"2024-09-11T00:04:15.869678+0000","last_peered":"2024-09-11T00:04:15.869678+0000","last_clean":"2024-09-11T00:04:15.869678+0000","last_became_active":"2024-09-11T00:04:03.826748+0000","last_became_peered":"2024-09-11T00:04:03.826748+0000","last_unstale":"2024-09-11T00:04:15.869678+0000","last_undegraded":"2024-09-11T00:04:15.869678+0000","last_fullsized":"2024-09-11T00:04:15.869678+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:53:15.656708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T00:04:16.882618+0000","last_change":"2024-09-11T00:04:03.835900+0000","last_active":"2024-09-11T00:04:16.882618+0000","last_peered":"2024-09-11T00:04:16.882618+0000","last_clean":"2024-09-11T00:04:16.882618+0000","last_became_active":"2024-09-11T00:04:03.835137+0000","last_became_peered":"2024-09-11T00:04:03.835137+0000","last_unstale":"2024-09-11T00:04:16.882618+0000","last_undegraded":"2024-09-11T00:04:16.882618+0000","last_fullsized":"2024-09-11T00:04:16.882618+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:23:31.184833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T00:04:16.751865+0000","last_change":"2024-09-11T00:04:03.833028+0000","last_active":"2024-09-11T00:04:16.751865+0000","last_peered":"2024-09-11T00:04:16.751865+0000","last_clean":"2024-09-11T00:04:16.751865+0000","last_became_active":"2024-09-11T00:04:03.832703+0000","last_became_peered":"2024-09-11T00:04:03.832703+0000","last_unstale":"2024-09-11T00:04:16.751865+0000","last_undegraded":"2024-09-11T00:04:16.751865+0000","last_fullsized":"2024-09-11T00:04:16.751865+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:38.115627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T00:04:17.435508+0000","last_change":"2024-09-11T00:04:03.832423+0000","last_active":"2024-09-11T00:04:17.435508+0000","last_peered":"2024-09-11T00:04:17.435508+0000","last_clean":"2024-09-11T00:04:17.435508+0000","last_became_active":"2024-09-11T00:04:03.832090+0000","last_became_peered":"2024-09-11T00:04:03.832090+0000","last_unstale":"2024-09-11T00:04:17.435508+0000","last_undegraded":"2024-09-11T00:04:17.435508+0000","last_fullsized":"2024-09-11T00:04:17.435508+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:01:27.258506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T00:04:16.751836+0000","last_change":"2024-09-11T00:04:03.842854+0000","last_active":"2024-09-11T00:04:16.751836+0000","last_peered":"2024-09-11T00:04:16.751836+0000","last_clean":"2024-09-11T00:04:16.751836+0000","last_became_active":"2024-09-11T00:04:03.838666+0000","last_became_peered":"2024-09-11T00:04:03.838666+0000","last_unstale":"2024-09-11T00:04:16.751836+0000","last_undegraded":"2024-09-11T00:04:16.751836+0000","last_fullsized":"2024-09-11T00:04:16.751836+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:45:48.575830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T00:04:17.436008+0000","last_change":"2024-09-11T00:04:03.834175+0000","last_active":"2024-09-11T00:04:17.436008+0000","last_peered":"2024-09-11T00:04:17.436008+0000","last_clean":"2024-09-11T00:04:17.436008+0000","last_became_active":"2024-09-11T00:04:03.833826+0000","last_became_peered":"2024-09-11T00:04:03.833826+0000","last_unstale":"2024-09-11T00:04:17.436008+0000","last_undegraded":"2024-09-11T00:04:17.436008+0000","last_fullsized":"2024-09-11T00:04:17.436008+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:45:57.323216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T00:04:15.869567+0000","last_change":"2024-09-11T00:04:03.836751+0000","last_active":"2024-09-11T00:04:15.869567+0000","last_peered":"2024-09-11T00:04:15.869567+0000","last_clean":"2024-09-11T00:04:15.869567+0000","last_became_active":"2024-09-11T00:04:03.834064+0000","last_became_peered":"2024-09-11T00:04:03.834064+0000","last_unstale":"2024-09-11T00:04:15.869567+0000","last_undegraded":"2024-09-11T00:04:15.869567+0000","last_fullsized":"2024-09-11T00:04:15.869567+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:06:23.059770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T00:04:15.869456+0000","last_change":"2024-09-11T00:04:03.830196+0000","last_active":"2024-09-11T00:04:15.869456+0000","last_peered":"2024-09-11T00:04:15.869456+0000","last_clean":"2024-09-11T00:04:15.869456+0000","last_became_active":"2024-09-11T00:04:03.829856+0000","last_became_peered":"2024-09-11T00:04:03.829856+0000","last_unstale":"2024-09-11T00:04:15.869456+0000","last_undegraded":"2024-09-11T00:04:15.869456+0000","last_fullsized":"2024-09-11T00:04:15.869456+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:34:47.489316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T00:04:16.882542+0000","last_change":"2024-09-11T00:04:03.835053+0000","last_active":"2024-09-11T00:04:16.882542+0000","last_peered":"2024-09-11T00:04:16.882542+0000","last_clean":"2024-09-11T00:04:16.882542+0000","last_became_active":"2024-09-11T00:04:03.834736+0000","last_became_peered":"2024-09-11T00:04:03.834736+0000","last_unstale":"2024-09-11T00:04:16.882542+0000","last_undegraded":"2024-09-11T00:04:16.882542+0000","last_fullsized":"2024-09-11T00:04:16.882542+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:53:40.253775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T00:04:17.435772+0000","last_change":"2024-09-11T00:04:03.834514+0000","last_active":"2024-09-11T00:04:17.435772+0000","last_peered":"2024-09-11T00:04:17.435772+0000","last_clean":"2024-09-11T00:04:17.435772+0000","last_became_active":"2024-09-11T00:04:03.834185+0000","last_became_peered":"2024-09-11T00:04:03.834185+0000","last_unstale":"2024-09-11T00:04:17.435772+0000","last_undegraded":"2024-09-11T00:04:17.435772+0000","last_fullsized":"2024-09-11T00:04:17.435772+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:33:52.949710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T00:04:15.869944+0000","last_change":"2024-09-11T00:04:03.840371+0000","last_active":"2024-09-11T00:04:15.869944+0000","last_peered":"2024-09-11T00:04:15.869944+0000","last_clean":"2024-09-11T00:04:15.869944+0000","last_became_active":"2024-09-11T00:04:03.839933+0000","last_became_peered":"2024-09-11T00:04:03.839933+0000","last_unstale":"2024-09-11T00:04:15.869944+0000","last_undegraded":"2024-09-11T00:04:15.869944+0000","last_fullsized":"2024-09-11T00:04:15.869944+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:24:53.890837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T00:04:17.435348+0000","last_change":"2024-09-11T00:04:03.838386+0000","last_active":"2024-09-11T00:04:17.435348+0000","last_peered":"2024-09-11T00:04:17.435348+0000","last_clean":"2024-09-11T00:04:17.435348+0000","last_became_active":"2024-09-11T00:04:03.837511+0000","last_became_peered":"2024-09-11T00:04:03.837511+0000","last_unstale":"2024-09-11T00:04:17.435348+0000","last_undegraded":"2024-09-11T00:04:17.435348+0000","last_fullsized":"2024-09-11T00:04:17.435348+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:27.399727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T00:04:17.304478+0000","last_change":"2024-09-11T00:04:03.831442+0000","last_active":"2024-09-11T00:04:17.304478+0000","last_peered":"2024-09-11T00:04:17.304478+0000","last_clean":"2024-09-11T00:04:17.304478+0000","last_became_active":"2024-09-11T00:04:03.831115+0000","last_became_peered":"2024-09-11T00:04:03.831115+0000","last_unstale":"2024-09-11T00:04:17.304478+0000","last_undegraded":"2024-09-11T00:04:17.304478+0000","last_fullsized":"2024-09-11T00:04:17.304478+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:53:13.475484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T00:04:17.304492+0000","last_change":"2024-09-11T00:04:03.832618+0000","last_active":"2024-09-11T00:04:17.304492+0000","last_peered":"2024-09-11T00:04:17.304492+0000","last_clean":"2024-09-11T00:04:17.304492+0000","last_became_active":"2024-09-11T00:04:03.831821+0000","last_became_peered":"2024-09-11T00:04:03.831821+0000","last_unstale":"2024-09-11T00:04:17.304492+0000","last_undegraded":"2024-09-11T00:04:17.304492+0000","last_fullsized":"2024-09-11T00:04:17.304492+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:40:40.402422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T00:04:15.869546+0000","last_change":"2024-09-11T00:04:03.825348+0000","last_active":"2024-09-11T00:04:15.869546+0000","last_peered":"2024-09-11T00:04:15.869546+0000","last_clean":"2024-09-11T00:04:15.869546+0000","last_became_active":"2024-09-11T00:04:03.824977+0000","last_became_peered":"2024-09-11T00:04:03.824977+0000","last_unstale":"2024-09-11T00:04:15.869546+0000","last_undegraded":"2024-09-11T00:04:15.869546+0000","last_fullsized":"2024-09-11T00:04:15.869546+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:19:53.412969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T00:04:17.436524+0000","last_change":"2024-09-11T00:04:03.837471+0000","last_active":"2024-09-11T00:04:17.436524+0000","last_peered":"2024-09-11T00:04:17.436524+0000","last_clean":"2024-09-11T00:04:17.436524+0000","last_became_active":"2024-09-11T00:04:03.836651+0000","last_became_peered":"2024-09-11T00:04:03.836651+0000","last_unstale":"2024-09-11T00:04:17.436524+0000","last_undegraded":"2024-09-11T00:04:17.436524+0000","last_fullsized":"2024-09-11T00:04:17.436524+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:34:19.902779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T00:04:15.912587+0000","last_change":"2024-09-11T00:04:03.840068+0000","last_active":"2024-09-11T00:04:15.912587+0000","last_peered":"2024-09-11T00:04:15.912587+0000","last_clean":"2024-09-11T00:04:15.912587+0000","last_became_active":"2024-09-11T00:04:03.835185+0000","last_became_peered":"2024-09-11T00:04:03.835185+0000","last_unstale":"2024-09-11T00:04:15.912587+0000","last_undegraded":"2024-09-11T00:04:15.912587+0000","last_fullsized":"2024-09-11T00:04:15.912587+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:19:10.699038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T00:04:15.870175+0000","last_change":"2024-09-11T00:04:03.828576+0000","last_active":"2024-09-11T00:04:15.870175+0000","last_peered":"2024-09-11T00:04:15.870175+0000","last_clean":"2024-09-11T00:04:15.870175+0000","last_became_active":"2024-09-11T00:04:03.828255+0000","last_became_peered":"2024-09-11T00:04:03.828255+0000","last_unstale":"2024-09-11T00:04:15.870175+0000","last_undegraded":"2024-09-11T00:04:15.870175+0000","last_fullsized":"2024-09-11T00:04:15.870175+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:29:54.164145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T00:04:17.436235+0000","last_change":"2024-09-11T00:04:03.841452+0000","last_active":"2024-09-11T00:04:17.436235+0000","last_peered":"2024-09-11T00:04:17.436235+0000","last_clean":"2024-09-11T00:04:17.436235+0000","last_became_active":"2024-09-11T00:04:03.840911+0000","last_became_peered":"2024-09-11T00:04:03.840911+0000","last_unstale":"2024-09-11T00:04:17.436235+0000","last_undegraded":"2024-09-11T00:04:17.436235+0000","last_fullsized":"2024-09-11T00:04:17.436235+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:56:40.121922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T00:04:16.751450+0000","last_change":"2024-09-11T00:04:03.842009+0000","last_active":"2024-09-11T00:04:16.751450+0000","last_peered":"2024-09-11T00:04:16.751450+0000","last_clean":"2024-09-11T00:04:16.751450+0000","last_became_active":"2024-09-11T00:04:03.841625+0000","last_became_peered":"2024-09-11T00:04:03.841625+0000","last_unstale":"2024-09-11T00:04:16.751450+0000","last_undegraded":"2024-09-11T00:04:16.751450+0000","last_fullsized":"2024-09-11T00:04:16.751450+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:15:11.855308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T00:04:15.912122+0000","last_change":"2024-09-11T00:04:03.829998+0000","last_active":"2024-09-11T00:04:15.912122+0000","last_peered":"2024-09-11T00:04:15.912122+0000","last_clean":"2024-09-11T00:04:15.912122+0000","last_became_active":"2024-09-11T00:04:03.829637+0000","last_became_peered":"2024-09-11T00:04:03.829637+0000","last_unstale":"2024-09-11T00:04:15.912122+0000","last_undegraded":"2024-09-11T00:04:15.912122+0000","last_fullsized":"2024-09-11T00:04:15.912122+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:05:10.144719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T00:04:15.869720+0000","last_change":"2024-09-11T00:04:03.834540+0000","last_active":"2024-09-11T00:04:15.869720+0000","last_peered":"2024-09-11T00:04:15.869720+0000","last_clean":"2024-09-11T00:04:15.869720+0000","last_became_active":"2024-09-11T00:04:03.834230+0000","last_became_peered":"2024-09-11T00:04:03.834230+0000","last_unstale":"2024-09-11T00:04:15.869720+0000","last_undegraded":"2024-09-11T00:04:15.869720+0000","last_fullsized":"2024-09-11T00:04:15.869720+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:14:26.125003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T00:04:17.435560+0000","last_change":"2024-09-11T00:04:03.836177+0000","last_active":"2024-09-11T00:04:17.435560+0000","last_peered":"2024-09-11T00:04:17.435560+0000","last_clean":"2024-09-11T00:04:17.435560+0000","last_became_active":"2024-09-11T00:04:03.835848+0000","last_became_peered":"2024-09-11T00:04:03.835848+0000","last_unstale":"2024-09-11T00:04:17.435560+0000","last_undegraded":"2024-09-11T00:04:17.435560+0000","last_fullsized":"2024-09-11T00:04:17.435560+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:57:53.854738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T00:04:17.436393+0000","last_change":"2024-09-11T00:04:03.837614+0000","last_active":"2024-09-11T00:04:17.436393+0000","last_peered":"2024-09-11T00:04:17.436393+0000","last_clean":"2024-09-11T00:04:17.436393+0000","last_became_active":"2024-09-11T00:04:03.837164+0000","last_became_peered":"2024-09-11T00:04:03.837164+0000","last_unstale":"2024-09-11T00:04:17.436393+0000","last_undegraded":"2024-09-11T00:04:17.436393+0000","last_fullsized":"2024-09-11T00:04:17.436393+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:10:38.467648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T00:04:17.304716+0000","last_change":"2024-09-11T00:04:03.833999+0000","last_active":"2024-09-11T00:04:17.304716+0000","last_peered":"2024-09-11T00:04:17.304716+0000","last_clean":"2024-09-11T00:04:17.304716+0000","last_became_active":"2024-09-11T00:04:03.833580+0000","last_became_peered":"2024-09-11T00:04:03.833580+0000","last_unstale":"2024-09-11T00:04:17.304716+0000","last_undegraded":"2024-09-11T00:04:17.304716+0000","last_fullsized":"2024-09-11T00:04:17.304716+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:47:40.902423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T00:04:17.304868+0000","last_change":"2024-09-11T00:04:03.832478+0000","last_active":"2024-09-11T00:04:17.304868+0000","last_peered":"2024-09-11T00:04:17.304868+0000","last_clean":"2024-09-11T00:04:17.304868+0000","last_became_active":"2024-09-11T00:04:03.831811+0000","last_became_peered":"2024-09-11T00:04:03.831811+0000","last_unstale":"2024-09-11T00:04:17.304868+0000","last_undegraded":"2024-09-11T00:04:17.304868+0000","last_fullsized":"2024-09-11T00:04:17.304868+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:53:05.839006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T00:04:15.869905+0000","last_change":"2024-09-11T00:04:03.837077+0000","last_active":"2024-09-11T00:04:15.869905+0000","last_peered":"2024-09-11T00:04:15.869905+0000","last_clean":"2024-09-11T00:04:15.869905+0000","last_became_active":"2024-09-11T00:04:03.835718+0000","last_became_peered":"2024-09-11T00:04:03.835718+0000","last_unstale":"2024-09-11T00:04:15.869905+0000","last_undegraded":"2024-09-11T00:04:15.869905+0000","last_fullsized":"2024-09-11T00:04:15.869905+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:17:00.633257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":63364,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":195,"num_read_kb":168,"num_write":242,"num_write_kb":132,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":552960,"data_stored":190092,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":209,"ondisk_log_size":209,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":68,"seq":292057776154,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36240,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708912,"statfs":{"total":95995035648,"available":95957925888,"internally_reserved":0,"allocated":917504,"data_stored":449299,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":36174276},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":63,"seq":270582939679,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28592,"kb_used_data":1568,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716560,"statfs":{"total":95995035648,"available":95965757440,"internally_reserved":0,"allocated":1605632,"data_stored":1091884,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":58,"seq":249108103202,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36268,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708884,"statfs":{"total":95995035648,"available":95957897216,"internally_reserved":0,"allocated":946176,"data_stored":452934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":52,"seq":223338299433,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36260,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708892,"statfs":{"total":95995035648,"available":95957905408,"internally_reserved":0,"allocated":937984,"data_stored":449216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":47,"seq":201863462957,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28524,"kb_used_data":1500,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716628,"statfs":{"total":95995035648,"available":95965827072,"internally_reserved":0,"allocated":1536000,"data_stored":1045713,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":42,"seq":180388626484,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32080,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713072,"statfs":{"total":95995035648,"available":95962185728,"internally_reserved":0,"allocated":917504,"data_stored":444733,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":36,"seq":154618822713,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28532,"kb_used_data":1508,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716620,"statfs":{"total":95995035648,"available":95965818880,"internally_reserved":0,"allocated":1544192,"data_stored":1040904,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986238,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27928,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717224,"statfs":{"total":95995035648,"available":95966437376,"internally_reserved":0,"allocated":925696,"data_stored":448416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182467,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32092,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713060,"statfs":{"total":95995035648,"available":95962173440,"internally_reserved":0,"allocated":929792,"data_stored":444050,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31914440},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411402,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28000,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717152,"statfs":{"total":95995035648,"available":95966363648,"internally_reserved":0,"allocated":999424,"data_stored":502143,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574926,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32140,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713012,"statfs":{"total":95995035648,"available":95962124288,"internally_reserved":0,"allocated":978944,"data_stored":501923,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738452,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32104,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713048,"statfs":{"total":95995035648,"available":95962161152,"internally_reserved":0,"allocated":942080,"data_stored":446189,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2569,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":58303,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":58523,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":700,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":2820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3533,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":57896,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1583,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T00:05:08.264 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-09-11T00:05:08.264 INFO:tasks.cephadm:Setting up client nodes... 2024-09-11T00:05:08.265 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ff3ad74a-6fcf-11ef-bcea-c7b262605968 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-09-11T00:05:08.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:05:07 smithi165 ceph-mon[35356]: pgmap v335: 129 pgs: 129 active+clean; 639 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 5.6 KiB/s wr, 17 op/s 2024-09-11T00:05:08.659 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:05:08.942 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:05:08 smithi012 ceph-mon[30333]: from='client.24896 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T00:05:09.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:05:08 smithi178 ceph-mon[35919]: from='client.24896 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T00:05:09.207 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:09.202+0000 7f782fc00640 1 -- 172.21.15.12:0/935882398 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7828104c40 msgr2=0x7f7828105040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:05:09.207 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:09.202+0000 7f782fc00640 1 --2- 172.21.15.12:0/935882398 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7828104c40 0x7f7828105040 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f7818009930 tx=0x7f781802a420 comp rx=0 tx=0).stop 2024-09-11T00:05:09.207 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:09.202+0000 7f782fc00640 1 -- 172.21.15.12:0/935882398 shutdown_connections 2024-09-11T00:05:09.207 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:09.202+0000 7f782fc00640 1 --2- 172.21.15.12:0/935882398 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7828106760 0x7f782810d0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:05:09.207 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:09.202+0000 7f782fc00640 1 --2- 172.21.15.12:0/935882398 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7828105da0 0x7f7828106220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:05:09.207 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:09.202+0000 7f782fc00640 1 --2- 172.21.15.12:0/935882398 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7828104c40 0x7f7828105040 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:05:09.207 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:09.202+0000 7f782fc00640 1 -- 172.21.15.12:0/935882398 >> 172.21.15.12:0/935882398 conn(0x7f7828100510 msgr2=0x7f7828102950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:05:09.207 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:09.202+0000 7f782fc00640 1 -- 172.21.15.12:0/935882398 shutdown_connections 2024-09-11T00:05:09.207 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:09.202+0000 7f782fc00640 1 -- 172.21.15.12:0/935882398 wait complete. 2024-09-11T00:05:09.208 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:09.203+0000 7f782fc00640 1 Processor -- start 2024-09-11T00:05:09.208 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:09.203+0000 7f782fc00640 1 -- start start 2024-09-11T00:05:09.208 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:09.203+0000 7f782fc00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7828104c40 0x7f782819b070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:05:09.208 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:09.203+0000 7f782fc00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7828105da0 0x7f782819b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:05:09.208 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:09.203+0000 7f782fc00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7828106760 0x7f782819c290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:05:09.208 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:09.203+0000 7f782fc00640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f78281108c0 con 0x7f7828104c40 2024-09-11T00:05:09.208 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:09.203+0000 7f782fc00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f7828110740 con 0x7f7828105da0 2024-09-11T00:05:09.208 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:09.203+0000 7f782fc00640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f7828110a40 con 0x7f7828106760 2024-09-11T00:05:09.208 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:09.204+0000 7f782d200640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7828104c40 0x7f782819b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:05:09.208 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:09.204+0000 7f782d200640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7828104c40 0x7f782819b070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:60924/0 (socket says 172.21.15.12:60924) 2024-09-11T00:05:09.208 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:09.204+0000 7f782d200640 1 -- 172.21.15.12:0/1166879242 learned_addr learned my addr 172.21.15.12:0/1166879242 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:05:09.208 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:09.204+0000 7f782dc00640 1 --2- 172.21.15.12:0/1166879242 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7828106760 0x7f782819c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:05:09.208 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:09.204+0000 7f781fe00640 1 --2- 172.21.15.12:0/1166879242 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7828105da0 0x7f782819b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:05:09.209 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:09.204+0000 7f782d200640 1 -- 172.21.15.12:0/1166879242 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7828106760 msgr2=0x7f782819c290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:05:09.209 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:09.204+0000 7f782d200640 1 --2- 172.21.15.12:0/1166879242 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7828106760 0x7f782819c290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:05:09.209 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:09.204+0000 7f782d200640 1 -- 172.21.15.12:0/1166879242 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7828105da0 msgr2=0x7f782819b6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:05:09.209 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:09.204+0000 7f782d200640 1 --2- 172.21.15.12:0/1166879242 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7828105da0 0x7f782819b6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:05:09.209 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:09.204+0000 7f782d200640 1 -- 172.21.15.12:0/1166879242 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f782819cb30 con 0x7f7828104c40 2024-09-11T00:05:09.209 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:09.204+0000 7f782d200640 1 --2- 172.21.15.12:0/1166879242 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7828104c40 0x7f782819b070 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f7818005ab0 tx=0x7f78180042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:05:09.209 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:09.205+0000 7f7826000640 1 -- 172.21.15.12:0/1166879242 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f781803f030 con 0x7f7828104c40 2024-09-11T00:05:09.209 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:09.205+0000 7f7826000640 1 -- 172.21.15.12:0/1166879242 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7818032e40 con 0x7f7828104c40 2024-09-11T00:05:09.209 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:09.205+0000 7f7826000640 1 -- 172.21.15.12:0/1166879242 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7818035070 con 0x7f7828104c40 2024-09-11T00:05:09.209 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:09.205+0000 7f782fc00640 1 -- 172.21.15.12:0/1166879242 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f782819cdc0 con 0x7f7828104c40 2024-09-11T00:05:09.209 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:09.205+0000 7f782fc00640 1 -- 172.21.15.12:0/1166879242 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78281a3500 con 0x7f7828104c40 2024-09-11T00:05:09.210 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:09.206+0000 7f782fc00640 1 -- 172.21.15.12:0/1166879242 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7828105040 con 0x7f7828104c40 2024-09-11T00:05:09.210 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:09.208+0000 7f7826000640 1 -- 172.21.15.12:0/1166879242 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 17) ==== 153301+0+0 (secure 0 0 0) 0x7f781803ba60 con 0x7f7828104c40 2024-09-11T00:05:09.212 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:09.211+0000 7f7826000640 1 --2- 172.21.15.12:0/1166879242 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f780c0b53f0 0x7f780c0b78b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:05:09.213 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:09.211+0000 7f7826000640 1 -- 172.21.15.12:0/1166879242 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8802+0+0 (secure 0 0 0) 0x7f78181066c0 con 0x7f7828104c40 2024-09-11T00:05:09.213 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:09.211+0000 7f781fe00640 1 --2- 172.21.15.12:0/1166879242 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f780c0b53f0 0x7f780c0b78b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:05:09.213 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:09.212+0000 7f781fe00640 1 --2- 172.21.15.12:0/1166879242 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f780c0b53f0 0x7f780c0b78b0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f7810009bd0 tx=0x7f7810005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:05:09.217 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:09.215+0000 7f7826000640 1 -- 172.21.15.12:0/1166879242 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f78180ccb30 con 0x7f7828104c40 2024-09-11T00:05:09.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:05:08 smithi165 ceph-mon[35356]: from='client.24896 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T00:05:09.487 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:09.485+0000 7f782fc00640 1 -- 172.21.15.12:0/1166879242 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) -- 0x7f782802e5c0 con 0x7f7828104c40 2024-09-11T00:05:09.490 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:09.488+0000 7f7826000640 1 -- 172.21.15.12:0/1166879242 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v32) ==== 170+0+59 (secure 0 0 0) 0x7f78180d1560 con 0x7f7828104c40 2024-09-11T00:05:09.491 INFO:teuthology.orchestra.run.smithi012.stdout:[client.0] 2024-09-11T00:05:09.491 INFO:teuthology.orchestra.run.smithi012.stdout: key = AQC13uBmlwoVHRAAa7LnNsHYvl2tww/cMagopw== 2024-09-11T00:05:09.493 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:09.491+0000 7f782fc00640 1 -- 172.21.15.12:0/1166879242 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f780c0b53f0 msgr2=0x7f780c0b78b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:05:09.493 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:09.491+0000 7f782fc00640 1 --2- 172.21.15.12:0/1166879242 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f780c0b53f0 0x7f780c0b78b0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f7810009bd0 tx=0x7f7810005f90 comp rx=0 tx=0).stop 2024-09-11T00:05:09.494 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:09.491+0000 7f782fc00640 1 -- 172.21.15.12:0/1166879242 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7828104c40 msgr2=0x7f782819b070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:05:09.494 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:09.491+0000 7f782fc00640 1 --2- 172.21.15.12:0/1166879242 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7828104c40 0x7f782819b070 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f7818005ab0 tx=0x7f78180042f0 comp rx=0 tx=0).stop 2024-09-11T00:05:09.494 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:09.491+0000 7f782fc00640 1 -- 172.21.15.12:0/1166879242 shutdown_connections 2024-09-11T00:05:09.494 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:09.491+0000 7f782fc00640 1 --2- 172.21.15.12:0/1166879242 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f780c0b53f0 0x7f780c0b78b0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:05:09.494 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:09.491+0000 7f782fc00640 1 --2- 172.21.15.12:0/1166879242 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7828106760 0x7f782819c290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:05:09.495 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:09.491+0000 7f782fc00640 1 --2- 172.21.15.12:0/1166879242 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7828105da0 0x7f782819b6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:05:09.495 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:09.491+0000 7f782fc00640 1 --2- 172.21.15.12:0/1166879242 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7828104c40 0x7f782819b070 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:05:09.495 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:09.491+0000 7f782fc00640 1 -- 172.21.15.12:0/1166879242 >> 172.21.15.12:0/1166879242 conn(0x7f7828100510 msgr2=0x7f7828109190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:05:09.495 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:09.491+0000 7f782fc00640 1 -- 172.21.15.12:0/1166879242 shutdown_connections 2024-09-11T00:05:09.495 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:09.491+0000 7f782fc00640 1 -- 172.21.15.12:0/1166879242 wait complete. 2024-09-11T00:05:10.063 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:05:09 smithi012 ceph-mon[30333]: pgmap v336: 129 pgs: 129 active+clean; 639 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 3.2 KiB/s wr, 10 op/s 2024-09-11T00:05:10.063 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:05:09 smithi012 ceph-mon[30333]: from='client.? 172.21.15.12:0/1166879242' entity='client.admin' cmd={"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} : dispatch 2024-09-11T00:05:10.063 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:05:09 smithi012 ceph-mon[30333]: from='client.? 172.21.15.12:0/1166879242' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-11T00:05:10.080 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:05:09 smithi178 ceph-mon[35919]: pgmap v336: 129 pgs: 129 active+clean; 639 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 3.2 KiB/s wr, 10 op/s 2024-09-11T00:05:10.080 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:05:09 smithi178 ceph-mon[35919]: from='client.? 172.21.15.12:0/1166879242' entity='client.admin' cmd={"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} : dispatch 2024-09-11T00:05:10.080 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:05:09 smithi178 ceph-mon[35919]: from='client.? 172.21.15.12:0/1166879242' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-11T00:05:10.274 DEBUG:teuthology.orchestra.run.smithi012:> set -ex 2024-09-11T00:05:10.274 DEBUG:teuthology.orchestra.run.smithi012:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-09-11T00:05:10.274 DEBUG:teuthology.orchestra.run.smithi012:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-09-11T00:05:10.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:05:09 smithi165 ceph-mon[35356]: pgmap v336: 129 pgs: 129 active+clean; 639 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 3.2 KiB/s wr, 10 op/s 2024-09-11T00:05:10.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:05:09 smithi165 ceph-mon[35356]: from='client.? 172.21.15.12:0/1166879242' entity='client.admin' cmd={"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} : dispatch 2024-09-11T00:05:10.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:05:09 smithi165 ceph-mon[35356]: from='client.? 172.21.15.12:0/1166879242' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-11T00:05:10.355 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid ff3ad74a-6fcf-11ef-bcea-c7b262605968 -- ceph config log 1 --format=json 2024-09-11T00:05:10.774 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:05:11.313 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:11.311+0000 7f06ca800640 1 -- 172.21.15.12:0/2430348672 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f06c4075540 msgr2=0x7f06c4075920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:05:11.313 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:11.311+0000 7f06ca800640 1 --2- 172.21.15.12:0/2430348672 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f06c4075540 0x7f06c4075920 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f06b4009900 tx=0x7f06b402a410 comp rx=0 tx=0).stop 2024-09-11T00:05:11.313 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:11.311+0000 7f06ca800640 1 -- 172.21.15.12:0/2430348672 shutdown_connections 2024-09-11T00:05:11.313 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:11.311+0000 7f06ca800640 1 --2- 172.21.15.12:0/2430348672 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f06c410f3e0 0x7f06c41117d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:05:11.313 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:11.311+0000 7f06ca800640 1 --2- 172.21.15.12:0/2430348672 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f06c4075f40 0x7f06c410ee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:05:11.313 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:11.311+0000 7f06ca800640 1 --2- 172.21.15.12:0/2430348672 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f06c4075540 0x7f06c4075920 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:05:11.314 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:11.311+0000 7f06ca800640 1 -- 172.21.15.12:0/2430348672 >> 172.21.15.12:0/2430348672 conn(0x7f06c40fe830 msgr2=0x7f06c4100c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:05:11.314 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:11.311+0000 7f06ca800640 1 -- 172.21.15.12:0/2430348672 shutdown_connections 2024-09-11T00:05:11.314 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:11.311+0000 7f06ca800640 1 -- 172.21.15.12:0/2430348672 wait complete. 2024-09-11T00:05:11.314 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:11.312+0000 7f06ca800640 1 Processor -- start 2024-09-11T00:05:11.314 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:11.312+0000 7f06ca800640 1 -- start start 2024-09-11T00:05:11.314 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:11.312+0000 7f06ca800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f06c4075540 0x7f06c419b4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:05:11.315 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:11.312+0000 7f06ca800640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f06c4075f40 0x7f06c419b9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:05:11.315 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:11.312+0000 7f06ca800640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f06c410f3e0 0x7f06c41a0d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:05:11.315 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:11.312+0000 7f06ca800640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f06c4114f70 con 0x7f06c410f3e0 2024-09-11T00:05:11.315 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:11.312+0000 7f06ca800640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f06c4114df0 con 0x7f06c4075540 2024-09-11T00:05:11.315 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:11.312+0000 7f06ca800640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f06c41150f0 con 0x7f06c4075f40 2024-09-11T00:05:11.316 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:11.313+0000 7f06c3400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f06c4075540 0x7f06c419b4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:05:11.316 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:11.313+0000 7f06c3400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f06c4075540 0x7f06c419b4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.12:44760/0 (socket says 172.21.15.12:44760) 2024-09-11T00:05:11.316 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:11.313+0000 7f06c3e00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f06c410f3e0 0x7f06c41a0d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:05:11.316 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:11.313+0000 7f06c3400640 1 -- 172.21.15.12:0/2284050461 learned_addr learned my addr 172.21.15.12:0/2284050461 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:05:11.316 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:11.313+0000 7f06baa00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f06c4075f40 0x7f06c419b9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:05:11.316 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:11.313+0000 7f06c3400640 1 -- 172.21.15.12:0/2284050461 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f06c4075f40 msgr2=0x7f06c419b9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:05:11.316 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:11.313+0000 7f06c3400640 1 --2- 172.21.15.12:0/2284050461 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f06c4075f40 0x7f06c419b9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:05:11.316 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:11.313+0000 7f06c3400640 1 -- 172.21.15.12:0/2284050461 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f06c410f3e0 msgr2=0x7f06c41a0d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:05:11.316 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:11.313+0000 7f06c3400640 1 --2- 172.21.15.12:0/2284050461 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f06c410f3e0 0x7f06c41a0d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:05:11.317 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:11.313+0000 7f06c3400640 1 -- 172.21.15.12:0/2284050461 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06c41a1550 con 0x7f06c4075540 2024-09-11T00:05:11.317 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:11.314+0000 7f06c3400640 1 --2- 172.21.15.12:0/2284050461 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f06c4075540 0x7f06c419b4a0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f06b4005940 tx=0x7f06b4004270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:05:11.317 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:11.314+0000 7f06c0c00640 1 -- 172.21.15.12:0/2284050461 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06b403f030 con 0x7f06c4075540 2024-09-11T00:05:11.317 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:11.314+0000 7f06ca800640 1 -- 172.21.15.12:0/2284050461 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f06c41a17e0 con 0x7f06c4075540 2024-09-11T00:05:11.317 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:11.314+0000 7f06c0c00640 1 -- 172.21.15.12:0/2284050461 <== mon.2 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f06b4003aa0 con 0x7f06c4075540 2024-09-11T00:05:11.317 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:11.314+0000 7f06c0c00640 1 -- 172.21.15.12:0/2284050461 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06b4035070 con 0x7f06c4075540 2024-09-11T00:05:11.317 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:11.314+0000 7f06ca800640 1 -- 172.21.15.12:0/2284050461 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06c41a7f00 con 0x7f06c4075540 2024-09-11T00:05:11.317 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:11.315+0000 7f06ca800640 1 -- 172.21.15.12:0/2284050461 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f06c4075920 con 0x7f06c4075540 2024-09-11T00:05:11.319 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:11.317+0000 7f06c0c00640 1 -- 172.21.15.12:0/2284050461 <== mon.2 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 17) ==== 153301+0+0 (secure 0 0 0) 0x7f06b403bcc0 con 0x7f06c4075540 2024-09-11T00:05:11.321 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:11.319+0000 7f06c0c00640 1 --2- 172.21.15.12:0/2284050461 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f06a80b5370 0x7f06a80b7830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:05:11.321 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:11.319+0000 7f06c0c00640 1 -- 172.21.15.12:0/2284050461 <== mon.2 v2:172.21.15.165:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8802+0+0 (secure 0 0 0) 0x7f06b4106bb0 con 0x7f06c4075540 2024-09-11T00:05:11.321 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:11.319+0000 7f06baa00640 1 --2- 172.21.15.12:0/2284050461 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f06a80b5370 0x7f06a80b7830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:05:11.321 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:11.320+0000 7f06baa00640 1 --2- 172.21.15.12:0/2284050461 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f06a80b5370 0x7f06a80b7830 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f06c4101040 tx=0x7f06ac00b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:05:11.322 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:11.321+0000 7f06c0c00640 1 -- 172.21.15.12:0/2284050461 <== mon.2 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f06b40cd020 con 0x7f06c4075540 2024-09-11T00:05:11.522 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:11.520+0000 7f06ca800640 1 -- 172.21.15.12:0/2284050461 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "config log", "num": 1, "format": "json"} v 0) -- 0x7f06c41a8230 con 0x7f06c4075540 2024-09-11T00:05:11.523 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:11.521+0000 7f06c0c00640 1 -- 172.21.15.12:0/2284050461 <== mon.2 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "config log", "num": 1, "format": "json"}]=0 v51) ==== 86+0+133 (secure 0 0 0) 0x7f06b40d1a50 con 0x7f06c4075540 2024-09-11T00:05:11.523 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-11T00:05:11.526 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:11.524+0000 7f06ca800640 1 -- 172.21.15.12:0/2284050461 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f06a80b5370 msgr2=0x7f06a80b7830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:05:11.526 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:11.524+0000 7f06ca800640 1 --2- 172.21.15.12:0/2284050461 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f06a80b5370 0x7f06a80b7830 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f06c4101040 tx=0x7f06ac00b390 comp rx=0 tx=0).stop 2024-09-11T00:05:11.526 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:11.524+0000 7f06ca800640 1 -- 172.21.15.12:0/2284050461 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f06c4075540 msgr2=0x7f06c419b4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:05:11.526 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:11.524+0000 7f06ca800640 1 --2- 172.21.15.12:0/2284050461 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f06c4075540 0x7f06c419b4a0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f06b4005940 tx=0x7f06b4004270 comp rx=0 tx=0).stop 2024-09-11T00:05:11.527 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:11.524+0000 7f06ca800640 1 -- 172.21.15.12:0/2284050461 shutdown_connections 2024-09-11T00:05:11.527 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:11.524+0000 7f06ca800640 1 --2- 172.21.15.12:0/2284050461 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f06a80b5370 0x7f06a80b7830 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:05:11.527 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:11.524+0000 7f06ca800640 1 --2- 172.21.15.12:0/2284050461 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f06c410f3e0 0x7f06c41a0d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:05:11.527 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:11.524+0000 7f06ca800640 1 --2- 172.21.15.12:0/2284050461 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f06c4075f40 0x7f06c419b9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:05:11.527 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:11.524+0000 7f06ca800640 1 --2- 172.21.15.12:0/2284050461 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f06c4075540 0x7f06c419b4a0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:05:11.527 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:11.524+0000 7f06ca800640 1 -- 172.21.15.12:0/2284050461 >> 172.21.15.12:0/2284050461 conn(0x7f06c40fe830 msgr2=0x7f06c41115d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:05:11.527 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:11.524+0000 7f06ca800640 1 -- 172.21.15.12:0/2284050461 shutdown_connections 2024-09-11T00:05:11.527 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:11.524+0000 7f06ca800640 1 -- 172.21.15.12:0/2284050461 wait complete. 2024-09-11T00:05:12.063 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:05:11 smithi012 ceph-mon[30333]: pgmap v337: 129 pgs: 129 active+clean; 639 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 3.2 KiB/s wr, 12 op/s 2024-09-11T00:05:12.063 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:05:11 smithi012 ceph-mon[30333]: from='client.? 172.21.15.12:0/2284050461' entity='client.admin' cmd={"prefix": "config log", "num": 1, "format": "json"} : dispatch 2024-09-11T00:05:12.080 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:05:11 smithi178 ceph-mon[35919]: pgmap v337: 129 pgs: 129 active+clean; 639 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 3.2 KiB/s wr, 12 op/s 2024-09-11T00:05:12.080 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:05:11 smithi178 ceph-mon[35919]: from='client.? 172.21.15.12:0/2284050461' entity='client.admin' cmd={"prefix": "config log", "num": 1, "format": "json"} : dispatch 2024-09-11T00:05:12.264 INFO:teuthology.orchestra.run.smithi012.stdout:[{"version":51,"timestamp":"2024-09-11T00:03:52.064450+0000","name":"","changes":[{"name":"mds.all/mds_join_fs","new_value":"all"}]}] 2024-09-11T00:05:12.264 INFO:tasks.ceph_manager:config epoch is 51 2024-09-11T00:05:12.265 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-09-11T00:05:12.265 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-09-11T00:05:12.265 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid ff3ad74a-6fcf-11ef-bcea-c7b262605968 -- ceph mgr dump --format=json 2024-09-11T00:05:12.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:05:11 smithi165 ceph-mon[35356]: pgmap v337: 129 pgs: 129 active+clean; 639 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 3.2 KiB/s wr, 12 op/s 2024-09-11T00:05:12.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:05:11 smithi165 ceph-mon[35356]: from='client.? 172.21.15.12:0/2284050461' entity='client.admin' cmd={"prefix": "config log", "num": 1, "format": "json"} : dispatch 2024-09-11T00:05:12.643 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:05:13.203 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:13.200+0000 7ff1a3200640 1 -- 172.21.15.12:0/4142895178 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff19c10ede0 msgr2=0x7ff19c10f220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:05:13.203 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:13.200+0000 7ff1a3200640 1 --2- 172.21.15.12:0/4142895178 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff19c10ede0 0x7ff19c10f220 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7ff188009900 tx=0x7ff18802a3f0 comp rx=0 tx=0).stop 2024-09-11T00:05:13.203 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:13.200+0000 7ff1a3200640 1 -- 172.21.15.12:0/4142895178 shutdown_connections 2024-09-11T00:05:13.203 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:13.200+0000 7ff1a3200640 1 --2- 172.21.15.12:0/4142895178 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff19c10f760 0x7ff19c115f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:05:13.203 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:13.200+0000 7ff1a3200640 1 --2- 172.21.15.12:0/4142895178 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff19c10ede0 0x7ff19c10f220 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:05:13.203 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:13.200+0000 7ff1a3200640 1 --2- 172.21.15.12:0/4142895178 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff19c1093d0 0x7ff19c1097b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:05:13.203 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:13.200+0000 7ff1a3200640 1 -- 172.21.15.12:0/4142895178 >> 172.21.15.12:0/4142895178 conn(0x7ff19c078a40 msgr2=0x7ff19c07ae60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:05:13.203 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:13.200+0000 7ff1a3200640 1 -- 172.21.15.12:0/4142895178 shutdown_connections 2024-09-11T00:05:13.204 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:13.200+0000 7ff1a3200640 1 -- 172.21.15.12:0/4142895178 wait complete. 2024-09-11T00:05:13.205 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:13.201+0000 7ff1a3200640 1 Processor -- start 2024-09-11T00:05:13.205 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:13.201+0000 7ff1a3200640 1 -- start start 2024-09-11T00:05:13.205 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:13.202+0000 7ff1a3200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff19c1093d0 0x7ff19c19f9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:05:13.205 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:13.202+0000 7ff1a3200640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff19c10ede0 0x7ff19c19fee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:05:13.205 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:13.202+0000 7ff1a3200640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff19c10f760 0x7ff19c1a5280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:05:13.206 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:13.202+0000 7ff1a3200640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7ff19c1193b0 con 0x7ff19c10f760 2024-09-11T00:05:13.206 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:13.202+0000 7ff1a3200640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7ff19c119230 con 0x7ff19c1093d0 2024-09-11T00:05:13.206 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:13.202+0000 7ff1a3200640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7ff19c119530 con 0x7ff19c10ede0 2024-09-11T00:05:13.206 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:13.202+0000 7ff19be00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff19c1093d0 0x7ff19c19f9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:05:13.206 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:13.202+0000 7ff19b400640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff19c10ede0 0x7ff19c19fee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:05:13.206 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:13.202+0000 7ff19be00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff19c1093d0 0x7ff19c19f9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.12:44768/0 (socket says 172.21.15.12:44768) 2024-09-11T00:05:13.206 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:13.202+0000 7ff19be00640 1 -- 172.21.15.12:0/1889245527 learned_addr learned my addr 172.21.15.12:0/1889245527 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:05:13.206 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:13.202+0000 7ff1a1200640 1 --2- 172.21.15.12:0/1889245527 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff19c10f760 0x7ff19c1a5280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:05:13.206 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:13.203+0000 7ff19be00640 1 -- 172.21.15.12:0/1889245527 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff19c10ede0 msgr2=0x7ff19c19fee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:05:13.207 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:13.203+0000 7ff19be00640 1 --2- 172.21.15.12:0/1889245527 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff19c10ede0 0x7ff19c19fee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:05:13.207 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:13.203+0000 7ff19be00640 1 -- 172.21.15.12:0/1889245527 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff19c10f760 msgr2=0x7ff19c1a5280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:05:13.207 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:13.203+0000 7ff19be00640 1 --2- 172.21.15.12:0/1889245527 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff19c10f760 0x7ff19c1a5280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:05:13.207 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:13.203+0000 7ff19be00640 1 -- 172.21.15.12:0/1889245527 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff19c1a5a50 con 0x7ff19c1093d0 2024-09-11T00:05:13.207 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:13.203+0000 7ff19be00640 1 --2- 172.21.15.12:0/1889245527 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff19c1093d0 0x7ff19c19f9a0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7ff18c00e9f0 tx=0x7ff18c00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:05:13.207 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:13.203+0000 7ff198c00640 1 -- 172.21.15.12:0/1889245527 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff18c00cd80 con 0x7ff19c1093d0 2024-09-11T00:05:13.207 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:13.203+0000 7ff1a3200640 1 -- 172.21.15.12:0/1889245527 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff19c1a5d40 con 0x7ff19c1093d0 2024-09-11T00:05:13.207 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:13.203+0000 7ff198c00640 1 -- 172.21.15.12:0/1889245527 <== mon.2 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff18c0108e0 con 0x7ff19c1093d0 2024-09-11T00:05:13.207 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:13.203+0000 7ff198c00640 1 -- 172.21.15.12:0/1889245527 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff18c018980 con 0x7ff19c1093d0 2024-09-11T00:05:13.207 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:13.203+0000 7ff1a3200640 1 -- 172.21.15.12:0/1889245527 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff19c1ac400 con 0x7ff19c1093d0 2024-09-11T00:05:13.208 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:13.204+0000 7ff1a3200640 1 -- 172.21.15.12:0/1889245527 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff19c1097b0 con 0x7ff19c1093d0 2024-09-11T00:05:13.208 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:13.207+0000 7ff198c00640 1 -- 172.21.15.12:0/1889245527 <== mon.2 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 17) ==== 153301+0+0 (secure 0 0 0) 0x7ff18c010430 con 0x7ff19c1093d0 2024-09-11T00:05:13.210 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:13.208+0000 7ff198c00640 1 --2- 172.21.15.12:0/1889245527 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7ff1800b5370 0x7ff1800b7830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:05:13.210 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:13.208+0000 7ff198c00640 1 -- 172.21.15.12:0/1889245527 <== mon.2 v2:172.21.15.165:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8802+0+0 (secure 0 0 0) 0x7ff18c0e6790 con 0x7ff19c1093d0 2024-09-11T00:05:13.210 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:13.208+0000 7ff19b400640 1 --2- 172.21.15.12:0/1889245527 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7ff1800b5370 0x7ff1800b7830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:05:13.211 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:13.209+0000 7ff19b400640 1 --2- 172.21.15.12:0/1889245527 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7ff1800b5370 0x7ff1800b7830 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7ff188005ab0 tx=0x7ff188005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:05:13.212 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:13.210+0000 7ff198c00640 1 -- 172.21.15.12:0/1889245527 <== mon.2 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7ff18c0acc00 con 0x7ff19c1093d0 2024-09-11T00:05:13.456 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:13.454+0000 7ff1a3200640 1 -- 172.21.15.12:0/1889245527 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7ff19c1a0ad0 con 0x7ff19c1093d0 2024-09-11T00:05:13.461 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:13.459+0000 7ff198c00640 1 -- 172.21.15.12:0/1889245527 <== mon.2 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v17) ==== 74+0+294215 (secure 0 0 0) 0x7ff18c0b1630 con 0x7ff19c1093d0 2024-09-11T00:05:13.466 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-11T00:05:13.468 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:13.464+0000 7ff1a3200640 1 -- 172.21.15.12:0/1889245527 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7ff1800b5370 msgr2=0x7ff1800b7830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:05:13.468 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:13.464+0000 7ff1a3200640 1 --2- 172.21.15.12:0/1889245527 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7ff1800b5370 0x7ff1800b7830 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7ff188005ab0 tx=0x7ff188005a20 comp rx=0 tx=0).stop 2024-09-11T00:05:13.468 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:13.464+0000 7ff1a3200640 1 -- 172.21.15.12:0/1889245527 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff19c1093d0 msgr2=0x7ff19c19f9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:05:13.468 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:13.464+0000 7ff1a3200640 1 --2- 172.21.15.12:0/1889245527 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff19c1093d0 0x7ff19c19f9a0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7ff18c00e9f0 tx=0x7ff18c00eeb0 comp rx=0 tx=0).stop 2024-09-11T00:05:13.468 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:13.464+0000 7ff1a3200640 1 -- 172.21.15.12:0/1889245527 shutdown_connections 2024-09-11T00:05:13.469 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:13.464+0000 7ff1a3200640 1 --2- 172.21.15.12:0/1889245527 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7ff1800b5370 0x7ff1800b7830 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:05:13.469 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:13.464+0000 7ff1a3200640 1 --2- 172.21.15.12:0/1889245527 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff19c10f760 0x7ff19c1a5280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:05:13.469 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:13.464+0000 7ff1a3200640 1 --2- 172.21.15.12:0/1889245527 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff19c10ede0 0x7ff19c19fee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:05:13.469 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:13.464+0000 7ff1a3200640 1 --2- 172.21.15.12:0/1889245527 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff19c1093d0 0x7ff19c19f9a0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:05:13.469 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:13.464+0000 7ff1a3200640 1 -- 172.21.15.12:0/1889245527 >> 172.21.15.12:0/1889245527 conn(0x7ff19c078a40 msgr2=0x7ff19c07ae30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:05:13.469 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:13.464+0000 7ff1a3200640 1 -- 172.21.15.12:0/1889245527 shutdown_connections 2024-09-11T00:05:13.469 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:13.464+0000 7ff1a3200640 1 -- 172.21.15.12:0/1889245527 wait complete. 2024-09-11T00:05:14.080 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:05:13 smithi178 ceph-mon[35919]: pgmap v338: 129 pgs: 129 active+clean; 643 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 3.4 KiB/s wr, 12 op/s 2024-09-11T00:05:14.080 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:05:13 smithi178 ceph-mon[35919]: from='client.? 172.21.15.12:0/1889245527' entity='client.admin' cmd={"prefix": "mgr dump", "format": "json"} : dispatch 2024-09-11T00:05:14.177 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:05:13 smithi012 ceph-mon[30333]: pgmap v338: 129 pgs: 129 active+clean; 643 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 3.4 KiB/s wr, 12 op/s 2024-09-11T00:05:14.177 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:05:13 smithi012 ceph-mon[30333]: from='client.? 172.21.15.12:0/1889245527' entity='client.admin' cmd={"prefix": "mgr dump", "format": "json"} : dispatch 2024-09-11T00:05:14.213 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":17,"flags":0,"active_gid":14150,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6800","nonce":3442939137},{"type":"v1","addr":"172.21.15.12:6801","nonce":3442939137}]},"active_addr":"172.21.15.12:6801/3442939137","active_change":"2024-09-10T23:56:17.999938+0000","active_mgr_features":4540719139924082687,"available":true,"standbys":[{"gid":24124,"name":"z","mgr_features":4540719139924082687,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to, use commas to separate multiple","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.27.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:10.4.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_nginx":{"name":"container_image_nginx","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nginx:1.26.1","min":"","max":"","enum_allowed":[],"desc":"Nginx container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.7.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.17","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_oauth2_proxy":{"name":"container_image_oauth2_proxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/oauth2-proxy/oauth2-proxy:v7.6.0","min":"","max":"","enum_allowed":[],"desc":"oauth2-proxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.51.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba_metrics":{"name":"container_image_samba_metrics","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-metrics:latest","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB metrics exporter container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_requests":{"name":"max_requests","type":"int","level":"advanced","flags":0,"default_value":"500","min":"","max":"","enum_allowed":[],"desc":"Maximum number of requests to keep in memory. When new request comes in, the oldest request will be removed if the number of requests exceeds the max request number.if un-finished request is removed, error message will be logged in the ceph-mgr log.","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"smb","can_run":true,"error_string":"","module_options":{"internal_store_backend":{"name":"internal_store_backend","type":"str","level":"dev","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"set internal store backend. for develoment and testing only","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_orchestration":{"name":"update_orchestration","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically update orchestration when smb resources are changed","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]},{"gid":24131,"name":"y","mgr_features":4540719139924082687,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to, use commas to separate multiple","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.27.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:10.4.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_nginx":{"name":"container_image_nginx","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nginx:1.26.1","min":"","max":"","enum_allowed":[],"desc":"Nginx container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.7.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.17","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_oauth2_proxy":{"name":"container_image_oauth2_proxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/oauth2-proxy/oauth2-proxy:v7.6.0","min":"","max":"","enum_allowed":[],"desc":"oauth2-proxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.51.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba_metrics":{"name":"container_image_samba_metrics","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-metrics:latest","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB metrics exporter container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_requests":{"name":"max_requests","type":"int","level":"advanced","flags":0,"default_value":"500","min":"","max":"","enum_allowed":[],"desc":"Maximum number of requests to keep in memory. When new request comes in, the oldest request will be removed if the number of requests exceeds the max request number.if un-finished request is removed, error message will be logged in the ceph-mgr log.","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"smb","can_run":true,"error_string":"","module_options":{"internal_store_backend":{"name":"internal_store_backend","type":"str","level":"dev","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"set internal store backend. for develoment and testing only","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_orchestration":{"name":"update_orchestration","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically update orchestration when smb resources are changed","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to, use commas to separate multiple","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.27.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:10.4.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_nginx":{"name":"container_image_nginx","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nginx:1.26.1","min":"","max":"","enum_allowed":[],"desc":"Nginx container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.7.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.17","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_oauth2_proxy":{"name":"container_image_oauth2_proxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/oauth2-proxy/oauth2-proxy:v7.6.0","min":"","max":"","enum_allowed":[],"desc":"oauth2-proxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.51.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba_metrics":{"name":"container_image_samba_metrics","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-metrics:latest","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB metrics exporter container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_requests":{"name":"max_requests","type":"int","level":"advanced","flags":0,"default_value":"500","min":"","max":"","enum_allowed":[],"desc":"Maximum number of requests to keep in memory. When new request comes in, the oldest request will be removed if the number of requests exceeds the max request number.if un-finished request is removed, error message will be logged in the ceph-mgr log.","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"smb","can_run":true,"error_string":"","module_options":{"internal_store_backend":{"name":"internal_store_backend","type":"str","level":"dev","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"set internal store backend. for develoment and testing only","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_orchestration":{"name":"update_orchestration","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically update orchestration when smb resources are changed","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.12:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"force_disabled_modules":{},"last_failure_osd_epoch":3,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.12:0","nonce":1700982273}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.12:0","nonce":3715726086}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.12:0","nonce":4005282650}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.12:0","nonce":1844819639}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.12:0","nonce":628715908}]}]} 2024-09-11T00:05:14.218 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-09-11T00:05:14.219 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-09-11T00:05:14.219 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid ff3ad74a-6fcf-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-11T00:05:14.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:05:13 smithi165 ceph-mon[35356]: pgmap v338: 129 pgs: 129 active+clean; 643 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 3.4 KiB/s wr, 12 op/s 2024-09-11T00:05:14.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:05:13 smithi165 ceph-mon[35356]: from='client.? 172.21.15.12:0/1889245527' entity='client.admin' cmd={"prefix": "mgr dump", "format": "json"} : dispatch 2024-09-11T00:05:14.599 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:05:15.128 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:15.125+0000 7f5497c00640 1 -- 172.21.15.12:0/993278003 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5490109150 msgr2=0x7f549010f950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:05:15.128 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:15.125+0000 7f5497c00640 1 --2- 172.21.15.12:0/993278003 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5490109150 0x7f549010f950 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f5488009be0 tx=0x7f548802a6c0 comp rx=0 tx=0).stop 2024-09-11T00:05:15.128 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:15.125+0000 7f5497c00640 1 -- 172.21.15.12:0/993278003 shutdown_connections 2024-09-11T00:05:15.128 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:15.125+0000 7f5497c00640 1 --2- 172.21.15.12:0/993278003 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5490109150 0x7f549010f950 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:05:15.128 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:15.126+0000 7f5497c00640 1 --2- 172.21.15.12:0/993278003 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f54901087d0 0x7f5490108c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:05:15.128 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:15.126+0000 7f5497c00640 1 --2- 172.21.15.12:0/993278003 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5490102dc0 0x7f54901031a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:05:15.129 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:15.126+0000 7f5497c00640 1 -- 172.21.15.12:0/993278003 >> 172.21.15.12:0/993278003 conn(0x7f54900fe830 msgr2=0x7f5490100c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:05:15.129 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:15.126+0000 7f5497c00640 1 -- 172.21.15.12:0/993278003 shutdown_connections 2024-09-11T00:05:15.129 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:15.126+0000 7f5497c00640 1 -- 172.21.15.12:0/993278003 wait complete. 2024-09-11T00:05:15.133 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:15.127+0000 7f5497c00640 1 Processor -- start 2024-09-11T00:05:15.133 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:15.127+0000 7f5497c00640 1 -- start start 2024-09-11T00:05:15.133 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:15.127+0000 7f5497c00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5490102dc0 0x7f54900754a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:05:15.133 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:15.127+0000 7f5497c00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f54901087d0 0x7f54900759e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:05:15.133 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:15.127+0000 7f5497c00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5490109150 0x7f5490075f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:05:15.133 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:15.127+0000 7f5497c00640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f5490112dd0 con 0x7f54901087d0 2024-09-11T00:05:15.133 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:15.127+0000 7f5497c00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f5490112c50 con 0x7f5490109150 2024-09-11T00:05:15.133 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:15.127+0000 7f5497c00640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f5490112f50 con 0x7f5490102dc0 2024-09-11T00:05:15.133 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:15.127+0000 7f548fe00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f54901087d0 0x7f54900759e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:05:15.133 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:15.127+0000 7f548fe00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f54901087d0 0x7f54900759e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:60998/0 (socket says 172.21.15.12:60998) 2024-09-11T00:05:15.134 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:15.127+0000 7f548fe00640 1 -- 172.21.15.12:0/4096496153 learned_addr learned my addr 172.21.15.12:0/4096496153 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:05:15.134 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:15.127+0000 7f5495200640 1 --2- 172.21.15.12:0/4096496153 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5490102dc0 0x7f54900754a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:05:15.134 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:15.128+0000 7f5495c00640 1 --2- 172.21.15.12:0/4096496153 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5490109150 0x7f5490075f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:05:15.134 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:15.128+0000 7f548fe00640 1 -- 172.21.15.12:0/4096496153 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5490102dc0 msgr2=0x7f54900754a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:05:15.134 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:15.128+0000 7f548fe00640 1 --2- 172.21.15.12:0/4096496153 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5490102dc0 0x7f54900754a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:05:15.134 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:15.128+0000 7f548fe00640 1 -- 172.21.15.12:0/4096496153 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5490109150 msgr2=0x7f5490075f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:05:15.134 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:15.128+0000 7f548fe00640 1 --2- 172.21.15.12:0/4096496153 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5490109150 0x7f5490075f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:05:15.135 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:15.128+0000 7f548fe00640 1 -- 172.21.15.12:0/4096496153 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54901a7680 con 0x7f54901087d0 2024-09-11T00:05:15.135 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:15.128+0000 7f548fe00640 1 --2- 172.21.15.12:0/4096496153 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f54901087d0 0x7f54900759e0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f547c00e9c0 tx=0x7f547c00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:05:15.135 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:15.128+0000 7f548d600640 1 -- 172.21.15.12:0/4096496153 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f547c00cde0 con 0x7f54901087d0 2024-09-11T00:05:15.135 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:15.129+0000 7f548d600640 1 -- 172.21.15.12:0/4096496153 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f547c0035f0 con 0x7f54901087d0 2024-09-11T00:05:15.135 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:15.129+0000 7f548d600640 1 -- 172.21.15.12:0/4096496153 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f547c010690 con 0x7f54901087d0 2024-09-11T00:05:15.135 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:15.129+0000 7f5497c00640 1 -- 172.21.15.12:0/4096496153 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f54901a7970 con 0x7f54901087d0 2024-09-11T00:05:15.135 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:15.129+0000 7f5497c00640 1 -- 172.21.15.12:0/4096496153 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54901a7e30 con 0x7f54901087d0 2024-09-11T00:05:15.135 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:15.130+0000 7f5497c00640 1 -- 172.21.15.12:0/4096496153 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f54901031a0 con 0x7f54901087d0 2024-09-11T00:05:15.136 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:15.132+0000 7f548d600640 1 -- 172.21.15.12:0/4096496153 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 17) ==== 153301+0+0 (secure 0 0 0) 0x7f547c004000 con 0x7f54901087d0 2024-09-11T00:05:15.136 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:15.134+0000 7f548d600640 1 --2- 172.21.15.12:0/4096496153 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f54780b53f0 0x7f54780b78b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:05:15.136 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:15.134+0000 7f548d600640 1 -- 172.21.15.12:0/4096496153 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8802+0+0 (secure 0 0 0) 0x7f547c0e6310 con 0x7f54901087d0 2024-09-11T00:05:15.136 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:15.134+0000 7f5495200640 1 --2- 172.21.15.12:0/4096496153 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f54780b53f0 0x7f54780b78b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:05:15.137 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:15.135+0000 7f5495200640 1 --2- 172.21.15.12:0/4096496153 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f54780b53f0 0x7f54780b78b0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f5484009bd0 tx=0x7f5484005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:05:15.137 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:15.136+0000 7f548d600640 1 -- 172.21.15.12:0/4096496153 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f547c0ace60 con 0x7f54901087d0 2024-09-11T00:05:15.339 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:15.337+0000 7f5497c00640 1 -- 172.21.15.12:0/4096496153 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f549002e5c0 con 0x7f54901087d0 2024-09-11T00:05:15.341 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:15.339+0000 7f548d600640 1 -- 172.21.15.12:0/4096496153 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) ==== 74+0+24236 (secure 0 0 0) 0x7f547c0ac760 con 0x7f54901087d0 2024-09-11T00:05:15.341 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-11T00:05:15.342 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":81,"fsid":"ff3ad74a-6fcf-11ef-bcea-c7b262605968","created":"2024-09-10T23:55:16.317643+0000","modified":"2024-09-11T00:04:15.795652+0000","last_up_change":"2024-09-11T00:03:08.959256+0000","last_in_change":"2024-09-11T00:02:46.564631+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T23:59:10.488728+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-11T00:04:00.065618+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"74","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-11T00:04:02.689888+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"74","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-11T00:04:09.862120+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"6382c200-a912-42a1-958b-79191b21249b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6802","nonce":1237101991},{"type":"v1","addr":"172.21.15.12:6803","nonce":1237101991}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6804","nonce":1237101991},{"type":"v1","addr":"172.21.15.12:6805","nonce":1237101991}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6808","nonce":1237101991},{"type":"v1","addr":"172.21.15.12:6809","nonce":1237101991}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6806","nonce":1237101991},{"type":"v1","addr":"172.21.15.12:6807","nonce":1237101991}]},"public_addr":"172.21.15.12:6803/1237101991","cluster_addr":"172.21.15.12:6805/1237101991","heartbeat_back_addr":"172.21.15.12:6809/1237101991","heartbeat_front_addr":"172.21.15.12:6807/1237101991","state":["exists","up"]},{"osd":1,"uuid":"fe0e1d0a-bdc5-4859-aa84-9038059a45f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6800","nonce":545848456},{"type":"v1","addr":"172.21.15.165:6801","nonce":545848456}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6802","nonce":545848456},{"type":"v1","addr":"172.21.15.165:6803","nonce":545848456}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6806","nonce":545848456},{"type":"v1","addr":"172.21.15.165:6807","nonce":545848456}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6804","nonce":545848456},{"type":"v1","addr":"172.21.15.165:6805","nonce":545848456}]},"public_addr":"172.21.15.165:6801/545848456","cluster_addr":"172.21.15.165:6803/545848456","heartbeat_back_addr":"172.21.15.165:6807/545848456","heartbeat_front_addr":"172.21.15.165:6805/545848456","state":["exists","up"]},{"osd":2,"uuid":"94f640dc-5f78-4e2c-87d9-03e96a6e7ba0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6800","nonce":4131162695},{"type":"v1","addr":"172.21.15.178:6801","nonce":4131162695}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":4131162695},{"type":"v1","addr":"172.21.15.178:6803","nonce":4131162695}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":4131162695},{"type":"v1","addr":"172.21.15.178:6807","nonce":4131162695}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":4131162695},{"type":"v1","addr":"172.21.15.178:6805","nonce":4131162695}]},"public_addr":"172.21.15.178:6801/4131162695","cluster_addr":"172.21.15.178:6803/4131162695","heartbeat_back_addr":"172.21.15.178:6807/4131162695","heartbeat_front_addr":"172.21.15.178:6805/4131162695","state":["exists","up"]},{"osd":3,"uuid":"af2f64d6-c9be-44a9-b041-8d599a7c080d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6810","nonce":452464245},{"type":"v1","addr":"172.21.15.12:6811","nonce":452464245}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6812","nonce":452464245},{"type":"v1","addr":"172.21.15.12:6813","nonce":452464245}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6816","nonce":452464245},{"type":"v1","addr":"172.21.15.12:6817","nonce":452464245}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6814","nonce":452464245},{"type":"v1","addr":"172.21.15.12:6815","nonce":452464245}]},"public_addr":"172.21.15.12:6811/452464245","cluster_addr":"172.21.15.12:6813/452464245","heartbeat_back_addr":"172.21.15.12:6817/452464245","heartbeat_front_addr":"172.21.15.12:6815/452464245","state":["exists","up"]},{"osd":4,"uuid":"ad76ac70-20c0-44ae-9866-5f9cfce2334c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6808","nonce":1899879280},{"type":"v1","addr":"172.21.15.165:6809","nonce":1899879280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6810","nonce":1899879280},{"type":"v1","addr":"172.21.15.165:6811","nonce":1899879280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6814","nonce":1899879280},{"type":"v1","addr":"172.21.15.165:6815","nonce":1899879280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6812","nonce":1899879280},{"type":"v1","addr":"172.21.15.165:6813","nonce":1899879280}]},"public_addr":"172.21.15.165:6809/1899879280","cluster_addr":"172.21.15.165:6811/1899879280","heartbeat_back_addr":"172.21.15.165:6815/1899879280","heartbeat_front_addr":"172.21.15.165:6813/1899879280","state":["exists","up"]},{"osd":5,"uuid":"aafda2cc-62fe-4fcb-9a96-8732e0b501d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":3042228555},{"type":"v1","addr":"172.21.15.178:6809","nonce":3042228555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":3042228555},{"type":"v1","addr":"172.21.15.178:6811","nonce":3042228555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6814","nonce":3042228555},{"type":"v1","addr":"172.21.15.178:6815","nonce":3042228555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":3042228555},{"type":"v1","addr":"172.21.15.178:6813","nonce":3042228555}]},"public_addr":"172.21.15.178:6809/3042228555","cluster_addr":"172.21.15.178:6811/3042228555","heartbeat_back_addr":"172.21.15.178:6815/3042228555","heartbeat_front_addr":"172.21.15.178:6813/3042228555","state":["exists","up"]},{"osd":6,"uuid":"375eca52-efe3-42a5-bbb1-d7d392d71f15","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6818","nonce":3868322467},{"type":"v1","addr":"172.21.15.12:6819","nonce":3868322467}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6820","nonce":3868322467},{"type":"v1","addr":"172.21.15.12:6821","nonce":3868322467}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6824","nonce":3868322467},{"type":"v1","addr":"172.21.15.12:6825","nonce":3868322467}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6822","nonce":3868322467},{"type":"v1","addr":"172.21.15.12:6823","nonce":3868322467}]},"public_addr":"172.21.15.12:6819/3868322467","cluster_addr":"172.21.15.12:6821/3868322467","heartbeat_back_addr":"172.21.15.12:6825/3868322467","heartbeat_front_addr":"172.21.15.12:6823/3868322467","state":["exists","up"]},{"osd":7,"uuid":"0fd619f5-44c3-4cf7-ae7a-388dd798de36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6816","nonce":1143773735},{"type":"v1","addr":"172.21.15.165:6817","nonce":1143773735}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6818","nonce":1143773735},{"type":"v1","addr":"172.21.15.165:6819","nonce":1143773735}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6822","nonce":1143773735},{"type":"v1","addr":"172.21.15.165:6823","nonce":1143773735}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6820","nonce":1143773735},{"type":"v1","addr":"172.21.15.165:6821","nonce":1143773735}]},"public_addr":"172.21.15.165:6817/1143773735","cluster_addr":"172.21.15.165:6819/1143773735","heartbeat_back_addr":"172.21.15.165:6823/1143773735","heartbeat_front_addr":"172.21.15.165:6821/1143773735","state":["exists","up"]},{"osd":8,"uuid":"b03d14ce-8427-462a-9bcc-27683c549f53","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":2891445058},{"type":"v1","addr":"172.21.15.178:6817","nonce":2891445058}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6818","nonce":2891445058},{"type":"v1","addr":"172.21.15.178:6819","nonce":2891445058}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6822","nonce":2891445058},{"type":"v1","addr":"172.21.15.178:6823","nonce":2891445058}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":2891445058},{"type":"v1","addr":"172.21.15.178:6821","nonce":2891445058}]},"public_addr":"172.21.15.178:6817/2891445058","cluster_addr":"172.21.15.178:6819/2891445058","heartbeat_back_addr":"172.21.15.178:6823/2891445058","heartbeat_front_addr":"172.21.15.178:6821/2891445058","state":["exists","up"]},{"osd":9,"uuid":"81a6cf69-2a24-4a6c-9adf-6b7df38cfacb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6826","nonce":3359634903},{"type":"v1","addr":"172.21.15.12:6827","nonce":3359634903}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6828","nonce":3359634903},{"type":"v1","addr":"172.21.15.12:6829","nonce":3359634903}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6832","nonce":3359634903},{"type":"v1","addr":"172.21.15.12:6833","nonce":3359634903}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6830","nonce":3359634903},{"type":"v1","addr":"172.21.15.12:6831","nonce":3359634903}]},"public_addr":"172.21.15.12:6827/3359634903","cluster_addr":"172.21.15.12:6829/3359634903","heartbeat_back_addr":"172.21.15.12:6833/3359634903","heartbeat_front_addr":"172.21.15.12:6831/3359634903","state":["exists","up"]},{"osd":10,"uuid":"c1ad1f0a-33f4-4c32-91f6-4d030a29f1dd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6824","nonce":3642325778},{"type":"v1","addr":"172.21.15.165:6825","nonce":3642325778}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6826","nonce":3642325778},{"type":"v1","addr":"172.21.15.165:6827","nonce":3642325778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6830","nonce":3642325778},{"type":"v1","addr":"172.21.15.165:6831","nonce":3642325778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6828","nonce":3642325778},{"type":"v1","addr":"172.21.15.165:6829","nonce":3642325778}]},"public_addr":"172.21.15.165:6825/3642325778","cluster_addr":"172.21.15.165:6827/3642325778","heartbeat_back_addr":"172.21.15.165:6831/3642325778","heartbeat_front_addr":"172.21.15.165:6829/3642325778","state":["exists","up"]},{"osd":11,"uuid":"8413c9b0-680b-4d02-aeab-b3b3ff16c3d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":2937059756},{"type":"v1","addr":"172.21.15.178:6825","nonce":2937059756}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6826","nonce":2937059756},{"type":"v1","addr":"172.21.15.178:6827","nonce":2937059756}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":2937059756},{"type":"v1","addr":"172.21.15.178:6831","nonce":2937059756}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":2937059756},{"type":"v1","addr":"172.21.15.178:6829","nonce":2937059756}]},"public_addr":"172.21.15.178:6825/2937059756","cluster_addr":"172.21.15.178:6827/2937059756","heartbeat_back_addr":"172.21.15.178:6831/2937059756","heartbeat_front_addr":"172.21.15.178:6829/2937059756","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:58:16.036348+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:58:41.483150+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:59:07.107283+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:59:35.706574+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T00:00:02.555720+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T00:00:28.092847+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T00:00:57.396495+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T00:01:24.216775+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T00:01:48.807850+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T00:02:16.838565+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T00:02:43.131736+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T00:03:07.353155+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.12:6801/4177251511":"2024-09-11T23:56:17.999820+0000","172.21.15.12:6800/4177251511":"2024-09-11T23:56:17.999820+0000","172.21.15.12:0/3273692817":"2024-09-11T23:56:17.999820+0000","172.21.15.12:0/4017778158":"2024-09-11T23:56:17.999820+0000","172.21.15.12:0/2499921051":"2024-09-11T23:55:50.302721+0000","172.21.15.12:6801/760140910":"2024-09-11T23:55:50.302721+0000","172.21.15.12:6800/760140910":"2024-09-11T23:55:50.302721+0000","172.21.15.12:0/2810703302":"2024-09-11T23:56:17.999820+0000","172.21.15.12:0/2325436065":"2024-09-11T23:55:50.302721+0000","172.21.15.12:0/3137053479":"2024-09-11T23:55:50.302721+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-11T00:05:15.344 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:15.342+0000 7f5497c00640 1 -- 172.21.15.12:0/4096496153 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f54780b53f0 msgr2=0x7f54780b78b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:05:15.344 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:15.342+0000 7f5497c00640 1 --2- 172.21.15.12:0/4096496153 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f54780b53f0 0x7f54780b78b0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f5484009bd0 tx=0x7f5484005f90 comp rx=0 tx=0).stop 2024-09-11T00:05:15.344 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:15.342+0000 7f5497c00640 1 -- 172.21.15.12:0/4096496153 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f54901087d0 msgr2=0x7f54900759e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:05:15.344 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:15.342+0000 7f5497c00640 1 --2- 172.21.15.12:0/4096496153 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f54901087d0 0x7f54900759e0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f547c00e9c0 tx=0x7f547c00ee80 comp rx=0 tx=0).stop 2024-09-11T00:05:15.344 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:15.342+0000 7f5497c00640 1 -- 172.21.15.12:0/4096496153 shutdown_connections 2024-09-11T00:05:15.345 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:15.342+0000 7f5497c00640 1 --2- 172.21.15.12:0/4096496153 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f54780b53f0 0x7f54780b78b0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:05:15.345 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:15.342+0000 7f5497c00640 1 --2- 172.21.15.12:0/4096496153 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5490109150 0x7f5490075f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:05:15.345 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:15.342+0000 7f5497c00640 1 --2- 172.21.15.12:0/4096496153 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f54901087d0 0x7f54900759e0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:05:15.345 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:15.342+0000 7f5497c00640 1 --2- 172.21.15.12:0/4096496153 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5490102dc0 0x7f54900754a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:05:15.345 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:15.342+0000 7f5497c00640 1 -- 172.21.15.12:0/4096496153 >> 172.21.15.12:0/4096496153 conn(0x7f54900fe830 msgr2=0x7f54900ff150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:05:15.345 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:15.342+0000 7f5497c00640 1 -- 172.21.15.12:0/4096496153 shutdown_connections 2024-09-11T00:05:15.345 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:15.342+0000 7f5497c00640 1 -- 172.21.15.12:0/4096496153 wait complete. 2024-09-11T00:05:16.001 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:05:15 smithi012 ceph-mon[30333]: pgmap v339: 129 pgs: 129 active+clean; 643 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 1.7 KiB/s wr, 9 op/s 2024-09-11T00:05:16.001 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:05:15 smithi012 ceph-mon[30333]: from='client.? 172.21.15.12:0/4096496153' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-11T00:05:16.037 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-09-11T00:05:16.038 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid ff3ad74a-6fcf-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-11T00:05:16.080 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:05:15 smithi178 ceph-mon[35919]: pgmap v339: 129 pgs: 129 active+clean; 643 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 1.7 KiB/s wr, 9 op/s 2024-09-11T00:05:16.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:05:15 smithi178 ceph-mon[35919]: from='client.? 172.21.15.12:0/4096496153' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-11T00:05:16.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:05:15 smithi165 ceph-mon[35356]: pgmap v339: 129 pgs: 129 active+clean; 643 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 1.7 KiB/s wr, 9 op/s 2024-09-11T00:05:16.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:05:15 smithi165 ceph-mon[35356]: from='client.? 172.21.15.12:0/4096496153' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-11T00:05:16.419 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:05:16.960 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:16.957+0000 7f899d400640 1 -- 172.21.15.12:0/2617618241 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f899810b330 msgr2=0x7f8998111b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:05:16.960 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:16.957+0000 7f899d400640 1 --2- 172.21.15.12:0/2617618241 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f899810b330 0x7f8998111b30 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f8990009ca0 tx=0x7f899002a8c0 comp rx=0 tx=0).stop 2024-09-11T00:05:16.960 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:16.958+0000 7f899d400640 1 -- 172.21.15.12:0/2617618241 shutdown_connections 2024-09-11T00:05:16.960 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:16.958+0000 7f899d400640 1 --2- 172.21.15.12:0/2617618241 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f899810b330 0x7f8998111b30 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:05:16.960 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:16.958+0000 7f899d400640 1 --2- 172.21.15.12:0/2617618241 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f899810a9b0 0x7f899810adf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:05:16.960 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:16.958+0000 7f899d400640 1 --2- 172.21.15.12:0/2617618241 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8998104fa0 0x7f8998105380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:05:16.960 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:16.958+0000 7f899d400640 1 -- 172.21.15.12:0/2617618241 >> 172.21.15.12:0/2617618241 conn(0x7f8998100a50 msgr2=0x7f8998102e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:05:16.960 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:16.958+0000 7f899d400640 1 -- 172.21.15.12:0/2617618241 shutdown_connections 2024-09-11T00:05:16.960 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:16.958+0000 7f899d400640 1 -- 172.21.15.12:0/2617618241 wait complete. 2024-09-11T00:05:16.961 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:16.959+0000 7f899d400640 1 Processor -- start 2024-09-11T00:05:16.961 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:16.959+0000 7f899d400640 1 -- start start 2024-09-11T00:05:16.961 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:16.959+0000 7f899d400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8998104fa0 0x7f899819b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:05:16.961 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:16.959+0000 7f899d400640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f899810a9b0 0x7f899819b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:05:16.962 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:16.959+0000 7f899d400640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f899810b330 0x7f89981a0d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:05:16.962 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:16.959+0000 7f899d400640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f8998114f60 con 0x7f899810a9b0 2024-09-11T00:05:16.962 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:16.959+0000 7f899d400640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f8998114de0 con 0x7f8998104fa0 2024-09-11T00:05:16.962 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:16.959+0000 7f899d400640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f89981150e0 con 0x7f899810b330 2024-09-11T00:05:16.962 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:16.959+0000 7f8996800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8998104fa0 0x7f899819b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:05:16.962 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:16.959+0000 7f8995e00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f899810a9b0 0x7f899819b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:05:16.962 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:16.960+0000 7f8996800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8998104fa0 0x7f899819b480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.12:44816/0 (socket says 172.21.15.12:44816) 2024-09-11T00:05:16.962 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:16.960+0000 7f8996800640 1 -- 172.21.15.12:0/134738697 learned_addr learned my addr 172.21.15.12:0/134738697 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:05:16.962 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:16.960+0000 7f8997200640 1 --2- 172.21.15.12:0/134738697 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f899810b330 0x7f89981a0d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:05:16.963 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:16.960+0000 7f8995e00640 1 -- 172.21.15.12:0/134738697 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f899810b330 msgr2=0x7f89981a0d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:05:16.963 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:16.960+0000 7f8995e00640 1 --2- 172.21.15.12:0/134738697 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f899810b330 0x7f89981a0d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:05:16.963 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:16.960+0000 7f8995e00640 1 -- 172.21.15.12:0/134738697 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8998104fa0 msgr2=0x7f899819b480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:05:16.963 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:16.960+0000 7f8995e00640 1 --2- 172.21.15.12:0/134738697 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8998104fa0 0x7f899819b480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:05:16.963 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:16.960+0000 7f8995e00640 1 -- 172.21.15.12:0/134738697 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89981a1530 con 0x7f899810a9b0 2024-09-11T00:05:16.963 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:16.960+0000 7f8996800640 1 --2- 172.21.15.12:0/134738697 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8998104fa0 0x7f899819b480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:05:16.963 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:16.960+0000 7f8995e00640 1 --2- 172.21.15.12:0/134738697 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f899810a9b0 0x7f899819b9c0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f898400e9f0 tx=0x7f898400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:05:16.963 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:16.961+0000 7f898f400640 1 -- 172.21.15.12:0/134738697 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f898400ccc0 con 0x7f899810a9b0 2024-09-11T00:05:16.964 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:16.961+0000 7f898f400640 1 -- 172.21.15.12:0/134738697 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8984005020 con 0x7f899810a9b0 2024-09-11T00:05:16.964 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:16.961+0000 7f899d400640 1 -- 172.21.15.12:0/134738697 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f89981a1820 con 0x7f899810a9b0 2024-09-11T00:05:16.964 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:16.961+0000 7f898f400640 1 -- 172.21.15.12:0/134738697 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8984004a00 con 0x7f899810a9b0 2024-09-11T00:05:16.964 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:16.961+0000 7f899d400640 1 -- 172.21.15.12:0/134738697 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89981a7ee0 con 0x7f899810a9b0 2024-09-11T00:05:16.964 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:16.962+0000 7f899d400640 1 -- 172.21.15.12:0/134738697 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8998105380 con 0x7f899810a9b0 2024-09-11T00:05:16.966 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:16.965+0000 7f898f400640 1 -- 172.21.15.12:0/134738697 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 17) ==== 153301+0+0 (secure 0 0 0) 0x7f8984004ba0 con 0x7f899810a9b0 2024-09-11T00:05:16.968 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:16.966+0000 7f898f400640 1 --2- 172.21.15.12:0/134738697 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f897c0b53f0 0x7f897c0b78b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:05:16.968 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:16.967+0000 7f898f400640 1 -- 172.21.15.12:0/134738697 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8802+0+0 (secure 0 0 0) 0x7f89840e5ec0 con 0x7f899810a9b0 2024-09-11T00:05:16.968 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:16.967+0000 7f8996800640 1 --2- 172.21.15.12:0/134738697 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f897c0b53f0 0x7f897c0b78b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:05:16.969 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:16.967+0000 7f8996800640 1 --2- 172.21.15.12:0/134738697 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f897c0b53f0 0x7f897c0b78b0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f8988005fd0 tx=0x7f898800b4a0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:05:16.972 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:16.970+0000 7f898f400640 1 -- 172.21.15.12:0/134738697 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f89840ac980 con 0x7f899810a9b0 2024-09-11T00:05:17.170 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:17.168+0000 7f899d400640 1 -- 172.21.15.12:0/134738697 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f899802e5c0 con 0x7f899810a9b0 2024-09-11T00:05:17.171 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:17.170+0000 7f898f400640 1 -- 172.21.15.12:0/134738697 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) ==== 74+0+24236 (secure 0 0 0) 0x7f89840ac280 con 0x7f899810a9b0 2024-09-11T00:05:17.172 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-11T00:05:17.172 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":81,"fsid":"ff3ad74a-6fcf-11ef-bcea-c7b262605968","created":"2024-09-10T23:55:16.317643+0000","modified":"2024-09-11T00:04:15.795652+0000","last_up_change":"2024-09-11T00:03:08.959256+0000","last_in_change":"2024-09-11T00:02:46.564631+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T23:59:10.488728+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-11T00:04:00.065618+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"74","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-11T00:04:02.689888+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"74","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-11T00:04:09.862120+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"6382c200-a912-42a1-958b-79191b21249b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6802","nonce":1237101991},{"type":"v1","addr":"172.21.15.12:6803","nonce":1237101991}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6804","nonce":1237101991},{"type":"v1","addr":"172.21.15.12:6805","nonce":1237101991}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6808","nonce":1237101991},{"type":"v1","addr":"172.21.15.12:6809","nonce":1237101991}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6806","nonce":1237101991},{"type":"v1","addr":"172.21.15.12:6807","nonce":1237101991}]},"public_addr":"172.21.15.12:6803/1237101991","cluster_addr":"172.21.15.12:6805/1237101991","heartbeat_back_addr":"172.21.15.12:6809/1237101991","heartbeat_front_addr":"172.21.15.12:6807/1237101991","state":["exists","up"]},{"osd":1,"uuid":"fe0e1d0a-bdc5-4859-aa84-9038059a45f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6800","nonce":545848456},{"type":"v1","addr":"172.21.15.165:6801","nonce":545848456}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6802","nonce":545848456},{"type":"v1","addr":"172.21.15.165:6803","nonce":545848456}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6806","nonce":545848456},{"type":"v1","addr":"172.21.15.165:6807","nonce":545848456}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6804","nonce":545848456},{"type":"v1","addr":"172.21.15.165:6805","nonce":545848456}]},"public_addr":"172.21.15.165:6801/545848456","cluster_addr":"172.21.15.165:6803/545848456","heartbeat_back_addr":"172.21.15.165:6807/545848456","heartbeat_front_addr":"172.21.15.165:6805/545848456","state":["exists","up"]},{"osd":2,"uuid":"94f640dc-5f78-4e2c-87d9-03e96a6e7ba0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6800","nonce":4131162695},{"type":"v1","addr":"172.21.15.178:6801","nonce":4131162695}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":4131162695},{"type":"v1","addr":"172.21.15.178:6803","nonce":4131162695}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":4131162695},{"type":"v1","addr":"172.21.15.178:6807","nonce":4131162695}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":4131162695},{"type":"v1","addr":"172.21.15.178:6805","nonce":4131162695}]},"public_addr":"172.21.15.178:6801/4131162695","cluster_addr":"172.21.15.178:6803/4131162695","heartbeat_back_addr":"172.21.15.178:6807/4131162695","heartbeat_front_addr":"172.21.15.178:6805/4131162695","state":["exists","up"]},{"osd":3,"uuid":"af2f64d6-c9be-44a9-b041-8d599a7c080d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6810","nonce":452464245},{"type":"v1","addr":"172.21.15.12:6811","nonce":452464245}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6812","nonce":452464245},{"type":"v1","addr":"172.21.15.12:6813","nonce":452464245}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6816","nonce":452464245},{"type":"v1","addr":"172.21.15.12:6817","nonce":452464245}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6814","nonce":452464245},{"type":"v1","addr":"172.21.15.12:6815","nonce":452464245}]},"public_addr":"172.21.15.12:6811/452464245","cluster_addr":"172.21.15.12:6813/452464245","heartbeat_back_addr":"172.21.15.12:6817/452464245","heartbeat_front_addr":"172.21.15.12:6815/452464245","state":["exists","up"]},{"osd":4,"uuid":"ad76ac70-20c0-44ae-9866-5f9cfce2334c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6808","nonce":1899879280},{"type":"v1","addr":"172.21.15.165:6809","nonce":1899879280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6810","nonce":1899879280},{"type":"v1","addr":"172.21.15.165:6811","nonce":1899879280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6814","nonce":1899879280},{"type":"v1","addr":"172.21.15.165:6815","nonce":1899879280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6812","nonce":1899879280},{"type":"v1","addr":"172.21.15.165:6813","nonce":1899879280}]},"public_addr":"172.21.15.165:6809/1899879280","cluster_addr":"172.21.15.165:6811/1899879280","heartbeat_back_addr":"172.21.15.165:6815/1899879280","heartbeat_front_addr":"172.21.15.165:6813/1899879280","state":["exists","up"]},{"osd":5,"uuid":"aafda2cc-62fe-4fcb-9a96-8732e0b501d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":3042228555},{"type":"v1","addr":"172.21.15.178:6809","nonce":3042228555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":3042228555},{"type":"v1","addr":"172.21.15.178:6811","nonce":3042228555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6814","nonce":3042228555},{"type":"v1","addr":"172.21.15.178:6815","nonce":3042228555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":3042228555},{"type":"v1","addr":"172.21.15.178:6813","nonce":3042228555}]},"public_addr":"172.21.15.178:6809/3042228555","cluster_addr":"172.21.15.178:6811/3042228555","heartbeat_back_addr":"172.21.15.178:6815/3042228555","heartbeat_front_addr":"172.21.15.178:6813/3042228555","state":["exists","up"]},{"osd":6,"uuid":"375eca52-efe3-42a5-bbb1-d7d392d71f15","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6818","nonce":3868322467},{"type":"v1","addr":"172.21.15.12:6819","nonce":3868322467}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6820","nonce":3868322467},{"type":"v1","addr":"172.21.15.12:6821","nonce":3868322467}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6824","nonce":3868322467},{"type":"v1","addr":"172.21.15.12:6825","nonce":3868322467}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6822","nonce":3868322467},{"type":"v1","addr":"172.21.15.12:6823","nonce":3868322467}]},"public_addr":"172.21.15.12:6819/3868322467","cluster_addr":"172.21.15.12:6821/3868322467","heartbeat_back_addr":"172.21.15.12:6825/3868322467","heartbeat_front_addr":"172.21.15.12:6823/3868322467","state":["exists","up"]},{"osd":7,"uuid":"0fd619f5-44c3-4cf7-ae7a-388dd798de36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6816","nonce":1143773735},{"type":"v1","addr":"172.21.15.165:6817","nonce":1143773735}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6818","nonce":1143773735},{"type":"v1","addr":"172.21.15.165:6819","nonce":1143773735}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6822","nonce":1143773735},{"type":"v1","addr":"172.21.15.165:6823","nonce":1143773735}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6820","nonce":1143773735},{"type":"v1","addr":"172.21.15.165:6821","nonce":1143773735}]},"public_addr":"172.21.15.165:6817/1143773735","cluster_addr":"172.21.15.165:6819/1143773735","heartbeat_back_addr":"172.21.15.165:6823/1143773735","heartbeat_front_addr":"172.21.15.165:6821/1143773735","state":["exists","up"]},{"osd":8,"uuid":"b03d14ce-8427-462a-9bcc-27683c549f53","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":2891445058},{"type":"v1","addr":"172.21.15.178:6817","nonce":2891445058}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6818","nonce":2891445058},{"type":"v1","addr":"172.21.15.178:6819","nonce":2891445058}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6822","nonce":2891445058},{"type":"v1","addr":"172.21.15.178:6823","nonce":2891445058}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":2891445058},{"type":"v1","addr":"172.21.15.178:6821","nonce":2891445058}]},"public_addr":"172.21.15.178:6817/2891445058","cluster_addr":"172.21.15.178:6819/2891445058","heartbeat_back_addr":"172.21.15.178:6823/2891445058","heartbeat_front_addr":"172.21.15.178:6821/2891445058","state":["exists","up"]},{"osd":9,"uuid":"81a6cf69-2a24-4a6c-9adf-6b7df38cfacb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6826","nonce":3359634903},{"type":"v1","addr":"172.21.15.12:6827","nonce":3359634903}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6828","nonce":3359634903},{"type":"v1","addr":"172.21.15.12:6829","nonce":3359634903}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6832","nonce":3359634903},{"type":"v1","addr":"172.21.15.12:6833","nonce":3359634903}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6830","nonce":3359634903},{"type":"v1","addr":"172.21.15.12:6831","nonce":3359634903}]},"public_addr":"172.21.15.12:6827/3359634903","cluster_addr":"172.21.15.12:6829/3359634903","heartbeat_back_addr":"172.21.15.12:6833/3359634903","heartbeat_front_addr":"172.21.15.12:6831/3359634903","state":["exists","up"]},{"osd":10,"uuid":"c1ad1f0a-33f4-4c32-91f6-4d030a29f1dd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6824","nonce":3642325778},{"type":"v1","addr":"172.21.15.165:6825","nonce":3642325778}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6826","nonce":3642325778},{"type":"v1","addr":"172.21.15.165:6827","nonce":3642325778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6830","nonce":3642325778},{"type":"v1","addr":"172.21.15.165:6831","nonce":3642325778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6828","nonce":3642325778},{"type":"v1","addr":"172.21.15.165:6829","nonce":3642325778}]},"public_addr":"172.21.15.165:6825/3642325778","cluster_addr":"172.21.15.165:6827/3642325778","heartbeat_back_addr":"172.21.15.165:6831/3642325778","heartbeat_front_addr":"172.21.15.165:6829/3642325778","state":["exists","up"]},{"osd":11,"uuid":"8413c9b0-680b-4d02-aeab-b3b3ff16c3d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":2937059756},{"type":"v1","addr":"172.21.15.178:6825","nonce":2937059756}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6826","nonce":2937059756},{"type":"v1","addr":"172.21.15.178:6827","nonce":2937059756}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":2937059756},{"type":"v1","addr":"172.21.15.178:6831","nonce":2937059756}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":2937059756},{"type":"v1","addr":"172.21.15.178:6829","nonce":2937059756}]},"public_addr":"172.21.15.178:6825/2937059756","cluster_addr":"172.21.15.178:6827/2937059756","heartbeat_back_addr":"172.21.15.178:6831/2937059756","heartbeat_front_addr":"172.21.15.178:6829/2937059756","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:58:16.036348+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:58:41.483150+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:59:07.107283+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:59:35.706574+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T00:00:02.555720+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T00:00:28.092847+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T00:00:57.396495+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T00:01:24.216775+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T00:01:48.807850+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T00:02:16.838565+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T00:02:43.131736+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T00:03:07.353155+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.12:6801/4177251511":"2024-09-11T23:56:17.999820+0000","172.21.15.12:6800/4177251511":"2024-09-11T23:56:17.999820+0000","172.21.15.12:0/3273692817":"2024-09-11T23:56:17.999820+0000","172.21.15.12:0/4017778158":"2024-09-11T23:56:17.999820+0000","172.21.15.12:0/2499921051":"2024-09-11T23:55:50.302721+0000","172.21.15.12:6801/760140910":"2024-09-11T23:55:50.302721+0000","172.21.15.12:6800/760140910":"2024-09-11T23:55:50.302721+0000","172.21.15.12:0/2810703302":"2024-09-11T23:56:17.999820+0000","172.21.15.12:0/2325436065":"2024-09-11T23:55:50.302721+0000","172.21.15.12:0/3137053479":"2024-09-11T23:55:50.302721+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-11T00:05:17.176 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:17.172+0000 7f899d400640 1 -- 172.21.15.12:0/134738697 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f897c0b53f0 msgr2=0x7f897c0b78b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:05:17.176 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:17.172+0000 7f899d400640 1 --2- 172.21.15.12:0/134738697 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f897c0b53f0 0x7f897c0b78b0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f8988005fd0 tx=0x7f898800b4a0 comp rx=0 tx=0).stop 2024-09-11T00:05:17.177 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:17.172+0000 7f899d400640 1 -- 172.21.15.12:0/134738697 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f899810a9b0 msgr2=0x7f899819b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:05:17.177 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:17.172+0000 7f899d400640 1 --2- 172.21.15.12:0/134738697 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f899810a9b0 0x7f899819b9c0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f898400e9f0 tx=0x7f898400eeb0 comp rx=0 tx=0).stop 2024-09-11T00:05:17.177 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:17.173+0000 7f899d400640 1 -- 172.21.15.12:0/134738697 shutdown_connections 2024-09-11T00:05:17.177 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:17.173+0000 7f899d400640 1 --2- 172.21.15.12:0/134738697 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f897c0b53f0 0x7f897c0b78b0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:05:17.177 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:17.173+0000 7f899d400640 1 --2- 172.21.15.12:0/134738697 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f899810b330 0x7f89981a0d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:05:17.177 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:17.173+0000 7f899d400640 1 --2- 172.21.15.12:0/134738697 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f899810a9b0 0x7f899819b9c0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:05:17.177 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:17.173+0000 7f899d400640 1 --2- 172.21.15.12:0/134738697 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8998104fa0 0x7f899819b480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:05:17.177 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:17.173+0000 7f899d400640 1 -- 172.21.15.12:0/134738697 >> 172.21.15.12:0/134738697 conn(0x7f8998100a50 msgr2=0x7f8998102e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:05:17.177 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:17.173+0000 7f899d400640 1 -- 172.21.15.12:0/134738697 shutdown_connections 2024-09-11T00:05:17.177 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:17.173+0000 7f899d400640 1 -- 172.21.15.12:0/134738697 wait complete. 2024-09-11T00:05:17.858 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid ff3ad74a-6fcf-11ef-bcea-c7b262605968 -- ceph tell osd.0 flush_pg_stats 2024-09-11T00:05:17.859 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid ff3ad74a-6fcf-11ef-bcea-c7b262605968 -- ceph tell osd.1 flush_pg_stats 2024-09-11T00:05:17.859 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid ff3ad74a-6fcf-11ef-bcea-c7b262605968 -- ceph tell osd.2 flush_pg_stats 2024-09-11T00:05:17.860 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid ff3ad74a-6fcf-11ef-bcea-c7b262605968 -- ceph tell osd.3 flush_pg_stats 2024-09-11T00:05:17.860 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid ff3ad74a-6fcf-11ef-bcea-c7b262605968 -- ceph tell osd.4 flush_pg_stats 2024-09-11T00:05:17.860 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid ff3ad74a-6fcf-11ef-bcea-c7b262605968 -- ceph tell osd.5 flush_pg_stats 2024-09-11T00:05:17.863 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid ff3ad74a-6fcf-11ef-bcea-c7b262605968 -- ceph tell osd.6 flush_pg_stats 2024-09-11T00:05:17.863 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid ff3ad74a-6fcf-11ef-bcea-c7b262605968 -- ceph tell osd.7 flush_pg_stats 2024-09-11T00:05:17.864 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid ff3ad74a-6fcf-11ef-bcea-c7b262605968 -- ceph tell osd.8 flush_pg_stats 2024-09-11T00:05:17.864 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid ff3ad74a-6fcf-11ef-bcea-c7b262605968 -- ceph tell osd.9 flush_pg_stats 2024-09-11T00:05:17.864 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid ff3ad74a-6fcf-11ef-bcea-c7b262605968 -- ceph tell osd.10 flush_pg_stats 2024-09-11T00:05:17.865 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid ff3ad74a-6fcf-11ef-bcea-c7b262605968 -- ceph tell osd.11 flush_pg_stats 2024-09-11T00:05:18.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:05:17 smithi178 ceph-mon[35919]: pgmap v340: 129 pgs: 129 active+clean; 645 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 2.2 KiB/s wr, 14 op/s 2024-09-11T00:05:18.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:05:17 smithi178 ceph-mon[35919]: from='client.? 172.21.15.12:0/134738697' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-11T00:05:18.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:05:17 smithi165 ceph-mon[35356]: pgmap v340: 129 pgs: 129 active+clean; 645 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 2.2 KiB/s wr, 14 op/s 2024-09-11T00:05:18.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:05:17 smithi165 ceph-mon[35356]: from='client.? 172.21.15.12:0/134738697' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-11T00:05:18.316 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:05:17 smithi012 ceph-mon[30333]: pgmap v340: 129 pgs: 129 active+clean; 645 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 2.2 KiB/s wr, 14 op/s 2024-09-11T00:05:18.317 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:05:17 smithi012 ceph-mon[30333]: from='client.? 172.21.15.12:0/134738697' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-11T00:05:18.810 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:05:18.943 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:05:18.966 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:05:18.968 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:05:18.969 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:05:18.971 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:05:19.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:05:18 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [8, 4]} : dispatch 2024-09-11T00:05:19.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:05:18 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]} : dispatch 2024-09-11T00:05:19.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:05:18 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 4]} : dispatch 2024-09-11T00:05:19.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:05:18 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [8, 11]} : dispatch 2024-09-11T00:05:19.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:05:18 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [5, 9]} : dispatch 2024-09-11T00:05:19.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:05:18 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1e", "id": [3, 9]} : dispatch 2024-09-11T00:05:19.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:05:18 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.23", "id": [4, 7]} : dispatch 2024-09-11T00:05:19.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:05:18 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.31", "id": [4, 8]} : dispatch 2024-09-11T00:05:19.082 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:05:18 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.34", "id": [4, 9]} : dispatch 2024-09-11T00:05:19.082 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:05:18 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.39", "id": [5, 6]} : dispatch 2024-09-11T00:05:19.110 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:05:18 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [8, 4]} : dispatch 2024-09-11T00:05:19.110 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:05:18 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]} : dispatch 2024-09-11T00:05:19.110 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:05:18 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 4]} : dispatch 2024-09-11T00:05:19.111 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:05:18 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [8, 11]} : dispatch 2024-09-11T00:05:19.111 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:05:18 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [5, 9]} : dispatch 2024-09-11T00:05:19.111 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:05:18 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1e", "id": [3, 9]} : dispatch 2024-09-11T00:05:19.111 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:05:18 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.23", "id": [4, 7]} : dispatch 2024-09-11T00:05:19.111 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:05:18 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.31", "id": [4, 8]} : dispatch 2024-09-11T00:05:19.111 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:05:18 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.34", "id": [4, 9]} : dispatch 2024-09-11T00:05:19.111 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:05:18 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.39", "id": [5, 6]} : dispatch 2024-09-11T00:05:19.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:05:18 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [8, 4]} : dispatch 2024-09-11T00:05:19.283 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:05:18 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]} : dispatch 2024-09-11T00:05:19.283 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:05:18 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 4]} : dispatch 2024-09-11T00:05:19.283 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:05:18 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [8, 11]} : dispatch 2024-09-11T00:05:19.283 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:05:18 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [5, 9]} : dispatch 2024-09-11T00:05:19.283 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:05:18 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1e", "id": [3, 9]} : dispatch 2024-09-11T00:05:19.283 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:05:18 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.23", "id": [4, 7]} : dispatch 2024-09-11T00:05:19.283 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:05:18 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.31", "id": [4, 8]} : dispatch 2024-09-11T00:05:19.283 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:05:18 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.34", "id": [4, 9]} : dispatch 2024-09-11T00:05:19.284 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:05:18 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.39", "id": [5, 6]} : dispatch 2024-09-11T00:05:19.573 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:19.561+0000 7f23e3e00640 1 -- 172.21.15.12:0/2382484842 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f23e4075de0 msgr2=0x7f23e410ece0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:05:19.573 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:19.561+0000 7f23e3e00640 1 --2- 172.21.15.12:0/2382484842 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f23e4075de0 0x7f23e410ece0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f23cc009900 tx=0x7f23cc02a3f0 comp rx=0 tx=0).stop 2024-09-11T00:05:19.573 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:19.561+0000 7f23e3e00640 1 -- 172.21.15.12:0/2382484842 shutdown_connections 2024-09-11T00:05:19.573 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:19.561+0000 7f23e3e00640 1 --2- 172.21.15.12:0/2382484842 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f23e410f220 0x7f23e4111610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:05:19.573 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:19.561+0000 7f23e3e00640 1 --2- 172.21.15.12:0/2382484842 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f23e4075de0 0x7f23e410ece0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:05:19.574 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:19.561+0000 7f23e3e00640 1 --2- 172.21.15.12:0/2382484842 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f23e40754c0 0x7f23e40758a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:05:19.574 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:19.561+0000 7f23e3e00640 1 -- 172.21.15.12:0/2382484842 >> 172.21.15.12:0/2382484842 conn(0x7f23e40fe660 msgr2=0x7f23e4100a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:05:19.574 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:19.561+0000 7f23e3e00640 1 -- 172.21.15.12:0/2382484842 shutdown_connections 2024-09-11T00:05:19.574 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:19.561+0000 7f23e3e00640 1 -- 172.21.15.12:0/2382484842 wait complete. 2024-09-11T00:05:19.574 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:19.562+0000 7f23e3e00640 1 Processor -- start 2024-09-11T00:05:19.574 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:19.562+0000 7f23e3e00640 1 -- start start 2024-09-11T00:05:19.574 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:19.562+0000 7f23e3e00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f23e40754c0 0x7f23e419b280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:05:19.574 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:19.562+0000 7f23e3e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f23e4075de0 0x7f23e419b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:05:19.574 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:19.562+0000 7f23e3e00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f23e410f220 0x7f23e41a0b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:05:19.574 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:19.562+0000 7f23e3e00640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f23e4114e50 con 0x7f23e40754c0 2024-09-11T00:05:19.575 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:19.562+0000 7f23e3e00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f23e4114cd0 con 0x7f23e4075de0 2024-09-11T00:05:19.575 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:19.562+0000 7f23e3e00640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f23e4114fd0 con 0x7f23e410f220 2024-09-11T00:05:19.575 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:19.563+0000 7f23da000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f23e4075de0 0x7f23e419b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:05:19.575 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:19.563+0000 7f23e2a00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f23e40754c0 0x7f23e419b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:05:19.575 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:19.563+0000 7f23da000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f23e4075de0 0x7f23e419b7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.12:36790/0 (socket says 172.21.15.12:36790) 2024-09-11T00:05:19.575 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:19.563+0000 7f23da000640 1 -- 172.21.15.12:0/2904545266 learned_addr learned my addr 172.21.15.12:0/2904545266 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:05:19.575 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:19.563+0000 7f23e3400640 1 --2- 172.21.15.12:0/2904545266 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f23e410f220 0x7f23e41a0b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:05:19.575 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:19.563+0000 7f23da000640 1 -- 172.21.15.12:0/2904545266 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f23e410f220 msgr2=0x7f23e41a0b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:05:19.575 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:19.563+0000 7f23da000640 1 --2- 172.21.15.12:0/2904545266 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f23e410f220 0x7f23e41a0b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:05:19.576 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:19.563+0000 7f23da000640 1 -- 172.21.15.12:0/2904545266 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f23e40754c0 msgr2=0x7f23e419b280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:05:19.576 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:19.563+0000 7f23da000640 1 --2- 172.21.15.12:0/2904545266 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f23e40754c0 0x7f23e419b280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:05:19.576 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:19.563+0000 7f23da000640 1 -- 172.21.15.12:0/2904545266 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23e41a1330 con 0x7f23e4075de0 2024-09-11T00:05:19.576 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:19.564+0000 7f23da000640 1 --2- 172.21.15.12:0/2904545266 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f23e4075de0 0x7f23e419b7c0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f23cc005ab0 tx=0x7f23cc003c10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:05:19.576 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:19.564+0000 7f23dbe00640 1 -- 172.21.15.12:0/2904545266 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23cc03f070 con 0x7f23e4075de0 2024-09-11T00:05:19.576 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:19.564+0000 7f23e3e00640 1 -- 172.21.15.12:0/2904545266 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f23e41a15c0 con 0x7f23e4075de0 2024-09-11T00:05:19.576 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:19.564+0000 7f23dbe00640 1 -- 172.21.15.12:0/2904545266 <== mon.2 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f23cc031b60 con 0x7f23e4075de0 2024-09-11T00:05:19.576 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:19.564+0000 7f23dbe00640 1 -- 172.21.15.12:0/2904545266 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23cc035030 con 0x7f23e4075de0 2024-09-11T00:05:19.577 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:19.564+0000 7f23e3e00640 1 -- 172.21.15.12:0/2904545266 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23e41a7ce0 con 0x7f23e4075de0 2024-09-11T00:05:19.577 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:19.565+0000 7f23e3e00640 1 -- 172.21.15.12:0/2904545266 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f23e4100580 con 0x7f23e4075de0 2024-09-11T00:05:19.577 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:19.568+0000 7f23dbe00640 1 -- 172.21.15.12:0/2904545266 <== mon.2 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 17) ==== 153301+0+0 (secure 0 0 0) 0x7f23cc03b460 con 0x7f23e4075de0 2024-09-11T00:05:19.577 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:19.569+0000 7f23dbe00640 1 --2- 172.21.15.12:0/2904545266 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f23c80b5410 0x7f23c80b78d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:05:19.577 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:19.569+0000 7f23dbe00640 1 -- 172.21.15.12:0/2904545266 <== mon.2 v2:172.21.15.165:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 9092+0+0 (secure 0 0 0) 0x7f23cc107130 con 0x7f23e4075de0 2024-09-11T00:05:19.577 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:19.569+0000 7f23e2a00640 1 --2- 172.21.15.12:0/2904545266 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f23c80b5410 0x7f23c80b78d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:05:19.577 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:19.569+0000 7f23dbe00640 1 --2- 172.21.15.12:0/2904545266 >> [v2:172.21.15.12:6810/452464245,v1:172.21.15.12:6811/452464245] conn(0x7f23c80c04d0 0x7f23c80c2970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:05:19.577 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:19.569+0000 7f23dbe00640 1 -- 172.21.15.12:0/2904545266 --> [v2:172.21.15.12:6810/452464245,v1:172.21.15.12:6811/452464245] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f23cc009080 con 0x7f23c80c04d0 2024-09-11T00:05:19.577 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:19.570+0000 7f23dbe00640 1 -- 172.21.15.12:0/2904545266 <== mon.2 v2:172.21.15.165:3300/0 6 ==== mon_get_version_reply(handle=1 version=82) ==== 24+0+0 (secure 0 0 0) 0x7f23cc1074e0 con 0x7f23e4075de0 2024-09-11T00:05:19.578 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:19.570+0000 7f23e3400640 1 --2- 172.21.15.12:0/2904545266 >> [v2:172.21.15.12:6810/452464245,v1:172.21.15.12:6811/452464245] conn(0x7f23c80c04d0 0x7f23c80c2970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:05:19.578 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:19.570+0000 7f23e2a00640 1 --2- 172.21.15.12:0/2904545266 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f23c80b5410 0x7f23c80b78d0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f23d4005fd0 tx=0x7f23d4009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:05:19.578 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:19.570+0000 7f23e3400640 1 --2- 172.21.15.12:0/2904545266 >> [v2:172.21.15.12:6810/452464245,v1:172.21.15.12:6811/452464245] conn(0x7f23c80c04d0 0x7f23c80c2970 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:05:19.578 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:19.571+0000 7f23dbe00640 1 -- 172.21.15.12:0/2904545266 <== osd.3 v2:172.21.15.12:6810/452464245 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f23cc009080 con 0x7f23c80c04d0 2024-09-11T00:05:19.588 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:19.586+0000 7f23e3e00640 1 -- 172.21.15.12:0/2904545266 --> [v2:172.21.15.12:6810/452464245,v1:172.21.15.12:6811/452464245] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f23e40769d0 con 0x7f23c80c04d0 2024-09-11T00:05:19.588 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:19.587+0000 7f23dbe00640 1 -- 172.21.15.12:0/2904545266 <== osd.3 v2:172.21.15.12:6810/452464245 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f23e40769d0 con 0x7f23c80c04d0 2024-09-11T00:05:19.589 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:19.587+0000 7f23e3e00640 1 -- 172.21.15.12:0/2904545266 >> [v2:172.21.15.12:6810/452464245,v1:172.21.15.12:6811/452464245] conn(0x7f23c80c04d0 msgr2=0x7f23c80c2970 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:05:19.589 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:19.587+0000 7f23e3e00640 1 --2- 172.21.15.12:0/2904545266 >> [v2:172.21.15.12:6810/452464245,v1:172.21.15.12:6811/452464245] conn(0x7f23c80c04d0 0x7f23c80c2970 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:05:19.589 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:19.587+0000 7f23e3e00640 1 -- 172.21.15.12:0/2904545266 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f23c80b5410 msgr2=0x7f23c80b78d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:05:19.590 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:19.587+0000 7f23e3e00640 1 --2- 172.21.15.12:0/2904545266 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f23c80b5410 0x7f23c80b78d0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f23d4005fd0 tx=0x7f23d4009500 comp rx=0 tx=0).stop 2024-09-11T00:05:19.590 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:19.587+0000 7f23e3e00640 1 -- 172.21.15.12:0/2904545266 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f23e4075de0 msgr2=0x7f23e419b7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:05:19.590 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:19.587+0000 7f23e3e00640 1 --2- 172.21.15.12:0/2904545266 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f23e4075de0 0x7f23e419b7c0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f23cc005ab0 tx=0x7f23cc003c10 comp rx=0 tx=0).stop 2024-09-11T00:05:19.590 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:19.587+0000 7f23e3400640 1 -- 172.21.15.12:0/2904545266 reap_dead start 2024-09-11T00:05:19.590 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:19.587+0000 7f23e3e00640 1 -- 172.21.15.12:0/2904545266 shutdown_connections 2024-09-11T00:05:19.590 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:19.587+0000 7f23e3e00640 1 -- 172.21.15.12:0/2904545266 >> 172.21.15.12:0/2904545266 conn(0x7f23e40fe660 msgr2=0x7f23e41113c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:05:19.590 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:19.588+0000 7f23e3e00640 1 -- 172.21.15.12:0/2904545266 shutdown_connections 2024-09-11T00:05:19.590 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:19.588+0000 7f23e3e00640 1 -- 172.21.15.12:0/2904545266 wait complete. 2024-09-11T00:05:20.063 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:05:19 smithi012 ceph-mon[30333]: pgmap v341: 129 pgs: 129 active+clean; 645 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 853 B/s wr, 8 op/s 2024-09-11T00:05:20.063 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:05:19 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [8, 4]}]': finished 2024-09-11T00:05:20.063 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:05:19 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]': finished 2024-09-11T00:05:20.063 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:05:19 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 4]}]': finished 2024-09-11T00:05:20.063 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:05:19 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [8, 11]}]': finished 2024-09-11T00:05:20.063 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:05:19 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [5, 9]}]': finished 2024-09-11T00:05:20.063 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:05:19 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1e", "id": [3, 9]}]': finished 2024-09-11T00:05:20.063 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:05:19 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.23", "id": [4, 7]}]': finished 2024-09-11T00:05:20.063 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:05:19 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.31", "id": [4, 8]}]': finished 2024-09-11T00:05:20.063 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:05:19 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.34", "id": [4, 9]}]': finished 2024-09-11T00:05:20.064 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:05:19 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.39", "id": [5, 6]}]': finished 2024-09-11T00:05:20.064 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:05:19 smithi012 ceph-mon[30333]: osdmap e82: 12 total, 12 up, 12 in 2024-09-11T00:05:20.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:05:19 smithi178 ceph-mon[35919]: pgmap v341: 129 pgs: 129 active+clean; 645 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 853 B/s wr, 8 op/s 2024-09-11T00:05:20.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:05:19 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [8, 4]}]': finished 2024-09-11T00:05:20.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:05:19 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]': finished 2024-09-11T00:05:20.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:05:19 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 4]}]': finished 2024-09-11T00:05:20.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:05:19 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [8, 11]}]': finished 2024-09-11T00:05:20.082 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:05:19 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [5, 9]}]': finished 2024-09-11T00:05:20.082 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:05:19 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1e", "id": [3, 9]}]': finished 2024-09-11T00:05:20.082 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:05:19 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.23", "id": [4, 7]}]': finished 2024-09-11T00:05:20.082 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:05:19 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.31", "id": [4, 8]}]': finished 2024-09-11T00:05:20.082 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:05:19 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.34", "id": [4, 9]}]': finished 2024-09-11T00:05:20.082 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:05:19 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.39", "id": [5, 6]}]': finished 2024-09-11T00:05:20.082 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:05:19 smithi178 ceph-mon[35919]: osdmap e82: 12 total, 12 up, 12 in 2024-09-11T00:05:20.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:05:19 smithi165 ceph-mon[35356]: pgmap v341: 129 pgs: 129 active+clean; 645 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 853 B/s wr, 8 op/s 2024-09-11T00:05:20.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:05:19 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [8, 4]}]': finished 2024-09-11T00:05:20.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:05:19 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]': finished 2024-09-11T00:05:20.283 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:05:19 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 4]}]': finished 2024-09-11T00:05:20.283 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:05:19 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [8, 11]}]': finished 2024-09-11T00:05:20.283 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:05:19 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [5, 9]}]': finished 2024-09-11T00:05:20.283 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:05:19 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1e", "id": [3, 9]}]': finished 2024-09-11T00:05:20.283 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:05:19 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.23", "id": [4, 7]}]': finished 2024-09-11T00:05:20.283 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:05:19 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.31", "id": [4, 8]}]': finished 2024-09-11T00:05:20.283 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:05:19 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.34", "id": [4, 9]}]': finished 2024-09-11T00:05:20.283 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:05:19 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.39", "id": [5, 6]}]': finished 2024-09-11T00:05:20.283 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:05:19 smithi165 ceph-mon[35356]: osdmap e82: 12 total, 12 up, 12 in 2024-09-11T00:05:20.694 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:20.692+0000 7fbe7bc00640 1 -- 172.21.15.12:0/3891663244 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbe7410a9b0 msgr2=0x7fbe7410adf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:05:20.694 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:20.692+0000 7fbe7bc00640 1 --2- 172.21.15.12:0/3891663244 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbe7410a9b0 0x7fbe7410adf0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fbe60009860 tx=0x7fbe6002a340 comp rx=0 tx=0).stop 2024-09-11T00:05:20.694 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:20.692+0000 7fbe7bc00640 1 -- 172.21.15.12:0/3891663244 shutdown_connections 2024-09-11T00:05:20.694 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:20.692+0000 7fbe7bc00640 1 --2- 172.21.15.12:0/3891663244 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbe7410b330 0x7fbe74111b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:05:20.695 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:20.692+0000 7fbe7bc00640 1 --2- 172.21.15.12:0/3891663244 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbe7410a9b0 0x7fbe7410adf0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:05:20.695 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:20.692+0000 7fbe7bc00640 1 --2- 172.21.15.12:0/3891663244 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbe74104fa0 0x7fbe74105380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:05:20.695 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:20.692+0000 7fbe7bc00640 1 -- 172.21.15.12:0/3891663244 >> 172.21.15.12:0/3891663244 conn(0x7fbe74100a50 msgr2=0x7fbe74102e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:05:20.695 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:20.692+0000 7fbe7bc00640 1 -- 172.21.15.12:0/3891663244 shutdown_connections 2024-09-11T00:05:20.695 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:20.692+0000 7fbe7bc00640 1 -- 172.21.15.12:0/3891663244 wait complete. 2024-09-11T00:05:20.697 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:20.693+0000 7fbe7bc00640 1 Processor -- start 2024-09-11T00:05:20.697 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:20.693+0000 7fbe7bc00640 1 -- start start 2024-09-11T00:05:20.697 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:20.693+0000 7fbe7bc00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbe74104fa0 0x7fbe7419b4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:05:20.697 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:20.693+0000 7fbe7bc00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbe7410a9b0 0x7fbe7419b9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:05:20.697 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:20.694+0000 7fbe7bc00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbe7410b330 0x7fbe741a0d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:05:20.698 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:20.694+0000 7fbe7bc00640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fbe74114f30 con 0x7fbe74104fa0 2024-09-11T00:05:20.698 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:20.694+0000 7fbe7bc00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fbe74114db0 con 0x7fbe7410a9b0 2024-09-11T00:05:20.698 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:20.694+0000 7fbe7bc00640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fbe741150b0 con 0x7fbe7410b330 2024-09-11T00:05:20.698 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:20.694+0000 7fbe73e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbe7410a9b0 0x7fbe7419b9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:05:20.698 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:20.694+0000 7fbe79200640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbe74104fa0 0x7fbe7419b4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:05:20.698 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:20.694+0000 7fbe73e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbe7410a9b0 0x7fbe7419b9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.12:36808/0 (socket says 172.21.15.12:36808) 2024-09-11T00:05:20.698 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:20.694+0000 7fbe73e00640 1 -- 172.21.15.12:0/145923564 learned_addr learned my addr 172.21.15.12:0/145923564 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:05:20.698 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:20.694+0000 7fbe79c00640 1 --2- 172.21.15.12:0/145923564 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbe7410b330 0x7fbe741a0d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:05:20.698 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:20.694+0000 7fbe79200640 1 -- 172.21.15.12:0/145923564 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbe7410b330 msgr2=0x7fbe741a0d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:05:20.698 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:20.694+0000 7fbe79200640 1 --2- 172.21.15.12:0/145923564 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbe7410b330 0x7fbe741a0d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:05:20.699 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:20.694+0000 7fbe79200640 1 -- 172.21.15.12:0/145923564 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbe7410a9b0 msgr2=0x7fbe7419b9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:05:20.699 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:20.694+0000 7fbe79200640 1 --2- 172.21.15.12:0/145923564 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbe7410a9b0 0x7fbe7419b9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:05:20.699 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:20.694+0000 7fbe79200640 1 -- 172.21.15.12:0/145923564 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe741a1550 con 0x7fbe74104fa0 2024-09-11T00:05:20.699 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:20.694+0000 7fbe73e00640 1 --2- 172.21.15.12:0/145923564 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbe7410a9b0 0x7fbe7419b9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:05:20.699 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:20.695+0000 7fbe79200640 1 --2- 172.21.15.12:0/145923564 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbe74104fa0 0x7fbe7419b4a0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fbe6800e9f0 tx=0x7fbe6800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:05:20.699 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:20.695+0000 7fbe71600640 1 -- 172.21.15.12:0/145923564 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe6800cd80 con 0x7fbe74104fa0 2024-09-11T00:05:20.699 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:20.695+0000 7fbe71600640 1 -- 172.21.15.12:0/145923564 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbe680035f0 con 0x7fbe74104fa0 2024-09-11T00:05:20.699 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:20.695+0000 7fbe71600640 1 -- 172.21.15.12:0/145923564 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe68010670 con 0x7fbe74104fa0 2024-09-11T00:05:20.699 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:20.695+0000 7fbe7bc00640 1 -- 172.21.15.12:0/145923564 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbe741a1840 con 0x7fbe74104fa0 2024-09-11T00:05:20.699 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:20.695+0000 7fbe7bc00640 1 -- 172.21.15.12:0/145923564 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbe741a80d0 con 0x7fbe74104fa0 2024-09-11T00:05:20.700 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:20.696+0000 7fbe7bc00640 1 -- 172.21.15.12:0/145923564 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fbe74063860 con 0x7fbe74104fa0 2024-09-11T00:05:20.700 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:20.699+0000 7fbe71600640 1 -- 172.21.15.12:0/145923564 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 17) ==== 153301+0+0 (secure 0 0 0) 0x7fbe68004000 con 0x7fbe74104fa0 2024-09-11T00:05:20.702 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:20.700+0000 7fbe71600640 1 --2- 172.21.15.12:0/145923564 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fbe5c0b53f0 0x7fbe5c0b78b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:05:20.702 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:20.700+0000 7fbe71600640 1 -- 172.21.15.12:0/145923564 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 9092+0+0 (secure 0 0 0) 0x7fbe680e6700 con 0x7fbe74104fa0 2024-09-11T00:05:20.703 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:20.700+0000 7fbe73e00640 1 --2- 172.21.15.12:0/145923564 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fbe5c0b53f0 0x7fbe5c0b78b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:05:20.703 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:20.701+0000 7fbe71600640 1 --2- 172.21.15.12:0/145923564 >> [v2:172.21.15.165:6800/545848456,v1:172.21.15.165:6801/545848456] conn(0x7fbe5c0c04b0 0x7fbe5c0c2950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:05:20.703 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:20.701+0000 7fbe71600640 1 -- 172.21.15.12:0/145923564 --> [v2:172.21.15.165:6800/545848456,v1:172.21.15.165:6801/545848456] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fbe68025ea0 con 0x7fbe5c0c04b0 2024-09-11T00:05:20.703 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:20.701+0000 7fbe71600640 1 -- 172.21.15.12:0/145923564 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_get_version_reply(handle=1 version=83) ==== 24+0+0 (secure 0 0 0) 0x7fbe680e6ab0 con 0x7fbe74104fa0 2024-09-11T00:05:20.704 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:20.701+0000 7fbe79c00640 1 --2- 172.21.15.12:0/145923564 >> [v2:172.21.15.165:6800/545848456,v1:172.21.15.165:6801/545848456] conn(0x7fbe5c0c04b0 0x7fbe5c0c2950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:05:20.704 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:20.701+0000 7fbe73e00640 1 --2- 172.21.15.12:0/145923564 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fbe5c0b53f0 0x7fbe5c0b78b0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fbe741096f0 tx=0x7fbe60030340 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:05:20.704 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:20.702+0000 7fbe79c00640 1 --2- 172.21.15.12:0/145923564 >> [v2:172.21.15.165:6800/545848456,v1:172.21.15.165:6801/545848456] conn(0x7fbe5c0c04b0 0x7fbe5c0c2950 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:05:20.704 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:20.703+0000 7fbe71600640 1 -- 172.21.15.12:0/145923564 <== osd.1 v2:172.21.15.165:6800/545848456 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fbe68025ea0 con 0x7fbe5c0c04b0 2024-09-11T00:05:20.719 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:20.717+0000 7fbe7bc00640 1 -- 172.21.15.12:0/145923564 --> [v2:172.21.15.165:6800/545848456,v1:172.21.15.165:6801/545848456] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fbe74063c60 con 0x7fbe5c0c04b0 2024-09-11T00:05:20.720 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:20.718+0000 7fbe71600640 1 -- 172.21.15.12:0/145923564 <== osd.1 v2:172.21.15.165:6800/545848456 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fbe74063c60 con 0x7fbe5c0c04b0 2024-09-11T00:05:20.721 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:20.718+0000 7fbe7bc00640 1 -- 172.21.15.12:0/145923564 >> [v2:172.21.15.165:6800/545848456,v1:172.21.15.165:6801/545848456] conn(0x7fbe5c0c04b0 msgr2=0x7fbe5c0c2950 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:05:20.721 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:20.718+0000 7fbe7bc00640 1 --2- 172.21.15.12:0/145923564 >> [v2:172.21.15.165:6800/545848456,v1:172.21.15.165:6801/545848456] conn(0x7fbe5c0c04b0 0x7fbe5c0c2950 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:05:20.721 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:20.718+0000 7fbe7bc00640 1 -- 172.21.15.12:0/145923564 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fbe5c0b53f0 msgr2=0x7fbe5c0b78b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:05:20.721 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:20.718+0000 7fbe7bc00640 1 --2- 172.21.15.12:0/145923564 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fbe5c0b53f0 0x7fbe5c0b78b0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fbe741096f0 tx=0x7fbe60030340 comp rx=0 tx=0).stop 2024-09-11T00:05:20.721 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:20.718+0000 7fbe7bc00640 1 -- 172.21.15.12:0/145923564 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbe74104fa0 msgr2=0x7fbe7419b4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:05:20.721 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:20.718+0000 7fbe7bc00640 1 --2- 172.21.15.12:0/145923564 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbe74104fa0 0x7fbe7419b4a0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fbe6800e9f0 tx=0x7fbe6800eeb0 comp rx=0 tx=0).stop 2024-09-11T00:05:20.721 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:20.719+0000 7fbe79c00640 1 -- 172.21.15.12:0/145923564 reap_dead start 2024-09-11T00:05:20.721 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:20.719+0000 7fbe7bc00640 1 -- 172.21.15.12:0/145923564 shutdown_connections 2024-09-11T00:05:20.721 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:20.719+0000 7fbe7bc00640 1 -- 172.21.15.12:0/145923564 >> 172.21.15.12:0/145923564 conn(0x7fbe74100a50 msgr2=0x7fbe74102e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:05:20.721 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:20.719+0000 7fbe7bc00640 1 -- 172.21.15.12:0/145923564 shutdown_connections 2024-09-11T00:05:20.722 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:20.719+0000 7fbe7bc00640 1 -- 172.21.15.12:0/145923564 wait complete. 2024-09-11T00:05:21.062 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:05:20 smithi012 ceph-mon[30333]: osdmap e83: 12 total, 12 up, 12 in 2024-09-11T00:05:21.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:05:20 smithi165 ceph-mon[35356]: osdmap e83: 12 total, 12 up, 12 in 2024-09-11T00:05:21.330 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:05:20 smithi178 ceph-mon[35919]: osdmap e83: 12 total, 12 up, 12 in 2024-09-11T00:05:21.469 INFO:teuthology.orchestra.run.smithi012.stdout:55834574930 2024-09-11T00:05:21.470 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid ff3ad74a-6fcf-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.1 2024-09-11T00:05:21.612 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:05:21.621 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:05:21.731 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:20.587+0000 7f7a72600640 1 -- 172.21.15.12:0/1117007240 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7a6c1018b0 msgr2=0x7f7a6c101c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:05:21.731 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:20.587+0000 7f7a72600640 1 --2- 172.21.15.12:0/1117007240 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7a6c1018b0 0x7f7a6c101c90 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f7a5c009900 tx=0x7f7a5c02a480 comp rx=0 tx=0).stop 2024-09-11T00:05:21.731 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:20.587+0000 7f7a72600640 1 -- 172.21.15.12:0/1117007240 shutdown_connections 2024-09-11T00:05:21.731 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:20.587+0000 7f7a72600640 1 --2- 172.21.15.12:0/1117007240 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7a6c10f460 0x7f7a6c111850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:05:21.731 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:20.587+0000 7f7a72600640 1 --2- 172.21.15.12:0/1117007240 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7a6c1021d0 0x7f7a6c10ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:05:21.731 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:20.587+0000 7f7a72600640 1 --2- 172.21.15.12:0/1117007240 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7a6c1018b0 0x7f7a6c101c90 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:05:21.731 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:20.587+0000 7f7a72600640 1 -- 172.21.15.12:0/1117007240 >> 172.21.15.12:0/1117007240 conn(0x7f7a6c0fd700 msgr2=0x7f7a6c0ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:05:21.732 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:20.587+0000 7f7a72600640 1 -- 172.21.15.12:0/1117007240 shutdown_connections 2024-09-11T00:05:21.732 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:20.587+0000 7f7a72600640 1 -- 172.21.15.12:0/1117007240 wait complete. 2024-09-11T00:05:21.732 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:20.588+0000 7f7a72600640 1 Processor -- start 2024-09-11T00:05:21.732 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:20.588+0000 7f7a72600640 1 -- start start 2024-09-11T00:05:21.732 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:20.589+0000 7f7a72600640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7a6c1018b0 0x7f7a6c19b4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:05:21.732 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:20.589+0000 7f7a72600640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7a6c1021d0 0x7f7a6c19ba30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:05:21.732 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:20.589+0000 7f7a72600640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7a6c10f460 0x7f7a6c1a0dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:05:21.732 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:20.589+0000 7f7a72600640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f7a6c114f80 con 0x7f7a6c1021d0 2024-09-11T00:05:21.732 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:20.589+0000 7f7a72600640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f7a6c114e00 con 0x7f7a6c1018b0 2024-09-11T00:05:21.732 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:20.589+0000 7f7a72600640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f7a6c115100 con 0x7f7a6c10f460 2024-09-11T00:05:21.732 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:20.589+0000 7f7a6be00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7a6c10f460 0x7f7a6c1a0dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:05:21.732 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:20.589+0000 7f7a6aa00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7a6c1021d0 0x7f7a6c19ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:05:21.733 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:20.589+0000 7f7a6be00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7a6c10f460 0x7f7a6c1a0dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.12:42950/0 (socket says 172.21.15.12:42950) 2024-09-11T00:05:21.733 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:20.589+0000 7f7a6b400640 1 --2- 172.21.15.12:0/2636226148 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7a6c1018b0 0x7f7a6c19b4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:05:21.733 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:20.589+0000 7f7a6be00640 1 -- 172.21.15.12:0/2636226148 learned_addr learned my addr 172.21.15.12:0/2636226148 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:05:21.733 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:20.590+0000 7f7a6be00640 1 -- 172.21.15.12:0/2636226148 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7a6c1018b0 msgr2=0x7f7a6c19b4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:05:21.733 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:20.590+0000 7f7a6be00640 1 --2- 172.21.15.12:0/2636226148 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7a6c1018b0 0x7f7a6c19b4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:05:21.733 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:20.590+0000 7f7a6be00640 1 -- 172.21.15.12:0/2636226148 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7a6c1021d0 msgr2=0x7f7a6c19ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:05:21.733 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:20.590+0000 7f7a6be00640 1 --2- 172.21.15.12:0/2636226148 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7a6c1021d0 0x7f7a6c19ba30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:05:21.733 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:20.590+0000 7f7a6be00640 1 -- 172.21.15.12:0/2636226148 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a6c1a15a0 con 0x7f7a6c10f460 2024-09-11T00:05:21.733 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:20.590+0000 7f7a6be00640 1 --2- 172.21.15.12:0/2636226148 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7a6c10f460 0x7f7a6c1a0dd0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f7a6400eb40 tx=0x7f7a6400c2d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:05:21.733 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:20.590+0000 7f7a63e00640 1 -- 172.21.15.12:0/2636226148 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a64004b20 con 0x7f7a6c10f460 2024-09-11T00:05:21.733 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:20.590+0000 7f7a72600640 1 -- 172.21.15.12:0/2636226148 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7a6c1a1890 con 0x7f7a6c10f460 2024-09-11T00:05:21.733 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:20.590+0000 7f7a63e00640 1 -- 172.21.15.12:0/2636226148 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7a64004cc0 con 0x7f7a6c10f460 2024-09-11T00:05:21.734 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:20.590+0000 7f7a63e00640 1 -- 172.21.15.12:0/2636226148 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a64010720 con 0x7f7a6c10f460 2024-09-11T00:05:21.734 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:20.591+0000 7f7a72600640 1 -- 172.21.15.12:0/2636226148 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7a6c1a7f50 con 0x7f7a6c10f460 2024-09-11T00:05:21.734 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:20.591+0000 7f7a72600640 1 -- 172.21.15.12:0/2636226148 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f7a6c110c60 con 0x7f7a6c10f460 2024-09-11T00:05:21.734 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:20.594+0000 7f7a63e00640 1 -- 172.21.15.12:0/2636226148 <== mon.1 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 17) ==== 153301+0+0 (secure 0 0 0) 0x7f7a64005560 con 0x7f7a6c10f460 2024-09-11T00:05:21.734 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:20.595+0000 7f7a63e00640 1 --2- 172.21.15.12:0/2636226148 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f7a500b5370 0x7f7a500b7830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:05:21.734 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:20.595+0000 7f7a63e00640 1 -- 172.21.15.12:0/2636226148 <== mon.1 v2:172.21.15.178:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 9092+0+0 (secure 0 0 0) 0x7f7a640e6a10 con 0x7f7a6c10f460 2024-09-11T00:05:21.734 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:20.595+0000 7f7a6b400640 1 --2- 172.21.15.12:0/2636226148 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f7a500b5370 0x7f7a500b7830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:05:21.734 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:20.596+0000 7f7a63e00640 1 --2- 172.21.15.12:0/2636226148 >> [v2:172.21.15.12:6802/1237101991,v1:172.21.15.12:6803/1237101991] conn(0x7f7a500c0430 0x7f7a500c28d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:05:21.734 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:20.596+0000 7f7a63e00640 1 -- 172.21.15.12:0/2636226148 --> [v2:172.21.15.12:6802/1237101991,v1:172.21.15.12:6803/1237101991] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f7a500c2fa0 con 0x7f7a500c0430 2024-09-11T00:05:21.734 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:20.596+0000 7f7a63e00640 1 -- 172.21.15.12:0/2636226148 <== mon.1 v2:172.21.15.178:3300/0 6 ==== mon_get_version_reply(handle=1 version=83) ==== 24+0+0 (secure 0 0 0) 0x7f7a640e6dc0 con 0x7f7a6c10f460 2024-09-11T00:05:21.734 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:20.596+0000 7f7a6aa00640 1 --2- 172.21.15.12:0/2636226148 >> [v2:172.21.15.12:6802/1237101991,v1:172.21.15.12:6803/1237101991] conn(0x7f7a500c0430 0x7f7a500c28d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:05:21.734 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:20.596+0000 7f7a6b400640 1 --2- 172.21.15.12:0/2636226148 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f7a500b5370 0x7f7a500b7830 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f7a5c031440 tx=0x7f7a5c005b50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:05:21.735 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:20.597+0000 7f7a6aa00640 1 --2- 172.21.15.12:0/2636226148 >> [v2:172.21.15.12:6802/1237101991,v1:172.21.15.12:6803/1237101991] conn(0x7f7a500c0430 0x7f7a500c28d0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:05:21.735 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:20.597+0000 7f7a63e00640 1 -- 172.21.15.12:0/2636226148 <== osd.0 v2:172.21.15.12:6802/1237101991 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f7a500c2fa0 con 0x7f7a500c0430 2024-09-11T00:05:21.735 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:20.612+0000 7f7a72600640 1 -- 172.21.15.12:0/2636226148 --> [v2:172.21.15.12:6802/1237101991,v1:172.21.15.12:6803/1237101991] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f7a6c06bb30 con 0x7f7a500c0430 2024-09-11T00:05:21.735 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:20.612+0000 7f7a63e00640 1 -- 172.21.15.12:0/2636226148 <== osd.0 v2:172.21.15.12:6802/1237101991 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f7a6c06bb30 con 0x7f7a500c0430 2024-09-11T00:05:21.735 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:20.613+0000 7f7a72600640 1 -- 172.21.15.12:0/2636226148 >> [v2:172.21.15.12:6802/1237101991,v1:172.21.15.12:6803/1237101991] conn(0x7f7a500c0430 msgr2=0x7f7a500c28d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:05:21.735 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:20.613+0000 7f7a72600640 1 --2- 172.21.15.12:0/2636226148 >> [v2:172.21.15.12:6802/1237101991,v1:172.21.15.12:6803/1237101991] conn(0x7f7a500c0430 0x7f7a500c28d0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:05:21.735 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:20.613+0000 7f7a72600640 1 -- 172.21.15.12:0/2636226148 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f7a500b5370 msgr2=0x7f7a500b7830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:05:21.736 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:20.613+0000 7f7a72600640 1 --2- 172.21.15.12:0/2636226148 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f7a500b5370 0x7f7a500b7830 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f7a5c031440 tx=0x7f7a5c005b50 comp rx=0 tx=0).stop 2024-09-11T00:05:21.736 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:20.613+0000 7f7a72600640 1 -- 172.21.15.12:0/2636226148 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7a6c10f460 msgr2=0x7f7a6c1a0dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:05:21.736 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:20.613+0000 7f7a72600640 1 --2- 172.21.15.12:0/2636226148 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7a6c10f460 0x7f7a6c1a0dd0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f7a6400eb40 tx=0x7f7a6400c2d0 comp rx=0 tx=0).stop 2024-09-11T00:05:21.736 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:20.613+0000 7f7a6be00640 1 -- 172.21.15.12:0/2636226148 reap_dead start 2024-09-11T00:05:21.736 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:20.613+0000 7f7a72600640 1 -- 172.21.15.12:0/2636226148 shutdown_connections 2024-09-11T00:05:21.736 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:20.613+0000 7f7a72600640 1 -- 172.21.15.12:0/2636226148 >> 172.21.15.12:0/2636226148 conn(0x7f7a6c0fd700 msgr2=0x7f7a6c0ffaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:05:21.736 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:20.613+0000 7f7a72600640 1 -- 172.21.15.12:0/2636226148 shutdown_connections 2024-09-11T00:05:21.736 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:20.613+0000 7f7a72600640 1 -- 172.21.15.12:0/2636226148 wait complete. 2024-09-11T00:05:21.987 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:05:21 smithi012 ceph-mon[30333]: pgmap v344: 129 pgs: 1 activating+degraded, 7 peering, 121 active+clean; 645 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 639 B/s wr, 9 op/s; 2/308 objects degraded (0.649%) 2024-09-11T00:05:21.987 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:05:21 smithi012 ceph-mon[30333]: Health check failed: Reduced data availability: 1 pg inactive, 5 pgs peering (PG_AVAILABILITY) 2024-09-11T00:05:21.987 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:05:21 smithi012 ceph-mon[30333]: Health check failed: Degraded data redundancy: 2/308 objects degraded (0.649%), 1 pg degraded (PG_DEGRADED) 2024-09-11T00:05:22.096 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:22.093+0000 7f0490a00640 1 -- 172.21.15.12:0/1159519241 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f048c108fa0 msgr2=0x7f048c10f7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:05:22.096 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:22.093+0000 7f0490a00640 1 --2- 172.21.15.12:0/1159519241 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f048c108fa0 0x7f048c10f7a0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f0484009be0 tx=0x7f048402a6c0 comp rx=0 tx=0).stop 2024-09-11T00:05:22.096 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:22.094+0000 7f0490a00640 1 -- 172.21.15.12:0/1159519241 shutdown_connections 2024-09-11T00:05:22.096 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:22.094+0000 7f0490a00640 1 --2- 172.21.15.12:0/1159519241 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f048c108fa0 0x7f048c10f7a0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:05:22.096 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:22.094+0000 7f0490a00640 1 --2- 172.21.15.12:0/1159519241 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f048c108620 0x7f048c108a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:05:22.097 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:22.094+0000 7f0490a00640 1 --2- 172.21.15.12:0/1159519241 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f048c102c10 0x7f048c102ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:05:22.097 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:22.094+0000 7f0490a00640 1 -- 172.21.15.12:0/1159519241 >> 172.21.15.12:0/1159519241 conn(0x7f048c0fe660 msgr2=0x7f048c100a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:05:22.097 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:22.094+0000 7f0490a00640 1 -- 172.21.15.12:0/1159519241 shutdown_connections 2024-09-11T00:05:22.097 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:22.094+0000 7f0490a00640 1 -- 172.21.15.12:0/1159519241 wait complete. 2024-09-11T00:05:22.097 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:22.095+0000 7f0490a00640 1 Processor -- start 2024-09-11T00:05:22.097 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:22.096+0000 7f0490a00640 1 -- start start 2024-09-11T00:05:22.098 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:22.096+0000 7f0490a00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f048c102c10 0x7f048c198fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:05:22.098 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:22.096+0000 7f0490a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f048c108620 0x7f048c199510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:05:22.098 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:22.096+0000 7f0490a00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f048c108fa0 0x7f048c19e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:05:22.098 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:22.096+0000 7f0490a00640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f048c112c60 con 0x7f048c108fa0 2024-09-11T00:05:22.098 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:22.096+0000 7f0490a00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f048c112ae0 con 0x7f048c108620 2024-09-11T00:05:22.098 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:22.096+0000 7f0490a00640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f048c112de0 con 0x7f048c102c10 2024-09-11T00:05:22.099 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:22.096+0000 7f0489e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f048c108620 0x7f048c199510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:05:22.099 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:22.096+0000 7f048a800640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f048c102c10 0x7f048c198fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:05:22.099 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:22.096+0000 7f0489e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f048c108620 0x7f048c199510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.12:36826/0 (socket says 172.21.15.12:36826) 2024-09-11T00:05:22.099 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:22.096+0000 7f0489e00640 1 -- 172.21.15.12:0/4051651996 learned_addr learned my addr 172.21.15.12:0/4051651996 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:05:22.099 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:22.097+0000 7f048b200640 1 --2- 172.21.15.12:0/4051651996 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f048c108fa0 0x7f048c19e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:05:22.099 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:22.097+0000 7f048a800640 1 -- 172.21.15.12:0/4051651996 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f048c108620 msgr2=0x7f048c199510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:05:22.099 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:22.097+0000 7f048a800640 1 --2- 172.21.15.12:0/4051651996 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f048c108620 0x7f048c199510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:05:22.099 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:22.097+0000 7f048a800640 1 -- 172.21.15.12:0/4051651996 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f048c108fa0 msgr2=0x7f048c19e8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:05:22.099 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:22.097+0000 7f048a800640 1 --2- 172.21.15.12:0/4051651996 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f048c108fa0 0x7f048c19e8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:05:22.100 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:22.097+0000 7f048a800640 1 -- 172.21.15.12:0/4051651996 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f048c19f080 con 0x7f048c102c10 2024-09-11T00:05:22.100 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:22.097+0000 7f048a800640 1 --2- 172.21.15.12:0/4051651996 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f048c102c10 0x7f048c198fd0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f047c00e9f0 tx=0x7f047c00eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:05:22.100 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:22.098+0000 7f0483400640 1 -- 172.21.15.12:0/4051651996 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f047c00ccc0 con 0x7f048c102c10 2024-09-11T00:05:22.100 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:22.098+0000 7f0490a00640 1 -- 172.21.15.12:0/4051651996 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f048c19f370 con 0x7f048c102c10 2024-09-11T00:05:22.100 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:22.098+0000 7f0490a00640 1 -- 172.21.15.12:0/4051651996 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f048c075380 con 0x7f048c102c10 2024-09-11T00:05:22.100 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:22.098+0000 7f0483400640 1 -- 172.21.15.12:0/4051651996 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f047c0044d0 con 0x7f048c102c10 2024-09-11T00:05:22.101 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:22.098+0000 7f0483400640 1 -- 172.21.15.12:0/4051651996 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f047c01c070 con 0x7f048c102c10 2024-09-11T00:05:22.101 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:22.099+0000 7f0490a00640 1 -- 172.21.15.12:0/4051651996 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f046c001060 con 0x7f048c102c10 2024-09-11T00:05:22.103 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:22.102+0000 7f0483400640 1 -- 172.21.15.12:0/4051651996 <== mon.1 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 17) ==== 153301+0+0 (secure 0 0 0) 0x7f047c005020 con 0x7f048c102c10 2024-09-11T00:05:22.106 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:22.103+0000 7f0483400640 1 --2- 172.21.15.12:0/4051651996 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f04700b5490 0x7f04700b7950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:05:22.106 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:22.103+0000 7f0483400640 1 -- 172.21.15.12:0/4051651996 <== mon.1 v2:172.21.15.178:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 9092+0+0 (secure 0 0 0) 0x7f047c0e5fe0 con 0x7f048c102c10 2024-09-11T00:05:22.106 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:22.104+0000 7f0489e00640 1 --2- 172.21.15.12:0/4051651996 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f04700b5490 0x7f04700b7950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:05:22.106 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:22.104+0000 7f0483400640 1 --2- 172.21.15.12:0/4051651996 >> [v2:172.21.15.178:6800/4131162695,v1:172.21.15.178:6801/4131162695] conn(0x7f04700c0550 0x7f04700c29f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:05:22.106 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:22.104+0000 7f0483400640 1 -- 172.21.15.12:0/4051651996 --> [v2:172.21.15.178:6800/4131162695,v1:172.21.15.178:6801/4131162695] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f04700c30c0 con 0x7f04700c0550 2024-09-11T00:05:22.107 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:22.104+0000 7f0483400640 1 -- 172.21.15.12:0/4051651996 <== mon.1 v2:172.21.15.178:3300/0 6 ==== mon_get_version_reply(handle=1 version=83) ==== 24+0+0 (secure 0 0 0) 0x7f047c0e63f0 con 0x7f048c102c10 2024-09-11T00:05:22.107 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:22.104+0000 7f048b200640 1 --2- 172.21.15.12:0/4051651996 >> [v2:172.21.15.178:6800/4131162695,v1:172.21.15.178:6801/4131162695] conn(0x7f04700c0550 0x7f04700c29f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:05:22.108 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:22.104+0000 7f0489e00640 1 --2- 172.21.15.12:0/4051651996 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f04700b5490 0x7f04700b7950 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f048c108150 tx=0x7f0478009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:05:22.108 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:22.105+0000 7f048b200640 1 --2- 172.21.15.12:0/4051651996 >> [v2:172.21.15.178:6800/4131162695,v1:172.21.15.178:6801/4131162695] conn(0x7f04700c0550 0x7f04700c29f0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:05:22.108 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:22.106+0000 7f0483400640 1 -- 172.21.15.12:0/4051651996 <== osd.2 v2:172.21.15.178:6800/4131162695 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f04700c30c0 con 0x7f04700c0550 2024-09-11T00:05:22.123 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:22.121+0000 7f0490a00640 1 -- 172.21.15.12:0/4051651996 --> [v2:172.21.15.178:6800/4131162695,v1:172.21.15.178:6801/4131162695] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f046c002e00 con 0x7f04700c0550 2024-09-11T00:05:22.124 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:22.122+0000 7f0483400640 1 -- 172.21.15.12:0/4051651996 <== osd.2 v2:172.21.15.178:6800/4131162695 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f046c002e00 con 0x7f04700c0550 2024-09-11T00:05:22.126 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:22.122+0000 7f0490a00640 1 -- 172.21.15.12:0/4051651996 >> [v2:172.21.15.178:6800/4131162695,v1:172.21.15.178:6801/4131162695] conn(0x7f04700c0550 msgr2=0x7f04700c29f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:05:22.126 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:22.122+0000 7f0490a00640 1 --2- 172.21.15.12:0/4051651996 >> [v2:172.21.15.178:6800/4131162695,v1:172.21.15.178:6801/4131162695] conn(0x7f04700c0550 0x7f04700c29f0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:05:22.126 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:22.123+0000 7f0490a00640 1 -- 172.21.15.12:0/4051651996 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f04700b5490 msgr2=0x7f04700b7950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:05:22.127 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:22.123+0000 7f0490a00640 1 --2- 172.21.15.12:0/4051651996 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f04700b5490 0x7f04700b7950 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f048c108150 tx=0x7f0478009260 comp rx=0 tx=0).stop 2024-09-11T00:05:22.127 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:22.123+0000 7f0490a00640 1 -- 172.21.15.12:0/4051651996 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f048c102c10 msgr2=0x7f048c198fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:05:22.127 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:22.123+0000 7f0490a00640 1 --2- 172.21.15.12:0/4051651996 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f048c102c10 0x7f048c198fd0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f047c00e9f0 tx=0x7f047c00eeb0 comp rx=0 tx=0).stop 2024-09-11T00:05:22.127 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:22.123+0000 7f048b200640 1 -- 172.21.15.12:0/4051651996 reap_dead start 2024-09-11T00:05:22.127 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:22.123+0000 7f0490a00640 1 -- 172.21.15.12:0/4051651996 shutdown_connections 2024-09-11T00:05:22.127 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:22.123+0000 7f0490a00640 1 -- 172.21.15.12:0/4051651996 >> 172.21.15.12:0/4051651996 conn(0x7f048c0fe660 msgr2=0x7f048c100a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:05:22.128 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:22.123+0000 7f0490a00640 1 -- 172.21.15.12:0/4051651996 shutdown_connections 2024-09-11T00:05:22.128 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:22.123+0000 7f0490a00640 1 -- 172.21.15.12:0/4051651996 wait complete. 2024-09-11T00:05:22.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:05:21 smithi165 ceph-mon[35356]: pgmap v344: 129 pgs: 1 activating+degraded, 7 peering, 121 active+clean; 645 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 639 B/s wr, 9 op/s; 2/308 objects degraded (0.649%) 2024-09-11T00:05:22.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:05:21 smithi165 ceph-mon[35356]: Health check failed: Reduced data availability: 1 pg inactive, 5 pgs peering (PG_AVAILABILITY) 2024-09-11T00:05:22.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:05:21 smithi165 ceph-mon[35356]: Health check failed: Degraded data redundancy: 2/308 objects degraded (0.649%), 1 pg degraded (PG_DEGRADED) 2024-09-11T00:05:22.324 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:22.317+0000 7f1b86c00640 1 -- 172.21.15.12:0/1500306653 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1b80104fa0 msgr2=0x7f1b80105380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:05:22.324 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:22.317+0000 7f1b86c00640 1 --2- 172.21.15.12:0/1500306653 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1b80104fa0 0x7f1b80105380 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f1b70009900 tx=0x7f1b7002a3f0 comp rx=0 tx=0).stop 2024-09-11T00:05:22.324 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:22.317+0000 7f1b86c00640 1 -- 172.21.15.12:0/1500306653 shutdown_connections 2024-09-11T00:05:22.324 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:22.317+0000 7f1b86c00640 1 --2- 172.21.15.12:0/1500306653 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1b8010b330 0x7f1b80111b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:05:22.324 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:22.317+0000 7f1b86c00640 1 --2- 172.21.15.12:0/1500306653 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1b8010a9b0 0x7f1b8010adf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:05:22.324 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:22.317+0000 7f1b86c00640 1 --2- 172.21.15.12:0/1500306653 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1b80104fa0 0x7f1b80105380 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:05:22.324 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:22.317+0000 7f1b86c00640 1 -- 172.21.15.12:0/1500306653 >> 172.21.15.12:0/1500306653 conn(0x7f1b80100a50 msgr2=0x7f1b80102e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:05:22.324 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:22.318+0000 7f1b86c00640 1 -- 172.21.15.12:0/1500306653 shutdown_connections 2024-09-11T00:05:22.325 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:22.318+0000 7f1b86c00640 1 -- 172.21.15.12:0/1500306653 wait complete. 2024-09-11T00:05:22.325 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:22.318+0000 7f1b86c00640 1 Processor -- start 2024-09-11T00:05:22.325 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:22.319+0000 7f1b86c00640 1 -- start start 2024-09-11T00:05:22.325 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:22.319+0000 7f1b86c00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1b80104fa0 0x7f1b8019b470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:05:22.325 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:22.319+0000 7f1b86c00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1b8010a9b0 0x7f1b8019b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:05:22.325 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:22.319+0000 7f1b86c00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1b8010b330 0x7f1b801a0d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:05:22.325 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:22.319+0000 7f1b86c00640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f1b80114f40 con 0x7f1b80104fa0 2024-09-11T00:05:22.325 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:22.319+0000 7f1b86c00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f1b80114dc0 con 0x7f1b8010b330 2024-09-11T00:05:22.325 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:22.319+0000 7f1b86c00640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f1b801150c0 con 0x7f1b8010a9b0 2024-09-11T00:05:22.325 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:22.319+0000 7f1b7fe00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1b80104fa0 0x7f1b8019b470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:05:22.325 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:22.319+0000 7f1b7fe00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1b80104fa0 0x7f1b8019b470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:41252/0 (socket says 172.21.15.12:41252) 2024-09-11T00:05:22.326 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:22.319+0000 7f1b7fe00640 1 -- 172.21.15.12:0/3574703329 learned_addr learned my addr 172.21.15.12:0/3574703329 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:05:22.326 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:22.319+0000 7f1b77e00640 1 --2- 172.21.15.12:0/3574703329 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1b8010a9b0 0x7f1b8019b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:05:22.326 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:22.320+0000 7f1b84c00640 1 --2- 172.21.15.12:0/3574703329 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1b8010b330 0x7f1b801a0d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:05:22.326 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:22.320+0000 7f1b7fe00640 1 -- 172.21.15.12:0/3574703329 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1b8010a9b0 msgr2=0x7f1b8019b9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:05:22.326 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:22.320+0000 7f1b7fe00640 1 --2- 172.21.15.12:0/3574703329 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1b8010a9b0 0x7f1b8019b9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:05:22.326 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:22.320+0000 7f1b7fe00640 1 -- 172.21.15.12:0/3574703329 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1b8010b330 msgr2=0x7f1b801a0d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:05:22.326 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:22.320+0000 7f1b7fe00640 1 --2- 172.21.15.12:0/3574703329 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1b8010b330 0x7f1b801a0d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:05:22.326 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:22.320+0000 7f1b7fe00640 1 -- 172.21.15.12:0/3574703329 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b801a1520 con 0x7f1b80104fa0 2024-09-11T00:05:22.326 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:22.320+0000 7f1b77e00640 1 --2- 172.21.15.12:0/3574703329 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1b8010a9b0 0x7f1b8019b9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:05:22.326 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:22.320+0000 7f1b84c00640 1 --2- 172.21.15.12:0/3574703329 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1b8010b330 0x7f1b801a0d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T00:05:22.326 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:22.320+0000 7f1b7fe00640 1 --2- 172.21.15.12:0/3574703329 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1b80104fa0 0x7f1b8019b470 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f1b70005940 tx=0x7f1b70004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:05:22.326 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:22.321+0000 7f1b7d600640 1 -- 172.21.15.12:0/3574703329 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b7003f030 con 0x7f1b80104fa0 2024-09-11T00:05:22.326 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:22.321+0000 7f1b86c00640 1 -- 172.21.15.12:0/3574703329 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1b801a17b0 con 0x7f1b80104fa0 2024-09-11T00:05:22.326 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:22.321+0000 7f1b7d600640 1 -- 172.21.15.12:0/3574703329 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1b70003aa0 con 0x7f1b80104fa0 2024-09-11T00:05:22.327 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:22.321+0000 7f1b7d600640 1 -- 172.21.15.12:0/3574703329 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b70035070 con 0x7f1b80104fa0 2024-09-11T00:05:22.327 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:22.321+0000 7f1b86c00640 1 -- 172.21.15.12:0/3574703329 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1b801a7ed0 con 0x7f1b80104fa0 2024-09-11T00:05:22.327 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:22.322+0000 7f1b86c00640 1 -- 172.21.15.12:0/3574703329 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f1b800638a0 con 0x7f1b80104fa0 2024-09-11T00:05:22.327 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:22.324+0000 7f1b7d600640 1 -- 172.21.15.12:0/3574703329 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 17) ==== 153301+0+0 (secure 0 0 0) 0x7f1b7003bcc0 con 0x7f1b80104fa0 2024-09-11T00:05:22.328 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:22.326+0000 7f1b7d600640 1 --2- 172.21.15.12:0/3574703329 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f1b640b53f0 0x7f1b640b78b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:05:22.328 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:22.326+0000 7f1b7d600640 1 -- 172.21.15.12:0/3574703329 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 9092+0+0 (secure 0 0 0) 0x7f1b70107160 con 0x7f1b80104fa0 2024-09-11T00:05:22.328 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:22.326+0000 7f1b77e00640 1 --2- 172.21.15.12:0/3574703329 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f1b640b53f0 0x7f1b640b78b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:05:22.328 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:22.326+0000 7f1b7d600640 1 --2- 172.21.15.12:0/3574703329 >> [v2:172.21.15.165:6808/1899879280,v1:172.21.15.165:6809/1899879280] conn(0x7f1b640c04b0 0x7f1b640c2950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:05:22.329 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:22.326+0000 7f1b7d600640 1 -- 172.21.15.12:0/3574703329 --> [v2:172.21.15.165:6808/1899879280,v1:172.21.15.165:6809/1899879280] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f1b640c3020 con 0x7f1b640c04b0 2024-09-11T00:05:22.329 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:22.326+0000 7f1b7d600640 1 -- 172.21.15.12:0/3574703329 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_get_version_reply(handle=1 version=83) ==== 24+0+0 (secure 0 0 0) 0x7f1b70107510 con 0x7f1b80104fa0 2024-09-11T00:05:22.329 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:22.327+0000 7f1b84c00640 1 --2- 172.21.15.12:0/3574703329 >> [v2:172.21.15.165:6808/1899879280,v1:172.21.15.165:6809/1899879280] conn(0x7f1b640c04b0 0x7f1b640c2950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:05:22.329 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:22.327+0000 7f1b77e00640 1 --2- 172.21.15.12:0/3574703329 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f1b640b53f0 0x7f1b640b78b0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f1b8010a150 tx=0x7f1b6800b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:05:22.329 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:22.327+0000 7f1b84c00640 1 --2- 172.21.15.12:0/3574703329 >> [v2:172.21.15.165:6808/1899879280,v1:172.21.15.165:6809/1899879280] conn(0x7f1b640c04b0 0x7f1b640c2950 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:05:22.330 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:22.328+0000 7f1b7d600640 1 -- 172.21.15.12:0/3574703329 <== osd.4 v2:172.21.15.165:6808/1899879280 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f1b640c3020 con 0x7f1b640c04b0 2024-09-11T00:05:22.330 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:05:21 smithi178 ceph-mon[35919]: pgmap v344: 129 pgs: 1 activating+degraded, 7 peering, 121 active+clean; 645 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 639 B/s wr, 9 op/s; 2/308 objects degraded (0.649%) 2024-09-11T00:05:22.330 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:05:21 smithi178 ceph-mon[35919]: Health check failed: Reduced data availability: 1 pg inactive, 5 pgs peering (PG_AVAILABILITY) 2024-09-11T00:05:22.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:05:21 smithi178 ceph-mon[35919]: Health check failed: Degraded data redundancy: 2/308 objects degraded (0.649%), 1 pg degraded (PG_DEGRADED) 2024-09-11T00:05:22.350 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:22.348+0000 7f1b86c00640 1 -- 172.21.15.12:0/3574703329 --> [v2:172.21.15.165:6808/1899879280,v1:172.21.15.165:6809/1899879280] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f1b80063ca0 con 0x7f1b640c04b0 2024-09-11T00:05:22.351 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:22.349+0000 7f1b7d600640 1 -- 172.21.15.12:0/3574703329 <== osd.4 v2:172.21.15.165:6808/1899879280 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f1b80063ca0 con 0x7f1b640c04b0 2024-09-11T00:05:22.352 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:22.349+0000 7f1b86c00640 1 -- 172.21.15.12:0/3574703329 >> [v2:172.21.15.165:6808/1899879280,v1:172.21.15.165:6809/1899879280] conn(0x7f1b640c04b0 msgr2=0x7f1b640c2950 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:05:22.352 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:22.349+0000 7f1b86c00640 1 --2- 172.21.15.12:0/3574703329 >> [v2:172.21.15.165:6808/1899879280,v1:172.21.15.165:6809/1899879280] conn(0x7f1b640c04b0 0x7f1b640c2950 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:05:22.352 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:22.350+0000 7f1b86c00640 1 -- 172.21.15.12:0/3574703329 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f1b640b53f0 msgr2=0x7f1b640b78b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:05:22.352 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:22.350+0000 7f1b86c00640 1 --2- 172.21.15.12:0/3574703329 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f1b640b53f0 0x7f1b640b78b0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f1b8010a150 tx=0x7f1b6800b390 comp rx=0 tx=0).stop 2024-09-11T00:05:22.352 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:22.350+0000 7f1b86c00640 1 -- 172.21.15.12:0/3574703329 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1b80104fa0 msgr2=0x7f1b8019b470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:05:22.352 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:22.350+0000 7f1b86c00640 1 --2- 172.21.15.12:0/3574703329 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1b80104fa0 0x7f1b8019b470 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f1b70005940 tx=0x7f1b70004270 comp rx=0 tx=0).stop 2024-09-11T00:05:22.352 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:22.350+0000 7f1b84c00640 1 -- 172.21.15.12:0/3574703329 reap_dead start 2024-09-11T00:05:22.352 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:22.350+0000 7f1b86c00640 1 -- 172.21.15.12:0/3574703329 shutdown_connections 2024-09-11T00:05:22.352 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:22.350+0000 7f1b86c00640 1 -- 172.21.15.12:0/3574703329 >> 172.21.15.12:0/3574703329 conn(0x7f1b80100a50 msgr2=0x7f1b801013b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:05:22.353 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:22.350+0000 7f1b86c00640 1 -- 172.21.15.12:0/3574703329 shutdown_connections 2024-09-11T00:05:22.353 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:22.350+0000 7f1b86c00640 1 -- 172.21.15.12:0/3574703329 wait complete. 2024-09-11T00:05:23.009 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:23.007+0000 7fb504800640 1 -- 172.21.15.12:0/92767415 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb4fc102dc0 msgr2=0x7fb4fc1031a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:05:23.009 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:23.007+0000 7fb504800640 1 --2- 172.21.15.12:0/92767415 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb4fc102dc0 0x7fb4fc1031a0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fb4f0009900 tx=0x7fb4f002a410 comp rx=0 tx=0).stop 2024-09-11T00:05:23.009 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:23.007+0000 7fb504800640 1 -- 172.21.15.12:0/92767415 shutdown_connections 2024-09-11T00:05:23.009 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:23.007+0000 7fb504800640 1 --2- 172.21.15.12:0/92767415 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb4fc109150 0x7fb4fc10f950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:05:23.009 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:23.007+0000 7fb504800640 1 --2- 172.21.15.12:0/92767415 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb4fc1087d0 0x7fb4fc108c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:05:23.009 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:23.007+0000 7fb504800640 1 --2- 172.21.15.12:0/92767415 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb4fc102dc0 0x7fb4fc1031a0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:05:23.010 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:23.007+0000 7fb504800640 1 -- 172.21.15.12:0/92767415 >> 172.21.15.12:0/92767415 conn(0x7fb4fc0fe830 msgr2=0x7fb4fc100c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:05:23.010 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:23.007+0000 7fb504800640 1 -- 172.21.15.12:0/92767415 shutdown_connections 2024-09-11T00:05:23.010 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:23.007+0000 7fb504800640 1 -- 172.21.15.12:0/92767415 wait complete. 2024-09-11T00:05:23.010 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:23.008+0000 7fb504800640 1 Processor -- start 2024-09-11T00:05:23.011 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:23.008+0000 7fb504800640 1 -- start start 2024-09-11T00:05:23.011 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:23.008+0000 7fb504800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb4fc102dc0 0x7fb4fc19b560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:05:23.011 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:23.008+0000 7fb504800640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb4fc1087d0 0x7fb4fc19baa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:05:23.011 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:23.008+0000 7fb504800640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb4fc109150 0x7fb4fc1a0e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:05:23.011 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:23.008+0000 7fb504800640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fb4fc112e10 con 0x7fb4fc109150 2024-09-11T00:05:23.011 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:23.008+0000 7fb504800640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fb4fc112c90 con 0x7fb4fc102dc0 2024-09-11T00:05:23.011 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:23.008+0000 7fb504800640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fb4fc112f90 con 0x7fb4fc1087d0 2024-09-11T00:05:23.011 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:23.009+0000 7fb502800640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb4fc109150 0x7fb4fc1a0e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:05:23.011 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:23.009+0000 7fb501e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb4fc102dc0 0x7fb4fc19b560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:05:23.011 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:23.009+0000 7fb502800640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb4fc109150 0x7fb4fc1a0e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:41278/0 (socket says 172.21.15.12:41278) 2024-09-11T00:05:23.011 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:23.009+0000 7fb502800640 1 -- 172.21.15.12:0/1551017380 learned_addr learned my addr 172.21.15.12:0/1551017380 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:05:23.011 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:23.009+0000 7fb501400640 1 --2- 172.21.15.12:0/1551017380 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb4fc1087d0 0x7fb4fc19baa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:05:23.012 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:23.009+0000 7fb502800640 1 -- 172.21.15.12:0/1551017380 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb4fc1087d0 msgr2=0x7fb4fc19baa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:05:23.012 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:23.009+0000 7fb502800640 1 --2- 172.21.15.12:0/1551017380 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb4fc1087d0 0x7fb4fc19baa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:05:23.012 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:23.009+0000 7fb502800640 1 -- 172.21.15.12:0/1551017380 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb4fc102dc0 msgr2=0x7fb4fc19b560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:05:23.012 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:23.009+0000 7fb502800640 1 --2- 172.21.15.12:0/1551017380 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb4fc102dc0 0x7fb4fc19b560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:05:23.013 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:23.009+0000 7fb502800640 1 -- 172.21.15.12:0/1551017380 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4fc1a1610 con 0x7fb4fc109150 2024-09-11T00:05:23.013 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:23.009+0000 7fb501e00640 1 --2- 172.21.15.12:0/1551017380 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb4fc102dc0 0x7fb4fc19b560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:05:23.013 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:23.010+0000 7fb502800640 1 --2- 172.21.15.12:0/1551017380 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb4fc109150 0x7fb4fc1a0e40 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fb4f800eb10 tx=0x7fb4f800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:05:23.013 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:23.010+0000 7fb4f6a00640 1 -- 172.21.15.12:0/1551017380 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4f8004b20 con 0x7fb4fc109150 2024-09-11T00:05:23.013 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:23.010+0000 7fb504800640 1 -- 172.21.15.12:0/1551017380 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb4fc1a1900 con 0x7fb4fc109150 2024-09-11T00:05:23.013 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:23.010+0000 7fb4f6a00640 1 -- 172.21.15.12:0/1551017380 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb4f8004cc0 con 0x7fb4fc109150 2024-09-11T00:05:23.013 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:23.010+0000 7fb4f6a00640 1 -- 172.21.15.12:0/1551017380 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4f8010670 con 0x7fb4fc109150 2024-09-11T00:05:23.013 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:23.010+0000 7fb504800640 1 -- 172.21.15.12:0/1551017380 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4fc1a7fc0 con 0x7fb4fc109150 2024-09-11T00:05:23.013 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:23.011+0000 7fb504800640 1 -- 172.21.15.12:0/1551017380 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fb4fc100980 con 0x7fb4fc109150 2024-09-11T00:05:23.015 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:23.014+0000 7fb4f6a00640 1 -- 172.21.15.12:0/1551017380 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 17) ==== 153301+0+0 (secure 0 0 0) 0x7fb4f8005000 con 0x7fb4fc109150 2024-09-11T00:05:23.017 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:23.015+0000 7fb4f6a00640 1 --2- 172.21.15.12:0/1551017380 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fb4e40b53f0 0x7fb4e40b78b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:05:23.017 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:23.015+0000 7fb4f6a00640 1 -- 172.21.15.12:0/1551017380 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 9092+0+0 (secure 0 0 0) 0x7fb4f80e6950 con 0x7fb4fc109150 2024-09-11T00:05:23.018 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:23.015+0000 7fb501e00640 1 --2- 172.21.15.12:0/1551017380 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fb4e40b53f0 0x7fb4e40b78b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:05:23.018 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:23.015+0000 7fb4f6a00640 1 --2- 172.21.15.12:0/1551017380 >> [v2:172.21.15.178:6808/3042228555,v1:172.21.15.178:6809/3042228555] conn(0x7fb4e40c04b0 0x7fb4e40c2950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:05:23.018 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:23.016+0000 7fb4f6a00640 1 -- 172.21.15.12:0/1551017380 --> [v2:172.21.15.178:6808/3042228555,v1:172.21.15.178:6809/3042228555] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fb4f8015e80 con 0x7fb4e40c04b0 2024-09-11T00:05:23.018 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:23.016+0000 7fb4f6a00640 1 -- 172.21.15.12:0/1551017380 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_get_version_reply(handle=1 version=83) ==== 24+0+0 (secure 0 0 0) 0x7fb4f80e6da0 con 0x7fb4fc109150 2024-09-11T00:05:23.019 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:23.016+0000 7fb501400640 1 --2- 172.21.15.12:0/1551017380 >> [v2:172.21.15.178:6808/3042228555,v1:172.21.15.178:6809/3042228555] conn(0x7fb4e40c04b0 0x7fb4e40c2950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:05:23.019 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:23.016+0000 7fb501e00640 1 --2- 172.21.15.12:0/1551017380 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fb4e40b53f0 0x7fb4e40b78b0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fb4f0031440 tx=0x7fb4f0005b50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:05:23.019 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:23.017+0000 7fb501400640 1 --2- 172.21.15.12:0/1551017380 >> [v2:172.21.15.178:6808/3042228555,v1:172.21.15.178:6809/3042228555] conn(0x7fb4e40c04b0 0x7fb4e40c2950 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:05:23.019 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:23.018+0000 7fb4f6a00640 1 -- 172.21.15.12:0/1551017380 <== osd.5 v2:172.21.15.178:6808/3042228555 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fb4f8015e80 con 0x7fb4e40c04b0 2024-09-11T00:05:23.034 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:23.032+0000 7fb504800640 1 -- 172.21.15.12:0/1551017380 --> [v2:172.21.15.178:6808/3042228555,v1:172.21.15.178:6809/3042228555] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fb4fc104070 con 0x7fb4e40c04b0 2024-09-11T00:05:23.034 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:23.033+0000 7fb4f6a00640 1 -- 172.21.15.12:0/1551017380 <== osd.5 v2:172.21.15.178:6808/3042228555 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fb4fc104070 con 0x7fb4e40c04b0 2024-09-11T00:05:23.035 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:23.033+0000 7fb504800640 1 -- 172.21.15.12:0/1551017380 >> [v2:172.21.15.178:6808/3042228555,v1:172.21.15.178:6809/3042228555] conn(0x7fb4e40c04b0 msgr2=0x7fb4e40c2950 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:05:23.035 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:23.033+0000 7fb504800640 1 --2- 172.21.15.12:0/1551017380 >> [v2:172.21.15.178:6808/3042228555,v1:172.21.15.178:6809/3042228555] conn(0x7fb4e40c04b0 0x7fb4e40c2950 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:05:23.035 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:23.033+0000 7fb504800640 1 -- 172.21.15.12:0/1551017380 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fb4e40b53f0 msgr2=0x7fb4e40b78b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:05:23.036 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:23.033+0000 7fb504800640 1 --2- 172.21.15.12:0/1551017380 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fb4e40b53f0 0x7fb4e40b78b0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fb4f0031440 tx=0x7fb4f0005b50 comp rx=0 tx=0).stop 2024-09-11T00:05:23.036 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:23.033+0000 7fb504800640 1 -- 172.21.15.12:0/1551017380 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb4fc109150 msgr2=0x7fb4fc1a0e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:05:23.036 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:23.033+0000 7fb504800640 1 --2- 172.21.15.12:0/1551017380 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb4fc109150 0x7fb4fc1a0e40 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fb4f800eb10 tx=0x7fb4f800c2d0 comp rx=0 tx=0).stop 2024-09-11T00:05:23.036 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:23.033+0000 7fb502800640 1 -- 172.21.15.12:0/1551017380 reap_dead start 2024-09-11T00:05:23.036 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:23.033+0000 7fb504800640 1 -- 172.21.15.12:0/1551017380 shutdown_connections 2024-09-11T00:05:23.036 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:23.033+0000 7fb504800640 1 -- 172.21.15.12:0/1551017380 >> 172.21.15.12:0/1551017380 conn(0x7fb4fc0fe830 msgr2=0x7fb4fc0fff20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:05:23.036 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:23.033+0000 7fb504800640 1 -- 172.21.15.12:0/1551017380 shutdown_connections 2024-09-11T00:05:23.036 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:23.033+0000 7fb504800640 1 -- 172.21.15.12:0/1551017380 wait complete. 2024-09-11T00:05:23.755 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:05:23.755 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:05:23.755 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:05:24.117 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:05:23 smithi012 ceph-mon[30333]: pgmap v345: 129 pgs: 1 activating+degraded, 5 peering, 123 active+clean; 645 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 639 B/s wr, 12 op/s; 2/308 objects degraded (0.649%); 0 B/s, 0 objects/s recovering 2024-09-11T00:05:24.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:05:23 smithi165 ceph-mon[35356]: pgmap v345: 129 pgs: 1 activating+degraded, 5 peering, 123 active+clean; 645 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 639 B/s wr, 12 op/s; 2/308 objects degraded (0.649%); 0 B/s, 0 objects/s recovering 2024-09-11T00:05:24.330 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:05:23 smithi178 ceph-mon[35919]: pgmap v345: 129 pgs: 1 activating+degraded, 5 peering, 123 active+clean; 645 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 639 B/s wr, 12 op/s; 2/308 objects degraded (0.649%); 0 B/s, 0 objects/s recovering 2024-09-11T00:05:24.611 INFO:teuthology.orchestra.run.smithi012.stdout:133143986242 2024-09-11T00:05:24.612 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid ff3ad74a-6fcf-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.4 2024-09-11T00:05:24.887 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:23.581+0000 7f8a4e200640 1 -- 172.21.15.12:0/3555776706 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8a4810cf70 msgr2=0x7f8a4810f3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:05:24.887 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:23.581+0000 7f8a4e200640 1 --2- 172.21.15.12:0/3555776706 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8a4810cf70 0x7f8a4810f3d0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f8a40009ca0 tx=0x7f8a4002a8c0 comp rx=0 tx=0).stop 2024-09-11T00:05:24.887 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:23.581+0000 7f8a4e200640 1 -- 172.21.15.12:0/3555776706 shutdown_connections 2024-09-11T00:05:24.887 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:23.581+0000 7f8a4e200640 1 --2- 172.21.15.12:0/3555776706 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8a4810cf70 0x7f8a4810f3d0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:05:24.887 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:23.581+0000 7f8a4e200640 1 --2- 172.21.15.12:0/3555776706 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8a48104430 0x7f8a4810c920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:05:24.887 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:23.581+0000 7f8a4e200640 1 --2- 172.21.15.12:0/3555776706 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8a48103b10 0x7f8a48103ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:05:24.887 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:23.581+0000 7f8a4e200640 1 -- 172.21.15.12:0/3555776706 >> 172.21.15.12:0/3555776706 conn(0x7f8a480fd5b0 msgr2=0x7f8a480ff9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:05:24.887 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:23.581+0000 7f8a4e200640 1 -- 172.21.15.12:0/3555776706 shutdown_connections 2024-09-11T00:05:24.887 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:23.581+0000 7f8a4e200640 1 -- 172.21.15.12:0/3555776706 wait complete. 2024-09-11T00:05:24.888 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:23.582+0000 7f8a4e200640 1 Processor -- start 2024-09-11T00:05:24.888 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:23.582+0000 7f8a4e200640 1 -- start start 2024-09-11T00:05:24.888 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:23.583+0000 7f8a4e200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8a48103b10 0x7f8a4819b200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:05:24.888 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:23.583+0000 7f8a4e200640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8a48104430 0x7f8a4819b740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:05:24.888 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:23.583+0000 7f8a4e200640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8a4810cf70 0x7f8a481a0ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:05:24.888 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:23.583+0000 7f8a4e200640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f8a48112c30 con 0x7f8a4810cf70 2024-09-11T00:05:24.888 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:23.583+0000 7f8a4e200640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f8a48112ab0 con 0x7f8a48103b10 2024-09-11T00:05:24.888 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:23.583+0000 7f8a4e200640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f8a48112db0 con 0x7f8a48104430 2024-09-11T00:05:24.888 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:23.583+0000 7f8a47e00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8a48104430 0x7f8a4819b740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:05:24.888 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:23.583+0000 7f8a4ce00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8a48103b10 0x7f8a4819b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:05:24.888 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:23.583+0000 7f8a47e00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8a48104430 0x7f8a4819b740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.12:43032/0 (socket says 172.21.15.12:43032) 2024-09-11T00:05:24.888 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:23.583+0000 7f8a4d800640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8a4810cf70 0x7f8a481a0ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:05:24.889 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:23.583+0000 7f8a47e00640 1 -- 172.21.15.12:0/3261285657 learned_addr learned my addr 172.21.15.12:0/3261285657 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:05:24.889 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:23.583+0000 7f8a4ce00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8a48103b10 0x7f8a4819b200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.12:36904/0 (socket says 172.21.15.12:36904) 2024-09-11T00:05:24.889 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:23.584+0000 7f8a47e00640 1 -- 172.21.15.12:0/3261285657 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8a48103b10 msgr2=0x7f8a4819b200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:05:24.889 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:23.584+0000 7f8a47e00640 1 --2- 172.21.15.12:0/3261285657 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8a48103b10 0x7f8a4819b200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:05:24.889 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:23.584+0000 7f8a47e00640 1 -- 172.21.15.12:0/3261285657 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8a4810cf70 msgr2=0x7f8a481a0ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:05:24.889 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:23.584+0000 7f8a47e00640 1 --2- 172.21.15.12:0/3261285657 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8a4810cf70 0x7f8a481a0ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:05:24.889 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:23.584+0000 7f8a47e00640 1 -- 172.21.15.12:0/3261285657 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a481a12b0 con 0x7f8a48104430 2024-09-11T00:05:24.889 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:23.584+0000 7f8a4ce00640 1 --2- 172.21.15.12:0/3261285657 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8a48103b10 0x7f8a4819b200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:05:24.889 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:23.584+0000 7f8a47e00640 1 --2- 172.21.15.12:0/3261285657 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8a48104430 0x7f8a4819b740 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f8a3400e9c0 tx=0x7f8a3400ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:05:24.889 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:23.584+0000 7f8a45600640 1 -- 172.21.15.12:0/3261285657 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a3400ccc0 con 0x7f8a48104430 2024-09-11T00:05:24.889 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:23.584+0000 7f8a45600640 1 -- 172.21.15.12:0/3261285657 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8a340035f0 con 0x7f8a48104430 2024-09-11T00:05:24.889 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:23.584+0000 7f8a45600640 1 -- 172.21.15.12:0/3261285657 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a3400ce60 con 0x7f8a48104430 2024-09-11T00:05:24.890 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:23.585+0000 7f8a4e200640 1 -- 172.21.15.12:0/3261285657 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a481a15a0 con 0x7f8a48104430 2024-09-11T00:05:24.890 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:23.585+0000 7f8a4e200640 1 -- 172.21.15.12:0/3261285657 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a481a7c60 con 0x7f8a48104430 2024-09-11T00:05:24.890 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:23.586+0000 7f8a4e200640 1 -- 172.21.15.12:0/3261285657 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f8a481031e0 con 0x7f8a48104430 2024-09-11T00:05:24.890 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:23.588+0000 7f8a45600640 1 -- 172.21.15.12:0/3261285657 <== mon.1 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 17) ==== 153301+0+0 (secure 0 0 0) 0x7f8a34004000 con 0x7f8a48104430 2024-09-11T00:05:24.890 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:23.590+0000 7f8a45600640 1 --2- 172.21.15.12:0/3261285657 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f8a300b5410 0x7f8a300b78d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:05:24.890 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:23.591+0000 7f8a45600640 1 -- 172.21.15.12:0/3261285657 <== mon.1 v2:172.21.15.178:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 9092+0+0 (secure 0 0 0) 0x7f8a340e65c0 con 0x7f8a48104430 2024-09-11T00:05:24.890 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:23.591+0000 7f8a4ce00640 1 --2- 172.21.15.12:0/3261285657 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f8a300b5410 0x7f8a300b78d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:05:24.890 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:23.591+0000 7f8a45600640 1 --2- 172.21.15.12:0/3261285657 >> [v2:172.21.15.178:6816/2891445058,v1:172.21.15.178:6817/2891445058] conn(0x7f8a300c04d0 0x7f8a300c2970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:05:24.890 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:23.591+0000 7f8a45600640 1 -- 172.21.15.12:0/3261285657 --> [v2:172.21.15.178:6816/2891445058,v1:172.21.15.178:6817/2891445058] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f8a3400be50 con 0x7f8a300c04d0 2024-09-11T00:05:24.890 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:23.591+0000 7f8a45600640 1 -- 172.21.15.12:0/3261285657 <== mon.1 v2:172.21.15.178:3300/0 6 ==== mon_get_version_reply(handle=1 version=83) ==== 24+0+0 (secure 0 0 0) 0x7f8a340e6970 con 0x7f8a48104430 2024-09-11T00:05:24.890 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:23.591+0000 7f8a4d800640 1 --2- 172.21.15.12:0/3261285657 >> [v2:172.21.15.178:6816/2891445058,v1:172.21.15.178:6817/2891445058] conn(0x7f8a300c04d0 0x7f8a300c2970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:05:24.890 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:23.591+0000 7f8a4ce00640 1 --2- 172.21.15.12:0/3261285657 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f8a300b5410 0x7f8a300b78d0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f8a3c009fc0 tx=0x7f8a3c005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:05:24.890 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:23.592+0000 7f8a4d800640 1 --2- 172.21.15.12:0/3261285657 >> [v2:172.21.15.178:6816/2891445058,v1:172.21.15.178:6817/2891445058] conn(0x7f8a300c04d0 0x7f8a300c2970 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:05:24.891 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:23.593+0000 7f8a45600640 1 -- 172.21.15.12:0/3261285657 <== osd.8 v2:172.21.15.178:6816/2891445058 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f8a3400be50 con 0x7f8a300c04d0 2024-09-11T00:05:24.891 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:23.608+0000 7f8a4e200640 1 -- 172.21.15.12:0/3261285657 --> [v2:172.21.15.178:6816/2891445058,v1:172.21.15.178:6817/2891445058] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f8a4806b1b0 con 0x7f8a300c04d0 2024-09-11T00:05:24.891 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:23.609+0000 7f8a45600640 1 -- 172.21.15.12:0/3261285657 <== osd.8 v2:172.21.15.178:6816/2891445058 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f8a4806b1b0 con 0x7f8a300c04d0 2024-09-11T00:05:24.891 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:23.609+0000 7f8a4e200640 1 -- 172.21.15.12:0/3261285657 >> [v2:172.21.15.178:6816/2891445058,v1:172.21.15.178:6817/2891445058] conn(0x7f8a300c04d0 msgr2=0x7f8a300c2970 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:05:24.891 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:23.609+0000 7f8a4e200640 1 --2- 172.21.15.12:0/3261285657 >> [v2:172.21.15.178:6816/2891445058,v1:172.21.15.178:6817/2891445058] conn(0x7f8a300c04d0 0x7f8a300c2970 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:05:24.891 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:23.609+0000 7f8a4e200640 1 -- 172.21.15.12:0/3261285657 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f8a300b5410 msgr2=0x7f8a300b78d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:05:24.891 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:23.609+0000 7f8a4e200640 1 --2- 172.21.15.12:0/3261285657 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f8a300b5410 0x7f8a300b78d0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f8a3c009fc0 tx=0x7f8a3c005f90 comp rx=0 tx=0).stop 2024-09-11T00:05:24.891 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:23.609+0000 7f8a4e200640 1 -- 172.21.15.12:0/3261285657 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8a48104430 msgr2=0x7f8a4819b740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:05:24.892 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:23.609+0000 7f8a4e200640 1 --2- 172.21.15.12:0/3261285657 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8a48104430 0x7f8a4819b740 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f8a3400e9c0 tx=0x7f8a3400ee80 comp rx=0 tx=0).stop 2024-09-11T00:05:24.892 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:23.609+0000 7f8a4d800640 1 -- 172.21.15.12:0/3261285657 reap_dead start 2024-09-11T00:05:24.892 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:23.609+0000 7f8a4e200640 1 -- 172.21.15.12:0/3261285657 shutdown_connections 2024-09-11T00:05:24.892 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:23.609+0000 7f8a4e200640 1 -- 172.21.15.12:0/3261285657 >> 172.21.15.12:0/3261285657 conn(0x7f8a480fd5b0 msgr2=0x7f8a4810f110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:05:24.892 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:23.610+0000 7f8a4e200640 1 -- 172.21.15.12:0/3261285657 shutdown_connections 2024-09-11T00:05:24.892 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:23.610+0000 7f8a4e200640 1 -- 172.21.15.12:0/3261285657 wait complete. 2024-09-11T00:05:25.604 INFO:teuthology.orchestra.run.smithi012.stdout:34359738456 2024-09-11T00:05:25.604 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid ff3ad74a-6fcf-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.0 2024-09-11T00:05:26.125 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:05:25 smithi012 ceph-mon[30333]: pgmap v346: 129 pgs: 1 activating+degraded, 5 peering, 123 active+clean; 645 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s; 2/308 objects degraded (0.649%); 0 B/s, 0 objects/s recovering 2024-09-11T00:05:26.164 INFO:teuthology.orchestra.run.smithi012.stdout:77309411406 2024-09-11T00:05:26.164 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid ff3ad74a-6fcf-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.2 2024-09-11T00:05:26.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:05:25 smithi165 ceph-mon[35356]: pgmap v346: 129 pgs: 1 activating+degraded, 5 peering, 123 active+clean; 645 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s; 2/308 objects degraded (0.649%); 0 B/s, 0 objects/s recovering 2024-09-11T00:05:26.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:05:25 smithi178 ceph-mon[35919]: pgmap v346: 129 pgs: 1 activating+degraded, 5 peering, 123 active+clean; 645 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s; 2/308 objects degraded (0.649%); 0 B/s, 0 objects/s recovering 2024-09-11T00:05:26.743 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:05:26.775 INFO:teuthology.orchestra.run.smithi012.stdout:107374182471 2024-09-11T00:05:26.775 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid ff3ad74a-6fcf-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.3 2024-09-11T00:05:26.938 INFO:teuthology.orchestra.run.smithi012.stdout:223338299438 2024-09-11T00:05:26.939 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid ff3ad74a-6fcf-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.8 2024-09-11T00:05:26.987 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:05:26 smithi012 ceph-mon[30333]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg inactive, 3 pgs peering) 2024-09-11T00:05:26.988 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:05:26 smithi012 ceph-mon[30333]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 2/308 objects degraded (0.649%), 1 pg degraded) 2024-09-11T00:05:26.988 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:05:26 smithi012 ceph-mon[30333]: Cluster is now healthy 2024-09-11T00:05:27.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:05:26 smithi165 ceph-mon[35356]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg inactive, 3 pgs peering) 2024-09-11T00:05:27.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:05:26 smithi165 ceph-mon[35356]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 2/308 objects degraded (0.649%), 1 pg degraded) 2024-09-11T00:05:27.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:05:26 smithi165 ceph-mon[35356]: Cluster is now healthy 2024-09-11T00:05:27.330 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:05:26 smithi178 ceph-mon[35919]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg inactive, 3 pgs peering) 2024-09-11T00:05:27.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:05:26 smithi178 ceph-mon[35919]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 2/308 objects degraded (0.649%), 1 pg degraded) 2024-09-11T00:05:27.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:05:26 smithi178 ceph-mon[35919]: Cluster is now healthy 2024-09-11T00:05:27.646 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:27.412+0000 7ffa68200640 1 -- 172.21.15.12:0/3061027819 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ffa60104f90 msgr2=0x7ffa60105370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:05:27.647 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:27.412+0000 7ffa68200640 1 --2- 172.21.15.12:0/3061027819 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ffa60104f90 0x7ffa60105370 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7ffa54009900 tx=0x7ffa5402a3f0 comp rx=0 tx=0).stop 2024-09-11T00:05:27.647 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:27.412+0000 7ffa68200640 1 -- 172.21.15.12:0/3061027819 shutdown_connections 2024-09-11T00:05:27.647 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:27.412+0000 7ffa68200640 1 --2- 172.21.15.12:0/3061027819 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ffa6010b320 0x7ffa60111b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:05:27.647 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:27.412+0000 7ffa68200640 1 --2- 172.21.15.12:0/3061027819 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ffa6010a9a0 0x7ffa6010ade0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:05:27.647 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:27.412+0000 7ffa68200640 1 --2- 172.21.15.12:0/3061027819 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ffa60104f90 0x7ffa60105370 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:05:27.647 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:27.412+0000 7ffa68200640 1 -- 172.21.15.12:0/3061027819 >> 172.21.15.12:0/3061027819 conn(0x7ffa60100a00 msgr2=0x7ffa60102e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:05:27.647 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:27.412+0000 7ffa68200640 1 -- 172.21.15.12:0/3061027819 shutdown_connections 2024-09-11T00:05:27.647 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:27.412+0000 7ffa68200640 1 -- 172.21.15.12:0/3061027819 wait complete. 2024-09-11T00:05:27.647 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:27.413+0000 7ffa68200640 1 Processor -- start 2024-09-11T00:05:27.647 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:27.413+0000 7ffa68200640 1 -- start start 2024-09-11T00:05:27.648 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:27.413+0000 7ffa68200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ffa60104f90 0x7ffa6019b460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:05:27.648 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:27.413+0000 7ffa68200640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ffa6010a9a0 0x7ffa6019b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:05:27.648 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:27.413+0000 7ffa68200640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ffa6010b320 0x7ffa601a0d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:05:27.648 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:27.413+0000 7ffa68200640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7ffa60114fa0 con 0x7ffa6010a9a0 2024-09-11T00:05:27.648 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:27.413+0000 7ffa68200640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7ffa60114e20 con 0x7ffa60104f90 2024-09-11T00:05:27.648 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:27.413+0000 7ffa68200640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7ffa60115120 con 0x7ffa6010b320 2024-09-11T00:05:27.648 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:27.414+0000 7ffa64e00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ffa6010a9a0 0x7ffa6019b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:05:27.648 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:27.414+0000 7ffa64e00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ffa6010a9a0 0x7ffa6019b9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:41314/0 (socket says 172.21.15.12:41314) 2024-09-11T00:05:27.648 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:27.414+0000 7ffa64e00640 1 -- 172.21.15.12:0/4010214474 learned_addr learned my addr 172.21.15.12:0/4010214474 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:05:27.648 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:27.414+0000 7ffa66200640 1 --2- 172.21.15.12:0/4010214474 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ffa6010b320 0x7ffa601a0d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:05:27.648 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:27.414+0000 7ffa65800640 1 --2- 172.21.15.12:0/4010214474 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ffa60104f90 0x7ffa6019b460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:05:27.648 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:27.414+0000 7ffa64e00640 1 -- 172.21.15.12:0/4010214474 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ffa6010b320 msgr2=0x7ffa601a0d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:05:27.649 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:27.414+0000 7ffa64e00640 1 --2- 172.21.15.12:0/4010214474 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ffa6010b320 0x7ffa601a0d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:05:27.649 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:27.414+0000 7ffa64e00640 1 -- 172.21.15.12:0/4010214474 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ffa60104f90 msgr2=0x7ffa6019b460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:05:27.649 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:27.414+0000 7ffa64e00640 1 --2- 172.21.15.12:0/4010214474 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ffa60104f90 0x7ffa6019b460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:05:27.649 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:27.414+0000 7ffa64e00640 1 -- 172.21.15.12:0/4010214474 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa601a1510 con 0x7ffa6010a9a0 2024-09-11T00:05:27.649 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:27.414+0000 7ffa64e00640 1 --2- 172.21.15.12:0/4010214474 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ffa6010a9a0 0x7ffa6019b9a0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7ffa4c00e9f0 tx=0x7ffa4c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:05:27.649 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:27.415+0000 7ffa5e000640 1 -- 172.21.15.12:0/4010214474 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa4c00cd80 con 0x7ffa6010a9a0 2024-09-11T00:05:27.649 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:27.415+0000 7ffa5e000640 1 -- 172.21.15.12:0/4010214474 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffa4c0108e0 con 0x7ffa6010a9a0 2024-09-11T00:05:27.649 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:27.415+0000 7ffa68200640 1 -- 172.21.15.12:0/4010214474 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa601a1800 con 0x7ffa6010a9a0 2024-09-11T00:05:27.649 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:27.415+0000 7ffa5e000640 1 -- 172.21.15.12:0/4010214474 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa4c018980 con 0x7ffa6010a9a0 2024-09-11T00:05:27.649 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:27.415+0000 7ffa68200640 1 -- 172.21.15.12:0/4010214474 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa601a7ec0 con 0x7ffa6010a9a0 2024-09-11T00:05:27.649 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:27.416+0000 7ffa68200640 1 -- 172.21.15.12:0/4010214474 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7ffa600638a0 con 0x7ffa6010a9a0 2024-09-11T00:05:27.649 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:27.418+0000 7ffa5e000640 1 -- 172.21.15.12:0/4010214474 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 17) ==== 153301+0+0 (secure 0 0 0) 0x7ffa4c010430 con 0x7ffa6010a9a0 2024-09-11T00:05:27.649 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:27.420+0000 7ffa5e000640 1 --2- 172.21.15.12:0/4010214474 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7ffa480b53f0 0x7ffa480b78b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:05:27.650 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:27.420+0000 7ffa5e000640 1 -- 172.21.15.12:0/4010214474 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 9092+0+0 (secure 0 0 0) 0x7ffa4c0e6b30 con 0x7ffa6010a9a0 2024-09-11T00:05:27.650 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:27.420+0000 7ffa65800640 1 --2- 172.21.15.12:0/4010214474 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7ffa480b53f0 0x7ffa480b78b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:05:27.650 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:27.421+0000 7ffa5e000640 1 --2- 172.21.15.12:0/4010214474 >> [v2:172.21.15.12:6826/3359634903,v1:172.21.15.12:6827/3359634903] conn(0x7ffa480c04b0 0x7ffa480c2950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:05:27.650 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:27.421+0000 7ffa5e000640 1 -- 172.21.15.12:0/4010214474 --> [v2:172.21.15.12:6826/3359634903,v1:172.21.15.12:6827/3359634903] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7ffa480c3020 con 0x7ffa480c04b0 2024-09-11T00:05:27.650 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:27.421+0000 7ffa5e000640 1 -- 172.21.15.12:0/4010214474 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_get_version_reply(handle=1 version=83) ==== 24+0+0 (secure 0 0 0) 0x7ffa4c0e6ee0 con 0x7ffa6010a9a0 2024-09-11T00:05:27.650 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:27.421+0000 7ffa66200640 1 --2- 172.21.15.12:0/4010214474 >> [v2:172.21.15.12:6826/3359634903,v1:172.21.15.12:6827/3359634903] conn(0x7ffa480c04b0 0x7ffa480c2950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:05:27.651 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:27.421+0000 7ffa65800640 1 --2- 172.21.15.12:0/4010214474 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7ffa480b53f0 0x7ffa480b78b0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7ffa54005ab0 tx=0x7ffa54005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:05:27.651 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:27.422+0000 7ffa66200640 1 --2- 172.21.15.12:0/4010214474 >> [v2:172.21.15.12:6826/3359634903,v1:172.21.15.12:6827/3359634903] conn(0x7ffa480c04b0 0x7ffa480c2950 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.9 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:05:27.651 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:27.422+0000 7ffa5e000640 1 -- 172.21.15.12:0/4010214474 <== osd.9 v2:172.21.15.12:6826/3359634903 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7ffa480c3020 con 0x7ffa480c04b0 2024-09-11T00:05:27.651 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:27.437+0000 7ffa68200640 1 -- 172.21.15.12:0/4010214474 --> [v2:172.21.15.12:6826/3359634903,v1:172.21.15.12:6827/3359634903] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7ffa60063ca0 con 0x7ffa480c04b0 2024-09-11T00:05:27.651 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:27.437+0000 7ffa5e000640 1 -- 172.21.15.12:0/4010214474 <== osd.9 v2:172.21.15.12:6826/3359634903 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7ffa60063ca0 con 0x7ffa480c04b0 2024-09-11T00:05:27.651 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:27.437+0000 7ffa68200640 1 -- 172.21.15.12:0/4010214474 >> [v2:172.21.15.12:6826/3359634903,v1:172.21.15.12:6827/3359634903] conn(0x7ffa480c04b0 msgr2=0x7ffa480c2950 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:05:27.651 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:27.437+0000 7ffa68200640 1 --2- 172.21.15.12:0/4010214474 >> [v2:172.21.15.12:6826/3359634903,v1:172.21.15.12:6827/3359634903] conn(0x7ffa480c04b0 0x7ffa480c2950 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:05:27.651 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:27.438+0000 7ffa68200640 1 -- 172.21.15.12:0/4010214474 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7ffa480b53f0 msgr2=0x7ffa480b78b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:05:27.651 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:27.438+0000 7ffa68200640 1 --2- 172.21.15.12:0/4010214474 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7ffa480b53f0 0x7ffa480b78b0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7ffa54005ab0 tx=0x7ffa54005a20 comp rx=0 tx=0).stop 2024-09-11T00:05:27.652 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:27.438+0000 7ffa68200640 1 -- 172.21.15.12:0/4010214474 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ffa6010a9a0 msgr2=0x7ffa6019b9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:05:27.652 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:27.438+0000 7ffa68200640 1 --2- 172.21.15.12:0/4010214474 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ffa6010a9a0 0x7ffa6019b9a0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7ffa4c00e9f0 tx=0x7ffa4c00eeb0 comp rx=0 tx=0).stop 2024-09-11T00:05:27.652 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:27.438+0000 7ffa66200640 1 -- 172.21.15.12:0/4010214474 reap_dead start 2024-09-11T00:05:27.652 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:27.438+0000 7ffa68200640 1 -- 172.21.15.12:0/4010214474 shutdown_connections 2024-09-11T00:05:27.652 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:27.438+0000 7ffa68200640 1 -- 172.21.15.12:0/4010214474 >> 172.21.15.12:0/4010214474 conn(0x7ffa60100a00 msgr2=0x7ffa60101300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:05:27.652 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:27.438+0000 7ffa68200640 1 -- 172.21.15.12:0/4010214474 shutdown_connections 2024-09-11T00:05:27.652 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:27.438+0000 7ffa68200640 1 -- 172.21.15.12:0/4010214474 wait complete. 2024-09-11T00:05:28.167 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:05:27 smithi012 ceph-mon[30333]: pgmap v347: 129 pgs: 129 active+clean; 647 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 255 B/s wr, 12 op/s; 13 B/s, 1 objects/s recovering 2024-09-11T00:05:28.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:05:27 smithi165 ceph-mon[35356]: pgmap v347: 129 pgs: 129 active+clean; 647 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 255 B/s wr, 12 op/s; 13 B/s, 1 objects/s recovering 2024-09-11T00:05:28.330 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:05:27 smithi178 ceph-mon[35919]: pgmap v347: 129 pgs: 129 active+clean; 647 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 255 B/s wr, 12 op/s; 13 B/s, 1 objects/s recovering 2024-09-11T00:05:28.772 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:28.457+0000 7ff746400640 1 -- 172.21.15.12:0/1799028273 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff74010b1a0 msgr2=0x7ff7401119a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:05:28.773 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:28.457+0000 7ff746400640 1 --2- 172.21.15.12:0/1799028273 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff74010b1a0 0x7ff7401119a0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7ff738009ca0 tx=0x7ff73802a8c0 comp rx=0 tx=0).stop 2024-09-11T00:05:28.773 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:28.457+0000 7ff746400640 1 -- 172.21.15.12:0/1799028273 shutdown_connections 2024-09-11T00:05:28.773 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:28.457+0000 7ff746400640 1 --2- 172.21.15.12:0/1799028273 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff74010b1a0 0x7ff7401119a0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:05:28.773 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:28.457+0000 7ff746400640 1 --2- 172.21.15.12:0/1799028273 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff74010a820 0x7ff74010ac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:05:28.773 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:28.457+0000 7ff746400640 1 --2- 172.21.15.12:0/1799028273 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff740104e10 0x7ff7401051f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:05:28.773 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:28.457+0000 7ff746400640 1 -- 172.21.15.12:0/1799028273 >> 172.21.15.12:0/1799028273 conn(0x7ff740100840 msgr2=0x7ff740102c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:05:28.773 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:28.457+0000 7ff746400640 1 -- 172.21.15.12:0/1799028273 shutdown_connections 2024-09-11T00:05:28.773 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:28.457+0000 7ff746400640 1 -- 172.21.15.12:0/1799028273 wait complete. 2024-09-11T00:05:28.773 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:28.458+0000 7ff746400640 1 Processor -- start 2024-09-11T00:05:28.773 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:28.458+0000 7ff746400640 1 -- start start 2024-09-11T00:05:28.774 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:28.459+0000 7ff746400640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff740104e10 0x7ff74019b200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:05:28.774 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:28.459+0000 7ff746400640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff74010a820 0x7ff74019b740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:05:28.774 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:28.459+0000 7ff746400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff74010b1a0 0x7ff7401a0ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:05:28.774 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:28.459+0000 7ff746400640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7ff740114e70 con 0x7ff740104e10 2024-09-11T00:05:28.774 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:28.459+0000 7ff746400640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7ff740114cf0 con 0x7ff74010b1a0 2024-09-11T00:05:28.774 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:28.459+0000 7ff746400640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7ff740114ff0 con 0x7ff74010a820 2024-09-11T00:05:28.774 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:28.459+0000 7ff745000640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff740104e10 0x7ff74019b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:05:28.774 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:28.459+0000 7ff745000640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff740104e10 0x7ff74019b200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:41334/0 (socket says 172.21.15.12:41334) 2024-09-11T00:05:28.774 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:28.459+0000 7ff745000640 1 -- 172.21.15.12:0/2517186224 learned_addr learned my addr 172.21.15.12:0/2517186224 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:05:28.774 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:28.459+0000 7ff73fe00640 1 --2- 172.21.15.12:0/2517186224 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff74010a820 0x7ff74019b740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:05:28.774 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:28.459+0000 7ff745a00640 1 --2- 172.21.15.12:0/2517186224 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff74010b1a0 0x7ff7401a0ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:05:28.775 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:28.460+0000 7ff745000640 1 -- 172.21.15.12:0/2517186224 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff74010a820 msgr2=0x7ff74019b740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:05:28.775 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:28.460+0000 7ff745000640 1 --2- 172.21.15.12:0/2517186224 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff74010a820 0x7ff74019b740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:05:28.775 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:28.460+0000 7ff745000640 1 -- 172.21.15.12:0/2517186224 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff74010b1a0 msgr2=0x7ff7401a0ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:05:28.775 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:28.460+0000 7ff745000640 1 --2- 172.21.15.12:0/2517186224 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff74010b1a0 0x7ff7401a0ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:05:28.775 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:28.460+0000 7ff745000640 1 -- 172.21.15.12:0/2517186224 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7401a12b0 con 0x7ff740104e10 2024-09-11T00:05:28.775 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:28.460+0000 7ff745a00640 1 --2- 172.21.15.12:0/2517186224 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff74010b1a0 0x7ff7401a0ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:05:28.775 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:28.460+0000 7ff745000640 1 --2- 172.21.15.12:0/2517186224 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff740104e10 0x7ff74019b200 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7ff73400e9f0 tx=0x7ff73400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:05:28.776 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:28.460+0000 7ff73d600640 1 -- 172.21.15.12:0/2517186224 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff73400cd80 con 0x7ff740104e10 2024-09-11T00:05:28.776 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:28.460+0000 7ff73d600640 1 -- 172.21.15.12:0/2517186224 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff7340108e0 con 0x7ff740104e10 2024-09-11T00:05:28.776 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:28.460+0000 7ff746400640 1 -- 172.21.15.12:0/2517186224 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff7401a15a0 con 0x7ff740104e10 2024-09-11T00:05:28.776 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:28.460+0000 7ff73d600640 1 -- 172.21.15.12:0/2517186224 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff734018980 con 0x7ff740104e10 2024-09-11T00:05:28.776 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:28.460+0000 7ff746400640 1 -- 172.21.15.12:0/2517186224 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7401a7c60 con 0x7ff740104e10 2024-09-11T00:05:28.776 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:28.461+0000 7ff746400640 1 -- 172.21.15.12:0/2517186224 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7ff74010a1b0 con 0x7ff740104e10 2024-09-11T00:05:28.776 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:28.464+0000 7ff73d600640 1 -- 172.21.15.12:0/2517186224 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 17) ==== 153301+0+0 (secure 0 0 0) 0x7ff734010430 con 0x7ff740104e10 2024-09-11T00:05:28.777 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:28.465+0000 7ff73d600640 1 --2- 172.21.15.12:0/2517186224 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7ff7280b5410 0x7ff7280b78d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:05:28.777 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:28.465+0000 7ff73d600640 1 -- 172.21.15.12:0/2517186224 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 9092+0+0 (secure 0 0 0) 0x7ff7340e6b80 con 0x7ff740104e10 2024-09-11T00:05:28.777 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:28.465+0000 7ff73fe00640 1 --2- 172.21.15.12:0/2517186224 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7ff7280b5410 0x7ff7280b78d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:05:28.777 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:28.465+0000 7ff73d600640 1 --2- 172.21.15.12:0/2517186224 >> [v2:172.21.15.178:6824/2937059756,v1:172.21.15.178:6825/2937059756] conn(0x7ff7280c04d0 0x7ff7280c2970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:05:28.777 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:28.465+0000 7ff73d600640 1 -- 172.21.15.12:0/2517186224 --> [v2:172.21.15.178:6824/2937059756,v1:172.21.15.178:6825/2937059756] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7ff734015e80 con 0x7ff7280c04d0 2024-09-11T00:05:28.777 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:28.465+0000 7ff73d600640 1 -- 172.21.15.12:0/2517186224 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_get_version_reply(handle=1 version=83) ==== 24+0+0 (secure 0 0 0) 0x7ff7340e6f30 con 0x7ff740104e10 2024-09-11T00:05:28.777 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:28.466+0000 7ff745a00640 1 --2- 172.21.15.12:0/2517186224 >> [v2:172.21.15.178:6824/2937059756,v1:172.21.15.178:6825/2937059756] conn(0x7ff7280c04d0 0x7ff7280c2970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:05:28.777 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:28.466+0000 7ff73fe00640 1 --2- 172.21.15.12:0/2517186224 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7ff7280b5410 0x7ff7280b78d0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7ff740107290 tx=0x7ff72c005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:05:28.778 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:28.467+0000 7ff745a00640 1 --2- 172.21.15.12:0/2517186224 >> [v2:172.21.15.178:6824/2937059756,v1:172.21.15.178:6825/2937059756] conn(0x7ff7280c04d0 0x7ff7280c2970 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.11 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:05:28.778 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:28.468+0000 7ff73d600640 1 -- 172.21.15.12:0/2517186224 <== osd.11 v2:172.21.15.178:6824/2937059756 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7ff734015e80 con 0x7ff7280c04d0 2024-09-11T00:05:28.778 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:28.482+0000 7ff746400640 1 -- 172.21.15.12:0/2517186224 --> [v2:172.21.15.178:6824/2937059756,v1:172.21.15.178:6825/2937059756] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7ff7401060c0 con 0x7ff7280c04d0 2024-09-11T00:05:28.778 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:28.483+0000 7ff73d600640 1 -- 172.21.15.12:0/2517186224 <== osd.11 v2:172.21.15.178:6824/2937059756 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7ff7401060c0 con 0x7ff7280c04d0 2024-09-11T00:05:28.778 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:28.483+0000 7ff746400640 1 -- 172.21.15.12:0/2517186224 >> [v2:172.21.15.178:6824/2937059756,v1:172.21.15.178:6825/2937059756] conn(0x7ff7280c04d0 msgr2=0x7ff7280c2970 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:05:28.778 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:28.483+0000 7ff746400640 1 --2- 172.21.15.12:0/2517186224 >> [v2:172.21.15.178:6824/2937059756,v1:172.21.15.178:6825/2937059756] conn(0x7ff7280c04d0 0x7ff7280c2970 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:05:28.778 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:28.484+0000 7ff746400640 1 -- 172.21.15.12:0/2517186224 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7ff7280b5410 msgr2=0x7ff7280b78d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:05:28.779 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:28.484+0000 7ff746400640 1 --2- 172.21.15.12:0/2517186224 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7ff7280b5410 0x7ff7280b78d0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7ff740107290 tx=0x7ff72c005f90 comp rx=0 tx=0).stop 2024-09-11T00:05:28.779 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:28.484+0000 7ff746400640 1 -- 172.21.15.12:0/2517186224 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff740104e10 msgr2=0x7ff74019b200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:05:28.779 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:28.484+0000 7ff746400640 1 --2- 172.21.15.12:0/2517186224 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff740104e10 0x7ff74019b200 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7ff73400e9f0 tx=0x7ff73400eeb0 comp rx=0 tx=0).stop 2024-09-11T00:05:28.779 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:28.484+0000 7ff745a00640 1 -- 172.21.15.12:0/2517186224 reap_dead start 2024-09-11T00:05:28.779 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:28.484+0000 7ff746400640 1 -- 172.21.15.12:0/2517186224 shutdown_connections 2024-09-11T00:05:28.779 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:28.484+0000 7ff746400640 1 -- 172.21.15.12:0/2517186224 >> 172.21.15.12:0/2517186224 conn(0x7ff740100840 msgr2=0x7ff740102c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:05:28.779 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:28.484+0000 7ff746400640 1 -- 172.21.15.12:0/2517186224 shutdown_connections 2024-09-11T00:05:28.779 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:28.484+0000 7ff746400640 1 -- 172.21.15.12:0/2517186224 wait complete. 2024-09-11T00:05:29.348 INFO:teuthology.orchestra.run.smithi012.stdout:249108103207 2024-09-11T00:05:29.348 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid ff3ad74a-6fcf-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.9 2024-09-11T00:05:29.688 INFO:teuthology.orchestra.run.smithi012.stdout:154618822717 2024-09-11T00:05:29.688 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid ff3ad74a-6fcf-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.5 2024-09-11T00:05:29.754 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:29.231+0000 7f1894c00640 1 -- 172.21.15.12:0/3522752533 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f18900754e0 msgr2=0x7f18900758c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:05:29.754 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:29.231+0000 7f1894c00640 1 --2- 172.21.15.12:0/3522752533 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f18900754e0 0x7f18900758c0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f1880009900 tx=0x7f188002a3f0 comp rx=0 tx=0).stop 2024-09-11T00:05:29.754 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:29.231+0000 7f1894c00640 1 -- 172.21.15.12:0/3522752533 shutdown_connections 2024-09-11T00:05:29.754 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:29.231+0000 7f1894c00640 1 --2- 172.21.15.12:0/3522752533 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f189010f200 0x7f18901115f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:05:29.754 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:29.231+0000 7f1894c00640 1 --2- 172.21.15.12:0/3522752533 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1890075e00 0x7f189010ecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:05:29.754 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:29.231+0000 7f1894c00640 1 --2- 172.21.15.12:0/3522752533 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f18900754e0 0x7f18900758c0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:05:29.755 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:29.231+0000 7f1894c00640 1 -- 172.21.15.12:0/3522752533 >> 172.21.15.12:0/3522752533 conn(0x7f18900fe660 msgr2=0x7f1890100a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:05:29.755 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:29.231+0000 7f1894c00640 1 -- 172.21.15.12:0/3522752533 shutdown_connections 2024-09-11T00:05:29.755 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:29.231+0000 7f1894c00640 1 -- 172.21.15.12:0/3522752533 wait complete. 2024-09-11T00:05:29.755 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:29.232+0000 7f1894c00640 1 Processor -- start 2024-09-11T00:05:29.755 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:29.232+0000 7f1894c00640 1 -- start start 2024-09-11T00:05:29.755 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:29.233+0000 7f1894c00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f18900754e0 0x7f189019b210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:05:29.755 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:29.233+0000 7f1894c00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1890075e00 0x7f189019b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:05:29.755 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:29.233+0000 7f1894c00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f189010f200 0x7f18901a0af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:05:29.755 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:29.233+0000 7f1894c00640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f1890114de0 con 0x7f18900754e0 2024-09-11T00:05:29.755 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:29.233+0000 7f1894c00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f1890114c60 con 0x7f189010f200 2024-09-11T00:05:29.755 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:29.233+0000 7f1894c00640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f1890114f60 con 0x7f1890075e00 2024-09-11T00:05:29.755 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:29.233+0000 7f188f400640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f18900754e0 0x7f189019b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:05:29.755 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:29.233+0000 7f188ea00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1890075e00 0x7f189019b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:05:29.756 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:29.233+0000 7f188fe00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f189010f200 0x7f18901a0af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:05:29.756 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:29.233+0000 7f188ea00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1890075e00 0x7f189019b750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.12:43210/0 (socket says 172.21.15.12:43210) 2024-09-11T00:05:29.756 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:29.233+0000 7f188ea00640 1 -- 172.21.15.12:0/2479873743 learned_addr learned my addr 172.21.15.12:0/2479873743 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:05:29.756 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:29.233+0000 7f188f400640 1 --2- 172.21.15.12:0/2479873743 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f18900754e0 0x7f189019b210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:60352/0 (socket says 172.21.15.12:60352) 2024-09-11T00:05:29.756 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:29.234+0000 7f188ea00640 1 -- 172.21.15.12:0/2479873743 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f189010f200 msgr2=0x7f18901a0af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:05:29.756 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:29.234+0000 7f188ea00640 1 --2- 172.21.15.12:0/2479873743 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f189010f200 0x7f18901a0af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:05:29.756 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:29.234+0000 7f188ea00640 1 -- 172.21.15.12:0/2479873743 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f18900754e0 msgr2=0x7f189019b210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:05:29.756 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:29.234+0000 7f188ea00640 1 --2- 172.21.15.12:0/2479873743 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f18900754e0 0x7f189019b210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:05:29.756 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:29.234+0000 7f188ea00640 1 -- 172.21.15.12:0/2479873743 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18901a12c0 con 0x7f1890075e00 2024-09-11T00:05:29.756 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:29.234+0000 7f188fe00640 1 --2- 172.21.15.12:0/2479873743 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f189010f200 0x7f18901a0af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:05:29.756 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:29.234+0000 7f188ea00640 1 --2- 172.21.15.12:0/2479873743 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1890075e00 0x7f189019b750 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f187c00d950 tx=0x7f187c00de10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:05:29.756 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:29.234+0000 7f1887e00640 1 -- 172.21.15.12:0/2479873743 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f187c017070 con 0x7f1890075e00 2024-09-11T00:05:29.757 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:29.234+0000 7f1894c00640 1 -- 172.21.15.12:0/2479873743 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18901a15b0 con 0x7f1890075e00 2024-09-11T00:05:29.757 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:29.234+0000 7f1887e00640 1 -- 172.21.15.12:0/2479873743 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f187c009c30 con 0x7f1890075e00 2024-09-11T00:05:29.757 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:29.234+0000 7f1887e00640 1 -- 172.21.15.12:0/2479873743 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f187c00f620 con 0x7f1890075e00 2024-09-11T00:05:29.757 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:29.234+0000 7f1894c00640 1 -- 172.21.15.12:0/2479873743 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18901a7c70 con 0x7f1890075e00 2024-09-11T00:05:29.757 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:29.235+0000 7f1894c00640 1 -- 172.21.15.12:0/2479873743 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f18901005c0 con 0x7f1890075e00 2024-09-11T00:05:29.757 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:29.238+0000 7f1887e00640 1 -- 172.21.15.12:0/2479873743 <== mon.1 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 17) ==== 153301+0+0 (secure 0 0 0) 0x7f187c010460 con 0x7f1890075e00 2024-09-11T00:05:29.757 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:29.240+0000 7f1887e00640 1 --2- 172.21.15.12:0/2479873743 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f18740b5410 0x7f18740b78d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:05:29.757 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:29.240+0000 7f1887e00640 1 -- 172.21.15.12:0/2479873743 <== mon.1 v2:172.21.15.178:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 9092+0+0 (secure 0 0 0) 0x7f187c0e6160 con 0x7f1890075e00 2024-09-11T00:05:29.757 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:29.240+0000 7f188f400640 1 --2- 172.21.15.12:0/2479873743 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f18740b5410 0x7f18740b78d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:05:29.757 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:29.240+0000 7f1887e00640 1 --2- 172.21.15.12:0/2479873743 >> [v2:172.21.15.165:6824/3642325778,v1:172.21.15.165:6825/3642325778] conn(0x7f18740c04d0 0x7f18740c2970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:05:29.757 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:29.240+0000 7f1887e00640 1 -- 172.21.15.12:0/2479873743 --> [v2:172.21.15.165:6824/3642325778,v1:172.21.15.165:6825/3642325778] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f187c022e70 con 0x7f18740c04d0 2024-09-11T00:05:29.757 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:29.240+0000 7f1887e00640 1 -- 172.21.15.12:0/2479873743 <== mon.1 v2:172.21.15.178:3300/0 6 ==== mon_get_version_reply(handle=1 version=83) ==== 24+0+0 (secure 0 0 0) 0x7f187c0e6510 con 0x7f1890075e00 2024-09-11T00:05:29.758 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:29.241+0000 7f188f400640 1 --2- 172.21.15.12:0/2479873743 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f18740b5410 0x7f18740b78d0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f1880031440 tx=0x7f1880005b50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:05:29.758 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:29.243+0000 7f188fe00640 1 --2- 172.21.15.12:0/2479873743 >> [v2:172.21.15.165:6824/3642325778,v1:172.21.15.165:6825/3642325778] conn(0x7f18740c04d0 0x7f18740c2970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:05:29.758 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:29.244+0000 7f188fe00640 1 --2- 172.21.15.12:0/2479873743 >> [v2:172.21.15.165:6824/3642325778,v1:172.21.15.165:6825/3642325778] conn(0x7f18740c04d0 0x7f18740c2970 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.10 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:05:29.758 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:29.245+0000 7f1887e00640 1 -- 172.21.15.12:0/2479873743 <== osd.10 v2:172.21.15.165:6824/3642325778 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f187c022e70 con 0x7f18740c04d0 2024-09-11T00:05:29.758 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:29.259+0000 7f1894c00640 1 -- 172.21.15.12:0/2479873743 --> [v2:172.21.15.165:6824/3642325778,v1:172.21.15.165:6825/3642325778] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f18900769f0 con 0x7f18740c04d0 2024-09-11T00:05:29.758 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:29.260+0000 7f1887e00640 1 -- 172.21.15.12:0/2479873743 <== osd.10 v2:172.21.15.165:6824/3642325778 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f18900769f0 con 0x7f18740c04d0 2024-09-11T00:05:29.758 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:29.260+0000 7f1894c00640 1 -- 172.21.15.12:0/2479873743 >> [v2:172.21.15.165:6824/3642325778,v1:172.21.15.165:6825/3642325778] conn(0x7f18740c04d0 msgr2=0x7f18740c2970 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:05:29.759 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:29.261+0000 7f1894c00640 1 --2- 172.21.15.12:0/2479873743 >> [v2:172.21.15.165:6824/3642325778,v1:172.21.15.165:6825/3642325778] conn(0x7f18740c04d0 0x7f18740c2970 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:05:29.759 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:29.261+0000 7f1894c00640 1 -- 172.21.15.12:0/2479873743 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f18740b5410 msgr2=0x7f18740b78d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:05:29.759 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:29.261+0000 7f1894c00640 1 --2- 172.21.15.12:0/2479873743 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f18740b5410 0x7f18740b78d0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f1880031440 tx=0x7f1880005b50 comp rx=0 tx=0).stop 2024-09-11T00:05:29.759 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:29.261+0000 7f1894c00640 1 -- 172.21.15.12:0/2479873743 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1890075e00 msgr2=0x7f189019b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:05:29.759 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:29.261+0000 7f1894c00640 1 --2- 172.21.15.12:0/2479873743 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1890075e00 0x7f189019b750 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f187c00d950 tx=0x7f187c00de10 comp rx=0 tx=0).stop 2024-09-11T00:05:29.759 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:29.261+0000 7f188fe00640 1 -- 172.21.15.12:0/2479873743 reap_dead start 2024-09-11T00:05:29.759 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:29.261+0000 7f1894c00640 1 -- 172.21.15.12:0/2479873743 shutdown_connections 2024-09-11T00:05:29.759 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:29.261+0000 7f1894c00640 1 -- 172.21.15.12:0/2479873743 >> 172.21.15.12:0/2479873743 conn(0x7f18900fe660 msgr2=0x7f1890111390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:05:29.759 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:29.261+0000 7f1894c00640 1 -- 172.21.15.12:0/2479873743 shutdown_connections 2024-09-11T00:05:29.759 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:29.261+0000 7f1894c00640 1 -- 172.21.15.12:0/2479873743 wait complete. 2024-09-11T00:05:30.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:05:29 smithi165 ceph-mon[35356]: pgmap v348: 129 pgs: 129 active+clean; 647 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 220 B/s wr, 11 op/s; 12 B/s, 1 objects/s recovering 2024-09-11T00:05:30.312 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:05:29 smithi012 ceph-mon[30333]: pgmap v348: 129 pgs: 129 active+clean; 647 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 220 B/s wr, 11 op/s; 12 B/s, 1 objects/s recovering 2024-09-11T00:05:30.330 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:05:29 smithi178 ceph-mon[35919]: pgmap v348: 129 pgs: 129 active+clean; 647 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 220 B/s wr, 11 op/s; 12 B/s, 1 objects/s recovering 2024-09-11T00:05:30.458 INFO:teuthology.orchestra.run.smithi012.stdout:270582939684 2024-09-11T00:05:30.458 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid ff3ad74a-6fcf-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.10 2024-09-11T00:05:30.813 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:30.805+0000 7f11fae00640 1 -- 172.21.15.12:0/3657210409 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f11f4075e00 msgr2=0x7f11f410ee50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:05:30.813 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:30.805+0000 7f11fae00640 1 --2- 172.21.15.12:0/3657210409 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f11f4075e00 0x7f11f410ee50 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f11e0009900 tx=0x7f11e002a3f0 comp rx=0 tx=0).stop 2024-09-11T00:05:30.813 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:30.805+0000 7f11fae00640 1 -- 172.21.15.12:0/3657210409 shutdown_connections 2024-09-11T00:05:30.813 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:30.805+0000 7f11fae00640 1 --2- 172.21.15.12:0/3657210409 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f11f410f390 0x7f11f4111780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:05:30.814 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:30.805+0000 7f11fae00640 1 --2- 172.21.15.12:0/3657210409 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f11f4075e00 0x7f11f410ee50 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:05:30.814 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:30.805+0000 7f11fae00640 1 --2- 172.21.15.12:0/3657210409 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f11f40754e0 0x7f11f40758c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:05:30.814 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:30.805+0000 7f11fae00640 1 -- 172.21.15.12:0/3657210409 >> 172.21.15.12:0/3657210409 conn(0x7f11f40fe810 msgr2=0x7f11f4100c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:05:30.814 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:30.805+0000 7f11fae00640 1 -- 172.21.15.12:0/3657210409 shutdown_connections 2024-09-11T00:05:30.814 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:30.805+0000 7f11fae00640 1 -- 172.21.15.12:0/3657210409 wait complete. 2024-09-11T00:05:30.814 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:30.806+0000 7f11fae00640 1 Processor -- start 2024-09-11T00:05:30.814 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:30.806+0000 7f11fae00640 1 -- start start 2024-09-11T00:05:30.814 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:30.807+0000 7f11fae00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f11f40754e0 0x7f11f419b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:05:30.814 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:30.807+0000 7f11fae00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f11f4075e00 0x7f11f419ba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:05:30.814 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:30.807+0000 7f11fae00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f11f410f390 0x7f11f41a0dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:05:30.814 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:30.807+0000 7f11fae00640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f11f4114f50 con 0x7f11f4075e00 2024-09-11T00:05:30.814 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:30.807+0000 7f11fae00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f11f4114dd0 con 0x7f11f410f390 2024-09-11T00:05:30.814 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:30.807+0000 7f11fae00640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f11f41150d0 con 0x7f11f40754e0 2024-09-11T00:05:30.815 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:30.807+0000 7f11f3400640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f11f4075e00 0x7f11f419ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:05:30.815 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:30.807+0000 7f11f3400640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f11f4075e00 0x7f11f419ba20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:60364/0 (socket says 172.21.15.12:60364) 2024-09-11T00:05:30.815 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:30.807+0000 7f11f8e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f11f410f390 0x7f11f41a0dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:05:30.815 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:30.807+0000 7f11f3400640 1 -- 172.21.15.12:0/296861689 learned_addr learned my addr 172.21.15.12:0/296861689 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:05:30.815 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:30.807+0000 7f11f3e00640 1 --2- 172.21.15.12:0/296861689 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f11f40754e0 0x7f11f419b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:05:30.815 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:30.807+0000 7f11f8e00640 1 -- 172.21.15.12:0/296861689 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f11f40754e0 msgr2=0x7f11f419b4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:05:30.815 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:30.808+0000 7f11f8e00640 1 --2- 172.21.15.12:0/296861689 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f11f40754e0 0x7f11f419b4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:05:30.815 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:30.808+0000 7f11f8e00640 1 -- 172.21.15.12:0/296861689 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f11f4075e00 msgr2=0x7f11f419ba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:05:30.815 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:30.808+0000 7f11f8e00640 1 --2- 172.21.15.12:0/296861689 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f11f4075e00 0x7f11f419ba20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:05:30.815 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:30.808+0000 7f11f8e00640 1 -- 172.21.15.12:0/296861689 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11f41a1590 con 0x7f11f410f390 2024-09-11T00:05:30.815 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:30.808+0000 7f11f3400640 1 --2- 172.21.15.12:0/296861689 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f11f4075e00 0x7f11f419ba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:05:30.815 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:30.808+0000 7f11f8e00640 1 --2- 172.21.15.12:0/296861689 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f11f410f390 0x7f11f41a0dc0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f11ec005fd0 tx=0x7f11ec00ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:05:30.815 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:30.808+0000 7f11f0c00640 1 -- 172.21.15.12:0/296861689 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11ec00cdf0 con 0x7f11f410f390 2024-09-11T00:05:30.816 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:30.808+0000 7f11fae00640 1 -- 172.21.15.12:0/296861689 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f11f41a1880 con 0x7f11f410f390 2024-09-11T00:05:30.816 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:30.808+0000 7f11f0c00640 1 -- 172.21.15.12:0/296861689 <== mon.2 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f11ec005020 con 0x7f11f410f390 2024-09-11T00:05:30.816 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:30.808+0000 7f11f0c00640 1 -- 172.21.15.12:0/296861689 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11ec010690 con 0x7f11f410f390 2024-09-11T00:05:30.816 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:30.808+0000 7f11fae00640 1 -- 172.21.15.12:0/296861689 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11f41a7f40 con 0x7f11f410f390 2024-09-11T00:05:30.816 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:30.809+0000 7f11fae00640 1 -- 172.21.15.12:0/296861689 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f11f4100e40 con 0x7f11f410f390 2024-09-11T00:05:30.817 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:30.812+0000 7f11f0c00640 1 -- 172.21.15.12:0/296861689 <== mon.2 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 17) ==== 153301+0+0 (secure 0 0 0) 0x7f11ec005460 con 0x7f11f410f390 2024-09-11T00:05:30.817 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:30.814+0000 7f11f0c00640 1 --2- 172.21.15.12:0/296861689 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f11d80b53f0 0x7f11d80b78b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:05:30.817 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:30.814+0000 7f11f0c00640 1 -- 172.21.15.12:0/296861689 <== mon.2 v2:172.21.15.165:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 9092+0+0 (secure 0 0 0) 0x7f11ec0e6970 con 0x7f11f410f390 2024-09-11T00:05:30.817 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:30.814+0000 7f11f3e00640 1 --2- 172.21.15.12:0/296861689 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f11d80b53f0 0x7f11d80b78b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:05:30.817 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:30.814+0000 7f11f0c00640 1 --2- 172.21.15.12:0/296861689 >> [v2:172.21.15.12:6818/3868322467,v1:172.21.15.12:6819/3868322467] conn(0x7f11d80c04b0 0x7f11d80c2950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:05:30.817 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:30.815+0000 7f11f0c00640 1 -- 172.21.15.12:0/296861689 --> [v2:172.21.15.12:6818/3868322467,v1:172.21.15.12:6819/3868322467] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f11ec022e50 con 0x7f11d80c04b0 2024-09-11T00:05:30.817 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:30.815+0000 7f11f0c00640 1 -- 172.21.15.12:0/296861689 <== mon.2 v2:172.21.15.165:3300/0 6 ==== mon_get_version_reply(handle=1 version=83) ==== 24+0+0 (secure 0 0 0) 0x7f11ec0e6d60 con 0x7f11f410f390 2024-09-11T00:05:30.817 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:30.815+0000 7f11f3400640 1 --2- 172.21.15.12:0/296861689 >> [v2:172.21.15.12:6818/3868322467,v1:172.21.15.12:6819/3868322467] conn(0x7f11d80c04b0 0x7f11d80c2950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:05:30.818 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:30.815+0000 7f11f3e00640 1 --2- 172.21.15.12:0/296861689 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f11d80b53f0 0x7f11d80b78b0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f11e4009bd0 tx=0x7f11e4005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:05:30.818 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:30.816+0000 7f11f3400640 1 --2- 172.21.15.12:0/296861689 >> [v2:172.21.15.12:6818/3868322467,v1:172.21.15.12:6819/3868322467] conn(0x7f11d80c04b0 0x7f11d80c2950 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:05:30.818 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:30.816+0000 7f11f0c00640 1 -- 172.21.15.12:0/296861689 <== osd.6 v2:172.21.15.12:6818/3868322467 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f11ec022e50 con 0x7f11d80c04b0 2024-09-11T00:05:30.833 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:30.831+0000 7f11fae00640 1 -- 172.21.15.12:0/296861689 --> [v2:172.21.15.12:6818/3868322467,v1:172.21.15.12:6819/3868322467] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f11f40769f0 con 0x7f11d80c04b0 2024-09-11T00:05:30.833 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:30.831+0000 7f11f0c00640 1 -- 172.21.15.12:0/296861689 <== osd.6 v2:172.21.15.12:6818/3868322467 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f11f40769f0 con 0x7f11d80c04b0 2024-09-11T00:05:30.835 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:30.832+0000 7f11fae00640 1 -- 172.21.15.12:0/296861689 >> [v2:172.21.15.12:6818/3868322467,v1:172.21.15.12:6819/3868322467] conn(0x7f11d80c04b0 msgr2=0x7f11d80c2950 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:05:30.835 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:30.832+0000 7f11fae00640 1 --2- 172.21.15.12:0/296861689 >> [v2:172.21.15.12:6818/3868322467,v1:172.21.15.12:6819/3868322467] conn(0x7f11d80c04b0 0x7f11d80c2950 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:05:30.835 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:30.832+0000 7f11fae00640 1 -- 172.21.15.12:0/296861689 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f11d80b53f0 msgr2=0x7f11d80b78b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:05:30.836 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:30.832+0000 7f11fae00640 1 --2- 172.21.15.12:0/296861689 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f11d80b53f0 0x7f11d80b78b0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f11e4009bd0 tx=0x7f11e4005f90 comp rx=0 tx=0).stop 2024-09-11T00:05:30.836 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:30.832+0000 7f11fae00640 1 -- 172.21.15.12:0/296861689 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f11f410f390 msgr2=0x7f11f41a0dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:05:30.836 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:30.832+0000 7f11fae00640 1 --2- 172.21.15.12:0/296861689 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f11f410f390 0x7f11f41a0dc0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f11ec005fd0 tx=0x7f11ec00ef10 comp rx=0 tx=0).stop 2024-09-11T00:05:30.836 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:30.832+0000 7f11f8e00640 1 -- 172.21.15.12:0/296861689 reap_dead start 2024-09-11T00:05:30.836 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:30.832+0000 7f11fae00640 1 -- 172.21.15.12:0/296861689 shutdown_connections 2024-09-11T00:05:30.836 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:30.832+0000 7f11fae00640 1 -- 172.21.15.12:0/296861689 >> 172.21.15.12:0/296861689 conn(0x7f11f40fe810 msgr2=0x7f11f40ff410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:05:30.836 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:30.833+0000 7f11fae00640 1 -- 172.21.15.12:0/296861689 shutdown_connections 2024-09-11T00:05:30.837 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:30.833+0000 7f11fae00640 1 -- 172.21.15.12:0/296861689 wait complete. 2024-09-11T00:05:31.622 INFO:teuthology.orchestra.run.smithi012.stdout:180388626490 2024-09-11T00:05:31.622 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid ff3ad74a-6fcf-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.6 2024-09-11T00:05:32.168 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:05:31 smithi012 ceph-mon[30333]: pgmap v349: 129 pgs: 129 active+clean; 647 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 199 B/s wr, 12 op/s; 10 B/s, 1 objects/s recovering 2024-09-11T00:05:32.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:05:31 smithi165 ceph-mon[35356]: pgmap v349: 129 pgs: 129 active+clean; 647 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 199 B/s wr, 12 op/s; 10 B/s, 1 objects/s recovering 2024-09-11T00:05:32.330 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:05:31 smithi178 ceph-mon[35919]: pgmap v349: 129 pgs: 129 active+clean; 647 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 199 B/s wr, 12 op/s; 10 B/s, 1 objects/s recovering 2024-09-11T00:05:32.377 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:32.161+0000 7f561a800640 1 -- 172.21.15.12:0/3125895336 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f561410b180 msgr2=0x7f5614111980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:05:32.377 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:32.161+0000 7f561a800640 1 --2- 172.21.15.12:0/3125895336 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f561410b180 0x7f5614111980 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f560c009ca0 tx=0x7f560c02a8c0 comp rx=0 tx=0).stop 2024-09-11T00:05:32.377 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:32.161+0000 7f561a800640 1 -- 172.21.15.12:0/3125895336 shutdown_connections 2024-09-11T00:05:32.377 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:32.161+0000 7f561a800640 1 --2- 172.21.15.12:0/3125895336 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f561410b180 0x7f5614111980 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:05:32.378 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:32.161+0000 7f561a800640 1 --2- 172.21.15.12:0/3125895336 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f561410a800 0x7f561410ac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:05:32.378 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:32.161+0000 7f561a800640 1 --2- 172.21.15.12:0/3125895336 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5614104df0 0x7f56141051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:05:32.378 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:32.161+0000 7f561a800640 1 -- 172.21.15.12:0/3125895336 >> 172.21.15.12:0/3125895336 conn(0x7f5614100840 msgr2=0x7f5614102c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:05:32.378 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:32.161+0000 7f561a800640 1 -- 172.21.15.12:0/3125895336 shutdown_connections 2024-09-11T00:05:32.378 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:32.161+0000 7f561a800640 1 -- 172.21.15.12:0/3125895336 wait complete. 2024-09-11T00:05:32.378 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:32.163+0000 7f561a800640 1 Processor -- start 2024-09-11T00:05:32.378 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:32.163+0000 7f561a800640 1 -- start start 2024-09-11T00:05:32.378 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:32.163+0000 7f561a800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5614104df0 0x7f561419b270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:05:32.378 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:32.163+0000 7f561a800640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f561410a800 0x7f561419b7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:05:32.378 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:32.163+0000 7f561a800640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f561410b180 0x7f56141a0b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:05:32.378 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:32.163+0000 7f561a800640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f5614114e00 con 0x7f561410a800 2024-09-11T00:05:32.378 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:32.163+0000 7f561a800640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f5614114c80 con 0x7f5614104df0 2024-09-11T00:05:32.378 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:32.163+0000 7f561a800640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f5614114f80 con 0x7f561410b180 2024-09-11T00:05:32.379 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:32.164+0000 7f5618a00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f561410a800 0x7f561419b7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:05:32.379 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:32.164+0000 7f5618a00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f561410a800 0x7f561419b7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:60382/0 (socket says 172.21.15.12:60382) 2024-09-11T00:05:32.379 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:32.164+0000 7f5618a00640 1 -- 172.21.15.12:0/2088641984 learned_addr learned my addr 172.21.15.12:0/2088641984 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:05:32.379 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:32.164+0000 7f5619e00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f561410b180 0x7f56141a0b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:05:32.379 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:32.164+0000 7f5619400640 1 --2- 172.21.15.12:0/2088641984 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5614104df0 0x7f561419b270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:05:32.379 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:32.164+0000 7f5618a00640 1 -- 172.21.15.12:0/2088641984 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f561410b180 msgr2=0x7f56141a0b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:05:32.379 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:32.164+0000 7f5618a00640 1 --2- 172.21.15.12:0/2088641984 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f561410b180 0x7f56141a0b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:05:32.379 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:32.164+0000 7f5618a00640 1 -- 172.21.15.12:0/2088641984 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5614104df0 msgr2=0x7f561419b270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:05:32.379 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:32.164+0000 7f5618a00640 1 --2- 172.21.15.12:0/2088641984 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5614104df0 0x7f561419b270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:05:32.379 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:32.164+0000 7f5618a00640 1 -- 172.21.15.12:0/2088641984 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56141a1320 con 0x7f561410a800 2024-09-11T00:05:32.379 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:32.164+0000 7f5619400640 1 --2- 172.21.15.12:0/2088641984 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5614104df0 0x7f561419b270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:05:32.379 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:32.164+0000 7f5618a00640 1 --2- 172.21.15.12:0/2088641984 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f561410a800 0x7f561419b7b0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f560000e9f0 tx=0x7f560000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:05:32.379 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:32.165+0000 7f5612000640 1 -- 172.21.15.12:0/2088641984 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f560000cd80 con 0x7f561410a800 2024-09-11T00:05:32.380 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:32.165+0000 7f5612000640 1 -- 172.21.15.12:0/2088641984 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f56000054d0 con 0x7f561410a800 2024-09-11T00:05:32.380 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:32.165+0000 7f561a800640 1 -- 172.21.15.12:0/2088641984 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f56141a1610 con 0x7f561410a800 2024-09-11T00:05:32.380 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:32.165+0000 7f5612000640 1 -- 172.21.15.12:0/2088641984 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5600010690 con 0x7f561410a800 2024-09-11T00:05:32.380 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:32.165+0000 7f561a800640 1 -- 172.21.15.12:0/2088641984 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56141a7cd0 con 0x7f561410a800 2024-09-11T00:05:32.380 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:32.166+0000 7f561a800640 1 -- 172.21.15.12:0/2088641984 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f5614063860 con 0x7f561410a800 2024-09-11T00:05:32.380 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:32.169+0000 7f5612000640 1 -- 172.21.15.12:0/2088641984 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 17) ==== 153301+0+0 (secure 0 0 0) 0x7f5600004e30 con 0x7f561410a800 2024-09-11T00:05:32.381 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:32.170+0000 7f5612000640 1 --2- 172.21.15.12:0/2088641984 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f55fc0b5410 0x7f55fc0b78d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:05:32.381 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:32.171+0000 7f5612000640 1 -- 172.21.15.12:0/2088641984 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 9092+0+0 (secure 0 0 0) 0x7f56000e68b0 con 0x7f561410a800 2024-09-11T00:05:32.381 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:32.171+0000 7f5619400640 1 --2- 172.21.15.12:0/2088641984 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f55fc0b5410 0x7f55fc0b78d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:05:32.381 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:32.171+0000 7f5612000640 1 --2- 172.21.15.12:0/2088641984 >> [v2:172.21.15.165:6816/1143773735,v1:172.21.15.165:6817/1143773735] conn(0x7f55fc0c04d0 0x7f55fc0c2970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:05:32.381 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:32.171+0000 7f5612000640 1 -- 172.21.15.12:0/2088641984 --> [v2:172.21.15.165:6816/1143773735,v1:172.21.15.165:6817/1143773735] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f5600015e80 con 0x7f55fc0c04d0 2024-09-11T00:05:32.381 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:32.171+0000 7f5612000640 1 -- 172.21.15.12:0/2088641984 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_get_version_reply(handle=1 version=83) ==== 24+0+0 (secure 0 0 0) 0x7f56000e6c60 con 0x7f561410a800 2024-09-11T00:05:32.381 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:32.171+0000 7f5619e00640 1 --2- 172.21.15.12:0/2088641984 >> [v2:172.21.15.165:6816/1143773735,v1:172.21.15.165:6817/1143773735] conn(0x7f55fc0c04d0 0x7f55fc0c2970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:05:32.381 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:32.171+0000 7f5619400640 1 --2- 172.21.15.12:0/2088641984 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f55fc0b5410 0x7f55fc0b78d0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f56080095f0 tx=0x7f5608005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:05:32.381 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:32.172+0000 7f5619e00640 1 --2- 172.21.15.12:0/2088641984 >> [v2:172.21.15.165:6816/1143773735,v1:172.21.15.165:6817/1143773735] conn(0x7f55fc0c04d0 0x7f55fc0c2970 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:05:32.381 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:32.173+0000 7f5612000640 1 -- 172.21.15.12:0/2088641984 <== osd.7 v2:172.21.15.165:6816/1143773735 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f5600015e80 con 0x7f55fc0c04d0 2024-09-11T00:05:32.381 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:32.187+0000 7f561a800640 1 -- 172.21.15.12:0/2088641984 --> [v2:172.21.15.165:6816/1143773735,v1:172.21.15.165:6817/1143773735] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f5614063ce0 con 0x7f55fc0c04d0 2024-09-11T00:05:32.381 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:32.188+0000 7f5612000640 1 -- 172.21.15.12:0/2088641984 <== osd.7 v2:172.21.15.165:6816/1143773735 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f5614063ce0 con 0x7f55fc0c04d0 2024-09-11T00:05:32.382 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:32.188+0000 7f561a800640 1 -- 172.21.15.12:0/2088641984 >> [v2:172.21.15.165:6816/1143773735,v1:172.21.15.165:6817/1143773735] conn(0x7f55fc0c04d0 msgr2=0x7f55fc0c2970 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:05:32.382 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:32.188+0000 7f561a800640 1 --2- 172.21.15.12:0/2088641984 >> [v2:172.21.15.165:6816/1143773735,v1:172.21.15.165:6817/1143773735] conn(0x7f55fc0c04d0 0x7f55fc0c2970 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:05:32.382 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:32.189+0000 7f561a800640 1 -- 172.21.15.12:0/2088641984 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f55fc0b5410 msgr2=0x7f55fc0b78d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:05:32.382 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:32.189+0000 7f561a800640 1 --2- 172.21.15.12:0/2088641984 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f55fc0b5410 0x7f55fc0b78d0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f56080095f0 tx=0x7f5608005f90 comp rx=0 tx=0).stop 2024-09-11T00:05:32.382 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:32.189+0000 7f561a800640 1 -- 172.21.15.12:0/2088641984 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f561410a800 msgr2=0x7f561419b7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:05:32.382 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:32.189+0000 7f561a800640 1 --2- 172.21.15.12:0/2088641984 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f561410a800 0x7f561419b7b0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f560000e9f0 tx=0x7f560000edf0 comp rx=0 tx=0).stop 2024-09-11T00:05:32.382 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:32.189+0000 7f5619e00640 1 -- 172.21.15.12:0/2088641984 reap_dead start 2024-09-11T00:05:32.382 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:32.189+0000 7f561a800640 1 -- 172.21.15.12:0/2088641984 shutdown_connections 2024-09-11T00:05:32.382 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:32.189+0000 7f561a800640 1 -- 172.21.15.12:0/2088641984 >> 172.21.15.12:0/2088641984 conn(0x7f5614100840 msgr2=0x7f5614108e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:05:32.382 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:32.189+0000 7f561a800640 1 -- 172.21.15.12:0/2088641984 shutdown_connections 2024-09-11T00:05:32.382 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:32.189+0000 7f561a800640 1 -- 172.21.15.12:0/2088641984 wait complete. 2024-09-11T00:05:32.583 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:05:32.591 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:05:32.838 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:05:32.863 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:05:32.869 INFO:teuthology.orchestra.run.smithi012.stdout:292057776159 2024-09-11T00:05:32.870 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid ff3ad74a-6fcf-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.11 2024-09-11T00:05:33.525 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:33.522+0000 7f60ef600640 1 -- 172.21.15.12:0/1158914378 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f60e8102dc0 msgr2=0x7f60e81031a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:05:33.525 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:33.522+0000 7f60ef600640 1 --2- 172.21.15.12:0/1158914378 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f60e8102dc0 0x7f60e81031a0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f60dc009900 tx=0x7f60dc02a3f0 comp rx=0 tx=0).stop 2024-09-11T00:05:33.525 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:33.522+0000 7f60ef600640 1 -- 172.21.15.12:0/1158914378 shutdown_connections 2024-09-11T00:05:33.525 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:33.522+0000 7f60ef600640 1 --2- 172.21.15.12:0/1158914378 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f60e8109150 0x7f60e810f950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:05:33.525 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:33.522+0000 7f60ef600640 1 --2- 172.21.15.12:0/1158914378 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f60e81087d0 0x7f60e8108c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:05:33.525 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:33.522+0000 7f60ef600640 1 --2- 172.21.15.12:0/1158914378 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f60e8102dc0 0x7f60e81031a0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:05:33.525 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:33.522+0000 7f60ef600640 1 -- 172.21.15.12:0/1158914378 >> 172.21.15.12:0/1158914378 conn(0x7f60e80fe830 msgr2=0x7f60e8100c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:05:33.525 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:33.522+0000 7f60ef600640 1 -- 172.21.15.12:0/1158914378 shutdown_connections 2024-09-11T00:05:33.526 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:33.522+0000 7f60ef600640 1 -- 172.21.15.12:0/1158914378 wait complete. 2024-09-11T00:05:33.526 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:33.523+0000 7f60ef600640 1 Processor -- start 2024-09-11T00:05:33.527 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:33.523+0000 7f60ef600640 1 -- start start 2024-09-11T00:05:33.527 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:33.524+0000 7f60ef600640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f60e8102dc0 0x7f60e819b4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:05:33.527 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:33.524+0000 7f60ef600640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f60e81087d0 0x7f60e819b9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:05:33.527 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:33.524+0000 7f60ef600640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f60e8109150 0x7f60e81a0d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:05:33.527 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:33.524+0000 7f60ef600640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f60e8112e20 con 0x7f60e8109150 2024-09-11T00:05:33.527 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:33.524+0000 7f60ef600640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f60e8112ca0 con 0x7f60e8102dc0 2024-09-11T00:05:33.527 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:33.524+0000 7f60ef600640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f60e8112fa0 con 0x7f60e81087d0 2024-09-11T00:05:33.527 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:33.524+0000 7f60e7e00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f60e81087d0 0x7f60e819b9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:05:33.527 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:33.524+0000 7f60e7e00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f60e81087d0 0x7f60e819b9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.12:43244/0 (socket says 172.21.15.12:43244) 2024-09-11T00:05:33.527 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:33.524+0000 7f60ed600640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f60e8109150 0x7f60e81a0d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:05:33.527 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:33.524+0000 7f60e7e00640 1 -- 172.21.15.12:0/1017769571 learned_addr learned my addr 172.21.15.12:0/1017769571 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:05:33.527 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:33.524+0000 7f60ecc00640 1 --2- 172.21.15.12:0/1017769571 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f60e8102dc0 0x7f60e819b4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:05:33.528 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:33.525+0000 7f60e7e00640 1 -- 172.21.15.12:0/1017769571 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f60e8102dc0 msgr2=0x7f60e819b4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:05:33.528 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:33.525+0000 7f60e7e00640 1 --2- 172.21.15.12:0/1017769571 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f60e8102dc0 0x7f60e819b4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:05:33.528 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:33.525+0000 7f60e7e00640 1 -- 172.21.15.12:0/1017769571 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f60e8109150 msgr2=0x7f60e81a0d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:05:33.528 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:33.525+0000 7f60e7e00640 1 --2- 172.21.15.12:0/1017769571 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f60e8109150 0x7f60e81a0d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:05:33.528 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:33.525+0000 7f60e7e00640 1 -- 172.21.15.12:0/1017769571 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60e81a1560 con 0x7f60e81087d0 2024-09-11T00:05:33.528 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:33.525+0000 7f60e7e00640 1 --2- 172.21.15.12:0/1017769571 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f60e81087d0 0x7f60e819b9f0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f60d400e9f0 tx=0x7f60d400eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:05:33.528 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:33.525+0000 7f60e5600640 1 -- 172.21.15.12:0/1017769571 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60d400cd80 con 0x7f60e81087d0 2024-09-11T00:05:33.529 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:33.525+0000 7f60ef600640 1 -- 172.21.15.12:0/1017769571 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f60e81a1850 con 0x7f60e81087d0 2024-09-11T00:05:33.529 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:33.525+0000 7f60e5600640 1 -- 172.21.15.12:0/1017769571 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f60d40108e0 con 0x7f60e81087d0 2024-09-11T00:05:33.529 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:33.525+0000 7f60e5600640 1 -- 172.21.15.12:0/1017769571 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60d4018980 con 0x7f60e81087d0 2024-09-11T00:05:33.529 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:33.525+0000 7f60ef600640 1 -- 172.21.15.12:0/1017769571 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60e81a7f10 con 0x7f60e81087d0 2024-09-11T00:05:33.529 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:33.527+0000 7f60ef600640 1 -- 172.21.15.12:0/1017769571 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f60e81031a0 con 0x7f60e81087d0 2024-09-11T00:05:33.532 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:33.530+0000 7f60e5600640 1 -- 172.21.15.12:0/1017769571 <== mon.1 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 17) ==== 153301+0+0 (secure 0 0 0) 0x7f60d4010430 con 0x7f60e81087d0 2024-09-11T00:05:33.533 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:33.532+0000 7f60e5600640 1 --2- 172.21.15.12:0/1017769571 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f60d00b5370 0x7f60d00b7830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:05:33.534 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:33.532+0000 7f60e5600640 1 -- 172.21.15.12:0/1017769571 <== mon.1 v2:172.21.15.178:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 9092+0+0 (secure 0 0 0) 0x7f60d40e63b0 con 0x7f60e81087d0 2024-09-11T00:05:33.534 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:33.532+0000 7f60ecc00640 1 --2- 172.21.15.12:0/1017769571 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f60d00b5370 0x7f60d00b7830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:05:33.534 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:33.533+0000 7f60ecc00640 1 --2- 172.21.15.12:0/1017769571 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f60d00b5370 0x7f60d00b7830 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f60dc005ab0 tx=0x7f60dc005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:05:33.537 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:33.536+0000 7f60e5600640 1 -- 172.21.15.12:0/1017769571 <== mon.1 v2:172.21.15.178:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f60d40ac760 con 0x7f60e81087d0 2024-09-11T00:05:33.740 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:33.738+0000 7f60ef600640 1 -- 172.21.15.12:0/1017769571 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7f60e810edb0 con 0x7f60e81087d0 2024-09-11T00:05:33.740 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:33.739+0000 7f60e5600640 1 -- 172.21.15.12:0/1017769571 <== mon.1 v2:172.21.15.178:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f60d40b1190 con 0x7f60e81087d0 2024-09-11T00:05:33.741 INFO:teuthology.orchestra.run.smithi012.stdout:55834574932 2024-09-11T00:05:33.743 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:33.741+0000 7f60ef600640 1 -- 172.21.15.12:0/1017769571 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f60d00b5370 msgr2=0x7f60d00b7830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:05:33.743 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:33.741+0000 7f60ef600640 1 --2- 172.21.15.12:0/1017769571 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f60d00b5370 0x7f60d00b7830 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f60dc005ab0 tx=0x7f60dc005a20 comp rx=0 tx=0).stop 2024-09-11T00:05:33.743 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:33.741+0000 7f60ef600640 1 -- 172.21.15.12:0/1017769571 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f60e81087d0 msgr2=0x7f60e819b9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:05:33.743 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:33.741+0000 7f60ef600640 1 --2- 172.21.15.12:0/1017769571 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f60e81087d0 0x7f60e819b9f0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f60d400e9f0 tx=0x7f60d400eeb0 comp rx=0 tx=0).stop 2024-09-11T00:05:33.743 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:33.741+0000 7f60ef600640 1 -- 172.21.15.12:0/1017769571 shutdown_connections 2024-09-11T00:05:33.744 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:33.741+0000 7f60ef600640 1 --2- 172.21.15.12:0/1017769571 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f60d00b5370 0x7f60d00b7830 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:05:33.744 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:33.741+0000 7f60ef600640 1 --2- 172.21.15.12:0/1017769571 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f60e8109150 0x7f60e81a0d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:05:33.744 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:33.741+0000 7f60ef600640 1 --2- 172.21.15.12:0/1017769571 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f60e81087d0 0x7f60e819b9f0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:05:33.744 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:33.741+0000 7f60ef600640 1 --2- 172.21.15.12:0/1017769571 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f60e8102dc0 0x7f60e819b4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:05:33.744 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:33.741+0000 7f60ef600640 1 -- 172.21.15.12:0/1017769571 >> 172.21.15.12:0/1017769571 conn(0x7f60e80fe830 msgr2=0x7f60e8104d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:05:33.744 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:33.741+0000 7f60ef600640 1 -- 172.21.15.12:0/1017769571 shutdown_connections 2024-09-11T00:05:33.744 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:33.741+0000 7f60ef600640 1 -- 172.21.15.12:0/1017769571 wait complete. 2024-09-11T00:05:34.062 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:05:33 smithi012 ceph-mon[30333]: pgmap v350: 129 pgs: 129 active+clean; 647 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 170 B/s wr, 11 op/s; 9 B/s, 1 objects/s recovering 2024-09-11T00:05:34.063 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:05:33 smithi012 ceph-mon[30333]: from='client.? 172.21.15.12:0/1017769571' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 1} : dispatch 2024-09-11T00:05:34.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:05:33 smithi165 ceph-mon[35356]: pgmap v350: 129 pgs: 129 active+clean; 647 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 170 B/s wr, 11 op/s; 9 B/s, 1 objects/s recovering 2024-09-11T00:05:34.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:05:33 smithi165 ceph-mon[35356]: from='client.? 172.21.15.12:0/1017769571' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 1} : dispatch 2024-09-11T00:05:34.330 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:05:33 smithi178 ceph-mon[35919]: pgmap v350: 129 pgs: 129 active+clean; 647 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 170 B/s wr, 11 op/s; 9 B/s, 1 objects/s recovering 2024-09-11T00:05:34.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:05:33 smithi178 ceph-mon[35919]: from='client.? 172.21.15.12:0/1017769571' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 1} : dispatch 2024-09-11T00:05:34.713 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:34.711+0000 7ff1d3e00640 1 -- 172.21.15.12:0/1409791337 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff1d4102c10 msgr2=0x7ff1d4102ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:05:34.713 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:34.711+0000 7ff1d3e00640 1 --2- 172.21.15.12:0/1409791337 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff1d4102c10 0x7ff1d4102ff0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7ff1c4009900 tx=0x7ff1c402a3f0 comp rx=0 tx=0).stop 2024-09-11T00:05:34.713 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:34.712+0000 7ff1d3e00640 1 -- 172.21.15.12:0/1409791337 shutdown_connections 2024-09-11T00:05:34.713 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:34.712+0000 7ff1d3e00640 1 --2- 172.21.15.12:0/1409791337 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff1d4108fa0 0x7ff1d410f7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:05:34.713 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:34.712+0000 7ff1d3e00640 1 --2- 172.21.15.12:0/1409791337 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff1d4108620 0x7ff1d4108a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:05:34.713 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:34.712+0000 7ff1d3e00640 1 --2- 172.21.15.12:0/1409791337 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff1d4102c10 0x7ff1d4102ff0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:05:34.713 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:34.712+0000 7ff1d3e00640 1 -- 172.21.15.12:0/1409791337 >> 172.21.15.12:0/1409791337 conn(0x7ff1d40fe660 msgr2=0x7ff1d4100a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:05:34.714 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:34.712+0000 7ff1d3e00640 1 -- 172.21.15.12:0/1409791337 shutdown_connections 2024-09-11T00:05:34.714 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:34.712+0000 7ff1d3e00640 1 -- 172.21.15.12:0/1409791337 wait complete. 2024-09-11T00:05:34.714 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:34.713+0000 7ff1d3e00640 1 Processor -- start 2024-09-11T00:05:34.715 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:34.713+0000 7ff1d3e00640 1 -- start start 2024-09-11T00:05:34.715 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:34.713+0000 7ff1d3e00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff1d4102c10 0x7ff1d419b2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:05:34.715 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:34.713+0000 7ff1d3e00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff1d4108620 0x7ff1d419b7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:05:34.715 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:34.713+0000 7ff1d3e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff1d4108fa0 0x7ff1d41a0b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:05:34.715 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:34.713+0000 7ff1d3e00640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7ff1d4112c30 con 0x7ff1d4108620 2024-09-11T00:05:34.715 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:34.713+0000 7ff1d3e00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7ff1d4112ab0 con 0x7ff1d4108fa0 2024-09-11T00:05:34.715 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:34.713+0000 7ff1d3e00640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7ff1d4112db0 con 0x7ff1d4102c10 2024-09-11T00:05:34.715 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:34.713+0000 7ff1d2000640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff1d4108620 0x7ff1d419b7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:05:34.715 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:34.713+0000 7ff1d2000640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff1d4108620 0x7ff1d419b7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:60438/0 (socket says 172.21.15.12:60438) 2024-09-11T00:05:34.715 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:34.713+0000 7ff1d2000640 1 -- 172.21.15.12:0/2383692096 learned_addr learned my addr 172.21.15.12:0/2383692096 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:05:34.715 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:34.713+0000 7ff1d2a00640 1 --2- 172.21.15.12:0/2383692096 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff1d4102c10 0x7ff1d419b2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:05:34.715 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:34.713+0000 7ff1d3400640 1 --2- 172.21.15.12:0/2383692096 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff1d4108fa0 0x7ff1d41a0b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:05:34.716 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:34.713+0000 7ff1d2000640 1 -- 172.21.15.12:0/2383692096 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff1d4102c10 msgr2=0x7ff1d419b2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:05:34.716 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:34.713+0000 7ff1d2000640 1 --2- 172.21.15.12:0/2383692096 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff1d4102c10 0x7ff1d419b2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:05:34.716 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:34.713+0000 7ff1d2000640 1 -- 172.21.15.12:0/2383692096 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff1d4108fa0 msgr2=0x7ff1d41a0b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:05:34.717 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:34.713+0000 7ff1d2000640 1 --2- 172.21.15.12:0/2383692096 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff1d4108fa0 0x7ff1d41a0b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:05:34.717 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:34.713+0000 7ff1d2000640 1 -- 172.21.15.12:0/2383692096 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1d41a1350 con 0x7ff1d4108620 2024-09-11T00:05:34.717 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:34.715+0000 7ff1d2000640 1 --2- 172.21.15.12:0/2383692096 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff1d4108620 0x7ff1d419b7e0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7ff1c000e9f0 tx=0x7ff1c000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:05:34.717 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:34.715+0000 7ff1cb400640 1 -- 172.21.15.12:0/2383692096 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1c000cd80 con 0x7ff1d4108620 2024-09-11T00:05:34.717 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:34.715+0000 7ff1cb400640 1 -- 172.21.15.12:0/2383692096 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff1c00108e0 con 0x7ff1d4108620 2024-09-11T00:05:34.717 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:34.715+0000 7ff1d3e00640 1 -- 172.21.15.12:0/2383692096 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff1d41a1640 con 0x7ff1d4108620 2024-09-11T00:05:34.717 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:34.715+0000 7ff1cb400640 1 -- 172.21.15.12:0/2383692096 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1c0018980 con 0x7ff1d4108620 2024-09-11T00:05:34.717 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:34.715+0000 7ff1d3e00640 1 -- 172.21.15.12:0/2383692096 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1d41a7d00 con 0x7ff1d4108620 2024-09-11T00:05:34.718 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:34.716+0000 7ff1d3e00640 1 -- 172.21.15.12:0/2383692096 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff1d4102ff0 con 0x7ff1d4108620 2024-09-11T00:05:34.719 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:34.718+0000 7ff1cb400640 1 -- 172.21.15.12:0/2383692096 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 17) ==== 153301+0+0 (secure 0 0 0) 0x7ff1c0010430 con 0x7ff1d4108620 2024-09-11T00:05:34.720 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:34.720+0000 7ff1cb400640 1 --2- 172.21.15.12:0/2383692096 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7ff1b80b5410 0x7ff1b80b78d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:05:34.721 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:34.720+0000 7ff1cb400640 1 -- 172.21.15.12:0/2383692096 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 9092+0+0 (secure 0 0 0) 0x7ff1c00e6410 con 0x7ff1d4108620 2024-09-11T00:05:34.721 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:34.720+0000 7ff1d2a00640 1 --2- 172.21.15.12:0/2383692096 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7ff1b80b5410 0x7ff1b80b78d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:05:34.722 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:34.721+0000 7ff1d2a00640 1 --2- 172.21.15.12:0/2383692096 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7ff1b80b5410 0x7ff1b80b78d0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7ff1c4031440 tx=0x7ff1c4005b50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:05:34.725 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:34.724+0000 7ff1cb400640 1 -- 172.21.15.12:0/2383692096 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7ff1c00ac760 con 0x7ff1d4108620 2024-09-11T00:05:34.820 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:34.587+0000 7f09b3000640 1 -- 172.21.15.12:0/1755059373 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f09ac10a820 msgr2=0x7f09ac10ac60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:05:34.820 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:34.587+0000 7f09b3000640 1 --2- 172.21.15.12:0/1755059373 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f09ac10a820 0x7f09ac10ac60 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f0998009900 tx=0x7f099802a410 comp rx=0 tx=0).stop 2024-09-11T00:05:34.820 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:34.588+0000 7f09b3000640 1 -- 172.21.15.12:0/1755059373 shutdown_connections 2024-09-11T00:05:34.821 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:34.588+0000 7f09b3000640 1 --2- 172.21.15.12:0/1755059373 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f09ac10b1a0 0x7f09ac1119a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:05:34.821 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:34.588+0000 7f09b3000640 1 --2- 172.21.15.12:0/1755059373 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f09ac10a820 0x7f09ac10ac60 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:05:34.821 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:34.588+0000 7f09b3000640 1 --2- 172.21.15.12:0/1755059373 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f09ac104e10 0x7f09ac1051f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:05:34.821 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:34.588+0000 7f09b3000640 1 -- 172.21.15.12:0/1755059373 >> 172.21.15.12:0/1755059373 conn(0x7f09ac100840 msgr2=0x7f09ac102c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:05:34.821 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:34.588+0000 7f09b3000640 1 -- 172.21.15.12:0/1755059373 shutdown_connections 2024-09-11T00:05:34.821 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:34.588+0000 7f09b3000640 1 -- 172.21.15.12:0/1755059373 wait complete. 2024-09-11T00:05:34.821 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:34.590+0000 7f09b3000640 1 Processor -- start 2024-09-11T00:05:34.821 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:34.590+0000 7f09b3000640 1 -- start start 2024-09-11T00:05:34.821 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:34.590+0000 7f09b3000640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f09ac104e10 0x7f09ac19b240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:05:34.821 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:34.590+0000 7f09b3000640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f09ac10a820 0x7f09ac19b780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:05:34.821 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:34.590+0000 7f09b3000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f09ac10b1a0 0x7f09ac1a0b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:05:34.822 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:34.590+0000 7f09b3000640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f09ac114eb0 con 0x7f09ac104e10 2024-09-11T00:05:34.822 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:34.590+0000 7f09b3000640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f09ac114d30 con 0x7f09ac10b1a0 2024-09-11T00:05:34.822 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:34.590+0000 7f09b3000640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f09ac115030 con 0x7f09ac10a820 2024-09-11T00:05:34.822 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:34.590+0000 7f09b1c00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f09ac104e10 0x7f09ac19b240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:05:34.822 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:34.590+0000 7f09b1c00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f09ac104e10 0x7f09ac19b240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:60424/0 (socket says 172.21.15.12:60424) 2024-09-11T00:05:34.822 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:34.590+0000 7f09b1c00640 1 -- 172.21.15.12:0/3556308714 learned_addr learned my addr 172.21.15.12:0/3556308714 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:05:34.822 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:34.590+0000 7f09b1200640 1 --2- 172.21.15.12:0/3556308714 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f09ac10a820 0x7f09ac19b780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:05:34.822 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:34.590+0000 7f09b2600640 1 --2- 172.21.15.12:0/3556308714 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f09ac10b1a0 0x7f09ac1a0b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:05:34.822 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:34.590+0000 7f09b1c00640 1 -- 172.21.15.12:0/3556308714 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f09ac10a820 msgr2=0x7f09ac19b780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:05:34.822 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:34.590+0000 7f09b1c00640 1 --2- 172.21.15.12:0/3556308714 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f09ac10a820 0x7f09ac19b780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:05:34.822 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:34.590+0000 7f09b1c00640 1 -- 172.21.15.12:0/3556308714 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f09ac10b1a0 msgr2=0x7f09ac1a0b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:05:34.822 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:34.590+0000 7f09b1c00640 1 --2- 172.21.15.12:0/3556308714 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f09ac10b1a0 0x7f09ac1a0b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:05:34.823 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:34.590+0000 7f09b1c00640 1 -- 172.21.15.12:0/3556308714 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09ac1a12f0 con 0x7f09ac104e10 2024-09-11T00:05:34.823 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:34.590+0000 7f09b1c00640 1 --2- 172.21.15.12:0/3556308714 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f09ac104e10 0x7f09ac19b240 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f09a000e9f0 tx=0x7f09a000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:05:34.823 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:34.592+0000 7f09aa000640 1 -- 172.21.15.12:0/3556308714 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09a000cd80 con 0x7f09ac104e10 2024-09-11T00:05:34.823 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:34.592+0000 7f09aa000640 1 -- 172.21.15.12:0/3556308714 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f09a00108e0 con 0x7f09ac104e10 2024-09-11T00:05:34.823 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:34.592+0000 7f09b3000640 1 -- 172.21.15.12:0/3556308714 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f09ac1a15e0 con 0x7f09ac104e10 2024-09-11T00:05:34.823 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:34.592+0000 7f09aa000640 1 -- 172.21.15.12:0/3556308714 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09a0018980 con 0x7f09ac104e10 2024-09-11T00:05:34.823 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:34.592+0000 7f09b3000640 1 -- 172.21.15.12:0/3556308714 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09ac1a7e70 con 0x7f09ac104e10 2024-09-11T00:05:34.823 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:34.593+0000 7f09b3000640 1 -- 172.21.15.12:0/3556308714 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f09ac1060c0 con 0x7f09ac104e10 2024-09-11T00:05:34.823 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:34.595+0000 7f09aa000640 1 -- 172.21.15.12:0/3556308714 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 17) ==== 153301+0+0 (secure 0 0 0) 0x7f09a0010430 con 0x7f09ac104e10 2024-09-11T00:05:34.823 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:34.596+0000 7f09aa000640 1 --2- 172.21.15.12:0/3556308714 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f09940b5490 0x7f09940b7950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:05:34.823 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:34.596+0000 7f09aa000640 1 -- 172.21.15.12:0/3556308714 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 9092+0+0 (secure 0 0 0) 0x7f09a00e63f0 con 0x7f09ac104e10 2024-09-11T00:05:34.823 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:34.596+0000 7f09b1200640 1 --2- 172.21.15.12:0/3556308714 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f09940b5490 0x7f09940b7950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:05:34.824 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:34.597+0000 7f09b1200640 1 --2- 172.21.15.12:0/3556308714 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f09940b5490 0x7f09940b7950 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f0998005ab0 tx=0x7f0998005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:05:34.824 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:34.600+0000 7f09aa000640 1 -- 172.21.15.12:0/3556308714 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f09a00ac740 con 0x7f09ac104e10 2024-09-11T00:05:34.824 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:34.823+0000 7f09b3000640 1 -- 172.21.15.12:0/3556308714 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7f09ac1a8040 con 0x7f09ac104e10 2024-09-11T00:05:34.824 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:34.824+0000 7f09aa000640 1 -- 172.21.15.12:0/3556308714 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f09a00b1170 con 0x7f09ac104e10 2024-09-11T00:05:34.825 INFO:teuthology.orchestra.run.smithi012.stdout:133143986244 2024-09-11T00:05:34.827 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:34.826+0000 7f09b3000640 1 -- 172.21.15.12:0/3556308714 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f09940b5490 msgr2=0x7f09940b7950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:05:34.827 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:34.826+0000 7f09b3000640 1 --2- 172.21.15.12:0/3556308714 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f09940b5490 0x7f09940b7950 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f0998005ab0 tx=0x7f0998005a20 comp rx=0 tx=0).stop 2024-09-11T00:05:34.827 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:34.826+0000 7f09b3000640 1 -- 172.21.15.12:0/3556308714 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f09ac104e10 msgr2=0x7f09ac19b240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:05:34.827 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:34.826+0000 7f09b3000640 1 --2- 172.21.15.12:0/3556308714 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f09ac104e10 0x7f09ac19b240 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f09a000e9f0 tx=0x7f09a000eeb0 comp rx=0 tx=0).stop 2024-09-11T00:05:34.828 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:34.826+0000 7f09b3000640 1 -- 172.21.15.12:0/3556308714 shutdown_connections 2024-09-11T00:05:34.828 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:34.826+0000 7f09b3000640 1 --2- 172.21.15.12:0/3556308714 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f09940b5490 0x7f09940b7950 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:05:34.828 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:34.826+0000 7f09b3000640 1 --2- 172.21.15.12:0/3556308714 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f09ac10b1a0 0x7f09ac1a0b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:05:34.828 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:34.826+0000 7f09b3000640 1 --2- 172.21.15.12:0/3556308714 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f09ac10a820 0x7f09ac19b780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:05:34.828 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:34.826+0000 7f09b3000640 1 --2- 172.21.15.12:0/3556308714 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f09ac104e10 0x7f09ac19b240 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:05:34.828 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:34.826+0000 7f09b3000640 1 -- 172.21.15.12:0/3556308714 >> 172.21.15.12:0/3556308714 conn(0x7f09ac100840 msgr2=0x7f09ac102c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:05:34.828 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:34.826+0000 7f09b3000640 1 -- 172.21.15.12:0/3556308714 shutdown_connections 2024-09-11T00:05:34.828 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:34.826+0000 7f09b3000640 1 -- 172.21.15.12:0/3556308714 wait complete. 2024-09-11T00:05:34.931 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:34.930+0000 7ff1d3e00640 1 -- 172.21.15.12:0/2383692096 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7ff1d402e5c0 con 0x7ff1d4108620 2024-09-11T00:05:34.931 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:34.930+0000 7ff1cb400640 1 -- 172.21.15.12:0/2383692096 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7ff1c00b1190 con 0x7ff1d4108620 2024-09-11T00:05:34.931 INFO:teuthology.orchestra.run.smithi012.stdout:34359738459 2024-09-11T00:05:34.933 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:34.932+0000 7ff1d3e00640 1 -- 172.21.15.12:0/2383692096 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7ff1b80b5410 msgr2=0x7ff1b80b78d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:05:34.933 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:34.932+0000 7ff1d3e00640 1 --2- 172.21.15.12:0/2383692096 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7ff1b80b5410 0x7ff1b80b78d0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7ff1c4031440 tx=0x7ff1c4005b50 comp rx=0 tx=0).stop 2024-09-11T00:05:34.934 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:34.932+0000 7ff1d3e00640 1 -- 172.21.15.12:0/2383692096 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff1d4108620 msgr2=0x7ff1d419b7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:05:34.934 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:34.932+0000 7ff1d3e00640 1 --2- 172.21.15.12:0/2383692096 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff1d4108620 0x7ff1d419b7e0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7ff1c000e9f0 tx=0x7ff1c000eeb0 comp rx=0 tx=0).stop 2024-09-11T00:05:34.934 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:34.932+0000 7ff1d3e00640 1 -- 172.21.15.12:0/2383692096 shutdown_connections 2024-09-11T00:05:34.934 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:34.932+0000 7ff1d3e00640 1 --2- 172.21.15.12:0/2383692096 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7ff1b80b5410 0x7ff1b80b78d0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:05:34.934 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:34.932+0000 7ff1d3e00640 1 --2- 172.21.15.12:0/2383692096 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff1d4108fa0 0x7ff1d41a0b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:05:34.934 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:34.932+0000 7ff1d3e00640 1 --2- 172.21.15.12:0/2383692096 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff1d4108620 0x7ff1d419b7e0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:05:34.934 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:34.932+0000 7ff1d3e00640 1 --2- 172.21.15.12:0/2383692096 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff1d4102c10 0x7ff1d419b2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:05:34.934 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:34.932+0000 7ff1d3e00640 1 -- 172.21.15.12:0/2383692096 >> 172.21.15.12:0/2383692096 conn(0x7ff1d40fe660 msgr2=0x7ff1d4104bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:05:34.934 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:34.932+0000 7ff1d3e00640 1 -- 172.21.15.12:0/2383692096 shutdown_connections 2024-09-11T00:05:34.934 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:34.932+0000 7ff1d3e00640 1 -- 172.21.15.12:0/2383692096 wait complete. 2024-09-11T00:05:34.961 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:05:34 smithi012 ceph-mon[30333]: from='client.? 172.21.15.12:0/3556308714' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 4} : dispatch 2024-09-11T00:05:35.283 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:05:34 smithi165 ceph-mon[35356]: from='client.? 172.21.15.12:0/3556308714' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 4} : dispatch 2024-09-11T00:05:35.330 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:05:34 smithi178 ceph-mon[35919]: from='client.? 172.21.15.12:0/3556308714' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 4} : dispatch 2024-09-11T00:05:35.464 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:05:35.466 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:05:35.491 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:05:35.496 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:05:35.497 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:05:35.512 INFO:teuthology.orchestra.run.smithi012.stdout:201863462963 2024-09-11T00:05:35.513 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid ff3ad74a-6fcf-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.7 2024-09-11T00:05:36.185 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:05:35 smithi012 ceph-mon[30333]: pgmap v351: 129 pgs: 129 active+clean; 647 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 170 B/s wr, 8 op/s; 9 B/s, 0 objects/s recovering 2024-09-11T00:05:36.185 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:05:35 smithi012 ceph-mon[30333]: from='client.? 172.21.15.12:0/2383692096' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 0} : dispatch 2024-09-11T00:05:36.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:05:35 smithi165 ceph-mon[35356]: pgmap v351: 129 pgs: 129 active+clean; 647 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 170 B/s wr, 8 op/s; 9 B/s, 0 objects/s recovering 2024-09-11T00:05:36.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:05:35 smithi165 ceph-mon[35356]: from='client.? 172.21.15.12:0/2383692096' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 0} : dispatch 2024-09-11T00:05:36.330 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:05:35 smithi178 ceph-mon[35919]: pgmap v351: 129 pgs: 129 active+clean; 647 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 170 B/s wr, 8 op/s; 9 B/s, 0 objects/s recovering 2024-09-11T00:05:36.330 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:05:35 smithi178 ceph-mon[35919]: from='client.? 172.21.15.12:0/2383692096' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 0} : dispatch 2024-09-11T00:05:36.796 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574930 got 55834574932 for osd.1 2024-09-11T00:05:36.796 DEBUG:teuthology.parallel:result is None 2024-09-11T00:05:37.065 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:05:36 smithi012 ceph-mon[30333]: pgmap v352: 129 pgs: 129 active+clean; 647 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 170 B/s wr, 12 op/s; 9 B/s, 0 objects/s recovering 2024-09-11T00:05:37.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:05:36 smithi165 ceph-mon[35356]: pgmap v352: 129 pgs: 129 active+clean; 647 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 170 B/s wr, 12 op/s; 9 B/s, 0 objects/s recovering 2024-09-11T00:05:37.330 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:05:36 smithi178 ceph-mon[35919]: pgmap v352: 129 pgs: 129 active+clean; 647 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 170 B/s wr, 12 op/s; 9 B/s, 0 objects/s recovering 2024-09-11T00:05:37.430 INFO:tasks.cephadm.ceph_manager.ceph:need seq 133143986242 got 133143986244 for osd.4 2024-09-11T00:05:37.430 DEBUG:teuthology.parallel:result is None 2024-09-11T00:05:37.812 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:37.095+0000 7efbfd800640 1 -- 172.21.15.12:0/1102734220 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efbf810b1a0 msgr2=0x7efbf81119a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:05:37.812 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:37.095+0000 7efbfd800640 1 --2- 172.21.15.12:0/1102734220 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efbf810b1a0 0x7efbf81119a0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7efbf0009ca0 tx=0x7efbf002a8c0 comp rx=0 tx=0).stop 2024-09-11T00:05:37.812 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:37.095+0000 7efbfd800640 1 -- 172.21.15.12:0/1102734220 shutdown_connections 2024-09-11T00:05:37.812 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:37.095+0000 7efbfd800640 1 --2- 172.21.15.12:0/1102734220 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efbf810b1a0 0x7efbf81119a0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:05:37.812 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:37.095+0000 7efbfd800640 1 --2- 172.21.15.12:0/1102734220 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7efbf810a820 0x7efbf810ac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:05:37.812 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:37.095+0000 7efbfd800640 1 --2- 172.21.15.12:0/1102734220 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efbf8104e10 0x7efbf81051f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:05:37.812 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:37.095+0000 7efbfd800640 1 -- 172.21.15.12:0/1102734220 >> 172.21.15.12:0/1102734220 conn(0x7efbf8100840 msgr2=0x7efbf8102c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:05:37.812 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:37.095+0000 7efbfd800640 1 -- 172.21.15.12:0/1102734220 shutdown_connections 2024-09-11T00:05:37.812 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:37.095+0000 7efbfd800640 1 -- 172.21.15.12:0/1102734220 wait complete. 2024-09-11T00:05:37.812 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:37.096+0000 7efbfd800640 1 Processor -- start 2024-09-11T00:05:37.813 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:37.096+0000 7efbfd800640 1 -- start start 2024-09-11T00:05:37.813 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:37.097+0000 7efbfd800640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efbf8104e10 0x7efbf819b250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:05:37.813 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:37.097+0000 7efbfd800640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efbf810a820 0x7efbf819b790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:05:37.813 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:37.097+0000 7efbfd800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7efbf810b1a0 0x7efbf81a0b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:05:37.813 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:37.097+0000 7efbfd800640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7efbf8114e20 con 0x7efbf810a820 2024-09-11T00:05:37.813 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:37.097+0000 7efbfd800640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7efbf8114ca0 con 0x7efbf810b1a0 2024-09-11T00:05:37.813 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:37.097+0000 7efbfd800640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7efbf8114fa0 con 0x7efbf8104e10 2024-09-11T00:05:37.813 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:37.097+0000 7efbf7400640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efbf810a820 0x7efbf819b790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:05:37.813 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:37.097+0000 7efbf7400640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efbf810a820 0x7efbf819b790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:60462/0 (socket says 172.21.15.12:60462) 2024-09-11T00:05:37.813 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:37.097+0000 7efbf7400640 1 -- 172.21.15.12:0/139493509 learned_addr learned my addr 172.21.15.12:0/139493509 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:05:37.813 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:37.097+0000 7efbf7e00640 1 --2- 172.21.15.12:0/139493509 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efbf8104e10 0x7efbf819b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:05:37.814 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:37.097+0000 7efbfce00640 1 --2- 172.21.15.12:0/139493509 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7efbf810b1a0 0x7efbf81a0b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:05:37.814 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:37.097+0000 7efbf7400640 1 -- 172.21.15.12:0/139493509 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efbf8104e10 msgr2=0x7efbf819b250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:05:37.814 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:37.097+0000 7efbf7400640 1 --2- 172.21.15.12:0/139493509 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efbf8104e10 0x7efbf819b250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:05:37.814 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:37.097+0000 7efbf7400640 1 -- 172.21.15.12:0/139493509 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7efbf810b1a0 msgr2=0x7efbf81a0b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:05:37.814 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:37.097+0000 7efbf7400640 1 --2- 172.21.15.12:0/139493509 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7efbf810b1a0 0x7efbf81a0b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:05:37.814 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:37.097+0000 7efbf7400640 1 -- 172.21.15.12:0/139493509 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efbf81a1300 con 0x7efbf810a820 2024-09-11T00:05:37.814 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:37.098+0000 7efbf7400640 1 --2- 172.21.15.12:0/139493509 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efbf810a820 0x7efbf819b790 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7efbe400e9f0 tx=0x7efbe400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:05:37.814 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:37.098+0000 7efbf4c00640 1 -- 172.21.15.12:0/139493509 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efbe400cd80 con 0x7efbf810a820 2024-09-11T00:05:37.814 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:37.098+0000 7efbf4c00640 1 -- 172.21.15.12:0/139493509 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efbe40108e0 con 0x7efbf810a820 2024-09-11T00:05:37.814 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:37.098+0000 7efbf4c00640 1 -- 172.21.15.12:0/139493509 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efbe4018980 con 0x7efbf810a820 2024-09-11T00:05:37.814 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:37.098+0000 7efbfd800640 1 -- 172.21.15.12:0/139493509 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efbf81a15f0 con 0x7efbf810a820 2024-09-11T00:05:37.814 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:37.098+0000 7efbfd800640 1 -- 172.21.15.12:0/139493509 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efbf81a7e80 con 0x7efbf810a820 2024-09-11T00:05:37.814 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:37.100+0000 7efbfd800640 1 -- 172.21.15.12:0/139493509 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efbf81051f0 con 0x7efbf810a820 2024-09-11T00:05:37.815 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:37.102+0000 7efbf4c00640 1 -- 172.21.15.12:0/139493509 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 17) ==== 153301+0+0 (secure 0 0 0) 0x7efbe4010430 con 0x7efbf810a820 2024-09-11T00:05:37.815 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:37.104+0000 7efbf4c00640 1 --2- 172.21.15.12:0/139493509 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7efbdc0b5490 0x7efbdc0b7950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:05:37.815 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:37.104+0000 7efbf4c00640 1 -- 172.21.15.12:0/139493509 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 9092+0+0 (secure 0 0 0) 0x7efbe40e6300 con 0x7efbf810a820 2024-09-11T00:05:37.815 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:37.104+0000 7efbf7e00640 1 --2- 172.21.15.12:0/139493509 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7efbdc0b5490 0x7efbdc0b7950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:05:37.815 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:37.105+0000 7efbf7e00640 1 --2- 172.21.15.12:0/139493509 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7efbdc0b5490 0x7efbdc0b7950 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7efbe8009bd0 tx=0x7efbe8005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:05:37.815 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:37.107+0000 7efbf4c00640 1 -- 172.21.15.12:0/139493509 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7efbe40ac6b0 con 0x7efbf810a820 2024-09-11T00:05:37.815 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:37.311+0000 7efbfd800640 1 -- 172.21.15.12:0/139493509 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 10} v 0) -- 0x7efbf810cf80 con 0x7efbf810a820 2024-09-11T00:05:37.815 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:37.311+0000 7efbf4c00640 1 -- 172.21.15.12:0/139493509 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 10}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7efbe40b10e0 con 0x7efbf810a820 2024-09-11T00:05:37.815 INFO:teuthology.orchestra.run.smithi012.stdout:270582939686 2024-09-11T00:05:37.816 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:37.315+0000 7efbfd800640 1 -- 172.21.15.12:0/139493509 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7efbdc0b5490 msgr2=0x7efbdc0b7950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:05:37.816 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:37.315+0000 7efbfd800640 1 --2- 172.21.15.12:0/139493509 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7efbdc0b5490 0x7efbdc0b7950 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7efbe8009bd0 tx=0x7efbe8005f90 comp rx=0 tx=0).stop 2024-09-11T00:05:37.816 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:37.315+0000 7efbfd800640 1 -- 172.21.15.12:0/139493509 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efbf810a820 msgr2=0x7efbf819b790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:05:37.816 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:37.315+0000 7efbfd800640 1 --2- 172.21.15.12:0/139493509 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efbf810a820 0x7efbf819b790 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7efbe400e9f0 tx=0x7efbe400eeb0 comp rx=0 tx=0).stop 2024-09-11T00:05:37.816 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:37.315+0000 7efbfd800640 1 -- 172.21.15.12:0/139493509 shutdown_connections 2024-09-11T00:05:37.816 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:37.315+0000 7efbfd800640 1 --2- 172.21.15.12:0/139493509 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7efbdc0b5490 0x7efbdc0b7950 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:05:37.816 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:37.315+0000 7efbfd800640 1 --2- 172.21.15.12:0/139493509 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7efbf810b1a0 0x7efbf81a0b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:05:37.816 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:37.315+0000 7efbfd800640 1 --2- 172.21.15.12:0/139493509 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efbf810a820 0x7efbf819b790 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:05:37.816 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:37.315+0000 7efbfd800640 1 --2- 172.21.15.12:0/139493509 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efbf8104e10 0x7efbf819b250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:05:37.816 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:37.315+0000 7efbfd800640 1 -- 172.21.15.12:0/139493509 >> 172.21.15.12:0/139493509 conn(0x7efbf8100840 msgr2=0x7efbf81010d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:05:37.816 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:37.315+0000 7efbfd800640 1 -- 172.21.15.12:0/139493509 shutdown_connections 2024-09-11T00:05:37.817 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:37.315+0000 7efbfd800640 1 -- 172.21.15.12:0/139493509 wait complete. 2024-09-11T00:05:38.112 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:37.412+0000 7fd8f0c00640 1 -- 172.21.15.12:0/1614951165 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd8ec075540 msgr2=0x7fd8ec075920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:05:38.113 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:37.412+0000 7fd8f0c00640 1 --2- 172.21.15.12:0/1614951165 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd8ec075540 0x7fd8ec075920 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fd8dc009860 tx=0x7fd8dc02a340 comp rx=0 tx=0).stop 2024-09-11T00:05:38.113 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:37.412+0000 7fd8f0c00640 1 -- 172.21.15.12:0/1614951165 shutdown_connections 2024-09-11T00:05:38.113 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:37.412+0000 7fd8f0c00640 1 --2- 172.21.15.12:0/1614951165 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd8ec10f3e0 0x7fd8ec1117d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:05:38.113 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:37.412+0000 7fd8f0c00640 1 --2- 172.21.15.12:0/1614951165 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd8ec075f40 0x7fd8ec10ee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:05:38.113 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:37.412+0000 7fd8f0c00640 1 --2- 172.21.15.12:0/1614951165 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd8ec075540 0x7fd8ec075920 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:05:38.113 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:37.412+0000 7fd8f0c00640 1 -- 172.21.15.12:0/1614951165 >> 172.21.15.12:0/1614951165 conn(0x7fd8ec0fe830 msgr2=0x7fd8ec100c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:05:38.113 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:37.412+0000 7fd8f0c00640 1 -- 172.21.15.12:0/1614951165 shutdown_connections 2024-09-11T00:05:38.113 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:37.413+0000 7fd8f0c00640 1 -- 172.21.15.12:0/1614951165 wait complete. 2024-09-11T00:05:38.113 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:37.414+0000 7fd8f0c00640 1 Processor -- start 2024-09-11T00:05:38.113 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:37.414+0000 7fd8f0c00640 1 -- start start 2024-09-11T00:05:38.113 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:37.415+0000 7fd8f0c00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd8ec075540 0x7fd8ec19b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:05:38.114 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:37.415+0000 7fd8f0c00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd8ec075f40 0x7fd8ec19ba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:05:38.114 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:37.415+0000 7fd8f0c00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd8ec10f3e0 0x7fd8ec1a0dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:05:38.114 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:37.415+0000 7fd8f0c00640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fd8ec114f70 con 0x7fd8ec075f40 2024-09-11T00:05:38.114 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:37.415+0000 7fd8f0c00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fd8ec114df0 con 0x7fd8ec10f3e0 2024-09-11T00:05:38.114 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:37.415+0000 7fd8f0c00640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fd8ec1150f0 con 0x7fd8ec075540 2024-09-11T00:05:38.114 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:37.415+0000 7fd8e3e00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd8ec075f40 0x7fd8ec19ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:05:38.114 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:37.415+0000 7fd8e3e00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd8ec075f40 0x7fd8ec19ba20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:60486/0 (socket says 172.21.15.12:60486) 2024-09-11T00:05:38.114 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:37.415+0000 7fd8e3e00640 1 -- 172.21.15.12:0/2345596114 learned_addr learned my addr 172.21.15.12:0/2345596114 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:05:38.114 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:37.415+0000 7fd8e9e00640 1 --2- 172.21.15.12:0/2345596114 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd8ec075540 0x7fd8ec19b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:05:38.114 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:37.415+0000 7fd8ea800640 1 --2- 172.21.15.12:0/2345596114 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd8ec10f3e0 0x7fd8ec1a0dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:05:38.114 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:37.415+0000 7fd8e9e00640 1 -- 172.21.15.12:0/2345596114 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd8ec10f3e0 msgr2=0x7fd8ec1a0dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:05:38.115 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:37.416+0000 7fd8e9e00640 1 --2- 172.21.15.12:0/2345596114 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd8ec10f3e0 0x7fd8ec1a0dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:05:38.115 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:37.416+0000 7fd8e9e00640 1 -- 172.21.15.12:0/2345596114 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd8ec075f40 msgr2=0x7fd8ec19ba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:05:38.115 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:37.416+0000 7fd8e9e00640 1 --2- 172.21.15.12:0/2345596114 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd8ec075f40 0x7fd8ec19ba20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:05:38.115 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:37.416+0000 7fd8e9e00640 1 -- 172.21.15.12:0/2345596114 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8ec1a1590 con 0x7fd8ec075540 2024-09-11T00:05:38.115 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:37.416+0000 7fd8e9e00640 1 --2- 172.21.15.12:0/2345596114 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd8ec075540 0x7fd8ec19b4e0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fd8dc035040 tx=0x7fd8dc032dc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:05:38.115 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:37.416+0000 7fd8e2a00640 1 -- 172.21.15.12:0/2345596114 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8dc040020 con 0x7fd8ec075540 2024-09-11T00:05:38.115 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:37.416+0000 7fd8f0c00640 1 -- 172.21.15.12:0/2345596114 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd8ec1a1820 con 0x7fd8ec075540 2024-09-11T00:05:38.115 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:37.416+0000 7fd8e2a00640 1 -- 172.21.15.12:0/2345596114 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd8dc030450 con 0x7fd8ec075540 2024-09-11T00:05:38.115 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:37.416+0000 7fd8e2a00640 1 -- 172.21.15.12:0/2345596114 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8dc037040 con 0x7fd8ec075540 2024-09-11T00:05:38.115 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:37.417+0000 7fd8f0c00640 1 -- 172.21.15.12:0/2345596114 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd8ec1a7f40 con 0x7fd8ec075540 2024-09-11T00:05:38.115 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:37.418+0000 7fd8f0c00640 1 -- 172.21.15.12:0/2345596114 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd8ec075920 con 0x7fd8ec075540 2024-09-11T00:05:38.116 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:37.420+0000 7fd8e2a00640 1 -- 172.21.15.12:0/2345596114 <== mon.1 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 17) ==== 153301+0+0 (secure 0 0 0) 0x7fd8dc004040 con 0x7fd8ec075540 2024-09-11T00:05:38.116 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:37.422+0000 7fd8e2a00640 1 --2- 172.21.15.12:0/2345596114 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fd8d00b53f0 0x7fd8d00b78b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:05:38.116 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:37.423+0000 7fd8e2a00640 1 -- 172.21.15.12:0/2345596114 <== mon.1 v2:172.21.15.178:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 9092+0+0 (secure 0 0 0) 0x7fd8dc105ec0 con 0x7fd8ec075540 2024-09-11T00:05:38.116 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:37.423+0000 7fd8e3e00640 1 --2- 172.21.15.12:0/2345596114 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fd8d00b53f0 0x7fd8d00b78b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:05:38.116 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:37.424+0000 7fd8e3e00640 1 --2- 172.21.15.12:0/2345596114 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fd8d00b53f0 0x7fd8d00b78b0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fd8ec111a40 tx=0x7fd8d4009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:05:38.116 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:37.426+0000 7fd8e2a00640 1 -- 172.21.15.12:0/2345596114 <== mon.1 v2:172.21.15.178:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7fd8dc04a020 con 0x7fd8ec075540 2024-09-11T00:05:38.116 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:37.643+0000 7fd8f0c00640 1 -- 172.21.15.12:0/2345596114 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7fd8ec1a8270 con 0x7fd8ec075540 2024-09-11T00:05:38.116 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:37.644+0000 7fd8e2a00640 1 -- 172.21.15.12:0/2345596114 <== mon.1 v2:172.21.15.178:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fd8dc0cc860 con 0x7fd8ec075540 2024-09-11T00:05:38.117 INFO:teuthology.orchestra.run.smithi012.stdout:154618822720 2024-09-11T00:05:38.117 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:37.647+0000 7fd8f0c00640 1 -- 172.21.15.12:0/2345596114 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fd8d00b53f0 msgr2=0x7fd8d00b78b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:05:38.117 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:37.647+0000 7fd8f0c00640 1 --2- 172.21.15.12:0/2345596114 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fd8d00b53f0 0x7fd8d00b78b0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fd8ec111a40 tx=0x7fd8d4009260 comp rx=0 tx=0).stop 2024-09-11T00:05:38.118 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:37.647+0000 7fd8f0c00640 1 -- 172.21.15.12:0/2345596114 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd8ec075540 msgr2=0x7fd8ec19b4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:05:38.118 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:37.647+0000 7fd8f0c00640 1 --2- 172.21.15.12:0/2345596114 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd8ec075540 0x7fd8ec19b4e0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fd8dc035040 tx=0x7fd8dc032dc0 comp rx=0 tx=0).stop 2024-09-11T00:05:38.118 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:37.647+0000 7fd8f0c00640 1 -- 172.21.15.12:0/2345596114 shutdown_connections 2024-09-11T00:05:38.118 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:37.647+0000 7fd8f0c00640 1 --2- 172.21.15.12:0/2345596114 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fd8d00b53f0 0x7fd8d00b78b0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:05:38.118 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:37.647+0000 7fd8f0c00640 1 --2- 172.21.15.12:0/2345596114 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd8ec10f3e0 0x7fd8ec1a0dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:05:38.118 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:37.647+0000 7fd8f0c00640 1 --2- 172.21.15.12:0/2345596114 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd8ec075f40 0x7fd8ec19ba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:05:38.118 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:37.647+0000 7fd8f0c00640 1 --2- 172.21.15.12:0/2345596114 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd8ec075540 0x7fd8ec19b4e0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:05:38.118 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:37.647+0000 7fd8f0c00640 1 -- 172.21.15.12:0/2345596114 >> 172.21.15.12:0/2345596114 conn(0x7fd8ec0fe830 msgr2=0x7fd8ec0ff430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:05:38.118 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:37.647+0000 7fd8f0c00640 1 -- 172.21.15.12:0/2345596114 shutdown_connections 2024-09-11T00:05:38.118 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:37.647+0000 7fd8f0c00640 1 -- 172.21.15.12:0/2345596114 wait complete. 2024-09-11T00:05:38.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:05:37 smithi165 ceph-mon[35356]: from='client.? 172.21.15.12:0/139493509' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 10} : dispatch 2024-09-11T00:05:38.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:05:37 smithi165 ceph-mon[35356]: from='client.? 172.21.15.12:0/2345596114' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 5} : dispatch 2024-09-11T00:05:38.300 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:05:37 smithi012 ceph-mon[30333]: from='client.? 172.21.15.12:0/139493509' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 10} : dispatch 2024-09-11T00:05:38.300 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:05:37 smithi012 ceph-mon[30333]: from='client.? 172.21.15.12:0/2345596114' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 5} : dispatch 2024-09-11T00:05:38.330 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:05:37 smithi178 ceph-mon[35919]: from='client.? 172.21.15.12:0/139493509' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 10} : dispatch 2024-09-11T00:05:38.330 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:05:37 smithi178 ceph-mon[35919]: from='client.? 172.21.15.12:0/2345596114' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 5} : dispatch 2024-09-11T00:05:38.585 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:38.412+0000 7fb84f600640 1 -- 172.21.15.12:0/2922433580 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb848102da0 msgr2=0x7fb848103180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:05:38.585 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:38.412+0000 7fb84f600640 1 --2- 172.21.15.12:0/2922433580 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb848102da0 0x7fb848103180 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fb83c009900 tx=0x7fb83c02a3f0 comp rx=0 tx=0).stop 2024-09-11T00:05:38.586 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:38.412+0000 7fb84f600640 1 -- 172.21.15.12:0/2922433580 shutdown_connections 2024-09-11T00:05:38.586 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:38.412+0000 7fb84f600640 1 --2- 172.21.15.12:0/2922433580 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb848109130 0x7fb84810f930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:05:38.586 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:38.412+0000 7fb84f600640 1 --2- 172.21.15.12:0/2922433580 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb8481087b0 0x7fb848108bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:05:38.586 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:38.412+0000 7fb84f600640 1 --2- 172.21.15.12:0/2922433580 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb848102da0 0x7fb848103180 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:05:38.586 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:38.412+0000 7fb84f600640 1 -- 172.21.15.12:0/2922433580 >> 172.21.15.12:0/2922433580 conn(0x7fb8480fe810 msgr2=0x7fb848100c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:05:38.586 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:38.412+0000 7fb84f600640 1 -- 172.21.15.12:0/2922433580 shutdown_connections 2024-09-11T00:05:38.586 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:38.412+0000 7fb84f600640 1 -- 172.21.15.12:0/2922433580 wait complete. 2024-09-11T00:05:38.586 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:38.413+0000 7fb84f600640 1 Processor -- start 2024-09-11T00:05:38.586 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:38.414+0000 7fb84f600640 1 -- start start 2024-09-11T00:05:38.586 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:38.414+0000 7fb84f600640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb848102da0 0x7fb84819b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:05:38.586 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:38.414+0000 7fb84f600640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb8481087b0 0x7fb84819ba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:05:38.586 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:38.414+0000 7fb84f600640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb848109130 0x7fb8481a0dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:05:38.586 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:38.414+0000 7fb84f600640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fb848112df0 con 0x7fb848109130 2024-09-11T00:05:38.587 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:38.414+0000 7fb84f600640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fb848112c70 con 0x7fb848102da0 2024-09-11T00:05:38.587 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:38.414+0000 7fb84f600640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fb848112f70 con 0x7fb8481087b0 2024-09-11T00:05:38.587 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:38.414+0000 7fb84cc00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb848102da0 0x7fb84819b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:05:38.587 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:38.414+0000 7fb84cc00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb848102da0 0x7fb84819b4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.12:59342/0 (socket says 172.21.15.12:59342) 2024-09-11T00:05:38.587 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:38.414+0000 7fb84cc00640 1 -- 172.21.15.12:0/3312855266 learned_addr learned my addr 172.21.15.12:0/3312855266 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:05:38.587 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:38.414+0000 7fb847e00640 1 --2- 172.21.15.12:0/3312855266 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb8481087b0 0x7fb84819ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:05:38.587 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:38.414+0000 7fb84d600640 1 --2- 172.21.15.12:0/3312855266 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb848109130 0x7fb8481a0dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:05:38.587 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:38.415+0000 7fb847e00640 1 -- 172.21.15.12:0/3312855266 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb848102da0 msgr2=0x7fb84819b4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:05:38.587 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:38.415+0000 7fb847e00640 1 --2- 172.21.15.12:0/3312855266 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb848102da0 0x7fb84819b4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:05:38.587 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:38.415+0000 7fb847e00640 1 -- 172.21.15.12:0/3312855266 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb848109130 msgr2=0x7fb8481a0dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:05:38.587 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:38.415+0000 7fb847e00640 1 --2- 172.21.15.12:0/3312855266 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb848109130 0x7fb8481a0dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:05:38.587 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:38.415+0000 7fb847e00640 1 -- 172.21.15.12:0/3312855266 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8481a1590 con 0x7fb8481087b0 2024-09-11T00:05:38.587 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:38.415+0000 7fb84cc00640 1 --2- 172.21.15.12:0/3312855266 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb848102da0 0x7fb84819b4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:05:38.587 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:38.415+0000 7fb847e00640 1 --2- 172.21.15.12:0/3312855266 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb8481087b0 0x7fb84819ba20 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fb83400e9f0 tx=0x7fb83400edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:05:38.588 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:38.415+0000 7fb845600640 1 -- 172.21.15.12:0/3312855266 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb83400ccc0 con 0x7fb8481087b0 2024-09-11T00:05:38.588 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:38.416+0000 7fb84f600640 1 -- 172.21.15.12:0/3312855266 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb8481a1880 con 0x7fb8481087b0 2024-09-11T00:05:38.588 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:38.416+0000 7fb845600640 1 -- 172.21.15.12:0/3312855266 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb8340044d0 con 0x7fb8481087b0 2024-09-11T00:05:38.588 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:38.416+0000 7fb84f600640 1 -- 172.21.15.12:0/3312855266 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8481a7f40 con 0x7fb8481087b0 2024-09-11T00:05:38.588 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:38.416+0000 7fb845600640 1 -- 172.21.15.12:0/3312855266 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb83401c070 con 0x7fb8481087b0 2024-09-11T00:05:38.588 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:38.417+0000 7fb84f600640 1 -- 172.21.15.12:0/3312855266 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb848103180 con 0x7fb8481087b0 2024-09-11T00:05:38.588 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:38.419+0000 7fb845600640 1 -- 172.21.15.12:0/3312855266 <== mon.1 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 17) ==== 153301+0+0 (secure 0 0 0) 0x7fb8340047c0 con 0x7fb8481087b0 2024-09-11T00:05:38.588 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:38.421+0000 7fb845600640 1 --2- 172.21.15.12:0/3312855266 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fb8300b5370 0x7fb8300b7830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:05:38.588 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:38.421+0000 7fb845600640 1 -- 172.21.15.12:0/3312855266 <== mon.1 v2:172.21.15.178:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 9092+0+0 (secure 0 0 0) 0x7fb8340e6140 con 0x7fb8481087b0 2024-09-11T00:05:38.588 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:38.422+0000 7fb84cc00640 1 --2- 172.21.15.12:0/3312855266 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fb8300b5370 0x7fb8300b7830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:05:38.588 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:38.422+0000 7fb845600640 1 -- 172.21.15.12:0/3312855266 <== mon.1 v2:172.21.15.178:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7fb8340acb70 con 0x7fb8481087b0 2024-09-11T00:05:38.588 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:38.422+0000 7fb84cc00640 1 --2- 172.21.15.12:0/3312855266 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fb8300b5370 0x7fb8300b7830 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fb83c031440 tx=0x7fb83c0058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:05:38.624 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:38.623+0000 7fb84f600640 1 -- 172.21.15.12:0/3312855266 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 8} v 0) -- 0x7fb84802e5c0 con 0x7fb8481087b0 2024-09-11T00:05:38.625 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:38.624+0000 7fb845600640 1 -- 172.21.15.12:0/3312855266 <== mon.1 v2:172.21.15.178:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 8}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fb8340ac470 con 0x7fb8481087b0 2024-09-11T00:05:38.625 INFO:teuthology.orchestra.run.smithi012.stdout:223338299440 2024-09-11T00:05:38.627 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:38.626+0000 7fb84f600640 1 -- 172.21.15.12:0/3312855266 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fb8300b5370 msgr2=0x7fb8300b7830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:05:38.627 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:38.626+0000 7fb84f600640 1 --2- 172.21.15.12:0/3312855266 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fb8300b5370 0x7fb8300b7830 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fb83c031440 tx=0x7fb83c0058d0 comp rx=0 tx=0).stop 2024-09-11T00:05:38.627 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:38.626+0000 7fb84f600640 1 -- 172.21.15.12:0/3312855266 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb8481087b0 msgr2=0x7fb84819ba20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:05:38.627 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:38.626+0000 7fb84f600640 1 --2- 172.21.15.12:0/3312855266 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb8481087b0 0x7fb84819ba20 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fb83400e9f0 tx=0x7fb83400edf0 comp rx=0 tx=0).stop 2024-09-11T00:05:38.628 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:38.626+0000 7fb84f600640 1 -- 172.21.15.12:0/3312855266 shutdown_connections 2024-09-11T00:05:38.628 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:38.626+0000 7fb84f600640 1 --2- 172.21.15.12:0/3312855266 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fb8300b5370 0x7fb8300b7830 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:05:38.628 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:38.626+0000 7fb84f600640 1 --2- 172.21.15.12:0/3312855266 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb848109130 0x7fb8481a0dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:05:38.628 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:38.626+0000 7fb84f600640 1 --2- 172.21.15.12:0/3312855266 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb8481087b0 0x7fb84819ba20 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:05:38.628 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:38.626+0000 7fb84f600640 1 --2- 172.21.15.12:0/3312855266 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb848102da0 0x7fb84819b4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:05:38.628 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:38.626+0000 7fb84f600640 1 -- 172.21.15.12:0/3312855266 >> 172.21.15.12:0/3312855266 conn(0x7fb8480fe810 msgr2=0x7fb848104d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:05:38.628 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:38.626+0000 7fb84f600640 1 -- 172.21.15.12:0/3312855266 shutdown_connections 2024-09-11T00:05:38.628 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:38.626+0000 7fb84f600640 1 -- 172.21.15.12:0/3312855266 wait complete. 2024-09-11T00:05:38.926 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:38.805+0000 7fe0e1a00640 1 -- 172.21.15.12:0/894113029 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe0dc104fa0 msgr2=0x7fe0dc105380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:05:38.927 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:38.805+0000 7fe0e1a00640 1 --2- 172.21.15.12:0/894113029 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe0dc104fa0 0x7fe0dc105380 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fe0cc009900 tx=0x7fe0cc02a410 comp rx=0 tx=0).stop 2024-09-11T00:05:38.927 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:38.805+0000 7fe0e1a00640 1 -- 172.21.15.12:0/894113029 shutdown_connections 2024-09-11T00:05:38.927 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:38.805+0000 7fe0e1a00640 1 --2- 172.21.15.12:0/894113029 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe0dc10b330 0x7fe0dc111b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:05:38.927 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:38.805+0000 7fe0e1a00640 1 --2- 172.21.15.12:0/894113029 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe0dc10a9b0 0x7fe0dc10adf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:05:38.927 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:38.805+0000 7fe0e1a00640 1 --2- 172.21.15.12:0/894113029 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe0dc104fa0 0x7fe0dc105380 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:05:38.927 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:38.805+0000 7fe0e1a00640 1 -- 172.21.15.12:0/894113029 >> 172.21.15.12:0/894113029 conn(0x7fe0dc100a50 msgr2=0x7fe0dc102e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:05:38.927 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:38.805+0000 7fe0e1a00640 1 -- 172.21.15.12:0/894113029 shutdown_connections 2024-09-11T00:05:38.927 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:38.805+0000 7fe0e1a00640 1 -- 172.21.15.12:0/894113029 wait complete. 2024-09-11T00:05:38.927 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:38.806+0000 7fe0e1a00640 1 Processor -- start 2024-09-11T00:05:38.927 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:38.807+0000 7fe0e1a00640 1 -- start start 2024-09-11T00:05:38.927 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:38.807+0000 7fe0e1a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe0dc104fa0 0x7fe0dc19b460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:05:38.928 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:38.807+0000 7fe0e1a00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe0dc10a9b0 0x7fe0dc19b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:05:38.928 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:38.807+0000 7fe0e1a00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe0dc10b330 0x7fe0dc1a0d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:05:38.928 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:38.807+0000 7fe0e1a00640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fe0dc114fb0 con 0x7fe0dc10a9b0 2024-09-11T00:05:38.928 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:38.807+0000 7fe0e1a00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fe0dc114e30 con 0x7fe0dc104fa0 2024-09-11T00:05:38.928 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:38.807+0000 7fe0e1a00640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fe0dc115130 con 0x7fe0dc10b330 2024-09-11T00:05:38.928 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:38.807+0000 7fe0da000640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe0dc10a9b0 0x7fe0dc19b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:05:38.928 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:38.807+0000 7fe0daa00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe0dc104fa0 0x7fe0dc19b460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:05:38.928 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:38.807+0000 7fe0da000640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe0dc10a9b0 0x7fe0dc19b9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:60522/0 (socket says 172.21.15.12:60522) 2024-09-11T00:05:38.928 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:38.807+0000 7fe0daa00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe0dc104fa0 0x7fe0dc19b460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.12:59368/0 (socket says 172.21.15.12:59368) 2024-09-11T00:05:38.928 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:38.807+0000 7fe0da000640 1 -- 172.21.15.12:0/95914234 learned_addr learned my addr 172.21.15.12:0/95914234 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:05:38.928 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:38.808+0000 7fe0db400640 1 --2- 172.21.15.12:0/95914234 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe0dc10b330 0x7fe0dc1a0d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:05:38.928 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:38.808+0000 7fe0da000640 1 -- 172.21.15.12:0/95914234 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe0dc10b330 msgr2=0x7fe0dc1a0d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:05:38.929 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:38.808+0000 7fe0da000640 1 --2- 172.21.15.12:0/95914234 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe0dc10b330 0x7fe0dc1a0d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:05:38.929 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:38.808+0000 7fe0da000640 1 -- 172.21.15.12:0/95914234 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe0dc104fa0 msgr2=0x7fe0dc19b460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:05:38.929 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:38.808+0000 7fe0da000640 1 --2- 172.21.15.12:0/95914234 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe0dc104fa0 0x7fe0dc19b460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:05:38.929 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:38.808+0000 7fe0da000640 1 -- 172.21.15.12:0/95914234 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0dc1a1510 con 0x7fe0dc10a9b0 2024-09-11T00:05:38.929 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:38.808+0000 7fe0da000640 1 --2- 172.21.15.12:0/95914234 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe0dc10a9b0 0x7fe0dc19b9a0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fe0c800e9f0 tx=0x7fe0c800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:05:38.929 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:38.808+0000 7fe0d3400640 1 -- 172.21.15.12:0/95914234 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0c800cd80 con 0x7fe0dc10a9b0 2024-09-11T00:05:38.929 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:38.808+0000 7fe0d3400640 1 -- 172.21.15.12:0/95914234 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe0c80108e0 con 0x7fe0dc10a9b0 2024-09-11T00:05:38.929 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:38.808+0000 7fe0e1a00640 1 -- 172.21.15.12:0/95914234 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe0dc1a1800 con 0x7fe0dc10a9b0 2024-09-11T00:05:38.929 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:38.808+0000 7fe0d3400640 1 -- 172.21.15.12:0/95914234 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0c8018980 con 0x7fe0dc10a9b0 2024-09-11T00:05:38.929 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:38.809+0000 7fe0e1a00640 1 -- 172.21.15.12:0/95914234 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe0dc1a7ec0 con 0x7fe0dc10a9b0 2024-09-11T00:05:38.929 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:38.810+0000 7fe0e1a00640 1 -- 172.21.15.12:0/95914234 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe0dc105380 con 0x7fe0dc10a9b0 2024-09-11T00:05:38.929 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:38.812+0000 7fe0d3400640 1 -- 172.21.15.12:0/95914234 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 17) ==== 153301+0+0 (secure 0 0 0) 0x7fe0c8010430 con 0x7fe0dc10a9b0 2024-09-11T00:05:38.929 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:38.813+0000 7fe0d3400640 1 --2- 172.21.15.12:0/95914234 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fe0c00b5370 0x7fe0c00b7830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:05:38.929 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:38.814+0000 7fe0d3400640 1 -- 172.21.15.12:0/95914234 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 9092+0+0 (secure 0 0 0) 0x7fe0c80e64c0 con 0x7fe0dc10a9b0 2024-09-11T00:05:38.930 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:38.814+0000 7fe0daa00640 1 --2- 172.21.15.12:0/95914234 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fe0c00b5370 0x7fe0c00b7830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:05:38.930 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:38.814+0000 7fe0daa00640 1 --2- 172.21.15.12:0/95914234 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fe0c00b5370 0x7fe0c00b7830 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fe0cc005ab0 tx=0x7fe0cc005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:05:38.930 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:38.817+0000 7fe0d3400640 1 -- 172.21.15.12:0/95914234 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7fe0c80ace60 con 0x7fe0dc10a9b0 2024-09-11T00:05:39.025 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:39.024+0000 7fe0e1a00640 1 -- 172.21.15.12:0/95914234 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7fe0dc10cb80 con 0x7fe0dc10a9b0 2024-09-11T00:05:39.026 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:39.025+0000 7fe0d3400640 1 -- 172.21.15.12:0/95914234 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fe0c80ac760 con 0x7fe0dc10a9b0 2024-09-11T00:05:39.026 INFO:teuthology.orchestra.run.smithi012.stdout:180388626491 2024-09-11T00:05:39.029 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:39.027+0000 7fe0e1a00640 1 -- 172.21.15.12:0/95914234 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fe0c00b5370 msgr2=0x7fe0c00b7830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:05:39.029 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:39.027+0000 7fe0e1a00640 1 --2- 172.21.15.12:0/95914234 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fe0c00b5370 0x7fe0c00b7830 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fe0cc005ab0 tx=0x7fe0cc005a20 comp rx=0 tx=0).stop 2024-09-11T00:05:39.029 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:39.027+0000 7fe0e1a00640 1 -- 172.21.15.12:0/95914234 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe0dc10a9b0 msgr2=0x7fe0dc19b9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:05:39.029 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:39.027+0000 7fe0e1a00640 1 --2- 172.21.15.12:0/95914234 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe0dc10a9b0 0x7fe0dc19b9a0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fe0c800e9f0 tx=0x7fe0c800eeb0 comp rx=0 tx=0).stop 2024-09-11T00:05:39.029 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:39.028+0000 7fe0e1a00640 1 -- 172.21.15.12:0/95914234 shutdown_connections 2024-09-11T00:05:39.029 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:39.028+0000 7fe0e1a00640 1 --2- 172.21.15.12:0/95914234 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fe0c00b5370 0x7fe0c00b7830 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:05:39.030 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:39.028+0000 7fe0e1a00640 1 --2- 172.21.15.12:0/95914234 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe0dc10b330 0x7fe0dc1a0d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:05:39.030 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:39.028+0000 7fe0e1a00640 1 --2- 172.21.15.12:0/95914234 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe0dc10a9b0 0x7fe0dc19b9a0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:05:39.030 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:39.028+0000 7fe0e1a00640 1 --2- 172.21.15.12:0/95914234 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe0dc104fa0 0x7fe0dc19b460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:05:39.030 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:39.028+0000 7fe0e1a00640 1 -- 172.21.15.12:0/95914234 >> 172.21.15.12:0/95914234 conn(0x7fe0dc100a50 msgr2=0x7fe0dc101300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:05:39.030 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:39.028+0000 7fe0e1a00640 1 -- 172.21.15.12:0/95914234 shutdown_connections 2024-09-11T00:05:39.030 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:39.028+0000 7fe0e1a00640 1 -- 172.21.15.12:0/95914234 wait complete. 2024-09-11T00:05:39.178 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:05:38 smithi012 ceph-mon[30333]: pgmap v353: 129 pgs: 129 active+clean; 647 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-11T00:05:39.178 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:05:38 smithi012 ceph-mon[30333]: from='client.? 172.21.15.12:0/3312855266' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 8} : dispatch 2024-09-11T00:05:39.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:05:38 smithi165 ceph-mon[35356]: pgmap v353: 129 pgs: 129 active+clean; 647 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-11T00:05:39.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:05:38 smithi165 ceph-mon[35356]: from='client.? 172.21.15.12:0/3312855266' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 8} : dispatch 2024-09-11T00:05:39.330 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:05:38 smithi178 ceph-mon[35919]: pgmap v353: 129 pgs: 129 active+clean; 647 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-11T00:05:39.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:05:38 smithi178 ceph-mon[35919]: from='client.? 172.21.15.12:0/3312855266' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 8} : dispatch 2024-09-11T00:05:39.438 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:38.941+0000 7f17cd000640 1 -- 172.21.15.12:0/644341335 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f17c81045e0 msgr2=0x7f17c810cad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:05:39.438 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:38.941+0000 7f17cd000640 1 --2- 172.21.15.12:0/644341335 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f17c81045e0 0x7f17c810cad0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f17b4009900 tx=0x7f17b402a3f0 comp rx=0 tx=0).stop 2024-09-11T00:05:39.438 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:38.941+0000 7f17cd000640 1 -- 172.21.15.12:0/644341335 shutdown_connections 2024-09-11T00:05:39.438 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:38.941+0000 7f17cd000640 1 --2- 172.21.15.12:0/644341335 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f17c810d120 0x7f17c810f580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:05:39.438 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:38.941+0000 7f17cd000640 1 --2- 172.21.15.12:0/644341335 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f17c81045e0 0x7f17c810cad0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:05:39.438 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:38.941+0000 7f17cd000640 1 --2- 172.21.15.12:0/644341335 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f17c8103cc0 0x7f17c81040a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:05:39.438 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:38.941+0000 7f17cd000640 1 -- 172.21.15.12:0/644341335 >> 172.21.15.12:0/644341335 conn(0x7f17c80fd700 msgr2=0x7f17c80ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:05:39.438 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:38.941+0000 7f17cd000640 1 -- 172.21.15.12:0/644341335 shutdown_connections 2024-09-11T00:05:39.438 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:38.941+0000 7f17cd000640 1 -- 172.21.15.12:0/644341335 wait complete. 2024-09-11T00:05:39.438 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:38.942+0000 7f17cd000640 1 Processor -- start 2024-09-11T00:05:39.439 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:38.942+0000 7f17cd000640 1 -- start start 2024-09-11T00:05:39.439 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:38.943+0000 7f17cd000640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f17c8103cc0 0x7f17c8101970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:05:39.439 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:38.943+0000 7f17cd000640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f17c81045e0 0x7f17c8101eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:05:39.439 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:38.943+0000 7f17cd000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f17c810d120 0x7f17c81023f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:05:39.439 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:38.943+0000 7f17cd000640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f17c8112d90 con 0x7f17c8103cc0 2024-09-11T00:05:39.439 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:38.943+0000 7f17cd000640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f17c8112c10 con 0x7f17c810d120 2024-09-11T00:05:39.439 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:38.943+0000 7f17cd000640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f17c8112f10 con 0x7f17c81045e0 2024-09-11T00:05:39.439 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:38.943+0000 7f17c5e00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f17c8103cc0 0x7f17c8101970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:05:39.439 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:38.943+0000 7f17c6800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f17c810d120 0x7f17c81023f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:05:39.439 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:38.943+0000 7f17c6800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f17c810d120 0x7f17c81023f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.12:58394/0 (socket says 172.21.15.12:58394) 2024-09-11T00:05:39.439 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:38.943+0000 7f17c6800640 1 -- 172.21.15.12:0/1604203759 learned_addr learned my addr 172.21.15.12:0/1604203759 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:05:39.439 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:38.943+0000 7f17c5400640 1 --2- 172.21.15.12:0/1604203759 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f17c81045e0 0x7f17c8101eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:05:39.439 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:38.944+0000 7f17c5e00640 1 -- 172.21.15.12:0/1604203759 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f17c81045e0 msgr2=0x7f17c8101eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:05:39.440 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:38.944+0000 7f17c5e00640 1 --2- 172.21.15.12:0/1604203759 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f17c81045e0 0x7f17c8101eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:05:39.440 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:38.944+0000 7f17c5e00640 1 -- 172.21.15.12:0/1604203759 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f17c810d120 msgr2=0x7f17c81023f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:05:39.440 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:38.944+0000 7f17c5e00640 1 --2- 172.21.15.12:0/1604203759 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f17c810d120 0x7f17c81023f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:05:39.440 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:38.944+0000 7f17c5e00640 1 -- 172.21.15.12:0/1604203759 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17c81a76e0 con 0x7f17c8103cc0 2024-09-11T00:05:39.440 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:38.944+0000 7f17c6800640 1 --2- 172.21.15.12:0/1604203759 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f17c810d120 0x7f17c81023f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:05:39.440 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:38.944+0000 7f17c5e00640 1 --2- 172.21.15.12:0/1604203759 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f17c8103cc0 0x7f17c8101970 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f17b800e9f0 tx=0x7f17b800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:05:39.440 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:38.944+0000 7f17bea00640 1 -- 172.21.15.12:0/1604203759 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17b800ccc0 con 0x7f17c8103cc0 2024-09-11T00:05:39.440 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:38.944+0000 7f17bea00640 1 -- 172.21.15.12:0/1604203759 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f17b80108e0 con 0x7f17c8103cc0 2024-09-11T00:05:39.440 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:38.944+0000 7f17cd000640 1 -- 172.21.15.12:0/1604203759 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f17c81a79d0 con 0x7f17c8103cc0 2024-09-11T00:05:39.440 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:38.944+0000 7f17bea00640 1 -- 172.21.15.12:0/1604203759 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17b8018880 con 0x7f17c8103cc0 2024-09-11T00:05:39.440 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:38.944+0000 7f17cd000640 1 -- 172.21.15.12:0/1604203759 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17c81a7f30 con 0x7f17c8103cc0 2024-09-11T00:05:39.440 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:38.946+0000 7f17cd000640 1 -- 172.21.15.12:0/1604203759 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f17c81040a0 con 0x7f17c8103cc0 2024-09-11T00:05:39.440 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:38.948+0000 7f17bea00640 1 -- 172.21.15.12:0/1604203759 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 17) ==== 153301+0+0 (secure 0 0 0) 0x7f17b8010430 con 0x7f17c8103cc0 2024-09-11T00:05:39.441 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:38.949+0000 7f17bea00640 1 --2- 172.21.15.12:0/1604203759 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f17ac0b53f0 0x7f17ac0b78b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:05:39.441 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:38.949+0000 7f17bea00640 1 -- 172.21.15.12:0/1604203759 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 9092+0+0 (secure 0 0 0) 0x7f17b80e63d0 con 0x7f17c8103cc0 2024-09-11T00:05:39.441 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:38.950+0000 7f17c5400640 1 --2- 172.21.15.12:0/1604203759 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f17ac0b53f0 0x7f17ac0b78b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:05:39.441 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:38.950+0000 7f17c5400640 1 --2- 172.21.15.12:0/1604203759 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f17ac0b53f0 0x7f17ac0b78b0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f17b4000c00 tx=0x7f17b4031400 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:05:39.441 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:38.953+0000 7f17bea00640 1 -- 172.21.15.12:0/1604203759 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f17b80ac720 con 0x7f17c8103cc0 2024-09-11T00:05:39.441 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:39.174+0000 7f17cd000640 1 -- 172.21.15.12:0/1604203759 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 9} v 0) -- 0x7f17c802e5c0 con 0x7f17c8103cc0 2024-09-11T00:05:39.441 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:39.175+0000 7f17bea00640 1 -- 172.21.15.12:0/1604203759 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 9}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f17b80a6e20 con 0x7f17c8103cc0 2024-09-11T00:05:39.441 INFO:teuthology.orchestra.run.smithi012.stdout:249108103209 2024-09-11T00:05:39.442 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:39.177+0000 7f17cd000640 1 -- 172.21.15.12:0/1604203759 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f17ac0b53f0 msgr2=0x7f17ac0b78b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:05:39.442 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:39.177+0000 7f17cd000640 1 --2- 172.21.15.12:0/1604203759 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f17ac0b53f0 0x7f17ac0b78b0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f17b4000c00 tx=0x7f17b4031400 comp rx=0 tx=0).stop 2024-09-11T00:05:39.442 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:39.177+0000 7f17cd000640 1 -- 172.21.15.12:0/1604203759 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f17c8103cc0 msgr2=0x7f17c8101970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:05:39.442 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:39.177+0000 7f17cd000640 1 --2- 172.21.15.12:0/1604203759 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f17c8103cc0 0x7f17c8101970 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f17b800e9f0 tx=0x7f17b800eeb0 comp rx=0 tx=0).stop 2024-09-11T00:05:39.442 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:39.177+0000 7f17cd000640 1 -- 172.21.15.12:0/1604203759 shutdown_connections 2024-09-11T00:05:39.442 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:39.177+0000 7f17cd000640 1 --2- 172.21.15.12:0/1604203759 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f17ac0b53f0 0x7f17ac0b78b0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:05:39.442 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:39.177+0000 7f17cd000640 1 --2- 172.21.15.12:0/1604203759 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f17c810d120 0x7f17c81023f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:05:39.442 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:39.177+0000 7f17cd000640 1 --2- 172.21.15.12:0/1604203759 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f17c81045e0 0x7f17c8101eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:05:39.442 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:39.177+0000 7f17cd000640 1 --2- 172.21.15.12:0/1604203759 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f17c8103cc0 0x7f17c8101970 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:05:39.442 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:39.177+0000 7f17cd000640 1 -- 172.21.15.12:0/1604203759 >> 172.21.15.12:0/1604203759 conn(0x7f17c80fd700 msgr2=0x7f17c80fe360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:05:39.442 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:39.177+0000 7f17cd000640 1 -- 172.21.15.12:0/1604203759 shutdown_connections 2024-09-11T00:05:39.442 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:39.178+0000 7f17cd000640 1 -- 172.21.15.12:0/1604203759 wait complete. 2024-09-11T00:05:40.063 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:05:39 smithi012 ceph-mon[30333]: from='client.? 172.21.15.12:0/95914234' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 6} : dispatch 2024-09-11T00:05:40.063 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:05:39 smithi012 ceph-mon[30333]: from='client.? 172.21.15.12:0/1604203759' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 9} : dispatch 2024-09-11T00:05:40.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:05:39 smithi165 ceph-mon[35356]: from='client.? 172.21.15.12:0/95914234' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 6} : dispatch 2024-09-11T00:05:40.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:05:39 smithi165 ceph-mon[35356]: from='client.? 172.21.15.12:0/1604203759' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 9} : dispatch 2024-09-11T00:05:40.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:05:39 smithi178 ceph-mon[35919]: from='client.? 172.21.15.12:0/95914234' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 6} : dispatch 2024-09-11T00:05:40.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:05:39 smithi178 ceph-mon[35919]: from='client.? 172.21.15.12:0/1604203759' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 9} : dispatch 2024-09-11T00:05:40.674 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:40.343+0000 7f83a0600640 1 -- 172.21.15.12:0/2131322282 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8398104fa0 msgr2=0x7f8398105380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:05:40.674 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:40.343+0000 7f83a0600640 1 --2- 172.21.15.12:0/2131322282 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8398104fa0 0x7f8398105380 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f838c009ca0 tx=0x7f838c02a8c0 comp rx=0 tx=0).stop 2024-09-11T00:05:40.674 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:40.343+0000 7f83a0600640 1 -- 172.21.15.12:0/2131322282 shutdown_connections 2024-09-11T00:05:40.674 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:40.343+0000 7f83a0600640 1 --2- 172.21.15.12:0/2131322282 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f839810b330 0x7f8398111b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:05:40.675 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:40.343+0000 7f83a0600640 1 --2- 172.21.15.12:0/2131322282 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f839810a9b0 0x7f839810adf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:05:40.675 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:40.343+0000 7f83a0600640 1 --2- 172.21.15.12:0/2131322282 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8398104fa0 0x7f8398105380 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:05:40.675 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:40.343+0000 7f83a0600640 1 -- 172.21.15.12:0/2131322282 >> 172.21.15.12:0/2131322282 conn(0x7f8398100a50 msgr2=0x7f8398102e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:05:40.675 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:40.343+0000 7f83a0600640 1 -- 172.21.15.12:0/2131322282 shutdown_connections 2024-09-11T00:05:40.675 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:40.343+0000 7f83a0600640 1 -- 172.21.15.12:0/2131322282 wait complete. 2024-09-11T00:05:40.675 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:40.344+0000 7f83a0600640 1 Processor -- start 2024-09-11T00:05:40.676 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:40.344+0000 7f83a0600640 1 -- start start 2024-09-11T00:05:40.676 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:40.345+0000 7f83a0600640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8398104fa0 0x7f839819b470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:05:40.676 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:40.345+0000 7f83a0600640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f839810a9b0 0x7f839819b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:05:40.676 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:40.345+0000 7f83a0600640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f839810b330 0x7f83981a0d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:05:40.676 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:40.345+0000 7f83a0600640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f8398114fb0 con 0x7f8398104fa0 2024-09-11T00:05:40.676 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:40.345+0000 7f83a0600640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f8398114e30 con 0x7f839810b330 2024-09-11T00:05:40.677 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:40.345+0000 7f83a0600640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f8398115130 con 0x7f839810a9b0 2024-09-11T00:05:40.677 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:40.345+0000 7f839dc00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8398104fa0 0x7f839819b470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:05:40.677 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:40.345+0000 7f839e600640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f839810b330 0x7f83981a0d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:05:40.677 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:40.345+0000 7f839dc00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8398104fa0 0x7f839819b470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:39854/0 (socket says 172.21.15.12:39854) 2024-09-11T00:05:40.677 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:40.345+0000 7f839dc00640 1 -- 172.21.15.12:0/602164988 learned_addr learned my addr 172.21.15.12:0/602164988 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:05:40.677 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:40.345+0000 7f839d200640 1 --2- 172.21.15.12:0/602164988 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f839810a9b0 0x7f839819b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:05:40.678 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:40.346+0000 7f839e600640 1 -- 172.21.15.12:0/602164988 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f839810a9b0 msgr2=0x7f839819b9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:05:40.678 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:40.346+0000 7f839e600640 1 --2- 172.21.15.12:0/602164988 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f839810a9b0 0x7f839819b9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:05:40.678 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:40.346+0000 7f839e600640 1 -- 172.21.15.12:0/602164988 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8398104fa0 msgr2=0x7f839819b470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:05:40.678 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:40.346+0000 7f839e600640 1 --2- 172.21.15.12:0/602164988 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8398104fa0 0x7f839819b470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:05:40.678 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:40.346+0000 7f839e600640 1 -- 172.21.15.12:0/602164988 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83981a1520 con 0x7f839810b330 2024-09-11T00:05:40.679 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:40.346+0000 7f839dc00640 1 --2- 172.21.15.12:0/602164988 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8398104fa0 0x7f839819b470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:05:40.679 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:40.346+0000 7f839e600640 1 --2- 172.21.15.12:0/602164988 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f839810b330 0x7f83981a0d50 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f839000e970 tx=0x7f839000ee30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:05:40.679 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:40.346+0000 7f8396000640 1 -- 172.21.15.12:0/602164988 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f839000ccc0 con 0x7f839810b330 2024-09-11T00:05:40.679 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:40.346+0000 7f83a0600640 1 -- 172.21.15.12:0/602164988 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f83981a1810 con 0x7f839810b330 2024-09-11T00:05:40.679 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:40.346+0000 7f8396000640 1 -- 172.21.15.12:0/602164988 <== mon.2 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f83900044d0 con 0x7f839810b330 2024-09-11T00:05:40.679 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:40.346+0000 7f8396000640 1 -- 172.21.15.12:0/602164988 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f839001c070 con 0x7f839810b330 2024-09-11T00:05:40.680 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:40.346+0000 7f83a0600640 1 -- 172.21.15.12:0/602164988 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83981a7ed0 con 0x7f839810b330 2024-09-11T00:05:40.680 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:40.347+0000 7f83a0600640 1 -- 172.21.15.12:0/602164988 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8398105380 con 0x7f839810b330 2024-09-11T00:05:40.680 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:40.350+0000 7f8396000640 1 -- 172.21.15.12:0/602164988 <== mon.2 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 17) ==== 153301+0+0 (secure 0 0 0) 0x7f8390004a00 con 0x7f839810b330 2024-09-11T00:05:40.680 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:40.351+0000 7f8396000640 1 --2- 172.21.15.12:0/602164988 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f83800b53f0 0x7f83800b78b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:05:40.680 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:40.352+0000 7f8396000640 1 -- 172.21.15.12:0/602164988 <== mon.2 v2:172.21.15.165:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 9092+0+0 (secure 0 0 0) 0x7f83900e62a0 con 0x7f839810b330 2024-09-11T00:05:40.680 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:40.352+0000 7f839dc00640 1 --2- 172.21.15.12:0/602164988 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f83800b53f0 0x7f83800b78b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:05:40.681 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:40.352+0000 7f839dc00640 1 --2- 172.21.15.12:0/602164988 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f83800b53f0 0x7f83800b78b0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f838c000f80 tx=0x7f838c005d80 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:05:40.681 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:40.353+0000 7f8396000640 1 -- 172.21.15.12:0/602164988 <== mon.2 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f83900accd0 con 0x7f839810b330 2024-09-11T00:05:40.681 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:40.551+0000 7f83a0600640 1 -- 172.21.15.12:0/602164988 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7f839819c5a0 con 0x7f839810b330 2024-09-11T00:05:40.681 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:40.551+0000 7f8396000640 1 -- 172.21.15.12:0/602164988 <== mon.2 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f839819c5a0 con 0x7f839810b330 2024-09-11T00:05:40.682 INFO:teuthology.orchestra.run.smithi012.stdout:77309411409 2024-09-11T00:05:40.682 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:40.553+0000 7f83a0600640 1 -- 172.21.15.12:0/602164988 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f83800b53f0 msgr2=0x7f83800b78b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:05:40.682 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:40.553+0000 7f83a0600640 1 --2- 172.21.15.12:0/602164988 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f83800b53f0 0x7f83800b78b0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f838c000f80 tx=0x7f838c005d80 comp rx=0 tx=0).stop 2024-09-11T00:05:40.683 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:40.553+0000 7f83a0600640 1 -- 172.21.15.12:0/602164988 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f839810b330 msgr2=0x7f83981a0d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:05:40.683 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:40.553+0000 7f83a0600640 1 --2- 172.21.15.12:0/602164988 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f839810b330 0x7f83981a0d50 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f839000e970 tx=0x7f839000ee30 comp rx=0 tx=0).stop 2024-09-11T00:05:40.683 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:40.553+0000 7f83a0600640 1 -- 172.21.15.12:0/602164988 shutdown_connections 2024-09-11T00:05:40.683 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:40.553+0000 7f83a0600640 1 --2- 172.21.15.12:0/602164988 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f83800b53f0 0x7f83800b78b0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:05:40.683 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:40.553+0000 7f83a0600640 1 --2- 172.21.15.12:0/602164988 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f839810b330 0x7f83981a0d50 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:05:40.684 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:40.553+0000 7f83a0600640 1 --2- 172.21.15.12:0/602164988 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f839810a9b0 0x7f839819b9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:05:40.684 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:40.553+0000 7f83a0600640 1 --2- 172.21.15.12:0/602164988 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8398104fa0 0x7f839819b470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:05:40.684 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:40.553+0000 7f83a0600640 1 -- 172.21.15.12:0/602164988 >> 172.21.15.12:0/602164988 conn(0x7f8398100a50 msgr2=0x7f8398101350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:05:40.684 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:40.554+0000 7f83a0600640 1 -- 172.21.15.12:0/602164988 shutdown_connections 2024-09-11T00:05:40.684 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:40.554+0000 7f83a0600640 1 -- 172.21.15.12:0/602164988 wait complete. 2024-09-11T00:05:40.930 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:05:41.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:05:41 smithi165 ceph-mon[35356]: pgmap v354: 129 pgs: 129 active+clean; 647 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 9 op/s 2024-09-11T00:05:41.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:05:41 smithi165 ceph-mon[35356]: from='client.? 172.21.15.12:0/602164988' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 2} : dispatch 2024-09-11T00:05:41.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:05:41 smithi012 ceph-mon[30333]: pgmap v354: 129 pgs: 129 active+clean; 647 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 9 op/s 2024-09-11T00:05:41.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:05:41 smithi012 ceph-mon[30333]: from='client.? 172.21.15.12:0/602164988' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 2} : dispatch 2024-09-11T00:05:41.330 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:05:41 smithi178 ceph-mon[35919]: pgmap v354: 129 pgs: 129 active+clean; 647 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 9 op/s 2024-09-11T00:05:41.330 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:05:41 smithi178 ceph-mon[35919]: from='client.? 172.21.15.12:0/602164988' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 2} : dispatch 2024-09-11T00:05:41.868 INFO:tasks.cephadm.ceph_manager.ceph:need seq 249108103207 got 249108103209 for osd.9 2024-09-11T00:05:41.868 DEBUG:teuthology.parallel:result is None 2024-09-11T00:05:42.399 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738456 got 34359738459 for osd.0 2024-09-11T00:05:42.400 DEBUG:teuthology.parallel:result is None 2024-09-11T00:05:42.539 INFO:tasks.cephadm.ceph_manager.ceph:need seq 270582939684 got 270582939686 for osd.10 2024-09-11T00:05:42.540 DEBUG:teuthology.parallel:result is None 2024-09-11T00:05:42.991 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411406 got 77309411409 for osd.2 2024-09-11T00:05:42.991 DEBUG:teuthology.parallel:result is None 2024-09-11T00:05:43.312 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:05:43 smithi012 ceph-mon[30333]: pgmap v355: 129 pgs: 129 active+clean; 647 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 9 op/s 2024-09-11T00:05:43.455 INFO:tasks.cephadm.ceph_manager.ceph:need seq 180388626490 got 180388626491 for osd.6 2024-09-11T00:05:43.455 DEBUG:teuthology.parallel:result is None 2024-09-11T00:05:43.532 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:05:43 smithi165 ceph-mon[35356]: pgmap v355: 129 pgs: 129 active+clean; 647 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 9 op/s 2024-09-11T00:05:43.581 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:05:43 smithi178 ceph-mon[35919]: pgmap v355: 129 pgs: 129 active+clean; 647 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 9 op/s 2024-09-11T00:05:43.728 INFO:tasks.cephadm.ceph_manager.ceph:need seq 223338299438 got 223338299440 for osd.8 2024-09-11T00:05:43.728 DEBUG:teuthology.parallel:result is None 2024-09-11T00:05:44.028 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:05:44.061 INFO:tasks.cephadm.ceph_manager.ceph:need seq 154618822717 got 154618822720 for osd.5 2024-09-11T00:05:44.061 DEBUG:teuthology.parallel:result is None 2024-09-11T00:05:44.739 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:44.737+0000 7f76e7e00640 1 -- 172.21.15.12:0/655208439 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f76e8102c10 msgr2=0x7f76e8102ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:05:44.739 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:44.737+0000 7f76e7e00640 1 --2- 172.21.15.12:0/655208439 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f76e8102c10 0x7f76e8102ff0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f76d8009900 tx=0x7f76d802a3f0 comp rx=0 tx=0).stop 2024-09-11T00:05:44.739 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:44.738+0000 7f76e7e00640 1 -- 172.21.15.12:0/655208439 shutdown_connections 2024-09-11T00:05:44.739 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:44.738+0000 7f76e7e00640 1 --2- 172.21.15.12:0/655208439 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f76e8108fa0 0x7f76e810f7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:05:44.739 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:44.738+0000 7f76e7e00640 1 --2- 172.21.15.12:0/655208439 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f76e8108620 0x7f76e8108a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:05:44.739 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:44.738+0000 7f76e7e00640 1 --2- 172.21.15.12:0/655208439 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f76e8102c10 0x7f76e8102ff0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:05:44.739 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:44.738+0000 7f76e7e00640 1 -- 172.21.15.12:0/655208439 >> 172.21.15.12:0/655208439 conn(0x7f76e80fe660 msgr2=0x7f76e8100a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:05:44.739 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:44.738+0000 7f76e7e00640 1 -- 172.21.15.12:0/655208439 shutdown_connections 2024-09-11T00:05:44.740 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:44.738+0000 7f76e7e00640 1 -- 172.21.15.12:0/655208439 wait complete. 2024-09-11T00:05:44.741 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:44.739+0000 7f76e7e00640 1 Processor -- start 2024-09-11T00:05:44.741 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:44.739+0000 7f76e7e00640 1 -- start start 2024-09-11T00:05:44.741 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:44.739+0000 7f76e7e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f76e8102c10 0x7f76e81990a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:05:44.741 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:44.739+0000 7f76e7e00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f76e8108620 0x7f76e81995e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:05:44.741 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:44.739+0000 7f76e7e00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f76e8108fa0 0x7f76e819e980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:05:44.741 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:44.739+0000 7f76e7e00640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f76e8112cb0 con 0x7f76e8108620 2024-09-11T00:05:44.741 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:44.739+0000 7f76e7e00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f76e8112b30 con 0x7f76e8102c10 2024-09-11T00:05:44.741 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:44.739+0000 7f76e7e00640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f76e8112e30 con 0x7f76e8108fa0 2024-09-11T00:05:44.741 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:44.740+0000 7f76e6a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f76e8102c10 0x7f76e81990a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:05:44.741 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:44.740+0000 7f76de000640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f76e8108620 0x7f76e81995e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:05:44.741 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:44.740+0000 7f76e6a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f76e8102c10 0x7f76e81990a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.12:58418/0 (socket says 172.21.15.12:58418) 2024-09-11T00:05:44.741 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:44.740+0000 7f76e6a00640 1 -- 172.21.15.12:0/261709197 learned_addr learned my addr 172.21.15.12:0/261709197 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:05:44.742 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:44.740+0000 7f76de000640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f76e8108620 0x7f76e81995e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:39886/0 (socket says 172.21.15.12:39886) 2024-09-11T00:05:44.742 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:44.740+0000 7f76e7400640 1 --2- 172.21.15.12:0/261709197 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f76e8108fa0 0x7f76e819e980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:05:44.742 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:44.740+0000 7f76de000640 1 -- 172.21.15.12:0/261709197 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f76e8108fa0 msgr2=0x7f76e819e980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:05:44.742 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:44.740+0000 7f76de000640 1 --2- 172.21.15.12:0/261709197 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f76e8108fa0 0x7f76e819e980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:05:44.743 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:44.740+0000 7f76de000640 1 -- 172.21.15.12:0/261709197 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f76e8102c10 msgr2=0x7f76e81990a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:05:44.743 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:44.740+0000 7f76de000640 1 --2- 172.21.15.12:0/261709197 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f76e8102c10 0x7f76e81990a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:05:44.743 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:44.740+0000 7f76de000640 1 -- 172.21.15.12:0/261709197 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76e819f150 con 0x7f76e8108620 2024-09-11T00:05:44.743 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:44.740+0000 7f76e6a00640 1 --2- 172.21.15.12:0/261709197 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f76e8102c10 0x7f76e81990a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:05:44.743 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:44.740+0000 7f76de000640 1 --2- 172.21.15.12:0/261709197 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f76e8108620 0x7f76e81995e0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f76d000e9f0 tx=0x7f76d000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:05:44.743 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:44.741+0000 7f76dfe00640 1 -- 172.21.15.12:0/261709197 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76d000ccc0 con 0x7f76e8108620 2024-09-11T00:05:44.743 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:44.741+0000 7f76dfe00640 1 -- 172.21.15.12:0/261709197 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f76d0005020 con 0x7f76e8108620 2024-09-11T00:05:44.743 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:44.741+0000 7f76e7e00640 1 -- 172.21.15.12:0/261709197 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f76e819f440 con 0x7f76e8108620 2024-09-11T00:05:44.743 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:44.741+0000 7f76dfe00640 1 -- 172.21.15.12:0/261709197 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76d000ce60 con 0x7f76e8108620 2024-09-11T00:05:44.743 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:44.741+0000 7f76e7e00640 1 -- 172.21.15.12:0/261709197 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76e81a5b00 con 0x7f76e8108620 2024-09-11T00:05:44.744 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:44.742+0000 7f76e7e00640 1 -- 172.21.15.12:0/261709197 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f76e8102ff0 con 0x7f76e8108620 2024-09-11T00:05:44.745 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:44.744+0000 7f76dfe00640 1 -- 172.21.15.12:0/261709197 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 17) ==== 153301+0+0 (secure 0 0 0) 0x7f76d0004a60 con 0x7f76e8108620 2024-09-11T00:05:44.747 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:44.746+0000 7f76dfe00640 1 --2- 172.21.15.12:0/261709197 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f76cc0b5490 0x7f76cc0b7950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:05:44.747 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:44.747+0000 7f76dfe00640 1 -- 172.21.15.12:0/261709197 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 9092+0+0 (secure 0 0 0) 0x7f76d00e5fe0 con 0x7f76e8108620 2024-09-11T00:05:44.747 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:44.747+0000 7f76e6a00640 1 --2- 172.21.15.12:0/261709197 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f76cc0b5490 0x7f76cc0b7950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:05:44.748 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:44.747+0000 7f76e6a00640 1 --2- 172.21.15.12:0/261709197 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f76cc0b5490 0x7f76cc0b7950 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f76d80096c0 tx=0x7f76d8005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:05:44.750 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:44.750+0000 7f76dfe00640 1 -- 172.21.15.12:0/261709197 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f76d00ac9c0 con 0x7f76e8108620 2024-09-11T00:05:44.948 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:44.947+0000 7f76e7e00640 1 -- 172.21.15.12:0/261709197 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7f76e81a5e90 con 0x7f76e8108620 2024-09-11T00:05:44.949 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:44.947+0000 7f76dfe00640 1 -- 172.21.15.12:0/261709197 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f76d00ac2c0 con 0x7f76e8108620 2024-09-11T00:05:44.949 INFO:teuthology.orchestra.run.smithi012.stdout:107374182476 2024-09-11T00:05:44.951 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:44.950+0000 7f76e7e00640 1 -- 172.21.15.12:0/261709197 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f76cc0b5490 msgr2=0x7f76cc0b7950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:05:44.951 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:44.950+0000 7f76e7e00640 1 --2- 172.21.15.12:0/261709197 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f76cc0b5490 0x7f76cc0b7950 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f76d80096c0 tx=0x7f76d8005a20 comp rx=0 tx=0).stop 2024-09-11T00:05:44.951 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:44.950+0000 7f76e7e00640 1 -- 172.21.15.12:0/261709197 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f76e8108620 msgr2=0x7f76e81995e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:05:44.951 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:44.950+0000 7f76e7e00640 1 --2- 172.21.15.12:0/261709197 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f76e8108620 0x7f76e81995e0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f76d000e9f0 tx=0x7f76d000edf0 comp rx=0 tx=0).stop 2024-09-11T00:05:44.951 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:44.950+0000 7f76e7e00640 1 -- 172.21.15.12:0/261709197 shutdown_connections 2024-09-11T00:05:44.951 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:44.950+0000 7f76e7e00640 1 --2- 172.21.15.12:0/261709197 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f76cc0b5490 0x7f76cc0b7950 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:05:44.951 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:44.950+0000 7f76e7e00640 1 --2- 172.21.15.12:0/261709197 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f76e8108fa0 0x7f76e819e980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:05:44.951 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:44.950+0000 7f76e7e00640 1 --2- 172.21.15.12:0/261709197 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f76e8108620 0x7f76e81995e0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:05:44.952 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:44.950+0000 7f76e7e00640 1 --2- 172.21.15.12:0/261709197 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f76e8102c10 0x7f76e81990a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:05:44.952 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:44.950+0000 7f76e7e00640 1 -- 172.21.15.12:0/261709197 >> 172.21.15.12:0/261709197 conn(0x7f76e80fe660 msgr2=0x7f76e8100a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:05:44.952 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:44.950+0000 7f76e7e00640 1 -- 172.21.15.12:0/261709197 shutdown_connections 2024-09-11T00:05:44.952 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:44.950+0000 7f76e7e00640 1 -- 172.21.15.12:0/261709197 wait complete. 2024-09-11T00:05:44.974 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:44.973+0000 7fad71400640 1 -- 172.21.15.12:0/3733693164 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fad6c104fa0 msgr2=0x7fad6c105380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:05:44.974 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:44.973+0000 7fad71400640 1 --2- 172.21.15.12:0/3733693164 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fad6c104fa0 0x7fad6c105380 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fad5c009900 tx=0x7fad5c02a3f0 comp rx=0 tx=0).stop 2024-09-11T00:05:44.974 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:44.973+0000 7fad71400640 1 -- 172.21.15.12:0/3733693164 shutdown_connections 2024-09-11T00:05:44.974 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:44.973+0000 7fad71400640 1 --2- 172.21.15.12:0/3733693164 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fad6c10b330 0x7fad6c111b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:05:44.974 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:44.973+0000 7fad71400640 1 --2- 172.21.15.12:0/3733693164 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fad6c10a9b0 0x7fad6c10adf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:05:44.975 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:44.973+0000 7fad71400640 1 --2- 172.21.15.12:0/3733693164 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fad6c104fa0 0x7fad6c105380 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:05:44.975 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:44.973+0000 7fad71400640 1 -- 172.21.15.12:0/3733693164 >> 172.21.15.12:0/3733693164 conn(0x7fad6c100a50 msgr2=0x7fad6c102e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:05:44.975 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:44.973+0000 7fad71400640 1 -- 172.21.15.12:0/3733693164 shutdown_connections 2024-09-11T00:05:44.975 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:44.973+0000 7fad71400640 1 -- 172.21.15.12:0/3733693164 wait complete. 2024-09-11T00:05:44.975 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:44.974+0000 7fad71400640 1 Processor -- start 2024-09-11T00:05:44.975 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:44.974+0000 7fad71400640 1 -- start start 2024-09-11T00:05:44.976 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:44.975+0000 7fad71400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fad6c104fa0 0x7fad6c19b440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:05:44.976 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:44.975+0000 7fad71400640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fad6c10a9b0 0x7fad6c19b980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:05:44.976 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:44.975+0000 7fad71400640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fad6c10b330 0x7fad6c1a0d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:05:44.976 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:44.975+0000 7fad71400640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fad6c114f90 con 0x7fad6c10a9b0 2024-09-11T00:05:44.976 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:44.975+0000 7fad71400640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fad6c114e10 con 0x7fad6c104fa0 2024-09-11T00:05:44.976 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:44.975+0000 7fad71400640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fad6c115110 con 0x7fad6c10b330 2024-09-11T00:05:44.977 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:44.975+0000 7fad6a800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fad6c104fa0 0x7fad6c19b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:05:44.977 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:44.975+0000 7fad69e00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fad6c10a9b0 0x7fad6c19b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:05:44.977 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:44.975+0000 7fad6a800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fad6c104fa0 0x7fad6c19b440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.12:58422/0 (socket says 172.21.15.12:58422) 2024-09-11T00:05:44.977 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:44.975+0000 7fad6a800640 1 -- 172.21.15.12:0/3972747029 learned_addr learned my addr 172.21.15.12:0/3972747029 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:05:44.977 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:44.976+0000 7fad6b200640 1 --2- 172.21.15.12:0/3972747029 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fad6c10b330 0x7fad6c1a0d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:05:44.977 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:44.976+0000 7fad6a800640 1 -- 172.21.15.12:0/3972747029 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fad6c10b330 msgr2=0x7fad6c1a0d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:05:44.977 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:44.976+0000 7fad6a800640 1 --2- 172.21.15.12:0/3972747029 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fad6c10b330 0x7fad6c1a0d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:05:44.977 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:44.976+0000 7fad6a800640 1 -- 172.21.15.12:0/3972747029 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fad6c10a9b0 msgr2=0x7fad6c19b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:05:44.978 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:44.976+0000 7fad6a800640 1 --2- 172.21.15.12:0/3972747029 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fad6c10a9b0 0x7fad6c19b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:05:44.978 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:44.976+0000 7fad6a800640 1 -- 172.21.15.12:0/3972747029 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad6c1a14f0 con 0x7fad6c104fa0 2024-09-11T00:05:44.978 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:44.976+0000 7fad6a800640 1 --2- 172.21.15.12:0/3972747029 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fad6c104fa0 0x7fad6c19b440 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fad5c005ab0 tx=0x7fad5c003cd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:05:44.978 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:44.976+0000 7fad63400640 1 -- 172.21.15.12:0/3972747029 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad5c03f070 con 0x7fad6c104fa0 2024-09-11T00:05:44.978 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:44.977+0000 7fad71400640 1 -- 172.21.15.12:0/3972747029 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fad6c1a1780 con 0x7fad6c104fa0 2024-09-11T00:05:44.978 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:44.977+0000 7fad71400640 1 -- 172.21.15.12:0/3972747029 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fad6c1a7ea0 con 0x7fad6c104fa0 2024-09-11T00:05:44.979 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:44.977+0000 7fad63400640 1 -- 172.21.15.12:0/3972747029 <== mon.2 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fad5c031d60 con 0x7fad6c104fa0 2024-09-11T00:05:44.979 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:44.977+0000 7fad63400640 1 -- 172.21.15.12:0/3972747029 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad5c049060 con 0x7fad6c104fa0 2024-09-11T00:05:44.979 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:44.978+0000 7fad71400640 1 -- 172.21.15.12:0/3972747029 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fad4c0052c0 con 0x7fad6c104fa0 2024-09-11T00:05:44.981 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:44.980+0000 7fad63400640 1 -- 172.21.15.12:0/3972747029 <== mon.2 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 17) ==== 153301+0+0 (secure 0 0 0) 0x7fad5c0318b0 con 0x7fad6c104fa0 2024-09-11T00:05:44.982 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:44.982+0000 7fad63400640 1 --2- 172.21.15.12:0/3972747029 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fad500b53f0 0x7fad500b78b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:05:44.983 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:44.982+0000 7fad63400640 1 -- 172.21.15.12:0/3972747029 <== mon.2 v2:172.21.15.165:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 9092+0+0 (secure 0 0 0) 0x7fad5c106c00 con 0x7fad6c104fa0 2024-09-11T00:05:44.983 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:44.982+0000 7fad69e00640 1 --2- 172.21.15.12:0/3972747029 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fad500b53f0 0x7fad500b78b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:05:44.983 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:44.983+0000 7fad69e00640 1 --2- 172.21.15.12:0/3972747029 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fad500b53f0 0x7fad500b78b0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fad58005fd0 tx=0x7fad58005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:05:44.984 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:44.983+0000 7fad63400640 1 -- 172.21.15.12:0/3972747029 <== mon.2 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7fad5c0cd5a0 con 0x7fad6c104fa0 2024-09-11T00:05:45.184 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:45.183+0000 7fad71400640 1 -- 172.21.15.12:0/3972747029 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 11} v 0) -- 0x7fad4c005880 con 0x7fad6c104fa0 2024-09-11T00:05:45.185 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:45.184+0000 7fad63400640 1 -- 172.21.15.12:0/3972747029 <== mon.2 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 11}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7fad5c0ccea0 con 0x7fad6c104fa0 2024-09-11T00:05:45.185 INFO:teuthology.orchestra.run.smithi012.stdout:292057776162 2024-09-11T00:05:45.188 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:45.186+0000 7fad71400640 1 -- 172.21.15.12:0/3972747029 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fad500b53f0 msgr2=0x7fad500b78b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:05:45.188 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:45.186+0000 7fad71400640 1 --2- 172.21.15.12:0/3972747029 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fad500b53f0 0x7fad500b78b0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fad58005fd0 tx=0x7fad58005850 comp rx=0 tx=0).stop 2024-09-11T00:05:45.188 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:45.186+0000 7fad71400640 1 -- 172.21.15.12:0/3972747029 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fad6c104fa0 msgr2=0x7fad6c19b440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:05:45.188 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:45.186+0000 7fad71400640 1 --2- 172.21.15.12:0/3972747029 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fad6c104fa0 0x7fad6c19b440 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fad5c005ab0 tx=0x7fad5c003cd0 comp rx=0 tx=0).stop 2024-09-11T00:05:45.188 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:45.186+0000 7fad71400640 1 -- 172.21.15.12:0/3972747029 shutdown_connections 2024-09-11T00:05:45.188 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:45.186+0000 7fad71400640 1 --2- 172.21.15.12:0/3972747029 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fad500b53f0 0x7fad500b78b0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:05:45.188 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:45.186+0000 7fad71400640 1 --2- 172.21.15.12:0/3972747029 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fad6c10b330 0x7fad6c1a0d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:05:45.188 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:45.186+0000 7fad71400640 1 --2- 172.21.15.12:0/3972747029 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fad6c10a9b0 0x7fad6c19b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:05:45.188 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:45.186+0000 7fad71400640 1 --2- 172.21.15.12:0/3972747029 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fad6c104fa0 0x7fad6c19b440 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:05:45.188 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:45.186+0000 7fad71400640 1 -- 172.21.15.12:0/3972747029 >> 172.21.15.12:0/3972747029 conn(0x7fad6c100a50 msgr2=0x7fad6c1012b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:05:45.188 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:45.187+0000 7fad71400640 1 -- 172.21.15.12:0/3972747029 shutdown_connections 2024-09-11T00:05:45.188 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:45.187+0000 7fad71400640 1 -- 172.21.15.12:0/3972747029 wait complete. 2024-09-11T00:05:45.506 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:05:45 smithi012 ceph-mon[30333]: pgmap v356: 129 pgs: 129 active+clean; 647 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-09-11T00:05:45.506 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:05:45 smithi012 ceph-mon[30333]: from='client.? 172.21.15.12:0/261709197' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 3} : dispatch 2024-09-11T00:05:45.532 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:05:45 smithi165 ceph-mon[35356]: pgmap v356: 129 pgs: 129 active+clean; 647 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-09-11T00:05:45.532 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:05:45 smithi165 ceph-mon[35356]: from='client.? 172.21.15.12:0/261709197' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 3} : dispatch 2024-09-11T00:05:45.580 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:05:45 smithi178 ceph-mon[35919]: pgmap v356: 129 pgs: 129 active+clean; 647 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-09-11T00:05:45.581 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:05:45 smithi178 ceph-mon[35919]: from='client.? 172.21.15.12:0/261709197' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 3} : dispatch 2024-09-11T00:05:45.960 INFO:tasks.cephadm.ceph_manager.ceph:need seq 292057776159 got 292057776162 for osd.11 2024-09-11T00:05:45.961 DEBUG:teuthology.parallel:result is None 2024-09-11T00:05:46.430 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:05:46.460 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:05:46 smithi012 ceph-mon[30333]: from='client.? 172.21.15.12:0/3972747029' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 11} : dispatch 2024-09-11T00:05:46.462 INFO:tasks.cephadm.ceph_manager.ceph:need seq 107374182471 got 107374182476 for osd.3 2024-09-11T00:05:46.462 DEBUG:teuthology.parallel:result is None 2024-09-11T00:05:46.532 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:05:46 smithi165 ceph-mon[35356]: from='client.? 172.21.15.12:0/3972747029' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 11} : dispatch 2024-09-11T00:05:46.580 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:05:46 smithi178 ceph-mon[35919]: from='client.? 172.21.15.12:0/3972747029' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 11} : dispatch 2024-09-11T00:05:47.011 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:47.009+0000 7f808c800640 1 -- 172.21.15.12:0/1344180211 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8084075f40 msgr2=0x7f808410edf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:05:47.011 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:47.009+0000 7f808c800640 1 --2- 172.21.15.12:0/1344180211 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8084075f40 0x7f808410edf0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f8074009900 tx=0x7f807402a400 comp rx=0 tx=0).stop 2024-09-11T00:05:47.011 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:47.010+0000 7f808c800640 1 -- 172.21.15.12:0/1344180211 shutdown_connections 2024-09-11T00:05:47.011 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:47.010+0000 7f808c800640 1 --2- 172.21.15.12:0/1344180211 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f808410f3c0 0x7f80841117b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:05:47.011 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:47.010+0000 7f808c800640 1 --2- 172.21.15.12:0/1344180211 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8084075f40 0x7f808410edf0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:05:47.011 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:47.010+0000 7f808c800640 1 --2- 172.21.15.12:0/1344180211 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8084075540 0x7f8084075920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:05:47.011 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:47.010+0000 7f808c800640 1 -- 172.21.15.12:0/1344180211 >> 172.21.15.12:0/1344180211 conn(0x7f80840fe810 msgr2=0x7f8084100c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:05:47.011 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:47.010+0000 7f808c800640 1 -- 172.21.15.12:0/1344180211 shutdown_connections 2024-09-11T00:05:47.011 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:47.010+0000 7f808c800640 1 -- 172.21.15.12:0/1344180211 wait complete. 2024-09-11T00:05:47.013 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:47.011+0000 7f808c800640 1 Processor -- start 2024-09-11T00:05:47.013 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:47.011+0000 7f808c800640 1 -- start start 2024-09-11T00:05:47.013 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:47.011+0000 7f808c800640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8084075540 0x7f808419b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:05:47.013 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:47.011+0000 7f808c800640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8084075f40 0x7f808419ba60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:05:47.013 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:47.011+0000 7f808c800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f808410f3c0 0x7f80841a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:05:47.013 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:47.011+0000 7f808c800640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f8084114fd0 con 0x7f8084075540 2024-09-11T00:05:47.013 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:47.011+0000 7f808c800640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f8084114e50 con 0x7f808410f3c0 2024-09-11T00:05:47.013 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:47.011+0000 7f808c800640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f8084115150 con 0x7f8084075f40 2024-09-11T00:05:47.013 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:47.011+0000 7f8089e00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8084075540 0x7f808419b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:05:47.013 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:47.012+0000 7f8089400640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8084075f40 0x7f808419ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:05:47.014 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:47.012+0000 7f8089e00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8084075540 0x7f808419b520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:39912/0 (socket says 172.21.15.12:39912) 2024-09-11T00:05:47.014 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:47.012+0000 7f8089e00640 1 -- 172.21.15.12:0/1808972012 learned_addr learned my addr 172.21.15.12:0/1808972012 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:05:47.014 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:47.012+0000 7f808a800640 1 --2- 172.21.15.12:0/1808972012 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f808410f3c0 0x7f80841a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:05:47.014 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:47.012+0000 7f8089e00640 1 -- 172.21.15.12:0/1808972012 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8084075f40 msgr2=0x7f808419ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:05:47.014 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:47.012+0000 7f8089e00640 1 --2- 172.21.15.12:0/1808972012 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8084075f40 0x7f808419ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:05:47.014 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:47.012+0000 7f8089e00640 1 -- 172.21.15.12:0/1808972012 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f808410f3c0 msgr2=0x7f80841a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:05:47.014 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:47.012+0000 7f8089e00640 1 --2- 172.21.15.12:0/1808972012 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f808410f3c0 0x7f80841a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:05:47.014 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:47.012+0000 7f8089e00640 1 -- 172.21.15.12:0/1808972012 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80841a15d0 con 0x7f8084075540 2024-09-11T00:05:47.015 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:47.012+0000 7f8089e00640 1 --2- 172.21.15.12:0/1808972012 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8084075540 0x7f808419b520 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f807800e9f0 tx=0x7f807800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:05:47.015 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:47.013+0000 7f807ea00640 1 -- 172.21.15.12:0/1808972012 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f807800cd80 con 0x7f8084075540 2024-09-11T00:05:47.015 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:47.013+0000 7f807ea00640 1 -- 172.21.15.12:0/1808972012 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f80780108e0 con 0x7f8084075540 2024-09-11T00:05:47.015 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:47.013+0000 7f808c800640 1 -- 172.21.15.12:0/1808972012 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f80841a18c0 con 0x7f8084075540 2024-09-11T00:05:47.015 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:47.013+0000 7f807ea00640 1 -- 172.21.15.12:0/1808972012 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8078018980 con 0x7f8084075540 2024-09-11T00:05:47.015 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:47.013+0000 7f808c800640 1 -- 172.21.15.12:0/1808972012 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80841a8150 con 0x7f8084075540 2024-09-11T00:05:47.015 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:47.014+0000 7f808c800640 1 -- 172.21.15.12:0/1808972012 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8084076a50 con 0x7f8084075540 2024-09-11T00:05:47.017 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:47.017+0000 7f807ea00640 1 -- 172.21.15.12:0/1808972012 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 17) ==== 153301+0+0 (secure 0 0 0) 0x7f8078010430 con 0x7f8084075540 2024-09-11T00:05:47.019 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:47.018+0000 7f807ea00640 1 --2- 172.21.15.12:0/1808972012 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f806c0b53f0 0x7f806c0b78b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:05:47.019 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:47.018+0000 7f807ea00640 1 -- 172.21.15.12:0/1808972012 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 9092+0+0 (secure 0 0 0) 0x7f80780e6d50 con 0x7f8084075540 2024-09-11T00:05:47.019 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:47.018+0000 7f8089400640 1 --2- 172.21.15.12:0/1808972012 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f806c0b53f0 0x7f806c0b78b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:05:47.020 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:47.019+0000 7f8089400640 1 --2- 172.21.15.12:0/1808972012 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f806c0b53f0 0x7f806c0b78b0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f8074005ab0 tx=0x7f8074005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:05:47.020 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:47.020+0000 7f807ea00640 1 -- 172.21.15.12:0/1808972012 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f80780ad6f0 con 0x7f8084075540 2024-09-11T00:05:47.217 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:47.216+0000 7f808c800640 1 -- 172.21.15.12:0/1808972012 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7f8084075920 con 0x7f8084075540 2024-09-11T00:05:47.217 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:47.216+0000 7f807ea00640 1 -- 172.21.15.12:0/1808972012 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f8084075920 con 0x7f8084075540 2024-09-11T00:05:47.218 INFO:teuthology.orchestra.run.smithi012.stdout:201863462966 2024-09-11T00:05:47.220 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:47.219+0000 7f808c800640 1 -- 172.21.15.12:0/1808972012 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f806c0b53f0 msgr2=0x7f806c0b78b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:05:47.220 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:47.219+0000 7f808c800640 1 --2- 172.21.15.12:0/1808972012 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f806c0b53f0 0x7f806c0b78b0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f8074005ab0 tx=0x7f8074005a20 comp rx=0 tx=0).stop 2024-09-11T00:05:47.220 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:47.219+0000 7f808c800640 1 -- 172.21.15.12:0/1808972012 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8084075540 msgr2=0x7f808419b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:05:47.220 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:47.219+0000 7f808c800640 1 --2- 172.21.15.12:0/1808972012 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8084075540 0x7f808419b520 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f807800e9f0 tx=0x7f807800eeb0 comp rx=0 tx=0).stop 2024-09-11T00:05:47.220 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:47.219+0000 7f808c800640 1 -- 172.21.15.12:0/1808972012 shutdown_connections 2024-09-11T00:05:47.220 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:47.219+0000 7f808c800640 1 --2- 172.21.15.12:0/1808972012 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f806c0b53f0 0x7f806c0b78b0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:05:47.221 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:47.219+0000 7f808c800640 1 --2- 172.21.15.12:0/1808972012 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f808410f3c0 0x7f80841a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:05:47.221 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:47.219+0000 7f808c800640 1 --2- 172.21.15.12:0/1808972012 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8084075f40 0x7f808419ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:05:47.221 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:47.219+0000 7f808c800640 1 --2- 172.21.15.12:0/1808972012 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8084075540 0x7f808419b520 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:05:47.221 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:47.219+0000 7f808c800640 1 -- 172.21.15.12:0/1808972012 >> 172.21.15.12:0/1808972012 conn(0x7f80840fe810 msgr2=0x7f8084111560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:05:47.221 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:47.219+0000 7f808c800640 1 -- 172.21.15.12:0/1808972012 shutdown_connections 2024-09-11T00:05:47.221 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:47.219+0000 7f808c800640 1 -- 172.21.15.12:0/1808972012 wait complete. 2024-09-11T00:05:47.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:05:47 smithi012 ceph-mon[30333]: pgmap v357: 129 pgs: 129 active+clean; 647 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-11T00:05:47.532 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:05:47 smithi165 ceph-mon[35356]: pgmap v357: 129 pgs: 129 active+clean; 647 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-11T00:05:47.581 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:05:47 smithi178 ceph-mon[35919]: pgmap v357: 129 pgs: 129 active+clean; 647 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-11T00:05:47.987 INFO:tasks.cephadm.ceph_manager.ceph:need seq 201863462963 got 201863462966 for osd.7 2024-09-11T00:05:47.987 DEBUG:teuthology.parallel:result is None 2024-09-11T00:05:47.987 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-09-11T00:05:47.988 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid ff3ad74a-6fcf-11ef-bcea-c7b262605968 -- ceph pg dump --format=json 2024-09-11T00:05:48.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:05:48 smithi012 ceph-mon[30333]: from='client.? 172.21.15.12:0/1808972012' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 7} : dispatch 2024-09-11T00:05:48.370 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:05:48.532 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:05:48 smithi165 ceph-mon[35356]: from='client.? 172.21.15.12:0/1808972012' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 7} : dispatch 2024-09-11T00:05:48.580 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:05:48 smithi178 ceph-mon[35919]: from='client.? 172.21.15.12:0/1808972012' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 7} : dispatch 2024-09-11T00:05:48.943 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:48.942+0000 7fb74e800640 1 -- 172.21.15.12:0/1648294157 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb748075f40 msgr2=0x7fb74810ee10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:05:48.943 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:48.942+0000 7fb74e800640 1 --2- 172.21.15.12:0/1648294157 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb748075f40 0x7fb74810ee10 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fb730009900 tx=0x7fb73002a410 comp rx=0 tx=0).stop 2024-09-11T00:05:48.944 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:48.942+0000 7fb74e800640 1 -- 172.21.15.12:0/1648294157 shutdown_connections 2024-09-11T00:05:48.944 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:48.942+0000 7fb74e800640 1 --2- 172.21.15.12:0/1648294157 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb74810f3e0 0x7fb7481117d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:05:48.944 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:48.942+0000 7fb74e800640 1 --2- 172.21.15.12:0/1648294157 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb748075f40 0x7fb74810ee10 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:05:48.944 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:48.942+0000 7fb74e800640 1 --2- 172.21.15.12:0/1648294157 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb748075540 0x7fb748075920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:05:48.944 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:48.942+0000 7fb74e800640 1 -- 172.21.15.12:0/1648294157 >> 172.21.15.12:0/1648294157 conn(0x7fb7480fe830 msgr2=0x7fb748100c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:05:48.944 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:48.942+0000 7fb74e800640 1 -- 172.21.15.12:0/1648294157 shutdown_connections 2024-09-11T00:05:48.945 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:48.942+0000 7fb74e800640 1 -- 172.21.15.12:0/1648294157 wait complete. 2024-09-11T00:05:48.952 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:48.943+0000 7fb74e800640 1 Processor -- start 2024-09-11T00:05:48.952 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:48.943+0000 7fb74e800640 1 -- start start 2024-09-11T00:05:48.952 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:48.943+0000 7fb74e800640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb748075540 0x7fb74819b4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:05:48.952 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:48.943+0000 7fb74e800640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb748075f40 0x7fb74819ba30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:05:48.952 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:48.943+0000 7fb74e800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb74810f3e0 0x7fb7481a0dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:05:48.952 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:48.943+0000 7fb74e800640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fb748114fe0 con 0x7fb748075f40 2024-09-11T00:05:48.953 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:48.943+0000 7fb74e800640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fb748114e60 con 0x7fb74810f3e0 2024-09-11T00:05:48.953 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:48.943+0000 7fb74e800640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fb748115160 con 0x7fb748075540 2024-09-11T00:05:48.953 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:48.944+0000 7fb73ea00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb748075f40 0x7fb74819ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:05:48.953 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:48.944+0000 7fb73ea00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb748075f40 0x7fb74819ba30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:56904/0 (socket says 172.21.15.12:56904) 2024-09-11T00:05:48.953 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:48.944+0000 7fb73ea00640 1 -- 172.21.15.12:0/2803238647 learned_addr learned my addr 172.21.15.12:0/2803238647 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:05:48.953 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:48.944+0000 7fb747400640 1 --2- 172.21.15.12:0/2803238647 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb748075540 0x7fb74819b4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:05:48.954 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:48.944+0000 7fb747e00640 1 --2- 172.21.15.12:0/2803238647 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb74810f3e0 0x7fb7481a0dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:05:48.954 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:48.944+0000 7fb73ea00640 1 -- 172.21.15.12:0/2803238647 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb748075540 msgr2=0x7fb74819b4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:05:48.954 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:48.944+0000 7fb73ea00640 1 --2- 172.21.15.12:0/2803238647 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb748075540 0x7fb74819b4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:05:48.954 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:48.944+0000 7fb73ea00640 1 -- 172.21.15.12:0/2803238647 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb74810f3e0 msgr2=0x7fb7481a0dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:05:48.954 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:48.944+0000 7fb73ea00640 1 --2- 172.21.15.12:0/2803238647 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb74810f3e0 0x7fb7481a0dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:05:48.954 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:48.944+0000 7fb73ea00640 1 -- 172.21.15.12:0/2803238647 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7481a15a0 con 0x7fb748075f40 2024-09-11T00:05:48.954 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:48.944+0000 7fb747400640 1 --2- 172.21.15.12:0/2803238647 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb748075540 0x7fb74819b4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:05:48.954 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:48.944+0000 7fb747e00640 1 --2- 172.21.15.12:0/2803238647 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb74810f3e0 0x7fb7481a0dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:05:48.954 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:48.945+0000 7fb73ea00640 1 --2- 172.21.15.12:0/2803238647 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb748075f40 0x7fb74819ba30 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fb730005ab0 tx=0x7fb730003c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:05:48.955 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:48.945+0000 7fb744c00640 1 -- 172.21.15.12:0/2803238647 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb73003f070 con 0x7fb748075f40 2024-09-11T00:05:48.955 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:48.945+0000 7fb744c00640 1 -- 172.21.15.12:0/2803238647 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb730031d60 con 0x7fb748075f40 2024-09-11T00:05:48.955 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:48.945+0000 7fb744c00640 1 -- 172.21.15.12:0/2803238647 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb730035030 con 0x7fb748075f40 2024-09-11T00:05:48.955 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:48.945+0000 7fb74e800640 1 -- 172.21.15.12:0/2803238647 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb7481a1830 con 0x7fb748075f40 2024-09-11T00:05:48.955 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:48.945+0000 7fb74e800640 1 -- 172.21.15.12:0/2803238647 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7481a7f50 con 0x7fb748075f40 2024-09-11T00:05:48.955 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:48.946+0000 7fb74e800640 1 -- 172.21.15.12:0/2803238647 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb748075920 con 0x7fb748075f40 2024-09-11T00:05:48.955 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:48.948+0000 7fb744c00640 1 -- 172.21.15.12:0/2803238647 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 17) ==== 153301+0+0 (secure 0 0 0) 0x7fb73003b460 con 0x7fb748075f40 2024-09-11T00:05:48.955 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:48.950+0000 7fb744c00640 1 --2- 172.21.15.12:0/2803238647 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fb72c0b53f0 0x7fb72c0b78b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:05:48.956 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:48.950+0000 7fb744c00640 1 -- 172.21.15.12:0/2803238647 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 9092+0+0 (secure 0 0 0) 0x7fb730106b50 con 0x7fb748075f40 2024-09-11T00:05:48.956 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:48.950+0000 7fb747400640 1 --2- 172.21.15.12:0/2803238647 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fb72c0b53f0 0x7fb72c0b78b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:05:48.956 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:48.951+0000 7fb747400640 1 --2- 172.21.15.12:0/2803238647 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fb72c0b53f0 0x7fb72c0b78b0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fb738005fd0 tx=0x7fb73800b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:05:48.956 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:48.954+0000 7fb744c00640 1 -- 172.21.15.12:0/2803238647 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7fb7300ccea0 con 0x7fb748075f40 2024-09-11T00:05:49.148 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:49.147+0000 7fb74e800640 1 -- 172.21.15.12:0/2803238647 --> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fb748063590 con 0x7fb72c0b53f0 2024-09-11T00:05:49.154 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:49.153+0000 7fb744c00640 1 -- 172.21.15.12:0/2803238647 <== mgr.14150 v2:172.21.15.12:6800/3442939137 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340395 (secure 0 0 0) 0x7fb748063590 con 0x7fb72c0b53f0 2024-09-11T00:05:49.160 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-11T00:05:49.161 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-11T00:05:49.164 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:49.157+0000 7fb74e800640 1 -- 172.21.15.12:0/2803238647 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fb72c0b53f0 msgr2=0x7fb72c0b78b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:05:49.165 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:49.157+0000 7fb74e800640 1 --2- 172.21.15.12:0/2803238647 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fb72c0b53f0 0x7fb72c0b78b0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fb738005fd0 tx=0x7fb73800b390 comp rx=0 tx=0).stop 2024-09-11T00:05:49.165 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:49.157+0000 7fb74e800640 1 -- 172.21.15.12:0/2803238647 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb748075f40 msgr2=0x7fb74819ba30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:05:49.165 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:49.157+0000 7fb74e800640 1 --2- 172.21.15.12:0/2803238647 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb748075f40 0x7fb74819ba30 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fb730005ab0 tx=0x7fb730003c80 comp rx=0 tx=0).stop 2024-09-11T00:05:49.165 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:49.158+0000 7fb74e800640 1 -- 172.21.15.12:0/2803238647 shutdown_connections 2024-09-11T00:05:49.165 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:49.158+0000 7fb74e800640 1 --2- 172.21.15.12:0/2803238647 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fb72c0b53f0 0x7fb72c0b78b0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:05:49.165 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:49.158+0000 7fb74e800640 1 --2- 172.21.15.12:0/2803238647 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb74810f3e0 0x7fb7481a0dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:05:49.165 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:49.158+0000 7fb74e800640 1 --2- 172.21.15.12:0/2803238647 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb748075f40 0x7fb74819ba30 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:05:49.165 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:49.158+0000 7fb74e800640 1 --2- 172.21.15.12:0/2803238647 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb748075540 0x7fb74819b4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:05:49.165 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:49.158+0000 7fb74e800640 1 -- 172.21.15.12:0/2803238647 >> 172.21.15.12:0/2803238647 conn(0x7fb7480fe830 msgr2=0x7fb7480ff430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:05:49.165 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:49.158+0000 7fb74e800640 1 -- 172.21.15.12:0/2803238647 shutdown_connections 2024-09-11T00:05:49.165 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:49.158+0000 7fb74e800640 1 -- 172.21.15.12:0/2803238647 wait complete. 2024-09-11T00:05:49.532 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:05:49 smithi165 ceph-mon[35356]: pgmap v358: 129 pgs: 129 active+clean; 647 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-11T00:05:49.549 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:05:49 smithi012 ceph-mon[30333]: pgmap v358: 129 pgs: 129 active+clean; 647 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-11T00:05:49.580 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:05:49 smithi178 ceph-mon[35919]: pgmap v358: 129 pgs: 129 active+clean; 647 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-11T00:05:49.962 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":358,"stamp":"2024-09-11T00:05:48.116499+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":662424,"num_objects":102,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":673,"num_read_kb":441,"num_write":374,"num_write_kb":2296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":23,"num_bytes_recovered":2951952,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":309,"ondisk_log_size":309,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":379244,"kb_used_data":13280,"kb_used_omap":18,"kb_used_meta":365677,"kb_avail":1124562580,"statfs":{"total":1151940427776,"available":1151552081920,"internally_reserved":0,"allocated":13598720,"data_stored":7569746,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19142,"internal_metadata":374453562},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":100,"num_read_kb":50,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003323"},"pg_stats":[{"pgid":"2.1f","version":"81'13","reported_seq":106,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T00:05:43.339219+0000","last_change":"2024-09-11T00:04:01.820872+0000","last_active":"2024-09-11T00:05:43.339219+0000","last_peered":"2024-09-11T00:05:43.339219+0000","last_clean":"2024-09-11T00:05:43.339219+0000","last_became_active":"2024-09-11T00:04:01.820404+0000","last_became_peered":"2024-09-11T00:04:01.820404+0000","last_unstale":"2024-09-11T00:05:43.339219+0000","last_undegraded":"2024-09-11T00:05:43.339219+0000","last_fullsized":"2024-09-11T00:05:43.339219+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_clean_scrub_stamp":"2024-09-11T00:04:00.794959+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:15:38.648289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":56,"num_read_kb":56,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T00:05:19.847074+0000","last_change":"2024-09-11T00:05:19.846978+0000","last_active":"2024-09-11T00:05:19.847074+0000","last_peered":"2024-09-11T00:05:19.847074+0000","last_clean":"2024-09-11T00:05:19.847074+0000","last_became_active":"2024-09-11T00:05:19.846569+0000","last_became_peered":"2024-09-11T00:05:19.846569+0000","last_unstale":"2024-09-11T00:05:19.847074+0000","last_undegraded":"2024-09-11T00:05:19.847074+0000","last_fullsized":"2024-09-11T00:05:19.847074+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:27:46.235552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,5],"acting":[9,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T00:05:19.836785+0000","last_change":"2024-09-11T00:04:11.872109+0000","last_active":"2024-09-11T00:05:19.836785+0000","last_peered":"2024-09-11T00:05:19.836785+0000","last_clean":"2024-09-11T00:05:19.836785+0000","last_became_active":"2024-09-11T00:04:11.871279+0000","last_became_peered":"2024-09-11T00:04:11.871279+0000","last_unstale":"2024-09-11T00:05:19.836785+0000","last_undegraded":"2024-09-11T00:05:19.836785+0000","last_fullsized":"2024-09-11T00:05:19.836785+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_clean_scrub_stamp":"2024-09-11T00:04:10.845136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:13:32.837555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"81'2","reported_seq":39,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T00:05:19.845403+0000","last_change":"2024-09-11T00:04:01.942288+0000","last_active":"2024-09-11T00:05:19.845403+0000","last_peered":"2024-09-11T00:05:19.845403+0000","last_clean":"2024-09-11T00:05:19.845403+0000","last_became_active":"2024-09-11T00:04:01.941838+0000","last_became_peered":"2024-09-11T00:04:01.941838+0000","last_unstale":"2024-09-11T00:05:19.845403+0000","last_undegraded":"2024-09-11T00:05:19.845403+0000","last_fullsized":"2024-09-11T00:05:19.845403+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_clean_scrub_stamp":"2024-09-11T00:04:00.794959+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:48:44.867267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T00:05:19.840736+0000","last_change":"2024-09-11T00:04:03.839951+0000","last_active":"2024-09-11T00:05:19.840736+0000","last_peered":"2024-09-11T00:05:19.840736+0000","last_clean":"2024-09-11T00:05:19.840736+0000","last_became_active":"2024-09-11T00:04:03.839591+0000","last_became_peered":"2024-09-11T00:04:03.839591+0000","last_unstale":"2024-09-11T00:05:19.840736+0000","last_undegraded":"2024-09-11T00:05:19.840736+0000","last_fullsized":"2024-09-11T00:05:19.840736+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:12:49.603310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T00:05:19.983148+0000","last_change":"2024-09-11T00:04:11.872026+0000","last_active":"2024-09-11T00:05:19.983148+0000","last_peered":"2024-09-11T00:05:19.983148+0000","last_clean":"2024-09-11T00:05:19.983148+0000","last_became_active":"2024-09-11T00:04:11.871699+0000","last_became_peered":"2024-09-11T00:04:11.871699+0000","last_unstale":"2024-09-11T00:05:19.983148+0000","last_undegraded":"2024-09-11T00:05:19.983148+0000","last_fullsized":"2024-09-11T00:05:19.983148+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_clean_scrub_stamp":"2024-09-11T00:04:10.845136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:07:47.757223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"81'12","reported_seq":50,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T00:05:19.842919+0000","last_change":"2024-09-11T00:04:01.819942+0000","last_active":"2024-09-11T00:05:19.842919+0000","last_peered":"2024-09-11T00:05:19.842919+0000","last_clean":"2024-09-11T00:05:19.842919+0000","last_became_active":"2024-09-11T00:04:01.819337+0000","last_became_peered":"2024-09-11T00:04:01.819337+0000","last_unstale":"2024-09-11T00:05:19.842919+0000","last_undegraded":"2024-09-11T00:05:19.842919+0000","last_fullsized":"2024-09-11T00:05:19.842919+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_clean_scrub_stamp":"2024-09-11T00:04:00.794959+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:22:26.947067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T00:05:19.837468+0000","last_change":"2024-09-11T00:04:03.839276+0000","last_active":"2024-09-11T00:05:19.837468+0000","last_peered":"2024-09-11T00:05:19.837468+0000","last_clean":"2024-09-11T00:05:19.837468+0000","last_became_active":"2024-09-11T00:04:03.838843+0000","last_became_peered":"2024-09-11T00:04:03.838843+0000","last_unstale":"2024-09-11T00:05:19.837468+0000","last_undegraded":"2024-09-11T00:05:19.837468+0000","last_fullsized":"2024-09-11T00:05:19.837468+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:26:42.640628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T00:05:19.983852+0000","last_change":"2024-09-11T00:04:11.872345+0000","last_active":"2024-09-11T00:05:19.983852+0000","last_peered":"2024-09-11T00:05:19.983852+0000","last_clean":"2024-09-11T00:05:19.983852+0000","last_became_active":"2024-09-11T00:04:11.871994+0000","last_became_peered":"2024-09-11T00:04:11.871994+0000","last_unstale":"2024-09-11T00:05:19.983852+0000","last_undegraded":"2024-09-11T00:05:19.983852+0000","last_fullsized":"2024-09-11T00:05:19.983852+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_clean_scrub_stamp":"2024-09-11T00:04:10.845136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:14:23.906778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"81'5","reported_seq":43,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T00:05:19.844344+0000","last_change":"2024-09-11T00:04:01.819581+0000","last_active":"2024-09-11T00:05:19.844344+0000","last_peered":"2024-09-11T00:05:19.844344+0000","last_clean":"2024-09-11T00:05:19.844344+0000","last_became_active":"2024-09-11T00:04:01.818699+0000","last_became_peered":"2024-09-11T00:04:01.818699+0000","last_unstale":"2024-09-11T00:05:19.844344+0000","last_undegraded":"2024-09-11T00:05:19.844344+0000","last_fullsized":"2024-09-11T00:05:19.844344+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_clean_scrub_stamp":"2024-09-11T00:04:00.794959+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:07:24.540825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T00:05:19.844097+0000","last_change":"2024-09-11T00:04:03.835107+0000","last_active":"2024-09-11T00:05:19.844097+0000","last_peered":"2024-09-11T00:05:19.844097+0000","last_clean":"2024-09-11T00:05:19.844097+0000","last_became_active":"2024-09-11T00:04:03.834782+0000","last_became_peered":"2024-09-11T00:04:03.834782+0000","last_unstale":"2024-09-11T00:05:19.844097+0000","last_undegraded":"2024-09-11T00:05:19.844097+0000","last_fullsized":"2024-09-11T00:05:19.844097+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:46:54.472270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T00:05:19.983645+0000","last_change":"2024-09-11T00:04:11.874168+0000","last_active":"2024-09-11T00:05:19.983645+0000","last_peered":"2024-09-11T00:05:19.983645+0000","last_clean":"2024-09-11T00:05:19.983645+0000","last_became_active":"2024-09-11T00:04:11.873810+0000","last_became_peered":"2024-09-11T00:04:11.873810+0000","last_unstale":"2024-09-11T00:05:19.983645+0000","last_undegraded":"2024-09-11T00:05:19.983645+0000","last_fullsized":"2024-09-11T00:05:19.983645+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_clean_scrub_stamp":"2024-09-11T00:04:10.845136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:06:05.130815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T00:05:19.840764+0000","last_change":"2024-09-11T00:04:03.834016+0000","last_active":"2024-09-11T00:05:19.840764+0000","last_peered":"2024-09-11T00:05:19.840764+0000","last_clean":"2024-09-11T00:05:19.840764+0000","last_became_active":"2024-09-11T00:04:03.833374+0000","last_became_peered":"2024-09-11T00:04:03.833374+0000","last_unstale":"2024-09-11T00:05:19.840764+0000","last_undegraded":"2024-09-11T00:05:19.840764+0000","last_fullsized":"2024-09-11T00:05:19.840764+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:13:34.068594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"81'4","reported_seq":42,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T00:05:19.842488+0000","last_change":"2024-09-11T00:04:01.821173+0000","last_active":"2024-09-11T00:05:19.842488+0000","last_peered":"2024-09-11T00:05:19.842488+0000","last_clean":"2024-09-11T00:05:19.842488+0000","last_became_active":"2024-09-11T00:04:01.820615+0000","last_became_peered":"2024-09-11T00:04:01.820615+0000","last_unstale":"2024-09-11T00:05:19.842488+0000","last_undegraded":"2024-09-11T00:05:19.842488+0000","last_fullsized":"2024-09-11T00:05:19.842488+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_clean_scrub_stamp":"2024-09-11T00:04:00.794959+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:22:11.842837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T00:05:19.983802+0000","last_change":"2024-09-11T00:04:12.077637+0000","last_active":"2024-09-11T00:05:19.983802+0000","last_peered":"2024-09-11T00:05:19.983802+0000","last_clean":"2024-09-11T00:05:19.983802+0000","last_became_active":"2024-09-11T00:04:12.076450+0000","last_became_peered":"2024-09-11T00:04:12.076450+0000","last_unstale":"2024-09-11T00:05:19.983802+0000","last_undegraded":"2024-09-11T00:05:19.983802+0000","last_fullsized":"2024-09-11T00:05:19.983802+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_clean_scrub_stamp":"2024-09-11T00:04:10.845136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:50:54.870468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T00:05:19.844069+0000","last_change":"2024-09-11T00:04:03.846398+0000","last_active":"2024-09-11T00:05:19.844069+0000","last_peered":"2024-09-11T00:05:19.844069+0000","last_clean":"2024-09-11T00:05:19.844069+0000","last_became_active":"2024-09-11T00:04:03.846051+0000","last_became_peered":"2024-09-11T00:04:03.846051+0000","last_unstale":"2024-09-11T00:05:19.844069+0000","last_undegraded":"2024-09-11T00:05:19.844069+0000","last_fullsized":"2024-09-11T00:05:19.844069+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:00:38.506122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"81'3","reported_seq":18,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T00:05:19.860144+0000","last_change":"2024-09-11T00:05:19.860048+0000","last_active":"2024-09-11T00:05:19.860144+0000","last_peered":"2024-09-11T00:05:19.860144+0000","last_clean":"2024-09-11T00:05:19.860144+0000","last_became_active":"2024-09-11T00:05:19.845210+0000","last_became_peered":"2024-09-11T00:05:19.845210+0000","last_unstale":"2024-09-11T00:05:19.860144+0000","last_undegraded":"2024-09-11T00:05:19.860144+0000","last_fullsized":"2024-09-11T00:05:19.860144+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_clean_scrub_stamp":"2024-09-11T00:04:00.794959+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:55:18.935136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"81'1","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T00:05:19.984147+0000","last_change":"2024-09-11T00:04:11.879582+0000","last_active":"2024-09-11T00:05:19.984147+0000","last_peered":"2024-09-11T00:05:19.984147+0000","last_clean":"2024-09-11T00:05:19.984147+0000","last_became_active":"2024-09-11T00:04:11.879098+0000","last_became_peered":"2024-09-11T00:04:11.879098+0000","last_unstale":"2024-09-11T00:05:19.984147+0000","last_undegraded":"2024-09-11T00:05:19.984147+0000","last_fullsized":"2024-09-11T00:05:19.984147+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_clean_scrub_stamp":"2024-09-11T00:04:10.845136+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:06.292841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T00:05:19.839912+0000","last_change":"2024-09-11T00:04:03.834615+0000","last_active":"2024-09-11T00:05:19.839912+0000","last_peered":"2024-09-11T00:05:19.839912+0000","last_clean":"2024-09-11T00:05:19.839912+0000","last_became_active":"2024-09-11T00:04:03.834284+0000","last_became_peered":"2024-09-11T00:04:03.834284+0000","last_unstale":"2024-09-11T00:05:19.839912+0000","last_undegraded":"2024-09-11T00:05:19.839912+0000","last_fullsized":"2024-09-11T00:05:19.839912+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:24:09.542774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"81'7","reported_seq":44,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T00:05:19.843350+0000","last_change":"2024-09-11T00:04:01.818156+0000","last_active":"2024-09-11T00:05:19.843350+0000","last_peered":"2024-09-11T00:05:19.843350+0000","last_clean":"2024-09-11T00:05:19.843350+0000","last_became_active":"2024-09-11T00:04:01.817820+0000","last_became_peered":"2024-09-11T00:04:01.817820+0000","last_unstale":"2024-09-11T00:05:19.843350+0000","last_undegraded":"2024-09-11T00:05:19.843350+0000","last_fullsized":"2024-09-11T00:05:19.843350+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_clean_scrub_stamp":"2024-09-11T00:04:00.794959+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:15:46.304276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T00:05:19.840361+0000","last_change":"2024-09-11T00:04:11.877704+0000","last_active":"2024-09-11T00:05:19.840361+0000","last_peered":"2024-09-11T00:05:19.840361+0000","last_clean":"2024-09-11T00:05:19.840361+0000","last_became_active":"2024-09-11T00:04:11.877320+0000","last_became_peered":"2024-09-11T00:04:11.877320+0000","last_unstale":"2024-09-11T00:05:19.840361+0000","last_undegraded":"2024-09-11T00:05:19.840361+0000","last_fullsized":"2024-09-11T00:05:19.840361+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_clean_scrub_stamp":"2024-09-11T00:04:10.845136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:31:35.046012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T00:05:19.841786+0000","last_change":"2024-09-11T00:04:03.828240+0000","last_active":"2024-09-11T00:05:19.841786+0000","last_peered":"2024-09-11T00:05:19.841786+0000","last_clean":"2024-09-11T00:05:19.841786+0000","last_became_active":"2024-09-11T00:04:03.827890+0000","last_became_peered":"2024-09-11T00:04:03.827890+0000","last_unstale":"2024-09-11T00:05:19.841786+0000","last_undegraded":"2024-09-11T00:05:19.841786+0000","last_fullsized":"2024-09-11T00:05:19.841786+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:43:12.146717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"81'2","reported_seq":39,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T00:05:19.837299+0000","last_change":"2024-09-11T00:04:01.942805+0000","last_active":"2024-09-11T00:05:19.837299+0000","last_peered":"2024-09-11T00:05:19.837299+0000","last_clean":"2024-09-11T00:05:19.837299+0000","last_became_active":"2024-09-11T00:04:01.942429+0000","last_became_peered":"2024-09-11T00:04:01.942429+0000","last_unstale":"2024-09-11T00:05:19.837299+0000","last_undegraded":"2024-09-11T00:05:19.837299+0000","last_fullsized":"2024-09-11T00:05:19.837299+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_clean_scrub_stamp":"2024-09-11T00:04:00.794959+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:51:22.403825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T00:05:19.983874+0000","last_change":"2024-09-11T00:04:12.077246+0000","last_active":"2024-09-11T00:05:19.983874+0000","last_peered":"2024-09-11T00:05:19.983874+0000","last_clean":"2024-09-11T00:05:19.983874+0000","last_became_active":"2024-09-11T00:04:12.076481+0000","last_became_peered":"2024-09-11T00:04:12.076481+0000","last_unstale":"2024-09-11T00:05:19.983874+0000","last_undegraded":"2024-09-11T00:05:19.983874+0000","last_fullsized":"2024-09-11T00:05:19.983874+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_clean_scrub_stamp":"2024-09-11T00:04:10.845136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:55:54.347334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T00:05:19.840940+0000","last_change":"2024-09-11T00:04:03.838431+0000","last_active":"2024-09-11T00:05:19.840940+0000","last_peered":"2024-09-11T00:05:19.840940+0000","last_clean":"2024-09-11T00:05:19.840940+0000","last_became_active":"2024-09-11T00:04:03.834451+0000","last_became_peered":"2024-09-11T00:04:03.834451+0000","last_unstale":"2024-09-11T00:05:19.840940+0000","last_undegraded":"2024-09-11T00:05:19.840940+0000","last_fullsized":"2024-09-11T00:05:19.840940+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:23:01.594299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"81'7","reported_seq":45,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T00:05:19.984326+0000","last_change":"2024-09-11T00:04:02.588834+0000","last_active":"2024-09-11T00:05:19.984326+0000","last_peered":"2024-09-11T00:05:19.984326+0000","last_clean":"2024-09-11T00:05:19.984326+0000","last_became_active":"2024-09-11T00:04:02.588477+0000","last_became_peered":"2024-09-11T00:04:02.588477+0000","last_unstale":"2024-09-11T00:05:19.984326+0000","last_undegraded":"2024-09-11T00:05:19.984326+0000","last_fullsized":"2024-09-11T00:05:19.984326+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_clean_scrub_stamp":"2024-09-11T00:04:00.794959+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:31:21.805591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T00:05:19.836911+0000","last_change":"2024-09-11T00:04:11.863381+0000","last_active":"2024-09-11T00:05:19.836911+0000","last_peered":"2024-09-11T00:05:19.836911+0000","last_clean":"2024-09-11T00:05:19.836911+0000","last_became_active":"2024-09-11T00:04:11.862944+0000","last_became_peered":"2024-09-11T00:04:11.862944+0000","last_unstale":"2024-09-11T00:05:19.836911+0000","last_undegraded":"2024-09-11T00:05:19.836911+0000","last_fullsized":"2024-09-11T00:05:19.836911+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_clean_scrub_stamp":"2024-09-11T00:04:10.845136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:00:14.185992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T00:05:19.845724+0000","last_change":"2024-09-11T00:04:03.829750+0000","last_active":"2024-09-11T00:05:19.845724+0000","last_peered":"2024-09-11T00:05:19.845724+0000","last_clean":"2024-09-11T00:05:19.845724+0000","last_became_active":"2024-09-11T00:04:03.828166+0000","last_became_peered":"2024-09-11T00:04:03.828166+0000","last_unstale":"2024-09-11T00:05:19.845724+0000","last_undegraded":"2024-09-11T00:05:19.845724+0000","last_fullsized":"2024-09-11T00:05:19.845724+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:55:01.667075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"81'15","reported_seq":52,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T00:05:19.841319+0000","last_change":"2024-09-11T00:04:01.818083+0000","last_active":"2024-09-11T00:05:19.841319+0000","last_peered":"2024-09-11T00:05:19.841319+0000","last_clean":"2024-09-11T00:05:19.841319+0000","last_became_active":"2024-09-11T00:04:01.817784+0000","last_became_peered":"2024-09-11T00:04:01.817784+0000","last_unstale":"2024-09-11T00:05:19.841319+0000","last_undegraded":"2024-09-11T00:05:19.841319+0000","last_fullsized":"2024-09-11T00:05:19.841319+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_clean_scrub_stamp":"2024-09-11T00:04:00.794959+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:38:55.466913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":26,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T00:05:19.842101+0000","last_change":"2024-09-11T00:04:12.077053+0000","last_active":"2024-09-11T00:05:19.842101+0000","last_peered":"2024-09-11T00:05:19.842101+0000","last_clean":"2024-09-11T00:05:19.842101+0000","last_became_active":"2024-09-11T00:04:12.076632+0000","last_became_peered":"2024-09-11T00:04:12.076632+0000","last_unstale":"2024-09-11T00:05:19.842101+0000","last_undegraded":"2024-09-11T00:05:19.842101+0000","last_fullsized":"2024-09-11T00:05:19.842101+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_clean_scrub_stamp":"2024-09-11T00:04:10.845136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:19:38.321429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T00:05:19.836751+0000","last_change":"2024-09-11T00:04:03.830316+0000","last_active":"2024-09-11T00:05:19.836751+0000","last_peered":"2024-09-11T00:05:19.836751+0000","last_clean":"2024-09-11T00:05:19.836751+0000","last_became_active":"2024-09-11T00:04:03.829932+0000","last_became_peered":"2024-09-11T00:04:03.829932+0000","last_unstale":"2024-09-11T00:05:19.836751+0000","last_undegraded":"2024-09-11T00:05:19.836751+0000","last_fullsized":"2024-09-11T00:05:19.836751+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:06:14.546651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"81'4","reported_seq":42,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T00:05:19.843047+0000","last_change":"2024-09-11T00:04:01.943004+0000","last_active":"2024-09-11T00:05:19.843047+0000","last_peered":"2024-09-11T00:05:19.843047+0000","last_clean":"2024-09-11T00:05:19.843047+0000","last_became_active":"2024-09-11T00:04:01.942432+0000","last_became_peered":"2024-09-11T00:04:01.942432+0000","last_unstale":"2024-09-11T00:05:19.843047+0000","last_undegraded":"2024-09-11T00:05:19.843047+0000","last_fullsized":"2024-09-11T00:05:19.843047+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_clean_scrub_stamp":"2024-09-11T00:04:00.794959+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:52:50.472596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T00:05:19.838838+0000","last_change":"2024-09-11T00:04:11.866778+0000","last_active":"2024-09-11T00:05:19.838838+0000","last_peered":"2024-09-11T00:05:19.838838+0000","last_clean":"2024-09-11T00:05:19.838838+0000","last_became_active":"2024-09-11T00:04:11.866333+0000","last_became_peered":"2024-09-11T00:04:11.866333+0000","last_unstale":"2024-09-11T00:05:19.838838+0000","last_undegraded":"2024-09-11T00:05:19.838838+0000","last_fullsized":"2024-09-11T00:05:19.838838+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_clean_scrub_stamp":"2024-09-11T00:04:10.845136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:33.680119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T00:05:19.983306+0000","last_change":"2024-09-11T00:04:03.832481+0000","last_active":"2024-09-11T00:05:19.983306+0000","last_peered":"2024-09-11T00:05:19.983306+0000","last_clean":"2024-09-11T00:05:19.983306+0000","last_became_active":"2024-09-11T00:04:03.832162+0000","last_became_peered":"2024-09-11T00:04:03.832162+0000","last_unstale":"2024-09-11T00:05:19.983306+0000","last_undegraded":"2024-09-11T00:05:19.983306+0000","last_fullsized":"2024-09-11T00:05:19.983306+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:32:52.550128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"81'1","reported_seq":38,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T00:05:19.839966+0000","last_change":"2024-09-11T00:04:01.943647+0000","last_active":"2024-09-11T00:05:19.839966+0000","last_peered":"2024-09-11T00:05:19.839966+0000","last_clean":"2024-09-11T00:05:19.839966+0000","last_became_active":"2024-09-11T00:04:01.942714+0000","last_became_peered":"2024-09-11T00:04:01.942714+0000","last_unstale":"2024-09-11T00:05:19.839966+0000","last_undegraded":"2024-09-11T00:05:19.839966+0000","last_fullsized":"2024-09-11T00:05:19.839966+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_clean_scrub_stamp":"2024-09-11T00:04:00.794959+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:22:23.687268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T00:05:19.841947+0000","last_change":"2024-09-11T00:04:12.077133+0000","last_active":"2024-09-11T00:05:19.841947+0000","last_peered":"2024-09-11T00:05:19.841947+0000","last_clean":"2024-09-11T00:05:19.841947+0000","last_became_active":"2024-09-11T00:04:12.076657+0000","last_became_peered":"2024-09-11T00:04:12.076657+0000","last_unstale":"2024-09-11T00:05:19.841947+0000","last_undegraded":"2024-09-11T00:05:19.841947+0000","last_fullsized":"2024-09-11T00:05:19.841947+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_clean_scrub_stamp":"2024-09-11T00:04:10.845136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:05:40.643405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T00:05:19.983081+0000","last_change":"2024-09-11T00:04:03.835454+0000","last_active":"2024-09-11T00:05:19.983081+0000","last_peered":"2024-09-11T00:05:19.983081+0000","last_clean":"2024-09-11T00:05:19.983081+0000","last_became_active":"2024-09-11T00:04:03.835122+0000","last_became_peered":"2024-09-11T00:04:03.835122+0000","last_unstale":"2024-09-11T00:05:19.983081+0000","last_undegraded":"2024-09-11T00:05:19.983081+0000","last_fullsized":"2024-09-11T00:05:19.983081+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:25:49.672766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"81'6","reported_seq":101,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T00:05:44.341301+0000","last_change":"2024-09-11T00:04:01.823738+0000","last_active":"2024-09-11T00:05:44.341301+0000","last_peered":"2024-09-11T00:05:44.341301+0000","last_clean":"2024-09-11T00:05:44.341301+0000","last_became_active":"2024-09-11T00:04:01.823320+0000","last_became_peered":"2024-09-11T00:04:01.823320+0000","last_unstale":"2024-09-11T00:05:44.341301+0000","last_undegraded":"2024-09-11T00:05:44.341301+0000","last_fullsized":"2024-09-11T00:05:44.341301+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_clean_scrub_stamp":"2024-09-11T00:04:00.794959+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:34:30.088454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":58,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T00:05:19.836596+0000","last_change":"2024-09-11T00:04:11.863906+0000","last_active":"2024-09-11T00:05:19.836596+0000","last_peered":"2024-09-11T00:05:19.836596+0000","last_clean":"2024-09-11T00:05:19.836596+0000","last_became_active":"2024-09-11T00:04:11.863571+0000","last_became_peered":"2024-09-11T00:04:11.863571+0000","last_unstale":"2024-09-11T00:05:19.836596+0000","last_undegraded":"2024-09-11T00:05:19.836596+0000","last_fullsized":"2024-09-11T00:05:19.836596+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_clean_scrub_stamp":"2024-09-11T00:04:10.845136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:04:40.288886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T00:05:19.843539+0000","last_change":"2024-09-11T00:04:03.826623+0000","last_active":"2024-09-11T00:05:19.843539+0000","last_peered":"2024-09-11T00:05:19.843539+0000","last_clean":"2024-09-11T00:05:19.843539+0000","last_became_active":"2024-09-11T00:04:03.826202+0000","last_became_peered":"2024-09-11T00:04:03.826202+0000","last_unstale":"2024-09-11T00:05:19.843539+0000","last_undegraded":"2024-09-11T00:05:19.843539+0000","last_fullsized":"2024-09-11T00:05:19.843539+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:23:13.222654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"81'5","reported_seq":98,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T00:05:43.339329+0000","last_change":"2024-09-11T00:04:01.942857+0000","last_active":"2024-09-11T00:05:43.339329+0000","last_peered":"2024-09-11T00:05:43.339329+0000","last_clean":"2024-09-11T00:05:43.339329+0000","last_became_active":"2024-09-11T00:04:01.942494+0000","last_became_peered":"2024-09-11T00:04:01.942494+0000","last_unstale":"2024-09-11T00:05:43.339329+0000","last_undegraded":"2024-09-11T00:05:43.339329+0000","last_fullsized":"2024-09-11T00:05:43.339329+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_clean_scrub_stamp":"2024-09-11T00:04:00.794959+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:40:38.587178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":56,"num_read_kb":56,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T00:05:19.836940+0000","last_change":"2024-09-11T00:04:12.077771+0000","last_active":"2024-09-11T00:05:19.836940+0000","last_peered":"2024-09-11T00:05:19.836940+0000","last_clean":"2024-09-11T00:05:19.836940+0000","last_became_active":"2024-09-11T00:04:12.077366+0000","last_became_peered":"2024-09-11T00:04:12.077366+0000","last_unstale":"2024-09-11T00:05:19.836940+0000","last_undegraded":"2024-09-11T00:05:19.836940+0000","last_fullsized":"2024-09-11T00:05:19.836940+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_clean_scrub_stamp":"2024-09-11T00:04:10.845136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:04:26.808201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T00:05:19.836365+0000","last_change":"2024-09-11T00:04:03.827872+0000","last_active":"2024-09-11T00:05:19.836365+0000","last_peered":"2024-09-11T00:05:19.836365+0000","last_clean":"2024-09-11T00:05:19.836365+0000","last_became_active":"2024-09-11T00:04:03.827577+0000","last_became_peered":"2024-09-11T00:04:03.827577+0000","last_unstale":"2024-09-11T00:05:19.836365+0000","last_undegraded":"2024-09-11T00:05:19.836365+0000","last_fullsized":"2024-09-11T00:05:19.836365+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:37:19.205218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"81'4","reported_seq":52,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T00:05:19.991478+0000","last_change":"2024-09-11T00:05:19.991372+0000","last_active":"2024-09-11T00:05:19.991478+0000","last_peered":"2024-09-11T00:05:19.991478+0000","last_clean":"2024-09-11T00:05:19.991478+0000","last_became_active":"2024-09-11T00:05:19.985328+0000","last_became_peered":"2024-09-11T00:05:19.985328+0000","last_unstale":"2024-09-11T00:05:19.991478+0000","last_undegraded":"2024-09-11T00:05:19.991478+0000","last_fullsized":"2024-09-11T00:05:19.991478+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_clean_scrub_stamp":"2024-09-11T00:04:00.794959+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:17:20.345615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,4],"acting":[7,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T00:05:19.843837+0000","last_change":"2024-09-11T00:04:12.077554+0000","last_active":"2024-09-11T00:05:19.843837+0000","last_peered":"2024-09-11T00:05:19.843837+0000","last_clean":"2024-09-11T00:05:19.843837+0000","last_became_active":"2024-09-11T00:04:12.076619+0000","last_became_peered":"2024-09-11T00:04:12.076619+0000","last_unstale":"2024-09-11T00:05:19.843837+0000","last_undegraded":"2024-09-11T00:05:19.843837+0000","last_fullsized":"2024-09-11T00:05:19.843837+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_clean_scrub_stamp":"2024-09-11T00:04:10.845136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:30:25.505200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T00:05:19.843455+0000","last_change":"2024-09-11T00:04:03.839615+0000","last_active":"2024-09-11T00:05:19.843455+0000","last_peered":"2024-09-11T00:05:19.843455+0000","last_clean":"2024-09-11T00:05:19.843455+0000","last_became_active":"2024-09-11T00:04:03.832175+0000","last_became_peered":"2024-09-11T00:04:03.832175+0000","last_unstale":"2024-09-11T00:05:19.843455+0000","last_undegraded":"2024-09-11T00:05:19.843455+0000","last_fullsized":"2024-09-11T00:05:19.843455+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:17:09.975345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"81'6","reported_seq":46,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T00:05:19.984323+0000","last_change":"2024-09-11T00:04:01.817248+0000","last_active":"2024-09-11T00:05:19.984323+0000","last_peered":"2024-09-11T00:05:19.984323+0000","last_clean":"2024-09-11T00:05:19.984323+0000","last_became_active":"2024-09-11T00:04:01.816896+0000","last_became_peered":"2024-09-11T00:04:01.816896+0000","last_unstale":"2024-09-11T00:05:19.984323+0000","last_undegraded":"2024-09-11T00:05:19.984323+0000","last_fullsized":"2024-09-11T00:05:19.984323+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_clean_scrub_stamp":"2024-09-11T00:04:00.794959+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:46:51.641881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'92","reported_seq":110,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T00:05:19.840658+0000","last_change":"2024-09-11T00:02:47.245193+0000","last_active":"2024-09-11T00:05:19.840658+0000","last_peered":"2024-09-11T00:05:19.840658+0000","last_clean":"2024-09-11T00:05:19.840658+0000","last_became_active":"2024-09-11T00:02:47.239773+0000","last_became_peered":"2024-09-11T00:02:47.239773+0000","last_unstale":"2024-09-11T00:05:19.840658+0000","last_undegraded":"2024-09-11T00:05:19.840658+0000","last_fullsized":"2024-09-11T00:05:19.840658+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:59:10.637655+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:59:10.637655+0000","last_clean_scrub_stamp":"2024-09-10T23:59:10.637655+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:51:52.914831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"81'1","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T00:05:19.841343+0000","last_change":"2024-09-11T00:04:11.867764+0000","last_active":"2024-09-11T00:05:19.841343+0000","last_peered":"2024-09-11T00:05:19.841343+0000","last_clean":"2024-09-11T00:05:19.841343+0000","last_became_active":"2024-09-11T00:04:11.867210+0000","last_became_peered":"2024-09-11T00:04:11.867210+0000","last_unstale":"2024-09-11T00:05:19.841343+0000","last_undegraded":"2024-09-11T00:05:19.841343+0000","last_fullsized":"2024-09-11T00:05:19.841343+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_clean_scrub_stamp":"2024-09-11T00:04:10.845136+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:04:15.439956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T00:05:19.841582+0000","last_change":"2024-09-11T00:04:03.834755+0000","last_active":"2024-09-11T00:05:19.841582+0000","last_peered":"2024-09-11T00:05:19.841582+0000","last_clean":"2024-09-11T00:05:19.841582+0000","last_became_active":"2024-09-11T00:04:03.833955+0000","last_became_peered":"2024-09-11T00:04:03.833955+0000","last_unstale":"2024-09-11T00:05:19.841582+0000","last_undegraded":"2024-09-11T00:05:19.841582+0000","last_fullsized":"2024-09-11T00:05:19.841582+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:47:13.863481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"81'6","reported_seq":101,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T00:05:43.340608+0000","last_change":"2024-09-11T00:04:02.589042+0000","last_active":"2024-09-11T00:05:43.340608+0000","last_peered":"2024-09-11T00:05:43.340608+0000","last_clean":"2024-09-11T00:05:43.340608+0000","last_became_active":"2024-09-11T00:04:02.588611+0000","last_became_peered":"2024-09-11T00:04:02.588611+0000","last_unstale":"2024-09-11T00:05:43.340608+0000","last_undegraded":"2024-09-11T00:05:43.340608+0000","last_fullsized":"2024-09-11T00:05:43.340608+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_clean_scrub_stamp":"2024-09-11T00:04:00.794959+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:41:08.257605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1561,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":58,"num_read_kb":3,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T00:05:19.837208+0000","last_change":"2024-09-11T00:04:11.860715+0000","last_active":"2024-09-11T00:05:19.837208+0000","last_peered":"2024-09-11T00:05:19.837208+0000","last_clean":"2024-09-11T00:05:19.837208+0000","last_became_active":"2024-09-11T00:04:11.860094+0000","last_became_peered":"2024-09-11T00:04:11.860094+0000","last_unstale":"2024-09-11T00:05:19.837208+0000","last_undegraded":"2024-09-11T00:05:19.837208+0000","last_fullsized":"2024-09-11T00:05:19.837208+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_clean_scrub_stamp":"2024-09-11T00:04:10.845136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:25:55.279079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T00:05:19.841217+0000","last_change":"2024-09-11T00:04:03.822665+0000","last_active":"2024-09-11T00:05:19.841217+0000","last_peered":"2024-09-11T00:05:19.841217+0000","last_clean":"2024-09-11T00:05:19.841217+0000","last_became_active":"2024-09-11T00:04:03.822333+0000","last_became_peered":"2024-09-11T00:04:03.822333+0000","last_unstale":"2024-09-11T00:05:19.841217+0000","last_undegraded":"2024-09-11T00:05:19.841217+0000","last_fullsized":"2024-09-11T00:05:19.841217+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:02:07.465335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"81'3","reported_seq":41,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T00:05:19.844583+0000","last_change":"2024-09-11T00:04:01.817244+0000","last_active":"2024-09-11T00:05:19.844583+0000","last_peered":"2024-09-11T00:05:19.844583+0000","last_clean":"2024-09-11T00:05:19.844583+0000","last_became_active":"2024-09-11T00:04:01.816846+0000","last_became_peered":"2024-09-11T00:04:01.816846+0000","last_unstale":"2024-09-11T00:05:19.844583+0000","last_undegraded":"2024-09-11T00:05:19.844583+0000","last_fullsized":"2024-09-11T00:05:19.844583+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_clean_scrub_stamp":"2024-09-11T00:04:00.794959+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:09:46.232332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T00:05:19.839674+0000","last_change":"2024-09-11T00:04:11.872439+0000","last_active":"2024-09-11T00:05:19.839674+0000","last_peered":"2024-09-11T00:05:19.839674+0000","last_clean":"2024-09-11T00:05:19.839674+0000","last_became_active":"2024-09-11T00:04:11.872064+0000","last_became_peered":"2024-09-11T00:04:11.872064+0000","last_unstale":"2024-09-11T00:05:19.839674+0000","last_undegraded":"2024-09-11T00:05:19.839674+0000","last_fullsized":"2024-09-11T00:05:19.839674+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_clean_scrub_stamp":"2024-09-11T00:04:10.845136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:15:47.902247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T00:05:19.841533+0000","last_change":"2024-09-11T00:04:03.829600+0000","last_active":"2024-09-11T00:05:19.841533+0000","last_peered":"2024-09-11T00:05:19.841533+0000","last_clean":"2024-09-11T00:05:19.841533+0000","last_became_active":"2024-09-11T00:04:03.829118+0000","last_became_peered":"2024-09-11T00:04:03.829118+0000","last_unstale":"2024-09-11T00:05:19.841533+0000","last_undegraded":"2024-09-11T00:05:19.841533+0000","last_fullsized":"2024-09-11T00:05:19.841533+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:41:07.069816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"81'5","reported_seq":55,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T00:05:19.989496+0000","last_change":"2024-09-11T00:05:19.989379+0000","last_active":"2024-09-11T00:05:19.989496+0000","last_peered":"2024-09-11T00:05:19.989496+0000","last_clean":"2024-09-11T00:05:19.989496+0000","last_became_active":"2024-09-11T00:05:19.984918+0000","last_became_peered":"2024-09-11T00:05:19.984918+0000","last_unstale":"2024-09-11T00:05:19.989496+0000","last_undegraded":"2024-09-11T00:05:19.989496+0000","last_fullsized":"2024-09-11T00:05:19.989496+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_clean_scrub_stamp":"2024-09-11T00:04:00.794959+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:27:52.063256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4],"acting":[6,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T00:05:19.843119+0000","last_change":"2024-09-11T00:04:11.863812+0000","last_active":"2024-09-11T00:05:19.843119+0000","last_peered":"2024-09-11T00:05:19.843119+0000","last_clean":"2024-09-11T00:05:19.843119+0000","last_became_active":"2024-09-11T00:04:11.863412+0000","last_became_peered":"2024-09-11T00:04:11.863412+0000","last_unstale":"2024-09-11T00:05:19.843119+0000","last_undegraded":"2024-09-11T00:05:19.843119+0000","last_fullsized":"2024-09-11T00:05:19.843119+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_clean_scrub_stamp":"2024-09-11T00:04:10.845136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:12:04.140034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":39,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T00:05:19.985326+0000","last_change":"2024-09-11T00:05:19.985248+0000","last_active":"2024-09-11T00:05:19.985326+0000","last_peered":"2024-09-11T00:05:19.985326+0000","last_clean":"2024-09-11T00:05:19.985326+0000","last_became_active":"2024-09-11T00:05:19.984921+0000","last_became_peered":"2024-09-11T00:05:19.984921+0000","last_unstale":"2024-09-11T00:05:19.985326+0000","last_undegraded":"2024-09-11T00:05:19.985326+0000","last_fullsized":"2024-09-11T00:05:19.985326+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:16:44.370849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,4],"acting":[11,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"81'2","reported_seq":40,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T00:05:19.843607+0000","last_change":"2024-09-11T00:04:01.814191+0000","last_active":"2024-09-11T00:05:19.843607+0000","last_peered":"2024-09-11T00:05:19.843607+0000","last_clean":"2024-09-11T00:05:19.843607+0000","last_became_active":"2024-09-11T00:04:01.813778+0000","last_became_peered":"2024-09-11T00:04:01.813778+0000","last_unstale":"2024-09-11T00:05:19.843607+0000","last_undegraded":"2024-09-11T00:05:19.843607+0000","last_fullsized":"2024-09-11T00:05:19.843607+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_clean_scrub_stamp":"2024-09-11T00:04:00.794959+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:01:49.044436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T00:05:19.843398+0000","last_change":"2024-09-11T00:04:11.870281+0000","last_active":"2024-09-11T00:05:19.843398+0000","last_peered":"2024-09-11T00:05:19.843398+0000","last_clean":"2024-09-11T00:05:19.843398+0000","last_became_active":"2024-09-11T00:04:11.869738+0000","last_became_peered":"2024-09-11T00:04:11.869738+0000","last_unstale":"2024-09-11T00:05:19.843398+0000","last_undegraded":"2024-09-11T00:05:19.843398+0000","last_fullsized":"2024-09-11T00:05:19.843398+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_clean_scrub_stamp":"2024-09-11T00:04:10.845136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:04:08.380199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T00:05:19.843745+0000","last_change":"2024-09-11T00:04:03.835240+0000","last_active":"2024-09-11T00:05:19.843745+0000","last_peered":"2024-09-11T00:05:19.843745+0000","last_clean":"2024-09-11T00:05:19.843745+0000","last_became_active":"2024-09-11T00:04:03.834910+0000","last_became_peered":"2024-09-11T00:04:03.834910+0000","last_unstale":"2024-09-11T00:05:19.843745+0000","last_undegraded":"2024-09-11T00:05:19.843745+0000","last_fullsized":"2024-09-11T00:05:19.843745+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:30:33.996409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.10","version":"81'17","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T00:05:19.856811+0000","last_change":"2024-09-11T00:05:19.856704+0000","last_active":"2024-09-11T00:05:19.856811+0000","last_peered":"2024-09-11T00:05:19.856811+0000","last_clean":"2024-09-11T00:05:19.856811+0000","last_became_active":"2024-09-11T00:05:19.843487+0000","last_became_peered":"2024-09-11T00:05:19.843487+0000","last_unstale":"2024-09-11T00:05:19.856811+0000","last_undegraded":"2024-09-11T00:05:19.856811+0000","last_fullsized":"2024-09-11T00:05:19.856811+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_clean_scrub_stamp":"2024-09-11T00:04:00.794959+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:03:51.375839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T00:05:19.836927+0000","last_change":"2024-09-11T00:04:11.874306+0000","last_active":"2024-09-11T00:05:19.836927+0000","last_peered":"2024-09-11T00:05:19.836927+0000","last_clean":"2024-09-11T00:05:19.836927+0000","last_became_active":"2024-09-11T00:04:11.873934+0000","last_became_peered":"2024-09-11T00:04:11.873934+0000","last_unstale":"2024-09-11T00:05:19.836927+0000","last_undegraded":"2024-09-11T00:05:19.836927+0000","last_fullsized":"2024-09-11T00:05:19.836927+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_clean_scrub_stamp":"2024-09-11T00:04:10.845136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:19:22.584367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T00:05:19.840426+0000","last_change":"2024-09-11T00:04:03.824219+0000","last_active":"2024-09-11T00:05:19.840426+0000","last_peered":"2024-09-11T00:05:19.840426+0000","last_clean":"2024-09-11T00:05:19.840426+0000","last_became_active":"2024-09-11T00:04:03.823877+0000","last_became_peered":"2024-09-11T00:04:03.823877+0000","last_unstale":"2024-09-11T00:05:19.840426+0000","last_undegraded":"2024-09-11T00:05:19.840426+0000","last_fullsized":"2024-09-11T00:05:19.840426+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:04:05.924113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"81'5","reported_seq":99,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T00:05:43.340032+0000","last_change":"2024-09-11T00:04:01.818055+0000","last_active":"2024-09-11T00:05:43.340032+0000","last_peered":"2024-09-11T00:05:43.340032+0000","last_clean":"2024-09-11T00:05:43.340032+0000","last_became_active":"2024-09-11T00:04:01.817483+0000","last_became_peered":"2024-09-11T00:04:01.817483+0000","last_unstale":"2024-09-11T00:05:43.340032+0000","last_undegraded":"2024-09-11T00:05:43.340032+0000","last_fullsized":"2024-09-11T00:05:43.340032+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_clean_scrub_stamp":"2024-09-11T00:04:00.794959+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:16:55.218673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":58,"num_read_kb":3,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T00:05:19.840396+0000","last_change":"2024-09-11T00:04:11.875918+0000","last_active":"2024-09-11T00:05:19.840396+0000","last_peered":"2024-09-11T00:05:19.840396+0000","last_clean":"2024-09-11T00:05:19.840396+0000","last_became_active":"2024-09-11T00:04:11.875549+0000","last_became_peered":"2024-09-11T00:04:11.875549+0000","last_unstale":"2024-09-11T00:05:19.840396+0000","last_undegraded":"2024-09-11T00:05:19.840396+0000","last_fullsized":"2024-09-11T00:05:19.840396+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_clean_scrub_stamp":"2024-09-11T00:04:10.845136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:16:48.685899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T00:05:19.841133+0000","last_change":"2024-09-11T00:04:03.827671+0000","last_active":"2024-09-11T00:05:19.841133+0000","last_peered":"2024-09-11T00:05:19.841133+0000","last_clean":"2024-09-11T00:05:19.841133+0000","last_became_active":"2024-09-11T00:04:03.827341+0000","last_became_peered":"2024-09-11T00:04:03.827341+0000","last_unstale":"2024-09-11T00:05:19.841133+0000","last_undegraded":"2024-09-11T00:05:19.841133+0000","last_fullsized":"2024-09-11T00:05:19.841133+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:56:16.614035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"81'5","reported_seq":43,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T00:05:19.844833+0000","last_change":"2024-09-11T00:04:01.820632+0000","last_active":"2024-09-11T00:05:19.844833+0000","last_peered":"2024-09-11T00:05:19.844833+0000","last_clean":"2024-09-11T00:05:19.844833+0000","last_became_active":"2024-09-11T00:04:01.820273+0000","last_became_peered":"2024-09-11T00:04:01.820273+0000","last_unstale":"2024-09-11T00:05:19.844833+0000","last_undegraded":"2024-09-11T00:05:19.844833+0000","last_fullsized":"2024-09-11T00:05:19.844833+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_clean_scrub_stamp":"2024-09-11T00:04:00.794959+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:34:53.444515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T00:05:19.838915+0000","last_change":"2024-09-11T00:04:11.881197+0000","last_active":"2024-09-11T00:05:19.838915+0000","last_peered":"2024-09-11T00:05:19.838915+0000","last_clean":"2024-09-11T00:05:19.838915+0000","last_became_active":"2024-09-11T00:04:11.880763+0000","last_became_peered":"2024-09-11T00:04:11.880763+0000","last_unstale":"2024-09-11T00:05:19.838915+0000","last_undegraded":"2024-09-11T00:05:19.838915+0000","last_fullsized":"2024-09-11T00:05:19.838915+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_clean_scrub_stamp":"2024-09-11T00:04:10.845136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:14:01.616095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T00:05:19.836503+0000","last_change":"2024-09-11T00:04:03.833470+0000","last_active":"2024-09-11T00:05:19.836503+0000","last_peered":"2024-09-11T00:05:19.836503+0000","last_clean":"2024-09-11T00:05:19.836503+0000","last_became_active":"2024-09-11T00:04:03.833120+0000","last_became_peered":"2024-09-11T00:04:03.833120+0000","last_unstale":"2024-09-11T00:05:19.836503+0000","last_undegraded":"2024-09-11T00:05:19.836503+0000","last_fullsized":"2024-09-11T00:05:19.836503+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:29:53.210230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"81'9","reported_seq":48,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T00:05:19.837431+0000","last_change":"2024-09-11T00:04:01.818081+0000","last_active":"2024-09-11T00:05:19.837431+0000","last_peered":"2024-09-11T00:05:19.837431+0000","last_clean":"2024-09-11T00:05:19.837431+0000","last_became_active":"2024-09-11T00:04:01.817716+0000","last_became_peered":"2024-09-11T00:04:01.817716+0000","last_unstale":"2024-09-11T00:05:19.837431+0000","last_undegraded":"2024-09-11T00:05:19.837431+0000","last_fullsized":"2024-09-11T00:05:19.837431+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_clean_scrub_stamp":"2024-09-11T00:04:00.794959+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:01:32.736319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T00:05:19.843999+0000","last_change":"2024-09-11T00:04:11.870182+0000","last_active":"2024-09-11T00:05:19.843999+0000","last_peered":"2024-09-11T00:05:19.843999+0000","last_clean":"2024-09-11T00:05:19.843999+0000","last_became_active":"2024-09-11T00:04:11.869744+0000","last_became_peered":"2024-09-11T00:04:11.869744+0000","last_unstale":"2024-09-11T00:05:19.843999+0000","last_undegraded":"2024-09-11T00:05:19.843999+0000","last_fullsized":"2024-09-11T00:05:19.843999+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_clean_scrub_stamp":"2024-09-11T00:04:10.845136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:50:24.070404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T00:05:19.842764+0000","last_change":"2024-09-11T00:04:03.828246+0000","last_active":"2024-09-11T00:05:19.842764+0000","last_peered":"2024-09-11T00:05:19.842764+0000","last_clean":"2024-09-11T00:05:19.842764+0000","last_became_active":"2024-09-11T00:04:03.827217+0000","last_became_peered":"2024-09-11T00:04:03.827217+0000","last_unstale":"2024-09-11T00:05:19.842764+0000","last_undegraded":"2024-09-11T00:05:19.842764+0000","last_fullsized":"2024-09-11T00:05:19.842764+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:51:46.018759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"83'36","reported_seq":195,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T00:05:45.342966+0000","last_change":"2024-09-11T00:04:01.819553+0000","last_active":"2024-09-11T00:05:45.342966+0000","last_peered":"2024-09-11T00:05:45.342966+0000","last_clean":"2024-09-11T00:05:45.342966+0000","last_became_active":"2024-09-11T00:04:01.819213+0000","last_became_peered":"2024-09-11T00:04:01.819213+0000","last_unstale":"2024-09-11T00:05:45.342966+0000","last_undegraded":"2024-09-11T00:05:45.342966+0000","last_fullsized":"2024-09-11T00:05:45.342966+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_clean_scrub_stamp":"2024-09-11T00:04:00.794959+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:44:11.039982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":64618,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":122,"num_read_kb":124,"num_write":38,"num_write_kb":79,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T00:05:19.839925+0000","last_change":"2024-09-11T00:04:11.871730+0000","last_active":"2024-09-11T00:05:19.839925+0000","last_peered":"2024-09-11T00:05:19.839925+0000","last_clean":"2024-09-11T00:05:19.839925+0000","last_became_active":"2024-09-11T00:04:11.870659+0000","last_became_peered":"2024-09-11T00:04:11.870659+0000","last_unstale":"2024-09-11T00:05:19.839925+0000","last_undegraded":"2024-09-11T00:05:19.839925+0000","last_fullsized":"2024-09-11T00:05:19.839925+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_clean_scrub_stamp":"2024-09-11T00:04:10.845136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:48:41.835602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T00:05:19.837548+0000","last_change":"2024-09-11T00:04:03.838763+0000","last_active":"2024-09-11T00:05:19.837548+0000","last_peered":"2024-09-11T00:05:19.837548+0000","last_clean":"2024-09-11T00:05:19.837548+0000","last_became_active":"2024-09-11T00:04:03.838409+0000","last_became_peered":"2024-09-11T00:04:03.838409+0000","last_unstale":"2024-09-11T00:05:19.837548+0000","last_undegraded":"2024-09-11T00:05:19.837548+0000","last_fullsized":"2024-09-11T00:05:19.837548+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:57:38.647351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"81'5","reported_seq":43,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T00:05:19.844476+0000","last_change":"2024-09-11T00:04:01.943027+0000","last_active":"2024-09-11T00:05:19.844476+0000","last_peered":"2024-09-11T00:05:19.844476+0000","last_clean":"2024-09-11T00:05:19.844476+0000","last_became_active":"2024-09-11T00:04:01.942558+0000","last_became_peered":"2024-09-11T00:04:01.942558+0000","last_unstale":"2024-09-11T00:05:19.844476+0000","last_undegraded":"2024-09-11T00:05:19.844476+0000","last_fullsized":"2024-09-11T00:05:19.844476+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_clean_scrub_stamp":"2024-09-11T00:04:00.794959+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:48:03.776476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T00:05:19.984081+0000","last_change":"2024-09-11T00:04:11.868069+0000","last_active":"2024-09-11T00:05:19.984081+0000","last_peered":"2024-09-11T00:05:19.984081+0000","last_clean":"2024-09-11T00:05:19.984081+0000","last_became_active":"2024-09-11T00:04:11.867544+0000","last_became_peered":"2024-09-11T00:04:11.867544+0000","last_unstale":"2024-09-11T00:05:19.984081+0000","last_undegraded":"2024-09-11T00:05:19.984081+0000","last_fullsized":"2024-09-11T00:05:19.984081+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_clean_scrub_stamp":"2024-09-11T00:04:10.845136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:09:20.455807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T00:05:19.840832+0000","last_change":"2024-09-11T00:04:03.836735+0000","last_active":"2024-09-11T00:05:19.840832+0000","last_peered":"2024-09-11T00:05:19.840832+0000","last_clean":"2024-09-11T00:05:19.840832+0000","last_became_active":"2024-09-11T00:04:03.833299+0000","last_became_peered":"2024-09-11T00:04:03.833299+0000","last_unstale":"2024-09-11T00:05:19.840832+0000","last_undegraded":"2024-09-11T00:05:19.840832+0000","last_fullsized":"2024-09-11T00:05:19.840832+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:32:15.924058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"81'4","reported_seq":41,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T00:05:19.841672+0000","last_change":"2024-09-11T00:04:01.823777+0000","last_active":"2024-09-11T00:05:19.841672+0000","last_peered":"2024-09-11T00:05:19.841672+0000","last_clean":"2024-09-11T00:05:19.841672+0000","last_became_active":"2024-09-11T00:04:01.823368+0000","last_became_peered":"2024-09-11T00:04:01.823368+0000","last_unstale":"2024-09-11T00:05:19.841672+0000","last_undegraded":"2024-09-11T00:05:19.841672+0000","last_fullsized":"2024-09-11T00:05:19.841672+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_clean_scrub_stamp":"2024-09-11T00:04:00.794959+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:05:12.584867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T00:05:19.839064+0000","last_change":"2024-09-11T00:04:11.868303+0000","last_active":"2024-09-11T00:05:19.839064+0000","last_peered":"2024-09-11T00:05:19.839064+0000","last_clean":"2024-09-11T00:05:19.839064+0000","last_became_active":"2024-09-11T00:04:11.867916+0000","last_became_peered":"2024-09-11T00:04:11.867916+0000","last_unstale":"2024-09-11T00:05:19.839064+0000","last_undegraded":"2024-09-11T00:05:19.839064+0000","last_fullsized":"2024-09-11T00:05:19.839064+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_clean_scrub_stamp":"2024-09-11T00:04:10.845136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:08:29.711763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T00:05:19.837642+0000","last_change":"2024-09-11T00:04:03.832712+0000","last_active":"2024-09-11T00:05:19.837642+0000","last_peered":"2024-09-11T00:05:19.837642+0000","last_clean":"2024-09-11T00:05:19.837642+0000","last_became_active":"2024-09-11T00:04:03.832295+0000","last_became_peered":"2024-09-11T00:04:03.832295+0000","last_unstale":"2024-09-11T00:05:19.837642+0000","last_undegraded":"2024-09-11T00:05:19.837642+0000","last_fullsized":"2024-09-11T00:05:19.837642+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:40:12.271930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"81'4","reported_seq":153,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T00:05:45.342929+0000","last_change":"2024-09-11T00:04:01.820891+0000","last_active":"2024-09-11T00:05:45.342929+0000","last_peered":"2024-09-11T00:05:45.342929+0000","last_clean":"2024-09-11T00:05:45.342929+0000","last_became_active":"2024-09-11T00:04:01.820528+0000","last_became_peered":"2024-09-11T00:04:01.820528+0000","last_unstale":"2024-09-11T00:05:45.342929+0000","last_undegraded":"2024-09-11T00:05:45.342929+0000","last_fullsized":"2024-09-11T00:05:45.342929+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_clean_scrub_stamp":"2024-09-11T00:04:00.794959+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:23:38.196618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":112,"num_read_kb":112,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T00:05:19.843062+0000","last_change":"2024-09-11T00:04:11.878864+0000","last_active":"2024-09-11T00:05:19.843062+0000","last_peered":"2024-09-11T00:05:19.843062+0000","last_clean":"2024-09-11T00:05:19.843062+0000","last_became_active":"2024-09-11T00:04:11.876701+0000","last_became_peered":"2024-09-11T00:04:11.876701+0000","last_unstale":"2024-09-11T00:05:19.843062+0000","last_undegraded":"2024-09-11T00:05:19.843062+0000","last_fullsized":"2024-09-11T00:05:19.843062+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_clean_scrub_stamp":"2024-09-11T00:04:10.845136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:26:05.650618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T00:05:19.840766+0000","last_change":"2024-09-11T00:04:11.868140+0000","last_active":"2024-09-11T00:05:19.840766+0000","last_peered":"2024-09-11T00:05:19.840766+0000","last_clean":"2024-09-11T00:05:19.840766+0000","last_became_active":"2024-09-11T00:04:11.866459+0000","last_became_peered":"2024-09-11T00:04:11.866459+0000","last_unstale":"2024-09-11T00:05:19.840766+0000","last_undegraded":"2024-09-11T00:05:19.840766+0000","last_fullsized":"2024-09-11T00:05:19.840766+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_clean_scrub_stamp":"2024-09-11T00:04:10.845136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:53:25.106039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T00:05:19.839391+0000","last_change":"2024-09-11T00:04:03.829839+0000","last_active":"2024-09-11T00:05:19.839391+0000","last_peered":"2024-09-11T00:05:19.839391+0000","last_clean":"2024-09-11T00:05:19.839391+0000","last_became_active":"2024-09-11T00:04:03.829553+0000","last_became_peered":"2024-09-11T00:04:03.829553+0000","last_unstale":"2024-09-11T00:05:19.839391+0000","last_undegraded":"2024-09-11T00:05:19.839391+0000","last_fullsized":"2024-09-11T00:05:19.839391+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:45:58.561389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"81'3","reported_seq":41,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T00:05:19.843660+0000","last_change":"2024-09-11T00:04:01.820204+0000","last_active":"2024-09-11T00:05:19.843660+0000","last_peered":"2024-09-11T00:05:19.843660+0000","last_clean":"2024-09-11T00:05:19.843660+0000","last_became_active":"2024-09-11T00:04:01.819797+0000","last_became_peered":"2024-09-11T00:04:01.819797+0000","last_unstale":"2024-09-11T00:05:19.843660+0000","last_undegraded":"2024-09-11T00:05:19.843660+0000","last_fullsized":"2024-09-11T00:05:19.843660+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_clean_scrub_stamp":"2024-09-11T00:04:00.794959+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:56:10.530023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T00:05:19.838877+0000","last_change":"2024-09-11T00:04:11.879430+0000","last_active":"2024-09-11T00:05:19.838877+0000","last_peered":"2024-09-11T00:05:19.838877+0000","last_clean":"2024-09-11T00:05:19.838877+0000","last_became_active":"2024-09-11T00:04:11.879010+0000","last_became_peered":"2024-09-11T00:04:11.879010+0000","last_unstale":"2024-09-11T00:05:19.838877+0000","last_undegraded":"2024-09-11T00:05:19.838877+0000","last_fullsized":"2024-09-11T00:05:19.838877+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_clean_scrub_stamp":"2024-09-11T00:04:10.845136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:57.737125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T00:05:19.842352+0000","last_change":"2024-09-11T00:04:03.843918+0000","last_active":"2024-09-11T00:05:19.842352+0000","last_peered":"2024-09-11T00:05:19.842352+0000","last_clean":"2024-09-11T00:05:19.842352+0000","last_became_active":"2024-09-11T00:04:03.843591+0000","last_became_peered":"2024-09-11T00:04:03.843591+0000","last_unstale":"2024-09-11T00:05:19.842352+0000","last_undegraded":"2024-09-11T00:05:19.842352+0000","last_fullsized":"2024-09-11T00:05:19.842352+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:26:41.228802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"81'3","reported_seq":40,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T00:05:19.836363+0000","last_change":"2024-09-11T00:04:01.813173+0000","last_active":"2024-09-11T00:05:19.836363+0000","last_peered":"2024-09-11T00:05:19.836363+0000","last_clean":"2024-09-11T00:05:19.836363+0000","last_became_active":"2024-09-11T00:04:01.811799+0000","last_became_peered":"2024-09-11T00:04:01.811799+0000","last_unstale":"2024-09-11T00:05:19.836363+0000","last_undegraded":"2024-09-11T00:05:19.836363+0000","last_fullsized":"2024-09-11T00:05:19.836363+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_clean_scrub_stamp":"2024-09-11T00:04:00.794959+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:16:47.304371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"81'7","reported_seq":44,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T00:05:19.983032+0000","last_change":"2024-09-11T00:04:01.818649+0000","last_active":"2024-09-11T00:05:19.983032+0000","last_peered":"2024-09-11T00:05:19.983032+0000","last_clean":"2024-09-11T00:05:19.983032+0000","last_became_active":"2024-09-11T00:04:01.818287+0000","last_became_peered":"2024-09-11T00:04:01.818287+0000","last_unstale":"2024-09-11T00:05:19.983032+0000","last_undegraded":"2024-09-11T00:05:19.983032+0000","last_fullsized":"2024-09-11T00:05:19.983032+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_clean_scrub_stamp":"2024-09-11T00:04:00.794959+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:42:00.764648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T00:05:19.844591+0000","last_change":"2024-09-11T00:04:03.832829+0000","last_active":"2024-09-11T00:05:19.844591+0000","last_peered":"2024-09-11T00:05:19.844591+0000","last_clean":"2024-09-11T00:05:19.844591+0000","last_became_active":"2024-09-11T00:04:03.832514+0000","last_became_peered":"2024-09-11T00:04:03.832514+0000","last_unstale":"2024-09-11T00:05:19.844591+0000","last_undegraded":"2024-09-11T00:05:19.844591+0000","last_fullsized":"2024-09-11T00:05:19.844591+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:36.643727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T00:05:19.837064+0000","last_change":"2024-09-11T00:04:11.877596+0000","last_active":"2024-09-11T00:05:19.837064+0000","last_peered":"2024-09-11T00:05:19.837064+0000","last_clean":"2024-09-11T00:05:19.837064+0000","last_became_active":"2024-09-11T00:04:11.877113+0000","last_became_peered":"2024-09-11T00:04:11.877113+0000","last_unstale":"2024-09-11T00:05:19.837064+0000","last_undegraded":"2024-09-11T00:05:19.837064+0000","last_fullsized":"2024-09-11T00:05:19.837064+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_clean_scrub_stamp":"2024-09-11T00:04:10.845136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:47:11.759687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"81'5","reported_seq":100,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T00:05:44.342213+0000","last_change":"2024-09-11T00:04:01.821519+0000","last_active":"2024-09-11T00:05:44.342213+0000","last_peered":"2024-09-11T00:05:44.342213+0000","last_clean":"2024-09-11T00:05:44.342213+0000","last_became_active":"2024-09-11T00:04:01.821207+0000","last_became_peered":"2024-09-11T00:04:01.821207+0000","last_unstale":"2024-09-11T00:05:44.342213+0000","last_undegraded":"2024-09-11T00:05:44.342213+0000","last_fullsized":"2024-09-11T00:05:44.342213+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_clean_scrub_stamp":"2024-09-11T00:04:00.794959+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:16:34.435040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":58,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T00:05:19.842126+0000","last_change":"2024-09-11T00:04:03.832691+0000","last_active":"2024-09-11T00:05:19.842126+0000","last_peered":"2024-09-11T00:05:19.842126+0000","last_clean":"2024-09-11T00:05:19.842126+0000","last_became_active":"2024-09-11T00:04:03.832250+0000","last_became_peered":"2024-09-11T00:04:03.832250+0000","last_unstale":"2024-09-11T00:05:19.842126+0000","last_undegraded":"2024-09-11T00:05:19.842126+0000","last_fullsized":"2024-09-11T00:05:19.842126+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:54.279820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T00:05:19.837089+0000","last_change":"2024-09-11T00:04:11.864995+0000","last_active":"2024-09-11T00:05:19.837089+0000","last_peered":"2024-09-11T00:05:19.837089+0000","last_clean":"2024-09-11T00:05:19.837089+0000","last_became_active":"2024-09-11T00:04:11.864501+0000","last_became_peered":"2024-09-11T00:04:11.864501+0000","last_unstale":"2024-09-11T00:05:19.837089+0000","last_undegraded":"2024-09-11T00:05:19.837089+0000","last_fullsized":"2024-09-11T00:05:19.837089+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_clean_scrub_stamp":"2024-09-11T00:04:10.845136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:08:08.637714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T00:05:19.839813+0000","last_change":"2024-09-11T00:04:03.829310+0000","last_active":"2024-09-11T00:05:19.839813+0000","last_peered":"2024-09-11T00:05:19.839813+0000","last_clean":"2024-09-11T00:05:19.839813+0000","last_became_active":"2024-09-11T00:04:03.828937+0000","last_became_peered":"2024-09-11T00:04:03.828937+0000","last_unstale":"2024-09-11T00:05:19.839813+0000","last_undegraded":"2024-09-11T00:05:19.839813+0000","last_fullsized":"2024-09-11T00:05:19.839813+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:32:58.263067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T00:05:19.837214+0000","last_change":"2024-09-11T00:04:03.828055+0000","last_active":"2024-09-11T00:05:19.837214+0000","last_peered":"2024-09-11T00:05:19.837214+0000","last_clean":"2024-09-11T00:05:19.837214+0000","last_became_active":"2024-09-11T00:04:03.827382+0000","last_became_peered":"2024-09-11T00:04:03.827382+0000","last_unstale":"2024-09-11T00:05:19.837214+0000","last_undegraded":"2024-09-11T00:05:19.837214+0000","last_fullsized":"2024-09-11T00:05:19.837214+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:02:47.107689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T00:05:19.836865+0000","last_change":"2024-09-11T00:04:03.843549+0000","last_active":"2024-09-11T00:05:19.836865+0000","last_peered":"2024-09-11T00:05:19.836865+0000","last_clean":"2024-09-11T00:05:19.836865+0000","last_became_active":"2024-09-11T00:04:03.843149+0000","last_became_peered":"2024-09-11T00:04:03.843149+0000","last_unstale":"2024-09-11T00:05:19.836865+0000","last_undegraded":"2024-09-11T00:05:19.836865+0000","last_fullsized":"2024-09-11T00:05:19.836865+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:47.288319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":39,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T00:05:19.844779+0000","last_change":"2024-09-11T00:05:19.844682+0000","last_active":"2024-09-11T00:05:19.844779+0000","last_peered":"2024-09-11T00:05:19.844779+0000","last_clean":"2024-09-11T00:05:19.844779+0000","last_became_active":"2024-09-11T00:05:19.844269+0000","last_became_peered":"2024-09-11T00:05:19.844269+0000","last_unstale":"2024-09-11T00:05:19.844779+0000","last_undegraded":"2024-09-11T00:05:19.844779+0000","last_fullsized":"2024-09-11T00:05:19.844779+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:47:22.001597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,7],"acting":[0,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T00:05:19.836704+0000","last_change":"2024-09-11T00:04:03.835068+0000","last_active":"2024-09-11T00:05:19.836704+0000","last_peered":"2024-09-11T00:05:19.836704+0000","last_clean":"2024-09-11T00:05:19.836704+0000","last_became_active":"2024-09-11T00:04:03.833166+0000","last_became_peered":"2024-09-11T00:04:03.833166+0000","last_unstale":"2024-09-11T00:05:19.836704+0000","last_undegraded":"2024-09-11T00:05:19.836704+0000","last_fullsized":"2024-09-11T00:05:19.836704+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:07:06.357597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T00:05:19.839328+0000","last_change":"2024-09-11T00:04:03.827310+0000","last_active":"2024-09-11T00:05:19.839328+0000","last_peered":"2024-09-11T00:05:19.839328+0000","last_clean":"2024-09-11T00:05:19.839328+0000","last_became_active":"2024-09-11T00:04:03.826748+0000","last_became_peered":"2024-09-11T00:04:03.826748+0000","last_unstale":"2024-09-11T00:05:19.839328+0000","last_undegraded":"2024-09-11T00:05:19.839328+0000","last_fullsized":"2024-09-11T00:05:19.839328+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:53:15.656708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T00:05:19.837302+0000","last_change":"2024-09-11T00:04:03.835900+0000","last_active":"2024-09-11T00:05:19.837302+0000","last_peered":"2024-09-11T00:05:19.837302+0000","last_clean":"2024-09-11T00:05:19.837302+0000","last_became_active":"2024-09-11T00:04:03.835137+0000","last_became_peered":"2024-09-11T00:04:03.835137+0000","last_unstale":"2024-09-11T00:05:19.837302+0000","last_undegraded":"2024-09-11T00:05:19.837302+0000","last_fullsized":"2024-09-11T00:05:19.837302+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:23:31.184833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T00:05:19.845643+0000","last_change":"2024-09-11T00:04:03.833028+0000","last_active":"2024-09-11T00:05:19.845643+0000","last_peered":"2024-09-11T00:05:19.845643+0000","last_clean":"2024-09-11T00:05:19.845643+0000","last_became_active":"2024-09-11T00:04:03.832703+0000","last_became_peered":"2024-09-11T00:04:03.832703+0000","last_unstale":"2024-09-11T00:05:19.845643+0000","last_undegraded":"2024-09-11T00:05:19.845643+0000","last_fullsized":"2024-09-11T00:05:19.845643+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:38.115627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T00:05:19.843431+0000","last_change":"2024-09-11T00:04:03.832423+0000","last_active":"2024-09-11T00:05:19.843431+0000","last_peered":"2024-09-11T00:05:19.843431+0000","last_clean":"2024-09-11T00:05:19.843431+0000","last_became_active":"2024-09-11T00:04:03.832090+0000","last_became_peered":"2024-09-11T00:04:03.832090+0000","last_unstale":"2024-09-11T00:05:19.843431+0000","last_undegraded":"2024-09-11T00:05:19.843431+0000","last_fullsized":"2024-09-11T00:05:19.843431+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:01:27.258506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T00:05:19.843194+0000","last_change":"2024-09-11T00:04:03.842854+0000","last_active":"2024-09-11T00:05:19.843194+0000","last_peered":"2024-09-11T00:05:19.843194+0000","last_clean":"2024-09-11T00:05:19.843194+0000","last_became_active":"2024-09-11T00:04:03.838666+0000","last_became_peered":"2024-09-11T00:04:03.838666+0000","last_unstale":"2024-09-11T00:05:19.843194+0000","last_undegraded":"2024-09-11T00:05:19.843194+0000","last_fullsized":"2024-09-11T00:05:19.843194+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:45:48.575830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T00:05:19.837513+0000","last_change":"2024-09-11T00:04:03.834175+0000","last_active":"2024-09-11T00:05:19.837513+0000","last_peered":"2024-09-11T00:05:19.837513+0000","last_clean":"2024-09-11T00:05:19.837513+0000","last_became_active":"2024-09-11T00:04:03.833826+0000","last_became_peered":"2024-09-11T00:04:03.833826+0000","last_unstale":"2024-09-11T00:05:19.837513+0000","last_undegraded":"2024-09-11T00:05:19.837513+0000","last_fullsized":"2024-09-11T00:05:19.837513+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:45:57.323216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T00:05:19.836787+0000","last_change":"2024-09-11T00:04:03.836751+0000","last_active":"2024-09-11T00:05:19.836787+0000","last_peered":"2024-09-11T00:05:19.836787+0000","last_clean":"2024-09-11T00:05:19.836787+0000","last_became_active":"2024-09-11T00:04:03.834064+0000","last_became_peered":"2024-09-11T00:04:03.834064+0000","last_unstale":"2024-09-11T00:05:19.836787+0000","last_undegraded":"2024-09-11T00:05:19.836787+0000","last_fullsized":"2024-09-11T00:05:19.836787+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:06:23.059770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T00:05:19.838906+0000","last_change":"2024-09-11T00:04:03.830196+0000","last_active":"2024-09-11T00:05:19.838906+0000","last_peered":"2024-09-11T00:05:19.838906+0000","last_clean":"2024-09-11T00:05:19.838906+0000","last_became_active":"2024-09-11T00:04:03.829856+0000","last_became_peered":"2024-09-11T00:04:03.829856+0000","last_unstale":"2024-09-11T00:05:19.838906+0000","last_undegraded":"2024-09-11T00:05:19.838906+0000","last_fullsized":"2024-09-11T00:05:19.838906+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:34:47.489316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T00:05:19.837396+0000","last_change":"2024-09-11T00:04:03.835053+0000","last_active":"2024-09-11T00:05:19.837396+0000","last_peered":"2024-09-11T00:05:19.837396+0000","last_clean":"2024-09-11T00:05:19.837396+0000","last_became_active":"2024-09-11T00:04:03.834736+0000","last_became_peered":"2024-09-11T00:04:03.834736+0000","last_unstale":"2024-09-11T00:05:19.837396+0000","last_undegraded":"2024-09-11T00:05:19.837396+0000","last_fullsized":"2024-09-11T00:05:19.837396+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:53:40.253775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T00:05:19.841222+0000","last_change":"2024-09-11T00:04:03.834514+0000","last_active":"2024-09-11T00:05:19.841222+0000","last_peered":"2024-09-11T00:05:19.841222+0000","last_clean":"2024-09-11T00:05:19.841222+0000","last_became_active":"2024-09-11T00:04:03.834185+0000","last_became_peered":"2024-09-11T00:04:03.834185+0000","last_unstale":"2024-09-11T00:05:19.841222+0000","last_undegraded":"2024-09-11T00:05:19.841222+0000","last_fullsized":"2024-09-11T00:05:19.841222+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:33:52.949710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T00:05:19.839504+0000","last_change":"2024-09-11T00:04:03.840371+0000","last_active":"2024-09-11T00:05:19.839504+0000","last_peered":"2024-09-11T00:05:19.839504+0000","last_clean":"2024-09-11T00:05:19.839504+0000","last_became_active":"2024-09-11T00:04:03.839933+0000","last_became_peered":"2024-09-11T00:04:03.839933+0000","last_unstale":"2024-09-11T00:05:19.839504+0000","last_undegraded":"2024-09-11T00:05:19.839504+0000","last_fullsized":"2024-09-11T00:05:19.839504+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:24:53.890837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T00:05:19.837159+0000","last_change":"2024-09-11T00:04:03.838386+0000","last_active":"2024-09-11T00:05:19.837159+0000","last_peered":"2024-09-11T00:05:19.837159+0000","last_clean":"2024-09-11T00:05:19.837159+0000","last_became_active":"2024-09-11T00:04:03.837511+0000","last_became_peered":"2024-09-11T00:04:03.837511+0000","last_unstale":"2024-09-11T00:05:19.837159+0000","last_undegraded":"2024-09-11T00:05:19.837159+0000","last_fullsized":"2024-09-11T00:05:19.837159+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:27.399727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":39,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T00:05:19.848091+0000","last_change":"2024-09-11T00:05:19.848001+0000","last_active":"2024-09-11T00:05:19.848091+0000","last_peered":"2024-09-11T00:05:19.848091+0000","last_clean":"2024-09-11T00:05:19.848091+0000","last_became_active":"2024-09-11T00:05:19.847175+0000","last_became_peered":"2024-09-11T00:05:19.847175+0000","last_unstale":"2024-09-11T00:05:19.848091+0000","last_undegraded":"2024-09-11T00:05:19.848091+0000","last_fullsized":"2024-09-11T00:05:19.848091+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:06:13.765740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,2],"acting":[0,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T00:05:19.840548+0000","last_change":"2024-09-11T00:04:03.832618+0000","last_active":"2024-09-11T00:05:19.840548+0000","last_peered":"2024-09-11T00:05:19.840548+0000","last_clean":"2024-09-11T00:05:19.840548+0000","last_became_active":"2024-09-11T00:04:03.831821+0000","last_became_peered":"2024-09-11T00:04:03.831821+0000","last_unstale":"2024-09-11T00:05:19.840548+0000","last_undegraded":"2024-09-11T00:05:19.840548+0000","last_fullsized":"2024-09-11T00:05:19.840548+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:40:40.402422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T00:05:19.836719+0000","last_change":"2024-09-11T00:04:03.825348+0000","last_active":"2024-09-11T00:05:19.836719+0000","last_peered":"2024-09-11T00:05:19.836719+0000","last_clean":"2024-09-11T00:05:19.836719+0000","last_became_active":"2024-09-11T00:04:03.824977+0000","last_became_peered":"2024-09-11T00:04:03.824977+0000","last_unstale":"2024-09-11T00:05:19.836719+0000","last_undegraded":"2024-09-11T00:05:19.836719+0000","last_fullsized":"2024-09-11T00:05:19.836719+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:19:53.412969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T00:05:19.983752+0000","last_change":"2024-09-11T00:05:19.983677+0000","last_active":"2024-09-11T00:05:19.983752+0000","last_peered":"2024-09-11T00:05:19.983752+0000","last_clean":"2024-09-11T00:05:19.983752+0000","last_became_active":"2024-09-11T00:05:19.983404+0000","last_became_peered":"2024-09-11T00:05:19.983404+0000","last_unstale":"2024-09-11T00:05:19.983752+0000","last_undegraded":"2024-09-11T00:05:19.983752+0000","last_fullsized":"2024-09-11T00:05:19.983752+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:58:11.624190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,10,8],"acting":[9,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T00:05:19.841363+0000","last_change":"2024-09-11T00:04:03.840068+0000","last_active":"2024-09-11T00:05:19.841363+0000","last_peered":"2024-09-11T00:05:19.841363+0000","last_clean":"2024-09-11T00:05:19.841363+0000","last_became_active":"2024-09-11T00:04:03.835185+0000","last_became_peered":"2024-09-11T00:04:03.835185+0000","last_unstale":"2024-09-11T00:05:19.841363+0000","last_undegraded":"2024-09-11T00:05:19.841363+0000","last_fullsized":"2024-09-11T00:05:19.841363+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:19:10.699038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T00:05:19.983754+0000","last_change":"2024-09-11T00:04:03.828576+0000","last_active":"2024-09-11T00:05:19.983754+0000","last_peered":"2024-09-11T00:05:19.983754+0000","last_clean":"2024-09-11T00:05:19.983754+0000","last_became_active":"2024-09-11T00:04:03.828255+0000","last_became_peered":"2024-09-11T00:04:03.828255+0000","last_unstale":"2024-09-11T00:05:19.983754+0000","last_undegraded":"2024-09-11T00:05:19.983754+0000","last_fullsized":"2024-09-11T00:05:19.983754+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:29:54.164145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T00:05:19.837616+0000","last_change":"2024-09-11T00:04:03.841452+0000","last_active":"2024-09-11T00:05:19.837616+0000","last_peered":"2024-09-11T00:05:19.837616+0000","last_clean":"2024-09-11T00:05:19.837616+0000","last_became_active":"2024-09-11T00:04:03.840911+0000","last_became_peered":"2024-09-11T00:04:03.840911+0000","last_unstale":"2024-09-11T00:05:19.837616+0000","last_undegraded":"2024-09-11T00:05:19.837616+0000","last_fullsized":"2024-09-11T00:05:19.837616+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:56:40.121922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T00:05:19.842259+0000","last_change":"2024-09-11T00:04:03.842009+0000","last_active":"2024-09-11T00:05:19.842259+0000","last_peered":"2024-09-11T00:05:19.842259+0000","last_clean":"2024-09-11T00:05:19.842259+0000","last_became_active":"2024-09-11T00:04:03.841625+0000","last_became_peered":"2024-09-11T00:04:03.841625+0000","last_unstale":"2024-09-11T00:05:19.842259+0000","last_undegraded":"2024-09-11T00:05:19.842259+0000","last_fullsized":"2024-09-11T00:05:19.842259+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:15:11.855308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":39,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T00:05:19.845975+0000","last_change":"2024-09-11T00:05:19.845891+0000","last_active":"2024-09-11T00:05:19.845975+0000","last_peered":"2024-09-11T00:05:19.845975+0000","last_clean":"2024-09-11T00:05:19.845975+0000","last_became_active":"2024-09-11T00:05:19.845547+0000","last_became_peered":"2024-09-11T00:05:19.845547+0000","last_unstale":"2024-09-11T00:05:19.845975+0000","last_undegraded":"2024-09-11T00:05:19.845975+0000","last_fullsized":"2024-09-11T00:05:19.845975+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:02:20.178667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T00:05:19.982904+0000","last_change":"2024-09-11T00:04:03.834540+0000","last_active":"2024-09-11T00:05:19.982904+0000","last_peered":"2024-09-11T00:05:19.982904+0000","last_clean":"2024-09-11T00:05:19.982904+0000","last_became_active":"2024-09-11T00:04:03.834230+0000","last_became_peered":"2024-09-11T00:04:03.834230+0000","last_unstale":"2024-09-11T00:05:19.982904+0000","last_undegraded":"2024-09-11T00:05:19.982904+0000","last_fullsized":"2024-09-11T00:05:19.982904+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:14:26.125003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T00:05:19.842391+0000","last_change":"2024-09-11T00:04:03.836177+0000","last_active":"2024-09-11T00:05:19.842391+0000","last_peered":"2024-09-11T00:05:19.842391+0000","last_clean":"2024-09-11T00:05:19.842391+0000","last_became_active":"2024-09-11T00:04:03.835848+0000","last_became_peered":"2024-09-11T00:04:03.835848+0000","last_unstale":"2024-09-11T00:05:19.842391+0000","last_undegraded":"2024-09-11T00:05:19.842391+0000","last_fullsized":"2024-09-11T00:05:19.842391+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:57:53.854738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T00:05:19.984468+0000","last_change":"2024-09-11T00:04:03.837614+0000","last_active":"2024-09-11T00:05:19.984468+0000","last_peered":"2024-09-11T00:05:19.984468+0000","last_clean":"2024-09-11T00:05:19.984468+0000","last_became_active":"2024-09-11T00:04:03.837164+0000","last_became_peered":"2024-09-11T00:04:03.837164+0000","last_unstale":"2024-09-11T00:05:19.984468+0000","last_undegraded":"2024-09-11T00:05:19.984468+0000","last_fullsized":"2024-09-11T00:05:19.984468+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:10:38.467648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T00:05:19.840634+0000","last_change":"2024-09-11T00:04:03.833999+0000","last_active":"2024-09-11T00:05:19.840634+0000","last_peered":"2024-09-11T00:05:19.840634+0000","last_clean":"2024-09-11T00:05:19.840634+0000","last_became_active":"2024-09-11T00:04:03.833580+0000","last_became_peered":"2024-09-11T00:04:03.833580+0000","last_unstale":"2024-09-11T00:05:19.840634+0000","last_undegraded":"2024-09-11T00:05:19.840634+0000","last_fullsized":"2024-09-11T00:05:19.840634+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:47:40.902423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T00:05:19.840856+0000","last_change":"2024-09-11T00:04:03.832478+0000","last_active":"2024-09-11T00:05:19.840856+0000","last_peered":"2024-09-11T00:05:19.840856+0000","last_clean":"2024-09-11T00:05:19.840856+0000","last_became_active":"2024-09-11T00:04:03.831811+0000","last_became_peered":"2024-09-11T00:04:03.831811+0000","last_unstale":"2024-09-11T00:05:19.840856+0000","last_undegraded":"2024-09-11T00:05:19.840856+0000","last_fullsized":"2024-09-11T00:05:19.840856+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:53:05.839006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T00:05:19.839432+0000","last_change":"2024-09-11T00:04:03.837077+0000","last_active":"2024-09-11T00:05:19.839432+0000","last_peered":"2024-09-11T00:05:19.839432+0000","last_clean":"2024-09-11T00:05:19.839432+0000","last_became_active":"2024-09-11T00:04:03.835718+0000","last_became_peered":"2024-09-11T00:04:03.835718+0000","last_unstale":"2024-09-11T00:05:19.839432+0000","last_undegraded":"2024-09-11T00:05:19.839432+0000","last_fullsized":"2024-09-11T00:05:19.839432+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:17:00.633257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":71822,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":597,"num_read_kb":377,"num_write":249,"num_write_kb":144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":577536,"data_stored":215466,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":215,"ondisk_log_size":215,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":68,"seq":292057776163,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36288,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708864,"statfs":{"total":95995035648,"available":95957876736,"internally_reserved":0,"allocated":958464,"data_stored":468325,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":36174276},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":63,"seq":270582939688,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28640,"kb_used_data":1608,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716512,"statfs":{"total":95995035648,"available":95965708288,"internally_reserved":0,"allocated":1646592,"data_stored":1119256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":58,"seq":249108103211,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36308,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708844,"statfs":{"total":95995035648,"available":95957856256,"internally_reserved":0,"allocated":978944,"data_stored":471848,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":52,"seq":223338299443,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36292,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708860,"statfs":{"total":95995035648,"available":95957872640,"internally_reserved":0,"allocated":962560,"data_stored":468018,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":47,"seq":201863462966,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28564,"kb_used_data":1532,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716588,"statfs":{"total":95995035648,"available":95965786112,"internally_reserved":0,"allocated":1568768,"data_stored":1064627,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":42,"seq":180388626493,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32120,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713032,"statfs":{"total":95995035648,"available":95962144768,"internally_reserved":0,"allocated":950272,"data_stored":463647,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":36,"seq":154618822722,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28568,"kb_used_data":1540,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716584,"statfs":{"total":95995035648,"available":95965782016,"internally_reserved":0,"allocated":1576960,"data_stored":1059818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986247,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27968,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717184,"statfs":{"total":95995035648,"available":95966396416,"internally_reserved":0,"allocated":958464,"data_stored":467330,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182477,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32128,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713024,"statfs":{"total":95995035648,"available":95962136576,"internally_reserved":0,"allocated":962560,"data_stored":462964,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31914440},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411411,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28044,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717108,"statfs":{"total":95995035648,"available":95966318592,"internally_reserved":0,"allocated":1040384,"data_stored":529515,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574935,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32184,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712968,"statfs":{"total":95995035648,"available":95962079232,"internally_reserved":0,"allocated":1019904,"data_stored":529295,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738461,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32140,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713012,"statfs":{"total":95995035648,"available":95962124288,"internally_reserved":0,"allocated":974848,"data_stored":465103,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2569,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":66761,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":66981,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":700,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":2820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3533,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1388,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":118784,"data_stored":66354,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1695,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T00:05:50.039 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid ff3ad74a-6fcf-11ef-bcea-c7b262605968 -- ceph pg dump --format=json 2024-09-11T00:05:50.420 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:05:50.532 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:05:50 smithi165 ceph-mon[35356]: from='client.15039 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T00:05:50.559 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:05:50 smithi012 ceph-mon[30333]: from='client.15039 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T00:05:50.580 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:05:50 smithi178 ceph-mon[35919]: from='client.15039 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T00:05:50.958 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:50.956+0000 7f2926a00640 1 -- 172.21.15.12:0/3043019108 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f29201087b0 msgr2=0x7f2920108bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:05:50.958 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:50.956+0000 7f2926a00640 1 --2- 172.21.15.12:0/3043019108 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f29201087b0 0x7f2920108bf0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f290c009900 tx=0x7f290c02a410 comp rx=0 tx=0).stop 2024-09-11T00:05:50.958 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:50.956+0000 7f2926a00640 1 -- 172.21.15.12:0/3043019108 shutdown_connections 2024-09-11T00:05:50.959 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:50.956+0000 7f2926a00640 1 --2- 172.21.15.12:0/3043019108 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2920109130 0x7f292010f930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:05:50.959 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:50.956+0000 7f2926a00640 1 --2- 172.21.15.12:0/3043019108 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f29201087b0 0x7f2920108bf0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:05:50.959 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:50.956+0000 7f2926a00640 1 --2- 172.21.15.12:0/3043019108 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2920102da0 0x7f2920103180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:05:50.959 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:50.956+0000 7f2926a00640 1 -- 172.21.15.12:0/3043019108 >> 172.21.15.12:0/3043019108 conn(0x7f29200fe810 msgr2=0x7f2920100c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:05:50.959 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:50.956+0000 7f2926a00640 1 -- 172.21.15.12:0/3043019108 shutdown_connections 2024-09-11T00:05:50.959 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:50.956+0000 7f2926a00640 1 -- 172.21.15.12:0/3043019108 wait complete. 2024-09-11T00:05:50.959 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:50.957+0000 7f2926a00640 1 Processor -- start 2024-09-11T00:05:50.959 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:50.958+0000 7f2926a00640 1 -- start start 2024-09-11T00:05:50.960 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:50.958+0000 7f2926a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2920102da0 0x7f29200754a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:05:50.961 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:50.958+0000 7f2926a00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f29201087b0 0x7f29200759e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:05:50.961 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:50.958+0000 7f2926a00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2920109130 0x7f2920075f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:05:50.961 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:50.958+0000 7f2926a00640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f2920112df0 con 0x7f2920109130 2024-09-11T00:05:50.961 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:50.958+0000 7f2926a00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f2920112c70 con 0x7f2920102da0 2024-09-11T00:05:50.961 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:50.958+0000 7f2926a00640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f2920112f70 con 0x7f29201087b0 2024-09-11T00:05:50.961 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:50.958+0000 7f291fe00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2920102da0 0x7f29200754a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:05:50.961 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:50.958+0000 7f2924a00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2920109130 0x7f2920075f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:05:50.961 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:50.958+0000 7f291fe00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2920102da0 0x7f29200754a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.12:48880/0 (socket says 172.21.15.12:48880) 2024-09-11T00:05:50.961 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:50.958+0000 7f291fe00640 1 -- 172.21.15.12:0/2698840108 learned_addr learned my addr 172.21.15.12:0/2698840108 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:05:50.961 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:50.958+0000 7f291f400640 1 --2- 172.21.15.12:0/2698840108 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f29201087b0 0x7f29200759e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:05:50.961 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:50.959+0000 7f291fe00640 1 -- 172.21.15.12:0/2698840108 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f29201087b0 msgr2=0x7f29200759e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:05:50.962 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:50.959+0000 7f291fe00640 1 --2- 172.21.15.12:0/2698840108 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f29201087b0 0x7f29200759e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:05:50.962 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:50.959+0000 7f291fe00640 1 -- 172.21.15.12:0/2698840108 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2920109130 msgr2=0x7f2920075f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:05:50.962 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:50.959+0000 7f291fe00640 1 --2- 172.21.15.12:0/2698840108 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2920109130 0x7f2920075f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:05:50.962 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:50.959+0000 7f291fe00640 1 -- 172.21.15.12:0/2698840108 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29201a76e0 con 0x7f2920102da0 2024-09-11T00:05:50.962 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:50.959+0000 7f291fe00640 1 --2- 172.21.15.12:0/2698840108 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2920102da0 0x7f29200754a0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f291000e9f0 tx=0x7f291000eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:05:50.962 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:50.959+0000 7f291cc00640 1 -- 172.21.15.12:0/2698840108 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f291000cd80 con 0x7f2920102da0 2024-09-11T00:05:50.962 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:50.959+0000 7f291cc00640 1 -- 172.21.15.12:0/2698840108 <== mon.2 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f29100108e0 con 0x7f2920102da0 2024-09-11T00:05:50.962 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:50.959+0000 7f2926a00640 1 -- 172.21.15.12:0/2698840108 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f29201a79d0 con 0x7f2920102da0 2024-09-11T00:05:50.962 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:50.960+0000 7f291cc00640 1 -- 172.21.15.12:0/2698840108 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2910018980 con 0x7f2920102da0 2024-09-11T00:05:50.962 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:50.960+0000 7f2926a00640 1 -- 172.21.15.12:0/2698840108 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29201a7f60 con 0x7f2920102da0 2024-09-11T00:05:50.963 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:50.961+0000 7f2926a00640 1 -- 172.21.15.12:0/2698840108 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2920103180 con 0x7f2920102da0 2024-09-11T00:05:50.964 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:50.963+0000 7f291cc00640 1 -- 172.21.15.12:0/2698840108 <== mon.2 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 17) ==== 153301+0+0 (secure 0 0 0) 0x7f2910010430 con 0x7f2920102da0 2024-09-11T00:05:50.965 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:50.965+0000 7f291cc00640 1 --2- 172.21.15.12:0/2698840108 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f29040b5370 0x7f29040b7830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:05:50.966 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:50.965+0000 7f291cc00640 1 -- 172.21.15.12:0/2698840108 <== mon.2 v2:172.21.15.165:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 9092+0+0 (secure 0 0 0) 0x7f29100e68b0 con 0x7f2920102da0 2024-09-11T00:05:50.966 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:50.965+0000 7f291f400640 1 --2- 172.21.15.12:0/2698840108 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f29040b5370 0x7f29040b7830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:05:50.966 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:50.966+0000 7f291f400640 1 --2- 172.21.15.12:0/2698840108 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f29040b5370 0x7f29040b7830 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f290c031440 tx=0x7f290c005b50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:05:50.967 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:50.966+0000 7f291cc00640 1 -- 172.21.15.12:0/2698840108 <== mon.2 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f29100acc00 con 0x7f2920102da0 2024-09-11T00:05:51.161 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:51.160+0000 7f2926a00640 1 -- 172.21.15.12:0/2698840108 --> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f2920063590 con 0x7f29040b5370 2024-09-11T00:05:51.167 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:51.166+0000 7f291cc00640 1 -- 172.21.15.12:0/2698840108 <== mgr.14150 v2:172.21.15.12:6800/3442939137 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340397 (secure 0 0 0) 0x7f2920063590 con 0x7f29040b5370 2024-09-11T00:05:51.171 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-11T00:05:51.173 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-11T00:05:51.175 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:51.171+0000 7f2926a00640 1 -- 172.21.15.12:0/2698840108 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f29040b5370 msgr2=0x7f29040b7830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:05:51.175 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:51.171+0000 7f2926a00640 1 --2- 172.21.15.12:0/2698840108 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f29040b5370 0x7f29040b7830 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f290c031440 tx=0x7f290c005b50 comp rx=0 tx=0).stop 2024-09-11T00:05:51.175 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:51.171+0000 7f2926a00640 1 -- 172.21.15.12:0/2698840108 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2920102da0 msgr2=0x7f29200754a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:05:51.175 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:51.171+0000 7f2926a00640 1 --2- 172.21.15.12:0/2698840108 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2920102da0 0x7f29200754a0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f291000e9f0 tx=0x7f291000eeb0 comp rx=0 tx=0).stop 2024-09-11T00:05:51.175 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:51.171+0000 7f2926a00640 1 -- 172.21.15.12:0/2698840108 shutdown_connections 2024-09-11T00:05:51.175 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:51.171+0000 7f2926a00640 1 --2- 172.21.15.12:0/2698840108 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f29040b5370 0x7f29040b7830 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:05:51.175 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:51.171+0000 7f2926a00640 1 --2- 172.21.15.12:0/2698840108 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2920109130 0x7f2920075f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:05:51.175 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:51.171+0000 7f2926a00640 1 --2- 172.21.15.12:0/2698840108 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f29201087b0 0x7f29200759e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:05:51.175 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:51.171+0000 7f2926a00640 1 --2- 172.21.15.12:0/2698840108 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2920102da0 0x7f29200754a0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:05:51.175 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:51.171+0000 7f2926a00640 1 -- 172.21.15.12:0/2698840108 >> 172.21.15.12:0/2698840108 conn(0x7f29200fe810 msgr2=0x7f2920100c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:05:51.175 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:51.172+0000 7f2926a00640 1 -- 172.21.15.12:0/2698840108 shutdown_connections 2024-09-11T00:05:51.176 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:51.172+0000 7f2926a00640 1 -- 172.21.15.12:0/2698840108 wait complete. 2024-09-11T00:05:51.532 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:05:51 smithi165 ceph-mon[35356]: pgmap v359: 129 pgs: 129 active+clean; 647 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-11T00:05:51.563 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:05:51 smithi012 ceph-mon[30333]: pgmap v359: 129 pgs: 129 active+clean; 647 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-11T00:05:51.580 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:05:51 smithi178 ceph-mon[35919]: pgmap v359: 129 pgs: 129 active+clean; 647 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-11T00:05:52.006 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":359,"stamp":"2024-09-11T00:05:50.116926+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":662424,"num_objects":102,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":693,"num_read_kb":451,"num_write":374,"num_write_kb":2296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":23,"num_bytes_recovered":2951952,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":309,"ondisk_log_size":309,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":379244,"kb_used_data":13280,"kb_used_omap":18,"kb_used_meta":365677,"kb_avail":1124562580,"statfs":{"total":1151940427776,"available":1151552081920,"internally_reserved":0,"allocated":13598720,"data_stored":7569746,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19142,"internal_metadata":374453562},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":120,"num_read_kb":60,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003167"},"pg_stats":[{"pgid":"2.1f","version":"81'13","reported_seq":111,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T00:05:48.346625+0000","last_change":"2024-09-11T00:04:01.820872+0000","last_active":"2024-09-11T00:05:48.346625+0000","last_peered":"2024-09-11T00:05:48.346625+0000","last_clean":"2024-09-11T00:05:48.346625+0000","last_became_active":"2024-09-11T00:04:01.820404+0000","last_became_peered":"2024-09-11T00:04:01.820404+0000","last_unstale":"2024-09-11T00:05:48.346625+0000","last_undegraded":"2024-09-11T00:05:48.346625+0000","last_fullsized":"2024-09-11T00:05:48.346625+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_clean_scrub_stamp":"2024-09-11T00:04:00.794959+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:15:38.648289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":61,"num_read_kb":61,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T00:05:19.847074+0000","last_change":"2024-09-11T00:05:19.846978+0000","last_active":"2024-09-11T00:05:19.847074+0000","last_peered":"2024-09-11T00:05:19.847074+0000","last_clean":"2024-09-11T00:05:19.847074+0000","last_became_active":"2024-09-11T00:05:19.846569+0000","last_became_peered":"2024-09-11T00:05:19.846569+0000","last_unstale":"2024-09-11T00:05:19.847074+0000","last_undegraded":"2024-09-11T00:05:19.847074+0000","last_fullsized":"2024-09-11T00:05:19.847074+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:27:46.235552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,5],"acting":[9,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T00:05:19.836785+0000","last_change":"2024-09-11T00:04:11.872109+0000","last_active":"2024-09-11T00:05:19.836785+0000","last_peered":"2024-09-11T00:05:19.836785+0000","last_clean":"2024-09-11T00:05:19.836785+0000","last_became_active":"2024-09-11T00:04:11.871279+0000","last_became_peered":"2024-09-11T00:04:11.871279+0000","last_unstale":"2024-09-11T00:05:19.836785+0000","last_undegraded":"2024-09-11T00:05:19.836785+0000","last_fullsized":"2024-09-11T00:05:19.836785+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_clean_scrub_stamp":"2024-09-11T00:04:10.845136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:13:32.837555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"81'2","reported_seq":39,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T00:05:19.845403+0000","last_change":"2024-09-11T00:04:01.942288+0000","last_active":"2024-09-11T00:05:19.845403+0000","last_peered":"2024-09-11T00:05:19.845403+0000","last_clean":"2024-09-11T00:05:19.845403+0000","last_became_active":"2024-09-11T00:04:01.941838+0000","last_became_peered":"2024-09-11T00:04:01.941838+0000","last_unstale":"2024-09-11T00:05:19.845403+0000","last_undegraded":"2024-09-11T00:05:19.845403+0000","last_fullsized":"2024-09-11T00:05:19.845403+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_clean_scrub_stamp":"2024-09-11T00:04:00.794959+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:48:44.867267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T00:05:19.840736+0000","last_change":"2024-09-11T00:04:03.839951+0000","last_active":"2024-09-11T00:05:19.840736+0000","last_peered":"2024-09-11T00:05:19.840736+0000","last_clean":"2024-09-11T00:05:19.840736+0000","last_became_active":"2024-09-11T00:04:03.839591+0000","last_became_peered":"2024-09-11T00:04:03.839591+0000","last_unstale":"2024-09-11T00:05:19.840736+0000","last_undegraded":"2024-09-11T00:05:19.840736+0000","last_fullsized":"2024-09-11T00:05:19.840736+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:12:49.603310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T00:05:19.983148+0000","last_change":"2024-09-11T00:04:11.872026+0000","last_active":"2024-09-11T00:05:19.983148+0000","last_peered":"2024-09-11T00:05:19.983148+0000","last_clean":"2024-09-11T00:05:19.983148+0000","last_became_active":"2024-09-11T00:04:11.871699+0000","last_became_peered":"2024-09-11T00:04:11.871699+0000","last_unstale":"2024-09-11T00:05:19.983148+0000","last_undegraded":"2024-09-11T00:05:19.983148+0000","last_fullsized":"2024-09-11T00:05:19.983148+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_clean_scrub_stamp":"2024-09-11T00:04:10.845136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:07:47.757223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"81'12","reported_seq":50,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T00:05:19.842919+0000","last_change":"2024-09-11T00:04:01.819942+0000","last_active":"2024-09-11T00:05:19.842919+0000","last_peered":"2024-09-11T00:05:19.842919+0000","last_clean":"2024-09-11T00:05:19.842919+0000","last_became_active":"2024-09-11T00:04:01.819337+0000","last_became_peered":"2024-09-11T00:04:01.819337+0000","last_unstale":"2024-09-11T00:05:19.842919+0000","last_undegraded":"2024-09-11T00:05:19.842919+0000","last_fullsized":"2024-09-11T00:05:19.842919+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_clean_scrub_stamp":"2024-09-11T00:04:00.794959+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:22:26.947067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T00:05:19.837468+0000","last_change":"2024-09-11T00:04:03.839276+0000","last_active":"2024-09-11T00:05:19.837468+0000","last_peered":"2024-09-11T00:05:19.837468+0000","last_clean":"2024-09-11T00:05:19.837468+0000","last_became_active":"2024-09-11T00:04:03.838843+0000","last_became_peered":"2024-09-11T00:04:03.838843+0000","last_unstale":"2024-09-11T00:05:19.837468+0000","last_undegraded":"2024-09-11T00:05:19.837468+0000","last_fullsized":"2024-09-11T00:05:19.837468+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:26:42.640628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T00:05:19.983852+0000","last_change":"2024-09-11T00:04:11.872345+0000","last_active":"2024-09-11T00:05:19.983852+0000","last_peered":"2024-09-11T00:05:19.983852+0000","last_clean":"2024-09-11T00:05:19.983852+0000","last_became_active":"2024-09-11T00:04:11.871994+0000","last_became_peered":"2024-09-11T00:04:11.871994+0000","last_unstale":"2024-09-11T00:05:19.983852+0000","last_undegraded":"2024-09-11T00:05:19.983852+0000","last_fullsized":"2024-09-11T00:05:19.983852+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_clean_scrub_stamp":"2024-09-11T00:04:10.845136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:14:23.906778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"81'5","reported_seq":43,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T00:05:19.844344+0000","last_change":"2024-09-11T00:04:01.819581+0000","last_active":"2024-09-11T00:05:19.844344+0000","last_peered":"2024-09-11T00:05:19.844344+0000","last_clean":"2024-09-11T00:05:19.844344+0000","last_became_active":"2024-09-11T00:04:01.818699+0000","last_became_peered":"2024-09-11T00:04:01.818699+0000","last_unstale":"2024-09-11T00:05:19.844344+0000","last_undegraded":"2024-09-11T00:05:19.844344+0000","last_fullsized":"2024-09-11T00:05:19.844344+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_clean_scrub_stamp":"2024-09-11T00:04:00.794959+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:07:24.540825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T00:05:19.844097+0000","last_change":"2024-09-11T00:04:03.835107+0000","last_active":"2024-09-11T00:05:19.844097+0000","last_peered":"2024-09-11T00:05:19.844097+0000","last_clean":"2024-09-11T00:05:19.844097+0000","last_became_active":"2024-09-11T00:04:03.834782+0000","last_became_peered":"2024-09-11T00:04:03.834782+0000","last_unstale":"2024-09-11T00:05:19.844097+0000","last_undegraded":"2024-09-11T00:05:19.844097+0000","last_fullsized":"2024-09-11T00:05:19.844097+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:46:54.472270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T00:05:19.983645+0000","last_change":"2024-09-11T00:04:11.874168+0000","last_active":"2024-09-11T00:05:19.983645+0000","last_peered":"2024-09-11T00:05:19.983645+0000","last_clean":"2024-09-11T00:05:19.983645+0000","last_became_active":"2024-09-11T00:04:11.873810+0000","last_became_peered":"2024-09-11T00:04:11.873810+0000","last_unstale":"2024-09-11T00:05:19.983645+0000","last_undegraded":"2024-09-11T00:05:19.983645+0000","last_fullsized":"2024-09-11T00:05:19.983645+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_clean_scrub_stamp":"2024-09-11T00:04:10.845136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:06:05.130815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T00:05:19.840764+0000","last_change":"2024-09-11T00:04:03.834016+0000","last_active":"2024-09-11T00:05:19.840764+0000","last_peered":"2024-09-11T00:05:19.840764+0000","last_clean":"2024-09-11T00:05:19.840764+0000","last_became_active":"2024-09-11T00:04:03.833374+0000","last_became_peered":"2024-09-11T00:04:03.833374+0000","last_unstale":"2024-09-11T00:05:19.840764+0000","last_undegraded":"2024-09-11T00:05:19.840764+0000","last_fullsized":"2024-09-11T00:05:19.840764+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:13:34.068594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"81'4","reported_seq":42,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T00:05:19.842488+0000","last_change":"2024-09-11T00:04:01.821173+0000","last_active":"2024-09-11T00:05:19.842488+0000","last_peered":"2024-09-11T00:05:19.842488+0000","last_clean":"2024-09-11T00:05:19.842488+0000","last_became_active":"2024-09-11T00:04:01.820615+0000","last_became_peered":"2024-09-11T00:04:01.820615+0000","last_unstale":"2024-09-11T00:05:19.842488+0000","last_undegraded":"2024-09-11T00:05:19.842488+0000","last_fullsized":"2024-09-11T00:05:19.842488+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_clean_scrub_stamp":"2024-09-11T00:04:00.794959+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:22:11.842837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T00:05:19.983802+0000","last_change":"2024-09-11T00:04:12.077637+0000","last_active":"2024-09-11T00:05:19.983802+0000","last_peered":"2024-09-11T00:05:19.983802+0000","last_clean":"2024-09-11T00:05:19.983802+0000","last_became_active":"2024-09-11T00:04:12.076450+0000","last_became_peered":"2024-09-11T00:04:12.076450+0000","last_unstale":"2024-09-11T00:05:19.983802+0000","last_undegraded":"2024-09-11T00:05:19.983802+0000","last_fullsized":"2024-09-11T00:05:19.983802+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_clean_scrub_stamp":"2024-09-11T00:04:10.845136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:50:54.870468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T00:05:19.844069+0000","last_change":"2024-09-11T00:04:03.846398+0000","last_active":"2024-09-11T00:05:19.844069+0000","last_peered":"2024-09-11T00:05:19.844069+0000","last_clean":"2024-09-11T00:05:19.844069+0000","last_became_active":"2024-09-11T00:04:03.846051+0000","last_became_peered":"2024-09-11T00:04:03.846051+0000","last_unstale":"2024-09-11T00:05:19.844069+0000","last_undegraded":"2024-09-11T00:05:19.844069+0000","last_fullsized":"2024-09-11T00:05:19.844069+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:00:38.506122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"81'3","reported_seq":18,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T00:05:19.860144+0000","last_change":"2024-09-11T00:05:19.860048+0000","last_active":"2024-09-11T00:05:19.860144+0000","last_peered":"2024-09-11T00:05:19.860144+0000","last_clean":"2024-09-11T00:05:19.860144+0000","last_became_active":"2024-09-11T00:05:19.845210+0000","last_became_peered":"2024-09-11T00:05:19.845210+0000","last_unstale":"2024-09-11T00:05:19.860144+0000","last_undegraded":"2024-09-11T00:05:19.860144+0000","last_fullsized":"2024-09-11T00:05:19.860144+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_clean_scrub_stamp":"2024-09-11T00:04:00.794959+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:55:18.935136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"81'1","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T00:05:19.984147+0000","last_change":"2024-09-11T00:04:11.879582+0000","last_active":"2024-09-11T00:05:19.984147+0000","last_peered":"2024-09-11T00:05:19.984147+0000","last_clean":"2024-09-11T00:05:19.984147+0000","last_became_active":"2024-09-11T00:04:11.879098+0000","last_became_peered":"2024-09-11T00:04:11.879098+0000","last_unstale":"2024-09-11T00:05:19.984147+0000","last_undegraded":"2024-09-11T00:05:19.984147+0000","last_fullsized":"2024-09-11T00:05:19.984147+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_clean_scrub_stamp":"2024-09-11T00:04:10.845136+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:06.292841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T00:05:19.839912+0000","last_change":"2024-09-11T00:04:03.834615+0000","last_active":"2024-09-11T00:05:19.839912+0000","last_peered":"2024-09-11T00:05:19.839912+0000","last_clean":"2024-09-11T00:05:19.839912+0000","last_became_active":"2024-09-11T00:04:03.834284+0000","last_became_peered":"2024-09-11T00:04:03.834284+0000","last_unstale":"2024-09-11T00:05:19.839912+0000","last_undegraded":"2024-09-11T00:05:19.839912+0000","last_fullsized":"2024-09-11T00:05:19.839912+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:24:09.542774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"81'7","reported_seq":44,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T00:05:19.843350+0000","last_change":"2024-09-11T00:04:01.818156+0000","last_active":"2024-09-11T00:05:19.843350+0000","last_peered":"2024-09-11T00:05:19.843350+0000","last_clean":"2024-09-11T00:05:19.843350+0000","last_became_active":"2024-09-11T00:04:01.817820+0000","last_became_peered":"2024-09-11T00:04:01.817820+0000","last_unstale":"2024-09-11T00:05:19.843350+0000","last_undegraded":"2024-09-11T00:05:19.843350+0000","last_fullsized":"2024-09-11T00:05:19.843350+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_clean_scrub_stamp":"2024-09-11T00:04:00.794959+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:15:46.304276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T00:05:19.840361+0000","last_change":"2024-09-11T00:04:11.877704+0000","last_active":"2024-09-11T00:05:19.840361+0000","last_peered":"2024-09-11T00:05:19.840361+0000","last_clean":"2024-09-11T00:05:19.840361+0000","last_became_active":"2024-09-11T00:04:11.877320+0000","last_became_peered":"2024-09-11T00:04:11.877320+0000","last_unstale":"2024-09-11T00:05:19.840361+0000","last_undegraded":"2024-09-11T00:05:19.840361+0000","last_fullsized":"2024-09-11T00:05:19.840361+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_clean_scrub_stamp":"2024-09-11T00:04:10.845136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:31:35.046012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T00:05:19.841786+0000","last_change":"2024-09-11T00:04:03.828240+0000","last_active":"2024-09-11T00:05:19.841786+0000","last_peered":"2024-09-11T00:05:19.841786+0000","last_clean":"2024-09-11T00:05:19.841786+0000","last_became_active":"2024-09-11T00:04:03.827890+0000","last_became_peered":"2024-09-11T00:04:03.827890+0000","last_unstale":"2024-09-11T00:05:19.841786+0000","last_undegraded":"2024-09-11T00:05:19.841786+0000","last_fullsized":"2024-09-11T00:05:19.841786+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:43:12.146717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"81'2","reported_seq":39,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T00:05:19.837299+0000","last_change":"2024-09-11T00:04:01.942805+0000","last_active":"2024-09-11T00:05:19.837299+0000","last_peered":"2024-09-11T00:05:19.837299+0000","last_clean":"2024-09-11T00:05:19.837299+0000","last_became_active":"2024-09-11T00:04:01.942429+0000","last_became_peered":"2024-09-11T00:04:01.942429+0000","last_unstale":"2024-09-11T00:05:19.837299+0000","last_undegraded":"2024-09-11T00:05:19.837299+0000","last_fullsized":"2024-09-11T00:05:19.837299+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_clean_scrub_stamp":"2024-09-11T00:04:00.794959+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:51:22.403825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T00:05:19.983874+0000","last_change":"2024-09-11T00:04:12.077246+0000","last_active":"2024-09-11T00:05:19.983874+0000","last_peered":"2024-09-11T00:05:19.983874+0000","last_clean":"2024-09-11T00:05:19.983874+0000","last_became_active":"2024-09-11T00:04:12.076481+0000","last_became_peered":"2024-09-11T00:04:12.076481+0000","last_unstale":"2024-09-11T00:05:19.983874+0000","last_undegraded":"2024-09-11T00:05:19.983874+0000","last_fullsized":"2024-09-11T00:05:19.983874+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_clean_scrub_stamp":"2024-09-11T00:04:10.845136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:55:54.347334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T00:05:19.840940+0000","last_change":"2024-09-11T00:04:03.838431+0000","last_active":"2024-09-11T00:05:19.840940+0000","last_peered":"2024-09-11T00:05:19.840940+0000","last_clean":"2024-09-11T00:05:19.840940+0000","last_became_active":"2024-09-11T00:04:03.834451+0000","last_became_peered":"2024-09-11T00:04:03.834451+0000","last_unstale":"2024-09-11T00:05:19.840940+0000","last_undegraded":"2024-09-11T00:05:19.840940+0000","last_fullsized":"2024-09-11T00:05:19.840940+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:23:01.594299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"81'7","reported_seq":45,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T00:05:19.984326+0000","last_change":"2024-09-11T00:04:02.588834+0000","last_active":"2024-09-11T00:05:19.984326+0000","last_peered":"2024-09-11T00:05:19.984326+0000","last_clean":"2024-09-11T00:05:19.984326+0000","last_became_active":"2024-09-11T00:04:02.588477+0000","last_became_peered":"2024-09-11T00:04:02.588477+0000","last_unstale":"2024-09-11T00:05:19.984326+0000","last_undegraded":"2024-09-11T00:05:19.984326+0000","last_fullsized":"2024-09-11T00:05:19.984326+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_clean_scrub_stamp":"2024-09-11T00:04:00.794959+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:31:21.805591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T00:05:19.836911+0000","last_change":"2024-09-11T00:04:11.863381+0000","last_active":"2024-09-11T00:05:19.836911+0000","last_peered":"2024-09-11T00:05:19.836911+0000","last_clean":"2024-09-11T00:05:19.836911+0000","last_became_active":"2024-09-11T00:04:11.862944+0000","last_became_peered":"2024-09-11T00:04:11.862944+0000","last_unstale":"2024-09-11T00:05:19.836911+0000","last_undegraded":"2024-09-11T00:05:19.836911+0000","last_fullsized":"2024-09-11T00:05:19.836911+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_clean_scrub_stamp":"2024-09-11T00:04:10.845136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:00:14.185992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T00:05:19.845724+0000","last_change":"2024-09-11T00:04:03.829750+0000","last_active":"2024-09-11T00:05:19.845724+0000","last_peered":"2024-09-11T00:05:19.845724+0000","last_clean":"2024-09-11T00:05:19.845724+0000","last_became_active":"2024-09-11T00:04:03.828166+0000","last_became_peered":"2024-09-11T00:04:03.828166+0000","last_unstale":"2024-09-11T00:05:19.845724+0000","last_undegraded":"2024-09-11T00:05:19.845724+0000","last_fullsized":"2024-09-11T00:05:19.845724+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:55:01.667075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"81'15","reported_seq":52,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T00:05:19.841319+0000","last_change":"2024-09-11T00:04:01.818083+0000","last_active":"2024-09-11T00:05:19.841319+0000","last_peered":"2024-09-11T00:05:19.841319+0000","last_clean":"2024-09-11T00:05:19.841319+0000","last_became_active":"2024-09-11T00:04:01.817784+0000","last_became_peered":"2024-09-11T00:04:01.817784+0000","last_unstale":"2024-09-11T00:05:19.841319+0000","last_undegraded":"2024-09-11T00:05:19.841319+0000","last_fullsized":"2024-09-11T00:05:19.841319+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_clean_scrub_stamp":"2024-09-11T00:04:00.794959+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:38:55.466913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":26,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T00:05:19.842101+0000","last_change":"2024-09-11T00:04:12.077053+0000","last_active":"2024-09-11T00:05:19.842101+0000","last_peered":"2024-09-11T00:05:19.842101+0000","last_clean":"2024-09-11T00:05:19.842101+0000","last_became_active":"2024-09-11T00:04:12.076632+0000","last_became_peered":"2024-09-11T00:04:12.076632+0000","last_unstale":"2024-09-11T00:05:19.842101+0000","last_undegraded":"2024-09-11T00:05:19.842101+0000","last_fullsized":"2024-09-11T00:05:19.842101+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_clean_scrub_stamp":"2024-09-11T00:04:10.845136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:19:38.321429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T00:05:19.836751+0000","last_change":"2024-09-11T00:04:03.830316+0000","last_active":"2024-09-11T00:05:19.836751+0000","last_peered":"2024-09-11T00:05:19.836751+0000","last_clean":"2024-09-11T00:05:19.836751+0000","last_became_active":"2024-09-11T00:04:03.829932+0000","last_became_peered":"2024-09-11T00:04:03.829932+0000","last_unstale":"2024-09-11T00:05:19.836751+0000","last_undegraded":"2024-09-11T00:05:19.836751+0000","last_fullsized":"2024-09-11T00:05:19.836751+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:06:14.546651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"81'4","reported_seq":42,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T00:05:19.843047+0000","last_change":"2024-09-11T00:04:01.943004+0000","last_active":"2024-09-11T00:05:19.843047+0000","last_peered":"2024-09-11T00:05:19.843047+0000","last_clean":"2024-09-11T00:05:19.843047+0000","last_became_active":"2024-09-11T00:04:01.942432+0000","last_became_peered":"2024-09-11T00:04:01.942432+0000","last_unstale":"2024-09-11T00:05:19.843047+0000","last_undegraded":"2024-09-11T00:05:19.843047+0000","last_fullsized":"2024-09-11T00:05:19.843047+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_clean_scrub_stamp":"2024-09-11T00:04:00.794959+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:52:50.472596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T00:05:19.838838+0000","last_change":"2024-09-11T00:04:11.866778+0000","last_active":"2024-09-11T00:05:19.838838+0000","last_peered":"2024-09-11T00:05:19.838838+0000","last_clean":"2024-09-11T00:05:19.838838+0000","last_became_active":"2024-09-11T00:04:11.866333+0000","last_became_peered":"2024-09-11T00:04:11.866333+0000","last_unstale":"2024-09-11T00:05:19.838838+0000","last_undegraded":"2024-09-11T00:05:19.838838+0000","last_fullsized":"2024-09-11T00:05:19.838838+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_clean_scrub_stamp":"2024-09-11T00:04:10.845136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:33.680119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T00:05:19.983306+0000","last_change":"2024-09-11T00:04:03.832481+0000","last_active":"2024-09-11T00:05:19.983306+0000","last_peered":"2024-09-11T00:05:19.983306+0000","last_clean":"2024-09-11T00:05:19.983306+0000","last_became_active":"2024-09-11T00:04:03.832162+0000","last_became_peered":"2024-09-11T00:04:03.832162+0000","last_unstale":"2024-09-11T00:05:19.983306+0000","last_undegraded":"2024-09-11T00:05:19.983306+0000","last_fullsized":"2024-09-11T00:05:19.983306+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:32:52.550128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"81'1","reported_seq":38,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T00:05:19.839966+0000","last_change":"2024-09-11T00:04:01.943647+0000","last_active":"2024-09-11T00:05:19.839966+0000","last_peered":"2024-09-11T00:05:19.839966+0000","last_clean":"2024-09-11T00:05:19.839966+0000","last_became_active":"2024-09-11T00:04:01.942714+0000","last_became_peered":"2024-09-11T00:04:01.942714+0000","last_unstale":"2024-09-11T00:05:19.839966+0000","last_undegraded":"2024-09-11T00:05:19.839966+0000","last_fullsized":"2024-09-11T00:05:19.839966+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_clean_scrub_stamp":"2024-09-11T00:04:00.794959+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:22:23.687268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T00:05:19.841947+0000","last_change":"2024-09-11T00:04:12.077133+0000","last_active":"2024-09-11T00:05:19.841947+0000","last_peered":"2024-09-11T00:05:19.841947+0000","last_clean":"2024-09-11T00:05:19.841947+0000","last_became_active":"2024-09-11T00:04:12.076657+0000","last_became_peered":"2024-09-11T00:04:12.076657+0000","last_unstale":"2024-09-11T00:05:19.841947+0000","last_undegraded":"2024-09-11T00:05:19.841947+0000","last_fullsized":"2024-09-11T00:05:19.841947+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_clean_scrub_stamp":"2024-09-11T00:04:10.845136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:05:40.643405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T00:05:19.983081+0000","last_change":"2024-09-11T00:04:03.835454+0000","last_active":"2024-09-11T00:05:19.983081+0000","last_peered":"2024-09-11T00:05:19.983081+0000","last_clean":"2024-09-11T00:05:19.983081+0000","last_became_active":"2024-09-11T00:04:03.835122+0000","last_became_peered":"2024-09-11T00:04:03.835122+0000","last_unstale":"2024-09-11T00:05:19.983081+0000","last_undegraded":"2024-09-11T00:05:19.983081+0000","last_fullsized":"2024-09-11T00:05:19.983081+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:25:49.672766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"81'6","reported_seq":101,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T00:05:44.341301+0000","last_change":"2024-09-11T00:04:01.823738+0000","last_active":"2024-09-11T00:05:44.341301+0000","last_peered":"2024-09-11T00:05:44.341301+0000","last_clean":"2024-09-11T00:05:44.341301+0000","last_became_active":"2024-09-11T00:04:01.823320+0000","last_became_peered":"2024-09-11T00:04:01.823320+0000","last_unstale":"2024-09-11T00:05:44.341301+0000","last_undegraded":"2024-09-11T00:05:44.341301+0000","last_fullsized":"2024-09-11T00:05:44.341301+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_clean_scrub_stamp":"2024-09-11T00:04:00.794959+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:34:30.088454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":58,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T00:05:19.836596+0000","last_change":"2024-09-11T00:04:11.863906+0000","last_active":"2024-09-11T00:05:19.836596+0000","last_peered":"2024-09-11T00:05:19.836596+0000","last_clean":"2024-09-11T00:05:19.836596+0000","last_became_active":"2024-09-11T00:04:11.863571+0000","last_became_peered":"2024-09-11T00:04:11.863571+0000","last_unstale":"2024-09-11T00:05:19.836596+0000","last_undegraded":"2024-09-11T00:05:19.836596+0000","last_fullsized":"2024-09-11T00:05:19.836596+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_clean_scrub_stamp":"2024-09-11T00:04:10.845136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:04:40.288886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T00:05:19.843539+0000","last_change":"2024-09-11T00:04:03.826623+0000","last_active":"2024-09-11T00:05:19.843539+0000","last_peered":"2024-09-11T00:05:19.843539+0000","last_clean":"2024-09-11T00:05:19.843539+0000","last_became_active":"2024-09-11T00:04:03.826202+0000","last_became_peered":"2024-09-11T00:04:03.826202+0000","last_unstale":"2024-09-11T00:05:19.843539+0000","last_undegraded":"2024-09-11T00:05:19.843539+0000","last_fullsized":"2024-09-11T00:05:19.843539+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:23:13.222654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"81'5","reported_seq":103,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T00:05:48.346626+0000","last_change":"2024-09-11T00:04:01.942857+0000","last_active":"2024-09-11T00:05:48.346626+0000","last_peered":"2024-09-11T00:05:48.346626+0000","last_clean":"2024-09-11T00:05:48.346626+0000","last_became_active":"2024-09-11T00:04:01.942494+0000","last_became_peered":"2024-09-11T00:04:01.942494+0000","last_unstale":"2024-09-11T00:05:48.346626+0000","last_undegraded":"2024-09-11T00:05:48.346626+0000","last_fullsized":"2024-09-11T00:05:48.346626+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_clean_scrub_stamp":"2024-09-11T00:04:00.794959+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:40:38.587178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":61,"num_read_kb":61,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T00:05:19.836940+0000","last_change":"2024-09-11T00:04:12.077771+0000","last_active":"2024-09-11T00:05:19.836940+0000","last_peered":"2024-09-11T00:05:19.836940+0000","last_clean":"2024-09-11T00:05:19.836940+0000","last_became_active":"2024-09-11T00:04:12.077366+0000","last_became_peered":"2024-09-11T00:04:12.077366+0000","last_unstale":"2024-09-11T00:05:19.836940+0000","last_undegraded":"2024-09-11T00:05:19.836940+0000","last_fullsized":"2024-09-11T00:05:19.836940+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_clean_scrub_stamp":"2024-09-11T00:04:10.845136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:04:26.808201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T00:05:19.836365+0000","last_change":"2024-09-11T00:04:03.827872+0000","last_active":"2024-09-11T00:05:19.836365+0000","last_peered":"2024-09-11T00:05:19.836365+0000","last_clean":"2024-09-11T00:05:19.836365+0000","last_became_active":"2024-09-11T00:04:03.827577+0000","last_became_peered":"2024-09-11T00:04:03.827577+0000","last_unstale":"2024-09-11T00:05:19.836365+0000","last_undegraded":"2024-09-11T00:05:19.836365+0000","last_fullsized":"2024-09-11T00:05:19.836365+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:37:19.205218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"81'4","reported_seq":52,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T00:05:19.991478+0000","last_change":"2024-09-11T00:05:19.991372+0000","last_active":"2024-09-11T00:05:19.991478+0000","last_peered":"2024-09-11T00:05:19.991478+0000","last_clean":"2024-09-11T00:05:19.991478+0000","last_became_active":"2024-09-11T00:05:19.985328+0000","last_became_peered":"2024-09-11T00:05:19.985328+0000","last_unstale":"2024-09-11T00:05:19.991478+0000","last_undegraded":"2024-09-11T00:05:19.991478+0000","last_fullsized":"2024-09-11T00:05:19.991478+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_clean_scrub_stamp":"2024-09-11T00:04:00.794959+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:17:20.345615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,4],"acting":[7,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T00:05:19.843837+0000","last_change":"2024-09-11T00:04:12.077554+0000","last_active":"2024-09-11T00:05:19.843837+0000","last_peered":"2024-09-11T00:05:19.843837+0000","last_clean":"2024-09-11T00:05:19.843837+0000","last_became_active":"2024-09-11T00:04:12.076619+0000","last_became_peered":"2024-09-11T00:04:12.076619+0000","last_unstale":"2024-09-11T00:05:19.843837+0000","last_undegraded":"2024-09-11T00:05:19.843837+0000","last_fullsized":"2024-09-11T00:05:19.843837+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_clean_scrub_stamp":"2024-09-11T00:04:10.845136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:30:25.505200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T00:05:19.843455+0000","last_change":"2024-09-11T00:04:03.839615+0000","last_active":"2024-09-11T00:05:19.843455+0000","last_peered":"2024-09-11T00:05:19.843455+0000","last_clean":"2024-09-11T00:05:19.843455+0000","last_became_active":"2024-09-11T00:04:03.832175+0000","last_became_peered":"2024-09-11T00:04:03.832175+0000","last_unstale":"2024-09-11T00:05:19.843455+0000","last_undegraded":"2024-09-11T00:05:19.843455+0000","last_fullsized":"2024-09-11T00:05:19.843455+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:17:09.975345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"81'6","reported_seq":46,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T00:05:19.984323+0000","last_change":"2024-09-11T00:04:01.817248+0000","last_active":"2024-09-11T00:05:19.984323+0000","last_peered":"2024-09-11T00:05:19.984323+0000","last_clean":"2024-09-11T00:05:19.984323+0000","last_became_active":"2024-09-11T00:04:01.816896+0000","last_became_peered":"2024-09-11T00:04:01.816896+0000","last_unstale":"2024-09-11T00:05:19.984323+0000","last_undegraded":"2024-09-11T00:05:19.984323+0000","last_fullsized":"2024-09-11T00:05:19.984323+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_clean_scrub_stamp":"2024-09-11T00:04:00.794959+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:46:51.641881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'92","reported_seq":110,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T00:05:19.840658+0000","last_change":"2024-09-11T00:02:47.245193+0000","last_active":"2024-09-11T00:05:19.840658+0000","last_peered":"2024-09-11T00:05:19.840658+0000","last_clean":"2024-09-11T00:05:19.840658+0000","last_became_active":"2024-09-11T00:02:47.239773+0000","last_became_peered":"2024-09-11T00:02:47.239773+0000","last_unstale":"2024-09-11T00:05:19.840658+0000","last_undegraded":"2024-09-11T00:05:19.840658+0000","last_fullsized":"2024-09-11T00:05:19.840658+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:59:10.637655+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:59:10.637655+0000","last_clean_scrub_stamp":"2024-09-10T23:59:10.637655+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:51:52.914831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"81'1","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T00:05:19.841343+0000","last_change":"2024-09-11T00:04:11.867764+0000","last_active":"2024-09-11T00:05:19.841343+0000","last_peered":"2024-09-11T00:05:19.841343+0000","last_clean":"2024-09-11T00:05:19.841343+0000","last_became_active":"2024-09-11T00:04:11.867210+0000","last_became_peered":"2024-09-11T00:04:11.867210+0000","last_unstale":"2024-09-11T00:05:19.841343+0000","last_undegraded":"2024-09-11T00:05:19.841343+0000","last_fullsized":"2024-09-11T00:05:19.841343+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_clean_scrub_stamp":"2024-09-11T00:04:10.845136+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:04:15.439956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T00:05:19.841582+0000","last_change":"2024-09-11T00:04:03.834755+0000","last_active":"2024-09-11T00:05:19.841582+0000","last_peered":"2024-09-11T00:05:19.841582+0000","last_clean":"2024-09-11T00:05:19.841582+0000","last_became_active":"2024-09-11T00:04:03.833955+0000","last_became_peered":"2024-09-11T00:04:03.833955+0000","last_unstale":"2024-09-11T00:05:19.841582+0000","last_undegraded":"2024-09-11T00:05:19.841582+0000","last_fullsized":"2024-09-11T00:05:19.841582+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:47:13.863481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"81'6","reported_seq":106,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T00:05:48.348386+0000","last_change":"2024-09-11T00:04:02.589042+0000","last_active":"2024-09-11T00:05:48.348386+0000","last_peered":"2024-09-11T00:05:48.348386+0000","last_clean":"2024-09-11T00:05:48.348386+0000","last_became_active":"2024-09-11T00:04:02.588611+0000","last_became_peered":"2024-09-11T00:04:02.588611+0000","last_unstale":"2024-09-11T00:05:48.348386+0000","last_undegraded":"2024-09-11T00:05:48.348386+0000","last_fullsized":"2024-09-11T00:05:48.348386+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_clean_scrub_stamp":"2024-09-11T00:04:00.794959+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:41:08.257605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1561,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":63,"num_read_kb":3,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T00:05:19.837208+0000","last_change":"2024-09-11T00:04:11.860715+0000","last_active":"2024-09-11T00:05:19.837208+0000","last_peered":"2024-09-11T00:05:19.837208+0000","last_clean":"2024-09-11T00:05:19.837208+0000","last_became_active":"2024-09-11T00:04:11.860094+0000","last_became_peered":"2024-09-11T00:04:11.860094+0000","last_unstale":"2024-09-11T00:05:19.837208+0000","last_undegraded":"2024-09-11T00:05:19.837208+0000","last_fullsized":"2024-09-11T00:05:19.837208+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_clean_scrub_stamp":"2024-09-11T00:04:10.845136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:25:55.279079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T00:05:19.841217+0000","last_change":"2024-09-11T00:04:03.822665+0000","last_active":"2024-09-11T00:05:19.841217+0000","last_peered":"2024-09-11T00:05:19.841217+0000","last_clean":"2024-09-11T00:05:19.841217+0000","last_became_active":"2024-09-11T00:04:03.822333+0000","last_became_peered":"2024-09-11T00:04:03.822333+0000","last_unstale":"2024-09-11T00:05:19.841217+0000","last_undegraded":"2024-09-11T00:05:19.841217+0000","last_fullsized":"2024-09-11T00:05:19.841217+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:02:07.465335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"81'3","reported_seq":41,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T00:05:19.844583+0000","last_change":"2024-09-11T00:04:01.817244+0000","last_active":"2024-09-11T00:05:19.844583+0000","last_peered":"2024-09-11T00:05:19.844583+0000","last_clean":"2024-09-11T00:05:19.844583+0000","last_became_active":"2024-09-11T00:04:01.816846+0000","last_became_peered":"2024-09-11T00:04:01.816846+0000","last_unstale":"2024-09-11T00:05:19.844583+0000","last_undegraded":"2024-09-11T00:05:19.844583+0000","last_fullsized":"2024-09-11T00:05:19.844583+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_clean_scrub_stamp":"2024-09-11T00:04:00.794959+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:09:46.232332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T00:05:19.839674+0000","last_change":"2024-09-11T00:04:11.872439+0000","last_active":"2024-09-11T00:05:19.839674+0000","last_peered":"2024-09-11T00:05:19.839674+0000","last_clean":"2024-09-11T00:05:19.839674+0000","last_became_active":"2024-09-11T00:04:11.872064+0000","last_became_peered":"2024-09-11T00:04:11.872064+0000","last_unstale":"2024-09-11T00:05:19.839674+0000","last_undegraded":"2024-09-11T00:05:19.839674+0000","last_fullsized":"2024-09-11T00:05:19.839674+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_clean_scrub_stamp":"2024-09-11T00:04:10.845136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:15:47.902247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T00:05:19.841533+0000","last_change":"2024-09-11T00:04:03.829600+0000","last_active":"2024-09-11T00:05:19.841533+0000","last_peered":"2024-09-11T00:05:19.841533+0000","last_clean":"2024-09-11T00:05:19.841533+0000","last_became_active":"2024-09-11T00:04:03.829118+0000","last_became_peered":"2024-09-11T00:04:03.829118+0000","last_unstale":"2024-09-11T00:05:19.841533+0000","last_undegraded":"2024-09-11T00:05:19.841533+0000","last_fullsized":"2024-09-11T00:05:19.841533+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:41:07.069816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"81'5","reported_seq":55,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T00:05:19.989496+0000","last_change":"2024-09-11T00:05:19.989379+0000","last_active":"2024-09-11T00:05:19.989496+0000","last_peered":"2024-09-11T00:05:19.989496+0000","last_clean":"2024-09-11T00:05:19.989496+0000","last_became_active":"2024-09-11T00:05:19.984918+0000","last_became_peered":"2024-09-11T00:05:19.984918+0000","last_unstale":"2024-09-11T00:05:19.989496+0000","last_undegraded":"2024-09-11T00:05:19.989496+0000","last_fullsized":"2024-09-11T00:05:19.989496+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_clean_scrub_stamp":"2024-09-11T00:04:00.794959+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:27:52.063256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4],"acting":[6,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T00:05:19.843119+0000","last_change":"2024-09-11T00:04:11.863812+0000","last_active":"2024-09-11T00:05:19.843119+0000","last_peered":"2024-09-11T00:05:19.843119+0000","last_clean":"2024-09-11T00:05:19.843119+0000","last_became_active":"2024-09-11T00:04:11.863412+0000","last_became_peered":"2024-09-11T00:04:11.863412+0000","last_unstale":"2024-09-11T00:05:19.843119+0000","last_undegraded":"2024-09-11T00:05:19.843119+0000","last_fullsized":"2024-09-11T00:05:19.843119+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_clean_scrub_stamp":"2024-09-11T00:04:10.845136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:12:04.140034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":39,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T00:05:19.985326+0000","last_change":"2024-09-11T00:05:19.985248+0000","last_active":"2024-09-11T00:05:19.985326+0000","last_peered":"2024-09-11T00:05:19.985326+0000","last_clean":"2024-09-11T00:05:19.985326+0000","last_became_active":"2024-09-11T00:05:19.984921+0000","last_became_peered":"2024-09-11T00:05:19.984921+0000","last_unstale":"2024-09-11T00:05:19.985326+0000","last_undegraded":"2024-09-11T00:05:19.985326+0000","last_fullsized":"2024-09-11T00:05:19.985326+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:16:44.370849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,4],"acting":[11,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"81'2","reported_seq":40,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T00:05:19.843607+0000","last_change":"2024-09-11T00:04:01.814191+0000","last_active":"2024-09-11T00:05:19.843607+0000","last_peered":"2024-09-11T00:05:19.843607+0000","last_clean":"2024-09-11T00:05:19.843607+0000","last_became_active":"2024-09-11T00:04:01.813778+0000","last_became_peered":"2024-09-11T00:04:01.813778+0000","last_unstale":"2024-09-11T00:05:19.843607+0000","last_undegraded":"2024-09-11T00:05:19.843607+0000","last_fullsized":"2024-09-11T00:05:19.843607+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_clean_scrub_stamp":"2024-09-11T00:04:00.794959+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:01:49.044436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T00:05:19.843398+0000","last_change":"2024-09-11T00:04:11.870281+0000","last_active":"2024-09-11T00:05:19.843398+0000","last_peered":"2024-09-11T00:05:19.843398+0000","last_clean":"2024-09-11T00:05:19.843398+0000","last_became_active":"2024-09-11T00:04:11.869738+0000","last_became_peered":"2024-09-11T00:04:11.869738+0000","last_unstale":"2024-09-11T00:05:19.843398+0000","last_undegraded":"2024-09-11T00:05:19.843398+0000","last_fullsized":"2024-09-11T00:05:19.843398+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_clean_scrub_stamp":"2024-09-11T00:04:10.845136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:04:08.380199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T00:05:19.843745+0000","last_change":"2024-09-11T00:04:03.835240+0000","last_active":"2024-09-11T00:05:19.843745+0000","last_peered":"2024-09-11T00:05:19.843745+0000","last_clean":"2024-09-11T00:05:19.843745+0000","last_became_active":"2024-09-11T00:04:03.834910+0000","last_became_peered":"2024-09-11T00:04:03.834910+0000","last_unstale":"2024-09-11T00:05:19.843745+0000","last_undegraded":"2024-09-11T00:05:19.843745+0000","last_fullsized":"2024-09-11T00:05:19.843745+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:30:33.996409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.10","version":"81'17","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T00:05:19.856811+0000","last_change":"2024-09-11T00:05:19.856704+0000","last_active":"2024-09-11T00:05:19.856811+0000","last_peered":"2024-09-11T00:05:19.856811+0000","last_clean":"2024-09-11T00:05:19.856811+0000","last_became_active":"2024-09-11T00:05:19.843487+0000","last_became_peered":"2024-09-11T00:05:19.843487+0000","last_unstale":"2024-09-11T00:05:19.856811+0000","last_undegraded":"2024-09-11T00:05:19.856811+0000","last_fullsized":"2024-09-11T00:05:19.856811+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_clean_scrub_stamp":"2024-09-11T00:04:00.794959+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:03:51.375839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T00:05:19.836927+0000","last_change":"2024-09-11T00:04:11.874306+0000","last_active":"2024-09-11T00:05:19.836927+0000","last_peered":"2024-09-11T00:05:19.836927+0000","last_clean":"2024-09-11T00:05:19.836927+0000","last_became_active":"2024-09-11T00:04:11.873934+0000","last_became_peered":"2024-09-11T00:04:11.873934+0000","last_unstale":"2024-09-11T00:05:19.836927+0000","last_undegraded":"2024-09-11T00:05:19.836927+0000","last_fullsized":"2024-09-11T00:05:19.836927+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_clean_scrub_stamp":"2024-09-11T00:04:10.845136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:19:22.584367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T00:05:19.840426+0000","last_change":"2024-09-11T00:04:03.824219+0000","last_active":"2024-09-11T00:05:19.840426+0000","last_peered":"2024-09-11T00:05:19.840426+0000","last_clean":"2024-09-11T00:05:19.840426+0000","last_became_active":"2024-09-11T00:04:03.823877+0000","last_became_peered":"2024-09-11T00:04:03.823877+0000","last_unstale":"2024-09-11T00:05:19.840426+0000","last_undegraded":"2024-09-11T00:05:19.840426+0000","last_fullsized":"2024-09-11T00:05:19.840426+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:04:05.924113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"81'5","reported_seq":104,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T00:05:48.347359+0000","last_change":"2024-09-11T00:04:01.818055+0000","last_active":"2024-09-11T00:05:48.347359+0000","last_peered":"2024-09-11T00:05:48.347359+0000","last_clean":"2024-09-11T00:05:48.347359+0000","last_became_active":"2024-09-11T00:04:01.817483+0000","last_became_peered":"2024-09-11T00:04:01.817483+0000","last_unstale":"2024-09-11T00:05:48.347359+0000","last_undegraded":"2024-09-11T00:05:48.347359+0000","last_fullsized":"2024-09-11T00:05:48.347359+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_clean_scrub_stamp":"2024-09-11T00:04:00.794959+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:16:55.218673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":63,"num_read_kb":3,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T00:05:19.840396+0000","last_change":"2024-09-11T00:04:11.875918+0000","last_active":"2024-09-11T00:05:19.840396+0000","last_peered":"2024-09-11T00:05:19.840396+0000","last_clean":"2024-09-11T00:05:19.840396+0000","last_became_active":"2024-09-11T00:04:11.875549+0000","last_became_peered":"2024-09-11T00:04:11.875549+0000","last_unstale":"2024-09-11T00:05:19.840396+0000","last_undegraded":"2024-09-11T00:05:19.840396+0000","last_fullsized":"2024-09-11T00:05:19.840396+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_clean_scrub_stamp":"2024-09-11T00:04:10.845136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:16:48.685899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T00:05:19.841133+0000","last_change":"2024-09-11T00:04:03.827671+0000","last_active":"2024-09-11T00:05:19.841133+0000","last_peered":"2024-09-11T00:05:19.841133+0000","last_clean":"2024-09-11T00:05:19.841133+0000","last_became_active":"2024-09-11T00:04:03.827341+0000","last_became_peered":"2024-09-11T00:04:03.827341+0000","last_unstale":"2024-09-11T00:05:19.841133+0000","last_undegraded":"2024-09-11T00:05:19.841133+0000","last_fullsized":"2024-09-11T00:05:19.841133+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:56:16.614035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"81'5","reported_seq":43,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T00:05:19.844833+0000","last_change":"2024-09-11T00:04:01.820632+0000","last_active":"2024-09-11T00:05:19.844833+0000","last_peered":"2024-09-11T00:05:19.844833+0000","last_clean":"2024-09-11T00:05:19.844833+0000","last_became_active":"2024-09-11T00:04:01.820273+0000","last_became_peered":"2024-09-11T00:04:01.820273+0000","last_unstale":"2024-09-11T00:05:19.844833+0000","last_undegraded":"2024-09-11T00:05:19.844833+0000","last_fullsized":"2024-09-11T00:05:19.844833+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_clean_scrub_stamp":"2024-09-11T00:04:00.794959+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:34:53.444515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T00:05:19.838915+0000","last_change":"2024-09-11T00:04:11.881197+0000","last_active":"2024-09-11T00:05:19.838915+0000","last_peered":"2024-09-11T00:05:19.838915+0000","last_clean":"2024-09-11T00:05:19.838915+0000","last_became_active":"2024-09-11T00:04:11.880763+0000","last_became_peered":"2024-09-11T00:04:11.880763+0000","last_unstale":"2024-09-11T00:05:19.838915+0000","last_undegraded":"2024-09-11T00:05:19.838915+0000","last_fullsized":"2024-09-11T00:05:19.838915+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_clean_scrub_stamp":"2024-09-11T00:04:10.845136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:14:01.616095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T00:05:19.836503+0000","last_change":"2024-09-11T00:04:03.833470+0000","last_active":"2024-09-11T00:05:19.836503+0000","last_peered":"2024-09-11T00:05:19.836503+0000","last_clean":"2024-09-11T00:05:19.836503+0000","last_became_active":"2024-09-11T00:04:03.833120+0000","last_became_peered":"2024-09-11T00:04:03.833120+0000","last_unstale":"2024-09-11T00:05:19.836503+0000","last_undegraded":"2024-09-11T00:05:19.836503+0000","last_fullsized":"2024-09-11T00:05:19.836503+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:29:53.210230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"81'9","reported_seq":48,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T00:05:19.837431+0000","last_change":"2024-09-11T00:04:01.818081+0000","last_active":"2024-09-11T00:05:19.837431+0000","last_peered":"2024-09-11T00:05:19.837431+0000","last_clean":"2024-09-11T00:05:19.837431+0000","last_became_active":"2024-09-11T00:04:01.817716+0000","last_became_peered":"2024-09-11T00:04:01.817716+0000","last_unstale":"2024-09-11T00:05:19.837431+0000","last_undegraded":"2024-09-11T00:05:19.837431+0000","last_fullsized":"2024-09-11T00:05:19.837431+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_clean_scrub_stamp":"2024-09-11T00:04:00.794959+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:01:32.736319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T00:05:19.843999+0000","last_change":"2024-09-11T00:04:11.870182+0000","last_active":"2024-09-11T00:05:19.843999+0000","last_peered":"2024-09-11T00:05:19.843999+0000","last_clean":"2024-09-11T00:05:19.843999+0000","last_became_active":"2024-09-11T00:04:11.869744+0000","last_became_peered":"2024-09-11T00:04:11.869744+0000","last_unstale":"2024-09-11T00:05:19.843999+0000","last_undegraded":"2024-09-11T00:05:19.843999+0000","last_fullsized":"2024-09-11T00:05:19.843999+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_clean_scrub_stamp":"2024-09-11T00:04:10.845136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:50:24.070404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T00:05:19.842764+0000","last_change":"2024-09-11T00:04:03.828246+0000","last_active":"2024-09-11T00:05:19.842764+0000","last_peered":"2024-09-11T00:05:19.842764+0000","last_clean":"2024-09-11T00:05:19.842764+0000","last_became_active":"2024-09-11T00:04:03.827217+0000","last_became_peered":"2024-09-11T00:04:03.827217+0000","last_unstale":"2024-09-11T00:05:19.842764+0000","last_undegraded":"2024-09-11T00:05:19.842764+0000","last_fullsized":"2024-09-11T00:05:19.842764+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:51:46.018759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"83'36","reported_seq":195,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T00:05:45.342966+0000","last_change":"2024-09-11T00:04:01.819553+0000","last_active":"2024-09-11T00:05:45.342966+0000","last_peered":"2024-09-11T00:05:45.342966+0000","last_clean":"2024-09-11T00:05:45.342966+0000","last_became_active":"2024-09-11T00:04:01.819213+0000","last_became_peered":"2024-09-11T00:04:01.819213+0000","last_unstale":"2024-09-11T00:05:45.342966+0000","last_undegraded":"2024-09-11T00:05:45.342966+0000","last_fullsized":"2024-09-11T00:05:45.342966+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_clean_scrub_stamp":"2024-09-11T00:04:00.794959+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:44:11.039982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":64618,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":122,"num_read_kb":124,"num_write":38,"num_write_kb":79,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T00:05:19.839925+0000","last_change":"2024-09-11T00:04:11.871730+0000","last_active":"2024-09-11T00:05:19.839925+0000","last_peered":"2024-09-11T00:05:19.839925+0000","last_clean":"2024-09-11T00:05:19.839925+0000","last_became_active":"2024-09-11T00:04:11.870659+0000","last_became_peered":"2024-09-11T00:04:11.870659+0000","last_unstale":"2024-09-11T00:05:19.839925+0000","last_undegraded":"2024-09-11T00:05:19.839925+0000","last_fullsized":"2024-09-11T00:05:19.839925+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_clean_scrub_stamp":"2024-09-11T00:04:10.845136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:48:41.835602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T00:05:19.837548+0000","last_change":"2024-09-11T00:04:03.838763+0000","last_active":"2024-09-11T00:05:19.837548+0000","last_peered":"2024-09-11T00:05:19.837548+0000","last_clean":"2024-09-11T00:05:19.837548+0000","last_became_active":"2024-09-11T00:04:03.838409+0000","last_became_peered":"2024-09-11T00:04:03.838409+0000","last_unstale":"2024-09-11T00:05:19.837548+0000","last_undegraded":"2024-09-11T00:05:19.837548+0000","last_fullsized":"2024-09-11T00:05:19.837548+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:57:38.647351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"81'5","reported_seq":43,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T00:05:19.844476+0000","last_change":"2024-09-11T00:04:01.943027+0000","last_active":"2024-09-11T00:05:19.844476+0000","last_peered":"2024-09-11T00:05:19.844476+0000","last_clean":"2024-09-11T00:05:19.844476+0000","last_became_active":"2024-09-11T00:04:01.942558+0000","last_became_peered":"2024-09-11T00:04:01.942558+0000","last_unstale":"2024-09-11T00:05:19.844476+0000","last_undegraded":"2024-09-11T00:05:19.844476+0000","last_fullsized":"2024-09-11T00:05:19.844476+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_clean_scrub_stamp":"2024-09-11T00:04:00.794959+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:48:03.776476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T00:05:19.984081+0000","last_change":"2024-09-11T00:04:11.868069+0000","last_active":"2024-09-11T00:05:19.984081+0000","last_peered":"2024-09-11T00:05:19.984081+0000","last_clean":"2024-09-11T00:05:19.984081+0000","last_became_active":"2024-09-11T00:04:11.867544+0000","last_became_peered":"2024-09-11T00:04:11.867544+0000","last_unstale":"2024-09-11T00:05:19.984081+0000","last_undegraded":"2024-09-11T00:05:19.984081+0000","last_fullsized":"2024-09-11T00:05:19.984081+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_clean_scrub_stamp":"2024-09-11T00:04:10.845136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:09:20.455807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T00:05:19.840832+0000","last_change":"2024-09-11T00:04:03.836735+0000","last_active":"2024-09-11T00:05:19.840832+0000","last_peered":"2024-09-11T00:05:19.840832+0000","last_clean":"2024-09-11T00:05:19.840832+0000","last_became_active":"2024-09-11T00:04:03.833299+0000","last_became_peered":"2024-09-11T00:04:03.833299+0000","last_unstale":"2024-09-11T00:05:19.840832+0000","last_undegraded":"2024-09-11T00:05:19.840832+0000","last_fullsized":"2024-09-11T00:05:19.840832+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:32:15.924058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"81'4","reported_seq":41,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T00:05:19.841672+0000","last_change":"2024-09-11T00:04:01.823777+0000","last_active":"2024-09-11T00:05:19.841672+0000","last_peered":"2024-09-11T00:05:19.841672+0000","last_clean":"2024-09-11T00:05:19.841672+0000","last_became_active":"2024-09-11T00:04:01.823368+0000","last_became_peered":"2024-09-11T00:04:01.823368+0000","last_unstale":"2024-09-11T00:05:19.841672+0000","last_undegraded":"2024-09-11T00:05:19.841672+0000","last_fullsized":"2024-09-11T00:05:19.841672+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_clean_scrub_stamp":"2024-09-11T00:04:00.794959+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:05:12.584867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T00:05:19.839064+0000","last_change":"2024-09-11T00:04:11.868303+0000","last_active":"2024-09-11T00:05:19.839064+0000","last_peered":"2024-09-11T00:05:19.839064+0000","last_clean":"2024-09-11T00:05:19.839064+0000","last_became_active":"2024-09-11T00:04:11.867916+0000","last_became_peered":"2024-09-11T00:04:11.867916+0000","last_unstale":"2024-09-11T00:05:19.839064+0000","last_undegraded":"2024-09-11T00:05:19.839064+0000","last_fullsized":"2024-09-11T00:05:19.839064+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_clean_scrub_stamp":"2024-09-11T00:04:10.845136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:08:29.711763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T00:05:19.837642+0000","last_change":"2024-09-11T00:04:03.832712+0000","last_active":"2024-09-11T00:05:19.837642+0000","last_peered":"2024-09-11T00:05:19.837642+0000","last_clean":"2024-09-11T00:05:19.837642+0000","last_became_active":"2024-09-11T00:04:03.832295+0000","last_became_peered":"2024-09-11T00:04:03.832295+0000","last_unstale":"2024-09-11T00:05:19.837642+0000","last_undegraded":"2024-09-11T00:05:19.837642+0000","last_fullsized":"2024-09-11T00:05:19.837642+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:40:12.271930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"81'4","reported_seq":153,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T00:05:45.342929+0000","last_change":"2024-09-11T00:04:01.820891+0000","last_active":"2024-09-11T00:05:45.342929+0000","last_peered":"2024-09-11T00:05:45.342929+0000","last_clean":"2024-09-11T00:05:45.342929+0000","last_became_active":"2024-09-11T00:04:01.820528+0000","last_became_peered":"2024-09-11T00:04:01.820528+0000","last_unstale":"2024-09-11T00:05:45.342929+0000","last_undegraded":"2024-09-11T00:05:45.342929+0000","last_fullsized":"2024-09-11T00:05:45.342929+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_clean_scrub_stamp":"2024-09-11T00:04:00.794959+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:23:38.196618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":112,"num_read_kb":112,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T00:05:19.843062+0000","last_change":"2024-09-11T00:04:11.878864+0000","last_active":"2024-09-11T00:05:19.843062+0000","last_peered":"2024-09-11T00:05:19.843062+0000","last_clean":"2024-09-11T00:05:19.843062+0000","last_became_active":"2024-09-11T00:04:11.876701+0000","last_became_peered":"2024-09-11T00:04:11.876701+0000","last_unstale":"2024-09-11T00:05:19.843062+0000","last_undegraded":"2024-09-11T00:05:19.843062+0000","last_fullsized":"2024-09-11T00:05:19.843062+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_clean_scrub_stamp":"2024-09-11T00:04:10.845136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:26:05.650618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T00:05:19.840766+0000","last_change":"2024-09-11T00:04:11.868140+0000","last_active":"2024-09-11T00:05:19.840766+0000","last_peered":"2024-09-11T00:05:19.840766+0000","last_clean":"2024-09-11T00:05:19.840766+0000","last_became_active":"2024-09-11T00:04:11.866459+0000","last_became_peered":"2024-09-11T00:04:11.866459+0000","last_unstale":"2024-09-11T00:05:19.840766+0000","last_undegraded":"2024-09-11T00:05:19.840766+0000","last_fullsized":"2024-09-11T00:05:19.840766+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_clean_scrub_stamp":"2024-09-11T00:04:10.845136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:53:25.106039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T00:05:19.839391+0000","last_change":"2024-09-11T00:04:03.829839+0000","last_active":"2024-09-11T00:05:19.839391+0000","last_peered":"2024-09-11T00:05:19.839391+0000","last_clean":"2024-09-11T00:05:19.839391+0000","last_became_active":"2024-09-11T00:04:03.829553+0000","last_became_peered":"2024-09-11T00:04:03.829553+0000","last_unstale":"2024-09-11T00:05:19.839391+0000","last_undegraded":"2024-09-11T00:05:19.839391+0000","last_fullsized":"2024-09-11T00:05:19.839391+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:45:58.561389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"81'3","reported_seq":41,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T00:05:19.843660+0000","last_change":"2024-09-11T00:04:01.820204+0000","last_active":"2024-09-11T00:05:19.843660+0000","last_peered":"2024-09-11T00:05:19.843660+0000","last_clean":"2024-09-11T00:05:19.843660+0000","last_became_active":"2024-09-11T00:04:01.819797+0000","last_became_peered":"2024-09-11T00:04:01.819797+0000","last_unstale":"2024-09-11T00:05:19.843660+0000","last_undegraded":"2024-09-11T00:05:19.843660+0000","last_fullsized":"2024-09-11T00:05:19.843660+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_clean_scrub_stamp":"2024-09-11T00:04:00.794959+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:56:10.530023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T00:05:19.838877+0000","last_change":"2024-09-11T00:04:11.879430+0000","last_active":"2024-09-11T00:05:19.838877+0000","last_peered":"2024-09-11T00:05:19.838877+0000","last_clean":"2024-09-11T00:05:19.838877+0000","last_became_active":"2024-09-11T00:04:11.879010+0000","last_became_peered":"2024-09-11T00:04:11.879010+0000","last_unstale":"2024-09-11T00:05:19.838877+0000","last_undegraded":"2024-09-11T00:05:19.838877+0000","last_fullsized":"2024-09-11T00:05:19.838877+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_clean_scrub_stamp":"2024-09-11T00:04:10.845136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:57.737125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T00:05:19.842352+0000","last_change":"2024-09-11T00:04:03.843918+0000","last_active":"2024-09-11T00:05:19.842352+0000","last_peered":"2024-09-11T00:05:19.842352+0000","last_clean":"2024-09-11T00:05:19.842352+0000","last_became_active":"2024-09-11T00:04:03.843591+0000","last_became_peered":"2024-09-11T00:04:03.843591+0000","last_unstale":"2024-09-11T00:05:19.842352+0000","last_undegraded":"2024-09-11T00:05:19.842352+0000","last_fullsized":"2024-09-11T00:05:19.842352+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:26:41.228802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"81'3","reported_seq":40,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T00:05:19.836363+0000","last_change":"2024-09-11T00:04:01.813173+0000","last_active":"2024-09-11T00:05:19.836363+0000","last_peered":"2024-09-11T00:05:19.836363+0000","last_clean":"2024-09-11T00:05:19.836363+0000","last_became_active":"2024-09-11T00:04:01.811799+0000","last_became_peered":"2024-09-11T00:04:01.811799+0000","last_unstale":"2024-09-11T00:05:19.836363+0000","last_undegraded":"2024-09-11T00:05:19.836363+0000","last_fullsized":"2024-09-11T00:05:19.836363+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_clean_scrub_stamp":"2024-09-11T00:04:00.794959+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:16:47.304371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"81'7","reported_seq":44,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T00:05:19.983032+0000","last_change":"2024-09-11T00:04:01.818649+0000","last_active":"2024-09-11T00:05:19.983032+0000","last_peered":"2024-09-11T00:05:19.983032+0000","last_clean":"2024-09-11T00:05:19.983032+0000","last_became_active":"2024-09-11T00:04:01.818287+0000","last_became_peered":"2024-09-11T00:04:01.818287+0000","last_unstale":"2024-09-11T00:05:19.983032+0000","last_undegraded":"2024-09-11T00:05:19.983032+0000","last_fullsized":"2024-09-11T00:05:19.983032+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_clean_scrub_stamp":"2024-09-11T00:04:00.794959+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:42:00.764648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T00:05:19.844591+0000","last_change":"2024-09-11T00:04:03.832829+0000","last_active":"2024-09-11T00:05:19.844591+0000","last_peered":"2024-09-11T00:05:19.844591+0000","last_clean":"2024-09-11T00:05:19.844591+0000","last_became_active":"2024-09-11T00:04:03.832514+0000","last_became_peered":"2024-09-11T00:04:03.832514+0000","last_unstale":"2024-09-11T00:05:19.844591+0000","last_undegraded":"2024-09-11T00:05:19.844591+0000","last_fullsized":"2024-09-11T00:05:19.844591+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:36.643727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T00:05:19.837064+0000","last_change":"2024-09-11T00:04:11.877596+0000","last_active":"2024-09-11T00:05:19.837064+0000","last_peered":"2024-09-11T00:05:19.837064+0000","last_clean":"2024-09-11T00:05:19.837064+0000","last_became_active":"2024-09-11T00:04:11.877113+0000","last_became_peered":"2024-09-11T00:04:11.877113+0000","last_unstale":"2024-09-11T00:05:19.837064+0000","last_undegraded":"2024-09-11T00:05:19.837064+0000","last_fullsized":"2024-09-11T00:05:19.837064+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_clean_scrub_stamp":"2024-09-11T00:04:10.845136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:47:11.759687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"81'5","reported_seq":100,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T00:05:44.342213+0000","last_change":"2024-09-11T00:04:01.821519+0000","last_active":"2024-09-11T00:05:44.342213+0000","last_peered":"2024-09-11T00:05:44.342213+0000","last_clean":"2024-09-11T00:05:44.342213+0000","last_became_active":"2024-09-11T00:04:01.821207+0000","last_became_peered":"2024-09-11T00:04:01.821207+0000","last_unstale":"2024-09-11T00:05:44.342213+0000","last_undegraded":"2024-09-11T00:05:44.342213+0000","last_fullsized":"2024-09-11T00:05:44.342213+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_clean_scrub_stamp":"2024-09-11T00:04:00.794959+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:16:34.435040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":58,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T00:05:19.842126+0000","last_change":"2024-09-11T00:04:03.832691+0000","last_active":"2024-09-11T00:05:19.842126+0000","last_peered":"2024-09-11T00:05:19.842126+0000","last_clean":"2024-09-11T00:05:19.842126+0000","last_became_active":"2024-09-11T00:04:03.832250+0000","last_became_peered":"2024-09-11T00:04:03.832250+0000","last_unstale":"2024-09-11T00:05:19.842126+0000","last_undegraded":"2024-09-11T00:05:19.842126+0000","last_fullsized":"2024-09-11T00:05:19.842126+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:54.279820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T00:05:19.837089+0000","last_change":"2024-09-11T00:04:11.864995+0000","last_active":"2024-09-11T00:05:19.837089+0000","last_peered":"2024-09-11T00:05:19.837089+0000","last_clean":"2024-09-11T00:05:19.837089+0000","last_became_active":"2024-09-11T00:04:11.864501+0000","last_became_peered":"2024-09-11T00:04:11.864501+0000","last_unstale":"2024-09-11T00:05:19.837089+0000","last_undegraded":"2024-09-11T00:05:19.837089+0000","last_fullsized":"2024-09-11T00:05:19.837089+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_clean_scrub_stamp":"2024-09-11T00:04:10.845136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:08:08.637714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T00:05:19.839813+0000","last_change":"2024-09-11T00:04:03.829310+0000","last_active":"2024-09-11T00:05:19.839813+0000","last_peered":"2024-09-11T00:05:19.839813+0000","last_clean":"2024-09-11T00:05:19.839813+0000","last_became_active":"2024-09-11T00:04:03.828937+0000","last_became_peered":"2024-09-11T00:04:03.828937+0000","last_unstale":"2024-09-11T00:05:19.839813+0000","last_undegraded":"2024-09-11T00:05:19.839813+0000","last_fullsized":"2024-09-11T00:05:19.839813+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:32:58.263067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T00:05:19.837214+0000","last_change":"2024-09-11T00:04:03.828055+0000","last_active":"2024-09-11T00:05:19.837214+0000","last_peered":"2024-09-11T00:05:19.837214+0000","last_clean":"2024-09-11T00:05:19.837214+0000","last_became_active":"2024-09-11T00:04:03.827382+0000","last_became_peered":"2024-09-11T00:04:03.827382+0000","last_unstale":"2024-09-11T00:05:19.837214+0000","last_undegraded":"2024-09-11T00:05:19.837214+0000","last_fullsized":"2024-09-11T00:05:19.837214+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:02:47.107689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T00:05:19.836865+0000","last_change":"2024-09-11T00:04:03.843549+0000","last_active":"2024-09-11T00:05:19.836865+0000","last_peered":"2024-09-11T00:05:19.836865+0000","last_clean":"2024-09-11T00:05:19.836865+0000","last_became_active":"2024-09-11T00:04:03.843149+0000","last_became_peered":"2024-09-11T00:04:03.843149+0000","last_unstale":"2024-09-11T00:05:19.836865+0000","last_undegraded":"2024-09-11T00:05:19.836865+0000","last_fullsized":"2024-09-11T00:05:19.836865+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:47.288319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":39,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T00:05:19.844779+0000","last_change":"2024-09-11T00:05:19.844682+0000","last_active":"2024-09-11T00:05:19.844779+0000","last_peered":"2024-09-11T00:05:19.844779+0000","last_clean":"2024-09-11T00:05:19.844779+0000","last_became_active":"2024-09-11T00:05:19.844269+0000","last_became_peered":"2024-09-11T00:05:19.844269+0000","last_unstale":"2024-09-11T00:05:19.844779+0000","last_undegraded":"2024-09-11T00:05:19.844779+0000","last_fullsized":"2024-09-11T00:05:19.844779+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:47:22.001597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,7],"acting":[0,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T00:05:19.836704+0000","last_change":"2024-09-11T00:04:03.835068+0000","last_active":"2024-09-11T00:05:19.836704+0000","last_peered":"2024-09-11T00:05:19.836704+0000","last_clean":"2024-09-11T00:05:19.836704+0000","last_became_active":"2024-09-11T00:04:03.833166+0000","last_became_peered":"2024-09-11T00:04:03.833166+0000","last_unstale":"2024-09-11T00:05:19.836704+0000","last_undegraded":"2024-09-11T00:05:19.836704+0000","last_fullsized":"2024-09-11T00:05:19.836704+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:07:06.357597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T00:05:19.839328+0000","last_change":"2024-09-11T00:04:03.827310+0000","last_active":"2024-09-11T00:05:19.839328+0000","last_peered":"2024-09-11T00:05:19.839328+0000","last_clean":"2024-09-11T00:05:19.839328+0000","last_became_active":"2024-09-11T00:04:03.826748+0000","last_became_peered":"2024-09-11T00:04:03.826748+0000","last_unstale":"2024-09-11T00:05:19.839328+0000","last_undegraded":"2024-09-11T00:05:19.839328+0000","last_fullsized":"2024-09-11T00:05:19.839328+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:53:15.656708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T00:05:19.837302+0000","last_change":"2024-09-11T00:04:03.835900+0000","last_active":"2024-09-11T00:05:19.837302+0000","last_peered":"2024-09-11T00:05:19.837302+0000","last_clean":"2024-09-11T00:05:19.837302+0000","last_became_active":"2024-09-11T00:04:03.835137+0000","last_became_peered":"2024-09-11T00:04:03.835137+0000","last_unstale":"2024-09-11T00:05:19.837302+0000","last_undegraded":"2024-09-11T00:05:19.837302+0000","last_fullsized":"2024-09-11T00:05:19.837302+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:23:31.184833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T00:05:19.845643+0000","last_change":"2024-09-11T00:04:03.833028+0000","last_active":"2024-09-11T00:05:19.845643+0000","last_peered":"2024-09-11T00:05:19.845643+0000","last_clean":"2024-09-11T00:05:19.845643+0000","last_became_active":"2024-09-11T00:04:03.832703+0000","last_became_peered":"2024-09-11T00:04:03.832703+0000","last_unstale":"2024-09-11T00:05:19.845643+0000","last_undegraded":"2024-09-11T00:05:19.845643+0000","last_fullsized":"2024-09-11T00:05:19.845643+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:38.115627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T00:05:19.843431+0000","last_change":"2024-09-11T00:04:03.832423+0000","last_active":"2024-09-11T00:05:19.843431+0000","last_peered":"2024-09-11T00:05:19.843431+0000","last_clean":"2024-09-11T00:05:19.843431+0000","last_became_active":"2024-09-11T00:04:03.832090+0000","last_became_peered":"2024-09-11T00:04:03.832090+0000","last_unstale":"2024-09-11T00:05:19.843431+0000","last_undegraded":"2024-09-11T00:05:19.843431+0000","last_fullsized":"2024-09-11T00:05:19.843431+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:01:27.258506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T00:05:19.843194+0000","last_change":"2024-09-11T00:04:03.842854+0000","last_active":"2024-09-11T00:05:19.843194+0000","last_peered":"2024-09-11T00:05:19.843194+0000","last_clean":"2024-09-11T00:05:19.843194+0000","last_became_active":"2024-09-11T00:04:03.838666+0000","last_became_peered":"2024-09-11T00:04:03.838666+0000","last_unstale":"2024-09-11T00:05:19.843194+0000","last_undegraded":"2024-09-11T00:05:19.843194+0000","last_fullsized":"2024-09-11T00:05:19.843194+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:45:48.575830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T00:05:19.837513+0000","last_change":"2024-09-11T00:04:03.834175+0000","last_active":"2024-09-11T00:05:19.837513+0000","last_peered":"2024-09-11T00:05:19.837513+0000","last_clean":"2024-09-11T00:05:19.837513+0000","last_became_active":"2024-09-11T00:04:03.833826+0000","last_became_peered":"2024-09-11T00:04:03.833826+0000","last_unstale":"2024-09-11T00:05:19.837513+0000","last_undegraded":"2024-09-11T00:05:19.837513+0000","last_fullsized":"2024-09-11T00:05:19.837513+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:45:57.323216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T00:05:19.836787+0000","last_change":"2024-09-11T00:04:03.836751+0000","last_active":"2024-09-11T00:05:19.836787+0000","last_peered":"2024-09-11T00:05:19.836787+0000","last_clean":"2024-09-11T00:05:19.836787+0000","last_became_active":"2024-09-11T00:04:03.834064+0000","last_became_peered":"2024-09-11T00:04:03.834064+0000","last_unstale":"2024-09-11T00:05:19.836787+0000","last_undegraded":"2024-09-11T00:05:19.836787+0000","last_fullsized":"2024-09-11T00:05:19.836787+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:06:23.059770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T00:05:19.838906+0000","last_change":"2024-09-11T00:04:03.830196+0000","last_active":"2024-09-11T00:05:19.838906+0000","last_peered":"2024-09-11T00:05:19.838906+0000","last_clean":"2024-09-11T00:05:19.838906+0000","last_became_active":"2024-09-11T00:04:03.829856+0000","last_became_peered":"2024-09-11T00:04:03.829856+0000","last_unstale":"2024-09-11T00:05:19.838906+0000","last_undegraded":"2024-09-11T00:05:19.838906+0000","last_fullsized":"2024-09-11T00:05:19.838906+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:34:47.489316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T00:05:19.837396+0000","last_change":"2024-09-11T00:04:03.835053+0000","last_active":"2024-09-11T00:05:19.837396+0000","last_peered":"2024-09-11T00:05:19.837396+0000","last_clean":"2024-09-11T00:05:19.837396+0000","last_became_active":"2024-09-11T00:04:03.834736+0000","last_became_peered":"2024-09-11T00:04:03.834736+0000","last_unstale":"2024-09-11T00:05:19.837396+0000","last_undegraded":"2024-09-11T00:05:19.837396+0000","last_fullsized":"2024-09-11T00:05:19.837396+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:53:40.253775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T00:05:19.841222+0000","last_change":"2024-09-11T00:04:03.834514+0000","last_active":"2024-09-11T00:05:19.841222+0000","last_peered":"2024-09-11T00:05:19.841222+0000","last_clean":"2024-09-11T00:05:19.841222+0000","last_became_active":"2024-09-11T00:04:03.834185+0000","last_became_peered":"2024-09-11T00:04:03.834185+0000","last_unstale":"2024-09-11T00:05:19.841222+0000","last_undegraded":"2024-09-11T00:05:19.841222+0000","last_fullsized":"2024-09-11T00:05:19.841222+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:33:52.949710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T00:05:19.839504+0000","last_change":"2024-09-11T00:04:03.840371+0000","last_active":"2024-09-11T00:05:19.839504+0000","last_peered":"2024-09-11T00:05:19.839504+0000","last_clean":"2024-09-11T00:05:19.839504+0000","last_became_active":"2024-09-11T00:04:03.839933+0000","last_became_peered":"2024-09-11T00:04:03.839933+0000","last_unstale":"2024-09-11T00:05:19.839504+0000","last_undegraded":"2024-09-11T00:05:19.839504+0000","last_fullsized":"2024-09-11T00:05:19.839504+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:24:53.890837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T00:05:19.837159+0000","last_change":"2024-09-11T00:04:03.838386+0000","last_active":"2024-09-11T00:05:19.837159+0000","last_peered":"2024-09-11T00:05:19.837159+0000","last_clean":"2024-09-11T00:05:19.837159+0000","last_became_active":"2024-09-11T00:04:03.837511+0000","last_became_peered":"2024-09-11T00:04:03.837511+0000","last_unstale":"2024-09-11T00:05:19.837159+0000","last_undegraded":"2024-09-11T00:05:19.837159+0000","last_fullsized":"2024-09-11T00:05:19.837159+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:27.399727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":39,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T00:05:19.848091+0000","last_change":"2024-09-11T00:05:19.848001+0000","last_active":"2024-09-11T00:05:19.848091+0000","last_peered":"2024-09-11T00:05:19.848091+0000","last_clean":"2024-09-11T00:05:19.848091+0000","last_became_active":"2024-09-11T00:05:19.847175+0000","last_became_peered":"2024-09-11T00:05:19.847175+0000","last_unstale":"2024-09-11T00:05:19.848091+0000","last_undegraded":"2024-09-11T00:05:19.848091+0000","last_fullsized":"2024-09-11T00:05:19.848091+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:06:13.765740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,2],"acting":[0,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T00:05:19.840548+0000","last_change":"2024-09-11T00:04:03.832618+0000","last_active":"2024-09-11T00:05:19.840548+0000","last_peered":"2024-09-11T00:05:19.840548+0000","last_clean":"2024-09-11T00:05:19.840548+0000","last_became_active":"2024-09-11T00:04:03.831821+0000","last_became_peered":"2024-09-11T00:04:03.831821+0000","last_unstale":"2024-09-11T00:05:19.840548+0000","last_undegraded":"2024-09-11T00:05:19.840548+0000","last_fullsized":"2024-09-11T00:05:19.840548+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:40:40.402422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T00:05:19.836719+0000","last_change":"2024-09-11T00:04:03.825348+0000","last_active":"2024-09-11T00:05:19.836719+0000","last_peered":"2024-09-11T00:05:19.836719+0000","last_clean":"2024-09-11T00:05:19.836719+0000","last_became_active":"2024-09-11T00:04:03.824977+0000","last_became_peered":"2024-09-11T00:04:03.824977+0000","last_unstale":"2024-09-11T00:05:19.836719+0000","last_undegraded":"2024-09-11T00:05:19.836719+0000","last_fullsized":"2024-09-11T00:05:19.836719+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:19:53.412969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T00:05:19.983752+0000","last_change":"2024-09-11T00:05:19.983677+0000","last_active":"2024-09-11T00:05:19.983752+0000","last_peered":"2024-09-11T00:05:19.983752+0000","last_clean":"2024-09-11T00:05:19.983752+0000","last_became_active":"2024-09-11T00:05:19.983404+0000","last_became_peered":"2024-09-11T00:05:19.983404+0000","last_unstale":"2024-09-11T00:05:19.983752+0000","last_undegraded":"2024-09-11T00:05:19.983752+0000","last_fullsized":"2024-09-11T00:05:19.983752+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:58:11.624190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,10,8],"acting":[9,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T00:05:19.841363+0000","last_change":"2024-09-11T00:04:03.840068+0000","last_active":"2024-09-11T00:05:19.841363+0000","last_peered":"2024-09-11T00:05:19.841363+0000","last_clean":"2024-09-11T00:05:19.841363+0000","last_became_active":"2024-09-11T00:04:03.835185+0000","last_became_peered":"2024-09-11T00:04:03.835185+0000","last_unstale":"2024-09-11T00:05:19.841363+0000","last_undegraded":"2024-09-11T00:05:19.841363+0000","last_fullsized":"2024-09-11T00:05:19.841363+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:19:10.699038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T00:05:19.983754+0000","last_change":"2024-09-11T00:04:03.828576+0000","last_active":"2024-09-11T00:05:19.983754+0000","last_peered":"2024-09-11T00:05:19.983754+0000","last_clean":"2024-09-11T00:05:19.983754+0000","last_became_active":"2024-09-11T00:04:03.828255+0000","last_became_peered":"2024-09-11T00:04:03.828255+0000","last_unstale":"2024-09-11T00:05:19.983754+0000","last_undegraded":"2024-09-11T00:05:19.983754+0000","last_fullsized":"2024-09-11T00:05:19.983754+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:29:54.164145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T00:05:19.837616+0000","last_change":"2024-09-11T00:04:03.841452+0000","last_active":"2024-09-11T00:05:19.837616+0000","last_peered":"2024-09-11T00:05:19.837616+0000","last_clean":"2024-09-11T00:05:19.837616+0000","last_became_active":"2024-09-11T00:04:03.840911+0000","last_became_peered":"2024-09-11T00:04:03.840911+0000","last_unstale":"2024-09-11T00:05:19.837616+0000","last_undegraded":"2024-09-11T00:05:19.837616+0000","last_fullsized":"2024-09-11T00:05:19.837616+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:56:40.121922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T00:05:19.842259+0000","last_change":"2024-09-11T00:04:03.842009+0000","last_active":"2024-09-11T00:05:19.842259+0000","last_peered":"2024-09-11T00:05:19.842259+0000","last_clean":"2024-09-11T00:05:19.842259+0000","last_became_active":"2024-09-11T00:04:03.841625+0000","last_became_peered":"2024-09-11T00:04:03.841625+0000","last_unstale":"2024-09-11T00:05:19.842259+0000","last_undegraded":"2024-09-11T00:05:19.842259+0000","last_fullsized":"2024-09-11T00:05:19.842259+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:15:11.855308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":39,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T00:05:19.845975+0000","last_change":"2024-09-11T00:05:19.845891+0000","last_active":"2024-09-11T00:05:19.845975+0000","last_peered":"2024-09-11T00:05:19.845975+0000","last_clean":"2024-09-11T00:05:19.845975+0000","last_became_active":"2024-09-11T00:05:19.845547+0000","last_became_peered":"2024-09-11T00:05:19.845547+0000","last_unstale":"2024-09-11T00:05:19.845975+0000","last_undegraded":"2024-09-11T00:05:19.845975+0000","last_fullsized":"2024-09-11T00:05:19.845975+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:02:20.178667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T00:05:19.982904+0000","last_change":"2024-09-11T00:04:03.834540+0000","last_active":"2024-09-11T00:05:19.982904+0000","last_peered":"2024-09-11T00:05:19.982904+0000","last_clean":"2024-09-11T00:05:19.982904+0000","last_became_active":"2024-09-11T00:04:03.834230+0000","last_became_peered":"2024-09-11T00:04:03.834230+0000","last_unstale":"2024-09-11T00:05:19.982904+0000","last_undegraded":"2024-09-11T00:05:19.982904+0000","last_fullsized":"2024-09-11T00:05:19.982904+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:14:26.125003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T00:05:19.842391+0000","last_change":"2024-09-11T00:04:03.836177+0000","last_active":"2024-09-11T00:05:19.842391+0000","last_peered":"2024-09-11T00:05:19.842391+0000","last_clean":"2024-09-11T00:05:19.842391+0000","last_became_active":"2024-09-11T00:04:03.835848+0000","last_became_peered":"2024-09-11T00:04:03.835848+0000","last_unstale":"2024-09-11T00:05:19.842391+0000","last_undegraded":"2024-09-11T00:05:19.842391+0000","last_fullsized":"2024-09-11T00:05:19.842391+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:57:53.854738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T00:05:19.984468+0000","last_change":"2024-09-11T00:04:03.837614+0000","last_active":"2024-09-11T00:05:19.984468+0000","last_peered":"2024-09-11T00:05:19.984468+0000","last_clean":"2024-09-11T00:05:19.984468+0000","last_became_active":"2024-09-11T00:04:03.837164+0000","last_became_peered":"2024-09-11T00:04:03.837164+0000","last_unstale":"2024-09-11T00:05:19.984468+0000","last_undegraded":"2024-09-11T00:05:19.984468+0000","last_fullsized":"2024-09-11T00:05:19.984468+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:10:38.467648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T00:05:19.840634+0000","last_change":"2024-09-11T00:04:03.833999+0000","last_active":"2024-09-11T00:05:19.840634+0000","last_peered":"2024-09-11T00:05:19.840634+0000","last_clean":"2024-09-11T00:05:19.840634+0000","last_became_active":"2024-09-11T00:04:03.833580+0000","last_became_peered":"2024-09-11T00:04:03.833580+0000","last_unstale":"2024-09-11T00:05:19.840634+0000","last_undegraded":"2024-09-11T00:05:19.840634+0000","last_fullsized":"2024-09-11T00:05:19.840634+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:47:40.902423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T00:05:19.840856+0000","last_change":"2024-09-11T00:04:03.832478+0000","last_active":"2024-09-11T00:05:19.840856+0000","last_peered":"2024-09-11T00:05:19.840856+0000","last_clean":"2024-09-11T00:05:19.840856+0000","last_became_active":"2024-09-11T00:04:03.831811+0000","last_became_peered":"2024-09-11T00:04:03.831811+0000","last_unstale":"2024-09-11T00:05:19.840856+0000","last_undegraded":"2024-09-11T00:05:19.840856+0000","last_fullsized":"2024-09-11T00:05:19.840856+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:53:05.839006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T00:05:19.839432+0000","last_change":"2024-09-11T00:04:03.837077+0000","last_active":"2024-09-11T00:05:19.839432+0000","last_peered":"2024-09-11T00:05:19.839432+0000","last_clean":"2024-09-11T00:05:19.839432+0000","last_became_active":"2024-09-11T00:04:03.835718+0000","last_became_peered":"2024-09-11T00:04:03.835718+0000","last_unstale":"2024-09-11T00:05:19.839432+0000","last_undegraded":"2024-09-11T00:05:19.839432+0000","last_fullsized":"2024-09-11T00:05:19.839432+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:17:00.633257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":71822,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":617,"num_read_kb":387,"num_write":249,"num_write_kb":144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":577536,"data_stored":215466,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":215,"ondisk_log_size":215,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":68,"seq":292057776164,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36288,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708864,"statfs":{"total":95995035648,"available":95957876736,"internally_reserved":0,"allocated":958464,"data_stored":468325,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":36174276},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":63,"seq":270582939688,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28640,"kb_used_data":1608,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716512,"statfs":{"total":95995035648,"available":95965708288,"internally_reserved":0,"allocated":1646592,"data_stored":1119256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":58,"seq":249108103212,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36308,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708844,"statfs":{"total":95995035648,"available":95957856256,"internally_reserved":0,"allocated":978944,"data_stored":471848,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":52,"seq":223338299443,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36292,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708860,"statfs":{"total":95995035648,"available":95957872640,"internally_reserved":0,"allocated":962560,"data_stored":468018,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":47,"seq":201863462966,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28564,"kb_used_data":1532,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716588,"statfs":{"total":95995035648,"available":95965786112,"internally_reserved":0,"allocated":1568768,"data_stored":1064627,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":42,"seq":180388626494,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32120,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713032,"statfs":{"total":95995035648,"available":95962144768,"internally_reserved":0,"allocated":950272,"data_stored":463647,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":36,"seq":154618822722,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28568,"kb_used_data":1540,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716584,"statfs":{"total":95995035648,"available":95965782016,"internally_reserved":0,"allocated":1576960,"data_stored":1059818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986248,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27968,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717184,"statfs":{"total":95995035648,"available":95966396416,"internally_reserved":0,"allocated":958464,"data_stored":467330,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182477,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32128,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713024,"statfs":{"total":95995035648,"available":95962136576,"internally_reserved":0,"allocated":962560,"data_stored":462964,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31914440},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411412,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28044,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717108,"statfs":{"total":95995035648,"available":95966318592,"internally_reserved":0,"allocated":1040384,"data_stored":529515,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574936,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32184,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712968,"statfs":{"total":95995035648,"available":95962079232,"internally_reserved":0,"allocated":1019904,"data_stored":529295,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738462,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32140,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713012,"statfs":{"total":95995035648,"available":95962124288,"internally_reserved":0,"allocated":974848,"data_stored":465103,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2569,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":66761,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":66981,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":700,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":2820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3533,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1388,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":118784,"data_stored":66354,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1695,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T00:05:52.012 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-09-11T00:05:52.012 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-09-11T00:05:52.012 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-09-11T00:05:52.012 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid ff3ad74a-6fcf-11ef-bcea-c7b262605968 -- ceph health --format=json 2024-09-11T00:05:52.312 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:05:52 smithi012 ceph-mon[30333]: from='client.25082 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T00:05:52.392 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:05:52.532 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:05:52 smithi165 ceph-mon[35356]: from='client.25082 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T00:05:52.581 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:05:52 smithi178 ceph-mon[35919]: from='client.25082 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T00:05:52.899 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:52.897+0000 7f1f10e00640 1 -- 172.21.15.12:0/2731471077 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1f0c1087d0 msgr2=0x7f1f0c108c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:05:52.899 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:52.897+0000 7f1f10e00640 1 --2- 172.21.15.12:0/2731471077 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1f0c1087d0 0x7f1f0c108c10 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f1ef4009900 tx=0x7f1ef402a410 comp rx=0 tx=0).stop 2024-09-11T00:05:52.899 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:52.897+0000 7f1f10e00640 1 -- 172.21.15.12:0/2731471077 shutdown_connections 2024-09-11T00:05:52.899 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:52.897+0000 7f1f10e00640 1 --2- 172.21.15.12:0/2731471077 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1f0c109150 0x7f1f0c10f950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:05:52.899 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:52.897+0000 7f1f10e00640 1 --2- 172.21.15.12:0/2731471077 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1f0c1087d0 0x7f1f0c108c10 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:05:52.899 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:52.897+0000 7f1f10e00640 1 --2- 172.21.15.12:0/2731471077 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1f0c102dc0 0x7f1f0c1031a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:05:52.899 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:52.897+0000 7f1f10e00640 1 -- 172.21.15.12:0/2731471077 >> 172.21.15.12:0/2731471077 conn(0x7f1f0c0fe830 msgr2=0x7f1f0c100c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:05:52.899 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:52.898+0000 7f1f10e00640 1 -- 172.21.15.12:0/2731471077 shutdown_connections 2024-09-11T00:05:52.899 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:52.898+0000 7f1f10e00640 1 -- 172.21.15.12:0/2731471077 wait complete. 2024-09-11T00:05:52.900 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:52.899+0000 7f1f10e00640 1 Processor -- start 2024-09-11T00:05:52.900 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:52.899+0000 7f1f10e00640 1 -- start start 2024-09-11T00:05:52.901 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:52.899+0000 7f1f10e00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1f0c102dc0 0x7f1f0c19b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:05:52.901 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:52.899+0000 7f1f10e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1f0c1087d0 0x7f1f0c19ba00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:05:52.901 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:52.899+0000 7f1f10e00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1f0c109150 0x7f1f0c1a0da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:05:52.901 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:52.899+0000 7f1f10e00640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f1f0c112dd0 con 0x7f1f0c102dc0 2024-09-11T00:05:52.901 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:52.899+0000 7f1f10e00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f1f0c112c50 con 0x7f1f0c1087d0 2024-09-11T00:05:52.901 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:52.899+0000 7f1f10e00640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f1f0c112f50 con 0x7f1f0c109150 2024-09-11T00:05:52.901 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:52.899+0000 7f1f09e00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1f0c102dc0 0x7f1f0c19b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:05:52.901 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:52.900+0000 7f1f09e00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1f0c102dc0 0x7f1f0c19b4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:56940/0 (socket says 172.21.15.12:56940) 2024-09-11T00:05:52.901 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:52.900+0000 7f1f09e00640 1 -- 172.21.15.12:0/1914139945 learned_addr learned my addr 172.21.15.12:0/1914139945 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:05:52.901 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:52.900+0000 7f1f0a800640 1 --2- 172.21.15.12:0/1914139945 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1f0c109150 0x7f1f0c1a0da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:05:52.901 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:52.900+0000 7f1f01400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1f0c1087d0 0x7f1f0c19ba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:05:52.902 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:52.900+0000 7f1f09e00640 1 -- 172.21.15.12:0/1914139945 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1f0c109150 msgr2=0x7f1f0c1a0da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:05:52.903 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:52.900+0000 7f1f09e00640 1 --2- 172.21.15.12:0/1914139945 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1f0c109150 0x7f1f0c1a0da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:05:52.903 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:52.900+0000 7f1f09e00640 1 -- 172.21.15.12:0/1914139945 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1f0c1087d0 msgr2=0x7f1f0c19ba00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:05:52.903 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:52.900+0000 7f1f09e00640 1 --2- 172.21.15.12:0/1914139945 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1f0c1087d0 0x7f1f0c19ba00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:05:52.903 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:52.900+0000 7f1f09e00640 1 -- 172.21.15.12:0/1914139945 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f0c1a1570 con 0x7f1f0c102dc0 2024-09-11T00:05:52.903 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:52.900+0000 7f1f0a800640 1 --2- 172.21.15.12:0/1914139945 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1f0c109150 0x7f1f0c1a0da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:05:52.903 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:52.900+0000 7f1f09e00640 1 --2- 172.21.15.12:0/1914139945 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1f0c102dc0 0x7f1f0c19b4c0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f1efc00e9f0 tx=0x7f1efc00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:05:52.903 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:52.901+0000 7f1f03400640 1 -- 172.21.15.12:0/1914139945 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1efc00cd80 con 0x7f1f0c102dc0 2024-09-11T00:05:52.903 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:52.901+0000 7f1f03400640 1 -- 172.21.15.12:0/1914139945 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1efc0108e0 con 0x7f1f0c102dc0 2024-09-11T00:05:52.903 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:52.901+0000 7f1f10e00640 1 -- 172.21.15.12:0/1914139945 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1f0c1a1860 con 0x7f1f0c102dc0 2024-09-11T00:05:52.903 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:52.901+0000 7f1f03400640 1 -- 172.21.15.12:0/1914139945 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1efc018980 con 0x7f1f0c102dc0 2024-09-11T00:05:52.903 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:52.901+0000 7f1f10e00640 1 -- 172.21.15.12:0/1914139945 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1f0c1a80f0 con 0x7f1f0c102dc0 2024-09-11T00:05:52.904 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:52.902+0000 7f1f10e00640 1 -- 172.21.15.12:0/1914139945 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1f0c104070 con 0x7f1f0c102dc0 2024-09-11T00:05:52.905 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:52.904+0000 7f1f03400640 1 -- 172.21.15.12:0/1914139945 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 17) ==== 153301+0+0 (secure 0 0 0) 0x7f1efc010430 con 0x7f1f0c102dc0 2024-09-11T00:05:52.908 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:52.907+0000 7f1f03400640 1 --2- 172.21.15.12:0/1914139945 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f1ef00b53f0 0x7f1ef00b78b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:05:52.908 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:52.907+0000 7f1f03400640 1 -- 172.21.15.12:0/1914139945 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 9092+0+0 (secure 0 0 0) 0x7f1efc0e6890 con 0x7f1f0c102dc0 2024-09-11T00:05:52.908 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:52.907+0000 7f1f01400640 1 --2- 172.21.15.12:0/1914139945 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f1ef00b53f0 0x7f1ef00b78b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:05:52.909 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:52.908+0000 7f1f01400640 1 --2- 172.21.15.12:0/1914139945 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f1ef00b53f0 0x7f1ef00b78b0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f1ef4031440 tx=0x7f1ef4005b50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:05:52.909 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:52.909+0000 7f1f03400640 1 -- 172.21.15.12:0/1914139945 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f1efc0acbe0 con 0x7f1f0c102dc0 2024-09-11T00:05:53.149 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:53.148+0000 7f1f10e00640 1 -- 172.21.15.12:0/1914139945 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f1f0c1031a0 con 0x7f1f0c102dc0 2024-09-11T00:05:53.149 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:53.149+0000 7f1f03400640 1 -- 172.21.15.12:0/1914139945 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7f1f0c1031a0 con 0x7f1f0c102dc0 2024-09-11T00:05:53.150 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-11T00:05:53.150 INFO:teuthology.orchestra.run.smithi012.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-09-11T00:05:53.152 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:53.151+0000 7f1f10e00640 1 -- 172.21.15.12:0/1914139945 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f1ef00b53f0 msgr2=0x7f1ef00b78b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:05:53.152 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:53.151+0000 7f1f10e00640 1 --2- 172.21.15.12:0/1914139945 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f1ef00b53f0 0x7f1ef00b78b0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f1ef4031440 tx=0x7f1ef4005b50 comp rx=0 tx=0).stop 2024-09-11T00:05:53.152 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:53.151+0000 7f1f10e00640 1 -- 172.21.15.12:0/1914139945 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1f0c102dc0 msgr2=0x7f1f0c19b4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:05:53.152 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:53.151+0000 7f1f10e00640 1 --2- 172.21.15.12:0/1914139945 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1f0c102dc0 0x7f1f0c19b4c0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f1efc00e9f0 tx=0x7f1efc00eeb0 comp rx=0 tx=0).stop 2024-09-11T00:05:53.152 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:53.151+0000 7f1f10e00640 1 -- 172.21.15.12:0/1914139945 shutdown_connections 2024-09-11T00:05:53.152 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:53.151+0000 7f1f10e00640 1 --2- 172.21.15.12:0/1914139945 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f1ef00b53f0 0x7f1ef00b78b0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:05:53.152 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:53.151+0000 7f1f10e00640 1 --2- 172.21.15.12:0/1914139945 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1f0c109150 0x7f1f0c1a0da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:05:53.153 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:53.151+0000 7f1f10e00640 1 --2- 172.21.15.12:0/1914139945 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1f0c1087d0 0x7f1f0c19ba00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:05:53.153 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:53.151+0000 7f1f10e00640 1 --2- 172.21.15.12:0/1914139945 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1f0c102dc0 0x7f1f0c19b4c0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:05:53.153 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:53.151+0000 7f1f10e00640 1 -- 172.21.15.12:0/1914139945 >> 172.21.15.12:0/1914139945 conn(0x7f1f0c0fe830 msgr2=0x7f1f0c105db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:05:53.153 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:53.151+0000 7f1f10e00640 1 -- 172.21.15.12:0/1914139945 shutdown_connections 2024-09-11T00:05:53.153 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:53.151+0000 7f1f10e00640 1 -- 172.21.15.12:0/1914139945 wait complete. 2024-09-11T00:05:53.532 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:05:53 smithi165 ceph-mon[35356]: pgmap v360: 129 pgs: 129 active+clean; 647 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-09-11T00:05:53.532 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:05:53 smithi165 ceph-mon[35356]: from='client.? 172.21.15.12:0/1914139945' entity='client.admin' cmd={"prefix": "health", "format": "json"} : dispatch 2024-09-11T00:05:53.537 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:05:53 smithi012 ceph-mon[30333]: pgmap v360: 129 pgs: 129 active+clean; 647 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-09-11T00:05:53.537 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:05:53 smithi012 ceph-mon[30333]: from='client.? 172.21.15.12:0/1914139945' entity='client.admin' cmd={"prefix": "health", "format": "json"} : dispatch 2024-09-11T00:05:53.581 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:05:53 smithi178 ceph-mon[35919]: pgmap v360: 129 pgs: 129 active+clean; 647 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-09-11T00:05:53.581 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:05:53 smithi178 ceph-mon[35919]: from='client.? 172.21.15.12:0/1914139945' entity='client.admin' cmd={"prefix": "health", "format": "json"} : dispatch 2024-09-11T00:05:53.916 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-09-11T00:05:53.916 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid ff3ad74a-6fcf-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T00:05:54.296 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:05:54.832 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:54.830+0000 7f822a400640 1 -- 172.21.15.12:0/1434988030 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f822410f3e0 msgr2=0x7f82241117d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:05:54.832 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:54.830+0000 7f822a400640 1 --2- 172.21.15.12:0/1434988030 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f822410f3e0 0x7f82241117d0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f821c009ca0 tx=0x7f821c02a8c0 comp rx=0 tx=0).stop 2024-09-11T00:05:54.832 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:54.831+0000 7f822a400640 1 -- 172.21.15.12:0/1434988030 shutdown_connections 2024-09-11T00:05:54.832 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:54.831+0000 7f822a400640 1 --2- 172.21.15.12:0/1434988030 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f822410f3e0 0x7f82241117d0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:05:54.832 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:54.831+0000 7f822a400640 1 --2- 172.21.15.12:0/1434988030 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8224075f40 0x7f822410ee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:05:54.832 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:54.831+0000 7f822a400640 1 --2- 172.21.15.12:0/1434988030 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8224075540 0x7f8224075920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:05:54.833 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:54.831+0000 7f822a400640 1 -- 172.21.15.12:0/1434988030 >> 172.21.15.12:0/1434988030 conn(0x7f82240fe830 msgr2=0x7f8224100c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:05:54.833 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:54.831+0000 7f822a400640 1 -- 172.21.15.12:0/1434988030 shutdown_connections 2024-09-11T00:05:54.833 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:54.831+0000 7f822a400640 1 -- 172.21.15.12:0/1434988030 wait complete. 2024-09-11T00:05:54.835 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:54.832+0000 7f822a400640 1 Processor -- start 2024-09-11T00:05:54.835 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:54.832+0000 7f822a400640 1 -- start start 2024-09-11T00:05:54.835 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:54.832+0000 7f822a400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8224075540 0x7f822419b460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:05:54.835 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:54.832+0000 7f822a400640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8224075f40 0x7f822419b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:05:54.835 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:54.832+0000 7f822a400640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f822410f3e0 0x7f82241a0d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:05:54.835 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:54.832+0000 7f822a400640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f8224114fb0 con 0x7f822410f3e0 2024-09-11T00:05:54.835 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:54.832+0000 7f822a400640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f8224114e30 con 0x7f8224075540 2024-09-11T00:05:54.835 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:54.832+0000 7f822a400640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f8224115130 con 0x7f8224075f40 2024-09-11T00:05:54.835 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:54.832+0000 7f8223e00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f822410f3e0 0x7f82241a0d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:05:54.835 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:54.832+0000 7f8223400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8224075540 0x7f822419b460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:05:54.836 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:54.832+0000 7f8222a00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8224075f40 0x7f822419b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:05:54.836 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:54.832+0000 7f8223e00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f822410f3e0 0x7f82241a0d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:56964/0 (socket says 172.21.15.12:56964) 2024-09-11T00:05:54.836 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:54.832+0000 7f8223e00640 1 -- 172.21.15.12:0/2254089626 learned_addr learned my addr 172.21.15.12:0/2254089626 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:05:54.836 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:54.833+0000 7f8222a00640 1 -- 172.21.15.12:0/2254089626 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8224075540 msgr2=0x7f822419b460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:05:54.836 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:54.833+0000 7f8222a00640 1 --2- 172.21.15.12:0/2254089626 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8224075540 0x7f822419b460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:05:54.836 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:54.833+0000 7f8222a00640 1 -- 172.21.15.12:0/2254089626 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f822410f3e0 msgr2=0x7f82241a0d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:05:54.836 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:54.833+0000 7f8222a00640 1 --2- 172.21.15.12:0/2254089626 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f822410f3e0 0x7f82241a0d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:05:54.836 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:54.833+0000 7f8222a00640 1 -- 172.21.15.12:0/2254089626 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82241a1510 con 0x7f8224075f40 2024-09-11T00:05:54.836 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:54.833+0000 7f8223e00640 1 --2- 172.21.15.12:0/2254089626 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f822410f3e0 0x7f82241a0d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:05:54.837 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:54.833+0000 7f8222a00640 1 --2- 172.21.15.12:0/2254089626 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8224075f40 0x7f822419b9a0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f821000e9f0 tx=0x7f821000eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:05:54.837 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:54.833+0000 7f821be00640 1 -- 172.21.15.12:0/2254089626 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f821000ccc0 con 0x7f8224075f40 2024-09-11T00:05:54.837 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:54.834+0000 7f822a400640 1 -- 172.21.15.12:0/2254089626 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82241a1800 con 0x7f8224075f40 2024-09-11T00:05:54.837 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:54.834+0000 7f821be00640 1 -- 172.21.15.12:0/2254089626 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f82100108e0 con 0x7f8224075f40 2024-09-11T00:05:54.837 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:54.834+0000 7f821be00640 1 -- 172.21.15.12:0/2254089626 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8210018880 con 0x7f8224075f40 2024-09-11T00:05:54.837 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:54.834+0000 7f822a400640 1 -- 172.21.15.12:0/2254089626 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82241a7ec0 con 0x7f8224075f40 2024-09-11T00:05:54.837 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:54.835+0000 7f822a400640 1 -- 172.21.15.12:0/2254089626 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8224075920 con 0x7f8224075f40 2024-09-11T00:05:54.839 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:54.838+0000 7f821be00640 1 -- 172.21.15.12:0/2254089626 <== mon.1 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 17) ==== 153301+0+0 (secure 0 0 0) 0x7f8210010430 con 0x7f8224075f40 2024-09-11T00:05:54.840 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:54.839+0000 7f821be00640 1 --2- 172.21.15.12:0/2254089626 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f82080b53f0 0x7f82080b78b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:05:54.841 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:54.840+0000 7f821be00640 1 -- 172.21.15.12:0/2254089626 <== mon.1 v2:172.21.15.178:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 9092+0+0 (secure 0 0 0) 0x7f82100e67b0 con 0x7f8224075f40 2024-09-11T00:05:54.841 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:54.840+0000 7f8223400640 1 --2- 172.21.15.12:0/2254089626 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f82080b53f0 0x7f82080b78b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:05:54.841 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:54.840+0000 7f821be00640 1 -- 172.21.15.12:0/2254089626 <== mon.1 v2:172.21.15.178:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f82100acb00 con 0x7f8224075f40 2024-09-11T00:05:54.842 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:54.840+0000 7f8223400640 1 --2- 172.21.15.12:0/2254089626 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f82080b53f0 0x7f82080b78b0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f8214005d00 tx=0x7f821400b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:05:55.072 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:55.071+0000 7f822a400640 1 -- 172.21.15.12:0/2254089626 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f822419c590 con 0x7f8224075f40 2024-09-11T00:05:55.073 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:55.072+0000 7f821be00640 1 -- 172.21.15.12:0/2254089626 <== mon.1 v2:172.21.15.178:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11965 (secure 0 0 0) 0x7f82100b1530 con 0x7f8224075f40 2024-09-11T00:05:55.074 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-11T00:05:55.074 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":28,"btime":"2024-09-11T00:04:50:247323+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14577,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6835/447171591","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":447171591},{"type":"v1","addr":"172.21.15.12:6835","nonce":447171591}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14589,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/901486040","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":901486040},{"type":"v1","addr":"172.21.15.12:6837","nonce":901486040}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T00:04:04.739881+0000","modified":"2024-09-11T00:04:50.247301+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24674,"mds_1":24643,"mds_2":24638,"mds_3":24637,"mds_4":24614},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14631":{"gid":14631,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.12:6841/3069919437","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6840","nonce":3069919437},{"type":"v1","addr":"172.21.15.12:6841","nonce":3069919437}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24562":{"gid":24562,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6833/1141636112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":1141636112},{"type":"v1","addr":"172.21.15.178:6833","nonce":1141636112}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24581":{"gid":24581,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6833/3111980970","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6832","nonce":3111980970},{"type":"v1","addr":"172.21.15.165:6833","nonce":3111980970}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24607":{"gid":24607,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6835/4084868287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":4084868287},{"type":"v1","addr":"172.21.15.178:6835","nonce":4084868287}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24613":{"gid":24613,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6837/526874763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":526874763},{"type":"v1","addr":"172.21.15.178:6837","nonce":526874763}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24614":{"gid":24614,"name":"e","rank":4,"incarnation":26,"state":"up:active","state_seq":21,"addr":"172.21.15.165:6835/2425395274","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":2425395274},{"type":"v1","addr":"172.21.15.165:6835","nonce":2425395274}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24637":{"gid":24637,"name":"i","rank":3,"incarnation":24,"state":"up:active","state_seq":18,"addr":"172.21.15.178:6839/70628097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6838","nonce":70628097},{"type":"v1","addr":"172.21.15.178:6839","nonce":70628097}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24638":{"gid":24638,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":18,"addr":"172.21.15.165:6837/6215088","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6836","nonce":6215088},{"type":"v1","addr":"172.21.15.165:6837","nonce":6215088}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24643":{"gid":24643,"name":"g","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.12:6839/3124093009","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6838","nonce":3124093009},{"type":"v1","addr":"172.21.15.12:6839","nonce":3124093009}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24674":{"gid":24674,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.165:6839/501312984","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6838","nonce":501312984},{"type":"v1","addr":"172.21.15.165:6839","nonce":501312984}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24674,"qdb_cluster":[24674,24643,24638,24637,24614]},"id":1}]} 2024-09-11T00:05:55.074 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 28 2024-09-11T00:05:55.076 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:55.075+0000 7f822a400640 1 -- 172.21.15.12:0/2254089626 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f82080b53f0 msgr2=0x7f82080b78b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:05:55.076 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:55.075+0000 7f822a400640 1 --2- 172.21.15.12:0/2254089626 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f82080b53f0 0x7f82080b78b0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f8214005d00 tx=0x7f821400b410 comp rx=0 tx=0).stop 2024-09-11T00:05:55.076 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:55.075+0000 7f822a400640 1 -- 172.21.15.12:0/2254089626 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8224075f40 msgr2=0x7f822419b9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:05:55.077 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:55.075+0000 7f822a400640 1 --2- 172.21.15.12:0/2254089626 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8224075f40 0x7f822419b9a0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f821000e9f0 tx=0x7f821000eeb0 comp rx=0 tx=0).stop 2024-09-11T00:05:55.077 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:55.075+0000 7f822a400640 1 -- 172.21.15.12:0/2254089626 shutdown_connections 2024-09-11T00:05:55.077 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:55.075+0000 7f822a400640 1 --2- 172.21.15.12:0/2254089626 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f82080b53f0 0x7f82080b78b0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:05:55.077 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:55.075+0000 7f822a400640 1 --2- 172.21.15.12:0/2254089626 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f822410f3e0 0x7f82241a0d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:05:55.077 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:55.075+0000 7f822a400640 1 --2- 172.21.15.12:0/2254089626 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8224075f40 0x7f822419b9a0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:05:55.077 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:55.075+0000 7f822a400640 1 --2- 172.21.15.12:0/2254089626 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8224075540 0x7f822419b460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:05:55.077 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:55.075+0000 7f822a400640 1 -- 172.21.15.12:0/2254089626 >> 172.21.15.12:0/2254089626 conn(0x7f82240fe830 msgr2=0x7f82241115a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:05:55.077 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:55.075+0000 7f822a400640 1 -- 172.21.15.12:0/2254089626 shutdown_connections 2024-09-11T00:05:55.077 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:55.075+0000 7f822a400640 1 -- 172.21.15.12:0/2254089626 wait complete. 2024-09-11T00:05:55.414 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:05:55 smithi012 ceph-mon[30333]: pgmap v361: 129 pgs: 129 active+clean; 647 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-11T00:05:55.414 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:05:55 smithi012 ceph-mon[30333]: from='client.? 172.21.15.12:0/2254089626' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T00:05:55.532 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:05:55 smithi165 ceph-mon[35356]: pgmap v361: 129 pgs: 129 active+clean; 647 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-11T00:05:55.532 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:05:55 smithi165 ceph-mon[35356]: from='client.? 172.21.15.12:0/2254089626' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T00:05:55.581 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:05:55 smithi178 ceph-mon[35919]: pgmap v361: 129 pgs: 129 active+clean; 647 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-11T00:05:55.581 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:05:55 smithi178 ceph-mon[35919]: from='client.? 172.21.15.12:0/2254089626' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T00:05:55.939 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid ff3ad74a-6fcf-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T00:05:56.321 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:05:56.873 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:56.871+0000 7fa53e000640 1 -- 172.21.15.12:0/3364643438 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa5381087b0 msgr2=0x7fa538108bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:05:56.873 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:56.871+0000 7fa53e000640 1 --2- 172.21.15.12:0/3364643438 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa5381087b0 0x7fa538108bf0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fa520009900 tx=0x7fa52002a410 comp rx=0 tx=0).stop 2024-09-11T00:05:56.873 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:56.871+0000 7fa53e000640 1 -- 172.21.15.12:0/3364643438 shutdown_connections 2024-09-11T00:05:56.873 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:56.871+0000 7fa53e000640 1 --2- 172.21.15.12:0/3364643438 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa538109130 0x7fa53810f930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:05:56.873 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:56.871+0000 7fa53e000640 1 --2- 172.21.15.12:0/3364643438 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa5381087b0 0x7fa538108bf0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:05:56.873 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:56.871+0000 7fa53e000640 1 --2- 172.21.15.12:0/3364643438 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa538102da0 0x7fa538103180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:05:56.873 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:56.872+0000 7fa53e000640 1 -- 172.21.15.12:0/3364643438 >> 172.21.15.12:0/3364643438 conn(0x7fa5380fe810 msgr2=0x7fa538100c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:05:56.873 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:56.872+0000 7fa53e000640 1 -- 172.21.15.12:0/3364643438 shutdown_connections 2024-09-11T00:05:56.873 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:56.872+0000 7fa53e000640 1 -- 172.21.15.12:0/3364643438 wait complete. 2024-09-11T00:05:56.875 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:56.872+0000 7fa53e000640 1 Processor -- start 2024-09-11T00:05:56.875 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:56.873+0000 7fa53e000640 1 -- start start 2024-09-11T00:05:56.875 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:56.873+0000 7fa53e000640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa538102da0 0x7fa538199290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:05:56.875 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:56.873+0000 7fa53e000640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa5381087b0 0x7fa5381997d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:05:56.875 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:56.873+0000 7fa53e000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa538109130 0x7fa53819eb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:05:56.875 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:56.873+0000 7fa53e000640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fa538112d80 con 0x7fa538102da0 2024-09-11T00:05:56.875 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:56.873+0000 7fa53e000640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fa538112c00 con 0x7fa538109130 2024-09-11T00:05:56.875 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:56.873+0000 7fa53e000640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fa538112f00 con 0x7fa5381087b0 2024-09-11T00:05:56.875 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:56.873+0000 7fa537400640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa538102da0 0x7fa538199290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:05:56.875 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:56.873+0000 7fa537400640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa538102da0 0x7fa538199290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:56986/0 (socket says 172.21.15.12:56986) 2024-09-11T00:05:56.876 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:56.873+0000 7fa537400640 1 -- 172.21.15.12:0/4239412840 learned_addr learned my addr 172.21.15.12:0/4239412840 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:05:56.876 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:56.873+0000 7fa52ea00640 1 --2- 172.21.15.12:0/4239412840 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa5381087b0 0x7fa5381997d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:05:56.876 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:56.874+0000 7fa537e00640 1 --2- 172.21.15.12:0/4239412840 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa538109130 0x7fa53819eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:05:56.876 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:56.874+0000 7fa537400640 1 -- 172.21.15.12:0/4239412840 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa5381087b0 msgr2=0x7fa5381997d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:05:56.876 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:56.874+0000 7fa537400640 1 --2- 172.21.15.12:0/4239412840 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa5381087b0 0x7fa5381997d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:05:56.876 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:56.874+0000 7fa537400640 1 -- 172.21.15.12:0/4239412840 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa538109130 msgr2=0x7fa53819eb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:05:56.876 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:56.874+0000 7fa537400640 1 --2- 172.21.15.12:0/4239412840 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa538109130 0x7fa53819eb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:05:56.876 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:56.874+0000 7fa537400640 1 -- 172.21.15.12:0/4239412840 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa53819f340 con 0x7fa538102da0 2024-09-11T00:05:56.877 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:56.874+0000 7fa537400640 1 --2- 172.21.15.12:0/4239412840 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa538102da0 0x7fa538199290 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fa52800d950 tx=0x7fa52800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:05:56.877 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:56.875+0000 7fa534c00640 1 -- 172.21.15.12:0/4239412840 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa528017030 con 0x7fa538102da0 2024-09-11T00:05:56.877 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:56.875+0000 7fa53e000640 1 -- 172.21.15.12:0/4239412840 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa53819f630 con 0x7fa538102da0 2024-09-11T00:05:56.877 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:56.875+0000 7fa534c00640 1 -- 172.21.15.12:0/4239412840 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa528004020 con 0x7fa538102da0 2024-09-11T00:05:56.877 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:56.875+0000 7fa534c00640 1 -- 172.21.15.12:0/4239412840 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5280047a0 con 0x7fa538102da0 2024-09-11T00:05:56.877 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:56.875+0000 7fa53e000640 1 -- 172.21.15.12:0/4239412840 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5381a5cf0 con 0x7fa538102da0 2024-09-11T00:05:56.877 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:56.876+0000 7fa53e000640 1 -- 172.21.15.12:0/4239412840 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa538103180 con 0x7fa538102da0 2024-09-11T00:05:56.879 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:56.878+0000 7fa534c00640 1 -- 172.21.15.12:0/4239412840 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 17) ==== 153301+0+0 (secure 0 0 0) 0x7fa528009a30 con 0x7fa538102da0 2024-09-11T00:05:56.880 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:56.879+0000 7fa534c00640 1 --2- 172.21.15.12:0/4239412840 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fa51c0b5370 0x7fa51c0b7830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:05:56.881 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:56.880+0000 7fa534c00640 1 -- 172.21.15.12:0/4239412840 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 9092+0+0 (secure 0 0 0) 0x7fa5280e5b50 con 0x7fa538102da0 2024-09-11T00:05:56.881 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:56.880+0000 7fa52ea00640 1 --2- 172.21.15.12:0/4239412840 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fa51c0b5370 0x7fa51c0b7830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:05:56.881 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:56.880+0000 7fa52ea00640 1 --2- 172.21.15.12:0/4239412840 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fa51c0b5370 0x7fa51c0b7830 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fa520031440 tx=0x7fa520005b50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:05:56.884 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:56.883+0000 7fa534c00640 1 -- 172.21.15.12:0/4239412840 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7fa5280ac4f0 con 0x7fa538102da0 2024-09-11T00:05:57.121 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:57.120+0000 7fa53e000640 1 -- 172.21.15.12:0/4239412840 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa53819a3c0 con 0x7fa538102da0 2024-09-11T00:05:57.121 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:57.120+0000 7fa534c00640 1 -- 172.21.15.12:0/4239412840 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11965 (secure 0 0 0) 0x7fa5280abdf0 con 0x7fa538102da0 2024-09-11T00:05:57.122 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-11T00:05:57.122 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":28,"btime":"2024-09-11T00:04:50:247323+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14577,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6835/447171591","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":447171591},{"type":"v1","addr":"172.21.15.12:6835","nonce":447171591}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14589,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/901486040","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":901486040},{"type":"v1","addr":"172.21.15.12:6837","nonce":901486040}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T00:04:04.739881+0000","modified":"2024-09-11T00:04:50.247301+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24674,"mds_1":24643,"mds_2":24638,"mds_3":24637,"mds_4":24614},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14631":{"gid":14631,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.12:6841/3069919437","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6840","nonce":3069919437},{"type":"v1","addr":"172.21.15.12:6841","nonce":3069919437}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24562":{"gid":24562,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6833/1141636112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":1141636112},{"type":"v1","addr":"172.21.15.178:6833","nonce":1141636112}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24581":{"gid":24581,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6833/3111980970","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6832","nonce":3111980970},{"type":"v1","addr":"172.21.15.165:6833","nonce":3111980970}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24607":{"gid":24607,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6835/4084868287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":4084868287},{"type":"v1","addr":"172.21.15.178:6835","nonce":4084868287}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24613":{"gid":24613,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6837/526874763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":526874763},{"type":"v1","addr":"172.21.15.178:6837","nonce":526874763}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24614":{"gid":24614,"name":"e","rank":4,"incarnation":26,"state":"up:active","state_seq":21,"addr":"172.21.15.165:6835/2425395274","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":2425395274},{"type":"v1","addr":"172.21.15.165:6835","nonce":2425395274}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24637":{"gid":24637,"name":"i","rank":3,"incarnation":24,"state":"up:active","state_seq":18,"addr":"172.21.15.178:6839/70628097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6838","nonce":70628097},{"type":"v1","addr":"172.21.15.178:6839","nonce":70628097}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24638":{"gid":24638,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":18,"addr":"172.21.15.165:6837/6215088","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6836","nonce":6215088},{"type":"v1","addr":"172.21.15.165:6837","nonce":6215088}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24643":{"gid":24643,"name":"g","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.12:6839/3124093009","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6838","nonce":3124093009},{"type":"v1","addr":"172.21.15.12:6839","nonce":3124093009}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24674":{"gid":24674,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.165:6839/501312984","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6838","nonce":501312984},{"type":"v1","addr":"172.21.15.165:6839","nonce":501312984}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24674,"qdb_cluster":[24674,24643,24638,24637,24614]},"id":1}]} 2024-09-11T00:05:57.122 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 28 2024-09-11T00:05:57.124 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:57.123+0000 7fa53e000640 1 -- 172.21.15.12:0/4239412840 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fa51c0b5370 msgr2=0x7fa51c0b7830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:05:57.124 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:57.123+0000 7fa53e000640 1 --2- 172.21.15.12:0/4239412840 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fa51c0b5370 0x7fa51c0b7830 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fa520031440 tx=0x7fa520005b50 comp rx=0 tx=0).stop 2024-09-11T00:05:57.124 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:57.123+0000 7fa53e000640 1 -- 172.21.15.12:0/4239412840 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa538102da0 msgr2=0x7fa538199290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:05:57.125 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:57.123+0000 7fa53e000640 1 --2- 172.21.15.12:0/4239412840 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa538102da0 0x7fa538199290 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fa52800d950 tx=0x7fa52800de10 comp rx=0 tx=0).stop 2024-09-11T00:05:57.125 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:57.123+0000 7fa53e000640 1 -- 172.21.15.12:0/4239412840 shutdown_connections 2024-09-11T00:05:57.125 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:57.123+0000 7fa53e000640 1 --2- 172.21.15.12:0/4239412840 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fa51c0b5370 0x7fa51c0b7830 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:05:57.125 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:57.123+0000 7fa53e000640 1 --2- 172.21.15.12:0/4239412840 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa538109130 0x7fa53819eb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:05:57.125 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:57.123+0000 7fa53e000640 1 --2- 172.21.15.12:0/4239412840 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa5381087b0 0x7fa5381997d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:05:57.125 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:57.123+0000 7fa53e000640 1 --2- 172.21.15.12:0/4239412840 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa538102da0 0x7fa538199290 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:05:57.125 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:57.123+0000 7fa53e000640 1 -- 172.21.15.12:0/4239412840 >> 172.21.15.12:0/4239412840 conn(0x7fa5380fe810 msgr2=0x7fa538100c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:05:57.125 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:57.123+0000 7fa53e000640 1 -- 172.21.15.12:0/4239412840 shutdown_connections 2024-09-11T00:05:57.125 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:57.123+0000 7fa53e000640 1 -- 172.21.15.12:0/4239412840 wait complete. 2024-09-11T00:05:57.478 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:05:57 smithi012 ceph-mon[30333]: pgmap v362: 129 pgs: 129 active+clean; 647 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-11T00:05:57.478 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:05:57 smithi012 ceph-mon[30333]: from='client.? 172.21.15.12:0/4239412840' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T00:05:57.532 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:05:57 smithi165 ceph-mon[35356]: pgmap v362: 129 pgs: 129 active+clean; 647 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-11T00:05:57.532 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:05:57 smithi165 ceph-mon[35356]: from='client.? 172.21.15.12:0/4239412840' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T00:05:57.581 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:05:57 smithi178 ceph-mon[35919]: pgmap v362: 129 pgs: 129 active+clean; 647 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-11T00:05:57.581 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:05:57 smithi178 ceph-mon[35919]: from='client.? 172.21.15.12:0/4239412840' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T00:05:57.967 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid ff3ad74a-6fcf-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-11T00:05:58.346 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:05:58.906 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:58.905+0000 7fb758800640 1 -- 172.21.15.12:0/3888598805 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb750075e00 msgr2=0x7fb75010ee50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:05:58.906 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:58.905+0000 7fb758800640 1 --2- 172.21.15.12:0/3888598805 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb750075e00 0x7fb75010ee50 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fb740009900 tx=0x7fb74002a410 comp rx=0 tx=0).stop 2024-09-11T00:05:58.906 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:58.905+0000 7fb758800640 1 -- 172.21.15.12:0/3888598805 shutdown_connections 2024-09-11T00:05:58.906 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:58.905+0000 7fb758800640 1 --2- 172.21.15.12:0/3888598805 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb75010f390 0x7fb750111780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:05:58.906 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:58.905+0000 7fb758800640 1 --2- 172.21.15.12:0/3888598805 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb750075e00 0x7fb75010ee50 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:05:58.906 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:58.905+0000 7fb758800640 1 --2- 172.21.15.12:0/3888598805 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb7500754e0 0x7fb7500758c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:05:58.907 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:58.905+0000 7fb758800640 1 -- 172.21.15.12:0/3888598805 >> 172.21.15.12:0/3888598805 conn(0x7fb7500fe810 msgr2=0x7fb750100c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:05:58.907 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:58.905+0000 7fb758800640 1 -- 172.21.15.12:0/3888598805 shutdown_connections 2024-09-11T00:05:58.907 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:58.905+0000 7fb758800640 1 -- 172.21.15.12:0/3888598805 wait complete. 2024-09-11T00:05:58.909 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:58.906+0000 7fb758800640 1 Processor -- start 2024-09-11T00:05:58.909 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:58.906+0000 7fb758800640 1 -- start start 2024-09-11T00:05:58.909 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:58.906+0000 7fb758800640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb7500754e0 0x7fb75019b510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:05:58.909 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:58.906+0000 7fb758800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb750075e00 0x7fb75019ba50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:05:58.909 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:58.906+0000 7fb758800640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb75010f390 0x7fb7501a0df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:05:58.909 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:58.906+0000 7fb758800640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fb750114fa0 con 0x7fb7500754e0 2024-09-11T00:05:58.909 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:58.906+0000 7fb758800640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fb750114e20 con 0x7fb750075e00 2024-09-11T00:05:58.909 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:58.906+0000 7fb758800640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fb750115120 con 0x7fb75010f390 2024-09-11T00:05:58.909 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:58.907+0000 7fb755e00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb7500754e0 0x7fb75019b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:05:58.910 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:58.907+0000 7fb755e00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb7500754e0 0x7fb75019b510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:38578/0 (socket says 172.21.15.12:38578) 2024-09-11T00:05:58.910 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:58.907+0000 7fb755e00640 1 -- 172.21.15.12:0/3495217849 learned_addr learned my addr 172.21.15.12:0/3495217849 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:05:58.910 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:58.907+0000 7fb755400640 1 --2- 172.21.15.12:0/3495217849 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb750075e00 0x7fb75019ba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:05:58.910 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:58.907+0000 7fb756800640 1 --2- 172.21.15.12:0/3495217849 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb75010f390 0x7fb7501a0df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:05:58.910 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:58.907+0000 7fb755e00640 1 -- 172.21.15.12:0/3495217849 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb75010f390 msgr2=0x7fb7501a0df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:05:58.910 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:58.907+0000 7fb755e00640 1 --2- 172.21.15.12:0/3495217849 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb75010f390 0x7fb7501a0df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:05:58.910 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:58.907+0000 7fb755e00640 1 -- 172.21.15.12:0/3495217849 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb750075e00 msgr2=0x7fb75019ba50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:05:58.911 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:58.907+0000 7fb755e00640 1 --2- 172.21.15.12:0/3495217849 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb750075e00 0x7fb75019ba50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:05:58.911 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:58.907+0000 7fb755e00640 1 -- 172.21.15.12:0/3495217849 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7501a15c0 con 0x7fb7500754e0 2024-09-11T00:05:58.911 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:58.907+0000 7fb755e00640 1 --2- 172.21.15.12:0/3495217849 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb7500754e0 0x7fb75019b510 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fb74400e9f0 tx=0x7fb74400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:05:58.911 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:58.908+0000 7fb74aa00640 1 -- 172.21.15.12:0/3495217849 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb74400cd80 con 0x7fb7500754e0 2024-09-11T00:05:58.911 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:58.908+0000 7fb74aa00640 1 -- 172.21.15.12:0/3495217849 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb7440108e0 con 0x7fb7500754e0 2024-09-11T00:05:58.911 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:58.908+0000 7fb74aa00640 1 -- 172.21.15.12:0/3495217849 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb744018980 con 0x7fb7500754e0 2024-09-11T00:05:58.911 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:58.908+0000 7fb758800640 1 -- 172.21.15.12:0/3495217849 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb7501a18b0 con 0x7fb7500754e0 2024-09-11T00:05:58.911 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:58.908+0000 7fb758800640 1 -- 172.21.15.12:0/3495217849 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7501a8140 con 0x7fb7500754e0 2024-09-11T00:05:58.912 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:58.909+0000 7fb758800640 1 -- 172.21.15.12:0/3495217849 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb7500769f0 con 0x7fb7500754e0 2024-09-11T00:05:58.912 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:58.911+0000 7fb74aa00640 1 -- 172.21.15.12:0/3495217849 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 17) ==== 153301+0+0 (secure 0 0 0) 0x7fb744010430 con 0x7fb7500754e0 2024-09-11T00:05:58.914 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:58.913+0000 7fb74aa00640 1 --2- 172.21.15.12:0/3495217849 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fb7380b53f0 0x7fb7380b78b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:05:58.914 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:58.913+0000 7fb74aa00640 1 -- 172.21.15.12:0/3495217849 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 9092+0+0 (secure 0 0 0) 0x7fb7440e64a0 con 0x7fb7500754e0 2024-09-11T00:05:58.914 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:58.913+0000 7fb755400640 1 --2- 172.21.15.12:0/3495217849 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fb7380b53f0 0x7fb7380b78b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:05:58.914 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:58.914+0000 7fb755400640 1 --2- 172.21.15.12:0/3495217849 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fb7380b53f0 0x7fb7380b78b0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fb740005ab0 tx=0x7fb740005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:05:58.917 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:58.916+0000 7fb74aa00640 1 -- 172.21.15.12:0/3495217849 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7fb7440ace40 con 0x7fb7500754e0 2024-09-11T00:05:59.117 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:59.117+0000 7fb758800640 1 -- 172.21.15.12:0/3495217849 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb7501a83a0 con 0x7fb7500754e0 2024-09-11T00:05:59.119 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:59.118+0000 7fb74aa00640 1 -- 172.21.15.12:0/3495217849 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v83) ==== 74+0+24762 (secure 0 0 0) 0x7fb7440ac740 con 0x7fb7500754e0 2024-09-11T00:05:59.119 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-11T00:05:59.120 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":83,"fsid":"ff3ad74a-6fcf-11ef-bcea-c7b262605968","created":"2024-09-10T23:55:16.317643+0000","modified":"2024-09-11T00:05:19.832883+0000","last_up_change":"2024-09-11T00:03:08.959256+0000","last_in_change":"2024-09-11T00:02:46.564631+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T23:59:10.488728+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-11T00:04:00.065618+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"74","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-11T00:04:02.689888+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"74","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-11T00:04:09.862120+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"6382c200-a912-42a1-958b-79191b21249b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6802","nonce":1237101991},{"type":"v1","addr":"172.21.15.12:6803","nonce":1237101991}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6804","nonce":1237101991},{"type":"v1","addr":"172.21.15.12:6805","nonce":1237101991}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6808","nonce":1237101991},{"type":"v1","addr":"172.21.15.12:6809","nonce":1237101991}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6806","nonce":1237101991},{"type":"v1","addr":"172.21.15.12:6807","nonce":1237101991}]},"public_addr":"172.21.15.12:6803/1237101991","cluster_addr":"172.21.15.12:6805/1237101991","heartbeat_back_addr":"172.21.15.12:6809/1237101991","heartbeat_front_addr":"172.21.15.12:6807/1237101991","state":["exists","up"]},{"osd":1,"uuid":"fe0e1d0a-bdc5-4859-aa84-9038059a45f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6800","nonce":545848456},{"type":"v1","addr":"172.21.15.165:6801","nonce":545848456}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6802","nonce":545848456},{"type":"v1","addr":"172.21.15.165:6803","nonce":545848456}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6806","nonce":545848456},{"type":"v1","addr":"172.21.15.165:6807","nonce":545848456}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6804","nonce":545848456},{"type":"v1","addr":"172.21.15.165:6805","nonce":545848456}]},"public_addr":"172.21.15.165:6801/545848456","cluster_addr":"172.21.15.165:6803/545848456","heartbeat_back_addr":"172.21.15.165:6807/545848456","heartbeat_front_addr":"172.21.15.165:6805/545848456","state":["exists","up"]},{"osd":2,"uuid":"94f640dc-5f78-4e2c-87d9-03e96a6e7ba0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6800","nonce":4131162695},{"type":"v1","addr":"172.21.15.178:6801","nonce":4131162695}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":4131162695},{"type":"v1","addr":"172.21.15.178:6803","nonce":4131162695}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":4131162695},{"type":"v1","addr":"172.21.15.178:6807","nonce":4131162695}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":4131162695},{"type":"v1","addr":"172.21.15.178:6805","nonce":4131162695}]},"public_addr":"172.21.15.178:6801/4131162695","cluster_addr":"172.21.15.178:6803/4131162695","heartbeat_back_addr":"172.21.15.178:6807/4131162695","heartbeat_front_addr":"172.21.15.178:6805/4131162695","state":["exists","up"]},{"osd":3,"uuid":"af2f64d6-c9be-44a9-b041-8d599a7c080d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6810","nonce":452464245},{"type":"v1","addr":"172.21.15.12:6811","nonce":452464245}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6812","nonce":452464245},{"type":"v1","addr":"172.21.15.12:6813","nonce":452464245}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6816","nonce":452464245},{"type":"v1","addr":"172.21.15.12:6817","nonce":452464245}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6814","nonce":452464245},{"type":"v1","addr":"172.21.15.12:6815","nonce":452464245}]},"public_addr":"172.21.15.12:6811/452464245","cluster_addr":"172.21.15.12:6813/452464245","heartbeat_back_addr":"172.21.15.12:6817/452464245","heartbeat_front_addr":"172.21.15.12:6815/452464245","state":["exists","up"]},{"osd":4,"uuid":"ad76ac70-20c0-44ae-9866-5f9cfce2334c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6808","nonce":1899879280},{"type":"v1","addr":"172.21.15.165:6809","nonce":1899879280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6810","nonce":1899879280},{"type":"v1","addr":"172.21.15.165:6811","nonce":1899879280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6814","nonce":1899879280},{"type":"v1","addr":"172.21.15.165:6815","nonce":1899879280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6812","nonce":1899879280},{"type":"v1","addr":"172.21.15.165:6813","nonce":1899879280}]},"public_addr":"172.21.15.165:6809/1899879280","cluster_addr":"172.21.15.165:6811/1899879280","heartbeat_back_addr":"172.21.15.165:6815/1899879280","heartbeat_front_addr":"172.21.15.165:6813/1899879280","state":["exists","up"]},{"osd":5,"uuid":"aafda2cc-62fe-4fcb-9a96-8732e0b501d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":3042228555},{"type":"v1","addr":"172.21.15.178:6809","nonce":3042228555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":3042228555},{"type":"v1","addr":"172.21.15.178:6811","nonce":3042228555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6814","nonce":3042228555},{"type":"v1","addr":"172.21.15.178:6815","nonce":3042228555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":3042228555},{"type":"v1","addr":"172.21.15.178:6813","nonce":3042228555}]},"public_addr":"172.21.15.178:6809/3042228555","cluster_addr":"172.21.15.178:6811/3042228555","heartbeat_back_addr":"172.21.15.178:6815/3042228555","heartbeat_front_addr":"172.21.15.178:6813/3042228555","state":["exists","up"]},{"osd":6,"uuid":"375eca52-efe3-42a5-bbb1-d7d392d71f15","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6818","nonce":3868322467},{"type":"v1","addr":"172.21.15.12:6819","nonce":3868322467}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6820","nonce":3868322467},{"type":"v1","addr":"172.21.15.12:6821","nonce":3868322467}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6824","nonce":3868322467},{"type":"v1","addr":"172.21.15.12:6825","nonce":3868322467}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6822","nonce":3868322467},{"type":"v1","addr":"172.21.15.12:6823","nonce":3868322467}]},"public_addr":"172.21.15.12:6819/3868322467","cluster_addr":"172.21.15.12:6821/3868322467","heartbeat_back_addr":"172.21.15.12:6825/3868322467","heartbeat_front_addr":"172.21.15.12:6823/3868322467","state":["exists","up"]},{"osd":7,"uuid":"0fd619f5-44c3-4cf7-ae7a-388dd798de36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6816","nonce":1143773735},{"type":"v1","addr":"172.21.15.165:6817","nonce":1143773735}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6818","nonce":1143773735},{"type":"v1","addr":"172.21.15.165:6819","nonce":1143773735}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6822","nonce":1143773735},{"type":"v1","addr":"172.21.15.165:6823","nonce":1143773735}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6820","nonce":1143773735},{"type":"v1","addr":"172.21.15.165:6821","nonce":1143773735}]},"public_addr":"172.21.15.165:6817/1143773735","cluster_addr":"172.21.15.165:6819/1143773735","heartbeat_back_addr":"172.21.15.165:6823/1143773735","heartbeat_front_addr":"172.21.15.165:6821/1143773735","state":["exists","up"]},{"osd":8,"uuid":"b03d14ce-8427-462a-9bcc-27683c549f53","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":2891445058},{"type":"v1","addr":"172.21.15.178:6817","nonce":2891445058}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6818","nonce":2891445058},{"type":"v1","addr":"172.21.15.178:6819","nonce":2891445058}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6822","nonce":2891445058},{"type":"v1","addr":"172.21.15.178:6823","nonce":2891445058}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":2891445058},{"type":"v1","addr":"172.21.15.178:6821","nonce":2891445058}]},"public_addr":"172.21.15.178:6817/2891445058","cluster_addr":"172.21.15.178:6819/2891445058","heartbeat_back_addr":"172.21.15.178:6823/2891445058","heartbeat_front_addr":"172.21.15.178:6821/2891445058","state":["exists","up"]},{"osd":9,"uuid":"81a6cf69-2a24-4a6c-9adf-6b7df38cfacb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6826","nonce":3359634903},{"type":"v1","addr":"172.21.15.12:6827","nonce":3359634903}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6828","nonce":3359634903},{"type":"v1","addr":"172.21.15.12:6829","nonce":3359634903}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6832","nonce":3359634903},{"type":"v1","addr":"172.21.15.12:6833","nonce":3359634903}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6830","nonce":3359634903},{"type":"v1","addr":"172.21.15.12:6831","nonce":3359634903}]},"public_addr":"172.21.15.12:6827/3359634903","cluster_addr":"172.21.15.12:6829/3359634903","heartbeat_back_addr":"172.21.15.12:6833/3359634903","heartbeat_front_addr":"172.21.15.12:6831/3359634903","state":["exists","up"]},{"osd":10,"uuid":"c1ad1f0a-33f4-4c32-91f6-4d030a29f1dd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6824","nonce":3642325778},{"type":"v1","addr":"172.21.15.165:6825","nonce":3642325778}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6826","nonce":3642325778},{"type":"v1","addr":"172.21.15.165:6827","nonce":3642325778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6830","nonce":3642325778},{"type":"v1","addr":"172.21.15.165:6831","nonce":3642325778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6828","nonce":3642325778},{"type":"v1","addr":"172.21.15.165:6829","nonce":3642325778}]},"public_addr":"172.21.15.165:6825/3642325778","cluster_addr":"172.21.15.165:6827/3642325778","heartbeat_back_addr":"172.21.15.165:6831/3642325778","heartbeat_front_addr":"172.21.15.165:6829/3642325778","state":["exists","up"]},{"osd":11,"uuid":"8413c9b0-680b-4d02-aeab-b3b3ff16c3d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":2937059756},{"type":"v1","addr":"172.21.15.178:6825","nonce":2937059756}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6826","nonce":2937059756},{"type":"v1","addr":"172.21.15.178:6827","nonce":2937059756}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":2937059756},{"type":"v1","addr":"172.21.15.178:6831","nonce":2937059756}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":2937059756},{"type":"v1","addr":"172.21.15.178:6829","nonce":2937059756}]},"public_addr":"172.21.15.178:6825/2937059756","cluster_addr":"172.21.15.178:6827/2937059756","heartbeat_back_addr":"172.21.15.178:6831/2937059756","heartbeat_front_addr":"172.21.15.178:6829/2937059756","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:58:16.036348+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:58:41.483150+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:59:07.107283+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:59:35.706574+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T00:00:02.555720+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T00:00:28.092847+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T00:00:57.396495+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T00:01:24.216775+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T00:01:48.807850+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T00:02:16.838565+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T00:02:43.131736+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T00:03:07.353155+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":8,"to":4}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"3.e","mappings":[{"from":5,"to":9}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9}]},{"pgid":"3.23","mappings":[{"from":4,"to":7}]},{"pgid":"3.31","mappings":[{"from":4,"to":8}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"3.39","mappings":[{"from":5,"to":6}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.12:6801/4177251511":"2024-09-11T23:56:17.999820+0000","172.21.15.12:6800/4177251511":"2024-09-11T23:56:17.999820+0000","172.21.15.12:0/3273692817":"2024-09-11T23:56:17.999820+0000","172.21.15.12:0/4017778158":"2024-09-11T23:56:17.999820+0000","172.21.15.12:0/2499921051":"2024-09-11T23:55:50.302721+0000","172.21.15.12:6801/760140910":"2024-09-11T23:55:50.302721+0000","172.21.15.12:6800/760140910":"2024-09-11T23:55:50.302721+0000","172.21.15.12:0/2810703302":"2024-09-11T23:56:17.999820+0000","172.21.15.12:0/2325436065":"2024-09-11T23:55:50.302721+0000","172.21.15.12:0/3137053479":"2024-09-11T23:55:50.302721+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-11T00:05:59.122 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:59.121+0000 7fb758800640 1 -- 172.21.15.12:0/3495217849 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fb7380b53f0 msgr2=0x7fb7380b78b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:05:59.122 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:59.121+0000 7fb758800640 1 --2- 172.21.15.12:0/3495217849 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fb7380b53f0 0x7fb7380b78b0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fb740005ab0 tx=0x7fb740005a20 comp rx=0 tx=0).stop 2024-09-11T00:05:59.122 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:59.121+0000 7fb758800640 1 -- 172.21.15.12:0/3495217849 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb7500754e0 msgr2=0x7fb75019b510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:05:59.122 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:59.121+0000 7fb758800640 1 --2- 172.21.15.12:0/3495217849 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb7500754e0 0x7fb75019b510 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fb74400e9f0 tx=0x7fb74400eeb0 comp rx=0 tx=0).stop 2024-09-11T00:05:59.122 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:59.121+0000 7fb758800640 1 -- 172.21.15.12:0/3495217849 shutdown_connections 2024-09-11T00:05:59.122 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:59.121+0000 7fb758800640 1 --2- 172.21.15.12:0/3495217849 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fb7380b53f0 0x7fb7380b78b0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:05:59.122 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:59.121+0000 7fb758800640 1 --2- 172.21.15.12:0/3495217849 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb75010f390 0x7fb7501a0df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:05:59.122 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:59.121+0000 7fb758800640 1 --2- 172.21.15.12:0/3495217849 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb750075e00 0x7fb75019ba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:05:59.122 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:59.121+0000 7fb758800640 1 --2- 172.21.15.12:0/3495217849 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb7500754e0 0x7fb75019b510 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:05:59.122 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:59.121+0000 7fb758800640 1 -- 172.21.15.12:0/3495217849 >> 172.21.15.12:0/3495217849 conn(0x7fb7500fe810 msgr2=0x7fb750111550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:05:59.123 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:59.121+0000 7fb758800640 1 -- 172.21.15.12:0/3495217849 shutdown_connections 2024-09-11T00:05:59.123 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:05:59.121+0000 7fb758800640 1 -- 172.21.15.12:0/3495217849 wait complete. 2024-09-11T00:05:59.514 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:05:59 smithi012 ceph-mon[30333]: pgmap v363: 129 pgs: 129 active+clean; 647 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-11T00:05:59.514 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:05:59 smithi012 ceph-mon[30333]: from='client.? 172.21.15.12:0/3495217849' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-11T00:05:59.532 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:05:59 smithi165 ceph-mon[35356]: pgmap v363: 129 pgs: 129 active+clean; 647 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-11T00:05:59.532 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:05:59 smithi165 ceph-mon[35356]: from='client.? 172.21.15.12:0/3495217849' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-11T00:05:59.580 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:05:59 smithi178 ceph-mon[35919]: pgmap v363: 129 pgs: 129 active+clean; 647 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-11T00:05:59.581 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:05:59 smithi178 ceph-mon[35919]: from='client.? 172.21.15.12:0/3495217849' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-11T00:05:59.926 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid ff3ad74a-6fcf-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T00:06:00.308 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:06:00.863 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:00.862+0000 7f55a1e00640 1 -- 172.21.15.12:0/3491479318 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f559c077ff0 msgr2=0x7f559c1132a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:06:00.864 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:00.862+0000 7f55a1e00640 1 --2- 172.21.15.12:0/3491479318 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f559c077ff0 0x7f559c1132a0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f5588009900 tx=0x7f558802a3f0 comp rx=0 tx=0).stop 2024-09-11T00:06:00.864 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:00.862+0000 7f55a1e00640 1 -- 172.21.15.12:0/3491479318 shutdown_connections 2024-09-11T00:06:00.864 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:00.862+0000 7f55a1e00640 1 --2- 172.21.15.12:0/3491479318 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f559c1137e0 0x7f559c115bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:00.864 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:00.862+0000 7f55a1e00640 1 --2- 172.21.15.12:0/3491479318 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f559c077ff0 0x7f559c1132a0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:00.864 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:00.862+0000 7f55a1e00640 1 --2- 172.21.15.12:0/3491479318 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f559c0776d0 0x7f559c077ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:00.864 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:00.862+0000 7f55a1e00640 1 -- 172.21.15.12:0/3491479318 >> 172.21.15.12:0/3491479318 conn(0x7f559c100840 msgr2=0x7f559c102c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:06:00.864 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:00.862+0000 7f55a1e00640 1 -- 172.21.15.12:0/3491479318 shutdown_connections 2024-09-11T00:06:00.864 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:00.862+0000 7f55a1e00640 1 -- 172.21.15.12:0/3491479318 wait complete. 2024-09-11T00:06:00.865 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:00.863+0000 7f55a1e00640 1 Processor -- start 2024-09-11T00:06:00.866 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:00.863+0000 7f55a1e00640 1 -- start start 2024-09-11T00:06:00.866 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:00.864+0000 7f55a1e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f559c0776d0 0x7f559c19f5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:06:00.866 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:00.864+0000 7f55a1e00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f559c077ff0 0x7f559c19fb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:06:00.866 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:00.864+0000 7f55a1e00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f559c1137e0 0x7f559c1a4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:06:00.866 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:00.864+0000 7f55a1e00640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f559c119220 con 0x7f559c077ff0 2024-09-11T00:06:00.866 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:00.864+0000 7f55a1e00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f559c1190a0 con 0x7f559c0776d0 2024-09-11T00:06:00.866 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:00.864+0000 7f55a1e00640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f559c1193a0 con 0x7f559c1137e0 2024-09-11T00:06:00.866 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:00.864+0000 7f559be00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f559c077ff0 0x7f559c19fb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:06:00.866 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:00.864+0000 7f559be00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f559c077ff0 0x7f559c19fb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:38588/0 (socket says 172.21.15.12:38588) 2024-09-11T00:06:00.866 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:00.864+0000 7f559be00640 1 -- 172.21.15.12:0/1461588200 learned_addr learned my addr 172.21.15.12:0/1461588200 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:06:00.866 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:00.864+0000 7f55a0a00640 1 --2- 172.21.15.12:0/1461588200 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f559c0776d0 0x7f559c19f5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:06:00.867 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:00.864+0000 7f55a1400640 1 --2- 172.21.15.12:0/1461588200 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f559c1137e0 0x7f559c1a4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:06:00.867 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:00.864+0000 7f559be00640 1 -- 172.21.15.12:0/1461588200 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f559c1137e0 msgr2=0x7f559c1a4ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:06:00.867 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:00.864+0000 7f559be00640 1 --2- 172.21.15.12:0/1461588200 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f559c1137e0 0x7f559c1a4ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:00.867 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:00.864+0000 7f559be00640 1 -- 172.21.15.12:0/1461588200 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f559c0776d0 msgr2=0x7f559c19f5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:06:00.867 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:00.864+0000 7f559be00640 1 --2- 172.21.15.12:0/1461588200 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f559c0776d0 0x7f559c19f5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:00.867 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:00.864+0000 7f559be00640 1 -- 172.21.15.12:0/1461588200 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f559c1a56a0 con 0x7f559c077ff0 2024-09-11T00:06:00.867 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:00.865+0000 7f559be00640 1 --2- 172.21.15.12:0/1461588200 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f559c077ff0 0x7f559c19fb30 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f5588005ab0 tx=0x7f5588003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:06:00.868 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:00.865+0000 7f5599600640 1 -- 172.21.15.12:0/1461588200 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f558803f070 con 0x7f559c077ff0 2024-09-11T00:06:00.868 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:00.865+0000 7f5599600640 1 -- 172.21.15.12:0/1461588200 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5588031b60 con 0x7f559c077ff0 2024-09-11T00:06:00.868 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:00.865+0000 7f5599600640 1 -- 172.21.15.12:0/1461588200 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5588035030 con 0x7f559c077ff0 2024-09-11T00:06:00.868 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:00.865+0000 7f55a1e00640 1 -- 172.21.15.12:0/1461588200 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f559c1a5930 con 0x7f559c077ff0 2024-09-11T00:06:00.868 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:00.865+0000 7f55a1e00640 1 -- 172.21.15.12:0/1461588200 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f559c1ac050 con 0x7f559c077ff0 2024-09-11T00:06:00.868 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:00.866+0000 7f55a1e00640 1 -- 172.21.15.12:0/1461588200 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f559c077ab0 con 0x7f559c077ff0 2024-09-11T00:06:00.869 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:00.869+0000 7f5599600640 1 -- 172.21.15.12:0/1461588200 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 17) ==== 153301+0+0 (secure 0 0 0) 0x7f558803b460 con 0x7f559c077ff0 2024-09-11T00:06:00.871 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:00.871+0000 7f5599600640 1 --2- 172.21.15.12:0/1461588200 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f55840b5490 0x7f55840b7950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:06:00.872 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:00.871+0000 7f5599600640 1 -- 172.21.15.12:0/1461588200 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 9092+0+0 (secure 0 0 0) 0x7f5588107030 con 0x7f559c077ff0 2024-09-11T00:06:00.872 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:00.871+0000 7f55a0a00640 1 --2- 172.21.15.12:0/1461588200 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f55840b5490 0x7f55840b7950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:06:00.872 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:00.872+0000 7f55a0a00640 1 --2- 172.21.15.12:0/1461588200 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f55840b5490 0x7f55840b7950 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f5590009bd0 tx=0x7f5590005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:06:00.873 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:00.872+0000 7f5599600640 1 -- 172.21.15.12:0/1461588200 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f55880cd3e0 con 0x7f559c077ff0 2024-09-11T00:06:01.106 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:01.105+0000 7f55a1e00640 1 -- 172.21.15.12:0/1461588200 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f559c1ac2b0 con 0x7f559c077ff0 2024-09-11T00:06:01.106 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:01.106+0000 7f5599600640 1 -- 172.21.15.12:0/1461588200 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11965 (secure 0 0 0) 0x7f55880d61e0 con 0x7f559c077ff0 2024-09-11T00:06:01.107 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-11T00:06:01.107 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":28,"btime":"2024-09-11T00:04:50:247323+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14577,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6835/447171591","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":447171591},{"type":"v1","addr":"172.21.15.12:6835","nonce":447171591}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14589,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/901486040","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":901486040},{"type":"v1","addr":"172.21.15.12:6837","nonce":901486040}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T00:04:04.739881+0000","modified":"2024-09-11T00:04:50.247301+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24674,"mds_1":24643,"mds_2":24638,"mds_3":24637,"mds_4":24614},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14631":{"gid":14631,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.12:6841/3069919437","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6840","nonce":3069919437},{"type":"v1","addr":"172.21.15.12:6841","nonce":3069919437}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24562":{"gid":24562,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6833/1141636112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":1141636112},{"type":"v1","addr":"172.21.15.178:6833","nonce":1141636112}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24581":{"gid":24581,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6833/3111980970","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6832","nonce":3111980970},{"type":"v1","addr":"172.21.15.165:6833","nonce":3111980970}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24607":{"gid":24607,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6835/4084868287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":4084868287},{"type":"v1","addr":"172.21.15.178:6835","nonce":4084868287}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24613":{"gid":24613,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6837/526874763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":526874763},{"type":"v1","addr":"172.21.15.178:6837","nonce":526874763}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24614":{"gid":24614,"name":"e","rank":4,"incarnation":26,"state":"up:active","state_seq":21,"addr":"172.21.15.165:6835/2425395274","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":2425395274},{"type":"v1","addr":"172.21.15.165:6835","nonce":2425395274}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24637":{"gid":24637,"name":"i","rank":3,"incarnation":24,"state":"up:active","state_seq":18,"addr":"172.21.15.178:6839/70628097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6838","nonce":70628097},{"type":"v1","addr":"172.21.15.178:6839","nonce":70628097}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24638":{"gid":24638,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":18,"addr":"172.21.15.165:6837/6215088","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6836","nonce":6215088},{"type":"v1","addr":"172.21.15.165:6837","nonce":6215088}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24643":{"gid":24643,"name":"g","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.12:6839/3124093009","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6838","nonce":3124093009},{"type":"v1","addr":"172.21.15.12:6839","nonce":3124093009}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24674":{"gid":24674,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.165:6839/501312984","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6838","nonce":501312984},{"type":"v1","addr":"172.21.15.165:6839","nonce":501312984}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24674,"qdb_cluster":[24674,24643,24638,24637,24614]},"id":1}]} 2024-09-11T00:06:01.107 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 28 2024-09-11T00:06:01.109 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:01.108+0000 7f55a1e00640 1 -- 172.21.15.12:0/1461588200 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f55840b5490 msgr2=0x7f55840b7950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:06:01.109 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:01.108+0000 7f55a1e00640 1 --2- 172.21.15.12:0/1461588200 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f55840b5490 0x7f55840b7950 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f5590009bd0 tx=0x7f5590005f90 comp rx=0 tx=0).stop 2024-09-11T00:06:01.109 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:01.108+0000 7f55a1e00640 1 -- 172.21.15.12:0/1461588200 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f559c077ff0 msgr2=0x7f559c19fb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:06:01.109 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:01.108+0000 7f55a1e00640 1 --2- 172.21.15.12:0/1461588200 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f559c077ff0 0x7f559c19fb30 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f5588005ab0 tx=0x7f5588003c10 comp rx=0 tx=0).stop 2024-09-11T00:06:01.109 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:01.108+0000 7f55a1e00640 1 -- 172.21.15.12:0/1461588200 shutdown_connections 2024-09-11T00:06:01.109 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:01.108+0000 7f55a1e00640 1 --2- 172.21.15.12:0/1461588200 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f55840b5490 0x7f55840b7950 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:01.109 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:01.108+0000 7f55a1e00640 1 --2- 172.21.15.12:0/1461588200 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f559c1137e0 0x7f559c1a4ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:01.109 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:01.108+0000 7f55a1e00640 1 --2- 172.21.15.12:0/1461588200 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f559c077ff0 0x7f559c19fb30 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:01.110 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:01.108+0000 7f55a1e00640 1 --2- 172.21.15.12:0/1461588200 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f559c0776d0 0x7f559c19f5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:01.110 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:01.108+0000 7f55a1e00640 1 -- 172.21.15.12:0/1461588200 >> 172.21.15.12:0/1461588200 conn(0x7f559c100840 msgr2=0x7f559c102c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:06:01.110 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:01.108+0000 7f55a1e00640 1 -- 172.21.15.12:0/1461588200 shutdown_connections 2024-09-11T00:06:01.110 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:01.108+0000 7f55a1e00640 1 -- 172.21.15.12:0/1461588200 wait complete. 2024-09-11T00:06:01.511 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:06:01 smithi012 ceph-mon[30333]: pgmap v364: 129 pgs: 129 active+clean; 647 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-11T00:06:01.511 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:06:01 smithi012 ceph-mon[30333]: from='client.? 172.21.15.12:0/1461588200' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T00:06:01.532 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:06:01 smithi165 ceph-mon[35356]: pgmap v364: 129 pgs: 129 active+clean; 647 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-11T00:06:01.532 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:06:01 smithi165 ceph-mon[35356]: from='client.? 172.21.15.12:0/1461588200' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T00:06:01.581 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:06:01 smithi178 ceph-mon[35919]: pgmap v364: 129 pgs: 129 active+clean; 647 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-11T00:06:01.581 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:06:01 smithi178 ceph-mon[35919]: from='client.? 172.21.15.12:0/1461588200' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T00:06:01.879 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 28, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-09-11T00:04:04.739881+0000', 'modified': '2024-09-11T00:04:50.247301+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 24674, 'mds_1': 24643, 'mds_2': 24638, 'mds_3': 24637, 'mds_4': 24614}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_14631': {'gid': 14631, 'name': 'a', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.12:6841/3069919437', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.12:6840', 'nonce': 3069919437}, {'type': 'v1', 'addr': '172.21.15.12:6841', 'nonce': 3069919437}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24562': {'gid': 24562, 'name': 'l', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.178:6833/1141636112', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.178:6832', 'nonce': 1141636112}, {'type': 'v1', 'addr': '172.21.15.178:6833', 'nonce': 1141636112}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24581': {'gid': 24581, 'name': 'k', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.165:6833/3111980970', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.165:6832', 'nonce': 3111980970}, {'type': 'v1', 'addr': '172.21.15.165:6833', 'nonce': 3111980970}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24607': {'gid': 24607, 'name': 'f', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.178:6835/4084868287', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.178:6834', 'nonce': 4084868287}, {'type': 'v1', 'addr': '172.21.15.178:6835', 'nonce': 4084868287}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24613': {'gid': 24613, 'name': 'c', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.178:6837/526874763', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.178:6836', 'nonce': 526874763}, {'type': 'v1', 'addr': '172.21.15.178:6837', 'nonce': 526874763}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24614': {'gid': 24614, 'name': 'e', 'rank': 4, 'incarnation': 26, 'state': 'up:active', 'state_seq': 21, 'addr': '172.21.15.165:6835/2425395274', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.165:6834', 'nonce': 2425395274}, {'type': 'v1', 'addr': '172.21.15.165:6835', 'nonce': 2425395274}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24637': {'gid': 24637, 'name': 'i', 'rank': 3, 'incarnation': 24, 'state': 'up:active', 'state_seq': 18, 'addr': '172.21.15.178:6839/70628097', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.178:6838', 'nonce': 70628097}, {'type': 'v1', 'addr': '172.21.15.178:6839', 'nonce': 70628097}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24638': {'gid': 24638, 'name': 'h', 'rank': 2, 'incarnation': 22, 'state': 'up:active', 'state_seq': 18, 'addr': '172.21.15.165:6837/6215088', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.165:6836', 'nonce': 6215088}, {'type': 'v1', 'addr': '172.21.15.165:6837', 'nonce': 6215088}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24643': {'gid': 24643, 'name': 'g', 'rank': 1, 'incarnation': 20, 'state': 'up:active', 'state_seq': 17, 'addr': '172.21.15.12:6839/3124093009', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.12:6838', 'nonce': 3124093009}, {'type': 'v1', 'addr': '172.21.15.12:6839', 'nonce': 3124093009}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24674': {'gid': 24674, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.165:6839/501312984', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.165:6838', 'nonce': 501312984}, {'type': 'v1', 'addr': '172.21.15.165:6839', 'nonce': 501312984}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24674, 'qdb_cluster': [24674, 24643, 24638, 24637, 24614]} 2024-09-11T00:06:01.879 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2024-09-11T00:06:01.879 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid ff3ad74a-6fcf-11ef-bcea-c7b262605968 -- ceph tell mds.e status 2024-09-11T00:06:02.263 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:06:02.780 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:02.779+0000 7f4684000640 1 -- 172.21.15.12:0/3897961349 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f467c1087b0 msgr2=0x7f467c108bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:06:02.780 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:02.779+0000 7f4684000640 1 --2- 172.21.15.12:0/3897961349 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f467c1087b0 0x7f467c108bf0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f4668009900 tx=0x7f466802a410 comp rx=0 tx=0).stop 2024-09-11T00:06:02.780 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:02.779+0000 7f4684000640 1 -- 172.21.15.12:0/3897961349 shutdown_connections 2024-09-11T00:06:02.780 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:02.779+0000 7f4684000640 1 --2- 172.21.15.12:0/3897961349 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f467c109130 0x7f467c10f930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:02.780 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:02.779+0000 7f4684000640 1 --2- 172.21.15.12:0/3897961349 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f467c1087b0 0x7f467c108bf0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:02.780 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:02.779+0000 7f4684000640 1 --2- 172.21.15.12:0/3897961349 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f467c102da0 0x7f467c103180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:02.780 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:02.779+0000 7f4684000640 1 -- 172.21.15.12:0/3897961349 >> 172.21.15.12:0/3897961349 conn(0x7f467c0fe810 msgr2=0x7f467c100c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:06:02.781 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:02.779+0000 7f4684000640 1 -- 172.21.15.12:0/3897961349 shutdown_connections 2024-09-11T00:06:02.781 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:02.779+0000 7f4684000640 1 -- 172.21.15.12:0/3897961349 wait complete. 2024-09-11T00:06:02.782 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:02.780+0000 7f4684000640 1 Processor -- start 2024-09-11T00:06:02.782 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:02.780+0000 7f4684000640 1 -- start start 2024-09-11T00:06:02.782 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:02.780+0000 7f4684000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f467c102da0 0x7f467c0754a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:06:02.782 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:02.780+0000 7f4684000640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f467c1087b0 0x7f467c0759e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:06:02.782 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:02.780+0000 7f4684000640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f467c109130 0x7f467c075f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:06:02.782 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:02.780+0000 7f4684000640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f467c112e40 con 0x7f467c109130 2024-09-11T00:06:02.782 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:02.780+0000 7f4684000640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f467c112cc0 con 0x7f467c102da0 2024-09-11T00:06:02.782 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:02.780+0000 7f4684000640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f467c112fc0 con 0x7f467c1087b0 2024-09-11T00:06:02.782 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:02.780+0000 7f4681600640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f467c102da0 0x7f467c0754a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:06:02.783 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:02.780+0000 7f4682000640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f467c109130 0x7f467c075f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:06:02.783 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:02.780+0000 7f4681600640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f467c102da0 0x7f467c0754a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.12:53476/0 (socket says 172.21.15.12:53476) 2024-09-11T00:06:02.783 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:02.780+0000 7f4681600640 1 -- 172.21.15.12:0/1257787172 learned_addr learned my addr 172.21.15.12:0/1257787172 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:06:02.783 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:02.781+0000 7f4680c00640 1 --2- 172.21.15.12:0/1257787172 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f467c1087b0 0x7f467c0759e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:06:02.783 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:02.781+0000 7f4681600640 1 -- 172.21.15.12:0/1257787172 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f467c1087b0 msgr2=0x7f467c0759e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:06:02.783 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:02.781+0000 7f4681600640 1 --2- 172.21.15.12:0/1257787172 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f467c1087b0 0x7f467c0759e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:02.783 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:02.781+0000 7f4681600640 1 -- 172.21.15.12:0/1257787172 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f467c109130 msgr2=0x7f467c075f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:06:02.783 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:02.781+0000 7f4681600640 1 --2- 172.21.15.12:0/1257787172 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f467c109130 0x7f467c075f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:02.783 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:02.781+0000 7f4681600640 1 -- 172.21.15.12:0/1257787172 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f467c1a76f0 con 0x7f467c102da0 2024-09-11T00:06:02.783 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:02.781+0000 7f4682000640 1 --2- 172.21.15.12:0/1257787172 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f467c109130 0x7f467c075f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:06:02.783 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:02.781+0000 7f4681600640 1 --2- 172.21.15.12:0/1257787172 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f467c102da0 0x7f467c0754a0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f467000e9f0 tx=0x7f467000edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:06:02.784 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:02.782+0000 7f467a000640 1 -- 172.21.15.12:0/1257787172 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f467000cd80 con 0x7f467c102da0 2024-09-11T00:06:02.784 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:02.782+0000 7f4684000640 1 -- 172.21.15.12:0/1257787172 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f467c1a79e0 con 0x7f467c102da0 2024-09-11T00:06:02.784 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:02.782+0000 7f467a000640 1 -- 172.21.15.12:0/1257787172 <== mon.2 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f46700054d0 con 0x7f467c102da0 2024-09-11T00:06:02.784 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:02.782+0000 7f467a000640 1 -- 172.21.15.12:0/1257787172 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4670010690 con 0x7f467c102da0 2024-09-11T00:06:02.784 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:02.782+0000 7f4684000640 1 -- 172.21.15.12:0/1257787172 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f467c1a7ef0 con 0x7f467c102da0 2024-09-11T00:06:02.785 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:02.785+0000 7f4686c2d740 1 Processor -- start 2024-09-11T00:06:02.786 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:02.785+0000 7f4686c2d740 1 -- start start 2024-09-11T00:06:02.786 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:02.785+0000 7f4686c2d740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563975600240 0x563975600620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:06:02.786 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:02.785+0000 7f4686c2d740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563975600b60 0x56397560c5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:06:02.787 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:02.785+0000 7f4686c2d740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56397560caf0 0x56397560eee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:06:02.787 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:02.785+0000 7f4686c2d740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x563975577d50 con 0x563975600b60 2024-09-11T00:06:02.787 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:02.785+0000 7f4686c2d740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x563975567180 con 0x56397560caf0 2024-09-11T00:06:02.787 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:02.785+0000 7f4686c2d740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x56397556af90 con 0x563975600240 2024-09-11T00:06:02.787 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:02.786+0000 7f4680c00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56397560caf0 0x56397560eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:06:02.787 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:02.786+0000 7f4682000640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563975600240 0x563975600620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:06:02.787 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:02.786+0000 7f4680c00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56397560caf0 0x56397560eee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.12:53488/0 (socket says 172.21.15.12:53488) 2024-09-11T00:06:02.787 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:02.786+0000 7f4680c00640 1 -- 172.21.15.12:0/4129136305 learned_addr learned my addr 172.21.15.12:0/4129136305 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:06:02.787 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:02.786+0000 7f4682000640 1 --2- 172.21.15.12:0/4129136305 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563975600240 0x563975600620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.12:49228/0 (socket says 172.21.15.12:49228) 2024-09-11T00:06:02.789 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:02.786+0000 7f4682000640 1 -- 172.21.15.12:0/4129136305 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56397560caf0 msgr2=0x56397560eee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:06:02.789 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:02.786+0000 7f4682000640 1 --2- 172.21.15.12:0/4129136305 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56397560caf0 0x56397560eee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:02.789 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:02.786+0000 7f4682000640 1 -- 172.21.15.12:0/4129136305 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563975600b60 msgr2=0x56397560c5b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-11T00:06:02.789 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:02.786+0000 7f4682000640 1 --2- 172.21.15.12:0/4129136305 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563975600b60 0x56397560c5b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:02.789 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:02.786+0000 7f4682000640 1 -- 172.21.15.12:0/4129136305 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563975531a70 con 0x563975600240 2024-09-11T00:06:02.789 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:02.786+0000 7f4680c00640 1 --2- 172.21.15.12:0/4129136305 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56397560caf0 0x56397560eee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:06:02.789 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:02.786+0000 7f467a000640 1 -- 172.21.15.12:0/1257787172 <== mon.2 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 17) ==== 153301+0+0 (secure 0 0 0) 0x7f4670004e90 con 0x7f467c102da0 2024-09-11T00:06:02.789 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:02.786+0000 7f4682000640 1 --2- 172.21.15.12:0/4129136305 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563975600240 0x563975600620 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f4674005fd0 tx=0x7f4674011bf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ab59758e4f0661ff server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:06:02.790 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:02.787+0000 7f466e600640 1 -- 172.21.15.12:0/4129136305 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4674004d20 con 0x563975600240 2024-09-11T00:06:02.790 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:02.787+0000 7f466e600640 1 -- 172.21.15.12:0/4129136305 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4674003970 con 0x563975600240 2024-09-11T00:06:02.790 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:02.787+0000 7f466e600640 1 -- 172.21.15.12:0/4129136305 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46740042c0 con 0x563975600240 2024-09-11T00:06:02.790 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:02.787+0000 7f4686c2d740 1 -- 172.21.15.12:0/4129136305 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563975600240 msgr2=0x563975600620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:06:02.790 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:02.787+0000 7f4686c2d740 1 --2- 172.21.15.12:0/4129136305 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563975600240 0x563975600620 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f4674005fd0 tx=0x7f4674011bf0 comp rx=0 tx=0).stop 2024-09-11T00:06:02.790 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:02.787+0000 7f4686c2d740 1 -- 172.21.15.12:0/4129136305 shutdown_connections 2024-09-11T00:06:02.790 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:02.787+0000 7f4686c2d740 1 --2- 172.21.15.12:0/4129136305 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56397560caf0 0x56397560eee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:02.790 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:02.787+0000 7f4686c2d740 1 --2- 172.21.15.12:0/4129136305 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563975600b60 0x56397560c5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:02.790 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:02.787+0000 7f4686c2d740 1 --2- 172.21.15.12:0/4129136305 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563975600240 0x563975600620 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:02.790 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:02.787+0000 7f4686c2d740 1 -- 172.21.15.12:0/4129136305 >> 172.21.15.12:0/4129136305 conn(0x5639755fe020 msgr2=0x5639755ff210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:06:02.790 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:02.788+0000 7f4686c2d740 1 -- 172.21.15.12:0/4129136305 shutdown_connections 2024-09-11T00:06:02.790 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:02.788+0000 7f4686c2d740 1 -- 172.21.15.12:0/4129136305 wait complete. 2024-09-11T00:06:02.790 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:02.788+0000 7f467a000640 1 --2- 172.21.15.12:0/1257787172 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f46640b5280 0x7f46640b7740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:06:02.791 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:02.788+0000 7f467a000640 1 -- 172.21.15.12:0/1257787172 <== mon.2 v2:172.21.15.165:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 9092+0+0 (secure 0 0 0) 0x7f46700e64a0 con 0x7f467c102da0 2024-09-11T00:06:02.791 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:02.788+0000 7f4680c00640 1 --2- 172.21.15.12:0/1257787172 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f46640b5280 0x7f46640b7740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:06:02.791 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:02.788+0000 7f4686c2d740 1 -- start start 2024-09-11T00:06:02.793 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:02.789+0000 7f4686c2d740 1 Processor -- start 2024-09-11T00:06:02.793 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:02.789+0000 7f4680c00640 1 --2- 172.21.15.12:0/1257787172 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f46640b5280 0x7f46640b7740 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f4668031440 tx=0x7f4668005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:06:02.793 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:02.789+0000 7f4686c2d740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563975600240 0x563975609b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:06:02.793 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:02.789+0000 7f4686c2d740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563975600b60 0x56397560a060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:06:02.793 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:02.789+0000 7f4686c2d740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56397560caf0 0x56397560a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:06:02.793 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:02.789+0000 7f4686c2d740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x5639753c85f0 con 0x56397560caf0 2024-09-11T00:06:02.793 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:02.789+0000 7f4686c2d740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x563975502460 con 0x563975600b60 2024-09-11T00:06:02.793 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:02.789+0000 7f4686c2d740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x56397560f840 con 0x563975600240 2024-09-11T00:06:02.794 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:02.789+0000 7f4686c2d740 10 client.0 _ll_register_callbacks cb 0x563975512210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:06:02.794 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:02.789+0000 7f4681600640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563975600240 0x563975609b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:06:02.794 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:02.789+0000 7f4680c00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563975600b60 0x56397560a060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:06:02.794 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:02.789+0000 7f4682000640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56397560caf0 0x56397560a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:06:02.794 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:02.789+0000 7f4680c00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563975600b60 0x56397560a060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.12:53490/0 (socket says 172.21.15.12:53490) 2024-09-11T00:06:02.794 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:02.789+0000 7f4680c00640 1 -- 172.21.15.12:0/175860469 learned_addr learned my addr 172.21.15.12:0/175860469 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:06:02.794 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:02.789+0000 7f4681600640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563975600240 0x563975609b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.12:49242/0 (socket says 172.21.15.12:49242) 2024-09-11T00:06:02.794 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:02.790+0000 7f4681600640 1 -- 172.21.15.12:0/175860469 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563975600b60 msgr2=0x56397560a060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:06:02.794 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:02.790+0000 7f4681600640 1 --2- 172.21.15.12:0/175860469 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563975600b60 0x56397560a060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:02.794 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:02.790+0000 7f4681600640 1 -- 172.21.15.12:0/175860469 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56397560caf0 msgr2=0x56397560a5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:06:02.794 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:02.790+0000 7f4681600640 1 --2- 172.21.15.12:0/175860469 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56397560caf0 0x56397560a5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:02.794 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:02.790+0000 7f4681600640 1 -- 172.21.15.12:0/175860469 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5639752c5540 con 0x563975600240 2024-09-11T00:06:02.794 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:02.790+0000 7f4682000640 1 --2- 172.21.15.12:0/175860469 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56397560caf0 0x56397560a5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:06:02.795 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:02.790+0000 7f4681600640 1 --2- 172.21.15.12:0/175860469 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563975600240 0x563975609b20 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f46700e3af0 tx=0x7f46700048f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ba464787ceee1193 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:06:02.795 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:02.790+0000 7f4662a00640 10 client.0 ms_handle_connect on v2:172.21.15.178:3300/0 2024-09-11T00:06:02.795 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:02.790+0000 7f4662a00640 1 -- 172.21.15.12:0/175860469 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46700ac2b0 con 0x563975600240 2024-09-11T00:06:02.795 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:02.790+0000 7f4686c2d740 1 -- 172.21.15.12:0/175860469 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x563975504a70 con 0x563975600240 2024-09-11T00:06:02.795 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:02.790+0000 7f4662a00640 1 -- 172.21.15.12:0/175860469 <== mon.1 v2:172.21.15.178:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 9092+0+0 (secure 0 0 0) 0x7f46700a8a10 con 0x563975600240 2024-09-11T00:06:02.795 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:02.791+0000 7f4662a00640 1 -- 172.21.15.12:0/175860469 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f46700acea0 con 0x563975600240 2024-09-11T00:06:02.795 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:02.791+0000 7f4686c2d740 10 client.25105 fetch_fsmap learned FSMap version 28 2024-09-11T00:06:02.795 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:02.791+0000 7f4686c2d740 1 -- 172.21.15.12:0/175860469 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x56397551f6f0 con 0x563975600240 2024-09-11T00:06:02.795 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:02.791+0000 7f4662a00640 1 -- 172.21.15.12:0/175860469 <== mon.1 v2:172.21.15.178:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x563975504a70 con 0x563975600240 2024-09-11T00:06:02.795 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:02.791+0000 7f4662a00640 1 -- 172.21.15.12:0/175860469 <== mon.1 v2:172.21.15.178:3300/0 5 ==== fsmap(e 28) ==== 8370+0+0 (secure 0 0 0) 0x7f46700b58a0 con 0x563975600240 2024-09-11T00:06:02.795 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:02.791+0000 7f4686c2d740 10 client.25105 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T00:06:02.795 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:02.791+0000 7f4686c2d740 10 client.25105 resolve_mds: resolved name 'e' to daemon mds.e 2024-09-11T00:06:02.795 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:02.791+0000 7f4686c2d740 20 client.25105 populate_metadata read hostname 'smithi012' 2024-09-11T00:06:02.796 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:02.791+0000 7f4686c2d740 1 --2- 172.21.15.12:0/175860469 >> [v2:172.21.15.165:6834/2425395274,v1:172.21.15.165:6835/2425395274] conn(0x56397561d8a0 0x56397561ed10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:06:02.796 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:02.791+0000 7f4686c2d740 4 client.25105 mds_command: new command op to 24614 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T00:06:02.796 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:02.791+0000 7f4686c2d740 1 -- 172.21.15.12:0/175860469 --> [v2:172.21.15.165:6834/2425395274,v1:172.21.15.165:6835/2425395274] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563975579ca0 con 0x56397561d8a0 2024-09-11T00:06:02.796 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:02.792+0000 7f4680c00640 1 --2- 172.21.15.12:0/175860469 >> [v2:172.21.15.165:6834/2425395274,v1:172.21.15.165:6835/2425395274] conn(0x56397561d8a0 0x56397561ed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:06:02.797 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:02.793+0000 7f4680c00640 1 --2- 172.21.15.12:0/175860469 >> [v2:172.21.15.165:6834/2425395274,v1:172.21.15.165:6835/2425395274] conn(0x56397561d8a0 0x56397561ed10 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=e0a76c769ce9b654 server_cookie=4ed479b703b52b5b in_seq=0 out_seq=0 2024-09-11T00:06:02.797 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:02.793+0000 7f4662a00640 10 client.25105 ms_handle_connect on v2:172.21.15.165:6834/2425395274 2024-09-11T00:06:02.797 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:02.793+0000 7f4662a00640 1 -- 172.21.15.12:0/175860469 <== mds.4 v2:172.21.15.165:6834/2425395274 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x563975579ca0 con 0x56397561d8a0 2024-09-11T00:06:02.797 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:02.793+0000 7f4662a00640 10 client.25105 handle_command_reply: tid=0 2024-09-11T00:06:02.797 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:02.794+0000 7f4686c2d740 1 client.25105 shutdown 2024-09-11T00:06:02.797 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:02.794+0000 7f4686c2d740 1 -- 172.21.15.12:0/175860469 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563975600240 msgr2=0x563975609b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:06:02.797 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:02.794+0000 7f4686c2d740 1 --2- 172.21.15.12:0/175860469 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563975600240 0x563975609b20 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f46700e3af0 tx=0x7f46700048f0 comp rx=0 tx=0).stop 2024-09-11T00:06:02.797 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:02.794+0000 7f4686c2d740 1 -- 172.21.15.12:0/175860469 shutdown_connections 2024-09-11T00:06:02.797 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:02.794+0000 7f4686c2d740 1 --2- 172.21.15.12:0/175860469 >> [v2:172.21.15.165:6834/2425395274,v1:172.21.15.165:6835/2425395274] conn(0x56397561d8a0 0x56397561ed10 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:02.797 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:02.794+0000 7f4686c2d740 1 --2- 172.21.15.12:0/175860469 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56397560caf0 0x56397560a5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:02.797 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:02.794+0000 7f4686c2d740 1 --2- 172.21.15.12:0/175860469 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563975600b60 0x56397560a060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:02.798 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:02.794+0000 7f4686c2d740 1 --2- 172.21.15.12:0/175860469 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563975600240 0x563975609b20 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:02.798 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:02.794+0000 7f4686c2d740 1 -- 172.21.15.12:0/175860469 >> 172.21.15.12:0/175860469 conn(0x5639755fe020 msgr2=0x563975603c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:06:02.798 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:02.794+0000 7f4662a00640 0 client.25105 ms_handle_reset on v2:172.21.15.165:6834/2425395274 2024-09-11T00:06:02.798 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:02.794+0000 7f4686c2d740 1 -- 172.21.15.12:0/175860469 shutdown_connections 2024-09-11T00:06:02.798 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:02.794+0000 7f4686c2d740 1 -- 172.21.15.12:0/175860469 wait complete. 2024-09-11T00:06:02.798 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:02.794+0000 7f4686c2d740 20 client.25105 trim_cache size 0 max 16384 2024-09-11T00:06:02.815 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:02.813+0000 7f4686c2d740 1 Processor -- start 2024-09-11T00:06:02.815 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:02.813+0000 7f4686c2d740 1 -- start start 2024-09-11T00:06:02.815 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:02.813+0000 7f4686c2d740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563975600240 0x56397560b860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:06:02.815 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:02.813+0000 7f4686c2d740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563975600b60 0x563975606d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:06:02.815 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:02.813+0000 7f4686c2d740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56397560caf0 0x5639756072d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:06:02.815 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:02.813+0000 7f4686c2d740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x56397561d050 con 0x563975600240 2024-09-11T00:06:02.815 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:02.813+0000 7f4686c2d740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x56397561d720 con 0x563975600b60 2024-09-11T00:06:02.815 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:02.813+0000 7f4686c2d740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x56397561ddf0 con 0x56397560caf0 2024-09-11T00:06:02.815 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:02.814+0000 7f4681600640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563975600240 0x56397560b860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:06:02.815 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:02.814+0000 7f4681600640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563975600240 0x56397560b860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:38640/0 (socket says 172.21.15.12:38640) 2024-09-11T00:06:02.815 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:02.814+0000 7f4681600640 1 -- 172.21.15.12:0/3927541465 learned_addr learned my addr 172.21.15.12:0/3927541465 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:06:02.816 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:02.814+0000 7f4682000640 1 --2- 172.21.15.12:0/3927541465 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56397560caf0 0x5639756072d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:06:02.816 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:02.814+0000 7f4680c00640 1 --2- 172.21.15.12:0/3927541465 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563975600b60 0x563975606d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:06:02.817 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:02.814+0000 7f4681600640 1 -- 172.21.15.12:0/3927541465 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56397560caf0 msgr2=0x5639756072d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:06:02.817 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:02.814+0000 7f4681600640 1 --2- 172.21.15.12:0/3927541465 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56397560caf0 0x5639756072d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:02.817 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:02.814+0000 7f4681600640 1 -- 172.21.15.12:0/3927541465 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563975600b60 msgr2=0x563975606d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:06:02.817 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:02.814+0000 7f4681600640 1 --2- 172.21.15.12:0/3927541465 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563975600b60 0x563975606d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:02.817 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:02.814+0000 7f4681600640 1 -- 172.21.15.12:0/3927541465 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563975600990 con 0x563975600240 2024-09-11T00:06:02.817 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:02.815+0000 7f4681600640 1 --2- 172.21.15.12:0/3927541465 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563975600240 0x56397560b860 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f46700b11f0 tx=0x7f46700e6430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85e676b0a35cf569 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:06:02.818 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:02.815+0000 7f466dc00640 1 -- 172.21.15.12:0/3927541465 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46700ac2b0 con 0x563975600240 2024-09-11T00:06:02.818 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:02.815+0000 7f466dc00640 1 -- 172.21.15.12:0/3927541465 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f46700cf070 con 0x563975600240 2024-09-11T00:06:02.818 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:02.815+0000 7f466dc00640 1 -- 172.21.15.12:0/3927541465 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46700b1280 con 0x563975600240 2024-09-11T00:06:02.818 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:02.815+0000 7f4686c2d740 1 -- 172.21.15.12:0/3927541465 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563975600240 msgr2=0x56397560b860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:06:02.818 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:02.815+0000 7f4686c2d740 1 --2- 172.21.15.12:0/3927541465 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563975600240 0x56397560b860 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f46700b11f0 tx=0x7f46700e6430 comp rx=0 tx=0).stop 2024-09-11T00:06:02.818 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:02.816+0000 7f4686c2d740 1 -- 172.21.15.12:0/3927541465 shutdown_connections 2024-09-11T00:06:02.818 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:02.816+0000 7f4686c2d740 1 --2- 172.21.15.12:0/3927541465 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56397560caf0 0x5639756072d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:02.818 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:02.816+0000 7f4686c2d740 1 --2- 172.21.15.12:0/3927541465 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563975600b60 0x563975606d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:02.818 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:02.816+0000 7f4686c2d740 1 --2- 172.21.15.12:0/3927541465 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563975600240 0x56397560b860 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:02.818 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:02.816+0000 7f4686c2d740 1 -- 172.21.15.12:0/3927541465 >> 172.21.15.12:0/3927541465 conn(0x5639755fe020 msgr2=0x56397560b150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:06:02.818 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:02.816+0000 7f4686c2d740 1 -- 172.21.15.12:0/3927541465 shutdown_connections 2024-09-11T00:06:02.818 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:02.816+0000 7f4686c2d740 1 -- 172.21.15.12:0/3927541465 wait complete. 2024-09-11T00:06:02.818 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:02.816+0000 7f4686c2d740 1 -- start start 2024-09-11T00:06:02.819 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:02.816+0000 7f4686c2d740 1 Processor -- start 2024-09-11T00:06:02.821 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:02.817+0000 7f4686c2d740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563975600240 0x563975621eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:06:02.821 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:02.817+0000 7f4686c2d740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563975600b60 0x5639756223f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:06:02.821 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:02.817+0000 7f4686c2d740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56397560caf0 0x563975622930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:06:02.821 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:02.817+0000 7f4686c2d740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x56397561eb90 con 0x563975600b60 2024-09-11T00:06:02.821 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:02.817+0000 7f4686c2d740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x56397561e4c0 con 0x56397560caf0 2024-09-11T00:06:02.821 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:02.817+0000 7f4686c2d740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x56397561f260 con 0x563975600240 2024-09-11T00:06:02.821 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:02.817+0000 7f4686c2d740 10 client.0 _ll_register_callbacks cb 0x7f46700a8640 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:06:02.821 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:02.817+0000 7f4680c00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563975600b60 0x5639756223f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:06:02.821 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:02.817+0000 7f4680c00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563975600b60 0x5639756223f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:38642/0 (socket says 172.21.15.12:38642) 2024-09-11T00:06:02.821 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:02.817+0000 7f4680c00640 1 -- 172.21.15.12:0/3783002866 learned_addr learned my addr 172.21.15.12:0/3783002866 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:06:02.822 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:02.817+0000 7f4682000640 1 --2- 172.21.15.12:0/3783002866 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56397560caf0 0x563975622930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:06:02.822 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:02.817+0000 7f4681600640 1 --2- 172.21.15.12:0/3783002866 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563975600240 0x563975621eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:06:02.822 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:02.818+0000 7f4680c00640 1 -- 172.21.15.12:0/3783002866 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563975600240 msgr2=0x563975621eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:06:02.822 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:02.818+0000 7f4680c00640 1 --2- 172.21.15.12:0/3783002866 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563975600240 0x563975621eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:02.822 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:02.818+0000 7f4680c00640 1 -- 172.21.15.12:0/3783002866 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56397560caf0 msgr2=0x563975622930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:06:02.822 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:02.818+0000 7f4680c00640 1 --2- 172.21.15.12:0/3783002866 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56397560caf0 0x563975622930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:02.822 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:02.818+0000 7f4680c00640 1 -- 172.21.15.12:0/3783002866 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563975627180 con 0x563975600b60 2024-09-11T00:06:02.822 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:02.818+0000 7f4680c00640 1 --2- 172.21.15.12:0/3783002866 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563975600b60 0x5639756223f0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f4668006010 tx=0x7f4668009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5cea0bd71fb818b3 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:06:02.822 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:02.818+0000 7f4662a00640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-09-11T00:06:02.822 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:02.818+0000 7f4662a00640 1 -- 172.21.15.12:0/3783002866 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4668060060 con 0x563975600b60 2024-09-11T00:06:02.822 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:02.818+0000 7f4662a00640 1 -- 172.21.15.12:0/3783002866 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 9092+0+0 (secure 0 0 0) 0x7f4668004b80 con 0x563975600b60 2024-09-11T00:06:02.822 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:02.818+0000 7f4686c2d740 1 -- 172.21.15.12:0/3783002866 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f46700cf070 con 0x563975600b60 2024-09-11T00:06:02.822 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:02.818+0000 7f4662a00640 1 -- 172.21.15.12:0/3783002866 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f4668067050 con 0x563975600b60 2024-09-11T00:06:02.823 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:02.818+0000 7f4686c2d740 10 client.15090 fetch_fsmap learned FSMap version 28 2024-09-11T00:06:02.823 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:02.819+0000 7f4686c2d740 1 -- 172.21.15.12:0/3783002866 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x56397568a7e0 con 0x563975600b60 2024-09-11T00:06:02.823 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:02.819+0000 7f4662a00640 1 -- 172.21.15.12:0/3783002866 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f46700cf070 con 0x563975600b60 2024-09-11T00:06:02.823 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:02.819+0000 7f4662a00640 1 -- 172.21.15.12:0/3783002866 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 28) ==== 8370+0+0 (secure 0 0 0) 0x7f46700051b0 con 0x563975600b60 2024-09-11T00:06:02.823 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:02.819+0000 7f4686c2d740 10 client.15090 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T00:06:02.823 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:02.819+0000 7f4686c2d740 10 client.15090 resolve_mds: resolved name 'e' to daemon mds.e 2024-09-11T00:06:02.823 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:02.819+0000 7f4686c2d740 20 client.15090 populate_metadata read hostname 'smithi012' 2024-09-11T00:06:02.823 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:02.819+0000 7f4686c2d740 1 --2- 172.21.15.12:0/3783002866 >> [v2:172.21.15.165:6834/2425395274,v1:172.21.15.165:6835/2425395274] conn(0x56397568f3d0 0x56397568e0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:06:02.823 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:02.819+0000 7f4686c2d740 4 client.15090 mds_command: new command op to 24614 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-11T00:06:02.823 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:02.819+0000 7f4686c2d740 1 -- 172.21.15.12:0/3783002866 --> [v2:172.21.15.165:6834/2425395274,v1:172.21.15.165:6835/2425395274] -- command(tid 0: {"prefix": "status"}) -- 0x563975575750 con 0x56397568f3d0 2024-09-11T00:06:02.823 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:02.820+0000 7f4681600640 1 --2- 172.21.15.12:0/3783002866 >> [v2:172.21.15.165:6834/2425395274,v1:172.21.15.165:6835/2425395274] conn(0x56397568f3d0 0x56397568e0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:06:02.823 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:02.820+0000 7f4681600640 1 --2- 172.21.15.12:0/3783002866 >> [v2:172.21.15.165:6834/2425395274,v1:172.21.15.165:6835/2425395274] conn(0x56397568f3d0 0x56397568e0e0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=72ab9a3f758ad2b9 server_cookie=5518df6c53873dc in_seq=0 out_seq=0 2024-09-11T00:06:02.823 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:02.820+0000 7f4662a00640 10 client.15090 ms_handle_connect on v2:172.21.15.165:6834/2425395274 2024-09-11T00:06:02.824 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:02.820+0000 7f4662a00640 1 -- 172.21.15.12:0/3783002866 <== mds.4 v2:172.21.15.165:6834/2425395274 1 ==== command_reply(tid 0: 0 ) ==== 8+0+326 (crc 0 0 0) 0x563975575750 con 0x56397568f3d0 2024-09-11T00:06:02.825 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:02.820+0000 7f4662a00640 10 client.15090 handle_command_reply: tid=0 2024-09-11T00:06:02.825 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:02.820+0000 7f4686c2d740 1 client.15090 shutdown 2024-09-11T00:06:02.825 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:02.821+0000 7f4686c2d740 1 -- 172.21.15.12:0/3783002866 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563975600b60 msgr2=0x5639756223f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:06:02.825 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:02.821+0000 7f4686c2d740 1 --2- 172.21.15.12:0/3783002866 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563975600b60 0x5639756223f0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f4668006010 tx=0x7f4668009260 comp rx=0 tx=0).stop 2024-09-11T00:06:02.825 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:02.821+0000 7f4686c2d740 1 -- 172.21.15.12:0/3783002866 shutdown_connections 2024-09-11T00:06:02.825 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:02.821+0000 7f4686c2d740 1 --2- 172.21.15.12:0/3783002866 >> [v2:172.21.15.165:6834/2425395274,v1:172.21.15.165:6835/2425395274] conn(0x56397568f3d0 0x56397568e0e0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:02.825 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:02.821+0000 7f4686c2d740 1 --2- 172.21.15.12:0/3783002866 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56397560caf0 0x563975622930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:02.825 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:02.821+0000 7f4686c2d740 1 --2- 172.21.15.12:0/3783002866 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563975600b60 0x5639756223f0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:02.825 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:02.821+0000 7f4686c2d740 1 --2- 172.21.15.12:0/3783002866 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563975600240 0x563975621eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:02.825 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:02.821+0000 7f4686c2d740 1 -- 172.21.15.12:0/3783002866 >> 172.21.15.12:0/3783002866 conn(0x5639755fe020 msgr2=0x56397560b970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:06:02.825 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:02.821+0000 7f4662a00640 0 client.15090 ms_handle_reset on v2:172.21.15.165:6834/2425395274 2024-09-11T00:06:02.825 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:02.821+0000 7f4686c2d740 1 -- 172.21.15.12:0/3783002866 shutdown_connections 2024-09-11T00:06:02.826 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:02.821+0000 7f4686c2d740 1 -- 172.21.15.12:0/3783002866 wait complete. 2024-09-11T00:06:02.826 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:02.821+0000 7f4686c2d740 20 client.15090 trim_cache size 0 max 16384 2024-09-11T00:06:02.826 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-09-11T00:06:02.826 INFO:teuthology.orchestra.run.smithi012.stdout: "cluster_fsid": "ff3ad74a-6fcf-11ef-bcea-c7b262605968", 2024-09-11T00:06:02.826 INFO:teuthology.orchestra.run.smithi012.stdout: "whoami": 4, 2024-09-11T00:06:02.826 INFO:teuthology.orchestra.run.smithi012.stdout: "id": 24614, 2024-09-11T00:06:02.826 INFO:teuthology.orchestra.run.smithi012.stdout: "want_state": "up:active", 2024-09-11T00:06:02.826 INFO:teuthology.orchestra.run.smithi012.stdout: "state": "up:active", 2024-09-11T00:06:02.826 INFO:teuthology.orchestra.run.smithi012.stdout: "fs_name": "cephfs", 2024-09-11T00:06:02.826 INFO:teuthology.orchestra.run.smithi012.stdout: "rank_uptime": 73.581653322999998, 2024-09-11T00:06:02.826 INFO:teuthology.orchestra.run.smithi012.stdout: "mdsmap_epoch": 28, 2024-09-11T00:06:02.827 INFO:teuthology.orchestra.run.smithi012.stdout: "osdmap_epoch": 83, 2024-09-11T00:06:02.827 INFO:teuthology.orchestra.run.smithi012.stdout: "osdmap_epoch_barrier": 81, 2024-09-11T00:06:02.827 INFO:teuthology.orchestra.run.smithi012.stdout: "uptime": 151.12559016 2024-09-11T00:06:02.827 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-09-11T00:06:02.827 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:02.822+0000 7f4684000640 1 -- 172.21.15.12:0/1257787172 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f46640b5280 msgr2=0x7f46640b7740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:06:02.828 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:02.822+0000 7f4684000640 1 --2- 172.21.15.12:0/1257787172 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f46640b5280 0x7f46640b7740 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f4668031440 tx=0x7f4668005a20 comp rx=0 tx=0).stop 2024-09-11T00:06:02.828 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:02.822+0000 7f4684000640 1 -- 172.21.15.12:0/1257787172 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f467c102da0 msgr2=0x7f467c0754a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:06:02.828 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:02.822+0000 7f4684000640 1 --2- 172.21.15.12:0/1257787172 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f467c102da0 0x7f467c0754a0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f467000e9f0 tx=0x7f467000edf0 comp rx=0 tx=0).stop 2024-09-11T00:06:02.828 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:02.822+0000 7f4684000640 1 -- 172.21.15.12:0/1257787172 shutdown_connections 2024-09-11T00:06:02.828 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:02.822+0000 7f4684000640 1 --2- 172.21.15.12:0/1257787172 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f46640b5280 0x7f46640b7740 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:02.828 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:02.822+0000 7f4684000640 1 --2- 172.21.15.12:0/1257787172 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f467c109130 0x7f467c075f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:02.828 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:02.822+0000 7f4684000640 1 --2- 172.21.15.12:0/1257787172 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f467c1087b0 0x7f467c0759e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:02.828 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:02.822+0000 7f4684000640 1 --2- 172.21.15.12:0/1257787172 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f467c102da0 0x7f467c0754a0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:02.828 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:02.822+0000 7f4684000640 1 -- 172.21.15.12:0/1257787172 >> 172.21.15.12:0/1257787172 conn(0x7f467c0fe810 msgr2=0x7f467c100c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:06:02.828 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:02.822+0000 7f4684000640 1 -- 172.21.15.12:0/1257787172 shutdown_connections 2024-09-11T00:06:02.828 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:02.822+0000 7f4684000640 1 -- 172.21.15.12:0/1257787172 wait complete. 2024-09-11T00:06:03.532 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:06:03 smithi165 ceph-mon[35356]: pgmap v365: 129 pgs: 129 active+clean; 647 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-09-11T00:06:03.546 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:06:03 smithi012 ceph-mon[30333]: pgmap v365: 129 pgs: 129 active+clean; 647 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-09-11T00:06:03.581 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:06:03 smithi178 ceph-mon[35919]: pgmap v365: 129 pgs: 129 active+clean; 647 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-09-11T00:06:03.583 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid ff3ad74a-6fcf-11ef-bcea-c7b262605968 -- ceph tell mds.i status 2024-09-11T00:06:03.970 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:06:04.399 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:06:04 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T00:06:04.514 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:04.512+0000 7f4ac0c00640 1 -- 172.21.15.12:0/257549708 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4abc108fa0 msgr2=0x7f4abc10f7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:06:04.514 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:04.512+0000 7f4ac0c00640 1 --2- 172.21.15.12:0/257549708 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4abc108fa0 0x7f4abc10f7a0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f4ab4009ca0 tx=0x7f4ab402a8c0 comp rx=0 tx=0).stop 2024-09-11T00:06:04.514 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:04.512+0000 7f4ac0c00640 1 -- 172.21.15.12:0/257549708 shutdown_connections 2024-09-11T00:06:04.514 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:04.512+0000 7f4ac0c00640 1 --2- 172.21.15.12:0/257549708 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4abc108fa0 0x7f4abc10f7a0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:04.514 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:04.512+0000 7f4ac0c00640 1 --2- 172.21.15.12:0/257549708 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4abc108620 0x7f4abc108a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:04.514 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:04.512+0000 7f4ac0c00640 1 --2- 172.21.15.12:0/257549708 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4abc102c10 0x7f4abc102ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:04.514 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:04.512+0000 7f4ac0c00640 1 -- 172.21.15.12:0/257549708 >> 172.21.15.12:0/257549708 conn(0x7f4abc0fe660 msgr2=0x7f4abc100a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:06:04.514 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:04.513+0000 7f4ac0c00640 1 -- 172.21.15.12:0/257549708 shutdown_connections 2024-09-11T00:06:04.514 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:04.513+0000 7f4ac0c00640 1 -- 172.21.15.12:0/257549708 wait complete. 2024-09-11T00:06:04.515 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:04.514+0000 7f4ac0c00640 1 Processor -- start 2024-09-11T00:06:04.515 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:04.514+0000 7f4ac0c00640 1 -- start start 2024-09-11T00:06:04.516 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:04.515+0000 7f4ac0c00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4abc102c10 0x7f4abc198fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:06:04.516 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:04.515+0000 7f4ac0c00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4abc108620 0x7f4abc199500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:06:04.516 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:04.515+0000 7f4ac0c00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4abc108fa0 0x7f4abc19e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:06:04.516 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:04.515+0000 7f4ac0c00640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f4abc112c30 con 0x7f4abc108620 2024-09-11T00:06:04.516 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:04.515+0000 7f4ac0c00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f4abc112ab0 con 0x7f4abc102c10 2024-09-11T00:06:04.516 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:04.515+0000 7f4ac0c00640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f4abc112db0 con 0x7f4abc108fa0 2024-09-11T00:06:04.516 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:04.515+0000 7f4abb400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4abc102c10 0x7f4abc198fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:06:04.516 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:04.515+0000 7f4abaa00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4abc108620 0x7f4abc199500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:06:04.516 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:04.515+0000 7f4abb400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4abc102c10 0x7f4abc198fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.12:53540/0 (socket says 172.21.15.12:53540) 2024-09-11T00:06:04.516 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:04.515+0000 7f4abb400640 1 -- 172.21.15.12:0/2779120497 learned_addr learned my addr 172.21.15.12:0/2779120497 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:06:04.516 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:04.515+0000 7f4abbe00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4abc108fa0 0x7f4abc19e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:06:04.517 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:04.515+0000 7f4abb400640 1 -- 172.21.15.12:0/2779120497 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4abc108fa0 msgr2=0x7f4abc19e8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:06:04.517 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:04.515+0000 7f4abb400640 1 --2- 172.21.15.12:0/2779120497 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4abc108fa0 0x7f4abc19e8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:04.518 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:04.515+0000 7f4abb400640 1 -- 172.21.15.12:0/2779120497 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4abc108620 msgr2=0x7f4abc199500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:06:04.518 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:04.515+0000 7f4abb400640 1 --2- 172.21.15.12:0/2779120497 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4abc108620 0x7f4abc199500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:04.518 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:04.515+0000 7f4abb400640 1 -- 172.21.15.12:0/2779120497 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4abc19f070 con 0x7f4abc102c10 2024-09-11T00:06:04.518 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:04.515+0000 7f4abaa00640 1 --2- 172.21.15.12:0/2779120497 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4abc108620 0x7f4abc199500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:06:04.518 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:04.516+0000 7f4abb400640 1 --2- 172.21.15.12:0/2779120497 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4abc102c10 0x7f4abc198fc0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f4aac00e9f0 tx=0x7f4aac00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:06:04.518 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:04.516+0000 7f4ab3e00640 1 -- 172.21.15.12:0/2779120497 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4aac00cd80 con 0x7f4abc102c10 2024-09-11T00:06:04.518 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:04.516+0000 7f4ab3e00640 1 -- 172.21.15.12:0/2779120497 <== mon.2 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4aac0108e0 con 0x7f4abc102c10 2024-09-11T00:06:04.518 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:04.516+0000 7f4ac0c00640 1 -- 172.21.15.12:0/2779120497 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4abc19f360 con 0x7f4abc102c10 2024-09-11T00:06:04.518 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:04.516+0000 7f4ab3e00640 1 -- 172.21.15.12:0/2779120497 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4aac018980 con 0x7f4abc102c10 2024-09-11T00:06:04.518 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:04.516+0000 7f4ac0c00640 1 -- 172.21.15.12:0/2779120497 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4abc1a5bf0 con 0x7f4abc102c10 2024-09-11T00:06:04.521 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:04.520+0000 7f4ac4e46740 1 Processor -- start 2024-09-11T00:06:04.521 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:04.520+0000 7f4ab3e00640 1 -- 172.21.15.12:0/2779120497 <== mon.2 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 17) ==== 153301+0+0 (secure 0 0 0) 0x7f4aac010430 con 0x7f4abc102c10 2024-09-11T00:06:04.521 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:04.520+0000 7f4ac4e46740 1 -- start start 2024-09-11T00:06:04.522 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:04.520+0000 7f4ac4e46740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ba52d4f400 0x55ba52d4f7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:06:04.522 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:04.520+0000 7f4ac4e46740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ba52d4fd20 0x55ba52d5fb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:06:04.522 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:04.520+0000 7f4ac4e46740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ba52d600c0 0x55ba52d624b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:06:04.522 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:04.520+0000 7f4ac4e46740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55ba52cc6d50 con 0x55ba52d600c0 2024-09-11T00:06:04.522 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:04.520+0000 7f4ac4e46740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55ba52cb6180 con 0x55ba52d4f400 2024-09-11T00:06:04.523 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:04.520+0000 7f4ac4e46740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55ba52cb9f90 con 0x55ba52d4fd20 2024-09-11T00:06:04.523 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:04.521+0000 7f4abbe00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ba52d4f400 0x55ba52d4f7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:06:04.523 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:04.521+0000 7f4abbe00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ba52d4f400 0x55ba52d4f7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.12:53556/0 (socket says 172.21.15.12:53556) 2024-09-11T00:06:04.523 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:04.521+0000 7f4abbe00640 1 -- 172.21.15.12:0/557374164 learned_addr learned my addr 172.21.15.12:0/557374164 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:06:04.523 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:04.521+0000 7f4abaa00640 1 --2- 172.21.15.12:0/557374164 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ba52d600c0 0x55ba52d624b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:06:04.523 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:04.521+0000 7f4abb400640 1 --2- 172.21.15.12:0/557374164 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ba52d4fd20 0x55ba52d5fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:06:04.523 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:04.521+0000 7f4abbe00640 1 -- 172.21.15.12:0/557374164 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ba52d4fd20 msgr2=0x55ba52d5fb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:06:04.523 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:04.521+0000 7f4abbe00640 1 --2- 172.21.15.12:0/557374164 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ba52d4fd20 0x55ba52d5fb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:04.523 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:04.521+0000 7f4abbe00640 1 -- 172.21.15.12:0/557374164 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ba52d600c0 msgr2=0x55ba52d624b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:06:04.523 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:04.521+0000 7f4abbe00640 1 --2- 172.21.15.12:0/557374164 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ba52d600c0 0x55ba52d624b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:04.523 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:04.521+0000 7f4abbe00640 1 -- 172.21.15.12:0/557374164 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ba52c80a70 con 0x55ba52d4f400 2024-09-11T00:06:04.523 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:04.521+0000 7f4abb400640 1 --2- 172.21.15.12:0/557374164 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ba52d4fd20 0x55ba52d5fb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T00:06:04.524 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:06:04 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T00:06:04.525 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:04.522+0000 7f4abbe00640 1 --2- 172.21.15.12:0/557374164 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ba52d4f400 0x55ba52d4f7e0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f4ab4009410 tx=0x7f4ab4005ef0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6cd533508ee4bea6 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:06:04.525 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:04.522+0000 7f4aa7e00640 1 -- 172.21.15.12:0/557374164 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ab4031cf0 con 0x55ba52d4f400 2024-09-11T00:06:04.525 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:04.522+0000 7f4aa7e00640 1 -- 172.21.15.12:0/557374164 <== mon.2 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4ab4036030 con 0x55ba52d4f400 2024-09-11T00:06:04.525 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:04.522+0000 7f4aa7e00640 1 -- 172.21.15.12:0/557374164 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ab40042a0 con 0x55ba52d4f400 2024-09-11T00:06:04.525 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:04.522+0000 7f4ab3e00640 1 --2- 172.21.15.12:0/2779120497 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f4aa00b5490 0x7f4aa00b7950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:06:04.525 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:04.523+0000 7f4ab3e00640 1 -- 172.21.15.12:0/2779120497 <== mon.2 v2:172.21.15.165:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 9092+0+0 (secure 0 0 0) 0x7f4aac0e6380 con 0x7f4abc102c10 2024-09-11T00:06:04.526 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:04.523+0000 7f4abb400640 1 --2- 172.21.15.12:0/2779120497 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f4aa00b5490 0x7f4aa00b7950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:06:04.526 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:04.523+0000 7f4ac4e46740 1 -- 172.21.15.12:0/557374164 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ba52d4f400 msgr2=0x55ba52d4f7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:06:04.526 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:04.523+0000 7f4ac4e46740 1 --2- 172.21.15.12:0/557374164 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ba52d4f400 0x55ba52d4f7e0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f4ab4009410 tx=0x7f4ab4005ef0 comp rx=0 tx=0).stop 2024-09-11T00:06:04.526 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:04.523+0000 7f4ac4e46740 1 -- 172.21.15.12:0/557374164 shutdown_connections 2024-09-11T00:06:04.526 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:04.523+0000 7f4ac4e46740 1 --2- 172.21.15.12:0/557374164 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ba52d600c0 0x55ba52d624b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:04.526 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:04.523+0000 7f4ac4e46740 1 --2- 172.21.15.12:0/557374164 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ba52d4fd20 0x55ba52d5fb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:04.526 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:04.523+0000 7f4ac4e46740 1 --2- 172.21.15.12:0/557374164 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ba52d4f400 0x55ba52d4f7e0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:04.526 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:04.523+0000 7f4ac4e46740 1 -- 172.21.15.12:0/557374164 >> 172.21.15.12:0/557374164 conn(0x55ba52d4d230 msgr2=0x55ba52d4e3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:06:04.526 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:04.523+0000 7f4ac4e46740 1 -- 172.21.15.12:0/557374164 shutdown_connections 2024-09-11T00:06:04.526 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:04.523+0000 7f4ac4e46740 1 -- 172.21.15.12:0/557374164 wait complete. 2024-09-11T00:06:04.526 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:04.523+0000 7f4abb400640 1 --2- 172.21.15.12:0/2779120497 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f4aa00b5490 0x7f4aa00b7950 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f4aac0ac3d0 tx=0x7f4aac000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:06:04.526 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:04.524+0000 7f4ac4e46740 1 -- start start 2024-09-11T00:06:04.526 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:04.524+0000 7f4ac4e46740 1 Processor -- start 2024-09-11T00:06:04.528 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:04.525+0000 7f4ac4e46740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ba52d4f400 0x55ba52d5d030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:06:04.528 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:04.525+0000 7f4ac4e46740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ba52d4fd20 0x55ba52d5d570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:06:04.528 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:04.525+0000 7f4ac4e46740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ba52d600c0 0x55ba52d5dab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:06:04.528 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:04.525+0000 7f4ac4e46740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55ba52b175f0 con 0x55ba52d4f400 2024-09-11T00:06:04.528 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:04.525+0000 7f4ac4e46740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55ba52c51460 con 0x55ba52d600c0 2024-09-11T00:06:04.528 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:04.525+0000 7f4ac4e46740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55ba52d62c30 con 0x55ba52d4fd20 2024-09-11T00:06:04.529 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:04.525+0000 7f4ac4e46740 10 client.0 _ll_register_callbacks cb 0x55ba52c61210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:06:04.529 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:04.525+0000 7f4abbe00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ba52d4f400 0x55ba52d5d030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:06:04.529 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:04.525+0000 7f4abbe00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ba52d4f400 0x55ba52d5d030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:38674/0 (socket says 172.21.15.12:38674) 2024-09-11T00:06:04.529 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:04.525+0000 7f4abbe00640 1 -- 172.21.15.12:0/560915540 learned_addr learned my addr 172.21.15.12:0/560915540 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:06:04.529 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:04.525+0000 7f4abaa00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ba52d4fd20 0x55ba52d5d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:06:04.529 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:04.525+0000 7f4abbe00640 1 --2- 172.21.15.12:0/560915540 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ba52d600c0 0x55ba52d5dab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:06:04.529 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:04.525+0000 7f4abaa00640 1 -- 172.21.15.12:0/560915540 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ba52d600c0 msgr2=0x55ba52d5dab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:06:04.529 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:04.525+0000 7f4abaa00640 1 --2- 172.21.15.12:0/560915540 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ba52d600c0 0x55ba52d5dab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:04.529 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:04.525+0000 7f4abaa00640 1 -- 172.21.15.12:0/560915540 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ba52d4f400 msgr2=0x55ba52d5d030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:06:04.529 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:04.525+0000 7f4abaa00640 1 --2- 172.21.15.12:0/560915540 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ba52d4f400 0x55ba52d5d030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:04.529 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:04.525+0000 7f4abaa00640 1 -- 172.21.15.12:0/560915540 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ba52a14540 con 0x55ba52d4fd20 2024-09-11T00:06:04.529 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:04.526+0000 7f4abaa00640 1 --2- 172.21.15.12:0/560915540 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ba52d4fd20 0x55ba52d5d570 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f4abc105080 tx=0x7f4aa800f790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e793b83f87ff8088 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:06:04.529 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:04.526+0000 7f4aa4c00640 10 client.0 ms_handle_connect on v2:172.21.15.178:3300/0 2024-09-11T00:06:04.530 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:04.526+0000 7f4aa4c00640 1 -- 172.21.15.12:0/560915540 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4aa8012110 con 0x55ba52d4fd20 2024-09-11T00:06:04.530 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:04.526+0000 7f4ac4e46740 1 -- 172.21.15.12:0/560915540 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55ba52c53a70 con 0x55ba52d4fd20 2024-09-11T00:06:04.530 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:04.526+0000 7f4aa4c00640 1 -- 172.21.15.12:0/560915540 <== mon.1 v2:172.21.15.178:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 9092+0+0 (secure 0 0 0) 0x7f4aa8004050 con 0x55ba52d4fd20 2024-09-11T00:06:04.530 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:04.526+0000 7f4aa4c00640 1 -- 172.21.15.12:0/560915540 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f4aa80122b0 con 0x55ba52d4fd20 2024-09-11T00:06:04.530 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:04.526+0000 7f4ac4e46740 10 client.25123 fetch_fsmap learned FSMap version 28 2024-09-11T00:06:04.530 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:04.526+0000 7f4ac4e46740 1 -- 172.21.15.12:0/560915540 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55ba52c6e6f0 con 0x55ba52d4fd20 2024-09-11T00:06:04.530 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:04.527+0000 7f4aa4c00640 1 -- 172.21.15.12:0/560915540 <== mon.1 v2:172.21.15.178:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55ba52c53a70 con 0x55ba52d4fd20 2024-09-11T00:06:04.530 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:04.527+0000 7f4aa4c00640 1 -- 172.21.15.12:0/560915540 <== mon.1 v2:172.21.15.178:3300/0 5 ==== fsmap(e 28) ==== 8370+0+0 (secure 0 0 0) 0x7f4aa8010430 con 0x55ba52d4fd20 2024-09-11T00:06:04.530 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:04.527+0000 7f4ac4e46740 10 client.25123 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T00:06:04.530 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:04.527+0000 7f4ac4e46740 10 client.25123 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-11T00:06:04.530 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:04.527+0000 7f4ac4e46740 20 client.25123 populate_metadata read hostname 'smithi012' 2024-09-11T00:06:04.530 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:04.527+0000 7f4ac4e46740 1 --2- 172.21.15.12:0/560915540 >> [v2:172.21.15.178:6838/70628097,v1:172.21.15.178:6839/70628097] conn(0x55ba52d70cd0 0x55ba52d72140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:06:04.530 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:04.527+0000 7f4ac4e46740 4 client.25123 mds_command: new command op to 24637 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T00:06:04.531 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:04.527+0000 7f4ac4e46740 1 -- 172.21.15.12:0/560915540 --> [v2:172.21.15.178:6838/70628097,v1:172.21.15.178:6839/70628097] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ba52cc8ca0 con 0x55ba52d70cd0 2024-09-11T00:06:04.531 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:04.527+0000 7f4abbe00640 1 --2- 172.21.15.12:0/560915540 >> [v2:172.21.15.178:6838/70628097,v1:172.21.15.178:6839/70628097] conn(0x55ba52d70cd0 0x55ba52d72140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:06:04.532 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:04.528+0000 7f4abbe00640 1 --2- 172.21.15.12:0/560915540 >> [v2:172.21.15.178:6838/70628097,v1:172.21.15.178:6839/70628097] conn(0x55ba52d70cd0 0x55ba52d72140 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=43efaf21c9e81077 server_cookie=66827dcf681b3de0 in_seq=0 out_seq=0 2024-09-11T00:06:04.532 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:04.528+0000 7f4aa4c00640 10 client.25123 ms_handle_connect on v2:172.21.15.178:6838/70628097 2024-09-11T00:06:04.532 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:04.528+0000 7f4aa4c00640 1 -- 172.21.15.12:0/560915540 <== mds.3 v2:172.21.15.178:6838/70628097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55ba52cc8ca0 con 0x55ba52d70cd0 2024-09-11T00:06:04.532 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:04.529+0000 7f4aa4c00640 10 client.25123 handle_command_reply: tid=0 2024-09-11T00:06:04.532 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:04.529+0000 7f4ac4e46740 1 client.25123 shutdown 2024-09-11T00:06:04.532 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:04.529+0000 7f4ac4e46740 1 -- 172.21.15.12:0/560915540 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ba52d4fd20 msgr2=0x55ba52d5d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:06:04.532 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:04.529+0000 7f4ac4e46740 1 --2- 172.21.15.12:0/560915540 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ba52d4fd20 0x55ba52d5d570 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f4abc105080 tx=0x7f4aa800f790 comp rx=0 tx=0).stop 2024-09-11T00:06:04.532 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:04.529+0000 7f4ac4e46740 1 -- 172.21.15.12:0/560915540 shutdown_connections 2024-09-11T00:06:04.532 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:04.529+0000 7f4ac4e46740 1 --2- 172.21.15.12:0/560915540 >> [v2:172.21.15.178:6838/70628097,v1:172.21.15.178:6839/70628097] conn(0x55ba52d70cd0 0x55ba52d72140 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:04.532 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:04.529+0000 7f4ac4e46740 1 --2- 172.21.15.12:0/560915540 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ba52d600c0 0x55ba52d5dab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:04.532 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:04.529+0000 7f4ac4e46740 1 --2- 172.21.15.12:0/560915540 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ba52d4fd20 0x55ba52d5d570 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:04.533 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:04.529+0000 7f4ac4e46740 1 --2- 172.21.15.12:0/560915540 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ba52d4f400 0x55ba52d5d030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:04.533 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:04.529+0000 7f4ac4e46740 1 -- 172.21.15.12:0/560915540 >> 172.21.15.12:0/560915540 conn(0x55ba52d4d230 msgr2=0x55ba52d4c350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:06:04.533 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:04.529+0000 7f4aa4c00640 0 client.25123 ms_handle_reset on v2:172.21.15.178:6838/70628097 2024-09-11T00:06:04.533 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:04.529+0000 7f4ac4e46740 1 -- 172.21.15.12:0/560915540 shutdown_connections 2024-09-11T00:06:04.533 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:04.529+0000 7f4ac4e46740 1 -- 172.21.15.12:0/560915540 wait complete. 2024-09-11T00:06:04.533 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:04.529+0000 7f4ac4e46740 20 client.25123 trim_cache size 0 max 16384 2024-09-11T00:06:04.560 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:04.559+0000 7f4ac4e46740 1 Processor -- start 2024-09-11T00:06:04.560 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:04.559+0000 7f4ac4e46740 1 -- start start 2024-09-11T00:06:04.562 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:04.560+0000 7f4ac4e46740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ba52d4f400 0x55ba52d483b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:06:04.562 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:04.560+0000 7f4ac4e46740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ba52d4fd20 0x55ba52d4cb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:06:04.562 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:04.560+0000 7f4ac4e46740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ba52d600c0 0x55ba52d61130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:06:04.562 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:04.560+0000 7f4ac4e46740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55ba52d70480 con 0x55ba52d4f400 2024-09-11T00:06:04.562 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:04.560+0000 7f4ac4e46740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55ba52d70b50 con 0x55ba52d600c0 2024-09-11T00:06:04.562 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:04.560+0000 7f4ac4e46740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55ba52d71220 con 0x55ba52d4fd20 2024-09-11T00:06:04.562 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:04.560+0000 7f4abbe00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ba52d4f400 0x55ba52d483b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:06:04.562 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:04.560+0000 7f4abbe00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ba52d4f400 0x55ba52d483b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:38680/0 (socket says 172.21.15.12:38680) 2024-09-11T00:06:04.562 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:04.560+0000 7f4abbe00640 1 -- 172.21.15.12:0/4277403491 learned_addr learned my addr 172.21.15.12:0/4277403491 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:06:04.562 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:04.560+0000 7f4abaa00640 1 --2- 172.21.15.12:0/4277403491 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ba52d4fd20 0x55ba52d4cb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:06:04.562 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:04.560+0000 7f4abbe00640 1 --2- 172.21.15.12:0/4277403491 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ba52d600c0 0x55ba52d61130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:06:04.563 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:04.561+0000 7f4abaa00640 1 -- 172.21.15.12:0/4277403491 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ba52d600c0 msgr2=0x55ba52d61130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:06:04.563 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:04.561+0000 7f4abaa00640 1 --2- 172.21.15.12:0/4277403491 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ba52d600c0 0x55ba52d61130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:04.563 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:04.561+0000 7f4abaa00640 1 -- 172.21.15.12:0/4277403491 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ba52d4f400 msgr2=0x55ba52d483b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:06:04.563 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:04.561+0000 7f4abaa00640 1 --2- 172.21.15.12:0/4277403491 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ba52d4f400 0x55ba52d483b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:04.563 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:04.561+0000 7f4abaa00640 1 -- 172.21.15.12:0/4277403491 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ba52d55560 con 0x55ba52d4fd20 2024-09-11T00:06:04.563 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:04.561+0000 7f4abaa00640 1 --2- 172.21.15.12:0/4277403491 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ba52d4fd20 0x55ba52d4cb40 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f4aa8010a40 tx=0x7f4aa8005850 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6a56ef51399ba4a server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:06:04.564 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:04.561+0000 7f4aa7400640 1 -- 172.21.15.12:0/4277403491 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4aa802e070 con 0x55ba52d4fd20 2024-09-11T00:06:04.565 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:04.561+0000 7f4aa7400640 1 -- 172.21.15.12:0/4277403491 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4aa8005590 con 0x55ba52d4fd20 2024-09-11T00:06:04.565 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:04.561+0000 7f4aa7400640 1 -- 172.21.15.12:0/4277403491 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4aa8003850 con 0x55ba52d4fd20 2024-09-11T00:06:04.565 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:04.562+0000 7f4ac4e46740 1 -- 172.21.15.12:0/4277403491 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ba52d4fd20 msgr2=0x55ba52d4cb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:06:04.565 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:04.562+0000 7f4ac4e46740 1 --2- 172.21.15.12:0/4277403491 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ba52d4fd20 0x55ba52d4cb40 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f4aa8010a40 tx=0x7f4aa8005850 comp rx=0 tx=0).stop 2024-09-11T00:06:04.565 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:04.562+0000 7f4ac4e46740 1 -- 172.21.15.12:0/4277403491 shutdown_connections 2024-09-11T00:06:04.565 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:04.562+0000 7f4ac4e46740 1 --2- 172.21.15.12:0/4277403491 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ba52d600c0 0x55ba52d61130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:04.565 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:04.562+0000 7f4ac4e46740 1 --2- 172.21.15.12:0/4277403491 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ba52d4fd20 0x55ba52d4cb40 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:04.565 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:04.562+0000 7f4ac4e46740 1 --2- 172.21.15.12:0/4277403491 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ba52d4f400 0x55ba52d483b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:04.565 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:04.562+0000 7f4ac4e46740 1 -- 172.21.15.12:0/4277403491 >> 172.21.15.12:0/4277403491 conn(0x55ba52d4d230 msgr2=0x55ba52d6fb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:06:04.565 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:04.562+0000 7f4ac4e46740 1 -- 172.21.15.12:0/4277403491 shutdown_connections 2024-09-11T00:06:04.565 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:04.562+0000 7f4ac4e46740 1 -- 172.21.15.12:0/4277403491 wait complete. 2024-09-11T00:06:04.565 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:04.563+0000 7f4ac4e46740 1 -- start start 2024-09-11T00:06:04.566 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:04.563+0000 7f4ac4e46740 1 Processor -- start 2024-09-11T00:06:04.566 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:04.563+0000 7f4ac4e46740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ba52d4f400 0x55ba52d509d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:06:04.566 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:04.563+0000 7f4ac4e46740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ba52d4fd20 0x55ba52d50f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:06:04.566 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:04.563+0000 7f4ac4e46740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ba52d600c0 0x55ba52d51450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:06:04.566 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:04.563+0000 7f4ac4e46740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55ba52d71fc0 con 0x55ba52d600c0 2024-09-11T00:06:04.566 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:04.563+0000 7f4ac4e46740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55ba52d718f0 con 0x55ba52d4fd20 2024-09-11T00:06:04.566 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:04.563+0000 7f4ac4e46740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55ba52d72690 con 0x55ba52d4f400 2024-09-11T00:06:04.566 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:04.563+0000 7f4ac4e46740 10 client.0 _ll_register_callbacks cb 0x7f4aa8012930 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:06:04.566 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:04.564+0000 7f4abaa00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ba52d4fd20 0x55ba52d50f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:06:04.566 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:04.564+0000 7f4abaa00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ba52d4fd20 0x55ba52d50f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.12:53586/0 (socket says 172.21.15.12:53586) 2024-09-11T00:06:04.566 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:04.564+0000 7f4abbe00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ba52d600c0 0x55ba52d51450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:06:04.567 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:04.564+0000 7f4abaa00640 1 -- 172.21.15.12:0/945794107 learned_addr learned my addr 172.21.15.12:0/945794107 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:06:04.567 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:04.564+0000 7f4abbe00640 1 --2- 172.21.15.12:0/945794107 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ba52d4f400 0x55ba52d509d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:06:04.568 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:04.564+0000 7f4abbe00640 1 -- 172.21.15.12:0/945794107 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ba52d4f400 msgr2=0x55ba52d509d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:06:04.568 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:04.564+0000 7f4abbe00640 1 --2- 172.21.15.12:0/945794107 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ba52d4f400 0x55ba52d509d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:04.568 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:04.564+0000 7f4abbe00640 1 -- 172.21.15.12:0/945794107 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ba52d4fd20 msgr2=0x55ba52d50f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:06:04.568 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:04.564+0000 7f4abbe00640 1 --2- 172.21.15.12:0/945794107 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ba52d4fd20 0x55ba52d50f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:04.568 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:04.564+0000 7f4abbe00640 1 -- 172.21.15.12:0/945794107 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ba52d7a530 con 0x55ba52d600c0 2024-09-11T00:06:04.568 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:04.564+0000 7f4abaa00640 1 --2- 172.21.15.12:0/945794107 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ba52d4fd20 0x55ba52d50f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:06:04.568 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:04.565+0000 7f4abbe00640 1 --2- 172.21.15.12:0/945794107 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ba52d600c0 0x55ba52d51450 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f4ab4003bf0 tx=0x7f4ab4003cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b1cff4c2d0755ac server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:06:04.568 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:04.565+0000 7f4aa4c00640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-09-11T00:06:04.569 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:04.565+0000 7f4aa4c00640 1 -- 172.21.15.12:0/945794107 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ab404a040 con 0x55ba52d600c0 2024-09-11T00:06:04.569 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:04.565+0000 7f4aa4c00640 1 -- 172.21.15.12:0/945794107 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 9092+0+0 (secure 0 0 0) 0x7f4ab402f660 con 0x55ba52d600c0 2024-09-11T00:06:04.569 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:04.565+0000 7f4ac4e46740 1 -- 172.21.15.12:0/945794107 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4aa8005590 con 0x55ba52d600c0 2024-09-11T00:06:04.569 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:04.565+0000 7f4aa4c00640 1 -- 172.21.15.12:0/945794107 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f4ab402f860 con 0x55ba52d600c0 2024-09-11T00:06:04.569 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:04.565+0000 7f4ac4e46740 10 client.15108 fetch_fsmap learned FSMap version 28 2024-09-11T00:06:04.569 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:04.565+0000 7f4ac4e46740 1 -- 172.21.15.12:0/945794107 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55ba52dddac0 con 0x55ba52d600c0 2024-09-11T00:06:04.569 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:04.566+0000 7f4aa4c00640 1 -- 172.21.15.12:0/945794107 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4aa8005590 con 0x55ba52d600c0 2024-09-11T00:06:04.569 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:04.566+0000 7f4aa4c00640 1 -- 172.21.15.12:0/945794107 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 28) ==== 8370+0+0 (secure 0 0 0) 0x7f4ab4043070 con 0x55ba52d600c0 2024-09-11T00:06:04.569 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:04.566+0000 7f4ac4e46740 10 client.15108 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T00:06:04.569 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:04.566+0000 7f4ac4e46740 10 client.15108 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-11T00:06:04.569 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:04.566+0000 7f4ac4e46740 20 client.15108 populate_metadata read hostname 'smithi012' 2024-09-11T00:06:04.569 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:04.566+0000 7f4ac4e46740 1 --2- 172.21.15.12:0/945794107 >> [v2:172.21.15.178:6838/70628097,v1:172.21.15.178:6839/70628097] conn(0x55ba52de22d0 0x55ba52de0fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:06:04.569 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:04.566+0000 7f4ac4e46740 4 client.15108 mds_command: new command op to 24637 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-11T00:06:04.570 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:04.566+0000 7f4ac4e46740 1 -- 172.21.15.12:0/945794107 --> [v2:172.21.15.178:6838/70628097,v1:172.21.15.178:6839/70628097] -- command(tid 0: {"prefix": "status"}) -- 0x55ba52cc4750 con 0x55ba52de22d0 2024-09-11T00:06:04.570 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:04.566+0000 7f4abaa00640 1 --2- 172.21.15.12:0/945794107 >> [v2:172.21.15.178:6838/70628097,v1:172.21.15.178:6839/70628097] conn(0x55ba52de22d0 0x55ba52de0fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:06:04.571 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:04.568+0000 7f4aa4c00640 10 client.15108 ms_handle_connect on v2:172.21.15.178:6838/70628097 2024-09-11T00:06:04.571 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:04.568+0000 7f4abaa00640 1 --2- 172.21.15.12:0/945794107 >> [v2:172.21.15.178:6838/70628097,v1:172.21.15.178:6839/70628097] conn(0x55ba52de22d0 0x55ba52de0fe0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=46f14a2c7ab585bb server_cookie=cfb8b65690b65e10 in_seq=0 out_seq=0 2024-09-11T00:06:04.571 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:04.568+0000 7f4aa4c00640 1 -- 172.21.15.12:0/945794107 <== mds.3 v2:172.21.15.178:6838/70628097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55ba52cc4750 con 0x55ba52de22d0 2024-09-11T00:06:04.571 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:04.568+0000 7f4aa4c00640 10 client.15108 handle_command_reply: tid=0 2024-09-11T00:06:04.571 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:04.568+0000 7f4ac4e46740 1 client.15108 shutdown 2024-09-11T00:06:04.571 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:04.568+0000 7f4ac4e46740 1 -- 172.21.15.12:0/945794107 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ba52d600c0 msgr2=0x55ba52d51450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:06:04.571 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:04.568+0000 7f4ac4e46740 1 --2- 172.21.15.12:0/945794107 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ba52d600c0 0x55ba52d51450 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f4ab4003bf0 tx=0x7f4ab4003cc0 comp rx=0 tx=0).stop 2024-09-11T00:06:04.571 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:04.568+0000 7f4ac4e46740 1 -- 172.21.15.12:0/945794107 shutdown_connections 2024-09-11T00:06:04.571 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:04.568+0000 7f4ac4e46740 1 --2- 172.21.15.12:0/945794107 >> [v2:172.21.15.178:6838/70628097,v1:172.21.15.178:6839/70628097] conn(0x55ba52de22d0 0x55ba52de0fe0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:04.571 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:04.568+0000 7f4ac4e46740 1 --2- 172.21.15.12:0/945794107 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ba52d600c0 0x55ba52d51450 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:04.571 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:04.568+0000 7f4ac4e46740 1 --2- 172.21.15.12:0/945794107 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ba52d4fd20 0x55ba52d50f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:04.571 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:04.568+0000 7f4ac4e46740 1 --2- 172.21.15.12:0/945794107 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ba52d4f400 0x55ba52d509d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:04.572 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:04.568+0000 7f4aa4c00640 0 client.15108 ms_handle_reset on v2:172.21.15.178:6838/70628097 2024-09-11T00:06:04.572 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:04.568+0000 7f4ac4e46740 1 -- 172.21.15.12:0/945794107 >> 172.21.15.12:0/945794107 conn(0x55ba52d4d230 msgr2=0x55ba52d6fb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:06:04.572 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:04.568+0000 7f4ac4e46740 1 -- 172.21.15.12:0/945794107 shutdown_connections 2024-09-11T00:06:04.572 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:04.568+0000 7f4ac4e46740 1 -- 172.21.15.12:0/945794107 wait complete. 2024-09-11T00:06:04.572 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:04.569+0000 7f4ac4e46740 20 client.15108 trim_cache size 0 max 16384 2024-09-11T00:06:04.572 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-09-11T00:06:04.572 INFO:teuthology.orchestra.run.smithi012.stdout: "cluster_fsid": "ff3ad74a-6fcf-11ef-bcea-c7b262605968", 2024-09-11T00:06:04.572 INFO:teuthology.orchestra.run.smithi012.stdout: "whoami": 3, 2024-09-11T00:06:04.572 INFO:teuthology.orchestra.run.smithi012.stdout: "id": 24637, 2024-09-11T00:06:04.573 INFO:teuthology.orchestra.run.smithi012.stdout: "want_state": "up:active", 2024-09-11T00:06:04.573 INFO:teuthology.orchestra.run.smithi012.stdout: "state": "up:active", 2024-09-11T00:06:04.573 INFO:teuthology.orchestra.run.smithi012.stdout: "fs_name": "cephfs", 2024-09-11T00:06:04.573 INFO:teuthology.orchestra.run.smithi012.stdout: "rank_uptime": 76.334057391000002, 2024-09-11T00:06:04.573 INFO:teuthology.orchestra.run.smithi012.stdout: "mdsmap_epoch": 28, 2024-09-11T00:06:04.573 INFO:teuthology.orchestra.run.smithi012.stdout: "osdmap_epoch": 83, 2024-09-11T00:06:04.573 INFO:teuthology.orchestra.run.smithi012.stdout: "osdmap_epoch_barrier": 81, 2024-09-11T00:06:04.573 INFO:teuthology.orchestra.run.smithi012.stdout: "uptime": 142.36165384700001 2024-09-11T00:06:04.573 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-09-11T00:06:04.574 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:04.570+0000 7f4ac0c00640 1 -- 172.21.15.12:0/2779120497 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f4aa00b5490 msgr2=0x7f4aa00b7950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:06:04.574 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:04.570+0000 7f4ac0c00640 1 --2- 172.21.15.12:0/2779120497 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f4aa00b5490 0x7f4aa00b7950 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f4aac0ac3d0 tx=0x7f4aac000bc0 comp rx=0 tx=0).stop 2024-09-11T00:06:04.574 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:04.570+0000 7f4ac0c00640 1 -- 172.21.15.12:0/2779120497 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4abc102c10 msgr2=0x7f4abc198fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:06:04.574 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:04.570+0000 7f4ac0c00640 1 --2- 172.21.15.12:0/2779120497 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4abc102c10 0x7f4abc198fc0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f4aac00e9f0 tx=0x7f4aac00eeb0 comp rx=0 tx=0).stop 2024-09-11T00:06:04.574 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:04.570+0000 7f4ac0c00640 1 -- 172.21.15.12:0/2779120497 shutdown_connections 2024-09-11T00:06:04.574 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:04.570+0000 7f4ac0c00640 1 --2- 172.21.15.12:0/2779120497 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f4aa00b5490 0x7f4aa00b7950 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:04.574 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:04.570+0000 7f4ac0c00640 1 --2- 172.21.15.12:0/2779120497 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4abc108fa0 0x7f4abc19e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:04.574 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:04.570+0000 7f4ac0c00640 1 --2- 172.21.15.12:0/2779120497 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4abc108620 0x7f4abc199500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:04.574 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:04.570+0000 7f4ac0c00640 1 --2- 172.21.15.12:0/2779120497 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4abc102c10 0x7f4abc198fc0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:04.574 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:04.570+0000 7f4ac0c00640 1 -- 172.21.15.12:0/2779120497 >> 172.21.15.12:0/2779120497 conn(0x7f4abc0fe660 msgr2=0x7f4abc100a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:06:04.574 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:04.570+0000 7f4ac0c00640 1 -- 172.21.15.12:0/2779120497 shutdown_connections 2024-09-11T00:06:04.575 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:04.570+0000 7f4ac0c00640 1 -- 172.21.15.12:0/2779120497 wait complete. 2024-09-11T00:06:04.580 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:06:04 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T00:06:05.340 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid ff3ad74a-6fcf-11ef-bcea-c7b262605968 -- ceph tell mds.h status 2024-09-11T00:06:05.532 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:06:05 smithi165 ceph-mon[35356]: pgmap v366: 129 pgs: 129 active+clean; 647 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-11T00:06:05.562 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:06:05 smithi012 ceph-mon[30333]: pgmap v366: 129 pgs: 129 active+clean; 647 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-11T00:06:05.580 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:06:05 smithi178 ceph-mon[35919]: pgmap v366: 129 pgs: 129 active+clean; 647 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-11T00:06:06.165 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:06:06.644 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:06:06 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:06:06.645 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:06:06 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:06:06.645 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:06:06 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:06:06.645 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:06:06 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:06:06.761 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:06.759+0000 7effc3400640 1 -- 172.21.15.12:0/4084038955 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7effbc077ff0 msgr2=0x7effbc113470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:06:06.761 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:06.759+0000 7effc3400640 1 --2- 172.21.15.12:0/4084038955 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7effbc077ff0 0x7effbc113470 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7effa8009900 tx=0x7effa802a3f0 comp rx=0 tx=0).stop 2024-09-11T00:06:06.761 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:06.759+0000 7effc3400640 1 -- 172.21.15.12:0/4084038955 shutdown_connections 2024-09-11T00:06:06.761 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:06.759+0000 7effc3400640 1 --2- 172.21.15.12:0/4084038955 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7effbc1139b0 0x7effbc115da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:06.761 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:06.759+0000 7effc3400640 1 --2- 172.21.15.12:0/4084038955 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7effbc077ff0 0x7effbc113470 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:06.761 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:06.759+0000 7effc3400640 1 --2- 172.21.15.12:0/4084038955 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7effbc0776d0 0x7effbc077ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:06.761 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:06.759+0000 7effc3400640 1 -- 172.21.15.12:0/4084038955 >> 172.21.15.12:0/4084038955 conn(0x7effbc1009f0 msgr2=0x7effbc102e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:06:06.761 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:06.760+0000 7effc3400640 1 -- 172.21.15.12:0/4084038955 shutdown_connections 2024-09-11T00:06:06.761 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:06.760+0000 7effc3400640 1 -- 172.21.15.12:0/4084038955 wait complete. 2024-09-11T00:06:06.762 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:06.761+0000 7effc3400640 1 Processor -- start 2024-09-11T00:06:06.762 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:06.761+0000 7effc3400640 1 -- start start 2024-09-11T00:06:06.762 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:06.761+0000 7effc3400640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7effbc0776d0 0x7effbc19f8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:06:06.762 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:06.761+0000 7effc3400640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7effbc077ff0 0x7effbc19fe10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:06:06.762 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:06.761+0000 7effc3400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7effbc1139b0 0x7effbc1a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:06:06.762 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:06.761+0000 7effc3400640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7effbc119400 con 0x7effbc0776d0 2024-09-11T00:06:06.762 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:06.761+0000 7effc3400640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7effbc119280 con 0x7effbc1139b0 2024-09-11T00:06:06.762 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:06.761+0000 7effc3400640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7effbc119580 con 0x7effbc077ff0 2024-09-11T00:06:06.763 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:06.762+0000 7effc0a00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7effbc0776d0 0x7effbc19f8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:06:06.763 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:06.762+0000 7effbbe00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7effbc077ff0 0x7effbc19fe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:06:06.763 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:06.762+0000 7effc0a00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7effbc0776d0 0x7effbc19f8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:38712/0 (socket says 172.21.15.12:38712) 2024-09-11T00:06:06.763 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:06.762+0000 7effc0a00640 1 -- 172.21.15.12:0/821579874 learned_addr learned my addr 172.21.15.12:0/821579874 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:06:06.763 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:06.762+0000 7effc1400640 1 --2- 172.21.15.12:0/821579874 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7effbc1139b0 0x7effbc1a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:06:06.764 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:06.762+0000 7effbbe00640 1 -- 172.21.15.12:0/821579874 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7effbc1139b0 msgr2=0x7effbc1a51b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:06:06.764 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:06.762+0000 7effbbe00640 1 --2- 172.21.15.12:0/821579874 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7effbc1139b0 0x7effbc1a51b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:06.764 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:06.762+0000 7effbbe00640 1 -- 172.21.15.12:0/821579874 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7effbc0776d0 msgr2=0x7effbc19f8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:06:06.764 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:06.762+0000 7effbbe00640 1 --2- 172.21.15.12:0/821579874 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7effbc0776d0 0x7effbc19f8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:06.764 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:06.762+0000 7effbbe00640 1 -- 172.21.15.12:0/821579874 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7effbc1a5980 con 0x7effbc077ff0 2024-09-11T00:06:06.764 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:06.763+0000 7effbbe00640 1 --2- 172.21.15.12:0/821579874 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7effbc077ff0 0x7effbc19fe10 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7effa802a9e0 tx=0x7effa8003b90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:06:06.764 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:06.763+0000 7effb9600640 1 -- 172.21.15.12:0/821579874 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effa803f070 con 0x7effbc077ff0 2024-09-11T00:06:06.765 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:06.763+0000 7effb9600640 1 -- 172.21.15.12:0/821579874 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7effa8031b50 con 0x7effbc077ff0 2024-09-11T00:06:06.765 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:06.763+0000 7effb9600640 1 -- 172.21.15.12:0/821579874 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effa8035030 con 0x7effbc077ff0 2024-09-11T00:06:06.765 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:06.763+0000 7effc3400640 1 -- 172.21.15.12:0/821579874 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7effbc1a5c10 con 0x7effbc077ff0 2024-09-11T00:06:06.765 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:06.764+0000 7effc3400640 1 -- 172.21.15.12:0/821579874 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7effbc1ac330 con 0x7effbc077ff0 2024-09-11T00:06:06.767 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:06.767+0000 7effc6000740 1 Processor -- start 2024-09-11T00:06:06.768 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:06.767+0000 7effc6000740 1 -- start start 2024-09-11T00:06:06.768 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:06.767+0000 7effc6000740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d6fed36240 0x55d6fed36620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:06:06.768 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:06.767+0000 7effc6000740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d6fed36b60 0x55d6fed425b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:06:06.768 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:06.767+0000 7effb9600640 1 -- 172.21.15.12:0/821579874 <== mon.1 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 17) ==== 153301+0+0 (secure 0 0 0) 0x7effa80316a0 con 0x7effbc077ff0 2024-09-11T00:06:06.768 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:06.767+0000 7effc6000740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d6fed42af0 0x55d6fed44ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:06:06.769 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:06.767+0000 7effc6000740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55d6fecadd50 con 0x55d6fed36240 2024-09-11T00:06:06.769 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:06.767+0000 7effc6000740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55d6fec9d180 con 0x55d6fed42af0 2024-09-11T00:06:06.769 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:06.767+0000 7effc6000740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55d6feca0f90 con 0x55d6fed36b60 2024-09-11T00:06:06.770 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:06.768+0000 7effc0a00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d6fed36b60 0x55d6fed425b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:06:06.770 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:06.768+0000 7effc0a00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d6fed36b60 0x55d6fed425b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.12:49352/0 (socket says 172.21.15.12:49352) 2024-09-11T00:06:06.770 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:06.768+0000 7effc0a00640 1 -- 172.21.15.12:0/1837762509 learned_addr learned my addr 172.21.15.12:0/1837762509 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:06:06.770 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:06.768+0000 7effc1400640 1 --2- 172.21.15.12:0/1837762509 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d6fed36240 0x55d6fed36620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:06:06.770 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:06.768+0000 7effbbe00640 1 --2- 172.21.15.12:0/1837762509 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d6fed42af0 0x55d6fed44ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:06:06.770 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:06.768+0000 7effc0a00640 1 -- 172.21.15.12:0/1837762509 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d6fed42af0 msgr2=0x55d6fed44ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:06:06.770 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:06.768+0000 7effc0a00640 1 --2- 172.21.15.12:0/1837762509 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d6fed42af0 0x55d6fed44ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:06.770 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:06.768+0000 7effc0a00640 1 -- 172.21.15.12:0/1837762509 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d6fed36240 msgr2=0x55d6fed36620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:06:06.770 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:06.768+0000 7effc0a00640 1 --2- 172.21.15.12:0/1837762509 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d6fed36240 0x55d6fed36620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:06.770 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:06.768+0000 7effc0a00640 1 -- 172.21.15.12:0/1837762509 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d6fec67a70 con 0x55d6fed36b60 2024-09-11T00:06:06.771 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:06.769+0000 7effc0a00640 1 --2- 172.21.15.12:0/1837762509 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d6fed36b60 0x55d6fed425b0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7effb0009d90 tx=0x7effb00117b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a776c540af47bf1c server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:06:06.771 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:06.769+0000 7effadc00640 1 -- 172.21.15.12:0/1837762509 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effb001b670 con 0x55d6fed36b60 2024-09-11T00:06:06.771 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:06.769+0000 7effadc00640 1 -- 172.21.15.12:0/1837762509 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7effb0016070 con 0x55d6fed36b60 2024-09-11T00:06:06.771 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:06.769+0000 7effadc00640 1 -- 172.21.15.12:0/1837762509 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effb000f680 con 0x55d6fed36b60 2024-09-11T00:06:06.771 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:06.770+0000 7effb9600640 1 --2- 172.21.15.12:0/821579874 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7effa40b54c0 0x7effa40b7980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:06:06.772 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:06.770+0000 7effc6000740 1 -- 172.21.15.12:0/1837762509 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d6fed36b60 msgr2=0x55d6fed425b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:06:06.772 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:06.770+0000 7effc6000740 1 --2- 172.21.15.12:0/1837762509 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d6fed36b60 0x55d6fed425b0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7effb0009d90 tx=0x7effb00117b0 comp rx=0 tx=0).stop 2024-09-11T00:06:06.772 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:06.770+0000 7effb9600640 1 -- 172.21.15.12:0/821579874 <== mon.1 v2:172.21.15.178:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 9092+0+0 (secure 0 0 0) 0x7effa81068e0 con 0x7effbc077ff0 2024-09-11T00:06:06.772 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:06.770+0000 7effc1400640 1 --2- 172.21.15.12:0/821579874 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7effa40b54c0 0x7effa40b7980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:06:06.772 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:06.771+0000 7effc6000740 1 -- 172.21.15.12:0/1837762509 shutdown_connections 2024-09-11T00:06:06.772 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:06.771+0000 7effc6000740 1 --2- 172.21.15.12:0/1837762509 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d6fed42af0 0x55d6fed44ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:06.772 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:06.771+0000 7effc6000740 1 --2- 172.21.15.12:0/1837762509 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d6fed36b60 0x55d6fed425b0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:06.772 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:06.771+0000 7effc6000740 1 --2- 172.21.15.12:0/1837762509 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d6fed36240 0x55d6fed36620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:06.772 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:06.771+0000 7effc6000740 1 -- 172.21.15.12:0/1837762509 >> 172.21.15.12:0/1837762509 conn(0x55d6fed34020 msgr2=0x55d6fed35210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:06:06.772 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:06.771+0000 7effc6000740 1 -- 172.21.15.12:0/1837762509 shutdown_connections 2024-09-11T00:06:06.773 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:06.771+0000 7effc6000740 1 -- 172.21.15.12:0/1837762509 wait complete. 2024-09-11T00:06:06.773 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:06.772+0000 7effc1400640 1 --2- 172.21.15.12:0/821579874 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7effa40b54c0 0x7effa40b7980 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7effbc06c140 tx=0x7effb4009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:06:06.773 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:06.772+0000 7effc6000740 1 -- start start 2024-09-11T00:06:06.773 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:06.772+0000 7effc6000740 1 Processor -- start 2024-09-11T00:06:06.774 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:06.773+0000 7effc6000740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d6fed36240 0x55d6fed425b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:06:06.774 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:06.773+0000 7effc6000740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d6fed36b60 0x55d6fed3fa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:06:06.774 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:06.773+0000 7effc6000740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d6fed42af0 0x55d6fed401d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:06:06.774 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:06.773+0000 7effc6000740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55d6feafe5f0 con 0x55d6fed36240 2024-09-11T00:06:06.774 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:06.773+0000 7effc6000740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55d6fec38460 con 0x55d6fed36b60 2024-09-11T00:06:06.774 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:06.773+0000 7effc6000740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55d6fed45840 con 0x55d6fed42af0 2024-09-11T00:06:06.774 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:06.773+0000 7effc6000740 10 client.0 _ll_register_callbacks cb 0x55d6fec48210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:06:06.775 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:06.773+0000 7effc1400640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d6fed42af0 0x55d6fed401d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:06:06.775 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:06.773+0000 7effc1400640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d6fed42af0 0x55d6fed401d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.12:49368/0 (socket says 172.21.15.12:49368) 2024-09-11T00:06:06.775 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:06.773+0000 7effc1400640 1 -- 172.21.15.12:0/3047792788 learned_addr learned my addr 172.21.15.12:0/3047792788 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:06:06.775 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:06.773+0000 7effbbe00640 1 --2- 172.21.15.12:0/3047792788 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d6fed36b60 0x55d6fed3fa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:06:06.775 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:06.773+0000 7effc0a00640 1 --2- 172.21.15.12:0/3047792788 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d6fed36240 0x55d6fed425b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:06:06.776 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:06.773+0000 7effc1400640 1 -- 172.21.15.12:0/3047792788 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d6fed36b60 msgr2=0x55d6fed3fa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:06:06.776 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:06.773+0000 7effc1400640 1 --2- 172.21.15.12:0/3047792788 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d6fed36b60 0x55d6fed3fa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:06.776 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:06.773+0000 7effc1400640 1 -- 172.21.15.12:0/3047792788 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d6fed36240 msgr2=0x55d6fed425b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:06:06.776 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:06.773+0000 7effc1400640 1 --2- 172.21.15.12:0/3047792788 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d6fed36240 0x55d6fed425b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:06.776 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:06.773+0000 7effc1400640 1 -- 172.21.15.12:0/3047792788 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d6fe9fb540 con 0x55d6fed42af0 2024-09-11T00:06:06.776 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:06.774+0000 7effc1400640 1 --2- 172.21.15.12:0/3047792788 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d6fed42af0 0x55d6fed401d0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7effb4003fc0 tx=0x7effb4004ab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bf801481e5874533 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:06:06.776 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:06.774+0000 7effa2000640 10 client.0 ms_handle_connect on v2:172.21.15.178:3300/0 2024-09-11T00:06:06.776 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:06.774+0000 7effa2000640 1 -- 172.21.15.12:0/3047792788 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effb402d060 con 0x55d6fed42af0 2024-09-11T00:06:06.776 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:06.774+0000 7effc6000740 1 -- 172.21.15.12:0/3047792788 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55d6fec3aa70 con 0x55d6fed42af0 2024-09-11T00:06:06.777 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:06.774+0000 7effa2000640 1 -- 172.21.15.12:0/3047792788 <== mon.1 v2:172.21.15.178:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 9092+0+0 (secure 0 0 0) 0x7effb4018070 con 0x55d6fed42af0 2024-09-11T00:06:06.777 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:06.775+0000 7effa2000640 1 -- 172.21.15.12:0/3047792788 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7effb4034050 con 0x55d6fed42af0 2024-09-11T00:06:06.777 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:06.775+0000 7effc6000740 10 client.25141 fetch_fsmap learned FSMap version 28 2024-09-11T00:06:06.777 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:06.775+0000 7effc6000740 1 -- 172.21.15.12:0/3047792788 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55d6fec556f0 con 0x55d6fed42af0 2024-09-11T00:06:06.777 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:06.775+0000 7effa2000640 1 -- 172.21.15.12:0/3047792788 <== mon.1 v2:172.21.15.178:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55d6fec3aa70 con 0x55d6fed42af0 2024-09-11T00:06:06.777 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:06.775+0000 7effa2000640 1 -- 172.21.15.12:0/3047792788 <== mon.1 v2:172.21.15.178:3300/0 5 ==== fsmap(e 28) ==== 8370+0+0 (secure 0 0 0) 0x7effb401e920 con 0x55d6fed42af0 2024-09-11T00:06:06.778 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:06.775+0000 7effc6000740 10 client.25141 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T00:06:06.778 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:06.775+0000 7effc6000740 10 client.25141 resolve_mds: resolved name 'h' to daemon mds.h 2024-09-11T00:06:06.778 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:06.775+0000 7effc6000740 20 client.25141 populate_metadata read hostname 'smithi012' 2024-09-11T00:06:06.778 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:06.775+0000 7effc6000740 1 --2- 172.21.15.12:0/3047792788 >> [v2:172.21.15.165:6836/6215088,v1:172.21.15.165:6837/6215088] conn(0x55d6fed53890 0x55d6fed54d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:06:06.778 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:06.775+0000 7effc6000740 4 client.25141 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T00:06:06.778 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:06.775+0000 7effc6000740 1 -- 172.21.15.12:0/3047792788 --> [v2:172.21.15.165:6836/6215088,v1:172.21.15.165:6837/6215088] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d6fecafca0 con 0x55d6fed53890 2024-09-11T00:06:06.778 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:06.776+0000 7effc0a00640 1 --2- 172.21.15.12:0/3047792788 >> [v2:172.21.15.165:6836/6215088,v1:172.21.15.165:6837/6215088] conn(0x55d6fed53890 0x55d6fed54d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:06:06.778 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:06.777+0000 7effc0a00640 1 --2- 172.21.15.12:0/3047792788 >> [v2:172.21.15.165:6836/6215088,v1:172.21.15.165:6837/6215088] conn(0x55d6fed53890 0x55d6fed54d00 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=72b81ef2e1110929 server_cookie=1a538366d8620d6e in_seq=0 out_seq=0 2024-09-11T00:06:06.778 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:06.777+0000 7effa2000640 10 client.25141 ms_handle_connect on v2:172.21.15.165:6836/6215088 2024-09-11T00:06:06.779 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:06.777+0000 7effa2000640 1 -- 172.21.15.12:0/3047792788 <== mds.2 v2:172.21.15.165:6836/6215088 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55d6fecafca0 con 0x55d6fed53890 2024-09-11T00:06:06.779 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:06.777+0000 7effa2000640 10 client.25141 handle_command_reply: tid=0 2024-09-11T00:06:06.779 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:06.778+0000 7effc6000740 1 client.25141 shutdown 2024-09-11T00:06:06.779 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:06.778+0000 7effc6000740 1 -- 172.21.15.12:0/3047792788 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d6fed42af0 msgr2=0x55d6fed401d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:06:06.779 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:06.778+0000 7effc6000740 1 --2- 172.21.15.12:0/3047792788 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d6fed42af0 0x55d6fed401d0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7effb4003fc0 tx=0x7effb4004ab0 comp rx=0 tx=0).stop 2024-09-11T00:06:06.779 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:06.778+0000 7effc6000740 1 -- 172.21.15.12:0/3047792788 shutdown_connections 2024-09-11T00:06:06.780 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:06.778+0000 7effc6000740 1 --2- 172.21.15.12:0/3047792788 >> [v2:172.21.15.165:6836/6215088,v1:172.21.15.165:6837/6215088] conn(0x55d6fed53890 0x55d6fed54d00 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:06.780 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:06.778+0000 7effc6000740 1 --2- 172.21.15.12:0/3047792788 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d6fed42af0 0x55d6fed401d0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:06.780 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:06.778+0000 7effa2000640 0 client.25141 ms_handle_reset on v2:172.21.15.165:6836/6215088 2024-09-11T00:06:06.780 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:06.778+0000 7effc6000740 1 --2- 172.21.15.12:0/3047792788 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d6fed36b60 0x55d6fed3fa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:06.780 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:06.778+0000 7effc6000740 1 --2- 172.21.15.12:0/3047792788 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d6fed36240 0x55d6fed425b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:06.780 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:06.778+0000 7effc6000740 1 -- 172.21.15.12:0/3047792788 >> 172.21.15.12:0/3047792788 conn(0x55d6fed34020 msgr2=0x55d6fed39c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:06:06.780 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:06.778+0000 7effc6000740 1 -- 172.21.15.12:0/3047792788 shutdown_connections 2024-09-11T00:06:06.780 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:06.778+0000 7effc6000740 1 -- 172.21.15.12:0/3047792788 wait complete. 2024-09-11T00:06:06.780 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:06.778+0000 7effc6000740 20 client.25141 trim_cache size 0 max 16384 2024-09-11T00:06:06.782 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:06:06 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:06:06.782 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:06:06 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:06:06.782 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:06:06 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:06:06.782 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:06:06 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:06:06.813 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:06.812+0000 7effc6000740 1 Processor -- start 2024-09-11T00:06:06.814 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:06.813+0000 7effc6000740 1 -- start start 2024-09-11T00:06:06.814 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:06.813+0000 7effc6000740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d6fed36240 0x55d6fed3a400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:06:06.814 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:06.813+0000 7effc6000740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d6fed36b60 0x55d6fed531c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:06:06.815 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:06.813+0000 7effc6000740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d6fed42af0 0x55d6fed54630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:06:06.815 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:06.813+0000 7effc6000740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55d6fed53040 con 0x55d6fed42af0 2024-09-11T00:06:06.815 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:06.813+0000 7effc6000740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55d6fed53710 con 0x55d6fed36b60 2024-09-11T00:06:06.815 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:06.813+0000 7effc6000740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55d6fed53de0 con 0x55d6fed36240 2024-09-11T00:06:06.815 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:06.813+0000 7effbbe00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d6fed36b60 0x55d6fed531c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:06:06.815 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:06.813+0000 7effc1400640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d6fed42af0 0x55d6fed54630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:06:06.815 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:06.813+0000 7effbbe00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d6fed36b60 0x55d6fed531c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.12:53636/0 (socket says 172.21.15.12:53636) 2024-09-11T00:06:06.815 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:06.813+0000 7effc0a00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d6fed36240 0x55d6fed3a400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:06:06.815 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:06.813+0000 7effbbe00640 1 -- 172.21.15.12:0/1881162548 learned_addr learned my addr 172.21.15.12:0/1881162548 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:06:06.815 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:06.814+0000 7effc0a00640 1 -- 172.21.15.12:0/1881162548 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d6fed36b60 msgr2=0x55d6fed531c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:06:06.815 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:06.814+0000 7effc0a00640 1 --2- 172.21.15.12:0/1881162548 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d6fed36b60 0x55d6fed531c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:06.815 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:06.814+0000 7effc0a00640 1 -- 172.21.15.12:0/1881162548 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d6fed42af0 msgr2=0x55d6fed54630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:06:06.816 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:06.814+0000 7effc0a00640 1 --2- 172.21.15.12:0/1881162548 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d6fed42af0 0x55d6fed54630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:06.816 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:06.814+0000 7effc0a00640 1 -- 172.21.15.12:0/1881162548 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d6fed36990 con 0x55d6fed36240 2024-09-11T00:06:06.817 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:06.814+0000 7effbbe00640 1 --2- 172.21.15.12:0/1881162548 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d6fed36b60 0x55d6fed531c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:06:06.817 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:06.814+0000 7effc0a00640 1 --2- 172.21.15.12:0/1881162548 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d6fed36240 0x55d6fed3a400 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7effb0012fd0 tx=0x7effb0012d70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4441d43c6aa359f4 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:06:06.817 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:06.814+0000 7effad200640 1 -- 172.21.15.12:0/1881162548 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effb0012650 con 0x55d6fed36240 2024-09-11T00:06:06.817 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:06.815+0000 7effad200640 1 -- 172.21.15.12:0/1881162548 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7effb0016040 con 0x55d6fed36240 2024-09-11T00:06:06.817 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:06.815+0000 7effad200640 1 -- 172.21.15.12:0/1881162548 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effb00048e0 con 0x55d6fed36240 2024-09-11T00:06:06.817 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:06.815+0000 7effc6000740 1 -- 172.21.15.12:0/1881162548 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d6fed36240 msgr2=0x55d6fed3a400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:06:06.817 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:06.815+0000 7effc6000740 1 --2- 172.21.15.12:0/1881162548 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d6fed36240 0x55d6fed3a400 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7effb0012fd0 tx=0x7effb0012d70 comp rx=0 tx=0).stop 2024-09-11T00:06:06.817 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:06.815+0000 7effc6000740 1 -- 172.21.15.12:0/1881162548 shutdown_connections 2024-09-11T00:06:06.817 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:06.815+0000 7effc6000740 1 --2- 172.21.15.12:0/1881162548 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d6fed42af0 0x55d6fed54630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:06.817 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:06.815+0000 7effc6000740 1 --2- 172.21.15.12:0/1881162548 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d6fed36b60 0x55d6fed531c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:06.817 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:06.815+0000 7effc6000740 1 --2- 172.21.15.12:0/1881162548 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d6fed36240 0x55d6fed3a400 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:06.817 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:06.815+0000 7effc6000740 1 -- 172.21.15.12:0/1881162548 >> 172.21.15.12:0/1881162548 conn(0x55d6fed34020 msgr2=0x55d6fed3ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:06:06.817 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:06.816+0000 7effc6000740 1 -- 172.21.15.12:0/1881162548 shutdown_connections 2024-09-11T00:06:06.817 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:06.816+0000 7effc6000740 1 -- 172.21.15.12:0/1881162548 wait complete. 2024-09-11T00:06:06.818 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:06.816+0000 7effc6000740 1 -- start start 2024-09-11T00:06:06.818 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:06.817+0000 7effc6000740 1 Processor -- start 2024-09-11T00:06:06.818 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:06.817+0000 7effc6000740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d6fed36240 0x55d6fed5d7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:06:06.818 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:06.817+0000 7effc6000740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d6fed36b60 0x55d6fed56f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:06:06.818 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:06.817+0000 7effc6000740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d6fed42af0 0x55d6fed57490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:06:06.819 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:06.817+0000 7effc6000740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55d6fed54b80 con 0x55d6fed42af0 2024-09-11T00:06:06.819 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:06.817+0000 7effc6000740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55d6fed544b0 con 0x55d6fed36240 2024-09-11T00:06:06.819 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:06.817+0000 7effc6000740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55d6fed55250 con 0x55d6fed36b60 2024-09-11T00:06:06.819 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:06.817+0000 7effc6000740 10 client.0 _ll_register_callbacks cb 0x7effb4003e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:06:06.820 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:06.818+0000 7effc1400640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d6fed42af0 0x55d6fed57490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:06:06.820 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:06.818+0000 7effc0a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d6fed36240 0x55d6fed5d7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:06:06.820 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:06.818+0000 7effbbe00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d6fed36b60 0x55d6fed56f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:06:06.820 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:06.818+0000 7effc0a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d6fed36240 0x55d6fed5d7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.12:53648/0 (socket says 172.21.15.12:53648) 2024-09-11T00:06:06.820 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:06.818+0000 7effbbe00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d6fed36b60 0x55d6fed56f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.12:49398/0 (socket says 172.21.15.12:49398) 2024-09-11T00:06:06.820 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:06.818+0000 7effc0a00640 1 -- 172.21.15.12:0/4114479225 learned_addr learned my addr 172.21.15.12:0/4114479225 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:06:06.820 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:06.818+0000 7effc0a00640 1 -- 172.21.15.12:0/4114479225 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d6fed36b60 msgr2=0x55d6fed56f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:06:06.820 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:06.818+0000 7effc0a00640 1 --2- 172.21.15.12:0/4114479225 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d6fed36b60 0x55d6fed56f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:06.820 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:06.818+0000 7effc0a00640 1 -- 172.21.15.12:0/4114479225 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d6fed42af0 msgr2=0x55d6fed57490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:06:06.820 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:06.818+0000 7effc0a00640 1 --2- 172.21.15.12:0/4114479225 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d6fed42af0 0x55d6fed57490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:06.821 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:06.818+0000 7effc0a00640 1 -- 172.21.15.12:0/4114479225 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d6fed57cc0 con 0x55d6fed36240 2024-09-11T00:06:06.821 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:06.818+0000 7effbbe00640 1 --2- 172.21.15.12:0/4114479225 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d6fed36b60 0x55d6fed56f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:06:06.821 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:06.819+0000 7effc0a00640 1 --2- 172.21.15.12:0/4114479225 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d6fed36240 0x55d6fed5d7b0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7effb0009810 tx=0x7effb0009840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d8a2789442ae4721 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:06:06.821 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:06.819+0000 7effa2000640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-09-11T00:06:06.821 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:06.819+0000 7effa2000640 1 -- 172.21.15.12:0/4114479225 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effb0012450 con 0x55d6fed36240 2024-09-11T00:06:06.821 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:06.819+0000 7effc6000740 1 -- 172.21.15.12:0/4114479225 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7effb0016040 con 0x55d6fed36240 2024-09-11T00:06:06.822 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:06.819+0000 7effa2000640 1 -- 172.21.15.12:0/4114479225 <== mon.2 v2:172.21.15.165:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 9092+0+0 (secure 0 0 0) 0x7effb00125f0 con 0x55d6fed36240 2024-09-11T00:06:06.822 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:06.819+0000 7effa2000640 1 -- 172.21.15.12:0/4114479225 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7effb0045050 con 0x55d6fed36240 2024-09-11T00:06:06.822 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:06.819+0000 7effc6000740 10 client.25166 fetch_fsmap learned FSMap version 28 2024-09-11T00:06:06.822 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:06.819+0000 7effc6000740 1 -- 172.21.15.12:0/4114479225 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55d6fedc0230 con 0x55d6fed36240 2024-09-11T00:06:06.822 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:06.820+0000 7effa2000640 1 -- 172.21.15.12:0/4114479225 <== mon.2 v2:172.21.15.165:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7effb0016040 con 0x55d6fed36240 2024-09-11T00:06:06.822 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:06.820+0000 7effa2000640 1 -- 172.21.15.12:0/4114479225 <== mon.2 v2:172.21.15.165:3300/0 5 ==== fsmap(e 28) ==== 8370+0+0 (secure 0 0 0) 0x7effb0040030 con 0x55d6fed36240 2024-09-11T00:06:06.822 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:06.820+0000 7effc6000740 10 client.25166 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T00:06:06.822 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:06.820+0000 7effc6000740 10 client.25166 resolve_mds: resolved name 'h' to daemon mds.h 2024-09-11T00:06:06.822 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:06.820+0000 7effc6000740 20 client.25166 populate_metadata read hostname 'smithi012' 2024-09-11T00:06:06.823 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:06.820+0000 7effc6000740 1 --2- 172.21.15.12:0/4114479225 >> [v2:172.21.15.165:6836/6215088,v1:172.21.15.165:6837/6215088] conn(0x55d6fedc47c0 0x55d6fedc34d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:06:06.823 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:06.820+0000 7effc6000740 4 client.25166 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-11T00:06:06.823 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:06.820+0000 7effc6000740 1 -- 172.21.15.12:0/4114479225 --> [v2:172.21.15.165:6836/6215088,v1:172.21.15.165:6837/6215088] -- command(tid 0: {"prefix": "status"}) -- 0x55d6fecab750 con 0x55d6fedc47c0 2024-09-11T00:06:06.823 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:06.821+0000 7effbbe00640 1 --2- 172.21.15.12:0/4114479225 >> [v2:172.21.15.165:6836/6215088,v1:172.21.15.165:6837/6215088] conn(0x55d6fedc47c0 0x55d6fedc34d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:06:06.824 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:06.821+0000 7effbbe00640 1 --2- 172.21.15.12:0/4114479225 >> [v2:172.21.15.165:6836/6215088,v1:172.21.15.165:6837/6215088] conn(0x55d6fedc47c0 0x55d6fedc34d0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=71050fba4284b796 server_cookie=3ff4d63a4666a150 in_seq=0 out_seq=0 2024-09-11T00:06:06.824 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:06.821+0000 7effa2000640 10 client.25166 ms_handle_connect on v2:172.21.15.165:6836/6215088 2024-09-11T00:06:06.824 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:06.822+0000 7effa2000640 1 -- 172.21.15.12:0/4114479225 <== mds.2 v2:172.21.15.165:6836/6215088 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55d6fecab750 con 0x55d6fedc47c0 2024-09-11T00:06:06.824 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:06.822+0000 7effa2000640 10 client.25166 handle_command_reply: tid=0 2024-09-11T00:06:06.824 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:06.822+0000 7effc6000740 1 client.25166 shutdown 2024-09-11T00:06:06.824 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:06.822+0000 7effc6000740 1 -- 172.21.15.12:0/4114479225 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d6fed36240 msgr2=0x55d6fed5d7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:06:06.824 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:06.822+0000 7effc6000740 1 --2- 172.21.15.12:0/4114479225 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d6fed36240 0x55d6fed5d7b0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7effb0009810 tx=0x7effb0009840 comp rx=0 tx=0).stop 2024-09-11T00:06:06.824 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:06.822+0000 7effc6000740 1 -- 172.21.15.12:0/4114479225 shutdown_connections 2024-09-11T00:06:06.824 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:06.822+0000 7effc6000740 1 --2- 172.21.15.12:0/4114479225 >> [v2:172.21.15.165:6836/6215088,v1:172.21.15.165:6837/6215088] conn(0x55d6fedc47c0 0x55d6fedc34d0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:06.825 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:06.822+0000 7effc6000740 1 --2- 172.21.15.12:0/4114479225 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d6fed42af0 0x55d6fed57490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:06.825 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:06.822+0000 7effc6000740 1 --2- 172.21.15.12:0/4114479225 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d6fed36b60 0x55d6fed56f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:06.825 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:06.822+0000 7effc6000740 1 --2- 172.21.15.12:0/4114479225 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d6fed36240 0x55d6fed5d7b0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:06.825 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:06.822+0000 7effc6000740 1 -- 172.21.15.12:0/4114479225 >> 172.21.15.12:0/4114479225 conn(0x55d6fed34020 msgr2=0x55d6fed3ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:06:06.825 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:06.822+0000 7effa2000640 0 client.25166 ms_handle_reset on v2:172.21.15.165:6836/6215088 2024-09-11T00:06:06.825 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:06.822+0000 7effc6000740 1 -- 172.21.15.12:0/4114479225 shutdown_connections 2024-09-11T00:06:06.825 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:06.822+0000 7effc6000740 1 -- 172.21.15.12:0/4114479225 wait complete. 2024-09-11T00:06:06.825 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:06.823+0000 7effc6000740 20 client.25166 trim_cache size 0 max 16384 2024-09-11T00:06:06.825 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-09-11T00:06:06.825 INFO:teuthology.orchestra.run.smithi012.stdout: "cluster_fsid": "ff3ad74a-6fcf-11ef-bcea-c7b262605968", 2024-09-11T00:06:06.826 INFO:teuthology.orchestra.run.smithi012.stdout: "whoami": 2, 2024-09-11T00:06:06.826 INFO:teuthology.orchestra.run.smithi012.stdout: "id": 24638, 2024-09-11T00:06:06.826 INFO:teuthology.orchestra.run.smithi012.stdout: "want_state": "up:active", 2024-09-11T00:06:06.826 INFO:teuthology.orchestra.run.smithi012.stdout: "state": "up:active", 2024-09-11T00:06:06.826 INFO:teuthology.orchestra.run.smithi012.stdout: "fs_name": "cephfs", 2024-09-11T00:06:06.826 INFO:teuthology.orchestra.run.smithi012.stdout: "rank_uptime": 79.594402880999993, 2024-09-11T00:06:06.826 INFO:teuthology.orchestra.run.smithi012.stdout: "mdsmap_epoch": 28, 2024-09-11T00:06:06.826 INFO:teuthology.orchestra.run.smithi012.stdout: "osdmap_epoch": 83, 2024-09-11T00:06:06.826 INFO:teuthology.orchestra.run.smithi012.stdout: "osdmap_epoch_barrier": 81, 2024-09-11T00:06:06.826 INFO:teuthology.orchestra.run.smithi012.stdout: "uptime": 147.306383385 2024-09-11T00:06:06.826 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-09-11T00:06:06.827 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:06.823+0000 7effc3400640 1 -- 172.21.15.12:0/821579874 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7effa40b54c0 msgr2=0x7effa40b7980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:06:06.827 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:06.823+0000 7effc3400640 1 --2- 172.21.15.12:0/821579874 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7effa40b54c0 0x7effa40b7980 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7effbc06c140 tx=0x7effb4009500 comp rx=0 tx=0).stop 2024-09-11T00:06:06.827 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:06.823+0000 7effc3400640 1 -- 172.21.15.12:0/821579874 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7effbc077ff0 msgr2=0x7effbc19fe10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:06:06.827 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:06.823+0000 7effc3400640 1 --2- 172.21.15.12:0/821579874 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7effbc077ff0 0x7effbc19fe10 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7effa802a9e0 tx=0x7effa8003b90 comp rx=0 tx=0).stop 2024-09-11T00:06:06.827 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:06.823+0000 7effc3400640 1 -- 172.21.15.12:0/821579874 shutdown_connections 2024-09-11T00:06:06.827 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:06.823+0000 7effc3400640 1 --2- 172.21.15.12:0/821579874 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7effa40b54c0 0x7effa40b7980 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:06.827 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:06.824+0000 7effc3400640 1 --2- 172.21.15.12:0/821579874 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7effbc1139b0 0x7effbc1a51b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:06.827 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:06.824+0000 7effc3400640 1 --2- 172.21.15.12:0/821579874 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7effbc077ff0 0x7effbc19fe10 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:06.827 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:06.824+0000 7effc3400640 1 --2- 172.21.15.12:0/821579874 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7effbc0776d0 0x7effbc19f8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:06.827 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:06.824+0000 7effc3400640 1 -- 172.21.15.12:0/821579874 >> 172.21.15.12:0/821579874 conn(0x7effbc1009f0 msgr2=0x7effbc102de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:06:06.827 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:06.824+0000 7effc3400640 1 -- 172.21.15.12:0/821579874 shutdown_connections 2024-09-11T00:06:06.828 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:06.824+0000 7effc3400640 1 -- 172.21.15.12:0/821579874 wait complete. 2024-09-11T00:06:06.830 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:06:06 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:06:06.830 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:06:06 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:06:06.830 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:06:06 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:06:06.831 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:06:06 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:06:07.782 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:06:07 smithi165 ceph-mon[35356]: pgmap v367: 129 pgs: 129 active+clean; 647 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-11T00:06:07.790 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:06:07 smithi012 ceph-mon[30333]: pgmap v367: 129 pgs: 129 active+clean; 647 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-11T00:06:07.829 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid ff3ad74a-6fcf-11ef-bcea-c7b262605968 -- ceph tell mds.g status 2024-09-11T00:06:07.830 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:06:07 smithi178 ceph-mon[35919]: pgmap v367: 129 pgs: 129 active+clean; 647 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-11T00:06:08.277 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:06:08.855 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:08.850+0000 7f9dd5c00640 1 -- 172.21.15.12:0/1037910981 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9dd0077ff0 msgr2=0x7f9dd0113470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:06:08.856 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:08.850+0000 7f9dd5c00640 1 --2- 172.21.15.12:0/1037910981 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9dd0077ff0 0x7f9dd0113470 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f9dbc009900 tx=0x7f9dbc02a3f0 comp rx=0 tx=0).stop 2024-09-11T00:06:08.856 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:08.850+0000 7f9dd5c00640 1 -- 172.21.15.12:0/1037910981 shutdown_connections 2024-09-11T00:06:08.856 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:08.850+0000 7f9dd5c00640 1 --2- 172.21.15.12:0/1037910981 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9dd01139b0 0x7f9dd0115da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:08.856 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:08.850+0000 7f9dd5c00640 1 --2- 172.21.15.12:0/1037910981 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9dd0077ff0 0x7f9dd0113470 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:08.857 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:08.850+0000 7f9dd5c00640 1 --2- 172.21.15.12:0/1037910981 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9dd00776d0 0x7f9dd0077ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:08.857 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:08.850+0000 7f9dd5c00640 1 -- 172.21.15.12:0/1037910981 >> 172.21.15.12:0/1037910981 conn(0x7f9dd01009f0 msgr2=0x7f9dd0102e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:06:08.857 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:08.850+0000 7f9dd5c00640 1 -- 172.21.15.12:0/1037910981 shutdown_connections 2024-09-11T00:06:08.857 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:08.850+0000 7f9dd5c00640 1 -- 172.21.15.12:0/1037910981 wait complete. 2024-09-11T00:06:08.858 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:08.851+0000 7f9dd5c00640 1 Processor -- start 2024-09-11T00:06:08.859 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:08.852+0000 7f9dd5c00640 1 -- start start 2024-09-11T00:06:08.859 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:08.852+0000 7f9dd5c00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9dd00776d0 0x7f9dd019f8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:06:08.859 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:08.852+0000 7f9dd5c00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9dd0077ff0 0x7f9dd019fe10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:06:08.859 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:08.852+0000 7f9dd5c00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9dd01139b0 0x7f9dd01a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:06:08.860 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:08.852+0000 7f9dd5c00640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f9dd0119400 con 0x7f9dd0077ff0 2024-09-11T00:06:08.860 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:08.852+0000 7f9dd5c00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f9dd0119280 con 0x7f9dd00776d0 2024-09-11T00:06:08.860 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:08.852+0000 7f9dd5c00640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f9dd0119580 con 0x7f9dd01139b0 2024-09-11T00:06:08.860 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:08.852+0000 7f9dcea00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9dd00776d0 0x7f9dd019f8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:06:08.860 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:08.852+0000 7f9dce000640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9dd0077ff0 0x7f9dd019fe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:06:08.861 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:08.852+0000 7f9dcea00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9dd00776d0 0x7f9dd019f8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.12:53662/0 (socket says 172.21.15.12:53662) 2024-09-11T00:06:08.861 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:08.852+0000 7f9dcea00640 1 -- 172.21.15.12:0/4024788476 learned_addr learned my addr 172.21.15.12:0/4024788476 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:06:08.861 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:08.853+0000 7f9dcf400640 1 --2- 172.21.15.12:0/4024788476 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9dd01139b0 0x7f9dd01a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:06:08.861 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:08.853+0000 7f9dcea00640 1 -- 172.21.15.12:0/4024788476 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9dd01139b0 msgr2=0x7f9dd01a51b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:06:08.861 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:08.853+0000 7f9dcea00640 1 --2- 172.21.15.12:0/4024788476 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9dd01139b0 0x7f9dd01a51b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:08.861 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:08.853+0000 7f9dcea00640 1 -- 172.21.15.12:0/4024788476 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9dd0077ff0 msgr2=0x7f9dd019fe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:06:08.862 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:08.853+0000 7f9dcea00640 1 --2- 172.21.15.12:0/4024788476 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9dd0077ff0 0x7f9dd019fe10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:08.862 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:08.853+0000 7f9dcea00640 1 -- 172.21.15.12:0/4024788476 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9dd01a5980 con 0x7f9dd00776d0 2024-09-11T00:06:08.862 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:08.853+0000 7f9dcf400640 1 --2- 172.21.15.12:0/4024788476 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9dd01139b0 0x7f9dd01a51b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T00:06:08.862 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:08.853+0000 7f9dcea00640 1 --2- 172.21.15.12:0/4024788476 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9dd00776d0 0x7f9dd019f8d0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f9dc000e9f0 tx=0x7f9dc000edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:06:08.862 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:08.853+0000 7f9dc7400640 1 -- 172.21.15.12:0/4024788476 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9dc000cd80 con 0x7f9dd00776d0 2024-09-11T00:06:08.862 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:08.854+0000 7f9dd5c00640 1 -- 172.21.15.12:0/4024788476 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9dd01a5c70 con 0x7f9dd00776d0 2024-09-11T00:06:08.862 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:08.854+0000 7f9dc7400640 1 -- 172.21.15.12:0/4024788476 <== mon.2 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9dc00054d0 con 0x7f9dd00776d0 2024-09-11T00:06:08.862 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:08.854+0000 7f9dc7400640 1 -- 172.21.15.12:0/4024788476 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9dc0010690 con 0x7f9dd00776d0 2024-09-11T00:06:08.862 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:08.854+0000 7f9dd5c00640 1 -- 172.21.15.12:0/4024788476 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9dd01ac330 con 0x7f9dd00776d0 2024-09-11T00:06:08.867 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:08.857+0000 7f9dd8850740 1 Processor -- start 2024-09-11T00:06:08.867 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:08.858+0000 7f9dd8850740 1 -- start start 2024-09-11T00:06:08.867 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:08.858+0000 7f9dd8850740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b35e182240 0x55b35e182620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:06:08.867 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:08.858+0000 7f9dd8850740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b35e182b60 0x55b35e18e5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:06:08.868 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:08.858+0000 7f9dd8850740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b35e18eaf0 0x55b35e190ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:06:08.868 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:08.858+0000 7f9dd8850740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55b35e0f9d50 con 0x55b35e182240 2024-09-11T00:06:08.868 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:08.858+0000 7f9dd8850740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55b35e0e9180 con 0x55b35e18eaf0 2024-09-11T00:06:08.868 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:08.858+0000 7f9dd8850740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55b35e0ecf90 con 0x55b35e182b60 2024-09-11T00:06:08.868 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:08.859+0000 7f9dcf400640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b35e182240 0x55b35e182620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:06:08.868 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:08.859+0000 7f9dcf400640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b35e182240 0x55b35e182620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:38776/0 (socket says 172.21.15.12:38776) 2024-09-11T00:06:08.868 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:08.859+0000 7f9dcf400640 1 -- 172.21.15.12:0/126209227 learned_addr learned my addr 172.21.15.12:0/126209227 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:06:08.868 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:08.859+0000 7f9dce000640 1 --2- 172.21.15.12:0/126209227 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b35e18eaf0 0x55b35e190ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:06:08.869 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:08.859+0000 7f9dc7400640 1 -- 172.21.15.12:0/4024788476 <== mon.2 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 17) ==== 153301+0+0 (secure 0 0 0) 0x7f9dc0004e90 con 0x7f9dd00776d0 2024-09-11T00:06:08.869 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:08.859+0000 7f9dcf400640 1 -- 172.21.15.12:0/126209227 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b35e182b60 msgr2=0x55b35e18e5b0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-11T00:06:08.869 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:08.859+0000 7f9dcf400640 1 --2- 172.21.15.12:0/126209227 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b35e182b60 0x55b35e18e5b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:08.869 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:08.859+0000 7f9dcf400640 1 -- 172.21.15.12:0/126209227 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b35e18eaf0 msgr2=0x55b35e190ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:06:08.869 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:08.859+0000 7f9dcf400640 1 --2- 172.21.15.12:0/126209227 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b35e18eaf0 0x55b35e190ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:08.869 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:08.859+0000 7f9dcf400640 1 -- 172.21.15.12:0/126209227 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b35e0b3a70 con 0x55b35e182240 2024-09-11T00:06:08.869 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:08.859+0000 7f9dce000640 1 --2- 172.21.15.12:0/126209227 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b35e18eaf0 0x55b35e190ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T00:06:08.869 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:08.859+0000 7f9dcf400640 1 --2- 172.21.15.12:0/126209227 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b35e182240 0x55b35e182620 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f9dc80104b0 tx=0x7f9dc8009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=130655b13daeeefa server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:06:08.869 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:08.860+0000 7f9dbbe00640 1 -- 172.21.15.12:0/126209227 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9dc8017070 con 0x55b35e182240 2024-09-11T00:06:08.869 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:08.860+0000 7f9dbbe00640 1 -- 172.21.15.12:0/126209227 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9dc8011c10 con 0x55b35e182240 2024-09-11T00:06:08.869 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:08.860+0000 7f9dbbe00640 1 -- 172.21.15.12:0/126209227 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9dc801bd30 con 0x55b35e182240 2024-09-11T00:06:08.870 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:08.861+0000 7f9dd8850740 1 -- 172.21.15.12:0/126209227 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b35e182240 msgr2=0x55b35e182620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:06:08.870 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:08.861+0000 7f9dd8850740 1 --2- 172.21.15.12:0/126209227 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b35e182240 0x55b35e182620 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f9dc80104b0 tx=0x7f9dc8009f90 comp rx=0 tx=0).stop 2024-09-11T00:06:08.870 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:08.861+0000 7f9dc7400640 1 --2- 172.21.15.12:0/4024788476 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f9db40b5370 0x7f9db40b7830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:06:08.870 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:08.861+0000 7f9dc7400640 1 -- 172.21.15.12:0/4024788476 <== mon.2 v2:172.21.15.165:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 9092+0+0 (secure 0 0 0) 0x7f9dc00e65b0 con 0x7f9dd00776d0 2024-09-11T00:06:08.870 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:08.861+0000 7f9dd8850740 1 -- 172.21.15.12:0/126209227 shutdown_connections 2024-09-11T00:06:08.870 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:08.861+0000 7f9dd8850740 1 --2- 172.21.15.12:0/126209227 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b35e18eaf0 0x55b35e190ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:08.870 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:08.861+0000 7f9dd8850740 1 --2- 172.21.15.12:0/126209227 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b35e182b60 0x55b35e18e5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:08.870 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:08.861+0000 7f9dd8850740 1 --2- 172.21.15.12:0/126209227 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b35e182240 0x55b35e182620 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:08.870 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:08.861+0000 7f9dd8850740 1 -- 172.21.15.12:0/126209227 >> 172.21.15.12:0/126209227 conn(0x55b35e180020 msgr2=0x55b35e181210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:06:08.870 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:08.861+0000 7f9dcf400640 1 --2- 172.21.15.12:0/4024788476 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f9db40b5370 0x7f9db40b7830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:06:08.870 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:08.862+0000 7f9dd8850740 1 -- 172.21.15.12:0/126209227 shutdown_connections 2024-09-11T00:06:08.871 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:08.862+0000 7f9dd8850740 1 -- 172.21.15.12:0/126209227 wait complete. 2024-09-11T00:06:08.871 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:08.862+0000 7f9dcf400640 1 --2- 172.21.15.12:0/4024788476 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f9db40b5370 0x7f9db40b7830 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f9dc801ba10 tx=0x7f9dc8000f40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:06:08.871 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:08.862+0000 7f9dd8850740 1 -- start start 2024-09-11T00:06:08.871 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:08.862+0000 7f9dd8850740 1 Processor -- start 2024-09-11T00:06:08.871 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:08.863+0000 7f9dd8850740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b35e182240 0x55b35e18bac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:06:08.871 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:08.863+0000 7f9dd8850740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b35e182b60 0x55b35e18c000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:06:08.871 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:08.863+0000 7f9dd8850740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b35e18eaf0 0x55b35e18c540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:06:08.871 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:08.863+0000 7f9dd8850740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55b35df4a5f0 con 0x55b35e18eaf0 2024-09-11T00:06:08.871 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:08.863+0000 7f9dd8850740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55b35e084460 con 0x55b35e182b60 2024-09-11T00:06:08.871 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:08.863+0000 7f9dd8850740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55b35e191840 con 0x55b35e182240 2024-09-11T00:06:08.871 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:08.863+0000 7f9dd8850740 10 client.0 _ll_register_callbacks cb 0x55b35e094210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:06:08.872 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:08.863+0000 7f9dcea00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b35e182240 0x55b35e18bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:06:08.872 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:08.863+0000 7f9dcea00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b35e182240 0x55b35e18bac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.12:49428/0 (socket says 172.21.15.12:49428) 2024-09-11T00:06:08.872 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:08.863+0000 7f9dcea00640 1 -- 172.21.15.12:0/2260970808 learned_addr learned my addr 172.21.15.12:0/2260970808 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:06:08.872 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:08.863+0000 7f9dce000640 1 --2- 172.21.15.12:0/2260970808 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b35e182b60 0x55b35e18c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:06:08.872 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:08.863+0000 7f9dcf400640 1 --2- 172.21.15.12:0/2260970808 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b35e18eaf0 0x55b35e18c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:06:08.872 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:08.864+0000 7f9dcea00640 1 -- 172.21.15.12:0/2260970808 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b35e182b60 msgr2=0x55b35e18c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:06:08.872 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:08.864+0000 7f9dcea00640 1 --2- 172.21.15.12:0/2260970808 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b35e182b60 0x55b35e18c000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:08.872 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:08.864+0000 7f9dcea00640 1 -- 172.21.15.12:0/2260970808 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b35e18eaf0 msgr2=0x55b35e18c540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:06:08.872 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:08.864+0000 7f9dcea00640 1 --2- 172.21.15.12:0/2260970808 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b35e18eaf0 0x55b35e18c540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:08.872 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:08.864+0000 7f9dcea00640 1 -- 172.21.15.12:0/2260970808 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b35de47540 con 0x55b35e182240 2024-09-11T00:06:08.873 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:08.864+0000 7f9dcf400640 1 --2- 172.21.15.12:0/2260970808 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b35e18eaf0 0x55b35e18c540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T00:06:08.873 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:08.864+0000 7f9dcea00640 1 --2- 172.21.15.12:0/2260970808 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b35e182240 0x55b35e18bac0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f9dc000c6a0 tx=0x7f9dc00048f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=46fbdb0ce5e6482b server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:06:08.873 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:08.864+0000 7f9db8c00640 10 client.0 ms_handle_connect on v2:172.21.15.178:3300/0 2024-09-11T00:06:08.873 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:08.864+0000 7f9db8c00640 1 -- 172.21.15.12:0/2260970808 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9dc00b1070 con 0x55b35e182240 2024-09-11T00:06:08.873 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:08.864+0000 7f9db8c00640 1 -- 172.21.15.12:0/2260970808 <== mon.1 v2:172.21.15.178:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 9092+0+0 (secure 0 0 0) 0x7f9dc00e3df0 con 0x55b35e182240 2024-09-11T00:06:08.873 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:08.864+0000 7f9dd8850740 1 -- 172.21.15.12:0/2260970808 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55b35e086a70 con 0x55b35e182240 2024-09-11T00:06:08.873 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:08.865+0000 7f9db8c00640 1 -- 172.21.15.12:0/2260970808 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f9dc00ac360 con 0x55b35e182240 2024-09-11T00:06:08.873 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:08.865+0000 7f9dd8850740 10 client.25156 fetch_fsmap learned FSMap version 28 2024-09-11T00:06:08.873 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:08.865+0000 7f9dd8850740 1 -- 172.21.15.12:0/2260970808 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55b35e0a16f0 con 0x55b35e182240 2024-09-11T00:06:08.873 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:08.865+0000 7f9db8c00640 1 -- 172.21.15.12:0/2260970808 <== mon.1 v2:172.21.15.178:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55b35e086a70 con 0x55b35e182240 2024-09-11T00:06:08.873 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:08.865+0000 7f9db8c00640 1 -- 172.21.15.12:0/2260970808 <== mon.1 v2:172.21.15.178:3300/0 5 ==== fsmap(e 28) ==== 8370+0+0 (secure 0 0 0) 0x7f9dc00b5950 con 0x55b35e182240 2024-09-11T00:06:08.873 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:08.865+0000 7f9dd8850740 10 client.25156 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T00:06:08.874 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:08.865+0000 7f9dd8850740 10 client.25156 resolve_mds: resolved name 'g' to daemon mds.g 2024-09-11T00:06:08.874 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:08.865+0000 7f9dd8850740 20 client.25156 populate_metadata read hostname 'smithi012' 2024-09-11T00:06:08.874 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:08.865+0000 7f9dd8850740 1 --2- 172.21.15.12:0/2260970808 >> [v2:172.21.15.12:6838/3124093009,v1:172.21.15.12:6839/3124093009] conn(0x55b35e19f7f0 0x55b35e1a0c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:06:08.874 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:08.865+0000 7f9dd8850740 4 client.25156 mds_command: new command op to 24643 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T00:06:08.874 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:08.865+0000 7f9dd8850740 1 -- 172.21.15.12:0/2260970808 --> [v2:172.21.15.12:6838/3124093009,v1:172.21.15.12:6839/3124093009] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b35e0fbca0 con 0x55b35e19f7f0 2024-09-11T00:06:08.874 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:08.866+0000 7f9dce000640 1 --2- 172.21.15.12:0/2260970808 >> [v2:172.21.15.12:6838/3124093009,v1:172.21.15.12:6839/3124093009] conn(0x55b35e19f7f0 0x55b35e1a0c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:06:08.874 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:08.866+0000 7f9dce000640 1 --2- 172.21.15.12:0/2260970808 >> [v2:172.21.15.12:6838/3124093009,v1:172.21.15.12:6839/3124093009] conn(0x55b35e19f7f0 0x55b35e1a0c60 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=cbc7d199280742be server_cookie=28fc18c7d98ded81 in_seq=0 out_seq=0 2024-09-11T00:06:08.874 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:08.866+0000 7f9db8c00640 10 client.25156 ms_handle_connect on v2:172.21.15.12:6838/3124093009 2024-09-11T00:06:08.875 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:08.868+0000 7f9db8c00640 1 -- 172.21.15.12:0/2260970808 <== mds.1 v2:172.21.15.12:6838/3124093009 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55b35e0fbca0 con 0x55b35e19f7f0 2024-09-11T00:06:08.875 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:08.868+0000 7f9db8c00640 10 client.25156 handle_command_reply: tid=0 2024-09-11T00:06:08.875 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:08.868+0000 7f9dd8850740 1 client.25156 shutdown 2024-09-11T00:06:08.875 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:08.868+0000 7f9dd8850740 1 -- 172.21.15.12:0/2260970808 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b35e182240 msgr2=0x55b35e18bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:06:08.876 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:08.868+0000 7f9dd8850740 1 --2- 172.21.15.12:0/2260970808 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b35e182240 0x55b35e18bac0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f9dc000c6a0 tx=0x7f9dc00048f0 comp rx=0 tx=0).stop 2024-09-11T00:06:08.876 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:08.868+0000 7f9dd8850740 1 -- 172.21.15.12:0/2260970808 shutdown_connections 2024-09-11T00:06:08.876 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:08.868+0000 7f9dd8850740 1 --2- 172.21.15.12:0/2260970808 >> [v2:172.21.15.12:6838/3124093009,v1:172.21.15.12:6839/3124093009] conn(0x55b35e19f7f0 0x55b35e1a0c60 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:08.876 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:08.868+0000 7f9dd8850740 1 --2- 172.21.15.12:0/2260970808 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b35e18eaf0 0x55b35e18c540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:08.876 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:08.868+0000 7f9dd8850740 1 --2- 172.21.15.12:0/2260970808 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b35e182b60 0x55b35e18c000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:08.876 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:08.868+0000 7f9dd8850740 1 --2- 172.21.15.12:0/2260970808 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b35e182240 0x55b35e18bac0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:08.876 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:08.868+0000 7f9dd8850740 1 -- 172.21.15.12:0/2260970808 >> 172.21.15.12:0/2260970808 conn(0x55b35e180020 msgr2=0x55b35e185c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:06:08.876 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:08.868+0000 7f9db8c00640 0 client.25156 ms_handle_reset on v2:172.21.15.12:6838/3124093009 2024-09-11T00:06:08.876 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:08.868+0000 7f9dd8850740 1 -- 172.21.15.12:0/2260970808 shutdown_connections 2024-09-11T00:06:08.876 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:08.868+0000 7f9dd8850740 1 -- 172.21.15.12:0/2260970808 wait complete. 2024-09-11T00:06:08.877 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:08.869+0000 7f9dd8850740 20 client.25156 trim_cache size 0 max 16384 2024-09-11T00:06:08.904 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:08.901+0000 7f9dd8850740 1 Processor -- start 2024-09-11T00:06:08.904 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:08.901+0000 7f9dd8850740 1 -- start start 2024-09-11T00:06:08.904 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:08.901+0000 7f9dd8850740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b35e182240 0x55b35e186290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:06:08.904 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:08.902+0000 7f9dd8850740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b35e182b60 0x55b35e186d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:06:08.904 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:08.902+0000 7f9dd8850740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b35e18eaf0 0x55b35e187270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:06:08.904 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:08.902+0000 7f9dd8850740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55b35e19efa0 con 0x55b35e182240 2024-09-11T00:06:08.904 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:08.902+0000 7f9dd8850740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55b35e19f670 con 0x55b35e18eaf0 2024-09-11T00:06:08.904 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:08.902+0000 7f9dd8850740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55b35e19fd40 con 0x55b35e182b60 2024-09-11T00:06:08.905 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:08.902+0000 7f9dcea00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b35e182240 0x55b35e186290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:06:08.905 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:08.902+0000 7f9dcea00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b35e182240 0x55b35e186290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:50880/0 (socket says 172.21.15.12:50880) 2024-09-11T00:06:08.905 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:08.902+0000 7f9dcea00640 1 -- 172.21.15.12:0/3405025683 learned_addr learned my addr 172.21.15.12:0/3405025683 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:06:08.905 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:08.902+0000 7f9dce000640 1 --2- 172.21.15.12:0/3405025683 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b35e182b60 0x55b35e186d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:06:08.905 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:08.902+0000 7f9dcf400640 1 --2- 172.21.15.12:0/3405025683 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b35e18eaf0 0x55b35e187270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:06:08.905 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:08.902+0000 7f9dcea00640 1 -- 172.21.15.12:0/3405025683 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b35e182b60 msgr2=0x55b35e186d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:06:08.905 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:08.902+0000 7f9dcea00640 1 --2- 172.21.15.12:0/3405025683 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b35e182b60 0x55b35e186d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:08.905 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:08.902+0000 7f9dcea00640 1 -- 172.21.15.12:0/3405025683 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b35e18eaf0 msgr2=0x55b35e187270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:06:08.905 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:08.902+0000 7f9dcea00640 1 --2- 172.21.15.12:0/3405025683 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b35e18eaf0 0x55b35e187270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:08.905 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:08.902+0000 7f9dcea00640 1 -- 172.21.15.12:0/3405025683 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b35e17fe50 con 0x55b35e182240 2024-09-11T00:06:08.905 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:08.903+0000 7f9dcea00640 1 --2- 172.21.15.12:0/3405025683 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b35e182240 0x55b35e186290 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f9dc00ac500 tx=0x7f9dc00057e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7857586f7b85764 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:06:08.905 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:08.903+0000 7f9dbb400640 1 -- 172.21.15.12:0/3405025683 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9dc00b1050 con 0x55b35e182240 2024-09-11T00:06:08.905 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:08.903+0000 7f9dbb400640 1 -- 172.21.15.12:0/3405025683 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9dc00acf50 con 0x55b35e182240 2024-09-11T00:06:08.906 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:08.903+0000 7f9dbb400640 1 -- 172.21.15.12:0/3405025683 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9dc00ac850 con 0x55b35e182240 2024-09-11T00:06:08.907 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:08.904+0000 7f9dd8850740 1 -- 172.21.15.12:0/3405025683 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b35e182240 msgr2=0x55b35e186290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:06:08.907 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:08.904+0000 7f9dd8850740 1 --2- 172.21.15.12:0/3405025683 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b35e182240 0x55b35e186290 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f9dc00ac500 tx=0x7f9dc00057e0 comp rx=0 tx=0).stop 2024-09-11T00:06:08.907 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:08.904+0000 7f9dd8850740 1 -- 172.21.15.12:0/3405025683 shutdown_connections 2024-09-11T00:06:08.907 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:08.904+0000 7f9dd8850740 1 --2- 172.21.15.12:0/3405025683 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b35e18eaf0 0x55b35e187270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:08.907 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:08.904+0000 7f9dd8850740 1 --2- 172.21.15.12:0/3405025683 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b35e182b60 0x55b35e186d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:08.907 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:08.904+0000 7f9dd8850740 1 --2- 172.21.15.12:0/3405025683 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b35e182240 0x55b35e186290 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:08.907 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:08.904+0000 7f9dd8850740 1 -- 172.21.15.12:0/3405025683 >> 172.21.15.12:0/3405025683 conn(0x55b35e180020 msgr2=0x55b35e181180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:06:08.907 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:08.904+0000 7f9dd8850740 1 -- 172.21.15.12:0/3405025683 shutdown_connections 2024-09-11T00:06:08.908 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:08.904+0000 7f9dd8850740 1 -- 172.21.15.12:0/3405025683 wait complete. 2024-09-11T00:06:08.908 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:08.905+0000 7f9dd8850740 1 -- start start 2024-09-11T00:06:08.908 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:08.905+0000 7f9dd8850740 1 Processor -- start 2024-09-11T00:06:08.908 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:08.905+0000 7f9dd8850740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b35e182240 0x55b35e1a9400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:06:08.908 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:08.905+0000 7f9dd8850740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b35e182b60 0x55b35e1a28d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:06:08.908 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:08.905+0000 7f9dd8850740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b35e18eaf0 0x55b35e1a2e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:06:08.908 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:08.905+0000 7f9dd8850740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55b35e1a0ae0 con 0x55b35e182b60 2024-09-11T00:06:08.908 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:08.905+0000 7f9dd8850740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55b35e1a0410 con 0x55b35e18eaf0 2024-09-11T00:06:08.908 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:08.905+0000 7f9dd8850740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55b35e1a11b0 con 0x55b35e182240 2024-09-11T00:06:08.908 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:08.905+0000 7f9dd8850740 10 client.0 _ll_register_callbacks cb 0x7f9dc00a8ea0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:06:08.908 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:08.906+0000 7f9dce000640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b35e182b60 0x55b35e1a28d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:06:08.908 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:08.906+0000 7f9dce000640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b35e182b60 0x55b35e1a28d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:50886/0 (socket says 172.21.15.12:50886) 2024-09-11T00:06:08.909 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:08.906+0000 7f9dcea00640 1 --2- 172.21.15.12:0/1662941401 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b35e182240 0x55b35e1a9400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:06:08.909 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:08.906+0000 7f9dce000640 1 -- 172.21.15.12:0/1662941401 learned_addr learned my addr 172.21.15.12:0/1662941401 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:06:08.909 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:08.906+0000 7f9dcf400640 1 --2- 172.21.15.12:0/1662941401 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b35e18eaf0 0x55b35e1a2e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:06:08.909 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:08.906+0000 7f9dcea00640 1 -- 172.21.15.12:0/1662941401 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b35e18eaf0 msgr2=0x55b35e1a2e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:06:08.909 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:08.906+0000 7f9dcea00640 1 --2- 172.21.15.12:0/1662941401 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b35e18eaf0 0x55b35e1a2e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:08.909 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:08.906+0000 7f9dcea00640 1 -- 172.21.15.12:0/1662941401 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b35e182b60 msgr2=0x55b35e1a28d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:06:08.909 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:08.906+0000 7f9dcea00640 1 --2- 172.21.15.12:0/1662941401 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b35e182b60 0x55b35e1a28d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:08.909 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:08.906+0000 7f9dcea00640 1 -- 172.21.15.12:0/1662941401 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b35e180f60 con 0x55b35e182240 2024-09-11T00:06:08.912 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:08.906+0000 7f9dcea00640 1 --2- 172.21.15.12:0/1662941401 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b35e182240 0x55b35e1a9400 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f9dc00a8010 tx=0x7f9dc00e3c50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=53dd003fd5042f1f server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:06:08.912 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:08.906+0000 7f9db8c00640 10 client.0 ms_handle_connect on v2:172.21.15.178:3300/0 2024-09-11T00:06:08.912 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:08.907+0000 7f9db8c00640 1 -- 172.21.15.12:0/1662941401 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9dc00b1040 con 0x55b35e182240 2024-09-11T00:06:08.912 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:08.907+0000 7f9db8c00640 1 -- 172.21.15.12:0/1662941401 <== mon.1 v2:172.21.15.178:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 9092+0+0 (secure 0 0 0) 0x7f9dc00e3df0 con 0x55b35e182240 2024-09-11T00:06:08.912 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:08.907+0000 7f9dd8850740 1 -- 172.21.15.12:0/1662941401 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9dc00acf50 con 0x55b35e182240 2024-09-11T00:06:08.912 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:08.907+0000 7f9db8c00640 1 -- 172.21.15.12:0/1662941401 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f9dc00ac1a0 con 0x55b35e182240 2024-09-11T00:06:08.913 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:08.907+0000 7f9dd8850740 10 client.25162 fetch_fsmap learned FSMap version 28 2024-09-11T00:06:08.913 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:08.907+0000 7f9dd8850740 1 -- 172.21.15.12:0/1662941401 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55b35e1a8c40 con 0x55b35e182240 2024-09-11T00:06:08.913 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:08.907+0000 7f9db8c00640 1 -- 172.21.15.12:0/1662941401 <== mon.1 v2:172.21.15.178:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9dc00acf50 con 0x55b35e182240 2024-09-11T00:06:08.913 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:08.907+0000 7f9db8c00640 1 -- 172.21.15.12:0/1662941401 <== mon.1 v2:172.21.15.178:3300/0 5 ==== fsmap(e 28) ==== 8370+0+0 (secure 0 0 0) 0x7f9dc0071350 con 0x55b35e182240 2024-09-11T00:06:08.913 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:08.907+0000 7f9dd8850740 10 client.25162 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T00:06:08.913 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:08.908+0000 7f9dd8850740 10 client.25162 resolve_mds: resolved name 'g' to daemon mds.g 2024-09-11T00:06:08.913 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:08.908+0000 7f9dd8850740 20 client.25162 populate_metadata read hostname 'smithi012' 2024-09-11T00:06:08.913 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:08.908+0000 7f9dd8850740 1 --2- 172.21.15.12:0/1662941401 >> [v2:172.21.15.12:6838/3124093009,v1:172.21.15.12:6839/3124093009] conn(0x55b35e210450 0x55b35e20f160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:06:08.913 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:08.908+0000 7f9dd8850740 4 client.25162 mds_command: new command op to 24643 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-11T00:06:08.913 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:08.908+0000 7f9dd8850740 1 -- 172.21.15.12:0/1662941401 --> [v2:172.21.15.12:6838/3124093009,v1:172.21.15.12:6839/3124093009] -- command(tid 0: {"prefix": "status"}) -- 0x55b35e0f7750 con 0x55b35e210450 2024-09-11T00:06:08.913 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:08.908+0000 7f9dce000640 1 --2- 172.21.15.12:0/1662941401 >> [v2:172.21.15.12:6838/3124093009,v1:172.21.15.12:6839/3124093009] conn(0x55b35e210450 0x55b35e20f160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:06:08.913 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:08.909+0000 7f9dce000640 1 --2- 172.21.15.12:0/1662941401 >> [v2:172.21.15.12:6838/3124093009,v1:172.21.15.12:6839/3124093009] conn(0x55b35e210450 0x55b35e20f160 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=5c2f47c45765b175 server_cookie=5bf241bc8babdae6 in_seq=0 out_seq=0 2024-09-11T00:06:08.913 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:08.909+0000 7f9db8c00640 10 client.25162 ms_handle_connect on v2:172.21.15.12:6838/3124093009 2024-09-11T00:06:08.914 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:08.909+0000 7f9db8c00640 1 -- 172.21.15.12:0/1662941401 <== mds.1 v2:172.21.15.12:6838/3124093009 1 ==== command_reply(tid 0: 0 ) ==== 8+0+326 (crc 0 0 0) 0x55b35e0f7750 con 0x55b35e210450 2024-09-11T00:06:08.914 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:08.909+0000 7f9db8c00640 10 client.25162 handle_command_reply: tid=0 2024-09-11T00:06:08.914 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:08.909+0000 7f9dd8850740 1 client.25162 shutdown 2024-09-11T00:06:08.914 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:08.909+0000 7f9dd8850740 1 -- 172.21.15.12:0/1662941401 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b35e182240 msgr2=0x55b35e1a9400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:06:08.914 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:08.909+0000 7f9dd8850740 1 --2- 172.21.15.12:0/1662941401 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b35e182240 0x55b35e1a9400 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f9dc00a8010 tx=0x7f9dc00e3c50 comp rx=0 tx=0).stop 2024-09-11T00:06:08.914 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:08.909+0000 7f9dd8850740 1 -- 172.21.15.12:0/1662941401 shutdown_connections 2024-09-11T00:06:08.914 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:08.909+0000 7f9dd8850740 1 --2- 172.21.15.12:0/1662941401 >> [v2:172.21.15.12:6838/3124093009,v1:172.21.15.12:6839/3124093009] conn(0x55b35e210450 0x55b35e20f160 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:08.914 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:08.909+0000 7f9dd8850740 1 --2- 172.21.15.12:0/1662941401 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b35e18eaf0 0x55b35e1a2e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:08.914 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:08.909+0000 7f9dd8850740 1 --2- 172.21.15.12:0/1662941401 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b35e182b60 0x55b35e1a28d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:08.914 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:08.909+0000 7f9dd8850740 1 --2- 172.21.15.12:0/1662941401 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b35e182240 0x55b35e1a9400 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:08.914 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:08.909+0000 7f9dd8850740 1 -- 172.21.15.12:0/1662941401 >> 172.21.15.12:0/1662941401 conn(0x55b35e180020 msgr2=0x55b35e186d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:06:08.914 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:08.909+0000 7f9db8c00640 0 client.25162 ms_handle_reset on v2:172.21.15.12:6838/3124093009 2024-09-11T00:06:08.915 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:08.910+0000 7f9dd8850740 1 -- 172.21.15.12:0/1662941401 shutdown_connections 2024-09-11T00:06:08.915 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:08.910+0000 7f9dd8850740 1 -- 172.21.15.12:0/1662941401 wait complete. 2024-09-11T00:06:08.915 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:08.910+0000 7f9dd8850740 20 client.25162 trim_cache size 0 max 16384 2024-09-11T00:06:08.915 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-09-11T00:06:08.915 INFO:teuthology.orchestra.run.smithi012.stdout: "cluster_fsid": "ff3ad74a-6fcf-11ef-bcea-c7b262605968", 2024-09-11T00:06:08.915 INFO:teuthology.orchestra.run.smithi012.stdout: "whoami": 1, 2024-09-11T00:06:08.915 INFO:teuthology.orchestra.run.smithi012.stdout: "id": 24643, 2024-09-11T00:06:08.915 INFO:teuthology.orchestra.run.smithi012.stdout: "want_state": "up:active", 2024-09-11T00:06:08.915 INFO:teuthology.orchestra.run.smithi012.stdout: "state": "up:active", 2024-09-11T00:06:08.916 INFO:teuthology.orchestra.run.smithi012.stdout: "fs_name": "cephfs", 2024-09-11T00:06:08.916 INFO:teuthology.orchestra.run.smithi012.stdout: "rank_uptime": 82.687681462, 2024-09-11T00:06:08.916 INFO:teuthology.orchestra.run.smithi012.stdout: "mdsmap_epoch": 28, 2024-09-11T00:06:08.916 INFO:teuthology.orchestra.run.smithi012.stdout: "osdmap_epoch": 83, 2024-09-11T00:06:08.916 INFO:teuthology.orchestra.run.smithi012.stdout: "osdmap_epoch_barrier": 81, 2024-09-11T00:06:08.916 INFO:teuthology.orchestra.run.smithi012.stdout: "uptime": 143.34178778699999 2024-09-11T00:06:08.916 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-09-11T00:06:08.917 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:08.910+0000 7f9dd5c00640 1 -- 172.21.15.12:0/4024788476 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f9db40b5370 msgr2=0x7f9db40b7830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:06:08.917 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:08.910+0000 7f9dd5c00640 1 --2- 172.21.15.12:0/4024788476 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f9db40b5370 0x7f9db40b7830 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f9dc801ba10 tx=0x7f9dc8000f40 comp rx=0 tx=0).stop 2024-09-11T00:06:08.917 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:08.910+0000 7f9dd5c00640 1 -- 172.21.15.12:0/4024788476 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9dd00776d0 msgr2=0x7f9dd019f8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:06:08.917 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:08.910+0000 7f9dd5c00640 1 --2- 172.21.15.12:0/4024788476 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9dd00776d0 0x7f9dd019f8d0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f9dc000e9f0 tx=0x7f9dc000edf0 comp rx=0 tx=0).stop 2024-09-11T00:06:08.917 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:08.911+0000 7f9dd5c00640 1 -- 172.21.15.12:0/4024788476 shutdown_connections 2024-09-11T00:06:08.917 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:08.911+0000 7f9dd5c00640 1 --2- 172.21.15.12:0/4024788476 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f9db40b5370 0x7f9db40b7830 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:08.917 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:08.911+0000 7f9dd5c00640 1 --2- 172.21.15.12:0/4024788476 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9dd01139b0 0x7f9dd01a51b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:08.917 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:08.911+0000 7f9dd5c00640 1 --2- 172.21.15.12:0/4024788476 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9dd0077ff0 0x7f9dd019fe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:08.917 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:08.911+0000 7f9dd5c00640 1 --2- 172.21.15.12:0/4024788476 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9dd00776d0 0x7f9dd019f8d0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:08.917 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:08.911+0000 7f9dd5c00640 1 -- 172.21.15.12:0/4024788476 >> 172.21.15.12:0/4024788476 conn(0x7f9dd01009f0 msgr2=0x7f9dd0102de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:06:08.917 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:08.911+0000 7f9dd5c00640 1 -- 172.21.15.12:0/4024788476 shutdown_connections 2024-09-11T00:06:08.918 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:08.911+0000 7f9dd5c00640 1 -- 172.21.15.12:0/4024788476 wait complete. 2024-09-11T00:06:09.428 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:06:09 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:06:09.429 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:06:09 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:06:09.429 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:06:09 smithi012 ceph-mon[30333]: pgmap v368: 129 pgs: 129 active+clean; 647 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-11T00:06:09.429 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:06:09 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:06:09.429 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:06:09 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T00:06:09.429 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:06:09 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:06:09.532 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:06:09 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:06:09.532 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:06:09 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:06:09.532 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:06:09 smithi165 ceph-mon[35356]: pgmap v368: 129 pgs: 129 active+clean; 647 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-11T00:06:09.532 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:06:09 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:06:09.532 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:06:09 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T00:06:09.533 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:06:09 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:06:09.581 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:06:09 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:06:09.581 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:06:09 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:06:09.581 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:06:09 smithi178 ceph-mon[35919]: pgmap v368: 129 pgs: 129 active+clean; 647 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-11T00:06:09.581 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:06:09 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:06:09.581 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:06:09 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T00:06:09.582 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:06:09 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:06:09.769 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid ff3ad74a-6fcf-11ef-bcea-c7b262605968 -- ceph tell mds.b status 2024-09-11T00:06:10.150 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:06:10.709 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:10.708+0000 7f98d6800640 1 -- 172.21.15.12:0/3976051943 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f98d0109130 msgr2=0x7f98d010f930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:06:10.709 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:10.708+0000 7f98d6800640 1 --2- 172.21.15.12:0/3976051943 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f98d0109130 0x7f98d010f930 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f98c8009ca0 tx=0x7f98c802a8c0 comp rx=0 tx=0).stop 2024-09-11T00:06:10.709 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:10.708+0000 7f98d6800640 1 -- 172.21.15.12:0/3976051943 shutdown_connections 2024-09-11T00:06:10.710 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:10.708+0000 7f98d6800640 1 --2- 172.21.15.12:0/3976051943 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f98d0109130 0x7f98d010f930 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:10.710 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:10.708+0000 7f98d6800640 1 --2- 172.21.15.12:0/3976051943 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f98d01087b0 0x7f98d0108bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:10.710 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:10.708+0000 7f98d6800640 1 --2- 172.21.15.12:0/3976051943 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f98d0102da0 0x7f98d0103180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:10.710 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:10.708+0000 7f98d6800640 1 -- 172.21.15.12:0/3976051943 >> 172.21.15.12:0/3976051943 conn(0x7f98d00fe810 msgr2=0x7f98d0100c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:06:10.710 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:10.708+0000 7f98d6800640 1 -- 172.21.15.12:0/3976051943 shutdown_connections 2024-09-11T00:06:10.710 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:10.708+0000 7f98d6800640 1 -- 172.21.15.12:0/3976051943 wait complete. 2024-09-11T00:06:10.711 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:10.709+0000 7f98d6800640 1 Processor -- start 2024-09-11T00:06:10.711 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:10.709+0000 7f98d6800640 1 -- start start 2024-09-11T00:06:10.711 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:10.710+0000 7f98d6800640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f98d0102da0 0x7f98d0199290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:06:10.711 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:10.710+0000 7f98d6800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f98d01087b0 0x7f98d01997d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:06:10.711 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:10.710+0000 7f98d6800640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f98d0109130 0x7f98d019eb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:06:10.711 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:10.710+0000 7f98d6800640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f98d0112e00 con 0x7f98d0102da0 2024-09-11T00:06:10.711 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:10.710+0000 7f98d6800640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f98d0112c80 con 0x7f98d01087b0 2024-09-11T00:06:10.712 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:10.710+0000 7f98d6800640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f98d0112f80 con 0x7f98d0109130 2024-09-11T00:06:10.712 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:10.710+0000 7f98cf400640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f98d0102da0 0x7f98d0199290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:06:10.712 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:10.710+0000 7f98cfe00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f98d0109130 0x7f98d019eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:06:10.712 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:10.710+0000 7f98cfe00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f98d0109130 0x7f98d019eb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.12:35370/0 (socket says 172.21.15.12:35370) 2024-09-11T00:06:10.712 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:10.710+0000 7f98cfe00640 1 -- 172.21.15.12:0/607296138 learned_addr learned my addr 172.21.15.12:0/607296138 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:06:10.712 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:10.710+0000 7f98c6a00640 1 --2- 172.21.15.12:0/607296138 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f98d01087b0 0x7f98d01997d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:06:10.713 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:10.711+0000 7f98cfe00640 1 -- 172.21.15.12:0/607296138 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f98d01087b0 msgr2=0x7f98d01997d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:06:10.713 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:10.711+0000 7f98cfe00640 1 --2- 172.21.15.12:0/607296138 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f98d01087b0 0x7f98d01997d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:10.713 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:10.711+0000 7f98cfe00640 1 -- 172.21.15.12:0/607296138 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f98d0102da0 msgr2=0x7f98d0199290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:06:10.713 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:10.711+0000 7f98cfe00640 1 --2- 172.21.15.12:0/607296138 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f98d0102da0 0x7f98d0199290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:10.713 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:10.711+0000 7f98cfe00640 1 -- 172.21.15.12:0/607296138 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98d019f340 con 0x7f98d0109130 2024-09-11T00:06:10.713 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:10.711+0000 7f98cf400640 1 --2- 172.21.15.12:0/607296138 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f98d0102da0 0x7f98d0199290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:06:10.713 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:10.711+0000 7f98cfe00640 1 --2- 172.21.15.12:0/607296138 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f98d0109130 0x7f98d019eb70 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f98c802f540 tx=0x7f98c80316f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:06:10.714 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:10.711+0000 7f98ccc00640 1 -- 172.21.15.12:0/607296138 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98c8003b70 con 0x7f98d0109130 2024-09-11T00:06:10.714 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:10.711+0000 7f98d6800640 1 -- 172.21.15.12:0/607296138 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98d019f5d0 con 0x7f98d0109130 2024-09-11T00:06:10.714 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:10.711+0000 7f98ccc00640 1 -- 172.21.15.12:0/607296138 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f98c8003d10 con 0x7f98d0109130 2024-09-11T00:06:10.714 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:10.711+0000 7f98ccc00640 1 -- 172.21.15.12:0/607296138 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98c802f990 con 0x7f98d0109130 2024-09-11T00:06:10.714 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:10.711+0000 7f98d6800640 1 -- 172.21.15.12:0/607296138 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98d01a5cf0 con 0x7f98d0109130 2024-09-11T00:06:10.715 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:10.714+0000 7f98d9440740 1 Processor -- start 2024-09-11T00:06:10.716 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:10.715+0000 7f98ccc00640 1 -- 172.21.15.12:0/607296138 <== mon.1 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 17) ==== 153301+0+0 (secure 0 0 0) 0x7f98c802fb30 con 0x7f98d0109130 2024-09-11T00:06:10.716 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:10.715+0000 7f98d9440740 1 -- start start 2024-09-11T00:06:10.716 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:10.715+0000 7f98d9440740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5606daab9650 0x5606daab9a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:06:10.716 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:10.715+0000 7f98d9440740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5606daab9f70 0x5606daac59c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:06:10.716 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:10.715+0000 7f98d9440740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5606daac5f00 0x5606daac82f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:06:10.716 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:10.715+0000 7f98d9440740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x5606daa2cd50 con 0x5606daac5f00 2024-09-11T00:06:10.716 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:10.715+0000 7f98d9440740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x5606daa1c180 con 0x5606daab9650 2024-09-11T00:06:10.716 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:10.715+0000 7f98d9440740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5606daa1ff90 con 0x5606daab9f70 2024-09-11T00:06:10.718 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:10.716+0000 7f98cf400640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5606daab9f70 0x5606daac59c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:06:10.718 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:10.716+0000 7f98cf400640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5606daab9f70 0x5606daac59c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.12:35376/0 (socket says 172.21.15.12:35376) 2024-09-11T00:06:10.718 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:10.716+0000 7f98cf400640 1 -- 172.21.15.12:0/2385286993 learned_addr learned my addr 172.21.15.12:0/2385286993 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:06:10.718 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:10.716+0000 7f98c6a00640 1 --2- 172.21.15.12:0/2385286993 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5606daac5f00 0x5606daac82f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:06:10.718 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:10.716+0000 7f98c6a00640 1 --2- 172.21.15.12:0/2385286993 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5606daab9650 0x5606daab9a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:06:10.718 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:10.716+0000 7f98cf400640 1 -- 172.21.15.12:0/2385286993 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5606daab9650 msgr2=0x5606daab9a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:06:10.718 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:10.716+0000 7f98cf400640 1 --2- 172.21.15.12:0/2385286993 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5606daab9650 0x5606daab9a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:10.718 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:10.716+0000 7f98cf400640 1 -- 172.21.15.12:0/2385286993 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5606daac5f00 msgr2=0x5606daac82f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:06:10.718 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:10.716+0000 7f98cf400640 1 --2- 172.21.15.12:0/2385286993 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5606daac5f00 0x5606daac82f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:10.718 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:10.716+0000 7f98cf400640 1 -- 172.21.15.12:0/2385286993 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5606da9e6a70 con 0x5606daab9f70 2024-09-11T00:06:10.718 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:10.716+0000 7f98c6a00640 1 --2- 172.21.15.12:0/2385286993 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5606daab9650 0x5606daab9a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T00:06:10.719 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:10.716+0000 7f98cf400640 1 --2- 172.21.15.12:0/2385286993 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5606daab9f70 0x5606daac59c0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f98c00115f0 tx=0x7f98c0011ab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=df3f2cc05c2ce6a1 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:06:10.719 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:10.717+0000 7f98c4c00640 1 -- 172.21.15.12:0/2385286993 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98c001e070 con 0x5606daab9f70 2024-09-11T00:06:10.719 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:10.717+0000 7f98c4c00640 1 -- 172.21.15.12:0/2385286993 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f98c000faf0 con 0x5606daab9f70 2024-09-11T00:06:10.719 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:10.717+0000 7f98c4c00640 1 -- 172.21.15.12:0/2385286993 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98c0019b60 con 0x5606daab9f70 2024-09-11T00:06:10.719 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:10.717+0000 7f98ccc00640 1 --2- 172.21.15.12:0/607296138 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f98b40b5370 0x7f98b40b7830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:06:10.719 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:10.717+0000 7f98ccc00640 1 -- 172.21.15.12:0/607296138 <== mon.1 v2:172.21.15.178:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 9092+0+0 (secure 0 0 0) 0x7f98c8107390 con 0x7f98d0109130 2024-09-11T00:06:10.719 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:10.717+0000 7f98c6a00640 1 --2- 172.21.15.12:0/607296138 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f98b40b5370 0x7f98b40b7830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:06:10.721 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:10.717+0000 7f98d9440740 1 -- 172.21.15.12:0/2385286993 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5606daab9f70 msgr2=0x5606daac59c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:06:10.721 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:10.717+0000 7f98d9440740 1 --2- 172.21.15.12:0/2385286993 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5606daab9f70 0x5606daac59c0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f98c00115f0 tx=0x7f98c0011ab0 comp rx=0 tx=0).stop 2024-09-11T00:06:10.722 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:10.718+0000 7f98d9440740 1 -- 172.21.15.12:0/2385286993 shutdown_connections 2024-09-11T00:06:10.722 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:10.718+0000 7f98d9440740 1 --2- 172.21.15.12:0/2385286993 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5606daac5f00 0x5606daac82f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:10.722 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:10.718+0000 7f98d9440740 1 --2- 172.21.15.12:0/2385286993 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5606daab9f70 0x5606daac59c0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:10.722 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:10.718+0000 7f98d9440740 1 --2- 172.21.15.12:0/2385286993 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5606daab9650 0x5606daab9a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:10.722 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:10.718+0000 7f98d9440740 1 -- 172.21.15.12:0/2385286993 >> 172.21.15.12:0/2385286993 conn(0x5606daab7430 msgr2=0x5606daab8620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:06:10.722 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:10.718+0000 7f98d9440740 1 -- 172.21.15.12:0/2385286993 shutdown_connections 2024-09-11T00:06:10.722 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:10.718+0000 7f98d9440740 1 -- 172.21.15.12:0/2385286993 wait complete. 2024-09-11T00:06:10.722 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:10.718+0000 7f98c6a00640 1 --2- 172.21.15.12:0/607296138 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f98b40b5370 0x7f98b40b7830 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f98d0102150 tx=0x7f98b8009640 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:06:10.722 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:10.718+0000 7f98d9440740 1 -- start start 2024-09-11T00:06:10.722 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:10.718+0000 7f98d9440740 1 Processor -- start 2024-09-11T00:06:10.722 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:10.719+0000 7f98d9440740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5606daab9650 0x5606daac2e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:06:10.722 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:10.719+0000 7f98d9440740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5606daab9f70 0x5606daac33b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:06:10.723 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:10.719+0000 7f98d9440740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5606daac5f00 0x5606daac38f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:06:10.723 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:10.719+0000 7f98d9440740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x5606da87d5f0 con 0x5606daab9650 2024-09-11T00:06:10.723 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:10.719+0000 7f98d9440740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x5606da9b7460 con 0x5606daab9f70 2024-09-11T00:06:10.723 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:10.719+0000 7f98d9440740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5606daac8c20 con 0x5606daac5f00 2024-09-11T00:06:10.723 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:10.719+0000 7f98d9440740 10 client.0 _ll_register_callbacks cb 0x5606da9c7210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:06:10.723 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:10.719+0000 7f98cf400640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5606daab9650 0x5606daac2e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:06:10.723 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:10.719+0000 7f98cf400640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5606daab9650 0x5606daac2e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:50908/0 (socket says 172.21.15.12:50908) 2024-09-11T00:06:10.723 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:10.719+0000 7f98cf400640 1 -- 172.21.15.12:0/681469037 learned_addr learned my addr 172.21.15.12:0/681469037 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:06:10.723 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:10.719+0000 7f98c6a00640 1 --2- 172.21.15.12:0/681469037 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5606daab9f70 0x5606daac33b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:06:10.723 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:10.719+0000 7f98cfe00640 1 --2- 172.21.15.12:0/681469037 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5606daac5f00 0x5606daac38f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:06:10.723 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:10.720+0000 7f98cf400640 1 -- 172.21.15.12:0/681469037 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5606daac5f00 msgr2=0x5606daac38f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:06:10.723 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:10.720+0000 7f98cf400640 1 --2- 172.21.15.12:0/681469037 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5606daac5f00 0x5606daac38f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:10.724 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:10.720+0000 7f98cf400640 1 -- 172.21.15.12:0/681469037 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5606daab9f70 msgr2=0x5606daac33b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:06:10.724 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:10.720+0000 7f98cf400640 1 --2- 172.21.15.12:0/681469037 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5606daab9f70 0x5606daac33b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:10.724 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:10.720+0000 7f98cf400640 1 -- 172.21.15.12:0/681469037 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5606da77a540 con 0x5606daab9650 2024-09-11T00:06:10.724 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:10.720+0000 7f98cf400640 1 --2- 172.21.15.12:0/681469037 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5606daab9650 0x5606daac2e70 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f98c001c040 tx=0x7f98c00056d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cfdefa1b3073829 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:06:10.724 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:10.720+0000 7f98bd600640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-09-11T00:06:10.724 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:10.720+0000 7f98bd600640 1 -- 172.21.15.12:0/681469037 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98c001e040 con 0x5606daab9650 2024-09-11T00:06:10.724 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:10.720+0000 7f98bd600640 1 -- 172.21.15.12:0/681469037 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 9092+0+0 (secure 0 0 0) 0x7f98c0019410 con 0x5606daab9650 2024-09-11T00:06:10.724 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:10.720+0000 7f98d9440740 1 -- 172.21.15.12:0/681469037 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5606da9b9a70 con 0x5606daab9650 2024-09-11T00:06:10.724 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:10.721+0000 7f98bd600640 1 -- 172.21.15.12:0/681469037 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f98c0030050 con 0x5606daab9650 2024-09-11T00:06:10.724 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:10.721+0000 7f98d9440740 10 client.15156 fetch_fsmap learned FSMap version 28 2024-09-11T00:06:10.724 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:10.721+0000 7f98d9440740 1 -- 172.21.15.12:0/681469037 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5606da9d46f0 con 0x5606daab9650 2024-09-11T00:06:10.726 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:10.721+0000 7f98bd600640 1 -- 172.21.15.12:0/681469037 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x5606da9b9a70 con 0x5606daab9650 2024-09-11T00:06:10.726 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:10.721+0000 7f98bd600640 1 -- 172.21.15.12:0/681469037 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 28) ==== 8370+0+0 (secure 0 0 0) 0x7f98c0021070 con 0x5606daab9650 2024-09-11T00:06:10.726 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:10.721+0000 7f98d9440740 10 client.15156 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T00:06:10.726 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:10.722+0000 7f98d9440740 10 client.15156 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-11T00:06:10.726 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:10.722+0000 7f98d9440740 20 client.15156 populate_metadata read hostname 'smithi012' 2024-09-11T00:06:10.726 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:10.722+0000 7f98d9440740 1 --2- 172.21.15.12:0/681469037 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x5606daad68d0 0x5606daad7d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:06:10.726 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:10.722+0000 7f98d9440740 4 client.15156 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T00:06:10.727 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:10.722+0000 7f98d9440740 1 -- 172.21.15.12:0/681469037 --> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5606daa2eca0 con 0x5606daad68d0 2024-09-11T00:06:10.727 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:10.722+0000 7f98c6a00640 1 --2- 172.21.15.12:0/681469037 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x5606daad68d0 0x5606daad7d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:06:10.727 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:10.723+0000 7f98c6a00640 1 --2- 172.21.15.12:0/681469037 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x5606daad68d0 0x5606daad7d40 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8e6fb64dd696fa32 server_cookie=aa3ff447f08030a3 in_seq=0 out_seq=0 2024-09-11T00:06:10.727 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:10.723+0000 7f98bd600640 10 client.15156 ms_handle_connect on v2:172.21.15.165:6838/501312984 2024-09-11T00:06:10.727 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:10.723+0000 7f98bd600640 1 -- 172.21.15.12:0/681469037 <== mds.0 v2:172.21.15.165:6838/501312984 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5606daa2eca0 con 0x5606daad68d0 2024-09-11T00:06:10.727 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:10.723+0000 7f98bd600640 10 client.15156 handle_command_reply: tid=0 2024-09-11T00:06:10.727 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:10.724+0000 7f98d9440740 1 client.15156 shutdown 2024-09-11T00:06:10.727 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:10.724+0000 7f98d9440740 1 -- 172.21.15.12:0/681469037 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5606daab9650 msgr2=0x5606daac2e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:06:10.727 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:10.724+0000 7f98d9440740 1 --2- 172.21.15.12:0/681469037 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5606daab9650 0x5606daac2e70 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f98c001c040 tx=0x7f98c00056d0 comp rx=0 tx=0).stop 2024-09-11T00:06:10.727 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:10.724+0000 7f98d9440740 1 -- 172.21.15.12:0/681469037 shutdown_connections 2024-09-11T00:06:10.727 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:10.724+0000 7f98d9440740 1 --2- 172.21.15.12:0/681469037 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x5606daad68d0 0x5606daad7d40 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:10.727 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:10.724+0000 7f98d9440740 1 --2- 172.21.15.12:0/681469037 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5606daac5f00 0x5606daac38f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:10.727 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:10.724+0000 7f98d9440740 1 --2- 172.21.15.12:0/681469037 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5606daab9f70 0x5606daac33b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:10.728 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:10.724+0000 7f98d9440740 1 --2- 172.21.15.12:0/681469037 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5606daab9650 0x5606daac2e70 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:10.728 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:10.724+0000 7f98d9440740 1 -- 172.21.15.12:0/681469037 >> 172.21.15.12:0/681469037 conn(0x5606daab7430 msgr2=0x5606daab2140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:06:10.728 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:10.724+0000 7f98bd600640 0 client.15156 ms_handle_reset on v2:172.21.15.165:6838/501312984 2024-09-11T00:06:10.728 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:10.724+0000 7f98d9440740 1 -- 172.21.15.12:0/681469037 shutdown_connections 2024-09-11T00:06:10.728 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:10.724+0000 7f98d9440740 1 -- 172.21.15.12:0/681469037 wait complete. 2024-09-11T00:06:10.728 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:10.724+0000 7f98d9440740 20 client.15156 trim_cache size 0 max 16384 2024-09-11T00:06:10.745 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:10.743+0000 7f98d9440740 1 Processor -- start 2024-09-11T00:06:10.745 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:10.743+0000 7f98d9440740 1 -- start start 2024-09-11T00:06:10.745 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:10.744+0000 7f98d9440740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5606daab9650 0x5606daabd6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:06:10.745 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:10.744+0000 7f98d9440740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5606daab9f70 0x5606daab84d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:06:10.745 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:10.744+0000 7f98d9440740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5606daac5f00 0x5606daab8a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:06:10.745 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:10.744+0000 7f98d9440740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x5606daad6080 con 0x5606daab9f70 2024-09-11T00:06:10.745 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:10.744+0000 7f98d9440740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x5606daad6750 con 0x5606daac5f00 2024-09-11T00:06:10.745 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:10.744+0000 7f98d9440740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5606daad6e20 con 0x5606daab9650 2024-09-11T00:06:10.747 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:10.744+0000 7f98c6a00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5606daab9f70 0x5606daab84d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:06:10.747 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:10.744+0000 7f98c6a00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5606daab9f70 0x5606daab84d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:50918/0 (socket says 172.21.15.12:50918) 2024-09-11T00:06:10.747 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:10.744+0000 7f98c6a00640 1 -- 172.21.15.12:0/2054423479 learned_addr learned my addr 172.21.15.12:0/2054423479 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:06:10.747 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:10.744+0000 7f98cfe00640 1 --2- 172.21.15.12:0/2054423479 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5606daac5f00 0x5606daab8a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:06:10.747 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:10.744+0000 7f98cf400640 1 --2- 172.21.15.12:0/2054423479 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5606daab9650 0x5606daabd6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:06:10.747 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:10.744+0000 7f98c6a00640 1 -- 172.21.15.12:0/2054423479 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5606daab9650 msgr2=0x5606daabd6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:06:10.747 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:10.744+0000 7f98c6a00640 1 --2- 172.21.15.12:0/2054423479 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5606daab9650 0x5606daabd6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:10.747 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:10.744+0000 7f98c6a00640 1 -- 172.21.15.12:0/2054423479 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5606daac5f00 msgr2=0x5606daab8a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:06:10.747 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:10.744+0000 7f98c6a00640 1 --2- 172.21.15.12:0/2054423479 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5606daac5f00 0x5606daab8a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:10.747 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:10.744+0000 7f98c6a00640 1 -- 172.21.15.12:0/2054423479 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5606daab9da0 con 0x5606daab9f70 2024-09-11T00:06:10.747 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:10.745+0000 7f98c6a00640 1 --2- 172.21.15.12:0/2054423479 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5606daab9f70 0x5606daab84d0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f98b8009fd0 tx=0x7f98b80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96487c6908a0170c server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:06:10.747 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:10.745+0000 7f98bfe00640 1 -- 172.21.15.12:0/2054423479 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98b802b640 con 0x5606daab9f70 2024-09-11T00:06:10.748 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:10.745+0000 7f98bfe00640 1 -- 172.21.15.12:0/2054423479 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f98b802bc90 con 0x5606daab9f70 2024-09-11T00:06:10.748 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:10.745+0000 7f98bfe00640 1 -- 172.21.15.12:0/2054423479 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98b8022d30 con 0x5606daab9f70 2024-09-11T00:06:10.748 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:10.746+0000 7f98d9440740 1 -- 172.21.15.12:0/2054423479 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5606daab9f70 msgr2=0x5606daab84d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:06:10.748 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:10.746+0000 7f98d9440740 1 --2- 172.21.15.12:0/2054423479 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5606daab9f70 0x5606daab84d0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f98b8009fd0 tx=0x7f98b80046c0 comp rx=0 tx=0).stop 2024-09-11T00:06:10.748 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:10.746+0000 7f98d9440740 1 -- 172.21.15.12:0/2054423479 shutdown_connections 2024-09-11T00:06:10.748 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:10.746+0000 7f98d9440740 1 --2- 172.21.15.12:0/2054423479 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5606daac5f00 0x5606daab8a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:10.748 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:10.746+0000 7f98d9440740 1 --2- 172.21.15.12:0/2054423479 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5606daab9f70 0x5606daab84d0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:10.748 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:10.746+0000 7f98d9440740 1 --2- 172.21.15.12:0/2054423479 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5606daab9650 0x5606daabd6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:10.748 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:10.746+0000 7f98d9440740 1 -- 172.21.15.12:0/2054423479 >> 172.21.15.12:0/2054423479 conn(0x5606daab7430 msgr2=0x5606daac16f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:06:10.748 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:10.746+0000 7f98d9440740 1 -- 172.21.15.12:0/2054423479 shutdown_connections 2024-09-11T00:06:10.748 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:10.746+0000 7f98d9440740 1 -- 172.21.15.12:0/2054423479 wait complete. 2024-09-11T00:06:10.750 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:10.746+0000 7f98d9440740 1 -- start start 2024-09-11T00:06:10.750 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:10.747+0000 7f98d9440740 1 Processor -- start 2024-09-11T00:06:10.750 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:10.747+0000 7f98d9440740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5606daab9650 0x5606daae08a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:06:10.750 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:10.747+0000 7f98d9440740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5606daab9f70 0x5606daabfec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:06:10.750 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:10.747+0000 7f98d9440740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5606daac5f00 0x5606daac04f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:06:10.751 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:10.747+0000 7f98d9440740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x5606daad7bc0 con 0x5606daac5f00 2024-09-11T00:06:10.751 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:10.747+0000 7f98d9440740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x5606daad74f0 con 0x5606daab9f70 2024-09-11T00:06:10.751 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:10.747+0000 7f98d9440740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5606daad8290 con 0x5606daab9650 2024-09-11T00:06:10.751 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:10.747+0000 7f98d9440740 10 client.0 _ll_register_callbacks cb 0x7f98c0011440 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:06:10.751 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:10.747+0000 7f98cfe00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5606daac5f00 0x5606daac04f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:06:10.751 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:10.747+0000 7f98cfe00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5606daac5f00 0x5606daac04f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:50932/0 (socket says 172.21.15.12:50932) 2024-09-11T00:06:10.751 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:10.747+0000 7f98cfe00640 1 -- 172.21.15.12:0/946516833 learned_addr learned my addr 172.21.15.12:0/946516833 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:06:10.751 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:10.747+0000 7f98cf400640 1 --2- 172.21.15.12:0/946516833 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5606daab9650 0x5606daae08a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:06:10.751 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:10.747+0000 7f98c6a00640 1 --2- 172.21.15.12:0/946516833 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5606daab9f70 0x5606daabfec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:06:10.751 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:10.748+0000 7f98cfe00640 1 -- 172.21.15.12:0/946516833 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5606daab9650 msgr2=0x5606daae08a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:06:10.751 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:10.748+0000 7f98cfe00640 1 --2- 172.21.15.12:0/946516833 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5606daab9650 0x5606daae08a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:10.751 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:10.748+0000 7f98cfe00640 1 -- 172.21.15.12:0/946516833 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5606daab9f70 msgr2=0x5606daabfec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:06:10.751 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:10.748+0000 7f98cfe00640 1 --2- 172.21.15.12:0/946516833 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5606daab9f70 0x5606daabfec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:10.752 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:10.748+0000 7f98cfe00640 1 -- 172.21.15.12:0/946516833 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5606daadf490 con 0x5606daac5f00 2024-09-11T00:06:10.752 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:10.748+0000 7f98c6a00640 1 --2- 172.21.15.12:0/946516833 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5606daab9f70 0x5606daabfec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T00:06:10.752 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:10.748+0000 7f98cf400640 1 --2- 172.21.15.12:0/946516833 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5606daab9650 0x5606daae08a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:06:10.752 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:10.748+0000 7f98cfe00640 1 --2- 172.21.15.12:0/946516833 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5606daac5f00 0x5606daac04f0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f98c80cd5c0 tx=0x7f98c80cd0b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf181aab92a08aed server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:06:10.752 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:10.748+0000 7f98bd600640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-09-11T00:06:10.752 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:10.748+0000 7f98bd600640 1 -- 172.21.15.12:0/946516833 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98c80cd630 con 0x5606daac5f00 2024-09-11T00:06:10.752 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:10.748+0000 7f98bd600640 1 -- 172.21.15.12:0/946516833 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 9092+0+0 (secure 0 0 0) 0x7f98c80cd890 con 0x5606daac5f00 2024-09-11T00:06:10.752 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:10.748+0000 7f98d9440740 1 -- 172.21.15.12:0/946516833 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f98b802bc90 con 0x5606daac5f00 2024-09-11T00:06:10.752 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:10.748+0000 7f98bd600640 1 -- 172.21.15.12:0/946516833 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f98c80d2110 con 0x5606daac5f00 2024-09-11T00:06:10.752 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:10.749+0000 7f98d9440740 10 client.15162 fetch_fsmap learned FSMap version 28 2024-09-11T00:06:10.752 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:10.749+0000 7f98d9440740 1 -- 172.21.15.12:0/946516833 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5606dab43360 con 0x5606daac5f00 2024-09-11T00:06:10.752 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:10.749+0000 7f98bd600640 1 -- 172.21.15.12:0/946516833 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f98b802bc90 con 0x5606daac5f00 2024-09-11T00:06:10.752 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:10.749+0000 7f98bd600640 1 -- 172.21.15.12:0/946516833 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 28) ==== 8370+0+0 (secure 0 0 0) 0x7f98c80d6730 con 0x5606daac5f00 2024-09-11T00:06:10.753 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:10.749+0000 7f98d9440740 10 client.15162 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T00:06:10.753 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:10.749+0000 7f98d9440740 10 client.15162 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-11T00:06:10.753 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:10.749+0000 7f98d9440740 20 client.15162 populate_metadata read hostname 'smithi012' 2024-09-11T00:06:10.753 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:10.749+0000 7f98d9440740 1 --2- 172.21.15.12:0/946516833 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x5606dab47f60 0x5606dab46c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:06:10.753 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:10.749+0000 7f98d9440740 4 client.15162 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-11T00:06:10.753 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:10.749+0000 7f98d9440740 1 -- 172.21.15.12:0/946516833 --> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] -- command(tid 0: {"prefix": "status"}) -- 0x5606daa2a750 con 0x5606dab47f60 2024-09-11T00:06:10.753 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:10.749+0000 7f98cf400640 1 --2- 172.21.15.12:0/946516833 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x5606dab47f60 0x5606dab46c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:06:10.754 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:10.750+0000 7f98cf400640 1 --2- 172.21.15.12:0/946516833 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x5606dab47f60 0x5606dab46c70 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e25e2d6520778887 server_cookie=6b5168f8ae4137a8 in_seq=0 out_seq=0 2024-09-11T00:06:10.754 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:10.750+0000 7f98bd600640 10 client.15162 ms_handle_connect on v2:172.21.15.165:6838/501312984 2024-09-11T00:06:10.754 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:10.750+0000 7f98bd600640 1 -- 172.21.15.12:0/946516833 <== mds.0 v2:172.21.15.165:6838/501312984 1 ==== command_reply(tid 0: 0 ) ==== 8+0+322 (crc 0 0 0) 0x5606daa2a750 con 0x5606dab47f60 2024-09-11T00:06:10.754 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:10.750+0000 7f98bd600640 10 client.15162 handle_command_reply: tid=0 2024-09-11T00:06:10.755 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:10.750+0000 7f98d9440740 1 client.15162 shutdown 2024-09-11T00:06:10.755 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:10.750+0000 7f98d9440740 1 -- 172.21.15.12:0/946516833 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5606daac5f00 msgr2=0x5606daac04f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:06:10.755 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:10.750+0000 7f98d9440740 1 --2- 172.21.15.12:0/946516833 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5606daac5f00 0x5606daac04f0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f98c80cd5c0 tx=0x7f98c80cd0b0 comp rx=0 tx=0).stop 2024-09-11T00:06:10.755 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:10.751+0000 7f98d9440740 1 -- 172.21.15.12:0/946516833 shutdown_connections 2024-09-11T00:06:10.755 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:10.751+0000 7f98d9440740 1 --2- 172.21.15.12:0/946516833 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x5606dab47f60 0x5606dab46c70 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:10.755 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:10.751+0000 7f98d9440740 1 --2- 172.21.15.12:0/946516833 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5606daac5f00 0x5606daac04f0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:10.755 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:10.751+0000 7f98d9440740 1 --2- 172.21.15.12:0/946516833 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5606daab9f70 0x5606daabfec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:10.755 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:10.751+0000 7f98d9440740 1 --2- 172.21.15.12:0/946516833 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5606daab9650 0x5606daae08a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:10.755 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:10.751+0000 7f98d9440740 1 -- 172.21.15.12:0/946516833 >> 172.21.15.12:0/946516833 conn(0x5606daab7430 msgr2=0x5606daab84d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:06:10.755 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:10.751+0000 7f98bd600640 0 client.15162 ms_handle_reset on v2:172.21.15.165:6838/501312984 2024-09-11T00:06:10.755 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:10.751+0000 7f98d9440740 1 -- 172.21.15.12:0/946516833 shutdown_connections 2024-09-11T00:06:10.755 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:10.751+0000 7f98d9440740 1 -- 172.21.15.12:0/946516833 wait complete. 2024-09-11T00:06:10.755 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:10.751+0000 7f98d9440740 20 client.15162 trim_cache size 0 max 16384 2024-09-11T00:06:10.756 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-09-11T00:06:10.756 INFO:teuthology.orchestra.run.smithi012.stdout: "cluster_fsid": "ff3ad74a-6fcf-11ef-bcea-c7b262605968", 2024-09-11T00:06:10.756 INFO:teuthology.orchestra.run.smithi012.stdout: "whoami": 0, 2024-09-11T00:06:10.756 INFO:teuthology.orchestra.run.smithi012.stdout: "id": 24674, 2024-09-11T00:06:10.756 INFO:teuthology.orchestra.run.smithi012.stdout: "want_state": "up:active", 2024-09-11T00:06:10.756 INFO:teuthology.orchestra.run.smithi012.stdout: "state": "up:active", 2024-09-11T00:06:10.756 INFO:teuthology.orchestra.run.smithi012.stdout: "fs_name": "cephfs", 2024-09-11T00:06:10.756 INFO:teuthology.orchestra.run.smithi012.stdout: "rank_uptime": 125.912029027, 2024-09-11T00:06:10.756 INFO:teuthology.orchestra.run.smithi012.stdout: "mdsmap_epoch": 28, 2024-09-11T00:06:10.756 INFO:teuthology.orchestra.run.smithi012.stdout: "osdmap_epoch": 83, 2024-09-11T00:06:10.756 INFO:teuthology.orchestra.run.smithi012.stdout: "osdmap_epoch_barrier": 81, 2024-09-11T00:06:10.757 INFO:teuthology.orchestra.run.smithi012.stdout: "uptime": 138.821076557 2024-09-11T00:06:10.757 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-09-11T00:06:10.757 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:10.752+0000 7f98d6800640 1 -- 172.21.15.12:0/607296138 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f98b40b5370 msgr2=0x7f98b40b7830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:06:10.757 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:10.752+0000 7f98d6800640 1 --2- 172.21.15.12:0/607296138 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f98b40b5370 0x7f98b40b7830 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f98d0102150 tx=0x7f98b8009640 comp rx=0 tx=0).stop 2024-09-11T00:06:10.757 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:10.752+0000 7f98d6800640 1 -- 172.21.15.12:0/607296138 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f98d0109130 msgr2=0x7f98d019eb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:06:10.757 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:10.752+0000 7f98d6800640 1 --2- 172.21.15.12:0/607296138 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f98d0109130 0x7f98d019eb70 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f98c802f540 tx=0x7f98c80316f0 comp rx=0 tx=0).stop 2024-09-11T00:06:10.757 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:10.752+0000 7f98d6800640 1 -- 172.21.15.12:0/607296138 shutdown_connections 2024-09-11T00:06:10.757 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:10.752+0000 7f98d6800640 1 --2- 172.21.15.12:0/607296138 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f98b40b5370 0x7f98b40b7830 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:10.758 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:10.752+0000 7f98d6800640 1 --2- 172.21.15.12:0/607296138 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f98d0109130 0x7f98d019eb70 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:10.758 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:10.752+0000 7f98d6800640 1 --2- 172.21.15.12:0/607296138 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f98d01087b0 0x7f98d01997d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:10.758 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:10.752+0000 7f98d6800640 1 --2- 172.21.15.12:0/607296138 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f98d0102da0 0x7f98d0199290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:10.758 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:10.752+0000 7f98d6800640 1 -- 172.21.15.12:0/607296138 >> 172.21.15.12:0/607296138 conn(0x7f98d00fe810 msgr2=0x7f98d0100c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:06:10.758 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:10.752+0000 7f98d6800640 1 -- 172.21.15.12:0/607296138 shutdown_connections 2024-09-11T00:06:10.758 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:10.752+0000 7f98d6800640 1 -- 172.21.15.12:0/607296138 wait complete. 2024-09-11T00:06:11.472 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:06:11 smithi012 ceph-mon[30333]: pgmap v369: 129 pgs: 129 active+clean; 647 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-11T00:06:11.513 INFO:tasks.cephadm:Setup complete, yielding 2024-09-11T00:06:11.513 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-09-11T00:06:11.587 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi012.front.sepia.ceph.com 2024-09-11T00:06:11.588 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ff3ad74a-6fcf-11ef-bcea-c7b262605968 -- bash -c 'ceph orch status' 2024-09-11T00:06:11.588 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:06:11 smithi178 ceph-mon[35919]: pgmap v369: 129 pgs: 129 active+clean; 647 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-11T00:06:11.589 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:06:11 smithi165 ceph-mon[35356]: pgmap v369: 129 pgs: 129 active+clean; 647 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-11T00:06:11.971 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:06:12.526 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:12.525+0000 7f95ed000640 1 -- 172.21.15.12:0/1094378604 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f95e8104c40 msgr2=0x7f95e8105040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:06:12.527 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:12.525+0000 7f95ed000640 1 --2- 172.21.15.12:0/1094378604 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f95e8104c40 0x7f95e8105040 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f95d8009930 tx=0x7f95d802a420 comp rx=0 tx=0).stop 2024-09-11T00:06:12.527 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:12.525+0000 7f95ed000640 1 -- 172.21.15.12:0/1094378604 shutdown_connections 2024-09-11T00:06:12.527 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:12.525+0000 7f95ed000640 1 --2- 172.21.15.12:0/1094378604 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f95e8106760 0x7f95e810d0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:12.527 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:12.525+0000 7f95ed000640 1 --2- 172.21.15.12:0/1094378604 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f95e8105da0 0x7f95e8106220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:12.527 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:12.525+0000 7f95ed000640 1 --2- 172.21.15.12:0/1094378604 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f95e8104c40 0x7f95e8105040 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:12.527 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:12.525+0000 7f95ed000640 1 -- 172.21.15.12:0/1094378604 >> 172.21.15.12:0/1094378604 conn(0x7f95e8100510 msgr2=0x7f95e8102950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:06:12.527 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:12.525+0000 7f95ed000640 1 -- 172.21.15.12:0/1094378604 shutdown_connections 2024-09-11T00:06:12.527 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:12.525+0000 7f95ed000640 1 -- 172.21.15.12:0/1094378604 wait complete. 2024-09-11T00:06:12.528 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:12.526+0000 7f95ed000640 1 Processor -- start 2024-09-11T00:06:12.528 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:12.526+0000 7f95ed000640 1 -- start start 2024-09-11T00:06:12.529 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:12.527+0000 7f95ed000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f95e8104c40 0x7f95e819b040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:06:12.529 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:12.527+0000 7f95ed000640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f95e8105da0 0x7f95e819b690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:06:12.529 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:12.527+0000 7f95ed000640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f95e8106760 0x7f95e819c2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:06:12.529 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:12.527+0000 7f95ed000640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f95e81108f0 con 0x7f95e8105da0 2024-09-11T00:06:12.529 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:12.527+0000 7f95ed000640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f95e8110770 con 0x7f95e8104c40 2024-09-11T00:06:12.529 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:12.527+0000 7f95ed000640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f95e8110a70 con 0x7f95e8106760 2024-09-11T00:06:12.529 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:12.527+0000 7f95dfe00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f95e8105da0 0x7f95e819b690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:06:12.529 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:12.527+0000 7f95dfe00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f95e8105da0 0x7f95e819b690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:50964/0 (socket says 172.21.15.12:50964) 2024-09-11T00:06:12.529 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:12.527+0000 7f95dfe00640 1 -- 172.21.15.12:0/1934333166 learned_addr learned my addr 172.21.15.12:0/1934333166 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:06:12.529 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:12.527+0000 7f95e5e00640 1 --2- 172.21.15.12:0/1934333166 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f95e8104c40 0x7f95e819b040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:06:12.529 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:12.527+0000 7f95e6800640 1 --2- 172.21.15.12:0/1934333166 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f95e8106760 0x7f95e819c2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:06:12.529 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:12.528+0000 7f95dfe00640 1 -- 172.21.15.12:0/1934333166 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f95e8106760 msgr2=0x7f95e819c2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:06:12.529 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:12.528+0000 7f95dfe00640 1 --2- 172.21.15.12:0/1934333166 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f95e8106760 0x7f95e819c2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:12.530 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:12.528+0000 7f95dfe00640 1 -- 172.21.15.12:0/1934333166 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f95e8104c40 msgr2=0x7f95e819b040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:06:12.530 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:12.528+0000 7f95dfe00640 1 --2- 172.21.15.12:0/1934333166 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f95e8104c40 0x7f95e819b040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:12.530 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:12.528+0000 7f95dfe00640 1 -- 172.21.15.12:0/1934333166 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95e819cb70 con 0x7f95e8105da0 2024-09-11T00:06:12.530 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:12.528+0000 7f95dfe00640 1 --2- 172.21.15.12:0/1934333166 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f95e8105da0 0x7f95e819b690 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f95d000e9f0 tx=0x7f95d000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:06:12.531 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:12.528+0000 7f95dea00640 1 -- 172.21.15.12:0/1934333166 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95d000cd80 con 0x7f95e8105da0 2024-09-11T00:06:12.531 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:12.528+0000 7f95dea00640 1 -- 172.21.15.12:0/1934333166 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f95d00108e0 con 0x7f95e8105da0 2024-09-11T00:06:12.531 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:12.528+0000 7f95ed000640 1 -- 172.21.15.12:0/1934333166 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f95e819ce60 con 0x7f95e8105da0 2024-09-11T00:06:12.531 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:12.528+0000 7f95dea00640 1 -- 172.21.15.12:0/1934333166 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95d0018980 con 0x7f95e8105da0 2024-09-11T00:06:12.531 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:12.528+0000 7f95ed000640 1 -- 172.21.15.12:0/1934333166 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95e81a3540 con 0x7f95e8105da0 2024-09-11T00:06:12.531 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:12.530+0000 7f95ed000640 1 -- 172.21.15.12:0/1934333166 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f95e810bfc0 con 0x7f95e8105da0 2024-09-11T00:06:12.532 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:12.532+0000 7f95dea00640 1 -- 172.21.15.12:0/1934333166 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 17) ==== 153301+0+0 (secure 0 0 0) 0x7f95d0010430 con 0x7f95e8105da0 2024-09-11T00:06:12.536 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:12.533+0000 7f95dea00640 1 --2- 172.21.15.12:0/1934333166 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f95cc0b53f0 0x7f95cc0b78b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:06:12.536 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:12.533+0000 7f95dea00640 1 -- 172.21.15.12:0/1934333166 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 9092+0+0 (secure 0 0 0) 0x7f95d00e6870 con 0x7f95e8105da0 2024-09-11T00:06:12.536 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:12.534+0000 7f95e5e00640 1 --2- 172.21.15.12:0/1934333166 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f95cc0b53f0 0x7f95cc0b78b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:06:12.537 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:12.534+0000 7f95e5e00640 1 --2- 172.21.15.12:0/1934333166 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f95cc0b53f0 0x7f95cc0b78b0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f95d8009900 tx=0x7f95d8031450 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:06:12.537 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:12.535+0000 7f95dea00640 1 -- 172.21.15.12:0/1934333166 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f95d00acbc0 con 0x7f95e8105da0 2024-09-11T00:06:12.751 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:12.750+0000 7f95ed000640 1 -- 172.21.15.12:0/1934333166 --> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) -- 0x7f95e8063590 con 0x7f95cc0b53f0 2024-09-11T00:06:12.753 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:12.752+0000 7f95dea00640 1 -- 172.21.15.12:0/1934333166 <== mgr.14150 v2:172.21.15.12:6800/3442939137 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+43 (secure 0 0 0) 0x7f95e8063590 con 0x7f95cc0b53f0 2024-09-11T00:06:12.753 INFO:teuthology.orchestra.run.smithi012.stdout:Backend: cephadm 2024-09-11T00:06:12.753 INFO:teuthology.orchestra.run.smithi012.stdout:Available: Yes 2024-09-11T00:06:12.753 INFO:teuthology.orchestra.run.smithi012.stdout:Paused: No 2024-09-11T00:06:12.756 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:12.754+0000 7f95ed000640 1 -- 172.21.15.12:0/1934333166 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f95cc0b53f0 msgr2=0x7f95cc0b78b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:06:12.756 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:12.754+0000 7f95ed000640 1 --2- 172.21.15.12:0/1934333166 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f95cc0b53f0 0x7f95cc0b78b0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f95d8009900 tx=0x7f95d8031450 comp rx=0 tx=0).stop 2024-09-11T00:06:12.756 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:12.754+0000 7f95ed000640 1 -- 172.21.15.12:0/1934333166 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f95e8105da0 msgr2=0x7f95e819b690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:06:12.756 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:12.754+0000 7f95ed000640 1 --2- 172.21.15.12:0/1934333166 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f95e8105da0 0x7f95e819b690 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f95d000e9f0 tx=0x7f95d000eeb0 comp rx=0 tx=0).stop 2024-09-11T00:06:12.756 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:12.755+0000 7f95ed000640 1 -- 172.21.15.12:0/1934333166 shutdown_connections 2024-09-11T00:06:12.756 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:12.755+0000 7f95ed000640 1 --2- 172.21.15.12:0/1934333166 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f95cc0b53f0 0x7f95cc0b78b0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:12.756 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:12.755+0000 7f95ed000640 1 --2- 172.21.15.12:0/1934333166 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f95e8106760 0x7f95e819c2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:12.756 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:12.755+0000 7f95ed000640 1 --2- 172.21.15.12:0/1934333166 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f95e8105da0 0x7f95e819b690 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:12.756 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:12.755+0000 7f95ed000640 1 --2- 172.21.15.12:0/1934333166 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f95e8104c40 0x7f95e819b040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:12.756 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:12.755+0000 7f95ed000640 1 -- 172.21.15.12:0/1934333166 >> 172.21.15.12:0/1934333166 conn(0x7f95e8100510 msgr2=0x7f95e8109190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:06:12.756 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:12.755+0000 7f95ed000640 1 -- 172.21.15.12:0/1934333166 shutdown_connections 2024-09-11T00:06:12.757 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:12.755+0000 7f95ed000640 1 -- 172.21.15.12:0/1934333166 wait complete. 2024-09-11T00:06:13.480 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:06:13 smithi012 ceph-mon[30333]: pgmap v370: 129 pgs: 129 active+clean; 647 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-09-11T00:06:13.480 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:06:13 smithi012 ceph-mon[30333]: from='client.15168 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T00:06:13.517 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ff3ad74a-6fcf-11ef-bcea-c7b262605968 -- bash -c 'ceph orch ps' 2024-09-11T00:06:13.532 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:06:13 smithi165 ceph-mon[35356]: pgmap v370: 129 pgs: 129 active+clean; 647 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-09-11T00:06:13.532 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:06:13 smithi165 ceph-mon[35356]: from='client.15168 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T00:06:13.580 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:06:13 smithi178 ceph-mon[35919]: pgmap v370: 129 pgs: 129 active+clean; 647 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-09-11T00:06:13.581 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:06:13 smithi178 ceph-mon[35919]: from='client.15168 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T00:06:13.895 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:06:14.460 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:14.458+0000 7f88c3a00640 1 -- 172.21.15.12:0/2087364466 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f88bc075b20 msgr2=0x7f88bc075fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:06:14.460 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:14.458+0000 7f88c3a00640 1 --2- 172.21.15.12:0/2087364466 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f88bc075b20 0x7f88bc075fa0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f88a4009900 tx=0x7f88a402a3f0 comp rx=0 tx=0).stop 2024-09-11T00:06:14.460 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:14.458+0000 7f88c3a00640 1 -- 172.21.15.12:0/2087364466 shutdown_connections 2024-09-11T00:06:14.460 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:14.458+0000 7f88c3a00640 1 --2- 172.21.15.12:0/2087364466 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f88bc106550 0x7f88bc111210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:14.460 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:14.458+0000 7f88c3a00640 1 --2- 172.21.15.12:0/2087364466 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f88bc075b20 0x7f88bc075fa0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:14.460 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:14.458+0000 7f88c3a00640 1 --2- 172.21.15.12:0/2087364466 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f88bc076f80 0x7f88bc0755e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:14.460 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:14.458+0000 7f88c3a00640 1 -- 172.21.15.12:0/2087364466 >> 172.21.15.12:0/2087364466 conn(0x7f88bc0fe320 msgr2=0x7f88bc100740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:06:14.461 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:14.459+0000 7f88c3a00640 1 -- 172.21.15.12:0/2087364466 shutdown_connections 2024-09-11T00:06:14.461 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:14.459+0000 7f88c3a00640 1 -- 172.21.15.12:0/2087364466 wait complete. 2024-09-11T00:06:14.464 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:14.459+0000 7f88c3a00640 1 Processor -- start 2024-09-11T00:06:14.464 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:14.460+0000 7f88c3a00640 1 -- start start 2024-09-11T00:06:14.464 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:14.460+0000 7f88c3a00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f88bc075b20 0x7f88bc19f430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:06:14.464 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:14.460+0000 7f88c3a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f88bc076f80 0x7f88bc19fa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:06:14.465 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:14.460+0000 7f88c3a00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f88bc1a0650 0x7f88bc1a0b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:06:14.465 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:14.460+0000 7f88c3a00640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f88bc114d10 con 0x7f88bc1a0650 2024-09-11T00:06:14.465 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:14.460+0000 7f88c3a00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f88bc114b90 con 0x7f88bc076f80 2024-09-11T00:06:14.465 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:14.460+0000 7f88c3a00640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f88bc114e90 con 0x7f88bc075b20 2024-09-11T00:06:14.465 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:14.460+0000 7f88c1a00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f88bc1a0650 0x7f88bc1a0b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:06:14.465 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:14.460+0000 7f88c1000640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f88bc075b20 0x7f88bc19f430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:06:14.465 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:14.460+0000 7f88c1a00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f88bc1a0650 0x7f88bc1a0b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:50980/0 (socket says 172.21.15.12:50980) 2024-09-11T00:06:14.466 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:14.460+0000 7f88c1a00640 1 -- 172.21.15.12:0/723189921 learned_addr learned my addr 172.21.15.12:0/723189921 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:06:14.466 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:14.460+0000 7f88b3e00640 1 --2- 172.21.15.12:0/723189921 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f88bc076f80 0x7f88bc19fa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:06:14.466 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:14.461+0000 7f88c1a00640 1 -- 172.21.15.12:0/723189921 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f88bc075b20 msgr2=0x7f88bc19f430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:06:14.466 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:14.461+0000 7f88c1a00640 1 --2- 172.21.15.12:0/723189921 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f88bc075b20 0x7f88bc19f430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:14.466 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:14.461+0000 7f88c1a00640 1 -- 172.21.15.12:0/723189921 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f88bc076f80 msgr2=0x7f88bc19fa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:06:14.466 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:14.461+0000 7f88c1a00640 1 --2- 172.21.15.12:0/723189921 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f88bc076f80 0x7f88bc19fa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:14.467 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:14.461+0000 7f88c1a00640 1 -- 172.21.15.12:0/723189921 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88bc1a12d0 con 0x7f88bc1a0650 2024-09-11T00:06:14.467 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:14.461+0000 7f88c1a00640 1 --2- 172.21.15.12:0/723189921 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f88bc1a0650 0x7f88bc1a0b00 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f88b400eb10 tx=0x7f88b400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:06:14.467 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:14.461+0000 7f88ba000640 1 -- 172.21.15.12:0/723189921 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88b4004b20 con 0x7f88bc1a0650 2024-09-11T00:06:14.467 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:14.461+0000 7f88ba000640 1 -- 172.21.15.12:0/723189921 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f88b4004cc0 con 0x7f88bc1a0650 2024-09-11T00:06:14.467 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:14.461+0000 7f88ba000640 1 -- 172.21.15.12:0/723189921 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88b4010670 con 0x7f88bc1a0650 2024-09-11T00:06:14.467 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:14.461+0000 7f88c3a00640 1 -- 172.21.15.12:0/723189921 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f88bc1a76f0 con 0x7f88bc1a0650 2024-09-11T00:06:14.468 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:14.462+0000 7f88c3a00640 1 -- 172.21.15.12:0/723189921 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88bc1a7b50 con 0x7f88bc1a0650 2024-09-11T00:06:14.468 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:14.463+0000 7f88c3a00640 1 -- 172.21.15.12:0/723189921 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f88bc106550 con 0x7f88bc1a0650 2024-09-11T00:06:14.469 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:14.465+0000 7f88ba000640 1 -- 172.21.15.12:0/723189921 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 17) ==== 153301+0+0 (secure 0 0 0) 0x7f88b4005000 con 0x7f88bc1a0650 2024-09-11T00:06:14.469 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:14.466+0000 7f88ba000640 1 --2- 172.21.15.12:0/723189921 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f88a00b53f0 0x7f88a00b78b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:06:14.469 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:14.467+0000 7f88ba000640 1 -- 172.21.15.12:0/723189921 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 9092+0+0 (secure 0 0 0) 0x7f88b40e6800 con 0x7f88bc1a0650 2024-09-11T00:06:14.470 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:14.467+0000 7f88c1000640 1 --2- 172.21.15.12:0/723189921 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f88a00b53f0 0x7f88a00b78b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:06:14.470 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:14.468+0000 7f88c1000640 1 --2- 172.21.15.12:0/723189921 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f88a00b53f0 0x7f88a00b78b0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f88ac009bd0 tx=0x7f88ac005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:06:14.470 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:14.468+0000 7f88ba000640 1 -- 172.21.15.12:0/723189921 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f88b4056220 con 0x7f88bc1a0650 2024-09-11T00:06:14.674 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:14.673+0000 7f88c3a00640 1 -- 172.21.15.12:0/723189921 --> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7f88bc0ff050 con 0x7f88a00b53f0 2024-09-11T00:06:14.687 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:14.686+0000 7f88ba000640 1 -- 172.21.15.12:0/723189921 <== mgr.14150 v2:172.21.15.12:6800/3442939137 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+4123 (secure 0 0 0) 0x7f88bc0ff050 con 0x7f88a00b53f0 2024-09-11T00:06:14.687 INFO:teuthology.orchestra.run.smithi012.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-11T00:06:14.687 INFO:teuthology.orchestra.run.smithi012.stdout:mds.a smithi012 running (2m) 6s ago 2m 17.2M - 19.3.0-4854-g8f219c47 589dd18248e4 8e26bf6033aa 2024-09-11T00:06:14.688 INFO:teuthology.orchestra.run.smithi012.stdout:mds.b smithi165 running (2m) 9s ago 2m 26.7M - 19.3.0-4854-g8f219c47 589dd18248e4 08c991ef5d62 2024-09-11T00:06:14.688 INFO:teuthology.orchestra.run.smithi012.stdout:mds.c smithi178 running (2m) 9s ago 2m 22.2M - 19.3.0-4854-g8f219c47 589dd18248e4 93a61ce72254 2024-09-11T00:06:14.688 INFO:teuthology.orchestra.run.smithi012.stdout:mds.d smithi012 running (2m) 6s ago 2m 13.3M - 19.3.0-4854-g8f219c47 589dd18248e4 2726b503ecaf 2024-09-11T00:06:14.688 INFO:teuthology.orchestra.run.smithi012.stdout:mds.e smithi165 running (2m) 9s ago 2m 22.0M - 19.3.0-4854-g8f219c47 589dd18248e4 ab3021c1e997 2024-09-11T00:06:14.688 INFO:teuthology.orchestra.run.smithi012.stdout:mds.f smithi178 running (2m) 9s ago 2m 20.3M - 19.3.0-4854-g8f219c47 589dd18248e4 d1d64697e699 2024-09-11T00:06:14.688 INFO:teuthology.orchestra.run.smithi012.stdout:mds.g smithi012 running (2m) 6s ago 2m 17.8M - 19.3.0-4854-g8f219c47 589dd18248e4 04ca535a4928 2024-09-11T00:06:14.688 INFO:teuthology.orchestra.run.smithi012.stdout:mds.h smithi165 running (2m) 9s ago 2m 19.3M - 19.3.0-4854-g8f219c47 589dd18248e4 e73b97112e11 2024-09-11T00:06:14.688 INFO:teuthology.orchestra.run.smithi012.stdout:mds.i smithi178 running (2m) 9s ago 2m 21.7M - 19.3.0-4854-g8f219c47 589dd18248e4 99c4f702fdec 2024-09-11T00:06:14.688 INFO:teuthology.orchestra.run.smithi012.stdout:mds.j smithi012 running (2m) 6s ago 2m 13.3M - 19.3.0-4854-g8f219c47 589dd18248e4 03bb22653329 2024-09-11T00:06:14.688 INFO:teuthology.orchestra.run.smithi012.stdout:mds.k smithi165 running (2m) 9s ago 2m 18.9M - 19.3.0-4854-g8f219c47 589dd18248e4 b4e8ce1b141f 2024-09-11T00:06:14.688 INFO:teuthology.orchestra.run.smithi012.stdout:mds.l smithi178 running (2m) 9s ago 2m 19.1M - 19.3.0-4854-g8f219c47 589dd18248e4 f02db3543a9a 2024-09-11T00:06:14.688 INFO:teuthology.orchestra.run.smithi012.stdout:mgr.x smithi012 *:9283,8765 running (10m) 6s ago 10m 525M - 19.3.0-4854-g8f219c47 589dd18248e4 a392e948a60b 2024-09-11T00:06:14.688 INFO:teuthology.orchestra.run.smithi012.stdout:mgr.y smithi165 *:8443,8765 running (8m) 9s ago 8m 456M - 19.3.0-4854-g8f219c47 589dd18248e4 7a55726343a6 2024-09-11T00:06:14.689 INFO:teuthology.orchestra.run.smithi012.stdout:mgr.z smithi178 *:8443,8765 running (8m) 9s ago 8m 457M - 19.3.0-4854-g8f219c47 589dd18248e4 14fc6152c232 2024-09-11T00:06:14.689 INFO:teuthology.orchestra.run.smithi012.stdout:mon.a smithi012 running (10m) 6s ago 10m 51.6M 2048M 19.3.0-4854-g8f219c47 589dd18248e4 08abb0cfbc0b 2024-09-11T00:06:14.689 INFO:teuthology.orchestra.run.smithi012.stdout:mon.b smithi165 running (9m) 9s ago 9m 42.7M 2048M 19.3.0-4854-g8f219c47 589dd18248e4 77449392964c 2024-09-11T00:06:14.689 INFO:teuthology.orchestra.run.smithi012.stdout:mon.c smithi178 running (9m) 9s ago 9m 42.9M 2048M 19.3.0-4854-g8f219c47 589dd18248e4 23ccd19ad543 2024-09-11T00:06:14.689 INFO:teuthology.orchestra.run.smithi012.stdout:osd.0 smithi012 running (8m) 6s ago 8m 52.2M 4273M 19.3.0-4854-g8f219c47 589dd18248e4 912c97baed4f 2024-09-11T00:06:14.689 INFO:teuthology.orchestra.run.smithi012.stdout:osd.1 smithi165 running (7m) 9s ago 7m 54.3M 4272M 19.3.0-4854-g8f219c47 589dd18248e4 2c5be6dbcc76 2024-09-11T00:06:14.689 INFO:teuthology.orchestra.run.smithi012.stdout:osd.2 smithi178 running (7m) 9s ago 7m 50.0M 4272M 19.3.0-4854-g8f219c47 589dd18248e4 f44b163d0fef 2024-09-11T00:06:14.689 INFO:teuthology.orchestra.run.smithi012.stdout:osd.3 smithi012 running (6m) 6s ago 6m 50.2M 4273M 19.3.0-4854-g8f219c47 589dd18248e4 61cfe8f7eaa5 2024-09-11T00:06:14.689 INFO:teuthology.orchestra.run.smithi012.stdout:osd.4 smithi165 running (6m) 9s ago 6m 46.4M 4272M 19.3.0-4854-g8f219c47 589dd18248e4 63c9846d2d83 2024-09-11T00:06:14.689 INFO:teuthology.orchestra.run.smithi012.stdout:osd.5 smithi178 running (5m) 9s ago 5m 47.6M 4272M 19.3.0-4854-g8f219c47 589dd18248e4 78e183446c6e 2024-09-11T00:06:14.689 INFO:teuthology.orchestra.run.smithi012.stdout:osd.6 smithi012 running (5m) 6s ago 5m 47.9M 4273M 19.3.0-4854-g8f219c47 589dd18248e4 f0cf9000529e 2024-09-11T00:06:14.689 INFO:teuthology.orchestra.run.smithi012.stdout:osd.7 smithi165 running (4m) 9s ago 4m 47.7M 4272M 19.3.0-4854-g8f219c47 589dd18248e4 8e149d1cf9eb 2024-09-11T00:06:14.689 INFO:teuthology.orchestra.run.smithi012.stdout:osd.8 smithi178 running (4m) 9s ago 4m 53.7M 4272M 19.3.0-4854-g8f219c47 589dd18248e4 cea9a534d126 2024-09-11T00:06:14.690 INFO:teuthology.orchestra.run.smithi012.stdout:osd.9 smithi012 running (4m) 6s ago 4m 48.4M 4273M 19.3.0-4854-g8f219c47 589dd18248e4 5d28c3dc6dac 2024-09-11T00:06:14.690 INFO:teuthology.orchestra.run.smithi012.stdout:osd.10 smithi165 running (3m) 9s ago 3m 44.9M 4272M 19.3.0-4854-g8f219c47 589dd18248e4 b557cfc179e9 2024-09-11T00:06:14.690 INFO:teuthology.orchestra.run.smithi012.stdout:osd.11 smithi178 running (3m) 9s ago 3m 49.8M 4272M 19.3.0-4854-g8f219c47 589dd18248e4 f73e807cea74 2024-09-11T00:06:14.690 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:14.688+0000 7f88c3a00640 1 -- 172.21.15.12:0/723189921 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f88a00b53f0 msgr2=0x7f88a00b78b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:06:14.690 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:14.688+0000 7f88c3a00640 1 --2- 172.21.15.12:0/723189921 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f88a00b53f0 0x7f88a00b78b0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f88ac009bd0 tx=0x7f88ac005f90 comp rx=0 tx=0).stop 2024-09-11T00:06:14.691 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:14.688+0000 7f88c3a00640 1 -- 172.21.15.12:0/723189921 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f88bc1a0650 msgr2=0x7f88bc1a0b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:06:14.691 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:14.688+0000 7f88c3a00640 1 --2- 172.21.15.12:0/723189921 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f88bc1a0650 0x7f88bc1a0b00 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f88b400eb10 tx=0x7f88b400c2d0 comp rx=0 tx=0).stop 2024-09-11T00:06:14.691 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:14.688+0000 7f88c3a00640 1 -- 172.21.15.12:0/723189921 shutdown_connections 2024-09-11T00:06:14.691 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:14.688+0000 7f88c3a00640 1 --2- 172.21.15.12:0/723189921 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f88a00b53f0 0x7f88a00b78b0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:14.691 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:14.688+0000 7f88c3a00640 1 --2- 172.21.15.12:0/723189921 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f88bc1a0650 0x7f88bc1a0b00 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:14.691 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:14.688+0000 7f88c3a00640 1 --2- 172.21.15.12:0/723189921 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f88bc076f80 0x7f88bc19fa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:14.691 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:14.688+0000 7f88c3a00640 1 --2- 172.21.15.12:0/723189921 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f88bc075b20 0x7f88bc19f430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:14.691 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:14.689+0000 7f88c3a00640 1 -- 172.21.15.12:0/723189921 >> 172.21.15.12:0/723189921 conn(0x7f88bc0fe320 msgr2=0x7f88bc0fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:06:14.691 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:14.689+0000 7f88c3a00640 1 -- 172.21.15.12:0/723189921 shutdown_connections 2024-09-11T00:06:14.691 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:14.689+0000 7f88c3a00640 1 -- 172.21.15.12:0/723189921 wait complete. 2024-09-11T00:06:15.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:06:15 smithi012 ceph-mon[30333]: pgmap v371: 129 pgs: 129 active+clean; 647 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-11T00:06:15.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:06:15 smithi012 ceph-mon[30333]: from='client.15174 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T00:06:15.351 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ff3ad74a-6fcf-11ef-bcea-c7b262605968 -- bash -c 'ceph orch ls' 2024-09-11T00:06:15.532 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:06:15 smithi165 ceph-mon[35356]: pgmap v371: 129 pgs: 129 active+clean; 647 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-11T00:06:15.532 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:06:15 smithi165 ceph-mon[35356]: from='client.15174 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T00:06:15.580 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:06:15 smithi178 ceph-mon[35919]: pgmap v371: 129 pgs: 129 active+clean; 647 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-11T00:06:15.581 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:06:15 smithi178 ceph-mon[35919]: from='client.15174 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T00:06:15.729 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:06:16.275 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:16.273+0000 7fedd3200640 1 -- 172.21.15.12:0/2582425805 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fedcc106760 msgr2=0x7fedcc10d0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:06:16.275 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:16.273+0000 7fedd3200640 1 --2- 172.21.15.12:0/2582425805 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fedcc106760 0x7fedcc10d0e0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fedc4009d00 tx=0x7fedc402a910 comp rx=0 tx=0).stop 2024-09-11T00:06:16.275 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:16.273+0000 7fedd3200640 1 -- 172.21.15.12:0/2582425805 shutdown_connections 2024-09-11T00:06:16.275 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:16.273+0000 7fedd3200640 1 --2- 172.21.15.12:0/2582425805 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fedcc106760 0x7fedcc10d0e0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:16.275 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:16.273+0000 7fedd3200640 1 --2- 172.21.15.12:0/2582425805 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fedcc105da0 0x7fedcc106220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:16.275 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:16.273+0000 7fedd3200640 1 --2- 172.21.15.12:0/2582425805 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fedcc104c40 0x7fedcc105040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:16.275 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:16.273+0000 7fedd3200640 1 -- 172.21.15.12:0/2582425805 >> 172.21.15.12:0/2582425805 conn(0x7fedcc100510 msgr2=0x7fedcc102950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:06:16.275 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:16.273+0000 7fedd3200640 1 -- 172.21.15.12:0/2582425805 shutdown_connections 2024-09-11T00:06:16.275 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:16.273+0000 7fedd3200640 1 -- 172.21.15.12:0/2582425805 wait complete. 2024-09-11T00:06:16.276 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:16.275+0000 7fedd3200640 1 Processor -- start 2024-09-11T00:06:16.276 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:16.275+0000 7fedd3200640 1 -- start start 2024-09-11T00:06:16.277 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:16.275+0000 7fedd3200640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fedcc104c40 0x7fedcc19b030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:06:16.277 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:16.275+0000 7fedd3200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fedcc105da0 0x7fedcc19b680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:06:16.277 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:16.275+0000 7fedd3200640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fedcc106760 0x7fedcc19c2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:06:16.277 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:16.275+0000 7fedd3200640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fedcc1108a0 con 0x7fedcc106760 2024-09-11T00:06:16.277 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:16.275+0000 7fedd3200640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fedcc110720 con 0x7fedcc105da0 2024-09-11T00:06:16.277 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:16.275+0000 7fedd3200640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fedcc110a20 con 0x7fedcc104c40 2024-09-11T00:06:16.277 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:16.276+0000 7fedcb400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fedcc105da0 0x7fedcc19b680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:06:16.277 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:16.276+0000 7fedd1200640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fedcc106760 0x7fedcc19c2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:06:16.277 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:16.276+0000 7fedcb400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fedcc105da0 0x7fedcc19b680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.12:45078/0 (socket says 172.21.15.12:45078) 2024-09-11T00:06:16.277 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:16.276+0000 7fedcb400640 1 -- 172.21.15.12:0/1058722098 learned_addr learned my addr 172.21.15.12:0/1058722098 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:06:16.278 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:16.276+0000 7fedcbe00640 1 --2- 172.21.15.12:0/1058722098 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fedcc104c40 0x7fedcc19b030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:06:16.278 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:16.276+0000 7fedcb400640 1 -- 172.21.15.12:0/1058722098 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fedcc104c40 msgr2=0x7fedcc19b030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:06:16.278 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:16.276+0000 7fedcb400640 1 --2- 172.21.15.12:0/1058722098 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fedcc104c40 0x7fedcc19b030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:16.279 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:16.276+0000 7fedcb400640 1 -- 172.21.15.12:0/1058722098 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fedcc106760 msgr2=0x7fedcc19c2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:06:16.279 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:16.276+0000 7fedcb400640 1 --2- 172.21.15.12:0/1058722098 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fedcc106760 0x7fedcc19c2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:16.279 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:16.276+0000 7fedcb400640 1 -- 172.21.15.12:0/1058722098 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fedcc19cb60 con 0x7fedcc105da0 2024-09-11T00:06:16.279 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:16.276+0000 7fedcb400640 1 --2- 172.21.15.12:0/1058722098 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fedcc105da0 0x7fedcc19b680 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fedb800e9f0 tx=0x7fedb800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:06:16.279 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:16.277+0000 7fedc8c00640 1 -- 172.21.15.12:0/1058722098 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedb800cd80 con 0x7fedcc105da0 2024-09-11T00:06:16.279 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:16.277+0000 7fedc8c00640 1 -- 172.21.15.12:0/1058722098 <== mon.2 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fedb80108e0 con 0x7fedcc105da0 2024-09-11T00:06:16.279 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:16.277+0000 7fedd3200640 1 -- 172.21.15.12:0/1058722098 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fedcc19ce50 con 0x7fedcc105da0 2024-09-11T00:06:16.279 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:16.277+0000 7fedc8c00640 1 -- 172.21.15.12:0/1058722098 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedb8018980 con 0x7fedcc105da0 2024-09-11T00:06:16.279 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:16.277+0000 7fedd3200640 1 -- 172.21.15.12:0/1058722098 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fedcc1a3530 con 0x7fedcc105da0 2024-09-11T00:06:16.280 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:16.278+0000 7fedd3200640 1 -- 172.21.15.12:0/1058722098 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fedcc10bf60 con 0x7fedcc105da0 2024-09-11T00:06:16.282 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:16.281+0000 7fedc8c00640 1 -- 172.21.15.12:0/1058722098 <== mon.2 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 17) ==== 153301+0+0 (secure 0 0 0) 0x7fedb8010430 con 0x7fedcc105da0 2024-09-11T00:06:16.283 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:16.282+0000 7fedc8c00640 1 --2- 172.21.15.12:0/1058722098 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fedb00b5370 0x7fedb00b7830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:06:16.283 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:16.282+0000 7fedc8c00640 1 -- 172.21.15.12:0/1058722098 <== mon.2 v2:172.21.15.165:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 9092+0+0 (secure 0 0 0) 0x7fedb80e63b0 con 0x7fedcc105da0 2024-09-11T00:06:16.283 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:16.283+0000 7fedcbe00640 1 --2- 172.21.15.12:0/1058722098 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fedb00b5370 0x7fedb00b7830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:06:16.284 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:16.283+0000 7fedcbe00640 1 --2- 172.21.15.12:0/1058722098 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fedb00b5370 0x7fedb00b7830 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fedbc009bd0 tx=0x7fedbc005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:06:16.287 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:16.286+0000 7fedc8c00640 1 -- 172.21.15.12:0/1058722098 <== mon.2 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7fedb80ac760 con 0x7fedcc105da0 2024-09-11T00:06:16.494 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:16.493+0000 7fedd3200640 1 -- 172.21.15.12:0/1058722098 --> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) -- 0x7fedcc063590 con 0x7fedb00b5370 2024-09-11T00:06:16.501 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:16.500+0000 7fedc8c00640 1 -- 172.21.15.12:0/1058722098 <== mgr.14150 v2:172.21.15.12:6800/3442939137 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+980 (secure 0 0 0) 0x7fedcc063590 con 0x7fedb00b5370 2024-09-11T00:06:16.501 INFO:teuthology.orchestra.run.smithi012.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2024-09-11T00:06:16.501 INFO:teuthology.orchestra.run.smithi012.stdout:mds.all 12/12 11s ago 3m smithi012=a;smithi012=d;smithi012=g;smithi012=j;smithi165=b;smithi165=e;smithi165=h;smithi165=k;smithi178=c;smithi178=f;smithi178=i;smithi178=l;count:12 2024-09-11T00:06:16.501 INFO:teuthology.orchestra.run.smithi012.stdout:mgr 3/3 11s ago 8m smithi012=x;smithi165=y;smithi178=z;count:3 2024-09-11T00:06:16.501 INFO:teuthology.orchestra.run.smithi012.stdout:mon 3/3 11s ago 9m smithi012:172.21.15.12=a;smithi165:172.21.15.165=b;smithi178:172.21.15.178=c;count:3 2024-09-11T00:06:16.501 INFO:teuthology.orchestra.run.smithi012.stdout:osd 12 11s ago - 2024-09-11T00:06:16.503 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:16.502+0000 7fedd3200640 1 -- 172.21.15.12:0/1058722098 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fedb00b5370 msgr2=0x7fedb00b7830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:06:16.503 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:16.502+0000 7fedd3200640 1 --2- 172.21.15.12:0/1058722098 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fedb00b5370 0x7fedb00b7830 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fedbc009bd0 tx=0x7fedbc005f90 comp rx=0 tx=0).stop 2024-09-11T00:06:16.504 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:16.502+0000 7fedd3200640 1 -- 172.21.15.12:0/1058722098 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fedcc105da0 msgr2=0x7fedcc19b680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:06:16.504 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:16.502+0000 7fedd3200640 1 --2- 172.21.15.12:0/1058722098 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fedcc105da0 0x7fedcc19b680 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fedb800e9f0 tx=0x7fedb800eeb0 comp rx=0 tx=0).stop 2024-09-11T00:06:16.504 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:16.502+0000 7fedd3200640 1 -- 172.21.15.12:0/1058722098 shutdown_connections 2024-09-11T00:06:16.504 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:16.502+0000 7fedd3200640 1 --2- 172.21.15.12:0/1058722098 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fedb00b5370 0x7fedb00b7830 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:16.504 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:16.502+0000 7fedd3200640 1 --2- 172.21.15.12:0/1058722098 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fedcc106760 0x7fedcc19c2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:16.504 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:16.502+0000 7fedd3200640 1 --2- 172.21.15.12:0/1058722098 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fedcc105da0 0x7fedcc19b680 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:16.504 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:16.502+0000 7fedd3200640 1 --2- 172.21.15.12:0/1058722098 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fedcc104c40 0x7fedcc19b030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:16.504 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:16.502+0000 7fedd3200640 1 -- 172.21.15.12:0/1058722098 >> 172.21.15.12:0/1058722098 conn(0x7fedcc100510 msgr2=0x7fedcc109130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:06:16.504 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:16.502+0000 7fedd3200640 1 -- 172.21.15.12:0/1058722098 shutdown_connections 2024-09-11T00:06:16.504 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:16.502+0000 7fedd3200640 1 -- 172.21.15.12:0/1058722098 wait complete. 2024-09-11T00:06:17.331 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ff3ad74a-6fcf-11ef-bcea-c7b262605968 -- bash -c 'ceph orch host ls' 2024-09-11T00:06:17.532 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:06:17 smithi165 ceph-mon[35356]: pgmap v372: 129 pgs: 129 active+clean; 647 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-11T00:06:17.532 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:06:17 smithi165 ceph-mon[35356]: from='client.25220 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T00:06:17.562 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:06:17 smithi012 ceph-mon[30333]: pgmap v372: 129 pgs: 129 active+clean; 647 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-11T00:06:17.563 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:06:17 smithi012 ceph-mon[30333]: from='client.25220 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T00:06:17.580 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:06:17 smithi178 ceph-mon[35919]: pgmap v372: 129 pgs: 129 active+clean; 647 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-11T00:06:17.581 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:06:17 smithi178 ceph-mon[35919]: from='client.25220 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T00:06:17.711 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:06:18.258 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:18.240+0000 7f5304e00640 1 -- 172.21.15.12:0/264008176 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5300105c00 msgr2=0x7f5300106080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:06:18.258 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:18.240+0000 7f5304e00640 1 --2- 172.21.15.12:0/264008176 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5300105c00 0x7f5300106080 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f52e8009960 tx=0x7f52e802a460 comp rx=0 tx=0).stop 2024-09-11T00:06:18.258 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:18.240+0000 7f5304e00640 1 -- 172.21.15.12:0/264008176 shutdown_connections 2024-09-11T00:06:18.258 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:18.240+0000 7f5304e00640 1 --2- 172.21.15.12:0/264008176 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f53001065c0 0x7f530010cf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:18.258 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:18.240+0000 7f5304e00640 1 --2- 172.21.15.12:0/264008176 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5300105c00 0x7f5300106080 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:18.258 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:18.240+0000 7f5304e00640 1 --2- 172.21.15.12:0/264008176 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5300104aa0 0x7f5300104ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:18.259 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:18.240+0000 7f5304e00640 1 -- 172.21.15.12:0/264008176 >> 172.21.15.12:0/264008176 conn(0x7f5300100370 msgr2=0x7f53001027b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:06:18.259 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:18.240+0000 7f5304e00640 1 -- 172.21.15.12:0/264008176 shutdown_connections 2024-09-11T00:06:18.259 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:18.240+0000 7f5304e00640 1 -- 172.21.15.12:0/264008176 wait complete. 2024-09-11T00:06:18.259 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:18.241+0000 7f5304e00640 1 Processor -- start 2024-09-11T00:06:18.259 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:18.241+0000 7f5304e00640 1 -- start start 2024-09-11T00:06:18.259 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:18.241+0000 7f5304e00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5300104aa0 0x7f530019add0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:06:18.259 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:18.241+0000 7f5304e00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5300105c00 0x7f530019b420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:06:18.259 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:18.241+0000 7f5304e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f53001065c0 0x7f530019c060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:06:18.260 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:18.241+0000 7f5304e00640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f5300110720 con 0x7f5300105c00 2024-09-11T00:06:18.260 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:18.241+0000 7f5304e00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f53001105a0 con 0x7f53001065c0 2024-09-11T00:06:18.260 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:18.241+0000 7f5304e00640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f53001108a0 con 0x7f5300104aa0 2024-09-11T00:06:18.260 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:18.242+0000 7f52ff400640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5300104aa0 0x7f530019add0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:06:18.260 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:18.242+0000 7f52f7e00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5300105c00 0x7f530019b420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:06:18.260 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:18.242+0000 7f52ff400640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5300104aa0 0x7f530019add0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.12:35466/0 (socket says 172.21.15.12:35466) 2024-09-11T00:06:18.260 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:18.242+0000 7f52ff400640 1 -- 172.21.15.12:0/3386462521 learned_addr learned my addr 172.21.15.12:0/3386462521 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:06:18.260 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:18.242+0000 7f52ffe00640 1 --2- 172.21.15.12:0/3386462521 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f53001065c0 0x7f530019c060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:06:18.261 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:18.242+0000 7f52ff400640 1 -- 172.21.15.12:0/3386462521 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f53001065c0 msgr2=0x7f530019c060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:06:18.261 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:18.242+0000 7f52ff400640 1 --2- 172.21.15.12:0/3386462521 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f53001065c0 0x7f530019c060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:18.261 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:18.242+0000 7f52ff400640 1 -- 172.21.15.12:0/3386462521 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5300105c00 msgr2=0x7f530019b420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:06:18.261 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:18.242+0000 7f52ff400640 1 --2- 172.21.15.12:0/3386462521 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5300105c00 0x7f530019b420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:18.261 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:18.242+0000 7f52ff400640 1 -- 172.21.15.12:0/3386462521 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f530019c900 con 0x7f5300104aa0 2024-09-11T00:06:18.261 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:18.242+0000 7f52ff400640 1 --2- 172.21.15.12:0/3386462521 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5300104aa0 0x7f530019add0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f52f000ea60 tx=0x7f52f000ef20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:06:18.261 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:18.242+0000 7f52fcc00640 1 -- 172.21.15.12:0/3386462521 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52f000cde0 con 0x7f5300104aa0 2024-09-11T00:06:18.262 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:18.243+0000 7f52fcc00640 1 -- 172.21.15.12:0/3386462521 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f52f00035f0 con 0x7f5300104aa0 2024-09-11T00:06:18.262 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:18.243+0000 7f5304e00640 1 -- 172.21.15.12:0/3386462521 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f530019cbf0 con 0x7f5300104aa0 2024-09-11T00:06:18.262 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:18.243+0000 7f52fcc00640 1 -- 172.21.15.12:0/3386462521 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52f0010690 con 0x7f5300104aa0 2024-09-11T00:06:18.262 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:18.243+0000 7f5304e00640 1 -- 172.21.15.12:0/3386462521 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53001a32d0 con 0x7f5300104aa0 2024-09-11T00:06:18.262 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:18.244+0000 7f5304e00640 1 -- 172.21.15.12:0/3386462521 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f530010bdc0 con 0x7f5300104aa0 2024-09-11T00:06:18.262 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:18.246+0000 7f52fcc00640 1 -- 172.21.15.12:0/3386462521 <== mon.1 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 17) ==== 153301+0+0 (secure 0 0 0) 0x7f52f0004000 con 0x7f5300104aa0 2024-09-11T00:06:18.262 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:18.247+0000 7f52fcc00640 1 --2- 172.21.15.12:0/3386462521 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f52e40b5490 0x7f52e40b7950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:06:18.262 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:18.248+0000 7f52fcc00640 1 -- 172.21.15.12:0/3386462521 <== mon.1 v2:172.21.15.178:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 9092+0+0 (secure 0 0 0) 0x7f52f00e6560 con 0x7f5300104aa0 2024-09-11T00:06:18.263 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:18.248+0000 7f52f7e00640 1 --2- 172.21.15.12:0/3386462521 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f52e40b5490 0x7f52e40b7950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:06:18.263 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:18.249+0000 7f52f7e00640 1 --2- 172.21.15.12:0/3386462521 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f52e40b5490 0x7f52e40b7950 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f52e8000c00 tx=0x7f52e8005d50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:06:18.263 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:18.249+0000 7f52fcc00640 1 -- 172.21.15.12:0/3386462521 <== mon.1 v2:172.21.15.178:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f52f0056220 con 0x7f5300104aa0 2024-09-11T00:06:18.451 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:18.450+0000 7f5304e00640 1 -- 172.21.15.12:0/3386462521 --> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) -- 0x7f5300063590 con 0x7f52e40b5490 2024-09-11T00:06:18.452 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:18.452+0000 7f52fcc00640 1 -- 172.21.15.12:0/3386462521 <== mgr.14150 v2:172.21.15.12:6800/3442939137 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+191 (secure 0 0 0) 0x7f5300063590 con 0x7f52e40b5490 2024-09-11T00:06:18.453 INFO:teuthology.orchestra.run.smithi012.stdout:HOST ADDR LABELS STATUS 2024-09-11T00:06:18.453 INFO:teuthology.orchestra.run.smithi012.stdout:smithi012 172.21.15.12 2024-09-11T00:06:18.453 INFO:teuthology.orchestra.run.smithi012.stdout:smithi165 172.21.15.165 2024-09-11T00:06:18.453 INFO:teuthology.orchestra.run.smithi012.stdout:smithi178 172.21.15.178 2024-09-11T00:06:18.453 INFO:teuthology.orchestra.run.smithi012.stdout:3 hosts in cluster 2024-09-11T00:06:18.455 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:18.454+0000 7f5304e00640 1 -- 172.21.15.12:0/3386462521 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f52e40b5490 msgr2=0x7f52e40b7950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:06:18.455 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:18.454+0000 7f5304e00640 1 --2- 172.21.15.12:0/3386462521 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f52e40b5490 0x7f52e40b7950 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f52e8000c00 tx=0x7f52e8005d50 comp rx=0 tx=0).stop 2024-09-11T00:06:18.455 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:18.454+0000 7f5304e00640 1 -- 172.21.15.12:0/3386462521 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5300104aa0 msgr2=0x7f530019add0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:06:18.455 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:18.454+0000 7f5304e00640 1 --2- 172.21.15.12:0/3386462521 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5300104aa0 0x7f530019add0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f52f000ea60 tx=0x7f52f000ef20 comp rx=0 tx=0).stop 2024-09-11T00:06:18.455 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:18.454+0000 7f5304e00640 1 -- 172.21.15.12:0/3386462521 shutdown_connections 2024-09-11T00:06:18.455 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:18.454+0000 7f5304e00640 1 --2- 172.21.15.12:0/3386462521 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f52e40b5490 0x7f52e40b7950 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:18.455 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:18.454+0000 7f5304e00640 1 --2- 172.21.15.12:0/3386462521 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f53001065c0 0x7f530019c060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:18.455 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:18.454+0000 7f5304e00640 1 --2- 172.21.15.12:0/3386462521 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5300105c00 0x7f530019b420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:18.456 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:18.454+0000 7f5304e00640 1 --2- 172.21.15.12:0/3386462521 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5300104aa0 0x7f530019add0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:18.456 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:18.454+0000 7f5304e00640 1 -- 172.21.15.12:0/3386462521 >> 172.21.15.12:0/3386462521 conn(0x7f5300100370 msgr2=0x7f5300108f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:06:18.456 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:18.454+0000 7f5304e00640 1 -- 172.21.15.12:0/3386462521 shutdown_connections 2024-09-11T00:06:18.456 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:18.454+0000 7f5304e00640 1 -- 172.21.15.12:0/3386462521 wait complete. 2024-09-11T00:06:19.169 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ff3ad74a-6fcf-11ef-bcea-c7b262605968 -- bash -c 'ceph orch device ls' 2024-09-11T00:06:19.532 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:06:19 smithi165 ceph-mon[35356]: pgmap v373: 129 pgs: 129 active+clean; 647 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-11T00:06:19.532 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:06:19 smithi165 ceph-mon[35356]: from='client.25204 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T00:06:19.532 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:06:19 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.5", "id": [9, 10]} : dispatch 2024-09-11T00:06:19.532 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:06:19 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.6", "id": [5, 8]} : dispatch 2024-09-11T00:06:19.533 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:06:19 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.b", "id": [9, 8]} : dispatch 2024-09-11T00:06:19.533 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:06:19 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.13", "id": [9, 8]} : dispatch 2024-09-11T00:06:19.553 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:06:19.563 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:06:19 smithi012 ceph-mon[30333]: pgmap v373: 129 pgs: 129 active+clean; 647 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-11T00:06:19.563 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:06:19 smithi012 ceph-mon[30333]: from='client.25204 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T00:06:19.563 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:06:19 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.5", "id": [9, 10]} : dispatch 2024-09-11T00:06:19.563 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:06:19 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.6", "id": [5, 8]} : dispatch 2024-09-11T00:06:19.563 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:06:19 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.b", "id": [9, 8]} : dispatch 2024-09-11T00:06:19.563 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:06:19 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.13", "id": [9, 8]} : dispatch 2024-09-11T00:06:19.581 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:06:19 smithi178 ceph-mon[35919]: pgmap v373: 129 pgs: 129 active+clean; 647 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-11T00:06:19.581 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:06:19 smithi178 ceph-mon[35919]: from='client.25204 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T00:06:19.581 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:06:19 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.5", "id": [9, 10]} : dispatch 2024-09-11T00:06:19.581 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:06:19 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.6", "id": [5, 8]} : dispatch 2024-09-11T00:06:19.581 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:06:19 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.b", "id": [9, 8]} : dispatch 2024-09-11T00:06:19.581 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:06:19 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.13", "id": [9, 8]} : dispatch 2024-09-11T00:06:20.098 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:20.097+0000 7ff851800640 1 -- 172.21.15.12:0/2477711369 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff84c076f20 msgr2=0x7ff84c075580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:06:20.098 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:20.097+0000 7ff851800640 1 --2- 172.21.15.12:0/2477711369 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff84c076f20 0x7ff84c075580 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7ff83c009ca0 tx=0x7ff83c02a8b0 comp rx=0 tx=0).stop 2024-09-11T00:06:20.098 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:20.097+0000 7ff851800640 1 -- 172.21.15.12:0/2477711369 shutdown_connections 2024-09-11T00:06:20.098 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:20.097+0000 7ff851800640 1 --2- 172.21.15.12:0/2477711369 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff84c10a820 0x7ff84c10cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:20.099 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:20.097+0000 7ff851800640 1 --2- 172.21.15.12:0/2477711369 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff84c075ac0 0x7ff84c075f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:20.099 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:20.097+0000 7ff851800640 1 --2- 172.21.15.12:0/2477711369 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff84c076f20 0x7ff84c075580 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:20.099 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:20.097+0000 7ff851800640 1 -- 172.21.15.12:0/2477711369 >> 172.21.15.12:0/2477711369 conn(0x7ff84c0fe180 msgr2=0x7ff84c1005a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:06:20.099 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:20.097+0000 7ff851800640 1 -- 172.21.15.12:0/2477711369 shutdown_connections 2024-09-11T00:06:20.099 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:20.097+0000 7ff851800640 1 -- 172.21.15.12:0/2477711369 wait complete. 2024-09-11T00:06:20.100 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:20.098+0000 7ff851800640 1 Processor -- start 2024-09-11T00:06:20.100 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:20.098+0000 7ff851800640 1 -- start start 2024-09-11T00:06:20.100 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:20.098+0000 7ff851800640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff84c075ac0 0x7ff84c19adb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:06:20.100 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:20.098+0000 7ff851800640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff84c076f20 0x7ff84c19b400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:06:20.100 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:20.098+0000 7ff851800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff84c10a820 0x7ff84c19c040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:06:20.100 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:20.098+0000 7ff851800640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7ff84c110760 con 0x7ff84c075ac0 2024-09-11T00:06:20.100 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:20.098+0000 7ff851800640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7ff84c1105e0 con 0x7ff84c10a820 2024-09-11T00:06:20.101 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:20.098+0000 7ff851800640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7ff84c1108e0 con 0x7ff84c076f20 2024-09-11T00:06:20.101 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:20.099+0000 7ff84be00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff84c075ac0 0x7ff84c19adb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:06:20.101 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:20.099+0000 7ff84be00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff84c075ac0 0x7ff84c19adb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:48662/0 (socket says 172.21.15.12:48662) 2024-09-11T00:06:20.101 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:20.099+0000 7ff84be00640 1 -- 172.21.15.12:0/2651496537 learned_addr learned my addr 172.21.15.12:0/2651496537 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:06:20.101 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:20.099+0000 7ff850e00640 1 --2- 172.21.15.12:0/2651496537 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff84c10a820 0x7ff84c19c040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:06:20.101 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:20.099+0000 7ff843400640 1 --2- 172.21.15.12:0/2651496537 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff84c076f20 0x7ff84c19b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:06:20.101 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:20.099+0000 7ff84be00640 1 -- 172.21.15.12:0/2651496537 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff84c076f20 msgr2=0x7ff84c19b400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:06:20.101 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:20.099+0000 7ff84be00640 1 --2- 172.21.15.12:0/2651496537 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff84c076f20 0x7ff84c19b400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:20.101 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:20.099+0000 7ff84be00640 1 -- 172.21.15.12:0/2651496537 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff84c10a820 msgr2=0x7ff84c19c040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:06:20.101 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:20.099+0000 7ff84be00640 1 --2- 172.21.15.12:0/2651496537 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff84c10a820 0x7ff84c19c040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:20.101 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:20.099+0000 7ff84be00640 1 -- 172.21.15.12:0/2651496537 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff84c19c8e0 con 0x7ff84c075ac0 2024-09-11T00:06:20.102 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:20.100+0000 7ff84be00640 1 --2- 172.21.15.12:0/2651496537 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff84c075ac0 0x7ff84c19adb0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7ff83c02aea0 tx=0x7ff83c003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:06:20.102 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:20.100+0000 7ff849600640 1 -- 172.21.15.12:0/2651496537 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff83c004450 con 0x7ff84c075ac0 2024-09-11T00:06:20.102 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:20.100+0000 7ff849600640 1 -- 172.21.15.12:0/2651496537 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff83c040070 con 0x7ff84c075ac0 2024-09-11T00:06:20.103 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:20.100+0000 7ff849600640 1 -- 172.21.15.12:0/2651496537 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff83c045cc0 con 0x7ff84c075ac0 2024-09-11T00:06:20.103 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:20.100+0000 7ff851800640 1 -- 172.21.15.12:0/2651496537 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff84c19cb70 con 0x7ff84c075ac0 2024-09-11T00:06:20.103 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:20.100+0000 7ff851800640 1 -- 172.21.15.12:0/2651496537 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff84c1a32b0 con 0x7ff84c075ac0 2024-09-11T00:06:20.103 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:20.101+0000 7ff851800640 1 -- 172.21.15.12:0/2651496537 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff84c109630 con 0x7ff84c075ac0 2024-09-11T00:06:20.104 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:20.103+0000 7ff849600640 1 -- 172.21.15.12:0/2651496537 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 17) ==== 153301+0+0 (secure 0 0 0) 0x7ff83c04d030 con 0x7ff84c075ac0 2024-09-11T00:06:20.105 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:20.105+0000 7ff849600640 1 --2- 172.21.15.12:0/2651496537 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7ff8300b5490 0x7ff8300b7950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:06:20.105 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:20.105+0000 7ff849600640 1 -- 172.21.15.12:0/2651496537 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 9208+0+0 (secure 0 0 0) 0x7ff83c1070b0 con 0x7ff84c075ac0 2024-09-11T00:06:20.106 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:20.105+0000 7ff843400640 1 --2- 172.21.15.12:0/2651496537 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7ff8300b5490 0x7ff8300b7950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:06:20.106 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:20.106+0000 7ff843400640 1 --2- 172.21.15.12:0/2651496537 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7ff8300b5490 0x7ff8300b7950 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7ff834005d00 tx=0x7ff83400b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:06:20.107 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:20.107+0000 7ff849600640 1 -- 172.21.15.12:0/2651496537 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7ff83c0cd9e0 con 0x7ff84c075ac0 2024-09-11T00:06:20.312 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:06:20 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.5", "id": [9, 10]}]': finished 2024-09-11T00:06:20.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:06:20 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.6", "id": [5, 8]}]': finished 2024-09-11T00:06:20.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:06:20 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.b", "id": [9, 8]}]': finished 2024-09-11T00:06:20.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:06:20 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.13", "id": [9, 8]}]': finished 2024-09-11T00:06:20.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:06:20 smithi012 ceph-mon[30333]: osdmap e84: 12 total, 12 up, 12 in 2024-09-11T00:06:20.322 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:20.322+0000 7ff851800640 1 -- 172.21.15.12:0/2651496537 --> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) -- 0x7ff84c063590 con 0x7ff8300b5490 2024-09-11T00:06:20.324 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:20.323+0000 7ff849600640 1 -- 172.21.15.12:0/2651496537 <== mgr.14150 v2:172.21.15.12:6800/3442939137 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+532 (secure 0 0 0) 0x7ff84c063590 con 0x7ff8300b5490 2024-09-11T00:06:20.324 INFO:teuthology.orchestra.run.smithi012.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2024-09-11T00:06:20.325 INFO:teuthology.orchestra.run.smithi012.stdout:smithi012 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT5331005J400BGN 372G No 4m ago Has a FileSystem, LVM detected 2024-09-11T00:06:20.325 INFO:teuthology.orchestra.run.smithi012.stdout:smithi165 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT623300CE400BGN 372G No 3m ago Has a FileSystem, LVM detected 2024-09-11T00:06:20.325 INFO:teuthology.orchestra.run.smithi012.stdout:smithi178 /dev/nvme0n1 ssd INTEL_SSDPED1D480GA_PHMB75130029480DGN 447G No 3m ago Has a FileSystem, LVM detected 2024-09-11T00:06:20.327 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:20.326+0000 7ff851800640 1 -- 172.21.15.12:0/2651496537 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7ff8300b5490 msgr2=0x7ff8300b7950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:06:20.327 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:20.326+0000 7ff851800640 1 --2- 172.21.15.12:0/2651496537 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7ff8300b5490 0x7ff8300b7950 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7ff834005d00 tx=0x7ff83400b410 comp rx=0 tx=0).stop 2024-09-11T00:06:20.327 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:20.326+0000 7ff851800640 1 -- 172.21.15.12:0/2651496537 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff84c075ac0 msgr2=0x7ff84c19adb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:06:20.327 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:20.326+0000 7ff851800640 1 --2- 172.21.15.12:0/2651496537 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff84c075ac0 0x7ff84c19adb0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7ff83c02aea0 tx=0x7ff83c003c20 comp rx=0 tx=0).stop 2024-09-11T00:06:20.327 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:20.326+0000 7ff851800640 1 -- 172.21.15.12:0/2651496537 shutdown_connections 2024-09-11T00:06:20.327 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:20.326+0000 7ff851800640 1 --2- 172.21.15.12:0/2651496537 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7ff8300b5490 0x7ff8300b7950 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:20.327 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:20.326+0000 7ff851800640 1 --2- 172.21.15.12:0/2651496537 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff84c10a820 0x7ff84c19c040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:20.327 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:20.326+0000 7ff851800640 1 --2- 172.21.15.12:0/2651496537 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff84c076f20 0x7ff84c19b400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:20.327 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:20.326+0000 7ff851800640 1 --2- 172.21.15.12:0/2651496537 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff84c075ac0 0x7ff84c19adb0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:20.328 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:20.326+0000 7ff851800640 1 -- 172.21.15.12:0/2651496537 >> 172.21.15.12:0/2651496537 conn(0x7ff84c0fe180 msgr2=0x7ff84c106800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:06:20.328 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:20.326+0000 7ff851800640 1 -- 172.21.15.12:0/2651496537 shutdown_connections 2024-09-11T00:06:20.328 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:20.326+0000 7ff851800640 1 -- 172.21.15.12:0/2651496537 wait complete. 2024-09-11T00:06:20.532 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:06:20 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.5", "id": [9, 10]}]': finished 2024-09-11T00:06:20.532 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:06:20 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.6", "id": [5, 8]}]': finished 2024-09-11T00:06:20.532 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:06:20 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.b", "id": [9, 8]}]': finished 2024-09-11T00:06:20.532 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:06:20 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.13", "id": [9, 8]}]': finished 2024-09-11T00:06:20.532 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:06:20 smithi165 ceph-mon[35356]: osdmap e84: 12 total, 12 up, 12 in 2024-09-11T00:06:20.581 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:06:20 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.5", "id": [9, 10]}]': finished 2024-09-11T00:06:20.581 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:06:20 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.6", "id": [5, 8]}]': finished 2024-09-11T00:06:20.581 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:06:20 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.b", "id": [9, 8]}]': finished 2024-09-11T00:06:20.581 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:06:20 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.13", "id": [9, 8]}]': finished 2024-09-11T00:06:20.581 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:06:20 smithi178 ceph-mon[35919]: osdmap e84: 12 total, 12 up, 12 in 2024-09-11T00:06:21.055 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-09-11T00:06:21.067 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi012.front.sepia.ceph.com 2024-09-11T00:06:21.068 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ff3ad74a-6fcf-11ef-bcea-c7b262605968 -- bash -c 'ceph fs dump' 2024-09-11T00:06:21.312 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:06:21 smithi012 ceph-mon[30333]: pgmap v375: 129 pgs: 1 peering, 128 active+clean; 647 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 204 B/s wr, 10 op/s 2024-09-11T00:06:21.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:06:21 smithi012 ceph-mon[30333]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2024-09-11T00:06:21.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:06:21 smithi012 ceph-mon[30333]: osdmap e85: 12 total, 12 up, 12 in 2024-09-11T00:06:21.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:06:21 smithi012 ceph-mon[30333]: from='client.15192 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T00:06:21.450 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:06:21.532 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:06:21 smithi165 ceph-mon[35356]: pgmap v375: 129 pgs: 1 peering, 128 active+clean; 647 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 204 B/s wr, 10 op/s 2024-09-11T00:06:21.532 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:06:21 smithi165 ceph-mon[35356]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2024-09-11T00:06:21.532 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:06:21 smithi165 ceph-mon[35356]: osdmap e85: 12 total, 12 up, 12 in 2024-09-11T00:06:21.532 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:06:21 smithi165 ceph-mon[35356]: from='client.15192 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T00:06:21.580 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:06:21 smithi178 ceph-mon[35919]: pgmap v375: 129 pgs: 1 peering, 128 active+clean; 647 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 204 B/s wr, 10 op/s 2024-09-11T00:06:21.581 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:06:21 smithi178 ceph-mon[35919]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2024-09-11T00:06:21.581 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:06:21 smithi178 ceph-mon[35919]: osdmap e85: 12 total, 12 up, 12 in 2024-09-11T00:06:21.581 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:06:21 smithi178 ceph-mon[35919]: from='client.15192 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T00:06:22.004 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:22.003+0000 7fddf7000640 1 -- 172.21.15.12:0/793988001 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fddf0102a30 msgr2=0x7fddf0102e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:06:22.005 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:22.003+0000 7fddf7000640 1 --2- 172.21.15.12:0/793988001 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fddf0102a30 0x7fddf0102e30 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fdde0009930 tx=0x7fdde002a420 comp rx=0 tx=0).stop 2024-09-11T00:06:22.005 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:22.003+0000 7fddf7000640 1 -- 172.21.15.12:0/793988001 shutdown_connections 2024-09-11T00:06:22.005 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:22.003+0000 7fddf7000640 1 --2- 172.21.15.12:0/793988001 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fddf0104550 0x7fddf010aed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:22.005 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:22.003+0000 7fddf7000640 1 --2- 172.21.15.12:0/793988001 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fddf0103b90 0x7fddf0104010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:22.005 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:22.003+0000 7fddf7000640 1 --2- 172.21.15.12:0/793988001 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fddf0102a30 0x7fddf0102e30 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:22.005 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:22.003+0000 7fddf7000640 1 -- 172.21.15.12:0/793988001 >> 172.21.15.12:0/793988001 conn(0x7fddf00fe320 msgr2=0x7fddf0100740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:06:22.005 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:22.003+0000 7fddf7000640 1 -- 172.21.15.12:0/793988001 shutdown_connections 2024-09-11T00:06:22.005 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:22.003+0000 7fddf7000640 1 -- 172.21.15.12:0/793988001 wait complete. 2024-09-11T00:06:22.007 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:22.004+0000 7fddf7000640 1 Processor -- start 2024-09-11T00:06:22.007 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:22.004+0000 7fddf7000640 1 -- start start 2024-09-11T00:06:22.007 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:22.005+0000 7fddf7000640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fddf0102a30 0x7fddf0076cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:06:22.007 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:22.005+0000 7fddf7000640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fddf0103b90 0x7fddf0075540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:06:22.007 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:22.005+0000 7fddf7000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fddf0104550 0x7fddf0075a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:06:22.007 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:22.005+0000 7fddf7000640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fddf010e690 con 0x7fddf0102a30 2024-09-11T00:06:22.007 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:22.005+0000 7fddf7000640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fddf010e510 con 0x7fddf0104550 2024-09-11T00:06:22.007 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:22.005+0000 7fddf7000640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fddf010e810 con 0x7fddf0103b90 2024-09-11T00:06:22.008 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:22.005+0000 7fddefe00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fddf0102a30 0x7fddf0076cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:06:22.008 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:22.005+0000 7fddefe00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fddf0102a30 0x7fddf0076cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:48676/0 (socket says 172.21.15.12:48676) 2024-09-11T00:06:22.008 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:22.005+0000 7fddefe00640 1 -- 172.21.15.12:0/1628956856 learned_addr learned my addr 172.21.15.12:0/1628956856 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:06:22.008 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:22.005+0000 7fdde7400640 1 --2- 172.21.15.12:0/1628956856 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fddf0103b90 0x7fddf0075540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:06:22.008 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:22.005+0000 7fddf5000640 1 --2- 172.21.15.12:0/1628956856 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fddf0104550 0x7fddf0075a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:06:22.008 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:22.006+0000 7fddefe00640 1 -- 172.21.15.12:0/1628956856 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fddf0103b90 msgr2=0x7fddf0075540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:06:22.008 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:22.006+0000 7fddefe00640 1 --2- 172.21.15.12:0/1628956856 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fddf0103b90 0x7fddf0075540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:22.008 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:22.006+0000 7fddefe00640 1 -- 172.21.15.12:0/1628956856 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fddf0104550 msgr2=0x7fddf0075a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:06:22.008 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:22.006+0000 7fddefe00640 1 --2- 172.21.15.12:0/1628956856 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fddf0104550 0x7fddf0075a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:22.008 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:22.006+0000 7fddefe00640 1 -- 172.21.15.12:0/1628956856 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fddf0076340 con 0x7fddf0102a30 2024-09-11T00:06:22.008 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:22.006+0000 7fdde7400640 1 --2- 172.21.15.12:0/1628956856 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fddf0103b90 0x7fddf0075540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:06:22.008 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:22.006+0000 7fddefe00640 1 --2- 172.21.15.12:0/1628956856 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fddf0102a30 0x7fddf0076cd0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fdde0005ab0 tx=0x7fdde00042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:06:22.009 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:22.006+0000 7fdded600640 1 -- 172.21.15.12:0/1628956856 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdde003f030 con 0x7fddf0102a30 2024-09-11T00:06:22.009 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:22.006+0000 7fdded600640 1 -- 172.21.15.12:0/1628956856 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdde0032e40 con 0x7fddf0102a30 2024-09-11T00:06:22.009 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:22.006+0000 7fdded600640 1 -- 172.21.15.12:0/1628956856 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdde0035070 con 0x7fddf0102a30 2024-09-11T00:06:22.010 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:22.006+0000 7fddf7000640 1 -- 172.21.15.12:0/1628956856 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fddf01a2ed0 con 0x7fddf0102a30 2024-09-11T00:06:22.010 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:22.007+0000 7fddf7000640 1 -- 172.21.15.12:0/1628956856 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fddf01a3270 con 0x7fddf0102a30 2024-09-11T00:06:22.010 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:22.008+0000 7fddf7000640 1 -- 172.21.15.12:0/1628956856 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fddf0109db0 con 0x7fddf0102a30 2024-09-11T00:06:22.010 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:22.010+0000 7fdded600640 1 -- 172.21.15.12:0/1628956856 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7fdde003ba60 con 0x7fddf0102a30 2024-09-11T00:06:22.012 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:22.011+0000 7fdded600640 1 --2- 172.21.15.12:0/1628956856 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fddd40b5360 0x7fddd40b7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:06:22.012 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:22.011+0000 7fdded600640 1 -- 172.21.15.12:0/1628956856 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9208+0+0 (secure 0 0 0) 0x7fdde01066a0 con 0x7fddf0102a30 2024-09-11T00:06:22.013 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:22.012+0000 7fdde7400640 1 --2- 172.21.15.12:0/1628956856 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fddd40b5360 0x7fddd40b7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:06:22.013 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:22.012+0000 7fdde7400640 1 --2- 172.21.15.12:0/1628956856 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fddd40b5360 0x7fddd40b7820 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fddd8009fc0 tx=0x7fddd8005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:06:22.016 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:22.015+0000 7fdded600640 1 -- 172.21.15.12:0/1628956856 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7fdde00ccae0 con 0x7fddf0102a30 2024-09-11T00:06:22.247 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:22.246+0000 7fddf7000640 1 -- 172.21.15.12:0/1628956856 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump"} v 0) -- 0x7fddf0063e20 con 0x7fddf0102a30 2024-09-11T00:06:22.248 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:22.247+0000 7fdded600640 1 -- 172.21.15.12:0/1628956856 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 28 v28) ==== 76+0+3124 (secure 0 0 0) 0x7fdde00d1510 con 0x7fddf0102a30 2024-09-11T00:06:22.248 INFO:teuthology.orchestra.run.smithi012.stdout:e28 2024-09-11T00:06:22.249 INFO:teuthology.orchestra.run.smithi012.stdout:btime 2024-09-11T00:04:50:247323+0000 2024-09-11T00:06:22.249 INFO:teuthology.orchestra.run.smithi012.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-11T00:06:22.249 INFO:teuthology.orchestra.run.smithi012.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-11T00:06:22.249 INFO:teuthology.orchestra.run.smithi012.stdout:legacy client fscid: 1 2024-09-11T00:06:22.249 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-11T00:06:22.249 INFO:teuthology.orchestra.run.smithi012.stdout:Filesystem 'cephfs' (1) 2024-09-11T00:06:22.249 INFO:teuthology.orchestra.run.smithi012.stdout:fs_name cephfs 2024-09-11T00:06:22.249 INFO:teuthology.orchestra.run.smithi012.stdout:epoch 28 2024-09-11T00:06:22.249 INFO:teuthology.orchestra.run.smithi012.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-09-11T00:06:22.249 INFO:teuthology.orchestra.run.smithi012.stdout:created 2024-09-11T00:04:04.739881+0000 2024-09-11T00:06:22.249 INFO:teuthology.orchestra.run.smithi012.stdout:modified 2024-09-11T00:04:50.247301+0000 2024-09-11T00:06:22.250 INFO:teuthology.orchestra.run.smithi012.stdout:tableserver 0 2024-09-11T00:06:22.250 INFO:teuthology.orchestra.run.smithi012.stdout:root 0 2024-09-11T00:06:22.250 INFO:teuthology.orchestra.run.smithi012.stdout:session_timeout 300 2024-09-11T00:06:22.250 INFO:teuthology.orchestra.run.smithi012.stdout:session_autoclose 300 2024-09-11T00:06:22.250 INFO:teuthology.orchestra.run.smithi012.stdout:max_file_size 1099511627776 2024-09-11T00:06:22.250 INFO:teuthology.orchestra.run.smithi012.stdout:max_xattr_size 65536 2024-09-11T00:06:22.250 INFO:teuthology.orchestra.run.smithi012.stdout:required_client_features {} 2024-09-11T00:06:22.250 INFO:teuthology.orchestra.run.smithi012.stdout:last_failure 0 2024-09-11T00:06:22.250 INFO:teuthology.orchestra.run.smithi012.stdout:last_failure_osd_epoch 0 2024-09-11T00:06:22.250 INFO:teuthology.orchestra.run.smithi012.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-11T00:06:22.250 INFO:teuthology.orchestra.run.smithi012.stdout:max_mds 5 2024-09-11T00:06:22.250 INFO:teuthology.orchestra.run.smithi012.stdout:in 0,1,2,3,4 2024-09-11T00:06:22.250 INFO:teuthology.orchestra.run.smithi012.stdout:up {0=24674,1=24643,2=24638,3=24637,4=24614} 2024-09-11T00:06:22.251 INFO:teuthology.orchestra.run.smithi012.stdout:failed 2024-09-11T00:06:22.251 INFO:teuthology.orchestra.run.smithi012.stdout:damaged 2024-09-11T00:06:22.251 INFO:teuthology.orchestra.run.smithi012.stdout:stopped 2024-09-11T00:06:22.251 INFO:teuthology.orchestra.run.smithi012.stdout:data_pools [3,4] 2024-09-11T00:06:22.251 INFO:teuthology.orchestra.run.smithi012.stdout:metadata_pool 2 2024-09-11T00:06:22.251 INFO:teuthology.orchestra.run.smithi012.stdout:inline_data disabled 2024-09-11T00:06:22.251 INFO:teuthology.orchestra.run.smithi012.stdout:balancer 2024-09-11T00:06:22.251 INFO:teuthology.orchestra.run.smithi012.stdout:bal_rank_mask -1 2024-09-11T00:06:22.251 INFO:teuthology.orchestra.run.smithi012.stdout:standby_count_wanted 0 2024-09-11T00:06:22.251 INFO:teuthology.orchestra.run.smithi012.stdout:qdb_cluster leader: 24674 members: 24674,24643,24638,24637,24614 2024-09-11T00:06:22.252 INFO:teuthology.orchestra.run.smithi012.stdout:[mds.b{0:24674} state up:active seq 5 addr [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] compat {c=[1],r=[1],i=[1fff]}] 2024-09-11T00:06:22.252 INFO:teuthology.orchestra.run.smithi012.stdout:[mds.c{0:24613} state up:standby-replay seq 1 addr [v2:172.21.15.178:6836/526874763,v1:172.21.15.178:6837/526874763] compat {c=[1],r=[1],i=[1fff]}] 2024-09-11T00:06:22.252 INFO:teuthology.orchestra.run.smithi012.stdout:[mds.g{1:24643} state up:active seq 17 addr [v2:172.21.15.12:6838/3124093009,v1:172.21.15.12:6839/3124093009] compat {c=[1],r=[1],i=[1fff]}] 2024-09-11T00:06:22.252 INFO:teuthology.orchestra.run.smithi012.stdout:[mds.f{1:24607} state up:standby-replay seq 1 addr [v2:172.21.15.178:6834/4084868287,v1:172.21.15.178:6835/4084868287] compat {c=[1],r=[1],i=[1fff]}] 2024-09-11T00:06:22.252 INFO:teuthology.orchestra.run.smithi012.stdout:[mds.h{2:24638} state up:active seq 18 addr [v2:172.21.15.165:6836/6215088,v1:172.21.15.165:6837/6215088] compat {c=[1],r=[1],i=[1fff]}] 2024-09-11T00:06:22.252 INFO:teuthology.orchestra.run.smithi012.stdout:[mds.k{2:24581} state up:standby-replay seq 1 addr [v2:172.21.15.165:6832/3111980970,v1:172.21.15.165:6833/3111980970] compat {c=[1],r=[1],i=[1fff]}] 2024-09-11T00:06:22.252 INFO:teuthology.orchestra.run.smithi012.stdout:[mds.i{3:24637} state up:active seq 18 addr [v2:172.21.15.178:6838/70628097,v1:172.21.15.178:6839/70628097] compat {c=[1],r=[1],i=[1fff]}] 2024-09-11T00:06:22.252 INFO:teuthology.orchestra.run.smithi012.stdout:[mds.l{3:24562} state up:standby-replay seq 1 addr [v2:172.21.15.178:6832/1141636112,v1:172.21.15.178:6833/1141636112] compat {c=[1],r=[1],i=[1fff]}] 2024-09-11T00:06:22.252 INFO:teuthology.orchestra.run.smithi012.stdout:[mds.e{4:24614} state up:active seq 21 addr [v2:172.21.15.165:6834/2425395274,v1:172.21.15.165:6835/2425395274] compat {c=[1],r=[1],i=[1fff]}] 2024-09-11T00:06:22.252 INFO:teuthology.orchestra.run.smithi012.stdout:[mds.a{4:14631} state up:standby-replay seq 1 addr [v2:172.21.15.12:6840/3069919437,v1:172.21.15.12:6841/3069919437] compat {c=[1],r=[1],i=[1fff]}] 2024-09-11T00:06:22.252 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-11T00:06:22.253 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-11T00:06:22.253 INFO:teuthology.orchestra.run.smithi012.stdout:Standby daemons: 2024-09-11T00:06:22.253 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-11T00:06:22.253 INFO:teuthology.orchestra.run.smithi012.stdout:[mds.j{-1:14577} state up:standby seq 1 addr [v2:172.21.15.12:6834/447171591,v1:172.21.15.12:6835/447171591] compat {c=[1],r=[1],i=[1fff]}] 2024-09-11T00:06:22.253 INFO:teuthology.orchestra.run.smithi012.stdout:[mds.d{-1:14589} state up:standby seq 1 addr [v2:172.21.15.12:6836/901486040,v1:172.21.15.12:6837/901486040] compat {c=[1],r=[1],i=[1fff]}] 2024-09-11T00:06:22.253 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 28 2024-09-11T00:06:22.254 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:22.249+0000 7fddf7000640 1 -- 172.21.15.12:0/1628956856 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fddd40b5360 msgr2=0x7fddd40b7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:06:22.254 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:22.249+0000 7fddf7000640 1 --2- 172.21.15.12:0/1628956856 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fddd40b5360 0x7fddd40b7820 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fddd8009fc0 tx=0x7fddd8005f90 comp rx=0 tx=0).stop 2024-09-11T00:06:22.254 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:22.249+0000 7fddf7000640 1 -- 172.21.15.12:0/1628956856 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fddf0102a30 msgr2=0x7fddf0076cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:06:22.254 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:22.249+0000 7fddf7000640 1 --2- 172.21.15.12:0/1628956856 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fddf0102a30 0x7fddf0076cd0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fdde0005ab0 tx=0x7fdde00042f0 comp rx=0 tx=0).stop 2024-09-11T00:06:22.254 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:22.250+0000 7fddf7000640 1 -- 172.21.15.12:0/1628956856 shutdown_connections 2024-09-11T00:06:22.254 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:22.250+0000 7fddf7000640 1 --2- 172.21.15.12:0/1628956856 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fddd40b5360 0x7fddd40b7820 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:22.254 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:22.250+0000 7fddf7000640 1 --2- 172.21.15.12:0/1628956856 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fddf0104550 0x7fddf0075a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:22.254 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:22.250+0000 7fddf7000640 1 --2- 172.21.15.12:0/1628956856 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fddf0103b90 0x7fddf0075540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:22.254 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:22.250+0000 7fddf7000640 1 --2- 172.21.15.12:0/1628956856 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fddf0102a30 0x7fddf0076cd0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:22.254 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:22.250+0000 7fddf7000640 1 -- 172.21.15.12:0/1628956856 >> 172.21.15.12:0/1628956856 conn(0x7fddf00fe320 msgr2=0x7fddf0106f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:06:22.254 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:22.250+0000 7fddf7000640 1 -- 172.21.15.12:0/1628956856 shutdown_connections 2024-09-11T00:06:22.255 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:22.250+0000 7fddf7000640 1 -- 172.21.15.12:0/1628956856 wait complete. 2024-09-11T00:06:22.532 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:06:22 smithi165 ceph-mon[35356]: mgrmap e18: x(active, since 10m), standbys: z, y 2024-09-11T00:06:22.563 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:06:22 smithi012 ceph-mon[30333]: mgrmap e18: x(active, since 10m), standbys: z, y 2024-09-11T00:06:22.580 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:06:22 smithi178 ceph-mon[35919]: mgrmap e18: x(active, since 10m), standbys: z, y 2024-09-11T00:06:22.971 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ff3ad74a-6fcf-11ef-bcea-c7b262605968 -- bash -c 'ceph osd dump' 2024-09-11T00:06:23.350 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:06:23.486 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:06:23 smithi012 ceph-mon[30333]: pgmap v377: 129 pgs: 1 peering, 128 active+clean; 647 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 895 B/s wr, 13 op/s; 14 B/s, 0 objects/s recovering 2024-09-11T00:06:23.486 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:06:23 smithi012 ceph-mon[30333]: from='client.? 172.21.15.12:0/1628956856' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2024-09-11T00:06:23.532 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:06:23 smithi165 ceph-mon[35356]: pgmap v377: 129 pgs: 1 peering, 128 active+clean; 647 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 895 B/s wr, 13 op/s; 14 B/s, 0 objects/s recovering 2024-09-11T00:06:23.532 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:06:23 smithi165 ceph-mon[35356]: from='client.? 172.21.15.12:0/1628956856' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2024-09-11T00:06:23.580 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:06:23 smithi178 ceph-mon[35919]: pgmap v377: 129 pgs: 1 peering, 128 active+clean; 647 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 895 B/s wr, 13 op/s; 14 B/s, 0 objects/s recovering 2024-09-11T00:06:23.581 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:06:23 smithi178 ceph-mon[35919]: from='client.? 172.21.15.12:0/1628956856' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2024-09-11T00:06:23.862 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:23.860+0000 7f8d34200640 1 -- 172.21.15.12:0/3748907702 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8d2c104c40 msgr2=0x7f8d2c105040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:06:23.862 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:23.860+0000 7f8d34200640 1 --2- 172.21.15.12:0/3748907702 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8d2c104c40 0x7f8d2c105040 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f8d20009930 tx=0x7f8d2002a420 comp rx=0 tx=0).stop 2024-09-11T00:06:23.862 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:23.860+0000 7f8d34200640 1 -- 172.21.15.12:0/3748907702 shutdown_connections 2024-09-11T00:06:23.862 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:23.860+0000 7f8d34200640 1 --2- 172.21.15.12:0/3748907702 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8d2c106760 0x7f8d2c10d0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:23.862 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:23.860+0000 7f8d34200640 1 --2- 172.21.15.12:0/3748907702 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8d2c105da0 0x7f8d2c106220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:23.863 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:23.860+0000 7f8d34200640 1 --2- 172.21.15.12:0/3748907702 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8d2c104c40 0x7f8d2c105040 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:23.863 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:23.860+0000 7f8d34200640 1 -- 172.21.15.12:0/3748907702 >> 172.21.15.12:0/3748907702 conn(0x7f8d2c100510 msgr2=0x7f8d2c102950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:06:23.863 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:23.860+0000 7f8d34200640 1 -- 172.21.15.12:0/3748907702 shutdown_connections 2024-09-11T00:06:23.863 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:23.861+0000 7f8d34200640 1 -- 172.21.15.12:0/3748907702 wait complete. 2024-09-11T00:06:23.864 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:23.862+0000 7f8d34200640 1 Processor -- start 2024-09-11T00:06:23.864 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:23.862+0000 7f8d34200640 1 -- start start 2024-09-11T00:06:23.864 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:23.862+0000 7f8d34200640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8d2c104c40 0x7f8d2c19b040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:06:23.864 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:23.862+0000 7f8d34200640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8d2c105da0 0x7f8d2c19b690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:06:23.864 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:23.862+0000 7f8d34200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8d2c106760 0x7f8d2c19c2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:06:23.864 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:23.862+0000 7f8d34200640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f8d2c1108c0 con 0x7f8d2c104c40 2024-09-11T00:06:23.864 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:23.862+0000 7f8d34200640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f8d2c110740 con 0x7f8d2c106760 2024-09-11T00:06:23.864 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:23.862+0000 7f8d34200640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f8d2c110a40 con 0x7f8d2c105da0 2024-09-11T00:06:23.864 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:23.863+0000 7f8d31800640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8d2c104c40 0x7f8d2c19b040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:06:23.864 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:23.863+0000 7f8d31800640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8d2c104c40 0x7f8d2c19b040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:48700/0 (socket says 172.21.15.12:48700) 2024-09-11T00:06:23.865 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:23.863+0000 7f8d31800640 1 -- 172.21.15.12:0/3967055416 learned_addr learned my addr 172.21.15.12:0/3967055416 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:06:23.865 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:23.863+0000 7f8d30e00640 1 --2- 172.21.15.12:0/3967055416 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8d2c105da0 0x7f8d2c19b690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:06:23.865 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:23.863+0000 7f8d32200640 1 --2- 172.21.15.12:0/3967055416 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8d2c106760 0x7f8d2c19c2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:06:23.865 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:23.863+0000 7f8d31800640 1 -- 172.21.15.12:0/3967055416 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8d2c105da0 msgr2=0x7f8d2c19b690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:06:23.865 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:23.863+0000 7f8d31800640 1 --2- 172.21.15.12:0/3967055416 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8d2c105da0 0x7f8d2c19b690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:23.866 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:23.863+0000 7f8d31800640 1 -- 172.21.15.12:0/3967055416 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8d2c106760 msgr2=0x7f8d2c19c2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:06:23.866 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:23.863+0000 7f8d31800640 1 --2- 172.21.15.12:0/3967055416 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8d2c106760 0x7f8d2c19c2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:23.866 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:23.863+0000 7f8d31800640 1 -- 172.21.15.12:0/3967055416 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d2c19cb70 con 0x7f8d2c104c40 2024-09-11T00:06:23.866 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:23.864+0000 7f8d31800640 1 --2- 172.21.15.12:0/3967055416 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8d2c104c40 0x7f8d2c19b040 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f8d20005ab0 tx=0x7f8d200042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:06:23.866 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:23.864+0000 7f8d2a000640 1 -- 172.21.15.12:0/3967055416 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d2003f030 con 0x7f8d2c104c40 2024-09-11T00:06:23.866 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:23.864+0000 7f8d2a000640 1 -- 172.21.15.12:0/3967055416 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8d20032e40 con 0x7f8d2c104c40 2024-09-11T00:06:23.866 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:23.864+0000 7f8d2a000640 1 -- 172.21.15.12:0/3967055416 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d20035070 con 0x7f8d2c104c40 2024-09-11T00:06:23.866 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:23.864+0000 7f8d34200640 1 -- 172.21.15.12:0/3967055416 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8d2c19ce00 con 0x7f8d2c104c40 2024-09-11T00:06:23.866 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:23.864+0000 7f8d34200640 1 -- 172.21.15.12:0/3967055416 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8d2c1a3370 con 0x7f8d2c104c40 2024-09-11T00:06:23.867 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:23.865+0000 7f8d34200640 1 -- 172.21.15.12:0/3967055416 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8d2c10bfc0 con 0x7f8d2c104c40 2024-09-11T00:06:23.868 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:23.867+0000 7f8d2a000640 1 -- 172.21.15.12:0/3967055416 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7f8d2003ba60 con 0x7f8d2c104c40 2024-09-11T00:06:23.869 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:23.869+0000 7f8d2a000640 1 --2- 172.21.15.12:0/3967055416 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f8d140b52e0 0x7f8d140b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:06:23.869 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:23.869+0000 7f8d2a000640 1 -- 172.21.15.12:0/3967055416 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9208+0+0 (secure 0 0 0) 0x7f8d20106b40 con 0x7f8d2c104c40 2024-09-11T00:06:23.870 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:23.869+0000 7f8d30e00640 1 --2- 172.21.15.12:0/3967055416 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f8d140b52e0 0x7f8d140b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:06:23.870 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:23.870+0000 7f8d30e00640 1 --2- 172.21.15.12:0/3967055416 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f8d140b52e0 0x7f8d140b77a0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f8d18009bd0 tx=0x7f8d18005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:06:23.872 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:23.871+0000 7f8d2a000640 1 -- 172.21.15.12:0/3967055416 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f8d200ccf80 con 0x7f8d2c104c40 2024-09-11T00:06:24.069 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:24.067+0000 7f8d34200640 1 -- 172.21.15.12:0/3967055416 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd dump"} v 0) -- 0x7f8d2c02e5c0 con 0x7f8d2c104c40 2024-09-11T00:06:24.070 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:24.068+0000 7f8d2a000640 1 -- 172.21.15.12:0/3967055416 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump"}]=0 v85) ==== 56+0+5707 (secure 0 0 0) 0x7f8d200d19b0 con 0x7f8d2c104c40 2024-09-11T00:06:24.070 INFO:teuthology.orchestra.run.smithi012.stdout:epoch 85 2024-09-11T00:06:24.071 INFO:teuthology.orchestra.run.smithi012.stdout:fsid ff3ad74a-6fcf-11ef-bcea-c7b262605968 2024-09-11T00:06:24.071 INFO:teuthology.orchestra.run.smithi012.stdout:created 2024-09-10T23:55:16.317643+0000 2024-09-11T00:06:24.071 INFO:teuthology.orchestra.run.smithi012.stdout:modified 2024-09-11T00:06:20.198537+0000 2024-09-11T00:06:24.071 INFO:teuthology.orchestra.run.smithi012.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-09-11T00:06:24.071 INFO:teuthology.orchestra.run.smithi012.stdout:crush_version 27 2024-09-11T00:06:24.071 INFO:teuthology.orchestra.run.smithi012.stdout:full_ratio 0.95 2024-09-11T00:06:24.071 INFO:teuthology.orchestra.run.smithi012.stdout:backfillfull_ratio 0.9 2024-09-11T00:06:24.071 INFO:teuthology.orchestra.run.smithi012.stdout:nearfull_ratio 0.85 2024-09-11T00:06:24.071 INFO:teuthology.orchestra.run.smithi012.stdout:require_min_compat_client luminous 2024-09-11T00:06:24.071 INFO:teuthology.orchestra.run.smithi012.stdout:min_compat_client luminous 2024-09-11T00:06:24.071 INFO:teuthology.orchestra.run.smithi012.stdout:require_osd_release squid 2024-09-11T00:06:24.071 INFO:teuthology.orchestra.run.smithi012.stdout:stretch_mode_enabled false 2024-09-11T00:06:24.072 INFO:teuthology.orchestra.run.smithi012.stdout:pool 1 '.mgr' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 22 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 12.00 2024-09-11T00:06:24.072 INFO:teuthology.orchestra.run.smithi012.stdout:pool 2 'cephfs_metadata' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 74 flags hashpspool stripe_width 0 pg_autoscale_bias 4 pg_num_min 16 recovery_priority 5 application cephfs read_balance_score 1.88 2024-09-11T00:06:24.072 INFO:teuthology.orchestra.run.smithi012.stdout:pool 3 'cephfs_data' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 64 pgp_num 64 autoscale_mode off last_change 74 flags hashpspool stripe_width 0 pg_num_min 64 target_size_ratio 0.05 application cephfs read_balance_score 1.50 2024-09-11T00:06:24.072 INFO:teuthology.orchestra.run.smithi012.stdout:pool 4 'cephfs_data_ec' erasure profile cephfs_data_ec size 4 min_size 3 crush_rule 1 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 81 flags hashpspool,ec_overwrites stripe_width 8192 pg_num_min 64 target_size_ratio 0.9 application cephfs 2024-09-11T00:06:24.072 INFO:teuthology.orchestra.run.smithi012.stdout:max_osd 12 2024-09-11T00:06:24.072 INFO:teuthology.orchestra.run.smithi012.stdout:osd.0 up in weight 1 up_from 8 up_thru 84 down_at 0 last_clean_interval [0,0) [v2:172.21.15.12:6802/1237101991,v1:172.21.15.12:6803/1237101991] [v2:172.21.15.12:6804/1237101991,v1:172.21.15.12:6805/1237101991] exists,up 6382c200-a912-42a1-958b-79191b21249b 2024-09-11T00:06:24.072 INFO:teuthology.orchestra.run.smithi012.stdout:osd.1 up in weight 1 up_from 13 up_thru 77 down_at 0 last_clean_interval [0,0) [v2:172.21.15.165:6800/545848456,v1:172.21.15.165:6801/545848456] [v2:172.21.15.165:6802/545848456,v1:172.21.15.165:6803/545848456] exists,up fe0e1d0a-bdc5-4859-aa84-9038059a45f6 2024-09-11T00:06:24.072 INFO:teuthology.orchestra.run.smithi012.stdout:osd.2 up in weight 1 up_from 18 up_thru 77 down_at 0 last_clean_interval [0,0) [v2:172.21.15.178:6800/4131162695,v1:172.21.15.178:6801/4131162695] [v2:172.21.15.178:6802/4131162695,v1:172.21.15.178:6803/4131162695] exists,up 94f640dc-5f78-4e2c-87d9-03e96a6e7ba0 2024-09-11T00:06:24.072 INFO:teuthology.orchestra.run.smithi012.stdout:osd.3 up in weight 1 up_from 25 up_thru 77 down_at 0 last_clean_interval [0,0) [v2:172.21.15.12:6810/452464245,v1:172.21.15.12:6811/452464245] [v2:172.21.15.12:6812/452464245,v1:172.21.15.12:6813/452464245] exists,up af2f64d6-c9be-44a9-b041-8d599a7c080d 2024-09-11T00:06:24.072 INFO:teuthology.orchestra.run.smithi012.stdout:osd.4 up in weight 1 up_from 31 up_thru 84 down_at 0 last_clean_interval [0,0) [v2:172.21.15.165:6808/1899879280,v1:172.21.15.165:6809/1899879280] [v2:172.21.15.165:6810/1899879280,v1:172.21.15.165:6811/1899879280] exists,up ad76ac70-20c0-44ae-9866-5f9cfce2334c 2024-09-11T00:06:24.072 INFO:teuthology.orchestra.run.smithi012.stdout:osd.5 up in weight 1 up_from 36 up_thru 77 down_at 0 last_clean_interval [0,0) [v2:172.21.15.178:6808/3042228555,v1:172.21.15.178:6809/3042228555] [v2:172.21.15.178:6810/3042228555,v1:172.21.15.178:6811/3042228555] exists,up aafda2cc-62fe-4fcb-9a96-8732e0b501d5 2024-09-11T00:06:24.072 INFO:teuthology.orchestra.run.smithi012.stdout:osd.6 up in weight 1 up_from 42 up_thru 82 down_at 0 last_clean_interval [0,0) [v2:172.21.15.12:6818/3868322467,v1:172.21.15.12:6819/3868322467] [v2:172.21.15.12:6820/3868322467,v1:172.21.15.12:6821/3868322467] exists,up 375eca52-efe3-42a5-bbb1-d7d392d71f15 2024-09-11T00:06:24.073 INFO:teuthology.orchestra.run.smithi012.stdout:osd.7 up in weight 1 up_from 47 up_thru 84 down_at 0 last_clean_interval [0,0) [v2:172.21.15.165:6816/1143773735,v1:172.21.15.165:6817/1143773735] [v2:172.21.15.165:6818/1143773735,v1:172.21.15.165:6819/1143773735] exists,up 0fd619f5-44c3-4cf7-ae7a-388dd798de36 2024-09-11T00:06:24.073 INFO:teuthology.orchestra.run.smithi012.stdout:osd.8 up in weight 1 up_from 52 up_thru 84 down_at 0 last_clean_interval [0,0) [v2:172.21.15.178:6816/2891445058,v1:172.21.15.178:6817/2891445058] [v2:172.21.15.178:6818/2891445058,v1:172.21.15.178:6819/2891445058] exists,up b03d14ce-8427-462a-9bcc-27683c549f53 2024-09-11T00:06:24.073 INFO:teuthology.orchestra.run.smithi012.stdout:osd.9 up in weight 1 up_from 58 up_thru 82 down_at 0 last_clean_interval [0,0) [v2:172.21.15.12:6826/3359634903,v1:172.21.15.12:6827/3359634903] [v2:172.21.15.12:6828/3359634903,v1:172.21.15.12:6829/3359634903] exists,up 81a6cf69-2a24-4a6c-9adf-6b7df38cfacb 2024-09-11T00:06:24.073 INFO:teuthology.orchestra.run.smithi012.stdout:osd.10 up in weight 1 up_from 63 up_thru 72 down_at 0 last_clean_interval [0,0) [v2:172.21.15.165:6824/3642325778,v1:172.21.15.165:6825/3642325778] [v2:172.21.15.165:6826/3642325778,v1:172.21.15.165:6827/3642325778] exists,up c1ad1f0a-33f4-4c32-91f6-4d030a29f1dd 2024-09-11T00:06:24.073 INFO:teuthology.orchestra.run.smithi012.stdout:osd.11 up in weight 1 up_from 68 up_thru 82 down_at 0 last_clean_interval [0,0) [v2:172.21.15.178:6824/2937059756,v1:172.21.15.178:6825/2937059756] [v2:172.21.15.178:6826/2937059756,v1:172.21.15.178:6827/2937059756] exists,up 8413c9b0-680b-4d02-aeab-b3b3ff16c3d6 2024-09-11T00:06:24.073 INFO:teuthology.orchestra.run.smithi012.stdout:pg_upmap_items 2.0 [8,4] 2024-09-11T00:06:24.073 INFO:teuthology.orchestra.run.smithi012.stdout:pg_upmap_items 2.a [2,11] 2024-09-11T00:06:24.073 INFO:teuthology.orchestra.run.smithi012.stdout:pg_upmap_items 2.e [2,4] 2024-09-11T00:06:24.073 INFO:teuthology.orchestra.run.smithi012.stdout:pg_upmap_items 2.10 [8,11] 2024-09-11T00:06:24.073 INFO:teuthology.orchestra.run.smithi012.stdout:pg_upmap_items 3.e [5,9] 2024-09-11T00:06:24.073 INFO:teuthology.orchestra.run.smithi012.stdout:pg_upmap_items 3.1e [3,9] 2024-09-11T00:06:24.074 INFO:teuthology.orchestra.run.smithi012.stdout:pg_upmap_items 3.23 [4,7] 2024-09-11T00:06:24.074 INFO:teuthology.orchestra.run.smithi012.stdout:pg_upmap_items 3.31 [4,8] 2024-09-11T00:06:24.074 INFO:teuthology.orchestra.run.smithi012.stdout:pg_upmap_items 3.34 [4,9] 2024-09-11T00:06:24.074 INFO:teuthology.orchestra.run.smithi012.stdout:pg_upmap_items 3.39 [5,6] 2024-09-11T00:06:24.074 INFO:teuthology.orchestra.run.smithi012.stdout:pg_upmap_items 4.5 [9,10] 2024-09-11T00:06:24.074 INFO:teuthology.orchestra.run.smithi012.stdout:pg_upmap_items 4.6 [5,8] 2024-09-11T00:06:24.074 INFO:teuthology.orchestra.run.smithi012.stdout:pg_upmap_items 4.b [9,8] 2024-09-11T00:06:24.074 INFO:teuthology.orchestra.run.smithi012.stdout:pg_upmap_items 4.13 [9,8] 2024-09-11T00:06:24.074 INFO:teuthology.orchestra.run.smithi012.stdout:blocklist 172.21.15.12:6801/4177251511 expires 2024-09-11T23:56:17.999820+0000 2024-09-11T00:06:24.074 INFO:teuthology.orchestra.run.smithi012.stdout:blocklist 172.21.15.12:6800/4177251511 expires 2024-09-11T23:56:17.999820+0000 2024-09-11T00:06:24.074 INFO:teuthology.orchestra.run.smithi012.stdout:blocklist 172.21.15.12:0/3273692817 expires 2024-09-11T23:56:17.999820+0000 2024-09-11T00:06:24.074 INFO:teuthology.orchestra.run.smithi012.stdout:blocklist 172.21.15.12:0/4017778158 expires 2024-09-11T23:56:17.999820+0000 2024-09-11T00:06:24.074 INFO:teuthology.orchestra.run.smithi012.stdout:blocklist 172.21.15.12:0/2499921051 expires 2024-09-11T23:55:50.302721+0000 2024-09-11T00:06:24.075 INFO:teuthology.orchestra.run.smithi012.stdout:blocklist 172.21.15.12:6801/760140910 expires 2024-09-11T23:55:50.302721+0000 2024-09-11T00:06:24.075 INFO:teuthology.orchestra.run.smithi012.stdout:blocklist 172.21.15.12:6800/760140910 expires 2024-09-11T23:55:50.302721+0000 2024-09-11T00:06:24.075 INFO:teuthology.orchestra.run.smithi012.stdout:blocklist 172.21.15.12:0/2810703302 expires 2024-09-11T23:56:17.999820+0000 2024-09-11T00:06:24.075 INFO:teuthology.orchestra.run.smithi012.stdout:blocklist 172.21.15.12:0/2325436065 expires 2024-09-11T23:55:50.302721+0000 2024-09-11T00:06:24.075 INFO:teuthology.orchestra.run.smithi012.stdout:blocklist 172.21.15.12:0/3137053479 expires 2024-09-11T23:55:50.302721+0000 2024-09-11T00:06:24.076 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:24.071+0000 7f8d34200640 1 -- 172.21.15.12:0/3967055416 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f8d140b52e0 msgr2=0x7f8d140b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:06:24.076 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:24.071+0000 7f8d34200640 1 --2- 172.21.15.12:0/3967055416 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f8d140b52e0 0x7f8d140b77a0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f8d18009bd0 tx=0x7f8d18005f90 comp rx=0 tx=0).stop 2024-09-11T00:06:24.076 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:24.071+0000 7f8d34200640 1 -- 172.21.15.12:0/3967055416 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8d2c104c40 msgr2=0x7f8d2c19b040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:06:24.076 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:24.071+0000 7f8d34200640 1 --2- 172.21.15.12:0/3967055416 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8d2c104c40 0x7f8d2c19b040 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f8d20005ab0 tx=0x7f8d200042f0 comp rx=0 tx=0).stop 2024-09-11T00:06:24.076 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:24.071+0000 7f8d34200640 1 -- 172.21.15.12:0/3967055416 shutdown_connections 2024-09-11T00:06:24.076 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:24.071+0000 7f8d34200640 1 --2- 172.21.15.12:0/3967055416 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f8d140b52e0 0x7f8d140b77a0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:24.076 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:24.071+0000 7f8d34200640 1 --2- 172.21.15.12:0/3967055416 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8d2c106760 0x7f8d2c19c2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:24.076 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:24.071+0000 7f8d34200640 1 --2- 172.21.15.12:0/3967055416 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8d2c105da0 0x7f8d2c19b690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:24.076 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:24.071+0000 7f8d34200640 1 --2- 172.21.15.12:0/3967055416 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8d2c104c40 0x7f8d2c19b040 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:24.077 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:24.071+0000 7f8d34200640 1 -- 172.21.15.12:0/3967055416 >> 172.21.15.12:0/3967055416 conn(0x7f8d2c100510 msgr2=0x7f8d2c109190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:06:24.077 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:24.071+0000 7f8d34200640 1 -- 172.21.15.12:0/3967055416 shutdown_connections 2024-09-11T00:06:24.077 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:24.071+0000 7f8d34200640 1 -- 172.21.15.12:0/3967055416 wait complete. 2024-09-11T00:06:24.422 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:06:24 smithi012 ceph-mon[30333]: from='client.? 172.21.15.12:0/3967055416' entity='client.admin' cmd={"prefix": "osd dump"} : dispatch 2024-09-11T00:06:24.532 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:06:24 smithi165 ceph-mon[35356]: from='client.? 172.21.15.12:0/3967055416' entity='client.admin' cmd={"prefix": "osd dump"} : dispatch 2024-09-11T00:06:24.580 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:06:24 smithi178 ceph-mon[35919]: from='client.? 172.21.15.12:0/3967055416' entity='client.admin' cmd={"prefix": "osd dump"} : dispatch 2024-09-11T00:06:24.773 INFO:teuthology.run_tasks:Running task fs.ready... 2024-09-11T00:06:24.790 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid ff3ad74a-6fcf-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T00:06:25.171 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:06:25.490 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:06:25 smithi012 ceph-mon[30333]: pgmap v378: 129 pgs: 1 peering, 128 active+clean; 647 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 895 B/s wr, 7 op/s; 14 B/s, 0 objects/s recovering 2024-09-11T00:06:25.532 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:06:25 smithi165 ceph-mon[35356]: pgmap v378: 129 pgs: 1 peering, 128 active+clean; 647 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 895 B/s wr, 7 op/s; 14 B/s, 0 objects/s recovering 2024-09-11T00:06:25.581 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:06:25 smithi178 ceph-mon[35919]: pgmap v378: 129 pgs: 1 peering, 128 active+clean; 647 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 895 B/s wr, 7 op/s; 14 B/s, 0 objects/s recovering 2024-09-11T00:06:25.710 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:25.707+0000 7f4fe9800640 1 -- 172.21.15.12:0/1393250372 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4fe4102da0 msgr2=0x7f4fe4103180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:06:25.710 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:25.707+0000 7f4fe9800640 1 --2- 172.21.15.12:0/1393250372 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4fe4102da0 0x7f4fe4103180 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f4fd4009900 tx=0x7f4fd402a3f0 comp rx=0 tx=0).stop 2024-09-11T00:06:25.710 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:25.707+0000 7f4fe9800640 1 -- 172.21.15.12:0/1393250372 shutdown_connections 2024-09-11T00:06:25.710 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:25.707+0000 7f4fe9800640 1 --2- 172.21.15.12:0/1393250372 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4fe4109130 0x7f4fe410f930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:25.710 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:25.707+0000 7f4fe9800640 1 --2- 172.21.15.12:0/1393250372 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4fe41087b0 0x7f4fe4108bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:25.710 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:25.707+0000 7f4fe9800640 1 --2- 172.21.15.12:0/1393250372 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4fe4102da0 0x7f4fe4103180 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:25.710 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:25.707+0000 7f4fe9800640 1 -- 172.21.15.12:0/1393250372 >> 172.21.15.12:0/1393250372 conn(0x7f4fe40fe810 msgr2=0x7f4fe4100c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:06:25.710 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:25.707+0000 7f4fe9800640 1 -- 172.21.15.12:0/1393250372 shutdown_connections 2024-09-11T00:06:25.710 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:25.707+0000 7f4fe9800640 1 -- 172.21.15.12:0/1393250372 wait complete. 2024-09-11T00:06:25.712 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:25.708+0000 7f4fe9800640 1 Processor -- start 2024-09-11T00:06:25.712 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:25.709+0000 7f4fe9800640 1 -- start start 2024-09-11T00:06:25.712 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:25.709+0000 7f4fe9800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4fe4102da0 0x7f4fe410ec60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:06:25.712 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:25.709+0000 7f4fe9800640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4fe41087b0 0x7f4fe410f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:06:25.712 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:25.709+0000 7f4fe9800640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4fe4109130 0x7f4fe41a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:06:25.712 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:25.709+0000 7f4fe9800640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f4fe4112df0 con 0x7f4fe4109130 2024-09-11T00:06:25.712 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:25.709+0000 7f4fe9800640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f4fe4112c70 con 0x7f4fe4102da0 2024-09-11T00:06:25.712 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:25.709+0000 7f4fe9800640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f4fe4112f70 con 0x7f4fe41087b0 2024-09-11T00:06:25.712 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:25.709+0000 7f4fe2a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4fe4102da0 0x7f4fe410ec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:06:25.712 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:25.709+0000 7f4fe2a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4fe4102da0 0x7f4fe410ec60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.12:43322/0 (socket says 172.21.15.12:43322) 2024-09-11T00:06:25.712 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:25.709+0000 7f4fe2a00640 1 -- 172.21.15.12:0/1454612391 learned_addr learned my addr 172.21.15.12:0/1454612391 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:06:25.713 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:25.709+0000 7f4fe2000640 1 --2- 172.21.15.12:0/1454612391 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4fe41087b0 0x7f4fe410f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:06:25.713 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:25.709+0000 7f4fe3400640 1 --2- 172.21.15.12:0/1454612391 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4fe4109130 0x7f4fe41a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:06:25.713 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:25.710+0000 7f4fe2a00640 1 -- 172.21.15.12:0/1454612391 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4fe41087b0 msgr2=0x7f4fe410f1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:06:25.713 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:25.710+0000 7f4fe2a00640 1 --2- 172.21.15.12:0/1454612391 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4fe41087b0 0x7f4fe410f1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:25.713 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:25.710+0000 7f4fe2a00640 1 -- 172.21.15.12:0/1454612391 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4fe4109130 msgr2=0x7f4fe41a51c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:06:25.713 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:25.710+0000 7f4fe2a00640 1 --2- 172.21.15.12:0/1454612391 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4fe4109130 0x7f4fe41a51c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:25.713 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:25.710+0000 7f4fe2a00640 1 -- 172.21.15.12:0/1454612391 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4fe41a5990 con 0x7f4fe4102da0 2024-09-11T00:06:25.713 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:25.710+0000 7f4fe2a00640 1 --2- 172.21.15.12:0/1454612391 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4fe4102da0 0x7f4fe410ec60 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f4fd4005940 tx=0x7f4fd40040c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:06:25.714 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:25.710+0000 7f4fdb400640 1 -- 172.21.15.12:0/1454612391 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fd403f030 con 0x7f4fe4102da0 2024-09-11T00:06:25.714 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:25.711+0000 7f4fe9800640 1 -- 172.21.15.12:0/1454612391 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4fe41a5c20 con 0x7f4fe4102da0 2024-09-11T00:06:25.714 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:25.711+0000 7f4fdb400640 1 -- 172.21.15.12:0/1454612391 <== mon.2 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4fd40044c0 con 0x7f4fe4102da0 2024-09-11T00:06:25.714 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:25.711+0000 7f4fdb400640 1 -- 172.21.15.12:0/1454612391 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fd4035070 con 0x7f4fe4102da0 2024-09-11T00:06:25.714 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:25.711+0000 7f4fe9800640 1 -- 172.21.15.12:0/1454612391 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4fe41ac340 con 0x7f4fe4102da0 2024-09-11T00:06:25.714 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:25.712+0000 7f4fe9800640 1 -- 172.21.15.12:0/1454612391 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4fe4103180 con 0x7f4fe4102da0 2024-09-11T00:06:25.716 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:25.714+0000 7f4fdb400640 1 -- 172.21.15.12:0/1454612391 <== mon.2 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7f4fd403b9d0 con 0x7f4fe4102da0 2024-09-11T00:06:25.717 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:25.716+0000 7f4fdb400640 1 --2- 172.21.15.12:0/1454612391 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f4fc80b5360 0x7f4fc80b7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:06:25.718 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:25.716+0000 7f4fdb400640 1 -- 172.21.15.12:0/1454612391 <== mon.2 v2:172.21.15.165:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9208+0+0 (secure 0 0 0) 0x7f4fd4106830 con 0x7f4fe4102da0 2024-09-11T00:06:25.718 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:25.716+0000 7f4fe2000640 1 --2- 172.21.15.12:0/1454612391 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f4fc80b5360 0x7f4fc80b7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:06:25.718 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:25.716+0000 7f4fe2000640 1 --2- 172.21.15.12:0/1454612391 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f4fc80b5360 0x7f4fc80b7820 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f4fe4108240 tx=0x7f4fd0005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:06:25.719 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:25.717+0000 7f4fdb400640 1 -- 172.21.15.12:0/1454612391 <== mon.2 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f4fd40cd2c0 con 0x7f4fe4102da0 2024-09-11T00:06:25.958 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:25.956+0000 7f4fe9800640 1 -- 172.21.15.12:0/1454612391 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4fe41a0a70 con 0x7f4fe4102da0 2024-09-11T00:06:25.959 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:25.957+0000 7f4fdb400640 1 -- 172.21.15.12:0/1454612391 <== mon.2 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11965 (secure 0 0 0) 0x7f4fd40ccbc0 con 0x7f4fe4102da0 2024-09-11T00:06:25.960 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-11T00:06:25.960 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":28,"btime":"2024-09-11T00:04:50:247323+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14577,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6835/447171591","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":447171591},{"type":"v1","addr":"172.21.15.12:6835","nonce":447171591}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14589,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/901486040","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":901486040},{"type":"v1","addr":"172.21.15.12:6837","nonce":901486040}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T00:04:04.739881+0000","modified":"2024-09-11T00:04:50.247301+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24674,"mds_1":24643,"mds_2":24638,"mds_3":24637,"mds_4":24614},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14631":{"gid":14631,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.12:6841/3069919437","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6840","nonce":3069919437},{"type":"v1","addr":"172.21.15.12:6841","nonce":3069919437}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24562":{"gid":24562,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6833/1141636112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":1141636112},{"type":"v1","addr":"172.21.15.178:6833","nonce":1141636112}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24581":{"gid":24581,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6833/3111980970","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6832","nonce":3111980970},{"type":"v1","addr":"172.21.15.165:6833","nonce":3111980970}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24607":{"gid":24607,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6835/4084868287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":4084868287},{"type":"v1","addr":"172.21.15.178:6835","nonce":4084868287}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24613":{"gid":24613,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6837/526874763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":526874763},{"type":"v1","addr":"172.21.15.178:6837","nonce":526874763}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24614":{"gid":24614,"name":"e","rank":4,"incarnation":26,"state":"up:active","state_seq":21,"addr":"172.21.15.165:6835/2425395274","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":2425395274},{"type":"v1","addr":"172.21.15.165:6835","nonce":2425395274}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24637":{"gid":24637,"name":"i","rank":3,"incarnation":24,"state":"up:active","state_seq":18,"addr":"172.21.15.178:6839/70628097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6838","nonce":70628097},{"type":"v1","addr":"172.21.15.178:6839","nonce":70628097}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24638":{"gid":24638,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":18,"addr":"172.21.15.165:6837/6215088","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6836","nonce":6215088},{"type":"v1","addr":"172.21.15.165:6837","nonce":6215088}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24643":{"gid":24643,"name":"g","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.12:6839/3124093009","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6838","nonce":3124093009},{"type":"v1","addr":"172.21.15.12:6839","nonce":3124093009}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24674":{"gid":24674,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.165:6839/501312984","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6838","nonce":501312984},{"type":"v1","addr":"172.21.15.165:6839","nonce":501312984}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24674,"qdb_cluster":[24674,24643,24638,24637,24614]},"id":1}]} 2024-09-11T00:06:25.960 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 28 2024-09-11T00:06:25.961 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:25.960+0000 7f4fe9800640 1 -- 172.21.15.12:0/1454612391 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f4fc80b5360 msgr2=0x7f4fc80b7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:06:25.962 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:25.960+0000 7f4fe9800640 1 --2- 172.21.15.12:0/1454612391 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f4fc80b5360 0x7f4fc80b7820 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f4fe4108240 tx=0x7f4fd0005f90 comp rx=0 tx=0).stop 2024-09-11T00:06:25.962 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:25.960+0000 7f4fe9800640 1 -- 172.21.15.12:0/1454612391 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4fe4102da0 msgr2=0x7f4fe410ec60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:06:25.962 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:25.960+0000 7f4fe9800640 1 --2- 172.21.15.12:0/1454612391 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4fe4102da0 0x7f4fe410ec60 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f4fd4005940 tx=0x7f4fd40040c0 comp rx=0 tx=0).stop 2024-09-11T00:06:25.962 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:25.960+0000 7f4fe9800640 1 -- 172.21.15.12:0/1454612391 shutdown_connections 2024-09-11T00:06:25.962 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:25.960+0000 7f4fe9800640 1 --2- 172.21.15.12:0/1454612391 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f4fc80b5360 0x7f4fc80b7820 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:25.962 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:25.960+0000 7f4fe9800640 1 --2- 172.21.15.12:0/1454612391 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4fe4109130 0x7f4fe41a51c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:25.962 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:25.960+0000 7f4fe9800640 1 --2- 172.21.15.12:0/1454612391 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4fe41087b0 0x7f4fe410f1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:25.962 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:25.960+0000 7f4fe9800640 1 --2- 172.21.15.12:0/1454612391 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4fe4102da0 0x7f4fe410ec60 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:25.963 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:25.960+0000 7f4fe9800640 1 -- 172.21.15.12:0/1454612391 >> 172.21.15.12:0/1454612391 conn(0x7f4fe40fe810 msgr2=0x7f4fe4104d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:06:25.963 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:25.960+0000 7f4fe9800640 1 -- 172.21.15.12:0/1454612391 shutdown_connections 2024-09-11T00:06:25.963 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:25.960+0000 7f4fe9800640 1 -- 172.21.15.12:0/1454612391 wait complete. 2024-09-11T00:06:26.532 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:06:26 smithi165 ceph-mon[35356]: from='client.? 172.21.15.12:0/1454612391' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T00:06:26.562 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:06:26 smithi012 ceph-mon[30333]: from='client.? 172.21.15.12:0/1454612391' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T00:06:26.580 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:06:26 smithi178 ceph-mon[35919]: from='client.? 172.21.15.12:0/1454612391' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T00:06:26.714 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid ff3ad74a-6fcf-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T00:06:27.091 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:06:27.532 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:06:27 smithi165 ceph-mon[35356]: pgmap v379: 129 pgs: 129 active+clean; 647 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 895 B/s wr, 13 op/s; 14 B/s, 0 objects/s recovering 2024-09-11T00:06:27.532 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:06:27 smithi165 ceph-mon[35356]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2024-09-11T00:06:27.532 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:06:27 smithi165 ceph-mon[35356]: Cluster is now healthy 2024-09-11T00:06:27.563 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:06:27 smithi012 ceph-mon[30333]: pgmap v379: 129 pgs: 129 active+clean; 647 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 895 B/s wr, 13 op/s; 14 B/s, 0 objects/s recovering 2024-09-11T00:06:27.563 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:06:27 smithi012 ceph-mon[30333]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2024-09-11T00:06:27.563 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:06:27 smithi012 ceph-mon[30333]: Cluster is now healthy 2024-09-11T00:06:27.580 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:06:27 smithi178 ceph-mon[35919]: pgmap v379: 129 pgs: 129 active+clean; 647 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 895 B/s wr, 13 op/s; 14 B/s, 0 objects/s recovering 2024-09-11T00:06:27.581 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:06:27 smithi178 ceph-mon[35919]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2024-09-11T00:06:27.581 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:06:27 smithi178 ceph-mon[35919]: Cluster is now healthy 2024-09-11T00:06:27.713 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:27.711+0000 7fb0fe200640 1 -- 172.21.15.12:0/2473699358 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb0f8109150 msgr2=0x7fb0f810f950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:06:27.714 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:27.711+0000 7fb0fe200640 1 --2- 172.21.15.12:0/2473699358 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb0f8109150 0x7fb0f810f950 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fb0f0009be0 tx=0x7fb0f002a6c0 comp rx=0 tx=0).stop 2024-09-11T00:06:27.714 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:27.711+0000 7fb0fe200640 1 -- 172.21.15.12:0/2473699358 shutdown_connections 2024-09-11T00:06:27.714 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:27.711+0000 7fb0fe200640 1 --2- 172.21.15.12:0/2473699358 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb0f8109150 0x7fb0f810f950 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:27.714 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:27.711+0000 7fb0fe200640 1 --2- 172.21.15.12:0/2473699358 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb0f81087d0 0x7fb0f8108c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:27.714 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:27.711+0000 7fb0fe200640 1 --2- 172.21.15.12:0/2473699358 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb0f8102dc0 0x7fb0f81031a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:27.714 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:27.711+0000 7fb0fe200640 1 -- 172.21.15.12:0/2473699358 >> 172.21.15.12:0/2473699358 conn(0x7fb0f80fe830 msgr2=0x7fb0f8100c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:06:27.715 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:27.711+0000 7fb0fe200640 1 -- 172.21.15.12:0/2473699358 shutdown_connections 2024-09-11T00:06:27.715 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:27.711+0000 7fb0fe200640 1 -- 172.21.15.12:0/2473699358 wait complete. 2024-09-11T00:06:27.717 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:27.712+0000 7fb0fe200640 1 Processor -- start 2024-09-11T00:06:27.717 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:27.713+0000 7fb0fe200640 1 -- start start 2024-09-11T00:06:27.717 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:27.713+0000 7fb0fe200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb0f8102dc0 0x7fb0f8199240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:06:27.717 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:27.713+0000 7fb0fe200640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb0f81087d0 0x7fb0f8199780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:06:27.718 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:27.713+0000 7fb0fe200640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb0f8109150 0x7fb0f819eb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:06:27.718 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:27.713+0000 7fb0fe200640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fb0f8112dd0 con 0x7fb0f81087d0 2024-09-11T00:06:27.718 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:27.713+0000 7fb0fe200640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fb0f8112c50 con 0x7fb0f8102dc0 2024-09-11T00:06:27.718 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:27.713+0000 7fb0fe200640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fb0f8112f50 con 0x7fb0f8109150 2024-09-11T00:06:27.718 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:27.713+0000 7fb0f6a00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb0f81087d0 0x7fb0f8199780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:06:27.718 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:27.713+0000 7fb0f7400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb0f8102dc0 0x7fb0f8199240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:06:27.719 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:27.714+0000 7fb0f7e00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb0f8109150 0x7fb0f819eb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:06:27.719 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:27.714+0000 7fb0f7400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb0f8102dc0 0x7fb0f8199240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.12:43330/0 (socket says 172.21.15.12:43330) 2024-09-11T00:06:27.719 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:27.714+0000 7fb0f7400640 1 -- 172.21.15.12:0/222432717 learned_addr learned my addr 172.21.15.12:0/222432717 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:06:27.719 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:27.714+0000 7fb0f6a00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb0f81087d0 0x7fb0f8199780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:48738/0 (socket says 172.21.15.12:48738) 2024-09-11T00:06:27.719 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:27.714+0000 7fb0f7400640 1 -- 172.21.15.12:0/222432717 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb0f8109150 msgr2=0x7fb0f819eb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:06:27.719 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:27.714+0000 7fb0f7400640 1 --2- 172.21.15.12:0/222432717 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb0f8109150 0x7fb0f819eb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:27.719 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:27.714+0000 7fb0f7400640 1 -- 172.21.15.12:0/222432717 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb0f81087d0 msgr2=0x7fb0f8199780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:06:27.720 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:27.714+0000 7fb0f7400640 1 --2- 172.21.15.12:0/222432717 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb0f81087d0 0x7fb0f8199780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:27.720 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:27.714+0000 7fb0f7400640 1 -- 172.21.15.12:0/222432717 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0f819f2f0 con 0x7fb0f8102dc0 2024-09-11T00:06:27.720 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:27.714+0000 7fb0f7400640 1 --2- 172.21.15.12:0/222432717 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb0f8102dc0 0x7fb0f8199240 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fb0e800e9c0 tx=0x7fb0e800ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:06:27.720 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:27.714+0000 7fb0efe00640 1 -- 172.21.15.12:0/222432717 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0e800cde0 con 0x7fb0f8102dc0 2024-09-11T00:06:27.720 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:27.715+0000 7fb0fe200640 1 -- 172.21.15.12:0/222432717 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb0f819f5e0 con 0x7fb0f8102dc0 2024-09-11T00:06:27.721 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:27.715+0000 7fb0efe00640 1 -- 172.21.15.12:0/222432717 <== mon.2 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb0e80035f0 con 0x7fb0f8102dc0 2024-09-11T00:06:27.721 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:27.715+0000 7fb0efe00640 1 -- 172.21.15.12:0/222432717 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0e8010690 con 0x7fb0f8102dc0 2024-09-11T00:06:27.721 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:27.715+0000 7fb0fe200640 1 -- 172.21.15.12:0/222432717 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0f81a5e70 con 0x7fb0f8102dc0 2024-09-11T00:06:27.722 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:27.716+0000 7fb0fe200640 1 -- 172.21.15.12:0/222432717 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb0f81031a0 con 0x7fb0f8102dc0 2024-09-11T00:06:27.723 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:27.718+0000 7fb0efe00640 1 -- 172.21.15.12:0/222432717 <== mon.2 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7fb0e8004000 con 0x7fb0f8102dc0 2024-09-11T00:06:27.723 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:27.720+0000 7fb0efe00640 1 --2- 172.21.15.12:0/222432717 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fb0dc0b52e0 0x7fb0dc0b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:06:27.723 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:27.720+0000 7fb0efe00640 1 -- 172.21.15.12:0/222432717 <== mon.2 v2:172.21.15.165:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9208+0+0 (secure 0 0 0) 0x7fb0e80e63a0 con 0x7fb0f8102dc0 2024-09-11T00:06:27.723 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:27.720+0000 7fb0f6a00640 1 --2- 172.21.15.12:0/222432717 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fb0dc0b52e0 0x7fb0dc0b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:06:27.723 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:27.720+0000 7fb0f6a00640 1 --2- 172.21.15.12:0/222432717 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fb0dc0b52e0 0x7fb0dc0b77a0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fb0f8104f80 tx=0x7fb0e4005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:06:27.724 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:27.721+0000 7fb0efe00640 1 -- 172.21.15.12:0/222432717 <== mon.2 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7fb0e80acec0 con 0x7fb0f8102dc0 2024-09-11T00:06:27.962 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:27.960+0000 7fb0fe200640 1 -- 172.21.15.12:0/222432717 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb0f819a370 con 0x7fb0f8102dc0 2024-09-11T00:06:27.963 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:27.961+0000 7fb0efe00640 1 -- 172.21.15.12:0/222432717 <== mon.2 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11965 (secure 0 0 0) 0x7fb0e80ac7c0 con 0x7fb0f8102dc0 2024-09-11T00:06:27.963 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-11T00:06:27.963 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":28,"btime":"2024-09-11T00:04:50:247323+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14577,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6835/447171591","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":447171591},{"type":"v1","addr":"172.21.15.12:6835","nonce":447171591}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14589,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/901486040","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":901486040},{"type":"v1","addr":"172.21.15.12:6837","nonce":901486040}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T00:04:04.739881+0000","modified":"2024-09-11T00:04:50.247301+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24674,"mds_1":24643,"mds_2":24638,"mds_3":24637,"mds_4":24614},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14631":{"gid":14631,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.12:6841/3069919437","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6840","nonce":3069919437},{"type":"v1","addr":"172.21.15.12:6841","nonce":3069919437}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24562":{"gid":24562,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6833/1141636112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":1141636112},{"type":"v1","addr":"172.21.15.178:6833","nonce":1141636112}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24581":{"gid":24581,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6833/3111980970","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6832","nonce":3111980970},{"type":"v1","addr":"172.21.15.165:6833","nonce":3111980970}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24607":{"gid":24607,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6835/4084868287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":4084868287},{"type":"v1","addr":"172.21.15.178:6835","nonce":4084868287}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24613":{"gid":24613,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6837/526874763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":526874763},{"type":"v1","addr":"172.21.15.178:6837","nonce":526874763}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24614":{"gid":24614,"name":"e","rank":4,"incarnation":26,"state":"up:active","state_seq":21,"addr":"172.21.15.165:6835/2425395274","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":2425395274},{"type":"v1","addr":"172.21.15.165:6835","nonce":2425395274}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24637":{"gid":24637,"name":"i","rank":3,"incarnation":24,"state":"up:active","state_seq":18,"addr":"172.21.15.178:6839/70628097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6838","nonce":70628097},{"type":"v1","addr":"172.21.15.178:6839","nonce":70628097}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24638":{"gid":24638,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":18,"addr":"172.21.15.165:6837/6215088","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6836","nonce":6215088},{"type":"v1","addr":"172.21.15.165:6837","nonce":6215088}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24643":{"gid":24643,"name":"g","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.12:6839/3124093009","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6838","nonce":3124093009},{"type":"v1","addr":"172.21.15.12:6839","nonce":3124093009}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24674":{"gid":24674,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.165:6839/501312984","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6838","nonce":501312984},{"type":"v1","addr":"172.21.15.165:6839","nonce":501312984}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24674,"qdb_cluster":[24674,24643,24638,24637,24614]},"id":1}]} 2024-09-11T00:06:27.964 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 28 2024-09-11T00:06:27.966 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:27.964+0000 7fb0fe200640 1 -- 172.21.15.12:0/222432717 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fb0dc0b52e0 msgr2=0x7fb0dc0b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:06:27.966 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:27.964+0000 7fb0fe200640 1 --2- 172.21.15.12:0/222432717 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fb0dc0b52e0 0x7fb0dc0b77a0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fb0f8104f80 tx=0x7fb0e4005f90 comp rx=0 tx=0).stop 2024-09-11T00:06:27.967 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:27.964+0000 7fb0fe200640 1 -- 172.21.15.12:0/222432717 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb0f8102dc0 msgr2=0x7fb0f8199240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:06:27.967 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:27.964+0000 7fb0fe200640 1 --2- 172.21.15.12:0/222432717 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb0f8102dc0 0x7fb0f8199240 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fb0e800e9c0 tx=0x7fb0e800ee80 comp rx=0 tx=0).stop 2024-09-11T00:06:27.967 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:27.964+0000 7fb0fe200640 1 -- 172.21.15.12:0/222432717 shutdown_connections 2024-09-11T00:06:27.967 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:27.964+0000 7fb0fe200640 1 --2- 172.21.15.12:0/222432717 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fb0dc0b52e0 0x7fb0dc0b77a0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:27.967 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:27.964+0000 7fb0fe200640 1 --2- 172.21.15.12:0/222432717 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb0f8109150 0x7fb0f819eb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:27.967 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:27.964+0000 7fb0fe200640 1 --2- 172.21.15.12:0/222432717 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb0f81087d0 0x7fb0f8199780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:27.967 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:27.964+0000 7fb0fe200640 1 --2- 172.21.15.12:0/222432717 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb0f8102dc0 0x7fb0f8199240 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:27.967 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:27.964+0000 7fb0fe200640 1 -- 172.21.15.12:0/222432717 >> 172.21.15.12:0/222432717 conn(0x7fb0f80fe830 msgr2=0x7fb0f8100c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:06:27.967 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:27.965+0000 7fb0fe200640 1 -- 172.21.15.12:0/222432717 shutdown_connections 2024-09-11T00:06:27.968 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:27.965+0000 7fb0fe200640 1 -- 172.21.15.12:0/222432717 wait complete. 2024-09-11T00:06:28.532 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:06:28 smithi165 ceph-mon[35356]: from='client.? 172.21.15.12:0/222432717' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T00:06:28.562 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:06:28 smithi012 ceph-mon[30333]: from='client.? 172.21.15.12:0/222432717' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T00:06:28.580 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:06:28 smithi178 ceph-mon[35919]: from='client.? 172.21.15.12:0/222432717' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T00:06:28.756 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid ff3ad74a-6fcf-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-11T00:06:29.134 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:06:29.532 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:06:29 smithi165 ceph-mon[35356]: pgmap v380: 129 pgs: 129 active+clean; 647 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 802 B/s wr, 12 op/s; 13 B/s, 0 objects/s recovering 2024-09-11T00:06:29.563 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:06:29 smithi012 ceph-mon[30333]: pgmap v380: 129 pgs: 129 active+clean; 647 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 802 B/s wr, 12 op/s; 13 B/s, 0 objects/s recovering 2024-09-11T00:06:29.580 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:06:29 smithi178 ceph-mon[35919]: pgmap v380: 129 pgs: 129 active+clean; 647 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 802 B/s wr, 12 op/s; 13 B/s, 0 objects/s recovering 2024-09-11T00:06:29.696 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:29.694+0000 7ffbc6e00640 1 -- 172.21.15.12:0/2799251412 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ffbc010a800 msgr2=0x7ffbc010ac40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:06:29.696 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:29.694+0000 7ffbc6e00640 1 --2- 172.21.15.12:0/2799251412 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ffbc010a800 0x7ffbc010ac40 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7ffbac009900 tx=0x7ffbac02a3f0 comp rx=0 tx=0).stop 2024-09-11T00:06:29.696 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:29.694+0000 7ffbc6e00640 1 -- 172.21.15.12:0/2799251412 shutdown_connections 2024-09-11T00:06:29.696 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:29.694+0000 7ffbc6e00640 1 --2- 172.21.15.12:0/2799251412 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ffbc010b180 0x7ffbc0111980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:29.696 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:29.694+0000 7ffbc6e00640 1 --2- 172.21.15.12:0/2799251412 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ffbc010a800 0x7ffbc010ac40 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:29.697 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:29.694+0000 7ffbc6e00640 1 --2- 172.21.15.12:0/2799251412 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ffbc0104df0 0x7ffbc01051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:29.697 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:29.694+0000 7ffbc6e00640 1 -- 172.21.15.12:0/2799251412 >> 172.21.15.12:0/2799251412 conn(0x7ffbc0100840 msgr2=0x7ffbc0102c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:06:29.697 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:29.694+0000 7ffbc6e00640 1 -- 172.21.15.12:0/2799251412 shutdown_connections 2024-09-11T00:06:29.697 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:29.694+0000 7ffbc6e00640 1 -- 172.21.15.12:0/2799251412 wait complete. 2024-09-11T00:06:29.698 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:29.695+0000 7ffbc6e00640 1 Processor -- start 2024-09-11T00:06:29.698 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:29.695+0000 7ffbc6e00640 1 -- start start 2024-09-11T00:06:29.698 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:29.695+0000 7ffbc6e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ffbc0104df0 0x7ffbc019b210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:06:29.699 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:29.695+0000 7ffbc6e00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ffbc010a800 0x7ffbc019b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:06:29.699 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:29.695+0000 7ffbc6e00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ffbc010b180 0x7ffbc01a0af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:06:29.699 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:29.695+0000 7ffbc6e00640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7ffbc0114e50 con 0x7ffbc010a800 2024-09-11T00:06:29.699 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:29.695+0000 7ffbc6e00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7ffbc0114cd0 con 0x7ffbc0104df0 2024-09-11T00:06:29.699 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:29.695+0000 7ffbc6e00640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7ffbc0114fd0 con 0x7ffbc010b180 2024-09-11T00:06:29.699 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:29.696+0000 7ffbc5000640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ffbc010a800 0x7ffbc019b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:06:29.699 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:29.696+0000 7ffbc5000640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ffbc010a800 0x7ffbc019b750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:43426/0 (socket says 172.21.15.12:43426) 2024-09-11T00:06:29.699 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:29.696+0000 7ffbc5000640 1 -- 172.21.15.12:0/1572819624 learned_addr learned my addr 172.21.15.12:0/1572819624 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:06:29.699 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:29.696+0000 7ffbc5a00640 1 --2- 172.21.15.12:0/1572819624 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ffbc0104df0 0x7ffbc019b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:06:29.699 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:29.696+0000 7ffbc6400640 1 --2- 172.21.15.12:0/1572819624 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ffbc010b180 0x7ffbc01a0af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:06:29.699 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:29.696+0000 7ffbc5000640 1 -- 172.21.15.12:0/1572819624 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ffbc010b180 msgr2=0x7ffbc01a0af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:06:29.699 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:29.696+0000 7ffbc5000640 1 --2- 172.21.15.12:0/1572819624 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ffbc010b180 0x7ffbc01a0af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:29.699 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:29.696+0000 7ffbc5000640 1 -- 172.21.15.12:0/1572819624 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ffbc0104df0 msgr2=0x7ffbc019b210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:06:29.700 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:29.696+0000 7ffbc5000640 1 --2- 172.21.15.12:0/1572819624 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ffbc0104df0 0x7ffbc019b210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:29.700 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:29.696+0000 7ffbc5000640 1 -- 172.21.15.12:0/1572819624 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffbc01a12c0 con 0x7ffbc010a800 2024-09-11T00:06:29.700 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:29.696+0000 7ffbc6400640 1 --2- 172.21.15.12:0/1572819624 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ffbc010b180 0x7ffbc01a0af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T00:06:29.700 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:29.697+0000 7ffbc5000640 1 --2- 172.21.15.12:0/1572819624 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ffbc010a800 0x7ffbc019b750 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7ffbac005ab0 tx=0x7ffbac003c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:06:29.700 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:29.697+0000 7ffbbe000640 1 -- 172.21.15.12:0/1572819624 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbac03f070 con 0x7ffbc010a800 2024-09-11T00:06:29.701 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:29.697+0000 7ffbbe000640 1 -- 172.21.15.12:0/1572819624 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffbac031d60 con 0x7ffbc010a800 2024-09-11T00:06:29.701 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:29.697+0000 7ffbc6e00640 1 -- 172.21.15.12:0/1572819624 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffbc01a1550 con 0x7ffbc010a800 2024-09-11T00:06:29.701 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:29.697+0000 7ffbbe000640 1 -- 172.21.15.12:0/1572819624 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbac035030 con 0x7ffbc010a800 2024-09-11T00:06:29.701 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:29.697+0000 7ffbc6e00640 1 -- 172.21.15.12:0/1572819624 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffbc01a7c70 con 0x7ffbc010a800 2024-09-11T00:06:29.701 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:29.698+0000 7ffbc6e00640 1 -- 172.21.15.12:0/1572819624 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffbc01051d0 con 0x7ffbc010a800 2024-09-11T00:06:29.702 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:29.700+0000 7ffbbe000640 1 -- 172.21.15.12:0/1572819624 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7ffbac03b460 con 0x7ffbc010a800 2024-09-11T00:06:29.704 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:29.702+0000 7ffbbe000640 1 --2- 172.21.15.12:0/1572819624 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7ffba80b5410 0x7ffba80b78d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:06:29.704 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:29.702+0000 7ffbbe000640 1 -- 172.21.15.12:0/1572819624 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9208+0+0 (secure 0 0 0) 0x7ffbac1066f0 con 0x7ffbc010a800 2024-09-11T00:06:29.704 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:29.702+0000 7ffbc5a00640 1 --2- 172.21.15.12:0/1572819624 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7ffba80b5410 0x7ffba80b78d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:06:29.704 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:29.703+0000 7ffbc5a00640 1 --2- 172.21.15.12:0/1572819624 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7ffba80b5410 0x7ffba80b78d0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7ffbb4005fd0 tx=0x7ffbb4009760 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:06:29.707 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:29.706+0000 7ffbbe000640 1 -- 172.21.15.12:0/1572819624 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7ffbac0cd180 con 0x7ffbc010a800 2024-09-11T00:06:29.904 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:29.902+0000 7ffbc6e00640 1 -- 172.21.15.12:0/1572819624 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ffbc019c340 con 0x7ffbc010a800 2024-09-11T00:06:29.906 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:29.904+0000 7ffbbe000640 1 -- 172.21.15.12:0/1572819624 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) ==== 74+0+24948 (secure 0 0 0) 0x7ffbac0cca80 con 0x7ffbc010a800 2024-09-11T00:06:29.906 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-11T00:06:29.907 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":85,"fsid":"ff3ad74a-6fcf-11ef-bcea-c7b262605968","created":"2024-09-10T23:55:16.317643+0000","modified":"2024-09-11T00:06:20.198537+0000","last_up_change":"2024-09-11T00:03:08.959256+0000","last_in_change":"2024-09-11T00:02:46.564631+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T23:59:10.488728+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-11T00:04:00.065618+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"74","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-11T00:04:02.689888+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"74","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-11T00:04:09.862120+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"6382c200-a912-42a1-958b-79191b21249b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6802","nonce":1237101991},{"type":"v1","addr":"172.21.15.12:6803","nonce":1237101991}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6804","nonce":1237101991},{"type":"v1","addr":"172.21.15.12:6805","nonce":1237101991}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6808","nonce":1237101991},{"type":"v1","addr":"172.21.15.12:6809","nonce":1237101991}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6806","nonce":1237101991},{"type":"v1","addr":"172.21.15.12:6807","nonce":1237101991}]},"public_addr":"172.21.15.12:6803/1237101991","cluster_addr":"172.21.15.12:6805/1237101991","heartbeat_back_addr":"172.21.15.12:6809/1237101991","heartbeat_front_addr":"172.21.15.12:6807/1237101991","state":["exists","up"]},{"osd":1,"uuid":"fe0e1d0a-bdc5-4859-aa84-9038059a45f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6800","nonce":545848456},{"type":"v1","addr":"172.21.15.165:6801","nonce":545848456}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6802","nonce":545848456},{"type":"v1","addr":"172.21.15.165:6803","nonce":545848456}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6806","nonce":545848456},{"type":"v1","addr":"172.21.15.165:6807","nonce":545848456}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6804","nonce":545848456},{"type":"v1","addr":"172.21.15.165:6805","nonce":545848456}]},"public_addr":"172.21.15.165:6801/545848456","cluster_addr":"172.21.15.165:6803/545848456","heartbeat_back_addr":"172.21.15.165:6807/545848456","heartbeat_front_addr":"172.21.15.165:6805/545848456","state":["exists","up"]},{"osd":2,"uuid":"94f640dc-5f78-4e2c-87d9-03e96a6e7ba0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6800","nonce":4131162695},{"type":"v1","addr":"172.21.15.178:6801","nonce":4131162695}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":4131162695},{"type":"v1","addr":"172.21.15.178:6803","nonce":4131162695}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":4131162695},{"type":"v1","addr":"172.21.15.178:6807","nonce":4131162695}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":4131162695},{"type":"v1","addr":"172.21.15.178:6805","nonce":4131162695}]},"public_addr":"172.21.15.178:6801/4131162695","cluster_addr":"172.21.15.178:6803/4131162695","heartbeat_back_addr":"172.21.15.178:6807/4131162695","heartbeat_front_addr":"172.21.15.178:6805/4131162695","state":["exists","up"]},{"osd":3,"uuid":"af2f64d6-c9be-44a9-b041-8d599a7c080d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6810","nonce":452464245},{"type":"v1","addr":"172.21.15.12:6811","nonce":452464245}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6812","nonce":452464245},{"type":"v1","addr":"172.21.15.12:6813","nonce":452464245}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6816","nonce":452464245},{"type":"v1","addr":"172.21.15.12:6817","nonce":452464245}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6814","nonce":452464245},{"type":"v1","addr":"172.21.15.12:6815","nonce":452464245}]},"public_addr":"172.21.15.12:6811/452464245","cluster_addr":"172.21.15.12:6813/452464245","heartbeat_back_addr":"172.21.15.12:6817/452464245","heartbeat_front_addr":"172.21.15.12:6815/452464245","state":["exists","up"]},{"osd":4,"uuid":"ad76ac70-20c0-44ae-9866-5f9cfce2334c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6808","nonce":1899879280},{"type":"v1","addr":"172.21.15.165:6809","nonce":1899879280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6810","nonce":1899879280},{"type":"v1","addr":"172.21.15.165:6811","nonce":1899879280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6814","nonce":1899879280},{"type":"v1","addr":"172.21.15.165:6815","nonce":1899879280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6812","nonce":1899879280},{"type":"v1","addr":"172.21.15.165:6813","nonce":1899879280}]},"public_addr":"172.21.15.165:6809/1899879280","cluster_addr":"172.21.15.165:6811/1899879280","heartbeat_back_addr":"172.21.15.165:6815/1899879280","heartbeat_front_addr":"172.21.15.165:6813/1899879280","state":["exists","up"]},{"osd":5,"uuid":"aafda2cc-62fe-4fcb-9a96-8732e0b501d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":3042228555},{"type":"v1","addr":"172.21.15.178:6809","nonce":3042228555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":3042228555},{"type":"v1","addr":"172.21.15.178:6811","nonce":3042228555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6814","nonce":3042228555},{"type":"v1","addr":"172.21.15.178:6815","nonce":3042228555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":3042228555},{"type":"v1","addr":"172.21.15.178:6813","nonce":3042228555}]},"public_addr":"172.21.15.178:6809/3042228555","cluster_addr":"172.21.15.178:6811/3042228555","heartbeat_back_addr":"172.21.15.178:6815/3042228555","heartbeat_front_addr":"172.21.15.178:6813/3042228555","state":["exists","up"]},{"osd":6,"uuid":"375eca52-efe3-42a5-bbb1-d7d392d71f15","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6818","nonce":3868322467},{"type":"v1","addr":"172.21.15.12:6819","nonce":3868322467}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6820","nonce":3868322467},{"type":"v1","addr":"172.21.15.12:6821","nonce":3868322467}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6824","nonce":3868322467},{"type":"v1","addr":"172.21.15.12:6825","nonce":3868322467}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6822","nonce":3868322467},{"type":"v1","addr":"172.21.15.12:6823","nonce":3868322467}]},"public_addr":"172.21.15.12:6819/3868322467","cluster_addr":"172.21.15.12:6821/3868322467","heartbeat_back_addr":"172.21.15.12:6825/3868322467","heartbeat_front_addr":"172.21.15.12:6823/3868322467","state":["exists","up"]},{"osd":7,"uuid":"0fd619f5-44c3-4cf7-ae7a-388dd798de36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6816","nonce":1143773735},{"type":"v1","addr":"172.21.15.165:6817","nonce":1143773735}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6818","nonce":1143773735},{"type":"v1","addr":"172.21.15.165:6819","nonce":1143773735}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6822","nonce":1143773735},{"type":"v1","addr":"172.21.15.165:6823","nonce":1143773735}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6820","nonce":1143773735},{"type":"v1","addr":"172.21.15.165:6821","nonce":1143773735}]},"public_addr":"172.21.15.165:6817/1143773735","cluster_addr":"172.21.15.165:6819/1143773735","heartbeat_back_addr":"172.21.15.165:6823/1143773735","heartbeat_front_addr":"172.21.15.165:6821/1143773735","state":["exists","up"]},{"osd":8,"uuid":"b03d14ce-8427-462a-9bcc-27683c549f53","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":2891445058},{"type":"v1","addr":"172.21.15.178:6817","nonce":2891445058}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6818","nonce":2891445058},{"type":"v1","addr":"172.21.15.178:6819","nonce":2891445058}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6822","nonce":2891445058},{"type":"v1","addr":"172.21.15.178:6823","nonce":2891445058}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":2891445058},{"type":"v1","addr":"172.21.15.178:6821","nonce":2891445058}]},"public_addr":"172.21.15.178:6817/2891445058","cluster_addr":"172.21.15.178:6819/2891445058","heartbeat_back_addr":"172.21.15.178:6823/2891445058","heartbeat_front_addr":"172.21.15.178:6821/2891445058","state":["exists","up"]},{"osd":9,"uuid":"81a6cf69-2a24-4a6c-9adf-6b7df38cfacb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6826","nonce":3359634903},{"type":"v1","addr":"172.21.15.12:6827","nonce":3359634903}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6828","nonce":3359634903},{"type":"v1","addr":"172.21.15.12:6829","nonce":3359634903}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6832","nonce":3359634903},{"type":"v1","addr":"172.21.15.12:6833","nonce":3359634903}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6830","nonce":3359634903},{"type":"v1","addr":"172.21.15.12:6831","nonce":3359634903}]},"public_addr":"172.21.15.12:6827/3359634903","cluster_addr":"172.21.15.12:6829/3359634903","heartbeat_back_addr":"172.21.15.12:6833/3359634903","heartbeat_front_addr":"172.21.15.12:6831/3359634903","state":["exists","up"]},{"osd":10,"uuid":"c1ad1f0a-33f4-4c32-91f6-4d030a29f1dd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6824","nonce":3642325778},{"type":"v1","addr":"172.21.15.165:6825","nonce":3642325778}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6826","nonce":3642325778},{"type":"v1","addr":"172.21.15.165:6827","nonce":3642325778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6830","nonce":3642325778},{"type":"v1","addr":"172.21.15.165:6831","nonce":3642325778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6828","nonce":3642325778},{"type":"v1","addr":"172.21.15.165:6829","nonce":3642325778}]},"public_addr":"172.21.15.165:6825/3642325778","cluster_addr":"172.21.15.165:6827/3642325778","heartbeat_back_addr":"172.21.15.165:6831/3642325778","heartbeat_front_addr":"172.21.15.165:6829/3642325778","state":["exists","up"]},{"osd":11,"uuid":"8413c9b0-680b-4d02-aeab-b3b3ff16c3d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":2937059756},{"type":"v1","addr":"172.21.15.178:6825","nonce":2937059756}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6826","nonce":2937059756},{"type":"v1","addr":"172.21.15.178:6827","nonce":2937059756}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":2937059756},{"type":"v1","addr":"172.21.15.178:6831","nonce":2937059756}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":2937059756},{"type":"v1","addr":"172.21.15.178:6829","nonce":2937059756}]},"public_addr":"172.21.15.178:6825/2937059756","cluster_addr":"172.21.15.178:6827/2937059756","heartbeat_back_addr":"172.21.15.178:6831/2937059756","heartbeat_front_addr":"172.21.15.178:6829/2937059756","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:58:16.036348+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:58:41.483150+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:59:07.107283+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:59:35.706574+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T00:00:02.555720+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T00:00:28.092847+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T00:00:57.396495+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T00:01:24.216775+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T00:01:48.807850+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T00:02:16.838565+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T00:02:43.131736+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T00:03:07.353155+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":8,"to":4}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"3.e","mappings":[{"from":5,"to":9}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9}]},{"pgid":"3.23","mappings":[{"from":4,"to":7}]},{"pgid":"3.31","mappings":[{"from":4,"to":8}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"3.39","mappings":[{"from":5,"to":6}]},{"pgid":"4.5","mappings":[{"from":9,"to":10}]},{"pgid":"4.6","mappings":[{"from":5,"to":8}]},{"pgid":"4.b","mappings":[{"from":9,"to":8}]},{"pgid":"4.13","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.12:6801/4177251511":"2024-09-11T23:56:17.999820+0000","172.21.15.12:6800/4177251511":"2024-09-11T23:56:17.999820+0000","172.21.15.12:0/3273692817":"2024-09-11T23:56:17.999820+0000","172.21.15.12:0/4017778158":"2024-09-11T23:56:17.999820+0000","172.21.15.12:0/2499921051":"2024-09-11T23:55:50.302721+0000","172.21.15.12:6801/760140910":"2024-09-11T23:55:50.302721+0000","172.21.15.12:6800/760140910":"2024-09-11T23:55:50.302721+0000","172.21.15.12:0/2810703302":"2024-09-11T23:56:17.999820+0000","172.21.15.12:0/2325436065":"2024-09-11T23:55:50.302721+0000","172.21.15.12:0/3137053479":"2024-09-11T23:55:50.302721+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-11T00:06:29.909 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:29.906+0000 7ffbc6e00640 1 -- 172.21.15.12:0/1572819624 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7ffba80b5410 msgr2=0x7ffba80b78d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:06:29.909 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:29.906+0000 7ffbc6e00640 1 --2- 172.21.15.12:0/1572819624 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7ffba80b5410 0x7ffba80b78d0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7ffbb4005fd0 tx=0x7ffbb4009760 comp rx=0 tx=0).stop 2024-09-11T00:06:29.909 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:29.906+0000 7ffbc6e00640 1 -- 172.21.15.12:0/1572819624 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ffbc010a800 msgr2=0x7ffbc019b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:06:29.909 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:29.906+0000 7ffbc6e00640 1 --2- 172.21.15.12:0/1572819624 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ffbc010a800 0x7ffbc019b750 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7ffbac005ab0 tx=0x7ffbac003c80 comp rx=0 tx=0).stop 2024-09-11T00:06:29.909 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:29.907+0000 7ffbc6e00640 1 -- 172.21.15.12:0/1572819624 shutdown_connections 2024-09-11T00:06:29.909 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:29.907+0000 7ffbc6e00640 1 --2- 172.21.15.12:0/1572819624 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7ffba80b5410 0x7ffba80b78d0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:29.909 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:29.907+0000 7ffbc6e00640 1 --2- 172.21.15.12:0/1572819624 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ffbc010b180 0x7ffbc01a0af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:29.909 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:29.907+0000 7ffbc6e00640 1 --2- 172.21.15.12:0/1572819624 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ffbc010a800 0x7ffbc019b750 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:29.909 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:29.907+0000 7ffbc6e00640 1 --2- 172.21.15.12:0/1572819624 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ffbc0104df0 0x7ffbc019b210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:29.909 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:29.907+0000 7ffbc6e00640 1 -- 172.21.15.12:0/1572819624 >> 172.21.15.12:0/1572819624 conn(0x7ffbc0100840 msgr2=0x7ffbc0107de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:06:29.909 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:29.907+0000 7ffbc6e00640 1 -- 172.21.15.12:0/1572819624 shutdown_connections 2024-09-11T00:06:29.910 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:29.907+0000 7ffbc6e00640 1 -- 172.21.15.12:0/1572819624 wait complete. 2024-09-11T00:06:30.452 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:06:30 smithi012 ceph-mon[30333]: from='client.? 172.21.15.12:0/1572819624' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-11T00:06:30.525 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 28, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-09-11T00:04:04.739881+0000', 'modified': '2024-09-11T00:04:50.247301+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 24674, 'mds_1': 24643, 'mds_2': 24638, 'mds_3': 24637, 'mds_4': 24614}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_14631': {'gid': 14631, 'name': 'a', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.12:6841/3069919437', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.12:6840', 'nonce': 3069919437}, {'type': 'v1', 'addr': '172.21.15.12:6841', 'nonce': 3069919437}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24562': {'gid': 24562, 'name': 'l', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.178:6833/1141636112', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.178:6832', 'nonce': 1141636112}, {'type': 'v1', 'addr': '172.21.15.178:6833', 'nonce': 1141636112}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24581': {'gid': 24581, 'name': 'k', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.165:6833/3111980970', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.165:6832', 'nonce': 3111980970}, {'type': 'v1', 'addr': '172.21.15.165:6833', 'nonce': 3111980970}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24607': {'gid': 24607, 'name': 'f', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.178:6835/4084868287', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.178:6834', 'nonce': 4084868287}, {'type': 'v1', 'addr': '172.21.15.178:6835', 'nonce': 4084868287}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24613': {'gid': 24613, 'name': 'c', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.178:6837/526874763', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.178:6836', 'nonce': 526874763}, {'type': 'v1', 'addr': '172.21.15.178:6837', 'nonce': 526874763}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24614': {'gid': 24614, 'name': 'e', 'rank': 4, 'incarnation': 26, 'state': 'up:active', 'state_seq': 21, 'addr': '172.21.15.165:6835/2425395274', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.165:6834', 'nonce': 2425395274}, {'type': 'v1', 'addr': '172.21.15.165:6835', 'nonce': 2425395274}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24637': {'gid': 24637, 'name': 'i', 'rank': 3, 'incarnation': 24, 'state': 'up:active', 'state_seq': 18, 'addr': '172.21.15.178:6839/70628097', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.178:6838', 'nonce': 70628097}, {'type': 'v1', 'addr': '172.21.15.178:6839', 'nonce': 70628097}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24638': {'gid': 24638, 'name': 'h', 'rank': 2, 'incarnation': 22, 'state': 'up:active', 'state_seq': 18, 'addr': '172.21.15.165:6837/6215088', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.165:6836', 'nonce': 6215088}, {'type': 'v1', 'addr': '172.21.15.165:6837', 'nonce': 6215088}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24643': {'gid': 24643, 'name': 'g', 'rank': 1, 'incarnation': 20, 'state': 'up:active', 'state_seq': 17, 'addr': '172.21.15.12:6839/3124093009', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.12:6838', 'nonce': 3124093009}, {'type': 'v1', 'addr': '172.21.15.12:6839', 'nonce': 3124093009}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24674': {'gid': 24674, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.165:6839/501312984', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.165:6838', 'nonce': 501312984}, {'type': 'v1', 'addr': '172.21.15.165:6839', 'nonce': 501312984}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24674, 'qdb_cluster': [24674, 24643, 24638, 24637, 24614]} 2024-09-11T00:06:30.526 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2024-09-11T00:06:30.526 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid ff3ad74a-6fcf-11ef-bcea-c7b262605968 -- ceph tell mds.e status 2024-09-11T00:06:30.532 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:06:30 smithi165 ceph-mon[35356]: from='client.? 172.21.15.12:0/1572819624' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-11T00:06:30.581 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:06:30 smithi178 ceph-mon[35919]: from='client.? 172.21.15.12:0/1572819624' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-11T00:06:30.903 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:06:31.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:06:31 smithi012 ceph-mon[30333]: pgmap v381: 129 pgs: 129 active+clean; 647 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 511 B/s wr, 10 op/s; 11 B/s, 0 objects/s recovering 2024-09-11T00:06:31.454 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:31.452+0000 7f6f4a000640 1 -- 172.21.15.12:0/3592745046 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6f44075540 msgr2=0x7f6f44075920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:06:31.454 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:31.452+0000 7f6f4a000640 1 --2- 172.21.15.12:0/3592745046 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6f44075540 0x7f6f44075920 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f6f34009900 tx=0x7f6f3402a3f0 comp rx=0 tx=0).stop 2024-09-11T00:06:31.454 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:31.452+0000 7f6f4a000640 1 -- 172.21.15.12:0/3592745046 shutdown_connections 2024-09-11T00:06:31.454 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:31.452+0000 7f6f4a000640 1 --2- 172.21.15.12:0/3592745046 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6f44113b20 0x7f6f44115f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:31.454 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:31.452+0000 7f6f4a000640 1 --2- 172.21.15.12:0/3592745046 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6f44075e60 0x7f6f441135e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:31.455 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:31.452+0000 7f6f4a000640 1 --2- 172.21.15.12:0/3592745046 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6f44075540 0x7f6f44075920 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:31.455 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:31.452+0000 7f6f4a000640 1 -- 172.21.15.12:0/3592745046 >> 172.21.15.12:0/3592745046 conn(0x7f6f440fe830 msgr2=0x7f6f44100c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:06:31.455 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:31.452+0000 7f6f4a000640 1 -- 172.21.15.12:0/3592745046 shutdown_connections 2024-09-11T00:06:31.455 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:31.452+0000 7f6f4a000640 1 -- 172.21.15.12:0/3592745046 wait complete. 2024-09-11T00:06:31.455 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:31.453+0000 7f6f4a000640 1 Processor -- start 2024-09-11T00:06:31.455 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:31.453+0000 7f6f4a000640 1 -- start start 2024-09-11T00:06:31.457 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:31.454+0000 7f6f4a000640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6f44075540 0x7f6f4419f8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:06:31.457 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:31.454+0000 7f6f4a000640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6f44075e60 0x7f6f4419fe30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:06:31.457 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:31.454+0000 7f6f4a000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6f44113b20 0x7f6f441a51d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:06:31.457 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:31.454+0000 7f6f4a000640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f6f44119400 con 0x7f6f44075e60 2024-09-11T00:06:31.457 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:31.454+0000 7f6f4a000640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f6f44119280 con 0x7f6f44113b20 2024-09-11T00:06:31.457 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:31.454+0000 7f6f4a000640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f6f44119580 con 0x7f6f44075540 2024-09-11T00:06:31.457 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:31.454+0000 7f6f3aa00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6f44075e60 0x7f6f4419fe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:06:31.457 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:31.454+0000 7f6f43400640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6f44075540 0x7f6f4419f8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:06:31.457 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:31.454+0000 7f6f43400640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6f44075540 0x7f6f4419f8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.12:47058/0 (socket says 172.21.15.12:47058) 2024-09-11T00:06:31.457 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:31.454+0000 7f6f43400640 1 -- 172.21.15.12:0/2104911097 learned_addr learned my addr 172.21.15.12:0/2104911097 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:06:31.457 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:31.454+0000 7f6f43e00640 1 --2- 172.21.15.12:0/2104911097 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6f44113b20 0x7f6f441a51d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:06:31.457 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:31.455+0000 7f6f43400640 1 -- 172.21.15.12:0/2104911097 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6f44113b20 msgr2=0x7f6f441a51d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:06:31.458 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:31.455+0000 7f6f43400640 1 --2- 172.21.15.12:0/2104911097 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6f44113b20 0x7f6f441a51d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:31.458 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:31.455+0000 7f6f43400640 1 -- 172.21.15.12:0/2104911097 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6f44075e60 msgr2=0x7f6f4419fe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:06:31.458 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:31.455+0000 7f6f43400640 1 --2- 172.21.15.12:0/2104911097 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6f44075e60 0x7f6f4419fe30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:31.458 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:31.455+0000 7f6f43400640 1 -- 172.21.15.12:0/2104911097 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f441a59a0 con 0x7f6f44075540 2024-09-11T00:06:31.458 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:31.455+0000 7f6f43e00640 1 --2- 172.21.15.12:0/2104911097 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6f44113b20 0x7f6f441a51d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T00:06:31.458 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:31.455+0000 7f6f3aa00640 1 --2- 172.21.15.12:0/2104911097 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6f44075e60 0x7f6f4419fe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:06:31.458 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:31.455+0000 7f6f43400640 1 --2- 172.21.15.12:0/2104911097 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6f44075540 0x7f6f4419f8f0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f6f34005ab0 tx=0x7f6f34003c10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:06:31.459 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:31.455+0000 7f6f40c00640 1 -- 172.21.15.12:0/2104911097 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f3403f070 con 0x7f6f44075540 2024-09-11T00:06:31.459 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:31.455+0000 7f6f40c00640 1 -- 172.21.15.12:0/2104911097 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6f34031b60 con 0x7f6f44075540 2024-09-11T00:06:31.459 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:31.455+0000 7f6f4a000640 1 -- 172.21.15.12:0/2104911097 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6f441a5c30 con 0x7f6f44075540 2024-09-11T00:06:31.459 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:31.455+0000 7f6f40c00640 1 -- 172.21.15.12:0/2104911097 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f34035030 con 0x7f6f44075540 2024-09-11T00:06:31.459 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:31.456+0000 7f6f4a000640 1 -- 172.21.15.12:0/2104911097 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6f441ac350 con 0x7f6f44075540 2024-09-11T00:06:31.461 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:31.459+0000 7f6f4cac1740 1 Processor -- start 2024-09-11T00:06:31.461 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:31.459+0000 7f6f4cac1740 1 -- start start 2024-09-11T00:06:31.462 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:31.459+0000 7f6f40c00640 1 -- 172.21.15.12:0/2104911097 <== mon.1 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7f6f3403b460 con 0x7f6f44075540 2024-09-11T00:06:31.462 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:31.459+0000 7f6f4cac1740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556898020240 0x556898020620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:06:31.462 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:31.459+0000 7f6f4cac1740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556898020b60 0x55689802c5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:06:31.462 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:31.459+0000 7f6f4cac1740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55689802caf0 0x55689802eee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:06:31.462 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:31.459+0000 7f6f4cac1740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x556897f97d50 con 0x55689802caf0 2024-09-11T00:06:31.462 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:31.459+0000 7f6f4cac1740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x556897f87180 con 0x556898020b60 2024-09-11T00:06:31.462 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:31.459+0000 7f6f4cac1740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x556897f8af90 con 0x556898020240 2024-09-11T00:06:31.462 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:31.460+0000 7f6f43e00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556898020240 0x556898020620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:06:31.462 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:31.460+0000 7f6f3aa00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55689802caf0 0x55689802eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:06:31.463 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:31.460+0000 7f6f43e00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556898020240 0x556898020620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.12:47062/0 (socket says 172.21.15.12:47062) 2024-09-11T00:06:31.463 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:31.460+0000 7f6f43e00640 1 -- 172.21.15.12:0/529566398 learned_addr learned my addr 172.21.15.12:0/529566398 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:06:31.463 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:31.460+0000 7f6f43400640 1 --2- 172.21.15.12:0/529566398 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556898020b60 0x55689802c5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:06:31.463 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:31.460+0000 7f6f43e00640 1 -- 172.21.15.12:0/529566398 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556898020b60 msgr2=0x55689802c5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:06:31.463 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:31.460+0000 7f6f43e00640 1 --2- 172.21.15.12:0/529566398 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556898020b60 0x55689802c5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:31.463 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:31.460+0000 7f6f43e00640 1 -- 172.21.15.12:0/529566398 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55689802caf0 msgr2=0x55689802eee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:06:31.463 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:31.460+0000 7f6f43e00640 1 --2- 172.21.15.12:0/529566398 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55689802caf0 0x55689802eee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:31.463 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:31.460+0000 7f6f43e00640 1 -- 172.21.15.12:0/529566398 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556897f51a70 con 0x556898020240 2024-09-11T00:06:31.465 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:31.460+0000 7f6f43e00640 1 --2- 172.21.15.12:0/529566398 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556898020240 0x556898020620 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f6f3c010480 tx=0x7f6f3c010940 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c355a8e94c356734 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:06:31.465 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:31.461+0000 7f6f33e00640 1 -- 172.21.15.12:0/529566398 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f3c017070 con 0x556898020240 2024-09-11T00:06:31.466 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:31.461+0000 7f6f33e00640 1 -- 172.21.15.12:0/529566398 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6f3c011c10 con 0x556898020240 2024-09-11T00:06:31.466 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:31.461+0000 7f6f33e00640 1 -- 172.21.15.12:0/529566398 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f3c01bcd0 con 0x556898020240 2024-09-11T00:06:31.466 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:31.461+0000 7f6f40c00640 1 --2- 172.21.15.12:0/2104911097 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f6f280b5360 0x7f6f280b7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:06:31.466 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:31.461+0000 7f6f40c00640 1 -- 172.21.15.12:0/2104911097 <== mon.1 v2:172.21.15.178:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9208+0+0 (secure 0 0 0) 0x7f6f341069e0 con 0x7f6f44075540 2024-09-11T00:06:31.466 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:31.461+0000 7f6f43400640 1 --2- 172.21.15.12:0/2104911097 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f6f280b5360 0x7f6f280b7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:06:31.466 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:31.461+0000 7f6f4cac1740 1 -- 172.21.15.12:0/529566398 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556898020240 msgr2=0x556898020620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:06:31.466 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:31.461+0000 7f6f4cac1740 1 --2- 172.21.15.12:0/529566398 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556898020240 0x556898020620 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f6f3c010480 tx=0x7f6f3c010940 comp rx=0 tx=0).stop 2024-09-11T00:06:31.466 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:31.462+0000 7f6f4cac1740 1 -- 172.21.15.12:0/529566398 shutdown_connections 2024-09-11T00:06:31.466 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:31.462+0000 7f6f4cac1740 1 --2- 172.21.15.12:0/529566398 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55689802caf0 0x55689802eee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:31.466 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:31.462+0000 7f6f4cac1740 1 --2- 172.21.15.12:0/529566398 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556898020b60 0x55689802c5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:31.466 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:31.462+0000 7f6f4cac1740 1 --2- 172.21.15.12:0/529566398 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556898020240 0x556898020620 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:31.466 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:31.462+0000 7f6f4cac1740 1 -- 172.21.15.12:0/529566398 >> 172.21.15.12:0/529566398 conn(0x55689801e020 msgr2=0x55689801f210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:06:31.467 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:31.462+0000 7f6f4cac1740 1 -- 172.21.15.12:0/529566398 shutdown_connections 2024-09-11T00:06:31.467 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:31.462+0000 7f6f4cac1740 1 -- 172.21.15.12:0/529566398 wait complete. 2024-09-11T00:06:31.467 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:31.462+0000 7f6f43400640 1 --2- 172.21.15.12:0/2104911097 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f6f280b5360 0x7f6f280b7820 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f6f340c9340 tx=0x7f6f3410c480 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:06:31.467 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:31.462+0000 7f6f4cac1740 1 -- start start 2024-09-11T00:06:31.467 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:31.463+0000 7f6f4cac1740 1 Processor -- start 2024-09-11T00:06:31.467 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:31.463+0000 7f6f4cac1740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556898020240 0x556898029a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:06:31.467 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:31.463+0000 7f6f4cac1740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556898020b60 0x556898029fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:06:31.467 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:31.463+0000 7f6f4cac1740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55689802caf0 0x55689802a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:06:31.467 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:31.463+0000 7f6f4cac1740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x556897de85f0 con 0x556898020240 2024-09-11T00:06:31.467 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:31.463+0000 7f6f4cac1740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x556897f22460 con 0x556898020b60 2024-09-11T00:06:31.467 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:31.463+0000 7f6f4cac1740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55689802f840 con 0x55689802caf0 2024-09-11T00:06:31.467 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:31.463+0000 7f6f4cac1740 10 client.0 _ll_register_callbacks cb 0x556897f32210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:06:31.467 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:31.463+0000 7f6f43e00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556898020240 0x556898029a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:06:31.468 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:31.463+0000 7f6f3aa00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556898020b60 0x556898029fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:06:31.468 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:31.464+0000 7f6f43e00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556898020240 0x556898029a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:43444/0 (socket says 172.21.15.12:43444) 2024-09-11T00:06:31.468 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:31.464+0000 7f6f43e00640 1 -- 172.21.15.12:0/490485231 learned_addr learned my addr 172.21.15.12:0/490485231 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:06:31.468 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:31.464+0000 7f6f43e00640 1 --2- 172.21.15.12:0/490485231 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55689802caf0 0x55689802a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:06:31.470 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:31.464+0000 7f6f3aa00640 1 -- 172.21.15.12:0/490485231 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55689802caf0 msgr2=0x55689802a4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:06:31.470 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:31.464+0000 7f6f3aa00640 1 --2- 172.21.15.12:0/490485231 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55689802caf0 0x55689802a4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:31.471 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:31.464+0000 7f6f3aa00640 1 -- 172.21.15.12:0/490485231 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556898020240 msgr2=0x556898029a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:06:31.471 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:31.464+0000 7f6f3aa00640 1 --2- 172.21.15.12:0/490485231 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556898020240 0x556898029a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:31.471 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:31.464+0000 7f6f3aa00640 1 -- 172.21.15.12:0/490485231 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556897ce5540 con 0x556898020b60 2024-09-11T00:06:31.471 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:31.464+0000 7f6f43e00640 1 --2- 172.21.15.12:0/490485231 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556898020240 0x556898029a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:06:31.471 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:31.464+0000 7f6f3aa00640 1 --2- 172.21.15.12:0/490485231 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556898020b60 0x556898029fb0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f6f441160e0 tx=0x7f6f2c00f790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bf7a08ee4bdb720a server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:06:31.471 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:31.464+0000 7f6f30c00640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-09-11T00:06:31.471 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:31.465+0000 7f6f30c00640 1 -- 172.21.15.12:0/490485231 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f2c012110 con 0x556898020b60 2024-09-11T00:06:31.471 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:31.465+0000 7f6f4cac1740 1 -- 172.21.15.12:0/490485231 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x556897f24a70 con 0x556898020b60 2024-09-11T00:06:31.471 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:31.465+0000 7f6f30c00640 1 -- 172.21.15.12:0/490485231 <== mon.2 v2:172.21.15.165:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 9208+0+0 (secure 0 0 0) 0x7f6f2c004050 con 0x556898020b60 2024-09-11T00:06:31.471 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:31.465+0000 7f6f30c00640 1 -- 172.21.15.12:0/490485231 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f6f2c025050 con 0x556898020b60 2024-09-11T00:06:31.471 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:31.465+0000 7f6f4cac1740 10 client.25274 fetch_fsmap learned FSMap version 28 2024-09-11T00:06:31.471 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:31.465+0000 7f6f4cac1740 1 -- 172.21.15.12:0/490485231 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x556897f3f6f0 con 0x556898020b60 2024-09-11T00:06:31.472 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:31.465+0000 7f6f30c00640 1 -- 172.21.15.12:0/490485231 <== mon.2 v2:172.21.15.165:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x556897f24a70 con 0x556898020b60 2024-09-11T00:06:31.472 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:31.465+0000 7f6f30c00640 1 -- 172.21.15.12:0/490485231 <== mon.2 v2:172.21.15.165:3300/0 5 ==== fsmap(e 28) ==== 8370+0+0 (secure 0 0 0) 0x7f6f2c010430 con 0x556898020b60 2024-09-11T00:06:31.472 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:31.466+0000 7f6f4cac1740 10 client.25274 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T00:06:31.472 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:31.466+0000 7f6f4cac1740 10 client.25274 resolve_mds: resolved name 'e' to daemon mds.e 2024-09-11T00:06:31.472 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:31.466+0000 7f6f4cac1740 20 client.25274 populate_metadata read hostname 'smithi012' 2024-09-11T00:06:31.472 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:31.466+0000 7f6f4cac1740 1 --2- 172.21.15.12:0/490485231 >> [v2:172.21.15.165:6834/2425395274,v1:172.21.15.165:6835/2425395274] conn(0x55689803d8a0 0x55689803ed10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:06:31.472 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:31.466+0000 7f6f4cac1740 4 client.25274 mds_command: new command op to 24614 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T00:06:31.472 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:31.466+0000 7f6f4cac1740 1 -- 172.21.15.12:0/490485231 --> [v2:172.21.15.165:6834/2425395274,v1:172.21.15.165:6835/2425395274] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556897f99ca0 con 0x55689803d8a0 2024-09-11T00:06:31.472 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:31.466+0000 7f6f43e00640 1 --2- 172.21.15.12:0/490485231 >> [v2:172.21.15.165:6834/2425395274,v1:172.21.15.165:6835/2425395274] conn(0x55689803d8a0 0x55689803ed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:06:31.472 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:31.467+0000 7f6f43e00640 1 --2- 172.21.15.12:0/490485231 >> [v2:172.21.15.165:6834/2425395274,v1:172.21.15.165:6835/2425395274] conn(0x55689803d8a0 0x55689803ed10 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=25801697e296b80e server_cookie=473afb6345b1a347 in_seq=0 out_seq=0 2024-09-11T00:06:31.472 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:31.467+0000 7f6f30c00640 10 client.25274 ms_handle_connect on v2:172.21.15.165:6834/2425395274 2024-09-11T00:06:31.472 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:31.468+0000 7f6f30c00640 1 -- 172.21.15.12:0/490485231 <== mds.4 v2:172.21.15.165:6834/2425395274 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x556897f99ca0 con 0x55689803d8a0 2024-09-11T00:06:31.472 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:31.468+0000 7f6f30c00640 10 client.25274 handle_command_reply: tid=0 2024-09-11T00:06:31.473 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:31.468+0000 7f6f4cac1740 1 client.25274 shutdown 2024-09-11T00:06:31.473 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:31.468+0000 7f6f4cac1740 1 -- 172.21.15.12:0/490485231 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556898020b60 msgr2=0x556898029fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:06:31.473 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:31.468+0000 7f6f4cac1740 1 --2- 172.21.15.12:0/490485231 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556898020b60 0x556898029fb0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f6f441160e0 tx=0x7f6f2c00f790 comp rx=0 tx=0).stop 2024-09-11T00:06:31.473 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:31.468+0000 7f6f4cac1740 1 -- 172.21.15.12:0/490485231 shutdown_connections 2024-09-11T00:06:31.473 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:31.468+0000 7f6f4cac1740 1 --2- 172.21.15.12:0/490485231 >> [v2:172.21.15.165:6834/2425395274,v1:172.21.15.165:6835/2425395274] conn(0x55689803d8a0 0x55689803ed10 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:31.473 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:31.468+0000 7f6f4cac1740 1 --2- 172.21.15.12:0/490485231 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55689802caf0 0x55689802a4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:31.473 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:31.468+0000 7f6f4cac1740 1 --2- 172.21.15.12:0/490485231 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556898020b60 0x556898029fb0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:31.473 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:31.468+0000 7f6f4cac1740 1 --2- 172.21.15.12:0/490485231 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556898020240 0x556898029a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:31.473 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:31.468+0000 7f6f4cac1740 1 -- 172.21.15.12:0/490485231 >> 172.21.15.12:0/490485231 conn(0x55689801e020 msgr2=0x556898023c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:06:31.473 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:31.468+0000 7f6f30c00640 0 client.25274 ms_handle_reset on v2:172.21.15.165:6834/2425395274 2024-09-11T00:06:31.473 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:31.468+0000 7f6f4cac1740 1 -- 172.21.15.12:0/490485231 shutdown_connections 2024-09-11T00:06:31.473 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:31.468+0000 7f6f4cac1740 1 -- 172.21.15.12:0/490485231 wait complete. 2024-09-11T00:06:31.473 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:31.468+0000 7f6f4cac1740 20 client.25274 trim_cache size 0 max 16384 2024-09-11T00:06:31.489 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:31.488+0000 7f6f4cac1740 1 Processor -- start 2024-09-11T00:06:31.489 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:31.488+0000 7f6f4cac1740 1 -- start start 2024-09-11T00:06:31.490 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:31.488+0000 7f6f4cac1740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556898020240 0x556898024400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:06:31.491 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:31.488+0000 7f6f4cac1740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556898020b60 0x55689801f0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:06:31.491 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:31.488+0000 7f6f4cac1740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55689802caf0 0x55689801f600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:06:31.491 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:31.488+0000 7f6f4cac1740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55689803d050 con 0x556898020b60 2024-09-11T00:06:31.491 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:31.488+0000 7f6f4cac1740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55689803d720 con 0x55689802caf0 2024-09-11T00:06:31.491 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:31.488+0000 7f6f4cac1740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55689803ddf0 con 0x556898020240 2024-09-11T00:06:31.491 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:31.488+0000 7f6f3aa00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556898020b60 0x55689801f0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:06:31.491 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:31.488+0000 7f6f3aa00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556898020b60 0x55689801f0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:43452/0 (socket says 172.21.15.12:43452) 2024-09-11T00:06:31.491 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:31.488+0000 7f6f3aa00640 1 -- 172.21.15.12:0/2260437726 learned_addr learned my addr 172.21.15.12:0/2260437726 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:06:31.491 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:31.488+0000 7f6f43e00640 1 --2- 172.21.15.12:0/2260437726 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556898020240 0x556898024400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:06:31.491 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:31.489+0000 7f6f43e00640 1 --2- 172.21.15.12:0/2260437726 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55689802caf0 0x55689801f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:06:31.491 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:31.489+0000 7f6f3aa00640 1 -- 172.21.15.12:0/2260437726 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556898020240 msgr2=0x556898024400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:06:31.491 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:31.489+0000 7f6f3aa00640 1 --2- 172.21.15.12:0/2260437726 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556898020240 0x556898024400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:31.492 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:31.489+0000 7f6f3aa00640 1 -- 172.21.15.12:0/2260437726 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55689802caf0 msgr2=0x55689801f600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:06:31.492 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:31.489+0000 7f6f3aa00640 1 --2- 172.21.15.12:0/2260437726 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55689802caf0 0x55689801f600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:31.492 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:31.489+0000 7f6f3aa00640 1 -- 172.21.15.12:0/2260437726 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55689803b320 con 0x556898020b60 2024-09-11T00:06:31.492 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:31.489+0000 7f6f43e00640 1 --2- 172.21.15.12:0/2260437726 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55689802caf0 0x55689801f600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T00:06:31.494 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:31.489+0000 7f6f3aa00640 1 --2- 172.21.15.12:0/2260437726 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556898020b60 0x55689801f0c0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f6f2c010850 tx=0x7f6f2c0123a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf4e845ffa72352c server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:06:31.494 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:31.489+0000 7f6f33400640 1 -- 172.21.15.12:0/2260437726 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f2c02e070 con 0x556898020b60 2024-09-11T00:06:31.494 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:31.489+0000 7f6f33400640 1 -- 172.21.15.12:0/2260437726 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6f2c005560 con 0x556898020b60 2024-09-11T00:06:31.494 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:31.489+0000 7f6f33400640 1 -- 172.21.15.12:0/2260437726 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f2c0035f0 con 0x556898020b60 2024-09-11T00:06:31.494 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:31.490+0000 7f6f4cac1740 1 -- 172.21.15.12:0/2260437726 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556898020b60 msgr2=0x55689801f0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:06:31.494 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:31.490+0000 7f6f4cac1740 1 --2- 172.21.15.12:0/2260437726 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556898020b60 0x55689801f0c0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f6f2c010850 tx=0x7f6f2c0123a0 comp rx=0 tx=0).stop 2024-09-11T00:06:31.494 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:31.490+0000 7f6f4cac1740 1 -- 172.21.15.12:0/2260437726 shutdown_connections 2024-09-11T00:06:31.494 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:31.490+0000 7f6f4cac1740 1 --2- 172.21.15.12:0/2260437726 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55689802caf0 0x55689801f600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:31.494 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:31.490+0000 7f6f4cac1740 1 --2- 172.21.15.12:0/2260437726 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556898020b60 0x55689801f0c0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:31.494 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:31.490+0000 7f6f4cac1740 1 --2- 172.21.15.12:0/2260437726 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556898020240 0x556898024400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:31.494 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:31.490+0000 7f6f4cac1740 1 -- 172.21.15.12:0/2260437726 >> 172.21.15.12:0/2260437726 conn(0x55689801e020 msgr2=0x556898018ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:06:31.495 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:31.490+0000 7f6f4cac1740 1 -- 172.21.15.12:0/2260437726 shutdown_connections 2024-09-11T00:06:31.495 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:31.490+0000 7f6f4cac1740 1 -- 172.21.15.12:0/2260437726 wait complete. 2024-09-11T00:06:31.495 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:31.491+0000 7f6f4cac1740 1 -- start start 2024-09-11T00:06:31.495 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:31.491+0000 7f6f4cac1740 1 Processor -- start 2024-09-11T00:06:31.495 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:31.491+0000 7f6f4cac1740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556898020240 0x556898041660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:06:31.495 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:31.491+0000 7f6f4cac1740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556898020b60 0x556898041ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:06:31.495 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:31.491+0000 7f6f4cac1740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55689802caf0 0x556898046bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:06:31.495 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:31.491+0000 7f6f4cac1740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55689803eb90 con 0x556898020b60 2024-09-11T00:06:31.495 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:31.491+0000 7f6f4cac1740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55689803e4c0 con 0x55689802caf0 2024-09-11T00:06:31.495 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:31.491+0000 7f6f4cac1740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55689803f260 con 0x556898020240 2024-09-11T00:06:31.495 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:31.491+0000 7f6f4cac1740 10 client.0 _ll_register_callbacks cb 0x7f6f2c012930 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:06:31.495 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:31.492+0000 7f6f3aa00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556898020b60 0x556898041ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:06:31.496 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:31.492+0000 7f6f43e00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556898020240 0x556898041660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:06:31.496 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:31.492+0000 7f6f3aa00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556898020b60 0x556898041ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:43462/0 (socket says 172.21.15.12:43462) 2024-09-11T00:06:31.496 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:31.492+0000 7f6f3aa00640 1 -- 172.21.15.12:0/2076189311 learned_addr learned my addr 172.21.15.12:0/2076189311 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:06:31.496 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:31.492+0000 7f6f43e00640 1 --2- 172.21.15.12:0/2076189311 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55689802caf0 0x556898046bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:06:31.498 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:31.492+0000 7f6f3aa00640 1 -- 172.21.15.12:0/2076189311 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556898020240 msgr2=0x556898041660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:06:31.498 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:31.492+0000 7f6f3aa00640 1 --2- 172.21.15.12:0/2076189311 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556898020240 0x556898041660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:31.498 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:31.492+0000 7f6f3aa00640 1 -- 172.21.15.12:0/2076189311 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55689802caf0 msgr2=0x556898046bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:06:31.498 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:31.492+0000 7f6f3aa00640 1 --2- 172.21.15.12:0/2076189311 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55689802caf0 0x556898046bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:31.499 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:31.492+0000 7f6f3aa00640 1 -- 172.21.15.12:0/2076189311 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556898047130 con 0x556898020b60 2024-09-11T00:06:31.499 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:31.492+0000 7f6f43e00640 1 --2- 172.21.15.12:0/2076189311 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55689802caf0 0x556898046bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T00:06:31.499 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:31.492+0000 7f6f3aa00640 1 --2- 172.21.15.12:0/2076189311 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556898020b60 0x556898041ba0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f6f2c0093b0 tx=0x7f6f2c005d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=465001bec4064b49 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:06:31.499 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:31.492+0000 7f6f30c00640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-09-11T00:06:31.499 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:31.493+0000 7f6f30c00640 1 -- 172.21.15.12:0/2076189311 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f2c02e070 con 0x556898020b60 2024-09-11T00:06:31.499 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:31.493+0000 7f6f30c00640 1 -- 172.21.15.12:0/2076189311 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 9208+0+0 (secure 0 0 0) 0x7f6f2c0050c0 con 0x556898020b60 2024-09-11T00:06:31.499 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:31.493+0000 7f6f4cac1740 1 -- 172.21.15.12:0/2076189311 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6f2c005560 con 0x556898020b60 2024-09-11T00:06:31.499 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:31.493+0000 7f6f30c00640 1 -- 172.21.15.12:0/2076189311 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f6f2c03b050 con 0x556898020b60 2024-09-11T00:06:31.499 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:31.493+0000 7f6f4cac1740 10 client.15240 fetch_fsmap learned FSMap version 28 2024-09-11T00:06:31.499 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:31.493+0000 7f6f4cac1740 1 -- 172.21.15.12:0/2076189311 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5568980aaa40 con 0x556898020b60 2024-09-11T00:06:31.499 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:31.493+0000 7f6f30c00640 1 -- 172.21.15.12:0/2076189311 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6f2c005560 con 0x556898020b60 2024-09-11T00:06:31.499 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:31.494+0000 7f6f30c00640 1 -- 172.21.15.12:0/2076189311 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 28) ==== 8370+0+0 (secure 0 0 0) 0x7f6f2c003bc0 con 0x556898020b60 2024-09-11T00:06:31.499 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:31.494+0000 7f6f4cac1740 10 client.15240 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T00:06:31.500 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:31.494+0000 7f6f4cac1740 10 client.15240 resolve_mds: resolved name 'e' to daemon mds.e 2024-09-11T00:06:31.500 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:31.494+0000 7f6f4cac1740 20 client.15240 populate_metadata read hostname 'smithi012' 2024-09-11T00:06:31.500 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:31.494+0000 7f6f4cac1740 1 --2- 172.21.15.12:0/2076189311 >> [v2:172.21.15.165:6834/2425395274,v1:172.21.15.165:6835/2425395274] conn(0x5568980af250 0x5568980adf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:06:31.500 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:31.494+0000 7f6f4cac1740 4 client.15240 mds_command: new command op to 24614 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-11T00:06:31.500 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:31.494+0000 7f6f4cac1740 1 -- 172.21.15.12:0/2076189311 --> [v2:172.21.15.165:6834/2425395274,v1:172.21.15.165:6835/2425395274] -- command(tid 0: {"prefix": "status"}) -- 0x556897f95750 con 0x5568980af250 2024-09-11T00:06:31.500 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:31.494+0000 7f6f43e00640 1 --2- 172.21.15.12:0/2076189311 >> [v2:172.21.15.165:6834/2425395274,v1:172.21.15.165:6835/2425395274] conn(0x5568980af250 0x5568980adf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:06:31.500 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:31.495+0000 7f6f43e00640 1 --2- 172.21.15.12:0/2076189311 >> [v2:172.21.15.165:6834/2425395274,v1:172.21.15.165:6835/2425395274] conn(0x5568980af250 0x5568980adf60 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=1be6b79aa0a5c214 server_cookie=ccec67a635634aac in_seq=0 out_seq=0 2024-09-11T00:06:31.500 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:31.495+0000 7f6f30c00640 10 client.15240 ms_handle_connect on v2:172.21.15.165:6834/2425395274 2024-09-11T00:06:31.500 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:31.495+0000 7f6f30c00640 1 -- 172.21.15.12:0/2076189311 <== mds.4 v2:172.21.15.165:6834/2425395274 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x556897f95750 con 0x5568980af250 2024-09-11T00:06:31.500 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:31.495+0000 7f6f30c00640 10 client.15240 handle_command_reply: tid=0 2024-09-11T00:06:31.500 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:31.495+0000 7f6f4cac1740 1 client.15240 shutdown 2024-09-11T00:06:31.501 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:31.495+0000 7f6f4cac1740 1 -- 172.21.15.12:0/2076189311 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556898020b60 msgr2=0x556898041ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:06:31.501 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:31.495+0000 7f6f4cac1740 1 --2- 172.21.15.12:0/2076189311 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556898020b60 0x556898041ba0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f6f2c0093b0 tx=0x7f6f2c005d30 comp rx=0 tx=0).stop 2024-09-11T00:06:31.501 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:31.496+0000 7f6f4cac1740 1 -- 172.21.15.12:0/2076189311 shutdown_connections 2024-09-11T00:06:31.501 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:31.496+0000 7f6f4cac1740 1 --2- 172.21.15.12:0/2076189311 >> [v2:172.21.15.165:6834/2425395274,v1:172.21.15.165:6835/2425395274] conn(0x5568980af250 0x5568980adf60 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:31.501 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:31.496+0000 7f6f4cac1740 1 --2- 172.21.15.12:0/2076189311 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55689802caf0 0x556898046bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:31.501 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:31.496+0000 7f6f4cac1740 1 --2- 172.21.15.12:0/2076189311 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556898020b60 0x556898041ba0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:31.501 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:31.496+0000 7f6f4cac1740 1 --2- 172.21.15.12:0/2076189311 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556898020240 0x556898041660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:31.501 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:31.496+0000 7f6f30c00640 0 client.15240 ms_handle_reset on v2:172.21.15.165:6834/2425395274 2024-09-11T00:06:31.501 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:31.496+0000 7f6f4cac1740 1 -- 172.21.15.12:0/2076189311 >> 172.21.15.12:0/2076189311 conn(0x55689801e020 msgr2=0x556898028270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:06:31.501 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:31.496+0000 7f6f4cac1740 1 -- 172.21.15.12:0/2076189311 shutdown_connections 2024-09-11T00:06:31.501 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:31.496+0000 7f6f4cac1740 1 -- 172.21.15.12:0/2076189311 wait complete. 2024-09-11T00:06:31.501 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:31.496+0000 7f6f4cac1740 20 client.15240 trim_cache size 0 max 16384 2024-09-11T00:06:31.502 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-09-11T00:06:31.502 INFO:teuthology.orchestra.run.smithi012.stdout: "cluster_fsid": "ff3ad74a-6fcf-11ef-bcea-c7b262605968", 2024-09-11T00:06:31.502 INFO:teuthology.orchestra.run.smithi012.stdout: "whoami": 4, 2024-09-11T00:06:31.502 INFO:teuthology.orchestra.run.smithi012.stdout: "id": 24614, 2024-09-11T00:06:31.502 INFO:teuthology.orchestra.run.smithi012.stdout: "want_state": "up:active", 2024-09-11T00:06:31.502 INFO:teuthology.orchestra.run.smithi012.stdout: "state": "up:active", 2024-09-11T00:06:31.502 INFO:teuthology.orchestra.run.smithi012.stdout: "fs_name": "cephfs", 2024-09-11T00:06:31.502 INFO:teuthology.orchestra.run.smithi012.stdout: "rank_uptime": 102.25718273299999, 2024-09-11T00:06:31.502 INFO:teuthology.orchestra.run.smithi012.stdout: "mdsmap_epoch": 28, 2024-09-11T00:06:31.502 INFO:teuthology.orchestra.run.smithi012.stdout: "osdmap_epoch": 85, 2024-09-11T00:06:31.502 INFO:teuthology.orchestra.run.smithi012.stdout: "osdmap_epoch_barrier": 81, 2024-09-11T00:06:31.503 INFO:teuthology.orchestra.run.smithi012.stdout: "uptime": 179.80114209600001 2024-09-11T00:06:31.503 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-09-11T00:06:31.503 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:31.496+0000 7f6f4a000640 1 -- 172.21.15.12:0/2104911097 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f6f280b5360 msgr2=0x7f6f280b7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:06:31.503 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:31.496+0000 7f6f4a000640 1 --2- 172.21.15.12:0/2104911097 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f6f280b5360 0x7f6f280b7820 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f6f340c9340 tx=0x7f6f3410c480 comp rx=0 tx=0).stop 2024-09-11T00:06:31.503 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:31.496+0000 7f6f4a000640 1 -- 172.21.15.12:0/2104911097 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6f44075540 msgr2=0x7f6f4419f8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:06:31.503 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:31.496+0000 7f6f4a000640 1 --2- 172.21.15.12:0/2104911097 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6f44075540 0x7f6f4419f8f0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f6f34005ab0 tx=0x7f6f34003c10 comp rx=0 tx=0).stop 2024-09-11T00:06:31.504 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:31.497+0000 7f6f4a000640 1 -- 172.21.15.12:0/2104911097 shutdown_connections 2024-09-11T00:06:31.504 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:31.497+0000 7f6f4a000640 1 --2- 172.21.15.12:0/2104911097 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f6f280b5360 0x7f6f280b7820 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:31.504 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:31.497+0000 7f6f4a000640 1 --2- 172.21.15.12:0/2104911097 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6f44113b20 0x7f6f441a51d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:31.504 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:31.497+0000 7f6f4a000640 1 --2- 172.21.15.12:0/2104911097 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6f44075e60 0x7f6f4419fe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:31.504 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:31.497+0000 7f6f4a000640 1 --2- 172.21.15.12:0/2104911097 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6f44075540 0x7f6f4419f8f0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:31.504 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:31.497+0000 7f6f4a000640 1 -- 172.21.15.12:0/2104911097 >> 172.21.15.12:0/2104911097 conn(0x7f6f440fe830 msgr2=0x7f6f440ff430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:06:31.504 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:31.497+0000 7f6f4a000640 1 -- 172.21.15.12:0/2104911097 shutdown_connections 2024-09-11T00:06:31.504 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:31.497+0000 7f6f4a000640 1 -- 172.21.15.12:0/2104911097 wait complete. 2024-09-11T00:06:31.532 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:06:31 smithi165 ceph-mon[35356]: pgmap v381: 129 pgs: 129 active+clean; 647 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 511 B/s wr, 10 op/s; 11 B/s, 0 objects/s recovering 2024-09-11T00:06:31.580 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:06:31 smithi178 ceph-mon[35919]: pgmap v381: 129 pgs: 129 active+clean; 647 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 511 B/s wr, 10 op/s; 11 B/s, 0 objects/s recovering 2024-09-11T00:06:32.293 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid ff3ad74a-6fcf-11ef-bcea-c7b262605968 -- ceph tell mds.i status 2024-09-11T00:06:32.679 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:06:33.228 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:33.224+0000 7fc8f8c00640 1 -- 172.21.15.12:0/815273071 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc8f410a9b0 msgr2=0x7fc8f410adf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:06:33.228 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:33.224+0000 7fc8f8c00640 1 --2- 172.21.15.12:0/815273071 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc8f410a9b0 0x7fc8f410adf0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fc8dc009900 tx=0x7fc8dc02a3f0 comp rx=0 tx=0).stop 2024-09-11T00:06:33.228 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:33.224+0000 7fc8f8c00640 1 -- 172.21.15.12:0/815273071 shutdown_connections 2024-09-11T00:06:33.228 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:33.224+0000 7fc8f8c00640 1 --2- 172.21.15.12:0/815273071 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc8f410b330 0x7fc8f4111b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:33.228 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:33.224+0000 7fc8f8c00640 1 --2- 172.21.15.12:0/815273071 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc8f410a9b0 0x7fc8f410adf0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:33.228 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:33.224+0000 7fc8f8c00640 1 --2- 172.21.15.12:0/815273071 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc8f4104fa0 0x7fc8f4105380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:33.229 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:33.224+0000 7fc8f8c00640 1 -- 172.21.15.12:0/815273071 >> 172.21.15.12:0/815273071 conn(0x7fc8f4100a50 msgr2=0x7fc8f4102e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:06:33.229 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:33.224+0000 7fc8f8c00640 1 -- 172.21.15.12:0/815273071 shutdown_connections 2024-09-11T00:06:33.229 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:33.224+0000 7fc8f8c00640 1 -- 172.21.15.12:0/815273071 wait complete. 2024-09-11T00:06:33.229 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:33.225+0000 7fc8f8c00640 1 Processor -- start 2024-09-11T00:06:33.229 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:33.225+0000 7fc8f8c00640 1 -- start start 2024-09-11T00:06:33.229 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:33.226+0000 7fc8f8c00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc8f4104fa0 0x7fc8f419b4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:06:33.229 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:33.226+0000 7fc8f8c00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc8f410a9b0 0x7fc8f419b9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:06:33.229 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:33.226+0000 7fc8f8c00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc8f410b330 0x7fc8f41a0d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:06:33.230 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:33.226+0000 7fc8f8c00640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fc8f4114f30 con 0x7fc8f4104fa0 2024-09-11T00:06:33.230 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:33.226+0000 7fc8f8c00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fc8f4114db0 con 0x7fc8f410b330 2024-09-11T00:06:33.230 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:33.226+0000 7fc8f8c00640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fc8f41150b0 con 0x7fc8f410a9b0 2024-09-11T00:06:33.231 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:33.226+0000 7fc8f1e00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc8f4104fa0 0x7fc8f419b4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:06:33.231 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:33.226+0000 7fc8ebe00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc8f410a9b0 0x7fc8f419b9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:06:33.231 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:33.226+0000 7fc8f2800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc8f410b330 0x7fc8f41a0d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:06:33.232 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:33.226+0000 7fc8ebe00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc8f410a9b0 0x7fc8f419b9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.12:47098/0 (socket says 172.21.15.12:47098) 2024-09-11T00:06:33.232 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:33.226+0000 7fc8f2800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc8f410b330 0x7fc8f41a0d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.12:51790/0 (socket says 172.21.15.12:51790) 2024-09-11T00:06:33.232 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:33.226+0000 7fc8ebe00640 1 -- 172.21.15.12:0/2005397494 learned_addr learned my addr 172.21.15.12:0/2005397494 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:06:33.232 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:33.227+0000 7fc8f1e00640 1 -- 172.21.15.12:0/2005397494 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc8f410a9b0 msgr2=0x7fc8f419b9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:06:33.232 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:33.227+0000 7fc8f1e00640 1 --2- 172.21.15.12:0/2005397494 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc8f410a9b0 0x7fc8f419b9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:33.232 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:33.227+0000 7fc8f1e00640 1 -- 172.21.15.12:0/2005397494 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc8f410b330 msgr2=0x7fc8f41a0d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:06:33.232 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:33.227+0000 7fc8f1e00640 1 --2- 172.21.15.12:0/2005397494 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc8f410b330 0x7fc8f41a0d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:33.233 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:33.227+0000 7fc8f1e00640 1 -- 172.21.15.12:0/2005397494 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8f41a1550 con 0x7fc8f4104fa0 2024-09-11T00:06:33.233 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:33.227+0000 7fc8f1e00640 1 --2- 172.21.15.12:0/2005397494 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc8f4104fa0 0x7fc8f419b4a0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fc8e400e9f0 tx=0x7fc8e400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:06:33.233 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:33.227+0000 7fc8eaa00640 1 -- 172.21.15.12:0/2005397494 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8e400cd80 con 0x7fc8f4104fa0 2024-09-11T00:06:33.233 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:33.227+0000 7fc8eaa00640 1 -- 172.21.15.12:0/2005397494 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc8e40035f0 con 0x7fc8f4104fa0 2024-09-11T00:06:33.233 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:33.227+0000 7fc8eaa00640 1 -- 172.21.15.12:0/2005397494 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8e4010670 con 0x7fc8f4104fa0 2024-09-11T00:06:33.233 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:33.227+0000 7fc8f8c00640 1 -- 172.21.15.12:0/2005397494 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc8f41a1840 con 0x7fc8f4104fa0 2024-09-11T00:06:33.233 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:33.228+0000 7fc8f8c00640 1 -- 172.21.15.12:0/2005397494 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8f41a7f00 con 0x7fc8f4104fa0 2024-09-11T00:06:33.234 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:33.231+0000 7fc8fb7ed740 1 Processor -- start 2024-09-11T00:06:33.234 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:33.231+0000 7fc8fb7ed740 1 -- start start 2024-09-11T00:06:33.234 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:33.231+0000 7fc8eaa00640 1 -- 172.21.15.12:0/2005397494 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7fc8e4004000 con 0x7fc8f4104fa0 2024-09-11T00:06:33.235 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:33.231+0000 7fc8fb7ed740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558ab7935240 0x558ab7935620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:06:33.235 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:33.231+0000 7fc8fb7ed740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558ab7935b60 0x558ab79415b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:06:33.235 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:33.231+0000 7fc8fb7ed740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558ab7941af0 0x558ab7943ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:06:33.235 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:33.231+0000 7fc8fb7ed740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x558ab78acd50 con 0x558ab7935b60 2024-09-11T00:06:33.235 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:33.231+0000 7fc8fb7ed740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x558ab789c180 con 0x558ab7941af0 2024-09-11T00:06:33.235 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:33.231+0000 7fc8fb7ed740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x558ab789ff90 con 0x558ab7935240 2024-09-11T00:06:33.235 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:33.232+0000 7fc8f1e00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558ab7935b60 0x558ab79415b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:06:33.235 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:33.232+0000 7fc8f1e00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558ab7935b60 0x558ab79415b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:43494/0 (socket says 172.21.15.12:43494) 2024-09-11T00:06:33.235 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:33.232+0000 7fc8f1e00640 1 -- 172.21.15.12:0/2327139553 learned_addr learned my addr 172.21.15.12:0/2327139553 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:06:33.236 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:33.232+0000 7fc8ebe00640 1 --2- 172.21.15.12:0/2327139553 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558ab7941af0 0x558ab7943ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:06:33.236 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:33.232+0000 7fc8f2800640 1 --2- 172.21.15.12:0/2327139553 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558ab7935240 0x558ab7935620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:06:33.236 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:33.232+0000 7fc8f1e00640 1 -- 172.21.15.12:0/2327139553 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558ab7935240 msgr2=0x558ab7935620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:06:33.236 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:33.232+0000 7fc8f1e00640 1 --2- 172.21.15.12:0/2327139553 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558ab7935240 0x558ab7935620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:33.236 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:33.232+0000 7fc8f1e00640 1 -- 172.21.15.12:0/2327139553 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558ab7941af0 msgr2=0x558ab7943ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:06:33.236 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:33.232+0000 7fc8f1e00640 1 --2- 172.21.15.12:0/2327139553 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558ab7941af0 0x558ab7943ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:33.236 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:33.232+0000 7fc8f1e00640 1 -- 172.21.15.12:0/2327139553 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558ab7866a70 con 0x558ab7935b60 2024-09-11T00:06:33.236 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:33.233+0000 7fc8f1e00640 1 --2- 172.21.15.12:0/2327139553 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558ab7935b60 0x558ab79415b0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fc8e40ee2c0 tx=0x7fc8e40ee780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c752a7e869d3dfa7 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:06:33.238 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:33.233+0000 7fc8e3000640 1 -- 172.21.15.12:0/2327139553 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8e40f9070 con 0x558ab7935b60 2024-09-11T00:06:33.239 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:33.233+0000 7fc8e3000640 1 -- 172.21.15.12:0/2327139553 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc8e40e84e0 con 0x558ab7935b60 2024-09-11T00:06:33.239 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:33.233+0000 7fc8e3000640 1 -- 172.21.15.12:0/2327139553 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8e40f4900 con 0x558ab7935b60 2024-09-11T00:06:33.239 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:33.233+0000 7fc8eaa00640 1 --2- 172.21.15.12:0/2005397494 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fc8d80b52e0 0x7fc8d80b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:06:33.239 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:33.234+0000 7fc8eaa00640 1 -- 172.21.15.12:0/2005397494 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9208+0+0 (secure 0 0 0) 0x7fc8e40e6280 con 0x7fc8f4104fa0 2024-09-11T00:06:33.239 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:33.234+0000 7fc8f2800640 1 --2- 172.21.15.12:0/2005397494 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fc8d80b52e0 0x7fc8d80b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:06:33.239 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:33.234+0000 7fc8fb7ed740 1 -- 172.21.15.12:0/2327139553 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558ab7935b60 msgr2=0x558ab79415b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:06:33.239 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:33.234+0000 7fc8fb7ed740 1 --2- 172.21.15.12:0/2327139553 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558ab7935b60 0x558ab79415b0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fc8e40ee2c0 tx=0x7fc8e40ee780 comp rx=0 tx=0).stop 2024-09-11T00:06:33.239 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:33.234+0000 7fc8fb7ed740 1 -- 172.21.15.12:0/2327139553 shutdown_connections 2024-09-11T00:06:33.239 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:33.234+0000 7fc8fb7ed740 1 --2- 172.21.15.12:0/2327139553 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558ab7941af0 0x558ab7943ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:33.239 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:33.234+0000 7fc8fb7ed740 1 --2- 172.21.15.12:0/2327139553 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558ab7935b60 0x558ab79415b0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:33.239 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:33.234+0000 7fc8fb7ed740 1 --2- 172.21.15.12:0/2327139553 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558ab7935240 0x558ab7935620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:33.239 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:33.234+0000 7fc8fb7ed740 1 -- 172.21.15.12:0/2327139553 >> 172.21.15.12:0/2327139553 conn(0x558ab7933020 msgr2=0x558ab7934210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:06:33.239 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:33.234+0000 7fc8fb7ed740 1 -- 172.21.15.12:0/2327139553 shutdown_connections 2024-09-11T00:06:33.240 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:33.234+0000 7fc8fb7ed740 1 -- 172.21.15.12:0/2327139553 wait complete. 2024-09-11T00:06:33.240 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:33.234+0000 7fc8f2800640 1 --2- 172.21.15.12:0/2005397494 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fc8d80b52e0 0x7fc8d80b77a0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fc8ec005fd0 tx=0x7fc8ec009600 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:06:33.240 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:33.235+0000 7fc8fb7ed740 1 -- start start 2024-09-11T00:06:33.240 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:33.235+0000 7fc8fb7ed740 1 Processor -- start 2024-09-11T00:06:33.240 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:33.235+0000 7fc8fb7ed740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558ab7935240 0x558ab793eb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:06:33.240 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:33.235+0000 7fc8fb7ed740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558ab7935b60 0x558ab793f060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:06:33.240 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:33.235+0000 7fc8fb7ed740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558ab7941af0 0x558ab793f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:06:33.240 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:33.236+0000 7fc8fb7ed740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x558ab76fd5f0 con 0x558ab7935240 2024-09-11T00:06:33.240 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:33.236+0000 7fc8fb7ed740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x558ab7837460 con 0x558ab7935b60 2024-09-11T00:06:33.241 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:33.236+0000 7fc8fb7ed740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x558ab7944840 con 0x558ab7941af0 2024-09-11T00:06:33.241 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:33.236+0000 7fc8fb7ed740 10 client.0 _ll_register_callbacks cb 0x558ab7847210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:06:33.241 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:33.236+0000 7fc8f1e00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558ab7935240 0x558ab793eb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:06:33.241 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:33.236+0000 7fc8f2800640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558ab7941af0 0x558ab793f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:06:33.241 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:33.236+0000 7fc8f1e00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558ab7935240 0x558ab793eb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:43498/0 (socket says 172.21.15.12:43498) 2024-09-11T00:06:33.241 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:33.236+0000 7fc8f1e00640 1 -- 172.21.15.12:0/437525472 learned_addr learned my addr 172.21.15.12:0/437525472 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:06:33.241 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:33.236+0000 7fc8ebe00640 1 --2- 172.21.15.12:0/437525472 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558ab7935b60 0x558ab793f060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:06:33.241 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:33.236+0000 7fc8f1e00640 1 -- 172.21.15.12:0/437525472 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558ab7941af0 msgr2=0x558ab793f5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:06:33.241 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:33.236+0000 7fc8f1e00640 1 --2- 172.21.15.12:0/437525472 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558ab7941af0 0x558ab793f5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:33.241 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:33.236+0000 7fc8f1e00640 1 -- 172.21.15.12:0/437525472 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558ab7935b60 msgr2=0x558ab793f060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:06:33.241 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:33.236+0000 7fc8f1e00640 1 --2- 172.21.15.12:0/437525472 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558ab7935b60 0x558ab793f060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:33.241 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:33.236+0000 7fc8f1e00640 1 -- 172.21.15.12:0/437525472 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558ab75fa540 con 0x558ab7935240 2024-09-11T00:06:33.241 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:33.237+0000 7fc8f1e00640 1 --2- 172.21.15.12:0/437525472 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558ab7935240 0x558ab793eb20 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fc8e40ac5a0 tx=0x7fc8e400c840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0c931d10eaa2289 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:06:33.241 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:33.237+0000 7fc8d7400640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-09-11T00:06:33.242 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:33.237+0000 7fc8d7400640 1 -- 172.21.15.12:0/437525472 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8e40a8c60 con 0x558ab7935240 2024-09-11T00:06:33.242 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:33.237+0000 7fc8d7400640 1 -- 172.21.15.12:0/437525472 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 9208+0+0 (secure 0 0 0) 0x7fc8e40037a0 con 0x558ab7935240 2024-09-11T00:06:33.242 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:33.237+0000 7fc8fb7ed740 1 -- 172.21.15.12:0/437525472 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x558ab7839a70 con 0x558ab7935240 2024-09-11T00:06:33.243 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:33.237+0000 7fc8d7400640 1 -- 172.21.15.12:0/437525472 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fc8e40abd80 con 0x558ab7935240 2024-09-11T00:06:33.244 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:33.237+0000 7fc8fb7ed740 10 client.15252 fetch_fsmap learned FSMap version 28 2024-09-11T00:06:33.244 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:33.237+0000 7fc8fb7ed740 1 -- 172.21.15.12:0/437525472 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x558ab78546f0 con 0x558ab7935240 2024-09-11T00:06:33.244 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:33.237+0000 7fc8d7400640 1 -- 172.21.15.12:0/437525472 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x558ab7839a70 con 0x558ab7935240 2024-09-11T00:06:33.244 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:33.238+0000 7fc8d7400640 1 -- 172.21.15.12:0/437525472 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 28) ==== 8370+0+0 (secure 0 0 0) 0x7fc8e4004320 con 0x558ab7935240 2024-09-11T00:06:33.244 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:33.238+0000 7fc8fb7ed740 10 client.15252 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T00:06:33.244 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:33.238+0000 7fc8fb7ed740 10 client.15252 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-11T00:06:33.244 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:33.238+0000 7fc8fb7ed740 20 client.15252 populate_metadata read hostname 'smithi012' 2024-09-11T00:06:33.244 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:33.238+0000 7fc8fb7ed740 1 --2- 172.21.15.12:0/437525472 >> [v2:172.21.15.178:6838/70628097,v1:172.21.15.178:6839/70628097] conn(0x558ab79523c0 0x558ab7953830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:06:33.244 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:33.238+0000 7fc8fb7ed740 4 client.15252 mds_command: new command op to 24637 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T00:06:33.244 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:33.238+0000 7fc8fb7ed740 1 -- 172.21.15.12:0/437525472 --> [v2:172.21.15.178:6838/70628097,v1:172.21.15.178:6839/70628097] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558ab78aeca0 con 0x558ab79523c0 2024-09-11T00:06:33.244 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:33.238+0000 7fc8ebe00640 1 --2- 172.21.15.12:0/437525472 >> [v2:172.21.15.178:6838/70628097,v1:172.21.15.178:6839/70628097] conn(0x558ab79523c0 0x558ab7953830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:06:33.244 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:33.239+0000 7fc8ebe00640 1 --2- 172.21.15.12:0/437525472 >> [v2:172.21.15.178:6838/70628097,v1:172.21.15.178:6839/70628097] conn(0x558ab79523c0 0x558ab7953830 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=860f0be4f371f440 server_cookie=ccdd75ba0b459865 in_seq=0 out_seq=0 2024-09-11T00:06:33.245 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:33.239+0000 7fc8d7400640 10 client.15252 ms_handle_connect on v2:172.21.15.178:6838/70628097 2024-09-11T00:06:33.245 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:33.239+0000 7fc8d7400640 1 -- 172.21.15.12:0/437525472 <== mds.3 v2:172.21.15.178:6838/70628097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x558ab78aeca0 con 0x558ab79523c0 2024-09-11T00:06:33.245 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:33.239+0000 7fc8d7400640 10 client.15252 handle_command_reply: tid=0 2024-09-11T00:06:33.245 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:33.239+0000 7fc8fb7ed740 1 client.15252 shutdown 2024-09-11T00:06:33.245 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:33.240+0000 7fc8fb7ed740 1 -- 172.21.15.12:0/437525472 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558ab7935240 msgr2=0x558ab793eb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:06:33.245 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:33.240+0000 7fc8fb7ed740 1 --2- 172.21.15.12:0/437525472 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558ab7935240 0x558ab793eb20 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fc8e40ac5a0 tx=0x7fc8e400c840 comp rx=0 tx=0).stop 2024-09-11T00:06:33.245 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:33.240+0000 7fc8fb7ed740 1 -- 172.21.15.12:0/437525472 shutdown_connections 2024-09-11T00:06:33.245 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:33.240+0000 7fc8fb7ed740 1 --2- 172.21.15.12:0/437525472 >> [v2:172.21.15.178:6838/70628097,v1:172.21.15.178:6839/70628097] conn(0x558ab79523c0 0x558ab7953830 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:33.245 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:33.240+0000 7fc8fb7ed740 1 --2- 172.21.15.12:0/437525472 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558ab7941af0 0x558ab793f5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:33.245 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:33.240+0000 7fc8fb7ed740 1 --2- 172.21.15.12:0/437525472 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558ab7935b60 0x558ab793f060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:33.245 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:33.240+0000 7fc8fb7ed740 1 --2- 172.21.15.12:0/437525472 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558ab7935240 0x558ab793eb20 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:33.245 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:33.240+0000 7fc8d7400640 0 client.15252 ms_handle_reset on v2:172.21.15.178:6838/70628097 2024-09-11T00:06:33.245 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:33.240+0000 7fc8fb7ed740 1 -- 172.21.15.12:0/437525472 >> 172.21.15.12:0/437525472 conn(0x558ab7933020 msgr2=0x558ab7938c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:06:33.245 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:33.240+0000 7fc8fb7ed740 1 -- 172.21.15.12:0/437525472 shutdown_connections 2024-09-11T00:06:33.246 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:33.240+0000 7fc8fb7ed740 1 -- 172.21.15.12:0/437525472 wait complete. 2024-09-11T00:06:33.246 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:33.240+0000 7fc8fb7ed740 20 client.15252 trim_cache size 0 max 16384 2024-09-11T00:06:33.261 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:33.259+0000 7fc8fb7ed740 1 Processor -- start 2024-09-11T00:06:33.262 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:33.259+0000 7fc8fb7ed740 1 -- start start 2024-09-11T00:06:33.262 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:33.260+0000 7fc8fb7ed740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558ab7935240 0x558ab7953830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:06:33.262 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:33.260+0000 7fc8fb7ed740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558ab7935b60 0x558ab79347d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:06:33.262 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:33.260+0000 7fc8fb7ed740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558ab7941af0 0x558ab793bd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:06:33.262 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:33.260+0000 7fc8fb7ed740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x558ab7951b70 con 0x558ab7935240 2024-09-11T00:06:33.262 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:33.260+0000 7fc8fb7ed740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x558ab7952240 con 0x558ab7935b60 2024-09-11T00:06:33.262 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:33.260+0000 7fc8fb7ed740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x558ab7952910 con 0x558ab7941af0 2024-09-11T00:06:33.263 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:33.260+0000 7fc8f1e00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558ab7935240 0x558ab7953830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:06:33.263 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:33.260+0000 7fc8ebe00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558ab7935b60 0x558ab79347d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:06:33.263 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:33.260+0000 7fc8f2800640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558ab7941af0 0x558ab793bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:06:33.263 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:33.260+0000 7fc8f1e00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558ab7935240 0x558ab7953830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:43500/0 (socket says 172.21.15.12:43500) 2024-09-11T00:06:33.263 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:33.260+0000 7fc8f1e00640 1 -- 172.21.15.12:0/3614302540 learned_addr learned my addr 172.21.15.12:0/3614302540 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:06:33.263 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:33.260+0000 7fc8ebe00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558ab7935b60 0x558ab79347d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.12:51828/0 (socket says 172.21.15.12:51828) 2024-09-11T00:06:33.264 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:33.260+0000 7fc8ebe00640 1 -- 172.21.15.12:0/3614302540 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558ab7941af0 msgr2=0x558ab793bd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:06:33.264 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:33.260+0000 7fc8ebe00640 1 --2- 172.21.15.12:0/3614302540 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558ab7941af0 0x558ab793bd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:33.264 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:33.260+0000 7fc8ebe00640 1 -- 172.21.15.12:0/3614302540 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558ab7935240 msgr2=0x558ab7953830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:06:33.264 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:33.260+0000 7fc8ebe00640 1 --2- 172.21.15.12:0/3614302540 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558ab7935240 0x558ab7953830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:33.264 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:33.260+0000 7fc8ebe00640 1 -- 172.21.15.12:0/3614302540 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558ab793d020 con 0x558ab7935b60 2024-09-11T00:06:33.264 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:33.260+0000 7fc8f1e00640 1 --2- 172.21.15.12:0/3614302540 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558ab7935240 0x558ab7953830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:06:33.264 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:33.261+0000 7fc8ebe00640 1 --2- 172.21.15.12:0/3614302540 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558ab7935b60 0x558ab79347d0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fc8dc004770 tx=0x7fc8dc0058d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9eb2c311d9810788 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:06:33.264 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:33.261+0000 7fc8e2600640 1 -- 172.21.15.12:0/3614302540 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8dc04a070 con 0x558ab7935b60 2024-09-11T00:06:33.264 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:33.261+0000 7fc8e2600640 1 -- 172.21.15.12:0/3614302540 <== mon.2 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc8dc031950 con 0x558ab7935b60 2024-09-11T00:06:33.264 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:33.261+0000 7fc8e2600640 1 -- 172.21.15.12:0/3614302540 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8dc035040 con 0x558ab7935b60 2024-09-11T00:06:33.267 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:33.262+0000 7fc8fb7ed740 1 -- 172.21.15.12:0/3614302540 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558ab7935b60 msgr2=0x558ab79347d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:06:33.267 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:33.262+0000 7fc8fb7ed740 1 --2- 172.21.15.12:0/3614302540 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558ab7935b60 0x558ab79347d0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fc8dc004770 tx=0x7fc8dc0058d0 comp rx=0 tx=0).stop 2024-09-11T00:06:33.267 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:33.262+0000 7fc8fb7ed740 1 -- 172.21.15.12:0/3614302540 shutdown_connections 2024-09-11T00:06:33.267 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:33.262+0000 7fc8fb7ed740 1 --2- 172.21.15.12:0/3614302540 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558ab7941af0 0x558ab793bd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:33.267 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:33.262+0000 7fc8fb7ed740 1 --2- 172.21.15.12:0/3614302540 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558ab7935b60 0x558ab79347d0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:33.267 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:33.262+0000 7fc8fb7ed740 1 --2- 172.21.15.12:0/3614302540 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558ab7935240 0x558ab7953830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:33.267 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:33.262+0000 7fc8fb7ed740 1 -- 172.21.15.12:0/3614302540 >> 172.21.15.12:0/3614302540 conn(0x558ab7933020 msgr2=0x558ab79340c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:06:33.267 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:33.262+0000 7fc8fb7ed740 1 -- 172.21.15.12:0/3614302540 shutdown_connections 2024-09-11T00:06:33.267 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:33.262+0000 7fc8fb7ed740 1 -- 172.21.15.12:0/3614302540 wait complete. 2024-09-11T00:06:33.268 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:33.262+0000 7fc8fb7ed740 1 -- start start 2024-09-11T00:06:33.268 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:33.263+0000 7fc8fb7ed740 1 Processor -- start 2024-09-11T00:06:33.268 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:33.263+0000 7fc8fb7ed740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558ab7935240 0x558ab7956210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:06:33.268 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:33.263+0000 7fc8fb7ed740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558ab7935b60 0x558ab7956750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:06:33.268 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:33.263+0000 7fc8fb7ed740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558ab7941af0 0x558ab795b720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:06:33.268 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:33.263+0000 7fc8fb7ed740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x558ab79536b0 con 0x558ab7941af0 2024-09-11T00:06:33.268 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:33.263+0000 7fc8fb7ed740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x558ab7952fe0 con 0x558ab7935b60 2024-09-11T00:06:33.268 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:33.263+0000 7fc8fb7ed740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x558ab7953d80 con 0x558ab7935240 2024-09-11T00:06:33.268 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:33.263+0000 7fc8fb7ed740 10 client.0 _ll_register_callbacks cb 0x7fc8ec009a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:06:33.268 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:33.263+0000 7fc8f1e00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558ab7935240 0x558ab7956210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:06:33.268 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:33.263+0000 7fc8ebe00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558ab7935b60 0x558ab7956750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:06:33.268 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:33.263+0000 7fc8f1e00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558ab7935240 0x558ab7956210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.12:47132/0 (socket says 172.21.15.12:47132) 2024-09-11T00:06:33.269 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:33.263+0000 7fc8f1e00640 1 -- 172.21.15.12:0/2485372624 learned_addr learned my addr 172.21.15.12:0/2485372624 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:06:33.269 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:33.263+0000 7fc8ebe00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558ab7935b60 0x558ab7956750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.12:51838/0 (socket says 172.21.15.12:51838) 2024-09-11T00:06:33.269 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:33.263+0000 7fc8f2800640 1 --2- 172.21.15.12:0/2485372624 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558ab7941af0 0x558ab795b720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:06:33.269 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:33.264+0000 7fc8f1e00640 1 -- 172.21.15.12:0/2485372624 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558ab7935b60 msgr2=0x558ab7956750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:06:33.269 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:33.264+0000 7fc8f1e00640 1 --2- 172.21.15.12:0/2485372624 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558ab7935b60 0x558ab7956750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:33.269 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:33.264+0000 7fc8f1e00640 1 -- 172.21.15.12:0/2485372624 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558ab7941af0 msgr2=0x558ab795b720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:06:33.269 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:33.264+0000 7fc8f1e00640 1 --2- 172.21.15.12:0/2485372624 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558ab7941af0 0x558ab795b720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:33.269 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:33.264+0000 7fc8f1e00640 1 -- 172.21.15.12:0/2485372624 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558ab795bce0 con 0x558ab7935240 2024-09-11T00:06:33.269 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:33.264+0000 7fc8ebe00640 1 --2- 172.21.15.12:0/2485372624 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558ab7935b60 0x558ab7956750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:06:33.269 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:33.264+0000 7fc8f2800640 1 --2- 172.21.15.12:0/2485372624 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558ab7941af0 0x558ab795b720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T00:06:33.269 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:33.264+0000 7fc8f1e00640 1 --2- 172.21.15.12:0/2485372624 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558ab7935240 0x558ab7956210 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fc8e400c6a0 tx=0x7fc8e4005bb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=68f86b5df6c1d717 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:06:33.269 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:33.264+0000 7fc8d7400640 10 client.0 ms_handle_connect on v2:172.21.15.178:3300/0 2024-09-11T00:06:33.269 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:33.264+0000 7fc8d7400640 1 -- 172.21.15.12:0/2485372624 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8e40abf60 con 0x558ab7935240 2024-09-11T00:06:33.270 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:33.264+0000 7fc8fb7ed740 1 -- 172.21.15.12:0/2485372624 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x558ab78aa750 con 0x558ab7935240 2024-09-11T00:06:33.270 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:33.264+0000 7fc8d7400640 1 -- 172.21.15.12:0/2485372624 <== mon.1 v2:172.21.15.178:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 9208+0+0 (secure 0 0 0) 0x7fc8e40a8880 con 0x558ab7935240 2024-09-11T00:06:33.270 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:33.265+0000 7fc8d7400640 1 -- 172.21.15.12:0/2485372624 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fc8e40ac3e0 con 0x558ab7935240 2024-09-11T00:06:33.270 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:33.265+0000 7fc8fb7ed740 10 client.25276 fetch_fsmap learned FSMap version 28 2024-09-11T00:06:33.270 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:33.265+0000 7fc8fb7ed740 1 -- 172.21.15.12:0/2485372624 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x558ab79bf5a0 con 0x558ab7935240 2024-09-11T00:06:33.270 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:33.265+0000 7fc8d7400640 1 -- 172.21.15.12:0/2485372624 <== mon.1 v2:172.21.15.178:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x558ab78aa750 con 0x558ab7935240 2024-09-11T00:06:33.270 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:33.265+0000 7fc8d7400640 1 -- 172.21.15.12:0/2485372624 <== mon.1 v2:172.21.15.178:3300/0 5 ==== fsmap(e 28) ==== 8370+0+0 (secure 0 0 0) 0x7fc8e40b5710 con 0x558ab7935240 2024-09-11T00:06:33.270 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:33.265+0000 7fc8fb7ed740 10 client.25276 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T00:06:33.270 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:33.265+0000 7fc8fb7ed740 10 client.25276 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-11T00:06:33.270 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:33.265+0000 7fc8fb7ed740 20 client.25276 populate_metadata read hostname 'smithi012' 2024-09-11T00:06:33.270 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:33.265+0000 7fc8fb7ed740 1 --2- 172.21.15.12:0/2485372624 >> [v2:172.21.15.178:6838/70628097,v1:172.21.15.178:6839/70628097] conn(0x558ab79c3f40 0x558ab79c2c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:06:33.270 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:33.265+0000 7fc8fb7ed740 4 client.25276 mds_command: new command op to 24637 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-11T00:06:33.270 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:33.265+0000 7fc8fb7ed740 1 -- 172.21.15.12:0/2485372624 --> [v2:172.21.15.178:6838/70628097,v1:172.21.15.178:6839/70628097] -- command(tid 0: {"prefix": "status"}) -- 0x558ab78a7b80 con 0x558ab79c3f40 2024-09-11T00:06:33.271 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:33.265+0000 7fc8ebe00640 1 --2- 172.21.15.12:0/2485372624 >> [v2:172.21.15.178:6838/70628097,v1:172.21.15.178:6839/70628097] conn(0x558ab79c3f40 0x558ab79c2c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:06:33.272 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:33.266+0000 7fc8ebe00640 1 --2- 172.21.15.12:0/2485372624 >> [v2:172.21.15.178:6838/70628097,v1:172.21.15.178:6839/70628097] conn(0x558ab79c3f40 0x558ab79c2c50 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=509521a39b55055 server_cookie=192a3479df0df4bb in_seq=0 out_seq=0 2024-09-11T00:06:33.272 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:33.266+0000 7fc8d7400640 10 client.25276 ms_handle_connect on v2:172.21.15.178:6838/70628097 2024-09-11T00:06:33.272 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:33.266+0000 7fc8d7400640 1 -- 172.21.15.12:0/2485372624 <== mds.3 v2:172.21.15.178:6838/70628097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x558ab78a7b80 con 0x558ab79c3f40 2024-09-11T00:06:33.272 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:33.266+0000 7fc8d7400640 10 client.25276 handle_command_reply: tid=0 2024-09-11T00:06:33.272 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:33.266+0000 7fc8fb7ed740 1 client.25276 shutdown 2024-09-11T00:06:33.272 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:33.266+0000 7fc8fb7ed740 1 -- 172.21.15.12:0/2485372624 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558ab7935240 msgr2=0x558ab7956210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:06:33.272 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:33.266+0000 7fc8fb7ed740 1 --2- 172.21.15.12:0/2485372624 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558ab7935240 0x558ab7956210 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fc8e400c6a0 tx=0x7fc8e4005bb0 comp rx=0 tx=0).stop 2024-09-11T00:06:33.272 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:33.267+0000 7fc8fb7ed740 1 -- 172.21.15.12:0/2485372624 shutdown_connections 2024-09-11T00:06:33.272 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:33.267+0000 7fc8fb7ed740 1 --2- 172.21.15.12:0/2485372624 >> [v2:172.21.15.178:6838/70628097,v1:172.21.15.178:6839/70628097] conn(0x558ab79c3f40 0x558ab79c2c50 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:33.272 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:33.267+0000 7fc8fb7ed740 1 --2- 172.21.15.12:0/2485372624 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558ab7941af0 0x558ab795b720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:33.272 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:33.267+0000 7fc8fb7ed740 1 --2- 172.21.15.12:0/2485372624 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558ab7935b60 0x558ab7956750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:33.272 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:33.267+0000 7fc8fb7ed740 1 --2- 172.21.15.12:0/2485372624 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558ab7935240 0x558ab7956210 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:33.272 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:33.267+0000 7fc8d7400640 0 client.25276 ms_handle_reset on v2:172.21.15.178:6838/70628097 2024-09-11T00:06:33.273 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:33.267+0000 7fc8fb7ed740 1 -- 172.21.15.12:0/2485372624 >> 172.21.15.12:0/2485372624 conn(0x558ab7933020 msgr2=0x558ab793ea90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:06:33.273 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:33.267+0000 7fc8fb7ed740 1 -- 172.21.15.12:0/2485372624 shutdown_connections 2024-09-11T00:06:33.273 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:33.267+0000 7fc8fb7ed740 1 -- 172.21.15.12:0/2485372624 wait complete. 2024-09-11T00:06:33.273 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:33.267+0000 7fc8fb7ed740 20 client.25276 trim_cache size 0 max 16384 2024-09-11T00:06:33.273 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-09-11T00:06:33.273 INFO:teuthology.orchestra.run.smithi012.stdout: "cluster_fsid": "ff3ad74a-6fcf-11ef-bcea-c7b262605968", 2024-09-11T00:06:33.273 INFO:teuthology.orchestra.run.smithi012.stdout: "whoami": 3, 2024-09-11T00:06:33.273 INFO:teuthology.orchestra.run.smithi012.stdout: "id": 24637, 2024-09-11T00:06:33.273 INFO:teuthology.orchestra.run.smithi012.stdout: "want_state": "up:active", 2024-09-11T00:06:33.273 INFO:teuthology.orchestra.run.smithi012.stdout: "state": "up:active", 2024-09-11T00:06:33.274 INFO:teuthology.orchestra.run.smithi012.stdout: "fs_name": "cephfs", 2024-09-11T00:06:33.274 INFO:teuthology.orchestra.run.smithi012.stdout: "rank_uptime": 105.033417571, 2024-09-11T00:06:33.274 INFO:teuthology.orchestra.run.smithi012.stdout: "mdsmap_epoch": 28, 2024-09-11T00:06:33.274 INFO:teuthology.orchestra.run.smithi012.stdout: "osdmap_epoch": 85, 2024-09-11T00:06:33.274 INFO:teuthology.orchestra.run.smithi012.stdout: "osdmap_epoch_barrier": 81, 2024-09-11T00:06:33.274 INFO:teuthology.orchestra.run.smithi012.stdout: "uptime": 171.06101495499999 2024-09-11T00:06:33.274 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-09-11T00:06:33.275 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:33.267+0000 7fc8f8c00640 1 -- 172.21.15.12:0/2005397494 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fc8d80b52e0 msgr2=0x7fc8d80b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:06:33.275 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:33.267+0000 7fc8f8c00640 1 --2- 172.21.15.12:0/2005397494 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fc8d80b52e0 0x7fc8d80b77a0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fc8ec005fd0 tx=0x7fc8ec009600 comp rx=0 tx=0).stop 2024-09-11T00:06:33.275 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:33.267+0000 7fc8f8c00640 1 -- 172.21.15.12:0/2005397494 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc8f4104fa0 msgr2=0x7fc8f419b4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:06:33.275 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:33.267+0000 7fc8f8c00640 1 --2- 172.21.15.12:0/2005397494 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc8f4104fa0 0x7fc8f419b4a0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fc8e400e9f0 tx=0x7fc8e400eeb0 comp rx=0 tx=0).stop 2024-09-11T00:06:33.275 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:33.268+0000 7fc8f8c00640 1 -- 172.21.15.12:0/2005397494 shutdown_connections 2024-09-11T00:06:33.275 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:33.268+0000 7fc8f8c00640 1 --2- 172.21.15.12:0/2005397494 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fc8d80b52e0 0x7fc8d80b77a0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:33.275 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:33.268+0000 7fc8f8c00640 1 --2- 172.21.15.12:0/2005397494 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc8f410b330 0x7fc8f41a0d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:33.275 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:33.268+0000 7fc8f8c00640 1 --2- 172.21.15.12:0/2005397494 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc8f410a9b0 0x7fc8f419b9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:33.275 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:33.268+0000 7fc8f8c00640 1 --2- 172.21.15.12:0/2005397494 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc8f4104fa0 0x7fc8f419b4a0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:33.276 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:33.268+0000 7fc8f8c00640 1 -- 172.21.15.12:0/2005397494 >> 172.21.15.12:0/2005397494 conn(0x7fc8f4100a50 msgr2=0x7fc8f4102e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:06:33.276 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:33.268+0000 7fc8f8c00640 1 -- 172.21.15.12:0/2005397494 shutdown_connections 2024-09-11T00:06:33.276 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:33.268+0000 7fc8f8c00640 1 -- 172.21.15.12:0/2005397494 wait complete. 2024-09-11T00:06:33.532 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:06:33 smithi165 ceph-mon[35356]: pgmap v382: 129 pgs: 129 active+clean; 647 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 429 B/s wr, 11 op/s; 9 B/s, 0 objects/s recovering 2024-09-11T00:06:33.562 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:06:33 smithi012 ceph-mon[30333]: pgmap v382: 129 pgs: 129 active+clean; 647 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 429 B/s wr, 11 op/s; 9 B/s, 0 objects/s recovering 2024-09-11T00:06:33.580 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:06:33 smithi178 ceph-mon[35919]: pgmap v382: 129 pgs: 129 active+clean; 647 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 429 B/s wr, 11 op/s; 9 B/s, 0 objects/s recovering 2024-09-11T00:06:34.080 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid ff3ad74a-6fcf-11ef-bcea-c7b262605968 -- ceph tell mds.h status 2024-09-11T00:06:34.460 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:06:35.035 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:35.033+0000 7fdb01800640 1 -- 172.21.15.12:0/2043535449 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdafc1087b0 msgr2=0x7fdafc108bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:06:35.036 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:35.033+0000 7fdb01800640 1 --2- 172.21.15.12:0/2043535449 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdafc1087b0 0x7fdafc108bf0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fdae8009900 tx=0x7fdae802a410 comp rx=0 tx=0).stop 2024-09-11T00:06:35.036 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:35.033+0000 7fdb01800640 1 -- 172.21.15.12:0/2043535449 shutdown_connections 2024-09-11T00:06:35.036 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:35.033+0000 7fdb01800640 1 --2- 172.21.15.12:0/2043535449 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdafc109130 0x7fdafc10f930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:35.036 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:35.033+0000 7fdb01800640 1 --2- 172.21.15.12:0/2043535449 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdafc1087b0 0x7fdafc108bf0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:35.036 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:35.033+0000 7fdb01800640 1 --2- 172.21.15.12:0/2043535449 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fdafc102da0 0x7fdafc103180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:35.036 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:35.033+0000 7fdb01800640 1 -- 172.21.15.12:0/2043535449 >> 172.21.15.12:0/2043535449 conn(0x7fdafc0fe810 msgr2=0x7fdafc100c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:06:35.036 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:35.033+0000 7fdb01800640 1 -- 172.21.15.12:0/2043535449 shutdown_connections 2024-09-11T00:06:35.036 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:35.033+0000 7fdb01800640 1 -- 172.21.15.12:0/2043535449 wait complete. 2024-09-11T00:06:35.038 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:35.034+0000 7fdb01800640 1 Processor -- start 2024-09-11T00:06:35.038 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:35.035+0000 7fdb01800640 1 -- start start 2024-09-11T00:06:35.038 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:35.035+0000 7fdb01800640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdafc102da0 0x7fdafc19b530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:06:35.038 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:35.035+0000 7fdb01800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fdafc1087b0 0x7fdafc19ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:06:35.038 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:35.035+0000 7fdb01800640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdafc109130 0x7fdafc1a0e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:06:35.038 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:35.035+0000 7fdb01800640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fdafc112d80 con 0x7fdafc102da0 2024-09-11T00:06:35.038 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:35.035+0000 7fdb01800640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fdafc112c00 con 0x7fdafc1087b0 2024-09-11T00:06:35.039 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:35.035+0000 7fdb01800640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fdafc112f00 con 0x7fdafc109130 2024-09-11T00:06:35.039 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:35.035+0000 7fdafaa00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdafc102da0 0x7fdafc19b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:06:35.039 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:35.035+0000 7fdafa000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fdafc1087b0 0x7fdafc19ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:06:35.039 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:35.035+0000 7fdafa000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fdafc1087b0 0x7fdafc19ba70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.12:51856/0 (socket says 172.21.15.12:51856) 2024-09-11T00:06:35.039 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:35.035+0000 7fdafa000640 1 -- 172.21.15.12:0/3967274721 learned_addr learned my addr 172.21.15.12:0/3967274721 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:06:35.039 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:35.035+0000 7fdafb400640 1 --2- 172.21.15.12:0/3967274721 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdafc109130 0x7fdafc1a0e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:06:35.039 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:35.036+0000 7fdafaa00640 1 -- 172.21.15.12:0/3967274721 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdafc109130 msgr2=0x7fdafc1a0e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:06:35.039 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:35.036+0000 7fdafaa00640 1 --2- 172.21.15.12:0/3967274721 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdafc109130 0x7fdafc1a0e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:35.039 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:35.036+0000 7fdafaa00640 1 -- 172.21.15.12:0/3967274721 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fdafc1087b0 msgr2=0x7fdafc19ba70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:06:35.039 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:35.036+0000 7fdafaa00640 1 --2- 172.21.15.12:0/3967274721 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fdafc1087b0 0x7fdafc19ba70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:35.040 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:35.036+0000 7fdafaa00640 1 -- 172.21.15.12:0/3967274721 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdafc1a15e0 con 0x7fdafc102da0 2024-09-11T00:06:35.040 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:35.036+0000 7fdafaa00640 1 --2- 172.21.15.12:0/3967274721 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdafc102da0 0x7fdafc19b530 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fdaec00e9f0 tx=0x7fdaec00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:06:35.040 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:35.036+0000 7fdaf3400640 1 -- 172.21.15.12:0/3967274721 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdaec00cd80 con 0x7fdafc102da0 2024-09-11T00:06:35.040 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:35.036+0000 7fdaf3400640 1 -- 172.21.15.12:0/3967274721 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdaec0035f0 con 0x7fdafc102da0 2024-09-11T00:06:35.040 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:35.036+0000 7fdb01800640 1 -- 172.21.15.12:0/3967274721 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdafc1a18d0 con 0x7fdafc102da0 2024-09-11T00:06:35.040 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:35.036+0000 7fdaf3400640 1 -- 172.21.15.12:0/3967274721 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdaec010670 con 0x7fdafc102da0 2024-09-11T00:06:35.040 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:35.036+0000 7fdb01800640 1 -- 172.21.15.12:0/3967274721 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdafc1a7f90 con 0x7fdafc102da0 2024-09-11T00:06:35.041 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:35.039+0000 7fdb04407740 1 Processor -- start 2024-09-11T00:06:35.042 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:35.040+0000 7fdb04407740 1 -- start start 2024-09-11T00:06:35.042 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:35.040+0000 7fdb04407740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5632a73d9240 0x5632a73d9620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:06:35.042 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:35.040+0000 7fdb04407740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5632a73d9b60 0x5632a73e55b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:06:35.042 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:35.040+0000 7fdb04407740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5632a73e5af0 0x5632a73e7ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:06:35.042 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:35.040+0000 7fdb04407740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x5632a7350d50 con 0x5632a73e5af0 2024-09-11T00:06:35.042 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:35.040+0000 7fdb04407740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x5632a7340180 con 0x5632a73d9b60 2024-09-11T00:06:35.042 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:35.040+0000 7fdb04407740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5632a7343f90 con 0x5632a73d9240 2024-09-11T00:06:35.042 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:35.040+0000 7fdaf3400640 1 -- 172.21.15.12:0/3967274721 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7fdaec004000 con 0x7fdafc102da0 2024-09-11T00:06:35.043 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:35.040+0000 7fdafa000640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5632a73e5af0 0x5632a73e7ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:06:35.043 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:35.040+0000 7fdafb400640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5632a73d9240 0x5632a73d9620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:06:35.043 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:35.040+0000 7fdafb400640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5632a73d9240 0x5632a73d9620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.12:47166/0 (socket says 172.21.15.12:47166) 2024-09-11T00:06:35.043 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:35.040+0000 7fdafb400640 1 -- 172.21.15.12:0/4288460868 learned_addr learned my addr 172.21.15.12:0/4288460868 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:06:35.044 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:35.040+0000 7fdafaa00640 1 --2- 172.21.15.12:0/4288460868 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5632a73d9b60 0x5632a73e55b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:06:35.044 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:35.041+0000 7fdafb400640 1 -- 172.21.15.12:0/4288460868 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5632a73d9b60 msgr2=0x5632a73e55b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:06:35.044 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:35.041+0000 7fdafb400640 1 --2- 172.21.15.12:0/4288460868 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5632a73d9b60 0x5632a73e55b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:35.044 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:35.041+0000 7fdafb400640 1 -- 172.21.15.12:0/4288460868 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5632a73e5af0 msgr2=0x5632a73e7ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:06:35.044 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:35.041+0000 7fdafb400640 1 --2- 172.21.15.12:0/4288460868 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5632a73e5af0 0x5632a73e7ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:35.044 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:35.041+0000 7fdafb400640 1 -- 172.21.15.12:0/4288460868 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5632a730aa70 con 0x5632a73d9240 2024-09-11T00:06:35.044 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:35.041+0000 7fdafa000640 1 --2- 172.21.15.12:0/4288460868 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5632a73e5af0 0x5632a73e7ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:06:35.044 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:35.041+0000 7fdafb400640 1 --2- 172.21.15.12:0/4288460868 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5632a73d9240 0x5632a73d9620 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fdaf4009fc0 tx=0x7fdaf4010860 comp rx=0 tx=0).ready entity=mon.1 client_cookie=adf9c0031775c983 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:06:35.044 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:35.041+0000 7fdae7e00640 1 -- 172.21.15.12:0/4288460868 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdaf4011700 con 0x5632a73d9240 2024-09-11T00:06:35.044 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:35.042+0000 7fdae7e00640 1 -- 172.21.15.12:0/4288460868 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdaf4011d50 con 0x5632a73d9240 2024-09-11T00:06:35.045 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:35.042+0000 7fdae7e00640 1 -- 172.21.15.12:0/4288460868 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdaf4019d20 con 0x5632a73d9240 2024-09-11T00:06:35.046 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:35.042+0000 7fdaf3400640 1 --2- 172.21.15.12:0/3967274721 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fdae00b5360 0x7fdae00b7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:06:35.046 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:35.042+0000 7fdb04407740 1 -- 172.21.15.12:0/4288460868 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5632a73d9240 msgr2=0x5632a73d9620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:06:35.047 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:35.042+0000 7fdb04407740 1 --2- 172.21.15.12:0/4288460868 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5632a73d9240 0x5632a73d9620 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fdaf4009fc0 tx=0x7fdaf4010860 comp rx=0 tx=0).stop 2024-09-11T00:06:35.047 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:35.043+0000 7fdb04407740 1 -- 172.21.15.12:0/4288460868 shutdown_connections 2024-09-11T00:06:35.047 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:35.043+0000 7fdb04407740 1 --2- 172.21.15.12:0/4288460868 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5632a73e5af0 0x5632a73e7ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:35.047 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:35.043+0000 7fdb04407740 1 --2- 172.21.15.12:0/4288460868 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5632a73d9b60 0x5632a73e55b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:35.047 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:35.043+0000 7fdb04407740 1 --2- 172.21.15.12:0/4288460868 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5632a73d9240 0x5632a73d9620 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:35.047 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:35.043+0000 7fdb04407740 1 -- 172.21.15.12:0/4288460868 >> 172.21.15.12:0/4288460868 conn(0x5632a73d7020 msgr2=0x5632a73d8210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:06:35.047 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:35.043+0000 7fdaf3400640 1 -- 172.21.15.12:0/3967274721 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9208+0+0 (secure 0 0 0) 0x7fdaec0e6280 con 0x7fdafc102da0 2024-09-11T00:06:35.047 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:35.043+0000 7fdb04407740 1 -- 172.21.15.12:0/4288460868 shutdown_connections 2024-09-11T00:06:35.047 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:35.043+0000 7fdafaa00640 1 --2- 172.21.15.12:0/3967274721 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fdae00b5360 0x7fdae00b7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:06:35.047 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:35.043+0000 7fdb04407740 1 -- 172.21.15.12:0/4288460868 wait complete. 2024-09-11T00:06:35.048 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:35.043+0000 7fdb04407740 1 -- start start 2024-09-11T00:06:35.048 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:35.044+0000 7fdb04407740 1 Processor -- start 2024-09-11T00:06:35.048 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:35.044+0000 7fdafaa00640 1 --2- 172.21.15.12:0/3967274721 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fdae00b5360 0x7fdae00b7820 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fdaec009230 tx=0x7fdaec0e9780 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:06:35.048 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:35.044+0000 7fdb04407740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5632a73d9240 0x5632a73e2a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:06:35.048 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:35.044+0000 7fdb04407740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5632a73d9b60 0x5632a73e2fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:06:35.048 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:35.044+0000 7fdb04407740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5632a73e5af0 0x5632a73e34f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:06:35.048 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:35.044+0000 7fdb04407740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x5632a71a15f0 con 0x5632a73e5af0 2024-09-11T00:06:35.048 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:35.044+0000 7fdb04407740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x5632a72db460 con 0x5632a73d9b60 2024-09-11T00:06:35.048 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:35.044+0000 7fdb04407740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5632a73e8840 con 0x5632a73d9240 2024-09-11T00:06:35.048 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:35.044+0000 7fdb04407740 10 client.0 _ll_register_callbacks cb 0x5632a72eb210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:06:35.048 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:35.044+0000 7fdafb400640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5632a73d9240 0x5632a73e2a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:06:35.048 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:35.044+0000 7fdafa000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5632a73d9b60 0x5632a73e2fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:06:35.048 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:35.044+0000 7fdafb400640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5632a73d9240 0x5632a73e2a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.12:47168/0 (socket says 172.21.15.12:47168) 2024-09-11T00:06:35.048 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:35.045+0000 7fdafb400640 1 -- 172.21.15.12:0/1281391910 learned_addr learned my addr 172.21.15.12:0/1281391910 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:06:35.049 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:35.045+0000 7fdafb400640 1 --2- 172.21.15.12:0/1281391910 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5632a73e5af0 0x5632a73e34f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:06:35.052 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:35.045+0000 7fdafa000640 1 -- 172.21.15.12:0/1281391910 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5632a73d9240 msgr2=0x5632a73e2a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:06:35.052 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:35.045+0000 7fdafa000640 1 --2- 172.21.15.12:0/1281391910 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5632a73d9240 0x5632a73e2a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:35.052 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:35.045+0000 7fdafa000640 1 -- 172.21.15.12:0/1281391910 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5632a73e5af0 msgr2=0x5632a73e34f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:06:35.052 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:35.045+0000 7fdafa000640 1 --2- 172.21.15.12:0/1281391910 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5632a73e5af0 0x5632a73e34f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:35.052 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:35.045+0000 7fdafa000640 1 -- 172.21.15.12:0/1281391910 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5632a709e540 con 0x5632a73d9b60 2024-09-11T00:06:35.052 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:35.045+0000 7fdafa000640 1 --2- 172.21.15.12:0/1281391910 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5632a73d9b60 0x5632a73e2fb0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fdae8004770 tx=0x7fdae8031f10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6ab29950bc163875 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:06:35.052 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:35.045+0000 7fdae4c00640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-09-11T00:06:35.052 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:35.046+0000 7fdae4c00640 1 -- 172.21.15.12:0/1281391910 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdae8035040 con 0x5632a73d9b60 2024-09-11T00:06:35.052 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:35.046+0000 7fdb04407740 1 -- 172.21.15.12:0/1281391910 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5632a72dda70 con 0x5632a73d9b60 2024-09-11T00:06:35.053 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:35.046+0000 7fdae4c00640 1 -- 172.21.15.12:0/1281391910 <== mon.2 v2:172.21.15.165:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 9208+0+0 (secure 0 0 0) 0x7fdae8031860 con 0x5632a73d9b60 2024-09-11T00:06:35.053 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:35.046+0000 7fdae4c00640 1 -- 172.21.15.12:0/1281391910 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fdae804f050 con 0x5632a73d9b60 2024-09-11T00:06:35.053 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:35.046+0000 7fdb04407740 10 client.25310 fetch_fsmap learned FSMap version 28 2024-09-11T00:06:35.053 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:35.046+0000 7fdb04407740 1 -- 172.21.15.12:0/1281391910 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5632a72f86f0 con 0x5632a73d9b60 2024-09-11T00:06:35.053 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:35.046+0000 7fdae4c00640 1 -- 172.21.15.12:0/1281391910 <== mon.2 v2:172.21.15.165:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x5632a72dda70 con 0x5632a73d9b60 2024-09-11T00:06:35.053 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:35.047+0000 7fdae4c00640 1 -- 172.21.15.12:0/1281391910 <== mon.2 v2:172.21.15.165:3300/0 5 ==== fsmap(e 28) ==== 8370+0+0 (secure 0 0 0) 0x7fdae8003c30 con 0x5632a73d9b60 2024-09-11T00:06:35.053 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:35.047+0000 7fdb04407740 10 client.25310 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T00:06:35.053 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:35.047+0000 7fdb04407740 10 client.25310 resolve_mds: resolved name 'h' to daemon mds.h 2024-09-11T00:06:35.053 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:35.047+0000 7fdb04407740 20 client.25310 populate_metadata read hostname 'smithi012' 2024-09-11T00:06:35.053 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:35.047+0000 7fdb04407740 1 --2- 172.21.15.12:0/1281391910 >> [v2:172.21.15.165:6836/6215088,v1:172.21.15.165:6837/6215088] conn(0x5632a73f67c0 0x5632a73f7c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:06:35.053 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:35.047+0000 7fdb04407740 4 client.25310 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T00:06:35.053 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:35.047+0000 7fdb04407740 1 -- 172.21.15.12:0/1281391910 --> [v2:172.21.15.165:6836/6215088,v1:172.21.15.165:6837/6215088] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5632a7352ca0 con 0x5632a73f67c0 2024-09-11T00:06:35.053 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:35.047+0000 7fdafb400640 1 --2- 172.21.15.12:0/1281391910 >> [v2:172.21.15.165:6836/6215088,v1:172.21.15.165:6837/6215088] conn(0x5632a73f67c0 0x5632a73f7c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:06:35.053 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:35.048+0000 7fdafb400640 1 --2- 172.21.15.12:0/1281391910 >> [v2:172.21.15.165:6836/6215088,v1:172.21.15.165:6837/6215088] conn(0x5632a73f67c0 0x5632a73f7c30 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=b0a07dfa2ede1aa1 server_cookie=1b55798323e3c834 in_seq=0 out_seq=0 2024-09-11T00:06:35.054 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:35.048+0000 7fdae4c00640 10 client.25310 ms_handle_connect on v2:172.21.15.165:6836/6215088 2024-09-11T00:06:35.054 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:35.048+0000 7fdae4c00640 1 -- 172.21.15.12:0/1281391910 <== mds.2 v2:172.21.15.165:6836/6215088 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5632a7352ca0 con 0x5632a73f67c0 2024-09-11T00:06:35.054 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:35.048+0000 7fdae4c00640 10 client.25310 handle_command_reply: tid=0 2024-09-11T00:06:35.054 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:35.049+0000 7fdb04407740 1 client.25310 shutdown 2024-09-11T00:06:35.054 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:35.049+0000 7fdb04407740 1 -- 172.21.15.12:0/1281391910 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5632a73d9b60 msgr2=0x5632a73e2fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:06:35.054 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:35.049+0000 7fdb04407740 1 --2- 172.21.15.12:0/1281391910 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5632a73d9b60 0x5632a73e2fb0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fdae8004770 tx=0x7fdae8031f10 comp rx=0 tx=0).stop 2024-09-11T00:06:35.054 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:35.049+0000 7fdb04407740 1 -- 172.21.15.12:0/1281391910 shutdown_connections 2024-09-11T00:06:35.054 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:35.049+0000 7fdb04407740 1 --2- 172.21.15.12:0/1281391910 >> [v2:172.21.15.165:6836/6215088,v1:172.21.15.165:6837/6215088] conn(0x5632a73f67c0 0x5632a73f7c30 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:35.054 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:35.049+0000 7fdb04407740 1 --2- 172.21.15.12:0/1281391910 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5632a73e5af0 0x5632a73e34f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:35.054 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:35.049+0000 7fdb04407740 1 --2- 172.21.15.12:0/1281391910 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5632a73d9b60 0x5632a73e2fb0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:35.054 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:35.049+0000 7fdb04407740 1 --2- 172.21.15.12:0/1281391910 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5632a73d9240 0x5632a73e2a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:35.054 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:35.049+0000 7fdae4c00640 0 client.25310 ms_handle_reset on v2:172.21.15.165:6836/6215088 2024-09-11T00:06:35.054 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:35.049+0000 7fdb04407740 1 -- 172.21.15.12:0/1281391910 >> 172.21.15.12:0/1281391910 conn(0x5632a73d7020 msgr2=0x5632a73dcc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:06:35.055 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:35.049+0000 7fdb04407740 1 -- 172.21.15.12:0/1281391910 shutdown_connections 2024-09-11T00:06:35.055 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:35.049+0000 7fdb04407740 1 -- 172.21.15.12:0/1281391910 wait complete. 2024-09-11T00:06:35.055 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:35.049+0000 7fdb04407740 20 client.25310 trim_cache size 0 max 16384 2024-09-11T00:06:35.072 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:35.068+0000 7fdb04407740 1 Processor -- start 2024-09-11T00:06:35.072 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:35.068+0000 7fdb04407740 1 -- start start 2024-09-11T00:06:35.072 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:35.069+0000 7fdb04407740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5632a73d9240 0x5632a73e12f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:06:35.072 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:35.069+0000 7fdb04407740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5632a73d9b60 0x5632a73e1830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:06:35.073 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:35.069+0000 7fdb04407740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5632a73e5af0 0x5632a73d80c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:06:35.073 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:35.069+0000 7fdb04407740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x5632a73f5f70 con 0x5632a73e5af0 2024-09-11T00:06:35.073 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:35.069+0000 7fdb04407740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x5632a73f6640 con 0x5632a73d9240 2024-09-11T00:06:35.073 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:35.069+0000 7fdb04407740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5632a73f6d10 con 0x5632a73d9b60 2024-09-11T00:06:35.073 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:35.069+0000 7fdafb400640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5632a73e5af0 0x5632a73d80c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:06:35.073 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:35.069+0000 7fdafa000640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5632a73d9b60 0x5632a73e1830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:06:35.073 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:35.069+0000 7fdafb400640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5632a73e5af0 0x5632a73d80c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:43550/0 (socket says 172.21.15.12:43550) 2024-09-11T00:06:35.073 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:35.069+0000 7fdafa000640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5632a73d9b60 0x5632a73e1830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.12:47170/0 (socket says 172.21.15.12:47170) 2024-09-11T00:06:35.073 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:35.069+0000 7fdafb400640 1 -- 172.21.15.12:0/3703167555 learned_addr learned my addr 172.21.15.12:0/3703167555 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:06:35.073 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:35.069+0000 7fdafb400640 1 --2- 172.21.15.12:0/3703167555 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5632a73d9240 0x5632a73e12f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:06:35.073 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:35.069+0000 7fdafa000640 1 -- 172.21.15.12:0/3703167555 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5632a73d9240 msgr2=0x5632a73e12f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:06:35.073 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:35.069+0000 7fdafa000640 1 --2- 172.21.15.12:0/3703167555 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5632a73d9240 0x5632a73e12f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:35.074 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:35.069+0000 7fdafa000640 1 -- 172.21.15.12:0/3703167555 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5632a73e5af0 msgr2=0x5632a73d80c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:06:35.074 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:35.069+0000 7fdafa000640 1 --2- 172.21.15.12:0/3703167555 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5632a73e5af0 0x5632a73d80c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:35.074 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:35.069+0000 7fdafa000640 1 -- 172.21.15.12:0/3703167555 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5632a73dd790 con 0x5632a73d9b60 2024-09-11T00:06:35.074 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:35.070+0000 7fdafa000640 1 --2- 172.21.15.12:0/3703167555 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5632a73d9b60 0x5632a73e1830 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fdae8046570 tx=0x7fdae8005a20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7645f6357d384d87 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:06:35.074 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:35.070+0000 7fdae7400640 1 -- 172.21.15.12:0/3703167555 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdae8035030 con 0x5632a73d9b60 2024-09-11T00:06:35.074 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:35.070+0000 7fdae7400640 1 -- 172.21.15.12:0/3703167555 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdae8031860 con 0x5632a73d9b60 2024-09-11T00:06:35.074 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:35.070+0000 7fdae7400640 1 -- 172.21.15.12:0/3703167555 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdae80035f0 con 0x5632a73d9b60 2024-09-11T00:06:35.079 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:35.071+0000 7fdb04407740 1 -- 172.21.15.12:0/3703167555 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5632a73d9b60 msgr2=0x5632a73e1830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:06:35.079 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:35.071+0000 7fdb04407740 1 --2- 172.21.15.12:0/3703167555 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5632a73d9b60 0x5632a73e1830 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fdae8046570 tx=0x7fdae8005a20 comp rx=0 tx=0).stop 2024-09-11T00:06:35.079 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:35.071+0000 7fdb04407740 1 -- 172.21.15.12:0/3703167555 shutdown_connections 2024-09-11T00:06:35.079 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:35.071+0000 7fdb04407740 1 --2- 172.21.15.12:0/3703167555 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5632a73e5af0 0x5632a73d80c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:35.079 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:35.071+0000 7fdb04407740 1 --2- 172.21.15.12:0/3703167555 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5632a73d9b60 0x5632a73e1830 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:35.079 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:35.071+0000 7fdb04407740 1 --2- 172.21.15.12:0/3703167555 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5632a73d9240 0x5632a73e12f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:35.079 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:35.071+0000 7fdb04407740 1 -- 172.21.15.12:0/3703167555 >> 172.21.15.12:0/3703167555 conn(0x5632a73d7020 msgr2=0x5632a73d1ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:06:35.080 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:35.071+0000 7fdb04407740 1 -- 172.21.15.12:0/3703167555 shutdown_connections 2024-09-11T00:06:35.080 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:35.071+0000 7fdb04407740 1 -- 172.21.15.12:0/3703167555 wait complete. 2024-09-11T00:06:35.080 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:35.071+0000 7fdb04407740 1 -- start start 2024-09-11T00:06:35.080 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:35.071+0000 7fdb04407740 1 Processor -- start 2024-09-11T00:06:35.080 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:35.072+0000 7fdb04407740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5632a73d9240 0x5632a73fa190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:06:35.080 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:35.072+0000 7fdb04407740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5632a73d9b60 0x5632a73fa6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:06:35.080 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:35.072+0000 7fdb04407740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5632a73e5af0 0x5632a73ff8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:06:35.080 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:35.072+0000 7fdb04407740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x5632a73f7ab0 con 0x5632a73e5af0 2024-09-11T00:06:35.080 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:35.072+0000 7fdb04407740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x5632a73f73e0 con 0x5632a73d9240 2024-09-11T00:06:35.080 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:35.072+0000 7fdb04407740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5632a73f8180 con 0x5632a73d9b60 2024-09-11T00:06:35.080 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:35.072+0000 7fdb04407740 10 client.0 _ll_register_callbacks cb 0x7fdae80095f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:06:35.080 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:35.072+0000 7fdafb400640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5632a73e5af0 0x5632a73ff8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:06:35.081 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:35.072+0000 7fdafb400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5632a73d9240 0x5632a73fa190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:06:35.081 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:35.072+0000 7fdafa000640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5632a73d9b60 0x5632a73fa6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:06:35.081 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:35.072+0000 7fdafb400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5632a73d9240 0x5632a73fa190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.12:51894/0 (socket says 172.21.15.12:51894) 2024-09-11T00:06:35.081 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:35.072+0000 7fdafb400640 1 -- 172.21.15.12:0/3110164658 learned_addr learned my addr 172.21.15.12:0/3110164658 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:06:35.081 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:35.073+0000 7fdafb400640 1 -- 172.21.15.12:0/3110164658 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5632a73d9b60 msgr2=0x5632a73fa6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:06:35.081 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:35.073+0000 7fdafb400640 1 --2- 172.21.15.12:0/3110164658 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5632a73d9b60 0x5632a73fa6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:35.081 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:35.073+0000 7fdafb400640 1 -- 172.21.15.12:0/3110164658 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5632a73e5af0 msgr2=0x5632a73ff8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:06:35.082 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:35.073+0000 7fdafb400640 1 --2- 172.21.15.12:0/3110164658 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5632a73e5af0 0x5632a73ff8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:35.082 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:35.073+0000 7fdafb400640 1 -- 172.21.15.12:0/3110164658 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5632a73fafd0 con 0x5632a73d9240 2024-09-11T00:06:35.082 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:35.073+0000 7fdafb400640 1 --2- 172.21.15.12:0/3110164658 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5632a73d9240 0x5632a73fa190 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fdaf4009a80 tx=0x7fdaf40096a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b651c9910dcc3f1e server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:06:35.082 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:35.073+0000 7fdae4c00640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-09-11T00:06:35.082 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:35.073+0000 7fdae4c00640 1 -- 172.21.15.12:0/3110164658 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdaf4019e90 con 0x5632a73d9240 2024-09-11T00:06:35.082 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:35.073+0000 7fdb04407740 1 -- 172.21.15.12:0/3110164658 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdae8031860 con 0x5632a73d9240 2024-09-11T00:06:35.082 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:35.073+0000 7fdae4c00640 1 -- 172.21.15.12:0/3110164658 <== mon.2 v2:172.21.15.165:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 9208+0+0 (secure 0 0 0) 0x7fdaf4032070 con 0x5632a73d9240 2024-09-11T00:06:35.082 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:35.074+0000 7fdae4c00640 1 -- 172.21.15.12:0/3110164658 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fdaf4003970 con 0x5632a73d9240 2024-09-11T00:06:35.083 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:35.074+0000 7fdb04407740 10 client.25316 fetch_fsmap learned FSMap version 28 2024-09-11T00:06:35.083 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:35.074+0000 7fdb04407740 1 -- 172.21.15.12:0/3110164658 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5632a7463760 con 0x5632a73d9240 2024-09-11T00:06:35.083 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:35.074+0000 7fdae4c00640 1 -- 172.21.15.12:0/3110164658 <== mon.2 v2:172.21.15.165:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdae8031860 con 0x5632a73d9240 2024-09-11T00:06:35.083 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:35.074+0000 7fdae4c00640 1 -- 172.21.15.12:0/3110164658 <== mon.2 v2:172.21.15.165:3300/0 5 ==== fsmap(e 28) ==== 8370+0+0 (secure 0 0 0) 0x7fdaf4011500 con 0x5632a73d9240 2024-09-11T00:06:35.083 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:35.074+0000 7fdb04407740 10 client.25316 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T00:06:35.083 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:35.074+0000 7fdb04407740 10 client.25316 resolve_mds: resolved name 'h' to daemon mds.h 2024-09-11T00:06:35.083 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:35.074+0000 7fdb04407740 20 client.25316 populate_metadata read hostname 'smithi012' 2024-09-11T00:06:35.083 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:35.074+0000 7fdb04407740 1 --2- 172.21.15.12:0/3110164658 >> [v2:172.21.15.165:6836/6215088,v1:172.21.15.165:6837/6215088] conn(0x5632a74680c0 0x5632a7466dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:06:35.083 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:35.074+0000 7fdb04407740 4 client.25316 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-11T00:06:35.083 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:35.075+0000 7fdb04407740 1 -- 172.21.15.12:0/3110164658 --> [v2:172.21.15.165:6836/6215088,v1:172.21.15.165:6837/6215088] -- command(tid 0: {"prefix": "status"}) -- 0x5632a734e750 con 0x5632a74680c0 2024-09-11T00:06:35.083 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:35.075+0000 7fdafa000640 1 --2- 172.21.15.12:0/3110164658 >> [v2:172.21.15.165:6836/6215088,v1:172.21.15.165:6837/6215088] conn(0x5632a74680c0 0x5632a7466dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:06:35.083 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:35.075+0000 7fdafa000640 1 --2- 172.21.15.12:0/3110164658 >> [v2:172.21.15.165:6836/6215088,v1:172.21.15.165:6837/6215088] conn(0x5632a74680c0 0x5632a7466dd0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=7989af08965fb6de server_cookie=cb1c051d8cbaf3db in_seq=0 out_seq=0 2024-09-11T00:06:35.084 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:35.075+0000 7fdae4c00640 10 client.25316 ms_handle_connect on v2:172.21.15.165:6836/6215088 2024-09-11T00:06:35.084 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:35.076+0000 7fdae4c00640 1 -- 172.21.15.12:0/3110164658 <== mds.2 v2:172.21.15.165:6836/6215088 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x5632a734e750 con 0x5632a74680c0 2024-09-11T00:06:35.084 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:35.076+0000 7fdae4c00640 10 client.25316 handle_command_reply: tid=0 2024-09-11T00:06:35.084 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:35.076+0000 7fdb04407740 1 client.25316 shutdown 2024-09-11T00:06:35.084 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:35.076+0000 7fdb04407740 1 -- 172.21.15.12:0/3110164658 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5632a73d9240 msgr2=0x5632a73fa190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:06:35.084 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:35.076+0000 7fdb04407740 1 --2- 172.21.15.12:0/3110164658 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5632a73d9240 0x5632a73fa190 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fdaf4009a80 tx=0x7fdaf40096a0 comp rx=0 tx=0).stop 2024-09-11T00:06:35.084 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:35.076+0000 7fdb04407740 1 -- 172.21.15.12:0/3110164658 shutdown_connections 2024-09-11T00:06:35.084 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:35.076+0000 7fdb04407740 1 --2- 172.21.15.12:0/3110164658 >> [v2:172.21.15.165:6836/6215088,v1:172.21.15.165:6837/6215088] conn(0x5632a74680c0 0x5632a7466dd0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:35.084 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:35.076+0000 7fdb04407740 1 --2- 172.21.15.12:0/3110164658 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5632a73e5af0 0x5632a73ff8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:35.084 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:35.076+0000 7fdb04407740 1 --2- 172.21.15.12:0/3110164658 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5632a73d9b60 0x5632a73fa6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:35.084 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:35.076+0000 7fdae4c00640 0 client.25316 ms_handle_reset on v2:172.21.15.165:6836/6215088 2024-09-11T00:06:35.084 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:35.076+0000 7fdb04407740 1 --2- 172.21.15.12:0/3110164658 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5632a73d9240 0x5632a73fa190 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:35.084 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:35.076+0000 7fdb04407740 1 -- 172.21.15.12:0/3110164658 >> 172.21.15.12:0/3110164658 conn(0x5632a73d7020 msgr2=0x5632a73e12b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:06:35.085 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:35.076+0000 7fdb04407740 1 -- 172.21.15.12:0/3110164658 shutdown_connections 2024-09-11T00:06:35.085 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:35.076+0000 7fdb04407740 1 -- 172.21.15.12:0/3110164658 wait complete. 2024-09-11T00:06:35.085 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:35.076+0000 7fdb04407740 20 client.25316 trim_cache size 0 max 16384 2024-09-11T00:06:35.085 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-09-11T00:06:35.085 INFO:teuthology.orchestra.run.smithi012.stdout: "cluster_fsid": "ff3ad74a-6fcf-11ef-bcea-c7b262605968", 2024-09-11T00:06:35.085 INFO:teuthology.orchestra.run.smithi012.stdout: "whoami": 2, 2024-09-11T00:06:35.086 INFO:teuthology.orchestra.run.smithi012.stdout: "id": 24638, 2024-09-11T00:06:35.086 INFO:teuthology.orchestra.run.smithi012.stdout: "want_state": "up:active", 2024-09-11T00:06:35.086 INFO:teuthology.orchestra.run.smithi012.stdout: "state": "up:active", 2024-09-11T00:06:35.086 INFO:teuthology.orchestra.run.smithi012.stdout: "fs_name": "cephfs", 2024-09-11T00:06:35.086 INFO:teuthology.orchestra.run.smithi012.stdout: "rank_uptime": 107.84925765200001, 2024-09-11T00:06:35.086 INFO:teuthology.orchestra.run.smithi012.stdout: "mdsmap_epoch": 28, 2024-09-11T00:06:35.086 INFO:teuthology.orchestra.run.smithi012.stdout: "osdmap_epoch": 85, 2024-09-11T00:06:35.086 INFO:teuthology.orchestra.run.smithi012.stdout: "osdmap_epoch_barrier": 81, 2024-09-11T00:06:35.086 INFO:teuthology.orchestra.run.smithi012.stdout: "uptime": 175.56123855300001 2024-09-11T00:06:35.086 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-09-11T00:06:35.087 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:35.077+0000 7fdb01800640 1 -- 172.21.15.12:0/3967274721 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fdae00b5360 msgr2=0x7fdae00b7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:06:35.087 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:35.077+0000 7fdb01800640 1 --2- 172.21.15.12:0/3967274721 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fdae00b5360 0x7fdae00b7820 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fdaec009230 tx=0x7fdaec0e9780 comp rx=0 tx=0).stop 2024-09-11T00:06:35.087 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:35.077+0000 7fdb01800640 1 -- 172.21.15.12:0/3967274721 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdafc102da0 msgr2=0x7fdafc19b530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:06:35.087 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:35.077+0000 7fdb01800640 1 --2- 172.21.15.12:0/3967274721 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdafc102da0 0x7fdafc19b530 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fdaec00e9f0 tx=0x7fdaec00eeb0 comp rx=0 tx=0).stop 2024-09-11T00:06:35.087 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:35.077+0000 7fdb01800640 1 -- 172.21.15.12:0/3967274721 shutdown_connections 2024-09-11T00:06:35.087 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:35.077+0000 7fdb01800640 1 --2- 172.21.15.12:0/3967274721 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fdae00b5360 0x7fdae00b7820 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:35.087 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:35.077+0000 7fdb01800640 1 --2- 172.21.15.12:0/3967274721 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdafc109130 0x7fdafc1a0e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:35.088 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:35.077+0000 7fdb01800640 1 --2- 172.21.15.12:0/3967274721 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fdafc1087b0 0x7fdafc19ba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:35.088 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:35.077+0000 7fdb01800640 1 --2- 172.21.15.12:0/3967274721 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdafc102da0 0x7fdafc19b530 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:35.088 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:35.077+0000 7fdb01800640 1 -- 172.21.15.12:0/3967274721 >> 172.21.15.12:0/3967274721 conn(0x7fdafc0fe810 msgr2=0x7fdafc105d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:06:35.088 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:35.077+0000 7fdb01800640 1 -- 172.21.15.12:0/3967274721 shutdown_connections 2024-09-11T00:06:35.088 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:35.077+0000 7fdb01800640 1 -- 172.21.15.12:0/3967274721 wait complete. 2024-09-11T00:06:35.532 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:06:35 smithi165 ceph-mon[35356]: pgmap v383: 129 pgs: 129 active+clean; 647 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-09-11T00:06:35.562 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:06:35 smithi012 ceph-mon[30333]: pgmap v383: 129 pgs: 129 active+clean; 647 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-09-11T00:06:35.580 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:06:35 smithi178 ceph-mon[35919]: pgmap v383: 129 pgs: 129 active+clean; 647 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-09-11T00:06:35.814 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid ff3ad74a-6fcf-11ef-bcea-c7b262605968 -- ceph tell mds.g status 2024-09-11T00:06:36.199 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:06:36.761 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:36.758+0000 7fdca8000640 1 -- 172.21.15.12:0/2933439402 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdca0102dc0 msgr2=0x7fdca01031a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:06:36.761 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:36.758+0000 7fdca8000640 1 --2- 172.21.15.12:0/2933439402 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdca0102dc0 0x7fdca01031a0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fdc94009900 tx=0x7fdc9402a410 comp rx=0 tx=0).stop 2024-09-11T00:06:36.761 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:36.758+0000 7fdca8000640 1 -- 172.21.15.12:0/2933439402 shutdown_connections 2024-09-11T00:06:36.761 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:36.758+0000 7fdca8000640 1 --2- 172.21.15.12:0/2933439402 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdca0109150 0x7fdca010f950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:36.761 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:36.758+0000 7fdca8000640 1 --2- 172.21.15.12:0/2933439402 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fdca01087d0 0x7fdca0108c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:36.761 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:36.758+0000 7fdca8000640 1 --2- 172.21.15.12:0/2933439402 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdca0102dc0 0x7fdca01031a0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:36.761 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:36.758+0000 7fdca8000640 1 -- 172.21.15.12:0/2933439402 >> 172.21.15.12:0/2933439402 conn(0x7fdca00fe830 msgr2=0x7fdca0100c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:06:36.761 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:36.758+0000 7fdca8000640 1 -- 172.21.15.12:0/2933439402 shutdown_connections 2024-09-11T00:06:36.762 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:36.758+0000 7fdca8000640 1 -- 172.21.15.12:0/2933439402 wait complete. 2024-09-11T00:06:36.763 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:36.759+0000 7fdca8000640 1 Processor -- start 2024-09-11T00:06:36.763 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:36.759+0000 7fdca8000640 1 -- start start 2024-09-11T00:06:36.763 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:36.760+0000 7fdca8000640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdca0102dc0 0x7fdca019b4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:06:36.763 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:36.760+0000 7fdca8000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fdca01087d0 0x7fdca019ba10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:06:36.763 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:36.760+0000 7fdca8000640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdca0109150 0x7fdca01a0db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:06:36.764 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:36.760+0000 7fdca8000640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fdca0112e60 con 0x7fdca0109150 2024-09-11T00:06:36.764 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:36.760+0000 7fdca8000640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fdca0112ce0 con 0x7fdca01087d0 2024-09-11T00:06:36.764 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:36.760+0000 7fdca8000640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fdca0112fe0 con 0x7fdca0102dc0 2024-09-11T00:06:36.764 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:36.760+0000 7fdca5600640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdca0102dc0 0x7fdca019b4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:06:36.764 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:36.760+0000 7fdca5600640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdca0102dc0 0x7fdca019b4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.12:47206/0 (socket says 172.21.15.12:47206) 2024-09-11T00:06:36.764 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:36.760+0000 7fdca5600640 1 -- 172.21.15.12:0/474593094 learned_addr learned my addr 172.21.15.12:0/474593094 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:06:36.764 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:36.760+0000 7fdca4c00640 1 --2- 172.21.15.12:0/474593094 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fdca01087d0 0x7fdca019ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:06:36.764 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:36.760+0000 7fdca6000640 1 --2- 172.21.15.12:0/474593094 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdca0109150 0x7fdca01a0db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:06:36.764 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:36.761+0000 7fdca6000640 1 -- 172.21.15.12:0/474593094 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdca0102dc0 msgr2=0x7fdca019b4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:06:36.764 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:36.761+0000 7fdca6000640 1 --2- 172.21.15.12:0/474593094 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdca0102dc0 0x7fdca019b4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:36.764 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:36.761+0000 7fdca6000640 1 -- 172.21.15.12:0/474593094 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fdca01087d0 msgr2=0x7fdca019ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:06:36.765 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:36.761+0000 7fdca6000640 1 --2- 172.21.15.12:0/474593094 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fdca01087d0 0x7fdca019ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:36.765 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:36.761+0000 7fdca6000640 1 -- 172.21.15.12:0/474593094 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdca01a1580 con 0x7fdca0109150 2024-09-11T00:06:36.765 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:36.761+0000 7fdca5600640 1 --2- 172.21.15.12:0/474593094 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdca0102dc0 0x7fdca019b4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:06:36.765 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:36.761+0000 7fdca4c00640 1 --2- 172.21.15.12:0/474593094 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fdca01087d0 0x7fdca019ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:06:36.765 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:36.761+0000 7fdca6000640 1 --2- 172.21.15.12:0/474593094 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdca0109150 0x7fdca01a0db0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fdc9800da10 tx=0x7fdc9800ded0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:06:36.765 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:36.761+0000 7fdc9e000640 1 -- 172.21.15.12:0/474593094 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc9800bd70 con 0x7fdca0109150 2024-09-11T00:06:36.765 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:36.761+0000 7fdc9e000640 1 -- 172.21.15.12:0/474593094 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdc98003e20 con 0x7fdca0109150 2024-09-11T00:06:36.765 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:36.761+0000 7fdc9e000640 1 -- 172.21.15.12:0/474593094 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc980106c0 con 0x7fdca0109150 2024-09-11T00:06:36.765 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:36.761+0000 7fdca8000640 1 -- 172.21.15.12:0/474593094 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdca01a1870 con 0x7fdca0109150 2024-09-11T00:06:36.765 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:36.761+0000 7fdca8000640 1 -- 172.21.15.12:0/474593094 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdca01a7f30 con 0x7fdca0109150 2024-09-11T00:06:36.766 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:36.764+0000 7fdcaaa6f740 1 Processor -- start 2024-09-11T00:06:36.766 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:36.765+0000 7fdc9e000640 1 -- 172.21.15.12:0/474593094 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7fdc98024060 con 0x7fdca0109150 2024-09-11T00:06:36.767 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:36.765+0000 7fdcaaa6f740 1 -- start start 2024-09-11T00:06:36.768 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:36.765+0000 7fdcaaa6f740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56139d281240 0x56139d281620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:06:36.768 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:36.765+0000 7fdcaaa6f740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56139d281b60 0x56139d28d5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:06:36.768 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:36.765+0000 7fdcaaa6f740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56139d28daf0 0x56139d28fee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:06:36.768 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:36.765+0000 7fdcaaa6f740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x56139d1f8d50 con 0x56139d28daf0 2024-09-11T00:06:36.768 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:36.765+0000 7fdcaaa6f740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x56139d1e8180 con 0x56139d281b60 2024-09-11T00:06:36.768 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:36.765+0000 7fdcaaa6f740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x56139d1ebf90 con 0x56139d281240 2024-09-11T00:06:36.768 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:36.766+0000 7fdca4c00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56139d28daf0 0x56139d28fee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:06:36.768 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:36.766+0000 7fdca5600640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56139d281b60 0x56139d28d5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:06:36.769 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:36.766+0000 7fdca5600640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56139d281b60 0x56139d28d5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.12:51924/0 (socket says 172.21.15.12:51924) 2024-09-11T00:06:36.769 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:36.766+0000 7fdca4c00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56139d28daf0 0x56139d28fee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:43586/0 (socket says 172.21.15.12:43586) 2024-09-11T00:06:36.769 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:36.766+0000 7fdca5600640 1 -- 172.21.15.12:0/548782176 learned_addr learned my addr 172.21.15.12:0/548782176 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:06:36.769 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:36.766+0000 7fdca4c00640 1 --2- 172.21.15.12:0/548782176 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56139d281240 0x56139d281620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:06:36.770 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:36.766+0000 7fdca5600640 1 -- 172.21.15.12:0/548782176 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56139d281240 msgr2=0x56139d281620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:06:36.771 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:36.766+0000 7fdca5600640 1 --2- 172.21.15.12:0/548782176 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56139d281240 0x56139d281620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:36.771 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:36.766+0000 7fdca5600640 1 -- 172.21.15.12:0/548782176 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56139d28daf0 msgr2=0x56139d28fee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:06:36.771 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:36.766+0000 7fdca5600640 1 --2- 172.21.15.12:0/548782176 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56139d28daf0 0x56139d28fee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:36.771 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:36.766+0000 7fdca5600640 1 -- 172.21.15.12:0/548782176 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56139d1b2a70 con 0x56139d281b60 2024-09-11T00:06:36.771 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:36.766+0000 7fdca4c00640 1 --2- 172.21.15.12:0/548782176 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56139d28daf0 0x56139d28fee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:06:36.771 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:36.767+0000 7fdca5600640 1 --2- 172.21.15.12:0/548782176 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56139d281b60 0x56139d28d5b0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fdc94006010 tx=0x7fdc94004560 comp rx=0 tx=0).ready entity=mon.2 client_cookie=402455b3a25ce8c7 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:06:36.771 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:36.767+0000 7fdc9e000640 1 --2- 172.21.15.12:0/474593094 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fdc880b5360 0x7fdc880b7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:06:36.771 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:36.767+0000 7fdc9e000640 1 -- 172.21.15.12:0/474593094 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9208+0+0 (secure 0 0 0) 0x7fdc980e5ff0 con 0x7fdca0109150 2024-09-11T00:06:36.771 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:36.767+0000 7fdca4c00640 1 --2- 172.21.15.12:0/474593094 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fdc880b5360 0x7fdc880b7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:06:36.772 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:36.767+0000 7fdc92600640 1 -- 172.21.15.12:0/548782176 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc94040070 con 0x56139d281b60 2024-09-11T00:06:36.772 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:36.767+0000 7fdc92600640 1 -- 172.21.15.12:0/548782176 <== mon.2 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdc940047a0 con 0x56139d281b60 2024-09-11T00:06:36.772 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:36.767+0000 7fdc92600640 1 -- 172.21.15.12:0/548782176 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc94035040 con 0x56139d281b60 2024-09-11T00:06:36.772 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:36.768+0000 7fdcaaa6f740 1 -- 172.21.15.12:0/548782176 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56139d281b60 msgr2=0x56139d28d5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:06:36.772 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:36.768+0000 7fdca4c00640 1 --2- 172.21.15.12:0/474593094 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fdc880b5360 0x7fdc880b7820 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x56139d2811f0 tx=0x7fdc8c005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:06:36.772 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:36.768+0000 7fdcaaa6f740 1 --2- 172.21.15.12:0/548782176 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56139d281b60 0x56139d28d5b0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fdc94006010 tx=0x7fdc94004560 comp rx=0 tx=0).stop 2024-09-11T00:06:36.772 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:36.768+0000 7fdcaaa6f740 1 -- 172.21.15.12:0/548782176 shutdown_connections 2024-09-11T00:06:36.772 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:36.768+0000 7fdcaaa6f740 1 --2- 172.21.15.12:0/548782176 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56139d28daf0 0x56139d28fee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:36.772 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:36.768+0000 7fdcaaa6f740 1 --2- 172.21.15.12:0/548782176 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56139d281b60 0x56139d28d5b0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:36.772 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:36.768+0000 7fdcaaa6f740 1 --2- 172.21.15.12:0/548782176 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56139d281240 0x56139d281620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:36.772 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:36.768+0000 7fdcaaa6f740 1 -- 172.21.15.12:0/548782176 >> 172.21.15.12:0/548782176 conn(0x56139d27f020 msgr2=0x56139d280210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:06:36.773 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:36.768+0000 7fdcaaa6f740 1 -- 172.21.15.12:0/548782176 shutdown_connections 2024-09-11T00:06:36.773 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:36.768+0000 7fdcaaa6f740 1 -- 172.21.15.12:0/548782176 wait complete. 2024-09-11T00:06:36.773 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:36.769+0000 7fdcaaa6f740 1 -- start start 2024-09-11T00:06:36.773 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:36.769+0000 7fdcaaa6f740 1 Processor -- start 2024-09-11T00:06:36.776 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:36.769+0000 7fdcaaa6f740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56139d281240 0x56139d28aa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:06:36.776 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:36.769+0000 7fdcaaa6f740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56139d281b60 0x56139d28af60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:06:36.776 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:36.769+0000 7fdcaaa6f740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56139d28daf0 0x56139d28b4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:06:36.776 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:36.769+0000 7fdcaaa6f740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x56139d0495f0 con 0x56139d28daf0 2024-09-11T00:06:36.776 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:36.769+0000 7fdcaaa6f740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x56139d183460 con 0x56139d281240 2024-09-11T00:06:36.776 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:36.769+0000 7fdcaaa6f740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x56139d2907e0 con 0x56139d281b60 2024-09-11T00:06:36.776 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:36.769+0000 7fdcaaa6f740 10 client.0 _ll_register_callbacks cb 0x56139d193210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:06:36.776 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:36.770+0000 7fdca6000640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56139d28daf0 0x56139d28b4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:06:36.777 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:36.770+0000 7fdca6000640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56139d28daf0 0x56139d28b4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:43602/0 (socket says 172.21.15.12:43602) 2024-09-11T00:06:36.777 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:36.770+0000 7fdca6000640 1 -- 172.21.15.12:0/859338759 learned_addr learned my addr 172.21.15.12:0/859338759 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:06:36.777 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:36.770+0000 7fdca5600640 1 --2- 172.21.15.12:0/859338759 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56139d281240 0x56139d28aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:06:36.777 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:36.770+0000 7fdca4c00640 1 --2- 172.21.15.12:0/859338759 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56139d281b60 0x56139d28af60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:06:36.777 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:36.770+0000 7fdca6000640 1 -- 172.21.15.12:0/859338759 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56139d281b60 msgr2=0x56139d28af60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:06:36.777 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:36.770+0000 7fdca6000640 1 --2- 172.21.15.12:0/859338759 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56139d281b60 0x56139d28af60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:36.777 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:36.770+0000 7fdca6000640 1 -- 172.21.15.12:0/859338759 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56139d281240 msgr2=0x56139d28aa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:06:36.777 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:36.770+0000 7fdca6000640 1 --2- 172.21.15.12:0/859338759 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56139d281240 0x56139d28aa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:36.777 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:36.770+0000 7fdca6000640 1 -- 172.21.15.12:0/859338759 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56139cf46540 con 0x56139d28daf0 2024-09-11T00:06:36.777 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:36.770+0000 7fdca5600640 1 --2- 172.21.15.12:0/859338759 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56139d281240 0x56139d28aa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:06:36.777 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:36.770+0000 7fdca6000640 1 --2- 172.21.15.12:0/859338759 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56139d28daf0 0x56139d28b4a0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fdc980a8950 tx=0x7fdc980abaf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91d19b8a693136a7 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:06:36.778 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:36.770+0000 7fdc86a00640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-09-11T00:06:36.778 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:36.771+0000 7fdc86a00640 1 -- 172.21.15.12:0/859338759 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc980ac380 con 0x56139d28daf0 2024-09-11T00:06:36.778 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:36.771+0000 7fdcaaa6f740 1 -- 172.21.15.12:0/859338759 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x56139d185a70 con 0x56139d28daf0 2024-09-11T00:06:36.778 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:36.771+0000 7fdc86a00640 1 -- 172.21.15.12:0/859338759 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 9208+0+0 (secure 0 0 0) 0x7fdc98004270 con 0x56139d28daf0 2024-09-11T00:06:36.778 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:36.771+0000 7fdc86a00640 1 -- 172.21.15.12:0/859338759 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fdc980b0e60 con 0x56139d28daf0 2024-09-11T00:06:36.778 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:36.771+0000 7fdcaaa6f740 10 client.15288 fetch_fsmap learned FSMap version 28 2024-09-11T00:06:36.778 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:36.771+0000 7fdcaaa6f740 1 -- 172.21.15.12:0/859338759 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x56139d1a06f0 con 0x56139d28daf0 2024-09-11T00:06:36.778 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:36.771+0000 7fdc86a00640 1 -- 172.21.15.12:0/859338759 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x56139d185a70 con 0x56139d28daf0 2024-09-11T00:06:36.778 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:36.771+0000 7fdc86a00640 1 -- 172.21.15.12:0/859338759 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 28) ==== 8370+0+0 (secure 0 0 0) 0x7fdc980e3b80 con 0x56139d28daf0 2024-09-11T00:06:36.778 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:36.771+0000 7fdcaaa6f740 10 client.15288 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T00:06:36.778 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:36.771+0000 7fdcaaa6f740 10 client.15288 resolve_mds: resolved name 'g' to daemon mds.g 2024-09-11T00:06:36.779 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:36.771+0000 7fdcaaa6f740 20 client.15288 populate_metadata read hostname 'smithi012' 2024-09-11T00:06:36.779 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:36.771+0000 7fdcaaa6f740 1 --2- 172.21.15.12:0/859338759 >> [v2:172.21.15.12:6838/3124093009,v1:172.21.15.12:6839/3124093009] conn(0x56139d29e7d0 0x56139d29fc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:06:36.779 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:36.772+0000 7fdcaaa6f740 4 client.15288 mds_command: new command op to 24643 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T00:06:36.779 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:36.772+0000 7fdcaaa6f740 1 -- 172.21.15.12:0/859338759 --> [v2:172.21.15.12:6838/3124093009,v1:172.21.15.12:6839/3124093009] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56139d1faca0 con 0x56139d29e7d0 2024-09-11T00:06:36.779 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:36.772+0000 7fdca5600640 1 --2- 172.21.15.12:0/859338759 >> [v2:172.21.15.12:6838/3124093009,v1:172.21.15.12:6839/3124093009] conn(0x56139d29e7d0 0x56139d29fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:06:36.779 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:36.772+0000 7fdca5600640 1 --2- 172.21.15.12:0/859338759 >> [v2:172.21.15.12:6838/3124093009,v1:172.21.15.12:6839/3124093009] conn(0x56139d29e7d0 0x56139d29fc40 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=37abe6e08ed65acc server_cookie=a92cda485aacab88 in_seq=0 out_seq=0 2024-09-11T00:06:36.779 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:36.772+0000 7fdc86a00640 10 client.15288 ms_handle_connect on v2:172.21.15.12:6838/3124093009 2024-09-11T00:06:36.779 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:36.773+0000 7fdc86a00640 1 -- 172.21.15.12:0/859338759 <== mds.1 v2:172.21.15.12:6838/3124093009 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x56139d1faca0 con 0x56139d29e7d0 2024-09-11T00:06:36.779 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:36.773+0000 7fdc86a00640 10 client.15288 handle_command_reply: tid=0 2024-09-11T00:06:36.779 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:36.773+0000 7fdcaaa6f740 1 client.15288 shutdown 2024-09-11T00:06:36.779 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:36.773+0000 7fdcaaa6f740 1 -- 172.21.15.12:0/859338759 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56139d28daf0 msgr2=0x56139d28b4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:06:36.780 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:36.773+0000 7fdcaaa6f740 1 --2- 172.21.15.12:0/859338759 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56139d28daf0 0x56139d28b4a0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fdc980a8950 tx=0x7fdc980abaf0 comp rx=0 tx=0).stop 2024-09-11T00:06:36.780 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:36.774+0000 7fdcaaa6f740 1 -- 172.21.15.12:0/859338759 shutdown_connections 2024-09-11T00:06:36.780 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:36.774+0000 7fdcaaa6f740 1 --2- 172.21.15.12:0/859338759 >> [v2:172.21.15.12:6838/3124093009,v1:172.21.15.12:6839/3124093009] conn(0x56139d29e7d0 0x56139d29fc40 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:36.780 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:36.774+0000 7fdcaaa6f740 1 --2- 172.21.15.12:0/859338759 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56139d28daf0 0x56139d28b4a0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:36.780 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:36.774+0000 7fdcaaa6f740 1 --2- 172.21.15.12:0/859338759 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56139d281b60 0x56139d28af60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:36.780 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:36.774+0000 7fdcaaa6f740 1 --2- 172.21.15.12:0/859338759 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56139d281240 0x56139d28aa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:36.780 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:36.774+0000 7fdcaaa6f740 1 -- 172.21.15.12:0/859338759 >> 172.21.15.12:0/859338759 conn(0x56139d27f020 msgr2=0x56139d284c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:06:36.780 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:36.774+0000 7fdc86a00640 0 client.15288 ms_handle_reset on v2:172.21.15.12:6838/3124093009 2024-09-11T00:06:36.780 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:36.774+0000 7fdcaaa6f740 1 -- 172.21.15.12:0/859338759 shutdown_connections 2024-09-11T00:06:36.780 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:36.774+0000 7fdcaaa6f740 1 -- 172.21.15.12:0/859338759 wait complete. 2024-09-11T00:06:36.780 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:36.774+0000 7fdcaaa6f740 20 client.15288 trim_cache size 0 max 16384 2024-09-11T00:06:36.795 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:36.793+0000 7fdcaaa6f740 1 Processor -- start 2024-09-11T00:06:36.795 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:36.793+0000 7fdcaaa6f740 1 -- start start 2024-09-11T00:06:36.795 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:36.793+0000 7fdcaaa6f740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56139d281240 0x56139d29e100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:06:36.795 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:36.793+0000 7fdcaaa6f740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56139d281b60 0x56139d29fc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:06:36.795 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:36.793+0000 7fdcaaa6f740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56139d28daf0 0x56139d285cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:06:36.795 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:36.793+0000 7fdcaaa6f740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x56139d29df80 con 0x56139d281240 2024-09-11T00:06:36.795 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:36.793+0000 7fdcaaa6f740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x56139d29e650 con 0x56139d281b60 2024-09-11T00:06:36.795 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:36.793+0000 7fdcaaa6f740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x56139d29ed20 con 0x56139d28daf0 2024-09-11T00:06:36.796 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:36.793+0000 7fdca5600640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56139d281240 0x56139d29e100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:06:36.797 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:36.794+0000 7fdca5600640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56139d281240 0x56139d29e100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:43604/0 (socket says 172.21.15.12:43604) 2024-09-11T00:06:36.797 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:36.794+0000 7fdca4c00640 1 --2- 172.21.15.12:0/623630327 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56139d281b60 0x56139d29fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:06:36.797 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:36.794+0000 7fdca5600640 1 -- 172.21.15.12:0/623630327 learned_addr learned my addr 172.21.15.12:0/623630327 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:06:36.797 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:36.794+0000 7fdca6000640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56139d28daf0 0x56139d285cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:06:36.797 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:36.794+0000 7fdca5600640 1 -- 172.21.15.12:0/623630327 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56139d28daf0 msgr2=0x56139d285cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:06:36.797 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:36.794+0000 7fdca5600640 1 --2- 172.21.15.12:0/623630327 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56139d28daf0 0x56139d285cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:36.797 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:36.794+0000 7fdca5600640 1 -- 172.21.15.12:0/623630327 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56139d281b60 msgr2=0x56139d29fc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:06:36.797 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:36.794+0000 7fdca5600640 1 --2- 172.21.15.12:0/623630327 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56139d281b60 0x56139d29fc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:36.797 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:36.794+0000 7fdca5600640 1 -- 172.21.15.12:0/623630327 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56139d2907e0 con 0x56139d281240 2024-09-11T00:06:36.797 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:36.794+0000 7fdca5600640 1 --2- 172.21.15.12:0/623630327 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56139d281240 0x56139d29e100 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fdc94004650 tx=0x7fdc9402a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b62f9e3b07beb61f server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:06:36.797 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:36.795+0000 7fdc91c00640 1 -- 172.21.15.12:0/623630327 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc94040040 con 0x56139d281240 2024-09-11T00:06:36.798 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:36.795+0000 7fdc91c00640 1 -- 172.21.15.12:0/623630327 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdc94003c60 con 0x56139d281240 2024-09-11T00:06:36.798 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:36.795+0000 7fdc91c00640 1 -- 172.21.15.12:0/623630327 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc94035040 con 0x56139d281240 2024-09-11T00:06:36.800 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:36.795+0000 7fdcaaa6f740 1 -- 172.21.15.12:0/623630327 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56139d281240 msgr2=0x56139d29e100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:06:36.800 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:36.795+0000 7fdcaaa6f740 1 --2- 172.21.15.12:0/623630327 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56139d281240 0x56139d29e100 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fdc94004650 tx=0x7fdc9402a410 comp rx=0 tx=0).stop 2024-09-11T00:06:36.800 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:36.795+0000 7fdcaaa6f740 1 -- 172.21.15.12:0/623630327 shutdown_connections 2024-09-11T00:06:36.800 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:36.795+0000 7fdcaaa6f740 1 --2- 172.21.15.12:0/623630327 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56139d28daf0 0x56139d285cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:36.800 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:36.795+0000 7fdcaaa6f740 1 --2- 172.21.15.12:0/623630327 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56139d281b60 0x56139d29fc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:36.800 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:36.795+0000 7fdcaaa6f740 1 --2- 172.21.15.12:0/623630327 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56139d281240 0x56139d29e100 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:36.800 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:36.795+0000 7fdcaaa6f740 1 -- 172.21.15.12:0/623630327 >> 172.21.15.12:0/623630327 conn(0x56139d27f020 msgr2=0x56139d279ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:06:36.800 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:36.795+0000 7fdcaaa6f740 1 -- 172.21.15.12:0/623630327 shutdown_connections 2024-09-11T00:06:36.800 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:36.795+0000 7fdcaaa6f740 1 -- 172.21.15.12:0/623630327 wait complete. 2024-09-11T00:06:36.800 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:36.796+0000 7fdcaaa6f740 1 -- start start 2024-09-11T00:06:36.801 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:36.796+0000 7fdcaaa6f740 1 Processor -- start 2024-09-11T00:06:36.801 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:36.796+0000 7fdcaaa6f740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56139d281240 0x56139d2a82c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:06:36.801 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:36.796+0000 7fdcaaa6f740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56139d281b60 0x56139d2a8820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:06:36.801 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:36.796+0000 7fdcaaa6f740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56139d28daf0 0x56139d29d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:06:36.801 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:36.796+0000 7fdcaaa6f740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x56139d29fac0 con 0x56139d281240 2024-09-11T00:06:36.801 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:36.796+0000 7fdcaaa6f740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x56139d29f3f0 con 0x56139d281b60 2024-09-11T00:06:36.801 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:36.797+0000 7fdcaaa6f740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x56139d2a0190 con 0x56139d28daf0 2024-09-11T00:06:36.801 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:36.797+0000 7fdcaaa6f740 10 client.0 _ll_register_callbacks cb 0x7fdc980ac220 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:06:36.801 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:36.797+0000 7fdca5600640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56139d281240 0x56139d2a82c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:06:36.801 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:36.797+0000 7fdca5600640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56139d281240 0x56139d2a82c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:43612/0 (socket says 172.21.15.12:43612) 2024-09-11T00:06:36.801 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:36.797+0000 7fdca5600640 1 -- 172.21.15.12:0/2975128826 learned_addr learned my addr 172.21.15.12:0/2975128826 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:06:36.801 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:36.797+0000 7fdca6000640 1 --2- 172.21.15.12:0/2975128826 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56139d28daf0 0x56139d29d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:06:36.802 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:36.797+0000 7fdca4c00640 1 --2- 172.21.15.12:0/2975128826 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56139d281b60 0x56139d2a8820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:06:36.802 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:36.797+0000 7fdca5600640 1 -- 172.21.15.12:0/2975128826 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56139d28daf0 msgr2=0x56139d29d7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:06:36.802 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:36.797+0000 7fdca5600640 1 --2- 172.21.15.12:0/2975128826 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56139d28daf0 0x56139d29d7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:36.802 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:36.797+0000 7fdca5600640 1 -- 172.21.15.12:0/2975128826 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56139d281b60 msgr2=0x56139d2a8820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:06:36.802 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:36.797+0000 7fdca5600640 1 --2- 172.21.15.12:0/2975128826 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56139d281b60 0x56139d2a8820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:36.802 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:36.797+0000 7fdca5600640 1 -- 172.21.15.12:0/2975128826 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56139d29dff0 con 0x56139d281240 2024-09-11T00:06:36.802 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:36.798+0000 7fdca5600640 1 --2- 172.21.15.12:0/2975128826 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56139d281240 0x56139d2a82c0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fdc94005a60 tx=0x7fdc94005e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e714d33c7424de9 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:06:36.802 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:36.798+0000 7fdc86a00640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-09-11T00:06:36.802 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:36.798+0000 7fdc86a00640 1 -- 172.21.15.12:0/2975128826 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc94060060 con 0x56139d281240 2024-09-11T00:06:36.802 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:36.798+0000 7fdc86a00640 1 -- 172.21.15.12:0/2975128826 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 9208+0+0 (secure 0 0 0) 0x7fdc940314a0 con 0x56139d281240 2024-09-11T00:06:36.802 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:36.798+0000 7fdcaaa6f740 1 -- 172.21.15.12:0/2975128826 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdc94003c60 con 0x56139d281240 2024-09-11T00:06:36.804 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:36.798+0000 7fdc86a00640 1 -- 172.21.15.12:0/2975128826 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fdc94067050 con 0x56139d281240 2024-09-11T00:06:36.805 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:36.798+0000 7fdcaaa6f740 10 client.15294 fetch_fsmap learned FSMap version 28 2024-09-11T00:06:36.805 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:36.798+0000 7fdcaaa6f740 1 -- 172.21.15.12:0/2975128826 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x56139d2a6ae0 con 0x56139d281240 2024-09-11T00:06:36.805 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:36.799+0000 7fdc86a00640 1 -- 172.21.15.12:0/2975128826 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdc94003c60 con 0x56139d281240 2024-09-11T00:06:36.805 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:36.799+0000 7fdc86a00640 1 -- 172.21.15.12:0/2975128826 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 28) ==== 8370+0+0 (secure 0 0 0) 0x7fdc9403c440 con 0x56139d281240 2024-09-11T00:06:36.805 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:36.799+0000 7fdcaaa6f740 10 client.15294 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T00:06:36.805 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:36.799+0000 7fdcaaa6f740 10 client.15294 resolve_mds: resolved name 'g' to daemon mds.g 2024-09-11T00:06:36.805 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:36.799+0000 7fdcaaa6f740 20 client.15294 populate_metadata read hostname 'smithi012' 2024-09-11T00:06:36.805 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:36.799+0000 7fdcaaa6f740 1 --2- 172.21.15.12:0/2975128826 >> [v2:172.21.15.12:6838/3124093009,v1:172.21.15.12:6839/3124093009] conn(0x56139d30f3c0 0x56139d30e0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:06:36.805 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:36.799+0000 7fdcaaa6f740 4 client.15294 mds_command: new command op to 24643 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-11T00:06:36.805 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:36.799+0000 7fdcaaa6f740 1 -- 172.21.15.12:0/2975128826 --> [v2:172.21.15.12:6838/3124093009,v1:172.21.15.12:6839/3124093009] -- command(tid 0: {"prefix": "status"}) -- 0x56139d1f6750 con 0x56139d30f3c0 2024-09-11T00:06:36.805 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:36.799+0000 7fdca4c00640 1 --2- 172.21.15.12:0/2975128826 >> [v2:172.21.15.12:6838/3124093009,v1:172.21.15.12:6839/3124093009] conn(0x56139d30f3c0 0x56139d30e0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:06:36.806 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:36.800+0000 7fdca4c00640 1 --2- 172.21.15.12:0/2975128826 >> [v2:172.21.15.12:6838/3124093009,v1:172.21.15.12:6839/3124093009] conn(0x56139d30f3c0 0x56139d30e0d0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=84ee742d7263318d server_cookie=f25fcae014850f80 in_seq=0 out_seq=0 2024-09-11T00:06:36.806 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:36.800+0000 7fdc86a00640 10 client.15294 ms_handle_connect on v2:172.21.15.12:6838/3124093009 2024-09-11T00:06:36.806 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:36.800+0000 7fdc86a00640 1 -- 172.21.15.12:0/2975128826 <== mds.1 v2:172.21.15.12:6838/3124093009 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x56139d1f6750 con 0x56139d30f3c0 2024-09-11T00:06:36.806 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:36.800+0000 7fdc86a00640 10 client.15294 handle_command_reply: tid=0 2024-09-11T00:06:36.806 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:36.800+0000 7fdcaaa6f740 1 client.15294 shutdown 2024-09-11T00:06:36.806 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:36.800+0000 7fdcaaa6f740 1 -- 172.21.15.12:0/2975128826 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56139d281240 msgr2=0x56139d2a82c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:06:36.806 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:36.800+0000 7fdcaaa6f740 1 --2- 172.21.15.12:0/2975128826 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56139d281240 0x56139d2a82c0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fdc94005a60 tx=0x7fdc94005e40 comp rx=0 tx=0).stop 2024-09-11T00:06:36.806 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:36.800+0000 7fdcaaa6f740 1 -- 172.21.15.12:0/2975128826 shutdown_connections 2024-09-11T00:06:36.806 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:36.800+0000 7fdcaaa6f740 1 --2- 172.21.15.12:0/2975128826 >> [v2:172.21.15.12:6838/3124093009,v1:172.21.15.12:6839/3124093009] conn(0x56139d30f3c0 0x56139d30e0d0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:36.806 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:36.800+0000 7fdcaaa6f740 1 --2- 172.21.15.12:0/2975128826 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56139d28daf0 0x56139d29d7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:36.806 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:36.800+0000 7fdcaaa6f740 1 --2- 172.21.15.12:0/2975128826 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56139d281b60 0x56139d2a8820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:36.806 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:36.800+0000 7fdcaaa6f740 1 --2- 172.21.15.12:0/2975128826 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56139d281240 0x56139d2a82c0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:36.807 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:36.800+0000 7fdcaaa6f740 1 -- 172.21.15.12:0/2975128826 >> 172.21.15.12:0/2975128826 conn(0x56139d27f020 msgr2=0x56139d298aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:06:36.807 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:36.800+0000 7fdc86a00640 0 client.15294 ms_handle_reset on v2:172.21.15.12:6838/3124093009 2024-09-11T00:06:36.807 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:36.801+0000 7fdcaaa6f740 1 -- 172.21.15.12:0/2975128826 shutdown_connections 2024-09-11T00:06:36.807 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:36.801+0000 7fdcaaa6f740 1 -- 172.21.15.12:0/2975128826 wait complete. 2024-09-11T00:06:36.807 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:36.801+0000 7fdcaaa6f740 20 client.15294 trim_cache size 0 max 16384 2024-09-11T00:06:36.807 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-09-11T00:06:36.807 INFO:teuthology.orchestra.run.smithi012.stdout: "cluster_fsid": "ff3ad74a-6fcf-11ef-bcea-c7b262605968", 2024-09-11T00:06:36.807 INFO:teuthology.orchestra.run.smithi012.stdout: "whoami": 1, 2024-09-11T00:06:36.807 INFO:teuthology.orchestra.run.smithi012.stdout: "id": 24643, 2024-09-11T00:06:36.808 INFO:teuthology.orchestra.run.smithi012.stdout: "want_state": "up:active", 2024-09-11T00:06:36.808 INFO:teuthology.orchestra.run.smithi012.stdout: "state": "up:active", 2024-09-11T00:06:36.808 INFO:teuthology.orchestra.run.smithi012.stdout: "fs_name": "cephfs", 2024-09-11T00:06:36.808 INFO:teuthology.orchestra.run.smithi012.stdout: "rank_uptime": 110.57982939199999, 2024-09-11T00:06:36.808 INFO:teuthology.orchestra.run.smithi012.stdout: "mdsmap_epoch": 28, 2024-09-11T00:06:36.808 INFO:teuthology.orchestra.run.smithi012.stdout: "osdmap_epoch": 85, 2024-09-11T00:06:36.808 INFO:teuthology.orchestra.run.smithi012.stdout: "osdmap_epoch_barrier": 81, 2024-09-11T00:06:36.808 INFO:teuthology.orchestra.run.smithi012.stdout: "uptime": 171.233935172 2024-09-11T00:06:36.808 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-09-11T00:06:36.809 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:36.801+0000 7fdca8000640 1 -- 172.21.15.12:0/474593094 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fdc880b5360 msgr2=0x7fdc880b7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:06:36.809 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:36.801+0000 7fdca8000640 1 --2- 172.21.15.12:0/474593094 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fdc880b5360 0x7fdc880b7820 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x56139d2811f0 tx=0x7fdc8c005850 comp rx=0 tx=0).stop 2024-09-11T00:06:36.809 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:36.801+0000 7fdca8000640 1 -- 172.21.15.12:0/474593094 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdca0109150 msgr2=0x7fdca01a0db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:06:36.810 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:36.801+0000 7fdca8000640 1 --2- 172.21.15.12:0/474593094 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdca0109150 0x7fdca01a0db0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fdc9800da10 tx=0x7fdc9800ded0 comp rx=0 tx=0).stop 2024-09-11T00:06:36.810 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:36.801+0000 7fdca8000640 1 -- 172.21.15.12:0/474593094 shutdown_connections 2024-09-11T00:06:36.810 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:36.801+0000 7fdca8000640 1 --2- 172.21.15.12:0/474593094 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fdc880b5360 0x7fdc880b7820 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:36.810 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:36.801+0000 7fdca8000640 1 --2- 172.21.15.12:0/474593094 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdca0109150 0x7fdca01a0db0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:36.810 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:36.801+0000 7fdca8000640 1 --2- 172.21.15.12:0/474593094 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fdca01087d0 0x7fdca019ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:36.810 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:36.801+0000 7fdca8000640 1 --2- 172.21.15.12:0/474593094 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdca0102dc0 0x7fdca019b4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:36.810 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:36.801+0000 7fdca8000640 1 -- 172.21.15.12:0/474593094 >> 172.21.15.12:0/474593094 conn(0x7fdca00fe830 msgr2=0x7fdca00fff10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:06:36.810 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:36.802+0000 7fdca8000640 1 -- 172.21.15.12:0/474593094 shutdown_connections 2024-09-11T00:06:36.810 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:36.802+0000 7fdca8000640 1 -- 172.21.15.12:0/474593094 wait complete. 2024-09-11T00:06:37.533 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:06:37 smithi165 ceph-mon[35356]: pgmap v384: 129 pgs: 129 active+clean; 647 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-11T00:06:37.534 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:06:37 smithi012 ceph-mon[30333]: pgmap v384: 129 pgs: 129 active+clean; 647 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-11T00:06:37.572 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid ff3ad74a-6fcf-11ef-bcea-c7b262605968 -- ceph tell mds.b status 2024-09-11T00:06:37.580 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:06:37 smithi178 ceph-mon[35919]: pgmap v384: 129 pgs: 129 active+clean; 647 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-11T00:06:37.955 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:06:38.488 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:38.485+0000 7f7614800640 1 -- 172.21.15.12:0/1071643516 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f760c1018d0 msgr2=0x7f760c101cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:06:38.488 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:38.485+0000 7f7614800640 1 --2- 172.21.15.12:0/1071643516 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f760c1018d0 0x7f760c101cb0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f7600009900 tx=0x7f760002a480 comp rx=0 tx=0).stop 2024-09-11T00:06:38.488 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:38.485+0000 7f7614800640 1 -- 172.21.15.12:0/1071643516 shutdown_connections 2024-09-11T00:06:38.488 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:38.485+0000 7f7614800640 1 --2- 172.21.15.12:0/1071643516 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f760c10f480 0x7f760c111870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:38.488 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:38.485+0000 7f7614800640 1 --2- 172.21.15.12:0/1071643516 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f760c1021f0 0x7f760c10ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:38.488 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:38.485+0000 7f7614800640 1 --2- 172.21.15.12:0/1071643516 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f760c1018d0 0x7f760c101cb0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:38.488 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:38.485+0000 7f7614800640 1 -- 172.21.15.12:0/1071643516 >> 172.21.15.12:0/1071643516 conn(0x7f760c0fd700 msgr2=0x7f760c0ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:06:38.488 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:38.485+0000 7f7614800640 1 -- 172.21.15.12:0/1071643516 shutdown_connections 2024-09-11T00:06:38.488 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:38.485+0000 7f7614800640 1 -- 172.21.15.12:0/1071643516 wait complete. 2024-09-11T00:06:38.489 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:38.486+0000 7f7614800640 1 Processor -- start 2024-09-11T00:06:38.489 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:38.486+0000 7f7614800640 1 -- start start 2024-09-11T00:06:38.490 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:38.487+0000 7f7614800640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f760c1018d0 0x7f760c19b4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:06:38.490 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:38.487+0000 7f7614800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f760c1021f0 0x7f760c19ba30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:06:38.490 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:38.487+0000 7f7614800640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f760c10f480 0x7f760c1a0dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:06:38.490 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:38.487+0000 7f7614800640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f760c114f60 con 0x7f760c10f480 2024-09-11T00:06:38.490 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:38.487+0000 7f7614800640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f760c114de0 con 0x7f760c1021f0 2024-09-11T00:06:38.490 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:38.487+0000 7f7614800640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f760c1150e0 con 0x7f760c1018d0 2024-09-11T00:06:38.490 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:38.487+0000 7f7611e00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f760c1018d0 0x7f760c19b4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:06:38.490 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:38.487+0000 7f7611e00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f760c1018d0 0x7f760c19b4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.12:47246/0 (socket says 172.21.15.12:47246) 2024-09-11T00:06:38.490 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:38.487+0000 7f7611e00640 1 -- 172.21.15.12:0/1876341730 learned_addr learned my addr 172.21.15.12:0/1876341730 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:06:38.490 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:38.487+0000 7f7612800640 1 --2- 172.21.15.12:0/1876341730 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f760c10f480 0x7f760c1a0dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:06:38.490 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:38.488+0000 7f7611e00640 1 -- 172.21.15.12:0/1876341730 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f760c1021f0 msgr2=0x7f760c19ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:06:38.490 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:38.488+0000 7f7611e00640 1 --2- 172.21.15.12:0/1876341730 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f760c1021f0 0x7f760c19ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:38.490 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:38.488+0000 7f7611e00640 1 -- 172.21.15.12:0/1876341730 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f760c10f480 msgr2=0x7f760c1a0dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:06:38.491 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:38.488+0000 7f7611e00640 1 --2- 172.21.15.12:0/1876341730 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f760c10f480 0x7f760c1a0dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:38.491 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:38.488+0000 7f7611e00640 1 -- 172.21.15.12:0/1876341730 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f760c1a15a0 con 0x7f760c1018d0 2024-09-11T00:06:38.491 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:38.488+0000 7f7611e00640 1 --2- 172.21.15.12:0/1876341730 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f760c1018d0 0x7f760c19b4f0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f7600005940 tx=0x7f76000040c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:06:38.491 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:38.488+0000 7f7606a00640 1 -- 172.21.15.12:0/1876341730 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f760003f030 con 0x7f760c1018d0 2024-09-11T00:06:38.491 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:38.488+0000 7f7606a00640 1 -- 172.21.15.12:0/1876341730 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f76000044c0 con 0x7f760c1018d0 2024-09-11T00:06:38.491 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:38.488+0000 7f7614800640 1 -- 172.21.15.12:0/1876341730 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f760c1a1830 con 0x7f760c1018d0 2024-09-11T00:06:38.492 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:38.489+0000 7f7606a00640 1 -- 172.21.15.12:0/1876341730 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7600035070 con 0x7f760c1018d0 2024-09-11T00:06:38.492 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:38.489+0000 7f7614800640 1 -- 172.21.15.12:0/1876341730 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f760c103ae0 con 0x7f760c1018d0 2024-09-11T00:06:38.493 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:38.492+0000 7f7617259740 1 Processor -- start 2024-09-11T00:06:38.494 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:38.492+0000 7f7617259740 1 -- start start 2024-09-11T00:06:38.494 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:38.492+0000 7f7617259740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563e64c0c240 0x563e64c0c620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:06:38.494 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:38.492+0000 7f7617259740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563e64c0cb60 0x563e64c185b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:06:38.494 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:38.492+0000 7f7617259740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563e64c18af0 0x563e64c1aee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:06:38.494 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:38.492+0000 7f7617259740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x563e64b83d50 con 0x563e64c0cb60 2024-09-11T00:06:38.494 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:38.492+0000 7f7617259740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x563e64b73180 con 0x563e64c18af0 2024-09-11T00:06:38.494 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:38.492+0000 7f7617259740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x563e64b76f90 con 0x563e64c0c240 2024-09-11T00:06:38.495 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:38.493+0000 7f7612800640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563e64c0c240 0x563e64c0c620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:06:38.496 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:38.493+0000 7f7612800640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563e64c0c240 0x563e64c0c620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.12:47258/0 (socket says 172.21.15.12:47258) 2024-09-11T00:06:38.496 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:38.493+0000 7f7612800640 1 -- 172.21.15.12:0/3101362488 learned_addr learned my addr 172.21.15.12:0/3101362488 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:06:38.496 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:38.493+0000 7f7611400640 1 --2- 172.21.15.12:0/3101362488 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563e64c18af0 0x563e64c1aee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:06:38.496 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:38.493+0000 7f7606a00640 1 -- 172.21.15.12:0/1876341730 <== mon.1 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7f760003b9f0 con 0x7f760c1018d0 2024-09-11T00:06:38.496 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:38.493+0000 7f7612800640 1 -- 172.21.15.12:0/3101362488 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563e64c18af0 msgr2=0x563e64c1aee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:06:38.496 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:38.493+0000 7f7612800640 1 --2- 172.21.15.12:0/3101362488 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563e64c18af0 0x563e64c1aee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:38.496 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:38.493+0000 7f7612800640 1 -- 172.21.15.12:0/3101362488 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563e64c0cb60 msgr2=0x563e64c185b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:06:38.496 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:38.493+0000 7f7612800640 1 --2- 172.21.15.12:0/3101362488 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563e64c0cb60 0x563e64c185b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:38.497 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:38.493+0000 7f7612800640 1 -- 172.21.15.12:0/3101362488 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563e64b3da70 con 0x563e64c0c240 2024-09-11T00:06:38.497 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:38.494+0000 7f7612800640 1 --2- 172.21.15.12:0/3101362488 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563e64c0c240 0x563e64c0c620 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f7608010400 tx=0x7f76080108c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=665710a5a2b1318f server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:06:38.497 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:38.494+0000 7f75fb000640 1 -- 172.21.15.12:0/3101362488 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7608003b70 con 0x563e64c0c240 2024-09-11T00:06:38.497 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:38.494+0000 7f75fb000640 1 -- 172.21.15.12:0/3101362488 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7608003d10 con 0x563e64c0c240 2024-09-11T00:06:38.497 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:38.494+0000 7f75fb000640 1 -- 172.21.15.12:0/3101362488 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7608011b00 con 0x563e64c0c240 2024-09-11T00:06:38.498 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:38.494+0000 7f7617259740 1 -- 172.21.15.12:0/3101362488 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563e64c0c240 msgr2=0x563e64c0c620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:06:38.498 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:38.494+0000 7f7617259740 1 --2- 172.21.15.12:0/3101362488 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563e64c0c240 0x563e64c0c620 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f7608010400 tx=0x7f76080108c0 comp rx=0 tx=0).stop 2024-09-11T00:06:38.498 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:38.495+0000 7f7617259740 1 -- 172.21.15.12:0/3101362488 shutdown_connections 2024-09-11T00:06:38.498 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:38.495+0000 7f7617259740 1 --2- 172.21.15.12:0/3101362488 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563e64c18af0 0x563e64c1aee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:38.498 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:38.495+0000 7f7617259740 1 --2- 172.21.15.12:0/3101362488 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563e64c0cb60 0x563e64c185b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:38.498 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:38.495+0000 7f7617259740 1 --2- 172.21.15.12:0/3101362488 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563e64c0c240 0x563e64c0c620 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:38.498 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:38.495+0000 7f7617259740 1 -- 172.21.15.12:0/3101362488 >> 172.21.15.12:0/3101362488 conn(0x563e64c0a020 msgr2=0x563e64c0b210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:06:38.498 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:38.495+0000 7f7617259740 1 -- 172.21.15.12:0/3101362488 shutdown_connections 2024-09-11T00:06:38.498 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:38.495+0000 7f7617259740 1 -- 172.21.15.12:0/3101362488 wait complete. 2024-09-11T00:06:38.498 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:38.495+0000 7f7617259740 1 -- start start 2024-09-11T00:06:38.498 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:38.495+0000 7f7606a00640 1 --2- 172.21.15.12:0/1876341730 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f75f40b5280 0x7f75f40b7740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:06:38.498 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:38.496+0000 7f7617259740 1 Processor -- start 2024-09-11T00:06:38.499 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:38.496+0000 7f7606a00640 1 -- 172.21.15.12:0/1876341730 <== mon.1 v2:172.21.15.178:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9208+0+0 (secure 0 0 0) 0x7f7600106430 con 0x7f760c1018d0 2024-09-11T00:06:38.499 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:38.496+0000 7f7612800640 1 --2- 172.21.15.12:0/1876341730 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f75f40b5280 0x7f75f40b7740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:06:38.500 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:38.496+0000 7f7617259740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563e64c0c240 0x563e64c15ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:06:38.501 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:38.496+0000 7f7617259740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563e64c0cb60 0x563e64c16020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:06:38.501 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:38.496+0000 7f7617259740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563e64c18af0 0x563e64c16560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:06:38.501 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:38.496+0000 7f7617259740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x563e649d45f0 con 0x563e64c0c240 2024-09-11T00:06:38.501 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:38.496+0000 7f7617259740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x563e64b0e460 con 0x563e64c0cb60 2024-09-11T00:06:38.501 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:38.496+0000 7f7617259740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x563e64c1b840 con 0x563e64c18af0 2024-09-11T00:06:38.501 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:38.496+0000 7f7617259740 10 client.0 _ll_register_callbacks cb 0x563e64b1e210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:06:38.501 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:38.497+0000 7f7611e00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563e64c0c240 0x563e64c15ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:06:38.501 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:38.497+0000 7f7611400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563e64c0cb60 0x563e64c16020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:06:38.501 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:38.497+0000 7f7612800640 1 --2- 172.21.15.12:0/1876341730 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f75f40b5280 0x7f75f40b7740 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f7608009dd0 tx=0x7f7608005fd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:06:38.501 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:38.497+0000 7f7612800640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563e64c18af0 0x563e64c16560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:06:38.501 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:38.497+0000 7f7611e00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563e64c0c240 0x563e64c15ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:43662/0 (socket says 172.21.15.12:43662) 2024-09-11T00:06:38.501 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:38.497+0000 7f7611e00640 1 -- 172.21.15.12:0/1550513067 learned_addr learned my addr 172.21.15.12:0/1550513067 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:06:38.502 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:38.497+0000 7f7611400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563e64c0cb60 0x563e64c16020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.12:51976/0 (socket says 172.21.15.12:51976) 2024-09-11T00:06:38.502 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:38.497+0000 7f7612800640 1 -- 172.21.15.12:0/1550513067 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563e64c0cb60 msgr2=0x563e64c16020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:06:38.502 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:38.497+0000 7f7612800640 1 --2- 172.21.15.12:0/1550513067 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563e64c0cb60 0x563e64c16020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:38.502 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:38.497+0000 7f7612800640 1 -- 172.21.15.12:0/1550513067 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563e64c0c240 msgr2=0x563e64c15ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:06:38.502 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:38.497+0000 7f7612800640 1 --2- 172.21.15.12:0/1550513067 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563e64c0c240 0x563e64c15ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:38.502 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:38.497+0000 7f7612800640 1 -- 172.21.15.12:0/1550513067 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563e648d1540 con 0x563e64c18af0 2024-09-11T00:06:38.502 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:38.497+0000 7f7611e00640 1 --2- 172.21.15.12:0/1550513067 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563e64c0c240 0x563e64c15ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:06:38.502 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:38.497+0000 7f7612800640 1 --2- 172.21.15.12:0/1550513067 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563e64c18af0 0x563e64c16560 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f76080103d0 tx=0x7f7608010fd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f5ba0d9abcc20dff server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:06:38.502 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:38.497+0000 7f75ef400640 10 client.0 ms_handle_connect on v2:172.21.15.178:3300/0 2024-09-11T00:06:38.502 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:38.498+0000 7f75ef400640 1 -- 172.21.15.12:0/1550513067 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7608035060 con 0x563e64c18af0 2024-09-11T00:06:38.502 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:38.498+0000 7f7617259740 1 -- 172.21.15.12:0/1550513067 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x563e64b10a70 con 0x563e64c18af0 2024-09-11T00:06:38.502 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:38.498+0000 7f75ef400640 1 -- 172.21.15.12:0/1550513067 <== mon.1 v2:172.21.15.178:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 9208+0+0 (secure 0 0 0) 0x7f760801b680 con 0x563e64c18af0 2024-09-11T00:06:38.503 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:38.498+0000 7f75ef400640 1 -- 172.21.15.12:0/1550513067 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f760803c050 con 0x563e64c18af0 2024-09-11T00:06:38.503 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:38.498+0000 7f7617259740 10 client.25324 fetch_fsmap learned FSMap version 28 2024-09-11T00:06:38.503 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:38.498+0000 7f7617259740 1 -- 172.21.15.12:0/1550513067 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x563e64b2b6f0 con 0x563e64c18af0 2024-09-11T00:06:38.503 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:38.498+0000 7f75ef400640 1 -- 172.21.15.12:0/1550513067 <== mon.1 v2:172.21.15.178:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x563e64b10a70 con 0x563e64c18af0 2024-09-11T00:06:38.503 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:38.498+0000 7f75ef400640 1 -- 172.21.15.12:0/1550513067 <== mon.1 v2:172.21.15.178:3300/0 5 ==== fsmap(e 28) ==== 8370+0+0 (secure 0 0 0) 0x7f760800e7c0 con 0x563e64c18af0 2024-09-11T00:06:38.503 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:38.499+0000 7f7617259740 10 client.25324 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T00:06:38.503 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:38.499+0000 7f7617259740 10 client.25324 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-11T00:06:38.503 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:38.499+0000 7f7617259740 20 client.25324 populate_metadata read hostname 'smithi012' 2024-09-11T00:06:38.503 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:38.499+0000 7f7617259740 1 --2- 172.21.15.12:0/1550513067 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x563e64c29640 0x563e64c2aab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:06:38.503 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:38.499+0000 7f7617259740 4 client.25324 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T00:06:38.503 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:38.499+0000 7f7617259740 1 -- 172.21.15.12:0/1550513067 --> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563e64b85ca0 con 0x563e64c29640 2024-09-11T00:06:38.504 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:38.499+0000 7f7611e00640 1 --2- 172.21.15.12:0/1550513067 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x563e64c29640 0x563e64c2aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:06:38.504 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:38.500+0000 7f7611e00640 1 --2- 172.21.15.12:0/1550513067 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x563e64c29640 0x563e64c2aab0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cd0fe3efe83f1635 server_cookie=a05b354386a1a1b0 in_seq=0 out_seq=0 2024-09-11T00:06:38.505 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:38.500+0000 7f75ef400640 10 client.25324 ms_handle_connect on v2:172.21.15.165:6838/501312984 2024-09-11T00:06:38.505 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:38.500+0000 7f75ef400640 1 -- 172.21.15.12:0/1550513067 <== mds.0 v2:172.21.15.165:6838/501312984 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x563e64b85ca0 con 0x563e64c29640 2024-09-11T00:06:38.505 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:38.500+0000 7f75ef400640 10 client.25324 handle_command_reply: tid=0 2024-09-11T00:06:38.505 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:38.501+0000 7f7617259740 1 client.25324 shutdown 2024-09-11T00:06:38.505 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:38.501+0000 7f7617259740 1 -- 172.21.15.12:0/1550513067 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563e64c18af0 msgr2=0x563e64c16560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:06:38.505 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:38.501+0000 7f7617259740 1 --2- 172.21.15.12:0/1550513067 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563e64c18af0 0x563e64c16560 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f76080103d0 tx=0x7f7608010fd0 comp rx=0 tx=0).stop 2024-09-11T00:06:38.505 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:38.501+0000 7f7617259740 1 -- 172.21.15.12:0/1550513067 shutdown_connections 2024-09-11T00:06:38.505 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:38.501+0000 7f7617259740 1 --2- 172.21.15.12:0/1550513067 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x563e64c29640 0x563e64c2aab0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:38.505 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:38.501+0000 7f7617259740 1 --2- 172.21.15.12:0/1550513067 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563e64c18af0 0x563e64c16560 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:38.505 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:38.501+0000 7f7617259740 1 --2- 172.21.15.12:0/1550513067 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563e64c0cb60 0x563e64c16020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:38.505 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:38.501+0000 7f75ef400640 0 client.25324 ms_handle_reset on v2:172.21.15.165:6838/501312984 2024-09-11T00:06:38.505 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:38.501+0000 7f7617259740 1 --2- 172.21.15.12:0/1550513067 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563e64c0c240 0x563e64c15ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:38.505 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:38.501+0000 7f7617259740 1 -- 172.21.15.12:0/1550513067 >> 172.21.15.12:0/1550513067 conn(0x563e64c0a020 msgr2=0x563e64c0fc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:06:38.506 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:38.501+0000 7f7617259740 1 -- 172.21.15.12:0/1550513067 shutdown_connections 2024-09-11T00:06:38.506 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:38.501+0000 7f7617259740 1 -- 172.21.15.12:0/1550513067 wait complete. 2024-09-11T00:06:38.506 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:38.501+0000 7f7617259740 20 client.25324 trim_cache size 0 max 16384 2024-09-11T00:06:38.523 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:38.521+0000 7f7617259740 1 Processor -- start 2024-09-11T00:06:38.523 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:38.521+0000 7f7617259740 1 -- start start 2024-09-11T00:06:38.523 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:38.521+0000 7f7617259740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563e64c0a020 0x563e64c0af80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:06:38.523 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:38.521+0000 7f7617259740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563e64c0c240 0x563e64c17820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:06:38.523 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:38.521+0000 7f7617259740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563e64c0cb60 0x563e64c12cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:06:38.523 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:38.521+0000 7f7617259740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x563e64c28df0 con 0x563e64c0c240 2024-09-11T00:06:38.523 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:38.521+0000 7f7617259740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x563e64c294c0 con 0x563e64c0cb60 2024-09-11T00:06:38.523 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:38.521+0000 7f7617259740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x563e64c29b90 con 0x563e64c0a020 2024-09-11T00:06:38.524 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:38.521+0000 7f7611e00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563e64c0a020 0x563e64c0af80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:06:38.525 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:38.521+0000 7f7611400640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563e64c0c240 0x563e64c17820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:06:38.525 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:38.521+0000 7f7611e00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563e64c0a020 0x563e64c0af80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.12:47274/0 (socket says 172.21.15.12:47274) 2024-09-11T00:06:38.525 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:38.521+0000 7f7611400640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563e64c0c240 0x563e64c17820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:43670/0 (socket says 172.21.15.12:43670) 2024-09-11T00:06:38.525 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:38.521+0000 7f7611e00640 1 -- 172.21.15.12:0/481174452 learned_addr learned my addr 172.21.15.12:0/481174452 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:06:38.525 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:38.522+0000 7f7611e00640 1 -- 172.21.15.12:0/481174452 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563e64c0cb60 msgr2=0x563e64c12cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:06:38.525 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:38.522+0000 7f7612800640 1 --2- 172.21.15.12:0/481174452 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563e64c0cb60 0x563e64c12cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:06:38.525 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:38.522+0000 7f7611e00640 1 --2- 172.21.15.12:0/481174452 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563e64c0cb60 0x563e64c12cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:38.525 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:38.522+0000 7f7611e00640 1 -- 172.21.15.12:0/481174452 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563e64c0c240 msgr2=0x563e64c17820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:06:38.525 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:38.522+0000 7f7611e00640 1 --2- 172.21.15.12:0/481174452 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563e64c0c240 0x563e64c17820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:38.525 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:38.522+0000 7f7611e00640 1 -- 172.21.15.12:0/481174452 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563e64c09e50 con 0x563e64c0a020 2024-09-11T00:06:38.525 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:38.522+0000 7f7612800640 1 --2- 172.21.15.12:0/481174452 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563e64c0cb60 0x563e64c12cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T00:06:38.525 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:38.522+0000 7f7611e00640 1 --2- 172.21.15.12:0/481174452 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563e64c0a020 0x563e64c0af80 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f7600032af0 tx=0x7f76000c86a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8d2d2e7f6ddd0bb6 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:06:38.525 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:38.522+0000 7f75fa600640 1 -- 172.21.15.12:0/481174452 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7600049040 con 0x563e64c0a020 2024-09-11T00:06:38.526 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:38.522+0000 7f75fa600640 1 -- 172.21.15.12:0/481174452 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f76000cc7c0 con 0x563e64c0a020 2024-09-11T00:06:38.526 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:38.522+0000 7f75fa600640 1 -- 172.21.15.12:0/481174452 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7600106200 con 0x563e64c0a020 2024-09-11T00:06:38.527 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:38.523+0000 7f7617259740 1 -- 172.21.15.12:0/481174452 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563e64c0a020 msgr2=0x563e64c0af80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:06:38.527 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:38.523+0000 7f7617259740 1 --2- 172.21.15.12:0/481174452 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563e64c0a020 0x563e64c0af80 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f7600032af0 tx=0x7f76000c86a0 comp rx=0 tx=0).stop 2024-09-11T00:06:38.527 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:38.523+0000 7f7617259740 1 -- 172.21.15.12:0/481174452 shutdown_connections 2024-09-11T00:06:38.527 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:38.523+0000 7f7617259740 1 --2- 172.21.15.12:0/481174452 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563e64c0cb60 0x563e64c12cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:38.527 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:38.523+0000 7f7617259740 1 --2- 172.21.15.12:0/481174452 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563e64c0c240 0x563e64c17820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:38.527 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:38.523+0000 7f7617259740 1 --2- 172.21.15.12:0/481174452 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563e64c0a020 0x563e64c0af80 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:38.527 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:38.523+0000 7f7617259740 1 -- 172.21.15.12:0/481174452 >> 172.21.15.12:0/481174452 conn(0x563e64c28a10 msgr2=0x563e64c17110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:06:38.527 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:38.523+0000 7f7617259740 1 -- 172.21.15.12:0/481174452 shutdown_connections 2024-09-11T00:06:38.528 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:38.524+0000 7f7617259740 1 -- 172.21.15.12:0/481174452 wait complete. 2024-09-11T00:06:38.528 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:38.524+0000 7f7617259740 1 -- start start 2024-09-11T00:06:38.528 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:38.524+0000 7f7617259740 1 Processor -- start 2024-09-11T00:06:38.528 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:38.525+0000 7f7617259740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563e64c0a020 0x563e64c334b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:06:38.528 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:38.525+0000 7f7617259740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563e64c0c240 0x563e64c2c980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:06:38.528 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:38.525+0000 7f7617259740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563e64c0cb60 0x563e64c2cec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:06:38.528 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:38.525+0000 7f7617259740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x563e64c2a930 con 0x563e64c0c240 2024-09-11T00:06:38.528 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:38.525+0000 7f7617259740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x563e64c2a260 con 0x563e64c0a020 2024-09-11T00:06:38.528 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:38.525+0000 7f7617259740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x563e64c2b000 con 0x563e64c0cb60 2024-09-11T00:06:38.528 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:38.525+0000 7f7617259740 10 client.0 _ll_register_callbacks cb 0x7f7608011c20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:06:38.528 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:38.525+0000 7f7611e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563e64c0a020 0x563e64c334b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:06:38.528 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:38.525+0000 7f7611e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563e64c0a020 0x563e64c334b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.12:51992/0 (socket says 172.21.15.12:51992) 2024-09-11T00:06:38.528 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:38.525+0000 7f7611e00640 1 -- 172.21.15.12:0/551911986 learned_addr learned my addr 172.21.15.12:0/551911986 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:06:38.529 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:38.525+0000 7f7612800640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563e64c0cb60 0x563e64c2cec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:06:38.529 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:38.525+0000 7f7611400640 1 --2- 172.21.15.12:0/551911986 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563e64c0c240 0x563e64c2c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:06:38.531 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:38.525+0000 7f7612800640 1 -- 172.21.15.12:0/551911986 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563e64c0a020 msgr2=0x563e64c334b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:06:38.531 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:38.525+0000 7f7612800640 1 --2- 172.21.15.12:0/551911986 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563e64c0a020 0x563e64c334b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:38.531 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:38.525+0000 7f7612800640 1 -- 172.21.15.12:0/551911986 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563e64c0c240 msgr2=0x563e64c2c980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:06:38.531 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:38.525+0000 7f7612800640 1 --2- 172.21.15.12:0/551911986 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563e64c0c240 0x563e64c2c980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:38.531 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:38.525+0000 7f7612800640 1 -- 172.21.15.12:0/551911986 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563e64c2d6a0 con 0x563e64c0cb60 2024-09-11T00:06:38.531 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:38.526+0000 7f7612800640 1 --2- 172.21.15.12:0/551911986 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563e64c0cb60 0x563e64c2cec0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f7608010900 tx=0x7f7608005ea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3e862e173c601cba server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:06:38.531 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:38.526+0000 7f75ef400640 10 client.0 ms_handle_connect on v2:172.21.15.178:3300/0 2024-09-11T00:06:38.531 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:38.526+0000 7f75ef400640 1 -- 172.21.15.12:0/551911986 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7608045060 con 0x563e64c0cb60 2024-09-11T00:06:38.531 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:38.526+0000 7f75ef400640 1 -- 172.21.15.12:0/551911986 <== mon.1 v2:172.21.15.178:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 9208+0+0 (secure 0 0 0) 0x7f760800e7f0 con 0x563e64c0cb60 2024-09-11T00:06:38.531 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:38.526+0000 7f7617259740 1 -- 172.21.15.12:0/551911986 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f76000cc7c0 con 0x563e64c0cb60 2024-09-11T00:06:38.532 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:38.526+0000 7f75ef400640 1 -- 172.21.15.12:0/551911986 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f760804c050 con 0x563e64c0cb60 2024-09-11T00:06:38.532 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:38.526+0000 7f7617259740 10 client.25330 fetch_fsmap learned FSMap version 28 2024-09-11T00:06:38.532 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:38.526+0000 7f7617259740 1 -- 172.21.15.12:0/551911986 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x563e64c95f80 con 0x563e64c0cb60 2024-09-11T00:06:38.532 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:38.527+0000 7f75ef400640 1 -- 172.21.15.12:0/551911986 <== mon.1 v2:172.21.15.178:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f76000cc7c0 con 0x563e64c0cb60 2024-09-11T00:06:38.532 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:38.527+0000 7f75ef400640 1 -- 172.21.15.12:0/551911986 <== mon.1 v2:172.21.15.178:3300/0 5 ==== fsmap(e 28) ==== 8370+0+0 (secure 0 0 0) 0x7f7608040a00 con 0x563e64c0cb60 2024-09-11T00:06:38.532 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:38.527+0000 7f7617259740 10 client.25330 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T00:06:38.533 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:38.527+0000 7f7617259740 10 client.25330 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-11T00:06:38.533 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:38.527+0000 7f7617259740 20 client.25330 populate_metadata read hostname 'smithi012' 2024-09-11T00:06:38.533 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:38.527+0000 7f7617259740 1 --2- 172.21.15.12:0/551911986 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x563e64c18af0 0x563e64c99750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:06:38.533 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:38.527+0000 7f7617259740 4 client.25330 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-11T00:06:38.533 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:38.527+0000 7f7617259740 1 -- 172.21.15.12:0/551911986 --> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] -- command(tid 0: {"prefix": "status"}) -- 0x563e64b81750 con 0x563e64c18af0 2024-09-11T00:06:38.533 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:38.528+0000 7f7611e00640 1 --2- 172.21.15.12:0/551911986 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x563e64c18af0 0x563e64c99750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:06:38.533 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:38.528+0000 7f7611e00640 1 --2- 172.21.15.12:0/551911986 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x563e64c18af0 0x563e64c99750 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e67cba0df346b09b server_cookie=e3735a504692c691 in_seq=0 out_seq=0 2024-09-11T00:06:38.533 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:38.528+0000 7f75ef400640 10 client.25330 ms_handle_connect on v2:172.21.15.165:6838/501312984 2024-09-11T00:06:38.533 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:38.528+0000 7f75ef400640 1 -- 172.21.15.12:0/551911986 <== mds.0 v2:172.21.15.165:6838/501312984 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x563e64b81750 con 0x563e64c18af0 2024-09-11T00:06:38.533 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:38.528+0000 7f75ef400640 10 client.25330 handle_command_reply: tid=0 2024-09-11T00:06:38.533 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:38.528+0000 7f7617259740 1 client.25330 shutdown 2024-09-11T00:06:38.533 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:38.528+0000 7f7617259740 1 -- 172.21.15.12:0/551911986 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563e64c0cb60 msgr2=0x563e64c2cec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:06:38.533 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:38.528+0000 7f7617259740 1 --2- 172.21.15.12:0/551911986 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563e64c0cb60 0x563e64c2cec0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f7608010900 tx=0x7f7608005ea0 comp rx=0 tx=0).stop 2024-09-11T00:06:38.534 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:38.529+0000 7f7617259740 1 -- 172.21.15.12:0/551911986 shutdown_connections 2024-09-11T00:06:38.534 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:38.529+0000 7f7617259740 1 --2- 172.21.15.12:0/551911986 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x563e64c18af0 0x563e64c99750 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:38.534 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:38.529+0000 7f7617259740 1 --2- 172.21.15.12:0/551911986 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563e64c0cb60 0x563e64c2cec0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:38.534 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:38.529+0000 7f7617259740 1 --2- 172.21.15.12:0/551911986 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563e64c0c240 0x563e64c2c980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:38.534 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:38.529+0000 7f7617259740 1 --2- 172.21.15.12:0/551911986 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563e64c0a020 0x563e64c334b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:38.534 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:38.529+0000 7f75ef400640 0 client.25330 ms_handle_reset on v2:172.21.15.165:6838/501312984 2024-09-11T00:06:38.534 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:38.529+0000 7f7617259740 1 -- 172.21.15.12:0/551911986 >> 172.21.15.12:0/551911986 conn(0x563e64c28a10 msgr2=0x563e64c17930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:06:38.534 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:38.529+0000 7f7617259740 1 -- 172.21.15.12:0/551911986 shutdown_connections 2024-09-11T00:06:38.534 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:38.529+0000 7f7617259740 1 -- 172.21.15.12:0/551911986 wait complete. 2024-09-11T00:06:38.534 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:38.529+0000 7f7617259740 20 client.25330 trim_cache size 0 max 16384 2024-09-11T00:06:38.535 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-09-11T00:06:38.535 INFO:teuthology.orchestra.run.smithi012.stdout: "cluster_fsid": "ff3ad74a-6fcf-11ef-bcea-c7b262605968", 2024-09-11T00:06:38.535 INFO:teuthology.orchestra.run.smithi012.stdout: "whoami": 0, 2024-09-11T00:06:38.535 INFO:teuthology.orchestra.run.smithi012.stdout: "id": 24674, 2024-09-11T00:06:38.535 INFO:teuthology.orchestra.run.smithi012.stdout: "want_state": "up:active", 2024-09-11T00:06:38.535 INFO:teuthology.orchestra.run.smithi012.stdout: "state": "up:active", 2024-09-11T00:06:38.535 INFO:teuthology.orchestra.run.smithi012.stdout: "fs_name": "cephfs", 2024-09-11T00:06:38.535 INFO:teuthology.orchestra.run.smithi012.stdout: "rank_uptime": 153.691055909, 2024-09-11T00:06:38.535 INFO:teuthology.orchestra.run.smithi012.stdout: "mdsmap_epoch": 28, 2024-09-11T00:06:38.535 INFO:teuthology.orchestra.run.smithi012.stdout: "osdmap_epoch": 85, 2024-09-11T00:06:38.535 INFO:teuthology.orchestra.run.smithi012.stdout: "osdmap_epoch_barrier": 81, 2024-09-11T00:06:38.535 INFO:teuthology.orchestra.run.smithi012.stdout: "uptime": 166.60010362400001 2024-09-11T00:06:38.536 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-09-11T00:06:38.536 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:38.530+0000 7f7614800640 1 -- 172.21.15.12:0/1876341730 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f75f40b5280 msgr2=0x7f75f40b7740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:06:38.536 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:38.530+0000 7f7614800640 1 --2- 172.21.15.12:0/1876341730 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f75f40b5280 0x7f75f40b7740 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f7608009dd0 tx=0x7f7608005fd0 comp rx=0 tx=0).stop 2024-09-11T00:06:38.536 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:38.530+0000 7f7614800640 1 -- 172.21.15.12:0/1876341730 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f760c1018d0 msgr2=0x7f760c19b4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:06:38.536 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:38.530+0000 7f7614800640 1 --2- 172.21.15.12:0/1876341730 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f760c1018d0 0x7f760c19b4f0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f7600005940 tx=0x7f76000040c0 comp rx=0 tx=0).stop 2024-09-11T00:06:38.536 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:38.530+0000 7f7614800640 1 -- 172.21.15.12:0/1876341730 shutdown_connections 2024-09-11T00:06:38.536 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:38.530+0000 7f7614800640 1 --2- 172.21.15.12:0/1876341730 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f75f40b5280 0x7f75f40b7740 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:38.537 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:38.530+0000 7f7614800640 1 --2- 172.21.15.12:0/1876341730 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f760c10f480 0x7f760c1a0dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:38.537 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:38.530+0000 7f7614800640 1 --2- 172.21.15.12:0/1876341730 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f760c1021f0 0x7f760c19ba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:38.537 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:38.530+0000 7f7614800640 1 --2- 172.21.15.12:0/1876341730 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f760c1018d0 0x7f760c19b4f0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:38.537 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:38.530+0000 7f7614800640 1 -- 172.21.15.12:0/1876341730 >> 172.21.15.12:0/1876341730 conn(0x7f760c0fd700 msgr2=0x7f760c0ffaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:06:38.537 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:38.530+0000 7f7614800640 1 -- 172.21.15.12:0/1876341730 shutdown_connections 2024-09-11T00:06:38.537 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:38.530+0000 7f7614800640 1 -- 172.21.15.12:0/1876341730 wait complete. 2024-09-11T00:06:39.292 INFO:teuthology.run_tasks:Running task kclient... 2024-09-11T00:06:39.305 INFO:tasks.kclient:Mounting kernel clients... 2024-09-11T00:06:39.305 INFO:tasks.kclient:config is {'client.0': {}} 2024-09-11T00:06:39.305 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T00:06:39.305 DEBUG:teuthology.orchestra.run.smithi012:> ip netns list 2024-09-11T00:06:39.342 INFO:teuthology.orchestra.run.smithi012.stdout:ceph-ns--home-ubuntu-cephtest-mnt.admin (id: 0) 2024-09-11T00:06:39.343 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T00:06:39.343 DEBUG:teuthology.orchestra.run.smithi012:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-09-11T00:06:39.443 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T00:06:39.444 DEBUG:teuthology.orchestra.run.smithi012:> sudo ip link delete ceph-brx 2024-09-11T00:06:39.532 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:06:39 smithi165 ceph-mon[35356]: pgmap v385: 129 pgs: 129 active+clean; 647 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-11T00:06:39.562 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:06:39 smithi012 ceph-mon[30333]: pgmap v385: 129 pgs: 129 active+clean; 647 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-11T00:06:39.580 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:06:39 smithi178 ceph-mon[35919]: pgmap v385: 129 pgs: 129 active+clean; 647 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-11T00:06:39.752 INFO:tasks.kclient:client.0 config is {'mntopts': ['ms_mode=legacy', 'nowsync'], 'syntax': 'v1', 'mount_subvol_num': 1} 2024-09-11T00:06:39.752 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-09-11T00:06:39.752 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-11T00:06:39.752 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid ff3ad74a-6fcf-11ef-bcea-c7b262605968 -- ceph fs ls 2024-09-11T00:06:40.135 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:06:40.721 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:40.718+0000 7f3530800640 1 -- 172.21.15.12:0/774862325 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f352810b330 msgr2=0x7f3528111b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:06:40.721 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:40.718+0000 7f3530800640 1 --2- 172.21.15.12:0/774862325 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f352810b330 0x7f3528111b30 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f3524009ca0 tx=0x7f352402a8c0 comp rx=0 tx=0).stop 2024-09-11T00:06:40.721 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:40.718+0000 7f3530800640 1 -- 172.21.15.12:0/774862325 shutdown_connections 2024-09-11T00:06:40.721 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:40.718+0000 7f3530800640 1 --2- 172.21.15.12:0/774862325 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f352810b330 0x7f3528111b30 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:40.721 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:40.718+0000 7f3530800640 1 --2- 172.21.15.12:0/774862325 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f352810a9b0 0x7f352810adf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:40.721 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:40.718+0000 7f3530800640 1 --2- 172.21.15.12:0/774862325 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3528104fa0 0x7f3528105380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:40.722 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:40.718+0000 7f3530800640 1 -- 172.21.15.12:0/774862325 >> 172.21.15.12:0/774862325 conn(0x7f3528100a50 msgr2=0x7f3528102e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:06:40.722 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:40.718+0000 7f3530800640 1 -- 172.21.15.12:0/774862325 shutdown_connections 2024-09-11T00:06:40.722 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:40.718+0000 7f3530800640 1 -- 172.21.15.12:0/774862325 wait complete. 2024-09-11T00:06:40.722 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:40.719+0000 7f3530800640 1 Processor -- start 2024-09-11T00:06:40.722 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:40.719+0000 7f3530800640 1 -- start start 2024-09-11T00:06:40.723 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:40.720+0000 7f3530800640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3528104fa0 0x7f352819b400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:06:40.723 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:40.720+0000 7f3530800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f352810a9b0 0x7f352819b940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:06:40.723 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:40.720+0000 7f3530800640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f352810b330 0x7f35281a0ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:06:40.723 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:40.720+0000 7f3530800640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f3528114fb0 con 0x7f3528104fa0 2024-09-11T00:06:40.723 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:40.720+0000 7f3530800640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f3528114e30 con 0x7f352810a9b0 2024-09-11T00:06:40.723 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:40.720+0000 7f3530800640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f3528115130 con 0x7f352810b330 2024-09-11T00:06:40.724 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:40.720+0000 7f352de00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3528104fa0 0x7f352819b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:06:40.724 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:40.720+0000 7f352e800640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f352810b330 0x7f35281a0ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:06:40.724 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:40.720+0000 7f352de00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3528104fa0 0x7f352819b400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:41708/0 (socket says 172.21.15.12:41708) 2024-09-11T00:06:40.724 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:40.720+0000 7f352de00640 1 -- 172.21.15.12:0/2014274492 learned_addr learned my addr 172.21.15.12:0/2014274492 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:06:40.724 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:40.720+0000 7f352d400640 1 --2- 172.21.15.12:0/2014274492 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f352810a9b0 0x7f352819b940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:06:40.724 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:40.720+0000 7f352de00640 1 -- 172.21.15.12:0/2014274492 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f352810b330 msgr2=0x7f35281a0ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:06:40.724 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:40.720+0000 7f352de00640 1 --2- 172.21.15.12:0/2014274492 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f352810b330 0x7f35281a0ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:40.724 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:40.721+0000 7f352de00640 1 -- 172.21.15.12:0/2014274492 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f352810a9b0 msgr2=0x7f352819b940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:06:40.724 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:40.721+0000 7f352de00640 1 --2- 172.21.15.12:0/2014274492 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f352810a9b0 0x7f352819b940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:40.724 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:40.721+0000 7f352de00640 1 -- 172.21.15.12:0/2014274492 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35281a14b0 con 0x7f3528104fa0 2024-09-11T00:06:40.724 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:40.721+0000 7f352de00640 1 --2- 172.21.15.12:0/2014274492 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3528104fa0 0x7f352819b400 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f351c00e9f0 tx=0x7f351c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:06:40.724 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:40.721+0000 7f3522a00640 1 -- 172.21.15.12:0/2014274492 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f351c00cd80 con 0x7f3528104fa0 2024-09-11T00:06:40.725 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:40.721+0000 7f3522a00640 1 -- 172.21.15.12:0/2014274492 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f351c0108e0 con 0x7f3528104fa0 2024-09-11T00:06:40.725 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:40.721+0000 7f3530800640 1 -- 172.21.15.12:0/2014274492 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f35281a17a0 con 0x7f3528104fa0 2024-09-11T00:06:40.725 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:40.721+0000 7f3522a00640 1 -- 172.21.15.12:0/2014274492 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f351c018980 con 0x7f3528104fa0 2024-09-11T00:06:40.725 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:40.721+0000 7f3530800640 1 -- 172.21.15.12:0/2014274492 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35281a8030 con 0x7f3528104fa0 2024-09-11T00:06:40.725 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:40.723+0000 7f3530800640 1 -- 172.21.15.12:0/2014274492 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3528106250 con 0x7f3528104fa0 2024-09-11T00:06:40.726 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:40.725+0000 7f3522a00640 1 -- 172.21.15.12:0/2014274492 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7f351c010430 con 0x7f3528104fa0 2024-09-11T00:06:40.728 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:40.726+0000 7f3522a00640 1 --2- 172.21.15.12:0/2014274492 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f35100b5360 0x7f35100b7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:06:40.728 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:40.726+0000 7f3522a00640 1 -- 172.21.15.12:0/2014274492 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9208+0+0 (secure 0 0 0) 0x7f351c0e62d0 con 0x7f3528104fa0 2024-09-11T00:06:40.728 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:40.726+0000 7f352d400640 1 --2- 172.21.15.12:0/2014274492 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f35100b5360 0x7f35100b7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:06:40.729 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:40.727+0000 7f352d400640 1 --2- 172.21.15.12:0/2014274492 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f35100b5360 0x7f35100b7820 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f352810a560 tx=0x7f351800b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:06:40.732 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:40.730+0000 7f3522a00640 1 -- 172.21.15.12:0/2014274492 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f351c0ac710 con 0x7f3528104fa0 2024-09-11T00:06:40.963 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:40.961+0000 7f3530800640 1 -- 172.21.15.12:0/2014274492 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f352802e5c0 con 0x7f3528104fa0 2024-09-11T00:06:40.964 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:40.962+0000 7f3522a00640 1 -- 172.21.15.12:0/2014274492 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v28) ==== 53+0+88 (secure 0 0 0) 0x7f351c0b1140 con 0x7f3528104fa0 2024-09-11T00:06:40.964 INFO:teuthology.orchestra.run.smithi012.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-09-11T00:06:40.966 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:40.964+0000 7f3530800640 1 -- 172.21.15.12:0/2014274492 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f35100b5360 msgr2=0x7f35100b7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:06:40.966 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:40.964+0000 7f3530800640 1 --2- 172.21.15.12:0/2014274492 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f35100b5360 0x7f35100b7820 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f352810a560 tx=0x7f351800b390 comp rx=0 tx=0).stop 2024-09-11T00:06:40.966 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:40.964+0000 7f3530800640 1 -- 172.21.15.12:0/2014274492 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3528104fa0 msgr2=0x7f352819b400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:06:40.966 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:40.964+0000 7f3530800640 1 --2- 172.21.15.12:0/2014274492 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3528104fa0 0x7f352819b400 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f351c00e9f0 tx=0x7f351c00eeb0 comp rx=0 tx=0).stop 2024-09-11T00:06:40.967 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:40.964+0000 7f3530800640 1 -- 172.21.15.12:0/2014274492 shutdown_connections 2024-09-11T00:06:40.967 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:40.964+0000 7f3530800640 1 --2- 172.21.15.12:0/2014274492 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f35100b5360 0x7f35100b7820 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:40.967 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:40.964+0000 7f3530800640 1 --2- 172.21.15.12:0/2014274492 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f352810b330 0x7f35281a0ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:40.967 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:40.964+0000 7f3530800640 1 --2- 172.21.15.12:0/2014274492 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f352810a9b0 0x7f352819b940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:40.967 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:40.964+0000 7f3530800640 1 --2- 172.21.15.12:0/2014274492 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3528104fa0 0x7f352819b400 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:40.967 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:40.964+0000 7f3530800640 1 -- 172.21.15.12:0/2014274492 >> 172.21.15.12:0/2014274492 conn(0x7f3528100a50 msgr2=0x7f3528102e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:06:40.967 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:40.964+0000 7f3530800640 1 -- 172.21.15.12:0/2014274492 shutdown_connections 2024-09-11T00:06:40.967 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:40.964+0000 7f3530800640 1 -- 172.21.15.12:0/2014274492 wait complete. 2024-09-11T00:06:41.532 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:06:41 smithi165 ceph-mon[35356]: pgmap v386: 129 pgs: 129 active+clean; 647 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-11T00:06:41.532 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:06:41 smithi165 ceph-mon[35356]: from='client.? 172.21.15.12:0/2014274492' entity='client.admin' cmd={"prefix": "fs ls"} : dispatch 2024-09-11T00:06:41.562 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:06:41 smithi012 ceph-mon[30333]: pgmap v386: 129 pgs: 129 active+clean; 647 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-11T00:06:41.563 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:06:41 smithi012 ceph-mon[30333]: from='client.? 172.21.15.12:0/2014274492' entity='client.admin' cmd={"prefix": "fs ls"} : dispatch 2024-09-11T00:06:41.581 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:06:41 smithi178 ceph-mon[35919]: pgmap v386: 129 pgs: 129 active+clean; 647 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-11T00:06:41.581 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:06:41 smithi178 ceph-mon[35919]: from='client.? 172.21.15.12:0/2014274492' entity='client.admin' cmd={"prefix": "fs ls"} : dispatch 2024-09-11T00:06:41.707 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-11T00:06:41.707 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-11T00:06:41.707 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi012.front.sepia.ceph.com 2024-09-11T00:06:41.707 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-11T00:06:41.707 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-11T00:06:41.707 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-11T00:06:41.707 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-11T00:06:41.707 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-11T00:06:41.707 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-11T00:06:41.708 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T00:06:41.708 DEBUG:teuthology.orchestra.run.smithi012:> ip addr 2024-09-11T00:06:41.744 INFO:teuthology.orchestra.run.smithi012.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-11T00:06:41.744 INFO:teuthology.orchestra.run.smithi012.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-11T00:06:41.744 INFO:teuthology.orchestra.run.smithi012.stdout: inet 127.0.0.1/8 scope host lo 2024-09-11T00:06:41.744 INFO:teuthology.orchestra.run.smithi012.stdout: valid_lft forever preferred_lft forever 2024-09-11T00:06:41.744 INFO:teuthology.orchestra.run.smithi012.stdout: inet6 ::1/128 scope host 2024-09-11T00:06:41.745 INFO:teuthology.orchestra.run.smithi012.stdout: valid_lft forever preferred_lft forever 2024-09-11T00:06:41.745 INFO:teuthology.orchestra.run.smithi012.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-11T00:06:41.745 INFO:teuthology.orchestra.run.smithi012.stdout: link/ether 0c:c4:7a:6b:fd:a4 brd ff:ff:ff:ff:ff:ff 2024-09-11T00:06:41.745 INFO:teuthology.orchestra.run.smithi012.stdout: altname enp7s0f0 2024-09-11T00:06:41.745 INFO:teuthology.orchestra.run.smithi012.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-11T00:06:41.745 INFO:teuthology.orchestra.run.smithi012.stdout: link/ether 0c:c4:7a:6b:fd:a5 brd ff:ff:ff:ff:ff:ff 2024-09-11T00:06:41.745 INFO:teuthology.orchestra.run.smithi012.stdout: altname enp7s0f1 2024-09-11T00:06:41.745 INFO:teuthology.orchestra.run.smithi012.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-11T00:06:41.745 INFO:teuthology.orchestra.run.smithi012.stdout: link/ether ac:1f:6b:f8:25:ea brd ff:ff:ff:ff:ff:ff 2024-09-11T00:06:41.745 INFO:teuthology.orchestra.run.smithi012.stdout: altname enp1s0f0 2024-09-11T00:06:41.746 INFO:teuthology.orchestra.run.smithi012.stdout: inet 172.21.15.12/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-09-11T00:06:41.746 INFO:teuthology.orchestra.run.smithi012.stdout: valid_lft 42048sec preferred_lft 42048sec 2024-09-11T00:06:41.746 INFO:teuthology.orchestra.run.smithi012.stdout: inet6 fe80::ae1f:6bff:fef8:25ea/64 scope link 2024-09-11T00:06:41.746 INFO:teuthology.orchestra.run.smithi012.stdout: valid_lft forever preferred_lft forever 2024-09-11T00:06:41.746 INFO:teuthology.orchestra.run.smithi012.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-11T00:06:41.746 INFO:teuthology.orchestra.run.smithi012.stdout: link/ether ac:1f:6b:f8:25:eb brd ff:ff:ff:ff:ff:ff 2024-09-11T00:06:41.746 INFO:teuthology.orchestra.run.smithi012.stdout: altname enp1s0f1 2024-09-11T00:06:41.747 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-09-11T00:06:41.747 DEBUG:teuthology.orchestra.run.smithi012:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-11T00:06:41.747 DEBUG:teuthology.orchestra.run.smithi012:> set -e 2024-09-11T00:06:41.747 DEBUG:teuthology.orchestra.run.smithi012:> sudo ip link add name ceph-brx type bridge 2024-09-11T00:06:41.747 DEBUG:teuthology.orchestra.run.smithi012:> sudo ip addr flush dev ceph-brx 2024-09-11T00:06:41.747 DEBUG:teuthology.orchestra.run.smithi012:> sudo ip link set ceph-brx up 2024-09-11T00:06:41.748 DEBUG:teuthology.orchestra.run.smithi012:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-09-11T00:06:41.748 DEBUG:teuthology.orchestra.run.smithi012:> ') 2024-09-11T00:06:41.869 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:41 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-11T00:06:41.968 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:41 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-11T00:06:41.976 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T00:06:41.976 DEBUG:teuthology.orchestra.run.smithi012:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-11T00:06:42.031 INFO:teuthology.orchestra.run.smithi012.stdout:1 2024-09-11T00:06:42.034 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T00:06:42.035 DEBUG:teuthology.orchestra.run.smithi012:> route 2024-09-11T00:06:42.072 INFO:teuthology.orchestra.run.smithi012.stdout:Kernel IP routing table 2024-09-11T00:06:42.072 INFO:teuthology.orchestra.run.smithi012.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-11T00:06:42.072 INFO:teuthology.orchestra.run.smithi012.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-09-11T00:06:42.072 INFO:teuthology.orchestra.run.smithi012.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-09-11T00:06:42.072 INFO:teuthology.orchestra.run.smithi012.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-11T00:06:42.072 DEBUG:teuthology.orchestra.run.smithi012:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-11T00:06:42.072 DEBUG:teuthology.orchestra.run.smithi012:> set -e 2024-09-11T00:06:42.073 DEBUG:teuthology.orchestra.run.smithi012:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-09-11T00:06:42.073 DEBUG:teuthology.orchestra.run.smithi012:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-09-11T00:06:42.073 DEBUG:teuthology.orchestra.run.smithi012:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-09-11T00:06:42.073 DEBUG:teuthology.orchestra.run.smithi012:> ') 2024-09-11T00:06:42.191 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:42 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-11T00:06:42.263 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:42 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-11T00:06:42.272 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T00:06:42.272 DEBUG:teuthology.orchestra.run.smithi012:> ip netns list 2024-09-11T00:06:42.308 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T00:06:42.308 DEBUG:teuthology.orchestra.run.smithi012:> ip netns list-id 2024-09-11T00:06:42.385 DEBUG:teuthology.orchestra.run.smithi012:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-11T00:06:42.385 DEBUG:teuthology.orchestra.run.smithi012:> set -e 2024-09-11T00:06:42.385 DEBUG:teuthology.orchestra.run.smithi012:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-11T00:06:42.385 DEBUG:teuthology.orchestra.run.smithi012:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-09-11T00:06:42.386 DEBUG:teuthology.orchestra.run.smithi012:> ') 2024-09-11T00:06:42.505 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:42 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-11T00:06:42.557 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:42 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-11T00:06:42.566 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-09-11T00:06:42.566 DEBUG:teuthology.orchestra.run.smithi012:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-11T00:06:42.566 DEBUG:teuthology.orchestra.run.smithi012:> set -e 2024-09-11T00:06:42.566 DEBUG:teuthology.orchestra.run.smithi012:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-09-11T00:06:42.566 DEBUG:teuthology.orchestra.run.smithi012:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-09-11T00:06:42.566 DEBUG:teuthology.orchestra.run.smithi012:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-09-11T00:06:42.566 DEBUG:teuthology.orchestra.run.smithi012:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-09-11T00:06:42.567 DEBUG:teuthology.orchestra.run.smithi012:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-09-11T00:06:42.567 DEBUG:teuthology.orchestra.run.smithi012:> ') 2024-09-11T00:06:42.645 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:42 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-11T00:06:42.777 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:42 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-11T00:06:42.786 DEBUG:teuthology.orchestra.run.smithi012:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-11T00:06:42.786 DEBUG:teuthology.orchestra.run.smithi012:> set -e 2024-09-11T00:06:42.786 DEBUG:teuthology.orchestra.run.smithi012:> sudo ip link set brx.0 up 2024-09-11T00:06:42.786 DEBUG:teuthology.orchestra.run.smithi012:> sudo ip link set dev brx.0 master ceph-brx 2024-09-11T00:06:42.786 DEBUG:teuthology.orchestra.run.smithi012:> ') 2024-09-11T00:06:42.865 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:42 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-11T00:06:42.944 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:42 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-11T00:06:42.952 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-11T00:06:42.953 DEBUG:teuthology.orchestra.run.smithi012:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-11T00:06:42.989 INFO:teuthology.orchestra.run.smithi012.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-11T00:06:42.989 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-11T00:06:42.989 DEBUG:teuthology.orchestra.run.smithi012:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-11T00:06:43.065 WARNING:tasks.cephfs.mount:You cannot specify both: cephfs_mntpt and mount_subvol_num 2024-09-11T00:06:43.065 INFO:tasks.cephfs.mount:Mounting subvol 1 for now 2024-09-11T00:06:43.065 INFO:tasks.cephfs.kernel_mount:mounting using device: :/volumes/qa/sv_1/40f120f8-5df9-4be4-beff-2e141f21ed64 2024-09-11T00:06:43.065 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T00:06:43.066 DEBUG:teuthology.orchestra.run.smithi012:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/volumes/qa/sv_1/40f120f8-5df9-4be4-beff-2e141f21ed64 /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=legacy,nowsync 2024-09-11T00:06:43.195 INFO:teuthology.orchestra.run.smithi012.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=legacy,nowsync 2024-09-11T00:06:43.195 INFO:teuthology.orchestra.run.smithi012.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=legacy,nowsync". 2024-09-11T00:06:43.195 INFO:teuthology.orchestra.run.smithi012.stdout:invalid new device string format 2024-09-11T00:06:43.324 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:06:43 smithi012 ceph-mon[30333]: pgmap v387: 129 pgs: 129 active+clean; 647 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-09-11T00:06:43.532 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:06:43 smithi165 ceph-mon[35356]: pgmap v387: 129 pgs: 129 active+clean; 647 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-09-11T00:06:43.580 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:06:43 smithi178 ceph-mon[35919]: pgmap v387: 129 pgs: 129 active+clean; 647 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-09-11T00:06:43.806 INFO:teuthology.orchestra.run.smithi012.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=legacy,nowsync 2024-09-11T00:06:43.807 INFO:teuthology.orchestra.run.smithi012.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=legacy,nowsync". 2024-09-11T00:06:43.807 INFO:teuthology.orchestra.run.smithi012.stdout:invalid new device string format 2024-09-11T00:06:43.807 INFO:teuthology.orchestra.run.smithi012.stdout:mount.ceph: resolved to: "172.21.15.12:6789,172.21.15.165:6789,172.21.15.178:6789" 2024-09-11T00:06:43.807 INFO:teuthology.orchestra.run.smithi012.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.12:6789,172.21.15.165:6789,172.21.15.178:6789:/volumes/qa/sv_1/40f120f8-5df9-4be4-beff-2e141f21ed64 2024-09-11T00:06:43.807 INFO:teuthology.orchestra.run.smithi012.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=legacy,nowsync,key=0,fsid=ff3ad74a-6fcf-11ef-bcea-c7b262605968" will pass to kernel 2024-09-11T00:06:43.810 INFO:teuthology.orchestra.run.smithi012.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-09-11T00:06:43.810 INFO:teuthology.orchestra.run.smithi012.stdout: You just mounted a file system that supports labels which does not 2024-09-11T00:06:43.810 INFO:teuthology.orchestra.run.smithi012.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-09-11T00:06:43.810 INFO:teuthology.orchestra.run.smithi012.stdout: applications will generate AVC messages and not be allowed access to 2024-09-11T00:06:43.810 INFO:teuthology.orchestra.run.smithi012.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-09-11T00:06:43.813 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-11T00:06:43.813 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T00:06:43.813 DEBUG:teuthology.orchestra.run.smithi012:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-11T00:06:43.878 DEBUG:teuthology.orchestra.run.smithi012:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-11T00:06:43.959 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:43 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-11T00:06:43.962 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:43 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-11T00:06:43.970 INFO:teuthology.orchestra.run.smithi012.stdout:client25355 2024-09-11T00:06:43.970 DEBUG:teuthology.orchestra.run.smithi012:> set -ex 2024-09-11T00:06:43.971 DEBUG:teuthology.orchestra.run.smithi012:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-11T00:06:44.007 DEBUG:teuthology.orchestra.run.smithi012:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-11T00:06:44.126 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:44 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-11T00:06:44.130 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:44 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-11T00:06:44.139 INFO:teuthology.orchestra.run.smithi012.stdout:client25355 2024-09-11T00:06:44.139 DEBUG:teuthology.orchestra.run.smithi012:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-11T00:06:44.217 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:44 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-11T00:06:44.220 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:44 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-11T00:06:44.228 INFO:teuthology.orchestra.run.smithi012.stdout:client25355 2024-09-11T00:06:44.229 DEBUG:teuthology.orchestra.run.smithi012:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968.client25355/status') 2024-09-11T00:06:44.309 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:44 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-11T00:06:44.331 INFO:teuthology.orchestra.run.smithi012.stdout:instance: client.25355 (3)172.21.15.12:0/2928550814 2024-09-11T00:06:44.331 INFO:teuthology.orchestra.run.smithi012.stdout:blocklisted: false 2024-09-11T00:06:44.332 INFO:teuthology.orchestra.run.smithi012.stderr:0+1 records in 2024-09-11T00:06:44.332 INFO:teuthology.orchestra.run.smithi012.stderr:0+1 records out 2024-09-11T00:06:44.332 INFO:teuthology.orchestra.run.smithi012.stderr:71 bytes copied, 5.7221e-05 s, 1.2 MB/s 2024-09-11T00:06:44.334 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:44 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-11T00:06:44.343 INFO:teuthology.run_tasks:Running task exec... 2024-09-11T00:06:44.351 INFO:teuthology.task.exec:Executing custom commands... 2024-09-11T00:06:44.352 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi012.front.sepia.ceph.com 2024-09-11T00:06:44.352 DEBUG:teuthology.orchestra.run.smithi012:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs set cephfs balance_automate true' 2024-09-11T00:06:44.509 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:44.507+0000 7f2b48600640 1 -- 172.21.15.12:0/3980658135 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2b40075b20 msgr2=0x7f2b40075fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:06:44.509 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:44.507+0000 7f2b48600640 1 --2- 172.21.15.12:0/3980658135 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2b40075b20 0x7f2b40075fa0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f2b2c009900 tx=0x7f2b2c02a410 comp rx=0 tx=0).stop 2024-09-11T00:06:44.509 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:44.507+0000 7f2b48600640 1 -- 172.21.15.12:0/3980658135 shutdown_connections 2024-09-11T00:06:44.509 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:44.507+0000 7f2b48600640 1 --2- 172.21.15.12:0/3980658135 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2b4010a890 0x7f2b4010cd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:44.510 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:44.507+0000 7f2b48600640 1 --2- 172.21.15.12:0/3980658135 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2b40075b20 0x7f2b40075fa0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:44.510 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:44.507+0000 7f2b48600640 1 --2- 172.21.15.12:0/3980658135 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2b40076f90 0x7f2b400755e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:44.510 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:44.507+0000 7f2b48600640 1 -- 172.21.15.12:0/3980658135 >> 172.21.15.12:0/3980658135 conn(0x7f2b400fe250 msgr2=0x7f2b40100670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:06:44.510 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:44.507+0000 7f2b48600640 1 -- 172.21.15.12:0/3980658135 shutdown_connections 2024-09-11T00:06:44.510 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:44.507+0000 7f2b48600640 1 -- 172.21.15.12:0/3980658135 wait complete. 2024-09-11T00:06:44.512 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:44.508+0000 7f2b48600640 1 Processor -- start 2024-09-11T00:06:44.512 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:44.508+0000 7f2b48600640 1 -- start start 2024-09-11T00:06:44.512 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:44.508+0000 7f2b48600640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2b40075b20 0x7f2b4019af10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:06:44.512 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:44.508+0000 7f2b48600640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2b40076f90 0x7f2b4019b560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:06:44.512 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:44.508+0000 7f2b48600640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2b4010a890 0x7f2b4019c190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:06:44.512 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:44.508+0000 7f2b48600640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f2b40110830 con 0x7f2b40076f90 2024-09-11T00:06:44.512 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:44.508+0000 7f2b48600640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f2b401106b0 con 0x7f2b4010a890 2024-09-11T00:06:44.512 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:44.508+0000 7f2b48600640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f2b401109b0 con 0x7f2b40075b20 2024-09-11T00:06:44.512 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:44.509+0000 7f2b45200640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2b40076f90 0x7f2b4019b560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:06:44.513 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:44.509+0000 7f2b45200640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2b40076f90 0x7f2b4019b560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:41734/0 (socket says 172.21.15.12:41734) 2024-09-11T00:06:44.513 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:44.509+0000 7f2b45200640 1 -- 172.21.15.12:0/2911381848 learned_addr learned my addr 172.21.15.12:0/2911381848 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:06:44.513 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:44.509+0000 7f2b45c00640 1 --2- 172.21.15.12:0/2911381848 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2b40075b20 0x7f2b4019af10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:06:44.513 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:44.509+0000 7f2b46600640 1 --2- 172.21.15.12:0/2911381848 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2b4010a890 0x7f2b4019c190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:06:44.513 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:44.509+0000 7f2b45200640 1 -- 172.21.15.12:0/2911381848 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2b40075b20 msgr2=0x7f2b4019af10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:06:44.513 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:44.509+0000 7f2b45200640 1 --2- 172.21.15.12:0/2911381848 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2b40075b20 0x7f2b4019af10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:44.513 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:44.509+0000 7f2b45200640 1 -- 172.21.15.12:0/2911381848 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2b4010a890 msgr2=0x7f2b4019c190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:06:44.513 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:44.509+0000 7f2b45200640 1 --2- 172.21.15.12:0/2911381848 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2b4010a890 0x7f2b4019c190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:44.513 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:44.509+0000 7f2b45200640 1 -- 172.21.15.12:0/2911381848 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b4019ca50 con 0x7f2b40076f90 2024-09-11T00:06:44.513 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:44.509+0000 7f2b45200640 1 --2- 172.21.15.12:0/2911381848 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2b40076f90 0x7f2b4019b560 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f2b2c005ab0 tx=0x7f2b2c003b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:06:44.513 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:44.510+0000 7f2b3e000640 1 -- 172.21.15.12:0/2911381848 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b2c03f070 con 0x7f2b40076f90 2024-09-11T00:06:44.513 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:44.510+0000 7f2b3e000640 1 -- 172.21.15.12:0/2911381848 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2b2c031c50 con 0x7f2b40076f90 2024-09-11T00:06:44.513 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:44.510+0000 7f2b48600640 1 -- 172.21.15.12:0/2911381848 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2b4019cce0 con 0x7f2b40076f90 2024-09-11T00:06:44.514 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:44.510+0000 7f2b3e000640 1 -- 172.21.15.12:0/2911381848 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b2c035030 con 0x7f2b40076f90 2024-09-11T00:06:44.514 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:44.510+0000 7f2b48600640 1 -- 172.21.15.12:0/2911381848 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2b401a3410 con 0x7f2b40076f90 2024-09-11T00:06:44.514 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:44.511+0000 7f2b48600640 1 -- 172.21.15.12:0/2911381848 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2b40075fa0 con 0x7f2b40076f90 2024-09-11T00:06:44.515 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:44.513+0000 7f2b3e000640 1 -- 172.21.15.12:0/2911381848 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7f2b2c03b460 con 0x7f2b40076f90 2024-09-11T00:06:44.518 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:44.516+0000 7f2b3e000640 1 --2- 172.21.15.12:0/2911381848 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f2b280b52e0 0x7f2b280b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:06:44.519 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:44.516+0000 7f2b3e000640 1 -- 172.21.15.12:0/2911381848 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9208+0+0 (secure 0 0 0) 0x7f2b2c1069e0 con 0x7f2b40076f90 2024-09-11T00:06:44.519 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:44.516+0000 7f2b45c00640 1 --2- 172.21.15.12:0/2911381848 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f2b280b52e0 0x7f2b280b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:06:44.520 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:44.517+0000 7f2b45c00640 1 --2- 172.21.15.12:0/2911381848 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f2b280b52e0 0x7f2b280b77a0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f2b34009bd0 tx=0x7f2b34005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:06:44.521 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:44.520+0000 7f2b3e000640 1 -- 172.21.15.12:0/2911381848 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f2b2c0cce20 con 0x7f2b40076f90 2024-09-11T00:06:44.754 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:44.751+0000 7f2b48600640 1 -- 172.21.15.12:0/2911381848 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"} v 0) -- 0x7f2b40063d50 con 0x7f2b40076f90 2024-09-11T00:06:45.282 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:45.280+0000 7f2b3e000640 1 -- 172.21.15.12:0/2911381848 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]=0 v29) ==== 117+0+0 (secure 0 0 0) 0x7f2b2c0d1850 con 0x7f2b40076f90 2024-09-11T00:06:45.285 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:45.283+0000 7f2b48600640 1 -- 172.21.15.12:0/2911381848 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f2b280b52e0 msgr2=0x7f2b280b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:06:45.285 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:45.283+0000 7f2b48600640 1 --2- 172.21.15.12:0/2911381848 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f2b280b52e0 0x7f2b280b77a0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f2b34009bd0 tx=0x7f2b34005f90 comp rx=0 tx=0).stop 2024-09-11T00:06:45.285 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:45.283+0000 7f2b48600640 1 -- 172.21.15.12:0/2911381848 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2b40076f90 msgr2=0x7f2b4019b560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:06:45.285 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:45.283+0000 7f2b48600640 1 --2- 172.21.15.12:0/2911381848 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2b40076f90 0x7f2b4019b560 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f2b2c005ab0 tx=0x7f2b2c003b90 comp rx=0 tx=0).stop 2024-09-11T00:06:45.285 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:45.283+0000 7f2b48600640 1 -- 172.21.15.12:0/2911381848 shutdown_connections 2024-09-11T00:06:45.285 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:45.283+0000 7f2b48600640 1 --2- 172.21.15.12:0/2911381848 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f2b280b52e0 0x7f2b280b77a0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:45.286 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:45.283+0000 7f2b48600640 1 --2- 172.21.15.12:0/2911381848 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2b4010a890 0x7f2b4019c190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:45.286 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:45.283+0000 7f2b48600640 1 --2- 172.21.15.12:0/2911381848 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2b40076f90 0x7f2b4019b560 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:45.286 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:45.283+0000 7f2b48600640 1 --2- 172.21.15.12:0/2911381848 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2b40075b20 0x7f2b4019af10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:45.286 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:45.283+0000 7f2b48600640 1 -- 172.21.15.12:0/2911381848 >> 172.21.15.12:0/2911381848 conn(0x7f2b400fe250 msgr2=0x7f2b40106870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:06:45.286 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:45.283+0000 7f2b48600640 1 -- 172.21.15.12:0/2911381848 shutdown_connections 2024-09-11T00:06:45.286 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:45.283+0000 7f2b48600640 1 -- 172.21.15.12:0/2911381848 wait complete. 2024-09-11T00:06:45.303 INFO:teuthology.run_tasks:Running task check-counter... 2024-09-11T00:06:45.317 INFO:teuthology.run_tasks:Running task background_exec... 2024-09-11T00:06:45.326 INFO:teuthology.task.background_exec:Running background command on role mon.a host ubuntu@smithi012.front.sepia.ceph.com 2024-09-11T00:06:45.327 DEBUG:teuthology.orchestra.run.smithi012:> sudo TESTDIR=/home/ubuntu/cephtest daemon-helper kill --kill-group bash -c 'while sleep 13; do ceph tell mds.cephfs:0 flush journal; done' 2024-09-11T00:06:45.331 INFO:teuthology.run_tasks:Running task quiescer... 2024-09-11T00:06:45.343 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-09-11T00:06:45.344 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid ff3ad74a-6fcf-11ef-bcea-c7b262605968 -- ceph pg dump --format=json 2024-09-11T00:06:45.532 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:06:45 smithi165 ceph-mon[35356]: pgmap v388: 129 pgs: 129 active+clean; 647 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-11T00:06:45.532 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:06:45 smithi165 ceph-mon[35356]: from='client.? 172.21.15.12:0/2911381848' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"} : dispatch 2024-09-11T00:06:45.562 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:06:45 smithi012 ceph-mon[30333]: pgmap v388: 129 pgs: 129 active+clean; 647 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-11T00:06:45.563 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:06:45 smithi012 ceph-mon[30333]: from='client.? 172.21.15.12:0/2911381848' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"} : dispatch 2024-09-11T00:06:45.581 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:06:45 smithi178 ceph-mon[35919]: pgmap v388: 129 pgs: 129 active+clean; 647 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-11T00:06:45.581 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:06:45 smithi178 ceph-mon[35919]: from='client.? 172.21.15.12:0/2911381848' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"} : dispatch 2024-09-11T00:06:45.727 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:06:46.323 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:46.321+0000 7fbf65600640 1 -- 172.21.15.12:0/461622297 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbf60109130 msgr2=0x7fbf6010f930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:06:46.323 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:46.321+0000 7fbf65600640 1 --2- 172.21.15.12:0/461622297 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbf60109130 0x7fbf6010f930 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fbf58009ca0 tx=0x7fbf5802a8b0 comp rx=0 tx=0).stop 2024-09-11T00:06:46.324 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:46.321+0000 7fbf65600640 1 -- 172.21.15.12:0/461622297 shutdown_connections 2024-09-11T00:06:46.324 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:46.321+0000 7fbf65600640 1 --2- 172.21.15.12:0/461622297 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbf60109130 0x7fbf6010f930 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:46.324 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:46.321+0000 7fbf65600640 1 --2- 172.21.15.12:0/461622297 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbf601087b0 0x7fbf60108bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:46.324 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:46.321+0000 7fbf65600640 1 --2- 172.21.15.12:0/461622297 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbf60102da0 0x7fbf60103180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:46.324 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:46.321+0000 7fbf65600640 1 -- 172.21.15.12:0/461622297 >> 172.21.15.12:0/461622297 conn(0x7fbf600fe810 msgr2=0x7fbf60100c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:06:46.324 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:46.321+0000 7fbf65600640 1 -- 172.21.15.12:0/461622297 shutdown_connections 2024-09-11T00:06:46.324 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:46.321+0000 7fbf65600640 1 -- 172.21.15.12:0/461622297 wait complete. 2024-09-11T00:06:46.325 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:46.322+0000 7fbf65600640 1 Processor -- start 2024-09-11T00:06:46.325 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:46.322+0000 7fbf65600640 1 -- start start 2024-09-11T00:06:46.325 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:46.323+0000 7fbf65600640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbf60102da0 0x7fbf60199210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:06:46.325 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:46.323+0000 7fbf65600640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbf601087b0 0x7fbf60199750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:06:46.325 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:46.323+0000 7fbf65600640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbf60109130 0x7fbf6019eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:06:46.325 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:46.323+0000 7fbf65600640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fbf60112db0 con 0x7fbf60109130 2024-09-11T00:06:46.325 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:46.323+0000 7fbf65600640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fbf60112c30 con 0x7fbf60102da0 2024-09-11T00:06:46.325 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:46.323+0000 7fbf65600640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fbf60112f30 con 0x7fbf601087b0 2024-09-11T00:06:46.326 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:46.323+0000 7fbf5ea00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbf60102da0 0x7fbf60199210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:06:46.326 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:46.323+0000 7fbf5ea00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbf60102da0 0x7fbf60199210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.12:59842/0 (socket says 172.21.15.12:59842) 2024-09-11T00:06:46.326 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:46.323+0000 7fbf5ea00640 1 -- 172.21.15.12:0/1506440460 learned_addr learned my addr 172.21.15.12:0/1506440460 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:06:46.326 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:46.323+0000 7fbf5f400640 1 --2- 172.21.15.12:0/1506440460 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbf60109130 0x7fbf6019eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:06:46.326 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:46.323+0000 7fbf56000640 1 --2- 172.21.15.12:0/1506440460 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbf601087b0 0x7fbf60199750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:06:46.327 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:46.324+0000 7fbf5ea00640 1 -- 172.21.15.12:0/1506440460 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbf601087b0 msgr2=0x7fbf60199750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:06:46.327 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:46.324+0000 7fbf5ea00640 1 --2- 172.21.15.12:0/1506440460 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbf601087b0 0x7fbf60199750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:46.327 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:46.324+0000 7fbf5ea00640 1 -- 172.21.15.12:0/1506440460 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbf60109130 msgr2=0x7fbf6019eaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:06:46.327 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:46.324+0000 7fbf5ea00640 1 --2- 172.21.15.12:0/1506440460 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbf60109130 0x7fbf6019eaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:46.327 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:46.324+0000 7fbf5ea00640 1 -- 172.21.15.12:0/1506440460 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf6019f2c0 con 0x7fbf60102da0 2024-09-11T00:06:46.327 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:46.324+0000 7fbf5ea00640 1 --2- 172.21.15.12:0/1506440460 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbf60102da0 0x7fbf60199210 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fbf5000e9f0 tx=0x7fbf5000eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:06:46.327 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:46.324+0000 7fbf57e00640 1 -- 172.21.15.12:0/1506440460 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf5000cd80 con 0x7fbf60102da0 2024-09-11T00:06:46.327 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:46.324+0000 7fbf57e00640 1 -- 172.21.15.12:0/1506440460 <== mon.2 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbf500108e0 con 0x7fbf60102da0 2024-09-11T00:06:46.327 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:46.324+0000 7fbf57e00640 1 -- 172.21.15.12:0/1506440460 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf50018980 con 0x7fbf60102da0 2024-09-11T00:06:46.327 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:46.324+0000 7fbf65600640 1 -- 172.21.15.12:0/1506440460 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbf6019f5b0 con 0x7fbf60102da0 2024-09-11T00:06:46.327 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:46.324+0000 7fbf65600640 1 -- 172.21.15.12:0/1506440460 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbf601a5c70 con 0x7fbf60102da0 2024-09-11T00:06:46.328 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:46.325+0000 7fbf65600640 1 -- 172.21.15.12:0/1506440460 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbf60103180 con 0x7fbf60102da0 2024-09-11T00:06:46.329 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:46.328+0000 7fbf57e00640 1 -- 172.21.15.12:0/1506440460 <== mon.2 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7fbf50010430 con 0x7fbf60102da0 2024-09-11T00:06:46.334 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:46.329+0000 7fbf57e00640 1 --2- 172.21.15.12:0/1506440460 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fbf440b52e0 0x7fbf440b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:06:46.334 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:46.329+0000 7fbf57e00640 1 -- 172.21.15.12:0/1506440460 <== mon.2 v2:172.21.15.165:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9208+0+0 (secure 0 0 0) 0x7fbf500e6560 con 0x7fbf60102da0 2024-09-11T00:06:46.334 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:46.329+0000 7fbf56000640 1 --2- 172.21.15.12:0/1506440460 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fbf440b52e0 0x7fbf440b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:06:46.334 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:46.330+0000 7fbf56000640 1 --2- 172.21.15.12:0/1506440460 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fbf440b52e0 0x7fbf440b77a0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fbf480095f0 tx=0x7fbf48005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:06:46.334 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:46.331+0000 7fbf57e00640 1 -- 172.21.15.12:0/1506440460 <== mon.2 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7fbf500aca00 con 0x7fbf60102da0 2024-09-11T00:06:46.525 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:46.523+0000 7fbf65600640 1 -- 172.21.15.12:0/1506440460 --> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fbf60063590 con 0x7fbf440b52e0 2024-09-11T00:06:46.525 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:06:46 smithi012 ceph-mon[30333]: from='client.? 172.21.15.12:0/2911381848' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-09-11T00:06:46.525 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:06:46 smithi012 ceph-mon[30333]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T00:06:46.531 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:46.529+0000 7fbf57e00640 1 -- 172.21.15.12:0/1506440460 <== mgr.14150 v2:172.21.15.12:6800/3442939137 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340533 (secure 0 0 0) 0x7fbf60063590 con 0x7fbf440b52e0 2024-09-11T00:06:46.539 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-11T00:06:46.541 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-11T00:06:46.543 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:46.534+0000 7fbf65600640 1 -- 172.21.15.12:0/1506440460 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fbf440b52e0 msgr2=0x7fbf440b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:06:46.544 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:46.534+0000 7fbf65600640 1 --2- 172.21.15.12:0/1506440460 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fbf440b52e0 0x7fbf440b77a0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fbf480095f0 tx=0x7fbf48005f90 comp rx=0 tx=0).stop 2024-09-11T00:06:46.544 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:46.534+0000 7fbf65600640 1 -- 172.21.15.12:0/1506440460 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbf60102da0 msgr2=0x7fbf60199210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:06:46.544 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:46.534+0000 7fbf65600640 1 --2- 172.21.15.12:0/1506440460 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbf60102da0 0x7fbf60199210 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fbf5000e9f0 tx=0x7fbf5000eeb0 comp rx=0 tx=0).stop 2024-09-11T00:06:46.544 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:46.534+0000 7fbf65600640 1 -- 172.21.15.12:0/1506440460 shutdown_connections 2024-09-11T00:06:46.544 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:46.534+0000 7fbf65600640 1 --2- 172.21.15.12:0/1506440460 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fbf440b52e0 0x7fbf440b77a0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:46.544 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:46.534+0000 7fbf65600640 1 --2- 172.21.15.12:0/1506440460 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbf60109130 0x7fbf6019eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:46.544 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:46.534+0000 7fbf65600640 1 --2- 172.21.15.12:0/1506440460 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbf601087b0 0x7fbf60199750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:46.544 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:46.534+0000 7fbf65600640 1 --2- 172.21.15.12:0/1506440460 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbf60102da0 0x7fbf60199210 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:46.544 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:46.534+0000 7fbf65600640 1 -- 172.21.15.12:0/1506440460 >> 172.21.15.12:0/1506440460 conn(0x7fbf600fe810 msgr2=0x7fbf600ff0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:06:46.544 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:46.534+0000 7fbf65600640 1 -- 172.21.15.12:0/1506440460 shutdown_connections 2024-09-11T00:06:46.544 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:46.534+0000 7fbf65600640 1 -- 172.21.15.12:0/1506440460 wait complete. 2024-09-11T00:06:46.581 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:06:46 smithi178 ceph-mon[35919]: from='client.? 172.21.15.12:0/2911381848' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-09-11T00:06:46.581 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:06:46 smithi178 ceph-mon[35919]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T00:06:46.782 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:06:46 smithi165 ceph-mon[35356]: from='client.? 172.21.15.12:0/2911381848' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-09-11T00:06:46.782 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:06:46 smithi165 ceph-mon[35356]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T00:06:47.419 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":389,"stamp":"2024-09-11T00:06:46.132411+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":663577,"num_objects":102,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":1277,"num_read_kb":745,"num_write":387,"num_write_kb":2307,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":24,"num_bytes_recovered":2952069,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":821,"ondisk_log_size":821,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":387940,"kb_used_data":13664,"kb_used_omap":18,"kb_used_meta":373997,"kb_avail":1124553884,"statfs":{"total":1151940427776,"available":1151543177216,"internally_reserved":0,"allocated":13991936,"data_stored":7800151,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19142,"internal_metadata":382973242},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":622,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":152,"num_read_kb":77,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003053"},"pg_stats":[{"pgid":"2.1f","version":"83'14","reported_seq":171,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T00:06:42.428729+0000","last_change":"2024-09-11T00:04:01.820872+0000","last_active":"2024-09-11T00:06:42.428729+0000","last_peered":"2024-09-11T00:06:42.428729+0000","last_clean":"2024-09-11T00:06:42.428729+0000","last_became_active":"2024-09-11T00:04:01.820404+0000","last_became_peered":"2024-09-11T00:04:01.820404+0000","last_unstale":"2024-09-11T00:06:42.428729+0000","last_undegraded":"2024-09-11T00:06:42.428729+0000","last_fullsized":"2024-09-11T00:06:42.428729+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_clean_scrub_stamp":"2024-09-11T00:04:00.794959+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:15:38.648289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":115,"num_read_kb":115,"num_write":15,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"83'5","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T00:06:20.218414+0000","last_change":"2024-09-11T00:05:19.846978+0000","last_active":"2024-09-11T00:06:20.218414+0000","last_peered":"2024-09-11T00:06:20.218414+0000","last_clean":"2024-09-11T00:06:20.218414+0000","last_became_active":"2024-09-11T00:05:19.846569+0000","last_became_peered":"2024-09-11T00:05:19.846569+0000","last_unstale":"2024-09-11T00:06:20.218414+0000","last_undegraded":"2024-09-11T00:06:20.218414+0000","last_fullsized":"2024-09-11T00:06:20.218414+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:27:46.235552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,5],"acting":[9,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":30,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T00:06:20.208028+0000","last_change":"2024-09-11T00:04:11.872109+0000","last_active":"2024-09-11T00:06:20.208028+0000","last_peered":"2024-09-11T00:06:20.208028+0000","last_clean":"2024-09-11T00:06:20.208028+0000","last_became_active":"2024-09-11T00:04:11.871279+0000","last_became_peered":"2024-09-11T00:04:11.871279+0000","last_unstale":"2024-09-11T00:06:20.208028+0000","last_undegraded":"2024-09-11T00:06:20.208028+0000","last_fullsized":"2024-09-11T00:06:20.208028+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_clean_scrub_stamp":"2024-09-11T00:04:10.845136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:13:32.837555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"81'2","reported_seq":44,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T00:06:20.224031+0000","last_change":"2024-09-11T00:04:01.942288+0000","last_active":"2024-09-11T00:06:20.224031+0000","last_peered":"2024-09-11T00:06:20.224031+0000","last_clean":"2024-09-11T00:06:20.224031+0000","last_became_active":"2024-09-11T00:04:01.941838+0000","last_became_peered":"2024-09-11T00:04:01.941838+0000","last_unstale":"2024-09-11T00:06:20.224031+0000","last_undegraded":"2024-09-11T00:06:20.224031+0000","last_fullsized":"2024-09-11T00:06:20.224031+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_clean_scrub_stamp":"2024-09-11T00:04:00.794959+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:48:44.867267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"83'6","reported_seq":43,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T00:06:20.209960+0000","last_change":"2024-09-11T00:04:03.839951+0000","last_active":"2024-09-11T00:06:20.209960+0000","last_peered":"2024-09-11T00:06:20.209960+0000","last_clean":"2024-09-11T00:06:20.209960+0000","last_became_active":"2024-09-11T00:04:03.839591+0000","last_became_peered":"2024-09-11T00:04:03.839591+0000","last_unstale":"2024-09-11T00:06:20.209960+0000","last_undegraded":"2024-09-11T00:06:20.209960+0000","last_fullsized":"2024-09-11T00:06:20.209960+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:12:49.603310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":30,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T00:06:20.210052+0000","last_change":"2024-09-11T00:04:11.872026+0000","last_active":"2024-09-11T00:06:20.210052+0000","last_peered":"2024-09-11T00:06:20.210052+0000","last_clean":"2024-09-11T00:06:20.210052+0000","last_became_active":"2024-09-11T00:04:11.871699+0000","last_became_peered":"2024-09-11T00:04:11.871699+0000","last_unstale":"2024-09-11T00:06:20.210052+0000","last_undegraded":"2024-09-11T00:06:20.210052+0000","last_fullsized":"2024-09-11T00:06:20.210052+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_clean_scrub_stamp":"2024-09-11T00:04:10.845136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:07:47.757223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"81'12","reported_seq":55,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T00:06:20.215846+0000","last_change":"2024-09-11T00:04:01.819942+0000","last_active":"2024-09-11T00:06:20.215846+0000","last_peered":"2024-09-11T00:06:20.215846+0000","last_clean":"2024-09-11T00:06:20.215846+0000","last_became_active":"2024-09-11T00:04:01.819337+0000","last_became_peered":"2024-09-11T00:04:01.819337+0000","last_unstale":"2024-09-11T00:06:20.215846+0000","last_undegraded":"2024-09-11T00:06:20.215846+0000","last_fullsized":"2024-09-11T00:06:20.215846+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_clean_scrub_stamp":"2024-09-11T00:04:00.794959+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:22:26.947067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"83'9","reported_seq":46,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T00:06:20.207881+0000","last_change":"2024-09-11T00:04:03.839276+0000","last_active":"2024-09-11T00:06:20.207881+0000","last_peered":"2024-09-11T00:06:20.207881+0000","last_clean":"2024-09-11T00:06:20.207881+0000","last_became_active":"2024-09-11T00:04:03.838843+0000","last_became_peered":"2024-09-11T00:04:03.838843+0000","last_unstale":"2024-09-11T00:06:20.207881+0000","last_undegraded":"2024-09-11T00:06:20.207881+0000","last_fullsized":"2024-09-11T00:06:20.207881+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:26:42.640628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":30,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T00:06:20.210814+0000","last_change":"2024-09-11T00:04:11.872345+0000","last_active":"2024-09-11T00:06:20.210814+0000","last_peered":"2024-09-11T00:06:20.210814+0000","last_clean":"2024-09-11T00:06:20.210814+0000","last_became_active":"2024-09-11T00:04:11.871994+0000","last_became_peered":"2024-09-11T00:04:11.871994+0000","last_unstale":"2024-09-11T00:06:20.210814+0000","last_undegraded":"2024-09-11T00:06:20.210814+0000","last_fullsized":"2024-09-11T00:06:20.210814+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_clean_scrub_stamp":"2024-09-11T00:04:10.845136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:14:23.906778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"81'5","reported_seq":48,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T00:06:20.215746+0000","last_change":"2024-09-11T00:04:01.819581+0000","last_active":"2024-09-11T00:06:20.215746+0000","last_peered":"2024-09-11T00:06:20.215746+0000","last_clean":"2024-09-11T00:06:20.215746+0000","last_became_active":"2024-09-11T00:04:01.818699+0000","last_became_peered":"2024-09-11T00:04:01.818699+0000","last_unstale":"2024-09-11T00:06:20.215746+0000","last_undegraded":"2024-09-11T00:06:20.215746+0000","last_fullsized":"2024-09-11T00:06:20.215746+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_clean_scrub_stamp":"2024-09-11T00:04:00.794959+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:07:24.540825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"83'15","reported_seq":52,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T00:06:20.218131+0000","last_change":"2024-09-11T00:04:03.835107+0000","last_active":"2024-09-11T00:06:20.218131+0000","last_peered":"2024-09-11T00:06:20.218131+0000","last_clean":"2024-09-11T00:06:20.218131+0000","last_became_active":"2024-09-11T00:04:03.834782+0000","last_became_peered":"2024-09-11T00:04:03.834782+0000","last_unstale":"2024-09-11T00:06:20.218131+0000","last_undegraded":"2024-09-11T00:06:20.218131+0000","last_fullsized":"2024-09-11T00:06:20.218131+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:46:54.472270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":30,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T00:06:20.209753+0000","last_change":"2024-09-11T00:04:11.874168+0000","last_active":"2024-09-11T00:06:20.209753+0000","last_peered":"2024-09-11T00:06:20.209753+0000","last_clean":"2024-09-11T00:06:20.209753+0000","last_became_active":"2024-09-11T00:04:11.873810+0000","last_became_peered":"2024-09-11T00:04:11.873810+0000","last_unstale":"2024-09-11T00:06:20.209753+0000","last_undegraded":"2024-09-11T00:06:20.209753+0000","last_fullsized":"2024-09-11T00:06:20.209753+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_clean_scrub_stamp":"2024-09-11T00:04:10.845136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:06:05.130815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"83'5","reported_seq":42,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T00:06:20.214579+0000","last_change":"2024-09-11T00:04:03.834016+0000","last_active":"2024-09-11T00:06:20.214579+0000","last_peered":"2024-09-11T00:06:20.214579+0000","last_clean":"2024-09-11T00:06:20.214579+0000","last_became_active":"2024-09-11T00:04:03.833374+0000","last_became_peered":"2024-09-11T00:04:03.833374+0000","last_unstale":"2024-09-11T00:06:20.214579+0000","last_undegraded":"2024-09-11T00:06:20.214579+0000","last_fullsized":"2024-09-11T00:06:20.214579+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:13:34.068594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"81'4","reported_seq":47,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T00:06:20.215423+0000","last_change":"2024-09-11T00:04:01.821173+0000","last_active":"2024-09-11T00:06:20.215423+0000","last_peered":"2024-09-11T00:06:20.215423+0000","last_clean":"2024-09-11T00:06:20.215423+0000","last_became_active":"2024-09-11T00:04:01.820615+0000","last_became_peered":"2024-09-11T00:04:01.820615+0000","last_unstale":"2024-09-11T00:06:20.215423+0000","last_undegraded":"2024-09-11T00:06:20.215423+0000","last_fullsized":"2024-09-11T00:06:20.215423+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_clean_scrub_stamp":"2024-09-11T00:04:00.794959+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:22:11.842837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":30,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T00:06:20.210190+0000","last_change":"2024-09-11T00:04:12.077637+0000","last_active":"2024-09-11T00:06:20.210190+0000","last_peered":"2024-09-11T00:06:20.210190+0000","last_clean":"2024-09-11T00:06:20.210190+0000","last_became_active":"2024-09-11T00:04:12.076450+0000","last_became_peered":"2024-09-11T00:04:12.076450+0000","last_unstale":"2024-09-11T00:06:20.210190+0000","last_undegraded":"2024-09-11T00:06:20.210190+0000","last_fullsized":"2024-09-11T00:06:20.210190+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_clean_scrub_stamp":"2024-09-11T00:04:10.845136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:50:54.870468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"83'8","reported_seq":45,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T00:06:20.223570+0000","last_change":"2024-09-11T00:04:03.846398+0000","last_active":"2024-09-11T00:06:20.223570+0000","last_peered":"2024-09-11T00:06:20.223570+0000","last_clean":"2024-09-11T00:06:20.223570+0000","last_became_active":"2024-09-11T00:04:03.846051+0000","last_became_peered":"2024-09-11T00:04:03.846051+0000","last_unstale":"2024-09-11T00:06:20.223570+0000","last_undegraded":"2024-09-11T00:06:20.223570+0000","last_fullsized":"2024-09-11T00:06:20.223570+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:00:38.506122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"81'3","reported_seq":22,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T00:06:20.217264+0000","last_change":"2024-09-11T00:05:19.860048+0000","last_active":"2024-09-11T00:06:20.217264+0000","last_peered":"2024-09-11T00:06:20.217264+0000","last_clean":"2024-09-11T00:06:20.217264+0000","last_became_active":"2024-09-11T00:05:19.845210+0000","last_became_peered":"2024-09-11T00:05:19.845210+0000","last_unstale":"2024-09-11T00:06:20.217264+0000","last_undegraded":"2024-09-11T00:06:20.217264+0000","last_fullsized":"2024-09-11T00:06:20.217264+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_clean_scrub_stamp":"2024-09-11T00:04:00.794959+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:55:18.935136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"81'1","reported_seq":31,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T00:06:20.210206+0000","last_change":"2024-09-11T00:04:11.879582+0000","last_active":"2024-09-11T00:06:20.210206+0000","last_peered":"2024-09-11T00:06:20.210206+0000","last_clean":"2024-09-11T00:06:20.210206+0000","last_became_active":"2024-09-11T00:04:11.879098+0000","last_became_peered":"2024-09-11T00:04:11.879098+0000","last_unstale":"2024-09-11T00:06:20.210206+0000","last_undegraded":"2024-09-11T00:06:20.210206+0000","last_fullsized":"2024-09-11T00:06:20.210206+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_clean_scrub_stamp":"2024-09-11T00:04:10.845136+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:06.292841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"83'9","reported_seq":46,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T00:06:20.218519+0000","last_change":"2024-09-11T00:04:03.834615+0000","last_active":"2024-09-11T00:06:20.218519+0000","last_peered":"2024-09-11T00:06:20.218519+0000","last_clean":"2024-09-11T00:06:20.218519+0000","last_became_active":"2024-09-11T00:04:03.834284+0000","last_became_peered":"2024-09-11T00:04:03.834284+0000","last_unstale":"2024-09-11T00:06:20.218519+0000","last_undegraded":"2024-09-11T00:06:20.218519+0000","last_fullsized":"2024-09-11T00:06:20.218519+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:24:09.542774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"81'7","reported_seq":49,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T00:06:20.223445+0000","last_change":"2024-09-11T00:04:01.818156+0000","last_active":"2024-09-11T00:06:20.223445+0000","last_peered":"2024-09-11T00:06:20.223445+0000","last_clean":"2024-09-11T00:06:20.223445+0000","last_became_active":"2024-09-11T00:04:01.817820+0000","last_became_peered":"2024-09-11T00:04:01.817820+0000","last_unstale":"2024-09-11T00:06:20.223445+0000","last_undegraded":"2024-09-11T00:06:20.223445+0000","last_fullsized":"2024-09-11T00:06:20.223445+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_clean_scrub_stamp":"2024-09-11T00:04:00.794959+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:15:46.304276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":30,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T00:06:20.218603+0000","last_change":"2024-09-11T00:04:11.877704+0000","last_active":"2024-09-11T00:06:20.218603+0000","last_peered":"2024-09-11T00:06:20.218603+0000","last_clean":"2024-09-11T00:06:20.218603+0000","last_became_active":"2024-09-11T00:04:11.877320+0000","last_became_peered":"2024-09-11T00:04:11.877320+0000","last_unstale":"2024-09-11T00:06:20.218603+0000","last_undegraded":"2024-09-11T00:06:20.218603+0000","last_fullsized":"2024-09-11T00:06:20.218603+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_clean_scrub_stamp":"2024-09-11T00:04:10.845136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:31:35.046012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.9","version":"83'2","reported_seq":39,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T00:06:20.215128+0000","last_change":"2024-09-11T00:04:03.828240+0000","last_active":"2024-09-11T00:06:20.215128+0000","last_peered":"2024-09-11T00:06:20.215128+0000","last_clean":"2024-09-11T00:06:20.215128+0000","last_became_active":"2024-09-11T00:04:03.827890+0000","last_became_peered":"2024-09-11T00:04:03.827890+0000","last_unstale":"2024-09-11T00:06:20.215128+0000","last_undegraded":"2024-09-11T00:06:20.215128+0000","last_fullsized":"2024-09-11T00:06:20.215128+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:43:12.146717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"81'2","reported_seq":44,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T00:06:20.207648+0000","last_change":"2024-09-11T00:04:01.942805+0000","last_active":"2024-09-11T00:06:20.207648+0000","last_peered":"2024-09-11T00:06:20.207648+0000","last_clean":"2024-09-11T00:06:20.207648+0000","last_became_active":"2024-09-11T00:04:01.942429+0000","last_became_peered":"2024-09-11T00:04:01.942429+0000","last_unstale":"2024-09-11T00:06:20.207648+0000","last_undegraded":"2024-09-11T00:06:20.207648+0000","last_fullsized":"2024-09-11T00:06:20.207648+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_clean_scrub_stamp":"2024-09-11T00:04:00.794959+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:51:22.403825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":30,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T00:06:20.210426+0000","last_change":"2024-09-11T00:04:12.077246+0000","last_active":"2024-09-11T00:06:20.210426+0000","last_peered":"2024-09-11T00:06:20.210426+0000","last_clean":"2024-09-11T00:06:20.210426+0000","last_became_active":"2024-09-11T00:04:12.076481+0000","last_became_peered":"2024-09-11T00:04:12.076481+0000","last_unstale":"2024-09-11T00:06:20.210426+0000","last_undegraded":"2024-09-11T00:06:20.210426+0000","last_fullsized":"2024-09-11T00:06:20.210426+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_clean_scrub_stamp":"2024-09-11T00:04:10.845136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:55:54.347334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"83'7","reported_seq":44,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T00:06:20.209811+0000","last_change":"2024-09-11T00:04:03.838431+0000","last_active":"2024-09-11T00:06:20.209811+0000","last_peered":"2024-09-11T00:06:20.209811+0000","last_clean":"2024-09-11T00:06:20.209811+0000","last_became_active":"2024-09-11T00:04:03.834451+0000","last_became_peered":"2024-09-11T00:04:03.834451+0000","last_unstale":"2024-09-11T00:06:20.209811+0000","last_undegraded":"2024-09-11T00:06:20.209811+0000","last_fullsized":"2024-09-11T00:06:20.209811+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:23:01.594299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"81'7","reported_seq":50,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T00:06:20.210677+0000","last_change":"2024-09-11T00:04:02.588834+0000","last_active":"2024-09-11T00:06:20.210677+0000","last_peered":"2024-09-11T00:06:20.210677+0000","last_clean":"2024-09-11T00:06:20.210677+0000","last_became_active":"2024-09-11T00:04:02.588477+0000","last_became_peered":"2024-09-11T00:04:02.588477+0000","last_unstale":"2024-09-11T00:06:20.210677+0000","last_undegraded":"2024-09-11T00:06:20.210677+0000","last_fullsized":"2024-09-11T00:06:20.210677+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_clean_scrub_stamp":"2024-09-11T00:04:00.794959+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:31:21.805591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":30,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T00:06:20.207762+0000","last_change":"2024-09-11T00:04:11.863381+0000","last_active":"2024-09-11T00:06:20.207762+0000","last_peered":"2024-09-11T00:06:20.207762+0000","last_clean":"2024-09-11T00:06:20.207762+0000","last_became_active":"2024-09-11T00:04:11.862944+0000","last_became_peered":"2024-09-11T00:04:11.862944+0000","last_unstale":"2024-09-11T00:06:20.207762+0000","last_undegraded":"2024-09-11T00:06:20.207762+0000","last_fullsized":"2024-09-11T00:06:20.207762+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_clean_scrub_stamp":"2024-09-11T00:04:10.845136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:00:14.185992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"83'5","reported_seq":42,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T00:06:20.224159+0000","last_change":"2024-09-11T00:04:03.829750+0000","last_active":"2024-09-11T00:06:20.224159+0000","last_peered":"2024-09-11T00:06:20.224159+0000","last_clean":"2024-09-11T00:06:20.224159+0000","last_became_active":"2024-09-11T00:04:03.828166+0000","last_became_peered":"2024-09-11T00:04:03.828166+0000","last_unstale":"2024-09-11T00:06:20.224159+0000","last_undegraded":"2024-09-11T00:06:20.224159+0000","last_fullsized":"2024-09-11T00:06:20.224159+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:55:01.667075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"83'16","reported_seq":58,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T00:06:20.218702+0000","last_change":"2024-09-11T00:04:01.818083+0000","last_active":"2024-09-11T00:06:20.218702+0000","last_peered":"2024-09-11T00:06:20.218702+0000","last_clean":"2024-09-11T00:06:20.218702+0000","last_became_active":"2024-09-11T00:04:01.817784+0000","last_became_peered":"2024-09-11T00:04:01.817784+0000","last_unstale":"2024-09-11T00:06:20.218702+0000","last_undegraded":"2024-09-11T00:06:20.218702+0000","last_fullsized":"2024-09-11T00:06:20.218702+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_clean_scrub_stamp":"2024-09-11T00:04:00.794959+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:38:55.466913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":28,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":30,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T00:06:20.223043+0000","last_change":"2024-09-11T00:04:12.077053+0000","last_active":"2024-09-11T00:06:20.223043+0000","last_peered":"2024-09-11T00:06:20.223043+0000","last_clean":"2024-09-11T00:06:20.223043+0000","last_became_active":"2024-09-11T00:04:12.076632+0000","last_became_peered":"2024-09-11T00:04:12.076632+0000","last_unstale":"2024-09-11T00:06:20.223043+0000","last_undegraded":"2024-09-11T00:06:20.223043+0000","last_fullsized":"2024-09-11T00:06:20.223043+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_clean_scrub_stamp":"2024-09-11T00:04:10.845136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:19:38.321429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"83'9","reported_seq":46,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T00:06:20.224608+0000","last_change":"2024-09-11T00:04:03.830316+0000","last_active":"2024-09-11T00:06:20.224608+0000","last_peered":"2024-09-11T00:06:20.224608+0000","last_clean":"2024-09-11T00:06:20.224608+0000","last_became_active":"2024-09-11T00:04:03.829932+0000","last_became_peered":"2024-09-11T00:04:03.829932+0000","last_unstale":"2024-09-11T00:06:20.224608+0000","last_undegraded":"2024-09-11T00:06:20.224608+0000","last_fullsized":"2024-09-11T00:06:20.224608+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:06:14.546651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"81'4","reported_seq":47,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T00:06:20.215888+0000","last_change":"2024-09-11T00:04:01.943004+0000","last_active":"2024-09-11T00:06:20.215888+0000","last_peered":"2024-09-11T00:06:20.215888+0000","last_clean":"2024-09-11T00:06:20.215888+0000","last_became_active":"2024-09-11T00:04:01.942432+0000","last_became_peered":"2024-09-11T00:04:01.942432+0000","last_unstale":"2024-09-11T00:06:20.215888+0000","last_undegraded":"2024-09-11T00:06:20.215888+0000","last_fullsized":"2024-09-11T00:06:20.215888+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_clean_scrub_stamp":"2024-09-11T00:04:00.794959+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:52:50.472596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":30,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T00:06:20.218250+0000","last_change":"2024-09-11T00:04:11.866778+0000","last_active":"2024-09-11T00:06:20.218250+0000","last_peered":"2024-09-11T00:06:20.218250+0000","last_clean":"2024-09-11T00:06:20.218250+0000","last_became_active":"2024-09-11T00:04:11.866333+0000","last_became_peered":"2024-09-11T00:04:11.866333+0000","last_unstale":"2024-09-11T00:06:20.218250+0000","last_undegraded":"2024-09-11T00:06:20.218250+0000","last_fullsized":"2024-09-11T00:06:20.218250+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_clean_scrub_stamp":"2024-09-11T00:04:10.845136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:33.680119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"83'4","reported_seq":41,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T00:06:20.224266+0000","last_change":"2024-09-11T00:04:03.832481+0000","last_active":"2024-09-11T00:06:20.224266+0000","last_peered":"2024-09-11T00:06:20.224266+0000","last_clean":"2024-09-11T00:06:20.224266+0000","last_became_active":"2024-09-11T00:04:03.832162+0000","last_became_peered":"2024-09-11T00:04:03.832162+0000","last_unstale":"2024-09-11T00:06:20.224266+0000","last_undegraded":"2024-09-11T00:06:20.224266+0000","last_fullsized":"2024-09-11T00:06:20.224266+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:32:52.550128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"81'1","reported_seq":43,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T00:06:20.218249+0000","last_change":"2024-09-11T00:04:01.943647+0000","last_active":"2024-09-11T00:06:20.218249+0000","last_peered":"2024-09-11T00:06:20.218249+0000","last_clean":"2024-09-11T00:06:20.218249+0000","last_became_active":"2024-09-11T00:04:01.942714+0000","last_became_peered":"2024-09-11T00:04:01.942714+0000","last_unstale":"2024-09-11T00:06:20.218249+0000","last_undegraded":"2024-09-11T00:06:20.218249+0000","last_fullsized":"2024-09-11T00:06:20.218249+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_clean_scrub_stamp":"2024-09-11T00:04:00.794959+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:22:23.687268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":30,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T00:06:20.215230+0000","last_change":"2024-09-11T00:04:12.077133+0000","last_active":"2024-09-11T00:06:20.215230+0000","last_peered":"2024-09-11T00:06:20.215230+0000","last_clean":"2024-09-11T00:06:20.215230+0000","last_became_active":"2024-09-11T00:04:12.076657+0000","last_became_peered":"2024-09-11T00:04:12.076657+0000","last_unstale":"2024-09-11T00:06:20.215230+0000","last_undegraded":"2024-09-11T00:06:20.215230+0000","last_fullsized":"2024-09-11T00:06:20.215230+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_clean_scrub_stamp":"2024-09-11T00:04:10.845136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:05:40.643405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"83'7","reported_seq":44,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T00:06:20.224150+0000","last_change":"2024-09-11T00:04:03.835454+0000","last_active":"2024-09-11T00:06:20.224150+0000","last_peered":"2024-09-11T00:06:20.224150+0000","last_clean":"2024-09-11T00:06:20.224150+0000","last_became_active":"2024-09-11T00:04:03.835122+0000","last_became_peered":"2024-09-11T00:04:03.835122+0000","last_unstale":"2024-09-11T00:06:20.224150+0000","last_undegraded":"2024-09-11T00:06:20.224150+0000","last_fullsized":"2024-09-11T00:06:20.224150+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:25:49.672766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"83'7","reported_seq":168,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T00:06:44.432221+0000","last_change":"2024-09-11T00:04:01.823738+0000","last_active":"2024-09-11T00:06:44.432221+0000","last_peered":"2024-09-11T00:06:44.432221+0000","last_clean":"2024-09-11T00:06:44.432221+0000","last_became_active":"2024-09-11T00:04:01.823320+0000","last_became_peered":"2024-09-11T00:04:01.823320+0000","last_unstale":"2024-09-11T00:06:44.432221+0000","last_undegraded":"2024-09-11T00:06:44.432221+0000","last_fullsized":"2024-09-11T00:06:44.432221+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_clean_scrub_stamp":"2024-09-11T00:04:00.794959+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:34:30.088454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1120,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":119,"num_read_kb":3,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":30,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T00:06:20.224438+0000","last_change":"2024-09-11T00:04:11.863906+0000","last_active":"2024-09-11T00:06:20.224438+0000","last_peered":"2024-09-11T00:06:20.224438+0000","last_clean":"2024-09-11T00:06:20.224438+0000","last_became_active":"2024-09-11T00:04:11.863571+0000","last_became_peered":"2024-09-11T00:04:11.863571+0000","last_unstale":"2024-09-11T00:06:20.224438+0000","last_undegraded":"2024-09-11T00:06:20.224438+0000","last_fullsized":"2024-09-11T00:06:20.224438+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_clean_scrub_stamp":"2024-09-11T00:04:10.845136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:04:40.288886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"83'8","reported_seq":45,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T00:06:20.217947+0000","last_change":"2024-09-11T00:04:03.826623+0000","last_active":"2024-09-11T00:06:20.217947+0000","last_peered":"2024-09-11T00:06:20.217947+0000","last_clean":"2024-09-11T00:06:20.217947+0000","last_became_active":"2024-09-11T00:04:03.826202+0000","last_became_peered":"2024-09-11T00:04:03.826202+0000","last_unstale":"2024-09-11T00:06:20.217947+0000","last_undegraded":"2024-09-11T00:06:20.217947+0000","last_fullsized":"2024-09-11T00:06:20.217947+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:23:13.222654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"85'6","reported_seq":164,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T00:06:43.803171+0000","last_change":"2024-09-11T00:04:01.942857+0000","last_active":"2024-09-11T00:06:43.803171+0000","last_peered":"2024-09-11T00:06:43.803171+0000","last_clean":"2024-09-11T00:06:43.803171+0000","last_became_active":"2024-09-11T00:04:01.942494+0000","last_became_peered":"2024-09-11T00:04:01.942494+0000","last_unstale":"2024-09-11T00:06:43.803171+0000","last_undegraded":"2024-09-11T00:06:43.803171+0000","last_fullsized":"2024-09-11T00:06:43.803171+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_clean_scrub_stamp":"2024-09-11T00:04:00.794959+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:40:38.587178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":116,"num_read_kb":116,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":30,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T00:06:20.224554+0000","last_change":"2024-09-11T00:04:12.077771+0000","last_active":"2024-09-11T00:06:20.224554+0000","last_peered":"2024-09-11T00:06:20.224554+0000","last_clean":"2024-09-11T00:06:20.224554+0000","last_became_active":"2024-09-11T00:04:12.077366+0000","last_became_peered":"2024-09-11T00:04:12.077366+0000","last_unstale":"2024-09-11T00:06:20.224554+0000","last_undegraded":"2024-09-11T00:06:20.224554+0000","last_fullsized":"2024-09-11T00:06:20.224554+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_clean_scrub_stamp":"2024-09-11T00:04:10.845136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:04:26.808201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"83'5","reported_seq":42,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T00:06:20.223536+0000","last_change":"2024-09-11T00:04:03.827872+0000","last_active":"2024-09-11T00:06:20.223536+0000","last_peered":"2024-09-11T00:06:20.223536+0000","last_clean":"2024-09-11T00:06:20.223536+0000","last_became_active":"2024-09-11T00:04:03.827577+0000","last_became_peered":"2024-09-11T00:04:03.827577+0000","last_unstale":"2024-09-11T00:06:20.223536+0000","last_undegraded":"2024-09-11T00:06:20.223536+0000","last_fullsized":"2024-09-11T00:06:20.223536+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:37:19.205218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"81'4","reported_seq":57,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T00:06:20.214717+0000","last_change":"2024-09-11T00:05:19.991372+0000","last_active":"2024-09-11T00:06:20.214717+0000","last_peered":"2024-09-11T00:06:20.214717+0000","last_clean":"2024-09-11T00:06:20.214717+0000","last_became_active":"2024-09-11T00:05:19.985328+0000","last_became_peered":"2024-09-11T00:05:19.985328+0000","last_unstale":"2024-09-11T00:06:20.214717+0000","last_undegraded":"2024-09-11T00:06:20.214717+0000","last_fullsized":"2024-09-11T00:06:20.214717+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_clean_scrub_stamp":"2024-09-11T00:04:00.794959+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:17:20.345615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,4],"acting":[7,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T00:06:20.226394+0000","last_change":"2024-09-11T00:06:20.225938+0000","last_active":"2024-09-11T00:06:20.226394+0000","last_peered":"2024-09-11T00:06:20.226394+0000","last_clean":"2024-09-11T00:06:20.226394+0000","last_became_active":"2024-09-11T00:06:20.225447+0000","last_became_peered":"2024-09-11T00:06:20.225447+0000","last_unstale":"2024-09-11T00:06:20.226394+0000","last_undegraded":"2024-09-11T00:06:20.226394+0000","last_fullsized":"2024-09-11T00:06:20.226394+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_clean_scrub_stamp":"2024-09-11T00:04:10.845136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:30:17.498206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,6,10],"acting":[8,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.2","version":"83'8","reported_seq":45,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T00:06:20.223732+0000","last_change":"2024-09-11T00:04:03.839615+0000","last_active":"2024-09-11T00:06:20.223732+0000","last_peered":"2024-09-11T00:06:20.223732+0000","last_clean":"2024-09-11T00:06:20.223732+0000","last_became_active":"2024-09-11T00:04:03.832175+0000","last_became_peered":"2024-09-11T00:04:03.832175+0000","last_unstale":"2024-09-11T00:06:20.223732+0000","last_undegraded":"2024-09-11T00:06:20.223732+0000","last_fullsized":"2024-09-11T00:06:20.223732+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:17:09.975345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"81'6","reported_seq":51,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T00:06:20.210941+0000","last_change":"2024-09-11T00:04:01.817248+0000","last_active":"2024-09-11T00:06:20.210941+0000","last_peered":"2024-09-11T00:06:20.210941+0000","last_clean":"2024-09-11T00:06:20.210941+0000","last_became_active":"2024-09-11T00:04:01.816896+0000","last_became_peered":"2024-09-11T00:04:01.816896+0000","last_unstale":"2024-09-11T00:06:20.210941+0000","last_undegraded":"2024-09-11T00:06:20.210941+0000","last_fullsized":"2024-09-11T00:06:20.210941+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_clean_scrub_stamp":"2024-09-11T00:04:00.794959+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:46:51.641881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'92","reported_seq":114,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T00:06:20.214947+0000","last_change":"2024-09-11T00:02:47.245193+0000","last_active":"2024-09-11T00:06:20.214947+0000","last_peered":"2024-09-11T00:06:20.214947+0000","last_clean":"2024-09-11T00:06:20.214947+0000","last_became_active":"2024-09-11T00:02:47.239773+0000","last_became_peered":"2024-09-11T00:02:47.239773+0000","last_unstale":"2024-09-11T00:06:20.214947+0000","last_undegraded":"2024-09-11T00:06:20.214947+0000","last_fullsized":"2024-09-11T00:06:20.214947+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:59:10.637655+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:59:10.637655+0000","last_clean_scrub_stamp":"2024-09-10T23:59:10.637655+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:51:52.914831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"81'1","reported_seq":43,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T00:06:20.230099+0000","last_change":"2024-09-11T00:06:20.229604+0000","last_active":"2024-09-11T00:06:20.230099+0000","last_peered":"2024-09-11T00:06:20.230099+0000","last_clean":"2024-09-11T00:06:20.230099+0000","last_became_active":"2024-09-11T00:06:20.225065+0000","last_became_peered":"2024-09-11T00:06:20.225065+0000","last_unstale":"2024-09-11T00:06:20.230099+0000","last_undegraded":"2024-09-11T00:06:20.230099+0000","last_fullsized":"2024-09-11T00:06:20.230099+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_clean_scrub_stamp":"2024-09-11T00:04:10.845136+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:52:41.717530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,10],"acting":[7,5,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"83'7","reported_seq":44,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T00:06:20.215116+0000","last_change":"2024-09-11T00:04:03.834755+0000","last_active":"2024-09-11T00:06:20.215116+0000","last_peered":"2024-09-11T00:06:20.215116+0000","last_clean":"2024-09-11T00:06:20.215116+0000","last_became_active":"2024-09-11T00:04:03.833955+0000","last_became_peered":"2024-09-11T00:04:03.833955+0000","last_unstale":"2024-09-11T00:06:20.215116+0000","last_undegraded":"2024-09-11T00:06:20.215116+0000","last_fullsized":"2024-09-11T00:06:20.215116+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:47:13.863481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"83'7","reported_seq":168,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T00:06:43.431261+0000","last_change":"2024-09-11T00:04:02.589042+0000","last_active":"2024-09-11T00:06:43.431261+0000","last_peered":"2024-09-11T00:06:43.431261+0000","last_clean":"2024-09-11T00:06:43.431261+0000","last_became_active":"2024-09-11T00:04:02.588611+0000","last_became_peered":"2024-09-11T00:04:02.588611+0000","last_unstale":"2024-09-11T00:06:43.431261+0000","last_undegraded":"2024-09-11T00:06:43.431261+0000","last_fullsized":"2024-09-11T00:06:43.431261+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_clean_scrub_stamp":"2024-09-11T00:04:00.794959+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:41:08.257605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1702,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":119,"num_read_kb":4,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":30,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T00:06:20.207731+0000","last_change":"2024-09-11T00:04:11.860715+0000","last_active":"2024-09-11T00:06:20.207731+0000","last_peered":"2024-09-11T00:06:20.207731+0000","last_clean":"2024-09-11T00:06:20.207731+0000","last_became_active":"2024-09-11T00:04:11.860094+0000","last_became_peered":"2024-09-11T00:04:11.860094+0000","last_unstale":"2024-09-11T00:06:20.207731+0000","last_undegraded":"2024-09-11T00:06:20.207731+0000","last_fullsized":"2024-09-11T00:06:20.207731+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_clean_scrub_stamp":"2024-09-11T00:04:10.845136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:25:55.279079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"83'6","reported_seq":43,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T00:06:20.214828+0000","last_change":"2024-09-11T00:04:03.822665+0000","last_active":"2024-09-11T00:06:20.214828+0000","last_peered":"2024-09-11T00:06:20.214828+0000","last_clean":"2024-09-11T00:06:20.214828+0000","last_became_active":"2024-09-11T00:04:03.822333+0000","last_became_peered":"2024-09-11T00:04:03.822333+0000","last_unstale":"2024-09-11T00:06:20.214828+0000","last_undegraded":"2024-09-11T00:06:20.214828+0000","last_fullsized":"2024-09-11T00:06:20.214828+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:02:07.465335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"81'3","reported_seq":46,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T00:06:20.218325+0000","last_change":"2024-09-11T00:04:01.817244+0000","last_active":"2024-09-11T00:06:20.218325+0000","last_peered":"2024-09-11T00:06:20.218325+0000","last_clean":"2024-09-11T00:06:20.218325+0000","last_became_active":"2024-09-11T00:04:01.816846+0000","last_became_peered":"2024-09-11T00:04:01.816846+0000","last_unstale":"2024-09-11T00:06:20.218325+0000","last_undegraded":"2024-09-11T00:06:20.218325+0000","last_fullsized":"2024-09-11T00:06:20.218325+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_clean_scrub_stamp":"2024-09-11T00:04:00.794959+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:09:46.232332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":38,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T00:06:20.217751+0000","last_change":"2024-09-11T00:06:20.217001+0000","last_active":"2024-09-11T00:06:20.217751+0000","last_peered":"2024-09-11T00:06:20.217751+0000","last_clean":"2024-09-11T00:06:20.217751+0000","last_became_active":"2024-09-11T00:06:20.216599+0000","last_became_peered":"2024-09-11T00:06:20.216599+0000","last_unstale":"2024-09-11T00:06:20.217751+0000","last_undegraded":"2024-09-11T00:06:20.217751+0000","last_fullsized":"2024-09-11T00:06:20.217751+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_clean_scrub_stamp":"2024-09-11T00:04:10.845136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:06:42.086069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,1,4],"acting":[0,8,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"83'9","reported_seq":46,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T00:06:20.214846+0000","last_change":"2024-09-11T00:04:03.829600+0000","last_active":"2024-09-11T00:06:20.214846+0000","last_peered":"2024-09-11T00:06:20.214846+0000","last_clean":"2024-09-11T00:06:20.214846+0000","last_became_active":"2024-09-11T00:04:03.829118+0000","last_became_peered":"2024-09-11T00:04:03.829118+0000","last_unstale":"2024-09-11T00:06:20.214846+0000","last_undegraded":"2024-09-11T00:06:20.214846+0000","last_fullsized":"2024-09-11T00:06:20.214846+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:41:07.069816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"81'5","reported_seq":60,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T00:06:20.224623+0000","last_change":"2024-09-11T00:05:19.989379+0000","last_active":"2024-09-11T00:06:20.224623+0000","last_peered":"2024-09-11T00:06:20.224623+0000","last_clean":"2024-09-11T00:06:20.224623+0000","last_became_active":"2024-09-11T00:05:19.984918+0000","last_became_peered":"2024-09-11T00:05:19.984918+0000","last_unstale":"2024-09-11T00:06:20.224623+0000","last_undegraded":"2024-09-11T00:06:20.224623+0000","last_fullsized":"2024-09-11T00:06:20.224623+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_clean_scrub_stamp":"2024-09-11T00:04:00.794959+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:27:52.063256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4],"acting":[6,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":30,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T00:06:20.217466+0000","last_change":"2024-09-11T00:04:11.863812+0000","last_active":"2024-09-11T00:06:20.217466+0000","last_peered":"2024-09-11T00:06:20.217466+0000","last_clean":"2024-09-11T00:06:20.217466+0000","last_became_active":"2024-09-11T00:04:11.863412+0000","last_became_peered":"2024-09-11T00:04:11.863412+0000","last_unstale":"2024-09-11T00:06:20.217466+0000","last_undegraded":"2024-09-11T00:06:20.217466+0000","last_fullsized":"2024-09-11T00:06:20.217466+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_clean_scrub_stamp":"2024-09-11T00:04:10.845136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:12:04.140034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"83'5","reported_seq":49,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T00:06:20.218684+0000","last_change":"2024-09-11T00:05:19.985248+0000","last_active":"2024-09-11T00:06:20.218684+0000","last_peered":"2024-09-11T00:06:20.218684+0000","last_clean":"2024-09-11T00:06:20.218684+0000","last_became_active":"2024-09-11T00:05:19.984921+0000","last_became_peered":"2024-09-11T00:05:19.984921+0000","last_unstale":"2024-09-11T00:06:20.218684+0000","last_undegraded":"2024-09-11T00:06:20.218684+0000","last_fullsized":"2024-09-11T00:06:20.218684+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:16:44.370849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,4],"acting":[11,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"81'2","reported_seq":45,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T00:06:20.215957+0000","last_change":"2024-09-11T00:04:01.814191+0000","last_active":"2024-09-11T00:06:20.215957+0000","last_peered":"2024-09-11T00:06:20.215957+0000","last_clean":"2024-09-11T00:06:20.215957+0000","last_became_active":"2024-09-11T00:04:01.813778+0000","last_became_peered":"2024-09-11T00:04:01.813778+0000","last_unstale":"2024-09-11T00:06:20.215957+0000","last_undegraded":"2024-09-11T00:06:20.215957+0000","last_fullsized":"2024-09-11T00:06:20.215957+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_clean_scrub_stamp":"2024-09-11T00:04:00.794959+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:01:49.044436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":30,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T00:06:20.217607+0000","last_change":"2024-09-11T00:04:11.870281+0000","last_active":"2024-09-11T00:06:20.217607+0000","last_peered":"2024-09-11T00:06:20.217607+0000","last_clean":"2024-09-11T00:06:20.217607+0000","last_became_active":"2024-09-11T00:04:11.869738+0000","last_became_peered":"2024-09-11T00:04:11.869738+0000","last_unstale":"2024-09-11T00:06:20.217607+0000","last_undegraded":"2024-09-11T00:06:20.217607+0000","last_fullsized":"2024-09-11T00:06:20.217607+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_clean_scrub_stamp":"2024-09-11T00:04:10.845136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:04:08.380199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"83'6","reported_seq":43,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T00:06:20.217988+0000","last_change":"2024-09-11T00:04:03.835240+0000","last_active":"2024-09-11T00:06:20.217988+0000","last_peered":"2024-09-11T00:06:20.217988+0000","last_clean":"2024-09-11T00:06:20.217988+0000","last_became_active":"2024-09-11T00:04:03.834910+0000","last_became_peered":"2024-09-11T00:04:03.834910+0000","last_unstale":"2024-09-11T00:06:20.217988+0000","last_undegraded":"2024-09-11T00:06:20.217988+0000","last_fullsized":"2024-09-11T00:06:20.217988+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:30:33.996409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.10","version":"81'17","reported_seq":34,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T00:06:20.218188+0000","last_change":"2024-09-11T00:05:19.856704+0000","last_active":"2024-09-11T00:06:20.218188+0000","last_peered":"2024-09-11T00:06:20.218188+0000","last_clean":"2024-09-11T00:06:20.218188+0000","last_became_active":"2024-09-11T00:05:19.843487+0000","last_became_peered":"2024-09-11T00:05:19.843487+0000","last_unstale":"2024-09-11T00:06:20.218188+0000","last_undegraded":"2024-09-11T00:06:20.218188+0000","last_fullsized":"2024-09-11T00:06:20.218188+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_clean_scrub_stamp":"2024-09-11T00:04:00.794959+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:03:51.375839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":30,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T00:06:20.224599+0000","last_change":"2024-09-11T00:04:11.874306+0000","last_active":"2024-09-11T00:06:20.224599+0000","last_peered":"2024-09-11T00:06:20.224599+0000","last_clean":"2024-09-11T00:06:20.224599+0000","last_became_active":"2024-09-11T00:04:11.873934+0000","last_became_peered":"2024-09-11T00:04:11.873934+0000","last_unstale":"2024-09-11T00:06:20.224599+0000","last_undegraded":"2024-09-11T00:06:20.224599+0000","last_fullsized":"2024-09-11T00:06:20.224599+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_clean_scrub_stamp":"2024-09-11T00:04:10.845136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:19:22.584367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"83'6","reported_seq":43,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T00:06:20.214810+0000","last_change":"2024-09-11T00:04:03.824219+0000","last_active":"2024-09-11T00:06:20.214810+0000","last_peered":"2024-09-11T00:06:20.214810+0000","last_clean":"2024-09-11T00:06:20.214810+0000","last_became_active":"2024-09-11T00:04:03.823877+0000","last_became_peered":"2024-09-11T00:04:03.823877+0000","last_unstale":"2024-09-11T00:06:20.214810+0000","last_undegraded":"2024-09-11T00:06:20.214810+0000","last_fullsized":"2024-09-11T00:06:20.214810+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:04:05.924113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"85'6","reported_seq":166,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T00:06:44.432477+0000","last_change":"2024-09-11T00:04:01.818055+0000","last_active":"2024-09-11T00:06:44.432477+0000","last_peered":"2024-09-11T00:06:44.432477+0000","last_clean":"2024-09-11T00:06:44.432477+0000","last_became_active":"2024-09-11T00:04:01.817483+0000","last_became_peered":"2024-09-11T00:04:01.817483+0000","last_unstale":"2024-09-11T00:06:44.432477+0000","last_undegraded":"2024-09-11T00:06:44.432477+0000","last_fullsized":"2024-09-11T00:06:44.432477+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_clean_scrub_stamp":"2024-09-11T00:04:00.794959+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:16:55.218673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":899,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":119,"num_read_kb":4,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":30,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T00:06:20.218592+0000","last_change":"2024-09-11T00:04:11.875918+0000","last_active":"2024-09-11T00:06:20.218592+0000","last_peered":"2024-09-11T00:06:20.218592+0000","last_clean":"2024-09-11T00:06:20.218592+0000","last_became_active":"2024-09-11T00:04:11.875549+0000","last_became_peered":"2024-09-11T00:04:11.875549+0000","last_unstale":"2024-09-11T00:06:20.218592+0000","last_undegraded":"2024-09-11T00:06:20.218592+0000","last_fullsized":"2024-09-11T00:06:20.218592+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_clean_scrub_stamp":"2024-09-11T00:04:10.845136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:16:48.685899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.13","version":"83'15","reported_seq":52,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T00:06:20.215033+0000","last_change":"2024-09-11T00:04:03.827671+0000","last_active":"2024-09-11T00:06:20.215033+0000","last_peered":"2024-09-11T00:06:20.215033+0000","last_clean":"2024-09-11T00:06:20.215033+0000","last_became_active":"2024-09-11T00:04:03.827341+0000","last_became_peered":"2024-09-11T00:04:03.827341+0000","last_unstale":"2024-09-11T00:06:20.215033+0000","last_undegraded":"2024-09-11T00:06:20.215033+0000","last_fullsized":"2024-09-11T00:06:20.215033+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:56:16.614035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"81'5","reported_seq":48,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T00:06:20.218496+0000","last_change":"2024-09-11T00:04:01.820632+0000","last_active":"2024-09-11T00:06:20.218496+0000","last_peered":"2024-09-11T00:06:20.218496+0000","last_clean":"2024-09-11T00:06:20.218496+0000","last_became_active":"2024-09-11T00:04:01.820273+0000","last_became_peered":"2024-09-11T00:04:01.820273+0000","last_unstale":"2024-09-11T00:06:20.218496+0000","last_undegraded":"2024-09-11T00:06:20.218496+0000","last_fullsized":"2024-09-11T00:06:20.218496+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_clean_scrub_stamp":"2024-09-11T00:04:00.794959+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:34:53.444515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":30,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T00:06:20.218420+0000","last_change":"2024-09-11T00:04:11.881197+0000","last_active":"2024-09-11T00:06:20.218420+0000","last_peered":"2024-09-11T00:06:20.218420+0000","last_clean":"2024-09-11T00:06:20.218420+0000","last_became_active":"2024-09-11T00:04:11.880763+0000","last_became_peered":"2024-09-11T00:04:11.880763+0000","last_unstale":"2024-09-11T00:06:20.218420+0000","last_undegraded":"2024-09-11T00:06:20.218420+0000","last_fullsized":"2024-09-11T00:06:20.218420+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_clean_scrub_stamp":"2024-09-11T00:04:10.845136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:14:01.616095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"83'9","reported_seq":46,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T00:06:20.224080+0000","last_change":"2024-09-11T00:04:03.833470+0000","last_active":"2024-09-11T00:06:20.224080+0000","last_peered":"2024-09-11T00:06:20.224080+0000","last_clean":"2024-09-11T00:06:20.224080+0000","last_became_active":"2024-09-11T00:04:03.833120+0000","last_became_peered":"2024-09-11T00:04:03.833120+0000","last_unstale":"2024-09-11T00:06:20.224080+0000","last_undegraded":"2024-09-11T00:06:20.224080+0000","last_fullsized":"2024-09-11T00:06:20.224080+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:29:53.210230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"81'9","reported_seq":53,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T00:06:20.208085+0000","last_change":"2024-09-11T00:04:01.818081+0000","last_active":"2024-09-11T00:06:20.208085+0000","last_peered":"2024-09-11T00:06:20.208085+0000","last_clean":"2024-09-11T00:06:20.208085+0000","last_became_active":"2024-09-11T00:04:01.817716+0000","last_became_peered":"2024-09-11T00:04:01.817716+0000","last_unstale":"2024-09-11T00:06:20.208085+0000","last_undegraded":"2024-09-11T00:06:20.208085+0000","last_fullsized":"2024-09-11T00:06:20.208085+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_clean_scrub_stamp":"2024-09-11T00:04:00.794959+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:01:32.736319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":30,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T00:06:20.218197+0000","last_change":"2024-09-11T00:04:11.870182+0000","last_active":"2024-09-11T00:06:20.218197+0000","last_peered":"2024-09-11T00:06:20.218197+0000","last_clean":"2024-09-11T00:06:20.218197+0000","last_became_active":"2024-09-11T00:04:11.869744+0000","last_became_peered":"2024-09-11T00:04:11.869744+0000","last_unstale":"2024-09-11T00:06:20.218197+0000","last_undegraded":"2024-09-11T00:06:20.218197+0000","last_fullsized":"2024-09-11T00:06:20.218197+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_clean_scrub_stamp":"2024-09-11T00:04:10.845136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:50:24.070404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"83'6","reported_seq":43,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T00:06:20.215695+0000","last_change":"2024-09-11T00:04:03.828246+0000","last_active":"2024-09-11T00:06:20.215695+0000","last_peered":"2024-09-11T00:06:20.215695+0000","last_clean":"2024-09-11T00:06:20.215695+0000","last_became_active":"2024-09-11T00:04:03.827217+0000","last_became_peered":"2024-09-11T00:04:03.827217+0000","last_unstale":"2024-09-11T00:06:20.215695+0000","last_undegraded":"2024-09-11T00:06:20.215695+0000","last_fullsized":"2024-09-11T00:06:20.215695+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:51:46.018759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"85'41","reported_seq":327,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T00:06:45.434193+0000","last_change":"2024-09-11T00:04:01.819553+0000","last_active":"2024-09-11T00:06:45.434193+0000","last_peered":"2024-09-11T00:06:45.434193+0000","last_clean":"2024-09-11T00:06:45.434193+0000","last_became_active":"2024-09-11T00:04:01.819213+0000","last_became_peered":"2024-09-11T00:04:01.819213+0000","last_unstale":"2024-09-11T00:06:45.434193+0000","last_undegraded":"2024-09-11T00:06:45.434193+0000","last_fullsized":"2024-09-11T00:06:45.434193+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_clean_scrub_stamp":"2024-09-11T00:04:00.794959+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:44:11.039982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":65178,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":244,"num_read_kb":186,"num_write":43,"num_write_kb":84,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":30,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T00:06:20.217633+0000","last_change":"2024-09-11T00:04:11.871730+0000","last_active":"2024-09-11T00:06:20.217633+0000","last_peered":"2024-09-11T00:06:20.217633+0000","last_clean":"2024-09-11T00:06:20.217633+0000","last_became_active":"2024-09-11T00:04:11.870659+0000","last_became_peered":"2024-09-11T00:04:11.870659+0000","last_unstale":"2024-09-11T00:06:20.217633+0000","last_undegraded":"2024-09-11T00:06:20.217633+0000","last_fullsized":"2024-09-11T00:06:20.217633+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_clean_scrub_stamp":"2024-09-11T00:04:10.845136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:48:41.835602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.14","version":"83'11","reported_seq":48,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T00:06:20.208703+0000","last_change":"2024-09-11T00:04:03.838763+0000","last_active":"2024-09-11T00:06:20.208703+0000","last_peered":"2024-09-11T00:06:20.208703+0000","last_clean":"2024-09-11T00:06:20.208703+0000","last_became_active":"2024-09-11T00:04:03.838409+0000","last_became_peered":"2024-09-11T00:04:03.838409+0000","last_unstale":"2024-09-11T00:06:20.208703+0000","last_undegraded":"2024-09-11T00:06:20.208703+0000","last_fullsized":"2024-09-11T00:06:20.208703+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:57:38.647351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"81'5","reported_seq":48,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T00:06:20.218329+0000","last_change":"2024-09-11T00:04:01.943027+0000","last_active":"2024-09-11T00:06:20.218329+0000","last_peered":"2024-09-11T00:06:20.218329+0000","last_clean":"2024-09-11T00:06:20.218329+0000","last_became_active":"2024-09-11T00:04:01.942558+0000","last_became_peered":"2024-09-11T00:04:01.942558+0000","last_unstale":"2024-09-11T00:06:20.218329+0000","last_undegraded":"2024-09-11T00:06:20.218329+0000","last_fullsized":"2024-09-11T00:06:20.218329+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_clean_scrub_stamp":"2024-09-11T00:04:00.794959+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:48:03.776476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":38,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T00:06:20.217115+0000","last_change":"2024-09-11T00:06:20.216571+0000","last_active":"2024-09-11T00:06:20.217115+0000","last_peered":"2024-09-11T00:06:20.217115+0000","last_clean":"2024-09-11T00:06:20.217115+0000","last_became_active":"2024-09-11T00:06:20.216166+0000","last_became_peered":"2024-09-11T00:06:20.216166+0000","last_unstale":"2024-09-11T00:06:20.217115+0000","last_undegraded":"2024-09-11T00:06:20.217115+0000","last_fullsized":"2024-09-11T00:06:20.217115+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_clean_scrub_stamp":"2024-09-11T00:04:10.845136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:58:59.803225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,8],"acting":[4,1,2,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"83'9","reported_seq":46,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T00:06:20.210061+0000","last_change":"2024-09-11T00:04:03.836735+0000","last_active":"2024-09-11T00:06:20.210061+0000","last_peered":"2024-09-11T00:06:20.210061+0000","last_clean":"2024-09-11T00:06:20.210061+0000","last_became_active":"2024-09-11T00:04:03.833299+0000","last_became_peered":"2024-09-11T00:04:03.833299+0000","last_unstale":"2024-09-11T00:06:20.210061+0000","last_undegraded":"2024-09-11T00:06:20.210061+0000","last_fullsized":"2024-09-11T00:06:20.210061+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:32:15.924058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"81'4","reported_seq":46,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T00:06:20.215069+0000","last_change":"2024-09-11T00:04:01.823777+0000","last_active":"2024-09-11T00:06:20.215069+0000","last_peered":"2024-09-11T00:06:20.215069+0000","last_clean":"2024-09-11T00:06:20.215069+0000","last_became_active":"2024-09-11T00:04:01.823368+0000","last_became_peered":"2024-09-11T00:04:01.823368+0000","last_unstale":"2024-09-11T00:06:20.215069+0000","last_undegraded":"2024-09-11T00:06:20.215069+0000","last_fullsized":"2024-09-11T00:06:20.215069+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_clean_scrub_stamp":"2024-09-11T00:04:00.794959+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:05:12.584867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":30,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T00:06:20.217038+0000","last_change":"2024-09-11T00:04:11.868303+0000","last_active":"2024-09-11T00:06:20.217038+0000","last_peered":"2024-09-11T00:06:20.217038+0000","last_clean":"2024-09-11T00:06:20.217038+0000","last_became_active":"2024-09-11T00:04:11.867916+0000","last_became_peered":"2024-09-11T00:04:11.867916+0000","last_unstale":"2024-09-11T00:06:20.217038+0000","last_undegraded":"2024-09-11T00:06:20.217038+0000","last_fullsized":"2024-09-11T00:06:20.217038+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_clean_scrub_stamp":"2024-09-11T00:04:10.845136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:08:29.711763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.16","version":"83'10","reported_seq":47,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T00:06:20.208348+0000","last_change":"2024-09-11T00:04:03.832712+0000","last_active":"2024-09-11T00:06:20.208348+0000","last_peered":"2024-09-11T00:06:20.208348+0000","last_clean":"2024-09-11T00:06:20.208348+0000","last_became_active":"2024-09-11T00:04:03.832295+0000","last_became_peered":"2024-09-11T00:04:03.832295+0000","last_unstale":"2024-09-11T00:06:20.208348+0000","last_undegraded":"2024-09-11T00:06:20.208348+0000","last_fullsized":"2024-09-11T00:06:20.208348+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:40:12.271930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"83'5","reported_seq":279,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T00:06:45.433903+0000","last_change":"2024-09-11T00:04:01.820891+0000","last_active":"2024-09-11T00:06:45.433903+0000","last_peered":"2024-09-11T00:06:45.433903+0000","last_clean":"2024-09-11T00:06:45.433903+0000","last_became_active":"2024-09-11T00:04:01.820528+0000","last_became_peered":"2024-09-11T00:04:01.820528+0000","last_unstale":"2024-09-11T00:06:45.433903+0000","last_undegraded":"2024-09-11T00:06:45.433903+0000","last_fullsized":"2024-09-11T00:06:45.433903+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_clean_scrub_stamp":"2024-09-11T00:04:00.794959+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:23:38.196618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":232,"num_read_kb":232,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":30,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T00:06:20.223208+0000","last_change":"2024-09-11T00:04:11.878864+0000","last_active":"2024-09-11T00:06:20.223208+0000","last_peered":"2024-09-11T00:06:20.223208+0000","last_clean":"2024-09-11T00:06:20.223208+0000","last_became_active":"2024-09-11T00:04:11.876701+0000","last_became_peered":"2024-09-11T00:04:11.876701+0000","last_unstale":"2024-09-11T00:06:20.223208+0000","last_undegraded":"2024-09-11T00:06:20.223208+0000","last_fullsized":"2024-09-11T00:06:20.223208+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_clean_scrub_stamp":"2024-09-11T00:04:10.845136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:26:05.650618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":30,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T00:06:20.209741+0000","last_change":"2024-09-11T00:04:11.868140+0000","last_active":"2024-09-11T00:06:20.209741+0000","last_peered":"2024-09-11T00:06:20.209741+0000","last_clean":"2024-09-11T00:06:20.209741+0000","last_became_active":"2024-09-11T00:04:11.866459+0000","last_became_peered":"2024-09-11T00:04:11.866459+0000","last_unstale":"2024-09-11T00:06:20.209741+0000","last_undegraded":"2024-09-11T00:06:20.209741+0000","last_fullsized":"2024-09-11T00:06:20.209741+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_clean_scrub_stamp":"2024-09-11T00:04:10.845136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:53:25.106039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"83'11","reported_seq":48,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T00:06:20.217686+0000","last_change":"2024-09-11T00:04:03.829839+0000","last_active":"2024-09-11T00:06:20.217686+0000","last_peered":"2024-09-11T00:06:20.217686+0000","last_clean":"2024-09-11T00:06:20.217686+0000","last_became_active":"2024-09-11T00:04:03.829553+0000","last_became_peered":"2024-09-11T00:04:03.829553+0000","last_unstale":"2024-09-11T00:06:20.217686+0000","last_undegraded":"2024-09-11T00:06:20.217686+0000","last_fullsized":"2024-09-11T00:06:20.217686+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:45:58.561389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"81'3","reported_seq":46,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T00:06:20.218461+0000","last_change":"2024-09-11T00:04:01.820204+0000","last_active":"2024-09-11T00:06:20.218461+0000","last_peered":"2024-09-11T00:06:20.218461+0000","last_clean":"2024-09-11T00:06:20.218461+0000","last_became_active":"2024-09-11T00:04:01.819797+0000","last_became_peered":"2024-09-11T00:04:01.819797+0000","last_unstale":"2024-09-11T00:06:20.218461+0000","last_undegraded":"2024-09-11T00:06:20.218461+0000","last_fullsized":"2024-09-11T00:06:20.218461+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_clean_scrub_stamp":"2024-09-11T00:04:00.794959+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:56:10.530023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":30,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T00:06:20.218728+0000","last_change":"2024-09-11T00:04:11.879430+0000","last_active":"2024-09-11T00:06:20.218728+0000","last_peered":"2024-09-11T00:06:20.218728+0000","last_clean":"2024-09-11T00:06:20.218728+0000","last_became_active":"2024-09-11T00:04:11.879010+0000","last_became_peered":"2024-09-11T00:04:11.879010+0000","last_unstale":"2024-09-11T00:06:20.218728+0000","last_undegraded":"2024-09-11T00:06:20.218728+0000","last_fullsized":"2024-09-11T00:06:20.218728+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_clean_scrub_stamp":"2024-09-11T00:04:10.845136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:57.737125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"83'8","reported_seq":45,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T00:06:20.223218+0000","last_change":"2024-09-11T00:04:03.843918+0000","last_active":"2024-09-11T00:06:20.223218+0000","last_peered":"2024-09-11T00:06:20.223218+0000","last_clean":"2024-09-11T00:06:20.223218+0000","last_became_active":"2024-09-11T00:04:03.843591+0000","last_became_peered":"2024-09-11T00:04:03.843591+0000","last_unstale":"2024-09-11T00:06:20.223218+0000","last_undegraded":"2024-09-11T00:06:20.223218+0000","last_fullsized":"2024-09-11T00:06:20.223218+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:26:41.228802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"81'3","reported_seq":45,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T00:06:20.223687+0000","last_change":"2024-09-11T00:04:01.813173+0000","last_active":"2024-09-11T00:06:20.223687+0000","last_peered":"2024-09-11T00:06:20.223687+0000","last_clean":"2024-09-11T00:06:20.223687+0000","last_became_active":"2024-09-11T00:04:01.811799+0000","last_became_peered":"2024-09-11T00:04:01.811799+0000","last_unstale":"2024-09-11T00:06:20.223687+0000","last_undegraded":"2024-09-11T00:06:20.223687+0000","last_fullsized":"2024-09-11T00:06:20.223687+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_clean_scrub_stamp":"2024-09-11T00:04:00.794959+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:16:47.304371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"81'7","reported_seq":49,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T00:06:20.224176+0000","last_change":"2024-09-11T00:04:01.818649+0000","last_active":"2024-09-11T00:06:20.224176+0000","last_peered":"2024-09-11T00:06:20.224176+0000","last_clean":"2024-09-11T00:06:20.224176+0000","last_became_active":"2024-09-11T00:04:01.818287+0000","last_became_peered":"2024-09-11T00:04:01.818287+0000","last_unstale":"2024-09-11T00:06:20.224176+0000","last_undegraded":"2024-09-11T00:06:20.224176+0000","last_fullsized":"2024-09-11T00:06:20.224176+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_clean_scrub_stamp":"2024-09-11T00:04:00.794959+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:42:00.764648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"83'8","reported_seq":45,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T00:06:20.218123+0000","last_change":"2024-09-11T00:04:03.832829+0000","last_active":"2024-09-11T00:06:20.218123+0000","last_peered":"2024-09-11T00:06:20.218123+0000","last_clean":"2024-09-11T00:06:20.218123+0000","last_became_active":"2024-09-11T00:04:03.832514+0000","last_became_peered":"2024-09-11T00:04:03.832514+0000","last_unstale":"2024-09-11T00:06:20.218123+0000","last_undegraded":"2024-09-11T00:06:20.218123+0000","last_fullsized":"2024-09-11T00:06:20.218123+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:36.643727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":30,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T00:06:20.208261+0000","last_change":"2024-09-11T00:04:11.877596+0000","last_active":"2024-09-11T00:06:20.208261+0000","last_peered":"2024-09-11T00:06:20.208261+0000","last_clean":"2024-09-11T00:06:20.208261+0000","last_became_active":"2024-09-11T00:04:11.877113+0000","last_became_peered":"2024-09-11T00:04:11.877113+0000","last_unstale":"2024-09-11T00:06:20.208261+0000","last_undegraded":"2024-09-11T00:06:20.208261+0000","last_fullsized":"2024-09-11T00:06:20.208261+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_clean_scrub_stamp":"2024-09-11T00:04:10.845136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:47:11.759687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"81'5","reported_seq":165,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T00:06:44.432852+0000","last_change":"2024-09-11T00:04:01.821519+0000","last_active":"2024-09-11T00:06:44.432852+0000","last_peered":"2024-09-11T00:06:44.432852+0000","last_clean":"2024-09-11T00:06:44.432852+0000","last_became_active":"2024-09-11T00:04:01.821207+0000","last_became_peered":"2024-09-11T00:04:01.821207+0000","last_unstale":"2024-09-11T00:06:44.432852+0000","last_undegraded":"2024-09-11T00:06:44.432852+0000","last_fullsized":"2024-09-11T00:06:44.432852+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_clean_scrub_stamp":"2024-09-11T00:04:00.794959+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:16:34.435040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":118,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"83'7","reported_seq":44,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T00:06:20.215339+0000","last_change":"2024-09-11T00:04:03.832691+0000","last_active":"2024-09-11T00:06:20.215339+0000","last_peered":"2024-09-11T00:06:20.215339+0000","last_clean":"2024-09-11T00:06:20.215339+0000","last_became_active":"2024-09-11T00:04:03.832250+0000","last_became_peered":"2024-09-11T00:04:03.832250+0000","last_unstale":"2024-09-11T00:06:20.215339+0000","last_undegraded":"2024-09-11T00:06:20.215339+0000","last_fullsized":"2024-09-11T00:06:20.215339+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:54.279820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":30,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T00:06:20.207934+0000","last_change":"2024-09-11T00:04:11.864995+0000","last_active":"2024-09-11T00:06:20.207934+0000","last_peered":"2024-09-11T00:06:20.207934+0000","last_clean":"2024-09-11T00:06:20.207934+0000","last_became_active":"2024-09-11T00:04:11.864501+0000","last_became_peered":"2024-09-11T00:04:11.864501+0000","last_unstale":"2024-09-11T00:06:20.207934+0000","last_undegraded":"2024-09-11T00:06:20.207934+0000","last_fullsized":"2024-09-11T00:06:20.207934+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_clean_scrub_stamp":"2024-09-11T00:04:10.845136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:08:08.637714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"83'9","reported_seq":46,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T00:06:20.218400+0000","last_change":"2024-09-11T00:04:03.829310+0000","last_active":"2024-09-11T00:06:20.218400+0000","last_peered":"2024-09-11T00:06:20.218400+0000","last_clean":"2024-09-11T00:06:20.218400+0000","last_became_active":"2024-09-11T00:04:03.828937+0000","last_became_peered":"2024-09-11T00:04:03.828937+0000","last_unstale":"2024-09-11T00:06:20.218400+0000","last_undegraded":"2024-09-11T00:06:20.218400+0000","last_fullsized":"2024-09-11T00:06:20.218400+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:32:58.263067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"83'12","reported_seq":49,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T00:06:20.208296+0000","last_change":"2024-09-11T00:04:03.828055+0000","last_active":"2024-09-11T00:06:20.208296+0000","last_peered":"2024-09-11T00:06:20.208296+0000","last_clean":"2024-09-11T00:06:20.208296+0000","last_became_active":"2024-09-11T00:04:03.827382+0000","last_became_peered":"2024-09-11T00:04:03.827382+0000","last_unstale":"2024-09-11T00:06:20.208296+0000","last_undegraded":"2024-09-11T00:06:20.208296+0000","last_fullsized":"2024-09-11T00:06:20.208296+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:02:47.107689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"83'8","reported_seq":45,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T00:06:20.207747+0000","last_change":"2024-09-11T00:04:03.843549+0000","last_active":"2024-09-11T00:06:20.207747+0000","last_peered":"2024-09-11T00:06:20.207747+0000","last_clean":"2024-09-11T00:06:20.207747+0000","last_became_active":"2024-09-11T00:04:03.843149+0000","last_became_peered":"2024-09-11T00:04:03.843149+0000","last_unstale":"2024-09-11T00:06:20.207747+0000","last_undegraded":"2024-09-11T00:06:20.207747+0000","last_fullsized":"2024-09-11T00:06:20.207747+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:47.288319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"83'7","reported_seq":51,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T00:06:20.209743+0000","last_change":"2024-09-11T00:05:19.844682+0000","last_active":"2024-09-11T00:06:20.209743+0000","last_peered":"2024-09-11T00:06:20.209743+0000","last_clean":"2024-09-11T00:06:20.209743+0000","last_became_active":"2024-09-11T00:05:19.844269+0000","last_became_peered":"2024-09-11T00:05:19.844269+0000","last_unstale":"2024-09-11T00:06:20.209743+0000","last_undegraded":"2024-09-11T00:06:20.209743+0000","last_fullsized":"2024-09-11T00:06:20.209743+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:47:22.001597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,7],"acting":[0,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"83'4","reported_seq":41,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T00:06:20.224595+0000","last_change":"2024-09-11T00:04:03.835068+0000","last_active":"2024-09-11T00:06:20.224595+0000","last_peered":"2024-09-11T00:06:20.224595+0000","last_clean":"2024-09-11T00:06:20.224595+0000","last_became_active":"2024-09-11T00:04:03.833166+0000","last_became_peered":"2024-09-11T00:04:03.833166+0000","last_unstale":"2024-09-11T00:06:20.224595+0000","last_undegraded":"2024-09-11T00:06:20.224595+0000","last_fullsized":"2024-09-11T00:06:20.224595+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:07:06.357597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"83'6","reported_seq":43,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T00:06:20.217982+0000","last_change":"2024-09-11T00:04:03.827310+0000","last_active":"2024-09-11T00:06:20.217982+0000","last_peered":"2024-09-11T00:06:20.217982+0000","last_clean":"2024-09-11T00:06:20.217982+0000","last_became_active":"2024-09-11T00:04:03.826748+0000","last_became_peered":"2024-09-11T00:04:03.826748+0000","last_unstale":"2024-09-11T00:06:20.217982+0000","last_undegraded":"2024-09-11T00:06:20.217982+0000","last_fullsized":"2024-09-11T00:06:20.217982+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:53:15.656708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"83'15","reported_seq":52,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T00:06:20.208245+0000","last_change":"2024-09-11T00:04:03.835900+0000","last_active":"2024-09-11T00:06:20.208245+0000","last_peered":"2024-09-11T00:06:20.208245+0000","last_clean":"2024-09-11T00:06:20.208245+0000","last_became_active":"2024-09-11T00:04:03.835137+0000","last_became_peered":"2024-09-11T00:04:03.835137+0000","last_unstale":"2024-09-11T00:06:20.208245+0000","last_undegraded":"2024-09-11T00:06:20.208245+0000","last_fullsized":"2024-09-11T00:06:20.208245+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:23:31.184833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"83'2","reported_seq":39,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T00:06:20.224137+0000","last_change":"2024-09-11T00:04:03.833028+0000","last_active":"2024-09-11T00:06:20.224137+0000","last_peered":"2024-09-11T00:06:20.224137+0000","last_clean":"2024-09-11T00:06:20.224137+0000","last_became_active":"2024-09-11T00:04:03.832703+0000","last_became_peered":"2024-09-11T00:04:03.832703+0000","last_unstale":"2024-09-11T00:06:20.224137+0000","last_undegraded":"2024-09-11T00:06:20.224137+0000","last_fullsized":"2024-09-11T00:06:20.224137+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:38.115627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"83'6","reported_seq":43,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T00:06:20.217825+0000","last_change":"2024-09-11T00:04:03.832423+0000","last_active":"2024-09-11T00:06:20.217825+0000","last_peered":"2024-09-11T00:06:20.217825+0000","last_clean":"2024-09-11T00:06:20.217825+0000","last_became_active":"2024-09-11T00:04:03.832090+0000","last_became_peered":"2024-09-11T00:04:03.832090+0000","last_unstale":"2024-09-11T00:06:20.217825+0000","last_undegraded":"2024-09-11T00:06:20.217825+0000","last_fullsized":"2024-09-11T00:06:20.217825+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:01:27.258506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"83'8","reported_seq":45,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T00:06:20.223374+0000","last_change":"2024-09-11T00:04:03.842854+0000","last_active":"2024-09-11T00:06:20.223374+0000","last_peered":"2024-09-11T00:06:20.223374+0000","last_clean":"2024-09-11T00:06:20.223374+0000","last_became_active":"2024-09-11T00:04:03.838666+0000","last_became_peered":"2024-09-11T00:04:03.838666+0000","last_unstale":"2024-09-11T00:06:20.223374+0000","last_undegraded":"2024-09-11T00:06:20.223374+0000","last_fullsized":"2024-09-11T00:06:20.223374+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:45:48.575830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"83'10","reported_seq":47,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T00:06:20.207908+0000","last_change":"2024-09-11T00:04:03.834175+0000","last_active":"2024-09-11T00:06:20.207908+0000","last_peered":"2024-09-11T00:06:20.207908+0000","last_clean":"2024-09-11T00:06:20.207908+0000","last_became_active":"2024-09-11T00:04:03.833826+0000","last_became_peered":"2024-09-11T00:04:03.833826+0000","last_unstale":"2024-09-11T00:06:20.207908+0000","last_undegraded":"2024-09-11T00:06:20.207908+0000","last_fullsized":"2024-09-11T00:06:20.207908+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:45:57.323216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"83'11","reported_seq":48,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T00:06:20.224407+0000","last_change":"2024-09-11T00:04:03.836751+0000","last_active":"2024-09-11T00:06:20.224407+0000","last_peered":"2024-09-11T00:06:20.224407+0000","last_clean":"2024-09-11T00:06:20.224407+0000","last_became_active":"2024-09-11T00:04:03.834064+0000","last_became_peered":"2024-09-11T00:04:03.834064+0000","last_unstale":"2024-09-11T00:06:20.224407+0000","last_undegraded":"2024-09-11T00:06:20.224407+0000","last_fullsized":"2024-09-11T00:06:20.224407+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:06:23.059770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"83'8","reported_seq":45,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T00:06:20.218478+0000","last_change":"2024-09-11T00:04:03.830196+0000","last_active":"2024-09-11T00:06:20.218478+0000","last_peered":"2024-09-11T00:06:20.218478+0000","last_clean":"2024-09-11T00:06:20.218478+0000","last_became_active":"2024-09-11T00:04:03.829856+0000","last_became_peered":"2024-09-11T00:04:03.829856+0000","last_unstale":"2024-09-11T00:06:20.218478+0000","last_undegraded":"2024-09-11T00:06:20.218478+0000","last_fullsized":"2024-09-11T00:06:20.218478+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:34:47.489316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"83'7","reported_seq":44,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T00:06:20.208071+0000","last_change":"2024-09-11T00:04:03.835053+0000","last_active":"2024-09-11T00:06:20.208071+0000","last_peered":"2024-09-11T00:06:20.208071+0000","last_clean":"2024-09-11T00:06:20.208071+0000","last_became_active":"2024-09-11T00:04:03.834736+0000","last_became_peered":"2024-09-11T00:04:03.834736+0000","last_unstale":"2024-09-11T00:06:20.208071+0000","last_undegraded":"2024-09-11T00:06:20.208071+0000","last_fullsized":"2024-09-11T00:06:20.208071+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:53:40.253775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"83'10","reported_seq":47,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T00:06:20.218686+0000","last_change":"2024-09-11T00:04:03.834514+0000","last_active":"2024-09-11T00:06:20.218686+0000","last_peered":"2024-09-11T00:06:20.218686+0000","last_clean":"2024-09-11T00:06:20.218686+0000","last_became_active":"2024-09-11T00:04:03.834185+0000","last_became_peered":"2024-09-11T00:04:03.834185+0000","last_unstale":"2024-09-11T00:06:20.218686+0000","last_undegraded":"2024-09-11T00:06:20.218686+0000","last_fullsized":"2024-09-11T00:06:20.218686+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:33:52.949710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"83'8","reported_seq":45,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T00:06:20.219019+0000","last_change":"2024-09-11T00:04:03.840371+0000","last_active":"2024-09-11T00:06:20.219019+0000","last_peered":"2024-09-11T00:06:20.219019+0000","last_clean":"2024-09-11T00:06:20.219019+0000","last_became_active":"2024-09-11T00:04:03.839933+0000","last_became_peered":"2024-09-11T00:04:03.839933+0000","last_unstale":"2024-09-11T00:06:20.219019+0000","last_undegraded":"2024-09-11T00:06:20.219019+0000","last_fullsized":"2024-09-11T00:06:20.219019+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:24:53.890837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"83'9","reported_seq":46,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T00:06:20.207585+0000","last_change":"2024-09-11T00:04:03.838386+0000","last_active":"2024-09-11T00:06:20.207585+0000","last_peered":"2024-09-11T00:06:20.207585+0000","last_clean":"2024-09-11T00:06:20.207585+0000","last_became_active":"2024-09-11T00:04:03.837511+0000","last_became_peered":"2024-09-11T00:04:03.837511+0000","last_unstale":"2024-09-11T00:06:20.207585+0000","last_undegraded":"2024-09-11T00:06:20.207585+0000","last_fullsized":"2024-09-11T00:06:20.207585+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:27.399727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"83'7","reported_seq":51,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T00:06:20.209348+0000","last_change":"2024-09-11T00:05:19.848001+0000","last_active":"2024-09-11T00:06:20.209348+0000","last_peered":"2024-09-11T00:06:20.209348+0000","last_clean":"2024-09-11T00:06:20.209348+0000","last_became_active":"2024-09-11T00:05:19.847175+0000","last_became_peered":"2024-09-11T00:05:19.847175+0000","last_unstale":"2024-09-11T00:06:20.209348+0000","last_undegraded":"2024-09-11T00:06:20.209348+0000","last_fullsized":"2024-09-11T00:06:20.209348+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:06:13.765740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,2],"acting":[0,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"83'7","reported_seq":44,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T00:06:20.209526+0000","last_change":"2024-09-11T00:04:03.832618+0000","last_active":"2024-09-11T00:06:20.209526+0000","last_peered":"2024-09-11T00:06:20.209526+0000","last_clean":"2024-09-11T00:06:20.209526+0000","last_became_active":"2024-09-11T00:04:03.831821+0000","last_became_peered":"2024-09-11T00:04:03.831821+0000","last_unstale":"2024-09-11T00:06:20.209526+0000","last_undegraded":"2024-09-11T00:06:20.209526+0000","last_fullsized":"2024-09-11T00:06:20.209526+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:40:40.402422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"83'10","reported_seq":47,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T00:06:20.224352+0000","last_change":"2024-09-11T00:04:03.825348+0000","last_active":"2024-09-11T00:06:20.224352+0000","last_peered":"2024-09-11T00:06:20.224352+0000","last_clean":"2024-09-11T00:06:20.224352+0000","last_became_active":"2024-09-11T00:04:03.824977+0000","last_became_peered":"2024-09-11T00:04:03.824977+0000","last_unstale":"2024-09-11T00:06:20.224352+0000","last_undegraded":"2024-09-11T00:06:20.224352+0000","last_fullsized":"2024-09-11T00:06:20.224352+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:19:53.412969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"83'7","reported_seq":23,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T00:06:20.217748+0000","last_change":"2024-09-11T00:05:19.983677+0000","last_active":"2024-09-11T00:06:20.217748+0000","last_peered":"2024-09-11T00:06:20.217748+0000","last_clean":"2024-09-11T00:06:20.217748+0000","last_became_active":"2024-09-11T00:05:19.983404+0000","last_became_peered":"2024-09-11T00:05:19.983404+0000","last_unstale":"2024-09-11T00:06:20.217748+0000","last_undegraded":"2024-09-11T00:06:20.217748+0000","last_fullsized":"2024-09-11T00:06:20.217748+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:58:11.624190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,10,8],"acting":[9,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.35","version":"83'6","reported_seq":43,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T00:06:20.214987+0000","last_change":"2024-09-11T00:04:03.840068+0000","last_active":"2024-09-11T00:06:20.214987+0000","last_peered":"2024-09-11T00:06:20.214987+0000","last_clean":"2024-09-11T00:06:20.214987+0000","last_became_active":"2024-09-11T00:04:03.835185+0000","last_became_peered":"2024-09-11T00:04:03.835185+0000","last_unstale":"2024-09-11T00:06:20.214987+0000","last_undegraded":"2024-09-11T00:06:20.214987+0000","last_fullsized":"2024-09-11T00:06:20.214987+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:19:10.699038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"83'16","reported_seq":53,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T00:06:20.224682+0000","last_change":"2024-09-11T00:04:03.828576+0000","last_active":"2024-09-11T00:06:20.224682+0000","last_peered":"2024-09-11T00:06:20.224682+0000","last_clean":"2024-09-11T00:06:20.224682+0000","last_became_active":"2024-09-11T00:04:03.828255+0000","last_became_peered":"2024-09-11T00:04:03.828255+0000","last_unstale":"2024-09-11T00:06:20.224682+0000","last_undegraded":"2024-09-11T00:06:20.224682+0000","last_fullsized":"2024-09-11T00:06:20.224682+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:29:54.164145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"83'11","reported_seq":48,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T00:06:20.208306+0000","last_change":"2024-09-11T00:04:03.841452+0000","last_active":"2024-09-11T00:06:20.208306+0000","last_peered":"2024-09-11T00:06:20.208306+0000","last_clean":"2024-09-11T00:06:20.208306+0000","last_became_active":"2024-09-11T00:04:03.840911+0000","last_became_peered":"2024-09-11T00:04:03.840911+0000","last_unstale":"2024-09-11T00:06:20.208306+0000","last_undegraded":"2024-09-11T00:06:20.208306+0000","last_fullsized":"2024-09-11T00:06:20.208306+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:56:40.121922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"83'9","reported_seq":46,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T00:06:20.223208+0000","last_change":"2024-09-11T00:04:03.842009+0000","last_active":"2024-09-11T00:06:20.223208+0000","last_peered":"2024-09-11T00:06:20.223208+0000","last_clean":"2024-09-11T00:06:20.223208+0000","last_became_active":"2024-09-11T00:04:03.841625+0000","last_became_peered":"2024-09-11T00:04:03.841625+0000","last_unstale":"2024-09-11T00:06:20.223208+0000","last_undegraded":"2024-09-11T00:06:20.223208+0000","last_fullsized":"2024-09-11T00:06:20.223208+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:15:11.855308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"83'5","reported_seq":49,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T00:06:20.214617+0000","last_change":"2024-09-11T00:05:19.845891+0000","last_active":"2024-09-11T00:06:20.214617+0000","last_peered":"2024-09-11T00:06:20.214617+0000","last_clean":"2024-09-11T00:06:20.214617+0000","last_became_active":"2024-09-11T00:05:19.845547+0000","last_became_peered":"2024-09-11T00:05:19.845547+0000","last_unstale":"2024-09-11T00:06:20.214617+0000","last_undegraded":"2024-09-11T00:06:20.214617+0000","last_fullsized":"2024-09-11T00:06:20.214617+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:02:20.178667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"83'6","reported_seq":43,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T00:06:20.224142+0000","last_change":"2024-09-11T00:04:03.834540+0000","last_active":"2024-09-11T00:06:20.224142+0000","last_peered":"2024-09-11T00:06:20.224142+0000","last_clean":"2024-09-11T00:06:20.224142+0000","last_became_active":"2024-09-11T00:04:03.834230+0000","last_became_peered":"2024-09-11T00:04:03.834230+0000","last_unstale":"2024-09-11T00:06:20.224142+0000","last_undegraded":"2024-09-11T00:06:20.224142+0000","last_fullsized":"2024-09-11T00:06:20.224142+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:14:26.125003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"83'4","reported_seq":41,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T00:06:20.215459+0000","last_change":"2024-09-11T00:04:03.836177+0000","last_active":"2024-09-11T00:06:20.215459+0000","last_peered":"2024-09-11T00:06:20.215459+0000","last_clean":"2024-09-11T00:06:20.215459+0000","last_became_active":"2024-09-11T00:04:03.835848+0000","last_became_peered":"2024-09-11T00:04:03.835848+0000","last_unstale":"2024-09-11T00:06:20.215459+0000","last_undegraded":"2024-09-11T00:06:20.215459+0000","last_fullsized":"2024-09-11T00:06:20.215459+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:57:53.854738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"83'8","reported_seq":45,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T00:06:20.210324+0000","last_change":"2024-09-11T00:04:03.837614+0000","last_active":"2024-09-11T00:06:20.210324+0000","last_peered":"2024-09-11T00:06:20.210324+0000","last_clean":"2024-09-11T00:06:20.210324+0000","last_became_active":"2024-09-11T00:04:03.837164+0000","last_became_peered":"2024-09-11T00:04:03.837164+0000","last_unstale":"2024-09-11T00:06:20.210324+0000","last_undegraded":"2024-09-11T00:06:20.210324+0000","last_fullsized":"2024-09-11T00:06:20.210324+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:10:38.467648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3d","version":"83'7","reported_seq":44,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T00:06:20.209452+0000","last_change":"2024-09-11T00:04:03.833999+0000","last_active":"2024-09-11T00:06:20.209452+0000","last_peered":"2024-09-11T00:06:20.209452+0000","last_clean":"2024-09-11T00:06:20.209452+0000","last_became_active":"2024-09-11T00:04:03.833580+0000","last_became_peered":"2024-09-11T00:04:03.833580+0000","last_unstale":"2024-09-11T00:06:20.209452+0000","last_undegraded":"2024-09-11T00:06:20.209452+0000","last_fullsized":"2024-09-11T00:06:20.209452+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:47:40.902423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"83'4","reported_seq":41,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T00:06:20.209745+0000","last_change":"2024-09-11T00:04:03.832478+0000","last_active":"2024-09-11T00:06:20.209745+0000","last_peered":"2024-09-11T00:06:20.209745+0000","last_clean":"2024-09-11T00:06:20.209745+0000","last_became_active":"2024-09-11T00:04:03.831811+0000","last_became_peered":"2024-09-11T00:04:03.831811+0000","last_unstale":"2024-09-11T00:06:20.209745+0000","last_undegraded":"2024-09-11T00:06:20.209745+0000","last_fullsized":"2024-09-11T00:06:20.209745+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:53:05.839006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"83'8","reported_seq":45,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T00:06:20.219013+0000","last_change":"2024-09-11T00:04:03.837077+0000","last_active":"2024-09-11T00:06:20.219013+0000","last_peered":"2024-09-11T00:06:20.219013+0000","last_clean":"2024-09-11T00:06:20.219013+0000","last_became_active":"2024-09-11T00:04:03.835718+0000","last_became_peered":"2024-09-11T00:04:03.835718+0000","last_unstale":"2024-09-11T00:06:20.219013+0000","last_undegraded":"2024-09-11T00:06:20.219013+0000","last_fullsized":"2024-09-11T00:06:20.219013+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:17:00.633257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":72975,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":1201,"num_read_kb":681,"num_write":262,"num_write_kb":155,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":577536,"data_stored":218925,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":227,"ondisk_log_size":227,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":68,"seq":292057776175,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36320,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708832,"statfs":{"total":95995035648,"available":95957843968,"internally_reserved":0,"allocated":991232,"data_stored":487430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":36174276},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":63,"seq":270582939700,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28668,"kb_used_data":1644,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716484,"statfs":{"total":95995035648,"available":95965679616,"internally_reserved":0,"allocated":1683456,"data_stored":1143187,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":58,"seq":249108103223,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36336,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708816,"statfs":{"total":95995035648,"available":95957827584,"internally_reserved":0,"allocated":1007616,"data_stored":486716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":52,"seq":223338299454,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":40484,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":93704668,"statfs":{"total":95995035648,"available":95953580032,"internally_reserved":0,"allocated":995328,"data_stored":486982,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":47,"seq":201863462978,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28596,"kb_used_data":1564,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716556,"statfs":{"total":95995035648,"available":95965753344,"internally_reserved":0,"allocated":1601536,"data_stored":1084043,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":42,"seq":180388626505,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32152,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713000,"statfs":{"total":95995035648,"available":95962112000,"internally_reserved":0,"allocated":983040,"data_stored":482752,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":36,"seq":154618822734,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28604,"kb_used_data":1572,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716548,"statfs":{"total":95995035648,"available":95965745152,"internally_reserved":0,"allocated":1609728,"data_stored":1078782,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986259,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27992,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717160,"statfs":{"total":95995035648,"available":95966371840,"internally_reserved":0,"allocated":991232,"data_stored":486605,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182488,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32164,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712988,"statfs":{"total":95995035648,"available":95962099712,"internally_reserved":0,"allocated":995328,"data_stored":481928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31914440},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411423,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28076,"kb_used_data":1048,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717076,"statfs":{"total":95995035648,"available":95966285824,"internally_reserved":0,"allocated":1073152,"data_stored":548869,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574947,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36376,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708776,"statfs":{"total":95995035648,"available":95957786624,"internally_reserved":0,"allocated":1052672,"data_stored":548649,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738473,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32172,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712980,"statfs":{"total":95995035648,"available":95962091520,"internally_reserved":0,"allocated":1007616,"data_stored":484208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2710,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":67462,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":67682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1011,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":2820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3985,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1388,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":118784,"data_stored":67225,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1836,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T00:06:47.430 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid ff3ad74a-6fcf-11ef-bcea-c7b262605968 -- ceph pg dump --format=json 2024-09-11T00:06:47.562 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:06:47 smithi012 ceph-mon[30333]: pgmap v389: 129 pgs: 129 active+clean; 648 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 341 B/s wr, 12 op/s 2024-09-11T00:06:47.563 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:06:47 smithi012 ceph-mon[30333]: from='client.25367 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T00:06:47.580 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:06:47 smithi178 ceph-mon[35919]: pgmap v389: 129 pgs: 129 active+clean; 648 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 341 B/s wr, 12 op/s 2024-09-11T00:06:47.581 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:06:47 smithi178 ceph-mon[35919]: from='client.25367 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T00:06:47.782 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:06:47 smithi165 ceph-mon[35356]: pgmap v389: 129 pgs: 129 active+clean; 648 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 341 B/s wr, 12 op/s 2024-09-11T00:06:47.782 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:06:47 smithi165 ceph-mon[35356]: from='client.25367 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T00:06:47.810 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:06:48.413 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:48.410+0000 7f4ba9400640 1 -- 172.21.15.12:0/2796677297 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4ba4075f40 msgr2=0x7f4ba410ee10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:06:48.413 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:48.410+0000 7f4ba9400640 1 --2- 172.21.15.12:0/2796677297 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4ba4075f40 0x7f4ba410ee10 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f4b8c009900 tx=0x7f4b8c02a410 comp rx=0 tx=0).stop 2024-09-11T00:06:48.413 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:48.410+0000 7f4ba9400640 1 -- 172.21.15.12:0/2796677297 shutdown_connections 2024-09-11T00:06:48.413 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:48.410+0000 7f4ba9400640 1 --2- 172.21.15.12:0/2796677297 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4ba410f3e0 0x7f4ba41117d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:48.414 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:48.410+0000 7f4ba9400640 1 --2- 172.21.15.12:0/2796677297 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4ba4075f40 0x7f4ba410ee10 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:48.414 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:48.410+0000 7f4ba9400640 1 --2- 172.21.15.12:0/2796677297 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4ba4075540 0x7f4ba4075920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:48.414 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:48.410+0000 7f4ba9400640 1 -- 172.21.15.12:0/2796677297 >> 172.21.15.12:0/2796677297 conn(0x7f4ba40fe830 msgr2=0x7f4ba4100c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:06:48.414 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:48.410+0000 7f4ba9400640 1 -- 172.21.15.12:0/2796677297 shutdown_connections 2024-09-11T00:06:48.414 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:48.410+0000 7f4ba9400640 1 -- 172.21.15.12:0/2796677297 wait complete. 2024-09-11T00:06:48.414 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:48.411+0000 7f4ba9400640 1 Processor -- start 2024-09-11T00:06:48.414 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:48.411+0000 7f4ba9400640 1 -- start start 2024-09-11T00:06:48.416 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:48.412+0000 7f4ba9400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4ba4075540 0x7f4ba419b4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:06:48.416 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:48.412+0000 7f4ba9400640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4ba4075f40 0x7f4ba419ba30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:06:48.416 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:48.412+0000 7f4ba9400640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4ba410f3e0 0x7f4ba41a0dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:06:48.416 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:48.412+0000 7f4ba9400640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f4ba4114fe0 con 0x7f4ba410f3e0 2024-09-11T00:06:48.416 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:48.412+0000 7f4ba9400640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f4ba4114e60 con 0x7f4ba4075540 2024-09-11T00:06:48.416 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:48.412+0000 7f4ba9400640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f4ba4115160 con 0x7f4ba4075f40 2024-09-11T00:06:48.416 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:48.412+0000 7f4ba2800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4ba4075540 0x7f4ba419b4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:06:48.416 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:48.412+0000 7f4b99e00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4ba4075f40 0x7f4ba419ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:06:48.416 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:48.412+0000 7f4ba2800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4ba4075540 0x7f4ba419b4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.12:59866/0 (socket says 172.21.15.12:59866) 2024-09-11T00:06:48.416 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:48.412+0000 7f4b99e00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4ba4075f40 0x7f4ba419ba30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.12:59740/0 (socket says 172.21.15.12:59740) 2024-09-11T00:06:48.416 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:48.412+0000 7f4ba2800640 1 -- 172.21.15.12:0/1639734956 learned_addr learned my addr 172.21.15.12:0/1639734956 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:06:48.417 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:48.412+0000 7f4ba3200640 1 --2- 172.21.15.12:0/1639734956 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4ba410f3e0 0x7f4ba41a0dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:06:48.417 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:48.413+0000 7f4ba2800640 1 -- 172.21.15.12:0/1639734956 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4ba4075f40 msgr2=0x7f4ba419ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:06:48.417 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:48.413+0000 7f4ba2800640 1 --2- 172.21.15.12:0/1639734956 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4ba4075f40 0x7f4ba419ba30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:48.417 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:48.413+0000 7f4ba2800640 1 -- 172.21.15.12:0/1639734956 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4ba410f3e0 msgr2=0x7f4ba41a0dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:06:48.417 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:48.413+0000 7f4ba2800640 1 --2- 172.21.15.12:0/1639734956 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4ba410f3e0 0x7f4ba41a0dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:48.417 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:48.413+0000 7f4ba2800640 1 -- 172.21.15.12:0/1639734956 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ba41a15a0 con 0x7f4ba4075540 2024-09-11T00:06:48.417 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:48.413+0000 7f4b99e00640 1 --2- 172.21.15.12:0/1639734956 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4ba4075f40 0x7f4ba419ba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:06:48.417 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:48.413+0000 7f4ba2800640 1 --2- 172.21.15.12:0/1639734956 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4ba4075540 0x7f4ba419b4f0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f4b9400e9f0 tx=0x7f4b9400edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:06:48.417 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:48.413+0000 7f4b9be00640 1 -- 172.21.15.12:0/1639734956 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b9400cd80 con 0x7f4ba4075540 2024-09-11T00:06:48.417 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:48.413+0000 7f4ba9400640 1 -- 172.21.15.12:0/1639734956 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4ba41a1890 con 0x7f4ba4075540 2024-09-11T00:06:48.417 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:48.413+0000 7f4b9be00640 1 -- 172.21.15.12:0/1639734956 <== mon.2 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4b940054d0 con 0x7f4ba4075540 2024-09-11T00:06:48.417 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:48.413+0000 7f4b9be00640 1 -- 172.21.15.12:0/1639734956 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b94010690 con 0x7f4ba4075540 2024-09-11T00:06:48.417 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:48.414+0000 7f4ba9400640 1 -- 172.21.15.12:0/1639734956 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4ba41a7f50 con 0x7f4ba4075540 2024-09-11T00:06:48.418 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:48.415+0000 7f4ba9400640 1 -- 172.21.15.12:0/1639734956 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4ba4075920 con 0x7f4ba4075540 2024-09-11T00:06:48.418 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:48.417+0000 7f4b9be00640 1 -- 172.21.15.12:0/1639734956 <== mon.2 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7f4b94004e90 con 0x7f4ba4075540 2024-09-11T00:06:48.420 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:48.418+0000 7f4b9be00640 1 --2- 172.21.15.12:0/1639734956 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f4b880b52e0 0x7f4b880b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:06:48.420 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:48.418+0000 7f4b9be00640 1 -- 172.21.15.12:0/1639734956 <== mon.2 v2:172.21.15.165:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9208+0+0 (secure 0 0 0) 0x7f4b940e63e0 con 0x7f4ba4075540 2024-09-11T00:06:48.420 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:48.418+0000 7f4b99e00640 1 --2- 172.21.15.12:0/1639734956 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f4b880b52e0 0x7f4b880b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:06:48.421 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:48.419+0000 7f4b99e00640 1 --2- 172.21.15.12:0/1639734956 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f4b880b52e0 0x7f4b880b77a0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f4b8c005ab0 tx=0x7f4b8c005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:06:48.423 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:48.422+0000 7f4b9be00640 1 -- 172.21.15.12:0/1639734956 <== mon.2 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f4b940ace70 con 0x7f4ba4075540 2024-09-11T00:06:48.621 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:48.619+0000 7f4ba9400640 1 -- 172.21.15.12:0/1639734956 --> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f4ba4063590 con 0x7f4b880b52e0 2024-09-11T00:06:48.626 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:48.625+0000 7f4b9be00640 1 -- 172.21.15.12:0/1639734956 <== mgr.14150 v2:172.21.15.12:6800/3442939137 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340533 (secure 0 0 0) 0x7f4ba4063590 con 0x7f4b880b52e0 2024-09-11T00:06:48.631 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-11T00:06:48.634 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-11T00:06:48.636 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:48.629+0000 7f4ba9400640 1 -- 172.21.15.12:0/1639734956 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f4b880b52e0 msgr2=0x7f4b880b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:06:48.636 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:48.629+0000 7f4ba9400640 1 --2- 172.21.15.12:0/1639734956 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f4b880b52e0 0x7f4b880b77a0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f4b8c005ab0 tx=0x7f4b8c005a20 comp rx=0 tx=0).stop 2024-09-11T00:06:48.636 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:48.629+0000 7f4ba9400640 1 -- 172.21.15.12:0/1639734956 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4ba4075540 msgr2=0x7f4ba419b4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:06:48.636 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:48.629+0000 7f4ba9400640 1 --2- 172.21.15.12:0/1639734956 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4ba4075540 0x7f4ba419b4f0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f4b9400e9f0 tx=0x7f4b9400edf0 comp rx=0 tx=0).stop 2024-09-11T00:06:48.636 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:48.629+0000 7f4ba9400640 1 -- 172.21.15.12:0/1639734956 shutdown_connections 2024-09-11T00:06:48.636 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:48.629+0000 7f4ba9400640 1 --2- 172.21.15.12:0/1639734956 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f4b880b52e0 0x7f4b880b77a0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:48.636 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:48.629+0000 7f4ba9400640 1 --2- 172.21.15.12:0/1639734956 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4ba410f3e0 0x7f4ba41a0dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:48.636 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:48.629+0000 7f4ba9400640 1 --2- 172.21.15.12:0/1639734956 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4ba4075f40 0x7f4ba419ba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:48.636 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:48.629+0000 7f4ba9400640 1 --2- 172.21.15.12:0/1639734956 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4ba4075540 0x7f4ba419b4f0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:48.636 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:48.629+0000 7f4ba9400640 1 -- 172.21.15.12:0/1639734956 >> 172.21.15.12:0/1639734956 conn(0x7f4ba40fe830 msgr2=0x7f4ba40ff430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:06:48.636 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:48.630+0000 7f4ba9400640 1 -- 172.21.15.12:0/1639734956 shutdown_connections 2024-09-11T00:06:48.636 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:48.630+0000 7f4ba9400640 1 -- 172.21.15.12:0/1639734956 wait complete. 2024-09-11T00:06:49.294 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:06:49 smithi012 ceph-mon[30333]: pgmap v390: 129 pgs: 129 active+clean; 648 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 426 B/s wr, 8 op/s 2024-09-11T00:06:49.295 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:06:49 smithi012 ceph-mon[30333]: from='client.25373 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T00:06:49.434 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":390,"stamp":"2024-09-11T00:06:48.133065+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":663577,"num_objects":102,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":1277,"num_read_kb":745,"num_write":389,"num_write_kb":2308,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":24,"num_bytes_recovered":2952069,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":822,"ondisk_log_size":822,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":387924,"kb_used_data":13664,"kb_used_omap":18,"kb_used_meta":373997,"kb_avail":1124553900,"statfs":{"total":1151940427776,"available":1151543193600,"internally_reserved":0,"allocated":13991936,"data_stored":7800151,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19142,"internal_metadata":382973242},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":622,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":102,"num_read_kb":52,"num_write":6,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003322"},"pg_stats":[{"pgid":"2.1f","version":"83'14","reported_seq":171,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T00:06:42.428729+0000","last_change":"2024-09-11T00:04:01.820872+0000","last_active":"2024-09-11T00:06:42.428729+0000","last_peered":"2024-09-11T00:06:42.428729+0000","last_clean":"2024-09-11T00:06:42.428729+0000","last_became_active":"2024-09-11T00:04:01.820404+0000","last_became_peered":"2024-09-11T00:04:01.820404+0000","last_unstale":"2024-09-11T00:06:42.428729+0000","last_undegraded":"2024-09-11T00:06:42.428729+0000","last_fullsized":"2024-09-11T00:06:42.428729+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_clean_scrub_stamp":"2024-09-11T00:04:00.794959+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:15:38.648289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":115,"num_read_kb":115,"num_write":15,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"83'5","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T00:06:20.218414+0000","last_change":"2024-09-11T00:05:19.846978+0000","last_active":"2024-09-11T00:06:20.218414+0000","last_peered":"2024-09-11T00:06:20.218414+0000","last_clean":"2024-09-11T00:06:20.218414+0000","last_became_active":"2024-09-11T00:05:19.846569+0000","last_became_peered":"2024-09-11T00:05:19.846569+0000","last_unstale":"2024-09-11T00:06:20.218414+0000","last_undegraded":"2024-09-11T00:06:20.218414+0000","last_fullsized":"2024-09-11T00:06:20.218414+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:27:46.235552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,5],"acting":[9,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":30,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T00:06:20.208028+0000","last_change":"2024-09-11T00:04:11.872109+0000","last_active":"2024-09-11T00:06:20.208028+0000","last_peered":"2024-09-11T00:06:20.208028+0000","last_clean":"2024-09-11T00:06:20.208028+0000","last_became_active":"2024-09-11T00:04:11.871279+0000","last_became_peered":"2024-09-11T00:04:11.871279+0000","last_unstale":"2024-09-11T00:06:20.208028+0000","last_undegraded":"2024-09-11T00:06:20.208028+0000","last_fullsized":"2024-09-11T00:06:20.208028+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_clean_scrub_stamp":"2024-09-11T00:04:10.845136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:13:32.837555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"81'2","reported_seq":44,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T00:06:20.224031+0000","last_change":"2024-09-11T00:04:01.942288+0000","last_active":"2024-09-11T00:06:20.224031+0000","last_peered":"2024-09-11T00:06:20.224031+0000","last_clean":"2024-09-11T00:06:20.224031+0000","last_became_active":"2024-09-11T00:04:01.941838+0000","last_became_peered":"2024-09-11T00:04:01.941838+0000","last_unstale":"2024-09-11T00:06:20.224031+0000","last_undegraded":"2024-09-11T00:06:20.224031+0000","last_fullsized":"2024-09-11T00:06:20.224031+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_clean_scrub_stamp":"2024-09-11T00:04:00.794959+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:48:44.867267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"83'6","reported_seq":43,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T00:06:20.209960+0000","last_change":"2024-09-11T00:04:03.839951+0000","last_active":"2024-09-11T00:06:20.209960+0000","last_peered":"2024-09-11T00:06:20.209960+0000","last_clean":"2024-09-11T00:06:20.209960+0000","last_became_active":"2024-09-11T00:04:03.839591+0000","last_became_peered":"2024-09-11T00:04:03.839591+0000","last_unstale":"2024-09-11T00:06:20.209960+0000","last_undegraded":"2024-09-11T00:06:20.209960+0000","last_fullsized":"2024-09-11T00:06:20.209960+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:12:49.603310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":30,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T00:06:20.210052+0000","last_change":"2024-09-11T00:04:11.872026+0000","last_active":"2024-09-11T00:06:20.210052+0000","last_peered":"2024-09-11T00:06:20.210052+0000","last_clean":"2024-09-11T00:06:20.210052+0000","last_became_active":"2024-09-11T00:04:11.871699+0000","last_became_peered":"2024-09-11T00:04:11.871699+0000","last_unstale":"2024-09-11T00:06:20.210052+0000","last_undegraded":"2024-09-11T00:06:20.210052+0000","last_fullsized":"2024-09-11T00:06:20.210052+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_clean_scrub_stamp":"2024-09-11T00:04:10.845136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:07:47.757223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"81'12","reported_seq":55,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T00:06:20.215846+0000","last_change":"2024-09-11T00:04:01.819942+0000","last_active":"2024-09-11T00:06:20.215846+0000","last_peered":"2024-09-11T00:06:20.215846+0000","last_clean":"2024-09-11T00:06:20.215846+0000","last_became_active":"2024-09-11T00:04:01.819337+0000","last_became_peered":"2024-09-11T00:04:01.819337+0000","last_unstale":"2024-09-11T00:06:20.215846+0000","last_undegraded":"2024-09-11T00:06:20.215846+0000","last_fullsized":"2024-09-11T00:06:20.215846+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_clean_scrub_stamp":"2024-09-11T00:04:00.794959+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:22:26.947067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"83'9","reported_seq":46,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T00:06:20.207881+0000","last_change":"2024-09-11T00:04:03.839276+0000","last_active":"2024-09-11T00:06:20.207881+0000","last_peered":"2024-09-11T00:06:20.207881+0000","last_clean":"2024-09-11T00:06:20.207881+0000","last_became_active":"2024-09-11T00:04:03.838843+0000","last_became_peered":"2024-09-11T00:04:03.838843+0000","last_unstale":"2024-09-11T00:06:20.207881+0000","last_undegraded":"2024-09-11T00:06:20.207881+0000","last_fullsized":"2024-09-11T00:06:20.207881+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:26:42.640628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":30,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T00:06:20.210814+0000","last_change":"2024-09-11T00:04:11.872345+0000","last_active":"2024-09-11T00:06:20.210814+0000","last_peered":"2024-09-11T00:06:20.210814+0000","last_clean":"2024-09-11T00:06:20.210814+0000","last_became_active":"2024-09-11T00:04:11.871994+0000","last_became_peered":"2024-09-11T00:04:11.871994+0000","last_unstale":"2024-09-11T00:06:20.210814+0000","last_undegraded":"2024-09-11T00:06:20.210814+0000","last_fullsized":"2024-09-11T00:06:20.210814+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_clean_scrub_stamp":"2024-09-11T00:04:10.845136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:14:23.906778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"81'5","reported_seq":48,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T00:06:20.215746+0000","last_change":"2024-09-11T00:04:01.819581+0000","last_active":"2024-09-11T00:06:20.215746+0000","last_peered":"2024-09-11T00:06:20.215746+0000","last_clean":"2024-09-11T00:06:20.215746+0000","last_became_active":"2024-09-11T00:04:01.818699+0000","last_became_peered":"2024-09-11T00:04:01.818699+0000","last_unstale":"2024-09-11T00:06:20.215746+0000","last_undegraded":"2024-09-11T00:06:20.215746+0000","last_fullsized":"2024-09-11T00:06:20.215746+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_clean_scrub_stamp":"2024-09-11T00:04:00.794959+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:07:24.540825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"83'15","reported_seq":52,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T00:06:20.218131+0000","last_change":"2024-09-11T00:04:03.835107+0000","last_active":"2024-09-11T00:06:20.218131+0000","last_peered":"2024-09-11T00:06:20.218131+0000","last_clean":"2024-09-11T00:06:20.218131+0000","last_became_active":"2024-09-11T00:04:03.834782+0000","last_became_peered":"2024-09-11T00:04:03.834782+0000","last_unstale":"2024-09-11T00:06:20.218131+0000","last_undegraded":"2024-09-11T00:06:20.218131+0000","last_fullsized":"2024-09-11T00:06:20.218131+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:46:54.472270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":30,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T00:06:20.209753+0000","last_change":"2024-09-11T00:04:11.874168+0000","last_active":"2024-09-11T00:06:20.209753+0000","last_peered":"2024-09-11T00:06:20.209753+0000","last_clean":"2024-09-11T00:06:20.209753+0000","last_became_active":"2024-09-11T00:04:11.873810+0000","last_became_peered":"2024-09-11T00:04:11.873810+0000","last_unstale":"2024-09-11T00:06:20.209753+0000","last_undegraded":"2024-09-11T00:06:20.209753+0000","last_fullsized":"2024-09-11T00:06:20.209753+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_clean_scrub_stamp":"2024-09-11T00:04:10.845136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:06:05.130815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"83'5","reported_seq":42,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T00:06:20.214579+0000","last_change":"2024-09-11T00:04:03.834016+0000","last_active":"2024-09-11T00:06:20.214579+0000","last_peered":"2024-09-11T00:06:20.214579+0000","last_clean":"2024-09-11T00:06:20.214579+0000","last_became_active":"2024-09-11T00:04:03.833374+0000","last_became_peered":"2024-09-11T00:04:03.833374+0000","last_unstale":"2024-09-11T00:06:20.214579+0000","last_undegraded":"2024-09-11T00:06:20.214579+0000","last_fullsized":"2024-09-11T00:06:20.214579+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:13:34.068594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"81'4","reported_seq":47,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T00:06:20.215423+0000","last_change":"2024-09-11T00:04:01.821173+0000","last_active":"2024-09-11T00:06:20.215423+0000","last_peered":"2024-09-11T00:06:20.215423+0000","last_clean":"2024-09-11T00:06:20.215423+0000","last_became_active":"2024-09-11T00:04:01.820615+0000","last_became_peered":"2024-09-11T00:04:01.820615+0000","last_unstale":"2024-09-11T00:06:20.215423+0000","last_undegraded":"2024-09-11T00:06:20.215423+0000","last_fullsized":"2024-09-11T00:06:20.215423+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_clean_scrub_stamp":"2024-09-11T00:04:00.794959+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:22:11.842837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":30,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T00:06:20.210190+0000","last_change":"2024-09-11T00:04:12.077637+0000","last_active":"2024-09-11T00:06:20.210190+0000","last_peered":"2024-09-11T00:06:20.210190+0000","last_clean":"2024-09-11T00:06:20.210190+0000","last_became_active":"2024-09-11T00:04:12.076450+0000","last_became_peered":"2024-09-11T00:04:12.076450+0000","last_unstale":"2024-09-11T00:06:20.210190+0000","last_undegraded":"2024-09-11T00:06:20.210190+0000","last_fullsized":"2024-09-11T00:06:20.210190+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_clean_scrub_stamp":"2024-09-11T00:04:10.845136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:50:54.870468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"83'8","reported_seq":45,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T00:06:20.223570+0000","last_change":"2024-09-11T00:04:03.846398+0000","last_active":"2024-09-11T00:06:20.223570+0000","last_peered":"2024-09-11T00:06:20.223570+0000","last_clean":"2024-09-11T00:06:20.223570+0000","last_became_active":"2024-09-11T00:04:03.846051+0000","last_became_peered":"2024-09-11T00:04:03.846051+0000","last_unstale":"2024-09-11T00:06:20.223570+0000","last_undegraded":"2024-09-11T00:06:20.223570+0000","last_fullsized":"2024-09-11T00:06:20.223570+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:00:38.506122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"81'3","reported_seq":22,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T00:06:20.217264+0000","last_change":"2024-09-11T00:05:19.860048+0000","last_active":"2024-09-11T00:06:20.217264+0000","last_peered":"2024-09-11T00:06:20.217264+0000","last_clean":"2024-09-11T00:06:20.217264+0000","last_became_active":"2024-09-11T00:05:19.845210+0000","last_became_peered":"2024-09-11T00:05:19.845210+0000","last_unstale":"2024-09-11T00:06:20.217264+0000","last_undegraded":"2024-09-11T00:06:20.217264+0000","last_fullsized":"2024-09-11T00:06:20.217264+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_clean_scrub_stamp":"2024-09-11T00:04:00.794959+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:55:18.935136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"81'1","reported_seq":31,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T00:06:20.210206+0000","last_change":"2024-09-11T00:04:11.879582+0000","last_active":"2024-09-11T00:06:20.210206+0000","last_peered":"2024-09-11T00:06:20.210206+0000","last_clean":"2024-09-11T00:06:20.210206+0000","last_became_active":"2024-09-11T00:04:11.879098+0000","last_became_peered":"2024-09-11T00:04:11.879098+0000","last_unstale":"2024-09-11T00:06:20.210206+0000","last_undegraded":"2024-09-11T00:06:20.210206+0000","last_fullsized":"2024-09-11T00:06:20.210206+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_clean_scrub_stamp":"2024-09-11T00:04:10.845136+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:06.292841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"83'9","reported_seq":46,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T00:06:20.218519+0000","last_change":"2024-09-11T00:04:03.834615+0000","last_active":"2024-09-11T00:06:20.218519+0000","last_peered":"2024-09-11T00:06:20.218519+0000","last_clean":"2024-09-11T00:06:20.218519+0000","last_became_active":"2024-09-11T00:04:03.834284+0000","last_became_peered":"2024-09-11T00:04:03.834284+0000","last_unstale":"2024-09-11T00:06:20.218519+0000","last_undegraded":"2024-09-11T00:06:20.218519+0000","last_fullsized":"2024-09-11T00:06:20.218519+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:24:09.542774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"81'7","reported_seq":49,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T00:06:20.223445+0000","last_change":"2024-09-11T00:04:01.818156+0000","last_active":"2024-09-11T00:06:20.223445+0000","last_peered":"2024-09-11T00:06:20.223445+0000","last_clean":"2024-09-11T00:06:20.223445+0000","last_became_active":"2024-09-11T00:04:01.817820+0000","last_became_peered":"2024-09-11T00:04:01.817820+0000","last_unstale":"2024-09-11T00:06:20.223445+0000","last_undegraded":"2024-09-11T00:06:20.223445+0000","last_fullsized":"2024-09-11T00:06:20.223445+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_clean_scrub_stamp":"2024-09-11T00:04:00.794959+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:15:46.304276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":30,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T00:06:20.218603+0000","last_change":"2024-09-11T00:04:11.877704+0000","last_active":"2024-09-11T00:06:20.218603+0000","last_peered":"2024-09-11T00:06:20.218603+0000","last_clean":"2024-09-11T00:06:20.218603+0000","last_became_active":"2024-09-11T00:04:11.877320+0000","last_became_peered":"2024-09-11T00:04:11.877320+0000","last_unstale":"2024-09-11T00:06:20.218603+0000","last_undegraded":"2024-09-11T00:06:20.218603+0000","last_fullsized":"2024-09-11T00:06:20.218603+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_clean_scrub_stamp":"2024-09-11T00:04:10.845136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:31:35.046012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.9","version":"83'2","reported_seq":39,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T00:06:20.215128+0000","last_change":"2024-09-11T00:04:03.828240+0000","last_active":"2024-09-11T00:06:20.215128+0000","last_peered":"2024-09-11T00:06:20.215128+0000","last_clean":"2024-09-11T00:06:20.215128+0000","last_became_active":"2024-09-11T00:04:03.827890+0000","last_became_peered":"2024-09-11T00:04:03.827890+0000","last_unstale":"2024-09-11T00:06:20.215128+0000","last_undegraded":"2024-09-11T00:06:20.215128+0000","last_fullsized":"2024-09-11T00:06:20.215128+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:43:12.146717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"81'2","reported_seq":44,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T00:06:20.207648+0000","last_change":"2024-09-11T00:04:01.942805+0000","last_active":"2024-09-11T00:06:20.207648+0000","last_peered":"2024-09-11T00:06:20.207648+0000","last_clean":"2024-09-11T00:06:20.207648+0000","last_became_active":"2024-09-11T00:04:01.942429+0000","last_became_peered":"2024-09-11T00:04:01.942429+0000","last_unstale":"2024-09-11T00:06:20.207648+0000","last_undegraded":"2024-09-11T00:06:20.207648+0000","last_fullsized":"2024-09-11T00:06:20.207648+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_clean_scrub_stamp":"2024-09-11T00:04:00.794959+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:51:22.403825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":30,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T00:06:20.210426+0000","last_change":"2024-09-11T00:04:12.077246+0000","last_active":"2024-09-11T00:06:20.210426+0000","last_peered":"2024-09-11T00:06:20.210426+0000","last_clean":"2024-09-11T00:06:20.210426+0000","last_became_active":"2024-09-11T00:04:12.076481+0000","last_became_peered":"2024-09-11T00:04:12.076481+0000","last_unstale":"2024-09-11T00:06:20.210426+0000","last_undegraded":"2024-09-11T00:06:20.210426+0000","last_fullsized":"2024-09-11T00:06:20.210426+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_clean_scrub_stamp":"2024-09-11T00:04:10.845136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:55:54.347334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"83'7","reported_seq":44,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T00:06:20.209811+0000","last_change":"2024-09-11T00:04:03.838431+0000","last_active":"2024-09-11T00:06:20.209811+0000","last_peered":"2024-09-11T00:06:20.209811+0000","last_clean":"2024-09-11T00:06:20.209811+0000","last_became_active":"2024-09-11T00:04:03.834451+0000","last_became_peered":"2024-09-11T00:04:03.834451+0000","last_unstale":"2024-09-11T00:06:20.209811+0000","last_undegraded":"2024-09-11T00:06:20.209811+0000","last_fullsized":"2024-09-11T00:06:20.209811+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:23:01.594299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"81'7","reported_seq":50,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T00:06:20.210677+0000","last_change":"2024-09-11T00:04:02.588834+0000","last_active":"2024-09-11T00:06:20.210677+0000","last_peered":"2024-09-11T00:06:20.210677+0000","last_clean":"2024-09-11T00:06:20.210677+0000","last_became_active":"2024-09-11T00:04:02.588477+0000","last_became_peered":"2024-09-11T00:04:02.588477+0000","last_unstale":"2024-09-11T00:06:20.210677+0000","last_undegraded":"2024-09-11T00:06:20.210677+0000","last_fullsized":"2024-09-11T00:06:20.210677+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_clean_scrub_stamp":"2024-09-11T00:04:00.794959+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:31:21.805591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":30,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T00:06:20.207762+0000","last_change":"2024-09-11T00:04:11.863381+0000","last_active":"2024-09-11T00:06:20.207762+0000","last_peered":"2024-09-11T00:06:20.207762+0000","last_clean":"2024-09-11T00:06:20.207762+0000","last_became_active":"2024-09-11T00:04:11.862944+0000","last_became_peered":"2024-09-11T00:04:11.862944+0000","last_unstale":"2024-09-11T00:06:20.207762+0000","last_undegraded":"2024-09-11T00:06:20.207762+0000","last_fullsized":"2024-09-11T00:06:20.207762+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_clean_scrub_stamp":"2024-09-11T00:04:10.845136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:00:14.185992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"83'5","reported_seq":42,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T00:06:20.224159+0000","last_change":"2024-09-11T00:04:03.829750+0000","last_active":"2024-09-11T00:06:20.224159+0000","last_peered":"2024-09-11T00:06:20.224159+0000","last_clean":"2024-09-11T00:06:20.224159+0000","last_became_active":"2024-09-11T00:04:03.828166+0000","last_became_peered":"2024-09-11T00:04:03.828166+0000","last_unstale":"2024-09-11T00:06:20.224159+0000","last_undegraded":"2024-09-11T00:06:20.224159+0000","last_fullsized":"2024-09-11T00:06:20.224159+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:55:01.667075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"83'16","reported_seq":58,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T00:06:20.218702+0000","last_change":"2024-09-11T00:04:01.818083+0000","last_active":"2024-09-11T00:06:20.218702+0000","last_peered":"2024-09-11T00:06:20.218702+0000","last_clean":"2024-09-11T00:06:20.218702+0000","last_became_active":"2024-09-11T00:04:01.817784+0000","last_became_peered":"2024-09-11T00:04:01.817784+0000","last_unstale":"2024-09-11T00:06:20.218702+0000","last_undegraded":"2024-09-11T00:06:20.218702+0000","last_fullsized":"2024-09-11T00:06:20.218702+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_clean_scrub_stamp":"2024-09-11T00:04:00.794959+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:38:55.466913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":28,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":30,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T00:06:20.223043+0000","last_change":"2024-09-11T00:04:12.077053+0000","last_active":"2024-09-11T00:06:20.223043+0000","last_peered":"2024-09-11T00:06:20.223043+0000","last_clean":"2024-09-11T00:06:20.223043+0000","last_became_active":"2024-09-11T00:04:12.076632+0000","last_became_peered":"2024-09-11T00:04:12.076632+0000","last_unstale":"2024-09-11T00:06:20.223043+0000","last_undegraded":"2024-09-11T00:06:20.223043+0000","last_fullsized":"2024-09-11T00:06:20.223043+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_clean_scrub_stamp":"2024-09-11T00:04:10.845136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:19:38.321429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"83'9","reported_seq":46,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T00:06:20.224608+0000","last_change":"2024-09-11T00:04:03.830316+0000","last_active":"2024-09-11T00:06:20.224608+0000","last_peered":"2024-09-11T00:06:20.224608+0000","last_clean":"2024-09-11T00:06:20.224608+0000","last_became_active":"2024-09-11T00:04:03.829932+0000","last_became_peered":"2024-09-11T00:04:03.829932+0000","last_unstale":"2024-09-11T00:06:20.224608+0000","last_undegraded":"2024-09-11T00:06:20.224608+0000","last_fullsized":"2024-09-11T00:06:20.224608+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:06:14.546651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"81'4","reported_seq":47,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T00:06:20.215888+0000","last_change":"2024-09-11T00:04:01.943004+0000","last_active":"2024-09-11T00:06:20.215888+0000","last_peered":"2024-09-11T00:06:20.215888+0000","last_clean":"2024-09-11T00:06:20.215888+0000","last_became_active":"2024-09-11T00:04:01.942432+0000","last_became_peered":"2024-09-11T00:04:01.942432+0000","last_unstale":"2024-09-11T00:06:20.215888+0000","last_undegraded":"2024-09-11T00:06:20.215888+0000","last_fullsized":"2024-09-11T00:06:20.215888+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_clean_scrub_stamp":"2024-09-11T00:04:00.794959+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:52:50.472596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":30,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T00:06:20.218250+0000","last_change":"2024-09-11T00:04:11.866778+0000","last_active":"2024-09-11T00:06:20.218250+0000","last_peered":"2024-09-11T00:06:20.218250+0000","last_clean":"2024-09-11T00:06:20.218250+0000","last_became_active":"2024-09-11T00:04:11.866333+0000","last_became_peered":"2024-09-11T00:04:11.866333+0000","last_unstale":"2024-09-11T00:06:20.218250+0000","last_undegraded":"2024-09-11T00:06:20.218250+0000","last_fullsized":"2024-09-11T00:06:20.218250+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_clean_scrub_stamp":"2024-09-11T00:04:10.845136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:33.680119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"83'4","reported_seq":41,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T00:06:20.224266+0000","last_change":"2024-09-11T00:04:03.832481+0000","last_active":"2024-09-11T00:06:20.224266+0000","last_peered":"2024-09-11T00:06:20.224266+0000","last_clean":"2024-09-11T00:06:20.224266+0000","last_became_active":"2024-09-11T00:04:03.832162+0000","last_became_peered":"2024-09-11T00:04:03.832162+0000","last_unstale":"2024-09-11T00:06:20.224266+0000","last_undegraded":"2024-09-11T00:06:20.224266+0000","last_fullsized":"2024-09-11T00:06:20.224266+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:32:52.550128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"81'1","reported_seq":43,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T00:06:20.218249+0000","last_change":"2024-09-11T00:04:01.943647+0000","last_active":"2024-09-11T00:06:20.218249+0000","last_peered":"2024-09-11T00:06:20.218249+0000","last_clean":"2024-09-11T00:06:20.218249+0000","last_became_active":"2024-09-11T00:04:01.942714+0000","last_became_peered":"2024-09-11T00:04:01.942714+0000","last_unstale":"2024-09-11T00:06:20.218249+0000","last_undegraded":"2024-09-11T00:06:20.218249+0000","last_fullsized":"2024-09-11T00:06:20.218249+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_clean_scrub_stamp":"2024-09-11T00:04:00.794959+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:22:23.687268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":30,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T00:06:20.215230+0000","last_change":"2024-09-11T00:04:12.077133+0000","last_active":"2024-09-11T00:06:20.215230+0000","last_peered":"2024-09-11T00:06:20.215230+0000","last_clean":"2024-09-11T00:06:20.215230+0000","last_became_active":"2024-09-11T00:04:12.076657+0000","last_became_peered":"2024-09-11T00:04:12.076657+0000","last_unstale":"2024-09-11T00:06:20.215230+0000","last_undegraded":"2024-09-11T00:06:20.215230+0000","last_fullsized":"2024-09-11T00:06:20.215230+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_clean_scrub_stamp":"2024-09-11T00:04:10.845136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:05:40.643405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"83'7","reported_seq":44,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T00:06:20.224150+0000","last_change":"2024-09-11T00:04:03.835454+0000","last_active":"2024-09-11T00:06:20.224150+0000","last_peered":"2024-09-11T00:06:20.224150+0000","last_clean":"2024-09-11T00:06:20.224150+0000","last_became_active":"2024-09-11T00:04:03.835122+0000","last_became_peered":"2024-09-11T00:04:03.835122+0000","last_unstale":"2024-09-11T00:06:20.224150+0000","last_undegraded":"2024-09-11T00:06:20.224150+0000","last_fullsized":"2024-09-11T00:06:20.224150+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:25:49.672766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"83'7","reported_seq":168,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T00:06:44.432221+0000","last_change":"2024-09-11T00:04:01.823738+0000","last_active":"2024-09-11T00:06:44.432221+0000","last_peered":"2024-09-11T00:06:44.432221+0000","last_clean":"2024-09-11T00:06:44.432221+0000","last_became_active":"2024-09-11T00:04:01.823320+0000","last_became_peered":"2024-09-11T00:04:01.823320+0000","last_unstale":"2024-09-11T00:06:44.432221+0000","last_undegraded":"2024-09-11T00:06:44.432221+0000","last_fullsized":"2024-09-11T00:06:44.432221+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_clean_scrub_stamp":"2024-09-11T00:04:00.794959+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:34:30.088454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1120,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":119,"num_read_kb":3,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":30,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T00:06:20.224438+0000","last_change":"2024-09-11T00:04:11.863906+0000","last_active":"2024-09-11T00:06:20.224438+0000","last_peered":"2024-09-11T00:06:20.224438+0000","last_clean":"2024-09-11T00:06:20.224438+0000","last_became_active":"2024-09-11T00:04:11.863571+0000","last_became_peered":"2024-09-11T00:04:11.863571+0000","last_unstale":"2024-09-11T00:06:20.224438+0000","last_undegraded":"2024-09-11T00:06:20.224438+0000","last_fullsized":"2024-09-11T00:06:20.224438+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_clean_scrub_stamp":"2024-09-11T00:04:10.845136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:04:40.288886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"83'8","reported_seq":45,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T00:06:20.217947+0000","last_change":"2024-09-11T00:04:03.826623+0000","last_active":"2024-09-11T00:06:20.217947+0000","last_peered":"2024-09-11T00:06:20.217947+0000","last_clean":"2024-09-11T00:06:20.217947+0000","last_became_active":"2024-09-11T00:04:03.826202+0000","last_became_peered":"2024-09-11T00:04:03.826202+0000","last_unstale":"2024-09-11T00:06:20.217947+0000","last_undegraded":"2024-09-11T00:06:20.217947+0000","last_fullsized":"2024-09-11T00:06:20.217947+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:23:13.222654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"85'6","reported_seq":164,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T00:06:43.803171+0000","last_change":"2024-09-11T00:04:01.942857+0000","last_active":"2024-09-11T00:06:43.803171+0000","last_peered":"2024-09-11T00:06:43.803171+0000","last_clean":"2024-09-11T00:06:43.803171+0000","last_became_active":"2024-09-11T00:04:01.942494+0000","last_became_peered":"2024-09-11T00:04:01.942494+0000","last_unstale":"2024-09-11T00:06:43.803171+0000","last_undegraded":"2024-09-11T00:06:43.803171+0000","last_fullsized":"2024-09-11T00:06:43.803171+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_clean_scrub_stamp":"2024-09-11T00:04:00.794959+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:40:38.587178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":116,"num_read_kb":116,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":30,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T00:06:20.224554+0000","last_change":"2024-09-11T00:04:12.077771+0000","last_active":"2024-09-11T00:06:20.224554+0000","last_peered":"2024-09-11T00:06:20.224554+0000","last_clean":"2024-09-11T00:06:20.224554+0000","last_became_active":"2024-09-11T00:04:12.077366+0000","last_became_peered":"2024-09-11T00:04:12.077366+0000","last_unstale":"2024-09-11T00:06:20.224554+0000","last_undegraded":"2024-09-11T00:06:20.224554+0000","last_fullsized":"2024-09-11T00:06:20.224554+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_clean_scrub_stamp":"2024-09-11T00:04:10.845136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:04:26.808201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"83'5","reported_seq":42,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T00:06:20.223536+0000","last_change":"2024-09-11T00:04:03.827872+0000","last_active":"2024-09-11T00:06:20.223536+0000","last_peered":"2024-09-11T00:06:20.223536+0000","last_clean":"2024-09-11T00:06:20.223536+0000","last_became_active":"2024-09-11T00:04:03.827577+0000","last_became_peered":"2024-09-11T00:04:03.827577+0000","last_unstale":"2024-09-11T00:06:20.223536+0000","last_undegraded":"2024-09-11T00:06:20.223536+0000","last_fullsized":"2024-09-11T00:06:20.223536+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:37:19.205218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"81'4","reported_seq":57,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T00:06:20.214717+0000","last_change":"2024-09-11T00:05:19.991372+0000","last_active":"2024-09-11T00:06:20.214717+0000","last_peered":"2024-09-11T00:06:20.214717+0000","last_clean":"2024-09-11T00:06:20.214717+0000","last_became_active":"2024-09-11T00:05:19.985328+0000","last_became_peered":"2024-09-11T00:05:19.985328+0000","last_unstale":"2024-09-11T00:06:20.214717+0000","last_undegraded":"2024-09-11T00:06:20.214717+0000","last_fullsized":"2024-09-11T00:06:20.214717+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_clean_scrub_stamp":"2024-09-11T00:04:00.794959+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:17:20.345615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,4],"acting":[7,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T00:06:20.226394+0000","last_change":"2024-09-11T00:06:20.225938+0000","last_active":"2024-09-11T00:06:20.226394+0000","last_peered":"2024-09-11T00:06:20.226394+0000","last_clean":"2024-09-11T00:06:20.226394+0000","last_became_active":"2024-09-11T00:06:20.225447+0000","last_became_peered":"2024-09-11T00:06:20.225447+0000","last_unstale":"2024-09-11T00:06:20.226394+0000","last_undegraded":"2024-09-11T00:06:20.226394+0000","last_fullsized":"2024-09-11T00:06:20.226394+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_clean_scrub_stamp":"2024-09-11T00:04:10.845136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:30:17.498206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,6,10],"acting":[8,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.2","version":"83'8","reported_seq":45,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T00:06:20.223732+0000","last_change":"2024-09-11T00:04:03.839615+0000","last_active":"2024-09-11T00:06:20.223732+0000","last_peered":"2024-09-11T00:06:20.223732+0000","last_clean":"2024-09-11T00:06:20.223732+0000","last_became_active":"2024-09-11T00:04:03.832175+0000","last_became_peered":"2024-09-11T00:04:03.832175+0000","last_unstale":"2024-09-11T00:06:20.223732+0000","last_undegraded":"2024-09-11T00:06:20.223732+0000","last_fullsized":"2024-09-11T00:06:20.223732+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:17:09.975345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"81'6","reported_seq":51,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T00:06:20.210941+0000","last_change":"2024-09-11T00:04:01.817248+0000","last_active":"2024-09-11T00:06:20.210941+0000","last_peered":"2024-09-11T00:06:20.210941+0000","last_clean":"2024-09-11T00:06:20.210941+0000","last_became_active":"2024-09-11T00:04:01.816896+0000","last_became_peered":"2024-09-11T00:04:01.816896+0000","last_unstale":"2024-09-11T00:06:20.210941+0000","last_undegraded":"2024-09-11T00:06:20.210941+0000","last_fullsized":"2024-09-11T00:06:20.210941+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_clean_scrub_stamp":"2024-09-11T00:04:00.794959+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:46:51.641881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'92","reported_seq":114,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T00:06:20.214947+0000","last_change":"2024-09-11T00:02:47.245193+0000","last_active":"2024-09-11T00:06:20.214947+0000","last_peered":"2024-09-11T00:06:20.214947+0000","last_clean":"2024-09-11T00:06:20.214947+0000","last_became_active":"2024-09-11T00:02:47.239773+0000","last_became_peered":"2024-09-11T00:02:47.239773+0000","last_unstale":"2024-09-11T00:06:20.214947+0000","last_undegraded":"2024-09-11T00:06:20.214947+0000","last_fullsized":"2024-09-11T00:06:20.214947+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:59:10.637655+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:59:10.637655+0000","last_clean_scrub_stamp":"2024-09-10T23:59:10.637655+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:51:52.914831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"81'1","reported_seq":43,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T00:06:20.230099+0000","last_change":"2024-09-11T00:06:20.229604+0000","last_active":"2024-09-11T00:06:20.230099+0000","last_peered":"2024-09-11T00:06:20.230099+0000","last_clean":"2024-09-11T00:06:20.230099+0000","last_became_active":"2024-09-11T00:06:20.225065+0000","last_became_peered":"2024-09-11T00:06:20.225065+0000","last_unstale":"2024-09-11T00:06:20.230099+0000","last_undegraded":"2024-09-11T00:06:20.230099+0000","last_fullsized":"2024-09-11T00:06:20.230099+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_clean_scrub_stamp":"2024-09-11T00:04:10.845136+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:52:41.717530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,10],"acting":[7,5,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"83'7","reported_seq":44,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T00:06:20.215116+0000","last_change":"2024-09-11T00:04:03.834755+0000","last_active":"2024-09-11T00:06:20.215116+0000","last_peered":"2024-09-11T00:06:20.215116+0000","last_clean":"2024-09-11T00:06:20.215116+0000","last_became_active":"2024-09-11T00:04:03.833955+0000","last_became_peered":"2024-09-11T00:04:03.833955+0000","last_unstale":"2024-09-11T00:06:20.215116+0000","last_undegraded":"2024-09-11T00:06:20.215116+0000","last_fullsized":"2024-09-11T00:06:20.215116+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:47:13.863481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"83'7","reported_seq":168,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T00:06:43.431261+0000","last_change":"2024-09-11T00:04:02.589042+0000","last_active":"2024-09-11T00:06:43.431261+0000","last_peered":"2024-09-11T00:06:43.431261+0000","last_clean":"2024-09-11T00:06:43.431261+0000","last_became_active":"2024-09-11T00:04:02.588611+0000","last_became_peered":"2024-09-11T00:04:02.588611+0000","last_unstale":"2024-09-11T00:06:43.431261+0000","last_undegraded":"2024-09-11T00:06:43.431261+0000","last_fullsized":"2024-09-11T00:06:43.431261+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_clean_scrub_stamp":"2024-09-11T00:04:00.794959+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:41:08.257605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1702,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":119,"num_read_kb":4,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":30,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T00:06:20.207731+0000","last_change":"2024-09-11T00:04:11.860715+0000","last_active":"2024-09-11T00:06:20.207731+0000","last_peered":"2024-09-11T00:06:20.207731+0000","last_clean":"2024-09-11T00:06:20.207731+0000","last_became_active":"2024-09-11T00:04:11.860094+0000","last_became_peered":"2024-09-11T00:04:11.860094+0000","last_unstale":"2024-09-11T00:06:20.207731+0000","last_undegraded":"2024-09-11T00:06:20.207731+0000","last_fullsized":"2024-09-11T00:06:20.207731+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_clean_scrub_stamp":"2024-09-11T00:04:10.845136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:25:55.279079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"83'6","reported_seq":43,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T00:06:20.214828+0000","last_change":"2024-09-11T00:04:03.822665+0000","last_active":"2024-09-11T00:06:20.214828+0000","last_peered":"2024-09-11T00:06:20.214828+0000","last_clean":"2024-09-11T00:06:20.214828+0000","last_became_active":"2024-09-11T00:04:03.822333+0000","last_became_peered":"2024-09-11T00:04:03.822333+0000","last_unstale":"2024-09-11T00:06:20.214828+0000","last_undegraded":"2024-09-11T00:06:20.214828+0000","last_fullsized":"2024-09-11T00:06:20.214828+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:02:07.465335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"81'3","reported_seq":46,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T00:06:20.218325+0000","last_change":"2024-09-11T00:04:01.817244+0000","last_active":"2024-09-11T00:06:20.218325+0000","last_peered":"2024-09-11T00:06:20.218325+0000","last_clean":"2024-09-11T00:06:20.218325+0000","last_became_active":"2024-09-11T00:04:01.816846+0000","last_became_peered":"2024-09-11T00:04:01.816846+0000","last_unstale":"2024-09-11T00:06:20.218325+0000","last_undegraded":"2024-09-11T00:06:20.218325+0000","last_fullsized":"2024-09-11T00:06:20.218325+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_clean_scrub_stamp":"2024-09-11T00:04:00.794959+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:09:46.232332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":38,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T00:06:20.217751+0000","last_change":"2024-09-11T00:06:20.217001+0000","last_active":"2024-09-11T00:06:20.217751+0000","last_peered":"2024-09-11T00:06:20.217751+0000","last_clean":"2024-09-11T00:06:20.217751+0000","last_became_active":"2024-09-11T00:06:20.216599+0000","last_became_peered":"2024-09-11T00:06:20.216599+0000","last_unstale":"2024-09-11T00:06:20.217751+0000","last_undegraded":"2024-09-11T00:06:20.217751+0000","last_fullsized":"2024-09-11T00:06:20.217751+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_clean_scrub_stamp":"2024-09-11T00:04:10.845136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:06:42.086069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,1,4],"acting":[0,8,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"83'9","reported_seq":46,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T00:06:20.214846+0000","last_change":"2024-09-11T00:04:03.829600+0000","last_active":"2024-09-11T00:06:20.214846+0000","last_peered":"2024-09-11T00:06:20.214846+0000","last_clean":"2024-09-11T00:06:20.214846+0000","last_became_active":"2024-09-11T00:04:03.829118+0000","last_became_peered":"2024-09-11T00:04:03.829118+0000","last_unstale":"2024-09-11T00:06:20.214846+0000","last_undegraded":"2024-09-11T00:06:20.214846+0000","last_fullsized":"2024-09-11T00:06:20.214846+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:41:07.069816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"81'5","reported_seq":60,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T00:06:20.224623+0000","last_change":"2024-09-11T00:05:19.989379+0000","last_active":"2024-09-11T00:06:20.224623+0000","last_peered":"2024-09-11T00:06:20.224623+0000","last_clean":"2024-09-11T00:06:20.224623+0000","last_became_active":"2024-09-11T00:05:19.984918+0000","last_became_peered":"2024-09-11T00:05:19.984918+0000","last_unstale":"2024-09-11T00:06:20.224623+0000","last_undegraded":"2024-09-11T00:06:20.224623+0000","last_fullsized":"2024-09-11T00:06:20.224623+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_clean_scrub_stamp":"2024-09-11T00:04:00.794959+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:27:52.063256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4],"acting":[6,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":30,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T00:06:20.217466+0000","last_change":"2024-09-11T00:04:11.863812+0000","last_active":"2024-09-11T00:06:20.217466+0000","last_peered":"2024-09-11T00:06:20.217466+0000","last_clean":"2024-09-11T00:06:20.217466+0000","last_became_active":"2024-09-11T00:04:11.863412+0000","last_became_peered":"2024-09-11T00:04:11.863412+0000","last_unstale":"2024-09-11T00:06:20.217466+0000","last_undegraded":"2024-09-11T00:06:20.217466+0000","last_fullsized":"2024-09-11T00:06:20.217466+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_clean_scrub_stamp":"2024-09-11T00:04:10.845136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:12:04.140034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"83'5","reported_seq":49,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T00:06:20.218684+0000","last_change":"2024-09-11T00:05:19.985248+0000","last_active":"2024-09-11T00:06:20.218684+0000","last_peered":"2024-09-11T00:06:20.218684+0000","last_clean":"2024-09-11T00:06:20.218684+0000","last_became_active":"2024-09-11T00:05:19.984921+0000","last_became_peered":"2024-09-11T00:05:19.984921+0000","last_unstale":"2024-09-11T00:06:20.218684+0000","last_undegraded":"2024-09-11T00:06:20.218684+0000","last_fullsized":"2024-09-11T00:06:20.218684+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:16:44.370849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,4],"acting":[11,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"85'3","reported_seq":46,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T00:06:20.216030+0000","last_change":"2024-09-11T00:04:01.814191+0000","last_active":"2024-09-11T00:06:20.216030+0000","last_peered":"2024-09-11T00:06:20.216030+0000","last_clean":"2024-09-11T00:06:20.216030+0000","last_became_active":"2024-09-11T00:04:01.813778+0000","last_became_peered":"2024-09-11T00:04:01.813778+0000","last_unstale":"2024-09-11T00:06:20.216030+0000","last_undegraded":"2024-09-11T00:06:20.216030+0000","last_fullsized":"2024-09-11T00:06:20.216030+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_clean_scrub_stamp":"2024-09-11T00:04:00.794959+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:01:49.044436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":30,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T00:06:20.217607+0000","last_change":"2024-09-11T00:04:11.870281+0000","last_active":"2024-09-11T00:06:20.217607+0000","last_peered":"2024-09-11T00:06:20.217607+0000","last_clean":"2024-09-11T00:06:20.217607+0000","last_became_active":"2024-09-11T00:04:11.869738+0000","last_became_peered":"2024-09-11T00:04:11.869738+0000","last_unstale":"2024-09-11T00:06:20.217607+0000","last_undegraded":"2024-09-11T00:06:20.217607+0000","last_fullsized":"2024-09-11T00:06:20.217607+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_clean_scrub_stamp":"2024-09-11T00:04:10.845136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:04:08.380199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"83'6","reported_seq":43,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T00:06:20.217988+0000","last_change":"2024-09-11T00:04:03.835240+0000","last_active":"2024-09-11T00:06:20.217988+0000","last_peered":"2024-09-11T00:06:20.217988+0000","last_clean":"2024-09-11T00:06:20.217988+0000","last_became_active":"2024-09-11T00:04:03.834910+0000","last_became_peered":"2024-09-11T00:04:03.834910+0000","last_unstale":"2024-09-11T00:06:20.217988+0000","last_undegraded":"2024-09-11T00:06:20.217988+0000","last_fullsized":"2024-09-11T00:06:20.217988+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:30:33.996409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.10","version":"81'17","reported_seq":34,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T00:06:20.218188+0000","last_change":"2024-09-11T00:05:19.856704+0000","last_active":"2024-09-11T00:06:20.218188+0000","last_peered":"2024-09-11T00:06:20.218188+0000","last_clean":"2024-09-11T00:06:20.218188+0000","last_became_active":"2024-09-11T00:05:19.843487+0000","last_became_peered":"2024-09-11T00:05:19.843487+0000","last_unstale":"2024-09-11T00:06:20.218188+0000","last_undegraded":"2024-09-11T00:06:20.218188+0000","last_fullsized":"2024-09-11T00:06:20.218188+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_clean_scrub_stamp":"2024-09-11T00:04:00.794959+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:03:51.375839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":30,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T00:06:20.224599+0000","last_change":"2024-09-11T00:04:11.874306+0000","last_active":"2024-09-11T00:06:20.224599+0000","last_peered":"2024-09-11T00:06:20.224599+0000","last_clean":"2024-09-11T00:06:20.224599+0000","last_became_active":"2024-09-11T00:04:11.873934+0000","last_became_peered":"2024-09-11T00:04:11.873934+0000","last_unstale":"2024-09-11T00:06:20.224599+0000","last_undegraded":"2024-09-11T00:06:20.224599+0000","last_fullsized":"2024-09-11T00:06:20.224599+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_clean_scrub_stamp":"2024-09-11T00:04:10.845136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:19:22.584367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"83'6","reported_seq":43,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T00:06:20.214810+0000","last_change":"2024-09-11T00:04:03.824219+0000","last_active":"2024-09-11T00:06:20.214810+0000","last_peered":"2024-09-11T00:06:20.214810+0000","last_clean":"2024-09-11T00:06:20.214810+0000","last_became_active":"2024-09-11T00:04:03.823877+0000","last_became_peered":"2024-09-11T00:04:03.823877+0000","last_unstale":"2024-09-11T00:06:20.214810+0000","last_undegraded":"2024-09-11T00:06:20.214810+0000","last_fullsized":"2024-09-11T00:06:20.214810+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:04:05.924113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"85'6","reported_seq":166,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T00:06:44.432477+0000","last_change":"2024-09-11T00:04:01.818055+0000","last_active":"2024-09-11T00:06:44.432477+0000","last_peered":"2024-09-11T00:06:44.432477+0000","last_clean":"2024-09-11T00:06:44.432477+0000","last_became_active":"2024-09-11T00:04:01.817483+0000","last_became_peered":"2024-09-11T00:04:01.817483+0000","last_unstale":"2024-09-11T00:06:44.432477+0000","last_undegraded":"2024-09-11T00:06:44.432477+0000","last_fullsized":"2024-09-11T00:06:44.432477+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_clean_scrub_stamp":"2024-09-11T00:04:00.794959+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:16:55.218673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":899,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":119,"num_read_kb":4,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":30,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T00:06:20.218592+0000","last_change":"2024-09-11T00:04:11.875918+0000","last_active":"2024-09-11T00:06:20.218592+0000","last_peered":"2024-09-11T00:06:20.218592+0000","last_clean":"2024-09-11T00:06:20.218592+0000","last_became_active":"2024-09-11T00:04:11.875549+0000","last_became_peered":"2024-09-11T00:04:11.875549+0000","last_unstale":"2024-09-11T00:06:20.218592+0000","last_undegraded":"2024-09-11T00:06:20.218592+0000","last_fullsized":"2024-09-11T00:06:20.218592+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_clean_scrub_stamp":"2024-09-11T00:04:10.845136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:16:48.685899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.13","version":"83'15","reported_seq":52,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T00:06:20.215033+0000","last_change":"2024-09-11T00:04:03.827671+0000","last_active":"2024-09-11T00:06:20.215033+0000","last_peered":"2024-09-11T00:06:20.215033+0000","last_clean":"2024-09-11T00:06:20.215033+0000","last_became_active":"2024-09-11T00:04:03.827341+0000","last_became_peered":"2024-09-11T00:04:03.827341+0000","last_unstale":"2024-09-11T00:06:20.215033+0000","last_undegraded":"2024-09-11T00:06:20.215033+0000","last_fullsized":"2024-09-11T00:06:20.215033+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:56:16.614035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"81'5","reported_seq":48,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T00:06:20.218496+0000","last_change":"2024-09-11T00:04:01.820632+0000","last_active":"2024-09-11T00:06:20.218496+0000","last_peered":"2024-09-11T00:06:20.218496+0000","last_clean":"2024-09-11T00:06:20.218496+0000","last_became_active":"2024-09-11T00:04:01.820273+0000","last_became_peered":"2024-09-11T00:04:01.820273+0000","last_unstale":"2024-09-11T00:06:20.218496+0000","last_undegraded":"2024-09-11T00:06:20.218496+0000","last_fullsized":"2024-09-11T00:06:20.218496+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_clean_scrub_stamp":"2024-09-11T00:04:00.794959+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:34:53.444515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":30,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T00:06:20.218420+0000","last_change":"2024-09-11T00:04:11.881197+0000","last_active":"2024-09-11T00:06:20.218420+0000","last_peered":"2024-09-11T00:06:20.218420+0000","last_clean":"2024-09-11T00:06:20.218420+0000","last_became_active":"2024-09-11T00:04:11.880763+0000","last_became_peered":"2024-09-11T00:04:11.880763+0000","last_unstale":"2024-09-11T00:06:20.218420+0000","last_undegraded":"2024-09-11T00:06:20.218420+0000","last_fullsized":"2024-09-11T00:06:20.218420+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_clean_scrub_stamp":"2024-09-11T00:04:10.845136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:14:01.616095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"83'9","reported_seq":46,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T00:06:20.224080+0000","last_change":"2024-09-11T00:04:03.833470+0000","last_active":"2024-09-11T00:06:20.224080+0000","last_peered":"2024-09-11T00:06:20.224080+0000","last_clean":"2024-09-11T00:06:20.224080+0000","last_became_active":"2024-09-11T00:04:03.833120+0000","last_became_peered":"2024-09-11T00:04:03.833120+0000","last_unstale":"2024-09-11T00:06:20.224080+0000","last_undegraded":"2024-09-11T00:06:20.224080+0000","last_fullsized":"2024-09-11T00:06:20.224080+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:29:53.210230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"81'9","reported_seq":53,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T00:06:20.208085+0000","last_change":"2024-09-11T00:04:01.818081+0000","last_active":"2024-09-11T00:06:20.208085+0000","last_peered":"2024-09-11T00:06:20.208085+0000","last_clean":"2024-09-11T00:06:20.208085+0000","last_became_active":"2024-09-11T00:04:01.817716+0000","last_became_peered":"2024-09-11T00:04:01.817716+0000","last_unstale":"2024-09-11T00:06:20.208085+0000","last_undegraded":"2024-09-11T00:06:20.208085+0000","last_fullsized":"2024-09-11T00:06:20.208085+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_clean_scrub_stamp":"2024-09-11T00:04:00.794959+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:01:32.736319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":30,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T00:06:20.218197+0000","last_change":"2024-09-11T00:04:11.870182+0000","last_active":"2024-09-11T00:06:20.218197+0000","last_peered":"2024-09-11T00:06:20.218197+0000","last_clean":"2024-09-11T00:06:20.218197+0000","last_became_active":"2024-09-11T00:04:11.869744+0000","last_became_peered":"2024-09-11T00:04:11.869744+0000","last_unstale":"2024-09-11T00:06:20.218197+0000","last_undegraded":"2024-09-11T00:06:20.218197+0000","last_fullsized":"2024-09-11T00:06:20.218197+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_clean_scrub_stamp":"2024-09-11T00:04:10.845136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:50:24.070404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"83'6","reported_seq":43,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T00:06:20.215695+0000","last_change":"2024-09-11T00:04:03.828246+0000","last_active":"2024-09-11T00:06:20.215695+0000","last_peered":"2024-09-11T00:06:20.215695+0000","last_clean":"2024-09-11T00:06:20.215695+0000","last_became_active":"2024-09-11T00:04:03.827217+0000","last_became_peered":"2024-09-11T00:04:03.827217+0000","last_unstale":"2024-09-11T00:06:20.215695+0000","last_undegraded":"2024-09-11T00:06:20.215695+0000","last_fullsized":"2024-09-11T00:06:20.215695+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:51:46.018759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"85'41","reported_seq":327,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T00:06:45.434193+0000","last_change":"2024-09-11T00:04:01.819553+0000","last_active":"2024-09-11T00:06:45.434193+0000","last_peered":"2024-09-11T00:06:45.434193+0000","last_clean":"2024-09-11T00:06:45.434193+0000","last_became_active":"2024-09-11T00:04:01.819213+0000","last_became_peered":"2024-09-11T00:04:01.819213+0000","last_unstale":"2024-09-11T00:06:45.434193+0000","last_undegraded":"2024-09-11T00:06:45.434193+0000","last_fullsized":"2024-09-11T00:06:45.434193+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_clean_scrub_stamp":"2024-09-11T00:04:00.794959+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:44:11.039982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":65178,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":244,"num_read_kb":186,"num_write":43,"num_write_kb":84,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":30,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T00:06:20.217633+0000","last_change":"2024-09-11T00:04:11.871730+0000","last_active":"2024-09-11T00:06:20.217633+0000","last_peered":"2024-09-11T00:06:20.217633+0000","last_clean":"2024-09-11T00:06:20.217633+0000","last_became_active":"2024-09-11T00:04:11.870659+0000","last_became_peered":"2024-09-11T00:04:11.870659+0000","last_unstale":"2024-09-11T00:06:20.217633+0000","last_undegraded":"2024-09-11T00:06:20.217633+0000","last_fullsized":"2024-09-11T00:06:20.217633+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_clean_scrub_stamp":"2024-09-11T00:04:10.845136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:48:41.835602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.14","version":"83'11","reported_seq":48,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T00:06:20.208703+0000","last_change":"2024-09-11T00:04:03.838763+0000","last_active":"2024-09-11T00:06:20.208703+0000","last_peered":"2024-09-11T00:06:20.208703+0000","last_clean":"2024-09-11T00:06:20.208703+0000","last_became_active":"2024-09-11T00:04:03.838409+0000","last_became_peered":"2024-09-11T00:04:03.838409+0000","last_unstale":"2024-09-11T00:06:20.208703+0000","last_undegraded":"2024-09-11T00:06:20.208703+0000","last_fullsized":"2024-09-11T00:06:20.208703+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:57:38.647351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"81'5","reported_seq":48,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T00:06:20.218329+0000","last_change":"2024-09-11T00:04:01.943027+0000","last_active":"2024-09-11T00:06:20.218329+0000","last_peered":"2024-09-11T00:06:20.218329+0000","last_clean":"2024-09-11T00:06:20.218329+0000","last_became_active":"2024-09-11T00:04:01.942558+0000","last_became_peered":"2024-09-11T00:04:01.942558+0000","last_unstale":"2024-09-11T00:06:20.218329+0000","last_undegraded":"2024-09-11T00:06:20.218329+0000","last_fullsized":"2024-09-11T00:06:20.218329+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_clean_scrub_stamp":"2024-09-11T00:04:00.794959+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:48:03.776476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":38,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T00:06:20.217115+0000","last_change":"2024-09-11T00:06:20.216571+0000","last_active":"2024-09-11T00:06:20.217115+0000","last_peered":"2024-09-11T00:06:20.217115+0000","last_clean":"2024-09-11T00:06:20.217115+0000","last_became_active":"2024-09-11T00:06:20.216166+0000","last_became_peered":"2024-09-11T00:06:20.216166+0000","last_unstale":"2024-09-11T00:06:20.217115+0000","last_undegraded":"2024-09-11T00:06:20.217115+0000","last_fullsized":"2024-09-11T00:06:20.217115+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_clean_scrub_stamp":"2024-09-11T00:04:10.845136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:58:59.803225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,8],"acting":[4,1,2,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"83'9","reported_seq":46,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T00:06:20.210061+0000","last_change":"2024-09-11T00:04:03.836735+0000","last_active":"2024-09-11T00:06:20.210061+0000","last_peered":"2024-09-11T00:06:20.210061+0000","last_clean":"2024-09-11T00:06:20.210061+0000","last_became_active":"2024-09-11T00:04:03.833299+0000","last_became_peered":"2024-09-11T00:04:03.833299+0000","last_unstale":"2024-09-11T00:06:20.210061+0000","last_undegraded":"2024-09-11T00:06:20.210061+0000","last_fullsized":"2024-09-11T00:06:20.210061+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:32:15.924058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"81'4","reported_seq":46,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T00:06:20.215069+0000","last_change":"2024-09-11T00:04:01.823777+0000","last_active":"2024-09-11T00:06:20.215069+0000","last_peered":"2024-09-11T00:06:20.215069+0000","last_clean":"2024-09-11T00:06:20.215069+0000","last_became_active":"2024-09-11T00:04:01.823368+0000","last_became_peered":"2024-09-11T00:04:01.823368+0000","last_unstale":"2024-09-11T00:06:20.215069+0000","last_undegraded":"2024-09-11T00:06:20.215069+0000","last_fullsized":"2024-09-11T00:06:20.215069+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_clean_scrub_stamp":"2024-09-11T00:04:00.794959+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:05:12.584867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":30,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T00:06:20.217038+0000","last_change":"2024-09-11T00:04:11.868303+0000","last_active":"2024-09-11T00:06:20.217038+0000","last_peered":"2024-09-11T00:06:20.217038+0000","last_clean":"2024-09-11T00:06:20.217038+0000","last_became_active":"2024-09-11T00:04:11.867916+0000","last_became_peered":"2024-09-11T00:04:11.867916+0000","last_unstale":"2024-09-11T00:06:20.217038+0000","last_undegraded":"2024-09-11T00:06:20.217038+0000","last_fullsized":"2024-09-11T00:06:20.217038+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_clean_scrub_stamp":"2024-09-11T00:04:10.845136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:08:29.711763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.16","version":"83'10","reported_seq":47,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T00:06:20.208348+0000","last_change":"2024-09-11T00:04:03.832712+0000","last_active":"2024-09-11T00:06:20.208348+0000","last_peered":"2024-09-11T00:06:20.208348+0000","last_clean":"2024-09-11T00:06:20.208348+0000","last_became_active":"2024-09-11T00:04:03.832295+0000","last_became_peered":"2024-09-11T00:04:03.832295+0000","last_unstale":"2024-09-11T00:06:20.208348+0000","last_undegraded":"2024-09-11T00:06:20.208348+0000","last_fullsized":"2024-09-11T00:06:20.208348+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:40:12.271930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"83'5","reported_seq":279,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T00:06:45.433903+0000","last_change":"2024-09-11T00:04:01.820891+0000","last_active":"2024-09-11T00:06:45.433903+0000","last_peered":"2024-09-11T00:06:45.433903+0000","last_clean":"2024-09-11T00:06:45.433903+0000","last_became_active":"2024-09-11T00:04:01.820528+0000","last_became_peered":"2024-09-11T00:04:01.820528+0000","last_unstale":"2024-09-11T00:06:45.433903+0000","last_undegraded":"2024-09-11T00:06:45.433903+0000","last_fullsized":"2024-09-11T00:06:45.433903+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_clean_scrub_stamp":"2024-09-11T00:04:00.794959+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:23:38.196618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":232,"num_read_kb":232,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":30,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T00:06:20.223208+0000","last_change":"2024-09-11T00:04:11.878864+0000","last_active":"2024-09-11T00:06:20.223208+0000","last_peered":"2024-09-11T00:06:20.223208+0000","last_clean":"2024-09-11T00:06:20.223208+0000","last_became_active":"2024-09-11T00:04:11.876701+0000","last_became_peered":"2024-09-11T00:04:11.876701+0000","last_unstale":"2024-09-11T00:06:20.223208+0000","last_undegraded":"2024-09-11T00:06:20.223208+0000","last_fullsized":"2024-09-11T00:06:20.223208+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_clean_scrub_stamp":"2024-09-11T00:04:10.845136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:26:05.650618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":30,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T00:06:20.209741+0000","last_change":"2024-09-11T00:04:11.868140+0000","last_active":"2024-09-11T00:06:20.209741+0000","last_peered":"2024-09-11T00:06:20.209741+0000","last_clean":"2024-09-11T00:06:20.209741+0000","last_became_active":"2024-09-11T00:04:11.866459+0000","last_became_peered":"2024-09-11T00:04:11.866459+0000","last_unstale":"2024-09-11T00:06:20.209741+0000","last_undegraded":"2024-09-11T00:06:20.209741+0000","last_fullsized":"2024-09-11T00:06:20.209741+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_clean_scrub_stamp":"2024-09-11T00:04:10.845136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:53:25.106039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"83'11","reported_seq":48,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T00:06:20.217686+0000","last_change":"2024-09-11T00:04:03.829839+0000","last_active":"2024-09-11T00:06:20.217686+0000","last_peered":"2024-09-11T00:06:20.217686+0000","last_clean":"2024-09-11T00:06:20.217686+0000","last_became_active":"2024-09-11T00:04:03.829553+0000","last_became_peered":"2024-09-11T00:04:03.829553+0000","last_unstale":"2024-09-11T00:06:20.217686+0000","last_undegraded":"2024-09-11T00:06:20.217686+0000","last_fullsized":"2024-09-11T00:06:20.217686+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:45:58.561389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"81'3","reported_seq":46,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T00:06:20.218461+0000","last_change":"2024-09-11T00:04:01.820204+0000","last_active":"2024-09-11T00:06:20.218461+0000","last_peered":"2024-09-11T00:06:20.218461+0000","last_clean":"2024-09-11T00:06:20.218461+0000","last_became_active":"2024-09-11T00:04:01.819797+0000","last_became_peered":"2024-09-11T00:04:01.819797+0000","last_unstale":"2024-09-11T00:06:20.218461+0000","last_undegraded":"2024-09-11T00:06:20.218461+0000","last_fullsized":"2024-09-11T00:06:20.218461+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_clean_scrub_stamp":"2024-09-11T00:04:00.794959+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:56:10.530023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":30,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T00:06:20.218728+0000","last_change":"2024-09-11T00:04:11.879430+0000","last_active":"2024-09-11T00:06:20.218728+0000","last_peered":"2024-09-11T00:06:20.218728+0000","last_clean":"2024-09-11T00:06:20.218728+0000","last_became_active":"2024-09-11T00:04:11.879010+0000","last_became_peered":"2024-09-11T00:04:11.879010+0000","last_unstale":"2024-09-11T00:06:20.218728+0000","last_undegraded":"2024-09-11T00:06:20.218728+0000","last_fullsized":"2024-09-11T00:06:20.218728+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_clean_scrub_stamp":"2024-09-11T00:04:10.845136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:57.737125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"83'8","reported_seq":45,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T00:06:20.223218+0000","last_change":"2024-09-11T00:04:03.843918+0000","last_active":"2024-09-11T00:06:20.223218+0000","last_peered":"2024-09-11T00:06:20.223218+0000","last_clean":"2024-09-11T00:06:20.223218+0000","last_became_active":"2024-09-11T00:04:03.843591+0000","last_became_peered":"2024-09-11T00:04:03.843591+0000","last_unstale":"2024-09-11T00:06:20.223218+0000","last_undegraded":"2024-09-11T00:06:20.223218+0000","last_fullsized":"2024-09-11T00:06:20.223218+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:26:41.228802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"81'3","reported_seq":45,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T00:06:20.223687+0000","last_change":"2024-09-11T00:04:01.813173+0000","last_active":"2024-09-11T00:06:20.223687+0000","last_peered":"2024-09-11T00:06:20.223687+0000","last_clean":"2024-09-11T00:06:20.223687+0000","last_became_active":"2024-09-11T00:04:01.811799+0000","last_became_peered":"2024-09-11T00:04:01.811799+0000","last_unstale":"2024-09-11T00:06:20.223687+0000","last_undegraded":"2024-09-11T00:06:20.223687+0000","last_fullsized":"2024-09-11T00:06:20.223687+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_clean_scrub_stamp":"2024-09-11T00:04:00.794959+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:16:47.304371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"81'7","reported_seq":49,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T00:06:20.224176+0000","last_change":"2024-09-11T00:04:01.818649+0000","last_active":"2024-09-11T00:06:20.224176+0000","last_peered":"2024-09-11T00:06:20.224176+0000","last_clean":"2024-09-11T00:06:20.224176+0000","last_became_active":"2024-09-11T00:04:01.818287+0000","last_became_peered":"2024-09-11T00:04:01.818287+0000","last_unstale":"2024-09-11T00:06:20.224176+0000","last_undegraded":"2024-09-11T00:06:20.224176+0000","last_fullsized":"2024-09-11T00:06:20.224176+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_clean_scrub_stamp":"2024-09-11T00:04:00.794959+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:42:00.764648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"83'8","reported_seq":45,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T00:06:20.218123+0000","last_change":"2024-09-11T00:04:03.832829+0000","last_active":"2024-09-11T00:06:20.218123+0000","last_peered":"2024-09-11T00:06:20.218123+0000","last_clean":"2024-09-11T00:06:20.218123+0000","last_became_active":"2024-09-11T00:04:03.832514+0000","last_became_peered":"2024-09-11T00:04:03.832514+0000","last_unstale":"2024-09-11T00:06:20.218123+0000","last_undegraded":"2024-09-11T00:06:20.218123+0000","last_fullsized":"2024-09-11T00:06:20.218123+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:36.643727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":30,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T00:06:20.208261+0000","last_change":"2024-09-11T00:04:11.877596+0000","last_active":"2024-09-11T00:06:20.208261+0000","last_peered":"2024-09-11T00:06:20.208261+0000","last_clean":"2024-09-11T00:06:20.208261+0000","last_became_active":"2024-09-11T00:04:11.877113+0000","last_became_peered":"2024-09-11T00:04:11.877113+0000","last_unstale":"2024-09-11T00:06:20.208261+0000","last_undegraded":"2024-09-11T00:06:20.208261+0000","last_fullsized":"2024-09-11T00:06:20.208261+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_clean_scrub_stamp":"2024-09-11T00:04:10.845136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:47:11.759687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"81'5","reported_seq":165,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T00:06:44.432852+0000","last_change":"2024-09-11T00:04:01.821519+0000","last_active":"2024-09-11T00:06:44.432852+0000","last_peered":"2024-09-11T00:06:44.432852+0000","last_clean":"2024-09-11T00:06:44.432852+0000","last_became_active":"2024-09-11T00:04:01.821207+0000","last_became_peered":"2024-09-11T00:04:01.821207+0000","last_unstale":"2024-09-11T00:06:44.432852+0000","last_undegraded":"2024-09-11T00:06:44.432852+0000","last_fullsized":"2024-09-11T00:06:44.432852+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_clean_scrub_stamp":"2024-09-11T00:04:00.794959+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:16:34.435040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":118,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"83'7","reported_seq":44,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T00:06:20.215339+0000","last_change":"2024-09-11T00:04:03.832691+0000","last_active":"2024-09-11T00:06:20.215339+0000","last_peered":"2024-09-11T00:06:20.215339+0000","last_clean":"2024-09-11T00:06:20.215339+0000","last_became_active":"2024-09-11T00:04:03.832250+0000","last_became_peered":"2024-09-11T00:04:03.832250+0000","last_unstale":"2024-09-11T00:06:20.215339+0000","last_undegraded":"2024-09-11T00:06:20.215339+0000","last_fullsized":"2024-09-11T00:06:20.215339+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:54.279820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":30,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T00:06:20.207934+0000","last_change":"2024-09-11T00:04:11.864995+0000","last_active":"2024-09-11T00:06:20.207934+0000","last_peered":"2024-09-11T00:06:20.207934+0000","last_clean":"2024-09-11T00:06:20.207934+0000","last_became_active":"2024-09-11T00:04:11.864501+0000","last_became_peered":"2024-09-11T00:04:11.864501+0000","last_unstale":"2024-09-11T00:06:20.207934+0000","last_undegraded":"2024-09-11T00:06:20.207934+0000","last_fullsized":"2024-09-11T00:06:20.207934+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_clean_scrub_stamp":"2024-09-11T00:04:10.845136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:08:08.637714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"83'9","reported_seq":46,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T00:06:20.218400+0000","last_change":"2024-09-11T00:04:03.829310+0000","last_active":"2024-09-11T00:06:20.218400+0000","last_peered":"2024-09-11T00:06:20.218400+0000","last_clean":"2024-09-11T00:06:20.218400+0000","last_became_active":"2024-09-11T00:04:03.828937+0000","last_became_peered":"2024-09-11T00:04:03.828937+0000","last_unstale":"2024-09-11T00:06:20.218400+0000","last_undegraded":"2024-09-11T00:06:20.218400+0000","last_fullsized":"2024-09-11T00:06:20.218400+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:32:58.263067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"83'12","reported_seq":49,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T00:06:20.208296+0000","last_change":"2024-09-11T00:04:03.828055+0000","last_active":"2024-09-11T00:06:20.208296+0000","last_peered":"2024-09-11T00:06:20.208296+0000","last_clean":"2024-09-11T00:06:20.208296+0000","last_became_active":"2024-09-11T00:04:03.827382+0000","last_became_peered":"2024-09-11T00:04:03.827382+0000","last_unstale":"2024-09-11T00:06:20.208296+0000","last_undegraded":"2024-09-11T00:06:20.208296+0000","last_fullsized":"2024-09-11T00:06:20.208296+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:02:47.107689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"83'8","reported_seq":45,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T00:06:20.207747+0000","last_change":"2024-09-11T00:04:03.843549+0000","last_active":"2024-09-11T00:06:20.207747+0000","last_peered":"2024-09-11T00:06:20.207747+0000","last_clean":"2024-09-11T00:06:20.207747+0000","last_became_active":"2024-09-11T00:04:03.843149+0000","last_became_peered":"2024-09-11T00:04:03.843149+0000","last_unstale":"2024-09-11T00:06:20.207747+0000","last_undegraded":"2024-09-11T00:06:20.207747+0000","last_fullsized":"2024-09-11T00:06:20.207747+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:47.288319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"83'7","reported_seq":51,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T00:06:20.209743+0000","last_change":"2024-09-11T00:05:19.844682+0000","last_active":"2024-09-11T00:06:20.209743+0000","last_peered":"2024-09-11T00:06:20.209743+0000","last_clean":"2024-09-11T00:06:20.209743+0000","last_became_active":"2024-09-11T00:05:19.844269+0000","last_became_peered":"2024-09-11T00:05:19.844269+0000","last_unstale":"2024-09-11T00:06:20.209743+0000","last_undegraded":"2024-09-11T00:06:20.209743+0000","last_fullsized":"2024-09-11T00:06:20.209743+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:47:22.001597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,7],"acting":[0,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"83'4","reported_seq":41,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T00:06:20.224595+0000","last_change":"2024-09-11T00:04:03.835068+0000","last_active":"2024-09-11T00:06:20.224595+0000","last_peered":"2024-09-11T00:06:20.224595+0000","last_clean":"2024-09-11T00:06:20.224595+0000","last_became_active":"2024-09-11T00:04:03.833166+0000","last_became_peered":"2024-09-11T00:04:03.833166+0000","last_unstale":"2024-09-11T00:06:20.224595+0000","last_undegraded":"2024-09-11T00:06:20.224595+0000","last_fullsized":"2024-09-11T00:06:20.224595+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:07:06.357597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"83'6","reported_seq":43,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T00:06:20.217982+0000","last_change":"2024-09-11T00:04:03.827310+0000","last_active":"2024-09-11T00:06:20.217982+0000","last_peered":"2024-09-11T00:06:20.217982+0000","last_clean":"2024-09-11T00:06:20.217982+0000","last_became_active":"2024-09-11T00:04:03.826748+0000","last_became_peered":"2024-09-11T00:04:03.826748+0000","last_unstale":"2024-09-11T00:06:20.217982+0000","last_undegraded":"2024-09-11T00:06:20.217982+0000","last_fullsized":"2024-09-11T00:06:20.217982+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:53:15.656708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"83'15","reported_seq":52,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T00:06:20.208245+0000","last_change":"2024-09-11T00:04:03.835900+0000","last_active":"2024-09-11T00:06:20.208245+0000","last_peered":"2024-09-11T00:06:20.208245+0000","last_clean":"2024-09-11T00:06:20.208245+0000","last_became_active":"2024-09-11T00:04:03.835137+0000","last_became_peered":"2024-09-11T00:04:03.835137+0000","last_unstale":"2024-09-11T00:06:20.208245+0000","last_undegraded":"2024-09-11T00:06:20.208245+0000","last_fullsized":"2024-09-11T00:06:20.208245+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:23:31.184833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"83'2","reported_seq":39,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T00:06:20.224137+0000","last_change":"2024-09-11T00:04:03.833028+0000","last_active":"2024-09-11T00:06:20.224137+0000","last_peered":"2024-09-11T00:06:20.224137+0000","last_clean":"2024-09-11T00:06:20.224137+0000","last_became_active":"2024-09-11T00:04:03.832703+0000","last_became_peered":"2024-09-11T00:04:03.832703+0000","last_unstale":"2024-09-11T00:06:20.224137+0000","last_undegraded":"2024-09-11T00:06:20.224137+0000","last_fullsized":"2024-09-11T00:06:20.224137+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:38.115627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"83'6","reported_seq":43,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T00:06:20.217825+0000","last_change":"2024-09-11T00:04:03.832423+0000","last_active":"2024-09-11T00:06:20.217825+0000","last_peered":"2024-09-11T00:06:20.217825+0000","last_clean":"2024-09-11T00:06:20.217825+0000","last_became_active":"2024-09-11T00:04:03.832090+0000","last_became_peered":"2024-09-11T00:04:03.832090+0000","last_unstale":"2024-09-11T00:06:20.217825+0000","last_undegraded":"2024-09-11T00:06:20.217825+0000","last_fullsized":"2024-09-11T00:06:20.217825+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:01:27.258506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"83'8","reported_seq":45,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T00:06:20.223374+0000","last_change":"2024-09-11T00:04:03.842854+0000","last_active":"2024-09-11T00:06:20.223374+0000","last_peered":"2024-09-11T00:06:20.223374+0000","last_clean":"2024-09-11T00:06:20.223374+0000","last_became_active":"2024-09-11T00:04:03.838666+0000","last_became_peered":"2024-09-11T00:04:03.838666+0000","last_unstale":"2024-09-11T00:06:20.223374+0000","last_undegraded":"2024-09-11T00:06:20.223374+0000","last_fullsized":"2024-09-11T00:06:20.223374+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:45:48.575830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"83'10","reported_seq":47,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T00:06:20.207908+0000","last_change":"2024-09-11T00:04:03.834175+0000","last_active":"2024-09-11T00:06:20.207908+0000","last_peered":"2024-09-11T00:06:20.207908+0000","last_clean":"2024-09-11T00:06:20.207908+0000","last_became_active":"2024-09-11T00:04:03.833826+0000","last_became_peered":"2024-09-11T00:04:03.833826+0000","last_unstale":"2024-09-11T00:06:20.207908+0000","last_undegraded":"2024-09-11T00:06:20.207908+0000","last_fullsized":"2024-09-11T00:06:20.207908+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:45:57.323216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"83'11","reported_seq":48,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T00:06:20.224407+0000","last_change":"2024-09-11T00:04:03.836751+0000","last_active":"2024-09-11T00:06:20.224407+0000","last_peered":"2024-09-11T00:06:20.224407+0000","last_clean":"2024-09-11T00:06:20.224407+0000","last_became_active":"2024-09-11T00:04:03.834064+0000","last_became_peered":"2024-09-11T00:04:03.834064+0000","last_unstale":"2024-09-11T00:06:20.224407+0000","last_undegraded":"2024-09-11T00:06:20.224407+0000","last_fullsized":"2024-09-11T00:06:20.224407+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:06:23.059770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"83'8","reported_seq":45,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T00:06:20.218478+0000","last_change":"2024-09-11T00:04:03.830196+0000","last_active":"2024-09-11T00:06:20.218478+0000","last_peered":"2024-09-11T00:06:20.218478+0000","last_clean":"2024-09-11T00:06:20.218478+0000","last_became_active":"2024-09-11T00:04:03.829856+0000","last_became_peered":"2024-09-11T00:04:03.829856+0000","last_unstale":"2024-09-11T00:06:20.218478+0000","last_undegraded":"2024-09-11T00:06:20.218478+0000","last_fullsized":"2024-09-11T00:06:20.218478+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:34:47.489316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"83'7","reported_seq":44,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T00:06:20.208071+0000","last_change":"2024-09-11T00:04:03.835053+0000","last_active":"2024-09-11T00:06:20.208071+0000","last_peered":"2024-09-11T00:06:20.208071+0000","last_clean":"2024-09-11T00:06:20.208071+0000","last_became_active":"2024-09-11T00:04:03.834736+0000","last_became_peered":"2024-09-11T00:04:03.834736+0000","last_unstale":"2024-09-11T00:06:20.208071+0000","last_undegraded":"2024-09-11T00:06:20.208071+0000","last_fullsized":"2024-09-11T00:06:20.208071+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:53:40.253775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"83'10","reported_seq":47,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T00:06:20.218686+0000","last_change":"2024-09-11T00:04:03.834514+0000","last_active":"2024-09-11T00:06:20.218686+0000","last_peered":"2024-09-11T00:06:20.218686+0000","last_clean":"2024-09-11T00:06:20.218686+0000","last_became_active":"2024-09-11T00:04:03.834185+0000","last_became_peered":"2024-09-11T00:04:03.834185+0000","last_unstale":"2024-09-11T00:06:20.218686+0000","last_undegraded":"2024-09-11T00:06:20.218686+0000","last_fullsized":"2024-09-11T00:06:20.218686+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:33:52.949710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"83'8","reported_seq":45,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T00:06:20.219019+0000","last_change":"2024-09-11T00:04:03.840371+0000","last_active":"2024-09-11T00:06:20.219019+0000","last_peered":"2024-09-11T00:06:20.219019+0000","last_clean":"2024-09-11T00:06:20.219019+0000","last_became_active":"2024-09-11T00:04:03.839933+0000","last_became_peered":"2024-09-11T00:04:03.839933+0000","last_unstale":"2024-09-11T00:06:20.219019+0000","last_undegraded":"2024-09-11T00:06:20.219019+0000","last_fullsized":"2024-09-11T00:06:20.219019+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:24:53.890837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"83'9","reported_seq":46,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T00:06:20.207585+0000","last_change":"2024-09-11T00:04:03.838386+0000","last_active":"2024-09-11T00:06:20.207585+0000","last_peered":"2024-09-11T00:06:20.207585+0000","last_clean":"2024-09-11T00:06:20.207585+0000","last_became_active":"2024-09-11T00:04:03.837511+0000","last_became_peered":"2024-09-11T00:04:03.837511+0000","last_unstale":"2024-09-11T00:06:20.207585+0000","last_undegraded":"2024-09-11T00:06:20.207585+0000","last_fullsized":"2024-09-11T00:06:20.207585+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:27.399727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"83'7","reported_seq":51,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T00:06:20.209348+0000","last_change":"2024-09-11T00:05:19.848001+0000","last_active":"2024-09-11T00:06:20.209348+0000","last_peered":"2024-09-11T00:06:20.209348+0000","last_clean":"2024-09-11T00:06:20.209348+0000","last_became_active":"2024-09-11T00:05:19.847175+0000","last_became_peered":"2024-09-11T00:05:19.847175+0000","last_unstale":"2024-09-11T00:06:20.209348+0000","last_undegraded":"2024-09-11T00:06:20.209348+0000","last_fullsized":"2024-09-11T00:06:20.209348+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:06:13.765740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,2],"acting":[0,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"83'7","reported_seq":44,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T00:06:20.209526+0000","last_change":"2024-09-11T00:04:03.832618+0000","last_active":"2024-09-11T00:06:20.209526+0000","last_peered":"2024-09-11T00:06:20.209526+0000","last_clean":"2024-09-11T00:06:20.209526+0000","last_became_active":"2024-09-11T00:04:03.831821+0000","last_became_peered":"2024-09-11T00:04:03.831821+0000","last_unstale":"2024-09-11T00:06:20.209526+0000","last_undegraded":"2024-09-11T00:06:20.209526+0000","last_fullsized":"2024-09-11T00:06:20.209526+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:40:40.402422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"83'10","reported_seq":47,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T00:06:20.224352+0000","last_change":"2024-09-11T00:04:03.825348+0000","last_active":"2024-09-11T00:06:20.224352+0000","last_peered":"2024-09-11T00:06:20.224352+0000","last_clean":"2024-09-11T00:06:20.224352+0000","last_became_active":"2024-09-11T00:04:03.824977+0000","last_became_peered":"2024-09-11T00:04:03.824977+0000","last_unstale":"2024-09-11T00:06:20.224352+0000","last_undegraded":"2024-09-11T00:06:20.224352+0000","last_fullsized":"2024-09-11T00:06:20.224352+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:19:53.412969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"83'7","reported_seq":23,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T00:06:20.217748+0000","last_change":"2024-09-11T00:05:19.983677+0000","last_active":"2024-09-11T00:06:20.217748+0000","last_peered":"2024-09-11T00:06:20.217748+0000","last_clean":"2024-09-11T00:06:20.217748+0000","last_became_active":"2024-09-11T00:05:19.983404+0000","last_became_peered":"2024-09-11T00:05:19.983404+0000","last_unstale":"2024-09-11T00:06:20.217748+0000","last_undegraded":"2024-09-11T00:06:20.217748+0000","last_fullsized":"2024-09-11T00:06:20.217748+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:58:11.624190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,10,8],"acting":[9,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.35","version":"83'6","reported_seq":43,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T00:06:20.214987+0000","last_change":"2024-09-11T00:04:03.840068+0000","last_active":"2024-09-11T00:06:20.214987+0000","last_peered":"2024-09-11T00:06:20.214987+0000","last_clean":"2024-09-11T00:06:20.214987+0000","last_became_active":"2024-09-11T00:04:03.835185+0000","last_became_peered":"2024-09-11T00:04:03.835185+0000","last_unstale":"2024-09-11T00:06:20.214987+0000","last_undegraded":"2024-09-11T00:06:20.214987+0000","last_fullsized":"2024-09-11T00:06:20.214987+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:19:10.699038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"83'16","reported_seq":53,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T00:06:20.224682+0000","last_change":"2024-09-11T00:04:03.828576+0000","last_active":"2024-09-11T00:06:20.224682+0000","last_peered":"2024-09-11T00:06:20.224682+0000","last_clean":"2024-09-11T00:06:20.224682+0000","last_became_active":"2024-09-11T00:04:03.828255+0000","last_became_peered":"2024-09-11T00:04:03.828255+0000","last_unstale":"2024-09-11T00:06:20.224682+0000","last_undegraded":"2024-09-11T00:06:20.224682+0000","last_fullsized":"2024-09-11T00:06:20.224682+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:29:54.164145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"83'11","reported_seq":48,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T00:06:20.208306+0000","last_change":"2024-09-11T00:04:03.841452+0000","last_active":"2024-09-11T00:06:20.208306+0000","last_peered":"2024-09-11T00:06:20.208306+0000","last_clean":"2024-09-11T00:06:20.208306+0000","last_became_active":"2024-09-11T00:04:03.840911+0000","last_became_peered":"2024-09-11T00:04:03.840911+0000","last_unstale":"2024-09-11T00:06:20.208306+0000","last_undegraded":"2024-09-11T00:06:20.208306+0000","last_fullsized":"2024-09-11T00:06:20.208306+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:56:40.121922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"83'9","reported_seq":46,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T00:06:20.223208+0000","last_change":"2024-09-11T00:04:03.842009+0000","last_active":"2024-09-11T00:06:20.223208+0000","last_peered":"2024-09-11T00:06:20.223208+0000","last_clean":"2024-09-11T00:06:20.223208+0000","last_became_active":"2024-09-11T00:04:03.841625+0000","last_became_peered":"2024-09-11T00:04:03.841625+0000","last_unstale":"2024-09-11T00:06:20.223208+0000","last_undegraded":"2024-09-11T00:06:20.223208+0000","last_fullsized":"2024-09-11T00:06:20.223208+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:15:11.855308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"83'5","reported_seq":49,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T00:06:20.214617+0000","last_change":"2024-09-11T00:05:19.845891+0000","last_active":"2024-09-11T00:06:20.214617+0000","last_peered":"2024-09-11T00:06:20.214617+0000","last_clean":"2024-09-11T00:06:20.214617+0000","last_became_active":"2024-09-11T00:05:19.845547+0000","last_became_peered":"2024-09-11T00:05:19.845547+0000","last_unstale":"2024-09-11T00:06:20.214617+0000","last_undegraded":"2024-09-11T00:06:20.214617+0000","last_fullsized":"2024-09-11T00:06:20.214617+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:02:20.178667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"83'6","reported_seq":43,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T00:06:20.224142+0000","last_change":"2024-09-11T00:04:03.834540+0000","last_active":"2024-09-11T00:06:20.224142+0000","last_peered":"2024-09-11T00:06:20.224142+0000","last_clean":"2024-09-11T00:06:20.224142+0000","last_became_active":"2024-09-11T00:04:03.834230+0000","last_became_peered":"2024-09-11T00:04:03.834230+0000","last_unstale":"2024-09-11T00:06:20.224142+0000","last_undegraded":"2024-09-11T00:06:20.224142+0000","last_fullsized":"2024-09-11T00:06:20.224142+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:14:26.125003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"83'4","reported_seq":41,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T00:06:20.215459+0000","last_change":"2024-09-11T00:04:03.836177+0000","last_active":"2024-09-11T00:06:20.215459+0000","last_peered":"2024-09-11T00:06:20.215459+0000","last_clean":"2024-09-11T00:06:20.215459+0000","last_became_active":"2024-09-11T00:04:03.835848+0000","last_became_peered":"2024-09-11T00:04:03.835848+0000","last_unstale":"2024-09-11T00:06:20.215459+0000","last_undegraded":"2024-09-11T00:06:20.215459+0000","last_fullsized":"2024-09-11T00:06:20.215459+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:57:53.854738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"83'8","reported_seq":45,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T00:06:20.210324+0000","last_change":"2024-09-11T00:04:03.837614+0000","last_active":"2024-09-11T00:06:20.210324+0000","last_peered":"2024-09-11T00:06:20.210324+0000","last_clean":"2024-09-11T00:06:20.210324+0000","last_became_active":"2024-09-11T00:04:03.837164+0000","last_became_peered":"2024-09-11T00:04:03.837164+0000","last_unstale":"2024-09-11T00:06:20.210324+0000","last_undegraded":"2024-09-11T00:06:20.210324+0000","last_fullsized":"2024-09-11T00:06:20.210324+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:10:38.467648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3d","version":"83'7","reported_seq":44,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T00:06:20.209452+0000","last_change":"2024-09-11T00:04:03.833999+0000","last_active":"2024-09-11T00:06:20.209452+0000","last_peered":"2024-09-11T00:06:20.209452+0000","last_clean":"2024-09-11T00:06:20.209452+0000","last_became_active":"2024-09-11T00:04:03.833580+0000","last_became_peered":"2024-09-11T00:04:03.833580+0000","last_unstale":"2024-09-11T00:06:20.209452+0000","last_undegraded":"2024-09-11T00:06:20.209452+0000","last_fullsized":"2024-09-11T00:06:20.209452+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:47:40.902423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"83'4","reported_seq":41,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T00:06:20.209745+0000","last_change":"2024-09-11T00:04:03.832478+0000","last_active":"2024-09-11T00:06:20.209745+0000","last_peered":"2024-09-11T00:06:20.209745+0000","last_clean":"2024-09-11T00:06:20.209745+0000","last_became_active":"2024-09-11T00:04:03.831811+0000","last_became_peered":"2024-09-11T00:04:03.831811+0000","last_unstale":"2024-09-11T00:06:20.209745+0000","last_undegraded":"2024-09-11T00:06:20.209745+0000","last_fullsized":"2024-09-11T00:06:20.209745+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:53:05.839006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"83'8","reported_seq":45,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T00:06:20.219013+0000","last_change":"2024-09-11T00:04:03.837077+0000","last_active":"2024-09-11T00:06:20.219013+0000","last_peered":"2024-09-11T00:06:20.219013+0000","last_clean":"2024-09-11T00:06:20.219013+0000","last_became_active":"2024-09-11T00:04:03.835718+0000","last_became_peered":"2024-09-11T00:04:03.835718+0000","last_unstale":"2024-09-11T00:06:20.219013+0000","last_undegraded":"2024-09-11T00:06:20.219013+0000","last_fullsized":"2024-09-11T00:06:20.219013+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:17:00.633257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":72975,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":1201,"num_read_kb":681,"num_write":264,"num_write_kb":156,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":577536,"data_stored":218925,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":228,"ondisk_log_size":228,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":68,"seq":292057776175,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36320,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708832,"statfs":{"total":95995035648,"available":95957843968,"internally_reserved":0,"allocated":991232,"data_stored":487430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":36174276},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":63,"seq":270582939700,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28668,"kb_used_data":1644,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716484,"statfs":{"total":95995035648,"available":95965679616,"internally_reserved":0,"allocated":1683456,"data_stored":1143187,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":58,"seq":249108103223,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36336,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708816,"statfs":{"total":95995035648,"available":95957827584,"internally_reserved":0,"allocated":1007616,"data_stored":486716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":52,"seq":223338299455,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":40476,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":93704676,"statfs":{"total":95995035648,"available":95953588224,"internally_reserved":0,"allocated":995328,"data_stored":486982,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":47,"seq":201863462978,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28596,"kb_used_data":1564,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716556,"statfs":{"total":95995035648,"available":95965753344,"internally_reserved":0,"allocated":1601536,"data_stored":1084043,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":42,"seq":180388626505,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32152,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713000,"statfs":{"total":95995035648,"available":95962112000,"internally_reserved":0,"allocated":983040,"data_stored":482752,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":36,"seq":154618822734,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28604,"kb_used_data":1572,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716548,"statfs":{"total":95995035648,"available":95965745152,"internally_reserved":0,"allocated":1609728,"data_stored":1078782,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986259,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27992,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717160,"statfs":{"total":95995035648,"available":95966371840,"internally_reserved":0,"allocated":991232,"data_stored":486605,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182489,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32156,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712996,"statfs":{"total":95995035648,"available":95962107904,"internally_reserved":0,"allocated":995328,"data_stored":481928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31914440},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411423,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28076,"kb_used_data":1048,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717076,"statfs":{"total":95995035648,"available":95966285824,"internally_reserved":0,"allocated":1073152,"data_stored":548869,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574947,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36376,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708776,"statfs":{"total":95995035648,"available":95957786624,"internally_reserved":0,"allocated":1052672,"data_stored":548649,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738473,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32172,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712980,"statfs":{"total":95995035648,"available":95962091520,"internally_reserved":0,"allocated":1007616,"data_stored":484208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2710,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":67462,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":67682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1011,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":2820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3985,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1388,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":118784,"data_stored":67225,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1836,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T00:06:49.439 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-09-11T00:06:49.439 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid ff3ad74a-6fcf-11ef-bcea-c7b262605968 -- ceph pg dump --format=json 2024-09-11T00:06:49.581 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:06:49 smithi178 ceph-mon[35919]: pgmap v390: 129 pgs: 129 active+clean; 648 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 426 B/s wr, 8 op/s 2024-09-11T00:06:49.581 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:06:49 smithi178 ceph-mon[35919]: from='client.25373 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T00:06:49.782 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:06:49 smithi165 ceph-mon[35356]: pgmap v390: 129 pgs: 129 active+clean; 648 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 426 B/s wr, 8 op/s 2024-09-11T00:06:49.782 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:06:49 smithi165 ceph-mon[35356]: from='client.25373 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T00:06:49.818 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:06:50.387 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:50.385+0000 7fce67200640 1 -- 172.21.15.12:0/1407303666 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fce6810f200 msgr2=0x7fce681115f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:06:50.387 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:50.385+0000 7fce67200640 1 --2- 172.21.15.12:0/1407303666 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fce6810f200 0x7fce681115f0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fce60009ca0 tx=0x7fce6002a8c0 comp rx=0 tx=0).stop 2024-09-11T00:06:50.388 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:50.385+0000 7fce67200640 1 -- 172.21.15.12:0/1407303666 shutdown_connections 2024-09-11T00:06:50.388 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:50.385+0000 7fce67200640 1 --2- 172.21.15.12:0/1407303666 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fce6810f200 0x7fce681115f0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:50.388 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:50.385+0000 7fce67200640 1 --2- 172.21.15.12:0/1407303666 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fce68075e00 0x7fce6810ecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:50.388 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:50.385+0000 7fce67200640 1 --2- 172.21.15.12:0/1407303666 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fce680754e0 0x7fce680758c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:50.388 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:50.385+0000 7fce67200640 1 -- 172.21.15.12:0/1407303666 >> 172.21.15.12:0/1407303666 conn(0x7fce680fe660 msgr2=0x7fce68100a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:06:50.388 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:50.385+0000 7fce67200640 1 -- 172.21.15.12:0/1407303666 shutdown_connections 2024-09-11T00:06:50.388 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:50.385+0000 7fce67200640 1 -- 172.21.15.12:0/1407303666 wait complete. 2024-09-11T00:06:50.389 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:50.386+0000 7fce67200640 1 Processor -- start 2024-09-11T00:06:50.389 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:50.386+0000 7fce67200640 1 -- start start 2024-09-11T00:06:50.389 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:50.387+0000 7fce67200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fce680754e0 0x7fce6819b280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:06:50.389 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:50.387+0000 7fce67200640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fce68075e00 0x7fce6819b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:06:50.389 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:50.387+0000 7fce67200640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fce6810f200 0x7fce681a0b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:06:50.389 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:50.387+0000 7fce67200640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fce68114e30 con 0x7fce6810f200 2024-09-11T00:06:50.389 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:50.387+0000 7fce67200640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fce68114cb0 con 0x7fce680754e0 2024-09-11T00:06:50.390 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:50.387+0000 7fce67200640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fce68114fb0 con 0x7fce68075e00 2024-09-11T00:06:50.390 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:50.387+0000 7fce66800640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fce6810f200 0x7fce681a0b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:06:50.390 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:50.387+0000 7fce65e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fce680754e0 0x7fce6819b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:06:50.390 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:50.387+0000 7fce65e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fce680754e0 0x7fce6819b280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.12:58240/0 (socket says 172.21.15.12:58240) 2024-09-11T00:06:50.390 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:50.387+0000 7fce5fe00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fce68075e00 0x7fce6819b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:06:50.390 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:50.387+0000 7fce65e00640 1 -- 172.21.15.12:0/3045407133 learned_addr learned my addr 172.21.15.12:0/3045407133 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:06:50.391 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:50.387+0000 7fce65e00640 1 -- 172.21.15.12:0/3045407133 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fce68075e00 msgr2=0x7fce6819b7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:06:50.391 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:50.387+0000 7fce65e00640 1 --2- 172.21.15.12:0/3045407133 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fce68075e00 0x7fce6819b7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:50.391 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:50.387+0000 7fce65e00640 1 -- 172.21.15.12:0/3045407133 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fce6810f200 msgr2=0x7fce681a0b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:06:50.391 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:50.387+0000 7fce65e00640 1 --2- 172.21.15.12:0/3045407133 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fce6810f200 0x7fce681a0b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:50.391 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:50.387+0000 7fce65e00640 1 -- 172.21.15.12:0/3045407133 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce681a1330 con 0x7fce680754e0 2024-09-11T00:06:50.391 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:50.388+0000 7fce65e00640 1 --2- 172.21.15.12:0/3045407133 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fce680754e0 0x7fce6819b280 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fce5800d950 tx=0x7fce5800de10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:06:50.391 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:50.388+0000 7fce5ea00640 1 -- 172.21.15.12:0/3045407133 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce58017070 con 0x7fce680754e0 2024-09-11T00:06:50.391 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:50.388+0000 7fce5ea00640 1 -- 172.21.15.12:0/3045407133 <== mon.2 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fce58009c30 con 0x7fce680754e0 2024-09-11T00:06:50.392 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:50.388+0000 7fce5ea00640 1 -- 172.21.15.12:0/3045407133 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce5800f620 con 0x7fce680754e0 2024-09-11T00:06:50.392 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:50.388+0000 7fce67200640 1 -- 172.21.15.12:0/3045407133 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fce681a1620 con 0x7fce680754e0 2024-09-11T00:06:50.392 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:50.388+0000 7fce67200640 1 -- 172.21.15.12:0/3045407133 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fce681a7ce0 con 0x7fce680754e0 2024-09-11T00:06:50.392 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:50.389+0000 7fce67200640 1 -- 172.21.15.12:0/3045407133 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fce680758c0 con 0x7fce680754e0 2024-09-11T00:06:50.393 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:50.391+0000 7fce5ea00640 1 -- 172.21.15.12:0/3045407133 <== mon.2 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7fce58010460 con 0x7fce680754e0 2024-09-11T00:06:50.394 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:50.393+0000 7fce5ea00640 1 --2- 172.21.15.12:0/3045407133 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fce4c0b5390 0x7fce4c0b7850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:06:50.395 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:50.393+0000 7fce5ea00640 1 -- 172.21.15.12:0/3045407133 <== mon.2 v2:172.21.15.165:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9208+0+0 (secure 0 0 0) 0x7fce580e5bf0 con 0x7fce680754e0 2024-09-11T00:06:50.395 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:50.393+0000 7fce5fe00640 1 --2- 172.21.15.12:0/3045407133 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fce4c0b5390 0x7fce4c0b7850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:06:50.395 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:50.394+0000 7fce5fe00640 1 --2- 172.21.15.12:0/3045407133 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fce4c0b5390 0x7fce4c0b7850 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fce68100e00 tx=0x7fce5000b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:06:50.398 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:50.397+0000 7fce5ea00640 1 -- 172.21.15.12:0/3045407133 <== mon.2 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7fce58009de0 con 0x7fce680754e0 2024-09-11T00:06:50.589 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:50.587+0000 7fce67200640 1 -- 172.21.15.12:0/3045407133 --> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fce68063590 con 0x7fce4c0b5390 2024-09-11T00:06:50.595 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:50.593+0000 7fce5ea00640 1 -- 172.21.15.12:0/3045407133 <== mgr.14150 v2:172.21.15.12:6800/3442939137 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340533 (secure 0 0 0) 0x7fce68063590 con 0x7fce4c0b5390 2024-09-11T00:06:50.601 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-11T00:06:50.603 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-11T00:06:50.605 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:50.598+0000 7fce67200640 1 -- 172.21.15.12:0/3045407133 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fce4c0b5390 msgr2=0x7fce4c0b7850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:06:50.605 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:50.598+0000 7fce67200640 1 --2- 172.21.15.12:0/3045407133 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fce4c0b5390 0x7fce4c0b7850 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fce68100e00 tx=0x7fce5000b390 comp rx=0 tx=0).stop 2024-09-11T00:06:50.605 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:50.598+0000 7fce67200640 1 -- 172.21.15.12:0/3045407133 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fce680754e0 msgr2=0x7fce6819b280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:06:50.605 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:50.598+0000 7fce67200640 1 --2- 172.21.15.12:0/3045407133 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fce680754e0 0x7fce6819b280 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fce5800d950 tx=0x7fce5800de10 comp rx=0 tx=0).stop 2024-09-11T00:06:50.605 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:50.598+0000 7fce67200640 1 -- 172.21.15.12:0/3045407133 shutdown_connections 2024-09-11T00:06:50.605 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:50.598+0000 7fce67200640 1 --2- 172.21.15.12:0/3045407133 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fce4c0b5390 0x7fce4c0b7850 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:50.605 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:50.598+0000 7fce67200640 1 --2- 172.21.15.12:0/3045407133 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fce6810f200 0x7fce681a0b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:50.605 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:50.598+0000 7fce67200640 1 --2- 172.21.15.12:0/3045407133 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fce68075e00 0x7fce6819b7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:50.605 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:50.598+0000 7fce67200640 1 --2- 172.21.15.12:0/3045407133 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fce680754e0 0x7fce6819b280 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:50.606 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:50.598+0000 7fce67200640 1 -- 172.21.15.12:0/3045407133 >> 172.21.15.12:0/3045407133 conn(0x7fce680fe660 msgr2=0x7fce68106c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:06:50.606 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:50.598+0000 7fce67200640 1 -- 172.21.15.12:0/3045407133 shutdown_connections 2024-09-11T00:06:50.606 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:50.598+0000 7fce67200640 1 -- 172.21.15.12:0/3045407133 wait complete. 2024-09-11T00:06:51.293 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":391,"stamp":"2024-09-11T00:06:50.133508+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":663577,"num_objects":102,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":1297,"num_read_kb":755,"num_write":391,"num_write_kb":2309,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":24,"num_bytes_recovered":2952069,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":823,"ondisk_log_size":823,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":387924,"kb_used_data":13672,"kb_used_omap":18,"kb_used_meta":373997,"kb_avail":1124553900,"statfs":{"total":1151940427776,"available":1151543193600,"internally_reserved":0,"allocated":14000128,"data_stored":7809401,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19142,"internal_metadata":382973242},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":622,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":122,"num_read_kb":62,"num_write":8,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003174"},"pg_stats":[{"pgid":"2.1f","version":"83'14","reported_seq":176,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T00:06:47.435978+0000","last_change":"2024-09-11T00:04:01.820872+0000","last_active":"2024-09-11T00:06:47.435978+0000","last_peered":"2024-09-11T00:06:47.435978+0000","last_clean":"2024-09-11T00:06:47.435978+0000","last_became_active":"2024-09-11T00:04:01.820404+0000","last_became_peered":"2024-09-11T00:04:01.820404+0000","last_unstale":"2024-09-11T00:06:47.435978+0000","last_undegraded":"2024-09-11T00:06:47.435978+0000","last_fullsized":"2024-09-11T00:06:47.435978+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_clean_scrub_stamp":"2024-09-11T00:04:00.794959+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:15:38.648289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":120,"num_read_kb":120,"num_write":15,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"83'5","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T00:06:20.218414+0000","last_change":"2024-09-11T00:05:19.846978+0000","last_active":"2024-09-11T00:06:20.218414+0000","last_peered":"2024-09-11T00:06:20.218414+0000","last_clean":"2024-09-11T00:06:20.218414+0000","last_became_active":"2024-09-11T00:05:19.846569+0000","last_became_peered":"2024-09-11T00:05:19.846569+0000","last_unstale":"2024-09-11T00:06:20.218414+0000","last_undegraded":"2024-09-11T00:06:20.218414+0000","last_fullsized":"2024-09-11T00:06:20.218414+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:27:46.235552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,5],"acting":[9,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":30,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T00:06:20.208028+0000","last_change":"2024-09-11T00:04:11.872109+0000","last_active":"2024-09-11T00:06:20.208028+0000","last_peered":"2024-09-11T00:06:20.208028+0000","last_clean":"2024-09-11T00:06:20.208028+0000","last_became_active":"2024-09-11T00:04:11.871279+0000","last_became_peered":"2024-09-11T00:04:11.871279+0000","last_unstale":"2024-09-11T00:06:20.208028+0000","last_undegraded":"2024-09-11T00:06:20.208028+0000","last_fullsized":"2024-09-11T00:06:20.208028+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_clean_scrub_stamp":"2024-09-11T00:04:10.845136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:13:32.837555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"81'2","reported_seq":44,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T00:06:20.224031+0000","last_change":"2024-09-11T00:04:01.942288+0000","last_active":"2024-09-11T00:06:20.224031+0000","last_peered":"2024-09-11T00:06:20.224031+0000","last_clean":"2024-09-11T00:06:20.224031+0000","last_became_active":"2024-09-11T00:04:01.941838+0000","last_became_peered":"2024-09-11T00:04:01.941838+0000","last_unstale":"2024-09-11T00:06:20.224031+0000","last_undegraded":"2024-09-11T00:06:20.224031+0000","last_fullsized":"2024-09-11T00:06:20.224031+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_clean_scrub_stamp":"2024-09-11T00:04:00.794959+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:48:44.867267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"83'6","reported_seq":43,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T00:06:20.209960+0000","last_change":"2024-09-11T00:04:03.839951+0000","last_active":"2024-09-11T00:06:20.209960+0000","last_peered":"2024-09-11T00:06:20.209960+0000","last_clean":"2024-09-11T00:06:20.209960+0000","last_became_active":"2024-09-11T00:04:03.839591+0000","last_became_peered":"2024-09-11T00:04:03.839591+0000","last_unstale":"2024-09-11T00:06:20.209960+0000","last_undegraded":"2024-09-11T00:06:20.209960+0000","last_fullsized":"2024-09-11T00:06:20.209960+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:12:49.603310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":30,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T00:06:20.210052+0000","last_change":"2024-09-11T00:04:11.872026+0000","last_active":"2024-09-11T00:06:20.210052+0000","last_peered":"2024-09-11T00:06:20.210052+0000","last_clean":"2024-09-11T00:06:20.210052+0000","last_became_active":"2024-09-11T00:04:11.871699+0000","last_became_peered":"2024-09-11T00:04:11.871699+0000","last_unstale":"2024-09-11T00:06:20.210052+0000","last_undegraded":"2024-09-11T00:06:20.210052+0000","last_fullsized":"2024-09-11T00:06:20.210052+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_clean_scrub_stamp":"2024-09-11T00:04:10.845136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:07:47.757223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"81'12","reported_seq":55,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T00:06:20.215846+0000","last_change":"2024-09-11T00:04:01.819942+0000","last_active":"2024-09-11T00:06:20.215846+0000","last_peered":"2024-09-11T00:06:20.215846+0000","last_clean":"2024-09-11T00:06:20.215846+0000","last_became_active":"2024-09-11T00:04:01.819337+0000","last_became_peered":"2024-09-11T00:04:01.819337+0000","last_unstale":"2024-09-11T00:06:20.215846+0000","last_undegraded":"2024-09-11T00:06:20.215846+0000","last_fullsized":"2024-09-11T00:06:20.215846+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_clean_scrub_stamp":"2024-09-11T00:04:00.794959+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:22:26.947067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"83'9","reported_seq":46,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T00:06:20.207881+0000","last_change":"2024-09-11T00:04:03.839276+0000","last_active":"2024-09-11T00:06:20.207881+0000","last_peered":"2024-09-11T00:06:20.207881+0000","last_clean":"2024-09-11T00:06:20.207881+0000","last_became_active":"2024-09-11T00:04:03.838843+0000","last_became_peered":"2024-09-11T00:04:03.838843+0000","last_unstale":"2024-09-11T00:06:20.207881+0000","last_undegraded":"2024-09-11T00:06:20.207881+0000","last_fullsized":"2024-09-11T00:06:20.207881+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:26:42.640628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":30,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T00:06:20.210814+0000","last_change":"2024-09-11T00:04:11.872345+0000","last_active":"2024-09-11T00:06:20.210814+0000","last_peered":"2024-09-11T00:06:20.210814+0000","last_clean":"2024-09-11T00:06:20.210814+0000","last_became_active":"2024-09-11T00:04:11.871994+0000","last_became_peered":"2024-09-11T00:04:11.871994+0000","last_unstale":"2024-09-11T00:06:20.210814+0000","last_undegraded":"2024-09-11T00:06:20.210814+0000","last_fullsized":"2024-09-11T00:06:20.210814+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_clean_scrub_stamp":"2024-09-11T00:04:10.845136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:14:23.906778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"81'5","reported_seq":48,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T00:06:20.215746+0000","last_change":"2024-09-11T00:04:01.819581+0000","last_active":"2024-09-11T00:06:20.215746+0000","last_peered":"2024-09-11T00:06:20.215746+0000","last_clean":"2024-09-11T00:06:20.215746+0000","last_became_active":"2024-09-11T00:04:01.818699+0000","last_became_peered":"2024-09-11T00:04:01.818699+0000","last_unstale":"2024-09-11T00:06:20.215746+0000","last_undegraded":"2024-09-11T00:06:20.215746+0000","last_fullsized":"2024-09-11T00:06:20.215746+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_clean_scrub_stamp":"2024-09-11T00:04:00.794959+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:07:24.540825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"83'15","reported_seq":52,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T00:06:20.218131+0000","last_change":"2024-09-11T00:04:03.835107+0000","last_active":"2024-09-11T00:06:20.218131+0000","last_peered":"2024-09-11T00:06:20.218131+0000","last_clean":"2024-09-11T00:06:20.218131+0000","last_became_active":"2024-09-11T00:04:03.834782+0000","last_became_peered":"2024-09-11T00:04:03.834782+0000","last_unstale":"2024-09-11T00:06:20.218131+0000","last_undegraded":"2024-09-11T00:06:20.218131+0000","last_fullsized":"2024-09-11T00:06:20.218131+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:46:54.472270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":30,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T00:06:20.209753+0000","last_change":"2024-09-11T00:04:11.874168+0000","last_active":"2024-09-11T00:06:20.209753+0000","last_peered":"2024-09-11T00:06:20.209753+0000","last_clean":"2024-09-11T00:06:20.209753+0000","last_became_active":"2024-09-11T00:04:11.873810+0000","last_became_peered":"2024-09-11T00:04:11.873810+0000","last_unstale":"2024-09-11T00:06:20.209753+0000","last_undegraded":"2024-09-11T00:06:20.209753+0000","last_fullsized":"2024-09-11T00:06:20.209753+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_clean_scrub_stamp":"2024-09-11T00:04:10.845136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:06:05.130815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"83'5","reported_seq":42,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T00:06:20.214579+0000","last_change":"2024-09-11T00:04:03.834016+0000","last_active":"2024-09-11T00:06:20.214579+0000","last_peered":"2024-09-11T00:06:20.214579+0000","last_clean":"2024-09-11T00:06:20.214579+0000","last_became_active":"2024-09-11T00:04:03.833374+0000","last_became_peered":"2024-09-11T00:04:03.833374+0000","last_unstale":"2024-09-11T00:06:20.214579+0000","last_undegraded":"2024-09-11T00:06:20.214579+0000","last_fullsized":"2024-09-11T00:06:20.214579+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:13:34.068594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"81'4","reported_seq":47,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T00:06:20.215423+0000","last_change":"2024-09-11T00:04:01.821173+0000","last_active":"2024-09-11T00:06:20.215423+0000","last_peered":"2024-09-11T00:06:20.215423+0000","last_clean":"2024-09-11T00:06:20.215423+0000","last_became_active":"2024-09-11T00:04:01.820615+0000","last_became_peered":"2024-09-11T00:04:01.820615+0000","last_unstale":"2024-09-11T00:06:20.215423+0000","last_undegraded":"2024-09-11T00:06:20.215423+0000","last_fullsized":"2024-09-11T00:06:20.215423+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_clean_scrub_stamp":"2024-09-11T00:04:00.794959+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:22:11.842837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":30,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T00:06:20.210190+0000","last_change":"2024-09-11T00:04:12.077637+0000","last_active":"2024-09-11T00:06:20.210190+0000","last_peered":"2024-09-11T00:06:20.210190+0000","last_clean":"2024-09-11T00:06:20.210190+0000","last_became_active":"2024-09-11T00:04:12.076450+0000","last_became_peered":"2024-09-11T00:04:12.076450+0000","last_unstale":"2024-09-11T00:06:20.210190+0000","last_undegraded":"2024-09-11T00:06:20.210190+0000","last_fullsized":"2024-09-11T00:06:20.210190+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_clean_scrub_stamp":"2024-09-11T00:04:10.845136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:50:54.870468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"83'8","reported_seq":45,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T00:06:20.223570+0000","last_change":"2024-09-11T00:04:03.846398+0000","last_active":"2024-09-11T00:06:20.223570+0000","last_peered":"2024-09-11T00:06:20.223570+0000","last_clean":"2024-09-11T00:06:20.223570+0000","last_became_active":"2024-09-11T00:04:03.846051+0000","last_became_peered":"2024-09-11T00:04:03.846051+0000","last_unstale":"2024-09-11T00:06:20.223570+0000","last_undegraded":"2024-09-11T00:06:20.223570+0000","last_fullsized":"2024-09-11T00:06:20.223570+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:00:38.506122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"81'3","reported_seq":22,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T00:06:20.217264+0000","last_change":"2024-09-11T00:05:19.860048+0000","last_active":"2024-09-11T00:06:20.217264+0000","last_peered":"2024-09-11T00:06:20.217264+0000","last_clean":"2024-09-11T00:06:20.217264+0000","last_became_active":"2024-09-11T00:05:19.845210+0000","last_became_peered":"2024-09-11T00:05:19.845210+0000","last_unstale":"2024-09-11T00:06:20.217264+0000","last_undegraded":"2024-09-11T00:06:20.217264+0000","last_fullsized":"2024-09-11T00:06:20.217264+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_clean_scrub_stamp":"2024-09-11T00:04:00.794959+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:55:18.935136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"81'1","reported_seq":31,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T00:06:20.210206+0000","last_change":"2024-09-11T00:04:11.879582+0000","last_active":"2024-09-11T00:06:20.210206+0000","last_peered":"2024-09-11T00:06:20.210206+0000","last_clean":"2024-09-11T00:06:20.210206+0000","last_became_active":"2024-09-11T00:04:11.879098+0000","last_became_peered":"2024-09-11T00:04:11.879098+0000","last_unstale":"2024-09-11T00:06:20.210206+0000","last_undegraded":"2024-09-11T00:06:20.210206+0000","last_fullsized":"2024-09-11T00:06:20.210206+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_clean_scrub_stamp":"2024-09-11T00:04:10.845136+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:06.292841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"83'9","reported_seq":46,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T00:06:20.218519+0000","last_change":"2024-09-11T00:04:03.834615+0000","last_active":"2024-09-11T00:06:20.218519+0000","last_peered":"2024-09-11T00:06:20.218519+0000","last_clean":"2024-09-11T00:06:20.218519+0000","last_became_active":"2024-09-11T00:04:03.834284+0000","last_became_peered":"2024-09-11T00:04:03.834284+0000","last_unstale":"2024-09-11T00:06:20.218519+0000","last_undegraded":"2024-09-11T00:06:20.218519+0000","last_fullsized":"2024-09-11T00:06:20.218519+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:24:09.542774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"81'7","reported_seq":49,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T00:06:20.223445+0000","last_change":"2024-09-11T00:04:01.818156+0000","last_active":"2024-09-11T00:06:20.223445+0000","last_peered":"2024-09-11T00:06:20.223445+0000","last_clean":"2024-09-11T00:06:20.223445+0000","last_became_active":"2024-09-11T00:04:01.817820+0000","last_became_peered":"2024-09-11T00:04:01.817820+0000","last_unstale":"2024-09-11T00:06:20.223445+0000","last_undegraded":"2024-09-11T00:06:20.223445+0000","last_fullsized":"2024-09-11T00:06:20.223445+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_clean_scrub_stamp":"2024-09-11T00:04:00.794959+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:15:46.304276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":30,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T00:06:20.218603+0000","last_change":"2024-09-11T00:04:11.877704+0000","last_active":"2024-09-11T00:06:20.218603+0000","last_peered":"2024-09-11T00:06:20.218603+0000","last_clean":"2024-09-11T00:06:20.218603+0000","last_became_active":"2024-09-11T00:04:11.877320+0000","last_became_peered":"2024-09-11T00:04:11.877320+0000","last_unstale":"2024-09-11T00:06:20.218603+0000","last_undegraded":"2024-09-11T00:06:20.218603+0000","last_fullsized":"2024-09-11T00:06:20.218603+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_clean_scrub_stamp":"2024-09-11T00:04:10.845136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:31:35.046012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.9","version":"83'2","reported_seq":39,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T00:06:20.215128+0000","last_change":"2024-09-11T00:04:03.828240+0000","last_active":"2024-09-11T00:06:20.215128+0000","last_peered":"2024-09-11T00:06:20.215128+0000","last_clean":"2024-09-11T00:06:20.215128+0000","last_became_active":"2024-09-11T00:04:03.827890+0000","last_became_peered":"2024-09-11T00:04:03.827890+0000","last_unstale":"2024-09-11T00:06:20.215128+0000","last_undegraded":"2024-09-11T00:06:20.215128+0000","last_fullsized":"2024-09-11T00:06:20.215128+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:43:12.146717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"81'2","reported_seq":44,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T00:06:20.207648+0000","last_change":"2024-09-11T00:04:01.942805+0000","last_active":"2024-09-11T00:06:20.207648+0000","last_peered":"2024-09-11T00:06:20.207648+0000","last_clean":"2024-09-11T00:06:20.207648+0000","last_became_active":"2024-09-11T00:04:01.942429+0000","last_became_peered":"2024-09-11T00:04:01.942429+0000","last_unstale":"2024-09-11T00:06:20.207648+0000","last_undegraded":"2024-09-11T00:06:20.207648+0000","last_fullsized":"2024-09-11T00:06:20.207648+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_clean_scrub_stamp":"2024-09-11T00:04:00.794959+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:51:22.403825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":30,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T00:06:20.210426+0000","last_change":"2024-09-11T00:04:12.077246+0000","last_active":"2024-09-11T00:06:20.210426+0000","last_peered":"2024-09-11T00:06:20.210426+0000","last_clean":"2024-09-11T00:06:20.210426+0000","last_became_active":"2024-09-11T00:04:12.076481+0000","last_became_peered":"2024-09-11T00:04:12.076481+0000","last_unstale":"2024-09-11T00:06:20.210426+0000","last_undegraded":"2024-09-11T00:06:20.210426+0000","last_fullsized":"2024-09-11T00:06:20.210426+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_clean_scrub_stamp":"2024-09-11T00:04:10.845136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:55:54.347334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"83'7","reported_seq":44,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T00:06:20.209811+0000","last_change":"2024-09-11T00:04:03.838431+0000","last_active":"2024-09-11T00:06:20.209811+0000","last_peered":"2024-09-11T00:06:20.209811+0000","last_clean":"2024-09-11T00:06:20.209811+0000","last_became_active":"2024-09-11T00:04:03.834451+0000","last_became_peered":"2024-09-11T00:04:03.834451+0000","last_unstale":"2024-09-11T00:06:20.209811+0000","last_undegraded":"2024-09-11T00:06:20.209811+0000","last_fullsized":"2024-09-11T00:06:20.209811+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:23:01.594299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"81'7","reported_seq":50,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T00:06:20.210677+0000","last_change":"2024-09-11T00:04:02.588834+0000","last_active":"2024-09-11T00:06:20.210677+0000","last_peered":"2024-09-11T00:06:20.210677+0000","last_clean":"2024-09-11T00:06:20.210677+0000","last_became_active":"2024-09-11T00:04:02.588477+0000","last_became_peered":"2024-09-11T00:04:02.588477+0000","last_unstale":"2024-09-11T00:06:20.210677+0000","last_undegraded":"2024-09-11T00:06:20.210677+0000","last_fullsized":"2024-09-11T00:06:20.210677+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_clean_scrub_stamp":"2024-09-11T00:04:00.794959+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:31:21.805591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":30,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T00:06:20.207762+0000","last_change":"2024-09-11T00:04:11.863381+0000","last_active":"2024-09-11T00:06:20.207762+0000","last_peered":"2024-09-11T00:06:20.207762+0000","last_clean":"2024-09-11T00:06:20.207762+0000","last_became_active":"2024-09-11T00:04:11.862944+0000","last_became_peered":"2024-09-11T00:04:11.862944+0000","last_unstale":"2024-09-11T00:06:20.207762+0000","last_undegraded":"2024-09-11T00:06:20.207762+0000","last_fullsized":"2024-09-11T00:06:20.207762+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_clean_scrub_stamp":"2024-09-11T00:04:10.845136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:00:14.185992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"83'5","reported_seq":42,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T00:06:20.224159+0000","last_change":"2024-09-11T00:04:03.829750+0000","last_active":"2024-09-11T00:06:20.224159+0000","last_peered":"2024-09-11T00:06:20.224159+0000","last_clean":"2024-09-11T00:06:20.224159+0000","last_became_active":"2024-09-11T00:04:03.828166+0000","last_became_peered":"2024-09-11T00:04:03.828166+0000","last_unstale":"2024-09-11T00:06:20.224159+0000","last_undegraded":"2024-09-11T00:06:20.224159+0000","last_fullsized":"2024-09-11T00:06:20.224159+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:55:01.667075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"85'17","reported_seq":59,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T00:06:20.218751+0000","last_change":"2024-09-11T00:04:01.818083+0000","last_active":"2024-09-11T00:06:20.218751+0000","last_peered":"2024-09-11T00:06:20.218751+0000","last_clean":"2024-09-11T00:06:20.218751+0000","last_became_active":"2024-09-11T00:04:01.817784+0000","last_became_peered":"2024-09-11T00:04:01.817784+0000","last_unstale":"2024-09-11T00:06:20.218751+0000","last_undegraded":"2024-09-11T00:06:20.218751+0000","last_fullsized":"2024-09-11T00:06:20.218751+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_clean_scrub_stamp":"2024-09-11T00:04:00.794959+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:38:55.466913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":30,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":30,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T00:06:20.223043+0000","last_change":"2024-09-11T00:04:12.077053+0000","last_active":"2024-09-11T00:06:20.223043+0000","last_peered":"2024-09-11T00:06:20.223043+0000","last_clean":"2024-09-11T00:06:20.223043+0000","last_became_active":"2024-09-11T00:04:12.076632+0000","last_became_peered":"2024-09-11T00:04:12.076632+0000","last_unstale":"2024-09-11T00:06:20.223043+0000","last_undegraded":"2024-09-11T00:06:20.223043+0000","last_fullsized":"2024-09-11T00:06:20.223043+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_clean_scrub_stamp":"2024-09-11T00:04:10.845136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:19:38.321429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"83'9","reported_seq":46,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T00:06:20.224608+0000","last_change":"2024-09-11T00:04:03.830316+0000","last_active":"2024-09-11T00:06:20.224608+0000","last_peered":"2024-09-11T00:06:20.224608+0000","last_clean":"2024-09-11T00:06:20.224608+0000","last_became_active":"2024-09-11T00:04:03.829932+0000","last_became_peered":"2024-09-11T00:04:03.829932+0000","last_unstale":"2024-09-11T00:06:20.224608+0000","last_undegraded":"2024-09-11T00:06:20.224608+0000","last_fullsized":"2024-09-11T00:06:20.224608+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:06:14.546651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"81'4","reported_seq":47,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T00:06:20.215888+0000","last_change":"2024-09-11T00:04:01.943004+0000","last_active":"2024-09-11T00:06:20.215888+0000","last_peered":"2024-09-11T00:06:20.215888+0000","last_clean":"2024-09-11T00:06:20.215888+0000","last_became_active":"2024-09-11T00:04:01.942432+0000","last_became_peered":"2024-09-11T00:04:01.942432+0000","last_unstale":"2024-09-11T00:06:20.215888+0000","last_undegraded":"2024-09-11T00:06:20.215888+0000","last_fullsized":"2024-09-11T00:06:20.215888+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_clean_scrub_stamp":"2024-09-11T00:04:00.794959+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:52:50.472596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":30,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T00:06:20.218250+0000","last_change":"2024-09-11T00:04:11.866778+0000","last_active":"2024-09-11T00:06:20.218250+0000","last_peered":"2024-09-11T00:06:20.218250+0000","last_clean":"2024-09-11T00:06:20.218250+0000","last_became_active":"2024-09-11T00:04:11.866333+0000","last_became_peered":"2024-09-11T00:04:11.866333+0000","last_unstale":"2024-09-11T00:06:20.218250+0000","last_undegraded":"2024-09-11T00:06:20.218250+0000","last_fullsized":"2024-09-11T00:06:20.218250+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_clean_scrub_stamp":"2024-09-11T00:04:10.845136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:57:33.680119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"83'4","reported_seq":41,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T00:06:20.224266+0000","last_change":"2024-09-11T00:04:03.832481+0000","last_active":"2024-09-11T00:06:20.224266+0000","last_peered":"2024-09-11T00:06:20.224266+0000","last_clean":"2024-09-11T00:06:20.224266+0000","last_became_active":"2024-09-11T00:04:03.832162+0000","last_became_peered":"2024-09-11T00:04:03.832162+0000","last_unstale":"2024-09-11T00:06:20.224266+0000","last_undegraded":"2024-09-11T00:06:20.224266+0000","last_fullsized":"2024-09-11T00:06:20.224266+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:32:52.550128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"81'1","reported_seq":43,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T00:06:20.218249+0000","last_change":"2024-09-11T00:04:01.943647+0000","last_active":"2024-09-11T00:06:20.218249+0000","last_peered":"2024-09-11T00:06:20.218249+0000","last_clean":"2024-09-11T00:06:20.218249+0000","last_became_active":"2024-09-11T00:04:01.942714+0000","last_became_peered":"2024-09-11T00:04:01.942714+0000","last_unstale":"2024-09-11T00:06:20.218249+0000","last_undegraded":"2024-09-11T00:06:20.218249+0000","last_fullsized":"2024-09-11T00:06:20.218249+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_clean_scrub_stamp":"2024-09-11T00:04:00.794959+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:22:23.687268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":30,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T00:06:20.215230+0000","last_change":"2024-09-11T00:04:12.077133+0000","last_active":"2024-09-11T00:06:20.215230+0000","last_peered":"2024-09-11T00:06:20.215230+0000","last_clean":"2024-09-11T00:06:20.215230+0000","last_became_active":"2024-09-11T00:04:12.076657+0000","last_became_peered":"2024-09-11T00:04:12.076657+0000","last_unstale":"2024-09-11T00:06:20.215230+0000","last_undegraded":"2024-09-11T00:06:20.215230+0000","last_fullsized":"2024-09-11T00:06:20.215230+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_clean_scrub_stamp":"2024-09-11T00:04:10.845136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:05:40.643405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"83'7","reported_seq":44,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T00:06:20.224150+0000","last_change":"2024-09-11T00:04:03.835454+0000","last_active":"2024-09-11T00:06:20.224150+0000","last_peered":"2024-09-11T00:06:20.224150+0000","last_clean":"2024-09-11T00:06:20.224150+0000","last_became_active":"2024-09-11T00:04:03.835122+0000","last_became_peered":"2024-09-11T00:04:03.835122+0000","last_unstale":"2024-09-11T00:06:20.224150+0000","last_undegraded":"2024-09-11T00:06:20.224150+0000","last_fullsized":"2024-09-11T00:06:20.224150+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:25:49.672766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"83'7","reported_seq":168,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T00:06:44.432221+0000","last_change":"2024-09-11T00:04:01.823738+0000","last_active":"2024-09-11T00:06:44.432221+0000","last_peered":"2024-09-11T00:06:44.432221+0000","last_clean":"2024-09-11T00:06:44.432221+0000","last_became_active":"2024-09-11T00:04:01.823320+0000","last_became_peered":"2024-09-11T00:04:01.823320+0000","last_unstale":"2024-09-11T00:06:44.432221+0000","last_undegraded":"2024-09-11T00:06:44.432221+0000","last_fullsized":"2024-09-11T00:06:44.432221+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_clean_scrub_stamp":"2024-09-11T00:04:00.794959+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:34:30.088454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1120,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":119,"num_read_kb":3,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":30,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T00:06:20.224438+0000","last_change":"2024-09-11T00:04:11.863906+0000","last_active":"2024-09-11T00:06:20.224438+0000","last_peered":"2024-09-11T00:06:20.224438+0000","last_clean":"2024-09-11T00:06:20.224438+0000","last_became_active":"2024-09-11T00:04:11.863571+0000","last_became_peered":"2024-09-11T00:04:11.863571+0000","last_unstale":"2024-09-11T00:06:20.224438+0000","last_undegraded":"2024-09-11T00:06:20.224438+0000","last_fullsized":"2024-09-11T00:06:20.224438+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_clean_scrub_stamp":"2024-09-11T00:04:10.845136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:04:40.288886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"83'8","reported_seq":45,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T00:06:20.217947+0000","last_change":"2024-09-11T00:04:03.826623+0000","last_active":"2024-09-11T00:06:20.217947+0000","last_peered":"2024-09-11T00:06:20.217947+0000","last_clean":"2024-09-11T00:06:20.217947+0000","last_became_active":"2024-09-11T00:04:03.826202+0000","last_became_peered":"2024-09-11T00:04:03.826202+0000","last_unstale":"2024-09-11T00:06:20.217947+0000","last_undegraded":"2024-09-11T00:06:20.217947+0000","last_fullsized":"2024-09-11T00:06:20.217947+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:23:13.222654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"85'6","reported_seq":169,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T00:06:48.438358+0000","last_change":"2024-09-11T00:04:01.942857+0000","last_active":"2024-09-11T00:06:48.438358+0000","last_peered":"2024-09-11T00:06:48.438358+0000","last_clean":"2024-09-11T00:06:48.438358+0000","last_became_active":"2024-09-11T00:04:01.942494+0000","last_became_peered":"2024-09-11T00:04:01.942494+0000","last_unstale":"2024-09-11T00:06:48.438358+0000","last_undegraded":"2024-09-11T00:06:48.438358+0000","last_fullsized":"2024-09-11T00:06:48.438358+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_clean_scrub_stamp":"2024-09-11T00:04:00.794959+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:40:38.587178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":121,"num_read_kb":121,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":30,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T00:06:20.224554+0000","last_change":"2024-09-11T00:04:12.077771+0000","last_active":"2024-09-11T00:06:20.224554+0000","last_peered":"2024-09-11T00:06:20.224554+0000","last_clean":"2024-09-11T00:06:20.224554+0000","last_became_active":"2024-09-11T00:04:12.077366+0000","last_became_peered":"2024-09-11T00:04:12.077366+0000","last_unstale":"2024-09-11T00:06:20.224554+0000","last_undegraded":"2024-09-11T00:06:20.224554+0000","last_fullsized":"2024-09-11T00:06:20.224554+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_clean_scrub_stamp":"2024-09-11T00:04:10.845136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:04:26.808201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"83'5","reported_seq":42,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T00:06:20.223536+0000","last_change":"2024-09-11T00:04:03.827872+0000","last_active":"2024-09-11T00:06:20.223536+0000","last_peered":"2024-09-11T00:06:20.223536+0000","last_clean":"2024-09-11T00:06:20.223536+0000","last_became_active":"2024-09-11T00:04:03.827577+0000","last_became_peered":"2024-09-11T00:04:03.827577+0000","last_unstale":"2024-09-11T00:06:20.223536+0000","last_undegraded":"2024-09-11T00:06:20.223536+0000","last_fullsized":"2024-09-11T00:06:20.223536+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:37:19.205218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"81'4","reported_seq":57,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T00:06:20.214717+0000","last_change":"2024-09-11T00:05:19.991372+0000","last_active":"2024-09-11T00:06:20.214717+0000","last_peered":"2024-09-11T00:06:20.214717+0000","last_clean":"2024-09-11T00:06:20.214717+0000","last_became_active":"2024-09-11T00:05:19.985328+0000","last_became_peered":"2024-09-11T00:05:19.985328+0000","last_unstale":"2024-09-11T00:06:20.214717+0000","last_undegraded":"2024-09-11T00:06:20.214717+0000","last_fullsized":"2024-09-11T00:06:20.214717+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_clean_scrub_stamp":"2024-09-11T00:04:00.794959+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:17:20.345615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,4],"acting":[7,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T00:06:20.226394+0000","last_change":"2024-09-11T00:06:20.225938+0000","last_active":"2024-09-11T00:06:20.226394+0000","last_peered":"2024-09-11T00:06:20.226394+0000","last_clean":"2024-09-11T00:06:20.226394+0000","last_became_active":"2024-09-11T00:06:20.225447+0000","last_became_peered":"2024-09-11T00:06:20.225447+0000","last_unstale":"2024-09-11T00:06:20.226394+0000","last_undegraded":"2024-09-11T00:06:20.226394+0000","last_fullsized":"2024-09-11T00:06:20.226394+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_clean_scrub_stamp":"2024-09-11T00:04:10.845136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:30:17.498206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,6,10],"acting":[8,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.2","version":"83'8","reported_seq":45,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T00:06:20.223732+0000","last_change":"2024-09-11T00:04:03.839615+0000","last_active":"2024-09-11T00:06:20.223732+0000","last_peered":"2024-09-11T00:06:20.223732+0000","last_clean":"2024-09-11T00:06:20.223732+0000","last_became_active":"2024-09-11T00:04:03.832175+0000","last_became_peered":"2024-09-11T00:04:03.832175+0000","last_unstale":"2024-09-11T00:06:20.223732+0000","last_undegraded":"2024-09-11T00:06:20.223732+0000","last_fullsized":"2024-09-11T00:06:20.223732+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:17:09.975345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"81'6","reported_seq":51,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T00:06:20.210941+0000","last_change":"2024-09-11T00:04:01.817248+0000","last_active":"2024-09-11T00:06:20.210941+0000","last_peered":"2024-09-11T00:06:20.210941+0000","last_clean":"2024-09-11T00:06:20.210941+0000","last_became_active":"2024-09-11T00:04:01.816896+0000","last_became_peered":"2024-09-11T00:04:01.816896+0000","last_unstale":"2024-09-11T00:06:20.210941+0000","last_undegraded":"2024-09-11T00:06:20.210941+0000","last_fullsized":"2024-09-11T00:06:20.210941+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_clean_scrub_stamp":"2024-09-11T00:04:00.794959+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:46:51.641881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'92","reported_seq":114,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T00:06:20.214947+0000","last_change":"2024-09-11T00:02:47.245193+0000","last_active":"2024-09-11T00:06:20.214947+0000","last_peered":"2024-09-11T00:06:20.214947+0000","last_clean":"2024-09-11T00:06:20.214947+0000","last_became_active":"2024-09-11T00:02:47.239773+0000","last_became_peered":"2024-09-11T00:02:47.239773+0000","last_unstale":"2024-09-11T00:06:20.214947+0000","last_undegraded":"2024-09-11T00:06:20.214947+0000","last_fullsized":"2024-09-11T00:06:20.214947+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T23:59:10.637655+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T23:59:10.637655+0000","last_clean_scrub_stamp":"2024-09-10T23:59:10.637655+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:51:52.914831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"81'1","reported_seq":43,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T00:06:20.230099+0000","last_change":"2024-09-11T00:06:20.229604+0000","last_active":"2024-09-11T00:06:20.230099+0000","last_peered":"2024-09-11T00:06:20.230099+0000","last_clean":"2024-09-11T00:06:20.230099+0000","last_became_active":"2024-09-11T00:06:20.225065+0000","last_became_peered":"2024-09-11T00:06:20.225065+0000","last_unstale":"2024-09-11T00:06:20.230099+0000","last_undegraded":"2024-09-11T00:06:20.230099+0000","last_fullsized":"2024-09-11T00:06:20.230099+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_clean_scrub_stamp":"2024-09-11T00:04:10.845136+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:52:41.717530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,10],"acting":[7,5,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"83'7","reported_seq":44,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T00:06:20.215116+0000","last_change":"2024-09-11T00:04:03.834755+0000","last_active":"2024-09-11T00:06:20.215116+0000","last_peered":"2024-09-11T00:06:20.215116+0000","last_clean":"2024-09-11T00:06:20.215116+0000","last_became_active":"2024-09-11T00:04:03.833955+0000","last_became_peered":"2024-09-11T00:04:03.833955+0000","last_unstale":"2024-09-11T00:06:20.215116+0000","last_undegraded":"2024-09-11T00:06:20.215116+0000","last_fullsized":"2024-09-11T00:06:20.215116+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:47:13.863481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"83'7","reported_seq":173,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T00:06:48.438807+0000","last_change":"2024-09-11T00:04:02.589042+0000","last_active":"2024-09-11T00:06:48.438807+0000","last_peered":"2024-09-11T00:06:48.438807+0000","last_clean":"2024-09-11T00:06:48.438807+0000","last_became_active":"2024-09-11T00:04:02.588611+0000","last_became_peered":"2024-09-11T00:04:02.588611+0000","last_unstale":"2024-09-11T00:06:48.438807+0000","last_undegraded":"2024-09-11T00:06:48.438807+0000","last_fullsized":"2024-09-11T00:06:48.438807+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_clean_scrub_stamp":"2024-09-11T00:04:00.794959+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:41:08.257605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1702,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":124,"num_read_kb":4,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":30,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T00:06:20.207731+0000","last_change":"2024-09-11T00:04:11.860715+0000","last_active":"2024-09-11T00:06:20.207731+0000","last_peered":"2024-09-11T00:06:20.207731+0000","last_clean":"2024-09-11T00:06:20.207731+0000","last_became_active":"2024-09-11T00:04:11.860094+0000","last_became_peered":"2024-09-11T00:04:11.860094+0000","last_unstale":"2024-09-11T00:06:20.207731+0000","last_undegraded":"2024-09-11T00:06:20.207731+0000","last_fullsized":"2024-09-11T00:06:20.207731+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_clean_scrub_stamp":"2024-09-11T00:04:10.845136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:25:55.279079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"83'6","reported_seq":43,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T00:06:20.214828+0000","last_change":"2024-09-11T00:04:03.822665+0000","last_active":"2024-09-11T00:06:20.214828+0000","last_peered":"2024-09-11T00:06:20.214828+0000","last_clean":"2024-09-11T00:06:20.214828+0000","last_became_active":"2024-09-11T00:04:03.822333+0000","last_became_peered":"2024-09-11T00:04:03.822333+0000","last_unstale":"2024-09-11T00:06:20.214828+0000","last_undegraded":"2024-09-11T00:06:20.214828+0000","last_fullsized":"2024-09-11T00:06:20.214828+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:02:07.465335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"81'3","reported_seq":46,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T00:06:20.218325+0000","last_change":"2024-09-11T00:04:01.817244+0000","last_active":"2024-09-11T00:06:20.218325+0000","last_peered":"2024-09-11T00:06:20.218325+0000","last_clean":"2024-09-11T00:06:20.218325+0000","last_became_active":"2024-09-11T00:04:01.816846+0000","last_became_peered":"2024-09-11T00:04:01.816846+0000","last_unstale":"2024-09-11T00:06:20.218325+0000","last_undegraded":"2024-09-11T00:06:20.218325+0000","last_fullsized":"2024-09-11T00:06:20.218325+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_clean_scrub_stamp":"2024-09-11T00:04:00.794959+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:09:46.232332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":38,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T00:06:20.217751+0000","last_change":"2024-09-11T00:06:20.217001+0000","last_active":"2024-09-11T00:06:20.217751+0000","last_peered":"2024-09-11T00:06:20.217751+0000","last_clean":"2024-09-11T00:06:20.217751+0000","last_became_active":"2024-09-11T00:06:20.216599+0000","last_became_peered":"2024-09-11T00:06:20.216599+0000","last_unstale":"2024-09-11T00:06:20.217751+0000","last_undegraded":"2024-09-11T00:06:20.217751+0000","last_fullsized":"2024-09-11T00:06:20.217751+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_clean_scrub_stamp":"2024-09-11T00:04:10.845136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:06:42.086069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,1,4],"acting":[0,8,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"83'9","reported_seq":46,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T00:06:20.214846+0000","last_change":"2024-09-11T00:04:03.829600+0000","last_active":"2024-09-11T00:06:20.214846+0000","last_peered":"2024-09-11T00:06:20.214846+0000","last_clean":"2024-09-11T00:06:20.214846+0000","last_became_active":"2024-09-11T00:04:03.829118+0000","last_became_peered":"2024-09-11T00:04:03.829118+0000","last_unstale":"2024-09-11T00:06:20.214846+0000","last_undegraded":"2024-09-11T00:06:20.214846+0000","last_fullsized":"2024-09-11T00:06:20.214846+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:41:07.069816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"81'5","reported_seq":60,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T00:06:20.224623+0000","last_change":"2024-09-11T00:05:19.989379+0000","last_active":"2024-09-11T00:06:20.224623+0000","last_peered":"2024-09-11T00:06:20.224623+0000","last_clean":"2024-09-11T00:06:20.224623+0000","last_became_active":"2024-09-11T00:05:19.984918+0000","last_became_peered":"2024-09-11T00:05:19.984918+0000","last_unstale":"2024-09-11T00:06:20.224623+0000","last_undegraded":"2024-09-11T00:06:20.224623+0000","last_fullsized":"2024-09-11T00:06:20.224623+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_clean_scrub_stamp":"2024-09-11T00:04:00.794959+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:27:52.063256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4],"acting":[6,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":30,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T00:06:20.217466+0000","last_change":"2024-09-11T00:04:11.863812+0000","last_active":"2024-09-11T00:06:20.217466+0000","last_peered":"2024-09-11T00:06:20.217466+0000","last_clean":"2024-09-11T00:06:20.217466+0000","last_became_active":"2024-09-11T00:04:11.863412+0000","last_became_peered":"2024-09-11T00:04:11.863412+0000","last_unstale":"2024-09-11T00:06:20.217466+0000","last_undegraded":"2024-09-11T00:06:20.217466+0000","last_fullsized":"2024-09-11T00:06:20.217466+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_clean_scrub_stamp":"2024-09-11T00:04:10.845136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:12:04.140034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"83'5","reported_seq":49,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T00:06:20.218684+0000","last_change":"2024-09-11T00:05:19.985248+0000","last_active":"2024-09-11T00:06:20.218684+0000","last_peered":"2024-09-11T00:06:20.218684+0000","last_clean":"2024-09-11T00:06:20.218684+0000","last_became_active":"2024-09-11T00:05:19.984921+0000","last_became_peered":"2024-09-11T00:05:19.984921+0000","last_unstale":"2024-09-11T00:06:20.218684+0000","last_undegraded":"2024-09-11T00:06:20.218684+0000","last_fullsized":"2024-09-11T00:06:20.218684+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:16:44.370849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,4],"acting":[11,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"85'3","reported_seq":46,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T00:06:20.216030+0000","last_change":"2024-09-11T00:04:01.814191+0000","last_active":"2024-09-11T00:06:20.216030+0000","last_peered":"2024-09-11T00:06:20.216030+0000","last_clean":"2024-09-11T00:06:20.216030+0000","last_became_active":"2024-09-11T00:04:01.813778+0000","last_became_peered":"2024-09-11T00:04:01.813778+0000","last_unstale":"2024-09-11T00:06:20.216030+0000","last_undegraded":"2024-09-11T00:06:20.216030+0000","last_fullsized":"2024-09-11T00:06:20.216030+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_clean_scrub_stamp":"2024-09-11T00:04:00.794959+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:01:49.044436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":30,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T00:06:20.217607+0000","last_change":"2024-09-11T00:04:11.870281+0000","last_active":"2024-09-11T00:06:20.217607+0000","last_peered":"2024-09-11T00:06:20.217607+0000","last_clean":"2024-09-11T00:06:20.217607+0000","last_became_active":"2024-09-11T00:04:11.869738+0000","last_became_peered":"2024-09-11T00:04:11.869738+0000","last_unstale":"2024-09-11T00:06:20.217607+0000","last_undegraded":"2024-09-11T00:06:20.217607+0000","last_fullsized":"2024-09-11T00:06:20.217607+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_clean_scrub_stamp":"2024-09-11T00:04:10.845136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:04:08.380199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"83'6","reported_seq":43,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T00:06:20.217988+0000","last_change":"2024-09-11T00:04:03.835240+0000","last_active":"2024-09-11T00:06:20.217988+0000","last_peered":"2024-09-11T00:06:20.217988+0000","last_clean":"2024-09-11T00:06:20.217988+0000","last_became_active":"2024-09-11T00:04:03.834910+0000","last_became_peered":"2024-09-11T00:04:03.834910+0000","last_unstale":"2024-09-11T00:06:20.217988+0000","last_undegraded":"2024-09-11T00:06:20.217988+0000","last_fullsized":"2024-09-11T00:06:20.217988+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:30:33.996409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.10","version":"81'17","reported_seq":34,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T00:06:20.218188+0000","last_change":"2024-09-11T00:05:19.856704+0000","last_active":"2024-09-11T00:06:20.218188+0000","last_peered":"2024-09-11T00:06:20.218188+0000","last_clean":"2024-09-11T00:06:20.218188+0000","last_became_active":"2024-09-11T00:05:19.843487+0000","last_became_peered":"2024-09-11T00:05:19.843487+0000","last_unstale":"2024-09-11T00:06:20.218188+0000","last_undegraded":"2024-09-11T00:06:20.218188+0000","last_fullsized":"2024-09-11T00:06:20.218188+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_clean_scrub_stamp":"2024-09-11T00:04:00.794959+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:03:51.375839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":30,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T00:06:20.224599+0000","last_change":"2024-09-11T00:04:11.874306+0000","last_active":"2024-09-11T00:06:20.224599+0000","last_peered":"2024-09-11T00:06:20.224599+0000","last_clean":"2024-09-11T00:06:20.224599+0000","last_became_active":"2024-09-11T00:04:11.873934+0000","last_became_peered":"2024-09-11T00:04:11.873934+0000","last_unstale":"2024-09-11T00:06:20.224599+0000","last_undegraded":"2024-09-11T00:06:20.224599+0000","last_fullsized":"2024-09-11T00:06:20.224599+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_clean_scrub_stamp":"2024-09-11T00:04:10.845136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:19:22.584367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"83'6","reported_seq":43,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T00:06:20.214810+0000","last_change":"2024-09-11T00:04:03.824219+0000","last_active":"2024-09-11T00:06:20.214810+0000","last_peered":"2024-09-11T00:06:20.214810+0000","last_clean":"2024-09-11T00:06:20.214810+0000","last_became_active":"2024-09-11T00:04:03.823877+0000","last_became_peered":"2024-09-11T00:04:03.823877+0000","last_unstale":"2024-09-11T00:06:20.214810+0000","last_undegraded":"2024-09-11T00:06:20.214810+0000","last_fullsized":"2024-09-11T00:06:20.214810+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:04:05.924113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"85'6","reported_seq":171,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T00:06:48.438864+0000","last_change":"2024-09-11T00:04:01.818055+0000","last_active":"2024-09-11T00:06:48.438864+0000","last_peered":"2024-09-11T00:06:48.438864+0000","last_clean":"2024-09-11T00:06:48.438864+0000","last_became_active":"2024-09-11T00:04:01.817483+0000","last_became_peered":"2024-09-11T00:04:01.817483+0000","last_unstale":"2024-09-11T00:06:48.438864+0000","last_undegraded":"2024-09-11T00:06:48.438864+0000","last_fullsized":"2024-09-11T00:06:48.438864+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_clean_scrub_stamp":"2024-09-11T00:04:00.794959+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:16:55.218673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":899,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":124,"num_read_kb":4,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":30,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T00:06:20.218592+0000","last_change":"2024-09-11T00:04:11.875918+0000","last_active":"2024-09-11T00:06:20.218592+0000","last_peered":"2024-09-11T00:06:20.218592+0000","last_clean":"2024-09-11T00:06:20.218592+0000","last_became_active":"2024-09-11T00:04:11.875549+0000","last_became_peered":"2024-09-11T00:04:11.875549+0000","last_unstale":"2024-09-11T00:06:20.218592+0000","last_undegraded":"2024-09-11T00:06:20.218592+0000","last_fullsized":"2024-09-11T00:06:20.218592+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_clean_scrub_stamp":"2024-09-11T00:04:10.845136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:16:48.685899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.13","version":"83'15","reported_seq":52,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T00:06:20.215033+0000","last_change":"2024-09-11T00:04:03.827671+0000","last_active":"2024-09-11T00:06:20.215033+0000","last_peered":"2024-09-11T00:06:20.215033+0000","last_clean":"2024-09-11T00:06:20.215033+0000","last_became_active":"2024-09-11T00:04:03.827341+0000","last_became_peered":"2024-09-11T00:04:03.827341+0000","last_unstale":"2024-09-11T00:06:20.215033+0000","last_undegraded":"2024-09-11T00:06:20.215033+0000","last_fullsized":"2024-09-11T00:06:20.215033+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:56:16.614035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"81'5","reported_seq":48,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T00:06:20.218496+0000","last_change":"2024-09-11T00:04:01.820632+0000","last_active":"2024-09-11T00:06:20.218496+0000","last_peered":"2024-09-11T00:06:20.218496+0000","last_clean":"2024-09-11T00:06:20.218496+0000","last_became_active":"2024-09-11T00:04:01.820273+0000","last_became_peered":"2024-09-11T00:04:01.820273+0000","last_unstale":"2024-09-11T00:06:20.218496+0000","last_undegraded":"2024-09-11T00:06:20.218496+0000","last_fullsized":"2024-09-11T00:06:20.218496+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_clean_scrub_stamp":"2024-09-11T00:04:00.794959+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:34:53.444515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":30,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T00:06:20.218420+0000","last_change":"2024-09-11T00:04:11.881197+0000","last_active":"2024-09-11T00:06:20.218420+0000","last_peered":"2024-09-11T00:06:20.218420+0000","last_clean":"2024-09-11T00:06:20.218420+0000","last_became_active":"2024-09-11T00:04:11.880763+0000","last_became_peered":"2024-09-11T00:04:11.880763+0000","last_unstale":"2024-09-11T00:06:20.218420+0000","last_undegraded":"2024-09-11T00:06:20.218420+0000","last_fullsized":"2024-09-11T00:06:20.218420+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_clean_scrub_stamp":"2024-09-11T00:04:10.845136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:14:01.616095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"83'9","reported_seq":46,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T00:06:20.224080+0000","last_change":"2024-09-11T00:04:03.833470+0000","last_active":"2024-09-11T00:06:20.224080+0000","last_peered":"2024-09-11T00:06:20.224080+0000","last_clean":"2024-09-11T00:06:20.224080+0000","last_became_active":"2024-09-11T00:04:03.833120+0000","last_became_peered":"2024-09-11T00:04:03.833120+0000","last_unstale":"2024-09-11T00:06:20.224080+0000","last_undegraded":"2024-09-11T00:06:20.224080+0000","last_fullsized":"2024-09-11T00:06:20.224080+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:29:53.210230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"81'9","reported_seq":53,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T00:06:20.208085+0000","last_change":"2024-09-11T00:04:01.818081+0000","last_active":"2024-09-11T00:06:20.208085+0000","last_peered":"2024-09-11T00:06:20.208085+0000","last_clean":"2024-09-11T00:06:20.208085+0000","last_became_active":"2024-09-11T00:04:01.817716+0000","last_became_peered":"2024-09-11T00:04:01.817716+0000","last_unstale":"2024-09-11T00:06:20.208085+0000","last_undegraded":"2024-09-11T00:06:20.208085+0000","last_fullsized":"2024-09-11T00:06:20.208085+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_clean_scrub_stamp":"2024-09-11T00:04:00.794959+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:01:32.736319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":30,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T00:06:20.218197+0000","last_change":"2024-09-11T00:04:11.870182+0000","last_active":"2024-09-11T00:06:20.218197+0000","last_peered":"2024-09-11T00:06:20.218197+0000","last_clean":"2024-09-11T00:06:20.218197+0000","last_became_active":"2024-09-11T00:04:11.869744+0000","last_became_peered":"2024-09-11T00:04:11.869744+0000","last_unstale":"2024-09-11T00:06:20.218197+0000","last_undegraded":"2024-09-11T00:06:20.218197+0000","last_fullsized":"2024-09-11T00:06:20.218197+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_clean_scrub_stamp":"2024-09-11T00:04:10.845136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:50:24.070404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"83'6","reported_seq":43,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T00:06:20.215695+0000","last_change":"2024-09-11T00:04:03.828246+0000","last_active":"2024-09-11T00:06:20.215695+0000","last_peered":"2024-09-11T00:06:20.215695+0000","last_clean":"2024-09-11T00:06:20.215695+0000","last_became_active":"2024-09-11T00:04:03.827217+0000","last_became_peered":"2024-09-11T00:04:03.827217+0000","last_unstale":"2024-09-11T00:06:20.215695+0000","last_undegraded":"2024-09-11T00:06:20.215695+0000","last_fullsized":"2024-09-11T00:06:20.215695+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:51:46.018759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"85'41","reported_seq":327,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T00:06:45.434193+0000","last_change":"2024-09-11T00:04:01.819553+0000","last_active":"2024-09-11T00:06:45.434193+0000","last_peered":"2024-09-11T00:06:45.434193+0000","last_clean":"2024-09-11T00:06:45.434193+0000","last_became_active":"2024-09-11T00:04:01.819213+0000","last_became_peered":"2024-09-11T00:04:01.819213+0000","last_unstale":"2024-09-11T00:06:45.434193+0000","last_undegraded":"2024-09-11T00:06:45.434193+0000","last_fullsized":"2024-09-11T00:06:45.434193+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_clean_scrub_stamp":"2024-09-11T00:04:00.794959+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:44:11.039982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":65178,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":244,"num_read_kb":186,"num_write":43,"num_write_kb":84,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":30,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T00:06:20.217633+0000","last_change":"2024-09-11T00:04:11.871730+0000","last_active":"2024-09-11T00:06:20.217633+0000","last_peered":"2024-09-11T00:06:20.217633+0000","last_clean":"2024-09-11T00:06:20.217633+0000","last_became_active":"2024-09-11T00:04:11.870659+0000","last_became_peered":"2024-09-11T00:04:11.870659+0000","last_unstale":"2024-09-11T00:06:20.217633+0000","last_undegraded":"2024-09-11T00:06:20.217633+0000","last_fullsized":"2024-09-11T00:06:20.217633+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_clean_scrub_stamp":"2024-09-11T00:04:10.845136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:48:41.835602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.14","version":"83'11","reported_seq":48,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T00:06:20.208703+0000","last_change":"2024-09-11T00:04:03.838763+0000","last_active":"2024-09-11T00:06:20.208703+0000","last_peered":"2024-09-11T00:06:20.208703+0000","last_clean":"2024-09-11T00:06:20.208703+0000","last_became_active":"2024-09-11T00:04:03.838409+0000","last_became_peered":"2024-09-11T00:04:03.838409+0000","last_unstale":"2024-09-11T00:06:20.208703+0000","last_undegraded":"2024-09-11T00:06:20.208703+0000","last_fullsized":"2024-09-11T00:06:20.208703+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:57:38.647351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"81'5","reported_seq":48,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T00:06:20.218329+0000","last_change":"2024-09-11T00:04:01.943027+0000","last_active":"2024-09-11T00:06:20.218329+0000","last_peered":"2024-09-11T00:06:20.218329+0000","last_clean":"2024-09-11T00:06:20.218329+0000","last_became_active":"2024-09-11T00:04:01.942558+0000","last_became_peered":"2024-09-11T00:04:01.942558+0000","last_unstale":"2024-09-11T00:06:20.218329+0000","last_undegraded":"2024-09-11T00:06:20.218329+0000","last_fullsized":"2024-09-11T00:06:20.218329+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_clean_scrub_stamp":"2024-09-11T00:04:00.794959+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:48:03.776476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":38,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T00:06:20.217115+0000","last_change":"2024-09-11T00:06:20.216571+0000","last_active":"2024-09-11T00:06:20.217115+0000","last_peered":"2024-09-11T00:06:20.217115+0000","last_clean":"2024-09-11T00:06:20.217115+0000","last_became_active":"2024-09-11T00:06:20.216166+0000","last_became_peered":"2024-09-11T00:06:20.216166+0000","last_unstale":"2024-09-11T00:06:20.217115+0000","last_undegraded":"2024-09-11T00:06:20.217115+0000","last_fullsized":"2024-09-11T00:06:20.217115+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_clean_scrub_stamp":"2024-09-11T00:04:10.845136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:58:59.803225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,8],"acting":[4,1,2,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"83'9","reported_seq":46,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T00:06:20.210061+0000","last_change":"2024-09-11T00:04:03.836735+0000","last_active":"2024-09-11T00:06:20.210061+0000","last_peered":"2024-09-11T00:06:20.210061+0000","last_clean":"2024-09-11T00:06:20.210061+0000","last_became_active":"2024-09-11T00:04:03.833299+0000","last_became_peered":"2024-09-11T00:04:03.833299+0000","last_unstale":"2024-09-11T00:06:20.210061+0000","last_undegraded":"2024-09-11T00:06:20.210061+0000","last_fullsized":"2024-09-11T00:06:20.210061+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:32:15.924058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"81'4","reported_seq":46,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T00:06:20.215069+0000","last_change":"2024-09-11T00:04:01.823777+0000","last_active":"2024-09-11T00:06:20.215069+0000","last_peered":"2024-09-11T00:06:20.215069+0000","last_clean":"2024-09-11T00:06:20.215069+0000","last_became_active":"2024-09-11T00:04:01.823368+0000","last_became_peered":"2024-09-11T00:04:01.823368+0000","last_unstale":"2024-09-11T00:06:20.215069+0000","last_undegraded":"2024-09-11T00:06:20.215069+0000","last_fullsized":"2024-09-11T00:06:20.215069+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_clean_scrub_stamp":"2024-09-11T00:04:00.794959+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:05:12.584867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":30,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T00:06:20.217038+0000","last_change":"2024-09-11T00:04:11.868303+0000","last_active":"2024-09-11T00:06:20.217038+0000","last_peered":"2024-09-11T00:06:20.217038+0000","last_clean":"2024-09-11T00:06:20.217038+0000","last_became_active":"2024-09-11T00:04:11.867916+0000","last_became_peered":"2024-09-11T00:04:11.867916+0000","last_unstale":"2024-09-11T00:06:20.217038+0000","last_undegraded":"2024-09-11T00:06:20.217038+0000","last_fullsized":"2024-09-11T00:06:20.217038+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_clean_scrub_stamp":"2024-09-11T00:04:10.845136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:08:29.711763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.16","version":"83'10","reported_seq":47,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T00:06:20.208348+0000","last_change":"2024-09-11T00:04:03.832712+0000","last_active":"2024-09-11T00:06:20.208348+0000","last_peered":"2024-09-11T00:06:20.208348+0000","last_clean":"2024-09-11T00:06:20.208348+0000","last_became_active":"2024-09-11T00:04:03.832295+0000","last_became_peered":"2024-09-11T00:04:03.832295+0000","last_unstale":"2024-09-11T00:06:20.208348+0000","last_undegraded":"2024-09-11T00:06:20.208348+0000","last_fullsized":"2024-09-11T00:06:20.208348+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:40:12.271930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"83'5","reported_seq":279,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T00:06:45.433903+0000","last_change":"2024-09-11T00:04:01.820891+0000","last_active":"2024-09-11T00:06:45.433903+0000","last_peered":"2024-09-11T00:06:45.433903+0000","last_clean":"2024-09-11T00:06:45.433903+0000","last_became_active":"2024-09-11T00:04:01.820528+0000","last_became_peered":"2024-09-11T00:04:01.820528+0000","last_unstale":"2024-09-11T00:06:45.433903+0000","last_undegraded":"2024-09-11T00:06:45.433903+0000","last_fullsized":"2024-09-11T00:06:45.433903+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_clean_scrub_stamp":"2024-09-11T00:04:00.794959+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:23:38.196618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":232,"num_read_kb":232,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":30,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T00:06:20.223208+0000","last_change":"2024-09-11T00:04:11.878864+0000","last_active":"2024-09-11T00:06:20.223208+0000","last_peered":"2024-09-11T00:06:20.223208+0000","last_clean":"2024-09-11T00:06:20.223208+0000","last_became_active":"2024-09-11T00:04:11.876701+0000","last_became_peered":"2024-09-11T00:04:11.876701+0000","last_unstale":"2024-09-11T00:06:20.223208+0000","last_undegraded":"2024-09-11T00:06:20.223208+0000","last_fullsized":"2024-09-11T00:06:20.223208+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_clean_scrub_stamp":"2024-09-11T00:04:10.845136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:26:05.650618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":30,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T00:06:20.209741+0000","last_change":"2024-09-11T00:04:11.868140+0000","last_active":"2024-09-11T00:06:20.209741+0000","last_peered":"2024-09-11T00:06:20.209741+0000","last_clean":"2024-09-11T00:06:20.209741+0000","last_became_active":"2024-09-11T00:04:11.866459+0000","last_became_peered":"2024-09-11T00:04:11.866459+0000","last_unstale":"2024-09-11T00:06:20.209741+0000","last_undegraded":"2024-09-11T00:06:20.209741+0000","last_fullsized":"2024-09-11T00:06:20.209741+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_clean_scrub_stamp":"2024-09-11T00:04:10.845136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:53:25.106039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"83'11","reported_seq":48,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T00:06:20.217686+0000","last_change":"2024-09-11T00:04:03.829839+0000","last_active":"2024-09-11T00:06:20.217686+0000","last_peered":"2024-09-11T00:06:20.217686+0000","last_clean":"2024-09-11T00:06:20.217686+0000","last_became_active":"2024-09-11T00:04:03.829553+0000","last_became_peered":"2024-09-11T00:04:03.829553+0000","last_unstale":"2024-09-11T00:06:20.217686+0000","last_undegraded":"2024-09-11T00:06:20.217686+0000","last_fullsized":"2024-09-11T00:06:20.217686+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:45:58.561389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"81'3","reported_seq":46,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T00:06:20.218461+0000","last_change":"2024-09-11T00:04:01.820204+0000","last_active":"2024-09-11T00:06:20.218461+0000","last_peered":"2024-09-11T00:06:20.218461+0000","last_clean":"2024-09-11T00:06:20.218461+0000","last_became_active":"2024-09-11T00:04:01.819797+0000","last_became_peered":"2024-09-11T00:04:01.819797+0000","last_unstale":"2024-09-11T00:06:20.218461+0000","last_undegraded":"2024-09-11T00:06:20.218461+0000","last_fullsized":"2024-09-11T00:06:20.218461+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_clean_scrub_stamp":"2024-09-11T00:04:00.794959+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:56:10.530023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":30,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T00:06:20.218728+0000","last_change":"2024-09-11T00:04:11.879430+0000","last_active":"2024-09-11T00:06:20.218728+0000","last_peered":"2024-09-11T00:06:20.218728+0000","last_clean":"2024-09-11T00:06:20.218728+0000","last_became_active":"2024-09-11T00:04:11.879010+0000","last_became_peered":"2024-09-11T00:04:11.879010+0000","last_unstale":"2024-09-11T00:06:20.218728+0000","last_undegraded":"2024-09-11T00:06:20.218728+0000","last_fullsized":"2024-09-11T00:06:20.218728+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_clean_scrub_stamp":"2024-09-11T00:04:10.845136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:57.737125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"83'8","reported_seq":45,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T00:06:20.223218+0000","last_change":"2024-09-11T00:04:03.843918+0000","last_active":"2024-09-11T00:06:20.223218+0000","last_peered":"2024-09-11T00:06:20.223218+0000","last_clean":"2024-09-11T00:06:20.223218+0000","last_became_active":"2024-09-11T00:04:03.843591+0000","last_became_peered":"2024-09-11T00:04:03.843591+0000","last_unstale":"2024-09-11T00:06:20.223218+0000","last_undegraded":"2024-09-11T00:06:20.223218+0000","last_fullsized":"2024-09-11T00:06:20.223218+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:26:41.228802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"81'3","reported_seq":45,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T00:06:20.223687+0000","last_change":"2024-09-11T00:04:01.813173+0000","last_active":"2024-09-11T00:06:20.223687+0000","last_peered":"2024-09-11T00:06:20.223687+0000","last_clean":"2024-09-11T00:06:20.223687+0000","last_became_active":"2024-09-11T00:04:01.811799+0000","last_became_peered":"2024-09-11T00:04:01.811799+0000","last_unstale":"2024-09-11T00:06:20.223687+0000","last_undegraded":"2024-09-11T00:06:20.223687+0000","last_fullsized":"2024-09-11T00:06:20.223687+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_clean_scrub_stamp":"2024-09-11T00:04:00.794959+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:16:47.304371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"81'7","reported_seq":49,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T00:06:20.224176+0000","last_change":"2024-09-11T00:04:01.818649+0000","last_active":"2024-09-11T00:06:20.224176+0000","last_peered":"2024-09-11T00:06:20.224176+0000","last_clean":"2024-09-11T00:06:20.224176+0000","last_became_active":"2024-09-11T00:04:01.818287+0000","last_became_peered":"2024-09-11T00:04:01.818287+0000","last_unstale":"2024-09-11T00:06:20.224176+0000","last_undegraded":"2024-09-11T00:06:20.224176+0000","last_fullsized":"2024-09-11T00:06:20.224176+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_clean_scrub_stamp":"2024-09-11T00:04:00.794959+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:42:00.764648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"83'8","reported_seq":45,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T00:06:20.218123+0000","last_change":"2024-09-11T00:04:03.832829+0000","last_active":"2024-09-11T00:06:20.218123+0000","last_peered":"2024-09-11T00:06:20.218123+0000","last_clean":"2024-09-11T00:06:20.218123+0000","last_became_active":"2024-09-11T00:04:03.832514+0000","last_became_peered":"2024-09-11T00:04:03.832514+0000","last_unstale":"2024-09-11T00:06:20.218123+0000","last_undegraded":"2024-09-11T00:06:20.218123+0000","last_fullsized":"2024-09-11T00:06:20.218123+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:36.643727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":30,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T00:06:20.208261+0000","last_change":"2024-09-11T00:04:11.877596+0000","last_active":"2024-09-11T00:06:20.208261+0000","last_peered":"2024-09-11T00:06:20.208261+0000","last_clean":"2024-09-11T00:06:20.208261+0000","last_became_active":"2024-09-11T00:04:11.877113+0000","last_became_peered":"2024-09-11T00:04:11.877113+0000","last_unstale":"2024-09-11T00:06:20.208261+0000","last_undegraded":"2024-09-11T00:06:20.208261+0000","last_fullsized":"2024-09-11T00:06:20.208261+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_clean_scrub_stamp":"2024-09-11T00:04:10.845136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:47:11.759687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"81'5","reported_seq":165,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T00:06:44.432852+0000","last_change":"2024-09-11T00:04:01.821519+0000","last_active":"2024-09-11T00:06:44.432852+0000","last_peered":"2024-09-11T00:06:44.432852+0000","last_clean":"2024-09-11T00:06:44.432852+0000","last_became_active":"2024-09-11T00:04:01.821207+0000","last_became_peered":"2024-09-11T00:04:01.821207+0000","last_unstale":"2024-09-11T00:06:44.432852+0000","last_undegraded":"2024-09-11T00:06:44.432852+0000","last_fullsized":"2024-09-11T00:06:44.432852+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:00.794959+0000","last_clean_scrub_stamp":"2024-09-11T00:04:00.794959+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:16:34.435040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":118,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"83'7","reported_seq":44,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T00:06:20.215339+0000","last_change":"2024-09-11T00:04:03.832691+0000","last_active":"2024-09-11T00:06:20.215339+0000","last_peered":"2024-09-11T00:06:20.215339+0000","last_clean":"2024-09-11T00:06:20.215339+0000","last_became_active":"2024-09-11T00:04:03.832250+0000","last_became_peered":"2024-09-11T00:04:03.832250+0000","last_unstale":"2024-09-11T00:06:20.215339+0000","last_undegraded":"2024-09-11T00:06:20.215339+0000","last_fullsized":"2024-09-11T00:06:20.215339+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:54.279820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":30,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T00:06:20.207934+0000","last_change":"2024-09-11T00:04:11.864995+0000","last_active":"2024-09-11T00:06:20.207934+0000","last_peered":"2024-09-11T00:06:20.207934+0000","last_clean":"2024-09-11T00:06:20.207934+0000","last_became_active":"2024-09-11T00:04:11.864501+0000","last_became_peered":"2024-09-11T00:04:11.864501+0000","last_unstale":"2024-09-11T00:06:20.207934+0000","last_undegraded":"2024-09-11T00:06:20.207934+0000","last_fullsized":"2024-09-11T00:06:20.207934+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:10.845136+0000","last_clean_scrub_stamp":"2024-09-11T00:04:10.845136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:08:08.637714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"83'9","reported_seq":46,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T00:06:20.218400+0000","last_change":"2024-09-11T00:04:03.829310+0000","last_active":"2024-09-11T00:06:20.218400+0000","last_peered":"2024-09-11T00:06:20.218400+0000","last_clean":"2024-09-11T00:06:20.218400+0000","last_became_active":"2024-09-11T00:04:03.828937+0000","last_became_peered":"2024-09-11T00:04:03.828937+0000","last_unstale":"2024-09-11T00:06:20.218400+0000","last_undegraded":"2024-09-11T00:06:20.218400+0000","last_fullsized":"2024-09-11T00:06:20.218400+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:32:58.263067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"83'12","reported_seq":49,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T00:06:20.208296+0000","last_change":"2024-09-11T00:04:03.828055+0000","last_active":"2024-09-11T00:06:20.208296+0000","last_peered":"2024-09-11T00:06:20.208296+0000","last_clean":"2024-09-11T00:06:20.208296+0000","last_became_active":"2024-09-11T00:04:03.827382+0000","last_became_peered":"2024-09-11T00:04:03.827382+0000","last_unstale":"2024-09-11T00:06:20.208296+0000","last_undegraded":"2024-09-11T00:06:20.208296+0000","last_fullsized":"2024-09-11T00:06:20.208296+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:02:47.107689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"83'8","reported_seq":45,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T00:06:20.207747+0000","last_change":"2024-09-11T00:04:03.843549+0000","last_active":"2024-09-11T00:06:20.207747+0000","last_peered":"2024-09-11T00:06:20.207747+0000","last_clean":"2024-09-11T00:06:20.207747+0000","last_became_active":"2024-09-11T00:04:03.843149+0000","last_became_peered":"2024-09-11T00:04:03.843149+0000","last_unstale":"2024-09-11T00:06:20.207747+0000","last_undegraded":"2024-09-11T00:06:20.207747+0000","last_fullsized":"2024-09-11T00:06:20.207747+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:47.288319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"83'7","reported_seq":51,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T00:06:20.209743+0000","last_change":"2024-09-11T00:05:19.844682+0000","last_active":"2024-09-11T00:06:20.209743+0000","last_peered":"2024-09-11T00:06:20.209743+0000","last_clean":"2024-09-11T00:06:20.209743+0000","last_became_active":"2024-09-11T00:05:19.844269+0000","last_became_peered":"2024-09-11T00:05:19.844269+0000","last_unstale":"2024-09-11T00:06:20.209743+0000","last_undegraded":"2024-09-11T00:06:20.209743+0000","last_fullsized":"2024-09-11T00:06:20.209743+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:47:22.001597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,7],"acting":[0,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"83'4","reported_seq":41,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T00:06:20.224595+0000","last_change":"2024-09-11T00:04:03.835068+0000","last_active":"2024-09-11T00:06:20.224595+0000","last_peered":"2024-09-11T00:06:20.224595+0000","last_clean":"2024-09-11T00:06:20.224595+0000","last_became_active":"2024-09-11T00:04:03.833166+0000","last_became_peered":"2024-09-11T00:04:03.833166+0000","last_unstale":"2024-09-11T00:06:20.224595+0000","last_undegraded":"2024-09-11T00:06:20.224595+0000","last_fullsized":"2024-09-11T00:06:20.224595+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:07:06.357597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"83'6","reported_seq":43,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T00:06:20.217982+0000","last_change":"2024-09-11T00:04:03.827310+0000","last_active":"2024-09-11T00:06:20.217982+0000","last_peered":"2024-09-11T00:06:20.217982+0000","last_clean":"2024-09-11T00:06:20.217982+0000","last_became_active":"2024-09-11T00:04:03.826748+0000","last_became_peered":"2024-09-11T00:04:03.826748+0000","last_unstale":"2024-09-11T00:06:20.217982+0000","last_undegraded":"2024-09-11T00:06:20.217982+0000","last_fullsized":"2024-09-11T00:06:20.217982+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:53:15.656708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"83'15","reported_seq":52,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T00:06:20.208245+0000","last_change":"2024-09-11T00:04:03.835900+0000","last_active":"2024-09-11T00:06:20.208245+0000","last_peered":"2024-09-11T00:06:20.208245+0000","last_clean":"2024-09-11T00:06:20.208245+0000","last_became_active":"2024-09-11T00:04:03.835137+0000","last_became_peered":"2024-09-11T00:04:03.835137+0000","last_unstale":"2024-09-11T00:06:20.208245+0000","last_undegraded":"2024-09-11T00:06:20.208245+0000","last_fullsized":"2024-09-11T00:06:20.208245+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:23:31.184833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"83'2","reported_seq":39,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T00:06:20.224137+0000","last_change":"2024-09-11T00:04:03.833028+0000","last_active":"2024-09-11T00:06:20.224137+0000","last_peered":"2024-09-11T00:06:20.224137+0000","last_clean":"2024-09-11T00:06:20.224137+0000","last_became_active":"2024-09-11T00:04:03.832703+0000","last_became_peered":"2024-09-11T00:04:03.832703+0000","last_unstale":"2024-09-11T00:06:20.224137+0000","last_undegraded":"2024-09-11T00:06:20.224137+0000","last_fullsized":"2024-09-11T00:06:20.224137+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:38.115627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"83'6","reported_seq":43,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T00:06:20.217825+0000","last_change":"2024-09-11T00:04:03.832423+0000","last_active":"2024-09-11T00:06:20.217825+0000","last_peered":"2024-09-11T00:06:20.217825+0000","last_clean":"2024-09-11T00:06:20.217825+0000","last_became_active":"2024-09-11T00:04:03.832090+0000","last_became_peered":"2024-09-11T00:04:03.832090+0000","last_unstale":"2024-09-11T00:06:20.217825+0000","last_undegraded":"2024-09-11T00:06:20.217825+0000","last_fullsized":"2024-09-11T00:06:20.217825+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:01:27.258506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"83'8","reported_seq":45,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T00:06:20.223374+0000","last_change":"2024-09-11T00:04:03.842854+0000","last_active":"2024-09-11T00:06:20.223374+0000","last_peered":"2024-09-11T00:06:20.223374+0000","last_clean":"2024-09-11T00:06:20.223374+0000","last_became_active":"2024-09-11T00:04:03.838666+0000","last_became_peered":"2024-09-11T00:04:03.838666+0000","last_unstale":"2024-09-11T00:06:20.223374+0000","last_undegraded":"2024-09-11T00:06:20.223374+0000","last_fullsized":"2024-09-11T00:06:20.223374+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:45:48.575830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"83'10","reported_seq":47,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T00:06:20.207908+0000","last_change":"2024-09-11T00:04:03.834175+0000","last_active":"2024-09-11T00:06:20.207908+0000","last_peered":"2024-09-11T00:06:20.207908+0000","last_clean":"2024-09-11T00:06:20.207908+0000","last_became_active":"2024-09-11T00:04:03.833826+0000","last_became_peered":"2024-09-11T00:04:03.833826+0000","last_unstale":"2024-09-11T00:06:20.207908+0000","last_undegraded":"2024-09-11T00:06:20.207908+0000","last_fullsized":"2024-09-11T00:06:20.207908+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:45:57.323216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"83'11","reported_seq":48,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T00:06:20.224407+0000","last_change":"2024-09-11T00:04:03.836751+0000","last_active":"2024-09-11T00:06:20.224407+0000","last_peered":"2024-09-11T00:06:20.224407+0000","last_clean":"2024-09-11T00:06:20.224407+0000","last_became_active":"2024-09-11T00:04:03.834064+0000","last_became_peered":"2024-09-11T00:04:03.834064+0000","last_unstale":"2024-09-11T00:06:20.224407+0000","last_undegraded":"2024-09-11T00:06:20.224407+0000","last_fullsized":"2024-09-11T00:06:20.224407+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:06:23.059770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"83'8","reported_seq":45,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T00:06:20.218478+0000","last_change":"2024-09-11T00:04:03.830196+0000","last_active":"2024-09-11T00:06:20.218478+0000","last_peered":"2024-09-11T00:06:20.218478+0000","last_clean":"2024-09-11T00:06:20.218478+0000","last_became_active":"2024-09-11T00:04:03.829856+0000","last_became_peered":"2024-09-11T00:04:03.829856+0000","last_unstale":"2024-09-11T00:06:20.218478+0000","last_undegraded":"2024-09-11T00:06:20.218478+0000","last_fullsized":"2024-09-11T00:06:20.218478+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:34:47.489316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"83'7","reported_seq":44,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T00:06:20.208071+0000","last_change":"2024-09-11T00:04:03.835053+0000","last_active":"2024-09-11T00:06:20.208071+0000","last_peered":"2024-09-11T00:06:20.208071+0000","last_clean":"2024-09-11T00:06:20.208071+0000","last_became_active":"2024-09-11T00:04:03.834736+0000","last_became_peered":"2024-09-11T00:04:03.834736+0000","last_unstale":"2024-09-11T00:06:20.208071+0000","last_undegraded":"2024-09-11T00:06:20.208071+0000","last_fullsized":"2024-09-11T00:06:20.208071+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:53:40.253775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"83'10","reported_seq":47,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T00:06:20.218686+0000","last_change":"2024-09-11T00:04:03.834514+0000","last_active":"2024-09-11T00:06:20.218686+0000","last_peered":"2024-09-11T00:06:20.218686+0000","last_clean":"2024-09-11T00:06:20.218686+0000","last_became_active":"2024-09-11T00:04:03.834185+0000","last_became_peered":"2024-09-11T00:04:03.834185+0000","last_unstale":"2024-09-11T00:06:20.218686+0000","last_undegraded":"2024-09-11T00:06:20.218686+0000","last_fullsized":"2024-09-11T00:06:20.218686+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:33:52.949710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"83'8","reported_seq":45,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T00:06:20.219019+0000","last_change":"2024-09-11T00:04:03.840371+0000","last_active":"2024-09-11T00:06:20.219019+0000","last_peered":"2024-09-11T00:06:20.219019+0000","last_clean":"2024-09-11T00:06:20.219019+0000","last_became_active":"2024-09-11T00:04:03.839933+0000","last_became_peered":"2024-09-11T00:04:03.839933+0000","last_unstale":"2024-09-11T00:06:20.219019+0000","last_undegraded":"2024-09-11T00:06:20.219019+0000","last_fullsized":"2024-09-11T00:06:20.219019+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:24:53.890837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"83'9","reported_seq":46,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T00:06:20.207585+0000","last_change":"2024-09-11T00:04:03.838386+0000","last_active":"2024-09-11T00:06:20.207585+0000","last_peered":"2024-09-11T00:06:20.207585+0000","last_clean":"2024-09-11T00:06:20.207585+0000","last_became_active":"2024-09-11T00:04:03.837511+0000","last_became_peered":"2024-09-11T00:04:03.837511+0000","last_unstale":"2024-09-11T00:06:20.207585+0000","last_undegraded":"2024-09-11T00:06:20.207585+0000","last_fullsized":"2024-09-11T00:06:20.207585+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:27.399727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"83'7","reported_seq":51,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T00:06:20.209348+0000","last_change":"2024-09-11T00:05:19.848001+0000","last_active":"2024-09-11T00:06:20.209348+0000","last_peered":"2024-09-11T00:06:20.209348+0000","last_clean":"2024-09-11T00:06:20.209348+0000","last_became_active":"2024-09-11T00:05:19.847175+0000","last_became_peered":"2024-09-11T00:05:19.847175+0000","last_unstale":"2024-09-11T00:06:20.209348+0000","last_undegraded":"2024-09-11T00:06:20.209348+0000","last_fullsized":"2024-09-11T00:06:20.209348+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:06:13.765740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,2],"acting":[0,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"83'7","reported_seq":44,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T00:06:20.209526+0000","last_change":"2024-09-11T00:04:03.832618+0000","last_active":"2024-09-11T00:06:20.209526+0000","last_peered":"2024-09-11T00:06:20.209526+0000","last_clean":"2024-09-11T00:06:20.209526+0000","last_became_active":"2024-09-11T00:04:03.831821+0000","last_became_peered":"2024-09-11T00:04:03.831821+0000","last_unstale":"2024-09-11T00:06:20.209526+0000","last_undegraded":"2024-09-11T00:06:20.209526+0000","last_fullsized":"2024-09-11T00:06:20.209526+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:40:40.402422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"83'10","reported_seq":47,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T00:06:20.224352+0000","last_change":"2024-09-11T00:04:03.825348+0000","last_active":"2024-09-11T00:06:20.224352+0000","last_peered":"2024-09-11T00:06:20.224352+0000","last_clean":"2024-09-11T00:06:20.224352+0000","last_became_active":"2024-09-11T00:04:03.824977+0000","last_became_peered":"2024-09-11T00:04:03.824977+0000","last_unstale":"2024-09-11T00:06:20.224352+0000","last_undegraded":"2024-09-11T00:06:20.224352+0000","last_fullsized":"2024-09-11T00:06:20.224352+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:19:53.412969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"83'7","reported_seq":23,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T00:06:20.217748+0000","last_change":"2024-09-11T00:05:19.983677+0000","last_active":"2024-09-11T00:06:20.217748+0000","last_peered":"2024-09-11T00:06:20.217748+0000","last_clean":"2024-09-11T00:06:20.217748+0000","last_became_active":"2024-09-11T00:05:19.983404+0000","last_became_peered":"2024-09-11T00:05:19.983404+0000","last_unstale":"2024-09-11T00:06:20.217748+0000","last_undegraded":"2024-09-11T00:06:20.217748+0000","last_fullsized":"2024-09-11T00:06:20.217748+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:58:11.624190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,10,8],"acting":[9,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.35","version":"83'6","reported_seq":43,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T00:06:20.214987+0000","last_change":"2024-09-11T00:04:03.840068+0000","last_active":"2024-09-11T00:06:20.214987+0000","last_peered":"2024-09-11T00:06:20.214987+0000","last_clean":"2024-09-11T00:06:20.214987+0000","last_became_active":"2024-09-11T00:04:03.835185+0000","last_became_peered":"2024-09-11T00:04:03.835185+0000","last_unstale":"2024-09-11T00:06:20.214987+0000","last_undegraded":"2024-09-11T00:06:20.214987+0000","last_fullsized":"2024-09-11T00:06:20.214987+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:19:10.699038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"83'16","reported_seq":53,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T00:06:20.224682+0000","last_change":"2024-09-11T00:04:03.828576+0000","last_active":"2024-09-11T00:06:20.224682+0000","last_peered":"2024-09-11T00:06:20.224682+0000","last_clean":"2024-09-11T00:06:20.224682+0000","last_became_active":"2024-09-11T00:04:03.828255+0000","last_became_peered":"2024-09-11T00:04:03.828255+0000","last_unstale":"2024-09-11T00:06:20.224682+0000","last_undegraded":"2024-09-11T00:06:20.224682+0000","last_fullsized":"2024-09-11T00:06:20.224682+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:29:54.164145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"83'11","reported_seq":48,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T00:06:20.208306+0000","last_change":"2024-09-11T00:04:03.841452+0000","last_active":"2024-09-11T00:06:20.208306+0000","last_peered":"2024-09-11T00:06:20.208306+0000","last_clean":"2024-09-11T00:06:20.208306+0000","last_became_active":"2024-09-11T00:04:03.840911+0000","last_became_peered":"2024-09-11T00:04:03.840911+0000","last_unstale":"2024-09-11T00:06:20.208306+0000","last_undegraded":"2024-09-11T00:06:20.208306+0000","last_fullsized":"2024-09-11T00:06:20.208306+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:56:40.121922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"83'9","reported_seq":46,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T00:06:20.223208+0000","last_change":"2024-09-11T00:04:03.842009+0000","last_active":"2024-09-11T00:06:20.223208+0000","last_peered":"2024-09-11T00:06:20.223208+0000","last_clean":"2024-09-11T00:06:20.223208+0000","last_became_active":"2024-09-11T00:04:03.841625+0000","last_became_peered":"2024-09-11T00:04:03.841625+0000","last_unstale":"2024-09-11T00:06:20.223208+0000","last_undegraded":"2024-09-11T00:06:20.223208+0000","last_fullsized":"2024-09-11T00:06:20.223208+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:15:11.855308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"83'5","reported_seq":49,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T00:06:20.214617+0000","last_change":"2024-09-11T00:05:19.845891+0000","last_active":"2024-09-11T00:06:20.214617+0000","last_peered":"2024-09-11T00:06:20.214617+0000","last_clean":"2024-09-11T00:06:20.214617+0000","last_became_active":"2024-09-11T00:05:19.845547+0000","last_became_peered":"2024-09-11T00:05:19.845547+0000","last_unstale":"2024-09-11T00:06:20.214617+0000","last_undegraded":"2024-09-11T00:06:20.214617+0000","last_fullsized":"2024-09-11T00:06:20.214617+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:02:20.178667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"83'6","reported_seq":43,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T00:06:20.224142+0000","last_change":"2024-09-11T00:04:03.834540+0000","last_active":"2024-09-11T00:06:20.224142+0000","last_peered":"2024-09-11T00:06:20.224142+0000","last_clean":"2024-09-11T00:06:20.224142+0000","last_became_active":"2024-09-11T00:04:03.834230+0000","last_became_peered":"2024-09-11T00:04:03.834230+0000","last_unstale":"2024-09-11T00:06:20.224142+0000","last_undegraded":"2024-09-11T00:06:20.224142+0000","last_fullsized":"2024-09-11T00:06:20.224142+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:14:26.125003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"83'4","reported_seq":41,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T00:06:20.215459+0000","last_change":"2024-09-11T00:04:03.836177+0000","last_active":"2024-09-11T00:06:20.215459+0000","last_peered":"2024-09-11T00:06:20.215459+0000","last_clean":"2024-09-11T00:06:20.215459+0000","last_became_active":"2024-09-11T00:04:03.835848+0000","last_became_peered":"2024-09-11T00:04:03.835848+0000","last_unstale":"2024-09-11T00:06:20.215459+0000","last_undegraded":"2024-09-11T00:06:20.215459+0000","last_fullsized":"2024-09-11T00:06:20.215459+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:57:53.854738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"83'8","reported_seq":45,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T00:06:20.210324+0000","last_change":"2024-09-11T00:04:03.837614+0000","last_active":"2024-09-11T00:06:20.210324+0000","last_peered":"2024-09-11T00:06:20.210324+0000","last_clean":"2024-09-11T00:06:20.210324+0000","last_became_active":"2024-09-11T00:04:03.837164+0000","last_became_peered":"2024-09-11T00:04:03.837164+0000","last_unstale":"2024-09-11T00:06:20.210324+0000","last_undegraded":"2024-09-11T00:06:20.210324+0000","last_fullsized":"2024-09-11T00:06:20.210324+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:10:38.467648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3d","version":"83'7","reported_seq":44,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T00:06:20.209452+0000","last_change":"2024-09-11T00:04:03.833999+0000","last_active":"2024-09-11T00:06:20.209452+0000","last_peered":"2024-09-11T00:06:20.209452+0000","last_clean":"2024-09-11T00:06:20.209452+0000","last_became_active":"2024-09-11T00:04:03.833580+0000","last_became_peered":"2024-09-11T00:04:03.833580+0000","last_unstale":"2024-09-11T00:06:20.209452+0000","last_undegraded":"2024-09-11T00:06:20.209452+0000","last_fullsized":"2024-09-11T00:06:20.209452+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:47:40.902423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"83'4","reported_seq":41,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T00:06:20.209745+0000","last_change":"2024-09-11T00:04:03.832478+0000","last_active":"2024-09-11T00:06:20.209745+0000","last_peered":"2024-09-11T00:06:20.209745+0000","last_clean":"2024-09-11T00:06:20.209745+0000","last_became_active":"2024-09-11T00:04:03.831811+0000","last_became_peered":"2024-09-11T00:04:03.831811+0000","last_unstale":"2024-09-11T00:06:20.209745+0000","last_undegraded":"2024-09-11T00:06:20.209745+0000","last_fullsized":"2024-09-11T00:06:20.209745+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:53:05.839006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"83'8","reported_seq":45,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T00:06:20.219013+0000","last_change":"2024-09-11T00:04:03.837077+0000","last_active":"2024-09-11T00:06:20.219013+0000","last_peered":"2024-09-11T00:06:20.219013+0000","last_clean":"2024-09-11T00:06:20.219013+0000","last_became_active":"2024-09-11T00:04:03.835718+0000","last_became_peered":"2024-09-11T00:04:03.835718+0000","last_unstale":"2024-09-11T00:06:20.219013+0000","last_undegraded":"2024-09-11T00:06:20.219013+0000","last_fullsized":"2024-09-11T00:06:20.219013+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:04:02.804031+0000","last_clean_scrub_stamp":"2024-09-11T00:04:02.804031+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:17:00.633257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":72975,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":1221,"num_read_kb":691,"num_write":266,"num_write_kb":157,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":585728,"data_stored":227553,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":229,"ondisk_log_size":229,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":68,"seq":292057776176,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36320,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708832,"statfs":{"total":95995035648,"available":95957843968,"internally_reserved":0,"allocated":991232,"data_stored":487430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":36174276},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":63,"seq":270582939700,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28668,"kb_used_data":1644,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716484,"statfs":{"total":95995035648,"available":95965679616,"internally_reserved":0,"allocated":1683456,"data_stored":1143187,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":58,"seq":249108103224,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36336,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708816,"statfs":{"total":95995035648,"available":95957827584,"internally_reserved":0,"allocated":1007616,"data_stored":486716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":52,"seq":223338299455,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":40476,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":93704676,"statfs":{"total":95995035648,"available":95953588224,"internally_reserved":0,"allocated":995328,"data_stored":486982,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":47,"seq":201863462978,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28596,"kb_used_data":1564,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716556,"statfs":{"total":95995035648,"available":95965753344,"internally_reserved":0,"allocated":1601536,"data_stored":1084043,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":42,"seq":180388626506,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32152,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713000,"statfs":{"total":95995035648,"available":95962112000,"internally_reserved":0,"allocated":983040,"data_stored":482752,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":36,"seq":154618822734,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28604,"kb_used_data":1572,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716548,"statfs":{"total":95995035648,"available":95965745152,"internally_reserved":0,"allocated":1609728,"data_stored":1078782,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986260,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27992,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717160,"statfs":{"total":95995035648,"available":95966371840,"internally_reserved":0,"allocated":991232,"data_stored":486605,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182489,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32156,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712996,"statfs":{"total":95995035648,"available":95962107904,"internally_reserved":0,"allocated":995328,"data_stored":481928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31914440},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411424,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28076,"kb_used_data":1052,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717076,"statfs":{"total":95995035648,"available":95966285824,"internally_reserved":0,"allocated":1077248,"data_stored":553494,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574948,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36376,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708776,"statfs":{"total":95995035648,"available":95957786624,"internally_reserved":0,"allocated":1056768,"data_stored":553274,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738474,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32172,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712980,"statfs":{"total":95995035648,"available":95962091520,"internally_reserved":0,"allocated":1007616,"data_stored":484208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2710,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":71776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":71996,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1011,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":2820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3985,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1388,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":118784,"data_stored":67225,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1836,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T00:06:51.299 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid ff3ad74a-6fcf-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T00:06:51.563 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:06:51 smithi012 ceph-mon[30333]: pgmap v391: 129 pgs: 129 active+clean; 648 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 511 B/s wr, 10 op/s 2024-09-11T00:06:51.563 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:06:51 smithi012 ceph-mon[30333]: from='client.25379 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T00:06:51.581 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:06:51 smithi178 ceph-mon[35919]: pgmap v391: 129 pgs: 129 active+clean; 648 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 511 B/s wr, 10 op/s 2024-09-11T00:06:51.581 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:06:51 smithi178 ceph-mon[35919]: from='client.25379 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T00:06:51.679 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:06:51.782 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:06:51 smithi165 ceph-mon[35356]: pgmap v391: 129 pgs: 129 active+clean; 648 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 511 B/s wr, 10 op/s 2024-09-11T00:06:51.782 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:06:51 smithi165 ceph-mon[35356]: from='client.25379 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T00:06:52.205 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:52.202+0000 7f0d8de00640 1 -- 172.21.15.12:0/3319433855 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0d8810f3e0 msgr2=0x7f0d881117d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:06:52.205 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:52.202+0000 7f0d8de00640 1 --2- 172.21.15.12:0/3319433855 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0d8810f3e0 0x7f0d881117d0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f0d80009ca0 tx=0x7f0d8002a8c0 comp rx=0 tx=0).stop 2024-09-11T00:06:52.205 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:52.203+0000 7f0d8de00640 1 -- 172.21.15.12:0/3319433855 shutdown_connections 2024-09-11T00:06:52.205 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:52.203+0000 7f0d8de00640 1 --2- 172.21.15.12:0/3319433855 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0d8810f3e0 0x7f0d881117d0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:52.205 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:52.203+0000 7f0d8de00640 1 --2- 172.21.15.12:0/3319433855 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0d88075f40 0x7f0d8810ee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:52.205 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:52.203+0000 7f0d8de00640 1 --2- 172.21.15.12:0/3319433855 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0d88075540 0x7f0d88075920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:52.205 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:52.203+0000 7f0d8de00640 1 -- 172.21.15.12:0/3319433855 >> 172.21.15.12:0/3319433855 conn(0x7f0d880fe830 msgr2=0x7f0d88100c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:06:52.205 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:52.203+0000 7f0d8de00640 1 -- 172.21.15.12:0/3319433855 shutdown_connections 2024-09-11T00:06:52.206 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:52.203+0000 7f0d8de00640 1 -- 172.21.15.12:0/3319433855 wait complete. 2024-09-11T00:06:52.207 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:52.204+0000 7f0d8de00640 1 Processor -- start 2024-09-11T00:06:52.207 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:52.204+0000 7f0d8de00640 1 -- start start 2024-09-11T00:06:52.207 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:52.204+0000 7f0d8de00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0d88075540 0x7f0d8819b440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:06:52.207 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:52.204+0000 7f0d8de00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0d88075f40 0x7f0d8819b980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:06:52.207 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:52.204+0000 7f0d8de00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0d8810f3e0 0x7f0d881a0d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:06:52.207 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:52.204+0000 7f0d8de00640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f0d88114f70 con 0x7f0d88075f40 2024-09-11T00:06:52.207 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:52.204+0000 7f0d8de00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f0d88114df0 con 0x7f0d88075540 2024-09-11T00:06:52.207 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:52.204+0000 7f0d8de00640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f0d881150f0 con 0x7f0d8810f3e0 2024-09-11T00:06:52.207 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:52.205+0000 7f0d86a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0d88075540 0x7f0d8819b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:06:52.207 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:52.205+0000 7f0d7fe00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0d88075f40 0x7f0d8819b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:06:52.207 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:52.205+0000 7f0d86a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0d88075540 0x7f0d8819b440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.12:58262/0 (socket says 172.21.15.12:58262) 2024-09-11T00:06:52.208 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:52.205+0000 7f0d86a00640 1 -- 172.21.15.12:0/2367547868 learned_addr learned my addr 172.21.15.12:0/2367547868 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:06:52.208 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:52.205+0000 7f0d87400640 1 --2- 172.21.15.12:0/2367547868 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0d8810f3e0 0x7f0d881a0d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:06:52.208 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:52.205+0000 7f0d86a00640 1 -- 172.21.15.12:0/2367547868 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0d8810f3e0 msgr2=0x7f0d881a0d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:06:52.208 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:52.205+0000 7f0d86a00640 1 --2- 172.21.15.12:0/2367547868 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0d8810f3e0 0x7f0d881a0d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:52.208 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:52.205+0000 7f0d86a00640 1 -- 172.21.15.12:0/2367547868 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0d88075f40 msgr2=0x7f0d8819b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:06:52.208 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:52.205+0000 7f0d86a00640 1 --2- 172.21.15.12:0/2367547868 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0d88075f40 0x7f0d8819b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:52.208 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:52.205+0000 7f0d86a00640 1 -- 172.21.15.12:0/2367547868 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d881a14f0 con 0x7f0d88075540 2024-09-11T00:06:52.208 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:52.205+0000 7f0d7fe00640 1 --2- 172.21.15.12:0/2367547868 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0d88075f40 0x7f0d8819b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:06:52.209 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:52.205+0000 7f0d86a00640 1 --2- 172.21.15.12:0/2367547868 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0d88075540 0x7f0d8819b440 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f0d7800e9f0 tx=0x7f0d7800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:06:52.209 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:52.206+0000 7f0d8ca00640 1 -- 172.21.15.12:0/2367547868 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d7800cd80 con 0x7f0d88075540 2024-09-11T00:06:52.209 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:52.206+0000 7f0d8de00640 1 -- 172.21.15.12:0/2367547868 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d881a17e0 con 0x7f0d88075540 2024-09-11T00:06:52.209 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:52.206+0000 7f0d8ca00640 1 -- 172.21.15.12:0/2367547868 <== mon.2 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0d780108e0 con 0x7f0d88075540 2024-09-11T00:06:52.209 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:52.206+0000 7f0d8ca00640 1 -- 172.21.15.12:0/2367547868 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d78018980 con 0x7f0d88075540 2024-09-11T00:06:52.209 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:52.206+0000 7f0d8de00640 1 -- 172.21.15.12:0/2367547868 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d881a7ea0 con 0x7f0d88075540 2024-09-11T00:06:52.209 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:52.207+0000 7f0d8de00640 1 -- 172.21.15.12:0/2367547868 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0d88075920 con 0x7f0d88075540 2024-09-11T00:06:52.211 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:52.209+0000 7f0d8ca00640 1 -- 172.21.15.12:0/2367547868 <== mon.2 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7f0d78010430 con 0x7f0d88075540 2024-09-11T00:06:52.213 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:52.211+0000 7f0d8ca00640 1 --2- 172.21.15.12:0/2367547868 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f0d6c0b52e0 0x7f0d6c0b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:06:52.213 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:52.211+0000 7f0d8ca00640 1 -- 172.21.15.12:0/2367547868 <== mon.2 v2:172.21.15.165:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9208+0+0 (secure 0 0 0) 0x7f0d780e6800 con 0x7f0d88075540 2024-09-11T00:06:52.213 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:52.211+0000 7f0d7fe00640 1 --2- 172.21.15.12:0/2367547868 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f0d6c0b52e0 0x7f0d6c0b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:06:52.213 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:52.212+0000 7f0d7fe00640 1 --2- 172.21.15.12:0/2367547868 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f0d6c0b52e0 0x7f0d6c0b77a0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f0d88101070 tx=0x7f0d70005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:06:52.214 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:52.212+0000 7f0d8ca00640 1 -- 172.21.15.12:0/2367547868 <== mon.2 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f0d780acc40 con 0x7f0d88075540 2024-09-11T00:06:52.450 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:52.448+0000 7f0d8de00640 1 -- 172.21.15.12:0/2367547868 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0d8819c570 con 0x7f0d88075540 2024-09-11T00:06:52.451 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:52.449+0000 7f0d8ca00640 1 -- 172.21.15.12:0/2367547868 <== mon.2 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11965 (secure 0 0 0) 0x7f0d780b1670 con 0x7f0d88075540 2024-09-11T00:06:52.452 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-11T00:06:52.452 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":29,"btime":"2024-09-11T00:06:45:273053+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14577,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6835/447171591","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":447171591},{"type":"v1","addr":"172.21.15.12:6835","nonce":447171591}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14589,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/901486040","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":901486040},{"type":"v1","addr":"172.21.15.12:6837","nonce":901486040}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T00:04:04.739881+0000","modified":"2024-09-11T00:06:44.753824+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24674,"mds_1":24643,"mds_2":24638,"mds_3":24637,"mds_4":24614},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14631":{"gid":14631,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.12:6841/3069919437","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6840","nonce":3069919437},{"type":"v1","addr":"172.21.15.12:6841","nonce":3069919437}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24562":{"gid":24562,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6833/1141636112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":1141636112},{"type":"v1","addr":"172.21.15.178:6833","nonce":1141636112}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24581":{"gid":24581,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6833/3111980970","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6832","nonce":3111980970},{"type":"v1","addr":"172.21.15.165:6833","nonce":3111980970}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24607":{"gid":24607,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6835/4084868287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":4084868287},{"type":"v1","addr":"172.21.15.178:6835","nonce":4084868287}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24613":{"gid":24613,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6837/526874763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":526874763},{"type":"v1","addr":"172.21.15.178:6837","nonce":526874763}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24614":{"gid":24614,"name":"e","rank":4,"incarnation":26,"state":"up:active","state_seq":21,"addr":"172.21.15.165:6835/2425395274","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":2425395274},{"type":"v1","addr":"172.21.15.165:6835","nonce":2425395274}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24637":{"gid":24637,"name":"i","rank":3,"incarnation":24,"state":"up:active","state_seq":18,"addr":"172.21.15.178:6839/70628097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6838","nonce":70628097},{"type":"v1","addr":"172.21.15.178:6839","nonce":70628097}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24638":{"gid":24638,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":18,"addr":"172.21.15.165:6837/6215088","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6836","nonce":6215088},{"type":"v1","addr":"172.21.15.165:6837","nonce":6215088}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24643":{"gid":24643,"name":"g","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.12:6839/3124093009","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6838","nonce":3124093009},{"type":"v1","addr":"172.21.15.12:6839","nonce":3124093009}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24674":{"gid":24674,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.165:6839/501312984","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6838","nonce":501312984},{"type":"v1","addr":"172.21.15.165:6839","nonce":501312984}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24674,"qdb_cluster":[24614,24637,24638,24643,24674]},"id":1}]} 2024-09-11T00:06:52.452 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 29 2024-09-11T00:06:52.454 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:52.452+0000 7f0d8de00640 1 -- 172.21.15.12:0/2367547868 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f0d6c0b52e0 msgr2=0x7f0d6c0b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:06:52.454 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:52.452+0000 7f0d8de00640 1 --2- 172.21.15.12:0/2367547868 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f0d6c0b52e0 0x7f0d6c0b77a0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f0d88101070 tx=0x7f0d70005f90 comp rx=0 tx=0).stop 2024-09-11T00:06:52.454 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:52.452+0000 7f0d8de00640 1 -- 172.21.15.12:0/2367547868 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0d88075540 msgr2=0x7f0d8819b440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:06:52.454 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:52.452+0000 7f0d8de00640 1 --2- 172.21.15.12:0/2367547868 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0d88075540 0x7f0d8819b440 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f0d7800e9f0 tx=0x7f0d7800eeb0 comp rx=0 tx=0).stop 2024-09-11T00:06:52.454 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:52.452+0000 7f0d8de00640 1 -- 172.21.15.12:0/2367547868 shutdown_connections 2024-09-11T00:06:52.454 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:52.452+0000 7f0d8de00640 1 --2- 172.21.15.12:0/2367547868 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f0d6c0b52e0 0x7f0d6c0b77a0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:52.454 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:52.452+0000 7f0d8de00640 1 --2- 172.21.15.12:0/2367547868 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0d8810f3e0 0x7f0d881a0d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:52.455 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:52.452+0000 7f0d8de00640 1 --2- 172.21.15.12:0/2367547868 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0d88075f40 0x7f0d8819b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:52.455 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:52.452+0000 7f0d8de00640 1 --2- 172.21.15.12:0/2367547868 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0d88075540 0x7f0d8819b440 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:52.455 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:52.452+0000 7f0d8de00640 1 -- 172.21.15.12:0/2367547868 >> 172.21.15.12:0/2367547868 conn(0x7f0d880fe830 msgr2=0x7f0d881115e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:06:52.455 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:52.452+0000 7f0d8de00640 1 -- 172.21.15.12:0/2367547868 shutdown_connections 2024-09-11T00:06:52.455 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:52.452+0000 7f0d8de00640 1 -- 172.21.15.12:0/2367547868 wait complete. 2024-09-11T00:06:53.301 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid ff3ad74a-6fcf-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T00:06:53.563 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:06:53 smithi012 ceph-mon[30333]: pgmap v392: 129 pgs: 129 active+clean; 652 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 938 B/s wr, 11 op/s 2024-09-11T00:06:53.563 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:06:53 smithi012 ceph-mon[30333]: from='client.? 172.21.15.12:0/2367547868' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T00:06:53.580 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:06:53 smithi178 ceph-mon[35919]: pgmap v392: 129 pgs: 129 active+clean; 652 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 938 B/s wr, 11 op/s 2024-09-11T00:06:53.581 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:06:53 smithi178 ceph-mon[35919]: from='client.? 172.21.15.12:0/2367547868' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T00:06:53.684 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:06:53.782 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:06:53 smithi165 ceph-mon[35356]: pgmap v392: 129 pgs: 129 active+clean; 652 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 938 B/s wr, 11 op/s 2024-09-11T00:06:53.782 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:06:53 smithi165 ceph-mon[35356]: from='client.? 172.21.15.12:0/2367547868' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T00:06:54.337 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:54.334+0000 7f6d81a00640 1 -- 172.21.15.12:0/1390006933 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6d7c104fa0 msgr2=0x7f6d7c105380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:06:54.337 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:54.334+0000 7f6d81a00640 1 --2- 172.21.15.12:0/1390006933 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6d7c104fa0 0x7f6d7c105380 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f6d6c009900 tx=0x7f6d6c02a3f0 comp rx=0 tx=0).stop 2024-09-11T00:06:54.337 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:54.335+0000 7f6d81a00640 1 -- 172.21.15.12:0/1390006933 shutdown_connections 2024-09-11T00:06:54.337 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:54.335+0000 7f6d81a00640 1 --2- 172.21.15.12:0/1390006933 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6d7c10b330 0x7f6d7c111b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:54.338 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:54.335+0000 7f6d81a00640 1 --2- 172.21.15.12:0/1390006933 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6d7c10a9b0 0x7f6d7c10adf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:54.338 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:54.335+0000 7f6d81a00640 1 --2- 172.21.15.12:0/1390006933 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6d7c104fa0 0x7f6d7c105380 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:54.338 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:54.335+0000 7f6d81a00640 1 -- 172.21.15.12:0/1390006933 >> 172.21.15.12:0/1390006933 conn(0x7f6d7c100a50 msgr2=0x7f6d7c102e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:06:54.338 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:54.335+0000 7f6d81a00640 1 -- 172.21.15.12:0/1390006933 shutdown_connections 2024-09-11T00:06:54.338 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:54.335+0000 7f6d81a00640 1 -- 172.21.15.12:0/1390006933 wait complete. 2024-09-11T00:06:54.339 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:54.336+0000 7f6d81a00640 1 Processor -- start 2024-09-11T00:06:54.339 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:54.336+0000 7f6d81a00640 1 -- start start 2024-09-11T00:06:54.339 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:54.336+0000 7f6d81a00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6d7c104fa0 0x7f6d7c19b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:06:54.339 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:54.336+0000 7f6d81a00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6d7c10a9b0 0x7f6d7c19ba00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:06:54.339 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:54.336+0000 7f6d81a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6d7c10b330 0x7f6d7c1a0da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:06:54.339 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:54.336+0000 7f6d81a00640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f6d7c114fb0 con 0x7f6d7c10a9b0 2024-09-11T00:06:54.339 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:54.336+0000 7f6d81a00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f6d7c114e30 con 0x7f6d7c10b330 2024-09-11T00:06:54.339 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:54.336+0000 7f6d81a00640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f6d7c115130 con 0x7f6d7c104fa0 2024-09-11T00:06:54.340 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:54.337+0000 7f6d72000640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6d7c10a9b0 0x7f6d7c19ba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:06:54.340 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:54.337+0000 7f6d7aa00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6d7c104fa0 0x7f6d7c19b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:06:54.340 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:54.337+0000 7f6d7aa00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6d7c104fa0 0x7f6d7c19b4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.12:58766/0 (socket says 172.21.15.12:58766) 2024-09-11T00:06:54.340 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:54.337+0000 7f6d7aa00640 1 -- 172.21.15.12:0/2568345346 learned_addr learned my addr 172.21.15.12:0/2568345346 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:06:54.340 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:54.337+0000 7f6d7b400640 1 --2- 172.21.15.12:0/2568345346 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6d7c10b330 0x7f6d7c1a0da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:06:54.340 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:54.337+0000 7f6d72000640 1 -- 172.21.15.12:0/2568345346 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6d7c104fa0 msgr2=0x7f6d7c19b4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:06:54.340 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:54.337+0000 7f6d72000640 1 --2- 172.21.15.12:0/2568345346 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6d7c104fa0 0x7f6d7c19b4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:54.340 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:54.337+0000 7f6d72000640 1 -- 172.21.15.12:0/2568345346 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6d7c10b330 msgr2=0x7f6d7c1a0da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:06:54.340 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:54.337+0000 7f6d72000640 1 --2- 172.21.15.12:0/2568345346 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6d7c10b330 0x7f6d7c1a0da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:54.340 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:54.337+0000 7f6d72000640 1 -- 172.21.15.12:0/2568345346 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d7c1a1570 con 0x7f6d7c10a9b0 2024-09-11T00:06:54.341 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:54.338+0000 7f6d72000640 1 --2- 172.21.15.12:0/2568345346 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6d7c10a9b0 0x7f6d7c19ba00 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f6d64009fd0 tx=0x7f6d6400ef90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:06:54.341 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:54.338+0000 7f6d73e00640 1 -- 172.21.15.12:0/2568345346 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d64004b20 con 0x7f6d7c10a9b0 2024-09-11T00:06:54.341 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:54.338+0000 7f6d73e00640 1 -- 172.21.15.12:0/2568345346 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6d64004cc0 con 0x7f6d7c10a9b0 2024-09-11T00:06:54.341 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:54.338+0000 7f6d73e00640 1 -- 172.21.15.12:0/2568345346 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d64010720 con 0x7f6d7c10a9b0 2024-09-11T00:06:54.341 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:54.338+0000 7f6d81a00640 1 -- 172.21.15.12:0/2568345346 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6d7c1a1860 con 0x7f6d7c10a9b0 2024-09-11T00:06:54.342 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:54.338+0000 7f6d81a00640 1 -- 172.21.15.12:0/2568345346 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6d7c1a7f20 con 0x7f6d7c10a9b0 2024-09-11T00:06:54.342 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:54.339+0000 7f6d81a00640 1 -- 172.21.15.12:0/2568345346 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6d7c105380 con 0x7f6d7c10a9b0 2024-09-11T00:06:54.343 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:54.341+0000 7f6d73e00640 1 -- 172.21.15.12:0/2568345346 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7f6d64005020 con 0x7f6d7c10a9b0 2024-09-11T00:06:54.345 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:54.343+0000 7f6d73e00640 1 --2- 172.21.15.12:0/2568345346 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f6d600b52e0 0x7f6d600b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:06:54.345 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:54.343+0000 7f6d73e00640 1 -- 172.21.15.12:0/2568345346 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9208+0+0 (secure 0 0 0) 0x7f6d640e6790 con 0x7f6d7c10a9b0 2024-09-11T00:06:54.345 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:54.343+0000 7f6d7aa00640 1 --2- 172.21.15.12:0/2568345346 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f6d600b52e0 0x7f6d600b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:06:54.346 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:54.344+0000 7f6d7aa00640 1 --2- 172.21.15.12:0/2568345346 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f6d600b52e0 0x7f6d600b77a0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f6d6c0096c0 tx=0x7f6d6c005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:06:54.349 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:54.347+0000 7f6d73e00640 1 -- 172.21.15.12:0/2568345346 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f6d640ad220 con 0x7f6d7c10a9b0 2024-09-11T00:06:54.584 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:54.582+0000 7f6d81a00640 1 -- 172.21.15.12:0/2568345346 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6d7c10cb80 con 0x7f6d7c10a9b0 2024-09-11T00:06:54.585 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:54.583+0000 7f6d73e00640 1 -- 172.21.15.12:0/2568345346 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11965 (secure 0 0 0) 0x7f6d640acb20 con 0x7f6d7c10a9b0 2024-09-11T00:06:54.585 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-11T00:06:54.585 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":29,"btime":"2024-09-11T00:06:45:273053+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14577,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6835/447171591","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":447171591},{"type":"v1","addr":"172.21.15.12:6835","nonce":447171591}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14589,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/901486040","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":901486040},{"type":"v1","addr":"172.21.15.12:6837","nonce":901486040}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T00:04:04.739881+0000","modified":"2024-09-11T00:06:44.753824+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24674,"mds_1":24643,"mds_2":24638,"mds_3":24637,"mds_4":24614},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14631":{"gid":14631,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.12:6841/3069919437","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6840","nonce":3069919437},{"type":"v1","addr":"172.21.15.12:6841","nonce":3069919437}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24562":{"gid":24562,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6833/1141636112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":1141636112},{"type":"v1","addr":"172.21.15.178:6833","nonce":1141636112}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24581":{"gid":24581,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6833/3111980970","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6832","nonce":3111980970},{"type":"v1","addr":"172.21.15.165:6833","nonce":3111980970}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24607":{"gid":24607,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6835/4084868287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":4084868287},{"type":"v1","addr":"172.21.15.178:6835","nonce":4084868287}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24613":{"gid":24613,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6837/526874763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":526874763},{"type":"v1","addr":"172.21.15.178:6837","nonce":526874763}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24614":{"gid":24614,"name":"e","rank":4,"incarnation":26,"state":"up:active","state_seq":21,"addr":"172.21.15.165:6835/2425395274","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":2425395274},{"type":"v1","addr":"172.21.15.165:6835","nonce":2425395274}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24637":{"gid":24637,"name":"i","rank":3,"incarnation":24,"state":"up:active","state_seq":18,"addr":"172.21.15.178:6839/70628097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6838","nonce":70628097},{"type":"v1","addr":"172.21.15.178:6839","nonce":70628097}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24638":{"gid":24638,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":18,"addr":"172.21.15.165:6837/6215088","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6836","nonce":6215088},{"type":"v1","addr":"172.21.15.165:6837","nonce":6215088}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24643":{"gid":24643,"name":"g","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.12:6839/3124093009","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6838","nonce":3124093009},{"type":"v1","addr":"172.21.15.12:6839","nonce":3124093009}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24674":{"gid":24674,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.165:6839/501312984","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6838","nonce":501312984},{"type":"v1","addr":"172.21.15.165:6839","nonce":501312984}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24674,"qdb_cluster":[24614,24637,24638,24643,24674]},"id":1}]} 2024-09-11T00:06:54.586 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 29 2024-09-11T00:06:54.587 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:54.585+0000 7f6d81a00640 1 -- 172.21.15.12:0/2568345346 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f6d600b52e0 msgr2=0x7f6d600b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:06:54.587 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:54.585+0000 7f6d81a00640 1 --2- 172.21.15.12:0/2568345346 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f6d600b52e0 0x7f6d600b77a0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f6d6c0096c0 tx=0x7f6d6c005a20 comp rx=0 tx=0).stop 2024-09-11T00:06:54.588 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:54.585+0000 7f6d81a00640 1 -- 172.21.15.12:0/2568345346 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6d7c10a9b0 msgr2=0x7f6d7c19ba00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:06:54.588 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:54.585+0000 7f6d81a00640 1 --2- 172.21.15.12:0/2568345346 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6d7c10a9b0 0x7f6d7c19ba00 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f6d64009fd0 tx=0x7f6d6400ef90 comp rx=0 tx=0).stop 2024-09-11T00:06:54.588 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:54.585+0000 7f6d81a00640 1 -- 172.21.15.12:0/2568345346 shutdown_connections 2024-09-11T00:06:54.588 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:54.585+0000 7f6d81a00640 1 --2- 172.21.15.12:0/2568345346 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f6d600b52e0 0x7f6d600b77a0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:54.588 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:54.585+0000 7f6d81a00640 1 --2- 172.21.15.12:0/2568345346 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6d7c10b330 0x7f6d7c1a0da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:54.588 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:54.585+0000 7f6d81a00640 1 --2- 172.21.15.12:0/2568345346 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6d7c10a9b0 0x7f6d7c19ba00 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:54.588 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:54.585+0000 7f6d81a00640 1 --2- 172.21.15.12:0/2568345346 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6d7c104fa0 0x7f6d7c19b4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:54.588 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:54.585+0000 7f6d81a00640 1 -- 172.21.15.12:0/2568345346 >> 172.21.15.12:0/2568345346 conn(0x7f6d7c100a50 msgr2=0x7f6d7c102e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:06:54.588 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:54.586+0000 7f6d81a00640 1 -- 172.21.15.12:0/2568345346 shutdown_connections 2024-09-11T00:06:54.588 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:54.586+0000 7f6d81a00640 1 -- 172.21.15.12:0/2568345346 wait complete. 2024-09-11T00:06:55.258 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid ff3ad74a-6fcf-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-11T00:06:55.563 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:06:55 smithi012 ceph-mon[30333]: pgmap v393: 129 pgs: 129 active+clean; 652 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 938 B/s wr, 9 op/s 2024-09-11T00:06:55.563 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:06:55 smithi012 ceph-mon[30333]: from='client.? 172.21.15.12:0/2568345346' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T00:06:55.580 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:06:55 smithi178 ceph-mon[35919]: pgmap v393: 129 pgs: 129 active+clean; 652 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 938 B/s wr, 9 op/s 2024-09-11T00:06:55.580 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:06:55 smithi178 ceph-mon[35919]: from='client.? 172.21.15.12:0/2568345346' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T00:06:55.638 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:06:55.782 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:06:55 smithi165 ceph-mon[35356]: pgmap v393: 129 pgs: 129 active+clean; 652 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 938 B/s wr, 9 op/s 2024-09-11T00:06:55.782 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:06:55 smithi165 ceph-mon[35356]: from='client.? 172.21.15.12:0/2568345346' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T00:06:56.404 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:56.402+0000 7f24adc00640 1 -- 172.21.15.12:0/784581243 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f24a81087d0 msgr2=0x7f24a8108c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:06:56.405 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:56.402+0000 7f24adc00640 1 --2- 172.21.15.12:0/784581243 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f24a81087d0 0x7f24a8108c10 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f2490009900 tx=0x7f249002a410 comp rx=0 tx=0).stop 2024-09-11T00:06:56.405 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:56.402+0000 7f24adc00640 1 -- 172.21.15.12:0/784581243 shutdown_connections 2024-09-11T00:06:56.405 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:56.402+0000 7f24adc00640 1 --2- 172.21.15.12:0/784581243 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f24a8109150 0x7f24a810f950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:56.405 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:56.402+0000 7f24adc00640 1 --2- 172.21.15.12:0/784581243 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f24a81087d0 0x7f24a8108c10 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:56.405 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:56.402+0000 7f24adc00640 1 --2- 172.21.15.12:0/784581243 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f24a8102dc0 0x7f24a81031a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:56.405 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:56.402+0000 7f24adc00640 1 -- 172.21.15.12:0/784581243 >> 172.21.15.12:0/784581243 conn(0x7f24a80fe830 msgr2=0x7f24a8100c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:06:56.405 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:56.402+0000 7f24adc00640 1 -- 172.21.15.12:0/784581243 shutdown_connections 2024-09-11T00:06:56.405 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:56.402+0000 7f24adc00640 1 -- 172.21.15.12:0/784581243 wait complete. 2024-09-11T00:06:56.406 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:56.403+0000 7f24adc00640 1 Processor -- start 2024-09-11T00:06:56.406 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:56.403+0000 7f24adc00640 1 -- start start 2024-09-11T00:06:56.406 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:56.404+0000 7f24adc00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f24a8102dc0 0x7f24a819b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:06:56.407 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:56.404+0000 7f24adc00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f24a81087d0 0x7f24a819ba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:06:56.407 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:56.404+0000 7f24adc00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f24a8109150 0x7f24a81a0dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:06:56.407 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:56.404+0000 7f24adc00640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f24a8112e20 con 0x7f24a8102dc0 2024-09-11T00:06:56.407 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:56.404+0000 7f24adc00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f24a8112ca0 con 0x7f24a81087d0 2024-09-11T00:06:56.407 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:56.404+0000 7f24adc00640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f24a8112fa0 con 0x7f24a8109150 2024-09-11T00:06:56.407 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:56.404+0000 7f24a6a00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f24a8102dc0 0x7f24a819b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:06:56.407 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:56.404+0000 7f24a6a00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f24a8102dc0 0x7f24a819b4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:54578/0 (socket says 172.21.15.12:54578) 2024-09-11T00:06:56.407 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:56.404+0000 7f24a6a00640 1 -- 172.21.15.12:0/846074235 learned_addr learned my addr 172.21.15.12:0/846074235 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:06:56.407 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:56.404+0000 7f249fe00640 1 --2- 172.21.15.12:0/846074235 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f24a81087d0 0x7f24a819ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:06:56.407 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:56.404+0000 7f24a7400640 1 --2- 172.21.15.12:0/846074235 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f24a8109150 0x7f24a81a0dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:06:56.407 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:56.405+0000 7f24a6a00640 1 -- 172.21.15.12:0/846074235 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f24a8109150 msgr2=0x7f24a81a0dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:06:56.407 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:56.405+0000 7f24a6a00640 1 --2- 172.21.15.12:0/846074235 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f24a8109150 0x7f24a81a0dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:56.407 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:56.405+0000 7f24a6a00640 1 -- 172.21.15.12:0/846074235 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f24a81087d0 msgr2=0x7f24a819ba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:06:56.408 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:56.405+0000 7f24a6a00640 1 --2- 172.21.15.12:0/846074235 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f24a81087d0 0x7f24a819ba20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:56.408 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:56.405+0000 7f24a6a00640 1 -- 172.21.15.12:0/846074235 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24a81a1590 con 0x7f24a8102dc0 2024-09-11T00:06:56.408 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:56.405+0000 7f24a6a00640 1 --2- 172.21.15.12:0/846074235 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f24a8102dc0 0x7f24a819b4e0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f249800e9f0 tx=0x7f249800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:06:56.409 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:56.405+0000 7f249f400640 1 -- 172.21.15.12:0/846074235 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f249800cd80 con 0x7f24a8102dc0 2024-09-11T00:06:56.409 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:56.405+0000 7f24adc00640 1 -- 172.21.15.12:0/846074235 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24a81a1880 con 0x7f24a8102dc0 2024-09-11T00:06:56.409 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:56.405+0000 7f249f400640 1 -- 172.21.15.12:0/846074235 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f24980054d0 con 0x7f24a8102dc0 2024-09-11T00:06:56.409 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:56.405+0000 7f249f400640 1 -- 172.21.15.12:0/846074235 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2498010690 con 0x7f24a8102dc0 2024-09-11T00:06:56.409 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:56.405+0000 7f24adc00640 1 -- 172.21.15.12:0/846074235 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24a81a7f40 con 0x7f24a8102dc0 2024-09-11T00:06:56.409 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:56.407+0000 7f24adc00640 1 -- 172.21.15.12:0/846074235 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f24a81031a0 con 0x7f24a8102dc0 2024-09-11T00:06:56.410 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:56.409+0000 7f249f400640 1 -- 172.21.15.12:0/846074235 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7f2498004e90 con 0x7f24a8102dc0 2024-09-11T00:06:56.412 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:56.410+0000 7f249f400640 1 --2- 172.21.15.12:0/846074235 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f248c0b52e0 0x7f248c0b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:06:56.412 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:56.410+0000 7f249f400640 1 -- 172.21.15.12:0/846074235 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9208+0+0 (secure 0 0 0) 0x7f24980e6410 con 0x7f24a8102dc0 2024-09-11T00:06:56.412 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:56.410+0000 7f249fe00640 1 --2- 172.21.15.12:0/846074235 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f248c0b52e0 0x7f248c0b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:06:56.413 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:56.411+0000 7f249fe00640 1 --2- 172.21.15.12:0/846074235 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f248c0b52e0 0x7f248c0b77a0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f2490005ab0 tx=0x7f2490005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:06:56.416 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:56.414+0000 7f249f400640 1 -- 172.21.15.12:0/846074235 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f24980acea0 con 0x7f24a8102dc0 2024-09-11T00:06:56.618 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:56.616+0000 7f24adc00640 1 -- 172.21.15.12:0/846074235 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f24a810edb0 con 0x7f24a8102dc0 2024-09-11T00:06:56.619 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:56.618+0000 7f249f400640 1 -- 172.21.15.12:0/846074235 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) ==== 74+0+24948 (secure 0 0 0) 0x7f24980ac7a0 con 0x7f24a8102dc0 2024-09-11T00:06:56.620 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-11T00:06:56.620 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":85,"fsid":"ff3ad74a-6fcf-11ef-bcea-c7b262605968","created":"2024-09-10T23:55:16.317643+0000","modified":"2024-09-11T00:06:20.198537+0000","last_up_change":"2024-09-11T00:03:08.959256+0000","last_in_change":"2024-09-11T00:02:46.564631+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T23:59:10.488728+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-11T00:04:00.065618+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"74","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-11T00:04:02.689888+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"74","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-11T00:04:09.862120+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"6382c200-a912-42a1-958b-79191b21249b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6802","nonce":1237101991},{"type":"v1","addr":"172.21.15.12:6803","nonce":1237101991}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6804","nonce":1237101991},{"type":"v1","addr":"172.21.15.12:6805","nonce":1237101991}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6808","nonce":1237101991},{"type":"v1","addr":"172.21.15.12:6809","nonce":1237101991}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6806","nonce":1237101991},{"type":"v1","addr":"172.21.15.12:6807","nonce":1237101991}]},"public_addr":"172.21.15.12:6803/1237101991","cluster_addr":"172.21.15.12:6805/1237101991","heartbeat_back_addr":"172.21.15.12:6809/1237101991","heartbeat_front_addr":"172.21.15.12:6807/1237101991","state":["exists","up"]},{"osd":1,"uuid":"fe0e1d0a-bdc5-4859-aa84-9038059a45f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6800","nonce":545848456},{"type":"v1","addr":"172.21.15.165:6801","nonce":545848456}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6802","nonce":545848456},{"type":"v1","addr":"172.21.15.165:6803","nonce":545848456}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6806","nonce":545848456},{"type":"v1","addr":"172.21.15.165:6807","nonce":545848456}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6804","nonce":545848456},{"type":"v1","addr":"172.21.15.165:6805","nonce":545848456}]},"public_addr":"172.21.15.165:6801/545848456","cluster_addr":"172.21.15.165:6803/545848456","heartbeat_back_addr":"172.21.15.165:6807/545848456","heartbeat_front_addr":"172.21.15.165:6805/545848456","state":["exists","up"]},{"osd":2,"uuid":"94f640dc-5f78-4e2c-87d9-03e96a6e7ba0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6800","nonce":4131162695},{"type":"v1","addr":"172.21.15.178:6801","nonce":4131162695}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":4131162695},{"type":"v1","addr":"172.21.15.178:6803","nonce":4131162695}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":4131162695},{"type":"v1","addr":"172.21.15.178:6807","nonce":4131162695}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":4131162695},{"type":"v1","addr":"172.21.15.178:6805","nonce":4131162695}]},"public_addr":"172.21.15.178:6801/4131162695","cluster_addr":"172.21.15.178:6803/4131162695","heartbeat_back_addr":"172.21.15.178:6807/4131162695","heartbeat_front_addr":"172.21.15.178:6805/4131162695","state":["exists","up"]},{"osd":3,"uuid":"af2f64d6-c9be-44a9-b041-8d599a7c080d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6810","nonce":452464245},{"type":"v1","addr":"172.21.15.12:6811","nonce":452464245}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6812","nonce":452464245},{"type":"v1","addr":"172.21.15.12:6813","nonce":452464245}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6816","nonce":452464245},{"type":"v1","addr":"172.21.15.12:6817","nonce":452464245}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6814","nonce":452464245},{"type":"v1","addr":"172.21.15.12:6815","nonce":452464245}]},"public_addr":"172.21.15.12:6811/452464245","cluster_addr":"172.21.15.12:6813/452464245","heartbeat_back_addr":"172.21.15.12:6817/452464245","heartbeat_front_addr":"172.21.15.12:6815/452464245","state":["exists","up"]},{"osd":4,"uuid":"ad76ac70-20c0-44ae-9866-5f9cfce2334c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6808","nonce":1899879280},{"type":"v1","addr":"172.21.15.165:6809","nonce":1899879280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6810","nonce":1899879280},{"type":"v1","addr":"172.21.15.165:6811","nonce":1899879280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6814","nonce":1899879280},{"type":"v1","addr":"172.21.15.165:6815","nonce":1899879280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6812","nonce":1899879280},{"type":"v1","addr":"172.21.15.165:6813","nonce":1899879280}]},"public_addr":"172.21.15.165:6809/1899879280","cluster_addr":"172.21.15.165:6811/1899879280","heartbeat_back_addr":"172.21.15.165:6815/1899879280","heartbeat_front_addr":"172.21.15.165:6813/1899879280","state":["exists","up"]},{"osd":5,"uuid":"aafda2cc-62fe-4fcb-9a96-8732e0b501d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":3042228555},{"type":"v1","addr":"172.21.15.178:6809","nonce":3042228555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":3042228555},{"type":"v1","addr":"172.21.15.178:6811","nonce":3042228555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6814","nonce":3042228555},{"type":"v1","addr":"172.21.15.178:6815","nonce":3042228555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":3042228555},{"type":"v1","addr":"172.21.15.178:6813","nonce":3042228555}]},"public_addr":"172.21.15.178:6809/3042228555","cluster_addr":"172.21.15.178:6811/3042228555","heartbeat_back_addr":"172.21.15.178:6815/3042228555","heartbeat_front_addr":"172.21.15.178:6813/3042228555","state":["exists","up"]},{"osd":6,"uuid":"375eca52-efe3-42a5-bbb1-d7d392d71f15","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6818","nonce":3868322467},{"type":"v1","addr":"172.21.15.12:6819","nonce":3868322467}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6820","nonce":3868322467},{"type":"v1","addr":"172.21.15.12:6821","nonce":3868322467}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6824","nonce":3868322467},{"type":"v1","addr":"172.21.15.12:6825","nonce":3868322467}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6822","nonce":3868322467},{"type":"v1","addr":"172.21.15.12:6823","nonce":3868322467}]},"public_addr":"172.21.15.12:6819/3868322467","cluster_addr":"172.21.15.12:6821/3868322467","heartbeat_back_addr":"172.21.15.12:6825/3868322467","heartbeat_front_addr":"172.21.15.12:6823/3868322467","state":["exists","up"]},{"osd":7,"uuid":"0fd619f5-44c3-4cf7-ae7a-388dd798de36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6816","nonce":1143773735},{"type":"v1","addr":"172.21.15.165:6817","nonce":1143773735}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6818","nonce":1143773735},{"type":"v1","addr":"172.21.15.165:6819","nonce":1143773735}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6822","nonce":1143773735},{"type":"v1","addr":"172.21.15.165:6823","nonce":1143773735}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6820","nonce":1143773735},{"type":"v1","addr":"172.21.15.165:6821","nonce":1143773735}]},"public_addr":"172.21.15.165:6817/1143773735","cluster_addr":"172.21.15.165:6819/1143773735","heartbeat_back_addr":"172.21.15.165:6823/1143773735","heartbeat_front_addr":"172.21.15.165:6821/1143773735","state":["exists","up"]},{"osd":8,"uuid":"b03d14ce-8427-462a-9bcc-27683c549f53","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":2891445058},{"type":"v1","addr":"172.21.15.178:6817","nonce":2891445058}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6818","nonce":2891445058},{"type":"v1","addr":"172.21.15.178:6819","nonce":2891445058}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6822","nonce":2891445058},{"type":"v1","addr":"172.21.15.178:6823","nonce":2891445058}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":2891445058},{"type":"v1","addr":"172.21.15.178:6821","nonce":2891445058}]},"public_addr":"172.21.15.178:6817/2891445058","cluster_addr":"172.21.15.178:6819/2891445058","heartbeat_back_addr":"172.21.15.178:6823/2891445058","heartbeat_front_addr":"172.21.15.178:6821/2891445058","state":["exists","up"]},{"osd":9,"uuid":"81a6cf69-2a24-4a6c-9adf-6b7df38cfacb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6826","nonce":3359634903},{"type":"v1","addr":"172.21.15.12:6827","nonce":3359634903}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6828","nonce":3359634903},{"type":"v1","addr":"172.21.15.12:6829","nonce":3359634903}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6832","nonce":3359634903},{"type":"v1","addr":"172.21.15.12:6833","nonce":3359634903}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6830","nonce":3359634903},{"type":"v1","addr":"172.21.15.12:6831","nonce":3359634903}]},"public_addr":"172.21.15.12:6827/3359634903","cluster_addr":"172.21.15.12:6829/3359634903","heartbeat_back_addr":"172.21.15.12:6833/3359634903","heartbeat_front_addr":"172.21.15.12:6831/3359634903","state":["exists","up"]},{"osd":10,"uuid":"c1ad1f0a-33f4-4c32-91f6-4d030a29f1dd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6824","nonce":3642325778},{"type":"v1","addr":"172.21.15.165:6825","nonce":3642325778}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6826","nonce":3642325778},{"type":"v1","addr":"172.21.15.165:6827","nonce":3642325778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6830","nonce":3642325778},{"type":"v1","addr":"172.21.15.165:6831","nonce":3642325778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6828","nonce":3642325778},{"type":"v1","addr":"172.21.15.165:6829","nonce":3642325778}]},"public_addr":"172.21.15.165:6825/3642325778","cluster_addr":"172.21.15.165:6827/3642325778","heartbeat_back_addr":"172.21.15.165:6831/3642325778","heartbeat_front_addr":"172.21.15.165:6829/3642325778","state":["exists","up"]},{"osd":11,"uuid":"8413c9b0-680b-4d02-aeab-b3b3ff16c3d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":2937059756},{"type":"v1","addr":"172.21.15.178:6825","nonce":2937059756}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6826","nonce":2937059756},{"type":"v1","addr":"172.21.15.178:6827","nonce":2937059756}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":2937059756},{"type":"v1","addr":"172.21.15.178:6831","nonce":2937059756}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":2937059756},{"type":"v1","addr":"172.21.15.178:6829","nonce":2937059756}]},"public_addr":"172.21.15.178:6825/2937059756","cluster_addr":"172.21.15.178:6827/2937059756","heartbeat_back_addr":"172.21.15.178:6831/2937059756","heartbeat_front_addr":"172.21.15.178:6829/2937059756","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:58:16.036348+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:58:41.483150+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:59:07.107283+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:59:35.706574+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T00:00:02.555720+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T00:00:28.092847+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T00:00:57.396495+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T00:01:24.216775+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T00:01:48.807850+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T00:02:16.838565+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T00:02:43.131736+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T00:03:07.353155+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":8,"to":4}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"3.e","mappings":[{"from":5,"to":9}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9}]},{"pgid":"3.23","mappings":[{"from":4,"to":7}]},{"pgid":"3.31","mappings":[{"from":4,"to":8}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"3.39","mappings":[{"from":5,"to":6}]},{"pgid":"4.5","mappings":[{"from":9,"to":10}]},{"pgid":"4.6","mappings":[{"from":5,"to":8}]},{"pgid":"4.b","mappings":[{"from":9,"to":8}]},{"pgid":"4.13","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.12:6801/4177251511":"2024-09-11T23:56:17.999820+0000","172.21.15.12:6800/4177251511":"2024-09-11T23:56:17.999820+0000","172.21.15.12:0/3273692817":"2024-09-11T23:56:17.999820+0000","172.21.15.12:0/4017778158":"2024-09-11T23:56:17.999820+0000","172.21.15.12:0/2499921051":"2024-09-11T23:55:50.302721+0000","172.21.15.12:6801/760140910":"2024-09-11T23:55:50.302721+0000","172.21.15.12:6800/760140910":"2024-09-11T23:55:50.302721+0000","172.21.15.12:0/2810703302":"2024-09-11T23:56:17.999820+0000","172.21.15.12:0/2325436065":"2024-09-11T23:55:50.302721+0000","172.21.15.12:0/3137053479":"2024-09-11T23:55:50.302721+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-11T00:06:56.622 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:56.620+0000 7f24adc00640 1 -- 172.21.15.12:0/846074235 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f248c0b52e0 msgr2=0x7f248c0b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:06:56.623 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:56.620+0000 7f24adc00640 1 --2- 172.21.15.12:0/846074235 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f248c0b52e0 0x7f248c0b77a0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f2490005ab0 tx=0x7f2490005a20 comp rx=0 tx=0).stop 2024-09-11T00:06:56.623 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:56.620+0000 7f24adc00640 1 -- 172.21.15.12:0/846074235 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f24a8102dc0 msgr2=0x7f24a819b4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:06:56.623 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:56.620+0000 7f24adc00640 1 --2- 172.21.15.12:0/846074235 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f24a8102dc0 0x7f24a819b4e0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f249800e9f0 tx=0x7f249800edf0 comp rx=0 tx=0).stop 2024-09-11T00:06:56.623 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:56.621+0000 7f24adc00640 1 -- 172.21.15.12:0/846074235 shutdown_connections 2024-09-11T00:06:56.623 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:56.621+0000 7f24adc00640 1 --2- 172.21.15.12:0/846074235 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f248c0b52e0 0x7f248c0b77a0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:56.623 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:56.621+0000 7f24adc00640 1 --2- 172.21.15.12:0/846074235 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f24a8109150 0x7f24a81a0dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:56.623 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:56.621+0000 7f24adc00640 1 --2- 172.21.15.12:0/846074235 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f24a81087d0 0x7f24a819ba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:56.623 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:56.621+0000 7f24adc00640 1 --2- 172.21.15.12:0/846074235 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f24a8102dc0 0x7f24a819b4e0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:56.623 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:56.621+0000 7f24adc00640 1 -- 172.21.15.12:0/846074235 >> 172.21.15.12:0/846074235 conn(0x7f24a80fe830 msgr2=0x7f24a8105db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:06:56.623 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:56.621+0000 7f24adc00640 1 -- 172.21.15.12:0/846074235 shutdown_connections 2024-09-11T00:06:56.623 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:06:56.621+0000 7f24adc00640 1 -- 172.21.15.12:0/846074235 wait complete. 2024-09-11T00:06:57.426 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:06:57 smithi012 ceph-mon[30333]: pgmap v394: 129 pgs: 129 active+clean; 652 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 938 B/s wr, 13 op/s 2024-09-11T00:06:57.426 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:06:57 smithi012 ceph-mon[30333]: from='client.? 172.21.15.12:0/846074235' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-11T00:06:57.470 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid ff3ad74a-6fcf-11ef-bcea-c7b262605968 -- ceph daemon mon.a --format=json config get run_dir 2024-09-11T00:06:57.580 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:06:57 smithi178 ceph-mon[35919]: pgmap v394: 129 pgs: 129 active+clean; 652 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 938 B/s wr, 13 op/s 2024-09-11T00:06:57.581 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:06:57 smithi178 ceph-mon[35919]: from='client.? 172.21.15.12:0/846074235' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-11T00:06:57.782 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:06:57 smithi165 ceph-mon[35356]: pgmap v394: 129 pgs: 129 active+clean; 652 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 938 B/s wr, 13 op/s 2024-09-11T00:06:57.782 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:06:57 smithi165 ceph-mon[35356]: from='client.? 172.21.15.12:0/846074235' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-11T00:06:57.850 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:06:58.534 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:06:58.531+0000 7f96aba00640 1 -- 172.21.15.12:0/2270174219 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f96a4076f30 msgr2=0x7f96a4075580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:06:58.534 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:06:58.531+0000 7f96aba00640 1 --2- 172.21.15.12:0/2270174219 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f96a4076f30 0x7f96a4075580 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f9694009900 tx=0x7f969402a410 comp rx=0 tx=0).stop 2024-09-11T00:06:58.534 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:06:58.531+0000 7f96aba00640 1 -- 172.21.15.12:0/2270174219 shutdown_connections 2024-09-11T00:06:58.534 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:06:58.531+0000 7f96aba00640 1 --2- 172.21.15.12:0/2270174219 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f96a410a910 0x7f96a410cd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:58.534 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:06:58.531+0000 7f96aba00640 1 --2- 172.21.15.12:0/2270174219 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f96a4075ac0 0x7f96a4075f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:58.534 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:06:58.531+0000 7f96aba00640 1 --2- 172.21.15.12:0/2270174219 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f96a4076f30 0x7f96a4075580 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:58.534 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:06:58.531+0000 7f96aba00640 1 -- 172.21.15.12:0/2270174219 >> 172.21.15.12:0/2270174219 conn(0x7f96a40fe250 msgr2=0x7f96a4100670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:06:58.534 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:06:58.532+0000 7f96aba00640 1 -- 172.21.15.12:0/2270174219 shutdown_connections 2024-09-11T00:06:58.535 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:06:58.532+0000 7f96aba00640 1 -- 172.21.15.12:0/2270174219 wait complete. 2024-09-11T00:06:58.536 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:06:58.532+0000 7f96aba00640 1 Processor -- start 2024-09-11T00:06:58.536 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:06:58.533+0000 7f96aba00640 1 -- start start 2024-09-11T00:06:58.536 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:06:58.533+0000 7f96aba00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f96a4075ac0 0x7f96a419afa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:06:58.537 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:06:58.533+0000 7f96aba00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f96a4076f30 0x7f96a419b5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:06:58.537 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:06:58.533+0000 7f96aba00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f96a410a910 0x7f96a419c220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:06:58.537 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:06:58.533+0000 7f96aba00640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f96a41107e0 con 0x7f96a4076f30 2024-09-11T00:06:58.537 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:06:58.533+0000 7f96aba00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f96a4110660 con 0x7f96a4075ac0 2024-09-11T00:06:58.537 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:06:58.533+0000 7f96aba00640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f96a4110960 con 0x7f96a410a910 2024-09-11T00:06:58.537 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:06:58.533+0000 7f96a3e00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f96a4076f30 0x7f96a419b5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:06:58.537 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:06:58.533+0000 7f96a3e00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f96a4076f30 0x7f96a419b5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:54592/0 (socket says 172.21.15.12:54592) 2024-09-11T00:06:58.537 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:06:58.533+0000 7f96a3e00640 1 -- 172.21.15.12:0/3452892606 learned_addr learned my addr 172.21.15.12:0/3452892606 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:06:58.537 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:06:58.533+0000 7f96a9000640 1 --2- 172.21.15.12:0/3452892606 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f96a4075ac0 0x7f96a419afa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:06:58.537 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:06:58.533+0000 7f96a9a00640 1 --2- 172.21.15.12:0/3452892606 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f96a410a910 0x7f96a419c220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:06:58.537 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:06:58.534+0000 7f96a3e00640 1 -- 172.21.15.12:0/3452892606 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f96a410a910 msgr2=0x7f96a419c220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:06:58.537 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:06:58.534+0000 7f96a3e00640 1 --2- 172.21.15.12:0/3452892606 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f96a410a910 0x7f96a419c220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:58.538 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:06:58.534+0000 7f96a3e00640 1 -- 172.21.15.12:0/3452892606 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f96a4075ac0 msgr2=0x7f96a419afa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:06:58.538 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:06:58.534+0000 7f96a3e00640 1 --2- 172.21.15.12:0/3452892606 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f96a4075ac0 0x7f96a419afa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:58.538 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:06:58.534+0000 7f96a3e00640 1 -- 172.21.15.12:0/3452892606 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96a419cae0 con 0x7f96a4076f30 2024-09-11T00:06:58.538 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:06:58.534+0000 7f96a3e00640 1 --2- 172.21.15.12:0/3452892606 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f96a4076f30 0x7f96a419b5f0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f968c00ebc0 tx=0x7f968c00c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:06:58.538 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:06:58.534+0000 7f96a1600640 1 -- 172.21.15.12:0/3452892606 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f968c010630 con 0x7f96a4076f30 2024-09-11T00:06:58.538 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:06:58.534+0000 7f96a1600640 1 -- 172.21.15.12:0/3452892606 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f968c010c80 con 0x7f96a4076f30 2024-09-11T00:06:58.538 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:06:58.534+0000 7f96aba00640 1 -- 172.21.15.12:0/3452892606 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96a419cdd0 con 0x7f96a4076f30 2024-09-11T00:06:58.538 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:06:58.534+0000 7f96a1600640 1 -- 172.21.15.12:0/3452892606 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f968c018b60 con 0x7f96a4076f30 2024-09-11T00:06:58.538 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:06:58.534+0000 7f96aba00640 1 -- 172.21.15.12:0/3452892606 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96a41a34a0 con 0x7f96a4076f30 2024-09-11T00:06:58.540 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:06:58.538+0000 7f96aea25740 1 Processor -- start 2024-09-11T00:06:58.540 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:06:58.539+0000 7f96aea25740 1 -- start start 2024-09-11T00:06:58.542 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:06:58.539+0000 7f96aea25740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a7f090f790 0x55a7f0911b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:06:58.542 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:06:58.539+0000 7f96aea25740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a7f09120c0 0x55a7f09144b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:06:58.542 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:06:58.539+0000 7f96aea25740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a7f0914b00 0x55a7f0916fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:06:58.542 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:06:58.539+0000 7f96aea25740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55a7f08826b0 con 0x55a7f0914b00 2024-09-11T00:06:58.542 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:06:58.539+0000 7f96aea25740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55a7f0871ae0 con 0x55a7f090f790 2024-09-11T00:06:58.542 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:06:58.539+0000 7f96aea25740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55a7f08758f0 con 0x55a7f09120c0 2024-09-11T00:06:58.542 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:06:58.539+0000 7f96a1600640 1 -- 172.21.15.12:0/3452892606 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7f968c0107d0 con 0x7f96a4076f30 2024-09-11T00:06:58.542 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:06:58.539+0000 7f96a9000640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a7f09120c0 0x55a7f09144b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:06:58.542 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:06:58.539+0000 7f96a9000640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a7f09120c0 0x55a7f09144b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.12:58800/0 (socket says 172.21.15.12:58800) 2024-09-11T00:06:58.542 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:06:58.539+0000 7f96a9000640 1 -- 172.21.15.12:0/3879150135 learned_addr learned my addr 172.21.15.12:0/3879150135 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:06:58.543 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:06:58.540+0000 7f96a9a00640 1 --2- 172.21.15.12:0/3879150135 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a7f090f790 0x55a7f0911b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:06:58.543 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:06:58.540+0000 7f96a3e00640 1 --2- 172.21.15.12:0/3879150135 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a7f0914b00 0x55a7f0916fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:06:58.543 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:06:58.540+0000 7f96a9000640 1 -- 172.21.15.12:0/3879150135 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a7f090f790 msgr2=0x55a7f0911b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:06:58.543 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:06:58.540+0000 7f96a9000640 1 --2- 172.21.15.12:0/3879150135 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a7f090f790 0x55a7f0911b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:58.543 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:06:58.540+0000 7f96a9000640 1 -- 172.21.15.12:0/3879150135 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a7f0914b00 msgr2=0x55a7f0916fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:06:58.543 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:06:58.540+0000 7f96a9000640 1 --2- 172.21.15.12:0/3879150135 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a7f0914b00 0x55a7f0916fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:58.543 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:06:58.540+0000 7f96a9000640 1 -- 172.21.15.12:0/3879150135 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a7f083e780 con 0x55a7f09120c0 2024-09-11T00:06:58.550 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:06:58.541+0000 7f96a9000640 1 --2- 172.21.15.12:0/3879150135 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a7f09120c0 0x55a7f09144b0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f9694009900 tx=0x7f9694031b10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=40f606002c31053b server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:06:58.550 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:06:58.541+0000 7f969a000640 1 -- 172.21.15.12:0/3879150135 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9694035030 con 0x55a7f09120c0 2024-09-11T00:06:58.550 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:06:58.541+0000 7f969a000640 1 -- 172.21.15.12:0/3879150135 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9694040070 con 0x55a7f09120c0 2024-09-11T00:06:58.551 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:06:58.541+0000 7f969a000640 1 -- 172.21.15.12:0/3879150135 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96940039e0 con 0x55a7f09120c0 2024-09-11T00:06:58.551 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:06:58.541+0000 7f96a1600640 1 --2- 172.21.15.12:0/3452892606 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f96840b5360 0x7f96840b7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:06:58.551 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:06:58.541+0000 7f96a1600640 1 -- 172.21.15.12:0/3452892606 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9208+0+0 (secure 0 0 0) 0x7f968c0e6510 con 0x7f96a4076f30 2024-09-11T00:06:58.551 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:06:58.541+0000 7f96a9a00640 1 --2- 172.21.15.12:0/3452892606 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f96840b5360 0x7f96840b7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:06:58.551 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:06:58.542+0000 7f96aea25740 1 -- 172.21.15.12:0/3879150135 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a7f09120c0 msgr2=0x55a7f09144b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:06:58.551 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:06:58.542+0000 7f96aea25740 1 --2- 172.21.15.12:0/3879150135 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a7f09120c0 0x55a7f09144b0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f9694009900 tx=0x7f9694031b10 comp rx=0 tx=0).stop 2024-09-11T00:06:58.551 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:06:58.542+0000 7f96aea25740 1 -- 172.21.15.12:0/3879150135 shutdown_connections 2024-09-11T00:06:58.552 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:06:58.542+0000 7f96aea25740 1 --2- 172.21.15.12:0/3879150135 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a7f0914b00 0x55a7f0916fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:58.552 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:06:58.542+0000 7f96aea25740 1 --2- 172.21.15.12:0/3879150135 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a7f09120c0 0x55a7f09144b0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:58.552 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:06:58.542+0000 7f96aea25740 1 --2- 172.21.15.12:0/3879150135 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a7f090f790 0x55a7f0911b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:58.552 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:06:58.542+0000 7f96aea25740 1 -- 172.21.15.12:0/3879150135 >> 172.21.15.12:0/3879150135 conn(0x55a7f090a8e0 msgr2=0x55a7f090b670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:06:58.553 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:06:58.542+0000 7f96aea25740 1 -- 172.21.15.12:0/3879150135 shutdown_connections 2024-09-11T00:06:58.553 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:06:58.542+0000 7f96aea25740 1 -- 172.21.15.12:0/3879150135 wait complete. 2024-09-11T00:06:58.553 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:06:58.542+0000 7f96a9a00640 1 --2- 172.21.15.12:0/3452892606 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f96840b5360 0x7f96840b7820 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f96a419bd20 tx=0x7f969c005cc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:06:58.553 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:06:58.543+0000 7f96aea25740 1 -- start start 2024-09-11T00:06:58.553 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:06:58.543+0000 7f96aea25740 1 Processor -- start 2024-09-11T00:06:58.553 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:06:58.543+0000 7f96aea25740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a7f090f790 0x55a7f0918e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:06:58.553 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:06:58.543+0000 7f96aea25740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a7f09120c0 0x55a7f09194e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:06:58.553 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:06:58.543+0000 7f96aea25740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a7f0914b00 0x55a7f091a0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:06:58.553 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:06:58.543+0000 7f96aea25740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55a7f07a9ab0 con 0x55a7f09120c0 2024-09-11T00:06:58.553 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:06:58.543+0000 7f96aea25740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55a7f0818570 con 0x55a7f0914b00 2024-09-11T00:06:58.554 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:06:58.543+0000 7f96aea25740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55a7f0917a80 con 0x55a7f090f790 2024-09-11T00:06:58.554 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:06:58.543+0000 7f96aea25740 10 client.0 _ll_register_callbacks cb 0x55a7f0827100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:06:58.554 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:06:58.544+0000 7f96a3e00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a7f09120c0 0x55a7f09194e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:06:58.554 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:06:58.544+0000 7f96a9000640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a7f090f790 0x55a7f0918e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:06:58.554 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:06:58.544+0000 7f96a9000640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a7f090f790 0x55a7f0918e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.12:58814/0 (socket says 172.21.15.12:58814) 2024-09-11T00:06:58.554 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:06:58.544+0000 7f96a9000640 1 -- 172.21.15.12:0/1143143719 learned_addr learned my addr 172.21.15.12:0/1143143719 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:06:58.554 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:06:58.544+0000 7f96a9a00640 1 --2- 172.21.15.12:0/1143143719 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a7f0914b00 0x55a7f091a0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:06:58.554 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:06:58.544+0000 7f96a9000640 1 -- 172.21.15.12:0/1143143719 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a7f0914b00 msgr2=0x55a7f091a0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:06:58.554 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:06:58.544+0000 7f96a9000640 1 --2- 172.21.15.12:0/1143143719 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a7f0914b00 0x55a7f091a0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:58.554 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:06:58.544+0000 7f96a9000640 1 -- 172.21.15.12:0/1143143719 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a7f09120c0 msgr2=0x55a7f09194e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:06:58.554 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:06:58.544+0000 7f96a9000640 1 --2- 172.21.15.12:0/1143143719 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a7f09120c0 0x55a7f09194e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:58.554 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:06:58.544+0000 7f96a9000640 1 -- 172.21.15.12:0/1143143719 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a7f05d3790 con 0x55a7f090f790 2024-09-11T00:06:58.555 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:06:58.544+0000 7f96a3e00640 1 --2- 172.21.15.12:0/1143143719 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a7f09120c0 0x55a7f09194e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:06:58.555 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:06:58.544+0000 7f96a9000640 1 --2- 172.21.15.12:0/1143143719 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a7f090f790 0x55a7f0918e40 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f9694000c00 tx=0x7f969402ab80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=eb97b16a39a64a6f server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:06:58.555 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:06:58.544+0000 7f9692a00640 10 client.0 ms_handle_connect on v2:172.21.15.178:3300/0 2024-09-11T00:06:58.555 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:06:58.545+0000 7f9692a00640 1 -- 172.21.15.12:0/1143143719 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9694053060 con 0x55a7f090f790 2024-09-11T00:06:58.555 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:06:58.545+0000 7f96aea25740 1 -- 172.21.15.12:0/1143143719 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9694040070 con 0x55a7f090f790 2024-09-11T00:06:58.555 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:06:58.545+0000 7f9692a00640 1 -- 172.21.15.12:0/1143143719 <== mon.1 v2:172.21.15.178:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 9208+0+0 (secure 0 0 0) 0x7f96940317a0 con 0x55a7f090f790 2024-09-11T00:06:58.555 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:06:58.545+0000 7f9692a00640 1 -- 172.21.15.12:0/1143143719 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f969405a050 con 0x55a7f090f790 2024-09-11T00:06:58.555 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:06:58.545+0000 7f96aea25740 10 client.25390 fetch_fsmap learned FSMap version 29 2024-09-11T00:06:58.555 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:06:58.545+0000 7f96aea25740 1 -- 172.21.15.12:0/1143143719 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55a7f064c620 con 0x55a7f090f790 2024-09-11T00:06:58.555 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:06:58.545+0000 7f9692a00640 1 -- 172.21.15.12:0/1143143719 <== mon.1 v2:172.21.15.178:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9694040070 con 0x55a7f090f790 2024-09-11T00:06:58.556 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:06:58.546+0000 7f9692a00640 1 -- 172.21.15.12:0/1143143719 <== mon.1 v2:172.21.15.178:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7f969404d030 con 0x55a7f090f790 2024-09-11T00:06:58.556 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:06:58.546+0000 7f96aea25740 10 client.25390 fetch_fsmap finished waiting for FSMap version 29 2024-09-11T00:06:58.556 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:06:58.546+0000 7f96aea25740 10 client.25390 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T00:06:58.556 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:06:58.546+0000 7f96aea25740 20 client.25390 populate_metadata read hostname 'smithi012' 2024-09-11T00:06:58.556 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:06:58.546+0000 7f96aea25740 1 --2- 172.21.15.12:0/1143143719 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55a7f09242e0 0x55a7f09266d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:06:58.556 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:06:58.546+0000 7f96aea25740 4 client.25390 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T00:06:58.557 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:06:58.546+0000 7f96aea25740 1 -- 172.21.15.12:0/1143143719 --> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a7f0884600 con 0x55a7f09242e0 2024-09-11T00:06:58.557 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:06:58.546+0000 7f96a3e00640 1 --2- 172.21.15.12:0/1143143719 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55a7f09242e0 0x55a7f09266d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:06:58.557 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:06:58.547+0000 7f96a3e00640 1 --2- 172.21.15.12:0/1143143719 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55a7f09242e0 0x55a7f09266d0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=276e92ec7b26299c server_cookie=afb21e226eeb1ee2 in_seq=0 out_seq=0 2024-09-11T00:06:58.557 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:06:58.547+0000 7f9692a00640 10 client.25390 ms_handle_connect on v2:172.21.15.165:6838/501312984 2024-09-11T00:06:58.557 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:06:58.547+0000 7f9692a00640 1 -- 172.21.15.12:0/1143143719 <== mds.0 v2:172.21.15.165:6838/501312984 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55a7f0884600 con 0x55a7f09242e0 2024-09-11T00:06:58.557 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:06:58.547+0000 7f9692a00640 10 client.25390 handle_command_reply: tid=0 2024-09-11T00:06:58.557 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:06:58.548+0000 7f96aea25740 1 client.25390 shutdown 2024-09-11T00:06:58.557 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:06:58.548+0000 7f96aea25740 1 -- 172.21.15.12:0/1143143719 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a7f090f790 msgr2=0x55a7f0918e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:06:58.557 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:06:58.548+0000 7f96aea25740 1 --2- 172.21.15.12:0/1143143719 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a7f090f790 0x55a7f0918e40 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f9694000c00 tx=0x7f969402ab80 comp rx=0 tx=0).stop 2024-09-11T00:06:58.557 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:06:58.548+0000 7f96aea25740 1 -- 172.21.15.12:0/1143143719 shutdown_connections 2024-09-11T00:06:58.558 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:06:58.548+0000 7f96aea25740 1 --2- 172.21.15.12:0/1143143719 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55a7f09242e0 0x55a7f09266d0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:58.558 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:06:58.548+0000 7f96aea25740 1 --2- 172.21.15.12:0/1143143719 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a7f0914b00 0x55a7f091a0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:58.558 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:06:58.548+0000 7f96aea25740 1 --2- 172.21.15.12:0/1143143719 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a7f09120c0 0x55a7f09194e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:58.558 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:06:58.548+0000 7f96aea25740 1 --2- 172.21.15.12:0/1143143719 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a7f090f790 0x55a7f0918e40 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:58.558 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:06:58.548+0000 7f9692a00640 0 client.25390 ms_handle_reset on v2:172.21.15.165:6838/501312984 2024-09-11T00:06:58.558 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:06:58.548+0000 7f96aea25740 1 -- 172.21.15.12:0/1143143719 >> 172.21.15.12:0/1143143719 conn(0x55a7f090a8e0 msgr2=0x55a7f0909a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:06:58.558 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:06:58.548+0000 7f96aea25740 1 -- 172.21.15.12:0/1143143719 shutdown_connections 2024-09-11T00:06:58.558 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:06:58.548+0000 7f96aea25740 1 -- 172.21.15.12:0/1143143719 wait complete. 2024-09-11T00:06:58.558 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:06:58.548+0000 7f96aea25740 20 client.25390 trim_cache size 0 max 16384 2024-09-11T00:06:58.572 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:06:58.564+0000 7f96aea25740 1 Processor -- start 2024-09-11T00:06:58.572 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:06:58.564+0000 7f96aea25740 1 -- start start 2024-09-11T00:06:58.572 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:06:58.564+0000 7f96aea25740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a7f090f790 0x55a7f090d930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:06:58.572 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:06:58.564+0000 7f96aea25740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a7f09120c0 0x55a7f090de70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:06:58.572 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:06:58.564+0000 7f96aea25740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a7f0914b00 0x55a7f090b140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:06:58.572 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:06:58.564+0000 7f96aea25740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f96a41ad960 con 0x55a7f090f790 2024-09-11T00:06:58.572 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:06:58.564+0000 7f96aea25740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f96a41af4a0 con 0x55a7f09120c0 2024-09-11T00:06:58.573 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:06:58.564+0000 7f96aea25740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f96a41ae030 con 0x55a7f0914b00 2024-09-11T00:06:58.573 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:06:58.565+0000 7f96a9000640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a7f090f790 0x55a7f090d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:06:58.573 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:06:58.565+0000 7f96a9000640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a7f090f790 0x55a7f090d930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:54622/0 (socket says 172.21.15.12:54622) 2024-09-11T00:06:58.573 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:06:58.565+0000 7f96a3e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a7f09120c0 0x55a7f090de70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:06:58.573 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:06:58.565+0000 7f96a9000640 1 -- 172.21.15.12:0/3385704676 learned_addr learned my addr 172.21.15.12:0/3385704676 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:06:58.573 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:06:58.565+0000 7f96a9a00640 1 --2- 172.21.15.12:0/3385704676 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a7f0914b00 0x55a7f090b140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:06:58.573 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:06:58.565+0000 7f96a3e00640 1 -- 172.21.15.12:0/3385704676 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a7f0914b00 msgr2=0x55a7f090b140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:06:58.573 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:06:58.565+0000 7f96a3e00640 1 --2- 172.21.15.12:0/3385704676 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a7f0914b00 0x55a7f090b140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:58.573 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:06:58.565+0000 7f96a3e00640 1 -- 172.21.15.12:0/3385704676 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a7f090f790 msgr2=0x55a7f090d930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:06:58.573 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:06:58.565+0000 7f96a3e00640 1 --2- 172.21.15.12:0/3385704676 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a7f090f790 0x55a7f090d930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:58.573 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:06:58.565+0000 7f96a3e00640 1 -- 172.21.15.12:0/3385704676 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a7f0911ef0 con 0x55a7f09120c0 2024-09-11T00:06:58.573 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:06:58.565+0000 7f96a9000640 1 --2- 172.21.15.12:0/3385704676 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a7f090f790 0x55a7f090d930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:06:58.574 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:06:58.565+0000 7f96a3e00640 1 --2- 172.21.15.12:0/3385704676 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a7f09120c0 0x55a7f090de70 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f968c0e3c90 tx=0x7f968c0e3b10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=abb4553f69c2e02c server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:06:58.574 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:06:58.566+0000 7f9699600640 1 -- 172.21.15.12:0/3385704676 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f968c0e6070 con 0x55a7f09120c0 2024-09-11T00:06:58.574 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:06:58.566+0000 7f9699600640 1 -- 172.21.15.12:0/3385704676 <== mon.2 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f968c018420 con 0x55a7f09120c0 2024-09-11T00:06:58.574 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:06:58.566+0000 7f9699600640 1 -- 172.21.15.12:0/3385704676 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f968c0ac4c0 con 0x55a7f09120c0 2024-09-11T00:06:58.574 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:06:58.566+0000 7f96aea25740 1 -- 172.21.15.12:0/3385704676 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a7f09120c0 msgr2=0x55a7f090de70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:06:58.574 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:06:58.566+0000 7f96aea25740 1 --2- 172.21.15.12:0/3385704676 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a7f09120c0 0x55a7f090de70 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f968c0e3c90 tx=0x7f968c0e3b10 comp rx=0 tx=0).stop 2024-09-11T00:06:58.574 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:06:58.566+0000 7f96aea25740 1 -- 172.21.15.12:0/3385704676 shutdown_connections 2024-09-11T00:06:58.574 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:06:58.566+0000 7f96aea25740 1 --2- 172.21.15.12:0/3385704676 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a7f0914b00 0x55a7f090b140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:58.574 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:06:58.566+0000 7f96aea25740 1 --2- 172.21.15.12:0/3385704676 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a7f09120c0 0x55a7f090de70 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:58.574 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:06:58.566+0000 7f96aea25740 1 --2- 172.21.15.12:0/3385704676 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a7f090f790 0x55a7f090d930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:58.574 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:06:58.566+0000 7f96aea25740 1 -- 172.21.15.12:0/3385704676 >> 172.21.15.12:0/3385704676 conn(0x55a7f090a8e0 msgr2=0x55a7f090ec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:06:58.574 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:06:58.567+0000 7f96aea25740 1 -- 172.21.15.12:0/3385704676 shutdown_connections 2024-09-11T00:06:58.575 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:06:58.567+0000 7f96aea25740 1 -- 172.21.15.12:0/3385704676 wait complete. 2024-09-11T00:06:58.575 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:06:58.567+0000 7f96aea25740 1 -- start start 2024-09-11T00:06:58.575 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:06:58.567+0000 7f96aea25740 1 Processor -- start 2024-09-11T00:06:58.575 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:06:58.567+0000 7f96aea25740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a7f090f790 0x55a7f091fa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:06:58.575 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:06:58.568+0000 7f96aea25740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a7f09120c0 0x55a7f091ff60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:06:58.575 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:06:58.568+0000 7f96aea25740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a7f0914b00 0x55a7f09204a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:06:58.575 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:06:58.568+0000 7f96aea25740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f96a41aedd0 con 0x55a7f0914b00 2024-09-11T00:06:58.575 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:06:58.568+0000 7f96aea25740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f96a41ae700 con 0x55a7f090f790 2024-09-11T00:06:58.575 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:06:58.568+0000 7f96aea25740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f96a41ab940 con 0x55a7f09120c0 2024-09-11T00:06:58.575 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:06:58.568+0000 7f96aea25740 10 client.0 _ll_register_callbacks cb 0x7f968c0e9ec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:06:58.575 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:06:58.568+0000 7f96a9a00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a7f0914b00 0x55a7f09204a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:06:58.575 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:06:58.568+0000 7f96a9a00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a7f0914b00 0x55a7f09204a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:54638/0 (socket says 172.21.15.12:54638) 2024-09-11T00:06:58.575 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:06:58.568+0000 7f96a9a00640 1 -- 172.21.15.12:0/546646346 learned_addr learned my addr 172.21.15.12:0/546646346 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:06:58.575 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:06:58.568+0000 7f96a3e00640 1 --2- 172.21.15.12:0/546646346 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a7f09120c0 0x55a7f091ff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:06:58.576 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:06:58.568+0000 7f96a9000640 1 --2- 172.21.15.12:0/546646346 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a7f090f790 0x55a7f091fa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:06:58.576 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:06:58.568+0000 7f96a9a00640 1 -- 172.21.15.12:0/546646346 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a7f09120c0 msgr2=0x55a7f091ff60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:06:58.576 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:06:58.568+0000 7f96a9a00640 1 --2- 172.21.15.12:0/546646346 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a7f09120c0 0x55a7f091ff60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:58.576 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:06:58.568+0000 7f96a9a00640 1 -- 172.21.15.12:0/546646346 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a7f090f790 msgr2=0x55a7f091fa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:06:58.576 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:06:58.568+0000 7f96a9a00640 1 --2- 172.21.15.12:0/546646346 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a7f090f790 0x55a7f091fa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:58.576 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:06:58.568+0000 7f96a9a00640 1 -- 172.21.15.12:0/546646346 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a7f091e880 con 0x55a7f0914b00 2024-09-11T00:06:58.576 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:06:58.569+0000 7f96a9a00640 1 --2- 172.21.15.12:0/546646346 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a7f0914b00 0x55a7f09204a0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f969c003cd0 tx=0x7f969c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ee1c0b872e186c1 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:06:58.576 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:06:58.569+0000 7f9692a00640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-09-11T00:06:58.576 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:06:58.569+0000 7f9692a00640 1 -- 172.21.15.12:0/546646346 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f969c02e060 con 0x55a7f0914b00 2024-09-11T00:06:58.576 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:06:58.569+0000 7f9692a00640 1 -- 172.21.15.12:0/546646346 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 9208+0+0 (secure 0 0 0) 0x7f969c01ebb0 con 0x55a7f0914b00 2024-09-11T00:06:58.576 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:06:58.569+0000 7f96aea25740 1 -- 172.21.15.12:0/546646346 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55a7f08800b0 con 0x55a7f0914b00 2024-09-11T00:06:58.576 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:06:58.569+0000 7f9692a00640 1 -- 172.21.15.12:0/546646346 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f969c035050 con 0x55a7f0914b00 2024-09-11T00:06:58.577 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:06:58.569+0000 7f96aea25740 10 client.15375 fetch_fsmap learned FSMap version 29 2024-09-11T00:06:58.577 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:06:58.569+0000 7f96aea25740 1 -- 172.21.15.12:0/546646346 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55a7f09916d0 con 0x55a7f0914b00 2024-09-11T00:06:58.577 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:06:58.570+0000 7f9692a00640 1 -- 172.21.15.12:0/546646346 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55a7f08800b0 con 0x55a7f0914b00 2024-09-11T00:06:58.577 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:06:58.570+0000 7f9692a00640 1 -- 172.21.15.12:0/546646346 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7f969c029400 con 0x55a7f0914b00 2024-09-11T00:06:58.577 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:06:58.570+0000 7f96aea25740 10 client.15375 fetch_fsmap finished waiting for FSMap version 29 2024-09-11T00:06:58.577 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:06:58.570+0000 7f96aea25740 10 client.15375 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T00:06:58.577 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:06:58.570+0000 7f96aea25740 20 client.15375 populate_metadata read hostname 'smithi012' 2024-09-11T00:06:58.577 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:06:58.570+0000 7f96aea25740 1 --2- 172.21.15.12:0/546646346 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55a7f09920e0 0x55a7f0994500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:06:58.578 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:06:58.570+0000 7f96aea25740 4 client.15375 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T00:06:58.578 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:06:58.570+0000 7f96aea25740 1 -- 172.21.15.12:0/546646346 --> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55a7f087d4e0 con 0x55a7f09920e0 2024-09-11T00:06:58.578 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:06:58.570+0000 7f96a9000640 1 --2- 172.21.15.12:0/546646346 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55a7f09920e0 0x55a7f0994500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:06:58.578 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:06:58.571+0000 7f96a9000640 1 --2- 172.21.15.12:0/546646346 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55a7f09920e0 0x55a7f0994500 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8426ecda1e76581d server_cookie=28fa3323607ea018 in_seq=0 out_seq=0 2024-09-11T00:06:58.578 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:06:58.571+0000 7f9692a00640 10 client.15375 ms_handle_connect on v2:172.21.15.165:6838/501312984 2024-09-11T00:06:58.585 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:06:58.583+0000 7f9692a00640 1 -- 172.21.15.12:0/546646346 <== mds.0 v2:172.21.15.165:6838/501312984 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55a7f087d4e0 con 0x55a7f09920e0 2024-09-11T00:06:58.586 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:06:58.583+0000 7f9692a00640 10 client.15375 handle_command_reply: tid=0 2024-09-11T00:06:58.586 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:06:58.583+0000 7f96aea25740 1 client.15375 shutdown 2024-09-11T00:06:58.586 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:06:58.583+0000 7f96aea25740 1 -- 172.21.15.12:0/546646346 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a7f0914b00 msgr2=0x55a7f09204a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:06:58.586 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:06:58.583+0000 7f96aea25740 1 --2- 172.21.15.12:0/546646346 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a7f0914b00 0x55a7f09204a0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f969c003cd0 tx=0x7f969c0046c0 comp rx=0 tx=0).stop 2024-09-11T00:06:58.586 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:06:58.583+0000 7f96aea25740 1 -- 172.21.15.12:0/546646346 shutdown_connections 2024-09-11T00:06:58.586 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:06:58.583+0000 7f96aea25740 1 --2- 172.21.15.12:0/546646346 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55a7f09920e0 0x55a7f0994500 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:58.586 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:06:58.583+0000 7f96aea25740 1 --2- 172.21.15.12:0/546646346 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a7f0914b00 0x55a7f09204a0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:58.586 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:06:58.583+0000 7f96aea25740 1 --2- 172.21.15.12:0/546646346 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a7f09120c0 0x55a7f091ff60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:58.586 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:06:58.583+0000 7f96aea25740 1 --2- 172.21.15.12:0/546646346 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a7f090f790 0x55a7f091fa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:58.586 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:06:58.583+0000 7f96aea25740 1 -- 172.21.15.12:0/546646346 >> 172.21.15.12:0/546646346 conn(0x55a7f090a8e0 msgr2=0x55a7f090eba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:06:58.587 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:06:58.583+0000 7f9692a00640 0 client.15375 ms_handle_reset on v2:172.21.15.165:6838/501312984 2024-09-11T00:06:58.587 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:06:58.583+0000 7f96aea25740 1 -- 172.21.15.12:0/546646346 shutdown_connections 2024-09-11T00:06:58.587 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:06:58.583+0000 7f96aea25740 1 -- 172.21.15.12:0/546646346 wait complete. 2024-09-11T00:06:58.587 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:06:58.584+0000 7f96aea25740 20 client.15375 trim_cache size 0 max 16384 2024-09-11T00:06:58.587 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:{ 2024-09-11T00:06:58.588 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "message": "", 2024-09-11T00:06:58.588 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "return_code": 0 2024-09-11T00:06:58.588 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:} 2024-09-11T00:06:58.589 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:06:58.584+0000 7f96aba00640 1 -- 172.21.15.12:0/3452892606 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f96840b5360 msgr2=0x7f96840b7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:06:58.589 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:06:58.584+0000 7f96aba00640 1 --2- 172.21.15.12:0/3452892606 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f96840b5360 0x7f96840b7820 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f96a419bd20 tx=0x7f969c005cc0 comp rx=0 tx=0).stop 2024-09-11T00:06:58.589 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:06:58.584+0000 7f96aba00640 1 -- 172.21.15.12:0/3452892606 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f96a4076f30 msgr2=0x7f96a419b5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:06:58.589 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:06:58.584+0000 7f96aba00640 1 --2- 172.21.15.12:0/3452892606 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f96a4076f30 0x7f96a419b5f0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f968c00ebc0 tx=0x7f968c00c6a0 comp rx=0 tx=0).stop 2024-09-11T00:06:58.589 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:06:58.584+0000 7f96aba00640 1 -- 172.21.15.12:0/3452892606 shutdown_connections 2024-09-11T00:06:58.590 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:06:58.584+0000 7f96aba00640 1 --2- 172.21.15.12:0/3452892606 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f96840b5360 0x7f96840b7820 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:58.590 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:06:58.584+0000 7f96aba00640 1 --2- 172.21.15.12:0/3452892606 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f96a410a910 0x7f96a419c220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:58.590 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:06:58.584+0000 7f96aba00640 1 --2- 172.21.15.12:0/3452892606 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f96a4076f30 0x7f96a419b5f0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:58.590 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:06:58.584+0000 7f96aba00640 1 --2- 172.21.15.12:0/3452892606 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f96a4075ac0 0x7f96a419afa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:06:58.590 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:06:58.584+0000 7f96aba00640 1 -- 172.21.15.12:0/3452892606 >> 172.21.15.12:0/3452892606 conn(0x7f96a40fe250 msgr2=0x7f96a41068f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:06:58.590 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:06:58.585+0000 7f96aba00640 1 -- 172.21.15.12:0/3452892606 shutdown_connections 2024-09-11T00:06:58.590 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:06:58.585+0000 7f96aba00640 1 -- 172.21.15.12:0/3452892606 wait complete. 2024-09-11T00:06:59.414 INFO:teuthology.orchestra.run.smithi012.stdout:{"run_dir":"/var/run/ceph"} 2024-09-11T00:06:59.415 DEBUG:tasks.cephfs.filesystem:_json_asok output { "run_dir": "/var/run/ceph" } 2024-09-11T00:06:59.415 INFO:tasks.quiescer.fs.[cephfs]:Initializing Quiescer with seed 6190 2024-09-11T00:06:59.416 DEBUG:tasks.quiescer:Yielding 2024-09-11T00:06:59.416 INFO:teuthology.run_tasks:Running task workunit... 2024-09-11T00:06:59.441 INFO:tasks.workunit:Pulling workunits from ref 8f219c476cb4f26fc5719c661c0323719a45cb30 2024-09-11T00:06:59.442 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-09-11T00:06:59.442 INFO:tasks.workunit:timeout=3h 2024-09-11T00:06:59.442 INFO:tasks.workunit:cleanup=True 2024-09-11T00:06:59.442 DEBUG:teuthology.orchestra.run.smithi012:> stat -- /home/ubuntu/cephtest/mnt.0 2024-09-11T00:06:59.443 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid ff3ad74a-6fcf-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T00:06:59.480 INFO:teuthology.orchestra.run.smithi012.stdout: File: /home/ubuntu/cephtest/mnt.0 2024-09-11T00:06:59.480 INFO:teuthology.orchestra.run.smithi012.stdout: Size: 0 Blocks: 0 IO Block: 65536 directory 2024-09-11T00:06:59.480 INFO:teuthology.orchestra.run.smithi012.stdout:Device: 78h/120d Inode: 1099511628283 Links: 2 2024-09-11T00:06:59.480 INFO:teuthology.orchestra.run.smithi012.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-11T00:06:59.480 INFO:teuthology.orchestra.run.smithi012.stdout:Context: system_u:object_r:unlabeled_t:s0 2024-09-11T00:06:59.480 INFO:teuthology.orchestra.run.smithi012.stdout:Access: 2024-09-11 00:04:57.566742941 +0000 2024-09-11T00:06:59.480 INFO:teuthology.orchestra.run.smithi012.stdout:Modify: 2024-09-11 00:04:57.566742941 +0000 2024-09-11T00:06:59.480 INFO:teuthology.orchestra.run.smithi012.stdout:Change: 2024-09-11 00:06:43.867821412 +0000 2024-09-11T00:06:59.481 INFO:teuthology.orchestra.run.smithi012.stdout: Birth: 2024-09-11 00:04:57.566742941 +0000 2024-09-11T00:06:59.481 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.0 2024-09-11T00:06:59.481 DEBUG:teuthology.orchestra.run.smithi012:> cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0 2024-09-11T00:06:59.569 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:06:59 smithi012 ceph-mon[30333]: pgmap v395: 129 pgs: 129 active+clean; 652 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 597 B/s wr, 8 op/s 2024-09-11T00:06:59.580 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:06:59 smithi178 ceph-mon[35919]: pgmap v395: 129 pgs: 129 active+clean; 652 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 597 B/s wr, 8 op/s 2024-09-11T00:06:59.593 DEBUG:teuthology.orchestra.run.smithi012:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://github.com/ceph/ceph-ci.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout 8f219c476cb4f26fc5719c661c0323719a45cb30 2024-09-11T00:06:59.631 INFO:tasks.workunit.client.0.smithi012.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-09-11T00:06:59.782 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:06:59 smithi165 ceph-mon[35356]: pgmap v395: 129 pgs: 129 active+clean; 652 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 597 B/s wr, 8 op/s 2024-09-11T00:06:59.843 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:07:00.396 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:00.383+0000 7f615ac00640 1 -- 172.21.15.12:0/2481899731 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6154113980 msgr2=0x7f6154115d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:07:00.397 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:00.383+0000 7f615ac00640 1 --2- 172.21.15.12:0/2481899731 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6154113980 0x7f6154115d70 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f614c009ca0 tx=0x7f614c02a8c0 comp rx=0 tx=0).stop 2024-09-11T00:07:00.397 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:00.384+0000 7f615ac00640 1 -- 172.21.15.12:0/2481899731 shutdown_connections 2024-09-11T00:07:00.397 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:00.384+0000 7f615ac00640 1 --2- 172.21.15.12:0/2481899731 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6154113980 0x7f6154115d70 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:07:00.397 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:00.384+0000 7f615ac00640 1 --2- 172.21.15.12:0/2481899731 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6154077ff0 0x7f6154113440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:07:00.397 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:00.384+0000 7f615ac00640 1 --2- 172.21.15.12:0/2481899731 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f61540776d0 0x7f6154077ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:07:00.398 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:00.384+0000 7f615ac00640 1 -- 172.21.15.12:0/2481899731 >> 172.21.15.12:0/2481899731 conn(0x7f6154100a00 msgr2=0x7f6154102e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:07:00.398 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:00.384+0000 7f615ac00640 1 -- 172.21.15.12:0/2481899731 shutdown_connections 2024-09-11T00:07:00.398 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:00.384+0000 7f615ac00640 1 -- 172.21.15.12:0/2481899731 wait complete. 2024-09-11T00:07:00.398 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:00.385+0000 7f615ac00640 1 Processor -- start 2024-09-11T00:07:00.398 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:00.385+0000 7f615ac00640 1 -- start start 2024-09-11T00:07:00.398 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:00.385+0000 7f615ac00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f61540776d0 0x7f615419f8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:07:00.399 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:00.385+0000 7f615ac00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6154077ff0 0x7f615419fe00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:07:00.399 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:00.385+0000 7f615ac00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6154113980 0x7f61541a51a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:07:00.399 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:00.385+0000 7f615ac00640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f6154119350 con 0x7f61540776d0 2024-09-11T00:07:00.399 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:00.385+0000 7f615ac00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f61541191d0 con 0x7f6154077ff0 2024-09-11T00:07:00.399 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:00.385+0000 7f615ac00640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f61541194d0 con 0x7f6154113980 2024-09-11T00:07:00.399 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:00.385+0000 7f6153e00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f61540776d0 0x7f615419f8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:07:00.400 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:00.385+0000 7f6153e00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f61540776d0 0x7f615419f8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:46916/0 (socket says 172.21.15.12:46916) 2024-09-11T00:07:00.400 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:00.385+0000 7f6153e00640 1 -- 172.21.15.12:0/2881854498 learned_addr learned my addr 172.21.15.12:0/2881854498 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:07:00.400 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:00.385+0000 7f614b400640 1 --2- 172.21.15.12:0/2881854498 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6154077ff0 0x7f615419fe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:07:00.400 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:00.385+0000 7f6158c00640 1 --2- 172.21.15.12:0/2881854498 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6154113980 0x7f61541a51a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:07:00.400 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:00.386+0000 7f6153e00640 1 -- 172.21.15.12:0/2881854498 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6154113980 msgr2=0x7f61541a51a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:07:00.401 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:00.386+0000 7f6153e00640 1 --2- 172.21.15.12:0/2881854498 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6154113980 0x7f61541a51a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:07:00.401 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:00.386+0000 7f6153e00640 1 -- 172.21.15.12:0/2881854498 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6154077ff0 msgr2=0x7f615419fe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:07:00.401 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:00.386+0000 7f6153e00640 1 --2- 172.21.15.12:0/2881854498 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6154077ff0 0x7f615419fe00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:07:00.401 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:00.386+0000 7f6153e00640 1 -- 172.21.15.12:0/2881854498 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61541a5970 con 0x7f61540776d0 2024-09-11T00:07:00.401 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:00.386+0000 7f6153e00640 1 --2- 172.21.15.12:0/2881854498 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f61540776d0 0x7f615419f8c0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f614400e9f0 tx=0x7f614400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:07:00.401 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:00.386+0000 7f6151600640 1 -- 172.21.15.12:0/2881854498 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f614400cd80 con 0x7f61540776d0 2024-09-11T00:07:00.402 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:00.386+0000 7f6151600640 1 -- 172.21.15.12:0/2881854498 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f61440108e0 con 0x7f61540776d0 2024-09-11T00:07:00.402 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:00.386+0000 7f6151600640 1 -- 172.21.15.12:0/2881854498 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6144018980 con 0x7f61540776d0 2024-09-11T00:07:00.402 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:00.387+0000 7f615ac00640 1 -- 172.21.15.12:0/2881854498 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f61541a5c60 con 0x7f61540776d0 2024-09-11T00:07:00.402 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:00.387+0000 7f615ac00640 1 -- 172.21.15.12:0/2881854498 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61541ac320 con 0x7f61540776d0 2024-09-11T00:07:00.402 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:00.388+0000 7f615ac00640 1 -- 172.21.15.12:0/2881854498 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6154077ab0 con 0x7f61540776d0 2024-09-11T00:07:00.403 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:00.390+0000 7f6151600640 1 -- 172.21.15.12:0/2881854498 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7f6144010430 con 0x7f61540776d0 2024-09-11T00:07:00.403 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:00.391+0000 7f6151600640 1 --2- 172.21.15.12:0/2881854498 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f61380b5360 0x7f61380b7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:07:00.403 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:00.391+0000 7f6151600640 1 -- 172.21.15.12:0/2881854498 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9208+0+0 (secure 0 0 0) 0x7f61440e6260 con 0x7f61540776d0 2024-09-11T00:07:00.403 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:00.391+0000 7f614b400640 1 --2- 172.21.15.12:0/2881854498 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f61380b5360 0x7f61380b7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:07:00.403 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:00.392+0000 7f614b400640 1 --2- 172.21.15.12:0/2881854498 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f61380b5360 0x7f61380b7820 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f6154115de0 tx=0x7f613c00b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:07:00.404 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:00.395+0000 7f6151600640 1 -- 172.21.15.12:0/2881854498 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f61440ac6a0 con 0x7f61540776d0 2024-09-11T00:07:00.629 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:00.627+0000 7f615ac00640 1 -- 172.21.15.12:0/2881854498 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f615402e5c0 con 0x7f61540776d0 2024-09-11T00:07:00.629 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:00.628+0000 7f6151600640 1 -- 172.21.15.12:0/2881854498 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11965 (secure 0 0 0) 0x7f61440b10d0 con 0x7f61540776d0 2024-09-11T00:07:00.630 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-11T00:07:00.630 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":29,"btime":"2024-09-11T00:06:45:273053+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14577,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6835/447171591","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":447171591},{"type":"v1","addr":"172.21.15.12:6835","nonce":447171591}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14589,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/901486040","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":901486040},{"type":"v1","addr":"172.21.15.12:6837","nonce":901486040}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T00:04:04.739881+0000","modified":"2024-09-11T00:06:44.753824+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24674,"mds_1":24643,"mds_2":24638,"mds_3":24637,"mds_4":24614},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14631":{"gid":14631,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.12:6841/3069919437","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6840","nonce":3069919437},{"type":"v1","addr":"172.21.15.12:6841","nonce":3069919437}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24562":{"gid":24562,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6833/1141636112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":1141636112},{"type":"v1","addr":"172.21.15.178:6833","nonce":1141636112}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24581":{"gid":24581,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6833/3111980970","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6832","nonce":3111980970},{"type":"v1","addr":"172.21.15.165:6833","nonce":3111980970}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24607":{"gid":24607,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6835/4084868287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":4084868287},{"type":"v1","addr":"172.21.15.178:6835","nonce":4084868287}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24613":{"gid":24613,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6837/526874763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":526874763},{"type":"v1","addr":"172.21.15.178:6837","nonce":526874763}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24614":{"gid":24614,"name":"e","rank":4,"incarnation":26,"state":"up:active","state_seq":21,"addr":"172.21.15.165:6835/2425395274","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":2425395274},{"type":"v1","addr":"172.21.15.165:6835","nonce":2425395274}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24637":{"gid":24637,"name":"i","rank":3,"incarnation":24,"state":"up:active","state_seq":18,"addr":"172.21.15.178:6839/70628097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6838","nonce":70628097},{"type":"v1","addr":"172.21.15.178:6839","nonce":70628097}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24638":{"gid":24638,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":18,"addr":"172.21.15.165:6837/6215088","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6836","nonce":6215088},{"type":"v1","addr":"172.21.15.165:6837","nonce":6215088}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24643":{"gid":24643,"name":"g","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.12:6839/3124093009","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6838","nonce":3124093009},{"type":"v1","addr":"172.21.15.12:6839","nonce":3124093009}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24674":{"gid":24674,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.165:6839/501312984","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6838","nonce":501312984},{"type":"v1","addr":"172.21.15.165:6839","nonce":501312984}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24674,"qdb_cluster":[24614,24637,24638,24643,24674]},"id":1}]} 2024-09-11T00:07:00.630 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 29 2024-09-11T00:07:00.632 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:00.630+0000 7f615ac00640 1 -- 172.21.15.12:0/2881854498 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f61380b5360 msgr2=0x7f61380b7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:07:00.632 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:00.630+0000 7f615ac00640 1 --2- 172.21.15.12:0/2881854498 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f61380b5360 0x7f61380b7820 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f6154115de0 tx=0x7f613c00b410 comp rx=0 tx=0).stop 2024-09-11T00:07:00.632 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:00.630+0000 7f615ac00640 1 -- 172.21.15.12:0/2881854498 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f61540776d0 msgr2=0x7f615419f8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:07:00.632 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:00.630+0000 7f615ac00640 1 --2- 172.21.15.12:0/2881854498 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f61540776d0 0x7f615419f8c0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f614400e9f0 tx=0x7f614400eeb0 comp rx=0 tx=0).stop 2024-09-11T00:07:00.633 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:00.630+0000 7f615ac00640 1 -- 172.21.15.12:0/2881854498 shutdown_connections 2024-09-11T00:07:00.633 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:00.630+0000 7f615ac00640 1 --2- 172.21.15.12:0/2881854498 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f61380b5360 0x7f61380b7820 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:07:00.633 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:00.630+0000 7f615ac00640 1 --2- 172.21.15.12:0/2881854498 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6154113980 0x7f61541a51a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:07:00.633 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:00.630+0000 7f615ac00640 1 --2- 172.21.15.12:0/2881854498 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6154077ff0 0x7f615419fe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:07:00.633 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:00.630+0000 7f615ac00640 1 --2- 172.21.15.12:0/2881854498 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f61540776d0 0x7f615419f8c0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:07:00.633 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:00.630+0000 7f615ac00640 1 -- 172.21.15.12:0/2881854498 >> 172.21.15.12:0/2881854498 conn(0x7f6154100a00 msgr2=0x7f6154102df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:07:00.633 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:00.630+0000 7f615ac00640 1 -- 172.21.15.12:0/2881854498 shutdown_connections 2024-09-11T00:07:00.633 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:00.631+0000 7f615ac00640 1 -- 172.21.15.12:0/2881854498 wait complete. 2024-09-11T00:07:01.581 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:07:01 smithi178 ceph-mon[35919]: pgmap v396: 129 pgs: 129 active+clean; 653 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 1.2 KiB/s wr, 12 op/s 2024-09-11T00:07:01.581 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:07:01 smithi178 ceph-mon[35919]: from='client.? 172.21.15.12:0/2881854498' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T00:07:01.610 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:07:01 smithi012 ceph-mon[30333]: pgmap v396: 129 pgs: 129 active+clean; 653 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 1.2 KiB/s wr, 12 op/s 2024-09-11T00:07:01.610 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:07:01 smithi012 ceph-mon[30333]: from='client.? 172.21.15.12:0/2881854498' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T00:07:01.650 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 29, 'flags': 306, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': True}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-09-11T00:04:04.739881+0000', 'modified': '2024-09-11T00:06:44.753824+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 24674, 'mds_1': 24643, 'mds_2': 24638, 'mds_3': 24637, 'mds_4': 24614}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_14631': {'gid': 14631, 'name': 'a', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.12:6841/3069919437', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.12:6840', 'nonce': 3069919437}, {'type': 'v1', 'addr': '172.21.15.12:6841', 'nonce': 3069919437}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24562': {'gid': 24562, 'name': 'l', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.178:6833/1141636112', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.178:6832', 'nonce': 1141636112}, {'type': 'v1', 'addr': '172.21.15.178:6833', 'nonce': 1141636112}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24581': {'gid': 24581, 'name': 'k', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.165:6833/3111980970', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.165:6832', 'nonce': 3111980970}, {'type': 'v1', 'addr': '172.21.15.165:6833', 'nonce': 3111980970}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24607': {'gid': 24607, 'name': 'f', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.178:6835/4084868287', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.178:6834', 'nonce': 4084868287}, {'type': 'v1', 'addr': '172.21.15.178:6835', 'nonce': 4084868287}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24613': {'gid': 24613, 'name': 'c', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.178:6837/526874763', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.178:6836', 'nonce': 526874763}, {'type': 'v1', 'addr': '172.21.15.178:6837', 'nonce': 526874763}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24614': {'gid': 24614, 'name': 'e', 'rank': 4, 'incarnation': 26, 'state': 'up:active', 'state_seq': 21, 'addr': '172.21.15.165:6835/2425395274', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.165:6834', 'nonce': 2425395274}, {'type': 'v1', 'addr': '172.21.15.165:6835', 'nonce': 2425395274}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24637': {'gid': 24637, 'name': 'i', 'rank': 3, 'incarnation': 24, 'state': 'up:active', 'state_seq': 18, 'addr': '172.21.15.178:6839/70628097', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.178:6838', 'nonce': 70628097}, {'type': 'v1', 'addr': '172.21.15.178:6839', 'nonce': 70628097}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24638': {'gid': 24638, 'name': 'h', 'rank': 2, 'incarnation': 22, 'state': 'up:active', 'state_seq': 18, 'addr': '172.21.15.165:6837/6215088', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.165:6836', 'nonce': 6215088}, {'type': 'v1', 'addr': '172.21.15.165:6837', 'nonce': 6215088}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24643': {'gid': 24643, 'name': 'g', 'rank': 1, 'incarnation': 20, 'state': 'up:active', 'state_seq': 17, 'addr': '172.21.15.12:6839/3124093009', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.12:6838', 'nonce': 3124093009}, {'type': 'v1', 'addr': '172.21.15.12:6839', 'nonce': 3124093009}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24674': {'gid': 24674, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.165:6839/501312984', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.165:6838', 'nonce': 501312984}, {'type': 'v1', 'addr': '172.21.15.165:6839', 'nonce': 501312984}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24674, 'qdb_cluster': [24614, 24637, 24638, 24643, 24674]} 2024-09-11T00:07:01.651 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2024-09-11T00:07:01.651 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid ff3ad74a-6fcf-11ef-bcea-c7b262605968 -- ceph tell mds.e status 2024-09-11T00:07:01.782 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:07:01 smithi165 ceph-mon[35356]: pgmap v396: 129 pgs: 129 active+clean; 653 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 1.2 KiB/s wr, 12 op/s 2024-09-11T00:07:01.782 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:07:01 smithi165 ceph-mon[35356]: from='client.? 172.21.15.12:0/2881854498' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T00:07:02.074 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:07:02.720 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:02.717+0000 7f6506c00640 1 -- 172.21.15.12:0/3588936366 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f650010ede0 msgr2=0x7f650010f220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:07:02.720 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:02.717+0000 7f6506c00640 1 --2- 172.21.15.12:0/3588936366 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f650010ede0 0x7f650010f220 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f64e8009900 tx=0x7f64e802a410 comp rx=0 tx=0).stop 2024-09-11T00:07:02.720 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:02.717+0000 7f6506c00640 1 -- 172.21.15.12:0/3588936366 shutdown_connections 2024-09-11T00:07:02.720 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:02.717+0000 7f6506c00640 1 --2- 172.21.15.12:0/3588936366 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f650010f760 0x7f6500115f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:07:02.720 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:02.717+0000 7f6506c00640 1 --2- 172.21.15.12:0/3588936366 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f650010ede0 0x7f650010f220 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:07:02.720 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:02.718+0000 7f6506c00640 1 --2- 172.21.15.12:0/3588936366 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f65001093d0 0x7f65001097b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:07:02.721 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:02.718+0000 7f6506c00640 1 -- 172.21.15.12:0/3588936366 >> 172.21.15.12:0/3588936366 conn(0x7f6500078a40 msgr2=0x7f650007ae60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:07:02.721 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:02.718+0000 7f6506c00640 1 -- 172.21.15.12:0/3588936366 shutdown_connections 2024-09-11T00:07:02.721 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:02.718+0000 7f6506c00640 1 -- 172.21.15.12:0/3588936366 wait complete. 2024-09-11T00:07:02.722 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:02.719+0000 7f6506c00640 1 Processor -- start 2024-09-11T00:07:02.722 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:02.719+0000 7f6506c00640 1 -- start start 2024-09-11T00:07:02.722 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:02.719+0000 7f6506c00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f65001093d0 0x7f650019f8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:07:02.722 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:02.719+0000 7f6506c00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f650010ede0 0x7f650019fde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:07:02.722 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:02.719+0000 7f6506c00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f650010f760 0x7f65001a5180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:07:02.722 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:02.719+0000 7f6506c00640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f6500119390 con 0x7f650010f760 2024-09-11T00:07:02.722 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:02.719+0000 7f6506c00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f6500119210 con 0x7f65001093d0 2024-09-11T00:07:02.722 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:02.719+0000 7f6506c00640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f6500119510 con 0x7f650010ede0 2024-09-11T00:07:02.722 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:02.720+0000 7f6504c00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f650010f760 0x7f65001a5180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:07:02.722 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:02.720+0000 7f64ffe00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f65001093d0 0x7f650019f8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:07:02.722 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:02.720+0000 7f6504c00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f650010f760 0x7f65001a5180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:46924/0 (socket says 172.21.15.12:46924) 2024-09-11T00:07:02.722 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:02.720+0000 7f6504c00640 1 -- 172.21.15.12:0/266414925 learned_addr learned my addr 172.21.15.12:0/266414925 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:07:02.723 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:02.720+0000 7f64f7400640 1 --2- 172.21.15.12:0/266414925 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f650010ede0 0x7f650019fde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:07:02.723 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:02.720+0000 7f6504c00640 1 -- 172.21.15.12:0/266414925 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f650010ede0 msgr2=0x7f650019fde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:07:02.723 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:02.720+0000 7f6504c00640 1 --2- 172.21.15.12:0/266414925 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f650010ede0 0x7f650019fde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:07:02.724 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:02.720+0000 7f6504c00640 1 -- 172.21.15.12:0/266414925 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f65001093d0 msgr2=0x7f650019f8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:07:02.724 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:02.720+0000 7f6504c00640 1 --2- 172.21.15.12:0/266414925 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f65001093d0 0x7f650019f8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:07:02.724 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:02.720+0000 7f6504c00640 1 -- 172.21.15.12:0/266414925 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65001a5950 con 0x7f650010f760 2024-09-11T00:07:02.724 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:02.720+0000 7f6504c00640 1 --2- 172.21.15.12:0/266414925 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f650010f760 0x7f65001a5180 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f64f800eb10 tx=0x7f64f800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:07:02.724 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:02.721+0000 7f64fd600640 1 -- 172.21.15.12:0/266414925 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64f8004b20 con 0x7f650010f760 2024-09-11T00:07:02.724 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:02.721+0000 7f64fd600640 1 -- 172.21.15.12:0/266414925 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f64f8004cc0 con 0x7f650010f760 2024-09-11T00:07:02.724 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:02.721+0000 7f64fd600640 1 -- 172.21.15.12:0/266414925 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64f8010670 con 0x7f650010f760 2024-09-11T00:07:02.724 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:02.721+0000 7f6506c00640 1 -- 172.21.15.12:0/266414925 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65001a5c40 con 0x7f650010f760 2024-09-11T00:07:02.724 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:02.721+0000 7f6506c00640 1 -- 172.21.15.12:0/266414925 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65001ac300 con 0x7f650010f760 2024-09-11T00:07:02.726 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:02.724+0000 7f6509811740 1 Processor -- start 2024-09-11T00:07:02.726 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:02.724+0000 7f64fd600640 1 -- 172.21.15.12:0/266414925 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7f64f8005020 con 0x7f650010f760 2024-09-11T00:07:02.726 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:02.725+0000 7f6509811740 1 -- start start 2024-09-11T00:07:02.728 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:02.725+0000 7f6509811740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5638ea064240 0x5638ea064620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:07:02.728 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:02.725+0000 7f6509811740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5638ea064b60 0x5638ea0705b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:07:02.728 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:02.725+0000 7f6509811740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5638ea070af0 0x5638ea072ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:07:02.728 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:02.725+0000 7f6509811740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x5638e9fdbd50 con 0x5638ea064b60 2024-09-11T00:07:02.728 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:02.725+0000 7f6509811740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x5638e9fcb180 con 0x5638ea070af0 2024-09-11T00:07:02.728 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:02.725+0000 7f6509811740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5638e9fcef90 con 0x5638ea064240 2024-09-11T00:07:02.728 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:02.725+0000 7f64ffe00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5638ea064b60 0x5638ea0705b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:07:02.728 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:02.725+0000 7f64f7400640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5638ea064240 0x5638ea064620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:07:02.728 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:02.725+0000 7f64ffe00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5638ea064b60 0x5638ea0705b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:46930/0 (socket says 172.21.15.12:46930) 2024-09-11T00:07:02.729 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:02.725+0000 7f64ffe00640 1 -- 172.21.15.12:0/1674684881 learned_addr learned my addr 172.21.15.12:0/1674684881 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:07:02.729 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:02.725+0000 7f64f7400640 1 --2- 172.21.15.12:0/1674684881 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5638ea070af0 0x5638ea072ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:07:02.729 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:02.726+0000 7f64ffe00640 1 -- 172.21.15.12:0/1674684881 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5638ea064240 msgr2=0x5638ea064620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:07:02.729 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:02.726+0000 7f64ffe00640 1 --2- 172.21.15.12:0/1674684881 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5638ea064240 0x5638ea064620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:07:02.729 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:02.726+0000 7f64ffe00640 1 -- 172.21.15.12:0/1674684881 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5638ea070af0 msgr2=0x5638ea072ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:07:02.729 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:02.726+0000 7f64ffe00640 1 --2- 172.21.15.12:0/1674684881 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5638ea070af0 0x5638ea072ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:07:02.729 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:02.726+0000 7f64ffe00640 1 -- 172.21.15.12:0/1674684881 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5638e9f95a70 con 0x5638ea064b60 2024-09-11T00:07:02.730 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:02.726+0000 7f64ffe00640 1 --2- 172.21.15.12:0/1674684881 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5638ea064b60 0x5638ea0705b0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f64f0009d20 tx=0x7f64f0011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1414356b252d882d server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:07:02.731 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:02.726+0000 7f64f5200640 1 -- 172.21.15.12:0/1674684881 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64f0004230 con 0x5638ea064b60 2024-09-11T00:07:02.731 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:02.727+0000 7f64fd600640 1 --2- 172.21.15.12:0/266414925 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f64e40b52e0 0x7f64e40b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:07:02.731 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:02.727+0000 7f64f5200640 1 -- 172.21.15.12:0/1674684881 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f64f00043d0 con 0x5638ea064b60 2024-09-11T00:07:02.731 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:02.727+0000 7f64f5200640 1 -- 172.21.15.12:0/1674684881 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64f00197e0 con 0x5638ea064b60 2024-09-11T00:07:02.731 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:02.727+0000 7f64fd600640 1 -- 172.21.15.12:0/266414925 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9208+0+0 (secure 0 0 0) 0x7f64f80e6500 con 0x7f650010f760 2024-09-11T00:07:02.731 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:02.727+0000 7f64f7400640 1 --2- 172.21.15.12:0/266414925 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f64e40b52e0 0x7f64e40b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:07:02.731 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:02.727+0000 7f6509811740 1 -- 172.21.15.12:0/1674684881 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5638ea064b60 msgr2=0x5638ea0705b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:07:02.731 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:02.727+0000 7f6509811740 1 --2- 172.21.15.12:0/1674684881 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5638ea064b60 0x5638ea0705b0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f64f0009d20 tx=0x7f64f0011810 comp rx=0 tx=0).stop 2024-09-11T00:07:02.731 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:02.728+0000 7f6509811740 1 -- 172.21.15.12:0/1674684881 shutdown_connections 2024-09-11T00:07:02.731 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:02.728+0000 7f6509811740 1 --2- 172.21.15.12:0/1674684881 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5638ea070af0 0x5638ea072ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:07:02.731 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:02.728+0000 7f6509811740 1 --2- 172.21.15.12:0/1674684881 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5638ea064b60 0x5638ea0705b0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:07:02.731 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:02.728+0000 7f6509811740 1 --2- 172.21.15.12:0/1674684881 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5638ea064240 0x5638ea064620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:07:02.732 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:02.728+0000 7f6509811740 1 -- 172.21.15.12:0/1674684881 >> 172.21.15.12:0/1674684881 conn(0x5638ea062020 msgr2=0x5638ea063210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:07:02.732 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:02.728+0000 7f64f7400640 1 --2- 172.21.15.12:0/266414925 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f64e40b52e0 0x7f64e40b77a0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f64e8031440 tx=0x7f64e8005780 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:07:02.732 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:02.728+0000 7f6509811740 1 -- 172.21.15.12:0/1674684881 shutdown_connections 2024-09-11T00:07:02.732 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:02.728+0000 7f6509811740 1 -- 172.21.15.12:0/1674684881 wait complete. 2024-09-11T00:07:02.732 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:02.728+0000 7f6509811740 1 -- start start 2024-09-11T00:07:02.732 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:02.729+0000 7f6509811740 1 Processor -- start 2024-09-11T00:07:02.734 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:02.729+0000 7f6509811740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5638ea064240 0x5638ea070590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:07:02.734 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:02.729+0000 7f6509811740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5638ea064b60 0x5638ea06da00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:07:02.734 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:02.729+0000 7f6509811740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5638ea070af0 0x5638ea06e190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:07:02.735 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:02.729+0000 7f6509811740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x5638e9e2c5f0 con 0x5638ea064b60 2024-09-11T00:07:02.735 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:02.729+0000 7f6509811740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x5638e9f66460 con 0x5638ea070af0 2024-09-11T00:07:02.735 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:02.729+0000 7f6509811740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5638ea073840 con 0x5638ea064240 2024-09-11T00:07:02.735 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:02.729+0000 7f6509811740 10 client.0 _ll_register_callbacks cb 0x5638e9f76210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:07:02.735 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:02.730+0000 7f64ffe00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5638ea064240 0x5638ea070590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:07:02.735 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:02.730+0000 7f64f7400640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5638ea064b60 0x5638ea06da00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:07:02.735 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:02.730+0000 7f64ffe00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5638ea064240 0x5638ea070590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.12:55960/0 (socket says 172.21.15.12:55960) 2024-09-11T00:07:02.735 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:02.730+0000 7f64ffe00640 1 -- 172.21.15.12:0/3327577337 learned_addr learned my addr 172.21.15.12:0/3327577337 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:07:02.735 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:02.730+0000 7f6504c00640 1 --2- 172.21.15.12:0/3327577337 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5638ea070af0 0x5638ea06e190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:07:02.736 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:02.730+0000 7f64ffe00640 1 -- 172.21.15.12:0/3327577337 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5638ea070af0 msgr2=0x5638ea06e190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:07:02.736 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:02.730+0000 7f64ffe00640 1 --2- 172.21.15.12:0/3327577337 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5638ea070af0 0x5638ea06e190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:07:02.736 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:02.730+0000 7f64ffe00640 1 -- 172.21.15.12:0/3327577337 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5638ea064b60 msgr2=0x5638ea06da00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:07:02.736 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:02.730+0000 7f64ffe00640 1 --2- 172.21.15.12:0/3327577337 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5638ea064b60 0x5638ea06da00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:07:02.736 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:02.730+0000 7f64ffe00640 1 -- 172.21.15.12:0/3327577337 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5638e9d29540 con 0x5638ea064240 2024-09-11T00:07:02.736 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:02.730+0000 7f64ffe00640 1 --2- 172.21.15.12:0/3327577337 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5638ea064240 0x5638ea070590 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f64f00045a0 tx=0x7f64f00045d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5f0c438dc01e9924 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:07:02.736 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:02.730+0000 7f64ed600640 10 client.0 ms_handle_connect on v2:172.21.15.178:3300/0 2024-09-11T00:07:02.736 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:02.731+0000 7f64ed600640 1 -- 172.21.15.12:0/3327577337 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64f0027070 con 0x5638ea064240 2024-09-11T00:07:02.736 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:02.731+0000 7f6509811740 1 -- 172.21.15.12:0/3327577337 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5638e9f68a70 con 0x5638ea064240 2024-09-11T00:07:02.736 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:02.731+0000 7f64ed600640 1 -- 172.21.15.12:0/3327577337 <== mon.1 v2:172.21.15.178:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 9208+0+0 (secure 0 0 0) 0x7f64f0005450 con 0x5638ea064240 2024-09-11T00:07:02.736 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:02.731+0000 7f64ed600640 1 -- 172.21.15.12:0/3327577337 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f64f0033050 con 0x5638ea064240 2024-09-11T00:07:02.737 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:02.731+0000 7f6509811740 10 client.25414 fetch_fsmap learned FSMap version 29 2024-09-11T00:07:02.737 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:02.731+0000 7f6509811740 1 -- 172.21.15.12:0/3327577337 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5638e9f836f0 con 0x5638ea064240 2024-09-11T00:07:02.737 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:02.731+0000 7f64ed600640 1 -- 172.21.15.12:0/3327577337 <== mon.1 v2:172.21.15.178:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x5638e9f68a70 con 0x5638ea064240 2024-09-11T00:07:02.737 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:02.731+0000 7f64ed600640 1 -- 172.21.15.12:0/3327577337 <== mon.1 v2:172.21.15.178:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7f64f00048c0 con 0x5638ea064240 2024-09-11T00:07:02.737 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:02.731+0000 7f6509811740 10 client.25414 fetch_fsmap finished waiting for FSMap version 29 2024-09-11T00:07:02.737 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:02.732+0000 7f6509811740 10 client.25414 resolve_mds: resolved name 'e' to daemon mds.e 2024-09-11T00:07:02.737 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:02.732+0000 7f6509811740 20 client.25414 populate_metadata read hostname 'smithi012' 2024-09-11T00:07:02.737 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:02.732+0000 7f6509811740 1 --2- 172.21.15.12:0/3327577337 >> [v2:172.21.15.165:6834/2425395274,v1:172.21.15.165:6835/2425395274] conn(0x5638ea0815c0 0x5638ea082a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:07:02.737 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:02.732+0000 7f6509811740 4 client.25414 mds_command: new command op to 24614 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T00:07:02.737 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:02.732+0000 7f6509811740 1 -- 172.21.15.12:0/3327577337 --> [v2:172.21.15.165:6834/2425395274,v1:172.21.15.165:6835/2425395274] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5638e9fddca0 con 0x5638ea0815c0 2024-09-11T00:07:02.738 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:02.732+0000 7f64f7400640 1 --2- 172.21.15.12:0/3327577337 >> [v2:172.21.15.165:6834/2425395274,v1:172.21.15.165:6835/2425395274] conn(0x5638ea0815c0 0x5638ea082a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:07:02.738 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:02.733+0000 7f64f7400640 1 --2- 172.21.15.12:0/3327577337 >> [v2:172.21.15.165:6834/2425395274,v1:172.21.15.165:6835/2425395274] conn(0x5638ea0815c0 0x5638ea082a30 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=5bf31d2efc36e180 server_cookie=338f5eee450634a1 in_seq=0 out_seq=0 2024-09-11T00:07:02.738 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:02.733+0000 7f64ed600640 10 client.25414 ms_handle_connect on v2:172.21.15.165:6834/2425395274 2024-09-11T00:07:02.739 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:02.733+0000 7f64ed600640 1 -- 172.21.15.12:0/3327577337 <== mds.4 v2:172.21.15.165:6834/2425395274 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5638e9fddca0 con 0x5638ea0815c0 2024-09-11T00:07:02.739 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:02.733+0000 7f64ed600640 10 client.25414 handle_command_reply: tid=0 2024-09-11T00:07:02.740 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:02.733+0000 7f6509811740 1 client.25414 shutdown 2024-09-11T00:07:02.740 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:02.734+0000 7f6509811740 1 -- 172.21.15.12:0/3327577337 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5638ea064240 msgr2=0x5638ea070590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:07:02.740 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:02.734+0000 7f6509811740 1 --2- 172.21.15.12:0/3327577337 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5638ea064240 0x5638ea070590 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f64f00045a0 tx=0x7f64f00045d0 comp rx=0 tx=0).stop 2024-09-11T00:07:02.740 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:02.734+0000 7f6509811740 1 -- 172.21.15.12:0/3327577337 shutdown_connections 2024-09-11T00:07:02.740 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:02.734+0000 7f6509811740 1 --2- 172.21.15.12:0/3327577337 >> [v2:172.21.15.165:6834/2425395274,v1:172.21.15.165:6835/2425395274] conn(0x5638ea0815c0 0x5638ea082a30 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:07:02.740 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:02.734+0000 7f6509811740 1 --2- 172.21.15.12:0/3327577337 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5638ea070af0 0x5638ea06e190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:07:02.741 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:02.734+0000 7f6509811740 1 --2- 172.21.15.12:0/3327577337 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5638ea064b60 0x5638ea06da00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:07:02.741 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:02.734+0000 7f6509811740 1 --2- 172.21.15.12:0/3327577337 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5638ea064240 0x5638ea070590 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:07:02.741 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:02.734+0000 7f6509811740 1 -- 172.21.15.12:0/3327577337 >> 172.21.15.12:0/3327577337 conn(0x5638ea062020 msgr2=0x5638ea067c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:07:02.741 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:02.734+0000 7f64ed600640 0 client.25414 ms_handle_reset on v2:172.21.15.165:6834/2425395274 2024-09-11T00:07:02.741 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:02.734+0000 7f6509811740 1 -- 172.21.15.12:0/3327577337 shutdown_connections 2024-09-11T00:07:02.741 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:02.734+0000 7f6509811740 1 -- 172.21.15.12:0/3327577337 wait complete. 2024-09-11T00:07:02.742 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:02.734+0000 7f6509811740 20 client.25414 trim_cache size 0 max 16384 2024-09-11T00:07:02.758 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:02.753+0000 7f6509811740 1 Processor -- start 2024-09-11T00:07:02.758 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:02.753+0000 7f6509811740 1 -- start start 2024-09-11T00:07:02.758 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:02.753+0000 7f6509811740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5638ea064240 0x5638ea080ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:07:02.758 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:02.754+0000 7f6509811740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5638ea064b60 0x5638ea082360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:07:02.758 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:02.754+0000 7f6509811740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5638ea070af0 0x5638ea07cc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:07:02.759 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:02.754+0000 7f6509811740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x5638ea080d70 con 0x5638ea064b60 2024-09-11T00:07:02.759 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:02.754+0000 7f6509811740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x5638ea081440 con 0x5638ea064240 2024-09-11T00:07:02.759 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:02.754+0000 7f6509811740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5638ea081b10 con 0x5638ea070af0 2024-09-11T00:07:02.760 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:02.754+0000 7f64f7400640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5638ea064b60 0x5638ea082360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:07:02.760 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:02.754+0000 7f64f7400640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5638ea064b60 0x5638ea082360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:46956/0 (socket says 172.21.15.12:46956) 2024-09-11T00:07:02.760 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:02.754+0000 7f64f7400640 1 -- 172.21.15.12:0/3405822519 learned_addr learned my addr 172.21.15.12:0/3405822519 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:07:02.760 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:02.754+0000 7f64ffe00640 1 --2- 172.21.15.12:0/3405822519 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5638ea064240 0x5638ea080ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:07:02.760 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:02.754+0000 7f6504c00640 1 --2- 172.21.15.12:0/3405822519 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5638ea070af0 0x5638ea07cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:07:02.760 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:02.754+0000 7f64f7400640 1 -- 172.21.15.12:0/3405822519 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5638ea070af0 msgr2=0x5638ea07cc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:07:02.761 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:02.754+0000 7f64f7400640 1 --2- 172.21.15.12:0/3405822519 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5638ea070af0 0x5638ea07cc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:07:02.761 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:02.754+0000 7f64f7400640 1 -- 172.21.15.12:0/3405822519 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5638ea064240 msgr2=0x5638ea080ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:07:02.761 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:02.754+0000 7f64f7400640 1 --2- 172.21.15.12:0/3405822519 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5638ea064240 0x5638ea080ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:07:02.761 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:02.754+0000 7f64f7400640 1 -- 172.21.15.12:0/3405822519 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5638ea064990 con 0x5638ea064b60 2024-09-11T00:07:02.761 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:02.754+0000 7f64ffe00640 1 --2- 172.21.15.12:0/3405822519 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5638ea064240 0x5638ea080ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:07:02.761 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:02.755+0000 7f64f7400640 1 --2- 172.21.15.12:0/3405822519 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5638ea064b60 0x5638ea082360 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f64e802a930 tx=0x7f64e802a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c12a33866e9d8c0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:07:02.761 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:02.755+0000 7f64efe00640 1 -- 172.21.15.12:0/3405822519 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64e8035040 con 0x5638ea064b60 2024-09-11T00:07:02.761 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:02.755+0000 7f64efe00640 1 -- 172.21.15.12:0/3405822519 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f64e8059070 con 0x5638ea064b60 2024-09-11T00:07:02.761 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:02.755+0000 7f64efe00640 1 -- 172.21.15.12:0/3405822519 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64e804b460 con 0x5638ea064b60 2024-09-11T00:07:02.761 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:02.756+0000 7f6509811740 1 -- 172.21.15.12:0/3405822519 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5638ea064b60 msgr2=0x5638ea082360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:07:02.762 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:02.756+0000 7f6509811740 1 --2- 172.21.15.12:0/3405822519 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5638ea064b60 0x5638ea082360 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f64e802a930 tx=0x7f64e802a410 comp rx=0 tx=0).stop 2024-09-11T00:07:02.762 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:02.756+0000 7f6509811740 1 -- 172.21.15.12:0/3405822519 shutdown_connections 2024-09-11T00:07:02.762 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:02.756+0000 7f6509811740 1 --2- 172.21.15.12:0/3405822519 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5638ea070af0 0x5638ea07cc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:07:02.762 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:02.756+0000 7f6509811740 1 --2- 172.21.15.12:0/3405822519 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5638ea064b60 0x5638ea082360 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:07:02.762 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:02.756+0000 7f6509811740 1 --2- 172.21.15.12:0/3405822519 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5638ea064240 0x5638ea080ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:07:02.762 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:02.756+0000 7f6509811740 1 -- 172.21.15.12:0/3405822519 >> 172.21.15.12:0/3405822519 conn(0x5638ea062020 msgr2=0x5638ea06d790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:07:02.762 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:02.756+0000 7f6509811740 1 -- 172.21.15.12:0/3405822519 shutdown_connections 2024-09-11T00:07:02.762 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:02.756+0000 7f6509811740 1 -- 172.21.15.12:0/3405822519 wait complete. 2024-09-11T00:07:02.766 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:02.757+0000 7f6509811740 1 -- start start 2024-09-11T00:07:02.766 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:02.757+0000 7f6509811740 1 Processor -- start 2024-09-11T00:07:02.766 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:02.757+0000 7f6509811740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5638ea064240 0x5638ea0850a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:07:02.766 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:02.757+0000 7f6509811740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5638ea064b60 0x5638ea0855e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:07:02.766 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:02.757+0000 7f6509811740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5638ea070af0 0x5638ea089f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:07:02.766 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:02.757+0000 7f6509811740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x5638ea0828b0 con 0x5638ea064b60 2024-09-11T00:07:02.766 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:02.757+0000 7f6509811740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x5638ea0821e0 con 0x5638ea070af0 2024-09-11T00:07:02.766 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:02.757+0000 7f6509811740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5638ea082f80 con 0x5638ea064240 2024-09-11T00:07:02.767 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:02.757+0000 7f6509811740 10 client.0 _ll_register_callbacks cb 0x7f64f0009b70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:07:02.767 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:02.758+0000 7f6504c00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5638ea070af0 0x5638ea089f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:07:02.767 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:02.758+0000 7f64f7400640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5638ea064b60 0x5638ea0855e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:07:02.767 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:02.758+0000 7f64ffe00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5638ea064240 0x5638ea0850a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:07:02.767 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:02.758+0000 7f6504c00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5638ea070af0 0x5638ea089f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.12:49004/0 (socket says 172.21.15.12:49004) 2024-09-11T00:07:02.767 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:02.758+0000 7f6504c00640 1 -- 172.21.15.12:0/3213572136 learned_addr learned my addr 172.21.15.12:0/3213572136 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:07:02.767 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:02.758+0000 7f64ffe00640 1 -- 172.21.15.12:0/3213572136 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5638ea070af0 msgr2=0x5638ea089f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:07:02.767 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:02.758+0000 7f64ffe00640 1 --2- 172.21.15.12:0/3213572136 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5638ea070af0 0x5638ea089f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:07:02.767 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:02.758+0000 7f64ffe00640 1 -- 172.21.15.12:0/3213572136 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5638ea064b60 msgr2=0x5638ea0855e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:07:02.767 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:02.758+0000 7f64ffe00640 1 --2- 172.21.15.12:0/3213572136 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5638ea064b60 0x5638ea0855e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:07:02.767 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:02.758+0000 7f64ffe00640 1 -- 172.21.15.12:0/3213572136 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5638ea08a480 con 0x5638ea064240 2024-09-11T00:07:02.768 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:02.758+0000 7f6504c00640 1 --2- 172.21.15.12:0/3213572136 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5638ea070af0 0x5638ea089f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:07:02.768 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:02.758+0000 7f64f7400640 1 --2- 172.21.15.12:0/3213572136 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5638ea064b60 0x5638ea0855e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:07:02.768 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:02.758+0000 7f64ffe00640 1 --2- 172.21.15.12:0/3213572136 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5638ea064240 0x5638ea0850a0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f64f0001eb0 tx=0x7f64f0009a10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=62ddf072a633c641 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:07:02.768 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:02.758+0000 7f64ed600640 10 client.0 ms_handle_connect on v2:172.21.15.178:3300/0 2024-09-11T00:07:02.768 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:02.759+0000 7f64ed600640 1 -- 172.21.15.12:0/3213572136 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64f003c060 con 0x5638ea064240 2024-09-11T00:07:02.768 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:02.759+0000 7f64ed600640 1 -- 172.21.15.12:0/3213572136 <== mon.1 v2:172.21.15.178:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 9208+0+0 (secure 0 0 0) 0x7f64f0004640 con 0x5638ea064240 2024-09-11T00:07:02.768 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:02.759+0000 7f6509811740 1 -- 172.21.15.12:0/3213572136 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f64e8059070 con 0x5638ea064240 2024-09-11T00:07:02.768 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:02.759+0000 7f64ed600640 1 -- 172.21.15.12:0/3213572136 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f64f0043050 con 0x5638ea064240 2024-09-11T00:07:02.768 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:02.759+0000 7f6509811740 10 client.25420 fetch_fsmap learned FSMap version 29 2024-09-11T00:07:02.768 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:02.759+0000 7f6509811740 1 -- 172.21.15.12:0/3213572136 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5638ea08b780 con 0x5638ea064240 2024-09-11T00:07:02.768 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:02.759+0000 7f64ed600640 1 -- 172.21.15.12:0/3213572136 <== mon.1 v2:172.21.15.178:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f64e8059070 con 0x5638ea064240 2024-09-11T00:07:02.768 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:02.760+0000 7f64ed600640 1 -- 172.21.15.12:0/3213572136 <== mon.1 v2:172.21.15.178:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7f64f0019ca0 con 0x5638ea064240 2024-09-11T00:07:02.769 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:02.760+0000 7f6509811740 10 client.25420 fetch_fsmap finished waiting for FSMap version 29 2024-09-11T00:07:02.769 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:02.760+0000 7f6509811740 10 client.25420 resolve_mds: resolved name 'e' to daemon mds.e 2024-09-11T00:07:02.769 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:02.760+0000 7f6509811740 20 client.25420 populate_metadata read hostname 'smithi012' 2024-09-11T00:07:02.769 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:02.760+0000 7f6509811740 1 --2- 172.21.15.12:0/3213572136 >> [v2:172.21.15.165:6834/2425395274,v1:172.21.15.165:6835/2425395274] conn(0x5638ea0f25b0 0x5638ea0f12c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:07:02.769 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:02.760+0000 7f6509811740 4 client.25420 mds_command: new command op to 24614 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-11T00:07:02.769 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:02.760+0000 7f6509811740 1 -- 172.21.15.12:0/3213572136 --> [v2:172.21.15.165:6834/2425395274,v1:172.21.15.165:6835/2425395274] -- command(tid 0: {"prefix": "status"}) -- 0x5638e9fd9750 con 0x5638ea0f25b0 2024-09-11T00:07:02.769 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:02.760+0000 7f64f7400640 1 --2- 172.21.15.12:0/3213572136 >> [v2:172.21.15.165:6834/2425395274,v1:172.21.15.165:6835/2425395274] conn(0x5638ea0f25b0 0x5638ea0f12c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:07:02.769 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:02.761+0000 7f64f7400640 1 --2- 172.21.15.12:0/3213572136 >> [v2:172.21.15.165:6834/2425395274,v1:172.21.15.165:6835/2425395274] conn(0x5638ea0f25b0 0x5638ea0f12c0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=7e727c3ca2acea95 server_cookie=9c7a66cc5dfec8aa in_seq=0 out_seq=0 2024-09-11T00:07:02.769 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:02.761+0000 7f64ed600640 10 client.25420 ms_handle_connect on v2:172.21.15.165:6834/2425395274 2024-09-11T00:07:02.769 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:02.761+0000 7f64ed600640 1 -- 172.21.15.12:0/3213572136 <== mds.4 v2:172.21.15.165:6834/2425395274 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x5638e9fd9750 con 0x5638ea0f25b0 2024-09-11T00:07:02.769 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:02.761+0000 7f64ed600640 10 client.25420 handle_command_reply: tid=0 2024-09-11T00:07:02.770 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:02.761+0000 7f6509811740 1 client.25420 shutdown 2024-09-11T00:07:02.770 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:02.761+0000 7f6509811740 1 -- 172.21.15.12:0/3213572136 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5638ea064240 msgr2=0x5638ea0850a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:07:02.770 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:02.761+0000 7f6509811740 1 --2- 172.21.15.12:0/3213572136 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5638ea064240 0x5638ea0850a0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f64f0001eb0 tx=0x7f64f0009a10 comp rx=0 tx=0).stop 2024-09-11T00:07:02.770 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:02.761+0000 7f6509811740 1 -- 172.21.15.12:0/3213572136 shutdown_connections 2024-09-11T00:07:02.770 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:02.761+0000 7f6509811740 1 --2- 172.21.15.12:0/3213572136 >> [v2:172.21.15.165:6834/2425395274,v1:172.21.15.165:6835/2425395274] conn(0x5638ea0f25b0 0x5638ea0f12c0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:07:02.770 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:02.761+0000 7f6509811740 1 --2- 172.21.15.12:0/3213572136 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5638ea070af0 0x5638ea089f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:07:02.770 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:02.761+0000 7f6509811740 1 --2- 172.21.15.12:0/3213572136 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5638ea064b60 0x5638ea0855e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:07:02.770 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:02.761+0000 7f64ed600640 0 client.25420 ms_handle_reset on v2:172.21.15.165:6834/2425395274 2024-09-11T00:07:02.770 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:02.761+0000 7f6509811740 1 --2- 172.21.15.12:0/3213572136 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5638ea064240 0x5638ea0850a0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:07:02.770 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:02.761+0000 7f6509811740 1 -- 172.21.15.12:0/3213572136 >> 172.21.15.12:0/3213572136 conn(0x5638ea062020 msgr2=0x5638ea05cea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:07:02.770 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:02.762+0000 7f6509811740 1 -- 172.21.15.12:0/3213572136 shutdown_connections 2024-09-11T00:07:02.770 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:02.762+0000 7f6509811740 1 -- 172.21.15.12:0/3213572136 wait complete. 2024-09-11T00:07:02.770 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:02.762+0000 7f6509811740 20 client.25420 trim_cache size 0 max 16384 2024-09-11T00:07:02.771 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-09-11T00:07:02.771 INFO:teuthology.orchestra.run.smithi012.stdout: "cluster_fsid": "ff3ad74a-6fcf-11ef-bcea-c7b262605968", 2024-09-11T00:07:02.771 INFO:teuthology.orchestra.run.smithi012.stdout: "whoami": 4, 2024-09-11T00:07:02.771 INFO:teuthology.orchestra.run.smithi012.stdout: "id": 24614, 2024-09-11T00:07:02.771 INFO:teuthology.orchestra.run.smithi012.stdout: "want_state": "up:active", 2024-09-11T00:07:02.771 INFO:teuthology.orchestra.run.smithi012.stdout: "state": "up:active", 2024-09-11T00:07:02.771 INFO:teuthology.orchestra.run.smithi012.stdout: "fs_name": "cephfs", 2024-09-11T00:07:02.771 INFO:teuthology.orchestra.run.smithi012.stdout: "rank_uptime": 133.52323959500001, 2024-09-11T00:07:02.771 INFO:teuthology.orchestra.run.smithi012.stdout: "mdsmap_epoch": 29, 2024-09-11T00:07:02.771 INFO:teuthology.orchestra.run.smithi012.stdout: "osdmap_epoch": 85, 2024-09-11T00:07:02.772 INFO:teuthology.orchestra.run.smithi012.stdout: "osdmap_epoch_barrier": 85, 2024-09-11T00:07:02.772 INFO:teuthology.orchestra.run.smithi012.stdout: "uptime": 211.06717816099999 2024-09-11T00:07:02.773 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-09-11T00:07:02.773 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:02.762+0000 7f6506c00640 1 -- 172.21.15.12:0/266414925 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f64e40b52e0 msgr2=0x7f64e40b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:07:02.773 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:02.762+0000 7f6506c00640 1 --2- 172.21.15.12:0/266414925 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f64e40b52e0 0x7f64e40b77a0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f64e8031440 tx=0x7f64e8005780 comp rx=0 tx=0).stop 2024-09-11T00:07:02.773 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:02.762+0000 7f6506c00640 1 -- 172.21.15.12:0/266414925 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f650010f760 msgr2=0x7f65001a5180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:07:02.774 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:02.762+0000 7f6506c00640 1 --2- 172.21.15.12:0/266414925 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f650010f760 0x7f65001a5180 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f64f800eb10 tx=0x7f64f800c2d0 comp rx=0 tx=0).stop 2024-09-11T00:07:02.774 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:02.763+0000 7f6506c00640 1 -- 172.21.15.12:0/266414925 shutdown_connections 2024-09-11T00:07:02.774 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:02.763+0000 7f6506c00640 1 --2- 172.21.15.12:0/266414925 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f64e40b52e0 0x7f64e40b77a0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:07:02.774 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:02.763+0000 7f6506c00640 1 --2- 172.21.15.12:0/266414925 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f650010f760 0x7f65001a5180 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:07:02.774 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:02.763+0000 7f6506c00640 1 --2- 172.21.15.12:0/266414925 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f650010ede0 0x7f650019fde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:07:02.774 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:02.763+0000 7f6506c00640 1 --2- 172.21.15.12:0/266414925 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f65001093d0 0x7f650019f8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:07:02.774 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:02.763+0000 7f6506c00640 1 -- 172.21.15.12:0/266414925 >> 172.21.15.12:0/266414925 conn(0x7f6500078a40 msgr2=0x7f650007ae30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:07:02.774 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:02.763+0000 7f6506c00640 1 -- 172.21.15.12:0/266414925 shutdown_connections 2024-09-11T00:07:02.774 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:02.763+0000 7f6506c00640 1 -- 172.21.15.12:0/266414925 wait complete. 2024-09-11T00:07:03.562 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:07:03 smithi012 ceph-mon[30333]: pgmap v397: 129 pgs: 129 active+clean; 663 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 2.9 KiB/s wr, 15 op/s 2024-09-11T00:07:03.581 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:07:03 smithi178 ceph-mon[35919]: pgmap v397: 129 pgs: 129 active+clean; 663 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 2.9 KiB/s wr, 15 op/s 2024-09-11T00:07:03.627 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid ff3ad74a-6fcf-11ef-bcea-c7b262605968 -- ceph tell mds.i status 2024-09-11T00:07:03.782 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:07:03 smithi165 ceph-mon[35356]: pgmap v397: 129 pgs: 129 active+clean; 663 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 2.9 KiB/s wr, 15 op/s 2024-09-11T00:07:04.006 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:07:04.554 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:04.551+0000 7f3547c00640 1 -- 172.21.15.12:0/920992095 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3540113980 msgr2=0x7f3540115d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:07:04.554 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:04.551+0000 7f3547c00640 1 --2- 172.21.15.12:0/920992095 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3540113980 0x7f3540115d70 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f3538009ca0 tx=0x7f353802a8c0 comp rx=0 tx=0).stop 2024-09-11T00:07:04.554 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:04.551+0000 7f3547c00640 1 -- 172.21.15.12:0/920992095 shutdown_connections 2024-09-11T00:07:04.554 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:04.551+0000 7f3547c00640 1 --2- 172.21.15.12:0/920992095 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3540113980 0x7f3540115d70 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:07:04.554 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:04.551+0000 7f3547c00640 1 --2- 172.21.15.12:0/920992095 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3540077ff0 0x7f3540113440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:07:04.554 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:04.551+0000 7f3547c00640 1 --2- 172.21.15.12:0/920992095 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f35400776d0 0x7f3540077ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:07:04.554 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:04.551+0000 7f3547c00640 1 -- 172.21.15.12:0/920992095 >> 172.21.15.12:0/920992095 conn(0x7f3540100a00 msgr2=0x7f3540102e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:07:04.554 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:04.552+0000 7f3547c00640 1 -- 172.21.15.12:0/920992095 shutdown_connections 2024-09-11T00:07:04.554 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:04.552+0000 7f3547c00640 1 -- 172.21.15.12:0/920992095 wait complete. 2024-09-11T00:07:04.556 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:04.552+0000 7f3547c00640 1 Processor -- start 2024-09-11T00:07:04.556 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:04.553+0000 7f3547c00640 1 -- start start 2024-09-11T00:07:04.556 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:04.553+0000 7f3547c00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f35400776d0 0x7f354019f840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:07:04.556 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:04.553+0000 7f3547c00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3540077ff0 0x7f354019fd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:07:04.556 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:04.553+0000 7f3547c00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3540113980 0x7f35401a5120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:07:04.556 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:04.553+0000 7f3547c00640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f3540119350 con 0x7f3540113980 2024-09-11T00:07:04.556 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:04.553+0000 7f3547c00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f35401191d0 con 0x7f3540077ff0 2024-09-11T00:07:04.556 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:04.553+0000 7f3547c00640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f35401194d0 con 0x7f35400776d0 2024-09-11T00:07:04.556 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:04.553+0000 7f353fe00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3540077ff0 0x7f354019fd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:07:04.556 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:04.553+0000 7f3545c00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3540113980 0x7f35401a5120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:07:04.557 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:04.553+0000 7f353fe00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3540077ff0 0x7f354019fd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.12:49024/0 (socket says 172.21.15.12:49024) 2024-09-11T00:07:04.557 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:04.553+0000 7f353fe00640 1 -- 172.21.15.12:0/221360776 learned_addr learned my addr 172.21.15.12:0/221360776 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:07:04.557 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:04.553+0000 7f3545200640 1 --2- 172.21.15.12:0/221360776 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f35400776d0 0x7f354019f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:07:04.557 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:04.554+0000 7f353fe00640 1 -- 172.21.15.12:0/221360776 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f35400776d0 msgr2=0x7f354019f840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:07:04.557 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:04.554+0000 7f353fe00640 1 --2- 172.21.15.12:0/221360776 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f35400776d0 0x7f354019f840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:07:04.557 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:04.554+0000 7f353fe00640 1 -- 172.21.15.12:0/221360776 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3540113980 msgr2=0x7f35401a5120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:07:04.557 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:04.554+0000 7f353fe00640 1 --2- 172.21.15.12:0/221360776 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3540113980 0x7f35401a5120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:07:04.557 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:04.554+0000 7f353fe00640 1 -- 172.21.15.12:0/221360776 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35401a58f0 con 0x7f3540077ff0 2024-09-11T00:07:04.557 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:04.554+0000 7f353fe00640 1 --2- 172.21.15.12:0/221360776 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3540077ff0 0x7f354019fd80 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f352c00e9f0 tx=0x7f352c00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:07:04.558 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:04.554+0000 7f353d600640 1 -- 172.21.15.12:0/221360776 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f352c00cd80 con 0x7f3540077ff0 2024-09-11T00:07:04.558 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:04.554+0000 7f3547c00640 1 -- 172.21.15.12:0/221360776 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f35401a5be0 con 0x7f3540077ff0 2024-09-11T00:07:04.558 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:04.554+0000 7f353d600640 1 -- 172.21.15.12:0/221360776 <== mon.2 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f352c0108e0 con 0x7f3540077ff0 2024-09-11T00:07:04.558 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:04.554+0000 7f353d600640 1 -- 172.21.15.12:0/221360776 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f352c018980 con 0x7f3540077ff0 2024-09-11T00:07:04.558 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:04.554+0000 7f3547c00640 1 -- 172.21.15.12:0/221360776 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35401ac2a0 con 0x7f3540077ff0 2024-09-11T00:07:04.560 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:04.558+0000 7f354a64b740 1 Processor -- start 2024-09-11T00:07:04.560 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:04.558+0000 7f354a64b740 1 -- start start 2024-09-11T00:07:04.561 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:04.558+0000 7f354a64b740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563b7718c240 0x563b7718c620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:07:04.561 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:04.558+0000 7f353d600640 1 -- 172.21.15.12:0/221360776 <== mon.2 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7f352c010430 con 0x7f3540077ff0 2024-09-11T00:07:04.561 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:04.558+0000 7f354a64b740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563b7718cb60 0x563b771985b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:07:04.561 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:04.558+0000 7f354a64b740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563b77198af0 0x563b7719aee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:07:04.562 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:04.558+0000 7f354a64b740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x563b77103d50 con 0x563b7718cb60 2024-09-11T00:07:04.562 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:04.558+0000 7f354a64b740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x563b770f3180 con 0x563b77198af0 2024-09-11T00:07:04.562 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:04.558+0000 7f354a64b740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x563b770f6f90 con 0x563b7718c240 2024-09-11T00:07:04.562 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:04.559+0000 7f3545200640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563b7718cb60 0x563b771985b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:07:04.562 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:04.559+0000 7f3545c00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563b7718c240 0x563b7718c620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:07:04.562 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:04.559+0000 7f3545c00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563b7718c240 0x563b7718c620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.12:55988/0 (socket says 172.21.15.12:55988) 2024-09-11T00:07:04.562 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:04.559+0000 7f3545c00640 1 -- 172.21.15.12:0/4187779866 learned_addr learned my addr 172.21.15.12:0/4187779866 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:07:04.562 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:04.559+0000 7f3545200640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563b7718cb60 0x563b771985b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:46988/0 (socket says 172.21.15.12:46988) 2024-09-11T00:07:04.562 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:04.559+0000 7f353fe00640 1 --2- 172.21.15.12:0/4187779866 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563b77198af0 0x563b7719aee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:07:04.562 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:04.559+0000 7f3545200640 1 -- 172.21.15.12:0/4187779866 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563b7718c240 msgr2=0x563b7718c620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:07:04.562 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:04.559+0000 7f3545200640 1 --2- 172.21.15.12:0/4187779866 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563b7718c240 0x563b7718c620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:07:04.562 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:04.559+0000 7f3545200640 1 -- 172.21.15.12:0/4187779866 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563b77198af0 msgr2=0x563b7719aee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:07:04.562 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:04.559+0000 7f3545200640 1 --2- 172.21.15.12:0/4187779866 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563b77198af0 0x563b7719aee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:07:04.563 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:04.559+0000 7f3545200640 1 -- 172.21.15.12:0/4187779866 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563b770bda70 con 0x563b7718cb60 2024-09-11T00:07:04.563 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:04.560+0000 7f3545200640 1 --2- 172.21.15.12:0/4187779866 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563b7718cb60 0x563b771985b0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f353400bc20 tx=0x7f35340107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d4ae31b49de1ebe server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:07:04.564 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:04.560+0000 7f3532000640 1 -- 172.21.15.12:0/4187779866 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3534004030 con 0x563b7718cb60 2024-09-11T00:07:04.565 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:04.560+0000 7f3532000640 1 -- 172.21.15.12:0/4187779866 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f35340035f0 con 0x563b7718cb60 2024-09-11T00:07:04.565 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:04.560+0000 7f3532000640 1 -- 172.21.15.12:0/4187779866 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3534011640 con 0x563b7718cb60 2024-09-11T00:07:04.565 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:04.560+0000 7f353d600640 1 --2- 172.21.15.12:0/221360776 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f35280b52e0 0x7f35280b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:07:04.565 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:04.560+0000 7f353d600640 1 -- 172.21.15.12:0/221360776 <== mon.2 v2:172.21.15.165:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9208+0+0 (secure 0 0 0) 0x7f352c0e62f0 con 0x7f3540077ff0 2024-09-11T00:07:04.565 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:04.561+0000 7f3545c00640 1 --2- 172.21.15.12:0/221360776 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f35280b52e0 0x7f35280b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:07:04.565 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:04.561+0000 7f354a64b740 1 -- 172.21.15.12:0/4187779866 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563b7718cb60 msgr2=0x563b771985b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:07:04.565 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:04.561+0000 7f354a64b740 1 --2- 172.21.15.12:0/4187779866 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563b7718cb60 0x563b771985b0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f353400bc20 tx=0x7f35340107a0 comp rx=0 tx=0).stop 2024-09-11T00:07:04.565 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:04.561+0000 7f354a64b740 1 -- 172.21.15.12:0/4187779866 shutdown_connections 2024-09-11T00:07:04.565 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:04.561+0000 7f354a64b740 1 --2- 172.21.15.12:0/4187779866 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563b77198af0 0x563b7719aee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:07:04.565 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:04.561+0000 7f354a64b740 1 --2- 172.21.15.12:0/4187779866 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563b7718cb60 0x563b771985b0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:07:04.565 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:04.561+0000 7f354a64b740 1 --2- 172.21.15.12:0/4187779866 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563b7718c240 0x563b7718c620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:07:04.566 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:04.561+0000 7f354a64b740 1 -- 172.21.15.12:0/4187779866 >> 172.21.15.12:0/4187779866 conn(0x563b7718a020 msgr2=0x563b7718b210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:07:04.566 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:04.561+0000 7f354a64b740 1 -- 172.21.15.12:0/4187779866 shutdown_connections 2024-09-11T00:07:04.566 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:04.561+0000 7f354a64b740 1 -- 172.21.15.12:0/4187779866 wait complete. 2024-09-11T00:07:04.566 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:04.562+0000 7f3545c00640 1 --2- 172.21.15.12:0/221360776 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f35280b52e0 0x7f35280b77a0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f3538009ca0 tx=0x7f35380099a0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:07:04.566 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:04.562+0000 7f354a64b740 1 -- start start 2024-09-11T00:07:04.566 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:04.562+0000 7f354a64b740 1 Processor -- start 2024-09-11T00:07:04.566 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:04.562+0000 7f354a64b740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563b7718c240 0x563b77195a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:07:04.566 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:04.563+0000 7f354a64b740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563b7718cb60 0x563b77195f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:07:04.566 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:04.563+0000 7f354a64b740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563b77198af0 0x563b771964b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:07:04.566 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:04.563+0000 7f354a64b740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x563b76f545f0 con 0x563b7718cb60 2024-09-11T00:07:04.566 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:04.563+0000 7f354a64b740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x563b7708e460 con 0x563b77198af0 2024-09-11T00:07:04.566 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:04.563+0000 7f354a64b740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x563b7719b890 con 0x563b7718c240 2024-09-11T00:07:04.566 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:04.563+0000 7f354a64b740 10 client.0 _ll_register_callbacks cb 0x563b7709e210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:07:04.567 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:04.563+0000 7f353fe00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563b7718cb60 0x563b77195f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:07:04.567 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:04.563+0000 7f3545c00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563b77198af0 0x563b771964b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:07:04.567 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:04.563+0000 7f3545200640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563b7718c240 0x563b77195a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:07:04.569 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:04.563+0000 7f3545200640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563b7718c240 0x563b77195a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.12:55992/0 (socket says 172.21.15.12:55992) 2024-09-11T00:07:04.569 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:04.563+0000 7f3545200640 1 -- 172.21.15.12:0/1281453412 learned_addr learned my addr 172.21.15.12:0/1281453412 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:07:04.569 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:04.563+0000 7f3545c00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563b77198af0 0x563b771964b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.12:49034/0 (socket says 172.21.15.12:49034) 2024-09-11T00:07:04.569 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:04.563+0000 7f3545200640 1 -- 172.21.15.12:0/1281453412 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563b77198af0 msgr2=0x563b771964b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:07:04.569 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:04.563+0000 7f3545200640 1 --2- 172.21.15.12:0/1281453412 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563b77198af0 0x563b771964b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:07:04.570 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:04.563+0000 7f3545200640 1 -- 172.21.15.12:0/1281453412 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563b7718cb60 msgr2=0x563b77195f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:07:04.570 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:04.563+0000 7f3545200640 1 --2- 172.21.15.12:0/1281453412 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563b7718cb60 0x563b77195f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:07:04.570 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:04.563+0000 7f3545200640 1 -- 172.21.15.12:0/1281453412 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563b76e51540 con 0x563b7718c240 2024-09-11T00:07:04.570 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:04.564+0000 7f3545200640 1 --2- 172.21.15.12:0/1281453412 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563b7718c240 0x563b77195a30 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f3534011010 tx=0x7f3534005280 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c906eeac28266069 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:07:04.570 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:04.564+0000 7f3526a00640 10 client.0 ms_handle_connect on v2:172.21.15.178:3300/0 2024-09-11T00:07:04.570 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:04.564+0000 7f3526a00640 1 -- 172.21.15.12:0/1281453412 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3534029060 con 0x563b7718c240 2024-09-11T00:07:04.570 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:04.564+0000 7f354a64b740 1 -- 172.21.15.12:0/1281453412 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x563b77090a70 con 0x563b7718c240 2024-09-11T00:07:04.570 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:04.564+0000 7f3526a00640 1 -- 172.21.15.12:0/1281453412 <== mon.1 v2:172.21.15.178:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 9208+0+0 (secure 0 0 0) 0x7f35340037f0 con 0x563b7718c240 2024-09-11T00:07:04.570 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:04.564+0000 7f3526a00640 1 -- 172.21.15.12:0/1281453412 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f3534030050 con 0x563b7718c240 2024-09-11T00:07:04.570 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:04.564+0000 7f354a64b740 10 client.25432 fetch_fsmap learned FSMap version 29 2024-09-11T00:07:04.570 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:04.564+0000 7f354a64b740 1 -- 172.21.15.12:0/1281453412 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x563b770ab6f0 con 0x563b7718c240 2024-09-11T00:07:04.571 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:04.565+0000 7f3526a00640 1 -- 172.21.15.12:0/1281453412 <== mon.1 v2:172.21.15.178:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x563b77090a70 con 0x563b7718c240 2024-09-11T00:07:04.571 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:04.565+0000 7f3526a00640 1 -- 172.21.15.12:0/1281453412 <== mon.1 v2:172.21.15.178:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7f3534023030 con 0x563b7718c240 2024-09-11T00:07:04.571 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:04.565+0000 7f354a64b740 10 client.25432 fetch_fsmap finished waiting for FSMap version 29 2024-09-11T00:07:04.571 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:04.565+0000 7f354a64b740 10 client.25432 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-11T00:07:04.571 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:04.565+0000 7f354a64b740 20 client.25432 populate_metadata read hostname 'smithi012' 2024-09-11T00:07:04.571 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:04.565+0000 7f354a64b740 1 --2- 172.21.15.12:0/1281453412 >> [v2:172.21.15.178:6838/70628097,v1:172.21.15.178:6839/70628097] conn(0x563b771a9620 0x563b771aaa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:07:04.571 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:04.565+0000 7f354a64b740 4 client.25432 mds_command: new command op to 24637 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T00:07:04.571 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:04.565+0000 7f354a64b740 1 -- 172.21.15.12:0/1281453412 --> [v2:172.21.15.178:6838/70628097,v1:172.21.15.178:6839/70628097] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563b77105ca0 con 0x563b771a9620 2024-09-11T00:07:04.571 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:04.566+0000 7f353fe00640 1 --2- 172.21.15.12:0/1281453412 >> [v2:172.21.15.178:6838/70628097,v1:172.21.15.178:6839/70628097] conn(0x563b771a9620 0x563b771aaa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:07:04.571 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:04.566+0000 7f353fe00640 1 --2- 172.21.15.12:0/1281453412 >> [v2:172.21.15.178:6838/70628097,v1:172.21.15.178:6839/70628097] conn(0x563b771a9620 0x563b771aaa90 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=d136c666815f3438 server_cookie=c938ea0dca3bef4b in_seq=0 out_seq=0 2024-09-11T00:07:04.571 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:04.566+0000 7f3526a00640 10 client.25432 ms_handle_connect on v2:172.21.15.178:6838/70628097 2024-09-11T00:07:04.571 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:04.567+0000 7f3526a00640 1 -- 172.21.15.12:0/1281453412 <== mds.3 v2:172.21.15.178:6838/70628097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x563b77105ca0 con 0x563b771a9620 2024-09-11T00:07:04.572 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:04.567+0000 7f3526a00640 10 client.25432 handle_command_reply: tid=0 2024-09-11T00:07:04.572 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:04.567+0000 7f354a64b740 1 client.25432 shutdown 2024-09-11T00:07:04.572 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:04.567+0000 7f354a64b740 1 -- 172.21.15.12:0/1281453412 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563b7718c240 msgr2=0x563b77195a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:07:04.572 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:04.567+0000 7f354a64b740 1 --2- 172.21.15.12:0/1281453412 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563b7718c240 0x563b77195a30 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f3534011010 tx=0x7f3534005280 comp rx=0 tx=0).stop 2024-09-11T00:07:04.572 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:04.567+0000 7f354a64b740 1 -- 172.21.15.12:0/1281453412 shutdown_connections 2024-09-11T00:07:04.572 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:04.567+0000 7f354a64b740 1 --2- 172.21.15.12:0/1281453412 >> [v2:172.21.15.178:6838/70628097,v1:172.21.15.178:6839/70628097] conn(0x563b771a9620 0x563b771aaa90 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:07:04.572 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:04.567+0000 7f354a64b740 1 --2- 172.21.15.12:0/1281453412 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563b77198af0 0x563b771964b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:07:04.572 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:04.567+0000 7f354a64b740 1 --2- 172.21.15.12:0/1281453412 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563b7718cb60 0x563b77195f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:07:04.572 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:04.567+0000 7f354a64b740 1 --2- 172.21.15.12:0/1281453412 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563b7718c240 0x563b77195a30 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:07:04.572 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:04.567+0000 7f3526a00640 0 client.25432 ms_handle_reset on v2:172.21.15.178:6838/70628097 2024-09-11T00:07:04.572 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:04.567+0000 7f354a64b740 1 -- 172.21.15.12:0/1281453412 >> 172.21.15.12:0/1281453412 conn(0x563b7718a020 msgr2=0x563b7718fc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:07:04.572 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:04.567+0000 7f354a64b740 1 -- 172.21.15.12:0/1281453412 shutdown_connections 2024-09-11T00:07:04.572 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:04.567+0000 7f354a64b740 1 -- 172.21.15.12:0/1281453412 wait complete. 2024-09-11T00:07:04.573 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:04.567+0000 7f354a64b740 20 client.25432 trim_cache size 0 max 16384 2024-09-11T00:07:04.590 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:04.587+0000 7f354a64b740 1 Processor -- start 2024-09-11T00:07:04.591 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:04.587+0000 7f354a64b740 1 -- start start 2024-09-11T00:07:04.591 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:04.587+0000 7f354a64b740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563b7718c240 0x563b771902b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:07:04.591 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:04.587+0000 7f354a64b740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563b7718cb60 0x563b771942b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:07:04.591 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:04.587+0000 7f354a64b740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563b77198af0 0x563b771947f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:07:04.591 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:04.587+0000 7f354a64b740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x563b771a8dd0 con 0x563b7718cb60 2024-09-11T00:07:04.591 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:04.587+0000 7f354a64b740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x563b771a94a0 con 0x563b7718c240 2024-09-11T00:07:04.591 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:04.587+0000 7f354a64b740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x563b771a9b70 con 0x563b77198af0 2024-09-11T00:07:04.592 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:04.587+0000 7f353fe00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563b7718cb60 0x563b771942b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:07:04.592 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:04.587+0000 7f353fe00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563b7718cb60 0x563b771942b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:47006/0 (socket says 172.21.15.12:47006) 2024-09-11T00:07:04.592 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:04.587+0000 7f353fe00640 1 -- 172.21.15.12:0/3642414525 learned_addr learned my addr 172.21.15.12:0/3642414525 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:07:04.592 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:04.587+0000 7f3545200640 1 --2- 172.21.15.12:0/3642414525 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563b7718c240 0x563b771902b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:07:04.592 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:04.587+0000 7f3545c00640 1 --2- 172.21.15.12:0/3642414525 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563b77198af0 0x563b771947f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:07:04.592 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:04.588+0000 7f353fe00640 1 -- 172.21.15.12:0/3642414525 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563b77198af0 msgr2=0x563b771947f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:07:04.592 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:04.588+0000 7f353fe00640 1 --2- 172.21.15.12:0/3642414525 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563b77198af0 0x563b771947f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:07:04.592 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:04.588+0000 7f353fe00640 1 -- 172.21.15.12:0/3642414525 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563b7718c240 msgr2=0x563b771902b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:07:04.593 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:04.588+0000 7f353fe00640 1 --2- 172.21.15.12:0/3642414525 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563b7718c240 0x563b771902b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:07:04.593 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:04.588+0000 7f353fe00640 1 -- 172.21.15.12:0/3642414525 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563b7719b890 con 0x563b7718cb60 2024-09-11T00:07:04.593 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:04.588+0000 7f3545200640 1 --2- 172.21.15.12:0/3642414525 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563b7718c240 0x563b771902b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:07:04.593 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:04.588+0000 7f353fe00640 1 --2- 172.21.15.12:0/3642414525 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563b7718cb60 0x563b771942b0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f352c0a8d00 tx=0x7f352c000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae16be87f6156b13 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:07:04.593 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:04.588+0000 7f3531600640 1 -- 172.21.15.12:0/3642414525 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f352c0b5e60 con 0x563b7718cb60 2024-09-11T00:07:04.593 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:04.588+0000 7f3531600640 1 -- 172.21.15.12:0/3642414525 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f352c0ac240 con 0x563b7718cb60 2024-09-11T00:07:04.593 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:04.588+0000 7f3531600640 1 -- 172.21.15.12:0/3642414525 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f352c0a89a0 con 0x563b7718cb60 2024-09-11T00:07:04.597 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:04.589+0000 7f354a64b740 1 -- 172.21.15.12:0/3642414525 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563b7718cb60 msgr2=0x563b771942b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:07:04.597 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:04.589+0000 7f354a64b740 1 --2- 172.21.15.12:0/3642414525 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563b7718cb60 0x563b771942b0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f352c0a8d00 tx=0x7f352c000bc0 comp rx=0 tx=0).stop 2024-09-11T00:07:04.597 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:04.589+0000 7f354a64b740 1 -- 172.21.15.12:0/3642414525 shutdown_connections 2024-09-11T00:07:04.597 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:04.589+0000 7f354a64b740 1 --2- 172.21.15.12:0/3642414525 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563b77198af0 0x563b771947f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:07:04.597 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:04.589+0000 7f354a64b740 1 --2- 172.21.15.12:0/3642414525 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563b7718cb60 0x563b771942b0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:07:04.597 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:04.589+0000 7f354a64b740 1 --2- 172.21.15.12:0/3642414525 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563b7718c240 0x563b771902b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:07:04.598 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:04.589+0000 7f354a64b740 1 -- 172.21.15.12:0/3642414525 >> 172.21.15.12:0/3642414525 conn(0x563b7718a020 msgr2=0x563b771a8630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:07:04.598 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:04.589+0000 7f354a64b740 1 -- 172.21.15.12:0/3642414525 shutdown_connections 2024-09-11T00:07:04.598 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:04.589+0000 7f354a64b740 1 -- 172.21.15.12:0/3642414525 wait complete. 2024-09-11T00:07:04.598 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:04.590+0000 7f354a64b740 1 -- start start 2024-09-11T00:07:04.598 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:04.590+0000 7f354a64b740 1 Processor -- start 2024-09-11T00:07:04.598 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:04.590+0000 7f354a64b740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563b7718c240 0x563b771b3110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:07:04.598 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:04.590+0000 7f354a64b740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563b7718cb60 0x563b771b36b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:07:04.598 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:04.590+0000 7f354a64b740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563b77198af0 0x563b771ac7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:07:04.598 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:04.590+0000 7f354a64b740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x563b771aa910 con 0x563b7718c240 2024-09-11T00:07:04.598 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:04.590+0000 7f354a64b740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x563b771aa240 con 0x563b77198af0 2024-09-11T00:07:04.598 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:04.590+0000 7f354a64b740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x563b771aafe0 con 0x563b7718cb60 2024-09-11T00:07:04.598 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:04.590+0000 7f354a64b740 10 client.0 _ll_register_callbacks cb 0x7f353400bdd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:07:04.598 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:04.591+0000 7f3545200640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563b7718c240 0x563b771b3110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:07:04.599 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:04.591+0000 7f3545200640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563b7718c240 0x563b771b3110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:47012/0 (socket says 172.21.15.12:47012) 2024-09-11T00:07:04.599 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:04.591+0000 7f3545200640 1 -- 172.21.15.12:0/1994312462 learned_addr learned my addr 172.21.15.12:0/1994312462 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:07:04.599 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:04.591+0000 7f3545c00640 1 --2- 172.21.15.12:0/1994312462 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563b77198af0 0x563b771ac7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:07:04.599 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:04.591+0000 7f353fe00640 1 --2- 172.21.15.12:0/1994312462 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563b7718cb60 0x563b771b36b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:07:04.599 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:04.591+0000 7f3545200640 1 -- 172.21.15.12:0/1994312462 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563b7718cb60 msgr2=0x563b771b36b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:07:04.599 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:04.591+0000 7f3545200640 1 --2- 172.21.15.12:0/1994312462 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563b7718cb60 0x563b771b36b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:07:04.599 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:04.591+0000 7f3545200640 1 -- 172.21.15.12:0/1994312462 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563b77198af0 msgr2=0x563b771ac7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:07:04.599 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:04.591+0000 7f3545200640 1 --2- 172.21.15.12:0/1994312462 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563b77198af0 0x563b771ac7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:07:04.599 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:04.591+0000 7f3545200640 1 -- 172.21.15.12:0/1994312462 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563b771ad020 con 0x563b7718c240 2024-09-11T00:07:04.599 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:04.591+0000 7f353fe00640 1 --2- 172.21.15.12:0/1994312462 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563b7718cb60 0x563b771b36b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T00:07:04.599 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:04.591+0000 7f3545200640 1 --2- 172.21.15.12:0/1994312462 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563b7718c240 0x563b771b3110 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f353400bac0 tx=0x7f3534005c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13ebe4a1782f1145 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:07:04.599 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:04.591+0000 7f3526a00640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-09-11T00:07:04.599 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:04.592+0000 7f3526a00640 1 -- 172.21.15.12:0/1994312462 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3534039060 con 0x563b7718c240 2024-09-11T00:07:04.600 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:04.592+0000 7f3526a00640 1 -- 172.21.15.12:0/1994312462 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 9208+0+0 (secure 0 0 0) 0x7f3534011850 con 0x563b7718c240 2024-09-11T00:07:04.600 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:04.592+0000 7f354a64b740 1 -- 172.21.15.12:0/1994312462 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f352c0ac240 con 0x563b7718c240 2024-09-11T00:07:04.600 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:04.592+0000 7f3526a00640 1 -- 172.21.15.12:0/1994312462 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f3534040050 con 0x563b7718c240 2024-09-11T00:07:04.600 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:04.592+0000 7f354a64b740 10 client.15417 fetch_fsmap learned FSMap version 29 2024-09-11T00:07:04.600 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:04.592+0000 7f354a64b740 1 -- 172.21.15.12:0/1994312462 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x563b771b2a00 con 0x563b7718c240 2024-09-11T00:07:04.600 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:04.592+0000 7f3526a00640 1 -- 172.21.15.12:0/1994312462 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f352c0ac240 con 0x563b7718c240 2024-09-11T00:07:04.600 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:04.592+0000 7f3526a00640 1 -- 172.21.15.12:0/1994312462 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7f3534023040 con 0x563b7718c240 2024-09-11T00:07:04.600 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:04.592+0000 7f354a64b740 10 client.15417 fetch_fsmap finished waiting for FSMap version 29 2024-09-11T00:07:04.600 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:04.593+0000 7f354a64b740 10 client.15417 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-11T00:07:04.600 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:04.593+0000 7f354a64b740 20 client.15417 populate_metadata read hostname 'smithi012' 2024-09-11T00:07:04.600 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:04.593+0000 7f354a64b740 1 --2- 172.21.15.12:0/1994312462 >> [v2:172.21.15.178:6838/70628097,v1:172.21.15.178:6839/70628097] conn(0x563b7721a700 0x563b77219410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:07:04.601 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:04.593+0000 7f354a64b740 4 client.15417 mds_command: new command op to 24637 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-11T00:07:04.601 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:04.593+0000 7f354a64b740 1 -- 172.21.15.12:0/1994312462 --> [v2:172.21.15.178:6838/70628097,v1:172.21.15.178:6839/70628097] -- command(tid 0: {"prefix": "status"}) -- 0x563b77101750 con 0x563b7721a700 2024-09-11T00:07:04.601 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:04.593+0000 7f353fe00640 1 --2- 172.21.15.12:0/1994312462 >> [v2:172.21.15.178:6838/70628097,v1:172.21.15.178:6839/70628097] conn(0x563b7721a700 0x563b77219410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:07:04.601 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:04.593+0000 7f353fe00640 1 --2- 172.21.15.12:0/1994312462 >> [v2:172.21.15.178:6838/70628097,v1:172.21.15.178:6839/70628097] conn(0x563b7721a700 0x563b77219410 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=d2ec7329774aef57 server_cookie=29c7c654ce3d4014 in_seq=0 out_seq=0 2024-09-11T00:07:04.601 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:04.593+0000 7f3526a00640 10 client.15417 ms_handle_connect on v2:172.21.15.178:6838/70628097 2024-09-11T00:07:04.601 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:04.594+0000 7f3526a00640 1 -- 172.21.15.12:0/1994312462 <== mds.3 v2:172.21.15.178:6838/70628097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x563b77101750 con 0x563b7721a700 2024-09-11T00:07:04.601 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:04.594+0000 7f3526a00640 10 client.15417 handle_command_reply: tid=0 2024-09-11T00:07:04.601 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:04.594+0000 7f354a64b740 1 client.15417 shutdown 2024-09-11T00:07:04.601 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:04.594+0000 7f354a64b740 1 -- 172.21.15.12:0/1994312462 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563b7718c240 msgr2=0x563b771b3110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:07:04.601 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:04.594+0000 7f354a64b740 1 --2- 172.21.15.12:0/1994312462 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563b7718c240 0x563b771b3110 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f353400bac0 tx=0x7f3534005c90 comp rx=0 tx=0).stop 2024-09-11T00:07:04.601 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:04.594+0000 7f354a64b740 1 -- 172.21.15.12:0/1994312462 shutdown_connections 2024-09-11T00:07:04.601 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:04.594+0000 7f354a64b740 1 --2- 172.21.15.12:0/1994312462 >> [v2:172.21.15.178:6838/70628097,v1:172.21.15.178:6839/70628097] conn(0x563b7721a700 0x563b77219410 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:07:04.602 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:04.594+0000 7f354a64b740 1 --2- 172.21.15.12:0/1994312462 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563b77198af0 0x563b771ac7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:07:04.602 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:04.594+0000 7f354a64b740 1 --2- 172.21.15.12:0/1994312462 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563b7718cb60 0x563b771b36b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:07:04.602 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:04.594+0000 7f354a64b740 1 --2- 172.21.15.12:0/1994312462 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563b7718c240 0x563b771b3110 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:07:04.603 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:04.594+0000 7f354a64b740 1 -- 172.21.15.12:0/1994312462 >> 172.21.15.12:0/1994312462 conn(0x563b7718a020 msgr2=0x563b771941f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:07:04.604 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:04.594+0000 7f3526a00640 0 client.15417 ms_handle_reset on v2:172.21.15.178:6838/70628097 2024-09-11T00:07:04.604 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:04.594+0000 7f354a64b740 1 -- 172.21.15.12:0/1994312462 shutdown_connections 2024-09-11T00:07:04.604 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:04.594+0000 7f354a64b740 1 -- 172.21.15.12:0/1994312462 wait complete. 2024-09-11T00:07:04.604 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:04.594+0000 7f354a64b740 20 client.15417 trim_cache size 0 max 16384 2024-09-11T00:07:04.605 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-09-11T00:07:04.605 INFO:teuthology.orchestra.run.smithi012.stdout: "cluster_fsid": "ff3ad74a-6fcf-11ef-bcea-c7b262605968", 2024-09-11T00:07:04.605 INFO:teuthology.orchestra.run.smithi012.stdout: "whoami": 3, 2024-09-11T00:07:04.605 INFO:teuthology.orchestra.run.smithi012.stdout: "id": 24637, 2024-09-11T00:07:04.605 INFO:teuthology.orchestra.run.smithi012.stdout: "want_state": "up:active", 2024-09-11T00:07:04.605 INFO:teuthology.orchestra.run.smithi012.stdout: "state": "up:active", 2024-09-11T00:07:04.605 INFO:teuthology.orchestra.run.smithi012.stdout: "fs_name": "cephfs", 2024-09-11T00:07:04.605 INFO:teuthology.orchestra.run.smithi012.stdout: "rank_uptime": 136.360868348, 2024-09-11T00:07:04.605 INFO:teuthology.orchestra.run.smithi012.stdout: "mdsmap_epoch": 29, 2024-09-11T00:07:04.605 INFO:teuthology.orchestra.run.smithi012.stdout: "osdmap_epoch": 85, 2024-09-11T00:07:04.606 INFO:teuthology.orchestra.run.smithi012.stdout: "osdmap_epoch_barrier": 85, 2024-09-11T00:07:04.606 INFO:teuthology.orchestra.run.smithi012.stdout: "uptime": 202.38846588800001 2024-09-11T00:07:04.606 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-09-11T00:07:04.607 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:04.595+0000 7f3547c00640 1 -- 172.21.15.12:0/221360776 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f35280b52e0 msgr2=0x7f35280b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:07:04.607 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:04.595+0000 7f3547c00640 1 --2- 172.21.15.12:0/221360776 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f35280b52e0 0x7f35280b77a0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f3538009ca0 tx=0x7f35380099a0 comp rx=0 tx=0).stop 2024-09-11T00:07:04.607 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:04.595+0000 7f3547c00640 1 -- 172.21.15.12:0/221360776 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3540077ff0 msgr2=0x7f354019fd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:07:04.607 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:04.595+0000 7f3547c00640 1 --2- 172.21.15.12:0/221360776 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3540077ff0 0x7f354019fd80 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f352c00e9f0 tx=0x7f352c00eeb0 comp rx=0 tx=0).stop 2024-09-11T00:07:04.607 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:04.595+0000 7f3547c00640 1 -- 172.21.15.12:0/221360776 shutdown_connections 2024-09-11T00:07:04.607 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:04.595+0000 7f3547c00640 1 --2- 172.21.15.12:0/221360776 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f35280b52e0 0x7f35280b77a0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:07:04.607 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:04.595+0000 7f3547c00640 1 --2- 172.21.15.12:0/221360776 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3540113980 0x7f35401a5120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:07:04.607 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:04.595+0000 7f3547c00640 1 --2- 172.21.15.12:0/221360776 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3540077ff0 0x7f354019fd80 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:07:04.607 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:04.595+0000 7f3547c00640 1 --2- 172.21.15.12:0/221360776 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f35400776d0 0x7f354019f840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:07:04.607 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:04.595+0000 7f3547c00640 1 -- 172.21.15.12:0/221360776 >> 172.21.15.12:0/221360776 conn(0x7f3540100a00 msgr2=0x7f3540102df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:07:04.607 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:04.595+0000 7f3547c00640 1 -- 172.21.15.12:0/221360776 shutdown_connections 2024-09-11T00:07:04.607 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:04.595+0000 7f3547c00640 1 -- 172.21.15.12:0/221360776 wait complete. 2024-09-11T00:07:05.418 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid ff3ad74a-6fcf-11ef-bcea-c7b262605968 -- ceph tell mds.h status 2024-09-11T00:07:05.580 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:07:05 smithi178 ceph-mon[35919]: pgmap v398: 129 pgs: 129 active+clean; 663 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 2.5 KiB/s wr, 13 op/s 2024-09-11T00:07:05.782 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:07:05 smithi165 ceph-mon[35356]: pgmap v398: 129 pgs: 129 active+clean; 663 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 2.5 KiB/s wr, 13 op/s 2024-09-11T00:07:05.798 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:07:05.812 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:07:05 smithi012 ceph-mon[30333]: pgmap v398: 129 pgs: 129 active+clean; 663 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 2.5 KiB/s wr, 13 op/s 2024-09-11T00:07:06.352 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:06.349+0000 7f4fe7200640 1 -- 172.21.15.12:0/3419788210 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4fe810b1a0 msgr2=0x7f4fe81119a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:07:06.352 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:06.349+0000 7f4fe7200640 1 --2- 172.21.15.12:0/3419788210 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4fe810b1a0 0x7f4fe81119a0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f4fe0009ca0 tx=0x7f4fe002a8c0 comp rx=0 tx=0).stop 2024-09-11T00:07:06.352 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:06.349+0000 7f4fe7200640 1 -- 172.21.15.12:0/3419788210 shutdown_connections 2024-09-11T00:07:06.352 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:06.349+0000 7f4fe7200640 1 --2- 172.21.15.12:0/3419788210 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4fe810b1a0 0x7f4fe81119a0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:07:06.352 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:06.349+0000 7f4fe7200640 1 --2- 172.21.15.12:0/3419788210 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4fe810a820 0x7f4fe810ac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:07:06.352 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:06.349+0000 7f4fe7200640 1 --2- 172.21.15.12:0/3419788210 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4fe8104e10 0x7f4fe81051f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:07:06.352 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:06.349+0000 7f4fe7200640 1 -- 172.21.15.12:0/3419788210 >> 172.21.15.12:0/3419788210 conn(0x7f4fe8100840 msgr2=0x7f4fe8102c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:07:06.352 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:06.349+0000 7f4fe7200640 1 -- 172.21.15.12:0/3419788210 shutdown_connections 2024-09-11T00:07:06.353 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:06.350+0000 7f4fe7200640 1 -- 172.21.15.12:0/3419788210 wait complete. 2024-09-11T00:07:06.353 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:06.351+0000 7f4fe7200640 1 Processor -- start 2024-09-11T00:07:06.353 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:06.351+0000 7f4fe7200640 1 -- start start 2024-09-11T00:07:06.354 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:06.352+0000 7f4fe7200640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4fe8104e10 0x7f4fe819b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:07:06.354 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:06.352+0000 7f4fe7200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4fe810a820 0x7f4fe819b850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:07:06.354 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:06.352+0000 7f4fe7200640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4fe810b1a0 0x7f4fe81a0bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:07:06.354 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:06.352+0000 7f4fe7200640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f4fe8114ea0 con 0x7f4fe8104e10 2024-09-11T00:07:06.354 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:06.352+0000 7f4fe7200640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f4fe8114d20 con 0x7f4fe810a820 2024-09-11T00:07:06.355 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:06.352+0000 7f4fe7200640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f4fe8115020 con 0x7f4fe810b1a0 2024-09-11T00:07:06.355 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:06.352+0000 7f4fe5e00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4fe8104e10 0x7f4fe819b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:07:06.355 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:06.352+0000 7f4fe5e00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4fe8104e10 0x7f4fe819b310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:47034/0 (socket says 172.21.15.12:47034) 2024-09-11T00:07:06.355 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:06.352+0000 7f4fe5e00640 1 -- 172.21.15.12:0/453083350 learned_addr learned my addr 172.21.15.12:0/453083350 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:07:06.355 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:06.352+0000 7f4fe5400640 1 --2- 172.21.15.12:0/453083350 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4fe810a820 0x7f4fe819b850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:07:06.355 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:06.352+0000 7f4fe6800640 1 --2- 172.21.15.12:0/453083350 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4fe810b1a0 0x7f4fe81a0bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:07:06.356 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:06.353+0000 7f4fe5e00640 1 -- 172.21.15.12:0/453083350 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4fe810b1a0 msgr2=0x7f4fe81a0bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:07:06.356 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:06.353+0000 7f4fe5e00640 1 --2- 172.21.15.12:0/453083350 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4fe810b1a0 0x7f4fe81a0bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:07:06.356 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:06.353+0000 7f4fe5e00640 1 -- 172.21.15.12:0/453083350 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4fe810a820 msgr2=0x7f4fe819b850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:07:06.356 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:06.353+0000 7f4fe5e00640 1 --2- 172.21.15.12:0/453083350 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4fe810a820 0x7f4fe819b850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:07:06.356 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:06.353+0000 7f4fe5e00640 1 -- 172.21.15.12:0/453083350 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4fe81a13c0 con 0x7f4fe8104e10 2024-09-11T00:07:06.357 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:06.353+0000 7f4fe5e00640 1 --2- 172.21.15.12:0/453083350 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4fe8104e10 0x7f4fe819b310 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f4fd800b4a0 tx=0x7f4fd800b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:07:06.357 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:06.353+0000 7f4fdea00640 1 -- 172.21.15.12:0/453083350 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fd8014070 con 0x7f4fe8104e10 2024-09-11T00:07:06.357 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:06.353+0000 7f4fdea00640 1 -- 172.21.15.12:0/453083350 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4fd8010ac0 con 0x7f4fe8104e10 2024-09-11T00:07:06.357 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:06.353+0000 7f4fdea00640 1 -- 172.21.15.12:0/453083350 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fd8017c30 con 0x7f4fe8104e10 2024-09-11T00:07:06.357 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:06.353+0000 7f4fe7200640 1 -- 172.21.15.12:0/453083350 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4fe81a16b0 con 0x7f4fe8104e10 2024-09-11T00:07:06.357 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:06.354+0000 7f4fe7200640 1 -- 172.21.15.12:0/453083350 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4fe81a7df0 con 0x7f4fe8104e10 2024-09-11T00:07:06.360 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:06.358+0000 7f4fef5e0740 1 Processor -- start 2024-09-11T00:07:06.360 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:06.358+0000 7f4fef5e0740 1 -- start start 2024-09-11T00:07:06.361 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:06.359+0000 7f4fef5e0740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fcf4a61400 0x55fcf4a617e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:07:06.363 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:06.359+0000 7f4fef5e0740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fcf4a61d20 0x55fcf4a6d770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:07:06.363 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:06.359+0000 7f4fef5e0740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fcf4a6dcb0 0x55fcf4a700a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:07:06.363 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:06.359+0000 7f4fef5e0740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55fcf49d8d50 con 0x55fcf4a61d20 2024-09-11T00:07:06.363 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:06.359+0000 7f4fef5e0740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55fcf49c8180 con 0x55fcf4a6dcb0 2024-09-11T00:07:06.364 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:06.359+0000 7f4fef5e0740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55fcf49cbf90 con 0x55fcf4a61400 2024-09-11T00:07:06.364 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:06.359+0000 7f4fe6800640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fcf4a61400 0x55fcf4a617e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:07:06.364 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:06.359+0000 7f4fe6800640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fcf4a61400 0x55fcf4a617e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.12:56034/0 (socket says 172.21.15.12:56034) 2024-09-11T00:07:06.364 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:06.359+0000 7f4fe6800640 1 -- 172.21.15.12:0/4234029472 learned_addr learned my addr 172.21.15.12:0/4234029472 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:07:06.364 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:06.360+0000 7f4fdea00640 1 -- 172.21.15.12:0/453083350 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7f4fd8010610 con 0x7f4fe8104e10 2024-09-11T00:07:06.365 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:06.360+0000 7f4fe5400640 1 --2- 172.21.15.12:0/4234029472 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fcf4a6dcb0 0x55fcf4a700a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:07:06.365 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:06.360+0000 7f4fe6800640 1 -- 172.21.15.12:0/4234029472 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fcf4a6dcb0 msgr2=0x55fcf4a700a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:07:06.365 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:06.360+0000 7f4fe6800640 1 --2- 172.21.15.12:0/4234029472 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fcf4a6dcb0 0x55fcf4a700a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:07:06.365 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:06.360+0000 7f4fe5e00640 1 --2- 172.21.15.12:0/4234029472 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fcf4a61d20 0x55fcf4a6d770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:07:06.365 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:06.360+0000 7f4fe6800640 1 -- 172.21.15.12:0/4234029472 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fcf4a61d20 msgr2=0x55fcf4a6d770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:07:06.365 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:06.360+0000 7f4fe6800640 1 --2- 172.21.15.12:0/4234029472 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fcf4a61d20 0x55fcf4a6d770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:07:06.366 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:06.360+0000 7f4fe6800640 1 -- 172.21.15.12:0/4234029472 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fcf4992a70 con 0x55fcf4a61400 2024-09-11T00:07:06.366 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:06.361+0000 7f4fe5e00640 1 --2- 172.21.15.12:0/4234029472 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fcf4a61d20 0x55fcf4a6d770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T00:07:06.366 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:06.361+0000 7f4fe6800640 1 --2- 172.21.15.12:0/4234029472 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fcf4a61400 0x55fcf4a617e0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f4fe002f4f0 tx=0x7f4fe0031af0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=df423d7a6e9eea68 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:07:06.373 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:06.362+0000 7f4fd3000640 1 -- 172.21.15.12:0/4234029472 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fe0031650 con 0x55fcf4a61400 2024-09-11T00:07:06.374 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:06.362+0000 7f4fd3000640 1 -- 172.21.15.12:0/4234029472 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4fe0036070 con 0x55fcf4a61400 2024-09-11T00:07:06.374 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:06.362+0000 7f4fd3000640 1 -- 172.21.15.12:0/4234029472 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fe0004860 con 0x55fcf4a61400 2024-09-11T00:07:06.374 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:06.362+0000 7f4fdea00640 1 --2- 172.21.15.12:0/453083350 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f4fcc0b5560 0x7f4fcc0b7a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:07:06.374 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:06.362+0000 7f4fdea00640 1 -- 172.21.15.12:0/453083350 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9208+0+0 (secure 0 0 0) 0x7f4fd80e5130 con 0x7f4fe8104e10 2024-09-11T00:07:06.374 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:06.363+0000 7f4fe5e00640 1 --2- 172.21.15.12:0/453083350 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f4fcc0b5560 0x7f4fcc0b7a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:07:06.375 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:06.363+0000 7f4fef5e0740 1 -- 172.21.15.12:0/4234029472 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fcf4a61400 msgr2=0x55fcf4a617e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:07:06.375 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:06.363+0000 7f4fef5e0740 1 --2- 172.21.15.12:0/4234029472 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fcf4a61400 0x55fcf4a617e0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f4fe002f4f0 tx=0x7f4fe0031af0 comp rx=0 tx=0).stop 2024-09-11T00:07:06.375 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:06.363+0000 7f4fef5e0740 1 -- 172.21.15.12:0/4234029472 shutdown_connections 2024-09-11T00:07:06.375 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:06.363+0000 7f4fef5e0740 1 --2- 172.21.15.12:0/4234029472 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fcf4a6dcb0 0x55fcf4a700a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:07:06.375 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:06.363+0000 7f4fef5e0740 1 --2- 172.21.15.12:0/4234029472 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fcf4a61d20 0x55fcf4a6d770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:07:06.375 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:06.363+0000 7f4fef5e0740 1 --2- 172.21.15.12:0/4234029472 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fcf4a61400 0x55fcf4a617e0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:07:06.376 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:06.363+0000 7f4fef5e0740 1 -- 172.21.15.12:0/4234029472 >> 172.21.15.12:0/4234029472 conn(0x55fcf4a5f230 msgr2=0x55fcf4a603d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:07:06.376 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:06.363+0000 7f4fef5e0740 1 -- 172.21.15.12:0/4234029472 shutdown_connections 2024-09-11T00:07:06.376 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:06.363+0000 7f4fef5e0740 1 -- 172.21.15.12:0/4234029472 wait complete. 2024-09-11T00:07:06.376 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:06.364+0000 7f4fe5e00640 1 --2- 172.21.15.12:0/453083350 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f4fcc0b5560 0x7f4fcc0b7a20 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f4fd800be60 tx=0x7f4fd8010f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:07:06.376 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:06.364+0000 7f4fef5e0740 1 -- start start 2024-09-11T00:07:06.376 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:06.364+0000 7f4fef5e0740 1 Processor -- start 2024-09-11T00:07:06.376 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:06.365+0000 7f4fef5e0740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fcf4a61400 0x55fcf4a6abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:07:06.376 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:06.365+0000 7f4fef5e0740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fcf4a61d20 0x55fcf4a6b110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:07:06.377 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:06.365+0000 7f4fef5e0740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fcf4a6dcb0 0x55fcf4a6b650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:07:06.377 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:06.365+0000 7f4fef5e0740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55fcf48295f0 con 0x55fcf4a61d20 2024-09-11T00:07:06.377 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:06.365+0000 7f4fef5e0740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55fcf4963460 con 0x55fcf4a6dcb0 2024-09-11T00:07:06.377 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:06.365+0000 7f4fef5e0740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55fcf4a70820 con 0x55fcf4a61400 2024-09-11T00:07:06.377 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:06.365+0000 7f4fef5e0740 10 client.0 _ll_register_callbacks cb 0x55fcf4973210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:07:06.377 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:06.365+0000 7f4fe6800640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fcf4a61400 0x55fcf4a6abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:07:06.377 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:06.365+0000 7f4fe6800640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fcf4a61400 0x55fcf4a6abd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.12:56044/0 (socket says 172.21.15.12:56044) 2024-09-11T00:07:06.377 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:06.365+0000 7f4fe6800640 1 -- 172.21.15.12:0/3861926092 learned_addr learned my addr 172.21.15.12:0/3861926092 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:07:06.378 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:06.365+0000 7f4fe5400640 1 --2- 172.21.15.12:0/3861926092 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fcf4a61d20 0x55fcf4a6b110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:07:06.378 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:06.365+0000 7f4fe6800640 1 --2- 172.21.15.12:0/3861926092 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fcf4a6dcb0 0x55fcf4a6b650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:07:06.378 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:06.366+0000 7f4fe6800640 1 -- 172.21.15.12:0/3861926092 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fcf4a61400 msgr2=0x55fcf4a6abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:07:06.378 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:06.366+0000 7f4fe6800640 1 --2- 172.21.15.12:0/3861926092 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fcf4a61400 0x55fcf4a6abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:07:06.378 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:06.366+0000 7f4fe6800640 1 -- 172.21.15.12:0/3861926092 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fcf4a61d20 msgr2=0x55fcf4a6b110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:07:06.378 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:06.366+0000 7f4fe6800640 1 --2- 172.21.15.12:0/3861926092 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fcf4a61d20 0x55fcf4a6b110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:07:06.378 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:06.366+0000 7f4fe6800640 1 -- 172.21.15.12:0/3861926092 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fcf4726540 con 0x55fcf4a6dcb0 2024-09-11T00:07:06.379 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:06.366+0000 7f4fe5400640 1 --2- 172.21.15.12:0/3861926092 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fcf4a61d20 0x55fcf4a6b110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:07:06.379 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:06.366+0000 7f4fe6800640 1 --2- 172.21.15.12:0/3861926092 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fcf4a6dcb0 0x55fcf4a6b650 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f4fe0006010 tx=0x7f4fe002a8c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=244fac60443d9c4b server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:07:06.379 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:06.366+0000 7f4fc7400640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-09-11T00:07:06.379 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:06.366+0000 7f4fc7400640 1 -- 172.21.15.12:0/3861926092 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fe0003c40 con 0x55fcf4a6dcb0 2024-09-11T00:07:06.379 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:06.367+0000 7f4fef5e0740 1 -- 172.21.15.12:0/3861926092 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55fcf4965a70 con 0x55fcf4a6dcb0 2024-09-11T00:07:06.380 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:06.367+0000 7f4fc7400640 1 -- 172.21.15.12:0/3861926092 <== mon.2 v2:172.21.15.165:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 9208+0+0 (secure 0 0 0) 0x7f4fe0003de0 con 0x55fcf4a6dcb0 2024-09-11T00:07:06.380 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:06.367+0000 7f4fc7400640 1 -- 172.21.15.12:0/3861926092 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f4fe005b050 con 0x55fcf4a6dcb0 2024-09-11T00:07:06.380 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:06.367+0000 7f4fef5e0740 10 client.25466 fetch_fsmap learned FSMap version 29 2024-09-11T00:07:06.380 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:06.367+0000 7f4fef5e0740 1 -- 172.21.15.12:0/3861926092 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55fcf49806f0 con 0x55fcf4a6dcb0 2024-09-11T00:07:06.380 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:06.368+0000 7f4fc7400640 1 -- 172.21.15.12:0/3861926092 <== mon.2 v2:172.21.15.165:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55fcf4965a70 con 0x55fcf4a6dcb0 2024-09-11T00:07:06.381 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:06.368+0000 7f4fc7400640 1 -- 172.21.15.12:0/3861926092 <== mon.2 v2:172.21.15.165:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7f4fe0043020 con 0x55fcf4a6dcb0 2024-09-11T00:07:06.381 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:06.368+0000 7f4fef5e0740 10 client.25466 fetch_fsmap finished waiting for FSMap version 29 2024-09-11T00:07:06.381 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:06.368+0000 7f4fef5e0740 10 client.25466 resolve_mds: resolved name 'h' to daemon mds.h 2024-09-11T00:07:06.381 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:06.368+0000 7f4fef5e0740 20 client.25466 populate_metadata read hostname 'smithi012' 2024-09-11T00:07:06.381 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:06.368+0000 7f4fef5e0740 1 --2- 172.21.15.12:0/3861926092 >> [v2:172.21.15.165:6836/6215088,v1:172.21.15.165:6837/6215088] conn(0x55fcf4a7e3e0 0x55fcf4a7f850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:07:06.382 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:06.368+0000 7f4fef5e0740 4 client.25466 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T00:07:06.382 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:06.368+0000 7f4fef5e0740 1 -- 172.21.15.12:0/3861926092 --> [v2:172.21.15.165:6836/6215088,v1:172.21.15.165:6837/6215088] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fcf49daca0 con 0x55fcf4a7e3e0 2024-09-11T00:07:06.382 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:06.369+0000 7f4fe5400640 1 --2- 172.21.15.12:0/3861926092 >> [v2:172.21.15.165:6836/6215088,v1:172.21.15.165:6837/6215088] conn(0x55fcf4a7e3e0 0x55fcf4a7f850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:07:06.382 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:06.369+0000 7f4fe5400640 1 --2- 172.21.15.12:0/3861926092 >> [v2:172.21.15.165:6836/6215088,v1:172.21.15.165:6837/6215088] conn(0x55fcf4a7e3e0 0x55fcf4a7f850 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=eb7730948e53cc17 server_cookie=539f25a49c456b6c in_seq=0 out_seq=0 2024-09-11T00:07:06.382 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:06.369+0000 7f4fc7400640 10 client.25466 ms_handle_connect on v2:172.21.15.165:6836/6215088 2024-09-11T00:07:06.382 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:06.370+0000 7f4fc7400640 1 -- 172.21.15.12:0/3861926092 <== mds.2 v2:172.21.15.165:6836/6215088 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55fcf49daca0 con 0x55fcf4a7e3e0 2024-09-11T00:07:06.382 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:06.370+0000 7f4fc7400640 10 client.25466 handle_command_reply: tid=0 2024-09-11T00:07:06.382 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:06.370+0000 7f4fef5e0740 1 client.25466 shutdown 2024-09-11T00:07:06.382 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:06.370+0000 7f4fef5e0740 1 -- 172.21.15.12:0/3861926092 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fcf4a6dcb0 msgr2=0x55fcf4a6b650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:07:06.382 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:06.370+0000 7f4fef5e0740 1 --2- 172.21.15.12:0/3861926092 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fcf4a6dcb0 0x55fcf4a6b650 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f4fe0006010 tx=0x7f4fe002a8c0 comp rx=0 tx=0).stop 2024-09-11T00:07:06.382 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:06.370+0000 7f4fef5e0740 1 -- 172.21.15.12:0/3861926092 shutdown_connections 2024-09-11T00:07:06.382 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:06.370+0000 7f4fef5e0740 1 --2- 172.21.15.12:0/3861926092 >> [v2:172.21.15.165:6836/6215088,v1:172.21.15.165:6837/6215088] conn(0x55fcf4a7e3e0 0x55fcf4a7f850 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:07:06.383 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:06.370+0000 7f4fef5e0740 1 --2- 172.21.15.12:0/3861926092 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fcf4a6dcb0 0x55fcf4a6b650 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:07:06.386 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:06.370+0000 7f4fef5e0740 1 --2- 172.21.15.12:0/3861926092 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fcf4a61d20 0x55fcf4a6b110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:07:06.386 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:06.370+0000 7f4fef5e0740 1 --2- 172.21.15.12:0/3861926092 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fcf4a61400 0x55fcf4a6abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:07:06.386 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:06.370+0000 7f4fef5e0740 1 -- 172.21.15.12:0/3861926092 >> 172.21.15.12:0/3861926092 conn(0x55fcf4a5f230 msgr2=0x55fcf4a64dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:07:06.386 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:06.370+0000 7f4fc7400640 0 client.25466 ms_handle_reset on v2:172.21.15.165:6836/6215088 2024-09-11T00:07:06.386 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:06.371+0000 7f4fef5e0740 1 -- 172.21.15.12:0/3861926092 shutdown_connections 2024-09-11T00:07:06.386 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:06.371+0000 7f4fef5e0740 1 -- 172.21.15.12:0/3861926092 wait complete. 2024-09-11T00:07:06.386 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:06.371+0000 7f4fef5e0740 20 client.25466 trim_cache size 0 max 16384 2024-09-11T00:07:06.393 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:06.391+0000 7f4fef5e0740 1 Processor -- start 2024-09-11T00:07:06.393 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:06.391+0000 7f4fef5e0740 1 -- start start 2024-09-11T00:07:06.393 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:06.391+0000 7f4fef5e0740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fcf4a61400 0x55fcf4a7f850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:07:06.393 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:06.391+0000 7f4fef5e0740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fcf4a61d20 0x55fcf4a655c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:07:06.393 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:06.391+0000 7f4fef5e0740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fcf4a6dcb0 0x55fcf4a65e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:07:06.393 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:06.391+0000 7f4fef5e0740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55fcf4a7db90 con 0x55fcf4a61d20 2024-09-11T00:07:06.393 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:06.391+0000 7f4fef5e0740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55fcf4a7e260 con 0x55fcf4a6dcb0 2024-09-11T00:07:06.394 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:06.391+0000 7f4fef5e0740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55fcf4a7e930 con 0x55fcf4a61400 2024-09-11T00:07:06.394 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:06.391+0000 7f4fe5400640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fcf4a61d20 0x55fcf4a655c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:07:06.395 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:06.391+0000 7f4fe5400640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fcf4a61d20 0x55fcf4a655c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:47068/0 (socket says 172.21.15.12:47068) 2024-09-11T00:07:06.395 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:06.391+0000 7f4fe5400640 1 -- 172.21.15.12:0/1562094623 learned_addr learned my addr 172.21.15.12:0/1562094623 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:07:06.395 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:06.392+0000 7f4fe6800640 1 --2- 172.21.15.12:0/1562094623 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fcf4a61400 0x55fcf4a7f850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:07:06.395 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:06.392+0000 7f4fe6800640 1 --2- 172.21.15.12:0/1562094623 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fcf4a6dcb0 0x55fcf4a65e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:07:06.395 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:06.392+0000 7f4fe5400640 1 -- 172.21.15.12:0/1562094623 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fcf4a61400 msgr2=0x55fcf4a7f850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:07:06.395 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:06.392+0000 7f4fe5400640 1 --2- 172.21.15.12:0/1562094623 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fcf4a61400 0x55fcf4a7f850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:07:06.395 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:06.392+0000 7f4fe5400640 1 -- 172.21.15.12:0/1562094623 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fcf4a6dcb0 msgr2=0x55fcf4a65e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:07:06.395 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:06.392+0000 7f4fe5400640 1 --2- 172.21.15.12:0/1562094623 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fcf4a6dcb0 0x55fcf4a65e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:07:06.395 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:06.392+0000 7f4fe5400640 1 -- 172.21.15.12:0/1562094623 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fcf4a5fcb0 con 0x55fcf4a61d20 2024-09-11T00:07:06.396 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:06.392+0000 7f4fe5400640 1 --2- 172.21.15.12:0/1562094623 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fcf4a61d20 0x55fcf4a655c0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f4fd4006a90 tx=0x7f4fd4008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6930d0de632a81a server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:07:06.396 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:06.393+0000 7f4fd2600640 1 -- 172.21.15.12:0/1562094623 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fd4024380 con 0x55fcf4a61d20 2024-09-11T00:07:06.396 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:06.393+0000 7f4fd2600640 1 -- 172.21.15.12:0/1562094623 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4fd40249d0 con 0x55fcf4a61d20 2024-09-11T00:07:06.396 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:06.393+0000 7f4fd2600640 1 -- 172.21.15.12:0/1562094623 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fd4014700 con 0x55fcf4a61d20 2024-09-11T00:07:06.399 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:06.393+0000 7f4fef5e0740 1 -- 172.21.15.12:0/1562094623 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fcf4a61d20 msgr2=0x55fcf4a655c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:07:06.399 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:06.393+0000 7f4fef5e0740 1 --2- 172.21.15.12:0/1562094623 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fcf4a61d20 0x55fcf4a655c0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f4fd4006a90 tx=0x7f4fd4008040 comp rx=0 tx=0).stop 2024-09-11T00:07:06.400 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:06.393+0000 7f4fef5e0740 1 -- 172.21.15.12:0/1562094623 shutdown_connections 2024-09-11T00:07:06.400 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:06.393+0000 7f4fef5e0740 1 --2- 172.21.15.12:0/1562094623 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fcf4a6dcb0 0x55fcf4a65e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:07:06.400 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:06.393+0000 7f4fef5e0740 1 --2- 172.21.15.12:0/1562094623 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fcf4a61d20 0x55fcf4a655c0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:07:06.400 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:06.393+0000 7f4fef5e0740 1 --2- 172.21.15.12:0/1562094623 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fcf4a61400 0x55fcf4a7f850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:07:06.400 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:06.393+0000 7f4fef5e0740 1 -- 172.21.15.12:0/1562094623 >> 172.21.15.12:0/1562094623 conn(0x55fcf4a5f230 msgr2=0x55fcf4a60850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:07:06.400 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:06.394+0000 7f4fef5e0740 1 -- 172.21.15.12:0/1562094623 shutdown_connections 2024-09-11T00:07:06.401 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:06.394+0000 7f4fef5e0740 1 -- 172.21.15.12:0/1562094623 wait complete. 2024-09-11T00:07:06.401 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:06.394+0000 7f4fef5e0740 1 -- start start 2024-09-11T00:07:06.401 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:06.394+0000 7f4fef5e0740 1 Processor -- start 2024-09-11T00:07:06.401 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:06.395+0000 7f4fef5e0740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fcf4a61400 0x55fcf4a81d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:07:06.401 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:06.395+0000 7f4fef5e0740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fcf4a61d20 0x55fcf4a82260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:07:06.401 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:06.395+0000 7f4fef5e0740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fcf4a6dcb0 0x55fcf4a874b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:07:06.402 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:06.395+0000 7f4fef5e0740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55fcf4a7f6d0 con 0x55fcf4a61d20 2024-09-11T00:07:06.402 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:06.395+0000 7f4fef5e0740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55fcf4a7f000 con 0x55fcf4a61400 2024-09-11T00:07:06.402 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:06.395+0000 7f4fef5e0740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55fcf4a7fda0 con 0x55fcf4a6dcb0 2024-09-11T00:07:06.402 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:06.395+0000 7f4fef5e0740 10 client.0 _ll_register_callbacks cb 0x7f4fe00317f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:07:06.402 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:06.395+0000 7f4fe6800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fcf4a61400 0x55fcf4a81d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:07:06.402 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:06.395+0000 7f4fe6800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fcf4a61400 0x55fcf4a81d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.12:49104/0 (socket says 172.21.15.12:49104) 2024-09-11T00:07:06.402 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:06.395+0000 7f4fe6800640 1 -- 172.21.15.12:0/2909820465 learned_addr learned my addr 172.21.15.12:0/2909820465 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:07:06.402 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:06.395+0000 7f4fe5400640 1 --2- 172.21.15.12:0/2909820465 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fcf4a61d20 0x55fcf4a82260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:07:06.403 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:06.395+0000 7f4fe6800640 1 --2- 172.21.15.12:0/2909820465 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fcf4a6dcb0 0x55fcf4a874b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:07:06.403 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:06.396+0000 7f4fe6800640 1 -- 172.21.15.12:0/2909820465 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fcf4a6dcb0 msgr2=0x55fcf4a874b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:07:06.403 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:06.396+0000 7f4fe6800640 1 --2- 172.21.15.12:0/2909820465 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fcf4a6dcb0 0x55fcf4a874b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:07:06.403 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:06.396+0000 7f4fe6800640 1 -- 172.21.15.12:0/2909820465 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fcf4a61d20 msgr2=0x55fcf4a82260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:07:06.403 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:06.396+0000 7f4fe6800640 1 --2- 172.21.15.12:0/2909820465 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fcf4a61d20 0x55fcf4a82260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:07:06.403 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:06.396+0000 7f4fe6800640 1 -- 172.21.15.12:0/2909820465 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fcf4a82b10 con 0x55fcf4a61400 2024-09-11T00:07:06.404 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:06.396+0000 7f4fe6800640 1 --2- 172.21.15.12:0/2909820465 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fcf4a61400 0x55fcf4a81d20 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f4fe00049d0 tx=0x7f4fe0031a00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=69bb57e1ee21c207 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:07:06.404 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:06.396+0000 7f4fc7400640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-09-11T00:07:06.404 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:06.397+0000 7f4fc7400640 1 -- 172.21.15.12:0/2909820465 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fe0030e80 con 0x55fcf4a61400 2024-09-11T00:07:06.404 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:06.397+0000 7f4fef5e0740 1 -- 172.21.15.12:0/2909820465 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4fd40249d0 con 0x55fcf4a61400 2024-09-11T00:07:06.404 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:06.397+0000 7f4fc7400640 1 -- 172.21.15.12:0/2909820465 <== mon.2 v2:172.21.15.165:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 9208+0+0 (secure 0 0 0) 0x7f4fe0003c40 con 0x55fcf4a61400 2024-09-11T00:07:06.405 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:06.397+0000 7f4fc7400640 1 -- 172.21.15.12:0/2909820465 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f4fe0003e40 con 0x55fcf4a61400 2024-09-11T00:07:06.405 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:06.397+0000 7f4fef5e0740 10 client.25472 fetch_fsmap learned FSMap version 29 2024-09-11T00:07:06.405 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:06.397+0000 7f4fef5e0740 1 -- 172.21.15.12:0/2909820465 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55fcf4af3b40 con 0x55fcf4a61400 2024-09-11T00:07:06.405 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:06.397+0000 7f4fc7400640 1 -- 172.21.15.12:0/2909820465 <== mon.2 v2:172.21.15.165:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4fd40249d0 con 0x55fcf4a61400 2024-09-11T00:07:06.407 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:06.398+0000 7f4fc7400640 1 -- 172.21.15.12:0/2909820465 <== mon.2 v2:172.21.15.165:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7f4fe0043040 con 0x55fcf4a61400 2024-09-11T00:07:06.407 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:06.398+0000 7f4fef5e0740 10 client.25472 fetch_fsmap finished waiting for FSMap version 29 2024-09-11T00:07:06.407 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:06.398+0000 7f4fef5e0740 10 client.25472 resolve_mds: resolved name 'h' to daemon mds.h 2024-09-11T00:07:06.407 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:06.398+0000 7f4fef5e0740 20 client.25472 populate_metadata read hostname 'smithi012' 2024-09-11T00:07:06.407 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:06.398+0000 7f4fef5e0740 1 --2- 172.21.15.12:0/2909820465 >> [v2:172.21.15.165:6836/6215088,v1:172.21.15.165:6837/6215088] conn(0x55fcf4af85c0 0x55fcf4af72d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:07:06.407 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:06.398+0000 7f4fef5e0740 4 client.25472 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-11T00:07:06.408 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:06.398+0000 7f4fef5e0740 1 -- 172.21.15.12:0/2909820465 --> [v2:172.21.15.165:6836/6215088,v1:172.21.15.165:6837/6215088] -- command(tid 0: {"prefix": "status"}) -- 0x55fcf49d6750 con 0x55fcf4af85c0 2024-09-11T00:07:06.408 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:06.399+0000 7f4fe5400640 1 --2- 172.21.15.12:0/2909820465 >> [v2:172.21.15.165:6836/6215088,v1:172.21.15.165:6837/6215088] conn(0x55fcf4af85c0 0x55fcf4af72d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:07:06.408 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:06.399+0000 7f4fc7400640 10 client.25472 ms_handle_connect on v2:172.21.15.165:6836/6215088 2024-09-11T00:07:06.408 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:06.399+0000 7f4fe5400640 1 --2- 172.21.15.12:0/2909820465 >> [v2:172.21.15.165:6836/6215088,v1:172.21.15.165:6837/6215088] conn(0x55fcf4af85c0 0x55fcf4af72d0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=46e8a0850963c4c7 server_cookie=d626aad5b09c5004 in_seq=0 out_seq=0 2024-09-11T00:07:06.408 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:06.400+0000 7f4fc7400640 1 -- 172.21.15.12:0/2909820465 <== mds.2 v2:172.21.15.165:6836/6215088 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55fcf49d6750 con 0x55fcf4af85c0 2024-09-11T00:07:06.409 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:06.400+0000 7f4fc7400640 10 client.25472 handle_command_reply: tid=0 2024-09-11T00:07:06.409 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:06.400+0000 7f4fef5e0740 1 client.25472 shutdown 2024-09-11T00:07:06.409 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:06.400+0000 7f4fef5e0740 1 -- 172.21.15.12:0/2909820465 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fcf4a61400 msgr2=0x55fcf4a81d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:07:06.409 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:06.400+0000 7f4fef5e0740 1 --2- 172.21.15.12:0/2909820465 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fcf4a61400 0x55fcf4a81d20 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f4fe00049d0 tx=0x7f4fe0031a00 comp rx=0 tx=0).stop 2024-09-11T00:07:06.409 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:06.400+0000 7f4fef5e0740 1 -- 172.21.15.12:0/2909820465 shutdown_connections 2024-09-11T00:07:06.409 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:06.400+0000 7f4fef5e0740 1 --2- 172.21.15.12:0/2909820465 >> [v2:172.21.15.165:6836/6215088,v1:172.21.15.165:6837/6215088] conn(0x55fcf4af85c0 0x55fcf4af72d0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:07:06.410 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:06.400+0000 7f4fef5e0740 1 --2- 172.21.15.12:0/2909820465 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fcf4a6dcb0 0x55fcf4a874b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:07:06.410 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:06.400+0000 7f4fef5e0740 1 --2- 172.21.15.12:0/2909820465 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fcf4a61d20 0x55fcf4a82260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:07:06.410 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:06.400+0000 7f4fef5e0740 1 --2- 172.21.15.12:0/2909820465 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fcf4a61400 0x55fcf4a81d20 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:07:06.410 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:06.400+0000 7f4fef5e0740 1 -- 172.21.15.12:0/2909820465 >> 172.21.15.12:0/2909820465 conn(0x55fcf4a5f230 msgr2=0x55fcf4a6c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:07:06.410 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:06.400+0000 7f4fc7400640 0 client.25472 ms_handle_reset on v2:172.21.15.165:6836/6215088 2024-09-11T00:07:06.411 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:06.400+0000 7f4fef5e0740 1 -- 172.21.15.12:0/2909820465 shutdown_connections 2024-09-11T00:07:06.411 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:06.400+0000 7f4fef5e0740 1 -- 172.21.15.12:0/2909820465 wait complete. 2024-09-11T00:07:06.411 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:06.401+0000 7f4fef5e0740 20 client.25472 trim_cache size 0 max 16384 2024-09-11T00:07:06.411 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-09-11T00:07:06.412 INFO:teuthology.orchestra.run.smithi012.stdout: "cluster_fsid": "ff3ad74a-6fcf-11ef-bcea-c7b262605968", 2024-09-11T00:07:06.412 INFO:teuthology.orchestra.run.smithi012.stdout: "whoami": 2, 2024-09-11T00:07:06.412 INFO:teuthology.orchestra.run.smithi012.stdout: "id": 24638, 2024-09-11T00:07:06.412 INFO:teuthology.orchestra.run.smithi012.stdout: "want_state": "up:active", 2024-09-11T00:07:06.412 INFO:teuthology.orchestra.run.smithi012.stdout: "state": "up:active", 2024-09-11T00:07:06.413 INFO:teuthology.orchestra.run.smithi012.stdout: "fs_name": "cephfs", 2024-09-11T00:07:06.413 INFO:teuthology.orchestra.run.smithi012.stdout: "rank_uptime": 139.17335293599999, 2024-09-11T00:07:06.413 INFO:teuthology.orchestra.run.smithi012.stdout: "mdsmap_epoch": 29, 2024-09-11T00:07:06.413 INFO:teuthology.orchestra.run.smithi012.stdout: "osdmap_epoch": 85, 2024-09-11T00:07:06.413 INFO:teuthology.orchestra.run.smithi012.stdout: "osdmap_epoch_barrier": 85, 2024-09-11T00:07:06.414 INFO:teuthology.orchestra.run.smithi012.stdout: "uptime": 206.88533413299999 2024-09-11T00:07:06.414 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-09-11T00:07:06.415 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:06.401+0000 7f4fe7200640 1 -- 172.21.15.12:0/453083350 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f4fcc0b5560 msgr2=0x7f4fcc0b7a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:07:06.416 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:06.401+0000 7f4fe7200640 1 --2- 172.21.15.12:0/453083350 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f4fcc0b5560 0x7f4fcc0b7a20 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f4fd800be60 tx=0x7f4fd8010f90 comp rx=0 tx=0).stop 2024-09-11T00:07:06.416 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:06.401+0000 7f4fe7200640 1 -- 172.21.15.12:0/453083350 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4fe8104e10 msgr2=0x7f4fe819b310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:07:06.416 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:06.401+0000 7f4fe7200640 1 --2- 172.21.15.12:0/453083350 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4fe8104e10 0x7f4fe819b310 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f4fd800b4a0 tx=0x7f4fd800b960 comp rx=0 tx=0).stop 2024-09-11T00:07:06.416 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:06.402+0000 7f4fe7200640 1 -- 172.21.15.12:0/453083350 shutdown_connections 2024-09-11T00:07:06.416 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:06.402+0000 7f4fe7200640 1 --2- 172.21.15.12:0/453083350 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f4fcc0b5560 0x7f4fcc0b7a20 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:07:06.416 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:06.402+0000 7f4fe7200640 1 --2- 172.21.15.12:0/453083350 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4fe810b1a0 0x7f4fe81a0bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:07:06.417 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:06.402+0000 7f4fe7200640 1 --2- 172.21.15.12:0/453083350 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4fe810a820 0x7f4fe819b850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:07:06.417 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:06.402+0000 7f4fe7200640 1 --2- 172.21.15.12:0/453083350 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4fe8104e10 0x7f4fe819b310 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:07:06.417 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:06.402+0000 7f4fe7200640 1 -- 172.21.15.12:0/453083350 >> 172.21.15.12:0/453083350 conn(0x7f4fe8100840 msgr2=0x7f4fe8102c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:07:06.417 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:06.402+0000 7f4fe7200640 1 -- 172.21.15.12:0/453083350 shutdown_connections 2024-09-11T00:07:06.417 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:06.402+0000 7f4fe7200640 1 -- 172.21.15.12:0/453083350 wait complete. 2024-09-11T00:07:07.202 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid ff3ad74a-6fcf-11ef-bcea-c7b262605968 -- ceph tell mds.g status 2024-09-11T00:07:07.563 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:07:07 smithi012 ceph-mon[30333]: pgmap v399: 129 pgs: 129 active+clean; 663 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 2.7 KiB/s wr, 17 op/s 2024-09-11T00:07:07.580 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:07:07 smithi178 ceph-mon[35919]: pgmap v399: 129 pgs: 129 active+clean; 663 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 2.7 KiB/s wr, 17 op/s 2024-09-11T00:07:07.584 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:07:07 smithi165 ceph-mon[35356]: pgmap v399: 129 pgs: 129 active+clean; 663 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 2.7 KiB/s wr, 17 op/s 2024-09-11T00:07:07.641 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:07:08.248 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:08.245+0000 7fdd1ca00640 1 -- 172.21.15.12:0/582521861 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdd18108fa0 msgr2=0x7fdd1810f7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:07:08.248 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:08.245+0000 7fdd1ca00640 1 --2- 172.21.15.12:0/582521861 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdd18108fa0 0x7fdd1810f7a0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fdd10009ca0 tx=0x7fdd1002a8c0 comp rx=0 tx=0).stop 2024-09-11T00:07:08.248 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:08.246+0000 7fdd1ca00640 1 -- 172.21.15.12:0/582521861 shutdown_connections 2024-09-11T00:07:08.248 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:08.246+0000 7fdd1ca00640 1 --2- 172.21.15.12:0/582521861 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdd18108fa0 0x7fdd1810f7a0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:07:08.248 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:08.246+0000 7fdd1ca00640 1 --2- 172.21.15.12:0/582521861 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fdd18108620 0x7fdd18108a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:07:08.249 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:08.246+0000 7fdd1ca00640 1 --2- 172.21.15.12:0/582521861 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdd18102c10 0x7fdd18102ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:07:08.249 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:08.246+0000 7fdd1ca00640 1 -- 172.21.15.12:0/582521861 >> 172.21.15.12:0/582521861 conn(0x7fdd180fe660 msgr2=0x7fdd18100a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:07:08.249 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:08.246+0000 7fdd1ca00640 1 -- 172.21.15.12:0/582521861 shutdown_connections 2024-09-11T00:07:08.249 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:08.246+0000 7fdd1ca00640 1 -- 172.21.15.12:0/582521861 wait complete. 2024-09-11T00:07:08.249 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:08.247+0000 7fdd1ca00640 1 Processor -- start 2024-09-11T00:07:08.250 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:08.247+0000 7fdd1ca00640 1 -- start start 2024-09-11T00:07:08.250 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:08.248+0000 7fdd1ca00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdd18102c10 0x7fdd1819b250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:07:08.250 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:08.248+0000 7fdd1ca00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdd18108620 0x7fdd1819b790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:07:08.250 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:08.248+0000 7fdd1ca00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fdd18108fa0 0x7fdd181a0b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:07:08.250 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:08.248+0000 7fdd1ca00640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fdd18112cb0 con 0x7fdd18102c10 2024-09-11T00:07:08.250 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:08.248+0000 7fdd1ca00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fdd18112b30 con 0x7fdd18108fa0 2024-09-11T00:07:08.250 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:08.248+0000 7fdd1ca00640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fdd18112e30 con 0x7fdd18108620 2024-09-11T00:07:08.251 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:08.248+0000 7fdd16800640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdd18102c10 0x7fdd1819b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:07:08.251 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:08.248+0000 7fdd15e00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdd18108620 0x7fdd1819b790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:07:08.251 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:08.248+0000 7fdd15e00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdd18108620 0x7fdd1819b790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.12:56070/0 (socket says 172.21.15.12:56070) 2024-09-11T00:07:08.251 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:08.248+0000 7fdd16800640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdd18102c10 0x7fdd1819b250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:47090/0 (socket says 172.21.15.12:47090) 2024-09-11T00:07:08.251 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:08.248+0000 7fdd15e00640 1 -- 172.21.15.12:0/4054622757 learned_addr learned my addr 172.21.15.12:0/4054622757 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:07:08.251 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:08.248+0000 7fdd17200640 1 --2- 172.21.15.12:0/4054622757 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fdd18108fa0 0x7fdd181a0b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:07:08.251 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:08.249+0000 7fdd15e00640 1 -- 172.21.15.12:0/4054622757 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fdd18108fa0 msgr2=0x7fdd181a0b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:07:08.251 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:08.249+0000 7fdd15e00640 1 --2- 172.21.15.12:0/4054622757 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fdd18108fa0 0x7fdd181a0b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:07:08.251 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:08.249+0000 7fdd15e00640 1 -- 172.21.15.12:0/4054622757 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdd18102c10 msgr2=0x7fdd1819b250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:07:08.251 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:08.249+0000 7fdd15e00640 1 --2- 172.21.15.12:0/4054622757 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdd18102c10 0x7fdd1819b250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:07:08.251 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:08.249+0000 7fdd15e00640 1 -- 172.21.15.12:0/4054622757 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd181a1300 con 0x7fdd18108620 2024-09-11T00:07:08.252 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:08.249+0000 7fdd15e00640 1 --2- 172.21.15.12:0/4054622757 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdd18108620 0x7fdd1819b790 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fdd0400e9f0 tx=0x7fdd0400eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:07:08.252 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:08.249+0000 7fdd0f400640 1 -- 172.21.15.12:0/4054622757 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd0400cd80 con 0x7fdd18108620 2024-09-11T00:07:08.253 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:08.249+0000 7fdd1ca00640 1 -- 172.21.15.12:0/4054622757 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdd181a15f0 con 0x7fdd18108620 2024-09-11T00:07:08.253 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:08.250+0000 7fdd0f400640 1 -- 172.21.15.12:0/4054622757 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdd040108e0 con 0x7fdd18108620 2024-09-11T00:07:08.253 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:08.250+0000 7fdd1ca00640 1 -- 172.21.15.12:0/4054622757 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdd181a7e80 con 0x7fdd18108620 2024-09-11T00:07:08.253 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:08.250+0000 7fdd0f400640 1 -- 172.21.15.12:0/4054622757 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd04018980 con 0x7fdd18108620 2024-09-11T00:07:08.254 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:08.253+0000 7fdd20038740 1 Processor -- start 2024-09-11T00:07:08.255 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:08.253+0000 7fdd20038740 1 -- start start 2024-09-11T00:07:08.255 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:08.253+0000 7fdd20038740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ea42972400 0x55ea429727e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:07:08.255 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:08.253+0000 7fdd20038740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ea42972d20 0x55ea42986f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:07:08.256 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:08.253+0000 7fdd20038740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ea429874c0 0x55ea429898b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:07:08.256 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:08.253+0000 7fdd20038740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55ea428e9d50 con 0x55ea42972400 2024-09-11T00:07:08.256 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:08.253+0000 7fdd20038740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55ea428d9180 con 0x55ea42972d20 2024-09-11T00:07:08.257 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:08.253+0000 7fdd20038740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55ea428dcf90 con 0x55ea429874c0 2024-09-11T00:07:08.257 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:08.253+0000 7fdd0f400640 1 -- 172.21.15.12:0/4054622757 <== mon.1 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7fdd04010430 con 0x7fdd18108620 2024-09-11T00:07:08.257 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:08.254+0000 7fdd17200640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ea42972400 0x55ea429727e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:07:08.257 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:08.254+0000 7fdd17200640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ea42972400 0x55ea429727e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:47096/0 (socket says 172.21.15.12:47096) 2024-09-11T00:07:08.258 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:08.254+0000 7fdd17200640 1 -- 172.21.15.12:0/1278269100 learned_addr learned my addr 172.21.15.12:0/1278269100 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:07:08.258 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:08.254+0000 7fdd16800640 1 --2- 172.21.15.12:0/1278269100 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ea42972d20 0x55ea42986f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:07:08.258 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:08.254+0000 7fdd15e00640 1 --2- 172.21.15.12:0/1278269100 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ea429874c0 0x55ea429898b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:07:08.258 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:08.254+0000 7fdd17200640 1 -- 172.21.15.12:0/1278269100 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ea429874c0 msgr2=0x55ea429898b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:07:08.258 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:08.254+0000 7fdd17200640 1 --2- 172.21.15.12:0/1278269100 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ea429874c0 0x55ea429898b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:07:08.258 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:08.254+0000 7fdd17200640 1 -- 172.21.15.12:0/1278269100 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ea42972d20 msgr2=0x55ea42986f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:07:08.258 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:08.254+0000 7fdd17200640 1 --2- 172.21.15.12:0/1278269100 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ea42972d20 0x55ea42986f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:07:08.258 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:08.254+0000 7fdd17200640 1 -- 172.21.15.12:0/1278269100 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ea428a3a70 con 0x55ea42972400 2024-09-11T00:07:08.258 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:08.254+0000 7fdd15e00640 1 --2- 172.21.15.12:0/1278269100 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ea429874c0 0x55ea429898b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T00:07:08.258 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:08.254+0000 7fdd16800640 1 --2- 172.21.15.12:0/1278269100 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ea42972d20 0x55ea42986f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:07:08.258 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:08.255+0000 7fdd17200640 1 --2- 172.21.15.12:0/1278269100 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ea42972400 0x55ea429727e0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fdd10009ca0 tx=0x7fdd100093e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9f21492408407d6 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:07:08.258 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:08.255+0000 7fdd0f400640 1 --2- 172.21.15.12:0/4054622757 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fdcfc0b5410 0x7fdcfc0b78d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:07:08.258 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:08.255+0000 7fdd03400640 1 -- 172.21.15.12:0/1278269100 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd1005a070 con 0x55ea42972400 2024-09-11T00:07:08.259 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:08.255+0000 7fdd03400640 1 -- 172.21.15.12:0/1278269100 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdd10036070 con 0x55ea42972400 2024-09-11T00:07:08.259 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:08.255+0000 7fdd03400640 1 -- 172.21.15.12:0/1278269100 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd10004040 con 0x55ea42972400 2024-09-11T00:07:08.260 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:08.256+0000 7fdd0f400640 1 -- 172.21.15.12:0/4054622757 <== mon.1 v2:172.21.15.178:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9208+0+0 (secure 0 0 0) 0x7fdd040e6060 con 0x7fdd18108620 2024-09-11T00:07:08.260 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:08.256+0000 7fdd16800640 1 --2- 172.21.15.12:0/4054622757 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fdcfc0b5410 0x7fdcfc0b78d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:07:08.260 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:08.256+0000 7fdd20038740 1 -- 172.21.15.12:0/1278269100 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ea42972400 msgr2=0x55ea429727e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:07:08.260 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:08.256+0000 7fdd20038740 1 --2- 172.21.15.12:0/1278269100 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ea42972400 0x55ea429727e0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fdd10009ca0 tx=0x7fdd100093e0 comp rx=0 tx=0).stop 2024-09-11T00:07:08.260 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:08.257+0000 7fdd16800640 1 --2- 172.21.15.12:0/4054622757 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fdcfc0b5410 0x7fdcfc0b78d0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x55ea42971110 tx=0x7fdd08005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:07:08.260 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:08.257+0000 7fdd20038740 1 -- 172.21.15.12:0/1278269100 shutdown_connections 2024-09-11T00:07:08.260 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:08.257+0000 7fdd20038740 1 --2- 172.21.15.12:0/1278269100 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ea429874c0 0x55ea429898b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:07:08.260 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:08.257+0000 7fdd20038740 1 --2- 172.21.15.12:0/1278269100 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ea42972d20 0x55ea42986f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:07:08.260 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:08.257+0000 7fdd20038740 1 --2- 172.21.15.12:0/1278269100 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ea42972400 0x55ea429727e0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:07:08.260 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:08.257+0000 7fdd20038740 1 -- 172.21.15.12:0/1278269100 >> 172.21.15.12:0/1278269100 conn(0x55ea42970230 msgr2=0x55ea429713d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:07:08.261 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:08.257+0000 7fdd20038740 1 -- 172.21.15.12:0/1278269100 shutdown_connections 2024-09-11T00:07:08.261 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:08.257+0000 7fdd20038740 1 -- 172.21.15.12:0/1278269100 wait complete. 2024-09-11T00:07:08.261 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:08.258+0000 7fdd20038740 1 -- start start 2024-09-11T00:07:08.261 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:08.258+0000 7fdd20038740 1 Processor -- start 2024-09-11T00:07:08.262 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:08.258+0000 7fdd20038740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ea42972400 0x55ea4297c460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:07:08.262 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:08.258+0000 7fdd20038740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ea42972d20 0x55ea42986cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:07:08.263 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:08.258+0000 7fdd20038740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ea429874c0 0x55ea4297c9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:07:08.263 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:08.258+0000 7fdd20038740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55ea4273a5f0 con 0x55ea42972d20 2024-09-11T00:07:08.263 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:08.258+0000 7fdd20038740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55ea42874460 con 0x55ea429874c0 2024-09-11T00:07:08.263 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:08.258+0000 7fdd20038740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55ea429789b0 con 0x55ea42972400 2024-09-11T00:07:08.263 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:08.258+0000 7fdd20038740 10 client.0 _ll_register_callbacks cb 0x55ea42884210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:07:08.263 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:08.259+0000 7fdd15e00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ea42972d20 0x55ea42986cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:07:08.263 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:08.259+0000 7fdd15e00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ea42972d20 0x55ea42986cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:47108/0 (socket says 172.21.15.12:47108) 2024-09-11T00:07:08.263 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:08.259+0000 7fdd15e00640 1 -- 172.21.15.12:0/1291500812 learned_addr learned my addr 172.21.15.12:0/1291500812 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:07:08.263 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:08.259+0000 7fdd17200640 1 --2- 172.21.15.12:0/1291500812 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ea429874c0 0x55ea4297c9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:07:08.263 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:08.259+0000 7fdd16800640 1 --2- 172.21.15.12:0/1291500812 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ea42972400 0x55ea4297c460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:07:08.263 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:08.259+0000 7fdd17200640 1 -- 172.21.15.12:0/1291500812 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ea42972400 msgr2=0x55ea4297c460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:07:08.264 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:08.259+0000 7fdd17200640 1 --2- 172.21.15.12:0/1291500812 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ea42972400 0x55ea4297c460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:07:08.264 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:08.259+0000 7fdd17200640 1 -- 172.21.15.12:0/1291500812 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ea42972d20 msgr2=0x55ea42986cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:07:08.264 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:08.259+0000 7fdd17200640 1 --2- 172.21.15.12:0/1291500812 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ea42972d20 0x55ea42986cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:07:08.264 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:08.259+0000 7fdd17200640 1 -- 172.21.15.12:0/1291500812 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ea42637540 con 0x55ea429874c0 2024-09-11T00:07:08.264 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:08.259+0000 7fdd16800640 1 --2- 172.21.15.12:0/1291500812 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ea42972400 0x55ea4297c460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:07:08.264 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:08.260+0000 7fdd17200640 1 --2- 172.21.15.12:0/1291500812 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ea429874c0 0x55ea4297c9e0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fdd1002aeb0 tx=0x7fdd10031ea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8f14bd8078f9e6f4 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:07:08.264 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:08.260+0000 7fdcf7e00640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-09-11T00:07:08.264 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:08.260+0000 7fdcf7e00640 1 -- 172.21.15.12:0/1291500812 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd1005a050 con 0x55ea429874c0 2024-09-11T00:07:08.265 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:08.260+0000 7fdd20038740 1 -- 172.21.15.12:0/1291500812 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55ea42876a70 con 0x55ea429874c0 2024-09-11T00:07:08.265 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:08.260+0000 7fdcf7e00640 1 -- 172.21.15.12:0/1291500812 <== mon.2 v2:172.21.15.165:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 9208+0+0 (secure 0 0 0) 0x7fdd10031650 con 0x55ea429874c0 2024-09-11T00:07:08.265 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:08.260+0000 7fdcf7e00640 1 -- 172.21.15.12:0/1291500812 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fdd1006f050 con 0x55ea429874c0 2024-09-11T00:07:08.265 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:08.260+0000 7fdd20038740 10 client.25484 fetch_fsmap learned FSMap version 29 2024-09-11T00:07:08.265 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:08.260+0000 7fdd20038740 1 -- 172.21.15.12:0/1291500812 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55ea428916f0 con 0x55ea429874c0 2024-09-11T00:07:08.266 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:08.261+0000 7fdcf7e00640 1 -- 172.21.15.12:0/1291500812 <== mon.2 v2:172.21.15.165:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55ea42876a70 con 0x55ea429874c0 2024-09-11T00:07:08.266 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:08.261+0000 7fdcf7e00640 1 -- 172.21.15.12:0/1291500812 <== mon.2 v2:172.21.15.165:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7fdd1002f610 con 0x55ea429874c0 2024-09-11T00:07:08.267 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:08.261+0000 7fdd20038740 10 client.25484 fetch_fsmap finished waiting for FSMap version 29 2024-09-11T00:07:08.267 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:08.261+0000 7fdd20038740 10 client.25484 resolve_mds: resolved name 'g' to daemon mds.g 2024-09-11T00:07:08.267 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:08.261+0000 7fdd20038740 20 client.25484 populate_metadata read hostname 'smithi012' 2024-09-11T00:07:08.267 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:08.261+0000 7fdd20038740 1 --2- 172.21.15.12:0/1291500812 >> [v2:172.21.15.12:6838/3124093009,v1:172.21.15.12:6839/3124093009] conn(0x55ea4298e7a0 0x55ea429909a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:07:08.267 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:08.261+0000 7fdd20038740 4 client.25484 mds_command: new command op to 24643 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T00:07:08.267 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:08.261+0000 7fdd20038740 1 -- 172.21.15.12:0/1291500812 --> [v2:172.21.15.12:6838/3124093009,v1:172.21.15.12:6839/3124093009] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ea428ebca0 con 0x55ea4298e7a0 2024-09-11T00:07:08.267 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:08.262+0000 7fdd16800640 1 --2- 172.21.15.12:0/1291500812 >> [v2:172.21.15.12:6838/3124093009,v1:172.21.15.12:6839/3124093009] conn(0x55ea4298e7a0 0x55ea429909a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:07:08.267 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:08.262+0000 7fdd16800640 1 --2- 172.21.15.12:0/1291500812 >> [v2:172.21.15.12:6838/3124093009,v1:172.21.15.12:6839/3124093009] conn(0x55ea4298e7a0 0x55ea429909a0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=b989eeb3b707e78a server_cookie=5ba7fe10759346e2 in_seq=0 out_seq=0 2024-09-11T00:07:08.267 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:08.262+0000 7fdcf7e00640 10 client.25484 ms_handle_connect on v2:172.21.15.12:6838/3124093009 2024-09-11T00:07:08.267 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:08.263+0000 7fdcf7e00640 1 -- 172.21.15.12:0/1291500812 <== mds.1 v2:172.21.15.12:6838/3124093009 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55ea428ebca0 con 0x55ea4298e7a0 2024-09-11T00:07:08.267 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:08.263+0000 7fdcf7e00640 10 client.25484 handle_command_reply: tid=0 2024-09-11T00:07:08.268 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:08.263+0000 7fdd20038740 1 client.25484 shutdown 2024-09-11T00:07:08.268 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:08.264+0000 7fdd20038740 1 -- 172.21.15.12:0/1291500812 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ea429874c0 msgr2=0x55ea4297c9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:07:08.268 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:08.264+0000 7fdd20038740 1 --2- 172.21.15.12:0/1291500812 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ea429874c0 0x55ea4297c9e0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fdd1002aeb0 tx=0x7fdd10031ea0 comp rx=0 tx=0).stop 2024-09-11T00:07:08.268 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:08.264+0000 7fdd20038740 1 -- 172.21.15.12:0/1291500812 shutdown_connections 2024-09-11T00:07:08.268 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:08.264+0000 7fdd20038740 1 --2- 172.21.15.12:0/1291500812 >> [v2:172.21.15.12:6838/3124093009,v1:172.21.15.12:6839/3124093009] conn(0x55ea4298e7a0 0x55ea429909a0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:07:08.268 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:08.264+0000 7fdd20038740 1 --2- 172.21.15.12:0/1291500812 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ea429874c0 0x55ea4297c9e0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:07:08.268 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:08.264+0000 7fdd20038740 1 --2- 172.21.15.12:0/1291500812 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ea42972d20 0x55ea42986cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:07:08.268 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:08.264+0000 7fdd20038740 1 --2- 172.21.15.12:0/1291500812 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ea42972400 0x55ea4297c460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:07:08.268 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:08.264+0000 7fdd20038740 1 -- 172.21.15.12:0/1291500812 >> 172.21.15.12:0/1291500812 conn(0x55ea42970230 msgr2=0x55ea42975da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:07:08.268 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:08.264+0000 7fdcf7e00640 0 client.25484 ms_handle_reset on v2:172.21.15.12:6838/3124093009 2024-09-11T00:07:08.268 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:08.264+0000 7fdd20038740 1 -- 172.21.15.12:0/1291500812 shutdown_connections 2024-09-11T00:07:08.268 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:08.264+0000 7fdd20038740 1 -- 172.21.15.12:0/1291500812 wait complete. 2024-09-11T00:07:08.269 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:08.264+0000 7fdd20038740 20 client.25484 trim_cache size 0 max 16384 2024-09-11T00:07:08.286 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:08.284+0000 7fdd20038740 1 Processor -- start 2024-09-11T00:07:08.286 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:08.284+0000 7fdd20038740 1 -- start start 2024-09-11T00:07:08.286 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:08.284+0000 7fdd20038740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ea42972400 0x55ea4296b250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:07:08.286 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:08.284+0000 7fdd20038740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ea42972d20 0x55ea4296fb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:07:08.287 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:08.284+0000 7fdd20038740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ea429874c0 0x55ea42979ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:07:08.287 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:08.284+0000 7fdd20038740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55ea4298ece0 con 0x55ea42972d20 2024-09-11T00:07:08.287 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:08.284+0000 7fdd20038740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55ea4298f3b0 con 0x55ea429874c0 2024-09-11T00:07:08.287 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:08.284+0000 7fdd20038740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55ea4298fa80 con 0x55ea42972400 2024-09-11T00:07:08.287 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:08.284+0000 7fdd16800640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ea42972400 0x55ea4296b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:07:08.287 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:08.284+0000 7fdd15e00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ea42972d20 0x55ea4296fb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:07:08.288 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:08.284+0000 7fdd16800640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ea42972400 0x55ea4296b250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.12:56100/0 (socket says 172.21.15.12:56100) 2024-09-11T00:07:08.288 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:08.284+0000 7fdd16800640 1 -- 172.21.15.12:0/3983042056 learned_addr learned my addr 172.21.15.12:0/3983042056 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:07:08.288 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:08.285+0000 7fdd17200640 1 --2- 172.21.15.12:0/3983042056 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ea429874c0 0x55ea42979ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:07:08.288 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:08.285+0000 7fdd16800640 1 -- 172.21.15.12:0/3983042056 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ea429874c0 msgr2=0x55ea42979ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:07:08.288 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:08.285+0000 7fdd16800640 1 --2- 172.21.15.12:0/3983042056 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ea429874c0 0x55ea42979ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:07:08.288 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:08.285+0000 7fdd16800640 1 -- 172.21.15.12:0/3983042056 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ea42972d20 msgr2=0x55ea4296fb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:07:08.288 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:08.285+0000 7fdd16800640 1 --2- 172.21.15.12:0/3983042056 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ea42972d20 0x55ea4296fb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:07:08.288 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:08.285+0000 7fdd16800640 1 -- 172.21.15.12:0/3983042056 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ea4297a1e0 con 0x55ea42972400 2024-09-11T00:07:08.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:08.285+0000 7fdd17200640 1 --2- 172.21.15.12:0/3983042056 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ea429874c0 0x55ea42979ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T00:07:08.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:08.285+0000 7fdd16800640 1 --2- 172.21.15.12:0/3983042056 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ea42972400 0x55ea4296b250 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fdd080045b0 tx=0x7fdd0801b000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9fb4b8ffa4ed384 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:07:08.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:08.286+0000 7fdd02a00640 1 -- 172.21.15.12:0/3983042056 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd08003b70 con 0x55ea42972400 2024-09-11T00:07:08.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:08.286+0000 7fdd02a00640 1 -- 172.21.15.12:0/3983042056 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdd08003d10 con 0x55ea42972400 2024-09-11T00:07:08.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:08.286+0000 7fdd02a00640 1 -- 172.21.15.12:0/3983042056 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd0801f740 con 0x55ea42972400 2024-09-11T00:07:08.291 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:08.286+0000 7fdd20038740 1 -- 172.21.15.12:0/3983042056 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ea42972400 msgr2=0x55ea4296b250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:07:08.291 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:08.287+0000 7fdd20038740 1 --2- 172.21.15.12:0/3983042056 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ea42972400 0x55ea4296b250 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fdd080045b0 tx=0x7fdd0801b000 comp rx=0 tx=0).stop 2024-09-11T00:07:08.291 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:08.287+0000 7fdd20038740 1 -- 172.21.15.12:0/3983042056 shutdown_connections 2024-09-11T00:07:08.291 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:08.287+0000 7fdd20038740 1 --2- 172.21.15.12:0/3983042056 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ea429874c0 0x55ea42979ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:07:08.291 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:08.287+0000 7fdd20038740 1 --2- 172.21.15.12:0/3983042056 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ea42972d20 0x55ea4296fb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:07:08.292 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:08.287+0000 7fdd20038740 1 --2- 172.21.15.12:0/3983042056 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ea42972400 0x55ea4296b250 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:07:08.292 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:08.287+0000 7fdd20038740 1 -- 172.21.15.12:0/3983042056 >> 172.21.15.12:0/3983042056 conn(0x55ea42970230 msgr2=0x55ea42971280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:07:08.292 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:08.287+0000 7fdd20038740 1 -- 172.21.15.12:0/3983042056 shutdown_connections 2024-09-11T00:07:08.292 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:08.287+0000 7fdd20038740 1 -- 172.21.15.12:0/3983042056 wait complete. 2024-09-11T00:07:08.292 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:08.287+0000 7fdd20038740 1 -- start start 2024-09-11T00:07:08.292 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:08.288+0000 7fdd20038740 1 Processor -- start 2024-09-11T00:07:08.292 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:08.288+0000 7fdd20038740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ea42972400 0x55ea42974800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:07:08.292 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:08.288+0000 7fdd20038740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ea42972d20 0x55ea42998bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:07:08.292 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:08.288+0000 7fdd20038740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ea429874c0 0x55ea42999130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:07:08.292 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:08.288+0000 7fdd20038740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55ea42990820 con 0x55ea42972d20 2024-09-11T00:07:08.292 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:08.288+0000 7fdd20038740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55ea42990150 con 0x55ea42972400 2024-09-11T00:07:08.292 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:08.288+0000 7fdd20038740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55ea42990ef0 con 0x55ea429874c0 2024-09-11T00:07:08.293 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:08.288+0000 7fdd20038740 10 client.0 _ll_register_callbacks cb 0x7fdd10009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:07:08.293 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:08.289+0000 7fdd15e00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ea42972d20 0x55ea42998bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:07:08.293 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:08.289+0000 7fdd16800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ea42972400 0x55ea42974800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:07:08.293 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:08.289+0000 7fdd16800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ea42972400 0x55ea42974800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.12:49154/0 (socket says 172.21.15.12:49154) 2024-09-11T00:07:08.293 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:08.289+0000 7fdd15e00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ea42972d20 0x55ea42998bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:47124/0 (socket says 172.21.15.12:47124) 2024-09-11T00:07:08.293 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:08.289+0000 7fdd16800640 1 -- 172.21.15.12:0/1136098177 learned_addr learned my addr 172.21.15.12:0/1136098177 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:07:08.293 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:08.289+0000 7fdd17200640 1 --2- 172.21.15.12:0/1136098177 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ea429874c0 0x55ea42999130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:07:08.293 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:08.289+0000 7fdd16800640 1 -- 172.21.15.12:0/1136098177 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ea429874c0 msgr2=0x55ea42999130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:07:08.293 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:08.289+0000 7fdd16800640 1 --2- 172.21.15.12:0/1136098177 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ea429874c0 0x55ea42999130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:07:08.293 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:08.289+0000 7fdd16800640 1 -- 172.21.15.12:0/1136098177 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ea42972d20 msgr2=0x55ea42998bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:07:08.293 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:08.289+0000 7fdd16800640 1 --2- 172.21.15.12:0/1136098177 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ea42972d20 0x55ea42998bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:07:08.293 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:08.289+0000 7fdd16800640 1 -- 172.21.15.12:0/1136098177 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ea429996a0 con 0x55ea42972400 2024-09-11T00:07:08.293 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:08.289+0000 7fdd16800640 1 --2- 172.21.15.12:0/1136098177 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ea42972400 0x55ea42974800 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fdd08005890 tx=0x7fdd08003f50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=603e973fd97ee1d2 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:07:08.294 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:08.289+0000 7fdcf7e00640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-09-11T00:07:08.295 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:08.290+0000 7fdcf7e00640 1 -- 172.21.15.12:0/1136098177 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd0803f060 con 0x55ea42972400 2024-09-11T00:07:08.295 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:08.290+0000 7fdcf7e00640 1 -- 172.21.15.12:0/1136098177 <== mon.2 v2:172.21.15.165:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 9208+0+0 (secure 0 0 0) 0x7fdd08005040 con 0x55ea42972400 2024-09-11T00:07:08.295 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:08.290+0000 7fdd20038740 1 -- 172.21.15.12:0/1136098177 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdd08003d10 con 0x55ea42972400 2024-09-11T00:07:08.295 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:08.290+0000 7fdcf7e00640 1 -- 172.21.15.12:0/1136098177 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fdd08046050 con 0x55ea42972400 2024-09-11T00:07:08.296 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:08.290+0000 7fdd20038740 10 client.25490 fetch_fsmap learned FSMap version 29 2024-09-11T00:07:08.296 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:08.290+0000 7fdd20038740 1 -- 172.21.15.12:0/1136098177 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55ea4296f9c0 con 0x55ea42972400 2024-09-11T00:07:08.296 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:08.290+0000 7fdcf7e00640 1 -- 172.21.15.12:0/1136098177 <== mon.2 v2:172.21.15.165:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdd08003d10 con 0x55ea42972400 2024-09-11T00:07:08.296 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:08.291+0000 7fdcf7e00640 1 -- 172.21.15.12:0/1136098177 <== mon.2 v2:172.21.15.165:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7fdd08030030 con 0x55ea42972400 2024-09-11T00:07:08.296 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:08.291+0000 7fdd20038740 10 client.25490 fetch_fsmap finished waiting for FSMap version 29 2024-09-11T00:07:08.296 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:08.291+0000 7fdd20038740 10 client.25490 resolve_mds: resolved name 'g' to daemon mds.g 2024-09-11T00:07:08.296 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:08.291+0000 7fdd20038740 20 client.25490 populate_metadata read hostname 'smithi012' 2024-09-11T00:07:08.296 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:08.291+0000 7fdd20038740 1 --2- 172.21.15.12:0/1136098177 >> [v2:172.21.15.12:6838/3124093009,v1:172.21.15.12:6839/3124093009] conn(0x55ea42a00450 0x55ea429ff160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:07:08.296 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:08.291+0000 7fdd20038740 4 client.25490 mds_command: new command op to 24643 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-11T00:07:08.296 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:08.291+0000 7fdd20038740 1 -- 172.21.15.12:0/1136098177 --> [v2:172.21.15.12:6838/3124093009,v1:172.21.15.12:6839/3124093009] -- command(tid 0: {"prefix": "status"}) -- 0x55ea428e7750 con 0x55ea42a00450 2024-09-11T00:07:08.296 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:08.291+0000 7fdd15e00640 1 --2- 172.21.15.12:0/1136098177 >> [v2:172.21.15.12:6838/3124093009,v1:172.21.15.12:6839/3124093009] conn(0x55ea42a00450 0x55ea429ff160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:07:08.296 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:08.292+0000 7fdd15e00640 1 --2- 172.21.15.12:0/1136098177 >> [v2:172.21.15.12:6838/3124093009,v1:172.21.15.12:6839/3124093009] conn(0x55ea42a00450 0x55ea429ff160 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=98f78d67e4ed8083 server_cookie=e495d0283cd6602b in_seq=0 out_seq=0 2024-09-11T00:07:08.296 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:08.292+0000 7fdcf7e00640 10 client.25490 ms_handle_connect on v2:172.21.15.12:6838/3124093009 2024-09-11T00:07:08.297 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:08.292+0000 7fdcf7e00640 1 -- 172.21.15.12:0/1136098177 <== mds.1 v2:172.21.15.12:6838/3124093009 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55ea428e7750 con 0x55ea42a00450 2024-09-11T00:07:08.297 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:08.292+0000 7fdcf7e00640 10 client.25490 handle_command_reply: tid=0 2024-09-11T00:07:08.297 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:08.292+0000 7fdd20038740 1 client.25490 shutdown 2024-09-11T00:07:08.297 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:08.292+0000 7fdd20038740 1 -- 172.21.15.12:0/1136098177 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ea42972400 msgr2=0x55ea42974800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:07:08.297 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:08.292+0000 7fdd20038740 1 --2- 172.21.15.12:0/1136098177 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ea42972400 0x55ea42974800 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fdd08005890 tx=0x7fdd08003f50 comp rx=0 tx=0).stop 2024-09-11T00:07:08.297 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:08.292+0000 7fdd20038740 1 -- 172.21.15.12:0/1136098177 shutdown_connections 2024-09-11T00:07:08.297 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:08.292+0000 7fdd20038740 1 --2- 172.21.15.12:0/1136098177 >> [v2:172.21.15.12:6838/3124093009,v1:172.21.15.12:6839/3124093009] conn(0x55ea42a00450 0x55ea429ff160 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:07:08.297 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:08.292+0000 7fdd20038740 1 --2- 172.21.15.12:0/1136098177 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ea429874c0 0x55ea42999130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:07:08.297 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:08.292+0000 7fdd20038740 1 --2- 172.21.15.12:0/1136098177 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ea42972d20 0x55ea42998bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:07:08.297 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:08.292+0000 7fdd20038740 1 --2- 172.21.15.12:0/1136098177 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ea42972400 0x55ea42974800 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:07:08.297 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:08.292+0000 7fdd20038740 1 -- 172.21.15.12:0/1136098177 >> 172.21.15.12:0/1136098177 conn(0x55ea42970230 msgr2=0x55ea429878a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:07:08.297 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:08.292+0000 7fdcf7e00640 0 client.25490 ms_handle_reset on v2:172.21.15.12:6838/3124093009 2024-09-11T00:07:08.297 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:08.293+0000 7fdd20038740 1 -- 172.21.15.12:0/1136098177 shutdown_connections 2024-09-11T00:07:08.298 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:08.293+0000 7fdd20038740 1 -- 172.21.15.12:0/1136098177 wait complete. 2024-09-11T00:07:08.298 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:08.293+0000 7fdd20038740 20 client.25490 trim_cache size 0 max 16384 2024-09-11T00:07:08.298 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-09-11T00:07:08.298 INFO:teuthology.orchestra.run.smithi012.stdout: "cluster_fsid": "ff3ad74a-6fcf-11ef-bcea-c7b262605968", 2024-09-11T00:07:08.298 INFO:teuthology.orchestra.run.smithi012.stdout: "whoami": 1, 2024-09-11T00:07:08.298 INFO:teuthology.orchestra.run.smithi012.stdout: "id": 24643, 2024-09-11T00:07:08.298 INFO:teuthology.orchestra.run.smithi012.stdout: "want_state": "up:active", 2024-09-11T00:07:08.298 INFO:teuthology.orchestra.run.smithi012.stdout: "state": "up:active", 2024-09-11T00:07:08.298 INFO:teuthology.orchestra.run.smithi012.stdout: "fs_name": "cephfs", 2024-09-11T00:07:08.298 INFO:teuthology.orchestra.run.smithi012.stdout: "rank_uptime": 142.071873428, 2024-09-11T00:07:08.299 INFO:teuthology.orchestra.run.smithi012.stdout: "mdsmap_epoch": 29, 2024-09-11T00:07:08.299 INFO:teuthology.orchestra.run.smithi012.stdout: "osdmap_epoch": 85, 2024-09-11T00:07:08.299 INFO:teuthology.orchestra.run.smithi012.stdout: "osdmap_epoch_barrier": 85, 2024-09-11T00:07:08.299 INFO:teuthology.orchestra.run.smithi012.stdout: "uptime": 202.72598758999999 2024-09-11T00:07:08.299 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-09-11T00:07:08.299 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:08.293+0000 7fdd1ca00640 1 -- 172.21.15.12:0/4054622757 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fdcfc0b5410 msgr2=0x7fdcfc0b78d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:07:08.299 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:08.293+0000 7fdd1ca00640 1 --2- 172.21.15.12:0/4054622757 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fdcfc0b5410 0x7fdcfc0b78d0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x55ea42971110 tx=0x7fdd08005f90 comp rx=0 tx=0).stop 2024-09-11T00:07:08.300 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:08.293+0000 7fdd1ca00640 1 -- 172.21.15.12:0/4054622757 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdd18108620 msgr2=0x7fdd1819b790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:07:08.300 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:08.293+0000 7fdd1ca00640 1 --2- 172.21.15.12:0/4054622757 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdd18108620 0x7fdd1819b790 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fdd0400e9f0 tx=0x7fdd0400eeb0 comp rx=0 tx=0).stop 2024-09-11T00:07:08.300 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:08.294+0000 7fdd1ca00640 1 -- 172.21.15.12:0/4054622757 shutdown_connections 2024-09-11T00:07:08.300 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:08.294+0000 7fdd1ca00640 1 --2- 172.21.15.12:0/4054622757 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fdcfc0b5410 0x7fdcfc0b78d0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:07:08.300 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:08.294+0000 7fdd1ca00640 1 --2- 172.21.15.12:0/4054622757 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fdd18108fa0 0x7fdd181a0b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:07:08.301 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:08.294+0000 7fdd1ca00640 1 --2- 172.21.15.12:0/4054622757 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdd18108620 0x7fdd1819b790 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:07:08.301 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:08.294+0000 7fdd1ca00640 1 --2- 172.21.15.12:0/4054622757 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdd18102c10 0x7fdd1819b250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:07:08.301 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:08.294+0000 7fdd1ca00640 1 -- 172.21.15.12:0/4054622757 >> 172.21.15.12:0/4054622757 conn(0x7fdd180fe660 msgr2=0x7fdd18104bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:07:08.301 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:08.294+0000 7fdd1ca00640 1 -- 172.21.15.12:0/4054622757 shutdown_connections 2024-09-11T00:07:08.301 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:08.294+0000 7fdd1ca00640 1 -- 172.21.15.12:0/4054622757 wait complete. 2024-09-11T00:07:09.229 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid ff3ad74a-6fcf-11ef-bcea-c7b262605968 -- ceph tell mds.b status 2024-09-11T00:07:09.562 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:07:09 smithi012 ceph-mon[30333]: pgmap v400: 129 pgs: 129 active+clean; 663 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 2.7 KiB/s wr, 13 op/s 2024-09-11T00:07:09.563 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:07:09 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T00:07:09.666 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:07:09.782 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:07:09 smithi165 ceph-mon[35356]: pgmap v400: 129 pgs: 129 active+clean; 663 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 2.7 KiB/s wr, 13 op/s 2024-09-11T00:07:09.782 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:07:09 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T00:07:09.831 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:07:09 smithi178 ceph-mon[35919]: pgmap v400: 129 pgs: 129 active+clean; 663 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 2.7 KiB/s wr, 13 op/s 2024-09-11T00:07:09.831 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:07:09 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T00:07:10.252 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:10.246+0000 7f6d11800640 1 -- 172.21.15.12:0/90115155 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6d0c1045e0 msgr2=0x7f6d0c10cad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:07:10.252 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:10.246+0000 7f6d11800640 1 --2- 172.21.15.12:0/90115155 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6d0c1045e0 0x7f6d0c10cad0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f6cf8009900 tx=0x7f6cf802a410 comp rx=0 tx=0).stop 2024-09-11T00:07:10.253 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:10.247+0000 7f6d11800640 1 -- 172.21.15.12:0/90115155 shutdown_connections 2024-09-11T00:07:10.253 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:10.247+0000 7f6d11800640 1 --2- 172.21.15.12:0/90115155 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6d0c10d120 0x7f6d0c10f580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:07:10.253 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:10.247+0000 7f6d11800640 1 --2- 172.21.15.12:0/90115155 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6d0c1045e0 0x7f6d0c10cad0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:07:10.253 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:10.247+0000 7f6d11800640 1 --2- 172.21.15.12:0/90115155 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6d0c103cc0 0x7f6d0c1040a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:07:10.253 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:10.247+0000 7f6d11800640 1 -- 172.21.15.12:0/90115155 >> 172.21.15.12:0/90115155 conn(0x7f6d0c0fd700 msgr2=0x7f6d0c0ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:07:10.253 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:10.247+0000 7f6d11800640 1 -- 172.21.15.12:0/90115155 shutdown_connections 2024-09-11T00:07:10.253 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:10.247+0000 7f6d11800640 1 -- 172.21.15.12:0/90115155 wait complete. 2024-09-11T00:07:10.253 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:10.248+0000 7f6d11800640 1 Processor -- start 2024-09-11T00:07:10.253 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:10.248+0000 7f6d11800640 1 -- start start 2024-09-11T00:07:10.253 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:10.248+0000 7f6d11800640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6d0c103cc0 0x7f6d0c19b4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:07:10.253 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:10.248+0000 7f6d11800640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6d0c1045e0 0x7f6d0c19ba10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:07:10.253 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:10.248+0000 7f6d11800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6d0c10d120 0x7f6d0c1a0db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:07:10.254 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:10.248+0000 7f6d11800640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f6d0c112d90 con 0x7f6d0c103cc0 2024-09-11T00:07:10.254 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:10.248+0000 7f6d11800640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f6d0c112c10 con 0x7f6d0c10d120 2024-09-11T00:07:10.254 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:10.248+0000 7f6d11800640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f6d0c112f10 con 0x7f6d0c1045e0 2024-09-11T00:07:10.254 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:10.249+0000 7f6d0aa00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6d0c103cc0 0x7f6d0c19b4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:07:10.254 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:10.249+0000 7f6d0a000640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6d0c1045e0 0x7f6d0c19ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:07:10.254 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:10.249+0000 7f6d0aa00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6d0c103cc0 0x7f6d0c19b4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:34952/0 (socket says 172.21.15.12:34952) 2024-09-11T00:07:10.254 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:10.249+0000 7f6d0a000640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6d0c1045e0 0x7f6d0c19ba10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.12:39584/0 (socket says 172.21.15.12:39584) 2024-09-11T00:07:10.254 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:10.249+0000 7f6d0a000640 1 -- 172.21.15.12:0/3533796436 learned_addr learned my addr 172.21.15.12:0/3533796436 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:07:10.254 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:10.249+0000 7f6d0b400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6d0c10d120 0x7f6d0c1a0db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:07:10.254 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:10.250+0000 7f6d0b400640 1 -- 172.21.15.12:0/3533796436 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6d0c1045e0 msgr2=0x7f6d0c19ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:07:10.254 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:10.250+0000 7f6d0b400640 1 --2- 172.21.15.12:0/3533796436 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6d0c1045e0 0x7f6d0c19ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:07:10.254 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:10.250+0000 7f6d0b400640 1 -- 172.21.15.12:0/3533796436 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6d0c103cc0 msgr2=0x7f6d0c19b4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:07:10.255 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:10.250+0000 7f6d0b400640 1 --2- 172.21.15.12:0/3533796436 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6d0c103cc0 0x7f6d0c19b4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:07:10.255 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:10.250+0000 7f6d0b400640 1 -- 172.21.15.12:0/3533796436 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d0c1a1580 con 0x7f6d0c10d120 2024-09-11T00:07:10.255 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:10.250+0000 7f6d0b400640 1 --2- 172.21.15.12:0/3533796436 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6d0c10d120 0x7f6d0c1a0db0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f6d04009fd0 tx=0x7f6d0400ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:07:10.255 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:10.250+0000 7f6d03400640 1 -- 172.21.15.12:0/3533796436 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d0400ce20 con 0x7f6d0c10d120 2024-09-11T00:07:10.255 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:10.250+0000 7f6d11800640 1 -- 172.21.15.12:0/3533796436 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6d0c1a1870 con 0x7f6d0c10d120 2024-09-11T00:07:10.255 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:10.250+0000 7f6d03400640 1 -- 172.21.15.12:0/3533796436 <== mon.2 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6d04005020 con 0x7f6d0c10d120 2024-09-11T00:07:10.255 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:10.250+0000 7f6d03400640 1 -- 172.21.15.12:0/3533796436 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d04010690 con 0x7f6d0c10d120 2024-09-11T00:07:10.255 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:10.251+0000 7f6d11800640 1 -- 172.21.15.12:0/3533796436 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6d0c1a7f30 con 0x7f6d0c10d120 2024-09-11T00:07:10.256 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:10.255+0000 7f6d1422f740 1 Processor -- start 2024-09-11T00:07:10.257 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:10.255+0000 7f6d03400640 1 -- 172.21.15.12:0/3533796436 <== mon.2 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7f6d040054d0 con 0x7f6d0c10d120 2024-09-11T00:07:10.258 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:10.256+0000 7f6d1422f740 1 -- start start 2024-09-11T00:07:10.258 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:10.256+0000 7f6d1422f740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5598bc2f9240 0x5598bc2f9620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:07:10.258 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:10.256+0000 7f6d1422f740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5598bc2f9b60 0x5598bc3055b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:07:10.258 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:10.256+0000 7f6d1422f740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5598bc305af0 0x5598bc307ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:07:10.258 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:10.256+0000 7f6d1422f740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x5598bc270d50 con 0x5598bc2f9b60 2024-09-11T00:07:10.259 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:10.256+0000 7f6d1422f740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x5598bc260180 con 0x5598bc2f9240 2024-09-11T00:07:10.259 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:10.256+0000 7f6d1422f740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5598bc263f90 con 0x5598bc305af0 2024-09-11T00:07:10.259 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:10.256+0000 7f6d0a000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5598bc2f9240 0x5598bc2f9620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:07:10.259 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:10.256+0000 7f6d0a000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5598bc2f9240 0x5598bc2f9620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.12:35848/0 (socket says 172.21.15.12:35848) 2024-09-11T00:07:10.259 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:10.256+0000 7f6d0a000640 1 -- 172.21.15.12:0/1862943419 learned_addr learned my addr 172.21.15.12:0/1862943419 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:07:10.259 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:10.257+0000 7f6d0aa00640 1 --2- 172.21.15.12:0/1862943419 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5598bc2f9b60 0x5598bc3055b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:07:10.260 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:10.257+0000 7f6d0a000640 1 --2- 172.21.15.12:0/1862943419 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5598bc305af0 0x5598bc307ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:07:10.260 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:10.257+0000 7f6d0aa00640 1 -- 172.21.15.12:0/1862943419 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5598bc305af0 msgr2=0x5598bc307ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:07:10.260 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:10.257+0000 7f6d0aa00640 1 --2- 172.21.15.12:0/1862943419 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5598bc305af0 0x5598bc307ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:07:10.260 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:10.257+0000 7f6d0aa00640 1 -- 172.21.15.12:0/1862943419 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5598bc2f9240 msgr2=0x5598bc2f9620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:07:10.260 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:10.257+0000 7f6d0aa00640 1 --2- 172.21.15.12:0/1862943419 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5598bc2f9240 0x5598bc2f9620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:07:10.260 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:10.257+0000 7f6d0aa00640 1 -- 172.21.15.12:0/1862943419 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5598bc22aa70 con 0x5598bc2f9b60 2024-09-11T00:07:10.260 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:10.258+0000 7f6d0aa00640 1 --2- 172.21.15.12:0/1862943419 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5598bc2f9b60 0x5598bc3055b0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f6cfc009c50 tx=0x7f6cfc011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=efd7a6e2aebfed20 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:07:10.261 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:10.258+0000 7f6d03400640 1 --2- 172.21.15.12:0/3533796436 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f6cf00b5360 0x7f6cf00b7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:07:10.261 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:10.258+0000 7f6cf7e00640 1 -- 172.21.15.12:0/1862943419 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cfc004050 con 0x5598bc2f9b60 2024-09-11T00:07:10.261 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:10.258+0000 7f6cf7e00640 1 -- 172.21.15.12:0/1862943419 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6cfc0054d0 con 0x5598bc2f9b60 2024-09-11T00:07:10.261 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:10.258+0000 7f6cf7e00640 1 -- 172.21.15.12:0/1862943419 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cfc004ca0 con 0x5598bc2f9b60 2024-09-11T00:07:10.261 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:10.258+0000 7f6d03400640 1 -- 172.21.15.12:0/3533796436 <== mon.2 v2:172.21.15.165:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9208+0+0 (secure 0 0 0) 0x7f6d040e66c0 con 0x7f6d0c10d120 2024-09-11T00:07:10.261 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:10.258+0000 7f6d0a000640 1 --2- 172.21.15.12:0/3533796436 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f6cf00b5360 0x7f6cf00b7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:07:10.261 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:10.259+0000 7f6d1422f740 1 -- 172.21.15.12:0/1862943419 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5598bc2f9b60 msgr2=0x5598bc3055b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:07:10.261 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:10.259+0000 7f6d1422f740 1 --2- 172.21.15.12:0/1862943419 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5598bc2f9b60 0x5598bc3055b0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f6cfc009c50 tx=0x7f6cfc011810 comp rx=0 tx=0).stop 2024-09-11T00:07:10.262 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:10.259+0000 7f6d1422f740 1 -- 172.21.15.12:0/1862943419 shutdown_connections 2024-09-11T00:07:10.262 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:10.259+0000 7f6d1422f740 1 --2- 172.21.15.12:0/1862943419 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5598bc305af0 0x5598bc307ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:07:10.262 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:10.259+0000 7f6d1422f740 1 --2- 172.21.15.12:0/1862943419 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5598bc2f9b60 0x5598bc3055b0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:07:10.262 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:10.259+0000 7f6d1422f740 1 --2- 172.21.15.12:0/1862943419 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5598bc2f9240 0x5598bc2f9620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:07:10.263 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:10.259+0000 7f6d1422f740 1 -- 172.21.15.12:0/1862943419 >> 172.21.15.12:0/1862943419 conn(0x5598bc2f7020 msgr2=0x5598bc2f8210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:07:10.263 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:10.259+0000 7f6d0a000640 1 --2- 172.21.15.12:0/3533796436 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f6cf00b5360 0x7f6cf00b7820 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f6cf8032fd0 tx=0x7f6cf8041040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:07:10.263 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:10.260+0000 7f6d1422f740 1 -- 172.21.15.12:0/1862943419 shutdown_connections 2024-09-11T00:07:10.263 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:10.260+0000 7f6d1422f740 1 -- 172.21.15.12:0/1862943419 wait complete. 2024-09-11T00:07:10.263 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:10.260+0000 7f6d1422f740 1 -- start start 2024-09-11T00:07:10.263 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:10.261+0000 7f6d1422f740 1 Processor -- start 2024-09-11T00:07:10.264 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:10.261+0000 7f6d1422f740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5598bc2f9240 0x5598bc3055b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:07:10.265 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:10.261+0000 7f6d1422f740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5598bc2f9b60 0x5598bc302a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:07:10.265 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:10.261+0000 7f6d1422f740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5598bc305af0 0x5598bc3031d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:07:10.265 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:10.261+0000 7f6d1422f740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x5598bc0c15f0 con 0x5598bc2f9240 2024-09-11T00:07:10.265 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:10.261+0000 7f6d1422f740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x5598bc1fb460 con 0x5598bc2f9b60 2024-09-11T00:07:10.265 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:10.261+0000 7f6d1422f740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5598bc308840 con 0x5598bc305af0 2024-09-11T00:07:10.265 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:10.261+0000 7f6d1422f740 10 client.0 _ll_register_callbacks cb 0x5598bc20b210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:07:10.265 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:10.262+0000 7f6d0aa00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5598bc2f9240 0x5598bc3055b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:07:10.265 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:10.262+0000 7f6d0a000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5598bc2f9b60 0x5598bc302a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:07:10.265 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:10.262+0000 7f6d0a000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5598bc2f9b60 0x5598bc302a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.12:35860/0 (socket says 172.21.15.12:35860) 2024-09-11T00:07:10.265 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:10.262+0000 7f6d0a000640 1 -- 172.21.15.12:0/1286788991 learned_addr learned my addr 172.21.15.12:0/1286788991 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:07:10.265 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:10.262+0000 7f6d0b400640 1 --2- 172.21.15.12:0/1286788991 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5598bc305af0 0x5598bc3031d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:07:10.265 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:10.262+0000 7f6d0b400640 1 -- 172.21.15.12:0/1286788991 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5598bc2f9b60 msgr2=0x5598bc302a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:07:10.265 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:10.262+0000 7f6d0b400640 1 --2- 172.21.15.12:0/1286788991 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5598bc2f9b60 0x5598bc302a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:07:10.266 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:10.262+0000 7f6d0b400640 1 -- 172.21.15.12:0/1286788991 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5598bc2f9240 msgr2=0x5598bc3055b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:07:10.266 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:10.262+0000 7f6d0b400640 1 --2- 172.21.15.12:0/1286788991 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5598bc2f9240 0x5598bc3055b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:07:10.266 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:10.262+0000 7f6d0b400640 1 -- 172.21.15.12:0/1286788991 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5598bbfbe540 con 0x5598bc305af0 2024-09-11T00:07:10.266 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:10.262+0000 7f6d0aa00640 1 --2- 172.21.15.12:0/1286788991 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5598bc2f9240 0x5598bc3055b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:07:10.266 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:10.263+0000 7f6d0b400640 1 --2- 172.21.15.12:0/1286788991 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5598bc305af0 0x5598bc3031d0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f6d040a9cc0 tx=0x7f6d040ac2f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6d9dba6de5ffe67b server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:07:10.266 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:10.263+0000 7f6cf4c00640 10 client.0 ms_handle_connect on v2:172.21.15.178:3300/0 2024-09-11T00:07:10.266 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:10.263+0000 7f6cf4c00640 1 -- 172.21.15.12:0/1286788991 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d040b12d0 con 0x5598bc305af0 2024-09-11T00:07:10.266 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:10.263+0000 7f6cf4c00640 1 -- 172.21.15.12:0/1286788991 <== mon.1 v2:172.21.15.178:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 9208+0+0 (secure 0 0 0) 0x7f6d040b5b50 con 0x5598bc305af0 2024-09-11T00:07:10.266 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:10.263+0000 7f6d1422f740 1 -- 172.21.15.12:0/1286788991 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5598bc1fda70 con 0x5598bc305af0 2024-09-11T00:07:10.268 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:10.263+0000 7f6cf4c00640 1 -- 172.21.15.12:0/1286788991 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f6d040c7050 con 0x5598bc305af0 2024-09-11T00:07:10.268 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:10.263+0000 7f6d1422f740 10 client.25483 fetch_fsmap learned FSMap version 29 2024-09-11T00:07:10.268 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:10.263+0000 7f6d1422f740 1 -- 172.21.15.12:0/1286788991 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5598bc2186f0 con 0x5598bc305af0 2024-09-11T00:07:10.268 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:10.263+0000 7f6cf4c00640 1 -- 172.21.15.12:0/1286788991 <== mon.1 v2:172.21.15.178:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x5598bc1fda70 con 0x5598bc305af0 2024-09-11T00:07:10.268 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:10.263+0000 7f6cf4c00640 1 -- 172.21.15.12:0/1286788991 <== mon.1 v2:172.21.15.178:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7f6d040a83a0 con 0x5598bc305af0 2024-09-11T00:07:10.268 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:10.264+0000 7f6d1422f740 10 client.25483 fetch_fsmap finished waiting for FSMap version 29 2024-09-11T00:07:10.268 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:10.264+0000 7f6d1422f740 10 client.25483 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-11T00:07:10.268 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:10.264+0000 7f6d1422f740 20 client.25483 populate_metadata read hostname 'smithi012' 2024-09-11T00:07:10.268 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:10.264+0000 7f6d1422f740 1 --2- 172.21.15.12:0/1286788991 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x5598bc3168a0 0x5598bc317d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:07:10.269 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:10.264+0000 7f6d1422f740 4 client.25483 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T00:07:10.269 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:10.264+0000 7f6d1422f740 1 -- 172.21.15.12:0/1286788991 --> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5598bc272ca0 con 0x5598bc3168a0 2024-09-11T00:07:10.269 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:10.264+0000 7f6d0aa00640 1 --2- 172.21.15.12:0/1286788991 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x5598bc3168a0 0x5598bc317d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:07:10.269 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:10.265+0000 7f6cf4c00640 10 client.25483 ms_handle_connect on v2:172.21.15.165:6838/501312984 2024-09-11T00:07:10.269 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:10.265+0000 7f6d0aa00640 1 --2- 172.21.15.12:0/1286788991 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x5598bc3168a0 0x5598bc317d10 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=28ff9d0ea742c0c server_cookie=747fd35396189061 in_seq=0 out_seq=0 2024-09-11T00:07:10.269 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:10.266+0000 7f6cf4c00640 1 -- 172.21.15.12:0/1286788991 <== mds.0 v2:172.21.15.165:6838/501312984 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5598bc272ca0 con 0x5598bc3168a0 2024-09-11T00:07:10.269 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:10.266+0000 7f6cf4c00640 10 client.25483 handle_command_reply: tid=0 2024-09-11T00:07:10.269 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:10.266+0000 7f6d1422f740 1 client.25483 shutdown 2024-09-11T00:07:10.270 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:10.266+0000 7f6d1422f740 1 -- 172.21.15.12:0/1286788991 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5598bc305af0 msgr2=0x5598bc3031d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:07:10.270 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:10.266+0000 7f6d1422f740 1 --2- 172.21.15.12:0/1286788991 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5598bc305af0 0x5598bc3031d0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f6d040a9cc0 tx=0x7f6d040ac2f0 comp rx=0 tx=0).stop 2024-09-11T00:07:10.270 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:10.266+0000 7f6d1422f740 1 -- 172.21.15.12:0/1286788991 shutdown_connections 2024-09-11T00:07:10.270 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:10.266+0000 7f6d1422f740 1 --2- 172.21.15.12:0/1286788991 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x5598bc3168a0 0x5598bc317d10 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:07:10.270 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:10.266+0000 7f6d1422f740 1 --2- 172.21.15.12:0/1286788991 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5598bc305af0 0x5598bc3031d0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:07:10.270 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:10.266+0000 7f6d1422f740 1 --2- 172.21.15.12:0/1286788991 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5598bc2f9b60 0x5598bc302a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:07:10.270 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:10.266+0000 7f6d1422f740 1 --2- 172.21.15.12:0/1286788991 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5598bc2f9240 0x5598bc3055b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:07:10.270 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:10.266+0000 7f6d1422f740 1 -- 172.21.15.12:0/1286788991 >> 172.21.15.12:0/1286788991 conn(0x5598bc2f7020 msgr2=0x5598bc2fcc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:07:10.270 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:10.266+0000 7f6cf4c00640 0 client.25483 ms_handle_reset on v2:172.21.15.165:6838/501312984 2024-09-11T00:07:10.270 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:10.267+0000 7f6d1422f740 1 -- 172.21.15.12:0/1286788991 shutdown_connections 2024-09-11T00:07:10.271 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:10.267+0000 7f6d1422f740 1 -- 172.21.15.12:0/1286788991 wait complete. 2024-09-11T00:07:10.271 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:10.267+0000 7f6d1422f740 20 client.25483 trim_cache size 0 max 16384 2024-09-11T00:07:10.307 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:10.305+0000 7f6d1422f740 1 Processor -- start 2024-09-11T00:07:10.308 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:10.305+0000 7f6d1422f740 1 -- start start 2024-09-11T00:07:10.308 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:10.306+0000 7f6d1422f740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5598bc2f9240 0x5598bc316f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:07:10.308 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:10.306+0000 7f6d1422f740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5598bc2f9b60 0x5598bc317640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:07:10.308 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:10.306+0000 7f6d1422f740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5598bc305af0 0x5598bc3161d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:07:10.308 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:10.306+0000 7f6d1422f740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x5598bc316050 con 0x5598bc2f9b60 2024-09-11T00:07:10.308 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:10.306+0000 7f6d1422f740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x5598bc316720 con 0x5598bc305af0 2024-09-11T00:07:10.308 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:10.306+0000 7f6d1422f740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5598bc316df0 con 0x5598bc2f9240 2024-09-11T00:07:10.309 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:10.306+0000 7f6d0a000640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5598bc2f9b60 0x5598bc317640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:07:10.309 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:10.306+0000 7f6d0aa00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5598bc2f9240 0x5598bc316f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:07:10.309 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:10.306+0000 7f6d0a000640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5598bc2f9b60 0x5598bc317640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:34982/0 (socket says 172.21.15.12:34982) 2024-09-11T00:07:10.309 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:10.306+0000 7f6d0a000640 1 -- 172.21.15.12:0/3453035132 learned_addr learned my addr 172.21.15.12:0/3453035132 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:07:10.309 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:10.306+0000 7f6d0b400640 1 --2- 172.21.15.12:0/3453035132 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5598bc305af0 0x5598bc3161d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:07:10.309 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:10.306+0000 7f6d0aa00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5598bc2f9240 0x5598bc316f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.12:39608/0 (socket says 172.21.15.12:39608) 2024-09-11T00:07:10.309 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:10.307+0000 7f6d0aa00640 1 -- 172.21.15.12:0/3453035132 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5598bc305af0 msgr2=0x5598bc3161d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:07:10.309 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:10.307+0000 7f6d0aa00640 1 --2- 172.21.15.12:0/3453035132 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5598bc305af0 0x5598bc3161d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:07:10.310 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:10.307+0000 7f6d0aa00640 1 -- 172.21.15.12:0/3453035132 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5598bc2f9b60 msgr2=0x5598bc317640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:07:10.310 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:10.307+0000 7f6d0aa00640 1 --2- 172.21.15.12:0/3453035132 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5598bc2f9b60 0x5598bc317640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:07:10.310 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:10.307+0000 7f6d0aa00640 1 -- 172.21.15.12:0/3453035132 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5598bc300f00 con 0x5598bc2f9240 2024-09-11T00:07:10.310 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:10.307+0000 7f6d0aa00640 1 --2- 172.21.15.12:0/3453035132 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5598bc2f9240 0x5598bc316f70 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f6cfc000c00 tx=0x7f6cfc004900 comp rx=0 tx=0).ready entity=mon.1 client_cookie=52acc70bcfce08df server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:07:10.310 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:10.308+0000 7f6cf7400640 1 -- 172.21.15.12:0/3453035132 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cfc004680 con 0x5598bc2f9240 2024-09-11T00:07:10.311 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:10.308+0000 7f6cf7400640 1 -- 172.21.15.12:0/3453035132 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6cfc031070 con 0x5598bc2f9240 2024-09-11T00:07:10.311 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:10.308+0000 7f6cf7400640 1 -- 172.21.15.12:0/3453035132 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cfc019430 con 0x5598bc2f9240 2024-09-11T00:07:10.311 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:10.309+0000 7f6d1422f740 1 -- 172.21.15.12:0/3453035132 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5598bc2f9240 msgr2=0x5598bc316f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:07:10.311 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:10.309+0000 7f6d1422f740 1 --2- 172.21.15.12:0/3453035132 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5598bc2f9240 0x5598bc316f70 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f6cfc000c00 tx=0x7f6cfc004900 comp rx=0 tx=0).stop 2024-09-11T00:07:10.311 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:10.309+0000 7f6d1422f740 1 -- 172.21.15.12:0/3453035132 shutdown_connections 2024-09-11T00:07:10.311 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:10.309+0000 7f6d1422f740 1 --2- 172.21.15.12:0/3453035132 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5598bc305af0 0x5598bc3161d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:07:10.311 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:10.309+0000 7f6d1422f740 1 --2- 172.21.15.12:0/3453035132 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5598bc2f9b60 0x5598bc317640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:07:10.311 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:10.309+0000 7f6d1422f740 1 --2- 172.21.15.12:0/3453035132 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5598bc2f9240 0x5598bc316f70 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:07:10.311 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:10.309+0000 7f6d1422f740 1 -- 172.21.15.12:0/3453035132 >> 172.21.15.12:0/3453035132 conn(0x5598bc2f7020 msgr2=0x5598bc2f7f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:07:10.311 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:10.309+0000 7f6d1422f740 1 -- 172.21.15.12:0/3453035132 shutdown_connections 2024-09-11T00:07:10.311 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:10.309+0000 7f6d1422f740 1 -- 172.21.15.12:0/3453035132 wait complete. 2024-09-11T00:07:10.312 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:10.309+0000 7f6d1422f740 1 -- start start 2024-09-11T00:07:10.312 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:10.310+0000 7f6d1422f740 1 Processor -- start 2024-09-11T00:07:10.312 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:10.310+0000 7f6d1422f740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5598bc2f9240 0x5598bc3205c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:07:10.312 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:10.310+0000 7f6d1422f740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5598bc2f9b60 0x5598bc310b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:07:10.313 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:10.310+0000 7f6d1422f740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5598bc305af0 0x5598bc3110a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:07:10.313 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:10.310+0000 7f6d1422f740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x5598bc317b90 con 0x5598bc2f9240 2024-09-11T00:07:10.313 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:10.310+0000 7f6d1422f740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x5598bc3174c0 con 0x5598bc305af0 2024-09-11T00:07:10.313 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:10.310+0000 7f6d1422f740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5598bc318260 con 0x5598bc2f9b60 2024-09-11T00:07:10.313 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:10.310+0000 7f6d1422f740 10 client.0 _ll_register_callbacks cb 0x7f6d040ac910 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:07:10.313 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:10.311+0000 7f6d0aa00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5598bc2f9240 0x5598bc3205c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:07:10.313 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:10.311+0000 7f6d0aa00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5598bc2f9240 0x5598bc3205c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:34986/0 (socket says 172.21.15.12:34986) 2024-09-11T00:07:10.313 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:10.311+0000 7f6d0aa00640 1 -- 172.21.15.12:0/4233327056 learned_addr learned my addr 172.21.15.12:0/4233327056 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:07:10.314 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:10.311+0000 7f6d0a000640 1 --2- 172.21.15.12:0/4233327056 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5598bc2f9b60 0x5598bc310b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:07:10.314 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:10.311+0000 7f6d0b400640 1 --2- 172.21.15.12:0/4233327056 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5598bc305af0 0x5598bc3110a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:07:10.314 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:10.311+0000 7f6d0aa00640 1 -- 172.21.15.12:0/4233327056 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5598bc2f9b60 msgr2=0x5598bc310b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:07:10.314 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:10.311+0000 7f6d0aa00640 1 --2- 172.21.15.12:0/4233327056 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5598bc2f9b60 0x5598bc310b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:07:10.314 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:10.311+0000 7f6d0aa00640 1 -- 172.21.15.12:0/4233327056 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5598bc305af0 msgr2=0x5598bc3110a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:07:10.314 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:10.311+0000 7f6d0aa00640 1 --2- 172.21.15.12:0/4233327056 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5598bc305af0 0x5598bc3110a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:07:10.314 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:10.311+0000 7f6d0aa00640 1 -- 172.21.15.12:0/4233327056 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5598bc3116e0 con 0x5598bc2f9240 2024-09-11T00:07:10.314 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:10.312+0000 7f6d0aa00640 1 --2- 172.21.15.12:0/4233327056 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5598bc2f9240 0x5598bc3205c0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f6cfc009bc0 tx=0x7f6cfc009390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5613032f9c21991e server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:07:10.314 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:10.312+0000 7f6cf4c00640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-09-11T00:07:10.315 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:10.312+0000 7f6cf4c00640 1 -- 172.21.15.12:0/4233327056 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cfc03b020 con 0x5598bc2f9240 2024-09-11T00:07:10.315 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:10.312+0000 7f6cf4c00640 1 -- 172.21.15.12:0/4233327056 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 9208+0+0 (secure 0 0 0) 0x7f6cfc0043e0 con 0x5598bc2f9240 2024-09-11T00:07:10.315 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:10.312+0000 7f6d1422f740 1 -- 172.21.15.12:0/4233327056 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6cfc031070 con 0x5598bc2f9240 2024-09-11T00:07:10.315 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:10.313+0000 7f6cf4c00640 1 -- 172.21.15.12:0/4233327056 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f6cfc0045e0 con 0x5598bc2f9240 2024-09-11T00:07:10.315 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:10.313+0000 7f6d1422f740 10 client.15468 fetch_fsmap learned FSMap version 29 2024-09-11T00:07:10.316 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:10.313+0000 7f6d1422f740 1 -- 172.21.15.12:0/4233327056 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5598bc38b9f0 con 0x5598bc2f9240 2024-09-11T00:07:10.316 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:10.313+0000 7f6cf4c00640 1 -- 172.21.15.12:0/4233327056 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6cfc031070 con 0x5598bc2f9240 2024-09-11T00:07:10.316 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:10.313+0000 7f6cf4c00640 1 -- 172.21.15.12:0/4233327056 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7f6cfc037840 con 0x5598bc2f9240 2024-09-11T00:07:10.316 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:10.313+0000 7f6d1422f740 10 client.15468 fetch_fsmap finished waiting for FSMap version 29 2024-09-11T00:07:10.316 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:10.313+0000 7f6d1422f740 10 client.15468 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-11T00:07:10.316 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:10.313+0000 7f6d1422f740 20 client.15468 populate_metadata read hostname 'smithi012' 2024-09-11T00:07:10.316 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:10.313+0000 7f6d1422f740 1 --2- 172.21.15.12:0/4233327056 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x5598bc38fe90 0x5598bc38eba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:07:10.316 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:10.313+0000 7f6d1422f740 4 client.15468 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-11T00:07:10.316 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:10.313+0000 7f6d1422f740 1 -- 172.21.15.12:0/4233327056 --> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] -- command(tid 0: {"prefix": "status"}) -- 0x5598bc26e750 con 0x5598bc38fe90 2024-09-11T00:07:10.317 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:10.314+0000 7f6d0a000640 1 --2- 172.21.15.12:0/4233327056 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x5598bc38fe90 0x5598bc38eba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:07:10.317 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:10.315+0000 7f6d0a000640 1 --2- 172.21.15.12:0/4233327056 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x5598bc38fe90 0x5598bc38eba0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3dfd97742fcf2810 server_cookie=5d3d1175b397d459 in_seq=0 out_seq=0 2024-09-11T00:07:10.317 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:10.315+0000 7f6cf4c00640 10 client.15468 ms_handle_connect on v2:172.21.15.165:6838/501312984 2024-09-11T00:07:10.317 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:10.315+0000 7f6cf4c00640 1 -- 172.21.15.12:0/4233327056 <== mds.0 v2:172.21.15.165:6838/501312984 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x5598bc26e750 con 0x5598bc38fe90 2024-09-11T00:07:10.317 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:10.315+0000 7f6cf4c00640 10 client.15468 handle_command_reply: tid=0 2024-09-11T00:07:10.318 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:10.315+0000 7f6d1422f740 1 client.15468 shutdown 2024-09-11T00:07:10.318 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:10.315+0000 7f6d1422f740 1 -- 172.21.15.12:0/4233327056 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5598bc2f9240 msgr2=0x5598bc3205c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:07:10.318 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:10.315+0000 7f6d1422f740 1 --2- 172.21.15.12:0/4233327056 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5598bc2f9240 0x5598bc3205c0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f6cfc009bc0 tx=0x7f6cfc009390 comp rx=0 tx=0).stop 2024-09-11T00:07:10.318 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:10.315+0000 7f6d1422f740 1 -- 172.21.15.12:0/4233327056 shutdown_connections 2024-09-11T00:07:10.318 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:10.315+0000 7f6d1422f740 1 --2- 172.21.15.12:0/4233327056 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x5598bc38fe90 0x5598bc38eba0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:07:10.318 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:10.315+0000 7f6d1422f740 1 --2- 172.21.15.12:0/4233327056 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5598bc305af0 0x5598bc3110a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:07:10.318 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:10.315+0000 7f6d1422f740 1 --2- 172.21.15.12:0/4233327056 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5598bc2f9b60 0x5598bc310b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:07:10.318 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:10.315+0000 7f6d1422f740 1 --2- 172.21.15.12:0/4233327056 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5598bc2f9240 0x5598bc3205c0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:07:10.318 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:10.315+0000 7f6d1422f740 1 -- 172.21.15.12:0/4233327056 >> 172.21.15.12:0/4233327056 conn(0x5598bc2f7020 msgr2=0x5598bc2f1ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:07:10.318 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:10.315+0000 7f6cf4c00640 0 client.15468 ms_handle_reset on v2:172.21.15.165:6838/501312984 2024-09-11T00:07:10.318 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:10.315+0000 7f6d1422f740 1 -- 172.21.15.12:0/4233327056 shutdown_connections 2024-09-11T00:07:10.318 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:10.316+0000 7f6d1422f740 1 -- 172.21.15.12:0/4233327056 wait complete. 2024-09-11T00:07:10.319 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:10.316+0000 7f6d1422f740 20 client.15468 trim_cache size 0 max 16384 2024-09-11T00:07:10.319 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-09-11T00:07:10.319 INFO:teuthology.orchestra.run.smithi012.stdout: "cluster_fsid": "ff3ad74a-6fcf-11ef-bcea-c7b262605968", 2024-09-11T00:07:10.319 INFO:teuthology.orchestra.run.smithi012.stdout: "whoami": 0, 2024-09-11T00:07:10.319 INFO:teuthology.orchestra.run.smithi012.stdout: "id": 24674, 2024-09-11T00:07:10.319 INFO:teuthology.orchestra.run.smithi012.stdout: "want_state": "up:active", 2024-09-11T00:07:10.319 INFO:teuthology.orchestra.run.smithi012.stdout: "state": "up:active", 2024-09-11T00:07:10.320 INFO:teuthology.orchestra.run.smithi012.stdout: "fs_name": "cephfs", 2024-09-11T00:07:10.320 INFO:teuthology.orchestra.run.smithi012.stdout: "rank_uptime": 185.477606775, 2024-09-11T00:07:10.320 INFO:teuthology.orchestra.run.smithi012.stdout: "mdsmap_epoch": 29, 2024-09-11T00:07:10.320 INFO:teuthology.orchestra.run.smithi012.stdout: "osdmap_epoch": 85, 2024-09-11T00:07:10.320 INFO:teuthology.orchestra.run.smithi012.stdout: "osdmap_epoch_barrier": 85, 2024-09-11T00:07:10.320 INFO:teuthology.orchestra.run.smithi012.stdout: "uptime": 198.38665325299999 2024-09-11T00:07:10.320 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-09-11T00:07:10.321 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:10.316+0000 7f6d11800640 1 -- 172.21.15.12:0/3533796436 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f6cf00b5360 msgr2=0x7f6cf00b7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:07:10.321 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:10.316+0000 7f6d11800640 1 --2- 172.21.15.12:0/3533796436 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f6cf00b5360 0x7f6cf00b7820 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f6cf8032fd0 tx=0x7f6cf8041040 comp rx=0 tx=0).stop 2024-09-11T00:07:10.321 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:10.316+0000 7f6d11800640 1 -- 172.21.15.12:0/3533796436 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6d0c10d120 msgr2=0x7f6d0c1a0db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:07:10.321 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:10.316+0000 7f6d11800640 1 --2- 172.21.15.12:0/3533796436 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6d0c10d120 0x7f6d0c1a0db0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f6d04009fd0 tx=0x7f6d0400ef10 comp rx=0 tx=0).stop 2024-09-11T00:07:10.321 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:10.317+0000 7f6d11800640 1 -- 172.21.15.12:0/3533796436 shutdown_connections 2024-09-11T00:07:10.321 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:10.317+0000 7f6d11800640 1 --2- 172.21.15.12:0/3533796436 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f6cf00b5360 0x7f6cf00b7820 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:07:10.321 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:10.317+0000 7f6d11800640 1 --2- 172.21.15.12:0/3533796436 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6d0c10d120 0x7f6d0c1a0db0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:07:10.321 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:10.317+0000 7f6d11800640 1 --2- 172.21.15.12:0/3533796436 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6d0c1045e0 0x7f6d0c19ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:07:10.321 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:10.317+0000 7f6d11800640 1 --2- 172.21.15.12:0/3533796436 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6d0c103cc0 0x7f6d0c19b4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:07:10.321 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:10.317+0000 7f6d11800640 1 -- 172.21.15.12:0/3533796436 >> 172.21.15.12:0/3533796436 conn(0x7f6d0c0fd700 msgr2=0x7f6d0c10f370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:07:10.321 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:10.317+0000 7f6d11800640 1 -- 172.21.15.12:0/3533796436 shutdown_connections 2024-09-11T00:07:10.321 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:10.317+0000 7f6d11800640 1 -- 172.21.15.12:0/3533796436 wait complete. 2024-09-11T00:07:10.782 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:07:10 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:07:10.783 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:07:10 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:07:10.783 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:07:10 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:07:10.783 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:07:10 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:07:10.783 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:07:10 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:07:10.783 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:07:10 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:07:10.813 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:07:10 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:07:10.813 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:07:10 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:07:10.813 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:07:10 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:07:10.813 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:07:10 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:07:10.813 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:07:10 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:07:10.813 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:07:10 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:07:10.830 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:07:10 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:07:10.830 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:07:10 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:07:10.831 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:07:10 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:07:10.831 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:07:10 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:07:10.831 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:07:10 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:07:10.831 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:07:10 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:07:11.057 INFO:tasks.quiescer:Ready to start quiesce thrashing; initial delay: 15 sec 2024-09-11T00:07:11.705 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:11.703+0000 7f9aa3400640 1 -- 172.21.15.12:0/2755162832 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9a9c105cf0 msgr2=0x7f9a9c106170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:07:11.705 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:11.703+0000 7f9aa3400640 1 --2- 172.21.15.12:0/2755162832 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9a9c105cf0 0x7f9a9c106170 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f9a88009930 tx=0x7f9a8802a420 comp rx=0 tx=0).stop 2024-09-11T00:07:11.705 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:11.703+0000 7f9aa3400640 1 -- 172.21.15.12:0/2755162832 shutdown_connections 2024-09-11T00:07:11.705 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:11.703+0000 7f9aa3400640 1 --2- 172.21.15.12:0/2755162832 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9a9c1066b0 0x7f9a9c10d030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:07:11.706 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:11.703+0000 7f9aa3400640 1 --2- 172.21.15.12:0/2755162832 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9a9c105cf0 0x7f9a9c106170 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:07:11.706 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:11.703+0000 7f9aa3400640 1 --2- 172.21.15.12:0/2755162832 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9a9c104ba0 0x7f9a9c104fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:07:11.706 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:11.703+0000 7f9aa3400640 1 -- 172.21.15.12:0/2755162832 >> 172.21.15.12:0/2755162832 conn(0x7f9a9c100460 msgr2=0x7f9a9c102880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:07:11.706 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:11.703+0000 7f9aa3400640 1 -- 172.21.15.12:0/2755162832 shutdown_connections 2024-09-11T00:07:11.706 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:11.703+0000 7f9aa3400640 1 -- 172.21.15.12:0/2755162832 wait complete. 2024-09-11T00:07:11.708 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:11.704+0000 7f9aa3400640 1 Processor -- start 2024-09-11T00:07:11.708 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:11.704+0000 7f9aa3400640 1 -- start start 2024-09-11T00:07:11.708 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:11.704+0000 7f9aa3400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9a9c104ba0 0x7f9a9c19af80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:07:11.708 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:11.704+0000 7f9aa3400640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9a9c105cf0 0x7f9a9c19b5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:07:11.708 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:11.704+0000 7f9aa3400640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9a9c1066b0 0x7f9a9c19c200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:07:11.708 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:11.704+0000 7f9aa3400640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f9a9c110860 con 0x7f9a9c105cf0 2024-09-11T00:07:11.708 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:11.704+0000 7f9aa3400640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f9a9c1106e0 con 0x7f9a9c104ba0 2024-09-11T00:07:11.709 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:11.704+0000 7f9aa3400640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f9a9c1109e0 con 0x7f9a9c1066b0 2024-09-11T00:07:11.709 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:11.705+0000 7f9a9be00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9a9c105cf0 0x7f9a9c19b5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:07:11.709 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:11.705+0000 7f9a9be00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9a9c105cf0 0x7f9a9c19b5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:35004/0 (socket says 172.21.15.12:35004) 2024-09-11T00:07:11.709 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:11.705+0000 7f9a9be00640 1 -- 172.21.15.12:0/132030862 learned_addr learned my addr 172.21.15.12:0/132030862 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:07:11.709 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:11.705+0000 7f9aa0a00640 1 --2- 172.21.15.12:0/132030862 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9a9c104ba0 0x7f9a9c19af80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:07:11.709 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:11.705+0000 7f9aa1400640 1 --2- 172.21.15.12:0/132030862 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9a9c1066b0 0x7f9a9c19c200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:07:11.709 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:11.705+0000 7f9a9be00640 1 -- 172.21.15.12:0/132030862 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9a9c1066b0 msgr2=0x7f9a9c19c200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:07:11.709 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:11.705+0000 7f9a9be00640 1 --2- 172.21.15.12:0/132030862 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9a9c1066b0 0x7f9a9c19c200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:07:11.709 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:11.705+0000 7f9a9be00640 1 -- 172.21.15.12:0/132030862 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9a9c104ba0 msgr2=0x7f9a9c19af80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:07:11.709 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:11.705+0000 7f9a9be00640 1 --2- 172.21.15.12:0/132030862 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9a9c104ba0 0x7f9a9c19af80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:07:11.709 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:11.705+0000 7f9a9be00640 1 -- 172.21.15.12:0/132030862 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a9c19cac0 con 0x7f9a9c105cf0 2024-09-11T00:07:11.709 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:11.705+0000 7f9aa0a00640 1 --2- 172.21.15.12:0/132030862 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9a9c104ba0 0x7f9a9c19af80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:07:11.710 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:11.706+0000 7f9a9be00640 1 --2- 172.21.15.12:0/132030862 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9a9c105cf0 0x7f9a9c19b5d0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f9a88031500 tx=0x7f9a88003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:07:11.710 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:11.706+0000 7f9a99600640 1 -- 172.21.15.12:0/132030862 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a8803f070 con 0x7f9a9c105cf0 2024-09-11T00:07:11.710 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:11.706+0000 7f9a99600640 1 -- 172.21.15.12:0/132030862 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9a88031c50 con 0x7f9a9c105cf0 2024-09-11T00:07:11.710 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:11.706+0000 7f9aa3400640 1 -- 172.21.15.12:0/132030862 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9a9c19cd50 con 0x7f9a9c105cf0 2024-09-11T00:07:11.710 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:11.706+0000 7f9a99600640 1 -- 172.21.15.12:0/132030862 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a88035030 con 0x7f9a9c105cf0 2024-09-11T00:07:11.710 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:11.706+0000 7f9aa3400640 1 -- 172.21.15.12:0/132030862 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9a9c1a3480 con 0x7f9a9c105cf0 2024-09-11T00:07:11.712 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:11.709+0000 7f9aa63ed740 1 Processor -- start 2024-09-11T00:07:11.712 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:11.710+0000 7f9aa63ed740 1 -- start start 2024-09-11T00:07:11.712 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:11.710+0000 7f9aa63ed740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564fc87a4790 0x564fc87a6b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:07:11.712 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:11.710+0000 7f9aa63ed740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564fc87a70c0 0x564fc87a94b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:07:11.712 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:11.710+0000 7f9aa63ed740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564fc87a9b00 0x564fc87abfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:07:11.713 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:11.710+0000 7f9aa63ed740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x564fc87176b0 con 0x564fc87a9b00 2024-09-11T00:07:11.713 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:11.710+0000 7f9aa63ed740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x564fc8706ae0 con 0x564fc87a4790 2024-09-11T00:07:11.713 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:11.710+0000 7f9aa63ed740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x564fc870a8f0 con 0x564fc87a70c0 2024-09-11T00:07:11.713 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:11.710+0000 7f9a99600640 1 -- 172.21.15.12:0/132030862 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7f9a8803b460 con 0x7f9a9c105cf0 2024-09-11T00:07:11.713 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:11.710+0000 7f9aa1400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564fc87a4790 0x564fc87a6b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:07:11.713 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:11.710+0000 7f9aa1400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564fc87a4790 0x564fc87a6b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.12:35920/0 (socket says 172.21.15.12:35920) 2024-09-11T00:07:11.713 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:11.710+0000 7f9aa1400640 1 -- 172.21.15.12:0/932625186 learned_addr learned my addr 172.21.15.12:0/932625186 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:07:11.713 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:11.710+0000 7f9aa0a00640 1 --2- 172.21.15.12:0/932625186 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564fc87a70c0 0x564fc87a94b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:07:11.713 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:11.710+0000 7f9a9be00640 1 --2- 172.21.15.12:0/932625186 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564fc87a9b00 0x564fc87abfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:07:11.715 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:11.711+0000 7f9aa1400640 1 -- 172.21.15.12:0/932625186 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564fc87a70c0 msgr2=0x564fc87a94b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:07:11.715 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:11.711+0000 7f9aa1400640 1 --2- 172.21.15.12:0/932625186 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564fc87a70c0 0x564fc87a94b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:07:11.715 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:11.711+0000 7f9aa1400640 1 -- 172.21.15.12:0/932625186 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564fc87a9b00 msgr2=0x564fc87abfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:07:11.715 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:11.711+0000 7f9aa1400640 1 --2- 172.21.15.12:0/932625186 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564fc87a9b00 0x564fc87abfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:07:11.715 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:11.711+0000 7f9aa1400640 1 -- 172.21.15.12:0/932625186 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564fc86d3780 con 0x564fc87a4790 2024-09-11T00:07:11.715 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:11.711+0000 7f9aa1400640 1 --2- 172.21.15.12:0/932625186 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564fc87a4790 0x564fc87a6b80 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f9a94009f90 tx=0x7f9a940118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b18919f079ea5451 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:07:11.715 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:11.711+0000 7f9a8dc00640 1 -- 172.21.15.12:0/932625186 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a94004d20 con 0x564fc87a4790 2024-09-11T00:07:11.715 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:11.711+0000 7f9a8dc00640 1 -- 172.21.15.12:0/932625186 <== mon.2 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9a94004050 con 0x564fc87a4790 2024-09-11T00:07:11.716 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:11.711+0000 7f9a8dc00640 1 -- 172.21.15.12:0/932625186 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a94005280 con 0x564fc87a4790 2024-09-11T00:07:11.716 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:11.712+0000 7f9aa63ed740 1 -- 172.21.15.12:0/932625186 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564fc87a4790 msgr2=0x564fc87a6b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:07:11.716 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:11.712+0000 7f9aa63ed740 1 --2- 172.21.15.12:0/932625186 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564fc87a4790 0x564fc87a6b80 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f9a94009f90 tx=0x7f9a940118d0 comp rx=0 tx=0).stop 2024-09-11T00:07:11.716 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:11.712+0000 7f9aa63ed740 1 -- 172.21.15.12:0/932625186 shutdown_connections 2024-09-11T00:07:11.716 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:11.712+0000 7f9aa63ed740 1 --2- 172.21.15.12:0/932625186 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564fc87a9b00 0x564fc87abfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:07:11.716 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:11.712+0000 7f9aa63ed740 1 --2- 172.21.15.12:0/932625186 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564fc87a70c0 0x564fc87a94b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:07:11.716 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:11.712+0000 7f9aa63ed740 1 --2- 172.21.15.12:0/932625186 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564fc87a4790 0x564fc87a6b80 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:07:11.716 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:11.712+0000 7f9aa63ed740 1 -- 172.21.15.12:0/932625186 >> 172.21.15.12:0/932625186 conn(0x564fc879f8e0 msgr2=0x564fc87a0670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:07:11.716 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:11.713+0000 7f9aa63ed740 1 -- 172.21.15.12:0/932625186 shutdown_connections 2024-09-11T00:07:11.716 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:11.713+0000 7f9aa63ed740 1 -- 172.21.15.12:0/932625186 wait complete. 2024-09-11T00:07:11.716 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:11.713+0000 7f9a99600640 1 --2- 172.21.15.12:0/132030862 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f9a840b5270 0x7f9a840b7730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:07:11.716 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:11.713+0000 7f9a99600640 1 -- 172.21.15.12:0/132030862 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9208+0+0 (secure 0 0 0) 0x7f9a88106960 con 0x7f9a9c105cf0 2024-09-11T00:07:11.716 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:11.713+0000 7f9aa63ed740 1 -- start start 2024-09-11T00:07:11.717 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:11.713+0000 7f9aa0a00640 1 --2- 172.21.15.12:0/132030862 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f9a840b5270 0x7f9a840b7730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:07:11.717 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:11.713+0000 7f9aa63ed740 1 Processor -- start 2024-09-11T00:07:11.719 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:11.714+0000 7f9aa63ed740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564fc87a4790 0x564fc87adf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:07:11.719 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:11.714+0000 7f9aa63ed740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564fc87a70c0 0x564fc87ae5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:07:11.719 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:11.714+0000 7f9aa0a00640 1 --2- 172.21.15.12:0/132030862 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f9a840b5270 0x7f9a840b7730 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f9a90012560 tx=0x7f9a90005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:07:11.719 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:11.714+0000 7f9aa63ed740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564fc87a9b00 0x564fc87af0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:07:11.720 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:11.714+0000 7f9aa63ed740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x564fc863eab0 con 0x564fc87a70c0 2024-09-11T00:07:11.720 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:11.714+0000 7f9aa63ed740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x564fc86ad570 con 0x564fc87a4790 2024-09-11T00:07:11.720 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:11.714+0000 7f9aa63ed740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x564fc87acae0 con 0x564fc87a9b00 2024-09-11T00:07:11.720 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:11.714+0000 7f9aa63ed740 10 client.0 _ll_register_callbacks cb 0x564fc86bc100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:07:11.720 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:11.714+0000 7f9a9be00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564fc87a70c0 0x564fc87ae5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:07:11.720 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:11.714+0000 7f9aa0a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564fc87a4790 0x564fc87adf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:07:11.720 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:11.714+0000 7f9aa0a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564fc87a4790 0x564fc87adf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.12:35922/0 (socket says 172.21.15.12:35922) 2024-09-11T00:07:11.720 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:11.714+0000 7f9aa0a00640 1 -- 172.21.15.12:0/1381953294 learned_addr learned my addr 172.21.15.12:0/1381953294 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:07:11.720 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:11.714+0000 7f9a9be00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564fc87a70c0 0x564fc87ae5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:35020/0 (socket says 172.21.15.12:35020) 2024-09-11T00:07:11.720 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:11.714+0000 7f9aa1400640 1 --2- 172.21.15.12:0/1381953294 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564fc87a9b00 0x564fc87af0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:07:11.720 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:11.715+0000 7f9aa0a00640 1 -- 172.21.15.12:0/1381953294 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564fc87a9b00 msgr2=0x564fc87af0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:07:11.720 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:11.715+0000 7f9aa0a00640 1 --2- 172.21.15.12:0/1381953294 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564fc87a9b00 0x564fc87af0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:07:11.721 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:11.715+0000 7f9aa0a00640 1 -- 172.21.15.12:0/1381953294 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564fc87a70c0 msgr2=0x564fc87ae5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:07:11.721 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:11.715+0000 7f9aa0a00640 1 --2- 172.21.15.12:0/1381953294 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564fc87a70c0 0x564fc87ae5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:07:11.721 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:11.715+0000 7f9aa0a00640 1 -- 172.21.15.12:0/1381953294 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564fc8468790 con 0x564fc87a4790 2024-09-11T00:07:11.721 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:11.715+0000 7f9a9be00640 1 --2- 172.21.15.12:0/1381953294 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564fc87a70c0 0x564fc87ae5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:07:11.721 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:11.715+0000 7f9aa0a00640 1 --2- 172.21.15.12:0/1381953294 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564fc87a4790 0x564fc87adf20 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f9a90009e90 tx=0x7f9a90004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2fd1db4135dc5823 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:07:11.721 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:11.715+0000 7f9a82000640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-09-11T00:07:11.721 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:11.715+0000 7f9a82000640 1 -- 172.21.15.12:0/1381953294 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a90005020 con 0x564fc87a4790 2024-09-11T00:07:11.721 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:11.716+0000 7f9aa63ed740 1 -- 172.21.15.12:0/1381953294 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9a94004050 con 0x564fc87a4790 2024-09-11T00:07:11.721 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:11.716+0000 7f9a82000640 1 -- 172.21.15.12:0/1381953294 <== mon.2 v2:172.21.15.165:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 9208+0+0 (secure 0 0 0) 0x7f9a900051c0 con 0x564fc87a4790 2024-09-11T00:07:11.721 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:11.716+0000 7f9a82000640 1 -- 172.21.15.12:0/1381953294 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f9a900053c0 con 0x564fc87a4790 2024-09-11T00:07:11.721 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:11.716+0000 7f9aa63ed740 10 client.25520 fetch_fsmap learned FSMap version 29 2024-09-11T00:07:11.721 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:11.716+0000 7f9aa63ed740 1 -- 172.21.15.12:0/1381953294 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x564fc84e1620 con 0x564fc87a4790 2024-09-11T00:07:11.721 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:11.716+0000 7f9a82000640 1 -- 172.21.15.12:0/1381953294 <== mon.2 v2:172.21.15.165:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9a94004050 con 0x564fc87a4790 2024-09-11T00:07:11.722 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:11.717+0000 7f9a82000640 1 -- 172.21.15.12:0/1381953294 <== mon.2 v2:172.21.15.165:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7f9a90027070 con 0x564fc87a4790 2024-09-11T00:07:11.722 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:11.717+0000 7f9aa63ed740 10 client.25520 fetch_fsmap finished waiting for FSMap version 29 2024-09-11T00:07:11.722 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:11.717+0000 7f9aa63ed740 10 client.25520 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T00:07:11.722 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:11.717+0000 7f9aa63ed740 20 client.25520 populate_metadata read hostname 'smithi012' 2024-09-11T00:07:11.722 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:11.717+0000 7f9aa63ed740 1 --2- 172.21.15.12:0/1381953294 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x564fc87b97f0 0x564fc87bbc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:07:11.722 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:11.717+0000 7f9aa63ed740 4 client.25520 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T00:07:11.722 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:11.717+0000 7f9aa63ed740 1 -- 172.21.15.12:0/1381953294 --> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564fc8719600 con 0x564fc87b97f0 2024-09-11T00:07:11.722 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:11.717+0000 7f9a9be00640 1 --2- 172.21.15.12:0/1381953294 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x564fc87b97f0 0x564fc87bbc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:07:11.722 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:11.718+0000 7f9a9be00640 1 --2- 172.21.15.12:0/1381953294 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x564fc87b97f0 0x564fc87bbc20 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4d4c7752792b0b3c server_cookie=13bc0708ebf3f97f in_seq=0 out_seq=0 2024-09-11T00:07:11.722 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:11.718+0000 7f9a82000640 10 client.25520 ms_handle_connect on v2:172.21.15.165:6838/501312984 2024-09-11T00:07:11.723 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:11.718+0000 7f9a82000640 1 -- 172.21.15.12:0/1381953294 <== mds.0 v2:172.21.15.165:6838/501312984 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x564fc8719600 con 0x564fc87b97f0 2024-09-11T00:07:11.724 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:11.718+0000 7f9a82000640 10 client.25520 handle_command_reply: tid=0 2024-09-11T00:07:11.724 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:11.719+0000 7f9aa63ed740 1 client.25520 shutdown 2024-09-11T00:07:11.724 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:11.719+0000 7f9aa63ed740 1 -- 172.21.15.12:0/1381953294 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564fc87a4790 msgr2=0x564fc87adf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:07:11.724 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:11.719+0000 7f9aa63ed740 1 --2- 172.21.15.12:0/1381953294 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564fc87a4790 0x564fc87adf20 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f9a90009e90 tx=0x7f9a90004730 comp rx=0 tx=0).stop 2024-09-11T00:07:11.724 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:11.719+0000 7f9aa63ed740 1 -- 172.21.15.12:0/1381953294 shutdown_connections 2024-09-11T00:07:11.724 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:11.719+0000 7f9aa63ed740 1 --2- 172.21.15.12:0/1381953294 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x564fc87b97f0 0x564fc87bbc20 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:07:11.724 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:11.719+0000 7f9aa63ed740 1 --2- 172.21.15.12:0/1381953294 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564fc87a9b00 0x564fc87af0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:07:11.724 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:11.719+0000 7f9aa63ed740 1 --2- 172.21.15.12:0/1381953294 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564fc87a70c0 0x564fc87ae5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:07:11.724 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:11.719+0000 7f9aa63ed740 1 --2- 172.21.15.12:0/1381953294 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564fc87a4790 0x564fc87adf20 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:07:11.724 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:11.719+0000 7f9aa63ed740 1 -- 172.21.15.12:0/1381953294 >> 172.21.15.12:0/1381953294 conn(0x564fc879f8e0 msgr2=0x564fc879ea60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:07:11.724 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:11.719+0000 7f9a82000640 0 client.25520 ms_handle_reset on v2:172.21.15.165:6838/501312984 2024-09-11T00:07:11.724 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:11.719+0000 7f9aa63ed740 1 -- 172.21.15.12:0/1381953294 shutdown_connections 2024-09-11T00:07:11.724 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:11.719+0000 7f9aa63ed740 1 -- 172.21.15.12:0/1381953294 wait complete. 2024-09-11T00:07:11.725 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:11.719+0000 7f9aa63ed740 20 client.25520 trim_cache size 0 max 16384 2024-09-11T00:07:11.737 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:11.735+0000 7f9aa63ed740 1 Processor -- start 2024-09-11T00:07:11.737 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:11.735+0000 7f9aa63ed740 1 -- start start 2024-09-11T00:07:11.737 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:11.735+0000 7f9aa63ed740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564fc87a4790 0x564fc87a0380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:07:11.737 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:11.735+0000 7f9aa63ed740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564fc87a70c0 0x564fc87a3c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:07:11.737 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:11.735+0000 7f9aa63ed740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564fc87a9b00 0x564fc87a41c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:07:11.737 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:11.735+0000 7f9aa63ed740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f9a9c1ad950 con 0x564fc87a4790 2024-09-11T00:07:11.737 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:11.735+0000 7f9aa63ed740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f9a9c1af490 con 0x564fc87a9b00 2024-09-11T00:07:11.737 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:11.735+0000 7f9aa63ed740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f9a9c1ae020 con 0x564fc87a70c0 2024-09-11T00:07:11.738 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:11.735+0000 7f9aa0a00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564fc87a4790 0x564fc87a0380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:07:11.738 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:11.735+0000 7f9aa0a00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564fc87a4790 0x564fc87a0380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:35034/0 (socket says 172.21.15.12:35034) 2024-09-11T00:07:11.739 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:11.735+0000 7f9aa0a00640 1 -- 172.21.15.12:0/3087492105 learned_addr learned my addr 172.21.15.12:0/3087492105 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:07:11.739 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:11.735+0000 7f9aa1400640 1 --2- 172.21.15.12:0/3087492105 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564fc87a9b00 0x564fc87a41c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:07:11.739 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:11.735+0000 7f9a9be00640 1 --2- 172.21.15.12:0/3087492105 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564fc87a70c0 0x564fc87a3c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:07:11.739 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:11.736+0000 7f9aa0a00640 1 -- 172.21.15.12:0/3087492105 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564fc87a70c0 msgr2=0x564fc87a3c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:07:11.739 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:11.736+0000 7f9aa0a00640 1 --2- 172.21.15.12:0/3087492105 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564fc87a70c0 0x564fc87a3c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:07:11.739 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:11.736+0000 7f9aa0a00640 1 -- 172.21.15.12:0/3087492105 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564fc87a9b00 msgr2=0x564fc87a41c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:07:11.739 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:11.736+0000 7f9aa0a00640 1 --2- 172.21.15.12:0/3087492105 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564fc87a9b00 0x564fc87a41c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:07:11.739 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:11.736+0000 7f9aa0a00640 1 -- 172.21.15.12:0/3087492105 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564fc87a6ef0 con 0x564fc87a4790 2024-09-11T00:07:11.739 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:11.736+0000 7f9aa0a00640 1 --2- 172.21.15.12:0/3087492105 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564fc87a4790 0x564fc87a0380 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f9a90004c20 tx=0x7f9a90009ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20dfe3cd924aa6ec server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:07:11.739 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:11.736+0000 7f9a8d200640 1 -- 172.21.15.12:0/3087492105 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a90005020 con 0x564fc87a4790 2024-09-11T00:07:11.739 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:11.736+0000 7f9a8d200640 1 -- 172.21.15.12:0/3087492105 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9a900051c0 con 0x564fc87a4790 2024-09-11T00:07:11.739 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:11.736+0000 7f9a8d200640 1 -- 172.21.15.12:0/3087492105 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a90036690 con 0x564fc87a4790 2024-09-11T00:07:11.741 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:11.737+0000 7f9aa63ed740 1 -- 172.21.15.12:0/3087492105 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564fc87a4790 msgr2=0x564fc87a0380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:07:11.741 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:11.737+0000 7f9aa63ed740 1 --2- 172.21.15.12:0/3087492105 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564fc87a4790 0x564fc87a0380 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f9a90004c20 tx=0x7f9a90009ec0 comp rx=0 tx=0).stop 2024-09-11T00:07:11.741 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:11.737+0000 7f9aa63ed740 1 -- 172.21.15.12:0/3087492105 shutdown_connections 2024-09-11T00:07:11.741 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:11.737+0000 7f9aa63ed740 1 --2- 172.21.15.12:0/3087492105 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564fc87a9b00 0x564fc87a41c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:07:11.741 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:11.737+0000 7f9aa63ed740 1 --2- 172.21.15.12:0/3087492105 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564fc87a70c0 0x564fc87a3c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:07:11.741 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:11.737+0000 7f9aa63ed740 1 --2- 172.21.15.12:0/3087492105 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564fc87a4790 0x564fc87a0380 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:07:11.741 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:11.737+0000 7f9aa63ed740 1 -- 172.21.15.12:0/3087492105 >> 172.21.15.12:0/3087492105 conn(0x564fc879f8e0 msgr2=0x564fc87bb800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:07:11.742 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:11.737+0000 7f9aa63ed740 1 -- 172.21.15.12:0/3087492105 shutdown_connections 2024-09-11T00:07:11.742 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:11.737+0000 7f9aa63ed740 1 -- 172.21.15.12:0/3087492105 wait complete. 2024-09-11T00:07:11.742 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:11.738+0000 7f9aa63ed740 1 -- start start 2024-09-11T00:07:11.742 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:11.738+0000 7f9aa63ed740 1 Processor -- start 2024-09-11T00:07:11.742 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:11.739+0000 7f9aa63ed740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564fc87a4790 0x564fc87b5450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:07:11.742 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:11.739+0000 7f9aa63ed740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564fc87a70c0 0x564fc87b5af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:07:11.742 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:11.739+0000 7f9aa63ed740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564fc87a9b00 0x564fc87b41b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:07:11.742 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:11.739+0000 7f9aa63ed740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f9a9c1aedc0 con 0x564fc87a70c0 2024-09-11T00:07:11.742 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:11.739+0000 7f9aa63ed740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f9a9c1ae6f0 con 0x564fc87a4790 2024-09-11T00:07:11.742 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:11.739+0000 7f9aa63ed740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f9a9c1ab930 con 0x564fc87a9b00 2024-09-11T00:07:11.742 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:11.739+0000 7f9aa63ed740 10 client.0 _ll_register_callbacks cb 0x7f9a90009de0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:07:11.742 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:11.739+0000 7f9a9be00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564fc87a70c0 0x564fc87b5af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:07:11.742 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:11.739+0000 7f9aa1400640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564fc87a9b00 0x564fc87b41b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:07:11.743 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:11.739+0000 7f9a9be00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564fc87a70c0 0x564fc87b5af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:35048/0 (socket says 172.21.15.12:35048) 2024-09-11T00:07:11.743 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:11.739+0000 7f9a9be00640 1 -- 172.21.15.12:0/3186670537 learned_addr learned my addr 172.21.15.12:0/3186670537 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:07:11.743 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:11.739+0000 7f9aa0a00640 1 --2- 172.21.15.12:0/3186670537 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564fc87a4790 0x564fc87b5450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:07:11.745 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:11.740+0000 7f9aa1400640 1 -- 172.21.15.12:0/3186670537 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564fc87a4790 msgr2=0x564fc87b5450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:07:11.745 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:11.740+0000 7f9aa1400640 1 --2- 172.21.15.12:0/3186670537 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564fc87a4790 0x564fc87b5450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:07:11.745 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:11.740+0000 7f9aa1400640 1 -- 172.21.15.12:0/3186670537 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564fc87a70c0 msgr2=0x564fc87b5af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:07:11.745 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:11.740+0000 7f9aa1400640 1 --2- 172.21.15.12:0/3186670537 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564fc87a70c0 0x564fc87b5af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:07:11.745 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:11.740+0000 7f9aa1400640 1 -- 172.21.15.12:0/3186670537 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564fc87b49e0 con 0x564fc87a9b00 2024-09-11T00:07:11.745 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:11.740+0000 7f9aa0a00640 1 --2- 172.21.15.12:0/3186670537 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564fc87a4790 0x564fc87b5450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:07:11.745 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:11.740+0000 7f9a9be00640 1 --2- 172.21.15.12:0/3186670537 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564fc87a70c0 0x564fc87b5af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:07:11.745 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:11.740+0000 7f9aa1400640 1 --2- 172.21.15.12:0/3186670537 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564fc87a9b00 0x564fc87b41b0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f9a9401b040 tx=0x7f9a94004050 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8c3fee0c414d4edf server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:07:11.745 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:11.740+0000 7f9a82000640 10 client.0 ms_handle_connect on v2:172.21.15.178:3300/0 2024-09-11T00:07:11.745 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:11.740+0000 7f9a82000640 1 -- 172.21.15.12:0/3186670537 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a94004850 con 0x564fc87a9b00 2024-09-11T00:07:11.745 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:11.740+0000 7f9aa63ed740 1 -- 172.21.15.12:0/3186670537 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9a900051c0 con 0x564fc87a9b00 2024-09-11T00:07:11.746 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:11.740+0000 7f9a82000640 1 -- 172.21.15.12:0/3186670537 <== mon.1 v2:172.21.15.178:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 9208+0+0 (secure 0 0 0) 0x7f9a940049f0 con 0x564fc87a9b00 2024-09-11T00:07:11.746 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:11.741+0000 7f9a82000640 1 -- 172.21.15.12:0/3186670537 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f9a94004bf0 con 0x564fc87a9b00 2024-09-11T00:07:11.746 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:11.741+0000 7f9aa63ed740 10 client.25507 fetch_fsmap learned FSMap version 29 2024-09-11T00:07:11.746 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:11.741+0000 7f9aa63ed740 1 -- 172.21.15.12:0/3186670537 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x564fc87b38c0 con 0x564fc87a9b00 2024-09-11T00:07:11.746 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:11.741+0000 7f9a82000640 1 -- 172.21.15.12:0/3186670537 <== mon.1 v2:172.21.15.178:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9a900051c0 con 0x564fc87a9b00 2024-09-11T00:07:11.746 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:11.741+0000 7f9a82000640 1 -- 172.21.15.12:0/3186670537 <== mon.1 v2:172.21.15.178:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7f9a9402b070 con 0x564fc87a9b00 2024-09-11T00:07:11.746 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:11.741+0000 7f9aa63ed740 10 client.25507 fetch_fsmap finished waiting for FSMap version 29 2024-09-11T00:07:11.746 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:11.741+0000 7f9aa63ed740 10 client.25507 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T00:07:11.746 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:11.741+0000 7f9aa63ed740 20 client.25507 populate_metadata read hostname 'smithi012' 2024-09-11T00:07:11.746 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:11.741+0000 7f9aa63ed740 1 --2- 172.21.15.12:0/3186670537 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x564fc8826bc0 0x564fc8828fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:07:11.746 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:11.741+0000 7f9aa63ed740 4 client.25507 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T00:07:11.746 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:11.741+0000 7f9aa63ed740 1 -- 172.21.15.12:0/3186670537 --> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] -- command(tid 0: {"prefix": "flush journal"}) -- 0x564fc87150b0 con 0x564fc8826bc0 2024-09-11T00:07:11.746 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:11.742+0000 7f9aa0a00640 1 --2- 172.21.15.12:0/3186670537 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x564fc8826bc0 0x564fc8828fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:07:11.746 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:11.742+0000 7f9aa0a00640 1 --2- 172.21.15.12:0/3186670537 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x564fc8826bc0 0x564fc8828fe0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=af18ed845ec1f50 server_cookie=f5d02fc596a3adde in_seq=0 out_seq=0 2024-09-11T00:07:11.747 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:11.742+0000 7f9a82000640 10 client.25507 ms_handle_connect on v2:172.21.15.165:6838/501312984 2024-09-11T00:07:11.753 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:11.751+0000 7f9a82000640 1 -- 172.21.15.12:0/3186670537 <== mds.0 v2:172.21.15.165:6838/501312984 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x564fc87150b0 con 0x564fc8826bc0 2024-09-11T00:07:11.753 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:11.751+0000 7f9a82000640 10 client.25507 handle_command_reply: tid=0 2024-09-11T00:07:11.753 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:11.751+0000 7f9aa63ed740 1 client.25507 shutdown 2024-09-11T00:07:11.753 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:11.751+0000 7f9aa63ed740 1 -- 172.21.15.12:0/3186670537 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564fc87a9b00 msgr2=0x564fc87b41b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:07:11.753 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:11.751+0000 7f9aa63ed740 1 --2- 172.21.15.12:0/3186670537 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564fc87a9b00 0x564fc87b41b0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f9a9401b040 tx=0x7f9a94004050 comp rx=0 tx=0).stop 2024-09-11T00:07:11.753 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:11.751+0000 7f9aa63ed740 1 -- 172.21.15.12:0/3186670537 shutdown_connections 2024-09-11T00:07:11.754 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:11.751+0000 7f9aa63ed740 1 --2- 172.21.15.12:0/3186670537 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x564fc8826bc0 0x564fc8828fe0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:07:11.754 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:11.751+0000 7f9aa63ed740 1 --2- 172.21.15.12:0/3186670537 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564fc87a9b00 0x564fc87b41b0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:07:11.754 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:11.751+0000 7f9aa63ed740 1 --2- 172.21.15.12:0/3186670537 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564fc87a70c0 0x564fc87b5af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:07:11.754 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:11.751+0000 7f9aa63ed740 1 --2- 172.21.15.12:0/3186670537 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564fc87a4790 0x564fc87b5450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:07:11.754 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:11.751+0000 7f9a82000640 0 client.25507 ms_handle_reset on v2:172.21.15.165:6838/501312984 2024-09-11T00:07:11.754 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:11.751+0000 7f9aa63ed740 1 -- 172.21.15.12:0/3186670537 >> 172.21.15.12:0/3186670537 conn(0x564fc879f8e0 msgr2=0x564fc87a0380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:07:11.754 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:11.751+0000 7f9aa63ed740 1 -- 172.21.15.12:0/3186670537 shutdown_connections 2024-09-11T00:07:11.754 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:11.751+0000 7f9aa63ed740 1 -- 172.21.15.12:0/3186670537 wait complete. 2024-09-11T00:07:11.754 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:11.751+0000 7f9aa63ed740 20 client.25507 trim_cache size 0 max 16384 2024-09-11T00:07:11.755 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:{ 2024-09-11T00:07:11.755 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "message": "", 2024-09-11T00:07:11.755 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "return_code": 0 2024-09-11T00:07:11.755 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:} 2024-09-11T00:07:11.755 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:11.752+0000 7f9aa3400640 1 -- 172.21.15.12:0/132030862 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f9a840b5270 msgr2=0x7f9a840b7730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:07:11.755 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:11.752+0000 7f9aa3400640 1 --2- 172.21.15.12:0/132030862 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f9a840b5270 0x7f9a840b7730 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f9a90012560 tx=0x7f9a90005f90 comp rx=0 tx=0).stop 2024-09-11T00:07:11.756 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:11.752+0000 7f9aa3400640 1 -- 172.21.15.12:0/132030862 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9a9c105cf0 msgr2=0x7f9a9c19b5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:07:11.756 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:11.752+0000 7f9aa3400640 1 --2- 172.21.15.12:0/132030862 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9a9c105cf0 0x7f9a9c19b5d0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f9a88031500 tx=0x7f9a88003c10 comp rx=0 tx=0).stop 2024-09-11T00:07:11.756 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:11.752+0000 7f9aa3400640 1 -- 172.21.15.12:0/132030862 shutdown_connections 2024-09-11T00:07:11.756 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:11.752+0000 7f9aa3400640 1 --2- 172.21.15.12:0/132030862 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f9a840b5270 0x7f9a840b7730 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:07:11.756 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:11.752+0000 7f9aa3400640 1 --2- 172.21.15.12:0/132030862 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9a9c1066b0 0x7f9a9c19c200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:07:11.756 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:11.752+0000 7f9aa3400640 1 --2- 172.21.15.12:0/132030862 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9a9c105cf0 0x7f9a9c19b5d0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:07:11.756 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:11.752+0000 7f9aa3400640 1 --2- 172.21.15.12:0/132030862 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9a9c104ba0 0x7f9a9c19af80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:07:11.756 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:11.752+0000 7f9aa3400640 1 -- 172.21.15.12:0/132030862 >> 172.21.15.12:0/132030862 conn(0x7f9a9c100460 msgr2=0x7f9a9c109080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:07:11.756 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:11.752+0000 7f9aa3400640 1 -- 172.21.15.12:0/132030862 shutdown_connections 2024-09-11T00:07:11.756 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:11.753+0000 7f9aa3400640 1 -- 172.21.15.12:0/132030862 wait complete. 2024-09-11T00:07:11.782 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:07:11 smithi165 ceph-mon[35356]: pgmap v401: 129 pgs: 129 active+clean; 663 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 2.7 KiB/s wr, 15 op/s 2024-09-11T00:07:11.783 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:07:11 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:07:11.783 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:07:11 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T00:07:11.783 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:07:11 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:07:11.813 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:07:11 smithi012 ceph-mon[30333]: pgmap v401: 129 pgs: 129 active+clean; 663 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 2.7 KiB/s wr, 15 op/s 2024-09-11T00:07:11.813 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:07:11 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:07:11.813 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:07:11 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T00:07:11.813 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:07:11 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:07:11.830 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:07:11 smithi178 ceph-mon[35919]: pgmap v401: 129 pgs: 129 active+clean; 663 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 2.7 KiB/s wr, 15 op/s 2024-09-11T00:07:11.831 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:07:11 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:07:11.831 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:07:11 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T00:07:11.831 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:07:11 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:07:13.782 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:07:13 smithi165 ceph-mon[35356]: pgmap v402: 129 pgs: 129 active+clean; 663 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 1.9 KiB/s wr, 13 op/s 2024-09-11T00:07:13.812 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:07:13 smithi012 ceph-mon[30333]: pgmap v402: 129 pgs: 129 active+clean; 663 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 1.9 KiB/s wr, 13 op/s 2024-09-11T00:07:13.830 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:07:13 smithi178 ceph-mon[35919]: pgmap v402: 129 pgs: 129 active+clean; 663 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 1.9 KiB/s wr, 13 op/s 2024-09-11T00:07:15.782 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:07:15 smithi165 ceph-mon[35356]: pgmap v403: 129 pgs: 129 active+clean; 663 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 170 B/s wr, 8 op/s 2024-09-11T00:07:15.812 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:07:15 smithi012 ceph-mon[30333]: pgmap v403: 129 pgs: 129 active+clean; 663 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 170 B/s wr, 8 op/s 2024-09-11T00:07:15.830 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:07:15 smithi178 ceph-mon[35919]: pgmap v403: 129 pgs: 129 active+clean; 663 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 170 B/s wr, 8 op/s 2024-09-11T00:07:17.782 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:07:17 smithi165 ceph-mon[35356]: pgmap v404: 129 pgs: 129 active+clean; 664 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 1023 B/s wr, 14 op/s 2024-09-11T00:07:17.812 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:07:17 smithi012 ceph-mon[30333]: pgmap v404: 129 pgs: 129 active+clean; 664 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 1023 B/s wr, 14 op/s 2024-09-11T00:07:17.830 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:07:17 smithi178 ceph-mon[35919]: pgmap v404: 129 pgs: 129 active+clean; 664 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 1023 B/s wr, 14 op/s 2024-09-11T00:07:19.782 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:07:19 smithi165 ceph-mon[35356]: pgmap v405: 129 pgs: 129 active+clean; 664 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 938 B/s wr, 10 op/s 2024-09-11T00:07:19.812 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:07:19 smithi012 ceph-mon[30333]: pgmap v405: 129 pgs: 129 active+clean; 664 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 938 B/s wr, 10 op/s 2024-09-11T00:07:19.830 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:07:19 smithi178 ceph-mon[35919]: pgmap v405: 129 pgs: 129 active+clean; 664 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 938 B/s wr, 10 op/s 2024-09-11T00:07:21.813 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:07:21 smithi012 ceph-mon[30333]: pgmap v406: 129 pgs: 129 active+clean; 664 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 938 B/s wr, 12 op/s 2024-09-11T00:07:21.830 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:07:21 smithi178 ceph-mon[35919]: pgmap v406: 129 pgs: 129 active+clean; 664 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 938 B/s wr, 12 op/s 2024-09-11T00:07:22.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:07:21 smithi165 ceph-mon[35356]: pgmap v406: 129 pgs: 129 active+clean; 664 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 938 B/s wr, 12 op/s 2024-09-11T00:07:23.813 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:07:23 smithi012 ceph-mon[30333]: pgmap v407: 129 pgs: 129 active+clean; 664 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 938 B/s wr, 13 op/s 2024-09-11T00:07:23.831 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:07:23 smithi178 ceph-mon[35919]: pgmap v407: 129 pgs: 129 active+clean; 664 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 938 B/s wr, 13 op/s 2024-09-11T00:07:24.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:07:23 smithi165 ceph-mon[35356]: pgmap v407: 129 pgs: 129 active+clean; 664 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 938 B/s wr, 13 op/s 2024-09-11T00:07:24.918 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:24.916+0000 7f760fe00640 1 -- 172.21.15.12:0/3995673936 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f761010a730 msgr2=0x7f761010cb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:07:24.919 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:24.916+0000 7f760fe00640 1 --2- 172.21.15.12:0/3995673936 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f761010a730 0x7f761010cb60 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f7608009ca0 tx=0x7f760802a8c0 comp rx=0 tx=0).stop 2024-09-11T00:07:24.919 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:24.916+0000 7f760fe00640 1 -- 172.21.15.12:0/3995673936 shutdown_connections 2024-09-11T00:07:24.919 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:24.916+0000 7f760fe00640 1 --2- 172.21.15.12:0/3995673936 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f761010a730 0x7f761010cb60 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:07:24.919 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:24.916+0000 7f760fe00640 1 --2- 172.21.15.12:0/3995673936 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7610075aa0 0x7f7610075f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:07:24.919 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:24.916+0000 7f760fe00640 1 --2- 172.21.15.12:0/3995673936 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7610076f10 0x7f7610075560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:07:24.919 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:24.916+0000 7f760fe00640 1 -- 172.21.15.12:0/3995673936 >> 172.21.15.12:0/3995673936 conn(0x7f76100fe030 msgr2=0x7f7610100470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:07:24.919 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:24.916+0000 7f760fe00640 1 -- 172.21.15.12:0/3995673936 shutdown_connections 2024-09-11T00:07:24.919 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:24.916+0000 7f760fe00640 1 -- 172.21.15.12:0/3995673936 wait complete. 2024-09-11T00:07:24.920 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:24.917+0000 7f760fe00640 1 Processor -- start 2024-09-11T00:07:24.920 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:24.918+0000 7f760fe00640 1 -- start start 2024-09-11T00:07:24.921 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:24.918+0000 7f760fe00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7610075aa0 0x7f761019ad60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:07:24.921 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:24.918+0000 7f760fe00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7610076f10 0x7f761019b3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:07:24.921 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:24.918+0000 7f760fe00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f761010a730 0x7f761019bfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:07:24.921 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:24.918+0000 7f760fe00640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f7610110640 con 0x7f7610075aa0 2024-09-11T00:07:24.921 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:24.918+0000 7f760fe00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f76101104c0 con 0x7f7610076f10 2024-09-11T00:07:24.921 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:24.918+0000 7f760fe00640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f76101107c0 con 0x7f761010a730 2024-09-11T00:07:24.921 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:24.918+0000 7f760ea00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7610075aa0 0x7f761019ad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:07:24.921 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:24.918+0000 7f760e000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7610076f10 0x7f761019b3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:07:24.921 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:24.918+0000 7f760e000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7610076f10 0x7f761019b3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.12:44134/0 (socket says 172.21.15.12:44134) 2024-09-11T00:07:24.921 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:24.918+0000 7f760e000640 1 -- 172.21.15.12:0/3763967610 learned_addr learned my addr 172.21.15.12:0/3763967610 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:07:24.921 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:24.918+0000 7f760ea00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7610075aa0 0x7f761019ad60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:36378/0 (socket says 172.21.15.12:36378) 2024-09-11T00:07:24.921 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:24.918+0000 7f760f400640 1 --2- 172.21.15.12:0/3763967610 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f761010a730 0x7f761019bfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:07:24.922 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:24.919+0000 7f760ea00640 1 -- 172.21.15.12:0/3763967610 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f761010a730 msgr2=0x7f761019bfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:07:24.922 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:24.919+0000 7f760ea00640 1 --2- 172.21.15.12:0/3763967610 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f761010a730 0x7f761019bfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:07:24.923 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:24.919+0000 7f760ea00640 1 -- 172.21.15.12:0/3763967610 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7610076f10 msgr2=0x7f761019b3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:07:24.923 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:24.919+0000 7f760ea00640 1 --2- 172.21.15.12:0/3763967610 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7610076f10 0x7f761019b3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:07:24.923 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:24.919+0000 7f760ea00640 1 -- 172.21.15.12:0/3763967610 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f761019c8a0 con 0x7f7610075aa0 2024-09-11T00:07:24.923 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:24.919+0000 7f760ea00640 1 --2- 172.21.15.12:0/3763967610 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7610075aa0 0x7f761019ad60 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f760000e9f0 tx=0x7f760000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:07:24.923 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:24.920+0000 7f7607400640 1 -- 172.21.15.12:0/3763967610 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f760000ccc0 con 0x7f7610075aa0 2024-09-11T00:07:24.923 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:24.920+0000 7f760fe00640 1 -- 172.21.15.12:0/3763967610 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f761019cb90 con 0x7f7610075aa0 2024-09-11T00:07:24.923 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:24.920+0000 7f7607400640 1 -- 172.21.15.12:0/3763967610 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7600005020 con 0x7f7610075aa0 2024-09-11T00:07:24.923 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:24.920+0000 7f7607400640 1 -- 172.21.15.12:0/3763967610 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7600004a00 con 0x7f7610075aa0 2024-09-11T00:07:24.923 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:24.920+0000 7f760fe00640 1 -- 172.21.15.12:0/3763967610 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76101a3260 con 0x7f7610075aa0 2024-09-11T00:07:24.926 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:24.923+0000 7f7618605740 1 Processor -- start 2024-09-11T00:07:24.926 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:24.924+0000 7f7618605740 1 -- start start 2024-09-11T00:07:24.927 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:24.924+0000 7f7618605740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55df35972970 0x55df35974d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:07:24.927 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:24.924+0000 7f7618605740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55df359752a0 0x55df35977690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:07:24.927 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:24.924+0000 7f7618605740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55df35977ce0 0x55df3597a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:07:24.927 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:24.924+0000 7f7618605740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55df358e56b0 con 0x55df35972970 2024-09-11T00:07:24.927 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:24.924+0000 7f7618605740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55df358d4ae0 con 0x55df35977ce0 2024-09-11T00:07:24.927 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:24.924+0000 7f7618605740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55df358d88f0 con 0x55df359752a0 2024-09-11T00:07:24.927 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:24.924+0000 7f760f400640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55df35972970 0x55df35974d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:07:24.927 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:24.924+0000 7f760f400640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55df35972970 0x55df35974d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:36388/0 (socket says 172.21.15.12:36388) 2024-09-11T00:07:24.927 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:24.924+0000 7f760f400640 1 -- 172.21.15.12:0/2974555378 learned_addr learned my addr 172.21.15.12:0/2974555378 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:07:24.927 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:24.925+0000 7f760e000640 1 --2- 172.21.15.12:0/2974555378 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55df35977ce0 0x55df3597a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:07:24.927 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:24.925+0000 7f760f400640 1 -- 172.21.15.12:0/2974555378 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55df359752a0 msgr2=0x55df35977690 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-11T00:07:24.927 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:24.925+0000 7f760f400640 1 --2- 172.21.15.12:0/2974555378 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55df359752a0 0x55df35977690 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:07:24.928 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:24.925+0000 7f760f400640 1 -- 172.21.15.12:0/2974555378 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55df35977ce0 msgr2=0x55df3597a1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:07:24.928 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:24.925+0000 7f760f400640 1 --2- 172.21.15.12:0/2974555378 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55df35977ce0 0x55df3597a1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:07:24.928 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:24.925+0000 7f760f400640 1 -- 172.21.15.12:0/2974555378 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55df358a1780 con 0x55df35972970 2024-09-11T00:07:24.928 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:24.925+0000 7f7607400640 1 -- 172.21.15.12:0/3763967610 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7f7600004ba0 con 0x7f7610075aa0 2024-09-11T00:07:24.928 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:24.925+0000 7f760f400640 1 --2- 172.21.15.12:0/2974555378 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55df35972970 0x55df35974d60 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f7608009ca0 tx=0x7f76080093c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5b4e2ee566d1024 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:07:24.930 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:24.926+0000 7f75fbe00640 1 -- 172.21.15.12:0/2974555378 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7608031650 con 0x55df35972970 2024-09-11T00:07:24.930 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:24.926+0000 7f75fbe00640 1 -- 172.21.15.12:0/2974555378 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7608036030 con 0x55df35972970 2024-09-11T00:07:24.930 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:24.926+0000 7f75fbe00640 1 -- 172.21.15.12:0/2974555378 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7608004620 con 0x55df35972970 2024-09-11T00:07:24.930 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:24.927+0000 7f7618605740 1 -- 172.21.15.12:0/2974555378 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55df35972970 msgr2=0x55df35974d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:07:24.930 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:24.927+0000 7f7618605740 1 --2- 172.21.15.12:0/2974555378 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55df35972970 0x55df35974d60 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f7608009ca0 tx=0x7f76080093c0 comp rx=0 tx=0).stop 2024-09-11T00:07:24.930 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:24.927+0000 7f7618605740 1 -- 172.21.15.12:0/2974555378 shutdown_connections 2024-09-11T00:07:24.930 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:24.927+0000 7f7618605740 1 --2- 172.21.15.12:0/2974555378 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55df35977ce0 0x55df3597a1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:07:24.930 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:24.927+0000 7f7618605740 1 --2- 172.21.15.12:0/2974555378 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55df359752a0 0x55df35977690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:07:24.931 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:24.927+0000 7f7618605740 1 --2- 172.21.15.12:0/2974555378 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55df35972970 0x55df35974d60 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:07:24.931 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:24.927+0000 7f7618605740 1 -- 172.21.15.12:0/2974555378 >> 172.21.15.12:0/2974555378 conn(0x55df3596dac0 msgr2=0x55df3596e850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:07:24.931 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:24.928+0000 7f7618605740 1 -- 172.21.15.12:0/2974555378 shutdown_connections 2024-09-11T00:07:24.931 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:24.928+0000 7f7618605740 1 -- 172.21.15.12:0/2974555378 wait complete. 2024-09-11T00:07:24.931 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:24.928+0000 7f7607400640 1 --2- 172.21.15.12:0/3763967610 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f75f40b51f0 0x7f75f40b76b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:07:24.932 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:24.929+0000 7f7618605740 1 -- start start 2024-09-11T00:07:24.932 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:24.929+0000 7f7607400640 1 -- 172.21.15.12:0/3763967610 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9208+0+0 (secure 0 0 0) 0x7f76000e5fb0 con 0x7f7610075aa0 2024-09-11T00:07:24.932 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:24.929+0000 7f7618605740 1 Processor -- start 2024-09-11T00:07:24.932 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:24.929+0000 7f760e000640 1 --2- 172.21.15.12:0/3763967610 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f75f40b51f0 0x7f75f40b76b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:07:24.932 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:24.930+0000 7f7618605740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55df35972970 0x55df35984750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:07:24.932 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:24.930+0000 7f7618605740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55df359752a0 0x55df35984df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:07:24.932 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:24.930+0000 7f7618605740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55df35977ce0 0x55df359859c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:07:24.932 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:24.930+0000 7f7618605740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55df3580cab0 con 0x55df35977ce0 2024-09-11T00:07:24.933 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:24.930+0000 7f7618605740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55df3587b570 con 0x55df359752a0 2024-09-11T00:07:24.933 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:24.930+0000 7f7618605740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55df3597aae0 con 0x55df35972970 2024-09-11T00:07:24.933 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:24.930+0000 7f7618605740 10 client.0 _ll_register_callbacks cb 0x55df3588a100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:07:24.933 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:24.930+0000 7f760ea00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55df35972970 0x55df35984750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:07:24.933 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:24.930+0000 7f760ea00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55df35972970 0x55df35984750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.12:47146/0 (socket says 172.21.15.12:47146) 2024-09-11T00:07:24.933 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:24.930+0000 7f760ea00640 1 -- 172.21.15.12:0/3154654140 learned_addr learned my addr 172.21.15.12:0/3154654140 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:07:24.935 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:24.930+0000 7f760f400640 1 --2- 172.21.15.12:0/3154654140 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55df35977ce0 0x55df359859c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:07:24.935 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:24.930+0000 7f760ea00640 1 -- 172.21.15.12:0/3154654140 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55df359752a0 msgr2=0x55df35984df0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-11T00:07:24.935 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:24.930+0000 7f760ea00640 1 --2- 172.21.15.12:0/3154654140 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55df359752a0 0x55df35984df0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:07:24.935 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:24.930+0000 7f760ea00640 1 -- 172.21.15.12:0/3154654140 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55df35977ce0 msgr2=0x55df359859c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:07:24.935 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:24.930+0000 7f760ea00640 1 --2- 172.21.15.12:0/3154654140 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55df35977ce0 0x55df359859c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:07:24.935 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:24.930+0000 7f760ea00640 1 -- 172.21.15.12:0/3154654140 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55df35636790 con 0x55df35972970 2024-09-11T00:07:24.935 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:24.931+0000 7f760e000640 1 --2- 172.21.15.12:0/3763967610 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f75f40b51f0 0x7f75f40b76b0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f75fc009d20 tx=0x7f75fc005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:07:24.935 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:24.931+0000 7f760ea00640 1 --2- 172.21.15.12:0/3154654140 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55df35972970 0x55df35984750 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f76000a95b0 tx=0x7f76000abab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=aa0cc755b1b51ad5 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:07:24.935 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:24.931+0000 7f75f8c00640 10 client.0 ms_handle_connect on v2:172.21.15.178:3300/0 2024-09-11T00:07:24.935 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:24.931+0000 7f75f8c00640 1 -- 172.21.15.12:0/3154654140 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f760001c040 con 0x55df35972970 2024-09-11T00:07:24.935 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:24.931+0000 7f7618605740 1 -- 172.21.15.12:0/3154654140 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7608036030 con 0x55df35972970 2024-09-11T00:07:24.936 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:24.931+0000 7f75f8c00640 1 -- 172.21.15.12:0/3154654140 <== mon.1 v2:172.21.15.178:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 9208+0+0 (secure 0 0 0) 0x7f76000b5640 con 0x55df35972970 2024-09-11T00:07:24.936 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:24.931+0000 7f75f8c00640 1 -- 172.21.15.12:0/3154654140 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f76000aca40 con 0x55df35972970 2024-09-11T00:07:24.936 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:24.931+0000 7f7618605740 10 client.25516 fetch_fsmap learned FSMap version 29 2024-09-11T00:07:24.936 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:24.931+0000 7f7618605740 1 -- 172.21.15.12:0/3154654140 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55df356af620 con 0x55df35972970 2024-09-11T00:07:24.936 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:24.932+0000 7f75f8c00640 1 -- 172.21.15.12:0/3154654140 <== mon.1 v2:172.21.15.178:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7608036030 con 0x55df35972970 2024-09-11T00:07:24.936 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:24.932+0000 7f75f8c00640 1 -- 172.21.15.12:0/3154654140 <== mon.1 v2:172.21.15.178:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7f76000cf110 con 0x55df35972970 2024-09-11T00:07:24.936 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:24.932+0000 7f7618605740 10 client.25516 fetch_fsmap finished waiting for FSMap version 29 2024-09-11T00:07:24.936 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:24.932+0000 7f7618605740 10 client.25516 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T00:07:24.936 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:24.932+0000 7f7618605740 20 client.25516 populate_metadata read hostname 'smithi012' 2024-09-11T00:07:24.936 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:24.932+0000 7f7618605740 1 --2- 172.21.15.12:0/3154654140 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55df35990040 0x55df35992430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:07:24.936 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:24.932+0000 7f7618605740 4 client.25516 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T00:07:24.936 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:24.932+0000 7f7618605740 1 -- 172.21.15.12:0/3154654140 --> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55df358e7600 con 0x55df35990040 2024-09-11T00:07:24.936 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:24.933+0000 7f760e000640 1 --2- 172.21.15.12:0/3154654140 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55df35990040 0x55df35992430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:07:24.938 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:24.933+0000 7f760e000640 1 --2- 172.21.15.12:0/3154654140 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55df35990040 0x55df35992430 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=18c2afd0c83f2077 server_cookie=416ec67f1999a76d in_seq=0 out_seq=0 2024-09-11T00:07:24.938 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:24.933+0000 7f75f8c00640 10 client.25516 ms_handle_connect on v2:172.21.15.165:6838/501312984 2024-09-11T00:07:24.938 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:24.934+0000 7f75f8c00640 1 -- 172.21.15.12:0/3154654140 <== mds.0 v2:172.21.15.165:6838/501312984 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55df358e7600 con 0x55df35990040 2024-09-11T00:07:24.938 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:24.934+0000 7f75f8c00640 10 client.25516 handle_command_reply: tid=0 2024-09-11T00:07:24.938 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:24.934+0000 7f7618605740 1 client.25516 shutdown 2024-09-11T00:07:24.938 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:24.934+0000 7f7618605740 1 -- 172.21.15.12:0/3154654140 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55df35972970 msgr2=0x55df35984750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:07:24.938 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:24.934+0000 7f7618605740 1 --2- 172.21.15.12:0/3154654140 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55df35972970 0x55df35984750 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f76000a95b0 tx=0x7f76000abab0 comp rx=0 tx=0).stop 2024-09-11T00:07:24.938 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:24.934+0000 7f7618605740 1 -- 172.21.15.12:0/3154654140 shutdown_connections 2024-09-11T00:07:24.938 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:24.934+0000 7f7618605740 1 --2- 172.21.15.12:0/3154654140 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55df35990040 0x55df35992430 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:07:24.938 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:24.935+0000 7f7618605740 1 --2- 172.21.15.12:0/3154654140 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55df35977ce0 0x55df359859c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:07:24.938 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:24.935+0000 7f7618605740 1 --2- 172.21.15.12:0/3154654140 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55df359752a0 0x55df35984df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:07:24.939 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:24.935+0000 7f7618605740 1 --2- 172.21.15.12:0/3154654140 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55df35972970 0x55df35984750 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:07:24.939 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:24.935+0000 7f7618605740 1 -- 172.21.15.12:0/3154654140 >> 172.21.15.12:0/3154654140 conn(0x55df3596dac0 msgr2=0x55df3596cbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:07:24.939 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:24.935+0000 7f75f8c00640 0 client.25516 ms_handle_reset on v2:172.21.15.165:6838/501312984 2024-09-11T00:07:24.939 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:24.935+0000 7f7618605740 1 -- 172.21.15.12:0/3154654140 shutdown_connections 2024-09-11T00:07:24.939 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:24.935+0000 7f7618605740 1 -- 172.21.15.12:0/3154654140 wait complete. 2024-09-11T00:07:24.939 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:24.935+0000 7f7618605740 20 client.25516 trim_cache size 0 max 16384 2024-09-11T00:07:24.953 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:24.950+0000 7f7618605740 1 Processor -- start 2024-09-11T00:07:24.954 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:24.950+0000 7f7618605740 1 -- start start 2024-09-11T00:07:24.954 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:24.950+0000 7f7618605740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55df35972970 0x55df359787d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:07:24.954 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:24.950+0000 7f7618605740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55df359752a0 0x55df3598ca00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:07:24.954 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:24.950+0000 7f7618605740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55df35977ce0 0x55df35992050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:07:24.954 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:24.950+0000 7f7618605740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f76101a92c0 con 0x55df359752a0 2024-09-11T00:07:24.954 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:24.950+0000 7f7618605740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f76101aae00 con 0x55df35972970 2024-09-11T00:07:24.954 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:24.950+0000 7f7618605740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f76101a9990 con 0x55df35977ce0 2024-09-11T00:07:24.954 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:24.951+0000 7f760e000640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55df359752a0 0x55df3598ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:07:24.954 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:24.951+0000 7f760e000640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55df359752a0 0x55df3598ca00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:36408/0 (socket says 172.21.15.12:36408) 2024-09-11T00:07:24.954 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:24.951+0000 7f760e000640 1 -- 172.21.15.12:0/1817145196 learned_addr learned my addr 172.21.15.12:0/1817145196 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:07:24.954 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:24.951+0000 7f760ea00640 1 --2- 172.21.15.12:0/1817145196 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55df35972970 0x55df359787d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:07:24.954 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:24.951+0000 7f760f400640 1 --2- 172.21.15.12:0/1817145196 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55df35977ce0 0x55df35992050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:07:24.954 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:24.951+0000 7f760ea00640 1 -- 172.21.15.12:0/1817145196 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55df35977ce0 msgr2=0x55df35992050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:07:24.955 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:24.951+0000 7f760ea00640 1 --2- 172.21.15.12:0/1817145196 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55df35977ce0 0x55df35992050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:07:24.955 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:24.951+0000 7f760ea00640 1 -- 172.21.15.12:0/1817145196 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55df359752a0 msgr2=0x55df3598ca00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:07:24.955 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:24.951+0000 7f760ea00640 1 --2- 172.21.15.12:0/1817145196 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55df359752a0 0x55df3598ca00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:07:24.955 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:24.951+0000 7f760ea00640 1 -- 172.21.15.12:0/1817145196 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55df359750d0 con 0x55df35972970 2024-09-11T00:07:24.955 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:24.952+0000 7f760ea00640 1 --2- 172.21.15.12:0/1817145196 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55df35972970 0x55df359787d0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f76000a88c0 tx=0x7f76000abbe0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1712dbc7d1eb3922 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:07:24.955 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:24.952+0000 7f75fb400640 1 -- 172.21.15.12:0/1817145196 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f760001c070 con 0x55df35972970 2024-09-11T00:07:24.955 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:24.952+0000 7f75fb400640 1 -- 172.21.15.12:0/1817145196 <== mon.2 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f76000b58d0 con 0x55df35972970 2024-09-11T00:07:24.955 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:24.952+0000 7f75fb400640 1 -- 172.21.15.12:0/1817145196 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76000b56a0 con 0x55df35972970 2024-09-11T00:07:24.957 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:24.953+0000 7f7618605740 1 -- 172.21.15.12:0/1817145196 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55df35972970 msgr2=0x55df359787d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:07:24.958 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:24.953+0000 7f7618605740 1 --2- 172.21.15.12:0/1817145196 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55df35972970 0x55df359787d0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f76000a88c0 tx=0x7f76000abbe0 comp rx=0 tx=0).stop 2024-09-11T00:07:24.958 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:24.953+0000 7f7618605740 1 -- 172.21.15.12:0/1817145196 shutdown_connections 2024-09-11T00:07:24.958 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:24.953+0000 7f7618605740 1 --2- 172.21.15.12:0/1817145196 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55df35977ce0 0x55df35992050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:07:24.958 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:24.953+0000 7f7618605740 1 --2- 172.21.15.12:0/1817145196 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55df359752a0 0x55df3598ca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:07:24.958 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:24.953+0000 7f7618605740 1 --2- 172.21.15.12:0/1817145196 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55df35972970 0x55df359787d0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:07:24.958 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:24.953+0000 7f7618605740 1 -- 172.21.15.12:0/1817145196 >> 172.21.15.12:0/1817145196 conn(0x55df3596dac0 msgr2=0x55df359780c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:07:24.958 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:24.953+0000 7f7618605740 1 -- 172.21.15.12:0/1817145196 shutdown_connections 2024-09-11T00:07:24.958 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:24.953+0000 7f7618605740 1 -- 172.21.15.12:0/1817145196 wait complete. 2024-09-11T00:07:24.958 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:24.954+0000 7f7618605740 1 -- start start 2024-09-11T00:07:24.958 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:24.954+0000 7f7618605740 1 Processor -- start 2024-09-11T00:07:24.958 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:24.954+0000 7f7618605740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55df35972970 0x55df35989bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:07:24.959 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:24.954+0000 7f7618605740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55df359752a0 0x55df3598a260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:07:24.959 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:24.954+0000 7f7618605740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55df35977ce0 0x55df3598ade0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:07:24.959 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:24.954+0000 7f7618605740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f76101aa730 con 0x55df35977ce0 2024-09-11T00:07:24.959 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:24.954+0000 7f7618605740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f76101aa060 con 0x55df35972970 2024-09-11T00:07:24.959 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:24.954+0000 7f7618605740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f76101a72a0 con 0x55df359752a0 2024-09-11T00:07:24.959 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:24.954+0000 7f7618605740 10 client.0 _ll_register_callbacks cb 0x7f76000ac110 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:07:24.959 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:24.955+0000 7f760ea00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55df35972970 0x55df35989bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:07:24.959 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:24.955+0000 7f760ea00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55df35972970 0x55df35989bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.12:44156/0 (socket says 172.21.15.12:44156) 2024-09-11T00:07:24.959 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:24.955+0000 7f760ea00640 1 -- 172.21.15.12:0/3790533848 learned_addr learned my addr 172.21.15.12:0/3790533848 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:07:24.959 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:24.955+0000 7f760e000640 1 --2- 172.21.15.12:0/3790533848 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55df359752a0 0x55df3598a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:07:24.959 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:24.955+0000 7f760f400640 1 --2- 172.21.15.12:0/3790533848 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55df35977ce0 0x55df3598ade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:07:24.959 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:24.955+0000 7f760ea00640 1 -- 172.21.15.12:0/3790533848 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55df359752a0 msgr2=0x55df3598a260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:07:24.959 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:24.955+0000 7f760ea00640 1 --2- 172.21.15.12:0/3790533848 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55df359752a0 0x55df3598a260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:07:24.960 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:24.955+0000 7f760ea00640 1 -- 172.21.15.12:0/3790533848 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55df35977ce0 msgr2=0x55df3598ade0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:07:24.960 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:24.955+0000 7f760ea00640 1 --2- 172.21.15.12:0/3790533848 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55df35977ce0 0x55df3598ade0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:07:24.960 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:24.955+0000 7f760ea00640 1 -- 172.21.15.12:0/3790533848 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55df35985d30 con 0x55df35972970 2024-09-11T00:07:24.960 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:24.955+0000 7f760e000640 1 --2- 172.21.15.12:0/3790533848 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55df359752a0 0x55df3598a260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:07:24.960 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:24.955+0000 7f760f400640 1 --2- 172.21.15.12:0/3790533848 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55df35977ce0 0x55df3598ade0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T00:07:24.960 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:24.955+0000 7f760ea00640 1 --2- 172.21.15.12:0/3790533848 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55df35972970 0x55df35989bc0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f76000b5720 tx=0x7f76000b5750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6fa4ce071b12ddc9 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:07:24.960 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:24.955+0000 7f75f8c00640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-09-11T00:07:24.960 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:24.956+0000 7f75f8c00640 1 -- 172.21.15.12:0/3790533848 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f760001c070 con 0x55df35972970 2024-09-11T00:07:24.960 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:24.956+0000 7f7618605740 1 -- 172.21.15.12:0/3790533848 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f76000b58d0 con 0x55df35972970 2024-09-11T00:07:24.960 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:24.956+0000 7f75f8c00640 1 -- 172.21.15.12:0/3790533848 <== mon.2 v2:172.21.15.165:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 9208+0+0 (secure 0 0 0) 0x7f76000a7c90 con 0x55df35972970 2024-09-11T00:07:24.960 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:24.956+0000 7f75f8c00640 1 -- 172.21.15.12:0/3790533848 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f76000b5210 con 0x55df35972970 2024-09-11T00:07:24.960 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:24.956+0000 7f7618605740 10 client.25541 fetch_fsmap learned FSMap version 29 2024-09-11T00:07:24.960 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:24.956+0000 7f7618605740 1 -- 172.21.15.12:0/3790533848 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55df35985d30 con 0x55df35972970 2024-09-11T00:07:24.961 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:24.956+0000 7f75f8c00640 1 -- 172.21.15.12:0/3790533848 <== mon.2 v2:172.21.15.165:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f76000b58d0 con 0x55df35972970 2024-09-11T00:07:24.961 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:24.956+0000 7f75f8c00640 1 -- 172.21.15.12:0/3790533848 <== mon.2 v2:172.21.15.165:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7f76000ab080 con 0x55df35972970 2024-09-11T00:07:24.962 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:24.956+0000 7f7618605740 10 client.25541 fetch_fsmap finished waiting for FSMap version 29 2024-09-11T00:07:24.962 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:24.956+0000 7f7618605740 10 client.25541 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T00:07:24.962 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:24.957+0000 7f7618605740 20 client.25541 populate_metadata read hostname 'smithi012' 2024-09-11T00:07:24.962 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:24.957+0000 7f7618605740 1 --2- 172.21.15.12:0/3790533848 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55df359fced0 0x55df359ff2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:07:24.962 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:24.957+0000 7f7618605740 4 client.25541 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T00:07:24.962 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:24.957+0000 7f7618605740 1 -- 172.21.15.12:0/3790533848 --> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55df358e30b0 con 0x55df359fced0 2024-09-11T00:07:24.962 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:24.957+0000 7f760e000640 1 --2- 172.21.15.12:0/3790533848 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55df359fced0 0x55df359ff2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:07:24.962 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:24.958+0000 7f760e000640 1 --2- 172.21.15.12:0/3790533848 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55df359fced0 0x55df359ff2c0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6d839c3280fab6ad server_cookie=8fb44828cb0c35cd in_seq=0 out_seq=0 2024-09-11T00:07:24.962 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:24.958+0000 7f75f8c00640 10 client.25541 ms_handle_connect on v2:172.21.15.165:6838/501312984 2024-09-11T00:07:24.963 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:24.961+0000 7f75f8c00640 1 -- 172.21.15.12:0/3790533848 <== mds.0 v2:172.21.15.165:6838/501312984 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55df358e30b0 con 0x55df359fced0 2024-09-11T00:07:24.963 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:24.961+0000 7f75f8c00640 10 client.25541 handle_command_reply: tid=0 2024-09-11T00:07:24.963 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:24.961+0000 7f7618605740 1 client.25541 shutdown 2024-09-11T00:07:24.964 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:24.962+0000 7f7618605740 1 -- 172.21.15.12:0/3790533848 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55df35972970 msgr2=0x55df35989bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:07:24.964 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:24.962+0000 7f7618605740 1 --2- 172.21.15.12:0/3790533848 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55df35972970 0x55df35989bc0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f76000b5720 tx=0x7f76000b5750 comp rx=0 tx=0).stop 2024-09-11T00:07:24.964 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:24.962+0000 7f7618605740 1 -- 172.21.15.12:0/3790533848 shutdown_connections 2024-09-11T00:07:24.964 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:24.962+0000 7f7618605740 1 --2- 172.21.15.12:0/3790533848 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55df359fced0 0x55df359ff2c0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:07:24.964 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:24.962+0000 7f7618605740 1 --2- 172.21.15.12:0/3790533848 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55df35977ce0 0x55df3598ade0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:07:24.964 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:24.962+0000 7f7618605740 1 --2- 172.21.15.12:0/3790533848 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55df359752a0 0x55df3598a260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:07:24.965 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:24.962+0000 7f7618605740 1 --2- 172.21.15.12:0/3790533848 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55df35972970 0x55df35989bc0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:07:24.965 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:24.962+0000 7f7618605740 1 -- 172.21.15.12:0/3790533848 >> 172.21.15.12:0/3790533848 conn(0x55df3596dac0 msgr2=0x55df3596e320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:07:24.965 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:24.962+0000 7f75f8c00640 0 client.25541 ms_handle_reset on v2:172.21.15.165:6838/501312984 2024-09-11T00:07:24.965 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:24.962+0000 7f7618605740 1 -- 172.21.15.12:0/3790533848 shutdown_connections 2024-09-11T00:07:24.965 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:24.962+0000 7f7618605740 1 -- 172.21.15.12:0/3790533848 wait complete. 2024-09-11T00:07:24.965 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:24.962+0000 7f7618605740 20 client.25541 trim_cache size 0 max 16384 2024-09-11T00:07:24.965 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:{ 2024-09-11T00:07:24.965 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "message": "", 2024-09-11T00:07:24.965 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "return_code": 0 2024-09-11T00:07:24.966 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:} 2024-09-11T00:07:24.966 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:24.963+0000 7f760fe00640 1 -- 172.21.15.12:0/3763967610 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f75f40b51f0 msgr2=0x7f75f40b76b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:07:24.966 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:24.963+0000 7f760fe00640 1 --2- 172.21.15.12:0/3763967610 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f75f40b51f0 0x7f75f40b76b0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f75fc009d20 tx=0x7f75fc005f90 comp rx=0 tx=0).stop 2024-09-11T00:07:24.966 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:24.963+0000 7f760fe00640 1 -- 172.21.15.12:0/3763967610 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7610075aa0 msgr2=0x7f761019ad60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:07:24.966 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:24.963+0000 7f760fe00640 1 --2- 172.21.15.12:0/3763967610 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7610075aa0 0x7f761019ad60 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f760000e9f0 tx=0x7f760000eeb0 comp rx=0 tx=0).stop 2024-09-11T00:07:24.966 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:24.963+0000 7f760fe00640 1 -- 172.21.15.12:0/3763967610 shutdown_connections 2024-09-11T00:07:24.966 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:24.963+0000 7f760fe00640 1 --2- 172.21.15.12:0/3763967610 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f75f40b51f0 0x7f75f40b76b0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:07:24.967 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:24.963+0000 7f760fe00640 1 --2- 172.21.15.12:0/3763967610 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f761010a730 0x7f761019bfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:07:24.967 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:24.963+0000 7f760fe00640 1 --2- 172.21.15.12:0/3763967610 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7610076f10 0x7f761019b3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:07:24.967 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:24.963+0000 7f760fe00640 1 --2- 172.21.15.12:0/3763967610 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7610075aa0 0x7f761019ad60 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:07:24.967 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:24.963+0000 7f760fe00640 1 -- 172.21.15.12:0/3763967610 >> 172.21.15.12:0/3763967610 conn(0x7f76100fe030 msgr2=0x7f7610106710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:07:24.967 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:24.963+0000 7f760fe00640 1 -- 172.21.15.12:0/3763967610 shutdown_connections 2024-09-11T00:07:24.967 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:24.964+0000 7f760fe00640 1 -- 172.21.15.12:0/3763967610 wait complete. 2024-09-11T00:07:25.813 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:07:25 smithi012 ceph-mon[30333]: pgmap v408: 129 pgs: 129 active+clean; 664 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 938 B/s wr, 10 op/s 2024-09-11T00:07:25.831 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:07:25 smithi178 ceph-mon[35919]: pgmap v408: 129 pgs: 129 active+clean; 664 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 938 B/s wr, 10 op/s 2024-09-11T00:07:26.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:07:25 smithi165 ceph-mon[35356]: pgmap v408: 129 pgs: 129 active+clean; 664 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 938 B/s wr, 10 op/s 2024-09-11T00:07:26.058 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 25.2 2024-09-11T00:07:26.058 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid ff3ad74a-6fcf-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T00:07:26.630 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:07:27.295 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:27.292+0000 7feff7e00640 1 -- 172.21.15.12:0/766351871 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7feff8102c10 msgr2=0x7feff8102ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:07:27.296 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:27.292+0000 7feff7e00640 1 --2- 172.21.15.12:0/766351871 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7feff8102c10 0x7feff8102ff0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fefe8009900 tx=0x7fefe802a3f0 comp rx=0 tx=0).stop 2024-09-11T00:07:27.297 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:27.293+0000 7feff7e00640 1 -- 172.21.15.12:0/766351871 shutdown_connections 2024-09-11T00:07:27.297 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:27.293+0000 7feff7e00640 1 --2- 172.21.15.12:0/766351871 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7feff8108fa0 0x7feff810f7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:07:27.297 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:27.293+0000 7feff7e00640 1 --2- 172.21.15.12:0/766351871 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7feff8108620 0x7feff8108a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:07:27.297 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:27.293+0000 7feff7e00640 1 --2- 172.21.15.12:0/766351871 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7feff8102c10 0x7feff8102ff0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:07:27.297 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:27.293+0000 7feff7e00640 1 -- 172.21.15.12:0/766351871 >> 172.21.15.12:0/766351871 conn(0x7feff80fe660 msgr2=0x7feff8100a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:07:27.297 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:27.293+0000 7feff7e00640 1 -- 172.21.15.12:0/766351871 shutdown_connections 2024-09-11T00:07:27.297 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:27.293+0000 7feff7e00640 1 -- 172.21.15.12:0/766351871 wait complete. 2024-09-11T00:07:27.297 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:27.294+0000 7feff7e00640 1 Processor -- start 2024-09-11T00:07:27.297 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:27.295+0000 7feff7e00640 1 -- start start 2024-09-11T00:07:27.298 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:27.295+0000 7feff7e00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7feff8102c10 0x7feff8199040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:07:27.298 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:27.295+0000 7feff7e00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7feff8108620 0x7feff8199580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:07:27.298 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:27.295+0000 7feff7e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7feff8108fa0 0x7feff819e920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:07:27.298 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:27.295+0000 7feff7e00640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7feff8112cb0 con 0x7feff8102c10 2024-09-11T00:07:27.298 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:27.295+0000 7feff7e00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7feff8112b30 con 0x7feff8108fa0 2024-09-11T00:07:27.298 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:27.295+0000 7feff7e00640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7feff8112e30 con 0x7feff8108620 2024-09-11T00:07:27.299 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:27.296+0000 7feff6a00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7feff8102c10 0x7feff8199040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:07:27.299 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:27.296+0000 7feff6a00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7feff8102c10 0x7feff8199040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:36450/0 (socket says 172.21.15.12:36450) 2024-09-11T00:07:27.299 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:27.296+0000 7feff6a00640 1 -- 172.21.15.12:0/1339374672 learned_addr learned my addr 172.21.15.12:0/1339374672 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:07:27.299 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:27.296+0000 7feff6000640 1 --2- 172.21.15.12:0/1339374672 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7feff8108620 0x7feff8199580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:07:27.299 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:27.296+0000 7feff7400640 1 --2- 172.21.15.12:0/1339374672 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7feff8108fa0 0x7feff819e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:07:27.299 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:27.296+0000 7feff6a00640 1 -- 172.21.15.12:0/1339374672 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7feff8108620 msgr2=0x7feff8199580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:07:27.299 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:27.296+0000 7feff6a00640 1 --2- 172.21.15.12:0/1339374672 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7feff8108620 0x7feff8199580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:07:27.300 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:27.296+0000 7feff6a00640 1 -- 172.21.15.12:0/1339374672 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7feff8108fa0 msgr2=0x7feff819e920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:07:27.300 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:27.296+0000 7feff6a00640 1 --2- 172.21.15.12:0/1339374672 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7feff8108fa0 0x7feff819e920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:07:27.300 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:27.296+0000 7feff6a00640 1 -- 172.21.15.12:0/1339374672 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feff819f0f0 con 0x7feff8102c10 2024-09-11T00:07:27.300 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:27.296+0000 7feff7400640 1 --2- 172.21.15.12:0/1339374672 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7feff8108fa0 0x7feff819e920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:07:27.300 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:27.296+0000 7feff6000640 1 --2- 172.21.15.12:0/1339374672 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7feff8108620 0x7feff8199580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:07:27.301 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:27.297+0000 7feff6a00640 1 --2- 172.21.15.12:0/1339374672 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7feff8102c10 0x7feff8199040 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fefe8005ab0 tx=0x7fefe8003c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:07:27.301 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:27.297+0000 7fefef400640 1 -- 172.21.15.12:0/1339374672 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefe803f070 con 0x7feff8102c10 2024-09-11T00:07:27.301 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:27.297+0000 7fefef400640 1 -- 172.21.15.12:0/1339374672 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fefe8031d60 con 0x7feff8102c10 2024-09-11T00:07:27.302 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:27.297+0000 7fefef400640 1 -- 172.21.15.12:0/1339374672 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefe8035030 con 0x7feff8102c10 2024-09-11T00:07:27.302 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:27.297+0000 7feff7e00640 1 -- 172.21.15.12:0/1339374672 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feff819f380 con 0x7feff8102c10 2024-09-11T00:07:27.302 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:27.298+0000 7feff7e00640 1 -- 172.21.15.12:0/1339374672 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feff8075380 con 0x7feff8102c10 2024-09-11T00:07:27.303 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:27.301+0000 7feff7e00640 1 -- 172.21.15.12:0/1339374672 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feff8103ec0 con 0x7feff8102c10 2024-09-11T00:07:27.306 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:27.304+0000 7fefef400640 1 -- 172.21.15.12:0/1339374672 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7fefe803b460 con 0x7feff8102c10 2024-09-11T00:07:27.309 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:27.307+0000 7fefef400640 1 --2- 172.21.15.12:0/1339374672 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fefdc0b5390 0x7fefdc0b7850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:07:27.309 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:27.307+0000 7fefef400640 1 -- 172.21.15.12:0/1339374672 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9208+0+0 (secure 0 0 0) 0x7fefe8106620 con 0x7feff8102c10 2024-09-11T00:07:27.310 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:27.308+0000 7feff6000640 1 --2- 172.21.15.12:0/1339374672 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fefdc0b5390 0x7fefdc0b7850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:07:27.311 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:27.309+0000 7feff6000640 1 --2- 172.21.15.12:0/1339374672 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fefdc0b5390 0x7fefdc0b7850 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7feff8108150 tx=0x7fefe4005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:07:27.317 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:27.315+0000 7fefef400640 1 -- 172.21.15.12:0/1339374672 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7fefe80cca60 con 0x7feff8102c10 2024-09-11T00:07:27.546 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:07:27 smithi012 ceph-mon[30333]: pgmap v409: 129 pgs: 129 active+clean; 665 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 1.2 KiB/s wr, 14 op/s 2024-09-11T00:07:27.751 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:27.749+0000 7feff7e00640 1 -- 172.21.15.12:0/1339374672 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7feff80756a0 con 0x7feff8102c10 2024-09-11T00:07:27.752 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:27.750+0000 7fefef400640 1 -- 172.21.15.12:0/1339374672 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11965 (secure 0 0 0) 0x7fefe80d1490 con 0x7feff8102c10 2024-09-11T00:07:27.752 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-11T00:07:27.753 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 29 2024-09-11T00:07:27.753 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":29,"btime":"2024-09-11T00:06:45:273053+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14577,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6835/447171591","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":447171591},{"type":"v1","addr":"172.21.15.12:6835","nonce":447171591}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14589,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/901486040","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":901486040},{"type":"v1","addr":"172.21.15.12:6837","nonce":901486040}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T00:04:04.739881+0000","modified":"2024-09-11T00:06:44.753824+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24674,"mds_1":24643,"mds_2":24638,"mds_3":24637,"mds_4":24614},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14631":{"gid":14631,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.12:6841/3069919437","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6840","nonce":3069919437},{"type":"v1","addr":"172.21.15.12:6841","nonce":3069919437}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24562":{"gid":24562,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6833/1141636112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":1141636112},{"type":"v1","addr":"172.21.15.178:6833","nonce":1141636112}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24581":{"gid":24581,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6833/3111980970","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6832","nonce":3111980970},{"type":"v1","addr":"172.21.15.165:6833","nonce":3111980970}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24607":{"gid":24607,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6835/4084868287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":4084868287},{"type":"v1","addr":"172.21.15.178:6835","nonce":4084868287}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24613":{"gid":24613,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6837/526874763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":526874763},{"type":"v1","addr":"172.21.15.178:6837","nonce":526874763}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24614":{"gid":24614,"name":"e","rank":4,"incarnation":26,"state":"up:active","state_seq":21,"addr":"172.21.15.165:6835/2425395274","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":2425395274},{"type":"v1","addr":"172.21.15.165:6835","nonce":2425395274}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24637":{"gid":24637,"name":"i","rank":3,"incarnation":24,"state":"up:active","state_seq":18,"addr":"172.21.15.178:6839/70628097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6838","nonce":70628097},{"type":"v1","addr":"172.21.15.178:6839","nonce":70628097}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24638":{"gid":24638,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":18,"addr":"172.21.15.165:6837/6215088","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6836","nonce":6215088},{"type":"v1","addr":"172.21.15.165:6837","nonce":6215088}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24643":{"gid":24643,"name":"g","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.12:6839/3124093009","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6838","nonce":3124093009},{"type":"v1","addr":"172.21.15.12:6839","nonce":3124093009}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24674":{"gid":24674,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.165:6839/501312984","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6838","nonce":501312984},{"type":"v1","addr":"172.21.15.165:6839","nonce":501312984}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24674,"qdb_cluster":[24614,24637,24638,24643,24674]},"id":1}]} 2024-09-11T00:07:27.755 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:27.753+0000 7feff7e00640 1 -- 172.21.15.12:0/1339374672 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fefdc0b5390 msgr2=0x7fefdc0b7850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:07:27.755 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:27.753+0000 7feff7e00640 1 --2- 172.21.15.12:0/1339374672 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fefdc0b5390 0x7fefdc0b7850 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7feff8108150 tx=0x7fefe4005f90 comp rx=0 tx=0).stop 2024-09-11T00:07:27.755 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:27.753+0000 7feff7e00640 1 -- 172.21.15.12:0/1339374672 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7feff8102c10 msgr2=0x7feff8199040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:07:27.755 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:27.753+0000 7feff7e00640 1 --2- 172.21.15.12:0/1339374672 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7feff8102c10 0x7feff8199040 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fefe8005ab0 tx=0x7fefe8003c80 comp rx=0 tx=0).stop 2024-09-11T00:07:27.756 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:27.753+0000 7feff7e00640 1 -- 172.21.15.12:0/1339374672 shutdown_connections 2024-09-11T00:07:27.756 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:27.753+0000 7feff7e00640 1 --2- 172.21.15.12:0/1339374672 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fefdc0b5390 0x7fefdc0b7850 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:07:27.756 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:27.753+0000 7feff7e00640 1 --2- 172.21.15.12:0/1339374672 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7feff8108fa0 0x7feff819e920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:07:27.756 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:27.753+0000 7feff7e00640 1 --2- 172.21.15.12:0/1339374672 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7feff8108620 0x7feff8199580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:07:27.756 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:27.753+0000 7feff7e00640 1 --2- 172.21.15.12:0/1339374672 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7feff8102c10 0x7feff8199040 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:07:27.756 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:27.753+0000 7feff7e00640 1 -- 172.21.15.12:0/1339374672 >> 172.21.15.12:0/1339374672 conn(0x7feff80fe660 msgr2=0x7feff8100a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:07:27.756 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:27.754+0000 7feff7e00640 1 -- 172.21.15.12:0/1339374672 shutdown_connections 2024-09-11T00:07:27.756 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:27.754+0000 7feff7e00640 1 -- 172.21.15.12:0/1339374672 wait complete. 2024-09-11T00:07:27.831 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:07:27 smithi178 ceph-mon[35919]: pgmap v409: 129 pgs: 129 active+clean; 665 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 1.2 KiB/s wr, 14 op/s 2024-09-11T00:07:28.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:07:27 smithi165 ceph-mon[35356]: pgmap v409: 129 pgs: 129 active+clean; 665 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 1.2 KiB/s wr, 14 op/s 2024-09-11T00:07:28.636 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24674 quiesce db / --timeout 90 --expiration 145.2 --await-for 45.0' 2024-09-11T00:07:28.636 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid ff3ad74a-6fcf-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24674 quiesce db / --timeout 90 --expiration 145.2 --await-for 45.0 2024-09-11T00:07:28.813 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:07:28 smithi012 ceph-mon[30333]: from='client.? 172.21.15.12:0/1339374672' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T00:07:28.830 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:07:28 smithi178 ceph-mon[35919]: from='client.? 172.21.15.12:0/1339374672' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T00:07:29.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:07:28 smithi165 ceph-mon[35356]: from='client.? 172.21.15.12:0/1339374672' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T00:07:29.104 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:07:29.813 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:07:29 smithi012 ceph-mon[30333]: pgmap v410: 129 pgs: 129 active+clean; 665 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 341 B/s wr, 8 op/s 2024-09-11T00:07:29.830 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:07:29 smithi178 ceph-mon[35919]: pgmap v410: 129 pgs: 129 active+clean; 665 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 341 B/s wr, 8 op/s 2024-09-11T00:07:29.854 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:29.852+0000 7f8ea3200640 1 -- 172.21.15.12:0/3408289308 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8ea410f200 msgr2=0x7f8ea41115f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:07:29.854 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:29.852+0000 7f8ea3200640 1 --2- 172.21.15.12:0/3408289308 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8ea410f200 0x7f8ea41115f0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f8e9c009be0 tx=0x7f8e9c02a6d0 comp rx=0 tx=0).stop 2024-09-11T00:07:29.854 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:29.852+0000 7f8ea3200640 1 -- 172.21.15.12:0/3408289308 shutdown_connections 2024-09-11T00:07:29.854 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:29.852+0000 7f8ea3200640 1 --2- 172.21.15.12:0/3408289308 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8ea410f200 0x7f8ea41115f0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:07:29.854 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:29.852+0000 7f8ea3200640 1 --2- 172.21.15.12:0/3408289308 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8ea4075e00 0x7f8ea410ecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:07:29.854 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:29.852+0000 7f8ea3200640 1 --2- 172.21.15.12:0/3408289308 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8ea40754e0 0x7f8ea40758c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:07:29.854 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:29.852+0000 7f8ea3200640 1 -- 172.21.15.12:0/3408289308 >> 172.21.15.12:0/3408289308 conn(0x7f8ea40fe660 msgr2=0x7f8ea4100a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:07:29.854 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:29.852+0000 7f8ea3200640 1 -- 172.21.15.12:0/3408289308 shutdown_connections 2024-09-11T00:07:29.854 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:29.852+0000 7f8ea3200640 1 -- 172.21.15.12:0/3408289308 wait complete. 2024-09-11T00:07:29.855 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:29.854+0000 7f8ea3200640 1 Processor -- start 2024-09-11T00:07:29.855 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:29.854+0000 7f8ea3200640 1 -- start start 2024-09-11T00:07:29.855 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:29.854+0000 7f8ea3200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8ea40754e0 0x7f8ea410e970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:07:29.855 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:29.854+0000 7f8ea3200640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8ea4075e00 0x7f8ea410bbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:07:29.856 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:29.855+0000 7f8ea3200640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8ea410f200 0x7f8ea410c100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:07:29.856 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:29.855+0000 7f8ea3200640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f8ea4114de0 con 0x7f8ea410f200 2024-09-11T00:07:29.856 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:29.855+0000 7f8ea3200640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f8ea4114c60 con 0x7f8ea40754e0 2024-09-11T00:07:29.856 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:29.855+0000 7f8ea3200640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f8ea4114f60 con 0x7f8ea4075e00 2024-09-11T00:07:29.857 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:29.855+0000 7f8ea1e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8ea40754e0 0x7f8ea410e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:07:29.857 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:29.855+0000 7f8ea1400640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8ea4075e00 0x7f8ea410bbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:07:29.857 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:29.855+0000 7f8ea1e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8ea40754e0 0x7f8ea410e970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.12:59966/0 (socket says 172.21.15.12:59966) 2024-09-11T00:07:29.857 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:29.855+0000 7f8ea1400640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8ea4075e00 0x7f8ea410bbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.12:58126/0 (socket says 172.21.15.12:58126) 2024-09-11T00:07:29.857 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:29.855+0000 7f8ea1e00640 1 -- 172.21.15.12:0/2564130263 learned_addr learned my addr 172.21.15.12:0/2564130263 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:07:29.857 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:29.855+0000 7f8ea2800640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8ea410f200 0x7f8ea410c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:07:29.858 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:29.857+0000 7f8ea1e00640 1 -- 172.21.15.12:0/2564130263 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8ea4075e00 msgr2=0x7f8ea410bbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:07:29.858 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:29.857+0000 7f8ea1e00640 1 --2- 172.21.15.12:0/2564130263 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8ea4075e00 0x7f8ea410bbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:07:29.858 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:29.857+0000 7f8ea1e00640 1 -- 172.21.15.12:0/2564130263 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8ea410f200 msgr2=0x7f8ea410c100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:07:29.858 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:29.857+0000 7f8ea1e00640 1 --2- 172.21.15.12:0/2564130263 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8ea410f200 0x7f8ea410c100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:07:29.858 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:29.857+0000 7f8ea1e00640 1 -- 172.21.15.12:0/2564130263 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ea410c900 con 0x7f8ea40754e0 2024-09-11T00:07:29.859 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:29.858+0000 7f8ea1e00640 1 --2- 172.21.15.12:0/2564130263 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8ea40754e0 0x7f8ea410e970 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f8e9400e9c0 tx=0x7f8e9400ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:07:29.859 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:29.858+0000 7f8e9aa00640 1 -- 172.21.15.12:0/2564130263 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e9400cde0 con 0x7f8ea40754e0 2024-09-11T00:07:29.860 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:29.858+0000 7f8e9aa00640 1 -- 172.21.15.12:0/2564130263 <== mon.2 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8e940035f0 con 0x7f8ea40754e0 2024-09-11T00:07:29.860 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:29.858+0000 7f8e9aa00640 1 -- 172.21.15.12:0/2564130263 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e94010690 con 0x7f8ea40754e0 2024-09-11T00:07:29.860 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:29.859+0000 7f8ea3200640 1 -- 172.21.15.12:0/2564130263 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ea41abb40 con 0x7f8ea40754e0 2024-09-11T00:07:29.860 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:29.859+0000 7f8ea3200640 1 -- 172.21.15.12:0/2564130263 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ea41ac000 con 0x7f8ea40754e0 2024-09-11T00:07:29.869 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:29.865+0000 7f8eab7c0740 1 Processor -- start 2024-09-11T00:07:29.870 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:29.865+0000 7f8eab7c0740 1 -- start start 2024-09-11T00:07:29.870 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:29.865+0000 7f8eab7c0740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5579091b22f0 0x5579091b26d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:07:29.870 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:29.865+0000 7f8eab7c0740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5579091b2c10 0x5579091cf670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:07:29.870 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:29.865+0000 7f8eab7c0740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5579091cfbb0 0x5579091d1fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:07:29.870 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:29.865+0000 7f8eab7c0740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x557909127ca0 con 0x5579091cfbb0 2024-09-11T00:07:29.870 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:29.865+0000 7f8eab7c0740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x5579091170d0 con 0x5579091b2c10 2024-09-11T00:07:29.870 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:29.865+0000 7f8eab7c0740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55790911aee0 con 0x5579091b22f0 2024-09-11T00:07:29.870 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:29.866+0000 7f8ea1400640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5579091cfbb0 0x5579091d1fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:07:29.870 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:29.866+0000 7f8ea1400640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5579091cfbb0 0x5579091d1fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:43238/0 (socket says 172.21.15.12:43238) 2024-09-11T00:07:29.870 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:29.866+0000 7f8ea1400640 1 -- 172.21.15.12:0/4273873558 learned_addr learned my addr 172.21.15.12:0/4273873558 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:07:29.870 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:29.866+0000 7f8ea2800640 1 --2- 172.21.15.12:0/4273873558 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5579091b22f0 0x5579091b26d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:07:29.870 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:29.866+0000 7f8ea1400640 1 -- 172.21.15.12:0/4273873558 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5579091b22f0 msgr2=0x5579091b26d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:07:29.871 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:29.866+0000 7f8ea1400640 1 --2- 172.21.15.12:0/4273873558 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5579091b22f0 0x5579091b26d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:07:29.871 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:29.866+0000 7f8ea1400640 1 -- 172.21.15.12:0/4273873558 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5579091b2c10 msgr2=0x5579091cf670 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-11T00:07:29.871 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:29.866+0000 7f8ea1400640 1 --2- 172.21.15.12:0/4273873558 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5579091b2c10 0x5579091cf670 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:07:29.871 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:29.866+0000 7f8ea1400640 1 -- 172.21.15.12:0/4273873558 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5579090c9400 con 0x5579091cfbb0 2024-09-11T00:07:29.871 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:29.867+0000 7f8ea1400640 1 --2- 172.21.15.12:0/4273873558 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5579091cfbb0 0x5579091d1fa0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f8e9000bce0 tx=0x7f8e900107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6379ed0b258cd476 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:07:29.871 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:29.868+0000 7f8e8f400640 1 -- 172.21.15.12:0/4273873558 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e90004030 con 0x5579091cfbb0 2024-09-11T00:07:29.871 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:29.868+0000 7f8e8f400640 1 -- 172.21.15.12:0/4273873558 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8e900035f0 con 0x5579091cfbb0 2024-09-11T00:07:29.871 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:29.868+0000 7f8e8f400640 1 -- 172.21.15.12:0/4273873558 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e90011720 con 0x5579091cfbb0 2024-09-11T00:07:29.871 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:29.868+0000 7f8e9aa00640 1 -- 172.21.15.12:0/2564130263 <== mon.2 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7f8e94004000 con 0x7f8ea40754e0 2024-09-11T00:07:29.871 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:29.869+0000 7f8eab7c0740 1 -- 172.21.15.12:0/4273873558 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5579091cfbb0 msgr2=0x5579091d1fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:07:29.872 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:29.869+0000 7f8eab7c0740 1 --2- 172.21.15.12:0/4273873558 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5579091cfbb0 0x5579091d1fa0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f8e9000bce0 tx=0x7f8e900107a0 comp rx=0 tx=0).stop 2024-09-11T00:07:29.872 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:29.869+0000 7f8eab7c0740 1 -- 172.21.15.12:0/4273873558 shutdown_connections 2024-09-11T00:07:29.872 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:29.869+0000 7f8eab7c0740 1 --2- 172.21.15.12:0/4273873558 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5579091cfbb0 0x5579091d1fa0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:07:29.872 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:29.869+0000 7f8eab7c0740 1 --2- 172.21.15.12:0/4273873558 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5579091b2c10 0x5579091cf670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:07:29.872 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:29.869+0000 7f8eab7c0740 1 --2- 172.21.15.12:0/4273873558 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5579091b22f0 0x5579091b26d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:07:29.872 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:29.869+0000 7f8eab7c0740 1 -- 172.21.15.12:0/4273873558 >> 172.21.15.12:0/4273873558 conn(0x5579091b0120 msgr2=0x5579091b12c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:07:29.872 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:29.870+0000 7f8eab7c0740 1 -- 172.21.15.12:0/4273873558 shutdown_connections 2024-09-11T00:07:29.872 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:29.871+0000 7f8eab7c0740 1 -- 172.21.15.12:0/4273873558 wait complete. 2024-09-11T00:07:29.873 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:29.872+0000 7f8e9aa00640 1 --2- 172.21.15.12:0/2564130263 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f8e880b53a0 0x7f8e880b7860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:07:29.874 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:29.872+0000 7f8eab7c0740 1 -- start start 2024-09-11T00:07:29.874 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:29.873+0000 7f8eab7c0740 1 Processor -- start 2024-09-11T00:07:29.875 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:29.873+0000 7f8e9aa00640 1 -- 172.21.15.12:0/2564130263 <== mon.2 v2:172.21.15.165:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9208+0+0 (secure 0 0 0) 0x7f8e940e62c0 con 0x7f8ea40754e0 2024-09-11T00:07:29.875 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:29.874+0000 7f8ea2800640 1 --2- 172.21.15.12:0/2564130263 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f8e880b53a0 0x7f8e880b7860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:07:29.876 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:29.875+0000 7f8eab7c0740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5579091b22f0 0x5579091cf4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:07:29.877 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:29.875+0000 7f8eab7c0740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5579091b2c10 0x5579091cc990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:07:29.877 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:29.875+0000 7f8eab7c0740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5579091cfbb0 0x5579091ccfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:07:29.877 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:29.875+0000 7f8eab7c0740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x557908f787a0 con 0x5579091b2c10 2024-09-11T00:07:29.877 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:29.875+0000 7f8eab7c0740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x5579090bca10 con 0x5579091b22f0 2024-09-11T00:07:29.877 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:29.875+0000 7f8eab7c0740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5579091d2860 con 0x5579091cfbb0 2024-09-11T00:07:29.877 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:29.875+0000 7f8eab7c0740 10 client.0 _ll_register_callbacks cb 0x5579090ccbe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:07:29.877 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:29.876+0000 7f8ea2800640 1 --2- 172.21.15.12:0/2564130263 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f8e880b53a0 0x7f8e880b7860 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f8e9c004af0 tx=0x7f8e9c004a40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:07:29.877 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:29.877+0000 7f8ea1400640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5579091b2c10 0x5579091cc990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:07:29.878 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:29.877+0000 7f8ea1400640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5579091b2c10 0x5579091cc990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:43248/0 (socket says 172.21.15.12:43248) 2024-09-11T00:07:29.878 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:29.877+0000 7f8ea1400640 1 -- 172.21.15.12:0/1554215250 learned_addr learned my addr 172.21.15.12:0/1554215250 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:07:29.879 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:29.877+0000 7f8ea2800640 1 --2- 172.21.15.12:0/1554215250 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5579091cfbb0 0x5579091ccfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:07:29.880 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:29.877+0000 7f8ea1400640 1 -- 172.21.15.12:0/1554215250 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5579091cfbb0 msgr2=0x5579091ccfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:07:29.880 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:29.877+0000 7f8ea1400640 1 --2- 172.21.15.12:0/1554215250 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5579091cfbb0 0x5579091ccfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:07:29.880 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:29.878+0000 7f8ea1400640 1 -- 172.21.15.12:0/1554215250 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5579091b22f0 msgr2=0x5579091cf4d0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-11T00:07:29.880 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:29.878+0000 7f8ea1400640 1 --2- 172.21.15.12:0/1554215250 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5579091b22f0 0x5579091cf4d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:07:29.880 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:29.878+0000 7f8ea1400640 1 -- 172.21.15.12:0/1554215250 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557908e84b90 con 0x5579091b2c10 2024-09-11T00:07:29.880 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:29.878+0000 7f8ea2800640 1 --2- 172.21.15.12:0/1554215250 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5579091cfbb0 0x5579091ccfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T00:07:29.880 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:29.878+0000 7f8ea1400640 1 --2- 172.21.15.12:0/1554215250 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5579091b2c10 0x5579091cc990 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f8e9000b740 tx=0x7f8e90004230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6937b121e9bf4e00 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:07:29.880 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:29.878+0000 7f8e83e00640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-09-11T00:07:29.881 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:29.879+0000 7f8e83e00640 1 -- 172.21.15.12:0/1554215250 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e90029060 con 0x5579091b2c10 2024-09-11T00:07:29.881 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:29.879+0000 7f8e83e00640 1 -- 172.21.15.12:0/1554215250 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 9208+0+0 (secure 0 0 0) 0x7f8e900037a0 con 0x5579091b2c10 2024-09-11T00:07:29.881 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:29.879+0000 7f8eab7c0740 1 -- 172.21.15.12:0/1554215250 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8e900035f0 con 0x5579091b2c10 2024-09-11T00:07:29.881 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:29.880+0000 7f8e83e00640 1 -- 172.21.15.12:0/1554215250 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f8e900039c0 con 0x5579091b2c10 2024-09-11T00:07:29.881 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:29.880+0000 7f8eab7c0740 10 client.15519 fetch_fsmap learned FSMap version 29 2024-09-11T00:07:29.882 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:29.880+0000 7f8eab7c0740 1 -- 172.21.15.12:0/1554215250 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5579091de270 con 0x5579091b2c10 2024-09-11T00:07:29.882 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:29.881+0000 7f8e83e00640 1 -- 172.21.15.12:0/1554215250 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8e900035f0 con 0x5579091b2c10 2024-09-11T00:07:29.882 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:29.881+0000 7f8e83e00640 1 -- 172.21.15.12:0/1554215250 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7f8e90023030 con 0x5579091b2c10 2024-09-11T00:07:29.883 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:29.882+0000 7f8eab7c0740 10 client.15519 fetch_fsmap finished waiting for FSMap version 29 2024-09-11T00:07:29.883 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:29.882+0000 7f8eab7c0740 10 client.15519 resolve_mds: validated gid 24674 aka daemon mds.b 2024-09-11T00:07:29.883 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:29.882+0000 7f8eab7c0740 20 client.15519 populate_metadata read hostname 'smithi012' 2024-09-11T00:07:29.884 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:29.882+0000 7f8eab7c0740 1 --2- 172.21.15.12:0/1554215250 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x5579091dfa40 0x5579091e1e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:07:29.884 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:29.882+0000 7f8eab7c0740 4 client.15519 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T00:07:29.884 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:29.882+0000 7f8eab7c0740 1 -- 172.21.15.12:0/1554215250 --> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557909129bf0 con 0x5579091dfa40 2024-09-11T00:07:29.884 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:29.883+0000 7f8ea1e00640 1 --2- 172.21.15.12:0/1554215250 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x5579091dfa40 0x5579091e1e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:07:29.885 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:29.884+0000 7f8ea1e00640 1 --2- 172.21.15.12:0/1554215250 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x5579091dfa40 0x5579091e1e60 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=969220784cdf7f9 server_cookie=64b3ffa4b7dac297 in_seq=0 out_seq=0 2024-09-11T00:07:29.885 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:29.884+0000 7f8e83e00640 10 client.15519 ms_handle_connect on v2:172.21.15.165:6838/501312984 2024-09-11T00:07:29.888 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:29.885+0000 7f8e83e00640 1 -- 172.21.15.12:0/1554215250 <== mds.0 v2:172.21.15.165:6838/501312984 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x557909129bf0 con 0x5579091dfa40 2024-09-11T00:07:29.888 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:29.885+0000 7f8e83e00640 10 client.15519 handle_command_reply: tid=0 2024-09-11T00:07:29.888 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:29.885+0000 7f8eab7c0740 1 client.15519 shutdown 2024-09-11T00:07:29.888 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:29.885+0000 7f8eab7c0740 1 -- 172.21.15.12:0/1554215250 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5579091b2c10 msgr2=0x5579091cc990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:07:29.888 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:29.885+0000 7f8eab7c0740 1 --2- 172.21.15.12:0/1554215250 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5579091b2c10 0x5579091cc990 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f8e9000b740 tx=0x7f8e90004230 comp rx=0 tx=0).stop 2024-09-11T00:07:29.888 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:29.886+0000 7f8eab7c0740 1 -- 172.21.15.12:0/1554215250 shutdown_connections 2024-09-11T00:07:29.888 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:29.886+0000 7f8eab7c0740 1 --2- 172.21.15.12:0/1554215250 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x5579091dfa40 0x5579091e1e60 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:07:29.888 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:29.886+0000 7f8eab7c0740 1 --2- 172.21.15.12:0/1554215250 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5579091cfbb0 0x5579091ccfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:07:29.888 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:29.886+0000 7f8eab7c0740 1 --2- 172.21.15.12:0/1554215250 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5579091b2c10 0x5579091cc990 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:07:29.888 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:29.886+0000 7f8e83e00640 0 client.15519 ms_handle_reset on v2:172.21.15.165:6838/501312984 2024-09-11T00:07:29.888 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:29.886+0000 7f8eab7c0740 1 --2- 172.21.15.12:0/1554215250 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5579091b22f0 0x5579091cf4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:07:29.888 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:29.886+0000 7f8eab7c0740 1 -- 172.21.15.12:0/1554215250 >> 172.21.15.12:0/1554215250 conn(0x5579091b0120 msgr2=0x5579091b5cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:07:29.889 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:29.886+0000 7f8eab7c0740 1 -- 172.21.15.12:0/1554215250 shutdown_connections 2024-09-11T00:07:29.889 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:29.886+0000 7f8eab7c0740 1 -- 172.21.15.12:0/1554215250 wait complete. 2024-09-11T00:07:29.889 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:29.887+0000 7f8eab7c0740 20 client.15519 trim_cache size 0 max 16384 2024-09-11T00:07:29.925 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:29.924+0000 7f8eab7c0740 1 Processor -- start 2024-09-11T00:07:29.926 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:29.925+0000 7f8eab7c0740 1 -- start start 2024-09-11T00:07:29.926 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:29.925+0000 7f8eab7c0740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5579091b22f0 0x5579091dbaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:07:29.927 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:29.925+0000 7f8eab7c0740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5579091cfbb0 0x5579091cb210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:07:29.927 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:29.925+0000 7f8eab7c0740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5579091ca640 0x5579091cb750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:07:29.927 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:29.925+0000 7f8eab7c0740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f8ea41ad910 con 0x5579091ca640 2024-09-11T00:07:29.927 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:29.925+0000 7f8eab7c0740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f8ea41af450 con 0x5579091b22f0 2024-09-11T00:07:29.927 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:29.925+0000 7f8eab7c0740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f8ea41adfe0 con 0x5579091cfbb0 2024-09-11T00:07:29.927 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:29.926+0000 7f8ea1e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5579091b22f0 0x5579091dbaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:07:29.927 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:29.926+0000 7f8ea1e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5579091b22f0 0x5579091dbaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.12:59982/0 (socket says 172.21.15.12:59982) 2024-09-11T00:07:29.927 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:29.926+0000 7f8ea1e00640 1 -- 172.21.15.12:0/484200603 learned_addr learned my addr 172.21.15.12:0/484200603 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:07:29.927 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:29.926+0000 7f8ea2800640 1 --2- 172.21.15.12:0/484200603 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5579091ca640 0x5579091cb750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:07:29.928 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:29.926+0000 7f8ea1400640 1 --2- 172.21.15.12:0/484200603 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5579091cfbb0 0x5579091cb210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:07:29.928 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:29.927+0000 7f8ea2800640 1 -- 172.21.15.12:0/484200603 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5579091cfbb0 msgr2=0x5579091cb210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:07:29.928 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:29.927+0000 7f8ea2800640 1 --2- 172.21.15.12:0/484200603 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5579091cfbb0 0x5579091cb210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:07:29.928 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:29.927+0000 7f8ea2800640 1 -- 172.21.15.12:0/484200603 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5579091b22f0 msgr2=0x5579091dbaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:07:29.928 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:29.927+0000 7f8ea2800640 1 --2- 172.21.15.12:0/484200603 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5579091b22f0 0x5579091dbaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:07:29.929 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:29.927+0000 7f8ea2800640 1 -- 172.21.15.12:0/484200603 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5579091d20d0 con 0x5579091ca640 2024-09-11T00:07:29.929 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:29.928+0000 7f8ea2800640 1 --2- 172.21.15.12:0/484200603 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5579091ca640 0x5579091cb750 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f8e9c034040 tx=0x7f8e9c03f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b0fa96556385270 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:07:29.929 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:29.928+0000 7f8e8ea00640 1 -- 172.21.15.12:0/484200603 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e9c04f070 con 0x5579091ca640 2024-09-11T00:07:29.929 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:29.928+0000 7f8e8ea00640 1 -- 172.21.15.12:0/484200603 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8e9c030cf0 con 0x5579091ca640 2024-09-11T00:07:29.929 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:29.928+0000 7f8e8ea00640 1 -- 172.21.15.12:0/484200603 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e9c036040 con 0x5579091ca640 2024-09-11T00:07:29.930 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:29.929+0000 7f8eab7c0740 1 -- 172.21.15.12:0/484200603 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5579091ca640 msgr2=0x5579091cb750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:07:29.930 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:29.929+0000 7f8eab7c0740 1 --2- 172.21.15.12:0/484200603 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5579091ca640 0x5579091cb750 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f8e9c034040 tx=0x7f8e9c03f000 comp rx=0 tx=0).stop 2024-09-11T00:07:29.931 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:29.930+0000 7f8eab7c0740 1 -- 172.21.15.12:0/484200603 shutdown_connections 2024-09-11T00:07:29.931 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:29.930+0000 7f8eab7c0740 1 --2- 172.21.15.12:0/484200603 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5579091ca640 0x5579091cb750 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:07:29.931 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:29.930+0000 7f8eab7c0740 1 --2- 172.21.15.12:0/484200603 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5579091cfbb0 0x5579091cb210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:07:29.931 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:29.930+0000 7f8eab7c0740 1 --2- 172.21.15.12:0/484200603 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5579091b22f0 0x5579091dbaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:07:29.932 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:29.930+0000 7f8eab7c0740 1 -- 172.21.15.12:0/484200603 >> 172.21.15.12:0/484200603 conn(0x5579091b0120 msgr2=0x5579091cc1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:07:29.932 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:29.931+0000 7f8eab7c0740 1 -- 172.21.15.12:0/484200603 shutdown_connections 2024-09-11T00:07:29.932 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:29.931+0000 7f8eab7c0740 1 -- 172.21.15.12:0/484200603 wait complete. 2024-09-11T00:07:29.933 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:29.932+0000 7f8eab7c0740 1 -- start start 2024-09-11T00:07:29.933 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:29.932+0000 7f8eab7c0740 1 Processor -- start 2024-09-11T00:07:29.934 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:29.933+0000 7f8eab7c0740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5579091b22f0 0x5579091d7b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:07:29.934 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:29.933+0000 7f8eab7c0740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5579091ca640 0x5579091d80d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:07:29.934 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:29.933+0000 7f8eab7c0740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5579091cfbb0 0x5579091d4b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:07:29.935 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:29.934+0000 7f8ea1400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5579091ca640 0x5579091d80d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:07:29.935 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:29.934+0000 7f8ea1e00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5579091b22f0 0x5579091d7b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:07:29.935 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:29.934+0000 7f8ea1e00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5579091b22f0 0x5579091d7b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.12:58166/0 (socket says 172.21.15.12:58166) 2024-09-11T00:07:29.936 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:29.935+0000 7f8ea1400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5579091ca640 0x5579091d80d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.12:59998/0 (socket says 172.21.15.12:59998) 2024-09-11T00:07:29.937 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:29.935+0000 7f8ea2800640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5579091cfbb0 0x5579091d4b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:07:29.937 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:29.935+0000 7f8ea2800640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5579091cfbb0 0x5579091d4b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:43266/0 (socket says 172.21.15.12:43266) 2024-09-11T00:07:29.937 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:29.935+0000 7f8ea2800640 1 -- 172.21.15.12:0/2320981333 learned_addr learned my addr 172.21.15.12:0/2320981333 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:07:29.937 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:29.935+0000 7f8eab7c0740 1 -- 172.21.15.12:0/2320981333 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f8ea41aed80 con 0x5579091cfbb0 2024-09-11T00:07:29.937 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:29.935+0000 7f8eab7c0740 1 -- 172.21.15.12:0/2320981333 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f8ea41ae6b0 con 0x5579091ca640 2024-09-11T00:07:29.937 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:29.935+0000 7f8eab7c0740 1 -- 172.21.15.12:0/2320981333 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f8ea410cec0 con 0x5579091b22f0 2024-09-11T00:07:29.937 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:29.936+0000 7f8eab7c0740 10 client.0 _ll_register_callbacks cb 0x7f8e9000bb00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:07:29.938 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:29.937+0000 7f8ea1400640 1 -- 172.21.15.12:0/2320981333 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5579091b22f0 msgr2=0x5579091d7b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:07:29.938 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:29.937+0000 7f8ea1400640 1 --2- 172.21.15.12:0/2320981333 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5579091b22f0 0x5579091d7b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:07:29.938 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:29.937+0000 7f8ea1400640 1 -- 172.21.15.12:0/2320981333 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5579091cfbb0 msgr2=0x5579091d4b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:07:29.938 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:29.937+0000 7f8ea1400640 1 --2- 172.21.15.12:0/2320981333 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5579091cfbb0 0x5579091d4b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:07:29.938 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:29.937+0000 7f8ea1400640 1 -- 172.21.15.12:0/2320981333 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5579091cd330 con 0x5579091ca640 2024-09-11T00:07:29.939 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:29.937+0000 7f8ea2800640 1 --2- 172.21.15.12:0/2320981333 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5579091cfbb0 0x5579091d4b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:07:29.939 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:29.938+0000 7f8ea1400640 1 --2- 172.21.15.12:0/2320981333 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5579091ca640 0x5579091d80d0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x5579091cc1a0 tx=0x7f8e90003e60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b337e1a0e69f6090 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:07:29.939 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:29.938+0000 7f8e83e00640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-09-11T00:07:29.939 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:29.938+0000 7f8e83e00640 1 -- 172.21.15.12:0/2320981333 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e90029040 con 0x5579091ca640 2024-09-11T00:07:29.940 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:29.938+0000 7f8e83e00640 1 -- 172.21.15.12:0/2320981333 <== mon.2 v2:172.21.15.165:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 9208+0+0 (secure 0 0 0) 0x7f8e90004630 con 0x5579091ca640 2024-09-11T00:07:29.940 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:29.939+0000 7f8eab7c0740 1 -- 172.21.15.12:0/2320981333 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8e9c030cf0 con 0x5579091ca640 2024-09-11T00:07:29.940 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:29.939+0000 7f8e83e00640 1 -- 172.21.15.12:0/2320981333 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f8e90033080 con 0x5579091ca640 2024-09-11T00:07:29.940 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:29.939+0000 7f8eab7c0740 10 client.25559 fetch_fsmap learned FSMap version 29 2024-09-11T00:07:29.940 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:29.939+0000 7f8eab7c0740 1 -- 172.21.15.12:0/2320981333 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55790924d040 con 0x5579091ca640 2024-09-11T00:07:29.942 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:29.940+0000 7f8e83e00640 1 -- 172.21.15.12:0/2320981333 <== mon.2 v2:172.21.15.165:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8e9c030cf0 con 0x5579091ca640 2024-09-11T00:07:29.942 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:29.940+0000 7f8e83e00640 1 -- 172.21.15.12:0/2320981333 <== mon.2 v2:172.21.15.165:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7f8e90023050 con 0x5579091ca640 2024-09-11T00:07:29.942 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:29.940+0000 7f8eab7c0740 10 client.25559 fetch_fsmap finished waiting for FSMap version 29 2024-09-11T00:07:29.942 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:29.940+0000 7f8eab7c0740 10 client.25559 resolve_mds: validated gid 24674 aka daemon mds.b 2024-09-11T00:07:29.942 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:29.940+0000 7f8eab7c0740 20 client.25559 populate_metadata read hostname 'smithi012' 2024-09-11T00:07:29.942 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:29.940+0000 7f8eab7c0740 1 --2- 172.21.15.12:0/2320981333 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55790924d740 0x55790924fb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:07:29.942 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:29.940+0000 7f8eab7c0740 4 client.25559 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 145.2, "await_for": 45.0}] 2024-09-11T00:07:29.942 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:29.940+0000 7f8eab7c0740 1 -- 172.21.15.12:0/2320981333 --> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 145.2, "await_for": 45.0}) -- 0x5579091256a0 con 0x55790924d740 2024-09-11T00:07:29.942 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:29.941+0000 7f8ea1e00640 1 --2- 172.21.15.12:0/2320981333 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55790924d740 0x55790924fb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:07:29.943 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:29.942+0000 7f8ea1e00640 1 --2- 172.21.15.12:0/2320981333 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55790924d740 0x55790924fb60 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4f5ce6b27ae95f1a server_cookie=38d0699f924be42e in_seq=0 out_seq=0 2024-09-11T00:07:29.943 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:29.942+0000 7f8e83e00640 10 client.25559 ms_handle_connect on v2:172.21.15.165:6838/501312984 2024-09-11T00:07:29.956 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:29.954+0000 7f8e83e00640 1 -- 172.21.15.12:0/2320981333 <== mds.0 v2:172.21.15.165:6838/501312984 1 ==== command_reply(tid 0: 0 ) ==== 8+0+589 (crc 0 0 0) 0x5579091256a0 con 0x55790924d740 2024-09-11T00:07:29.956 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:29.955+0000 7f8e83e00640 10 client.25559 handle_command_reply: tid=0 2024-09-11T00:07:29.956 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:29.955+0000 7f8eab7c0740 1 client.25559 shutdown 2024-09-11T00:07:29.957 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:29.955+0000 7f8eab7c0740 1 -- 172.21.15.12:0/2320981333 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5579091ca640 msgr2=0x5579091d80d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:07:29.957 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:29.955+0000 7f8eab7c0740 1 --2- 172.21.15.12:0/2320981333 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5579091ca640 0x5579091d80d0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x5579091cc1a0 tx=0x7f8e90003e60 comp rx=0 tx=0).stop 2024-09-11T00:07:29.957 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:29.955+0000 7f8eab7c0740 1 -- 172.21.15.12:0/2320981333 shutdown_connections 2024-09-11T00:07:29.957 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:29.955+0000 7f8eab7c0740 1 --2- 172.21.15.12:0/2320981333 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55790924d740 0x55790924fb60 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:07:29.957 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:29.956+0000 7f8eab7c0740 1 --2- 172.21.15.12:0/2320981333 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5579091cfbb0 0x5579091d4b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:07:29.957 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:29.956+0000 7f8eab7c0740 1 --2- 172.21.15.12:0/2320981333 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5579091ca640 0x5579091d80d0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:07:29.957 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:29.956+0000 7f8eab7c0740 1 --2- 172.21.15.12:0/2320981333 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5579091b22f0 0x5579091d7b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:07:29.957 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:29.956+0000 7f8eab7c0740 1 -- 172.21.15.12:0/2320981333 >> 172.21.15.12:0/2320981333 conn(0x5579091b0120 msgr2=0x5579091cb1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:07:29.957 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:29.956+0000 7f8e83e00640 0 client.25559 ms_handle_reset on v2:172.21.15.165:6838/501312984 2024-09-11T00:07:29.958 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:29.956+0000 7f8eab7c0740 1 -- 172.21.15.12:0/2320981333 shutdown_connections 2024-09-11T00:07:29.958 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:29.956+0000 7f8eab7c0740 1 -- 172.21.15.12:0/2320981333 wait complete. 2024-09-11T00:07:29.958 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:29.956+0000 7f8eab7c0740 20 client.25559 trim_cache size 0 max 16384 2024-09-11T00:07:29.958 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-09-11T00:07:29.958 INFO:teuthology.orchestra.run.smithi012.stdout: "epoch": 29, 2024-09-11T00:07:29.959 INFO:teuthology.orchestra.run.smithi012.stdout: "leader": 24674, 2024-09-11T00:07:29.959 INFO:teuthology.orchestra.run.smithi012.stdout: "set_version": 2, 2024-09-11T00:07:29.959 INFO:teuthology.orchestra.run.smithi012.stdout: "sets": { 2024-09-11T00:07:29.959 INFO:teuthology.orchestra.run.smithi012.stdout: "6a295f84": { 2024-09-11T00:07:29.959 INFO:teuthology.orchestra.run.smithi012.stdout: "version": 2, 2024-09-11T00:07:29.959 INFO:teuthology.orchestra.run.smithi012.stdout: "age_ref": 0.0, 2024-09-11T00:07:29.959 INFO:teuthology.orchestra.run.smithi012.stdout: "state": { 2024-09-11T00:07:29.959 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "QUIESCED", 2024-09-11T00:07:29.959 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 0.0 2024-09-11T00:07:29.959 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:07:29.959 INFO:teuthology.orchestra.run.smithi012.stdout: "timeout": 90.0, 2024-09-11T00:07:29.959 INFO:teuthology.orchestra.run.smithi012.stdout: "expiration": 145.2, 2024-09-11T00:07:29.959 INFO:teuthology.orchestra.run.smithi012.stdout: "members": { 2024-09-11T00:07:29.960 INFO:teuthology.orchestra.run.smithi012.stdout: "file:/": { 2024-09-11T00:07:29.960 INFO:teuthology.orchestra.run.smithi012.stdout: "excluded": false, 2024-09-11T00:07:29.960 INFO:teuthology.orchestra.run.smithi012.stdout: "state": { 2024-09-11T00:07:29.960 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "QUIESCED", 2024-09-11T00:07:29.960 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 0.0 2024-09-11T00:07:29.960 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:07:29.960 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:07:29.960 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:07:29.960 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:07:29.960 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:07:29.960 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-09-11T00:07:29.961 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:29.959+0000 7f8ea3200640 1 -- 172.21.15.12:0/2564130263 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f8e880b53a0 msgr2=0x7f8e880b7860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:07:29.961 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:29.959+0000 7f8ea3200640 1 --2- 172.21.15.12:0/2564130263 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f8e880b53a0 0x7f8e880b7860 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f8e9c004af0 tx=0x7f8e9c004a40 comp rx=0 tx=0).stop 2024-09-11T00:07:29.961 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:29.959+0000 7f8ea3200640 1 -- 172.21.15.12:0/2564130263 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8ea40754e0 msgr2=0x7f8ea410e970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:07:29.962 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:29.959+0000 7f8ea3200640 1 --2- 172.21.15.12:0/2564130263 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8ea40754e0 0x7f8ea410e970 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f8e9400e9c0 tx=0x7f8e9400ee80 comp rx=0 tx=0).stop 2024-09-11T00:07:29.962 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:29.959+0000 7f8ea3200640 1 -- 172.21.15.12:0/2564130263 shutdown_connections 2024-09-11T00:07:29.962 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:29.959+0000 7f8ea3200640 1 --2- 172.21.15.12:0/2564130263 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f8e880b53a0 0x7f8e880b7860 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:07:29.962 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:29.959+0000 7f8ea3200640 1 --2- 172.21.15.12:0/2564130263 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8ea410f200 0x7f8ea410c100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:07:29.962 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:29.959+0000 7f8ea3200640 1 --2- 172.21.15.12:0/2564130263 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8ea4075e00 0x7f8ea410bbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:07:29.962 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:29.959+0000 7f8ea3200640 1 --2- 172.21.15.12:0/2564130263 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8ea40754e0 0x7f8ea410e970 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:07:29.962 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:29.959+0000 7f8ea3200640 1 -- 172.21.15.12:0/2564130263 >> 172.21.15.12:0/2564130263 conn(0x7f8ea40fe660 msgr2=0x7f8ea40feae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:07:29.962 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:29.960+0000 7f8ea3200640 1 -- 172.21.15.12:0/2564130263 shutdown_connections 2024-09-11T00:07:29.962 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:29.960+0000 7f8ea3200640 1 -- 172.21.15.12:0/2564130263 wait complete. 2024-09-11T00:07:30.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:07:29 smithi165 ceph-mon[35356]: pgmap v410: 129 pgs: 129 active+clean; 665 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 341 B/s wr, 8 op/s 2024-09-11T00:07:30.728 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '6a295f84', quiesce took 4.7 seconds. Will release after: 20.5 2024-09-11T00:07:31.813 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:07:31 smithi012 ceph-mon[30333]: pgmap v411: 129 pgs: 129 active+clean; 665 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 255 B/s wr, 10 op/s 2024-09-11T00:07:31.830 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:07:31 smithi178 ceph-mon[35919]: pgmap v411: 129 pgs: 129 active+clean; 665 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 255 B/s wr, 10 op/s 2024-09-11T00:07:32.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:07:31 smithi165 ceph-mon[35356]: pgmap v411: 129 pgs: 129 active+clean; 665 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 255 B/s wr, 10 op/s 2024-09-11T00:07:33.830 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:07:33 smithi178 ceph-mon[35919]: pgmap v412: 129 pgs: 129 active+clean; 670 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 767 B/s wr, 11 op/s 2024-09-11T00:07:34.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:07:33 smithi165 ceph-mon[35356]: pgmap v412: 129 pgs: 129 active+clean; 670 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 767 B/s wr, 11 op/s 2024-09-11T00:07:34.063 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:07:33 smithi012 ceph-mon[30333]: pgmap v412: 129 pgs: 129 active+clean; 670 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 767 B/s wr, 11 op/s 2024-09-11T00:07:35.830 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:07:35 smithi178 ceph-mon[35919]: pgmap v413: 129 pgs: 129 active+clean; 670 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 767 B/s wr, 8 op/s 2024-09-11T00:07:36.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:07:35 smithi165 ceph-mon[35356]: pgmap v413: 129 pgs: 129 active+clean; 670 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 767 B/s wr, 8 op/s 2024-09-11T00:07:36.064 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:07:35 smithi012 ceph-mon[30333]: pgmap v413: 129 pgs: 129 active+clean; 670 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 767 B/s wr, 8 op/s 2024-09-11T00:07:37.831 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:07:37 smithi178 ceph-mon[35919]: pgmap v414: 129 pgs: 129 active+clean; 670 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 767 B/s wr, 13 op/s 2024-09-11T00:07:37.834 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:07:37 smithi165 ceph-mon[35356]: pgmap v414: 129 pgs: 129 active+clean; 670 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 767 B/s wr, 13 op/s 2024-09-11T00:07:37.866 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:07:37 smithi012 ceph-mon[30333]: pgmap v414: 129 pgs: 129 active+clean; 670 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 767 B/s wr, 13 op/s 2024-09-11T00:07:38.147 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:38.146+0000 7f67f1800640 1 -- 172.21.15.12:0/923014591 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f67ec105b20 msgr2=0x7f67ec105fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:07:38.148 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:38.146+0000 7f67f1800640 1 --2- 172.21.15.12:0/923014591 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f67ec105b20 0x7f67ec105fa0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f67d8009890 tx=0x7f67d802a380 comp rx=0 tx=0).stop 2024-09-11T00:07:38.148 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:38.147+0000 7f67f1800640 1 -- 172.21.15.12:0/923014591 shutdown_connections 2024-09-11T00:07:38.148 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:38.147+0000 7f67f1800640 1 --2- 172.21.15.12:0/923014591 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f67ec1064e0 0x7f67ec10ce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:07:38.148 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:38.147+0000 7f67f1800640 1 --2- 172.21.15.12:0/923014591 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f67ec105b20 0x7f67ec105fa0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:07:38.148 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:38.147+0000 7f67f1800640 1 --2- 172.21.15.12:0/923014591 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f67ec1049d0 0x7f67ec104dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:07:38.148 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:38.147+0000 7f67f1800640 1 -- 172.21.15.12:0/923014591 >> 172.21.15.12:0/923014591 conn(0x7f67ec100270 msgr2=0x7f67ec1026b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:07:38.149 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:38.147+0000 7f67f1800640 1 -- 172.21.15.12:0/923014591 shutdown_connections 2024-09-11T00:07:38.149 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:38.147+0000 7f67f1800640 1 -- 172.21.15.12:0/923014591 wait complete. 2024-09-11T00:07:38.149 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:38.148+0000 7f67f1800640 1 Processor -- start 2024-09-11T00:07:38.149 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:38.149+0000 7f67f1800640 1 -- start start 2024-09-11T00:07:38.150 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:38.149+0000 7f67f1800640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f67ec1049d0 0x7f67ec071c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:07:38.150 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:38.149+0000 7f67f1800640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f67ec105b20 0x7f67ec072260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:07:38.150 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:38.149+0000 7f67f1800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f67ec1064e0 0x7f67ec072e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:07:38.150 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:38.149+0000 7f67f1800640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f67ec110640 con 0x7f67ec105b20 2024-09-11T00:07:38.150 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:38.149+0000 7f67f1800640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f67ec1104c0 con 0x7f67ec1064e0 2024-09-11T00:07:38.150 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:38.149+0000 7f67f1800640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f67ec1107c0 con 0x7f67ec1049d0 2024-09-11T00:07:38.151 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:38.149+0000 7f67ebe00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f67ec1049d0 0x7f67ec071c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:07:38.151 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:38.149+0000 7f67eb400640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f67ec105b20 0x7f67ec072260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:07:38.152 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:38.149+0000 7f67ebe00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f67ec1049d0 0x7f67ec071c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.12:58190/0 (socket says 172.21.15.12:58190) 2024-09-11T00:07:38.152 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:38.149+0000 7f67ebe00640 1 -- 172.21.15.12:0/3533137181 learned_addr learned my addr 172.21.15.12:0/3533137181 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:07:38.152 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:38.149+0000 7f67f0e00640 1 --2- 172.21.15.12:0/3533137181 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f67ec1064e0 0x7f67ec072e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:07:38.152 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:38.150+0000 7f67f0e00640 1 -- 172.21.15.12:0/3533137181 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f67ec1049d0 msgr2=0x7f67ec071c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:07:38.152 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:38.150+0000 7f67f0e00640 1 --2- 172.21.15.12:0/3533137181 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f67ec1049d0 0x7f67ec071c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:07:38.152 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:38.150+0000 7f67f0e00640 1 -- 172.21.15.12:0/3533137181 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f67ec105b20 msgr2=0x7f67ec072260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:07:38.152 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:38.150+0000 7f67f0e00640 1 --2- 172.21.15.12:0/3533137181 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f67ec105b20 0x7f67ec072260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:07:38.152 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:38.150+0000 7f67f0e00640 1 -- 172.21.15.12:0/3533137181 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67ec073750 con 0x7f67ec1064e0 2024-09-11T00:07:38.153 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:38.150+0000 7f67f0e00640 1 --2- 172.21.15.12:0/3533137181 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f67ec1064e0 0x7f67ec072e90 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f67e4009fd0 tx=0x7f67e400ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:07:38.153 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:38.151+0000 7f67e8c00640 1 -- 172.21.15.12:0/3533137181 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67e400ce20 con 0x7f67ec1064e0 2024-09-11T00:07:38.153 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:38.151+0000 7f67e8c00640 1 -- 172.21.15.12:0/3533137181 <== mon.2 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f67e4005020 con 0x7f67ec1064e0 2024-09-11T00:07:38.153 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:38.151+0000 7f67f1800640 1 -- 172.21.15.12:0/3533137181 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f67ec073a40 con 0x7f67ec1064e0 2024-09-11T00:07:38.153 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:38.151+0000 7f67e8c00640 1 -- 172.21.15.12:0/3533137181 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67e4010690 con 0x7f67ec1064e0 2024-09-11T00:07:38.153 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:38.151+0000 7f67f1800640 1 -- 172.21.15.12:0/3533137181 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67ec0740e0 con 0x7f67ec1064e0 2024-09-11T00:07:38.156 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:38.155+0000 7f67f5e0a740 1 Processor -- start 2024-09-11T00:07:38.157 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:38.156+0000 7f67e8c00640 1 -- 172.21.15.12:0/3533137181 <== mon.2 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7f67e40054d0 con 0x7f67ec1064e0 2024-09-11T00:07:38.157 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:38.156+0000 7f67f5e0a740 1 -- start start 2024-09-11T00:07:38.157 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:38.156+0000 7f67f5e0a740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561e79fc7970 0x561e79fc9d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:07:38.157 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:38.156+0000 7f67f5e0a740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561e79fca2a0 0x561e79fcc690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:07:38.157 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:38.156+0000 7f67f5e0a740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561e79fccce0 0x561e79fcf1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:07:38.158 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:38.156+0000 7f67f5e0a740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x561e79f3a6b0 con 0x561e79fca2a0 2024-09-11T00:07:38.158 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:38.156+0000 7f67f5e0a740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x561e79f29ae0 con 0x561e79fc7970 2024-09-11T00:07:38.158 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:38.156+0000 7f67f5e0a740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x561e79f2d8f0 con 0x561e79fccce0 2024-09-11T00:07:38.159 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:38.157+0000 7f67eb400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561e79fc7970 0x561e79fc9d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:07:38.159 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:38.157+0000 7f67eb400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561e79fc7970 0x561e79fc9d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.12:60032/0 (socket says 172.21.15.12:60032) 2024-09-11T00:07:38.159 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:38.157+0000 7f67eb400640 1 -- 172.21.15.12:0/3908458924 learned_addr learned my addr 172.21.15.12:0/3908458924 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:07:38.159 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:38.157+0000 7f67ebe00640 1 --2- 172.21.15.12:0/3908458924 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561e79fca2a0 0x561e79fcc690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:07:38.159 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:38.157+0000 7f67eb400640 1 --2- 172.21.15.12:0/3908458924 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561e79fccce0 0x561e79fcf1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:07:38.159 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:38.157+0000 7f67eb400640 1 -- 172.21.15.12:0/3908458924 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561e79fccce0 msgr2=0x561e79fcf1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:07:38.159 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:38.157+0000 7f67eb400640 1 --2- 172.21.15.12:0/3908458924 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561e79fccce0 0x561e79fcf1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:07:38.159 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:38.157+0000 7f67eb400640 1 -- 172.21.15.12:0/3908458924 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561e79fca2a0 msgr2=0x561e79fcc690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:07:38.159 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:38.157+0000 7f67eb400640 1 --2- 172.21.15.12:0/3908458924 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561e79fca2a0 0x561e79fcc690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:07:38.159 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:38.158+0000 7f67eb400640 1 -- 172.21.15.12:0/3908458924 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561e79ef6780 con 0x561e79fc7970 2024-09-11T00:07:38.159 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:38.158+0000 7f67eb400640 1 --2- 172.21.15.12:0/3908458924 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561e79fc7970 0x561e79fc9d60 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f67d802a8a0 tx=0x7f67d8005760 comp rx=0 tx=0).ready entity=mon.2 client_cookie=715acd85bb6cb741 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:07:38.160 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:38.158+0000 7f67e1600640 1 -- 172.21.15.12:0/3908458924 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67d8046070 con 0x561e79fc7970 2024-09-11T00:07:38.161 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:38.158+0000 7f67e1600640 1 -- 172.21.15.12:0/3908458924 <== mon.2 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f67d8004040 con 0x561e79fc7970 2024-09-11T00:07:38.161 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:38.158+0000 7f67e1600640 1 -- 172.21.15.12:0/3908458924 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67d8037040 con 0x561e79fc7970 2024-09-11T00:07:38.161 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:38.158+0000 7f67e8c00640 1 --2- 172.21.15.12:0/3533137181 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f67d00b5360 0x7f67d00b7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:07:38.161 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:38.159+0000 7f67e8c00640 1 -- 172.21.15.12:0/3533137181 <== mon.2 v2:172.21.15.165:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9208+0+0 (secure 0 0 0) 0x7f67e40e66c0 con 0x7f67ec1064e0 2024-09-11T00:07:38.161 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:38.159+0000 7f67f5e0a740 1 -- 172.21.15.12:0/3908458924 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561e79fc7970 msgr2=0x561e79fc9d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:07:38.161 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:38.159+0000 7f67f5e0a740 1 --2- 172.21.15.12:0/3908458924 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561e79fc7970 0x561e79fc9d60 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f67d802a8a0 tx=0x7f67d8005760 comp rx=0 tx=0).stop 2024-09-11T00:07:38.161 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:38.159+0000 7f67f5e0a740 1 -- 172.21.15.12:0/3908458924 shutdown_connections 2024-09-11T00:07:38.161 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:38.159+0000 7f67f5e0a740 1 --2- 172.21.15.12:0/3908458924 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561e79fccce0 0x561e79fcf1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:07:38.161 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:38.159+0000 7f67f5e0a740 1 --2- 172.21.15.12:0/3908458924 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561e79fca2a0 0x561e79fcc690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:07:38.161 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:38.159+0000 7f67f5e0a740 1 --2- 172.21.15.12:0/3908458924 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561e79fc7970 0x561e79fc9d60 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:07:38.161 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:38.159+0000 7f67f5e0a740 1 -- 172.21.15.12:0/3908458924 >> 172.21.15.12:0/3908458924 conn(0x561e79fc2ac0 msgr2=0x561e79fc3850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:07:38.161 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:38.159+0000 7f67f5e0a740 1 -- 172.21.15.12:0/3908458924 shutdown_connections 2024-09-11T00:07:38.161 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:38.159+0000 7f67ebe00640 1 --2- 172.21.15.12:0/3533137181 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f67d00b5360 0x7f67d00b7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:07:38.162 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:38.160+0000 7f67f5e0a740 1 -- 172.21.15.12:0/3908458924 wait complete. 2024-09-11T00:07:38.162 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:38.160+0000 7f67f5e0a740 1 -- start start 2024-09-11T00:07:38.163 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:38.160+0000 7f67f5e0a740 1 Processor -- start 2024-09-11T00:07:38.163 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:38.161+0000 7f67ebe00640 1 --2- 172.21.15.12:0/3533137181 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f67d00b5360 0x7f67d00b7820 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f67dc0058c0 tx=0x7f67dc009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:07:38.163 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:38.161+0000 7f67f5e0a740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561e79fc7970 0x561e79fd9890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:07:38.163 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:38.161+0000 7f67f5e0a740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561e79fca2a0 0x561e79fd9f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:07:38.163 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:38.161+0000 7f67f5e0a740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561e79fccce0 0x561e79fdab00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:07:38.163 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:38.161+0000 7f67f5e0a740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x561e79e61ab0 con 0x561e79fccce0 2024-09-11T00:07:38.163 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:38.161+0000 7f67f5e0a740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x561e79ed0570 con 0x561e79fc7970 2024-09-11T00:07:38.163 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:38.161+0000 7f67f5e0a740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x561e79fcfc60 con 0x561e79fca2a0 2024-09-11T00:07:38.163 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:38.161+0000 7f67f5e0a740 10 client.0 _ll_register_callbacks cb 0x561e79edf100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:07:38.163 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:38.161+0000 7f67eb400640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561e79fca2a0 0x561e79fd9f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:07:38.164 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:38.161+0000 7f67ebe00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561e79fc7970 0x561e79fd9890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:07:38.164 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:38.161+0000 7f67f0e00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561e79fccce0 0x561e79fdab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:07:38.164 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:38.161+0000 7f67eb400640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561e79fca2a0 0x561e79fd9f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.12:58202/0 (socket says 172.21.15.12:58202) 2024-09-11T00:07:38.164 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:38.161+0000 7f67eb400640 1 -- 172.21.15.12:0/886663748 learned_addr learned my addr 172.21.15.12:0/886663748 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:07:38.164 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:38.162+0000 7f67f0e00640 1 -- 172.21.15.12:0/886663748 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561e79fca2a0 msgr2=0x561e79fd9f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:07:38.164 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:38.162+0000 7f67f0e00640 1 --2- 172.21.15.12:0/886663748 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561e79fca2a0 0x561e79fd9f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:07:38.164 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:38.162+0000 7f67f0e00640 1 -- 172.21.15.12:0/886663748 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561e79fc7970 msgr2=0x561e79fd9890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:07:38.164 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:38.162+0000 7f67f0e00640 1 --2- 172.21.15.12:0/886663748 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561e79fc7970 0x561e79fd9890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:07:38.164 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:38.162+0000 7f67f0e00640 1 -- 172.21.15.12:0/886663748 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561e79c8b790 con 0x561e79fccce0 2024-09-11T00:07:38.166 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:38.162+0000 7f67f0e00640 1 --2- 172.21.15.12:0/886663748 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561e79fccce0 0x561e79fdab00 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f67e40ac750 tx=0x7f67e40ac280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b31548dda88e1a8 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:07:38.166 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:38.162+0000 7f67d6000640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-09-11T00:07:38.166 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:38.163+0000 7f67d6000640 1 -- 172.21.15.12:0/886663748 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67e40b12d0 con 0x561e79fccce0 2024-09-11T00:07:38.166 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:38.163+0000 7f67d6000640 1 -- 172.21.15.12:0/886663748 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 9208+0+0 (secure 0 0 0) 0x7f67e40b5b50 con 0x561e79fccce0 2024-09-11T00:07:38.166 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:38.163+0000 7f67f5e0a740 1 -- 172.21.15.12:0/886663748 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f67d8004040 con 0x561e79fccce0 2024-09-11T00:07:38.166 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:38.163+0000 7f67d6000640 1 -- 172.21.15.12:0/886663748 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f67e40c7050 con 0x561e79fccce0 2024-09-11T00:07:38.166 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:38.163+0000 7f67f5e0a740 10 client.15537 fetch_fsmap learned FSMap version 29 2024-09-11T00:07:38.166 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:38.163+0000 7f67f5e0a740 1 -- 172.21.15.12:0/886663748 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x561e79d04620 con 0x561e79fccce0 2024-09-11T00:07:38.166 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:38.163+0000 7f67d6000640 1 -- 172.21.15.12:0/886663748 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f67d8004040 con 0x561e79fccce0 2024-09-11T00:07:38.166 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:38.164+0000 7f67d6000640 1 -- 172.21.15.12:0/886663748 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7f67e40a83a0 con 0x561e79fccce0 2024-09-11T00:07:38.166 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:38.164+0000 7f67f5e0a740 10 client.15537 fetch_fsmap finished waiting for FSMap version 29 2024-09-11T00:07:38.167 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:38.164+0000 7f67f5e0a740 10 client.15537 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T00:07:38.167 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:38.164+0000 7f67f5e0a740 20 client.15537 populate_metadata read hostname 'smithi012' 2024-09-11T00:07:38.167 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:38.164+0000 7f67f5e0a740 1 --2- 172.21.15.12:0/886663748 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x561e79fe50e0 0x561e79fe7530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:07:38.167 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:38.164+0000 7f67f5e0a740 4 client.15537 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T00:07:38.167 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:38.164+0000 7f67f5e0a740 1 -- 172.21.15.12:0/886663748 --> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561e79f3c600 con 0x561e79fe50e0 2024-09-11T00:07:38.167 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:38.164+0000 7f67ebe00640 1 --2- 172.21.15.12:0/886663748 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x561e79fe50e0 0x561e79fe7530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:07:38.167 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:38.165+0000 7f67ebe00640 1 --2- 172.21.15.12:0/886663748 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x561e79fe50e0 0x561e79fe7530 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8f2e478311fb1347 server_cookie=4c4ab4f2483178ce in_seq=0 out_seq=0 2024-09-11T00:07:38.167 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:38.165+0000 7f67d6000640 10 client.15537 ms_handle_connect on v2:172.21.15.165:6838/501312984 2024-09-11T00:07:38.168 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:38.166+0000 7f67d6000640 1 -- 172.21.15.12:0/886663748 <== mds.0 v2:172.21.15.165:6838/501312984 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x561e79f3c600 con 0x561e79fe50e0 2024-09-11T00:07:38.168 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:38.166+0000 7f67d6000640 10 client.15537 handle_command_reply: tid=0 2024-09-11T00:07:38.168 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:38.166+0000 7f67f5e0a740 1 client.15537 shutdown 2024-09-11T00:07:38.168 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:38.166+0000 7f67f5e0a740 1 -- 172.21.15.12:0/886663748 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561e79fccce0 msgr2=0x561e79fdab00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:07:38.168 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:38.166+0000 7f67f5e0a740 1 --2- 172.21.15.12:0/886663748 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561e79fccce0 0x561e79fdab00 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f67e40ac750 tx=0x7f67e40ac280 comp rx=0 tx=0).stop 2024-09-11T00:07:38.169 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:38.166+0000 7f67f5e0a740 1 -- 172.21.15.12:0/886663748 shutdown_connections 2024-09-11T00:07:38.169 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:38.166+0000 7f67f5e0a740 1 --2- 172.21.15.12:0/886663748 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x561e79fe50e0 0x561e79fe7530 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:07:38.170 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:38.166+0000 7f67f5e0a740 1 --2- 172.21.15.12:0/886663748 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561e79fccce0 0x561e79fdab00 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:07:38.171 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:38.166+0000 7f67f5e0a740 1 --2- 172.21.15.12:0/886663748 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561e79fca2a0 0x561e79fd9f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:07:38.171 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:38.166+0000 7f67f5e0a740 1 --2- 172.21.15.12:0/886663748 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561e79fc7970 0x561e79fd9890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:07:38.171 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:38.166+0000 7f67f5e0a740 1 -- 172.21.15.12:0/886663748 >> 172.21.15.12:0/886663748 conn(0x561e79fc2ac0 msgr2=0x561e79fc1bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:07:38.172 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:38.167+0000 7f67d6000640 0 client.15537 ms_handle_reset on v2:172.21.15.165:6838/501312984 2024-09-11T00:07:38.172 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:38.167+0000 7f67f5e0a740 1 -- 172.21.15.12:0/886663748 shutdown_connections 2024-09-11T00:07:38.172 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:38.167+0000 7f67f5e0a740 1 -- 172.21.15.12:0/886663748 wait complete. 2024-09-11T00:07:38.172 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:38.167+0000 7f67f5e0a740 20 client.15537 trim_cache size 0 max 16384 2024-09-11T00:07:38.184 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:38.182+0000 7f67f5e0a740 1 Processor -- start 2024-09-11T00:07:38.184 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:38.182+0000 7f67f5e0a740 1 -- start start 2024-09-11T00:07:38.184 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:38.182+0000 7f67f5e0a740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561e79fc7970 0x561e79fcd7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:07:38.184 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:38.183+0000 7f67f5e0a740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561e79fca2a0 0x561e79fc5920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:07:38.184 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:38.183+0000 7f67f5e0a740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561e79fccce0 0x561e79fc5e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:07:38.184 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:38.183+0000 7f67f5e0a740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f67ec1b1be0 con 0x561e79fc7970 2024-09-11T00:07:38.184 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:38.183+0000 7f67f5e0a740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f67ec1b3720 con 0x561e79fca2a0 2024-09-11T00:07:38.185 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:38.183+0000 7f67f5e0a740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f67ec1b22b0 con 0x561e79fccce0 2024-09-11T00:07:38.185 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:38.183+0000 7f67eb400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561e79fca2a0 0x561e79fc5920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:07:38.185 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:38.183+0000 7f67eb400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561e79fca2a0 0x561e79fc5920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.12:60052/0 (socket says 172.21.15.12:60052) 2024-09-11T00:07:38.185 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:38.183+0000 7f67eb400640 1 -- 172.21.15.12:0/3167506402 learned_addr learned my addr 172.21.15.12:0/3167506402 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:07:38.185 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:38.183+0000 7f67ebe00640 1 --2- 172.21.15.12:0/3167506402 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561e79fc7970 0x561e79fcd7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:07:38.185 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:38.183+0000 7f67f0e00640 1 --2- 172.21.15.12:0/3167506402 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561e79fccce0 0x561e79fc5e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:07:38.185 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:38.183+0000 7f67eb400640 1 -- 172.21.15.12:0/3167506402 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561e79fccce0 msgr2=0x561e79fc5e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:07:38.185 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:38.183+0000 7f67eb400640 1 --2- 172.21.15.12:0/3167506402 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561e79fccce0 0x561e79fc5e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:07:38.185 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:38.183+0000 7f67eb400640 1 -- 172.21.15.12:0/3167506402 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561e79fc7970 msgr2=0x561e79fcd7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:07:38.186 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:38.183+0000 7f67eb400640 1 --2- 172.21.15.12:0/3167506402 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561e79fc7970 0x561e79fcd7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:07:38.186 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:38.183+0000 7f67eb400640 1 -- 172.21.15.12:0/3167506402 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561e79fca0d0 con 0x561e79fca2a0 2024-09-11T00:07:38.187 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:38.183+0000 7f67ebe00640 1 --2- 172.21.15.12:0/3167506402 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561e79fc7970 0x561e79fcd7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T00:07:38.187 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:38.184+0000 7f67eb400640 1 --2- 172.21.15.12:0/3167506402 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561e79fca2a0 0x561e79fc5920 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f67d802cfd0 tx=0x7f67d8009450 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d711b887bc905c69 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:07:38.187 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:38.184+0000 7f67e0c00640 1 -- 172.21.15.12:0/3167506402 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67d8046070 con 0x561e79fca2a0 2024-09-11T00:07:38.187 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:38.184+0000 7f67e0c00640 1 -- 172.21.15.12:0/3167506402 <== mon.2 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f67d8004040 con 0x561e79fca2a0 2024-09-11T00:07:38.187 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:38.184+0000 7f67e0c00640 1 -- 172.21.15.12:0/3167506402 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67d8037040 con 0x561e79fca2a0 2024-09-11T00:07:38.187 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:38.185+0000 7f67f5e0a740 1 -- 172.21.15.12:0/3167506402 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561e79fca2a0 msgr2=0x561e79fc5920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:07:38.187 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:38.185+0000 7f67f5e0a740 1 --2- 172.21.15.12:0/3167506402 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561e79fca2a0 0x561e79fc5920 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f67d802cfd0 tx=0x7f67d8009450 comp rx=0 tx=0).stop 2024-09-11T00:07:38.187 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:38.185+0000 7f67f5e0a740 1 -- 172.21.15.12:0/3167506402 shutdown_connections 2024-09-11T00:07:38.187 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:38.185+0000 7f67f5e0a740 1 --2- 172.21.15.12:0/3167506402 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561e79fccce0 0x561e79fc5e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:07:38.187 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:38.185+0000 7f67f5e0a740 1 --2- 172.21.15.12:0/3167506402 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561e79fca2a0 0x561e79fc5920 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:07:38.187 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:38.185+0000 7f67f5e0a740 1 --2- 172.21.15.12:0/3167506402 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561e79fc7970 0x561e79fcd7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:07:38.187 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:38.185+0000 7f67f5e0a740 1 -- 172.21.15.12:0/3167506402 >> 172.21.15.12:0/3167506402 conn(0x561e79fc2ac0 msgr2=0x561e79fcd0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:07:38.187 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:38.185+0000 7f67f5e0a740 1 -- 172.21.15.12:0/3167506402 shutdown_connections 2024-09-11T00:07:38.188 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:38.185+0000 7f67f5e0a740 1 -- 172.21.15.12:0/3167506402 wait complete. 2024-09-11T00:07:38.188 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:38.185+0000 7f67f5e0a740 1 -- start start 2024-09-11T00:07:38.188 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:38.186+0000 7f67f5e0a740 1 Processor -- start 2024-09-11T00:07:38.190 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:38.186+0000 7f67f5e0a740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561e79fc7970 0x561e79fe04b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:07:38.190 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:38.186+0000 7f67f5e0a740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561e79fca2a0 0x561e79fe09f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:07:38.190 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:38.186+0000 7f67f5e0a740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561e79fccce0 0x561e79fe0f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:07:38.190 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:38.186+0000 7f67f5e0a740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f67ec1b3050 con 0x561e79fca2a0 2024-09-11T00:07:38.191 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:38.186+0000 7f67f5e0a740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f67ec1b2980 con 0x561e79fccce0 2024-09-11T00:07:38.191 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:38.186+0000 7f67f5e0a740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f67ec1afbc0 con 0x561e79fc7970 2024-09-11T00:07:38.191 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:38.186+0000 7f67f5e0a740 10 client.0 _ll_register_callbacks cb 0x7f67e40ac3a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:07:38.191 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:38.187+0000 7f67eb400640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561e79fca2a0 0x561e79fe09f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:07:38.191 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:38.187+0000 7f67f0e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561e79fccce0 0x561e79fe0f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:07:38.191 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:38.187+0000 7f67ebe00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561e79fc7970 0x561e79fe04b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:07:38.191 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:38.187+0000 7f67f0e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561e79fccce0 0x561e79fe0f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.12:60054/0 (socket says 172.21.15.12:60054) 2024-09-11T00:07:38.191 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:38.187+0000 7f67ebe00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561e79fc7970 0x561e79fe04b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.12:58224/0 (socket says 172.21.15.12:58224) 2024-09-11T00:07:38.191 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:38.187+0000 7f67f0e00640 1 -- 172.21.15.12:0/2943720279 learned_addr learned my addr 172.21.15.12:0/2943720279 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:07:38.191 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:38.187+0000 7f67f0e00640 1 -- 172.21.15.12:0/2943720279 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561e79fc7970 msgr2=0x561e79fe04b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:07:38.191 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:38.187+0000 7f67f0e00640 1 --2- 172.21.15.12:0/2943720279 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561e79fc7970 0x561e79fe04b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:07:38.191 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:38.187+0000 7f67f0e00640 1 -- 172.21.15.12:0/2943720279 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561e79fca2a0 msgr2=0x561e79fe09f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:07:38.191 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:38.187+0000 7f67f0e00640 1 --2- 172.21.15.12:0/2943720279 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561e79fca2a0 0x561e79fe09f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:07:38.192 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:38.187+0000 7f67f0e00640 1 -- 172.21.15.12:0/2943720279 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561e79fdf150 con 0x561e79fccce0 2024-09-11T00:07:38.192 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:38.187+0000 7f67eb400640 1 --2- 172.21.15.12:0/2943720279 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561e79fca2a0 0x561e79fe09f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T00:07:38.192 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:38.187+0000 7f67ebe00640 1 --2- 172.21.15.12:0/2943720279 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561e79fc7970 0x561e79fe04b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:07:38.192 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:38.187+0000 7f67f0e00640 1 --2- 172.21.15.12:0/2943720279 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561e79fccce0 0x561e79fe0f30 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x561e79fda5e0 tx=0x7f67e40ad290 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a4e9813f28a54f8f server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:07:38.192 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:38.187+0000 7f67d6000640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-09-11T00:07:38.192 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:38.187+0000 7f67d6000640 1 -- 172.21.15.12:0/2943720279 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67e40cb020 con 0x561e79fccce0 2024-09-11T00:07:38.192 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:38.188+0000 7f67d6000640 1 -- 172.21.15.12:0/2943720279 <== mon.2 v2:172.21.15.165:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 9208+0+0 (secure 0 0 0) 0x7f67e40b10d0 con 0x561e79fccce0 2024-09-11T00:07:38.192 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:38.188+0000 7f67f5e0a740 1 -- 172.21.15.12:0/2943720279 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f67d8004040 con 0x561e79fccce0 2024-09-11T00:07:38.192 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:38.188+0000 7f67d6000640 1 -- 172.21.15.12:0/2943720279 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f67e408e050 con 0x561e79fccce0 2024-09-11T00:07:38.192 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:38.188+0000 7f67f5e0a740 10 client.25577 fetch_fsmap learned FSMap version 29 2024-09-11T00:07:38.192 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:38.188+0000 7f67f5e0a740 1 -- 172.21.15.12:0/2943720279 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x561e7a052560 con 0x561e79fccce0 2024-09-11T00:07:38.192 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:38.188+0000 7f67d6000640 1 -- 172.21.15.12:0/2943720279 <== mon.2 v2:172.21.15.165:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f67d8004040 con 0x561e79fccce0 2024-09-11T00:07:38.192 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:38.188+0000 7f67d6000640 1 -- 172.21.15.12:0/2943720279 <== mon.2 v2:172.21.15.165:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7f67e4088ae0 con 0x561e79fccce0 2024-09-11T00:07:38.193 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:38.188+0000 7f67f5e0a740 10 client.25577 fetch_fsmap finished waiting for FSMap version 29 2024-09-11T00:07:38.193 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:38.188+0000 7f67f5e0a740 10 client.25577 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T00:07:38.193 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:38.188+0000 7f67f5e0a740 20 client.25577 populate_metadata read hostname 'smithi012' 2024-09-11T00:07:38.193 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:38.188+0000 7f67f5e0a740 1 --2- 172.21.15.12:0/2943720279 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x561e7a052ee0 0x561e7a055300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:07:38.193 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:38.188+0000 7f67f5e0a740 4 client.25577 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T00:07:38.193 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:38.188+0000 7f67f5e0a740 1 -- 172.21.15.12:0/2943720279 --> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] -- command(tid 0: {"prefix": "flush journal"}) -- 0x561e79f380b0 con 0x561e7a052ee0 2024-09-11T00:07:38.193 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:38.189+0000 7f67ebe00640 1 --2- 172.21.15.12:0/2943720279 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x561e7a052ee0 0x561e7a055300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:07:38.193 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:38.189+0000 7f67d6000640 10 client.25577 ms_handle_connect on v2:172.21.15.165:6838/501312984 2024-09-11T00:07:38.193 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:38.189+0000 7f67ebe00640 1 --2- 172.21.15.12:0/2943720279 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x561e7a052ee0 0x561e7a055300 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=791349efde6dc60 server_cookie=70f8d45c7c9a745d in_seq=0 out_seq=0 2024-09-11T00:07:38.199 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:38.197+0000 7f67d6000640 1 -- 172.21.15.12:0/2943720279 <== mds.0 v2:172.21.15.165:6838/501312984 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x561e79f380b0 con 0x561e7a052ee0 2024-09-11T00:07:38.199 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:38.197+0000 7f67d6000640 10 client.25577 handle_command_reply: tid=0 2024-09-11T00:07:38.199 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:38.197+0000 7f67f5e0a740 1 client.25577 shutdown 2024-09-11T00:07:38.199 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:38.197+0000 7f67f5e0a740 1 -- 172.21.15.12:0/2943720279 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561e79fccce0 msgr2=0x561e79fe0f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:07:38.199 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:38.197+0000 7f67f5e0a740 1 --2- 172.21.15.12:0/2943720279 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561e79fccce0 0x561e79fe0f30 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x561e79fda5e0 tx=0x7f67e40ad290 comp rx=0 tx=0).stop 2024-09-11T00:07:38.199 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:38.197+0000 7f67f5e0a740 1 -- 172.21.15.12:0/2943720279 shutdown_connections 2024-09-11T00:07:38.199 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:38.197+0000 7f67f5e0a740 1 --2- 172.21.15.12:0/2943720279 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x561e7a052ee0 0x561e7a055300 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:07:38.199 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:38.197+0000 7f67f5e0a740 1 --2- 172.21.15.12:0/2943720279 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561e79fccce0 0x561e79fe0f30 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:07:38.199 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:38.197+0000 7f67f5e0a740 1 --2- 172.21.15.12:0/2943720279 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561e79fca2a0 0x561e79fe09f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:07:38.199 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:38.197+0000 7f67f5e0a740 1 --2- 172.21.15.12:0/2943720279 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561e79fc7970 0x561e79fe04b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:07:38.199 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:38.197+0000 7f67f5e0a740 1 -- 172.21.15.12:0/2943720279 >> 172.21.15.12:0/2943720279 conn(0x561e79fc2ac0 msgr2=0x561e79fc13a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:07:38.199 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:38.198+0000 7f67d6000640 0 client.25577 ms_handle_reset on v2:172.21.15.165:6838/501312984 2024-09-11T00:07:38.200 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:38.198+0000 7f67f5e0a740 1 -- 172.21.15.12:0/2943720279 shutdown_connections 2024-09-11T00:07:38.200 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:38.198+0000 7f67f5e0a740 1 -- 172.21.15.12:0/2943720279 wait complete. 2024-09-11T00:07:38.200 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:38.198+0000 7f67f5e0a740 20 client.25577 trim_cache size 0 max 16384 2024-09-11T00:07:38.200 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:{ 2024-09-11T00:07:38.200 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "message": "", 2024-09-11T00:07:38.200 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "return_code": 0 2024-09-11T00:07:38.201 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:} 2024-09-11T00:07:38.201 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:38.199+0000 7f67f1800640 1 -- 172.21.15.12:0/3533137181 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f67d00b5360 msgr2=0x7f67d00b7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:07:38.201 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:38.199+0000 7f67f1800640 1 --2- 172.21.15.12:0/3533137181 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f67d00b5360 0x7f67d00b7820 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f67dc0058c0 tx=0x7f67dc009260 comp rx=0 tx=0).stop 2024-09-11T00:07:38.202 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:38.199+0000 7f67f1800640 1 -- 172.21.15.12:0/3533137181 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f67ec1064e0 msgr2=0x7f67ec072e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:07:38.202 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:38.199+0000 7f67f1800640 1 --2- 172.21.15.12:0/3533137181 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f67ec1064e0 0x7f67ec072e90 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f67e4009fd0 tx=0x7f67e400ef10 comp rx=0 tx=0).stop 2024-09-11T00:07:38.202 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:38.199+0000 7f67f1800640 1 -- 172.21.15.12:0/3533137181 shutdown_connections 2024-09-11T00:07:38.202 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:38.199+0000 7f67f1800640 1 --2- 172.21.15.12:0/3533137181 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f67d00b5360 0x7f67d00b7820 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:07:38.202 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:38.199+0000 7f67f1800640 1 --2- 172.21.15.12:0/3533137181 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f67ec1064e0 0x7f67ec072e90 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:07:38.202 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:38.199+0000 7f67f1800640 1 --2- 172.21.15.12:0/3533137181 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f67ec105b20 0x7f67ec072260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:07:38.202 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:38.199+0000 7f67f1800640 1 --2- 172.21.15.12:0/3533137181 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f67ec1049d0 0x7f67ec071c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:07:38.202 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:38.199+0000 7f67f1800640 1 -- 172.21.15.12:0/3533137181 >> 172.21.15.12:0/3533137181 conn(0x7f67ec100270 msgr2=0x7f67ec108f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:07:38.202 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:38.199+0000 7f67f1800640 1 -- 172.21.15.12:0/3533137181 shutdown_connections 2024-09-11T00:07:38.202 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:38.199+0000 7f67f1800640 1 -- 172.21.15.12:0/3533137181 wait complete. 2024-09-11T00:07:39.830 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:07:39 smithi178 ceph-mon[35919]: pgmap v415: 129 pgs: 129 active+clean; 670 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 511 B/s wr, 8 op/s 2024-09-11T00:07:40.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:07:39 smithi165 ceph-mon[35356]: pgmap v415: 129 pgs: 129 active+clean; 670 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 511 B/s wr, 8 op/s 2024-09-11T00:07:40.063 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:07:39 smithi012 ceph-mon[30333]: pgmap v415: 129 pgs: 129 active+clean; 670 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 511 B/s wr, 8 op/s 2024-09-11T00:07:41.830 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:07:41 smithi178 ceph-mon[35919]: pgmap v416: 129 pgs: 129 active+clean; 670 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 938 B/s wr, 11 op/s 2024-09-11T00:07:42.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:07:41 smithi165 ceph-mon[35356]: pgmap v416: 129 pgs: 129 active+clean; 670 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 938 B/s wr, 11 op/s 2024-09-11T00:07:42.063 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:07:41 smithi012 ceph-mon[30333]: pgmap v416: 129 pgs: 129 active+clean; 670 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 938 B/s wr, 11 op/s 2024-09-11T00:07:44.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:07:43 smithi165 ceph-mon[35356]: pgmap v417: 129 pgs: 129 active+clean; 670 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 1.2 KiB/s wr, 12 op/s 2024-09-11T00:07:44.062 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:07:43 smithi012 ceph-mon[30333]: pgmap v417: 129 pgs: 129 active+clean; 670 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 1.2 KiB/s wr, 12 op/s 2024-09-11T00:07:44.080 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:07:43 smithi178 ceph-mon[35919]: pgmap v417: 129 pgs: 129 active+clean; 670 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 1.2 KiB/s wr, 12 op/s 2024-09-11T00:07:46.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:07:45 smithi165 ceph-mon[35356]: pgmap v418: 129 pgs: 129 active+clean; 670 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 767 B/s wr, 10 op/s 2024-09-11T00:07:46.062 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:07:45 smithi012 ceph-mon[30333]: pgmap v418: 129 pgs: 129 active+clean; 670 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 767 B/s wr, 10 op/s 2024-09-11T00:07:46.080 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:07:45 smithi178 ceph-mon[35919]: pgmap v418: 129 pgs: 129 active+clean; 670 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 767 B/s wr, 10 op/s 2024-09-11T00:07:48.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:07:47 smithi165 ceph-mon[35356]: pgmap v419: 129 pgs: 129 active+clean; 670 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 853 B/s wr, 14 op/s 2024-09-11T00:07:48.063 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:07:47 smithi012 ceph-mon[30333]: pgmap v419: 129 pgs: 129 active+clean; 670 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 853 B/s wr, 14 op/s 2024-09-11T00:07:48.080 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:07:47 smithi178 ceph-mon[35919]: pgmap v419: 129 pgs: 129 active+clean; 670 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 853 B/s wr, 14 op/s 2024-09-11T00:07:50.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:07:49 smithi165 ceph-mon[35356]: pgmap v420: 129 pgs: 129 active+clean; 670 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 853 B/s wr, 10 op/s 2024-09-11T00:07:50.063 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:07:49 smithi012 ceph-mon[30333]: pgmap v420: 129 pgs: 129 active+clean; 670 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 853 B/s wr, 10 op/s 2024-09-11T00:07:50.080 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:07:49 smithi178 ceph-mon[35919]: pgmap v420: 129 pgs: 129 active+clean; 670 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 853 B/s wr, 10 op/s 2024-09-11T00:07:51.230 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid ff3ad74a-6fcf-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T00:07:51.393 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:51.391+0000 7fea85400640 1 -- 172.21.15.12:0/3100531234 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fea80077a20 msgr2=0x7fea80077e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:07:51.393 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:51.391+0000 7fea85400640 1 --2- 172.21.15.12:0/3100531234 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fea80077a20 0x7fea80077e20 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fea70009900 tx=0x7fea7002a410 comp rx=0 tx=0).stop 2024-09-11T00:07:51.393 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:51.391+0000 7fea85400640 1 -- 172.21.15.12:0/3100531234 shutdown_connections 2024-09-11T00:07:51.393 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:51.391+0000 7fea85400640 1 --2- 172.21.15.12:0/3100531234 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fea80079530 0x7fea80079e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:07:51.393 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:51.391+0000 7fea85400640 1 --2- 172.21.15.12:0/3100531234 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fea80078b70 0x7fea80078ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:07:51.393 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:51.391+0000 7fea85400640 1 --2- 172.21.15.12:0/3100531234 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fea80077a20 0x7fea80077e20 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:07:51.393 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:51.391+0000 7fea85400640 1 -- 172.21.15.12:0/3100531234 >> 172.21.15.12:0/3100531234 conn(0x7fea80100270 msgr2=0x7fea801026b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:07:51.393 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:51.391+0000 7fea85400640 1 -- 172.21.15.12:0/3100531234 shutdown_connections 2024-09-11T00:07:51.393 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:51.391+0000 7fea85400640 1 -- 172.21.15.12:0/3100531234 wait complete. 2024-09-11T00:07:51.395 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:51.393+0000 7fea85400640 1 Processor -- start 2024-09-11T00:07:51.395 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:51.393+0000 7fea85400640 1 -- start start 2024-09-11T00:07:51.395 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:51.393+0000 7fea85400640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fea80077a20 0x7fea8019f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:07:51.395 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:51.393+0000 7fea85400640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fea80078b70 0x7fea8019f800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:07:51.395 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:51.393+0000 7fea85400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fea80079530 0x7fea801a0430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:07:51.395 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:51.393+0000 7fea85400640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fea80114a10 con 0x7fea80077a20 2024-09-11T00:07:51.395 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:51.393+0000 7fea85400640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fea80114890 con 0x7fea80079530 2024-09-11T00:07:51.395 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:51.393+0000 7fea85400640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fea80114b90 con 0x7fea80078b70 2024-09-11T00:07:51.395 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:51.394+0000 7fea7fe00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fea80077a20 0x7fea8019f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:07:51.395 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:51.394+0000 7fea7fe00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fea80077a20 0x7fea8019f1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:47420/0 (socket says 172.21.15.12:47420) 2024-09-11T00:07:51.395 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:51.394+0000 7fea7fe00640 1 -- 172.21.15.12:0/3113899166 learned_addr learned my addr 172.21.15.12:0/3113899166 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:07:51.396 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:51.394+0000 7fea84a00640 1 --2- 172.21.15.12:0/3113899166 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fea80079530 0x7fea801a0430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:07:51.397 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:51.394+0000 7fea7fe00640 1 -- 172.21.15.12:0/3113899166 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fea80078b70 msgr2=0x7fea8019f800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:07:51.397 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:51.394+0000 7fea7fe00640 1 --2- 172.21.15.12:0/3113899166 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fea80078b70 0x7fea8019f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:07:51.397 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:51.394+0000 7fea7fe00640 1 -- 172.21.15.12:0/3113899166 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fea80079530 msgr2=0x7fea801a0430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:07:51.397 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:51.394+0000 7fea7fe00640 1 --2- 172.21.15.12:0/3113899166 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fea80079530 0x7fea801a0430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:07:51.397 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:51.394+0000 7fea7fe00640 1 -- 172.21.15.12:0/3113899166 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea801a0cf0 con 0x7fea80077a20 2024-09-11T00:07:51.397 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:51.395+0000 7fea84a00640 1 --2- 172.21.15.12:0/3113899166 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fea80079530 0x7fea801a0430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:07:51.397 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:51.395+0000 7fea7fe00640 1 --2- 172.21.15.12:0/3113899166 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fea80077a20 0x7fea8019f1b0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fea70000c00 tx=0x7fea70005210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:07:51.397 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:51.395+0000 7fea7cc00640 1 -- 172.21.15.12:0/3113899166 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea70040030 con 0x7fea80077a20 2024-09-11T00:07:51.397 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:51.395+0000 7fea85400640 1 -- 172.21.15.12:0/3113899166 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea801a0f80 con 0x7fea80077a20 2024-09-11T00:07:51.398 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:51.395+0000 7fea7cc00640 1 -- 172.21.15.12:0/3113899166 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fea70005610 con 0x7fea80077a20 2024-09-11T00:07:51.398 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:51.396+0000 7fea7cc00640 1 -- 172.21.15.12:0/3113899166 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea7003c830 con 0x7fea80077a20 2024-09-11T00:07:51.398 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:51.396+0000 7fea85400640 1 -- 172.21.15.12:0/3113899166 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fea801a74d0 con 0x7fea80077a20 2024-09-11T00:07:51.401 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:51.400+0000 7fea89a49740 1 Processor -- start 2024-09-11T00:07:51.401 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:51.400+0000 7fea89a49740 1 -- start start 2024-09-11T00:07:51.401 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:51.400+0000 7fea89a49740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564600f4e970 0x564600f50d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:07:51.401 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:51.401+0000 7fea89a49740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564600f512a0 0x564600f53690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:07:51.402 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:51.401+0000 7fea89a49740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564600f53ce0 0x564600f561a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:07:51.402 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:51.401+0000 7fea89a49740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x564600ec16b0 con 0x564600f53ce0 2024-09-11T00:07:51.402 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:51.401+0000 7fea89a49740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x564600eb0ae0 con 0x564600f4e970 2024-09-11T00:07:51.402 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:51.401+0000 7fea89a49740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x564600eb48f0 con 0x564600f512a0 2024-09-11T00:07:51.402 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:51.401+0000 7fea7cc00640 1 -- 172.21.15.12:0/3113899166 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7fea70003f10 con 0x7fea80077a20 2024-09-11T00:07:51.402 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:51.401+0000 7fea7f400640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564600f53ce0 0x564600f561a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:07:51.403 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:51.401+0000 7fea7fe00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564600f512a0 0x564600f53690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:07:51.403 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:51.401+0000 7fea7f400640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564600f53ce0 0x564600f561a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:47436/0 (socket says 172.21.15.12:47436) 2024-09-11T00:07:51.403 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:51.401+0000 7fea7f400640 1 -- 172.21.15.12:0/9522526 learned_addr learned my addr 172.21.15.12:0/9522526 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:07:51.403 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:51.402+0000 7fea84a00640 1 --2- 172.21.15.12:0/9522526 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564600f4e970 0x564600f50d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:07:51.403 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:51.402+0000 7fea7fe00640 1 -- 172.21.15.12:0/9522526 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564600f4e970 msgr2=0x564600f50d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:07:51.403 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:51.402+0000 7fea7fe00640 1 --2- 172.21.15.12:0/9522526 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564600f4e970 0x564600f50d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:07:51.403 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:51.402+0000 7fea7fe00640 1 -- 172.21.15.12:0/9522526 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564600f53ce0 msgr2=0x564600f561a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:07:51.403 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:51.402+0000 7fea7fe00640 1 --2- 172.21.15.12:0/9522526 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564600f53ce0 0x564600f561a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:07:51.403 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:51.402+0000 7fea7fe00640 1 -- 172.21.15.12:0/9522526 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564600e7d780 con 0x564600f512a0 2024-09-11T00:07:51.403 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:51.402+0000 7fea7f400640 1 --2- 172.21.15.12:0/9522526 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564600f53ce0 0x564600f561a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:07:51.403 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:51.402+0000 7fea7fe00640 1 --2- 172.21.15.12:0/9522526 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564600f512a0 0x564600f53690 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fea701040d0 tx=0x7fea7010ccd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=92d2b90c459d455 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:07:51.404 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:51.402+0000 7fea75200640 1 -- 172.21.15.12:0/9522526 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea70110670 con 0x564600f512a0 2024-09-11T00:07:51.404 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:51.402+0000 7fea75200640 1 -- 172.21.15.12:0/9522526 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fea70116020 con 0x564600f512a0 2024-09-11T00:07:51.405 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:51.402+0000 7fea75200640 1 -- 172.21.15.12:0/9522526 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea70111ac0 con 0x564600f512a0 2024-09-11T00:07:51.405 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:51.403+0000 7fea7cc00640 1 --2- 172.21.15.12:0/3113899166 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fea640b5360 0x7fea640b7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:07:51.405 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:51.403+0000 7fea7cc00640 1 -- 172.21.15.12:0/3113899166 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9208+0+0 (secure 0 0 0) 0x7fea701069d0 con 0x7fea80077a20 2024-09-11T00:07:51.405 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:51.403+0000 7fea84a00640 1 --2- 172.21.15.12:0/3113899166 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fea640b5360 0x7fea640b7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:07:51.405 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:51.403+0000 7fea89a49740 1 -- 172.21.15.12:0/9522526 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564600f512a0 msgr2=0x564600f53690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:07:51.406 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:51.403+0000 7fea89a49740 1 --2- 172.21.15.12:0/9522526 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564600f512a0 0x564600f53690 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fea701040d0 tx=0x7fea7010ccd0 comp rx=0 tx=0).stop 2024-09-11T00:07:51.406 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:51.404+0000 7fea89a49740 1 -- 172.21.15.12:0/9522526 shutdown_connections 2024-09-11T00:07:51.406 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:51.404+0000 7fea89a49740 1 --2- 172.21.15.12:0/9522526 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564600f53ce0 0x564600f561a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:07:51.406 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:51.404+0000 7fea89a49740 1 --2- 172.21.15.12:0/9522526 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564600f512a0 0x564600f53690 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:07:51.406 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:51.404+0000 7fea89a49740 1 --2- 172.21.15.12:0/9522526 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564600f4e970 0x564600f50d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:07:51.406 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:51.404+0000 7fea89a49740 1 -- 172.21.15.12:0/9522526 >> 172.21.15.12:0/9522526 conn(0x564600f49ac0 msgr2=0x564600f4a850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:07:51.408 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:51.404+0000 7fea84a00640 1 --2- 172.21.15.12:0/3113899166 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fea640b5360 0x7fea640b7820 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fea8019ff30 tx=0x7fea78009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:07:51.408 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:51.404+0000 7fea89a49740 1 -- 172.21.15.12:0/9522526 shutdown_connections 2024-09-11T00:07:51.408 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:51.404+0000 7fea89a49740 1 -- 172.21.15.12:0/9522526 wait complete. 2024-09-11T00:07:51.408 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:51.405+0000 7fea89a49740 1 -- start start 2024-09-11T00:07:51.408 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:51.405+0000 7fea89a49740 1 Processor -- start 2024-09-11T00:07:51.408 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:51.405+0000 7fea89a49740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564600f4e970 0x564600f57d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:07:51.408 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:51.405+0000 7fea89a49740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564600f512a0 0x564600f58400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:07:51.408 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:51.405+0000 7fea89a49740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564600f53ce0 0x564600f58fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:07:51.408 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:51.405+0000 7fea89a49740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x564600de8ab0 con 0x564600f512a0 2024-09-11T00:07:51.408 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:51.405+0000 7fea89a49740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x564600e57570 con 0x564600f53ce0 2024-09-11T00:07:51.408 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:51.405+0000 7fea89a49740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x564600f56a80 con 0x564600f4e970 2024-09-11T00:07:51.409 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:51.405+0000 7fea89a49740 10 client.0 _ll_register_callbacks cb 0x564600e66100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:07:51.409 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:51.406+0000 7fea7fe00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564600f4e970 0x564600f57d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:07:51.409 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:51.406+0000 7fea84a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564600f53ce0 0x564600f58fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:07:51.409 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:51.406+0000 7fea84a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564600f53ce0 0x564600f58fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.12:39108/0 (socket says 172.21.15.12:39108) 2024-09-11T00:07:51.409 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:51.406+0000 7fea84a00640 1 -- 172.21.15.12:0/1129345939 learned_addr learned my addr 172.21.15.12:0/1129345939 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:07:51.409 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:51.406+0000 7fea84a00640 1 -- 172.21.15.12:0/1129345939 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564600f4e970 msgr2=0x564600f57d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:07:51.409 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:51.406+0000 7fea84a00640 1 --2- 172.21.15.12:0/1129345939 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564600f4e970 0x564600f57d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:07:51.409 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:51.407+0000 7fea84a00640 1 -- 172.21.15.12:0/1129345939 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564600f512a0 msgr2=0x564600f58400 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-11T00:07:51.409 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:51.407+0000 7fea84a00640 1 --2- 172.21.15.12:0/1129345939 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564600f512a0 0x564600f58400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:07:51.409 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:51.407+0000 7fea84a00640 1 -- 172.21.15.12:0/1129345939 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564600c12790 con 0x564600f53ce0 2024-09-11T00:07:51.409 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:51.407+0000 7fea84a00640 1 --2- 172.21.15.12:0/1129345939 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564600f53ce0 0x564600f58fd0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fea780040b0 tx=0x7fea78004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b968e88a4754e2e9 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:07:51.409 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:51.407+0000 7fea69600640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-09-11T00:07:51.411 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:51.408+0000 7fea69600640 1 -- 172.21.15.12:0/1129345939 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea78011c40 con 0x564600f53ce0 2024-09-11T00:07:51.411 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:51.408+0000 7fea89a49740 1 -- 172.21.15.12:0/1129345939 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fea70116020 con 0x564600f53ce0 2024-09-11T00:07:51.411 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:51.408+0000 7fea69600640 1 -- 172.21.15.12:0/1129345939 <== mon.2 v2:172.21.15.165:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 9208+0+0 (secure 0 0 0) 0x7fea78011de0 con 0x564600f53ce0 2024-09-11T00:07:51.411 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:51.408+0000 7fea69600640 1 -- 172.21.15.12:0/1129345939 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fea7801f430 con 0x564600f53ce0 2024-09-11T00:07:51.411 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:51.408+0000 7fea89a49740 10 client.25586 fetch_fsmap learned FSMap version 29 2024-09-11T00:07:51.411 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:51.408+0000 7fea89a49740 1 -- 172.21.15.12:0/1129345939 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x564600c8b620 con 0x564600f53ce0 2024-09-11T00:07:51.411 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:51.409+0000 7fea69600640 1 -- 172.21.15.12:0/1129345939 <== mon.2 v2:172.21.15.165:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fea70116020 con 0x564600f53ce0 2024-09-11T00:07:51.411 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:51.409+0000 7fea69600640 1 -- 172.21.15.12:0/1129345939 <== mon.2 v2:172.21.15.165:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7fea78020710 con 0x564600f53ce0 2024-09-11T00:07:51.411 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:51.409+0000 7fea89a49740 10 client.25586 fetch_fsmap finished waiting for FSMap version 29 2024-09-11T00:07:51.411 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:51.409+0000 7fea89a49740 10 client.25586 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T00:07:51.411 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:51.409+0000 7fea89a49740 20 client.25586 populate_metadata read hostname 'smithi012' 2024-09-11T00:07:51.411 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:51.409+0000 7fea89a49740 1 --2- 172.21.15.12:0/1129345939 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x564600f63710 0x564600f65b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:07:51.412 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:51.409+0000 7fea89a49740 4 client.25586 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T00:07:51.412 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:51.409+0000 7fea89a49740 1 -- 172.21.15.12:0/1129345939 --> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564600ec3600 con 0x564600f63710 2024-09-11T00:07:51.412 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:51.410+0000 7fea7fe00640 1 --2- 172.21.15.12:0/1129345939 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x564600f63710 0x564600f65b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:07:51.413 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:51.410+0000 7fea7fe00640 1 --2- 172.21.15.12:0/1129345939 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x564600f63710 0x564600f65b20 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=45cbe29a81e5de5a server_cookie=36e1941acbff5c66 in_seq=0 out_seq=0 2024-09-11T00:07:51.413 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:51.410+0000 7fea69600640 10 client.25586 ms_handle_connect on v2:172.21.15.165:6838/501312984 2024-09-11T00:07:51.413 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:51.411+0000 7fea69600640 1 -- 172.21.15.12:0/1129345939 <== mds.0 v2:172.21.15.165:6838/501312984 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x564600ec3600 con 0x564600f63710 2024-09-11T00:07:51.413 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:51.411+0000 7fea69600640 10 client.25586 handle_command_reply: tid=0 2024-09-11T00:07:51.413 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:51.411+0000 7fea89a49740 1 client.25586 shutdown 2024-09-11T00:07:51.413 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:51.411+0000 7fea89a49740 1 -- 172.21.15.12:0/1129345939 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564600f53ce0 msgr2=0x564600f58fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:07:51.413 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:51.411+0000 7fea89a49740 1 --2- 172.21.15.12:0/1129345939 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564600f53ce0 0x564600f58fd0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fea780040b0 tx=0x7fea78004a40 comp rx=0 tx=0).stop 2024-09-11T00:07:51.413 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:51.411+0000 7fea89a49740 1 -- 172.21.15.12:0/1129345939 shutdown_connections 2024-09-11T00:07:51.413 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:51.411+0000 7fea89a49740 1 --2- 172.21.15.12:0/1129345939 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x564600f63710 0x564600f65b20 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:07:51.414 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:51.411+0000 7fea89a49740 1 --2- 172.21.15.12:0/1129345939 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564600f53ce0 0x564600f58fd0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:07:51.414 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:51.411+0000 7fea89a49740 1 --2- 172.21.15.12:0/1129345939 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564600f512a0 0x564600f58400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:07:51.414 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:51.411+0000 7fea69600640 0 client.25586 ms_handle_reset on v2:172.21.15.165:6838/501312984 2024-09-11T00:07:51.414 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:51.411+0000 7fea89a49740 1 --2- 172.21.15.12:0/1129345939 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564600f4e970 0x564600f57d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:07:51.414 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:51.411+0000 7fea89a49740 1 -- 172.21.15.12:0/1129345939 >> 172.21.15.12:0/1129345939 conn(0x564600f49ac0 msgr2=0x564600f48bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:07:51.414 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:51.412+0000 7fea89a49740 1 -- 172.21.15.12:0/1129345939 shutdown_connections 2024-09-11T00:07:51.414 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:51.412+0000 7fea89a49740 1 -- 172.21.15.12:0/1129345939 wait complete. 2024-09-11T00:07:51.414 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:51.412+0000 7fea89a49740 20 client.25586 trim_cache size 0 max 16384 2024-09-11T00:07:51.441 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:51.437+0000 7fea89a49740 1 Processor -- start 2024-09-11T00:07:51.441 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:51.437+0000 7fea89a49740 1 -- start start 2024-09-11T00:07:51.441 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:51.438+0000 7fea89a49740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564600f4e970 0x564600f4dda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:07:51.441 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:51.438+0000 7fea89a49740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564600f512a0 0x564600f4e2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:07:51.441 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:51.438+0000 7fea89a49740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564600f53ce0 0x564600f5f8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:07:51.441 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:51.438+0000 7fea89a49740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fea801b1c00 con 0x564600f53ce0 2024-09-11T00:07:51.441 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:51.438+0000 7fea89a49740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fea801b3740 con 0x564600f4e970 2024-09-11T00:07:51.441 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:51.438+0000 7fea89a49740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fea801b22d0 con 0x564600f512a0 2024-09-11T00:07:51.441 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:51.438+0000 7fea84a00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564600f53ce0 0x564600f5f8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:07:51.441 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:51.438+0000 7fea84a00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564600f53ce0 0x564600f5f8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:47444/0 (socket says 172.21.15.12:47444) 2024-09-11T00:07:51.442 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:51.438+0000 7fea84a00640 1 -- 172.21.15.12:0/3380387100 learned_addr learned my addr 172.21.15.12:0/3380387100 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:07:51.442 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:51.438+0000 7fea7fe00640 1 --2- 172.21.15.12:0/3380387100 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564600f4e970 0x564600f4dda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:07:51.442 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:51.438+0000 7fea7f400640 1 --2- 172.21.15.12:0/3380387100 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564600f512a0 0x564600f4e2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:07:51.442 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:51.439+0000 7fea84a00640 1 -- 172.21.15.12:0/3380387100 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564600f512a0 msgr2=0x564600f4e2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:07:51.442 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:51.439+0000 7fea84a00640 1 --2- 172.21.15.12:0/3380387100 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564600f512a0 0x564600f4e2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:07:51.442 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:51.439+0000 7fea84a00640 1 -- 172.21.15.12:0/3380387100 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564600f4e970 msgr2=0x564600f4dda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:07:51.442 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:51.439+0000 7fea84a00640 1 --2- 172.21.15.12:0/3380387100 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564600f4e970 0x564600f4dda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:07:51.442 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:51.439+0000 7fea84a00640 1 -- 172.21.15.12:0/3380387100 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564600f510d0 con 0x564600f53ce0 2024-09-11T00:07:51.442 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:51.439+0000 7fea84a00640 1 --2- 172.21.15.12:0/3380387100 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564600f53ce0 0x564600f5f8d0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fea78009600 tx=0x7fea780116f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9a804ca0508824 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:07:51.442 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:51.439+0000 7fea6be00640 1 -- 172.21.15.12:0/3380387100 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea78004cb0 con 0x564600f53ce0 2024-09-11T00:07:51.442 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:51.440+0000 7fea6be00640 1 -- 172.21.15.12:0/3380387100 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fea78024070 con 0x564600f53ce0 2024-09-11T00:07:51.442 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:51.440+0000 7fea6be00640 1 -- 172.21.15.12:0/3380387100 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea7801f830 con 0x564600f53ce0 2024-09-11T00:07:51.444 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:51.441+0000 7fea89a49740 1 -- 172.21.15.12:0/3380387100 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564600f53ce0 msgr2=0x564600f5f8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:07:51.445 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:51.441+0000 7fea89a49740 1 --2- 172.21.15.12:0/3380387100 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564600f53ce0 0x564600f5f8d0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fea78009600 tx=0x7fea780116f0 comp rx=0 tx=0).stop 2024-09-11T00:07:51.445 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:51.441+0000 7fea89a49740 1 -- 172.21.15.12:0/3380387100 shutdown_connections 2024-09-11T00:07:51.445 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:51.441+0000 7fea89a49740 1 --2- 172.21.15.12:0/3380387100 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564600f53ce0 0x564600f5f8d0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:07:51.445 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:51.441+0000 7fea89a49740 1 --2- 172.21.15.12:0/3380387100 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564600f512a0 0x564600f4e2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:07:51.445 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:51.441+0000 7fea89a49740 1 --2- 172.21.15.12:0/3380387100 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564600f4e970 0x564600f4dda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:07:51.445 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:51.441+0000 7fea89a49740 1 -- 172.21.15.12:0/3380387100 >> 172.21.15.12:0/3380387100 conn(0x564600f49ac0 msgr2=0x564600f540c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:07:51.445 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:51.441+0000 7fea89a49740 1 -- 172.21.15.12:0/3380387100 shutdown_connections 2024-09-11T00:07:51.445 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:51.441+0000 7fea89a49740 1 -- 172.21.15.12:0/3380387100 wait complete. 2024-09-11T00:07:51.445 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:51.442+0000 7fea89a49740 1 -- start start 2024-09-11T00:07:51.445 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:51.442+0000 7fea89a49740 1 Processor -- start 2024-09-11T00:07:51.445 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:51.442+0000 7fea89a49740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564600f4e970 0x564600f5e9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:07:51.445 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:51.442+0000 7fea89a49740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564600f512a0 0x564600f5ef10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:07:51.445 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:51.442+0000 7fea89a49740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564600f53ce0 0x564600f5f450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:07:51.446 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:51.442+0000 7fea89a49740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fea801b3070 con 0x564600f4e970 2024-09-11T00:07:51.446 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:51.442+0000 7fea89a49740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fea801b29a0 con 0x564600f512a0 2024-09-11T00:07:51.446 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:51.442+0000 7fea89a49740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fea801afbe0 con 0x564600f53ce0 2024-09-11T00:07:51.446 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:51.442+0000 7fea89a49740 10 client.0 _ll_register_callbacks cb 0x7fea78003f20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:07:51.446 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:51.443+0000 7fea7fe00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564600f4e970 0x564600f5e9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:07:51.446 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:51.443+0000 7fea7fe00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564600f4e970 0x564600f5e9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:47458/0 (socket says 172.21.15.12:47458) 2024-09-11T00:07:51.446 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:51.443+0000 7fea7fe00640 1 -- 172.21.15.12:0/1229687875 learned_addr learned my addr 172.21.15.12:0/1229687875 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:07:51.446 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:51.443+0000 7fea7f400640 1 --2- 172.21.15.12:0/1229687875 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564600f512a0 0x564600f5ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:07:51.446 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:51.443+0000 7fea84a00640 1 --2- 172.21.15.12:0/1229687875 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564600f53ce0 0x564600f5f450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:07:51.446 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:51.443+0000 7fea7fe00640 1 -- 172.21.15.12:0/1229687875 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564600f53ce0 msgr2=0x564600f5f450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:07:51.446 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:51.443+0000 7fea7fe00640 1 --2- 172.21.15.12:0/1229687875 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564600f53ce0 0x564600f5f450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:07:51.446 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:51.443+0000 7fea7fe00640 1 -- 172.21.15.12:0/1229687875 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564600f512a0 msgr2=0x564600f5ef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:07:51.446 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:51.443+0000 7fea7fe00640 1 --2- 172.21.15.12:0/1229687875 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564600f512a0 0x564600f5ef10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:07:51.447 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:51.444+0000 7fea7fe00640 1 -- 172.21.15.12:0/1229687875 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564600f5d6f0 con 0x564600f4e970 2024-09-11T00:07:51.447 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:51.444+0000 7fea7f400640 1 --2- 172.21.15.12:0/1229687875 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564600f512a0 0x564600f5ef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T00:07:51.448 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:51.444+0000 7fea7fe00640 1 --2- 172.21.15.12:0/1229687875 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564600f4e970 0x564600f5e9d0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fea700c9390 tx=0x7fea700042b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8fee2187f600a87e server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:07:51.448 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:51.444+0000 7fea69600640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-09-11T00:07:51.448 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:51.444+0000 7fea69600640 1 -- 172.21.15.12:0/1229687875 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea7004a040 con 0x564600f4e970 2024-09-11T00:07:51.448 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:51.444+0000 7fea69600640 1 -- 172.21.15.12:0/1229687875 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 9208+0+0 (secure 0 0 0) 0x7fea70104180 con 0x564600f4e970 2024-09-11T00:07:51.448 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:51.444+0000 7fea89a49740 1 -- 172.21.15.12:0/1229687875 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fea78024070 con 0x564600f4e970 2024-09-11T00:07:51.448 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:51.445+0000 7fea69600640 1 -- 172.21.15.12:0/1229687875 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fea700f2070 con 0x564600f4e970 2024-09-11T00:07:51.448 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:51.445+0000 7fea89a49740 10 client.15558 fetch_fsmap learned FSMap version 29 2024-09-11T00:07:51.448 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:51.445+0000 7fea89a49740 1 -- 172.21.15.12:0/1229687875 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x564600fd0bc0 con 0x564600f4e970 2024-09-11T00:07:51.448 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:51.445+0000 7fea69600640 1 -- 172.21.15.12:0/1229687875 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fea78024070 con 0x564600f4e970 2024-09-11T00:07:51.448 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:51.445+0000 7fea69600640 1 -- 172.21.15.12:0/1229687875 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7fea700c9030 con 0x564600f4e970 2024-09-11T00:07:51.449 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:51.445+0000 7fea89a49740 10 client.15558 fetch_fsmap finished waiting for FSMap version 29 2024-09-11T00:07:51.449 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:51.445+0000 7fea89a49740 10 client.15558 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T00:07:51.449 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:51.445+0000 7fea89a49740 20 client.15558 populate_metadata read hostname 'smithi012' 2024-09-11T00:07:51.449 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:51.445+0000 7fea89a49740 1 --2- 172.21.15.12:0/1229687875 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x564600fd13d0 0x564600fd37f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:07:51.449 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:51.445+0000 7fea89a49740 4 client.15558 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T00:07:51.449 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:51.445+0000 7fea89a49740 1 -- 172.21.15.12:0/1229687875 --> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] -- command(tid 0: {"prefix": "flush journal"}) -- 0x564600ebf0b0 con 0x564600fd13d0 2024-09-11T00:07:51.449 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:51.446+0000 7fea7f400640 1 --2- 172.21.15.12:0/1229687875 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x564600fd13d0 0x564600fd37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:07:51.449 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:51.447+0000 7fea7f400640 1 --2- 172.21.15.12:0/1229687875 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x564600fd13d0 0x564600fd37f0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=420bf84cb78635f5 server_cookie=9e354b6a8101ba1c in_seq=0 out_seq=0 2024-09-11T00:07:51.449 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:51.447+0000 7fea69600640 10 client.15558 ms_handle_connect on v2:172.21.15.165:6838/501312984 2024-09-11T00:07:51.451 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:51.451+0000 7fea69600640 1 -- 172.21.15.12:0/1229687875 <== mds.0 v2:172.21.15.165:6838/501312984 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x564600ebf0b0 con 0x564600fd13d0 2024-09-11T00:07:51.451 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:51.451+0000 7fea69600640 10 client.15558 handle_command_reply: tid=0 2024-09-11T00:07:51.452 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:51.451+0000 7fea89a49740 1 client.15558 shutdown 2024-09-11T00:07:51.453 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:51.451+0000 7fea89a49740 1 -- 172.21.15.12:0/1229687875 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564600f4e970 msgr2=0x564600f5e9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:07:51.453 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:51.451+0000 7fea89a49740 1 --2- 172.21.15.12:0/1229687875 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564600f4e970 0x564600f5e9d0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fea700c9390 tx=0x7fea700042b0 comp rx=0 tx=0).stop 2024-09-11T00:07:51.453 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:51.451+0000 7fea89a49740 1 -- 172.21.15.12:0/1229687875 shutdown_connections 2024-09-11T00:07:51.453 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:51.451+0000 7fea89a49740 1 --2- 172.21.15.12:0/1229687875 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x564600fd13d0 0x564600fd37f0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:07:51.453 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:51.451+0000 7fea89a49740 1 --2- 172.21.15.12:0/1229687875 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564600f53ce0 0x564600f5f450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:07:51.453 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:51.451+0000 7fea89a49740 1 --2- 172.21.15.12:0/1229687875 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564600f512a0 0x564600f5ef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:07:51.453 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:51.451+0000 7fea69600640 0 client.15558 ms_handle_reset on v2:172.21.15.165:6838/501312984 2024-09-11T00:07:51.453 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:51.451+0000 7fea89a49740 1 --2- 172.21.15.12:0/1229687875 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564600f4e970 0x564600f5e9d0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:07:51.453 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:51.451+0000 7fea89a49740 1 -- 172.21.15.12:0/1229687875 >> 172.21.15.12:0/1229687875 conn(0x564600f49ac0 msgr2=0x564600f4d970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:07:51.453 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:51.452+0000 7fea89a49740 1 -- 172.21.15.12:0/1229687875 shutdown_connections 2024-09-11T00:07:51.453 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:51.452+0000 7fea89a49740 1 -- 172.21.15.12:0/1229687875 wait complete. 2024-09-11T00:07:51.453 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:51.452+0000 7fea89a49740 20 client.15558 trim_cache size 0 max 16384 2024-09-11T00:07:51.454 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:{ 2024-09-11T00:07:51.454 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "message": "", 2024-09-11T00:07:51.454 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "return_code": 0 2024-09-11T00:07:51.454 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:} 2024-09-11T00:07:51.454 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:51.454+0000 7fea85400640 1 -- 172.21.15.12:0/3113899166 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fea640b5360 msgr2=0x7fea640b7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:07:51.454 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:51.454+0000 7fea85400640 1 --2- 172.21.15.12:0/3113899166 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fea640b5360 0x7fea640b7820 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fea8019ff30 tx=0x7fea78009500 comp rx=0 tx=0).stop 2024-09-11T00:07:51.454 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:51.454+0000 7fea85400640 1 -- 172.21.15.12:0/3113899166 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fea80077a20 msgr2=0x7fea8019f1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:07:51.454 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:51.454+0000 7fea85400640 1 --2- 172.21.15.12:0/3113899166 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fea80077a20 0x7fea8019f1b0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fea70000c00 tx=0x7fea70005210 comp rx=0 tx=0).stop 2024-09-11T00:07:51.455 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:51.454+0000 7fea85400640 1 -- 172.21.15.12:0/3113899166 shutdown_connections 2024-09-11T00:07:51.455 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:51.454+0000 7fea85400640 1 --2- 172.21.15.12:0/3113899166 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fea640b5360 0x7fea640b7820 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:07:51.455 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:51.454+0000 7fea85400640 1 --2- 172.21.15.12:0/3113899166 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fea80079530 0x7fea801a0430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:07:51.455 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:51.454+0000 7fea85400640 1 --2- 172.21.15.12:0/3113899166 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fea80078b70 0x7fea8019f800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:07:51.455 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:51.454+0000 7fea85400640 1 --2- 172.21.15.12:0/3113899166 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fea80077a20 0x7fea8019f1b0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:07:51.455 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:51.454+0000 7fea85400640 1 -- 172.21.15.12:0/3113899166 >> 172.21.15.12:0/3113899166 conn(0x7fea80100270 msgr2=0x7fea80101d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:07:51.455 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:51.454+0000 7fea85400640 1 -- 172.21.15.12:0/3113899166 shutdown_connections 2024-09-11T00:07:51.455 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:07:51.454+0000 7fea85400640 1 -- 172.21.15.12:0/3113899166 wait complete. 2024-09-11T00:07:51.806 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:07:51.949 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:07:51 smithi012 ceph-mon[30333]: pgmap v421: 129 pgs: 129 active+clean; 670 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 853 B/s wr, 11 op/s 2024-09-11T00:07:52.033 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:07:51 smithi165 ceph-mon[35356]: pgmap v421: 129 pgs: 129 active+clean; 670 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 853 B/s wr, 11 op/s 2024-09-11T00:07:52.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:07:51 smithi178 ceph-mon[35919]: pgmap v421: 129 pgs: 129 active+clean; 670 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 853 B/s wr, 11 op/s 2024-09-11T00:07:52.416 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:52.414+0000 7efe6b600640 1 -- 172.21.15.12:0/1086443168 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7efe64104fa0 msgr2=0x7efe64105380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:07:52.416 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:52.414+0000 7efe6b600640 1 --2- 172.21.15.12:0/1086443168 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7efe64104fa0 0x7efe64105380 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7efe58009860 tx=0x7efe5802a340 comp rx=0 tx=0).stop 2024-09-11T00:07:52.416 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:52.414+0000 7efe6b600640 1 -- 172.21.15.12:0/1086443168 shutdown_connections 2024-09-11T00:07:52.416 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:52.414+0000 7efe6b600640 1 --2- 172.21.15.12:0/1086443168 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efe6410b330 0x7efe64111b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:07:52.416 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:52.414+0000 7efe6b600640 1 --2- 172.21.15.12:0/1086443168 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efe6410a9b0 0x7efe6410adf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:07:52.416 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:52.414+0000 7efe6b600640 1 --2- 172.21.15.12:0/1086443168 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7efe64104fa0 0x7efe64105380 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:07:52.416 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:52.414+0000 7efe6b600640 1 -- 172.21.15.12:0/1086443168 >> 172.21.15.12:0/1086443168 conn(0x7efe64100a50 msgr2=0x7efe64102e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:07:52.416 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:52.414+0000 7efe6b600640 1 -- 172.21.15.12:0/1086443168 shutdown_connections 2024-09-11T00:07:52.417 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:52.414+0000 7efe6b600640 1 -- 172.21.15.12:0/1086443168 wait complete. 2024-09-11T00:07:52.417 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:52.416+0000 7efe6b600640 1 Processor -- start 2024-09-11T00:07:52.417 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:52.416+0000 7efe6b600640 1 -- start start 2024-09-11T00:07:52.417 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:52.416+0000 7efe6b600640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efe64104fa0 0x7efe6419b490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:07:52.417 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:52.416+0000 7efe6b600640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efe6410a9b0 0x7efe6419b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:07:52.418 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:52.416+0000 7efe6b600640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7efe6410b330 0x7efe641a0d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:07:52.418 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:52.416+0000 7efe6b600640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7efe64114f60 con 0x7efe6410a9b0 2024-09-11T00:07:52.418 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:52.416+0000 7efe6b600640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7efe64114de0 con 0x7efe6410b330 2024-09-11T00:07:52.418 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:52.416+0000 7efe6b600640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7efe641150e0 con 0x7efe64104fa0 2024-09-11T00:07:52.419 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:52.417+0000 7efe68c00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efe64104fa0 0x7efe6419b490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:07:52.419 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:52.417+0000 7efe68c00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efe64104fa0 0x7efe6419b490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.12:47548/0 (socket says 172.21.15.12:47548) 2024-09-11T00:07:52.419 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:52.417+0000 7efe68c00640 1 -- 172.21.15.12:0/3632627658 learned_addr learned my addr 172.21.15.12:0/3632627658 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:07:52.419 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:52.417+0000 7efe63e00640 1 --2- 172.21.15.12:0/3632627658 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efe6410a9b0 0x7efe6419b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:07:52.419 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:52.417+0000 7efe69600640 1 --2- 172.21.15.12:0/3632627658 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7efe6410b330 0x7efe641a0d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:07:52.419 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:52.417+0000 7efe68c00640 1 -- 172.21.15.12:0/3632627658 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7efe6410b330 msgr2=0x7efe641a0d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:07:52.419 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:52.417+0000 7efe68c00640 1 --2- 172.21.15.12:0/3632627658 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7efe6410b330 0x7efe641a0d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:07:52.419 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:52.417+0000 7efe68c00640 1 -- 172.21.15.12:0/3632627658 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efe6410a9b0 msgr2=0x7efe6419b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:07:52.419 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:52.417+0000 7efe68c00640 1 --2- 172.21.15.12:0/3632627658 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efe6410a9b0 0x7efe6419b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:07:52.419 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:52.417+0000 7efe68c00640 1 -- 172.21.15.12:0/3632627658 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe641a1540 con 0x7efe64104fa0 2024-09-11T00:07:52.419 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:52.418+0000 7efe68c00640 1 --2- 172.21.15.12:0/3632627658 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efe64104fa0 0x7efe6419b490 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7efe580303f0 tx=0x7efe58032da0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:07:52.420 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:52.418+0000 7efe61600640 1 -- 172.21.15.12:0/3632627658 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe58040020 con 0x7efe64104fa0 2024-09-11T00:07:52.420 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:52.418+0000 7efe61600640 1 -- 172.21.15.12:0/3632627658 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efe58030dd0 con 0x7efe64104fa0 2024-09-11T00:07:52.420 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:52.418+0000 7efe61600640 1 -- 172.21.15.12:0/3632627658 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe58037040 con 0x7efe64104fa0 2024-09-11T00:07:52.420 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:52.418+0000 7efe6b600640 1 -- 172.21.15.12:0/3632627658 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe641a17d0 con 0x7efe64104fa0 2024-09-11T00:07:52.420 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:52.418+0000 7efe6b600640 1 -- 172.21.15.12:0/3632627658 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe641a7ef0 con 0x7efe64104fa0 2024-09-11T00:07:52.421 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:52.420+0000 7efe6b600640 1 -- 172.21.15.12:0/3632627658 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efe64105380 con 0x7efe64104fa0 2024-09-11T00:07:52.425 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:52.424+0000 7efe61600640 1 -- 172.21.15.12:0/3632627658 <== mon.1 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7efe5804d060 con 0x7efe64104fa0 2024-09-11T00:07:52.426 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:52.425+0000 7efe61600640 1 --2- 172.21.15.12:0/3632627658 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7efe4c0b52e0 0x7efe4c0b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:07:52.427 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:52.426+0000 7efe61600640 1 -- 172.21.15.12:0/3632627658 <== mon.1 v2:172.21.15.178:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9208+0+0 (secure 0 0 0) 0x7efe58105bb0 con 0x7efe64104fa0 2024-09-11T00:07:52.427 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:52.426+0000 7efe63e00640 1 --2- 172.21.15.12:0/3632627658 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7efe4c0b52e0 0x7efe4c0b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:07:52.427 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:52.426+0000 7efe61600640 1 -- 172.21.15.12:0/3632627658 <== mon.1 v2:172.21.15.178:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7efe5804a020 con 0x7efe64104fa0 2024-09-11T00:07:52.428 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:52.427+0000 7efe63e00640 1 --2- 172.21.15.12:0/3632627658 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7efe4c0b52e0 0x7efe4c0b77a0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7efe64109fa0 tx=0x7efe50009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:07:52.720 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:52.719+0000 7efe6b600640 1 -- 172.21.15.12:0/3632627658 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7efe6419c5c0 con 0x7efe64104fa0 2024-09-11T00:07:52.721 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:52.720+0000 7efe61600640 1 -- 172.21.15.12:0/3632627658 <== mon.1 v2:172.21.15.178:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11965 (secure 0 0 0) 0x7efe580cc720 con 0x7efe64104fa0 2024-09-11T00:07:52.721 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-11T00:07:52.722 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":29,"btime":"2024-09-11T00:06:45:273053+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14577,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6835/447171591","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":447171591},{"type":"v1","addr":"172.21.15.12:6835","nonce":447171591}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14589,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/901486040","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":901486040},{"type":"v1","addr":"172.21.15.12:6837","nonce":901486040}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T00:04:04.739881+0000","modified":"2024-09-11T00:06:44.753824+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24674,"mds_1":24643,"mds_2":24638,"mds_3":24637,"mds_4":24614},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14631":{"gid":14631,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.12:6841/3069919437","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6840","nonce":3069919437},{"type":"v1","addr":"172.21.15.12:6841","nonce":3069919437}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24562":{"gid":24562,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6833/1141636112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":1141636112},{"type":"v1","addr":"172.21.15.178:6833","nonce":1141636112}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24581":{"gid":24581,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6833/3111980970","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6832","nonce":3111980970},{"type":"v1","addr":"172.21.15.165:6833","nonce":3111980970}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24607":{"gid":24607,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6835/4084868287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":4084868287},{"type":"v1","addr":"172.21.15.178:6835","nonce":4084868287}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24613":{"gid":24613,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6837/526874763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":526874763},{"type":"v1","addr":"172.21.15.178:6837","nonce":526874763}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24614":{"gid":24614,"name":"e","rank":4,"incarnation":26,"state":"up:active","state_seq":21,"addr":"172.21.15.165:6835/2425395274","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":2425395274},{"type":"v1","addr":"172.21.15.165:6835","nonce":2425395274}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24637":{"gid":24637,"name":"i","rank":3,"incarnation":24,"state":"up:active","state_seq":18,"addr":"172.21.15.178:6839/70628097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6838","nonce":70628097},{"type":"v1","addr":"172.21.15.178:6839","nonce":70628097}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24638":{"gid":24638,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":18,"addr":"172.21.15.165:6837/6215088","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6836","nonce":6215088},{"type":"v1","addr":"172.21.15.165:6837","nonce":6215088}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24643":{"gid":24643,"name":"g","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.12:6839/3124093009","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6838","nonce":3124093009},{"type":"v1","addr":"172.21.15.12:6839","nonce":3124093009}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24674":{"gid":24674,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.165:6839/501312984","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6838","nonce":501312984},{"type":"v1","addr":"172.21.15.165:6839","nonce":501312984}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24674,"qdb_cluster":[24614,24637,24638,24643,24674]},"id":1}]} 2024-09-11T00:07:52.722 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 29 2024-09-11T00:07:52.723 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:52.722+0000 7efe6b600640 1 -- 172.21.15.12:0/3632627658 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7efe4c0b52e0 msgr2=0x7efe4c0b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:07:52.724 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:52.722+0000 7efe6b600640 1 --2- 172.21.15.12:0/3632627658 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7efe4c0b52e0 0x7efe4c0b77a0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7efe64109fa0 tx=0x7efe50009260 comp rx=0 tx=0).stop 2024-09-11T00:07:52.724 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:52.723+0000 7efe6b600640 1 -- 172.21.15.12:0/3632627658 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efe64104fa0 msgr2=0x7efe6419b490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:07:52.724 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:52.723+0000 7efe6b600640 1 --2- 172.21.15.12:0/3632627658 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efe64104fa0 0x7efe6419b490 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7efe580303f0 tx=0x7efe58032da0 comp rx=0 tx=0).stop 2024-09-11T00:07:52.724 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:52.723+0000 7efe6b600640 1 -- 172.21.15.12:0/3632627658 shutdown_connections 2024-09-11T00:07:52.724 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:52.723+0000 7efe6b600640 1 --2- 172.21.15.12:0/3632627658 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7efe4c0b52e0 0x7efe4c0b77a0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:07:52.724 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:52.723+0000 7efe6b600640 1 --2- 172.21.15.12:0/3632627658 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7efe6410b330 0x7efe641a0d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:07:52.725 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:52.723+0000 7efe6b600640 1 --2- 172.21.15.12:0/3632627658 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efe6410a9b0 0x7efe6419b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:07:52.725 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:52.723+0000 7efe6b600640 1 --2- 172.21.15.12:0/3632627658 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efe64104fa0 0x7efe6419b490 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:07:52.725 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:52.723+0000 7efe6b600640 1 -- 172.21.15.12:0/3632627658 >> 172.21.15.12:0/3632627658 conn(0x7efe64100a50 msgr2=0x7efe641013a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:07:52.725 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:52.723+0000 7efe6b600640 1 -- 172.21.15.12:0/3632627658 shutdown_connections 2024-09-11T00:07:52.725 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:52.723+0000 7efe6b600640 1 -- 172.21.15.12:0/3632627658 wait complete. 2024-09-11T00:07:53.492 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24674 quiesce db --set-id 6a295f84 --release --await' 2024-09-11T00:07:53.492 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid ff3ad74a-6fcf-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24674 quiesce db --set-id 6a295f84 --release --await 2024-09-11T00:07:53.813 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:07:53 smithi012 ceph-mon[30333]: pgmap v422: 129 pgs: 129 active+clean; 670 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 426 B/s wr, 11 op/s 2024-09-11T00:07:53.813 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:07:53 smithi012 ceph-mon[30333]: from='client.? 172.21.15.12:0/3632627658' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T00:07:53.925 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:07:54.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:07:53 smithi165 ceph-mon[35356]: pgmap v422: 129 pgs: 129 active+clean; 670 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 426 B/s wr, 11 op/s 2024-09-11T00:07:54.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:07:53 smithi165 ceph-mon[35356]: from='client.? 172.21.15.12:0/3632627658' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T00:07:54.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:07:53 smithi178 ceph-mon[35919]: pgmap v422: 129 pgs: 129 active+clean; 670 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 426 B/s wr, 11 op/s 2024-09-11T00:07:54.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:07:53 smithi178 ceph-mon[35919]: from='client.? 172.21.15.12:0/3632627658' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T00:07:54.921 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:54.497+0000 7f5e75a00640 1 -- 172.21.15.12:0/3337969451 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5e70108620 msgr2=0x7f5e70108a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:07:54.922 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:54.497+0000 7f5e75a00640 1 --2- 172.21.15.12:0/3337969451 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5e70108620 0x7f5e70108a60 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f5e58009900 tx=0x7f5e5802a410 comp rx=0 tx=0).stop 2024-09-11T00:07:54.922 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:54.497+0000 7f5e75a00640 1 -- 172.21.15.12:0/3337969451 shutdown_connections 2024-09-11T00:07:54.922 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:54.497+0000 7f5e75a00640 1 --2- 172.21.15.12:0/3337969451 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5e70108fa0 0x7f5e7010f7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:07:54.922 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:54.497+0000 7f5e75a00640 1 --2- 172.21.15.12:0/3337969451 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5e70108620 0x7f5e70108a60 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:07:54.922 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:54.497+0000 7f5e75a00640 1 --2- 172.21.15.12:0/3337969451 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5e70102c10 0x7f5e70102ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:07:54.922 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:54.497+0000 7f5e75a00640 1 -- 172.21.15.12:0/3337969451 >> 172.21.15.12:0/3337969451 conn(0x7f5e700fe660 msgr2=0x7f5e70100a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:07:54.922 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:54.497+0000 7f5e75a00640 1 -- 172.21.15.12:0/3337969451 shutdown_connections 2024-09-11T00:07:54.922 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:54.497+0000 7f5e75a00640 1 -- 172.21.15.12:0/3337969451 wait complete. 2024-09-11T00:07:54.922 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:54.498+0000 7f5e75a00640 1 Processor -- start 2024-09-11T00:07:54.922 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:54.498+0000 7f5e75a00640 1 -- start start 2024-09-11T00:07:54.923 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:54.499+0000 7f5e75a00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5e70102c10 0x7f5e70199040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:07:54.923 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:54.499+0000 7f5e75a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5e70108620 0x7f5e70199580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:07:54.923 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:54.499+0000 7f5e75a00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5e70108fa0 0x7f5e7019e920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:07:54.923 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:54.499+0000 7f5e75a00640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f5e70112c30 con 0x7f5e70102c10 2024-09-11T00:07:54.923 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:54.499+0000 7f5e75a00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f5e70112ab0 con 0x7f5e70108620 2024-09-11T00:07:54.923 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:54.499+0000 7f5e75a00640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f5e70112db0 con 0x7f5e70108fa0 2024-09-11T00:07:54.923 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:54.499+0000 7f5e6fe00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5e70102c10 0x7f5e70199040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:07:54.923 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:54.499+0000 7f5e6fe00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5e70102c10 0x7f5e70199040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:47490/0 (socket says 172.21.15.12:47490) 2024-09-11T00:07:54.923 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:54.499+0000 7f5e6fe00640 1 -- 172.21.15.12:0/2648313793 learned_addr learned my addr 172.21.15.12:0/2648313793 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:07:54.923 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:54.499+0000 7f5e67400640 1 --2- 172.21.15.12:0/2648313793 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5e70108620 0x7f5e70199580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:07:54.923 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:54.499+0000 7f5e75000640 1 --2- 172.21.15.12:0/2648313793 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5e70108fa0 0x7f5e7019e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:07:54.923 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:54.499+0000 7f5e6fe00640 1 -- 172.21.15.12:0/2648313793 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5e70108fa0 msgr2=0x7f5e7019e920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:07:54.923 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:54.499+0000 7f5e6fe00640 1 --2- 172.21.15.12:0/2648313793 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5e70108fa0 0x7f5e7019e920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:07:54.924 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:54.499+0000 7f5e6fe00640 1 -- 172.21.15.12:0/2648313793 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5e70108620 msgr2=0x7f5e70199580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:07:54.924 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:54.499+0000 7f5e6fe00640 1 --2- 172.21.15.12:0/2648313793 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5e70108620 0x7f5e70199580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:07:54.924 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:54.499+0000 7f5e6fe00640 1 -- 172.21.15.12:0/2648313793 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e7019f0f0 con 0x7f5e70102c10 2024-09-11T00:07:54.924 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:54.499+0000 7f5e67400640 1 --2- 172.21.15.12:0/2648313793 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5e70108620 0x7f5e70199580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T00:07:54.924 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:54.500+0000 7f5e6fe00640 1 --2- 172.21.15.12:0/2648313793 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5e70102c10 0x7f5e70199040 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f5e6000e9f0 tx=0x7f5e6000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:07:54.924 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:54.500+0000 7f5e6d600640 1 -- 172.21.15.12:0/2648313793 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e6000cd80 con 0x7f5e70102c10 2024-09-11T00:07:54.924 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:54.500+0000 7f5e6d600640 1 -- 172.21.15.12:0/2648313793 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5e600108e0 con 0x7f5e70102c10 2024-09-11T00:07:54.924 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:54.500+0000 7f5e75a00640 1 -- 172.21.15.12:0/2648313793 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5e7019f3e0 con 0x7f5e70102c10 2024-09-11T00:07:54.924 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:54.500+0000 7f5e6d600640 1 -- 172.21.15.12:0/2648313793 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e60018980 con 0x7f5e70102c10 2024-09-11T00:07:54.924 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:54.500+0000 7f5e75a00640 1 -- 172.21.15.12:0/2648313793 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5e701a5c70 con 0x7f5e70102c10 2024-09-11T00:07:54.924 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:54.503+0000 7f5e79c75740 1 Processor -- start 2024-09-11T00:07:54.924 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:54.503+0000 7f5e79c75740 1 -- start start 2024-09-11T00:07:54.924 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:54.504+0000 7f5e79c75740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f2c78ab1d0 0x55f2c78ab5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:07:54.925 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:54.504+0000 7f5e79c75740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f2c78abaf0 0x55f2c78bb950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:07:54.925 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:54.504+0000 7f5e79c75740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f2c78bbe90 0x55f2c78be280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:07:54.925 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:54.504+0000 7f5e79c75740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55f2c7822290 con 0x55f2c78abaf0 2024-09-11T00:07:54.925 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:54.504+0000 7f5e79c75740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55f2c78116c0 con 0x55f2c78bbe90 2024-09-11T00:07:54.925 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:54.504+0000 7f5e79c75740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55f2c78154d0 con 0x55f2c78ab1d0 2024-09-11T00:07:54.925 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:54.504+0000 7f5e6d600640 1 -- 172.21.15.12:0/2648313793 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7f5e60010430 con 0x7f5e70102c10 2024-09-11T00:07:54.925 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:54.504+0000 7f5e75000640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f2c78ab1d0 0x55f2c78ab5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:07:54.925 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:54.504+0000 7f5e75000640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f2c78ab1d0 0x55f2c78ab5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.12:47576/0 (socket says 172.21.15.12:47576) 2024-09-11T00:07:54.925 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:54.504+0000 7f5e75000640 1 -- 172.21.15.12:0/1749510417 learned_addr learned my addr 172.21.15.12:0/1749510417 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:07:54.925 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:54.505+0000 7f5e67400640 1 --2- 172.21.15.12:0/1749510417 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f2c78bbe90 0x55f2c78be280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:07:54.925 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:54.505+0000 7f5e6fe00640 1 --2- 172.21.15.12:0/1749510417 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f2c78abaf0 0x55f2c78bb950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:07:54.925 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:54.505+0000 7f5e75000640 1 -- 172.21.15.12:0/1749510417 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f2c78bbe90 msgr2=0x55f2c78be280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:07:54.926 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:54.505+0000 7f5e75000640 1 --2- 172.21.15.12:0/1749510417 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f2c78bbe90 0x55f2c78be280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:07:54.926 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:54.505+0000 7f5e75000640 1 -- 172.21.15.12:0/1749510417 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f2c78abaf0 msgr2=0x55f2c78bb950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:07:54.926 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:54.505+0000 7f5e75000640 1 --2- 172.21.15.12:0/1749510417 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f2c78abaf0 0x55f2c78bb950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:07:54.926 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:54.505+0000 7f5e75000640 1 -- 172.21.15.12:0/1749510417 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f2c779e460 con 0x55f2c78ab1d0 2024-09-11T00:07:54.926 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:54.505+0000 7f5e75000640 1 --2- 172.21.15.12:0/1749510417 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f2c78ab1d0 0x55f2c78ab5b0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f5e68010400 tx=0x7f5e680108c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d364e8883e2c7a49 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:07:54.926 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:54.505+0000 7f5e67400640 1 --2- 172.21.15.12:0/1749510417 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f2c78bbe90 0x55f2c78be280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T00:07:54.926 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:54.505+0000 7f5e65600640 1 -- 172.21.15.12:0/1749510417 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e68003b70 con 0x55f2c78ab1d0 2024-09-11T00:07:54.926 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:54.506+0000 7f5e65600640 1 -- 172.21.15.12:0/1749510417 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5e68003d10 con 0x55f2c78ab1d0 2024-09-11T00:07:54.926 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:54.506+0000 7f5e65600640 1 -- 172.21.15.12:0/1749510417 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e68011a60 con 0x55f2c78ab1d0 2024-09-11T00:07:54.926 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:54.506+0000 7f5e79c75740 1 -- 172.21.15.12:0/1749510417 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f2c78ab1d0 msgr2=0x55f2c78ab5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:07:54.926 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:54.506+0000 7f5e79c75740 1 --2- 172.21.15.12:0/1749510417 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f2c78ab1d0 0x55f2c78ab5b0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f5e68010400 tx=0x7f5e680108c0 comp rx=0 tx=0).stop 2024-09-11T00:07:54.926 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:54.506+0000 7f5e6d600640 1 --2- 172.21.15.12:0/2648313793 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f5e540b5490 0x7f5e540b7950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:07:54.926 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:54.507+0000 7f5e79c75740 1 -- 172.21.15.12:0/1749510417 shutdown_connections 2024-09-11T00:07:54.927 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:54.507+0000 7f5e79c75740 1 --2- 172.21.15.12:0/1749510417 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f2c78bbe90 0x55f2c78be280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:07:54.927 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:54.507+0000 7f5e79c75740 1 --2- 172.21.15.12:0/1749510417 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f2c78abaf0 0x55f2c78bb950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:07:54.927 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:54.507+0000 7f5e79c75740 1 --2- 172.21.15.12:0/1749510417 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f2c78ab1d0 0x55f2c78ab5b0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:07:54.927 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:54.507+0000 7f5e79c75740 1 -- 172.21.15.12:0/1749510417 >> 172.21.15.12:0/1749510417 conn(0x55f2c78a9000 msgr2=0x55f2c78aa1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:07:54.927 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:54.507+0000 7f5e6d600640 1 -- 172.21.15.12:0/2648313793 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9208+0+0 (secure 0 0 0) 0x7f5e600e63c0 con 0x7f5e70102c10 2024-09-11T00:07:54.927 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:54.507+0000 7f5e75000640 1 --2- 172.21.15.12:0/2648313793 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f5e540b5490 0x7f5e540b7950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:07:54.927 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:54.507+0000 7f5e79c75740 1 -- 172.21.15.12:0/1749510417 shutdown_connections 2024-09-11T00:07:54.927 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:54.507+0000 7f5e79c75740 1 -- 172.21.15.12:0/1749510417 wait complete. 2024-09-11T00:07:54.927 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:54.507+0000 7f5e75000640 1 --2- 172.21.15.12:0/2648313793 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f5e540b5490 0x7f5e540b7950 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f5e68004980 tx=0x7f5e68011560 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:07:54.927 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:54.507+0000 7f5e79c75740 1 -- start start 2024-09-11T00:07:54.927 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:54.508+0000 7f5e79c75740 1 Processor -- start 2024-09-11T00:07:54.927 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:54.508+0000 7f5e79c75740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f2c78ab1d0 0x55f2c78bb950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:07:54.927 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:54.508+0000 7f5e79c75740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f2c78abaf0 0x55f2c78b8de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:07:54.928 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:54.508+0000 7f5e79c75740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f2c78bbe90 0x55f2c78b9570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:07:54.928 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:54.508+0000 7f5e79c75740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55f2c7672bb0 con 0x55f2c78abaf0 2024-09-11T00:07:54.928 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:54.508+0000 7f5e79c75740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55f2c77ac6c0 con 0x55f2c78bbe90 2024-09-11T00:07:54.928 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:54.508+0000 7f5e79c75740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55f2c78beb40 con 0x55f2c78ab1d0 2024-09-11T00:07:54.928 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:54.508+0000 7f5e79c75740 10 client.0 _ll_register_callbacks cb 0x55f2c77c7490 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:07:54.928 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:54.508+0000 7f5e67400640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f2c78abaf0 0x55f2c78b8de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:07:54.928 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:54.509+0000 7f5e67400640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f2c78abaf0 0x55f2c78b8de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:47502/0 (socket says 172.21.15.12:47502) 2024-09-11T00:07:54.928 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:54.509+0000 7f5e67400640 1 -- 172.21.15.12:0/3838025742 learned_addr learned my addr 172.21.15.12:0/3838025742 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:07:54.928 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:54.509+0000 7f5e75000640 1 --2- 172.21.15.12:0/3838025742 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f2c78bbe90 0x55f2c78b9570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:07:54.928 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:54.509+0000 7f5e6fe00640 1 --2- 172.21.15.12:0/3838025742 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f2c78ab1d0 0x55f2c78bb950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:07:54.928 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:54.509+0000 7f5e67400640 1 -- 172.21.15.12:0/3838025742 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f2c78ab1d0 msgr2=0x55f2c78bb950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:07:54.928 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:54.509+0000 7f5e67400640 1 --2- 172.21.15.12:0/3838025742 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f2c78ab1d0 0x55f2c78bb950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:07:54.929 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:54.509+0000 7f5e67400640 1 -- 172.21.15.12:0/3838025742 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f2c78bbe90 msgr2=0x55f2c78b9570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:07:54.929 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:54.509+0000 7f5e67400640 1 --2- 172.21.15.12:0/3838025742 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f2c78bbe90 0x55f2c78b9570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:07:54.929 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:54.509+0000 7f5e67400640 1 -- 172.21.15.12:0/3838025742 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f2c756fb00 con 0x55f2c78abaf0 2024-09-11T00:07:54.929 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:54.509+0000 7f5e75000640 1 --2- 172.21.15.12:0/3838025742 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f2c78bbe90 0x55f2c78b9570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:07:54.929 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:54.510+0000 7f5e5e000640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-09-11T00:07:54.929 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:54.510+0000 7f5e67400640 1 --2- 172.21.15.12:0/3838025742 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f2c78abaf0 0x55f2c78b8de0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f5e58009900 tx=0x7f5e58031750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf46da438c2eb50d server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:07:54.929 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:54.510+0000 7f5e5e000640 1 -- 172.21.15.12:0/3838025742 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e58035030 con 0x55f2c78abaf0 2024-09-11T00:07:54.929 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:54.510+0000 7f5e79c75740 1 -- 172.21.15.12:0/3838025742 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5e68003d10 con 0x55f2c78abaf0 2024-09-11T00:07:54.929 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:54.510+0000 7f5e5e000640 1 -- 172.21.15.12:0/3838025742 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 9208+0+0 (secure 0 0 0) 0x7f5e58004460 con 0x55f2c78abaf0 2024-09-11T00:07:54.929 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:54.510+0000 7f5e5e000640 1 -- 172.21.15.12:0/3838025742 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f5e5804c050 con 0x55f2c78abaf0 2024-09-11T00:07:54.929 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:54.510+0000 7f5e79c75740 10 client.15576 fetch_fsmap learned FSMap version 29 2024-09-11T00:07:54.929 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:54.510+0000 7f5e79c75740 1 -- 172.21.15.12:0/3838025742 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55f2c75e6960 con 0x55f2c78abaf0 2024-09-11T00:07:54.929 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:54.511+0000 7f5e5e000640 1 -- 172.21.15.12:0/3838025742 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5e68003d10 con 0x55f2c78abaf0 2024-09-11T00:07:54.930 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:54.511+0000 7f5e5e000640 1 -- 172.21.15.12:0/3838025742 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7f5e580049c0 con 0x55f2c78abaf0 2024-09-11T00:07:54.930 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:54.511+0000 7f5e79c75740 10 client.15576 fetch_fsmap finished waiting for FSMap version 29 2024-09-11T00:07:54.930 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:54.511+0000 7f5e79c75740 10 client.15576 resolve_mds: validated gid 24674 aka daemon mds.b 2024-09-11T00:07:54.930 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:54.511+0000 7f5e79c75740 20 client.15576 populate_metadata read hostname 'smithi012' 2024-09-11T00:07:54.930 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:54.511+0000 7f5e79c75740 1 --2- 172.21.15.12:0/3838025742 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55f2c78cb9b0 0x55f2c78cddd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:07:54.930 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:54.511+0000 7f5e79c75740 4 client.15576 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T00:07:54.930 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:54.511+0000 7f5e79c75740 1 -- 172.21.15.12:0/3838025742 --> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f2c78241e0 con 0x55f2c78cb9b0 2024-09-11T00:07:54.930 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:54.511+0000 7f5e6fe00640 1 --2- 172.21.15.12:0/3838025742 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55f2c78cb9b0 0x55f2c78cddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:07:54.930 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:54.512+0000 7f5e6fe00640 1 --2- 172.21.15.12:0/3838025742 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55f2c78cb9b0 0x55f2c78cddd0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b065ec1f254f70a server_cookie=d07d6a6c71b8ea5 in_seq=0 out_seq=0 2024-09-11T00:07:54.930 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:54.512+0000 7f5e5e000640 10 client.15576 ms_handle_connect on v2:172.21.15.165:6838/501312984 2024-09-11T00:07:54.930 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:54.513+0000 7f5e5e000640 1 -- 172.21.15.12:0/3838025742 <== mds.0 v2:172.21.15.165:6838/501312984 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55f2c78241e0 con 0x55f2c78cb9b0 2024-09-11T00:07:54.930 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:54.513+0000 7f5e5e000640 10 client.15576 handle_command_reply: tid=0 2024-09-11T00:07:54.930 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:54.513+0000 7f5e79c75740 1 client.15576 shutdown 2024-09-11T00:07:54.930 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:54.513+0000 7f5e79c75740 1 -- 172.21.15.12:0/3838025742 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f2c78abaf0 msgr2=0x55f2c78b8de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:07:54.931 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:54.513+0000 7f5e79c75740 1 --2- 172.21.15.12:0/3838025742 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f2c78abaf0 0x55f2c78b8de0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f5e58009900 tx=0x7f5e58031750 comp rx=0 tx=0).stop 2024-09-11T00:07:54.931 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:54.513+0000 7f5e79c75740 1 -- 172.21.15.12:0/3838025742 shutdown_connections 2024-09-11T00:07:54.931 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:54.513+0000 7f5e79c75740 1 --2- 172.21.15.12:0/3838025742 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55f2c78cb9b0 0x55f2c78cddd0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:07:54.931 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:54.513+0000 7f5e79c75740 1 --2- 172.21.15.12:0/3838025742 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f2c78bbe90 0x55f2c78b9570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:07:54.931 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:54.513+0000 7f5e79c75740 1 --2- 172.21.15.12:0/3838025742 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f2c78abaf0 0x55f2c78b8de0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:07:54.931 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:54.513+0000 7f5e79c75740 1 --2- 172.21.15.12:0/3838025742 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f2c78ab1d0 0x55f2c78bb950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:07:54.931 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:54.513+0000 7f5e79c75740 1 -- 172.21.15.12:0/3838025742 >> 172.21.15.12:0/3838025742 conn(0x55f2c78a9000 msgr2=0x55f2c78aeb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:07:54.931 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:54.513+0000 7f5e5e000640 0 client.15576 ms_handle_reset on v2:172.21.15.165:6838/501312984 2024-09-11T00:07:54.931 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:54.513+0000 7f5e79c75740 1 -- 172.21.15.12:0/3838025742 shutdown_connections 2024-09-11T00:07:54.931 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:54.513+0000 7f5e79c75740 1 -- 172.21.15.12:0/3838025742 wait complete. 2024-09-11T00:07:54.931 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:54.513+0000 7f5e79c75740 20 client.15576 trim_cache size 0 max 16384 2024-09-11T00:07:54.931 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:54.544+0000 7f5e79c75740 1 Processor -- start 2024-09-11T00:07:54.931 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:54.544+0000 7f5e79c75740 1 -- start start 2024-09-11T00:07:54.932 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:54.544+0000 7f5e79c75740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f2c78ab1d0 0x55f2c78bde90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:07:54.932 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:54.544+0000 7f5e79c75740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f2c78abaf0 0x55f2c78b6920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:07:54.932 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:54.544+0000 7f5e79c75740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f2c78bbe90 0x55f2c78d36a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:07:54.932 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:54.544+0000 7f5e79c75740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f5e701ad7c0 con 0x55f2c78abaf0 2024-09-11T00:07:54.932 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:54.544+0000 7f5e79c75740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f5e701af300 con 0x55f2c78bbe90 2024-09-11T00:07:54.932 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:54.544+0000 7f5e79c75740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f5e701ade90 con 0x55f2c78ab1d0 2024-09-11T00:07:54.932 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:54.544+0000 7f5e6fe00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f2c78ab1d0 0x55f2c78bde90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:07:54.932 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:54.544+0000 7f5e6fe00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f2c78ab1d0 0x55f2c78bde90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.12:47586/0 (socket says 172.21.15.12:47586) 2024-09-11T00:07:54.932 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:54.544+0000 7f5e6fe00640 1 -- 172.21.15.12:0/2287248181 learned_addr learned my addr 172.21.15.12:0/2287248181 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:07:54.932 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:54.544+0000 7f5e67400640 1 --2- 172.21.15.12:0/2287248181 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f2c78abaf0 0x55f2c78b6920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:07:54.932 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:54.544+0000 7f5e75000640 1 --2- 172.21.15.12:0/2287248181 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f2c78bbe90 0x55f2c78d36a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:07:54.932 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:54.545+0000 7f5e6fe00640 1 -- 172.21.15.12:0/2287248181 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f2c78bbe90 msgr2=0x55f2c78d36a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:07:54.932 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:54.545+0000 7f5e6fe00640 1 --2- 172.21.15.12:0/2287248181 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f2c78bbe90 0x55f2c78d36a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:07:54.933 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:54.545+0000 7f5e6fe00640 1 -- 172.21.15.12:0/2287248181 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f2c78abaf0 msgr2=0x55f2c78b6920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:07:54.933 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:54.545+0000 7f5e6fe00640 1 --2- 172.21.15.12:0/2287248181 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f2c78abaf0 0x55f2c78b6920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:07:54.933 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:54.545+0000 7f5e6fe00640 1 -- 172.21.15.12:0/2287248181 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f2c78ab920 con 0x55f2c78ab1d0 2024-09-11T00:07:54.933 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:54.545+0000 7f5e6fe00640 1 --2- 172.21.15.12:0/2287248181 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f2c78ab1d0 0x55f2c78bde90 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f5e600a8d20 tx=0x7f5e6000cf20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=92cd971d23c3e782 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:07:54.933 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:54.545+0000 7f5e64c00640 1 -- 172.21.15.12:0/2287248181 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e600abec0 con 0x55f2c78ab1d0 2024-09-11T00:07:54.933 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:54.545+0000 7f5e64c00640 1 -- 172.21.15.12:0/2287248181 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5e600a6e50 con 0x55f2c78ab1d0 2024-09-11T00:07:54.933 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:54.545+0000 7f5e64c00640 1 -- 172.21.15.12:0/2287248181 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e600b5ce0 con 0x55f2c78ab1d0 2024-09-11T00:07:54.933 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:54.546+0000 7f5e79c75740 1 -- 172.21.15.12:0/2287248181 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f2c78ab1d0 msgr2=0x55f2c78bde90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:07:54.933 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:54.546+0000 7f5e79c75740 1 --2- 172.21.15.12:0/2287248181 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f2c78ab1d0 0x55f2c78bde90 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f5e600a8d20 tx=0x7f5e6000cf20 comp rx=0 tx=0).stop 2024-09-11T00:07:54.933 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:54.546+0000 7f5e79c75740 1 -- 172.21.15.12:0/2287248181 shutdown_connections 2024-09-11T00:07:54.933 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:54.546+0000 7f5e79c75740 1 --2- 172.21.15.12:0/2287248181 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f2c78bbe90 0x55f2c78d36a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:07:54.933 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:54.546+0000 7f5e79c75740 1 --2- 172.21.15.12:0/2287248181 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f2c78abaf0 0x55f2c78b6920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:07:54.933 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:54.546+0000 7f5e79c75740 1 --2- 172.21.15.12:0/2287248181 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f2c78ab1d0 0x55f2c78bde90 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:07:54.934 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:54.546+0000 7f5e79c75740 1 -- 172.21.15.12:0/2287248181 >> 172.21.15.12:0/2287248181 conn(0x55f2c78a9000 msgr2=0x55f2c78b7620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:07:54.934 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:54.546+0000 7f5e79c75740 1 -- 172.21.15.12:0/2287248181 shutdown_connections 2024-09-11T00:07:54.934 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:54.546+0000 7f5e79c75740 1 -- 172.21.15.12:0/2287248181 wait complete. 2024-09-11T00:07:54.934 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:54.547+0000 7f5e79c75740 1 -- start start 2024-09-11T00:07:54.934 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:54.547+0000 7f5e79c75740 1 Processor -- start 2024-09-11T00:07:54.934 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:54.547+0000 7f5e79c75740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f2c78ab1d0 0x55f2c78c43e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:07:54.934 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:54.547+0000 7f5e79c75740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f2c78abaf0 0x55f2c78c0dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:07:54.934 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:54.547+0000 7f5e79c75740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f2c78bbe90 0x55f2c78c1310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:07:54.934 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:54.547+0000 7f5e79c75740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f5e701aec30 con 0x55f2c78ab1d0 2024-09-11T00:07:54.934 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:54.547+0000 7f5e79c75740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f5e701ae560 con 0x55f2c78abaf0 2024-09-11T00:07:54.934 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:54.547+0000 7f5e79c75740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f5e701ab7a0 con 0x55f2c78bbe90 2024-09-11T00:07:54.935 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:54.547+0000 7f5e79c75740 10 client.0 _ll_register_callbacks cb 0x7f5e60005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:07:54.935 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:54.548+0000 7f5e6fe00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f2c78ab1d0 0x55f2c78c43e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:07:54.935 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:54.548+0000 7f5e6fe00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f2c78ab1d0 0x55f2c78c43e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:47522/0 (socket says 172.21.15.12:47522) 2024-09-11T00:07:54.935 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:54.548+0000 7f5e6fe00640 1 -- 172.21.15.12:0/2504039304 learned_addr learned my addr 172.21.15.12:0/2504039304 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:07:54.935 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:54.548+0000 7f5e75000640 1 --2- 172.21.15.12:0/2504039304 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f2c78bbe90 0x55f2c78c1310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:07:54.935 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:54.548+0000 7f5e67400640 1 --2- 172.21.15.12:0/2504039304 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f2c78abaf0 0x55f2c78c0dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:07:54.935 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:54.548+0000 7f5e6fe00640 1 -- 172.21.15.12:0/2504039304 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f2c78bbe90 msgr2=0x55f2c78c1310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:07:54.935 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:54.548+0000 7f5e6fe00640 1 --2- 172.21.15.12:0/2504039304 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f2c78bbe90 0x55f2c78c1310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:07:54.935 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:54.548+0000 7f5e6fe00640 1 -- 172.21.15.12:0/2504039304 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f2c78abaf0 msgr2=0x55f2c78c0dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:07:54.935 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:54.548+0000 7f5e6fe00640 1 --2- 172.21.15.12:0/2504039304 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f2c78abaf0 0x55f2c78c0dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:07:54.935 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:54.548+0000 7f5e6fe00640 1 -- 172.21.15.12:0/2504039304 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f2c78c1980 con 0x55f2c78ab1d0 2024-09-11T00:07:54.935 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:54.548+0000 7f5e6fe00640 1 --2- 172.21.15.12:0/2504039304 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f2c78ab1d0 0x55f2c78c43e0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f5e600a99c0 tx=0x7f5e600107d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a107678ad6227fe6 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:07:54.935 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:54.548+0000 7f5e5e000640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-09-11T00:07:54.936 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:54.549+0000 7f5e5e000640 1 -- 172.21.15.12:0/2504039304 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e600ac750 con 0x55f2c78ab1d0 2024-09-11T00:07:54.936 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:54.549+0000 7f5e5e000640 1 -- 172.21.15.12:0/2504039304 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 9208+0+0 (secure 0 0 0) 0x7f5e600a89c0 con 0x55f2c78ab1d0 2024-09-11T00:07:54.936 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:54.549+0000 7f5e79c75740 1 -- 172.21.15.12:0/2504039304 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5e600a6e50 con 0x55f2c78ab1d0 2024-09-11T00:07:54.936 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:54.549+0000 7f5e5e000640 1 -- 172.21.15.12:0/2504039304 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f5e600ac260 con 0x55f2c78ab1d0 2024-09-11T00:07:54.936 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:54.549+0000 7f5e79c75740 10 client.15582 fetch_fsmap learned FSMap version 29 2024-09-11T00:07:54.936 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:54.549+0000 7f5e79c75740 1 -- 172.21.15.12:0/2504039304 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55f2c79397a0 con 0x55f2c78ab1d0 2024-09-11T00:07:54.936 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:54.549+0000 7f5e5e000640 1 -- 172.21.15.12:0/2504039304 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5e600a6e50 con 0x55f2c78ab1d0 2024-09-11T00:07:54.936 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:54.549+0000 7f5e5e000640 1 -- 172.21.15.12:0/2504039304 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7f5e60018b20 con 0x55f2c78ab1d0 2024-09-11T00:07:54.936 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:54.549+0000 7f5e79c75740 10 client.15582 fetch_fsmap finished waiting for FSMap version 29 2024-09-11T00:07:54.936 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:54.549+0000 7f5e79c75740 10 client.15582 resolve_mds: validated gid 24674 aka daemon mds.b 2024-09-11T00:07:54.936 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:54.549+0000 7f5e79c75740 20 client.15582 populate_metadata read hostname 'smithi012' 2024-09-11T00:07:54.936 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:54.550+0000 7f5e79c75740 1 --2- 172.21.15.12:0/2504039304 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55f2c7939eb0 0x55f2c793c2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:07:54.936 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:54.550+0000 7f5e79c75740 4 client.15582 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "6a295f84", "release": true, "await": true}] 2024-09-11T00:07:54.937 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:54.550+0000 7f5e79c75740 1 -- 172.21.15.12:0/2504039304 --> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] -- command(tid 0: {"prefix": "quiesce db", "set_id": "6a295f84", "release": true, "await": true}) -- 0x55f2c781fc90 con 0x55f2c7939eb0 2024-09-11T00:07:54.937 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:54.550+0000 7f5e67400640 1 --2- 172.21.15.12:0/2504039304 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55f2c7939eb0 0x55f2c793c2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:07:54.937 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:54.550+0000 7f5e67400640 1 --2- 172.21.15.12:0/2504039304 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55f2c7939eb0 0x55f2c793c2d0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b7373e7f3af6187b server_cookie=eb16bb52967120a0 in_seq=0 out_seq=0 2024-09-11T00:07:54.937 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:54.550+0000 7f5e5e000640 10 client.15582 ms_handle_connect on v2:172.21.15.165:6838/501312984 2024-09-11T00:07:54.937 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:54.552+0000 7f5e5e000640 1 -- 172.21.15.12:0/2504039304 <== mds.0 v2:172.21.15.165:6838/501312984 1 ==== command_reply(tid 0: 0 ) ==== 8+0+589 (crc 0 0 0) 0x55f2c781fc90 con 0x55f2c7939eb0 2024-09-11T00:07:54.937 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:54.552+0000 7f5e5e000640 10 client.15582 handle_command_reply: tid=0 2024-09-11T00:07:54.937 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:54.552+0000 7f5e79c75740 1 client.15582 shutdown 2024-09-11T00:07:54.937 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:54.552+0000 7f5e79c75740 1 -- 172.21.15.12:0/2504039304 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f2c78ab1d0 msgr2=0x55f2c78c43e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:07:54.937 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:54.552+0000 7f5e79c75740 1 --2- 172.21.15.12:0/2504039304 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f2c78ab1d0 0x55f2c78c43e0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f5e600a99c0 tx=0x7f5e600107d0 comp rx=0 tx=0).stop 2024-09-11T00:07:54.937 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-09-11T00:07:54.938 INFO:teuthology.orchestra.run.smithi012.stdout: "epoch": 29, 2024-09-11T00:07:54.938 INFO:teuthology.orchestra.run.smithi012.stdout: "leader": 24674, 2024-09-11T00:07:54.938 INFO:teuthology.orchestra.run.smithi012.stdout: "set_version": 4, 2024-09-11T00:07:54.938 INFO:teuthology.orchestra.run.smithi012.stdout: "sets": { 2024-09-11T00:07:54.938 INFO:teuthology.orchestra.run.smithi012.stdout: "6a295f84": { 2024-09-11T00:07:54.938 INFO:teuthology.orchestra.run.smithi012.stdout: "version": 4, 2024-09-11T00:07:54.938 INFO:teuthology.orchestra.run.smithi012.stdout: "age_ref": 0.0, 2024-09-11T00:07:54.938 INFO:teuthology.orchestra.run.smithi012.stdout: "state": { 2024-09-11T00:07:54.938 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "RELEASED", 2024-09-11T00:07:54.938 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 0.0 2024-09-11T00:07:54.938 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:07:54.938 INFO:teuthology.orchestra.run.smithi012.stdout: "timeout": 90.0, 2024-09-11T00:07:54.939 INFO:teuthology.orchestra.run.smithi012.stdout: "expiration": 145.2, 2024-09-11T00:07:54.939 INFO:teuthology.orchestra.run.smithi012.stdout: "members": { 2024-09-11T00:07:54.939 INFO:teuthology.orchestra.run.smithi012.stdout: "file:/": { 2024-09-11T00:07:54.939 INFO:teuthology.orchestra.run.smithi012.stdout: "excluded": false, 2024-09-11T00:07:54.939 INFO:teuthology.orchestra.run.smithi012.stdout: "state": { 2024-09-11T00:07:54.939 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "RELEASED", 2024-09-11T00:07:54.939 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 0.0 2024-09-11T00:07:54.939 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:07:54.939 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:07:54.939 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:07:54.939 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:07:54.939 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:07:54.939 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-09-11T00:07:54.940 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:54.553+0000 7f5e79c75740 1 -- 172.21.15.12:0/2504039304 shutdown_connections 2024-09-11T00:07:54.940 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:54.553+0000 7f5e79c75740 1 --2- 172.21.15.12:0/2504039304 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55f2c7939eb0 0x55f2c793c2d0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:07:54.940 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:54.553+0000 7f5e79c75740 1 --2- 172.21.15.12:0/2504039304 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f2c78bbe90 0x55f2c78c1310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:07:54.940 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:54.553+0000 7f5e79c75740 1 --2- 172.21.15.12:0/2504039304 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f2c78abaf0 0x55f2c78c0dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:07:54.940 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:54.553+0000 7f5e79c75740 1 --2- 172.21.15.12:0/2504039304 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f2c78ab1d0 0x55f2c78c43e0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:07:54.940 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:54.553+0000 7f5e79c75740 1 -- 172.21.15.12:0/2504039304 >> 172.21.15.12:0/2504039304 conn(0x55f2c78a9000 msgr2=0x55f2c78c6d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:07:54.940 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:54.553+0000 7f5e5e000640 0 client.15582 ms_handle_reset on v2:172.21.15.165:6838/501312984 2024-09-11T00:07:54.940 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:54.553+0000 7f5e79c75740 1 -- 172.21.15.12:0/2504039304 shutdown_connections 2024-09-11T00:07:54.941 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:54.553+0000 7f5e79c75740 1 -- 172.21.15.12:0/2504039304 wait complete. 2024-09-11T00:07:54.941 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:54.553+0000 7f5e79c75740 20 client.15582 trim_cache size 0 max 16384 2024-09-11T00:07:54.941 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:54.554+0000 7f5e75a00640 1 -- 172.21.15.12:0/2648313793 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f5e540b5490 msgr2=0x7f5e540b7950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:07:54.941 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:54.554+0000 7f5e75a00640 1 --2- 172.21.15.12:0/2648313793 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f5e540b5490 0x7f5e540b7950 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f5e68004980 tx=0x7f5e68011560 comp rx=0 tx=0).stop 2024-09-11T00:07:54.941 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:54.554+0000 7f5e75a00640 1 -- 172.21.15.12:0/2648313793 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5e70102c10 msgr2=0x7f5e70199040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:07:54.941 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:54.554+0000 7f5e75a00640 1 --2- 172.21.15.12:0/2648313793 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5e70102c10 0x7f5e70199040 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f5e6000e9f0 tx=0x7f5e6000eeb0 comp rx=0 tx=0).stop 2024-09-11T00:07:54.941 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:54.554+0000 7f5e75a00640 1 -- 172.21.15.12:0/2648313793 shutdown_connections 2024-09-11T00:07:54.941 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:54.554+0000 7f5e75a00640 1 --2- 172.21.15.12:0/2648313793 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f5e540b5490 0x7f5e540b7950 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:07:54.941 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:54.554+0000 7f5e75a00640 1 --2- 172.21.15.12:0/2648313793 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5e70108fa0 0x7f5e7019e920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:07:54.941 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:54.554+0000 7f5e75a00640 1 --2- 172.21.15.12:0/2648313793 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5e70108620 0x7f5e70199580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:07:54.941 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:54.554+0000 7f5e75a00640 1 --2- 172.21.15.12:0/2648313793 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5e70102c10 0x7f5e70199040 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:07:54.941 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:54.554+0000 7f5e75a00640 1 -- 172.21.15.12:0/2648313793 >> 172.21.15.12:0/2648313793 conn(0x7f5e700fe660 msgr2=0x7f5e70100a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:07:54.941 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:54.554+0000 7f5e75a00640 1 -- 172.21.15.12:0/2648313793 shutdown_connections 2024-09-11T00:07:54.941 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:07:54.554+0000 7f5e75a00640 1 -- 172.21.15.12:0/2648313793 wait complete. 2024-09-11T00:07:55.813 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:07:55 smithi012 ceph-mon[30333]: pgmap v423: 129 pgs: 129 active+clean; 670 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-09-11T00:07:56.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:07:55 smithi165 ceph-mon[35356]: pgmap v423: 129 pgs: 129 active+clean; 670 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-09-11T00:07:56.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:07:55 smithi178 ceph-mon[35919]: pgmap v423: 129 pgs: 129 active+clean; 670 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-09-11T00:07:58.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:07:57 smithi165 ceph-mon[35356]: pgmap v424: 129 pgs: 129 active+clean; 671 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 341 B/s wr, 13 op/s 2024-09-11T00:07:58.063 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:07:57 smithi012 ceph-mon[30333]: pgmap v424: 129 pgs: 129 active+clean; 671 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 341 B/s wr, 13 op/s 2024-09-11T00:07:58.080 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:07:57 smithi178 ceph-mon[35919]: pgmap v424: 129 pgs: 129 active+clean; 671 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 341 B/s wr, 13 op/s 2024-09-11T00:07:58.820 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '6a295f84', total seconds elapsed: 32.8 2024-09-11T00:08:00.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:07:59 smithi165 ceph-mon[35356]: pgmap v425: 129 pgs: 129 active+clean; 671 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-09-11T00:08:00.062 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:07:59 smithi012 ceph-mon[30333]: pgmap v425: 129 pgs: 129 active+clean; 671 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-09-11T00:08:00.080 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:07:59 smithi178 ceph-mon[35919]: pgmap v425: 129 pgs: 129 active+clean; 671 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-09-11T00:08:00.905 INFO:tasks.workunit.client.0.smithi012.stderr:Updating files: 63% (7699/12088) Updating files: 64% (7737/12088) Updating files: 65% (7858/12088) Updating files: 66% (7979/12088) Updating files: 67% (8099/12088) Updating files: 68% (8220/12088) Updating files: 69% (8341/12088) Updating files: 70% (8462/12088) Updating files: 71% (8583/12088) Updating files: 72% (8704/12088) Updating files: 73% (8825/12088) Updating files: 74% (8946/12088) Updating files: 75% (9066/12088) Updating files: 76% (9187/12088) Updating files: 77% (9308/12088) Updating files: 78% (9429/12088) Updating files: 79% (9550/12088) Updating files: 80% (9671/12088) Updating files: 81% (9792/12088) Updating files: 82% (9913/12088) Updating files: 83% (10034/12088) Updating files: 84% (10154/12088) Updating files: 85% (10275/12088) Updating files: 86% (10396/12088) Updating files: 87% (10517/12088) Updating files: 88% (10638/12088) Updating files: 89% (10759/12088) Updating files: 90% (10880/12088) Updating files: 91% (11001/12088) Updating files: 92% (11121/12088) Updating files: 93% (11242/12088) Updating files: 94% (11363/12088) Updating files: 95% (11484/12088) Updating files: 96% (11605/12088) Updating files: 97% (11726/12088) Updating files: 98% (11847/12088) Updating files: 99% (11968/12088) Updating files: 100% (12088/12088) Updating files: 100% (12088/12088), done. 2024-09-11T00:08:02.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:08:01 smithi165 ceph-mon[35356]: pgmap v426: 129 pgs: 129 active+clean; 671 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 255 B/s wr, 10 op/s 2024-09-11T00:08:02.062 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:08:01 smithi012 ceph-mon[30333]: pgmap v426: 129 pgs: 129 active+clean; 671 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 255 B/s wr, 10 op/s 2024-09-11T00:08:02.080 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:08:01 smithi178 ceph-mon[35919]: pgmap v426: 129 pgs: 129 active+clean; 671 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 255 B/s wr, 10 op/s 2024-09-11T00:08:02.335 INFO:tasks.workunit.client.0.smithi012.stderr:Note: switching to '8f219c476cb4f26fc5719c661c0323719a45cb30'. 2024-09-11T00:08:02.335 INFO:tasks.workunit.client.0.smithi012.stderr: 2024-09-11T00:08:02.335 INFO:tasks.workunit.client.0.smithi012.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-09-11T00:08:02.335 INFO:tasks.workunit.client.0.smithi012.stderr:changes and commit them, and you can discard any commits you make in this 2024-09-11T00:08:02.335 INFO:tasks.workunit.client.0.smithi012.stderr:state without impacting any branches by switching back to a branch. 2024-09-11T00:08:02.335 INFO:tasks.workunit.client.0.smithi012.stderr: 2024-09-11T00:08:02.335 INFO:tasks.workunit.client.0.smithi012.stderr:If you want to create a new branch to retain commits you create, you may 2024-09-11T00:08:02.335 INFO:tasks.workunit.client.0.smithi012.stderr:do so (now or later) by using -c with the switch command. Example: 2024-09-11T00:08:02.335 INFO:tasks.workunit.client.0.smithi012.stderr: 2024-09-11T00:08:02.335 INFO:tasks.workunit.client.0.smithi012.stderr: git switch -c 2024-09-11T00:08:02.335 INFO:tasks.workunit.client.0.smithi012.stderr: 2024-09-11T00:08:02.336 INFO:tasks.workunit.client.0.smithi012.stderr:Or undo this operation with: 2024-09-11T00:08:02.336 INFO:tasks.workunit.client.0.smithi012.stderr: 2024-09-11T00:08:02.336 INFO:tasks.workunit.client.0.smithi012.stderr: git switch - 2024-09-11T00:08:02.336 INFO:tasks.workunit.client.0.smithi012.stderr: 2024-09-11T00:08:02.336 INFO:tasks.workunit.client.0.smithi012.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-09-11T00:08:02.336 INFO:tasks.workunit.client.0.smithi012.stderr: 2024-09-11T00:08:02.336 INFO:tasks.workunit.client.0.smithi012.stderr:HEAD is now at 8f219c476cb Merge branch 'mgr-vol-mod-disable' of https://github.com/rishabh-d-dave/ceph into wip-hemanth-testing-2024-09-10-0723 2024-09-11T00:08:02.348 DEBUG:teuthology.orchestra.run.smithi012:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-09-11T00:08:02.394 INFO:tasks.workunit.client.0.smithi012.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-09-11T00:08:02.399 INFO:tasks.workunit.client.0.smithi012.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-09-11T00:08:02.399 INFO:tasks.workunit.client.0.smithi012.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-09-11T00:08:03.081 INFO:tasks.workunit.client.0.smithi012.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-09-11T00:08:03.170 INFO:tasks.workunit.client.0.smithi012.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-09-11T00:08:03.236 INFO:tasks.workunit.client.0.smithi012.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-09-11T00:08:03.239 INFO:tasks.workunit.client.0.smithi012.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-09-11T00:08:03.239 INFO:tasks.workunit.client.0.smithi012.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-09-11T00:08:03.305 INFO:tasks.workunit.client.0.smithi012.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-09-11T00:08:03.312 DEBUG:teuthology.orchestra.run.smithi012:> set -ex 2024-09-11T00:08:03.312 DEBUG:teuthology.orchestra.run.smithi012:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-09-11T00:08:03.388 INFO:tasks.workunit:Running workunits matching suites/dbench.sh on client.0... 2024-09-11T00:08:03.390 INFO:tasks.workunit:Running workunit suites/dbench.sh... 2024-09-11T00:08:03.390 DEBUG:teuthology.orchestra.run.smithi012:workunit test suites/dbench.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=8f219c476cb4f26fc5719c661c0323719a45cb30 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/dbench.sh 2024-09-11T00:08:03.478 INFO:tasks.workunit.client.0.smithi012.stderr:+ dbench 1 2024-09-11T00:08:03.487 INFO:tasks.workunit.client.0.smithi012.stdout:dbench version 4.00 - Copyright Andrew Tridgell 1999-2004 2024-09-11T00:08:03.487 INFO:tasks.workunit.client.0.smithi012.stdout: 2024-09-11T00:08:03.487 INFO:tasks.workunit.client.0.smithi012.stdout:Running for 600 seconds with load '/usr/share/dbench/client.txt' and minimum warmup 120 secs 2024-09-11T00:08:03.488 INFO:tasks.workunit.client.0.smithi012.stdout:0 of 1 processes prepared for launch 0 sec 2024-09-11T00:08:03.588 INFO:tasks.workunit.client.0.smithi012.stdout:1 of 1 processes prepared for launch 0 sec 2024-09-11T00:08:03.588 INFO:tasks.workunit.client.0.smithi012.stdout:releasing clients 2024-09-11T00:08:04.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:08:03 smithi165 ceph-mon[35356]: pgmap v427: 129 pgs: 129 active+clean; 671 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 255 B/s wr, 11 op/s 2024-09-11T00:08:04.062 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:08:03 smithi012 ceph-mon[30333]: pgmap v427: 129 pgs: 129 active+clean; 671 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 255 B/s wr, 11 op/s 2024-09-11T00:08:04.080 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:08:03 smithi178 ceph-mon[35919]: pgmap v427: 129 pgs: 129 active+clean; 671 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 255 B/s wr, 11 op/s 2024-09-11T00:08:04.580 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:04.579+0000 7f1fbee00640 1 -- 172.21.15.12:0/2879739742 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1fb8105b00 msgr2=0x7f1fb8105f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:08:04.581 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:04.579+0000 7f1fbee00640 1 --2- 172.21.15.12:0/2879739742 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1fb8105b00 0x7f1fb8105f80 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f1fa4009900 tx=0x7f1fa402a3f0 comp rx=0 tx=0).stop 2024-09-11T00:08:04.581 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:04.579+0000 7f1fbee00640 1 -- 172.21.15.12:0/2879739742 shutdown_connections 2024-09-11T00:08:04.581 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:04.579+0000 7f1fbee00640 1 --2- 172.21.15.12:0/2879739742 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1fb81064c0 0x7f1fb810ce40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:08:04.581 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:04.579+0000 7f1fbee00640 1 --2- 172.21.15.12:0/2879739742 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1fb8105b00 0x7f1fb8105f80 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:08:04.581 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:04.579+0000 7f1fbee00640 1 --2- 172.21.15.12:0/2879739742 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1fb81049b0 0x7f1fb8104db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:08:04.581 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:04.579+0000 7f1fbee00640 1 -- 172.21.15.12:0/2879739742 >> 172.21.15.12:0/2879739742 conn(0x7f1fb8100270 msgr2=0x7f1fb8102690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:08:04.581 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:04.579+0000 7f1fbee00640 1 -- 172.21.15.12:0/2879739742 shutdown_connections 2024-09-11T00:08:04.581 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:04.579+0000 7f1fbee00640 1 -- 172.21.15.12:0/2879739742 wait complete. 2024-09-11T00:08:04.584 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:04.580+0000 7f1fbee00640 1 Processor -- start 2024-09-11T00:08:04.584 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:04.580+0000 7f1fbee00640 1 -- start start 2024-09-11T00:08:04.584 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:04.580+0000 7f1fbee00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1fb81049b0 0x7f1fb819ad50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:08:04.584 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:04.580+0000 7f1fbee00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1fb8105b00 0x7f1fb819b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:08:04.584 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:04.580+0000 7f1fbee00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1fb81064c0 0x7f1fb819bfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:08:04.584 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:04.580+0000 7f1fbee00640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f1fb8110670 con 0x7f1fb81064c0 2024-09-11T00:08:04.584 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:04.580+0000 7f1fbee00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f1fb81104f0 con 0x7f1fb8105b00 2024-09-11T00:08:04.584 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:04.580+0000 7f1fbee00640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f1fb81107f0 con 0x7f1fb81049b0 2024-09-11T00:08:04.585 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:04.581+0000 7f1fbd000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1fb8105b00 0x7f1fb819b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:08:04.585 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:04.581+0000 7f1fbd000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1fb8105b00 0x7f1fb819b3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.12:34978/0 (socket says 172.21.15.12:34978) 2024-09-11T00:08:04.585 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:04.581+0000 7f1fbd000640 1 -- 172.21.15.12:0/1222628100 learned_addr learned my addr 172.21.15.12:0/1222628100 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:08:04.585 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:04.581+0000 7f1fbe400640 1 --2- 172.21.15.12:0/1222628100 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1fb81064c0 0x7f1fb819bfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:08:04.585 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:04.581+0000 7f1fbda00640 1 --2- 172.21.15.12:0/1222628100 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1fb81049b0 0x7f1fb819ad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:08:04.585 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:04.581+0000 7f1fbd000640 1 -- 172.21.15.12:0/1222628100 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1fb81049b0 msgr2=0x7f1fb819ad50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:08:04.585 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:04.581+0000 7f1fbd000640 1 --2- 172.21.15.12:0/1222628100 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1fb81049b0 0x7f1fb819ad50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:08:04.585 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:04.581+0000 7f1fbd000640 1 -- 172.21.15.12:0/1222628100 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1fb81064c0 msgr2=0x7f1fb819bfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:08:04.585 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:04.581+0000 7f1fbd000640 1 --2- 172.21.15.12:0/1222628100 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1fb81064c0 0x7f1fb819bfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:08:04.585 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:04.581+0000 7f1fbd000640 1 -- 172.21.15.12:0/1222628100 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1fb819c890 con 0x7f1fb8105b00 2024-09-11T00:08:04.586 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:04.582+0000 7f1fbd000640 1 --2- 172.21.15.12:0/1222628100 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1fb8105b00 0x7f1fb819b3a0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f1fa4031490 tx=0x7f1fa4003c10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:08:04.586 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:04.582+0000 7f1fb6000640 1 -- 172.21.15.12:0/1222628100 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fa403f070 con 0x7f1fb8105b00 2024-09-11T00:08:04.586 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:04.582+0000 7f1fbee00640 1 -- 172.21.15.12:0/1222628100 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1fb819cb20 con 0x7f1fb8105b00 2024-09-11T00:08:04.586 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:04.582+0000 7f1fb6000640 1 -- 172.21.15.12:0/1222628100 <== mon.2 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1fa4031bf0 con 0x7f1fb8105b00 2024-09-11T00:08:04.586 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:04.582+0000 7f1fb6000640 1 -- 172.21.15.12:0/1222628100 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fa4035030 con 0x7f1fb8105b00 2024-09-11T00:08:04.586 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:04.582+0000 7f1fbee00640 1 -- 172.21.15.12:0/1222628100 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1fb81a3250 con 0x7f1fb8105b00 2024-09-11T00:08:04.587 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:04.586+0000 7f1fc31d1740 1 Processor -- start 2024-09-11T00:08:04.587 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:04.586+0000 7f1fc31d1740 1 -- start start 2024-09-11T00:08:04.588 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:04.587+0000 7f1fc31d1740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555eb8cd8970 0x555eb8cdad60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:08:04.588 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:04.587+0000 7f1fc31d1740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555eb8cdb2a0 0x555eb8cdd690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:08:04.588 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:04.587+0000 7f1fc31d1740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555eb8cddce0 0x555eb8ce01a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:08:04.588 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:04.587+0000 7f1fc31d1740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x555eb8c4b6b0 con 0x555eb8cd8970 2024-09-11T00:08:04.589 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:04.587+0000 7f1fc31d1740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x555eb8c3aae0 con 0x555eb8cddce0 2024-09-11T00:08:04.589 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:04.587+0000 7f1fc31d1740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x555eb8c3e8f0 con 0x555eb8cdb2a0 2024-09-11T00:08:04.589 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:04.587+0000 7f1fbe400640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555eb8cd8970 0x555eb8cdad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:08:04.589 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:04.587+0000 7f1fbe400640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555eb8cd8970 0x555eb8cdad60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:50064/0 (socket says 172.21.15.12:50064) 2024-09-11T00:08:04.589 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:04.587+0000 7f1fbe400640 1 -- 172.21.15.12:0/1810373372 learned_addr learned my addr 172.21.15.12:0/1810373372 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:08:04.589 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:04.587+0000 7f1fbda00640 1 --2- 172.21.15.12:0/1810373372 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555eb8cdb2a0 0x555eb8cdd690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:08:04.589 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:04.587+0000 7f1fb6000640 1 -- 172.21.15.12:0/1222628100 <== mon.2 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7f1fa403b460 con 0x7f1fb8105b00 2024-09-11T00:08:04.589 INFO:tasks.workunit.client.0.smithi012.stdout: 1 3513 38.36 MB/sec warmup 1 sec latency 6.773 ms 2024-09-11T00:08:04.593 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:04.587+0000 7f1fbda00640 1 -- 172.21.15.12:0/1810373372 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555eb8cddce0 msgr2=0x555eb8ce01a0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-11T00:08:04.593 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:04.587+0000 7f1fbda00640 1 --2- 172.21.15.12:0/1810373372 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555eb8cddce0 0x555eb8ce01a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:08:04.593 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:04.587+0000 7f1fbda00640 1 -- 172.21.15.12:0/1810373372 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555eb8cd8970 msgr2=0x555eb8cdad60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:08:04.593 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:04.587+0000 7f1fbda00640 1 --2- 172.21.15.12:0/1810373372 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555eb8cd8970 0x555eb8cdad60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:08:04.593 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:04.587+0000 7f1fbda00640 1 -- 172.21.15.12:0/1810373372 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555eb8c07780 con 0x555eb8cdb2a0 2024-09-11T00:08:04.593 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:04.588+0000 7f1fbe400640 1 --2- 172.21.15.12:0/1810373372 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555eb8cd8970 0x555eb8cdad60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:08:04.594 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:04.588+0000 7f1fbda00640 1 --2- 172.21.15.12:0/1810373372 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555eb8cdb2a0 0x555eb8cdd690 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f1fac009fa0 tx=0x7f1fac0118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5a7b0e2f1fd5cba4 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:08:04.594 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:04.588+0000 7f1faaa00640 1 -- 172.21.15.12:0/1810373372 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fac003c30 con 0x555eb8cdb2a0 2024-09-11T00:08:04.594 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:04.588+0000 7f1faaa00640 1 -- 172.21.15.12:0/1810373372 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1fac003dd0 con 0x555eb8cdb2a0 2024-09-11T00:08:04.594 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:04.588+0000 7f1faaa00640 1 -- 172.21.15.12:0/1810373372 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fac00fb20 con 0x555eb8cdb2a0 2024-09-11T00:08:04.594 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:04.589+0000 7f1fc31d1740 1 -- 172.21.15.12:0/1810373372 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555eb8cdb2a0 msgr2=0x555eb8cdd690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:08:04.594 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:04.589+0000 7f1fc31d1740 1 --2- 172.21.15.12:0/1810373372 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555eb8cdb2a0 0x555eb8cdd690 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f1fac009fa0 tx=0x7f1fac0118d0 comp rx=0 tx=0).stop 2024-09-11T00:08:04.594 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:04.589+0000 7f1fc31d1740 1 -- 172.21.15.12:0/1810373372 shutdown_connections 2024-09-11T00:08:04.594 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:04.589+0000 7f1fc31d1740 1 --2- 172.21.15.12:0/1810373372 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555eb8cddce0 0x555eb8ce01a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:08:04.594 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:04.589+0000 7f1fc31d1740 1 --2- 172.21.15.12:0/1810373372 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555eb8cdb2a0 0x555eb8cdd690 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:08:04.594 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:04.589+0000 7f1fc31d1740 1 --2- 172.21.15.12:0/1810373372 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555eb8cd8970 0x555eb8cdad60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:08:04.594 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:04.589+0000 7f1fc31d1740 1 -- 172.21.15.12:0/1810373372 >> 172.21.15.12:0/1810373372 conn(0x555eb8cd3ac0 msgr2=0x555eb8cd4850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:08:04.594 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:04.589+0000 7f1fc31d1740 1 -- 172.21.15.12:0/1810373372 shutdown_connections 2024-09-11T00:08:04.595 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:04.589+0000 7f1fc31d1740 1 -- 172.21.15.12:0/1810373372 wait complete. 2024-09-11T00:08:04.595 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:04.589+0000 7f1fb6000640 1 --2- 172.21.15.12:0/1222628100 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f1fa00b5270 0x7f1fa00b7730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:08:04.595 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:04.590+0000 7f1fb6000640 1 -- 172.21.15.12:0/1222628100 <== mon.2 v2:172.21.15.165:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9208+0+0 (secure 0 0 0) 0x7f1fa4106a00 con 0x7f1fb8105b00 2024-09-11T00:08:04.595 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:04.590+0000 7f1fbe400640 1 --2- 172.21.15.12:0/1222628100 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f1fa00b5270 0x7f1fa00b7730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:08:04.595 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:04.590+0000 7f1fc31d1740 1 -- start start 2024-09-11T00:08:04.595 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:04.590+0000 7f1fc31d1740 1 Processor -- start 2024-09-11T00:08:04.595 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:04.590+0000 7f1fbe400640 1 --2- 172.21.15.12:0/1222628100 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f1fa00b5270 0x7f1fa00b7730 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f1fb0009d60 tx=0x7f1fb0005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:08:04.595 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:04.591+0000 7f1fc31d1740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555eb8cd8970 0x555eb8ce20d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:08:04.595 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:04.591+0000 7f1fc31d1740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555eb8cdb2a0 0x555eb8ce2770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:08:04.595 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:04.591+0000 7f1fc31d1740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555eb8cddce0 0x555eb8ce3340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:08:04.596 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:04.591+0000 7f1fc31d1740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x555eb8b72ab0 con 0x555eb8cdb2a0 2024-09-11T00:08:04.596 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:04.591+0000 7f1fc31d1740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x555eb8be1570 con 0x555eb8cddce0 2024-09-11T00:08:04.596 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:04.591+0000 7f1fc31d1740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x555eb8ce0cc0 con 0x555eb8cd8970 2024-09-11T00:08:04.596 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:04.591+0000 7f1fc31d1740 10 client.0 _ll_register_callbacks cb 0x555eb8bf0100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:08:04.596 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:04.591+0000 7f1fbda00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555eb8cd8970 0x555eb8ce20d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:08:04.596 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:04.591+0000 7f1fbe400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555eb8cddce0 0x555eb8ce3340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:08:04.596 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:04.591+0000 7f1fbda00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555eb8cd8970 0x555eb8ce20d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.12:56290/0 (socket says 172.21.15.12:56290) 2024-09-11T00:08:04.596 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:04.591+0000 7f1fbda00640 1 -- 172.21.15.12:0/521668554 learned_addr learned my addr 172.21.15.12:0/521668554 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:08:04.596 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:04.591+0000 7f1fbd000640 1 --2- 172.21.15.12:0/521668554 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555eb8cdb2a0 0x555eb8ce2770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:08:04.596 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:04.591+0000 7f1fbe400640 1 -- 172.21.15.12:0/521668554 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555eb8cd8970 msgr2=0x555eb8ce20d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:08:04.596 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:04.591+0000 7f1fbe400640 1 --2- 172.21.15.12:0/521668554 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555eb8cd8970 0x555eb8ce20d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:08:04.597 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:04.591+0000 7f1fbe400640 1 -- 172.21.15.12:0/521668554 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555eb8cdb2a0 msgr2=0x555eb8ce2770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:08:04.597 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:04.591+0000 7f1fbe400640 1 --2- 172.21.15.12:0/521668554 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555eb8cdb2a0 0x555eb8ce2770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:08:04.597 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:04.591+0000 7f1fbe400640 1 -- 172.21.15.12:0/521668554 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555eb899c790 con 0x555eb8cddce0 2024-09-11T00:08:04.597 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:04.591+0000 7f1fbd000640 1 --2- 172.21.15.12:0/521668554 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555eb8cdb2a0 0x555eb8ce2770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:08:04.597 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:04.592+0000 7f1fbe400640 1 --2- 172.21.15.12:0/521668554 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555eb8cddce0 0x555eb8ce3340 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f1fb0009b40 tx=0x7f1fb0004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6829f1e392e9095a server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:08:04.597 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:04.592+0000 7f1f9f400640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-09-11T00:08:04.597 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:04.592+0000 7f1f9f400640 1 -- 172.21.15.12:0/521668554 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fb0009e50 con 0x555eb8cddce0 2024-09-11T00:08:04.597 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:04.592+0000 7f1fc31d1740 1 -- 172.21.15.12:0/521668554 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1fac003dd0 con 0x555eb8cddce0 2024-09-11T00:08:04.597 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:04.592+0000 7f1f9f400640 1 -- 172.21.15.12:0/521668554 <== mon.2 v2:172.21.15.165:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 9208+0+0 (secure 0 0 0) 0x7f1fb001f640 con 0x555eb8cddce0 2024-09-11T00:08:04.597 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:04.592+0000 7f1f9f400640 1 -- 172.21.15.12:0/521668554 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f1fb001f840 con 0x555eb8cddce0 2024-09-11T00:08:04.597 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:04.592+0000 7f1fc31d1740 10 client.25625 fetch_fsmap learned FSMap version 29 2024-09-11T00:08:04.597 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:04.592+0000 7f1fc31d1740 1 -- 172.21.15.12:0/521668554 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x555eb8a15620 con 0x555eb8cddce0 2024-09-11T00:08:04.599 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:04.593+0000 7f1f9f400640 1 -- 172.21.15.12:0/521668554 <== mon.2 v2:172.21.15.165:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1fac003dd0 con 0x555eb8cddce0 2024-09-11T00:08:04.599 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:04.593+0000 7f1f9f400640 1 -- 172.21.15.12:0/521668554 <== mon.2 v2:172.21.15.165:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7f1fb0027070 con 0x555eb8cddce0 2024-09-11T00:08:04.599 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:04.593+0000 7f1fc31d1740 10 client.25625 fetch_fsmap finished waiting for FSMap version 29 2024-09-11T00:08:04.600 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:04.593+0000 7f1fc31d1740 10 client.25625 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T00:08:04.600 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:04.593+0000 7f1fc31d1740 20 client.25625 populate_metadata read hostname 'smithi012' 2024-09-11T00:08:04.600 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:04.593+0000 7f1fc31d1740 1 --2- 172.21.15.12:0/521668554 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x555eb8cedad0 0x555eb8cefee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:08:04.600 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:04.593+0000 7f1fc31d1740 4 client.25625 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T00:08:04.600 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:04.593+0000 7f1fc31d1740 1 -- 172.21.15.12:0/521668554 --> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555eb8c4d600 con 0x555eb8cedad0 2024-09-11T00:08:04.600 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:04.593+0000 7f1fbda00640 1 --2- 172.21.15.12:0/521668554 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x555eb8cedad0 0x555eb8cefee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:08:04.600 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:04.594+0000 7f1fbda00640 1 --2- 172.21.15.12:0/521668554 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x555eb8cedad0 0x555eb8cefee0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6a4be75b11562695 server_cookie=e9ae52094062136f in_seq=0 out_seq=0 2024-09-11T00:08:04.600 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:04.594+0000 7f1f9f400640 10 client.25625 ms_handle_connect on v2:172.21.15.165:6838/501312984 2024-09-11T00:08:04.600 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:04.594+0000 7f1f9f400640 1 -- 172.21.15.12:0/521668554 <== mds.0 v2:172.21.15.165:6838/501312984 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x555eb8c4d600 con 0x555eb8cedad0 2024-09-11T00:08:04.601 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:04.594+0000 7f1f9f400640 10 client.25625 handle_command_reply: tid=0 2024-09-11T00:08:04.601 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:04.594+0000 7f1fc31d1740 1 client.25625 shutdown 2024-09-11T00:08:04.601 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:04.595+0000 7f1fc31d1740 1 -- 172.21.15.12:0/521668554 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555eb8cddce0 msgr2=0x555eb8ce3340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:08:04.601 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:04.595+0000 7f1fc31d1740 1 --2- 172.21.15.12:0/521668554 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555eb8cddce0 0x555eb8ce3340 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f1fb0009b40 tx=0x7f1fb0004730 comp rx=0 tx=0).stop 2024-09-11T00:08:04.601 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:04.595+0000 7f1fc31d1740 1 -- 172.21.15.12:0/521668554 shutdown_connections 2024-09-11T00:08:04.601 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:04.595+0000 7f1fc31d1740 1 --2- 172.21.15.12:0/521668554 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x555eb8cedad0 0x555eb8cefee0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:08:04.601 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:04.595+0000 7f1fc31d1740 1 --2- 172.21.15.12:0/521668554 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555eb8cddce0 0x555eb8ce3340 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:08:04.601 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:04.595+0000 7f1fc31d1740 1 --2- 172.21.15.12:0/521668554 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555eb8cdb2a0 0x555eb8ce2770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:08:04.601 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:04.595+0000 7f1f9f400640 0 client.25625 ms_handle_reset on v2:172.21.15.165:6838/501312984 2024-09-11T00:08:04.601 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:04.595+0000 7f1fc31d1740 1 --2- 172.21.15.12:0/521668554 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555eb8cd8970 0x555eb8ce20d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:08:04.601 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:04.595+0000 7f1fc31d1740 1 -- 172.21.15.12:0/521668554 >> 172.21.15.12:0/521668554 conn(0x555eb8cd3ac0 msgr2=0x555eb8cd2bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:08:04.601 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:04.595+0000 7f1fc31d1740 1 -- 172.21.15.12:0/521668554 shutdown_connections 2024-09-11T00:08:04.601 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:04.595+0000 7f1fc31d1740 1 -- 172.21.15.12:0/521668554 wait complete. 2024-09-11T00:08:04.602 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:04.595+0000 7f1fc31d1740 20 client.25625 trim_cache size 0 max 16384 2024-09-11T00:08:04.614 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:04.613+0000 7f1fc31d1740 1 Processor -- start 2024-09-11T00:08:04.614 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:04.613+0000 7f1fc31d1740 1 -- start start 2024-09-11T00:08:04.615 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:04.613+0000 7f1fc31d1740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555eb8cd8970 0x555eb8ce3320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:08:04.615 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:04.613+0000 7f1fc31d1740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555eb8cdb2a0 0x555eb8cea3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:08:04.615 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:04.613+0000 7f1fc31d1740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555eb8cddce0 0x555eb8cde110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:08:04.616 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:04.613+0000 7f1fc31d1740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f1fb81ad740 con 0x555eb8cd8970 2024-09-11T00:08:04.616 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:04.613+0000 7f1fc31d1740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f1fb81af280 con 0x555eb8cddce0 2024-09-11T00:08:04.616 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:04.613+0000 7f1fc31d1740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f1fb81ade10 con 0x555eb8cdb2a0 2024-09-11T00:08:04.616 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:04.614+0000 7f1fbda00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555eb8cd8970 0x555eb8ce3320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:08:04.616 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:04.614+0000 7f1fbd000640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555eb8cdb2a0 0x555eb8cea3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:08:04.616 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:04.614+0000 7f1fbda00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555eb8cd8970 0x555eb8ce3320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:50090/0 (socket says 172.21.15.12:50090) 2024-09-11T00:08:04.616 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:04.614+0000 7f1fbda00640 1 -- 172.21.15.12:0/3218952163 learned_addr learned my addr 172.21.15.12:0/3218952163 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:08:04.616 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:04.614+0000 7f1fbe400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555eb8cddce0 0x555eb8cde110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:08:04.616 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:04.614+0000 7f1fbda00640 1 -- 172.21.15.12:0/3218952163 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555eb8cdb2a0 msgr2=0x555eb8cea3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:08:04.616 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:04.614+0000 7f1fbda00640 1 --2- 172.21.15.12:0/3218952163 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555eb8cdb2a0 0x555eb8cea3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:08:04.616 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:04.614+0000 7f1fbda00640 1 -- 172.21.15.12:0/3218952163 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555eb8cddce0 msgr2=0x555eb8cde110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:08:04.616 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:04.614+0000 7f1fbda00640 1 --2- 172.21.15.12:0/3218952163 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555eb8cddce0 0x555eb8cde110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:08:04.616 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:04.614+0000 7f1fbda00640 1 -- 172.21.15.12:0/3218952163 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555eb8cdb0d0 con 0x555eb8cd8970 2024-09-11T00:08:04.617 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:04.614+0000 7f1fbe400640 1 --2- 172.21.15.12:0/3218952163 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555eb8cddce0 0x555eb8cde110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:08:04.617 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:04.614+0000 7f1fbda00640 1 --2- 172.21.15.12:0/3218952163 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555eb8cd8970 0x555eb8ce3320 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f1fac009a00 tx=0x7f1fac009a70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6bc9abe35b42d112 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:08:04.618 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:04.617+0000 7f1faa000640 1 -- 172.21.15.12:0/3218952163 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fac003d70 con 0x555eb8cd8970 2024-09-11T00:08:04.618 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:04.617+0000 7f1faa000640 1 -- 172.21.15.12:0/3218952163 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1fac00faa0 con 0x555eb8cd8970 2024-09-11T00:08:04.618 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:04.617+0000 7f1faa000640 1 -- 172.21.15.12:0/3218952163 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fac005520 con 0x555eb8cd8970 2024-09-11T00:08:04.619 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:04.618+0000 7f1fc31d1740 1 -- 172.21.15.12:0/3218952163 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555eb8cd8970 msgr2=0x555eb8ce3320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:08:04.619 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:04.618+0000 7f1fc31d1740 1 --2- 172.21.15.12:0/3218952163 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555eb8cd8970 0x555eb8ce3320 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f1fac009a00 tx=0x7f1fac009a70 comp rx=0 tx=0).stop 2024-09-11T00:08:04.619 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:04.618+0000 7f1fc31d1740 1 -- 172.21.15.12:0/3218952163 shutdown_connections 2024-09-11T00:08:04.619 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:04.618+0000 7f1fc31d1740 1 --2- 172.21.15.12:0/3218952163 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555eb8cddce0 0x555eb8cde110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:08:04.619 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:04.618+0000 7f1fc31d1740 1 --2- 172.21.15.12:0/3218952163 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555eb8cdb2a0 0x555eb8cea3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:08:04.620 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:04.618+0000 7f1fc31d1740 1 --2- 172.21.15.12:0/3218952163 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555eb8cd8970 0x555eb8ce3320 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:08:04.620 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:04.618+0000 7f1fc31d1740 1 -- 172.21.15.12:0/3218952163 >> 172.21.15.12:0/3218952163 conn(0x555eb8cd3ac0 msgr2=0x555eb8cd7e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:08:04.620 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:04.618+0000 7f1fc31d1740 1 -- 172.21.15.12:0/3218952163 shutdown_connections 2024-09-11T00:08:04.620 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:04.618+0000 7f1fc31d1740 1 -- 172.21.15.12:0/3218952163 wait complete. 2024-09-11T00:08:04.620 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:04.618+0000 7f1fc31d1740 1 -- start start 2024-09-11T00:08:04.620 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:04.619+0000 7f1fc31d1740 1 Processor -- start 2024-09-11T00:08:04.623 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:04.619+0000 7f1fc31d1740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555eb8cd8970 0x555eb8ce92d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:08:04.623 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:04.619+0000 7f1fc31d1740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555eb8cdb2a0 0x555eb8ce9810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:08:04.623 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:04.619+0000 7f1fc31d1740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555eb8cddce0 0x555eb8ce9d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:08:04.623 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:04.619+0000 7f1fc31d1740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f1fb81aebb0 con 0x555eb8cd8970 2024-09-11T00:08:04.623 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:04.619+0000 7f1fc31d1740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f1fb81ae4e0 con 0x555eb8cddce0 2024-09-11T00:08:04.623 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:04.619+0000 7f1fc31d1740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f1fb81ab720 con 0x555eb8cdb2a0 2024-09-11T00:08:04.624 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:04.619+0000 7f1fc31d1740 10 client.0 _ll_register_callbacks cb 0x7f1fb00099b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:08:04.624 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:04.619+0000 7f1fbda00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555eb8cd8970 0x555eb8ce92d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:08:04.624 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:04.619+0000 7f1fbd000640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555eb8cdb2a0 0x555eb8ce9810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:08:04.624 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:04.619+0000 7f1fbe400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555eb8cddce0 0x555eb8ce9d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:08:04.624 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:04.619+0000 7f1fbd000640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555eb8cdb2a0 0x555eb8ce9810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.12:56306/0 (socket says 172.21.15.12:56306) 2024-09-11T00:08:04.624 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:04.619+0000 7f1fbd000640 1 -- 172.21.15.12:0/3742641963 learned_addr learned my addr 172.21.15.12:0/3742641963 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:08:04.624 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:04.620+0000 7f1fbd000640 1 -- 172.21.15.12:0/3742641963 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555eb8cddce0 msgr2=0x555eb8ce9d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:08:04.624 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:04.620+0000 7f1fbd000640 1 --2- 172.21.15.12:0/3742641963 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555eb8cddce0 0x555eb8ce9d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:08:04.624 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:04.620+0000 7f1fbd000640 1 -- 172.21.15.12:0/3742641963 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555eb8cd8970 msgr2=0x555eb8ce92d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:08:04.624 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:04.620+0000 7f1fbd000640 1 --2- 172.21.15.12:0/3742641963 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555eb8cd8970 0x555eb8ce92d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:08:04.624 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:04.620+0000 7f1fbd000640 1 -- 172.21.15.12:0/3742641963 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555eb8ce8060 con 0x555eb8cdb2a0 2024-09-11T00:08:04.625 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:04.620+0000 7f1fbd000640 1 --2- 172.21.15.12:0/3742641963 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555eb8cdb2a0 0x555eb8ce9810 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f1fa40ccdd0 tx=0x7f1fa40cd460 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fbf96a43777d169c server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:08:04.625 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:04.620+0000 7f1f9f400640 10 client.0 ms_handle_connect on v2:172.21.15.178:3300/0 2024-09-11T00:08:04.625 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:04.620+0000 7f1f9f400640 1 -- 172.21.15.12:0/3742641963 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fa4049040 con 0x555eb8cdb2a0 2024-09-11T00:08:04.625 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:04.620+0000 7f1fc31d1740 1 -- 172.21.15.12:0/3742641963 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1fac00faa0 con 0x555eb8cdb2a0 2024-09-11T00:08:04.625 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:04.620+0000 7f1f9f400640 1 -- 172.21.15.12:0/3742641963 <== mon.1 v2:172.21.15.178:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 9208+0+0 (secure 0 0 0) 0x7f1fa4031740 con 0x555eb8cdb2a0 2024-09-11T00:08:04.625 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:04.620+0000 7f1f9f400640 1 -- 172.21.15.12:0/3742641963 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f1fa40cc950 con 0x555eb8cdb2a0 2024-09-11T00:08:04.625 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:04.620+0000 7f1fc31d1740 10 client.25615 fetch_fsmap learned FSMap version 29 2024-09-11T00:08:04.625 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:04.620+0000 7f1fc31d1740 1 -- 172.21.15.12:0/3742641963 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x555eb8d5b090 con 0x555eb8cdb2a0 2024-09-11T00:08:04.625 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:04.620+0000 7f1f9f400640 1 -- 172.21.15.12:0/3742641963 <== mon.1 v2:172.21.15.178:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1fac00faa0 con 0x555eb8cdb2a0 2024-09-11T00:08:04.625 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:04.621+0000 7f1f9f400640 1 -- 172.21.15.12:0/3742641963 <== mon.1 v2:172.21.15.178:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7f1fa40c8790 con 0x555eb8cdb2a0 2024-09-11T00:08:04.625 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:04.621+0000 7f1fc31d1740 10 client.25615 fetch_fsmap finished waiting for FSMap version 29 2024-09-11T00:08:04.625 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:04.621+0000 7f1fc31d1740 10 client.25615 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T00:08:04.625 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:04.621+0000 7f1fc31d1740 20 client.25615 populate_metadata read hostname 'smithi012' 2024-09-11T00:08:04.626 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:04.621+0000 7f1fc31d1740 1 --2- 172.21.15.12:0/3742641963 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x555eb8d5b9f0 0x555eb8d5de10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:08:04.626 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:04.621+0000 7f1fc31d1740 4 client.25615 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T00:08:04.626 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:04.621+0000 7f1fc31d1740 1 -- 172.21.15.12:0/3742641963 --> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] -- command(tid 0: {"prefix": "flush journal"}) -- 0x555eb8c490b0 con 0x555eb8d5b9f0 2024-09-11T00:08:04.626 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:04.621+0000 7f1fbda00640 1 --2- 172.21.15.12:0/3742641963 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x555eb8d5b9f0 0x555eb8d5de10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:08:04.626 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:04.621+0000 7f1fbda00640 1 --2- 172.21.15.12:0/3742641963 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x555eb8d5b9f0 0x555eb8d5de10 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b8ee0af4c46d54b4 server_cookie=bcd4dc99026cde1c in_seq=0 out_seq=0 2024-09-11T00:08:04.626 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:04.621+0000 7f1f9f400640 10 client.25615 ms_handle_connect on v2:172.21.15.165:6838/501312984 2024-09-11T00:08:04.694 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:04.692+0000 7f1f9f400640 1 -- 172.21.15.12:0/3742641963 <== mds.0 v2:172.21.15.165:6838/501312984 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x555eb8c490b0 con 0x555eb8d5b9f0 2024-09-11T00:08:04.694 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:04.692+0000 7f1f9f400640 10 client.25615 handle_command_reply: tid=0 2024-09-11T00:08:04.694 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:04.692+0000 7f1fc31d1740 1 client.25615 shutdown 2024-09-11T00:08:04.694 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:04.693+0000 7f1fc31d1740 1 -- 172.21.15.12:0/3742641963 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555eb8cdb2a0 msgr2=0x555eb8ce9810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:08:04.694 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:04.693+0000 7f1fc31d1740 1 --2- 172.21.15.12:0/3742641963 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555eb8cdb2a0 0x555eb8ce9810 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f1fa40ccdd0 tx=0x7f1fa40cd460 comp rx=0 tx=0).stop 2024-09-11T00:08:04.694 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:04.693+0000 7f1fc31d1740 1 -- 172.21.15.12:0/3742641963 shutdown_connections 2024-09-11T00:08:04.695 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:04.693+0000 7f1fc31d1740 1 --2- 172.21.15.12:0/3742641963 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x555eb8d5b9f0 0x555eb8d5de10 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:08:04.695 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:04.693+0000 7f1fc31d1740 1 --2- 172.21.15.12:0/3742641963 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555eb8cddce0 0x555eb8ce9d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:08:04.695 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:04.693+0000 7f1fc31d1740 1 --2- 172.21.15.12:0/3742641963 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555eb8cdb2a0 0x555eb8ce9810 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:08:04.695 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:04.693+0000 7f1f9f400640 0 client.25615 ms_handle_reset on v2:172.21.15.165:6838/501312984 2024-09-11T00:08:04.695 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:04.693+0000 7f1fc31d1740 1 --2- 172.21.15.12:0/3742641963 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555eb8cd8970 0x555eb8ce92d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:08:04.695 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:04.693+0000 7f1fc31d1740 1 -- 172.21.15.12:0/3742641963 >> 172.21.15.12:0/3742641963 conn(0x555eb8cd3ac0 msgr2=0x555eb8cd7a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:08:04.695 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:04.693+0000 7f1fc31d1740 1 -- 172.21.15.12:0/3742641963 shutdown_connections 2024-09-11T00:08:04.695 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:04.693+0000 7f1fc31d1740 1 -- 172.21.15.12:0/3742641963 wait complete. 2024-09-11T00:08:04.695 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:04.693+0000 7f1fc31d1740 20 client.25615 trim_cache size 0 max 16384 2024-09-11T00:08:04.696 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:{ 2024-09-11T00:08:04.696 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "message": "", 2024-09-11T00:08:04.696 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "return_code": 0 2024-09-11T00:08:04.696 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:} 2024-09-11T00:08:04.697 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:04.694+0000 7f1fbee00640 1 -- 172.21.15.12:0/1222628100 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f1fa00b5270 msgr2=0x7f1fa00b7730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:08:04.697 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:04.694+0000 7f1fbee00640 1 --2- 172.21.15.12:0/1222628100 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f1fa00b5270 0x7f1fa00b7730 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f1fb0009d60 tx=0x7f1fb0005f90 comp rx=0 tx=0).stop 2024-09-11T00:08:04.697 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:04.694+0000 7f1fbee00640 1 -- 172.21.15.12:0/1222628100 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1fb8105b00 msgr2=0x7f1fb819b3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:08:04.697 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:04.694+0000 7f1fbee00640 1 --2- 172.21.15.12:0/1222628100 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1fb8105b00 0x7f1fb819b3a0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f1fa4031490 tx=0x7f1fa4003c10 comp rx=0 tx=0).stop 2024-09-11T00:08:04.697 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:04.694+0000 7f1fbee00640 1 -- 172.21.15.12:0/1222628100 shutdown_connections 2024-09-11T00:08:04.697 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:04.694+0000 7f1fbee00640 1 --2- 172.21.15.12:0/1222628100 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f1fa00b5270 0x7f1fa00b7730 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:08:04.697 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:04.694+0000 7f1fbee00640 1 --2- 172.21.15.12:0/1222628100 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1fb81064c0 0x7f1fb819bfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:08:04.697 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:04.694+0000 7f1fbee00640 1 --2- 172.21.15.12:0/1222628100 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1fb8105b00 0x7f1fb819b3a0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:08:04.697 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:04.694+0000 7f1fbee00640 1 --2- 172.21.15.12:0/1222628100 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1fb81049b0 0x7f1fb819ad50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:08:04.697 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:04.694+0000 7f1fbee00640 1 -- 172.21.15.12:0/1222628100 >> 172.21.15.12:0/1222628100 conn(0x7f1fb8100270 msgr2=0x7f1fb8108e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:08:04.697 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:04.694+0000 7f1fbee00640 1 -- 172.21.15.12:0/1222628100 shutdown_connections 2024-09-11T00:08:04.698 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:04.694+0000 7f1fbee00640 1 -- 172.21.15.12:0/1222628100 wait complete. 2024-09-11T00:08:05.588 INFO:tasks.workunit.client.0.smithi012.stdout: 1 6435 25.66 MB/sec warmup 2 sec latency 34.148 ms 2024-09-11T00:08:06.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:08:05 smithi165 ceph-mon[35356]: pgmap v428: 129 pgs: 129 active+clean; 671 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-09-11T00:08:06.062 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:08:05 smithi012 ceph-mon[30333]: pgmap v428: 129 pgs: 129 active+clean; 671 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-09-11T00:08:06.080 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:08:05 smithi178 ceph-mon[35919]: pgmap v428: 129 pgs: 129 active+clean; 671 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-09-11T00:08:06.588 INFO:tasks.workunit.client.0.smithi012.stdout: 1 9504 22.07 MB/sec warmup 3 sec latency 14.490 ms 2024-09-11T00:08:07.588 INFO:tasks.workunit.client.0.smithi012.stdout: 1 12121 20.85 MB/sec warmup 4 sec latency 15.754 ms 2024-09-11T00:08:07.882 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:08:07 smithi178 ceph-mon[35919]: pgmap v429: 129 pgs: 129 active+clean; 3.2 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 135 KiB/s rd, 255 KiB/s wr, 71 op/s 2024-09-11T00:08:08.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:08:07 smithi165 ceph-mon[35356]: pgmap v429: 129 pgs: 129 active+clean; 3.2 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 135 KiB/s rd, 255 KiB/s wr, 71 op/s 2024-09-11T00:08:08.062 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:08:07 smithi012 ceph-mon[30333]: pgmap v429: 129 pgs: 129 active+clean; 3.2 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 135 KiB/s rd, 255 KiB/s wr, 71 op/s 2024-09-11T00:08:08.588 INFO:tasks.workunit.client.0.smithi012.stdout: 1 15109 19.63 MB/sec warmup 5 sec latency 11.533 ms 2024-09-11T00:08:09.589 INFO:tasks.workunit.client.0.smithi012.stdout: 1 18410 19.44 MB/sec warmup 6 sec latency 15.398 ms 2024-09-11T00:08:10.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:08:09 smithi165 ceph-mon[35356]: pgmap v430: 129 pgs: 129 active+clean; 3.5 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 133 KiB/s rd, 294 KiB/s wr, 78 op/s 2024-09-11T00:08:10.066 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:08:09 smithi012 ceph-mon[30333]: pgmap v430: 129 pgs: 129 active+clean; 3.5 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 133 KiB/s rd, 294 KiB/s wr, 78 op/s 2024-09-11T00:08:10.080 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:08:09 smithi178 ceph-mon[35919]: pgmap v430: 129 pgs: 129 active+clean; 3.5 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 133 KiB/s rd, 294 KiB/s wr, 78 op/s 2024-09-11T00:08:10.589 INFO:tasks.workunit.client.0.smithi012.stdout: 1 21716 19.39 MB/sec warmup 7 sec latency 13.614 ms 2024-09-11T00:08:11.589 INFO:tasks.workunit.client.0.smithi012.stdout: 1 24955 19.18 MB/sec warmup 8 sec latency 10.795 ms 2024-09-11T00:08:12.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:08:11 smithi165 ceph-mon[35356]: pgmap v431: 129 pgs: 129 active+clean; 5.9 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 135 KiB/s rd, 585 KiB/s wr, 111 op/s 2024-09-11T00:08:12.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:08:11 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T00:08:12.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:08:11 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:08:12.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:08:11 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T00:08:12.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:08:11 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:08:12.063 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:08:11 smithi012 ceph-mon[30333]: pgmap v431: 129 pgs: 129 active+clean; 5.9 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 135 KiB/s rd, 585 KiB/s wr, 111 op/s 2024-09-11T00:08:12.063 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:08:11 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T00:08:12.063 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:08:11 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:08:12.063 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:08:11 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T00:08:12.063 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:08:11 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:08:12.080 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:08:11 smithi178 ceph-mon[35919]: pgmap v431: 129 pgs: 129 active+clean; 5.9 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 135 KiB/s rd, 585 KiB/s wr, 111 op/s 2024-09-11T00:08:12.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:08:11 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T00:08:12.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:08:11 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:08:12.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:08:11 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T00:08:12.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:08:11 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:08:12.589 INFO:tasks.workunit.client.0.smithi012.stdout: 1 27342 18.06 MB/sec warmup 9 sec latency 16.464 ms 2024-09-11T00:08:13.589 INFO:tasks.workunit.client.0.smithi012.stdout: 1 30055 17.92 MB/sec warmup 10 sec latency 11.306 ms 2024-09-11T00:08:14.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:08:13 smithi165 ceph-mon[35356]: pgmap v432: 129 pgs: 129 active+clean; 11 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 510 KiB/s rd, 1.1 MiB/s wr, 183 op/s 2024-09-11T00:08:14.062 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:08:13 smithi012 ceph-mon[30333]: pgmap v432: 129 pgs: 129 active+clean; 11 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 510 KiB/s rd, 1.1 MiB/s wr, 183 op/s 2024-09-11T00:08:14.080 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:08:13 smithi178 ceph-mon[35919]: pgmap v432: 129 pgs: 129 active+clean; 11 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 510 KiB/s rd, 1.1 MiB/s wr, 183 op/s 2024-09-11T00:08:14.589 INFO:tasks.workunit.client.0.smithi012.stdout: 1 32879 17.62 MB/sec warmup 11 sec latency 15.784 ms 2024-09-11T00:08:15.589 INFO:tasks.workunit.client.0.smithi012.stdout: 1 36072 17.68 MB/sec warmup 12 sec latency 14.628 ms 2024-09-11T00:08:16.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:08:15 smithi165 ceph-mon[35356]: pgmap v433: 129 pgs: 129 active+clean; 11 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 509 KiB/s rd, 1.1 MiB/s wr, 181 op/s 2024-09-11T00:08:16.063 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:08:15 smithi012 ceph-mon[30333]: pgmap v433: 129 pgs: 129 active+clean; 11 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 509 KiB/s rd, 1.1 MiB/s wr, 181 op/s 2024-09-11T00:08:16.080 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:08:15 smithi178 ceph-mon[35919]: pgmap v433: 129 pgs: 129 active+clean; 11 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 509 KiB/s rd, 1.1 MiB/s wr, 181 op/s 2024-09-11T00:08:16.589 INFO:tasks.workunit.client.0.smithi012.stdout: 1 39296 17.77 MB/sec warmup 13 sec latency 15.599 ms 2024-09-11T00:08:17.589 INFO:tasks.workunit.client.0.smithi012.stdout: 1 42350 17.55 MB/sec warmup 14 sec latency 11.305 ms 2024-09-11T00:08:17.812 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:17.810+0000 7f2903e00640 1 -- 172.21.15.12:0/1063526963 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2904105b20 msgr2=0x7f2904105fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:08:17.812 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:17.810+0000 7f2903e00640 1 --2- 172.21.15.12:0/1063526963 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2904105b20 0x7f2904105fa0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f28f0009900 tx=0x7f28f002a3f0 comp rx=0 tx=0).stop 2024-09-11T00:08:17.813 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:17.810+0000 7f2903e00640 1 -- 172.21.15.12:0/1063526963 shutdown_connections 2024-09-11T00:08:17.813 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:17.810+0000 7f2903e00640 1 --2- 172.21.15.12:0/1063526963 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f29041064e0 0x7f290410ce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:08:17.813 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:17.810+0000 7f2903e00640 1 --2- 172.21.15.12:0/1063526963 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2904105b20 0x7f2904105fa0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:08:17.813 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:17.810+0000 7f2903e00640 1 --2- 172.21.15.12:0/1063526963 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f29041049d0 0x7f2904104dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:08:17.813 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:17.810+0000 7f2903e00640 1 -- 172.21.15.12:0/1063526963 >> 172.21.15.12:0/1063526963 conn(0x7f2904100270 msgr2=0x7f29041026b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:08:17.813 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:17.810+0000 7f2903e00640 1 -- 172.21.15.12:0/1063526963 shutdown_connections 2024-09-11T00:08:17.813 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:17.810+0000 7f2903e00640 1 -- 172.21.15.12:0/1063526963 wait complete. 2024-09-11T00:08:17.815 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:17.811+0000 7f2903e00640 1 Processor -- start 2024-09-11T00:08:17.815 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:17.811+0000 7f2903e00640 1 -- start start 2024-09-11T00:08:17.815 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:17.811+0000 7f2903e00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f29041049d0 0x7f290419add0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:08:17.815 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:17.811+0000 7f2903e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2904105b20 0x7f290419b420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:08:17.815 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:17.812+0000 7f2903e00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f29041064e0 0x7f290419c050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:08:17.815 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:17.812+0000 7f2903e00640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f2904110670 con 0x7f29041064e0 2024-09-11T00:08:17.815 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:17.812+0000 7f2903e00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f29041104f0 con 0x7f2904105b20 2024-09-11T00:08:17.815 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:17.812+0000 7f2903e00640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f29041107f0 con 0x7f29041049d0 2024-09-11T00:08:17.816 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:17.812+0000 7f2902000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2904105b20 0x7f290419b420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:08:17.816 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:17.812+0000 7f2902a00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f29041049d0 0x7f290419add0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:08:17.816 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:17.812+0000 7f2903400640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f29041064e0 0x7f290419c050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:08:17.816 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:17.812+0000 7f2902000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2904105b20 0x7f290419b420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.12:48844/0 (socket says 172.21.15.12:48844) 2024-09-11T00:08:17.816 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:17.812+0000 7f2902a00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f29041049d0 0x7f290419add0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.12:53952/0 (socket says 172.21.15.12:53952) 2024-09-11T00:08:17.816 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:17.812+0000 7f2902000640 1 -- 172.21.15.12:0/15221900 learned_addr learned my addr 172.21.15.12:0/15221900 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:08:17.816 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:17.812+0000 7f2902a00640 1 -- 172.21.15.12:0/15221900 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2904105b20 msgr2=0x7f290419b420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:08:17.816 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:17.812+0000 7f2902a00640 1 --2- 172.21.15.12:0/15221900 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2904105b20 0x7f290419b420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:08:17.816 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:17.812+0000 7f2902a00640 1 -- 172.21.15.12:0/15221900 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f29041064e0 msgr2=0x7f290419c050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:08:17.816 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:17.812+0000 7f2902a00640 1 --2- 172.21.15.12:0/15221900 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f29041064e0 0x7f290419c050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:08:17.816 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:17.812+0000 7f2902a00640 1 -- 172.21.15.12:0/15221900 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f290419c910 con 0x7f29041049d0 2024-09-11T00:08:17.816 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:17.812+0000 7f2903400640 1 --2- 172.21.15.12:0/15221900 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f29041064e0 0x7f290419c050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:08:17.817 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:17.813+0000 7f2902a00640 1 --2- 172.21.15.12:0/15221900 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f29041049d0 0x7f290419add0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f28f4005fd0 tx=0x7f28f400dd20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:08:17.817 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:17.813+0000 7f28fb400640 1 -- 172.21.15.12:0/15221900 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28f4009c30 con 0x7f29041049d0 2024-09-11T00:08:17.817 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:17.813+0000 7f2903e00640 1 -- 172.21.15.12:0/15221900 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f290419cc00 con 0x7f29041049d0 2024-09-11T00:08:17.817 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:17.813+0000 7f28fb400640 1 -- 172.21.15.12:0/15221900 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f28f400be30 con 0x7f29041049d0 2024-09-11T00:08:17.817 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:17.813+0000 7f28fb400640 1 -- 172.21.15.12:0/15221900 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28f4003da0 con 0x7f29041049d0 2024-09-11T00:08:17.817 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:17.813+0000 7f2903e00640 1 -- 172.21.15.12:0/15221900 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29041a32d0 con 0x7f29041049d0 2024-09-11T00:08:17.818 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:17.816+0000 7f290c67f740 1 Processor -- start 2024-09-11T00:08:17.818 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:17.816+0000 7f290c67f740 1 -- start start 2024-09-11T00:08:17.819 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:17.816+0000 7f290c67f740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563fc3562970 0x563fc3564d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:08:17.819 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:17.816+0000 7f290c67f740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563fc35652a0 0x563fc3567690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:08:17.819 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:17.816+0000 7f290c67f740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563fc3567ce0 0x563fc356a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:08:17.820 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:17.816+0000 7f290c67f740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x563fc34d56b0 con 0x563fc3562970 2024-09-11T00:08:17.820 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:17.816+0000 7f290c67f740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x563fc34c4ae0 con 0x563fc35652a0 2024-09-11T00:08:17.820 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:17.816+0000 7f290c67f740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x563fc34c88f0 con 0x563fc3567ce0 2024-09-11T00:08:17.820 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:17.816+0000 7f28fb400640 1 -- 172.21.15.12:0/15221900 <== mon.1 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7f28f4003f40 con 0x7f29041049d0 2024-09-11T00:08:17.820 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:17.817+0000 7f2902000640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563fc3567ce0 0x563fc356a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:08:17.820 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:17.817+0000 7f2902000640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563fc3567ce0 0x563fc356a1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.12:53958/0 (socket says 172.21.15.12:53958) 2024-09-11T00:08:17.820 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:17.817+0000 7f2903400640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563fc3562970 0x563fc3564d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:08:17.820 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:17.817+0000 7f2902000640 1 -- 172.21.15.12:0/110047013 learned_addr learned my addr 172.21.15.12:0/110047013 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:08:17.820 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:17.817+0000 7f2902a00640 1 --2- 172.21.15.12:0/110047013 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563fc35652a0 0x563fc3567690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:08:17.820 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:17.817+0000 7f2902000640 1 -- 172.21.15.12:0/110047013 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563fc35652a0 msgr2=0x563fc3567690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:08:17.820 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:17.817+0000 7f2902000640 1 --2- 172.21.15.12:0/110047013 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563fc35652a0 0x563fc3567690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:08:17.820 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:17.817+0000 7f2902000640 1 -- 172.21.15.12:0/110047013 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563fc3562970 msgr2=0x563fc3564d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:08:17.820 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:17.817+0000 7f2902000640 1 --2- 172.21.15.12:0/110047013 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563fc3562970 0x563fc3564d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:08:17.821 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:17.817+0000 7f2902000640 1 -- 172.21.15.12:0/110047013 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563fc3491780 con 0x563fc3567ce0 2024-09-11T00:08:17.821 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:17.817+0000 7f2902a00640 1 --2- 172.21.15.12:0/110047013 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563fc35652a0 0x563fc3567690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T00:08:17.821 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:17.817+0000 7f2902000640 1 --2- 172.21.15.12:0/110047013 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563fc3567ce0 0x563fc356a1a0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f28f002a9e0 tx=0x7f28f0004170 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6362c62845f83ff server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:08:17.821 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:17.818+0000 7f28efe00640 1 -- 172.21.15.12:0/110047013 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28f0035030 con 0x563fc3567ce0 2024-09-11T00:08:17.821 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:17.818+0000 7f28efe00640 1 -- 172.21.15.12:0/110047013 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f28f0040070 con 0x563fc3567ce0 2024-09-11T00:08:17.821 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:17.818+0000 7f28efe00640 1 -- 172.21.15.12:0/110047013 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28f00319a0 con 0x563fc3567ce0 2024-09-11T00:08:17.825 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:17.818+0000 7f28fb400640 1 --2- 172.21.15.12:0/15221900 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f28e80b52e0 0x7f28e80b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:08:17.825 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:17.818+0000 7f290c67f740 1 -- 172.21.15.12:0/110047013 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563fc3567ce0 msgr2=0x563fc356a1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:08:17.825 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:17.818+0000 7f290c67f740 1 --2- 172.21.15.12:0/110047013 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563fc3567ce0 0x563fc356a1a0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f28f002a9e0 tx=0x7f28f0004170 comp rx=0 tx=0).stop 2024-09-11T00:08:17.825 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:17.818+0000 7f28fb400640 1 -- 172.21.15.12:0/15221900 <== mon.1 v2:172.21.15.178:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9208+0+0 (secure 0 0 0) 0x7f28f40e5de0 con 0x7f29041049d0 2024-09-11T00:08:17.825 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:17.818+0000 7f2903400640 1 --2- 172.21.15.12:0/15221900 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f28e80b52e0 0x7f28e80b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:08:17.825 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:17.819+0000 7f290c67f740 1 -- 172.21.15.12:0/110047013 shutdown_connections 2024-09-11T00:08:17.825 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:17.819+0000 7f290c67f740 1 --2- 172.21.15.12:0/110047013 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563fc3567ce0 0x563fc356a1a0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:08:17.825 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:17.819+0000 7f290c67f740 1 --2- 172.21.15.12:0/110047013 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563fc35652a0 0x563fc3567690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:08:17.825 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:17.819+0000 7f290c67f740 1 --2- 172.21.15.12:0/110047013 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563fc3562970 0x563fc3564d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:08:17.825 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:17.819+0000 7f290c67f740 1 -- 172.21.15.12:0/110047013 >> 172.21.15.12:0/110047013 conn(0x563fc355dac0 msgr2=0x563fc355e850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:08:17.825 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:17.819+0000 7f290c67f740 1 -- 172.21.15.12:0/110047013 shutdown_connections 2024-09-11T00:08:17.825 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:17.819+0000 7f290c67f740 1 -- 172.21.15.12:0/110047013 wait complete. 2024-09-11T00:08:17.826 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:17.819+0000 7f2903400640 1 --2- 172.21.15.12:0/15221900 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f28e80b52e0 0x7f28e80b77a0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f290419bb50 tx=0x7f28fc009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:08:17.826 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:17.819+0000 7f290c67f740 1 -- start start 2024-09-11T00:08:17.826 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:17.820+0000 7f290c67f740 1 Processor -- start 2024-09-11T00:08:17.826 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:17.820+0000 7f290c67f740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563fc3562970 0x563fc356c040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:08:17.826 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:17.820+0000 7f290c67f740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563fc35652a0 0x563fc356c6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:08:17.826 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:17.820+0000 7f290c67f740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563fc3567ce0 0x563fc356d2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:08:17.826 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:17.820+0000 7f290c67f740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x563fc33fcab0 con 0x563fc3562970 2024-09-11T00:08:17.826 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:17.820+0000 7f290c67f740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x563fc346b570 con 0x563fc35652a0 2024-09-11T00:08:17.826 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:17.820+0000 7f290c67f740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x563fc356ac60 con 0x563fc3567ce0 2024-09-11T00:08:17.826 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:17.820+0000 7f290c67f740 10 client.0 _ll_register_callbacks cb 0x563fc347a100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:08:17.826 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:17.820+0000 7f2902000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563fc35652a0 0x563fc356c6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:08:17.826 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:17.820+0000 7f2902a00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563fc3562970 0x563fc356c040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:08:17.827 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:17.820+0000 7f2902000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563fc35652a0 0x563fc356c6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.12:48854/0 (socket says 172.21.15.12:48854) 2024-09-11T00:08:17.827 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:17.820+0000 7f2902000640 1 -- 172.21.15.12:0/3527735276 learned_addr learned my addr 172.21.15.12:0/3527735276 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:08:17.827 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:17.820+0000 7f2903400640 1 --2- 172.21.15.12:0/3527735276 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563fc3567ce0 0x563fc356d2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:08:17.827 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:17.821+0000 7f2902000640 1 -- 172.21.15.12:0/3527735276 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563fc3567ce0 msgr2=0x563fc356d2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:08:17.827 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:17.821+0000 7f2902000640 1 --2- 172.21.15.12:0/3527735276 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563fc3567ce0 0x563fc356d2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:08:17.827 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:17.821+0000 7f2902000640 1 -- 172.21.15.12:0/3527735276 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563fc3562970 msgr2=0x563fc356c040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:08:17.827 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:17.821+0000 7f2902000640 1 --2- 172.21.15.12:0/3527735276 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563fc3562970 0x563fc356c040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:08:17.827 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:17.821+0000 7f2902000640 1 -- 172.21.15.12:0/3527735276 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563fc3226790 con 0x563fc35652a0 2024-09-11T00:08:17.827 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:17.821+0000 7f2902a00640 1 --2- 172.21.15.12:0/3527735276 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563fc3562970 0x563fc356c040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:08:17.827 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:17.821+0000 7f2902000640 1 --2- 172.21.15.12:0/3527735276 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563fc35652a0 0x563fc356c6e0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f28f002a430 tx=0x7f28f0009590 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dbcd2a2f05e7b089 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:08:17.827 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:17.821+0000 7f28ecc00640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-09-11T00:08:17.827 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:17.821+0000 7f28ecc00640 1 -- 172.21.15.12:0/3527735276 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28f0053060 con 0x563fc35652a0 2024-09-11T00:08:17.827 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:17.821+0000 7f28ecc00640 1 -- 172.21.15.12:0/3527735276 <== mon.2 v2:172.21.15.165:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 9208+0+0 (secure 0 0 0) 0x7f28f0003b00 con 0x563fc35652a0 2024-09-11T00:08:17.828 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:17.821+0000 7f290c67f740 1 -- 172.21.15.12:0/3527735276 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f28f0040070 con 0x563fc35652a0 2024-09-11T00:08:17.828 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:17.822+0000 7f28ecc00640 1 -- 172.21.15.12:0/3527735276 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f28f005a050 con 0x563fc35652a0 2024-09-11T00:08:17.828 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:17.822+0000 7f290c67f740 10 client.25643 fetch_fsmap learned FSMap version 29 2024-09-11T00:08:17.828 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:17.822+0000 7f290c67f740 1 -- 172.21.15.12:0/3527735276 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x563fc329f620 con 0x563fc35652a0 2024-09-11T00:08:17.828 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:17.822+0000 7f28ecc00640 1 -- 172.21.15.12:0/3527735276 <== mon.2 v2:172.21.15.165:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f28f0040070 con 0x563fc35652a0 2024-09-11T00:08:17.828 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:17.822+0000 7f28ecc00640 1 -- 172.21.15.12:0/3527735276 <== mon.2 v2:172.21.15.165:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7f28f004d030 con 0x563fc35652a0 2024-09-11T00:08:17.828 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:17.822+0000 7f290c67f740 10 client.25643 fetch_fsmap finished waiting for FSMap version 29 2024-09-11T00:08:17.828 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:17.822+0000 7f290c67f740 10 client.25643 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T00:08:17.828 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:17.822+0000 7f290c67f740 20 client.25643 populate_metadata read hostname 'smithi012' 2024-09-11T00:08:17.829 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:17.822+0000 7f290c67f740 1 --2- 172.21.15.12:0/3527735276 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x563fc35774e0 0x563fc35798d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:08:17.829 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:17.823+0000 7f290c67f740 4 client.25643 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T00:08:17.829 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:17.823+0000 7f290c67f740 1 -- 172.21.15.12:0/3527735276 --> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563fc34d7600 con 0x563fc35774e0 2024-09-11T00:08:17.829 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:17.823+0000 7f2902a00640 1 --2- 172.21.15.12:0/3527735276 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x563fc35774e0 0x563fc35798d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:08:17.830 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:17.823+0000 7f2902a00640 1 --2- 172.21.15.12:0/3527735276 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x563fc35774e0 0x563fc35798d0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=25c3394850bf10d8 server_cookie=779219d9d377c9e9 in_seq=0 out_seq=0 2024-09-11T00:08:17.830 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:17.823+0000 7f28ecc00640 10 client.25643 ms_handle_connect on v2:172.21.15.165:6838/501312984 2024-09-11T00:08:17.830 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:17.825+0000 7f28ecc00640 1 -- 172.21.15.12:0/3527735276 <== mds.0 v2:172.21.15.165:6838/501312984 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x563fc34d7600 con 0x563fc35774e0 2024-09-11T00:08:17.830 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:17.825+0000 7f28ecc00640 10 client.25643 handle_command_reply: tid=0 2024-09-11T00:08:17.830 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:17.825+0000 7f290c67f740 1 client.25643 shutdown 2024-09-11T00:08:17.830 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:17.825+0000 7f290c67f740 1 -- 172.21.15.12:0/3527735276 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563fc35652a0 msgr2=0x563fc356c6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:08:17.830 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:17.825+0000 7f290c67f740 1 --2- 172.21.15.12:0/3527735276 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563fc35652a0 0x563fc356c6e0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f28f002a430 tx=0x7f28f0009590 comp rx=0 tx=0).stop 2024-09-11T00:08:17.830 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:17.825+0000 7f290c67f740 1 -- 172.21.15.12:0/3527735276 shutdown_connections 2024-09-11T00:08:17.831 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:17.825+0000 7f290c67f740 1 --2- 172.21.15.12:0/3527735276 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x563fc35774e0 0x563fc35798d0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:08:17.831 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:17.825+0000 7f290c67f740 1 --2- 172.21.15.12:0/3527735276 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563fc3567ce0 0x563fc356d2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:08:17.831 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:17.825+0000 7f290c67f740 1 --2- 172.21.15.12:0/3527735276 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563fc35652a0 0x563fc356c6e0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:08:17.831 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:17.825+0000 7f290c67f740 1 --2- 172.21.15.12:0/3527735276 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563fc3562970 0x563fc356c040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:08:17.831 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:17.825+0000 7f28ecc00640 0 client.25643 ms_handle_reset on v2:172.21.15.165:6838/501312984 2024-09-11T00:08:17.831 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:17.825+0000 7f290c67f740 1 -- 172.21.15.12:0/3527735276 >> 172.21.15.12:0/3527735276 conn(0x563fc355dac0 msgr2=0x563fc355cbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:08:17.831 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:17.826+0000 7f290c67f740 1 -- 172.21.15.12:0/3527735276 shutdown_connections 2024-09-11T00:08:17.831 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:17.826+0000 7f290c67f740 1 -- 172.21.15.12:0/3527735276 wait complete. 2024-09-11T00:08:17.831 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:17.826+0000 7f290c67f740 20 client.25643 trim_cache size 0 max 16384 2024-09-11T00:08:17.845 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:17.843+0000 7f290c67f740 1 Processor -- start 2024-09-11T00:08:17.845 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:17.843+0000 7f290c67f740 1 -- start start 2024-09-11T00:08:17.845 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:17.843+0000 7f290c67f740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563fc3562970 0x563fc355e320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:08:17.845 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:17.843+0000 7f290c67f740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563fc35652a0 0x563fc355e860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:08:17.845 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:17.843+0000 7f290c67f740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563fc3567ce0 0x563fc356c040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:08:17.845 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:17.843+0000 7f290c67f740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f29041ad760 con 0x563fc3562970 2024-09-11T00:08:17.845 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:17.843+0000 7f290c67f740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f29041af2a0 con 0x563fc3567ce0 2024-09-11T00:08:17.846 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:17.843+0000 7f290c67f740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f29041ade30 con 0x563fc35652a0 2024-09-11T00:08:17.847 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:17.844+0000 7f2903400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563fc3567ce0 0x563fc356c040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:08:17.847 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:17.844+0000 7f2902000640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563fc35652a0 0x563fc355e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:08:17.847 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:17.844+0000 7f2903400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563fc3567ce0 0x563fc356c040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.12:48860/0 (socket says 172.21.15.12:48860) 2024-09-11T00:08:17.847 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:17.844+0000 7f2902a00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563fc3562970 0x563fc355e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:08:17.848 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:17.844+0000 7f2903400640 1 -- 172.21.15.12:0/3904862248 learned_addr learned my addr 172.21.15.12:0/3904862248 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:08:17.848 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:17.844+0000 7f2902000640 1 -- 172.21.15.12:0/3904862248 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563fc3567ce0 msgr2=0x563fc356c040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:08:17.848 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:17.844+0000 7f2902000640 1 --2- 172.21.15.12:0/3904862248 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563fc3567ce0 0x563fc356c040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:08:17.848 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:17.844+0000 7f2902000640 1 -- 172.21.15.12:0/3904862248 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563fc3562970 msgr2=0x563fc355e320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:08:17.848 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:17.844+0000 7f2902000640 1 --2- 172.21.15.12:0/3904862248 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563fc3562970 0x563fc355e320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:08:17.848 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:17.844+0000 7f2902000640 1 -- 172.21.15.12:0/3904862248 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563fc3579c40 con 0x563fc35652a0 2024-09-11T00:08:17.848 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:17.844+0000 7f2902000640 1 --2- 172.21.15.12:0/3904862248 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563fc35652a0 0x563fc355e860 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f28f0006010 tx=0x7f28f0004080 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8b91e72601779db1 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:08:17.848 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:17.844+0000 7f28ef400640 1 -- 172.21.15.12:0/3904862248 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28f0053070 con 0x563fc35652a0 2024-09-11T00:08:17.848 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:17.844+0000 7f28ef400640 1 -- 172.21.15.12:0/3904862248 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f28f0040040 con 0x563fc35652a0 2024-09-11T00:08:17.848 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:17.844+0000 7f28ef400640 1 -- 172.21.15.12:0/3904862248 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28f0035070 con 0x563fc35652a0 2024-09-11T00:08:17.848 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:17.845+0000 7f290c67f740 1 -- 172.21.15.12:0/3904862248 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563fc35652a0 msgr2=0x563fc355e860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:08:17.848 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:17.845+0000 7f290c67f740 1 --2- 172.21.15.12:0/3904862248 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563fc35652a0 0x563fc355e860 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f28f0006010 tx=0x7f28f0004080 comp rx=0 tx=0).stop 2024-09-11T00:08:17.848 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:17.845+0000 7f290c67f740 1 -- 172.21.15.12:0/3904862248 shutdown_connections 2024-09-11T00:08:17.849 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:17.845+0000 7f290c67f740 1 --2- 172.21.15.12:0/3904862248 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563fc3567ce0 0x563fc356c040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:08:17.849 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:17.845+0000 7f290c67f740 1 --2- 172.21.15.12:0/3904862248 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563fc35652a0 0x563fc355e860 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:08:17.849 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:17.845+0000 7f290c67f740 1 --2- 172.21.15.12:0/3904862248 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563fc3562970 0x563fc355e320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:08:17.849 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:17.845+0000 7f290c67f740 1 -- 172.21.15.12:0/3904862248 >> 172.21.15.12:0/3904862248 conn(0x563fc355dac0 msgr2=0x563fc3561e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:08:17.849 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:17.845+0000 7f290c67f740 1 -- 172.21.15.12:0/3904862248 shutdown_connections 2024-09-11T00:08:17.849 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:17.845+0000 7f290c67f740 1 -- 172.21.15.12:0/3904862248 wait complete. 2024-09-11T00:08:17.849 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:17.846+0000 7f290c67f740 1 -- start start 2024-09-11T00:08:17.851 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:17.846+0000 7f290c67f740 1 Processor -- start 2024-09-11T00:08:17.851 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:17.846+0000 7f290c67f740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563fc3562970 0x563fc3573d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:08:17.851 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:17.846+0000 7f290c67f740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563fc35652a0 0x563fc3572af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:08:17.851 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:17.846+0000 7f290c67f740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563fc3567ce0 0x563fc35731e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:08:17.852 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:17.846+0000 7f290c67f740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f29041aebd0 con 0x563fc3562970 2024-09-11T00:08:17.852 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:17.846+0000 7f290c67f740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f29041ae500 con 0x563fc3567ce0 2024-09-11T00:08:17.852 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:17.846+0000 7f290c67f740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f29041ab740 con 0x563fc35652a0 2024-09-11T00:08:17.852 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:17.846+0000 7f290c67f740 10 client.0 _ll_register_callbacks cb 0x7f28fc009900 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:08:17.852 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:17.846+0000 7f2902a00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563fc3562970 0x563fc3573d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:08:17.852 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:17.846+0000 7f2902a00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563fc3562970 0x563fc3573d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:55508/0 (socket says 172.21.15.12:55508) 2024-09-11T00:08:17.852 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:17.847+0000 7f2902000640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563fc35652a0 0x563fc3572af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:08:17.852 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:17.847+0000 7f2902a00640 1 -- 172.21.15.12:0/1584895526 learned_addr learned my addr 172.21.15.12:0/1584895526 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:08:17.852 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:17.847+0000 7f2903400640 1 --2- 172.21.15.12:0/1584895526 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563fc3567ce0 0x563fc35731e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:08:17.852 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:17.847+0000 7f2902000640 1 -- 172.21.15.12:0/1584895526 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563fc3567ce0 msgr2=0x563fc35731e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:08:17.852 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:17.847+0000 7f2902000640 1 --2- 172.21.15.12:0/1584895526 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563fc3567ce0 0x563fc35731e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:08:17.852 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:17.847+0000 7f2902000640 1 -- 172.21.15.12:0/1584895526 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563fc3562970 msgr2=0x563fc3573d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:08:17.853 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:17.847+0000 7f2902000640 1 --2- 172.21.15.12:0/1584895526 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563fc3562970 0x563fc3573d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:08:17.853 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:17.847+0000 7f2902000640 1 -- 172.21.15.12:0/1584895526 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563fc3571590 con 0x563fc35652a0 2024-09-11T00:08:17.853 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:17.847+0000 7f2902a00640 1 --2- 172.21.15.12:0/1584895526 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563fc3562970 0x563fc3573d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:08:17.853 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:17.847+0000 7f2902000640 1 --2- 172.21.15.12:0/1584895526 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563fc35652a0 0x563fc3572af0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f28f0009800 tx=0x7f28f00456c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6609aa0270cc1d22 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:08:17.853 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:17.847+0000 7f28ecc00640 10 client.0 ms_handle_connect on v2:172.21.15.178:3300/0 2024-09-11T00:08:17.853 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:17.847+0000 7f28ecc00640 1 -- 172.21.15.12:0/1584895526 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28f0053070 con 0x563fc35652a0 2024-09-11T00:08:17.853 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:17.847+0000 7f290c67f740 1 -- 172.21.15.12:0/1584895526 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f28f0040040 con 0x563fc35652a0 2024-09-11T00:08:17.853 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:17.847+0000 7f28ecc00640 1 -- 172.21.15.12:0/1584895526 <== mon.1 v2:172.21.15.178:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 9208+0+0 (secure 0 0 0) 0x7f28f0031630 con 0x563fc35652a0 2024-09-11T00:08:17.853 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:17.847+0000 7f28ecc00640 1 -- 172.21.15.12:0/1584895526 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f28f006f050 con 0x563fc35652a0 2024-09-11T00:08:17.853 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:17.847+0000 7f290c67f740 10 client.25633 fetch_fsmap learned FSMap version 29 2024-09-11T00:08:17.853 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:17.848+0000 7f290c67f740 1 -- 172.21.15.12:0/1584895526 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x563fc35e4650 con 0x563fc35652a0 2024-09-11T00:08:17.853 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:17.848+0000 7f28ecc00640 1 -- 172.21.15.12:0/1584895526 <== mon.1 v2:172.21.15.178:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f28f0040040 con 0x563fc35652a0 2024-09-11T00:08:17.854 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:17.848+0000 7f28ecc00640 1 -- 172.21.15.12:0/1584895526 <== mon.1 v2:172.21.15.178:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7f28f004d040 con 0x563fc35652a0 2024-09-11T00:08:17.854 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:17.848+0000 7f290c67f740 10 client.25633 fetch_fsmap finished waiting for FSMap version 29 2024-09-11T00:08:17.854 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:17.848+0000 7f290c67f740 10 client.25633 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T00:08:17.854 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:17.848+0000 7f290c67f740 20 client.25633 populate_metadata read hostname 'smithi012' 2024-09-11T00:08:17.854 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:17.848+0000 7f290c67f740 1 --2- 172.21.15.12:0/1584895526 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x563fc35e4fd0 0x563fc35e73f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:08:17.854 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:17.848+0000 7f290c67f740 4 client.25633 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T00:08:17.854 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:17.848+0000 7f290c67f740 1 -- 172.21.15.12:0/1584895526 --> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] -- command(tid 0: {"prefix": "flush journal"}) -- 0x563fc34d30b0 con 0x563fc35e4fd0 2024-09-11T00:08:17.854 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:17.848+0000 7f2902a00640 1 --2- 172.21.15.12:0/1584895526 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x563fc35e4fd0 0x563fc35e73f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:08:17.854 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:17.849+0000 7f2902a00640 1 --2- 172.21.15.12:0/1584895526 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x563fc35e4fd0 0x563fc35e73f0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e745555dd4bd44e3 server_cookie=8b30b9a22b15e925 in_seq=0 out_seq=0 2024-09-11T00:08:17.854 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:17.849+0000 7f28ecc00640 10 client.25633 ms_handle_connect on v2:172.21.15.165:6838/501312984 2024-09-11T00:08:18.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:08:17 smithi165 ceph-mon[35356]: pgmap v434: 129 pgs: 129 active+clean; 15 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 687 KiB/s rd, 1.5 MiB/s wr, 234 op/s 2024-09-11T00:08:18.063 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:08:17 smithi012 ceph-mon[30333]: pgmap v434: 129 pgs: 129 active+clean; 15 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 687 KiB/s rd, 1.5 MiB/s wr, 234 op/s 2024-09-11T00:08:18.080 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:08:17 smithi178 ceph-mon[35919]: pgmap v434: 129 pgs: 129 active+clean; 15 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 687 KiB/s rd, 1.5 MiB/s wr, 234 op/s 2024-09-11T00:08:18.130 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:18.128+0000 7f28ecc00640 1 -- 172.21.15.12:0/1584895526 <== mds.0 v2:172.21.15.165:6838/501312984 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x563fc34d30b0 con 0x563fc35e4fd0 2024-09-11T00:08:18.130 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:18.128+0000 7f28ecc00640 10 client.25633 handle_command_reply: tid=0 2024-09-11T00:08:18.131 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:18.128+0000 7f290c67f740 1 client.25633 shutdown 2024-09-11T00:08:18.131 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:18.128+0000 7f290c67f740 1 -- 172.21.15.12:0/1584895526 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563fc35652a0 msgr2=0x563fc3572af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:08:18.131 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:18.128+0000 7f290c67f740 1 --2- 172.21.15.12:0/1584895526 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563fc35652a0 0x563fc3572af0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f28f0009800 tx=0x7f28f00456c0 comp rx=0 tx=0).stop 2024-09-11T00:08:18.131 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:18.128+0000 7f290c67f740 1 -- 172.21.15.12:0/1584895526 shutdown_connections 2024-09-11T00:08:18.131 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:18.128+0000 7f290c67f740 1 --2- 172.21.15.12:0/1584895526 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x563fc35e4fd0 0x563fc35e73f0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:08:18.131 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:18.128+0000 7f290c67f740 1 --2- 172.21.15.12:0/1584895526 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563fc3567ce0 0x563fc35731e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:08:18.131 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:18.128+0000 7f28ecc00640 0 client.25633 ms_handle_reset on v2:172.21.15.165:6838/501312984 2024-09-11T00:08:18.131 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:18.128+0000 7f290c67f740 1 --2- 172.21.15.12:0/1584895526 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563fc35652a0 0x563fc3572af0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:08:18.131 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:18.128+0000 7f290c67f740 1 --2- 172.21.15.12:0/1584895526 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563fc3562970 0x563fc3573d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:08:18.131 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:18.128+0000 7f290c67f740 1 -- 172.21.15.12:0/1584895526 >> 172.21.15.12:0/1584895526 conn(0x563fc355dac0 msgr2=0x563fc356c020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:08:18.131 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:18.128+0000 7f290c67f740 1 -- 172.21.15.12:0/1584895526 shutdown_connections 2024-09-11T00:08:18.131 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:18.128+0000 7f290c67f740 1 -- 172.21.15.12:0/1584895526 wait complete. 2024-09-11T00:08:18.131 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:18.128+0000 7f290c67f740 20 client.25633 trim_cache size 0 max 16384 2024-09-11T00:08:18.132 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:{ 2024-09-11T00:08:18.132 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "message": "", 2024-09-11T00:08:18.132 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "return_code": 0 2024-09-11T00:08:18.132 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:} 2024-09-11T00:08:18.133 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:18.129+0000 7f2903e00640 1 -- 172.21.15.12:0/15221900 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f28e80b52e0 msgr2=0x7f28e80b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:08:18.133 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:18.129+0000 7f2903e00640 1 --2- 172.21.15.12:0/15221900 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f28e80b52e0 0x7f28e80b77a0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f290419bb50 tx=0x7f28fc009500 comp rx=0 tx=0).stop 2024-09-11T00:08:18.133 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:18.129+0000 7f2903e00640 1 -- 172.21.15.12:0/15221900 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f29041049d0 msgr2=0x7f290419add0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:08:18.133 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:18.129+0000 7f2903e00640 1 --2- 172.21.15.12:0/15221900 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f29041049d0 0x7f290419add0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f28f4005fd0 tx=0x7f28f400dd20 comp rx=0 tx=0).stop 2024-09-11T00:08:18.133 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:18.129+0000 7f2903e00640 1 -- 172.21.15.12:0/15221900 shutdown_connections 2024-09-11T00:08:18.133 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:18.129+0000 7f2903e00640 1 --2- 172.21.15.12:0/15221900 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f28e80b52e0 0x7f28e80b77a0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:08:18.133 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:18.129+0000 7f2903e00640 1 --2- 172.21.15.12:0/15221900 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f29041064e0 0x7f290419c050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:08:18.133 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:18.129+0000 7f2903e00640 1 --2- 172.21.15.12:0/15221900 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2904105b20 0x7f290419b420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:08:18.134 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:18.129+0000 7f2903e00640 1 --2- 172.21.15.12:0/15221900 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f29041049d0 0x7f290419add0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:08:18.134 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:18.129+0000 7f2903e00640 1 -- 172.21.15.12:0/15221900 >> 172.21.15.12:0/15221900 conn(0x7f2904100270 msgr2=0x7f2904108eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:08:18.134 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:18.129+0000 7f2903e00640 1 -- 172.21.15.12:0/15221900 shutdown_connections 2024-09-11T00:08:18.134 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:18.129+0000 7f2903e00640 1 -- 172.21.15.12:0/15221900 wait complete. 2024-09-11T00:08:18.589 INFO:tasks.workunit.client.0.smithi012.stdout: 1 44156 17.04 MB/sec warmup 15 sec latency 108.460 ms 2024-09-11T00:08:19.589 INFO:tasks.workunit.client.0.smithi012.stdout: 1 46833 16.86 MB/sec warmup 16 sec latency 19.396 ms 2024-09-11T00:08:19.913 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:08:19 smithi012 ceph-mon[30333]: pgmap v435: 129 pgs: 129 active+clean; 18 MiB data, 473 MiB used, 1.0 TiB / 1.0 TiB avail; 730 KiB/s rd, 1.5 MiB/s wr, 214 op/s 2024-09-11T00:08:20.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:08:19 smithi165 ceph-mon[35356]: pgmap v435: 129 pgs: 129 active+clean; 18 MiB data, 473 MiB used, 1.0 TiB / 1.0 TiB avail; 730 KiB/s rd, 1.5 MiB/s wr, 214 op/s 2024-09-11T00:08:20.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:08:19 smithi178 ceph-mon[35919]: pgmap v435: 129 pgs: 129 active+clean; 18 MiB data, 473 MiB used, 1.0 TiB / 1.0 TiB avail; 730 KiB/s rd, 1.5 MiB/s wr, 214 op/s 2024-09-11T00:08:20.589 INFO:tasks.workunit.client.0.smithi012.stdout: 1 50009 16.96 MB/sec warmup 17 sec latency 8.274 ms 2024-09-11T00:08:21.589 INFO:tasks.workunit.client.0.smithi012.stdout: 1 53016 16.84 MB/sec warmup 18 sec latency 18.212 ms 2024-09-11T00:08:22.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:08:21 smithi165 ceph-mon[35356]: pgmap v436: 129 pgs: 129 active+clean; 19 MiB data, 480 MiB used, 1.0 TiB / 1.0 TiB avail; 768 KiB/s rd, 1.8 MiB/s wr, 376 op/s 2024-09-11T00:08:22.062 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:08:21 smithi012 ceph-mon[30333]: pgmap v436: 129 pgs: 129 active+clean; 19 MiB data, 480 MiB used, 1.0 TiB / 1.0 TiB avail; 768 KiB/s rd, 1.8 MiB/s wr, 376 op/s 2024-09-11T00:08:22.080 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:08:21 smithi178 ceph-mon[35919]: pgmap v436: 129 pgs: 129 active+clean; 19 MiB data, 480 MiB used, 1.0 TiB / 1.0 TiB avail; 768 KiB/s rd, 1.8 MiB/s wr, 376 op/s 2024-09-11T00:08:22.589 INFO:tasks.workunit.client.0.smithi012.stdout: 1 55691 16.58 MB/sec warmup 19 sec latency 10.565 ms 2024-09-11T00:08:23.589 INFO:tasks.workunit.client.0.smithi012.stdout: 1 58268 16.60 MB/sec warmup 20 sec latency 10.979 ms 2024-09-11T00:08:24.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:08:23 smithi165 ceph-mon[35356]: pgmap v437: 129 pgs: 129 active+clean; 14 MiB data, 485 MiB used, 1.0 TiB / 1.0 TiB avail; 935 KiB/s rd, 1.8 MiB/s wr, 451 op/s 2024-09-11T00:08:24.063 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:08:23 smithi012 ceph-mon[30333]: pgmap v437: 129 pgs: 129 active+clean; 14 MiB data, 485 MiB used, 1.0 TiB / 1.0 TiB avail; 935 KiB/s rd, 1.8 MiB/s wr, 451 op/s 2024-09-11T00:08:24.080 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:08:23 smithi178 ceph-mon[35919]: pgmap v437: 129 pgs: 129 active+clean; 14 MiB data, 485 MiB used, 1.0 TiB / 1.0 TiB avail; 935 KiB/s rd, 1.8 MiB/s wr, 451 op/s 2024-09-11T00:08:24.590 INFO:tasks.workunit.client.0.smithi012.stdout: 1 61014 16.49 MB/sec warmup 21 sec latency 10.858 ms 2024-09-11T00:08:25.590 INFO:tasks.workunit.client.0.smithi012.stdout: 1 64215 16.60 MB/sec warmup 22 sec latency 6.749 ms 2024-09-11T00:08:26.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:08:25 smithi165 ceph-mon[35356]: pgmap v438: 129 pgs: 129 active+clean; 14 MiB data, 490 MiB used, 1.0 TiB / 1.0 TiB avail; 559 KiB/s rd, 1.3 MiB/s wr, 379 op/s 2024-09-11T00:08:26.063 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:08:25 smithi012 ceph-mon[30333]: pgmap v438: 129 pgs: 129 active+clean; 14 MiB data, 490 MiB used, 1.0 TiB / 1.0 TiB avail; 559 KiB/s rd, 1.3 MiB/s wr, 379 op/s 2024-09-11T00:08:26.080 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:08:25 smithi178 ceph-mon[35919]: pgmap v438: 129 pgs: 129 active+clean; 14 MiB data, 490 MiB used, 1.0 TiB / 1.0 TiB avail; 559 KiB/s rd, 1.3 MiB/s wr, 379 op/s 2024-09-11T00:08:26.590 INFO:tasks.workunit.client.0.smithi012.stdout: 1 66922 16.43 MB/sec warmup 23 sec latency 25.235 ms 2024-09-11T00:08:27.589 INFO:tasks.workunit.client.0.smithi012.stdout: 1 69480 16.25 MB/sec warmup 24 sec latency 12.385 ms 2024-09-11T00:08:28.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:08:27 smithi165 ceph-mon[35356]: pgmap v439: 129 pgs: 129 active+clean; 21 MiB data, 487 MiB used, 1.0 TiB / 1.0 TiB avail; 972 KiB/s rd, 2.0 MiB/s wr, 481 op/s 2024-09-11T00:08:28.062 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:08:27 smithi012 ceph-mon[30333]: pgmap v439: 129 pgs: 129 active+clean; 21 MiB data, 487 MiB used, 1.0 TiB / 1.0 TiB avail; 972 KiB/s rd, 2.0 MiB/s wr, 481 op/s 2024-09-11T00:08:28.080 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:08:27 smithi178 ceph-mon[35919]: pgmap v439: 129 pgs: 129 active+clean; 21 MiB data, 487 MiB used, 1.0 TiB / 1.0 TiB avail; 972 KiB/s rd, 2.0 MiB/s wr, 481 op/s 2024-09-11T00:08:28.590 INFO:tasks.workunit.client.0.smithi012.stdout: 1 71765 16.16 MB/sec warmup 25 sec latency 13.643 ms 2024-09-11T00:08:29.590 INFO:tasks.workunit.client.0.smithi012.stdout: 1 74868 16.24 MB/sec warmup 26 sec latency 12.673 ms 2024-09-11T00:08:30.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:08:29 smithi165 ceph-mon[35356]: pgmap v440: 129 pgs: 129 active+clean; 21 MiB data, 491 MiB used, 1.0 TiB / 1.0 TiB avail; 794 KiB/s rd, 1.6 MiB/s wr, 431 op/s 2024-09-11T00:08:30.062 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:08:29 smithi012 ceph-mon[30333]: pgmap v440: 129 pgs: 129 active+clean; 21 MiB data, 491 MiB used, 1.0 TiB / 1.0 TiB avail; 794 KiB/s rd, 1.6 MiB/s wr, 431 op/s 2024-09-11T00:08:30.080 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:08:29 smithi178 ceph-mon[35919]: pgmap v440: 129 pgs: 129 active+clean; 21 MiB data, 491 MiB used, 1.0 TiB / 1.0 TiB avail; 794 KiB/s rd, 1.6 MiB/s wr, 431 op/s 2024-09-11T00:08:30.590 INFO:tasks.workunit.client.0.smithi012.stdout: 1 77746 16.14 MB/sec warmup 27 sec latency 10.646 ms 2024-09-11T00:08:31.265 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:31.262+0000 7f223be00640 1 -- 172.21.15.12:0/1786391401 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f223c105b20 msgr2=0x7f223c105fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:08:31.265 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:31.262+0000 7f223be00640 1 --2- 172.21.15.12:0/1786391401 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f223c105b20 0x7f223c105fa0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f2228009900 tx=0x7f222802a3f0 comp rx=0 tx=0).stop 2024-09-11T00:08:31.265 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:31.262+0000 7f223be00640 1 -- 172.21.15.12:0/1786391401 shutdown_connections 2024-09-11T00:08:31.265 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:31.262+0000 7f223be00640 1 --2- 172.21.15.12:0/1786391401 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f223c1064e0 0x7f223c10ce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:08:31.265 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:31.262+0000 7f223be00640 1 --2- 172.21.15.12:0/1786391401 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f223c105b20 0x7f223c105fa0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:08:31.265 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:31.262+0000 7f223be00640 1 --2- 172.21.15.12:0/1786391401 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f223c1049d0 0x7f223c104dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:08:31.265 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:31.262+0000 7f223be00640 1 -- 172.21.15.12:0/1786391401 >> 172.21.15.12:0/1786391401 conn(0x7f223c100270 msgr2=0x7f223c1026b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:08:31.265 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:31.262+0000 7f223be00640 1 -- 172.21.15.12:0/1786391401 shutdown_connections 2024-09-11T00:08:31.265 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:31.262+0000 7f223be00640 1 -- 172.21.15.12:0/1786391401 wait complete. 2024-09-11T00:08:31.267 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:31.264+0000 7f223be00640 1 Processor -- start 2024-09-11T00:08:31.268 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:31.264+0000 7f223be00640 1 -- start start 2024-09-11T00:08:31.268 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:31.264+0000 7f223be00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f223c1049d0 0x7f223c19ae70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:08:31.268 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:31.264+0000 7f223be00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f223c105b20 0x7f223c19b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:08:31.268 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:31.264+0000 7f223be00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f223c1064e0 0x7f223c19c080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:08:31.268 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:31.264+0000 7f223be00640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f223c110690 con 0x7f223c1049d0 2024-09-11T00:08:31.268 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:31.264+0000 7f223be00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f223c110510 con 0x7f223c105b20 2024-09-11T00:08:31.268 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:31.264+0000 7f223be00640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f223c110810 con 0x7f223c1064e0 2024-09-11T00:08:31.268 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:31.264+0000 7f223a000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f223c105b20 0x7f223c19b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:08:31.268 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:31.264+0000 7f223aa00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f223c1049d0 0x7f223c19ae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:08:31.268 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:31.264+0000 7f223a000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f223c105b20 0x7f223c19b4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.12:33376/0 (socket says 172.21.15.12:33376) 2024-09-11T00:08:31.268 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:31.264+0000 7f223a000640 1 -- 172.21.15.12:0/3306246232 learned_addr learned my addr 172.21.15.12:0/3306246232 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:08:31.268 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:31.264+0000 7f223b400640 1 --2- 172.21.15.12:0/3306246232 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f223c1064e0 0x7f223c19c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:08:31.269 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:31.265+0000 7f223a000640 1 -- 172.21.15.12:0/3306246232 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f223c1064e0 msgr2=0x7f223c19c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:08:31.269 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:31.265+0000 7f223a000640 1 --2- 172.21.15.12:0/3306246232 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f223c1064e0 0x7f223c19c080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:08:31.269 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:31.265+0000 7f223a000640 1 -- 172.21.15.12:0/3306246232 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f223c1049d0 msgr2=0x7f223c19ae70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:08:31.269 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:31.265+0000 7f223a000640 1 --2- 172.21.15.12:0/3306246232 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f223c1049d0 0x7f223c19ae70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:08:31.269 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:31.265+0000 7f223a000640 1 -- 172.21.15.12:0/3306246232 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f223c19c940 con 0x7f223c105b20 2024-09-11T00:08:31.269 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:31.265+0000 7f223b400640 1 --2- 172.21.15.12:0/3306246232 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f223c1064e0 0x7f223c19c080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:08:31.269 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:31.265+0000 7f223a000640 1 --2- 172.21.15.12:0/3306246232 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f223c105b20 0x7f223c19b4c0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f2228031490 tx=0x7f2228003c10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:08:31.269 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:31.265+0000 7f2233400640 1 -- 172.21.15.12:0/3306246232 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f222803f070 con 0x7f223c105b20 2024-09-11T00:08:31.269 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:31.265+0000 7f2233400640 1 -- 172.21.15.12:0/3306246232 <== mon.2 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2228031bf0 con 0x7f223c105b20 2024-09-11T00:08:31.269 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:31.265+0000 7f2233400640 1 -- 172.21.15.12:0/3306246232 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2228035030 con 0x7f223c105b20 2024-09-11T00:08:31.269 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:31.265+0000 7f223be00640 1 -- 172.21.15.12:0/3306246232 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f223c19cbd0 con 0x7f223c105b20 2024-09-11T00:08:31.269 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:31.266+0000 7f223be00640 1 -- 172.21.15.12:0/3306246232 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f223c1a3120 con 0x7f223c105b20 2024-09-11T00:08:31.270 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:31.268+0000 7f224484e740 1 Processor -- start 2024-09-11T00:08:31.270 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:31.269+0000 7f224484e740 1 -- start start 2024-09-11T00:08:31.271 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:31.269+0000 7f224484e740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55efd40da970 0x55efd40dcd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:08:31.272 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:31.269+0000 7f224484e740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55efd40dd2a0 0x55efd40df690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:08:31.272 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:31.269+0000 7f224484e740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55efd40dfce0 0x55efd40e21a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:08:31.272 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:31.269+0000 7f224484e740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55efd404d6b0 con 0x55efd40dd2a0 2024-09-11T00:08:31.272 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:31.269+0000 7f224484e740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55efd403cae0 con 0x55efd40dfce0 2024-09-11T00:08:31.272 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:31.269+0000 7f224484e740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55efd40408f0 con 0x55efd40da970 2024-09-11T00:08:31.272 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:31.269+0000 7f2233400640 1 -- 172.21.15.12:0/3306246232 <== mon.2 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7f2228031740 con 0x7f223c105b20 2024-09-11T00:08:31.272 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:31.269+0000 7f223b400640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55efd40da970 0x55efd40dcd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:08:31.272 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:31.269+0000 7f223b400640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55efd40da970 0x55efd40dcd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.12:51372/0 (socket says 172.21.15.12:51372) 2024-09-11T00:08:31.272 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:31.269+0000 7f223b400640 1 -- 172.21.15.12:0/1489474476 learned_addr learned my addr 172.21.15.12:0/1489474476 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:08:31.272 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:31.269+0000 7f223aa00640 1 --2- 172.21.15.12:0/1489474476 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55efd40dd2a0 0x55efd40df690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:08:31.272 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:31.269+0000 7f223a000640 1 --2- 172.21.15.12:0/1489474476 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55efd40dfce0 0x55efd40e21a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:08:31.272 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:31.270+0000 7f223aa00640 1 -- 172.21.15.12:0/1489474476 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55efd40da970 msgr2=0x55efd40dcd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:08:31.273 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:31.270+0000 7f223aa00640 1 --2- 172.21.15.12:0/1489474476 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55efd40da970 0x55efd40dcd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:08:31.273 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:31.270+0000 7f223aa00640 1 -- 172.21.15.12:0/1489474476 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55efd40dfce0 msgr2=0x55efd40e21a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:08:31.273 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:31.270+0000 7f223aa00640 1 --2- 172.21.15.12:0/1489474476 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55efd40dfce0 0x55efd40e21a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:08:31.273 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:31.270+0000 7f223aa00640 1 -- 172.21.15.12:0/1489474476 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55efd4009780 con 0x55efd40dd2a0 2024-09-11T00:08:31.274 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:31.270+0000 7f223aa00640 1 --2- 172.21.15.12:0/1489474476 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55efd40dd2a0 0x55efd40df690 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f222c009c20 tx=0x7f222c011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac1e5c734a184ffb server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:08:31.274 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:31.270+0000 7f2227e00640 1 -- 172.21.15.12:0/1489474476 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f222c004050 con 0x55efd40dd2a0 2024-09-11T00:08:31.275 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:31.271+0000 7f2227e00640 1 -- 172.21.15.12:0/1489474476 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f222c0054d0 con 0x55efd40dd2a0 2024-09-11T00:08:31.275 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:31.271+0000 7f2227e00640 1 -- 172.21.15.12:0/1489474476 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f222c004ca0 con 0x55efd40dd2a0 2024-09-11T00:08:31.275 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:31.271+0000 7f2233400640 1 --2- 172.21.15.12:0/3306246232 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f22200b5360 0x7f22200b7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:08:31.275 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:31.271+0000 7f224484e740 1 -- 172.21.15.12:0/1489474476 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55efd40dd2a0 msgr2=0x55efd40df690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:08:31.275 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:31.271+0000 7f224484e740 1 --2- 172.21.15.12:0/1489474476 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55efd40dd2a0 0x55efd40df690 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f222c009c20 tx=0x7f222c011810 comp rx=0 tx=0).stop 2024-09-11T00:08:31.275 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:31.272+0000 7f2233400640 1 -- 172.21.15.12:0/3306246232 <== mon.2 v2:172.21.15.165:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9208+0+0 (secure 0 0 0) 0x7f2228106980 con 0x7f223c105b20 2024-09-11T00:08:31.275 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:31.272+0000 7f223b400640 1 --2- 172.21.15.12:0/3306246232 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f22200b5360 0x7f22200b7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:08:31.275 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:31.272+0000 7f224484e740 1 -- 172.21.15.12:0/1489474476 shutdown_connections 2024-09-11T00:08:31.275 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:31.272+0000 7f224484e740 1 --2- 172.21.15.12:0/1489474476 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55efd40dfce0 0x55efd40e21a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:08:31.275 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:31.272+0000 7f224484e740 1 --2- 172.21.15.12:0/1489474476 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55efd40dd2a0 0x55efd40df690 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:08:31.275 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:31.272+0000 7f224484e740 1 --2- 172.21.15.12:0/1489474476 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55efd40da970 0x55efd40dcd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:08:31.275 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:31.272+0000 7f224484e740 1 -- 172.21.15.12:0/1489474476 >> 172.21.15.12:0/1489474476 conn(0x55efd40d5ac0 msgr2=0x55efd40d6850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:08:31.275 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:31.272+0000 7f224484e740 1 -- 172.21.15.12:0/1489474476 shutdown_connections 2024-09-11T00:08:31.276 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:31.272+0000 7f224484e740 1 -- 172.21.15.12:0/1489474476 wait complete. 2024-09-11T00:08:31.276 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:31.272+0000 7f223b400640 1 --2- 172.21.15.12:0/3306246232 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f22200b5360 0x7f22200b7820 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f2234009fd0 tx=0x7f2234009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:08:31.276 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:31.272+0000 7f224484e740 1 -- start start 2024-09-11T00:08:31.276 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:31.273+0000 7f224484e740 1 Processor -- start 2024-09-11T00:08:31.279 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:31.273+0000 7f224484e740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55efd40da970 0x55efd40e40d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:08:31.279 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:31.273+0000 7f224484e740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55efd40dd2a0 0x55efd40e4770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:08:31.279 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:31.273+0000 7f224484e740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55efd40dfce0 0x55efd40e5340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:08:31.279 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:31.273+0000 7f224484e740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55efd3f74ab0 con 0x55efd40da970 2024-09-11T00:08:31.279 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:31.273+0000 7f224484e740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55efd3fe3570 con 0x55efd40dd2a0 2024-09-11T00:08:31.279 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:31.273+0000 7f224484e740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55efd40e2cc0 con 0x55efd40dfce0 2024-09-11T00:08:31.279 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:31.273+0000 7f224484e740 10 client.0 _ll_register_callbacks cb 0x55efd3ff2100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:08:31.279 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:31.274+0000 7f223b400640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55efd40dfce0 0x55efd40e5340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:08:31.279 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:31.274+0000 7f223aa00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55efd40da970 0x55efd40e40d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:08:31.279 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:31.274+0000 7f223a000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55efd40dd2a0 0x55efd40e4770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:08:31.280 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:31.274+0000 7f223a000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55efd40dd2a0 0x55efd40e4770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.12:33406/0 (socket says 172.21.15.12:33406) 2024-09-11T00:08:31.280 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:31.274+0000 7f223a000640 1 -- 172.21.15.12:0/3304570142 learned_addr learned my addr 172.21.15.12:0/3304570142 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:08:31.280 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:31.274+0000 7f223b400640 1 --2- 172.21.15.12:0/3304570142 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55efd40dfce0 0x55efd40e5340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.12:51380/0 (socket says 172.21.15.12:51380) 2024-09-11T00:08:31.280 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:31.274+0000 7f223a000640 1 -- 172.21.15.12:0/3304570142 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55efd40dfce0 msgr2=0x55efd40e5340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:08:31.280 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:31.274+0000 7f223a000640 1 --2- 172.21.15.12:0/3304570142 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55efd40dfce0 0x55efd40e5340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:08:31.280 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:31.274+0000 7f223a000640 1 -- 172.21.15.12:0/3304570142 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55efd40da970 msgr2=0x55efd40e40d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:08:31.280 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:31.274+0000 7f223a000640 1 --2- 172.21.15.12:0/3304570142 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55efd40da970 0x55efd40e40d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:08:31.280 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:31.274+0000 7f223a000640 1 -- 172.21.15.12:0/3304570142 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55efd3d9e790 con 0x55efd40dd2a0 2024-09-11T00:08:31.280 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:31.274+0000 7f223b400640 1 --2- 172.21.15.12:0/3304570142 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55efd40dfce0 0x55efd40e5340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:08:31.280 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:31.274+0000 7f223a000640 1 --2- 172.21.15.12:0/3304570142 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55efd40dd2a0 0x55efd40e4770 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f22280ccac0 tx=0x7f222802a3f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d55167ae519bfe2 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:08:31.280 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:31.274+0000 7f2224c00640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-09-11T00:08:31.280 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:31.275+0000 7f2224c00640 1 -- 172.21.15.12:0/3304570142 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2228049050 con 0x55efd40dd2a0 2024-09-11T00:08:31.281 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:31.275+0000 7f224484e740 1 -- 172.21.15.12:0/3304570142 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f222c0054d0 con 0x55efd40dd2a0 2024-09-11T00:08:31.281 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:31.275+0000 7f2224c00640 1 -- 172.21.15.12:0/3304570142 <== mon.2 v2:172.21.15.165:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 9208+0+0 (secure 0 0 0) 0x7f22280c9030 con 0x55efd40dd2a0 2024-09-11T00:08:31.281 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:31.275+0000 7f2224c00640 1 -- 172.21.15.12:0/3304570142 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f22280cc530 con 0x55efd40dd2a0 2024-09-11T00:08:31.281 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:31.275+0000 7f224484e740 10 client.25661 fetch_fsmap learned FSMap version 29 2024-09-11T00:08:31.281 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:31.275+0000 7f224484e740 1 -- 172.21.15.12:0/3304570142 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55efd3e17620 con 0x55efd40dd2a0 2024-09-11T00:08:31.281 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:31.275+0000 7f2224c00640 1 -- 172.21.15.12:0/3304570142 <== mon.2 v2:172.21.15.165:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f222c0054d0 con 0x55efd40dd2a0 2024-09-11T00:08:31.281 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:31.276+0000 7f2224c00640 1 -- 172.21.15.12:0/3304570142 <== mon.2 v2:172.21.15.165:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7f22280b40e0 con 0x55efd40dd2a0 2024-09-11T00:08:31.281 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:31.276+0000 7f224484e740 10 client.25661 fetch_fsmap finished waiting for FSMap version 29 2024-09-11T00:08:31.281 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:31.276+0000 7f224484e740 10 client.25661 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T00:08:31.281 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:31.276+0000 7f224484e740 20 client.25661 populate_metadata read hostname 'smithi012' 2024-09-11T00:08:31.281 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:31.276+0000 7f224484e740 1 --2- 172.21.15.12:0/3304570142 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55efd40ef9b0 0x55efd40f1dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:08:31.281 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:31.276+0000 7f224484e740 4 client.25661 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T00:08:31.281 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:31.276+0000 7f224484e740 1 -- 172.21.15.12:0/3304570142 --> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55efd404f600 con 0x55efd40ef9b0 2024-09-11T00:08:31.282 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:31.276+0000 7f223aa00640 1 --2- 172.21.15.12:0/3304570142 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55efd40ef9b0 0x55efd40f1dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:08:31.282 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:31.277+0000 7f223aa00640 1 --2- 172.21.15.12:0/3304570142 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55efd40ef9b0 0x55efd40f1dd0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fcab4eec32ebab2d server_cookie=e783561e210bfe48 in_seq=0 out_seq=0 2024-09-11T00:08:31.282 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:31.277+0000 7f2224c00640 10 client.25661 ms_handle_connect on v2:172.21.15.165:6838/501312984 2024-09-11T00:08:31.283 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:31.278+0000 7f2224c00640 1 -- 172.21.15.12:0/3304570142 <== mds.0 v2:172.21.15.165:6838/501312984 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55efd404f600 con 0x55efd40ef9b0 2024-09-11T00:08:31.283 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:31.278+0000 7f2224c00640 10 client.25661 handle_command_reply: tid=0 2024-09-11T00:08:31.283 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:31.278+0000 7f224484e740 1 client.25661 shutdown 2024-09-11T00:08:31.283 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:31.278+0000 7f224484e740 1 -- 172.21.15.12:0/3304570142 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55efd40dd2a0 msgr2=0x55efd40e4770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:08:31.283 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:31.278+0000 7f224484e740 1 --2- 172.21.15.12:0/3304570142 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55efd40dd2a0 0x55efd40e4770 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f22280ccac0 tx=0x7f222802a3f0 comp rx=0 tx=0).stop 2024-09-11T00:08:31.283 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:31.278+0000 7f224484e740 1 -- 172.21.15.12:0/3304570142 shutdown_connections 2024-09-11T00:08:31.283 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:31.278+0000 7f224484e740 1 --2- 172.21.15.12:0/3304570142 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55efd40ef9b0 0x55efd40f1dd0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:08:31.283 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:31.278+0000 7f224484e740 1 --2- 172.21.15.12:0/3304570142 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55efd40dfce0 0x55efd40e5340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:08:31.283 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:31.278+0000 7f224484e740 1 --2- 172.21.15.12:0/3304570142 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55efd40dd2a0 0x55efd40e4770 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:08:31.283 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:31.278+0000 7f224484e740 1 --2- 172.21.15.12:0/3304570142 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55efd40da970 0x55efd40e40d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:08:31.283 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:31.278+0000 7f2224c00640 0 client.25661 ms_handle_reset on v2:172.21.15.165:6838/501312984 2024-09-11T00:08:31.284 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:31.278+0000 7f224484e740 1 -- 172.21.15.12:0/3304570142 >> 172.21.15.12:0/3304570142 conn(0x55efd40d5ac0 msgr2=0x55efd40d4bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:08:31.284 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:31.278+0000 7f224484e740 1 -- 172.21.15.12:0/3304570142 shutdown_connections 2024-09-11T00:08:31.284 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:31.278+0000 7f224484e740 1 -- 172.21.15.12:0/3304570142 wait complete. 2024-09-11T00:08:31.284 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:31.279+0000 7f224484e740 20 client.25661 trim_cache size 0 max 16384 2024-09-11T00:08:31.304 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:31.301+0000 7f224484e740 1 Processor -- start 2024-09-11T00:08:31.304 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:31.301+0000 7f224484e740 1 -- start start 2024-09-11T00:08:31.304 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:31.301+0000 7f224484e740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55efd40da970 0x55efd40d6320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:08:31.304 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:31.301+0000 7f224484e740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55efd40dd2a0 0x55efd40d6860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:08:31.304 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:31.301+0000 7f224484e740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55efd40dfce0 0x55efd40f6690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:08:31.304 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:31.301+0000 7f224484e740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f223c1ad760 con 0x55efd40dfce0 2024-09-11T00:08:31.305 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:31.302+0000 7f224484e740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f223c1af2a0 con 0x55efd40da970 2024-09-11T00:08:31.305 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:31.302+0000 7f224484e740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f223c1ade30 con 0x55efd40dd2a0 2024-09-11T00:08:31.305 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:31.302+0000 7f223aa00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55efd40da970 0x55efd40d6320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:08:31.305 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:31.302+0000 7f223a000640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55efd40dd2a0 0x55efd40d6860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:08:31.305 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:31.302+0000 7f223b400640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55efd40dfce0 0x55efd40f6690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:08:31.305 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:31.302+0000 7f223aa00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55efd40da970 0x55efd40d6320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.12:33410/0 (socket says 172.21.15.12:33410) 2024-09-11T00:08:31.305 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:31.302+0000 7f223aa00640 1 -- 172.21.15.12:0/2900790975 learned_addr learned my addr 172.21.15.12:0/2900790975 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:08:31.305 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:31.302+0000 7f223a000640 1 -- 172.21.15.12:0/2900790975 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55efd40da970 msgr2=0x55efd40d6320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:08:31.305 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:31.302+0000 7f223a000640 1 --2- 172.21.15.12:0/2900790975 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55efd40da970 0x55efd40d6320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:08:31.305 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:31.302+0000 7f223a000640 1 -- 172.21.15.12:0/2900790975 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55efd40dfce0 msgr2=0x55efd40f6690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:08:31.305 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:31.302+0000 7f223a000640 1 --2- 172.21.15.12:0/2900790975 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55efd40dfce0 0x55efd40f6690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:08:31.305 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:31.302+0000 7f223a000640 1 -- 172.21.15.12:0/2900790975 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55efd40d1460 con 0x55efd40dd2a0 2024-09-11T00:08:31.309 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:31.302+0000 7f223a000640 1 --2- 172.21.15.12:0/2900790975 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55efd40dd2a0 0x55efd40d6860 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f222802a9b0 tx=0x7f22280d6400 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1abd2c6808686b24 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:08:31.309 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:31.303+0000 7f2227400640 1 -- 172.21.15.12:0/2900790975 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2228049070 con 0x55efd40dd2a0 2024-09-11T00:08:31.309 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:31.303+0000 7f2227400640 1 -- 172.21.15.12:0/2900790975 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f22280cc8d0 con 0x55efd40dd2a0 2024-09-11T00:08:31.309 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:31.303+0000 7f2227400640 1 -- 172.21.15.12:0/2900790975 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22280b40b0 con 0x55efd40dd2a0 2024-09-11T00:08:31.309 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:31.303+0000 7f224484e740 1 -- 172.21.15.12:0/2900790975 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55efd40dd2a0 msgr2=0x55efd40d6860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:08:31.310 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:31.303+0000 7f224484e740 1 --2- 172.21.15.12:0/2900790975 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55efd40dd2a0 0x55efd40d6860 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f222802a9b0 tx=0x7f22280d6400 comp rx=0 tx=0).stop 2024-09-11T00:08:31.310 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:31.303+0000 7f224484e740 1 -- 172.21.15.12:0/2900790975 shutdown_connections 2024-09-11T00:08:31.310 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:31.303+0000 7f224484e740 1 --2- 172.21.15.12:0/2900790975 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55efd40dfce0 0x55efd40f6690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:08:31.310 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:31.303+0000 7f224484e740 1 --2- 172.21.15.12:0/2900790975 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55efd40dd2a0 0x55efd40d6860 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:08:31.310 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:31.303+0000 7f224484e740 1 --2- 172.21.15.12:0/2900790975 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55efd40da970 0x55efd40d6320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:08:31.310 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:31.303+0000 7f224484e740 1 -- 172.21.15.12:0/2900790975 >> 172.21.15.12:0/2900790975 conn(0x55efd40d5ac0 msgr2=0x55efd40d8b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:08:31.310 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:31.304+0000 7f224484e740 1 -- 172.21.15.12:0/2900790975 shutdown_connections 2024-09-11T00:08:31.311 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:31.304+0000 7f224484e740 1 -- 172.21.15.12:0/2900790975 wait complete. 2024-09-11T00:08:31.311 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:31.304+0000 7f224484e740 1 -- start start 2024-09-11T00:08:31.311 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:31.304+0000 7f224484e740 1 Processor -- start 2024-09-11T00:08:31.311 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:31.304+0000 7f224484e740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55efd40da970 0x55efd40eb990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:08:31.311 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:31.305+0000 7f224484e740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55efd40dd2a0 0x55efd40ea1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:08:31.311 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:31.305+0000 7f224484e740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55efd40dfce0 0x55efd40ea700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:08:31.311 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:31.305+0000 7f224484e740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f223c1aebd0 con 0x55efd40dd2a0 2024-09-11T00:08:31.311 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:31.305+0000 7f224484e740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f223c1ae500 con 0x55efd40da970 2024-09-11T00:08:31.311 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:31.305+0000 7f224484e740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f223c1ab740 con 0x55efd40dfce0 2024-09-11T00:08:31.311 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:31.305+0000 7f224484e740 10 client.0 _ll_register_callbacks cb 0x7f2228005b70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:08:31.311 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:31.305+0000 7f223aa00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55efd40da970 0x55efd40eb990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:08:31.311 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:31.305+0000 7f223aa00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55efd40da970 0x55efd40eb990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.12:33420/0 (socket says 172.21.15.12:33420) 2024-09-11T00:08:31.311 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:31.305+0000 7f223aa00640 1 -- 172.21.15.12:0/4286777163 learned_addr learned my addr 172.21.15.12:0/4286777163 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:08:31.312 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:31.305+0000 7f223a000640 1 --2- 172.21.15.12:0/4286777163 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55efd40dd2a0 0x55efd40ea1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:08:31.312 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:31.305+0000 7f223b400640 1 --2- 172.21.15.12:0/4286777163 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55efd40dfce0 0x55efd40ea700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:08:31.312 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:31.305+0000 7f223aa00640 1 -- 172.21.15.12:0/4286777163 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55efd40dfce0 msgr2=0x55efd40ea700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:08:31.312 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:31.305+0000 7f223aa00640 1 --2- 172.21.15.12:0/4286777163 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55efd40dfce0 0x55efd40ea700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:08:31.312 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:31.305+0000 7f223aa00640 1 -- 172.21.15.12:0/4286777163 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55efd40dd2a0 msgr2=0x55efd40ea1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:08:31.312 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:31.305+0000 7f223aa00640 1 --2- 172.21.15.12:0/4286777163 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55efd40dd2a0 0x55efd40ea1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:08:31.312 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:31.305+0000 7f223aa00640 1 -- 172.21.15.12:0/4286777163 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55efd40eacf0 con 0x55efd40da970 2024-09-11T00:08:31.312 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:31.305+0000 7f223aa00640 1 --2- 172.21.15.12:0/4286777163 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55efd40da970 0x55efd40eb990 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f222c01b040 tx=0x7f222c004270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7bad3ae1653630ca server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:08:31.312 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:31.305+0000 7f2224c00640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-09-11T00:08:31.312 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:31.306+0000 7f2224c00640 1 -- 172.21.15.12:0/4286777163 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f222c005240 con 0x55efd40da970 2024-09-11T00:08:31.312 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:31.306+0000 7f2224c00640 1 -- 172.21.15.12:0/4286777163 <== mon.2 v2:172.21.15.165:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 9208+0+0 (secure 0 0 0) 0x7f222c0053e0 con 0x55efd40da970 2024-09-11T00:08:31.312 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:31.306+0000 7f224484e740 1 -- 172.21.15.12:0/4286777163 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f22280cc8d0 con 0x55efd40da970 2024-09-11T00:08:31.312 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:31.306+0000 7f2224c00640 1 -- 172.21.15.12:0/4286777163 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f222c03d050 con 0x55efd40da970 2024-09-11T00:08:31.313 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:31.306+0000 7f224484e740 10 client.25667 fetch_fsmap learned FSMap version 29 2024-09-11T00:08:31.313 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:31.306+0000 7f224484e740 1 -- 172.21.15.12:0/4286777163 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55efd415d770 con 0x55efd40da970 2024-09-11T00:08:31.313 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:31.306+0000 7f2224c00640 1 -- 172.21.15.12:0/4286777163 <== mon.2 v2:172.21.15.165:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f22280cc8d0 con 0x55efd40da970 2024-09-11T00:08:31.313 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:31.306+0000 7f2224c00640 1 -- 172.21.15.12:0/4286777163 <== mon.2 v2:172.21.15.165:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7f222c003680 con 0x55efd40da970 2024-09-11T00:08:31.313 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:31.306+0000 7f224484e740 10 client.25667 fetch_fsmap finished waiting for FSMap version 29 2024-09-11T00:08:31.313 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:31.306+0000 7f224484e740 10 client.25667 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T00:08:31.313 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:31.306+0000 7f224484e740 20 client.25667 populate_metadata read hostname 'smithi012' 2024-09-11T00:08:31.313 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:31.306+0000 7f224484e740 1 --2- 172.21.15.12:0/4286777163 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55efd415df60 0x55efd4160380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:08:31.313 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:31.306+0000 7f224484e740 4 client.25667 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T00:08:31.313 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:31.306+0000 7f224484e740 1 -- 172.21.15.12:0/4286777163 --> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55efd40d5890 con 0x55efd415df60 2024-09-11T00:08:31.313 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:31.307+0000 7f223a000640 1 --2- 172.21.15.12:0/4286777163 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55efd415df60 0x55efd4160380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:08:31.313 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:31.307+0000 7f2224c00640 10 client.25667 ms_handle_connect on v2:172.21.15.165:6838/501312984 2024-09-11T00:08:31.314 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:31.307+0000 7f223a000640 1 --2- 172.21.15.12:0/4286777163 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55efd415df60 0x55efd4160380 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b9e97ee73971f14f server_cookie=e83151945b034c3 in_seq=0 out_seq=0 2024-09-11T00:08:31.573 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:31.571+0000 7f2224c00640 1 -- 172.21.15.12:0/4286777163 <== mds.0 v2:172.21.15.165:6838/501312984 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55efd40d5890 con 0x55efd415df60 2024-09-11T00:08:31.574 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:31.571+0000 7f2224c00640 10 client.25667 handle_command_reply: tid=0 2024-09-11T00:08:31.574 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:31.571+0000 7f224484e740 1 client.25667 shutdown 2024-09-11T00:08:31.574 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:31.571+0000 7f224484e740 1 -- 172.21.15.12:0/4286777163 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55efd40da970 msgr2=0x55efd40eb990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:08:31.574 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:31.571+0000 7f224484e740 1 --2- 172.21.15.12:0/4286777163 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55efd40da970 0x55efd40eb990 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f222c01b040 tx=0x7f222c004270 comp rx=0 tx=0).stop 2024-09-11T00:08:31.574 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:31.571+0000 7f224484e740 1 -- 172.21.15.12:0/4286777163 shutdown_connections 2024-09-11T00:08:31.574 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:31.571+0000 7f224484e740 1 --2- 172.21.15.12:0/4286777163 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55efd415df60 0x55efd4160380 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:08:31.574 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:31.571+0000 7f224484e740 1 --2- 172.21.15.12:0/4286777163 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55efd40dfce0 0x55efd40ea700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:08:31.574 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:31.571+0000 7f224484e740 1 --2- 172.21.15.12:0/4286777163 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55efd40dd2a0 0x55efd40ea1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:08:31.574 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:31.571+0000 7f224484e740 1 --2- 172.21.15.12:0/4286777163 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55efd40da970 0x55efd40eb990 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:08:31.574 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:31.571+0000 7f224484e740 1 -- 172.21.15.12:0/4286777163 >> 172.21.15.12:0/4286777163 conn(0x55efd40d5ac0 msgr2=0x55efd40d9a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:08:31.574 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:31.571+0000 7f2224c00640 0 client.25667 ms_handle_reset on v2:172.21.15.165:6838/501312984 2024-09-11T00:08:31.574 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:31.571+0000 7f224484e740 1 -- 172.21.15.12:0/4286777163 shutdown_connections 2024-09-11T00:08:31.575 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:31.572+0000 7f224484e740 1 -- 172.21.15.12:0/4286777163 wait complete. 2024-09-11T00:08:31.575 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:31.572+0000 7f224484e740 20 client.25667 trim_cache size 0 max 16384 2024-09-11T00:08:31.575 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:{ 2024-09-11T00:08:31.575 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "message": "", 2024-09-11T00:08:31.575 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "return_code": 0 2024-09-11T00:08:31.575 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:} 2024-09-11T00:08:31.576 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:31.573+0000 7f223be00640 1 -- 172.21.15.12:0/3306246232 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f22200b5360 msgr2=0x7f22200b7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:08:31.576 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:31.573+0000 7f223be00640 1 --2- 172.21.15.12:0/3306246232 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f22200b5360 0x7f22200b7820 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f2234009fd0 tx=0x7f2234009500 comp rx=0 tx=0).stop 2024-09-11T00:08:31.576 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:31.573+0000 7f223be00640 1 -- 172.21.15.12:0/3306246232 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f223c105b20 msgr2=0x7f223c19b4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:08:31.576 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:31.573+0000 7f223be00640 1 --2- 172.21.15.12:0/3306246232 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f223c105b20 0x7f223c19b4c0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f2228031490 tx=0x7f2228003c10 comp rx=0 tx=0).stop 2024-09-11T00:08:31.576 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:31.573+0000 7f223be00640 1 -- 172.21.15.12:0/3306246232 shutdown_connections 2024-09-11T00:08:31.577 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:31.573+0000 7f223be00640 1 --2- 172.21.15.12:0/3306246232 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f22200b5360 0x7f22200b7820 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:08:31.577 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:31.573+0000 7f223be00640 1 --2- 172.21.15.12:0/3306246232 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f223c1064e0 0x7f223c19c080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:08:31.580 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:31.573+0000 7f223be00640 1 --2- 172.21.15.12:0/3306246232 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f223c105b20 0x7f223c19b4c0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:08:31.580 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:31.573+0000 7f223be00640 1 --2- 172.21.15.12:0/3306246232 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f223c1049d0 0x7f223c19ae70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:08:31.581 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:31.573+0000 7f223be00640 1 -- 172.21.15.12:0/3306246232 >> 172.21.15.12:0/3306246232 conn(0x7f223c100270 msgr2=0x7f223c106e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:08:31.581 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:31.573+0000 7f223be00640 1 -- 172.21.15.12:0/3306246232 shutdown_connections 2024-09-11T00:08:31.581 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:31.573+0000 7f223be00640 1 -- 172.21.15.12:0/3306246232 wait complete. 2024-09-11T00:08:31.590 INFO:tasks.workunit.client.0.smithi012.stdout: 1 79030 15.80 MB/sec warmup 28 sec latency 148.536 ms 2024-09-11T00:08:32.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:08:31 smithi165 ceph-mon[35356]: pgmap v441: 129 pgs: 129 active+clean; 23 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 624 KiB/s rd, 1.6 MiB/s wr, 405 op/s 2024-09-11T00:08:32.063 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:08:31 smithi012 ceph-mon[30333]: pgmap v441: 129 pgs: 129 active+clean; 23 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 624 KiB/s rd, 1.6 MiB/s wr, 405 op/s 2024-09-11T00:08:32.080 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:08:31 smithi178 ceph-mon[35919]: pgmap v441: 129 pgs: 129 active+clean; 23 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 624 KiB/s rd, 1.6 MiB/s wr, 405 op/s 2024-09-11T00:08:32.590 INFO:tasks.workunit.client.0.smithi012.stdout: 1 81971 15.89 MB/sec warmup 29 sec latency 14.053 ms 2024-09-11T00:08:33.590 INFO:tasks.workunit.client.0.smithi012.stdout: 1 84533 15.76 MB/sec warmup 30 sec latency 18.995 ms 2024-09-11T00:08:34.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:08:33 smithi165 ceph-mon[35356]: pgmap v442: 129 pgs: 129 active+clean; 28 MiB data, 508 MiB used, 1.0 TiB / 1.0 TiB avail; 966 KiB/s rd, 1.8 MiB/s wr, 326 op/s 2024-09-11T00:08:34.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:08:33 smithi165 ceph-mon[35356]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T00:08:34.062 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:08:33 smithi012 ceph-mon[30333]: pgmap v442: 129 pgs: 129 active+clean; 28 MiB data, 508 MiB used, 1.0 TiB / 1.0 TiB avail; 966 KiB/s rd, 1.8 MiB/s wr, 326 op/s 2024-09-11T00:08:34.063 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:08:33 smithi012 ceph-mon[30333]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T00:08:34.080 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:08:33 smithi178 ceph-mon[35919]: pgmap v442: 129 pgs: 129 active+clean; 28 MiB data, 508 MiB used, 1.0 TiB / 1.0 TiB avail; 966 KiB/s rd, 1.8 MiB/s wr, 326 op/s 2024-09-11T00:08:34.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:08:33 smithi178 ceph-mon[35919]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T00:08:34.590 INFO:tasks.workunit.client.0.smithi012.stdout: 1 85161 15.43 MB/sec warmup 31 sec latency 803.946 ms 2024-09-11T00:08:35.590 INFO:tasks.workunit.client.0.smithi012.stdout: 1 85267 14.95 MB/sec warmup 32 sec latency 999.201 ms 2024-09-11T00:08:35.812 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:08:35 smithi012 ceph-mon[30333]: pgmap v443: 129 pgs: 129 active+clean; 28 MiB data, 500 MiB used, 1.0 TiB / 1.0 TiB avail; 796 KiB/s rd, 1.5 MiB/s wr, 237 op/s 2024-09-11T00:08:36.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:08:35 smithi165 ceph-mon[35356]: pgmap v443: 129 pgs: 129 active+clean; 28 MiB data, 500 MiB used, 1.0 TiB / 1.0 TiB avail; 796 KiB/s rd, 1.5 MiB/s wr, 237 op/s 2024-09-11T00:08:36.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:08:35 smithi178 ceph-mon[35919]: pgmap v443: 129 pgs: 129 active+clean; 28 MiB data, 500 MiB used, 1.0 TiB / 1.0 TiB avail; 796 KiB/s rd, 1.5 MiB/s wr, 237 op/s 2024-09-11T00:08:36.590 INFO:tasks.workunit.client.0.smithi012.stdout: 1 85318 14.50 MB/sec warmup 33 sec latency 962.675 ms 2024-09-11T00:08:37.590 INFO:tasks.workunit.client.0.smithi012.stdout: 1 85318 14.08 MB/sec warmup 34 sec latency 1962.828 ms 2024-09-11T00:08:38.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:08:37 smithi165 ceph-mon[35356]: pgmap v444: 129 pgs: 129 active+clean; 37 MiB data, 487 MiB used, 1.0 TiB / 1.0 TiB avail; 813 KiB/s rd, 3.0 MiB/s wr, 422 op/s 2024-09-11T00:08:38.063 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:08:37 smithi012 ceph-mon[30333]: pgmap v444: 129 pgs: 129 active+clean; 37 MiB data, 487 MiB used, 1.0 TiB / 1.0 TiB avail; 813 KiB/s rd, 3.0 MiB/s wr, 422 op/s 2024-09-11T00:08:38.080 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:08:37 smithi178 ceph-mon[35919]: pgmap v444: 129 pgs: 129 active+clean; 37 MiB data, 487 MiB used, 1.0 TiB / 1.0 TiB avail; 813 KiB/s rd, 3.0 MiB/s wr, 422 op/s 2024-09-11T00:08:38.590 INFO:tasks.workunit.client.0.smithi012.stdout: 1 85318 13.67 MB/sec warmup 35 sec latency 2962.957 ms 2024-09-11T00:08:39.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:08:38 smithi165 ceph-mon[35356]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T00:08:39.063 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:08:38 smithi012 ceph-mon[30333]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T00:08:39.080 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:08:38 smithi178 ceph-mon[35919]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T00:08:39.591 INFO:tasks.workunit.client.0.smithi012.stdout: 1 85318 13.29 MB/sec warmup 36 sec latency 3963.034 ms 2024-09-11T00:08:39.751 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:08:39 smithi012 ceph-mon[30333]: pgmap v445: 129 pgs: 129 active+clean; 37 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 544 KiB/s rd, 2.7 MiB/s wr, 361 op/s 2024-09-11T00:08:40.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:08:39 smithi165 ceph-mon[35356]: pgmap v445: 129 pgs: 129 active+clean; 37 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 544 KiB/s rd, 2.7 MiB/s wr, 361 op/s 2024-09-11T00:08:40.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:08:39 smithi178 ceph-mon[35919]: pgmap v445: 129 pgs: 129 active+clean; 37 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 544 KiB/s rd, 2.7 MiB/s wr, 361 op/s 2024-09-11T00:08:40.590 INFO:tasks.workunit.client.0.smithi012.stdout: 1 85318 12.94 MB/sec warmup 37 sec latency 4963.095 ms 2024-09-11T00:08:41.591 INFO:tasks.workunit.client.0.smithi012.stdout: 1 85318 12.60 MB/sec warmup 38 sec latency 5963.157 ms 2024-09-11T00:08:41.958 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:08:41 smithi012 ceph-mon[30333]: pgmap v446: 129 pgs: 129 active+clean; 44 MiB data, 488 MiB used, 1.0 TiB / 1.0 TiB avail; 551 KiB/s rd, 3.3 MiB/s wr, 375 op/s 2024-09-11T00:08:42.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:08:41 smithi165 ceph-mon[35356]: pgmap v446: 129 pgs: 129 active+clean; 44 MiB data, 488 MiB used, 1.0 TiB / 1.0 TiB avail; 551 KiB/s rd, 3.3 MiB/s wr, 375 op/s 2024-09-11T00:08:42.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:08:41 smithi178 ceph-mon[35919]: pgmap v446: 129 pgs: 129 active+clean; 44 MiB data, 488 MiB used, 1.0 TiB / 1.0 TiB avail; 551 KiB/s rd, 3.3 MiB/s wr, 375 op/s 2024-09-11T00:08:42.591 INFO:tasks.workunit.client.0.smithi012.stdout: 1 85337 12.27 MB/sec warmup 39 sec latency 6383.485 ms 2024-09-11T00:08:43.591 INFO:tasks.workunit.client.0.smithi012.stdout: 1 85337 11.97 MB/sec warmup 40 sec latency 1563.298 ms 2024-09-11T00:08:44.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:08:43 smithi165 ceph-mon[35356]: pgmap v447: 129 pgs: 129 active+clean; 45 MiB data, 513 MiB used, 1.0 TiB / 1.0 TiB avail; 593 KiB/s rd, 3.4 MiB/s wr, 373 op/s 2024-09-11T00:08:44.063 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:08:43 smithi012 ceph-mon[30333]: pgmap v447: 129 pgs: 129 active+clean; 45 MiB data, 513 MiB used, 1.0 TiB / 1.0 TiB avail; 593 KiB/s rd, 3.4 MiB/s wr, 373 op/s 2024-09-11T00:08:44.080 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:08:43 smithi178 ceph-mon[35919]: pgmap v447: 129 pgs: 129 active+clean; 45 MiB data, 513 MiB used, 1.0 TiB / 1.0 TiB avail; 593 KiB/s rd, 3.4 MiB/s wr, 373 op/s 2024-09-11T00:08:44.591 INFO:tasks.workunit.client.0.smithi012.stdout: 1 85337 11.67 MB/sec warmup 41 sec latency 2563.387 ms 2024-09-11T00:08:44.759 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:44.756+0000 7f430b800640 1 -- 172.21.15.12:0/1864348247 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4304105cf0 msgr2=0x7f4304106170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:08:44.759 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:44.756+0000 7f430b800640 1 --2- 172.21.15.12:0/1864348247 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4304105cf0 0x7f4304106170 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f42ec009930 tx=0x7f42ec02a420 comp rx=0 tx=0).stop 2024-09-11T00:08:44.759 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:44.757+0000 7f430b800640 1 -- 172.21.15.12:0/1864348247 shutdown_connections 2024-09-11T00:08:44.759 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:44.757+0000 7f430b800640 1 --2- 172.21.15.12:0/1864348247 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f43041066b0 0x7f430410d030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:08:44.759 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:44.757+0000 7f430b800640 1 --2- 172.21.15.12:0/1864348247 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4304105cf0 0x7f4304106170 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:08:44.759 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:44.757+0000 7f430b800640 1 --2- 172.21.15.12:0/1864348247 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4304104ba0 0x7f4304104fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:08:44.759 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:44.757+0000 7f430b800640 1 -- 172.21.15.12:0/1864348247 >> 172.21.15.12:0/1864348247 conn(0x7f4304100460 msgr2=0x7f4304102880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:08:44.760 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:44.757+0000 7f430b800640 1 -- 172.21.15.12:0/1864348247 shutdown_connections 2024-09-11T00:08:44.760 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:44.757+0000 7f430b800640 1 -- 172.21.15.12:0/1864348247 wait complete. 2024-09-11T00:08:44.762 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:44.758+0000 7f430b800640 1 Processor -- start 2024-09-11T00:08:44.762 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:44.758+0000 7f430b800640 1 -- start start 2024-09-11T00:08:44.762 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:44.758+0000 7f430b800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4304104ba0 0x7f430419aff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:08:44.762 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:44.758+0000 7f430b800640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4304105cf0 0x7f430419b640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:08:44.762 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:44.758+0000 7f430b800640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f43041066b0 0x7f430419c270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:08:44.762 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:44.758+0000 7f430b800640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f43041108c0 con 0x7f43041066b0 2024-09-11T00:08:44.762 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:44.758+0000 7f430b800640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f4304110740 con 0x7f4304104ba0 2024-09-11T00:08:44.762 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:44.758+0000 7f430b800640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f4304110a40 con 0x7f4304105cf0 2024-09-11T00:08:44.762 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:44.758+0000 7f4309800640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f43041066b0 0x7f430419c270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:08:44.762 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:44.759+0000 7f4309800640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f43041066b0 0x7f430419c270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:34026/0 (socket says 172.21.15.12:34026) 2024-09-11T00:08:44.762 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:44.759+0000 7f4309800640 1 -- 172.21.15.12:0/997665760 learned_addr learned my addr 172.21.15.12:0/997665760 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:08:44.762 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:44.759+0000 7f42fbe00640 1 --2- 172.21.15.12:0/997665760 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4304105cf0 0x7f430419b640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:08:44.763 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:44.759+0000 7f4308e00640 1 --2- 172.21.15.12:0/997665760 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4304104ba0 0x7f430419aff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:08:44.763 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:44.759+0000 7f42fbe00640 1 -- 172.21.15.12:0/997665760 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4304104ba0 msgr2=0x7f430419aff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:08:44.763 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:44.759+0000 7f42fbe00640 1 --2- 172.21.15.12:0/997665760 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4304104ba0 0x7f430419aff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:08:44.763 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:44.759+0000 7f42fbe00640 1 -- 172.21.15.12:0/997665760 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f43041066b0 msgr2=0x7f430419c270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:08:44.763 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:44.759+0000 7f42fbe00640 1 --2- 172.21.15.12:0/997665760 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f43041066b0 0x7f430419c270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:08:44.763 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:44.759+0000 7f42fbe00640 1 -- 172.21.15.12:0/997665760 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f430419cb30 con 0x7f4304105cf0 2024-09-11T00:08:44.763 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:44.759+0000 7f4309800640 1 --2- 172.21.15.12:0/997665760 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f43041066b0 0x7f430419c270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:08:44.763 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:44.759+0000 7f42fbe00640 1 --2- 172.21.15.12:0/997665760 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4304105cf0 0x7f430419b640 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f42ec02a940 tx=0x7f42ec003c80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:08:44.763 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:44.760+0000 7f4302000640 1 -- 172.21.15.12:0/997665760 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42ec03f070 con 0x7f4304105cf0 2024-09-11T00:08:44.763 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:44.760+0000 7f4302000640 1 -- 172.21.15.12:0/997665760 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f42ec003e70 con 0x7f4304105cf0 2024-09-11T00:08:44.763 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:44.760+0000 7f4302000640 1 -- 172.21.15.12:0/997665760 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42ec035030 con 0x7f4304105cf0 2024-09-11T00:08:44.763 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:44.760+0000 7f430b800640 1 -- 172.21.15.12:0/997665760 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f430419cdc0 con 0x7f4304105cf0 2024-09-11T00:08:44.763 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:44.760+0000 7f430b800640 1 -- 172.21.15.12:0/997665760 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43041a3390 con 0x7f4304105cf0 2024-09-11T00:08:44.764 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:44.763+0000 7f430e7e5740 1 Processor -- start 2024-09-11T00:08:44.764 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:44.763+0000 7f430e7e5740 1 -- start start 2024-09-11T00:08:44.766 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:44.763+0000 7f430e7e5740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e232ab3790 0x55e232ab5b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:08:44.766 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:44.763+0000 7f430e7e5740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e232ab60c0 0x55e232ab84b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:08:44.766 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:44.763+0000 7f430e7e5740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e232ab8b00 0x55e232abafc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:08:44.766 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:44.763+0000 7f430e7e5740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55e232a266b0 con 0x55e232ab60c0 2024-09-11T00:08:44.766 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:44.763+0000 7f430e7e5740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55e232a15ae0 con 0x55e232ab8b00 2024-09-11T00:08:44.766 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:44.763+0000 7f430e7e5740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55e232a198f0 con 0x55e232ab3790 2024-09-11T00:08:44.766 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:44.763+0000 7f4302000640 1 -- 172.21.15.12:0/997665760 <== mon.1 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7f42ec03b460 con 0x7f4304105cf0 2024-09-11T00:08:44.766 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:44.763+0000 7f4309800640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e232ab3790 0x55e232ab5b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:08:44.766 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:44.763+0000 7f4308e00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e232ab60c0 0x55e232ab84b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:08:44.767 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:44.763+0000 7f4309800640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e232ab3790 0x55e232ab5b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.12:56418/0 (socket says 172.21.15.12:56418) 2024-09-11T00:08:44.767 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:44.763+0000 7f4309800640 1 -- 172.21.15.12:0/3812981030 learned_addr learned my addr 172.21.15.12:0/3812981030 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:08:44.767 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:44.764+0000 7f42fbe00640 1 --2- 172.21.15.12:0/3812981030 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e232ab8b00 0x55e232abafc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:08:44.767 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:44.764+0000 7f4309800640 1 -- 172.21.15.12:0/3812981030 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e232ab8b00 msgr2=0x55e232abafc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:08:44.767 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:44.764+0000 7f4309800640 1 --2- 172.21.15.12:0/3812981030 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e232ab8b00 0x55e232abafc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:08:44.767 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:44.764+0000 7f4309800640 1 -- 172.21.15.12:0/3812981030 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e232ab60c0 msgr2=0x55e232ab84b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:08:44.767 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:44.764+0000 7f4309800640 1 --2- 172.21.15.12:0/3812981030 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e232ab60c0 0x55e232ab84b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:08:44.767 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:44.764+0000 7f4309800640 1 -- 172.21.15.12:0/3812981030 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e2329e2780 con 0x55e232ab3790 2024-09-11T00:08:44.767 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:44.764+0000 7f42fbe00640 1 --2- 172.21.15.12:0/3812981030 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e232ab8b00 0x55e232abafc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T00:08:44.767 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:44.764+0000 7f4309800640 1 --2- 172.21.15.12:0/3812981030 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e232ab3790 0x55e232ab5b80 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f42fc0117d0 tx=0x7f42fc011c90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e4c836a995dc464e server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:08:44.770 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:44.764+0000 7f42fa000640 1 -- 172.21.15.12:0/3812981030 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42fc00fb40 con 0x55e232ab3790 2024-09-11T00:08:44.770 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:44.764+0000 7f42fa000640 1 -- 172.21.15.12:0/3812981030 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f42fc00fce0 con 0x55e232ab3790 2024-09-11T00:08:44.770 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:44.764+0000 7f42fa000640 1 -- 172.21.15.12:0/3812981030 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42fc01a650 con 0x55e232ab3790 2024-09-11T00:08:44.770 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:44.765+0000 7f430e7e5740 1 -- 172.21.15.12:0/3812981030 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e232ab3790 msgr2=0x55e232ab5b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:08:44.770 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:44.765+0000 7f430e7e5740 1 --2- 172.21.15.12:0/3812981030 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e232ab3790 0x55e232ab5b80 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f42fc0117d0 tx=0x7f42fc011c90 comp rx=0 tx=0).stop 2024-09-11T00:08:44.770 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:44.765+0000 7f4302000640 1 --2- 172.21.15.12:0/997665760 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f42e80b5360 0x7f42e80b7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:08:44.771 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:44.765+0000 7f4302000640 1 -- 172.21.15.12:0/997665760 <== mon.1 v2:172.21.15.178:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9208+0+0 (secure 0 0 0) 0x7f42ec106a50 con 0x7f4304105cf0 2024-09-11T00:08:44.771 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:44.765+0000 7f430e7e5740 1 -- 172.21.15.12:0/3812981030 shutdown_connections 2024-09-11T00:08:44.771 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:44.765+0000 7f430e7e5740 1 --2- 172.21.15.12:0/3812981030 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e232ab8b00 0x55e232abafc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:08:44.771 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:44.765+0000 7f430e7e5740 1 --2- 172.21.15.12:0/3812981030 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e232ab60c0 0x55e232ab84b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:08:44.771 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:44.765+0000 7f430e7e5740 1 --2- 172.21.15.12:0/3812981030 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e232ab3790 0x55e232ab5b80 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:08:44.771 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:44.765+0000 7f430e7e5740 1 -- 172.21.15.12:0/3812981030 >> 172.21.15.12:0/3812981030 conn(0x55e232aae8e0 msgr2=0x55e232aaf670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:08:44.771 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:44.766+0000 7f4309800640 1 --2- 172.21.15.12:0/997665760 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f42e80b5360 0x7f42e80b7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:08:44.771 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:44.766+0000 7f430e7e5740 1 -- 172.21.15.12:0/3812981030 shutdown_connections 2024-09-11T00:08:44.772 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:44.766+0000 7f430e7e5740 1 -- 172.21.15.12:0/3812981030 wait complete. 2024-09-11T00:08:44.772 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:44.766+0000 7f430e7e5740 1 -- start start 2024-09-11T00:08:44.772 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:44.766+0000 7f4309800640 1 --2- 172.21.15.12:0/997665760 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f42e80b5360 0x7f42e80b7820 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f42fc012bc0 tx=0x7f42fc005bd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:08:44.772 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:44.767+0000 7f430e7e5740 1 Processor -- start 2024-09-11T00:08:44.772 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:44.767+0000 7f430e7e5740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e232ab3790 0x55e232abcea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:08:44.772 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:44.767+0000 7f430e7e5740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e232ab60c0 0x55e232abd540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:08:44.772 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:44.767+0000 7f430e7e5740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e232ab8b00 0x55e232abe110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:08:44.772 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:44.767+0000 7f430e7e5740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55e23294dab0 con 0x55e232ab8b00 2024-09-11T00:08:44.772 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:44.767+0000 7f430e7e5740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55e2329bc570 con 0x55e232ab3790 2024-09-11T00:08:44.772 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:44.767+0000 7f430e7e5740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55e232abbae0 con 0x55e232ab60c0 2024-09-11T00:08:44.773 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:44.767+0000 7f430e7e5740 10 client.0 _ll_register_callbacks cb 0x55e2329cb100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:08:44.773 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:44.767+0000 7f42fbe00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e232ab60c0 0x55e232abd540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:08:44.773 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:44.767+0000 7f42fbe00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e232ab60c0 0x55e232abd540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.12:56434/0 (socket says 172.21.15.12:56434) 2024-09-11T00:08:44.773 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:44.767+0000 7f4309800640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e232ab8b00 0x55e232abe110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:08:44.773 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:44.767+0000 7f42fbe00640 1 -- 172.21.15.12:0/3451037367 learned_addr learned my addr 172.21.15.12:0/3451037367 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:08:44.773 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:44.768+0000 7f42fbe00640 1 -- 172.21.15.12:0/3451037367 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e232ab3790 msgr2=0x55e232abcea0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-11T00:08:44.773 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:44.768+0000 7f42fbe00640 1 --2- 172.21.15.12:0/3451037367 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e232ab3790 0x55e232abcea0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:08:44.773 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:44.768+0000 7f42fbe00640 1 -- 172.21.15.12:0/3451037367 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e232ab8b00 msgr2=0x55e232abe110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:08:44.774 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:44.768+0000 7f42fbe00640 1 --2- 172.21.15.12:0/3451037367 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e232ab8b00 0x55e232abe110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:08:44.774 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:44.768+0000 7f42fbe00640 1 -- 172.21.15.12:0/3451037367 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e232777790 con 0x55e232ab60c0 2024-09-11T00:08:44.774 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:44.768+0000 7f42fbe00640 1 --2- 172.21.15.12:0/3451037367 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e232ab60c0 0x55e232abd540 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f42ec106cf0 tx=0x7f42ec0c8d40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=78f79a8e2bab5d05 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:08:44.774 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:44.768+0000 7f42f2a00640 10 client.0 ms_handle_connect on v2:172.21.15.178:3300/0 2024-09-11T00:08:44.774 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:44.768+0000 7f42f2a00640 1 -- 172.21.15.12:0/3451037367 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42ec049070 con 0x55e232ab60c0 2024-09-11T00:08:44.774 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:44.768+0000 7f42f2a00640 1 -- 172.21.15.12:0/3451037367 <== mon.1 v2:172.21.15.178:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 9208+0+0 (secure 0 0 0) 0x7f42ec0d5ee0 con 0x55e232ab60c0 2024-09-11T00:08:44.774 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:44.768+0000 7f430e7e5740 1 -- 172.21.15.12:0/3451037367 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f42fc00fce0 con 0x55e232ab60c0 2024-09-11T00:08:44.774 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:44.768+0000 7f42f2a00640 1 -- 172.21.15.12:0/3451037367 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f42ec0cc8f0 con 0x55e232ab60c0 2024-09-11T00:08:44.774 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:44.768+0000 7f430e7e5740 10 client.25660 fetch_fsmap learned FSMap version 31 2024-09-11T00:08:44.775 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:44.768+0000 7f430e7e5740 1 -- 172.21.15.12:0/3451037367 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55e2327f0620 con 0x55e232ab60c0 2024-09-11T00:08:44.775 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:44.768+0000 7f42f2a00640 1 -- 172.21.15.12:0/3451037367 <== mon.1 v2:172.21.15.178:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f42fc00fce0 con 0x55e232ab60c0 2024-09-11T00:08:44.776 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:44.769+0000 7f42f2a00640 1 -- 172.21.15.12:0/3451037367 <== mon.1 v2:172.21.15.178:3300/0 5 ==== fsmap(e 31) ==== 8386+0+0 (secure 0 0 0) 0x7f42ec0c8730 con 0x55e232ab60c0 2024-09-11T00:08:44.777 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:44.769+0000 7f430e7e5740 10 client.25660 fetch_fsmap finished waiting for FSMap version 31 2024-09-11T00:08:44.777 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:44.769+0000 7f430e7e5740 10 client.25660 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T00:08:44.777 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:44.769+0000 7f430e7e5740 20 client.25660 populate_metadata read hostname 'smithi012' 2024-09-11T00:08:44.777 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:44.769+0000 7f430e7e5740 1 --2- 172.21.15.12:0/3451037367 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55e232ac8690 0x55e232acab50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:08:44.777 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:44.769+0000 7f430e7e5740 4 client.25660 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T00:08:44.777 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:44.769+0000 7f430e7e5740 1 -- 172.21.15.12:0/3451037367 --> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e232a28600 con 0x55e232ac8690 2024-09-11T00:08:44.777 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:44.769+0000 7f4308e00640 1 --2- 172.21.15.12:0/3451037367 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55e232ac8690 0x55e232acab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:08:44.777 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:44.770+0000 7f4308e00640 1 --2- 172.21.15.12:0/3451037367 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55e232ac8690 0x55e232acab50 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ecbf1432b6957313 server_cookie=4f7ba76f406d8b36 in_seq=0 out_seq=0 2024-09-11T00:08:44.777 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:44.770+0000 7f42f2a00640 10 client.25660 ms_handle_connect on v2:172.21.15.165:6838/501312984 2024-09-11T00:08:44.777 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:44.770+0000 7f42f2a00640 1 -- 172.21.15.12:0/3451037367 <== mds.0 v2:172.21.15.165:6838/501312984 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55e232a28600 con 0x55e232ac8690 2024-09-11T00:08:44.778 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:44.770+0000 7f42f2a00640 10 client.25660 handle_command_reply: tid=0 2024-09-11T00:08:44.778 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:44.771+0000 7f430e7e5740 1 client.25660 shutdown 2024-09-11T00:08:44.778 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:44.771+0000 7f430e7e5740 1 -- 172.21.15.12:0/3451037367 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e232ab60c0 msgr2=0x55e232abd540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:08:44.778 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:44.771+0000 7f430e7e5740 1 --2- 172.21.15.12:0/3451037367 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e232ab60c0 0x55e232abd540 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f42ec106cf0 tx=0x7f42ec0c8d40 comp rx=0 tx=0).stop 2024-09-11T00:08:44.778 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:44.771+0000 7f430e7e5740 1 -- 172.21.15.12:0/3451037367 shutdown_connections 2024-09-11T00:08:44.778 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:44.771+0000 7f430e7e5740 1 --2- 172.21.15.12:0/3451037367 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55e232ac8690 0x55e232acab50 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:08:44.778 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:44.771+0000 7f430e7e5740 1 --2- 172.21.15.12:0/3451037367 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e232ab8b00 0x55e232abe110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:08:44.778 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:44.771+0000 7f430e7e5740 1 --2- 172.21.15.12:0/3451037367 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e232ab60c0 0x55e232abd540 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:08:44.778 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:44.771+0000 7f430e7e5740 1 --2- 172.21.15.12:0/3451037367 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e232ab3790 0x55e232abcea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:08:44.778 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:44.771+0000 7f430e7e5740 1 -- 172.21.15.12:0/3451037367 >> 172.21.15.12:0/3451037367 conn(0x55e232aae8e0 msgr2=0x55e232aada60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:08:44.778 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:44.771+0000 7f42f2a00640 0 client.25660 ms_handle_reset on v2:172.21.15.165:6838/501312984 2024-09-11T00:08:44.779 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:44.771+0000 7f430e7e5740 1 -- 172.21.15.12:0/3451037367 shutdown_connections 2024-09-11T00:08:44.779 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:44.771+0000 7f430e7e5740 1 -- 172.21.15.12:0/3451037367 wait complete. 2024-09-11T00:08:44.779 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:44.771+0000 7f430e7e5740 20 client.25660 trim_cache size 0 max 16384 2024-09-11T00:08:44.788 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:44.786+0000 7f430e7e5740 1 Processor -- start 2024-09-11T00:08:44.788 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:44.786+0000 7f430e7e5740 1 -- start start 2024-09-11T00:08:44.789 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:44.787+0000 7f430e7e5740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e232ab3790 0x55e232abbee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:08:44.789 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:44.787+0000 7f430e7e5740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e232ab60c0 0x55e232abc420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:08:44.789 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:44.787+0000 7f430e7e5740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e232ab8b00 0x55e232ab0710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:08:44.789 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:44.787+0000 7f430e7e5740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f43041ad930 con 0x55e232ab60c0 2024-09-11T00:08:44.789 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:44.787+0000 7f430e7e5740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f43041af530 con 0x55e232ab8b00 2024-09-11T00:08:44.789 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:44.787+0000 7f430e7e5740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f43041ae000 con 0x55e232ab3790 2024-09-11T00:08:44.792 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:44.787+0000 7f42fbe00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e232ab60c0 0x55e232abc420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:08:44.793 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:44.787+0000 7f42fbe00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e232ab60c0 0x55e232abc420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:34044/0 (socket says 172.21.15.12:34044) 2024-09-11T00:08:44.793 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:44.787+0000 7f42fbe00640 1 -- 172.21.15.12:0/2812982122 learned_addr learned my addr 172.21.15.12:0/2812982122 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:08:44.793 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:44.787+0000 7f4308e00640 1 --2- 172.21.15.12:0/2812982122 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e232ab3790 0x55e232abbee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:08:44.793 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:44.787+0000 7f4309800640 1 --2- 172.21.15.12:0/2812982122 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e232ab8b00 0x55e232ab0710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:08:44.793 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:44.787+0000 7f42fbe00640 1 -- 172.21.15.12:0/2812982122 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e232ab3790 msgr2=0x55e232abbee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:08:44.793 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:44.787+0000 7f42fbe00640 1 --2- 172.21.15.12:0/2812982122 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e232ab3790 0x55e232abbee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:08:44.793 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:44.787+0000 7f42fbe00640 1 -- 172.21.15.12:0/2812982122 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e232ab8b00 msgr2=0x55e232ab0710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:08:44.793 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:44.787+0000 7f42fbe00640 1 --2- 172.21.15.12:0/2812982122 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e232ab8b00 0x55e232ab0710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:08:44.793 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:44.787+0000 7f42fbe00640 1 -- 172.21.15.12:0/2812982122 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e232acb1d0 con 0x55e232ab60c0 2024-09-11T00:08:44.793 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:44.787+0000 7f4309800640 1 --2- 172.21.15.12:0/2812982122 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e232ab8b00 0x55e232ab0710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T00:08:44.793 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:44.787+0000 7f42fbe00640 1 --2- 172.21.15.12:0/2812982122 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e232ab60c0 0x55e232abc420 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f42ec0ca050 tx=0x7f42ec0c8db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30f2df746e8deaaf server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:08:44.794 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:44.788+0000 7f42f9600640 1 -- 172.21.15.12:0/2812982122 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42ec049070 con 0x55e232ab60c0 2024-09-11T00:08:44.794 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:44.788+0000 7f42f9600640 1 -- 172.21.15.12:0/2812982122 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f42ec0f2050 con 0x55e232ab60c0 2024-09-11T00:08:44.794 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:44.788+0000 7f42f9600640 1 -- 172.21.15.12:0/2812982122 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42ec0c8730 con 0x55e232ab60c0 2024-09-11T00:08:44.794 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:44.788+0000 7f430e7e5740 1 -- 172.21.15.12:0/2812982122 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e232ab60c0 msgr2=0x55e232abc420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:08:44.794 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:44.788+0000 7f430e7e5740 1 --2- 172.21.15.12:0/2812982122 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e232ab60c0 0x55e232abc420 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f42ec0ca050 tx=0x7f42ec0c8db0 comp rx=0 tx=0).stop 2024-09-11T00:08:44.794 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:44.788+0000 7f430e7e5740 1 -- 172.21.15.12:0/2812982122 shutdown_connections 2024-09-11T00:08:44.794 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:44.788+0000 7f430e7e5740 1 --2- 172.21.15.12:0/2812982122 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e232ab8b00 0x55e232ab0710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:08:44.794 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:44.788+0000 7f430e7e5740 1 --2- 172.21.15.12:0/2812982122 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e232ab60c0 0x55e232abc420 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:08:44.794 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:44.788+0000 7f430e7e5740 1 --2- 172.21.15.12:0/2812982122 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e232ab3790 0x55e232abbee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:08:44.794 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:44.788+0000 7f430e7e5740 1 -- 172.21.15.12:0/2812982122 >> 172.21.15.12:0/2812982122 conn(0x55e232aae8e0 msgr2=0x55e232aaf260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:08:44.794 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:44.789+0000 7f430e7e5740 1 -- 172.21.15.12:0/2812982122 shutdown_connections 2024-09-11T00:08:44.794 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:44.789+0000 7f430e7e5740 1 -- 172.21.15.12:0/2812982122 wait complete. 2024-09-11T00:08:44.795 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:44.789+0000 7f430e7e5740 1 -- start start 2024-09-11T00:08:44.795 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:44.789+0000 7f430e7e5740 1 Processor -- start 2024-09-11T00:08:44.795 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:44.789+0000 7f430e7e5740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e232ab3790 0x55e232ac4b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:08:44.795 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:44.790+0000 7f430e7e5740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e232ab60c0 0x55e232ac3800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:08:44.795 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:44.790+0000 7f430e7e5740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e232ab8b00 0x55e232ac3ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:08:44.795 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:44.790+0000 7f430e7e5740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f43041aeda0 con 0x55e232ab60c0 2024-09-11T00:08:44.795 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:44.790+0000 7f430e7e5740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f43041ae6d0 con 0x55e232ab3790 2024-09-11T00:08:44.795 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:44.790+0000 7f430e7e5740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f43041ab910 con 0x55e232ab8b00 2024-09-11T00:08:44.795 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:44.790+0000 7f430e7e5740 10 client.0 _ll_register_callbacks cb 0x7f42ec0c9430 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:08:44.795 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:44.790+0000 7f4308e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e232ab3790 0x55e232ac4b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:08:44.795 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:44.790+0000 7f42fbe00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e232ab60c0 0x55e232ac3800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:08:44.796 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:44.790+0000 7f4308e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e232ab3790 0x55e232ac4b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.12:33066/0 (socket says 172.21.15.12:33066) 2024-09-11T00:08:44.796 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:44.790+0000 7f4308e00640 1 -- 172.21.15.12:0/716059121 learned_addr learned my addr 172.21.15.12:0/716059121 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:08:44.796 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:44.790+0000 7f4309800640 1 --2- 172.21.15.12:0/716059121 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e232ab8b00 0x55e232ac3ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:08:44.796 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:44.790+0000 7f42fbe00640 1 -- 172.21.15.12:0/716059121 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e232ab8b00 msgr2=0x55e232ac3ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:08:44.796 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:44.790+0000 7f42fbe00640 1 --2- 172.21.15.12:0/716059121 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e232ab8b00 0x55e232ac3ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:08:44.796 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:44.790+0000 7f42fbe00640 1 -- 172.21.15.12:0/716059121 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e232ab3790 msgr2=0x55e232ac4b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:08:44.796 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:44.790+0000 7f42fbe00640 1 --2- 172.21.15.12:0/716059121 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e232ab3790 0x55e232ac4b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:08:44.796 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:44.790+0000 7f42fbe00640 1 -- 172.21.15.12:0/716059121 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e232ac2320 con 0x55e232ab60c0 2024-09-11T00:08:44.796 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:44.791+0000 7f42fbe00640 1 --2- 172.21.15.12:0/716059121 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e232ab60c0 0x55e232ac3800 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f42ec0c8b20 tx=0x7f42ec0b82e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=718162b704ebea92 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:08:44.796 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:44.791+0000 7f42f2a00640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-09-11T00:08:44.797 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:44.791+0000 7f42f2a00640 1 -- 172.21.15.12:0/716059121 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42ec0e6070 con 0x55e232ab60c0 2024-09-11T00:08:44.797 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:44.791+0000 7f42f2a00640 1 -- 172.21.15.12:0/716059121 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 9208+0+0 (secure 0 0 0) 0x7f42ec0cc040 con 0x55e232ab60c0 2024-09-11T00:08:44.797 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:44.791+0000 7f430e7e5740 1 -- 172.21.15.12:0/716059121 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f42ec0f2050 con 0x55e232ab60c0 2024-09-11T00:08:44.798 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:44.791+0000 7f42f2a00640 1 -- 172.21.15.12:0/716059121 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f42ec049070 con 0x55e232ab60c0 2024-09-11T00:08:44.798 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:44.791+0000 7f430e7e5740 10 client.15654 fetch_fsmap learned FSMap version 31 2024-09-11T00:08:44.798 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:44.791+0000 7f430e7e5740 1 -- 172.21.15.12:0/716059121 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55e232ac2320 con 0x55e232ab60c0 2024-09-11T00:08:44.798 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:44.791+0000 7f42f2a00640 1 -- 172.21.15.12:0/716059121 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f42ec0f2050 con 0x55e232ab60c0 2024-09-11T00:08:44.798 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:44.791+0000 7f42f2a00640 1 -- 172.21.15.12:0/716059121 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 31) ==== 8386+0+0 (secure 0 0 0) 0x7f42ec0d2a50 con 0x55e232ab60c0 2024-09-11T00:08:44.798 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:44.791+0000 7f430e7e5740 10 client.15654 fetch_fsmap finished waiting for FSMap version 31 2024-09-11T00:08:44.798 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:44.792+0000 7f430e7e5740 10 client.15654 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T00:08:44.798 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:44.792+0000 7f430e7e5740 20 client.15654 populate_metadata read hostname 'smithi012' 2024-09-11T00:08:44.799 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:44.792+0000 7f430e7e5740 1 --2- 172.21.15.12:0/716059121 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55e232b35e20 0x55e232b38240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:08:44.799 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:44.792+0000 7f430e7e5740 4 client.15654 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T00:08:44.799 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:44.792+0000 7f430e7e5740 1 -- 172.21.15.12:0/716059121 --> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55e232a240b0 con 0x55e232b35e20 2024-09-11T00:08:44.815 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:44.814+0000 7f4308e00640 1 --2- 172.21.15.12:0/716059121 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55e232b35e20 0x55e232b38240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:08:44.875 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:44.873+0000 7f4308e00640 1 --2- 172.21.15.12:0/716059121 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55e232b35e20 0x55e232b38240 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f678cb7d0f0f9816 server_cookie=c226a676665dd7bf in_seq=0 out_seq=0 2024-09-11T00:08:44.875 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:44.873+0000 7f42f2a00640 10 client.15654 ms_handle_connect on v2:172.21.15.165:6838/501312984 2024-09-11T00:08:44.948 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:44.945+0000 7f42f2a00640 1 -- 172.21.15.12:0/716059121 <== mds.0 v2:172.21.15.165:6838/501312984 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55e232a240b0 con 0x55e232b35e20 2024-09-11T00:08:44.948 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:44.945+0000 7f42f2a00640 10 client.15654 handle_command_reply: tid=0 2024-09-11T00:08:44.948 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:44.945+0000 7f430e7e5740 1 client.15654 shutdown 2024-09-11T00:08:44.948 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:44.945+0000 7f430e7e5740 1 -- 172.21.15.12:0/716059121 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e232ab60c0 msgr2=0x55e232ac3800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:08:44.948 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:44.945+0000 7f430e7e5740 1 --2- 172.21.15.12:0/716059121 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e232ab60c0 0x55e232ac3800 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f42ec0c8b20 tx=0x7f42ec0b82e0 comp rx=0 tx=0).stop 2024-09-11T00:08:44.948 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:44.945+0000 7f430e7e5740 1 -- 172.21.15.12:0/716059121 shutdown_connections 2024-09-11T00:08:44.948 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:44.945+0000 7f430e7e5740 1 --2- 172.21.15.12:0/716059121 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55e232b35e20 0x55e232b38240 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:08:44.948 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:44.945+0000 7f430e7e5740 1 --2- 172.21.15.12:0/716059121 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e232ab8b00 0x55e232ac3ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:08:44.948 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:44.945+0000 7f430e7e5740 1 --2- 172.21.15.12:0/716059121 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e232ab60c0 0x55e232ac3800 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:08:44.948 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:44.945+0000 7f430e7e5740 1 --2- 172.21.15.12:0/716059121 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e232ab3790 0x55e232ac4b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:08:44.949 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:44.945+0000 7f430e7e5740 1 -- 172.21.15.12:0/716059121 >> 172.21.15.12:0/716059121 conn(0x55e232aae8e0 msgr2=0x55e232aaf140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:08:44.949 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:44.945+0000 7f42f2a00640 0 client.15654 ms_handle_reset on v2:172.21.15.165:6838/501312984 2024-09-11T00:08:44.949 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:44.945+0000 7f430e7e5740 1 -- 172.21.15.12:0/716059121 shutdown_connections 2024-09-11T00:08:44.949 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:44.946+0000 7f430e7e5740 1 -- 172.21.15.12:0/716059121 wait complete. 2024-09-11T00:08:44.949 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:44.946+0000 7f430e7e5740 20 client.15654 trim_cache size 0 max 16384 2024-09-11T00:08:44.949 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:{ 2024-09-11T00:08:44.949 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "message": "", 2024-09-11T00:08:44.950 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "return_code": 0 2024-09-11T00:08:44.950 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:} 2024-09-11T00:08:44.950 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:44.946+0000 7f430b800640 1 -- 172.21.15.12:0/997665760 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f42e80b5360 msgr2=0x7f42e80b7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:08:44.951 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:44.946+0000 7f430b800640 1 --2- 172.21.15.12:0/997665760 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f42e80b5360 0x7f42e80b7820 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f42fc012bc0 tx=0x7f42fc005bd0 comp rx=0 tx=0).stop 2024-09-11T00:08:44.951 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:44.946+0000 7f430b800640 1 -- 172.21.15.12:0/997665760 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4304105cf0 msgr2=0x7f430419b640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:08:44.951 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:44.946+0000 7f430b800640 1 --2- 172.21.15.12:0/997665760 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4304105cf0 0x7f430419b640 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f42ec02a940 tx=0x7f42ec003c80 comp rx=0 tx=0).stop 2024-09-11T00:08:44.951 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:44.947+0000 7f430b800640 1 -- 172.21.15.12:0/997665760 shutdown_connections 2024-09-11T00:08:44.951 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:44.947+0000 7f430b800640 1 --2- 172.21.15.12:0/997665760 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f42e80b5360 0x7f42e80b7820 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:08:44.951 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:44.947+0000 7f430b800640 1 --2- 172.21.15.12:0/997665760 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f43041066b0 0x7f430419c270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:08:44.951 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:44.947+0000 7f430b800640 1 --2- 172.21.15.12:0/997665760 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4304105cf0 0x7f430419b640 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:08:44.951 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:44.947+0000 7f430b800640 1 --2- 172.21.15.12:0/997665760 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4304104ba0 0x7f430419aff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:08:44.951 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:44.947+0000 7f430b800640 1 -- 172.21.15.12:0/997665760 >> 172.21.15.12:0/997665760 conn(0x7f4304100460 msgr2=0x7f4304109080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:08:44.951 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:44.947+0000 7f430b800640 1 -- 172.21.15.12:0/997665760 shutdown_connections 2024-09-11T00:08:44.951 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:44.947+0000 7f430b800640 1 -- 172.21.15.12:0/997665760 wait complete. 2024-09-11T00:08:45.591 INFO:tasks.workunit.client.0.smithi012.stdout: 1 85337 11.40 MB/sec warmup 42 sec latency 3563.472 ms 2024-09-11T00:08:45.621 INFO:tasks.quiescer.fs.[cephfs]:Total duration (41.0) is longer than `split_if_longer` (35.0), will split into two consecutive quiesces 2024-09-11T00:08:45.621 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 20.5 2024-09-11T00:08:45.622 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid ff3ad74a-6fcf-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T00:08:45.705 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:08:45 smithi012 ceph-mon[30333]: pgmap v448: 129 pgs: 129 active+clean; 45 MiB data, 513 MiB used, 1.0 TiB / 1.0 TiB avail; 213 KiB/s rd, 2.9 MiB/s wr, 283 op/s 2024-09-11T00:08:46.017 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:08:46.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:08:45 smithi165 ceph-mon[35356]: pgmap v448: 129 pgs: 129 active+clean; 45 MiB data, 513 MiB used, 1.0 TiB / 1.0 TiB avail; 213 KiB/s rd, 2.9 MiB/s wr, 283 op/s 2024-09-11T00:08:46.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:08:45 smithi178 ceph-mon[35919]: pgmap v448: 129 pgs: 129 active+clean; 45 MiB data, 513 MiB used, 1.0 TiB / 1.0 TiB avail; 213 KiB/s rd, 2.9 MiB/s wr, 283 op/s 2024-09-11T00:08:46.081 INFO:journalctl@ceph.mds.c.smithi178.stdout:Sep 11 00:08:45 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-mds-c[66074]: ignoring --setuser ceph since I am not root 2024-09-11T00:08:46.081 INFO:journalctl@ceph.mds.c.smithi178.stdout:Sep 11 00:08:45 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-mds-c[66074]: ignoring --setgroup ceph since I am not root 2024-09-11T00:08:46.081 INFO:journalctl@ceph.mds.c.smithi178.stdout:Sep 11 00:08:45 smithi178 ceph-mds[66078]: -- 172.21.15.178:0/3276600490 <== mon.1 v2:172.21.15.178:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x558f06393860 con 0x558f06370800 2024-09-11T00:08:46.081 INFO:journalctl@ceph.mds.c.smithi178.stdout:Sep 11 00:08:45 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-mds-c[66074]: starting mds.c at 2024-09-11T00:08:46.591 INFO:tasks.workunit.client.0.smithi012.stdout: 1 85337 11.13 MB/sec warmup 43 sec latency 4563.532 ms 2024-09-11T00:08:46.839 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:08:46 smithi012 ceph-mon[30333]: daemon mds.c restarted 2024-09-11T00:08:46.890 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:08:46.877+0000 7f909fe00640 1 -- 172.21.15.12:0/2087383783 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f90a0104430 msgr2=0x7f90a010c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:08:46.891 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:08:46.877+0000 7f909fe00640 1 --2- 172.21.15.12:0/2087383783 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f90a0104430 0x7f90a010c920 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f908c009900 tx=0x7f908c02a3f0 comp rx=0 tx=0).stop 2024-09-11T00:08:46.891 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:08:46.877+0000 7f909fe00640 1 -- 172.21.15.12:0/2087383783 shutdown_connections 2024-09-11T00:08:46.891 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:08:46.877+0000 7f909fe00640 1 --2- 172.21.15.12:0/2087383783 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f90a010cf70 0x7f90a010f3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:08:46.891 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:08:46.877+0000 7f909fe00640 1 --2- 172.21.15.12:0/2087383783 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f90a0104430 0x7f90a010c920 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:08:46.891 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:08:46.877+0000 7f909fe00640 1 --2- 172.21.15.12:0/2087383783 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f90a0103b10 0x7f90a0103ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:08:46.891 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:08:46.877+0000 7f909fe00640 1 -- 172.21.15.12:0/2087383783 >> 172.21.15.12:0/2087383783 conn(0x7f90a00fd5b0 msgr2=0x7f90a00ff9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:08:46.891 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:08:46.877+0000 7f909fe00640 1 -- 172.21.15.12:0/2087383783 shutdown_connections 2024-09-11T00:08:46.891 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:08:46.877+0000 7f909fe00640 1 -- 172.21.15.12:0/2087383783 wait complete. 2024-09-11T00:08:46.891 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:08:46.878+0000 7f909fe00640 1 Processor -- start 2024-09-11T00:08:46.891 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:08:46.879+0000 7f909fe00640 1 -- start start 2024-09-11T00:08:46.891 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:08:46.879+0000 7f909fe00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f90a0103b10 0x7f90a019b240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:08:46.891 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:08:46.879+0000 7f909fe00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f90a0104430 0x7f90a019b780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:08:46.892 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:08:46.879+0000 7f909fe00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f90a010cf70 0x7f90a01a0b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:08:46.892 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:08:46.879+0000 7f909fe00640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f90a0112c30 con 0x7f90a0104430 2024-09-11T00:08:46.892 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:08:46.879+0000 7f909fe00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f90a0112ab0 con 0x7f90a010cf70 2024-09-11T00:08:46.892 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:08:46.879+0000 7f909fe00640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f90a0112db0 con 0x7f90a0103b10 2024-09-11T00:08:46.892 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:08:46.879+0000 7f909e000640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f90a0104430 0x7f90a019b780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:08:46.892 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:08:46.879+0000 7f909ea00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f90a0103b10 0x7f90a019b240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:08:46.892 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:08:46.879+0000 7f909e000640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f90a0104430 0x7f90a019b780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:34054/0 (socket says 172.21.15.12:34054) 2024-09-11T00:08:46.892 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:08:46.879+0000 7f909e000640 1 -- 172.21.15.12:0/3359356978 learned_addr learned my addr 172.21.15.12:0/3359356978 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:08:46.892 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:08:46.879+0000 7f909f400640 1 --2- 172.21.15.12:0/3359356978 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f90a010cf70 0x7f90a01a0b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:08:46.892 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:08:46.880+0000 7f909e000640 1 -- 172.21.15.12:0/3359356978 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f90a0103b10 msgr2=0x7f90a019b240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:08:46.892 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:08:46.880+0000 7f909e000640 1 --2- 172.21.15.12:0/3359356978 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f90a0103b10 0x7f90a019b240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:08:46.892 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:08:46.880+0000 7f909e000640 1 -- 172.21.15.12:0/3359356978 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f90a010cf70 msgr2=0x7f90a01a0b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:08:46.893 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:08:46.880+0000 7f909e000640 1 --2- 172.21.15.12:0/3359356978 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f90a010cf70 0x7f90a01a0b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:08:46.893 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:08:46.880+0000 7f909e000640 1 -- 172.21.15.12:0/3359356978 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90a01a12f0 con 0x7f90a0104430 2024-09-11T00:08:46.893 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:08:46.880+0000 7f909ea00640 1 --2- 172.21.15.12:0/3359356978 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f90a0103b10 0x7f90a019b240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:08:46.893 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:08:46.880+0000 7f909f400640 1 --2- 172.21.15.12:0/3359356978 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f90a010cf70 0x7f90a01a0b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T00:08:46.893 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:08:46.880+0000 7f909e000640 1 --2- 172.21.15.12:0/3359356978 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f90a0104430 0x7f90a019b780 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f908c005ab0 tx=0x7f908c003ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:08:46.893 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:08:46.880+0000 7f9097400640 1 -- 172.21.15.12:0/3359356978 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f908c03f070 con 0x7f90a0104430 2024-09-11T00:08:46.893 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:08:46.880+0000 7f9097400640 1 -- 172.21.15.12:0/3359356978 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f908c031b50 con 0x7f90a0104430 2024-09-11T00:08:46.893 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:08:46.881+0000 7f909fe00640 1 -- 172.21.15.12:0/3359356978 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f90a01a1580 con 0x7f90a0104430 2024-09-11T00:08:46.893 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:08:46.881+0000 7f9097400640 1 -- 172.21.15.12:0/3359356978 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f908c035030 con 0x7f90a0104430 2024-09-11T00:08:46.893 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:08:46.881+0000 7f909fe00640 1 -- 172.21.15.12:0/3359356978 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90a01a7ca0 con 0x7f90a0104430 2024-09-11T00:08:46.893 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:08:46.882+0000 7f909fe00640 1 -- 172.21.15.12:0/3359356978 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f90a0103ef0 con 0x7f90a0104430 2024-09-11T00:08:46.893 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:08:46.885+0000 7f9097400640 1 -- 172.21.15.12:0/3359356978 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7f908c0316a0 con 0x7f90a0104430 2024-09-11T00:08:46.894 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:08:46.887+0000 7f9097400640 1 --2- 172.21.15.12:0/3359356978 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f90840b5410 0x7f90840b78d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:08:46.894 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:08:46.887+0000 7f9097400640 1 -- 172.21.15.12:0/3359356978 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9208+0+0 (secure 0 0 0) 0x7f908c1067f0 con 0x7f90a0104430 2024-09-11T00:08:46.894 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:08:46.887+0000 7f909ea00640 1 --2- 172.21.15.12:0/3359356978 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f90840b5410 0x7f90840b78d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:08:46.894 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:08:46.888+0000 7f909ea00640 1 --2- 172.21.15.12:0/3359356978 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f90840b5410 0x7f90840b78d0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f9090009fc0 tx=0x7f9090005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:08:46.894 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:08:46.891+0000 7f9097400640 1 -- 172.21.15.12:0/3359356978 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f908c0cd280 con 0x7f90a0104430 2024-09-11T00:08:47.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:08:46 smithi165 ceph-mon[35356]: daemon mds.c restarted 2024-09-11T00:08:47.080 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:08:46 smithi178 ceph-mon[35919]: daemon mds.c restarted 2024-09-11T00:08:47.224 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:08:47.221+0000 7f909fe00640 1 -- 172.21.15.12:0/3359356978 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f90a01a7f30 con 0x7f90a0104430 2024-09-11T00:08:47.224 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:08:47.223+0000 7f9097400640 1 -- 172.21.15.12:0/3359356978 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) ==== 94+0+11972 (secure 0 0 0) 0x7f908c0ccb80 con 0x7f90a0104430 2024-09-11T00:08:47.225 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-11T00:08:47.225 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":33,"btime":"2024-09-11T00:08:46:706969+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14577,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6835/447171591","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":447171591},{"type":"v1","addr":"172.21.15.12:6835","nonce":447171591}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14589,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/901486040","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":901486040},{"type":"v1","addr":"172.21.15.12:6837","nonce":901486040}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":33,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T00:04:04.739881+0000","modified":"2024-09-11T00:08:46.706950+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24674,"mds_1":24643,"mds_2":24638,"mds_3":24637,"mds_4":24614},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14631":{"gid":14631,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.12:6841/3069919437","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6840","nonce":3069919437},{"type":"v1","addr":"172.21.15.12:6841","nonce":3069919437}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24562":{"gid":24562,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6833/1141636112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":1141636112},{"type":"v1","addr":"172.21.15.178:6833","nonce":1141636112}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24581":{"gid":24581,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6833/3111980970","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6832","nonce":3111980970},{"type":"v1","addr":"172.21.15.165:6833","nonce":3111980970}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24607":{"gid":24607,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6835/4084868287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":4084868287},{"type":"v1","addr":"172.21.15.178:6835","nonce":4084868287}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24614":{"gid":24614,"name":"e","rank":4,"incarnation":26,"state":"up:active","state_seq":21,"addr":"172.21.15.165:6835/2425395274","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":2425395274},{"type":"v1","addr":"172.21.15.165:6835","nonce":2425395274}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24637":{"gid":24637,"name":"i","rank":3,"incarnation":24,"state":"up:active","state_seq":18,"addr":"172.21.15.178:6839/70628097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6838","nonce":70628097},{"type":"v1","addr":"172.21.15.178:6839","nonce":70628097}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24638":{"gid":24638,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":18,"addr":"172.21.15.165:6837/6215088","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6836","nonce":6215088},{"type":"v1","addr":"172.21.15.165:6837","nonce":6215088}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24643":{"gid":24643,"name":"g","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.12:6839/3124093009","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6838","nonce":3124093009},{"type":"v1","addr":"172.21.15.12:6839","nonce":3124093009}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24674":{"gid":24674,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.165:6839/501312984","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6838","nonce":501312984},{"type":"v1","addr":"172.21.15.165:6839","nonce":501312984}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25672":{"gid":25672,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6837/308601741","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":308601741},{"type":"v1","addr":"172.21.15.178:6837","nonce":308601741}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24674,"qdb_cluster":[24614,24637,24638,24643,24674]},"id":1}]} 2024-09-11T00:08:47.225 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 33 2024-09-11T00:08:47.227 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:08:47.225+0000 7f909fe00640 1 -- 172.21.15.12:0/3359356978 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f90840b5410 msgr2=0x7f90840b78d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:08:47.227 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:08:47.225+0000 7f909fe00640 1 --2- 172.21.15.12:0/3359356978 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f90840b5410 0x7f90840b78d0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f9090009fc0 tx=0x7f9090005f90 comp rx=0 tx=0).stop 2024-09-11T00:08:47.227 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:08:47.225+0000 7f909fe00640 1 -- 172.21.15.12:0/3359356978 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f90a0104430 msgr2=0x7f90a019b780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:08:47.227 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:08:47.225+0000 7f909fe00640 1 --2- 172.21.15.12:0/3359356978 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f90a0104430 0x7f90a019b780 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f908c005ab0 tx=0x7f908c003ae0 comp rx=0 tx=0).stop 2024-09-11T00:08:47.228 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:08:47.225+0000 7f909fe00640 1 -- 172.21.15.12:0/3359356978 shutdown_connections 2024-09-11T00:08:47.228 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:08:47.225+0000 7f909fe00640 1 --2- 172.21.15.12:0/3359356978 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f90840b5410 0x7f90840b78d0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:08:47.228 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:08:47.225+0000 7f909fe00640 1 --2- 172.21.15.12:0/3359356978 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f90a010cf70 0x7f90a01a0b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:08:47.228 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:08:47.225+0000 7f909fe00640 1 --2- 172.21.15.12:0/3359356978 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f90a0104430 0x7f90a019b780 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:08:47.228 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:08:47.225+0000 7f909fe00640 1 --2- 172.21.15.12:0/3359356978 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f90a0103b10 0x7f90a019b240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:08:47.228 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:08:47.225+0000 7f909fe00640 1 -- 172.21.15.12:0/3359356978 >> 172.21.15.12:0/3359356978 conn(0x7f90a00fd5b0 msgr2=0x7f90a010f130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:08:47.228 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:08:47.225+0000 7f909fe00640 1 -- 172.21.15.12:0/3359356978 shutdown_connections 2024-09-11T00:08:47.228 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:08:47.226+0000 7f909fe00640 1 -- 172.21.15.12:0/3359356978 wait complete. 2024-09-11T00:08:47.594 INFO:tasks.workunit.client.0.smithi012.stdout: 1 85738 10.91 MB/sec warmup 44 sec latency 4998.078 ms 2024-09-11T00:08:48.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:08:47 smithi165 ceph-mon[35356]: pgmap v449: 129 pgs: 129 active+clean; 39 MiB data, 516 MiB used, 1.0 TiB / 1.0 TiB avail; 219 KiB/s rd, 3.5 MiB/s wr, 324 op/s 2024-09-11T00:08:48.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:08:47 smithi165 ceph-mon[35356]: osdmap e86: 12 total, 12 up, 12 in 2024-09-11T00:08:48.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:08:47 smithi165 ceph-mon[35356]: mds.? [v2:172.21.15.178:6836/308601741,v1:172.21.15.178:6837/308601741] up:boot 2024-09-11T00:08:48.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:08:47 smithi165 ceph-mon[35356]: fsmap cephfs:5 4 up:standby-replay 3 up:standby 5 up:active 2024-09-11T00:08:48.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:08:47 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "c"} : dispatch 2024-09-11T00:08:48.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:08:47 smithi165 ceph-mon[35356]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T00:08:48.033 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:08:47 smithi165 ceph-mon[35356]: from='client.? 172.21.15.12:0/3359356978' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T00:08:48.063 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:08:47 smithi012 ceph-mon[30333]: pgmap v449: 129 pgs: 129 active+clean; 39 MiB data, 516 MiB used, 1.0 TiB / 1.0 TiB avail; 219 KiB/s rd, 3.5 MiB/s wr, 324 op/s 2024-09-11T00:08:48.063 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:08:47 smithi012 ceph-mon[30333]: osdmap e86: 12 total, 12 up, 12 in 2024-09-11T00:08:48.063 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:08:47 smithi012 ceph-mon[30333]: mds.? [v2:172.21.15.178:6836/308601741,v1:172.21.15.178:6837/308601741] up:boot 2024-09-11T00:08:48.063 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:08:47 smithi012 ceph-mon[30333]: fsmap cephfs:5 4 up:standby-replay 3 up:standby 5 up:active 2024-09-11T00:08:48.063 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:08:47 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "c"} : dispatch 2024-09-11T00:08:48.063 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:08:47 smithi012 ceph-mon[30333]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T00:08:48.063 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:08:47 smithi012 ceph-mon[30333]: from='client.? 172.21.15.12:0/3359356978' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T00:08:48.080 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:08:47 smithi178 ceph-mon[35919]: pgmap v449: 129 pgs: 129 active+clean; 39 MiB data, 516 MiB used, 1.0 TiB / 1.0 TiB avail; 219 KiB/s rd, 3.5 MiB/s wr, 324 op/s 2024-09-11T00:08:48.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:08:47 smithi178 ceph-mon[35919]: osdmap e86: 12 total, 12 up, 12 in 2024-09-11T00:08:48.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:08:47 smithi178 ceph-mon[35919]: mds.? [v2:172.21.15.178:6836/308601741,v1:172.21.15.178:6837/308601741] up:boot 2024-09-11T00:08:48.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:08:47 smithi178 ceph-mon[35919]: fsmap cephfs:5 4 up:standby-replay 3 up:standby 5 up:active 2024-09-11T00:08:48.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:08:47 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "c"} : dispatch 2024-09-11T00:08:48.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:08:47 smithi178 ceph-mon[35919]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T00:08:48.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:08:47 smithi178 ceph-mon[35919]: from='client.? 172.21.15.12:0/3359356978' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T00:08:48.592 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24674 quiesce db / --timeout 90 --expiration 140.5 --await-for 45.0' 2024-09-11T00:08:48.592 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid ff3ad74a-6fcf-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24674 quiesce db / --timeout 90 --expiration 140.5 --await-for 45.0 2024-09-11T00:08:48.597 INFO:tasks.workunit.client.0.smithi012.stdout: 1 85843 10.67 MB/sec warmup 45 sec latency 899.275 ms 2024-09-11T00:08:48.980 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:08:49.563 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:08:49.560+0000 7f7892000640 1 -- 172.21.15.12:0/1255080136 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f788c104fc0 msgr2=0x7f788c1053a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:08:49.563 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:08:49.560+0000 7f7892000640 1 --2- 172.21.15.12:0/1255080136 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f788c104fc0 0x7f788c1053a0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f787c009900 tx=0x7f787c02a410 comp rx=0 tx=0).stop 2024-09-11T00:08:49.563 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:08:49.560+0000 7f7892000640 1 -- 172.21.15.12:0/1255080136 shutdown_connections 2024-09-11T00:08:49.563 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:08:49.560+0000 7f7892000640 1 --2- 172.21.15.12:0/1255080136 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f788c10b350 0x7f788c111b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:08:49.563 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:08:49.560+0000 7f7892000640 1 --2- 172.21.15.12:0/1255080136 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f788c10a9d0 0x7f788c10ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:08:49.563 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:08:49.560+0000 7f7892000640 1 --2- 172.21.15.12:0/1255080136 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f788c104fc0 0x7f788c1053a0 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:08:49.563 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:08:49.560+0000 7f7892000640 1 -- 172.21.15.12:0/1255080136 >> 172.21.15.12:0/1255080136 conn(0x7f788c1009f0 msgr2=0x7f788c102e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:08:49.563 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:08:49.560+0000 7f7892000640 1 -- 172.21.15.12:0/1255080136 shutdown_connections 2024-09-11T00:08:49.563 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:08:49.561+0000 7f7892000640 1 -- 172.21.15.12:0/1255080136 wait complete. 2024-09-11T00:08:49.565 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:08:49.561+0000 7f7892000640 1 Processor -- start 2024-09-11T00:08:49.565 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:08:49.562+0000 7f7892000640 1 -- start start 2024-09-11T00:08:49.565 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:08:49.562+0000 7f7892000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f788c104fc0 0x7f788c19b470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:08:49.565 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:08:49.562+0000 7f7892000640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f788c10a9d0 0x7f788c19b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:08:49.565 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:08:49.562+0000 7f7892000640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f788c10b350 0x7f788c1a0d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:08:49.565 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:08:49.562+0000 7f7892000640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f788c114f60 con 0x7f788c10b350 2024-09-11T00:08:49.565 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:08:49.562+0000 7f7892000640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f788c114de0 con 0x7f788c104fc0 2024-09-11T00:08:49.566 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:08:49.562+0000 7f7892000640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f788c1150e0 con 0x7f788c10a9d0 2024-09-11T00:08:49.566 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:08:49.562+0000 7f788b400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f788c104fc0 0x7f788c19b470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:08:49.566 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:08:49.562+0000 7f788aa00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f788c10a9d0 0x7f788c19b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:08:49.566 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:08:49.562+0000 7f788b400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f788c104fc0 0x7f788c19b470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.12:51396/0 (socket says 172.21.15.12:51396) 2024-09-11T00:08:49.566 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:08:49.562+0000 7f788b400640 1 -- 172.21.15.12:0/3789963994 learned_addr learned my addr 172.21.15.12:0/3789963994 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:08:49.566 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:08:49.563+0000 7f788be00640 1 --2- 172.21.15.12:0/3789963994 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f788c10b350 0x7f788c1a0d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:08:49.566 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:08:49.563+0000 7f788aa00640 1 -- 172.21.15.12:0/3789963994 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f788c104fc0 msgr2=0x7f788c19b470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:08:49.566 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:08:49.563+0000 7f788aa00640 1 --2- 172.21.15.12:0/3789963994 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f788c104fc0 0x7f788c19b470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:08:49.566 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:08:49.563+0000 7f788aa00640 1 -- 172.21.15.12:0/3789963994 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f788c10b350 msgr2=0x7f788c1a0d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:08:49.566 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:08:49.563+0000 7f788aa00640 1 --2- 172.21.15.12:0/3789963994 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f788c10b350 0x7f788c1a0d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:08:49.566 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:08:49.563+0000 7f788aa00640 1 -- 172.21.15.12:0/3789963994 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f788c1a1520 con 0x7f788c10a9d0 2024-09-11T00:08:49.567 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:08:49.563+0000 7f788b400640 1 --2- 172.21.15.12:0/3789963994 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f788c104fc0 0x7f788c19b470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:08:49.567 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:08:49.563+0000 7f788aa00640 1 --2- 172.21.15.12:0/3789963994 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f788c10a9d0 0x7f788c19b9b0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f787800e9f0 tx=0x7f787800eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:08:49.567 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:08:49.564+0000 7f7883e00640 1 -- 172.21.15.12:0/3789963994 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f787800ccc0 con 0x7f788c10a9d0 2024-09-11T00:08:49.567 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:08:49.564+0000 7f7883e00640 1 -- 172.21.15.12:0/3789963994 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f78780108e0 con 0x7f788c10a9d0 2024-09-11T00:08:49.567 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:08:49.564+0000 7f7892000640 1 -- 172.21.15.12:0/3789963994 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f788c1a1810 con 0x7f788c10a9d0 2024-09-11T00:08:49.568 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:08:49.564+0000 7f7883e00640 1 -- 172.21.15.12:0/3789963994 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7878018880 con 0x7f788c10a9d0 2024-09-11T00:08:49.568 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:08:49.564+0000 7f7892000640 1 -- 172.21.15.12:0/3789963994 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f788c1a7ed0 con 0x7f788c10a9d0 2024-09-11T00:08:49.569 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:08:49.567+0000 7f7894c08740 1 Processor -- start 2024-09-11T00:08:49.570 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:08:49.567+0000 7f7894c08740 1 -- start start 2024-09-11T00:08:49.570 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:08:49.568+0000 7f7894c08740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564a79865180 0x564a79865560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:08:49.570 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:08:49.568+0000 7f7894c08740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564a79865aa0 0x564a798714f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:08:49.570 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:08:49.568+0000 7f7894c08740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564a79871a30 0x564a79873e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:08:49.570 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:08:49.568+0000 7f7894c08740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x564a797daca0 con 0x564a79865180 2024-09-11T00:08:49.570 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:08:49.568+0000 7f7894c08740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x564a797ca0d0 con 0x564a79871a30 2024-09-11T00:08:49.571 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:08:49.568+0000 7f7894c08740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x564a797cdee0 con 0x564a79865aa0 2024-09-11T00:08:49.571 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:08:49.568+0000 7f7883e00640 1 -- 172.21.15.12:0/3789963994 <== mon.1 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7f7878010430 con 0x7f788c10a9d0 2024-09-11T00:08:49.571 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:08:49.568+0000 7f788be00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564a79865180 0x564a79865560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:08:49.571 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:08:49.568+0000 7f788be00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564a79865180 0x564a79865560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:45750/0 (socket says 172.21.15.12:45750) 2024-09-11T00:08:49.571 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:08:49.568+0000 7f788be00640 1 -- 172.21.15.12:0/58204324 learned_addr learned my addr 172.21.15.12:0/58204324 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:08:49.571 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:08:49.568+0000 7f788b400640 1 --2- 172.21.15.12:0/58204324 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564a79865aa0 0x564a798714f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:08:49.571 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:08:49.568+0000 7f788aa00640 1 --2- 172.21.15.12:0/58204324 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564a79871a30 0x564a79873e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:08:49.572 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:08:49.569+0000 7f788be00640 1 -- 172.21.15.12:0/58204324 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564a79865aa0 msgr2=0x564a798714f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:08:49.573 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:08:49.569+0000 7f788be00640 1 --2- 172.21.15.12:0/58204324 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564a79865aa0 0x564a798714f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:08:49.573 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:08:49.569+0000 7f788be00640 1 -- 172.21.15.12:0/58204324 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564a79871a30 msgr2=0x564a79873e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:08:49.573 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:08:49.569+0000 7f788be00640 1 --2- 172.21.15.12:0/58204324 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564a79871a30 0x564a79873e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:08:49.573 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:08:49.569+0000 7f788be00640 1 -- 172.21.15.12:0/58204324 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564a7977c400 con 0x564a79865180 2024-09-11T00:08:49.573 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:08:49.569+0000 7f788b400640 1 --2- 172.21.15.12:0/58204324 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564a79865aa0 0x564a798714f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:08:49.573 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:08:49.569+0000 7f788be00640 1 --2- 172.21.15.12:0/58204324 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564a79865180 0x564a79865560 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f7884010370 tx=0x7f7884010830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5924a785868fe5d server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:08:49.573 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:08:49.569+0000 7f7877e00640 1 -- 172.21.15.12:0/58204324 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7884016070 con 0x564a79865180 2024-09-11T00:08:49.573 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:08:49.569+0000 7f7877e00640 1 -- 172.21.15.12:0/58204324 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7884004d20 con 0x564a79865180 2024-09-11T00:08:49.573 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:08:49.569+0000 7f7877e00640 1 -- 172.21.15.12:0/58204324 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7884005420 con 0x564a79865180 2024-09-11T00:08:49.573 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:08:49.570+0000 7f7883e00640 1 --2- 172.21.15.12:0/3789963994 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f78700b5360 0x7f78700b7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:08:49.574 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:08:49.570+0000 7f7894c08740 1 -- 172.21.15.12:0/58204324 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564a79865180 msgr2=0x564a79865560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:08:49.574 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:08:49.570+0000 7f7894c08740 1 --2- 172.21.15.12:0/58204324 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564a79865180 0x564a79865560 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f7884010370 tx=0x7f7884010830 comp rx=0 tx=0).stop 2024-09-11T00:08:49.574 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:08:49.570+0000 7f7883e00640 1 -- 172.21.15.12:0/3789963994 <== mon.1 v2:172.21.15.178:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9208+0+0 (secure 0 0 0) 0x7f78780e62b0 con 0x7f788c10a9d0 2024-09-11T00:08:49.574 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:08:49.570+0000 7f7894c08740 1 -- 172.21.15.12:0/58204324 shutdown_connections 2024-09-11T00:08:49.574 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:08:49.570+0000 7f7894c08740 1 --2- 172.21.15.12:0/58204324 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564a79871a30 0x564a79873e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:08:49.574 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:08:49.570+0000 7f7894c08740 1 --2- 172.21.15.12:0/58204324 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564a79865aa0 0x564a798714f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:08:49.574 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:08:49.570+0000 7f7894c08740 1 --2- 172.21.15.12:0/58204324 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564a79865180 0x564a79865560 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:08:49.574 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:08:49.570+0000 7f7894c08740 1 -- 172.21.15.12:0/58204324 >> 172.21.15.12:0/58204324 conn(0x564a79862fb0 msgr2=0x564a79864150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:08:49.574 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:08:49.570+0000 7f788b400640 1 --2- 172.21.15.12:0/3789963994 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f78700b5360 0x7f78700b7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:08:49.574 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:08:49.570+0000 7f7894c08740 1 -- 172.21.15.12:0/58204324 shutdown_connections 2024-09-11T00:08:49.574 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:08:49.571+0000 7f7894c08740 1 -- 172.21.15.12:0/58204324 wait complete. 2024-09-11T00:08:49.576 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:08:49.571+0000 7f7894c08740 1 -- start start 2024-09-11T00:08:49.576 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:08:49.571+0000 7f788b400640 1 --2- 172.21.15.12:0/3789963994 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f78700b5360 0x7f78700b7820 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f787c031440 tx=0x7f787c0058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:08:49.577 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:08:49.571+0000 7f7894c08740 1 Processor -- start 2024-09-11T00:08:49.577 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:08:49.572+0000 7f7894c08740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564a79865180 0x564a79871280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:08:49.577 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:08:49.572+0000 7f7894c08740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564a79865aa0 0x564a7986e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:08:49.577 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:08:49.572+0000 7f7894c08740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564a79871a30 0x564a7986ec30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:08:49.577 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:08:49.572+0000 7f7894c08740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x564a7962b7a0 con 0x564a79871a30 2024-09-11T00:08:49.577 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:08:49.572+0000 7f7894c08740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x564a7976fa10 con 0x564a79865180 2024-09-11T00:08:49.577 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:08:49.572+0000 7f7894c08740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x564a798746b0 con 0x564a79865aa0 2024-09-11T00:08:49.577 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:08:49.572+0000 7f7894c08740 10 client.0 _ll_register_callbacks cb 0x564a7977fbe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:08:49.577 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:08:49.572+0000 7f788b400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564a79865180 0x564a79871280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:08:49.577 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:08:49.572+0000 7f788b400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564a79865180 0x564a79871280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.12:51414/0 (socket says 172.21.15.12:51414) 2024-09-11T00:08:49.577 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:08:49.572+0000 7f788be00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564a79871a30 0x564a7986ec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:08:49.578 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:08:49.572+0000 7f788b400640 1 -- 172.21.15.12:0/3077582197 learned_addr learned my addr 172.21.15.12:0/3077582197 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:08:49.578 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:08:49.572+0000 7f788aa00640 1 --2- 172.21.15.12:0/3077582197 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564a79865aa0 0x564a7986e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:08:49.578 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:08:49.572+0000 7f788b400640 1 -- 172.21.15.12:0/3077582197 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564a79865aa0 msgr2=0x564a7986e6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:08:49.578 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:08:49.572+0000 7f788b400640 1 --2- 172.21.15.12:0/3077582197 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564a79865aa0 0x564a7986e6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:08:49.578 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:08:49.573+0000 7f788b400640 1 -- 172.21.15.12:0/3077582197 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564a79871a30 msgr2=0x564a7986ec30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:08:49.578 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:08:49.573+0000 7f788b400640 1 --2- 172.21.15.12:0/3077582197 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564a79871a30 0x564a7986ec30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:08:49.578 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:08:49.573+0000 7f788b400640 1 -- 172.21.15.12:0/3077582197 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564a79537b90 con 0x564a79865180 2024-09-11T00:08:49.578 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:08:49.573+0000 7f788aa00640 1 --2- 172.21.15.12:0/3077582197 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564a79865aa0 0x564a7986e6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:08:49.578 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:08:49.573+0000 7f788b400640 1 --2- 172.21.15.12:0/3077582197 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564a79865180 0x564a79871280 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f787c009a20 tx=0x7f787c009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=834fa6dbb76ae02f server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:08:49.578 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:08:49.573+0000 7f7874c00640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-09-11T00:08:49.578 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:08:49.573+0000 7f7874c00640 1 -- 172.21.15.12:0/3077582197 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f787c04d070 con 0x564a79865180 2024-09-11T00:08:49.578 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:08:49.573+0000 7f7894c08740 1 -- 172.21.15.12:0/3077582197 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7884004d20 con 0x564a79865180 2024-09-11T00:08:49.579 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:08:49.573+0000 7f7874c00640 1 -- 172.21.15.12:0/3077582197 <== mon.2 v2:172.21.15.165:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9208+0+0 (secure 0 0 0) 0x7f787c005220 con 0x564a79865180 2024-09-11T00:08:49.579 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:08:49.574+0000 7f7874c00640 1 -- 172.21.15.12:0/3077582197 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f787c043070 con 0x564a79865180 2024-09-11T00:08:49.579 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:08:49.574+0000 7f7894c08740 10 client.25703 fetch_fsmap learned FSMap version 33 2024-09-11T00:08:49.579 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:08:49.574+0000 7f7894c08740 1 -- 172.21.15.12:0/3077582197 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x564a7987ff90 con 0x564a79865180 2024-09-11T00:08:49.579 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:08:49.574+0000 7f7874c00640 1 -- 172.21.15.12:0/3077582197 <== mon.2 v2:172.21.15.165:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7884004d20 con 0x564a79865180 2024-09-11T00:08:49.579 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:08:49.574+0000 7f7874c00640 1 -- 172.21.15.12:0/3077582197 <== mon.2 v2:172.21.15.165:3300/0 5 ==== fsmap(e 33) ==== 8386+0+0 (secure 0 0 0) 0x7f787c003680 con 0x564a79865180 2024-09-11T00:08:49.579 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:08:49.574+0000 7f7894c08740 10 client.25703 fetch_fsmap finished waiting for FSMap version 33 2024-09-11T00:08:49.579 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:08:49.574+0000 7f7894c08740 10 client.25703 resolve_mds: validated gid 24674 aka daemon mds.b 2024-09-11T00:08:49.579 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:08:49.574+0000 7f7894c08740 20 client.25703 populate_metadata read hostname 'smithi012' 2024-09-11T00:08:49.579 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:08:49.574+0000 7f7894c08740 1 --2- 172.21.15.12:0/3077582197 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x564a798814c0 0x564a79883940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:08:49.579 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:08:49.574+0000 7f7894c08740 4 client.25703 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T00:08:49.579 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:08:49.574+0000 7f7894c08740 1 -- 172.21.15.12:0/3077582197 --> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564a797dcbf0 con 0x564a798814c0 2024-09-11T00:08:49.580 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:08:49.575+0000 7f788aa00640 1 --2- 172.21.15.12:0/3077582197 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x564a798814c0 0x564a79883940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:08:49.581 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:08:49.575+0000 7f788aa00640 1 --2- 172.21.15.12:0/3077582197 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x564a798814c0 0x564a79883940 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3bdc51bf8b85608e server_cookie=699d69091441ac2e in_seq=0 out_seq=0 2024-09-11T00:08:49.581 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:08:49.575+0000 7f7874c00640 10 client.25703 ms_handle_connect on v2:172.21.15.165:6838/501312984 2024-09-11T00:08:49.581 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:08:49.576+0000 7f7874c00640 1 -- 172.21.15.12:0/3077582197 <== mds.0 v2:172.21.15.165:6838/501312984 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x564a797dcbf0 con 0x564a798814c0 2024-09-11T00:08:49.581 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:08:49.576+0000 7f7874c00640 10 client.25703 handle_command_reply: tid=0 2024-09-11T00:08:49.582 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:08:49.576+0000 7f7894c08740 1 client.25703 shutdown 2024-09-11T00:08:49.582 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:08:49.576+0000 7f7894c08740 1 -- 172.21.15.12:0/3077582197 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564a79865180 msgr2=0x564a79871280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:08:49.582 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:08:49.576+0000 7f7894c08740 1 --2- 172.21.15.12:0/3077582197 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564a79865180 0x564a79871280 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f787c009a20 tx=0x7f787c009260 comp rx=0 tx=0).stop 2024-09-11T00:08:49.582 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:08:49.576+0000 7f7894c08740 1 -- 172.21.15.12:0/3077582197 shutdown_connections 2024-09-11T00:08:49.582 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:08:49.576+0000 7f7894c08740 1 --2- 172.21.15.12:0/3077582197 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x564a798814c0 0x564a79883940 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:08:49.582 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:08:49.576+0000 7f7894c08740 1 --2- 172.21.15.12:0/3077582197 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564a79871a30 0x564a7986ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:08:49.582 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:08:49.577+0000 7f7894c08740 1 --2- 172.21.15.12:0/3077582197 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564a79865aa0 0x564a7986e6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:08:49.582 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:08:49.577+0000 7f7894c08740 1 --2- 172.21.15.12:0/3077582197 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564a79865180 0x564a79871280 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:08:49.582 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:08:49.577+0000 7f7894c08740 1 -- 172.21.15.12:0/3077582197 >> 172.21.15.12:0/3077582197 conn(0x564a79862fb0 msgr2=0x564a798620b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:08:49.582 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:08:49.577+0000 7f7874c00640 0 client.25703 ms_handle_reset on v2:172.21.15.165:6838/501312984 2024-09-11T00:08:49.582 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:08:49.577+0000 7f7894c08740 1 -- 172.21.15.12:0/3077582197 shutdown_connections 2024-09-11T00:08:49.582 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:08:49.577+0000 7f7894c08740 1 -- 172.21.15.12:0/3077582197 wait complete. 2024-09-11T00:08:49.583 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:08:49.577+0000 7f7894c08740 20 client.25703 trim_cache size 0 max 16384 2024-09-11T00:08:49.594 INFO:tasks.workunit.client.0.smithi012.stdout: 1 85899 10.44 MB/sec warmup 46 sec latency 1838.707 ms 2024-09-11T00:08:49.597 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:08:49.595+0000 7f7894c08740 1 Processor -- start 2024-09-11T00:08:49.597 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:08:49.595+0000 7f7894c08740 1 -- start start 2024-09-11T00:08:49.598 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:08:49.596+0000 7f7894c08740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564a79865180 0x564a7986dfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:08:49.598 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:08:49.596+0000 7f7894c08740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564a79865aa0 0x564a79872e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:08:49.599 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:08:49.596+0000 7f7894c08740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564a79871a30 0x564a79873360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:08:49.599 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:08:49.596+0000 7f7894c08740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f788c1b1e60 con 0x564a79871a30 2024-09-11T00:08:49.599 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:08:49.596+0000 7f7894c08740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f788c1b3a60 con 0x564a79865180 2024-09-11T00:08:49.599 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:08:49.596+0000 7f7894c08740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f788c1b2530 con 0x564a79865aa0 2024-09-11T00:08:49.599 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:08:49.596+0000 7f788be00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564a79871a30 0x564a79873360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:08:49.599 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:08:49.596+0000 7f788be00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564a79871a30 0x564a79873360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:45778/0 (socket says 172.21.15.12:45778) 2024-09-11T00:08:49.599 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:08:49.596+0000 7f788be00640 1 -- 172.21.15.12:0/2650477315 learned_addr learned my addr 172.21.15.12:0/2650477315 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:08:49.599 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:08:49.596+0000 7f788b400640 1 --2- 172.21.15.12:0/2650477315 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564a79865180 0x564a7986dfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:08:49.599 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:08:49.596+0000 7f788aa00640 1 --2- 172.21.15.12:0/2650477315 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564a79865aa0 0x564a79872e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:08:49.599 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:08:49.596+0000 7f788b400640 1 -- 172.21.15.12:0/2650477315 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564a79865aa0 msgr2=0x564a79872e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:08:49.599 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:08:49.596+0000 7f788b400640 1 --2- 172.21.15.12:0/2650477315 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564a79865aa0 0x564a79872e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:08:49.600 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:08:49.596+0000 7f788b400640 1 -- 172.21.15.12:0/2650477315 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564a79871a30 msgr2=0x564a79873360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:08:49.600 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:08:49.596+0000 7f788b400640 1 --2- 172.21.15.12:0/2650477315 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564a79871a30 0x564a79873360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:08:49.600 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:08:49.596+0000 7f788b400640 1 -- 172.21.15.12:0/2650477315 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564a7986efa0 con 0x564a79865180 2024-09-11T00:08:49.600 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:08:49.596+0000 7f788be00640 1 --2- 172.21.15.12:0/2650477315 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564a79871a30 0x564a79873360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:08:49.600 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:08:49.597+0000 7f788b400640 1 --2- 172.21.15.12:0/2650477315 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564a79865180 0x564a7986dfa0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f787c006010 tx=0x7f787c0314a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=544eb8e3275ef7b1 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:08:49.602 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:08:49.597+0000 7f7877400640 1 -- 172.21.15.12:0/2650477315 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f787c04d070 con 0x564a79865180 2024-09-11T00:08:49.602 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:08:49.597+0000 7f7877400640 1 -- 172.21.15.12:0/2650477315 <== mon.2 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f787c009520 con 0x564a79865180 2024-09-11T00:08:49.602 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:08:49.597+0000 7f7877400640 1 -- 172.21.15.12:0/2650477315 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f787c035040 con 0x564a79865180 2024-09-11T00:08:49.602 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:08:49.598+0000 7f7894c08740 1 -- 172.21.15.12:0/2650477315 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564a79865180 msgr2=0x564a7986dfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:08:49.602 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:08:49.598+0000 7f7894c08740 1 --2- 172.21.15.12:0/2650477315 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564a79865180 0x564a7986dfa0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f787c006010 tx=0x7f787c0314a0 comp rx=0 tx=0).stop 2024-09-11T00:08:49.602 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:08:49.598+0000 7f7894c08740 1 -- 172.21.15.12:0/2650477315 shutdown_connections 2024-09-11T00:08:49.603 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:08:49.598+0000 7f7894c08740 1 --2- 172.21.15.12:0/2650477315 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564a79871a30 0x564a79873360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:08:49.603 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:08:49.598+0000 7f7894c08740 1 --2- 172.21.15.12:0/2650477315 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564a79865aa0 0x564a79872e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:08:49.603 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:08:49.598+0000 7f7894c08740 1 --2- 172.21.15.12:0/2650477315 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564a79865180 0x564a7986dfa0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:08:49.603 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:08:49.598+0000 7f7894c08740 1 -- 172.21.15.12:0/2650477315 >> 172.21.15.12:0/2650477315 conn(0x564a79862fb0 msgr2=0x564a79864070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:08:49.603 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:08:49.598+0000 7f7894c08740 1 -- 172.21.15.12:0/2650477315 shutdown_connections 2024-09-11T00:08:49.603 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:08:49.598+0000 7f7894c08740 1 -- 172.21.15.12:0/2650477315 wait complete. 2024-09-11T00:08:49.603 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:08:49.598+0000 7f7894c08740 1 -- start start 2024-09-11T00:08:49.603 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:08:49.598+0000 7f7894c08740 1 Processor -- start 2024-09-11T00:08:49.603 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:08:49.599+0000 7f7894c08740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564a79865180 0x564a79879940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:08:49.603 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:08:49.599+0000 7f7894c08740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564a79865aa0 0x564a79879e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:08:49.603 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:08:49.599+0000 7f7894c08740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564a79871a30 0x564a7986a930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:08:49.603 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:08:49.599+0000 7f7894c08740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f788c1b3390 con 0x564a79865aa0 2024-09-11T00:08:49.604 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:08:49.599+0000 7f7894c08740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f788c1b2c00 con 0x564a79871a30 2024-09-11T00:08:49.604 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:08:49.599+0000 7f7894c08740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f788c1afe40 con 0x564a79865180 2024-09-11T00:08:49.604 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:08:49.599+0000 7f7894c08740 10 client.0 _ll_register_callbacks cb 0x7f787c032ed0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:08:49.604 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:08:49.599+0000 7f788aa00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564a79865aa0 0x564a79879e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:08:49.604 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:08:49.599+0000 7f788aa00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564a79865aa0 0x564a79879e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:45784/0 (socket says 172.21.15.12:45784) 2024-09-11T00:08:49.604 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:08:49.599+0000 7f788aa00640 1 -- 172.21.15.12:0/2873058486 learned_addr learned my addr 172.21.15.12:0/2873058486 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:08:49.604 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:08:49.599+0000 7f788b400640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564a79865180 0x564a79879940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:08:49.604 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:08:49.599+0000 7f788be00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564a79871a30 0x564a7986a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:08:49.604 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:08:49.600+0000 7f788b400640 1 -- 172.21.15.12:0/2873058486 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564a79871a30 msgr2=0x564a7986a930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:08:49.604 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:08:49.600+0000 7f788b400640 1 --2- 172.21.15.12:0/2873058486 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564a79871a30 0x564a7986a930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:08:49.604 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:08:49.600+0000 7f788b400640 1 -- 172.21.15.12:0/2873058486 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564a79865aa0 msgr2=0x564a79879e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:08:49.604 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:08:49.600+0000 7f788b400640 1 --2- 172.21.15.12:0/2873058486 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564a79865aa0 0x564a79879e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:08:49.605 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:08:49.600+0000 7f788b400640 1 -- 172.21.15.12:0/2873058486 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564a7986ae70 con 0x564a79865180 2024-09-11T00:08:49.605 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:08:49.600+0000 7f788aa00640 1 --2- 172.21.15.12:0/2873058486 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564a79865aa0 0x564a79879e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:08:49.605 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:08:49.600+0000 7f788b400640 1 --2- 172.21.15.12:0/2873058486 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564a79865180 0x564a79879940 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f787c006010 tx=0x7f787c003820 comp rx=0 tx=0).ready entity=mon.1 client_cookie=36b03fda7fa0000e server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:08:49.605 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:08:49.600+0000 7f7874c00640 10 client.0 ms_handle_connect on v2:172.21.15.178:3300/0 2024-09-11T00:08:49.605 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:08:49.600+0000 7f7874c00640 1 -- 172.21.15.12:0/2873058486 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f787c04d040 con 0x564a79865180 2024-09-11T00:08:49.605 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:08:49.600+0000 7f7894c08740 1 -- 172.21.15.12:0/2873058486 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f787c009520 con 0x564a79865180 2024-09-11T00:08:49.605 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:08:49.600+0000 7f7874c00640 1 -- 172.21.15.12:0/2873058486 <== mon.1 v2:172.21.15.178:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9208+0+0 (secure 0 0 0) 0x7f787c005450 con 0x564a79865180 2024-09-11T00:08:49.606 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:08:49.601+0000 7f7874c00640 1 -- 172.21.15.12:0/2873058486 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f787c074050 con 0x564a79865180 2024-09-11T00:08:49.606 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:08:49.601+0000 7f7894c08740 10 client.25696 fetch_fsmap learned FSMap version 33 2024-09-11T00:08:49.606 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:08:49.601+0000 7f7894c08740 1 -- 172.21.15.12:0/2873058486 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x564a7986c150 con 0x564a79865180 2024-09-11T00:08:49.606 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:08:49.601+0000 7f7874c00640 1 -- 172.21.15.12:0/2873058486 <== mon.1 v2:172.21.15.178:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f787c009520 con 0x564a79865180 2024-09-11T00:08:49.606 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:08:49.601+0000 7f7874c00640 1 -- 172.21.15.12:0/2873058486 <== mon.1 v2:172.21.15.178:3300/0 5 ==== fsmap(e 33) ==== 8386+0+0 (secure 0 0 0) 0x7f787c03d850 con 0x564a79865180 2024-09-11T00:08:49.607 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:08:49.601+0000 7f7894c08740 10 client.25696 fetch_fsmap finished waiting for FSMap version 33 2024-09-11T00:08:49.607 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:08:49.601+0000 7f7894c08740 10 client.25696 resolve_mds: validated gid 24674 aka daemon mds.b 2024-09-11T00:08:49.607 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:08:49.601+0000 7f7894c08740 20 client.25696 populate_metadata read hostname 'smithi012' 2024-09-11T00:08:49.607 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:08:49.601+0000 7f7894c08740 1 --2- 172.21.15.12:0/2873058486 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x564a798ef420 0x564a798f1810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:08:49.607 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:08:49.601+0000 7f7894c08740 4 client.25696 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 140.5, "await_for": 45.0}] 2024-09-11T00:08:49.607 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:08:49.601+0000 7f7894c08740 1 -- 172.21.15.12:0/2873058486 --> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 140.5, "await_for": 45.0}) -- 0x564a797d86a0 con 0x564a798ef420 2024-09-11T00:08:49.607 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:08:49.602+0000 7f788aa00640 1 --2- 172.21.15.12:0/2873058486 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x564a798ef420 0x564a798f1810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:08:49.607 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:08:49.602+0000 7f788aa00640 1 --2- 172.21.15.12:0/2873058486 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x564a798ef420 0x564a798f1810 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4b0efe29de6649bb server_cookie=c007e12d0a847379 in_seq=0 out_seq=0 2024-09-11T00:08:49.607 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:08:49.602+0000 7f7874c00640 10 client.25696 ms_handle_connect on v2:172.21.15.165:6838/501312984 2024-09-11T00:08:49.676 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:08:49.674+0000 7f7874c00640 1 -- 172.21.15.12:0/2873058486 <== mds.0 v2:172.21.15.165:6838/501312984 1 ==== command_reply(tid 0: 0 ) ==== 8+0+589 (crc 0 0 0) 0x564a797d86a0 con 0x564a798ef420 2024-09-11T00:08:49.677 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:08:49.674+0000 7f7874c00640 10 client.25696 handle_command_reply: tid=0 2024-09-11T00:08:49.677 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:08:49.674+0000 7f7894c08740 1 client.25696 shutdown 2024-09-11T00:08:49.677 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:08:49.674+0000 7f7894c08740 1 -- 172.21.15.12:0/2873058486 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564a79865180 msgr2=0x564a79879940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:08:49.677 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:08:49.674+0000 7f7894c08740 1 --2- 172.21.15.12:0/2873058486 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564a79865180 0x564a79879940 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f787c006010 tx=0x7f787c003820 comp rx=0 tx=0).stop 2024-09-11T00:08:49.677 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:08:49.674+0000 7f7894c08740 1 -- 172.21.15.12:0/2873058486 shutdown_connections 2024-09-11T00:08:49.677 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:08:49.674+0000 7f7894c08740 1 --2- 172.21.15.12:0/2873058486 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x564a798ef420 0x564a798f1810 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:08:49.677 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:08:49.674+0000 7f7894c08740 1 --2- 172.21.15.12:0/2873058486 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564a79871a30 0x564a7986a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:08:49.677 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:08:49.674+0000 7f7894c08740 1 --2- 172.21.15.12:0/2873058486 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564a79865aa0 0x564a79879e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:08:49.677 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:08:49.674+0000 7f7894c08740 1 --2- 172.21.15.12:0/2873058486 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564a79865180 0x564a79879940 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:08:49.677 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:08:49.674+0000 7f7894c08740 1 -- 172.21.15.12:0/2873058486 >> 172.21.15.12:0/2873058486 conn(0x564a79862fb0 msgr2=0x564a79861890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:08:49.677 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:08:49.674+0000 7f7874c00640 0 client.25696 ms_handle_reset on v2:172.21.15.165:6838/501312984 2024-09-11T00:08:49.678 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:08:49.675+0000 7f7894c08740 1 -- 172.21.15.12:0/2873058486 shutdown_connections 2024-09-11T00:08:49.678 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:08:49.675+0000 7f7894c08740 1 -- 172.21.15.12:0/2873058486 wait complete. 2024-09-11T00:08:49.678 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:08:49.675+0000 7f7894c08740 20 client.25696 trim_cache size 0 max 16384 2024-09-11T00:08:49.678 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-09-11T00:08:49.678 INFO:teuthology.orchestra.run.smithi012.stdout: "epoch": 33, 2024-09-11T00:08:49.678 INFO:teuthology.orchestra.run.smithi012.stdout: "leader": 24674, 2024-09-11T00:08:49.678 INFO:teuthology.orchestra.run.smithi012.stdout: "set_version": 6, 2024-09-11T00:08:49.679 INFO:teuthology.orchestra.run.smithi012.stdout: "sets": { 2024-09-11T00:08:49.679 INFO:teuthology.orchestra.run.smithi012.stdout: "b4f324a8": { 2024-09-11T00:08:49.679 INFO:teuthology.orchestra.run.smithi012.stdout: "version": 6, 2024-09-11T00:08:49.679 INFO:teuthology.orchestra.run.smithi012.stdout: "age_ref": 0.0, 2024-09-11T00:08:49.679 INFO:teuthology.orchestra.run.smithi012.stdout: "state": { 2024-09-11T00:08:49.679 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "QUIESCED", 2024-09-11T00:08:49.679 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 0.0 2024-09-11T00:08:49.679 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:08:49.679 INFO:teuthology.orchestra.run.smithi012.stdout: "timeout": 90.0, 2024-09-11T00:08:49.679 INFO:teuthology.orchestra.run.smithi012.stdout: "expiration": 140.5, 2024-09-11T00:08:49.679 INFO:teuthology.orchestra.run.smithi012.stdout: "members": { 2024-09-11T00:08:49.679 INFO:teuthology.orchestra.run.smithi012.stdout: "file:/": { 2024-09-11T00:08:49.679 INFO:teuthology.orchestra.run.smithi012.stdout: "excluded": false, 2024-09-11T00:08:49.680 INFO:teuthology.orchestra.run.smithi012.stdout: "state": { 2024-09-11T00:08:49.680 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "QUIESCED", 2024-09-11T00:08:49.680 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 0.0 2024-09-11T00:08:49.680 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:08:49.680 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:08:49.680 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:08:49.680 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:08:49.680 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:08:49.680 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-09-11T00:08:49.681 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:08:49.675+0000 7f7892000640 1 -- 172.21.15.12:0/3789963994 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f78700b5360 msgr2=0x7f78700b7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:08:49.681 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:08:49.675+0000 7f7892000640 1 --2- 172.21.15.12:0/3789963994 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f78700b5360 0x7f78700b7820 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f787c031440 tx=0x7f787c0058d0 comp rx=0 tx=0).stop 2024-09-11T00:08:49.681 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:08:49.675+0000 7f7892000640 1 -- 172.21.15.12:0/3789963994 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f788c10a9d0 msgr2=0x7f788c19b9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:08:49.681 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:08:49.675+0000 7f7892000640 1 --2- 172.21.15.12:0/3789963994 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f788c10a9d0 0x7f788c19b9b0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f787800e9f0 tx=0x7f787800eeb0 comp rx=0 tx=0).stop 2024-09-11T00:08:49.681 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:08:49.676+0000 7f7892000640 1 -- 172.21.15.12:0/3789963994 shutdown_connections 2024-09-11T00:08:49.681 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:08:49.676+0000 7f7892000640 1 --2- 172.21.15.12:0/3789963994 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f78700b5360 0x7f78700b7820 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:08:49.681 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:08:49.676+0000 7f7892000640 1 --2- 172.21.15.12:0/3789963994 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f788c10b350 0x7f788c1a0d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:08:49.681 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:08:49.676+0000 7f7892000640 1 --2- 172.21.15.12:0/3789963994 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f788c10a9d0 0x7f788c19b9b0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:08:49.682 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:08:49.676+0000 7f7892000640 1 --2- 172.21.15.12:0/3789963994 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f788c104fc0 0x7f788c19b470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:08:49.682 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:08:49.676+0000 7f7892000640 1 -- 172.21.15.12:0/3789963994 >> 172.21.15.12:0/3789963994 conn(0x7f788c1009f0 msgr2=0x7f788c1012f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:08:49.682 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:08:49.676+0000 7f7892000640 1 -- 172.21.15.12:0/3789963994 shutdown_connections 2024-09-11T00:08:49.682 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:08:49.676+0000 7f7892000640 1 -- 172.21.15.12:0/3789963994 wait complete. 2024-09-11T00:08:49.964 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:08:49 smithi012 ceph-mon[30333]: pgmap v451: 129 pgs: 129 active+clean; 38 MiB data, 514 MiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 2.2 MiB/s wr, 125 op/s 2024-09-11T00:08:50.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:08:49 smithi165 ceph-mon[35356]: pgmap v451: 129 pgs: 129 active+clean; 38 MiB data, 514 MiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 2.2 MiB/s wr, 125 op/s 2024-09-11T00:08:50.080 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:08:49 smithi178 ceph-mon[35919]: pgmap v451: 129 pgs: 129 active+clean; 38 MiB data, 514 MiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 2.2 MiB/s wr, 125 op/s 2024-09-11T00:08:50.595 INFO:tasks.workunit.client.0.smithi012.stdout: 1 85900 10.22 MB/sec warmup 47 sec latency 972.781 ms 2024-09-11T00:08:50.776 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'b4f324a8', quiesce took 5.2 seconds. Will release after: 15.3 2024-09-11T00:08:51.595 INFO:tasks.workunit.client.0.smithi012.stdout: 1 85900 10.00 MB/sec warmup 48 sec latency 1972.829 ms 2024-09-11T00:08:51.945 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:08:51 smithi012 ceph-mon[30333]: pgmap v452: 129 pgs: 129 active+clean; 39 MiB data, 516 MiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 2.3 MiB/s wr, 167 op/s 2024-09-11T00:08:52.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:08:51 smithi165 ceph-mon[35356]: pgmap v452: 129 pgs: 129 active+clean; 39 MiB data, 516 MiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 2.3 MiB/s wr, 167 op/s 2024-09-11T00:08:52.080 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:08:51 smithi178 ceph-mon[35919]: pgmap v452: 129 pgs: 129 active+clean; 39 MiB data, 516 MiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 2.3 MiB/s wr, 167 op/s 2024-09-11T00:08:52.595 INFO:tasks.workunit.client.0.smithi012.stdout: 1 85900 9.80 MB/sec warmup 49 sec latency 2972.953 ms 2024-09-11T00:08:53.595 INFO:tasks.workunit.client.0.smithi012.stdout: 1 85900 9.60 MB/sec warmup 50 sec latency 3973.088 ms 2024-09-11T00:08:54.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:08:53 smithi165 ceph-mon[35356]: pgmap v453: 129 pgs: 129 active+clean; 36 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 183 KiB/s rd, 2.3 MiB/s wr, 190 op/s 2024-09-11T00:08:54.063 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:08:53 smithi012 ceph-mon[30333]: pgmap v453: 129 pgs: 129 active+clean; 36 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 183 KiB/s rd, 2.3 MiB/s wr, 190 op/s 2024-09-11T00:08:54.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:08:53 smithi178 ceph-mon[35919]: pgmap v453: 129 pgs: 129 active+clean; 36 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 183 KiB/s rd, 2.3 MiB/s wr, 190 op/s 2024-09-11T00:08:54.595 INFO:tasks.workunit.client.0.smithi012.stdout: 1 85900 9.41 MB/sec warmup 51 sec latency 4973.245 ms 2024-09-11T00:08:55.595 INFO:tasks.workunit.client.0.smithi012.stdout: 1 85900 9.23 MB/sec warmup 52 sec latency 5973.326 ms 2024-09-11T00:08:55.936 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:08:55 smithi012 ceph-mon[30333]: pgmap v454: 129 pgs: 129 active+clean; 36 MiB data, 519 MiB used, 1.0 TiB / 1.0 TiB avail; 183 KiB/s rd, 2.5 MiB/s wr, 193 op/s 2024-09-11T00:08:56.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:08:55 smithi165 ceph-mon[35356]: pgmap v454: 129 pgs: 129 active+clean; 36 MiB data, 519 MiB used, 1.0 TiB / 1.0 TiB avail; 183 KiB/s rd, 2.5 MiB/s wr, 193 op/s 2024-09-11T00:08:56.080 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:08:55 smithi178 ceph-mon[35919]: pgmap v454: 129 pgs: 129 active+clean; 36 MiB data, 519 MiB used, 1.0 TiB / 1.0 TiB avail; 183 KiB/s rd, 2.5 MiB/s wr, 193 op/s 2024-09-11T00:08:56.595 INFO:tasks.workunit.client.0.smithi012.stdout: 1 85900 9.06 MB/sec warmup 53 sec latency 6973.370 ms 2024-09-11T00:08:57.595 INFO:tasks.workunit.client.0.smithi012.stdout: 1 85900 8.89 MB/sec warmup 54 sec latency 7973.480 ms 2024-09-11T00:08:58.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:08:57 smithi165 ceph-mon[35356]: pgmap v455: 129 pgs: 129 active+clean; 34 MiB data, 517 MiB used, 1.0 TiB / 1.0 TiB avail; 183 KiB/s rd, 2.6 MiB/s wr, 175 op/s 2024-09-11T00:08:58.062 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:08:57 smithi012 ceph-mon[30333]: pgmap v455: 129 pgs: 129 active+clean; 34 MiB data, 517 MiB used, 1.0 TiB / 1.0 TiB avail; 183 KiB/s rd, 2.6 MiB/s wr, 175 op/s 2024-09-11T00:08:58.072 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:58.070+0000 7fb4af200640 1 -- 172.21.15.12:0/1854520362 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb4b0103900 msgr2=0x7fb4b0103d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:08:58.072 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:58.070+0000 7fb4af200640 1 --2- 172.21.15.12:0/1854520362 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb4b0103900 0x7fb4b0103d80 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fb49c009890 tx=0x7fb49c02a370 comp rx=0 tx=0).stop 2024-09-11T00:08:58.072 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:58.070+0000 7fb4af200640 1 -- 172.21.15.12:0/1854520362 shutdown_connections 2024-09-11T00:08:58.072 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:58.070+0000 7fb4af200640 1 --2- 172.21.15.12:0/1854520362 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb4b01042c0 0x7fb4b010ac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:08:58.072 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:58.070+0000 7fb4af200640 1 --2- 172.21.15.12:0/1854520362 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb4b0103900 0x7fb4b0103d80 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:08:58.072 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:58.070+0000 7fb4af200640 1 --2- 172.21.15.12:0/1854520362 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb4b01027b0 0x7fb4b0102bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:08:58.072 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:58.070+0000 7fb4af200640 1 -- 172.21.15.12:0/1854520362 >> 172.21.15.12:0/1854520362 conn(0x7fb4b00fe050 msgr2=0x7fb4b0100490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:08:58.072 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:58.070+0000 7fb4af200640 1 -- 172.21.15.12:0/1854520362 shutdown_connections 2024-09-11T00:08:58.072 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:58.070+0000 7fb4af200640 1 -- 172.21.15.12:0/1854520362 wait complete. 2024-09-11T00:08:58.074 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:58.071+0000 7fb4af200640 1 Processor -- start 2024-09-11T00:08:58.074 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:58.071+0000 7fb4af200640 1 -- start start 2024-09-11T00:08:58.074 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:58.071+0000 7fb4af200640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb4b01027b0 0x7fb4b019add0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:08:58.074 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:58.071+0000 7fb4af200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb4b0103900 0x7fb4b019b420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:08:58.074 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:58.071+0000 7fb4af200640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb4b01042c0 0x7fb4b019c050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:08:58.074 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:58.071+0000 7fb4af200640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fb4b010e400 con 0x7fb4b01027b0 2024-09-11T00:08:58.074 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:58.071+0000 7fb4af200640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fb4b010e280 con 0x7fb4b0103900 2024-09-11T00:08:58.074 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:58.071+0000 7fb4af200640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fb4b010e580 con 0x7fb4b01042c0 2024-09-11T00:08:58.075 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:58.072+0000 7fb4ade00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb4b01027b0 0x7fb4b019add0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:08:58.075 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:58.072+0000 7fb4ade00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb4b01027b0 0x7fb4b019add0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:45808/0 (socket says 172.21.15.12:45808) 2024-09-11T00:08:58.075 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:58.072+0000 7fb4ade00640 1 -- 172.21.15.12:0/670401076 learned_addr learned my addr 172.21.15.12:0/670401076 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:08:58.075 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:58.072+0000 7fb4ad400640 1 --2- 172.21.15.12:0/670401076 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb4b0103900 0x7fb4b019b420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:08:58.075 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:58.072+0000 7fb4ae800640 1 --2- 172.21.15.12:0/670401076 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb4b01042c0 0x7fb4b019c050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:08:58.075 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:58.072+0000 7fb4ade00640 1 -- 172.21.15.12:0/670401076 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb4b01042c0 msgr2=0x7fb4b019c050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:08:58.075 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:58.072+0000 7fb4ade00640 1 --2- 172.21.15.12:0/670401076 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb4b01042c0 0x7fb4b019c050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:08:58.075 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:58.072+0000 7fb4ade00640 1 -- 172.21.15.12:0/670401076 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb4b0103900 msgr2=0x7fb4b019b420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:08:58.075 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:58.072+0000 7fb4ade00640 1 --2- 172.21.15.12:0/670401076 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb4b0103900 0x7fb4b019b420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:08:58.075 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:58.072+0000 7fb4ade00640 1 -- 172.21.15.12:0/670401076 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4b019c910 con 0x7fb4b01027b0 2024-09-11T00:08:58.075 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:58.072+0000 7fb4ade00640 1 --2- 172.21.15.12:0/670401076 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb4b01027b0 0x7fb4b019add0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fb4a000e9c0 tx=0x7fb4a000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:08:58.076 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:58.073+0000 7fb4a6a00640 1 -- 172.21.15.12:0/670401076 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4a000cde0 con 0x7fb4b01027b0 2024-09-11T00:08:58.076 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:58.073+0000 7fb4af200640 1 -- 172.21.15.12:0/670401076 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb4b019cc00 con 0x7fb4b01027b0 2024-09-11T00:08:58.076 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:58.073+0000 7fb4a6a00640 1 -- 172.21.15.12:0/670401076 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb4a00035f0 con 0x7fb4b01027b0 2024-09-11T00:08:58.076 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:58.073+0000 7fb4a6a00640 1 -- 172.21.15.12:0/670401076 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4a0010690 con 0x7fb4b01027b0 2024-09-11T00:08:58.076 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:58.073+0000 7fb4af200640 1 -- 172.21.15.12:0/670401076 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4b01a32d0 con 0x7fb4b01027b0 2024-09-11T00:08:58.077 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:58.076+0000 7fb4b79d5740 1 Processor -- start 2024-09-11T00:08:58.078 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:58.076+0000 7fb4b79d5740 1 -- start start 2024-09-11T00:08:58.078 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:58.076+0000 7fb4b79d5740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b6ffc8e970 0x55b6ffc90d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:08:58.078 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:58.076+0000 7fb4a6a00640 1 -- 172.21.15.12:0/670401076 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7fb4a0004000 con 0x7fb4b01027b0 2024-09-11T00:08:58.078 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:58.076+0000 7fb4b79d5740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b6ffc912a0 0x55b6ffc93690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:08:58.079 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:58.076+0000 7fb4b79d5740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b6ffc93ce0 0x55b6ffc961a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:08:58.079 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:58.076+0000 7fb4b79d5740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55b6ffc016b0 con 0x55b6ffc8e970 2024-09-11T00:08:58.079 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:58.076+0000 7fb4b79d5740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55b6ffbf0ae0 con 0x55b6ffc93ce0 2024-09-11T00:08:58.079 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:58.076+0000 7fb4b79d5740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55b6ffbf48f0 con 0x55b6ffc912a0 2024-09-11T00:08:58.079 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:58.076+0000 7fb4ae800640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b6ffc8e970 0x55b6ffc90d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:08:58.079 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:58.076+0000 7fb4ae800640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b6ffc8e970 0x55b6ffc90d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:45814/0 (socket says 172.21.15.12:45814) 2024-09-11T00:08:58.079 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:58.076+0000 7fb4ae800640 1 -- 172.21.15.12:0/3408123507 learned_addr learned my addr 172.21.15.12:0/3408123507 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:08:58.079 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:58.077+0000 7fb4ade00640 1 --2- 172.21.15.12:0/3408123507 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b6ffc912a0 0x55b6ffc93690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:08:58.079 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:58.077+0000 7fb4ad400640 1 --2- 172.21.15.12:0/3408123507 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b6ffc93ce0 0x55b6ffc961a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:08:58.080 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:58.077+0000 7fb4ae800640 1 -- 172.21.15.12:0/3408123507 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b6ffc912a0 msgr2=0x55b6ffc93690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:08:58.080 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:58.077+0000 7fb4ae800640 1 --2- 172.21.15.12:0/3408123507 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b6ffc912a0 0x55b6ffc93690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:08:58.080 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:58.077+0000 7fb4ae800640 1 -- 172.21.15.12:0/3408123507 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b6ffc93ce0 msgr2=0x55b6ffc961a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:08:58.080 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:58.077+0000 7fb4ae800640 1 --2- 172.21.15.12:0/3408123507 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b6ffc93ce0 0x55b6ffc961a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:08:58.080 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:58.077+0000 7fb4ae800640 1 -- 172.21.15.12:0/3408123507 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b6ffbbd780 con 0x55b6ffc8e970 2024-09-11T00:08:58.080 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:58.077+0000 7fb4ade00640 1 --2- 172.21.15.12:0/3408123507 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b6ffc912a0 0x55b6ffc93690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T00:08:58.080 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:58.078+0000 7fb4ae800640 1 --2- 172.21.15.12:0/3408123507 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b6ffc8e970 0x55b6ffc90d60 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fb4a8009f90 tx=0x7fb4a80118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be28de9961bbb51e server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:08:58.081 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:58.078+0000 7fb49b000640 1 -- 172.21.15.12:0/3408123507 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4a8004d20 con 0x55b6ffc8e970 2024-09-11T00:08:58.081 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:58.078+0000 7fb49b000640 1 -- 172.21.15.12:0/3408123507 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb4a8004050 con 0x55b6ffc8e970 2024-09-11T00:08:58.081 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:58.078+0000 7fb49b000640 1 -- 172.21.15.12:0/3408123507 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4a8005280 con 0x55b6ffc8e970 2024-09-11T00:08:58.081 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:58.078+0000 7fb4a6a00640 1 --2- 172.21.15.12:0/670401076 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fb4940b52e0 0x7fb4940b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:08:58.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:08:57 smithi178 ceph-mon[35919]: pgmap v455: 129 pgs: 129 active+clean; 34 MiB data, 517 MiB used, 1.0 TiB / 1.0 TiB avail; 183 KiB/s rd, 2.6 MiB/s wr, 175 op/s 2024-09-11T00:08:58.084 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:58.079+0000 7fb4a6a00640 1 -- 172.21.15.12:0/670401076 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9208+0+0 (secure 0 0 0) 0x7fb4a00e6310 con 0x7fb4b01027b0 2024-09-11T00:08:58.084 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:58.079+0000 7fb4ade00640 1 --2- 172.21.15.12:0/670401076 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fb4940b52e0 0x7fb4940b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:08:58.084 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:58.079+0000 7fb4b79d5740 1 -- 172.21.15.12:0/3408123507 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b6ffc8e970 msgr2=0x55b6ffc90d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:08:58.084 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:58.079+0000 7fb4b79d5740 1 --2- 172.21.15.12:0/3408123507 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b6ffc8e970 0x55b6ffc90d60 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fb4a8009f90 tx=0x7fb4a80118d0 comp rx=0 tx=0).stop 2024-09-11T00:08:58.084 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:58.079+0000 7fb4b79d5740 1 -- 172.21.15.12:0/3408123507 shutdown_connections 2024-09-11T00:08:58.084 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:58.079+0000 7fb4b79d5740 1 --2- 172.21.15.12:0/3408123507 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b6ffc93ce0 0x55b6ffc961a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:08:58.084 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:58.079+0000 7fb4b79d5740 1 --2- 172.21.15.12:0/3408123507 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b6ffc912a0 0x55b6ffc93690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:08:58.084 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:58.079+0000 7fb4b79d5740 1 --2- 172.21.15.12:0/3408123507 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b6ffc8e970 0x55b6ffc90d60 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:08:58.084 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:58.079+0000 7fb4b79d5740 1 -- 172.21.15.12:0/3408123507 >> 172.21.15.12:0/3408123507 conn(0x55b6ffc89ac0 msgr2=0x55b6ffc8a850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:08:58.084 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:58.079+0000 7fb4b79d5740 1 -- 172.21.15.12:0/3408123507 shutdown_connections 2024-09-11T00:08:58.085 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:58.079+0000 7fb4b79d5740 1 -- 172.21.15.12:0/3408123507 wait complete. 2024-09-11T00:08:58.085 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:58.079+0000 7fb4ade00640 1 --2- 172.21.15.12:0/670401076 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fb4940b52e0 0x7fb4940b77a0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fb4a00ac630 tx=0x7fb4a000c870 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:08:58.085 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:58.080+0000 7fb4b79d5740 1 -- start start 2024-09-11T00:08:58.085 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:58.080+0000 7fb4b79d5740 1 Processor -- start 2024-09-11T00:08:58.085 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:58.080+0000 7fb4b79d5740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b6ffc8e970 0x55b6ffc980b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:08:58.085 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:58.080+0000 7fb4b79d5740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b6ffc912a0 0x55b6ffc98750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:08:58.085 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:58.080+0000 7fb4b79d5740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b6ffc93ce0 0x55b6ffc99320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:08:58.085 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:58.080+0000 7fb4b79d5740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55b6ffb28ab0 con 0x55b6ffc93ce0 2024-09-11T00:08:58.085 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:58.080+0000 7fb4b79d5740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55b6ffb97570 con 0x55b6ffc912a0 2024-09-11T00:08:58.085 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:58.080+0000 7fb4b79d5740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55b6ffc96cc0 con 0x55b6ffc8e970 2024-09-11T00:08:58.085 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:58.080+0000 7fb4b79d5740 10 client.0 _ll_register_callbacks cb 0x55b6ffba6100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:08:58.085 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:58.081+0000 7fb4ae800640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b6ffc93ce0 0x55b6ffc99320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:08:58.085 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:58.081+0000 7fb4ad400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b6ffc912a0 0x55b6ffc98750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:08:58.086 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:58.081+0000 7fb4ae800640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b6ffc8e970 0x55b6ffc980b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:08:58.086 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:58.081+0000 7fb4ad400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b6ffc912a0 0x55b6ffc98750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.12:51474/0 (socket says 172.21.15.12:51474) 2024-09-11T00:08:58.086 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:58.081+0000 7fb4ad400640 1 -- 172.21.15.12:0/2835506519 learned_addr learned my addr 172.21.15.12:0/2835506519 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:08:58.086 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:58.081+0000 7fb4ad400640 1 -- 172.21.15.12:0/2835506519 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b6ffc8e970 msgr2=0x55b6ffc980b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:08:58.086 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:58.081+0000 7fb4ad400640 1 --2- 172.21.15.12:0/2835506519 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b6ffc8e970 0x55b6ffc980b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:08:58.086 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:58.081+0000 7fb4ad400640 1 -- 172.21.15.12:0/2835506519 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b6ffc93ce0 msgr2=0x55b6ffc99320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:08:58.086 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:58.081+0000 7fb4ad400640 1 --2- 172.21.15.12:0/2835506519 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b6ffc93ce0 0x55b6ffc99320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:08:58.086 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:58.081+0000 7fb4ad400640 1 -- 172.21.15.12:0/2835506519 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b6ff952790 con 0x55b6ffc912a0 2024-09-11T00:08:58.086 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:58.082+0000 7fb4ad400640 1 --2- 172.21.15.12:0/2835506519 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b6ffc912a0 0x55b6ffc98750 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fb49c02a890 tx=0x7fb49c0097f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d620a11a97e28d4a server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:08:58.086 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:58.082+0000 7fb48f400640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-09-11T00:08:58.086 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:58.082+0000 7fb48f400640 1 -- 172.21.15.12:0/2835506519 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb49c037040 con 0x55b6ffc912a0 2024-09-11T00:08:58.086 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:58.082+0000 7fb4b79d5740 1 -- 172.21.15.12:0/2835506519 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb4a8004050 con 0x55b6ffc912a0 2024-09-11T00:08:58.086 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:58.082+0000 7fb48f400640 1 -- 172.21.15.12:0/2835506519 <== mon.2 v2:172.21.15.165:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9208+0+0 (secure 0 0 0) 0x7fb49c004240 con 0x55b6ffc912a0 2024-09-11T00:08:58.088 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:58.082+0000 7fb48f400640 1 -- 172.21.15.12:0/2835506519 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fb49c004440 con 0x55b6ffc912a0 2024-09-11T00:08:58.089 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:58.082+0000 7fb4b79d5740 10 client.25721 fetch_fsmap learned FSMap version 33 2024-09-11T00:08:58.089 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:58.082+0000 7fb4b79d5740 1 -- 172.21.15.12:0/2835506519 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55b6ff9cb620 con 0x55b6ffc912a0 2024-09-11T00:08:58.089 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:58.083+0000 7fb48f400640 1 -- 172.21.15.12:0/2835506519 <== mon.2 v2:172.21.15.165:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb4a8004050 con 0x55b6ffc912a0 2024-09-11T00:08:58.089 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:58.083+0000 7fb48f400640 1 -- 172.21.15.12:0/2835506519 <== mon.2 v2:172.21.15.165:3300/0 5 ==== fsmap(e 33) ==== 8386+0+0 (secure 0 0 0) 0x7fb49c03d040 con 0x55b6ffc912a0 2024-09-11T00:08:58.089 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:58.083+0000 7fb4b79d5740 10 client.25721 fetch_fsmap finished waiting for FSMap version 33 2024-09-11T00:08:58.089 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:58.083+0000 7fb4b79d5740 10 client.25721 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T00:08:58.089 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:58.083+0000 7fb4b79d5740 20 client.25721 populate_metadata read hostname 'smithi012' 2024-09-11T00:08:58.089 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:58.083+0000 7fb4b79d5740 1 --2- 172.21.15.12:0/2835506519 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55b6ffca3aa0 0x55b6ffca5ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:08:58.089 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:58.083+0000 7fb4b79d5740 4 client.25721 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T00:08:58.089 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:58.083+0000 7fb4b79d5740 1 -- 172.21.15.12:0/2835506519 --> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b6ffc03600 con 0x55b6ffca3aa0 2024-09-11T00:08:58.089 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:58.083+0000 7fb4ae800640 1 --2- 172.21.15.12:0/2835506519 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55b6ffca3aa0 0x55b6ffca5ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:08:58.089 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:58.084+0000 7fb4ae800640 1 --2- 172.21.15.12:0/2835506519 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55b6ffca3aa0 0x55b6ffca5ee0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ff3e6b10b8d87dde server_cookie=1dd2f2db914127d7 in_seq=0 out_seq=0 2024-09-11T00:08:58.090 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:58.084+0000 7fb48f400640 10 client.25721 ms_handle_connect on v2:172.21.15.165:6838/501312984 2024-09-11T00:08:58.090 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:58.085+0000 7fb48f400640 1 -- 172.21.15.12:0/2835506519 <== mds.0 v2:172.21.15.165:6838/501312984 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55b6ffc03600 con 0x55b6ffca3aa0 2024-09-11T00:08:58.090 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:58.085+0000 7fb48f400640 10 client.25721 handle_command_reply: tid=0 2024-09-11T00:08:58.090 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:58.085+0000 7fb4b79d5740 1 client.25721 shutdown 2024-09-11T00:08:58.090 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:58.085+0000 7fb4b79d5740 1 -- 172.21.15.12:0/2835506519 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b6ffc912a0 msgr2=0x55b6ffc98750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:08:58.090 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:58.085+0000 7fb4b79d5740 1 --2- 172.21.15.12:0/2835506519 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b6ffc912a0 0x55b6ffc98750 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fb49c02a890 tx=0x7fb49c0097f0 comp rx=0 tx=0).stop 2024-09-11T00:08:58.090 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:58.085+0000 7fb4b79d5740 1 -- 172.21.15.12:0/2835506519 shutdown_connections 2024-09-11T00:08:58.090 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:58.085+0000 7fb4b79d5740 1 --2- 172.21.15.12:0/2835506519 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55b6ffca3aa0 0x55b6ffca5ee0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:08:58.090 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:58.085+0000 7fb4b79d5740 1 --2- 172.21.15.12:0/2835506519 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b6ffc93ce0 0x55b6ffc99320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:08:58.090 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:58.085+0000 7fb4b79d5740 1 --2- 172.21.15.12:0/2835506519 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b6ffc912a0 0x55b6ffc98750 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:08:58.090 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:58.085+0000 7fb48f400640 0 client.25721 ms_handle_reset on v2:172.21.15.165:6838/501312984 2024-09-11T00:08:58.090 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:58.085+0000 7fb4b79d5740 1 --2- 172.21.15.12:0/2835506519 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b6ffc8e970 0x55b6ffc980b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:08:58.090 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:58.085+0000 7fb4b79d5740 1 -- 172.21.15.12:0/2835506519 >> 172.21.15.12:0/2835506519 conn(0x55b6ffc89ac0 msgr2=0x55b6ffc88bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:08:58.091 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:58.085+0000 7fb4b79d5740 1 -- 172.21.15.12:0/2835506519 shutdown_connections 2024-09-11T00:08:58.091 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:58.085+0000 7fb4b79d5740 1 -- 172.21.15.12:0/2835506519 wait complete. 2024-09-11T00:08:58.091 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:58.086+0000 7fb4b79d5740 20 client.25721 trim_cache size 0 max 16384 2024-09-11T00:08:58.104 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:58.101+0000 7fb4b79d5740 1 Processor -- start 2024-09-11T00:08:58.104 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:58.101+0000 7fb4b79d5740 1 -- start start 2024-09-11T00:08:58.104 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:58.101+0000 7fb4b79d5740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b6ffc8e970 0x55b6ffc940c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:08:58.104 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:58.101+0000 7fb4b79d5740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b6ffc912a0 0x55b6ffc94600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:08:58.104 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:58.101+0000 7fb4b79d5740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b6ffc93ce0 0x55b6ffc970c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:08:58.104 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:58.101+0000 7fb4b79d5740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fb4b01ad7f0 con 0x55b6ffc8e970 2024-09-11T00:08:58.104 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:58.101+0000 7fb4b79d5740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fb4b01af3f0 con 0x55b6ffc912a0 2024-09-11T00:08:58.104 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:58.101+0000 7fb4b79d5740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fb4b01adec0 con 0x55b6ffc93ce0 2024-09-11T00:08:58.104 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:58.102+0000 7fb4ae800640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b6ffc8e970 0x55b6ffc940c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:08:58.104 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:58.102+0000 7fb4ae800640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b6ffc8e970 0x55b6ffc940c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:45842/0 (socket says 172.21.15.12:45842) 2024-09-11T00:08:58.104 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:58.102+0000 7fb4ae800640 1 -- 172.21.15.12:0/2880806734 learned_addr learned my addr 172.21.15.12:0/2880806734 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:08:58.105 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:58.102+0000 7fb4ad400640 1 --2- 172.21.15.12:0/2880806734 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b6ffc912a0 0x55b6ffc94600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:08:58.105 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:58.102+0000 7fb4ae800640 1 --2- 172.21.15.12:0/2880806734 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b6ffc93ce0 0x55b6ffc970c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:08:58.105 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:58.102+0000 7fb4ae800640 1 -- 172.21.15.12:0/2880806734 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b6ffc93ce0 msgr2=0x55b6ffc970c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:08:58.105 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:58.102+0000 7fb4ae800640 1 --2- 172.21.15.12:0/2880806734 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b6ffc93ce0 0x55b6ffc970c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:08:58.105 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:58.102+0000 7fb4ae800640 1 -- 172.21.15.12:0/2880806734 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b6ffc912a0 msgr2=0x55b6ffc94600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:08:58.105 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:58.102+0000 7fb4ae800640 1 --2- 172.21.15.12:0/2880806734 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b6ffc912a0 0x55b6ffc94600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:08:58.105 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:58.102+0000 7fb4ae800640 1 -- 172.21.15.12:0/2880806734 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b6ffc910d0 con 0x55b6ffc8e970 2024-09-11T00:08:58.108 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:58.102+0000 7fb4ae800640 1 --2- 172.21.15.12:0/2880806734 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b6ffc8e970 0x55b6ffc940c0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fb4a8006010 tx=0x7fb4a8009d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f919345867163cbb server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:08:58.108 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:58.102+0000 7fb49a600640 1 -- 172.21.15.12:0/2880806734 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4a802f030 con 0x55b6ffc8e970 2024-09-11T00:08:58.108 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:58.103+0000 7fb49a600640 1 -- 172.21.15.12:0/2880806734 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb4a8004570 con 0x55b6ffc8e970 2024-09-11T00:08:58.108 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:58.103+0000 7fb49a600640 1 -- 172.21.15.12:0/2880806734 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4a800fbc0 con 0x55b6ffc8e970 2024-09-11T00:08:58.108 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:58.103+0000 7fb4b79d5740 1 -- 172.21.15.12:0/2880806734 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b6ffc8e970 msgr2=0x55b6ffc940c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:08:58.109 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:58.103+0000 7fb4b79d5740 1 --2- 172.21.15.12:0/2880806734 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b6ffc8e970 0x55b6ffc940c0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fb4a8006010 tx=0x7fb4a8009d00 comp rx=0 tx=0).stop 2024-09-11T00:08:58.109 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:58.103+0000 7fb4b79d5740 1 -- 172.21.15.12:0/2880806734 shutdown_connections 2024-09-11T00:08:58.109 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:58.103+0000 7fb4b79d5740 1 --2- 172.21.15.12:0/2880806734 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b6ffc93ce0 0x55b6ffc970c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:08:58.109 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:58.103+0000 7fb4b79d5740 1 --2- 172.21.15.12:0/2880806734 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b6ffc912a0 0x55b6ffc94600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:08:58.109 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:58.103+0000 7fb4b79d5740 1 --2- 172.21.15.12:0/2880806734 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b6ffc8e970 0x55b6ffc940c0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:08:58.109 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:58.103+0000 7fb4b79d5740 1 -- 172.21.15.12:0/2880806734 >> 172.21.15.12:0/2880806734 conn(0x55b6ffc89ac0 msgr2=0x55b6ffc980b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:08:58.109 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:58.103+0000 7fb4b79d5740 1 -- 172.21.15.12:0/2880806734 shutdown_connections 2024-09-11T00:08:58.109 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:58.103+0000 7fb4b79d5740 1 -- 172.21.15.12:0/2880806734 wait complete. 2024-09-11T00:08:58.109 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:58.104+0000 7fb4b79d5740 1 -- start start 2024-09-11T00:08:58.109 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:58.104+0000 7fb4b79d5740 1 Processor -- start 2024-09-11T00:08:58.109 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:58.104+0000 7fb4b79d5740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b6ffc8e970 0x55b6ffc9ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:08:58.109 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:58.104+0000 7fb4b79d5740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b6ffc912a0 0x55b6ffc9f1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:08:58.110 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:58.104+0000 7fb4b79d5740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b6ffc93ce0 0x55b6ffc9f730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:08:58.110 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:58.104+0000 7fb4b79d5740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fb4b01aed20 con 0x55b6ffc93ce0 2024-09-11T00:08:58.110 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:58.104+0000 7fb4b79d5740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fb4b01ae590 con 0x55b6ffc912a0 2024-09-11T00:08:58.110 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:58.104+0000 7fb4b79d5740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fb4b01ab7d0 con 0x55b6ffc8e970 2024-09-11T00:08:58.110 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:58.104+0000 7fb4b79d5740 10 client.0 _ll_register_callbacks cb 0x7fb4a0005ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:08:58.110 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:58.104+0000 7fb4ae800640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b6ffc8e970 0x55b6ffc9ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:08:58.110 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:58.105+0000 7fb4ae800640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b6ffc8e970 0x55b6ffc9ecb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.12:54850/0 (socket says 172.21.15.12:54850) 2024-09-11T00:08:58.110 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:58.105+0000 7fb4ae800640 1 -- 172.21.15.12:0/4111316093 learned_addr learned my addr 172.21.15.12:0/4111316093 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:08:58.110 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:58.105+0000 7fb4ad400640 1 --2- 172.21.15.12:0/4111316093 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b6ffc912a0 0x55b6ffc9f1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:08:58.110 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:58.105+0000 7fb4ae800640 1 --2- 172.21.15.12:0/4111316093 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b6ffc93ce0 0x55b6ffc9f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:08:58.110 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:58.105+0000 7fb4ad400640 1 -- 172.21.15.12:0/4111316093 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b6ffc8e970 msgr2=0x55b6ffc9ecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:08:58.110 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:58.105+0000 7fb4ad400640 1 --2- 172.21.15.12:0/4111316093 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b6ffc8e970 0x55b6ffc9ecb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:08:58.110 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:58.105+0000 7fb4ad400640 1 -- 172.21.15.12:0/4111316093 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b6ffc93ce0 msgr2=0x55b6ffc9f730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:08:58.110 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:58.105+0000 7fb4ad400640 1 --2- 172.21.15.12:0/4111316093 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b6ffc93ce0 0x55b6ffc9f730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:08:58.111 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:58.105+0000 7fb4ad400640 1 -- 172.21.15.12:0/4111316093 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b6ffc9d930 con 0x55b6ffc912a0 2024-09-11T00:08:58.111 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:58.105+0000 7fb4ae800640 1 --2- 172.21.15.12:0/4111316093 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b6ffc8e970 0x55b6ffc9ecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:08:58.111 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:58.105+0000 7fb4ad400640 1 --2- 172.21.15.12:0/4111316093 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b6ffc912a0 0x55b6ffc9f1f0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fb49c032720 tx=0x7fb49c0307b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=755da93fdab06d22 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:08:58.111 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:58.105+0000 7fb48f400640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-09-11T00:08:58.111 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:58.106+0000 7fb48f400640 1 -- 172.21.15.12:0/4111316093 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb49c037040 con 0x55b6ffc912a0 2024-09-11T00:08:58.111 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:58.106+0000 7fb4b79d5740 1 -- 172.21.15.12:0/4111316093 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb4a8004570 con 0x55b6ffc912a0 2024-09-11T00:08:58.111 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:58.106+0000 7fb48f400640 1 -- 172.21.15.12:0/4111316093 <== mon.2 v2:172.21.15.165:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9208+0+0 (secure 0 0 0) 0x7fb49c005450 con 0x55b6ffc912a0 2024-09-11T00:08:58.111 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:58.106+0000 7fb48f400640 1 -- 172.21.15.12:0/4111316093 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fb49c009450 con 0x55b6ffc912a0 2024-09-11T00:08:58.111 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:58.106+0000 7fb4b79d5740 10 client.25727 fetch_fsmap learned FSMap version 33 2024-09-11T00:08:58.111 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:58.106+0000 7fb4b79d5740 1 -- 172.21.15.12:0/4111316093 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55b6ffd10ed0 con 0x55b6ffc912a0 2024-09-11T00:08:58.111 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:58.106+0000 7fb48f400640 1 -- 172.21.15.12:0/4111316093 <== mon.2 v2:172.21.15.165:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb4a8004570 con 0x55b6ffc912a0 2024-09-11T00:08:58.111 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:58.106+0000 7fb48f400640 1 -- 172.21.15.12:0/4111316093 <== mon.2 v2:172.21.15.165:3300/0 5 ==== fsmap(e 33) ==== 8386+0+0 (secure 0 0 0) 0x7fb49c03d070 con 0x55b6ffc912a0 2024-09-11T00:08:58.111 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:58.106+0000 7fb4b79d5740 10 client.25727 fetch_fsmap finished waiting for FSMap version 33 2024-09-11T00:08:58.112 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:58.106+0000 7fb4b79d5740 10 client.25727 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T00:08:58.112 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:58.106+0000 7fb4b79d5740 20 client.25727 populate_metadata read hostname 'smithi012' 2024-09-11T00:08:58.112 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:58.106+0000 7fb4b79d5740 1 --2- 172.21.15.12:0/4111316093 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55b6ffd117c0 0x55b6ffd13be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:08:58.112 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:58.107+0000 7fb4b79d5740 4 client.25727 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T00:08:58.112 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:58.107+0000 7fb4b79d5740 1 -- 172.21.15.12:0/4111316093 --> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55b6ffbff0b0 con 0x55b6ffd117c0 2024-09-11T00:08:58.112 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:58.107+0000 7fb4ae800640 1 --2- 172.21.15.12:0/4111316093 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55b6ffd117c0 0x55b6ffd13be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:08:58.112 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:58.108+0000 7fb4ae800640 1 --2- 172.21.15.12:0/4111316093 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55b6ffd117c0 0x55b6ffd13be0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4840aa6ff9e9e516 server_cookie=e52484e3ec92d316 in_seq=0 out_seq=0 2024-09-11T00:08:58.113 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:58.108+0000 7fb48f400640 10 client.25727 ms_handle_connect on v2:172.21.15.165:6838/501312984 2024-09-11T00:08:58.143 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:58.140+0000 7fb48f400640 1 -- 172.21.15.12:0/4111316093 <== mds.0 v2:172.21.15.165:6838/501312984 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55b6ffbff0b0 con 0x55b6ffd117c0 2024-09-11T00:08:58.143 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:58.140+0000 7fb48f400640 10 client.25727 handle_command_reply: tid=0 2024-09-11T00:08:58.143 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:58.141+0000 7fb4b79d5740 1 client.25727 shutdown 2024-09-11T00:08:58.143 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:58.141+0000 7fb4b79d5740 1 -- 172.21.15.12:0/4111316093 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b6ffc912a0 msgr2=0x55b6ffc9f1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:08:58.143 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:58.141+0000 7fb4b79d5740 1 --2- 172.21.15.12:0/4111316093 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b6ffc912a0 0x55b6ffc9f1f0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fb49c032720 tx=0x7fb49c0307b0 comp rx=0 tx=0).stop 2024-09-11T00:08:58.143 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:58.141+0000 7fb4b79d5740 1 -- 172.21.15.12:0/4111316093 shutdown_connections 2024-09-11T00:08:58.143 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:58.141+0000 7fb4b79d5740 1 --2- 172.21.15.12:0/4111316093 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55b6ffd117c0 0x55b6ffd13be0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:08:58.143 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:58.141+0000 7fb4b79d5740 1 --2- 172.21.15.12:0/4111316093 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b6ffc93ce0 0x55b6ffc9f730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:08:58.144 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:58.141+0000 7fb4b79d5740 1 --2- 172.21.15.12:0/4111316093 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b6ffc912a0 0x55b6ffc9f1f0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:08:58.144 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:58.141+0000 7fb4b79d5740 1 --2- 172.21.15.12:0/4111316093 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b6ffc8e970 0x55b6ffc9ecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:08:58.144 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:58.141+0000 7fb48f400640 0 client.25727 ms_handle_reset on v2:172.21.15.165:6838/501312984 2024-09-11T00:08:58.144 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:58.141+0000 7fb4b79d5740 1 -- 172.21.15.12:0/4111316093 >> 172.21.15.12:0/4111316093 conn(0x55b6ffc89ac0 msgr2=0x55b6ffc940c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:08:58.144 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:58.141+0000 7fb4b79d5740 1 -- 172.21.15.12:0/4111316093 shutdown_connections 2024-09-11T00:08:58.144 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:58.141+0000 7fb4b79d5740 1 -- 172.21.15.12:0/4111316093 wait complete. 2024-09-11T00:08:58.144 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:58.141+0000 7fb4b79d5740 20 client.25727 trim_cache size 0 max 16384 2024-09-11T00:08:58.144 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:{ 2024-09-11T00:08:58.144 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "message": "", 2024-09-11T00:08:58.145 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "return_code": 0 2024-09-11T00:08:58.145 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:} 2024-09-11T00:08:58.145 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:58.142+0000 7fb4af200640 1 -- 172.21.15.12:0/670401076 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fb4940b52e0 msgr2=0x7fb4940b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:08:58.145 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:58.142+0000 7fb4af200640 1 --2- 172.21.15.12:0/670401076 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fb4940b52e0 0x7fb4940b77a0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fb4a00ac630 tx=0x7fb4a000c870 comp rx=0 tx=0).stop 2024-09-11T00:08:58.145 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:58.142+0000 7fb4af200640 1 -- 172.21.15.12:0/670401076 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb4b01027b0 msgr2=0x7fb4b019add0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:08:58.146 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:58.142+0000 7fb4af200640 1 --2- 172.21.15.12:0/670401076 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb4b01027b0 0x7fb4b019add0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fb4a000e9c0 tx=0x7fb4a000ee80 comp rx=0 tx=0).stop 2024-09-11T00:08:58.146 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:58.142+0000 7fb4af200640 1 -- 172.21.15.12:0/670401076 shutdown_connections 2024-09-11T00:08:58.146 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:58.142+0000 7fb4af200640 1 --2- 172.21.15.12:0/670401076 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fb4940b52e0 0x7fb4940b77a0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:08:58.146 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:58.142+0000 7fb4af200640 1 --2- 172.21.15.12:0/670401076 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb4b01042c0 0x7fb4b019c050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:08:58.146 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:58.142+0000 7fb4af200640 1 --2- 172.21.15.12:0/670401076 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb4b0103900 0x7fb4b019b420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:08:58.146 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:58.142+0000 7fb4af200640 1 --2- 172.21.15.12:0/670401076 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb4b01027b0 0x7fb4b019add0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:08:58.146 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:58.142+0000 7fb4af200640 1 -- 172.21.15.12:0/670401076 >> 172.21.15.12:0/670401076 conn(0x7fb4b00fe050 msgr2=0x7fb4b0075380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:08:58.146 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:58.142+0000 7fb4af200640 1 -- 172.21.15.12:0/670401076 shutdown_connections 2024-09-11T00:08:58.146 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:08:58.142+0000 7fb4af200640 1 -- 172.21.15.12:0/670401076 wait complete. 2024-09-11T00:08:58.596 INFO:tasks.workunit.client.0.smithi012.stdout: 1 85900 8.73 MB/sec warmup 55 sec latency 8973.605 ms 2024-09-11T00:08:59.596 INFO:tasks.workunit.client.0.smithi012.stdout: 1 85900 8.57 MB/sec warmup 56 sec latency 9973.765 ms 2024-09-11T00:09:00.004 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:08:59 smithi012 ceph-mon[30333]: pgmap v456: 129 pgs: 129 active+clean; 34 MiB data, 516 MiB used, 1.0 TiB / 1.0 TiB avail; 203 KiB/s rd, 2.3 MiB/s wr, 155 op/s 2024-09-11T00:09:00.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:08:59 smithi165 ceph-mon[35356]: pgmap v456: 129 pgs: 129 active+clean; 34 MiB data, 516 MiB used, 1.0 TiB / 1.0 TiB avail; 203 KiB/s rd, 2.3 MiB/s wr, 155 op/s 2024-09-11T00:09:00.080 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:08:59 smithi178 ceph-mon[35919]: pgmap v456: 129 pgs: 129 active+clean; 34 MiB data, 516 MiB used, 1.0 TiB / 1.0 TiB avail; 203 KiB/s rd, 2.3 MiB/s wr, 155 op/s 2024-09-11T00:09:00.596 INFO:tasks.workunit.client.0.smithi012.stdout: 1 85900 8.42 MB/sec warmup 57 sec latency 10973.838 ms 2024-09-11T00:09:01.596 INFO:tasks.workunit.client.0.smithi012.stdout: 1 85900 8.28 MB/sec warmup 58 sec latency 11973.880 ms 2024-09-11T00:09:01.965 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:09:01 smithi012 ceph-mon[30333]: pgmap v457: 129 pgs: 129 active+clean; 32 MiB data, 518 MiB used, 1.0 TiB / 1.0 TiB avail; 180 KiB/s rd, 2.5 MiB/s wr, 186 op/s 2024-09-11T00:09:02.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:09:01 smithi165 ceph-mon[35356]: pgmap v457: 129 pgs: 129 active+clean; 32 MiB data, 518 MiB used, 1.0 TiB / 1.0 TiB avail; 180 KiB/s rd, 2.5 MiB/s wr, 186 op/s 2024-09-11T00:09:02.080 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:09:01 smithi178 ceph-mon[35919]: pgmap v457: 129 pgs: 129 active+clean; 32 MiB data, 518 MiB used, 1.0 TiB / 1.0 TiB avail; 180 KiB/s rd, 2.5 MiB/s wr, 186 op/s 2024-09-11T00:09:02.596 INFO:tasks.workunit.client.0.smithi012.stdout: 1 85900 8.14 MB/sec warmup 59 sec latency 12973.991 ms 2024-09-11T00:09:03.596 INFO:tasks.workunit.client.0.smithi012.stdout: 1 85900 8.00 MB/sec warmup 60 sec latency 13974.129 ms 2024-09-11T00:09:04.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:09:03 smithi165 ceph-mon[35356]: pgmap v458: 129 pgs: 129 active+clean; 30 MiB data, 521 MiB used, 1.0 TiB / 1.0 TiB avail; 183 KiB/s rd, 1.9 MiB/s wr, 156 op/s 2024-09-11T00:09:04.063 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:09:03 smithi012 ceph-mon[30333]: pgmap v458: 129 pgs: 129 active+clean; 30 MiB data, 521 MiB used, 1.0 TiB / 1.0 TiB avail; 183 KiB/s rd, 1.9 MiB/s wr, 156 op/s 2024-09-11T00:09:04.080 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:09:03 smithi178 ceph-mon[35919]: pgmap v458: 129 pgs: 129 active+clean; 30 MiB data, 521 MiB used, 1.0 TiB / 1.0 TiB avail; 183 KiB/s rd, 1.9 MiB/s wr, 156 op/s 2024-09-11T00:09:04.596 INFO:tasks.workunit.client.0.smithi012.stdout: 1 85900 7.87 MB/sec warmup 61 sec latency 14974.229 ms 2024-09-11T00:09:05.596 INFO:tasks.workunit.client.0.smithi012.stdout: 1 85900 7.74 MB/sec warmup 62 sec latency 15974.316 ms 2024-09-11T00:09:05.905 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:09:05 smithi012 ceph-mon[30333]: pgmap v459: 129 pgs: 129 active+clean; 29 MiB data, 521 MiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 1.7 MiB/s wr, 125 op/s 2024-09-11T00:09:06.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:09:05 smithi165 ceph-mon[35356]: pgmap v459: 129 pgs: 129 active+clean; 29 MiB data, 521 MiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 1.7 MiB/s wr, 125 op/s 2024-09-11T00:09:06.078 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid ff3ad74a-6fcf-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T00:09:06.080 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:09:05 smithi178 ceph-mon[35919]: pgmap v459: 129 pgs: 129 active+clean; 29 MiB data, 521 MiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 1.7 MiB/s wr, 125 op/s 2024-09-11T00:09:06.478 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:09:06.603 INFO:tasks.workunit.client.0.smithi012.stdout: 1 85900 7.62 MB/sec warmup 63 sec latency 16981.353 ms 2024-09-11T00:09:07.134 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:07.131+0000 7fe413800640 1 -- 172.21.15.12:0/4116070140 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe40c102da0 msgr2=0x7fe40c103180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:09:07.134 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:07.131+0000 7fe413800640 1 --2- 172.21.15.12:0/4116070140 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe40c102da0 0x7fe40c103180 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fe3f4009900 tx=0x7fe3f402a400 comp rx=0 tx=0).stop 2024-09-11T00:09:07.134 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:07.131+0000 7fe413800640 1 -- 172.21.15.12:0/4116070140 shutdown_connections 2024-09-11T00:09:07.134 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:07.131+0000 7fe413800640 1 --2- 172.21.15.12:0/4116070140 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe40c109130 0x7fe40c10f930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:09:07.134 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:07.131+0000 7fe413800640 1 --2- 172.21.15.12:0/4116070140 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe40c1087b0 0x7fe40c108bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:09:07.134 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:07.131+0000 7fe413800640 1 --2- 172.21.15.12:0/4116070140 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe40c102da0 0x7fe40c103180 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:09:07.134 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:07.131+0000 7fe413800640 1 -- 172.21.15.12:0/4116070140 >> 172.21.15.12:0/4116070140 conn(0x7fe40c0fe810 msgr2=0x7fe40c100c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:09:07.134 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:07.131+0000 7fe413800640 1 -- 172.21.15.12:0/4116070140 shutdown_connections 2024-09-11T00:09:07.135 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:07.131+0000 7fe413800640 1 -- 172.21.15.12:0/4116070140 wait complete. 2024-09-11T00:09:07.135 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:07.133+0000 7fe413800640 1 Processor -- start 2024-09-11T00:09:07.136 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:07.134+0000 7fe413800640 1 -- start start 2024-09-11T00:09:07.136 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:07.134+0000 7fe413800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe40c102da0 0x7fe40c10f650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:09:07.136 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:07.134+0000 7fe413800640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe40c1087b0 0x7fe40c10c8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:09:07.136 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:07.134+0000 7fe413800640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe40c109130 0x7fe40c10cde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:09:07.137 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:07.134+0000 7fe413800640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fe40c076eb0 con 0x7fe40c1087b0 2024-09-11T00:09:07.137 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:07.134+0000 7fe413800640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fe40c076d30 con 0x7fe40c102da0 2024-09-11T00:09:07.137 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:07.134+0000 7fe413800640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fe40c077030 con 0x7fe40c109130 2024-09-11T00:09:07.137 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:07.135+0000 7fe410e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe40c102da0 0x7fe40c10f650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:09:07.138 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:07.135+0000 7fe403e00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe40c1087b0 0x7fe40c10c8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:09:07.138 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:07.135+0000 7fe410e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe40c102da0 0x7fe40c10f650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.12:55134/0 (socket says 172.21.15.12:55134) 2024-09-11T00:09:07.138 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:07.135+0000 7fe410e00640 1 -- 172.21.15.12:0/49001865 learned_addr learned my addr 172.21.15.12:0/49001865 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:09:07.138 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:07.135+0000 7fe411800640 1 --2- 172.21.15.12:0/49001865 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe40c109130 0x7fe40c10cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:09:07.138 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:07.135+0000 7fe411800640 1 -- 172.21.15.12:0/49001865 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe40c102da0 msgr2=0x7fe40c10f650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:09:07.138 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:07.135+0000 7fe411800640 1 --2- 172.21.15.12:0/49001865 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe40c102da0 0x7fe40c10f650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:09:07.138 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:07.135+0000 7fe411800640 1 -- 172.21.15.12:0/49001865 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe40c1087b0 msgr2=0x7fe40c10c8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:09:07.138 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:07.135+0000 7fe411800640 1 --2- 172.21.15.12:0/49001865 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe40c1087b0 0x7fe40c10c8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:09:07.138 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:07.135+0000 7fe411800640 1 -- 172.21.15.12:0/49001865 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe40c10d640 con 0x7fe40c109130 2024-09-11T00:09:07.138 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:07.136+0000 7fe403e00640 1 --2- 172.21.15.12:0/49001865 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe40c1087b0 0x7fe40c10c8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T00:09:07.139 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:07.136+0000 7fe411800640 1 --2- 172.21.15.12:0/49001865 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe40c109130 0x7fe40c10cde0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fe404009fd0 tx=0x7fe40400ef10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:09:07.139 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:07.137+0000 7fe40a000640 1 -- 172.21.15.12:0/49001865 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe40400ce20 con 0x7fe40c109130 2024-09-11T00:09:07.139 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:07.137+0000 7fe40a000640 1 -- 172.21.15.12:0/49001865 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe404005020 con 0x7fe40c109130 2024-09-11T00:09:07.139 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:07.137+0000 7fe40a000640 1 -- 172.21.15.12:0/49001865 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe404010690 con 0x7fe40c109130 2024-09-11T00:09:07.139 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:07.137+0000 7fe413800640 1 -- 172.21.15.12:0/49001865 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe40c06cc30 con 0x7fe40c109130 2024-09-11T00:09:07.139 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:07.137+0000 7fe413800640 1 -- 172.21.15.12:0/49001865 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe40c06d120 con 0x7fe40c109130 2024-09-11T00:09:07.142 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:07.140+0000 7fe413800640 1 -- 172.21.15.12:0/49001865 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe40c103180 con 0x7fe40c109130 2024-09-11T00:09:07.144 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:07.142+0000 7fe40a000640 1 -- 172.21.15.12:0/49001865 <== mon.1 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7fe4040054d0 con 0x7fe40c109130 2024-09-11T00:09:07.146 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:07.144+0000 7fe40a000640 1 --2- 172.21.15.12:0/49001865 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fe3f00b5470 0x7fe3f00b7930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:09:07.146 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:07.144+0000 7fe40a000640 1 -- 172.21.15.12:0/49001865 <== mon.1 v2:172.21.15.178:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9208+0+0 (secure 0 0 0) 0x7fe4040e66d0 con 0x7fe40c109130 2024-09-11T00:09:07.146 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:07.144+0000 7fe410e00640 1 --2- 172.21.15.12:0/49001865 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fe3f00b5470 0x7fe3f00b7930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:09:07.147 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:07.145+0000 7fe410e00640 1 --2- 172.21.15.12:0/49001865 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fe3f00b5470 0x7fe3f00b7930 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fe3f40096c0 tx=0x7fe3f40046c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:09:07.150 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:07.148+0000 7fe40a000640 1 -- 172.21.15.12:0/49001865 <== mon.1 v2:172.21.15.178:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7fe4040ad160 con 0x7fe40c109130 2024-09-11T00:09:07.394 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:07.392+0000 7fe413800640 1 -- 172.21.15.12:0/49001865 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe40c112a40 con 0x7fe40c109130 2024-09-11T00:09:07.395 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:07.393+0000 7fe40a000640 1 -- 172.21.15.12:0/49001865 <== mon.1 v2:172.21.15.178:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) ==== 94+0+11972 (secure 0 0 0) 0x7fe4040aca60 con 0x7fe40c109130 2024-09-11T00:09:07.395 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-11T00:09:07.396 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":33,"btime":"2024-09-11T00:08:46:706969+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14577,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6835/447171591","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":447171591},{"type":"v1","addr":"172.21.15.12:6835","nonce":447171591}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14589,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/901486040","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":901486040},{"type":"v1","addr":"172.21.15.12:6837","nonce":901486040}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":33,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T00:04:04.739881+0000","modified":"2024-09-11T00:08:46.706950+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24674,"mds_1":24643,"mds_2":24638,"mds_3":24637,"mds_4":24614},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14631":{"gid":14631,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.12:6841/3069919437","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6840","nonce":3069919437},{"type":"v1","addr":"172.21.15.12:6841","nonce":3069919437}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24562":{"gid":24562,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6833/1141636112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":1141636112},{"type":"v1","addr":"172.21.15.178:6833","nonce":1141636112}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24581":{"gid":24581,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6833/3111980970","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6832","nonce":3111980970},{"type":"v1","addr":"172.21.15.165:6833","nonce":3111980970}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24607":{"gid":24607,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6835/4084868287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":4084868287},{"type":"v1","addr":"172.21.15.178:6835","nonce":4084868287}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24614":{"gid":24614,"name":"e","rank":4,"incarnation":26,"state":"up:active","state_seq":21,"addr":"172.21.15.165:6835/2425395274","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":2425395274},{"type":"v1","addr":"172.21.15.165:6835","nonce":2425395274}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24637":{"gid":24637,"name":"i","rank":3,"incarnation":24,"state":"up:active","state_seq":18,"addr":"172.21.15.178:6839/70628097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6838","nonce":70628097},{"type":"v1","addr":"172.21.15.178:6839","nonce":70628097}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24638":{"gid":24638,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":18,"addr":"172.21.15.165:6837/6215088","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6836","nonce":6215088},{"type":"v1","addr":"172.21.15.165:6837","nonce":6215088}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24643":{"gid":24643,"name":"g","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.12:6839/3124093009","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6838","nonce":3124093009},{"type":"v1","addr":"172.21.15.12:6839","nonce":3124093009}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24674":{"gid":24674,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.165:6839/501312984","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6838","nonce":501312984},{"type":"v1","addr":"172.21.15.165:6839","nonce":501312984}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25672":{"gid":25672,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6837/308601741","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":308601741},{"type":"v1","addr":"172.21.15.178:6837","nonce":308601741}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24674,"qdb_cluster":[24614,24637,24638,24643,24674]},"id":1}]} 2024-09-11T00:09:07.396 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 33 2024-09-11T00:09:07.398 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:07.396+0000 7fe413800640 1 -- 172.21.15.12:0/49001865 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fe3f00b5470 msgr2=0x7fe3f00b7930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:09:07.398 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:07.396+0000 7fe413800640 1 --2- 172.21.15.12:0/49001865 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fe3f00b5470 0x7fe3f00b7930 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fe3f40096c0 tx=0x7fe3f40046c0 comp rx=0 tx=0).stop 2024-09-11T00:09:07.398 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:07.396+0000 7fe413800640 1 -- 172.21.15.12:0/49001865 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe40c109130 msgr2=0x7fe40c10cde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:09:07.398 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:07.396+0000 7fe413800640 1 --2- 172.21.15.12:0/49001865 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe40c109130 0x7fe40c10cde0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fe404009fd0 tx=0x7fe40400ef10 comp rx=0 tx=0).stop 2024-09-11T00:09:07.398 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:07.396+0000 7fe413800640 1 -- 172.21.15.12:0/49001865 shutdown_connections 2024-09-11T00:09:07.398 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:07.396+0000 7fe413800640 1 --2- 172.21.15.12:0/49001865 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fe3f00b5470 0x7fe3f00b7930 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:09:07.398 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:07.396+0000 7fe413800640 1 --2- 172.21.15.12:0/49001865 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe40c109130 0x7fe40c10cde0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:09:07.398 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:07.396+0000 7fe413800640 1 --2- 172.21.15.12:0/49001865 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe40c1087b0 0x7fe40c10c8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:09:07.399 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:07.396+0000 7fe413800640 1 --2- 172.21.15.12:0/49001865 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe40c102da0 0x7fe40c10f650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:09:07.399 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:07.396+0000 7fe413800640 1 -- 172.21.15.12:0/49001865 >> 172.21.15.12:0/49001865 conn(0x7fe40c0fe810 msgr2=0x7fe40c100c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:09:07.399 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:07.396+0000 7fe413800640 1 -- 172.21.15.12:0/49001865 shutdown_connections 2024-09-11T00:09:07.399 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:07.396+0000 7fe413800640 1 -- 172.21.15.12:0/49001865 wait complete. 2024-09-11T00:09:07.603 INFO:tasks.workunit.client.0.smithi012.stdout: 1 85900 7.50 MB/sec warmup 64 sec latency 17981.416 ms 2024-09-11T00:09:08.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:09:07 smithi165 ceph-mon[35356]: pgmap v460: 129 pgs: 129 active+clean; 27 MiB data, 516 MiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 2.2 MiB/s wr, 159 op/s 2024-09-11T00:09:08.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:09:07 smithi165 ceph-mon[35356]: from='client.? 172.21.15.12:0/49001865' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T00:09:08.063 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:09:07 smithi012 ceph-mon[30333]: pgmap v460: 129 pgs: 129 active+clean; 27 MiB data, 516 MiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 2.2 MiB/s wr, 159 op/s 2024-09-11T00:09:08.063 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:09:07 smithi012 ceph-mon[30333]: from='client.? 172.21.15.12:0/49001865' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T00:09:08.080 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:09:07 smithi178 ceph-mon[35919]: pgmap v460: 129 pgs: 129 active+clean; 27 MiB data, 516 MiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 2.2 MiB/s wr, 159 op/s 2024-09-11T00:09:08.080 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:09:07 smithi178 ceph-mon[35919]: from='client.? 172.21.15.12:0/49001865' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T00:09:08.226 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24674 quiesce db --set-id b4f324a8 --release --await' 2024-09-11T00:09:08.226 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid ff3ad74a-6fcf-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24674 quiesce db --set-id b4f324a8 --release --await 2024-09-11T00:09:08.603 INFO:tasks.workunit.client.0.smithi012.stdout: 1 85900 7.39 MB/sec warmup 65 sec latency 18981.466 ms 2024-09-11T00:09:08.604 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:09:09.216 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:09.213+0000 7f119f200640 1 -- 172.21.15.12:0/2900256707 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f11a010b180 msgr2=0x7f11a0111980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:09:09.216 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:09.213+0000 7f119f200640 1 --2- 172.21.15.12:0/2900256707 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f11a010b180 0x7f11a0111980 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f1198009ca0 tx=0x7f119802a8c0 comp rx=0 tx=0).stop 2024-09-11T00:09:09.216 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:09.214+0000 7f119f200640 1 -- 172.21.15.12:0/2900256707 shutdown_connections 2024-09-11T00:09:09.216 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:09.214+0000 7f119f200640 1 --2- 172.21.15.12:0/2900256707 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f11a010b180 0x7f11a0111980 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:09:09.216 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:09.214+0000 7f119f200640 1 --2- 172.21.15.12:0/2900256707 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f11a010a800 0x7f11a010ac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:09:09.216 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:09.214+0000 7f119f200640 1 --2- 172.21.15.12:0/2900256707 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f11a0104df0 0x7f11a01051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:09:09.217 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:09.214+0000 7f119f200640 1 -- 172.21.15.12:0/2900256707 >> 172.21.15.12:0/2900256707 conn(0x7f11a0100840 msgr2=0x7f11a0102c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:09:09.217 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:09.214+0000 7f119f200640 1 -- 172.21.15.12:0/2900256707 shutdown_connections 2024-09-11T00:09:09.217 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:09.214+0000 7f119f200640 1 -- 172.21.15.12:0/2900256707 wait complete. 2024-09-11T00:09:09.218 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:09.215+0000 7f119f200640 1 Processor -- start 2024-09-11T00:09:09.218 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:09.215+0000 7f119f200640 1 -- start start 2024-09-11T00:09:09.218 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:09.215+0000 7f119f200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f11a0104df0 0x7f11a019b200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:09:09.218 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:09.215+0000 7f119f200640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f11a010a800 0x7f11a019b740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:09:09.218 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:09.215+0000 7f119f200640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f11a010b180 0x7f11a01a0ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:09:09.218 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:09.215+0000 7f119f200640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f11a0114e50 con 0x7f11a010b180 2024-09-11T00:09:09.218 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:09.215+0000 7f119f200640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f11a0114cd0 con 0x7f11a0104df0 2024-09-11T00:09:09.218 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:09.215+0000 7f119f200640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f11a0114fd0 con 0x7f11a010a800 2024-09-11T00:09:09.219 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:09.216+0000 7f119e800640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f11a010b180 0x7f11a01a0ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:09:09.219 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:09.216+0000 7f119de00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f11a0104df0 0x7f11a019b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:09:09.219 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:09.216+0000 7f119e800640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f11a010b180 0x7f11a01a0ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:55486/0 (socket says 172.21.15.12:55486) 2024-09-11T00:09:09.219 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:09.216+0000 7f119e800640 1 -- 172.21.15.12:0/969279467 learned_addr learned my addr 172.21.15.12:0/969279467 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:09:09.219 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:09.216+0000 7f119de00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f11a0104df0 0x7f11a019b200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.12:40700/0 (socket says 172.21.15.12:40700) 2024-09-11T00:09:09.219 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:09.216+0000 7f1195400640 1 --2- 172.21.15.12:0/969279467 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f11a010a800 0x7f11a019b740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:09:09.219 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:09.216+0000 7f119de00640 1 -- 172.21.15.12:0/969279467 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f11a010a800 msgr2=0x7f11a019b740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:09:09.219 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:09.216+0000 7f119de00640 1 --2- 172.21.15.12:0/969279467 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f11a010a800 0x7f11a019b740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:09:09.219 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:09.216+0000 7f119de00640 1 -- 172.21.15.12:0/969279467 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f11a010b180 msgr2=0x7f11a01a0ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:09:09.219 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:09.216+0000 7f119de00640 1 --2- 172.21.15.12:0/969279467 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f11a010b180 0x7f11a01a0ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:09:09.219 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:09.216+0000 7f119de00640 1 -- 172.21.15.12:0/969279467 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11a01a12b0 con 0x7f11a0104df0 2024-09-11T00:09:09.219 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:09.216+0000 7f119e800640 1 --2- 172.21.15.12:0/969279467 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f11a010b180 0x7f11a01a0ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:09:09.220 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:09.216+0000 7f119de00640 1 --2- 172.21.15.12:0/969279467 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f11a0104df0 0x7f11a019b200 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f119000e9f0 tx=0x7f119000eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:09:09.220 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:09.217+0000 7f1197400640 1 -- 172.21.15.12:0/969279467 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f119000cd80 con 0x7f11a0104df0 2024-09-11T00:09:09.220 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:09.217+0000 7f119f200640 1 -- 172.21.15.12:0/969279467 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f11a01a15a0 con 0x7f11a0104df0 2024-09-11T00:09:09.220 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:09.217+0000 7f1197400640 1 -- 172.21.15.12:0/969279467 <== mon.2 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f11900035f0 con 0x7f11a0104df0 2024-09-11T00:09:09.220 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:09.217+0000 7f1197400640 1 -- 172.21.15.12:0/969279467 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1190010690 con 0x7f11a0104df0 2024-09-11T00:09:09.220 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:09.217+0000 7f119f200640 1 -- 172.21.15.12:0/969279467 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11a01a7e30 con 0x7f11a0104df0 2024-09-11T00:09:09.222 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:09.220+0000 7f11a7e75740 1 Processor -- start 2024-09-11T00:09:09.222 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:09.220+0000 7f11a7e75740 1 -- start start 2024-09-11T00:09:09.224 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:09.220+0000 7f11a7e75740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562e8ca401d0 0x562e8ca405b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:09:09.224 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:09.220+0000 7f11a7e75740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562e8ca40af0 0x562e8ca4c540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:09:09.224 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:09.220+0000 7f11a7e75740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562e8ca4ca80 0x562e8ca4ee70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:09:09.224 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:09.220+0000 7f11a7e75740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x562e8c9b7290 con 0x562e8ca4ca80 2024-09-11T00:09:09.224 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:09.220+0000 7f11a7e75740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x562e8c9a66c0 con 0x562e8ca401d0 2024-09-11T00:09:09.224 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:09.220+0000 7f11a7e75740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x562e8c9aa4d0 con 0x562e8ca40af0 2024-09-11T00:09:09.224 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:09.221+0000 7f1197400640 1 -- 172.21.15.12:0/969279467 <== mon.2 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7f1190004000 con 0x7f11a0104df0 2024-09-11T00:09:09.224 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:09.221+0000 7f1195400640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562e8ca4ca80 0x562e8ca4ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:09:09.224 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:09.221+0000 7f1195400640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562e8ca4ca80 0x562e8ca4ee70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:55488/0 (socket says 172.21.15.12:55488) 2024-09-11T00:09:09.224 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:09.221+0000 7f1195400640 1 -- 172.21.15.12:0/1927994292 learned_addr learned my addr 172.21.15.12:0/1927994292 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:09:09.224 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:09.221+0000 7f119e800640 1 --2- 172.21.15.12:0/1927994292 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562e8ca401d0 0x562e8ca405b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:09:09.225 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:09.221+0000 7f119de00640 1 --2- 172.21.15.12:0/1927994292 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562e8ca40af0 0x562e8ca4c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:09:09.225 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:09.221+0000 7f1195400640 1 -- 172.21.15.12:0/1927994292 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562e8ca40af0 msgr2=0x562e8ca4c540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:09:09.225 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:09.221+0000 7f1195400640 1 --2- 172.21.15.12:0/1927994292 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562e8ca40af0 0x562e8ca4c540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:09:09.225 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:09.221+0000 7f1195400640 1 -- 172.21.15.12:0/1927994292 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562e8ca401d0 msgr2=0x562e8ca405b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:09:09.225 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:09.221+0000 7f1195400640 1 --2- 172.21.15.12:0/1927994292 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562e8ca401d0 0x562e8ca405b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:09:09.225 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:09.221+0000 7f1195400640 1 -- 172.21.15.12:0/1927994292 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562e8c933460 con 0x562e8ca4ca80 2024-09-11T00:09:09.225 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:09.221+0000 7f119e800640 1 --2- 172.21.15.12:0/1927994292 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562e8ca401d0 0x562e8ca405b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:09:09.225 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:09.222+0000 7f1195400640 1 --2- 172.21.15.12:0/1927994292 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562e8ca4ca80 0x562e8ca4ee70 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f1188009c40 tx=0x7f1188011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=190724d5e3eb483 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:09:09.225 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:09.222+0000 7f118f400640 1 -- 172.21.15.12:0/1927994292 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1188004230 con 0x562e8ca4ca80 2024-09-11T00:09:09.225 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:09.222+0000 7f118f400640 1 -- 172.21.15.12:0/1927994292 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f11880043d0 con 0x562e8ca4ca80 2024-09-11T00:09:09.225 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:09.222+0000 7f118f400640 1 -- 172.21.15.12:0/1927994292 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1188004d20 con 0x562e8ca4ca80 2024-09-11T00:09:09.227 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:09.222+0000 7f1197400640 1 --2- 172.21.15.12:0/969279467 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f11840b5390 0x7f11840b7850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:09:09.227 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:09.223+0000 7f1197400640 1 -- 172.21.15.12:0/969279467 <== mon.2 v2:172.21.15.165:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9208+0+0 (secure 0 0 0) 0x7f11900e62c0 con 0x7f11a0104df0 2024-09-11T00:09:09.227 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:09.223+0000 7f119e800640 1 --2- 172.21.15.12:0/969279467 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f11840b5390 0x7f11840b7850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:09:09.227 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:09.223+0000 7f11a7e75740 1 -- 172.21.15.12:0/1927994292 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562e8ca4ca80 msgr2=0x562e8ca4ee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:09:09.227 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:09.223+0000 7f11a7e75740 1 --2- 172.21.15.12:0/1927994292 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562e8ca4ca80 0x562e8ca4ee70 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f1188009c40 tx=0x7f1188011810 comp rx=0 tx=0).stop 2024-09-11T00:09:09.227 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:09.223+0000 7f11a7e75740 1 -- 172.21.15.12:0/1927994292 shutdown_connections 2024-09-11T00:09:09.228 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:09.223+0000 7f11a7e75740 1 --2- 172.21.15.12:0/1927994292 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562e8ca4ca80 0x562e8ca4ee70 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:09:09.228 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:09.223+0000 7f11a7e75740 1 --2- 172.21.15.12:0/1927994292 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562e8ca40af0 0x562e8ca4c540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:09:09.228 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:09.223+0000 7f11a7e75740 1 --2- 172.21.15.12:0/1927994292 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562e8ca401d0 0x562e8ca405b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:09:09.228 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:09.223+0000 7f11a7e75740 1 -- 172.21.15.12:0/1927994292 >> 172.21.15.12:0/1927994292 conn(0x562e8ca3e000 msgr2=0x562e8ca3f1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:09:09.228 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:09.223+0000 7f11a7e75740 1 -- 172.21.15.12:0/1927994292 shutdown_connections 2024-09-11T00:09:09.228 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:09.223+0000 7f11a7e75740 1 -- 172.21.15.12:0/1927994292 wait complete. 2024-09-11T00:09:09.228 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:09.223+0000 7f119e800640 1 --2- 172.21.15.12:0/969279467 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f11840b5390 0x7f11840b7850 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f1198004af0 tx=0x7f1198005d80 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:09:09.228 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:09.224+0000 7f11a7e75740 1 -- start start 2024-09-11T00:09:09.228 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:09.224+0000 7f11a7e75740 1 Processor -- start 2024-09-11T00:09:09.228 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:09.225+0000 7f11a7e75740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562e8ca401d0 0x562e8ca49a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:09:09.228 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:09.225+0000 7f11a7e75740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562e8ca40af0 0x562e8ca49f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:09:09.228 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:09.225+0000 7f11a7e75740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562e8ca4ca80 0x562e8ca4a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:09:09.229 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:09.225+0000 7f11a7e75740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x562e8c807bb0 con 0x562e8ca40af0 2024-09-11T00:09:09.229 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:09.225+0000 7f11a7e75740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x562e8c9416c0 con 0x562e8ca401d0 2024-09-11T00:09:09.229 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:09.225+0000 7f11a7e75740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x562e8ca4f730 con 0x562e8ca4ca80 2024-09-11T00:09:09.229 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:09.225+0000 7f11a7e75740 10 client.0 _ll_register_callbacks cb 0x562e8c95c490 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:09:09.229 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:09.225+0000 7f1195400640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562e8ca40af0 0x562e8ca49f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:09:09.229 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:09.225+0000 7f119e800640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562e8ca4ca80 0x562e8ca4a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:09:09.229 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:09.225+0000 7f1195400640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562e8ca40af0 0x562e8ca49f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:55502/0 (socket says 172.21.15.12:55502) 2024-09-11T00:09:09.229 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:09.225+0000 7f1195400640 1 -- 172.21.15.12:0/1376089816 learned_addr learned my addr 172.21.15.12:0/1376089816 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:09:09.229 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:09.225+0000 7f119de00640 1 --2- 172.21.15.12:0/1376089816 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562e8ca401d0 0x562e8ca49a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:09:09.231 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:09.225+0000 7f1195400640 1 -- 172.21.15.12:0/1376089816 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562e8ca4ca80 msgr2=0x562e8ca4a4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:09:09.231 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:09.225+0000 7f1195400640 1 --2- 172.21.15.12:0/1376089816 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562e8ca4ca80 0x562e8ca4a4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:09:09.231 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:09.225+0000 7f1195400640 1 -- 172.21.15.12:0/1376089816 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562e8ca401d0 msgr2=0x562e8ca49a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:09:09.231 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:09.225+0000 7f1195400640 1 --2- 172.21.15.12:0/1376089816 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562e8ca401d0 0x562e8ca49a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:09:09.232 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:09.225+0000 7f1195400640 1 -- 172.21.15.12:0/1376089816 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562e8c704b00 con 0x562e8ca40af0 2024-09-11T00:09:09.232 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:09.225+0000 7f119de00640 1 --2- 172.21.15.12:0/1376089816 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562e8ca401d0 0x562e8ca49a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:09:09.232 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:09.226+0000 7f1195400640 1 --2- 172.21.15.12:0/1376089816 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562e8ca40af0 0x562e8ca49f90 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f118801ab10 tx=0x7f118801ab40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2257f9fdf25ba24d server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:09:09.232 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:09.226+0000 7f1183e00640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-09-11T00:09:09.232 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:09.226+0000 7f1183e00640 1 -- 172.21.15.12:0/1376089816 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1188028020 con 0x562e8ca40af0 2024-09-11T00:09:09.232 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:09.226+0000 7f11a7e75740 1 -- 172.21.15.12:0/1376089816 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f11880043d0 con 0x562e8ca40af0 2024-09-11T00:09:09.232 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:09.226+0000 7f1183e00640 1 -- 172.21.15.12:0/1376089816 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9208+0+0 (secure 0 0 0) 0x7f1188024420 con 0x562e8ca40af0 2024-09-11T00:09:09.232 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:09.226+0000 7f1183e00640 1 -- 172.21.15.12:0/1376089816 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f1188033050 con 0x562e8ca40af0 2024-09-11T00:09:09.232 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:09.226+0000 7f11a7e75740 10 client.15717 fetch_fsmap learned FSMap version 33 2024-09-11T00:09:09.232 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:09.226+0000 7f11a7e75740 1 -- 172.21.15.12:0/1376089816 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x562e8c77b960 con 0x562e8ca40af0 2024-09-11T00:09:09.232 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:09.227+0000 7f1183e00640 1 -- 172.21.15.12:0/1376089816 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f11880043d0 con 0x562e8ca40af0 2024-09-11T00:09:09.232 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:09.227+0000 7f1183e00640 1 -- 172.21.15.12:0/1376089816 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 33) ==== 8386+0+0 (secure 0 0 0) 0x7f1188018070 con 0x562e8ca40af0 2024-09-11T00:09:09.233 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:09.227+0000 7f11a7e75740 10 client.15717 fetch_fsmap finished waiting for FSMap version 33 2024-09-11T00:09:09.233 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:09.227+0000 7f11a7e75740 10 client.15717 resolve_mds: validated gid 24674 aka daemon mds.b 2024-09-11T00:09:09.233 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:09.227+0000 7f11a7e75740 20 client.15717 populate_metadata read hostname 'smithi012' 2024-09-11T00:09:09.233 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:09.227+0000 7f11a7e75740 1 --2- 172.21.15.12:0/1376089816 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x562e8ca5c5b0 0x562e8ca5e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:09:09.233 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:09.227+0000 7f11a7e75740 4 client.15717 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T00:09:09.233 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:09.227+0000 7f11a7e75740 1 -- 172.21.15.12:0/1376089816 --> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562e8c9b91e0 con 0x562e8ca5c5b0 2024-09-11T00:09:09.233 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:09.228+0000 7f119de00640 1 --2- 172.21.15.12:0/1376089816 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x562e8ca5c5b0 0x562e8ca5e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:09:09.233 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:09.228+0000 7f119de00640 1 --2- 172.21.15.12:0/1376089816 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x562e8ca5c5b0 0x562e8ca5e9a0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b39867f367426c4c server_cookie=45503f3f1990d2e8 in_seq=0 out_seq=0 2024-09-11T00:09:09.233 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:09.228+0000 7f1183e00640 10 client.15717 ms_handle_connect on v2:172.21.15.165:6838/501312984 2024-09-11T00:09:09.233 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:09.229+0000 7f1183e00640 1 -- 172.21.15.12:0/1376089816 <== mds.0 v2:172.21.15.165:6838/501312984 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x562e8c9b91e0 con 0x562e8ca5c5b0 2024-09-11T00:09:09.233 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:09.229+0000 7f1183e00640 10 client.15717 handle_command_reply: tid=0 2024-09-11T00:09:09.233 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:09.229+0000 7f11a7e75740 1 client.15717 shutdown 2024-09-11T00:09:09.233 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:09.229+0000 7f11a7e75740 1 -- 172.21.15.12:0/1376089816 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562e8ca40af0 msgr2=0x562e8ca49f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:09:09.233 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:09.229+0000 7f11a7e75740 1 --2- 172.21.15.12:0/1376089816 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562e8ca40af0 0x562e8ca49f90 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f118801ab10 tx=0x7f118801ab40 comp rx=0 tx=0).stop 2024-09-11T00:09:09.234 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:09.229+0000 7f11a7e75740 1 -- 172.21.15.12:0/1376089816 shutdown_connections 2024-09-11T00:09:09.234 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:09.229+0000 7f11a7e75740 1 --2- 172.21.15.12:0/1376089816 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x562e8ca5c5b0 0x562e8ca5e9a0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:09:09.234 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:09.229+0000 7f11a7e75740 1 --2- 172.21.15.12:0/1376089816 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562e8ca4ca80 0x562e8ca4a4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:09:09.234 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:09.229+0000 7f11a7e75740 1 --2- 172.21.15.12:0/1376089816 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562e8ca40af0 0x562e8ca49f90 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:09:09.234 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:09.229+0000 7f11a7e75740 1 --2- 172.21.15.12:0/1376089816 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562e8ca401d0 0x562e8ca49a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:09:09.234 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:09.229+0000 7f11a7e75740 1 -- 172.21.15.12:0/1376089816 >> 172.21.15.12:0/1376089816 conn(0x562e8ca3e000 msgr2=0x562e8ca43ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:09:09.234 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:09.229+0000 7f1183e00640 0 client.15717 ms_handle_reset on v2:172.21.15.165:6838/501312984 2024-09-11T00:09:09.234 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:09.229+0000 7f11a7e75740 1 -- 172.21.15.12:0/1376089816 shutdown_connections 2024-09-11T00:09:09.234 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:09.229+0000 7f11a7e75740 1 -- 172.21.15.12:0/1376089816 wait complete. 2024-09-11T00:09:09.234 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:09.230+0000 7f11a7e75740 20 client.15717 trim_cache size 0 max 16384 2024-09-11T00:09:09.256 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:09.247+0000 7f11a7e75740 1 Processor -- start 2024-09-11T00:09:09.256 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:09.248+0000 7f11a7e75740 1 -- start start 2024-09-11T00:09:09.256 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:09.248+0000 7f11a7e75740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562e8ca401d0 0x562e8ca46c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:09:09.257 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:09.248+0000 7f11a7e75740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562e8ca40af0 0x562e8ca471a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:09:09.257 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:09.248+0000 7f11a7e75740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562e8ca4ca80 0x562e8ca64220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:09:09.257 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:09.248+0000 7f11a7e75740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f11a01b1c00 con 0x562e8ca40af0 2024-09-11T00:09:09.257 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:09.248+0000 7f11a7e75740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f11a01b3800 con 0x562e8ca4ca80 2024-09-11T00:09:09.257 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:09.248+0000 7f11a7e75740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f11a01b22d0 con 0x562e8ca401d0 2024-09-11T00:09:09.257 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:09.248+0000 7f1195400640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562e8ca40af0 0x562e8ca471a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:09:09.257 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:09.248+0000 7f119de00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562e8ca401d0 0x562e8ca46c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:09:09.257 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:09.248+0000 7f119de00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562e8ca401d0 0x562e8ca46c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.12:33816/0 (socket says 172.21.15.12:33816) 2024-09-11T00:09:09.257 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:09.248+0000 7f119de00640 1 -- 172.21.15.12:0/2645551185 learned_addr learned my addr 172.21.15.12:0/2645551185 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:09:09.257 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:09.248+0000 7f119e800640 1 --2- 172.21.15.12:0/2645551185 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562e8ca4ca80 0x562e8ca64220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:09:09.257 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:09.249+0000 7f119de00640 1 -- 172.21.15.12:0/2645551185 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562e8ca4ca80 msgr2=0x562e8ca64220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:09:09.258 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:09.249+0000 7f119de00640 1 --2- 172.21.15.12:0/2645551185 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562e8ca4ca80 0x562e8ca64220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:09:09.258 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:09.249+0000 7f119de00640 1 -- 172.21.15.12:0/2645551185 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562e8ca40af0 msgr2=0x562e8ca471a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:09:09.258 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:09.249+0000 7f119de00640 1 --2- 172.21.15.12:0/2645551185 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562e8ca40af0 0x562e8ca471a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:09:09.258 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:09.249+0000 7f119de00640 1 -- 172.21.15.12:0/2645551185 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562e8ca5ed10 con 0x562e8ca401d0 2024-09-11T00:09:09.258 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:09.249+0000 7f1195400640 1 --2- 172.21.15.12:0/2645551185 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562e8ca40af0 0x562e8ca471a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:09:09.258 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:09.249+0000 7f119de00640 1 --2- 172.21.15.12:0/2645551185 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562e8ca401d0 0x562e8ca46c60 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f119000c6a0 tx=0x7f1190005bb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c0b9f1315c2f2157 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:09:09.258 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:09.249+0000 7f118ea00640 1 -- 172.21.15.12:0/2645551185 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11900acd50 con 0x562e8ca401d0 2024-09-11T00:09:09.258 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:09.249+0000 7f118ea00640 1 -- 172.21.15.12:0/2645551185 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f11900e3e50 con 0x562e8ca401d0 2024-09-11T00:09:09.258 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:09.249+0000 7f118ea00640 1 -- 172.21.15.12:0/2645551185 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1190009310 con 0x562e8ca401d0 2024-09-11T00:09:09.258 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:09.250+0000 7f11a7e75740 1 -- 172.21.15.12:0/2645551185 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562e8ca401d0 msgr2=0x562e8ca46c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:09:09.258 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:09.250+0000 7f11a7e75740 1 --2- 172.21.15.12:0/2645551185 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562e8ca401d0 0x562e8ca46c60 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f119000c6a0 tx=0x7f1190005bb0 comp rx=0 tx=0).stop 2024-09-11T00:09:09.258 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:09.250+0000 7f11a7e75740 1 -- 172.21.15.12:0/2645551185 shutdown_connections 2024-09-11T00:09:09.258 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:09.250+0000 7f11a7e75740 1 --2- 172.21.15.12:0/2645551185 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562e8ca4ca80 0x562e8ca64220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:09:09.259 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:09.250+0000 7f11a7e75740 1 --2- 172.21.15.12:0/2645551185 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562e8ca40af0 0x562e8ca471a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:09:09.259 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:09.250+0000 7f11a7e75740 1 --2- 172.21.15.12:0/2645551185 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562e8ca401d0 0x562e8ca46c60 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:09:09.259 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:09.250+0000 7f11a7e75740 1 -- 172.21.15.12:0/2645551185 >> 172.21.15.12:0/2645551185 conn(0x562e8ca3e000 msgr2=0x562e8ca4b8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:09:09.259 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:09.250+0000 7f11a7e75740 1 -- 172.21.15.12:0/2645551185 shutdown_connections 2024-09-11T00:09:09.259 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:09.250+0000 7f11a7e75740 1 -- 172.21.15.12:0/2645551185 wait complete. 2024-09-11T00:09:09.260 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:09.250+0000 7f11a7e75740 1 -- start start 2024-09-11T00:09:09.260 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:09.251+0000 7f11a7e75740 1 Processor -- start 2024-09-11T00:09:09.260 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:09.251+0000 7f11a7e75740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562e8ca401d0 0x562e8ca551b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:09:09.261 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:09.251+0000 7f11a7e75740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562e8ca40af0 0x562e8ca516e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:09:09.261 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:09.251+0000 7f11a7e75740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562e8ca4ca80 0x562e8ca51c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:09:09.261 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:09.251+0000 7f11a7e75740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f11a01b3130 con 0x562e8ca4ca80 2024-09-11T00:09:09.261 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:09.251+0000 7f11a7e75740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f11a01b29a0 con 0x562e8ca401d0 2024-09-11T00:09:09.261 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:09.251+0000 7f11a7e75740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f11a01afbe0 con 0x562e8ca40af0 2024-09-11T00:09:09.261 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:09.251+0000 7f11a7e75740 10 client.0 _ll_register_callbacks cb 0x7f1188005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:09:09.261 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:09.251+0000 7f119e800640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562e8ca4ca80 0x562e8ca51c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:09:09.262 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:09.251+0000 7f119e800640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562e8ca4ca80 0x562e8ca51c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:55524/0 (socket says 172.21.15.12:55524) 2024-09-11T00:09:09.262 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:09.251+0000 7f119e800640 1 -- 172.21.15.12:0/3591055265 learned_addr learned my addr 172.21.15.12:0/3591055265 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:09:09.262 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:09.251+0000 7f119de00640 1 --2- 172.21.15.12:0/3591055265 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562e8ca401d0 0x562e8ca551b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:09:09.262 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:09.251+0000 7f1195400640 1 --2- 172.21.15.12:0/3591055265 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562e8ca40af0 0x562e8ca516e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:09:09.262 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:09.252+0000 7f119e800640 1 -- 172.21.15.12:0/3591055265 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562e8ca40af0 msgr2=0x562e8ca516e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:09:09.262 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:09.252+0000 7f119e800640 1 --2- 172.21.15.12:0/3591055265 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562e8ca40af0 0x562e8ca516e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:09:09.262 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:09.252+0000 7f119e800640 1 -- 172.21.15.12:0/3591055265 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562e8ca401d0 msgr2=0x562e8ca551b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:09:09.263 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:09.252+0000 7f119e800640 1 --2- 172.21.15.12:0/3591055265 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562e8ca401d0 0x562e8ca551b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:09:09.263 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:09.252+0000 7f119e800640 1 -- 172.21.15.12:0/3591055265 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562e8ca52470 con 0x562e8ca4ca80 2024-09-11T00:09:09.263 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:09.252+0000 7f119de00640 1 --2- 172.21.15.12:0/3591055265 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562e8ca401d0 0x562e8ca551b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:09:09.263 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:09.252+0000 7f119e800640 1 --2- 172.21.15.12:0/3591055265 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562e8ca4ca80 0x562e8ca51c40 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f1198003970 tx=0x7f119802a8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f46a4161062f55e2 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:09:09.263 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:09.252+0000 7f1183e00640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-09-11T00:09:09.263 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:09.252+0000 7f1183e00640 1 -- 172.21.15.12:0/3591055265 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f119802f610 con 0x562e8ca4ca80 2024-09-11T00:09:09.263 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:09.252+0000 7f1183e00640 1 -- 172.21.15.12:0/3591055265 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9208+0+0 (secure 0 0 0) 0x7f1198043030 con 0x562e8ca4ca80 2024-09-11T00:09:09.264 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:09.253+0000 7f11a7e75740 1 -- 172.21.15.12:0/3591055265 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f11900e3e50 con 0x562e8ca4ca80 2024-09-11T00:09:09.264 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:09.253+0000 7f1183e00640 1 -- 172.21.15.12:0/3591055265 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f119802f7b0 con 0x562e8ca4ca80 2024-09-11T00:09:09.264 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:09.253+0000 7f11a7e75740 10 client.15723 fetch_fsmap learned FSMap version 33 2024-09-11T00:09:09.264 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:09.253+0000 7f11a7e75740 1 -- 172.21.15.12:0/3591055265 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x562e8caca500 con 0x562e8ca4ca80 2024-09-11T00:09:09.264 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:09.253+0000 7f1183e00640 1 -- 172.21.15.12:0/3591055265 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f11900e3e50 con 0x562e8ca4ca80 2024-09-11T00:09:09.264 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:09.253+0000 7f1183e00640 1 -- 172.21.15.12:0/3591055265 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 33) ==== 8386+0+0 (secure 0 0 0) 0x7f1198030ca0 con 0x562e8ca4ca80 2024-09-11T00:09:09.264 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:09.253+0000 7f11a7e75740 10 client.15723 fetch_fsmap finished waiting for FSMap version 33 2024-09-11T00:09:09.265 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:09.253+0000 7f11a7e75740 10 client.15723 resolve_mds: validated gid 24674 aka daemon mds.b 2024-09-11T00:09:09.265 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:09.253+0000 7f11a7e75740 20 client.15723 populate_metadata read hostname 'smithi012' 2024-09-11T00:09:09.265 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:09.253+0000 7f11a7e75740 1 --2- 172.21.15.12:0/3591055265 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x562e8cacada0 0x562e8cacd1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:09:09.265 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:09.253+0000 7f11a7e75740 4 client.15723 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "b4f324a8", "release": true, "await": true}] 2024-09-11T00:09:09.265 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:09.253+0000 7f11a7e75740 1 -- 172.21.15.12:0/3591055265 --> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] -- command(tid 0: {"prefix": "quiesce db", "set_id": "b4f324a8", "release": true, "await": true}) -- 0x562e8c9b4c90 con 0x562e8cacada0 2024-09-11T00:09:09.265 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:09.254+0000 7f119de00640 1 --2- 172.21.15.12:0/3591055265 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x562e8cacada0 0x562e8cacd1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:09:09.267 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:09.254+0000 7f119de00640 1 --2- 172.21.15.12:0/3591055265 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x562e8cacada0 0x562e8cacd1c0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2e1c035929d03552 server_cookie=f75befd68c51e75e in_seq=0 out_seq=0 2024-09-11T00:09:09.267 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:09.254+0000 7f1183e00640 10 client.15723 ms_handle_connect on v2:172.21.15.165:6838/501312984 2024-09-11T00:09:09.267 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:09.261+0000 7f1183e00640 1 -- 172.21.15.12:0/3591055265 <== mds.0 v2:172.21.15.165:6838/501312984 1 ==== command_reply(tid 0: 0 ) ==== 8+0+589 (crc 0 0 0) 0x562e8c9b4c90 con 0x562e8cacada0 2024-09-11T00:09:09.267 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:09.261+0000 7f1183e00640 10 client.15723 handle_command_reply: tid=0 2024-09-11T00:09:09.267 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:09.261+0000 7f11a7e75740 1 client.15723 shutdown 2024-09-11T00:09:09.268 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:09.261+0000 7f11a7e75740 1 -- 172.21.15.12:0/3591055265 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562e8ca4ca80 msgr2=0x562e8ca51c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:09:09.268 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:09.261+0000 7f11a7e75740 1 --2- 172.21.15.12:0/3591055265 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562e8ca4ca80 0x562e8ca51c40 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f1198003970 tx=0x7f119802a8c0 comp rx=0 tx=0).stop 2024-09-11T00:09:09.268 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:09.261+0000 7f11a7e75740 1 -- 172.21.15.12:0/3591055265 shutdown_connections 2024-09-11T00:09:09.268 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:09.261+0000 7f11a7e75740 1 --2- 172.21.15.12:0/3591055265 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x562e8cacada0 0x562e8cacd1c0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:09:09.268 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:09.261+0000 7f11a7e75740 1 --2- 172.21.15.12:0/3591055265 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562e8ca4ca80 0x562e8ca51c40 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:09:09.268 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:09.261+0000 7f11a7e75740 1 --2- 172.21.15.12:0/3591055265 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562e8ca40af0 0x562e8ca516e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:09:09.268 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:09.261+0000 7f11a7e75740 1 --2- 172.21.15.12:0/3591055265 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562e8ca401d0 0x562e8ca551b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:09:09.269 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:09.261+0000 7f1183e00640 0 client.15723 ms_handle_reset on v2:172.21.15.165:6838/501312984 2024-09-11T00:09:09.269 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:09.261+0000 7f11a7e75740 1 -- 172.21.15.12:0/3591055265 >> 172.21.15.12:0/3591055265 conn(0x562e8ca3e000 msgr2=0x562e8ca4b8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:09:09.269 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:09.262+0000 7f11a7e75740 1 -- 172.21.15.12:0/3591055265 shutdown_connections 2024-09-11T00:09:09.269 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:09.262+0000 7f11a7e75740 1 -- 172.21.15.12:0/3591055265 wait complete. 2024-09-11T00:09:09.269 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:09.262+0000 7f11a7e75740 20 client.15723 trim_cache size 0 max 16384 2024-09-11T00:09:09.270 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-09-11T00:09:09.270 INFO:teuthology.orchestra.run.smithi012.stdout: "epoch": 33, 2024-09-11T00:09:09.270 INFO:teuthology.orchestra.run.smithi012.stdout: "leader": 24674, 2024-09-11T00:09:09.270 INFO:teuthology.orchestra.run.smithi012.stdout: "set_version": 8, 2024-09-11T00:09:09.270 INFO:teuthology.orchestra.run.smithi012.stdout: "sets": { 2024-09-11T00:09:09.270 INFO:teuthology.orchestra.run.smithi012.stdout: "b4f324a8": { 2024-09-11T00:09:09.270 INFO:teuthology.orchestra.run.smithi012.stdout: "version": 8, 2024-09-11T00:09:09.271 INFO:teuthology.orchestra.run.smithi012.stdout: "age_ref": 0.0, 2024-09-11T00:09:09.271 INFO:teuthology.orchestra.run.smithi012.stdout: "state": { 2024-09-11T00:09:09.271 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "RELEASED", 2024-09-11T00:09:09.271 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 0.0 2024-09-11T00:09:09.272 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:09:09.272 INFO:teuthology.orchestra.run.smithi012.stdout: "timeout": 90.0, 2024-09-11T00:09:09.272 INFO:teuthology.orchestra.run.smithi012.stdout: "expiration": 140.5, 2024-09-11T00:09:09.272 INFO:teuthology.orchestra.run.smithi012.stdout: "members": { 2024-09-11T00:09:09.272 INFO:teuthology.orchestra.run.smithi012.stdout: "file:/": { 2024-09-11T00:09:09.273 INFO:teuthology.orchestra.run.smithi012.stdout: "excluded": false, 2024-09-11T00:09:09.273 INFO:teuthology.orchestra.run.smithi012.stdout: "state": { 2024-09-11T00:09:09.273 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "RELEASED", 2024-09-11T00:09:09.273 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 0.0 2024-09-11T00:09:09.273 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:09:09.273 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:09:09.273 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:09:09.273 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:09:09.273 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:09:09.273 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-09-11T00:09:09.274 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:09.263+0000 7f119f200640 1 -- 172.21.15.12:0/969279467 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f11840b5390 msgr2=0x7f11840b7850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:09:09.274 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:09.263+0000 7f119f200640 1 --2- 172.21.15.12:0/969279467 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f11840b5390 0x7f11840b7850 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f1198004af0 tx=0x7f1198005d80 comp rx=0 tx=0).stop 2024-09-11T00:09:09.274 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:09.263+0000 7f119f200640 1 -- 172.21.15.12:0/969279467 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f11a0104df0 msgr2=0x7f11a019b200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:09:09.274 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:09.263+0000 7f119f200640 1 --2- 172.21.15.12:0/969279467 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f11a0104df0 0x7f11a019b200 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f119000e9f0 tx=0x7f119000eeb0 comp rx=0 tx=0).stop 2024-09-11T00:09:09.274 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:09.263+0000 7f119f200640 1 -- 172.21.15.12:0/969279467 shutdown_connections 2024-09-11T00:09:09.274 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:09.263+0000 7f119f200640 1 --2- 172.21.15.12:0/969279467 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f11840b5390 0x7f11840b7850 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:09:09.275 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:09.263+0000 7f119f200640 1 --2- 172.21.15.12:0/969279467 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f11a010b180 0x7f11a01a0ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:09:09.275 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:09.263+0000 7f119f200640 1 --2- 172.21.15.12:0/969279467 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f11a010a800 0x7f11a019b740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:09:09.275 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:09.263+0000 7f119f200640 1 --2- 172.21.15.12:0/969279467 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f11a0104df0 0x7f11a019b200 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:09:09.275 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:09.263+0000 7f119f200640 1 -- 172.21.15.12:0/969279467 >> 172.21.15.12:0/969279467 conn(0x7f11a0100840 msgr2=0x7f11a0100f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:09:09.275 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:09.263+0000 7f119f200640 1 -- 172.21.15.12:0/969279467 shutdown_connections 2024-09-11T00:09:09.275 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:09.263+0000 7f119f200640 1 -- 172.21.15.12:0/969279467 wait complete. 2024-09-11T00:09:09.604 INFO:tasks.workunit.client.0.smithi012.stdout: 1 86089 7.28 MB/sec warmup 66 sec latency 19643.180 ms 2024-09-11T00:09:10.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:09:09 smithi165 ceph-mon[35356]: pgmap v461: 129 pgs: 129 active+clean; 26 MiB data, 518 MiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 1.6 MiB/s wr, 122 op/s 2024-09-11T00:09:10.038 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:09:09 smithi012 ceph-mon[30333]: pgmap v461: 129 pgs: 129 active+clean; 26 MiB data, 518 MiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 1.6 MiB/s wr, 122 op/s 2024-09-11T00:09:10.079 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'b4f324a8', total seconds elapsed: 24.5 2024-09-11T00:09:10.079 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 20.5 2024-09-11T00:09:10.080 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid ff3ad74a-6fcf-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T00:09:10.080 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:09:09 smithi178 ceph-mon[35919]: pgmap v461: 129 pgs: 129 active+clean; 26 MiB data, 518 MiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 1.6 MiB/s wr, 122 op/s 2024-09-11T00:09:10.486 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:09:10.603 INFO:tasks.workunit.client.0.smithi012.stdout: 1 88539 7.38 MB/sec warmup 67 sec latency 32.985 ms 2024-09-11T00:09:11.074 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:11.071+0000 7f1127e00640 1 -- 172.21.15.12:0/3592447932 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1128075e00 msgr2=0x7f112810ecc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:09:11.074 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:11.071+0000 7f1127e00640 1 --2- 172.21.15.12:0/3592447932 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1128075e00 0x7f112810ecc0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f1114009900 tx=0x7f111402a410 comp rx=0 tx=0).stop 2024-09-11T00:09:11.074 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:11.071+0000 7f1127e00640 1 -- 172.21.15.12:0/3592447932 shutdown_connections 2024-09-11T00:09:11.074 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:11.071+0000 7f1127e00640 1 --2- 172.21.15.12:0/3592447932 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f112810f200 0x7f11281115f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:09:11.075 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:11.071+0000 7f1127e00640 1 --2- 172.21.15.12:0/3592447932 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1128075e00 0x7f112810ecc0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:09:11.075 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:11.071+0000 7f1127e00640 1 --2- 172.21.15.12:0/3592447932 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f11280754e0 0x7f11280758c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:09:11.075 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:11.071+0000 7f1127e00640 1 -- 172.21.15.12:0/3592447932 >> 172.21.15.12:0/3592447932 conn(0x7f11280fe660 msgr2=0x7f1128100a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:09:11.075 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:11.071+0000 7f1127e00640 1 -- 172.21.15.12:0/3592447932 shutdown_connections 2024-09-11T00:09:11.075 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:11.071+0000 7f1127e00640 1 -- 172.21.15.12:0/3592447932 wait complete. 2024-09-11T00:09:11.077 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:11.072+0000 7f1127e00640 1 Processor -- start 2024-09-11T00:09:11.077 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:11.073+0000 7f1127e00640 1 -- start start 2024-09-11T00:09:11.077 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:11.073+0000 7f1127e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f11280754e0 0x7f112819b250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:09:11.077 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:11.073+0000 7f1127e00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1128075e00 0x7f112819b790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:09:11.077 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:11.073+0000 7f1127e00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f112810f200 0x7f11281a0b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:09:11.077 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:11.073+0000 7f1127e00640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f1128114de0 con 0x7f1128075e00 2024-09-11T00:09:11.077 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:11.073+0000 7f1127e00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f1128114c60 con 0x7f11280754e0 2024-09-11T00:09:11.077 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:11.073+0000 7f1127e00640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f1128114f60 con 0x7f112810f200 2024-09-11T00:09:11.077 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:11.073+0000 7f1126a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f11280754e0 0x7f112819b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:09:11.078 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:11.073+0000 7f1126a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f11280754e0 0x7f112819b250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.12:40758/0 (socket says 172.21.15.12:40758) 2024-09-11T00:09:11.078 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:11.073+0000 7f1126a00640 1 -- 172.21.15.12:0/2277356914 learned_addr learned my addr 172.21.15.12:0/2277356914 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:09:11.078 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:11.073+0000 7f1127400640 1 --2- 172.21.15.12:0/2277356914 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f112810f200 0x7f11281a0b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:09:11.078 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:11.073+0000 7f1126000640 1 --2- 172.21.15.12:0/2277356914 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1128075e00 0x7f112819b790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:09:11.078 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:11.074+0000 7f1126a00640 1 -- 172.21.15.12:0/2277356914 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f112810f200 msgr2=0x7f11281a0b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:09:11.078 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:11.074+0000 7f1126a00640 1 --2- 172.21.15.12:0/2277356914 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f112810f200 0x7f11281a0b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:09:11.078 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:11.074+0000 7f1126a00640 1 -- 172.21.15.12:0/2277356914 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1128075e00 msgr2=0x7f112819b790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:09:11.078 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:11.074+0000 7f1126a00640 1 --2- 172.21.15.12:0/2277356914 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1128075e00 0x7f112819b790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:09:11.078 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:11.074+0000 7f1126a00640 1 -- 172.21.15.12:0/2277356914 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11281a1300 con 0x7f11280754e0 2024-09-11T00:09:11.078 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:11.074+0000 7f1126a00640 1 --2- 172.21.15.12:0/2277356914 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f11280754e0 0x7f112819b250 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f111800e9f0 tx=0x7f111800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:09:11.079 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:11.074+0000 7f111f400640 1 -- 172.21.15.12:0/2277356914 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f111800cd80 con 0x7f11280754e0 2024-09-11T00:09:11.079 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:11.074+0000 7f1127e00640 1 -- 172.21.15.12:0/2277356914 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f11281a15f0 con 0x7f11280754e0 2024-09-11T00:09:11.079 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:11.074+0000 7f111f400640 1 -- 172.21.15.12:0/2277356914 <== mon.2 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f11180108e0 con 0x7f11280754e0 2024-09-11T00:09:11.079 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:11.075+0000 7f111f400640 1 -- 172.21.15.12:0/2277356914 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1118018980 con 0x7f11280754e0 2024-09-11T00:09:11.079 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:11.075+0000 7f1127e00640 1 -- 172.21.15.12:0/2277356914 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11281a7e80 con 0x7f11280754e0 2024-09-11T00:09:11.079 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:11.076+0000 7f1127e00640 1 -- 172.21.15.12:0/2277356914 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f11080052c0 con 0x7f11280754e0 2024-09-11T00:09:11.080 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:11.078+0000 7f111f400640 1 -- 172.21.15.12:0/2277356914 <== mon.2 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7f1118010430 con 0x7f11280754e0 2024-09-11T00:09:11.081 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:11.079+0000 7f111f400640 1 --2- 172.21.15.12:0/2277356914 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f110c0b5410 0x7f110c0b78d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:09:11.081 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:11.079+0000 7f111f400640 1 -- 172.21.15.12:0/2277356914 <== mon.2 v2:172.21.15.165:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9208+0+0 (secure 0 0 0) 0x7f11180e60a0 con 0x7f11280754e0 2024-09-11T00:09:11.082 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:11.080+0000 7f1126000640 1 --2- 172.21.15.12:0/2277356914 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f110c0b5410 0x7f110c0b78d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:09:11.082 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:11.080+0000 7f1126000640 1 --2- 172.21.15.12:0/2277356914 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f110c0b5410 0x7f110c0b78d0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f1114031440 tx=0x7f11140095f0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:09:11.083 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:11.081+0000 7f111f400640 1 -- 172.21.15.12:0/2277356914 <== mon.2 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f11180acb30 con 0x7f11280754e0 2024-09-11T00:09:11.275 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:11.272+0000 7fb81e800640 1 -- 172.21.15.12:0/4137415546 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb81806b470 msgr2=0x7fb818105360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:09:11.275 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:11.272+0000 7fb81e800640 1 --2- 172.21.15.12:0/4137415546 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb81806b470 0x7fb818105360 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fb80c009900 tx=0x7fb80c02a410 comp rx=0 tx=0).stop 2024-09-11T00:09:11.275 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:11.273+0000 7fb81e800640 1 -- 172.21.15.12:0/4137415546 shutdown_connections 2024-09-11T00:09:11.275 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:11.273+0000 7fb81e800640 1 --2- 172.21.15.12:0/4137415546 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb8181084a0 0x7fb81810a960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:09:11.275 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:11.273+0000 7fb81e800640 1 --2- 172.21.15.12:0/4137415546 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb8181058f0 0x7fb818107d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:09:11.275 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:11.273+0000 7fb81e800640 1 --2- 172.21.15.12:0/4137415546 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb81806b470 0x7fb818105360 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:09:11.275 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:11.273+0000 7fb81e800640 1 -- 172.21.15.12:0/4137415546 >> 172.21.15.12:0/4137415546 conn(0x7fb8180fcf30 msgr2=0x7fb8180ff370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:09:11.276 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:11.273+0000 7fb81e800640 1 -- 172.21.15.12:0/4137415546 shutdown_connections 2024-09-11T00:09:11.276 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:11.273+0000 7fb81e800640 1 -- 172.21.15.12:0/4137415546 wait complete. 2024-09-11T00:09:11.277 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:11.274+0000 7fb81e800640 1 Processor -- start 2024-09-11T00:09:11.277 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:11.274+0000 7fb81e800640 1 -- start start 2024-09-11T00:09:11.278 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:11.274+0000 7fb81e800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb81806b470 0x7fb818102aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:09:11.278 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:11.274+0000 7fb81e800640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb8181058f0 0x7fb818101300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:09:11.278 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:11.274+0000 7fb81e800640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb8181084a0 0x7fb818101840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:09:11.278 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:11.274+0000 7fb81e800640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fb81810e3f0 con 0x7fb8181058f0 2024-09-11T00:09:11.278 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:11.274+0000 7fb81e800640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fb81810e270 con 0x7fb81806b470 2024-09-11T00:09:11.278 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:11.274+0000 7fb81e800640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fb81810e570 con 0x7fb8181084a0 2024-09-11T00:09:11.278 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:11.274+0000 7fb81d400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb81806b470 0x7fb818102aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:09:11.278 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:11.274+0000 7fb81ca00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb8181058f0 0x7fb818101300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:09:11.278 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:11.274+0000 7fb81de00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb8181084a0 0x7fb818101840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:09:11.278 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:11.274+0000 7fb81d400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb81806b470 0x7fb818102aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.12:40766/0 (socket says 172.21.15.12:40766) 2024-09-11T00:09:11.278 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:11.274+0000 7fb81d400640 1 -- 172.21.15.12:0/860780313 learned_addr learned my addr 172.21.15.12:0/860780313 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:09:11.279 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:11.275+0000 7fb81d400640 1 -- 172.21.15.12:0/860780313 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb8181084a0 msgr2=0x7fb818101840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:09:11.279 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:11.275+0000 7fb81d400640 1 --2- 172.21.15.12:0/860780313 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb8181084a0 0x7fb818101840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:09:11.279 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:11.275+0000 7fb81d400640 1 -- 172.21.15.12:0/860780313 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb8181058f0 msgr2=0x7fb818101300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:09:11.279 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:11.275+0000 7fb81d400640 1 --2- 172.21.15.12:0/860780313 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb8181058f0 0x7fb818101300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:09:11.279 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:11.275+0000 7fb81d400640 1 -- 172.21.15.12:0/860780313 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8181020b0 con 0x7fb81806b470 2024-09-11T00:09:11.279 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:11.275+0000 7fb81d400640 1 --2- 172.21.15.12:0/860780313 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb81806b470 0x7fb818102aa0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fb80c005940 tx=0x7fb80c004430 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:09:11.279 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:11.275+0000 7fb816000640 1 -- 172.21.15.12:0/860780313 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb80c03f030 con 0x7fb81806b470 2024-09-11T00:09:11.279 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:11.275+0000 7fb81e800640 1 -- 172.21.15.12:0/860780313 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb8181a2bf0 con 0x7fb81806b470 2024-09-11T00:09:11.279 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:11.275+0000 7fb816000640 1 -- 172.21.15.12:0/860780313 <== mon.2 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb80c003cb0 con 0x7fb81806b470 2024-09-11T00:09:11.279 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:11.275+0000 7fb816000640 1 -- 172.21.15.12:0/860780313 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb80c035070 con 0x7fb81806b470 2024-09-11T00:09:11.279 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:11.276+0000 7fb81e800640 1 -- 172.21.15.12:0/860780313 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8181a30b0 con 0x7fb81806b470 2024-09-11T00:09:11.281 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:11.278+0000 7fb822e0f740 1 Processor -- start 2024-09-11T00:09:11.281 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:11.279+0000 7fb822e0f740 1 -- start start 2024-09-11T00:09:11.281 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:11.279+0000 7fb822e0f740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556372d1e970 0x556372d20d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:09:11.281 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:11.279+0000 7fb822e0f740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556372d212a0 0x556372d23690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:09:11.281 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:11.279+0000 7fb822e0f740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556372d23ce0 0x556372d261a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:09:11.281 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:11.279+0000 7fb816000640 1 -- 172.21.15.12:0/860780313 <== mon.2 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7fb80c03b460 con 0x7fb81806b470 2024-09-11T00:09:11.281 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:11.279+0000 7fb822e0f740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x556372c916b0 con 0x556372d1e970 2024-09-11T00:09:11.281 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:11.279+0000 7fb822e0f740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x556372c80ae0 con 0x556372d212a0 2024-09-11T00:09:11.282 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:11.279+0000 7fb822e0f740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x556372c848f0 con 0x556372d23ce0 2024-09-11T00:09:11.282 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:11.279+0000 7fb81de00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556372d1e970 0x556372d20d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:09:11.282 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:11.279+0000 7fb81de00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556372d1e970 0x556372d20d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:55564/0 (socket says 172.21.15.12:55564) 2024-09-11T00:09:11.282 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:11.279+0000 7fb81de00640 1 -- 172.21.15.12:0/453627158 learned_addr learned my addr 172.21.15.12:0/453627158 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:09:11.282 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:11.279+0000 7fb81d400640 1 --2- 172.21.15.12:0/453627158 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556372d212a0 0x556372d23690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:09:11.282 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:11.279+0000 7fb81ca00640 1 --2- 172.21.15.12:0/453627158 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556372d23ce0 0x556372d261a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:09:11.284 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:11.280+0000 7fb81ca00640 1 -- 172.21.15.12:0/453627158 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556372d212a0 msgr2=0x556372d23690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:09:11.284 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:11.280+0000 7fb81ca00640 1 --2- 172.21.15.12:0/453627158 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556372d212a0 0x556372d23690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:09:11.284 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:11.280+0000 7fb81ca00640 1 -- 172.21.15.12:0/453627158 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556372d1e970 msgr2=0x556372d20d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:09:11.284 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:11.280+0000 7fb81ca00640 1 --2- 172.21.15.12:0/453627158 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556372d1e970 0x556372d20d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:09:11.284 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:11.280+0000 7fb81ca00640 1 -- 172.21.15.12:0/453627158 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556372c4d780 con 0x556372d23ce0 2024-09-11T00:09:11.284 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:11.280+0000 7fb81de00640 1 --2- 172.21.15.12:0/453627158 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556372d1e970 0x556372d20d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:09:11.284 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:11.280+0000 7fb81ca00640 1 --2- 172.21.15.12:0/453627158 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556372d23ce0 0x556372d261a0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fb804005fd0 tx=0x7fb8040107a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1beaa532245cf663 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:09:11.284 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:11.280+0000 7fb80a600640 1 -- 172.21.15.12:0/453627158 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb80401c680 con 0x556372d23ce0 2024-09-11T00:09:11.285 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:11.280+0000 7fb80a600640 1 -- 172.21.15.12:0/453627158 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb80400be20 con 0x556372d23ce0 2024-09-11T00:09:11.285 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:11.280+0000 7fb80a600640 1 -- 172.21.15.12:0/453627158 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb804011c00 con 0x556372d23ce0 2024-09-11T00:09:11.285 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:11.281+0000 7fb816000640 1 --2- 172.21.15.12:0/860780313 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fb8000b52e0 0x7fb8000b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:09:11.285 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:11.281+0000 7fb822e0f740 1 -- 172.21.15.12:0/453627158 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556372d23ce0 msgr2=0x556372d261a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:09:11.285 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:11.281+0000 7fb822e0f740 1 --2- 172.21.15.12:0/453627158 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556372d23ce0 0x556372d261a0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fb804005fd0 tx=0x7fb8040107a0 comp rx=0 tx=0).stop 2024-09-11T00:09:11.285 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:11.281+0000 7fb816000640 1 -- 172.21.15.12:0/860780313 <== mon.2 v2:172.21.15.165:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9208+0+0 (secure 0 0 0) 0x7fb80c106b70 con 0x7fb81806b470 2024-09-11T00:09:11.285 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:11.281+0000 7fb81de00640 1 --2- 172.21.15.12:0/860780313 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fb8000b52e0 0x7fb8000b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:09:11.285 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:11.281+0000 7fb822e0f740 1 -- 172.21.15.12:0/453627158 shutdown_connections 2024-09-11T00:09:11.285 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:11.281+0000 7fb822e0f740 1 --2- 172.21.15.12:0/453627158 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556372d23ce0 0x556372d261a0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:09:11.285 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:11.281+0000 7fb822e0f740 1 --2- 172.21.15.12:0/453627158 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556372d212a0 0x556372d23690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:09:11.285 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:11.281+0000 7fb822e0f740 1 --2- 172.21.15.12:0/453627158 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556372d1e970 0x556372d20d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:09:11.285 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:11.281+0000 7fb822e0f740 1 -- 172.21.15.12:0/453627158 >> 172.21.15.12:0/453627158 conn(0x556372d19ac0 msgr2=0x556372d1a850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:09:11.285 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:11.281+0000 7fb822e0f740 1 -- 172.21.15.12:0/453627158 shutdown_connections 2024-09-11T00:09:11.286 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:11.281+0000 7fb822e0f740 1 -- 172.21.15.12:0/453627158 wait complete. 2024-09-11T00:09:11.286 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:11.282+0000 7fb822e0f740 1 -- start start 2024-09-11T00:09:11.286 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:11.282+0000 7fb81de00640 1 --2- 172.21.15.12:0/860780313 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fb8000b52e0 0x7fb8000b77a0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fb810005f40 tx=0x7fb810009600 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:09:11.286 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:11.282+0000 7fb822e0f740 1 Processor -- start 2024-09-11T00:09:11.289 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:11.283+0000 7fb822e0f740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556372d1e970 0x556372d280e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:09:11.292 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:11.283+0000 7fb822e0f740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556372d212a0 0x556372d28780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:09:11.292 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:11.283+0000 7fb822e0f740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556372d23ce0 0x556372d292e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:09:11.292 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:11.283+0000 7fb822e0f740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x556372bb8ab0 con 0x556372d23ce0 2024-09-11T00:09:11.292 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:11.283+0000 7fb822e0f740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x556372c27570 con 0x556372d212a0 2024-09-11T00:09:11.292 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:11.283+0000 7fb822e0f740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x556372d26cc0 con 0x556372d1e970 2024-09-11T00:09:11.292 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:11.283+0000 7fb822e0f740 10 client.0 _ll_register_callbacks cb 0x556372c36100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:09:11.293 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:11.283+0000 7fb81ca00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556372d212a0 0x556372d28780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:09:11.293 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:11.283+0000 7fb81d400640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556372d1e970 0x556372d280e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:09:11.293 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:11.283+0000 7fb81ca00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556372d212a0 0x556372d28780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.12:40784/0 (socket says 172.21.15.12:40784) 2024-09-11T00:09:11.293 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:11.283+0000 7fb81ca00640 1 -- 172.21.15.12:0/116260599 learned_addr learned my addr 172.21.15.12:0/116260599 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:09:11.293 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:11.283+0000 7fb81de00640 1 --2- 172.21.15.12:0/116260599 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556372d23ce0 0x556372d292e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:09:11.293 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:11.283+0000 7fb81ca00640 1 -- 172.21.15.12:0/116260599 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556372d1e970 msgr2=0x556372d280e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:09:11.293 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:11.283+0000 7fb81ca00640 1 --2- 172.21.15.12:0/116260599 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556372d1e970 0x556372d280e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:09:11.293 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:11.283+0000 7fb81ca00640 1 -- 172.21.15.12:0/116260599 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556372d23ce0 msgr2=0x556372d292e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:09:11.293 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:11.283+0000 7fb81ca00640 1 --2- 172.21.15.12:0/116260599 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556372d23ce0 0x556372d292e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:09:11.293 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:11.283+0000 7fb81ca00640 1 -- 172.21.15.12:0/116260599 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5563729e2790 con 0x556372d212a0 2024-09-11T00:09:11.293 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:11.283+0000 7fb81d400640 1 --2- 172.21.15.12:0/116260599 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556372d1e970 0x556372d280e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:09:11.293 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:11.283+0000 7fb81de00640 1 --2- 172.21.15.12:0/116260599 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556372d23ce0 0x556372d292e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T00:09:11.293 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:11.284+0000 7fb81ca00640 1 --2- 172.21.15.12:0/116260599 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556372d212a0 0x556372d28780 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fb804011670 tx=0x7fb804005900 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f3ec6778fb4ca051 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:09:11.294 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:11.284+0000 7fb7fea00640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-09-11T00:09:11.294 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:11.284+0000 7fb7fea00640 1 -- 172.21.15.12:0/116260599 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb804011dd0 con 0x556372d212a0 2024-09-11T00:09:11.294 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:11.284+0000 7fb822e0f740 1 -- 172.21.15.12:0/116260599 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb80400be20 con 0x556372d212a0 2024-09-11T00:09:11.294 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:11.284+0000 7fb7fea00640 1 -- 172.21.15.12:0/116260599 <== mon.2 v2:172.21.15.165:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9208+0+0 (secure 0 0 0) 0x7fb804004020 con 0x556372d212a0 2024-09-11T00:09:11.294 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:11.284+0000 7fb7fea00640 1 -- 172.21.15.12:0/116260599 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fb804030050 con 0x556372d212a0 2024-09-11T00:09:11.294 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:11.284+0000 7fb822e0f740 10 client.25769 fetch_fsmap learned FSMap version 33 2024-09-11T00:09:11.294 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:11.284+0000 7fb822e0f740 1 -- 172.21.15.12:0/116260599 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x556372a5b620 con 0x556372d212a0 2024-09-11T00:09:11.294 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:11.285+0000 7fb7fea00640 1 -- 172.21.15.12:0/116260599 <== mon.2 v2:172.21.15.165:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb80400be20 con 0x556372d212a0 2024-09-11T00:09:11.294 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:11.285+0000 7fb7fea00640 1 -- 172.21.15.12:0/116260599 <== mon.2 v2:172.21.15.165:3300/0 5 ==== fsmap(e 33) ==== 8386+0+0 (secure 0 0 0) 0x7fb8040048f0 con 0x556372d212a0 2024-09-11T00:09:11.294 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:11.285+0000 7fb822e0f740 10 client.25769 fetch_fsmap finished waiting for FSMap version 33 2024-09-11T00:09:11.294 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:11.285+0000 7fb822e0f740 10 client.25769 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T00:09:11.294 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:11.285+0000 7fb822e0f740 20 client.25769 populate_metadata read hostname 'smithi012' 2024-09-11T00:09:11.294 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:11.285+0000 7fb822e0f740 1 --2- 172.21.15.12:0/116260599 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x556372d335d0 0x556372d359c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:09:11.295 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:11.285+0000 7fb822e0f740 4 client.25769 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T00:09:11.295 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:11.285+0000 7fb822e0f740 1 -- 172.21.15.12:0/116260599 --> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556372c93600 con 0x556372d335d0 2024-09-11T00:09:11.295 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:11.285+0000 7fb81d400640 1 --2- 172.21.15.12:0/116260599 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x556372d335d0 0x556372d359c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:09:11.295 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:11.286+0000 7fb81d400640 1 --2- 172.21.15.12:0/116260599 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x556372d335d0 0x556372d359c0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=719443e5339b8bf1 server_cookie=634208d6fb9e4135 in_seq=0 out_seq=0 2024-09-11T00:09:11.295 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:11.286+0000 7fb7fea00640 10 client.25769 ms_handle_connect on v2:172.21.15.165:6838/501312984 2024-09-11T00:09:11.295 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:11.286+0000 7fb7fea00640 1 -- 172.21.15.12:0/116260599 <== mds.0 v2:172.21.15.165:6838/501312984 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x556372c93600 con 0x556372d335d0 2024-09-11T00:09:11.295 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:11.286+0000 7fb7fea00640 10 client.25769 handle_command_reply: tid=0 2024-09-11T00:09:11.295 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:11.286+0000 7fb822e0f740 1 client.25769 shutdown 2024-09-11T00:09:11.295 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:11.287+0000 7fb822e0f740 1 -- 172.21.15.12:0/116260599 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556372d212a0 msgr2=0x556372d28780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:09:11.295 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:11.287+0000 7fb822e0f740 1 --2- 172.21.15.12:0/116260599 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556372d212a0 0x556372d28780 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fb804011670 tx=0x7fb804005900 comp rx=0 tx=0).stop 2024-09-11T00:09:11.295 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:11.287+0000 7fb822e0f740 1 -- 172.21.15.12:0/116260599 shutdown_connections 2024-09-11T00:09:11.295 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:11.287+0000 7fb822e0f740 1 --2- 172.21.15.12:0/116260599 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x556372d335d0 0x556372d359c0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:09:11.295 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:11.287+0000 7fb822e0f740 1 --2- 172.21.15.12:0/116260599 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556372d23ce0 0x556372d292e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:09:11.296 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:11.287+0000 7fb822e0f740 1 --2- 172.21.15.12:0/116260599 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556372d212a0 0x556372d28780 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:09:11.296 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:11.287+0000 7fb822e0f740 1 --2- 172.21.15.12:0/116260599 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556372d1e970 0x556372d280e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:09:11.296 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:11.287+0000 7fb822e0f740 1 -- 172.21.15.12:0/116260599 >> 172.21.15.12:0/116260599 conn(0x556372d19ac0 msgr2=0x556372d18bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:09:11.296 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:11.287+0000 7fb7fea00640 0 client.25769 ms_handle_reset on v2:172.21.15.165:6838/501312984 2024-09-11T00:09:11.296 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:11.287+0000 7fb822e0f740 1 -- 172.21.15.12:0/116260599 shutdown_connections 2024-09-11T00:09:11.296 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:11.287+0000 7fb822e0f740 1 -- 172.21.15.12:0/116260599 wait complete. 2024-09-11T00:09:11.296 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:11.287+0000 7fb822e0f740 20 client.25769 trim_cache size 0 max 16384 2024-09-11T00:09:11.309 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:11.307+0000 7fb822e0f740 1 Processor -- start 2024-09-11T00:09:11.309 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:11.307+0000 7fb822e0f740 1 -- start start 2024-09-11T00:09:11.310 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:11.307+0000 7fb822e0f740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556372d1e970 0x556372d280e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:09:11.310 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:11.307+0000 7fb822e0f740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556372d212a0 0x556372d28620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:09:11.310 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:11.307+0000 7fb822e0f740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556372d23ce0 0x556372d1b8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:09:11.310 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:11.307+0000 7fb822e0f740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fb8181ad710 con 0x556372d1e970 2024-09-11T00:09:11.310 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:11.307+0000 7fb822e0f740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fb8181af310 con 0x556372d212a0 2024-09-11T00:09:11.310 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:11.307+0000 7fb822e0f740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fb8181adde0 con 0x556372d23ce0 2024-09-11T00:09:11.310 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:11.307+0000 7fb81d400640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556372d1e970 0x556372d280e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:09:11.310 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:11.307+0000 7fb81d400640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556372d1e970 0x556372d280e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:55592/0 (socket says 172.21.15.12:55592) 2024-09-11T00:09:11.310 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:11.307+0000 7fb81d400640 1 -- 172.21.15.12:0/3945828820 learned_addr learned my addr 172.21.15.12:0/3945828820 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:09:11.310 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:11.307+0000 7fb81de00640 1 --2- 172.21.15.12:0/3945828820 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556372d23ce0 0x556372d1b8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:09:11.310 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:11.308+0000 7fb81ca00640 1 --2- 172.21.15.12:0/3945828820 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556372d212a0 0x556372d28620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:09:11.312 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:11.308+0000 7fb81de00640 1 -- 172.21.15.12:0/3945828820 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556372d212a0 msgr2=0x556372d28620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:09:11.312 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:11.308+0000 7fb81de00640 1 --2- 172.21.15.12:0/3945828820 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556372d212a0 0x556372d28620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:09:11.312 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:11.308+0000 7fb81de00640 1 -- 172.21.15.12:0/3945828820 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556372d1e970 msgr2=0x556372d280e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:09:11.312 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:11.308+0000 7fb81de00640 1 --2- 172.21.15.12:0/3945828820 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556372d1e970 0x556372d280e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:09:11.312 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:11.308+0000 7fb81de00640 1 -- 172.21.15.12:0/3945828820 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556372d29650 con 0x556372d23ce0 2024-09-11T00:09:11.312 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:11.308+0000 7fb81d400640 1 --2- 172.21.15.12:0/3945828820 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556372d1e970 0x556372d280e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:09:11.312 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:11.308+0000 7fb81de00640 1 --2- 172.21.15.12:0/3945828820 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556372d23ce0 0x556372d1b8f0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fb810009fd0 tx=0x7fb810016000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9d524f4b98613ce3 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:09:11.312 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:11.309+0000 7fb809c00640 1 -- 172.21.15.12:0/3945828820 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb810004d20 con 0x556372d23ce0 2024-09-11T00:09:11.313 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:11.309+0000 7fb809c00640 1 -- 172.21.15.12:0/3945828820 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb810021aa0 con 0x556372d23ce0 2024-09-11T00:09:11.313 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:11.309+0000 7fb809c00640 1 -- 172.21.15.12:0/3945828820 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8100056f0 con 0x556372d23ce0 2024-09-11T00:09:11.313 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:11.309+0000 7fb822e0f740 1 -- 172.21.15.12:0/3945828820 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556372d23ce0 msgr2=0x556372d1b8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:09:11.313 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:11.309+0000 7fb822e0f740 1 --2- 172.21.15.12:0/3945828820 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556372d23ce0 0x556372d1b8f0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fb810009fd0 tx=0x7fb810016000 comp rx=0 tx=0).stop 2024-09-11T00:09:11.313 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:11.310+0000 7fb822e0f740 1 -- 172.21.15.12:0/3945828820 shutdown_connections 2024-09-11T00:09:11.313 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:11.310+0000 7fb822e0f740 1 --2- 172.21.15.12:0/3945828820 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556372d23ce0 0x556372d1b8f0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:09:11.313 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:11.310+0000 7fb822e0f740 1 --2- 172.21.15.12:0/3945828820 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556372d212a0 0x556372d28620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:09:11.313 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:11.310+0000 7fb822e0f740 1 --2- 172.21.15.12:0/3945828820 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556372d1e970 0x556372d280e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:09:11.313 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:11.310+0000 7fb822e0f740 1 -- 172.21.15.12:0/3945828820 >> 172.21.15.12:0/3945828820 conn(0x556372d19ac0 msgr2=0x556372d240c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:09:11.313 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:11.310+0000 7fb822e0f740 1 -- 172.21.15.12:0/3945828820 shutdown_connections 2024-09-11T00:09:11.313 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:11.310+0000 7fb822e0f740 1 -- 172.21.15.12:0/3945828820 wait complete. 2024-09-11T00:09:11.313 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:11.310+0000 7fb822e0f740 1 -- start start 2024-09-11T00:09:11.313 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:11.310+0000 7fb822e0f740 1 Processor -- start 2024-09-11T00:09:11.315 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:11.311+0000 7fb822e0f740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556372d1e970 0x556372d2ee90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:09:11.316 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:11.311+0000 7fb822e0f740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556372d212a0 0x556372d2f3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:09:11.316 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:11.311+0000 7fb822e0f740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556372d23ce0 0x556372d2d9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:09:11.316 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:11.311+0000 7fb822e0f740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fb8181aec40 con 0x556372d23ce0 2024-09-11T00:09:11.316 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:11.311+0000 7fb822e0f740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fb8181ae4b0 con 0x556372d1e970 2024-09-11T00:09:11.316 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:11.311+0000 7fb822e0f740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fb8181ab6f0 con 0x556372d212a0 2024-09-11T00:09:11.316 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:11.311+0000 7fb822e0f740 10 client.0 _ll_register_callbacks cb 0x7fb810009a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:09:11.316 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:11.311+0000 7fb81d400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556372d1e970 0x556372d2ee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:09:11.316 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:11.311+0000 7fb81d400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556372d1e970 0x556372d2ee90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.12:40804/0 (socket says 172.21.15.12:40804) 2024-09-11T00:09:11.316 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:11.311+0000 7fb81d400640 1 -- 172.21.15.12:0/1183025612 learned_addr learned my addr 172.21.15.12:0/1183025612 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:09:11.316 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:11.311+0000 7fb81de00640 1 --2- 172.21.15.12:0/1183025612 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556372d23ce0 0x556372d2d9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:09:11.316 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:11.311+0000 7fb81ca00640 1 --2- 172.21.15.12:0/1183025612 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556372d212a0 0x556372d2f3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:09:11.317 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:11.312+0000 7fb81ca00640 1 -- 172.21.15.12:0/1183025612 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556372d1e970 msgr2=0x556372d2ee90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:09:11.317 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:11.312+0000 7fb81ca00640 1 --2- 172.21.15.12:0/1183025612 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556372d1e970 0x556372d2ee90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:09:11.317 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:11.312+0000 7fb81ca00640 1 -- 172.21.15.12:0/1183025612 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556372d23ce0 msgr2=0x556372d2d9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:09:11.317 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:11.312+0000 7fb81ca00640 1 --2- 172.21.15.12:0/1183025612 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556372d23ce0 0x556372d2d9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:09:11.318 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:11.312+0000 7fb81ca00640 1 -- 172.21.15.12:0/1183025612 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556372d2def0 con 0x556372d212a0 2024-09-11T00:09:11.318 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:11.312+0000 7fb81ca00640 1 --2- 172.21.15.12:0/1183025612 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556372d212a0 0x556372d2f3d0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fb804004cc0 tx=0x7fb80400bf40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=893520fb593e51d6 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:09:11.318 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:11.312+0000 7fb7fea00640 10 client.0 ms_handle_connect on v2:172.21.15.178:3300/0 2024-09-11T00:09:11.318 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:11.313+0000 7fb7fea00640 1 -- 172.21.15.12:0/1183025612 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb804011dd0 con 0x556372d212a0 2024-09-11T00:09:11.318 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:11.313+0000 7fb7fea00640 1 -- 172.21.15.12:0/1183025612 <== mon.1 v2:172.21.15.178:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9208+0+0 (secure 0 0 0) 0x7fb804004660 con 0x556372d212a0 2024-09-11T00:09:11.318 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:11.313+0000 7fb822e0f740 1 -- 172.21.15.12:0/1183025612 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb810021aa0 con 0x556372d212a0 2024-09-11T00:09:11.318 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:11.313+0000 7fb7fea00640 1 -- 172.21.15.12:0/1183025612 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fb804045050 con 0x556372d212a0 2024-09-11T00:09:11.318 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:11.313+0000 7fb822e0f740 10 client.25762 fetch_fsmap learned FSMap version 33 2024-09-11T00:09:11.318 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:11.313+0000 7fb822e0f740 1 -- 172.21.15.12:0/1183025612 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x556372da0b50 con 0x556372d212a0 2024-09-11T00:09:11.318 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:11.313+0000 7fb7fea00640 1 -- 172.21.15.12:0/1183025612 <== mon.1 v2:172.21.15.178:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb810021aa0 con 0x556372d212a0 2024-09-11T00:09:11.319 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:11.314+0000 7fb7fea00640 1 -- 172.21.15.12:0/1183025612 <== mon.1 v2:172.21.15.178:3300/0 5 ==== fsmap(e 33) ==== 8386+0+0 (secure 0 0 0) 0x7fb80401c480 con 0x556372d212a0 2024-09-11T00:09:11.319 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:11.314+0000 7fb822e0f740 10 client.25762 fetch_fsmap finished waiting for FSMap version 33 2024-09-11T00:09:11.319 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:11.314+0000 7fb822e0f740 10 client.25762 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T00:09:11.319 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:11.314+0000 7fb822e0f740 20 client.25762 populate_metadata read hostname 'smithi012' 2024-09-11T00:09:11.319 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:11.314+0000 7fb822e0f740 1 --2- 172.21.15.12:0/1183025612 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x556372da1530 0x556372da3950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:09:11.319 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:11.314+0000 7fb822e0f740 4 client.25762 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T00:09:11.319 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:11.314+0000 7fb822e0f740 1 -- 172.21.15.12:0/1183025612 --> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] -- command(tid 0: {"prefix": "flush journal"}) -- 0x556372c8f0b0 con 0x556372da1530 2024-09-11T00:09:11.319 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:11.314+0000 7fb81d400640 1 --2- 172.21.15.12:0/1183025612 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x556372da1530 0x556372da3950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:09:11.320 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:11.315+0000 7fb81d400640 1 --2- 172.21.15.12:0/1183025612 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x556372da1530 0x556372da3950 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=94d9454e354aac4 server_cookie=ef66176b71828f46 in_seq=0 out_seq=0 2024-09-11T00:09:11.320 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:11.315+0000 7fb7fea00640 10 client.25762 ms_handle_connect on v2:172.21.15.165:6838/501312984 2024-09-11T00:09:11.355 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:11.353+0000 7f1127e00640 1 -- 172.21.15.12:0/2277356914 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f11080055b0 con 0x7f11280754e0 2024-09-11T00:09:11.356 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:11.354+0000 7fb7fea00640 1 -- 172.21.15.12:0/1183025612 <== mds.0 v2:172.21.15.165:6838/501312984 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x556372c8f0b0 con 0x556372da1530 2024-09-11T00:09:11.356 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:11.354+0000 7fb7fea00640 10 client.25762 handle_command_reply: tid=0 2024-09-11T00:09:11.356 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:11.354+0000 7fb822e0f740 1 client.25762 shutdown 2024-09-11T00:09:11.356 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:11.354+0000 7fb822e0f740 1 -- 172.21.15.12:0/1183025612 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556372d212a0 msgr2=0x556372d2f3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:09:11.356 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:11.354+0000 7fb822e0f740 1 --2- 172.21.15.12:0/1183025612 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556372d212a0 0x556372d2f3d0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fb804004cc0 tx=0x7fb80400bf40 comp rx=0 tx=0).stop 2024-09-11T00:09:11.356 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:11.354+0000 7f111f400640 1 -- 172.21.15.12:0/2277356914 <== mon.2 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) ==== 94+0+11972 (secure 0 0 0) 0x7f11180ac430 con 0x7f11280754e0 2024-09-11T00:09:11.357 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-11T00:09:11.357 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:11.354+0000 7fb822e0f740 1 -- 172.21.15.12:0/1183025612 shutdown_connections 2024-09-11T00:09:11.357 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":33,"btime":"2024-09-11T00:08:46:706969+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14577,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6835/447171591","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":447171591},{"type":"v1","addr":"172.21.15.12:6835","nonce":447171591}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14589,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/901486040","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":901486040},{"type":"v1","addr":"172.21.15.12:6837","nonce":901486040}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":33,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T00:04:04.739881+0000","modified":"2024-09-11T00:08:46.706950+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24674,"mds_1":24643,"mds_2":24638,"mds_3":24637,"mds_4":24614},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14631":{"gid":14631,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.12:6841/3069919437","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6840","nonce":3069919437},{"type":"v1","addr":"172.21.15.12:6841","nonce":3069919437}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24562":{"gid":24562,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6833/1141636112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":1141636112},{"type":"v1","addr":"172.21.15.178:6833","nonce":1141636112}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24581":{"gid":24581,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6833/3111980970","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6832","nonce":3111980970},{"type":"v1","addr":"172.21.15.165:6833","nonce":3111980970}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24607":{"gid":24607,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6835/4084868287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":4084868287},{"type":"v1","addr":"172.21.15.178:6835","nonce":4084868287}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24614":{"gid":24614,"name":"e","rank":4,"incarnation":26,"state":"up:active","state_seq":21,"addr":"172.21.15.165:6835/2425395274","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":2425395274},{"type":"v1","addr":"172.21.15.165:6835","nonce":2425395274}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24637":{"gid":24637,"name":"i","rank":3,"incarnation":24,"state":"up:active","state_seq":18,"addr":"172.21.15.178:6839/70628097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6838","nonce":70628097},{"type":"v1","addr":"172.21.15.178:6839","nonce":70628097}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24638":{"gid":24638,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":18,"addr":"172.21.15.165:6837/6215088","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6836","nonce":6215088},{"type":"v1","addr":"172.21.15.165:6837","nonce":6215088}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24643":{"gid":24643,"name":"g","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.12:6839/3124093009","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6838","nonce":3124093009},{"type":"v1","addr":"172.21.15.12:6839","nonce":3124093009}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24674":{"gid":24674,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.165:6839/501312984","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6838","nonce":501312984},{"type":"v1","addr":"172.21.15.165:6839","nonce":501312984}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25672":{"gid":25672,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6837/308601741","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":308601741},{"type":"v1","addr":"172.21.15.178:6837","nonce":308601741}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24674,"qdb_cluster":[24614,24637,24638,24643,24674]},"id":1}]} 2024-09-11T00:09:11.358 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 33 2024-09-11T00:09:11.358 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:11.354+0000 7fb822e0f740 1 --2- 172.21.15.12:0/1183025612 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x556372da1530 0x556372da3950 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:09:11.358 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:11.354+0000 7fb822e0f740 1 --2- 172.21.15.12:0/1183025612 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556372d23ce0 0x556372d2d9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:09:11.358 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:11.354+0000 7fb822e0f740 1 --2- 172.21.15.12:0/1183025612 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556372d212a0 0x556372d2f3d0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:09:11.358 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:11.354+0000 7fb822e0f740 1 --2- 172.21.15.12:0/1183025612 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556372d1e970 0x556372d2ee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:09:11.358 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:11.354+0000 7fb7fea00640 0 client.25762 ms_handle_reset on v2:172.21.15.165:6838/501312984 2024-09-11T00:09:11.358 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:11.354+0000 7fb822e0f740 1 -- 172.21.15.12:0/1183025612 >> 172.21.15.12:0/1183025612 conn(0x556372d19ac0 msgr2=0x556372d1a320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:09:11.359 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:11.354+0000 7fb822e0f740 1 -- 172.21.15.12:0/1183025612 shutdown_connections 2024-09-11T00:09:11.359 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:11.354+0000 7fb822e0f740 1 -- 172.21.15.12:0/1183025612 wait complete. 2024-09-11T00:09:11.359 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:11.354+0000 7fb822e0f740 20 client.25762 trim_cache size 0 max 16384 2024-09-11T00:09:11.359 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:{ 2024-09-11T00:09:11.359 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "message": "", 2024-09-11T00:09:11.359 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "return_code": 0 2024-09-11T00:09:11.359 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:} 2024-09-11T00:09:11.360 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:11.355+0000 7fb81e800640 1 -- 172.21.15.12:0/860780313 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fb8000b52e0 msgr2=0x7fb8000b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:09:11.360 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:11.355+0000 7fb81e800640 1 --2- 172.21.15.12:0/860780313 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fb8000b52e0 0x7fb8000b77a0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fb810005f40 tx=0x7fb810009600 comp rx=0 tx=0).stop 2024-09-11T00:09:11.360 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:11.355+0000 7fb81e800640 1 -- 172.21.15.12:0/860780313 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb81806b470 msgr2=0x7fb818102aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:09:11.360 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:11.355+0000 7fb81e800640 1 --2- 172.21.15.12:0/860780313 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb81806b470 0x7fb818102aa0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fb80c005940 tx=0x7fb80c004430 comp rx=0 tx=0).stop 2024-09-11T00:09:11.360 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:11.355+0000 7fb81e800640 1 -- 172.21.15.12:0/860780313 shutdown_connections 2024-09-11T00:09:11.360 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:11.355+0000 7fb81e800640 1 --2- 172.21.15.12:0/860780313 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fb8000b52e0 0x7fb8000b77a0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:09:11.360 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:11.355+0000 7fb81e800640 1 --2- 172.21.15.12:0/860780313 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb8181084a0 0x7fb818101840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:09:11.361 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:11.355+0000 7fb81e800640 1 --2- 172.21.15.12:0/860780313 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb8181058f0 0x7fb818101300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:09:11.361 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:11.355+0000 7fb81e800640 1 --2- 172.21.15.12:0/860780313 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb81806b470 0x7fb818102aa0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:09:11.361 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:11.355+0000 7fb81e800640 1 -- 172.21.15.12:0/860780313 >> 172.21.15.12:0/860780313 conn(0x7fb8180fcf30 msgr2=0x7fb818103b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:09:11.361 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:11.355+0000 7fb81e800640 1 -- 172.21.15.12:0/860780313 shutdown_connections 2024-09-11T00:09:11.361 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:11.356+0000 7fb81e800640 1 -- 172.21.15.12:0/860780313 wait complete. 2024-09-11T00:09:11.361 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:11.357+0000 7f1127e00640 1 -- 172.21.15.12:0/2277356914 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f110c0b5410 msgr2=0x7f110c0b78d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:09:11.362 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:11.357+0000 7f1127e00640 1 --2- 172.21.15.12:0/2277356914 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f110c0b5410 0x7f110c0b78d0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f1114031440 tx=0x7f11140095f0 comp rx=0 tx=0).stop 2024-09-11T00:09:11.362 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:11.357+0000 7f1127e00640 1 -- 172.21.15.12:0/2277356914 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f11280754e0 msgr2=0x7f112819b250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:09:11.362 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:11.357+0000 7f1127e00640 1 --2- 172.21.15.12:0/2277356914 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f11280754e0 0x7f112819b250 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f111800e9f0 tx=0x7f111800eeb0 comp rx=0 tx=0).stop 2024-09-11T00:09:11.362 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:11.357+0000 7f1127e00640 1 -- 172.21.15.12:0/2277356914 shutdown_connections 2024-09-11T00:09:11.362 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:11.357+0000 7f1127e00640 1 --2- 172.21.15.12:0/2277356914 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f110c0b5410 0x7f110c0b78d0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:09:11.362 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:11.357+0000 7f1127e00640 1 --2- 172.21.15.12:0/2277356914 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f112810f200 0x7f11281a0b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:09:11.362 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:11.357+0000 7f1127e00640 1 --2- 172.21.15.12:0/2277356914 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1128075e00 0x7f112819b790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:09:11.362 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:11.357+0000 7f1127e00640 1 --2- 172.21.15.12:0/2277356914 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f11280754e0 0x7f112819b250 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:09:11.362 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:11.357+0000 7f1127e00640 1 -- 172.21.15.12:0/2277356914 >> 172.21.15.12:0/2277356914 conn(0x7f11280fe660 msgr2=0x7f11281113b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:09:11.362 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:11.357+0000 7f1127e00640 1 -- 172.21.15.12:0/2277356914 shutdown_connections 2024-09-11T00:09:11.362 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:11.357+0000 7f1127e00640 1 -- 172.21.15.12:0/2277356914 wait complete. 2024-09-11T00:09:11.603 INFO:tasks.workunit.client.0.smithi012.stdout: 1 89448 7.34 MB/sec warmup 68 sec latency 168.628 ms 2024-09-11T00:09:12.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:09:11 smithi165 ceph-mon[35356]: pgmap v462: 129 pgs: 129 active+clean; 26 MiB data, 526 MiB used, 1.0 TiB / 1.0 TiB avail; 254 KiB/s rd, 2.0 MiB/s wr, 147 op/s 2024-09-11T00:09:12.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:09:11 smithi165 ceph-mon[35356]: from='client.? 172.21.15.12:0/2277356914' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T00:09:12.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:09:11 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T00:09:12.063 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:09:11 smithi012 ceph-mon[30333]: pgmap v462: 129 pgs: 129 active+clean; 26 MiB data, 526 MiB used, 1.0 TiB / 1.0 TiB avail; 254 KiB/s rd, 2.0 MiB/s wr, 147 op/s 2024-09-11T00:09:12.063 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:09:11 smithi012 ceph-mon[30333]: from='client.? 172.21.15.12:0/2277356914' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T00:09:12.063 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:09:11 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T00:09:12.080 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:09:11 smithi178 ceph-mon[35919]: pgmap v462: 129 pgs: 129 active+clean; 26 MiB data, 526 MiB used, 1.0 TiB / 1.0 TiB avail; 254 KiB/s rd, 2.0 MiB/s wr, 147 op/s 2024-09-11T00:09:12.080 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:09:11 smithi178 ceph-mon[35919]: from='client.? 172.21.15.12:0/2277356914' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T00:09:12.080 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:09:11 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T00:09:12.232 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24674 quiesce db / --timeout 90 --expiration 140.5 --await-for 45.0' 2024-09-11T00:09:12.233 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid ff3ad74a-6fcf-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24674 quiesce db / --timeout 90 --expiration 140.5 --await-for 45.0 2024-09-11T00:09:12.438 INFO:journalctl@ceph.osd.0.smithi012.stdout:Sep 11 00:09:12 smithi012 sudo[90781]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-11T00:09:12.438 INFO:journalctl@ceph.osd.0.smithi012.stdout:Sep 11 00:09:12 smithi012 sudo[90781]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-11T00:09:12.438 INFO:journalctl@ceph.osd.0.smithi012.stdout:Sep 11 00:09:12 smithi012 sudo[90781]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-11T00:09:12.438 INFO:journalctl@ceph.osd.0.smithi012.stdout:Sep 11 00:09:12 smithi012 sudo[90781]: pam_unix(sudo:session): session closed for user root 2024-09-11T00:09:12.438 INFO:journalctl@ceph.osd.0.smithi012.stdout:Sep 11 00:09:12 smithi012 sudo[90787]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-11T00:09:12.439 INFO:journalctl@ceph.osd.0.smithi012.stdout:Sep 11 00:09:12 smithi012 sudo[90787]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-11T00:09:12.439 INFO:journalctl@ceph.osd.0.smithi012.stdout:Sep 11 00:09:12 smithi012 sudo[90787]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-11T00:09:12.439 INFO:journalctl@ceph.osd.0.smithi012.stdout:Sep 11 00:09:12 smithi012 sudo[90787]: pam_unix(sudo:session): session closed for user root 2024-09-11T00:09:12.532 INFO:journalctl@ceph.osd.1.smithi165.stdout:Sep 11 00:09:12 smithi165 sudo[67860]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-11T00:09:12.532 INFO:journalctl@ceph.osd.1.smithi165.stdout:Sep 11 00:09:12 smithi165 sudo[67860]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-11T00:09:12.532 INFO:journalctl@ceph.osd.1.smithi165.stdout:Sep 11 00:09:12 smithi165 sudo[67860]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-11T00:09:12.532 INFO:journalctl@ceph.osd.1.smithi165.stdout:Sep 11 00:09:12 smithi165 sudo[67860]: pam_unix(sudo:session): session closed for user root 2024-09-11T00:09:12.533 INFO:journalctl@ceph.osd.1.smithi165.stdout:Sep 11 00:09:12 smithi165 sudo[67863]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-11T00:09:12.533 INFO:journalctl@ceph.osd.1.smithi165.stdout:Sep 11 00:09:12 smithi165 sudo[67863]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-11T00:09:12.533 INFO:journalctl@ceph.osd.1.smithi165.stdout:Sep 11 00:09:12 smithi165 sudo[67863]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-11T00:09:12.533 INFO:journalctl@ceph.osd.1.smithi165.stdout:Sep 11 00:09:12 smithi165 sudo[67863]: pam_unix(sudo:session): session closed for user root 2024-09-11T00:09:12.580 INFO:journalctl@ceph.osd.2.smithi178.stdout:Sep 11 00:09:12 smithi178 sudo[68766]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-11T00:09:12.581 INFO:journalctl@ceph.osd.2.smithi178.stdout:Sep 11 00:09:12 smithi178 sudo[68766]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-11T00:09:12.581 INFO:journalctl@ceph.osd.2.smithi178.stdout:Sep 11 00:09:12 smithi178 sudo[68766]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-11T00:09:12.581 INFO:journalctl@ceph.osd.2.smithi178.stdout:Sep 11 00:09:12 smithi178 sudo[68766]: pam_unix(sudo:session): session closed for user root 2024-09-11T00:09:12.581 INFO:journalctl@ceph.osd.2.smithi178.stdout:Sep 11 00:09:12 smithi178 sudo[68769]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-11T00:09:12.581 INFO:journalctl@ceph.osd.2.smithi178.stdout:Sep 11 00:09:12 smithi178 sudo[68769]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-11T00:09:12.581 INFO:journalctl@ceph.osd.2.smithi178.stdout:Sep 11 00:09:12 smithi178 sudo[68769]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-11T00:09:12.581 INFO:journalctl@ceph.osd.2.smithi178.stdout:Sep 11 00:09:12 smithi178 sudo[68769]: pam_unix(sudo:session): session closed for user root 2024-09-11T00:09:12.604 INFO:tasks.workunit.client.0.smithi012.stdout: 1 89448 7.24 MB/sec warmup 69 sec latency 1169.029 ms 2024-09-11T00:09:12.673 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:09:12.708 INFO:journalctl@ceph.osd.3.smithi012.stdout:Sep 11 00:09:12 smithi012 sudo[90822]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-11T00:09:12.708 INFO:journalctl@ceph.osd.3.smithi012.stdout:Sep 11 00:09:12 smithi012 sudo[90822]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-11T00:09:12.709 INFO:journalctl@ceph.osd.3.smithi012.stdout:Sep 11 00:09:12 smithi012 sudo[90822]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-11T00:09:12.709 INFO:journalctl@ceph.osd.3.smithi012.stdout:Sep 11 00:09:12 smithi012 sudo[90822]: pam_unix(sudo:session): session closed for user root 2024-09-11T00:09:12.709 INFO:journalctl@ceph.osd.3.smithi012.stdout:Sep 11 00:09:12 smithi012 sudo[90825]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-11T00:09:12.709 INFO:journalctl@ceph.osd.3.smithi012.stdout:Sep 11 00:09:12 smithi012 sudo[90825]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-11T00:09:12.709 INFO:journalctl@ceph.osd.3.smithi012.stdout:Sep 11 00:09:12 smithi012 sudo[90825]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-11T00:09:12.709 INFO:journalctl@ceph.osd.3.smithi012.stdout:Sep 11 00:09:12 smithi012 sudo[90825]: pam_unix(sudo:session): session closed for user root 2024-09-11T00:09:12.709 INFO:journalctl@ceph.osd.6.smithi012.stdout:Sep 11 00:09:12 smithi012 sudo[90890]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-11T00:09:12.709 INFO:journalctl@ceph.osd.6.smithi012.stdout:Sep 11 00:09:12 smithi012 sudo[90890]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-11T00:09:12.709 INFO:journalctl@ceph.osd.6.smithi012.stdout:Sep 11 00:09:12 smithi012 sudo[90890]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-11T00:09:12.710 INFO:journalctl@ceph.osd.6.smithi012.stdout:Sep 11 00:09:12 smithi012 sudo[90890]: pam_unix(sudo:session): session closed for user root 2024-09-11T00:09:12.932 INFO:journalctl@ceph.osd.7.smithi165.stdout:Sep 11 00:09:12 smithi165 sudo[67904]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-11T00:09:12.932 INFO:journalctl@ceph.osd.7.smithi165.stdout:Sep 11 00:09:12 smithi165 sudo[67904]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-11T00:09:12.933 INFO:journalctl@ceph.osd.7.smithi165.stdout:Sep 11 00:09:12 smithi165 sudo[67904]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-11T00:09:12.933 INFO:journalctl@ceph.osd.7.smithi165.stdout:Sep 11 00:09:12 smithi165 sudo[67904]: pam_unix(sudo:session): session closed for user root 2024-09-11T00:09:12.933 INFO:journalctl@ceph.osd.7.smithi165.stdout:Sep 11 00:09:12 smithi165 sudo[67907]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-11T00:09:12.933 INFO:journalctl@ceph.osd.7.smithi165.stdout:Sep 11 00:09:12 smithi165 sudo[67907]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-11T00:09:12.933 INFO:journalctl@ceph.osd.7.smithi165.stdout:Sep 11 00:09:12 smithi165 sudo[67907]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-11T00:09:12.933 INFO:journalctl@ceph.osd.7.smithi165.stdout:Sep 11 00:09:12 smithi165 sudo[67907]: pam_unix(sudo:session): session closed for user root 2024-09-11T00:09:12.933 INFO:journalctl@ceph.osd.4.smithi165.stdout:Sep 11 00:09:12 smithi165 sudo[67866]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-11T00:09:12.933 INFO:journalctl@ceph.osd.4.smithi165.stdout:Sep 11 00:09:12 smithi165 sudo[67866]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-11T00:09:12.933 INFO:journalctl@ceph.osd.4.smithi165.stdout:Sep 11 00:09:12 smithi165 sudo[67866]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-11T00:09:12.933 INFO:journalctl@ceph.osd.4.smithi165.stdout:Sep 11 00:09:12 smithi165 sudo[67866]: pam_unix(sudo:session): session closed for user root 2024-09-11T00:09:12.934 INFO:journalctl@ceph.osd.4.smithi165.stdout:Sep 11 00:09:12 smithi165 sudo[67869]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-11T00:09:12.934 INFO:journalctl@ceph.osd.4.smithi165.stdout:Sep 11 00:09:12 smithi165 sudo[67869]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-11T00:09:12.934 INFO:journalctl@ceph.osd.4.smithi165.stdout:Sep 11 00:09:12 smithi165 sudo[67869]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-11T00:09:12.934 INFO:journalctl@ceph.osd.4.smithi165.stdout:Sep 11 00:09:12 smithi165 sudo[67869]: pam_unix(sudo:session): session closed for user root 2024-09-11T00:09:12.974 INFO:journalctl@ceph.osd.6.smithi012.stdout:Sep 11 00:09:12 smithi012 sudo[90907]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-11T00:09:12.974 INFO:journalctl@ceph.osd.6.smithi012.stdout:Sep 11 00:09:12 smithi012 sudo[90907]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-11T00:09:12.974 INFO:journalctl@ceph.osd.6.smithi012.stdout:Sep 11 00:09:12 smithi012 sudo[90907]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-11T00:09:12.974 INFO:journalctl@ceph.osd.6.smithi012.stdout:Sep 11 00:09:12 smithi012 sudo[90907]: pam_unix(sudo:session): session closed for user root 2024-09-11T00:09:12.974 INFO:journalctl@ceph.osd.9.smithi012.stdout:Sep 11 00:09:12 smithi012 sudo[90917]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-11T00:09:12.974 INFO:journalctl@ceph.osd.9.smithi012.stdout:Sep 11 00:09:12 smithi012 sudo[90917]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-11T00:09:12.974 INFO:journalctl@ceph.osd.9.smithi012.stdout:Sep 11 00:09:12 smithi012 sudo[90917]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-11T00:09:12.975 INFO:journalctl@ceph.osd.9.smithi012.stdout:Sep 11 00:09:12 smithi012 sudo[90917]: pam_unix(sudo:session): session closed for user root 2024-09-11T00:09:12.975 INFO:journalctl@ceph.osd.9.smithi012.stdout:Sep 11 00:09:12 smithi012 sudo[90920]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-11T00:09:12.975 INFO:journalctl@ceph.osd.9.smithi012.stdout:Sep 11 00:09:12 smithi012 sudo[90920]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-11T00:09:12.975 INFO:journalctl@ceph.osd.9.smithi012.stdout:Sep 11 00:09:12 smithi012 sudo[90920]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-11T00:09:12.975 INFO:journalctl@ceph.osd.9.smithi012.stdout:Sep 11 00:09:12 smithi012 sudo[90920]: pam_unix(sudo:session): session closed for user root 2024-09-11T00:09:12.977 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 11 00:09:12 smithi178 sudo[68811]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-11T00:09:12.977 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 11 00:09:12 smithi178 sudo[68811]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-11T00:09:12.977 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 11 00:09:12 smithi178 sudo[68811]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-11T00:09:12.978 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 11 00:09:12 smithi178 sudo[68811]: pam_unix(sudo:session): session closed for user root 2024-09-11T00:09:12.978 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 11 00:09:12 smithi178 sudo[68814]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-11T00:09:12.978 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 11 00:09:12 smithi178 sudo[68814]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-11T00:09:12.978 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 11 00:09:12 smithi178 sudo[68814]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-11T00:09:12.978 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 11 00:09:12 smithi178 sudo[68814]: pam_unix(sudo:session): session closed for user root 2024-09-11T00:09:12.978 INFO:journalctl@ceph.osd.5.smithi178.stdout:Sep 11 00:09:12 smithi178 sudo[68772]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-11T00:09:12.978 INFO:journalctl@ceph.osd.5.smithi178.stdout:Sep 11 00:09:12 smithi178 sudo[68772]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-11T00:09:12.979 INFO:journalctl@ceph.osd.5.smithi178.stdout:Sep 11 00:09:12 smithi178 sudo[68772]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-11T00:09:12.979 INFO:journalctl@ceph.osd.5.smithi178.stdout:Sep 11 00:09:12 smithi178 sudo[68772]: pam_unix(sudo:session): session closed for user root 2024-09-11T00:09:12.979 INFO:journalctl@ceph.osd.5.smithi178.stdout:Sep 11 00:09:12 smithi178 sudo[68775]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-11T00:09:12.979 INFO:journalctl@ceph.osd.5.smithi178.stdout:Sep 11 00:09:12 smithi178 sudo[68775]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-11T00:09:12.979 INFO:journalctl@ceph.osd.5.smithi178.stdout:Sep 11 00:09:12 smithi178 sudo[68775]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-11T00:09:12.979 INFO:journalctl@ceph.osd.5.smithi178.stdout:Sep 11 00:09:12 smithi178 sudo[68775]: pam_unix(sudo:session): session closed for user root 2024-09-11T00:09:13.227 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:13.225+0000 7f6731200640 1 -- 172.21.15.12:0/1851727476 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f672c077ff0 msgr2=0x7f672c1132a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:09:13.228 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:13.225+0000 7f6731200640 1 --2- 172.21.15.12:0/1851727476 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f672c077ff0 0x7f672c1132a0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f6714009900 tx=0x7f671402a410 comp rx=0 tx=0).stop 2024-09-11T00:09:13.228 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:13.225+0000 7f6731200640 1 -- 172.21.15.12:0/1851727476 shutdown_connections 2024-09-11T00:09:13.228 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:13.225+0000 7f6731200640 1 --2- 172.21.15.12:0/1851727476 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f672c1137e0 0x7f672c115bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:09:13.228 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:13.225+0000 7f6731200640 1 --2- 172.21.15.12:0/1851727476 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f672c077ff0 0x7f672c1132a0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:09:13.228 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:13.225+0000 7f6731200640 1 --2- 172.21.15.12:0/1851727476 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f672c0776d0 0x7f672c077ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:09:13.228 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:13.225+0000 7f6731200640 1 -- 172.21.15.12:0/1851727476 >> 172.21.15.12:0/1851727476 conn(0x7f672c100840 msgr2=0x7f672c102c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:09:13.228 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:13.225+0000 7f6731200640 1 -- 172.21.15.12:0/1851727476 shutdown_connections 2024-09-11T00:09:13.228 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:13.225+0000 7f6731200640 1 -- 172.21.15.12:0/1851727476 wait complete. 2024-09-11T00:09:13.229 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:09:13 smithi012 sudo[90927]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-11T00:09:13.229 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:09:13 smithi012 sudo[90927]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-11T00:09:13.229 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:09:13 smithi012 sudo[90927]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-11T00:09:13.229 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:09:13 smithi012 sudo[90927]: pam_unix(sudo:session): session closed for user root 2024-09-11T00:09:13.229 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:09:13 smithi012 sudo[90930]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-11T00:09:13.229 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:09:13 smithi012 sudo[90930]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-11T00:09:13.229 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:09:13 smithi012 sudo[90930]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-11T00:09:13.229 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:09:13 smithi012 sudo[90930]: pam_unix(sudo:session): session closed for user root 2024-09-11T00:09:13.230 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:13.226+0000 7f6731200640 1 Processor -- start 2024-09-11T00:09:13.230 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:13.226+0000 7f6731200640 1 -- start start 2024-09-11T00:09:13.230 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:13.227+0000 7f6731200640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f672c0776d0 0x7f672c19f600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:09:13.230 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:13.227+0000 7f6731200640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f672c077ff0 0x7f672c19fb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:09:13.231 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:13.227+0000 7f6731200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f672c1137e0 0x7f672c1a4ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:09:13.231 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:13.227+0000 7f6731200640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f672c1191d0 con 0x7f672c077ff0 2024-09-11T00:09:13.231 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:13.227+0000 7f6731200640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f672c119050 con 0x7f672c1137e0 2024-09-11T00:09:13.231 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:13.227+0000 7f6731200640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f672c119350 con 0x7f672c0776d0 2024-09-11T00:09:13.231 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:13.227+0000 7f672b400640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f672c0776d0 0x7f672c19f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:09:13.231 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:13.227+0000 7f672b400640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f672c0776d0 0x7f672c19f600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.12:33930/0 (socket says 172.21.15.12:33930) 2024-09-11T00:09:13.231 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:13.227+0000 7f672b400640 1 -- 172.21.15.12:0/494838590 learned_addr learned my addr 172.21.15.12:0/494838590 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:09:13.231 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:13.227+0000 7f672be00640 1 --2- 172.21.15.12:0/494838590 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f672c1137e0 0x7f672c1a4ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:09:13.231 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:13.227+0000 7f6722a00640 1 --2- 172.21.15.12:0/494838590 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f672c077ff0 0x7f672c19fb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:09:13.231 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:13.227+0000 7f672b400640 1 -- 172.21.15.12:0/494838590 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f672c1137e0 msgr2=0x7f672c1a4ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:09:13.231 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:13.227+0000 7f672b400640 1 --2- 172.21.15.12:0/494838590 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f672c1137e0 0x7f672c1a4ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:09:13.231 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:13.227+0000 7f672b400640 1 -- 172.21.15.12:0/494838590 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f672c077ff0 msgr2=0x7f672c19fb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:09:13.231 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:13.228+0000 7f672b400640 1 --2- 172.21.15.12:0/494838590 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f672c077ff0 0x7f672c19fb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:09:13.232 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:13.228+0000 7f672b400640 1 -- 172.21.15.12:0/494838590 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f672c1a56b0 con 0x7f672c0776d0 2024-09-11T00:09:13.232 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:13.228+0000 7f672be00640 1 --2- 172.21.15.12:0/494838590 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f672c1137e0 0x7f672c1a4ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:09:13.232 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:13.228+0000 7f6722a00640 1 --2- 172.21.15.12:0/494838590 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f672c077ff0 0x7f672c19fb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T00:09:13.232 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:13.228+0000 7f672b400640 1 --2- 172.21.15.12:0/494838590 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f672c0776d0 0x7f672c19f600 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f671c00e9f0 tx=0x7f671c00eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:09:13.232 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:13.228+0000 7f6728c00640 1 -- 172.21.15.12:0/494838590 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f671c00cd80 con 0x7f672c0776d0 2024-09-11T00:09:13.232 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:13.228+0000 7f6731200640 1 -- 172.21.15.12:0/494838590 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f672c1a59a0 con 0x7f672c0776d0 2024-09-11T00:09:13.232 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:13.228+0000 7f6728c00640 1 -- 172.21.15.12:0/494838590 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f671c0108e0 con 0x7f672c0776d0 2024-09-11T00:09:13.232 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:13.228+0000 7f6728c00640 1 -- 172.21.15.12:0/494838590 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f671c018980 con 0x7f672c0776d0 2024-09-11T00:09:13.232 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:13.229+0000 7f6731200640 1 -- 172.21.15.12:0/494838590 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f672c1ac230 con 0x7f672c0776d0 2024-09-11T00:09:13.233 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:13.232+0000 7f6735456740 1 Processor -- start 2024-09-11T00:09:13.234 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:13.232+0000 7f6735456740 1 -- start start 2024-09-11T00:09:13.234 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:13.232+0000 7f6735456740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563d9d5202f0 0x563d9d5206d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:09:13.235 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:13.232+0000 7f6735456740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563d9d520c10 0x563d9d52c660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:09:13.235 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:13.232+0000 7f6735456740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563d9d52cba0 0x563d9d52ef90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:09:13.235 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:13.232+0000 7f6735456740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x563d9d495ca0 con 0x563d9d5202f0 2024-09-11T00:09:13.235 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:13.232+0000 7f6735456740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x563d9d4850d0 con 0x563d9d52cba0 2024-09-11T00:09:13.235 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:13.232+0000 7f6735456740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x563d9d488ee0 con 0x563d9d520c10 2024-09-11T00:09:13.237 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:13.232+0000 7f672be00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563d9d5202f0 0x563d9d5206d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:09:13.237 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:13.233+0000 7f6722a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563d9d52cba0 0x563d9d52ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:09:13.238 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:13.233+0000 7f672be00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563d9d5202f0 0x563d9d5206d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:55642/0 (socket says 172.21.15.12:55642) 2024-09-11T00:09:13.238 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:13.233+0000 7f672be00640 1 -- 172.21.15.12:0/247738048 learned_addr learned my addr 172.21.15.12:0/247738048 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:09:13.238 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:13.233+0000 7f6728c00640 1 -- 172.21.15.12:0/494838590 <== mon.1 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7f671c010430 con 0x7f672c0776d0 2024-09-11T00:09:13.238 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:13.233+0000 7f672b400640 1 --2- 172.21.15.12:0/247738048 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563d9d520c10 0x563d9d52c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:09:13.238 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:13.233+0000 7f6722a00640 1 -- 172.21.15.12:0/247738048 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563d9d520c10 msgr2=0x563d9d52c660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:09:13.238 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:13.233+0000 7f6722a00640 1 --2- 172.21.15.12:0/247738048 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563d9d520c10 0x563d9d52c660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:09:13.238 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:13.233+0000 7f6722a00640 1 -- 172.21.15.12:0/247738048 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563d9d5202f0 msgr2=0x563d9d5206d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:09:13.238 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:13.233+0000 7f6722a00640 1 --2- 172.21.15.12:0/247738048 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563d9d5202f0 0x563d9d5206d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:09:13.238 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:13.233+0000 7f6722a00640 1 -- 172.21.15.12:0/247738048 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563d9d437400 con 0x563d9d52cba0 2024-09-11T00:09:13.238 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:13.233+0000 7f672be00640 1 --2- 172.21.15.12:0/247738048 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563d9d5202f0 0x563d9d5206d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:09:13.238 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:13.233+0000 7f6722a00640 1 --2- 172.21.15.12:0/247738048 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563d9d52cba0 0x563d9d52ef90 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f6714009230 tx=0x7f6714032c20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c18a282ba4813616 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:09:13.238 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:13.234+0000 7f6720c00640 1 -- 172.21.15.12:0/247738048 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f671403f070 con 0x563d9d52cba0 2024-09-11T00:09:13.239 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:13.234+0000 7f6720c00640 1 -- 172.21.15.12:0/247738048 <== mon.2 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6714032e10 con 0x563d9d52cba0 2024-09-11T00:09:13.239 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:13.234+0000 7f6720c00640 1 -- 172.21.15.12:0/247738048 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6714035070 con 0x563d9d52cba0 2024-09-11T00:09:13.239 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:13.234+0000 7f6728c00640 1 --2- 172.21.15.12:0/494838590 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f67100b5410 0x7f67100b78d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:09:13.239 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:13.235+0000 7f6728c00640 1 -- 172.21.15.12:0/494838590 <== mon.1 v2:172.21.15.178:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9208+0+0 (secure 0 0 0) 0x7f671c0e63a0 con 0x7f672c0776d0 2024-09-11T00:09:13.239 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:13.235+0000 7f672be00640 1 --2- 172.21.15.12:0/494838590 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f67100b5410 0x7f67100b78d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:09:13.239 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:13.235+0000 7f6735456740 1 -- 172.21.15.12:0/247738048 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563d9d52cba0 msgr2=0x563d9d52ef90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:09:13.239 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:13.235+0000 7f6735456740 1 --2- 172.21.15.12:0/247738048 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563d9d52cba0 0x563d9d52ef90 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f6714009230 tx=0x7f6714032c20 comp rx=0 tx=0).stop 2024-09-11T00:09:13.239 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:13.235+0000 7f6735456740 1 -- 172.21.15.12:0/247738048 shutdown_connections 2024-09-11T00:09:13.239 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:13.235+0000 7f6735456740 1 --2- 172.21.15.12:0/247738048 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563d9d52cba0 0x563d9d52ef90 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:09:13.239 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:13.235+0000 7f6735456740 1 --2- 172.21.15.12:0/247738048 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563d9d520c10 0x563d9d52c660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:09:13.239 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:13.235+0000 7f6735456740 1 --2- 172.21.15.12:0/247738048 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563d9d5202f0 0x563d9d5206d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:09:13.239 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:13.235+0000 7f6735456740 1 -- 172.21.15.12:0/247738048 >> 172.21.15.12:0/247738048 conn(0x563d9d51e120 msgr2=0x563d9d51f2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:09:13.239 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:13.235+0000 7f6735456740 1 -- 172.21.15.12:0/247738048 shutdown_connections 2024-09-11T00:09:13.240 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:13.235+0000 7f6735456740 1 -- 172.21.15.12:0/247738048 wait complete. 2024-09-11T00:09:13.240 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:13.236+0000 7f672be00640 1 --2- 172.21.15.12:0/494838590 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f67100b5410 0x7f67100b78d0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f6724009fd0 tx=0x7f6724009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:09:13.242 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:13.236+0000 7f6735456740 1 -- start start 2024-09-11T00:09:13.242 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:13.236+0000 7f6735456740 1 Processor -- start 2024-09-11T00:09:13.242 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:13.237+0000 7f6735456740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563d9d5202f0 0x563d9d52c490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:09:13.242 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:13.237+0000 7f6735456740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563d9d520c10 0x563d9d5298e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:09:13.242 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:13.237+0000 7f6735456740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563d9d52cba0 0x563d9d529f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:09:13.242 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:13.237+0000 7f6735456740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x563d9d2e67a0 con 0x563d9d5202f0 2024-09-11T00:09:13.243 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:13.237+0000 7f6735456740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x563d9d42aa10 con 0x563d9d52cba0 2024-09-11T00:09:13.243 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:13.237+0000 7f6735456740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x563d9d52f7c0 con 0x563d9d520c10 2024-09-11T00:09:13.243 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:13.237+0000 7f6735456740 10 client.0 _ll_register_callbacks cb 0x563d9d43abe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:09:13.243 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:13.237+0000 7f6722a00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563d9d520c10 0x563d9d5298e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:09:13.243 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:13.237+0000 7f6722a00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563d9d520c10 0x563d9d5298e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.12:33960/0 (socket says 172.21.15.12:33960) 2024-09-11T00:09:13.243 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:13.237+0000 7f6722a00640 1 -- 172.21.15.12:0/1115831802 learned_addr learned my addr 172.21.15.12:0/1115831802 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:09:13.243 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:13.237+0000 7f672b400640 1 --2- 172.21.15.12:0/1115831802 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563d9d5202f0 0x563d9d52c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:09:13.244 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:13.237+0000 7f672be00640 1 --2- 172.21.15.12:0/1115831802 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563d9d52cba0 0x563d9d529f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:09:13.244 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:13.237+0000 7f6722a00640 1 -- 172.21.15.12:0/1115831802 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563d9d52cba0 msgr2=0x563d9d529f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:09:13.244 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:13.237+0000 7f6722a00640 1 --2- 172.21.15.12:0/1115831802 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563d9d52cba0 0x563d9d529f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:09:13.244 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:13.237+0000 7f6722a00640 1 -- 172.21.15.12:0/1115831802 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563d9d5202f0 msgr2=0x563d9d52c490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:09:13.244 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:13.237+0000 7f6722a00640 1 --2- 172.21.15.12:0/1115831802 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563d9d5202f0 0x563d9d52c490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:09:13.244 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:13.237+0000 7f6722a00640 1 -- 172.21.15.12:0/1115831802 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563d9d1f2b90 con 0x563d9d520c10 2024-09-11T00:09:13.244 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:13.238+0000 7f6722a00640 1 --2- 172.21.15.12:0/1115831802 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563d9d520c10 0x563d9d5298e0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f6714031440 tx=0x7f6714031780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=897fe2982feda085 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:09:13.244 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:13.238+0000 7f6719600640 10 client.0 ms_handle_connect on v2:172.21.15.178:3300/0 2024-09-11T00:09:13.244 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:13.238+0000 7f6719600640 1 -- 172.21.15.12:0/1115831802 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f671403f040 con 0x563d9d520c10 2024-09-11T00:09:13.244 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:13.238+0000 7f6719600640 1 -- 172.21.15.12:0/1115831802 <== mon.1 v2:172.21.15.178:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9208+0+0 (secure 0 0 0) 0x7f6714004440 con 0x563d9d520c10 2024-09-11T00:09:13.245 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:13.238+0000 7f6735456740 1 -- 172.21.15.12:0/1115831802 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6714032e10 con 0x563d9d520c10 2024-09-11T00:09:13.245 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:13.238+0000 7f6719600640 1 -- 172.21.15.12:0/1115831802 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f6714055050 con 0x563d9d520c10 2024-09-11T00:09:13.245 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:13.238+0000 7f6735456740 10 client.25777 fetch_fsmap learned FSMap version 33 2024-09-11T00:09:13.245 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:13.238+0000 7f6735456740 1 -- 172.21.15.12:0/1115831802 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x563d9d53b0f0 con 0x563d9d520c10 2024-09-11T00:09:13.245 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:13.239+0000 7f6719600640 1 -- 172.21.15.12:0/1115831802 <== mon.1 v2:172.21.15.178:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6714032e10 con 0x563d9d520c10 2024-09-11T00:09:13.245 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:13.239+0000 7f6719600640 1 -- 172.21.15.12:0/1115831802 <== mon.1 v2:172.21.15.178:3300/0 5 ==== fsmap(e 33) ==== 8386+0+0 (secure 0 0 0) 0x7f671404c760 con 0x563d9d520c10 2024-09-11T00:09:13.245 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:13.239+0000 7f6735456740 10 client.25777 fetch_fsmap finished waiting for FSMap version 33 2024-09-11T00:09:13.245 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:13.239+0000 7f6735456740 10 client.25777 resolve_mds: validated gid 24674 aka daemon mds.b 2024-09-11T00:09:13.245 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:13.239+0000 7f6735456740 20 client.25777 populate_metadata read hostname 'smithi012' 2024-09-11T00:09:13.245 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:13.239+0000 7f6735456740 1 --2- 172.21.15.12:0/1115831802 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x563d9d53c440 0x563d9d53e830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:09:13.245 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:13.239+0000 7f6735456740 4 client.25777 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T00:09:13.245 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:13.239+0000 7f6735456740 1 -- 172.21.15.12:0/1115831802 --> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563d9d497bf0 con 0x563d9d53c440 2024-09-11T00:09:13.245 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:13.239+0000 7f672b400640 1 --2- 172.21.15.12:0/1115831802 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x563d9d53c440 0x563d9d53e830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:09:13.246 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:13.240+0000 7f672b400640 1 --2- 172.21.15.12:0/1115831802 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x563d9d53c440 0x563d9d53e830 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f5cf8d05c32ca403 server_cookie=1bf4e39ae5427a22 in_seq=0 out_seq=0 2024-09-11T00:09:13.246 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:13.240+0000 7f6719600640 10 client.25777 ms_handle_connect on v2:172.21.15.165:6838/501312984 2024-09-11T00:09:13.247 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 00:09:12 smithi178 sudo[68817]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-11T00:09:13.247 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 00:09:12 smithi178 sudo[68817]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-11T00:09:13.247 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 00:09:12 smithi178 sudo[68817]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-11T00:09:13.247 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 00:09:12 smithi178 sudo[68817]: pam_unix(sudo:session): session closed for user root 2024-09-11T00:09:13.247 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 00:09:13 smithi178 sudo[68820]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-11T00:09:13.247 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 00:09:13 smithi178 sudo[68820]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-11T00:09:13.247 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 00:09:13 smithi178 sudo[68820]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-11T00:09:13.247 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 00:09:13 smithi178 sudo[68820]: pam_unix(sudo:session): session closed for user root 2024-09-11T00:09:13.247 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:09:13 smithi165 sudo[67916]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-11T00:09:13.247 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:09:13 smithi165 sudo[67916]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-11T00:09:13.248 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:09:13 smithi165 sudo[67916]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-11T00:09:13.248 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:09:13 smithi165 sudo[67916]: pam_unix(sudo:session): session closed for user root 2024-09-11T00:09:13.248 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:09:13 smithi165 sudo[67919]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-11T00:09:13.248 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:09:13 smithi165 sudo[67919]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-11T00:09:13.248 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:09:13 smithi165 sudo[67919]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-11T00:09:13.248 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:09:13 smithi165 sudo[67919]: pam_unix(sudo:session): session closed for user root 2024-09-11T00:09:13.248 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:09:13 smithi178 sudo[68823]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-11T00:09:13.248 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:09:13 smithi178 sudo[68823]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-11T00:09:13.249 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:09:13 smithi178 sudo[68823]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-11T00:09:13.249 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:09:13 smithi178 sudo[68823]: pam_unix(sudo:session): session closed for user root 2024-09-11T00:09:13.249 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:09:13 smithi178 sudo[68826]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-11T00:09:13.249 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:09:13 smithi178 sudo[68826]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-11T00:09:13.249 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:09:13 smithi178 sudo[68826]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-11T00:09:13.249 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:09:13 smithi178 sudo[68826]: pam_unix(sudo:session): session closed for user root 2024-09-11T00:09:13.249 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 11 00:09:12 smithi165 sudo[67910]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-11T00:09:13.249 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 11 00:09:12 smithi165 sudo[67910]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-11T00:09:13.249 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 11 00:09:12 smithi165 sudo[67910]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-11T00:09:13.249 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 11 00:09:12 smithi165 sudo[67910]: pam_unix(sudo:session): session closed for user root 2024-09-11T00:09:13.249 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 11 00:09:12 smithi165 sudo[67913]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-11T00:09:13.249 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 11 00:09:12 smithi165 sudo[67913]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-11T00:09:13.250 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 11 00:09:12 smithi165 sudo[67913]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-11T00:09:13.250 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 11 00:09:12 smithi165 sudo[67913]: pam_unix(sudo:session): session closed for user root 2024-09-11T00:09:13.250 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:13.240+0000 7f6719600640 1 -- 172.21.15.12:0/1115831802 <== mds.0 v2:172.21.15.165:6838/501312984 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x563d9d497bf0 con 0x563d9d53c440 2024-09-11T00:09:13.250 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:13.240+0000 7f6719600640 10 client.25777 handle_command_reply: tid=0 2024-09-11T00:09:13.251 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:13.241+0000 7f6735456740 1 client.25777 shutdown 2024-09-11T00:09:13.251 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:13.241+0000 7f6735456740 1 -- 172.21.15.12:0/1115831802 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563d9d520c10 msgr2=0x563d9d5298e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:09:13.251 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:13.241+0000 7f6735456740 1 --2- 172.21.15.12:0/1115831802 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563d9d520c10 0x563d9d5298e0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f6714031440 tx=0x7f6714031780 comp rx=0 tx=0).stop 2024-09-11T00:09:13.251 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:13.241+0000 7f6735456740 1 -- 172.21.15.12:0/1115831802 shutdown_connections 2024-09-11T00:09:13.251 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:13.241+0000 7f6735456740 1 --2- 172.21.15.12:0/1115831802 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x563d9d53c440 0x563d9d53e830 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:09:13.251 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:13.241+0000 7f6735456740 1 --2- 172.21.15.12:0/1115831802 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563d9d52cba0 0x563d9d529f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:09:13.251 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:13.241+0000 7f6735456740 1 --2- 172.21.15.12:0/1115831802 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563d9d520c10 0x563d9d5298e0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:09:13.251 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:13.241+0000 7f6735456740 1 --2- 172.21.15.12:0/1115831802 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563d9d5202f0 0x563d9d52c490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:09:13.251 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:13.241+0000 7f6719600640 0 client.25777 ms_handle_reset on v2:172.21.15.165:6838/501312984 2024-09-11T00:09:13.251 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:13.241+0000 7f6735456740 1 -- 172.21.15.12:0/1115831802 >> 172.21.15.12:0/1115831802 conn(0x563d9d51e120 msgr2=0x563d9d523c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:09:13.251 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:13.241+0000 7f6735456740 1 -- 172.21.15.12:0/1115831802 shutdown_connections 2024-09-11T00:09:13.251 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:13.241+0000 7f6735456740 1 -- 172.21.15.12:0/1115831802 wait complete. 2024-09-11T00:09:13.251 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:13.241+0000 7f6735456740 20 client.25777 trim_cache size 0 max 16384 2024-09-11T00:09:13.265 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:13.262+0000 7f6735456740 1 Processor -- start 2024-09-11T00:09:13.265 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:13.263+0000 7f6735456740 1 -- start start 2024-09-11T00:09:13.265 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:13.263+0000 7f6735456740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563d9d5202f0 0x563d9d52b620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:09:13.265 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:13.263+0000 7f6735456740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563d9d520c10 0x563d9d526af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:09:13.265 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:13.263+0000 7f6735456740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563d9d52cba0 0x563d9d527030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:09:13.265 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:13.263+0000 7f6735456740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f672c1b5f40 con 0x563d9d520c10 2024-09-11T00:09:13.265 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:13.263+0000 7f6735456740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f672c1b7b40 con 0x563d9d5202f0 2024-09-11T00:09:13.265 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:13.263+0000 7f6735456740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f672c1b6610 con 0x563d9d52cba0 2024-09-11T00:09:13.266 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:13.263+0000 7f6722a00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563d9d520c10 0x563d9d526af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:09:13.266 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:13.263+0000 7f6722a00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563d9d520c10 0x563d9d526af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:55660/0 (socket says 172.21.15.12:55660) 2024-09-11T00:09:13.266 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:13.263+0000 7f6722a00640 1 -- 172.21.15.12:0/2245725210 learned_addr learned my addr 172.21.15.12:0/2245725210 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:09:13.267 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:13.263+0000 7f672be00640 1 --2- 172.21.15.12:0/2245725210 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563d9d52cba0 0x563d9d527030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:09:13.267 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:13.263+0000 7f672b400640 1 --2- 172.21.15.12:0/2245725210 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563d9d5202f0 0x563d9d52b620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:09:13.267 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:13.264+0000 7f6722a00640 1 -- 172.21.15.12:0/2245725210 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563d9d52cba0 msgr2=0x563d9d527030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:09:13.267 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:13.264+0000 7f6722a00640 1 --2- 172.21.15.12:0/2245725210 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563d9d52cba0 0x563d9d527030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:09:13.267 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:13.264+0000 7f6722a00640 1 -- 172.21.15.12:0/2245725210 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563d9d5202f0 msgr2=0x563d9d52b620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:09:13.267 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:13.264+0000 7f6722a00640 1 --2- 172.21.15.12:0/2245725210 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563d9d5202f0 0x563d9d52b620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:09:13.267 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:13.264+0000 7f6722a00640 1 -- 172.21.15.12:0/2245725210 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563d9d5251f0 con 0x563d9d520c10 2024-09-11T00:09:13.267 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:13.264+0000 7f6722a00640 1 --2- 172.21.15.12:0/2245725210 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563d9d520c10 0x563d9d526af0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f6714009800 tx=0x7f6714005b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b4bc38fe8f9f7bf server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:09:13.267 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:13.264+0000 7f671be00640 1 -- 172.21.15.12:0/2245725210 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f671403f030 con 0x563d9d520c10 2024-09-11T00:09:13.267 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:13.264+0000 7f671be00640 1 -- 172.21.15.12:0/2245725210 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f671404ce50 con 0x563d9d520c10 2024-09-11T00:09:13.267 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:13.264+0000 7f671be00640 1 -- 172.21.15.12:0/2245725210 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6714035040 con 0x563d9d520c10 2024-09-11T00:09:13.269 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:13.265+0000 7f6735456740 1 -- 172.21.15.12:0/2245725210 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563d9d520c10 msgr2=0x563d9d526af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:09:13.269 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:13.265+0000 7f6735456740 1 --2- 172.21.15.12:0/2245725210 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563d9d520c10 0x563d9d526af0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f6714009800 tx=0x7f6714005b50 comp rx=0 tx=0).stop 2024-09-11T00:09:13.269 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:13.265+0000 7f6735456740 1 -- 172.21.15.12:0/2245725210 shutdown_connections 2024-09-11T00:09:13.269 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:13.265+0000 7f6735456740 1 --2- 172.21.15.12:0/2245725210 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563d9d52cba0 0x563d9d527030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:09:13.269 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:13.265+0000 7f6735456740 1 --2- 172.21.15.12:0/2245725210 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563d9d520c10 0x563d9d526af0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:09:13.269 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:13.265+0000 7f6735456740 1 --2- 172.21.15.12:0/2245725210 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563d9d5202f0 0x563d9d52b620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:09:13.269 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:13.265+0000 7f6735456740 1 -- 172.21.15.12:0/2245725210 >> 172.21.15.12:0/2245725210 conn(0x563d9d51e120 msgr2=0x563d9d52af10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:09:13.269 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:13.265+0000 7f6735456740 1 -- 172.21.15.12:0/2245725210 shutdown_connections 2024-09-11T00:09:13.269 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:13.265+0000 7f6735456740 1 -- 172.21.15.12:0/2245725210 wait complete. 2024-09-11T00:09:13.269 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:13.266+0000 7f6735456740 1 -- start start 2024-09-11T00:09:13.270 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:13.266+0000 7f6735456740 1 Processor -- start 2024-09-11T00:09:13.270 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:13.266+0000 7f6735456740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563d9d5202f0 0x563d9d540970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:09:13.270 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:13.266+0000 7f6735456740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563d9d520c10 0x563d9d540eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:09:13.270 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:13.266+0000 7f6735456740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563d9d52cba0 0x563d9d5413f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:09:13.270 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:13.266+0000 7f6735456740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f672c1b7470 con 0x563d9d520c10 2024-09-11T00:09:13.270 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:13.266+0000 7f6735456740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f672c1b6ce0 con 0x563d9d52cba0 2024-09-11T00:09:13.270 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:13.266+0000 7f6735456740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f672c1b3f20 con 0x563d9d5202f0 2024-09-11T00:09:13.270 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:13.267+0000 7f6735456740 10 client.0 _ll_register_callbacks cb 0x7f6724009ee0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:09:13.270 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:13.267+0000 7f672be00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563d9d52cba0 0x563d9d5413f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:09:13.270 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:13.267+0000 7f672be00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563d9d52cba0 0x563d9d5413f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.12:40858/0 (socket says 172.21.15.12:40858) 2024-09-11T00:09:13.270 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:13.267+0000 7f672be00640 1 -- 172.21.15.12:0/612213552 learned_addr learned my addr 172.21.15.12:0/612213552 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:09:13.270 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:13.267+0000 7f6722a00640 1 --2- 172.21.15.12:0/612213552 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563d9d520c10 0x563d9d540eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:09:13.270 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:13.267+0000 7f672b400640 1 --2- 172.21.15.12:0/612213552 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563d9d5202f0 0x563d9d540970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:09:13.272 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:13.267+0000 7f672b400640 1 -- 172.21.15.12:0/612213552 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563d9d52cba0 msgr2=0x563d9d5413f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:09:13.272 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:13.267+0000 7f672b400640 1 --2- 172.21.15.12:0/612213552 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563d9d52cba0 0x563d9d5413f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:09:13.272 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:13.267+0000 7f672b400640 1 -- 172.21.15.12:0/612213552 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563d9d520c10 msgr2=0x563d9d540eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:09:13.272 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:13.267+0000 7f672b400640 1 --2- 172.21.15.12:0/612213552 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563d9d520c10 0x563d9d540eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:09:13.272 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:13.267+0000 7f672b400640 1 -- 172.21.15.12:0/612213552 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563d9d5a91c0 con 0x563d9d5202f0 2024-09-11T00:09:13.273 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:13.267+0000 7f6722a00640 1 --2- 172.21.15.12:0/612213552 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563d9d520c10 0x563d9d540eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:09:13.273 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:13.268+0000 7f672b400640 1 --2- 172.21.15.12:0/612213552 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563d9d5202f0 0x563d9d540970 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x563d9d51f000 tx=0x7f671c005f10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=961bfc0b36132358 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:09:13.273 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:13.268+0000 7f6719600640 10 client.0 ms_handle_connect on v2:172.21.15.178:3300/0 2024-09-11T00:09:13.273 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:13.268+0000 7f6719600640 1 -- 172.21.15.12:0/612213552 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f671c0abea0 con 0x563d9d5202f0 2024-09-11T00:09:13.273 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:13.268+0000 7f6735456740 1 -- 172.21.15.12:0/612213552 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f671404ce50 con 0x563d9d5202f0 2024-09-11T00:09:13.273 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:13.268+0000 7f6719600640 1 -- 172.21.15.12:0/612213552 <== mon.1 v2:172.21.15.178:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9208+0+0 (secure 0 0 0) 0x7f671c0a89a0 con 0x563d9d5202f0 2024-09-11T00:09:13.273 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:13.268+0000 7f6719600640 1 -- 172.21.15.12:0/612213552 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f671c0ac240 con 0x563d9d5202f0 2024-09-11T00:09:13.273 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:13.268+0000 7f6735456740 10 client.25783 fetch_fsmap learned FSMap version 33 2024-09-11T00:09:13.273 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:13.268+0000 7f6735456740 1 -- 172.21.15.12:0/612213552 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x563d9d5aa4b0 con 0x563d9d5202f0 2024-09-11T00:09:13.273 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:13.268+0000 7f6719600640 1 -- 172.21.15.12:0/612213552 <== mon.1 v2:172.21.15.178:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f671404ce50 con 0x563d9d5202f0 2024-09-11T00:09:13.273 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:13.268+0000 7f6719600640 1 -- 172.21.15.12:0/612213552 <== mon.1 v2:172.21.15.178:3300/0 5 ==== fsmap(e 33) ==== 8386+0+0 (secure 0 0 0) 0x7f671c018b20 con 0x563d9d5202f0 2024-09-11T00:09:13.273 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:13.269+0000 7f6735456740 10 client.25783 fetch_fsmap finished waiting for FSMap version 33 2024-09-11T00:09:13.274 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:13.269+0000 7f6735456740 10 client.25783 resolve_mds: validated gid 24674 aka daemon mds.b 2024-09-11T00:09:13.274 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:13.269+0000 7f6735456740 20 client.25783 populate_metadata read hostname 'smithi012' 2024-09-11T00:09:13.274 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:13.269+0000 7f6735456740 1 --2- 172.21.15.12:0/612213552 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x563d9d5aad30 0x563d9d5ad150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:09:13.274 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:13.269+0000 7f6735456740 4 client.25783 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 140.5, "await_for": 45.0}] 2024-09-11T00:09:13.274 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:13.269+0000 7f6735456740 1 -- 172.21.15.12:0/612213552 --> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 140.5, "await_for": 45.0}) -- 0x563d9d4936a0 con 0x563d9d5aad30 2024-09-11T00:09:13.274 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:13.269+0000 7f6722a00640 1 --2- 172.21.15.12:0/612213552 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x563d9d5aad30 0x563d9d5ad150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:09:13.274 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:13.269+0000 7f6722a00640 1 --2- 172.21.15.12:0/612213552 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x563d9d5aad30 0x563d9d5ad150 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=561c02f3b99d1575 server_cookie=14c9b161381281a9 in_seq=0 out_seq=0 2024-09-11T00:09:13.274 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:13.269+0000 7f6719600640 10 client.25783 ms_handle_connect on v2:172.21.15.165:6838/501312984 2024-09-11T00:09:13.325 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:13.322+0000 7f6719600640 1 -- 172.21.15.12:0/612213552 <== mds.0 v2:172.21.15.165:6838/501312984 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x563d9d4936a0 con 0x563d9d5aad30 2024-09-11T00:09:13.325 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:13.322+0000 7f6719600640 10 client.25783 handle_command_reply: tid=0 2024-09-11T00:09:13.325 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:13.322+0000 7f6735456740 1 client.25783 shutdown 2024-09-11T00:09:13.325 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:13.322+0000 7f6735456740 1 -- 172.21.15.12:0/612213552 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563d9d5202f0 msgr2=0x563d9d540970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:09:13.325 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:13.322+0000 7f6735456740 1 --2- 172.21.15.12:0/612213552 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563d9d5202f0 0x563d9d540970 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x563d9d51f000 tx=0x7f671c005f10 comp rx=0 tx=0).stop 2024-09-11T00:09:13.325 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:13.322+0000 7f6735456740 1 -- 172.21.15.12:0/612213552 shutdown_connections 2024-09-11T00:09:13.325 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:13.322+0000 7f6735456740 1 --2- 172.21.15.12:0/612213552 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x563d9d5aad30 0x563d9d5ad150 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:09:13.325 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:13.323+0000 7f6735456740 1 --2- 172.21.15.12:0/612213552 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563d9d52cba0 0x563d9d5413f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:09:13.325 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:13.323+0000 7f6735456740 1 --2- 172.21.15.12:0/612213552 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563d9d520c10 0x563d9d540eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:09:13.325 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:13.323+0000 7f6735456740 1 --2- 172.21.15.12:0/612213552 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563d9d5202f0 0x563d9d540970 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:09:13.325 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:13.323+0000 7f6719600640 0 client.25783 ms_handle_reset on v2:172.21.15.165:6838/501312984 2024-09-11T00:09:13.326 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:13.323+0000 7f6735456740 1 -- 172.21.15.12:0/612213552 >> 172.21.15.12:0/612213552 conn(0x563d9d51e120 msgr2=0x563d9d52af10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:09:13.326 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:13.323+0000 7f6735456740 1 -- 172.21.15.12:0/612213552 shutdown_connections 2024-09-11T00:09:13.326 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:13.323+0000 7f6735456740 1 -- 172.21.15.12:0/612213552 wait complete. 2024-09-11T00:09:13.326 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:13.323+0000 7f6735456740 20 client.25783 trim_cache size 0 max 16384 2024-09-11T00:09:13.326 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-09-11T00:09:13.326 INFO:teuthology.orchestra.run.smithi012.stdout: "epoch": 33, 2024-09-11T00:09:13.326 INFO:teuthology.orchestra.run.smithi012.stdout: "leader": 24674, 2024-09-11T00:09:13.326 INFO:teuthology.orchestra.run.smithi012.stdout: "set_version": 10, 2024-09-11T00:09:13.327 INFO:teuthology.orchestra.run.smithi012.stdout: "sets": { 2024-09-11T00:09:13.327 INFO:teuthology.orchestra.run.smithi012.stdout: "68ee5ab7": { 2024-09-11T00:09:13.327 INFO:teuthology.orchestra.run.smithi012.stdout: "version": 10, 2024-09-11T00:09:13.327 INFO:teuthology.orchestra.run.smithi012.stdout: "age_ref": 0.0, 2024-09-11T00:09:13.327 INFO:teuthology.orchestra.run.smithi012.stdout: "state": { 2024-09-11T00:09:13.327 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "QUIESCED", 2024-09-11T00:09:13.327 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 0.0 2024-09-11T00:09:13.327 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:09:13.327 INFO:teuthology.orchestra.run.smithi012.stdout: "timeout": 90.0, 2024-09-11T00:09:13.328 INFO:teuthology.orchestra.run.smithi012.stdout: "expiration": 140.5, 2024-09-11T00:09:13.328 INFO:teuthology.orchestra.run.smithi012.stdout: "members": { 2024-09-11T00:09:13.328 INFO:teuthology.orchestra.run.smithi012.stdout: "file:/": { 2024-09-11T00:09:13.328 INFO:teuthology.orchestra.run.smithi012.stdout: "excluded": false, 2024-09-11T00:09:13.328 INFO:teuthology.orchestra.run.smithi012.stdout: "state": { 2024-09-11T00:09:13.328 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "QUIESCED", 2024-09-11T00:09:13.328 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 0.0 2024-09-11T00:09:13.328 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:09:13.328 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:09:13.328 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:09:13.328 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:09:13.328 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:09:13.328 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-09-11T00:09:13.329 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:13.324+0000 7f6731200640 1 -- 172.21.15.12:0/494838590 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f67100b5410 msgr2=0x7f67100b78d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:09:13.329 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:13.324+0000 7f6731200640 1 --2- 172.21.15.12:0/494838590 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f67100b5410 0x7f67100b78d0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f6724009fd0 tx=0x7f6724009500 comp rx=0 tx=0).stop 2024-09-11T00:09:13.329 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:13.324+0000 7f6731200640 1 -- 172.21.15.12:0/494838590 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f672c0776d0 msgr2=0x7f672c19f600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:09:13.329 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:13.324+0000 7f6731200640 1 --2- 172.21.15.12:0/494838590 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f672c0776d0 0x7f672c19f600 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f671c00e9f0 tx=0x7f671c00eeb0 comp rx=0 tx=0).stop 2024-09-11T00:09:13.329 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:13.324+0000 7f6731200640 1 -- 172.21.15.12:0/494838590 shutdown_connections 2024-09-11T00:09:13.330 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:13.324+0000 7f6731200640 1 --2- 172.21.15.12:0/494838590 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f67100b5410 0x7f67100b78d0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:09:13.330 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:13.324+0000 7f6731200640 1 --2- 172.21.15.12:0/494838590 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f672c1137e0 0x7f672c1a4ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:09:13.330 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:13.324+0000 7f6731200640 1 --2- 172.21.15.12:0/494838590 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f672c077ff0 0x7f672c19fb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:09:13.330 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:13.324+0000 7f6731200640 1 --2- 172.21.15.12:0/494838590 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f672c0776d0 0x7f672c19f600 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:09:13.330 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:13.324+0000 7f6731200640 1 -- 172.21.15.12:0/494838590 >> 172.21.15.12:0/494838590 conn(0x7f672c100840 msgr2=0x7f672c102c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:09:13.330 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:13.324+0000 7f6731200640 1 -- 172.21.15.12:0/494838590 shutdown_connections 2024-09-11T00:09:13.330 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:13.324+0000 7f6731200640 1 -- 172.21.15.12:0/494838590 wait complete. 2024-09-11T00:09:13.532 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:09:13 smithi165 ceph-mon[35356]: pgmap v463: 129 pgs: 129 active+clean; 26 MiB data, 506 MiB used, 1.0 TiB / 1.0 TiB avail; 500 KiB/s rd, 1.6 MiB/s wr, 137 op/s 2024-09-11T00:09:13.532 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:09:13 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:09:13.532 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:09:13 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:09:13.533 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:09:13 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:09:13.533 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:09:13 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:09:13.533 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:09:13 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T00:09:13.533 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:09:13 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:09:13.533 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:09:13 smithi165 ceph-mon[35356]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-11T00:09:13.533 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:09:13 smithi165 ceph-mon[35356]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-11T00:09:13.533 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:09:13 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-11T00:09:13.533 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:09:13 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-11T00:09:13.533 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:09:13 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-11T00:09:13.533 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:09:13 smithi165 ceph-mon[35356]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-11T00:09:13.533 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:09:13 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-11T00:09:13.533 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:09:13 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-11T00:09:13.534 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:09:13 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-11T00:09:13.534 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:09:13 smithi165 ceph-mon[35356]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-11T00:09:13.534 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:09:13 smithi165 ceph-mon[35356]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-11T00:09:13.534 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:09:13 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-11T00:09:13.534 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:09:13 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-11T00:09:13.534 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:09:13 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-11T00:09:13.534 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:09:13 smithi165 ceph-mon[35356]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-11T00:09:13.563 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:09:13 smithi012 ceph-mon[30333]: pgmap v463: 129 pgs: 129 active+clean; 26 MiB data, 506 MiB used, 1.0 TiB / 1.0 TiB avail; 500 KiB/s rd, 1.6 MiB/s wr, 137 op/s 2024-09-11T00:09:13.563 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:09:13 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:09:13.563 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:09:13 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:09:13.563 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:09:13 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:09:13.563 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:09:13 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:09:13.563 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:09:13 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T00:09:13.563 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:09:13 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:09:13.563 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:09:13 smithi012 ceph-mon[30333]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-11T00:09:13.564 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:09:13 smithi012 ceph-mon[30333]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-11T00:09:13.564 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:09:13 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-11T00:09:13.564 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:09:13 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-11T00:09:13.564 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:09:13 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-11T00:09:13.564 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:09:13 smithi012 ceph-mon[30333]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-11T00:09:13.564 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:09:13 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-11T00:09:13.564 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:09:13 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-11T00:09:13.564 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:09:13 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-11T00:09:13.564 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:09:13 smithi012 ceph-mon[30333]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-11T00:09:13.564 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:09:13 smithi012 ceph-mon[30333]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-11T00:09:13.564 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:09:13 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-11T00:09:13.564 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:09:13 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-11T00:09:13.564 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:09:13 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-11T00:09:13.564 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:09:13 smithi012 ceph-mon[30333]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-11T00:09:13.581 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:09:13 smithi178 ceph-mon[35919]: pgmap v463: 129 pgs: 129 active+clean; 26 MiB data, 506 MiB used, 1.0 TiB / 1.0 TiB avail; 500 KiB/s rd, 1.6 MiB/s wr, 137 op/s 2024-09-11T00:09:13.581 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:09:13 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:09:13.581 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:09:13 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:09:13.581 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:09:13 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:09:13.581 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:09:13 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:09:13.581 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:09:13 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T00:09:13.581 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:09:13 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:09:13.581 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:09:13 smithi178 ceph-mon[35919]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-11T00:09:13.581 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:09:13 smithi178 ceph-mon[35919]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-11T00:09:13.581 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:09:13 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-11T00:09:13.581 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:09:13 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-11T00:09:13.581 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:09:13 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-11T00:09:13.582 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:09:13 smithi178 ceph-mon[35919]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-11T00:09:13.582 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:09:13 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-11T00:09:13.582 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:09:13 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-11T00:09:13.582 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:09:13 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-11T00:09:13.582 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:09:13 smithi178 ceph-mon[35919]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-11T00:09:13.582 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:09:13 smithi178 ceph-mon[35919]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-11T00:09:13.582 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:09:13 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-11T00:09:13.582 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:09:13 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-11T00:09:13.582 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:09:13 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-11T00:09:13.582 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:09:13 smithi178 ceph-mon[35919]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-11T00:09:13.604 INFO:tasks.workunit.client.0.smithi012.stdout: 1 89449 7.13 MB/sec warmup 70 sec latency 1854.063 ms 2024-09-11T00:09:14.395 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '68ee5ab7', quiesce took 4.3 seconds. Will release after: 16.2 2024-09-11T00:09:14.604 INFO:tasks.workunit.client.0.smithi012.stdout: 1 89449 7.03 MB/sec warmup 71 sec latency 1315.186 ms 2024-09-11T00:09:15.532 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:09:15 smithi165 ceph-mon[35356]: pgmap v464: 129 pgs: 129 active+clean; 25 MiB data, 506 MiB used, 1.0 TiB / 1.0 TiB avail; 527 KiB/s rd, 1.5 MiB/s wr, 117 op/s 2024-09-11T00:09:15.563 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:09:15 smithi012 ceph-mon[30333]: pgmap v464: 129 pgs: 129 active+clean; 25 MiB data, 506 MiB used, 1.0 TiB / 1.0 TiB avail; 527 KiB/s rd, 1.5 MiB/s wr, 117 op/s 2024-09-11T00:09:15.580 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:09:15 smithi178 ceph-mon[35919]: pgmap v464: 129 pgs: 129 active+clean; 25 MiB data, 506 MiB used, 1.0 TiB / 1.0 TiB avail; 527 KiB/s rd, 1.5 MiB/s wr, 117 op/s 2024-09-11T00:09:15.604 INFO:tasks.workunit.client.0.smithi012.stdout: 1 89449 6.93 MB/sec warmup 72 sec latency 2315.309 ms 2024-09-11T00:09:16.605 INFO:tasks.workunit.client.0.smithi012.stdout: 1 89449 6.84 MB/sec warmup 73 sec latency 3315.467 ms 2024-09-11T00:09:17.532 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:09:17 smithi165 ceph-mon[35356]: pgmap v465: 129 pgs: 129 active+clean; 26 MiB data, 516 MiB used, 1.0 TiB / 1.0 TiB avail; 882 KiB/s rd, 1.6 MiB/s wr, 157 op/s 2024-09-11T00:09:17.563 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:09:17 smithi012 ceph-mon[30333]: pgmap v465: 129 pgs: 129 active+clean; 26 MiB data, 516 MiB used, 1.0 TiB / 1.0 TiB avail; 882 KiB/s rd, 1.6 MiB/s wr, 157 op/s 2024-09-11T00:09:17.580 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:09:17 smithi178 ceph-mon[35919]: pgmap v465: 129 pgs: 129 active+clean; 26 MiB data, 516 MiB used, 1.0 TiB / 1.0 TiB avail; 882 KiB/s rd, 1.6 MiB/s wr, 157 op/s 2024-09-11T00:09:17.604 INFO:tasks.workunit.client.0.smithi012.stdout: 1 89449 6.75 MB/sec warmup 74 sec latency 4315.540 ms 2024-09-11T00:09:18.605 INFO:tasks.workunit.client.0.smithi012.stdout: 1 89449 6.66 MB/sec warmup 75 sec latency 5315.668 ms 2024-09-11T00:09:19.532 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:09:19 smithi165 ceph-mon[35356]: pgmap v466: 129 pgs: 129 active+clean; 26 MiB data, 516 MiB used, 1.0 TiB / 1.0 TiB avail; 892 KiB/s rd, 1009 KiB/s wr, 120 op/s 2024-09-11T00:09:19.563 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:09:19 smithi012 ceph-mon[30333]: pgmap v466: 129 pgs: 129 active+clean; 26 MiB data, 516 MiB used, 1.0 TiB / 1.0 TiB avail; 892 KiB/s rd, 1009 KiB/s wr, 120 op/s 2024-09-11T00:09:19.581 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:09:19 smithi178 ceph-mon[35919]: pgmap v466: 129 pgs: 129 active+clean; 26 MiB data, 516 MiB used, 1.0 TiB / 1.0 TiB avail; 892 KiB/s rd, 1009 KiB/s wr, 120 op/s 2024-09-11T00:09:19.605 INFO:tasks.workunit.client.0.smithi012.stdout: 1 89449 6.57 MB/sec warmup 76 sec latency 6315.813 ms 2024-09-11T00:09:20.605 INFO:tasks.workunit.client.0.smithi012.stdout: 1 89449 6.48 MB/sec warmup 77 sec latency 7315.950 ms 2024-09-11T00:09:21.533 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:09:21 smithi165 ceph-mon[35356]: pgmap v467: 129 pgs: 129 active+clean; 26 MiB data, 516 MiB used, 1.0 TiB / 1.0 TiB avail; 843 KiB/s rd, 887 KiB/s wr, 115 op/s 2024-09-11T00:09:21.563 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:09:21 smithi012 ceph-mon[30333]: pgmap v467: 129 pgs: 129 active+clean; 26 MiB data, 516 MiB used, 1.0 TiB / 1.0 TiB avail; 843 KiB/s rd, 887 KiB/s wr, 115 op/s 2024-09-11T00:09:21.581 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:09:21 smithi178 ceph-mon[35919]: pgmap v467: 129 pgs: 129 active+clean; 26 MiB data, 516 MiB used, 1.0 TiB / 1.0 TiB avail; 843 KiB/s rd, 887 KiB/s wr, 115 op/s 2024-09-11T00:09:21.605 INFO:tasks.workunit.client.0.smithi012.stdout: 1 89449 6.40 MB/sec warmup 78 sec latency 8316.071 ms 2024-09-11T00:09:22.605 INFO:tasks.workunit.client.0.smithi012.stdout: 1 89449 6.32 MB/sec warmup 79 sec latency 9316.190 ms 2024-09-11T00:09:23.532 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:09:23 smithi165 ceph-mon[35356]: pgmap v468: 129 pgs: 129 active+clean; 26 MiB data, 516 MiB used, 1.0 TiB / 1.0 TiB avail; 668 KiB/s rd, 377 KiB/s wr, 86 op/s 2024-09-11T00:09:23.562 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:09:23 smithi012 ceph-mon[30333]: pgmap v468: 129 pgs: 129 active+clean; 26 MiB data, 516 MiB used, 1.0 TiB / 1.0 TiB avail; 668 KiB/s rd, 377 KiB/s wr, 86 op/s 2024-09-11T00:09:23.581 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:09:23 smithi178 ceph-mon[35919]: pgmap v468: 129 pgs: 129 active+clean; 26 MiB data, 516 MiB used, 1.0 TiB / 1.0 TiB avail; 668 KiB/s rd, 377 KiB/s wr, 86 op/s 2024-09-11T00:09:23.605 INFO:tasks.workunit.client.0.smithi012.stdout: 1 89449 6.24 MB/sec warmup 80 sec latency 10316.307 ms 2024-09-11T00:09:24.486 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:24.485+0000 7f4536200640 1 -- 172.21.15.12:0/2971169581 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4530105ce0 msgr2=0x7f4530106160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:09:24.486 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:24.485+0000 7f4536200640 1 --2- 172.21.15.12:0/2971169581 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4530105ce0 0x7f4530106160 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f4518009930 tx=0x7f451802a420 comp rx=0 tx=0).stop 2024-09-11T00:09:24.486 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:24.485+0000 7f4536200640 1 -- 172.21.15.12:0/2971169581 shutdown_connections 2024-09-11T00:09:24.487 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:24.485+0000 7f4536200640 1 --2- 172.21.15.12:0/2971169581 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f45301066a0 0x7f453010d020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:09:24.487 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:24.485+0000 7f4536200640 1 --2- 172.21.15.12:0/2971169581 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4530105ce0 0x7f4530106160 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:09:24.487 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:24.485+0000 7f4536200640 1 --2- 172.21.15.12:0/2971169581 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4530104b90 0x7f4530104f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:09:24.487 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:24.485+0000 7f4536200640 1 -- 172.21.15.12:0/2971169581 >> 172.21.15.12:0/2971169581 conn(0x7f4530100470 msgr2=0x7f4530102890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:09:24.487 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:24.485+0000 7f4536200640 1 -- 172.21.15.12:0/2971169581 shutdown_connections 2024-09-11T00:09:24.487 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:24.485+0000 7f4536200640 1 -- 172.21.15.12:0/2971169581 wait complete. 2024-09-11T00:09:24.489 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:24.486+0000 7f4536200640 1 Processor -- start 2024-09-11T00:09:24.489 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:24.486+0000 7f4536200640 1 -- start start 2024-09-11T00:09:24.489 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:24.486+0000 7f4536200640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4530104b90 0x7f453019af70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:09:24.489 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:24.486+0000 7f4536200640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4530105ce0 0x7f453019b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:09:24.489 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:24.486+0000 7f4536200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f45301066a0 0x7f453019c1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:09:24.489 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:24.486+0000 7f4536200640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f4530110850 con 0x7f4530104b90 2024-09-11T00:09:24.489 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:24.486+0000 7f4536200640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f45301106d0 con 0x7f45301066a0 2024-09-11T00:09:24.489 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:24.486+0000 7f4536200640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f45301109d0 con 0x7f4530105ce0 2024-09-11T00:09:24.490 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:24.487+0000 7f452f400640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4530104b90 0x7f453019af70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:09:24.490 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:24.487+0000 7f4526a00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4530105ce0 0x7f453019b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:09:24.490 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:24.487+0000 7f452f400640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4530104b90 0x7f453019af70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:56478/0 (socket says 172.21.15.12:56478) 2024-09-11T00:09:24.490 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:24.487+0000 7f452f400640 1 -- 172.21.15.12:0/1079635166 learned_addr learned my addr 172.21.15.12:0/1079635166 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:09:24.490 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:24.487+0000 7f452fe00640 1 --2- 172.21.15.12:0/1079635166 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f45301066a0 0x7f453019c1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:09:24.490 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:24.487+0000 7f4526a00640 1 -- 172.21.15.12:0/1079635166 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f45301066a0 msgr2=0x7f453019c1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:09:24.490 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:24.487+0000 7f4526a00640 1 --2- 172.21.15.12:0/1079635166 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f45301066a0 0x7f453019c1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:09:24.490 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:24.487+0000 7f4526a00640 1 -- 172.21.15.12:0/1079635166 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4530104b90 msgr2=0x7f453019af70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:09:24.490 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:24.487+0000 7f4526a00640 1 --2- 172.21.15.12:0/1079635166 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4530104b90 0x7f453019af70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:09:24.490 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:24.487+0000 7f4526a00640 1 -- 172.21.15.12:0/1079635166 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f453019cab0 con 0x7f4530105ce0 2024-09-11T00:09:24.490 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:24.487+0000 7f4526a00640 1 --2- 172.21.15.12:0/1079635166 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4530105ce0 0x7f453019b5c0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f451802a940 tx=0x7f4518003c10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:09:24.490 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:24.488+0000 7f452cc00640 1 -- 172.21.15.12:0/1079635166 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f451803f070 con 0x7f4530105ce0 2024-09-11T00:09:24.490 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:24.488+0000 7f452cc00640 1 -- 172.21.15.12:0/1079635166 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4518031c50 con 0x7f4530105ce0 2024-09-11T00:09:24.491 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:24.488+0000 7f4536200640 1 -- 172.21.15.12:0/1079635166 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f453019cd40 con 0x7f4530105ce0 2024-09-11T00:09:24.491 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:24.488+0000 7f452cc00640 1 -- 172.21.15.12:0/1079635166 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4518035030 con 0x7f4530105ce0 2024-09-11T00:09:24.491 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:24.488+0000 7f4536200640 1 -- 172.21.15.12:0/1079635166 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45301a3290 con 0x7f4530105ce0 2024-09-11T00:09:24.492 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:24.491+0000 7f45391e9740 1 Processor -- start 2024-09-11T00:09:24.492 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:24.491+0000 7f45391e9740 1 -- start start 2024-09-11T00:09:24.493 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:24.491+0000 7f452cc00640 1 -- 172.21.15.12:0/1079635166 <== mon.1 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7f451803b4f0 con 0x7f4530105ce0 2024-09-11T00:09:24.493 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:24.491+0000 7f45391e9740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55db677ff790 0x55db67801b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:09:24.493 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:24.491+0000 7f45391e9740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55db678020c0 0x55db678044b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:09:24.493 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:24.491+0000 7f45391e9740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55db67804b00 0x55db67806fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:09:24.493 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:24.491+0000 7f45391e9740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55db677726b0 con 0x55db677ff790 2024-09-11T00:09:24.493 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:24.491+0000 7f45391e9740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55db67761ae0 con 0x55db678020c0 2024-09-11T00:09:24.493 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:24.491+0000 7f45391e9740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55db677658f0 con 0x55db67804b00 2024-09-11T00:09:24.493 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:24.492+0000 7f452fe00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55db677ff790 0x55db67801b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:09:24.494 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:24.492+0000 7f452f400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55db678020c0 0x55db678044b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:09:24.494 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:24.492+0000 7f452f400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55db678020c0 0x55db678044b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.12:48438/0 (socket says 172.21.15.12:48438) 2024-09-11T00:09:24.494 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:24.492+0000 7f452f400640 1 -- 172.21.15.12:0/2491395050 learned_addr learned my addr 172.21.15.12:0/2491395050 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:09:24.494 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:24.492+0000 7f4526a00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55db67804b00 0x55db67806fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:09:24.494 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:24.492+0000 7f452f400640 1 -- 172.21.15.12:0/2491395050 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55db67804b00 msgr2=0x55db67806fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:09:24.494 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:24.492+0000 7f452f400640 1 --2- 172.21.15.12:0/2491395050 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55db67804b00 0x55db67806fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:09:24.494 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:24.492+0000 7f452f400640 1 -- 172.21.15.12:0/2491395050 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55db677ff790 msgr2=0x55db67801b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:09:24.494 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:24.492+0000 7f452f400640 1 --2- 172.21.15.12:0/2491395050 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55db677ff790 0x55db67801b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:09:24.494 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:24.492+0000 7f452f400640 1 -- 172.21.15.12:0/2491395050 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55db6772e780 con 0x55db678020c0 2024-09-11T00:09:24.497 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:24.492+0000 7f452f400640 1 --2- 172.21.15.12:0/2491395050 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55db678020c0 0x55db678044b0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f452000b890 tx=0x7f452000bd50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=96a7076705143ed9 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:09:24.497 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:24.493+0000 7f451fe00640 1 -- 172.21.15.12:0/2491395050 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4520003dd0 con 0x55db678020c0 2024-09-11T00:09:24.497 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:24.493+0000 7f451fe00640 1 -- 172.21.15.12:0/2491395050 <== mon.2 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4520015070 con 0x55db678020c0 2024-09-11T00:09:24.497 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:24.493+0000 7f451fe00640 1 -- 172.21.15.12:0/2491395050 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45200117c0 con 0x55db678020c0 2024-09-11T00:09:24.498 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:24.493+0000 7f452cc00640 1 --2- 172.21.15.12:0/1079635166 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f45140b52e0 0x7f45140b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:09:24.498 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:24.493+0000 7f45391e9740 1 -- 172.21.15.12:0/2491395050 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55db678020c0 msgr2=0x55db678044b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:09:24.498 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:24.493+0000 7f45391e9740 1 --2- 172.21.15.12:0/2491395050 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55db678020c0 0x55db678044b0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f452000b890 tx=0x7f452000bd50 comp rx=0 tx=0).stop 2024-09-11T00:09:24.498 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:24.493+0000 7f452cc00640 1 -- 172.21.15.12:0/1079635166 <== mon.1 v2:172.21.15.178:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9208+0+0 (secure 0 0 0) 0x7f45181069b0 con 0x7f4530105ce0 2024-09-11T00:09:24.498 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:24.493+0000 7f45391e9740 1 -- 172.21.15.12:0/2491395050 shutdown_connections 2024-09-11T00:09:24.498 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:24.493+0000 7f45391e9740 1 --2- 172.21.15.12:0/2491395050 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55db67804b00 0x55db67806fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:09:24.498 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:24.493+0000 7f45391e9740 1 --2- 172.21.15.12:0/2491395050 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55db678020c0 0x55db678044b0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:09:24.498 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:24.493+0000 7f45391e9740 1 --2- 172.21.15.12:0/2491395050 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55db677ff790 0x55db67801b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:09:24.498 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:24.493+0000 7f45391e9740 1 -- 172.21.15.12:0/2491395050 >> 172.21.15.12:0/2491395050 conn(0x55db677fa8e0 msgr2=0x55db677fb670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:09:24.498 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:24.494+0000 7f452fe00640 1 --2- 172.21.15.12:0/1079635166 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f45140b52e0 0x7f45140b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:09:24.498 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:24.494+0000 7f45391e9740 1 -- 172.21.15.12:0/2491395050 shutdown_connections 2024-09-11T00:09:24.499 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:24.494+0000 7f45391e9740 1 -- 172.21.15.12:0/2491395050 wait complete. 2024-09-11T00:09:24.499 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:24.494+0000 7f45391e9740 1 -- start start 2024-09-11T00:09:24.499 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:24.494+0000 7f452fe00640 1 --2- 172.21.15.12:0/1079635166 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f45140b52e0 0x7f45140b77a0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f453019bcf0 tx=0x7f4528009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:09:24.499 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:24.495+0000 7f45391e9740 1 Processor -- start 2024-09-11T00:09:24.499 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:24.495+0000 7f45391e9740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55db677ff790 0x55db67808e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:09:24.499 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:24.495+0000 7f45391e9740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55db678020c0 0x55db67809530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:09:24.499 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:24.495+0000 7f45391e9740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55db67804b00 0x55db6780a100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:09:24.499 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:24.495+0000 7f45391e9740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55db67699ab0 con 0x55db677ff790 2024-09-11T00:09:24.499 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:24.495+0000 7f45391e9740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55db67708570 con 0x55db67804b00 2024-09-11T00:09:24.499 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:24.495+0000 7f45391e9740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55db67807ae0 con 0x55db678020c0 2024-09-11T00:09:24.499 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:24.495+0000 7f45391e9740 10 client.0 _ll_register_callbacks cb 0x55db67717100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:09:24.499 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:24.495+0000 7f452f400640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55db677ff790 0x55db67808e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:09:24.500 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:24.495+0000 7f4526a00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55db678020c0 0x55db67809530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:09:24.500 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:24.495+0000 7f452fe00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55db67804b00 0x55db6780a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:09:24.500 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:24.495+0000 7f4526a00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55db678020c0 0x55db67809530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.12:60666/0 (socket says 172.21.15.12:60666) 2024-09-11T00:09:24.500 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:24.495+0000 7f4526a00640 1 -- 172.21.15.12:0/292661413 learned_addr learned my addr 172.21.15.12:0/292661413 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:09:24.500 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:24.495+0000 7f452f400640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55db677ff790 0x55db67808e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:56498/0 (socket says 172.21.15.12:56498) 2024-09-11T00:09:24.500 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:24.496+0000 7f4526a00640 1 -- 172.21.15.12:0/292661413 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55db67804b00 msgr2=0x55db6780a100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:09:24.500 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:24.496+0000 7f4526a00640 1 --2- 172.21.15.12:0/292661413 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55db67804b00 0x55db6780a100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:09:24.500 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:24.496+0000 7f4526a00640 1 -- 172.21.15.12:0/292661413 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55db677ff790 msgr2=0x55db67808e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:09:24.500 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:24.496+0000 7f4526a00640 1 --2- 172.21.15.12:0/292661413 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55db677ff790 0x55db67808e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:09:24.500 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:24.496+0000 7f4526a00640 1 -- 172.21.15.12:0/292661413 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55db674c3790 con 0x55db678020c0 2024-09-11T00:09:24.500 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:24.496+0000 7f4526a00640 1 --2- 172.21.15.12:0/292661413 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55db678020c0 0x55db67809530 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f45180ca060 tx=0x7f451803b460 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4d3ed10a7922225e server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:09:24.500 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:24.496+0000 7f451cc00640 10 client.0 ms_handle_connect on v2:172.21.15.178:3300/0 2024-09-11T00:09:24.500 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:24.496+0000 7f451cc00640 1 -- 172.21.15.12:0/292661413 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4518049070 con 0x55db678020c0 2024-09-11T00:09:24.501 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:24.496+0000 7f451cc00640 1 -- 172.21.15.12:0/292661413 <== mon.1 v2:172.21.15.178:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9208+0+0 (secure 0 0 0) 0x7f45180d5e40 con 0x55db678020c0 2024-09-11T00:09:24.501 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:24.496+0000 7f45391e9740 1 -- 172.21.15.12:0/292661413 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4520015070 con 0x55db678020c0 2024-09-11T00:09:24.501 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:24.496+0000 7f451cc00640 1 -- 172.21.15.12:0/292661413 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f45180ccbc0 con 0x55db678020c0 2024-09-11T00:09:24.501 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:24.496+0000 7f45391e9740 10 client.25795 fetch_fsmap learned FSMap version 33 2024-09-11T00:09:24.501 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:24.496+0000 7f45391e9740 1 -- 172.21.15.12:0/292661413 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55db6753c620 con 0x55db678020c0 2024-09-11T00:09:24.501 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:24.497+0000 7f451cc00640 1 -- 172.21.15.12:0/292661413 <== mon.1 v2:172.21.15.178:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4520015070 con 0x55db678020c0 2024-09-11T00:09:24.502 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:24.497+0000 7f451cc00640 1 -- 172.21.15.12:0/292661413 <== mon.1 v2:172.21.15.178:3300/0 5 ==== fsmap(e 33) ==== 8386+0+0 (secure 0 0 0) 0x7f45180cc050 con 0x55db678020c0 2024-09-11T00:09:24.502 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:24.497+0000 7f45391e9740 10 client.25795 fetch_fsmap finished waiting for FSMap version 33 2024-09-11T00:09:24.503 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:24.497+0000 7f45391e9740 10 client.25795 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T00:09:24.503 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:24.497+0000 7f45391e9740 20 client.25795 populate_metadata read hostname 'smithi012' 2024-09-11T00:09:24.503 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:24.497+0000 7f45391e9740 1 --2- 172.21.15.12:0/292661413 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55db67814840 0x55db67816cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:09:24.503 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:24.497+0000 7f45391e9740 4 client.25795 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T00:09:24.503 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:24.497+0000 7f45391e9740 1 -- 172.21.15.12:0/292661413 --> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55db67774600 con 0x55db67814840 2024-09-11T00:09:24.503 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:24.497+0000 7f452f400640 1 --2- 172.21.15.12:0/292661413 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55db67814840 0x55db67816cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:09:24.503 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:24.498+0000 7f452f400640 1 --2- 172.21.15.12:0/292661413 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55db67814840 0x55db67816cc0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e24a1720d3440b25 server_cookie=fee84d818ed3c530 in_seq=0 out_seq=0 2024-09-11T00:09:24.503 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:24.498+0000 7f451cc00640 10 client.25795 ms_handle_connect on v2:172.21.15.165:6838/501312984 2024-09-11T00:09:24.503 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:24.499+0000 7f451cc00640 1 -- 172.21.15.12:0/292661413 <== mds.0 v2:172.21.15.165:6838/501312984 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55db67774600 con 0x55db67814840 2024-09-11T00:09:24.503 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:24.499+0000 7f451cc00640 10 client.25795 handle_command_reply: tid=0 2024-09-11T00:09:24.503 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:24.499+0000 7f45391e9740 1 client.25795 shutdown 2024-09-11T00:09:24.503 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:24.499+0000 7f45391e9740 1 -- 172.21.15.12:0/292661413 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55db678020c0 msgr2=0x55db67809530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:09:24.504 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:24.499+0000 7f45391e9740 1 --2- 172.21.15.12:0/292661413 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55db678020c0 0x55db67809530 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f45180ca060 tx=0x7f451803b460 comp rx=0 tx=0).stop 2024-09-11T00:09:24.504 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:24.499+0000 7f45391e9740 1 -- 172.21.15.12:0/292661413 shutdown_connections 2024-09-11T00:09:24.504 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:24.499+0000 7f45391e9740 1 --2- 172.21.15.12:0/292661413 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55db67814840 0x55db67816cc0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:09:24.504 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:24.499+0000 7f45391e9740 1 --2- 172.21.15.12:0/292661413 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55db67804b00 0x55db6780a100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:09:24.504 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:24.499+0000 7f45391e9740 1 --2- 172.21.15.12:0/292661413 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55db678020c0 0x55db67809530 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:09:24.504 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:24.499+0000 7f45391e9740 1 --2- 172.21.15.12:0/292661413 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55db677ff790 0x55db67808e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:09:24.504 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:24.499+0000 7f451cc00640 0 client.25795 ms_handle_reset on v2:172.21.15.165:6838/501312984 2024-09-11T00:09:24.504 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:24.499+0000 7f45391e9740 1 -- 172.21.15.12:0/292661413 >> 172.21.15.12:0/292661413 conn(0x55db677fa8e0 msgr2=0x55db677f9a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:09:24.504 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:24.499+0000 7f45391e9740 1 -- 172.21.15.12:0/292661413 shutdown_connections 2024-09-11T00:09:24.504 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:24.499+0000 7f45391e9740 1 -- 172.21.15.12:0/292661413 wait complete. 2024-09-11T00:09:24.504 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:24.499+0000 7f45391e9740 20 client.25795 trim_cache size 0 max 16384 2024-09-11T00:09:24.517 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:24.515+0000 7f45391e9740 1 Processor -- start 2024-09-11T00:09:24.517 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:24.515+0000 7f45391e9740 1 -- start start 2024-09-11T00:09:24.517 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:24.515+0000 7f45391e9740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55db677ff790 0x55db67804ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:09:24.517 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:24.515+0000 7f45391e9740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55db678020c0 0x55db67805420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:09:24.518 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:24.515+0000 7f45391e9740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55db67804b00 0x55db677febf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:09:24.518 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:24.515+0000 7f45391e9740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f45301ad930 con 0x55db678020c0 2024-09-11T00:09:24.518 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:24.515+0000 7f45391e9740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f45301af530 con 0x55db67804b00 2024-09-11T00:09:24.518 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:24.515+0000 7f45391e9740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f45301ae000 con 0x55db677ff790 2024-09-11T00:09:24.518 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:24.515+0000 7f4526a00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55db678020c0 0x55db67805420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:09:24.518 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:24.515+0000 7f4526a00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55db678020c0 0x55db67805420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:56510/0 (socket says 172.21.15.12:56510) 2024-09-11T00:09:24.518 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:24.516+0000 7f4526a00640 1 -- 172.21.15.12:0/1471980166 learned_addr learned my addr 172.21.15.12:0/1471980166 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:09:24.518 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:24.516+0000 7f452fe00640 1 --2- 172.21.15.12:0/1471980166 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55db67804b00 0x55db677febf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:09:24.518 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:24.516+0000 7f452f400640 1 --2- 172.21.15.12:0/1471980166 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55db677ff790 0x55db67804ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:09:24.518 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:24.516+0000 7f4526a00640 1 -- 172.21.15.12:0/1471980166 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55db677ff790 msgr2=0x55db67804ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:09:24.518 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:24.516+0000 7f4526a00640 1 --2- 172.21.15.12:0/1471980166 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55db677ff790 0x55db67804ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:09:24.518 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:24.516+0000 7f4526a00640 1 -- 172.21.15.12:0/1471980166 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55db67804b00 msgr2=0x55db677febf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:09:24.518 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:24.516+0000 7f4526a00640 1 --2- 172.21.15.12:0/1471980166 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55db67804b00 0x55db677febf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:09:24.519 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:24.516+0000 7f4526a00640 1 -- 172.21.15.12:0/1471980166 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55db67804820 con 0x55db678020c0 2024-09-11T00:09:24.519 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:24.516+0000 7f452f400640 1 --2- 172.21.15.12:0/1471980166 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55db677ff790 0x55db67804ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T00:09:24.519 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:24.516+0000 7f4526a00640 1 --2- 172.21.15.12:0/1471980166 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55db678020c0 0x55db67805420 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f45180ccd80 tx=0x7f4518009560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac4cd7732078de2b server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:09:24.519 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:24.516+0000 7f451f400640 1 -- 172.21.15.12:0/1471980166 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45180d1030 con 0x55db678020c0 2024-09-11T00:09:24.519 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:24.516+0000 7f451f400640 1 -- 172.21.15.12:0/1471980166 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4518031e00 con 0x55db678020c0 2024-09-11T00:09:24.519 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:24.516+0000 7f451f400640 1 -- 172.21.15.12:0/1471980166 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4518049040 con 0x55db678020c0 2024-09-11T00:09:24.522 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:24.517+0000 7f45391e9740 1 -- 172.21.15.12:0/1471980166 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55db678020c0 msgr2=0x55db67805420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:09:24.522 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:24.517+0000 7f45391e9740 1 --2- 172.21.15.12:0/1471980166 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55db678020c0 0x55db67805420 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f45180ccd80 tx=0x7f4518009560 comp rx=0 tx=0).stop 2024-09-11T00:09:24.522 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:24.517+0000 7f45391e9740 1 -- 172.21.15.12:0/1471980166 shutdown_connections 2024-09-11T00:09:24.522 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:24.517+0000 7f45391e9740 1 --2- 172.21.15.12:0/1471980166 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55db67804b00 0x55db677febf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:09:24.522 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:24.517+0000 7f45391e9740 1 --2- 172.21.15.12:0/1471980166 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55db678020c0 0x55db67805420 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:09:24.522 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:24.517+0000 7f45391e9740 1 --2- 172.21.15.12:0/1471980166 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55db677ff790 0x55db67804ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:09:24.522 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:24.517+0000 7f45391e9740 1 -- 172.21.15.12:0/1471980166 >> 172.21.15.12:0/1471980166 conn(0x55db677fa8e0 msgr2=0x55db677f5a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:09:24.522 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:24.517+0000 7f45391e9740 1 -- 172.21.15.12:0/1471980166 shutdown_connections 2024-09-11T00:09:24.523 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:24.517+0000 7f45391e9740 1 -- 172.21.15.12:0/1471980166 wait complete. 2024-09-11T00:09:24.523 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:24.518+0000 7f45391e9740 1 -- start start 2024-09-11T00:09:24.523 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:24.518+0000 7f45391e9740 1 Processor -- start 2024-09-11T00:09:24.523 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:24.518+0000 7f45391e9740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55db677ff790 0x55db6780fc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:09:24.523 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:24.518+0000 7f45391e9740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55db678020c0 0x55db67810160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:09:24.523 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:24.518+0000 7f45391e9740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55db67804b00 0x55db6780e740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:09:24.523 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:24.518+0000 7f45391e9740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f45301aee60 con 0x55db678020c0 2024-09-11T00:09:24.523 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:24.518+0000 7f45391e9740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f45301ae6d0 con 0x55db67804b00 2024-09-11T00:09:24.523 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:24.518+0000 7f45391e9740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f45301ab910 con 0x55db677ff790 2024-09-11T00:09:24.523 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:24.518+0000 7f45391e9740 10 client.0 _ll_register_callbacks cb 0x7f451810c220 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:09:24.523 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:24.518+0000 7f452f400640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55db677ff790 0x55db6780fc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:09:24.523 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:24.518+0000 7f452f400640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55db677ff790 0x55db6780fc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.12:60690/0 (socket says 172.21.15.12:60690) 2024-09-11T00:09:24.523 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:24.518+0000 7f452f400640 1 -- 172.21.15.12:0/3854047244 learned_addr learned my addr 172.21.15.12:0/3854047244 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:09:24.524 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:24.518+0000 7f4526a00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55db678020c0 0x55db67810160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:09:24.524 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:24.519+0000 7f452fe00640 1 --2- 172.21.15.12:0/3854047244 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55db67804b00 0x55db6780e740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:09:24.524 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:24.519+0000 7f452f400640 1 -- 172.21.15.12:0/3854047244 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55db67804b00 msgr2=0x55db6780e740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:09:24.524 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:24.519+0000 7f452f400640 1 --2- 172.21.15.12:0/3854047244 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55db67804b00 0x55db6780e740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:09:24.524 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:24.519+0000 7f452f400640 1 -- 172.21.15.12:0/3854047244 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55db678020c0 msgr2=0x55db67810160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:09:24.524 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:24.519+0000 7f452f400640 1 --2- 172.21.15.12:0/3854047244 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55db678020c0 0x55db67810160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:09:24.524 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:24.519+0000 7f452f400640 1 -- 172.21.15.12:0/3854047244 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55db678109f0 con 0x55db677ff790 2024-09-11T00:09:24.524 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:24.519+0000 7f452f400640 1 --2- 172.21.15.12:0/3854047244 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55db677ff790 0x55db6780fc20 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f452000b890 tx=0x7f4520003e00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8608c4fd3b58bd29 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:09:24.524 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:24.519+0000 7f451cc00640 10 client.0 ms_handle_connect on v2:172.21.15.178:3300/0 2024-09-11T00:09:24.524 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:24.519+0000 7f451cc00640 1 -- 172.21.15.12:0/3854047244 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4520005220 con 0x55db677ff790 2024-09-11T00:09:24.524 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:24.519+0000 7f451cc00640 1 -- 172.21.15.12:0/3854047244 <== mon.1 v2:172.21.15.178:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9208+0+0 (secure 0 0 0) 0x7f45200053c0 con 0x55db677ff790 2024-09-11T00:09:24.524 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:24.519+0000 7f45391e9740 1 -- 172.21.15.12:0/3854047244 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55db677700b0 con 0x55db677ff790 2024-09-11T00:09:24.524 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:24.520+0000 7f451cc00640 1 -- 172.21.15.12:0/3854047244 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f4520005bf0 con 0x55db677ff790 2024-09-11T00:09:24.525 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:24.520+0000 7f45391e9740 10 client.25801 fetch_fsmap learned FSMap version 33 2024-09-11T00:09:24.525 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:24.520+0000 7f45391e9740 1 -- 172.21.15.12:0/3854047244 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55db67881e10 con 0x55db677ff790 2024-09-11T00:09:24.525 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:24.520+0000 7f451cc00640 1 -- 172.21.15.12:0/3854047244 <== mon.1 v2:172.21.15.178:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55db677700b0 con 0x55db677ff790 2024-09-11T00:09:24.525 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:24.520+0000 7f451cc00640 1 -- 172.21.15.12:0/3854047244 <== mon.1 v2:172.21.15.178:3300/0 5 ==== fsmap(e 33) ==== 8386+0+0 (secure 0 0 0) 0x7f45200049b0 con 0x55db677ff790 2024-09-11T00:09:24.525 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:24.520+0000 7f45391e9740 10 client.25801 fetch_fsmap finished waiting for FSMap version 33 2024-09-11T00:09:24.525 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:24.520+0000 7f45391e9740 10 client.25801 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T00:09:24.525 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:24.520+0000 7f45391e9740 20 client.25801 populate_metadata read hostname 'smithi012' 2024-09-11T00:09:24.525 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:24.520+0000 7f45391e9740 1 --2- 172.21.15.12:0/3854047244 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55db67882750 0x55db67884b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:09:24.525 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:24.520+0000 7f45391e9740 4 client.25801 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T00:09:24.525 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:24.520+0000 7f45391e9740 1 -- 172.21.15.12:0/3854047244 --> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55db6776d4e0 con 0x55db67882750 2024-09-11T00:09:24.525 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:24.520+0000 7f4526a00640 1 --2- 172.21.15.12:0/3854047244 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55db67882750 0x55db67884b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:09:24.525 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:24.521+0000 7f4526a00640 1 --2- 172.21.15.12:0/3854047244 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55db67882750 0x55db67884b70 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8207b4300770f6cc server_cookie=23794458cf07b2c4 in_seq=0 out_seq=0 2024-09-11T00:09:24.525 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:24.521+0000 7f451cc00640 10 client.25801 ms_handle_connect on v2:172.21.15.165:6838/501312984 2024-09-11T00:09:24.549 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:24.547+0000 7f451cc00640 1 -- 172.21.15.12:0/3854047244 <== mds.0 v2:172.21.15.165:6838/501312984 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55db6776d4e0 con 0x55db67882750 2024-09-11T00:09:24.549 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:24.547+0000 7f451cc00640 10 client.25801 handle_command_reply: tid=0 2024-09-11T00:09:24.549 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:24.548+0000 7f45391e9740 1 client.25801 shutdown 2024-09-11T00:09:24.549 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:24.548+0000 7f45391e9740 1 -- 172.21.15.12:0/3854047244 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55db677ff790 msgr2=0x55db6780fc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:09:24.549 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:24.548+0000 7f45391e9740 1 --2- 172.21.15.12:0/3854047244 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55db677ff790 0x55db6780fc20 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f452000b890 tx=0x7f4520003e00 comp rx=0 tx=0).stop 2024-09-11T00:09:24.550 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:24.548+0000 7f45391e9740 1 -- 172.21.15.12:0/3854047244 shutdown_connections 2024-09-11T00:09:24.550 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:24.548+0000 7f45391e9740 1 --2- 172.21.15.12:0/3854047244 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55db67882750 0x55db67884b70 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:09:24.550 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:24.548+0000 7f45391e9740 1 --2- 172.21.15.12:0/3854047244 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55db67804b00 0x55db6780e740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:09:24.550 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:24.548+0000 7f451cc00640 0 client.25801 ms_handle_reset on v2:172.21.15.165:6838/501312984 2024-09-11T00:09:24.550 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:24.548+0000 7f45391e9740 1 --2- 172.21.15.12:0/3854047244 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55db678020c0 0x55db67810160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:09:24.550 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:24.548+0000 7f45391e9740 1 --2- 172.21.15.12:0/3854047244 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55db677ff790 0x55db6780fc20 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:09:24.550 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:24.548+0000 7f45391e9740 1 -- 172.21.15.12:0/3854047244 >> 172.21.15.12:0/3854047244 conn(0x55db677fa8e0 msgr2=0x55db67807ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:09:24.550 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:24.548+0000 7f45391e9740 1 -- 172.21.15.12:0/3854047244 shutdown_connections 2024-09-11T00:09:24.550 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:24.548+0000 7f45391e9740 1 -- 172.21.15.12:0/3854047244 wait complete. 2024-09-11T00:09:24.550 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:24.548+0000 7f45391e9740 20 client.25801 trim_cache size 0 max 16384 2024-09-11T00:09:24.551 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:{ 2024-09-11T00:09:24.551 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "message": "", 2024-09-11T00:09:24.551 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "return_code": 0 2024-09-11T00:09:24.551 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:} 2024-09-11T00:09:24.551 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:24.549+0000 7f4536200640 1 -- 172.21.15.12:0/1079635166 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f45140b52e0 msgr2=0x7f45140b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:09:24.551 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:24.549+0000 7f4536200640 1 --2- 172.21.15.12:0/1079635166 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f45140b52e0 0x7f45140b77a0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f453019bcf0 tx=0x7f4528009500 comp rx=0 tx=0).stop 2024-09-11T00:09:24.552 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:24.549+0000 7f4536200640 1 -- 172.21.15.12:0/1079635166 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4530105ce0 msgr2=0x7f453019b5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:09:24.552 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:24.549+0000 7f4536200640 1 --2- 172.21.15.12:0/1079635166 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4530105ce0 0x7f453019b5c0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f451802a940 tx=0x7f4518003c10 comp rx=0 tx=0).stop 2024-09-11T00:09:24.552 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:24.549+0000 7f4536200640 1 -- 172.21.15.12:0/1079635166 shutdown_connections 2024-09-11T00:09:24.552 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:24.549+0000 7f4536200640 1 --2- 172.21.15.12:0/1079635166 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f45140b52e0 0x7f45140b77a0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:09:24.552 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:24.549+0000 7f4536200640 1 --2- 172.21.15.12:0/1079635166 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f45301066a0 0x7f453019c1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:09:24.552 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:24.549+0000 7f4536200640 1 --2- 172.21.15.12:0/1079635166 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4530105ce0 0x7f453019b5c0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:09:24.552 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:24.549+0000 7f4536200640 1 --2- 172.21.15.12:0/1079635166 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4530104b90 0x7f453019af70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:09:24.552 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:24.549+0000 7f4536200640 1 -- 172.21.15.12:0/1079635166 >> 172.21.15.12:0/1079635166 conn(0x7f4530100470 msgr2=0x7f4530109070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:09:24.552 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:24.549+0000 7f4536200640 1 -- 172.21.15.12:0/1079635166 shutdown_connections 2024-09-11T00:09:24.552 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:24.549+0000 7f4536200640 1 -- 172.21.15.12:0/1079635166 wait complete. 2024-09-11T00:09:24.605 INFO:tasks.workunit.client.0.smithi012.stdout: 1 89449 6.16 MB/sec warmup 81 sec latency 11316.350 ms 2024-09-11T00:09:25.532 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:09:25 smithi165 ceph-mon[35356]: pgmap v469: 129 pgs: 129 active+clean; 26 MiB data, 516 MiB used, 1.0 TiB / 1.0 TiB avail; 419 KiB/s rd, 283 KiB/s wr, 54 op/s 2024-09-11T00:09:25.563 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:09:25 smithi012 ceph-mon[30333]: pgmap v469: 129 pgs: 129 active+clean; 26 MiB data, 516 MiB used, 1.0 TiB / 1.0 TiB avail; 419 KiB/s rd, 283 KiB/s wr, 54 op/s 2024-09-11T00:09:25.580 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:09:25 smithi178 ceph-mon[35919]: pgmap v469: 129 pgs: 129 active+clean; 26 MiB data, 516 MiB used, 1.0 TiB / 1.0 TiB avail; 419 KiB/s rd, 283 KiB/s wr, 54 op/s 2024-09-11T00:09:25.606 INFO:tasks.workunit.client.0.smithi012.stdout: 1 89449 6.09 MB/sec warmup 82 sec latency 12316.464 ms 2024-09-11T00:09:26.606 INFO:tasks.workunit.client.0.smithi012.stdout: 1 89449 6.01 MB/sec warmup 83 sec latency 13316.558 ms 2024-09-11T00:09:27.532 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:09:27 smithi165 ceph-mon[35356]: pgmap v470: 129 pgs: 129 active+clean; 26 MiB data, 516 MiB used, 1.0 TiB / 1.0 TiB avail; 380 KiB/s rd, 334 KiB/s wr, 61 op/s 2024-09-11T00:09:27.563 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:09:27 smithi012 ceph-mon[30333]: pgmap v470: 129 pgs: 129 active+clean; 26 MiB data, 516 MiB used, 1.0 TiB / 1.0 TiB avail; 380 KiB/s rd, 334 KiB/s wr, 61 op/s 2024-09-11T00:09:27.580 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:09:27 smithi178 ceph-mon[35919]: pgmap v470: 129 pgs: 129 active+clean; 26 MiB data, 516 MiB used, 1.0 TiB / 1.0 TiB avail; 380 KiB/s rd, 334 KiB/s wr, 61 op/s 2024-09-11T00:09:27.606 INFO:tasks.workunit.client.0.smithi012.stdout: 1 89449 5.94 MB/sec warmup 84 sec latency 14316.707 ms 2024-09-11T00:09:28.606 INFO:tasks.workunit.client.0.smithi012.stdout: 1 89449 5.87 MB/sec warmup 85 sec latency 15316.801 ms 2024-09-11T00:09:29.532 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:09:29 smithi165 ceph-mon[35356]: pgmap v471: 129 pgs: 129 active+clean; 26 MiB data, 517 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 127 KiB/s wr, 21 op/s 2024-09-11T00:09:29.563 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:09:29 smithi012 ceph-mon[30333]: pgmap v471: 129 pgs: 129 active+clean; 26 MiB data, 517 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 127 KiB/s wr, 21 op/s 2024-09-11T00:09:29.581 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:09:29 smithi178 ceph-mon[35919]: pgmap v471: 129 pgs: 129 active+clean; 26 MiB data, 517 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 127 KiB/s wr, 21 op/s 2024-09-11T00:09:29.606 INFO:tasks.workunit.client.0.smithi012.stdout: 1 89449 5.81 MB/sec warmup 86 sec latency 16316.924 ms 2024-09-11T00:09:30.596 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid ff3ad74a-6fcf-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T00:09:30.606 INFO:tasks.workunit.client.0.smithi012.stdout: 1 89449 5.74 MB/sec warmup 87 sec latency 17317.021 ms 2024-09-11T00:09:31.041 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:09:31.364 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:09:31 smithi012 ceph-mon[30333]: pgmap v472: 129 pgs: 129 active+clean; 26 MiB data, 517 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 120 KiB/s wr, 21 op/s 2024-09-11T00:09:31.532 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:09:31 smithi165 ceph-mon[35356]: pgmap v472: 129 pgs: 129 active+clean; 26 MiB data, 517 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 120 KiB/s wr, 21 op/s 2024-09-11T00:09:31.581 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:09:31 smithi178 ceph-mon[35919]: pgmap v472: 129 pgs: 129 active+clean; 26 MiB data, 517 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 120 KiB/s wr, 21 op/s 2024-09-11T00:09:31.606 INFO:tasks.workunit.client.0.smithi012.stdout: 1 89449 5.67 MB/sec warmup 88 sec latency 18317.129 ms 2024-09-11T00:09:31.738 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:31.737+0000 7f5354a00640 1 -- 172.21.15.12:0/2682329560 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5350077ff0 msgr2=0x7f53501132a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:09:31.738 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:31.737+0000 7f5354a00640 1 --2- 172.21.15.12:0/2682329560 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5350077ff0 0x7f53501132a0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f5338009900 tx=0x7f533802a3f0 comp rx=0 tx=0).stop 2024-09-11T00:09:31.738 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:31.737+0000 7f5354a00640 1 -- 172.21.15.12:0/2682329560 shutdown_connections 2024-09-11T00:09:31.738 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:31.737+0000 7f5354a00640 1 --2- 172.21.15.12:0/2682329560 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f53501137e0 0x7f5350115bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:09:31.738 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:31.737+0000 7f5354a00640 1 --2- 172.21.15.12:0/2682329560 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5350077ff0 0x7f53501132a0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:09:31.739 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:31.737+0000 7f5354a00640 1 --2- 172.21.15.12:0/2682329560 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f53500776d0 0x7f5350077ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:09:31.739 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:31.737+0000 7f5354a00640 1 -- 172.21.15.12:0/2682329560 >> 172.21.15.12:0/2682329560 conn(0x7f5350100840 msgr2=0x7f5350102c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:09:31.739 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:31.737+0000 7f5354a00640 1 -- 172.21.15.12:0/2682329560 shutdown_connections 2024-09-11T00:09:31.739 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:31.737+0000 7f5354a00640 1 -- 172.21.15.12:0/2682329560 wait complete. 2024-09-11T00:09:31.739 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:31.738+0000 7f5354a00640 1 Processor -- start 2024-09-11T00:09:31.739 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:31.739+0000 7f5354a00640 1 -- start start 2024-09-11T00:09:31.740 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:31.739+0000 7f5354a00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f53500776d0 0x7f535019f610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:09:31.740 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:31.739+0000 7f5354a00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5350077ff0 0x7f535019fb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:09:31.740 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:31.739+0000 7f5354a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f53501137e0 0x7f53501a4ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:09:31.740 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:31.739+0000 7f5354a00640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f5350119220 con 0x7f5350077ff0 2024-09-11T00:09:31.740 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:31.739+0000 7f5354a00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f53501190a0 con 0x7f53501137e0 2024-09-11T00:09:31.740 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:31.739+0000 7f5354a00640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f53501193a0 con 0x7f53500776d0 2024-09-11T00:09:31.740 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:31.739+0000 7f5345e00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5350077ff0 0x7f535019fb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:09:31.741 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:31.739+0000 7f5345e00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5350077ff0 0x7f535019fb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:40226/0 (socket says 172.21.15.12:40226) 2024-09-11T00:09:31.741 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:31.739+0000 7f5345e00640 1 -- 172.21.15.12:0/178202227 learned_addr learned my addr 172.21.15.12:0/178202227 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:09:31.742 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:31.739+0000 7f534f200640 1 --2- 172.21.15.12:0/178202227 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f53501137e0 0x7f53501a4ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:09:31.742 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:31.740+0000 7f534e800640 1 --2- 172.21.15.12:0/178202227 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f53500776d0 0x7f535019f610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:09:31.742 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:31.740+0000 7f5345e00640 1 -- 172.21.15.12:0/178202227 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f53500776d0 msgr2=0x7f535019f610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:09:31.742 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:31.740+0000 7f5345e00640 1 --2- 172.21.15.12:0/178202227 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f53500776d0 0x7f535019f610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:09:31.742 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:31.740+0000 7f5345e00640 1 -- 172.21.15.12:0/178202227 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f53501137e0 msgr2=0x7f53501a4ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:09:31.742 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:31.740+0000 7f5345e00640 1 --2- 172.21.15.12:0/178202227 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f53501137e0 0x7f53501a4ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:09:31.742 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:31.740+0000 7f5345e00640 1 -- 172.21.15.12:0/178202227 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53501a56c0 con 0x7f5350077ff0 2024-09-11T00:09:31.742 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:31.740+0000 7f5345e00640 1 --2- 172.21.15.12:0/178202227 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5350077ff0 0x7f535019fb50 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f5338005ab0 tx=0x7f5338003cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:09:31.742 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:31.740+0000 7f5347e00640 1 -- 172.21.15.12:0/178202227 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f533803f070 con 0x7f5350077ff0 2024-09-11T00:09:31.742 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:31.740+0000 7f5347e00640 1 -- 172.21.15.12:0/178202227 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5338031d60 con 0x7f5350077ff0 2024-09-11T00:09:31.742 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:31.740+0000 7f5347e00640 1 -- 172.21.15.12:0/178202227 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5338035030 con 0x7f5350077ff0 2024-09-11T00:09:31.742 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:31.741+0000 7f5354a00640 1 -- 172.21.15.12:0/178202227 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f53501a5950 con 0x7f5350077ff0 2024-09-11T00:09:31.742 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:31.741+0000 7f5354a00640 1 -- 172.21.15.12:0/178202227 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53501ac070 con 0x7f5350077ff0 2024-09-11T00:09:31.743 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:31.742+0000 7f5354a00640 1 -- 172.21.15.12:0/178202227 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5350077ab0 con 0x7f5350077ff0 2024-09-11T00:09:31.745 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:31.744+0000 7f5347e00640 1 -- 172.21.15.12:0/178202227 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7f533803b560 con 0x7f5350077ff0 2024-09-11T00:09:31.748 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:31.746+0000 7f5347e00640 1 --2- 172.21.15.12:0/178202227 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f53340b5410 0x7f53340b78d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:09:31.748 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:31.747+0000 7f5347e00640 1 -- 172.21.15.12:0/178202227 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9208+0+0 (secure 0 0 0) 0x7f53381070a0 con 0x7f5350077ff0 2024-09-11T00:09:31.748 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:31.747+0000 7f534e800640 1 --2- 172.21.15.12:0/178202227 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f53340b5410 0x7f53340b78d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:09:31.748 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:31.748+0000 7f534e800640 1 --2- 172.21.15.12:0/178202227 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f53340b5410 0x7f53340b78d0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f5340005d00 tx=0x7f534000b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:09:31.749 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:31.748+0000 7f5347e00640 1 -- 172.21.15.12:0/178202227 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f53380cd4e0 con 0x7f5350077ff0 2024-09-11T00:09:31.992 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:31.991+0000 7f5354a00640 1 -- 172.21.15.12:0/178202227 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f53501a0740 con 0x7f5350077ff0 2024-09-11T00:09:31.993 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:31.992+0000 7f5347e00640 1 -- 172.21.15.12:0/178202227 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) ==== 94+0+11972 (secure 0 0 0) 0x7f53501a0740 con 0x7f5350077ff0 2024-09-11T00:09:31.994 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-11T00:09:31.994 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":33,"btime":"2024-09-11T00:08:46:706969+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14577,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6835/447171591","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":447171591},{"type":"v1","addr":"172.21.15.12:6835","nonce":447171591}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14589,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/901486040","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":901486040},{"type":"v1","addr":"172.21.15.12:6837","nonce":901486040}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":33,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T00:04:04.739881+0000","modified":"2024-09-11T00:08:46.706950+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24674,"mds_1":24643,"mds_2":24638,"mds_3":24637,"mds_4":24614},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14631":{"gid":14631,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.12:6841/3069919437","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6840","nonce":3069919437},{"type":"v1","addr":"172.21.15.12:6841","nonce":3069919437}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24562":{"gid":24562,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6833/1141636112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":1141636112},{"type":"v1","addr":"172.21.15.178:6833","nonce":1141636112}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24581":{"gid":24581,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6833/3111980970","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6832","nonce":3111980970},{"type":"v1","addr":"172.21.15.165:6833","nonce":3111980970}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24607":{"gid":24607,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6835/4084868287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":4084868287},{"type":"v1","addr":"172.21.15.178:6835","nonce":4084868287}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24614":{"gid":24614,"name":"e","rank":4,"incarnation":26,"state":"up:active","state_seq":21,"addr":"172.21.15.165:6835/2425395274","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":2425395274},{"type":"v1","addr":"172.21.15.165:6835","nonce":2425395274}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24637":{"gid":24637,"name":"i","rank":3,"incarnation":24,"state":"up:active","state_seq":18,"addr":"172.21.15.178:6839/70628097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6838","nonce":70628097},{"type":"v1","addr":"172.21.15.178:6839","nonce":70628097}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24638":{"gid":24638,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":18,"addr":"172.21.15.165:6837/6215088","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6836","nonce":6215088},{"type":"v1","addr":"172.21.15.165:6837","nonce":6215088}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24643":{"gid":24643,"name":"g","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.12:6839/3124093009","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6838","nonce":3124093009},{"type":"v1","addr":"172.21.15.12:6839","nonce":3124093009}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24674":{"gid":24674,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.165:6839/501312984","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6838","nonce":501312984},{"type":"v1","addr":"172.21.15.165:6839","nonce":501312984}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25672":{"gid":25672,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6837/308601741","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":308601741},{"type":"v1","addr":"172.21.15.178:6837","nonce":308601741}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24674,"qdb_cluster":[24614,24637,24638,24643,24674]},"id":1}]} 2024-09-11T00:09:31.995 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 33 2024-09-11T00:09:31.996 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:31.995+0000 7f5354a00640 1 -- 172.21.15.12:0/178202227 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f53340b5410 msgr2=0x7f53340b78d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:09:31.996 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:31.995+0000 7f5354a00640 1 --2- 172.21.15.12:0/178202227 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f53340b5410 0x7f53340b78d0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f5340005d00 tx=0x7f534000b410 comp rx=0 tx=0).stop 2024-09-11T00:09:31.997 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:31.995+0000 7f5354a00640 1 -- 172.21.15.12:0/178202227 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5350077ff0 msgr2=0x7f535019fb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:09:31.997 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:31.995+0000 7f5354a00640 1 --2- 172.21.15.12:0/178202227 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5350077ff0 0x7f535019fb50 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f5338005ab0 tx=0x7f5338003cd0 comp rx=0 tx=0).stop 2024-09-11T00:09:31.997 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:31.995+0000 7f5354a00640 1 -- 172.21.15.12:0/178202227 shutdown_connections 2024-09-11T00:09:31.997 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:31.995+0000 7f5354a00640 1 --2- 172.21.15.12:0/178202227 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f53340b5410 0x7f53340b78d0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:09:31.997 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:31.995+0000 7f5354a00640 1 --2- 172.21.15.12:0/178202227 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f53501137e0 0x7f53501a4ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:09:31.997 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:31.995+0000 7f5354a00640 1 --2- 172.21.15.12:0/178202227 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5350077ff0 0x7f535019fb50 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:09:31.997 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:31.995+0000 7f5354a00640 1 --2- 172.21.15.12:0/178202227 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f53500776d0 0x7f535019f610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:09:31.997 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:31.995+0000 7f5354a00640 1 -- 172.21.15.12:0/178202227 >> 172.21.15.12:0/178202227 conn(0x7f5350100840 msgr2=0x7f5350102c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:09:31.997 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:31.995+0000 7f5354a00640 1 -- 172.21.15.12:0/178202227 shutdown_connections 2024-09-11T00:09:31.997 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:31.996+0000 7f5354a00640 1 -- 172.21.15.12:0/178202227 wait complete. 2024-09-11T00:09:32.532 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:09:32 smithi165 ceph-mon[35356]: from='client.? 172.21.15.12:0/178202227' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T00:09:32.563 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:09:32 smithi012 ceph-mon[30333]: from='client.? 172.21.15.12:0/178202227' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T00:09:32.581 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:09:32 smithi178 ceph-mon[35919]: from='client.? 172.21.15.12:0/178202227' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T00:09:32.607 INFO:tasks.workunit.client.0.smithi012.stdout: 1 89449 5.61 MB/sec warmup 89 sec latency 19317.276 ms 2024-09-11T00:09:32.859 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24674 quiesce db --set-id 68ee5ab7 --release --await' 2024-09-11T00:09:32.859 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid ff3ad74a-6fcf-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24674 quiesce db --set-id 68ee5ab7 --release --await 2024-09-11T00:09:33.282 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:09:33.532 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:09:33 smithi165 ceph-mon[35356]: pgmap v473: 129 pgs: 129 active+clean; 26 MiB data, 518 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 135 KiB/s wr, 22 op/s 2024-09-11T00:09:33.563 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:09:33 smithi012 ceph-mon[30333]: pgmap v473: 129 pgs: 129 active+clean; 26 MiB data, 518 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 135 KiB/s wr, 22 op/s 2024-09-11T00:09:33.580 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:09:33 smithi178 ceph-mon[35919]: pgmap v473: 129 pgs: 129 active+clean; 26 MiB data, 518 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 135 KiB/s wr, 22 op/s 2024-09-11T00:09:33.606 INFO:tasks.workunit.client.0.smithi012.stdout: 1 89449 5.55 MB/sec warmup 90 sec latency 20317.343 ms 2024-09-11T00:09:33.860 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:33.858+0000 7f9deb200640 1 -- 172.21.15.12:0/4135476576 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9de410a9b0 msgr2=0x7f9de410adf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:09:33.860 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:33.858+0000 7f9deb200640 1 --2- 172.21.15.12:0/4135476576 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9de410a9b0 0x7f9de410adf0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f9dcc009900 tx=0x7f9dcc02a3f0 comp rx=0 tx=0).stop 2024-09-11T00:09:33.860 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:33.858+0000 7f9deb200640 1 -- 172.21.15.12:0/4135476576 shutdown_connections 2024-09-11T00:09:33.860 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:33.858+0000 7f9deb200640 1 --2- 172.21.15.12:0/4135476576 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9de410b330 0x7f9de4111b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:09:33.860 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:33.858+0000 7f9deb200640 1 --2- 172.21.15.12:0/4135476576 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9de410a9b0 0x7f9de410adf0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:09:33.860 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:33.858+0000 7f9deb200640 1 --2- 172.21.15.12:0/4135476576 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9de4104fa0 0x7f9de4105380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:09:33.860 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:33.858+0000 7f9deb200640 1 -- 172.21.15.12:0/4135476576 >> 172.21.15.12:0/4135476576 conn(0x7f9de4100a50 msgr2=0x7f9de4102e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:09:33.860 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:33.858+0000 7f9deb200640 1 -- 172.21.15.12:0/4135476576 shutdown_connections 2024-09-11T00:09:33.860 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:33.858+0000 7f9deb200640 1 -- 172.21.15.12:0/4135476576 wait complete. 2024-09-11T00:09:33.862 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:33.859+0000 7f9deb200640 1 Processor -- start 2024-09-11T00:09:33.862 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:33.859+0000 7f9deb200640 1 -- start start 2024-09-11T00:09:33.862 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:33.859+0000 7f9deb200640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9de4104fa0 0x7f9de419b470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:09:33.862 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:33.859+0000 7f9deb200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9de410a9b0 0x7f9de419b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:09:33.862 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:33.860+0000 7f9deb200640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9de410b330 0x7f9de41a0d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:09:33.862 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:33.860+0000 7f9deb200640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f9de4114f60 con 0x7f9de410b330 2024-09-11T00:09:33.862 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:33.860+0000 7f9deb200640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f9de4114de0 con 0x7f9de410a9b0 2024-09-11T00:09:33.862 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:33.860+0000 7f9deb200640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f9de41150e0 con 0x7f9de4104fa0 2024-09-11T00:09:33.862 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:33.860+0000 7f9de9200640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9de410b330 0x7f9de41a0d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:09:33.862 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:33.860+0000 7f9de9200640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9de410b330 0x7f9de41a0d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:40246/0 (socket says 172.21.15.12:40246) 2024-09-11T00:09:33.862 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:33.860+0000 7f9de9200640 1 -- 172.21.15.12:0/1108356628 learned_addr learned my addr 172.21.15.12:0/1108356628 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:09:33.863 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:33.860+0000 7f9de3e00640 1 --2- 172.21.15.12:0/1108356628 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9de4104fa0 0x7f9de419b470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:09:33.863 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:33.860+0000 7f9de3400640 1 --2- 172.21.15.12:0/1108356628 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9de410a9b0 0x7f9de419b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:09:33.863 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:33.860+0000 7f9de3e00640 1 -- 172.21.15.12:0/1108356628 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9de410a9b0 msgr2=0x7f9de419b9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:09:33.863 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:33.860+0000 7f9de3e00640 1 --2- 172.21.15.12:0/1108356628 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9de410a9b0 0x7f9de419b9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:09:33.863 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:33.860+0000 7f9de3e00640 1 -- 172.21.15.12:0/1108356628 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9de410b330 msgr2=0x7f9de41a0d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:09:33.863 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:33.860+0000 7f9de3e00640 1 --2- 172.21.15.12:0/1108356628 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9de410b330 0x7f9de41a0d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:09:33.863 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:33.860+0000 7f9de3e00640 1 -- 172.21.15.12:0/1108356628 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9de41a1520 con 0x7f9de4104fa0 2024-09-11T00:09:33.863 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:33.860+0000 7f9de9200640 1 --2- 172.21.15.12:0/1108356628 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9de410b330 0x7f9de41a0d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:09:33.863 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:33.861+0000 7f9de3e00640 1 --2- 172.21.15.12:0/1108356628 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9de4104fa0 0x7f9de419b470 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f9dd400eb00 tx=0x7f9dd400c2d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:09:33.864 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:33.861+0000 7f9de0c00640 1 -- 172.21.15.12:0/1108356628 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9dd4010430 con 0x7f9de4104fa0 2024-09-11T00:09:33.864 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:33.861+0000 7f9de0c00640 1 -- 172.21.15.12:0/1108356628 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9dd4010a80 con 0x7f9de4104fa0 2024-09-11T00:09:33.864 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:33.861+0000 7f9deb200640 1 -- 172.21.15.12:0/1108356628 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9de41a1810 con 0x7f9de4104fa0 2024-09-11T00:09:33.864 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:33.861+0000 7f9de0c00640 1 -- 172.21.15.12:0/1108356628 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9dd4017a90 con 0x7f9de4104fa0 2024-09-11T00:09:33.864 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:33.861+0000 7f9deb200640 1 -- 172.21.15.12:0/1108356628 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9de41a7ed0 con 0x7f9de4104fa0 2024-09-11T00:09:33.865 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:33.864+0000 7f9deddff740 1 Processor -- start 2024-09-11T00:09:33.865 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:33.864+0000 7f9deddff740 1 -- start start 2024-09-11T00:09:33.866 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:33.865+0000 7f9deddff740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556dd7205ff0 0x556dd72063d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:09:33.866 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:33.865+0000 7f9deddff740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556dd7206910 0x556dd7212360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:09:33.866 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:33.865+0000 7f9deddff740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556dd72128a0 0x556dd7214c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:09:33.866 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:33.865+0000 7f9deddff740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x556dd717d290 con 0x556dd72128a0 2024-09-11T00:09:33.866 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:33.865+0000 7f9deddff740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x556dd716c6c0 con 0x556dd7205ff0 2024-09-11T00:09:33.866 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:33.865+0000 7f9deddff740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x556dd71704d0 con 0x556dd7206910 2024-09-11T00:09:33.867 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:33.865+0000 7f9de0c00640 1 -- 172.21.15.12:0/1108356628 <== mon.1 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7f9dd4017c70 con 0x7f9de4104fa0 2024-09-11T00:09:33.867 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:33.865+0000 7f9de3400640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556dd72128a0 0x556dd7214c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:09:33.867 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:33.865+0000 7f9de9200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556dd7205ff0 0x556dd72063d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:09:33.867 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:33.865+0000 7f9de3400640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556dd72128a0 0x556dd7214c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:40250/0 (socket says 172.21.15.12:40250) 2024-09-11T00:09:33.867 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:33.865+0000 7f9de3400640 1 -- 172.21.15.12:0/3302754382 learned_addr learned my addr 172.21.15.12:0/3302754382 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:09:33.867 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:33.865+0000 7f9de3e00640 1 --2- 172.21.15.12:0/3302754382 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556dd7206910 0x556dd7212360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:09:33.868 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:33.866+0000 7f9de9200640 1 -- 172.21.15.12:0/3302754382 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556dd7206910 msgr2=0x556dd7212360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:09:33.868 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:33.866+0000 7f9de9200640 1 --2- 172.21.15.12:0/3302754382 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556dd7206910 0x556dd7212360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:09:33.868 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:33.866+0000 7f9de9200640 1 -- 172.21.15.12:0/3302754382 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556dd72128a0 msgr2=0x556dd7214c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:09:33.868 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:33.866+0000 7f9de9200640 1 --2- 172.21.15.12:0/3302754382 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556dd72128a0 0x556dd7214c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:09:33.868 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:33.866+0000 7f9de9200640 1 -- 172.21.15.12:0/3302754382 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556dd70f9460 con 0x556dd7205ff0 2024-09-11T00:09:33.868 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:33.866+0000 7f9de3e00640 1 --2- 172.21.15.12:0/3302754382 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556dd7206910 0x556dd7212360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T00:09:33.868 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:33.866+0000 7f9de9200640 1 --2- 172.21.15.12:0/3302754382 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556dd7205ff0 0x556dd72063d0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f9ddc011490 tx=0x7f9ddc011950 comp rx=0 tx=0).ready entity=mon.2 client_cookie=16199e14ff4cc964 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:09:33.868 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:33.866+0000 7f9dd9600640 1 -- 172.21.15.12:0/3302754382 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ddc01e070 con 0x556dd7205ff0 2024-09-11T00:09:33.868 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:33.866+0000 7f9dd9600640 1 -- 172.21.15.12:0/3302754382 <== mon.2 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9ddc00f9f0 con 0x556dd7205ff0 2024-09-11T00:09:33.868 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:33.866+0000 7f9dd9600640 1 -- 172.21.15.12:0/3302754382 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ddc019ac0 con 0x556dd7205ff0 2024-09-11T00:09:33.869 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:33.867+0000 7f9deddff740 1 -- 172.21.15.12:0/3302754382 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556dd7205ff0 msgr2=0x556dd72063d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:09:33.869 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:33.867+0000 7f9deddff740 1 --2- 172.21.15.12:0/3302754382 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556dd7205ff0 0x556dd72063d0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f9ddc011490 tx=0x7f9ddc011950 comp rx=0 tx=0).stop 2024-09-11T00:09:33.870 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:33.867+0000 7f9deddff740 1 -- 172.21.15.12:0/3302754382 shutdown_connections 2024-09-11T00:09:33.870 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:33.867+0000 7f9deddff740 1 --2- 172.21.15.12:0/3302754382 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556dd72128a0 0x556dd7214c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:09:33.870 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:33.867+0000 7f9deddff740 1 --2- 172.21.15.12:0/3302754382 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556dd7206910 0x556dd7212360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:09:33.870 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:33.867+0000 7f9deddff740 1 --2- 172.21.15.12:0/3302754382 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556dd7205ff0 0x556dd72063d0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:09:33.870 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:33.867+0000 7f9de0c00640 1 --2- 172.21.15.12:0/1108356628 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f9dc80b52e0 0x7f9dc80b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:09:33.870 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:33.867+0000 7f9deddff740 1 -- 172.21.15.12:0/3302754382 >> 172.21.15.12:0/3302754382 conn(0x556dd7203e20 msgr2=0x556dd7204fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:09:33.870 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:33.867+0000 7f9de0c00640 1 -- 172.21.15.12:0/1108356628 <== mon.1 v2:172.21.15.178:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9208+0+0 (secure 0 0 0) 0x7f9dd40e6a50 con 0x7f9de4104fa0 2024-09-11T00:09:33.870 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:33.867+0000 7f9de9200640 1 --2- 172.21.15.12:0/1108356628 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f9dc80b52e0 0x7f9dc80b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:09:33.870 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:33.868+0000 7f9deddff740 1 -- 172.21.15.12:0/3302754382 shutdown_connections 2024-09-11T00:09:33.870 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:33.868+0000 7f9deddff740 1 -- 172.21.15.12:0/3302754382 wait complete. 2024-09-11T00:09:33.870 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:33.868+0000 7f9deddff740 1 -- start start 2024-09-11T00:09:33.870 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:33.868+0000 7f9de9200640 1 --2- 172.21.15.12:0/1108356628 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f9dc80b52e0 0x7f9dc80b77a0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f9ddc009640 tx=0x7f9ddc0097a0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:09:33.870 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:33.868+0000 7f9deddff740 1 Processor -- start 2024-09-11T00:09:33.872 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:33.869+0000 7f9deddff740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556dd7205ff0 0x556dd720f7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:09:33.872 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:33.869+0000 7f9deddff740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556dd7206910 0x556dd720fd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:09:33.872 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:33.869+0000 7f9deddff740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556dd72128a0 0x556dd7210260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:09:33.872 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:33.869+0000 7f9deddff740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x556dd6fcdbb0 con 0x556dd72128a0 2024-09-11T00:09:33.872 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:33.869+0000 7f9deddff740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x556dd71076c0 con 0x556dd7206910 2024-09-11T00:09:33.872 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:33.869+0000 7f9deddff740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x556dd7215550 con 0x556dd7205ff0 2024-09-11T00:09:33.872 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:33.869+0000 7f9deddff740 10 client.0 _ll_register_callbacks cb 0x556dd7122490 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:09:33.872 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:33.869+0000 7f9de9200640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556dd72128a0 0x556dd7210260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:09:33.872 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:33.869+0000 7f9de3400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556dd7206910 0x556dd720fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:09:33.873 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:33.869+0000 7f9de9200640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556dd72128a0 0x556dd7210260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:40262/0 (socket says 172.21.15.12:40262) 2024-09-11T00:09:33.873 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:33.869+0000 7f9de9200640 1 -- 172.21.15.12:0/1450307139 learned_addr learned my addr 172.21.15.12:0/1450307139 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:09:33.873 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:33.869+0000 7f9de3e00640 1 --2- 172.21.15.12:0/1450307139 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556dd7205ff0 0x556dd720f7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:09:33.873 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:33.870+0000 7f9de3400640 1 -- 172.21.15.12:0/1450307139 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556dd7205ff0 msgr2=0x556dd720f7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:09:33.873 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:33.870+0000 7f9de3400640 1 --2- 172.21.15.12:0/1450307139 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556dd7205ff0 0x556dd720f7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:09:33.873 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:33.870+0000 7f9de3400640 1 -- 172.21.15.12:0/1450307139 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556dd72128a0 msgr2=0x556dd7210260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:09:33.873 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:33.870+0000 7f9de3400640 1 --2- 172.21.15.12:0/1450307139 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556dd72128a0 0x556dd7210260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:09:33.873 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:33.870+0000 7f9de3400640 1 -- 172.21.15.12:0/1450307139 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556dd6ecab00 con 0x556dd7206910 2024-09-11T00:09:33.873 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:33.870+0000 7f9de3400640 1 --2- 172.21.15.12:0/1450307139 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556dd7206910 0x556dd720fd20 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f9dcc009230 tx=0x7f9dcc032f20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=df3e5df0f21cf025 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:09:33.873 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:33.870+0000 7f9dd2000640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-09-11T00:09:33.873 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:33.870+0000 7f9dd2000640 1 -- 172.21.15.12:0/1450307139 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9dcc035040 con 0x556dd7206910 2024-09-11T00:09:33.873 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:33.871+0000 7f9dd2000640 1 -- 172.21.15.12:0/1450307139 <== mon.2 v2:172.21.15.165:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9208+0+0 (secure 0 0 0) 0x7f9dcc031860 con 0x556dd7206910 2024-09-11T00:09:33.873 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:33.871+0000 7f9deddff740 1 -- 172.21.15.12:0/1450307139 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9ddc00f9f0 con 0x556dd7206910 2024-09-11T00:09:33.874 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:33.871+0000 7f9dd2000640 1 -- 172.21.15.12:0/1450307139 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f9dcc04f050 con 0x556dd7206910 2024-09-11T00:09:33.874 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:33.871+0000 7f9deddff740 10 client.25832 fetch_fsmap learned FSMap version 33 2024-09-11T00:09:33.874 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:33.871+0000 7f9deddff740 1 -- 172.21.15.12:0/1450307139 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x556dd6f41960 con 0x556dd7206910 2024-09-11T00:09:33.875 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:33.871+0000 7f9dd2000640 1 -- 172.21.15.12:0/1450307139 <== mon.2 v2:172.21.15.165:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9ddc00f9f0 con 0x556dd7206910 2024-09-11T00:09:33.875 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:33.872+0000 7f9dd2000640 1 -- 172.21.15.12:0/1450307139 <== mon.2 v2:172.21.15.165:3300/0 5 ==== fsmap(e 33) ==== 8386+0+0 (secure 0 0 0) 0x7f9dcc004750 con 0x556dd7206910 2024-09-11T00:09:33.875 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:33.872+0000 7f9deddff740 10 client.25832 fetch_fsmap finished waiting for FSMap version 33 2024-09-11T00:09:33.876 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:33.872+0000 7f9deddff740 10 client.25832 resolve_mds: validated gid 24674 aka daemon mds.b 2024-09-11T00:09:33.876 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:33.872+0000 7f9deddff740 20 client.25832 populate_metadata read hostname 'smithi012' 2024-09-11T00:09:33.876 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:33.872+0000 7f9deddff740 1 --2- 172.21.15.12:0/1450307139 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x556dd72223e0 0x556dd7224840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:09:33.876 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:33.872+0000 7f9deddff740 4 client.25832 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T00:09:33.876 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:33.872+0000 7f9deddff740 1 -- 172.21.15.12:0/1450307139 --> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556dd717f1e0 con 0x556dd72223e0 2024-09-11T00:09:33.876 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:33.872+0000 7f9de3e00640 1 --2- 172.21.15.12:0/1450307139 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x556dd72223e0 0x556dd7224840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:09:33.876 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:33.873+0000 7f9de3e00640 1 --2- 172.21.15.12:0/1450307139 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x556dd72223e0 0x556dd7224840 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eaa62e73ddd28c5c server_cookie=927f6c630d4e5f6e in_seq=0 out_seq=0 2024-09-11T00:09:33.876 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:33.873+0000 7f9dd2000640 10 client.25832 ms_handle_connect on v2:172.21.15.165:6838/501312984 2024-09-11T00:09:33.876 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:33.874+0000 7f9dd2000640 1 -- 172.21.15.12:0/1450307139 <== mds.0 v2:172.21.15.165:6838/501312984 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x556dd717f1e0 con 0x556dd72223e0 2024-09-11T00:09:33.876 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:33.874+0000 7f9dd2000640 10 client.25832 handle_command_reply: tid=0 2024-09-11T00:09:33.876 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:33.874+0000 7f9deddff740 1 client.25832 shutdown 2024-09-11T00:09:33.876 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:33.874+0000 7f9deddff740 1 -- 172.21.15.12:0/1450307139 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556dd7206910 msgr2=0x556dd720fd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:09:33.877 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:33.874+0000 7f9deddff740 1 --2- 172.21.15.12:0/1450307139 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556dd7206910 0x556dd720fd20 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f9dcc009230 tx=0x7f9dcc032f20 comp rx=0 tx=0).stop 2024-09-11T00:09:33.877 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:33.874+0000 7f9deddff740 1 -- 172.21.15.12:0/1450307139 shutdown_connections 2024-09-11T00:09:33.877 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:33.874+0000 7f9deddff740 1 --2- 172.21.15.12:0/1450307139 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x556dd72223e0 0x556dd7224840 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:09:33.877 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:33.874+0000 7f9deddff740 1 --2- 172.21.15.12:0/1450307139 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556dd72128a0 0x556dd7210260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:09:33.877 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:33.874+0000 7f9deddff740 1 --2- 172.21.15.12:0/1450307139 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556dd7206910 0x556dd720fd20 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:09:33.877 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:33.874+0000 7f9dd2000640 0 client.25832 ms_handle_reset on v2:172.21.15.165:6838/501312984 2024-09-11T00:09:33.877 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:33.874+0000 7f9deddff740 1 --2- 172.21.15.12:0/1450307139 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556dd7205ff0 0x556dd720f7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:09:33.877 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:33.874+0000 7f9deddff740 1 -- 172.21.15.12:0/1450307139 >> 172.21.15.12:0/1450307139 conn(0x556dd7203e20 msgr2=0x556dd72099c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:09:33.877 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:33.874+0000 7f9deddff740 1 -- 172.21.15.12:0/1450307139 shutdown_connections 2024-09-11T00:09:33.877 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:33.874+0000 7f9deddff740 1 -- 172.21.15.12:0/1450307139 wait complete. 2024-09-11T00:09:33.877 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:33.874+0000 7f9deddff740 20 client.25832 trim_cache size 0 max 16384 2024-09-11T00:09:33.894 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:33.893+0000 7f9deddff740 1 Processor -- start 2024-09-11T00:09:33.894 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:33.893+0000 7f9deddff740 1 -- start start 2024-09-11T00:09:33.895 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:33.893+0000 7f9deddff740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556dd7205ff0 0x556dd7211520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:09:33.895 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:33.893+0000 7f9deddff740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556dd7206910 0x556dd720c9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:09:33.895 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:33.893+0000 7f9deddff740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556dd72128a0 0x556dd720cf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:09:33.895 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:33.893+0000 7f9deddff740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f9de41b1d30 con 0x556dd7205ff0 2024-09-11T00:09:33.895 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:33.893+0000 7f9deddff740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f9de41b3930 con 0x556dd7206910 2024-09-11T00:09:33.895 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:33.893+0000 7f9deddff740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f9de41b2400 con 0x556dd72128a0 2024-09-11T00:09:33.895 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:33.893+0000 7f9de3e00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556dd7205ff0 0x556dd7211520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:09:33.895 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:33.893+0000 7f9de3e00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556dd7205ff0 0x556dd7211520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:40268/0 (socket says 172.21.15.12:40268) 2024-09-11T00:09:33.895 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:33.893+0000 7f9de3e00640 1 -- 172.21.15.12:0/1003685176 learned_addr learned my addr 172.21.15.12:0/1003685176 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:09:33.895 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:33.893+0000 7f9de9200640 1 --2- 172.21.15.12:0/1003685176 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556dd72128a0 0x556dd720cf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:09:33.896 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:33.894+0000 7f9de3400640 1 --2- 172.21.15.12:0/1003685176 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556dd7206910 0x556dd720c9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:09:33.896 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:33.894+0000 7f9de9200640 1 -- 172.21.15.12:0/1003685176 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556dd7206910 msgr2=0x556dd720c9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:09:33.896 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:33.894+0000 7f9de9200640 1 --2- 172.21.15.12:0/1003685176 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556dd7206910 0x556dd720c9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:09:33.896 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:33.894+0000 7f9de9200640 1 -- 172.21.15.12:0/1003685176 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556dd7205ff0 msgr2=0x556dd7211520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:09:33.896 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:33.894+0000 7f9de9200640 1 --2- 172.21.15.12:0/1003685176 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556dd7205ff0 0x556dd7211520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:09:33.896 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:33.894+0000 7f9de9200640 1 -- 172.21.15.12:0/1003685176 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556dd72048a0 con 0x556dd72128a0 2024-09-11T00:09:33.896 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:33.894+0000 7f9de3e00640 1 --2- 172.21.15.12:0/1003685176 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556dd7205ff0 0x556dd7211520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:09:33.898 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:33.894+0000 7f9de9200640 1 --2- 172.21.15.12:0/1003685176 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556dd72128a0 0x556dd720cf30 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f9ddc011e50 tx=0x7f9ddc021000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=132605217d9df57a server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:09:33.898 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:33.895+0000 7f9dd8c00640 1 -- 172.21.15.12:0/1003685176 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ddc00f600 con 0x556dd72128a0 2024-09-11T00:09:33.898 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:33.895+0000 7f9dd8c00640 1 -- 172.21.15.12:0/1003685176 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9ddc035070 con 0x556dd72128a0 2024-09-11T00:09:33.898 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:33.895+0000 7f9dd8c00640 1 -- 172.21.15.12:0/1003685176 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ddc030a10 con 0x556dd72128a0 2024-09-11T00:09:33.898 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:33.895+0000 7f9deddff740 1 -- 172.21.15.12:0/1003685176 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556dd72128a0 msgr2=0x556dd720cf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:09:33.898 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:33.895+0000 7f9deddff740 1 --2- 172.21.15.12:0/1003685176 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556dd72128a0 0x556dd720cf30 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f9ddc011e50 tx=0x7f9ddc021000 comp rx=0 tx=0).stop 2024-09-11T00:09:33.898 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:33.895+0000 7f9deddff740 1 -- 172.21.15.12:0/1003685176 shutdown_connections 2024-09-11T00:09:33.899 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:33.895+0000 7f9deddff740 1 --2- 172.21.15.12:0/1003685176 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556dd72128a0 0x556dd720cf30 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:09:33.899 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:33.895+0000 7f9deddff740 1 --2- 172.21.15.12:0/1003685176 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556dd7206910 0x556dd720c9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:09:33.899 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:33.895+0000 7f9deddff740 1 --2- 172.21.15.12:0/1003685176 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556dd7205ff0 0x556dd7211520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:09:33.899 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:33.895+0000 7f9deddff740 1 -- 172.21.15.12:0/1003685176 >> 172.21.15.12:0/1003685176 conn(0x556dd7203e20 msgr2=0x556dd7210e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:09:33.899 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:33.895+0000 7f9deddff740 1 -- 172.21.15.12:0/1003685176 shutdown_connections 2024-09-11T00:09:33.899 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:33.895+0000 7f9deddff740 1 -- 172.21.15.12:0/1003685176 wait complete. 2024-09-11T00:09:33.899 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:33.896+0000 7f9deddff740 1 -- start start 2024-09-11T00:09:33.899 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:33.896+0000 7f9deddff740 1 Processor -- start 2024-09-11T00:09:33.899 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:33.896+0000 7f9deddff740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556dd7205ff0 0x556dd721af10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:09:33.899 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:33.896+0000 7f9deddff740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556dd7206910 0x556dd72173f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:09:33.899 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:33.896+0000 7f9deddff740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556dd72128a0 0x556dd7217930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:09:33.899 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:33.896+0000 7f9deddff740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f9de41b3260 con 0x556dd72128a0 2024-09-11T00:09:33.899 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:33.896+0000 7f9deddff740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f9de41b2ad0 con 0x556dd7205ff0 2024-09-11T00:09:33.900 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:33.896+0000 7f9deddff740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f9de41afd10 con 0x556dd7206910 2024-09-11T00:09:33.900 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:33.896+0000 7f9deddff740 10 client.0 _ll_register_callbacks cb 0x7f9dcc009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:09:33.900 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:33.897+0000 7f9de3e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556dd7205ff0 0x556dd721af10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:09:33.900 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:33.897+0000 7f9de3e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556dd7205ff0 0x556dd721af10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.12:33198/0 (socket says 172.21.15.12:33198) 2024-09-11T00:09:33.900 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:33.897+0000 7f9de3e00640 1 -- 172.21.15.12:0/33999013 learned_addr learned my addr 172.21.15.12:0/33999013 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:09:33.900 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:33.897+0000 7f9de3400640 1 --2- 172.21.15.12:0/33999013 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556dd7206910 0x556dd72173f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:09:33.900 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:33.897+0000 7f9de9200640 1 --2- 172.21.15.12:0/33999013 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556dd72128a0 0x556dd7217930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:09:33.900 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:33.897+0000 7f9de3e00640 1 -- 172.21.15.12:0/33999013 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556dd7206910 msgr2=0x556dd72173f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:09:33.900 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:33.897+0000 7f9de3e00640 1 --2- 172.21.15.12:0/33999013 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556dd7206910 0x556dd72173f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:09:33.900 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:33.897+0000 7f9de3e00640 1 -- 172.21.15.12:0/33999013 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556dd72128a0 msgr2=0x556dd7217930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:09:33.900 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:33.897+0000 7f9de3e00640 1 --2- 172.21.15.12:0/33999013 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556dd72128a0 0x556dd7217930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:09:33.900 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:33.897+0000 7f9de3e00640 1 -- 172.21.15.12:0/33999013 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556dd72180c0 con 0x556dd7205ff0 2024-09-11T00:09:33.902 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:33.897+0000 7f9de3e00640 1 --2- 172.21.15.12:0/33999013 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556dd7205ff0 0x556dd721af10 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f9dd400c7d0 tx=0x7f9dd400cf10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f9c06d9fa9bc6292 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:09:33.902 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:33.897+0000 7f9dd2000640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-09-11T00:09:33.902 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:33.898+0000 7f9dd2000640 1 -- 172.21.15.12:0/33999013 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9dd40b0070 con 0x556dd7205ff0 2024-09-11T00:09:33.902 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:33.898+0000 7f9deddff740 1 -- 172.21.15.12:0/33999013 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9ddc035070 con 0x556dd7205ff0 2024-09-11T00:09:33.902 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:33.898+0000 7f9dd2000640 1 -- 172.21.15.12:0/33999013 <== mon.2 v2:172.21.15.165:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9208+0+0 (secure 0 0 0) 0x7f9dd40a9050 con 0x556dd7205ff0 2024-09-11T00:09:33.902 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:33.898+0000 7f9dd2000640 1 -- 172.21.15.12:0/33999013 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f9dd40981b0 con 0x556dd7205ff0 2024-09-11T00:09:33.902 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:33.898+0000 7f9deddff740 10 client.25838 fetch_fsmap learned FSMap version 33 2024-09-11T00:09:33.902 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:33.898+0000 7f9deddff740 1 -- 172.21.15.12:0/33999013 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x556dd7290330 con 0x556dd7205ff0 2024-09-11T00:09:33.902 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:33.898+0000 7f9dd2000640 1 -- 172.21.15.12:0/33999013 <== mon.2 v2:172.21.15.165:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9ddc035070 con 0x556dd7205ff0 2024-09-11T00:09:33.903 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:33.898+0000 7f9dd2000640 1 -- 172.21.15.12:0/33999013 <== mon.2 v2:172.21.15.165:3300/0 5 ==== fsmap(e 33) ==== 8386+0+0 (secure 0 0 0) 0x7f9dd40ac040 con 0x556dd7205ff0 2024-09-11T00:09:33.903 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:33.898+0000 7f9deddff740 10 client.25838 fetch_fsmap finished waiting for FSMap version 33 2024-09-11T00:09:33.903 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:33.898+0000 7f9deddff740 10 client.25838 resolve_mds: validated gid 24674 aka daemon mds.b 2024-09-11T00:09:33.903 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:33.898+0000 7f9deddff740 20 client.25838 populate_metadata read hostname 'smithi012' 2024-09-11T00:09:33.903 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:33.898+0000 7f9deddff740 1 --2- 172.21.15.12:0/33999013 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x556dd7290b20 0x556dd7292f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:09:33.903 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:33.899+0000 7f9deddff740 4 client.25838 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "68ee5ab7", "release": true, "await": true}] 2024-09-11T00:09:33.903 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:33.899+0000 7f9deddff740 1 -- 172.21.15.12:0/33999013 --> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] -- command(tid 0: {"prefix": "quiesce db", "set_id": "68ee5ab7", "release": true, "await": true}) -- 0x556dd717ac90 con 0x556dd7290b20 2024-09-11T00:09:33.903 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:33.899+0000 7f9de3400640 1 --2- 172.21.15.12:0/33999013 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x556dd7290b20 0x556dd7292f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:09:33.903 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:33.899+0000 7f9de3400640 1 --2- 172.21.15.12:0/33999013 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x556dd7290b20 0x556dd7292f40 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b096a7fd2385c875 server_cookie=a6cf0a2baa0feb5e in_seq=0 out_seq=0 2024-09-11T00:09:33.903 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:33.899+0000 7f9dd2000640 10 client.25838 ms_handle_connect on v2:172.21.15.165:6838/501312984 2024-09-11T00:09:33.911 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:33.909+0000 7f9dd2000640 1 -- 172.21.15.12:0/33999013 <== mds.0 v2:172.21.15.165:6838/501312984 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x556dd717ac90 con 0x556dd7290b20 2024-09-11T00:09:33.911 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:33.909+0000 7f9dd2000640 10 client.25838 handle_command_reply: tid=0 2024-09-11T00:09:33.911 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:33.909+0000 7f9deddff740 1 client.25838 shutdown 2024-09-11T00:09:33.911 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:33.909+0000 7f9deddff740 1 -- 172.21.15.12:0/33999013 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556dd7205ff0 msgr2=0x556dd721af10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:09:33.911 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:33.909+0000 7f9deddff740 1 --2- 172.21.15.12:0/33999013 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556dd7205ff0 0x556dd721af10 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f9dd400c7d0 tx=0x7f9dd400cf10 comp rx=0 tx=0).stop 2024-09-11T00:09:33.911 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:33.909+0000 7f9deddff740 1 -- 172.21.15.12:0/33999013 shutdown_connections 2024-09-11T00:09:33.911 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:33.909+0000 7f9deddff740 1 --2- 172.21.15.12:0/33999013 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x556dd7290b20 0x556dd7292f40 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:09:33.911 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:33.909+0000 7f9deddff740 1 --2- 172.21.15.12:0/33999013 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556dd72128a0 0x556dd7217930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:09:33.911 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:33.909+0000 7f9deddff740 1 --2- 172.21.15.12:0/33999013 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556dd7206910 0x556dd72173f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:09:33.911 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:33.909+0000 7f9deddff740 1 --2- 172.21.15.12:0/33999013 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556dd7205ff0 0x556dd721af10 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:09:33.911 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:33.909+0000 7f9dd2000640 0 client.25838 ms_handle_reset on v2:172.21.15.165:6838/501312984 2024-09-11T00:09:33.912 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:33.909+0000 7f9deddff740 1 -- 172.21.15.12:0/33999013 >> 172.21.15.12:0/33999013 conn(0x556dd7203e20 msgr2=0x556dd720e880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:09:33.912 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:33.909+0000 7f9deddff740 1 -- 172.21.15.12:0/33999013 shutdown_connections 2024-09-11T00:09:33.912 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:33.909+0000 7f9deddff740 1 -- 172.21.15.12:0/33999013 wait complete. 2024-09-11T00:09:33.912 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:33.910+0000 7f9deddff740 20 client.25838 trim_cache size 0 max 16384 2024-09-11T00:09:33.912 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-09-11T00:09:33.912 INFO:teuthology.orchestra.run.smithi012.stdout: "epoch": 33, 2024-09-11T00:09:33.912 INFO:teuthology.orchestra.run.smithi012.stdout: "leader": 24674, 2024-09-11T00:09:33.913 INFO:teuthology.orchestra.run.smithi012.stdout: "set_version": 12, 2024-09-11T00:09:33.913 INFO:teuthology.orchestra.run.smithi012.stdout: "sets": { 2024-09-11T00:09:33.913 INFO:teuthology.orchestra.run.smithi012.stdout: "68ee5ab7": { 2024-09-11T00:09:33.913 INFO:teuthology.orchestra.run.smithi012.stdout: "version": 12, 2024-09-11T00:09:33.913 INFO:teuthology.orchestra.run.smithi012.stdout: "age_ref": 0.0, 2024-09-11T00:09:33.913 INFO:teuthology.orchestra.run.smithi012.stdout: "state": { 2024-09-11T00:09:33.913 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "RELEASED", 2024-09-11T00:09:33.913 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 0.0 2024-09-11T00:09:33.913 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:09:33.913 INFO:teuthology.orchestra.run.smithi012.stdout: "timeout": 90.0, 2024-09-11T00:09:33.913 INFO:teuthology.orchestra.run.smithi012.stdout: "expiration": 140.5, 2024-09-11T00:09:33.913 INFO:teuthology.orchestra.run.smithi012.stdout: "members": { 2024-09-11T00:09:33.913 INFO:teuthology.orchestra.run.smithi012.stdout: "file:/": { 2024-09-11T00:09:33.914 INFO:teuthology.orchestra.run.smithi012.stdout: "excluded": false, 2024-09-11T00:09:33.914 INFO:teuthology.orchestra.run.smithi012.stdout: "state": { 2024-09-11T00:09:33.914 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "RELEASED", 2024-09-11T00:09:33.914 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 0.0 2024-09-11T00:09:33.914 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:09:33.914 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:09:33.914 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:09:33.914 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:09:33.914 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:09:33.914 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-09-11T00:09:33.915 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:33.910+0000 7f9deb200640 1 -- 172.21.15.12:0/1108356628 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f9dc80b52e0 msgr2=0x7f9dc80b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:09:33.915 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:33.910+0000 7f9deb200640 1 --2- 172.21.15.12:0/1108356628 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f9dc80b52e0 0x7f9dc80b77a0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f9ddc009640 tx=0x7f9ddc0097a0 comp rx=0 tx=0).stop 2024-09-11T00:09:33.915 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:33.910+0000 7f9deb200640 1 -- 172.21.15.12:0/1108356628 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9de4104fa0 msgr2=0x7f9de419b470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:09:33.915 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:33.910+0000 7f9deb200640 1 --2- 172.21.15.12:0/1108356628 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9de4104fa0 0x7f9de419b470 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f9dd400eb00 tx=0x7f9dd400c2d0 comp rx=0 tx=0).stop 2024-09-11T00:09:33.915 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:33.910+0000 7f9deb200640 1 -- 172.21.15.12:0/1108356628 shutdown_connections 2024-09-11T00:09:33.915 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:33.910+0000 7f9deb200640 1 --2- 172.21.15.12:0/1108356628 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f9dc80b52e0 0x7f9dc80b77a0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:09:33.915 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:33.910+0000 7f9deb200640 1 --2- 172.21.15.12:0/1108356628 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9de410b330 0x7f9de41a0d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:09:33.915 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:33.910+0000 7f9deb200640 1 --2- 172.21.15.12:0/1108356628 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9de410a9b0 0x7f9de419b9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:09:33.915 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:33.911+0000 7f9deb200640 1 --2- 172.21.15.12:0/1108356628 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9de4104fa0 0x7f9de419b470 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:09:33.915 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:33.911+0000 7f9deb200640 1 -- 172.21.15.12:0/1108356628 >> 172.21.15.12:0/1108356628 conn(0x7f9de4100a50 msgr2=0x7f9de4102e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:09:33.916 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:33.911+0000 7f9deb200640 1 -- 172.21.15.12:0/1108356628 shutdown_connections 2024-09-11T00:09:33.916 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:09:33.911+0000 7f9deb200640 1 -- 172.21.15.12:0/1108356628 wait complete. 2024-09-11T00:09:34.608 INFO:tasks.workunit.client.0.smithi012.stdout: 1 91116 5.56 MB/sec warmup 91 sec latency 20621.595 ms 2024-09-11T00:09:34.711 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '68ee5ab7', total seconds elapsed: 24.6 2024-09-11T00:09:35.563 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:09:35 smithi012 ceph-mon[30333]: pgmap v474: 129 pgs: 129 active+clean; 27 MiB data, 518 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 141 KiB/s wr, 18 op/s 2024-09-11T00:09:35.580 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:09:35 smithi178 ceph-mon[35919]: pgmap v474: 129 pgs: 129 active+clean; 27 MiB data, 518 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 141 KiB/s wr, 18 op/s 2024-09-11T00:09:35.606 INFO:tasks.workunit.client.0.smithi012.stdout: 1 92733 5.63 MB/sec warmup 92 sec latency 9.129 ms 2024-09-11T00:09:35.782 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:09:35 smithi165 ceph-mon[35356]: pgmap v474: 129 pgs: 129 active+clean; 27 MiB data, 518 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 141 KiB/s wr, 18 op/s 2024-09-11T00:09:36.607 INFO:tasks.workunit.client.0.smithi012.stdout: 1 92936 5.57 MB/sec warmup 93 sec latency 853.462 ms 2024-09-11T00:09:37.563 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:09:37 smithi012 ceph-mon[30333]: pgmap v475: 129 pgs: 129 active+clean; 28 MiB data, 508 MiB used, 1.0 TiB / 1.0 TiB avail; 450 KiB/s rd, 409 KiB/s wr, 50 op/s 2024-09-11T00:09:37.580 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:09:37 smithi178 ceph-mon[35919]: pgmap v475: 129 pgs: 129 active+clean; 28 MiB data, 508 MiB used, 1.0 TiB / 1.0 TiB avail; 450 KiB/s rd, 409 KiB/s wr, 50 op/s 2024-09-11T00:09:37.584 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:09:37 smithi165 ceph-mon[35356]: pgmap v475: 129 pgs: 129 active+clean; 28 MiB data, 508 MiB used, 1.0 TiB / 1.0 TiB avail; 450 KiB/s rd, 409 KiB/s wr, 50 op/s 2024-09-11T00:09:37.607 INFO:tasks.workunit.client.0.smithi012.stdout: 1 92936 5.51 MB/sec warmup 94 sec latency 1853.527 ms 2024-09-11T00:09:37.668 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:37.667+0000 7effd5400640 1 -- 172.21.15.12:0/1431159589 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7effd01062a0 msgr2=0x7effd0110f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:09:37.668 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:37.667+0000 7effd5400640 1 --2- 172.21.15.12:0/1431159589 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7effd01062a0 0x7effd0110f80 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7effc8009ca0 tx=0x7effc802a8c0 comp rx=0 tx=0).stop 2024-09-11T00:09:37.669 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:37.667+0000 7effd5400640 1 -- 172.21.15.12:0/1431159589 shutdown_connections 2024-09-11T00:09:37.669 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:37.667+0000 7effd5400640 1 --2- 172.21.15.12:0/1431159589 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7effd01062a0 0x7effd0110f80 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:09:37.669 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:37.667+0000 7effd5400640 1 --2- 172.21.15.12:0/1431159589 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7effd0075b20 0x7effd0075fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:09:37.669 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:37.667+0000 7effd5400640 1 --2- 172.21.15.12:0/1431159589 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7effd0076f90 0x7effd00755e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:09:37.669 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:37.667+0000 7effd5400640 1 -- 172.21.15.12:0/1431159589 >> 172.21.15.12:0/1431159589 conn(0x7effd00fe030 msgr2=0x7effd0100470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:09:37.669 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:37.667+0000 7effd5400640 1 -- 172.21.15.12:0/1431159589 shutdown_connections 2024-09-11T00:09:37.669 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:37.667+0000 7effd5400640 1 -- 172.21.15.12:0/1431159589 wait complete. 2024-09-11T00:09:37.671 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:37.668+0000 7effd5400640 1 Processor -- start 2024-09-11T00:09:37.671 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:37.668+0000 7effd5400640 1 -- start start 2024-09-11T00:09:37.671 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:37.668+0000 7effd5400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7effd0075b20 0x7effd019f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:09:37.671 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:37.668+0000 7effd5400640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7effd0076f90 0x7effd019f720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:09:37.671 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:37.668+0000 7effd5400640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7effd01a0350 0x7effd01a0800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:09:37.671 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:37.668+0000 7effd5400640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7effd0114a40 con 0x7effd01a0350 2024-09-11T00:09:37.672 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:37.668+0000 7effd5400640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7effd01148c0 con 0x7effd0075b20 2024-09-11T00:09:37.672 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:37.668+0000 7effd5400640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7effd0114bc0 con 0x7effd0076f90 2024-09-11T00:09:37.672 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:37.669+0000 7effd4a00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7effd01a0350 0x7effd01a0800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:09:37.672 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:37.669+0000 7effd4a00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7effd01a0350 0x7effd01a0800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:40288/0 (socket says 172.21.15.12:40288) 2024-09-11T00:09:37.672 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:37.669+0000 7effd4a00640 1 -- 172.21.15.12:0/2248840129 learned_addr learned my addr 172.21.15.12:0/2248840129 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:09:37.672 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:37.669+0000 7effc7e00640 1 --2- 172.21.15.12:0/2248840129 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7effd0076f90 0x7effd019f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:09:37.672 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:37.669+0000 7effcfe00640 1 --2- 172.21.15.12:0/2248840129 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7effd0075b20 0x7effd019f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:09:37.672 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:37.669+0000 7effd4a00640 1 -- 172.21.15.12:0/2248840129 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7effd0076f90 msgr2=0x7effd019f720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:09:37.672 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:37.669+0000 7effd4a00640 1 --2- 172.21.15.12:0/2248840129 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7effd0076f90 0x7effd019f720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:09:37.672 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:37.669+0000 7effd4a00640 1 -- 172.21.15.12:0/2248840129 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7effd0075b20 msgr2=0x7effd019f0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:09:37.672 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:37.669+0000 7effd4a00640 1 --2- 172.21.15.12:0/2248840129 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7effd0075b20 0x7effd019f0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:09:37.672 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:37.669+0000 7effd4a00640 1 -- 172.21.15.12:0/2248840129 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7effd01a1000 con 0x7effd01a0350 2024-09-11T00:09:37.673 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:37.669+0000 7effd4a00640 1 --2- 172.21.15.12:0/2248840129 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7effd01a0350 0x7effd01a0800 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7effc8033040 tx=0x7effc80317c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:09:37.673 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:37.670+0000 7effcd600640 1 -- 172.21.15.12:0/2248840129 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effc8003b70 con 0x7effd01a0350 2024-09-11T00:09:37.673 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:37.670+0000 7effcd600640 1 -- 172.21.15.12:0/2248840129 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7effc8003d10 con 0x7effd01a0350 2024-09-11T00:09:37.673 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:37.670+0000 7effd5400640 1 -- 172.21.15.12:0/2248840129 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7effd01a73f0 con 0x7effd01a0350 2024-09-11T00:09:37.673 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:37.670+0000 7effcd600640 1 -- 172.21.15.12:0/2248840129 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effc802fbd0 con 0x7effd01a0350 2024-09-11T00:09:37.673 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:37.670+0000 7effd5400640 1 -- 172.21.15.12:0/2248840129 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7effd01a78e0 con 0x7effd01a0350 2024-09-11T00:09:37.674 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:37.673+0000 7effd9a14740 1 Processor -- start 2024-09-11T00:09:37.676 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:37.674+0000 7effcd600640 1 -- 172.21.15.12:0/2248840129 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7effc8004040 con 0x7effd01a0350 2024-09-11T00:09:37.676 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:37.674+0000 7effd9a14740 1 -- start start 2024-09-11T00:09:37.676 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:37.675+0000 7effd9a14740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560dd946b970 0x560dd946dd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:09:37.676 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:37.675+0000 7effd9a14740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560dd946e2a0 0x560dd9470690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:09:37.676 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:37.675+0000 7effd9a14740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560dd9470ce0 0x560dd94731a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:09:37.676 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:37.675+0000 7effd9a14740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x560dd93de6b0 con 0x560dd946e2a0 2024-09-11T00:09:37.676 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:37.675+0000 7effd9a14740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x560dd93cdae0 con 0x560dd946b970 2024-09-11T00:09:37.676 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:37.675+0000 7effd9a14740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x560dd93d18f0 con 0x560dd9470ce0 2024-09-11T00:09:37.676 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:37.675+0000 7effcfe00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560dd946e2a0 0x560dd9470690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:09:37.677 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:37.675+0000 7effc7e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560dd946b970 0x560dd946dd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:09:37.677 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:37.675+0000 7effcfe00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560dd946e2a0 0x560dd9470690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:40300/0 (socket says 172.21.15.12:40300) 2024-09-11T00:09:37.677 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:37.675+0000 7effc7e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560dd946b970 0x560dd946dd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.12:33226/0 (socket says 172.21.15.12:33226) 2024-09-11T00:09:37.677 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:37.675+0000 7effcfe00640 1 -- 172.21.15.12:0/2580970710 learned_addr learned my addr 172.21.15.12:0/2580970710 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:09:37.677 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:37.675+0000 7effc7e00640 1 --2- 172.21.15.12:0/2580970710 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560dd9470ce0 0x560dd94731a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:09:37.679 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:37.675+0000 7effc7e00640 1 -- 172.21.15.12:0/2580970710 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560dd9470ce0 msgr2=0x560dd94731a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:09:37.679 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:37.675+0000 7effc7e00640 1 --2- 172.21.15.12:0/2580970710 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560dd9470ce0 0x560dd94731a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:09:37.679 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:37.675+0000 7effc7e00640 1 -- 172.21.15.12:0/2580970710 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560dd946e2a0 msgr2=0x560dd9470690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:09:37.679 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:37.675+0000 7effc7e00640 1 --2- 172.21.15.12:0/2580970710 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560dd946e2a0 0x560dd9470690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:09:37.679 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:37.675+0000 7effc7e00640 1 -- 172.21.15.12:0/2580970710 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560dd939a780 con 0x560dd946b970 2024-09-11T00:09:37.680 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:37.676+0000 7effcfe00640 1 --2- 172.21.15.12:0/2580970710 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560dd946e2a0 0x560dd9470690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:09:37.680 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:37.676+0000 7effc7e00640 1 --2- 172.21.15.12:0/2580970710 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560dd946b970 0x560dd946dd60 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7effb800b6c0 tx=0x7effb800b650 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c2ce046f3bd56d65 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:09:37.680 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:37.676+0000 7effc5200640 1 -- 172.21.15.12:0/2580970710 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effb8012d40 con 0x560dd946b970 2024-09-11T00:09:37.680 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:37.676+0000 7effc5200640 1 -- 172.21.15.12:0/2580970710 <== mon.2 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7effb8005020 con 0x560dd946b970 2024-09-11T00:09:37.680 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:37.676+0000 7effc5200640 1 -- 172.21.15.12:0/2580970710 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effb80049a0 con 0x560dd946b970 2024-09-11T00:09:37.680 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:37.676+0000 7effcd600640 1 --2- 172.21.15.12:0/2248840129 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7effb40b5360 0x7effb40b7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:09:37.680 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:37.677+0000 7effcd600640 1 -- 172.21.15.12:0/2248840129 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9208+0+0 (secure 0 0 0) 0x7effc8107280 con 0x7effd01a0350 2024-09-11T00:09:37.680 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:37.677+0000 7effcfe00640 1 --2- 172.21.15.12:0/2248840129 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7effb40b5360 0x7effb40b7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:09:37.680 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:37.677+0000 7effd9a14740 1 -- 172.21.15.12:0/2580970710 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560dd946b970 msgr2=0x560dd946dd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:09:37.680 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:37.677+0000 7effd9a14740 1 --2- 172.21.15.12:0/2580970710 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560dd946b970 0x560dd946dd60 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7effb800b6c0 tx=0x7effb800b650 comp rx=0 tx=0).stop 2024-09-11T00:09:37.681 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:37.677+0000 7effd9a14740 1 -- 172.21.15.12:0/2580970710 shutdown_connections 2024-09-11T00:09:37.681 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:37.677+0000 7effd9a14740 1 --2- 172.21.15.12:0/2580970710 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560dd9470ce0 0x560dd94731a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:09:37.681 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:37.677+0000 7effd9a14740 1 --2- 172.21.15.12:0/2580970710 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560dd946e2a0 0x560dd9470690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:09:37.681 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:37.677+0000 7effd9a14740 1 --2- 172.21.15.12:0/2580970710 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560dd946b970 0x560dd946dd60 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:09:37.681 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:37.677+0000 7effd9a14740 1 -- 172.21.15.12:0/2580970710 >> 172.21.15.12:0/2580970710 conn(0x560dd9466ac0 msgr2=0x560dd9467850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:09:37.681 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:37.677+0000 7effcfe00640 1 --2- 172.21.15.12:0/2248840129 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7effb40b5360 0x7effb40b7820 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7effc0005bc0 tx=0x7effc0005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:09:37.681 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:37.677+0000 7effd9a14740 1 -- 172.21.15.12:0/2580970710 shutdown_connections 2024-09-11T00:09:37.681 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:37.678+0000 7effd9a14740 1 -- 172.21.15.12:0/2580970710 wait complete. 2024-09-11T00:09:37.681 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:37.678+0000 7effd9a14740 1 -- start start 2024-09-11T00:09:37.681 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:37.678+0000 7effd9a14740 1 Processor -- start 2024-09-11T00:09:37.685 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:37.679+0000 7effd9a14740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560dd946b970 0x560dd9475160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:09:37.685 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:37.679+0000 7effd9a14740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560dd946e2a0 0x560dd9475800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:09:37.685 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:37.679+0000 7effd9a14740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560dd9470ce0 0x560dd94763d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:09:37.685 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:37.679+0000 7effd9a14740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x560dd9305ab0 con 0x560dd946e2a0 2024-09-11T00:09:37.685 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:37.679+0000 7effd9a14740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x560dd9374570 con 0x560dd9470ce0 2024-09-11T00:09:37.685 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:37.679+0000 7effd9a14740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x560dd9473cc0 con 0x560dd946b970 2024-09-11T00:09:37.685 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:37.679+0000 7effd9a14740 10 client.0 _ll_register_callbacks cb 0x560dd9383100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:09:37.685 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:37.679+0000 7effc7e00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560dd946e2a0 0x560dd9475800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:09:37.685 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:37.679+0000 7effc7e00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560dd946e2a0 0x560dd9475800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:40314/0 (socket says 172.21.15.12:40314) 2024-09-11T00:09:37.685 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:37.679+0000 7effc7e00640 1 -- 172.21.15.12:0/1227101874 learned_addr learned my addr 172.21.15.12:0/1227101874 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:09:37.686 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:37.679+0000 7effd4a00640 1 --2- 172.21.15.12:0/1227101874 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560dd9470ce0 0x560dd94763d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:09:37.686 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:37.679+0000 7effcfe00640 1 --2- 172.21.15.12:0/1227101874 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560dd946b970 0x560dd9475160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:09:37.686 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:37.679+0000 7effc7e00640 1 -- 172.21.15.12:0/1227101874 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560dd946b970 msgr2=0x560dd9475160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:09:37.686 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:37.679+0000 7effc7e00640 1 --2- 172.21.15.12:0/1227101874 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560dd946b970 0x560dd9475160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:09:37.686 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:37.679+0000 7effc7e00640 1 -- 172.21.15.12:0/1227101874 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560dd9470ce0 msgr2=0x560dd94763d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:09:37.686 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:37.679+0000 7effc7e00640 1 --2- 172.21.15.12:0/1227101874 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560dd9470ce0 0x560dd94763d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:09:37.686 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:37.679+0000 7effc7e00640 1 -- 172.21.15.12:0/1227101874 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560dd912f790 con 0x560dd946e2a0 2024-09-11T00:09:37.686 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:37.679+0000 7effd4a00640 1 --2- 172.21.15.12:0/1227101874 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560dd9470ce0 0x560dd94763d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:09:37.686 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:37.680+0000 7effc7e00640 1 --2- 172.21.15.12:0/1227101874 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560dd946e2a0 0x560dd9475800 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7effb8015040 tx=0x7effb80042d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=161dc2389efd662b server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:09:37.686 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:37.680+0000 7effbd600640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-09-11T00:09:37.686 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:37.680+0000 7effbd600640 1 -- 172.21.15.12:0/1227101874 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effb8012d40 con 0x560dd946e2a0 2024-09-11T00:09:37.686 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:37.680+0000 7effd9a14740 1 -- 172.21.15.12:0/1227101874 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7effb8005020 con 0x560dd946e2a0 2024-09-11T00:09:37.687 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:37.680+0000 7effbd600640 1 -- 172.21.15.12:0/1227101874 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9208+0+0 (secure 0 0 0) 0x7effb8010740 con 0x560dd946e2a0 2024-09-11T00:09:37.687 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:37.680+0000 7effbd600640 1 -- 172.21.15.12:0/1227101874 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7effb8030050 con 0x560dd946e2a0 2024-09-11T00:09:37.687 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:37.680+0000 7effd9a14740 10 client.15822 fetch_fsmap learned FSMap version 33 2024-09-11T00:09:37.687 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:37.680+0000 7effd9a14740 1 -- 172.21.15.12:0/1227101874 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x560dd91a8620 con 0x560dd946e2a0 2024-09-11T00:09:37.687 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:37.681+0000 7effbd600640 1 -- 172.21.15.12:0/1227101874 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7effb8005020 con 0x560dd946e2a0 2024-09-11T00:09:37.687 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:37.681+0000 7effbd600640 1 -- 172.21.15.12:0/1227101874 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 33) ==== 8386+0+0 (secure 0 0 0) 0x7effb8004a60 con 0x560dd946e2a0 2024-09-11T00:09:37.687 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:37.681+0000 7effd9a14740 10 client.15822 fetch_fsmap finished waiting for FSMap version 33 2024-09-11T00:09:37.687 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:37.681+0000 7effd9a14740 10 client.15822 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T00:09:37.687 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:37.681+0000 7effd9a14740 20 client.15822 populate_metadata read hostname 'smithi012' 2024-09-11T00:09:37.687 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:37.681+0000 7effd9a14740 1 --2- 172.21.15.12:0/1227101874 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x560dd9480860 0x560dd9482c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:09:37.687 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:37.681+0000 7effd9a14740 4 client.15822 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T00:09:37.687 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:37.681+0000 7effd9a14740 1 -- 172.21.15.12:0/1227101874 --> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560dd93e0600 con 0x560dd9480860 2024-09-11T00:09:37.687 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:37.682+0000 7effcfe00640 1 --2- 172.21.15.12:0/1227101874 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x560dd9480860 0x560dd9482c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:09:37.687 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:37.682+0000 7effcfe00640 1 --2- 172.21.15.12:0/1227101874 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x560dd9480860 0x560dd9482c70 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=48a920e90cbc0f7b server_cookie=1656b128d1b7896b in_seq=0 out_seq=0 2024-09-11T00:09:37.688 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:37.682+0000 7effbd600640 10 client.15822 ms_handle_connect on v2:172.21.15.165:6838/501312984 2024-09-11T00:09:37.688 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:37.683+0000 7effbd600640 1 -- 172.21.15.12:0/1227101874 <== mds.0 v2:172.21.15.165:6838/501312984 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x560dd93e0600 con 0x560dd9480860 2024-09-11T00:09:37.688 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:37.683+0000 7effbd600640 10 client.15822 handle_command_reply: tid=0 2024-09-11T00:09:37.688 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:37.683+0000 7effd9a14740 1 client.15822 shutdown 2024-09-11T00:09:37.688 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:37.683+0000 7effd9a14740 1 -- 172.21.15.12:0/1227101874 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560dd946e2a0 msgr2=0x560dd9475800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:09:37.688 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:37.683+0000 7effd9a14740 1 --2- 172.21.15.12:0/1227101874 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560dd946e2a0 0x560dd9475800 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7effb8015040 tx=0x7effb80042d0 comp rx=0 tx=0).stop 2024-09-11T00:09:37.688 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:37.683+0000 7effd9a14740 1 -- 172.21.15.12:0/1227101874 shutdown_connections 2024-09-11T00:09:37.688 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:37.683+0000 7effd9a14740 1 --2- 172.21.15.12:0/1227101874 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x560dd9480860 0x560dd9482c70 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:09:37.688 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:37.683+0000 7effd9a14740 1 --2- 172.21.15.12:0/1227101874 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560dd9470ce0 0x560dd94763d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:09:37.688 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:37.683+0000 7effd9a14740 1 --2- 172.21.15.12:0/1227101874 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560dd946e2a0 0x560dd9475800 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:09:37.688 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:37.683+0000 7effbd600640 0 client.15822 ms_handle_reset on v2:172.21.15.165:6838/501312984 2024-09-11T00:09:37.688 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:37.683+0000 7effd9a14740 1 --2- 172.21.15.12:0/1227101874 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560dd946b970 0x560dd9475160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:09:37.689 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:37.683+0000 7effd9a14740 1 -- 172.21.15.12:0/1227101874 >> 172.21.15.12:0/1227101874 conn(0x560dd9466ac0 msgr2=0x560dd9465bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:09:37.689 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:37.684+0000 7effd9a14740 1 -- 172.21.15.12:0/1227101874 shutdown_connections 2024-09-11T00:09:37.689 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:37.684+0000 7effd9a14740 1 -- 172.21.15.12:0/1227101874 wait complete. 2024-09-11T00:09:37.689 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:37.684+0000 7effd9a14740 20 client.15822 trim_cache size 0 max 16384 2024-09-11T00:09:37.701 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:37.699+0000 7effd9a14740 1 Processor -- start 2024-09-11T00:09:37.701 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:37.699+0000 7effd9a14740 1 -- start start 2024-09-11T00:09:37.701 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:37.699+0000 7effd9a14740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560dd946b970 0x560dd94710c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:09:37.702 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:37.699+0000 7effd9a14740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560dd946e2a0 0x560dd9471600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:09:37.702 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:37.700+0000 7effd9a14740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560dd9470ce0 0x560dd94740c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:09:37.702 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:37.700+0000 7effd9a14740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7effd01b1a90 con 0x560dd946b970 2024-09-11T00:09:37.702 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:37.700+0000 7effd9a14740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7effd01b3690 con 0x560dd946e2a0 2024-09-11T00:09:37.702 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:37.700+0000 7effd9a14740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7effd01b2160 con 0x560dd9470ce0 2024-09-11T00:09:37.702 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:37.700+0000 7effcfe00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560dd946b970 0x560dd94710c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:09:37.702 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:37.700+0000 7effcfe00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560dd946b970 0x560dd94710c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:40316/0 (socket says 172.21.15.12:40316) 2024-09-11T00:09:37.702 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:37.700+0000 7effcfe00640 1 -- 172.21.15.12:0/2821462473 learned_addr learned my addr 172.21.15.12:0/2821462473 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:09:37.702 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:37.700+0000 7effd4a00640 1 --2- 172.21.15.12:0/2821462473 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560dd9470ce0 0x560dd94740c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:09:37.702 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:37.700+0000 7effc7e00640 1 --2- 172.21.15.12:0/2821462473 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560dd946e2a0 0x560dd9471600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:09:37.702 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:37.700+0000 7effcfe00640 1 -- 172.21.15.12:0/2821462473 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560dd9470ce0 msgr2=0x560dd94740c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:09:37.702 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:37.700+0000 7effcfe00640 1 --2- 172.21.15.12:0/2821462473 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560dd9470ce0 0x560dd94740c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:09:37.703 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:37.700+0000 7effcfe00640 1 -- 172.21.15.12:0/2821462473 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560dd946e2a0 msgr2=0x560dd9471600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:09:37.703 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:37.700+0000 7effcfe00640 1 --2- 172.21.15.12:0/2821462473 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560dd946e2a0 0x560dd9471600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:09:37.703 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:37.700+0000 7effcfe00640 1 -- 172.21.15.12:0/2821462473 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560dd94832f0 con 0x560dd946b970 2024-09-11T00:09:37.703 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:37.700+0000 7effcfe00640 1 --2- 172.21.15.12:0/2821462473 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560dd946b970 0x560dd94710c0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7effc0013010 tx=0x7effc0022040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a61f2cfd790b715e server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:09:37.706 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:37.701+0000 7effbfe00640 1 -- 172.21.15.12:0/2821462473 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effc0003ac0 con 0x560dd946b970 2024-09-11T00:09:37.706 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:37.701+0000 7effbfe00640 1 -- 172.21.15.12:0/2821462473 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7effc00054d0 con 0x560dd946b970 2024-09-11T00:09:37.706 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:37.701+0000 7effbfe00640 1 -- 172.21.15.12:0/2821462473 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effc0004d60 con 0x560dd946b970 2024-09-11T00:09:37.706 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:37.702+0000 7effd9a14740 1 -- 172.21.15.12:0/2821462473 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560dd946b970 msgr2=0x560dd94710c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:09:37.706 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:37.702+0000 7effd9a14740 1 --2- 172.21.15.12:0/2821462473 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560dd946b970 0x560dd94710c0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7effc0013010 tx=0x7effc0022040 comp rx=0 tx=0).stop 2024-09-11T00:09:37.706 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:37.702+0000 7effd9a14740 1 -- 172.21.15.12:0/2821462473 shutdown_connections 2024-09-11T00:09:37.706 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:37.702+0000 7effd9a14740 1 --2- 172.21.15.12:0/2821462473 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560dd9470ce0 0x560dd94740c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:09:37.706 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:37.702+0000 7effd9a14740 1 --2- 172.21.15.12:0/2821462473 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560dd946e2a0 0x560dd9471600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:09:37.706 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:37.702+0000 7effd9a14740 1 --2- 172.21.15.12:0/2821462473 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560dd946b970 0x560dd94710c0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:09:37.706 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:37.702+0000 7effd9a14740 1 -- 172.21.15.12:0/2821462473 >> 172.21.15.12:0/2821462473 conn(0x560dd9466ac0 msgr2=0x560dd9475160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:09:37.706 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:37.702+0000 7effd9a14740 1 -- 172.21.15.12:0/2821462473 shutdown_connections 2024-09-11T00:09:37.707 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:37.702+0000 7effd9a14740 1 -- 172.21.15.12:0/2821462473 wait complete. 2024-09-11T00:09:37.707 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:37.702+0000 7effd9a14740 1 -- start start 2024-09-11T00:09:37.707 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:37.702+0000 7effd9a14740 1 Processor -- start 2024-09-11T00:09:37.707 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:37.703+0000 7effd9a14740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560dd946b970 0x560dd947a1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:09:37.707 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:37.703+0000 7effd9a14740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560dd946e2a0 0x560dd947a850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:09:37.707 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:37.703+0000 7effd9a14740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560dd9470ce0 0x560dd947b380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:09:37.707 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:37.703+0000 7effd9a14740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7effd01b2fc0 con 0x560dd9470ce0 2024-09-11T00:09:37.707 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:37.703+0000 7effd9a14740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7effd01b2830 con 0x560dd946b970 2024-09-11T00:09:37.707 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:37.703+0000 7effd9a14740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7effd01afa70 con 0x560dd946e2a0 2024-09-11T00:09:37.707 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:37.703+0000 7effd9a14740 10 client.0 _ll_register_callbacks cb 0x7effc0009570 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:09:37.707 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:37.703+0000 7effcfe00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560dd946b970 0x560dd947a1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:09:37.707 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:37.703+0000 7effd4a00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560dd9470ce0 0x560dd947b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:09:37.707 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:37.703+0000 7effcfe00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560dd946b970 0x560dd947a1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.12:33256/0 (socket says 172.21.15.12:33256) 2024-09-11T00:09:37.708 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:37.703+0000 7effcfe00640 1 -- 172.21.15.12:0/1466995435 learned_addr learned my addr 172.21.15.12:0/1466995435 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:09:37.708 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:37.703+0000 7effc7e00640 1 --2- 172.21.15.12:0/1466995435 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560dd946e2a0 0x560dd947a850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:09:37.708 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:37.704+0000 7effc7e00640 1 -- 172.21.15.12:0/1466995435 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560dd946b970 msgr2=0x560dd947a1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:09:37.708 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:37.704+0000 7effc7e00640 1 --2- 172.21.15.12:0/1466995435 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560dd946b970 0x560dd947a1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:09:37.708 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:37.704+0000 7effc7e00640 1 -- 172.21.15.12:0/1466995435 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560dd9470ce0 msgr2=0x560dd947b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:09:37.708 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:37.704+0000 7effc7e00640 1 --2- 172.21.15.12:0/1466995435 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560dd9470ce0 0x560dd947b380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:09:37.708 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:37.704+0000 7effc7e00640 1 -- 172.21.15.12:0/1466995435 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560dd947bbb0 con 0x560dd946e2a0 2024-09-11T00:09:37.708 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:37.704+0000 7effd4a00640 1 --2- 172.21.15.12:0/1466995435 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560dd9470ce0 0x560dd947b380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:09:37.708 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:37.704+0000 7effcfe00640 1 --2- 172.21.15.12:0/1466995435 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560dd946b970 0x560dd947a1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:09:37.708 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:37.704+0000 7effc7e00640 1 --2- 172.21.15.12:0/1466995435 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560dd946e2a0 0x560dd947a850 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7effb8005d00 tx=0x7effb8012460 comp rx=0 tx=0).ready entity=mon.1 client_cookie=af5e75a7878579fe server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:09:37.708 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:37.704+0000 7effbd600640 10 client.0 ms_handle_connect on v2:172.21.15.178:3300/0 2024-09-11T00:09:37.708 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:37.704+0000 7effbd600640 1 -- 172.21.15.12:0/1466995435 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effb800bb00 con 0x560dd946e2a0 2024-09-11T00:09:37.708 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:37.704+0000 7effd9a14740 1 -- 172.21.15.12:0/1466995435 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7effc00054d0 con 0x560dd946e2a0 2024-09-11T00:09:37.709 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:37.705+0000 7effbd600640 1 -- 172.21.15.12:0/1466995435 <== mon.1 v2:172.21.15.178:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9208+0+0 (secure 0 0 0) 0x7effb80053e0 con 0x560dd946e2a0 2024-09-11T00:09:37.709 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:37.705+0000 7effbd600640 1 -- 172.21.15.12:0/1466995435 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7effb8005600 con 0x560dd946e2a0 2024-09-11T00:09:37.709 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:37.705+0000 7effd9a14740 10 client.25840 fetch_fsmap learned FSMap version 33 2024-09-11T00:09:37.709 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:37.705+0000 7effd9a14740 1 -- 172.21.15.12:0/1466995435 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x560dd947cf20 con 0x560dd946e2a0 2024-09-11T00:09:37.709 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:37.705+0000 7effbd600640 1 -- 172.21.15.12:0/1466995435 <== mon.1 v2:172.21.15.178:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7effc00054d0 con 0x560dd946e2a0 2024-09-11T00:09:37.710 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:37.705+0000 7effbd600640 1 -- 172.21.15.12:0/1466995435 <== mon.1 v2:172.21.15.178:3300/0 5 ==== fsmap(e 33) ==== 8386+0+0 (secure 0 0 0) 0x7effb8004860 con 0x560dd946e2a0 2024-09-11T00:09:37.710 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:37.705+0000 7effd9a14740 10 client.25840 fetch_fsmap finished waiting for FSMap version 33 2024-09-11T00:09:37.710 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:37.705+0000 7effd9a14740 10 client.25840 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T00:09:37.710 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:37.705+0000 7effd9a14740 20 client.25840 populate_metadata read hostname 'smithi012' 2024-09-11T00:09:37.710 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:37.705+0000 7effd9a14740 1 --2- 172.21.15.12:0/1466995435 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x560dd94edd00 0x560dd94f0120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:09:37.710 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:37.705+0000 7effd9a14740 4 client.25840 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T00:09:37.710 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:37.705+0000 7effd9a14740 1 -- 172.21.15.12:0/1466995435 --> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] -- command(tid 0: {"prefix": "flush journal"}) -- 0x560dd93dc0b0 con 0x560dd94edd00 2024-09-11T00:09:37.710 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:37.706+0000 7effcfe00640 1 --2- 172.21.15.12:0/1466995435 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x560dd94edd00 0x560dd94f0120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:09:37.710 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:37.706+0000 7effcfe00640 1 --2- 172.21.15.12:0/1466995435 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x560dd94edd00 0x560dd94f0120 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3117276a09dd584b server_cookie=b40fa1230bfc4301 in_seq=0 out_seq=0 2024-09-11T00:09:37.710 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:37.707+0000 7effbd600640 10 client.25840 ms_handle_connect on v2:172.21.15.165:6838/501312984 2024-09-11T00:09:37.737 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:37.736+0000 7effbd600640 1 -- 172.21.15.12:0/1466995435 <== mds.0 v2:172.21.15.165:6838/501312984 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x560dd93dc0b0 con 0x560dd94edd00 2024-09-11T00:09:37.738 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:37.736+0000 7effbd600640 10 client.25840 handle_command_reply: tid=0 2024-09-11T00:09:37.738 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:37.736+0000 7effd9a14740 1 client.25840 shutdown 2024-09-11T00:09:37.738 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:37.736+0000 7effd9a14740 1 -- 172.21.15.12:0/1466995435 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560dd946e2a0 msgr2=0x560dd947a850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:09:37.738 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:37.736+0000 7effd9a14740 1 --2- 172.21.15.12:0/1466995435 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560dd946e2a0 0x560dd947a850 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7effb8005d00 tx=0x7effb8012460 comp rx=0 tx=0).stop 2024-09-11T00:09:37.738 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:37.736+0000 7effd9a14740 1 -- 172.21.15.12:0/1466995435 shutdown_connections 2024-09-11T00:09:37.738 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:37.736+0000 7effd9a14740 1 --2- 172.21.15.12:0/1466995435 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x560dd94edd00 0x560dd94f0120 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:09:37.738 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:37.736+0000 7effd9a14740 1 --2- 172.21.15.12:0/1466995435 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560dd9470ce0 0x560dd947b380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:09:37.738 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:37.736+0000 7effd9a14740 1 --2- 172.21.15.12:0/1466995435 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560dd946e2a0 0x560dd947a850 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:09:37.738 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:37.736+0000 7effbd600640 0 client.25840 ms_handle_reset on v2:172.21.15.165:6838/501312984 2024-09-11T00:09:37.738 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:37.736+0000 7effd9a14740 1 --2- 172.21.15.12:0/1466995435 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560dd946b970 0x560dd947a1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:09:37.739 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:37.736+0000 7effd9a14740 1 -- 172.21.15.12:0/1466995435 >> 172.21.15.12:0/1466995435 conn(0x560dd9466ac0 msgr2=0x560dd94740c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:09:37.739 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:37.736+0000 7effd9a14740 1 -- 172.21.15.12:0/1466995435 shutdown_connections 2024-09-11T00:09:37.739 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:37.736+0000 7effd9a14740 1 -- 172.21.15.12:0/1466995435 wait complete. 2024-09-11T00:09:37.739 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:37.736+0000 7effd9a14740 20 client.25840 trim_cache size 0 max 16384 2024-09-11T00:09:37.739 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:{ 2024-09-11T00:09:37.739 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "message": "", 2024-09-11T00:09:37.739 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "return_code": 0 2024-09-11T00:09:37.739 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:} 2024-09-11T00:09:37.740 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:37.737+0000 7effd5400640 1 -- 172.21.15.12:0/2248840129 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7effb40b5360 msgr2=0x7effb40b7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:09:37.740 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:37.737+0000 7effd5400640 1 --2- 172.21.15.12:0/2248840129 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7effb40b5360 0x7effb40b7820 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7effc0005bc0 tx=0x7effc0005f90 comp rx=0 tx=0).stop 2024-09-11T00:09:37.740 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:37.737+0000 7effd5400640 1 -- 172.21.15.12:0/2248840129 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7effd01a0350 msgr2=0x7effd01a0800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:09:37.740 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:37.737+0000 7effd5400640 1 --2- 172.21.15.12:0/2248840129 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7effd01a0350 0x7effd01a0800 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7effc8033040 tx=0x7effc80317c0 comp rx=0 tx=0).stop 2024-09-11T00:09:37.740 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:37.737+0000 7effd5400640 1 -- 172.21.15.12:0/2248840129 shutdown_connections 2024-09-11T00:09:37.740 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:37.737+0000 7effd5400640 1 --2- 172.21.15.12:0/2248840129 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7effb40b5360 0x7effb40b7820 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:09:37.741 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:37.737+0000 7effd5400640 1 --2- 172.21.15.12:0/2248840129 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7effd01a0350 0x7effd01a0800 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:09:37.741 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:37.737+0000 7effd5400640 1 --2- 172.21.15.12:0/2248840129 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7effd0076f90 0x7effd019f720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:09:37.741 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:37.737+0000 7effd5400640 1 --2- 172.21.15.12:0/2248840129 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7effd0075b20 0x7effd019f0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:09:37.741 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:37.737+0000 7effd5400640 1 -- 172.21.15.12:0/2248840129 >> 172.21.15.12:0/2248840129 conn(0x7effd00fe030 msgr2=0x7effd00fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:09:37.741 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:37.737+0000 7effd5400640 1 -- 172.21.15.12:0/2248840129 shutdown_connections 2024-09-11T00:09:37.741 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:37.737+0000 7effd5400640 1 -- 172.21.15.12:0/2248840129 wait complete. 2024-09-11T00:09:38.607 INFO:tasks.workunit.client.0.smithi012.stdout: 1 92936 5.46 MB/sec warmup 95 sec latency 2853.601 ms 2024-09-11T00:09:39.563 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:09:39 smithi012 ceph-mon[30333]: pgmap v476: 129 pgs: 129 active+clean; 28 MiB data, 512 MiB used, 1.0 TiB / 1.0 TiB avail; 578 KiB/s rd, 412 KiB/s wr, 61 op/s 2024-09-11T00:09:39.581 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:09:39 smithi178 ceph-mon[35919]: pgmap v476: 129 pgs: 129 active+clean; 28 MiB data, 512 MiB used, 1.0 TiB / 1.0 TiB avail; 578 KiB/s rd, 412 KiB/s wr, 61 op/s 2024-09-11T00:09:39.607 INFO:tasks.workunit.client.0.smithi012.stdout: 1 92992 5.40 MB/sec warmup 96 sec latency 3783.256 ms 2024-09-11T00:09:39.782 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:09:39 smithi165 ceph-mon[35356]: pgmap v476: 129 pgs: 129 active+clean; 28 MiB data, 512 MiB used, 1.0 TiB / 1.0 TiB avail; 578 KiB/s rd, 412 KiB/s wr, 61 op/s 2024-09-11T00:09:40.607 INFO:tasks.workunit.client.0.smithi012.stdout: 1 92992 5.34 MB/sec warmup 97 sec latency 1026.675 ms 2024-09-11T00:09:41.563 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:09:41 smithi012 ceph-mon[30333]: pgmap v477: 129 pgs: 129 active+clean; 28 MiB data, 516 MiB used, 1.0 TiB / 1.0 TiB avail; 814 KiB/s rd, 441 KiB/s wr, 73 op/s 2024-09-11T00:09:41.581 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:09:41 smithi178 ceph-mon[35919]: pgmap v477: 129 pgs: 129 active+clean; 28 MiB data, 516 MiB used, 1.0 TiB / 1.0 TiB avail; 814 KiB/s rd, 441 KiB/s wr, 73 op/s 2024-09-11T00:09:41.607 INFO:tasks.workunit.client.0.smithi012.stdout: 1 92992 5.29 MB/sec warmup 98 sec latency 2026.834 ms 2024-09-11T00:09:41.782 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:09:41 smithi165 ceph-mon[35356]: pgmap v477: 129 pgs: 129 active+clean; 28 MiB data, 516 MiB used, 1.0 TiB / 1.0 TiB avail; 814 KiB/s rd, 441 KiB/s wr, 73 op/s 2024-09-11T00:09:42.608 INFO:tasks.workunit.client.0.smithi012.stdout: 1 92992 5.24 MB/sec warmup 99 sec latency 3027.012 ms 2024-09-11T00:09:43.563 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:09:43 smithi012 ceph-mon[30333]: pgmap v478: 129 pgs: 129 active+clean; 28 MiB data, 539 MiB used, 1.0 TiB / 1.0 TiB avail; 824 KiB/s rd, 454 KiB/s wr, 78 op/s 2024-09-11T00:09:43.580 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:09:43 smithi178 ceph-mon[35919]: pgmap v478: 129 pgs: 129 active+clean; 28 MiB data, 539 MiB used, 1.0 TiB / 1.0 TiB avail; 824 KiB/s rd, 454 KiB/s wr, 78 op/s 2024-09-11T00:09:43.608 INFO:tasks.workunit.client.0.smithi012.stdout: 1 92992 5.18 MB/sec warmup 100 sec latency 4027.144 ms 2024-09-11T00:09:43.782 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:09:43 smithi165 ceph-mon[35356]: pgmap v478: 129 pgs: 129 active+clean; 28 MiB data, 539 MiB used, 1.0 TiB / 1.0 TiB avail; 824 KiB/s rd, 454 KiB/s wr, 78 op/s 2024-09-11T00:09:44.608 INFO:tasks.workunit.client.0.smithi012.stdout: 1 93060 5.13 MB/sec warmup 101 sec latency 4955.157 ms 2024-09-11T00:09:45.563 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:09:45 smithi012 ceph-mon[30333]: pgmap v479: 129 pgs: 129 active+clean; 28 MiB data, 539 MiB used, 1.0 TiB / 1.0 TiB avail; 821 KiB/s rd, 435 KiB/s wr, 74 op/s 2024-09-11T00:09:45.580 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:09:45 smithi178 ceph-mon[35919]: pgmap v479: 129 pgs: 129 active+clean; 28 MiB data, 539 MiB used, 1.0 TiB / 1.0 TiB avail; 821 KiB/s rd, 435 KiB/s wr, 74 op/s 2024-09-11T00:09:45.607 INFO:tasks.workunit.client.0.smithi012.stdout: 1 93060 5.08 MB/sec warmup 102 sec latency 1009.254 ms 2024-09-11T00:09:45.782 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:09:45 smithi165 ceph-mon[35356]: pgmap v479: 129 pgs: 129 active+clean; 28 MiB data, 539 MiB used, 1.0 TiB / 1.0 TiB avail; 821 KiB/s rd, 435 KiB/s wr, 74 op/s 2024-09-11T00:09:46.608 INFO:tasks.workunit.client.0.smithi012.stdout: 1 93060 5.03 MB/sec warmup 103 sec latency 2009.396 ms 2024-09-11T00:09:47.563 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:09:47 smithi012 ceph-mon[30333]: pgmap v480: 129 pgs: 129 active+clean; 28 MiB data, 539 MiB used, 1.0 TiB / 1.0 TiB avail; 828 KiB/s rd, 396 KiB/s wr, 81 op/s 2024-09-11T00:09:47.580 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:09:47 smithi178 ceph-mon[35919]: pgmap v480: 129 pgs: 129 active+clean; 28 MiB data, 539 MiB used, 1.0 TiB / 1.0 TiB avail; 828 KiB/s rd, 396 KiB/s wr, 81 op/s 2024-09-11T00:09:47.608 INFO:tasks.workunit.client.0.smithi012.stdout: 1 93060 4.99 MB/sec warmup 104 sec latency 3009.513 ms 2024-09-11T00:09:47.782 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:09:47 smithi165 ceph-mon[35356]: pgmap v480: 129 pgs: 129 active+clean; 28 MiB data, 539 MiB used, 1.0 TiB / 1.0 TiB avail; 828 KiB/s rd, 396 KiB/s wr, 81 op/s 2024-09-11T00:09:48.608 INFO:tasks.workunit.client.0.smithi012.stdout: 1 93060 4.94 MB/sec warmup 105 sec latency 4009.619 ms 2024-09-11T00:09:49.563 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:09:49 smithi012 ceph-mon[30333]: pgmap v481: 129 pgs: 129 active+clean; 28 MiB data, 539 MiB used, 1.0 TiB / 1.0 TiB avail; 396 KiB/s rd, 131 KiB/s wr, 51 op/s 2024-09-11T00:09:49.581 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:09:49 smithi178 ceph-mon[35919]: pgmap v481: 129 pgs: 129 active+clean; 28 MiB data, 539 MiB used, 1.0 TiB / 1.0 TiB avail; 396 KiB/s rd, 131 KiB/s wr, 51 op/s 2024-09-11T00:09:49.608 INFO:tasks.workunit.client.0.smithi012.stdout: 1 93153 4.89 MB/sec warmup 106 sec latency 4937.114 ms 2024-09-11T00:09:49.782 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:09:49 smithi165 ceph-mon[35356]: pgmap v481: 129 pgs: 129 active+clean; 28 MiB data, 539 MiB used, 1.0 TiB / 1.0 TiB avail; 396 KiB/s rd, 131 KiB/s wr, 51 op/s 2024-09-11T00:09:50.608 INFO:tasks.workunit.client.0.smithi012.stdout: 1 94095 4.89 MB/sec warmup 107 sec latency 476.315 ms 2024-09-11T00:09:50.858 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:50.857+0000 7f4abec00640 1 -- 172.21.15.12:0/2139881518 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4ab8105cf0 msgr2=0x7f4ab8106170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:09:50.858 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:50.857+0000 7f4abec00640 1 --2- 172.21.15.12:0/2139881518 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4ab8105cf0 0x7f4ab8106170 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f4aa0009860 tx=0x7f4aa002a340 comp rx=0 tx=0).stop 2024-09-11T00:09:50.859 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:50.857+0000 7f4abec00640 1 -- 172.21.15.12:0/2139881518 shutdown_connections 2024-09-11T00:09:50.859 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:50.857+0000 7f4abec00640 1 --2- 172.21.15.12:0/2139881518 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4ab81066b0 0x7f4ab810d030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:09:50.859 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:50.857+0000 7f4abec00640 1 --2- 172.21.15.12:0/2139881518 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4ab8105cf0 0x7f4ab8106170 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:09:50.859 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:50.857+0000 7f4abec00640 1 --2- 172.21.15.12:0/2139881518 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4ab8104ba0 0x7f4ab8104fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:09:50.859 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:50.857+0000 7f4abec00640 1 -- 172.21.15.12:0/2139881518 >> 172.21.15.12:0/2139881518 conn(0x7f4ab8100460 msgr2=0x7f4ab8102880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:09:50.859 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:50.857+0000 7f4abec00640 1 -- 172.21.15.12:0/2139881518 shutdown_connections 2024-09-11T00:09:50.859 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:50.857+0000 7f4abec00640 1 -- 172.21.15.12:0/2139881518 wait complete. 2024-09-11T00:09:50.861 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:50.858+0000 7f4abec00640 1 Processor -- start 2024-09-11T00:09:50.861 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:50.858+0000 7f4abec00640 1 -- start start 2024-09-11T00:09:50.861 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:50.859+0000 7f4abec00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4ab8104ba0 0x7f4ab819af90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:09:50.861 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:50.859+0000 7f4abec00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4ab8105cf0 0x7f4ab819b5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:09:50.861 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:50.859+0000 7f4abec00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4ab81066b0 0x7f4ab819c1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:09:50.862 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:50.859+0000 7f4abec00640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f4ab8110810 con 0x7f4ab8105cf0 2024-09-11T00:09:50.862 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:50.859+0000 7f4abec00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f4ab8110690 con 0x7f4ab8104ba0 2024-09-11T00:09:50.862 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:50.859+0000 7f4abec00640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f4ab8110990 con 0x7f4ab81066b0 2024-09-11T00:09:50.862 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:50.859+0000 7f4aafe00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4ab8105cf0 0x7f4ab819b5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:09:50.862 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:50.859+0000 7f4ab7e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4ab8104ba0 0x7f4ab819af90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:09:50.862 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:50.859+0000 7f4abcc00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4ab81066b0 0x7f4ab819c1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:09:50.862 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:50.859+0000 7f4abcc00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4ab81066b0 0x7f4ab819c1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.12:52976/0 (socket says 172.21.15.12:52976) 2024-09-11T00:09:50.862 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:50.859+0000 7f4abcc00640 1 -- 172.21.15.12:0/1290145530 learned_addr learned my addr 172.21.15.12:0/1290145530 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:09:50.862 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:50.859+0000 7f4ab7e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4ab8104ba0 0x7f4ab819af90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.12:43694/0 (socket says 172.21.15.12:43694) 2024-09-11T00:09:50.862 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:50.859+0000 7f4ab7e00640 1 -- 172.21.15.12:0/1290145530 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4ab81066b0 msgr2=0x7f4ab819c1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:09:50.862 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:50.859+0000 7f4ab7e00640 1 --2- 172.21.15.12:0/1290145530 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4ab81066b0 0x7f4ab819c1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:09:50.862 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:50.859+0000 7f4ab7e00640 1 -- 172.21.15.12:0/1290145530 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4ab8105cf0 msgr2=0x7f4ab819b5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:09:50.863 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:50.859+0000 7f4ab7e00640 1 --2- 172.21.15.12:0/1290145530 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4ab8105cf0 0x7f4ab819b5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:09:50.863 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:50.859+0000 7f4ab7e00640 1 -- 172.21.15.12:0/1290145530 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ab819ca60 con 0x7f4ab8104ba0 2024-09-11T00:09:50.863 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:50.859+0000 7f4abcc00640 1 --2- 172.21.15.12:0/1290145530 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4ab81066b0 0x7f4ab819c1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:09:50.863 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:50.860+0000 7f4ab7e00640 1 --2- 172.21.15.12:0/1290145530 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4ab8104ba0 0x7f4ab819af90 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f4aa800e9f0 tx=0x7f4aa800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:09:50.863 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:50.860+0000 7f4ab5600640 1 -- 172.21.15.12:0/1290145530 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4aa800ccc0 con 0x7f4ab8104ba0 2024-09-11T00:09:50.863 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:50.860+0000 7f4ab5600640 1 -- 172.21.15.12:0/1290145530 <== mon.2 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4aa80108e0 con 0x7f4ab8104ba0 2024-09-11T00:09:50.863 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:50.860+0000 7f4ab5600640 1 -- 172.21.15.12:0/1290145530 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4aa8018880 con 0x7f4ab8104ba0 2024-09-11T00:09:50.863 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:50.860+0000 7f4abec00640 1 -- 172.21.15.12:0/1290145530 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4ab819cd50 con 0x7f4ab8104ba0 2024-09-11T00:09:50.863 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:50.860+0000 7f4abec00640 1 -- 172.21.15.12:0/1290145530 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4ab81a3420 con 0x7f4ab8104ba0 2024-09-11T00:09:50.864 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:50.863+0000 7f4ac1c62740 1 Processor -- start 2024-09-11T00:09:50.864 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:50.863+0000 7f4ac1c62740 1 -- start start 2024-09-11T00:09:50.865 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:50.864+0000 7f4ac1c62740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e0a5b33790 0x55e0a5b35b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:09:50.865 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:50.864+0000 7f4ac1c62740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e0a5b360c0 0x55e0a5b384b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:09:50.866 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:50.864+0000 7f4ac1c62740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e0a5b38b00 0x55e0a5b3afc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:09:50.866 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:50.864+0000 7f4ac1c62740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55e0a5aa66b0 con 0x55e0a5b360c0 2024-09-11T00:09:50.866 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:50.864+0000 7f4ac1c62740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55e0a5a95ae0 con 0x55e0a5b38b00 2024-09-11T00:09:50.866 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:50.864+0000 7f4ac1c62740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55e0a5a998f0 con 0x55e0a5b33790 2024-09-11T00:09:50.866 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:50.864+0000 7f4abcc00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e0a5b33790 0x55e0a5b35b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:09:50.866 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:50.864+0000 7f4abcc00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e0a5b33790 0x55e0a5b35b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.12:52982/0 (socket says 172.21.15.12:52982) 2024-09-11T00:09:50.866 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:50.864+0000 7f4abcc00640 1 -- 172.21.15.12:0/3018544629 learned_addr learned my addr 172.21.15.12:0/3018544629 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:09:50.866 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:50.864+0000 7f4aafe00640 1 --2- 172.21.15.12:0/3018544629 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e0a5b38b00 0x55e0a5b3afc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:09:50.866 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:50.864+0000 7f4ab5600640 1 -- 172.21.15.12:0/1290145530 <== mon.2 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7f4aa8010430 con 0x7f4ab8104ba0 2024-09-11T00:09:50.866 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:50.864+0000 7f4abcc00640 1 -- 172.21.15.12:0/3018544629 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e0a5b38b00 msgr2=0x55e0a5b3afc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:09:50.866 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:50.864+0000 7f4abcc00640 1 --2- 172.21.15.12:0/3018544629 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e0a5b38b00 0x55e0a5b3afc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:09:50.867 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:50.864+0000 7f4abcc00640 1 -- 172.21.15.12:0/3018544629 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e0a5b360c0 msgr2=0x55e0a5b384b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-11T00:09:50.867 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:50.864+0000 7f4abcc00640 1 --2- 172.21.15.12:0/3018544629 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e0a5b360c0 0x55e0a5b384b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:09:50.867 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:50.864+0000 7f4abcc00640 1 -- 172.21.15.12:0/3018544629 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e0a5a62780 con 0x55e0a5b33790 2024-09-11T00:09:50.867 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:50.865+0000 7f4abcc00640 1 --2- 172.21.15.12:0/3018544629 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e0a5b33790 0x55e0a5b35b80 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f4ab000eac0 tx=0x7f4ab000ef80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1ad03e59501a2f87 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:09:50.870 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:50.865+0000 7f4aad200640 1 -- 172.21.15.12:0/3018544629 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ab000cda0 con 0x55e0a5b33790 2024-09-11T00:09:50.870 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:50.865+0000 7f4aad200640 1 -- 172.21.15.12:0/3018544629 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4ab001d070 con 0x55e0a5b33790 2024-09-11T00:09:50.870 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:50.865+0000 7f4aad200640 1 -- 172.21.15.12:0/3018544629 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ab000b5c0 con 0x55e0a5b33790 2024-09-11T00:09:50.870 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:50.866+0000 7f4ac1c62740 1 -- 172.21.15.12:0/3018544629 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e0a5b33790 msgr2=0x55e0a5b35b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:09:50.870 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:50.866+0000 7f4ac1c62740 1 --2- 172.21.15.12:0/3018544629 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e0a5b33790 0x55e0a5b35b80 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f4ab000eac0 tx=0x7f4ab000ef80 comp rx=0 tx=0).stop 2024-09-11T00:09:50.870 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:50.866+0000 7f4ac1c62740 1 -- 172.21.15.12:0/3018544629 shutdown_connections 2024-09-11T00:09:50.870 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:50.866+0000 7f4ac1c62740 1 --2- 172.21.15.12:0/3018544629 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e0a5b38b00 0x55e0a5b3afc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:09:50.870 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:50.866+0000 7f4ac1c62740 1 --2- 172.21.15.12:0/3018544629 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e0a5b360c0 0x55e0a5b384b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:09:50.870 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:50.866+0000 7f4ac1c62740 1 --2- 172.21.15.12:0/3018544629 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e0a5b33790 0x55e0a5b35b80 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:09:50.871 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:50.866+0000 7f4ac1c62740 1 -- 172.21.15.12:0/3018544629 >> 172.21.15.12:0/3018544629 conn(0x55e0a5b2e8e0 msgr2=0x55e0a5b2f670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:09:50.871 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:50.866+0000 7f4ab5600640 1 --2- 172.21.15.12:0/1290145530 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f4a9c0b5360 0x7f4a9c0b7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:09:50.871 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:50.866+0000 7f4ac1c62740 1 -- 172.21.15.12:0/3018544629 shutdown_connections 2024-09-11T00:09:50.871 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:50.866+0000 7f4ac1c62740 1 -- 172.21.15.12:0/3018544629 wait complete. 2024-09-11T00:09:50.871 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:50.866+0000 7f4ab5600640 1 -- 172.21.15.12:0/1290145530 <== mon.2 v2:172.21.15.165:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9208+0+0 (secure 0 0 0) 0x7f4aa80e62b0 con 0x7f4ab8104ba0 2024-09-11T00:09:50.871 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:50.866+0000 7f4aafe00640 1 --2- 172.21.15.12:0/1290145530 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f4a9c0b5360 0x7f4a9c0b7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:09:50.871 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:50.867+0000 7f4ac1c62740 1 -- start start 2024-09-11T00:09:50.871 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:50.867+0000 7f4ac1c62740 1 Processor -- start 2024-09-11T00:09:50.871 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:50.867+0000 7f4aafe00640 1 --2- 172.21.15.12:0/1290145530 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f4a9c0b5360 0x7f4a9c0b7820 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f4aa0009830 tx=0x7f4aa0030480 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:09:50.871 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:50.867+0000 7f4ac1c62740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e0a5b33790 0x55e0a5b3cf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:09:50.871 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:50.867+0000 7f4ac1c62740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e0a5b360c0 0x55e0a5b3d5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:09:50.871 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:50.867+0000 7f4ac1c62740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e0a5b38b00 0x55e0a5b3e1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:09:50.872 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:50.867+0000 7f4ac1c62740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55e0a59cdab0 con 0x55e0a5b360c0 2024-09-11T00:09:50.872 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:50.867+0000 7f4ac1c62740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55e0a5a3c570 con 0x55e0a5b38b00 2024-09-11T00:09:50.872 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:50.867+0000 7f4ac1c62740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55e0a5b3bae0 con 0x55e0a5b33790 2024-09-11T00:09:50.872 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:50.868+0000 7f4ac1c62740 10 client.0 _ll_register_callbacks cb 0x55e0a5a4b100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:09:50.872 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:50.868+0000 7f4ab7e00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e0a5b33790 0x55e0a5b3cf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:09:50.872 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:50.868+0000 7f4ab7e00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e0a5b33790 0x55e0a5b3cf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.12:52984/0 (socket says 172.21.15.12:52984) 2024-09-11T00:09:50.872 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:50.868+0000 7f4ab7e00640 1 -- 172.21.15.12:0/3005572627 learned_addr learned my addr 172.21.15.12:0/3005572627 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:09:50.872 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:50.868+0000 7f4abcc00640 1 --2- 172.21.15.12:0/3005572627 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e0a5b38b00 0x55e0a5b3e1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:09:50.872 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:50.868+0000 7f4aafe00640 1 --2- 172.21.15.12:0/3005572627 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e0a5b360c0 0x55e0a5b3d5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:09:50.872 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:50.868+0000 7f4ab7e00640 1 -- 172.21.15.12:0/3005572627 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e0a5b38b00 msgr2=0x55e0a5b3e1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:09:50.872 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:50.868+0000 7f4ab7e00640 1 --2- 172.21.15.12:0/3005572627 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e0a5b38b00 0x55e0a5b3e1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:09:50.872 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:50.868+0000 7f4ab7e00640 1 -- 172.21.15.12:0/3005572627 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e0a5b360c0 msgr2=0x55e0a5b3d5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:09:50.872 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:50.868+0000 7f4ab7e00640 1 --2- 172.21.15.12:0/3005572627 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e0a5b360c0 0x55e0a5b3d5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:09:50.873 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:50.868+0000 7f4ab7e00640 1 -- 172.21.15.12:0/3005572627 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e0a57f7790 con 0x55e0a5b33790 2024-09-11T00:09:50.873 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:50.868+0000 7f4abcc00640 1 --2- 172.21.15.12:0/3005572627 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e0a5b38b00 0x55e0a5b3e1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:09:50.873 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:50.868+0000 7f4ab7e00640 1 --2- 172.21.15.12:0/3005572627 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e0a5b33790 0x55e0a5b3cf50 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f4aa80ac680 tx=0x7f4aa80ac100 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5200610765c43d61 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:09:50.873 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:50.868+0000 7f4aa5600640 10 client.0 ms_handle_connect on v2:172.21.15.178:3300/0 2024-09-11T00:09:50.873 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:50.869+0000 7f4aa5600640 1 -- 172.21.15.12:0/3005572627 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4aa80b1120 con 0x55e0a5b33790 2024-09-11T00:09:50.873 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:50.869+0000 7f4aa5600640 1 -- 172.21.15.12:0/3005572627 <== mon.1 v2:172.21.15.178:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9208+0+0 (secure 0 0 0) 0x7f4aa80ab950 con 0x55e0a5b33790 2024-09-11T00:09:50.873 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:50.869+0000 7f4ac1c62740 1 -- 172.21.15.12:0/3005572627 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4ab001d070 con 0x55e0a5b33790 2024-09-11T00:09:50.873 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:50.869+0000 7f4aa5600640 1 -- 172.21.15.12:0/3005572627 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f4aa80b5bd0 con 0x55e0a5b33790 2024-09-11T00:09:50.873 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:50.869+0000 7f4ac1c62740 10 client.25852 fetch_fsmap learned FSMap version 33 2024-09-11T00:09:50.873 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:50.869+0000 7f4ac1c62740 1 -- 172.21.15.12:0/3005572627 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55e0a5870620 con 0x55e0a5b33790 2024-09-11T00:09:50.873 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:50.869+0000 7f4aa5600640 1 -- 172.21.15.12:0/3005572627 <== mon.1 v2:172.21.15.178:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4ab001d070 con 0x55e0a5b33790 2024-09-11T00:09:50.875 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:50.869+0000 7f4aa5600640 1 -- 172.21.15.12:0/3005572627 <== mon.1 v2:172.21.15.178:3300/0 5 ==== fsmap(e 33) ==== 8386+0+0 (secure 0 0 0) 0x7f4aa80b22b0 con 0x55e0a5b33790 2024-09-11T00:09:50.875 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:50.869+0000 7f4ac1c62740 10 client.25852 fetch_fsmap finished waiting for FSMap version 33 2024-09-11T00:09:50.875 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:50.870+0000 7f4ac1c62740 10 client.25852 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T00:09:50.875 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:50.870+0000 7f4ac1c62740 20 client.25852 populate_metadata read hostname 'smithi012' 2024-09-11T00:09:50.875 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:50.870+0000 7f4ac1c62740 1 --2- 172.21.15.12:0/3005572627 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55e0a5b48790 0x55e0a5b4ac10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:09:50.875 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:50.870+0000 7f4ac1c62740 4 client.25852 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T00:09:50.876 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:50.870+0000 7f4ac1c62740 1 -- 172.21.15.12:0/3005572627 --> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e0a5aa8600 con 0x55e0a5b48790 2024-09-11T00:09:50.876 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:50.870+0000 7f4aafe00640 1 --2- 172.21.15.12:0/3005572627 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55e0a5b48790 0x55e0a5b4ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:09:50.876 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:50.871+0000 7f4aafe00640 1 --2- 172.21.15.12:0/3005572627 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55e0a5b48790 0x55e0a5b4ac10 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d69b9387a57c33e server_cookie=3c74e86a4cba6f99 in_seq=0 out_seq=0 2024-09-11T00:09:50.876 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:50.871+0000 7f4aa5600640 10 client.25852 ms_handle_connect on v2:172.21.15.165:6838/501312984 2024-09-11T00:09:50.876 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:50.871+0000 7f4aa5600640 1 -- 172.21.15.12:0/3005572627 <== mds.0 v2:172.21.15.165:6838/501312984 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55e0a5aa8600 con 0x55e0a5b48790 2024-09-11T00:09:50.876 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:50.871+0000 7f4aa5600640 10 client.25852 handle_command_reply: tid=0 2024-09-11T00:09:50.876 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:50.871+0000 7f4ac1c62740 1 client.25852 shutdown 2024-09-11T00:09:50.876 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:50.872+0000 7f4ac1c62740 1 -- 172.21.15.12:0/3005572627 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e0a5b33790 msgr2=0x55e0a5b3cf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:09:50.876 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:50.872+0000 7f4ac1c62740 1 --2- 172.21.15.12:0/3005572627 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e0a5b33790 0x55e0a5b3cf50 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f4aa80ac680 tx=0x7f4aa80ac100 comp rx=0 tx=0).stop 2024-09-11T00:09:50.876 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:50.872+0000 7f4ac1c62740 1 -- 172.21.15.12:0/3005572627 shutdown_connections 2024-09-11T00:09:50.876 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:50.872+0000 7f4ac1c62740 1 --2- 172.21.15.12:0/3005572627 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55e0a5b48790 0x55e0a5b4ac10 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:09:50.876 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:50.872+0000 7f4ac1c62740 1 --2- 172.21.15.12:0/3005572627 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e0a5b38b00 0x55e0a5b3e1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:09:50.877 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:50.872+0000 7f4aa5600640 0 client.25852 ms_handle_reset on v2:172.21.15.165:6838/501312984 2024-09-11T00:09:50.877 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:50.872+0000 7f4ac1c62740 1 --2- 172.21.15.12:0/3005572627 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e0a5b360c0 0x55e0a5b3d5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:09:50.877 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:50.872+0000 7f4ac1c62740 1 --2- 172.21.15.12:0/3005572627 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e0a5b33790 0x55e0a5b3cf50 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:09:50.877 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:50.872+0000 7f4ac1c62740 1 -- 172.21.15.12:0/3005572627 >> 172.21.15.12:0/3005572627 conn(0x55e0a5b2e8e0 msgr2=0x55e0a5b2da60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:09:50.877 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:50.872+0000 7f4ac1c62740 1 -- 172.21.15.12:0/3005572627 shutdown_connections 2024-09-11T00:09:50.877 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:50.872+0000 7f4ac1c62740 1 -- 172.21.15.12:0/3005572627 wait complete. 2024-09-11T00:09:50.877 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:50.872+0000 7f4ac1c62740 20 client.25852 trim_cache size 0 max 16384 2024-09-11T00:09:50.889 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:50.888+0000 7f4ac1c62740 1 Processor -- start 2024-09-11T00:09:50.889 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:50.888+0000 7f4ac1c62740 1 -- start start 2024-09-11T00:09:50.889 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:50.888+0000 7f4ac1c62740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e0a5b33790 0x55e0a5b38ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:09:50.889 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:50.888+0000 7f4ac1c62740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e0a5b360c0 0x55e0a5b39420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:09:50.890 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:50.888+0000 7f4ac1c62740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e0a5b38b00 0x55e0a5b30710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:09:50.890 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:50.888+0000 7f4ac1c62740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f4ab81ad900 con 0x55e0a5b360c0 2024-09-11T00:09:50.890 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:50.888+0000 7f4ac1c62740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f4ab81af500 con 0x55e0a5b38b00 2024-09-11T00:09:50.890 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:50.888+0000 7f4ac1c62740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f4ab81adfd0 con 0x55e0a5b33790 2024-09-11T00:09:50.890 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:50.888+0000 7f4ab7e00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e0a5b33790 0x55e0a5b38ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:09:50.890 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:50.888+0000 7f4ab7e00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e0a5b33790 0x55e0a5b38ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.12:52988/0 (socket says 172.21.15.12:52988) 2024-09-11T00:09:50.890 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:50.888+0000 7f4aafe00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e0a5b360c0 0x55e0a5b39420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:09:50.890 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:50.888+0000 7f4ab7e00640 1 -- 172.21.15.12:0/1441553143 learned_addr learned my addr 172.21.15.12:0/1441553143 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:09:50.890 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:50.888+0000 7f4abcc00640 1 --2- 172.21.15.12:0/1441553143 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e0a5b38b00 0x55e0a5b30710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:09:50.894 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:50.889+0000 7f4ab7e00640 1 -- 172.21.15.12:0/1441553143 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e0a5b38b00 msgr2=0x55e0a5b30710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:09:50.894 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:50.889+0000 7f4ab7e00640 1 --2- 172.21.15.12:0/1441553143 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e0a5b38b00 0x55e0a5b30710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:09:50.894 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:50.889+0000 7f4ab7e00640 1 -- 172.21.15.12:0/1441553143 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e0a5b360c0 msgr2=0x55e0a5b39420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:09:50.894 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:50.889+0000 7f4ab7e00640 1 --2- 172.21.15.12:0/1441553143 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e0a5b360c0 0x55e0a5b39420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:09:50.894 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:50.889+0000 7f4ab7e00640 1 -- 172.21.15.12:0/1441553143 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e0a5b35ef0 con 0x55e0a5b33790 2024-09-11T00:09:50.894 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:50.889+0000 7f4abcc00640 1 --2- 172.21.15.12:0/1441553143 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e0a5b38b00 0x55e0a5b30710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:09:50.894 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:50.889+0000 7f4ab7e00640 1 --2- 172.21.15.12:0/1441553143 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e0a5b33790 0x55e0a5b38ee0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f4aa8000c00 tx=0x7f4aa80ac170 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c51b5fccfbf2f2ee server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:09:50.894 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:50.889+0000 7f4aa7e00640 1 -- 172.21.15.12:0/1441553143 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4aa80ab3c0 con 0x55e0a5b33790 2024-09-11T00:09:50.894 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:50.889+0000 7f4aa7e00640 1 -- 172.21.15.12:0/1441553143 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4aa80ab560 con 0x55e0a5b33790 2024-09-11T00:09:50.895 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:50.889+0000 7f4aa7e00640 1 -- 172.21.15.12:0/1441553143 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4aa80a8040 con 0x55e0a5b33790 2024-09-11T00:09:50.895 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:50.890+0000 7f4ac1c62740 1 -- 172.21.15.12:0/1441553143 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e0a5b33790 msgr2=0x55e0a5b38ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:09:50.895 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:50.890+0000 7f4ac1c62740 1 --2- 172.21.15.12:0/1441553143 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e0a5b33790 0x55e0a5b38ee0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f4aa8000c00 tx=0x7f4aa80ac170 comp rx=0 tx=0).stop 2024-09-11T00:09:50.895 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:50.890+0000 7f4ac1c62740 1 -- 172.21.15.12:0/1441553143 shutdown_connections 2024-09-11T00:09:50.895 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:50.890+0000 7f4ac1c62740 1 --2- 172.21.15.12:0/1441553143 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e0a5b38b00 0x55e0a5b30710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:09:50.895 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:50.890+0000 7f4ac1c62740 1 --2- 172.21.15.12:0/1441553143 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e0a5b360c0 0x55e0a5b39420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:09:50.895 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:50.890+0000 7f4ac1c62740 1 --2- 172.21.15.12:0/1441553143 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e0a5b33790 0x55e0a5b38ee0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:09:50.895 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:50.890+0000 7f4ac1c62740 1 -- 172.21.15.12:0/1441553143 >> 172.21.15.12:0/1441553143 conn(0x55e0a5b2e8e0 msgr2=0x55e0a5b3bee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:09:50.895 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:50.890+0000 7f4ac1c62740 1 -- 172.21.15.12:0/1441553143 shutdown_connections 2024-09-11T00:09:50.895 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:50.890+0000 7f4ac1c62740 1 -- 172.21.15.12:0/1441553143 wait complete. 2024-09-11T00:09:50.895 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:50.890+0000 7f4ac1c62740 1 -- start start 2024-09-11T00:09:50.895 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:50.891+0000 7f4ac1c62740 1 Processor -- start 2024-09-11T00:09:50.896 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:50.891+0000 7f4ac1c62740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e0a5b33790 0x55e0a5b44aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:09:50.896 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:50.891+0000 7f4ac1c62740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e0a5b360c0 0x55e0a5b437a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:09:50.896 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:50.891+0000 7f4ac1c62740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e0a5b38b00 0x55e0a5b43d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:09:50.896 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:50.891+0000 7f4ac1c62740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f4ab81aee30 con 0x55e0a5b33790 2024-09-11T00:09:50.896 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:50.891+0000 7f4ac1c62740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f4ab81ae6a0 con 0x55e0a5b38b00 2024-09-11T00:09:50.896 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:50.891+0000 7f4ac1c62740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f4ab81ab8e0 con 0x55e0a5b360c0 2024-09-11T00:09:50.896 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:50.891+0000 7f4ac1c62740 10 client.0 _ll_register_callbacks cb 0x7f4aa80a8750 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:09:50.896 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:50.891+0000 7f4ab7e00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e0a5b33790 0x55e0a5b44aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:09:50.896 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:50.891+0000 7f4aafe00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e0a5b360c0 0x55e0a5b437a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:09:50.896 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:50.891+0000 7f4abcc00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e0a5b38b00 0x55e0a5b43d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:09:50.896 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:50.891+0000 7f4aafe00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e0a5b360c0 0x55e0a5b437a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.12:53004/0 (socket says 172.21.15.12:53004) 2024-09-11T00:09:50.896 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:50.891+0000 7f4aafe00640 1 -- 172.21.15.12:0/3316028022 learned_addr learned my addr 172.21.15.12:0/3316028022 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:09:50.896 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:50.892+0000 7f4aafe00640 1 -- 172.21.15.12:0/3316028022 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e0a5b38b00 msgr2=0x55e0a5b43d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:09:50.897 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:50.892+0000 7f4aafe00640 1 --2- 172.21.15.12:0/3316028022 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e0a5b38b00 0x55e0a5b43d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:09:50.897 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:50.892+0000 7f4aafe00640 1 -- 172.21.15.12:0/3316028022 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e0a5b33790 msgr2=0x55e0a5b44aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:09:50.897 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:50.892+0000 7f4aafe00640 1 --2- 172.21.15.12:0/3316028022 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e0a5b33790 0x55e0a5b44aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:09:50.897 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:50.892+0000 7f4aafe00640 1 -- 172.21.15.12:0/3316028022 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e0a5b44570 con 0x55e0a5b360c0 2024-09-11T00:09:50.897 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:50.892+0000 7f4aafe00640 1 --2- 172.21.15.12:0/3316028022 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e0a5b360c0 0x55e0a5b437a0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f4aa002a930 tx=0x7f4aa002a340 comp rx=0 tx=0).ready entity=mon.1 client_cookie=95caa8e4810424be server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:09:50.897 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:50.892+0000 7f4aa5600640 10 client.0 ms_handle_connect on v2:172.21.15.178:3300/0 2024-09-11T00:09:50.897 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:50.892+0000 7f4aa5600640 1 -- 172.21.15.12:0/3316028022 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4aa0058060 con 0x55e0a5b360c0 2024-09-11T00:09:50.897 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:50.892+0000 7f4ac1c62740 1 -- 172.21.15.12:0/3316028022 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55e0a5aa40b0 con 0x55e0a5b360c0 2024-09-11T00:09:50.897 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:50.892+0000 7f4aa5600640 1 -- 172.21.15.12:0/3316028022 <== mon.1 v2:172.21.15.178:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9208+0+0 (secure 0 0 0) 0x7f4aa00037f0 con 0x55e0a5b360c0 2024-09-11T00:09:50.897 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:50.892+0000 7f4aa5600640 1 -- 172.21.15.12:0/3316028022 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f4aa005f050 con 0x55e0a5b360c0 2024-09-11T00:09:50.897 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:50.892+0000 7f4ac1c62740 10 client.25858 fetch_fsmap learned FSMap version 33 2024-09-11T00:09:50.897 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:50.892+0000 7f4ac1c62740 1 -- 172.21.15.12:0/3316028022 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55e0a5bb58a0 con 0x55e0a5b360c0 2024-09-11T00:09:50.898 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:50.893+0000 7f4aa5600640 1 -- 172.21.15.12:0/3316028022 <== mon.1 v2:172.21.15.178:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55e0a5aa40b0 con 0x55e0a5b360c0 2024-09-11T00:09:50.898 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:50.893+0000 7f4aa5600640 1 -- 172.21.15.12:0/3316028022 <== mon.1 v2:172.21.15.178:3300/0 5 ==== fsmap(e 33) ==== 8386+0+0 (secure 0 0 0) 0x7f4aa003da60 con 0x55e0a5b360c0 2024-09-11T00:09:50.898 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:50.893+0000 7f4ac1c62740 10 client.25858 fetch_fsmap finished waiting for FSMap version 33 2024-09-11T00:09:50.898 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:50.893+0000 7f4ac1c62740 10 client.25858 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T00:09:50.898 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:50.893+0000 7f4ac1c62740 20 client.25858 populate_metadata read hostname 'smithi012' 2024-09-11T00:09:50.898 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:50.893+0000 7f4ac1c62740 1 --2- 172.21.15.12:0/3316028022 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55e0a5bb62c0 0x55e0a5bb86e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:09:50.898 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:50.893+0000 7f4ac1c62740 4 client.25858 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T00:09:50.898 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:50.893+0000 7f4ac1c62740 1 -- 172.21.15.12:0/3316028022 --> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55e0a5aa14e0 con 0x55e0a5bb62c0 2024-09-11T00:09:50.898 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:50.893+0000 7f4ab7e00640 1 --2- 172.21.15.12:0/3316028022 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55e0a5bb62c0 0x55e0a5bb86e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:09:50.898 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:50.894+0000 7f4ab7e00640 1 --2- 172.21.15.12:0/3316028022 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55e0a5bb62c0 0x55e0a5bb86e0 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ef103b2de8fd6ca7 server_cookie=edc82c312a7d9f34 in_seq=0 out_seq=0 2024-09-11T00:09:50.899 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:50.894+0000 7f4aa5600640 10 client.25858 ms_handle_connect on v2:172.21.15.165:6838/501312984 2024-09-11T00:09:50.920 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:50.918+0000 7f4aa5600640 1 -- 172.21.15.12:0/3316028022 <== mds.0 v2:172.21.15.165:6838/501312984 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55e0a5aa14e0 con 0x55e0a5bb62c0 2024-09-11T00:09:50.920 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:50.918+0000 7f4aa5600640 10 client.25858 handle_command_reply: tid=0 2024-09-11T00:09:50.920 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:50.918+0000 7f4ac1c62740 1 client.25858 shutdown 2024-09-11T00:09:50.920 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:50.918+0000 7f4ac1c62740 1 -- 172.21.15.12:0/3316028022 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e0a5b360c0 msgr2=0x55e0a5b437a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:09:50.920 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:50.918+0000 7f4ac1c62740 1 --2- 172.21.15.12:0/3316028022 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e0a5b360c0 0x55e0a5b437a0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f4aa002a930 tx=0x7f4aa002a340 comp rx=0 tx=0).stop 2024-09-11T00:09:50.920 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:50.918+0000 7f4ac1c62740 1 -- 172.21.15.12:0/3316028022 shutdown_connections 2024-09-11T00:09:50.920 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:50.918+0000 7f4ac1c62740 1 --2- 172.21.15.12:0/3316028022 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55e0a5bb62c0 0x55e0a5bb86e0 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:09:50.920 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:50.918+0000 7f4ac1c62740 1 --2- 172.21.15.12:0/3316028022 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e0a5b38b00 0x55e0a5b43d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:09:50.920 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:50.918+0000 7f4ac1c62740 1 --2- 172.21.15.12:0/3316028022 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e0a5b360c0 0x55e0a5b437a0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:09:50.920 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:50.918+0000 7f4ac1c62740 1 --2- 172.21.15.12:0/3316028022 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e0a5b33790 0x55e0a5b44aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:09:50.921 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:50.918+0000 7f4aa5600640 0 client.25858 ms_handle_reset on v2:172.21.15.165:6838/501312984 2024-09-11T00:09:50.921 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:50.918+0000 7f4ac1c62740 1 -- 172.21.15.12:0/3316028022 >> 172.21.15.12:0/3316028022 conn(0x55e0a5b2e8e0 msgr2=0x55e0a5b38ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:09:50.921 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:50.919+0000 7f4ac1c62740 1 -- 172.21.15.12:0/3316028022 shutdown_connections 2024-09-11T00:09:50.921 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:50.919+0000 7f4ac1c62740 1 -- 172.21.15.12:0/3316028022 wait complete. 2024-09-11T00:09:50.921 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:50.919+0000 7f4ac1c62740 20 client.25858 trim_cache size 0 max 16384 2024-09-11T00:09:50.921 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:{ 2024-09-11T00:09:50.921 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "message": "", 2024-09-11T00:09:50.921 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "return_code": 0 2024-09-11T00:09:50.921 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:} 2024-09-11T00:09:50.922 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:50.919+0000 7f4abec00640 1 -- 172.21.15.12:0/1290145530 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f4a9c0b5360 msgr2=0x7f4a9c0b7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:09:50.922 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:50.919+0000 7f4abec00640 1 --2- 172.21.15.12:0/1290145530 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f4a9c0b5360 0x7f4a9c0b7820 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f4aa0009830 tx=0x7f4aa0030480 comp rx=0 tx=0).stop 2024-09-11T00:09:50.922 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:50.919+0000 7f4abec00640 1 -- 172.21.15.12:0/1290145530 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4ab8104ba0 msgr2=0x7f4ab819af90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:09:50.922 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:50.919+0000 7f4abec00640 1 --2- 172.21.15.12:0/1290145530 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4ab8104ba0 0x7f4ab819af90 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f4aa800e9f0 tx=0x7f4aa800eeb0 comp rx=0 tx=0).stop 2024-09-11T00:09:50.922 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:50.919+0000 7f4abec00640 1 -- 172.21.15.12:0/1290145530 shutdown_connections 2024-09-11T00:09:50.922 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:50.920+0000 7f4abec00640 1 --2- 172.21.15.12:0/1290145530 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f4a9c0b5360 0x7f4a9c0b7820 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:09:50.923 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:50.920+0000 7f4abec00640 1 --2- 172.21.15.12:0/1290145530 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4ab81066b0 0x7f4ab819c1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:09:50.923 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:50.920+0000 7f4abec00640 1 --2- 172.21.15.12:0/1290145530 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4ab8105cf0 0x7f4ab819b5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:09:50.923 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:50.920+0000 7f4abec00640 1 --2- 172.21.15.12:0/1290145530 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4ab8104ba0 0x7f4ab819af90 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:09:50.923 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:50.920+0000 7f4abec00640 1 -- 172.21.15.12:0/1290145530 >> 172.21.15.12:0/1290145530 conn(0x7f4ab8100460 msgr2=0x7f4ab8109080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:09:50.923 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:50.920+0000 7f4abec00640 1 -- 172.21.15.12:0/1290145530 shutdown_connections 2024-09-11T00:09:50.923 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:09:50.920+0000 7f4abec00640 1 -- 172.21.15.12:0/1290145530 wait complete. 2024-09-11T00:09:51.438 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:09:51 smithi012 ceph-mon[30333]: pgmap v482: 129 pgs: 129 active+clean; 29 MiB data, 540 MiB used, 1.0 TiB / 1.0 TiB avail; 268 KiB/s rd, 201 KiB/s wr, 39 op/s 2024-09-11T00:09:51.581 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:09:51 smithi178 ceph-mon[35919]: pgmap v482: 129 pgs: 129 active+clean; 29 MiB data, 540 MiB used, 1.0 TiB / 1.0 TiB avail; 268 KiB/s rd, 201 KiB/s wr, 39 op/s 2024-09-11T00:09:51.608 INFO:tasks.workunit.client.0.smithi012.stdout: 1 96206 4.97 MB/sec warmup 108 sec latency 20.162 ms 2024-09-11T00:09:51.782 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:09:51 smithi165 ceph-mon[35356]: pgmap v482: 129 pgs: 129 active+clean; 29 MiB data, 540 MiB used, 1.0 TiB / 1.0 TiB avail; 268 KiB/s rd, 201 KiB/s wr, 39 op/s 2024-09-11T00:09:52.608 INFO:tasks.workunit.client.0.smithi012.stdout: 1 96483 4.93 MB/sec warmup 109 sec latency 748.370 ms 2024-09-11T00:09:53.581 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:09:53 smithi178 ceph-mon[35919]: pgmap v483: 129 pgs: 129 active+clean; 25 MiB data, 541 MiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 315 KiB/s wr, 46 op/s 2024-09-11T00:09:53.608 INFO:tasks.workunit.client.0.smithi012.stdout: 1 96483 4.88 MB/sec warmup 110 sec latency 1748.465 ms 2024-09-11T00:09:53.782 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:09:53 smithi165 ceph-mon[35356]: pgmap v483: 129 pgs: 129 active+clean; 25 MiB data, 541 MiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 315 KiB/s wr, 46 op/s 2024-09-11T00:09:53.813 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:09:53 smithi012 ceph-mon[30333]: pgmap v483: 129 pgs: 129 active+clean; 25 MiB data, 541 MiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 315 KiB/s wr, 46 op/s 2024-09-11T00:09:54.609 INFO:tasks.workunit.client.0.smithi012.stdout: 1 96539 4.84 MB/sec warmup 111 sec latency 2676.088 ms 2024-09-11T00:09:55.581 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:09:55 smithi178 ceph-mon[35919]: pgmap v484: 129 pgs: 129 active+clean; 25 MiB data, 546 MiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 317 KiB/s wr, 40 op/s 2024-09-11T00:09:55.609 INFO:tasks.workunit.client.0.smithi012.stdout: 1 96539 4.80 MB/sec warmup 112 sec latency 1021.143 ms 2024-09-11T00:09:55.782 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:09:55 smithi165 ceph-mon[35356]: pgmap v484: 129 pgs: 129 active+clean; 25 MiB data, 546 MiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 317 KiB/s wr, 40 op/s 2024-09-11T00:09:55.813 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:09:55 smithi012 ceph-mon[30333]: pgmap v484: 129 pgs: 129 active+clean; 25 MiB data, 546 MiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 317 KiB/s wr, 40 op/s 2024-09-11T00:09:56.609 INFO:tasks.workunit.client.0.smithi012.stdout: 1 96539 4.76 MB/sec warmup 113 sec latency 2021.251 ms 2024-09-11T00:09:57.581 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:09:57 smithi178 ceph-mon[35919]: pgmap v485: 129 pgs: 129 active+clean; 25 MiB data, 551 MiB used, 1.0 TiB / 1.0 TiB avail; 170 KiB/s rd, 506 KiB/s wr, 74 op/s 2024-09-11T00:09:57.609 INFO:tasks.workunit.client.0.smithi012.stdout: 1 96539 4.71 MB/sec warmup 114 sec latency 3021.364 ms 2024-09-11T00:09:57.782 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:09:57 smithi165 ceph-mon[35356]: pgmap v485: 129 pgs: 129 active+clean; 25 MiB data, 551 MiB used, 1.0 TiB / 1.0 TiB avail; 170 KiB/s rd, 506 KiB/s wr, 74 op/s 2024-09-11T00:09:57.812 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:09:57 smithi012 ceph-mon[30333]: pgmap v485: 129 pgs: 129 active+clean; 25 MiB data, 551 MiB used, 1.0 TiB / 1.0 TiB avail; 170 KiB/s rd, 506 KiB/s wr, 74 op/s 2024-09-11T00:09:58.609 INFO:tasks.workunit.client.0.smithi012.stdout: 1 96539 4.67 MB/sec warmup 115 sec latency 4021.435 ms 2024-09-11T00:09:59.580 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:09:59 smithi178 ceph-mon[35919]: pgmap v486: 129 pgs: 129 active+clean; 25 MiB data, 551 MiB used, 1.0 TiB / 1.0 TiB avail; 181 KiB/s rd, 505 KiB/s wr, 72 op/s 2024-09-11T00:09:59.609 INFO:tasks.workunit.client.0.smithi012.stdout: 1 96607 4.63 MB/sec warmup 116 sec latency 4948.133 ms 2024-09-11T00:09:59.782 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:09:59 smithi165 ceph-mon[35356]: pgmap v486: 129 pgs: 129 active+clean; 25 MiB data, 551 MiB used, 1.0 TiB / 1.0 TiB avail; 181 KiB/s rd, 505 KiB/s wr, 72 op/s 2024-09-11T00:09:59.812 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:09:59 smithi012 ceph-mon[30333]: pgmap v486: 129 pgs: 129 active+clean; 25 MiB data, 551 MiB used, 1.0 TiB / 1.0 TiB avail; 181 KiB/s rd, 505 KiB/s wr, 72 op/s 2024-09-11T00:10:00.609 INFO:tasks.workunit.client.0.smithi012.stdout: 1 96607 4.60 MB/sec warmup 117 sec latency 1018.191 ms 2024-09-11T00:10:00.782 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:10:00 smithi165 ceph-mon[35356]: overall HEALTH_OK 2024-09-11T00:10:00.812 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:10:00 smithi012 ceph-mon[30333]: overall HEALTH_OK 2024-09-11T00:10:00.830 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:10:00 smithi178 ceph-mon[35919]: overall HEALTH_OK 2024-09-11T00:10:01.610 INFO:tasks.workunit.client.0.smithi012.stdout: 1 96607 4.56 MB/sec warmup 118 sec latency 2018.328 ms 2024-09-11T00:10:01.782 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:10:01 smithi165 ceph-mon[35356]: pgmap v487: 129 pgs: 129 active+clean; 25 MiB data, 551 MiB used, 1.0 TiB / 1.0 TiB avail; 181 KiB/s rd, 521 KiB/s wr, 74 op/s 2024-09-11T00:10:01.812 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:10:01 smithi012 ceph-mon[30333]: pgmap v487: 129 pgs: 129 active+clean; 25 MiB data, 551 MiB used, 1.0 TiB / 1.0 TiB avail; 181 KiB/s rd, 521 KiB/s wr, 74 op/s 2024-09-11T00:10:01.830 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:10:01 smithi178 ceph-mon[35919]: pgmap v487: 129 pgs: 129 active+clean; 25 MiB data, 551 MiB used, 1.0 TiB / 1.0 TiB avail; 181 KiB/s rd, 521 KiB/s wr, 74 op/s 2024-09-11T00:10:02.609 INFO:tasks.workunit.client.0.smithi012.stdout: 1 96607 4.52 MB/sec warmup 119 sec latency 3018.463 ms 2024-09-11T00:10:03.782 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:10:03 smithi165 ceph-mon[35356]: pgmap v488: 129 pgs: 129 active+clean; 25 MiB data, 551 MiB used, 1.0 TiB / 1.0 TiB avail; 187 KiB/s rd, 398 KiB/s wr, 74 op/s 2024-09-11T00:10:03.813 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:10:03 smithi012 ceph-mon[30333]: pgmap v488: 129 pgs: 129 active+clean; 25 MiB data, 551 MiB used, 1.0 TiB / 1.0 TiB avail; 187 KiB/s rd, 398 KiB/s wr, 74 op/s 2024-09-11T00:10:03.831 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:10:03 smithi178 ceph-mon[35919]: pgmap v488: 129 pgs: 129 active+clean; 25 MiB data, 551 MiB used, 1.0 TiB / 1.0 TiB avail; 187 KiB/s rd, 398 KiB/s wr, 74 op/s 2024-09-11T00:10:04.051 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:04.049+0000 7f1965c00640 1 -- 172.21.15.12:0/518530705 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f19601084a0 msgr2=0x7f196010a960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:10:04.051 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:04.049+0000 7f1965c00640 1 --2- 172.21.15.12:0/518530705 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f19601084a0 0x7f196010a960 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f1958009ca0 tx=0x7f195802a8c0 comp rx=0 tx=0).stop 2024-09-11T00:10:04.051 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:04.049+0000 7f1965c00640 1 -- 172.21.15.12:0/518530705 shutdown_connections 2024-09-11T00:10:04.051 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:04.049+0000 7f1965c00640 1 --2- 172.21.15.12:0/518530705 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f19601084a0 0x7f196010a960 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:10:04.051 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:04.049+0000 7f1965c00640 1 --2- 172.21.15.12:0/518530705 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f19601058f0 0x7f1960107d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:10:04.051 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:04.049+0000 7f1965c00640 1 --2- 172.21.15.12:0/518530705 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f196006b470 0x7f1960105360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:10:04.051 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:04.049+0000 7f1965c00640 1 -- 172.21.15.12:0/518530705 >> 172.21.15.12:0/518530705 conn(0x7f19600fcf30 msgr2=0x7f19600ff370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:10:04.051 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:04.049+0000 7f1965c00640 1 -- 172.21.15.12:0/518530705 shutdown_connections 2024-09-11T00:10:04.051 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:04.050+0000 7f1965c00640 1 -- 172.21.15.12:0/518530705 wait complete. 2024-09-11T00:10:04.054 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:04.050+0000 7f1965c00640 1 Processor -- start 2024-09-11T00:10:04.054 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:04.051+0000 7f1965c00640 1 -- start start 2024-09-11T00:10:04.054 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:04.051+0000 7f1965c00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f196006b470 0x7f1960102aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:10:04.054 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:04.051+0000 7f1965c00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f19601058f0 0x7f1960101300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:10:04.054 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:04.051+0000 7f1965c00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f19601084a0 0x7f1960101840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:10:04.054 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:04.051+0000 7f1965c00640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f196010e440 con 0x7f196006b470 2024-09-11T00:10:04.054 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:04.051+0000 7f1965c00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f196010e2c0 con 0x7f19601058f0 2024-09-11T00:10:04.054 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:04.051+0000 7f1965c00640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f196010e5c0 con 0x7f19601084a0 2024-09-11T00:10:04.054 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:04.051+0000 7f195fe00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f196006b470 0x7f1960102aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:10:04.054 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:04.051+0000 7f195fe00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f196006b470 0x7f1960102aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:50188/0 (socket says 172.21.15.12:50188) 2024-09-11T00:10:04.055 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:04.051+0000 7f195fe00640 1 -- 172.21.15.12:0/4049962597 learned_addr learned my addr 172.21.15.12:0/4049962597 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:10:04.055 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:04.051+0000 7f195f400640 1 --2- 172.21.15.12:0/4049962597 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f19601058f0 0x7f1960101300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:10:04.055 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:04.051+0000 7f1965200640 1 --2- 172.21.15.12:0/4049962597 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f19601084a0 0x7f1960101840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:10:04.055 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:04.052+0000 7f195fe00640 1 -- 172.21.15.12:0/4049962597 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f19601084a0 msgr2=0x7f1960101840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:10:04.055 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:04.052+0000 7f195fe00640 1 --2- 172.21.15.12:0/4049962597 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f19601084a0 0x7f1960101840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:10:04.055 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:04.052+0000 7f195fe00640 1 -- 172.21.15.12:0/4049962597 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f19601058f0 msgr2=0x7f1960101300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:10:04.055 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:04.052+0000 7f195fe00640 1 --2- 172.21.15.12:0/4049962597 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f19601058f0 0x7f1960101300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:10:04.055 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:04.052+0000 7f195fe00640 1 -- 172.21.15.12:0/4049962597 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19601020b0 con 0x7f196006b470 2024-09-11T00:10:04.055 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:04.052+0000 7f195f400640 1 --2- 172.21.15.12:0/4049962597 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f19601058f0 0x7f1960101300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:10:04.055 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:04.052+0000 7f195fe00640 1 --2- 172.21.15.12:0/4049962597 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f196006b470 0x7f1960102aa0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f195000e9f0 tx=0x7f195000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:10:04.055 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:04.052+0000 7f195cc00640 1 -- 172.21.15.12:0/4049962597 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f195000cd80 con 0x7f196006b470 2024-09-11T00:10:04.055 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:04.052+0000 7f195cc00640 1 -- 172.21.15.12:0/4049962597 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f19500054d0 con 0x7f196006b470 2024-09-11T00:10:04.055 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:04.052+0000 7f1965c00640 1 -- 172.21.15.12:0/4049962597 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f19601a2be0 con 0x7f196006b470 2024-09-11T00:10:04.056 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:04.052+0000 7f195cc00640 1 -- 172.21.15.12:0/4049962597 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1950010690 con 0x7f196006b470 2024-09-11T00:10:04.056 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:04.052+0000 7f1965c00640 1 -- 172.21.15.12:0/4049962597 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19601a3100 con 0x7f196006b470 2024-09-11T00:10:04.057 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:04.056+0000 7f196a0ca740 1 Processor -- start 2024-09-11T00:10:04.057 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:04.056+0000 7f196a0ca740 1 -- start start 2024-09-11T00:10:04.058 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:04.057+0000 7f196a0ca740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d2cdf16970 0x55d2cdf18d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:10:04.059 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:04.057+0000 7f196a0ca740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d2cdf192a0 0x55d2cdf1b690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:10:04.059 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:04.057+0000 7f196a0ca740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d2cdf1bce0 0x55d2cdf1e1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:10:04.059 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:04.057+0000 7f196a0ca740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55d2cde896b0 con 0x55d2cdf16970 2024-09-11T00:10:04.059 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:04.057+0000 7f196a0ca740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55d2cde78ae0 con 0x55d2cdf192a0 2024-09-11T00:10:04.059 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:04.057+0000 7f196a0ca740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55d2cde7c8f0 con 0x55d2cdf1bce0 2024-09-11T00:10:04.059 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:04.057+0000 7f1965200640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d2cdf16970 0x55d2cdf18d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:10:04.059 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:04.057+0000 7f1965200640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d2cdf16970 0x55d2cdf18d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:50196/0 (socket says 172.21.15.12:50196) 2024-09-11T00:10:04.059 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:04.057+0000 7f1965200640 1 -- 172.21.15.12:0/3974123827 learned_addr learned my addr 172.21.15.12:0/3974123827 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:10:04.059 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:04.057+0000 7f195f400640 1 --2- 172.21.15.12:0/3974123827 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d2cdf1bce0 0x55d2cdf1e1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:10:04.059 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:04.058+0000 7f195cc00640 1 -- 172.21.15.12:0/4049962597 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7f1950004e90 con 0x7f196006b470 2024-09-11T00:10:04.059 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:04.058+0000 7f1965200640 1 -- 172.21.15.12:0/3974123827 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d2cdf1bce0 msgr2=0x55d2cdf1e1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:10:04.059 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:04.058+0000 7f1965200640 1 --2- 172.21.15.12:0/3974123827 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d2cdf1bce0 0x55d2cdf1e1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:10:04.059 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:04.058+0000 7f1965200640 1 -- 172.21.15.12:0/3974123827 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d2cdf192a0 msgr2=0x55d2cdf1b690 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-11T00:10:04.060 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:04.058+0000 7f1965200640 1 --2- 172.21.15.12:0/3974123827 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d2cdf192a0 0x55d2cdf1b690 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:10:04.060 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:04.058+0000 7f1965200640 1 -- 172.21.15.12:0/3974123827 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d2cde45780 con 0x55d2cdf16970 2024-09-11T00:10:04.062 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:04.058+0000 7f1965200640 1 --2- 172.21.15.12:0/3974123827 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d2cdf16970 0x55d2cdf18d60 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f1958009ca0 tx=0x7f19580093c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7fc86b347f048d6 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:10:04.062 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:04.058+0000 7f1955600640 1 -- 172.21.15.12:0/3974123827 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1958031650 con 0x55d2cdf16970 2024-09-11T00:10:04.062 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:04.058+0000 7f1955600640 1 -- 172.21.15.12:0/3974123827 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1958036030 con 0x55d2cdf16970 2024-09-11T00:10:04.062 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:04.058+0000 7f1955600640 1 -- 172.21.15.12:0/3974123827 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1958004660 con 0x55d2cdf16970 2024-09-11T00:10:04.062 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:04.059+0000 7f196a0ca740 1 -- 172.21.15.12:0/3974123827 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d2cdf16970 msgr2=0x55d2cdf18d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:10:04.062 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:04.059+0000 7f196a0ca740 1 --2- 172.21.15.12:0/3974123827 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d2cdf16970 0x55d2cdf18d60 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f1958009ca0 tx=0x7f19580093c0 comp rx=0 tx=0).stop 2024-09-11T00:10:04.062 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:04.059+0000 7f196a0ca740 1 -- 172.21.15.12:0/3974123827 shutdown_connections 2024-09-11T00:10:04.062 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:04.059+0000 7f196a0ca740 1 --2- 172.21.15.12:0/3974123827 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d2cdf1bce0 0x55d2cdf1e1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:10:04.062 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:04.059+0000 7f196a0ca740 1 --2- 172.21.15.12:0/3974123827 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d2cdf192a0 0x55d2cdf1b690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:10:04.062 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:04.059+0000 7f196a0ca740 1 --2- 172.21.15.12:0/3974123827 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d2cdf16970 0x55d2cdf18d60 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:10:04.062 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:04.059+0000 7f196a0ca740 1 -- 172.21.15.12:0/3974123827 >> 172.21.15.12:0/3974123827 conn(0x55d2cdf11ac0 msgr2=0x55d2cdf12850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:10:04.063 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:04.059+0000 7f196a0ca740 1 -- 172.21.15.12:0/3974123827 shutdown_connections 2024-09-11T00:10:04.063 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:04.059+0000 7f196a0ca740 1 -- 172.21.15.12:0/3974123827 wait complete. 2024-09-11T00:10:04.063 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:04.059+0000 7f195cc00640 1 --2- 172.21.15.12:0/4049962597 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f19440b5280 0x7f19440b7740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:10:04.063 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:04.060+0000 7f195cc00640 1 -- 172.21.15.12:0/4049962597 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9208+0+0 (secure 0 0 0) 0x7f19500e6380 con 0x7f196006b470 2024-09-11T00:10:04.063 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:04.060+0000 7f196a0ca740 1 -- start start 2024-09-11T00:10:04.063 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:04.060+0000 7f195f400640 1 --2- 172.21.15.12:0/4049962597 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f19440b5280 0x7f19440b7740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:10:04.063 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:04.060+0000 7f196a0ca740 1 Processor -- start 2024-09-11T00:10:04.063 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:04.061+0000 7f196a0ca740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d2cdf16970 0x55d2cdf1ff00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:10:04.063 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:04.061+0000 7f196a0ca740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d2cdf192a0 0x55d2cdf205a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:10:04.063 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:04.061+0000 7f196a0ca740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d2cdf1bce0 0x55d2cdf21170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:10:04.063 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:04.061+0000 7f195f400640 1 --2- 172.21.15.12:0/4049962597 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f19440b5280 0x7f19440b7740 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f194c012580 tx=0x7f194c005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:10:04.063 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:04.061+0000 7f196a0ca740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55d2cddb0ab0 con 0x55d2cdf1bce0 2024-09-11T00:10:04.063 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:04.061+0000 7f196a0ca740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55d2cde1f570 con 0x55d2cdf192a0 2024-09-11T00:10:04.064 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:04.061+0000 7f196a0ca740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55d2cdf1eae0 con 0x55d2cdf16970 2024-09-11T00:10:04.064 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:04.061+0000 7f196a0ca740 10 client.0 _ll_register_callbacks cb 0x55d2cde2e100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:10:04.066 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:04.061+0000 7f1965200640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d2cdf1bce0 0x55d2cdf21170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:10:04.067 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:04.061+0000 7f1965200640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d2cdf1bce0 0x55d2cdf21170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:50200/0 (socket says 172.21.15.12:50200) 2024-09-11T00:10:04.067 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:04.061+0000 7f1965200640 1 -- 172.21.15.12:0/1403613223 learned_addr learned my addr 172.21.15.12:0/1403613223 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:10:04.067 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:04.061+0000 7f195fe00640 1 --2- 172.21.15.12:0/1403613223 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d2cdf16970 0x55d2cdf1ff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:10:04.067 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:04.061+0000 7f195f400640 1 --2- 172.21.15.12:0/1403613223 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d2cdf192a0 0x55d2cdf205a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:10:04.067 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:04.061+0000 7f1965200640 1 -- 172.21.15.12:0/1403613223 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d2cdf16970 msgr2=0x55d2cdf1ff00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:10:04.067 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:04.061+0000 7f1965200640 1 --2- 172.21.15.12:0/1403613223 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d2cdf16970 0x55d2cdf1ff00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:10:04.067 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:04.061+0000 7f1965200640 1 -- 172.21.15.12:0/1403613223 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d2cdf192a0 msgr2=0x55d2cdf205a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:10:04.067 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:04.061+0000 7f1965200640 1 --2- 172.21.15.12:0/1403613223 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d2cdf192a0 0x55d2cdf205a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:10:04.067 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:04.061+0000 7f1965200640 1 -- 172.21.15.12:0/1403613223 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d2cdbda790 con 0x55d2cdf1bce0 2024-09-11T00:10:04.067 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:04.062+0000 7f1965200640 1 --2- 172.21.15.12:0/1403613223 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d2cdf1bce0 0x55d2cdf21170 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f195802ade0 tx=0x7f1958004690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=512106f4da1cbba3 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:10:04.067 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:04.062+0000 7f194a000640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-09-11T00:10:04.068 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:04.062+0000 7f194a000640 1 -- 172.21.15.12:0/1403613223 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f195804a040 con 0x55d2cdf1bce0 2024-09-11T00:10:04.068 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:04.062+0000 7f194a000640 1 -- 172.21.15.12:0/1403613223 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9208+0+0 (secure 0 0 0) 0x7f1958003b70 con 0x55d2cdf1bce0 2024-09-11T00:10:04.068 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:04.062+0000 7f196a0ca740 1 -- 172.21.15.12:0/1403613223 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1958036030 con 0x55d2cdf1bce0 2024-09-11T00:10:04.068 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:04.062+0000 7f194a000640 1 -- 172.21.15.12:0/1403613223 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f195805e050 con 0x55d2cdf1bce0 2024-09-11T00:10:04.068 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:04.062+0000 7f196a0ca740 10 client.15855 fetch_fsmap learned FSMap version 33 2024-09-11T00:10:04.068 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:04.062+0000 7f196a0ca740 1 -- 172.21.15.12:0/1403613223 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55d2cdc53620 con 0x55d2cdf1bce0 2024-09-11T00:10:04.068 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:04.063+0000 7f194a000640 1 -- 172.21.15.12:0/1403613223 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1958036030 con 0x55d2cdf1bce0 2024-09-11T00:10:04.068 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:04.063+0000 7f194a000640 1 -- 172.21.15.12:0/1403613223 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 33) ==== 8386+0+0 (secure 0 0 0) 0x7f195804d030 con 0x55d2cdf1bce0 2024-09-11T00:10:04.068 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:04.063+0000 7f196a0ca740 10 client.15855 fetch_fsmap finished waiting for FSMap version 33 2024-09-11T00:10:04.068 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:04.063+0000 7f196a0ca740 10 client.15855 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T00:10:04.068 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:04.063+0000 7f196a0ca740 20 client.15855 populate_metadata read hostname 'smithi012' 2024-09-11T00:10:04.069 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:04.063+0000 7f196a0ca740 1 --2- 172.21.15.12:0/1403613223 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55d2cdf2b590 0x55d2cdf2d980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:10:04.069 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:04.063+0000 7f196a0ca740 4 client.15855 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T00:10:04.069 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:04.063+0000 7f196a0ca740 1 -- 172.21.15.12:0/1403613223 --> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d2cde8b600 con 0x55d2cdf2b590 2024-09-11T00:10:04.069 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:04.063+0000 7f195fe00640 1 --2- 172.21.15.12:0/1403613223 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55d2cdf2b590 0x55d2cdf2d980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:10:04.069 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:04.064+0000 7f195fe00640 1 --2- 172.21.15.12:0/1403613223 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55d2cdf2b590 0x55d2cdf2d980 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=272956c6795b5459 server_cookie=cb02ee9daef6f2c1 in_seq=0 out_seq=0 2024-09-11T00:10:04.069 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:04.064+0000 7f194a000640 10 client.15855 ms_handle_connect on v2:172.21.15.165:6838/501312984 2024-09-11T00:10:04.069 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:04.065+0000 7f194a000640 1 -- 172.21.15.12:0/1403613223 <== mds.0 v2:172.21.15.165:6838/501312984 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55d2cde8b600 con 0x55d2cdf2b590 2024-09-11T00:10:04.069 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:04.065+0000 7f194a000640 10 client.15855 handle_command_reply: tid=0 2024-09-11T00:10:04.069 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:04.065+0000 7f196a0ca740 1 client.15855 shutdown 2024-09-11T00:10:04.069 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:04.065+0000 7f196a0ca740 1 -- 172.21.15.12:0/1403613223 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d2cdf1bce0 msgr2=0x55d2cdf21170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:10:04.069 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:04.065+0000 7f196a0ca740 1 --2- 172.21.15.12:0/1403613223 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d2cdf1bce0 0x55d2cdf21170 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f195802ade0 tx=0x7f1958004690 comp rx=0 tx=0).stop 2024-09-11T00:10:04.070 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:04.065+0000 7f196a0ca740 1 -- 172.21.15.12:0/1403613223 shutdown_connections 2024-09-11T00:10:04.070 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:04.065+0000 7f196a0ca740 1 --2- 172.21.15.12:0/1403613223 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55d2cdf2b590 0x55d2cdf2d980 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:10:04.070 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:04.065+0000 7f196a0ca740 1 --2- 172.21.15.12:0/1403613223 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d2cdf1bce0 0x55d2cdf21170 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:10:04.070 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:04.065+0000 7f196a0ca740 1 --2- 172.21.15.12:0/1403613223 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d2cdf192a0 0x55d2cdf205a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:10:04.070 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:04.065+0000 7f196a0ca740 1 --2- 172.21.15.12:0/1403613223 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d2cdf16970 0x55d2cdf1ff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:10:04.070 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:04.065+0000 7f196a0ca740 1 -- 172.21.15.12:0/1403613223 >> 172.21.15.12:0/1403613223 conn(0x55d2cdf11ac0 msgr2=0x55d2cdf10bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:10:04.070 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:04.065+0000 7f194a000640 0 client.15855 ms_handle_reset on v2:172.21.15.165:6838/501312984 2024-09-11T00:10:04.070 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:04.065+0000 7f196a0ca740 1 -- 172.21.15.12:0/1403613223 shutdown_connections 2024-09-11T00:10:04.070 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:04.065+0000 7f196a0ca740 1 -- 172.21.15.12:0/1403613223 wait complete. 2024-09-11T00:10:04.070 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:04.065+0000 7f196a0ca740 20 client.15855 trim_cache size 0 max 16384 2024-09-11T00:10:04.082 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:04.081+0000 7f196a0ca740 1 Processor -- start 2024-09-11T00:10:04.082 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:04.081+0000 7f196a0ca740 1 -- start start 2024-09-11T00:10:04.082 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:04.081+0000 7f196a0ca740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d2cdf16970 0x55d2cdf1c7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:10:04.082 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:04.081+0000 7f196a0ca740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d2cdf192a0 0x55d2cdf1f0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:10:04.082 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:04.081+0000 7f196a0ca740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d2cdf1bce0 0x55d2cdf113b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:10:04.082 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:04.081+0000 7f196a0ca740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f19601ad540 con 0x55d2cdf16970 2024-09-11T00:10:04.082 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:04.081+0000 7f196a0ca740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f19601af140 con 0x55d2cdf1bce0 2024-09-11T00:10:04.082 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:04.081+0000 7f196a0ca740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f19601adc10 con 0x55d2cdf192a0 2024-09-11T00:10:04.084 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:04.081+0000 7f195fe00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d2cdf16970 0x55d2cdf1c7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:10:04.084 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:04.081+0000 7f195fe00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d2cdf16970 0x55d2cdf1c7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:50212/0 (socket says 172.21.15.12:50212) 2024-09-11T00:10:04.085 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:04.081+0000 7f195fe00640 1 -- 172.21.15.12:0/50786950 learned_addr learned my addr 172.21.15.12:0/50786950 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:10:04.085 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:04.082+0000 7f195f400640 1 --2- 172.21.15.12:0/50786950 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d2cdf192a0 0x55d2cdf1f0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:10:04.085 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:04.082+0000 7f1965200640 1 --2- 172.21.15.12:0/50786950 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d2cdf1bce0 0x55d2cdf113b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:10:04.085 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:04.082+0000 7f195fe00640 1 -- 172.21.15.12:0/50786950 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d2cdf192a0 msgr2=0x55d2cdf1f0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:10:04.085 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:04.082+0000 7f195fe00640 1 --2- 172.21.15.12:0/50786950 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d2cdf192a0 0x55d2cdf1f0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:10:04.085 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:04.082+0000 7f195fe00640 1 -- 172.21.15.12:0/50786950 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d2cdf1bce0 msgr2=0x55d2cdf113b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:10:04.085 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:04.082+0000 7f195fe00640 1 --2- 172.21.15.12:0/50786950 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d2cdf1bce0 0x55d2cdf113b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:10:04.085 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:04.082+0000 7f195fe00640 1 -- 172.21.15.12:0/50786950 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d2cdf20910 con 0x55d2cdf16970 2024-09-11T00:10:04.085 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:04.082+0000 7f1965200640 1 --2- 172.21.15.12:0/50786950 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d2cdf1bce0 0x55d2cdf113b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T00:10:04.085 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:04.082+0000 7f195fe00640 1 --2- 172.21.15.12:0/50786950 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d2cdf16970 0x55d2cdf1c7d0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f195000cfd0 tx=0x7f1950004a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af71d2b1f5babe9b server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:10:04.085 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:04.082+0000 7f1954c00640 1 -- 172.21.15.12:0/50786950 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19500d3070 con 0x55d2cdf16970 2024-09-11T00:10:04.086 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:04.082+0000 7f1954c00640 1 -- 172.21.15.12:0/50786950 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f19500ac280 con 0x55d2cdf16970 2024-09-11T00:10:04.086 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:04.082+0000 7f1954c00640 1 -- 172.21.15.12:0/50786950 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19500046c0 con 0x55d2cdf16970 2024-09-11T00:10:04.086 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:04.083+0000 7f196a0ca740 1 -- 172.21.15.12:0/50786950 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d2cdf16970 msgr2=0x55d2cdf1c7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:10:04.086 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:04.083+0000 7f196a0ca740 1 --2- 172.21.15.12:0/50786950 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d2cdf16970 0x55d2cdf1c7d0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f195000cfd0 tx=0x7f1950004a90 comp rx=0 tx=0).stop 2024-09-11T00:10:04.086 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:04.083+0000 7f196a0ca740 1 -- 172.21.15.12:0/50786950 shutdown_connections 2024-09-11T00:10:04.086 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:04.083+0000 7f196a0ca740 1 --2- 172.21.15.12:0/50786950 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d2cdf1bce0 0x55d2cdf113b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:10:04.086 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:04.083+0000 7f196a0ca740 1 --2- 172.21.15.12:0/50786950 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d2cdf192a0 0x55d2cdf1f0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:10:04.086 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:04.083+0000 7f196a0ca740 1 --2- 172.21.15.12:0/50786950 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d2cdf16970 0x55d2cdf1c7d0 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:10:04.086 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:04.083+0000 7f196a0ca740 1 -- 172.21.15.12:0/50786950 >> 172.21.15.12:0/50786950 conn(0x55d2cdf11ac0 msgr2=0x55d2cdf1c0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:10:04.086 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:04.083+0000 7f196a0ca740 1 -- 172.21.15.12:0/50786950 shutdown_connections 2024-09-11T00:10:04.086 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:04.083+0000 7f196a0ca740 1 -- 172.21.15.12:0/50786950 wait complete. 2024-09-11T00:10:04.086 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:04.083+0000 7f196a0ca740 1 -- start start 2024-09-11T00:10:04.086 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:04.084+0000 7f196a0ca740 1 Processor -- start 2024-09-11T00:10:04.089 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:04.084+0000 7f196a0ca740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d2cdf16970 0x55d2cdf27290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:10:04.089 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:04.084+0000 7f196a0ca740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d2cdf192a0 0x55d2cdf27870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:10:04.089 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:04.084+0000 7f196a0ca740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d2cdf1bce0 0x55d2cdf25b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:10:04.089 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:04.084+0000 7f196a0ca740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f19601aea70 con 0x55d2cdf16970 2024-09-11T00:10:04.090 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:04.084+0000 7f196a0ca740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f19601ae2e0 con 0x55d2cdf192a0 2024-09-11T00:10:04.090 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:04.084+0000 7f196a0ca740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f19601ab520 con 0x55d2cdf1bce0 2024-09-11T00:10:04.090 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:04.084+0000 7f196a0ca740 10 client.0 _ll_register_callbacks cb 0x7f194c009b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:10:04.090 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:04.084+0000 7f195fe00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d2cdf16970 0x55d2cdf27290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:10:04.090 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:04.084+0000 7f195fe00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d2cdf16970 0x55d2cdf27290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:50220/0 (socket says 172.21.15.12:50220) 2024-09-11T00:10:04.090 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:04.084+0000 7f195fe00640 1 -- 172.21.15.12:0/2495375619 learned_addr learned my addr 172.21.15.12:0/2495375619 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:10:04.090 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:04.084+0000 7f1965200640 1 --2- 172.21.15.12:0/2495375619 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d2cdf1bce0 0x55d2cdf25b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:10:04.090 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:04.085+0000 7f195f400640 1 --2- 172.21.15.12:0/2495375619 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d2cdf192a0 0x55d2cdf27870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:10:04.090 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:04.085+0000 7f195fe00640 1 -- 172.21.15.12:0/2495375619 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d2cdf1bce0 msgr2=0x55d2cdf25b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:10:04.090 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:04.085+0000 7f195fe00640 1 --2- 172.21.15.12:0/2495375619 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d2cdf1bce0 0x55d2cdf25b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:10:04.090 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:04.085+0000 7f195fe00640 1 -- 172.21.15.12:0/2495375619 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d2cdf192a0 msgr2=0x55d2cdf27870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:10:04.090 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:04.085+0000 7f195fe00640 1 --2- 172.21.15.12:0/2495375619 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d2cdf192a0 0x55d2cdf27870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:10:04.091 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:04.085+0000 7f195fe00640 1 -- 172.21.15.12:0/2495375619 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d2cdf190d0 con 0x55d2cdf16970 2024-09-11T00:10:04.091 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:04.085+0000 7f195fe00640 1 --2- 172.21.15.12:0/2495375619 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d2cdf16970 0x55d2cdf27290 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f1950004a40 tx=0x7f19500e66f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b3fdf792fd33a3b server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:10:04.091 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:04.085+0000 7f194a000640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-09-11T00:10:04.091 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:04.085+0000 7f194a000640 1 -- 172.21.15.12:0/2495375619 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19500d3070 con 0x55d2cdf16970 2024-09-11T00:10:04.091 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:04.085+0000 7f194a000640 1 -- 172.21.15.12:0/2495375619 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9208+0+0 (secure 0 0 0) 0x7f19500051b0 con 0x55d2cdf16970 2024-09-11T00:10:04.091 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:04.085+0000 7f196a0ca740 1 -- 172.21.15.12:0/2495375619 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f19500ac280 con 0x55d2cdf16970 2024-09-11T00:10:04.091 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:04.085+0000 7f194a000640 1 -- 172.21.15.12:0/2495375619 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f19500ac770 con 0x55d2cdf16970 2024-09-11T00:10:04.091 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:04.085+0000 7f196a0ca740 10 client.15861 fetch_fsmap learned FSMap version 33 2024-09-11T00:10:04.091 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:04.085+0000 7f196a0ca740 1 -- 172.21.15.12:0/2495375619 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55d2cdf98c20 con 0x55d2cdf16970 2024-09-11T00:10:04.091 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:04.086+0000 7f194a000640 1 -- 172.21.15.12:0/2495375619 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f19500ac280 con 0x55d2cdf16970 2024-09-11T00:10:04.091 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:04.086+0000 7f194a000640 1 -- 172.21.15.12:0/2495375619 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 33) ==== 8386+0+0 (secure 0 0 0) 0x7f19500c3070 con 0x55d2cdf16970 2024-09-11T00:10:04.091 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:04.086+0000 7f196a0ca740 10 client.15861 fetch_fsmap finished waiting for FSMap version 33 2024-09-11T00:10:04.091 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:04.086+0000 7f196a0ca740 10 client.15861 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T00:10:04.092 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:04.086+0000 7f196a0ca740 20 client.15861 populate_metadata read hostname 'smithi012' 2024-09-11T00:10:04.092 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:04.086+0000 7f196a0ca740 1 --2- 172.21.15.12:0/2495375619 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55d2cdf99630 0x55d2cdf9ba50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:10:04.092 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:04.086+0000 7f196a0ca740 4 client.15861 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T00:10:04.092 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:04.086+0000 7f196a0ca740 1 -- 172.21.15.12:0/2495375619 --> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55d2cde870b0 con 0x55d2cdf99630 2024-09-11T00:10:04.092 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:04.086+0000 7f195f400640 1 --2- 172.21.15.12:0/2495375619 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55d2cdf99630 0x55d2cdf9ba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:10:04.092 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:04.087+0000 7f195f400640 1 --2- 172.21.15.12:0/2495375619 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55d2cdf99630 0x55d2cdf9ba50 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a918a9767fdb5a65 server_cookie=2d60790d7bd5983f in_seq=0 out_seq=0 2024-09-11T00:10:04.092 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:04.087+0000 7f194a000640 10 client.15861 ms_handle_connect on v2:172.21.15.165:6838/501312984 2024-09-11T00:10:04.111 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:04.110+0000 7f194a000640 1 -- 172.21.15.12:0/2495375619 <== mds.0 v2:172.21.15.165:6838/501312984 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55d2cde870b0 con 0x55d2cdf99630 2024-09-11T00:10:04.111 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:04.110+0000 7f194a000640 10 client.15861 handle_command_reply: tid=0 2024-09-11T00:10:04.111 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:04.110+0000 7f196a0ca740 1 client.15861 shutdown 2024-09-11T00:10:04.112 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:04.110+0000 7f196a0ca740 1 -- 172.21.15.12:0/2495375619 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d2cdf16970 msgr2=0x55d2cdf27290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:10:04.112 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:04.110+0000 7f196a0ca740 1 --2- 172.21.15.12:0/2495375619 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d2cdf16970 0x55d2cdf27290 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f1950004a40 tx=0x7f19500e66f0 comp rx=0 tx=0).stop 2024-09-11T00:10:04.112 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:04.110+0000 7f196a0ca740 1 -- 172.21.15.12:0/2495375619 shutdown_connections 2024-09-11T00:10:04.112 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:04.110+0000 7f196a0ca740 1 --2- 172.21.15.12:0/2495375619 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55d2cdf99630 0x55d2cdf9ba50 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:10:04.112 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:04.110+0000 7f196a0ca740 1 --2- 172.21.15.12:0/2495375619 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d2cdf1bce0 0x55d2cdf25b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:10:04.112 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:04.110+0000 7f194a000640 0 client.15861 ms_handle_reset on v2:172.21.15.165:6838/501312984 2024-09-11T00:10:04.112 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:04.110+0000 7f196a0ca740 1 --2- 172.21.15.12:0/2495375619 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d2cdf192a0 0x55d2cdf27870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:10:04.112 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:04.110+0000 7f196a0ca740 1 --2- 172.21.15.12:0/2495375619 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d2cdf16970 0x55d2cdf27290 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:10:04.112 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:04.110+0000 7f196a0ca740 1 -- 172.21.15.12:0/2495375619 >> 172.21.15.12:0/2495375619 conn(0x55d2cdf11ac0 msgr2=0x55d2cdf16250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:10:04.112 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:04.110+0000 7f196a0ca740 1 -- 172.21.15.12:0/2495375619 shutdown_connections 2024-09-11T00:10:04.112 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:04.110+0000 7f196a0ca740 1 -- 172.21.15.12:0/2495375619 wait complete. 2024-09-11T00:10:04.112 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:04.110+0000 7f196a0ca740 20 client.15861 trim_cache size 0 max 16384 2024-09-11T00:10:04.113 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:{ 2024-09-11T00:10:04.113 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "message": "", 2024-09-11T00:10:04.113 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "return_code": 0 2024-09-11T00:10:04.113 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:} 2024-09-11T00:10:04.114 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:04.111+0000 7f1965c00640 1 -- 172.21.15.12:0/4049962597 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f19440b5280 msgr2=0x7f19440b7740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:10:04.114 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:04.111+0000 7f1965c00640 1 --2- 172.21.15.12:0/4049962597 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f19440b5280 0x7f19440b7740 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f194c012580 tx=0x7f194c005f90 comp rx=0 tx=0).stop 2024-09-11T00:10:04.114 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:04.111+0000 7f1965c00640 1 -- 172.21.15.12:0/4049962597 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f196006b470 msgr2=0x7f1960102aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:10:04.114 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:04.111+0000 7f1965c00640 1 --2- 172.21.15.12:0/4049962597 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f196006b470 0x7f1960102aa0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f195000e9f0 tx=0x7f195000edf0 comp rx=0 tx=0).stop 2024-09-11T00:10:04.114 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:04.111+0000 7f1965c00640 1 -- 172.21.15.12:0/4049962597 shutdown_connections 2024-09-11T00:10:04.114 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:04.111+0000 7f1965c00640 1 --2- 172.21.15.12:0/4049962597 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f19440b5280 0x7f19440b7740 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:10:04.114 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:04.111+0000 7f1965c00640 1 --2- 172.21.15.12:0/4049962597 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f19601084a0 0x7f1960101840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:10:04.114 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:04.111+0000 7f1965c00640 1 --2- 172.21.15.12:0/4049962597 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f19601058f0 0x7f1960101300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:10:04.114 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:04.111+0000 7f1965c00640 1 --2- 172.21.15.12:0/4049962597 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f196006b470 0x7f1960102aa0 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:10:04.114 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:04.111+0000 7f1965c00640 1 -- 172.21.15.12:0/4049962597 >> 172.21.15.12:0/4049962597 conn(0x7f19600fcf30 msgr2=0x7f1960103bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:10:04.114 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:04.111+0000 7f1965c00640 1 -- 172.21.15.12:0/4049962597 shutdown_connections 2024-09-11T00:10:04.115 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:04.111+0000 7f1965c00640 1 -- 172.21.15.12:0/4049962597 wait complete. 2024-09-11T00:10:04.610 INFO:tasks.workunit.client.0.smithi012.stdout: 1 96719 0.32 MB/sec execute 1 sec latency 4944.174 ms 2024-09-11T00:10:05.610 INFO:tasks.workunit.client.0.smithi012.stdout: 1 99433 8.62 MB/sec execute 2 sec latency 10.470 ms 2024-09-11T00:10:05.782 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:10:05 smithi165 ceph-mon[35356]: pgmap v489: 129 pgs: 129 active+clean; 25 MiB data, 551 MiB used, 1.0 TiB / 1.0 TiB avail; 154 KiB/s rd, 247 KiB/s wr, 52 op/s 2024-09-11T00:10:05.812 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:10:05 smithi012 ceph-mon[30333]: pgmap v489: 129 pgs: 129 active+clean; 25 MiB data, 551 MiB used, 1.0 TiB / 1.0 TiB avail; 154 KiB/s rd, 247 KiB/s wr, 52 op/s 2024-09-11T00:10:05.830 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:10:05 smithi178 ceph-mon[35919]: pgmap v489: 129 pgs: 129 active+clean; 25 MiB data, 551 MiB used, 1.0 TiB / 1.0 TiB avail; 154 KiB/s rd, 247 KiB/s wr, 52 op/s 2024-09-11T00:10:06.610 INFO:tasks.workunit.client.0.smithi012.stdout: 1 100152 6.38 MB/sec execute 3 sec latency 285.382 ms 2024-09-11T00:10:07.610 INFO:tasks.workunit.client.0.smithi012.stdout: 1 100152 4.79 MB/sec execute 4 sec latency 1285.488 ms 2024-09-11T00:10:07.615 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:10:07 smithi012 ceph-mon[30333]: pgmap v490: 129 pgs: 129 active+clean; 25 MiB data, 536 MiB used, 1.0 TiB / 1.0 TiB avail; 169 KiB/s rd, 271 KiB/s wr, 63 op/s 2024-09-11T00:10:07.633 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:10:07 smithi178 ceph-mon[35919]: pgmap v490: 129 pgs: 129 active+clean; 25 MiB data, 536 MiB used, 1.0 TiB / 1.0 TiB avail; 169 KiB/s rd, 271 KiB/s wr, 63 op/s 2024-09-11T00:10:07.782 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:10:07 smithi165 ceph-mon[35356]: pgmap v490: 129 pgs: 129 active+clean; 25 MiB data, 536 MiB used, 1.0 TiB / 1.0 TiB avail; 169 KiB/s rd, 271 KiB/s wr, 63 op/s 2024-09-11T00:10:08.610 INFO:tasks.workunit.client.0.smithi012.stdout: 1 100152 3.83 MB/sec execute 5 sec latency 2285.638 ms 2024-09-11T00:10:09.610 INFO:tasks.workunit.client.0.smithi012.stdout: 1 100269 3.24 MB/sec execute 6 sec latency 3213.846 ms 2024-09-11T00:10:09.782 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:10:09 smithi165 ceph-mon[35356]: pgmap v491: 129 pgs: 129 active+clean; 26 MiB data, 536 MiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 123 KiB/s wr, 45 op/s 2024-09-11T00:10:09.813 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:10:09 smithi012 ceph-mon[30333]: pgmap v491: 129 pgs: 129 active+clean; 26 MiB data, 536 MiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 123 KiB/s wr, 45 op/s 2024-09-11T00:10:09.830 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:10:09 smithi178 ceph-mon[35919]: pgmap v491: 129 pgs: 129 active+clean; 26 MiB data, 536 MiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 123 KiB/s wr, 45 op/s 2024-09-11T00:10:10.610 INFO:tasks.workunit.client.0.smithi012.stdout: 1 102888 5.17 MB/sec execute 7 sec latency 9.748 ms 2024-09-11T00:10:11.611 INFO:tasks.workunit.client.0.smithi012.stdout: 1 103576 4.75 MB/sec execute 8 sec latency 357.855 ms 2024-09-11T00:10:11.782 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:10:11 smithi165 ceph-mon[35356]: pgmap v492: 129 pgs: 129 active+clean; 27 MiB data, 556 MiB used, 1.0 TiB / 1.0 TiB avail; 181 KiB/s rd, 211 KiB/s wr, 58 op/s 2024-09-11T00:10:11.813 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:10:11 smithi012 ceph-mon[30333]: pgmap v492: 129 pgs: 129 active+clean; 27 MiB data, 556 MiB used, 1.0 TiB / 1.0 TiB avail; 181 KiB/s rd, 211 KiB/s wr, 58 op/s 2024-09-11T00:10:11.830 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:10:11 smithi178 ceph-mon[35919]: pgmap v492: 129 pgs: 129 active+clean; 27 MiB data, 556 MiB used, 1.0 TiB / 1.0 TiB avail; 181 KiB/s rd, 211 KiB/s wr, 58 op/s 2024-09-11T00:10:12.611 INFO:tasks.workunit.client.0.smithi012.stdout: 1 103576 4.22 MB/sec execute 9 sec latency 1357.996 ms 2024-09-11T00:10:13.610 INFO:tasks.workunit.client.0.smithi012.stdout: 1 103576 3.80 MB/sec execute 10 sec latency 2358.062 ms 2024-09-11T00:10:13.782 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:10:13 smithi165 ceph-mon[35356]: pgmap v493: 129 pgs: 129 active+clean; 27 MiB data, 576 MiB used, 1.0 TiB / 1.0 TiB avail; 235 KiB/s rd, 259 KiB/s wr, 75 op/s 2024-09-11T00:10:13.782 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:10:13 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T00:10:13.813 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:10:13 smithi012 ceph-mon[30333]: pgmap v493: 129 pgs: 129 active+clean; 27 MiB data, 576 MiB used, 1.0 TiB / 1.0 TiB avail; 235 KiB/s rd, 259 KiB/s wr, 75 op/s 2024-09-11T00:10:13.813 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:10:13 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T00:10:13.830 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:10:13 smithi178 ceph-mon[35919]: pgmap v493: 129 pgs: 129 active+clean; 27 MiB data, 576 MiB used, 1.0 TiB / 1.0 TiB avail; 235 KiB/s rd, 259 KiB/s wr, 75 op/s 2024-09-11T00:10:13.830 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:10:13 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T00:10:14.611 INFO:tasks.workunit.client.0.smithi012.stdout: 1 103649 3.47 MB/sec execute 11 sec latency 3285.093 ms 2024-09-11T00:10:14.782 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:10:14 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:10:14.782 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:10:14 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T00:10:14.782 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:10:14 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:10:14.812 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:10:14 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:10:14.813 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:10:14 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T00:10:14.813 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:10:14 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:10:14.830 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:10:14 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:10:14.830 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:10:14 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T00:10:14.830 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:10:14 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:10:15.611 INFO:tasks.workunit.client.0.smithi012.stdout: 1 105930 4.20 MB/sec execute 12 sec latency 10.773 ms 2024-09-11T00:10:15.782 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:10:15 smithi165 ceph-mon[35356]: pgmap v494: 129 pgs: 129 active+clean; 27 MiB data, 576 MiB used, 1.0 TiB / 1.0 TiB avail; 226 KiB/s rd, 251 KiB/s wr, 69 op/s 2024-09-11T00:10:15.812 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:10:15 smithi012 ceph-mon[30333]: pgmap v494: 129 pgs: 129 active+clean; 27 MiB data, 576 MiB used, 1.0 TiB / 1.0 TiB avail; 226 KiB/s rd, 251 KiB/s wr, 69 op/s 2024-09-11T00:10:15.830 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:10:15 smithi178 ceph-mon[35919]: pgmap v494: 129 pgs: 129 active+clean; 27 MiB data, 576 MiB used, 1.0 TiB / 1.0 TiB avail; 226 KiB/s rd, 251 KiB/s wr, 69 op/s 2024-09-11T00:10:16.611 INFO:tasks.workunit.client.0.smithi012.stdout: 1 107173 4.40 MB/sec execute 13 sec latency 138.220 ms 2024-09-11T00:10:17.235 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:17.233+0000 7f43a4800640 1 -- 172.21.15.12:0/1797621263 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f439c105cf0 msgr2=0x7f439c106170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:10:17.236 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:17.233+0000 7f43a4800640 1 --2- 172.21.15.12:0/1797621263 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f439c105cf0 0x7f439c106170 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f438c009890 tx=0x7f438c02a360 comp rx=0 tx=0).stop 2024-09-11T00:10:17.236 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:17.233+0000 7f43a4800640 1 -- 172.21.15.12:0/1797621263 shutdown_connections 2024-09-11T00:10:17.236 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:17.233+0000 7f43a4800640 1 --2- 172.21.15.12:0/1797621263 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f439c1066b0 0x7f439c10d030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:10:17.236 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:17.233+0000 7f43a4800640 1 --2- 172.21.15.12:0/1797621263 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f439c105cf0 0x7f439c106170 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:10:17.236 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:17.233+0000 7f43a4800640 1 --2- 172.21.15.12:0/1797621263 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f439c104ba0 0x7f439c104fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:10:17.236 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:17.233+0000 7f43a4800640 1 -- 172.21.15.12:0/1797621263 >> 172.21.15.12:0/1797621263 conn(0x7f439c100460 msgr2=0x7f439c102880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:10:17.236 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:17.233+0000 7f43a4800640 1 -- 172.21.15.12:0/1797621263 shutdown_connections 2024-09-11T00:10:17.236 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:17.233+0000 7f43a4800640 1 -- 172.21.15.12:0/1797621263 wait complete. 2024-09-11T00:10:17.238 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:17.234+0000 7f43a4800640 1 Processor -- start 2024-09-11T00:10:17.238 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:17.234+0000 7f43a4800640 1 -- start start 2024-09-11T00:10:17.238 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:17.234+0000 7f43a4800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f439c104ba0 0x7f439c19afa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:10:17.238 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:17.234+0000 7f43a4800640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f439c105cf0 0x7f439c19b5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:10:17.238 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:17.234+0000 7f43a4800640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f439c1066b0 0x7f439c19c220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:10:17.238 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:17.234+0000 7f43a4800640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f439c110810 con 0x7f439c1066b0 2024-09-11T00:10:17.238 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:17.234+0000 7f43a4800640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f439c110690 con 0x7f439c104ba0 2024-09-11T00:10:17.239 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:17.234+0000 7f43a4800640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f439c110990 con 0x7f439c105cf0 2024-09-11T00:10:17.239 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:17.235+0000 7f43a1400640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f439c105cf0 0x7f439c19b5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:10:17.239 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:17.235+0000 7f43a2800640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f439c1066b0 0x7f439c19c220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:10:17.239 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:17.235+0000 7f43a1400640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f439c105cf0 0x7f439c19b5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.12:37488/0 (socket says 172.21.15.12:37488) 2024-09-11T00:10:17.239 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:17.235+0000 7f43a1400640 1 -- 172.21.15.12:0/75769782 learned_addr learned my addr 172.21.15.12:0/75769782 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:10:17.239 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:17.235+0000 7f43a1e00640 1 --2- 172.21.15.12:0/75769782 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f439c104ba0 0x7f439c19afa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:10:17.239 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:17.235+0000 7f43a1400640 1 -- 172.21.15.12:0/75769782 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f439c104ba0 msgr2=0x7f439c19afa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:10:17.239 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:17.235+0000 7f43a1400640 1 --2- 172.21.15.12:0/75769782 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f439c104ba0 0x7f439c19afa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:10:17.239 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:17.235+0000 7f43a1400640 1 -- 172.21.15.12:0/75769782 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f439c1066b0 msgr2=0x7f439c19c220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:10:17.239 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:17.235+0000 7f43a1400640 1 --2- 172.21.15.12:0/75769782 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f439c1066b0 0x7f439c19c220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:10:17.239 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:17.235+0000 7f43a1400640 1 -- 172.21.15.12:0/75769782 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f439c19cae0 con 0x7f439c105cf0 2024-09-11T00:10:17.239 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:17.236+0000 7f43a1400640 1 --2- 172.21.15.12:0/75769782 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f439c105cf0 0x7f439c19b5f0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f438c0057c0 tx=0x7f438c032d30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:10:17.239 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:17.236+0000 7f4396a00640 1 -- 172.21.15.12:0/75769782 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f438c037040 con 0x7f439c105cf0 2024-09-11T00:10:17.239 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:17.236+0000 7f4396a00640 1 -- 172.21.15.12:0/75769782 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f438c030520 con 0x7f439c105cf0 2024-09-11T00:10:17.240 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:17.236+0000 7f4396a00640 1 -- 172.21.15.12:0/75769782 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f438c005280 con 0x7f439c105cf0 2024-09-11T00:10:17.240 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:17.236+0000 7f43a4800640 1 -- 172.21.15.12:0/75769782 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f439c19cd70 con 0x7f439c105cf0 2024-09-11T00:10:17.240 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:17.236+0000 7f43a4800640 1 -- 172.21.15.12:0/75769782 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f439c1a34a0 con 0x7f439c105cf0 2024-09-11T00:10:17.242 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:17.239+0000 7f43a7669740 1 Processor -- start 2024-09-11T00:10:17.242 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:17.239+0000 7f43a7669740 1 -- start start 2024-09-11T00:10:17.242 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:17.239+0000 7f43a7669740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f4a9597790 0x55f4a9599b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:10:17.242 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:17.239+0000 7f43a7669740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f4a959a0c0 0x55f4a959c4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:10:17.242 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:17.239+0000 7f43a7669740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f4a959cb00 0x55f4a959efc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:10:17.242 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:17.239+0000 7f43a7669740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55f4a950a6b0 con 0x55f4a9597790 2024-09-11T00:10:17.242 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:17.239+0000 7f43a7669740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55f4a94f9ae0 con 0x55f4a959cb00 2024-09-11T00:10:17.243 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:17.239+0000 7f43a7669740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55f4a94fd8f0 con 0x55f4a959a0c0 2024-09-11T00:10:17.243 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:17.240+0000 7f43a2800640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f4a9597790 0x55f4a9599b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:10:17.243 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:17.240+0000 7f43a1e00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f4a959a0c0 0x55f4a959c4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:10:17.243 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:17.240+0000 7f43a2800640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f4a9597790 0x55f4a9599b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:54286/0 (socket says 172.21.15.12:54286) 2024-09-11T00:10:17.243 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:17.240+0000 7f43a1e00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f4a959a0c0 0x55f4a959c4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.12:37504/0 (socket says 172.21.15.12:37504) 2024-09-11T00:10:17.243 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:17.240+0000 7f43a2800640 1 -- 172.21.15.12:0/3146948772 learned_addr learned my addr 172.21.15.12:0/3146948772 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:10:17.243 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:17.240+0000 7f4396a00640 1 -- 172.21.15.12:0/75769782 <== mon.1 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7f438c030b20 con 0x7f439c105cf0 2024-09-11T00:10:17.243 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:17.240+0000 7f43a1e00640 1 -- 172.21.15.12:0/3146948772 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f4a959cb00 msgr2=0x55f4a959efc0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-11T00:10:17.243 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:17.240+0000 7f43a1e00640 1 --2- 172.21.15.12:0/3146948772 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f4a959cb00 0x55f4a959efc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:10:17.243 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:17.240+0000 7f43a1e00640 1 -- 172.21.15.12:0/3146948772 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f4a9597790 msgr2=0x55f4a9599b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:10:17.243 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:17.240+0000 7f43a1e00640 1 --2- 172.21.15.12:0/3146948772 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f4a9597790 0x55f4a9599b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:10:17.243 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:17.240+0000 7f43a1e00640 1 -- 172.21.15.12:0/3146948772 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f4a94c6780 con 0x55f4a959a0c0 2024-09-11T00:10:17.243 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:17.240+0000 7f43a1e00640 1 --2- 172.21.15.12:0/3146948772 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f4a959a0c0 0x55f4a959c4b0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f4390009c00 tx=0x7f4390011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=44c5da129fc9cdfd server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:10:17.246 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:17.240+0000 7f438b000640 1 -- 172.21.15.12:0/3146948772 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4390004230 con 0x55f4a959a0c0 2024-09-11T00:10:17.246 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:17.241+0000 7f438b000640 1 -- 172.21.15.12:0/3146948772 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f43900043d0 con 0x55f4a959a0c0 2024-09-11T00:10:17.246 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:17.241+0000 7f438b000640 1 -- 172.21.15.12:0/3146948772 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4390004c70 con 0x55f4a959a0c0 2024-09-11T00:10:17.246 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:17.241+0000 7f43a7669740 1 -- 172.21.15.12:0/3146948772 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f4a959a0c0 msgr2=0x55f4a959c4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:10:17.246 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:17.241+0000 7f43a7669740 1 --2- 172.21.15.12:0/3146948772 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f4a959a0c0 0x55f4a959c4b0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f4390009c00 tx=0x7f4390011810 comp rx=0 tx=0).stop 2024-09-11T00:10:17.247 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:17.241+0000 7f43a7669740 1 -- 172.21.15.12:0/3146948772 shutdown_connections 2024-09-11T00:10:17.247 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:17.241+0000 7f43a7669740 1 --2- 172.21.15.12:0/3146948772 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f4a959cb00 0x55f4a959efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:10:17.247 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:17.241+0000 7f43a7669740 1 --2- 172.21.15.12:0/3146948772 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f4a959a0c0 0x55f4a959c4b0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:10:17.247 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:17.241+0000 7f43a7669740 1 --2- 172.21.15.12:0/3146948772 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f4a9597790 0x55f4a9599b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:10:17.247 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:17.241+0000 7f43a7669740 1 -- 172.21.15.12:0/3146948772 >> 172.21.15.12:0/3146948772 conn(0x55f4a95928e0 msgr2=0x55f4a9593670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:10:17.247 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:17.241+0000 7f4396a00640 1 --2- 172.21.15.12:0/75769782 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f43840b5360 0x7f43840b7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:10:17.247 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:17.242+0000 7f4396a00640 1 -- 172.21.15.12:0/75769782 <== mon.1 v2:172.21.15.178:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9208+0+0 (secure 0 0 0) 0x7f438c1056c0 con 0x7f439c105cf0 2024-09-11T00:10:17.247 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:17.242+0000 7f43a2800640 1 --2- 172.21.15.12:0/75769782 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f43840b5360 0x7f43840b7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:10:17.247 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:17.242+0000 7f43a7669740 1 -- 172.21.15.12:0/3146948772 shutdown_connections 2024-09-11T00:10:17.247 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:17.242+0000 7f43a7669740 1 -- 172.21.15.12:0/3146948772 wait complete. 2024-09-11T00:10:17.247 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:17.242+0000 7f43a2800640 1 --2- 172.21.15.12:0/75769782 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f43840b5360 0x7f43840b7820 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f4398005fd0 tx=0x7f4398010040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:10:17.247 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:17.242+0000 7f43a7669740 1 -- start start 2024-09-11T00:10:17.247 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:17.243+0000 7f43a7669740 1 Processor -- start 2024-09-11T00:10:17.248 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:17.243+0000 7f43a7669740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f4a9597790 0x55f4a95a0ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:10:17.248 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:17.243+0000 7f43a7669740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f4a959a0c0 0x55f4a95a1590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:10:17.248 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:17.243+0000 7f43a7669740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f4a959cb00 0x55f4a95a2160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:10:17.248 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:17.243+0000 7f43a7669740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55f4a9431ab0 con 0x55f4a959cb00 2024-09-11T00:10:17.248 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:17.243+0000 7f43a7669740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55f4a94a0570 con 0x55f4a9597790 2024-09-11T00:10:17.248 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:17.243+0000 7f43a7669740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55f4a959fae0 con 0x55f4a959a0c0 2024-09-11T00:10:17.248 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:17.244+0000 7f43a7669740 10 client.0 _ll_register_callbacks cb 0x55f4a94af100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:10:17.248 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:17.244+0000 7f43a2800640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f4a959cb00 0x55f4a95a2160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:10:17.248 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:17.244+0000 7f43a2800640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f4a959cb00 0x55f4a95a2160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:54288/0 (socket says 172.21.15.12:54288) 2024-09-11T00:10:17.248 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:17.244+0000 7f43a2800640 1 -- 172.21.15.12:0/2764055231 learned_addr learned my addr 172.21.15.12:0/2764055231 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:10:17.248 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:17.244+0000 7f43a1400640 1 --2- 172.21.15.12:0/2764055231 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f4a959a0c0 0x55f4a95a1590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:10:17.248 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:17.244+0000 7f43a1e00640 1 --2- 172.21.15.12:0/2764055231 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f4a9597790 0x55f4a95a0ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:10:17.249 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:17.244+0000 7f43a2800640 1 -- 172.21.15.12:0/2764055231 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f4a959a0c0 msgr2=0x55f4a95a1590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:10:17.249 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:17.244+0000 7f43a2800640 1 --2- 172.21.15.12:0/2764055231 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f4a959a0c0 0x55f4a95a1590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:10:17.249 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:17.244+0000 7f43a2800640 1 -- 172.21.15.12:0/2764055231 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f4a9597790 msgr2=0x55f4a95a0ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:10:17.249 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:17.244+0000 7f43a2800640 1 --2- 172.21.15.12:0/2764055231 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f4a9597790 0x55f4a95a0ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:10:17.249 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:17.244+0000 7f43a2800640 1 -- 172.21.15.12:0/2764055231 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f4a925b790 con 0x55f4a959cb00 2024-09-11T00:10:17.249 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:17.244+0000 7f43a1400640 1 --2- 172.21.15.12:0/2764055231 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f4a959a0c0 0x55f4a95a1590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:10:17.251 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:17.245+0000 7f43a2800640 1 --2- 172.21.15.12:0/2764055231 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f4a959cb00 0x55f4a95a2160 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f439800dfd0 tx=0x7f4398004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=400154f4aedcf46d server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:10:17.251 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:17.245+0000 7f437f400640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-09-11T00:10:17.251 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:17.245+0000 7f437f400640 1 -- 172.21.15.12:0/2764055231 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4398004b20 con 0x55f4a959cb00 2024-09-11T00:10:17.251 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:17.245+0000 7f43a7669740 1 -- 172.21.15.12:0/2764055231 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f43900043d0 con 0x55f4a959cb00 2024-09-11T00:10:17.251 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:17.245+0000 7f437f400640 1 -- 172.21.15.12:0/2764055231 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9208+0+0 (secure 0 0 0) 0x7f4398004cc0 con 0x55f4a959cb00 2024-09-11T00:10:17.251 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:17.245+0000 7f437f400640 1 -- 172.21.15.12:0/2764055231 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f4398031050 con 0x55f4a959cb00 2024-09-11T00:10:17.252 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:17.245+0000 7f43a7669740 10 client.15873 fetch_fsmap learned FSMap version 33 2024-09-11T00:10:17.252 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:17.245+0000 7f43a7669740 1 -- 172.21.15.12:0/2764055231 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55f4a92d4620 con 0x55f4a959cb00 2024-09-11T00:10:17.252 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:17.245+0000 7f437f400640 1 -- 172.21.15.12:0/2764055231 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f43900043d0 con 0x55f4a959cb00 2024-09-11T00:10:17.252 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:17.246+0000 7f437f400640 1 -- 172.21.15.12:0/2764055231 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 33) ==== 8386+0+0 (secure 0 0 0) 0x7f439800b040 con 0x55f4a959cb00 2024-09-11T00:10:17.252 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:17.246+0000 7f43a7669740 10 client.15873 fetch_fsmap finished waiting for FSMap version 33 2024-09-11T00:10:17.252 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:17.246+0000 7f43a7669740 10 client.15873 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T00:10:17.252 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:17.246+0000 7f43a7669740 20 client.15873 populate_metadata read hostname 'smithi012' 2024-09-11T00:10:17.252 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:17.246+0000 7f43a7669740 1 --2- 172.21.15.12:0/2764055231 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55f4a95ac9b0 0x55f4a95aee10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:10:17.252 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:17.246+0000 7f43a7669740 4 client.15873 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T00:10:17.252 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:17.246+0000 7f43a7669740 1 -- 172.21.15.12:0/2764055231 --> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f4a950c600 con 0x55f4a95ac9b0 2024-09-11T00:10:17.252 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:17.246+0000 7f43a1e00640 1 --2- 172.21.15.12:0/2764055231 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55f4a95ac9b0 0x55f4a95aee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:10:17.252 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:17.247+0000 7f43a1e00640 1 --2- 172.21.15.12:0/2764055231 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55f4a95ac9b0 0x55f4a95aee10 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dfd1cb840dcd17e0 server_cookie=df7a3722238e0d32 in_seq=0 out_seq=0 2024-09-11T00:10:17.252 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:17.247+0000 7f437f400640 10 client.15873 ms_handle_connect on v2:172.21.15.165:6838/501312984 2024-09-11T00:10:17.253 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:17.248+0000 7f437f400640 1 -- 172.21.15.12:0/2764055231 <== mds.0 v2:172.21.15.165:6838/501312984 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55f4a950c600 con 0x55f4a95ac9b0 2024-09-11T00:10:17.253 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:17.248+0000 7f437f400640 10 client.15873 handle_command_reply: tid=0 2024-09-11T00:10:17.253 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:17.248+0000 7f43a7669740 1 client.15873 shutdown 2024-09-11T00:10:17.253 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:17.248+0000 7f43a7669740 1 -- 172.21.15.12:0/2764055231 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f4a959cb00 msgr2=0x55f4a95a2160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:10:17.253 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:17.248+0000 7f43a7669740 1 --2- 172.21.15.12:0/2764055231 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f4a959cb00 0x55f4a95a2160 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f439800dfd0 tx=0x7f4398004ab0 comp rx=0 tx=0).stop 2024-09-11T00:10:17.253 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:17.248+0000 7f43a7669740 1 -- 172.21.15.12:0/2764055231 shutdown_connections 2024-09-11T00:10:17.253 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:17.248+0000 7f43a7669740 1 --2- 172.21.15.12:0/2764055231 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55f4a95ac9b0 0x55f4a95aee10 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:10:17.253 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:17.248+0000 7f43a7669740 1 --2- 172.21.15.12:0/2764055231 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f4a959cb00 0x55f4a95a2160 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:10:17.253 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:17.248+0000 7f43a7669740 1 --2- 172.21.15.12:0/2764055231 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f4a959a0c0 0x55f4a95a1590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:10:17.253 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:17.248+0000 7f43a7669740 1 --2- 172.21.15.12:0/2764055231 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f4a9597790 0x55f4a95a0ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:10:17.253 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:17.248+0000 7f43a7669740 1 -- 172.21.15.12:0/2764055231 >> 172.21.15.12:0/2764055231 conn(0x55f4a95928e0 msgr2=0x55f4a9591a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:10:17.253 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:17.248+0000 7f437f400640 0 client.15873 ms_handle_reset on v2:172.21.15.165:6838/501312984 2024-09-11T00:10:17.253 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:17.248+0000 7f43a7669740 1 -- 172.21.15.12:0/2764055231 shutdown_connections 2024-09-11T00:10:17.254 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:17.248+0000 7f43a7669740 1 -- 172.21.15.12:0/2764055231 wait complete. 2024-09-11T00:10:17.254 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:17.248+0000 7f43a7669740 20 client.15873 trim_cache size 0 max 16384 2024-09-11T00:10:17.266 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:17.264+0000 7f43a7669740 1 Processor -- start 2024-09-11T00:10:17.266 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:17.264+0000 7f43a7669740 1 -- start start 2024-09-11T00:10:17.266 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:17.264+0000 7f43a7669740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f4a9597790 0x55f4a95a2140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:10:17.266 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:17.264+0000 7f43a7669740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f4a959a0c0 0x55f4a9593140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:10:17.266 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:17.264+0000 7f43a7669740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f4a959cb00 0x55f4a9593680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:10:17.266 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:17.264+0000 7f43a7669740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f439c1ad880 con 0x55f4a959cb00 2024-09-11T00:10:17.266 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:17.264+0000 7f43a7669740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f439c1af480 con 0x55f4a959a0c0 2024-09-11T00:10:17.267 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:17.264+0000 7f43a7669740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f439c1adf50 con 0x55f4a9597790 2024-09-11T00:10:17.268 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:17.264+0000 7f43a1400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f4a959a0c0 0x55f4a9593140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:10:17.268 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:17.264+0000 7f43a2800640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f4a959cb00 0x55f4a9593680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:10:17.268 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:17.264+0000 7f43a1400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f4a959a0c0 0x55f4a9593140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.12:55238/0 (socket says 172.21.15.12:55238) 2024-09-11T00:10:17.269 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:17.264+0000 7f43a1400640 1 -- 172.21.15.12:0/2352320828 learned_addr learned my addr 172.21.15.12:0/2352320828 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:10:17.269 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:17.264+0000 7f43a1e00640 1 --2- 172.21.15.12:0/2352320828 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f4a9597790 0x55f4a95a2140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:10:17.269 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:17.265+0000 7f43a1400640 1 -- 172.21.15.12:0/2352320828 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f4a9597790 msgr2=0x55f4a95a2140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:10:17.269 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:17.265+0000 7f43a1400640 1 --2- 172.21.15.12:0/2352320828 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f4a9597790 0x55f4a95a2140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:10:17.269 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:17.265+0000 7f43a1400640 1 -- 172.21.15.12:0/2352320828 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f4a959cb00 msgr2=0x55f4a9593680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:10:17.269 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:17.265+0000 7f43a1400640 1 --2- 172.21.15.12:0/2352320828 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f4a959cb00 0x55f4a9593680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:10:17.269 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:17.265+0000 7f43a1400640 1 -- 172.21.15.12:0/2352320828 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f4a9599ef0 con 0x55f4a959a0c0 2024-09-11T00:10:17.269 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:17.265+0000 7f43a1e00640 1 --2- 172.21.15.12:0/2352320828 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f4a9597790 0x55f4a95a2140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:10:17.269 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:17.265+0000 7f43a1400640 1 --2- 172.21.15.12:0/2352320828 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f4a959a0c0 0x55f4a9593140 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f438c0c8030 tx=0x7f438c005f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3a2bdd3e42e5ae89 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:10:17.269 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:17.265+0000 7f438a600640 1 -- 172.21.15.12:0/2352320828 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f438c0c7220 con 0x55f4a959a0c0 2024-09-11T00:10:17.269 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:17.265+0000 7f438a600640 1 -- 172.21.15.12:0/2352320828 <== mon.2 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f438c0328f0 con 0x55f4a959a0c0 2024-09-11T00:10:17.269 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:17.265+0000 7f438a600640 1 -- 172.21.15.12:0/2352320828 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f438c0d5040 con 0x55f4a959a0c0 2024-09-11T00:10:17.269 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:17.266+0000 7f43a7669740 1 -- 172.21.15.12:0/2352320828 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f4a959a0c0 msgr2=0x55f4a9593140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:10:17.270 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:17.266+0000 7f43a7669740 1 --2- 172.21.15.12:0/2352320828 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f4a959a0c0 0x55f4a9593140 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f438c0c8030 tx=0x7f438c005f90 comp rx=0 tx=0).stop 2024-09-11T00:10:17.270 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:17.266+0000 7f43a7669740 1 -- 172.21.15.12:0/2352320828 shutdown_connections 2024-09-11T00:10:17.270 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:17.266+0000 7f43a7669740 1 --2- 172.21.15.12:0/2352320828 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f4a959cb00 0x55f4a9593680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:10:17.270 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:17.266+0000 7f43a7669740 1 --2- 172.21.15.12:0/2352320828 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f4a959a0c0 0x55f4a9593140 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:10:17.270 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:17.266+0000 7f43a7669740 1 --2- 172.21.15.12:0/2352320828 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f4a9597790 0x55f4a95a2140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:10:17.270 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:17.266+0000 7f43a7669740 1 -- 172.21.15.12:0/2352320828 >> 172.21.15.12:0/2352320828 conn(0x55f4a95928e0 msgr2=0x55f4a959cf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:10:17.270 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:17.266+0000 7f43a7669740 1 -- 172.21.15.12:0/2352320828 shutdown_connections 2024-09-11T00:10:17.270 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:17.266+0000 7f43a7669740 1 -- 172.21.15.12:0/2352320828 wait complete. 2024-09-11T00:10:17.270 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:17.266+0000 7f43a7669740 1 -- start start 2024-09-11T00:10:17.270 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:17.267+0000 7f43a7669740 1 Processor -- start 2024-09-11T00:10:17.273 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:17.267+0000 7f43a7669740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f4a9597790 0x55f4a95a8980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:10:17.273 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:17.267+0000 7f43a7669740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f4a959a0c0 0x55f4a95a7680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:10:17.273 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:17.267+0000 7f43a7669740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f4a959cb00 0x55f4a95a7bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:10:17.273 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:17.267+0000 7f43a7669740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f439c1aedb0 con 0x55f4a9597790 2024-09-11T00:10:17.273 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:17.267+0000 7f43a7669740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f439c1ae620 con 0x55f4a959a0c0 2024-09-11T00:10:17.273 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:17.267+0000 7f43a7669740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f439c1ab860 con 0x55f4a959cb00 2024-09-11T00:10:17.273 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:17.267+0000 7f43a7669740 10 client.0 _ll_register_callbacks cb 0x7f43980097e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:10:17.273 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:17.267+0000 7f43a1e00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f4a9597790 0x55f4a95a8980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:10:17.274 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:17.267+0000 7f43a1e00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f4a9597790 0x55f4a95a8980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:54308/0 (socket says 172.21.15.12:54308) 2024-09-11T00:10:17.274 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:17.267+0000 7f43a1e00640 1 -- 172.21.15.12:0/1745791480 learned_addr learned my addr 172.21.15.12:0/1745791480 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:10:17.274 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:17.267+0000 7f43a2800640 1 --2- 172.21.15.12:0/1745791480 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f4a959cb00 0x55f4a95a7bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:10:17.274 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:17.267+0000 7f43a1400640 1 --2- 172.21.15.12:0/1745791480 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f4a959a0c0 0x55f4a95a7680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:10:17.274 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:17.268+0000 7f43a2800640 1 -- 172.21.15.12:0/1745791480 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f4a959a0c0 msgr2=0x55f4a95a7680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:10:17.274 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:17.268+0000 7f43a2800640 1 --2- 172.21.15.12:0/1745791480 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f4a959a0c0 0x55f4a95a7680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:10:17.274 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:17.268+0000 7f43a2800640 1 -- 172.21.15.12:0/1745791480 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f4a9597790 msgr2=0x55f4a95a8980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:10:17.274 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:17.268+0000 7f43a2800640 1 --2- 172.21.15.12:0/1745791480 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f4a9597790 0x55f4a95a8980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:10:17.274 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:17.268+0000 7f43a2800640 1 -- 172.21.15.12:0/1745791480 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f4a95a83f0 con 0x55f4a959cb00 2024-09-11T00:10:17.274 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:17.268+0000 7f43a1e00640 1 --2- 172.21.15.12:0/1745791480 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f4a9597790 0x55f4a95a8980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:10:17.274 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:17.268+0000 7f43a2800640 1 --2- 172.21.15.12:0/1745791480 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f4a959cb00 0x55f4a95a7bc0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f4398000f80 tx=0x7f4398004dd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1f687d418c4d5cf3 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:10:17.274 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:17.268+0000 7f437f400640 10 client.0 ms_handle_connect on v2:172.21.15.178:3300/0 2024-09-11T00:10:17.274 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:17.268+0000 7f437f400640 1 -- 172.21.15.12:0/1745791480 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4398039070 con 0x55f4a959cb00 2024-09-11T00:10:17.275 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:17.268+0000 7f43a7669740 1 -- 172.21.15.12:0/1745791480 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f438c0328f0 con 0x55f4a959cb00 2024-09-11T00:10:17.275 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:17.268+0000 7f437f400640 1 -- 172.21.15.12:0/1745791480 <== mon.1 v2:172.21.15.178:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9208+0+0 (secure 0 0 0) 0x7f4398004f40 con 0x55f4a959cb00 2024-09-11T00:10:17.275 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:17.269+0000 7f437f400640 1 -- 172.21.15.12:0/1745791480 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f4398044050 con 0x55f4a959cb00 2024-09-11T00:10:17.275 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:17.269+0000 7f43a7669740 10 client.25894 fetch_fsmap learned FSMap version 33 2024-09-11T00:10:17.275 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:17.269+0000 7f43a7669740 1 -- 172.21.15.12:0/1745791480 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55f4a9619990 con 0x55f4a959cb00 2024-09-11T00:10:17.275 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:17.269+0000 7f437f400640 1 -- 172.21.15.12:0/1745791480 <== mon.1 v2:172.21.15.178:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f438c0328f0 con 0x55f4a959cb00 2024-09-11T00:10:17.275 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:17.269+0000 7f437f400640 1 -- 172.21.15.12:0/1745791480 <== mon.1 v2:172.21.15.178:3300/0 5 ==== fsmap(e 33) ==== 8386+0+0 (secure 0 0 0) 0x7f439800b070 con 0x55f4a959cb00 2024-09-11T00:10:17.275 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:17.269+0000 7f43a7669740 10 client.25894 fetch_fsmap finished waiting for FSMap version 33 2024-09-11T00:10:17.275 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:17.269+0000 7f43a7669740 10 client.25894 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T00:10:17.275 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:17.269+0000 7f43a7669740 20 client.25894 populate_metadata read hostname 'smithi012' 2024-09-11T00:10:17.275 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:17.269+0000 7f43a7669740 1 --2- 172.21.15.12:0/1745791480 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55f4a961a430 0x55f4a961c850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:10:17.275 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:17.269+0000 7f43a7669740 4 client.25894 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T00:10:17.276 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:17.269+0000 7f43a7669740 1 -- 172.21.15.12:0/1745791480 --> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55f4a95080b0 con 0x55f4a961a430 2024-09-11T00:10:17.276 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:17.270+0000 7f43a1e00640 1 --2- 172.21.15.12:0/1745791480 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55f4a961a430 0x55f4a961c850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:10:17.276 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:17.270+0000 7f43a1e00640 1 --2- 172.21.15.12:0/1745791480 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55f4a961a430 0x55f4a961c850 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=54d533281ecf9f2d server_cookie=31c0225b331f35a2 in_seq=0 out_seq=0 2024-09-11T00:10:17.276 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:17.270+0000 7f437f400640 10 client.25894 ms_handle_connect on v2:172.21.15.165:6838/501312984 2024-09-11T00:10:17.312 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:17.309+0000 7f437f400640 1 -- 172.21.15.12:0/1745791480 <== mds.0 v2:172.21.15.165:6838/501312984 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55f4a95080b0 con 0x55f4a961a430 2024-09-11T00:10:17.312 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:17.309+0000 7f437f400640 10 client.25894 handle_command_reply: tid=0 2024-09-11T00:10:17.312 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:17.309+0000 7f43a7669740 1 client.25894 shutdown 2024-09-11T00:10:17.313 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:17.310+0000 7f43a7669740 1 -- 172.21.15.12:0/1745791480 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f4a959cb00 msgr2=0x55f4a95a7bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:10:17.313 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:17.310+0000 7f43a7669740 1 --2- 172.21.15.12:0/1745791480 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f4a959cb00 0x55f4a95a7bc0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f4398000f80 tx=0x7f4398004dd0 comp rx=0 tx=0).stop 2024-09-11T00:10:17.313 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:17.310+0000 7f43a7669740 1 -- 172.21.15.12:0/1745791480 shutdown_connections 2024-09-11T00:10:17.313 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:17.310+0000 7f43a7669740 1 --2- 172.21.15.12:0/1745791480 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55f4a961a430 0x55f4a961c850 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:10:17.313 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:17.310+0000 7f43a7669740 1 --2- 172.21.15.12:0/1745791480 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f4a959cb00 0x55f4a95a7bc0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:10:17.313 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:17.310+0000 7f43a7669740 1 --2- 172.21.15.12:0/1745791480 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f4a959a0c0 0x55f4a95a7680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:10:17.313 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:17.310+0000 7f43a7669740 1 --2- 172.21.15.12:0/1745791480 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f4a9597790 0x55f4a95a8980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:10:17.313 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:17.310+0000 7f437f400640 0 client.25894 ms_handle_reset on v2:172.21.15.165:6838/501312984 2024-09-11T00:10:17.313 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:17.310+0000 7f43a7669740 1 -- 172.21.15.12:0/1745791480 >> 172.21.15.12:0/1745791480 conn(0x55f4a95928e0 msgr2=0x55f4a959cf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:10:17.313 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:17.310+0000 7f43a7669740 1 -- 172.21.15.12:0/1745791480 shutdown_connections 2024-09-11T00:10:17.313 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:17.310+0000 7f43a7669740 1 -- 172.21.15.12:0/1745791480 wait complete. 2024-09-11T00:10:17.313 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:17.310+0000 7f43a7669740 20 client.25894 trim_cache size 0 max 16384 2024-09-11T00:10:17.314 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:{ 2024-09-11T00:10:17.314 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "message": "", 2024-09-11T00:10:17.314 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "return_code": 0 2024-09-11T00:10:17.314 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:} 2024-09-11T00:10:17.315 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:17.311+0000 7f43a4800640 1 -- 172.21.15.12:0/75769782 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f43840b5360 msgr2=0x7f43840b7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:10:17.315 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:17.311+0000 7f43a4800640 1 --2- 172.21.15.12:0/75769782 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f43840b5360 0x7f43840b7820 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f4398005fd0 tx=0x7f4398010040 comp rx=0 tx=0).stop 2024-09-11T00:10:17.315 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:17.311+0000 7f43a4800640 1 -- 172.21.15.12:0/75769782 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f439c105cf0 msgr2=0x7f439c19b5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:10:17.315 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:17.311+0000 7f43a4800640 1 --2- 172.21.15.12:0/75769782 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f439c105cf0 0x7f439c19b5f0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f438c0057c0 tx=0x7f438c032d30 comp rx=0 tx=0).stop 2024-09-11T00:10:17.315 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:17.311+0000 7f43a4800640 1 -- 172.21.15.12:0/75769782 shutdown_connections 2024-09-11T00:10:17.315 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:17.311+0000 7f43a4800640 1 --2- 172.21.15.12:0/75769782 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f43840b5360 0x7f43840b7820 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:10:17.315 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:17.311+0000 7f43a4800640 1 --2- 172.21.15.12:0/75769782 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f439c1066b0 0x7f439c19c220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:10:17.315 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:17.311+0000 7f43a4800640 1 --2- 172.21.15.12:0/75769782 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f439c105cf0 0x7f439c19b5f0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:10:17.315 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:17.311+0000 7f43a4800640 1 --2- 172.21.15.12:0/75769782 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f439c104ba0 0x7f439c19afa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:10:17.315 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:17.311+0000 7f43a4800640 1 -- 172.21.15.12:0/75769782 >> 172.21.15.12:0/75769782 conn(0x7f439c100460 msgr2=0x7f439c1090c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:10:17.316 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:17.311+0000 7f43a4800640 1 -- 172.21.15.12:0/75769782 shutdown_connections 2024-09-11T00:10:17.316 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:17.311+0000 7f43a4800640 1 -- 172.21.15.12:0/75769782 wait complete. 2024-09-11T00:10:17.611 INFO:tasks.workunit.client.0.smithi012.stdout: 1 107173 4.09 MB/sec execute 14 sec latency 1138.322 ms 2024-09-11T00:10:17.782 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:10:17 smithi165 ceph-mon[35356]: pgmap v495: 129 pgs: 129 active+clean; 29 MiB data, 592 MiB used, 1.0 TiB / 1.0 TiB avail; 464 KiB/s rd, 400 KiB/s wr, 95 op/s 2024-09-11T00:10:17.813 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:10:17 smithi012 ceph-mon[30333]: pgmap v495: 129 pgs: 129 active+clean; 29 MiB data, 592 MiB used, 1.0 TiB / 1.0 TiB avail; 464 KiB/s rd, 400 KiB/s wr, 95 op/s 2024-09-11T00:10:17.830 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:10:17 smithi178 ceph-mon[35919]: pgmap v495: 129 pgs: 129 active+clean; 29 MiB data, 592 MiB used, 1.0 TiB / 1.0 TiB avail; 464 KiB/s rd, 400 KiB/s wr, 95 op/s 2024-09-11T00:10:18.613 INFO:tasks.workunit.client.0.smithi012.stdout: 1 107173 3.81 MB/sec execute 15 sec latency 2138.418 ms 2024-09-11T00:10:19.611 INFO:tasks.workunit.client.0.smithi012.stdout: 1 107241 3.59 MB/sec execute 16 sec latency 3064.007 ms 2024-09-11T00:10:19.782 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:10:19 smithi165 ceph-mon[35356]: pgmap v496: 129 pgs: 129 active+clean; 29 MiB data, 592 MiB used, 1.0 TiB / 1.0 TiB avail; 512 KiB/s rd, 413 KiB/s wr, 99 op/s 2024-09-11T00:10:19.813 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:10:19 smithi012 ceph-mon[30333]: pgmap v496: 129 pgs: 129 active+clean; 29 MiB data, 592 MiB used, 1.0 TiB / 1.0 TiB avail; 512 KiB/s rd, 413 KiB/s wr, 99 op/s 2024-09-11T00:10:19.830 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:10:19 smithi178 ceph-mon[35919]: pgmap v496: 129 pgs: 129 active+clean; 29 MiB data, 592 MiB used, 1.0 TiB / 1.0 TiB avail; 512 KiB/s rd, 413 KiB/s wr, 99 op/s 2024-09-11T00:10:20.611 INFO:tasks.workunit.client.0.smithi012.stdout: 1 107241 3.38 MB/sec execute 17 sec latency 1026.291 ms 2024-09-11T00:10:21.621 INFO:tasks.workunit.client.0.smithi012.stdout: 1 107241 3.19 MB/sec execute 18 sec latency 2036.314 ms 2024-09-11T00:10:21.782 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:10:21 smithi165 ceph-mon[35356]: pgmap v497: 129 pgs: 129 active+clean; 30 MiB data, 583 MiB used, 1.0 TiB / 1.0 TiB avail; 681 KiB/s rd, 453 KiB/s wr, 109 op/s 2024-09-11T00:10:21.812 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:10:21 smithi012 ceph-mon[30333]: pgmap v497: 129 pgs: 129 active+clean; 30 MiB data, 583 MiB used, 1.0 TiB / 1.0 TiB avail; 681 KiB/s rd, 453 KiB/s wr, 109 op/s 2024-09-11T00:10:21.830 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:10:21 smithi178 ceph-mon[35919]: pgmap v497: 129 pgs: 129 active+clean; 30 MiB data, 583 MiB used, 1.0 TiB / 1.0 TiB avail; 681 KiB/s rd, 453 KiB/s wr, 109 op/s 2024-09-11T00:10:22.621 INFO:tasks.workunit.client.0.smithi012.stdout: 1 107241 3.02 MB/sec execute 19 sec latency 3036.380 ms 2024-09-11T00:10:23.622 INFO:tasks.workunit.client.0.smithi012.stdout: 1 107419 2.89 MB/sec execute 20 sec latency 3915.370 ms 2024-09-11T00:10:23.782 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:10:23 smithi165 ceph-mon[35356]: pgmap v498: 129 pgs: 129 active+clean; 30 MiB data, 590 MiB used, 1.0 TiB / 1.0 TiB avail; 605 KiB/s rd, 412 KiB/s wr, 110 op/s 2024-09-11T00:10:23.812 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:10:23 smithi012 ceph-mon[30333]: pgmap v498: 129 pgs: 129 active+clean; 30 MiB data, 590 MiB used, 1.0 TiB / 1.0 TiB avail; 605 KiB/s rd, 412 KiB/s wr, 110 op/s 2024-09-11T00:10:23.830 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:10:23 smithi178 ceph-mon[35919]: pgmap v498: 129 pgs: 129 active+clean; 30 MiB data, 590 MiB used, 1.0 TiB / 1.0 TiB avail; 605 KiB/s rd, 412 KiB/s wr, 110 op/s 2024-09-11T00:10:24.622 INFO:tasks.workunit.client.0.smithi012.stdout: 1 110070 3.55 MB/sec execute 21 sec latency 9.695 ms 2024-09-11T00:10:25.622 INFO:tasks.workunit.client.0.smithi012.stdout: 1 110667 3.46 MB/sec execute 22 sec latency 448.992 ms 2024-09-11T00:10:25.782 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:10:25 smithi165 ceph-mon[35356]: pgmap v499: 129 pgs: 129 active+clean; 31 MiB data, 591 MiB used, 1.0 TiB / 1.0 TiB avail; 548 KiB/s rd, 434 KiB/s wr, 91 op/s 2024-09-11T00:10:25.812 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:10:25 smithi012 ceph-mon[30333]: pgmap v499: 129 pgs: 129 active+clean; 31 MiB data, 591 MiB used, 1.0 TiB / 1.0 TiB avail; 548 KiB/s rd, 434 KiB/s wr, 91 op/s 2024-09-11T00:10:25.830 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:10:25 smithi178 ceph-mon[35919]: pgmap v499: 129 pgs: 129 active+clean; 31 MiB data, 591 MiB used, 1.0 TiB / 1.0 TiB avail; 548 KiB/s rd, 434 KiB/s wr, 91 op/s 2024-09-11T00:10:26.622 INFO:tasks.workunit.client.0.smithi012.stdout: 1 110667 3.31 MB/sec execute 23 sec latency 1449.145 ms 2024-09-11T00:10:27.622 INFO:tasks.workunit.client.0.smithi012.stdout: 1 110667 3.17 MB/sec execute 24 sec latency 2449.243 ms 2024-09-11T00:10:27.782 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:10:27 smithi165 ceph-mon[35356]: pgmap v500: 129 pgs: 129 active+clean; 30 MiB data, 616 MiB used, 1.0 TiB / 1.0 TiB avail; 674 KiB/s rd, 1.4 MiB/s wr, 148 op/s 2024-09-11T00:10:27.813 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:10:27 smithi012 ceph-mon[30333]: pgmap v500: 129 pgs: 129 active+clean; 30 MiB data, 616 MiB used, 1.0 TiB / 1.0 TiB avail; 674 KiB/s rd, 1.4 MiB/s wr, 148 op/s 2024-09-11T00:10:27.830 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:10:27 smithi178 ceph-mon[35919]: pgmap v500: 129 pgs: 129 active+clean; 30 MiB data, 616 MiB used, 1.0 TiB / 1.0 TiB avail; 674 KiB/s rd, 1.4 MiB/s wr, 148 op/s 2024-09-11T00:10:28.622 INFO:tasks.workunit.client.0.smithi012.stdout: 1 110667 3.05 MB/sec execute 25 sec latency 3449.385 ms 2024-09-11T00:10:29.622 INFO:tasks.workunit.client.0.smithi012.stdout: 1 110755 2.94 MB/sec execute 26 sec latency 4366.007 ms 2024-09-11T00:10:29.782 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:10:29 smithi165 ceph-mon[35356]: pgmap v501: 129 pgs: 129 active+clean; 31 MiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 479 KiB/s rd, 1.3 MiB/s wr, 140 op/s 2024-09-11T00:10:29.812 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:10:29 smithi012 ceph-mon[30333]: pgmap v501: 129 pgs: 129 active+clean; 31 MiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 479 KiB/s rd, 1.3 MiB/s wr, 140 op/s 2024-09-11T00:10:29.830 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:10:29 smithi178 ceph-mon[35919]: pgmap v501: 129 pgs: 129 active+clean; 31 MiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 479 KiB/s rd, 1.3 MiB/s wr, 140 op/s 2024-09-11T00:10:30.437 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:30.435+0000 7f7377a00640 1 -- 172.21.15.12:0/3417524939 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7370104ba0 msgr2=0x7f7370104fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:10:30.438 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:30.435+0000 7f7377a00640 1 --2- 172.21.15.12:0/3417524939 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7370104ba0 0x7f7370104fa0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f7360009930 tx=0x7f736002a420 comp rx=0 tx=0).stop 2024-09-11T00:10:30.438 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:30.435+0000 7f7377a00640 1 -- 172.21.15.12:0/3417524939 shutdown_connections 2024-09-11T00:10:30.438 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:30.435+0000 7f7377a00640 1 --2- 172.21.15.12:0/3417524939 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f73701066b0 0x7f737010d030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:10:30.438 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:30.435+0000 7f7377a00640 1 --2- 172.21.15.12:0/3417524939 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7370105cf0 0x7f7370106170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:10:30.438 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:30.435+0000 7f7377a00640 1 --2- 172.21.15.12:0/3417524939 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7370104ba0 0x7f7370104fa0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:10:30.438 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:30.435+0000 7f7377a00640 1 -- 172.21.15.12:0/3417524939 >> 172.21.15.12:0/3417524939 conn(0x7f7370100460 msgr2=0x7f7370102880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:10:30.438 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:30.435+0000 7f7377a00640 1 -- 172.21.15.12:0/3417524939 shutdown_connections 2024-09-11T00:10:30.438 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:30.435+0000 7f7377a00640 1 -- 172.21.15.12:0/3417524939 wait complete. 2024-09-11T00:10:30.440 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:30.436+0000 7f7377a00640 1 Processor -- start 2024-09-11T00:10:30.441 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:30.436+0000 7f7377a00640 1 -- start start 2024-09-11T00:10:30.441 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:30.437+0000 7f7377a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7370104ba0 0x7f737019aff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:10:30.441 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:30.437+0000 7f7377a00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7370105cf0 0x7f737019b640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:10:30.441 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:30.437+0000 7f7377a00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f73701066b0 0x7f737019c270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:10:30.441 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:30.437+0000 7f7377a00640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f7370110840 con 0x7f73701066b0 2024-09-11T00:10:30.441 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:30.437+0000 7f7377a00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f73701106c0 con 0x7f7370104ba0 2024-09-11T00:10:30.441 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:30.437+0000 7f7377a00640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f73701109c0 con 0x7f7370105cf0 2024-09-11T00:10:30.441 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:30.437+0000 7f7375000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7370104ba0 0x7f737019aff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:10:30.441 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:30.437+0000 7f7367e00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7370105cf0 0x7f737019b640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:10:30.442 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:30.437+0000 7f7375000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7370104ba0 0x7f737019aff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.12:51320/0 (socket says 172.21.15.12:51320) 2024-09-11T00:10:30.442 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:30.437+0000 7f7375000640 1 -- 172.21.15.12:0/3334980094 learned_addr learned my addr 172.21.15.12:0/3334980094 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:10:30.442 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:30.437+0000 7f7375a00640 1 --2- 172.21.15.12:0/3334980094 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f73701066b0 0x7f737019c270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:10:30.442 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:30.437+0000 7f7367e00640 1 -- 172.21.15.12:0/3334980094 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7370104ba0 msgr2=0x7f737019aff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:10:30.442 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:30.437+0000 7f7367e00640 1 --2- 172.21.15.12:0/3334980094 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7370104ba0 0x7f737019aff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:10:30.442 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:30.437+0000 7f7367e00640 1 -- 172.21.15.12:0/3334980094 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f73701066b0 msgr2=0x7f737019c270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:10:30.442 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:30.437+0000 7f7367e00640 1 --2- 172.21.15.12:0/3334980094 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f73701066b0 0x7f737019c270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:10:30.442 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:30.437+0000 7f7367e00640 1 -- 172.21.15.12:0/3334980094 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f737019cb30 con 0x7f7370105cf0 2024-09-11T00:10:30.442 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:30.438+0000 7f7367e00640 1 --2- 172.21.15.12:0/3334980094 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7370105cf0 0x7f737019b640 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f735c00e9f0 tx=0x7f735c00eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:10:30.442 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:30.438+0000 7f736e000640 1 -- 172.21.15.12:0/3334980094 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f735c00ccc0 con 0x7f7370105cf0 2024-09-11T00:10:30.443 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:30.438+0000 7f7377a00640 1 -- 172.21.15.12:0/3334980094 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f737019ce20 con 0x7f7370105cf0 2024-09-11T00:10:30.443 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:30.438+0000 7f736e000640 1 -- 172.21.15.12:0/3334980094 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f735c0108e0 con 0x7f7370105cf0 2024-09-11T00:10:30.443 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:30.438+0000 7f736e000640 1 -- 172.21.15.12:0/3334980094 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f735c018880 con 0x7f7370105cf0 2024-09-11T00:10:30.443 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:30.438+0000 7f7377a00640 1 -- 172.21.15.12:0/3334980094 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73701a34f0 con 0x7f7370105cf0 2024-09-11T00:10:30.444 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:30.441+0000 7f737aa64740 1 Processor -- start 2024-09-11T00:10:30.444 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:30.441+0000 7f737aa64740 1 -- start start 2024-09-11T00:10:30.444 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:30.441+0000 7f737aa64740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5644f4fe9790 0x5644f4febb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:10:30.444 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:30.441+0000 7f737aa64740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5644f4fec0c0 0x5644f4fee4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:10:30.445 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:30.441+0000 7f737aa64740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5644f4feeb00 0x5644f4ff0fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:10:30.445 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:30.441+0000 7f737aa64740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x5644f4f5c6b0 con 0x5644f4fec0c0 2024-09-11T00:10:30.445 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:30.441+0000 7f737aa64740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x5644f4f4bae0 con 0x5644f4fe9790 2024-09-11T00:10:30.445 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:30.441+0000 7f737aa64740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5644f4f4f8f0 con 0x5644f4feeb00 2024-09-11T00:10:30.445 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:30.442+0000 7f7375000640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5644f4fec0c0 0x5644f4fee4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:10:30.445 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:30.442+0000 7f7375a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5644f4fe9790 0x5644f4febb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:10:30.445 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:30.442+0000 7f7375a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5644f4fe9790 0x5644f4febb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.12:51322/0 (socket says 172.21.15.12:51322) 2024-09-11T00:10:30.445 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:30.442+0000 7f7375a00640 1 -- 172.21.15.12:0/924905073 learned_addr learned my addr 172.21.15.12:0/924905073 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:10:30.445 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:30.442+0000 7f736e000640 1 -- 172.21.15.12:0/3334980094 <== mon.1 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7f735c010430 con 0x7f7370105cf0 2024-09-11T00:10:30.445 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:30.442+0000 7f7375a00640 1 -- 172.21.15.12:0/924905073 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5644f4feeb00 msgr2=0x5644f4ff0fc0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-11T00:10:30.446 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:30.442+0000 7f7375a00640 1 --2- 172.21.15.12:0/924905073 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5644f4feeb00 0x5644f4ff0fc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:10:30.446 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:30.442+0000 7f7375a00640 1 -- 172.21.15.12:0/924905073 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5644f4fec0c0 msgr2=0x5644f4fee4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:10:30.446 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:30.442+0000 7f7375a00640 1 --2- 172.21.15.12:0/924905073 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5644f4fec0c0 0x5644f4fee4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:10:30.446 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:30.442+0000 7f7375a00640 1 -- 172.21.15.12:0/924905073 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5644f4f18780 con 0x5644f4fe9790 2024-09-11T00:10:30.446 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:30.442+0000 7f7375000640 1 --2- 172.21.15.12:0/924905073 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5644f4fec0c0 0x5644f4fee4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:10:30.447 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:30.443+0000 7f7375a00640 1 --2- 172.21.15.12:0/924905073 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5644f4fe9790 0x5644f4febb80 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f7368009f30 tx=0x7f73680118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=584133c4e0582712 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:10:30.448 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:30.443+0000 7f7366000640 1 -- 172.21.15.12:0/924905073 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7368004d20 con 0x5644f4fe9790 2024-09-11T00:10:30.448 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:30.443+0000 7f7366000640 1 -- 172.21.15.12:0/924905073 <== mon.2 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7368004050 con 0x5644f4fe9790 2024-09-11T00:10:30.448 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:30.443+0000 7f7366000640 1 -- 172.21.15.12:0/924905073 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7368005260 con 0x5644f4fe9790 2024-09-11T00:10:30.448 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:30.444+0000 7f737aa64740 1 -- 172.21.15.12:0/924905073 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5644f4fe9790 msgr2=0x5644f4febb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:10:30.448 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:30.444+0000 7f737aa64740 1 --2- 172.21.15.12:0/924905073 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5644f4fe9790 0x5644f4febb80 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f7368009f30 tx=0x7f73680118d0 comp rx=0 tx=0).stop 2024-09-11T00:10:30.448 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:30.444+0000 7f737aa64740 1 -- 172.21.15.12:0/924905073 shutdown_connections 2024-09-11T00:10:30.448 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:30.444+0000 7f737aa64740 1 --2- 172.21.15.12:0/924905073 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5644f4feeb00 0x5644f4ff0fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:10:30.448 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:30.444+0000 7f737aa64740 1 --2- 172.21.15.12:0/924905073 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5644f4fec0c0 0x5644f4fee4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:10:30.448 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:30.444+0000 7f737aa64740 1 --2- 172.21.15.12:0/924905073 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5644f4fe9790 0x5644f4febb80 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:10:30.448 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:30.444+0000 7f737aa64740 1 -- 172.21.15.12:0/924905073 >> 172.21.15.12:0/924905073 conn(0x5644f4fe48e0 msgr2=0x5644f4fe5670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:10:30.448 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:30.444+0000 7f737aa64740 1 -- 172.21.15.12:0/924905073 shutdown_connections 2024-09-11T00:10:30.448 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:30.444+0000 7f737aa64740 1 -- 172.21.15.12:0/924905073 wait complete. 2024-09-11T00:10:30.449 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:30.445+0000 7f736e000640 1 --2- 172.21.15.12:0/3334980094 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f73540b5270 0x7f73540b7730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:10:30.449 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:30.445+0000 7f736e000640 1 -- 172.21.15.12:0/3334980094 <== mon.1 v2:172.21.15.178:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9208+0+0 (secure 0 0 0) 0x7f735c0e6250 con 0x7f7370105cf0 2024-09-11T00:10:30.449 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:30.445+0000 7f737aa64740 1 -- start start 2024-09-11T00:10:30.449 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:30.445+0000 7f7375000640 1 --2- 172.21.15.12:0/3334980094 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f73540b5270 0x7f73540b7730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:10:30.449 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:30.445+0000 7f737aa64740 1 Processor -- start 2024-09-11T00:10:30.450 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:30.446+0000 7f737aa64740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5644f4fe9790 0x5644f4ffb7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:10:30.450 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:30.446+0000 7f737aa64740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5644f4fec0c0 0x5644f4ffbe40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:10:30.451 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:30.446+0000 7f737aa64740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5644f4feeb00 0x5644f4ffca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:10:30.451 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:30.446+0000 7f737aa64740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x5644f4e83ab0 con 0x5644f4feeb00 2024-09-11T00:10:30.451 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:30.446+0000 7f737aa64740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x5644f4ef2570 con 0x5644f4fec0c0 2024-09-11T00:10:30.451 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:30.446+0000 7f737aa64740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5644f4ff1ae0 con 0x5644f4fe9790 2024-09-11T00:10:30.451 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:30.446+0000 7f737aa64740 10 client.0 _ll_register_callbacks cb 0x5644f4f01100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:10:30.451 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:30.446+0000 7f7367e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5644f4fec0c0 0x5644f4ffbe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:10:30.451 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:30.446+0000 7f7367e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5644f4fec0c0 0x5644f4ffbe40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.12:51332/0 (socket says 172.21.15.12:51332) 2024-09-11T00:10:30.451 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:30.446+0000 7f7367e00640 1 -- 172.21.15.12:0/226583613 learned_addr learned my addr 172.21.15.12:0/226583613 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:10:30.451 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:30.446+0000 7f7375a00640 1 --2- 172.21.15.12:0/226583613 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5644f4feeb00 0x5644f4ffca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:10:30.451 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:30.446+0000 7f7375000640 1 --2- 172.21.15.12:0/3334980094 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f73540b5270 0x7f73540b7730 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f7360000c00 tx=0x7f7360005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:10:30.451 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:30.447+0000 7f7375000640 1 --2- 172.21.15.12:0/226583613 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5644f4fe9790 0x5644f4ffb7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:10:30.451 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:30.447+0000 7f7375a00640 1 -- 172.21.15.12:0/226583613 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5644f4fe9790 msgr2=0x5644f4ffb7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:10:30.452 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:30.447+0000 7f7375a00640 1 --2- 172.21.15.12:0/226583613 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5644f4fe9790 0x5644f4ffb7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:10:30.452 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:30.447+0000 7f7375a00640 1 -- 172.21.15.12:0/226583613 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5644f4fec0c0 msgr2=0x5644f4ffbe40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:10:30.452 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:30.447+0000 7f7375a00640 1 --2- 172.21.15.12:0/226583613 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5644f4fec0c0 0x5644f4ffbe40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:10:30.452 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:30.447+0000 7f7375a00640 1 -- 172.21.15.12:0/226583613 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5644f4cad790 con 0x5644f4feeb00 2024-09-11T00:10:30.452 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:30.447+0000 7f7367e00640 1 --2- 172.21.15.12:0/226583613 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5644f4fec0c0 0x5644f4ffbe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:10:30.452 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:30.448+0000 7f7375a00640 1 --2- 172.21.15.12:0/226583613 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5644f4feeb00 0x5644f4ffca10 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f736801b040 tx=0x7f7368004cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8704e6ae89784ffb server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:10:30.452 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:30.449+0000 7f735aa00640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-09-11T00:10:30.453 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:30.449+0000 7f735aa00640 1 -- 172.21.15.12:0/226583613 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7368029020 con 0x5644f4feeb00 2024-09-11T00:10:30.454 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:30.449+0000 7f735aa00640 1 -- 172.21.15.12:0/226583613 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9208+0+0 (secure 0 0 0) 0x7f7368005040 con 0x5644f4feeb00 2024-09-11T00:10:30.454 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:30.450+0000 7f737aa64740 1 -- 172.21.15.12:0/226583613 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7368004050 con 0x5644f4feeb00 2024-09-11T00:10:30.454 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:30.450+0000 7f735aa00640 1 -- 172.21.15.12:0/226583613 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f7368003e40 con 0x5644f4feeb00 2024-09-11T00:10:30.454 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:30.450+0000 7f737aa64740 10 client.15891 fetch_fsmap learned FSMap version 33 2024-09-11T00:10:30.454 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:30.450+0000 7f737aa64740 1 -- 172.21.15.12:0/226583613 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x5644f4d26620 con 0x5644f4feeb00 2024-09-11T00:10:30.454 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:30.451+0000 7f735aa00640 1 -- 172.21.15.12:0/226583613 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7368004050 con 0x5644f4feeb00 2024-09-11T00:10:30.454 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:30.451+0000 7f735aa00640 1 -- 172.21.15.12:0/226583613 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 33) ==== 8386+0+0 (secure 0 0 0) 0x7f7368025670 con 0x5644f4feeb00 2024-09-11T00:10:30.454 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:30.451+0000 7f737aa64740 10 client.15891 fetch_fsmap finished waiting for FSMap version 33 2024-09-11T00:10:30.454 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:30.451+0000 7f737aa64740 10 client.15891 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T00:10:30.454 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:30.451+0000 7f737aa64740 20 client.15891 populate_metadata read hostname 'smithi012' 2024-09-11T00:10:30.455 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:30.451+0000 7f737aa64740 1 --2- 172.21.15.12:0/226583613 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x5644f5006e20 0x5644f5009210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:10:30.455 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:30.451+0000 7f737aa64740 4 client.15891 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T00:10:30.455 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:30.451+0000 7f737aa64740 1 -- 172.21.15.12:0/226583613 --> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5644f4f5e600 con 0x5644f5006e20 2024-09-11T00:10:30.455 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:30.452+0000 7f7375000640 1 --2- 172.21.15.12:0/226583613 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x5644f5006e20 0x5644f5009210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:10:30.455 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:30.453+0000 7f7375000640 1 --2- 172.21.15.12:0/226583613 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x5644f5006e20 0x5644f5009210 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=145e3be91be5e2e5 server_cookie=b84e6d9b68c50bb1 in_seq=0 out_seq=0 2024-09-11T00:10:30.455 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:30.453+0000 7f735aa00640 10 client.15891 ms_handle_connect on v2:172.21.15.165:6838/501312984 2024-09-11T00:10:30.457 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:30.455+0000 7f735aa00640 1 -- 172.21.15.12:0/226583613 <== mds.0 v2:172.21.15.165:6838/501312984 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5644f4f5e600 con 0x5644f5006e20 2024-09-11T00:10:30.457 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:30.455+0000 7f735aa00640 10 client.15891 handle_command_reply: tid=0 2024-09-11T00:10:30.457 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:30.455+0000 7f737aa64740 1 client.15891 shutdown 2024-09-11T00:10:30.457 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:30.456+0000 7f737aa64740 1 -- 172.21.15.12:0/226583613 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5644f4feeb00 msgr2=0x5644f4ffca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:10:30.458 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:30.456+0000 7f737aa64740 1 --2- 172.21.15.12:0/226583613 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5644f4feeb00 0x5644f4ffca10 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f736801b040 tx=0x7f7368004cf0 comp rx=0 tx=0).stop 2024-09-11T00:10:30.458 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:30.456+0000 7f737aa64740 1 -- 172.21.15.12:0/226583613 shutdown_connections 2024-09-11T00:10:30.458 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:30.456+0000 7f737aa64740 1 --2- 172.21.15.12:0/226583613 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x5644f5006e20 0x5644f5009210 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:10:30.458 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:30.456+0000 7f737aa64740 1 --2- 172.21.15.12:0/226583613 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5644f4feeb00 0x5644f4ffca10 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:10:30.458 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:30.456+0000 7f737aa64740 1 --2- 172.21.15.12:0/226583613 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5644f4fec0c0 0x5644f4ffbe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:10:30.459 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:30.456+0000 7f735aa00640 0 client.15891 ms_handle_reset on v2:172.21.15.165:6838/501312984 2024-09-11T00:10:30.459 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:30.456+0000 7f737aa64740 1 --2- 172.21.15.12:0/226583613 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5644f4fe9790 0x5644f4ffb7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:10:30.459 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:30.456+0000 7f737aa64740 1 -- 172.21.15.12:0/226583613 >> 172.21.15.12:0/226583613 conn(0x5644f4fe48e0 msgr2=0x5644f4fe3a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:10:30.459 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:30.457+0000 7f737aa64740 1 -- 172.21.15.12:0/226583613 shutdown_connections 2024-09-11T00:10:30.460 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:30.457+0000 7f737aa64740 1 -- 172.21.15.12:0/226583613 wait complete. 2024-09-11T00:10:30.460 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:30.457+0000 7f737aa64740 20 client.15891 trim_cache size 0 max 16384 2024-09-11T00:10:30.477 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:30.475+0000 7f737aa64740 1 Processor -- start 2024-09-11T00:10:30.477 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:30.475+0000 7f737aa64740 1 -- start start 2024-09-11T00:10:30.478 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:30.475+0000 7f737aa64740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5644f4fe9790 0x5644f4fef660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:10:30.478 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:30.475+0000 7f737aa64740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5644f4fec0c0 0x5644f500da90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:10:30.478 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:30.475+0000 7f737aa64740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5644f4feeb00 0x5644f500dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:10:30.478 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:30.475+0000 7f737aa64740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f73701a9780 con 0x5644f4fec0c0 2024-09-11T00:10:30.478 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:30.475+0000 7f737aa64740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f73701ab380 con 0x5644f4fe9790 2024-09-11T00:10:30.478 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:30.475+0000 7f737aa64740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f73701a9e50 con 0x5644f4feeb00 2024-09-11T00:10:30.478 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:30.475+0000 7f7375000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5644f4fe9790 0x5644f4fef660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:10:30.478 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:30.475+0000 7f7375000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5644f4fe9790 0x5644f4fef660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.12:51346/0 (socket says 172.21.15.12:51346) 2024-09-11T00:10:30.478 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:30.475+0000 7f7375000640 1 -- 172.21.15.12:0/320779089 learned_addr learned my addr 172.21.15.12:0/320779089 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:10:30.478 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:30.475+0000 7f7367e00640 1 --2- 172.21.15.12:0/320779089 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5644f4fec0c0 0x5644f500da90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:10:30.478 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:30.475+0000 7f7375a00640 1 --2- 172.21.15.12:0/320779089 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5644f4feeb00 0x5644f500dfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:10:30.478 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:30.476+0000 7f7375000640 1 -- 172.21.15.12:0/320779089 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5644f4feeb00 msgr2=0x5644f500dfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:10:30.479 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:30.476+0000 7f7375000640 1 --2- 172.21.15.12:0/320779089 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5644f4feeb00 0x5644f500dfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:10:30.479 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:30.476+0000 7f7375000640 1 -- 172.21.15.12:0/320779089 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5644f4fec0c0 msgr2=0x5644f500da90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:10:30.479 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:30.476+0000 7f7375000640 1 --2- 172.21.15.12:0/320779089 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5644f4fec0c0 0x5644f500da90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:10:30.479 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:30.476+0000 7f7375000640 1 -- 172.21.15.12:0/320779089 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5644f4febef0 con 0x5644f4fe9790 2024-09-11T00:10:30.482 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:30.476+0000 7f7375000640 1 --2- 172.21.15.12:0/320779089 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5644f4fe9790 0x5644f4fef660 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f7360009900 tx=0x7f7360040000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9c4de9978d10782a server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:10:30.482 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:30.476+0000 7f7365600640 1 -- 172.21.15.12:0/320779089 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7360035040 con 0x5644f4fe9790 2024-09-11T00:10:30.482 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:30.476+0000 7f7365600640 1 -- 172.21.15.12:0/320779089 <== mon.2 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7360003cf0 con 0x5644f4fe9790 2024-09-11T00:10:30.482 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:30.476+0000 7f7365600640 1 -- 172.21.15.12:0/320779089 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f736003d7d0 con 0x5644f4fe9790 2024-09-11T00:10:30.483 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:30.477+0000 7f737aa64740 1 -- 172.21.15.12:0/320779089 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5644f4fe9790 msgr2=0x5644f4fef660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:10:30.483 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:30.477+0000 7f737aa64740 1 --2- 172.21.15.12:0/320779089 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5644f4fe9790 0x5644f4fef660 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f7360009900 tx=0x7f7360040000 comp rx=0 tx=0).stop 2024-09-11T00:10:30.483 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:30.477+0000 7f737aa64740 1 -- 172.21.15.12:0/320779089 shutdown_connections 2024-09-11T00:10:30.483 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:30.477+0000 7f737aa64740 1 --2- 172.21.15.12:0/320779089 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5644f4feeb00 0x5644f500dfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:10:30.483 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:30.477+0000 7f737aa64740 1 --2- 172.21.15.12:0/320779089 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5644f4fec0c0 0x5644f500da90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:10:30.483 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:30.477+0000 7f737aa64740 1 --2- 172.21.15.12:0/320779089 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5644f4fe9790 0x5644f4fef660 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:10:30.483 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:30.477+0000 7f737aa64740 1 -- 172.21.15.12:0/320779089 >> 172.21.15.12:0/320779089 conn(0x5644f4fe48e0 msgr2=0x5644f4feef50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:10:30.483 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:30.477+0000 7f737aa64740 1 -- 172.21.15.12:0/320779089 shutdown_connections 2024-09-11T00:10:30.483 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:30.477+0000 7f737aa64740 1 -- 172.21.15.12:0/320779089 wait complete. 2024-09-11T00:10:30.483 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:30.477+0000 7f737aa64740 1 -- start start 2024-09-11T00:10:30.483 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:30.478+0000 7f737aa64740 1 Processor -- start 2024-09-11T00:10:30.483 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:30.478+0000 7f737aa64740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5644f4fe9790 0x5644f5003310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:10:30.483 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:30.478+0000 7f737aa64740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5644f4fec0c0 0x5644f5002010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:10:30.484 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:30.478+0000 7f737aa64740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5644f4feeb00 0x5644f50026e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:10:30.484 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:30.478+0000 7f737aa64740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f73701aacb0 con 0x5644f4fec0c0 2024-09-11T00:10:30.484 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:30.478+0000 7f737aa64740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f73701aa520 con 0x5644f4feeb00 2024-09-11T00:10:30.484 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:30.478+0000 7f737aa64740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f73701a7760 con 0x5644f4fe9790 2024-09-11T00:10:30.484 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:30.478+0000 7f737aa64740 10 client.0 _ll_register_callbacks cb 0x7f73680093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:10:30.484 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:30.478+0000 7f7375000640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5644f4fe9790 0x5644f5003310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:10:30.484 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:30.478+0000 7f7375000640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5644f4fe9790 0x5644f5003310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.12:46160/0 (socket says 172.21.15.12:46160) 2024-09-11T00:10:30.484 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:30.478+0000 7f7375000640 1 -- 172.21.15.12:0/684617214 learned_addr learned my addr 172.21.15.12:0/684617214 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:10:30.484 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:30.478+0000 7f7375a00640 1 --2- 172.21.15.12:0/684617214 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5644f4feeb00 0x5644f50026e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:10:30.484 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:30.478+0000 7f7367e00640 1 --2- 172.21.15.12:0/684617214 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5644f4fec0c0 0x5644f5002010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:10:30.484 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:30.479+0000 7f7375000640 1 -- 172.21.15.12:0/684617214 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5644f4feeb00 msgr2=0x5644f50026e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:10:30.484 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:30.479+0000 7f7375000640 1 --2- 172.21.15.12:0/684617214 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5644f4feeb00 0x5644f50026e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:10:30.485 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:30.479+0000 7f7375000640 1 -- 172.21.15.12:0/684617214 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5644f4fec0c0 msgr2=0x5644f5002010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:10:30.485 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:30.479+0000 7f7375000640 1 --2- 172.21.15.12:0/684617214 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5644f4fec0c0 0x5644f5002010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:10:30.485 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:30.479+0000 7f7375000640 1 -- 172.21.15.12:0/684617214 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5644f5000b80 con 0x5644f4fe9790 2024-09-11T00:10:30.485 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:30.479+0000 7f7375000640 1 --2- 172.21.15.12:0/684617214 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5644f4fe9790 0x5644f5003310 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f7360009a50 tx=0x7f7360004370 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5a7e9018ec56663 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:10:30.485 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:30.479+0000 7f735aa00640 10 client.0 ms_handle_connect on v2:172.21.15.178:3300/0 2024-09-11T00:10:30.485 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:30.479+0000 7f735aa00640 1 -- 172.21.15.12:0/684617214 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7360035030 con 0x5644f4fe9790 2024-09-11T00:10:30.485 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:30.479+0000 7f737aa64740 1 -- 172.21.15.12:0/684617214 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7360003cf0 con 0x5644f4fe9790 2024-09-11T00:10:30.485 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:30.479+0000 7f735aa00640 1 -- 172.21.15.12:0/684617214 <== mon.1 v2:172.21.15.178:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9208+0+0 (secure 0 0 0) 0x7f7360031930 con 0x5644f4fe9790 2024-09-11T00:10:30.485 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:30.479+0000 7f735aa00640 1 -- 172.21.15.12:0/684617214 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f7360031b30 con 0x5644f4fe9790 2024-09-11T00:10:30.485 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:30.479+0000 7f737aa64740 10 client.25909 fetch_fsmap learned FSMap version 33 2024-09-11T00:10:30.485 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:30.479+0000 7f737aa64740 1 -- 172.21.15.12:0/684617214 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x5644f5073e80 con 0x5644f4fe9790 2024-09-11T00:10:30.485 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:30.480+0000 7f735aa00640 1 -- 172.21.15.12:0/684617214 <== mon.1 v2:172.21.15.178:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7360003cf0 con 0x5644f4fe9790 2024-09-11T00:10:30.485 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:30.480+0000 7f735aa00640 1 -- 172.21.15.12:0/684617214 <== mon.1 v2:172.21.15.178:3300/0 5 ==== fsmap(e 33) ==== 8386+0+0 (secure 0 0 0) 0x7f7360044040 con 0x5644f4fe9790 2024-09-11T00:10:30.486 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:30.480+0000 7f737aa64740 10 client.25909 fetch_fsmap finished waiting for FSMap version 33 2024-09-11T00:10:30.486 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:30.480+0000 7f737aa64740 10 client.25909 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T00:10:30.486 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:30.480+0000 7f737aa64740 20 client.25909 populate_metadata read hostname 'smithi012' 2024-09-11T00:10:30.486 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:30.480+0000 7f737aa64740 1 --2- 172.21.15.12:0/684617214 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x5644f50748d0 0x5644f5076cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:10:30.486 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:30.480+0000 7f737aa64740 4 client.25909 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T00:10:30.486 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:30.480+0000 7f737aa64740 1 -- 172.21.15.12:0/684617214 --> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5644f4f5a0b0 con 0x5644f50748d0 2024-09-11T00:10:30.486 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:30.480+0000 7f7367e00640 1 --2- 172.21.15.12:0/684617214 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x5644f50748d0 0x5644f5076cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:10:30.486 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:30.481+0000 7f7367e00640 1 --2- 172.21.15.12:0/684617214 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x5644f50748d0 0x5644f5076cf0 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a3ed4526ccfb65f7 server_cookie=a3e96ff8d5c93ca7 in_seq=0 out_seq=0 2024-09-11T00:10:30.486 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:30.481+0000 7f735aa00640 10 client.25909 ms_handle_connect on v2:172.21.15.165:6838/501312984 2024-09-11T00:10:30.518 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:30.516+0000 7f735aa00640 1 -- 172.21.15.12:0/684617214 <== mds.0 v2:172.21.15.165:6838/501312984 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5644f4f5a0b0 con 0x5644f50748d0 2024-09-11T00:10:30.518 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:30.516+0000 7f735aa00640 10 client.25909 handle_command_reply: tid=0 2024-09-11T00:10:30.518 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:30.516+0000 7f737aa64740 1 client.25909 shutdown 2024-09-11T00:10:30.518 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:30.516+0000 7f737aa64740 1 -- 172.21.15.12:0/684617214 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5644f4fe9790 msgr2=0x5644f5003310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:10:30.519 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:30.516+0000 7f737aa64740 1 --2- 172.21.15.12:0/684617214 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5644f4fe9790 0x5644f5003310 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f7360009a50 tx=0x7f7360004370 comp rx=0 tx=0).stop 2024-09-11T00:10:30.519 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:30.516+0000 7f737aa64740 1 -- 172.21.15.12:0/684617214 shutdown_connections 2024-09-11T00:10:30.519 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:30.516+0000 7f737aa64740 1 --2- 172.21.15.12:0/684617214 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x5644f50748d0 0x5644f5076cf0 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:10:30.519 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:30.516+0000 7f737aa64740 1 --2- 172.21.15.12:0/684617214 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5644f4feeb00 0x5644f50026e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:10:30.519 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:30.516+0000 7f735aa00640 0 client.25909 ms_handle_reset on v2:172.21.15.165:6838/501312984 2024-09-11T00:10:30.519 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:30.516+0000 7f737aa64740 1 --2- 172.21.15.12:0/684617214 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5644f4fec0c0 0x5644f5002010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:10:30.519 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:30.516+0000 7f737aa64740 1 --2- 172.21.15.12:0/684617214 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5644f4fe9790 0x5644f5003310 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:10:30.519 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:30.516+0000 7f737aa64740 1 -- 172.21.15.12:0/684617214 >> 172.21.15.12:0/684617214 conn(0x5644f4fe48e0 msgr2=0x5644f4ffa740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:10:30.519 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:30.516+0000 7f737aa64740 1 -- 172.21.15.12:0/684617214 shutdown_connections 2024-09-11T00:10:30.519 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:30.516+0000 7f737aa64740 1 -- 172.21.15.12:0/684617214 wait complete. 2024-09-11T00:10:30.519 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:30.516+0000 7f737aa64740 20 client.25909 trim_cache size 0 max 16384 2024-09-11T00:10:30.520 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:{ 2024-09-11T00:10:30.520 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "message": "", 2024-09-11T00:10:30.520 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "return_code": 0 2024-09-11T00:10:30.520 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:} 2024-09-11T00:10:30.521 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:30.517+0000 7f7377a00640 1 -- 172.21.15.12:0/3334980094 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f73540b5270 msgr2=0x7f73540b7730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:10:30.521 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:30.517+0000 7f7377a00640 1 --2- 172.21.15.12:0/3334980094 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f73540b5270 0x7f73540b7730 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f7360000c00 tx=0x7f7360005a20 comp rx=0 tx=0).stop 2024-09-11T00:10:30.521 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:30.517+0000 7f7377a00640 1 -- 172.21.15.12:0/3334980094 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7370105cf0 msgr2=0x7f737019b640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:10:30.521 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:30.517+0000 7f7377a00640 1 --2- 172.21.15.12:0/3334980094 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7370105cf0 0x7f737019b640 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f735c00e9f0 tx=0x7f735c00eeb0 comp rx=0 tx=0).stop 2024-09-11T00:10:30.521 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:30.517+0000 7f7377a00640 1 -- 172.21.15.12:0/3334980094 shutdown_connections 2024-09-11T00:10:30.521 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:30.517+0000 7f7377a00640 1 --2- 172.21.15.12:0/3334980094 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f73540b5270 0x7f73540b7730 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:10:30.521 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:30.517+0000 7f7377a00640 1 --2- 172.21.15.12:0/3334980094 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f73701066b0 0x7f737019c270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:10:30.521 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:30.517+0000 7f7377a00640 1 --2- 172.21.15.12:0/3334980094 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7370105cf0 0x7f737019b640 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:10:30.521 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:30.517+0000 7f7377a00640 1 --2- 172.21.15.12:0/3334980094 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7370104ba0 0x7f737019aff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:10:30.522 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:30.517+0000 7f7377a00640 1 -- 172.21.15.12:0/3334980094 >> 172.21.15.12:0/3334980094 conn(0x7f7370100460 msgr2=0x7f7370107000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:10:30.522 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:30.517+0000 7f7377a00640 1 -- 172.21.15.12:0/3334980094 shutdown_connections 2024-09-11T00:10:30.522 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:30.517+0000 7f7377a00640 1 -- 172.21.15.12:0/3334980094 wait complete. 2024-09-11T00:10:30.622 INFO:tasks.workunit.client.0.smithi012.stdout: 1 113172 3.30 MB/sec execute 27 sec latency 30.924 ms 2024-09-11T00:10:31.623 INFO:tasks.workunit.client.0.smithi012.stdout: 1 114266 3.41 MB/sec execute 28 sec latency 186.862 ms 2024-09-11T00:10:31.782 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:10:31 smithi165 ceph-mon[35356]: pgmap v502: 129 pgs: 129 active+clean; 30 MiB data, 615 MiB used, 1.0 TiB / 1.0 TiB avail; 444 KiB/s rd, 1.3 MiB/s wr, 132 op/s 2024-09-11T00:10:31.812 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:10:31 smithi012 ceph-mon[30333]: pgmap v502: 129 pgs: 129 active+clean; 30 MiB data, 615 MiB used, 1.0 TiB / 1.0 TiB avail; 444 KiB/s rd, 1.3 MiB/s wr, 132 op/s 2024-09-11T00:10:31.830 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:10:31 smithi178 ceph-mon[35919]: pgmap v502: 129 pgs: 129 active+clean; 30 MiB data, 615 MiB used, 1.0 TiB / 1.0 TiB avail; 444 KiB/s rd, 1.3 MiB/s wr, 132 op/s 2024-09-11T00:10:32.623 INFO:tasks.workunit.client.0.smithi012.stdout: 1 114266 3.29 MB/sec execute 29 sec latency 1187.019 ms 2024-09-11T00:10:33.623 INFO:tasks.workunit.client.0.smithi012.stdout: 1 114266 3.18 MB/sec execute 30 sec latency 2187.123 ms 2024-09-11T00:10:33.782 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:10:33 smithi165 ceph-mon[35356]: pgmap v503: 129 pgs: 129 active+clean; 31 MiB data, 642 MiB used, 1.0 TiB / 1.0 TiB avail; 290 KiB/s rd, 1.3 MiB/s wr, 132 op/s 2024-09-11T00:10:33.812 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:10:33 smithi012 ceph-mon[30333]: pgmap v503: 129 pgs: 129 active+clean; 31 MiB data, 642 MiB used, 1.0 TiB / 1.0 TiB avail; 290 KiB/s rd, 1.3 MiB/s wr, 132 op/s 2024-09-11T00:10:33.830 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:10:33 smithi178 ceph-mon[35919]: pgmap v503: 129 pgs: 129 active+clean; 31 MiB data, 642 MiB used, 1.0 TiB / 1.0 TiB avail; 290 KiB/s rd, 1.3 MiB/s wr, 132 op/s 2024-09-11T00:10:34.623 INFO:tasks.workunit.client.0.smithi012.stdout: 1 114356 3.09 MB/sec execute 31 sec latency 3099.501 ms 2024-09-11T00:10:35.623 INFO:tasks.workunit.client.0.smithi012.stdout: 1 116869 3.42 MB/sec execute 32 sec latency 12.807 ms 2024-09-11T00:10:35.782 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:10:35 smithi165 ceph-mon[35356]: pgmap v504: 129 pgs: 129 active+clean; 31 MiB data, 643 MiB used, 1.0 TiB / 1.0 TiB avail; 246 KiB/s rd, 1.3 MiB/s wr, 114 op/s 2024-09-11T00:10:35.812 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:10:35 smithi012 ceph-mon[30333]: pgmap v504: 129 pgs: 129 active+clean; 31 MiB data, 643 MiB used, 1.0 TiB / 1.0 TiB avail; 246 KiB/s rd, 1.3 MiB/s wr, 114 op/s 2024-09-11T00:10:35.830 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:10:35 smithi178 ceph-mon[35919]: pgmap v504: 129 pgs: 129 active+clean; 31 MiB data, 643 MiB used, 1.0 TiB / 1.0 TiB avail; 246 KiB/s rd, 1.3 MiB/s wr, 114 op/s 2024-09-11T00:10:36.623 INFO:tasks.workunit.client.0.smithi012.stdout: 1 117812 3.47 MB/sec execute 33 sec latency 169.463 ms 2024-09-11T00:10:37.623 INFO:tasks.workunit.client.0.smithi012.stdout: 1 117812 3.37 MB/sec execute 34 sec latency 1169.584 ms 2024-09-11T00:10:37.782 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:10:37 smithi165 ceph-mon[35356]: pgmap v505: 129 pgs: 129 active+clean; 32 MiB data, 634 MiB used, 1.0 TiB / 1.0 TiB avail; 363 KiB/s rd, 1.3 MiB/s wr, 144 op/s 2024-09-11T00:10:37.813 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:10:37 smithi012 ceph-mon[30333]: pgmap v505: 129 pgs: 129 active+clean; 32 MiB data, 634 MiB used, 1.0 TiB / 1.0 TiB avail; 363 KiB/s rd, 1.3 MiB/s wr, 144 op/s 2024-09-11T00:10:37.830 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:10:37 smithi178 ceph-mon[35919]: pgmap v505: 129 pgs: 129 active+clean; 32 MiB data, 634 MiB used, 1.0 TiB / 1.0 TiB avail; 363 KiB/s rd, 1.3 MiB/s wr, 144 op/s 2024-09-11T00:10:38.623 INFO:tasks.workunit.client.0.smithi012.stdout: 1 117812 3.27 MB/sec execute 35 sec latency 2169.752 ms 2024-09-11T00:10:38.813 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:10:38 smithi012 ceph-mon[30333]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T00:10:38.830 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:10:38 smithi178 ceph-mon[35919]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T00:10:39.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:10:38 smithi165 ceph-mon[35356]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T00:10:39.623 INFO:tasks.workunit.client.0.smithi012.stdout: 1 117880 3.19 MB/sec execute 36 sec latency 3084.669 ms 2024-09-11T00:10:39.813 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:10:39 smithi012 ceph-mon[30333]: pgmap v506: 129 pgs: 129 active+clean; 33 MiB data, 635 MiB used, 1.0 TiB / 1.0 TiB avail; 295 KiB/s rd, 355 KiB/s wr, 103 op/s 2024-09-11T00:10:39.830 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:10:39 smithi178 ceph-mon[35919]: pgmap v506: 129 pgs: 129 active+clean; 33 MiB data, 635 MiB used, 1.0 TiB / 1.0 TiB avail; 295 KiB/s rd, 355 KiB/s wr, 103 op/s 2024-09-11T00:10:40.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:10:39 smithi165 ceph-mon[35356]: pgmap v506: 129 pgs: 129 active+clean; 33 MiB data, 635 MiB used, 1.0 TiB / 1.0 TiB avail; 295 KiB/s rd, 355 KiB/s wr, 103 op/s 2024-09-11T00:10:40.624 INFO:tasks.workunit.client.0.smithi012.stdout: 1 117880 3.10 MB/sec execute 37 sec latency 1030.265 ms 2024-09-11T00:10:41.624 INFO:tasks.workunit.client.0.smithi012.stdout: 1 117880 3.02 MB/sec execute 38 sec latency 2030.413 ms 2024-09-11T00:10:41.813 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:10:41 smithi012 ceph-mon[30333]: pgmap v507: 129 pgs: 129 active+clean; 33 MiB data, 655 MiB used, 1.0 TiB / 1.0 TiB avail; 391 KiB/s rd, 390 KiB/s wr, 103 op/s 2024-09-11T00:10:41.830 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:10:41 smithi178 ceph-mon[35919]: pgmap v507: 129 pgs: 129 active+clean; 33 MiB data, 655 MiB used, 1.0 TiB / 1.0 TiB avail; 391 KiB/s rd, 390 KiB/s wr, 103 op/s 2024-09-11T00:10:42.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:10:41 smithi165 ceph-mon[35356]: pgmap v507: 129 pgs: 129 active+clean; 33 MiB data, 655 MiB used, 1.0 TiB / 1.0 TiB avail; 391 KiB/s rd, 390 KiB/s wr, 103 op/s 2024-09-11T00:10:42.624 INFO:tasks.workunit.client.0.smithi012.stdout: 1 117880 2.94 MB/sec execute 39 sec latency 3030.544 ms 2024-09-11T00:10:43.624 INFO:tasks.workunit.client.0.smithi012.stdout: 1 117880 2.87 MB/sec execute 40 sec latency 4030.614 ms 2024-09-11T00:10:43.647 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:43.645+0000 7f49d7000640 1 -- 172.21.15.12:0/482985137 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f49d01066a0 msgr2=0x7f49d010d020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:10:43.647 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:43.645+0000 7f49d7000640 1 --2- 172.21.15.12:0/482985137 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f49d01066a0 0x7f49d010d020 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f49c8009cd0 tx=0x7f49c802a8f0 comp rx=0 tx=0).stop 2024-09-11T00:10:43.647 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:43.645+0000 7f49d7000640 1 -- 172.21.15.12:0/482985137 shutdown_connections 2024-09-11T00:10:43.647 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:43.645+0000 7f49d7000640 1 --2- 172.21.15.12:0/482985137 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f49d01066a0 0x7f49d010d020 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:10:43.647 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:43.645+0000 7f49d7000640 1 --2- 172.21.15.12:0/482985137 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f49d0105ce0 0x7f49d0106160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:10:43.647 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:43.645+0000 7f49d7000640 1 --2- 172.21.15.12:0/482985137 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f49d0104b90 0x7f49d0104f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:10:43.648 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:43.645+0000 7f49d7000640 1 -- 172.21.15.12:0/482985137 >> 172.21.15.12:0/482985137 conn(0x7f49d0100470 msgr2=0x7f49d0102890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:10:43.648 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:43.645+0000 7f49d7000640 1 -- 172.21.15.12:0/482985137 shutdown_connections 2024-09-11T00:10:43.648 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:43.645+0000 7f49d7000640 1 -- 172.21.15.12:0/482985137 wait complete. 2024-09-11T00:10:43.649 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:43.646+0000 7f49d7000640 1 Processor -- start 2024-09-11T00:10:43.649 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:43.646+0000 7f49d7000640 1 -- start start 2024-09-11T00:10:43.649 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:43.646+0000 7f49d7000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f49d0104b90 0x7f49d019af30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:10:43.649 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:43.646+0000 7f49d7000640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f49d0105ce0 0x7f49d019b580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:10:43.649 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:43.647+0000 7f49d7000640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f49d01066a0 0x7f49d019c1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:10:43.649 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:43.647+0000 7f49d7000640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f49d0110800 con 0x7f49d01066a0 2024-09-11T00:10:43.649 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:43.647+0000 7f49d7000640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f49d0110680 con 0x7f49d0104b90 2024-09-11T00:10:43.649 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:43.647+0000 7f49d7000640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f49d0110980 con 0x7f49d0105ce0 2024-09-11T00:10:43.649 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:43.647+0000 7f49d5000640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f49d01066a0 0x7f49d019c1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:10:43.650 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:43.647+0000 7f49cfe00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f49d0104b90 0x7f49d019af30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:10:43.650 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:43.647+0000 7f49d5000640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f49d01066a0 0x7f49d019c1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:32902/0 (socket says 172.21.15.12:32902) 2024-09-11T00:10:43.650 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:43.647+0000 7f49d5000640 1 -- 172.21.15.12:0/601876653 learned_addr learned my addr 172.21.15.12:0/601876653 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:10:43.650 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:43.647+0000 7f49c7400640 1 --2- 172.21.15.12:0/601876653 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f49d0105ce0 0x7f49d019b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:10:43.651 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:43.647+0000 7f49cfe00640 1 -- 172.21.15.12:0/601876653 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f49d0105ce0 msgr2=0x7f49d019b580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:10:43.651 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:43.647+0000 7f49cfe00640 1 --2- 172.21.15.12:0/601876653 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f49d0105ce0 0x7f49d019b580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:10:43.651 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:43.647+0000 7f49cfe00640 1 -- 172.21.15.12:0/601876653 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f49d01066a0 msgr2=0x7f49d019c1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:10:43.651 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:43.647+0000 7f49cfe00640 1 --2- 172.21.15.12:0/601876653 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f49d01066a0 0x7f49d019c1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:10:43.651 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:43.647+0000 7f49cfe00640 1 -- 172.21.15.12:0/601876653 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49d019ca70 con 0x7f49d0104b90 2024-09-11T00:10:43.651 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:43.647+0000 7f49d5000640 1 --2- 172.21.15.12:0/601876653 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f49d01066a0 0x7f49d019c1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:10:43.651 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:43.648+0000 7f49cfe00640 1 --2- 172.21.15.12:0/601876653 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f49d0104b90 0x7f49d019af30 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f49c000e9f0 tx=0x7f49c000eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:10:43.651 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:43.648+0000 7f49cd600640 1 -- 172.21.15.12:0/601876653 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49c000ccc0 con 0x7f49d0104b90 2024-09-11T00:10:43.651 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:43.648+0000 7f49cd600640 1 -- 172.21.15.12:0/601876653 <== mon.2 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f49c00108e0 con 0x7f49d0104b90 2024-09-11T00:10:43.652 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:43.648+0000 7f49cd600640 1 -- 172.21.15.12:0/601876653 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49c0018880 con 0x7f49d0104b90 2024-09-11T00:10:43.652 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:43.648+0000 7f49d7000640 1 -- 172.21.15.12:0/601876653 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49d019cd60 con 0x7f49d0104b90 2024-09-11T00:10:43.652 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:43.648+0000 7f49d7000640 1 -- 172.21.15.12:0/601876653 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49d01a3430 con 0x7f49d0104b90 2024-09-11T00:10:43.653 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:43.651+0000 7f49d9df5740 1 Processor -- start 2024-09-11T00:10:43.653 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:43.651+0000 7f49d9df5740 1 -- start start 2024-09-11T00:10:43.654 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:43.652+0000 7f49d9df5740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f72cb38790 0x55f72cb3ab80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:10:43.654 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:43.652+0000 7f49d9df5740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f72cb3b0c0 0x55f72cb3d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:10:43.654 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:43.652+0000 7f49d9df5740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f72cb3db00 0x55f72cb3ffc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:10:43.654 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:43.652+0000 7f49d9df5740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55f72caab6b0 con 0x55f72cb3b0c0 2024-09-11T00:10:43.654 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:43.652+0000 7f49d9df5740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55f72ca9aae0 con 0x55f72cb38790 2024-09-11T00:10:43.654 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:43.652+0000 7f49d9df5740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55f72ca9e8f0 con 0x55f72cb3db00 2024-09-11T00:10:43.654 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:43.652+0000 7f49cd600640 1 -- 172.21.15.12:0/601876653 <== mon.2 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7f49c0010430 con 0x7f49d0104b90 2024-09-11T00:10:43.655 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:43.652+0000 7f49d5000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f72cb38790 0x55f72cb3ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:10:43.655 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:43.652+0000 7f49c7400640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f72cb3db00 0x55f72cb3ffc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:10:43.655 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:43.652+0000 7f49d5000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f72cb38790 0x55f72cb3ab80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.12:42654/0 (socket says 172.21.15.12:42654) 2024-09-11T00:10:43.655 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:43.652+0000 7f49d5000640 1 -- 172.21.15.12:0/4141622315 learned_addr learned my addr 172.21.15.12:0/4141622315 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:10:43.655 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:43.652+0000 7f49cfe00640 1 --2- 172.21.15.12:0/4141622315 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f72cb3b0c0 0x55f72cb3d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:10:43.657 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:43.652+0000 7f49d5000640 1 -- 172.21.15.12:0/4141622315 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f72cb3db00 msgr2=0x55f72cb3ffc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:10:43.657 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:43.652+0000 7f49d5000640 1 --2- 172.21.15.12:0/4141622315 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f72cb3db00 0x55f72cb3ffc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:10:43.657 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:43.652+0000 7f49d5000640 1 -- 172.21.15.12:0/4141622315 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f72cb3b0c0 msgr2=0x55f72cb3d4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:10:43.657 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:43.652+0000 7f49d5000640 1 --2- 172.21.15.12:0/4141622315 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f72cb3b0c0 0x55f72cb3d4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:10:43.657 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:43.652+0000 7f49d5000640 1 -- 172.21.15.12:0/4141622315 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f72ca67780 con 0x55f72cb38790 2024-09-11T00:10:43.657 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:43.653+0000 7f49c7400640 1 --2- 172.21.15.12:0/4141622315 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f72cb3db00 0x55f72cb3ffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:10:43.657 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:43.653+0000 7f49d5000640 1 --2- 172.21.15.12:0/4141622315 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f72cb38790 0x55f72cb3ab80 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f49c8009a20 tx=0x7f49c8030f50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ce5b23084948de1e server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:10:43.657 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:43.653+0000 7f49c5600640 1 -- 172.21.15.12:0/4141622315 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49c8003970 con 0x55f72cb38790 2024-09-11T00:10:43.657 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:43.653+0000 7f49c5600640 1 -- 172.21.15.12:0/4141622315 <== mon.2 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f49c8036030 con 0x55f72cb38790 2024-09-11T00:10:43.657 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:43.653+0000 7f49c5600640 1 -- 172.21.15.12:0/4141622315 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49c8031440 con 0x55f72cb38790 2024-09-11T00:10:43.658 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:43.653+0000 7f49cd600640 1 --2- 172.21.15.12:0/601876653 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f49b40b52e0 0x7f49b40b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:10:43.658 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:43.654+0000 7f49cd600640 1 -- 172.21.15.12:0/601876653 <== mon.2 v2:172.21.15.165:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9208+0+0 (secure 0 0 0) 0x7f49c00e6320 con 0x7f49d0104b90 2024-09-11T00:10:43.658 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:43.654+0000 7f49cfe00640 1 --2- 172.21.15.12:0/601876653 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f49b40b52e0 0x7f49b40b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:10:43.658 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:43.654+0000 7f49d9df5740 1 -- 172.21.15.12:0/4141622315 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f72cb38790 msgr2=0x55f72cb3ab80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:10:43.658 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:43.654+0000 7f49d9df5740 1 --2- 172.21.15.12:0/4141622315 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f72cb38790 0x55f72cb3ab80 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f49c8009a20 tx=0x7f49c8030f50 comp rx=0 tx=0).stop 2024-09-11T00:10:43.658 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:43.654+0000 7f49d9df5740 1 -- 172.21.15.12:0/4141622315 shutdown_connections 2024-09-11T00:10:43.658 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:43.654+0000 7f49d9df5740 1 --2- 172.21.15.12:0/4141622315 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f72cb3db00 0x55f72cb3ffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:10:43.658 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:43.654+0000 7f49d9df5740 1 --2- 172.21.15.12:0/4141622315 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f72cb3b0c0 0x55f72cb3d4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:10:43.658 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:43.654+0000 7f49d9df5740 1 --2- 172.21.15.12:0/4141622315 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f72cb38790 0x55f72cb3ab80 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:10:43.658 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:43.654+0000 7f49d9df5740 1 -- 172.21.15.12:0/4141622315 >> 172.21.15.12:0/4141622315 conn(0x55f72cb338e0 msgr2=0x55f72cb34670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:10:43.658 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:43.654+0000 7f49d9df5740 1 -- 172.21.15.12:0/4141622315 shutdown_connections 2024-09-11T00:10:43.658 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:43.654+0000 7f49d9df5740 1 -- 172.21.15.12:0/4141622315 wait complete. 2024-09-11T00:10:43.658 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:43.654+0000 7f49cfe00640 1 --2- 172.21.15.12:0/601876653 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f49b40b52e0 0x7f49b40b77a0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f49c00ac6f0 tx=0x7f49c00e9d60 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:10:43.658 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:43.655+0000 7f49d9df5740 1 -- start start 2024-09-11T00:10:43.659 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:43.655+0000 7f49d9df5740 1 Processor -- start 2024-09-11T00:10:43.661 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:43.655+0000 7f49d9df5740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f72cb38790 0x55f72cb41e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:10:43.661 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:43.655+0000 7f49d9df5740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f72cb3b0c0 0x55f72cb42500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:10:43.661 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:43.655+0000 7f49d9df5740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f72cb3db00 0x55f72cb430d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:10:43.661 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:43.655+0000 7f49d9df5740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55f72c9d2ab0 con 0x55f72cb3b0c0 2024-09-11T00:10:43.661 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:43.655+0000 7f49d9df5740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55f72ca41570 con 0x55f72cb3db00 2024-09-11T00:10:43.661 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:43.655+0000 7f49d9df5740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55f72cb40a80 con 0x55f72cb38790 2024-09-11T00:10:43.661 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:43.656+0000 7f49d9df5740 10 client.0 _ll_register_callbacks cb 0x55f72ca50100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:10:43.661 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:43.656+0000 7f49c7400640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f72cb3b0c0 0x55f72cb42500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:10:43.662 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:43.656+0000 7f49c7400640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f72cb3b0c0 0x55f72cb42500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:32920/0 (socket says 172.21.15.12:32920) 2024-09-11T00:10:43.662 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:43.656+0000 7f49c7400640 1 -- 172.21.15.12:0/433968798 learned_addr learned my addr 172.21.15.12:0/433968798 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:10:43.662 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:43.656+0000 7f49d5000640 1 --2- 172.21.15.12:0/433968798 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f72cb3db00 0x55f72cb430d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:10:43.662 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:43.656+0000 7f49d5000640 1 --2- 172.21.15.12:0/433968798 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f72cb38790 0x55f72cb41e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:10:43.662 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:43.656+0000 7f49c7400640 1 -- 172.21.15.12:0/433968798 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f72cb38790 msgr2=0x55f72cb41e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:10:43.662 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:43.656+0000 7f49c7400640 1 --2- 172.21.15.12:0/433968798 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f72cb38790 0x55f72cb41e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:10:43.662 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:43.656+0000 7f49c7400640 1 -- 172.21.15.12:0/433968798 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f72cb3db00 msgr2=0x55f72cb430d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:10:43.662 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:43.656+0000 7f49c7400640 1 --2- 172.21.15.12:0/433968798 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f72cb3db00 0x55f72cb430d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:10:43.662 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:43.656+0000 7f49c7400640 1 -- 172.21.15.12:0/433968798 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f72c7fc790 con 0x55f72cb3b0c0 2024-09-11T00:10:43.662 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:43.656+0000 7f49c7400640 1 --2- 172.21.15.12:0/433968798 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f72cb3b0c0 0x55f72cb42500 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f49b80126f0 tx=0x7f49b8012bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9970c3a28eb71d68 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:10:43.662 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:43.656+0000 7f49be000640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-09-11T00:10:43.662 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:43.657+0000 7f49be000640 1 -- 172.21.15.12:0/433968798 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49b800f920 con 0x55f72cb3b0c0 2024-09-11T00:10:43.662 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:43.657+0000 7f49be000640 1 -- 172.21.15.12:0/433968798 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9208+0+0 (secure 0 0 0) 0x7f49b8012e00 con 0x55f72cb3b0c0 2024-09-11T00:10:43.663 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:43.657+0000 7f49d9df5740 1 -- 172.21.15.12:0/433968798 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f49c8036030 con 0x55f72cb3b0c0 2024-09-11T00:10:43.663 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:43.657+0000 7f49be000640 1 -- 172.21.15.12:0/433968798 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f49b800fac0 con 0x55f72cb3b0c0 2024-09-11T00:10:43.663 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:43.657+0000 7f49d9df5740 10 client.15909 fetch_fsmap learned FSMap version 34 2024-09-11T00:10:43.663 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:43.657+0000 7f49d9df5740 1 -- 172.21.15.12:0/433968798 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55f72c875620 con 0x55f72cb3b0c0 2024-09-11T00:10:43.663 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:43.657+0000 7f49be000640 1 -- 172.21.15.12:0/433968798 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f49c8036030 con 0x55f72cb3b0c0 2024-09-11T00:10:43.663 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:43.658+0000 7f49be000640 1 -- 172.21.15.12:0/433968798 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 34) ==== 8382+0+0 (secure 0 0 0) 0x7f49b801c420 con 0x55f72cb3b0c0 2024-09-11T00:10:43.663 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:43.658+0000 7f49d9df5740 10 client.15909 fetch_fsmap finished waiting for FSMap version 34 2024-09-11T00:10:43.663 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:43.658+0000 7f49d9df5740 10 client.15909 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T00:10:43.663 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:43.658+0000 7f49d9df5740 20 client.15909 populate_metadata read hostname 'smithi012' 2024-09-11T00:10:43.663 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:43.658+0000 7f49d9df5740 1 --2- 172.21.15.12:0/433968798 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55f72cb4d8b0 0x55f72cb4fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:10:43.663 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:43.658+0000 7f49d9df5740 4 client.15909 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T00:10:43.663 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:43.658+0000 7f49d9df5740 1 -- 172.21.15.12:0/433968798 --> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f72caad600 con 0x55f72cb4d8b0 2024-09-11T00:10:43.663 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:43.658+0000 7f49d5000640 1 --2- 172.21.15.12:0/433968798 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55f72cb4d8b0 0x55f72cb4fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:10:43.663 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:43.659+0000 7f49d5000640 1 --2- 172.21.15.12:0/433968798 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55f72cb4d8b0 0x55f72cb4fcd0 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cc8cde2c366c1f06 server_cookie=c1775c1600a3374c in_seq=0 out_seq=0 2024-09-11T00:10:43.664 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:43.659+0000 7f49be000640 10 client.15909 ms_handle_connect on v2:172.21.15.165:6838/501312984 2024-09-11T00:10:43.665 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:43.660+0000 7f49be000640 1 -- 172.21.15.12:0/433968798 <== mds.0 v2:172.21.15.165:6838/501312984 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55f72caad600 con 0x55f72cb4d8b0 2024-09-11T00:10:43.665 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:43.660+0000 7f49be000640 10 client.15909 handle_command_reply: tid=0 2024-09-11T00:10:43.665 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:43.660+0000 7f49d9df5740 1 client.15909 shutdown 2024-09-11T00:10:43.665 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:43.660+0000 7f49d9df5740 1 -- 172.21.15.12:0/433968798 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f72cb3b0c0 msgr2=0x55f72cb42500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:10:43.665 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:43.660+0000 7f49d9df5740 1 --2- 172.21.15.12:0/433968798 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f72cb3b0c0 0x55f72cb42500 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f49b80126f0 tx=0x7f49b8012bb0 comp rx=0 tx=0).stop 2024-09-11T00:10:43.665 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:43.660+0000 7f49d9df5740 1 -- 172.21.15.12:0/433968798 shutdown_connections 2024-09-11T00:10:43.665 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:43.660+0000 7f49d9df5740 1 --2- 172.21.15.12:0/433968798 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55f72cb4d8b0 0x55f72cb4fcd0 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:10:43.665 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:43.660+0000 7f49d9df5740 1 --2- 172.21.15.12:0/433968798 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f72cb3db00 0x55f72cb430d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:10:43.665 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:43.660+0000 7f49d9df5740 1 --2- 172.21.15.12:0/433968798 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f72cb3b0c0 0x55f72cb42500 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:10:43.665 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:43.660+0000 7f49d9df5740 1 --2- 172.21.15.12:0/433968798 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f72cb38790 0x55f72cb41e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:10:43.665 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:43.660+0000 7f49be000640 0 client.15909 ms_handle_reset on v2:172.21.15.165:6838/501312984 2024-09-11T00:10:43.665 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:43.660+0000 7f49d9df5740 1 -- 172.21.15.12:0/433968798 >> 172.21.15.12:0/433968798 conn(0x55f72cb338e0 msgr2=0x55f72cb32a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:10:43.665 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:43.660+0000 7f49d9df5740 1 -- 172.21.15.12:0/433968798 shutdown_connections 2024-09-11T00:10:43.666 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:43.660+0000 7f49d9df5740 1 -- 172.21.15.12:0/433968798 wait complete. 2024-09-11T00:10:43.666 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:43.660+0000 7f49d9df5740 20 client.15909 trim_cache size 0 max 16384 2024-09-11T00:10:43.678 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:43.676+0000 7f49d9df5740 1 Processor -- start 2024-09-11T00:10:43.678 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:43.676+0000 7f49d9df5740 1 -- start start 2024-09-11T00:10:43.678 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:43.676+0000 7f49d9df5740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f72cb38790 0x55f72cb40e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:10:43.678 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:43.676+0000 7f49d9df5740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f72cb3b0c0 0x55f72cb41390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:10:43.678 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:43.676+0000 7f49d9df5740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f72cb3db00 0x55f72cb41e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:10:43.678 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:43.676+0000 7f49d9df5740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f49d01ad900 con 0x55f72cb3b0c0 2024-09-11T00:10:43.679 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:43.676+0000 7f49d9df5740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f49d01af4d0 con 0x55f72cb3db00 2024-09-11T00:10:43.679 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:43.676+0000 7f49d9df5740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f49d01adfd0 con 0x55f72cb38790 2024-09-11T00:10:43.680 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:43.676+0000 7f49c7400640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f72cb3b0c0 0x55f72cb41390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:10:43.680 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:43.676+0000 7f49c7400640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f72cb3b0c0 0x55f72cb41390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:32924/0 (socket says 172.21.15.12:32924) 2024-09-11T00:10:43.680 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:43.676+0000 7f49d5000640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f72cb38790 0x55f72cb40e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:10:43.681 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:43.676+0000 7f49c7400640 1 -- 172.21.15.12:0/1840193307 learned_addr learned my addr 172.21.15.12:0/1840193307 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:10:43.681 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:43.677+0000 7f49d5000640 1 --2- 172.21.15.12:0/1840193307 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f72cb3db00 0x55f72cb41e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:10:43.681 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:43.677+0000 7f49c7400640 1 -- 172.21.15.12:0/1840193307 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f72cb38790 msgr2=0x55f72cb40e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:10:43.681 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:43.677+0000 7f49c7400640 1 --2- 172.21.15.12:0/1840193307 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f72cb38790 0x55f72cb40e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:10:43.681 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:43.677+0000 7f49c7400640 1 -- 172.21.15.12:0/1840193307 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f72cb3db00 msgr2=0x55f72cb41e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:10:43.681 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:43.677+0000 7f49c7400640 1 --2- 172.21.15.12:0/1840193307 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f72cb3db00 0x55f72cb41e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:10:43.681 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:43.677+0000 7f49c7400640 1 -- 172.21.15.12:0/1840193307 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f72cb436b0 con 0x55f72cb3b0c0 2024-09-11T00:10:43.681 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:43.677+0000 7f49c7400640 1 --2- 172.21.15.12:0/1840193307 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f72cb3b0c0 0x55f72cb41390 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f49b8000c00 tx=0x7f49b800f810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86311804e97dd5a3 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:10:43.681 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:43.677+0000 7f49c4c00640 1 -- 172.21.15.12:0/1840193307 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49b801cb00 con 0x55f72cb3b0c0 2024-09-11T00:10:43.681 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:43.677+0000 7f49c4c00640 1 -- 172.21.15.12:0/1840193307 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f49b801cca0 con 0x55f72cb3b0c0 2024-09-11T00:10:43.681 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:43.677+0000 7f49c4c00640 1 -- 172.21.15.12:0/1840193307 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49b8005030 con 0x55f72cb3b0c0 2024-09-11T00:10:43.681 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:43.678+0000 7f49d9df5740 1 -- 172.21.15.12:0/1840193307 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f72cb3b0c0 msgr2=0x55f72cb41390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:10:43.681 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:43.678+0000 7f49d9df5740 1 --2- 172.21.15.12:0/1840193307 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f72cb3b0c0 0x55f72cb41390 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f49b8000c00 tx=0x7f49b800f810 comp rx=0 tx=0).stop 2024-09-11T00:10:43.682 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:43.678+0000 7f49d9df5740 1 -- 172.21.15.12:0/1840193307 shutdown_connections 2024-09-11T00:10:43.682 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:43.678+0000 7f49d9df5740 1 --2- 172.21.15.12:0/1840193307 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f72cb3db00 0x55f72cb41e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:10:43.682 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:43.678+0000 7f49d9df5740 1 --2- 172.21.15.12:0/1840193307 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f72cb3b0c0 0x55f72cb41390 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:10:43.682 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:43.678+0000 7f49d9df5740 1 --2- 172.21.15.12:0/1840193307 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f72cb38790 0x55f72cb40e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:10:43.682 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:43.678+0000 7f49d9df5740 1 -- 172.21.15.12:0/1840193307 >> 172.21.15.12:0/1840193307 conn(0x55f72cb338e0 msgr2=0x55f72cb36930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:10:43.682 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:43.678+0000 7f49d9df5740 1 -- 172.21.15.12:0/1840193307 shutdown_connections 2024-09-11T00:10:43.682 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:43.678+0000 7f49d9df5740 1 -- 172.21.15.12:0/1840193307 wait complete. 2024-09-11T00:10:43.682 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:43.678+0000 7f49d9df5740 1 -- start start 2024-09-11T00:10:43.682 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:43.679+0000 7f49d9df5740 1 Processor -- start 2024-09-11T00:10:43.684 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:43.679+0000 7f49d9df5740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f72cb38790 0x55f72cb48b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:10:43.684 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:43.679+0000 7f49d9df5740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f72cb3db00 0x55f72cb49070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:10:43.684 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:43.679+0000 7f49d9df5740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f72cb495b0 0x55f72cb47650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:10:43.684 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:43.679+0000 7f49d9df5740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f49d01aee00 con 0x55f72cb495b0 2024-09-11T00:10:43.685 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:43.679+0000 7f49d9df5740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f49d01ae6a0 con 0x55f72cb3db00 2024-09-11T00:10:43.685 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:43.679+0000 7f49d9df5740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f49d01ab8e0 con 0x55f72cb38790 2024-09-11T00:10:43.685 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:43.679+0000 7f49d9df5740 10 client.0 _ll_register_callbacks cb 0x7f49b8009830 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:10:43.685 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:43.679+0000 7f49d5000640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f72cb495b0 0x55f72cb47650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:10:43.685 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:43.679+0000 7f49d5000640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f72cb495b0 0x55f72cb47650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:32928/0 (socket says 172.21.15.12:32928) 2024-09-11T00:10:43.685 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:43.679+0000 7f49c7400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f72cb3db00 0x55f72cb49070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:10:43.685 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:43.679+0000 7f49d5000640 1 -- 172.21.15.12:0/1459963901 learned_addr learned my addr 172.21.15.12:0/1459963901 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:10:43.685 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:43.679+0000 7f49d5000640 1 --2- 172.21.15.12:0/1459963901 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f72cb38790 0x55f72cb48b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:10:43.685 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:43.680+0000 7f49c7400640 1 -- 172.21.15.12:0/1459963901 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f72cb38790 msgr2=0x55f72cb48b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:10:43.685 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:43.680+0000 7f49c7400640 1 --2- 172.21.15.12:0/1459963901 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f72cb38790 0x55f72cb48b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:10:43.685 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:43.680+0000 7f49c7400640 1 -- 172.21.15.12:0/1459963901 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f72cb495b0 msgr2=0x55f72cb47650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:10:43.685 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:43.680+0000 7f49c7400640 1 --2- 172.21.15.12:0/1459963901 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f72cb495b0 0x55f72cb47650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:10:43.685 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:43.680+0000 7f49c7400640 1 -- 172.21.15.12:0/1459963901 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f72cb47d40 con 0x55f72cb3db00 2024-09-11T00:10:43.686 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:43.680+0000 7f49c7400640 1 --2- 172.21.15.12:0/1459963901 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f72cb3db00 0x55f72cb49070 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x55f72cb34d80 tx=0x7f49b8005260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8066f08c4adb7bb8 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:10:43.686 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:43.680+0000 7f49be000640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-09-11T00:10:43.686 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:43.680+0000 7f49be000640 1 -- 172.21.15.12:0/1459963901 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49b801cb00 con 0x55f72cb3db00 2024-09-11T00:10:43.686 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:43.680+0000 7f49d9df5740 1 -- 172.21.15.12:0/1459963901 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f49b801cca0 con 0x55f72cb3db00 2024-09-11T00:10:43.686 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:43.680+0000 7f49be000640 1 -- 172.21.15.12:0/1459963901 <== mon.2 v2:172.21.15.165:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9208+0+0 (secure 0 0 0) 0x7f49b8018040 con 0x55f72cb3db00 2024-09-11T00:10:43.686 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:43.681+0000 7f49be000640 1 -- 172.21.15.12:0/1459963901 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f49b801c670 con 0x55f72cb3db00 2024-09-11T00:10:43.686 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:43.681+0000 7f49d9df5740 10 client.25937 fetch_fsmap learned FSMap version 34 2024-09-11T00:10:43.686 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:43.681+0000 7f49d9df5740 1 -- 172.21.15.12:0/1459963901 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55f72cbbb150 con 0x55f72cb3db00 2024-09-11T00:10:43.686 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:43.681+0000 7f49be000640 1 -- 172.21.15.12:0/1459963901 <== mon.2 v2:172.21.15.165:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f49b801cca0 con 0x55f72cb3db00 2024-09-11T00:10:43.686 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:43.681+0000 7f49be000640 1 -- 172.21.15.12:0/1459963901 <== mon.2 v2:172.21.15.165:3300/0 5 ==== fsmap(e 34) ==== 8382+0+0 (secure 0 0 0) 0x7f49b802a440 con 0x55f72cb3db00 2024-09-11T00:10:43.686 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:43.681+0000 7f49d9df5740 10 client.25937 fetch_fsmap finished waiting for FSMap version 34 2024-09-11T00:10:43.686 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:43.681+0000 7f49d9df5740 10 client.25937 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T00:10:43.686 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:43.681+0000 7f49d9df5740 20 client.25937 populate_metadata read hostname 'smithi012' 2024-09-11T00:10:43.687 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:43.681+0000 7f49d9df5740 1 --2- 172.21.15.12:0/1459963901 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55f72cbbbae0 0x55f72cbbdf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:10:43.687 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:43.681+0000 7f49d9df5740 4 client.25937 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T00:10:43.687 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:43.681+0000 7f49d9df5740 1 -- 172.21.15.12:0/1459963901 --> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55f72caa90b0 con 0x55f72cbbbae0 2024-09-11T00:10:43.687 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:43.682+0000 7f49d5000640 1 --2- 172.21.15.12:0/1459963901 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55f72cbbbae0 0x55f72cbbdf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:10:43.687 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:43.682+0000 7f49d5000640 1 --2- 172.21.15.12:0/1459963901 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55f72cbbbae0 0x55f72cbbdf00 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ba96732c6c6fdcf2 server_cookie=c5b1a0ad0a945448 in_seq=0 out_seq=0 2024-09-11T00:10:43.687 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:43.682+0000 7f49be000640 10 client.25937 ms_handle_connect on v2:172.21.15.165:6838/501312984 2024-09-11T00:10:43.715 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:43.713+0000 7f49be000640 1 -- 172.21.15.12:0/1459963901 <== mds.0 v2:172.21.15.165:6838/501312984 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55f72caa90b0 con 0x55f72cbbbae0 2024-09-11T00:10:43.716 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:43.713+0000 7f49be000640 10 client.25937 handle_command_reply: tid=0 2024-09-11T00:10:43.716 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:43.713+0000 7f49d9df5740 1 client.25937 shutdown 2024-09-11T00:10:43.716 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:43.713+0000 7f49d9df5740 1 -- 172.21.15.12:0/1459963901 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f72cb3db00 msgr2=0x55f72cb49070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:10:43.716 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:43.713+0000 7f49d9df5740 1 --2- 172.21.15.12:0/1459963901 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f72cb3db00 0x55f72cb49070 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x55f72cb34d80 tx=0x7f49b8005260 comp rx=0 tx=0).stop 2024-09-11T00:10:43.716 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:43.713+0000 7f49d9df5740 1 -- 172.21.15.12:0/1459963901 shutdown_connections 2024-09-11T00:10:43.716 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:43.713+0000 7f49d9df5740 1 --2- 172.21.15.12:0/1459963901 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55f72cbbbae0 0x55f72cbbdf00 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:10:43.716 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:43.713+0000 7f49d9df5740 1 --2- 172.21.15.12:0/1459963901 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f72cb495b0 0x55f72cb47650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:10:43.716 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:43.713+0000 7f49d9df5740 1 --2- 172.21.15.12:0/1459963901 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f72cb3db00 0x55f72cb49070 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:10:43.716 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:43.713+0000 7f49be000640 0 client.25937 ms_handle_reset on v2:172.21.15.165:6838/501312984 2024-09-11T00:10:43.716 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:43.713+0000 7f49d9df5740 1 --2- 172.21.15.12:0/1459963901 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f72cb38790 0x55f72cb48b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:10:43.716 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:43.713+0000 7f49d9df5740 1 -- 172.21.15.12:0/1459963901 >> 172.21.15.12:0/1459963901 conn(0x55f72cb338e0 msgr2=0x55f72cb40e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:10:43.716 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:43.713+0000 7f49d9df5740 1 -- 172.21.15.12:0/1459963901 shutdown_connections 2024-09-11T00:10:43.717 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:43.713+0000 7f49d9df5740 1 -- 172.21.15.12:0/1459963901 wait complete. 2024-09-11T00:10:43.717 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:43.713+0000 7f49d9df5740 20 client.25937 trim_cache size 0 max 16384 2024-09-11T00:10:43.717 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:{ 2024-09-11T00:10:43.717 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "message": "", 2024-09-11T00:10:43.717 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "return_code": 0 2024-09-11T00:10:43.717 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:} 2024-09-11T00:10:43.718 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:43.714+0000 7f49d7000640 1 -- 172.21.15.12:0/601876653 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f49b40b52e0 msgr2=0x7f49b40b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:10:43.718 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:43.714+0000 7f49d7000640 1 --2- 172.21.15.12:0/601876653 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f49b40b52e0 0x7f49b40b77a0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f49c00ac6f0 tx=0x7f49c00e9d60 comp rx=0 tx=0).stop 2024-09-11T00:10:43.718 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:43.714+0000 7f49d7000640 1 -- 172.21.15.12:0/601876653 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f49d0104b90 msgr2=0x7f49d019af30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:10:43.718 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:43.714+0000 7f49d7000640 1 --2- 172.21.15.12:0/601876653 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f49d0104b90 0x7f49d019af30 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f49c000e9f0 tx=0x7f49c000eeb0 comp rx=0 tx=0).stop 2024-09-11T00:10:43.718 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:43.714+0000 7f49d7000640 1 -- 172.21.15.12:0/601876653 shutdown_connections 2024-09-11T00:10:43.718 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:43.714+0000 7f49d7000640 1 --2- 172.21.15.12:0/601876653 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f49b40b52e0 0x7f49b40b77a0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:10:43.718 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:43.714+0000 7f49d7000640 1 --2- 172.21.15.12:0/601876653 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f49d01066a0 0x7f49d019c1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:10:43.718 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:43.714+0000 7f49d7000640 1 --2- 172.21.15.12:0/601876653 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f49d0105ce0 0x7f49d019b580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:10:43.719 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:43.714+0000 7f49d7000640 1 --2- 172.21.15.12:0/601876653 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f49d0104b90 0x7f49d019af30 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:10:43.719 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:43.714+0000 7f49d7000640 1 -- 172.21.15.12:0/601876653 >> 172.21.15.12:0/601876653 conn(0x7f49d0100470 msgr2=0x7f49d01090d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:10:43.719 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:43.714+0000 7f49d7000640 1 -- 172.21.15.12:0/601876653 shutdown_connections 2024-09-11T00:10:43.719 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:43.714+0000 7f49d7000640 1 -- 172.21.15.12:0/601876653 wait complete. 2024-09-11T00:10:43.813 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:10:43 smithi012 ceph-mon[30333]: pgmap v508: 129 pgs: 129 active+clean; 34 MiB data, 656 MiB used, 1.0 TiB / 1.0 TiB avail; 394 KiB/s rd, 384 KiB/s wr, 102 op/s 2024-09-11T00:10:43.830 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:10:43 smithi178 ceph-mon[35919]: pgmap v508: 129 pgs: 129 active+clean; 34 MiB data, 656 MiB used, 1.0 TiB / 1.0 TiB avail; 394 KiB/s rd, 384 KiB/s wr, 102 op/s 2024-09-11T00:10:44.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:10:43 smithi165 ceph-mon[35356]: pgmap v508: 129 pgs: 129 active+clean; 34 MiB data, 656 MiB used, 1.0 TiB / 1.0 TiB avail; 394 KiB/s rd, 384 KiB/s wr, 102 op/s 2024-09-11T00:10:44.624 INFO:tasks.workunit.client.0.smithi012.stdout: 1 117880 2.80 MB/sec execute 41 sec latency 5030.678 ms 2024-09-11T00:10:45.613 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:10:45 smithi012 ceph-mon[30333]: pgmap v509: 129 pgs: 129 active+clean; 34 MiB data, 656 MiB used, 1.0 TiB / 1.0 TiB avail; 346 KiB/s rd, 325 KiB/s wr, 77 op/s 2024-09-11T00:10:45.624 INFO:tasks.workunit.client.0.smithi012.stdout: 1 117880 2.73 MB/sec execute 42 sec latency 6030.735 ms 2024-09-11T00:10:45.830 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:10:45 smithi178 ceph-mon[35919]: pgmap v509: 129 pgs: 129 active+clean; 34 MiB data, 656 MiB used, 1.0 TiB / 1.0 TiB avail; 346 KiB/s rd, 325 KiB/s wr, 77 op/s 2024-09-11T00:10:46.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:10:45 smithi165 ceph-mon[35356]: pgmap v509: 129 pgs: 129 active+clean; 34 MiB data, 656 MiB used, 1.0 TiB / 1.0 TiB avail; 346 KiB/s rd, 325 KiB/s wr, 77 op/s 2024-09-11T00:10:46.624 INFO:tasks.workunit.client.0.smithi012.stdout: 1 118921 2.78 MB/sec execute 43 sec latency 6469.901 ms 2024-09-11T00:10:47.625 INFO:tasks.workunit.client.0.smithi012.stdout: 1 121036 3.03 MB/sec execute 44 sec latency 10.792 ms 2024-09-11T00:10:47.813 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:10:47 smithi012 ceph-mon[30333]: pgmap v510: 129 pgs: 129 active+clean; 39 MiB data, 676 MiB used, 1.0 TiB / 1.0 TiB avail; 359 KiB/s rd, 986 KiB/s wr, 101 op/s 2024-09-11T00:10:47.830 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:10:47 smithi178 ceph-mon[35919]: pgmap v510: 129 pgs: 129 active+clean; 39 MiB data, 676 MiB used, 1.0 TiB / 1.0 TiB avail; 359 KiB/s rd, 986 KiB/s wr, 101 op/s 2024-09-11T00:10:48.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:10:47 smithi165 ceph-mon[35356]: pgmap v510: 129 pgs: 129 active+clean; 39 MiB data, 676 MiB used, 1.0 TiB / 1.0 TiB avail; 359 KiB/s rd, 986 KiB/s wr, 101 op/s 2024-09-11T00:10:48.624 INFO:tasks.workunit.client.0.smithi012.stdout: 1 121300 2.97 MB/sec execute 45 sec latency 770.600 ms 2024-09-11T00:10:49.624 INFO:tasks.workunit.client.0.smithi012.stdout: 1 121356 2.91 MB/sec execute 46 sec latency 1684.169 ms 2024-09-11T00:10:49.813 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:10:49 smithi012 ceph-mon[30333]: pgmap v511: 129 pgs: 129 active+clean; 39 MiB data, 684 MiB used, 1.0 TiB / 1.0 TiB avail; 285 KiB/s rd, 986 KiB/s wr, 80 op/s 2024-09-11T00:10:49.831 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:10:49 smithi178 ceph-mon[35919]: pgmap v511: 129 pgs: 129 active+clean; 39 MiB data, 684 MiB used, 1.0 TiB / 1.0 TiB avail; 285 KiB/s rd, 986 KiB/s wr, 80 op/s 2024-09-11T00:10:50.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:10:49 smithi165 ceph-mon[35356]: pgmap v511: 129 pgs: 129 active+clean; 39 MiB data, 684 MiB used, 1.0 TiB / 1.0 TiB avail; 285 KiB/s rd, 986 KiB/s wr, 80 op/s 2024-09-11T00:10:50.625 INFO:tasks.workunit.client.0.smithi012.stdout: 1 121356 2.84 MB/sec execute 47 sec latency 1038.704 ms 2024-09-11T00:10:50.855 INFO:tasks.quiescer.fs.[cephfs]:Total duration (60) is longer than `split_if_longer` (35.0), will split into two consecutive quiesces 2024-09-11T00:10:50.855 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 30.0 2024-09-11T00:10:50.855 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid ff3ad74a-6fcf-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T00:10:51.245 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:10:51.625 INFO:tasks.workunit.client.0.smithi012.stdout: 1 121356 2.78 MB/sec execute 48 sec latency 2038.799 ms 2024-09-11T00:10:51.700 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:10:51 smithi012 ceph-mon[30333]: pgmap v512: 129 pgs: 129 active+clean; 33 MiB data, 689 MiB used, 1.0 TiB / 1.0 TiB avail; 592 KiB/s rd, 1.0 MiB/s wr, 95 op/s 2024-09-11T00:10:51.814 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:10:51.812+0000 7fd365400640 1 -- 172.21.15.12:0/1185439086 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd360102dc0 msgr2=0x7fd3601031a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:10:51.815 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:10:51.812+0000 7fd365400640 1 --2- 172.21.15.12:0/1185439086 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd360102dc0 0x7fd3601031a0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fd350009900 tx=0x7fd35002a410 comp rx=0 tx=0).stop 2024-09-11T00:10:51.815 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:10:51.812+0000 7fd365400640 1 -- 172.21.15.12:0/1185439086 shutdown_connections 2024-09-11T00:10:51.815 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:10:51.812+0000 7fd365400640 1 --2- 172.21.15.12:0/1185439086 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd360109150 0x7fd36010f950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:10:51.815 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:10:51.812+0000 7fd365400640 1 --2- 172.21.15.12:0/1185439086 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd3601087d0 0x7fd360108c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:10:51.815 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:10:51.812+0000 7fd365400640 1 --2- 172.21.15.12:0/1185439086 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd360102dc0 0x7fd3601031a0 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:10:51.815 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:10:51.812+0000 7fd365400640 1 -- 172.21.15.12:0/1185439086 >> 172.21.15.12:0/1185439086 conn(0x7fd3600fe830 msgr2=0x7fd360100c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:10:51.815 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:10:51.812+0000 7fd365400640 1 -- 172.21.15.12:0/1185439086 shutdown_connections 2024-09-11T00:10:51.815 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:10:51.812+0000 7fd365400640 1 -- 172.21.15.12:0/1185439086 wait complete. 2024-09-11T00:10:51.816 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:10:51.813+0000 7fd365400640 1 Processor -- start 2024-09-11T00:10:51.817 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:10:51.813+0000 7fd365400640 1 -- start start 2024-09-11T00:10:51.817 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:10:51.814+0000 7fd365400640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd360102dc0 0x7fd36019b500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:10:51.817 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:10:51.814+0000 7fd365400640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd3601087d0 0x7fd36019ba40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:10:51.817 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:10:51.814+0000 7fd365400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd360109150 0x7fd3601a0de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:10:51.817 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:10:51.814+0000 7fd365400640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fd360112e20 con 0x7fd360102dc0 2024-09-11T00:10:51.817 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:10:51.814+0000 7fd365400640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fd360112ca0 con 0x7fd360109150 2024-09-11T00:10:51.817 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:10:51.814+0000 7fd365400640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fd360112fa0 con 0x7fd3601087d0 2024-09-11T00:10:51.817 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:10:51.814+0000 7fd35e800640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd360102dc0 0x7fd36019b500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:10:51.817 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:10:51.814+0000 7fd35e800640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd360102dc0 0x7fd36019b500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:44156/0 (socket says 172.21.15.12:44156) 2024-09-11T00:10:51.818 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:10:51.814+0000 7fd35e800640 1 -- 172.21.15.12:0/4264867679 learned_addr learned my addr 172.21.15.12:0/4264867679 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:10:51.818 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:10:51.814+0000 7fd355e00640 1 --2- 172.21.15.12:0/4264867679 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd3601087d0 0x7fd36019ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:10:51.818 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:10:51.814+0000 7fd35f200640 1 --2- 172.21.15.12:0/4264867679 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd360109150 0x7fd3601a0de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:10:51.818 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:10:51.814+0000 7fd35e800640 1 -- 172.21.15.12:0/4264867679 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd3601087d0 msgr2=0x7fd36019ba40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:10:51.818 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:10:51.814+0000 7fd35e800640 1 --2- 172.21.15.12:0/4264867679 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd3601087d0 0x7fd36019ba40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:10:51.818 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:10:51.814+0000 7fd35e800640 1 -- 172.21.15.12:0/4264867679 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd360109150 msgr2=0x7fd3601a0de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:10:51.818 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:10:51.814+0000 7fd35e800640 1 --2- 172.21.15.12:0/4264867679 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd360109150 0x7fd3601a0de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:10:51.818 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:10:51.815+0000 7fd35e800640 1 -- 172.21.15.12:0/4264867679 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3601a15b0 con 0x7fd360102dc0 2024-09-11T00:10:51.819 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:10:51.815+0000 7fd35e800640 1 --2- 172.21.15.12:0/4264867679 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd360102dc0 0x7fd36019b500 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fd350005ab0 tx=0x7fd350003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:10:51.819 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:10:51.815+0000 7fd357e00640 1 -- 172.21.15.12:0/4264867679 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd35003f070 con 0x7fd360102dc0 2024-09-11T00:10:51.819 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:10:51.815+0000 7fd357e00640 1 -- 172.21.15.12:0/4264867679 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd350031b60 con 0x7fd360102dc0 2024-09-11T00:10:51.819 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:10:51.815+0000 7fd357e00640 1 -- 172.21.15.12:0/4264867679 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd350035030 con 0x7fd360102dc0 2024-09-11T00:10:51.819 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:10:51.815+0000 7fd365400640 1 -- 172.21.15.12:0/4264867679 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd3601a1840 con 0x7fd360102dc0 2024-09-11T00:10:51.819 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:10:51.815+0000 7fd365400640 1 -- 172.21.15.12:0/4264867679 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3601a7f60 con 0x7fd360102dc0 2024-09-11T00:10:51.819 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:10:51.816+0000 7fd365400640 1 -- 172.21.15.12:0/4264867679 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd3601031a0 con 0x7fd360102dc0 2024-09-11T00:10:51.820 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:10:51.818+0000 7fd357e00640 1 -- 172.21.15.12:0/4264867679 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7fd35003b460 con 0x7fd360102dc0 2024-09-11T00:10:51.822 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:10:51.820+0000 7fd357e00640 1 --2- 172.21.15.12:0/4264867679 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fd3440b5360 0x7fd3440b7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:10:51.822 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:10:51.820+0000 7fd357e00640 1 -- 172.21.15.12:0/4264867679 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9208+0+0 (secure 0 0 0) 0x7fd350106930 con 0x7fd360102dc0 2024-09-11T00:10:51.822 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:10:51.820+0000 7fd355e00640 1 --2- 172.21.15.12:0/4264867679 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fd3440b5360 0x7fd3440b7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:10:51.823 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:10:51.821+0000 7fd355e00640 1 --2- 172.21.15.12:0/4264867679 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fd3440b5360 0x7fd3440b7820 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fd3601080c0 tx=0x7fd348005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:10:51.826 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:10:51.824+0000 7fd357e00640 1 -- 172.21.15.12:0/4264867679 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7fd3500ccd70 con 0x7fd360102dc0 2024-09-11T00:10:51.830 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:10:51 smithi178 ceph-mon[35919]: pgmap v512: 129 pgs: 129 active+clean; 33 MiB data, 689 MiB used, 1.0 TiB / 1.0 TiB avail; 592 KiB/s rd, 1.0 MiB/s wr, 95 op/s 2024-09-11T00:10:52.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:10:51 smithi165 ceph-mon[35356]: pgmap v512: 129 pgs: 129 active+clean; 33 MiB data, 689 MiB used, 1.0 TiB / 1.0 TiB avail; 592 KiB/s rd, 1.0 MiB/s wr, 95 op/s 2024-09-11T00:10:52.071 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:10:52.069+0000 7fd365400640 1 -- 172.21.15.12:0/4264867679 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd36002e5c0 con 0x7fd360102dc0 2024-09-11T00:10:52.071 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:10:52.069+0000 7fd357e00640 1 -- 172.21.15.12:0/4264867679 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) ==== 94+0+11970 (secure 0 0 0) 0x7fd3500d17a0 con 0x7fd360102dc0 2024-09-11T00:10:52.072 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-11T00:10:52.072 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":34,"btime":"2024-09-11T00:10:37:521144+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14577,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6835/447171591","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":447171591},{"type":"v1","addr":"172.21.15.12:6835","nonce":447171591}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14589,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/901486040","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":901486040},{"type":"v1","addr":"172.21.15.12:6837","nonce":901486040}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":34,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T00:04:04.739881+0000","modified":"2024-09-11T00:10:36.942981+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24674,"mds_1":24643,"mds_2":24638,"mds_3":24637,"mds_4":24614},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14631":{"gid":14631,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.12:6841/3069919437","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6840","nonce":3069919437},{"type":"v1","addr":"172.21.15.12:6841","nonce":3069919437}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24562":{"gid":24562,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6833/1141636112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":1141636112},{"type":"v1","addr":"172.21.15.178:6833","nonce":1141636112}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24581":{"gid":24581,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6833/3111980970","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6832","nonce":3111980970},{"type":"v1","addr":"172.21.15.165:6833","nonce":3111980970}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24607":{"gid":24607,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6835/4084868287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":4084868287},{"type":"v1","addr":"172.21.15.178:6835","nonce":4084868287}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24614":{"gid":24614,"name":"e","rank":4,"incarnation":26,"state":"up:active","state_seq":21,"addr":"172.21.15.165:6835/2425395274","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":2425395274},{"type":"v1","addr":"172.21.15.165:6835","nonce":2425395274}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24637":{"gid":24637,"name":"i","rank":3,"incarnation":24,"state":"up:active","state_seq":18,"addr":"172.21.15.178:6839/70628097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6838","nonce":70628097},{"type":"v1","addr":"172.21.15.178:6839","nonce":70628097}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24638":{"gid":24638,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":18,"addr":"172.21.15.165:6837/6215088","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6836","nonce":6215088},{"type":"v1","addr":"172.21.15.165:6837","nonce":6215088}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24643":{"gid":24643,"name":"g","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.12:6839/3124093009","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6838","nonce":3124093009},{"type":"v1","addr":"172.21.15.12:6839","nonce":3124093009}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24674":{"gid":24674,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.165:6839/501312984","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6838","nonce":501312984},{"type":"v1","addr":"172.21.15.165:6839","nonce":501312984}]},"join_fscid":-1,"export_targets":[1,2,3],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25672":{"gid":25672,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6837/308601741","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":308601741},{"type":"v1","addr":"172.21.15.178:6837","nonce":308601741}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24674,"qdb_cluster":[24674,24643,24638,24637,24614]},"id":1}]} 2024-09-11T00:10:52.072 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 34 2024-09-11T00:10:52.074 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:10:52.072+0000 7fd365400640 1 -- 172.21.15.12:0/4264867679 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fd3440b5360 msgr2=0x7fd3440b7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:10:52.074 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:10:52.072+0000 7fd365400640 1 --2- 172.21.15.12:0/4264867679 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fd3440b5360 0x7fd3440b7820 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fd3601080c0 tx=0x7fd348005f90 comp rx=0 tx=0).stop 2024-09-11T00:10:52.074 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:10:52.072+0000 7fd365400640 1 -- 172.21.15.12:0/4264867679 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd360102dc0 msgr2=0x7fd36019b500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:10:52.074 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:10:52.072+0000 7fd365400640 1 --2- 172.21.15.12:0/4264867679 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd360102dc0 0x7fd36019b500 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fd350005ab0 tx=0x7fd350003c10 comp rx=0 tx=0).stop 2024-09-11T00:10:52.075 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:10:52.072+0000 7fd365400640 1 -- 172.21.15.12:0/4264867679 shutdown_connections 2024-09-11T00:10:52.075 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:10:52.072+0000 7fd365400640 1 --2- 172.21.15.12:0/4264867679 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fd3440b5360 0x7fd3440b7820 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:10:52.075 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:10:52.072+0000 7fd365400640 1 --2- 172.21.15.12:0/4264867679 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd360109150 0x7fd3601a0de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:10:52.075 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:10:52.072+0000 7fd365400640 1 --2- 172.21.15.12:0/4264867679 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd3601087d0 0x7fd36019ba40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:10:52.075 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:10:52.072+0000 7fd365400640 1 --2- 172.21.15.12:0/4264867679 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd360102dc0 0x7fd36019b500 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:10:52.075 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:10:52.072+0000 7fd365400640 1 -- 172.21.15.12:0/4264867679 >> 172.21.15.12:0/4264867679 conn(0x7fd3600fe830 msgr2=0x7fd360104d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:10:52.075 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:10:52.072+0000 7fd365400640 1 -- 172.21.15.12:0/4264867679 shutdown_connections 2024-09-11T00:10:52.075 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:10:52.072+0000 7fd365400640 1 -- 172.21.15.12:0/4264867679 wait complete. 2024-09-11T00:10:52.625 INFO:tasks.workunit.client.0.smithi012.stdout: 1 121356 2.73 MB/sec execute 49 sec latency 3038.887 ms 2024-09-11T00:10:52.813 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:10:52 smithi012 ceph-mon[30333]: from='client.? 172.21.15.12:0/4264867679' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T00:10:52.830 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:10:52 smithi178 ceph-mon[35919]: from='client.? 172.21.15.12:0/4264867679' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T00:10:52.959 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24674 quiesce db / --timeout 90 --expiration 150.0 --await-for 45.0' 2024-09-11T00:10:52.960 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid ff3ad74a-6fcf-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24674 quiesce db / --timeout 90 --expiration 150.0 --await-for 45.0 2024-09-11T00:10:53.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:10:52 smithi165 ceph-mon[35356]: from='client.? 172.21.15.12:0/4264867679' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T00:10:53.340 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:10:53.625 INFO:tasks.workunit.client.0.smithi012.stdout: 1 121356 2.67 MB/sec execute 50 sec latency 4038.969 ms 2024-09-11T00:10:53.799 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:10:53 smithi012 ceph-mon[30333]: pgmap v513: 129 pgs: 129 active+clean; 34 MiB data, 695 MiB used, 1.0 TiB / 1.0 TiB avail; 618 KiB/s rd, 1.0 MiB/s wr, 104 op/s 2024-09-11T00:10:53.799 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:10:53 smithi012 ceph-mon[30333]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T00:10:53.830 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:10:53 smithi178 ceph-mon[35919]: pgmap v513: 129 pgs: 129 active+clean; 34 MiB data, 695 MiB used, 1.0 TiB / 1.0 TiB avail; 618 KiB/s rd, 1.0 MiB/s wr, 104 op/s 2024-09-11T00:10:53.831 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:10:53 smithi178 ceph-mon[35919]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T00:10:53.925 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:10:53.923+0000 7f7498800640 1 -- 172.21.15.12:0/2308118771 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f749010f3c0 msgr2=0x7f74901117b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:10:53.925 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:10:53.923+0000 7f7498800640 1 --2- 172.21.15.12:0/2308118771 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f749010f3c0 0x7f74901117b0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f748c009ca0 tx=0x7f748c02a8c0 comp rx=0 tx=0).stop 2024-09-11T00:10:53.925 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:10:53.923+0000 7f7498800640 1 -- 172.21.15.12:0/2308118771 shutdown_connections 2024-09-11T00:10:53.925 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:10:53.923+0000 7f7498800640 1 --2- 172.21.15.12:0/2308118771 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f749010f3c0 0x7f74901117b0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:10:53.926 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:10:53.923+0000 7f7498800640 1 --2- 172.21.15.12:0/2308118771 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7490075f40 0x7f749010edf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:10:53.926 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:10:53.923+0000 7f7498800640 1 --2- 172.21.15.12:0/2308118771 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7490075540 0x7f7490075920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:10:53.926 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:10:53.923+0000 7f7498800640 1 -- 172.21.15.12:0/2308118771 >> 172.21.15.12:0/2308118771 conn(0x7f74900fe810 msgr2=0x7f7490100c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:10:53.926 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:10:53.923+0000 7f7498800640 1 -- 172.21.15.12:0/2308118771 shutdown_connections 2024-09-11T00:10:53.926 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:10:53.923+0000 7f7498800640 1 -- 172.21.15.12:0/2308118771 wait complete. 2024-09-11T00:10:53.928 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:10:53.924+0000 7f7498800640 1 Processor -- start 2024-09-11T00:10:53.928 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:10:53.924+0000 7f7498800640 1 -- start start 2024-09-11T00:10:53.928 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:10:53.924+0000 7f7498800640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7490075540 0x7f749019b420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:10:53.928 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:10:53.924+0000 7f7498800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7490075f40 0x7f749019b960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:10:53.929 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:10:53.924+0000 7f7498800640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f749010f3c0 0x7f74901a0d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:10:53.929 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:10:53.924+0000 7f7498800640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f7490114f50 con 0x7f7490075540 2024-09-11T00:10:53.929 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:10:53.924+0000 7f7498800640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f7490114dd0 con 0x7f7490075f40 2024-09-11T00:10:53.929 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:10:53.924+0000 7f7498800640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f74901150d0 con 0x7f749010f3c0 2024-09-11T00:10:53.929 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:10:53.925+0000 7f7495e00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7490075540 0x7f749019b420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:10:53.929 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:10:53.925+0000 7f7495e00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7490075540 0x7f749019b420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:44168/0 (socket says 172.21.15.12:44168) 2024-09-11T00:10:53.929 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:10:53.925+0000 7f7495e00640 1 -- 172.21.15.12:0/771039303 learned_addr learned my addr 172.21.15.12:0/771039303 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:10:53.929 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:10:53.925+0000 7f7495400640 1 --2- 172.21.15.12:0/771039303 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7490075f40 0x7f749019b960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:10:53.929 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:10:53.925+0000 7f7496800640 1 --2- 172.21.15.12:0/771039303 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f749010f3c0 0x7f74901a0d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:10:53.929 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:10:53.925+0000 7f7495400640 1 -- 172.21.15.12:0/771039303 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f749010f3c0 msgr2=0x7f74901a0d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:10:53.929 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:10:53.925+0000 7f7495400640 1 --2- 172.21.15.12:0/771039303 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f749010f3c0 0x7f74901a0d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:10:53.929 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:10:53.925+0000 7f7495400640 1 -- 172.21.15.12:0/771039303 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7490075540 msgr2=0x7f749019b420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:10:53.929 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:10:53.925+0000 7f7495400640 1 --2- 172.21.15.12:0/771039303 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7490075540 0x7f749019b420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:10:53.930 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:10:53.925+0000 7f7495400640 1 -- 172.21.15.12:0/771039303 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74901a14d0 con 0x7f7490075f40 2024-09-11T00:10:53.930 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:10:53.925+0000 7f7496800640 1 --2- 172.21.15.12:0/771039303 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f749010f3c0 0x7f74901a0d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T00:10:53.930 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:10:53.925+0000 7f7495e00640 1 --2- 172.21.15.12:0/771039303 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7490075540 0x7f749019b420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:10:53.930 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:10:53.926+0000 7f7495400640 1 --2- 172.21.15.12:0/771039303 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7490075f40 0x7f749019b960 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f748000e9f0 tx=0x7f748000eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:10:53.930 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:10:53.926+0000 7f748aa00640 1 -- 172.21.15.12:0/771039303 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f748000ccc0 con 0x7f7490075f40 2024-09-11T00:10:53.930 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:10:53.926+0000 7f7498800640 1 -- 172.21.15.12:0/771039303 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f74901a17c0 con 0x7f7490075f40 2024-09-11T00:10:53.930 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:10:53.926+0000 7f748aa00640 1 -- 172.21.15.12:0/771039303 <== mon.2 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7480005020 con 0x7f7490075f40 2024-09-11T00:10:53.930 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:10:53.926+0000 7f748aa00640 1 -- 172.21.15.12:0/771039303 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7480004a00 con 0x7f7490075f40 2024-09-11T00:10:53.930 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:10:53.926+0000 7f7498800640 1 -- 172.21.15.12:0/771039303 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74901a7e80 con 0x7f7490075f40 2024-09-11T00:10:53.931 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:10:53.929+0000 7f749b261740 1 Processor -- start 2024-09-11T00:10:53.931 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:10:53.929+0000 7f749b261740 1 -- start start 2024-09-11T00:10:53.932 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:10:53.929+0000 7f749b261740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562da7d98180 0x562da7d98560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:10:53.932 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:10:53.929+0000 7f749b261740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562da7d98aa0 0x562da7da44f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:10:53.932 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:10:53.930+0000 7f749b261740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562da7da4a30 0x562da7da6e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:10:53.932 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:10:53.930+0000 7f749b261740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x562da7d0dca0 con 0x562da7d98aa0 2024-09-11T00:10:53.932 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:10:53.930+0000 7f749b261740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x562da7cfd0d0 con 0x562da7da4a30 2024-09-11T00:10:53.933 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:10:53.930+0000 7f749b261740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x562da7d00ee0 con 0x562da7d98180 2024-09-11T00:10:53.933 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:10:53.930+0000 7f748aa00640 1 -- 172.21.15.12:0/771039303 <== mon.2 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7f7480004ba0 con 0x7f7490075f40 2024-09-11T00:10:53.933 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:10:53.930+0000 7f7495e00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562da7d98aa0 0x562da7da44f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:10:53.933 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:10:53.930+0000 7f7496800640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562da7d98180 0x562da7d98560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:10:53.933 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:10:53.930+0000 7f7495e00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562da7d98aa0 0x562da7da44f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:44178/0 (socket says 172.21.15.12:44178) 2024-09-11T00:10:53.933 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:10:53.930+0000 7f7495e00640 1 -- 172.21.15.12:0/4101389641 learned_addr learned my addr 172.21.15.12:0/4101389641 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:10:53.933 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:10:53.930+0000 7f7495400640 1 --2- 172.21.15.12:0/4101389641 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562da7da4a30 0x562da7da6e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:10:53.935 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:10:53.931+0000 7f7496800640 1 -- 172.21.15.12:0/4101389641 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562da7da4a30 msgr2=0x562da7da6e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:10:53.935 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:10:53.931+0000 7f7496800640 1 --2- 172.21.15.12:0/4101389641 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562da7da4a30 0x562da7da6e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:10:53.935 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:10:53.931+0000 7f7496800640 1 -- 172.21.15.12:0/4101389641 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562da7d98aa0 msgr2=0x562da7da44f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:10:53.935 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:10:53.931+0000 7f7496800640 1 --2- 172.21.15.12:0/4101389641 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562da7d98aa0 0x562da7da44f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:10:53.935 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:10:53.931+0000 7f7496800640 1 -- 172.21.15.12:0/4101389641 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562da7caf400 con 0x562da7d98180 2024-09-11T00:10:53.935 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:10:53.931+0000 7f7495e00640 1 --2- 172.21.15.12:0/4101389641 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562da7d98aa0 0x562da7da44f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:10:53.935 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:10:53.931+0000 7f7496800640 1 --2- 172.21.15.12:0/4101389641 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562da7d98180 0x562da7d98560 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f748c02ade0 tx=0x7f748c009360 comp rx=0 tx=0).ready entity=mon.1 client_cookie=653f14450b9cdb0b server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:10:53.935 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:10:53.932+0000 7f747f000640 1 -- 172.21.15.12:0/4101389641 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f748c031cf0 con 0x562da7d98180 2024-09-11T00:10:53.935 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:10:53.932+0000 7f747f000640 1 -- 172.21.15.12:0/4101389641 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f748c036040 con 0x562da7d98180 2024-09-11T00:10:53.935 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:10:53.932+0000 7f748aa00640 1 --2- 172.21.15.12:0/771039303 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f74780b5360 0x7f74780b7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:10:53.936 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:10:53.932+0000 7f747f000640 1 -- 172.21.15.12:0/4101389641 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f748c0042a0 con 0x562da7d98180 2024-09-11T00:10:53.936 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:10:53.932+0000 7f748aa00640 1 -- 172.21.15.12:0/771039303 <== mon.2 v2:172.21.15.165:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9208+0+0 (secure 0 0 0) 0x7f74800e5f50 con 0x7f7490075f40 2024-09-11T00:10:53.936 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:10:53.932+0000 7f7495e00640 1 --2- 172.21.15.12:0/771039303 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f74780b5360 0x7f74780b7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:10:53.936 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:10:53.932+0000 7f749b261740 1 -- 172.21.15.12:0/4101389641 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562da7d98180 msgr2=0x562da7d98560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:10:53.936 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:10:53.932+0000 7f749b261740 1 --2- 172.21.15.12:0/4101389641 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562da7d98180 0x562da7d98560 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f748c02ade0 tx=0x7f748c009360 comp rx=0 tx=0).stop 2024-09-11T00:10:53.936 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:10:53.933+0000 7f749b261740 1 -- 172.21.15.12:0/4101389641 shutdown_connections 2024-09-11T00:10:53.936 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:10:53.933+0000 7f749b261740 1 --2- 172.21.15.12:0/4101389641 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562da7da4a30 0x562da7da6e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:10:53.936 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:10:53.933+0000 7f749b261740 1 --2- 172.21.15.12:0/4101389641 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562da7d98aa0 0x562da7da44f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:10:53.936 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:10:53.933+0000 7f749b261740 1 --2- 172.21.15.12:0/4101389641 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562da7d98180 0x562da7d98560 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:10:53.936 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:10:53.933+0000 7f749b261740 1 -- 172.21.15.12:0/4101389641 >> 172.21.15.12:0/4101389641 conn(0x562da7d95fb0 msgr2=0x562da7d97150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:10:53.936 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:10:53.933+0000 7f749b261740 1 -- 172.21.15.12:0/4101389641 shutdown_connections 2024-09-11T00:10:53.936 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:10:53.933+0000 7f7495e00640 1 --2- 172.21.15.12:0/771039303 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f74780b5360 0x7f74780b7820 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f7484009fd0 tx=0x7f7484005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:10:53.936 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:10:53.933+0000 7f749b261740 1 -- 172.21.15.12:0/4101389641 wait complete. 2024-09-11T00:10:53.939 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:10:53.933+0000 7f749b261740 1 -- start start 2024-09-11T00:10:53.939 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:10:53.933+0000 7f749b261740 1 Processor -- start 2024-09-11T00:10:53.939 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:10:53.934+0000 7f749b261740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562da7d98180 0x562da7da4320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:10:53.939 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:10:53.934+0000 7f749b261740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562da7d98aa0 0x562da7da1770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:10:53.939 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:10:53.934+0000 7f749b261740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562da7da4a30 0x562da7da1da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:10:53.939 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:10:53.934+0000 7f749b261740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x562da7b5e7a0 con 0x562da7da4a30 2024-09-11T00:10:53.939 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:10:53.934+0000 7f749b261740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x562da7ca2a10 con 0x562da7d98aa0 2024-09-11T00:10:53.940 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:10:53.934+0000 7f749b261740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x562da7da7650 con 0x562da7d98180 2024-09-11T00:10:53.940 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:10:53.934+0000 7f749b261740 10 client.0 _ll_register_callbacks cb 0x562da7cb2be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:10:53.940 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:10:53.934+0000 7f7496800640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562da7da4a30 0x562da7da1da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:10:53.940 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:10:53.934+0000 7f7496800640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562da7da4a30 0x562da7da1da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:44194/0 (socket says 172.21.15.12:44194) 2024-09-11T00:10:53.940 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:10:53.934+0000 7f7495400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562da7d98aa0 0x562da7da1770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:10:53.940 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:10:53.934+0000 7f7496800640 1 -- 172.21.15.12:0/2245175924 learned_addr learned my addr 172.21.15.12:0/2245175924 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:10:53.940 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:10:53.934+0000 7f7495e00640 1 --2- 172.21.15.12:0/2245175924 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562da7d98180 0x562da7da4320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:10:53.940 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:10:53.935+0000 7f7495400640 1 -- 172.21.15.12:0/2245175924 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562da7d98180 msgr2=0x562da7da4320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:10:53.940 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:10:53.935+0000 7f7495400640 1 --2- 172.21.15.12:0/2245175924 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562da7d98180 0x562da7da4320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:10:53.940 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:10:53.935+0000 7f7495400640 1 -- 172.21.15.12:0/2245175924 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562da7da4a30 msgr2=0x562da7da1da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:10:53.940 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:10:53.935+0000 7f7495400640 1 --2- 172.21.15.12:0/2245175924 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562da7da4a30 0x562da7da1da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:10:53.940 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:10:53.935+0000 7f7495400640 1 -- 172.21.15.12:0/2245175924 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562da7a6ab90 con 0x562da7d98aa0 2024-09-11T00:10:53.941 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:10:53.935+0000 7f7495400640 1 --2- 172.21.15.12:0/2245175924 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562da7d98aa0 0x562da7da1770 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f74800e36d0 tx=0x7f7480005b30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=536f6c122283cf62 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:10:53.941 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:10:53.935+0000 7f7473400640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-09-11T00:10:53.941 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:10:53.935+0000 7f7473400640 1 -- 172.21.15.12:0/2245175924 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f748001c070 con 0x562da7d98aa0 2024-09-11T00:10:53.941 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:10:53.935+0000 7f749b261740 1 -- 172.21.15.12:0/2245175924 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f748c036040 con 0x562da7d98aa0 2024-09-11T00:10:53.941 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:10:53.935+0000 7f7473400640 1 -- 172.21.15.12:0/2245175924 <== mon.2 v2:172.21.15.165:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9208+0+0 (secure 0 0 0) 0x7f74800e8110 con 0x562da7d98aa0 2024-09-11T00:10:53.941 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:10:53.935+0000 7f7473400640 1 -- 172.21.15.12:0/2245175924 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f74800ac110 con 0x562da7d98aa0 2024-09-11T00:10:53.941 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:10:53.936+0000 7f749b261740 10 client.25955 fetch_fsmap learned FSMap version 35 2024-09-11T00:10:53.941 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:10:53.936+0000 7f749b261740 1 -- 172.21.15.12:0/2245175924 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x562da7db2fd0 con 0x562da7d98aa0 2024-09-11T00:10:53.941 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:10:53.936+0000 7f7473400640 1 -- 172.21.15.12:0/2245175924 <== mon.2 v2:172.21.15.165:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f748c036040 con 0x562da7d98aa0 2024-09-11T00:10:53.941 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:10:53.936+0000 7f7473400640 1 -- 172.21.15.12:0/2245175924 <== mon.2 v2:172.21.15.165:3300/0 5 ==== fsmap(e 35) ==== 8378+0+0 (secure 0 0 0) 0x7f74800ac5a0 con 0x562da7d98aa0 2024-09-11T00:10:53.941 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:10:53.936+0000 7f749b261740 10 client.25955 fetch_fsmap finished waiting for FSMap version 35 2024-09-11T00:10:53.941 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:10:53.936+0000 7f749b261740 10 client.25955 resolve_mds: validated gid 24674 aka daemon mds.b 2024-09-11T00:10:53.941 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:10:53.936+0000 7f749b261740 20 client.25955 populate_metadata read hostname 'smithi012' 2024-09-11T00:10:53.942 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:10:53.936+0000 7f749b261740 1 --2- 172.21.15.12:0/2245175924 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x562da7db4480 0x562da7db68a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:10:53.942 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:10:53.936+0000 7f749b261740 4 client.25955 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T00:10:53.942 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:10:53.936+0000 7f749b261740 1 -- 172.21.15.12:0/2245175924 --> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562da7d0fbf0 con 0x562da7db4480 2024-09-11T00:10:53.942 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:10:53.936+0000 7f7495e00640 1 --2- 172.21.15.12:0/2245175924 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x562da7db4480 0x562da7db68a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:10:53.942 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:10:53.937+0000 7f7495e00640 1 --2- 172.21.15.12:0/2245175924 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x562da7db4480 0x562da7db68a0 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8a497923923d0b8 server_cookie=e027b64b1dc6cabe in_seq=0 out_seq=0 2024-09-11T00:10:53.942 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:10:53.937+0000 7f7473400640 10 client.25955 ms_handle_connect on v2:172.21.15.165:6838/501312984 2024-09-11T00:10:53.943 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:10:53.938+0000 7f7473400640 1 -- 172.21.15.12:0/2245175924 <== mds.0 v2:172.21.15.165:6838/501312984 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x562da7d0fbf0 con 0x562da7db4480 2024-09-11T00:10:53.943 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:10:53.938+0000 7f7473400640 10 client.25955 handle_command_reply: tid=0 2024-09-11T00:10:53.943 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:10:53.938+0000 7f749b261740 1 client.25955 shutdown 2024-09-11T00:10:53.943 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:10:53.938+0000 7f749b261740 1 -- 172.21.15.12:0/2245175924 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562da7d98aa0 msgr2=0x562da7da1770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:10:53.943 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:10:53.938+0000 7f749b261740 1 --2- 172.21.15.12:0/2245175924 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562da7d98aa0 0x562da7da1770 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f74800e36d0 tx=0x7f7480005b30 comp rx=0 tx=0).stop 2024-09-11T00:10:53.943 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:10:53.938+0000 7f749b261740 1 -- 172.21.15.12:0/2245175924 shutdown_connections 2024-09-11T00:10:53.944 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:10:53.938+0000 7f749b261740 1 --2- 172.21.15.12:0/2245175924 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x562da7db4480 0x562da7db68a0 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:10:53.944 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:10:53.938+0000 7f749b261740 1 --2- 172.21.15.12:0/2245175924 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562da7da4a30 0x562da7da1da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:10:53.944 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:10:53.938+0000 7f749b261740 1 --2- 172.21.15.12:0/2245175924 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562da7d98aa0 0x562da7da1770 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:10:53.944 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:10:53.938+0000 7f749b261740 1 --2- 172.21.15.12:0/2245175924 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562da7d98180 0x562da7da4320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:10:53.944 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:10:53.938+0000 7f7473400640 0 client.25955 ms_handle_reset on v2:172.21.15.165:6838/501312984 2024-09-11T00:10:53.944 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:10:53.938+0000 7f749b261740 1 -- 172.21.15.12:0/2245175924 >> 172.21.15.12:0/2245175924 conn(0x562da7d95fb0 msgr2=0x562da7d9bb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:10:53.944 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:10:53.938+0000 7f749b261740 1 -- 172.21.15.12:0/2245175924 shutdown_connections 2024-09-11T00:10:53.944 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:10:53.938+0000 7f749b261740 1 -- 172.21.15.12:0/2245175924 wait complete. 2024-09-11T00:10:53.944 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:10:53.939+0000 7f749b261740 20 client.25955 trim_cache size 0 max 16384 2024-09-11T00:10:53.958 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:10:53.957+0000 7f749b261740 1 Processor -- start 2024-09-11T00:10:53.959 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:10:53.957+0000 7f749b261740 1 -- start start 2024-09-11T00:10:53.960 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:10:53.957+0000 7f749b261740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562da7d98180 0x562da7da1000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:10:53.960 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:10:53.957+0000 7f749b261740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562da7d98aa0 0x562da7da5e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:10:53.960 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:10:53.957+0000 7f749b261740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562da7da4a30 0x562da7da6360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:10:53.960 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:10:53.957+0000 7f749b261740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f74901b1dc0 con 0x562da7da4a30 2024-09-11T00:10:53.960 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:10:53.957+0000 7f749b261740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f74901b3960 con 0x562da7d98aa0 2024-09-11T00:10:53.960 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:10:53.957+0000 7f749b261740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f74901b2490 con 0x562da7d98180 2024-09-11T00:10:53.960 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:10:53.957+0000 7f7496800640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562da7da4a30 0x562da7da6360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:10:53.960 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:10:53.957+0000 7f7496800640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562da7da4a30 0x562da7da6360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:44198/0 (socket says 172.21.15.12:44198) 2024-09-11T00:10:53.961 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:10:53.957+0000 7f7496800640 1 -- 172.21.15.12:0/2371834714 learned_addr learned my addr 172.21.15.12:0/2371834714 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:10:53.961 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:10:53.957+0000 7f7495400640 1 --2- 172.21.15.12:0/2371834714 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562da7d98aa0 0x562da7da5e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:10:53.961 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:10:53.957+0000 7f7495e00640 1 --2- 172.21.15.12:0/2371834714 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562da7d98180 0x562da7da1000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:10:53.961 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:10:53.958+0000 7f7496800640 1 -- 172.21.15.12:0/2371834714 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562da7d98180 msgr2=0x562da7da1000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:10:53.961 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:10:53.958+0000 7f7496800640 1 --2- 172.21.15.12:0/2371834714 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562da7d98180 0x562da7da1000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:10:53.961 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:10:53.958+0000 7f7496800640 1 -- 172.21.15.12:0/2371834714 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562da7d98aa0 msgr2=0x562da7da5e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:10:53.961 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:10:53.958+0000 7f7496800640 1 --2- 172.21.15.12:0/2371834714 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562da7d98aa0 0x562da7da5e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:10:53.961 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:10:53.958+0000 7f7496800640 1 -- 172.21.15.12:0/2371834714 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562da7da1540 con 0x562da7da4a30 2024-09-11T00:10:53.961 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:10:53.958+0000 7f7495400640 1 --2- 172.21.15.12:0/2371834714 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562da7d98aa0 0x562da7da5e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:10:53.961 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:10:53.958+0000 7f7496800640 1 --2- 172.21.15.12:0/2371834714 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562da7da4a30 0x562da7da6360 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f748c009ca0 tx=0x7f748c005d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9de0abf49af2814f server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:10:53.963 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:10:53.959+0000 7f747e600640 1 -- 172.21.15.12:0/2371834714 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f748c052020 con 0x562da7da4a30 2024-09-11T00:10:53.963 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:10:53.959+0000 7f747e600640 1 -- 172.21.15.12:0/2371834714 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f748c036040 con 0x562da7da4a30 2024-09-11T00:10:53.963 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:10:53.959+0000 7f747e600640 1 -- 172.21.15.12:0/2371834714 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f748c04a040 con 0x562da7da4a30 2024-09-11T00:10:53.963 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:10:53.959+0000 7f749b261740 1 -- 172.21.15.12:0/2371834714 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562da7da4a30 msgr2=0x562da7da6360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:10:53.963 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:10:53.959+0000 7f749b261740 1 --2- 172.21.15.12:0/2371834714 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562da7da4a30 0x562da7da6360 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f748c009ca0 tx=0x7f748c005d80 comp rx=0 tx=0).stop 2024-09-11T00:10:53.963 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:10:53.959+0000 7f749b261740 1 -- 172.21.15.12:0/2371834714 shutdown_connections 2024-09-11T00:10:53.963 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:10:53.959+0000 7f749b261740 1 --2- 172.21.15.12:0/2371834714 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562da7da4a30 0x562da7da6360 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:10:53.963 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:10:53.959+0000 7f749b261740 1 --2- 172.21.15.12:0/2371834714 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562da7d98aa0 0x562da7da5e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:10:53.964 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:10:53.959+0000 7f749b261740 1 --2- 172.21.15.12:0/2371834714 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562da7d98180 0x562da7da1000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:10:53.964 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:10:53.959+0000 7f749b261740 1 -- 172.21.15.12:0/2371834714 >> 172.21.15.12:0/2371834714 conn(0x562da7d95fb0 msgr2=0x562da7d9c9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:10:53.964 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:10:53.959+0000 7f749b261740 1 -- 172.21.15.12:0/2371834714 shutdown_connections 2024-09-11T00:10:53.964 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:10:53.960+0000 7f749b261740 1 -- 172.21.15.12:0/2371834714 wait complete. 2024-09-11T00:10:53.964 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:10:53.960+0000 7f749b261740 1 -- start start 2024-09-11T00:10:53.964 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:10:53.960+0000 7f749b261740 1 Processor -- start 2024-09-11T00:10:53.964 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:10:53.961+0000 7f749b261740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562da7d98180 0x562da7dacce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:10:53.964 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:10:53.961+0000 7f749b261740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562da7d98aa0 0x562da7dad260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:10:53.964 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:10:53.961+0000 7f749b261740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562da7da4a30 0x562da7d9db20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:10:53.964 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:10:53.961+0000 7f749b261740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f74901b3290 con 0x562da7d98aa0 2024-09-11T00:10:53.964 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:10:53.961+0000 7f749b261740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f74901b2b60 con 0x562da7d98180 2024-09-11T00:10:53.964 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:10:53.961+0000 7f749b261740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f74901afda0 con 0x562da7da4a30 2024-09-11T00:10:53.964 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:10:53.961+0000 7f749b261740 10 client.0 _ll_register_callbacks cb 0x7f74800e9720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:10:53.965 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:10:53.961+0000 7f7495400640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562da7d98aa0 0x562da7dad260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:10:53.965 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:10:53.961+0000 7f7495400640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562da7d98aa0 0x562da7dad260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:44206/0 (socket says 172.21.15.12:44206) 2024-09-11T00:10:53.965 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:10:53.961+0000 7f7495400640 1 -- 172.21.15.12:0/1681169687 learned_addr learned my addr 172.21.15.12:0/1681169687 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:10:53.965 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:10:53.961+0000 7f7495e00640 1 --2- 172.21.15.12:0/1681169687 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562da7d98180 0x562da7dacce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:10:53.965 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:10:53.961+0000 7f7496800640 1 --2- 172.21.15.12:0/1681169687 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562da7da4a30 0x562da7d9db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:10:53.966 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:10:53.961+0000 7f7495e00640 1 -- 172.21.15.12:0/1681169687 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562da7da4a30 msgr2=0x562da7d9db20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:10:53.967 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:10:53.961+0000 7f7495e00640 1 --2- 172.21.15.12:0/1681169687 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562da7da4a30 0x562da7d9db20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:10:53.967 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:10:53.961+0000 7f7495e00640 1 -- 172.21.15.12:0/1681169687 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562da7d98aa0 msgr2=0x562da7dad260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:10:53.967 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:10:53.961+0000 7f7495e00640 1 --2- 172.21.15.12:0/1681169687 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562da7d98aa0 0x562da7dad260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:10:53.967 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:10:53.961+0000 7f7495e00640 1 -- 172.21.15.12:0/1681169687 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562da7d9e350 con 0x562da7d98180 2024-09-11T00:10:53.967 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:10:53.961+0000 7f7495400640 1 --2- 172.21.15.12:0/1681169687 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562da7d98aa0 0x562da7dad260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:10:53.967 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:10:53.962+0000 7f7495e00640 1 --2- 172.21.15.12:0/1681169687 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562da7d98180 0x562da7dacce0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x562da7db70e0 tx=0x7f74840046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cd124773a1dc24ba server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:10:53.967 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:10:53.962+0000 7f7473400640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-09-11T00:10:53.967 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:10:53.962+0000 7f7473400640 1 -- 172.21.15.12:0/1681169687 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f748400fe60 con 0x562da7d98180 2024-09-11T00:10:53.967 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:10:53.962+0000 7f749b261740 1 -- 172.21.15.12:0/1681169687 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f748c036040 con 0x562da7d98180 2024-09-11T00:10:53.967 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:10:53.962+0000 7f7473400640 1 -- 172.21.15.12:0/1681169687 <== mon.2 v2:172.21.15.165:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9208+0+0 (secure 0 0 0) 0x7f7484005020 con 0x562da7d98180 2024-09-11T00:10:53.967 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:10:53.962+0000 7f7473400640 1 -- 172.21.15.12:0/1681169687 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f748403a050 con 0x562da7d98180 2024-09-11T00:10:53.967 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:10:53.962+0000 7f749b261740 10 client.25961 fetch_fsmap learned FSMap version 35 2024-09-11T00:10:53.968 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:10:53.962+0000 7f749b261740 1 -- 172.21.15.12:0/1681169687 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x562da7d9f690 con 0x562da7d98180 2024-09-11T00:10:53.968 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:10:53.963+0000 7f7473400640 1 -- 172.21.15.12:0/1681169687 <== mon.2 v2:172.21.15.165:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f748c036040 con 0x562da7d98180 2024-09-11T00:10:53.968 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:10:53.963+0000 7f7473400640 1 -- 172.21.15.12:0/1681169687 <== mon.2 v2:172.21.15.165:3300/0 5 ==== fsmap(e 35) ==== 8378+0+0 (secure 0 0 0) 0x7f748402b410 con 0x562da7d98180 2024-09-11T00:10:53.968 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:10:53.963+0000 7f749b261740 10 client.25961 fetch_fsmap finished waiting for FSMap version 35 2024-09-11T00:10:53.968 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:10:53.963+0000 7f749b261740 10 client.25961 resolve_mds: validated gid 24674 aka daemon mds.b 2024-09-11T00:10:53.968 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:10:53.963+0000 7f749b261740 20 client.25961 populate_metadata read hostname 'smithi012' 2024-09-11T00:10:53.968 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:10:53.963+0000 7f749b261740 1 --2- 172.21.15.12:0/1681169687 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x562da7e227f0 0x562da7e24c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:10:53.968 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:10:53.963+0000 7f749b261740 4 client.25961 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 150.0, "await_for": 45.0}] 2024-09-11T00:10:53.968 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:10:53.963+0000 7f749b261740 1 -- 172.21.15.12:0/1681169687 --> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 150.0, "await_for": 45.0}) -- 0x562da7d0b6a0 con 0x562da7e227f0 2024-09-11T00:10:53.968 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:10:53.963+0000 7f7495400640 1 --2- 172.21.15.12:0/1681169687 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x562da7e227f0 0x562da7e24c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:10:53.968 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:10:53.964+0000 7f7495400640 1 --2- 172.21.15.12:0/1681169687 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x562da7e227f0 0x562da7e24c10 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=52aa562d0f574b66 server_cookie=5d12906f674834cc in_seq=0 out_seq=0 2024-09-11T00:10:53.968 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:10:53.964+0000 7f7473400640 10 client.25961 ms_handle_connect on v2:172.21.15.165:6838/501312984 2024-09-11T00:10:54.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:10:53 smithi165 ceph-mon[35356]: pgmap v513: 129 pgs: 129 active+clean; 34 MiB data, 695 MiB used, 1.0 TiB / 1.0 TiB avail; 618 KiB/s rd, 1.0 MiB/s wr, 104 op/s 2024-09-11T00:10:54.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:10:53 smithi165 ceph-mon[35356]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T00:10:54.052 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:10:54.049+0000 7f7473400640 1 -- 172.21.15.12:0/1681169687 <== mds.0 v2:172.21.15.165:6838/501312984 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x562da7d0b6a0 con 0x562da7e227f0 2024-09-11T00:10:54.052 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:10:54.049+0000 7f7473400640 10 client.25961 handle_command_reply: tid=0 2024-09-11T00:10:54.052 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:10:54.049+0000 7f749b261740 1 client.25961 shutdown 2024-09-11T00:10:54.052 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:10:54.049+0000 7f749b261740 1 -- 172.21.15.12:0/1681169687 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562da7d98180 msgr2=0x562da7dacce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:10:54.052 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:10:54.049+0000 7f749b261740 1 --2- 172.21.15.12:0/1681169687 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562da7d98180 0x562da7dacce0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x562da7db70e0 tx=0x7f74840046c0 comp rx=0 tx=0).stop 2024-09-11T00:10:54.053 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:10:54.050+0000 7f749b261740 1 -- 172.21.15.12:0/1681169687 shutdown_connections 2024-09-11T00:10:54.053 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:10:54.050+0000 7f749b261740 1 --2- 172.21.15.12:0/1681169687 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x562da7e227f0 0x562da7e24c10 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:10:54.053 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:10:54.050+0000 7f749b261740 1 --2- 172.21.15.12:0/1681169687 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562da7da4a30 0x562da7d9db20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:10:54.053 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:10:54.050+0000 7f749b261740 1 --2- 172.21.15.12:0/1681169687 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562da7d98aa0 0x562da7dad260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:10:54.053 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:10:54.050+0000 7f749b261740 1 --2- 172.21.15.12:0/1681169687 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562da7d98180 0x562da7dacce0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:10:54.053 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:10:54.050+0000 7f7473400640 0 client.25961 ms_handle_reset on v2:172.21.15.165:6838/501312984 2024-09-11T00:10:54.053 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:10:54.050+0000 7f749b261740 1 -- 172.21.15.12:0/1681169687 >> 172.21.15.12:0/1681169687 conn(0x562da7d95fb0 msgr2=0x562da7da2da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:10:54.053 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:10:54.050+0000 7f749b261740 1 -- 172.21.15.12:0/1681169687 shutdown_connections 2024-09-11T00:10:54.053 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:10:54.051+0000 7f749b261740 1 -- 172.21.15.12:0/1681169687 wait complete. 2024-09-11T00:10:54.053 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:10:54.051+0000 7f749b261740 20 client.25961 trim_cache size 0 max 16384 2024-09-11T00:10:54.054 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-09-11T00:10:54.054 INFO:teuthology.orchestra.run.smithi012.stdout: "epoch": 35, 2024-09-11T00:10:54.054 INFO:teuthology.orchestra.run.smithi012.stdout: "leader": 24674, 2024-09-11T00:10:54.054 INFO:teuthology.orchestra.run.smithi012.stdout: "set_version": 14, 2024-09-11T00:10:54.054 INFO:teuthology.orchestra.run.smithi012.stdout: "sets": { 2024-09-11T00:10:54.054 INFO:teuthology.orchestra.run.smithi012.stdout: "8c2b0c3c": { 2024-09-11T00:10:54.054 INFO:teuthology.orchestra.run.smithi012.stdout: "version": 14, 2024-09-11T00:10:54.054 INFO:teuthology.orchestra.run.smithi012.stdout: "age_ref": 0.0, 2024-09-11T00:10:54.055 INFO:teuthology.orchestra.run.smithi012.stdout: "state": { 2024-09-11T00:10:54.055 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "QUIESCED", 2024-09-11T00:10:54.055 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 0.0 2024-09-11T00:10:54.055 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:10:54.055 INFO:teuthology.orchestra.run.smithi012.stdout: "timeout": 90.0, 2024-09-11T00:10:54.055 INFO:teuthology.orchestra.run.smithi012.stdout: "expiration": 150.0, 2024-09-11T00:10:54.055 INFO:teuthology.orchestra.run.smithi012.stdout: "members": { 2024-09-11T00:10:54.055 INFO:teuthology.orchestra.run.smithi012.stdout: "file:/": { 2024-09-11T00:10:54.055 INFO:teuthology.orchestra.run.smithi012.stdout: "excluded": false, 2024-09-11T00:10:54.055 INFO:teuthology.orchestra.run.smithi012.stdout: "state": { 2024-09-11T00:10:54.055 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "QUIESCED", 2024-09-11T00:10:54.055 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 0.0 2024-09-11T00:10:54.056 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:10:54.056 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:10:54.056 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:10:54.056 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:10:54.056 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:10:54.056 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-09-11T00:10:54.056 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:10:54.052+0000 7f7498800640 1 -- 172.21.15.12:0/771039303 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f74780b5360 msgr2=0x7f74780b7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:10:54.057 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:10:54.052+0000 7f7498800640 1 --2- 172.21.15.12:0/771039303 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f74780b5360 0x7f74780b7820 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f7484009fd0 tx=0x7f7484005f90 comp rx=0 tx=0).stop 2024-09-11T00:10:54.057 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:10:54.052+0000 7f7498800640 1 -- 172.21.15.12:0/771039303 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7490075f40 msgr2=0x7f749019b960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:10:54.057 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:10:54.052+0000 7f7498800640 1 --2- 172.21.15.12:0/771039303 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7490075f40 0x7f749019b960 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f748000e9f0 tx=0x7f748000eeb0 comp rx=0 tx=0).stop 2024-09-11T00:10:54.057 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:10:54.052+0000 7f7498800640 1 -- 172.21.15.12:0/771039303 shutdown_connections 2024-09-11T00:10:54.057 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:10:54.052+0000 7f7498800640 1 --2- 172.21.15.12:0/771039303 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f74780b5360 0x7f74780b7820 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:10:54.057 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:10:54.052+0000 7f7498800640 1 --2- 172.21.15.12:0/771039303 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f749010f3c0 0x7f74901a0d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:10:54.057 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:10:54.052+0000 7f7498800640 1 --2- 172.21.15.12:0/771039303 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7490075f40 0x7f749019b960 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:10:54.057 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:10:54.052+0000 7f7498800640 1 --2- 172.21.15.12:0/771039303 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7490075540 0x7f749019b420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:10:54.057 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:10:54.052+0000 7f7498800640 1 -- 172.21.15.12:0/771039303 >> 172.21.15.12:0/771039303 conn(0x7f74900fe810 msgr2=0x7f74901115c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:10:54.057 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:10:54.052+0000 7f7498800640 1 -- 172.21.15.12:0/771039303 shutdown_connections 2024-09-11T00:10:54.057 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:10:54.052+0000 7f7498800640 1 -- 172.21.15.12:0/771039303 wait complete. 2024-09-11T00:10:54.625 INFO:tasks.workunit.client.0.smithi012.stdout: 1 121357 2.62 MB/sec execute 51 sec latency 4435.231 ms 2024-09-11T00:10:54.979 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '8c2b0c3c', quiesce took 4.1 seconds. Will release after: 25.9 2024-09-11T00:10:55.625 INFO:tasks.workunit.client.0.smithi012.stdout: 1 121357 2.57 MB/sec execute 52 sec latency 1604.103 ms 2024-09-11T00:10:55.830 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:10:55 smithi178 ceph-mon[35919]: pgmap v514: 129 pgs: 129 active+clean; 34 MiB data, 695 MiB used, 1.0 TiB / 1.0 TiB avail; 587 KiB/s rd, 1008 KiB/s wr, 96 op/s 2024-09-11T00:10:56.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:10:55 smithi165 ceph-mon[35356]: pgmap v514: 129 pgs: 129 active+clean; 34 MiB data, 695 MiB used, 1.0 TiB / 1.0 TiB avail; 587 KiB/s rd, 1008 KiB/s wr, 96 op/s 2024-09-11T00:10:56.063 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:10:55 smithi012 ceph-mon[30333]: pgmap v514: 129 pgs: 129 active+clean; 34 MiB data, 695 MiB used, 1.0 TiB / 1.0 TiB avail; 587 KiB/s rd, 1008 KiB/s wr, 96 op/s 2024-09-11T00:10:56.625 INFO:tasks.workunit.client.0.smithi012.stdout: 1 121357 2.52 MB/sec execute 53 sec latency 2604.236 ms 2024-09-11T00:10:56.845 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:56.842+0000 7f733d600640 1 -- 172.21.15.12:0/2098991444 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7338078b70 msgr2=0x7f7338078ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:10:56.845 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:56.842+0000 7f733d600640 1 --2- 172.21.15.12:0/2098991444 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7338078b70 0x7f7338078ff0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f7324009900 tx=0x7f732402a410 comp rx=0 tx=0).stop 2024-09-11T00:10:56.845 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:56.843+0000 7f733d600640 1 -- 172.21.15.12:0/2098991444 shutdown_connections 2024-09-11T00:10:56.845 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:56.843+0000 7f733d600640 1 --2- 172.21.15.12:0/2098991444 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7338079530 0x7f7338079e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:10:56.845 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:56.843+0000 7f733d600640 1 --2- 172.21.15.12:0/2098991444 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7338078b70 0x7f7338078ff0 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:10:56.845 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:56.843+0000 7f733d600640 1 --2- 172.21.15.12:0/2098991444 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7338077a20 0x7f7338077e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:10:56.845 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:56.843+0000 7f733d600640 1 -- 172.21.15.12:0/2098991444 >> 172.21.15.12:0/2098991444 conn(0x7f7338100270 msgr2=0x7f7338102690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:10:56.846 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:56.843+0000 7f733d600640 1 -- 172.21.15.12:0/2098991444 shutdown_connections 2024-09-11T00:10:56.846 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:56.843+0000 7f733d600640 1 -- 172.21.15.12:0/2098991444 wait complete. 2024-09-11T00:10:56.848 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:56.844+0000 7f733d600640 1 Processor -- start 2024-09-11T00:10:56.848 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:56.844+0000 7f733d600640 1 -- start start 2024-09-11T00:10:56.848 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:56.844+0000 7f733d600640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7338077a20 0x7f733819f190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:10:56.848 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:56.844+0000 7f733d600640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7338078b70 0x7f733819f7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:10:56.848 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:56.844+0000 7f733d600640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7338079530 0x7f73381a0410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:10:56.848 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:56.844+0000 7f733d600640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f7338114a30 con 0x7f7338078b70 2024-09-11T00:10:56.848 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:56.844+0000 7f733d600640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f73381148b0 con 0x7f7338077a20 2024-09-11T00:10:56.848 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:56.844+0000 7f733d600640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f7338114bb0 con 0x7f7338079530 2024-09-11T00:10:56.848 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:56.844+0000 7f7337400640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7338078b70 0x7f733819f7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:10:56.848 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:56.844+0000 7f733cc00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7338079530 0x7f73381a0410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:10:56.849 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:56.844+0000 7f7337400640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7338078b70 0x7f733819f7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:44224/0 (socket says 172.21.15.12:44224) 2024-09-11T00:10:56.849 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:56.844+0000 7f733cc00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7338079530 0x7f73381a0410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.12:60842/0 (socket says 172.21.15.12:60842) 2024-09-11T00:10:56.849 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:56.844+0000 7f7337400640 1 -- 172.21.15.12:0/2183087856 learned_addr learned my addr 172.21.15.12:0/2183087856 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:10:56.849 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:56.844+0000 7f7337e00640 1 --2- 172.21.15.12:0/2183087856 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7338077a20 0x7f733819f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:10:56.849 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:56.845+0000 7f733cc00640 1 -- 172.21.15.12:0/2183087856 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7338077a20 msgr2=0x7f733819f190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:10:56.849 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:56.845+0000 7f733cc00640 1 --2- 172.21.15.12:0/2183087856 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7338077a20 0x7f733819f190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:10:56.849 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:56.845+0000 7f733cc00640 1 -- 172.21.15.12:0/2183087856 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7338078b70 msgr2=0x7f733819f7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:10:56.849 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:56.845+0000 7f733cc00640 1 --2- 172.21.15.12:0/2183087856 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7338078b70 0x7f733819f7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:10:56.849 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:56.845+0000 7f733cc00640 1 -- 172.21.15.12:0/2183087856 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73381a0cd0 con 0x7f7338079530 2024-09-11T00:10:56.849 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:56.845+0000 7f733cc00640 1 --2- 172.21.15.12:0/2183087856 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7338079530 0x7f73381a0410 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f7330009fd0 tx=0x7f733000ef10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:10:56.849 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:56.845+0000 7f7334c00640 1 -- 172.21.15.12:0/2183087856 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f733000ce20 con 0x7f7338079530 2024-09-11T00:10:56.849 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:56.845+0000 7f7334c00640 1 -- 172.21.15.12:0/2183087856 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7330005020 con 0x7f7338079530 2024-09-11T00:10:56.849 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:56.845+0000 7f733d600640 1 -- 172.21.15.12:0/2183087856 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f73381a0fc0 con 0x7f7338079530 2024-09-11T00:10:56.850 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:56.845+0000 7f7334c00640 1 -- 172.21.15.12:0/2183087856 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7330010690 con 0x7f7338079530 2024-09-11T00:10:56.852 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:56.846+0000 7f733d600640 1 -- 172.21.15.12:0/2183087856 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73381a7690 con 0x7f7338079530 2024-09-11T00:10:56.855 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:56.848+0000 7f7341a95740 1 Processor -- start 2024-09-11T00:10:56.855 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:56.849+0000 7f7334c00640 1 -- 172.21.15.12:0/2183087856 <== mon.1 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7f73300054d0 con 0x7f7338079530 2024-09-11T00:10:56.856 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:56.849+0000 7f7341a95740 1 -- start start 2024-09-11T00:10:56.856 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:56.849+0000 7f7341a95740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5648977a8970 0x5648977aad60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:10:56.856 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:56.849+0000 7f7341a95740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5648977ab2a0 0x5648977ad690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:10:56.856 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:56.849+0000 7f7341a95740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5648977adce0 0x5648977b01a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:10:56.856 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:56.849+0000 7f7341a95740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x56489771b6b0 con 0x5648977a8970 2024-09-11T00:10:56.856 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:56.849+0000 7f7341a95740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x56489770aae0 con 0x5648977ab2a0 2024-09-11T00:10:56.856 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:56.849+0000 7f7341a95740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x56489770e8f0 con 0x5648977adce0 2024-09-11T00:10:56.856 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:56.850+0000 7f7337400640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5648977a8970 0x5648977aad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:10:56.856 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:56.850+0000 7f7337400640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5648977a8970 0x5648977aad60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:44228/0 (socket says 172.21.15.12:44228) 2024-09-11T00:10:56.856 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:56.850+0000 7f7337400640 1 -- 172.21.15.12:0/897408360 learned_addr learned my addr 172.21.15.12:0/897408360 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:10:56.856 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:56.850+0000 7f7337e00640 1 --2- 172.21.15.12:0/897408360 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5648977ab2a0 0x5648977ad690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:10:56.857 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:56.850+0000 7f7337400640 1 --2- 172.21.15.12:0/897408360 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5648977adce0 0x5648977b01a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:10:56.857 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:56.850+0000 7f7337400640 1 -- 172.21.15.12:0/897408360 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5648977adce0 msgr2=0x5648977b01a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:10:56.857 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:56.850+0000 7f7337400640 1 --2- 172.21.15.12:0/897408360 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5648977adce0 0x5648977b01a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:10:56.857 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:56.850+0000 7f7337400640 1 -- 172.21.15.12:0/897408360 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5648977ab2a0 msgr2=0x5648977ad690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:10:56.857 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:56.850+0000 7f7337400640 1 --2- 172.21.15.12:0/897408360 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5648977ab2a0 0x5648977ad690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:10:56.857 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:56.850+0000 7f7337400640 1 -- 172.21.15.12:0/897408360 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5648976d7780 con 0x5648977a8970 2024-09-11T00:10:56.857 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:56.850+0000 7f7337400640 1 --2- 172.21.15.12:0/897408360 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5648977a8970 0x5648977aad60 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f7324006010 tx=0x7f7324005e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0be2bb9075cab5 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:10:56.857 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:56.851+0000 7f732d200640 1 -- 172.21.15.12:0/897408360 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7324035070 con 0x5648977a8970 2024-09-11T00:10:56.857 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:56.851+0000 7f732d200640 1 -- 172.21.15.12:0/897408360 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f73240317a0 con 0x5648977a8970 2024-09-11T00:10:56.857 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:56.851+0000 7f732d200640 1 -- 172.21.15.12:0/897408360 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7324003de0 con 0x5648977a8970 2024-09-11T00:10:56.857 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:56.851+0000 7f7334c00640 1 --2- 172.21.15.12:0/2183087856 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f731c0b52e0 0x7f731c0b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:10:56.857 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:56.851+0000 7f7334c00640 1 -- 172.21.15.12:0/2183087856 <== mon.1 v2:172.21.15.178:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9208+0+0 (secure 0 0 0) 0x7f73300e66c0 con 0x7f7338079530 2024-09-11T00:10:56.857 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:56.851+0000 7f7337e00640 1 --2- 172.21.15.12:0/2183087856 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f731c0b52e0 0x7f731c0b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:10:56.858 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:56.851+0000 7f7341a95740 1 -- 172.21.15.12:0/897408360 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5648977a8970 msgr2=0x5648977aad60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:10:56.858 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:56.851+0000 7f7341a95740 1 --2- 172.21.15.12:0/897408360 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5648977a8970 0x5648977aad60 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f7324006010 tx=0x7f7324005e50 comp rx=0 tx=0).stop 2024-09-11T00:10:56.858 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:56.852+0000 7f7341a95740 1 -- 172.21.15.12:0/897408360 shutdown_connections 2024-09-11T00:10:56.858 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:56.852+0000 7f7341a95740 1 --2- 172.21.15.12:0/897408360 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5648977adce0 0x5648977b01a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:10:56.858 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:56.852+0000 7f7341a95740 1 --2- 172.21.15.12:0/897408360 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5648977ab2a0 0x5648977ad690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:10:56.858 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:56.852+0000 7f7341a95740 1 --2- 172.21.15.12:0/897408360 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5648977a8970 0x5648977aad60 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:10:56.858 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:56.852+0000 7f7341a95740 1 -- 172.21.15.12:0/897408360 >> 172.21.15.12:0/897408360 conn(0x5648977a3ac0 msgr2=0x5648977a4850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:10:56.858 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:56.852+0000 7f7341a95740 1 -- 172.21.15.12:0/897408360 shutdown_connections 2024-09-11T00:10:56.858 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:56.852+0000 7f7341a95740 1 -- 172.21.15.12:0/897408360 wait complete. 2024-09-11T00:10:56.858 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:56.852+0000 7f7337e00640 1 --2- 172.21.15.12:0/2183087856 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f731c0b52e0 0x7f731c0b77a0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f7328009da0 tx=0x7f7328005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:10:56.858 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:56.852+0000 7f7341a95740 1 -- start start 2024-09-11T00:10:56.858 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:56.852+0000 7f7341a95740 1 Processor -- start 2024-09-11T00:10:56.859 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:56.853+0000 7f7341a95740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5648977a8970 0x5648977b2100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:10:56.859 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:56.853+0000 7f7341a95740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5648977ab2a0 0x5648977b27a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:10:56.859 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:56.853+0000 7f7341a95740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5648977adce0 0x5648977b3300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:10:56.859 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:56.853+0000 7f7341a95740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x564897642ab0 con 0x5648977ab2a0 2024-09-11T00:10:56.859 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:56.853+0000 7f7341a95740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x5648976b1570 con 0x5648977a8970 2024-09-11T00:10:56.859 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:56.853+0000 7f7341a95740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5648977b0c60 con 0x5648977adce0 2024-09-11T00:10:56.859 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:56.853+0000 7f7341a95740 10 client.0 _ll_register_callbacks cb 0x5648976c0100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:10:56.859 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:56.853+0000 7f7337400640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5648977ab2a0 0x5648977b27a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:10:56.859 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:56.853+0000 7f7337e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5648977a8970 0x5648977b2100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:10:56.860 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:56.853+0000 7f7337e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5648977a8970 0x5648977b2100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.12:36164/0 (socket says 172.21.15.12:36164) 2024-09-11T00:10:56.860 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:56.853+0000 7f7337e00640 1 -- 172.21.15.12:0/3049087185 learned_addr learned my addr 172.21.15.12:0/3049087185 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:10:56.860 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:56.853+0000 7f733cc00640 1 --2- 172.21.15.12:0/3049087185 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5648977adce0 0x5648977b3300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:10:56.860 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:56.854+0000 7f7337400640 1 -- 172.21.15.12:0/3049087185 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5648977adce0 msgr2=0x5648977b3300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:10:56.860 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:56.854+0000 7f7337400640 1 --2- 172.21.15.12:0/3049087185 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5648977adce0 0x5648977b3300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:10:56.860 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:56.854+0000 7f7337400640 1 -- 172.21.15.12:0/3049087185 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5648977a8970 msgr2=0x5648977b2100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:10:56.860 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:56.854+0000 7f7337400640 1 --2- 172.21.15.12:0/3049087185 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5648977a8970 0x5648977b2100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:10:56.860 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:56.854+0000 7f7337400640 1 -- 172.21.15.12:0/3049087185 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56489746c790 con 0x5648977ab2a0 2024-09-11T00:10:56.860 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:56.854+0000 7f7337e00640 1 --2- 172.21.15.12:0/3049087185 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5648977a8970 0x5648977b2100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:10:56.862 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:56.854+0000 7f7337400640 1 --2- 172.21.15.12:0/3049087185 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5648977ab2a0 0x5648977b27a0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f7324006010 tx=0x7f7324003880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=611d1d0ff84b00d6 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:10:56.863 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:56.854+0000 7f7321600640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-09-11T00:10:56.863 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:56.854+0000 7f7321600640 1 -- 172.21.15.12:0/3049087185 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7324035070 con 0x5648977ab2a0 2024-09-11T00:10:56.863 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:56.854+0000 7f7321600640 1 -- 172.21.15.12:0/3049087185 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9208+0+0 (secure 0 0 0) 0x7f7324003b60 con 0x5648977ab2a0 2024-09-11T00:10:56.863 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:56.854+0000 7f7341a95740 1 -- 172.21.15.12:0/3049087185 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f73240317a0 con 0x5648977ab2a0 2024-09-11T00:10:56.863 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:56.855+0000 7f7321600640 1 -- 172.21.15.12:0/3049087185 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f7324003d60 con 0x5648977ab2a0 2024-09-11T00:10:56.863 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:56.855+0000 7f7341a95740 10 client.15951 fetch_fsmap learned FSMap version 35 2024-09-11T00:10:56.863 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:56.855+0000 7f7341a95740 1 -- 172.21.15.12:0/3049087185 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x5648974e5620 con 0x5648977ab2a0 2024-09-11T00:10:56.863 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:56.855+0000 7f7321600640 1 -- 172.21.15.12:0/3049087185 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f73240317a0 con 0x5648977ab2a0 2024-09-11T00:10:56.863 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:56.855+0000 7f7321600640 1 -- 172.21.15.12:0/3049087185 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 35) ==== 8378+0+0 (secure 0 0 0) 0x7f732403b600 con 0x5648977ab2a0 2024-09-11T00:10:56.863 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:56.855+0000 7f7341a95740 10 client.15951 fetch_fsmap finished waiting for FSMap version 35 2024-09-11T00:10:56.863 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:56.855+0000 7f7341a95740 10 client.15951 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T00:10:56.864 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:56.855+0000 7f7341a95740 20 client.15951 populate_metadata read hostname 'smithi012' 2024-09-11T00:10:56.864 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:56.855+0000 7f7341a95740 1 --2- 172.21.15.12:0/3049087185 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x5648977bd530 0x5648977bf990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:10:56.864 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:56.855+0000 7f7341a95740 4 client.15951 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T00:10:56.864 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:56.855+0000 7f7341a95740 1 -- 172.21.15.12:0/3049087185 --> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56489771d600 con 0x5648977bd530 2024-09-11T00:10:56.864 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:56.856+0000 7f7337e00640 1 --2- 172.21.15.12:0/3049087185 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x5648977bd530 0x5648977bf990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:10:56.864 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:56.856+0000 7f7337e00640 1 --2- 172.21.15.12:0/3049087185 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x5648977bd530 0x5648977bf990 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ad85b75c157dd3c6 server_cookie=85fdff65b51f18d3 in_seq=0 out_seq=0 2024-09-11T00:10:56.864 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:56.856+0000 7f7321600640 10 client.15951 ms_handle_connect on v2:172.21.15.165:6838/501312984 2024-09-11T00:10:56.864 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:56.857+0000 7f7321600640 1 -- 172.21.15.12:0/3049087185 <== mds.0 v2:172.21.15.165:6838/501312984 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x56489771d600 con 0x5648977bd530 2024-09-11T00:10:56.864 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:56.857+0000 7f7321600640 10 client.15951 handle_command_reply: tid=0 2024-09-11T00:10:56.864 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:56.857+0000 7f7341a95740 1 client.15951 shutdown 2024-09-11T00:10:56.864 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:56.857+0000 7f7341a95740 1 -- 172.21.15.12:0/3049087185 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5648977ab2a0 msgr2=0x5648977b27a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:10:56.864 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:56.857+0000 7f7341a95740 1 --2- 172.21.15.12:0/3049087185 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5648977ab2a0 0x5648977b27a0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f7324006010 tx=0x7f7324003880 comp rx=0 tx=0).stop 2024-09-11T00:10:56.865 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:56.857+0000 7f7341a95740 1 -- 172.21.15.12:0/3049087185 shutdown_connections 2024-09-11T00:10:56.865 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:56.857+0000 7f7341a95740 1 --2- 172.21.15.12:0/3049087185 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x5648977bd530 0x5648977bf990 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:10:56.865 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:56.857+0000 7f7341a95740 1 --2- 172.21.15.12:0/3049087185 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5648977adce0 0x5648977b3300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:10:56.865 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:56.857+0000 7f7341a95740 1 --2- 172.21.15.12:0/3049087185 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5648977ab2a0 0x5648977b27a0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:10:56.865 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:56.857+0000 7f7341a95740 1 --2- 172.21.15.12:0/3049087185 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5648977a8970 0x5648977b2100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:10:56.865 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:56.857+0000 7f7321600640 0 client.15951 ms_handle_reset on v2:172.21.15.165:6838/501312984 2024-09-11T00:10:56.865 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:56.857+0000 7f7341a95740 1 -- 172.21.15.12:0/3049087185 >> 172.21.15.12:0/3049087185 conn(0x5648977a3ac0 msgr2=0x5648977a2bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:10:56.865 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:56.858+0000 7f7341a95740 1 -- 172.21.15.12:0/3049087185 shutdown_connections 2024-09-11T00:10:56.865 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:56.858+0000 7f7341a95740 1 -- 172.21.15.12:0/3049087185 wait complete. 2024-09-11T00:10:56.865 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:56.858+0000 7f7341a95740 20 client.15951 trim_cache size 0 max 16384 2024-09-11T00:10:56.876 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:56.873+0000 7f7341a95740 1 Processor -- start 2024-09-11T00:10:56.876 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:56.873+0000 7f7341a95740 1 -- start start 2024-09-11T00:10:56.876 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:56.873+0000 7f7341a95740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5648977a8970 0x5648977b2810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:10:56.876 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:56.873+0000 7f7341a95740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5648977ab2a0 0x5648977ba380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:10:56.876 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:56.873+0000 7f7341a95740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5648977adce0 0x5648977b1060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:10:56.876 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:56.873+0000 7f7341a95740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f73381b1a50 con 0x5648977ab2a0 2024-09-11T00:10:56.876 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:56.873+0000 7f7341a95740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f73381b35f0 con 0x5648977a8970 2024-09-11T00:10:56.876 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:56.873+0000 7f7341a95740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f73381b2120 con 0x5648977adce0 2024-09-11T00:10:56.877 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:56.874+0000 7f7337400640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5648977ab2a0 0x5648977ba380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:10:56.877 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:56.874+0000 7f7337400640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5648977ab2a0 0x5648977ba380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:44246/0 (socket says 172.21.15.12:44246) 2024-09-11T00:10:56.877 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:56.874+0000 7f7337400640 1 -- 172.21.15.12:0/2671893528 learned_addr learned my addr 172.21.15.12:0/2671893528 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:10:56.877 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:56.874+0000 7f7337e00640 1 --2- 172.21.15.12:0/2671893528 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5648977a8970 0x5648977b2810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:10:56.877 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:56.874+0000 7f733cc00640 1 --2- 172.21.15.12:0/2671893528 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5648977adce0 0x5648977b1060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:10:56.877 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:56.874+0000 7f7337400640 1 -- 172.21.15.12:0/2671893528 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5648977adce0 msgr2=0x5648977b1060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:10:56.877 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:56.874+0000 7f7337400640 1 --2- 172.21.15.12:0/2671893528 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5648977adce0 0x5648977b1060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:10:56.877 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:56.874+0000 7f7337400640 1 -- 172.21.15.12:0/2671893528 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5648977a8970 msgr2=0x5648977b2810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:10:56.877 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:56.874+0000 7f7337400640 1 --2- 172.21.15.12:0/2671893528 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5648977a8970 0x5648977b2810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:10:56.877 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:56.874+0000 7f7337400640 1 -- 172.21.15.12:0/2671893528 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5648977ab0d0 con 0x5648977ab2a0 2024-09-11T00:10:56.877 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:56.874+0000 7f7337e00640 1 --2- 172.21.15.12:0/2671893528 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5648977a8970 0x5648977b2810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:10:56.881 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:56.874+0000 7f7337400640 1 --2- 172.21.15.12:0/2671893528 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5648977ab2a0 0x5648977ba380 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f732403ba20 tx=0x7f7324003f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6fefcf0c1246b888 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:10:56.881 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:56.875+0000 7f7323e00640 1 -- 172.21.15.12:0/2671893528 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7324035070 con 0x5648977ab2a0 2024-09-11T00:10:56.881 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:56.875+0000 7f7323e00640 1 -- 172.21.15.12:0/2671893528 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7324003af0 con 0x5648977ab2a0 2024-09-11T00:10:56.881 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:56.875+0000 7f7323e00640 1 -- 172.21.15.12:0/2671893528 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7324004390 con 0x5648977ab2a0 2024-09-11T00:10:56.881 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:56.875+0000 7f7341a95740 1 -- 172.21.15.12:0/2671893528 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5648977ab2a0 msgr2=0x5648977ba380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:10:56.881 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:56.875+0000 7f7341a95740 1 --2- 172.21.15.12:0/2671893528 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5648977ab2a0 0x5648977ba380 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f732403ba20 tx=0x7f7324003f00 comp rx=0 tx=0).stop 2024-09-11T00:10:56.881 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:56.875+0000 7f7341a95740 1 -- 172.21.15.12:0/2671893528 shutdown_connections 2024-09-11T00:10:56.881 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:56.875+0000 7f7341a95740 1 --2- 172.21.15.12:0/2671893528 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5648977adce0 0x5648977b1060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:10:56.881 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:56.875+0000 7f7341a95740 1 --2- 172.21.15.12:0/2671893528 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5648977ab2a0 0x5648977ba380 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:10:56.882 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:56.875+0000 7f7341a95740 1 --2- 172.21.15.12:0/2671893528 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5648977a8970 0x5648977b2810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:10:56.882 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:56.875+0000 7f7341a95740 1 -- 172.21.15.12:0/2671893528 >> 172.21.15.12:0/2671893528 conn(0x5648977a3ac0 msgr2=0x5648977b2100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:10:56.882 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:56.875+0000 7f7341a95740 1 -- 172.21.15.12:0/2671893528 shutdown_connections 2024-09-11T00:10:56.882 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:56.875+0000 7f7341a95740 1 -- 172.21.15.12:0/2671893528 wait complete. 2024-09-11T00:10:56.882 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:56.876+0000 7f7341a95740 1 -- start start 2024-09-11T00:10:56.882 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:56.876+0000 7f7341a95740 1 Processor -- start 2024-09-11T00:10:56.882 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:56.876+0000 7f7341a95740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5648977a8970 0x5648977b9290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:10:56.882 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:56.876+0000 7f7341a95740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5648977ab2a0 0x5648977b97d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:10:56.882 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:56.876+0000 7f7341a95740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5648977adce0 0x5648977b9d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:10:56.882 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:56.876+0000 7f7341a95740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f73381b2f20 con 0x5648977ab2a0 2024-09-11T00:10:56.882 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:56.876+0000 7f7341a95740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f73381b27f0 con 0x5648977a8970 2024-09-11T00:10:56.882 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:56.876+0000 7f7341a95740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f73381afa30 con 0x5648977adce0 2024-09-11T00:10:56.882 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:56.876+0000 7f7341a95740 10 client.0 _ll_register_callbacks cb 0x7f73280097a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:10:56.882 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:56.877+0000 7f7337400640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5648977ab2a0 0x5648977b97d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:10:56.883 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:56.877+0000 7f7337400640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5648977ab2a0 0x5648977b97d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:44258/0 (socket says 172.21.15.12:44258) 2024-09-11T00:10:56.883 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:56.877+0000 7f7337400640 1 -- 172.21.15.12:0/11980555 learned_addr learned my addr 172.21.15.12:0/11980555 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:10:56.883 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:56.877+0000 7f733cc00640 1 --2- 172.21.15.12:0/11980555 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5648977adce0 0x5648977b9d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:10:56.883 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:56.877+0000 7f7337e00640 1 --2- 172.21.15.12:0/11980555 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5648977a8970 0x5648977b9290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:10:56.883 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:56.877+0000 7f7337400640 1 -- 172.21.15.12:0/11980555 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5648977adce0 msgr2=0x5648977b9d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:10:56.883 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:56.877+0000 7f7337400640 1 --2- 172.21.15.12:0/11980555 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5648977adce0 0x5648977b9d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:10:56.883 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:56.877+0000 7f7337400640 1 -- 172.21.15.12:0/11980555 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5648977a8970 msgr2=0x5648977b9290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:10:56.883 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:56.877+0000 7f7337400640 1 --2- 172.21.15.12:0/11980555 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5648977a8970 0x5648977b9290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:10:56.883 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:56.877+0000 7f7337400640 1 -- 172.21.15.12:0/11980555 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5648977b80a0 con 0x5648977ab2a0 2024-09-11T00:10:56.883 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:56.877+0000 7f733cc00640 1 --2- 172.21.15.12:0/11980555 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5648977adce0 0x5648977b9d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T00:10:56.883 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:56.877+0000 7f7337400640 1 --2- 172.21.15.12:0/11980555 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5648977ab2a0 0x5648977b97d0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f732402a450 tx=0x7f732403baa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a12cddc18c74d0f3 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:10:56.883 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:56.877+0000 7f7321600640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-09-11T00:10:56.883 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:56.878+0000 7f7321600640 1 -- 172.21.15.12:0/11980555 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7324035030 con 0x5648977ab2a0 2024-09-11T00:10:56.884 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:56.878+0000 7f7341a95740 1 -- 172.21.15.12:0/11980555 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7324003af0 con 0x5648977ab2a0 2024-09-11T00:10:56.884 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:56.878+0000 7f7321600640 1 -- 172.21.15.12:0/11980555 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9208+0+0 (secure 0 0 0) 0x7f732403b670 con 0x5648977ab2a0 2024-09-11T00:10:56.884 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:56.878+0000 7f7321600640 1 -- 172.21.15.12:0/11980555 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f73240315a0 con 0x5648977ab2a0 2024-09-11T00:10:56.884 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:56.878+0000 7f7341a95740 10 client.15957 fetch_fsmap learned FSMap version 35 2024-09-11T00:10:56.884 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:56.878+0000 7f7341a95740 1 -- 172.21.15.12:0/11980555 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x56489782aa70 con 0x5648977ab2a0 2024-09-11T00:10:56.884 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:56.878+0000 7f7321600640 1 -- 172.21.15.12:0/11980555 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7324003af0 con 0x5648977ab2a0 2024-09-11T00:10:56.884 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:56.878+0000 7f7321600640 1 -- 172.21.15.12:0/11980555 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 35) ==== 8378+0+0 (secure 0 0 0) 0x7f732406a760 con 0x5648977ab2a0 2024-09-11T00:10:56.884 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:56.878+0000 7f7341a95740 10 client.15957 fetch_fsmap finished waiting for FSMap version 35 2024-09-11T00:10:56.884 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:56.878+0000 7f7341a95740 10 client.15957 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T00:10:56.884 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:56.878+0000 7f7341a95740 20 client.15957 populate_metadata read hostname 'smithi012' 2024-09-11T00:10:56.884 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:56.878+0000 7f7341a95740 1 --2- 172.21.15.12:0/11980555 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x56489782b240 0x56489782d660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:10:56.884 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:56.878+0000 7f7341a95740 4 client.15957 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T00:10:56.885 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:56.878+0000 7f7341a95740 1 -- 172.21.15.12:0/11980555 --> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5648977190b0 con 0x56489782b240 2024-09-11T00:10:56.885 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:56.879+0000 7f7337e00640 1 --2- 172.21.15.12:0/11980555 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x56489782b240 0x56489782d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:10:56.885 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:56.879+0000 7f7337e00640 1 --2- 172.21.15.12:0/11980555 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x56489782b240 0x56489782d660 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bca9213b94e81e0e server_cookie=5535d1a12bd59b30 in_seq=0 out_seq=0 2024-09-11T00:10:56.885 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:56.879+0000 7f7321600640 10 client.15957 ms_handle_connect on v2:172.21.15.165:6838/501312984 2024-09-11T00:10:56.913 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:56.910+0000 7f7321600640 1 -- 172.21.15.12:0/11980555 <== mds.0 v2:172.21.15.165:6838/501312984 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5648977190b0 con 0x56489782b240 2024-09-11T00:10:56.913 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:56.910+0000 7f7321600640 10 client.15957 handle_command_reply: tid=0 2024-09-11T00:10:56.913 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:56.910+0000 7f7341a95740 1 client.15957 shutdown 2024-09-11T00:10:56.913 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:56.910+0000 7f7341a95740 1 -- 172.21.15.12:0/11980555 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5648977ab2a0 msgr2=0x5648977b97d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:10:56.913 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:56.910+0000 7f7341a95740 1 --2- 172.21.15.12:0/11980555 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5648977ab2a0 0x5648977b97d0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f732402a450 tx=0x7f732403baa0 comp rx=0 tx=0).stop 2024-09-11T00:10:56.913 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:56.911+0000 7f7341a95740 1 -- 172.21.15.12:0/11980555 shutdown_connections 2024-09-11T00:10:56.913 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:56.911+0000 7f7341a95740 1 --2- 172.21.15.12:0/11980555 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x56489782b240 0x56489782d660 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:10:56.913 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:56.911+0000 7f7341a95740 1 --2- 172.21.15.12:0/11980555 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5648977adce0 0x5648977b9d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:10:56.913 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:56.911+0000 7f7341a95740 1 --2- 172.21.15.12:0/11980555 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5648977ab2a0 0x5648977b97d0 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:10:56.914 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:56.911+0000 7f7321600640 0 client.15957 ms_handle_reset on v2:172.21.15.165:6838/501312984 2024-09-11T00:10:56.914 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:56.911+0000 7f7341a95740 1 --2- 172.21.15.12:0/11980555 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5648977a8970 0x5648977b9290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:10:56.914 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:56.911+0000 7f7341a95740 1 -- 172.21.15.12:0/11980555 >> 172.21.15.12:0/11980555 conn(0x5648977a3ac0 msgr2=0x5648977b1060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:10:56.914 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:56.911+0000 7f7341a95740 1 -- 172.21.15.12:0/11980555 shutdown_connections 2024-09-11T00:10:56.914 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:56.911+0000 7f7341a95740 1 -- 172.21.15.12:0/11980555 wait complete. 2024-09-11T00:10:56.914 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:56.911+0000 7f7341a95740 20 client.15957 trim_cache size 0 max 16384 2024-09-11T00:10:56.914 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:{ 2024-09-11T00:10:56.914 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "message": "", 2024-09-11T00:10:56.914 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "return_code": 0 2024-09-11T00:10:56.915 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:} 2024-09-11T00:10:56.915 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:56.912+0000 7f733d600640 1 -- 172.21.15.12:0/2183087856 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f731c0b52e0 msgr2=0x7f731c0b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:10:56.915 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:56.912+0000 7f733d600640 1 --2- 172.21.15.12:0/2183087856 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f731c0b52e0 0x7f731c0b77a0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f7328009da0 tx=0x7f7328005f90 comp rx=0 tx=0).stop 2024-09-11T00:10:56.915 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:56.912+0000 7f733d600640 1 -- 172.21.15.12:0/2183087856 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7338079530 msgr2=0x7f73381a0410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:10:56.916 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:56.912+0000 7f733d600640 1 --2- 172.21.15.12:0/2183087856 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7338079530 0x7f73381a0410 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f7330009fd0 tx=0x7f733000ef10 comp rx=0 tx=0).stop 2024-09-11T00:10:56.916 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:56.912+0000 7f733d600640 1 -- 172.21.15.12:0/2183087856 shutdown_connections 2024-09-11T00:10:56.916 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:56.912+0000 7f733d600640 1 --2- 172.21.15.12:0/2183087856 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f731c0b52e0 0x7f731c0b77a0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:10:56.916 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:56.912+0000 7f733d600640 1 --2- 172.21.15.12:0/2183087856 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7338079530 0x7f73381a0410 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:10:56.916 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:56.912+0000 7f733d600640 1 --2- 172.21.15.12:0/2183087856 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7338078b70 0x7f733819f7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:10:56.916 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:56.912+0000 7f733d600640 1 --2- 172.21.15.12:0/2183087856 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7338077a20 0x7f733819f190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:10:56.916 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:56.912+0000 7f733d600640 1 -- 172.21.15.12:0/2183087856 >> 172.21.15.12:0/2183087856 conn(0x7f7338100270 msgr2=0x7f7338101d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:10:56.916 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:56.912+0000 7f733d600640 1 -- 172.21.15.12:0/2183087856 shutdown_connections 2024-09-11T00:10:56.916 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:10:56.912+0000 7f733d600640 1 -- 172.21.15.12:0/2183087856 wait complete. 2024-09-11T00:10:57.625 INFO:tasks.workunit.client.0.smithi012.stdout: 1 121357 2.48 MB/sec execute 54 sec latency 3604.427 ms 2024-09-11T00:10:57.831 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:10:57 smithi178 ceph-mon[35919]: pgmap v515: 129 pgs: 129 active+clean; 34 MiB data, 695 MiB used, 1.0 TiB / 1.0 TiB avail; 613 KiB/s rd, 1.1 MiB/s wr, 110 op/s 2024-09-11T00:10:58.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:10:57 smithi165 ceph-mon[35356]: pgmap v515: 129 pgs: 129 active+clean; 34 MiB data, 695 MiB used, 1.0 TiB / 1.0 TiB avail; 613 KiB/s rd, 1.1 MiB/s wr, 110 op/s 2024-09-11T00:10:58.063 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:10:57 smithi012 ceph-mon[30333]: pgmap v515: 129 pgs: 129 active+clean; 34 MiB data, 695 MiB used, 1.0 TiB / 1.0 TiB avail; 613 KiB/s rd, 1.1 MiB/s wr, 110 op/s 2024-09-11T00:10:58.626 INFO:tasks.workunit.client.0.smithi012.stdout: 1 121357 2.43 MB/sec execute 55 sec latency 4604.590 ms 2024-09-11T00:10:59.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:10:58 smithi165 ceph-mon[35356]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T00:10:59.063 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:10:58 smithi012 ceph-mon[30333]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T00:10:59.080 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:10:58 smithi178 ceph-mon[35919]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T00:10:59.626 INFO:tasks.workunit.client.0.smithi012.stdout: 1 121357 2.39 MB/sec execute 56 sec latency 5604.772 ms 2024-09-11T00:11:00.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:10:59 smithi165 ceph-mon[35356]: pgmap v516: 129 pgs: 129 active+clean; 34 MiB data, 695 MiB used, 1.0 TiB / 1.0 TiB avail; 600 KiB/s rd, 434 KiB/s wr, 87 op/s 2024-09-11T00:11:00.063 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:10:59 smithi012 ceph-mon[30333]: pgmap v516: 129 pgs: 129 active+clean; 34 MiB data, 695 MiB used, 1.0 TiB / 1.0 TiB avail; 600 KiB/s rd, 434 KiB/s wr, 87 op/s 2024-09-11T00:11:00.080 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:10:59 smithi178 ceph-mon[35919]: pgmap v516: 129 pgs: 129 active+clean; 34 MiB data, 695 MiB used, 1.0 TiB / 1.0 TiB avail; 600 KiB/s rd, 434 KiB/s wr, 87 op/s 2024-09-11T00:11:00.626 INFO:tasks.workunit.client.0.smithi012.stdout: 1 121357 2.35 MB/sec execute 57 sec latency 6604.872 ms 2024-09-11T00:11:01.626 INFO:tasks.workunit.client.0.smithi012.stdout: 1 121357 2.30 MB/sec execute 58 sec latency 7605.007 ms 2024-09-11T00:11:02.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:11:01 smithi165 ceph-mon[35356]: pgmap v517: 129 pgs: 129 active+clean; 34 MiB data, 691 MiB used, 1.0 TiB / 1.0 TiB avail; 559 KiB/s rd, 285 KiB/s wr, 82 op/s 2024-09-11T00:11:02.063 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:11:01 smithi012 ceph-mon[30333]: pgmap v517: 129 pgs: 129 active+clean; 34 MiB data, 691 MiB used, 1.0 TiB / 1.0 TiB avail; 559 KiB/s rd, 285 KiB/s wr, 82 op/s 2024-09-11T00:11:02.080 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:11:01 smithi178 ceph-mon[35919]: pgmap v517: 129 pgs: 129 active+clean; 34 MiB data, 691 MiB used, 1.0 TiB / 1.0 TiB avail; 559 KiB/s rd, 285 KiB/s wr, 82 op/s 2024-09-11T00:11:02.626 INFO:tasks.workunit.client.0.smithi012.stdout: 1 121357 2.27 MB/sec execute 59 sec latency 8605.164 ms 2024-09-11T00:11:03.627 INFO:tasks.workunit.client.0.smithi012.stdout: 1 121357 2.23 MB/sec execute 60 sec latency 9605.323 ms 2024-09-11T00:11:04.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:11:03 smithi165 ceph-mon[35356]: pgmap v518: 129 pgs: 129 active+clean; 30 MiB data, 683 MiB used, 1.0 TiB / 1.0 TiB avail; 200 KiB/s rd, 196 KiB/s wr, 58 op/s 2024-09-11T00:11:04.063 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:11:03 smithi012 ceph-mon[30333]: pgmap v518: 129 pgs: 129 active+clean; 30 MiB data, 683 MiB used, 1.0 TiB / 1.0 TiB avail; 200 KiB/s rd, 196 KiB/s wr, 58 op/s 2024-09-11T00:11:04.080 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:11:03 smithi178 ceph-mon[35919]: pgmap v518: 129 pgs: 129 active+clean; 30 MiB data, 683 MiB used, 1.0 TiB / 1.0 TiB avail; 200 KiB/s rd, 196 KiB/s wr, 58 op/s 2024-09-11T00:11:04.627 INFO:tasks.workunit.client.0.smithi012.stdout: 1 121357 2.19 MB/sec execute 61 sec latency 10605.450 ms 2024-09-11T00:11:05.627 INFO:tasks.workunit.client.0.smithi012.stdout: 1 121357 2.16 MB/sec execute 62 sec latency 11605.525 ms 2024-09-11T00:11:06.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:11:05 smithi165 ceph-mon[35356]: pgmap v519: 129 pgs: 129 active+clean; 30 MiB data, 683 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 117 KiB/s wr, 32 op/s 2024-09-11T00:11:06.063 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:11:05 smithi012 ceph-mon[30333]: pgmap v519: 129 pgs: 129 active+clean; 30 MiB data, 683 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 117 KiB/s wr, 32 op/s 2024-09-11T00:11:06.080 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:11:05 smithi178 ceph-mon[35919]: pgmap v519: 129 pgs: 129 active+clean; 30 MiB data, 683 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 117 KiB/s wr, 32 op/s 2024-09-11T00:11:06.627 INFO:tasks.workunit.client.0.smithi012.stdout: 1 121357 2.12 MB/sec execute 63 sec latency 12605.631 ms 2024-09-11T00:11:07.627 INFO:tasks.workunit.client.0.smithi012.stdout: 1 121357 2.09 MB/sec execute 64 sec latency 13605.713 ms 2024-09-11T00:11:07.813 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:11:07 smithi012 ceph-mon[30333]: pgmap v520: 129 pgs: 129 active+clean; 30 MiB data, 683 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 117 KiB/s wr, 37 op/s 2024-09-11T00:11:07.882 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:11:07 smithi178 ceph-mon[35919]: pgmap v520: 129 pgs: 129 active+clean; 30 MiB data, 683 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 117 KiB/s wr, 37 op/s 2024-09-11T00:11:08.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:11:07 smithi165 ceph-mon[35356]: pgmap v520: 129 pgs: 129 active+clean; 30 MiB data, 683 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 117 KiB/s wr, 37 op/s 2024-09-11T00:11:08.627 INFO:tasks.workunit.client.0.smithi012.stdout: 1 121357 2.06 MB/sec execute 65 sec latency 14605.843 ms 2024-09-11T00:11:09.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:11:08 smithi165 ceph-mon[35356]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T00:11:09.063 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:11:08 smithi012 ceph-mon[30333]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T00:11:09.080 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:11:08 smithi178 ceph-mon[35919]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T00:11:09.627 INFO:tasks.workunit.client.0.smithi012.stdout: 1 121357 2.03 MB/sec execute 66 sec latency 15605.964 ms 2024-09-11T00:11:10.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:11:09 smithi165 ceph-mon[35356]: pgmap v521: 129 pgs: 129 active+clean; 30 MiB data, 683 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 KiB/s wr, 21 op/s 2024-09-11T00:11:10.041 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:10.039+0000 7f5609000640 1 -- 172.21.15.12:0/2352030042 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5604107ca0 msgr2=0x7f560410a090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:11:10.041 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:10.039+0000 7f5609000640 1 --2- 172.21.15.12:0/2352030042 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5604107ca0 0x7f560410a090 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f55ec009900 tx=0x7f55ec02a410 comp rx=0 tx=0).stop 2024-09-11T00:11:10.042 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:10.039+0000 7f5609000640 1 -- 172.21.15.12:0/2352030042 shutdown_connections 2024-09-11T00:11:10.042 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:10.039+0000 7f5609000640 1 --2- 172.21.15.12:0/2352030042 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f560410a6e0 0x7f560410cb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:11:10.042 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:10.039+0000 7f5609000640 1 --2- 172.21.15.12:0/2352030042 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5604107ca0 0x7f560410a090 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:11:10.042 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:10.039+0000 7f5609000640 1 --2- 172.21.15.12:0/2352030042 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f560406b360 0x7f5604107760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:11:10.042 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:10.039+0000 7f5609000640 1 -- 172.21.15.12:0/2352030042 >> 172.21.15.12:0/2352030042 conn(0x7f56040fcf30 msgr2=0x7f56040ff370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:11:10.042 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:10.039+0000 7f5609000640 1 -- 172.21.15.12:0/2352030042 shutdown_connections 2024-09-11T00:11:10.042 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:10.039+0000 7f5609000640 1 -- 172.21.15.12:0/2352030042 wait complete. 2024-09-11T00:11:10.044 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:10.040+0000 7f5609000640 1 Processor -- start 2024-09-11T00:11:10.044 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:10.040+0000 7f5609000640 1 -- start start 2024-09-11T00:11:10.044 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:10.041+0000 7f5609000640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f560406b360 0x7f560419adc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:11:10.044 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:10.041+0000 7f5609000640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5604107ca0 0x7f560419b410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:11:10.044 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:10.041+0000 7f5609000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f560410a6e0 0x7f560419c040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:11:10.044 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:10.041+0000 7f5609000640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f5604110610 con 0x7f560406b360 2024-09-11T00:11:10.044 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:10.041+0000 7f5609000640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f5604110490 con 0x7f560410a6e0 2024-09-11T00:11:10.044 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:10.041+0000 7f5609000640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f5604110790 con 0x7f5604107ca0 2024-09-11T00:11:10.044 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:10.041+0000 7f5603400640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f560406b360 0x7f560419adc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:11:10.044 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:10.041+0000 7f5603400640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f560406b360 0x7f560419adc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:40570/0 (socket says 172.21.15.12:40570) 2024-09-11T00:11:10.045 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:10.041+0000 7f5603400640 1 -- 172.21.15.12:0/3024881641 learned_addr learned my addr 172.21.15.12:0/3024881641 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:11:10.045 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:10.041+0000 7f5603e00640 1 --2- 172.21.15.12:0/3024881641 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f560410a6e0 0x7f560419c040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:11:10.045 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:10.041+0000 7f55faa00640 1 --2- 172.21.15.12:0/3024881641 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5604107ca0 0x7f560419b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:11:10.045 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:10.042+0000 7f5603400640 1 -- 172.21.15.12:0/3024881641 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5604107ca0 msgr2=0x7f560419b410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:11:10.045 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:10.042+0000 7f5603400640 1 --2- 172.21.15.12:0/3024881641 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5604107ca0 0x7f560419b410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:11:10.045 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:10.042+0000 7f5603400640 1 -- 172.21.15.12:0/3024881641 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f560410a6e0 msgr2=0x7f560419c040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:11:10.045 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:10.042+0000 7f5603400640 1 --2- 172.21.15.12:0/3024881641 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f560410a6e0 0x7f560419c040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:11:10.045 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:10.042+0000 7f5603400640 1 -- 172.21.15.12:0/3024881641 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f560419c900 con 0x7f560406b360 2024-09-11T00:11:10.045 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:10.042+0000 7f5603e00640 1 --2- 172.21.15.12:0/3024881641 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f560410a6e0 0x7f560419c040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:11:10.045 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:10.042+0000 7f5603400640 1 --2- 172.21.15.12:0/3024881641 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f560406b360 0x7f560419adc0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f55f400eb90 tx=0x7f55f400c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:11:10.046 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:10.042+0000 7f5600c00640 1 -- 172.21.15.12:0/3024881641 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55f4004d20 con 0x7f560406b360 2024-09-11T00:11:10.046 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:10.042+0000 7f5600c00640 1 -- 172.21.15.12:0/3024881641 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f55f400ee90 con 0x7f560406b360 2024-09-11T00:11:10.046 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:10.042+0000 7f5600c00640 1 -- 172.21.15.12:0/3024881641 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55f4010670 con 0x7f560406b360 2024-09-11T00:11:10.046 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:10.042+0000 7f5609000640 1 -- 172.21.15.12:0/3024881641 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f560419cbf0 con 0x7f560406b360 2024-09-11T00:11:10.046 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:10.042+0000 7f5609000640 1 -- 172.21.15.12:0/3024881641 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56041a32c0 con 0x7f560406b360 2024-09-11T00:11:10.047 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:10.045+0000 7f560d647740 1 Processor -- start 2024-09-11T00:11:10.047 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:10.045+0000 7f560d647740 1 -- start start 2024-09-11T00:11:10.049 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:10.046+0000 7f560d647740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5570aa72b970 0x5570aa72dd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:11:10.049 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:10.046+0000 7f560d647740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5570aa72e2a0 0x5570aa730690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:11:10.049 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:10.046+0000 7f560d647740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5570aa730ce0 0x5570aa7331a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:11:10.049 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:10.046+0000 7f560d647740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x5570aa69e6b0 con 0x5570aa72e2a0 2024-09-11T00:11:10.049 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:10.046+0000 7f560d647740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x5570aa68dae0 con 0x5570aa730ce0 2024-09-11T00:11:10.049 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:10.046+0000 7f560d647740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5570aa6918f0 con 0x5570aa72b970 2024-09-11T00:11:10.049 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:10.046+0000 7f5600c00640 1 -- 172.21.15.12:0/3024881641 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7f55f4005020 con 0x7f560406b360 2024-09-11T00:11:10.049 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:10.046+0000 7f5603e00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5570aa72b970 0x5570aa72dd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:11:10.049 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:10.046+0000 7f5603e00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5570aa72b970 0x5570aa72dd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.12:58714/0 (socket says 172.21.15.12:58714) 2024-09-11T00:11:10.049 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:10.046+0000 7f5603e00640 1 -- 172.21.15.12:0/2526318632 learned_addr learned my addr 172.21.15.12:0/2526318632 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:11:10.050 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:10.046+0000 7f5603400640 1 --2- 172.21.15.12:0/2526318632 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5570aa72e2a0 0x5570aa730690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:11:10.050 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:10.046+0000 7f55faa00640 1 --2- 172.21.15.12:0/2526318632 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5570aa730ce0 0x5570aa7331a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:11:10.050 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:10.046+0000 7f5603e00640 1 -- 172.21.15.12:0/2526318632 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5570aa730ce0 msgr2=0x5570aa7331a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:11:10.050 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:10.046+0000 7f5603e00640 1 --2- 172.21.15.12:0/2526318632 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5570aa730ce0 0x5570aa7331a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:11:10.050 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:10.046+0000 7f5603e00640 1 -- 172.21.15.12:0/2526318632 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5570aa72e2a0 msgr2=0x5570aa730690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:11:10.050 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:10.046+0000 7f5603e00640 1 --2- 172.21.15.12:0/2526318632 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5570aa72e2a0 0x5570aa730690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:11:10.050 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:10.046+0000 7f5603e00640 1 -- 172.21.15.12:0/2526318632 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5570aa65a780 con 0x5570aa72b970 2024-09-11T00:11:10.050 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:10.046+0000 7f5603400640 1 --2- 172.21.15.12:0/2526318632 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5570aa72e2a0 0x5570aa730690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T00:11:10.050 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:10.047+0000 7f5603e00640 1 --2- 172.21.15.12:0/2526318632 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5570aa72b970 0x5570aa72dd60 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f55fc011470 tx=0x7f55fc011930 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c8d4ba1768bb2fc9 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:11:10.050 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:10.047+0000 7f55f8c00640 1 -- 172.21.15.12:0/2526318632 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55fc004020 con 0x5570aa72b970 2024-09-11T00:11:10.053 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:10.047+0000 7f55f8c00640 1 -- 172.21.15.12:0/2526318632 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f55fc0041c0 con 0x5570aa72b970 2024-09-11T00:11:10.054 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:10.047+0000 7f55f8c00640 1 -- 172.21.15.12:0/2526318632 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55fc00f7a0 con 0x5570aa72b970 2024-09-11T00:11:10.054 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:10.048+0000 7f5600c00640 1 --2- 172.21.15.12:0/3024881641 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f55e80b5360 0x7f55e80b7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:11:10.054 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:10.048+0000 7f560d647740 1 -- 172.21.15.12:0/2526318632 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5570aa72b970 msgr2=0x5570aa72dd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:11:10.054 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:10.048+0000 7f560d647740 1 --2- 172.21.15.12:0/2526318632 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5570aa72b970 0x5570aa72dd60 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f55fc011470 tx=0x7f55fc011930 comp rx=0 tx=0).stop 2024-09-11T00:11:10.054 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:10.048+0000 7f560d647740 1 -- 172.21.15.12:0/2526318632 shutdown_connections 2024-09-11T00:11:10.054 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:10.048+0000 7f560d647740 1 --2- 172.21.15.12:0/2526318632 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5570aa730ce0 0x5570aa7331a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:11:10.054 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:10.048+0000 7f560d647740 1 --2- 172.21.15.12:0/2526318632 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5570aa72e2a0 0x5570aa730690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:11:10.054 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:10.048+0000 7f560d647740 1 --2- 172.21.15.12:0/2526318632 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5570aa72b970 0x5570aa72dd60 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:11:10.054 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:10.048+0000 7f560d647740 1 -- 172.21.15.12:0/2526318632 >> 172.21.15.12:0/2526318632 conn(0x5570aa726ac0 msgr2=0x5570aa727850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:11:10.054 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:10.048+0000 7f5600c00640 1 -- 172.21.15.12:0/3024881641 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9208+0+0 (secure 0 0 0) 0x7f55f40e65c0 con 0x7f560406b360 2024-09-11T00:11:10.055 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:10.048+0000 7f5603400640 1 --2- 172.21.15.12:0/3024881641 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f55e80b5360 0x7f55e80b7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:11:10.055 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:10.048+0000 7f560d647740 1 -- 172.21.15.12:0/2526318632 shutdown_connections 2024-09-11T00:11:10.055 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:10.048+0000 7f560d647740 1 -- 172.21.15.12:0/2526318632 wait complete. 2024-09-11T00:11:10.055 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:10.049+0000 7f560d647740 1 -- start start 2024-09-11T00:11:10.055 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:10.049+0000 7f5603400640 1 --2- 172.21.15.12:0/3024881641 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f55e80b5360 0x7f55e80b7820 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f55f40ac8e0 tx=0x7f55f400cf90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:11:10.055 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:10.049+0000 7f560d647740 1 Processor -- start 2024-09-11T00:11:10.055 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:10.049+0000 7f560d647740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5570aa72b970 0x5570aa735050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:11:10.055 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:10.049+0000 7f560d647740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5570aa72e2a0 0x5570aa7356f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:11:10.055 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:10.049+0000 7f560d647740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5570aa730ce0 0x5570aa7362c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:11:10.055 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:10.049+0000 7f560d647740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x5570aa5c5ab0 con 0x5570aa730ce0 2024-09-11T00:11:10.055 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:10.049+0000 7f560d647740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x5570aa634570 con 0x5570aa72e2a0 2024-09-11T00:11:10.056 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:10.049+0000 7f560d647740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5570aa733cc0 con 0x5570aa72b970 2024-09-11T00:11:10.056 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:10.049+0000 7f560d647740 10 client.0 _ll_register_callbacks cb 0x5570aa643100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:11:10.056 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:10.050+0000 7f5603e00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5570aa730ce0 0x5570aa7362c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:11:10.056 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:10.050+0000 7f5603e00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5570aa730ce0 0x5570aa7362c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:40586/0 (socket says 172.21.15.12:40586) 2024-09-11T00:11:10.056 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:10.050+0000 7f5603e00640 1 -- 172.21.15.12:0/2837257693 learned_addr learned my addr 172.21.15.12:0/2837257693 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:11:10.056 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:10.050+0000 7f5603e00640 1 --2- 172.21.15.12:0/2837257693 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5570aa72b970 0x5570aa735050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:11:10.056 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:10.050+0000 7f55faa00640 1 --2- 172.21.15.12:0/2837257693 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5570aa72e2a0 0x5570aa7356f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:11:10.056 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:10.050+0000 7f5603e00640 1 -- 172.21.15.12:0/2837257693 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5570aa72b970 msgr2=0x5570aa735050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:11:10.056 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:10.050+0000 7f5603e00640 1 --2- 172.21.15.12:0/2837257693 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5570aa72b970 0x5570aa735050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:11:10.056 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:10.050+0000 7f5603e00640 1 -- 172.21.15.12:0/2837257693 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5570aa72e2a0 msgr2=0x5570aa7356f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:11:10.056 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:10.050+0000 7f5603e00640 1 --2- 172.21.15.12:0/2837257693 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5570aa72e2a0 0x5570aa7356f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:11:10.056 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:10.050+0000 7f5603e00640 1 -- 172.21.15.12:0/2837257693 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5570aa3ef790 con 0x5570aa730ce0 2024-09-11T00:11:10.057 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:10.050+0000 7f5603e00640 1 --2- 172.21.15.12:0/2837257693 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5570aa730ce0 0x5570aa7362c0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f55fc004190 tx=0x7f55fc009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ccc750202fa0ebca server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:11:10.057 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:10.050+0000 7f55f1600640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-09-11T00:11:10.057 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:10.051+0000 7f55f1600640 1 -- 172.21.15.12:0/2837257693 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55fc0039b0 con 0x5570aa730ce0 2024-09-11T00:11:10.057 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:10.051+0000 7f55f1600640 1 -- 172.21.15.12:0/2837257693 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9208+0+0 (secure 0 0 0) 0x7f55fc01f040 con 0x5570aa730ce0 2024-09-11T00:11:10.057 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:10.051+0000 7f560d647740 1 -- 172.21.15.12:0/2837257693 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f55fc0041c0 con 0x5570aa730ce0 2024-09-11T00:11:10.057 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:10.051+0000 7f55f1600640 1 -- 172.21.15.12:0/2837257693 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f55fc033050 con 0x5570aa730ce0 2024-09-11T00:11:10.057 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:10.051+0000 7f560d647740 10 client.15969 fetch_fsmap learned FSMap version 37 2024-09-11T00:11:10.057 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:10.051+0000 7f560d647740 1 -- 172.21.15.12:0/2837257693 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x5570aa468620 con 0x5570aa730ce0 2024-09-11T00:11:10.057 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:10.051+0000 7f55f1600640 1 -- 172.21.15.12:0/2837257693 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f55fc0041c0 con 0x5570aa730ce0 2024-09-11T00:11:10.057 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:10.052+0000 7f55f1600640 1 -- 172.21.15.12:0/2837257693 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 37) ==== 8370+0+0 (secure 0 0 0) 0x7f55fc004c60 con 0x5570aa730ce0 2024-09-11T00:11:10.057 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:10.052+0000 7f560d647740 10 client.15969 fetch_fsmap finished waiting for FSMap version 37 2024-09-11T00:11:10.057 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:10.052+0000 7f560d647740 10 client.15969 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T00:11:10.057 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:10.052+0000 7f560d647740 20 client.15969 populate_metadata read hostname 'smithi012' 2024-09-11T00:11:10.058 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:10.052+0000 7f560d647740 1 --2- 172.21.15.12:0/2837257693 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x5570aa740700 0x5570aa742af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:11:10.058 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:10.052+0000 7f560d647740 4 client.15969 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T00:11:10.058 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:10.052+0000 7f560d647740 1 -- 172.21.15.12:0/2837257693 --> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5570aa6a0600 con 0x5570aa740700 2024-09-11T00:11:10.059 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:10.052+0000 7f55faa00640 1 --2- 172.21.15.12:0/2837257693 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x5570aa740700 0x5570aa742af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:11:10.059 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:10.053+0000 7f55faa00640 1 --2- 172.21.15.12:0/2837257693 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x5570aa740700 0x5570aa742af0 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dcfeda634b6f7e70 server_cookie=cfbbb0ce97be2cc3 in_seq=0 out_seq=0 2024-09-11T00:11:10.059 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:10.053+0000 7f55f1600640 10 client.15969 ms_handle_connect on v2:172.21.15.165:6838/501312984 2024-09-11T00:11:10.059 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:10.054+0000 7f55f1600640 1 -- 172.21.15.12:0/2837257693 <== mds.0 v2:172.21.15.165:6838/501312984 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5570aa6a0600 con 0x5570aa740700 2024-09-11T00:11:10.059 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:10.054+0000 7f55f1600640 10 client.15969 handle_command_reply: tid=0 2024-09-11T00:11:10.059 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:10.054+0000 7f560d647740 1 client.15969 shutdown 2024-09-11T00:11:10.059 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:10.054+0000 7f560d647740 1 -- 172.21.15.12:0/2837257693 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5570aa730ce0 msgr2=0x5570aa7362c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:11:10.059 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:10.054+0000 7f560d647740 1 --2- 172.21.15.12:0/2837257693 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5570aa730ce0 0x5570aa7362c0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f55fc004190 tx=0x7f55fc009310 comp rx=0 tx=0).stop 2024-09-11T00:11:10.059 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:10.054+0000 7f560d647740 1 -- 172.21.15.12:0/2837257693 shutdown_connections 2024-09-11T00:11:10.059 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:10.054+0000 7f560d647740 1 --2- 172.21.15.12:0/2837257693 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x5570aa740700 0x5570aa742af0 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:11:10.060 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:10.054+0000 7f560d647740 1 --2- 172.21.15.12:0/2837257693 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5570aa730ce0 0x5570aa7362c0 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:11:10.060 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:10.054+0000 7f560d647740 1 --2- 172.21.15.12:0/2837257693 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5570aa72e2a0 0x5570aa7356f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:11:10.060 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:10.054+0000 7f560d647740 1 --2- 172.21.15.12:0/2837257693 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5570aa72b970 0x5570aa735050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:11:10.060 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:10.054+0000 7f55f1600640 0 client.15969 ms_handle_reset on v2:172.21.15.165:6838/501312984 2024-09-11T00:11:10.060 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:10.054+0000 7f560d647740 1 -- 172.21.15.12:0/2837257693 >> 172.21.15.12:0/2837257693 conn(0x5570aa726ac0 msgr2=0x5570aa725bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:11:10.060 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:10.054+0000 7f560d647740 1 -- 172.21.15.12:0/2837257693 shutdown_connections 2024-09-11T00:11:10.060 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:10.054+0000 7f560d647740 1 -- 172.21.15.12:0/2837257693 wait complete. 2024-09-11T00:11:10.060 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:10.054+0000 7f560d647740 20 client.15969 trim_cache size 0 max 16384 2024-09-11T00:11:10.062 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:11:09 smithi012 ceph-mon[30333]: pgmap v521: 129 pgs: 129 active+clean; 30 MiB data, 683 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 KiB/s wr, 21 op/s 2024-09-11T00:11:10.072 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:10.070+0000 7f560d647740 1 Processor -- start 2024-09-11T00:11:10.072 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:10.070+0000 7f560d647740 1 -- start start 2024-09-11T00:11:10.072 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:10.070+0000 7f560d647740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5570aa72b970 0x5570aa721be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:11:10.072 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:10.070+0000 7f560d647740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5570aa72e2a0 0x5570aa7263b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:11:10.072 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:10.070+0000 7f560d647740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5570aa730ce0 0x5570aa7310c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:11:10.072 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:10.070+0000 7f560d647740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f56041ad790 con 0x5570aa72b970 2024-09-11T00:11:10.072 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:10.070+0000 7f560d647740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f56041af2d0 con 0x5570aa730ce0 2024-09-11T00:11:10.072 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:10.070+0000 7f560d647740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f56041ade60 con 0x5570aa72e2a0 2024-09-11T00:11:10.074 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:10.070+0000 7f5603e00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5570aa72b970 0x5570aa721be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:11:10.074 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:10.070+0000 7f5603e00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5570aa72b970 0x5570aa721be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:40598/0 (socket says 172.21.15.12:40598) 2024-09-11T00:11:10.074 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:10.070+0000 7f5603e00640 1 -- 172.21.15.12:0/2851284522 learned_addr learned my addr 172.21.15.12:0/2851284522 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:11:10.074 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:10.070+0000 7f55faa00640 1 --2- 172.21.15.12:0/2851284522 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5570aa72e2a0 0x5570aa7263b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:11:10.075 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:10.071+0000 7f5603e00640 1 --2- 172.21.15.12:0/2851284522 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5570aa730ce0 0x5570aa7310c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:11:10.075 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:10.071+0000 7f5603e00640 1 -- 172.21.15.12:0/2851284522 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5570aa72e2a0 msgr2=0x5570aa7263b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:11:10.075 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:10.071+0000 7f5603e00640 1 --2- 172.21.15.12:0/2851284522 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5570aa72e2a0 0x5570aa7263b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:11:10.075 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:10.071+0000 7f5603e00640 1 -- 172.21.15.12:0/2851284522 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5570aa730ce0 msgr2=0x5570aa7310c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:11:10.075 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:10.071+0000 7f5603e00640 1 --2- 172.21.15.12:0/2851284522 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5570aa730ce0 0x5570aa7310c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:11:10.075 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:10.071+0000 7f5603e00640 1 -- 172.21.15.12:0/2851284522 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5570aa742e60 con 0x5570aa72b970 2024-09-11T00:11:10.075 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:10.071+0000 7f5603e00640 1 --2- 172.21.15.12:0/2851284522 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5570aa72b970 0x5570aa721be0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f55fc00a010 tx=0x7f55fc005970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4faaafc8df4c036b server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:11:10.075 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:10.071+0000 7f55f3e00640 1 -- 172.21.15.12:0/2851284522 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55fc0041c0 con 0x5570aa72b970 2024-09-11T00:11:10.075 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:10.071+0000 7f55f3e00640 1 -- 172.21.15.12:0/2851284522 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f55fc004360 con 0x5570aa72b970 2024-09-11T00:11:10.076 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:10.071+0000 7f55f3e00640 1 -- 172.21.15.12:0/2851284522 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55fc00f750 con 0x5570aa72b970 2024-09-11T00:11:10.076 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:10.072+0000 7f560d647740 1 -- 172.21.15.12:0/2851284522 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5570aa72b970 msgr2=0x5570aa721be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:11:10.076 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:10.072+0000 7f560d647740 1 --2- 172.21.15.12:0/2851284522 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5570aa72b970 0x5570aa721be0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f55fc00a010 tx=0x7f55fc005970 comp rx=0 tx=0).stop 2024-09-11T00:11:10.076 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:10.072+0000 7f560d647740 1 -- 172.21.15.12:0/2851284522 shutdown_connections 2024-09-11T00:11:10.077 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:10.072+0000 7f560d647740 1 --2- 172.21.15.12:0/2851284522 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5570aa730ce0 0x5570aa7310c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:11:10.077 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:10.072+0000 7f560d647740 1 --2- 172.21.15.12:0/2851284522 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5570aa72e2a0 0x5570aa7263b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:11:10.077 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:10.072+0000 7f560d647740 1 --2- 172.21.15.12:0/2851284522 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5570aa72b970 0x5570aa721be0 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:11:10.077 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:10.072+0000 7f560d647740 1 -- 172.21.15.12:0/2851284522 >> 172.21.15.12:0/2851284522 conn(0x5570aa726ac0 msgr2=0x5570aa72add0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:11:10.077 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:10.072+0000 7f560d647740 1 -- 172.21.15.12:0/2851284522 shutdown_connections 2024-09-11T00:11:10.077 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:10.072+0000 7f560d647740 1 -- 172.21.15.12:0/2851284522 wait complete. 2024-09-11T00:11:10.077 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:10.072+0000 7f560d647740 1 -- start start 2024-09-11T00:11:10.077 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:10.073+0000 7f560d647740 1 Processor -- start 2024-09-11T00:11:10.081 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:10.073+0000 7f560d647740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5570aa72b970 0x5570aa73c900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:11:10.081 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:10.073+0000 7f560d647740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5570aa72e2a0 0x5570aa73b600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:11:10.081 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:10.073+0000 7f560d647740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5570aa730ce0 0x5570aa73bb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:11:10.081 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:10.073+0000 7f560d647740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f56041aec00 con 0x5570aa72e2a0 2024-09-11T00:11:10.081 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:10.073+0000 7f560d647740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f56041ae530 con 0x5570aa72b970 2024-09-11T00:11:10.081 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:10.073+0000 7f560d647740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f56041ab770 con 0x5570aa730ce0 2024-09-11T00:11:10.081 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:10.073+0000 7f560d647740 10 client.0 _ll_register_callbacks cb 0x7f55f40093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:11:10.082 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:10.073+0000 7f55faa00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5570aa72e2a0 0x5570aa73b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:11:10.082 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:10.073+0000 7f5603e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5570aa72b970 0x5570aa73c900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:11:10.082 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:10.073+0000 7f55faa00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5570aa72e2a0 0x5570aa73b600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:40604/0 (socket says 172.21.15.12:40604) 2024-09-11T00:11:10.082 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:10.073+0000 7f55faa00640 1 -- 172.21.15.12:0/3015734184 learned_addr learned my addr 172.21.15.12:0/3015734184 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:11:10.082 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:10.073+0000 7f5603e00640 1 --2- 172.21.15.12:0/3015734184 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5570aa730ce0 0x5570aa73bb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:11:10.082 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:10.074+0000 7f55faa00640 1 -- 172.21.15.12:0/3015734184 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5570aa730ce0 msgr2=0x5570aa73bb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:11:10.082 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:10.074+0000 7f55faa00640 1 --2- 172.21.15.12:0/3015734184 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5570aa730ce0 0x5570aa73bb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:11:10.082 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:10.074+0000 7f55faa00640 1 -- 172.21.15.12:0/3015734184 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5570aa72b970 msgr2=0x5570aa73c900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:11:10.082 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:10.074+0000 7f55faa00640 1 --2- 172.21.15.12:0/3015734184 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5570aa72b970 0x5570aa73c900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:11:10.082 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:10.074+0000 7f55faa00640 1 -- 172.21.15.12:0/3015734184 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5570aa73c130 con 0x5570aa72e2a0 2024-09-11T00:11:10.082 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:10.074+0000 7f5603e00640 1 --2- 172.21.15.12:0/3015734184 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5570aa72b970 0x5570aa73c900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T00:11:10.082 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:10.074+0000 7f55faa00640 1 --2- 172.21.15.12:0/3015734184 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5570aa72e2a0 0x5570aa73b600 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f5604103320 tx=0x7f55ec031850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=231cf10375b31cd4 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:11:10.083 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:10.074+0000 7f55f1600640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-09-11T00:11:10.083 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:10.074+0000 7f55f1600640 1 -- 172.21.15.12:0/3015734184 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55ec052060 con 0x5570aa72e2a0 2024-09-11T00:11:10.083 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:10.074+0000 7f55f1600640 1 -- 172.21.15.12:0/3015734184 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9208+0+0 (secure 0 0 0) 0x7f55ec004450 con 0x5570aa72e2a0 2024-09-11T00:11:10.083 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:10.074+0000 7f560d647740 1 -- 172.21.15.12:0/3015734184 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f55fc004360 con 0x5570aa72e2a0 2024-09-11T00:11:10.083 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:10.075+0000 7f55f1600640 1 -- 172.21.15.12:0/3015734184 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f55ec059050 con 0x5570aa72e2a0 2024-09-11T00:11:10.083 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:10.075+0000 7f560d647740 10 client.15975 fetch_fsmap learned FSMap version 37 2024-09-11T00:11:10.083 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:10.075+0000 7f560d647740 1 -- 172.21.15.12:0/3015734184 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x5570aa7ad5e0 con 0x5570aa72e2a0 2024-09-11T00:11:10.083 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:10.075+0000 7f55f1600640 1 -- 172.21.15.12:0/3015734184 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f55fc004360 con 0x5570aa72e2a0 2024-09-11T00:11:10.083 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:10.075+0000 7f55f1600640 1 -- 172.21.15.12:0/3015734184 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 37) ==== 8370+0+0 (secure 0 0 0) 0x7f55ec04d640 con 0x5570aa72e2a0 2024-09-11T00:11:10.083 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:10.075+0000 7f560d647740 10 client.15975 fetch_fsmap finished waiting for FSMap version 37 2024-09-11T00:11:10.083 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:10.075+0000 7f560d647740 10 client.15975 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T00:11:10.083 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:10.075+0000 7f560d647740 20 client.15975 populate_metadata read hostname 'smithi012' 2024-09-11T00:11:10.083 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:10.075+0000 7f560d647740 1 --2- 172.21.15.12:0/3015734184 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x5570aa7adef0 0x5570aa7b0310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:11:10.084 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:10.075+0000 7f560d647740 4 client.15975 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T00:11:10.084 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:10.075+0000 7f560d647740 1 -- 172.21.15.12:0/3015734184 --> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5570aa69c0b0 con 0x5570aa7adef0 2024-09-11T00:11:10.084 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:10.076+0000 7f5603e00640 1 --2- 172.21.15.12:0/3015734184 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x5570aa7adef0 0x5570aa7b0310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:11:10.084 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:10.076+0000 7f5603e00640 1 --2- 172.21.15.12:0/3015734184 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x5570aa7adef0 0x5570aa7b0310 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bc1abc820e33d1b7 server_cookie=dddee17c2fd35790 in_seq=0 out_seq=0 2024-09-11T00:11:10.084 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:10.076+0000 7f55f1600640 10 client.15975 ms_handle_connect on v2:172.21.15.165:6838/501312984 2024-09-11T00:11:10.084 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:11:09 smithi178 ceph-mon[35919]: pgmap v521: 129 pgs: 129 active+clean; 30 MiB data, 683 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 KiB/s wr, 21 op/s 2024-09-11T00:11:10.101 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:10.099+0000 7f55f1600640 1 -- 172.21.15.12:0/3015734184 <== mds.0 v2:172.21.15.165:6838/501312984 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5570aa69c0b0 con 0x5570aa7adef0 2024-09-11T00:11:10.101 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:10.099+0000 7f55f1600640 10 client.15975 handle_command_reply: tid=0 2024-09-11T00:11:10.101 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:10.099+0000 7f560d647740 1 client.15975 shutdown 2024-09-11T00:11:10.101 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:10.099+0000 7f560d647740 1 -- 172.21.15.12:0/3015734184 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5570aa72e2a0 msgr2=0x5570aa73b600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:11:10.101 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:10.099+0000 7f560d647740 1 --2- 172.21.15.12:0/3015734184 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5570aa72e2a0 0x5570aa73b600 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f5604103320 tx=0x7f55ec031850 comp rx=0 tx=0).stop 2024-09-11T00:11:10.101 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:10.099+0000 7f560d647740 1 -- 172.21.15.12:0/3015734184 shutdown_connections 2024-09-11T00:11:10.101 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:10.099+0000 7f560d647740 1 --2- 172.21.15.12:0/3015734184 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x5570aa7adef0 0x5570aa7b0310 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:11:10.101 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:10.099+0000 7f560d647740 1 --2- 172.21.15.12:0/3015734184 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5570aa730ce0 0x5570aa73bb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:11:10.101 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:10.099+0000 7f560d647740 1 --2- 172.21.15.12:0/3015734184 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5570aa72e2a0 0x5570aa73b600 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:11:10.101 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:10.099+0000 7f560d647740 1 --2- 172.21.15.12:0/3015734184 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5570aa72b970 0x5570aa73c900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:11:10.102 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:10.099+0000 7f55f1600640 0 client.15975 ms_handle_reset on v2:172.21.15.165:6838/501312984 2024-09-11T00:11:10.102 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:10.099+0000 7f560d647740 1 -- 172.21.15.12:0/3015734184 >> 172.21.15.12:0/3015734184 conn(0x5570aa726ac0 msgr2=0x5570aa72a9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:11:10.102 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:10.099+0000 7f560d647740 1 -- 172.21.15.12:0/3015734184 shutdown_connections 2024-09-11T00:11:10.102 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:10.099+0000 7f560d647740 1 -- 172.21.15.12:0/3015734184 wait complete. 2024-09-11T00:11:10.102 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:10.099+0000 7f560d647740 20 client.15975 trim_cache size 0 max 16384 2024-09-11T00:11:10.102 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:{ 2024-09-11T00:11:10.102 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "message": "", 2024-09-11T00:11:10.102 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "return_code": 0 2024-09-11T00:11:10.102 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:} 2024-09-11T00:11:10.103 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:10.099+0000 7f5609000640 1 -- 172.21.15.12:0/3024881641 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f55e80b5360 msgr2=0x7f55e80b7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:11:10.103 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:10.099+0000 7f5609000640 1 --2- 172.21.15.12:0/3024881641 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f55e80b5360 0x7f55e80b7820 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f55f40ac8e0 tx=0x7f55f400cf90 comp rx=0 tx=0).stop 2024-09-11T00:11:10.103 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:10.099+0000 7f5609000640 1 -- 172.21.15.12:0/3024881641 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f560406b360 msgr2=0x7f560419adc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:11:10.103 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:10.099+0000 7f5609000640 1 --2- 172.21.15.12:0/3024881641 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f560406b360 0x7f560419adc0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f55f400eb90 tx=0x7f55f400c6a0 comp rx=0 tx=0).stop 2024-09-11T00:11:10.103 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:10.100+0000 7f5609000640 1 -- 172.21.15.12:0/3024881641 shutdown_connections 2024-09-11T00:11:10.103 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:10.100+0000 7f5609000640 1 --2- 172.21.15.12:0/3024881641 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f55e80b5360 0x7f55e80b7820 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:11:10.104 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:10.100+0000 7f5609000640 1 --2- 172.21.15.12:0/3024881641 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f560410a6e0 0x7f560419c040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:11:10.104 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:10.100+0000 7f5609000640 1 --2- 172.21.15.12:0/3024881641 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5604107ca0 0x7f560419b410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:11:10.104 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:10.100+0000 7f5609000640 1 --2- 172.21.15.12:0/3024881641 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f560406b360 0x7f560419adc0 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:11:10.104 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:10.100+0000 7f5609000640 1 -- 172.21.15.12:0/3024881641 >> 172.21.15.12:0/3024881641 conn(0x7f56040fcf30 msgr2=0x7f56041088a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:11:10.104 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:10.100+0000 7f5609000640 1 -- 172.21.15.12:0/3024881641 shutdown_connections 2024-09-11T00:11:10.104 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:10.100+0000 7f5609000640 1 -- 172.21.15.12:0/3024881641 wait complete. 2024-09-11T00:11:10.627 INFO:tasks.workunit.client.0.smithi012.stdout: 1 121357 2.00 MB/sec execute 67 sec latency 16606.014 ms 2024-09-11T00:11:11.627 INFO:tasks.workunit.client.0.smithi012.stdout: 1 121357 1.97 MB/sec execute 68 sec latency 17606.141 ms 2024-09-11T00:11:12.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:11:11 smithi165 ceph-mon[35356]: pgmap v522: 129 pgs: 129 active+clean; 31 MiB data, 685 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 177 KiB/s wr, 27 op/s 2024-09-11T00:11:12.063 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:11:11 smithi012 ceph-mon[30333]: pgmap v522: 129 pgs: 129 active+clean; 31 MiB data, 685 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 177 KiB/s wr, 27 op/s 2024-09-11T00:11:12.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:11:11 smithi178 ceph-mon[35919]: pgmap v522: 129 pgs: 129 active+clean; 31 MiB data, 685 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 177 KiB/s wr, 27 op/s 2024-09-11T00:11:12.627 INFO:tasks.workunit.client.0.smithi012.stdout: 1 121357 1.94 MB/sec execute 69 sec latency 18606.235 ms 2024-09-11T00:11:13.628 INFO:tasks.workunit.client.0.smithi012.stdout: 1 121357 1.91 MB/sec execute 70 sec latency 19606.291 ms 2024-09-11T00:11:14.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:11:13 smithi165 ceph-mon[35356]: pgmap v523: 129 pgs: 129 active+clean; 31 MiB data, 686 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 258 KiB/s wr, 30 op/s 2024-09-11T00:11:14.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:11:13 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T00:11:14.063 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:11:13 smithi012 ceph-mon[30333]: pgmap v523: 129 pgs: 129 active+clean; 31 MiB data, 686 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 258 KiB/s wr, 30 op/s 2024-09-11T00:11:14.063 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:11:13 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T00:11:14.080 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:11:13 smithi178 ceph-mon[35919]: pgmap v523: 129 pgs: 129 active+clean; 31 MiB data, 686 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 258 KiB/s wr, 30 op/s 2024-09-11T00:11:14.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:11:13 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T00:11:14.627 INFO:tasks.workunit.client.0.smithi012.stdout: 1 121357 1.88 MB/sec execute 71 sec latency 20606.399 ms 2024-09-11T00:11:15.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:11:14 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:11:15.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:11:14 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T00:11:15.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:11:14 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:11:15.064 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:11:14 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:11:15.064 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:11:14 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T00:11:15.065 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:11:14 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:11:15.080 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:11:14 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:11:15.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:11:14 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T00:11:15.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:11:14 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:11:15.628 INFO:tasks.workunit.client.0.smithi012.stdout: 1 121357 1.86 MB/sec execute 72 sec latency 21606.512 ms 2024-09-11T00:11:16.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:11:15 smithi165 ceph-mon[35356]: pgmap v524: 129 pgs: 129 active+clean; 31 MiB data, 685 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 251 KiB/s wr, 24 op/s 2024-09-11T00:11:16.062 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:11:15 smithi012 ceph-mon[30333]: pgmap v524: 129 pgs: 129 active+clean; 31 MiB data, 685 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 251 KiB/s wr, 24 op/s 2024-09-11T00:11:16.080 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:11:15 smithi178 ceph-mon[35919]: pgmap v524: 129 pgs: 129 active+clean; 31 MiB data, 685 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 251 KiB/s wr, 24 op/s 2024-09-11T00:11:16.628 INFO:tasks.workunit.client.0.smithi012.stdout: 1 121357 1.83 MB/sec execute 73 sec latency 22606.670 ms 2024-09-11T00:11:17.628 INFO:tasks.workunit.client.0.smithi012.stdout: 1 121357 1.81 MB/sec execute 74 sec latency 23606.813 ms 2024-09-11T00:11:18.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:11:17 smithi165 ceph-mon[35356]: pgmap v525: 129 pgs: 129 active+clean; 30 MiB data, 687 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 256 KiB/s wr, 33 op/s 2024-09-11T00:11:18.063 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:11:17 smithi012 ceph-mon[30333]: pgmap v525: 129 pgs: 129 active+clean; 30 MiB data, 687 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 256 KiB/s wr, 33 op/s 2024-09-11T00:11:18.080 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:11:17 smithi178 ceph-mon[35919]: pgmap v525: 129 pgs: 129 active+clean; 30 MiB data, 687 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 256 KiB/s wr, 33 op/s 2024-09-11T00:11:18.628 INFO:tasks.workunit.client.0.smithi012.stdout: 1 121357 1.78 MB/sec execute 75 sec latency 24606.953 ms 2024-09-11T00:11:19.628 INFO:tasks.workunit.client.0.smithi012.stdout: 1 121357 1.76 MB/sec execute 76 sec latency 25607.100 ms 2024-09-11T00:11:20.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:11:19 smithi165 ceph-mon[35356]: pgmap v526: 129 pgs: 129 active+clean; 30 MiB data, 687 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 254 KiB/s wr, 27 op/s 2024-09-11T00:11:20.063 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:11:19 smithi012 ceph-mon[30333]: pgmap v526: 129 pgs: 129 active+clean; 30 MiB data, 687 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 254 KiB/s wr, 27 op/s 2024-09-11T00:11:20.080 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:11:19 smithi178 ceph-mon[35919]: pgmap v526: 129 pgs: 129 active+clean; 30 MiB data, 687 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 254 KiB/s wr, 27 op/s 2024-09-11T00:11:20.628 INFO:tasks.workunit.client.0.smithi012.stdout: 1 121357 1.74 MB/sec execute 77 sec latency 26607.195 ms 2024-09-11T00:11:20.880 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid ff3ad74a-6fcf-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T00:11:21.273 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:11:21.629 INFO:tasks.workunit.client.0.smithi012.stdout: 1 121357 1.71 MB/sec execute 78 sec latency 27607.300 ms 2024-09-11T00:11:21.756 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:11:21 smithi012 ceph-mon[30333]: pgmap v527: 129 pgs: 129 active+clean; 30 MiB data, 687 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 253 KiB/s wr, 29 op/s 2024-09-11T00:11:21.905 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:21.869+0000 7fadf5c00640 1 -- 172.21.15.12:0/922356307 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fadf0102da0 msgr2=0x7fadf0103180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:11:21.905 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:21.869+0000 7fadf5c00640 1 --2- 172.21.15.12:0/922356307 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fadf0102da0 0x7fadf0103180 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fade0009900 tx=0x7fade002a3f0 comp rx=0 tx=0).stop 2024-09-11T00:11:21.906 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:21.870+0000 7fadf5c00640 1 -- 172.21.15.12:0/922356307 shutdown_connections 2024-09-11T00:11:21.906 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:21.870+0000 7fadf5c00640 1 --2- 172.21.15.12:0/922356307 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fadf0109130 0x7fadf010f930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:11:21.906 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:21.870+0000 7fadf5c00640 1 --2- 172.21.15.12:0/922356307 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fadf01087b0 0x7fadf0108bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:11:21.906 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:21.870+0000 7fadf5c00640 1 --2- 172.21.15.12:0/922356307 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fadf0102da0 0x7fadf0103180 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:11:21.906 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:21.870+0000 7fadf5c00640 1 -- 172.21.15.12:0/922356307 >> 172.21.15.12:0/922356307 conn(0x7fadf00fe810 msgr2=0x7fadf0100c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:11:21.906 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:21.870+0000 7fadf5c00640 1 -- 172.21.15.12:0/922356307 shutdown_connections 2024-09-11T00:11:21.906 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:21.870+0000 7fadf5c00640 1 -- 172.21.15.12:0/922356307 wait complete. 2024-09-11T00:11:21.906 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:21.871+0000 7fadf5c00640 1 Processor -- start 2024-09-11T00:11:21.906 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:21.871+0000 7fadf5c00640 1 -- start start 2024-09-11T00:11:21.906 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:21.871+0000 7fadf5c00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fadf0102da0 0x7fadf00754a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:11:21.906 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:21.871+0000 7fadf5c00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fadf01087b0 0x7fadf00759e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:11:21.906 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:21.871+0000 7fadf5c00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fadf0109130 0x7fadf0075f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:11:21.907 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:21.871+0000 7fadf5c00640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fadf0112df0 con 0x7fadf0102da0 2024-09-11T00:11:21.907 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:21.871+0000 7fadf5c00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fadf0112c70 con 0x7fadf01087b0 2024-09-11T00:11:21.907 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:21.871+0000 7fadf5c00640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fadf0112f70 con 0x7fadf0109130 2024-09-11T00:11:21.907 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:21.871+0000 7fadeea00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fadf0102da0 0x7fadf00754a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:11:21.907 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:21.871+0000 7fadeea00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fadf0102da0 0x7fadf00754a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:43888/0 (socket says 172.21.15.12:43888) 2024-09-11T00:11:21.907 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:21.871+0000 7fadeea00640 1 -- 172.21.15.12:0/2042002414 learned_addr learned my addr 172.21.15.12:0/2042002414 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:11:21.907 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:21.871+0000 7fade7e00640 1 --2- 172.21.15.12:0/2042002414 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fadf01087b0 0x7fadf00759e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:11:21.907 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:21.872+0000 7fadef400640 1 --2- 172.21.15.12:0/2042002414 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fadf0109130 0x7fadf0075f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:11:21.907 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:21.872+0000 7fadeea00640 1 -- 172.21.15.12:0/2042002414 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fadf0109130 msgr2=0x7fadf0075f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:11:21.907 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:21.872+0000 7fadeea00640 1 --2- 172.21.15.12:0/2042002414 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fadf0109130 0x7fadf0075f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:11:21.907 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:21.872+0000 7fadeea00640 1 -- 172.21.15.12:0/2042002414 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fadf01087b0 msgr2=0x7fadf00759e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:11:21.908 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:21.872+0000 7fadeea00640 1 --2- 172.21.15.12:0/2042002414 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fadf01087b0 0x7fadf00759e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:11:21.908 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:21.872+0000 7fadeea00640 1 -- 172.21.15.12:0/2042002414 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fadf01a76a0 con 0x7fadf0102da0 2024-09-11T00:11:21.908 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:21.872+0000 7fadef400640 1 --2- 172.21.15.12:0/2042002414 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fadf0109130 0x7fadf0075f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T00:11:21.908 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:21.872+0000 7fadeea00640 1 --2- 172.21.15.12:0/2042002414 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fadf0102da0 0x7fadf00754a0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fade0005940 tx=0x7fade0004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:11:21.908 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:21.872+0000 7fade7400640 1 -- 172.21.15.12:0/2042002414 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fade003f030 con 0x7fadf0102da0 2024-09-11T00:11:21.908 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:21.872+0000 7fade7400640 1 -- 172.21.15.12:0/2042002414 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fade0003aa0 con 0x7fadf0102da0 2024-09-11T00:11:21.908 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:21.872+0000 7fadf5c00640 1 -- 172.21.15.12:0/2042002414 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fadf01a7930 con 0x7fadf0102da0 2024-09-11T00:11:21.908 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:21.873+0000 7fade7400640 1 -- 172.21.15.12:0/2042002414 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fade0035070 con 0x7fadf0102da0 2024-09-11T00:11:21.908 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:21.873+0000 7fadf5c00640 1 -- 172.21.15.12:0/2042002414 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fadf01a7e60 con 0x7fadf0102da0 2024-09-11T00:11:21.908 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:21.874+0000 7fadf5c00640 1 -- 172.21.15.12:0/2042002414 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fadf0103180 con 0x7fadf0102da0 2024-09-11T00:11:21.908 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:21.876+0000 7fade7400640 1 -- 172.21.15.12:0/2042002414 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7fade0003c50 con 0x7fadf0102da0 2024-09-11T00:11:21.908 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:21.877+0000 7fade7400640 1 --2- 172.21.15.12:0/2042002414 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fadd40b5360 0x7fadd40b7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:11:21.909 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:21.877+0000 7fade7400640 1 -- 172.21.15.12:0/2042002414 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9208+0+0 (secure 0 0 0) 0x7fade0106b20 con 0x7fadf0102da0 2024-09-11T00:11:21.909 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:21.878+0000 7fade7e00640 1 --2- 172.21.15.12:0/2042002414 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fadd40b5360 0x7fadd40b7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:11:21.909 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:21.878+0000 7fade7e00640 1 --2- 172.21.15.12:0/2042002414 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fadd40b5360 0x7fadd40b7820 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fadf0076f90 tx=0x7fadd8005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:11:21.909 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:21.879+0000 7fade7400640 1 -- 172.21.15.12:0/2042002414 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7fade00cd5b0 con 0x7fadf0102da0 2024-09-11T00:11:22.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:11:21 smithi165 ceph-mon[35356]: pgmap v527: 129 pgs: 129 active+clean; 30 MiB data, 687 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 253 KiB/s wr, 29 op/s 2024-09-11T00:11:22.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:11:21 smithi178 ceph-mon[35919]: pgmap v527: 129 pgs: 129 active+clean; 30 MiB data, 687 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 253 KiB/s wr, 29 op/s 2024-09-11T00:11:22.116 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:22.115+0000 7fadf5c00640 1 -- 172.21.15.12:0/2042002414 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fadf00767a0 con 0x7fadf0102da0 2024-09-11T00:11:22.117 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:22.116+0000 7fade7400640 1 -- 172.21.15.12:0/2042002414 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 37 v37) ==== 94+0+11965 (secure 0 0 0) 0x7fade00cceb0 con 0x7fadf0102da0 2024-09-11T00:11:22.117 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-11T00:11:22.117 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":37,"btime":"2024-09-11T00:11:07:606386+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14577,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6835/447171591","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":447171591},{"type":"v1","addr":"172.21.15.12:6835","nonce":447171591}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14589,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/901486040","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":901486040},{"type":"v1","addr":"172.21.15.12:6837","nonce":901486040}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":37,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T00:04:04.739881+0000","modified":"2024-09-11T00:11:06.943614+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24674,"mds_1":24643,"mds_2":24638,"mds_3":24637,"mds_4":24614},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14631":{"gid":14631,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.12:6841/3069919437","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6840","nonce":3069919437},{"type":"v1","addr":"172.21.15.12:6841","nonce":3069919437}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24562":{"gid":24562,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6833/1141636112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":1141636112},{"type":"v1","addr":"172.21.15.178:6833","nonce":1141636112}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24581":{"gid":24581,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6833/3111980970","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6832","nonce":3111980970},{"type":"v1","addr":"172.21.15.165:6833","nonce":3111980970}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24607":{"gid":24607,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6835/4084868287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":4084868287},{"type":"v1","addr":"172.21.15.178:6835","nonce":4084868287}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24614":{"gid":24614,"name":"e","rank":4,"incarnation":26,"state":"up:active","state_seq":21,"addr":"172.21.15.165:6835/2425395274","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":2425395274},{"type":"v1","addr":"172.21.15.165:6835","nonce":2425395274}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24637":{"gid":24637,"name":"i","rank":3,"incarnation":24,"state":"up:active","state_seq":18,"addr":"172.21.15.178:6839/70628097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6838","nonce":70628097},{"type":"v1","addr":"172.21.15.178:6839","nonce":70628097}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24638":{"gid":24638,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":18,"addr":"172.21.15.165:6837/6215088","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6836","nonce":6215088},{"type":"v1","addr":"172.21.15.165:6837","nonce":6215088}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24643":{"gid":24643,"name":"g","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.12:6839/3124093009","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6838","nonce":3124093009},{"type":"v1","addr":"172.21.15.12:6839","nonce":3124093009}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24674":{"gid":24674,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.165:6839/501312984","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6838","nonce":501312984},{"type":"v1","addr":"172.21.15.165:6839","nonce":501312984}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25672":{"gid":25672,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6837/308601741","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":308601741},{"type":"v1","addr":"172.21.15.178:6837","nonce":308601741}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24674,"qdb_cluster":[24614,24637,24638,24643,24674]},"id":1}]} 2024-09-11T00:11:22.118 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 37 2024-09-11T00:11:22.120 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:22.118+0000 7fadf5c00640 1 -- 172.21.15.12:0/2042002414 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fadd40b5360 msgr2=0x7fadd40b7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:11:22.120 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:22.118+0000 7fadf5c00640 1 --2- 172.21.15.12:0/2042002414 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fadd40b5360 0x7fadd40b7820 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fadf0076f90 tx=0x7fadd8005f90 comp rx=0 tx=0).stop 2024-09-11T00:11:22.120 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:22.118+0000 7fadf5c00640 1 -- 172.21.15.12:0/2042002414 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fadf0102da0 msgr2=0x7fadf00754a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:11:22.120 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:22.118+0000 7fadf5c00640 1 --2- 172.21.15.12:0/2042002414 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fadf0102da0 0x7fadf00754a0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fade0005940 tx=0x7fade0004270 comp rx=0 tx=0).stop 2024-09-11T00:11:22.120 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:22.119+0000 7fadf5c00640 1 -- 172.21.15.12:0/2042002414 shutdown_connections 2024-09-11T00:11:22.121 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:22.119+0000 7fadf5c00640 1 --2- 172.21.15.12:0/2042002414 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fadd40b5360 0x7fadd40b7820 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:11:22.121 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:22.119+0000 7fadf5c00640 1 --2- 172.21.15.12:0/2042002414 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fadf0109130 0x7fadf0075f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:11:22.121 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:22.119+0000 7fadf5c00640 1 --2- 172.21.15.12:0/2042002414 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fadf01087b0 0x7fadf00759e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:11:22.121 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:22.119+0000 7fadf5c00640 1 --2- 172.21.15.12:0/2042002414 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fadf0102da0 0x7fadf00754a0 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:11:22.121 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:22.119+0000 7fadf5c00640 1 -- 172.21.15.12:0/2042002414 >> 172.21.15.12:0/2042002414 conn(0x7fadf00fe810 msgr2=0x7fadf01000a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:11:22.121 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:22.119+0000 7fadf5c00640 1 -- 172.21.15.12:0/2042002414 shutdown_connections 2024-09-11T00:11:22.121 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:22.119+0000 7fadf5c00640 1 -- 172.21.15.12:0/2042002414 wait complete. 2024-09-11T00:11:22.632 INFO:tasks.workunit.client.0.smithi012.stdout: 1 121357 1.69 MB/sec execute 79 sec latency 28607.425 ms 2024-09-11T00:11:22.813 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:11:22 smithi012 ceph-mon[30333]: from='client.? 172.21.15.12:0/2042002414' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T00:11:22.947 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24674 quiesce db --set-id 8c2b0c3c --release --await' 2024-09-11T00:11:22.948 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid ff3ad74a-6fcf-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24674 quiesce db --set-id 8c2b0c3c --release --await 2024-09-11T00:11:23.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:11:22 smithi165 ceph-mon[35356]: from='client.? 172.21.15.12:0/2042002414' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T00:11:23.080 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:11:22 smithi178 ceph-mon[35919]: from='client.? 172.21.15.12:0/2042002414' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T00:11:23.222 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:23.220+0000 7ffb58a00640 1 -- 172.21.15.12:0/431530488 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ffb5406bde0 msgr2=0x7ffb541079c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:11:23.222 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:23.220+0000 7ffb58a00640 1 --2- 172.21.15.12:0/431530488 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ffb5406bde0 0x7ffb541079c0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7ffb44009900 tx=0x7ffb4402a3f0 comp rx=0 tx=0).stop 2024-09-11T00:11:23.222 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:23.220+0000 7ffb58a00640 1 -- 172.21.15.12:0/431530488 shutdown_connections 2024-09-11T00:11:23.222 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:23.220+0000 7ffb58a00640 1 --2- 172.21.15.12:0/431530488 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ffb5410a8c0 0x7ffb5410cd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:11:23.222 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:23.220+0000 7ffb58a00640 1 --2- 172.21.15.12:0/431530488 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ffb54107f00 0x7ffb5410a2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:11:23.222 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:23.220+0000 7ffb58a00640 1 --2- 172.21.15.12:0/431530488 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ffb5406bde0 0x7ffb541079c0 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:11:23.222 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:23.220+0000 7ffb58a00640 1 -- 172.21.15.12:0/431530488 >> 172.21.15.12:0/431530488 conn(0x7ffb540fd150 msgr2=0x7ffb540ff570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:11:23.222 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:23.220+0000 7ffb58a00640 1 -- 172.21.15.12:0/431530488 shutdown_connections 2024-09-11T00:11:23.223 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:23.220+0000 7ffb58a00640 1 -- 172.21.15.12:0/431530488 wait complete. 2024-09-11T00:11:23.223 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:23.222+0000 7ffb58a00640 1 Processor -- start 2024-09-11T00:11:23.223 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:23.222+0000 7ffb58a00640 1 -- start start 2024-09-11T00:11:23.224 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:23.223+0000 7ffb58a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ffb5406bde0 0x7ffb5419afc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:11:23.224 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:23.223+0000 7ffb58a00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ffb54107f00 0x7ffb5419b610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:11:23.224 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:23.223+0000 7ffb58a00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ffb5410a8c0 0x7ffb5419c240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:11:23.224 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:23.223+0000 7ffb58a00640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7ffb54110840 con 0x7ffb54107f00 2024-09-11T00:11:23.224 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:23.223+0000 7ffb58a00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7ffb541106c0 con 0x7ffb5406bde0 2024-09-11T00:11:23.224 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:23.223+0000 7ffb58a00640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7ffb541109c0 con 0x7ffb5410a8c0 2024-09-11T00:11:23.225 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:23.223+0000 7ffb4be00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ffb54107f00 0x7ffb5419b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:11:23.225 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:23.223+0000 7ffb51e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ffb5406bde0 0x7ffb5419afc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:11:23.225 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:23.223+0000 7ffb4be00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ffb54107f00 0x7ffb5419b610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:43912/0 (socket says 172.21.15.12:43912) 2024-09-11T00:11:23.225 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:23.223+0000 7ffb51e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ffb5406bde0 0x7ffb5419afc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.12:41562/0 (socket says 172.21.15.12:41562) 2024-09-11T00:11:23.225 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:23.223+0000 7ffb4be00640 1 -- 172.21.15.12:0/1619807881 learned_addr learned my addr 172.21.15.12:0/1619807881 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:11:23.225 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:23.223+0000 7ffb52800640 1 --2- 172.21.15.12:0/1619807881 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ffb5410a8c0 0x7ffb5419c240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:11:23.226 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:23.224+0000 7ffb4be00640 1 -- 172.21.15.12:0/1619807881 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ffb5410a8c0 msgr2=0x7ffb5419c240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:11:23.226 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:23.224+0000 7ffb4be00640 1 --2- 172.21.15.12:0/1619807881 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ffb5410a8c0 0x7ffb5419c240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:11:23.226 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:23.224+0000 7ffb4be00640 1 -- 172.21.15.12:0/1619807881 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ffb5406bde0 msgr2=0x7ffb5419afc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:11:23.226 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:23.224+0000 7ffb4be00640 1 --2- 172.21.15.12:0/1619807881 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ffb5406bde0 0x7ffb5419afc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:11:23.226 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:23.224+0000 7ffb4be00640 1 -- 172.21.15.12:0/1619807881 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb5419cb00 con 0x7ffb54107f00 2024-09-11T00:11:23.226 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:23.224+0000 7ffb51e00640 1 --2- 172.21.15.12:0/1619807881 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ffb5406bde0 0x7ffb5419afc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:11:23.226 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:23.224+0000 7ffb4be00640 1 --2- 172.21.15.12:0/1619807881 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ffb54107f00 0x7ffb5419b610 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7ffb3c00e9f0 tx=0x7ffb3c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:11:23.227 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:23.224+0000 7ffb4aa00640 1 -- 172.21.15.12:0/1619807881 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb3c00ccc0 con 0x7ffb54107f00 2024-09-11T00:11:23.227 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:23.224+0000 7ffb4aa00640 1 -- 172.21.15.12:0/1619807881 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffb3c0054d0 con 0x7ffb54107f00 2024-09-11T00:11:23.227 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:23.224+0000 7ffb4aa00640 1 -- 172.21.15.12:0/1619807881 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb3c010430 con 0x7ffb54107f00 2024-09-11T00:11:23.227 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:23.224+0000 7ffb58a00640 1 -- 172.21.15.12:0/1619807881 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffb5419cdf0 con 0x7ffb54107f00 2024-09-11T00:11:23.227 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:23.224+0000 7ffb58a00640 1 -- 172.21.15.12:0/1619807881 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffb541a34c0 con 0x7ffb54107f00 2024-09-11T00:11:23.228 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:23.227+0000 7ffb5b887740 1 Processor -- start 2024-09-11T00:11:23.229 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:23.227+0000 7ffb5b887740 1 -- start start 2024-09-11T00:11:23.229 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:23.228+0000 7ffb5b887740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bd4577a790 0x55bd4577cb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:11:23.229 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:23.228+0000 7ffb5b887740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bd4577d0c0 0x55bd4577f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:11:23.229 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:23.228+0000 7ffb5b887740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bd4577fb00 0x55bd45781fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:11:23.229 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:23.228+0000 7ffb5b887740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55bd456ed6b0 con 0x55bd4577d0c0 2024-09-11T00:11:23.229 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:23.228+0000 7ffb5b887740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55bd456dcae0 con 0x55bd4577a790 2024-09-11T00:11:23.229 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:23.228+0000 7ffb5b887740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55bd456e08f0 con 0x55bd4577fb00 2024-09-11T00:11:23.231 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:23.228+0000 7ffb51e00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bd4577d0c0 0x55bd4577f4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:11:23.231 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:23.228+0000 7ffb51e00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bd4577d0c0 0x55bd4577f4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:43916/0 (socket says 172.21.15.12:43916) 2024-09-11T00:11:23.231 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:23.228+0000 7ffb51e00640 1 -- 172.21.15.12:0/725576167 learned_addr learned my addr 172.21.15.12:0/725576167 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:11:23.231 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:23.228+0000 7ffb52800640 1 --2- 172.21.15.12:0/725576167 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bd4577a790 0x55bd4577cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:11:23.232 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:23.228+0000 7ffb4aa00640 1 -- 172.21.15.12:0/1619807881 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7ffb3c0046c0 con 0x7ffb54107f00 2024-09-11T00:11:23.232 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:23.228+0000 7ffb51e00640 1 -- 172.21.15.12:0/725576167 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bd4577fb00 msgr2=0x55bd45781fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:11:23.232 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:23.228+0000 7ffb51e00640 1 --2- 172.21.15.12:0/725576167 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bd4577fb00 0x55bd45781fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:11:23.232 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:23.228+0000 7ffb51e00640 1 -- 172.21.15.12:0/725576167 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bd4577a790 msgr2=0x55bd4577cb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:11:23.232 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:23.228+0000 7ffb51e00640 1 --2- 172.21.15.12:0/725576167 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bd4577a790 0x55bd4577cb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:11:23.232 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:23.229+0000 7ffb51e00640 1 -- 172.21.15.12:0/725576167 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bd456a9780 con 0x55bd4577d0c0 2024-09-11T00:11:23.232 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:23.229+0000 7ffb52800640 1 --2- 172.21.15.12:0/725576167 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bd4577a790 0x55bd4577cb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:11:23.232 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:23.229+0000 7ffb51e00640 1 --2- 172.21.15.12:0/725576167 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bd4577d0c0 0x55bd4577f4b0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7ffb44009900 tx=0x7ffb44005df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53b088d0870b9a36 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:11:23.232 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:23.229+0000 7ffb43000640 1 -- 172.21.15.12:0/725576167 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb44040070 con 0x55bd4577d0c0 2024-09-11T00:11:23.232 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:23.229+0000 7ffb43000640 1 -- 172.21.15.12:0/725576167 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffb440318a0 con 0x55bd4577d0c0 2024-09-11T00:11:23.232 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:23.229+0000 7ffb43000640 1 -- 172.21.15.12:0/725576167 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb44035040 con 0x55bd4577d0c0 2024-09-11T00:11:23.232 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:23.230+0000 7ffb4aa00640 1 --2- 172.21.15.12:0/1619807881 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7ffb380b5360 0x7ffb380b7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:11:23.233 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:23.230+0000 7ffb5b887740 1 -- 172.21.15.12:0/725576167 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bd4577d0c0 msgr2=0x55bd4577f4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:11:23.233 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:23.230+0000 7ffb5b887740 1 --2- 172.21.15.12:0/725576167 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bd4577d0c0 0x55bd4577f4b0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7ffb44009900 tx=0x7ffb44005df0 comp rx=0 tx=0).stop 2024-09-11T00:11:23.233 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:23.230+0000 7ffb4aa00640 1 -- 172.21.15.12:0/1619807881 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9208+0+0 (secure 0 0 0) 0x7ffb3c0e62d0 con 0x7ffb54107f00 2024-09-11T00:11:23.233 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:23.230+0000 7ffb5b887740 1 -- 172.21.15.12:0/725576167 shutdown_connections 2024-09-11T00:11:23.233 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:23.230+0000 7ffb5b887740 1 --2- 172.21.15.12:0/725576167 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bd4577fb00 0x55bd45781fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:11:23.233 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:23.230+0000 7ffb5b887740 1 --2- 172.21.15.12:0/725576167 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bd4577d0c0 0x55bd4577f4b0 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:11:23.233 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:23.230+0000 7ffb5b887740 1 --2- 172.21.15.12:0/725576167 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bd4577a790 0x55bd4577cb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:11:23.233 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:23.230+0000 7ffb5b887740 1 -- 172.21.15.12:0/725576167 >> 172.21.15.12:0/725576167 conn(0x55bd457758e0 msgr2=0x55bd45776670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:11:23.233 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:23.230+0000 7ffb52800640 1 --2- 172.21.15.12:0/1619807881 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7ffb380b5360 0x7ffb380b7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:11:23.233 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:23.230+0000 7ffb5b887740 1 -- 172.21.15.12:0/725576167 shutdown_connections 2024-09-11T00:11:23.233 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:23.230+0000 7ffb5b887740 1 -- 172.21.15.12:0/725576167 wait complete. 2024-09-11T00:11:23.237 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:23.231+0000 7ffb5b887740 1 -- start start 2024-09-11T00:11:23.237 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:23.231+0000 7ffb52800640 1 --2- 172.21.15.12:0/1619807881 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7ffb380b5360 0x7ffb380b7820 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7ffb5419bd40 tx=0x7ffb4c009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:11:23.237 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:23.231+0000 7ffb5b887740 1 Processor -- start 2024-09-11T00:11:23.237 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:23.232+0000 7ffb5b887740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bd4577a790 0x55bd45783ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:11:23.237 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:23.232+0000 7ffb5b887740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bd4577d0c0 0x55bd45784540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:11:23.238 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:23.232+0000 7ffb5b887740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bd4577fb00 0x55bd45785110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:11:23.238 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:23.232+0000 7ffb5b887740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55bd45614ab0 con 0x55bd4577a790 2024-09-11T00:11:23.238 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:23.232+0000 7ffb5b887740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55bd45683570 con 0x55bd4577fb00 2024-09-11T00:11:23.238 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:23.232+0000 7ffb5b887740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55bd45782a80 con 0x55bd4577d0c0 2024-09-11T00:11:23.238 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:23.232+0000 7ffb5b887740 10 client.0 _ll_register_callbacks cb 0x55bd45692100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:11:23.238 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:23.232+0000 7ffb51e00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bd4577a790 0x55bd45783ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:11:23.238 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:23.232+0000 7ffb4be00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bd4577d0c0 0x55bd45784540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:11:23.238 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:23.232+0000 7ffb52800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bd4577fb00 0x55bd45785110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:11:23.238 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:23.232+0000 7ffb51e00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bd4577a790 0x55bd45783ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:43926/0 (socket says 172.21.15.12:43926) 2024-09-11T00:11:23.238 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:23.232+0000 7ffb4be00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bd4577d0c0 0x55bd45784540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.12:47986/0 (socket says 172.21.15.12:47986) 2024-09-11T00:11:23.238 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:23.232+0000 7ffb51e00640 1 -- 172.21.15.12:0/1428174613 learned_addr learned my addr 172.21.15.12:0/1428174613 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:11:23.238 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:23.232+0000 7ffb4be00640 1 -- 172.21.15.12:0/1428174613 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bd4577fb00 msgr2=0x55bd45785110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:11:23.238 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:23.232+0000 7ffb4be00640 1 --2- 172.21.15.12:0/1428174613 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bd4577fb00 0x55bd45785110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:11:23.238 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:23.232+0000 7ffb4be00640 1 -- 172.21.15.12:0/1428174613 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bd4577a790 msgr2=0x55bd45783ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:11:23.239 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:23.232+0000 7ffb4be00640 1 --2- 172.21.15.12:0/1428174613 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bd4577a790 0x55bd45783ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:11:23.239 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:23.232+0000 7ffb4be00640 1 -- 172.21.15.12:0/1428174613 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bd4543e790 con 0x55bd4577d0c0 2024-09-11T00:11:23.239 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:23.232+0000 7ffb52800640 1 --2- 172.21.15.12:0/1428174613 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bd4577fb00 0x55bd45785110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:11:23.239 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:23.233+0000 7ffb4be00640 1 --2- 172.21.15.12:0/1428174613 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bd4577d0c0 0x55bd45784540 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7ffb3c0a98d0 tx=0x7ffb3c0abdd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=421b2b4583005cff server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:11:23.239 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:23.233+0000 7ffb37400640 10 client.0 ms_handle_connect on v2:172.21.15.178:3300/0 2024-09-11T00:11:23.239 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:23.233+0000 7ffb37400640 1 -- 172.21.15.12:0/1428174613 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb3c0e3e60 con 0x55bd4577d0c0 2024-09-11T00:11:23.239 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:23.233+0000 7ffb5b887740 1 -- 172.21.15.12:0/1428174613 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffb440318a0 con 0x55bd4577d0c0 2024-09-11T00:11:23.239 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:23.233+0000 7ffb37400640 1 -- 172.21.15.12:0/1428174613 <== mon.1 v2:172.21.15.178:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9208+0+0 (secure 0 0 0) 0x7ffb3c0d3510 con 0x55bd4577d0c0 2024-09-11T00:11:23.239 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:23.233+0000 7ffb37400640 1 -- 172.21.15.12:0/1428174613 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7ffb3c0acd60 con 0x55bd4577d0c0 2024-09-11T00:11:23.239 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:23.233+0000 7ffb5b887740 10 client.26002 fetch_fsmap learned FSMap version 37 2024-09-11T00:11:23.239 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:23.233+0000 7ffb5b887740 1 -- 172.21.15.12:0/1428174613 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55bd454b7620 con 0x55bd4577d0c0 2024-09-11T00:11:23.239 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:23.233+0000 7ffb37400640 1 -- 172.21.15.12:0/1428174613 <== mon.1 v2:172.21.15.178:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffb440318a0 con 0x55bd4577d0c0 2024-09-11T00:11:23.239 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:23.234+0000 7ffb37400640 1 -- 172.21.15.12:0/1428174613 <== mon.1 v2:172.21.15.178:3300/0 5 ==== fsmap(e 37) ==== 8370+0+0 (secure 0 0 0) 0x7ffb3c0ab8c0 con 0x55bd4577d0c0 2024-09-11T00:11:23.239 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:23.234+0000 7ffb5b887740 10 client.26002 fetch_fsmap finished waiting for FSMap version 37 2024-09-11T00:11:23.240 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:23.234+0000 7ffb5b887740 10 client.26002 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T00:11:23.240 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:23.234+0000 7ffb5b887740 20 client.26002 populate_metadata read hostname 'smithi012' 2024-09-11T00:11:23.240 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:23.234+0000 7ffb5b887740 1 --2- 172.21.15.12:0/1428174613 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55bd4578f710 0x55bd45791b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:11:23.240 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:23.234+0000 7ffb5b887740 4 client.26002 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T00:11:23.240 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:23.234+0000 7ffb5b887740 1 -- 172.21.15.12:0/1428174613 --> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bd456ef600 con 0x55bd4578f710 2024-09-11T00:11:23.240 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:23.234+0000 7ffb51e00640 1 --2- 172.21.15.12:0/1428174613 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55bd4578f710 0x55bd45791b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:11:23.240 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:23.235+0000 7ffb51e00640 1 --2- 172.21.15.12:0/1428174613 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55bd4578f710 0x55bd45791b00 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d214039f77a71295 server_cookie=7ca832261a49fabe in_seq=0 out_seq=0 2024-09-11T00:11:23.240 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:23.235+0000 7ffb37400640 10 client.26002 ms_handle_connect on v2:172.21.15.165:6838/501312984 2024-09-11T00:11:23.240 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:23.236+0000 7ffb37400640 1 -- 172.21.15.12:0/1428174613 <== mds.0 v2:172.21.15.165:6838/501312984 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55bd456ef600 con 0x55bd4578f710 2024-09-11T00:11:23.240 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:23.236+0000 7ffb37400640 10 client.26002 handle_command_reply: tid=0 2024-09-11T00:11:23.240 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:23.236+0000 7ffb5b887740 1 client.26002 shutdown 2024-09-11T00:11:23.240 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:23.236+0000 7ffb5b887740 1 -- 172.21.15.12:0/1428174613 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bd4577d0c0 msgr2=0x55bd45784540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:11:23.241 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:23.236+0000 7ffb5b887740 1 --2- 172.21.15.12:0/1428174613 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bd4577d0c0 0x55bd45784540 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7ffb3c0a98d0 tx=0x7ffb3c0abdd0 comp rx=0 tx=0).stop 2024-09-11T00:11:23.241 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:23.236+0000 7ffb5b887740 1 -- 172.21.15.12:0/1428174613 shutdown_connections 2024-09-11T00:11:23.241 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:23.236+0000 7ffb5b887740 1 --2- 172.21.15.12:0/1428174613 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55bd4578f710 0x55bd45791b00 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:11:23.241 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:23.236+0000 7ffb5b887740 1 --2- 172.21.15.12:0/1428174613 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bd4577fb00 0x55bd45785110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:11:23.241 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:23.236+0000 7ffb5b887740 1 --2- 172.21.15.12:0/1428174613 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bd4577d0c0 0x55bd45784540 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:11:23.241 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:23.236+0000 7ffb5b887740 1 --2- 172.21.15.12:0/1428174613 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bd4577a790 0x55bd45783ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:11:23.241 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:23.236+0000 7ffb37400640 0 client.26002 ms_handle_reset on v2:172.21.15.165:6838/501312984 2024-09-11T00:11:23.241 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:23.236+0000 7ffb5b887740 1 -- 172.21.15.12:0/1428174613 >> 172.21.15.12:0/1428174613 conn(0x55bd457758e0 msgr2=0x55bd45774a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:11:23.241 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:23.236+0000 7ffb5b887740 1 -- 172.21.15.12:0/1428174613 shutdown_connections 2024-09-11T00:11:23.241 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:23.236+0000 7ffb5b887740 1 -- 172.21.15.12:0/1428174613 wait complete. 2024-09-11T00:11:23.241 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:23.236+0000 7ffb5b887740 20 client.26002 trim_cache size 0 max 16384 2024-09-11T00:11:23.254 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:23.252+0000 7ffb5b887740 1 Processor -- start 2024-09-11T00:11:23.254 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:23.252+0000 7ffb5b887740 1 -- start start 2024-09-11T00:11:23.254 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:23.253+0000 7ffb5b887740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bd4577a790 0x55bd45779c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:11:23.254 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:23.253+0000 7ffb5b887740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bd4577d0c0 0x55bd4578c190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:11:23.254 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:23.253+0000 7ffb5b887740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bd4577fb00 0x55bd4577a1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:11:23.254 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:23.253+0000 7ffb5b887740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7ffb541ad960 con 0x55bd4577a790 2024-09-11T00:11:23.254 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:23.253+0000 7ffb5b887740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7ffb541af4a0 con 0x55bd4577fb00 2024-09-11T00:11:23.254 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:23.253+0000 7ffb5b887740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7ffb541ae030 con 0x55bd4577d0c0 2024-09-11T00:11:23.255 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:23.253+0000 7ffb52800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bd4577fb00 0x55bd4577a1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:11:23.255 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:23.253+0000 7ffb51e00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bd4577a790 0x55bd45779c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:11:23.255 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:23.253+0000 7ffb52800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bd4577fb00 0x55bd4577a1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.12:41582/0 (socket says 172.21.15.12:41582) 2024-09-11T00:11:23.256 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:23.253+0000 7ffb52800640 1 -- 172.21.15.12:0/3827957309 learned_addr learned my addr 172.21.15.12:0/3827957309 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:11:23.256 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:23.253+0000 7ffb4be00640 1 --2- 172.21.15.12:0/3827957309 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bd4577d0c0 0x55bd4578c190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:11:23.256 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:23.253+0000 7ffb52800640 1 -- 172.21.15.12:0/3827957309 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bd4577d0c0 msgr2=0x55bd4578c190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:11:23.256 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:23.253+0000 7ffb52800640 1 --2- 172.21.15.12:0/3827957309 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bd4577d0c0 0x55bd4578c190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:11:23.256 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:23.253+0000 7ffb52800640 1 -- 172.21.15.12:0/3827957309 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bd4577a790 msgr2=0x55bd45779c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:11:23.256 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:23.253+0000 7ffb52800640 1 --2- 172.21.15.12:0/3827957309 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bd4577a790 0x55bd45779c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:11:23.256 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:23.253+0000 7ffb52800640 1 -- 172.21.15.12:0/3827957309 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bd45791e70 con 0x55bd4577fb00 2024-09-11T00:11:23.256 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:23.253+0000 7ffb51e00640 1 --2- 172.21.15.12:0/3827957309 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bd4577a790 0x55bd45779c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:11:23.256 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:23.254+0000 7ffb52800640 1 --2- 172.21.15.12:0/3827957309 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bd4577fb00 0x55bd4577a1c0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7ffb4c0043c0 tx=0x7ffb4c01a000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a32e98620374e6e7 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:11:23.256 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:23.254+0000 7ffb42600640 1 -- 172.21.15.12:0/3827957309 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb4c003e30 con 0x55bd4577fb00 2024-09-11T00:11:23.256 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:23.254+0000 7ffb42600640 1 -- 172.21.15.12:0/3827957309 <== mon.2 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffb4c01e430 con 0x55bd4577fb00 2024-09-11T00:11:23.256 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:23.254+0000 7ffb42600640 1 -- 172.21.15.12:0/3827957309 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb4c01c640 con 0x55bd4577fb00 2024-09-11T00:11:23.259 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:23.255+0000 7ffb5b887740 1 -- 172.21.15.12:0/3827957309 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bd4577fb00 msgr2=0x55bd4577a1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:11:23.259 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:23.255+0000 7ffb5b887740 1 --2- 172.21.15.12:0/3827957309 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bd4577fb00 0x55bd4577a1c0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7ffb4c0043c0 tx=0x7ffb4c01a000 comp rx=0 tx=0).stop 2024-09-11T00:11:23.260 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:23.255+0000 7ffb5b887740 1 -- 172.21.15.12:0/3827957309 shutdown_connections 2024-09-11T00:11:23.260 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:23.255+0000 7ffb5b887740 1 --2- 172.21.15.12:0/3827957309 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bd4577fb00 0x55bd4577a1c0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:11:23.260 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:23.255+0000 7ffb5b887740 1 --2- 172.21.15.12:0/3827957309 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bd4577d0c0 0x55bd4578c190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:11:23.260 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:23.255+0000 7ffb5b887740 1 --2- 172.21.15.12:0/3827957309 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bd4577a790 0x55bd45779c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:11:23.260 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:23.255+0000 7ffb5b887740 1 -- 172.21.15.12:0/3827957309 >> 172.21.15.12:0/3827957309 conn(0x55bd457758e0 msgr2=0x55bd45783060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:11:23.260 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:23.255+0000 7ffb5b887740 1 -- 172.21.15.12:0/3827957309 shutdown_connections 2024-09-11T00:11:23.260 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:23.255+0000 7ffb5b887740 1 -- 172.21.15.12:0/3827957309 wait complete. 2024-09-11T00:11:23.260 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:23.255+0000 7ffb5b887740 1 -- start start 2024-09-11T00:11:23.260 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:23.256+0000 7ffb5b887740 1 Processor -- start 2024-09-11T00:11:23.260 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:23.256+0000 7ffb5b887740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bd4577a790 0x55bd4578c150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:11:23.260 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:23.256+0000 7ffb5b887740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bd4577d0c0 0x55bd4578af90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:11:23.260 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:23.256+0000 7ffb5b887740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bd4577fb00 0x55bd4578b660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:11:23.261 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:23.256+0000 7ffb5b887740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7ffb541aedd0 con 0x55bd4577d0c0 2024-09-11T00:11:23.261 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:23.256+0000 7ffb5b887740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7ffb541ae700 con 0x55bd4577a790 2024-09-11T00:11:23.261 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:23.256+0000 7ffb5b887740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7ffb541ab940 con 0x55bd4577fb00 2024-09-11T00:11:23.261 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:23.256+0000 7ffb5b887740 10 client.0 _ll_register_callbacks cb 0x7ffb3c00cef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:11:23.261 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:23.256+0000 7ffb4be00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bd4577d0c0 0x55bd4578af90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:11:23.261 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:23.256+0000 7ffb51e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bd4577a790 0x55bd4578c150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:11:23.261 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:23.257+0000 7ffb4be00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bd4577d0c0 0x55bd4578af90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:43944/0 (socket says 172.21.15.12:43944) 2024-09-11T00:11:23.261 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:23.257+0000 7ffb4be00640 1 -- 172.21.15.12:0/1082586931 learned_addr learned my addr 172.21.15.12:0/1082586931 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:11:23.261 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:23.257+0000 7ffb52800640 1 --2- 172.21.15.12:0/1082586931 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bd4577fb00 0x55bd4578b660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:11:23.261 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:23.257+0000 7ffb51e00640 1 -- 172.21.15.12:0/1082586931 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bd4577fb00 msgr2=0x55bd4578b660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:11:23.261 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:23.257+0000 7ffb51e00640 1 --2- 172.21.15.12:0/1082586931 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bd4577fb00 0x55bd4578b660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:11:23.261 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:23.257+0000 7ffb51e00640 1 -- 172.21.15.12:0/1082586931 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bd4577d0c0 msgr2=0x55bd4578af90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:11:23.261 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:23.257+0000 7ffb51e00640 1 --2- 172.21.15.12:0/1082586931 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bd4577d0c0 0x55bd4578af90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:11:23.262 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:23.257+0000 7ffb51e00640 1 -- 172.21.15.12:0/1082586931 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bd457899a0 con 0x55bd4577a790 2024-09-11T00:11:23.262 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:23.257+0000 7ffb4be00640 1 --2- 172.21.15.12:0/1082586931 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bd4577d0c0 0x55bd4578af90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:11:23.262 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:23.257+0000 7ffb51e00640 1 --2- 172.21.15.12:0/1082586931 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bd4577a790 0x55bd4578c150 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x55bd45792380 tx=0x7ffb44005620 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7b3f94efa999a40b server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:11:23.262 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:23.257+0000 7ffb37400640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-09-11T00:11:23.262 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:23.258+0000 7ffb37400640 1 -- 172.21.15.12:0/1082586931 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb4405b060 con 0x55bd4577a790 2024-09-11T00:11:23.262 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:23.258+0000 7ffb5b887740 1 -- 172.21.15.12:0/1082586931 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffb4c01e430 con 0x55bd4577a790 2024-09-11T00:11:23.262 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:23.258+0000 7ffb37400640 1 -- 172.21.15.12:0/1082586931 <== mon.2 v2:172.21.15.165:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9208+0+0 (secure 0 0 0) 0x7ffb44004300 con 0x55bd4577a790 2024-09-11T00:11:23.262 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:23.258+0000 7ffb37400640 1 -- 172.21.15.12:0/1082586931 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7ffb44062050 con 0x55bd4577a790 2024-09-11T00:11:23.262 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:23.258+0000 7ffb5b887740 10 client.26021 fetch_fsmap learned FSMap version 37 2024-09-11T00:11:23.262 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:23.258+0000 7ffb5b887740 1 -- 172.21.15.12:0/1082586931 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55bd457fca10 con 0x55bd4577a790 2024-09-11T00:11:23.262 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:23.258+0000 7ffb37400640 1 -- 172.21.15.12:0/1082586931 <== mon.2 v2:172.21.15.165:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffb4c01e430 con 0x55bd4577a790 2024-09-11T00:11:23.262 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:23.258+0000 7ffb37400640 1 -- 172.21.15.12:0/1082586931 <== mon.2 v2:172.21.15.165:3300/0 5 ==== fsmap(e 37) ==== 8370+0+0 (secure 0 0 0) 0x7ffb44050510 con 0x55bd4577a790 2024-09-11T00:11:23.262 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:23.259+0000 7ffb5b887740 10 client.26021 fetch_fsmap finished waiting for FSMap version 37 2024-09-11T00:11:23.263 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:23.259+0000 7ffb5b887740 10 client.26021 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T00:11:23.263 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:23.259+0000 7ffb5b887740 20 client.26021 populate_metadata read hostname 'smithi012' 2024-09-11T00:11:23.263 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:23.259+0000 7ffb5b887740 1 --2- 172.21.15.12:0/1082586931 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55bd457fcf80 0x55bd457ff370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:11:23.263 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:23.259+0000 7ffb5b887740 4 client.26021 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T00:11:23.263 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:23.259+0000 7ffb5b887740 1 -- 172.21.15.12:0/1082586931 --> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55bd456eb0b0 con 0x55bd457fcf80 2024-09-11T00:11:23.263 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:23.259+0000 7ffb4be00640 1 --2- 172.21.15.12:0/1082586931 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55bd457fcf80 0x55bd457ff370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:11:23.263 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:23.259+0000 7ffb4be00640 1 --2- 172.21.15.12:0/1082586931 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55bd457fcf80 0x55bd457ff370 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=22d3387ea6f7b716 server_cookie=5c6172d091f0195 in_seq=0 out_seq=0 2024-09-11T00:11:23.263 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:23.259+0000 7ffb37400640 10 client.26021 ms_handle_connect on v2:172.21.15.165:6838/501312984 2024-09-11T00:11:23.277 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:23.275+0000 7ffb37400640 1 -- 172.21.15.12:0/1082586931 <== mds.0 v2:172.21.15.165:6838/501312984 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55bd456eb0b0 con 0x55bd457fcf80 2024-09-11T00:11:23.277 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:23.275+0000 7ffb37400640 10 client.26021 handle_command_reply: tid=0 2024-09-11T00:11:23.277 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:23.275+0000 7ffb5b887740 1 client.26021 shutdown 2024-09-11T00:11:23.277 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:23.275+0000 7ffb5b887740 1 -- 172.21.15.12:0/1082586931 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bd4577a790 msgr2=0x55bd4578c150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:11:23.277 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:23.275+0000 7ffb5b887740 1 --2- 172.21.15.12:0/1082586931 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bd4577a790 0x55bd4578c150 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x55bd45792380 tx=0x7ffb44005620 comp rx=0 tx=0).stop 2024-09-11T00:11:23.277 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:23.275+0000 7ffb5b887740 1 -- 172.21.15.12:0/1082586931 shutdown_connections 2024-09-11T00:11:23.277 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:23.275+0000 7ffb5b887740 1 --2- 172.21.15.12:0/1082586931 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55bd457fcf80 0x55bd457ff370 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:11:23.277 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:23.275+0000 7ffb5b887740 1 --2- 172.21.15.12:0/1082586931 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bd4577fb00 0x55bd4578b660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:11:23.277 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:23.275+0000 7ffb5b887740 1 --2- 172.21.15.12:0/1082586931 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bd4577d0c0 0x55bd4578af90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:11:23.277 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:23.275+0000 7ffb5b887740 1 --2- 172.21.15.12:0/1082586931 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bd4577a790 0x55bd4578c150 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:11:23.277 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:23.275+0000 7ffb37400640 0 client.26021 ms_handle_reset on v2:172.21.15.165:6838/501312984 2024-09-11T00:11:23.277 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:23.275+0000 7ffb5b887740 1 -- 172.21.15.12:0/1082586931 >> 172.21.15.12:0/1082586931 conn(0x55bd457758e0 msgr2=0x55bd4577ff90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:11:23.278 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:23.275+0000 7ffb5b887740 1 -- 172.21.15.12:0/1082586931 shutdown_connections 2024-09-11T00:11:23.278 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:23.276+0000 7ffb5b887740 1 -- 172.21.15.12:0/1082586931 wait complete. 2024-09-11T00:11:23.278 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:23.276+0000 7ffb5b887740 20 client.26021 trim_cache size 0 max 16384 2024-09-11T00:11:23.278 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:{ 2024-09-11T00:11:23.278 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "message": "", 2024-09-11T00:11:23.278 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "return_code": 0 2024-09-11T00:11:23.278 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:} 2024-09-11T00:11:23.279 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:23.276+0000 7ffb58a00640 1 -- 172.21.15.12:0/1619807881 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7ffb380b5360 msgr2=0x7ffb380b7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:11:23.279 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:23.276+0000 7ffb58a00640 1 --2- 172.21.15.12:0/1619807881 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7ffb380b5360 0x7ffb380b7820 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7ffb5419bd40 tx=0x7ffb4c009500 comp rx=0 tx=0).stop 2024-09-11T00:11:23.279 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:23.276+0000 7ffb58a00640 1 -- 172.21.15.12:0/1619807881 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ffb54107f00 msgr2=0x7ffb5419b610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:11:23.279 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:23.276+0000 7ffb58a00640 1 --2- 172.21.15.12:0/1619807881 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ffb54107f00 0x7ffb5419b610 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7ffb3c00e9f0 tx=0x7ffb3c00eeb0 comp rx=0 tx=0).stop 2024-09-11T00:11:23.279 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:23.276+0000 7ffb58a00640 1 -- 172.21.15.12:0/1619807881 shutdown_connections 2024-09-11T00:11:23.279 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:23.276+0000 7ffb58a00640 1 --2- 172.21.15.12:0/1619807881 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7ffb380b5360 0x7ffb380b7820 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:11:23.279 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:23.276+0000 7ffb58a00640 1 --2- 172.21.15.12:0/1619807881 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ffb5410a8c0 0x7ffb5419c240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:11:23.279 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:23.277+0000 7ffb58a00640 1 --2- 172.21.15.12:0/1619807881 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ffb54107f00 0x7ffb5419b610 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:11:23.279 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:23.277+0000 7ffb58a00640 1 --2- 172.21.15.12:0/1619807881 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ffb5406bde0 0x7ffb5419afc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:11:23.280 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:23.277+0000 7ffb58a00640 1 -- 172.21.15.12:0/1619807881 >> 172.21.15.12:0/1619807881 conn(0x7ffb540fd150 msgr2=0x7ffb54108b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:11:23.280 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:23.277+0000 7ffb58a00640 1 -- 172.21.15.12:0/1619807881 shutdown_connections 2024-09-11T00:11:23.280 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:23.277+0000 7ffb58a00640 1 -- 172.21.15.12:0/1619807881 wait complete. 2024-09-11T00:11:23.342 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:11:23.629 INFO:tasks.workunit.client.0.smithi012.stdout: 1 121357 1.67 MB/sec execute 80 sec latency 29607.508 ms 2024-09-11T00:11:23.782 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:11:23 smithi012 ceph-mon[30333]: pgmap v528: 129 pgs: 129 active+clean; 30 MiB data, 687 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 89 KiB/s wr, 24 op/s 2024-09-11T00:11:23.908 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:23.907+0000 7f9449a00640 1 -- 172.21.15.12:0/1974547522 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f944410b350 msgr2=0x7f9444111b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:11:23.909 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:23.907+0000 7f9449a00640 1 --2- 172.21.15.12:0/1974547522 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f944410b350 0x7f9444111b50 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f943c009ca0 tx=0x7f943c02a8c0 comp rx=0 tx=0).stop 2024-09-11T00:11:23.909 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:23.907+0000 7f9449a00640 1 -- 172.21.15.12:0/1974547522 shutdown_connections 2024-09-11T00:11:23.909 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:23.907+0000 7f9449a00640 1 --2- 172.21.15.12:0/1974547522 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f944410b350 0x7f9444111b50 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:11:23.909 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:23.907+0000 7f9449a00640 1 --2- 172.21.15.12:0/1974547522 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f944410a9d0 0x7f944410ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:11:23.909 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:23.907+0000 7f9449a00640 1 --2- 172.21.15.12:0/1974547522 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9444104fc0 0x7f94441053a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:11:23.909 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:23.907+0000 7f9449a00640 1 -- 172.21.15.12:0/1974547522 >> 172.21.15.12:0/1974547522 conn(0x7f94441009f0 msgr2=0x7f9444102e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:11:23.909 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:23.907+0000 7f9449a00640 1 -- 172.21.15.12:0/1974547522 shutdown_connections 2024-09-11T00:11:23.909 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:23.907+0000 7f9449a00640 1 -- 172.21.15.12:0/1974547522 wait complete. 2024-09-11T00:11:23.910 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:23.908+0000 7f9449a00640 1 Processor -- start 2024-09-11T00:11:23.910 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:23.909+0000 7f9449a00640 1 -- start start 2024-09-11T00:11:23.910 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:23.909+0000 7f9449a00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9444104fc0 0x7f944419b450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:11:23.910 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:23.909+0000 7f9449a00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f944410a9d0 0x7f944419b990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:11:23.910 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:23.909+0000 7f9449a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f944410b350 0x7f94441a0d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:11:23.910 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:23.909+0000 7f9449a00640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f9444114f80 con 0x7f944410a9d0 2024-09-11T00:11:23.910 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:23.909+0000 7f9449a00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f9444114e00 con 0x7f944410b350 2024-09-11T00:11:23.911 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:23.909+0000 7f9449a00640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f9444115100 con 0x7f9444104fc0 2024-09-11T00:11:23.912 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:23.909+0000 7f9442000640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f944410a9d0 0x7f944419b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:11:23.912 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:23.909+0000 7f9442a00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9444104fc0 0x7f944419b450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:11:23.912 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:23.909+0000 7f9442a00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9444104fc0 0x7f944419b450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.12:48012/0 (socket says 172.21.15.12:48012) 2024-09-11T00:11:23.912 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:23.909+0000 7f9442a00640 1 -- 172.21.15.12:0/2954783838 learned_addr learned my addr 172.21.15.12:0/2954783838 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:11:23.912 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:23.910+0000 7f9443400640 1 --2- 172.21.15.12:0/2954783838 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f944410b350 0x7f94441a0d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:11:23.912 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:23.910+0000 7f9442a00640 1 -- 172.21.15.12:0/2954783838 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f944410b350 msgr2=0x7f94441a0d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:11:23.913 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:23.910+0000 7f9442a00640 1 --2- 172.21.15.12:0/2954783838 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f944410b350 0x7f94441a0d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:11:23.913 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:23.910+0000 7f9442a00640 1 -- 172.21.15.12:0/2954783838 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f944410a9d0 msgr2=0x7f944419b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:11:23.913 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:23.910+0000 7f9442a00640 1 --2- 172.21.15.12:0/2954783838 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f944410a9d0 0x7f944419b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:11:23.913 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:23.910+0000 7f9442a00640 1 -- 172.21.15.12:0/2954783838 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94441a1500 con 0x7f9444104fc0 2024-09-11T00:11:23.913 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:23.910+0000 7f9442a00640 1 --2- 172.21.15.12:0/2954783838 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9444104fc0 0x7f944419b450 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f943400e9f0 tx=0x7f943400edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:11:23.913 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:23.911+0000 7f943b400640 1 -- 172.21.15.12:0/2954783838 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f943400cd80 con 0x7f9444104fc0 2024-09-11T00:11:23.913 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:23.911+0000 7f9449a00640 1 -- 172.21.15.12:0/2954783838 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f94441a17f0 con 0x7f9444104fc0 2024-09-11T00:11:23.913 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:23.911+0000 7f943b400640 1 -- 172.21.15.12:0/2954783838 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f94340054d0 con 0x7f9444104fc0 2024-09-11T00:11:23.913 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:23.911+0000 7f9449a00640 1 -- 172.21.15.12:0/2954783838 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94441a7eb0 con 0x7f9444104fc0 2024-09-11T00:11:23.913 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:23.911+0000 7f943b400640 1 -- 172.21.15.12:0/2954783838 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9434010430 con 0x7f9444104fc0 2024-09-11T00:11:23.915 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:23.914+0000 7f944c5c2740 1 Processor -- start 2024-09-11T00:11:23.915 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:23.914+0000 7f944c5c2740 1 -- start start 2024-09-11T00:11:23.915 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:23.914+0000 7f944c5c2740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56245fb45ff0 0x56245fb463d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:11:23.915 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:23.914+0000 7f944c5c2740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56245fb46910 0x56245fb52360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:11:23.916 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:23.914+0000 7f944c5c2740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56245fb528a0 0x56245fb54c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:11:23.916 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:23.914+0000 7f944c5c2740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x56245fabd290 con 0x56245fb528a0 2024-09-11T00:11:23.916 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:23.914+0000 7f944c5c2740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x56245faac6c0 con 0x56245fb46910 2024-09-11T00:11:23.916 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:23.914+0000 7f944c5c2740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x56245fab04d0 con 0x56245fb45ff0 2024-09-11T00:11:23.916 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:23.914+0000 7f943b400640 1 -- 172.21.15.12:0/2954783838 <== mon.1 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7f94340046c0 con 0x7f9444104fc0 2024-09-11T00:11:23.917 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:23.915+0000 7f9443400640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56245fb45ff0 0x56245fb463d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:11:23.917 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:23.915+0000 7f9443400640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56245fb45ff0 0x56245fb463d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.12:48024/0 (socket says 172.21.15.12:48024) 2024-09-11T00:11:23.917 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:23.915+0000 7f9443400640 1 -- 172.21.15.12:0/752508358 learned_addr learned my addr 172.21.15.12:0/752508358 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:11:23.917 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:23.915+0000 7f9442000640 1 --2- 172.21.15.12:0/752508358 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56245fb528a0 0x56245fb54c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:11:23.917 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:23.915+0000 7f9442a00640 1 --2- 172.21.15.12:0/752508358 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56245fb46910 0x56245fb52360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:11:23.918 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:23.915+0000 7f9443400640 1 -- 172.21.15.12:0/752508358 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56245fb46910 msgr2=0x56245fb52360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:11:23.918 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:23.915+0000 7f9443400640 1 --2- 172.21.15.12:0/752508358 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56245fb46910 0x56245fb52360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:11:23.918 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:23.915+0000 7f9443400640 1 -- 172.21.15.12:0/752508358 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56245fb528a0 msgr2=0x56245fb54c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:11:23.918 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:23.915+0000 7f9443400640 1 --2- 172.21.15.12:0/752508358 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56245fb528a0 0x56245fb54c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:11:23.918 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:23.915+0000 7f9443400640 1 -- 172.21.15.12:0/752508358 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56245fa39460 con 0x56245fb45ff0 2024-09-11T00:11:23.918 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:23.915+0000 7f9442000640 1 --2- 172.21.15.12:0/752508358 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56245fb528a0 0x56245fb54c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:11:23.918 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:23.915+0000 7f9443400640 1 --2- 172.21.15.12:0/752508358 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56245fb45ff0 0x56245fb463d0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f943c005ef0 tx=0x7f943c005f20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=357f4a6c99e1c309 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:11:23.918 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:23.916+0000 7f942fe00640 1 -- 172.21.15.12:0/752508358 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f943c031cf0 con 0x56245fb45ff0 2024-09-11T00:11:23.918 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:23.916+0000 7f942fe00640 1 -- 172.21.15.12:0/752508358 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f943c036070 con 0x56245fb45ff0 2024-09-11T00:11:23.918 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:23.916+0000 7f942fe00640 1 -- 172.21.15.12:0/752508358 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f943c0042a0 con 0x56245fb45ff0 2024-09-11T00:11:23.918 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:23.916+0000 7f943b400640 1 --2- 172.21.15.12:0/2954783838 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f94280b5360 0x7f94280b7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:11:23.921 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:23.916+0000 7f943b400640 1 -- 172.21.15.12:0/2954783838 <== mon.1 v2:172.21.15.178:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9208+0+0 (secure 0 0 0) 0x7f94340e6300 con 0x7f9444104fc0 2024-09-11T00:11:23.921 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:23.916+0000 7f944c5c2740 1 -- 172.21.15.12:0/752508358 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56245fb45ff0 msgr2=0x56245fb463d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:11:23.921 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:23.916+0000 7f944c5c2740 1 --2- 172.21.15.12:0/752508358 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56245fb45ff0 0x56245fb463d0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f943c005ef0 tx=0x7f943c005f20 comp rx=0 tx=0).stop 2024-09-11T00:11:23.921 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:23.916+0000 7f9442a00640 1 --2- 172.21.15.12:0/2954783838 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f94280b5360 0x7f94280b7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:11:23.921 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:23.917+0000 7f944c5c2740 1 -- 172.21.15.12:0/752508358 shutdown_connections 2024-09-11T00:11:23.921 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:23.917+0000 7f944c5c2740 1 --2- 172.21.15.12:0/752508358 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56245fb528a0 0x56245fb54c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:11:23.921 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:23.917+0000 7f944c5c2740 1 --2- 172.21.15.12:0/752508358 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56245fb46910 0x56245fb52360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:11:23.922 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:23.917+0000 7f944c5c2740 1 --2- 172.21.15.12:0/752508358 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56245fb45ff0 0x56245fb463d0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:11:23.922 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:23.917+0000 7f944c5c2740 1 -- 172.21.15.12:0/752508358 >> 172.21.15.12:0/752508358 conn(0x56245fb43e20 msgr2=0x56245fb44fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:11:23.922 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:23.917+0000 7f944c5c2740 1 -- 172.21.15.12:0/752508358 shutdown_connections 2024-09-11T00:11:23.922 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:23.917+0000 7f944c5c2740 1 -- 172.21.15.12:0/752508358 wait complete. 2024-09-11T00:11:23.922 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:23.917+0000 7f9442a00640 1 --2- 172.21.15.12:0/2954783838 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f94280b5360 0x7f94280b7820 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f94340a8c60 tx=0x7f9434009720 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:11:23.922 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:23.917+0000 7f944c5c2740 1 -- start start 2024-09-11T00:11:23.922 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:23.918+0000 7f944c5c2740 1 Processor -- start 2024-09-11T00:11:23.922 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:23.918+0000 7f944c5c2740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56245fb45ff0 0x56245fb4f8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:11:23.923 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:23.918+0000 7f944c5c2740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56245fb46910 0x56245fb4fde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:11:23.923 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:23.918+0000 7f944c5c2740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56245fb528a0 0x56245fb50320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:11:23.923 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:23.918+0000 7f944c5c2740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x56245f90dbb0 con 0x56245fb46910 2024-09-11T00:11:23.923 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:23.918+0000 7f944c5c2740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x56245fa476c0 con 0x56245fb45ff0 2024-09-11T00:11:23.923 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:23.918+0000 7f944c5c2740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x56245fb55370 con 0x56245fb528a0 2024-09-11T00:11:23.923 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:23.918+0000 7f944c5c2740 10 client.0 _ll_register_callbacks cb 0x56245fa62490 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:11:23.923 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:23.919+0000 7f9442000640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56245fb46910 0x56245fb4fde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:11:23.923 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:23.919+0000 7f9443400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56245fb45ff0 0x56245fb4f8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:11:23.923 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:23.919+0000 7f9442000640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56245fb46910 0x56245fb4fde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:43982/0 (socket says 172.21.15.12:43982) 2024-09-11T00:11:23.923 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:23.919+0000 7f9442000640 1 -- 172.21.15.12:0/1171323678 learned_addr learned my addr 172.21.15.12:0/1171323678 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:11:23.923 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:23.919+0000 7f9443400640 1 --2- 172.21.15.12:0/1171323678 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56245fb528a0 0x56245fb50320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:11:23.923 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:23.919+0000 7f9442000640 1 -- 172.21.15.12:0/1171323678 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56245fb528a0 msgr2=0x56245fb50320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:11:23.924 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:23.919+0000 7f9442000640 1 --2- 172.21.15.12:0/1171323678 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56245fb528a0 0x56245fb50320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:11:23.924 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:23.919+0000 7f9442000640 1 -- 172.21.15.12:0/1171323678 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56245fb45ff0 msgr2=0x56245fb4f8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:11:23.924 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:23.919+0000 7f9442000640 1 --2- 172.21.15.12:0/1171323678 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56245fb45ff0 0x56245fb4f8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:11:23.924 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:23.919+0000 7f9442000640 1 -- 172.21.15.12:0/1171323678 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56245f80ab00 con 0x56245fb46910 2024-09-11T00:11:23.924 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:23.919+0000 7f9443400640 1 --2- 172.21.15.12:0/1171323678 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56245fb45ff0 0x56245fb4f8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:11:23.924 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:23.919+0000 7f9442000640 1 --2- 172.21.15.12:0/1171323678 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56245fb46910 0x56245fb4fde0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f9430012720 tx=0x7f9430012be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78d85961a44a8c2 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:11:23.924 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:23.919+0000 7f942cc00640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-09-11T00:11:23.924 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:23.920+0000 7f942cc00640 1 -- 172.21.15.12:0/1171323678 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9430004030 con 0x56245fb46910 2024-09-11T00:11:23.924 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:23.920+0000 7f942cc00640 1 -- 172.21.15.12:0/1171323678 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9208+0+0 (secure 0 0 0) 0x7f94300041d0 con 0x56245fb46910 2024-09-11T00:11:23.924 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:23.920+0000 7f944c5c2740 1 -- 172.21.15.12:0/1171323678 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f943c036070 con 0x56245fb46910 2024-09-11T00:11:23.925 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:23.920+0000 7f942cc00640 1 -- 172.21.15.12:0/1171323678 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f9430004410 con 0x56245fb46910 2024-09-11T00:11:23.925 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:23.920+0000 7f944c5c2740 10 client.16011 fetch_fsmap learned FSMap version 37 2024-09-11T00:11:23.925 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:23.920+0000 7f944c5c2740 1 -- 172.21.15.12:0/1171323678 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x56245f881960 con 0x56245fb46910 2024-09-11T00:11:23.927 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:23.920+0000 7f942cc00640 1 -- 172.21.15.12:0/1171323678 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f943c036070 con 0x56245fb46910 2024-09-11T00:11:23.927 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:23.921+0000 7f942cc00640 1 -- 172.21.15.12:0/1171323678 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 37) ==== 8370+0+0 (secure 0 0 0) 0x7f9430003ba0 con 0x56245fb46910 2024-09-11T00:11:23.927 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:23.921+0000 7f944c5c2740 10 client.16011 fetch_fsmap finished waiting for FSMap version 37 2024-09-11T00:11:23.927 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:23.921+0000 7f944c5c2740 10 client.16011 resolve_mds: validated gid 24674 aka daemon mds.b 2024-09-11T00:11:23.927 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:23.921+0000 7f944c5c2740 20 client.16011 populate_metadata read hostname 'smithi012' 2024-09-11T00:11:23.927 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:23.921+0000 7f944c5c2740 1 --2- 172.21.15.12:0/1171323678 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x56245fb622e0 0x56245fb646d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:11:23.927 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:23.921+0000 7f944c5c2740 4 client.16011 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T00:11:23.928 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:23.921+0000 7f944c5c2740 1 -- 172.21.15.12:0/1171323678 --> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56245fabf1e0 con 0x56245fb622e0 2024-09-11T00:11:23.928 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:23.921+0000 7f9443400640 1 --2- 172.21.15.12:0/1171323678 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x56245fb622e0 0x56245fb646d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:11:23.928 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:23.922+0000 7f9443400640 1 --2- 172.21.15.12:0/1171323678 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x56245fb622e0 0x56245fb646d0 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=72423e8eed2affc8 server_cookie=225f91f3d7507171 in_seq=0 out_seq=0 2024-09-11T00:11:23.928 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:23.922+0000 7f942cc00640 10 client.16011 ms_handle_connect on v2:172.21.15.165:6838/501312984 2024-09-11T00:11:23.928 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:23.923+0000 7f942cc00640 1 -- 172.21.15.12:0/1171323678 <== mds.0 v2:172.21.15.165:6838/501312984 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x56245fabf1e0 con 0x56245fb622e0 2024-09-11T00:11:23.928 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:23.923+0000 7f942cc00640 10 client.16011 handle_command_reply: tid=0 2024-09-11T00:11:23.928 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:23.923+0000 7f944c5c2740 1 client.16011 shutdown 2024-09-11T00:11:23.928 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:23.923+0000 7f944c5c2740 1 -- 172.21.15.12:0/1171323678 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56245fb46910 msgr2=0x56245fb4fde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:11:23.928 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:23.923+0000 7f944c5c2740 1 --2- 172.21.15.12:0/1171323678 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56245fb46910 0x56245fb4fde0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f9430012720 tx=0x7f9430012be0 comp rx=0 tx=0).stop 2024-09-11T00:11:23.928 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:23.923+0000 7f944c5c2740 1 -- 172.21.15.12:0/1171323678 shutdown_connections 2024-09-11T00:11:23.928 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:23.923+0000 7f944c5c2740 1 --2- 172.21.15.12:0/1171323678 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x56245fb622e0 0x56245fb646d0 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:11:23.928 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:23.923+0000 7f944c5c2740 1 --2- 172.21.15.12:0/1171323678 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56245fb528a0 0x56245fb50320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:11:23.928 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:23.923+0000 7f942cc00640 0 client.16011 ms_handle_reset on v2:172.21.15.165:6838/501312984 2024-09-11T00:11:23.929 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:23.923+0000 7f944c5c2740 1 --2- 172.21.15.12:0/1171323678 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56245fb46910 0x56245fb4fde0 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:11:23.929 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:23.923+0000 7f944c5c2740 1 --2- 172.21.15.12:0/1171323678 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56245fb45ff0 0x56245fb4f8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:11:23.929 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:23.923+0000 7f944c5c2740 1 -- 172.21.15.12:0/1171323678 >> 172.21.15.12:0/1171323678 conn(0x56245fb43e20 msgr2=0x56245fb499c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:11:23.929 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:23.923+0000 7f944c5c2740 1 -- 172.21.15.12:0/1171323678 shutdown_connections 2024-09-11T00:11:23.929 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:23.924+0000 7f944c5c2740 1 -- 172.21.15.12:0/1171323678 wait complete. 2024-09-11T00:11:23.929 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:23.924+0000 7f944c5c2740 20 client.16011 trim_cache size 0 max 16384 2024-09-11T00:11:23.943 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:23.942+0000 7f944c5c2740 1 Processor -- start 2024-09-11T00:11:23.943 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:23.942+0000 7f944c5c2740 1 -- start start 2024-09-11T00:11:23.946 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:23.942+0000 7f944c5c2740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56245fb45ff0 0x56245fb4cab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:11:23.946 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:23.942+0000 7f944c5c2740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56245fb46910 0x56245fb4cff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:11:23.946 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:23.942+0000 7f944c5c2740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56245fb528a0 0x56245fb6a030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:11:23.946 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:23.942+0000 7f944c5c2740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f94441b1d10 con 0x56245fb528a0 2024-09-11T00:11:23.946 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:23.942+0000 7f944c5c2740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f94441b3850 con 0x56245fb45ff0 2024-09-11T00:11:23.946 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:23.942+0000 7f944c5c2740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f94441b23e0 con 0x56245fb46910 2024-09-11T00:11:23.946 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:23.943+0000 7f9442000640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56245fb46910 0x56245fb4cff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:11:23.946 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:23.943+0000 7f9442000640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56245fb46910 0x56245fb4cff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.12:48044/0 (socket says 172.21.15.12:48044) 2024-09-11T00:11:23.946 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:23.943+0000 7f9442000640 1 -- 172.21.15.12:0/1189293907 learned_addr learned my addr 172.21.15.12:0/1189293907 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:11:23.946 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:23.943+0000 7f9443400640 1 --2- 172.21.15.12:0/1189293907 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56245fb45ff0 0x56245fb4cab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:11:23.947 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:23.943+0000 7f9443400640 1 --2- 172.21.15.12:0/1189293907 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56245fb528a0 0x56245fb6a030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:11:23.947 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:23.943+0000 7f9442000640 1 -- 172.21.15.12:0/1189293907 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56245fb45ff0 msgr2=0x56245fb4cab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:11:23.947 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:23.943+0000 7f9442000640 1 --2- 172.21.15.12:0/1189293907 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56245fb45ff0 0x56245fb4cab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:11:23.947 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:23.943+0000 7f9442000640 1 -- 172.21.15.12:0/1189293907 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56245fb528a0 msgr2=0x56245fb6a030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:11:23.947 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:23.943+0000 7f9442000640 1 --2- 172.21.15.12:0/1189293907 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56245fb528a0 0x56245fb6a030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:11:23.947 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:23.943+0000 7f9442000640 1 -- 172.21.15.12:0/1189293907 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56245fb45500 con 0x56245fb46910 2024-09-11T00:11:23.947 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:23.943+0000 7f9443400640 1 --2- 172.21.15.12:0/1189293907 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56245fb528a0 0x56245fb6a030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T00:11:23.947 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:23.943+0000 7f9442000640 1 --2- 172.21.15.12:0/1189293907 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56245fb46910 0x56245fb4cff0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f94300105f0 tx=0x7f943000f810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4380e3557797c30e server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:11:23.947 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:23.944+0000 7f942f400640 1 -- 172.21.15.12:0/1189293907 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f943002e070 con 0x56245fb46910 2024-09-11T00:11:23.947 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:23.944+0000 7f942f400640 1 -- 172.21.15.12:0/1189293907 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9430005740 con 0x56245fb46910 2024-09-11T00:11:23.947 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:23.944+0000 7f942f400640 1 -- 172.21.15.12:0/1189293907 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f943000fb80 con 0x56245fb46910 2024-09-11T00:11:23.947 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:23.944+0000 7f944c5c2740 1 -- 172.21.15.12:0/1189293907 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56245fb46910 msgr2=0x56245fb4cff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:11:23.947 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:23.944+0000 7f944c5c2740 1 --2- 172.21.15.12:0/1189293907 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56245fb46910 0x56245fb4cff0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f94300105f0 tx=0x7f943000f810 comp rx=0 tx=0).stop 2024-09-11T00:11:23.948 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:23.944+0000 7f944c5c2740 1 -- 172.21.15.12:0/1189293907 shutdown_connections 2024-09-11T00:11:23.948 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:23.944+0000 7f944c5c2740 1 --2- 172.21.15.12:0/1189293907 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56245fb528a0 0x56245fb6a030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:11:23.948 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:23.944+0000 7f944c5c2740 1 --2- 172.21.15.12:0/1189293907 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56245fb46910 0x56245fb4cff0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:11:23.948 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:23.944+0000 7f944c5c2740 1 --2- 172.21.15.12:0/1189293907 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56245fb45ff0 0x56245fb4cab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:11:23.948 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:23.944+0000 7f944c5c2740 1 -- 172.21.15.12:0/1189293907 >> 172.21.15.12:0/1189293907 conn(0x56245fb43e20 msgr2=0x56245fb5e870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:11:23.948 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:23.944+0000 7f944c5c2740 1 -- 172.21.15.12:0/1189293907 shutdown_connections 2024-09-11T00:11:23.948 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:23.944+0000 7f944c5c2740 1 -- 172.21.15.12:0/1189293907 wait complete. 2024-09-11T00:11:23.948 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:23.945+0000 7f944c5c2740 1 -- start start 2024-09-11T00:11:23.948 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:23.945+0000 7f944c5c2740 1 Processor -- start 2024-09-11T00:11:23.952 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:23.945+0000 7f944c5c2740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56245fb45ff0 0x56245fb569f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:11:23.952 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:23.945+0000 7f944c5c2740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56245fb46910 0x56245fb56f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:11:23.952 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:23.945+0000 7f944c5c2740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56245fb528a0 0x56245fbce230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:11:23.952 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:23.945+0000 7f944c5c2740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f94441b3180 con 0x56245fb46910 2024-09-11T00:11:23.952 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:23.945+0000 7f944c5c2740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f94441b2ab0 con 0x56245fb528a0 2024-09-11T00:11:23.952 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:23.945+0000 7f944c5c2740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f94441afcf0 con 0x56245fb45ff0 2024-09-11T00:11:23.952 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:23.945+0000 7f944c5c2740 10 client.0 _ll_register_callbacks cb 0x7f9430009930 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:11:23.952 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:23.946+0000 7f9442000640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56245fb46910 0x56245fb56f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:11:23.952 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:23.946+0000 7f9442000640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56245fb46910 0x56245fb56f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:44002/0 (socket says 172.21.15.12:44002) 2024-09-11T00:11:23.953 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:23.946+0000 7f9442000640 1 -- 172.21.15.12:0/2731844538 learned_addr learned my addr 172.21.15.12:0/2731844538 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:11:23.953 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:23.946+0000 7f9443400640 1 --2- 172.21.15.12:0/2731844538 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56245fb45ff0 0x56245fb569f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:11:23.953 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:23.946+0000 7f9443400640 1 --2- 172.21.15.12:0/2731844538 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56245fb528a0 0x56245fbce230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:11:23.953 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:23.946+0000 7f9442000640 1 -- 172.21.15.12:0/2731844538 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56245fb45ff0 msgr2=0x56245fb569f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:11:23.953 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:23.946+0000 7f9442000640 1 --2- 172.21.15.12:0/2731844538 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56245fb45ff0 0x56245fb569f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:11:23.953 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:23.946+0000 7f9442000640 1 -- 172.21.15.12:0/2731844538 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56245fb528a0 msgr2=0x56245fbce230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:11:23.954 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:23.946+0000 7f9442000640 1 --2- 172.21.15.12:0/2731844538 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56245fb528a0 0x56245fbce230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:11:23.954 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:23.946+0000 7f9442000640 1 -- 172.21.15.12:0/2731844538 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56245fb46740 con 0x56245fb46910 2024-09-11T00:11:23.954 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:23.946+0000 7f9442000640 1 --2- 172.21.15.12:0/2731844538 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56245fb46910 0x56245fb56f30 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f94300043e0 tx=0x7f9430012450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=731eee7f9afdedaf server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:11:23.954 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:23.946+0000 7f942cc00640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-09-11T00:11:23.954 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:23.947+0000 7f942cc00640 1 -- 172.21.15.12:0/2731844538 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f943002e040 con 0x56245fb46910 2024-09-11T00:11:23.954 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:23.947+0000 7f944c5c2740 1 -- 172.21.15.12:0/2731844538 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9430005740 con 0x56245fb46910 2024-09-11T00:11:23.955 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:23.947+0000 7f942cc00640 1 -- 172.21.15.12:0/2731844538 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9208+0+0 (secure 0 0 0) 0x7f943000fcd0 con 0x56245fb46910 2024-09-11T00:11:23.955 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:23.947+0000 7f942cc00640 1 -- 172.21.15.12:0/2731844538 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f9430005020 con 0x56245fb46910 2024-09-11T00:11:23.955 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:23.947+0000 7f944c5c2740 10 client.16017 fetch_fsmap learned FSMap version 37 2024-09-11T00:11:23.955 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:23.947+0000 7f944c5c2740 1 -- 172.21.15.12:0/2731844538 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x56245fbcf750 con 0x56245fb46910 2024-09-11T00:11:23.955 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:23.947+0000 7f942cc00640 1 -- 172.21.15.12:0/2731844538 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9430005740 con 0x56245fb46910 2024-09-11T00:11:23.955 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:23.947+0000 7f942cc00640 1 -- 172.21.15.12:0/2731844538 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 37) ==== 8370+0+0 (secure 0 0 0) 0x7f9430031020 con 0x56245fb46910 2024-09-11T00:11:23.955 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:23.948+0000 7f944c5c2740 10 client.16017 fetch_fsmap finished waiting for FSMap version 37 2024-09-11T00:11:23.955 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:23.948+0000 7f944c5c2740 10 client.16017 resolve_mds: validated gid 24674 aka daemon mds.b 2024-09-11T00:11:23.955 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:23.948+0000 7f944c5c2740 20 client.16017 populate_metadata read hostname 'smithi012' 2024-09-11T00:11:23.955 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:23.948+0000 7f944c5c2740 1 --2- 172.21.15.12:0/2731844538 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x56245fbcfe30 0x56245fbd2250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:11:23.956 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:23.948+0000 7f944c5c2740 4 client.16017 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "8c2b0c3c", "release": true, "await": true}] 2024-09-11T00:11:23.956 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:23.948+0000 7f944c5c2740 1 -- 172.21.15.12:0/2731844538 --> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] -- command(tid 0: {"prefix": "quiesce db", "set_id": "8c2b0c3c", "release": true, "await": true}) -- 0x56245fabac90 con 0x56245fbcfe30 2024-09-11T00:11:23.956 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:23.948+0000 7f9443400640 1 --2- 172.21.15.12:0/2731844538 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x56245fbcfe30 0x56245fbd2250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:11:23.956 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:23.949+0000 7f9443400640 1 --2- 172.21.15.12:0/2731844538 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x56245fbcfe30 0x56245fbd2250 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=124e11f03dbf9573 server_cookie=78d37ec1a701ae43 in_seq=0 out_seq=0 2024-09-11T00:11:23.956 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:23.949+0000 7f942cc00640 10 client.16017 ms_handle_connect on v2:172.21.15.165:6838/501312984 2024-09-11T00:11:23.958 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:23.956+0000 7f942cc00640 1 -- 172.21.15.12:0/2731844538 <== mds.0 v2:172.21.15.165:6838/501312984 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x56245fabac90 con 0x56245fbcfe30 2024-09-11T00:11:23.958 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:23.956+0000 7f942cc00640 10 client.16017 handle_command_reply: tid=0 2024-09-11T00:11:23.958 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:23.956+0000 7f944c5c2740 1 client.16017 shutdown 2024-09-11T00:11:23.958 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:23.956+0000 7f944c5c2740 1 -- 172.21.15.12:0/2731844538 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56245fb46910 msgr2=0x56245fb56f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:11:23.958 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:23.956+0000 7f944c5c2740 1 --2- 172.21.15.12:0/2731844538 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56245fb46910 0x56245fb56f30 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f94300043e0 tx=0x7f9430012450 comp rx=0 tx=0).stop 2024-09-11T00:11:23.959 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:23.956+0000 7f944c5c2740 1 -- 172.21.15.12:0/2731844538 shutdown_connections 2024-09-11T00:11:23.959 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:23.956+0000 7f944c5c2740 1 --2- 172.21.15.12:0/2731844538 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x56245fbcfe30 0x56245fbd2250 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:11:23.959 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:23.956+0000 7f944c5c2740 1 --2- 172.21.15.12:0/2731844538 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56245fb528a0 0x56245fbce230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:11:23.959 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:23.956+0000 7f944c5c2740 1 --2- 172.21.15.12:0/2731844538 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56245fb46910 0x56245fb56f30 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:11:23.959 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:23.956+0000 7f944c5c2740 1 --2- 172.21.15.12:0/2731844538 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56245fb45ff0 0x56245fb569f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:11:23.959 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:23.956+0000 7f944c5c2740 1 -- 172.21.15.12:0/2731844538 >> 172.21.15.12:0/2731844538 conn(0x56245fb43e20 msgr2=0x56245fb50ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:11:23.959 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:23.956+0000 7f942cc00640 0 client.16017 ms_handle_reset on v2:172.21.15.165:6838/501312984 2024-09-11T00:11:23.959 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:23.956+0000 7f944c5c2740 1 -- 172.21.15.12:0/2731844538 shutdown_connections 2024-09-11T00:11:23.959 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:23.957+0000 7f944c5c2740 1 -- 172.21.15.12:0/2731844538 wait complete. 2024-09-11T00:11:23.959 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:23.957+0000 7f944c5c2740 20 client.16017 trim_cache size 0 max 16384 2024-09-11T00:11:23.960 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-09-11T00:11:23.960 INFO:teuthology.orchestra.run.smithi012.stdout: "epoch": 37, 2024-09-11T00:11:23.960 INFO:teuthology.orchestra.run.smithi012.stdout: "leader": 24674, 2024-09-11T00:11:23.960 INFO:teuthology.orchestra.run.smithi012.stdout: "set_version": 16, 2024-09-11T00:11:23.960 INFO:teuthology.orchestra.run.smithi012.stdout: "sets": { 2024-09-11T00:11:23.960 INFO:teuthology.orchestra.run.smithi012.stdout: "8c2b0c3c": { 2024-09-11T00:11:23.960 INFO:teuthology.orchestra.run.smithi012.stdout: "version": 16, 2024-09-11T00:11:23.960 INFO:teuthology.orchestra.run.smithi012.stdout: "age_ref": 0.0, 2024-09-11T00:11:23.961 INFO:teuthology.orchestra.run.smithi012.stdout: "state": { 2024-09-11T00:11:23.961 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "RELEASED", 2024-09-11T00:11:23.961 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 0.0 2024-09-11T00:11:23.961 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:11:23.961 INFO:teuthology.orchestra.run.smithi012.stdout: "timeout": 90.0, 2024-09-11T00:11:23.961 INFO:teuthology.orchestra.run.smithi012.stdout: "expiration": 150.0, 2024-09-11T00:11:23.961 INFO:teuthology.orchestra.run.smithi012.stdout: "members": { 2024-09-11T00:11:23.961 INFO:teuthology.orchestra.run.smithi012.stdout: "file:/": { 2024-09-11T00:11:23.961 INFO:teuthology.orchestra.run.smithi012.stdout: "excluded": false, 2024-09-11T00:11:23.961 INFO:teuthology.orchestra.run.smithi012.stdout: "state": { 2024-09-11T00:11:23.961 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "RELEASED", 2024-09-11T00:11:23.961 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 0.0 2024-09-11T00:11:23.961 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:11:23.962 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:11:23.962 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:11:23.962 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:11:23.962 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:11:23.962 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-09-11T00:11:23.963 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:23.957+0000 7f9449a00640 1 -- 172.21.15.12:0/2954783838 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f94280b5360 msgr2=0x7f94280b7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:11:23.963 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:23.957+0000 7f9449a00640 1 --2- 172.21.15.12:0/2954783838 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f94280b5360 0x7f94280b7820 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f94340a8c60 tx=0x7f9434009720 comp rx=0 tx=0).stop 2024-09-11T00:11:23.963 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:23.957+0000 7f9449a00640 1 -- 172.21.15.12:0/2954783838 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9444104fc0 msgr2=0x7f944419b450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:11:23.963 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:23.957+0000 7f9449a00640 1 --2- 172.21.15.12:0/2954783838 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9444104fc0 0x7f944419b450 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f943400e9f0 tx=0x7f943400edf0 comp rx=0 tx=0).stop 2024-09-11T00:11:23.963 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:23.958+0000 7f9449a00640 1 -- 172.21.15.12:0/2954783838 shutdown_connections 2024-09-11T00:11:23.963 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:23.958+0000 7f9449a00640 1 --2- 172.21.15.12:0/2954783838 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f94280b5360 0x7f94280b7820 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:11:23.963 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:23.958+0000 7f9449a00640 1 --2- 172.21.15.12:0/2954783838 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f944410b350 0x7f94441a0d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:11:23.963 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:23.958+0000 7f9449a00640 1 --2- 172.21.15.12:0/2954783838 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f944410a9d0 0x7f944419b990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:11:23.963 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:23.958+0000 7f9449a00640 1 --2- 172.21.15.12:0/2954783838 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9444104fc0 0x7f944419b450 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:11:23.963 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:23.958+0000 7f9449a00640 1 -- 172.21.15.12:0/2954783838 >> 172.21.15.12:0/2954783838 conn(0x7f94441009f0 msgr2=0x7f9444101270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:11:23.964 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:23.958+0000 7f9449a00640 1 -- 172.21.15.12:0/2954783838 shutdown_connections 2024-09-11T00:11:23.964 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:23.958+0000 7f9449a00640 1 -- 172.21.15.12:0/2954783838 wait complete. 2024-09-11T00:11:24.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:11:23 smithi165 ceph-mon[35356]: pgmap v528: 129 pgs: 129 active+clean; 30 MiB data, 687 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 89 KiB/s wr, 24 op/s 2024-09-11T00:11:24.080 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:11:23 smithi178 ceph-mon[35919]: pgmap v528: 129 pgs: 129 active+clean; 30 MiB data, 687 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 89 KiB/s wr, 24 op/s 2024-09-11T00:11:24.629 INFO:tasks.workunit.client.0.smithi012.stdout: 1 123033 1.73 MB/sec execute 81 sec latency 29939.080 ms 2024-09-11T00:11:24.944 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '8c2b0c3c', total seconds elapsed: 34.1 2024-09-11T00:11:24.944 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 30.0 2024-09-11T00:11:24.944 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid ff3ad74a-6fcf-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T00:11:25.372 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:11:25.629 INFO:tasks.workunit.client.0.smithi012.stdout: 1 124588 1.86 MB/sec execute 82 sec latency 47.901 ms 2024-09-11T00:11:25.949 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:11:25 smithi012 ceph-mon[30333]: pgmap v529: 129 pgs: 129 active+clean; 30 MiB data, 687 MiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 5.9 KiB/s wr, 14 op/s 2024-09-11T00:11:25.961 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:25.959+0000 7fbb9c200640 1 -- 172.21.15.12:0/3604212676 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbb94104fa0 msgr2=0x7fbb94105380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:11:25.961 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:25.959+0000 7fbb9c200640 1 --2- 172.21.15.12:0/3604212676 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbb94104fa0 0x7fbb94105380 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fbb88009900 tx=0x7fbb8802a3f0 comp rx=0 tx=0).stop 2024-09-11T00:11:25.961 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:25.960+0000 7fbb9c200640 1 -- 172.21.15.12:0/3604212676 shutdown_connections 2024-09-11T00:11:25.962 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:25.960+0000 7fbb9c200640 1 --2- 172.21.15.12:0/3604212676 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbb9410b330 0x7fbb94111b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:11:25.962 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:25.960+0000 7fbb9c200640 1 --2- 172.21.15.12:0/3604212676 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbb9410a9b0 0x7fbb9410adf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:11:25.962 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:25.960+0000 7fbb9c200640 1 --2- 172.21.15.12:0/3604212676 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbb94104fa0 0x7fbb94105380 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:11:25.962 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:25.960+0000 7fbb9c200640 1 -- 172.21.15.12:0/3604212676 >> 172.21.15.12:0/3604212676 conn(0x7fbb94100a50 msgr2=0x7fbb94102e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:11:25.962 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:25.960+0000 7fbb9c200640 1 -- 172.21.15.12:0/3604212676 shutdown_connections 2024-09-11T00:11:25.962 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:25.960+0000 7fbb9c200640 1 -- 172.21.15.12:0/3604212676 wait complete. 2024-09-11T00:11:25.963 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:25.960+0000 7fbb9c200640 1 Processor -- start 2024-09-11T00:11:25.963 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:25.961+0000 7fbb9c200640 1 -- start start 2024-09-11T00:11:25.963 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:25.961+0000 7fbb9c200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbb94104fa0 0x7fbb9419b510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:11:25.963 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:25.961+0000 7fbb9c200640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbb9410a9b0 0x7fbb9419ba50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:11:25.963 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:25.961+0000 7fbb9c200640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbb9410b330 0x7fbb941a0df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:11:25.963 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:25.961+0000 7fbb9c200640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fbb94114f60 con 0x7fbb9410a9b0 2024-09-11T00:11:25.963 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:25.961+0000 7fbb9c200640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fbb94114de0 con 0x7fbb94104fa0 2024-09-11T00:11:25.964 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:25.961+0000 7fbb9c200640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fbb941150e0 con 0x7fbb9410b330 2024-09-11T00:11:25.964 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:25.962+0000 7fbb98e00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbb9410a9b0 0x7fbb9419ba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:11:25.964 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:25.962+0000 7fbb9a200640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbb9410b330 0x7fbb941a0df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:11:25.964 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:25.962+0000 7fbb99800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbb94104fa0 0x7fbb9419b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:11:25.964 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:25.962+0000 7fbb9a200640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbb9410b330 0x7fbb941a0df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.12:48080/0 (socket says 172.21.15.12:48080) 2024-09-11T00:11:25.964 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:25.962+0000 7fbb99800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbb94104fa0 0x7fbb9419b510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.12:41688/0 (socket says 172.21.15.12:41688) 2024-09-11T00:11:25.964 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:25.962+0000 7fbb9a200640 1 -- 172.21.15.12:0/565991495 learned_addr learned my addr 172.21.15.12:0/565991495 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:11:25.964 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:25.962+0000 7fbb98e00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbb9410a9b0 0x7fbb9419ba50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:44020/0 (socket says 172.21.15.12:44020) 2024-09-11T00:11:25.964 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:25.962+0000 7fbb9a200640 1 -- 172.21.15.12:0/565991495 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbb94104fa0 msgr2=0x7fbb9419b510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:11:25.964 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:25.962+0000 7fbb9a200640 1 --2- 172.21.15.12:0/565991495 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbb94104fa0 0x7fbb9419b510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:11:25.964 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:25.962+0000 7fbb9a200640 1 -- 172.21.15.12:0/565991495 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbb9410a9b0 msgr2=0x7fbb9419ba50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:11:25.965 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:25.962+0000 7fbb9a200640 1 --2- 172.21.15.12:0/565991495 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbb9410a9b0 0x7fbb9419ba50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:11:25.965 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:25.962+0000 7fbb9a200640 1 -- 172.21.15.12:0/565991495 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb941a15c0 con 0x7fbb9410b330 2024-09-11T00:11:25.965 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:25.962+0000 7fbb9a200640 1 --2- 172.21.15.12:0/565991495 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbb9410b330 0x7fbb941a0df0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fbb8c00eb10 tx=0x7fbb8c00c2d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:11:25.965 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:25.963+0000 7fbb92000640 1 -- 172.21.15.12:0/565991495 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb8c004b20 con 0x7fbb9410b330 2024-09-11T00:11:25.966 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:25.963+0000 7fbb92000640 1 -- 172.21.15.12:0/565991495 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbb8c004cc0 con 0x7fbb9410b330 2024-09-11T00:11:25.966 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:25.963+0000 7fbb92000640 1 -- 172.21.15.12:0/565991495 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb8c010670 con 0x7fbb9410b330 2024-09-11T00:11:25.966 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:25.963+0000 7fbb9c200640 1 -- 172.21.15.12:0/565991495 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbb941a18b0 con 0x7fbb9410b330 2024-09-11T00:11:25.966 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:25.963+0000 7fbb9c200640 1 -- 172.21.15.12:0/565991495 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbb941a7f70 con 0x7fbb9410b330 2024-09-11T00:11:25.966 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:25.964+0000 7fbb9c200640 1 -- 172.21.15.12:0/565991495 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbb94105380 con 0x7fbb9410b330 2024-09-11T00:11:25.967 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:25.966+0000 7fbb92000640 1 -- 172.21.15.12:0/565991495 <== mon.1 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7fbb8c005020 con 0x7fbb9410b330 2024-09-11T00:11:25.968 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:25.967+0000 7fbb92000640 1 --2- 172.21.15.12:0/565991495 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fbb7c0b5360 0x7fbb7c0b7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:11:25.969 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:25.968+0000 7fbb92000640 1 -- 172.21.15.12:0/565991495 <== mon.1 v2:172.21.15.178:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9208+0+0 (secure 0 0 0) 0x7fbb8c0e64b0 con 0x7fbb9410b330 2024-09-11T00:11:25.969 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:25.968+0000 7fbb99800640 1 --2- 172.21.15.12:0/565991495 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fbb7c0b5360 0x7fbb7c0b7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:11:25.970 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:25.969+0000 7fbb99800640 1 --2- 172.21.15.12:0/565991495 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fbb7c0b5360 0x7fbb7c0b7820 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fbb88031440 tx=0x7fbb88005b50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:11:25.972 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:25.971+0000 7fbb92000640 1 -- 172.21.15.12:0/565991495 <== mon.1 v2:172.21.15.178:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7fbb8c0acf40 con 0x7fbb9410b330 2024-09-11T00:11:26.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:11:25 smithi165 ceph-mon[35356]: pgmap v529: 129 pgs: 129 active+clean; 30 MiB data, 687 MiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 5.9 KiB/s wr, 14 op/s 2024-09-11T00:11:26.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:11:25 smithi178 ceph-mon[35919]: pgmap v529: 129 pgs: 129 active+clean; 30 MiB data, 687 MiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 5.9 KiB/s wr, 14 op/s 2024-09-11T00:11:26.212 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:26.210+0000 7fbb9c200640 1 -- 172.21.15.12:0/565991495 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbb9402e5c0 con 0x7fbb9410b330 2024-09-11T00:11:26.213 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:26.211+0000 7fbb92000640 1 -- 172.21.15.12:0/565991495 <== mon.1 v2:172.21.15.178:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 37 v37) ==== 94+0+11965 (secure 0 0 0) 0x7fbb8c0ac840 con 0x7fbb9410b330 2024-09-11T00:11:26.213 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-11T00:11:26.213 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":37,"btime":"2024-09-11T00:11:07:606386+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14577,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6835/447171591","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":447171591},{"type":"v1","addr":"172.21.15.12:6835","nonce":447171591}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14589,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/901486040","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":901486040},{"type":"v1","addr":"172.21.15.12:6837","nonce":901486040}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":37,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T00:04:04.739881+0000","modified":"2024-09-11T00:11:06.943614+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24674,"mds_1":24643,"mds_2":24638,"mds_3":24637,"mds_4":24614},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14631":{"gid":14631,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.12:6841/3069919437","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6840","nonce":3069919437},{"type":"v1","addr":"172.21.15.12:6841","nonce":3069919437}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24562":{"gid":24562,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6833/1141636112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":1141636112},{"type":"v1","addr":"172.21.15.178:6833","nonce":1141636112}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24581":{"gid":24581,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6833/3111980970","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6832","nonce":3111980970},{"type":"v1","addr":"172.21.15.165:6833","nonce":3111980970}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24607":{"gid":24607,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6835/4084868287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":4084868287},{"type":"v1","addr":"172.21.15.178:6835","nonce":4084868287}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24614":{"gid":24614,"name":"e","rank":4,"incarnation":26,"state":"up:active","state_seq":21,"addr":"172.21.15.165:6835/2425395274","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":2425395274},{"type":"v1","addr":"172.21.15.165:6835","nonce":2425395274}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24637":{"gid":24637,"name":"i","rank":3,"incarnation":24,"state":"up:active","state_seq":18,"addr":"172.21.15.178:6839/70628097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6838","nonce":70628097},{"type":"v1","addr":"172.21.15.178:6839","nonce":70628097}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24638":{"gid":24638,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":18,"addr":"172.21.15.165:6837/6215088","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6836","nonce":6215088},{"type":"v1","addr":"172.21.15.165:6837","nonce":6215088}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24643":{"gid":24643,"name":"g","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.12:6839/3124093009","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6838","nonce":3124093009},{"type":"v1","addr":"172.21.15.12:6839","nonce":3124093009}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24674":{"gid":24674,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.165:6839/501312984","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6838","nonce":501312984},{"type":"v1","addr":"172.21.15.165:6839","nonce":501312984}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25672":{"gid":25672,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6837/308601741","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":308601741},{"type":"v1","addr":"172.21.15.178:6837","nonce":308601741}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24674,"qdb_cluster":[24614,24637,24638,24643,24674]},"id":1}]} 2024-09-11T00:11:26.213 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 37 2024-09-11T00:11:26.215 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:26.214+0000 7fbb9c200640 1 -- 172.21.15.12:0/565991495 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fbb7c0b5360 msgr2=0x7fbb7c0b7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:11:26.215 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:26.214+0000 7fbb9c200640 1 --2- 172.21.15.12:0/565991495 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fbb7c0b5360 0x7fbb7c0b7820 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fbb88031440 tx=0x7fbb88005b50 comp rx=0 tx=0).stop 2024-09-11T00:11:26.215 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:26.214+0000 7fbb9c200640 1 -- 172.21.15.12:0/565991495 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbb9410b330 msgr2=0x7fbb941a0df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:11:26.215 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:26.214+0000 7fbb9c200640 1 --2- 172.21.15.12:0/565991495 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbb9410b330 0x7fbb941a0df0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fbb8c00eb10 tx=0x7fbb8c00c2d0 comp rx=0 tx=0).stop 2024-09-11T00:11:26.215 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:26.214+0000 7fbb9c200640 1 -- 172.21.15.12:0/565991495 shutdown_connections 2024-09-11T00:11:26.215 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:26.214+0000 7fbb9c200640 1 --2- 172.21.15.12:0/565991495 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fbb7c0b5360 0x7fbb7c0b7820 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:11:26.216 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:26.214+0000 7fbb9c200640 1 --2- 172.21.15.12:0/565991495 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbb9410b330 0x7fbb941a0df0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:11:26.216 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:26.214+0000 7fbb9c200640 1 --2- 172.21.15.12:0/565991495 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbb9410a9b0 0x7fbb9419ba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:11:26.216 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:26.214+0000 7fbb9c200640 1 --2- 172.21.15.12:0/565991495 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbb94104fa0 0x7fbb9419b510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:11:26.216 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:26.214+0000 7fbb9c200640 1 -- 172.21.15.12:0/565991495 >> 172.21.15.12:0/565991495 conn(0x7fbb94100a50 msgr2=0x7fbb94102e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:11:26.216 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:26.214+0000 7fbb9c200640 1 -- 172.21.15.12:0/565991495 shutdown_connections 2024-09-11T00:11:26.216 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:26.214+0000 7fbb9c200640 1 -- 172.21.15.12:0/565991495 wait complete. 2024-09-11T00:11:26.629 INFO:tasks.workunit.client.0.smithi012.stdout: 1 124905 1.84 MB/sec execute 83 sec latency 667.101 ms 2024-09-11T00:11:27.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:11:26 smithi165 ceph-mon[35356]: from='client.? 172.21.15.12:0/565991495' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T00:11:27.063 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:11:26 smithi012 ceph-mon[30333]: from='client.? 172.21.15.12:0/565991495' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T00:11:27.080 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:11:26 smithi178 ceph-mon[35919]: from='client.? 172.21.15.12:0/565991495' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T00:11:27.132 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24674 quiesce db / --timeout 90 --expiration 150.0 --await-for 45.0' 2024-09-11T00:11:27.133 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid ff3ad74a-6fcf-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24674 quiesce db / --timeout 90 --expiration 150.0 --await-for 45.0 2024-09-11T00:11:27.512 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:11:27.629 INFO:tasks.workunit.client.0.smithi012.stdout: 1 124905 1.82 MB/sec execute 84 sec latency 1667.198 ms 2024-09-11T00:11:27.932 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:11:27 smithi012 ceph-mon[30333]: pgmap v530: 129 pgs: 129 active+clean; 31 MiB data, 689 MiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 136 KiB/s wr, 51 op/s 2024-09-11T00:11:28.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:11:27 smithi165 ceph-mon[35356]: pgmap v530: 129 pgs: 129 active+clean; 31 MiB data, 689 MiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 136 KiB/s wr, 51 op/s 2024-09-11T00:11:28.059 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:28.058+0000 7f1cea200640 1 -- 172.21.15.12:0/3096692960 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1ce41087d0 msgr2=0x7f1ce4108c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:11:28.059 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:28.058+0000 7f1cea200640 1 --2- 172.21.15.12:0/3096692960 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1ce41087d0 0x7f1ce4108c10 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f1ccc009900 tx=0x7f1ccc02a410 comp rx=0 tx=0).stop 2024-09-11T00:11:28.059 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:28.058+0000 7f1cea200640 1 -- 172.21.15.12:0/3096692960 shutdown_connections 2024-09-11T00:11:28.059 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:28.058+0000 7f1cea200640 1 --2- 172.21.15.12:0/3096692960 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1ce4109150 0x7f1ce410f950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:11:28.060 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:28.058+0000 7f1cea200640 1 --2- 172.21.15.12:0/3096692960 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1ce41087d0 0x7f1ce4108c10 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:11:28.060 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:28.058+0000 7f1cea200640 1 --2- 172.21.15.12:0/3096692960 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1ce4102dc0 0x7f1ce41031a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:11:28.060 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:28.058+0000 7f1cea200640 1 -- 172.21.15.12:0/3096692960 >> 172.21.15.12:0/3096692960 conn(0x7f1ce40fe830 msgr2=0x7f1ce4100c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:11:28.060 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:28.058+0000 7f1cea200640 1 -- 172.21.15.12:0/3096692960 shutdown_connections 2024-09-11T00:11:28.060 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:28.058+0000 7f1cea200640 1 -- 172.21.15.12:0/3096692960 wait complete. 2024-09-11T00:11:28.061 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:28.059+0000 7f1cea200640 1 Processor -- start 2024-09-11T00:11:28.061 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:28.059+0000 7f1cea200640 1 -- start start 2024-09-11T00:11:28.061 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:28.059+0000 7f1cea200640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1ce4102dc0 0x7f1ce4199290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:11:28.061 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:28.059+0000 7f1cea200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1ce41087d0 0x7f1ce41997d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:11:28.061 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:28.059+0000 7f1cea200640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1ce4109150 0x7f1ce419eb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:11:28.061 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:28.059+0000 7f1cea200640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f1ce4112e60 con 0x7f1ce4109150 2024-09-11T00:11:28.062 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:28.059+0000 7f1cea200640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f1ce4112ce0 con 0x7f1ce41087d0 2024-09-11T00:11:28.062 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:28.059+0000 7f1cea200640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f1ce4112fe0 con 0x7f1ce4102dc0 2024-09-11T00:11:28.062 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:28.059+0000 7f1ce3e00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1ce4109150 0x7f1ce419eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:11:28.062 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:28.060+0000 7f1ce3400640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1ce4102dc0 0x7f1ce4199290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:11:28.062 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:28.060+0000 7f1ce3e00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1ce4109150 0x7f1ce419eb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:44040/0 (socket says 172.21.15.12:44040) 2024-09-11T00:11:28.062 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:28.060+0000 7f1cdaa00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1ce41087d0 0x7f1ce41997d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:11:28.062 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:28.060+0000 7f1ce3e00640 1 -- 172.21.15.12:0/3153005508 learned_addr learned my addr 172.21.15.12:0/3153005508 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:11:28.062 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:28.060+0000 7f1ce3400640 1 -- 172.21.15.12:0/3153005508 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1ce41087d0 msgr2=0x7f1ce41997d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:11:28.062 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:28.060+0000 7f1ce3400640 1 --2- 172.21.15.12:0/3153005508 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1ce41087d0 0x7f1ce41997d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:11:28.062 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:28.060+0000 7f1ce3400640 1 -- 172.21.15.12:0/3153005508 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1ce4109150 msgr2=0x7f1ce419eb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:11:28.062 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:28.060+0000 7f1ce3400640 1 --2- 172.21.15.12:0/3153005508 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1ce4109150 0x7f1ce419eb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:11:28.062 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:28.060+0000 7f1ce3400640 1 -- 172.21.15.12:0/3153005508 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ce419f340 con 0x7f1ce4102dc0 2024-09-11T00:11:28.062 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:28.060+0000 7f1cdaa00640 1 --2- 172.21.15.12:0/3153005508 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1ce41087d0 0x7f1ce41997d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:11:28.063 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:28.060+0000 7f1ce3e00640 1 --2- 172.21.15.12:0/3153005508 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1ce4109150 0x7f1ce419eb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:11:28.063 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:28.060+0000 7f1ce3400640 1 --2- 172.21.15.12:0/3153005508 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1ce4102dc0 0x7f1ce4199290 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f1cd400e9f0 tx=0x7f1cd400eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:11:28.063 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:28.061+0000 7f1ce0c00640 1 -- 172.21.15.12:0/3153005508 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cd400ccc0 con 0x7f1ce4102dc0 2024-09-11T00:11:28.063 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:28.061+0000 7f1ce0c00640 1 -- 172.21.15.12:0/3153005508 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1cd40108e0 con 0x7f1ce4102dc0 2024-09-11T00:11:28.064 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:28.061+0000 7f1cea200640 1 -- 172.21.15.12:0/3153005508 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ce419f630 con 0x7f1ce4102dc0 2024-09-11T00:11:28.064 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:28.061+0000 7f1ce0c00640 1 -- 172.21.15.12:0/3153005508 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cd4018880 con 0x7f1ce4102dc0 2024-09-11T00:11:28.064 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:28.061+0000 7f1cea200640 1 -- 172.21.15.12:0/3153005508 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ce41a5ec0 con 0x7f1ce4102dc0 2024-09-11T00:11:28.065 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:28.064+0000 7f1cece63740 1 Processor -- start 2024-09-11T00:11:28.065 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:28.064+0000 7f1cece63740 1 -- start start 2024-09-11T00:11:28.066 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:28.064+0000 7f1ce0c00640 1 -- 172.21.15.12:0/3153005508 <== mon.1 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7f1cd4010430 con 0x7f1ce4102dc0 2024-09-11T00:11:28.066 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:28.064+0000 7f1cece63740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55629c58a180 0x55629c58a560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:11:28.066 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:28.064+0000 7f1cece63740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55629c58aaa0 0x55629c59a900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:11:28.066 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:28.064+0000 7f1cece63740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55629c59ae40 0x55629c59d230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:11:28.066 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:28.064+0000 7f1cece63740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55629c4ffca0 con 0x55629c58aaa0 2024-09-11T00:11:28.066 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:28.064+0000 7f1cece63740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55629c4ef0d0 con 0x55629c58a180 2024-09-11T00:11:28.066 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:28.064+0000 7f1cece63740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55629c4f2ee0 con 0x55629c59ae40 2024-09-11T00:11:28.066 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:28.065+0000 7f1ce3400640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55629c58aaa0 0x55629c59a900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:11:28.066 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:28.065+0000 7f1ce3e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55629c58a180 0x55629c58a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:11:28.066 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:28.065+0000 7f1ce3400640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55629c58aaa0 0x55629c59a900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:44050/0 (socket says 172.21.15.12:44050) 2024-09-11T00:11:28.066 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:28.065+0000 7f1ce3400640 1 -- 172.21.15.12:0/3912146170 learned_addr learned my addr 172.21.15.12:0/3912146170 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:11:28.067 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:28.065+0000 7f1cdaa00640 1 --2- 172.21.15.12:0/3912146170 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55629c59ae40 0x55629c59d230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:11:28.068 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:28.065+0000 7f1cdaa00640 1 -- 172.21.15.12:0/3912146170 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55629c58a180 msgr2=0x55629c58a560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:11:28.068 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:28.065+0000 7f1cdaa00640 1 --2- 172.21.15.12:0/3912146170 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55629c58a180 0x55629c58a560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:11:28.068 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:28.065+0000 7f1cdaa00640 1 -- 172.21.15.12:0/3912146170 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55629c58aaa0 msgr2=0x55629c59a900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:11:28.068 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:28.065+0000 7f1cdaa00640 1 --2- 172.21.15.12:0/3912146170 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55629c58aaa0 0x55629c59a900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:11:28.068 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:28.065+0000 7f1cdaa00640 1 -- 172.21.15.12:0/3912146170 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55629c4a1400 con 0x55629c59ae40 2024-09-11T00:11:28.068 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:28.066+0000 7f1cdaa00640 1 --2- 172.21.15.12:0/3912146170 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55629c59ae40 0x55629c59d230 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f1ccc02aa00 tx=0x7f1ccc004520 comp rx=0 tx=0).ready entity=mon.1 client_cookie=49031d00b2bef522 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:11:28.068 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:28.066+0000 7f1cd3e00640 1 -- 172.21.15.12:0/3912146170 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ccc035040 con 0x55629c59ae40 2024-09-11T00:11:28.068 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:28.066+0000 7f1cd3e00640 1 -- 172.21.15.12:0/3912146170 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1ccc004c20 con 0x55629c59ae40 2024-09-11T00:11:28.068 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:28.066+0000 7f1cd3e00640 1 -- 172.21.15.12:0/3912146170 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ccc003920 con 0x55629c59ae40 2024-09-11T00:11:28.069 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:28.066+0000 7f1ce0c00640 1 --2- 172.21.15.12:0/3153005508 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f1cc80b5360 0x7f1cc80b7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:11:28.069 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:28.067+0000 7f1cece63740 1 -- 172.21.15.12:0/3912146170 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55629c59ae40 msgr2=0x55629c59d230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:11:28.069 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:28.067+0000 7f1cece63740 1 --2- 172.21.15.12:0/3912146170 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55629c59ae40 0x55629c59d230 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f1ccc02aa00 tx=0x7f1ccc004520 comp rx=0 tx=0).stop 2024-09-11T00:11:28.069 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:28.067+0000 7f1ce0c00640 1 -- 172.21.15.12:0/3153005508 <== mon.1 v2:172.21.15.178:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9208+0+0 (secure 0 0 0) 0x7f1cd40e6250 con 0x7f1ce4102dc0 2024-09-11T00:11:28.069 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:28.067+0000 7f1cece63740 1 -- 172.21.15.12:0/3912146170 shutdown_connections 2024-09-11T00:11:28.069 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:28.067+0000 7f1ce3e00640 1 --2- 172.21.15.12:0/3153005508 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f1cc80b5360 0x7f1cc80b7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:11:28.069 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:28.067+0000 7f1cece63740 1 --2- 172.21.15.12:0/3912146170 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55629c59ae40 0x55629c59d230 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:11:28.069 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:28.067+0000 7f1cece63740 1 --2- 172.21.15.12:0/3912146170 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55629c58aaa0 0x55629c59a900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:11:28.069 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:28.067+0000 7f1cece63740 1 --2- 172.21.15.12:0/3912146170 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55629c58a180 0x55629c58a560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:11:28.069 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:28.067+0000 7f1cece63740 1 -- 172.21.15.12:0/3912146170 >> 172.21.15.12:0/3912146170 conn(0x55629c587fb0 msgr2=0x55629c589150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:11:28.069 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:28.067+0000 7f1cece63740 1 -- 172.21.15.12:0/3912146170 shutdown_connections 2024-09-11T00:11:28.069 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:28.067+0000 7f1cece63740 1 -- 172.21.15.12:0/3912146170 wait complete. 2024-09-11T00:11:28.072 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:28.067+0000 7f1cece63740 1 -- start start 2024-09-11T00:11:28.072 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:28.068+0000 7f1ce3e00640 1 --2- 172.21.15.12:0/3153005508 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f1cc80b5360 0x7f1cc80b7820 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f1cdc009270 tx=0x7f1cdc009680 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:11:28.072 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:28.068+0000 7f1cece63740 1 Processor -- start 2024-09-11T00:11:28.072 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:28.068+0000 7f1cece63740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55629c58a180 0x55629c59a390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:11:28.072 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:28.068+0000 7f1cece63740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55629c58aaa0 0x55629c5937c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:11:28.073 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:28.068+0000 7f1cece63740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55629c59ae40 0x55629c593d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:11:28.073 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:28.068+0000 7f1cece63740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55629c3507a0 con 0x55629c58aaa0 2024-09-11T00:11:28.073 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:28.068+0000 7f1cece63740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55629c494a10 con 0x55629c59ae40 2024-09-11T00:11:28.073 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:28.068+0000 7f1cece63740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55629c59da90 con 0x55629c58a180 2024-09-11T00:11:28.073 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:28.068+0000 7f1cece63740 10 client.0 _ll_register_callbacks cb 0x55629c4a4be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:11:28.073 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:28.069+0000 7f1ce3400640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55629c58a180 0x55629c59a390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:11:28.073 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:28.069+0000 7f1cdaa00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55629c58aaa0 0x55629c5937c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:11:28.073 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:28.069+0000 7f1ce3400640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55629c58a180 0x55629c59a390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.12:48120/0 (socket says 172.21.15.12:48120) 2024-09-11T00:11:28.073 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:28.069+0000 7f1ce3400640 1 -- 172.21.15.12:0/3995080008 learned_addr learned my addr 172.21.15.12:0/3995080008 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:11:28.073 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:28.069+0000 7f1ce3e00640 1 --2- 172.21.15.12:0/3995080008 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55629c59ae40 0x55629c593d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:11:28.073 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:28.069+0000 7f1ce3400640 1 -- 172.21.15.12:0/3995080008 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55629c59ae40 msgr2=0x55629c593d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:11:28.073 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:28.069+0000 7f1ce3400640 1 --2- 172.21.15.12:0/3995080008 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55629c59ae40 0x55629c593d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:11:28.073 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:28.069+0000 7f1ce3400640 1 -- 172.21.15.12:0/3995080008 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55629c58aaa0 msgr2=0x55629c5937c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:11:28.074 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:28.069+0000 7f1ce3400640 1 --2- 172.21.15.12:0/3995080008 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55629c58aaa0 0x55629c5937c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:11:28.074 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:28.069+0000 7f1ce3400640 1 -- 172.21.15.12:0/3995080008 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55629c25cb90 con 0x55629c58a180 2024-09-11T00:11:28.074 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:28.069+0000 7f1ce3400640 1 --2- 172.21.15.12:0/3995080008 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55629c58a180 0x55629c59a390 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f1cd4000c00 tx=0x7f1cd400ce60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=aa4bd1f9065fd8d9 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:11:28.074 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:28.069+0000 7f1cd0c00640 10 client.0 ms_handle_connect on v2:172.21.15.178:3300/0 2024-09-11T00:11:28.074 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:28.069+0000 7f1cd0c00640 1 -- 172.21.15.12:0/3995080008 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cd40a8590 con 0x55629c58a180 2024-09-11T00:11:28.074 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:28.070+0000 7f1cece63740 1 -- 172.21.15.12:0/3995080008 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1ccc004c20 con 0x55629c58a180 2024-09-11T00:11:28.074 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:28.070+0000 7f1cd0c00640 1 -- 172.21.15.12:0/3995080008 <== mon.1 v2:172.21.15.178:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9208+0+0 (secure 0 0 0) 0x7f1cd40ab950 con 0x55629c58a180 2024-09-11T00:11:28.074 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:28.070+0000 7f1cd0c00640 1 -- 172.21.15.12:0/3995080008 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f1cd40abe60 con 0x55629c58a180 2024-09-11T00:11:28.074 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:28.070+0000 7f1cece63740 10 client.26044 fetch_fsmap learned FSMap version 37 2024-09-11T00:11:28.074 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:28.070+0000 7f1cece63740 1 -- 172.21.15.12:0/3995080008 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55629c5ad860 con 0x55629c58a180 2024-09-11T00:11:28.074 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:28.070+0000 7f1cd0c00640 1 -- 172.21.15.12:0/3995080008 <== mon.1 v2:172.21.15.178:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1ccc004c20 con 0x55629c58a180 2024-09-11T00:11:28.074 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:28.070+0000 7f1cd0c00640 1 -- 172.21.15.12:0/3995080008 <== mon.1 v2:172.21.15.178:3300/0 5 ==== fsmap(e 37) ==== 8370+0+0 (secure 0 0 0) 0x7f1cd40ab3c0 con 0x55629c58a180 2024-09-11T00:11:28.074 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:28.070+0000 7f1cece63740 10 client.26044 fetch_fsmap finished waiting for FSMap version 37 2024-09-11T00:11:28.074 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:28.070+0000 7f1cece63740 10 client.26044 resolve_mds: validated gid 24674 aka daemon mds.b 2024-09-11T00:11:28.075 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:28.070+0000 7f1cece63740 20 client.26044 populate_metadata read hostname 'smithi012' 2024-09-11T00:11:28.075 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:28.070+0000 7f1cece63740 1 --2- 172.21.15.12:0/3995080008 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55629c5aebb0 0x55629c5b0fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:11:28.075 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:28.070+0000 7f1cece63740 4 client.26044 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T00:11:28.075 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:28.070+0000 7f1cece63740 1 -- 172.21.15.12:0/3995080008 --> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55629c501bf0 con 0x55629c5aebb0 2024-09-11T00:11:28.075 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:28.071+0000 7f1cdaa00640 1 --2- 172.21.15.12:0/3995080008 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55629c5aebb0 0x55629c5b0fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:11:28.076 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:28.071+0000 7f1cdaa00640 1 --2- 172.21.15.12:0/3995080008 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55629c5aebb0 0x55629c5b0fc0 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f52e62361badd33d server_cookie=8100a0cd1c79b220 in_seq=0 out_seq=0 2024-09-11T00:11:28.076 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:28.071+0000 7f1cd0c00640 10 client.26044 ms_handle_connect on v2:172.21.15.165:6838/501312984 2024-09-11T00:11:28.076 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:28.072+0000 7f1cd0c00640 1 -- 172.21.15.12:0/3995080008 <== mds.0 v2:172.21.15.165:6838/501312984 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55629c501bf0 con 0x55629c5aebb0 2024-09-11T00:11:28.076 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:28.072+0000 7f1cd0c00640 10 client.26044 handle_command_reply: tid=0 2024-09-11T00:11:28.076 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:28.072+0000 7f1cece63740 1 client.26044 shutdown 2024-09-11T00:11:28.076 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:28.073+0000 7f1cece63740 1 -- 172.21.15.12:0/3995080008 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55629c58a180 msgr2=0x55629c59a390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:11:28.076 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:28.073+0000 7f1cece63740 1 --2- 172.21.15.12:0/3995080008 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55629c58a180 0x55629c59a390 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f1cd4000c00 tx=0x7f1cd400ce60 comp rx=0 tx=0).stop 2024-09-11T00:11:28.076 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:28.073+0000 7f1cece63740 1 -- 172.21.15.12:0/3995080008 shutdown_connections 2024-09-11T00:11:28.077 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:28.073+0000 7f1cece63740 1 --2- 172.21.15.12:0/3995080008 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55629c5aebb0 0x55629c5b0fc0 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:11:28.077 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:28.073+0000 7f1cece63740 1 --2- 172.21.15.12:0/3995080008 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55629c59ae40 0x55629c593d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:11:28.077 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:28.073+0000 7f1cece63740 1 --2- 172.21.15.12:0/3995080008 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55629c58aaa0 0x55629c5937c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:11:28.077 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:28.073+0000 7f1cd0c00640 0 client.26044 ms_handle_reset on v2:172.21.15.165:6838/501312984 2024-09-11T00:11:28.077 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:28.073+0000 7f1cece63740 1 --2- 172.21.15.12:0/3995080008 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55629c58a180 0x55629c59a390 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:11:28.077 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:28.073+0000 7f1cece63740 1 -- 172.21.15.12:0/3995080008 >> 172.21.15.12:0/3995080008 conn(0x55629c587fb0 msgr2=0x55629c5870b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:11:28.077 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:28.073+0000 7f1cece63740 1 -- 172.21.15.12:0/3995080008 shutdown_connections 2024-09-11T00:11:28.077 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:28.073+0000 7f1cece63740 1 -- 172.21.15.12:0/3995080008 wait complete. 2024-09-11T00:11:28.077 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:28.073+0000 7f1cece63740 20 client.26044 trim_cache size 0 max 16384 2024-09-11T00:11:28.080 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:11:27 smithi178 ceph-mon[35919]: pgmap v530: 129 pgs: 129 active+clean; 31 MiB data, 689 MiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 136 KiB/s wr, 51 op/s 2024-09-11T00:11:28.093 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:28.091+0000 7f1cece63740 1 Processor -- start 2024-09-11T00:11:28.093 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:28.091+0000 7f1cece63740 1 -- start start 2024-09-11T00:11:28.093 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:28.092+0000 7f1cece63740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55629c58a180 0x55629c589000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:11:28.093 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:28.092+0000 7f1cece63740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55629c58aaa0 0x55629c589540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:11:28.093 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:28.092+0000 7f1cece63740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55629c59ae40 0x55629c5909d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:11:28.093 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:28.092+0000 7f1cece63740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f1ce41ada30 con 0x55629c58a180 2024-09-11T00:11:28.093 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:28.092+0000 7f1cece63740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f1ce41af570 con 0x55629c59ae40 2024-09-11T00:11:28.093 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:28.092+0000 7f1cece63740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f1ce41ae100 con 0x55629c58aaa0 2024-09-11T00:11:28.094 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:28.092+0000 7f1ce3400640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55629c58a180 0x55629c589000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:11:28.094 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:28.092+0000 7f1ce3400640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55629c58a180 0x55629c589000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:44060/0 (socket says 172.21.15.12:44060) 2024-09-11T00:11:28.094 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:28.092+0000 7f1ce3400640 1 -- 172.21.15.12:0/800925412 learned_addr learned my addr 172.21.15.12:0/800925412 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:11:28.094 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:28.092+0000 7f1cdaa00640 1 --2- 172.21.15.12:0/800925412 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55629c58aaa0 0x55629c589540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:11:28.094 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:28.092+0000 7f1ce3e00640 1 --2- 172.21.15.12:0/800925412 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55629c59ae40 0x55629c5909d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:11:28.095 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:28.092+0000 7f1ce3400640 1 -- 172.21.15.12:0/800925412 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55629c58aaa0 msgr2=0x55629c589540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:11:28.095 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:28.092+0000 7f1ce3400640 1 --2- 172.21.15.12:0/800925412 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55629c58aaa0 0x55629c589540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:11:28.095 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:28.092+0000 7f1ce3400640 1 -- 172.21.15.12:0/800925412 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55629c59ae40 msgr2=0x55629c5909d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:11:28.095 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:28.092+0000 7f1ce3400640 1 --2- 172.21.15.12:0/800925412 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55629c59ae40 0x55629c5909d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:11:28.095 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:28.092+0000 7f1ce3400640 1 -- 172.21.15.12:0/800925412 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55629c58f120 con 0x55629c58a180 2024-09-11T00:11:28.096 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:28.093+0000 7f1ce3400640 1 --2- 172.21.15.12:0/800925412 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55629c58a180 0x55629c589000 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f1cd400e9c0 tx=0x7f1cd40e6170 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9453431b7ba1bcf server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:11:28.096 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:28.093+0000 7f1cd3400640 1 -- 172.21.15.12:0/800925412 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cd40ac040 con 0x55629c58a180 2024-09-11T00:11:28.096 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:28.093+0000 7f1cd3400640 1 -- 172.21.15.12:0/800925412 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1cd40ac200 con 0x55629c58a180 2024-09-11T00:11:28.096 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:28.093+0000 7f1cd3400640 1 -- 172.21.15.12:0/800925412 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cd40ac4c0 con 0x55629c58a180 2024-09-11T00:11:28.096 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:28.093+0000 7f1cece63740 1 -- 172.21.15.12:0/800925412 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55629c58a180 msgr2=0x55629c589000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:11:28.096 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:28.093+0000 7f1cece63740 1 --2- 172.21.15.12:0/800925412 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55629c58a180 0x55629c589000 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f1cd400e9c0 tx=0x7f1cd40e6170 comp rx=0 tx=0).stop 2024-09-11T00:11:28.096 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:28.093+0000 7f1cece63740 1 -- 172.21.15.12:0/800925412 shutdown_connections 2024-09-11T00:11:28.096 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:28.093+0000 7f1cece63740 1 --2- 172.21.15.12:0/800925412 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55629c59ae40 0x55629c5909d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:11:28.096 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:28.093+0000 7f1cece63740 1 --2- 172.21.15.12:0/800925412 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55629c58aaa0 0x55629c589540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:11:28.096 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:28.093+0000 7f1cece63740 1 --2- 172.21.15.12:0/800925412 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55629c58a180 0x55629c589000 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:11:28.096 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:28.094+0000 7f1cece63740 1 -- 172.21.15.12:0/800925412 >> 172.21.15.12:0/800925412 conn(0x55629c587fb0 msgr2=0x55629c593790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:11:28.096 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:28.094+0000 7f1cece63740 1 -- 172.21.15.12:0/800925412 shutdown_connections 2024-09-11T00:11:28.096 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:28.094+0000 7f1cece63740 1 -- 172.21.15.12:0/800925412 wait complete. 2024-09-11T00:11:28.097 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:28.094+0000 7f1cece63740 1 -- start start 2024-09-11T00:11:28.097 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:28.094+0000 7f1cece63740 1 Processor -- start 2024-09-11T00:11:28.098 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:28.095+0000 7f1cece63740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55629c58a180 0x55629c5abf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:11:28.098 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:28.095+0000 7f1cece63740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55629c58aaa0 0x55629c5a8460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:11:28.099 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:28.095+0000 7f1cece63740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55629c59ae40 0x55629c5a8b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:11:28.099 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:28.095+0000 7f1cece63740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f1ce41aeea0 con 0x55629c59ae40 2024-09-11T00:11:28.099 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:28.095+0000 7f1cece63740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f1ce41ae7d0 con 0x55629c58aaa0 2024-09-11T00:11:28.099 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:28.095+0000 7f1cece63740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f1ce41aba10 con 0x55629c58a180 2024-09-11T00:11:28.099 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:28.095+0000 7f1cece63740 10 client.0 _ll_register_callbacks cb 0x7f1cd40e9ee0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:11:28.099 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:28.095+0000 7f1ce3e00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55629c59ae40 0x55629c5a8b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:11:28.099 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:28.095+0000 7f1ce3400640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55629c58a180 0x55629c5abf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:11:28.099 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:28.095+0000 7f1ce3400640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55629c58a180 0x55629c5abf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.12:48136/0 (socket says 172.21.15.12:48136) 2024-09-11T00:11:28.099 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:28.095+0000 7f1ce3e00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55629c59ae40 0x55629c5a8b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:44076/0 (socket says 172.21.15.12:44076) 2024-09-11T00:11:28.099 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:28.095+0000 7f1ce3400640 1 -- 172.21.15.12:0/2599363864 learned_addr learned my addr 172.21.15.12:0/2599363864 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:11:28.099 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:28.095+0000 7f1cdaa00640 1 --2- 172.21.15.12:0/2599363864 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55629c58aaa0 0x55629c5a8460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:11:28.099 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:28.096+0000 7f1ce3400640 1 -- 172.21.15.12:0/2599363864 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55629c58aaa0 msgr2=0x55629c5a8460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:11:28.099 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:28.096+0000 7f1ce3400640 1 --2- 172.21.15.12:0/2599363864 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55629c58aaa0 0x55629c5a8460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:11:28.100 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:28.096+0000 7f1ce3400640 1 -- 172.21.15.12:0/2599363864 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55629c59ae40 msgr2=0x55629c5a8b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:11:28.100 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:28.096+0000 7f1ce3400640 1 --2- 172.21.15.12:0/2599363864 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55629c59ae40 0x55629c5a8b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:11:28.100 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:28.096+0000 7f1ce3400640 1 -- 172.21.15.12:0/2599363864 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55629c59a700 con 0x55629c58a180 2024-09-11T00:11:28.100 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:28.096+0000 7f1ce3400640 1 --2- 172.21.15.12:0/2599363864 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55629c58a180 0x55629c5abf80 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f1cd4000c00 tx=0x7f1cd40abb80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=996466fb37bae016 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:11:28.100 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:28.096+0000 7f1cd0c00640 10 client.0 ms_handle_connect on v2:172.21.15.178:3300/0 2024-09-11T00:11:28.100 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:28.096+0000 7f1cd0c00640 1 -- 172.21.15.12:0/2599363864 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cd40a8240 con 0x55629c58a180 2024-09-11T00:11:28.100 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:28.096+0000 7f1cece63740 1 -- 172.21.15.12:0/2599363864 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1cd40ac200 con 0x55629c58a180 2024-09-11T00:11:28.100 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:28.096+0000 7f1cd0c00640 1 -- 172.21.15.12:0/2599363864 <== mon.1 v2:172.21.15.178:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9208+0+0 (secure 0 0 0) 0x7f1cd40b22b0 con 0x55629c58a180 2024-09-11T00:11:28.100 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:28.096+0000 7f1cd0c00640 1 -- 172.21.15.12:0/2599363864 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f1cd40b5bd0 con 0x55629c58a180 2024-09-11T00:11:28.100 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:28.096+0000 7f1cece63740 10 client.26050 fetch_fsmap learned FSMap version 37 2024-09-11T00:11:28.100 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:28.096+0000 7f1cece63740 1 -- 172.21.15.12:0/2599363864 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55629c61cac0 con 0x55629c58a180 2024-09-11T00:11:28.100 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:28.097+0000 7f1cd0c00640 1 -- 172.21.15.12:0/2599363864 <== mon.1 v2:172.21.15.178:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1cd40ac200 con 0x55629c58a180 2024-09-11T00:11:28.100 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:28.097+0000 7f1cd0c00640 1 -- 172.21.15.12:0/2599363864 <== mon.1 v2:172.21.15.178:3300/0 5 ==== fsmap(e 37) ==== 8370+0+0 (secure 0 0 0) 0x7f1cd4018a20 con 0x55629c58a180 2024-09-11T00:11:28.101 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:28.097+0000 7f1cece63740 10 client.26050 fetch_fsmap finished waiting for FSMap version 37 2024-09-11T00:11:28.101 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:28.097+0000 7f1cece63740 10 client.26050 resolve_mds: validated gid 24674 aka daemon mds.b 2024-09-11T00:11:28.101 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:28.097+0000 7f1cece63740 20 client.26050 populate_metadata read hostname 'smithi012' 2024-09-11T00:11:28.101 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:28.097+0000 7f1cece63740 1 --2- 172.21.15.12:0/2599363864 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55629c61d000 0x55629c61f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:11:28.101 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:28.097+0000 7f1cece63740 4 client.26050 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 150.0, "await_for": 45.0}] 2024-09-11T00:11:28.101 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:28.097+0000 7f1cece63740 1 -- 172.21.15.12:0/2599363864 --> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 150.0, "await_for": 45.0}) -- 0x55629c4fd6a0 con 0x55629c61d000 2024-09-11T00:11:28.101 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:28.097+0000 7f1cdaa00640 1 --2- 172.21.15.12:0/2599363864 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55629c61d000 0x55629c61f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:11:28.101 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:28.098+0000 7f1cdaa00640 1 --2- 172.21.15.12:0/2599363864 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55629c61d000 0x55629c61f3f0 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d1341ca323bb1732 server_cookie=8c197c6e8a0cdeb1 in_seq=0 out_seq=0 2024-09-11T00:11:28.101 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:28.098+0000 7f1cd0c00640 10 client.26050 ms_handle_connect on v2:172.21.15.165:6838/501312984 2024-09-11T00:11:28.163 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:28.161+0000 7f1cd0c00640 1 -- 172.21.15.12:0/2599363864 <== mds.0 v2:172.21.15.165:6838/501312984 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55629c4fd6a0 con 0x55629c61d000 2024-09-11T00:11:28.163 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:28.161+0000 7f1cd0c00640 10 client.26050 handle_command_reply: tid=0 2024-09-11T00:11:28.163 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:28.161+0000 7f1cece63740 1 client.26050 shutdown 2024-09-11T00:11:28.163 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:28.162+0000 7f1cece63740 1 -- 172.21.15.12:0/2599363864 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55629c58a180 msgr2=0x55629c5abf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:11:28.163 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:28.162+0000 7f1cece63740 1 --2- 172.21.15.12:0/2599363864 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55629c58a180 0x55629c5abf80 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f1cd4000c00 tx=0x7f1cd40abb80 comp rx=0 tx=0).stop 2024-09-11T00:11:28.163 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:28.162+0000 7f1cece63740 1 -- 172.21.15.12:0/2599363864 shutdown_connections 2024-09-11T00:11:28.163 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:28.162+0000 7f1cece63740 1 --2- 172.21.15.12:0/2599363864 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55629c61d000 0x55629c61f3f0 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:11:28.164 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:28.162+0000 7f1cece63740 1 --2- 172.21.15.12:0/2599363864 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55629c59ae40 0x55629c5a8b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:11:28.164 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:28.162+0000 7f1cece63740 1 --2- 172.21.15.12:0/2599363864 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55629c58aaa0 0x55629c5a8460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:11:28.164 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:28.162+0000 7f1cd0c00640 0 client.26050 ms_handle_reset on v2:172.21.15.165:6838/501312984 2024-09-11T00:11:28.164 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:28.162+0000 7f1cece63740 1 --2- 172.21.15.12:0/2599363864 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55629c58a180 0x55629c5abf80 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:11:28.164 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:28.162+0000 7f1cece63740 1 -- 172.21.15.12:0/2599363864 >> 172.21.15.12:0/2599363864 conn(0x55629c587fb0 msgr2=0x55629c589000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:11:28.164 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:28.162+0000 7f1cece63740 1 -- 172.21.15.12:0/2599363864 shutdown_connections 2024-09-11T00:11:28.164 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:28.162+0000 7f1cece63740 1 -- 172.21.15.12:0/2599363864 wait complete. 2024-09-11T00:11:28.164 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:28.162+0000 7f1cece63740 20 client.26050 trim_cache size 0 max 16384 2024-09-11T00:11:28.165 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-09-11T00:11:28.165 INFO:teuthology.orchestra.run.smithi012.stdout: "epoch": 37, 2024-09-11T00:11:28.165 INFO:teuthology.orchestra.run.smithi012.stdout: "leader": 24674, 2024-09-11T00:11:28.165 INFO:teuthology.orchestra.run.smithi012.stdout: "set_version": 18, 2024-09-11T00:11:28.165 INFO:teuthology.orchestra.run.smithi012.stdout: "sets": { 2024-09-11T00:11:28.165 INFO:teuthology.orchestra.run.smithi012.stdout: "8f1ca7f2": { 2024-09-11T00:11:28.165 INFO:teuthology.orchestra.run.smithi012.stdout: "version": 18, 2024-09-11T00:11:28.165 INFO:teuthology.orchestra.run.smithi012.stdout: "age_ref": 0.0, 2024-09-11T00:11:28.165 INFO:teuthology.orchestra.run.smithi012.stdout: "state": { 2024-09-11T00:11:28.165 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "QUIESCED", 2024-09-11T00:11:28.165 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 0.0 2024-09-11T00:11:28.165 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:11:28.166 INFO:teuthology.orchestra.run.smithi012.stdout: "timeout": 90.0, 2024-09-11T00:11:28.166 INFO:teuthology.orchestra.run.smithi012.stdout: "expiration": 150.0, 2024-09-11T00:11:28.166 INFO:teuthology.orchestra.run.smithi012.stdout: "members": { 2024-09-11T00:11:28.166 INFO:teuthology.orchestra.run.smithi012.stdout: "file:/": { 2024-09-11T00:11:28.166 INFO:teuthology.orchestra.run.smithi012.stdout: "excluded": false, 2024-09-11T00:11:28.166 INFO:teuthology.orchestra.run.smithi012.stdout: "state": { 2024-09-11T00:11:28.166 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "QUIESCED", 2024-09-11T00:11:28.166 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 0.0 2024-09-11T00:11:28.166 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:11:28.166 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:11:28.166 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:11:28.166 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:11:28.166 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:11:28.166 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-09-11T00:11:28.167 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:28.163+0000 7f1cea200640 1 -- 172.21.15.12:0/3153005508 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f1cc80b5360 msgr2=0x7f1cc80b7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:11:28.167 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:28.163+0000 7f1cea200640 1 --2- 172.21.15.12:0/3153005508 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f1cc80b5360 0x7f1cc80b7820 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f1cdc009270 tx=0x7f1cdc009680 comp rx=0 tx=0).stop 2024-09-11T00:11:28.167 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:28.163+0000 7f1cea200640 1 -- 172.21.15.12:0/3153005508 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1ce4102dc0 msgr2=0x7f1ce4199290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:11:28.167 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:28.163+0000 7f1cea200640 1 --2- 172.21.15.12:0/3153005508 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1ce4102dc0 0x7f1ce4199290 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f1cd400e9f0 tx=0x7f1cd400eeb0 comp rx=0 tx=0).stop 2024-09-11T00:11:28.167 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:28.163+0000 7f1cea200640 1 -- 172.21.15.12:0/3153005508 shutdown_connections 2024-09-11T00:11:28.167 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:28.163+0000 7f1cea200640 1 --2- 172.21.15.12:0/3153005508 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f1cc80b5360 0x7f1cc80b7820 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:11:28.167 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:28.163+0000 7f1cea200640 1 --2- 172.21.15.12:0/3153005508 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1ce4109150 0x7f1ce419eb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:11:28.168 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:28.163+0000 7f1cea200640 1 --2- 172.21.15.12:0/3153005508 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1ce41087d0 0x7f1ce41997d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:11:28.168 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:28.163+0000 7f1cea200640 1 --2- 172.21.15.12:0/3153005508 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1ce4102dc0 0x7f1ce4199290 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:11:28.168 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:28.163+0000 7f1cea200640 1 -- 172.21.15.12:0/3153005508 >> 172.21.15.12:0/3153005508 conn(0x7f1ce40fe830 msgr2=0x7f1ce4100c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:11:28.168 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:28.163+0000 7f1cea200640 1 -- 172.21.15.12:0/3153005508 shutdown_connections 2024-09-11T00:11:28.168 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:28.163+0000 7f1cea200640 1 -- 172.21.15.12:0/3153005508 wait complete. 2024-09-11T00:11:28.629 INFO:tasks.workunit.client.0.smithi012.stdout: 1 124906 1.80 MB/sec execute 85 sec latency 2203.930 ms 2024-09-11T00:11:28.959 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '8f1ca7f2', quiesce took 4.0 seconds. Will release after: 26.0 2024-09-11T00:11:29.629 INFO:tasks.workunit.client.0.smithi012.stdout: 1 124906 1.78 MB/sec execute 86 sec latency 1463.517 ms 2024-09-11T00:11:30.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:11:29 smithi165 ceph-mon[35356]: pgmap v531: 129 pgs: 129 active+clean; 31 MiB data, 689 MiB used, 1.0 TiB / 1.0 TiB avail; 132 KiB/s rd, 142 KiB/s wr, 45 op/s 2024-09-11T00:11:30.063 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:11:29 smithi012 ceph-mon[30333]: pgmap v531: 129 pgs: 129 active+clean; 31 MiB data, 689 MiB used, 1.0 TiB / 1.0 TiB avail; 132 KiB/s rd, 142 KiB/s wr, 45 op/s 2024-09-11T00:11:30.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:11:29 smithi178 ceph-mon[35919]: pgmap v531: 129 pgs: 129 active+clean; 31 MiB data, 689 MiB used, 1.0 TiB / 1.0 TiB avail; 132 KiB/s rd, 142 KiB/s wr, 45 op/s 2024-09-11T00:11:30.629 INFO:tasks.workunit.client.0.smithi012.stdout: 1 124906 1.76 MB/sec execute 87 sec latency 2463.593 ms 2024-09-11T00:11:31.631 INFO:tasks.workunit.client.0.smithi012.stdout: 1 124906 1.74 MB/sec execute 88 sec latency 3463.732 ms 2024-09-11T00:11:32.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:11:31 smithi165 ceph-mon[35356]: pgmap v532: 129 pgs: 129 active+clean; 32 MiB data, 692 MiB used, 1.0 TiB / 1.0 TiB avail; 188 KiB/s rd, 194 KiB/s wr, 55 op/s 2024-09-11T00:11:32.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:11:31 smithi165 ceph-mon[35356]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T00:11:32.063 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:11:31 smithi012 ceph-mon[30333]: pgmap v532: 129 pgs: 129 active+clean; 32 MiB data, 692 MiB used, 1.0 TiB / 1.0 TiB avail; 188 KiB/s rd, 194 KiB/s wr, 55 op/s 2024-09-11T00:11:32.063 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:11:31 smithi012 ceph-mon[30333]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T00:11:32.080 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:11:31 smithi178 ceph-mon[35919]: pgmap v532: 129 pgs: 129 active+clean; 32 MiB data, 692 MiB used, 1.0 TiB / 1.0 TiB avail; 188 KiB/s rd, 194 KiB/s wr, 55 op/s 2024-09-11T00:11:32.080 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:11:31 smithi178 ceph-mon[35919]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T00:11:32.630 INFO:tasks.workunit.client.0.smithi012.stdout: 1 124906 1.72 MB/sec execute 89 sec latency 4463.891 ms 2024-09-11T00:11:33.630 INFO:tasks.workunit.client.0.smithi012.stdout: 1 124906 1.70 MB/sec execute 90 sec latency 5464.024 ms 2024-09-11T00:11:34.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:11:33 smithi165 ceph-mon[35356]: pgmap v533: 129 pgs: 129 active+clean; 32 MiB data, 692 MiB used, 1.0 TiB / 1.0 TiB avail; 222 KiB/s rd, 207 KiB/s wr, 59 op/s 2024-09-11T00:11:34.063 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:11:33 smithi012 ceph-mon[30333]: pgmap v533: 129 pgs: 129 active+clean; 32 MiB data, 692 MiB used, 1.0 TiB / 1.0 TiB avail; 222 KiB/s rd, 207 KiB/s wr, 59 op/s 2024-09-11T00:11:34.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:11:33 smithi178 ceph-mon[35919]: pgmap v533: 129 pgs: 129 active+clean; 32 MiB data, 692 MiB used, 1.0 TiB / 1.0 TiB avail; 222 KiB/s rd, 207 KiB/s wr, 59 op/s 2024-09-11T00:11:34.630 INFO:tasks.workunit.client.0.smithi012.stdout: 1 124906 1.68 MB/sec execute 91 sec latency 6464.075 ms 2024-09-11T00:11:35.630 INFO:tasks.workunit.client.0.smithi012.stdout: 1 124906 1.66 MB/sec execute 92 sec latency 7464.197 ms 2024-09-11T00:11:36.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:11:35 smithi165 ceph-mon[35356]: pgmap v534: 129 pgs: 129 active+clean; 32 MiB data, 692 MiB used, 1.0 TiB / 1.0 TiB avail; 219 KiB/s rd, 206 KiB/s wr, 56 op/s 2024-09-11T00:11:36.063 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:11:35 smithi012 ceph-mon[30333]: pgmap v534: 129 pgs: 129 active+clean; 32 MiB data, 692 MiB used, 1.0 TiB / 1.0 TiB avail; 219 KiB/s rd, 206 KiB/s wr, 56 op/s 2024-09-11T00:11:36.080 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:11:35 smithi178 ceph-mon[35919]: pgmap v534: 129 pgs: 129 active+clean; 32 MiB data, 692 MiB used, 1.0 TiB / 1.0 TiB avail; 219 KiB/s rd, 206 KiB/s wr, 56 op/s 2024-09-11T00:11:36.406 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:36.404+0000 7f29e7200640 1 -- 172.21.15.12:0/498441063 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f29e0104ba0 msgr2=0x7f29e0104fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:11:36.406 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:36.404+0000 7f29e7200640 1 --2- 172.21.15.12:0/498441063 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f29e0104ba0 0x7f29e0104fa0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f29d0009cd0 tx=0x7f29d002a8e0 comp rx=0 tx=0).stop 2024-09-11T00:11:36.406 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:36.405+0000 7f29e7200640 1 -- 172.21.15.12:0/498441063 shutdown_connections 2024-09-11T00:11:36.406 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:36.405+0000 7f29e7200640 1 --2- 172.21.15.12:0/498441063 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f29e01066b0 0x7f29e010d030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:11:36.406 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:36.405+0000 7f29e7200640 1 --2- 172.21.15.12:0/498441063 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f29e0105cf0 0x7f29e0106170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:11:36.406 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:36.405+0000 7f29e7200640 1 --2- 172.21.15.12:0/498441063 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f29e0104ba0 0x7f29e0104fa0 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:11:36.406 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:36.405+0000 7f29e7200640 1 -- 172.21.15.12:0/498441063 >> 172.21.15.12:0/498441063 conn(0x7f29e0100460 msgr2=0x7f29e0102880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:11:36.406 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:36.405+0000 7f29e7200640 1 -- 172.21.15.12:0/498441063 shutdown_connections 2024-09-11T00:11:36.407 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:36.405+0000 7f29e7200640 1 -- 172.21.15.12:0/498441063 wait complete. 2024-09-11T00:11:36.409 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:36.406+0000 7f29e7200640 1 Processor -- start 2024-09-11T00:11:36.409 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:36.406+0000 7f29e7200640 1 -- start start 2024-09-11T00:11:36.409 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:36.406+0000 7f29e7200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f29e0104ba0 0x7f29e019afd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:11:36.409 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:36.406+0000 7f29e7200640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f29e0105cf0 0x7f29e019b620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:11:36.409 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:36.406+0000 7f29e7200640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f29e01066b0 0x7f29e019c250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:11:36.409 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:36.406+0000 7f29e7200640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f29e0110840 con 0x7f29e01066b0 2024-09-11T00:11:36.409 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:36.406+0000 7f29e7200640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f29e01106c0 con 0x7f29e0104ba0 2024-09-11T00:11:36.409 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:36.406+0000 7f29e7200640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f29e01109c0 con 0x7f29e0105cf0 2024-09-11T00:11:36.409 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:36.406+0000 7f29e5200640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f29e01066b0 0x7f29e019c250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:11:36.409 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:36.406+0000 7f29d7400640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f29e0105cf0 0x7f29e019b620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:11:36.410 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:36.406+0000 7f29e5200640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f29e01066b0 0x7f29e019c250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:50840/0 (socket says 172.21.15.12:50840) 2024-09-11T00:11:36.410 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:36.406+0000 7f29e5200640 1 -- 172.21.15.12:0/2631361644 learned_addr learned my addr 172.21.15.12:0/2631361644 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:11:36.410 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:36.406+0000 7f29d7400640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f29e0105cf0 0x7f29e019b620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.12:54194/0 (socket says 172.21.15.12:54194) 2024-09-11T00:11:36.410 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:36.406+0000 7f29dfe00640 1 --2- 172.21.15.12:0/2631361644 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f29e0104ba0 0x7f29e019afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:11:36.410 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:36.407+0000 7f29d7400640 1 -- 172.21.15.12:0/2631361644 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f29e0104ba0 msgr2=0x7f29e019afd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:11:36.410 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:36.407+0000 7f29d7400640 1 --2- 172.21.15.12:0/2631361644 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f29e0104ba0 0x7f29e019afd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:11:36.410 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:36.407+0000 7f29d7400640 1 -- 172.21.15.12:0/2631361644 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f29e01066b0 msgr2=0x7f29e019c250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:11:36.410 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:36.407+0000 7f29d7400640 1 --2- 172.21.15.12:0/2631361644 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f29e01066b0 0x7f29e019c250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:11:36.410 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:36.407+0000 7f29d7400640 1 -- 172.21.15.12:0/2631361644 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29e019cb10 con 0x7f29e0105cf0 2024-09-11T00:11:36.410 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:36.407+0000 7f29e5200640 1 --2- 172.21.15.12:0/2631361644 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f29e01066b0 0x7f29e019c250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:11:36.410 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:36.407+0000 7f29d7400640 1 --2- 172.21.15.12:0/2631361644 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f29e0105cf0 0x7f29e019b620 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f29c800e9f0 tx=0x7f29c800eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:11:36.410 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:36.407+0000 7f29dd600640 1 -- 172.21.15.12:0/2631361644 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29c800ccc0 con 0x7f29e0105cf0 2024-09-11T00:11:36.410 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:36.408+0000 7f29e7200640 1 -- 172.21.15.12:0/2631361644 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f29e019ce00 con 0x7f29e0105cf0 2024-09-11T00:11:36.410 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:36.408+0000 7f29dd600640 1 -- 172.21.15.12:0/2631361644 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f29c80108e0 con 0x7f29e0105cf0 2024-09-11T00:11:36.411 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:36.408+0000 7f29dd600640 1 -- 172.21.15.12:0/2631361644 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29c8018880 con 0x7f29e0105cf0 2024-09-11T00:11:36.411 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:36.408+0000 7f29e7200640 1 -- 172.21.15.12:0/2631361644 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29e01a34d0 con 0x7f29e0105cf0 2024-09-11T00:11:36.411 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:36.411+0000 7f29ea28e740 1 Processor -- start 2024-09-11T00:11:36.412 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:36.411+0000 7f29ea28e740 1 -- start start 2024-09-11T00:11:36.413 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:36.411+0000 7f29ea28e740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fc07391790 0x55fc07393b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:11:36.413 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:36.411+0000 7f29ea28e740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fc073940c0 0x55fc073964b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:11:36.413 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:36.411+0000 7f29ea28e740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fc07396b00 0x55fc07398fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:11:36.413 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:36.411+0000 7f29ea28e740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55fc073046b0 con 0x55fc073940c0 2024-09-11T00:11:36.413 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:36.411+0000 7f29ea28e740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55fc072f3ae0 con 0x55fc07396b00 2024-09-11T00:11:36.413 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:36.411+0000 7f29ea28e740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55fc072f78f0 con 0x55fc07391790 2024-09-11T00:11:36.413 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:36.411+0000 7f29dfe00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fc073940c0 0x55fc073964b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:11:36.413 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:36.411+0000 7f29dfe00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fc073940c0 0x55fc073964b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:50854/0 (socket says 172.21.15.12:50854) 2024-09-11T00:11:36.413 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:36.411+0000 7f29dfe00640 1 -- 172.21.15.12:0/3594149071 learned_addr learned my addr 172.21.15.12:0/3594149071 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:11:36.414 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:36.411+0000 7f29e5200640 1 --2- 172.21.15.12:0/3594149071 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fc07391790 0x55fc07393b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:11:36.414 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:36.412+0000 7f29dd600640 1 -- 172.21.15.12:0/2631361644 <== mon.1 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7f29c8010430 con 0x7f29e0105cf0 2024-09-11T00:11:36.414 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:36.412+0000 7f29dfe00640 1 -- 172.21.15.12:0/3594149071 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fc07391790 msgr2=0x55fc07393b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:11:36.414 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:36.412+0000 7f29dfe00640 1 --2- 172.21.15.12:0/3594149071 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fc07391790 0x55fc07393b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:11:36.414 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:36.412+0000 7f29dfe00640 1 -- 172.21.15.12:0/3594149071 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fc07396b00 msgr2=0x55fc07398fc0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-11T00:11:36.414 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:36.412+0000 7f29dfe00640 1 --2- 172.21.15.12:0/3594149071 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fc07396b00 0x55fc07398fc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:11:36.414 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:36.412+0000 7f29dfe00640 1 -- 172.21.15.12:0/3594149071 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fc072c0780 con 0x55fc073940c0 2024-09-11T00:11:36.414 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:36.412+0000 7f29dfe00640 1 --2- 172.21.15.12:0/3594149071 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fc073940c0 0x55fc073964b0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f29d0009ca0 tx=0x7f29d0009a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be7fee1c87d67bfb server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:11:36.416 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:36.412+0000 7f29d5600640 1 -- 172.21.15.12:0/3594149071 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29d0048070 con 0x55fc073940c0 2024-09-11T00:11:36.416 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:36.412+0000 7f29d5600640 1 -- 172.21.15.12:0/3594149071 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f29d0036070 con 0x55fc073940c0 2024-09-11T00:11:36.417 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:36.412+0000 7f29d5600640 1 -- 172.21.15.12:0/3594149071 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29d0004500 con 0x55fc073940c0 2024-09-11T00:11:36.417 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:36.413+0000 7f29ea28e740 1 -- 172.21.15.12:0/3594149071 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fc073940c0 msgr2=0x55fc073964b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:11:36.417 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:36.413+0000 7f29ea28e740 1 --2- 172.21.15.12:0/3594149071 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fc073940c0 0x55fc073964b0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f29d0009ca0 tx=0x7f29d0009a20 comp rx=0 tx=0).stop 2024-09-11T00:11:36.417 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:36.413+0000 7f29dd600640 1 --2- 172.21.15.12:0/2631361644 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f29c40b52e0 0x7f29c40b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:11:36.417 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:36.413+0000 7f29ea28e740 1 -- 172.21.15.12:0/3594149071 shutdown_connections 2024-09-11T00:11:36.417 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:36.413+0000 7f29ea28e740 1 --2- 172.21.15.12:0/3594149071 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fc07396b00 0x55fc07398fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:11:36.417 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:36.413+0000 7f29ea28e740 1 --2- 172.21.15.12:0/3594149071 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fc073940c0 0x55fc073964b0 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:11:36.417 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:36.413+0000 7f29ea28e740 1 --2- 172.21.15.12:0/3594149071 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fc07391790 0x55fc07393b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:11:36.417 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:36.413+0000 7f29dd600640 1 -- 172.21.15.12:0/2631361644 <== mon.1 v2:172.21.15.178:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9208+0+0 (secure 0 0 0) 0x7f29c80e62c0 con 0x7f29e0105cf0 2024-09-11T00:11:36.417 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:36.413+0000 7f29ea28e740 1 -- 172.21.15.12:0/3594149071 >> 172.21.15.12:0/3594149071 conn(0x55fc0738c8e0 msgr2=0x55fc0738d670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:11:36.417 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:36.413+0000 7f29e5200640 1 --2- 172.21.15.12:0/2631361644 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f29c40b52e0 0x7f29c40b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:11:36.417 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:36.413+0000 7f29ea28e740 1 -- 172.21.15.12:0/3594149071 shutdown_connections 2024-09-11T00:11:36.417 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:36.414+0000 7f29ea28e740 1 -- 172.21.15.12:0/3594149071 wait complete. 2024-09-11T00:11:36.418 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:36.414+0000 7f29ea28e740 1 -- start start 2024-09-11T00:11:36.418 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:36.414+0000 7f29ea28e740 1 Processor -- start 2024-09-11T00:11:36.418 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:36.414+0000 7f29e5200640 1 --2- 172.21.15.12:0/2631361644 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f29c40b52e0 0x7f29c40b77a0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f29d800c010 tx=0x7f29d800b420 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:11:36.418 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:36.415+0000 7f29ea28e740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fc07391790 0x55fc0739af20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:11:36.418 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:36.415+0000 7f29ea28e740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fc073940c0 0x55fc0739b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:11:36.418 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:36.415+0000 7f29ea28e740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fc07396b00 0x55fc0739c120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:11:36.418 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:36.415+0000 7f29ea28e740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55fc0722bab0 con 0x55fc073940c0 2024-09-11T00:11:36.418 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:36.415+0000 7f29ea28e740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55fc0729a570 con 0x55fc07396b00 2024-09-11T00:11:36.418 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:36.415+0000 7f29ea28e740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55fc07399a80 con 0x55fc07391790 2024-09-11T00:11:36.418 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:36.415+0000 7f29ea28e740 10 client.0 _ll_register_callbacks cb 0x55fc072a9100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:11:36.418 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:36.415+0000 7f29d7400640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fc073940c0 0x55fc0739b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:11:36.418 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:36.415+0000 7f29dfe00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fc07391790 0x55fc0739af20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:11:36.419 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:36.415+0000 7f29e5200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fc07396b00 0x55fc0739c120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:11:36.419 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:36.415+0000 7f29dfe00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fc07391790 0x55fc0739af20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.12:54214/0 (socket says 172.21.15.12:54214) 2024-09-11T00:11:36.419 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:36.415+0000 7f29dfe00640 1 -- 172.21.15.12:0/3313821514 learned_addr learned my addr 172.21.15.12:0/3313821514 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:11:36.419 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:36.415+0000 7f29dfe00640 1 -- 172.21.15.12:0/3313821514 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fc07396b00 msgr2=0x55fc0739c120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:11:36.419 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:36.415+0000 7f29dfe00640 1 --2- 172.21.15.12:0/3313821514 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fc07396b00 0x55fc0739c120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:11:36.419 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:36.415+0000 7f29dfe00640 1 -- 172.21.15.12:0/3313821514 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fc073940c0 msgr2=0x55fc0739b5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:11:36.419 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:36.415+0000 7f29dfe00640 1 --2- 172.21.15.12:0/3313821514 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fc073940c0 0x55fc0739b5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:11:36.419 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:36.415+0000 7f29dfe00640 1 -- 172.21.15.12:0/3313821514 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fc07055790 con 0x55fc07391790 2024-09-11T00:11:36.419 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:36.416+0000 7f29dfe00640 1 --2- 172.21.15.12:0/3313821514 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fc07391790 0x55fc0739af20 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f29d0006010 tx=0x7f29d00095c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=df365373526baf56 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:11:36.419 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:36.416+0000 7f29ce000640 10 client.0 ms_handle_connect on v2:172.21.15.178:3300/0 2024-09-11T00:11:36.421 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:36.416+0000 7f29ce000640 1 -- 172.21.15.12:0/3313821514 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29d0048050 con 0x55fc07391790 2024-09-11T00:11:36.421 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:36.416+0000 7f29ce000640 1 -- 172.21.15.12:0/3313821514 <== mon.1 v2:172.21.15.178:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9208+0+0 (secure 0 0 0) 0x7f29d0003b80 con 0x55fc07391790 2024-09-11T00:11:36.422 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:36.416+0000 7f29ea28e740 1 -- 172.21.15.12:0/3313821514 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f29d0036070 con 0x55fc07391790 2024-09-11T00:11:36.422 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:36.416+0000 7f29ce000640 1 -- 172.21.15.12:0/3313821514 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f29d005c050 con 0x55fc07391790 2024-09-11T00:11:36.422 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:36.416+0000 7f29ea28e740 10 client.26062 fetch_fsmap learned FSMap version 38 2024-09-11T00:11:36.422 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:36.416+0000 7f29ea28e740 1 -- 172.21.15.12:0/3313821514 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x55fc070ce620 con 0x55fc07391790 2024-09-11T00:11:36.422 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:36.417+0000 7f29ce000640 1 -- 172.21.15.12:0/3313821514 <== mon.1 v2:172.21.15.178:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f29d0036070 con 0x55fc07391790 2024-09-11T00:11:36.422 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:36.417+0000 7f29ce000640 1 -- 172.21.15.12:0/3313821514 <== mon.1 v2:172.21.15.178:3300/0 5 ==== fsmap(e 38) ==== 8374+0+0 (secure 0 0 0) 0x7f29d0057030 con 0x55fc07391790 2024-09-11T00:11:36.422 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:36.417+0000 7f29ea28e740 10 client.26062 fetch_fsmap finished waiting for FSMap version 38 2024-09-11T00:11:36.422 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:36.417+0000 7f29ea28e740 10 client.26062 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T00:11:36.422 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:36.417+0000 7f29ea28e740 20 client.26062 populate_metadata read hostname 'smithi012' 2024-09-11T00:11:36.423 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:36.417+0000 7f29ea28e740 1 --2- 172.21.15.12:0/3313821514 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55fc073a63a0 0x55fc073a8790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:11:36.423 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:36.417+0000 7f29ea28e740 4 client.26062 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T00:11:36.423 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:36.417+0000 7f29ea28e740 1 -- 172.21.15.12:0/3313821514 --> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fc07306600 con 0x55fc073a63a0 2024-09-11T00:11:36.423 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:36.418+0000 7f29d7400640 1 --2- 172.21.15.12:0/3313821514 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55fc073a63a0 0x55fc073a8790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:11:36.423 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:36.418+0000 7f29d7400640 1 --2- 172.21.15.12:0/3313821514 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55fc073a63a0 0x55fc073a8790 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d8cc8c57d6f5084f server_cookie=80913b93fb7a028e in_seq=0 out_seq=0 2024-09-11T00:11:36.423 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:36.418+0000 7f29ce000640 10 client.26062 ms_handle_connect on v2:172.21.15.165:6838/501312984 2024-09-11T00:11:36.423 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:36.419+0000 7f29ce000640 1 -- 172.21.15.12:0/3313821514 <== mds.0 v2:172.21.15.165:6838/501312984 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55fc07306600 con 0x55fc073a63a0 2024-09-11T00:11:36.423 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:36.419+0000 7f29ce000640 10 client.26062 handle_command_reply: tid=0 2024-09-11T00:11:36.423 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:36.419+0000 7f29ea28e740 1 client.26062 shutdown 2024-09-11T00:11:36.423 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:36.419+0000 7f29ea28e740 1 -- 172.21.15.12:0/3313821514 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fc07391790 msgr2=0x55fc0739af20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:11:36.423 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:36.419+0000 7f29ea28e740 1 --2- 172.21.15.12:0/3313821514 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fc07391790 0x55fc0739af20 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f29d0006010 tx=0x7f29d00095c0 comp rx=0 tx=0).stop 2024-09-11T00:11:36.423 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:36.419+0000 7f29ea28e740 1 -- 172.21.15.12:0/3313821514 shutdown_connections 2024-09-11T00:11:36.423 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:36.419+0000 7f29ea28e740 1 --2- 172.21.15.12:0/3313821514 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55fc073a63a0 0x55fc073a8790 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:11:36.424 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:36.419+0000 7f29ea28e740 1 --2- 172.21.15.12:0/3313821514 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fc07396b00 0x55fc0739c120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:11:36.424 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:36.419+0000 7f29ea28e740 1 --2- 172.21.15.12:0/3313821514 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fc073940c0 0x55fc0739b5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:11:36.424 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:36.419+0000 7f29ea28e740 1 --2- 172.21.15.12:0/3313821514 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fc07391790 0x55fc0739af20 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:11:36.424 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:36.419+0000 7f29ce000640 0 client.26062 ms_handle_reset on v2:172.21.15.165:6838/501312984 2024-09-11T00:11:36.424 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:36.419+0000 7f29ea28e740 1 -- 172.21.15.12:0/3313821514 >> 172.21.15.12:0/3313821514 conn(0x55fc0738c8e0 msgr2=0x55fc0738ba60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:11:36.424 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:36.419+0000 7f29ea28e740 1 -- 172.21.15.12:0/3313821514 shutdown_connections 2024-09-11T00:11:36.424 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:36.419+0000 7f29ea28e740 1 -- 172.21.15.12:0/3313821514 wait complete. 2024-09-11T00:11:36.424 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:36.419+0000 7f29ea28e740 20 client.26062 trim_cache size 0 max 16384 2024-09-11T00:11:36.437 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:36.435+0000 7f29ea28e740 1 Processor -- start 2024-09-11T00:11:36.437 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:36.435+0000 7f29ea28e740 1 -- start start 2024-09-11T00:11:36.437 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:36.435+0000 7f29ea28e740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fc07391790 0x55fc0739ac90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:11:36.437 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:36.435+0000 7f29ea28e740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fc073940c0 0x55fc073a31a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:11:36.437 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:36.435+0000 7f29ea28e740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fc07396b00 0x55fc07396f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:11:36.437 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:36.435+0000 7f29ea28e740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f29e01ad8c0 con 0x55fc073940c0 2024-09-11T00:11:36.437 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:36.435+0000 7f29ea28e740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f29e01af430 con 0x55fc07396b00 2024-09-11T00:11:36.437 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:36.435+0000 7f29ea28e740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f29e01adf90 con 0x55fc07391790 2024-09-11T00:11:36.437 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:36.436+0000 7f29d7400640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fc073940c0 0x55fc073a31a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:11:36.438 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:36.436+0000 7f29d7400640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fc073940c0 0x55fc073a31a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:50868/0 (socket says 172.21.15.12:50868) 2024-09-11T00:11:36.438 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:36.436+0000 7f29d7400640 1 -- 172.21.15.12:0/2566230821 learned_addr learned my addr 172.21.15.12:0/2566230821 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:11:36.438 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:36.436+0000 7f29e5200640 1 --2- 172.21.15.12:0/2566230821 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fc07396b00 0x55fc07396f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:11:36.438 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:36.436+0000 7f29dfe00640 1 --2- 172.21.15.12:0/2566230821 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fc07391790 0x55fc0739ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:11:36.438 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:36.436+0000 7f29d7400640 1 -- 172.21.15.12:0/2566230821 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fc07391790 msgr2=0x55fc0739ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:11:36.438 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:36.436+0000 7f29d7400640 1 --2- 172.21.15.12:0/2566230821 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fc07391790 0x55fc0739ac90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:11:36.438 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:36.436+0000 7f29d7400640 1 -- 172.21.15.12:0/2566230821 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fc07396b00 msgr2=0x55fc07396f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:11:36.438 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:36.436+0000 7f29d7400640 1 --2- 172.21.15.12:0/2566230821 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fc07396b00 0x55fc07396f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:11:36.438 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:36.436+0000 7f29d7400640 1 -- 172.21.15.12:0/2566230821 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fc073a8b00 con 0x55fc073940c0 2024-09-11T00:11:36.438 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:36.436+0000 7f29d7400640 1 --2- 172.21.15.12:0/2566230821 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fc073940c0 0x55fc073a31a0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f29c8005ba0 tx=0x7f29c80e38c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74fb357edcff33f3 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:11:36.442 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:36.436+0000 7f29d4c00640 1 -- 172.21.15.12:0/2566230821 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29c80a6e60 con 0x55fc073940c0 2024-09-11T00:11:36.442 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:36.436+0000 7f29d4c00640 1 -- 172.21.15.12:0/2566230821 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f29c80b5c40 con 0x55fc073940c0 2024-09-11T00:11:36.442 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:36.437+0000 7f29d4c00640 1 -- 172.21.15.12:0/2566230821 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29c80a89d0 con 0x55fc073940c0 2024-09-11T00:11:36.442 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:36.437+0000 7f29ea28e740 1 -- 172.21.15.12:0/2566230821 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fc073940c0 msgr2=0x55fc073a31a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:11:36.442 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:36.437+0000 7f29ea28e740 1 --2- 172.21.15.12:0/2566230821 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fc073940c0 0x55fc073a31a0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f29c8005ba0 tx=0x7f29c80e38c0 comp rx=0 tx=0).stop 2024-09-11T00:11:36.442 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:36.437+0000 7f29ea28e740 1 -- 172.21.15.12:0/2566230821 shutdown_connections 2024-09-11T00:11:36.442 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:36.437+0000 7f29ea28e740 1 --2- 172.21.15.12:0/2566230821 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fc07396b00 0x55fc07396f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:11:36.442 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:36.437+0000 7f29ea28e740 1 --2- 172.21.15.12:0/2566230821 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fc073940c0 0x55fc073a31a0 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:11:36.442 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:36.437+0000 7f29ea28e740 1 --2- 172.21.15.12:0/2566230821 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fc07391790 0x55fc0739ac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:11:36.442 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:36.437+0000 7f29ea28e740 1 -- 172.21.15.12:0/2566230821 >> 172.21.15.12:0/2566230821 conn(0x55fc0738c8e0 msgr2=0x55fc0738d380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:11:36.443 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:36.437+0000 7f29ea28e740 1 -- 172.21.15.12:0/2566230821 shutdown_connections 2024-09-11T00:11:36.443 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:36.437+0000 7f29ea28e740 1 -- 172.21.15.12:0/2566230821 wait complete. 2024-09-11T00:11:36.443 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:36.437+0000 7f29ea28e740 1 -- start start 2024-09-11T00:11:36.443 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:36.438+0000 7f29ea28e740 1 Processor -- start 2024-09-11T00:11:36.443 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:36.438+0000 7f29ea28e740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fc07391790 0x55fc073a23b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:11:36.443 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:36.438+0000 7f29ea28e740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fc073940c0 0x55fc073a2a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:11:36.443 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:36.438+0000 7f29ea28e740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fc07396b00 0x55fc073a0f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:11:36.443 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:36.438+0000 7f29ea28e740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f29e01aed60 con 0x55fc073940c0 2024-09-11T00:11:36.443 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:36.438+0000 7f29ea28e740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f29e01ae690 con 0x55fc07396b00 2024-09-11T00:11:36.443 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:36.438+0000 7f29ea28e740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f29e01ab8a0 con 0x55fc07391790 2024-09-11T00:11:36.443 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:36.438+0000 7f29ea28e740 10 client.0 _ll_register_callbacks cb 0x7f29d0046070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:11:36.444 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:36.438+0000 7f29d7400640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fc073940c0 0x55fc073a2a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:11:36.444 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:36.438+0000 7f29d7400640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fc073940c0 0x55fc073a2a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:50870/0 (socket says 172.21.15.12:50870) 2024-09-11T00:11:36.444 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:36.438+0000 7f29d7400640 1 -- 172.21.15.12:0/3902662830 learned_addr learned my addr 172.21.15.12:0/3902662830 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:11:36.444 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:36.438+0000 7f29e5200640 1 --2- 172.21.15.12:0/3902662830 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fc07396b00 0x55fc073a0f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:11:36.444 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:36.438+0000 7f29dfe00640 1 --2- 172.21.15.12:0/3902662830 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fc07391790 0x55fc073a23b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:11:36.444 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:36.439+0000 7f29d7400640 1 -- 172.21.15.12:0/3902662830 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fc07391790 msgr2=0x55fc073a23b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:11:36.444 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:36.439+0000 7f29d7400640 1 --2- 172.21.15.12:0/3902662830 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fc07391790 0x55fc073a23b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:11:36.444 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:36.439+0000 7f29d7400640 1 -- 172.21.15.12:0/3902662830 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fc07396b00 msgr2=0x55fc073a0f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:11:36.444 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:36.439+0000 7f29d7400640 1 --2- 172.21.15.12:0/3902662830 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fc07396b00 0x55fc073a0f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:11:36.444 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:36.439+0000 7f29d7400640 1 -- 172.21.15.12:0/3902662830 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fc073a1490 con 0x55fc073940c0 2024-09-11T00:11:36.444 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:36.439+0000 7f29d7400640 1 --2- 172.21.15.12:0/3902662830 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fc073940c0 0x55fc073a2a10 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f29c80108b0 tx=0x7f29c8009700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94784b94f9757be8 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:11:36.444 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:36.439+0000 7f29ce000640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-09-11T00:11:36.444 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:36.439+0000 7f29ce000640 1 -- 172.21.15.12:0/3902662830 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29c80a6e60 con 0x55fc073940c0 2024-09-11T00:11:36.445 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:36.439+0000 7f29ce000640 1 -- 172.21.15.12:0/3902662830 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9208+0+0 (secure 0 0 0) 0x7f29c80a89d0 con 0x55fc073940c0 2024-09-11T00:11:36.445 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:36.439+0000 7f29ea28e740 1 -- 172.21.15.12:0/3902662830 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f29c80b5c40 con 0x55fc073940c0 2024-09-11T00:11:36.445 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:36.439+0000 7f29ce000640 1 -- 172.21.15.12:0/3902662830 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f29c80ac270 con 0x55fc073940c0 2024-09-11T00:11:36.445 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:36.439+0000 7f29ea28e740 10 client.16059 fetch_fsmap learned FSMap version 38 2024-09-11T00:11:36.445 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:36.439+0000 7f29ea28e740 1 -- 172.21.15.12:0/3902662830 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x55fc07413b20 con 0x55fc073940c0 2024-09-11T00:11:36.445 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:36.440+0000 7f29ce000640 1 -- 172.21.15.12:0/3902662830 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f29c80b5c40 con 0x55fc073940c0 2024-09-11T00:11:36.445 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:36.440+0000 7f29ce000640 1 -- 172.21.15.12:0/3902662830 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 38) ==== 8374+0+0 (secure 0 0 0) 0x7f29c80ab4c0 con 0x55fc073940c0 2024-09-11T00:11:36.445 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:36.440+0000 7f29ea28e740 10 client.16059 fetch_fsmap finished waiting for FSMap version 38 2024-09-11T00:11:36.445 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:36.440+0000 7f29ea28e740 10 client.16059 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T00:11:36.445 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:36.440+0000 7f29ea28e740 20 client.16059 populate_metadata read hostname 'smithi012' 2024-09-11T00:11:36.445 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:36.440+0000 7f29ea28e740 1 --2- 172.21.15.12:0/3902662830 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55fc074144d0 0x55fc074168f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:11:36.445 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:36.440+0000 7f29ea28e740 4 client.16059 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T00:11:36.446 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:36.440+0000 7f29ea28e740 1 -- 172.21.15.12:0/3902662830 --> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55fc073020b0 con 0x55fc074144d0 2024-09-11T00:11:36.446 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:36.440+0000 7f29dfe00640 1 --2- 172.21.15.12:0/3902662830 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55fc074144d0 0x55fc074168f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:11:36.446 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:36.441+0000 7f29dfe00640 1 --2- 172.21.15.12:0/3902662830 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55fc074144d0 0x55fc074168f0 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b28d0ba749bed942 server_cookie=293b60f04dc582f7 in_seq=0 out_seq=0 2024-09-11T00:11:36.446 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:36.441+0000 7f29ce000640 10 client.16059 ms_handle_connect on v2:172.21.15.165:6838/501312984 2024-09-11T00:11:36.473 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:36.472+0000 7f29ce000640 1 -- 172.21.15.12:0/3902662830 <== mds.0 v2:172.21.15.165:6838/501312984 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55fc073020b0 con 0x55fc074144d0 2024-09-11T00:11:36.473 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:36.472+0000 7f29ce000640 10 client.16059 handle_command_reply: tid=0 2024-09-11T00:11:36.474 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:36.472+0000 7f29ea28e740 1 client.16059 shutdown 2024-09-11T00:11:36.474 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:36.472+0000 7f29ea28e740 1 -- 172.21.15.12:0/3902662830 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fc073940c0 msgr2=0x55fc073a2a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:11:36.474 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:36.472+0000 7f29ea28e740 1 --2- 172.21.15.12:0/3902662830 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fc073940c0 0x55fc073a2a10 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f29c80108b0 tx=0x7f29c8009700 comp rx=0 tx=0).stop 2024-09-11T00:11:36.474 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:36.472+0000 7f29ea28e740 1 -- 172.21.15.12:0/3902662830 shutdown_connections 2024-09-11T00:11:36.474 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:36.472+0000 7f29ea28e740 1 --2- 172.21.15.12:0/3902662830 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55fc074144d0 0x55fc074168f0 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:11:36.474 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:36.472+0000 7f29ea28e740 1 --2- 172.21.15.12:0/3902662830 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fc07396b00 0x55fc073a0f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:11:36.474 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:36.472+0000 7f29ea28e740 1 --2- 172.21.15.12:0/3902662830 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fc073940c0 0x55fc073a2a10 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:11:36.474 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:36.472+0000 7f29ce000640 0 client.16059 ms_handle_reset on v2:172.21.15.165:6838/501312984 2024-09-11T00:11:36.474 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:36.472+0000 7f29ea28e740 1 --2- 172.21.15.12:0/3902662830 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fc07391790 0x55fc073a23b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:11:36.474 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:36.472+0000 7f29ea28e740 1 -- 172.21.15.12:0/3902662830 >> 172.21.15.12:0/3902662830 conn(0x55fc0738c8e0 msgr2=0x55fc07396ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:11:36.474 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:36.472+0000 7f29ea28e740 1 -- 172.21.15.12:0/3902662830 shutdown_connections 2024-09-11T00:11:36.474 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:36.472+0000 7f29ea28e740 1 -- 172.21.15.12:0/3902662830 wait complete. 2024-09-11T00:11:36.474 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:36.472+0000 7f29ea28e740 20 client.16059 trim_cache size 0 max 16384 2024-09-11T00:11:36.475 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:{ 2024-09-11T00:11:36.475 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "message": "", 2024-09-11T00:11:36.475 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "return_code": 0 2024-09-11T00:11:36.475 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:} 2024-09-11T00:11:36.476 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:36.473+0000 7f29e7200640 1 -- 172.21.15.12:0/2631361644 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f29c40b52e0 msgr2=0x7f29c40b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:11:36.476 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:36.473+0000 7f29e7200640 1 --2- 172.21.15.12:0/2631361644 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f29c40b52e0 0x7f29c40b77a0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f29d800c010 tx=0x7f29d800b420 comp rx=0 tx=0).stop 2024-09-11T00:11:36.476 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:36.473+0000 7f29e7200640 1 -- 172.21.15.12:0/2631361644 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f29e0105cf0 msgr2=0x7f29e019b620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:11:36.476 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:36.473+0000 7f29e7200640 1 --2- 172.21.15.12:0/2631361644 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f29e0105cf0 0x7f29e019b620 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f29c800e9f0 tx=0x7f29c800eeb0 comp rx=0 tx=0).stop 2024-09-11T00:11:36.476 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:36.473+0000 7f29e7200640 1 -- 172.21.15.12:0/2631361644 shutdown_connections 2024-09-11T00:11:36.476 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:36.473+0000 7f29e7200640 1 --2- 172.21.15.12:0/2631361644 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f29c40b52e0 0x7f29c40b77a0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:11:36.476 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:36.473+0000 7f29e7200640 1 --2- 172.21.15.12:0/2631361644 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f29e01066b0 0x7f29e019c250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:11:36.476 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:36.473+0000 7f29e7200640 1 --2- 172.21.15.12:0/2631361644 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f29e0105cf0 0x7f29e019b620 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:11:36.476 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:36.473+0000 7f29e7200640 1 --2- 172.21.15.12:0/2631361644 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f29e0104ba0 0x7f29e019afd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:11:36.477 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:36.473+0000 7f29e7200640 1 -- 172.21.15.12:0/2631361644 >> 172.21.15.12:0/2631361644 conn(0x7f29e0100460 msgr2=0x7f29e01090e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:11:36.477 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:36.473+0000 7f29e7200640 1 -- 172.21.15.12:0/2631361644 shutdown_connections 2024-09-11T00:11:36.477 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:36.473+0000 7f29e7200640 1 -- 172.21.15.12:0/2631361644 wait complete. 2024-09-11T00:11:36.630 INFO:tasks.workunit.client.0.smithi012.stdout: 1 124906 1.64 MB/sec execute 93 sec latency 8464.273 ms 2024-09-11T00:11:37.630 INFO:tasks.workunit.client.0.smithi012.stdout: 1 124906 1.63 MB/sec execute 94 sec latency 9464.355 ms 2024-09-11T00:11:38.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:11:37 smithi165 ceph-mon[35356]: pgmap v535: 129 pgs: 129 active+clean; 33 MiB data, 694 MiB used, 1.0 TiB / 1.0 TiB avail; 228 KiB/s rd, 423 KiB/s wr, 66 op/s 2024-09-11T00:11:38.062 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:11:37 smithi012 ceph-mon[30333]: pgmap v535: 129 pgs: 129 active+clean; 33 MiB data, 694 MiB used, 1.0 TiB / 1.0 TiB avail; 228 KiB/s rd, 423 KiB/s wr, 66 op/s 2024-09-11T00:11:38.080 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:11:37 smithi178 ceph-mon[35919]: pgmap v535: 129 pgs: 129 active+clean; 33 MiB data, 694 MiB used, 1.0 TiB / 1.0 TiB avail; 228 KiB/s rd, 423 KiB/s wr, 66 op/s 2024-09-11T00:11:38.630 INFO:tasks.workunit.client.0.smithi012.stdout: 1 124906 1.61 MB/sec execute 95 sec latency 10464.421 ms 2024-09-11T00:11:39.630 INFO:tasks.workunit.client.0.smithi012.stdout: 1 124906 1.59 MB/sec execute 96 sec latency 11464.503 ms 2024-09-11T00:11:40.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:11:39 smithi165 ceph-mon[35356]: pgmap v536: 129 pgs: 129 active+clean; 34 MiB data, 695 MiB used, 1.0 TiB / 1.0 TiB avail; 141 KiB/s rd, 352 KiB/s wr, 30 op/s 2024-09-11T00:11:40.063 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:11:39 smithi012 ceph-mon[30333]: pgmap v536: 129 pgs: 129 active+clean; 34 MiB data, 695 MiB used, 1.0 TiB / 1.0 TiB avail; 141 KiB/s rd, 352 KiB/s wr, 30 op/s 2024-09-11T00:11:40.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:11:39 smithi178 ceph-mon[35919]: pgmap v536: 129 pgs: 129 active+clean; 34 MiB data, 695 MiB used, 1.0 TiB / 1.0 TiB avail; 141 KiB/s rd, 352 KiB/s wr, 30 op/s 2024-09-11T00:11:40.630 INFO:tasks.workunit.client.0.smithi012.stdout: 1 124906 1.58 MB/sec execute 97 sec latency 12464.655 ms 2024-09-11T00:11:41.631 INFO:tasks.workunit.client.0.smithi012.stdout: 1 124906 1.56 MB/sec execute 98 sec latency 13464.821 ms 2024-09-11T00:11:42.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:11:41 smithi165 ceph-mon[35356]: pgmap v537: 129 pgs: 129 active+clean; 34 MiB data, 696 MiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 393 KiB/s wr, 42 op/s 2024-09-11T00:11:42.063 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:11:41 smithi012 ceph-mon[30333]: pgmap v537: 129 pgs: 129 active+clean; 34 MiB data, 696 MiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 393 KiB/s wr, 42 op/s 2024-09-11T00:11:42.080 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:11:41 smithi178 ceph-mon[35919]: pgmap v537: 129 pgs: 129 active+clean; 34 MiB data, 696 MiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 393 KiB/s wr, 42 op/s 2024-09-11T00:11:42.631 INFO:tasks.workunit.client.0.smithi012.stdout: 1 124906 1.54 MB/sec execute 99 sec latency 14465.019 ms 2024-09-11T00:11:43.631 INFO:tasks.workunit.client.0.smithi012.stdout: 1 124906 1.53 MB/sec execute 100 sec latency 15465.171 ms 2024-09-11T00:11:44.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:11:43 smithi165 ceph-mon[35356]: pgmap v538: 129 pgs: 129 active+clean; 33 MiB data, 694 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 352 KiB/s wr, 39 op/s 2024-09-11T00:11:44.063 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:11:43 smithi012 ceph-mon[30333]: pgmap v538: 129 pgs: 129 active+clean; 33 MiB data, 694 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 352 KiB/s wr, 39 op/s 2024-09-11T00:11:44.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:11:43 smithi178 ceph-mon[35919]: pgmap v538: 129 pgs: 129 active+clean; 33 MiB data, 694 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 352 KiB/s wr, 39 op/s 2024-09-11T00:11:44.631 INFO:tasks.workunit.client.0.smithi012.stdout: 1 124906 1.51 MB/sec execute 101 sec latency 16465.264 ms 2024-09-11T00:11:45.631 INFO:tasks.workunit.client.0.smithi012.stdout: 1 124906 1.50 MB/sec execute 102 sec latency 17465.385 ms 2024-09-11T00:11:46.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:11:45 smithi165 ceph-mon[35356]: pgmap v539: 129 pgs: 129 active+clean; 33 MiB data, 693 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 339 KiB/s wr, 33 op/s 2024-09-11T00:11:46.063 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:11:45 smithi012 ceph-mon[30333]: pgmap v539: 129 pgs: 129 active+clean; 33 MiB data, 693 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 339 KiB/s wr, 33 op/s 2024-09-11T00:11:46.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:11:45 smithi178 ceph-mon[35919]: pgmap v539: 129 pgs: 129 active+clean; 33 MiB data, 693 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 339 KiB/s wr, 33 op/s 2024-09-11T00:11:46.631 INFO:tasks.workunit.client.0.smithi012.stdout: 1 124906 1.48 MB/sec execute 103 sec latency 18465.530 ms 2024-09-11T00:11:47.631 INFO:tasks.workunit.client.0.smithi012.stdout: 1 124906 1.47 MB/sec execute 104 sec latency 19465.600 ms 2024-09-11T00:11:48.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:11:47 smithi165 ceph-mon[35356]: pgmap v540: 129 pgs: 129 active+clean; 32 MiB data, 675 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 345 KiB/s wr, 41 op/s 2024-09-11T00:11:48.063 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:11:47 smithi012 ceph-mon[30333]: pgmap v540: 129 pgs: 129 active+clean; 32 MiB data, 675 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 345 KiB/s wr, 41 op/s 2024-09-11T00:11:48.080 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:11:47 smithi178 ceph-mon[35919]: pgmap v540: 129 pgs: 129 active+clean; 32 MiB data, 675 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 345 KiB/s wr, 41 op/s 2024-09-11T00:11:48.631 INFO:tasks.workunit.client.0.smithi012.stdout: 1 124906 1.46 MB/sec execute 105 sec latency 20465.725 ms 2024-09-11T00:11:49.604 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:49.602+0000 7f728fe00640 1 -- 172.21.15.12:0/3403045609 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7290105b20 msgr2=0x7f7290105fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:11:49.604 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:49.602+0000 7f728fe00640 1 --2- 172.21.15.12:0/3403045609 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7290105b20 0x7f7290105fa0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f727c009930 tx=0x7f727c02a420 comp rx=0 tx=0).stop 2024-09-11T00:11:49.604 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:49.602+0000 7f728fe00640 1 -- 172.21.15.12:0/3403045609 shutdown_connections 2024-09-11T00:11:49.604 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:49.603+0000 7f728fe00640 1 --2- 172.21.15.12:0/3403045609 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f72901064e0 0x7f729010ce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:11:49.604 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:49.603+0000 7f728fe00640 1 --2- 172.21.15.12:0/3403045609 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7290105b20 0x7f7290105fa0 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:11:49.604 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:49.603+0000 7f728fe00640 1 --2- 172.21.15.12:0/3403045609 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f72901049d0 0x7f7290104dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:11:49.604 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:49.603+0000 7f728fe00640 1 -- 172.21.15.12:0/3403045609 >> 172.21.15.12:0/3403045609 conn(0x7f7290100270 msgr2=0x7f72901026b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:11:49.605 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:49.603+0000 7f728fe00640 1 -- 172.21.15.12:0/3403045609 shutdown_connections 2024-09-11T00:11:49.605 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:49.603+0000 7f728fe00640 1 -- 172.21.15.12:0/3403045609 wait complete. 2024-09-11T00:11:49.606 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:49.604+0000 7f728fe00640 1 Processor -- start 2024-09-11T00:11:49.606 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:49.604+0000 7f728fe00640 1 -- start start 2024-09-11T00:11:49.606 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:49.604+0000 7f728fe00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f72901049d0 0x7f729019ae30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:11:49.606 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:49.604+0000 7f728fe00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7290105b20 0x7f729019b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:11:49.606 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:49.604+0000 7f728fe00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f72901064e0 0x7f729019c040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:11:49.606 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:49.604+0000 7f728fe00640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f7290110670 con 0x7f72901049d0 2024-09-11T00:11:49.606 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:49.604+0000 7f728fe00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f72901104f0 con 0x7f7290105b20 2024-09-11T00:11:49.606 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:49.604+0000 7f728fe00640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f72901107f0 con 0x7f72901064e0 2024-09-11T00:11:49.606 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:49.604+0000 7f728ea00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f72901049d0 0x7f729019ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:11:49.607 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:49.604+0000 7f728ea00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f72901049d0 0x7f729019ae30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:54218/0 (socket says 172.21.15.12:54218) 2024-09-11T00:11:49.607 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:49.604+0000 7f728ea00640 1 -- 172.21.15.12:0/1740356786 learned_addr learned my addr 172.21.15.12:0/1740356786 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:11:49.607 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:49.604+0000 7f728e000640 1 --2- 172.21.15.12:0/1740356786 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7290105b20 0x7f729019b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:11:49.607 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:49.604+0000 7f728f400640 1 --2- 172.21.15.12:0/1740356786 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f72901064e0 0x7f729019c040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:11:49.607 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:49.605+0000 7f728ea00640 1 -- 172.21.15.12:0/1740356786 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f72901064e0 msgr2=0x7f729019c040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:11:49.607 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:49.605+0000 7f728ea00640 1 --2- 172.21.15.12:0/1740356786 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f72901064e0 0x7f729019c040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:11:49.607 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:49.605+0000 7f728ea00640 1 -- 172.21.15.12:0/1740356786 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7290105b20 msgr2=0x7f729019b480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:11:49.607 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:49.605+0000 7f728ea00640 1 --2- 172.21.15.12:0/1740356786 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7290105b20 0x7f729019b480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:11:49.607 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:49.605+0000 7f728ea00640 1 -- 172.21.15.12:0/1740356786 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f729019c900 con 0x7f72901049d0 2024-09-11T00:11:49.607 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:49.605+0000 7f728ea00640 1 --2- 172.21.15.12:0/1740356786 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f72901049d0 0x7f729019ae30 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f728000e9f0 tx=0x7f728000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:11:49.608 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:49.605+0000 7f7287400640 1 -- 172.21.15.12:0/1740356786 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f728000cd80 con 0x7f72901049d0 2024-09-11T00:11:49.608 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:49.605+0000 7f7287400640 1 -- 172.21.15.12:0/1740356786 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f72800108e0 con 0x7f72901049d0 2024-09-11T00:11:49.608 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:49.605+0000 7f728fe00640 1 -- 172.21.15.12:0/1740356786 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f729019cbf0 con 0x7f72901049d0 2024-09-11T00:11:49.608 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:49.605+0000 7f7287400640 1 -- 172.21.15.12:0/1740356786 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7280018980 con 0x7f72901049d0 2024-09-11T00:11:49.608 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:49.606+0000 7f728fe00640 1 -- 172.21.15.12:0/1740356786 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f72901a32c0 con 0x7f72901049d0 2024-09-11T00:11:49.611 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:49.609+0000 7f72989dc740 1 Processor -- start 2024-09-11T00:11:49.611 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:49.609+0000 7f72989dc740 1 -- start start 2024-09-11T00:11:49.611 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:49.609+0000 7f7287400640 1 -- 172.21.15.12:0/1740356786 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7f7280010430 con 0x7f72901049d0 2024-09-11T00:11:49.611 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:49.609+0000 7f72989dc740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f7fec86970 0x55f7fec88d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:11:49.611 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:49.609+0000 7f72989dc740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f7fec892a0 0x55f7fec8b690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:11:49.611 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:49.609+0000 7f72989dc740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f7fec8bce0 0x55f7fec8e1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:11:49.611 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:49.609+0000 7f72989dc740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55f7febf96b0 con 0x55f7fec8bce0 2024-09-11T00:11:49.611 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:49.609+0000 7f72989dc740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55f7febe8ae0 con 0x55f7fec86970 2024-09-11T00:11:49.612 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:49.609+0000 7f72989dc740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55f7febec8f0 con 0x55f7fec892a0 2024-09-11T00:11:49.612 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:49.610+0000 7f728e000640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f7fec8bce0 0x55f7fec8e1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:11:49.613 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:49.610+0000 7f728f400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f7fec86970 0x55f7fec88d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:11:49.613 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:49.610+0000 7f728f400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f7fec86970 0x55f7fec88d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.12:47456/0 (socket says 172.21.15.12:47456) 2024-09-11T00:11:49.613 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:49.610+0000 7f728e000640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f7fec8bce0 0x55f7fec8e1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:54222/0 (socket says 172.21.15.12:54222) 2024-09-11T00:11:49.613 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:49.610+0000 7f728f400640 1 -- 172.21.15.12:0/3060972855 learned_addr learned my addr 172.21.15.12:0/3060972855 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:11:49.613 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:49.610+0000 7f728ea00640 1 --2- 172.21.15.12:0/3060972855 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f7fec892a0 0x55f7fec8b690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:11:49.617 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:49.610+0000 7f728e000640 1 -- 172.21.15.12:0/3060972855 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f7fec892a0 msgr2=0x55f7fec8b690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:11:49.618 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:49.610+0000 7f728e000640 1 --2- 172.21.15.12:0/3060972855 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f7fec892a0 0x55f7fec8b690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:11:49.618 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:49.610+0000 7f728e000640 1 -- 172.21.15.12:0/3060972855 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f7fec86970 msgr2=0x55f7fec88d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:11:49.618 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:49.610+0000 7f728e000640 1 --2- 172.21.15.12:0/3060972855 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f7fec86970 0x55f7fec88d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:11:49.618 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:49.610+0000 7f728e000640 1 -- 172.21.15.12:0/3060972855 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f7febb5780 con 0x55f7fec8bce0 2024-09-11T00:11:49.618 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:49.610+0000 7f728f400640 1 --2- 172.21.15.12:0/3060972855 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f7fec86970 0x55f7fec88d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:11:49.618 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:49.611+0000 7f728e000640 1 --2- 172.21.15.12:0/3060972855 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f7fec8bce0 0x55f7fec8e1a0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f727c02a940 tx=0x7f727c004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ece19d79ecd0b862 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:11:49.618 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:49.611+0000 7f727be00640 1 -- 172.21.15.12:0/3060972855 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f727c035030 con 0x55f7fec8bce0 2024-09-11T00:11:49.618 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:49.611+0000 7f727be00640 1 -- 172.21.15.12:0/3060972855 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f727c040070 con 0x55f7fec8bce0 2024-09-11T00:11:49.618 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:49.611+0000 7f727be00640 1 -- 172.21.15.12:0/3060972855 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f727c0317e0 con 0x55f7fec8bce0 2024-09-11T00:11:49.618 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:49.611+0000 7f7287400640 1 --2- 172.21.15.12:0/1740356786 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f72740b5360 0x7f72740b7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:11:49.618 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:49.611+0000 7f7287400640 1 -- 172.21.15.12:0/1740356786 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9208+0+0 (secure 0 0 0) 0x7f72800e6260 con 0x7f72901049d0 2024-09-11T00:11:49.618 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:49.611+0000 7f728f400640 1 --2- 172.21.15.12:0/1740356786 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f72740b5360 0x7f72740b7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:11:49.619 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:49.612+0000 7f72989dc740 1 -- 172.21.15.12:0/3060972855 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f7fec8bce0 msgr2=0x55f7fec8e1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:11:49.619 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:49.612+0000 7f72989dc740 1 --2- 172.21.15.12:0/3060972855 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f7fec8bce0 0x55f7fec8e1a0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f727c02a940 tx=0x7f727c004040 comp rx=0 tx=0).stop 2024-09-11T00:11:49.619 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:49.612+0000 7f72989dc740 1 -- 172.21.15.12:0/3060972855 shutdown_connections 2024-09-11T00:11:49.619 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:49.612+0000 7f72989dc740 1 --2- 172.21.15.12:0/3060972855 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f7fec8bce0 0x55f7fec8e1a0 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:11:49.619 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:49.612+0000 7f72989dc740 1 --2- 172.21.15.12:0/3060972855 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f7fec892a0 0x55f7fec8b690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:11:49.619 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:49.612+0000 7f72989dc740 1 --2- 172.21.15.12:0/3060972855 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f7fec86970 0x55f7fec88d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:11:49.619 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:49.612+0000 7f72989dc740 1 -- 172.21.15.12:0/3060972855 >> 172.21.15.12:0/3060972855 conn(0x55f7fec81ac0 msgr2=0x55f7fec82850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:11:49.619 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:49.612+0000 7f72989dc740 1 -- 172.21.15.12:0/3060972855 shutdown_connections 2024-09-11T00:11:49.619 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:49.612+0000 7f72989dc740 1 -- 172.21.15.12:0/3060972855 wait complete. 2024-09-11T00:11:49.619 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:49.612+0000 7f728f400640 1 --2- 172.21.15.12:0/1740356786 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f72740b5360 0x7f72740b7820 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f7288009fd0 tx=0x7f7288009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:11:49.619 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:49.613+0000 7f72989dc740 1 -- start start 2024-09-11T00:11:49.619 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:49.613+0000 7f72989dc740 1 Processor -- start 2024-09-11T00:11:49.620 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:49.613+0000 7f72989dc740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f7fec86970 0x55f7fec8fff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:11:49.620 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:49.613+0000 7f72989dc740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f7fec892a0 0x55f7fec90690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:11:49.620 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:49.613+0000 7f72989dc740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f7fec8bce0 0x55f7fec91260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:11:49.620 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:49.613+0000 7f72989dc740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55f7feb20ab0 con 0x55f7fec86970 2024-09-11T00:11:49.620 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:49.613+0000 7f72989dc740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55f7feb8f570 con 0x55f7fec892a0 2024-09-11T00:11:49.620 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:49.613+0000 7f72989dc740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55f7fec8ec60 con 0x55f7fec8bce0 2024-09-11T00:11:49.620 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:49.613+0000 7f72989dc740 10 client.0 _ll_register_callbacks cb 0x55f7feb9e100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:11:49.620 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:49.613+0000 7f728ea00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f7fec86970 0x55f7fec8fff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:11:49.620 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:49.614+0000 7f728ea00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f7fec86970 0x55f7fec8fff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:54226/0 (socket says 172.21.15.12:54226) 2024-09-11T00:11:49.620 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:49.614+0000 7f728ea00640 1 -- 172.21.15.12:0/141110635 learned_addr learned my addr 172.21.15.12:0/141110635 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:11:49.620 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:49.614+0000 7f728f400640 1 --2- 172.21.15.12:0/141110635 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f7fec8bce0 0x55f7fec91260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:11:49.620 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:49.614+0000 7f728e000640 1 --2- 172.21.15.12:0/141110635 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f7fec892a0 0x55f7fec90690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:11:49.621 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:49.614+0000 7f728ea00640 1 -- 172.21.15.12:0/141110635 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f7fec8bce0 msgr2=0x55f7fec91260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:11:49.621 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:49.614+0000 7f728ea00640 1 --2- 172.21.15.12:0/141110635 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f7fec8bce0 0x55f7fec91260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:11:49.621 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:49.614+0000 7f728ea00640 1 -- 172.21.15.12:0/141110635 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f7fec892a0 msgr2=0x55f7fec90690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:11:49.621 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:49.614+0000 7f728ea00640 1 --2- 172.21.15.12:0/141110635 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f7fec892a0 0x55f7fec90690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:11:49.621 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:49.614+0000 7f728ea00640 1 -- 172.21.15.12:0/141110635 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f7fe94a790 con 0x55f7fec86970 2024-09-11T00:11:49.621 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:49.614+0000 7f728ea00640 1 --2- 172.21.15.12:0/141110635 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f7fec86970 0x55f7fec8fff0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f72800a8c70 tx=0x7f72800a8cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e13e1561e103013 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:11:49.621 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:49.614+0000 7f7278c00640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-09-11T00:11:49.621 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:49.615+0000 7f7278c00640 1 -- 172.21.15.12:0/141110635 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72800abff0 con 0x55f7fec86970 2024-09-11T00:11:49.621 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:49.615+0000 7f7278c00640 1 -- 172.21.15.12:0/141110635 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9208+0+0 (secure 0 0 0) 0x7f72800e9d90 con 0x55f7fec86970 2024-09-11T00:11:49.621 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:49.615+0000 7f72989dc740 1 -- 172.21.15.12:0/141110635 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f727c040070 con 0x55f7fec86970 2024-09-11T00:11:49.621 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:49.615+0000 7f7278c00640 1 -- 172.21.15.12:0/141110635 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f72800abe10 con 0x55f7fec86970 2024-09-11T00:11:49.621 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:49.615+0000 7f72989dc740 10 client.16071 fetch_fsmap learned FSMap version 38 2024-09-11T00:11:49.622 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:49.615+0000 7f72989dc740 1 -- 172.21.15.12:0/141110635 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x55f7fe9c3620 con 0x55f7fec86970 2024-09-11T00:11:49.622 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:49.615+0000 7f7278c00640 1 -- 172.21.15.12:0/141110635 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f727c040070 con 0x55f7fec86970 2024-09-11T00:11:49.622 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:49.615+0000 7f7278c00640 1 -- 172.21.15.12:0/141110635 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 38) ==== 8374+0+0 (secure 0 0 0) 0x7f7280018b20 con 0x55f7fec86970 2024-09-11T00:11:49.622 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:49.616+0000 7f72989dc740 10 client.16071 fetch_fsmap finished waiting for FSMap version 38 2024-09-11T00:11:49.622 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:49.616+0000 7f72989dc740 10 client.16071 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T00:11:49.622 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:49.616+0000 7f72989dc740 20 client.16071 populate_metadata read hostname 'smithi012' 2024-09-11T00:11:49.622 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:49.616+0000 7f72989dc740 1 --2- 172.21.15.12:0/141110635 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55f7fec9b880 0x55f7fec9dc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:11:49.622 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:49.616+0000 7f72989dc740 4 client.16071 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T00:11:49.622 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:49.616+0000 7f72989dc740 1 -- 172.21.15.12:0/141110635 --> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f7febfb600 con 0x55f7fec9b880 2024-09-11T00:11:49.622 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:49.616+0000 7f728e000640 1 --2- 172.21.15.12:0/141110635 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55f7fec9b880 0x55f7fec9dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:11:49.622 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:49.616+0000 7f728e000640 1 --2- 172.21.15.12:0/141110635 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55f7fec9b880 0x55f7fec9dc90 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=473ca883a1c5502e server_cookie=a3e5cb8af797ac7d in_seq=0 out_seq=0 2024-09-11T00:11:49.622 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:49.616+0000 7f7278c00640 10 client.16071 ms_handle_connect on v2:172.21.15.165:6838/501312984 2024-09-11T00:11:49.623 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:49.617+0000 7f7278c00640 1 -- 172.21.15.12:0/141110635 <== mds.0 v2:172.21.15.165:6838/501312984 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55f7febfb600 con 0x55f7fec9b880 2024-09-11T00:11:49.624 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:49.617+0000 7f7278c00640 10 client.16071 handle_command_reply: tid=0 2024-09-11T00:11:49.624 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:49.617+0000 7f72989dc740 1 client.16071 shutdown 2024-09-11T00:11:49.624 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:49.617+0000 7f72989dc740 1 -- 172.21.15.12:0/141110635 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f7fec86970 msgr2=0x55f7fec8fff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:11:49.624 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:49.617+0000 7f72989dc740 1 --2- 172.21.15.12:0/141110635 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f7fec86970 0x55f7fec8fff0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f72800a8c70 tx=0x7f72800a8cf0 comp rx=0 tx=0).stop 2024-09-11T00:11:49.624 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:49.618+0000 7f72989dc740 1 -- 172.21.15.12:0/141110635 shutdown_connections 2024-09-11T00:11:49.624 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:49.618+0000 7f72989dc740 1 --2- 172.21.15.12:0/141110635 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55f7fec9b880 0x55f7fec9dc90 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:11:49.624 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:49.618+0000 7f72989dc740 1 --2- 172.21.15.12:0/141110635 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f7fec8bce0 0x55f7fec91260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:11:49.624 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:49.618+0000 7f72989dc740 1 --2- 172.21.15.12:0/141110635 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f7fec892a0 0x55f7fec90690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:11:49.624 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:49.618+0000 7f72989dc740 1 --2- 172.21.15.12:0/141110635 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f7fec86970 0x55f7fec8fff0 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:11:49.624 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:49.618+0000 7f7278c00640 0 client.16071 ms_handle_reset on v2:172.21.15.165:6838/501312984 2024-09-11T00:11:49.624 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:49.618+0000 7f72989dc740 1 -- 172.21.15.12:0/141110635 >> 172.21.15.12:0/141110635 conn(0x55f7fec81ac0 msgr2=0x55f7fec80bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:11:49.624 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:49.618+0000 7f72989dc740 1 -- 172.21.15.12:0/141110635 shutdown_connections 2024-09-11T00:11:49.625 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:49.618+0000 7f72989dc740 1 -- 172.21.15.12:0/141110635 wait complete. 2024-09-11T00:11:49.625 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:49.618+0000 7f72989dc740 20 client.16071 trim_cache size 0 max 16384 2024-09-11T00:11:49.631 INFO:tasks.workunit.client.0.smithi012.stdout: 1 124906 1.44 MB/sec execute 106 sec latency 21465.786 ms 2024-09-11T00:11:49.638 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:49.634+0000 7f72989dc740 1 Processor -- start 2024-09-11T00:11:49.638 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:49.634+0000 7f72989dc740 1 -- start start 2024-09-11T00:11:49.638 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:49.634+0000 7f72989dc740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f7fec86970 0x55f7fec8fd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:11:49.638 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:49.634+0000 7f72989dc740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f7fec892a0 0x55f7fec85e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:11:49.639 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:49.634+0000 7f72989dc740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f7fec8bce0 0x55f7fec86340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:11:49.639 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:49.634+0000 7f72989dc740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f72901ad710 con 0x55f7fec86970 2024-09-11T00:11:49.639 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:49.634+0000 7f72989dc740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f72901af280 con 0x55f7fec8bce0 2024-09-11T00:11:49.639 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:49.634+0000 7f72989dc740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f72901adde0 con 0x55f7fec892a0 2024-09-11T00:11:49.639 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:49.634+0000 7f728ea00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f7fec86970 0x55f7fec8fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:11:49.639 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:49.634+0000 7f728ea00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f7fec86970 0x55f7fec8fd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:54240/0 (socket says 172.21.15.12:54240) 2024-09-11T00:11:49.639 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:49.634+0000 7f728ea00640 1 -- 172.21.15.12:0/99415882 learned_addr learned my addr 172.21.15.12:0/99415882 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:11:49.639 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:49.634+0000 7f728e000640 1 --2- 172.21.15.12:0/99415882 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f7fec892a0 0x55f7fec85e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:11:49.639 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:49.634+0000 7f728f400640 1 --2- 172.21.15.12:0/99415882 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f7fec8bce0 0x55f7fec86340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:11:49.639 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:49.635+0000 7f728e000640 1 -- 172.21.15.12:0/99415882 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f7fec8bce0 msgr2=0x55f7fec86340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:11:49.640 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:49.635+0000 7f728e000640 1 --2- 172.21.15.12:0/99415882 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f7fec8bce0 0x55f7fec86340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:11:49.640 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:49.635+0000 7f728e000640 1 -- 172.21.15.12:0/99415882 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f7fec86970 msgr2=0x55f7fec8fd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:11:49.640 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:49.635+0000 7f728e000640 1 --2- 172.21.15.12:0/99415882 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f7fec86970 0x55f7fec8fd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:11:49.640 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:49.635+0000 7f728e000640 1 -- 172.21.15.12:0/99415882 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f7fec890d0 con 0x55f7fec892a0 2024-09-11T00:11:49.640 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:49.635+0000 7f728ea00640 1 --2- 172.21.15.12:0/99415882 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f7fec86970 0x55f7fec8fd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:11:49.640 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:49.635+0000 7f728e000640 1 --2- 172.21.15.12:0/99415882 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f7fec892a0 0x55f7fec85e00 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f727c031500 tx=0x7f727c005c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=eb9f6cbcb0bda7d0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:11:49.640 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:49.635+0000 7f727b400640 1 -- 172.21.15.12:0/99415882 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f727c035070 con 0x55f7fec892a0 2024-09-11T00:11:49.640 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:49.635+0000 7f727b400640 1 -- 172.21.15.12:0/99415882 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f727c040070 con 0x55f7fec892a0 2024-09-11T00:11:49.640 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:49.635+0000 7f727b400640 1 -- 172.21.15.12:0/99415882 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f727c031b70 con 0x55f7fec892a0 2024-09-11T00:11:49.640 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:49.636+0000 7f72989dc740 1 -- 172.21.15.12:0/99415882 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f7fec892a0 msgr2=0x55f7fec85e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:11:49.640 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:49.636+0000 7f72989dc740 1 --2- 172.21.15.12:0/99415882 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f7fec892a0 0x55f7fec85e00 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f727c031500 tx=0x7f727c005c00 comp rx=0 tx=0).stop 2024-09-11T00:11:49.640 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:49.636+0000 7f72989dc740 1 -- 172.21.15.12:0/99415882 shutdown_connections 2024-09-11T00:11:49.641 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:49.636+0000 7f72989dc740 1 --2- 172.21.15.12:0/99415882 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f7fec8bce0 0x55f7fec86340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:11:49.641 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:49.636+0000 7f72989dc740 1 --2- 172.21.15.12:0/99415882 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f7fec892a0 0x55f7fec85e00 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:11:49.641 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:49.636+0000 7f72989dc740 1 --2- 172.21.15.12:0/99415882 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f7fec86970 0x55f7fec8fd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:11:49.641 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:49.636+0000 7f72989dc740 1 -- 172.21.15.12:0/99415882 >> 172.21.15.12:0/99415882 conn(0x55f7fec81ac0 msgr2=0x55f7fec8c0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:11:49.641 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:49.636+0000 7f72989dc740 1 -- 172.21.15.12:0/99415882 shutdown_connections 2024-09-11T00:11:49.641 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:49.637+0000 7f72989dc740 1 -- 172.21.15.12:0/99415882 wait complete. 2024-09-11T00:11:49.641 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:49.637+0000 7f72989dc740 1 -- start start 2024-09-11T00:11:49.641 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:49.637+0000 7f72989dc740 1 Processor -- start 2024-09-11T00:11:49.643 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:49.637+0000 7f72989dc740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f7fec86970 0x55f7fec97a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:11:49.643 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:49.638+0000 7f72989dc740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f7fec892a0 0x55f7fec96760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:11:49.644 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:49.638+0000 7f72989dc740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f7fec8bce0 0x55f7fec96ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:11:49.644 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:49.638+0000 7f72989dc740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f72901aebb0 con 0x55f7fec892a0 2024-09-11T00:11:49.644 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:49.638+0000 7f72989dc740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f72901ae4e0 con 0x55f7fec8bce0 2024-09-11T00:11:49.644 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:49.638+0000 7f72989dc740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f72901ab6f0 con 0x55f7fec86970 2024-09-11T00:11:49.644 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:49.638+0000 7f72989dc740 10 client.0 _ll_register_callbacks cb 0x7f72800e9990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:11:49.644 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:49.638+0000 7f728ea00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f7fec86970 0x55f7fec97a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:11:49.644 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:49.638+0000 7f728e000640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f7fec892a0 0x55f7fec96760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:11:49.644 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:49.638+0000 7f728ea00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f7fec86970 0x55f7fec97a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.12:54090/0 (socket says 172.21.15.12:54090) 2024-09-11T00:11:49.644 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:49.638+0000 7f728ea00640 1 -- 172.21.15.12:0/1894209674 learned_addr learned my addr 172.21.15.12:0/1894209674 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:11:49.644 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:49.638+0000 7f728f400640 1 --2- 172.21.15.12:0/1894209674 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f7fec8bce0 0x55f7fec96ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:11:49.644 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:49.638+0000 7f728ea00640 1 -- 172.21.15.12:0/1894209674 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f7fec8bce0 msgr2=0x55f7fec96ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:11:49.644 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:49.638+0000 7f728ea00640 1 --2- 172.21.15.12:0/1894209674 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f7fec8bce0 0x55f7fec96ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:11:49.644 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:49.638+0000 7f728ea00640 1 -- 172.21.15.12:0/1894209674 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f7fec892a0 msgr2=0x55f7fec96760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:11:49.645 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:49.638+0000 7f728ea00640 1 --2- 172.21.15.12:0/1894209674 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f7fec892a0 0x55f7fec96760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:11:49.645 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:49.638+0000 7f728ea00640 1 -- 172.21.15.12:0/1894209674 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f7fec974d0 con 0x55f7fec86970 2024-09-11T00:11:49.645 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:49.639+0000 7f728ea00640 1 --2- 172.21.15.12:0/1894209674 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f7fec86970 0x55f7fec97a60 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f72800a8c20 tx=0x7f72800a8e20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=88b334fa37eb9607 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:11:49.645 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:49.639+0000 7f7278c00640 10 client.0 ms_handle_connect on v2:172.21.15.178:3300/0 2024-09-11T00:11:49.645 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:49.639+0000 7f7278c00640 1 -- 172.21.15.12:0/1894209674 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72800a8540 con 0x55f7fec86970 2024-09-11T00:11:49.645 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:49.639+0000 7f72989dc740 1 -- 172.21.15.12:0/1894209674 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f727c040070 con 0x55f7fec86970 2024-09-11T00:11:49.645 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:49.639+0000 7f7278c00640 1 -- 172.21.15.12:0/1894209674 <== mon.1 v2:172.21.15.178:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9208+0+0 (secure 0 0 0) 0x7f72800a8040 con 0x55f7fec86970 2024-09-11T00:11:49.645 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:49.639+0000 7f7278c00640 1 -- 172.21.15.12:0/1894209674 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f72800c3050 con 0x55f7fec86970 2024-09-11T00:11:49.645 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:49.639+0000 7f72989dc740 10 client.26086 fetch_fsmap learned FSMap version 38 2024-09-11T00:11:49.645 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:49.639+0000 7f72989dc740 1 -- 172.21.15.12:0/1894209674 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x55f7fed08810 con 0x55f7fec86970 2024-09-11T00:11:49.645 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:49.639+0000 7f7278c00640 1 -- 172.21.15.12:0/1894209674 <== mon.1 v2:172.21.15.178:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f727c040070 con 0x55f7fec86970 2024-09-11T00:11:49.645 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:49.640+0000 7f7278c00640 1 -- 172.21.15.12:0/1894209674 <== mon.1 v2:172.21.15.178:3300/0 5 ==== fsmap(e 38) ==== 8374+0+0 (secure 0 0 0) 0x7f72800b2260 con 0x55f7fec86970 2024-09-11T00:11:49.645 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:49.640+0000 7f72989dc740 10 client.26086 fetch_fsmap finished waiting for FSMap version 38 2024-09-11T00:11:49.646 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:49.640+0000 7f72989dc740 10 client.26086 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T00:11:49.646 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:49.640+0000 7f72989dc740 20 client.26086 populate_metadata read hostname 'smithi012' 2024-09-11T00:11:49.646 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:49.640+0000 7f72989dc740 1 --2- 172.21.15.12:0/1894209674 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55f7fed08fd0 0x55f7fed0b3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:11:49.646 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:49.640+0000 7f72989dc740 4 client.26086 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T00:11:49.646 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:49.640+0000 7f72989dc740 1 -- 172.21.15.12:0/1894209674 --> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55f7febf70b0 con 0x55f7fed08fd0 2024-09-11T00:11:49.646 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:49.640+0000 7f728e000640 1 --2- 172.21.15.12:0/1894209674 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55f7fed08fd0 0x55f7fed0b3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:11:49.646 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:49.641+0000 7f728e000640 1 --2- 172.21.15.12:0/1894209674 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55f7fed08fd0 0x55f7fed0b3c0 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=85c779fb3f170852 server_cookie=319e28bd46917579 in_seq=0 out_seq=0 2024-09-11T00:11:49.646 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:49.641+0000 7f7278c00640 10 client.26086 ms_handle_connect on v2:172.21.15.165:6838/501312984 2024-09-11T00:11:49.662 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:49.660+0000 7f7278c00640 1 -- 172.21.15.12:0/1894209674 <== mds.0 v2:172.21.15.165:6838/501312984 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55f7febf70b0 con 0x55f7fed08fd0 2024-09-11T00:11:49.662 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:49.660+0000 7f7278c00640 10 client.26086 handle_command_reply: tid=0 2024-09-11T00:11:49.662 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:49.660+0000 7f72989dc740 1 client.26086 shutdown 2024-09-11T00:11:49.662 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:49.660+0000 7f72989dc740 1 -- 172.21.15.12:0/1894209674 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f7fec86970 msgr2=0x55f7fec97a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:11:49.662 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:49.660+0000 7f72989dc740 1 --2- 172.21.15.12:0/1894209674 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f7fec86970 0x55f7fec97a60 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f72800a8c20 tx=0x7f72800a8e20 comp rx=0 tx=0).stop 2024-09-11T00:11:49.662 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:49.660+0000 7f72989dc740 1 -- 172.21.15.12:0/1894209674 shutdown_connections 2024-09-11T00:11:49.662 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:49.660+0000 7f72989dc740 1 --2- 172.21.15.12:0/1894209674 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55f7fed08fd0 0x55f7fed0b3c0 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:11:49.662 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:49.660+0000 7f72989dc740 1 --2- 172.21.15.12:0/1894209674 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f7fec8bce0 0x55f7fec96ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:11:49.662 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:49.660+0000 7f72989dc740 1 --2- 172.21.15.12:0/1894209674 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f7fec892a0 0x55f7fec96760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:11:49.662 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:49.660+0000 7f72989dc740 1 --2- 172.21.15.12:0/1894209674 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f7fec86970 0x55f7fec97a60 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:11:49.662 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:49.660+0000 7f7278c00640 0 client.26086 ms_handle_reset on v2:172.21.15.165:6838/501312984 2024-09-11T00:11:49.662 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:49.660+0000 7f72989dc740 1 -- 172.21.15.12:0/1894209674 >> 172.21.15.12:0/1894209674 conn(0x55f7fec81ac0 msgr2=0x55f7fec82320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:11:49.663 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:49.660+0000 7f72989dc740 1 -- 172.21.15.12:0/1894209674 shutdown_connections 2024-09-11T00:11:49.663 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:49.661+0000 7f72989dc740 1 -- 172.21.15.12:0/1894209674 wait complete. 2024-09-11T00:11:49.663 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:49.661+0000 7f72989dc740 20 client.26086 trim_cache size 0 max 16384 2024-09-11T00:11:49.663 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:{ 2024-09-11T00:11:49.663 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "message": "", 2024-09-11T00:11:49.663 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "return_code": 0 2024-09-11T00:11:49.663 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:} 2024-09-11T00:11:49.664 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:49.661+0000 7f728fe00640 1 -- 172.21.15.12:0/1740356786 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f72740b5360 msgr2=0x7f72740b7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:11:49.664 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:49.661+0000 7f728fe00640 1 --2- 172.21.15.12:0/1740356786 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f72740b5360 0x7f72740b7820 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f7288009fd0 tx=0x7f7288009500 comp rx=0 tx=0).stop 2024-09-11T00:11:49.664 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:49.661+0000 7f728fe00640 1 -- 172.21.15.12:0/1740356786 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f72901049d0 msgr2=0x7f729019ae30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:11:49.664 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:49.661+0000 7f728fe00640 1 --2- 172.21.15.12:0/1740356786 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f72901049d0 0x7f729019ae30 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f728000e9f0 tx=0x7f728000eeb0 comp rx=0 tx=0).stop 2024-09-11T00:11:49.664 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:49.661+0000 7f728fe00640 1 -- 172.21.15.12:0/1740356786 shutdown_connections 2024-09-11T00:11:49.664 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:49.661+0000 7f728fe00640 1 --2- 172.21.15.12:0/1740356786 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f72740b5360 0x7f72740b7820 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:11:49.664 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:49.661+0000 7f728fe00640 1 --2- 172.21.15.12:0/1740356786 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f72901064e0 0x7f729019c040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:11:49.664 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:49.661+0000 7f728fe00640 1 --2- 172.21.15.12:0/1740356786 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7290105b20 0x7f729019b480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:11:49.665 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:49.661+0000 7f728fe00640 1 --2- 172.21.15.12:0/1740356786 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f72901049d0 0x7f729019ae30 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:11:49.665 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:49.661+0000 7f728fe00640 1 -- 172.21.15.12:0/1740356786 >> 172.21.15.12:0/1740356786 conn(0x7f7290100270 msgr2=0x7f7290108eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:11:49.665 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:49.662+0000 7f728fe00640 1 -- 172.21.15.12:0/1740356786 shutdown_connections 2024-09-11T00:11:49.665 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:11:49.662+0000 7f728fe00640 1 -- 172.21.15.12:0/1740356786 wait complete. 2024-09-11T00:11:50.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:11:49 smithi165 ceph-mon[35356]: pgmap v541: 129 pgs: 129 active+clean; 32 MiB data, 675 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 128 KiB/s wr, 31 op/s 2024-09-11T00:11:50.063 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:11:49 smithi012 ceph-mon[30333]: pgmap v541: 129 pgs: 129 active+clean; 32 MiB data, 675 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 128 KiB/s wr, 31 op/s 2024-09-11T00:11:50.080 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:11:49 smithi178 ceph-mon[35919]: pgmap v541: 129 pgs: 129 active+clean; 32 MiB data, 675 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 128 KiB/s wr, 31 op/s 2024-09-11T00:11:50.632 INFO:tasks.workunit.client.0.smithi012.stdout: 1 124906 1.43 MB/sec execute 107 sec latency 22465.944 ms 2024-09-11T00:11:51.632 INFO:tasks.workunit.client.0.smithi012.stdout: 1 124906 1.41 MB/sec execute 108 sec latency 23466.044 ms 2024-09-11T00:11:52.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:11:51 smithi165 ceph-mon[35356]: pgmap v542: 129 pgs: 129 active+clean; 32 MiB data, 676 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 70 KiB/s wr, 33 op/s 2024-09-11T00:11:52.063 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:11:51 smithi012 ceph-mon[30333]: pgmap v542: 129 pgs: 129 active+clean; 32 MiB data, 676 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 70 KiB/s wr, 33 op/s 2024-09-11T00:11:52.080 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:11:51 smithi178 ceph-mon[35919]: pgmap v542: 129 pgs: 129 active+clean; 32 MiB data, 676 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 70 KiB/s wr, 33 op/s 2024-09-11T00:11:52.632 INFO:tasks.workunit.client.0.smithi012.stdout: 1 124906 1.40 MB/sec execute 109 sec latency 24466.183 ms 2024-09-11T00:11:53.632 INFO:tasks.workunit.client.0.smithi012.stdout: 1 124906 1.39 MB/sec execute 110 sec latency 25466.274 ms 2024-09-11T00:11:54.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:11:53 smithi165 ceph-mon[35356]: pgmap v543: 129 pgs: 129 active+clean; 32 MiB data, 676 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 23 KiB/s wr, 24 op/s 2024-09-11T00:11:54.063 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:11:53 smithi012 ceph-mon[30333]: pgmap v543: 129 pgs: 129 active+clean; 32 MiB data, 676 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 23 KiB/s wr, 24 op/s 2024-09-11T00:11:54.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:11:53 smithi178 ceph-mon[35919]: pgmap v543: 129 pgs: 129 active+clean; 32 MiB data, 676 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 23 KiB/s wr, 24 op/s 2024-09-11T00:11:54.632 INFO:tasks.workunit.client.0.smithi012.stdout: 1 124906 1.38 MB/sec execute 111 sec latency 26466.367 ms 2024-09-11T00:11:54.960 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid ff3ad74a-6fcf-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T00:11:55.356 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:11:55.633 INFO:tasks.workunit.client.0.smithi012.stdout: 1 124906 1.36 MB/sec execute 112 sec latency 27466.677 ms 2024-09-11T00:11:55.813 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:11:55 smithi012 ceph-mon[30333]: pgmap v544: 129 pgs: 129 active+clean; 32 MiB data, 676 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 12 KiB/s wr, 15 op/s 2024-09-11T00:11:55.949 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:55.947+0000 7f4d00400640 1 -- 172.21.15.12:0/1505466361 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4cf8075f40 msgr2=0x7f4cf810ee10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:11:55.949 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:55.947+0000 7f4d00400640 1 --2- 172.21.15.12:0/1505466361 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4cf8075f40 0x7f4cf810ee10 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f4ce4009900 tx=0x7f4ce402a3f0 comp rx=0 tx=0).stop 2024-09-11T00:11:55.949 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:55.948+0000 7f4d00400640 1 -- 172.21.15.12:0/1505466361 shutdown_connections 2024-09-11T00:11:55.950 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:55.948+0000 7f4d00400640 1 --2- 172.21.15.12:0/1505466361 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4cf810f3e0 0x7f4cf81117d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:11:55.950 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:55.948+0000 7f4d00400640 1 --2- 172.21.15.12:0/1505466361 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4cf8075f40 0x7f4cf810ee10 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:11:55.950 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:55.948+0000 7f4d00400640 1 --2- 172.21.15.12:0/1505466361 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4cf8075540 0x7f4cf8075920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:11:55.950 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:55.948+0000 7f4d00400640 1 -- 172.21.15.12:0/1505466361 >> 172.21.15.12:0/1505466361 conn(0x7f4cf80fe830 msgr2=0x7f4cf8100c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:11:55.950 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:55.948+0000 7f4d00400640 1 -- 172.21.15.12:0/1505466361 shutdown_connections 2024-09-11T00:11:55.950 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:55.948+0000 7f4d00400640 1 -- 172.21.15.12:0/1505466361 wait complete. 2024-09-11T00:11:55.951 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:55.949+0000 7f4d00400640 1 Processor -- start 2024-09-11T00:11:55.951 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:55.949+0000 7f4d00400640 1 -- start start 2024-09-11T00:11:55.951 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:55.949+0000 7f4d00400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4cf8075540 0x7f4cf819b4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:11:55.951 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:55.949+0000 7f4d00400640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4cf8075f40 0x7f4cf819ba10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:11:55.951 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:55.949+0000 7f4d00400640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4cf810f3e0 0x7f4cf81a0db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:11:55.952 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:55.949+0000 7f4d00400640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f4cf8114f80 con 0x7f4cf8075f40 2024-09-11T00:11:55.952 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:55.949+0000 7f4d00400640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f4cf8114e00 con 0x7f4cf8075540 2024-09-11T00:11:55.952 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:55.949+0000 7f4d00400640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f4cf8115100 con 0x7f4cf810f3e0 2024-09-11T00:11:55.952 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:55.950+0000 7f4cfd000640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4cf8075f40 0x7f4cf819ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:11:55.952 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:55.950+0000 7f4cfd000640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4cf8075f40 0x7f4cf819ba10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:54268/0 (socket says 172.21.15.12:54268) 2024-09-11T00:11:55.952 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:55.950+0000 7f4cfda00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4cf8075540 0x7f4cf819b4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:11:55.952 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:55.950+0000 7f4cfd000640 1 -- 172.21.15.12:0/431936235 learned_addr learned my addr 172.21.15.12:0/431936235 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:11:55.952 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:55.950+0000 7f4cfe400640 1 --2- 172.21.15.12:0/431936235 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4cf810f3e0 0x7f4cf81a0db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:11:55.952 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:55.950+0000 7f4cfd000640 1 -- 172.21.15.12:0/431936235 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4cf810f3e0 msgr2=0x7f4cf81a0db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:11:55.952 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:55.950+0000 7f4cfd000640 1 --2- 172.21.15.12:0/431936235 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4cf810f3e0 0x7f4cf81a0db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:11:55.952 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:55.950+0000 7f4cfd000640 1 -- 172.21.15.12:0/431936235 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4cf8075540 msgr2=0x7f4cf819b4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:11:55.952 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:55.950+0000 7f4cfd000640 1 --2- 172.21.15.12:0/431936235 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4cf8075540 0x7f4cf819b4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:11:55.953 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:55.950+0000 7f4cfd000640 1 -- 172.21.15.12:0/431936235 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4cf81a1580 con 0x7f4cf8075f40 2024-09-11T00:11:55.953 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:55.950+0000 7f4cfda00640 1 --2- 172.21.15.12:0/431936235 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4cf8075540 0x7f4cf819b4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:11:55.953 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:55.951+0000 7f4cfd000640 1 --2- 172.21.15.12:0/431936235 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4cf8075f40 0x7f4cf819ba10 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f4ce4005940 tx=0x7f4ce4004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:11:55.954 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:55.951+0000 7f4cf6000640 1 -- 172.21.15.12:0/431936235 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ce403f030 con 0x7f4cf8075f40 2024-09-11T00:11:55.954 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:55.951+0000 7f4cf6000640 1 -- 172.21.15.12:0/431936235 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4ce4003aa0 con 0x7f4cf8075f40 2024-09-11T00:11:55.954 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:55.951+0000 7f4d00400640 1 -- 172.21.15.12:0/431936235 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4cf81a1810 con 0x7f4cf8075f40 2024-09-11T00:11:55.954 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:55.951+0000 7f4cf6000640 1 -- 172.21.15.12:0/431936235 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ce4035070 con 0x7f4cf8075f40 2024-09-11T00:11:55.954 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:55.951+0000 7f4d00400640 1 -- 172.21.15.12:0/431936235 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4cf81a7f30 con 0x7f4cf8075f40 2024-09-11T00:11:55.954 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:55.952+0000 7f4d00400640 1 -- 172.21.15.12:0/431936235 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4cf8075920 con 0x7f4cf8075f40 2024-09-11T00:11:55.955 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:55.954+0000 7f4cf6000640 1 -- 172.21.15.12:0/431936235 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7f4ce403bce0 con 0x7f4cf8075f40 2024-09-11T00:11:55.957 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:55.956+0000 7f4cf6000640 1 --2- 172.21.15.12:0/431936235 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f4ce00b5360 0x7f4ce00b7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:11:55.957 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:55.956+0000 7f4cf6000640 1 -- 172.21.15.12:0/431936235 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9208+0+0 (secure 0 0 0) 0x7f4ce41068d0 con 0x7f4cf8075f40 2024-09-11T00:11:55.957 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:55.956+0000 7f4cfda00640 1 --2- 172.21.15.12:0/431936235 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f4ce00b5360 0x7f4ce00b7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:11:55.958 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:55.957+0000 7f4cfda00640 1 --2- 172.21.15.12:0/431936235 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f4ce00b5360 0x7f4ce00b7820 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f4cec009fc0 tx=0x7f4cec005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:11:55.961 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:55.960+0000 7f4cf6000640 1 -- 172.21.15.12:0/431936235 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f4ce40ccd10 con 0x7f4cf8075f40 2024-09-11T00:11:56.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:11:55 smithi165 ceph-mon[35356]: pgmap v544: 129 pgs: 129 active+clean; 32 MiB data, 676 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 12 KiB/s wr, 15 op/s 2024-09-11T00:11:56.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:11:55 smithi178 ceph-mon[35919]: pgmap v544: 129 pgs: 129 active+clean; 32 MiB data, 676 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 12 KiB/s wr, 15 op/s 2024-09-11T00:11:56.194 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:56.193+0000 7f4d00400640 1 -- 172.21.15.12:0/431936235 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4cf81a82a0 con 0x7f4cf8075f40 2024-09-11T00:11:56.196 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-11T00:11:56.196 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":38,"btime":"2024-09-11T00:11:30:673202+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14577,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6835/447171591","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":447171591},{"type":"v1","addr":"172.21.15.12:6835","nonce":447171591}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14589,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/901486040","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":901486040},{"type":"v1","addr":"172.21.15.12:6837","nonce":901486040}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":38,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T00:04:04.739881+0000","modified":"2024-09-11T00:11:30.581332+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24674,"mds_1":24643,"mds_2":24638,"mds_3":24637,"mds_4":24614},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14631":{"gid":14631,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.12:6841/3069919437","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6840","nonce":3069919437},{"type":"v1","addr":"172.21.15.12:6841","nonce":3069919437}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24562":{"gid":24562,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6833/1141636112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":1141636112},{"type":"v1","addr":"172.21.15.178:6833","nonce":1141636112}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24581":{"gid":24581,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6833/3111980970","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6832","nonce":3111980970},{"type":"v1","addr":"172.21.15.165:6833","nonce":3111980970}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24607":{"gid":24607,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6835/4084868287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":4084868287},{"type":"v1","addr":"172.21.15.178:6835","nonce":4084868287}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24614":{"gid":24614,"name":"e","rank":4,"incarnation":26,"state":"up:active","state_seq":21,"addr":"172.21.15.165:6835/2425395274","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":2425395274},{"type":"v1","addr":"172.21.15.165:6835","nonce":2425395274}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24637":{"gid":24637,"name":"i","rank":3,"incarnation":24,"state":"up:active","state_seq":18,"addr":"172.21.15.178:6839/70628097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6838","nonce":70628097},{"type":"v1","addr":"172.21.15.178:6839","nonce":70628097}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24638":{"gid":24638,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":18,"addr":"172.21.15.165:6837/6215088","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6836","nonce":6215088},{"type":"v1","addr":"172.21.15.165:6837","nonce":6215088}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24643":{"gid":24643,"name":"g","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.12:6839/3124093009","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6838","nonce":3124093009},{"type":"v1","addr":"172.21.15.12:6839","nonce":3124093009}]},"join_fscid":-1,"export_targets":[4],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24674":{"gid":24674,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.165:6839/501312984","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6838","nonce":501312984},{"type":"v1","addr":"172.21.15.165:6839","nonce":501312984}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25672":{"gid":25672,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6837/308601741","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":308601741},{"type":"v1","addr":"172.21.15.178:6837","nonce":308601741}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24674,"qdb_cluster":[24674,24643,24638,24637,24614]},"id":1}]} 2024-09-11T00:11:56.196 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:56.194+0000 7f4cf6000640 1 -- 172.21.15.12:0/431936235 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 38 v38) ==== 94+0+11966 (secure 0 0 0) 0x7f4ce40d1740 con 0x7f4cf8075f40 2024-09-11T00:11:56.197 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 38 2024-09-11T00:11:56.198 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:56.197+0000 7f4d00400640 1 -- 172.21.15.12:0/431936235 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f4ce00b5360 msgr2=0x7f4ce00b7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:11:56.198 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:56.197+0000 7f4d00400640 1 --2- 172.21.15.12:0/431936235 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f4ce00b5360 0x7f4ce00b7820 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f4cec009fc0 tx=0x7f4cec005f90 comp rx=0 tx=0).stop 2024-09-11T00:11:56.199 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:56.197+0000 7f4d00400640 1 -- 172.21.15.12:0/431936235 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4cf8075f40 msgr2=0x7f4cf819ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:11:56.199 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:56.197+0000 7f4d00400640 1 --2- 172.21.15.12:0/431936235 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4cf8075f40 0x7f4cf819ba10 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f4ce4005940 tx=0x7f4ce4004270 comp rx=0 tx=0).stop 2024-09-11T00:11:56.199 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:56.197+0000 7f4d00400640 1 -- 172.21.15.12:0/431936235 shutdown_connections 2024-09-11T00:11:56.199 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:56.197+0000 7f4d00400640 1 --2- 172.21.15.12:0/431936235 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f4ce00b5360 0x7f4ce00b7820 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:11:56.199 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:56.197+0000 7f4d00400640 1 --2- 172.21.15.12:0/431936235 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4cf810f3e0 0x7f4cf81a0db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:11:56.199 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:56.197+0000 7f4d00400640 1 --2- 172.21.15.12:0/431936235 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4cf8075f40 0x7f4cf819ba10 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:11:56.199 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:56.197+0000 7f4d00400640 1 --2- 172.21.15.12:0/431936235 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4cf8075540 0x7f4cf819b4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:11:56.200 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:56.197+0000 7f4d00400640 1 -- 172.21.15.12:0/431936235 >> 172.21.15.12:0/431936235 conn(0x7f4cf80fe830 msgr2=0x7f4cf80ff430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:11:56.200 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:56.197+0000 7f4d00400640 1 -- 172.21.15.12:0/431936235 shutdown_connections 2024-09-11T00:11:56.200 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:56.197+0000 7f4d00400640 1 -- 172.21.15.12:0/431936235 wait complete. 2024-09-11T00:11:56.632 INFO:tasks.workunit.client.0.smithi012.stdout: 1 124906 1.35 MB/sec execute 113 sec latency 28466.789 ms 2024-09-11T00:11:56.982 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:11:56 smithi012 ceph-mon[30333]: from='client.? 172.21.15.12:0/431936235' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T00:11:57.022 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24674 quiesce db --set-id 8f1ca7f2 --release --await' 2024-09-11T00:11:57.022 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid ff3ad74a-6fcf-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24674 quiesce db --set-id 8f1ca7f2 --release --await 2024-09-11T00:11:57.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:11:56 smithi165 ceph-mon[35356]: from='client.? 172.21.15.12:0/431936235' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T00:11:57.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:11:56 smithi178 ceph-mon[35919]: from='client.? 172.21.15.12:0/431936235' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T00:11:57.402 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:11:57.633 INFO:tasks.workunit.client.0.smithi012.stdout: 1 124906 1.34 MB/sec execute 114 sec latency 29466.903 ms 2024-09-11T00:11:57.802 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:11:57 smithi012 ceph-mon[30333]: pgmap v545: 129 pgs: 129 active+clean; 32 MiB data, 676 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 14 KiB/s wr, 21 op/s 2024-09-11T00:11:57.924 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:57.923+0000 7f2ab8e00640 1 -- 172.21.15.12:0/2017356070 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2ab410a800 msgr2=0x7f2ab410ac40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:11:57.924 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:57.923+0000 7f2ab8e00640 1 --2- 172.21.15.12:0/2017356070 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2ab410a800 0x7f2ab410ac40 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f2a9c009900 tx=0x7f2a9c02a410 comp rx=0 tx=0).stop 2024-09-11T00:11:57.924 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:57.923+0000 7f2ab8e00640 1 -- 172.21.15.12:0/2017356070 shutdown_connections 2024-09-11T00:11:57.925 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:57.923+0000 7f2ab8e00640 1 --2- 172.21.15.12:0/2017356070 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2ab410b180 0x7f2ab4111980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:11:57.925 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:57.923+0000 7f2ab8e00640 1 --2- 172.21.15.12:0/2017356070 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2ab410a800 0x7f2ab410ac40 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:11:57.925 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:57.923+0000 7f2ab8e00640 1 --2- 172.21.15.12:0/2017356070 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2ab4104df0 0x7f2ab41051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:11:57.925 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:57.923+0000 7f2ab8e00640 1 -- 172.21.15.12:0/2017356070 >> 172.21.15.12:0/2017356070 conn(0x7f2ab4100840 msgr2=0x7f2ab4102c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:11:57.925 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:57.923+0000 7f2ab8e00640 1 -- 172.21.15.12:0/2017356070 shutdown_connections 2024-09-11T00:11:57.925 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:57.923+0000 7f2ab8e00640 1 -- 172.21.15.12:0/2017356070 wait complete. 2024-09-11T00:11:57.926 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:57.924+0000 7f2ab8e00640 1 Processor -- start 2024-09-11T00:11:57.926 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:57.924+0000 7f2ab8e00640 1 -- start start 2024-09-11T00:11:57.926 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:57.924+0000 7f2ab8e00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2ab4104df0 0x7f2ab419b1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:11:57.926 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:57.924+0000 7f2ab8e00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2ab410a800 0x7f2ab419b6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:11:57.927 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:57.924+0000 7f2ab8e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2ab410b180 0x7f2ab41a0a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:11:57.927 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:57.924+0000 7f2ab8e00640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f2ab4114e50 con 0x7f2ab4104df0 2024-09-11T00:11:57.927 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:57.924+0000 7f2ab8e00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f2ab4114cd0 con 0x7f2ab410b180 2024-09-11T00:11:57.927 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:57.924+0000 7f2ab8e00640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f2ab4114fd0 con 0x7f2ab410a800 2024-09-11T00:11:57.927 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:57.925+0000 7f2ab3400640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2ab4104df0 0x7f2ab419b1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:11:57.927 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:57.925+0000 7f2ab3400640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2ab4104df0 0x7f2ab419b1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:54278/0 (socket says 172.21.15.12:54278) 2024-09-11T00:11:57.927 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:57.925+0000 7f2ab3400640 1 -- 172.21.15.12:0/923699427 learned_addr learned my addr 172.21.15.12:0/923699427 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:11:57.927 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:57.925+0000 7f2aaaa00640 1 --2- 172.21.15.12:0/923699427 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2ab410a800 0x7f2ab419b6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:11:57.927 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:57.925+0000 7f2ab3e00640 1 --2- 172.21.15.12:0/923699427 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2ab410b180 0x7f2ab41a0a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:11:57.927 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:57.925+0000 7f2ab3400640 1 -- 172.21.15.12:0/923699427 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2ab410a800 msgr2=0x7f2ab419b6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:11:57.927 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:57.925+0000 7f2ab3400640 1 --2- 172.21.15.12:0/923699427 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2ab410a800 0x7f2ab419b6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:11:57.927 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:57.925+0000 7f2ab3400640 1 -- 172.21.15.12:0/923699427 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2ab410b180 msgr2=0x7f2ab41a0a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:11:57.928 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:57.925+0000 7f2ab3400640 1 --2- 172.21.15.12:0/923699427 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2ab410b180 0x7f2ab41a0a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:11:57.928 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:57.925+0000 7f2ab3400640 1 -- 172.21.15.12:0/923699427 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ab41a1250 con 0x7f2ab4104df0 2024-09-11T00:11:57.928 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:57.926+0000 7f2ab3400640 1 --2- 172.21.15.12:0/923699427 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2ab4104df0 0x7f2ab419b1a0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f2aa400e9f0 tx=0x7f2aa400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:11:57.928 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:57.926+0000 7f2ab0c00640 1 -- 172.21.15.12:0/923699427 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2aa400cd80 con 0x7f2ab4104df0 2024-09-11T00:11:57.928 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:57.926+0000 7f2ab0c00640 1 -- 172.21.15.12:0/923699427 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2aa40108e0 con 0x7f2ab4104df0 2024-09-11T00:11:57.928 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:57.926+0000 7f2ab8e00640 1 -- 172.21.15.12:0/923699427 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ab41a1540 con 0x7f2ab4104df0 2024-09-11T00:11:57.928 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:57.926+0000 7f2ab0c00640 1 -- 172.21.15.12:0/923699427 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2aa4018980 con 0x7f2ab4104df0 2024-09-11T00:11:57.929 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:57.926+0000 7f2ab8e00640 1 -- 172.21.15.12:0/923699427 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ab41a7c00 con 0x7f2ab4104df0 2024-09-11T00:11:57.930 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:57.929+0000 7f2abce82740 1 Processor -- start 2024-09-11T00:11:57.930 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:57.929+0000 7f2abce82740 1 -- start start 2024-09-11T00:11:57.930 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:57.930+0000 7f2ab0c00640 1 -- 172.21.15.12:0/923699427 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7f2aa4010430 con 0x7f2ab4104df0 2024-09-11T00:11:57.931 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:57.930+0000 7f2abce82740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557565c071d0 0x557565c075b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:11:57.931 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:57.930+0000 7f2abce82740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557565c07af0 0x557565c13540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:11:57.931 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:57.930+0000 7f2abce82740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557565c13a80 0x557565c15e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:11:57.931 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:57.930+0000 7f2abce82740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x557565b7e290 con 0x557565c13a80 2024-09-11T00:11:57.931 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:57.930+0000 7f2abce82740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x557565b6d6c0 con 0x557565c07af0 2024-09-11T00:11:57.931 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:57.930+0000 7f2abce82740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x557565b714d0 con 0x557565c071d0 2024-09-11T00:11:57.932 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:57.930+0000 7f2aaaa00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557565c13a80 0x557565c15e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:11:57.932 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:57.930+0000 7f2ab3400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557565c07af0 0x557565c13540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:11:57.932 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:57.930+0000 7f2ab3e00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557565c071d0 0x557565c075b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:11:57.932 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:57.930+0000 7f2aaaa00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557565c13a80 0x557565c15e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:54288/0 (socket says 172.21.15.12:54288) 2024-09-11T00:11:57.932 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:57.930+0000 7f2aaaa00640 1 -- 172.21.15.12:0/637425458 learned_addr learned my addr 172.21.15.12:0/637425458 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:11:57.932 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:57.931+0000 7f2ab3e00640 1 -- 172.21.15.12:0/637425458 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557565c07af0 msgr2=0x557565c13540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:11:57.933 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:57.931+0000 7f2ab3e00640 1 --2- 172.21.15.12:0/637425458 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557565c07af0 0x557565c13540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:11:57.933 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:57.931+0000 7f2ab3e00640 1 -- 172.21.15.12:0/637425458 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557565c13a80 msgr2=0x557565c15e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:11:57.933 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:57.931+0000 7f2ab3e00640 1 --2- 172.21.15.12:0/637425458 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557565c13a80 0x557565c15e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:11:57.933 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:57.931+0000 7f2ab3e00640 1 -- 172.21.15.12:0/637425458 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557565afa460 con 0x557565c071d0 2024-09-11T00:11:57.933 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:57.931+0000 7f2ab3e00640 1 --2- 172.21.15.12:0/637425458 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557565c071d0 0x557565c075b0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f2aac009f30 tx=0x7f2aac0118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=88dd02dd285c42d0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:11:57.934 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:57.931+0000 7f2aa8c00640 1 -- 172.21.15.12:0/637425458 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2aac004d20 con 0x557565c071d0 2024-09-11T00:11:57.934 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:57.931+0000 7f2ab0c00640 1 --2- 172.21.15.12:0/923699427 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f2a980b5390 0x7f2a980b7850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:11:57.934 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:57.931+0000 7f2aa8c00640 1 -- 172.21.15.12:0/637425458 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2aac004050 con 0x557565c071d0 2024-09-11T00:11:57.935 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:57.931+0000 7f2aa8c00640 1 -- 172.21.15.12:0/637425458 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2aac005280 con 0x557565c071d0 2024-09-11T00:11:57.935 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:57.932+0000 7f2ab0c00640 1 -- 172.21.15.12:0/923699427 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9208+0+0 (secure 0 0 0) 0x7f2aa40e6350 con 0x7f2ab4104df0 2024-09-11T00:11:57.935 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:57.932+0000 7f2ab3400640 1 --2- 172.21.15.12:0/923699427 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f2a980b5390 0x7f2a980b7850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:11:57.935 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:57.932+0000 7f2abce82740 1 -- 172.21.15.12:0/637425458 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557565c071d0 msgr2=0x557565c075b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:11:57.935 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:57.932+0000 7f2abce82740 1 --2- 172.21.15.12:0/637425458 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557565c071d0 0x557565c075b0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f2aac009f30 tx=0x7f2aac0118d0 comp rx=0 tx=0).stop 2024-09-11T00:11:57.935 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:57.932+0000 7f2ab3400640 1 --2- 172.21.15.12:0/923699427 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f2a980b5390 0x7f2a980b7850 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f2aa40ac720 tx=0x7f2aa40e9d90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:11:57.935 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:57.932+0000 7f2abce82740 1 -- 172.21.15.12:0/637425458 shutdown_connections 2024-09-11T00:11:57.935 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:57.933+0000 7f2abce82740 1 --2- 172.21.15.12:0/637425458 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557565c13a80 0x557565c15e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:11:57.935 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:57.933+0000 7f2abce82740 1 --2- 172.21.15.12:0/637425458 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557565c07af0 0x557565c13540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:11:57.935 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:57.933+0000 7f2abce82740 1 --2- 172.21.15.12:0/637425458 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557565c071d0 0x557565c075b0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:11:57.935 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:57.933+0000 7f2abce82740 1 -- 172.21.15.12:0/637425458 >> 172.21.15.12:0/637425458 conn(0x557565c05000 msgr2=0x557565c061a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:11:57.935 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:57.933+0000 7f2abce82740 1 -- 172.21.15.12:0/637425458 shutdown_connections 2024-09-11T00:11:57.935 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:57.933+0000 7f2abce82740 1 -- 172.21.15.12:0/637425458 wait complete. 2024-09-11T00:11:57.935 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:57.933+0000 7f2abce82740 1 -- start start 2024-09-11T00:11:57.936 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:57.933+0000 7f2abce82740 1 Processor -- start 2024-09-11T00:11:57.938 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:57.934+0000 7f2abce82740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557565c071d0 0x557565c10a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:11:57.938 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:57.934+0000 7f2abce82740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557565c07af0 0x557565c10f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:11:57.938 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:57.934+0000 7f2abce82740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557565c13a80 0x557565c114d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:11:57.938 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:57.934+0000 7f2abce82740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x5575659cebb0 con 0x557565c071d0 2024-09-11T00:11:57.938 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:57.934+0000 7f2abce82740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x557565b086c0 con 0x557565c13a80 2024-09-11T00:11:57.938 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:57.934+0000 7f2abce82740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x557565c16730 con 0x557565c07af0 2024-09-11T00:11:57.938 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:57.934+0000 7f2abce82740 10 client.0 _ll_register_callbacks cb 0x557565b23490 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:11:57.938 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:57.934+0000 7f2ab3e00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557565c071d0 0x557565c10a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:11:57.939 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:57.934+0000 7f2ab3e00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557565c071d0 0x557565c10a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:54300/0 (socket says 172.21.15.12:54300) 2024-09-11T00:11:57.939 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:57.934+0000 7f2ab3e00640 1 -- 172.21.15.12:0/72530594 learned_addr learned my addr 172.21.15.12:0/72530594 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:11:57.939 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:57.934+0000 7f2aaaa00640 1 --2- 172.21.15.12:0/72530594 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557565c07af0 0x557565c10f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:11:57.939 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:57.935+0000 7f2ab3e00640 1 --2- 172.21.15.12:0/72530594 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557565c13a80 0x557565c114d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:11:57.939 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:57.935+0000 7f2aaaa00640 1 -- 172.21.15.12:0/72530594 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557565c13a80 msgr2=0x557565c114d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:11:57.939 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:57.935+0000 7f2aaaa00640 1 --2- 172.21.15.12:0/72530594 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557565c13a80 0x557565c114d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:11:57.939 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:57.935+0000 7f2aaaa00640 1 -- 172.21.15.12:0/72530594 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557565c071d0 msgr2=0x557565c10a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:11:57.939 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:57.935+0000 7f2aaaa00640 1 --2- 172.21.15.12:0/72530594 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557565c071d0 0x557565c10a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:11:57.939 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:57.935+0000 7f2aaaa00640 1 -- 172.21.15.12:0/72530594 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5575658cbb00 con 0x557565c07af0 2024-09-11T00:11:57.939 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:57.935+0000 7f2ab3e00640 1 --2- 172.21.15.12:0/72530594 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557565c071d0 0x557565c10a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:11:57.940 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:57.935+0000 7f2aaaa00640 1 --2- 172.21.15.12:0/72530594 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557565c07af0 0x557565c10f90 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f2a9c006010 tx=0x7f2a9c004440 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a6267f563a4c887a server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:11:57.940 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:57.935+0000 7f2aa1600640 10 client.0 ms_handle_connect on v2:172.21.15.178:3300/0 2024-09-11T00:11:57.940 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:57.936+0000 7f2aa1600640 1 -- 172.21.15.12:0/72530594 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a9c035040 con 0x557565c07af0 2024-09-11T00:11:57.940 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:57.936+0000 7f2abce82740 1 -- 172.21.15.12:0/72530594 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2aac004050 con 0x557565c07af0 2024-09-11T00:11:57.940 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:57.936+0000 7f2aa1600640 1 -- 172.21.15.12:0/72530594 <== mon.1 v2:172.21.15.178:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9208+0+0 (secure 0 0 0) 0x7f2a9c005510 con 0x557565c07af0 2024-09-11T00:11:57.940 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:57.936+0000 7f2aa1600640 1 -- 172.21.15.12:0/72530594 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f2a9c04f050 con 0x557565c07af0 2024-09-11T00:11:57.940 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:57.936+0000 7f2abce82740 10 client.26104 fetch_fsmap learned FSMap version 38 2024-09-11T00:11:57.940 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:57.936+0000 7f2abce82740 1 -- 172.21.15.12:0/72530594 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x557565942960 con 0x557565c07af0 2024-09-11T00:11:57.940 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:57.936+0000 7f2aa1600640 1 -- 172.21.15.12:0/72530594 <== mon.1 v2:172.21.15.178:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2aac004050 con 0x557565c07af0 2024-09-11T00:11:57.940 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:57.937+0000 7f2aa1600640 1 -- 172.21.15.12:0/72530594 <== mon.1 v2:172.21.15.178:3300/0 5 ==== fsmap(e 38) ==== 8374+0+0 (secure 0 0 0) 0x7f2a9c003940 con 0x557565c07af0 2024-09-11T00:11:57.941 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:57.937+0000 7f2abce82740 10 client.26104 fetch_fsmap finished waiting for FSMap version 38 2024-09-11T00:11:57.941 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:57.937+0000 7f2abce82740 10 client.26104 resolve_mds: validated gid 24674 aka daemon mds.b 2024-09-11T00:11:57.941 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:57.937+0000 7f2abce82740 20 client.26104 populate_metadata read hostname 'smithi012' 2024-09-11T00:11:57.941 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:57.937+0000 7f2abce82740 1 --2- 172.21.15.12:0/72530594 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x557565c235b0 0x557565c259a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:11:57.941 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:57.937+0000 7f2abce82740 4 client.26104 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T00:11:57.941 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:57.937+0000 7f2abce82740 1 -- 172.21.15.12:0/72530594 --> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557565b801e0 con 0x557565c235b0 2024-09-11T00:11:57.942 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:57.937+0000 7f2ab3e00640 1 --2- 172.21.15.12:0/72530594 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x557565c235b0 0x557565c259a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:11:57.942 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:57.938+0000 7f2ab3e00640 1 --2- 172.21.15.12:0/72530594 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x557565c235b0 0x557565c259a0 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eb1ea33da4676b96 server_cookie=81bd928182c8788 in_seq=0 out_seq=0 2024-09-11T00:11:57.942 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:57.938+0000 7f2aa1600640 10 client.26104 ms_handle_connect on v2:172.21.15.165:6838/501312984 2024-09-11T00:11:57.943 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:57.939+0000 7f2aa1600640 1 -- 172.21.15.12:0/72530594 <== mds.0 v2:172.21.15.165:6838/501312984 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x557565b801e0 con 0x557565c235b0 2024-09-11T00:11:57.943 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:57.939+0000 7f2aa1600640 10 client.26104 handle_command_reply: tid=0 2024-09-11T00:11:57.943 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:57.939+0000 7f2abce82740 1 client.26104 shutdown 2024-09-11T00:11:57.943 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:57.939+0000 7f2abce82740 1 -- 172.21.15.12:0/72530594 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557565c07af0 msgr2=0x557565c10f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:11:57.943 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:57.939+0000 7f2abce82740 1 --2- 172.21.15.12:0/72530594 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557565c07af0 0x557565c10f90 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f2a9c006010 tx=0x7f2a9c004440 comp rx=0 tx=0).stop 2024-09-11T00:11:57.943 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:57.939+0000 7f2abce82740 1 -- 172.21.15.12:0/72530594 shutdown_connections 2024-09-11T00:11:57.943 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:57.939+0000 7f2abce82740 1 --2- 172.21.15.12:0/72530594 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x557565c235b0 0x557565c259a0 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:11:57.943 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:57.939+0000 7f2abce82740 1 --2- 172.21.15.12:0/72530594 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557565c13a80 0x557565c114d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:11:57.943 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:57.939+0000 7f2abce82740 1 --2- 172.21.15.12:0/72530594 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557565c07af0 0x557565c10f90 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:11:57.943 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:57.939+0000 7f2abce82740 1 --2- 172.21.15.12:0/72530594 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557565c071d0 0x557565c10a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:11:57.943 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:57.939+0000 7f2abce82740 1 -- 172.21.15.12:0/72530594 >> 172.21.15.12:0/72530594 conn(0x557565c05000 msgr2=0x557565c0aba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:11:57.943 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:57.939+0000 7f2aa1600640 0 client.26104 ms_handle_reset on v2:172.21.15.165:6838/501312984 2024-09-11T00:11:57.944 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:57.939+0000 7f2abce82740 1 -- 172.21.15.12:0/72530594 shutdown_connections 2024-09-11T00:11:57.944 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:57.939+0000 7f2abce82740 1 -- 172.21.15.12:0/72530594 wait complete. 2024-09-11T00:11:57.944 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:57.939+0000 7f2abce82740 20 client.26104 trim_cache size 0 max 16384 2024-09-11T00:11:57.959 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:57.957+0000 7f2abce82740 1 Processor -- start 2024-09-11T00:11:57.959 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:57.957+0000 7f2abce82740 1 -- start start 2024-09-11T00:11:57.959 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:57.958+0000 7f2abce82740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557565c05000 0x557565c0dc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:11:57.959 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:57.958+0000 7f2abce82740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557565c071d0 0x557565c0e1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:11:57.960 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:57.958+0000 7f2abce82740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557565c07af0 0x557565c2b220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:11:57.960 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:57.958+0000 7f2abce82740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f2ab41b1ae0 con 0x557565c071d0 2024-09-11T00:11:57.960 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:57.958+0000 7f2abce82740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f2ab41b3650 con 0x557565c07af0 2024-09-11T00:11:57.960 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:57.958+0000 7f2abce82740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f2ab41b21b0 con 0x557565c05000 2024-09-11T00:11:57.960 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:57.958+0000 7f2aaaa00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557565c071d0 0x557565c0e1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:11:57.960 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:57.958+0000 7f2aaaa00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557565c071d0 0x557565c0e1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:54312/0 (socket says 172.21.15.12:54312) 2024-09-11T00:11:57.960 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:57.958+0000 7f2aaaa00640 1 -- 172.21.15.12:0/675888729 learned_addr learned my addr 172.21.15.12:0/675888729 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:11:57.960 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:57.958+0000 7f2ab3e00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557565c05000 0x557565c0dc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:11:57.960 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:57.958+0000 7f2ab3e00640 1 --2- 172.21.15.12:0/675888729 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557565c07af0 0x557565c2b220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:11:57.961 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:57.959+0000 7f2aaaa00640 1 -- 172.21.15.12:0/675888729 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557565c05000 msgr2=0x557565c0dc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:11:57.961 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:57.959+0000 7f2aaaa00640 1 --2- 172.21.15.12:0/675888729 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557565c05000 0x557565c0dc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:11:57.962 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:57.959+0000 7f2aaaa00640 1 -- 172.21.15.12:0/675888729 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557565c07af0 msgr2=0x557565c2b220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:11:57.962 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:57.959+0000 7f2aaaa00640 1 --2- 172.21.15.12:0/675888729 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557565c07af0 0x557565c2b220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:11:57.962 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:57.959+0000 7f2aaaa00640 1 -- 172.21.15.12:0/675888729 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557565c161b0 con 0x557565c071d0 2024-09-11T00:11:57.962 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:57.959+0000 7f2aaaa00640 1 --2- 172.21.15.12:0/675888729 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557565c071d0 0x557565c0e1a0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f2a9c009900 tx=0x7f2a9c009520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0373f69b56f9ce server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:11:57.962 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:57.959+0000 7f2aa3e00640 1 -- 172.21.15.12:0/675888729 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a9c058070 con 0x557565c071d0 2024-09-11T00:11:57.962 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:57.959+0000 7f2aa3e00640 1 -- 172.21.15.12:0/675888729 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2a9c041070 con 0x557565c071d0 2024-09-11T00:11:57.962 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:57.959+0000 7f2aa3e00640 1 -- 172.21.15.12:0/675888729 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a9c035070 con 0x557565c071d0 2024-09-11T00:11:57.962 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:57.960+0000 7f2abce82740 1 -- 172.21.15.12:0/675888729 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557565c071d0 msgr2=0x557565c0e1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:11:57.962 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:57.960+0000 7f2abce82740 1 --2- 172.21.15.12:0/675888729 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557565c071d0 0x557565c0e1a0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f2a9c009900 tx=0x7f2a9c009520 comp rx=0 tx=0).stop 2024-09-11T00:11:57.962 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:57.960+0000 7f2abce82740 1 -- 172.21.15.12:0/675888729 shutdown_connections 2024-09-11T00:11:57.962 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:57.960+0000 7f2abce82740 1 --2- 172.21.15.12:0/675888729 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557565c07af0 0x557565c2b220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:11:57.962 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:57.960+0000 7f2abce82740 1 --2- 172.21.15.12:0/675888729 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557565c071d0 0x557565c0e1a0 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:11:57.962 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:57.960+0000 7f2abce82740 1 --2- 172.21.15.12:0/675888729 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557565c05000 0x557565c0dc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:11:57.963 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:57.960+0000 7f2abce82740 1 -- 172.21.15.12:0/675888729 >> 172.21.15.12:0/675888729 conn(0x557565c065f0 msgr2=0x557565c12080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:11:57.963 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:57.960+0000 7f2abce82740 1 -- 172.21.15.12:0/675888729 shutdown_connections 2024-09-11T00:11:57.963 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:57.960+0000 7f2abce82740 1 -- 172.21.15.12:0/675888729 wait complete. 2024-09-11T00:11:57.963 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:57.960+0000 7f2abce82740 1 -- start start 2024-09-11T00:11:57.963 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:57.961+0000 7f2abce82740 1 Processor -- start 2024-09-11T00:11:57.965 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:57.961+0000 7f2abce82740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557565c05000 0x557565c18c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:11:57.965 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:57.961+0000 7f2abce82740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557565c071d0 0x557565c19190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:11:57.965 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:57.961+0000 7f2abce82740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557565c07af0 0x557565c196d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:11:57.965 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:57.961+0000 7f2abce82740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f2ab41b2f80 con 0x557565c071d0 2024-09-11T00:11:57.965 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:57.961+0000 7f2abce82740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f2ab41b28b0 con 0x557565c05000 2024-09-11T00:11:57.965 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:57.961+0000 7f2abce82740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f2ab41afac0 con 0x557565c07af0 2024-09-11T00:11:57.965 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:57.961+0000 7f2abce82740 10 client.0 _ll_register_callbacks cb 0x7f2a9c009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:11:57.966 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:57.961+0000 7f2aaaa00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557565c071d0 0x557565c19190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:11:57.966 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:57.961+0000 7f2aaaa00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557565c071d0 0x557565c19190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:54314/0 (socket says 172.21.15.12:54314) 2024-09-11T00:11:57.966 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:57.961+0000 7f2aaaa00640 1 -- 172.21.15.12:0/1143272957 learned_addr learned my addr 172.21.15.12:0/1143272957 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:11:57.966 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:57.961+0000 7f2ab3e00640 1 --2- 172.21.15.12:0/1143272957 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557565c05000 0x557565c18c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:11:57.966 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:57.961+0000 7f2ab3e00640 1 --2- 172.21.15.12:0/1143272957 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557565c07af0 0x557565c196d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:11:57.966 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:57.962+0000 7f2aaaa00640 1 -- 172.21.15.12:0/1143272957 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557565c07af0 msgr2=0x557565c196d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:11:57.966 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:57.962+0000 7f2aaaa00640 1 --2- 172.21.15.12:0/1143272957 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557565c07af0 0x557565c196d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:11:57.966 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:57.962+0000 7f2aaaa00640 1 -- 172.21.15.12:0/1143272957 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557565c05000 msgr2=0x557565c18c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:11:57.966 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:57.962+0000 7f2aaaa00640 1 --2- 172.21.15.12:0/1143272957 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557565c05000 0x557565c18c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:11:57.966 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:57.962+0000 7f2aaaa00640 1 -- 172.21.15.12:0/1143272957 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557565c906d0 con 0x557565c071d0 2024-09-11T00:11:57.967 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:57.962+0000 7f2ab3e00640 1 --2- 172.21.15.12:0/1143272957 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557565c07af0 0x557565c196d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T00:11:57.967 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:57.962+0000 7f2aaaa00640 1 --2- 172.21.15.12:0/1143272957 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557565c071d0 0x557565c19190 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f2a9c006010 tx=0x7f2a9c004120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea867e219d6f5ce2 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:11:57.967 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:57.962+0000 7f2aa1600640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-09-11T00:11:57.967 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:57.962+0000 7f2aa1600640 1 -- 172.21.15.12:0/1143272957 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a9c058070 con 0x557565c071d0 2024-09-11T00:11:57.967 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:57.962+0000 7f2abce82740 1 -- 172.21.15.12:0/1143272957 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2a9c041070 con 0x557565c071d0 2024-09-11T00:11:57.967 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:57.962+0000 7f2aa1600640 1 -- 172.21.15.12:0/1143272957 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9208+0+0 (secure 0 0 0) 0x7f2a9c005520 con 0x557565c071d0 2024-09-11T00:11:57.967 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:57.963+0000 7f2aa1600640 1 -- 172.21.15.12:0/1143272957 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f2a9c065050 con 0x557565c071d0 2024-09-11T00:11:57.967 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:57.963+0000 7f2abce82740 10 client.16101 fetch_fsmap learned FSMap version 38 2024-09-11T00:11:57.967 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:57.963+0000 7f2abce82740 1 -- 172.21.15.12:0/1143272957 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x557565c91a10 con 0x557565c071d0 2024-09-11T00:11:57.967 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:57.963+0000 7f2aa1600640 1 -- 172.21.15.12:0/1143272957 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2a9c041070 con 0x557565c071d0 2024-09-11T00:11:57.967 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:57.963+0000 7f2aa1600640 1 -- 172.21.15.12:0/1143272957 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 38) ==== 8374+0+0 (secure 0 0 0) 0x7f2a9c0314a0 con 0x557565c071d0 2024-09-11T00:11:57.967 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:57.963+0000 7f2abce82740 10 client.16101 fetch_fsmap finished waiting for FSMap version 38 2024-09-11T00:11:57.967 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:57.963+0000 7f2abce82740 10 client.16101 resolve_mds: validated gid 24674 aka daemon mds.b 2024-09-11T00:11:57.968 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:57.963+0000 7f2abce82740 20 client.16101 populate_metadata read hostname 'smithi012' 2024-09-11T00:11:57.968 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:57.963+0000 7f2abce82740 1 --2- 172.21.15.12:0/1143272957 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x557565c13a80 0x557565c93fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:11:57.968 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:57.963+0000 7f2abce82740 4 client.16101 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "8f1ca7f2", "release": true, "await": true}] 2024-09-11T00:11:57.968 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:57.963+0000 7f2abce82740 1 -- 172.21.15.12:0/1143272957 --> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] -- command(tid 0: {"prefix": "quiesce db", "set_id": "8f1ca7f2", "release": true, "await": true}) -- 0x557565b7bc90 con 0x557565c13a80 2024-09-11T00:11:57.968 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:57.964+0000 7f2ab3e00640 1 --2- 172.21.15.12:0/1143272957 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x557565c13a80 0x557565c93fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:11:57.968 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:57.964+0000 7f2ab3e00640 1 --2- 172.21.15.12:0/1143272957 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x557565c13a80 0x557565c93fd0 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1403e9ee49d700a0 server_cookie=7964ef61bdafe8b in_seq=0 out_seq=0 2024-09-11T00:11:57.968 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:57.964+0000 7f2aa1600640 10 client.16101 ms_handle_connect on v2:172.21.15.165:6838/501312984 2024-09-11T00:11:57.973 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:57.972+0000 7f2aa1600640 1 -- 172.21.15.12:0/1143272957 <== mds.0 v2:172.21.15.165:6838/501312984 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x557565b7bc90 con 0x557565c13a80 2024-09-11T00:11:57.973 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:57.972+0000 7f2aa1600640 10 client.16101 handle_command_reply: tid=0 2024-09-11T00:11:57.973 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:57.972+0000 7f2abce82740 1 client.16101 shutdown 2024-09-11T00:11:57.973 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:57.972+0000 7f2abce82740 1 -- 172.21.15.12:0/1143272957 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557565c071d0 msgr2=0x557565c19190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:11:57.973 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:57.972+0000 7f2abce82740 1 --2- 172.21.15.12:0/1143272957 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557565c071d0 0x557565c19190 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f2a9c006010 tx=0x7f2a9c004120 comp rx=0 tx=0).stop 2024-09-11T00:11:57.974 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:57.972+0000 7f2abce82740 1 -- 172.21.15.12:0/1143272957 shutdown_connections 2024-09-11T00:11:57.974 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:57.972+0000 7f2abce82740 1 --2- 172.21.15.12:0/1143272957 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x557565c13a80 0x557565c93fd0 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:11:57.974 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:57.972+0000 7f2abce82740 1 --2- 172.21.15.12:0/1143272957 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557565c07af0 0x557565c196d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:11:57.974 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:57.972+0000 7f2abce82740 1 --2- 172.21.15.12:0/1143272957 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557565c071d0 0x557565c19190 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:11:57.974 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:57.972+0000 7f2aa1600640 0 client.16101 ms_handle_reset on v2:172.21.15.165:6838/501312984 2024-09-11T00:11:57.974 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:57.972+0000 7f2abce82740 1 --2- 172.21.15.12:0/1143272957 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557565c05000 0x557565c18c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:11:57.974 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:57.972+0000 7f2abce82740 1 -- 172.21.15.12:0/1143272957 >> 172.21.15.12:0/1143272957 conn(0x557565c065f0 msgr2=0x557565c1e980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:11:57.974 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:57.972+0000 7f2abce82740 1 -- 172.21.15.12:0/1143272957 shutdown_connections 2024-09-11T00:11:57.974 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:57.972+0000 7f2abce82740 1 -- 172.21.15.12:0/1143272957 wait complete. 2024-09-11T00:11:57.975 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:57.973+0000 7f2abce82740 20 client.16101 trim_cache size 0 max 16384 2024-09-11T00:11:57.975 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-09-11T00:11:57.975 INFO:teuthology.orchestra.run.smithi012.stdout: "epoch": 38, 2024-09-11T00:11:57.975 INFO:teuthology.orchestra.run.smithi012.stdout: "leader": 24674, 2024-09-11T00:11:57.975 INFO:teuthology.orchestra.run.smithi012.stdout: "set_version": 20, 2024-09-11T00:11:57.975 INFO:teuthology.orchestra.run.smithi012.stdout: "sets": { 2024-09-11T00:11:57.975 INFO:teuthology.orchestra.run.smithi012.stdout: "8f1ca7f2": { 2024-09-11T00:11:57.975 INFO:teuthology.orchestra.run.smithi012.stdout: "version": 20, 2024-09-11T00:11:57.976 INFO:teuthology.orchestra.run.smithi012.stdout: "age_ref": 0.0, 2024-09-11T00:11:57.976 INFO:teuthology.orchestra.run.smithi012.stdout: "state": { 2024-09-11T00:11:57.976 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "RELEASED", 2024-09-11T00:11:57.976 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 0.0 2024-09-11T00:11:57.976 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:11:57.976 INFO:teuthology.orchestra.run.smithi012.stdout: "timeout": 90.0, 2024-09-11T00:11:57.976 INFO:teuthology.orchestra.run.smithi012.stdout: "expiration": 150.0, 2024-09-11T00:11:57.976 INFO:teuthology.orchestra.run.smithi012.stdout: "members": { 2024-09-11T00:11:57.976 INFO:teuthology.orchestra.run.smithi012.stdout: "file:/": { 2024-09-11T00:11:57.976 INFO:teuthology.orchestra.run.smithi012.stdout: "excluded": false, 2024-09-11T00:11:57.976 INFO:teuthology.orchestra.run.smithi012.stdout: "state": { 2024-09-11T00:11:57.976 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "RELEASED", 2024-09-11T00:11:57.977 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 0.0 2024-09-11T00:11:57.977 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:11:57.977 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:11:57.977 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:11:57.977 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:11:57.977 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:11:57.977 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-09-11T00:11:57.977 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:57.973+0000 7f2ab8e00640 1 -- 172.21.15.12:0/923699427 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f2a980b5390 msgr2=0x7f2a980b7850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:11:57.978 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:57.973+0000 7f2ab8e00640 1 --2- 172.21.15.12:0/923699427 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f2a980b5390 0x7f2a980b7850 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f2aa40ac720 tx=0x7f2aa40e9d90 comp rx=0 tx=0).stop 2024-09-11T00:11:57.978 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:57.973+0000 7f2ab8e00640 1 -- 172.21.15.12:0/923699427 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2ab4104df0 msgr2=0x7f2ab419b1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:11:57.978 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:57.973+0000 7f2ab8e00640 1 --2- 172.21.15.12:0/923699427 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2ab4104df0 0x7f2ab419b1a0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f2aa400e9f0 tx=0x7f2aa400eeb0 comp rx=0 tx=0).stop 2024-09-11T00:11:57.978 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:57.973+0000 7f2ab8e00640 1 -- 172.21.15.12:0/923699427 shutdown_connections 2024-09-11T00:11:57.978 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:57.973+0000 7f2ab8e00640 1 --2- 172.21.15.12:0/923699427 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f2a980b5390 0x7f2a980b7850 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:11:57.978 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:57.973+0000 7f2ab8e00640 1 --2- 172.21.15.12:0/923699427 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2ab410b180 0x7f2ab41a0a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:11:57.978 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:57.973+0000 7f2ab8e00640 1 --2- 172.21.15.12:0/923699427 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2ab410a800 0x7f2ab419b6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:11:57.978 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:57.973+0000 7f2ab8e00640 1 --2- 172.21.15.12:0/923699427 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2ab4104df0 0x7f2ab419b1a0 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:11:57.978 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:57.973+0000 7f2ab8e00640 1 -- 172.21.15.12:0/923699427 >> 172.21.15.12:0/923699427 conn(0x7f2ab4100840 msgr2=0x7f2ab4102c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:11:57.978 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:57.974+0000 7f2ab8e00640 1 -- 172.21.15.12:0/923699427 shutdown_connections 2024-09-11T00:11:57.978 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:11:57.974+0000 7f2ab8e00640 1 -- 172.21.15.12:0/923699427 wait complete. 2024-09-11T00:11:58.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:11:57 smithi165 ceph-mon[35356]: pgmap v545: 129 pgs: 129 active+clean; 32 MiB data, 676 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 14 KiB/s wr, 21 op/s 2024-09-11T00:11:58.080 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:11:57 smithi178 ceph-mon[35919]: pgmap v545: 129 pgs: 129 active+clean; 32 MiB data, 676 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 14 KiB/s wr, 21 op/s 2024-09-11T00:11:58.633 INFO:tasks.workunit.client.0.smithi012.stdout: 1 126364 1.38 MB/sec execute 115 sec latency 29809.847 ms 2024-09-11T00:11:58.900 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '8f1ca7f2', total seconds elapsed: 34.0 2024-09-11T00:11:59.633 INFO:tasks.workunit.client.0.smithi012.stdout: 1 127825 1.47 MB/sec execute 116 sec latency 220.140 ms 2024-09-11T00:12:00.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:11:59 smithi165 ceph-mon[35356]: pgmap v546: 129 pgs: 129 active+clean; 32 MiB data, 676 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 8.2 KiB/s wr, 14 op/s 2024-09-11T00:12:00.063 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:11:59 smithi012 ceph-mon[30333]: pgmap v546: 129 pgs: 129 active+clean; 32 MiB data, 676 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 8.2 KiB/s wr, 14 op/s 2024-09-11T00:12:00.080 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:11:59 smithi178 ceph-mon[35919]: pgmap v546: 129 pgs: 129 active+clean; 32 MiB data, 676 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 8.2 KiB/s wr, 14 op/s 2024-09-11T00:12:00.633 INFO:tasks.workunit.client.0.smithi012.stdout: 1 128170 1.47 MB/sec execute 117 sec latency 752.696 ms 2024-09-11T00:12:01.633 INFO:tasks.workunit.client.0.smithi012.stdout: 1 128518 1.46 MB/sec execute 118 sec latency 684.685 ms 2024-09-11T00:12:02.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:12:01 smithi165 ceph-mon[35356]: pgmap v547: 129 pgs: 129 active+clean; 32 MiB data, 681 MiB used, 1.0 TiB / 1.0 TiB avail; 9.0 MiB/s rd, 83 KiB/s wr, 67 op/s 2024-09-11T00:12:02.063 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:12:01 smithi012 ceph-mon[30333]: pgmap v547: 129 pgs: 129 active+clean; 32 MiB data, 681 MiB used, 1.0 TiB / 1.0 TiB avail; 9.0 MiB/s rd, 83 KiB/s wr, 67 op/s 2024-09-11T00:12:02.080 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:12:01 smithi178 ceph-mon[35919]: pgmap v547: 129 pgs: 129 active+clean; 32 MiB data, 681 MiB used, 1.0 TiB / 1.0 TiB avail; 9.0 MiB/s rd, 83 KiB/s wr, 67 op/s 2024-09-11T00:12:02.634 INFO:tasks.workunit.client.0.smithi012.stdout: 1 128518 1.45 MB/sec execute 119 sec latency 1684.824 ms 2024-09-11T00:12:02.793 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:02.791+0000 7f20d1000640 1 -- 172.21.15.12:0/4147171252 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f20cc102790 msgr2=0x7f20cc102b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:12:02.793 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:02.791+0000 7f20d1000640 1 --2- 172.21.15.12:0/4147171252 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f20cc102790 0x7f20cc102b90 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f20bc009930 tx=0x7f20bc02a420 comp rx=0 tx=0).stop 2024-09-11T00:12:02.793 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:02.792+0000 7f20d1000640 1 -- 172.21.15.12:0/4147171252 shutdown_connections 2024-09-11T00:12:02.793 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:02.792+0000 7f20d1000640 1 --2- 172.21.15.12:0/4147171252 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f20cc1042a0 0x7f20cc10ac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:12:02.793 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:02.792+0000 7f20d1000640 1 --2- 172.21.15.12:0/4147171252 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f20cc1038e0 0x7f20cc103d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:12:02.794 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:02.792+0000 7f20d1000640 1 --2- 172.21.15.12:0/4147171252 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f20cc102790 0x7f20cc102b90 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:12:02.794 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:02.792+0000 7f20d1000640 1 -- 172.21.15.12:0/4147171252 >> 172.21.15.12:0/4147171252 conn(0x7f20cc0fe030 msgr2=0x7f20cc100470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:12:02.794 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:02.792+0000 7f20d1000640 1 -- 172.21.15.12:0/4147171252 shutdown_connections 2024-09-11T00:12:02.794 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:02.792+0000 7f20d1000640 1 -- 172.21.15.12:0/4147171252 wait complete. 2024-09-11T00:12:02.796 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:02.793+0000 7f20d1000640 1 Processor -- start 2024-09-11T00:12:02.796 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:02.793+0000 7f20d1000640 1 -- start start 2024-09-11T00:12:02.796 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:02.793+0000 7f20d1000640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f20cc102790 0x7f20cc19ae40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:12:02.796 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:02.793+0000 7f20d1000640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f20cc1038e0 0x7f20cc19b490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:12:02.796 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:02.793+0000 7f20d1000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f20cc1042a0 0x7f20cc19c0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:12:02.796 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:02.793+0000 7f20d1000640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f20cc10e450 con 0x7f20cc102790 2024-09-11T00:12:02.796 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:02.793+0000 7f20d1000640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f20cc10e2d0 con 0x7f20cc1042a0 2024-09-11T00:12:02.797 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:02.793+0000 7f20d1000640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f20cc10e5d0 con 0x7f20cc1038e0 2024-09-11T00:12:02.797 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:02.793+0000 7f20cb400640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f20cc102790 0x7f20cc19ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:12:02.797 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:02.793+0000 7f20cb400640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f20cc102790 0x7f20cc19ae40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:48348/0 (socket says 172.21.15.12:48348) 2024-09-11T00:12:02.797 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:02.793+0000 7f20cb400640 1 -- 172.21.15.12:0/4291049155 learned_addr learned my addr 172.21.15.12:0/4291049155 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:12:02.797 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:02.793+0000 7f20caa00640 1 --2- 172.21.15.12:0/4291049155 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f20cc1038e0 0x7f20cc19b490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:12:02.797 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:02.794+0000 7f20cbe00640 1 --2- 172.21.15.12:0/4291049155 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f20cc1042a0 0x7f20cc19c0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:12:02.797 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:02.794+0000 7f20caa00640 1 -- 172.21.15.12:0/4291049155 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f20cc1042a0 msgr2=0x7f20cc19c0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:12:02.797 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:02.794+0000 7f20caa00640 1 --2- 172.21.15.12:0/4291049155 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f20cc1042a0 0x7f20cc19c0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:12:02.797 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:02.794+0000 7f20caa00640 1 -- 172.21.15.12:0/4291049155 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f20cc102790 msgr2=0x7f20cc19ae40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:12:02.797 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:02.794+0000 7f20caa00640 1 --2- 172.21.15.12:0/4291049155 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f20cc102790 0x7f20cc19ae40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:12:02.797 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:02.794+0000 7f20caa00640 1 -- 172.21.15.12:0/4291049155 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20cc19c980 con 0x7f20cc1038e0 2024-09-11T00:12:02.797 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:02.794+0000 7f20cb400640 1 --2- 172.21.15.12:0/4291049155 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f20cc102790 0x7f20cc19ae40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:12:02.798 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:02.794+0000 7f20caa00640 1 --2- 172.21.15.12:0/4291049155 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f20cc1038e0 0x7f20cc19b490 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f20b800e9f0 tx=0x7f20b800eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:12:02.798 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:02.794+0000 7f20c3e00640 1 -- 172.21.15.12:0/4291049155 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20b800ccc0 con 0x7f20cc1038e0 2024-09-11T00:12:02.798 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:02.794+0000 7f20c3e00640 1 -- 172.21.15.12:0/4291049155 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f20b8005020 con 0x7f20cc1038e0 2024-09-11T00:12:02.798 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:02.794+0000 7f20d1000640 1 -- 172.21.15.12:0/4291049155 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f20cc19cc70 con 0x7f20cc1038e0 2024-09-11T00:12:02.798 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:02.795+0000 7f20c3e00640 1 -- 172.21.15.12:0/4291049155 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20b8004a00 con 0x7f20cc1038e0 2024-09-11T00:12:02.798 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:02.795+0000 7f20d1000640 1 -- 172.21.15.12:0/4291049155 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20cc1a3340 con 0x7f20cc1038e0 2024-09-11T00:12:02.798 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:02.798+0000 7f20d54bd740 1 Processor -- start 2024-09-11T00:12:02.798 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:02.798+0000 7f20d54bd740 1 -- start start 2024-09-11T00:12:02.800 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:02.798+0000 7f20d54bd740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5596e662c970 0x5596e662ed60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:12:02.800 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:02.798+0000 7f20d54bd740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5596e662f2a0 0x5596e6631690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:12:02.801 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:02.798+0000 7f20d54bd740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5596e6631ce0 0x5596e66341a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:12:02.801 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:02.798+0000 7f20d54bd740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x5596e659f6b0 con 0x5596e662c970 2024-09-11T00:12:02.801 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:02.798+0000 7f20d54bd740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x5596e658eae0 con 0x5596e662f2a0 2024-09-11T00:12:02.801 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:02.798+0000 7f20d54bd740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5596e65928f0 con 0x5596e6631ce0 2024-09-11T00:12:02.801 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:02.798+0000 7f20cbe00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5596e662c970 0x5596e662ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:12:02.801 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:02.798+0000 7f20cbe00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5596e662c970 0x5596e662ed60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:48350/0 (socket says 172.21.15.12:48350) 2024-09-11T00:12:02.801 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:02.798+0000 7f20cbe00640 1 -- 172.21.15.12:0/1172550037 learned_addr learned my addr 172.21.15.12:0/1172550037 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:12:02.801 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:02.798+0000 7f20c3e00640 1 -- 172.21.15.12:0/4291049155 <== mon.1 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7f20b8004ba0 con 0x7f20cc1038e0 2024-09-11T00:12:02.801 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:02.799+0000 7f20cb400640 1 --2- 172.21.15.12:0/1172550037 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5596e662f2a0 0x5596e6631690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:12:02.801 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:02.799+0000 7f20cbe00640 1 -- 172.21.15.12:0/1172550037 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5596e6631ce0 msgr2=0x5596e66341a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:12:02.801 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:02.799+0000 7f20cbe00640 1 --2- 172.21.15.12:0/1172550037 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5596e6631ce0 0x5596e66341a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:12:02.801 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:02.799+0000 7f20cbe00640 1 -- 172.21.15.12:0/1172550037 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5596e662f2a0 msgr2=0x5596e6631690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:12:02.801 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:02.799+0000 7f20cbe00640 1 --2- 172.21.15.12:0/1172550037 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5596e662f2a0 0x5596e6631690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:12:02.802 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:02.799+0000 7f20cbe00640 1 -- 172.21.15.12:0/1172550037 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5596e655b780 con 0x5596e662c970 2024-09-11T00:12:02.802 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:02.799+0000 7f20cb400640 1 --2- 172.21.15.12:0/1172550037 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5596e662f2a0 0x5596e6631690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T00:12:02.802 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:02.799+0000 7f20cbe00640 1 --2- 172.21.15.12:0/1172550037 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5596e662c970 0x5596e662ed60 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f20c4009f30 tx=0x7f20c40118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9de3090d5144f3e6 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:12:02.802 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:02.799+0000 7f20c0c00640 1 -- 172.21.15.12:0/1172550037 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20c4004d20 con 0x5596e662c970 2024-09-11T00:12:02.802 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:02.799+0000 7f20c0c00640 1 -- 172.21.15.12:0/1172550037 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f20c4004050 con 0x5596e662c970 2024-09-11T00:12:02.802 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:02.800+0000 7f20c0c00640 1 -- 172.21.15.12:0/1172550037 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20c4005280 con 0x5596e662c970 2024-09-11T00:12:02.805 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:02.800+0000 7f20c3e00640 1 --2- 172.21.15.12:0/4291049155 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f20b00b5360 0x7f20b00b7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:12:02.806 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:02.800+0000 7f20d54bd740 1 -- 172.21.15.12:0/1172550037 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5596e662c970 msgr2=0x5596e662ed60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:12:02.806 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:02.800+0000 7f20d54bd740 1 --2- 172.21.15.12:0/1172550037 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5596e662c970 0x5596e662ed60 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f20c4009f30 tx=0x7f20c40118d0 comp rx=0 tx=0).stop 2024-09-11T00:12:02.806 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:02.800+0000 7f20c3e00640 1 -- 172.21.15.12:0/4291049155 <== mon.1 v2:172.21.15.178:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9208+0+0 (secure 0 0 0) 0x7f20b80e5f30 con 0x7f20cc1038e0 2024-09-11T00:12:02.806 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:02.800+0000 7f20d54bd740 1 -- 172.21.15.12:0/1172550037 shutdown_connections 2024-09-11T00:12:02.806 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:02.800+0000 7f20d54bd740 1 --2- 172.21.15.12:0/1172550037 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5596e6631ce0 0x5596e66341a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:12:02.806 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:02.800+0000 7f20d54bd740 1 --2- 172.21.15.12:0/1172550037 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5596e662f2a0 0x5596e6631690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:12:02.806 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:02.800+0000 7f20d54bd740 1 --2- 172.21.15.12:0/1172550037 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5596e662c970 0x5596e662ed60 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:12:02.806 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:02.800+0000 7f20d54bd740 1 -- 172.21.15.12:0/1172550037 >> 172.21.15.12:0/1172550037 conn(0x5596e6627ac0 msgr2=0x5596e6628850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:12:02.806 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:02.800+0000 7f20cb400640 1 --2- 172.21.15.12:0/4291049155 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f20b00b5360 0x7f20b00b7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:12:02.806 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:02.800+0000 7f20d54bd740 1 -- 172.21.15.12:0/1172550037 shutdown_connections 2024-09-11T00:12:02.806 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:02.801+0000 7f20d54bd740 1 -- 172.21.15.12:0/1172550037 wait complete. 2024-09-11T00:12:02.807 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:02.801+0000 7f20d54bd740 1 -- start start 2024-09-11T00:12:02.807 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:02.801+0000 7f20cb400640 1 --2- 172.21.15.12:0/4291049155 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f20b00b5360 0x7f20b00b7820 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f20bc000c00 tx=0x7f20bc005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:12:02.807 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:02.801+0000 7f20d54bd740 1 Processor -- start 2024-09-11T00:12:02.807 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:02.802+0000 7f20d54bd740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5596e662c970 0x5596e6636130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:12:02.807 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:02.802+0000 7f20d54bd740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5596e662f2a0 0x5596e66367d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:12:02.807 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:02.802+0000 7f20d54bd740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5596e6631ce0 0x5596e6637330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:12:02.807 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:02.802+0000 7f20d54bd740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x5596e64c6ab0 con 0x5596e662c970 2024-09-11T00:12:02.807 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:02.802+0000 7f20d54bd740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x5596e6535570 con 0x5596e6631ce0 2024-09-11T00:12:02.807 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:02.802+0000 7f20d54bd740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5596e6634cc0 con 0x5596e662f2a0 2024-09-11T00:12:02.807 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:02.802+0000 7f20d54bd740 10 client.0 _ll_register_callbacks cb 0x5596e6544100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:12:02.807 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:02.802+0000 7f20cb400640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5596e662c970 0x5596e6636130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:12:02.807 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:02.802+0000 7f20caa00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5596e662f2a0 0x5596e66367d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:12:02.807 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:02.802+0000 7f20caa00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5596e662f2a0 0x5596e66367d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.12:40484/0 (socket says 172.21.15.12:40484) 2024-09-11T00:12:02.808 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:02.802+0000 7f20caa00640 1 -- 172.21.15.12:0/2117359807 learned_addr learned my addr 172.21.15.12:0/2117359807 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:12:02.808 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:02.802+0000 7f20cbe00640 1 --2- 172.21.15.12:0/2117359807 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5596e6631ce0 0x5596e6637330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:12:02.808 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:02.803+0000 7f20caa00640 1 -- 172.21.15.12:0/2117359807 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5596e6631ce0 msgr2=0x5596e6637330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:12:02.808 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:02.803+0000 7f20caa00640 1 --2- 172.21.15.12:0/2117359807 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5596e6631ce0 0x5596e6637330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:12:02.808 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:02.803+0000 7f20caa00640 1 -- 172.21.15.12:0/2117359807 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5596e662c970 msgr2=0x5596e6636130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:12:02.808 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:02.803+0000 7f20caa00640 1 --2- 172.21.15.12:0/2117359807 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5596e662c970 0x5596e6636130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:12:02.808 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:02.803+0000 7f20caa00640 1 -- 172.21.15.12:0/2117359807 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5596e62f0790 con 0x5596e662f2a0 2024-09-11T00:12:02.808 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:02.803+0000 7f20caa00640 1 --2- 172.21.15.12:0/2117359807 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5596e662f2a0 0x5596e66367d0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f20b8009230 tx=0x7f20b80abaf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c7662f40edbb48d0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:12:02.808 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:02.803+0000 7f20b5600640 10 client.0 ms_handle_connect on v2:172.21.15.178:3300/0 2024-09-11T00:12:02.808 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:02.803+0000 7f20b5600640 1 -- 172.21.15.12:0/2117359807 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20b801c040 con 0x5596e662f2a0 2024-09-11T00:12:02.808 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:02.803+0000 7f20d54bd740 1 -- 172.21.15.12:0/2117359807 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f20c4004050 con 0x5596e662f2a0 2024-09-11T00:12:02.808 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:02.803+0000 7f20b5600640 1 -- 172.21.15.12:0/2117359807 <== mon.1 v2:172.21.15.178:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9208+0+0 (secure 0 0 0) 0x7f20b80054f0 con 0x5596e662f2a0 2024-09-11T00:12:02.809 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:02.803+0000 7f20b5600640 1 -- 172.21.15.12:0/2117359807 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f20b8095e60 con 0x5596e662f2a0 2024-09-11T00:12:02.809 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:02.803+0000 7f20d54bd740 10 client.26119 fetch_fsmap learned FSMap version 38 2024-09-11T00:12:02.809 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:02.803+0000 7f20d54bd740 1 -- 172.21.15.12:0/2117359807 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x5596e6369620 con 0x5596e662f2a0 2024-09-11T00:12:02.809 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:02.804+0000 7f20b5600640 1 -- 172.21.15.12:0/2117359807 <== mon.1 v2:172.21.15.178:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f20c4004050 con 0x5596e662f2a0 2024-09-11T00:12:02.809 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:02.804+0000 7f20b5600640 1 -- 172.21.15.12:0/2117359807 <== mon.1 v2:172.21.15.178:3300/0 5 ==== fsmap(e 38) ==== 8374+0+0 (secure 0 0 0) 0x7f20b80b53c0 con 0x5596e662f2a0 2024-09-11T00:12:02.809 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:02.804+0000 7f20d54bd740 10 client.26119 fetch_fsmap finished waiting for FSMap version 38 2024-09-11T00:12:02.809 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:02.804+0000 7f20d54bd740 10 client.26119 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T00:12:02.809 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:02.804+0000 7f20d54bd740 20 client.26119 populate_metadata read hostname 'smithi012' 2024-09-11T00:12:02.809 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:02.804+0000 7f20d54bd740 1 --2- 172.21.15.12:0/2117359807 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x5596e66419a0 0x5596e6643d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:12:02.809 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:02.804+0000 7f20d54bd740 4 client.26119 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T00:12:02.809 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:02.804+0000 7f20d54bd740 1 -- 172.21.15.12:0/2117359807 --> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5596e65a1600 con 0x5596e66419a0 2024-09-11T00:12:02.809 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:02.805+0000 7f20cb400640 1 --2- 172.21.15.12:0/2117359807 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x5596e66419a0 0x5596e6643d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:12:02.810 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:02.805+0000 7f20cb400640 1 --2- 172.21.15.12:0/2117359807 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x5596e66419a0 0x5596e6643d90 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ffdd5d0cbc9e6559 server_cookie=9367ab4aa8a105c7 in_seq=0 out_seq=0 2024-09-11T00:12:02.811 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:02.805+0000 7f20b5600640 10 client.26119 ms_handle_connect on v2:172.21.15.165:6838/501312984 2024-09-11T00:12:02.811 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:02.806+0000 7f20b5600640 1 -- 172.21.15.12:0/2117359807 <== mds.0 v2:172.21.15.165:6838/501312984 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5596e65a1600 con 0x5596e66419a0 2024-09-11T00:12:02.811 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:02.806+0000 7f20b5600640 10 client.26119 handle_command_reply: tid=0 2024-09-11T00:12:02.811 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:02.806+0000 7f20d54bd740 1 client.26119 shutdown 2024-09-11T00:12:02.811 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:02.806+0000 7f20d54bd740 1 -- 172.21.15.12:0/2117359807 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5596e662f2a0 msgr2=0x5596e66367d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:12:02.811 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:02.806+0000 7f20d54bd740 1 --2- 172.21.15.12:0/2117359807 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5596e662f2a0 0x5596e66367d0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f20b8009230 tx=0x7f20b80abaf0 comp rx=0 tx=0).stop 2024-09-11T00:12:02.811 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:02.806+0000 7f20d54bd740 1 -- 172.21.15.12:0/2117359807 shutdown_connections 2024-09-11T00:12:02.811 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:02.806+0000 7f20d54bd740 1 --2- 172.21.15.12:0/2117359807 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x5596e66419a0 0x5596e6643d90 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:12:02.811 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:02.806+0000 7f20d54bd740 1 --2- 172.21.15.12:0/2117359807 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5596e6631ce0 0x5596e6637330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:12:02.811 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:02.806+0000 7f20d54bd740 1 --2- 172.21.15.12:0/2117359807 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5596e662f2a0 0x5596e66367d0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:12:02.811 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:02.806+0000 7f20d54bd740 1 --2- 172.21.15.12:0/2117359807 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5596e662c970 0x5596e6636130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:12:02.811 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:02.806+0000 7f20d54bd740 1 -- 172.21.15.12:0/2117359807 >> 172.21.15.12:0/2117359807 conn(0x5596e6627ac0 msgr2=0x5596e6626bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:12:02.811 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:02.806+0000 7f20b5600640 0 client.26119 ms_handle_reset on v2:172.21.15.165:6838/501312984 2024-09-11T00:12:02.812 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:02.807+0000 7f20d54bd740 1 -- 172.21.15.12:0/2117359807 shutdown_connections 2024-09-11T00:12:02.812 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:02.807+0000 7f20d54bd740 1 -- 172.21.15.12:0/2117359807 wait complete. 2024-09-11T00:12:02.812 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:02.807+0000 7f20d54bd740 20 client.26119 trim_cache size 0 max 16384 2024-09-11T00:12:02.824 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:02.822+0000 7f20d54bd740 1 Processor -- start 2024-09-11T00:12:02.824 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:02.822+0000 7f20d54bd740 1 -- start start 2024-09-11T00:12:02.824 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:02.822+0000 7f20d54bd740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5596e662c970 0x5596e662beb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:12:02.824 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:02.822+0000 7f20d54bd740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5596e662f2a0 0x5596e662c3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:12:02.824 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:02.822+0000 7f20d54bd740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5596e6631ce0 0x5596e6632170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:12:02.825 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:02.822+0000 7f20d54bd740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f20cc1ad880 con 0x5596e662f2a0 2024-09-11T00:12:02.825 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:02.822+0000 7f20d54bd740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f20cc1af3f0 con 0x5596e662c970 2024-09-11T00:12:02.825 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:02.822+0000 7f20d54bd740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f20cc1adf50 con 0x5596e6631ce0 2024-09-11T00:12:02.825 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:02.823+0000 7f20caa00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5596e662f2a0 0x5596e662c3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:12:02.825 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:02.823+0000 7f20caa00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5596e662f2a0 0x5596e662c3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:48370/0 (socket says 172.21.15.12:48370) 2024-09-11T00:12:02.825 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:02.823+0000 7f20caa00640 1 -- 172.21.15.12:0/3273221379 learned_addr learned my addr 172.21.15.12:0/3273221379 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:12:02.825 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:02.823+0000 7f20cbe00640 1 --2- 172.21.15.12:0/3273221379 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5596e6631ce0 0x5596e6632170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:12:02.825 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:02.823+0000 7f20cb400640 1 --2- 172.21.15.12:0/3273221379 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5596e662c970 0x5596e662beb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:12:02.825 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:02.823+0000 7f20caa00640 1 -- 172.21.15.12:0/3273221379 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5596e6631ce0 msgr2=0x5596e6632170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:12:02.825 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:02.823+0000 7f20caa00640 1 --2- 172.21.15.12:0/3273221379 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5596e6631ce0 0x5596e6632170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:12:02.825 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:02.823+0000 7f20caa00640 1 -- 172.21.15.12:0/3273221379 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5596e662c970 msgr2=0x5596e662beb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:12:02.825 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:02.823+0000 7f20caa00640 1 --2- 172.21.15.12:0/3273221379 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5596e662c970 0x5596e662beb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:12:02.825 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:02.823+0000 7f20caa00640 1 -- 172.21.15.12:0/3273221379 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5596e6644100 con 0x5596e662f2a0 2024-09-11T00:12:02.826 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:02.823+0000 7f20caa00640 1 --2- 172.21.15.12:0/3273221379 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5596e662f2a0 0x5596e662c3f0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f20b80ac250 tx=0x7f20b80096e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb63929d9ed20914 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:12:02.828 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:02.824+0000 7f20b7e00640 1 -- 172.21.15.12:0/3273221379 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20b801c070 con 0x5596e662f2a0 2024-09-11T00:12:02.829 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:02.824+0000 7f20b7e00640 1 -- 172.21.15.12:0/3273221379 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f20b80b0da0 con 0x5596e662f2a0 2024-09-11T00:12:02.829 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:02.824+0000 7f20b7e00640 1 -- 172.21.15.12:0/3273221379 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20b80b53c0 con 0x5596e662f2a0 2024-09-11T00:12:02.829 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:02.824+0000 7f20d54bd740 1 -- 172.21.15.12:0/3273221379 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5596e662f2a0 msgr2=0x5596e662c3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:12:02.829 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:02.824+0000 7f20d54bd740 1 --2- 172.21.15.12:0/3273221379 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5596e662f2a0 0x5596e662c3f0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f20b80ac250 tx=0x7f20b80096e0 comp rx=0 tx=0).stop 2024-09-11T00:12:02.829 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:02.824+0000 7f20d54bd740 1 -- 172.21.15.12:0/3273221379 shutdown_connections 2024-09-11T00:12:02.829 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:02.824+0000 7f20d54bd740 1 --2- 172.21.15.12:0/3273221379 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5596e6631ce0 0x5596e6632170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:12:02.829 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:02.824+0000 7f20d54bd740 1 --2- 172.21.15.12:0/3273221379 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5596e662f2a0 0x5596e662c3f0 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:12:02.829 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:02.824+0000 7f20d54bd740 1 --2- 172.21.15.12:0/3273221379 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5596e662c970 0x5596e662beb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:12:02.829 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:02.824+0000 7f20d54bd740 1 -- 172.21.15.12:0/3273221379 >> 172.21.15.12:0/3273221379 conn(0x5596e6627ac0 msgr2=0x5596e6628560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:12:02.829 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:02.824+0000 7f20d54bd740 1 -- 172.21.15.12:0/3273221379 shutdown_connections 2024-09-11T00:12:02.829 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:02.824+0000 7f20d54bd740 1 -- 172.21.15.12:0/3273221379 wait complete. 2024-09-11T00:12:02.829 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:02.825+0000 7f20d54bd740 1 -- start start 2024-09-11T00:12:02.829 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:02.825+0000 7f20d54bd740 1 Processor -- start 2024-09-11T00:12:02.830 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:02.825+0000 7f20d54bd740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5596e662c970 0x5596e663da50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:12:02.830 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:02.825+0000 7f20d54bd740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5596e6631ce0 0x5596e663c750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:12:02.830 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:02.825+0000 7f20d54bd740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5596e663cc90 0x5596e663d110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:12:02.830 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:02.825+0000 7f20d54bd740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f20cc1aed20 con 0x5596e663cc90 2024-09-11T00:12:02.830 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:02.825+0000 7f20d54bd740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f20cc1ae650 con 0x5596e6631ce0 2024-09-11T00:12:02.830 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:02.825+0000 7f20d54bd740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f20cc1ab860 con 0x5596e662c970 2024-09-11T00:12:02.830 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:02.825+0000 7f20d54bd740 10 client.0 _ll_register_callbacks cb 0x7f20b80ac120 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:12:02.830 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:02.826+0000 7f20cbe00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5596e663cc90 0x5596e663d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:12:02.830 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:02.826+0000 7f20caa00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5596e6631ce0 0x5596e663c750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:12:02.830 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:02.826+0000 7f20cbe00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5596e663cc90 0x5596e663d110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:48386/0 (socket says 172.21.15.12:48386) 2024-09-11T00:12:02.830 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:02.826+0000 7f20cbe00640 1 -- 172.21.15.12:0/3581661960 learned_addr learned my addr 172.21.15.12:0/3581661960 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:12:02.830 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:02.826+0000 7f20cb400640 1 --2- 172.21.15.12:0/3581661960 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5596e662c970 0x5596e663da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:12:02.830 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:02.826+0000 7f20caa00640 1 -- 172.21.15.12:0/3581661960 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5596e662c970 msgr2=0x5596e663da50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:12:02.831 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:02.826+0000 7f20caa00640 1 --2- 172.21.15.12:0/3581661960 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5596e662c970 0x5596e663da50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:12:02.831 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:02.826+0000 7f20caa00640 1 -- 172.21.15.12:0/3581661960 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5596e663cc90 msgr2=0x5596e663d110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:12:02.831 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:02.826+0000 7f20caa00640 1 --2- 172.21.15.12:0/3581661960 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5596e663cc90 0x5596e663d110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:12:02.831 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:02.826+0000 7f20caa00640 1 -- 172.21.15.12:0/3581661960 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5596e663b270 con 0x5596e6631ce0 2024-09-11T00:12:02.831 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:02.826+0000 7f20caa00640 1 --2- 172.21.15.12:0/3581661960 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5596e6631ce0 0x5596e663c750 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f20b80ac250 tx=0x7f20b8009f60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=417c53968cb776c6 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:12:02.831 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:02.826+0000 7f20b5600640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-09-11T00:12:02.831 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:02.826+0000 7f20b5600640 1 -- 172.21.15.12:0/3581661960 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20b801c070 con 0x5596e6631ce0 2024-09-11T00:12:02.831 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:02.827+0000 7f20d54bd740 1 -- 172.21.15.12:0/3581661960 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f20b80b0da0 con 0x5596e6631ce0 2024-09-11T00:12:02.831 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:02.827+0000 7f20b5600640 1 -- 172.21.15.12:0/3581661960 <== mon.2 v2:172.21.15.165:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9208+0+0 (secure 0 0 0) 0x7f20b80a7d50 con 0x5596e6631ce0 2024-09-11T00:12:02.831 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:02.827+0000 7f20b5600640 1 -- 172.21.15.12:0/3581661960 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f20b80b5850 con 0x5596e6631ce0 2024-09-11T00:12:02.831 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:02.827+0000 7f20d54bd740 10 client.26138 fetch_fsmap learned FSMap version 38 2024-09-11T00:12:02.831 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:02.827+0000 7f20d54bd740 1 -- 172.21.15.12:0/3581661960 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x5596e663b270 con 0x5596e6631ce0 2024-09-11T00:12:02.831 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:02.827+0000 7f20b5600640 1 -- 172.21.15.12:0/3581661960 <== mon.2 v2:172.21.15.165:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f20b80b0da0 con 0x5596e6631ce0 2024-09-11T00:12:02.831 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:02.827+0000 7f20b5600640 1 -- 172.21.15.12:0/3581661960 <== mon.2 v2:172.21.15.165:3300/0 5 ==== fsmap(e 38) ==== 8374+0+0 (secure 0 0 0) 0x7f20b80aaf90 con 0x5596e6631ce0 2024-09-11T00:12:02.832 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:02.827+0000 7f20d54bd740 10 client.26138 fetch_fsmap finished waiting for FSMap version 38 2024-09-11T00:12:02.832 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:02.827+0000 7f20d54bd740 10 client.26138 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T00:12:02.832 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:02.827+0000 7f20d54bd740 20 client.26138 populate_metadata read hostname 'smithi012' 2024-09-11T00:12:02.832 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:02.827+0000 7f20d54bd740 1 --2- 172.21.15.12:0/3581661960 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x5596e66af060 0x5596e66b1480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:12:02.832 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:02.827+0000 7f20d54bd740 4 client.26138 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T00:12:02.832 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:02.827+0000 7f20d54bd740 1 -- 172.21.15.12:0/3581661960 --> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5596e659d0b0 con 0x5596e66af060 2024-09-11T00:12:02.832 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:02.828+0000 7f20cb400640 1 --2- 172.21.15.12:0/3581661960 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x5596e66af060 0x5596e66b1480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:12:02.832 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:02.828+0000 7f20cb400640 1 --2- 172.21.15.12:0/3581661960 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x5596e66af060 0x5596e66b1480 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d2a7977f26fb3047 server_cookie=fe1be2d5b926eaa6 in_seq=0 out_seq=0 2024-09-11T00:12:02.832 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:02.828+0000 7f20b5600640 10 client.26138 ms_handle_connect on v2:172.21.15.165:6838/501312984 2024-09-11T00:12:02.861 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:02.860+0000 7f20b5600640 1 -- 172.21.15.12:0/3581661960 <== mds.0 v2:172.21.15.165:6838/501312984 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5596e659d0b0 con 0x5596e66af060 2024-09-11T00:12:02.862 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:02.860+0000 7f20b5600640 10 client.26138 handle_command_reply: tid=0 2024-09-11T00:12:02.862 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:02.860+0000 7f20d54bd740 1 client.26138 shutdown 2024-09-11T00:12:02.862 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:02.860+0000 7f20d54bd740 1 -- 172.21.15.12:0/3581661960 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5596e6631ce0 msgr2=0x5596e663c750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:12:02.862 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:02.860+0000 7f20d54bd740 1 --2- 172.21.15.12:0/3581661960 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5596e6631ce0 0x5596e663c750 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f20b80ac250 tx=0x7f20b8009f60 comp rx=0 tx=0).stop 2024-09-11T00:12:02.862 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:02.860+0000 7f20d54bd740 1 -- 172.21.15.12:0/3581661960 shutdown_connections 2024-09-11T00:12:02.862 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:02.860+0000 7f20d54bd740 1 --2- 172.21.15.12:0/3581661960 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x5596e66af060 0x5596e66b1480 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:12:02.862 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:02.860+0000 7f20d54bd740 1 --2- 172.21.15.12:0/3581661960 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5596e663cc90 0x5596e663d110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:12:02.862 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:02.860+0000 7f20d54bd740 1 --2- 172.21.15.12:0/3581661960 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5596e6631ce0 0x5596e663c750 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:12:02.862 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:02.860+0000 7f20d54bd740 1 --2- 172.21.15.12:0/3581661960 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5596e662c970 0x5596e663da50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:12:02.862 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:02.860+0000 7f20b5600640 0 client.26138 ms_handle_reset on v2:172.21.15.165:6838/501312984 2024-09-11T00:12:02.862 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:02.860+0000 7f20d54bd740 1 -- 172.21.15.12:0/3581661960 >> 172.21.15.12:0/3581661960 conn(0x5596e6627ac0 msgr2=0x5596e6632170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:12:02.862 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:02.860+0000 7f20d54bd740 1 -- 172.21.15.12:0/3581661960 shutdown_connections 2024-09-11T00:12:02.863 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:02.860+0000 7f20d54bd740 1 -- 172.21.15.12:0/3581661960 wait complete. 2024-09-11T00:12:02.863 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:02.861+0000 7f20d54bd740 20 client.26138 trim_cache size 0 max 16384 2024-09-11T00:12:02.863 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:{ 2024-09-11T00:12:02.863 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "message": "", 2024-09-11T00:12:02.863 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "return_code": 0 2024-09-11T00:12:02.863 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:} 2024-09-11T00:12:02.864 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:02.861+0000 7f20d1000640 1 -- 172.21.15.12:0/4291049155 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f20b00b5360 msgr2=0x7f20b00b7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:12:02.864 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:02.861+0000 7f20d1000640 1 --2- 172.21.15.12:0/4291049155 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f20b00b5360 0x7f20b00b7820 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f20bc000c00 tx=0x7f20bc005a20 comp rx=0 tx=0).stop 2024-09-11T00:12:02.864 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:02.861+0000 7f20d1000640 1 -- 172.21.15.12:0/4291049155 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f20cc1038e0 msgr2=0x7f20cc19b490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:12:02.864 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:02.861+0000 7f20d1000640 1 --2- 172.21.15.12:0/4291049155 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f20cc1038e0 0x7f20cc19b490 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f20b800e9f0 tx=0x7f20b800eeb0 comp rx=0 tx=0).stop 2024-09-11T00:12:02.864 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:02.861+0000 7f20d1000640 1 -- 172.21.15.12:0/4291049155 shutdown_connections 2024-09-11T00:12:02.864 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:02.861+0000 7f20d1000640 1 --2- 172.21.15.12:0/4291049155 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f20b00b5360 0x7f20b00b7820 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:12:02.864 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:02.861+0000 7f20d1000640 1 --2- 172.21.15.12:0/4291049155 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f20cc1042a0 0x7f20cc19c0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:12:02.865 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:02.861+0000 7f20d1000640 1 --2- 172.21.15.12:0/4291049155 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f20cc1038e0 0x7f20cc19b490 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:12:02.865 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:02.861+0000 7f20d1000640 1 --2- 172.21.15.12:0/4291049155 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f20cc102790 0x7f20cc19ae40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:12:02.865 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:02.861+0000 7f20d1000640 1 -- 172.21.15.12:0/4291049155 >> 172.21.15.12:0/4291049155 conn(0x7f20cc0fe030 msgr2=0x7f20cc109000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:12:02.865 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:02.861+0000 7f20d1000640 1 -- 172.21.15.12:0/4291049155 shutdown_connections 2024-09-11T00:12:02.865 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:02.861+0000 7f20d1000640 1 -- 172.21.15.12:0/4291049155 wait complete. 2024-09-11T00:12:03.634 INFO:tasks.workunit.client.0.smithi012.stdout: 1 128980 1.45 MB/sec execute 120 sec latency 2385.278 ms 2024-09-11T00:12:04.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:12:03 smithi165 ceph-mon[35356]: pgmap v548: 129 pgs: 129 active+clean; 33 MiB data, 675 MiB used, 1.0 TiB / 1.0 TiB avail; 9.4 MiB/s rd, 174 KiB/s wr, 90 op/s 2024-09-11T00:12:04.062 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:12:03 smithi012 ceph-mon[30333]: pgmap v548: 129 pgs: 129 active+clean; 33 MiB data, 675 MiB used, 1.0 TiB / 1.0 TiB avail; 9.4 MiB/s rd, 174 KiB/s wr, 90 op/s 2024-09-11T00:12:04.080 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:12:03 smithi178 ceph-mon[35919]: pgmap v548: 129 pgs: 129 active+clean; 33 MiB data, 675 MiB used, 1.0 TiB / 1.0 TiB avail; 9.4 MiB/s rd, 174 KiB/s wr, 90 op/s 2024-09-11T00:12:04.633 INFO:tasks.workunit.client.0.smithi012.stdout: 1 131509 1.57 MB/sec execute 121 sec latency 10.028 ms 2024-09-11T00:12:05.634 INFO:tasks.workunit.client.0.smithi012.stdout: 1 133055 1.61 MB/sec execute 122 sec latency 11.048 ms 2024-09-11T00:12:06.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:12:05 smithi165 ceph-mon[35356]: pgmap v549: 129 pgs: 129 active+clean; 33 MiB data, 675 MiB used, 1.0 TiB / 1.0 TiB avail; 9.4 MiB/s rd, 175 KiB/s wr, 87 op/s 2024-09-11T00:12:06.063 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:12:05 smithi012 ceph-mon[30333]: pgmap v549: 129 pgs: 129 active+clean; 33 MiB data, 675 MiB used, 1.0 TiB / 1.0 TiB avail; 9.4 MiB/s rd, 175 KiB/s wr, 87 op/s 2024-09-11T00:12:06.080 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:12:05 smithi178 ceph-mon[35919]: pgmap v549: 129 pgs: 129 active+clean; 33 MiB data, 675 MiB used, 1.0 TiB / 1.0 TiB avail; 9.4 MiB/s rd, 175 KiB/s wr, 87 op/s 2024-09-11T00:12:06.634 INFO:tasks.workunit.client.0.smithi012.stdout: 1 134811 1.69 MB/sec execute 123 sec latency 342.056 ms 2024-09-11T00:12:07.634 INFO:tasks.workunit.client.0.smithi012.stdout: 1 134918 1.68 MB/sec execute 124 sec latency 685.595 ms 2024-09-11T00:12:08.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:12:07 smithi165 ceph-mon[35356]: pgmap v550: 129 pgs: 129 active+clean; 35 MiB data, 737 MiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 706 KiB/s wr, 163 op/s 2024-09-11T00:12:08.064 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:12:07 smithi012 ceph-mon[30333]: pgmap v550: 129 pgs: 129 active+clean; 35 MiB data, 737 MiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 706 KiB/s wr, 163 op/s 2024-09-11T00:12:08.080 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:12:07 smithi178 ceph-mon[35919]: pgmap v550: 129 pgs: 129 active+clean; 35 MiB data, 737 MiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 706 KiB/s wr, 163 op/s 2024-09-11T00:12:08.634 INFO:tasks.workunit.client.0.smithi012.stdout: 1 136004 1.70 MB/sec execute 125 sec latency 48.697 ms 2024-09-11T00:12:09.634 INFO:tasks.workunit.client.0.smithi012.stdout: 1 138357 1.80 MB/sec execute 126 sec latency 78.744 ms 2024-09-11T00:12:10.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:12:09 smithi165 ceph-mon[35356]: pgmap v551: 129 pgs: 129 active+clean; 35 MiB data, 738 MiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 721 KiB/s wr, 161 op/s 2024-09-11T00:12:10.062 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:12:09 smithi012 ceph-mon[30333]: pgmap v551: 129 pgs: 129 active+clean; 35 MiB data, 738 MiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 721 KiB/s wr, 161 op/s 2024-09-11T00:12:10.080 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:12:09 smithi178 ceph-mon[35919]: pgmap v551: 129 pgs: 129 active+clean; 35 MiB data, 738 MiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 721 KiB/s wr, 161 op/s 2024-09-11T00:12:10.634 INFO:tasks.workunit.client.0.smithi012.stdout: 1 138464 1.79 MB/sec execute 127 sec latency 559.766 ms 2024-09-11T00:12:11.634 INFO:tasks.workunit.client.0.smithi012.stdout: 1 138902 1.79 MB/sec execute 128 sec latency 636.729 ms 2024-09-11T00:12:12.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:12:11 smithi165 ceph-mon[35356]: pgmap v552: 129 pgs: 129 active+clean; 36 MiB data, 746 MiB used, 1.0 TiB / 1.0 TiB avail; 39 MiB/s rd, 905 KiB/s wr, 301 op/s 2024-09-11T00:12:12.062 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:12:11 smithi012 ceph-mon[30333]: pgmap v552: 129 pgs: 129 active+clean; 36 MiB data, 746 MiB used, 1.0 TiB / 1.0 TiB avail; 39 MiB/s rd, 905 KiB/s wr, 301 op/s 2024-09-11T00:12:12.080 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:12:11 smithi178 ceph-mon[35919]: pgmap v552: 129 pgs: 129 active+clean; 36 MiB data, 746 MiB used, 1.0 TiB / 1.0 TiB avail; 39 MiB/s rd, 905 KiB/s wr, 301 op/s 2024-09-11T00:12:12.634 INFO:tasks.workunit.client.0.smithi012.stdout: 1 141268 1.87 MB/sec execute 129 sec latency 9.328 ms 2024-09-11T00:12:13.634 INFO:tasks.workunit.client.0.smithi012.stdout: 1 141999 1.90 MB/sec execute 130 sec latency 553.943 ms 2024-09-11T00:12:14.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:12:13 smithi165 ceph-mon[35356]: pgmap v553: 129 pgs: 129 active+clean; 38 MiB data, 753 MiB used, 1.0 TiB / 1.0 TiB avail; 30 MiB/s rd, 943 KiB/s wr, 280 op/s 2024-09-11T00:12:14.063 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:12:13 smithi012 ceph-mon[30333]: pgmap v553: 129 pgs: 129 active+clean; 38 MiB data, 753 MiB used, 1.0 TiB / 1.0 TiB avail; 30 MiB/s rd, 943 KiB/s wr, 280 op/s 2024-09-11T00:12:14.080 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:12:13 smithi178 ceph-mon[35919]: pgmap v553: 129 pgs: 129 active+clean; 38 MiB data, 753 MiB used, 1.0 TiB / 1.0 TiB avail; 30 MiB/s rd, 943 KiB/s wr, 280 op/s 2024-09-11T00:12:14.634 INFO:tasks.workunit.client.0.smithi012.stdout: 1 142819 1.90 MB/sec execute 131 sec latency 158.111 ms 2024-09-11T00:12:15.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:12:14 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T00:12:15.063 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:12:14 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T00:12:15.080 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:12:14 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T00:12:15.634 INFO:tasks.workunit.client.0.smithi012.stdout: 1 145431 2.01 MB/sec execute 132 sec latency 12.820 ms 2024-09-11T00:12:16.001 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:15.999+0000 7f162ea00640 1 -- 172.21.15.12:0/1395757249 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1628107d00 msgr2=0x7f162810a0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:12:16.001 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:15.999+0000 7f162ea00640 1 --2- 172.21.15.12:0/1395757249 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1628107d00 0x7f162810a0f0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f1614009900 tx=0x7f161402a3f0 comp rx=0 tx=0).stop 2024-09-11T00:12:16.001 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:15.999+0000 7f162ea00640 1 -- 172.21.15.12:0/1395757249 shutdown_connections 2024-09-11T00:12:16.002 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:15.999+0000 7f162ea00640 1 --2- 172.21.15.12:0/1395757249 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f162810a6c0 0x7f162810cb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:12:16.002 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:15.999+0000 7f162ea00640 1 --2- 172.21.15.12:0/1395757249 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1628107d00 0x7f162810a0f0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:12:16.002 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:15.999+0000 7f162ea00640 1 --2- 172.21.15.12:0/1395757249 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f162806b300 0x7f16281077c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:12:16.002 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:15.999+0000 7f162ea00640 1 -- 172.21.15.12:0/1395757249 >> 172.21.15.12:0/1395757249 conn(0x7f16280fcf30 msgr2=0x7f16280ff370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:12:16.002 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:15.999+0000 7f162ea00640 1 -- 172.21.15.12:0/1395757249 shutdown_connections 2024-09-11T00:12:16.002 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:15.999+0000 7f162ea00640 1 -- 172.21.15.12:0/1395757249 wait complete. 2024-09-11T00:12:16.003 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:16.000+0000 7f162ea00640 1 Processor -- start 2024-09-11T00:12:16.003 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:16.000+0000 7f162ea00640 1 -- start start 2024-09-11T00:12:16.003 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:16.001+0000 7f162ea00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f162806b300 0x7f162819adc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:12:16.003 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:16.001+0000 7f162ea00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1628107d00 0x7f162819b410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:12:16.003 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:16.001+0000 7f162ea00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f162810a6c0 0x7f162819c040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:12:16.003 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:16.001+0000 7f162ea00640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f1628110640 con 0x7f162810a6c0 2024-09-11T00:12:16.004 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:16.001+0000 7f162ea00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f16281104c0 con 0x7f162806b300 2024-09-11T00:12:16.004 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:16.001+0000 7f162ea00640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f16281107c0 con 0x7f1628107d00 2024-09-11T00:12:16.004 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:16.001+0000 7f162cc00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1628107d00 0x7f162819b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:12:16.004 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:16.001+0000 7f162cc00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1628107d00 0x7f162819b410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.12:39154/0 (socket says 172.21.15.12:39154) 2024-09-11T00:12:16.004 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:16.001+0000 7f162cc00640 1 -- 172.21.15.12:0/1175663471 learned_addr learned my addr 172.21.15.12:0/1175663471 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:12:16.004 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:16.001+0000 7f162d600640 1 --2- 172.21.15.12:0/1175663471 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f162806b300 0x7f162819adc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:12:16.004 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:16.001+0000 7f162e000640 1 --2- 172.21.15.12:0/1175663471 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f162810a6c0 0x7f162819c040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:12:16.005 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:16.001+0000 7f162cc00640 1 -- 172.21.15.12:0/1175663471 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f162806b300 msgr2=0x7f162819adc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:12:16.005 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:16.001+0000 7f162cc00640 1 --2- 172.21.15.12:0/1175663471 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f162806b300 0x7f162819adc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:12:16.005 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:16.001+0000 7f162cc00640 1 -- 172.21.15.12:0/1175663471 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f162810a6c0 msgr2=0x7f162819c040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:12:16.005 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:16.001+0000 7f162cc00640 1 --2- 172.21.15.12:0/1175663471 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f162810a6c0 0x7f162819c040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:12:16.005 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:16.001+0000 7f162cc00640 1 -- 172.21.15.12:0/1175663471 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f162819c900 con 0x7f1628107d00 2024-09-11T00:12:16.006 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:16.001+0000 7f162d600640 1 --2- 172.21.15.12:0/1175663471 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f162806b300 0x7f162819adc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:12:16.006 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:16.002+0000 7f162cc00640 1 --2- 172.21.15.12:0/1175663471 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1628107d00 0x7f162819b410 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f1614031500 tx=0x7f1614003c10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:12:16.006 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:16.002+0000 7f1626000640 1 -- 172.21.15.12:0/1175663471 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f161403f070 con 0x7f1628107d00 2024-09-11T00:12:16.006 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:16.002+0000 7f1626000640 1 -- 172.21.15.12:0/1175663471 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1614031c50 con 0x7f1628107d00 2024-09-11T00:12:16.006 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:16.002+0000 7f162ea00640 1 -- 172.21.15.12:0/1175663471 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f162819cb90 con 0x7f1628107d00 2024-09-11T00:12:16.006 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:16.002+0000 7f1626000640 1 -- 172.21.15.12:0/1175663471 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1614035030 con 0x7f1628107d00 2024-09-11T00:12:16.006 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:16.002+0000 7f162ea00640 1 -- 172.21.15.12:0/1175663471 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16281a32c0 con 0x7f1628107d00 2024-09-11T00:12:16.008 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:16.006+0000 7f1632df5740 1 Processor -- start 2024-09-11T00:12:16.008 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:16.006+0000 7f1632df5740 1 -- start start 2024-09-11T00:12:16.008 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:16.006+0000 7f1626000640 1 -- 172.21.15.12:0/1175663471 <== mon.1 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7f161403b460 con 0x7f1628107d00 2024-09-11T00:12:16.009 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:16.006+0000 7f1632df5740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b1054ae970 0x55b1054b0d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:12:16.009 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:16.006+0000 7f1632df5740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b1054b12a0 0x55b1054b3690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:12:16.009 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:16.006+0000 7f1632df5740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b1054b3ce0 0x55b1054b61a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:12:16.009 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:16.006+0000 7f1632df5740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55b1054216b0 con 0x55b1054b12a0 2024-09-11T00:12:16.009 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:16.006+0000 7f1632df5740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55b105410ae0 con 0x55b1054b3ce0 2024-09-11T00:12:16.009 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:16.006+0000 7f1632df5740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55b1054148f0 con 0x55b1054ae970 2024-09-11T00:12:16.010 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:16.007+0000 7f162e000640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b1054ae970 0x55b1054b0d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:12:16.010 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:16.007+0000 7f162cc00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b1054b3ce0 0x55b1054b61a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:12:16.011 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:16.007+0000 7f162d600640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b1054b12a0 0x55b1054b3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:12:16.011 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:16.007+0000 7f162e000640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b1054ae970 0x55b1054b0d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.12:39168/0 (socket says 172.21.15.12:39168) 2024-09-11T00:12:16.011 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:16.007+0000 7f162e000640 1 -- 172.21.15.12:0/4290719209 learned_addr learned my addr 172.21.15.12:0/4290719209 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:12:16.011 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:16.007+0000 7f162e000640 1 -- 172.21.15.12:0/4290719209 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b1054b3ce0 msgr2=0x55b1054b61a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:12:16.011 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:16.007+0000 7f162e000640 1 --2- 172.21.15.12:0/4290719209 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b1054b3ce0 0x55b1054b61a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:12:16.011 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:16.007+0000 7f162e000640 1 -- 172.21.15.12:0/4290719209 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b1054b12a0 msgr2=0x55b1054b3690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:12:16.011 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:16.007+0000 7f162e000640 1 --2- 172.21.15.12:0/4290719209 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b1054b12a0 0x55b1054b3690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:12:16.011 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:16.007+0000 7f162e000640 1 -- 172.21.15.12:0/4290719209 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b1053dd780 con 0x55b1054ae970 2024-09-11T00:12:16.011 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:16.007+0000 7f162d600640 1 --2- 172.21.15.12:0/4290719209 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b1054b12a0 0x55b1054b3690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T00:12:16.011 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:16.007+0000 7f162e000640 1 --2- 172.21.15.12:0/4290719209 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b1054ae970 0x55b1054b0d60 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f1620009b20 tx=0x7f16200113e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e51e7cd954b6dee1 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:12:16.011 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:16.008+0000 7f161a600640 1 -- 172.21.15.12:0/4290719209 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1620011e10 con 0x55b1054ae970 2024-09-11T00:12:16.011 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:16.008+0000 7f161a600640 1 -- 172.21.15.12:0/4290719209 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1620012900 con 0x55b1054ae970 2024-09-11T00:12:16.012 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:16.008+0000 7f161a600640 1 -- 172.21.15.12:0/4290719209 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f162000f940 con 0x55b1054ae970 2024-09-11T00:12:16.012 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:16.008+0000 7f1626000640 1 --2- 172.21.15.12:0/1175663471 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f16100b5360 0x7f16100b7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:12:16.012 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:16.008+0000 7f1626000640 1 -- 172.21.15.12:0/1175663471 <== mon.1 v2:172.21.15.178:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9208+0+0 (secure 0 0 0) 0x7f1614106ac0 con 0x7f1628107d00 2024-09-11T00:12:16.012 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:16.009+0000 7f1632df5740 1 -- 172.21.15.12:0/4290719209 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b1054ae970 msgr2=0x55b1054b0d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:12:16.012 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:16.009+0000 7f1632df5740 1 --2- 172.21.15.12:0/4290719209 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b1054ae970 0x55b1054b0d60 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f1620009b20 tx=0x7f16200113e0 comp rx=0 tx=0).stop 2024-09-11T00:12:16.014 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:16.009+0000 7f162d600640 1 --2- 172.21.15.12:0/1175663471 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f16100b5360 0x7f16100b7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:12:16.014 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:16.009+0000 7f1632df5740 1 -- 172.21.15.12:0/4290719209 shutdown_connections 2024-09-11T00:12:16.014 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:16.009+0000 7f1632df5740 1 --2- 172.21.15.12:0/4290719209 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b1054b3ce0 0x55b1054b61a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:12:16.014 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:16.009+0000 7f1632df5740 1 --2- 172.21.15.12:0/4290719209 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b1054b12a0 0x55b1054b3690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:12:16.015 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:16.009+0000 7f1632df5740 1 --2- 172.21.15.12:0/4290719209 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b1054ae970 0x55b1054b0d60 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:12:16.015 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:16.009+0000 7f1632df5740 1 -- 172.21.15.12:0/4290719209 >> 172.21.15.12:0/4290719209 conn(0x55b1054a9ac0 msgr2=0x55b1054aa850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:12:16.015 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:16.010+0000 7f162d600640 1 --2- 172.21.15.12:0/1175663471 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f16100b5360 0x7f16100b7820 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f161c005fd0 tx=0x7f161c009760 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:12:16.015 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:16.010+0000 7f1632df5740 1 -- 172.21.15.12:0/4290719209 shutdown_connections 2024-09-11T00:12:16.015 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:16.010+0000 7f1632df5740 1 -- 172.21.15.12:0/4290719209 wait complete. 2024-09-11T00:12:16.015 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:16.010+0000 7f1632df5740 1 -- start start 2024-09-11T00:12:16.015 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:16.010+0000 7f1632df5740 1 Processor -- start 2024-09-11T00:12:16.015 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:16.011+0000 7f1632df5740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b1054ae970 0x55b1054b80e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:12:16.015 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:16.011+0000 7f1632df5740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b1054b12a0 0x55b1054b8780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:12:16.015 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:16.011+0000 7f1632df5740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b1054b3ce0 0x55b1054b9350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:12:16.015 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:16.011+0000 7f1632df5740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55b105348ab0 con 0x55b1054ae970 2024-09-11T00:12:16.015 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:16.011+0000 7f1632df5740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55b1053b7570 con 0x55b1054b12a0 2024-09-11T00:12:16.016 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:16.011+0000 7f1632df5740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55b1054b6cc0 con 0x55b1054b3ce0 2024-09-11T00:12:16.016 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:16.011+0000 7f1632df5740 10 client.0 _ll_register_callbacks cb 0x55b1053c6100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:12:16.016 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:16.011+0000 7f162d600640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b1054ae970 0x55b1054b80e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:12:16.016 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:16.011+0000 7f162cc00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b1054b12a0 0x55b1054b8780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:12:16.016 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:16.011+0000 7f162e000640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b1054b3ce0 0x55b1054b9350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:12:16.016 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:16.011+0000 7f162cc00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b1054b12a0 0x55b1054b8780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.12:55860/0 (socket says 172.21.15.12:55860) 2024-09-11T00:12:16.016 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:16.011+0000 7f162cc00640 1 -- 172.21.15.12:0/2261739591 learned_addr learned my addr 172.21.15.12:0/2261739591 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:12:16.016 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:16.012+0000 7f162cc00640 1 -- 172.21.15.12:0/2261739591 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b1054b3ce0 msgr2=0x55b1054b9350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:12:16.016 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:16.012+0000 7f162cc00640 1 --2- 172.21.15.12:0/2261739591 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b1054b3ce0 0x55b1054b9350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:12:16.016 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:16.012+0000 7f162cc00640 1 -- 172.21.15.12:0/2261739591 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b1054ae970 msgr2=0x55b1054b80e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:12:16.016 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:16.012+0000 7f162cc00640 1 --2- 172.21.15.12:0/2261739591 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b1054ae970 0x55b1054b80e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:12:16.016 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:16.012+0000 7f162cc00640 1 -- 172.21.15.12:0/2261739591 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b105172790 con 0x55b1054b12a0 2024-09-11T00:12:16.017 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:16.012+0000 7f162cc00640 1 --2- 172.21.15.12:0/2261739591 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b1054b12a0 0x55b1054b8780 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f1614000c00 tx=0x7f161403b8e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ba8c43a8332d94c0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:12:16.017 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:16.012+0000 7f160ea00640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-09-11T00:12:16.017 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:16.012+0000 7f160ea00640 1 -- 172.21.15.12:0/2261739591 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1614049070 con 0x55b1054b12a0 2024-09-11T00:12:16.017 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:16.012+0000 7f160ea00640 1 -- 172.21.15.12:0/2261739591 <== mon.2 v2:172.21.15.165:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9208+0+0 (secure 0 0 0) 0x7f16140b42f0 con 0x55b1054b12a0 2024-09-11T00:12:16.017 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:16.012+0000 7f1632df5740 1 -- 172.21.15.12:0/2261739591 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1620012900 con 0x55b1054b12a0 2024-09-11T00:12:16.017 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:16.012+0000 7f160ea00640 1 -- 172.21.15.12:0/2261739591 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f16140c8d50 con 0x55b1054b12a0 2024-09-11T00:12:16.017 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:16.012+0000 7f1632df5740 10 client.26150 fetch_fsmap learned FSMap version 38 2024-09-11T00:12:16.017 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:16.012+0000 7f1632df5740 1 -- 172.21.15.12:0/2261739591 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x55b1051eb620 con 0x55b1054b12a0 2024-09-11T00:12:16.019 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:16.013+0000 7f160ea00640 1 -- 172.21.15.12:0/2261739591 <== mon.2 v2:172.21.15.165:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1620012900 con 0x55b1054b12a0 2024-09-11T00:12:16.019 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:16.013+0000 7f160ea00640 1 -- 172.21.15.12:0/2261739591 <== mon.2 v2:172.21.15.165:3300/0 5 ==== fsmap(e 38) ==== 8374+0+0 (secure 0 0 0) 0x7f16140d5fb0 con 0x55b1054b12a0 2024-09-11T00:12:16.019 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:16.013+0000 7f1632df5740 10 client.26150 fetch_fsmap finished waiting for FSMap version 38 2024-09-11T00:12:16.019 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:16.013+0000 7f1632df5740 10 client.26150 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T00:12:16.019 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:16.013+0000 7f1632df5740 20 client.26150 populate_metadata read hostname 'smithi012' 2024-09-11T00:12:16.019 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:16.013+0000 7f1632df5740 1 --2- 172.21.15.12:0/2261739591 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55b1054c3850 0x55b1054c5c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:12:16.019 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:16.013+0000 7f1632df5740 4 client.26150 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T00:12:16.019 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:16.013+0000 7f1632df5740 1 -- 172.21.15.12:0/2261739591 --> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b105423600 con 0x55b1054c3850 2024-09-11T00:12:16.020 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:16.014+0000 7f162d600640 1 --2- 172.21.15.12:0/2261739591 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55b1054c3850 0x55b1054c5c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:12:16.020 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:16.014+0000 7f162d600640 1 --2- 172.21.15.12:0/2261739591 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55b1054c3850 0x55b1054c5c60 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=23d2239262f2a63e server_cookie=1cf8d3373402d3ea in_seq=0 out_seq=0 2024-09-11T00:12:16.020 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:16.014+0000 7f160ea00640 10 client.26150 ms_handle_connect on v2:172.21.15.165:6838/501312984 2024-09-11T00:12:16.020 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:16.015+0000 7f160ea00640 1 -- 172.21.15.12:0/2261739591 <== mds.0 v2:172.21.15.165:6838/501312984 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55b105423600 con 0x55b1054c3850 2024-09-11T00:12:16.020 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:16.015+0000 7f160ea00640 10 client.26150 handle_command_reply: tid=0 2024-09-11T00:12:16.020 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:16.015+0000 7f1632df5740 1 client.26150 shutdown 2024-09-11T00:12:16.020 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:16.015+0000 7f1632df5740 1 -- 172.21.15.12:0/2261739591 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b1054b12a0 msgr2=0x55b1054b8780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:12:16.020 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:16.015+0000 7f1632df5740 1 --2- 172.21.15.12:0/2261739591 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b1054b12a0 0x55b1054b8780 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f1614000c00 tx=0x7f161403b8e0 comp rx=0 tx=0).stop 2024-09-11T00:12:16.020 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:16.015+0000 7f1632df5740 1 -- 172.21.15.12:0/2261739591 shutdown_connections 2024-09-11T00:12:16.020 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:16.015+0000 7f1632df5740 1 --2- 172.21.15.12:0/2261739591 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55b1054c3850 0x55b1054c5c60 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:12:16.020 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:16.015+0000 7f1632df5740 1 --2- 172.21.15.12:0/2261739591 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b1054b3ce0 0x55b1054b9350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:12:16.021 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:16.015+0000 7f1632df5740 1 --2- 172.21.15.12:0/2261739591 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b1054b12a0 0x55b1054b8780 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:12:16.021 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:16.015+0000 7f1632df5740 1 --2- 172.21.15.12:0/2261739591 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b1054ae970 0x55b1054b80e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:12:16.021 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:16.015+0000 7f1632df5740 1 -- 172.21.15.12:0/2261739591 >> 172.21.15.12:0/2261739591 conn(0x55b1054a9ac0 msgr2=0x55b1054a8bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:12:16.021 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:16.015+0000 7f160ea00640 0 client.26150 ms_handle_reset on v2:172.21.15.165:6838/501312984 2024-09-11T00:12:16.021 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:16.016+0000 7f1632df5740 1 -- 172.21.15.12:0/2261739591 shutdown_connections 2024-09-11T00:12:16.021 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:16.016+0000 7f1632df5740 1 -- 172.21.15.12:0/2261739591 wait complete. 2024-09-11T00:12:16.021 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:16.016+0000 7f1632df5740 20 client.26150 trim_cache size 0 max 16384 2024-09-11T00:12:16.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:12:15 smithi165 ceph-mon[35356]: pgmap v554: 129 pgs: 129 active+clean; 38 MiB data, 758 MiB used, 1.0 TiB / 1.0 TiB avail; 30 MiB/s rd, 866 KiB/s wr, 255 op/s 2024-09-11T00:12:16.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:12:15 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:12:16.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:12:15 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T00:12:16.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:12:15 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:12:16.036 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:16.034+0000 7f1632df5740 1 Processor -- start 2024-09-11T00:12:16.037 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:16.034+0000 7f1632df5740 1 -- start start 2024-09-11T00:12:16.037 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:16.034+0000 7f1632df5740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b1054ae970 0x55b1054b80e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:12:16.037 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:16.034+0000 7f1632df5740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b1054b12a0 0x55b1054b8620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:12:16.037 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:16.035+0000 7f1632df5740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b1054b3ce0 0x55b1054b40c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:12:16.037 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:16.035+0000 7f1632df5740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f16281ad740 con 0x55b1054ae970 2024-09-11T00:12:16.037 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:16.035+0000 7f1632df5740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f16281af2b0 con 0x55b1054b12a0 2024-09-11T00:12:16.037 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:16.035+0000 7f1632df5740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f16281ade10 con 0x55b1054b3ce0 2024-09-11T00:12:16.038 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:16.035+0000 7f162e000640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b1054b3ce0 0x55b1054b40c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:12:16.038 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:16.035+0000 7f162d600640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b1054ae970 0x55b1054b80e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:12:16.038 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:16.035+0000 7f162e000640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b1054b3ce0 0x55b1054b40c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.12:39178/0 (socket says 172.21.15.12:39178) 2024-09-11T00:12:16.038 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:16.035+0000 7f162e000640 1 -- 172.21.15.12:0/2912839061 learned_addr learned my addr 172.21.15.12:0/2912839061 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:12:16.039 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:16.035+0000 7f162cc00640 1 --2- 172.21.15.12:0/2912839061 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b1054b12a0 0x55b1054b8620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:12:16.039 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:16.035+0000 7f162e000640 1 -- 172.21.15.12:0/2912839061 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b1054b12a0 msgr2=0x55b1054b8620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:12:16.039 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:16.035+0000 7f162e000640 1 --2- 172.21.15.12:0/2912839061 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b1054b12a0 0x55b1054b8620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:12:16.039 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:16.035+0000 7f162e000640 1 -- 172.21.15.12:0/2912839061 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b1054ae970 msgr2=0x55b1054b80e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:12:16.039 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:16.035+0000 7f162e000640 1 --2- 172.21.15.12:0/2912839061 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b1054ae970 0x55b1054b80e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:12:16.039 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:16.035+0000 7f162e000640 1 -- 172.21.15.12:0/2912839061 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b1054b10d0 con 0x55b1054b3ce0 2024-09-11T00:12:16.039 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:16.036+0000 7f162e000640 1 --2- 172.21.15.12:0/2912839061 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b1054b3ce0 0x55b1054b40c0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f1620009b20 tx=0x7f1620005db0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8dbb5e46d969f273 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:12:16.039 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:16.036+0000 7f1619c00640 1 -- 172.21.15.12:0/2912839061 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1620029070 con 0x55b1054b3ce0 2024-09-11T00:12:16.039 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:16.036+0000 7f1619c00640 1 -- 172.21.15.12:0/2912839061 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1620011e10 con 0x55b1054b3ce0 2024-09-11T00:12:16.039 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:16.036+0000 7f1619c00640 1 -- 172.21.15.12:0/2912839061 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16200048e0 con 0x55b1054b3ce0 2024-09-11T00:12:16.042 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:16.037+0000 7f1632df5740 1 -- 172.21.15.12:0/2912839061 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b1054b3ce0 msgr2=0x55b1054b40c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:12:16.042 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:16.037+0000 7f1632df5740 1 --2- 172.21.15.12:0/2912839061 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b1054b3ce0 0x55b1054b40c0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f1620009b20 tx=0x7f1620005db0 comp rx=0 tx=0).stop 2024-09-11T00:12:16.042 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:16.037+0000 7f1632df5740 1 -- 172.21.15.12:0/2912839061 shutdown_connections 2024-09-11T00:12:16.042 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:16.037+0000 7f1632df5740 1 --2- 172.21.15.12:0/2912839061 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b1054b3ce0 0x55b1054b40c0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:12:16.042 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:16.037+0000 7f1632df5740 1 --2- 172.21.15.12:0/2912839061 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b1054b12a0 0x55b1054b8620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:12:16.042 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:16.037+0000 7f1632df5740 1 --2- 172.21.15.12:0/2912839061 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b1054ae970 0x55b1054b80e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:12:16.042 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:16.037+0000 7f1632df5740 1 -- 172.21.15.12:0/2912839061 >> 172.21.15.12:0/2912839061 conn(0x55b1054a9ac0 msgr2=0x55b1054aa320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:12:16.042 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:16.037+0000 7f1632df5740 1 -- 172.21.15.12:0/2912839061 shutdown_connections 2024-09-11T00:12:16.043 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:16.037+0000 7f1632df5740 1 -- 172.21.15.12:0/2912839061 wait complete. 2024-09-11T00:12:16.043 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:16.037+0000 7f1632df5740 1 -- start start 2024-09-11T00:12:16.043 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:16.038+0000 7f1632df5740 1 Processor -- start 2024-09-11T00:12:16.043 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:16.038+0000 7f1632df5740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b1054ae970 0x55b1054bf930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:12:16.043 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:16.038+0000 7f1632df5740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b1054b12a0 0x55b1054be630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:12:16.043 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:16.038+0000 7f1632df5740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b1054b3ce0 0x55b1054beb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:12:16.043 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:16.038+0000 7f1632df5740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f16281aebe0 con 0x55b1054b12a0 2024-09-11T00:12:16.043 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:16.038+0000 7f1632df5740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f16281ae510 con 0x55b1054b3ce0 2024-09-11T00:12:16.043 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:16.038+0000 7f1632df5740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f16281ab720 con 0x55b1054ae970 2024-09-11T00:12:16.043 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:16.038+0000 7f1632df5740 10 client.0 _ll_register_callbacks cb 0x7f161410a280 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:12:16.043 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:16.038+0000 7f162cc00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b1054b12a0 0x55b1054be630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:12:16.044 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:16.039+0000 7f162d600640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b1054ae970 0x55b1054bf930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:12:16.044 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:16.039+0000 7f162cc00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b1054b12a0 0x55b1054be630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:35280/0 (socket says 172.21.15.12:35280) 2024-09-11T00:12:16.044 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:16.039+0000 7f162cc00640 1 -- 172.21.15.12:0/1407785297 learned_addr learned my addr 172.21.15.12:0/1407785297 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:12:16.044 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:16.039+0000 7f162e000640 1 --2- 172.21.15.12:0/1407785297 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b1054b3ce0 0x55b1054beb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:12:16.044 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:16.039+0000 7f162cc00640 1 -- 172.21.15.12:0/1407785297 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b1054ae970 msgr2=0x55b1054bf930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:12:16.044 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:16.039+0000 7f162cc00640 1 --2- 172.21.15.12:0/1407785297 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b1054ae970 0x55b1054bf930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:12:16.044 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:16.039+0000 7f162cc00640 1 -- 172.21.15.12:0/1407785297 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b1054b3ce0 msgr2=0x55b1054beb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:12:16.044 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:16.039+0000 7f162cc00640 1 --2- 172.21.15.12:0/1407785297 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b1054b3ce0 0x55b1054beb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:12:16.044 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:16.039+0000 7f162cc00640 1 -- 172.21.15.12:0/1407785297 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b1054bf130 con 0x55b1054b12a0 2024-09-11T00:12:16.044 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:16.039+0000 7f162e000640 1 --2- 172.21.15.12:0/1407785297 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b1054b3ce0 0x55b1054beb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T00:12:16.044 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:16.039+0000 7f162d600640 1 --2- 172.21.15.12:0/1407785297 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b1054ae970 0x55b1054bf930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:12:16.044 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:16.039+0000 7f162cc00640 1 --2- 172.21.15.12:0/1407785297 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b1054b12a0 0x55b1054be630 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f16140d1b50 tx=0x7f1614106e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1988a4784dad275 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:12:16.044 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:16.039+0000 7f160ea00640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-09-11T00:12:16.045 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:16.039+0000 7f160ea00640 1 -- 172.21.15.12:0/1407785297 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1614049040 con 0x55b1054b12a0 2024-09-11T00:12:16.045 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:16.040+0000 7f160ea00640 1 -- 172.21.15.12:0/1407785297 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9208+0+0 (secure 0 0 0) 0x7f16140b40f0 con 0x55b1054b12a0 2024-09-11T00:12:16.045 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:16.040+0000 7f1632df5740 1 -- 172.21.15.12:0/1407785297 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1620011e10 con 0x55b1054b12a0 2024-09-11T00:12:16.045 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:16.040+0000 7f160ea00640 1 -- 172.21.15.12:0/1407785297 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f16140cc620 con 0x55b1054b12a0 2024-09-11T00:12:16.045 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:16.040+0000 7f1632df5740 10 client.16137 fetch_fsmap learned FSMap version 38 2024-09-11T00:12:16.045 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:16.040+0000 7f1632df5740 1 -- 172.21.15.12:0/1407785297 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x55b1054be1c0 con 0x55b1054b12a0 2024-09-11T00:12:16.045 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:16.040+0000 7f160ea00640 1 -- 172.21.15.12:0/1407785297 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1620011e10 con 0x55b1054b12a0 2024-09-11T00:12:16.046 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:16.040+0000 7f160ea00640 1 -- 172.21.15.12:0/1407785297 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 38) ==== 8374+0+0 (secure 0 0 0) 0x7f16140cc110 con 0x55b1054b12a0 2024-09-11T00:12:16.046 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:16.040+0000 7f1632df5740 10 client.16137 fetch_fsmap finished waiting for FSMap version 38 2024-09-11T00:12:16.046 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:16.040+0000 7f1632df5740 10 client.16137 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T00:12:16.046 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:16.040+0000 7f1632df5740 20 client.16137 populate_metadata read hostname 'smithi012' 2024-09-11T00:12:16.046 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:16.041+0000 7f1632df5740 1 --2- 172.21.15.12:0/1407785297 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55b105530c50 0x55b105533070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:12:16.046 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:16.041+0000 7f1632df5740 4 client.16137 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T00:12:16.046 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:16.041+0000 7f1632df5740 1 -- 172.21.15.12:0/1407785297 --> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55b10541f0b0 con 0x55b105530c50 2024-09-11T00:12:16.047 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:16.041+0000 7f162d600640 1 --2- 172.21.15.12:0/1407785297 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55b105530c50 0x55b105533070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:12:16.047 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:16.042+0000 7f162d600640 1 --2- 172.21.15.12:0/1407785297 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55b105530c50 0x55b105533070 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=49432bb60cca165b server_cookie=a890332e83e25109 in_seq=0 out_seq=0 2024-09-11T00:12:16.047 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:16.042+0000 7f160ea00640 10 client.16137 ms_handle_connect on v2:172.21.15.165:6838/501312984 2024-09-11T00:12:16.064 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:12:15 smithi012 ceph-mon[30333]: pgmap v554: 129 pgs: 129 active+clean; 38 MiB data, 758 MiB used, 1.0 TiB / 1.0 TiB avail; 30 MiB/s rd, 866 KiB/s wr, 255 op/s 2024-09-11T00:12:16.064 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:12:15 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:12:16.065 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:12:15 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T00:12:16.065 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:12:15 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:12:16.080 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:12:15 smithi178 ceph-mon[35919]: pgmap v554: 129 pgs: 129 active+clean; 38 MiB data, 758 MiB used, 1.0 TiB / 1.0 TiB avail; 30 MiB/s rd, 866 KiB/s wr, 255 op/s 2024-09-11T00:12:16.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:12:15 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:12:16.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:12:15 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T00:12:16.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:12:15 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:12:16.103 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:16.100+0000 7f160ea00640 1 -- 172.21.15.12:0/1407785297 <== mds.0 v2:172.21.15.165:6838/501312984 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55b10541f0b0 con 0x55b105530c50 2024-09-11T00:12:16.103 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:16.100+0000 7f160ea00640 10 client.16137 handle_command_reply: tid=0 2024-09-11T00:12:16.103 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:16.100+0000 7f1632df5740 1 client.16137 shutdown 2024-09-11T00:12:16.103 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:16.100+0000 7f1632df5740 1 -- 172.21.15.12:0/1407785297 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b1054b12a0 msgr2=0x55b1054be630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:12:16.103 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:16.100+0000 7f1632df5740 1 --2- 172.21.15.12:0/1407785297 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b1054b12a0 0x55b1054be630 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f16140d1b50 tx=0x7f1614106e20 comp rx=0 tx=0).stop 2024-09-11T00:12:16.103 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:16.100+0000 7f1632df5740 1 -- 172.21.15.12:0/1407785297 shutdown_connections 2024-09-11T00:12:16.103 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:16.100+0000 7f1632df5740 1 --2- 172.21.15.12:0/1407785297 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55b105530c50 0x55b105533070 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:12:16.103 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:16.100+0000 7f1632df5740 1 --2- 172.21.15.12:0/1407785297 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b1054b3ce0 0x55b1054beb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:12:16.103 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:16.100+0000 7f1632df5740 1 --2- 172.21.15.12:0/1407785297 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b1054b12a0 0x55b1054be630 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:12:16.104 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:16.100+0000 7f1632df5740 1 --2- 172.21.15.12:0/1407785297 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b1054ae970 0x55b1054bf930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:12:16.104 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:16.100+0000 7f1632df5740 1 -- 172.21.15.12:0/1407785297 >> 172.21.15.12:0/1407785297 conn(0x55b1054a9ac0 msgr2=0x55b1054aa320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:12:16.104 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:16.100+0000 7f160ea00640 0 client.16137 ms_handle_reset on v2:172.21.15.165:6838/501312984 2024-09-11T00:12:16.104 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:16.100+0000 7f1632df5740 1 -- 172.21.15.12:0/1407785297 shutdown_connections 2024-09-11T00:12:16.104 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:16.101+0000 7f1632df5740 1 -- 172.21.15.12:0/1407785297 wait complete. 2024-09-11T00:12:16.104 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:16.101+0000 7f1632df5740 20 client.16137 trim_cache size 0 max 16384 2024-09-11T00:12:16.104 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:{ 2024-09-11T00:12:16.104 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "message": "", 2024-09-11T00:12:16.104 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "return_code": 0 2024-09-11T00:12:16.105 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:} 2024-09-11T00:12:16.105 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:16.101+0000 7f162ea00640 1 -- 172.21.15.12:0/1175663471 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f16100b5360 msgr2=0x7f16100b7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:12:16.105 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:16.101+0000 7f162ea00640 1 --2- 172.21.15.12:0/1175663471 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f16100b5360 0x7f16100b7820 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f161c005fd0 tx=0x7f161c009760 comp rx=0 tx=0).stop 2024-09-11T00:12:16.105 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:16.101+0000 7f162ea00640 1 -- 172.21.15.12:0/1175663471 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1628107d00 msgr2=0x7f162819b410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:12:16.105 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:16.101+0000 7f162ea00640 1 --2- 172.21.15.12:0/1175663471 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1628107d00 0x7f162819b410 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f1614031500 tx=0x7f1614003c10 comp rx=0 tx=0).stop 2024-09-11T00:12:16.106 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:16.101+0000 7f162ea00640 1 -- 172.21.15.12:0/1175663471 shutdown_connections 2024-09-11T00:12:16.106 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:16.101+0000 7f162ea00640 1 --2- 172.21.15.12:0/1175663471 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f16100b5360 0x7f16100b7820 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:12:16.106 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:16.101+0000 7f162ea00640 1 --2- 172.21.15.12:0/1175663471 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f162810a6c0 0x7f162819c040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:12:16.106 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:16.101+0000 7f162ea00640 1 --2- 172.21.15.12:0/1175663471 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1628107d00 0x7f162819b410 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:12:16.106 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:16.101+0000 7f162ea00640 1 --2- 172.21.15.12:0/1175663471 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f162806b300 0x7f162819adc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:12:16.106 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:16.101+0000 7f162ea00640 1 -- 172.21.15.12:0/1175663471 >> 172.21.15.12:0/1175663471 conn(0x7f16280fcf30 msgr2=0x7f1628108960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:12:16.106 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:16.102+0000 7f162ea00640 1 -- 172.21.15.12:0/1175663471 shutdown_connections 2024-09-11T00:12:16.106 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:16.102+0000 7f162ea00640 1 -- 172.21.15.12:0/1175663471 wait complete. 2024-09-11T00:12:16.634 INFO:tasks.workunit.client.0.smithi012.stdout: 1 145769 2.01 MB/sec execute 133 sec latency 342.294 ms 2024-09-11T00:12:17.634 INFO:tasks.workunit.client.0.smithi012.stdout: 1 147850 2.05 MB/sec execute 134 sec latency 11.726 ms 2024-09-11T00:12:18.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:12:17 smithi165 ceph-mon[35356]: pgmap v555: 129 pgs: 129 active+clean; 40 MiB data, 799 MiB used, 1.0 TiB / 1.0 TiB avail; 56 MiB/s rd, 1.1 MiB/s wr, 426 op/s 2024-09-11T00:12:18.063 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:12:17 smithi012 ceph-mon[30333]: pgmap v555: 129 pgs: 129 active+clean; 40 MiB data, 799 MiB used, 1.0 TiB / 1.0 TiB avail; 56 MiB/s rd, 1.1 MiB/s wr, 426 op/s 2024-09-11T00:12:18.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:12:17 smithi178 ceph-mon[35919]: pgmap v555: 129 pgs: 129 active+clean; 40 MiB data, 799 MiB used, 1.0 TiB / 1.0 TiB avail; 56 MiB/s rd, 1.1 MiB/s wr, 426 op/s 2024-09-11T00:12:18.635 INFO:tasks.workunit.client.0.smithi012.stdout: 1 149249 2.12 MB/sec execute 135 sec latency 117.114 ms 2024-09-11T00:12:19.634 INFO:tasks.workunit.client.0.smithi012.stdout: 1 150619 2.14 MB/sec execute 136 sec latency 11.166 ms 2024-09-11T00:12:20.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:12:19 smithi165 ceph-mon[35356]: pgmap v556: 129 pgs: 129 active+clean; 40 MiB data, 821 MiB used, 1.0 TiB / 1.0 TiB avail; 52 MiB/s rd, 598 KiB/s wr, 356 op/s 2024-09-11T00:12:20.063 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:12:19 smithi012 ceph-mon[30333]: pgmap v556: 129 pgs: 129 active+clean; 40 MiB data, 821 MiB used, 1.0 TiB / 1.0 TiB avail; 52 MiB/s rd, 598 KiB/s wr, 356 op/s 2024-09-11T00:12:20.080 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:12:19 smithi178 ceph-mon[35919]: pgmap v556: 129 pgs: 129 active+clean; 40 MiB data, 821 MiB used, 1.0 TiB / 1.0 TiB avail; 52 MiB/s rd, 598 KiB/s wr, 356 op/s 2024-09-11T00:12:20.635 INFO:tasks.workunit.client.0.smithi012.stdout: 1 152635 2.22 MB/sec execute 137 sec latency 131.333 ms 2024-09-11T00:12:21.635 INFO:tasks.workunit.client.0.smithi012.stdout: 1 153053 2.21 MB/sec execute 138 sec latency 281.471 ms 2024-09-11T00:12:22.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:12:21 smithi165 ceph-mon[35356]: pgmap v557: 129 pgs: 129 active+clean; 43 MiB data, 815 MiB used, 1.0 TiB / 1.0 TiB avail; 66 MiB/s rd, 905 KiB/s wr, 489 op/s 2024-09-11T00:12:22.063 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:12:21 smithi012 ceph-mon[30333]: pgmap v557: 129 pgs: 129 active+clean; 43 MiB data, 815 MiB used, 1.0 TiB / 1.0 TiB avail; 66 MiB/s rd, 905 KiB/s wr, 489 op/s 2024-09-11T00:12:22.080 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:12:21 smithi178 ceph-mon[35919]: pgmap v557: 129 pgs: 129 active+clean; 43 MiB data, 815 MiB used, 1.0 TiB / 1.0 TiB avail; 66 MiB/s rd, 905 KiB/s wr, 489 op/s 2024-09-11T00:12:22.635 INFO:tasks.workunit.client.0.smithi012.stdout: 1 155311 2.28 MB/sec execute 139 sec latency 13.244 ms 2024-09-11T00:12:23.635 INFO:tasks.workunit.client.0.smithi012.stdout: 1 156036 2.31 MB/sec execute 140 sec latency 712.529 ms 2024-09-11T00:12:24.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:12:23 smithi165 ceph-mon[35356]: pgmap v558: 129 pgs: 129 active+clean; 40 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 40 MiB/s rd, 813 KiB/s wr, 373 op/s 2024-09-11T00:12:24.063 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:12:23 smithi012 ceph-mon[30333]: pgmap v558: 129 pgs: 129 active+clean; 40 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 40 MiB/s rd, 813 KiB/s wr, 373 op/s 2024-09-11T00:12:24.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:12:23 smithi178 ceph-mon[35919]: pgmap v558: 129 pgs: 129 active+clean; 40 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 40 MiB/s rd, 813 KiB/s wr, 373 op/s 2024-09-11T00:12:24.635 INFO:tasks.workunit.client.0.smithi012.stdout: 1 156036 2.29 MB/sec execute 141 sec latency 1712.649 ms 2024-09-11T00:12:25.635 INFO:tasks.workunit.client.0.smithi012.stdout: 1 156074 2.27 MB/sec execute 142 sec latency 2685.974 ms 2024-09-11T00:12:26.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:12:25 smithi165 ceph-mon[35356]: pgmap v559: 129 pgs: 129 active+clean; 40 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 40 MiB/s rd, 705 KiB/s wr, 343 op/s 2024-09-11T00:12:26.063 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:12:25 smithi012 ceph-mon[30333]: pgmap v559: 129 pgs: 129 active+clean; 40 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 40 MiB/s rd, 705 KiB/s wr, 343 op/s 2024-09-11T00:12:26.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:12:25 smithi178 ceph-mon[35919]: pgmap v559: 129 pgs: 129 active+clean; 40 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 40 MiB/s rd, 705 KiB/s wr, 343 op/s 2024-09-11T00:12:26.635 INFO:tasks.workunit.client.0.smithi012.stdout: 1 156074 2.26 MB/sec execute 143 sec latency 1002.082 ms 2024-09-11T00:12:27.635 INFO:tasks.workunit.client.0.smithi012.stdout: 1 156099 2.24 MB/sec execute 144 sec latency 1087.382 ms 2024-09-11T00:12:28.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:12:27 smithi165 ceph-mon[35356]: pgmap v560: 129 pgs: 129 active+clean; 42 MiB data, 790 MiB used, 1.0 TiB / 1.0 TiB avail; 51 MiB/s rd, 890 KiB/s wr, 428 op/s 2024-09-11T00:12:28.063 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:12:27 smithi012 ceph-mon[30333]: pgmap v560: 129 pgs: 129 active+clean; 42 MiB data, 790 MiB used, 1.0 TiB / 1.0 TiB avail; 51 MiB/s rd, 890 KiB/s wr, 428 op/s 2024-09-11T00:12:28.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:12:27 smithi178 ceph-mon[35919]: pgmap v560: 129 pgs: 129 active+clean; 42 MiB data, 790 MiB used, 1.0 TiB / 1.0 TiB avail; 51 MiB/s rd, 890 KiB/s wr, 428 op/s 2024-09-11T00:12:28.635 INFO:tasks.workunit.client.0.smithi012.stdout: 1 156099 2.23 MB/sec execute 145 sec latency 1892.218 ms 2024-09-11T00:12:29.229 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:29.227+0000 7f9c0f200640 1 -- 172.21.15.12:0/970884219 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9c10102790 msgr2=0x7f9c10102b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:12:29.229 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:29.227+0000 7f9c0f200640 1 --2- 172.21.15.12:0/970884219 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9c10102790 0x7f9c10102b90 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f9c00009930 tx=0x7f9c0002a460 comp rx=0 tx=0).stop 2024-09-11T00:12:29.229 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:29.227+0000 7f9c0f200640 1 -- 172.21.15.12:0/970884219 shutdown_connections 2024-09-11T00:12:29.229 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:29.227+0000 7f9c0f200640 1 --2- 172.21.15.12:0/970884219 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9c101042a0 0x7f9c1010ac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:12:29.229 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:29.227+0000 7f9c0f200640 1 --2- 172.21.15.12:0/970884219 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9c101038e0 0x7f9c10103d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:12:29.230 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:29.227+0000 7f9c0f200640 1 --2- 172.21.15.12:0/970884219 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9c10102790 0x7f9c10102b90 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:12:29.230 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:29.227+0000 7f9c0f200640 1 -- 172.21.15.12:0/970884219 >> 172.21.15.12:0/970884219 conn(0x7f9c100fe030 msgr2=0x7f9c10100470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:12:29.231 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:29.227+0000 7f9c0f200640 1 -- 172.21.15.12:0/970884219 shutdown_connections 2024-09-11T00:12:29.231 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:29.227+0000 7f9c0f200640 1 -- 172.21.15.12:0/970884219 wait complete. 2024-09-11T00:12:29.232 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:29.228+0000 7f9c0f200640 1 Processor -- start 2024-09-11T00:12:29.232 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:29.228+0000 7f9c0f200640 1 -- start start 2024-09-11T00:12:29.232 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:29.228+0000 7f9c0f200640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9c10102790 0x7f9c1019ae90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:12:29.232 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:29.228+0000 7f9c0f200640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9c101038e0 0x7f9c1019b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:12:29.232 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:29.228+0000 7f9c0f200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9c101042a0 0x7f9c1019c110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:12:29.232 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:29.228+0000 7f9c0f200640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f9c1010e4a0 con 0x7f9c101038e0 2024-09-11T00:12:29.232 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:29.228+0000 7f9c0f200640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f9c1010e320 con 0x7f9c101042a0 2024-09-11T00:12:29.232 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:29.228+0000 7f9c0f200640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f9c1010e620 con 0x7f9c10102790 2024-09-11T00:12:29.232 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:29.229+0000 7f9c0d400640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9c101038e0 0x7f9c1019b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:12:29.232 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:29.229+0000 7f9c0d400640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9c101038e0 0x7f9c1019b4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:48906/0 (socket says 172.21.15.12:48906) 2024-09-11T00:12:29.232 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:29.229+0000 7f9c0d400640 1 -- 172.21.15.12:0/2863758771 learned_addr learned my addr 172.21.15.12:0/2863758771 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:12:29.233 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:29.229+0000 7f9c0de00640 1 --2- 172.21.15.12:0/2863758771 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9c10102790 0x7f9c1019ae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:12:29.234 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:29.229+0000 7f9c0e800640 1 --2- 172.21.15.12:0/2863758771 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9c101042a0 0x7f9c1019c110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:12:29.234 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:29.229+0000 7f9c0d400640 1 -- 172.21.15.12:0/2863758771 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9c10102790 msgr2=0x7f9c1019ae90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:12:29.234 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:29.229+0000 7f9c0d400640 1 --2- 172.21.15.12:0/2863758771 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9c10102790 0x7f9c1019ae90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:12:29.234 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:29.229+0000 7f9c0d400640 1 -- 172.21.15.12:0/2863758771 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9c101042a0 msgr2=0x7f9c1019c110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:12:29.234 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:29.229+0000 7f9c0d400640 1 --2- 172.21.15.12:0/2863758771 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9c101042a0 0x7f9c1019c110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:12:29.234 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:29.229+0000 7f9c0d400640 1 -- 172.21.15.12:0/2863758771 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c1019c9d0 con 0x7f9c101038e0 2024-09-11T00:12:29.234 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:29.229+0000 7f9c0d400640 1 --2- 172.21.15.12:0/2863758771 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9c101038e0 0x7f9c1019b4e0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f9bfc00e9f0 tx=0x7f9bfc00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:12:29.234 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:29.230+0000 7f9c06a00640 1 -- 172.21.15.12:0/2863758771 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bfc00cd80 con 0x7f9c101038e0 2024-09-11T00:12:29.234 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:29.230+0000 7f9c06a00640 1 -- 172.21.15.12:0/2863758771 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9bfc0054d0 con 0x7f9c101038e0 2024-09-11T00:12:29.234 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:29.230+0000 7f9c0f200640 1 -- 172.21.15.12:0/2863758771 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9c1019ccc0 con 0x7f9c101038e0 2024-09-11T00:12:29.234 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:29.230+0000 7f9c06a00640 1 -- 172.21.15.12:0/2863758771 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bfc010690 con 0x7f9c101038e0 2024-09-11T00:12:29.235 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:29.230+0000 7f9c0f200640 1 -- 172.21.15.12:0/2863758771 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9c101a3390 con 0x7f9c101038e0 2024-09-11T00:12:29.235 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:29.233+0000 7f9c17e2f740 1 Processor -- start 2024-09-11T00:12:29.235 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:29.233+0000 7f9c17e2f740 1 -- start start 2024-09-11T00:12:29.236 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:29.233+0000 7f9c17e2f740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d2c4a8f970 0x55d2c4a91d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:12:29.236 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:29.233+0000 7f9c17e2f740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d2c4a922a0 0x55d2c4a94690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:12:29.236 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:29.234+0000 7f9c17e2f740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d2c4a94ce0 0x55d2c4a971a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:12:29.236 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:29.234+0000 7f9c17e2f740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55d2c4a026b0 con 0x55d2c4a94ce0 2024-09-11T00:12:29.236 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:29.234+0000 7f9c17e2f740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55d2c49f1ae0 con 0x55d2c4a8f970 2024-09-11T00:12:29.236 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:29.234+0000 7f9c17e2f740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55d2c49f58f0 con 0x55d2c4a922a0 2024-09-11T00:12:29.237 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:29.235+0000 7f9c06a00640 1 -- 172.21.15.12:0/2863758771 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7f9bfc004e90 con 0x7f9c101038e0 2024-09-11T00:12:29.237 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:29.235+0000 7f9c0d400640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d2c4a94ce0 0x55d2c4a971a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:12:29.237 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:29.235+0000 7f9c0d400640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d2c4a94ce0 0x55d2c4a971a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:48918/0 (socket says 172.21.15.12:48918) 2024-09-11T00:12:29.237 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:29.235+0000 7f9c0d400640 1 -- 172.21.15.12:0/552844473 learned_addr learned my addr 172.21.15.12:0/552844473 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:12:29.238 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:29.236+0000 7f9c0de00640 1 --2- 172.21.15.12:0/552844473 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d2c4a922a0 0x55d2c4a94690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:12:29.247 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:29.236+0000 7f9c0d400640 1 -- 172.21.15.12:0/552844473 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d2c4a922a0 msgr2=0x55d2c4a94690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:12:29.247 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:29.236+0000 7f9c0d400640 1 --2- 172.21.15.12:0/552844473 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d2c4a922a0 0x55d2c4a94690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:12:29.247 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:29.236+0000 7f9c0d400640 1 -- 172.21.15.12:0/552844473 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d2c4a8f970 msgr2=0x55d2c4a91d60 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-11T00:12:29.247 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:29.236+0000 7f9c0d400640 1 --2- 172.21.15.12:0/552844473 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d2c4a8f970 0x55d2c4a91d60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:12:29.247 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:29.236+0000 7f9c0d400640 1 -- 172.21.15.12:0/552844473 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d2c49be780 con 0x55d2c4a94ce0 2024-09-11T00:12:29.247 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:29.236+0000 7f9c0d400640 1 --2- 172.21.15.12:0/552844473 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d2c4a94ce0 0x55d2c4a971a0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f9bfc0ee3e0 tx=0x7f9bfc0ee4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2974a3b4ab7d5d7f server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:12:29.247 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:29.237+0000 7f9bfb400640 1 -- 172.21.15.12:0/552844473 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bfc0e8310 con 0x55d2c4a94ce0 2024-09-11T00:12:29.247 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:29.237+0000 7f9bfb400640 1 -- 172.21.15.12:0/552844473 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9bfc0e84b0 con 0x55d2c4a94ce0 2024-09-11T00:12:29.247 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:29.237+0000 7f9c06a00640 1 --2- 172.21.15.12:0/2863758771 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f9bf40b5360 0x7f9bf40b7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:12:29.247 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:29.237+0000 7f9bfb400640 1 -- 172.21.15.12:0/552844473 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bfc0f5aa0 con 0x55d2c4a94ce0 2024-09-11T00:12:29.247 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:29.237+0000 7f9c06a00640 1 -- 172.21.15.12:0/2863758771 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9208+0+0 (secure 0 0 0) 0x7f9bfc0e63e0 con 0x7f9c101038e0 2024-09-11T00:12:29.248 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:29.237+0000 7f9c0e800640 1 --2- 172.21.15.12:0/2863758771 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f9bf40b5360 0x7f9bf40b7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:12:29.248 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:29.238+0000 7f9c17e2f740 1 -- 172.21.15.12:0/552844473 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d2c4a94ce0 msgr2=0x55d2c4a971a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:12:29.248 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:29.238+0000 7f9c17e2f740 1 --2- 172.21.15.12:0/552844473 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d2c4a94ce0 0x55d2c4a971a0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f9bfc0ee3e0 tx=0x7f9bfc0ee4d0 comp rx=0 tx=0).stop 2024-09-11T00:12:29.248 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:29.238+0000 7f9c17e2f740 1 -- 172.21.15.12:0/552844473 shutdown_connections 2024-09-11T00:12:29.248 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:29.238+0000 7f9c17e2f740 1 --2- 172.21.15.12:0/552844473 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d2c4a94ce0 0x55d2c4a971a0 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:12:29.248 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:29.238+0000 7f9c17e2f740 1 --2- 172.21.15.12:0/552844473 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d2c4a922a0 0x55d2c4a94690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:12:29.248 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:29.238+0000 7f9c17e2f740 1 --2- 172.21.15.12:0/552844473 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d2c4a8f970 0x55d2c4a91d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:12:29.248 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:29.238+0000 7f9c17e2f740 1 -- 172.21.15.12:0/552844473 >> 172.21.15.12:0/552844473 conn(0x55d2c4a8aac0 msgr2=0x55d2c4a8b850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:12:29.248 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:29.238+0000 7f9c17e2f740 1 -- 172.21.15.12:0/552844473 shutdown_connections 2024-09-11T00:12:29.248 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:29.238+0000 7f9c17e2f740 1 -- 172.21.15.12:0/552844473 wait complete. 2024-09-11T00:12:29.248 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:29.238+0000 7f9c0e800640 1 --2- 172.21.15.12:0/2863758771 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f9bf40b5360 0x7f9bf40b7820 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f9c1019bc10 tx=0x7f9c0800d040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:12:29.249 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:29.239+0000 7f9c17e2f740 1 -- start start 2024-09-11T00:12:29.249 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:29.239+0000 7f9c17e2f740 1 Processor -- start 2024-09-11T00:12:29.249 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:29.239+0000 7f9c17e2f740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d2c4a8f970 0x55d2c4a990b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:12:29.249 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:29.239+0000 7f9c17e2f740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d2c4a922a0 0x55d2c4a99750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:12:29.249 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:29.239+0000 7f9c17e2f740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d2c4a94ce0 0x55d2c4a9a320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:12:29.249 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:29.239+0000 7f9c17e2f740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55d2c4929ab0 con 0x55d2c4a922a0 2024-09-11T00:12:29.249 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:29.239+0000 7f9c17e2f740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55d2c4998570 con 0x55d2c4a8f970 2024-09-11T00:12:29.249 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:29.239+0000 7f9c17e2f740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55d2c4a97cc0 con 0x55d2c4a94ce0 2024-09-11T00:12:29.249 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:29.239+0000 7f9c17e2f740 10 client.0 _ll_register_callbacks cb 0x55d2c49a7100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:12:29.249 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:29.240+0000 7f9c0d400640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d2c4a922a0 0x55d2c4a99750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:12:29.249 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:29.240+0000 7f9c0d400640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d2c4a922a0 0x55d2c4a99750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:48928/0 (socket says 172.21.15.12:48928) 2024-09-11T00:12:29.249 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:29.240+0000 7f9c0d400640 1 -- 172.21.15.12:0/3888724639 learned_addr learned my addr 172.21.15.12:0/3888724639 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:12:29.249 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:29.240+0000 7f9c0e800640 1 --2- 172.21.15.12:0/3888724639 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d2c4a94ce0 0x55d2c4a9a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:12:29.250 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:29.240+0000 7f9c0d400640 1 -- 172.21.15.12:0/3888724639 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d2c4a94ce0 msgr2=0x55d2c4a9a320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:12:29.250 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:29.240+0000 7f9c0d400640 1 --2- 172.21.15.12:0/3888724639 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d2c4a94ce0 0x55d2c4a9a320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:12:29.250 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:29.240+0000 7f9c0d400640 1 -- 172.21.15.12:0/3888724639 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d2c4a8f970 msgr2=0x55d2c4a990b0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-11T00:12:29.250 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:29.240+0000 7f9c0d400640 1 --2- 172.21.15.12:0/3888724639 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d2c4a8f970 0x55d2c4a990b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:12:29.250 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:29.240+0000 7f9c0d400640 1 -- 172.21.15.12:0/3888724639 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d2c4753790 con 0x55d2c4a922a0 2024-09-11T00:12:29.250 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:29.240+0000 7f9c0d400640 1 --2- 172.21.15.12:0/3888724639 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d2c4a922a0 0x55d2c4a99750 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f9bfc0a8d40 tx=0x7f9bfc009720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6fdd30eb85fa6190 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:12:29.250 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:29.240+0000 7f9befe00640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-09-11T00:12:29.250 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:29.241+0000 7f9befe00640 1 -- 172.21.15.12:0/3888724639 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bfc0a8dc0 con 0x55d2c4a922a0 2024-09-11T00:12:29.250 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:29.241+0000 7f9c17e2f740 1 -- 172.21.15.12:0/3888724639 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9bfc0e84b0 con 0x55d2c4a922a0 2024-09-11T00:12:29.250 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:29.241+0000 7f9befe00640 1 -- 172.21.15.12:0/3888724639 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9208+0+0 (secure 0 0 0) 0x7f9bfc0046c0 con 0x55d2c4a922a0 2024-09-11T00:12:29.250 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:29.241+0000 7f9befe00640 1 -- 172.21.15.12:0/3888724639 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f9bfc0abee0 con 0x55d2c4a922a0 2024-09-11T00:12:29.250 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:29.241+0000 7f9c17e2f740 10 client.16149 fetch_fsmap learned FSMap version 38 2024-09-11T00:12:29.250 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:29.241+0000 7f9c17e2f740 1 -- 172.21.15.12:0/3888724639 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x55d2c47cc620 con 0x55d2c4a922a0 2024-09-11T00:12:29.251 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:29.241+0000 7f9befe00640 1 -- 172.21.15.12:0/3888724639 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9bfc0e84b0 con 0x55d2c4a922a0 2024-09-11T00:12:29.251 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:29.241+0000 7f9befe00640 1 -- 172.21.15.12:0/3888724639 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 38) ==== 8374+0+0 (secure 0 0 0) 0x7f9bfc0a89e0 con 0x55d2c4a922a0 2024-09-11T00:12:29.251 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:29.241+0000 7f9c17e2f740 10 client.16149 fetch_fsmap finished waiting for FSMap version 38 2024-09-11T00:12:29.251 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:29.241+0000 7f9c17e2f740 10 client.16149 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T00:12:29.251 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:29.241+0000 7f9c17e2f740 20 client.16149 populate_metadata read hostname 'smithi012' 2024-09-11T00:12:29.251 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:29.241+0000 7f9c17e2f740 1 --2- 172.21.15.12:0/3888724639 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55d2c4aa4550 0x55d2c4aa6940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:12:29.251 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:29.242+0000 7f9c17e2f740 4 client.16149 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T00:12:29.251 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:29.242+0000 7f9c17e2f740 1 -- 172.21.15.12:0/3888724639 --> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d2c4a04600 con 0x55d2c4aa4550 2024-09-11T00:12:29.256 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:29.254+0000 7f9c0de00640 1 --2- 172.21.15.12:0/3888724639 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55d2c4aa4550 0x55d2c4aa6940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:12:29.327 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:29.325+0000 7f9c0de00640 1 --2- 172.21.15.12:0/3888724639 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55d2c4aa4550 0x55d2c4aa6940 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=caee53029d61e89 server_cookie=8d18dbf981c29ee5 in_seq=0 out_seq=0 2024-09-11T00:12:29.327 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:29.325+0000 7f9befe00640 10 client.16149 ms_handle_connect on v2:172.21.15.165:6838/501312984 2024-09-11T00:12:29.350 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:29.348+0000 7f9befe00640 1 -- 172.21.15.12:0/3888724639 <== mds.0 v2:172.21.15.165:6838/501312984 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55d2c4a04600 con 0x55d2c4aa4550 2024-09-11T00:12:29.350 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:29.348+0000 7f9befe00640 10 client.16149 handle_command_reply: tid=0 2024-09-11T00:12:29.351 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:29.349+0000 7f9c17e2f740 1 client.16149 shutdown 2024-09-11T00:12:29.352 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:29.349+0000 7f9c17e2f740 1 -- 172.21.15.12:0/3888724639 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d2c4a922a0 msgr2=0x55d2c4a99750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:12:29.352 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:29.349+0000 7f9c17e2f740 1 --2- 172.21.15.12:0/3888724639 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d2c4a922a0 0x55d2c4a99750 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f9bfc0a8d40 tx=0x7f9bfc009720 comp rx=0 tx=0).stop 2024-09-11T00:12:29.352 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:29.349+0000 7f9c17e2f740 1 -- 172.21.15.12:0/3888724639 shutdown_connections 2024-09-11T00:12:29.352 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:29.349+0000 7f9c17e2f740 1 --2- 172.21.15.12:0/3888724639 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55d2c4aa4550 0x55d2c4aa6940 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:12:29.352 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:29.349+0000 7f9c17e2f740 1 --2- 172.21.15.12:0/3888724639 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d2c4a94ce0 0x55d2c4a9a320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:12:29.352 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:29.349+0000 7f9c17e2f740 1 --2- 172.21.15.12:0/3888724639 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d2c4a922a0 0x55d2c4a99750 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:12:29.352 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:29.349+0000 7f9c17e2f740 1 --2- 172.21.15.12:0/3888724639 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d2c4a8f970 0x55d2c4a990b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:12:29.352 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:29.349+0000 7f9c17e2f740 1 -- 172.21.15.12:0/3888724639 >> 172.21.15.12:0/3888724639 conn(0x55d2c4a8aac0 msgr2=0x55d2c4a89bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:12:29.352 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:29.349+0000 7f9befe00640 0 client.16149 ms_handle_reset on v2:172.21.15.165:6838/501312984 2024-09-11T00:12:29.352 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:29.349+0000 7f9c17e2f740 1 -- 172.21.15.12:0/3888724639 shutdown_connections 2024-09-11T00:12:29.352 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:29.350+0000 7f9c17e2f740 1 -- 172.21.15.12:0/3888724639 wait complete. 2024-09-11T00:12:29.352 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:29.350+0000 7f9c17e2f740 20 client.16149 trim_cache size 0 max 16384 2024-09-11T00:12:29.377 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:29.374+0000 7f9c17e2f740 1 Processor -- start 2024-09-11T00:12:29.377 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:29.374+0000 7f9c17e2f740 1 -- start start 2024-09-11T00:12:29.377 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:29.374+0000 7f9c17e2f740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d2c4a8f970 0x55d2c4a95130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:12:29.377 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:29.374+0000 7f9c17e2f740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d2c4a922a0 0x55d2c4a95670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:12:29.377 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:29.374+0000 7f9c17e2f740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d2c4a94ce0 0x55d2c4a8b320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:12:29.378 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:29.374+0000 7f9c17e2f740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f9c101ad860 con 0x55d2c4a94ce0 2024-09-11T00:12:29.378 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:29.374+0000 7f9c17e2f740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f9c101af3d0 con 0x55d2c4a8f970 2024-09-11T00:12:29.378 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:29.374+0000 7f9c17e2f740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f9c101adf30 con 0x55d2c4a922a0 2024-09-11T00:12:29.378 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:29.374+0000 7f9c0d400640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d2c4a922a0 0x55d2c4a95670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:12:29.378 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:29.374+0000 7f9c0d400640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d2c4a922a0 0x55d2c4a95670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.12:53532/0 (socket says 172.21.15.12:53532) 2024-09-11T00:12:29.378 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:29.374+0000 7f9c0d400640 1 -- 172.21.15.12:0/3628547546 learned_addr learned my addr 172.21.15.12:0/3628547546 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:12:29.378 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:29.375+0000 7f9c0e800640 1 --2- 172.21.15.12:0/3628547546 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d2c4a94ce0 0x55d2c4a8b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:12:29.378 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:29.375+0000 7f9c0d400640 1 -- 172.21.15.12:0/3628547546 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d2c4a8f970 msgr2=0x55d2c4a95130 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-11T00:12:29.379 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:29.375+0000 7f9c0d400640 1 --2- 172.21.15.12:0/3628547546 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d2c4a8f970 0x55d2c4a95130 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:12:29.379 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:29.375+0000 7f9c0d400640 1 -- 172.21.15.12:0/3628547546 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d2c4a94ce0 msgr2=0x55d2c4a8b320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:12:29.379 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:29.375+0000 7f9c0d400640 1 --2- 172.21.15.12:0/3628547546 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d2c4a94ce0 0x55d2c4a8b320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:12:29.379 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:29.375+0000 7f9c0d400640 1 -- 172.21.15.12:0/3628547546 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d2c4a920d0 con 0x55d2c4a922a0 2024-09-11T00:12:29.379 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:29.375+0000 7f9c0d400640 1 --2- 172.21.15.12:0/3628547546 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d2c4a922a0 0x55d2c4a95670 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f9bfc009380 tx=0x7f9bfc0eefd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8905b175057682ea server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:12:29.379 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:29.375+0000 7f9bfaa00640 1 -- 172.21.15.12:0/3628547546 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bfc0ac0c0 con 0x55d2c4a922a0 2024-09-11T00:12:29.379 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:29.375+0000 7f9bfaa00640 1 -- 172.21.15.12:0/3628547546 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9bfc0e9950 con 0x55d2c4a922a0 2024-09-11T00:12:29.379 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:29.375+0000 7f9bfaa00640 1 -- 172.21.15.12:0/3628547546 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bfc0ac540 con 0x55d2c4a922a0 2024-09-11T00:12:29.382 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:29.376+0000 7f9c17e2f740 1 -- 172.21.15.12:0/3628547546 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d2c4a922a0 msgr2=0x55d2c4a95670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:12:29.382 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:29.376+0000 7f9c17e2f740 1 --2- 172.21.15.12:0/3628547546 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d2c4a922a0 0x55d2c4a95670 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f9bfc009380 tx=0x7f9bfc0eefd0 comp rx=0 tx=0).stop 2024-09-11T00:12:29.382 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:29.376+0000 7f9c17e2f740 1 -- 172.21.15.12:0/3628547546 shutdown_connections 2024-09-11T00:12:29.382 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:29.376+0000 7f9c17e2f740 1 --2- 172.21.15.12:0/3628547546 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d2c4a94ce0 0x55d2c4a8b320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:12:29.382 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:29.376+0000 7f9c17e2f740 1 --2- 172.21.15.12:0/3628547546 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d2c4a922a0 0x55d2c4a95670 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:12:29.382 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:29.376+0000 7f9c17e2f740 1 --2- 172.21.15.12:0/3628547546 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d2c4a8f970 0x55d2c4a95130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:12:29.382 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:29.376+0000 7f9c17e2f740 1 -- 172.21.15.12:0/3628547546 >> 172.21.15.12:0/3628547546 conn(0x55d2c4a8aac0 msgr2=0x55d2c4a98270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:12:29.382 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:29.376+0000 7f9c17e2f740 1 -- 172.21.15.12:0/3628547546 shutdown_connections 2024-09-11T00:12:29.382 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:29.376+0000 7f9c17e2f740 1 -- 172.21.15.12:0/3628547546 wait complete. 2024-09-11T00:12:29.382 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:29.376+0000 7f9c17e2f740 1 -- start start 2024-09-11T00:12:29.382 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:29.377+0000 7f9c17e2f740 1 Processor -- start 2024-09-11T00:12:29.382 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:29.377+0000 7f9c17e2f740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d2c4a8f970 0x55d2c4aa0d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:12:29.383 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:29.377+0000 7f9c17e2f740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d2c4a922a0 0x55d2c4a9faa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:12:29.383 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:29.377+0000 7f9c17e2f740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d2c4a94ce0 0x55d2c4aa0190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:12:29.383 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:29.377+0000 7f9c17e2f740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f9c101aed00 con 0x55d2c4a94ce0 2024-09-11T00:12:29.383 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:29.377+0000 7f9c17e2f740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f9c101ae630 con 0x55d2c4a922a0 2024-09-11T00:12:29.383 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:29.377+0000 7f9c17e2f740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f9c101ab840 con 0x55d2c4a8f970 2024-09-11T00:12:29.383 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:29.377+0000 7f9c17e2f740 10 client.0 _ll_register_callbacks cb 0x7f9c00009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:12:29.383 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:29.377+0000 7f9c0e800640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d2c4a94ce0 0x55d2c4aa0190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:12:29.383 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:29.377+0000 7f9c0de00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d2c4a8f970 0x55d2c4aa0d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:12:29.383 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:29.377+0000 7f9c0d400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d2c4a922a0 0x55d2c4a9faa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:12:29.383 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:29.377+0000 7f9c0de00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d2c4a8f970 0x55d2c4aa0d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.12:53538/0 (socket says 172.21.15.12:53538) 2024-09-11T00:12:29.383 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:29.377+0000 7f9c0de00640 1 -- 172.21.15.12:0/657323045 learned_addr learned my addr 172.21.15.12:0/657323045 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:12:29.383 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:29.377+0000 7f9c0e800640 1 --2- 172.21.15.12:0/657323045 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d2c4a94ce0 0x55d2c4aa0190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:48940/0 (socket says 172.21.15.12:48940) 2024-09-11T00:12:29.384 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:29.378+0000 7f9c0de00640 1 -- 172.21.15.12:0/657323045 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d2c4a922a0 msgr2=0x55d2c4a9faa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:12:29.384 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:29.378+0000 7f9c0de00640 1 --2- 172.21.15.12:0/657323045 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d2c4a922a0 0x55d2c4a9faa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:12:29.384 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:29.378+0000 7f9c0de00640 1 -- 172.21.15.12:0/657323045 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d2c4a94ce0 msgr2=0x55d2c4aa0190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:12:29.384 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:29.378+0000 7f9c0de00640 1 --2- 172.21.15.12:0/657323045 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d2c4a94ce0 0x55d2c4aa0190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:12:29.384 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:29.378+0000 7f9c0de00640 1 -- 172.21.15.12:0/657323045 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d2c4a9e5a0 con 0x55d2c4a8f970 2024-09-11T00:12:29.384 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:29.378+0000 7f9c0de00640 1 --2- 172.21.15.12:0/657323045 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d2c4a8f970 0x55d2c4aa0d80 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f9c00009930 tx=0x7f9c00031df0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5394ce7c4eaec1d2 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:12:29.384 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:29.378+0000 7f9befe00640 10 client.0 ms_handle_connect on v2:172.21.15.178:3300/0 2024-09-11T00:12:29.384 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:29.378+0000 7f9befe00640 1 -- 172.21.15.12:0/657323045 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c00035040 con 0x55d2c4a8f970 2024-09-11T00:12:29.384 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:29.378+0000 7f9befe00640 1 -- 172.21.15.12:0/657323045 <== mon.1 v2:172.21.15.178:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9208+0+0 (secure 0 0 0) 0x7f9c00004050 con 0x55d2c4a8f970 2024-09-11T00:12:29.385 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:29.378+0000 7f9c17e2f740 1 -- 172.21.15.12:0/657323045 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55d2c4a000b0 con 0x55d2c4a8f970 2024-09-11T00:12:29.385 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:29.379+0000 7f9befe00640 1 -- 172.21.15.12:0/657323045 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f9c00053050 con 0x55d2c4a8f970 2024-09-11T00:12:29.385 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:29.379+0000 7f9c17e2f740 10 client.26161 fetch_fsmap learned FSMap version 38 2024-09-11T00:12:29.385 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:29.379+0000 7f9c17e2f740 1 -- 172.21.15.12:0/657323045 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x55d2c4b115f0 con 0x55d2c4a8f970 2024-09-11T00:12:29.385 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:29.379+0000 7f9befe00640 1 -- 172.21.15.12:0/657323045 <== mon.1 v2:172.21.15.178:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55d2c4a000b0 con 0x55d2c4a8f970 2024-09-11T00:12:29.385 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:29.379+0000 7f9befe00640 1 -- 172.21.15.12:0/657323045 <== mon.1 v2:172.21.15.178:3300/0 5 ==== fsmap(e 38) ==== 8374+0+0 (secure 0 0 0) 0x7f9bfc0051b0 con 0x55d2c4a8f970 2024-09-11T00:12:29.386 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:29.379+0000 7f9c17e2f740 10 client.26161 fetch_fsmap finished waiting for FSMap version 38 2024-09-11T00:12:29.386 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:29.379+0000 7f9c17e2f740 10 client.26161 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T00:12:29.386 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:29.379+0000 7f9c17e2f740 20 client.26161 populate_metadata read hostname 'smithi012' 2024-09-11T00:12:29.386 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:29.379+0000 7f9c17e2f740 1 --2- 172.21.15.12:0/657323045 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55d2c4b11fd0 0x55d2c4b143f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:12:29.386 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:29.379+0000 7f9c17e2f740 4 client.26161 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T00:12:29.386 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:29.379+0000 7f9c17e2f740 1 -- 172.21.15.12:0/657323045 --> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55d2c49fd4e0 con 0x55d2c4b11fd0 2024-09-11T00:12:29.386 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:29.380+0000 7f9c0d400640 1 --2- 172.21.15.12:0/657323045 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55d2c4b11fd0 0x55d2c4b143f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:12:29.386 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:29.380+0000 7f9c0d400640 1 --2- 172.21.15.12:0/657323045 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55d2c4b11fd0 0x55d2c4b143f0 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7fb2d42aec4520a9 server_cookie=c67a941f993e4baf in_seq=0 out_seq=0 2024-09-11T00:12:29.386 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:29.380+0000 7f9befe00640 10 client.26161 ms_handle_connect on v2:172.21.15.165:6838/501312984 2024-09-11T00:12:29.433 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:29.431+0000 7f9befe00640 1 -- 172.21.15.12:0/657323045 <== mds.0 v2:172.21.15.165:6838/501312984 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55d2c49fd4e0 con 0x55d2c4b11fd0 2024-09-11T00:12:29.433 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:29.431+0000 7f9befe00640 10 client.26161 handle_command_reply: tid=0 2024-09-11T00:12:29.434 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:29.431+0000 7f9c17e2f740 1 client.26161 shutdown 2024-09-11T00:12:29.434 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:29.431+0000 7f9c17e2f740 1 -- 172.21.15.12:0/657323045 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d2c4a8f970 msgr2=0x55d2c4aa0d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:12:29.434 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:29.431+0000 7f9c17e2f740 1 --2- 172.21.15.12:0/657323045 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d2c4a8f970 0x55d2c4aa0d80 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f9c00009930 tx=0x7f9c00031df0 comp rx=0 tx=0).stop 2024-09-11T00:12:29.434 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:29.431+0000 7f9c17e2f740 1 -- 172.21.15.12:0/657323045 shutdown_connections 2024-09-11T00:12:29.434 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:29.431+0000 7f9c17e2f740 1 --2- 172.21.15.12:0/657323045 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55d2c4b11fd0 0x55d2c4b143f0 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:12:29.434 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:29.431+0000 7f9c17e2f740 1 --2- 172.21.15.12:0/657323045 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d2c4a94ce0 0x55d2c4aa0190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:12:29.434 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:29.431+0000 7f9c17e2f740 1 --2- 172.21.15.12:0/657323045 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d2c4a922a0 0x55d2c4a9faa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:12:29.434 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:29.431+0000 7f9befe00640 0 client.26161 ms_handle_reset on v2:172.21.15.165:6838/501312984 2024-09-11T00:12:29.434 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:29.431+0000 7f9c17e2f740 1 --2- 172.21.15.12:0/657323045 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d2c4a8f970 0x55d2c4aa0d80 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:12:29.434 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:29.431+0000 7f9c17e2f740 1 -- 172.21.15.12:0/657323045 >> 172.21.15.12:0/657323045 conn(0x55d2c4a8aac0 msgr2=0x55d2c4a8ed50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:12:29.434 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:29.431+0000 7f9c17e2f740 1 -- 172.21.15.12:0/657323045 shutdown_connections 2024-09-11T00:12:29.434 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:29.431+0000 7f9c17e2f740 1 -- 172.21.15.12:0/657323045 wait complete. 2024-09-11T00:12:29.434 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:29.431+0000 7f9c17e2f740 20 client.26161 trim_cache size 0 max 16384 2024-09-11T00:12:29.435 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:{ 2024-09-11T00:12:29.435 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "message": "", 2024-09-11T00:12:29.435 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "return_code": 0 2024-09-11T00:12:29.435 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:} 2024-09-11T00:12:29.436 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:29.432+0000 7f9c0f200640 1 -- 172.21.15.12:0/2863758771 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f9bf40b5360 msgr2=0x7f9bf40b7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:12:29.436 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:29.432+0000 7f9c0f200640 1 --2- 172.21.15.12:0/2863758771 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f9bf40b5360 0x7f9bf40b7820 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f9c1019bc10 tx=0x7f9c0800d040 comp rx=0 tx=0).stop 2024-09-11T00:12:29.436 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:29.432+0000 7f9c0f200640 1 -- 172.21.15.12:0/2863758771 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9c101038e0 msgr2=0x7f9c1019b4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:12:29.436 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:29.432+0000 7f9c0f200640 1 --2- 172.21.15.12:0/2863758771 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9c101038e0 0x7f9c1019b4e0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f9bfc00e9f0 tx=0x7f9bfc00edf0 comp rx=0 tx=0).stop 2024-09-11T00:12:29.436 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:29.433+0000 7f9c0f200640 1 -- 172.21.15.12:0/2863758771 shutdown_connections 2024-09-11T00:12:29.436 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:29.433+0000 7f9c0f200640 1 --2- 172.21.15.12:0/2863758771 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f9bf40b5360 0x7f9bf40b7820 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:12:29.436 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:29.433+0000 7f9c0f200640 1 --2- 172.21.15.12:0/2863758771 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9c101042a0 0x7f9c1019c110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:12:29.436 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:29.433+0000 7f9c0f200640 1 --2- 172.21.15.12:0/2863758771 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9c101038e0 0x7f9c1019b4e0 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:12:29.436 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:29.433+0000 7f9c0f200640 1 --2- 172.21.15.12:0/2863758771 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9c10102790 0x7f9c1019ae90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:12:29.436 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:29.433+0000 7f9c0f200640 1 -- 172.21.15.12:0/2863758771 >> 172.21.15.12:0/2863758771 conn(0x7f9c100fe030 msgr2=0x7f9c10109070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:12:29.436 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:29.433+0000 7f9c0f200640 1 -- 172.21.15.12:0/2863758771 shutdown_connections 2024-09-11T00:12:29.437 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:29.433+0000 7f9c0f200640 1 -- 172.21.15.12:0/2863758771 wait complete. 2024-09-11T00:12:29.635 INFO:tasks.workunit.client.0.smithi012.stdout: 1 156099 2.21 MB/sec execute 146 sec latency 2892.297 ms 2024-09-11T00:12:30.063 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:12:29 smithi012 ceph-mon[30333]: pgmap v561: 129 pgs: 129 active+clean; 42 MiB data, 795 MiB used, 1.0 TiB / 1.0 TiB avail; 24 MiB/s rd, 639 KiB/s wr, 259 op/s 2024-09-11T00:12:30.080 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:12:29 smithi178 ceph-mon[35919]: pgmap v561: 129 pgs: 129 active+clean; 42 MiB data, 795 MiB used, 1.0 TiB / 1.0 TiB avail; 24 MiB/s rd, 639 KiB/s wr, 259 op/s 2024-09-11T00:12:30.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:12:29 smithi165 ceph-mon[35356]: pgmap v561: 129 pgs: 129 active+clean; 42 MiB data, 795 MiB used, 1.0 TiB / 1.0 TiB avail; 24 MiB/s rd, 639 KiB/s wr, 259 op/s 2024-09-11T00:12:30.635 INFO:tasks.workunit.client.0.smithi012.stdout: 1 156099 2.20 MB/sec execute 147 sec latency 3892.350 ms 2024-09-11T00:12:31.635 INFO:tasks.workunit.client.0.smithi012.stdout: 1 156099 2.18 MB/sec execute 148 sec latency 4892.399 ms 2024-09-11T00:12:32.053 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:12:31 smithi012 ceph-mon[30333]: pgmap v562: 129 pgs: 129 active+clean; 54 MiB data, 788 MiB used, 1.0 TiB / 1.0 TiB avail; 24 MiB/s rd, 1.7 MiB/s wr, 288 op/s 2024-09-11T00:12:32.053 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:12:31 smithi012 ceph-mon[30333]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T00:12:32.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:12:31 smithi178 ceph-mon[35919]: pgmap v562: 129 pgs: 129 active+clean; 54 MiB data, 788 MiB used, 1.0 TiB / 1.0 TiB avail; 24 MiB/s rd, 1.7 MiB/s wr, 288 op/s 2024-09-11T00:12:32.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:12:31 smithi178 ceph-mon[35919]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T00:12:32.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:12:31 smithi165 ceph-mon[35356]: pgmap v562: 129 pgs: 129 active+clean; 54 MiB data, 788 MiB used, 1.0 TiB / 1.0 TiB avail; 24 MiB/s rd, 1.7 MiB/s wr, 288 op/s 2024-09-11T00:12:32.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:12:31 smithi165 ceph-mon[35356]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T00:12:32.635 INFO:tasks.workunit.client.0.smithi012.stdout: 1 156099 2.17 MB/sec execute 149 sec latency 5892.451 ms 2024-09-11T00:12:33.636 INFO:tasks.workunit.client.0.smithi012.stdout: 1 156164 2.15 MB/sec execute 150 sec latency 6105.658 ms 2024-09-11T00:12:34.063 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:12:33 smithi012 ceph-mon[30333]: pgmap v563: 129 pgs: 129 active+clean; 55 MiB data, 770 MiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 1.8 MiB/s wr, 175 op/s 2024-09-11T00:12:34.080 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:12:33 smithi178 ceph-mon[35919]: pgmap v563: 129 pgs: 129 active+clean; 55 MiB data, 770 MiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 1.8 MiB/s wr, 175 op/s 2024-09-11T00:12:34.284 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:12:33 smithi165 ceph-mon[35356]: pgmap v563: 129 pgs: 129 active+clean; 55 MiB data, 770 MiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 1.8 MiB/s wr, 175 op/s 2024-09-11T00:12:34.636 INFO:tasks.workunit.client.0.smithi012.stdout: 1 156164 2.14 MB/sec execute 151 sec latency 1745.945 ms 2024-09-11T00:12:35.636 INFO:tasks.workunit.client.0.smithi012.stdout: 1 156164 2.13 MB/sec execute 152 sec latency 2745.990 ms 2024-09-11T00:12:36.063 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:12:35 smithi012 ceph-mon[30333]: pgmap v564: 129 pgs: 129 active+clean; 54 MiB data, 768 MiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 1.7 MiB/s wr, 156 op/s 2024-09-11T00:12:36.080 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:12:35 smithi178 ceph-mon[35919]: pgmap v564: 129 pgs: 129 active+clean; 54 MiB data, 768 MiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 1.7 MiB/s wr, 156 op/s 2024-09-11T00:12:36.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:12:35 smithi165 ceph-mon[35356]: pgmap v564: 129 pgs: 129 active+clean; 54 MiB data, 768 MiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 1.7 MiB/s wr, 156 op/s 2024-09-11T00:12:36.636 INFO:tasks.workunit.client.0.smithi012.stdout: 1 156164 2.11 MB/sec execute 153 sec latency 3746.053 ms 2024-09-11T00:12:37.636 INFO:tasks.workunit.client.0.smithi012.stdout: 1 156209 2.10 MB/sec execute 154 sec latency 3831.976 ms 2024-09-11T00:12:38.063 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:12:37 smithi012 ceph-mon[30333]: pgmap v565: 129 pgs: 129 active+clean; 42 MiB data, 755 MiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 2.0 MiB/s wr, 196 op/s 2024-09-11T00:12:38.080 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:12:37 smithi178 ceph-mon[35919]: pgmap v565: 129 pgs: 129 active+clean; 42 MiB data, 755 MiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 2.0 MiB/s wr, 196 op/s 2024-09-11T00:12:38.084 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:12:37 smithi165 ceph-mon[35356]: pgmap v565: 129 pgs: 129 active+clean; 42 MiB data, 755 MiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 2.0 MiB/s wr, 196 op/s 2024-09-11T00:12:38.637 INFO:tasks.workunit.client.0.smithi012.stdout: 1 156209 2.09 MB/sec execute 155 sec latency 1850.912 ms 2024-09-11T00:12:39.636 INFO:tasks.workunit.client.0.smithi012.stdout: 1 156209 2.07 MB/sec execute 156 sec latency 2850.981 ms 2024-09-11T00:12:40.063 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:12:39 smithi012 ceph-mon[30333]: pgmap v566: 129 pgs: 129 active+clean; 42 MiB data, 752 MiB used, 1.0 TiB / 1.0 TiB avail; 169 KiB/s rd, 1.9 MiB/s wr, 114 op/s 2024-09-11T00:12:40.080 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:12:39 smithi178 ceph-mon[35919]: pgmap v566: 129 pgs: 129 active+clean; 42 MiB data, 752 MiB used, 1.0 TiB / 1.0 TiB avail; 169 KiB/s rd, 1.9 MiB/s wr, 114 op/s 2024-09-11T00:12:40.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:12:39 smithi165 ceph-mon[35356]: pgmap v566: 129 pgs: 129 active+clean; 42 MiB data, 752 MiB used, 1.0 TiB / 1.0 TiB avail; 169 KiB/s rd, 1.9 MiB/s wr, 114 op/s 2024-09-11T00:12:40.636 INFO:tasks.workunit.client.0.smithi012.stdout: 1 156209 2.06 MB/sec execute 157 sec latency 3851.060 ms 2024-09-11T00:12:41.636 INFO:tasks.workunit.client.0.smithi012.stdout: 1 156209 2.05 MB/sec execute 158 sec latency 4851.178 ms 2024-09-11T00:12:42.063 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:12:41 smithi012 ceph-mon[30333]: pgmap v567: 129 pgs: 129 active+clean; 41 MiB data, 759 MiB used, 1.0 TiB / 1.0 TiB avail; 244 KiB/s rd, 2.0 MiB/s wr, 123 op/s 2024-09-11T00:12:42.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:12:41 smithi178 ceph-mon[35919]: pgmap v567: 129 pgs: 129 active+clean; 41 MiB data, 759 MiB used, 1.0 TiB / 1.0 TiB avail; 244 KiB/s rd, 2.0 MiB/s wr, 123 op/s 2024-09-11T00:12:42.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:12:41 smithi165 ceph-mon[35356]: pgmap v567: 129 pgs: 129 active+clean; 41 MiB data, 759 MiB used, 1.0 TiB / 1.0 TiB avail; 244 KiB/s rd, 2.0 MiB/s wr, 123 op/s 2024-09-11T00:12:42.566 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:42.563+0000 7f9bc8800640 1 -- 172.21.15.12:0/2410797468 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9bc0105ce0 msgr2=0x7f9bc0106160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:12:42.566 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:42.563+0000 7f9bc8800640 1 --2- 172.21.15.12:0/2410797468 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9bc0105ce0 0x7f9bc0106160 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f9bb0009930 tx=0x7f9bb002a420 comp rx=0 tx=0).stop 2024-09-11T00:12:42.566 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:42.563+0000 7f9bc8800640 1 -- 172.21.15.12:0/2410797468 shutdown_connections 2024-09-11T00:12:42.566 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:42.563+0000 7f9bc8800640 1 --2- 172.21.15.12:0/2410797468 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9bc01066a0 0x7f9bc010d020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:12:42.566 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:42.563+0000 7f9bc8800640 1 --2- 172.21.15.12:0/2410797468 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9bc0105ce0 0x7f9bc0106160 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:12:42.566 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:42.563+0000 7f9bc8800640 1 --2- 172.21.15.12:0/2410797468 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9bc0104b90 0x7f9bc0104f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:12:42.566 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:42.563+0000 7f9bc8800640 1 -- 172.21.15.12:0/2410797468 >> 172.21.15.12:0/2410797468 conn(0x7f9bc0100470 msgr2=0x7f9bc0102890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:12:42.566 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:42.563+0000 7f9bc8800640 1 -- 172.21.15.12:0/2410797468 shutdown_connections 2024-09-11T00:12:42.566 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:42.563+0000 7f9bc8800640 1 -- 172.21.15.12:0/2410797468 wait complete. 2024-09-11T00:12:42.568 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:42.564+0000 7f9bc8800640 1 Processor -- start 2024-09-11T00:12:42.568 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:42.564+0000 7f9bc8800640 1 -- start start 2024-09-11T00:12:42.568 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:42.565+0000 7f9bc8800640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9bc0104b90 0x7f9bc019af70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:12:42.569 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:42.565+0000 7f9bc8800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9bc0105ce0 0x7f9bc019b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:12:42.569 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:42.565+0000 7f9bc8800640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9bc01066a0 0x7f9bc019c1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:12:42.569 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:42.565+0000 7f9bc8800640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f9bc0110850 con 0x7f9bc01066a0 2024-09-11T00:12:42.569 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:42.565+0000 7f9bc8800640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f9bc01106d0 con 0x7f9bc0105ce0 2024-09-11T00:12:42.569 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:42.565+0000 7f9bc8800640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f9bc01109d0 con 0x7f9bc0104b90 2024-09-11T00:12:42.569 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:42.565+0000 7f9bc6800640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9bc01066a0 0x7f9bc019c1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:12:42.569 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:42.565+0000 7f9bc6800640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9bc01066a0 0x7f9bc019c1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:50660/0 (socket says 172.21.15.12:50660) 2024-09-11T00:12:42.569 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:42.565+0000 7f9bc6800640 1 -- 172.21.15.12:0/522808684 learned_addr learned my addr 172.21.15.12:0/522808684 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:12:42.569 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:42.565+0000 7f9bc5e00640 1 --2- 172.21.15.12:0/522808684 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9bc0104b90 0x7f9bc019af70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:12:42.569 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:42.565+0000 7f9bc5400640 1 --2- 172.21.15.12:0/522808684 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9bc0105ce0 0x7f9bc019b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:12:42.569 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:42.565+0000 7f9bc6800640 1 -- 172.21.15.12:0/522808684 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9bc0104b90 msgr2=0x7f9bc019af70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:12:42.569 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:42.565+0000 7f9bc6800640 1 --2- 172.21.15.12:0/522808684 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9bc0104b90 0x7f9bc019af70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:12:42.570 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:42.566+0000 7f9bc6800640 1 -- 172.21.15.12:0/522808684 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9bc0105ce0 msgr2=0x7f9bc019b5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:12:42.570 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:42.566+0000 7f9bc6800640 1 --2- 172.21.15.12:0/522808684 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9bc0105ce0 0x7f9bc019b5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:12:42.570 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:42.566+0000 7f9bc6800640 1 -- 172.21.15.12:0/522808684 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9bc019cab0 con 0x7f9bc01066a0 2024-09-11T00:12:42.570 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:42.566+0000 7f9bc5e00640 1 --2- 172.21.15.12:0/522808684 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9bc0104b90 0x7f9bc019af70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T00:12:42.570 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:42.566+0000 7f9bc6800640 1 --2- 172.21.15.12:0/522808684 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9bc01066a0 0x7f9bc019c1f0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f9bbc00eaa0 tx=0x7f9bbc00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:12:42.570 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:42.566+0000 7f9bbaa00640 1 -- 172.21.15.12:0/522808684 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bbc00cdb0 con 0x7f9bc01066a0 2024-09-11T00:12:42.570 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:42.566+0000 7f9bbaa00640 1 -- 172.21.15.12:0/522808684 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9bbc005020 con 0x7f9bc01066a0 2024-09-11T00:12:42.570 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:42.566+0000 7f9bc8800640 1 -- 172.21.15.12:0/522808684 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9bc019cda0 con 0x7f9bc01066a0 2024-09-11T00:12:42.570 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:42.566+0000 7f9bbaa00640 1 -- 172.21.15.12:0/522808684 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bbc0116d0 con 0x7f9bc01066a0 2024-09-11T00:12:42.570 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:42.566+0000 7f9bc8800640 1 -- 172.21.15.12:0/522808684 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9bc01a3470 con 0x7f9bc01066a0 2024-09-11T00:12:42.571 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:42.569+0000 7f9bcb6d4740 1 Processor -- start 2024-09-11T00:12:42.572 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:42.570+0000 7f9bbaa00640 1 -- 172.21.15.12:0/522808684 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7f9bbc00f040 con 0x7f9bc01066a0 2024-09-11T00:12:42.572 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:42.570+0000 7f9bcb6d4740 1 -- start start 2024-09-11T00:12:42.573 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:42.570+0000 7f9bcb6d4740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556b5e280790 0x556b5e282b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:12:42.573 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:42.570+0000 7f9bcb6d4740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556b5e2830c0 0x556b5e2854b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:12:42.573 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:42.570+0000 7f9bcb6d4740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556b5e285b00 0x556b5e287fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:12:42.573 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:42.570+0000 7f9bcb6d4740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x556b5e1f36b0 con 0x556b5e285b00 2024-09-11T00:12:42.573 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:42.570+0000 7f9bcb6d4740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x556b5e1e2ae0 con 0x556b5e280790 2024-09-11T00:12:42.573 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:42.570+0000 7f9bcb6d4740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x556b5e1e68f0 con 0x556b5e2830c0 2024-09-11T00:12:42.573 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:42.570+0000 7f9bc5400640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556b5e285b00 0x556b5e287fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:12:42.573 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:42.571+0000 7f9bc5400640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556b5e285b00 0x556b5e287fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:50666/0 (socket says 172.21.15.12:50666) 2024-09-11T00:12:42.573 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:42.571+0000 7f9bc5400640 1 -- 172.21.15.12:0/689196745 learned_addr learned my addr 172.21.15.12:0/689196745 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:12:42.574 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:42.571+0000 7f9bc5e00640 1 --2- 172.21.15.12:0/689196745 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556b5e2830c0 0x556b5e2854b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:12:42.574 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:42.571+0000 7f9bc5400640 1 --2- 172.21.15.12:0/689196745 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556b5e280790 0x556b5e282b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:12:42.574 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:42.571+0000 7f9bc5e00640 1 -- 172.21.15.12:0/689196745 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556b5e280790 msgr2=0x556b5e282b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:12:42.574 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:42.571+0000 7f9bc5e00640 1 --2- 172.21.15.12:0/689196745 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556b5e280790 0x556b5e282b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:12:42.574 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:42.571+0000 7f9bc5e00640 1 -- 172.21.15.12:0/689196745 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556b5e285b00 msgr2=0x556b5e287fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:12:42.574 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:42.571+0000 7f9bc5e00640 1 --2- 172.21.15.12:0/689196745 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556b5e285b00 0x556b5e287fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:12:42.574 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:42.571+0000 7f9bc5e00640 1 -- 172.21.15.12:0/689196745 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556b5e1af780 con 0x556b5e2830c0 2024-09-11T00:12:42.577 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:42.571+0000 7f9bc5e00640 1 --2- 172.21.15.12:0/689196745 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556b5e2830c0 0x556b5e2854b0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f9bb4009c30 tx=0x7f9bb40117b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=70af441242826aa server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:12:42.577 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:42.572+0000 7f9baf000640 1 -- 172.21.15.12:0/689196745 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bb4012450 con 0x556b5e2830c0 2024-09-11T00:12:42.577 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:42.572+0000 7f9baf000640 1 -- 172.21.15.12:0/689196745 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9bb4012aa0 con 0x556b5e2830c0 2024-09-11T00:12:42.577 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:42.572+0000 7f9bbaa00640 1 --2- 172.21.15.12:0/522808684 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f9ba80b52e0 0x7f9ba80b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:12:42.577 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:42.572+0000 7f9baf000640 1 -- 172.21.15.12:0/689196745 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bb400fa80 con 0x556b5e2830c0 2024-09-11T00:12:42.577 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:42.572+0000 7f9bbaa00640 1 -- 172.21.15.12:0/522808684 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9208+0+0 (secure 0 0 0) 0x7f9bbc0e5e70 con 0x7f9bc01066a0 2024-09-11T00:12:42.577 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:42.572+0000 7f9bc5400640 1 --2- 172.21.15.12:0/522808684 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f9ba80b52e0 0x7f9ba80b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:12:42.577 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:42.572+0000 7f9bcb6d4740 1 -- 172.21.15.12:0/689196745 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556b5e2830c0 msgr2=0x556b5e2854b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:12:42.577 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:42.572+0000 7f9bcb6d4740 1 --2- 172.21.15.12:0/689196745 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556b5e2830c0 0x556b5e2854b0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f9bb4009c30 tx=0x7f9bb40117b0 comp rx=0 tx=0).stop 2024-09-11T00:12:42.578 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:42.572+0000 7f9bcb6d4740 1 -- 172.21.15.12:0/689196745 shutdown_connections 2024-09-11T00:12:42.578 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:42.572+0000 7f9bcb6d4740 1 --2- 172.21.15.12:0/689196745 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556b5e285b00 0x556b5e287fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:12:42.578 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:42.572+0000 7f9bcb6d4740 1 --2- 172.21.15.12:0/689196745 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556b5e2830c0 0x556b5e2854b0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:12:42.578 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:42.572+0000 7f9bcb6d4740 1 --2- 172.21.15.12:0/689196745 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556b5e280790 0x556b5e282b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:12:42.578 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:42.572+0000 7f9bcb6d4740 1 -- 172.21.15.12:0/689196745 >> 172.21.15.12:0/689196745 conn(0x556b5e27b8e0 msgr2=0x556b5e27c670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:12:42.578 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:42.573+0000 7f9bcb6d4740 1 -- 172.21.15.12:0/689196745 shutdown_connections 2024-09-11T00:12:42.578 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:42.573+0000 7f9bc5400640 1 --2- 172.21.15.12:0/522808684 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f9ba80b52e0 0x7f9ba80b77a0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f9bb0031500 tx=0x7f9bb0005d50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:12:42.578 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:42.573+0000 7f9bcb6d4740 1 -- 172.21.15.12:0/689196745 wait complete. 2024-09-11T00:12:42.578 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:42.573+0000 7f9bcb6d4740 1 -- start start 2024-09-11T00:12:42.578 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:42.573+0000 7f9bcb6d4740 1 Processor -- start 2024-09-11T00:12:42.578 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:42.574+0000 7f9bcb6d4740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556b5e280790 0x556b5e289ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:12:42.578 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:42.574+0000 7f9bcb6d4740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556b5e2830c0 0x556b5e28a540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:12:42.578 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:42.574+0000 7f9bcb6d4740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556b5e285b00 0x556b5e28b110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:12:42.579 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:42.574+0000 7f9bcb6d4740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x556b5e11aab0 con 0x556b5e2830c0 2024-09-11T00:12:42.579 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:42.574+0000 7f9bcb6d4740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x556b5e189570 con 0x556b5e285b00 2024-09-11T00:12:42.579 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:42.574+0000 7f9bcb6d4740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x556b5e288ae0 con 0x556b5e280790 2024-09-11T00:12:42.579 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:42.574+0000 7f9bcb6d4740 10 client.0 _ll_register_callbacks cb 0x556b5e198100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:12:42.579 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:42.574+0000 7f9bc5400640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556b5e2830c0 0x556b5e28a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:12:42.579 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:42.574+0000 7f9bc6800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556b5e285b00 0x556b5e28b110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:12:42.579 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:42.574+0000 7f9bc5e00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556b5e280790 0x556b5e289ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:12:42.579 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:42.574+0000 7f9bc5400640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556b5e2830c0 0x556b5e28a540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:50678/0 (socket says 172.21.15.12:50678) 2024-09-11T00:12:42.579 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:42.574+0000 7f9bc5400640 1 -- 172.21.15.12:0/2207315172 learned_addr learned my addr 172.21.15.12:0/2207315172 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:12:42.579 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:42.575+0000 7f9bc6800640 1 -- 172.21.15.12:0/2207315172 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556b5e280790 msgr2=0x556b5e289ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:12:42.579 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:42.575+0000 7f9bc6800640 1 --2- 172.21.15.12:0/2207315172 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556b5e280790 0x556b5e289ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:12:42.579 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:42.575+0000 7f9bc6800640 1 -- 172.21.15.12:0/2207315172 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556b5e2830c0 msgr2=0x556b5e28a540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:12:42.579 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:42.575+0000 7f9bc6800640 1 --2- 172.21.15.12:0/2207315172 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556b5e2830c0 0x556b5e28a540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:12:42.580 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:42.575+0000 7f9bc6800640 1 -- 172.21.15.12:0/2207315172 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556b5df44790 con 0x556b5e285b00 2024-09-11T00:12:42.580 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:42.575+0000 7f9bc6800640 1 --2- 172.21.15.12:0/2207315172 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556b5e285b00 0x556b5e28b110 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f9bbc0a9470 tx=0x7f9bbc0abd10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a0c0dc52a880996e server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:12:42.580 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:42.575+0000 7f9ba3400640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-09-11T00:12:42.582 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:42.575+0000 7f9ba3400640 1 -- 172.21.15.12:0/2207315172 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bbc01f030 con 0x556b5e285b00 2024-09-11T00:12:42.582 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:42.575+0000 7f9ba3400640 1 -- 172.21.15.12:0/2207315172 <== mon.2 v2:172.21.15.165:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9208+0+0 (secure 0 0 0) 0x7f9bbc097510 con 0x556b5e285b00 2024-09-11T00:12:42.582 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:42.575+0000 7f9bcb6d4740 1 -- 172.21.15.12:0/2207315172 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9bb4012aa0 con 0x556b5e285b00 2024-09-11T00:12:42.582 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:42.576+0000 7f9ba3400640 1 -- 172.21.15.12:0/2207315172 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f9bbc0b0ce0 con 0x556b5e285b00 2024-09-11T00:12:42.582 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:42.576+0000 7f9bcb6d4740 10 client.26174 fetch_fsmap learned FSMap version 39 2024-09-11T00:12:42.582 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:42.576+0000 7f9bcb6d4740 1 -- 172.21.15.12:0/2207315172 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x556b5dfbd620 con 0x556b5e285b00 2024-09-11T00:12:42.583 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:42.576+0000 7f9ba3400640 1 -- 172.21.15.12:0/2207315172 <== mon.2 v2:172.21.15.165:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9bb4012aa0 con 0x556b5e285b00 2024-09-11T00:12:42.583 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:42.576+0000 7f9ba3400640 1 -- 172.21.15.12:0/2207315172 <== mon.2 v2:172.21.15.165:3300/0 5 ==== fsmap(e 39) ==== 8378+0+0 (secure 0 0 0) 0x7f9bbc0cf070 con 0x556b5e285b00 2024-09-11T00:12:42.583 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:42.576+0000 7f9bcb6d4740 10 client.26174 fetch_fsmap finished waiting for FSMap version 39 2024-09-11T00:12:42.583 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:42.576+0000 7f9bcb6d4740 10 client.26174 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T00:12:42.583 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:42.576+0000 7f9bcb6d4740 20 client.26174 populate_metadata read hostname 'smithi012' 2024-09-11T00:12:42.583 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:42.576+0000 7f9bcb6d4740 1 --2- 172.21.15.12:0/2207315172 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x556b5e2957b0 0x556b5e297c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:12:42.583 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:42.576+0000 7f9bcb6d4740 4 client.26174 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T00:12:42.583 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:42.576+0000 7f9bcb6d4740 1 -- 172.21.15.12:0/2207315172 --> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556b5e1f5600 con 0x556b5e2957b0 2024-09-11T00:12:42.583 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:42.576+0000 7f9bc5e00640 1 --2- 172.21.15.12:0/2207315172 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x556b5e2957b0 0x556b5e297c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:12:42.583 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:42.577+0000 7f9bc5e00640 1 --2- 172.21.15.12:0/2207315172 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x556b5e2957b0 0x556b5e297c00 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=79a26023c023fb6b server_cookie=4580574fc3c8395e in_seq=0 out_seq=0 2024-09-11T00:12:42.583 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:42.577+0000 7f9ba3400640 10 client.26174 ms_handle_connect on v2:172.21.15.165:6838/501312984 2024-09-11T00:12:42.583 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:42.578+0000 7f9ba3400640 1 -- 172.21.15.12:0/2207315172 <== mds.0 v2:172.21.15.165:6838/501312984 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x556b5e1f5600 con 0x556b5e2957b0 2024-09-11T00:12:42.583 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:42.578+0000 7f9ba3400640 10 client.26174 handle_command_reply: tid=0 2024-09-11T00:12:42.584 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:42.578+0000 7f9bcb6d4740 1 client.26174 shutdown 2024-09-11T00:12:42.584 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:42.578+0000 7f9bcb6d4740 1 -- 172.21.15.12:0/2207315172 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556b5e285b00 msgr2=0x556b5e28b110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:12:42.584 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:42.578+0000 7f9bcb6d4740 1 --2- 172.21.15.12:0/2207315172 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556b5e285b00 0x556b5e28b110 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f9bbc0a9470 tx=0x7f9bbc0abd10 comp rx=0 tx=0).stop 2024-09-11T00:12:42.584 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:42.578+0000 7f9bcb6d4740 1 -- 172.21.15.12:0/2207315172 shutdown_connections 2024-09-11T00:12:42.584 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:42.578+0000 7f9bcb6d4740 1 --2- 172.21.15.12:0/2207315172 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x556b5e2957b0 0x556b5e297c00 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:12:42.584 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:42.578+0000 7f9bcb6d4740 1 --2- 172.21.15.12:0/2207315172 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556b5e285b00 0x556b5e28b110 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:12:42.584 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:42.578+0000 7f9bcb6d4740 1 --2- 172.21.15.12:0/2207315172 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556b5e2830c0 0x556b5e28a540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:12:42.584 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:42.578+0000 7f9ba3400640 0 client.26174 ms_handle_reset on v2:172.21.15.165:6838/501312984 2024-09-11T00:12:42.584 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:42.578+0000 7f9bcb6d4740 1 --2- 172.21.15.12:0/2207315172 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556b5e280790 0x556b5e289ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:12:42.584 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:42.578+0000 7f9bcb6d4740 1 -- 172.21.15.12:0/2207315172 >> 172.21.15.12:0/2207315172 conn(0x556b5e27b8e0 msgr2=0x556b5e27aa60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:12:42.584 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:42.578+0000 7f9bcb6d4740 1 -- 172.21.15.12:0/2207315172 shutdown_connections 2024-09-11T00:12:42.584 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:42.578+0000 7f9bcb6d4740 1 -- 172.21.15.12:0/2207315172 wait complete. 2024-09-11T00:12:42.584 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:42.578+0000 7f9bcb6d4740 20 client.26174 trim_cache size 0 max 16384 2024-09-11T00:12:42.596 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:42.594+0000 7f9bcb6d4740 1 Processor -- start 2024-09-11T00:12:42.596 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:42.594+0000 7f9bcb6d4740 1 -- start start 2024-09-11T00:12:42.596 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:42.594+0000 7f9bcb6d4740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556b5e280790 0x556b5e27a240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:12:42.596 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:42.594+0000 7f9bcb6d4740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556b5e2830c0 0x556b5e292190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:12:42.597 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:42.594+0000 7f9bcb6d4740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556b5e285b00 0x556b5e27a780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:12:42.597 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:42.594+0000 7f9bcb6d4740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f9bc01ad9d0 con 0x556b5e280790 2024-09-11T00:12:42.597 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:42.594+0000 7f9bcb6d4740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f9bc01af570 con 0x556b5e285b00 2024-09-11T00:12:42.597 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:42.594+0000 7f9bcb6d4740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f9bc01ae0a0 con 0x556b5e2830c0 2024-09-11T00:12:42.598 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:42.595+0000 7f9bc5e00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556b5e280790 0x556b5e27a240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:12:42.599 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:42.595+0000 7f9bc5e00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556b5e280790 0x556b5e27a240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:50688/0 (socket says 172.21.15.12:50688) 2024-09-11T00:12:42.599 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:42.595+0000 7f9bc5e00640 1 -- 172.21.15.12:0/188557045 learned_addr learned my addr 172.21.15.12:0/188557045 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:12:42.599 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:42.595+0000 7f9bc5400640 1 --2- 172.21.15.12:0/188557045 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556b5e2830c0 0x556b5e292190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:12:42.599 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:42.595+0000 7f9bc6800640 1 --2- 172.21.15.12:0/188557045 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556b5e285b00 0x556b5e27a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:12:42.599 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:42.595+0000 7f9bc5e00640 1 -- 172.21.15.12:0/188557045 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556b5e2830c0 msgr2=0x556b5e292190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:12:42.599 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:42.595+0000 7f9bc5e00640 1 --2- 172.21.15.12:0/188557045 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556b5e2830c0 0x556b5e292190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:12:42.599 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:42.595+0000 7f9bc5e00640 1 -- 172.21.15.12:0/188557045 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556b5e285b00 msgr2=0x556b5e27a780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:12:42.599 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:42.595+0000 7f9bc5e00640 1 --2- 172.21.15.12:0/188557045 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556b5e285b00 0x556b5e27a780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:12:42.599 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:42.595+0000 7f9bc5e00640 1 -- 172.21.15.12:0/188557045 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556b5e285820 con 0x556b5e280790 2024-09-11T00:12:42.599 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:42.595+0000 7f9bc5e00640 1 --2- 172.21.15.12:0/188557045 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556b5e280790 0x556b5e27a240 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f9bb4014010 tx=0x7f9bb40058d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf325714271fcccd server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:12:42.599 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:42.596+0000 7f9bae600640 1 -- 172.21.15.12:0/188557045 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bb4009d20 con 0x556b5e280790 2024-09-11T00:12:42.599 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:42.596+0000 7f9bae600640 1 -- 172.21.15.12:0/188557045 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9bb40054f0 con 0x556b5e280790 2024-09-11T00:12:42.600 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:42.596+0000 7f9bae600640 1 -- 172.21.15.12:0/188557045 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bb4003850 con 0x556b5e280790 2024-09-11T00:12:42.600 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:42.596+0000 7f9bcb6d4740 1 -- 172.21.15.12:0/188557045 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556b5e280790 msgr2=0x556b5e27a240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:12:42.600 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:42.596+0000 7f9bcb6d4740 1 --2- 172.21.15.12:0/188557045 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556b5e280790 0x556b5e27a240 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f9bb4014010 tx=0x7f9bb40058d0 comp rx=0 tx=0).stop 2024-09-11T00:12:42.600 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:42.596+0000 7f9bcb6d4740 1 -- 172.21.15.12:0/188557045 shutdown_connections 2024-09-11T00:12:42.600 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:42.596+0000 7f9bcb6d4740 1 --2- 172.21.15.12:0/188557045 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556b5e285b00 0x556b5e27a780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:12:42.600 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:42.596+0000 7f9bcb6d4740 1 --2- 172.21.15.12:0/188557045 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556b5e2830c0 0x556b5e292190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:12:42.600 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:42.596+0000 7f9bcb6d4740 1 --2- 172.21.15.12:0/188557045 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556b5e280790 0x556b5e27a240 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:12:42.600 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:42.596+0000 7f9bcb6d4740 1 -- 172.21.15.12:0/188557045 >> 172.21.15.12:0/188557045 conn(0x556b5e27b8e0 msgr2=0x556b5e27c140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:12:42.600 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:42.596+0000 7f9bcb6d4740 1 -- 172.21.15.12:0/188557045 shutdown_connections 2024-09-11T00:12:42.600 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:42.596+0000 7f9bcb6d4740 1 -- 172.21.15.12:0/188557045 wait complete. 2024-09-11T00:12:42.600 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:42.597+0000 7f9bcb6d4740 1 -- start start 2024-09-11T00:12:42.603 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:42.597+0000 7f9bcb6d4740 1 Processor -- start 2024-09-11T00:12:42.603 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:42.597+0000 7f9bcb6d4740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556b5e280790 0x556b5e2912b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:12:42.603 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:42.597+0000 7f9bcb6d4740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556b5e2830c0 0x556b5e291830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:12:42.603 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:42.597+0000 7f9bcb6d4740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556b5e285b00 0x556b5e28fdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:12:42.603 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:42.597+0000 7f9bcb6d4740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f9bc01aeea0 con 0x556b5e285b00 2024-09-11T00:12:42.603 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:42.597+0000 7f9bcb6d4740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f9bc01ae7d0 con 0x556b5e280790 2024-09-11T00:12:42.603 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:42.597+0000 7f9bcb6d4740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f9bc01ab9b0 con 0x556b5e2830c0 2024-09-11T00:12:42.604 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:42.597+0000 7f9bcb6d4740 10 client.0 _ll_register_callbacks cb 0x7f9bbc0a8850 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:12:42.604 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:42.598+0000 7f9bc5e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556b5e280790 0x556b5e2912b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:12:42.604 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:42.598+0000 7f9bc5e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556b5e280790 0x556b5e2912b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.12:52280/0 (socket says 172.21.15.12:52280) 2024-09-11T00:12:42.604 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:42.598+0000 7f9bc6800640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556b5e285b00 0x556b5e28fdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:12:42.604 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:42.598+0000 7f9bc5e00640 1 -- 172.21.15.12:0/2589299307 learned_addr learned my addr 172.21.15.12:0/2589299307 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:12:42.604 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:42.598+0000 7f9bc5400640 1 --2- 172.21.15.12:0/2589299307 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556b5e2830c0 0x556b5e291830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:12:42.604 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:42.598+0000 7f9bc5e00640 1 -- 172.21.15.12:0/2589299307 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556b5e2830c0 msgr2=0x556b5e291830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:12:42.604 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:42.598+0000 7f9bc5e00640 1 --2- 172.21.15.12:0/2589299307 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556b5e2830c0 0x556b5e291830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:12:42.604 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:42.598+0000 7f9bc5e00640 1 -- 172.21.15.12:0/2589299307 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556b5e285b00 msgr2=0x556b5e28fdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:12:42.604 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:42.598+0000 7f9bc5e00640 1 --2- 172.21.15.12:0/2589299307 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556b5e285b00 0x556b5e28fdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:12:42.604 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:42.598+0000 7f9bc5e00640 1 -- 172.21.15.12:0/2589299307 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556b5e290310 con 0x556b5e280790 2024-09-11T00:12:42.604 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:42.598+0000 7f9bc6800640 1 --2- 172.21.15.12:0/2589299307 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556b5e285b00 0x556b5e28fdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:12:42.605 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:42.598+0000 7f9bc5e00640 1 --2- 172.21.15.12:0/2589299307 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556b5e280790 0x556b5e2912b0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f9bb4012c00 tx=0x7f9bb4009310 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f173710fde3da86e server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:12:42.605 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:42.598+0000 7f9ba3400640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-09-11T00:12:42.605 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:42.599+0000 7f9ba3400640 1 -- 172.21.15.12:0/2589299307 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bb4012450 con 0x556b5e280790 2024-09-11T00:12:42.605 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:42.599+0000 7f9bcb6d4740 1 -- 172.21.15.12:0/2589299307 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9bb40054f0 con 0x556b5e280790 2024-09-11T00:12:42.605 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:42.599+0000 7f9ba3400640 1 -- 172.21.15.12:0/2589299307 <== mon.2 v2:172.21.15.165:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9208+0+0 (secure 0 0 0) 0x7f9bb40125f0 con 0x556b5e280790 2024-09-11T00:12:42.605 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:42.599+0000 7f9ba3400640 1 -- 172.21.15.12:0/2589299307 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f9bb4043050 con 0x556b5e280790 2024-09-11T00:12:42.605 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:42.599+0000 7f9bcb6d4740 10 client.26180 fetch_fsmap learned FSMap version 39 2024-09-11T00:12:42.605 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:42.599+0000 7f9bcb6d4740 1 -- 172.21.15.12:0/2589299307 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x556b5e302ec0 con 0x556b5e280790 2024-09-11T00:12:42.605 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:42.599+0000 7f9ba3400640 1 -- 172.21.15.12:0/2589299307 <== mon.2 v2:172.21.15.165:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9bb40054f0 con 0x556b5e280790 2024-09-11T00:12:42.605 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:42.599+0000 7f9ba3400640 1 -- 172.21.15.12:0/2589299307 <== mon.2 v2:172.21.15.165:3300/0 5 ==== fsmap(e 39) ==== 8378+0+0 (secure 0 0 0) 0x7f9bb4025680 con 0x556b5e280790 2024-09-11T00:12:42.605 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:42.600+0000 7f9bcb6d4740 10 client.26180 fetch_fsmap finished waiting for FSMap version 39 2024-09-11T00:12:42.605 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:42.600+0000 7f9bcb6d4740 10 client.26180 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T00:12:42.605 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:42.600+0000 7f9bcb6d4740 20 client.26180 populate_metadata read hostname 'smithi012' 2024-09-11T00:12:42.606 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:42.600+0000 7f9bcb6d4740 1 --2- 172.21.15.12:0/2589299307 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x556b5e3036c0 0x556b5e305ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:12:42.606 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:42.600+0000 7f9bcb6d4740 4 client.26180 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T00:12:42.606 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:42.600+0000 7f9bcb6d4740 1 -- 172.21.15.12:0/2589299307 --> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] -- command(tid 0: {"prefix": "flush journal"}) -- 0x556b5e1f10b0 con 0x556b5e3036c0 2024-09-11T00:12:42.606 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:42.600+0000 7f9bc5400640 1 --2- 172.21.15.12:0/2589299307 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x556b5e3036c0 0x556b5e305ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:12:42.606 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:42.601+0000 7f9bc5400640 1 --2- 172.21.15.12:0/2589299307 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x556b5e3036c0 0x556b5e305ae0 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=627f66fb2a87e2d5 server_cookie=943b42e5ac5f2a6e in_seq=0 out_seq=0 2024-09-11T00:12:42.606 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:42.601+0000 7f9ba3400640 10 client.26180 ms_handle_connect on v2:172.21.15.165:6838/501312984 2024-09-11T00:12:42.624 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:42.621+0000 7f9ba3400640 1 -- 172.21.15.12:0/2589299307 <== mds.0 v2:172.21.15.165:6838/501312984 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x556b5e1f10b0 con 0x556b5e3036c0 2024-09-11T00:12:42.624 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:42.621+0000 7f9ba3400640 10 client.26180 handle_command_reply: tid=0 2024-09-11T00:12:42.624 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:42.621+0000 7f9bcb6d4740 1 client.26180 shutdown 2024-09-11T00:12:42.624 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:42.621+0000 7f9bcb6d4740 1 -- 172.21.15.12:0/2589299307 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556b5e280790 msgr2=0x556b5e2912b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:12:42.624 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:42.621+0000 7f9bcb6d4740 1 --2- 172.21.15.12:0/2589299307 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556b5e280790 0x556b5e2912b0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f9bb4012c00 tx=0x7f9bb4009310 comp rx=0 tx=0).stop 2024-09-11T00:12:42.624 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:42.621+0000 7f9bcb6d4740 1 -- 172.21.15.12:0/2589299307 shutdown_connections 2024-09-11T00:12:42.625 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:42.621+0000 7f9bcb6d4740 1 --2- 172.21.15.12:0/2589299307 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x556b5e3036c0 0x556b5e305ae0 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:12:42.625 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:42.621+0000 7f9bcb6d4740 1 --2- 172.21.15.12:0/2589299307 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556b5e285b00 0x556b5e28fdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:12:42.625 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:42.621+0000 7f9bcb6d4740 1 --2- 172.21.15.12:0/2589299307 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556b5e2830c0 0x556b5e291830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:12:42.625 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:42.621+0000 7f9bcb6d4740 1 --2- 172.21.15.12:0/2589299307 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556b5e280790 0x556b5e2912b0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:12:42.625 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:42.621+0000 7f9ba3400640 0 client.26180 ms_handle_reset on v2:172.21.15.165:6838/501312984 2024-09-11T00:12:42.625 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:42.621+0000 7f9bcb6d4740 1 -- 172.21.15.12:0/2589299307 >> 172.21.15.12:0/2589299307 conn(0x556b5e27b8e0 msgr2=0x556b5e27a240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:12:42.625 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:42.621+0000 7f9bcb6d4740 1 -- 172.21.15.12:0/2589299307 shutdown_connections 2024-09-11T00:12:42.625 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:42.621+0000 7f9bcb6d4740 1 -- 172.21.15.12:0/2589299307 wait complete. 2024-09-11T00:12:42.625 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:42.621+0000 7f9bcb6d4740 20 client.26180 trim_cache size 0 max 16384 2024-09-11T00:12:42.626 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:{ 2024-09-11T00:12:42.626 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "message": "", 2024-09-11T00:12:42.626 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "return_code": 0 2024-09-11T00:12:42.626 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:} 2024-09-11T00:12:42.627 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:42.622+0000 7f9bc8800640 1 -- 172.21.15.12:0/522808684 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f9ba80b52e0 msgr2=0x7f9ba80b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:12:42.627 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:42.622+0000 7f9bc8800640 1 --2- 172.21.15.12:0/522808684 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f9ba80b52e0 0x7f9ba80b77a0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f9bb0031500 tx=0x7f9bb0005d50 comp rx=0 tx=0).stop 2024-09-11T00:12:42.627 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:42.622+0000 7f9bc8800640 1 -- 172.21.15.12:0/522808684 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9bc01066a0 msgr2=0x7f9bc019c1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:12:42.627 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:42.622+0000 7f9bc8800640 1 --2- 172.21.15.12:0/522808684 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9bc01066a0 0x7f9bc019c1f0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f9bbc00eaa0 tx=0x7f9bbc00ef60 comp rx=0 tx=0).stop 2024-09-11T00:12:42.627 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:42.622+0000 7f9bc8800640 1 -- 172.21.15.12:0/522808684 shutdown_connections 2024-09-11T00:12:42.628 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:42.622+0000 7f9bc8800640 1 --2- 172.21.15.12:0/522808684 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f9ba80b52e0 0x7f9ba80b77a0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:12:42.628 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:42.622+0000 7f9bc8800640 1 --2- 172.21.15.12:0/522808684 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9bc01066a0 0x7f9bc019c1f0 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:12:42.628 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:42.622+0000 7f9bc8800640 1 --2- 172.21.15.12:0/522808684 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9bc0105ce0 0x7f9bc019b5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:12:42.628 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:42.622+0000 7f9bc8800640 1 --2- 172.21.15.12:0/522808684 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9bc0104b90 0x7f9bc019af70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:12:42.628 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:42.622+0000 7f9bc8800640 1 -- 172.21.15.12:0/522808684 >> 172.21.15.12:0/522808684 conn(0x7f9bc0100470 msgr2=0x7f9bc0109070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:12:42.628 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:42.622+0000 7f9bc8800640 1 -- 172.21.15.12:0/522808684 shutdown_connections 2024-09-11T00:12:42.628 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:42.622+0000 7f9bc8800640 1 -- 172.21.15.12:0/522808684 wait complete. 2024-09-11T00:12:42.636 INFO:tasks.workunit.client.0.smithi012.stdout: 1 156250 2.03 MB/sec execute 159 sec latency 4934.792 ms 2024-09-11T00:12:43.637 INFO:tasks.workunit.client.0.smithi012.stdout: 1 156250 2.02 MB/sec execute 160 sec latency 1876.266 ms 2024-09-11T00:12:44.063 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:12:43 smithi012 ceph-mon[30333]: pgmap v568: 129 pgs: 129 active+clean; 41 MiB data, 757 MiB used, 1.0 TiB / 1.0 TiB avail; 236 KiB/s rd, 938 KiB/s wr, 92 op/s 2024-09-11T00:12:44.080 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:12:43 smithi178 ceph-mon[35919]: pgmap v568: 129 pgs: 129 active+clean; 41 MiB data, 757 MiB used, 1.0 TiB / 1.0 TiB avail; 236 KiB/s rd, 938 KiB/s wr, 92 op/s 2024-09-11T00:12:44.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:12:43 smithi165 ceph-mon[35356]: pgmap v568: 129 pgs: 129 active+clean; 41 MiB data, 757 MiB used, 1.0 TiB / 1.0 TiB avail; 236 KiB/s rd, 938 KiB/s wr, 92 op/s 2024-09-11T00:12:44.637 INFO:tasks.workunit.client.0.smithi012.stdout: 1 156250 2.01 MB/sec execute 161 sec latency 2876.443 ms 2024-09-11T00:12:45.637 INFO:tasks.workunit.client.0.smithi012.stdout: 1 156250 2.00 MB/sec execute 162 sec latency 3876.571 ms 2024-09-11T00:12:46.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:12:45 smithi178 ceph-mon[35919]: pgmap v569: 129 pgs: 129 active+clean; 41 MiB data, 757 MiB used, 1.0 TiB / 1.0 TiB avail; 217 KiB/s rd, 512 KiB/s wr, 69 op/s 2024-09-11T00:12:46.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:12:45 smithi165 ceph-mon[35356]: pgmap v569: 129 pgs: 129 active+clean; 41 MiB data, 757 MiB used, 1.0 TiB / 1.0 TiB avail; 217 KiB/s rd, 512 KiB/s wr, 69 op/s 2024-09-11T00:12:46.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:12:45 smithi012 ceph-mon[30333]: pgmap v569: 129 pgs: 129 active+clean; 41 MiB data, 757 MiB used, 1.0 TiB / 1.0 TiB avail; 217 KiB/s rd, 512 KiB/s wr, 69 op/s 2024-09-11T00:12:46.637 INFO:tasks.workunit.client.0.smithi012.stdout: 1 156250 1.98 MB/sec execute 163 sec latency 4876.706 ms 2024-09-11T00:12:47.637 INFO:tasks.workunit.client.0.smithi012.stdout: 1 157263 1.99 MB/sec execute 164 sec latency 4960.864 ms 2024-09-11T00:12:48.080 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:12:47 smithi178 ceph-mon[35919]: pgmap v570: 129 pgs: 129 active+clean; 41 MiB data, 761 MiB used, 1.0 TiB / 1.0 TiB avail; 225 KiB/s rd, 520 KiB/s wr, 73 op/s 2024-09-11T00:12:48.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:12:47 smithi165 ceph-mon[35356]: pgmap v570: 129 pgs: 129 active+clean; 41 MiB data, 761 MiB used, 1.0 TiB / 1.0 TiB avail; 225 KiB/s rd, 520 KiB/s wr, 73 op/s 2024-09-11T00:12:48.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:12:47 smithi012 ceph-mon[30333]: pgmap v570: 129 pgs: 129 active+clean; 41 MiB data, 761 MiB used, 1.0 TiB / 1.0 TiB avail; 225 KiB/s rd, 520 KiB/s wr, 73 op/s 2024-09-11T00:12:48.637 INFO:tasks.workunit.client.0.smithi012.stdout: 1 159594 2.07 MB/sec execute 165 sec latency 112.577 ms 2024-09-11T00:12:49.637 INFO:tasks.workunit.client.0.smithi012.stdout: 1 160119 2.07 MB/sec execute 166 sec latency 145.110 ms 2024-09-11T00:12:50.080 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:12:49 smithi178 ceph-mon[35919]: pgmap v571: 129 pgs: 129 active+clean; 41 MiB data, 761 MiB used, 1.0 TiB / 1.0 TiB avail; 132 KiB/s rd, 229 KiB/s wr, 34 op/s 2024-09-11T00:12:50.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:12:49 smithi165 ceph-mon[35356]: pgmap v571: 129 pgs: 129 active+clean; 41 MiB data, 761 MiB used, 1.0 TiB / 1.0 TiB avail; 132 KiB/s rd, 229 KiB/s wr, 34 op/s 2024-09-11T00:12:50.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:12:49 smithi012 ceph-mon[30333]: pgmap v571: 129 pgs: 129 active+clean; 41 MiB data, 761 MiB used, 1.0 TiB / 1.0 TiB avail; 132 KiB/s rd, 229 KiB/s wr, 34 op/s 2024-09-11T00:12:50.637 INFO:tasks.workunit.client.0.smithi012.stdout: 1 162403 2.12 MB/sec execute 167 sec latency 11.775 ms 2024-09-11T00:12:51.637 INFO:tasks.workunit.client.0.smithi012.stdout: 1 163281 2.15 MB/sec execute 168 sec latency 475.173 ms 2024-09-11T00:12:52.080 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:12:51 smithi178 ceph-mon[35919]: pgmap v572: 129 pgs: 129 active+clean; 42 MiB data, 803 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 290 KiB/s wr, 79 op/s 2024-09-11T00:12:52.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:12:51 smithi165 ceph-mon[35356]: pgmap v572: 129 pgs: 129 active+clean; 42 MiB data, 803 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 290 KiB/s wr, 79 op/s 2024-09-11T00:12:52.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:12:51 smithi012 ceph-mon[30333]: pgmap v572: 129 pgs: 129 active+clean; 42 MiB data, 803 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 290 KiB/s wr, 79 op/s 2024-09-11T00:12:52.637 INFO:tasks.workunit.client.0.smithi012.stdout: 1 164127 2.15 MB/sec execute 169 sec latency 613.808 ms 2024-09-11T00:12:53.637 INFO:tasks.workunit.client.0.smithi012.stdout: 1 166544 2.22 MB/sec execute 170 sec latency 61.605 ms 2024-09-11T00:12:54.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:12:53 smithi178 ceph-mon[35919]: pgmap v573: 129 pgs: 129 active+clean; 43 MiB data, 837 MiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 274 KiB/s wr, 109 op/s 2024-09-11T00:12:54.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:12:53 smithi165 ceph-mon[35356]: pgmap v573: 129 pgs: 129 active+clean; 43 MiB data, 837 MiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 274 KiB/s wr, 109 op/s 2024-09-11T00:12:54.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:12:53 smithi012 ceph-mon[30333]: pgmap v573: 129 pgs: 129 active+clean; 43 MiB data, 837 MiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 274 KiB/s wr, 109 op/s 2024-09-11T00:12:54.637 INFO:tasks.workunit.client.0.smithi012.stdout: 1 167414 2.24 MB/sec execute 171 sec latency 94.809 ms 2024-09-11T00:12:55.637 INFO:tasks.workunit.client.0.smithi012.stdout: 1 169713 2.29 MB/sec execute 172 sec latency 9.601 ms 2024-09-11T00:12:55.751 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:55.748+0000 7f40f3e00640 1 -- 172.21.15.12:0/955167778 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f40ec104ba0 msgr2=0x7f40ec104fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:12:55.751 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:55.748+0000 7f40f3e00640 1 --2- 172.21.15.12:0/955167778 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f40ec104ba0 0x7f40ec104fa0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f40e0009900 tx=0x7f40e002a410 comp rx=0 tx=0).stop 2024-09-11T00:12:55.751 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:55.748+0000 7f40f3e00640 1 -- 172.21.15.12:0/955167778 shutdown_connections 2024-09-11T00:12:55.751 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:55.748+0000 7f40f3e00640 1 --2- 172.21.15.12:0/955167778 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f40ec1066b0 0x7f40ec10d030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:12:55.751 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:55.748+0000 7f40f3e00640 1 --2- 172.21.15.12:0/955167778 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f40ec105cf0 0x7f40ec106170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:12:55.751 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:55.748+0000 7f40f3e00640 1 --2- 172.21.15.12:0/955167778 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f40ec104ba0 0x7f40ec104fa0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:12:55.751 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:55.748+0000 7f40f3e00640 1 -- 172.21.15.12:0/955167778 >> 172.21.15.12:0/955167778 conn(0x7f40ec100460 msgr2=0x7f40ec102880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:12:55.751 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:55.749+0000 7f40f3e00640 1 -- 172.21.15.12:0/955167778 shutdown_connections 2024-09-11T00:12:55.752 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:55.749+0000 7f40f3e00640 1 -- 172.21.15.12:0/955167778 wait complete. 2024-09-11T00:12:55.753 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:55.749+0000 7f40f3e00640 1 Processor -- start 2024-09-11T00:12:55.753 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:55.750+0000 7f40f3e00640 1 -- start start 2024-09-11T00:12:55.753 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:55.750+0000 7f40f3e00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f40ec104ba0 0x7f40ec19af60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:12:55.753 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:55.750+0000 7f40f3e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f40ec105cf0 0x7f40ec19b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:12:55.753 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:55.750+0000 7f40f3e00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f40ec1066b0 0x7f40ec19c1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:12:55.753 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:55.750+0000 7f40f3e00640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f40ec110810 con 0x7f40ec104ba0 2024-09-11T00:12:55.753 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:55.750+0000 7f40f3e00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f40ec110690 con 0x7f40ec105cf0 2024-09-11T00:12:55.753 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:55.750+0000 7f40f3e00640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f40ec110990 con 0x7f40ec1066b0 2024-09-11T00:12:55.754 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:55.750+0000 7f40f1e00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f40ec1066b0 0x7f40ec19c1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:12:55.754 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:55.750+0000 7f40f1e00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f40ec1066b0 0x7f40ec19c1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.12:50312/0 (socket says 172.21.15.12:50312) 2024-09-11T00:12:55.754 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:55.750+0000 7f40f1e00640 1 -- 172.21.15.12:0/2340651411 learned_addr learned my addr 172.21.15.12:0/2340651411 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:12:55.754 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:55.750+0000 7f40f1400640 1 --2- 172.21.15.12:0/2340651411 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f40ec104ba0 0x7f40ec19af60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:12:55.754 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:55.751+0000 7f40f0a00640 1 --2- 172.21.15.12:0/2340651411 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f40ec105cf0 0x7f40ec19b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:12:55.754 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:55.751+0000 7f40f0a00640 1 -- 172.21.15.12:0/2340651411 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f40ec1066b0 msgr2=0x7f40ec19c1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:12:55.754 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:55.751+0000 7f40f0a00640 1 --2- 172.21.15.12:0/2340651411 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f40ec1066b0 0x7f40ec19c1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:12:55.754 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:55.751+0000 7f40f0a00640 1 -- 172.21.15.12:0/2340651411 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f40ec104ba0 msgr2=0x7f40ec19af60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:12:55.754 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:55.751+0000 7f40f0a00640 1 --2- 172.21.15.12:0/2340651411 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f40ec104ba0 0x7f40ec19af60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:12:55.754 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:55.751+0000 7f40f0a00640 1 -- 172.21.15.12:0/2340651411 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40ec19caa0 con 0x7f40ec105cf0 2024-09-11T00:12:55.755 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:55.751+0000 7f40f0a00640 1 --2- 172.21.15.12:0/2340651411 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f40ec105cf0 0x7f40ec19b5b0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f40d8005fd0 tx=0x7f40d800dde0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:12:55.755 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:55.751+0000 7f40ea000640 1 -- 172.21.15.12:0/2340651411 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40d8009c30 con 0x7f40ec105cf0 2024-09-11T00:12:55.755 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:55.752+0000 7f40ea000640 1 -- 172.21.15.12:0/2340651411 <== mon.2 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f40d8009dd0 con 0x7f40ec105cf0 2024-09-11T00:12:55.755 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:55.752+0000 7f40f3e00640 1 -- 172.21.15.12:0/2340651411 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f40ec19cd90 con 0x7f40ec105cf0 2024-09-11T00:12:55.755 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:55.752+0000 7f40ea000640 1 -- 172.21.15.12:0/2340651411 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40d8003e20 con 0x7f40ec105cf0 2024-09-11T00:12:55.756 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:55.752+0000 7f40f3e00640 1 -- 172.21.15.12:0/2340651411 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f40ec1a3460 con 0x7f40ec105cf0 2024-09-11T00:12:55.757 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:55.755+0000 7f40f6c1c740 1 Processor -- start 2024-09-11T00:12:55.758 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:55.755+0000 7f40f6c1c740 1 -- start start 2024-09-11T00:12:55.758 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:55.756+0000 7f40f6c1c740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ca937d6fa0 0x55ca937d9390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:12:55.758 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:55.756+0000 7f40f6c1c740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ca937d98d0 0x55ca937dbcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:12:55.758 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:55.756+0000 7f40f6c1c740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ca937dc310 0x55ca937de7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:12:55.758 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:55.756+0000 7f40f6c1c740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55ca937416b0 con 0x55ca937d6fa0 2024-09-11T00:12:55.759 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:55.756+0000 7f40f6c1c740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55ca93730ae0 con 0x55ca937d98d0 2024-09-11T00:12:55.759 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:55.756+0000 7f40f6c1c740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55ca937348f0 con 0x55ca937dc310 2024-09-11T00:12:55.759 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:55.756+0000 7f40f1400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ca937d98d0 0x55ca937dbcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:12:55.759 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:55.756+0000 7f40f1e00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ca937d6fa0 0x55ca937d9390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:12:55.759 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:55.756+0000 7f40f1400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ca937d98d0 0x55ca937dbcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.12:42694/0 (socket says 172.21.15.12:42694) 2024-09-11T00:12:55.759 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:55.756+0000 7f40f1400640 1 -- 172.21.15.12:0/2229902658 learned_addr learned my addr 172.21.15.12:0/2229902658 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:12:55.759 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:55.756+0000 7f40ea000640 1 -- 172.21.15.12:0/2340651411 <== mon.2 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7f40d8003fc0 con 0x7f40ec105cf0 2024-09-11T00:12:55.759 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:55.756+0000 7f40f1400640 1 -- 172.21.15.12:0/2229902658 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ca937dc310 msgr2=0x55ca937de7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:12:55.761 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:55.756+0000 7f40f1400640 1 --2- 172.21.15.12:0/2229902658 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ca937dc310 0x55ca937de7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:12:55.761 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:55.756+0000 7f40f1400640 1 -- 172.21.15.12:0/2229902658 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ca937d6fa0 msgr2=0x55ca937d9390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:12:55.761 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:55.756+0000 7f40f1400640 1 --2- 172.21.15.12:0/2229902658 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ca937d6fa0 0x55ca937d9390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:12:55.761 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:55.756+0000 7f40f1400640 1 -- 172.21.15.12:0/2229902658 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ca936fd780 con 0x55ca937d98d0 2024-09-11T00:12:55.761 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:55.757+0000 7f40f1e00640 1 --2- 172.21.15.12:0/2229902658 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ca937d6fa0 0x55ca937d9390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:12:55.761 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:55.757+0000 7f40f1400640 1 --2- 172.21.15.12:0/2229902658 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ca937d98d0 0x55ca937dbcc0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f40e002a930 tx=0x7f40e0005c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b5b5811b678b2c07 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:12:55.761 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:55.757+0000 7f40de600640 1 -- 172.21.15.12:0/2229902658 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40e0035030 con 0x55ca937d98d0 2024-09-11T00:12:55.761 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:55.757+0000 7f40de600640 1 -- 172.21.15.12:0/2229902658 <== mon.2 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f40e0040070 con 0x55ca937d98d0 2024-09-11T00:12:55.761 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:55.757+0000 7f40de600640 1 -- 172.21.15.12:0/2229902658 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40e00038e0 con 0x55ca937d98d0 2024-09-11T00:12:55.762 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:55.758+0000 7f40f6c1c740 1 -- 172.21.15.12:0/2229902658 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ca937d98d0 msgr2=0x55ca937dbcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:12:55.762 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:55.758+0000 7f40f6c1c740 1 --2- 172.21.15.12:0/2229902658 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ca937d98d0 0x55ca937dbcc0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f40e002a930 tx=0x7f40e0005c00 comp rx=0 tx=0).stop 2024-09-11T00:12:55.762 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:55.758+0000 7f40f6c1c740 1 -- 172.21.15.12:0/2229902658 shutdown_connections 2024-09-11T00:12:55.762 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:55.758+0000 7f40f6c1c740 1 --2- 172.21.15.12:0/2229902658 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ca937dc310 0x55ca937de7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:12:55.762 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:55.758+0000 7f40f6c1c740 1 --2- 172.21.15.12:0/2229902658 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ca937d98d0 0x55ca937dbcc0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:12:55.762 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:55.758+0000 7f40f6c1c740 1 --2- 172.21.15.12:0/2229902658 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ca937d6fa0 0x55ca937d9390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:12:55.762 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:55.758+0000 7f40f6c1c740 1 -- 172.21.15.12:0/2229902658 >> 172.21.15.12:0/2229902658 conn(0x55ca937c98e0 msgr2=0x55ca937ca670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:12:55.762 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:55.758+0000 7f40f6c1c740 1 -- 172.21.15.12:0/2229902658 shutdown_connections 2024-09-11T00:12:55.762 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:55.758+0000 7f40f6c1c740 1 -- 172.21.15.12:0/2229902658 wait complete. 2024-09-11T00:12:55.762 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:55.758+0000 7f40ea000640 1 --2- 172.21.15.12:0/2340651411 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f40d40b5200 0x7f40d40b76c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:12:55.762 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:55.759+0000 7f40ea000640 1 -- 172.21.15.12:0/2340651411 <== mon.2 v2:172.21.15.165:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9208+0+0 (secure 0 0 0) 0x7f40d80e5b40 con 0x7f40ec105cf0 2024-09-11T00:12:55.762 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:55.759+0000 7f40f1400640 1 --2- 172.21.15.12:0/2340651411 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f40d40b5200 0x7f40d40b76c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:12:55.762 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:55.759+0000 7f40f6c1c740 1 -- start start 2024-09-11T00:12:55.763 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:55.759+0000 7f40f6c1c740 1 Processor -- start 2024-09-11T00:12:55.766 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:55.759+0000 7f40f6c1c740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ca937d6fa0 0x55ca937e0690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:12:55.766 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:55.759+0000 7f40f6c1c740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ca937d98d0 0x55ca937e0d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:12:55.766 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:55.759+0000 7f40f6c1c740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ca937dc310 0x55ca937e1900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:12:55.766 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:55.759+0000 7f40f6c1c740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55ca93668ab0 con 0x55ca937dc310 2024-09-11T00:12:55.766 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:55.759+0000 7f40f6c1c740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55ca936d7570 con 0x55ca937d98d0 2024-09-11T00:12:55.767 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:55.759+0000 7f40f6c1c740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55ca937df290 con 0x55ca937d6fa0 2024-09-11T00:12:55.767 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:55.760+0000 7f40f1400640 1 --2- 172.21.15.12:0/2340651411 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f40d40b5200 0x7f40d40b76c0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f40e00096c0 tx=0x7f40e0046040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:12:55.767 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:55.760+0000 7f40f6c1c740 10 client.0 _ll_register_callbacks cb 0x55ca936e6100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:12:55.767 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:55.760+0000 7f40f1400640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ca937d6fa0 0x55ca937e0690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:12:55.767 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:55.760+0000 7f40f0a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ca937d98d0 0x55ca937e0d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:12:55.767 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:55.760+0000 7f40f1400640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ca937d6fa0 0x55ca937e0690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.12:50326/0 (socket says 172.21.15.12:50326) 2024-09-11T00:12:55.767 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:55.760+0000 7f40f1400640 1 -- 172.21.15.12:0/59208613 learned_addr learned my addr 172.21.15.12:0/59208613 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:12:55.767 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:55.760+0000 7f40f1e00640 1 --2- 172.21.15.12:0/59208613 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ca937dc310 0x55ca937e1900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:12:55.767 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:55.760+0000 7f40f1400640 1 -- 172.21.15.12:0/59208613 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ca937d98d0 msgr2=0x55ca937e0d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:12:55.767 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:55.760+0000 7f40f1400640 1 --2- 172.21.15.12:0/59208613 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ca937d98d0 0x55ca937e0d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:12:55.767 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:55.760+0000 7f40f1400640 1 -- 172.21.15.12:0/59208613 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ca937dc310 msgr2=0x55ca937e1900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:12:55.767 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:55.760+0000 7f40f1400640 1 --2- 172.21.15.12:0/59208613 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ca937dc310 0x55ca937e1900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:12:55.767 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:55.760+0000 7f40f1400640 1 -- 172.21.15.12:0/59208613 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ca93492790 con 0x55ca937d6fa0 2024-09-11T00:12:55.768 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:55.760+0000 7f40f1e00640 1 --2- 172.21.15.12:0/59208613 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ca937dc310 0x55ca937e1900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:12:55.768 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:55.761+0000 7f40f1400640 1 --2- 172.21.15.12:0/59208613 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ca937d6fa0 0x55ca937e0690 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f40e0043000 tx=0x7f40e00046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3d28d06990733ead server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:12:55.768 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:55.761+0000 7f40d2a00640 10 client.0 ms_handle_connect on v2:172.21.15.178:3300/0 2024-09-11T00:12:55.768 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:55.761+0000 7f40d2a00640 1 -- 172.21.15.12:0/59208613 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40e003c7d0 con 0x55ca937d6fa0 2024-09-11T00:12:55.768 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:55.761+0000 7f40f6c1c740 1 -- 172.21.15.12:0/59208613 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f40e0040070 con 0x55ca937d6fa0 2024-09-11T00:12:55.768 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:55.761+0000 7f40d2a00640 1 -- 172.21.15.12:0/59208613 <== mon.1 v2:172.21.15.178:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9208+0+0 (secure 0 0 0) 0x7f40e003c970 con 0x55ca937d6fa0 2024-09-11T00:12:55.768 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:55.761+0000 7f40d2a00640 1 -- 172.21.15.12:0/59208613 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f40e0062050 con 0x55ca937d6fa0 2024-09-11T00:12:55.768 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:55.761+0000 7f40f6c1c740 10 client.26188 fetch_fsmap learned FSMap version 39 2024-09-11T00:12:55.768 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:55.761+0000 7f40f6c1c740 1 -- 172.21.15.12:0/59208613 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55ca9350b620 con 0x55ca937d6fa0 2024-09-11T00:12:55.768 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:55.761+0000 7f40d2a00640 1 -- 172.21.15.12:0/59208613 <== mon.1 v2:172.21.15.178:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f40e0040070 con 0x55ca937d6fa0 2024-09-11T00:12:55.768 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:55.762+0000 7f40d2a00640 1 -- 172.21.15.12:0/59208613 <== mon.1 v2:172.21.15.178:3300/0 5 ==== fsmap(e 39) ==== 8378+0+0 (secure 0 0 0) 0x7f40e004d070 con 0x55ca937d6fa0 2024-09-11T00:12:55.768 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:55.762+0000 7f40f6c1c740 10 client.26188 fetch_fsmap finished waiting for FSMap version 39 2024-09-11T00:12:55.768 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:55.762+0000 7f40f6c1c740 10 client.26188 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T00:12:55.769 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:55.762+0000 7f40f6c1c740 20 client.26188 populate_metadata read hostname 'smithi012' 2024-09-11T00:12:55.769 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:55.762+0000 7f40f6c1c740 1 --2- 172.21.15.12:0/59208613 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55ca937ebb30 0x55ca937edf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:12:55.769 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:55.762+0000 7f40f6c1c740 4 client.26188 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T00:12:55.769 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:55.762+0000 7f40f6c1c740 1 -- 172.21.15.12:0/59208613 --> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ca93743600 con 0x55ca937ebb30 2024-09-11T00:12:55.769 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:55.762+0000 7f40f0a00640 1 --2- 172.21.15.12:0/59208613 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55ca937ebb30 0x55ca937edf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:12:55.769 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:55.762+0000 7f40f0a00640 1 --2- 172.21.15.12:0/59208613 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55ca937ebb30 0x55ca937edf20 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5a82d3691c017c70 server_cookie=1bc06a947c292a8b in_seq=0 out_seq=0 2024-09-11T00:12:55.769 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:55.762+0000 7f40d2a00640 10 client.26188 ms_handle_connect on v2:172.21.15.165:6838/501312984 2024-09-11T00:12:55.769 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:55.763+0000 7f40d2a00640 1 -- 172.21.15.12:0/59208613 <== mds.0 v2:172.21.15.165:6838/501312984 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55ca93743600 con 0x55ca937ebb30 2024-09-11T00:12:55.769 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:55.763+0000 7f40d2a00640 10 client.26188 handle_command_reply: tid=0 2024-09-11T00:12:55.769 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:55.763+0000 7f40f6c1c740 1 client.26188 shutdown 2024-09-11T00:12:55.769 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:55.764+0000 7f40f6c1c740 1 -- 172.21.15.12:0/59208613 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ca937d6fa0 msgr2=0x55ca937e0690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:12:55.769 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:55.764+0000 7f40f6c1c740 1 --2- 172.21.15.12:0/59208613 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ca937d6fa0 0x55ca937e0690 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f40e0043000 tx=0x7f40e00046c0 comp rx=0 tx=0).stop 2024-09-11T00:12:55.769 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:55.764+0000 7f40f6c1c740 1 -- 172.21.15.12:0/59208613 shutdown_connections 2024-09-11T00:12:55.770 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:55.764+0000 7f40f6c1c740 1 --2- 172.21.15.12:0/59208613 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55ca937ebb30 0x55ca937edf20 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:12:55.770 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:55.764+0000 7f40f6c1c740 1 --2- 172.21.15.12:0/59208613 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ca937dc310 0x55ca937e1900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:12:55.770 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:55.764+0000 7f40f6c1c740 1 --2- 172.21.15.12:0/59208613 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ca937d98d0 0x55ca937e0d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:12:55.770 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:55.764+0000 7f40d2a00640 0 client.26188 ms_handle_reset on v2:172.21.15.165:6838/501312984 2024-09-11T00:12:55.770 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:55.764+0000 7f40f6c1c740 1 --2- 172.21.15.12:0/59208613 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ca937d6fa0 0x55ca937e0690 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:12:55.770 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:55.764+0000 7f40f6c1c740 1 -- 172.21.15.12:0/59208613 >> 172.21.15.12:0/59208613 conn(0x55ca937c98e0 msgr2=0x55ca937c8a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:12:55.770 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:55.764+0000 7f40f6c1c740 1 -- 172.21.15.12:0/59208613 shutdown_connections 2024-09-11T00:12:55.770 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:55.764+0000 7f40f6c1c740 1 -- 172.21.15.12:0/59208613 wait complete. 2024-09-11T00:12:55.770 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:55.764+0000 7f40f6c1c740 20 client.26188 trim_cache size 0 max 16384 2024-09-11T00:12:55.785 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:55.782+0000 7f40f6c1c740 1 Processor -- start 2024-09-11T00:12:55.785 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:55.782+0000 7f40f6c1c740 1 -- start start 2024-09-11T00:12:55.785 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:55.782+0000 7f40f6c1c740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ca937d6fa0 0x55ca937ca380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:12:55.785 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:55.782+0000 7f40f6c1c740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ca937d98d0 0x55ca937ca8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:12:55.785 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:55.782+0000 7f40f6c1c740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ca937dc310 0x55ca937dc7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:12:55.785 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:55.782+0000 7f40f6c1c740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f40ec1a9520 con 0x55ca937d98d0 2024-09-11T00:12:55.785 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:55.782+0000 7f40f6c1c740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f40ec1ab0c0 con 0x55ca937d6fa0 2024-09-11T00:12:55.785 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:55.782+0000 7f40f6c1c740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f40ec1a9bf0 con 0x55ca937dc310 2024-09-11T00:12:55.785 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:55.782+0000 7f40f0a00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ca937d98d0 0x55ca937ca8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:12:55.785 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:55.782+0000 7f40f1e00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ca937dc310 0x55ca937dc7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:12:55.786 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:55.782+0000 7f40f0a00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ca937d98d0 0x55ca937ca8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:41962/0 (socket says 172.21.15.12:41962) 2024-09-11T00:12:55.786 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:55.782+0000 7f40f0a00640 1 -- 172.21.15.12:0/2796320358 learned_addr learned my addr 172.21.15.12:0/2796320358 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:12:55.786 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:55.782+0000 7f40f1400640 1 --2- 172.21.15.12:0/2796320358 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ca937d6fa0 0x55ca937ca380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:12:55.786 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:55.783+0000 7f40f0a00640 1 -- 172.21.15.12:0/2796320358 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ca937dc310 msgr2=0x55ca937dc7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:12:55.786 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:55.783+0000 7f40f0a00640 1 --2- 172.21.15.12:0/2796320358 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ca937dc310 0x55ca937dc7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:12:55.786 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:55.783+0000 7f40f0a00640 1 -- 172.21.15.12:0/2796320358 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ca937d6fa0 msgr2=0x55ca937ca380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:12:55.786 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:55.783+0000 7f40f0a00640 1 --2- 172.21.15.12:0/2796320358 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ca937d6fa0 0x55ca937ca380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:12:55.786 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:55.783+0000 7f40f0a00640 1 -- 172.21.15.12:0/2796320358 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ca937ee290 con 0x55ca937d98d0 2024-09-11T00:12:55.786 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:55.783+0000 7f40f1e00640 1 --2- 172.21.15.12:0/2796320358 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ca937dc310 0x55ca937dc7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:12:55.786 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:55.783+0000 7f40f1400640 1 --2- 172.21.15.12:0/2796320358 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ca937d6fa0 0x55ca937ca380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:12:55.786 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:55.783+0000 7f40f0a00640 1 --2- 172.21.15.12:0/2796320358 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ca937d98d0 0x55ca937ca8c0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f40d80a91f0 tx=0x7f40d80038b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8952e0a91b26f3e server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:12:55.790 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:55.783+0000 7f40ddc00640 1 -- 172.21.15.12:0/2796320358 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40d8020040 con 0x55ca937d98d0 2024-09-11T00:12:55.790 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:55.783+0000 7f40ddc00640 1 -- 172.21.15.12:0/2796320358 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f40d80b09b0 con 0x55ca937d98d0 2024-09-11T00:12:55.790 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:55.783+0000 7f40ddc00640 1 -- 172.21.15.12:0/2796320358 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40d80a81f0 con 0x55ca937d98d0 2024-09-11T00:12:55.790 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:55.784+0000 7f40f6c1c740 1 -- 172.21.15.12:0/2796320358 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ca937d98d0 msgr2=0x55ca937ca8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:12:55.790 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:55.784+0000 7f40f6c1c740 1 --2- 172.21.15.12:0/2796320358 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ca937d98d0 0x55ca937ca8c0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f40d80a91f0 tx=0x7f40d80038b0 comp rx=0 tx=0).stop 2024-09-11T00:12:55.790 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:55.784+0000 7f40f6c1c740 1 -- 172.21.15.12:0/2796320358 shutdown_connections 2024-09-11T00:12:55.790 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:55.784+0000 7f40f6c1c740 1 --2- 172.21.15.12:0/2796320358 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ca937dc310 0x55ca937dc7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:12:55.791 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:55.784+0000 7f40f6c1c740 1 --2- 172.21.15.12:0/2796320358 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ca937d98d0 0x55ca937ca8c0 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:12:55.791 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:55.784+0000 7f40f6c1c740 1 --2- 172.21.15.12:0/2796320358 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ca937d6fa0 0x55ca937ca380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:12:55.791 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:55.784+0000 7f40f6c1c740 1 -- 172.21.15.12:0/2796320358 >> 172.21.15.12:0/2796320358 conn(0x55ca937c98e0 msgr2=0x55ca937e0420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:12:55.791 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:55.784+0000 7f40f6c1c740 1 -- 172.21.15.12:0/2796320358 shutdown_connections 2024-09-11T00:12:55.791 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:55.784+0000 7f40f6c1c740 1 -- 172.21.15.12:0/2796320358 wait complete. 2024-09-11T00:12:55.791 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:55.785+0000 7f40f6c1c740 1 -- start start 2024-09-11T00:12:55.791 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:55.785+0000 7f40f6c1c740 1 Processor -- start 2024-09-11T00:12:55.791 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:55.785+0000 7f40f6c1c740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ca937d6fa0 0x55ca937e81f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:12:55.791 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:55.785+0000 7f40f6c1c740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ca937d98d0 0x55ca937e6ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:12:55.792 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:55.785+0000 7f40f6c1c740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ca937dc310 0x55ca937e7490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:12:55.792 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:55.785+0000 7f40f6c1c740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f40ec1aa9f0 con 0x55ca937dc310 2024-09-11T00:12:55.792 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:55.785+0000 7f40f6c1c740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f40ec1aa320 con 0x55ca937d6fa0 2024-09-11T00:12:55.792 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:55.785+0000 7f40f6c1c740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f40ec1a7500 con 0x55ca937d98d0 2024-09-11T00:12:55.792 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:55.785+0000 7f40f6c1c740 10 client.0 _ll_register_callbacks cb 0x7f40e0005dc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:12:55.792 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:55.785+0000 7f40f1400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ca937d6fa0 0x55ca937e81f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:12:55.792 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:55.786+0000 7f40f0a00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ca937d98d0 0x55ca937e6ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:12:55.792 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:55.786+0000 7f40f1400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ca937d6fa0 0x55ca937e81f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.12:42720/0 (socket says 172.21.15.12:42720) 2024-09-11T00:12:55.792 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:55.786+0000 7f40f1400640 1 -- 172.21.15.12:0/1154001175 learned_addr learned my addr 172.21.15.12:0/1154001175 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:12:55.792 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:55.786+0000 7f40f1e00640 1 --2- 172.21.15.12:0/1154001175 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ca937dc310 0x55ca937e7490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:12:55.792 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:55.786+0000 7f40f0a00640 1 -- 172.21.15.12:0/1154001175 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ca937d6fa0 msgr2=0x55ca937e81f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:12:55.793 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:55.786+0000 7f40f0a00640 1 --2- 172.21.15.12:0/1154001175 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ca937d6fa0 0x55ca937e81f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:12:55.793 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:55.786+0000 7f40f0a00640 1 -- 172.21.15.12:0/1154001175 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ca937dc310 msgr2=0x55ca937e7490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:12:55.793 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:55.786+0000 7f40f0a00640 1 --2- 172.21.15.12:0/1154001175 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ca937dc310 0x55ca937e7490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:12:55.793 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:55.786+0000 7f40f0a00640 1 -- 172.21.15.12:0/1154001175 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ca937e1c70 con 0x55ca937d98d0 2024-09-11T00:12:55.793 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:55.786+0000 7f40f1400640 1 --2- 172.21.15.12:0/1154001175 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ca937d6fa0 0x55ca937e81f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:12:55.793 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:55.786+0000 7f40f0a00640 1 --2- 172.21.15.12:0/1154001175 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ca937d98d0 0x55ca937e6ef0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f40d80a91f0 tx=0x7f40d800df90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=512b31b090edda9 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:12:55.793 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:55.786+0000 7f40d2a00640 10 client.0 ms_handle_connect on v2:172.21.15.178:3300/0 2024-09-11T00:12:55.793 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:55.786+0000 7f40d2a00640 1 -- 172.21.15.12:0/1154001175 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40d8020040 con 0x55ca937d98d0 2024-09-11T00:12:55.793 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:55.786+0000 7f40f6c1c740 1 -- 172.21.15.12:0/1154001175 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f40d80b09b0 con 0x55ca937d98d0 2024-09-11T00:12:55.793 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:55.786+0000 7f40d2a00640 1 -- 172.21.15.12:0/1154001175 <== mon.1 v2:172.21.15.178:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9208+0+0 (secure 0 0 0) 0x7f40d8097170 con 0x55ca937d98d0 2024-09-11T00:12:55.793 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:55.787+0000 7f40d2a00640 1 -- 172.21.15.12:0/1154001175 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f40d80b5460 con 0x55ca937d98d0 2024-09-11T00:12:55.793 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:55.787+0000 7f40f6c1c740 10 client.26194 fetch_fsmap learned FSMap version 39 2024-09-11T00:12:55.794 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:55.787+0000 7f40f6c1c740 1 -- 172.21.15.12:0/1154001175 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55ca93858a30 con 0x55ca937d98d0 2024-09-11T00:12:55.794 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:55.787+0000 7f40d2a00640 1 -- 172.21.15.12:0/1154001175 <== mon.1 v2:172.21.15.178:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f40d80b09b0 con 0x55ca937d98d0 2024-09-11T00:12:55.794 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:55.787+0000 7f40d2a00640 1 -- 172.21.15.12:0/1154001175 <== mon.1 v2:172.21.15.178:3300/0 5 ==== fsmap(e 39) ==== 8378+0+0 (secure 0 0 0) 0x7f40d80930b0 con 0x55ca937d98d0 2024-09-11T00:12:55.794 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:55.787+0000 7f40f6c1c740 10 client.26194 fetch_fsmap finished waiting for FSMap version 39 2024-09-11T00:12:55.794 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:55.787+0000 7f40f6c1c740 10 client.26194 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T00:12:55.794 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:55.787+0000 7f40f6c1c740 20 client.26194 populate_metadata read hostname 'smithi012' 2024-09-11T00:12:55.794 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:55.787+0000 7f40f6c1c740 1 --2- 172.21.15.12:0/1154001175 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55ca93859280 0x55ca9385b670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:12:55.794 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:55.787+0000 7f40f6c1c740 4 client.26194 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T00:12:55.794 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:55.787+0000 7f40f6c1c740 1 -- 172.21.15.12:0/1154001175 --> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55ca9373f0b0 con 0x55ca93859280 2024-09-11T00:12:55.794 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:55.787+0000 7f40f1400640 1 --2- 172.21.15.12:0/1154001175 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55ca93859280 0x55ca9385b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:12:55.794 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:55.788+0000 7f40f1400640 1 --2- 172.21.15.12:0/1154001175 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55ca93859280 0x55ca9385b670 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f10ecf1242421c40 server_cookie=b33196c9ac8177be in_seq=0 out_seq=0 2024-09-11T00:12:55.794 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:55.788+0000 7f40d2a00640 10 client.26194 ms_handle_connect on v2:172.21.15.165:6838/501312984 2024-09-11T00:12:55.839 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:55.836+0000 7f40d2a00640 1 -- 172.21.15.12:0/1154001175 <== mds.0 v2:172.21.15.165:6838/501312984 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55ca9373f0b0 con 0x55ca93859280 2024-09-11T00:12:55.839 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:55.836+0000 7f40d2a00640 10 client.26194 handle_command_reply: tid=0 2024-09-11T00:12:55.839 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:55.836+0000 7f40f6c1c740 1 client.26194 shutdown 2024-09-11T00:12:55.839 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:55.836+0000 7f40f6c1c740 1 -- 172.21.15.12:0/1154001175 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ca937d98d0 msgr2=0x55ca937e6ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:12:55.839 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:55.836+0000 7f40f6c1c740 1 --2- 172.21.15.12:0/1154001175 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ca937d98d0 0x55ca937e6ef0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f40d80a91f0 tx=0x7f40d800df90 comp rx=0 tx=0).stop 2024-09-11T00:12:55.839 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:55.837+0000 7f40f6c1c740 1 -- 172.21.15.12:0/1154001175 shutdown_connections 2024-09-11T00:12:55.840 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:55.837+0000 7f40f6c1c740 1 --2- 172.21.15.12:0/1154001175 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55ca93859280 0x55ca9385b670 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:12:55.840 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:55.837+0000 7f40f6c1c740 1 --2- 172.21.15.12:0/1154001175 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ca937dc310 0x55ca937e7490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:12:55.840 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:55.837+0000 7f40f6c1c740 1 --2- 172.21.15.12:0/1154001175 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ca937d98d0 0x55ca937e6ef0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:12:55.840 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:55.837+0000 7f40d2a00640 0 client.26194 ms_handle_reset on v2:172.21.15.165:6838/501312984 2024-09-11T00:12:55.840 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:55.837+0000 7f40f6c1c740 1 --2- 172.21.15.12:0/1154001175 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ca937d6fa0 0x55ca937e81f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:12:55.840 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:55.837+0000 7f40f6c1c740 1 -- 172.21.15.12:0/1154001175 >> 172.21.15.12:0/1154001175 conn(0x55ca937c98e0 msgr2=0x55ca937dc7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:12:55.840 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:55.837+0000 7f40f6c1c740 1 -- 172.21.15.12:0/1154001175 shutdown_connections 2024-09-11T00:12:55.840 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:55.837+0000 7f40f6c1c740 1 -- 172.21.15.12:0/1154001175 wait complete. 2024-09-11T00:12:55.840 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:55.837+0000 7f40f6c1c740 20 client.26194 trim_cache size 0 max 16384 2024-09-11T00:12:55.841 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:{ 2024-09-11T00:12:55.841 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "message": "", 2024-09-11T00:12:55.841 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "return_code": 0 2024-09-11T00:12:55.841 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:} 2024-09-11T00:12:55.842 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:55.838+0000 7f40f3e00640 1 -- 172.21.15.12:0/2340651411 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f40d40b5200 msgr2=0x7f40d40b76c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:12:55.842 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:55.838+0000 7f40f3e00640 1 --2- 172.21.15.12:0/2340651411 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f40d40b5200 0x7f40d40b76c0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f40e00096c0 tx=0x7f40e0046040 comp rx=0 tx=0).stop 2024-09-11T00:12:55.842 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:55.838+0000 7f40f3e00640 1 -- 172.21.15.12:0/2340651411 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f40ec105cf0 msgr2=0x7f40ec19b5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:12:55.842 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:55.838+0000 7f40f3e00640 1 --2- 172.21.15.12:0/2340651411 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f40ec105cf0 0x7f40ec19b5b0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f40d8005fd0 tx=0x7f40d800dde0 comp rx=0 tx=0).stop 2024-09-11T00:12:55.842 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:55.838+0000 7f40f3e00640 1 -- 172.21.15.12:0/2340651411 shutdown_connections 2024-09-11T00:12:55.842 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:55.838+0000 7f40f3e00640 1 --2- 172.21.15.12:0/2340651411 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f40d40b5200 0x7f40d40b76c0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:12:55.842 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:55.838+0000 7f40f3e00640 1 --2- 172.21.15.12:0/2340651411 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f40ec1066b0 0x7f40ec19c1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:12:55.843 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:55.838+0000 7f40f3e00640 1 --2- 172.21.15.12:0/2340651411 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f40ec105cf0 0x7f40ec19b5b0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:12:55.843 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:55.838+0000 7f40f3e00640 1 --2- 172.21.15.12:0/2340651411 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f40ec104ba0 0x7f40ec19af60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:12:55.843 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:55.838+0000 7f40f3e00640 1 -- 172.21.15.12:0/2340651411 >> 172.21.15.12:0/2340651411 conn(0x7f40ec100460 msgr2=0x7f40ec109080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:12:55.843 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:55.838+0000 7f40f3e00640 1 -- 172.21.15.12:0/2340651411 shutdown_connections 2024-09-11T00:12:55.843 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:12:55.838+0000 7f40f3e00640 1 -- 172.21.15.12:0/2340651411 wait complete. 2024-09-11T00:12:56.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:12:55 smithi165 ceph-mon[35356]: pgmap v574: 129 pgs: 129 active+clean; 43 MiB data, 837 MiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 287 KiB/s wr, 106 op/s 2024-09-11T00:12:56.312 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:12:55 smithi012 ceph-mon[30333]: pgmap v574: 129 pgs: 129 active+clean; 43 MiB data, 837 MiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 287 KiB/s wr, 106 op/s 2024-09-11T00:12:56.330 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:12:55 smithi178 ceph-mon[35919]: pgmap v574: 129 pgs: 129 active+clean; 43 MiB data, 837 MiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 287 KiB/s wr, 106 op/s 2024-09-11T00:12:56.638 INFO:tasks.workunit.client.0.smithi012.stdout: 1 170716 2.32 MB/sec execute 173 sec latency 105.999 ms 2024-09-11T00:12:57.638 INFO:tasks.workunit.client.0.smithi012.stdout: 1 172832 2.35 MB/sec execute 174 sec latency 12.610 ms 2024-09-11T00:12:58.284 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:12:57 smithi165 ceph-mon[35356]: pgmap v575: 129 pgs: 129 active+clean; 44 MiB data, 844 MiB used, 1.0 TiB / 1.0 TiB avail; 20 MiB/s rd, 592 KiB/s wr, 224 op/s 2024-09-11T00:12:58.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:12:57 smithi012 ceph-mon[30333]: pgmap v575: 129 pgs: 129 active+clean; 44 MiB data, 844 MiB used, 1.0 TiB / 1.0 TiB avail; 20 MiB/s rd, 592 KiB/s wr, 224 op/s 2024-09-11T00:12:58.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:12:57 smithi178 ceph-mon[35919]: pgmap v575: 129 pgs: 129 active+clean; 44 MiB data, 844 MiB used, 1.0 TiB / 1.0 TiB avail; 20 MiB/s rd, 592 KiB/s wr, 224 op/s 2024-09-11T00:12:58.637 INFO:tasks.workunit.client.0.smithi012.stdout: 1 173916 2.39 MB/sec execute 175 sec latency 205.091 ms 2024-09-11T00:12:59.638 INFO:tasks.workunit.client.0.smithi012.stdout: 1 174048 2.39 MB/sec execute 176 sec latency 864.957 ms 2024-09-11T00:13:00.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:12:59 smithi165 ceph-mon[35356]: pgmap v576: 129 pgs: 129 active+clean; 45 MiB data, 846 MiB used, 1.0 TiB / 1.0 TiB avail; 23 MiB/s rd, 655 KiB/s wr, 249 op/s 2024-09-11T00:13:00.312 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:12:59 smithi012 ceph-mon[30333]: pgmap v576: 129 pgs: 129 active+clean; 45 MiB data, 846 MiB used, 1.0 TiB / 1.0 TiB avail; 23 MiB/s rd, 655 KiB/s wr, 249 op/s 2024-09-11T00:13:00.330 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:12:59 smithi178 ceph-mon[35919]: pgmap v576: 129 pgs: 129 active+clean; 45 MiB data, 846 MiB used, 1.0 TiB / 1.0 TiB avail; 23 MiB/s rd, 655 KiB/s wr, 249 op/s 2024-09-11T00:13:00.638 INFO:tasks.workunit.client.0.smithi012.stdout: 1 175598 2.40 MB/sec execute 177 sec latency 10.752 ms 2024-09-11T00:13:01.638 INFO:tasks.workunit.client.0.smithi012.stdout: 1 177414 2.46 MB/sec execute 178 sec latency 293.162 ms 2024-09-11T00:13:02.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:13:01 smithi165 ceph-mon[35356]: pgmap v577: 129 pgs: 129 active+clean; 46 MiB data, 876 MiB used, 1.0 TiB / 1.0 TiB avail; 42 MiB/s rd, 854 KiB/s wr, 381 op/s 2024-09-11T00:13:02.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:13:01 smithi012 ceph-mon[30333]: pgmap v577: 129 pgs: 129 active+clean; 46 MiB data, 876 MiB used, 1.0 TiB / 1.0 TiB avail; 42 MiB/s rd, 854 KiB/s wr, 381 op/s 2024-09-11T00:13:02.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:13:01 smithi178 ceph-mon[35919]: pgmap v577: 129 pgs: 129 active+clean; 46 MiB data, 876 MiB used, 1.0 TiB / 1.0 TiB avail; 42 MiB/s rd, 854 KiB/s wr, 381 op/s 2024-09-11T00:13:02.638 INFO:tasks.workunit.client.0.smithi012.stdout: 1 177983 2.45 MB/sec execute 179 sec latency 328.250 ms 2024-09-11T00:13:03.638 INFO:tasks.workunit.client.0.smithi012.stdout: 1 180374 2.51 MB/sec execute 180 sec latency 13.267 ms 2024-09-11T00:13:04.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:13:03 smithi165 ceph-mon[35356]: pgmap v578: 129 pgs: 129 active+clean; 48 MiB data, 888 MiB used, 1.0 TiB / 1.0 TiB avail; 49 MiB/s rd, 891 KiB/s wr, 421 op/s 2024-09-11T00:13:04.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:13:03 smithi012 ceph-mon[30333]: pgmap v578: 129 pgs: 129 active+clean; 48 MiB data, 888 MiB used, 1.0 TiB / 1.0 TiB avail; 49 MiB/s rd, 891 KiB/s wr, 421 op/s 2024-09-11T00:13:04.330 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:13:03 smithi178 ceph-mon[35919]: pgmap v578: 129 pgs: 129 active+clean; 48 MiB data, 888 MiB used, 1.0 TiB / 1.0 TiB avail; 49 MiB/s rd, 891 KiB/s wr, 421 op/s 2024-09-11T00:13:04.638 INFO:tasks.workunit.client.0.smithi012.stdout: 1 181076 2.53 MB/sec execute 181 sec latency 374.276 ms 2024-09-11T00:13:05.638 INFO:tasks.workunit.client.0.smithi012.stdout: 1 181704 2.52 MB/sec execute 182 sec latency 378.973 ms 2024-09-11T00:13:06.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:13:05 smithi165 ceph-mon[35356]: pgmap v579: 129 pgs: 129 active+clean; 48 MiB data, 894 MiB used, 1.0 TiB / 1.0 TiB avail; 46 MiB/s rd, 793 KiB/s wr, 380 op/s 2024-09-11T00:13:06.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:13:05 smithi012 ceph-mon[30333]: pgmap v579: 129 pgs: 129 active+clean; 48 MiB data, 894 MiB used, 1.0 TiB / 1.0 TiB avail; 46 MiB/s rd, 793 KiB/s wr, 380 op/s 2024-09-11T00:13:06.330 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:13:05 smithi178 ceph-mon[35919]: pgmap v579: 129 pgs: 129 active+clean; 48 MiB data, 894 MiB used, 1.0 TiB / 1.0 TiB avail; 46 MiB/s rd, 793 KiB/s wr, 380 op/s 2024-09-11T00:13:06.638 INFO:tasks.workunit.client.0.smithi012.stdout: 1 184251 2.58 MB/sec execute 183 sec latency 10.195 ms 2024-09-11T00:13:07.638 INFO:tasks.workunit.client.0.smithi012.stdout: 1 184875 2.60 MB/sec execute 184 sec latency 331.481 ms 2024-09-11T00:13:08.117 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:13:07 smithi012 ceph-mon[30333]: pgmap v580: 129 pgs: 129 active+clean; 51 MiB data, 896 MiB used, 1.0 TiB / 1.0 TiB avail; 53 MiB/s rd, 1023 KiB/s wr, 469 op/s 2024-09-11T00:13:08.132 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:13:07 smithi178 ceph-mon[35919]: pgmap v580: 129 pgs: 129 active+clean; 51 MiB data, 896 MiB used, 1.0 TiB / 1.0 TiB avail; 53 MiB/s rd, 1023 KiB/s wr, 469 op/s 2024-09-11T00:13:08.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:13:07 smithi165 ceph-mon[35356]: pgmap v580: 129 pgs: 129 active+clean; 51 MiB data, 896 MiB used, 1.0 TiB / 1.0 TiB avail; 53 MiB/s rd, 1023 KiB/s wr, 469 op/s 2024-09-11T00:13:08.638 INFO:tasks.workunit.client.0.smithi012.stdout: 1 187037 2.62 MB/sec execute 185 sec latency 14.158 ms 2024-09-11T00:13:08.970 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:08.967+0000 7f28cd800640 1 -- 172.21.15.12:0/3118710190 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f28c8105ce0 msgr2=0x7f28c8106160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:13:08.970 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:08.967+0000 7f28cd800640 1 --2- 172.21.15.12:0/3118710190 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f28c8105ce0 0x7f28c8106160 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f28b4009900 tx=0x7f28b402a3f0 comp rx=0 tx=0).stop 2024-09-11T00:13:08.970 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:08.967+0000 7f28cd800640 1 -- 172.21.15.12:0/3118710190 shutdown_connections 2024-09-11T00:13:08.970 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:08.967+0000 7f28cd800640 1 --2- 172.21.15.12:0/3118710190 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f28c81066a0 0x7f28c810d020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:13:08.970 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:08.967+0000 7f28cd800640 1 --2- 172.21.15.12:0/3118710190 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f28c8105ce0 0x7f28c8106160 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:13:08.970 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:08.967+0000 7f28cd800640 1 --2- 172.21.15.12:0/3118710190 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f28c8104b90 0x7f28c8104f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:13:08.970 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:08.967+0000 7f28cd800640 1 -- 172.21.15.12:0/3118710190 >> 172.21.15.12:0/3118710190 conn(0x7f28c8100470 msgr2=0x7f28c8102890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:13:08.970 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:08.967+0000 7f28cd800640 1 -- 172.21.15.12:0/3118710190 shutdown_connections 2024-09-11T00:13:08.971 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:08.967+0000 7f28cd800640 1 -- 172.21.15.12:0/3118710190 wait complete. 2024-09-11T00:13:08.972 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:08.968+0000 7f28cd800640 1 Processor -- start 2024-09-11T00:13:08.973 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:08.968+0000 7f28cd800640 1 -- start start 2024-09-11T00:13:08.973 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:08.969+0000 7f28cd800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f28c8104b90 0x7f28c819af40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:13:08.973 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:08.969+0000 7f28cd800640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f28c8105ce0 0x7f28c819b590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:13:08.973 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:08.969+0000 7f28cd800640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f28c81066a0 0x7f28c819c1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:13:08.973 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:08.969+0000 7f28cd800640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f28c81107e0 con 0x7f28c81066a0 2024-09-11T00:13:08.973 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:08.969+0000 7f28cd800640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f28c8110660 con 0x7f28c8104b90 2024-09-11T00:13:08.973 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:08.969+0000 7f28cd800640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f28c8110960 con 0x7f28c8105ce0 2024-09-11T00:13:08.973 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:08.969+0000 7f28c6000640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f28c8105ce0 0x7f28c819b590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:13:08.973 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:08.969+0000 7f28c6000640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f28c8105ce0 0x7f28c819b590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.12:40544/0 (socket says 172.21.15.12:40544) 2024-09-11T00:13:08.973 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:08.969+0000 7f28c6000640 1 -- 172.21.15.12:0/322008537 learned_addr learned my addr 172.21.15.12:0/322008537 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:13:08.973 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:08.969+0000 7f28c6a00640 1 --2- 172.21.15.12:0/322008537 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f28c8104b90 0x7f28c819af40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:13:08.974 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:08.969+0000 7f28c7400640 1 --2- 172.21.15.12:0/322008537 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f28c81066a0 0x7f28c819c1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:13:08.974 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:08.969+0000 7f28c6000640 1 -- 172.21.15.12:0/322008537 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f28c8104b90 msgr2=0x7f28c819af40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:13:08.974 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:08.969+0000 7f28c6000640 1 --2- 172.21.15.12:0/322008537 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f28c8104b90 0x7f28c819af40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:13:08.974 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:08.969+0000 7f28c6000640 1 -- 172.21.15.12:0/322008537 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f28c81066a0 msgr2=0x7f28c819c1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:13:08.974 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:08.969+0000 7f28c6000640 1 --2- 172.21.15.12:0/322008537 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f28c81066a0 0x7f28c819c1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:13:08.974 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:08.969+0000 7f28c6000640 1 -- 172.21.15.12:0/322008537 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28c819ca80 con 0x7f28c8105ce0 2024-09-11T00:13:08.974 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:08.970+0000 7f28c6000640 1 --2- 172.21.15.12:0/322008537 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f28c8105ce0 0x7f28c819b590 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f28b4009230 tx=0x7f28b40042f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:13:08.974 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:08.970+0000 7f28bf400640 1 -- 172.21.15.12:0/322008537 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28b403f030 con 0x7f28c8105ce0 2024-09-11T00:13:08.974 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:08.970+0000 7f28bf400640 1 -- 172.21.15.12:0/322008537 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f28b4032e40 con 0x7f28c8105ce0 2024-09-11T00:13:08.974 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:08.970+0000 7f28cd800640 1 -- 172.21.15.12:0/322008537 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f28c819cd10 con 0x7f28c8105ce0 2024-09-11T00:13:08.974 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:08.970+0000 7f28bf400640 1 -- 172.21.15.12:0/322008537 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28b4035070 con 0x7f28c8105ce0 2024-09-11T00:13:08.974 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:08.971+0000 7f28cd800640 1 -- 172.21.15.12:0/322008537 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28c81a3440 con 0x7f28c8105ce0 2024-09-11T00:13:08.976 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:08.974+0000 7f28d0322740 1 Processor -- start 2024-09-11T00:13:08.976 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:08.974+0000 7f28bf400640 1 -- 172.21.15.12:0/322008537 <== mon.1 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7f28b403ba60 con 0x7f28c8105ce0 2024-09-11T00:13:08.977 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:08.974+0000 7f28d0322740 1 -- start start 2024-09-11T00:13:08.977 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:08.974+0000 7f28d0322740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563c4eaef790 0x563c4eaf1b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:13:08.977 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:08.974+0000 7f28d0322740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563c4eaf20c0 0x563c4eaf44b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:13:08.977 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:08.974+0000 7f28d0322740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563c4eaf4b00 0x563c4eaf6fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:13:08.977 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:08.974+0000 7f28d0322740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x563c4ea626b0 con 0x563c4eaef790 2024-09-11T00:13:08.977 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:08.974+0000 7f28d0322740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x563c4ea51ae0 con 0x563c4eaf20c0 2024-09-11T00:13:08.977 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:08.974+0000 7f28d0322740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x563c4ea558f0 con 0x563c4eaf4b00 2024-09-11T00:13:08.977 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:08.974+0000 7f28c6a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563c4eaf20c0 0x563c4eaf44b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:13:08.977 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:08.974+0000 7f28c6a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563c4eaf20c0 0x563c4eaf44b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.12:35600/0 (socket says 172.21.15.12:35600) 2024-09-11T00:13:08.977 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:08.974+0000 7f28c6a00640 1 -- 172.21.15.12:0/1028077172 learned_addr learned my addr 172.21.15.12:0/1028077172 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:13:08.977 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:08.974+0000 7f28c7400640 1 --2- 172.21.15.12:0/1028077172 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563c4eaef790 0x563c4eaf1b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:13:08.978 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:08.975+0000 7f28c6000640 1 --2- 172.21.15.12:0/1028077172 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563c4eaf4b00 0x563c4eaf6fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:13:08.979 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:08.975+0000 7f28c6a00640 1 -- 172.21.15.12:0/1028077172 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563c4eaf4b00 msgr2=0x563c4eaf6fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:13:08.979 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:08.975+0000 7f28c6a00640 1 --2- 172.21.15.12:0/1028077172 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563c4eaf4b00 0x563c4eaf6fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:13:08.979 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:08.975+0000 7f28c6a00640 1 -- 172.21.15.12:0/1028077172 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563c4eaef790 msgr2=0x563c4eaf1b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:13:08.979 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:08.975+0000 7f28c6a00640 1 --2- 172.21.15.12:0/1028077172 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563c4eaef790 0x563c4eaf1b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:13:08.979 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:08.975+0000 7f28c6a00640 1 -- 172.21.15.12:0/1028077172 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563c4ea1e780 con 0x563c4eaf20c0 2024-09-11T00:13:08.980 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:08.975+0000 7f28c6a00640 1 --2- 172.21.15.12:0/1028077172 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563c4eaf20c0 0x563c4eaf44b0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f28b8009cf0 tx=0x7f28b8011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=35b96767678294d3 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:13:08.980 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:08.975+0000 7f28b3e00640 1 -- 172.21.15.12:0/1028077172 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28b801e070 con 0x563c4eaf20c0 2024-09-11T00:13:08.980 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:08.975+0000 7f28b3e00640 1 -- 172.21.15.12:0/1028077172 <== mon.2 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f28b8019ae0 con 0x563c4eaf20c0 2024-09-11T00:13:08.980 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:08.975+0000 7f28b3e00640 1 -- 172.21.15.12:0/1028077172 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28b800e980 con 0x563c4eaf20c0 2024-09-11T00:13:08.980 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:08.976+0000 7f28bf400640 1 --2- 172.21.15.12:0/322008537 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f28ac0b52e0 0x7f28ac0b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:13:08.980 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:08.976+0000 7f28bf400640 1 -- 172.21.15.12:0/322008537 <== mon.1 v2:172.21.15.178:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9208+0+0 (secure 0 0 0) 0x7f28b41067b0 con 0x7f28c8105ce0 2024-09-11T00:13:08.980 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:08.976+0000 7f28c7400640 1 --2- 172.21.15.12:0/322008537 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f28ac0b52e0 0x7f28ac0b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:13:08.980 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:08.976+0000 7f28d0322740 1 -- 172.21.15.12:0/1028077172 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563c4eaf20c0 msgr2=0x563c4eaf44b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:13:08.980 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:08.976+0000 7f28d0322740 1 --2- 172.21.15.12:0/1028077172 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563c4eaf20c0 0x563c4eaf44b0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f28b8009cf0 tx=0x7f28b8011810 comp rx=0 tx=0).stop 2024-09-11T00:13:08.980 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:08.976+0000 7f28d0322740 1 -- 172.21.15.12:0/1028077172 shutdown_connections 2024-09-11T00:13:08.980 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:08.977+0000 7f28d0322740 1 --2- 172.21.15.12:0/1028077172 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563c4eaf4b00 0x563c4eaf6fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:13:08.980 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:08.977+0000 7f28d0322740 1 --2- 172.21.15.12:0/1028077172 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563c4eaf20c0 0x563c4eaf44b0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:13:08.981 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:08.977+0000 7f28d0322740 1 --2- 172.21.15.12:0/1028077172 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563c4eaef790 0x563c4eaf1b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:13:08.981 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:08.977+0000 7f28d0322740 1 -- 172.21.15.12:0/1028077172 >> 172.21.15.12:0/1028077172 conn(0x563c4eaea8e0 msgr2=0x563c4eaeb670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:13:08.981 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:08.977+0000 7f28d0322740 1 -- 172.21.15.12:0/1028077172 shutdown_connections 2024-09-11T00:13:08.981 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:08.977+0000 7f28c7400640 1 --2- 172.21.15.12:0/322008537 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f28ac0b52e0 0x7f28ac0b77a0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f28c819bcc0 tx=0x7f28c0009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:13:08.981 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:08.977+0000 7f28d0322740 1 -- 172.21.15.12:0/1028077172 wait complete. 2024-09-11T00:13:08.981 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:08.977+0000 7f28d0322740 1 -- start start 2024-09-11T00:13:08.984 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:08.977+0000 7f28d0322740 1 Processor -- start 2024-09-11T00:13:08.985 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:08.978+0000 7f28d0322740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563c4eaef790 0x563c4eaf8e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:13:08.985 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:08.978+0000 7f28d0322740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563c4eaf20c0 0x563c4eaf9510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:13:08.985 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:08.978+0000 7f28d0322740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563c4eaf4b00 0x563c4eafa0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:13:08.985 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:08.978+0000 7f28d0322740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x563c4e989ab0 con 0x563c4eaf20c0 2024-09-11T00:13:08.985 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:08.978+0000 7f28d0322740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x563c4e9f8570 con 0x563c4eaef790 2024-09-11T00:13:08.985 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:08.978+0000 7f28d0322740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x563c4eaf7ae0 con 0x563c4eaf4b00 2024-09-11T00:13:08.985 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:08.978+0000 7f28d0322740 10 client.0 _ll_register_callbacks cb 0x563c4ea07100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:13:08.985 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:08.978+0000 7f28c6a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563c4eaef790 0x563c4eaf8e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:13:08.985 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:08.978+0000 7f28c6000640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563c4eaf20c0 0x563c4eaf9510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:13:08.985 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:08.978+0000 7f28c6a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563c4eaef790 0x563c4eaf8e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.12:35614/0 (socket says 172.21.15.12:35614) 2024-09-11T00:13:08.986 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:08.978+0000 7f28c6a00640 1 -- 172.21.15.12:0/1124428083 learned_addr learned my addr 172.21.15.12:0/1124428083 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:13:08.986 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:08.978+0000 7f28c7400640 1 --2- 172.21.15.12:0/1124428083 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563c4eaf4b00 0x563c4eafa0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:13:08.986 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:08.979+0000 7f28c6a00640 1 -- 172.21.15.12:0/1124428083 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563c4eaf4b00 msgr2=0x563c4eafa0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:13:08.986 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:08.979+0000 7f28c6a00640 1 --2- 172.21.15.12:0/1124428083 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563c4eaf4b00 0x563c4eafa0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:13:08.986 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:08.979+0000 7f28c6a00640 1 -- 172.21.15.12:0/1124428083 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563c4eaf20c0 msgr2=0x563c4eaf9510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:13:08.986 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:08.979+0000 7f28c6a00640 1 --2- 172.21.15.12:0/1124428083 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563c4eaf20c0 0x563c4eaf9510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:13:08.986 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:08.979+0000 7f28c6a00640 1 -- 172.21.15.12:0/1124428083 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563c4e7b3790 con 0x563c4eaef790 2024-09-11T00:13:08.986 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:08.979+0000 7f28c6000640 1 --2- 172.21.15.12:0/1124428083 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563c4eaf20c0 0x563c4eaf9510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:13:08.986 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:08.979+0000 7f28c7400640 1 --2- 172.21.15.12:0/1124428083 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563c4eaf4b00 0x563c4eafa0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T00:13:08.986 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:08.979+0000 7f28c6a00640 1 --2- 172.21.15.12:0/1124428083 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563c4eaef790 0x563c4eaf8e70 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f28b8000c00 tx=0x7f28b8019660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=928a7989383ae01b server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:13:08.986 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:08.979+0000 7f28b0c00640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-09-11T00:13:08.987 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:08.979+0000 7f28b0c00640 1 -- 172.21.15.12:0/1124428083 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28b801e050 con 0x563c4eaef790 2024-09-11T00:13:08.987 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:08.979+0000 7f28d0322740 1 -- 172.21.15.12:0/1124428083 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f28b8019ae0 con 0x563c4eaef790 2024-09-11T00:13:08.987 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:08.979+0000 7f28b0c00640 1 -- 172.21.15.12:0/1124428083 <== mon.2 v2:172.21.15.165:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9208+0+0 (secure 0 0 0) 0x7f28b8005240 con 0x563c4eaef790 2024-09-11T00:13:08.987 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:08.979+0000 7f28b0c00640 1 -- 172.21.15.12:0/1124428083 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f28b8033050 con 0x563c4eaef790 2024-09-11T00:13:08.987 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:08.979+0000 7f28d0322740 10 client.26210 fetch_fsmap learned FSMap version 39 2024-09-11T00:13:08.987 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:08.979+0000 7f28d0322740 1 -- 172.21.15.12:0/1124428083 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x563c4e82c620 con 0x563c4eaef790 2024-09-11T00:13:08.987 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:08.980+0000 7f28b0c00640 1 -- 172.21.15.12:0/1124428083 <== mon.2 v2:172.21.15.165:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f28b8019ae0 con 0x563c4eaef790 2024-09-11T00:13:08.987 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:08.980+0000 7f28b0c00640 1 -- 172.21.15.12:0/1124428083 <== mon.2 v2:172.21.15.165:3300/0 5 ==== fsmap(e 39) ==== 8378+0+0 (secure 0 0 0) 0x7f28b80046c0 con 0x563c4eaef790 2024-09-11T00:13:08.987 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:08.980+0000 7f28d0322740 10 client.26210 fetch_fsmap finished waiting for FSMap version 39 2024-09-11T00:13:08.987 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:08.980+0000 7f28d0322740 10 client.26210 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T00:13:08.987 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:08.980+0000 7f28d0322740 20 client.26210 populate_metadata read hostname 'smithi012' 2024-09-11T00:13:08.987 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:08.980+0000 7f28d0322740 1 --2- 172.21.15.12:0/1124428083 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x563c4eb04580 0x563c4eb069e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:13:08.988 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:08.980+0000 7f28d0322740 4 client.26210 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T00:13:08.988 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:08.980+0000 7f28d0322740 1 -- 172.21.15.12:0/1124428083 --> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563c4ea64600 con 0x563c4eb04580 2024-09-11T00:13:08.988 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:08.980+0000 7f28c6000640 1 --2- 172.21.15.12:0/1124428083 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x563c4eb04580 0x563c4eb069e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:13:08.988 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:08.981+0000 7f28c6000640 1 --2- 172.21.15.12:0/1124428083 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x563c4eb04580 0x563c4eb069e0 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=91593a792cb9329c server_cookie=daa7ba2608c0b1c2 in_seq=0 out_seq=0 2024-09-11T00:13:08.988 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:08.981+0000 7f28b0c00640 10 client.26210 ms_handle_connect on v2:172.21.15.165:6838/501312984 2024-09-11T00:13:08.988 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:08.981+0000 7f28b0c00640 1 -- 172.21.15.12:0/1124428083 <== mds.0 v2:172.21.15.165:6838/501312984 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x563c4ea64600 con 0x563c4eb04580 2024-09-11T00:13:08.988 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:08.981+0000 7f28b0c00640 10 client.26210 handle_command_reply: tid=0 2024-09-11T00:13:08.988 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:08.982+0000 7f28d0322740 1 client.26210 shutdown 2024-09-11T00:13:08.988 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:08.982+0000 7f28d0322740 1 -- 172.21.15.12:0/1124428083 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563c4eaef790 msgr2=0x563c4eaf8e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:13:08.988 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:08.982+0000 7f28d0322740 1 --2- 172.21.15.12:0/1124428083 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563c4eaef790 0x563c4eaf8e70 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f28b8000c00 tx=0x7f28b8019660 comp rx=0 tx=0).stop 2024-09-11T00:13:08.988 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:08.982+0000 7f28d0322740 1 -- 172.21.15.12:0/1124428083 shutdown_connections 2024-09-11T00:13:08.989 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:08.982+0000 7f28d0322740 1 --2- 172.21.15.12:0/1124428083 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x563c4eb04580 0x563c4eb069e0 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:13:08.989 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:08.982+0000 7f28d0322740 1 --2- 172.21.15.12:0/1124428083 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563c4eaf4b00 0x563c4eafa0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:13:08.989 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:08.982+0000 7f28d0322740 1 --2- 172.21.15.12:0/1124428083 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563c4eaf20c0 0x563c4eaf9510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:13:08.989 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:08.982+0000 7f28b0c00640 0 client.26210 ms_handle_reset on v2:172.21.15.165:6838/501312984 2024-09-11T00:13:08.989 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:08.982+0000 7f28d0322740 1 --2- 172.21.15.12:0/1124428083 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563c4eaef790 0x563c4eaf8e70 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:13:08.989 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:08.982+0000 7f28d0322740 1 -- 172.21.15.12:0/1124428083 >> 172.21.15.12:0/1124428083 conn(0x563c4eaea8e0 msgr2=0x563c4eae9a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:13:08.989 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:08.982+0000 7f28d0322740 1 -- 172.21.15.12:0/1124428083 shutdown_connections 2024-09-11T00:13:08.989 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:08.982+0000 7f28d0322740 1 -- 172.21.15.12:0/1124428083 wait complete. 2024-09-11T00:13:08.989 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:08.982+0000 7f28d0322740 20 client.26210 trim_cache size 0 max 16384 2024-09-11T00:13:09.002 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:08.999+0000 7f28d0322740 1 Processor -- start 2024-09-11T00:13:09.002 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:08.999+0000 7f28d0322740 1 -- start start 2024-09-11T00:13:09.002 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:08.999+0000 7f28d0322740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563c4eaef790 0x563c4eaf8be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:13:09.002 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:08.999+0000 7f28d0322740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563c4eaf20c0 0x563c4eaf4ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:13:09.003 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:08.999+0000 7f28d0322740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563c4eaf4b00 0x563c4eaf5420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:13:09.003 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:08.999+0000 7f28d0322740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f28c81ad900 con 0x563c4eaf4b00 2024-09-11T00:13:09.003 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:08.999+0000 7f28d0322740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f28c81af4a0 con 0x563c4eaef790 2024-09-11T00:13:09.003 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:08.999+0000 7f28d0322740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f28c81adfd0 con 0x563c4eaf20c0 2024-09-11T00:13:09.003 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:09.000+0000 7f28c6a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563c4eaef790 0x563c4eaf8be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:13:09.003 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:09.000+0000 7f28c6000640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563c4eaf20c0 0x563c4eaf4ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:13:09.003 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:09.000+0000 7f28c6a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563c4eaef790 0x563c4eaf8be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.12:35626/0 (socket says 172.21.15.12:35626) 2024-09-11T00:13:09.003 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:09.000+0000 7f28c6a00640 1 -- 172.21.15.12:0/873438789 learned_addr learned my addr 172.21.15.12:0/873438789 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:13:09.003 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:09.000+0000 7f28c7400640 1 --2- 172.21.15.12:0/873438789 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563c4eaf4b00 0x563c4eaf5420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:13:09.003 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:09.000+0000 7f28c6000640 1 -- 172.21.15.12:0/873438789 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563c4eaef790 msgr2=0x563c4eaf8be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:13:09.003 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:09.000+0000 7f28c6000640 1 --2- 172.21.15.12:0/873438789 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563c4eaef790 0x563c4eaf8be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:13:09.004 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:09.000+0000 7f28c6000640 1 -- 172.21.15.12:0/873438789 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563c4eaf4b00 msgr2=0x563c4eaf5420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:13:09.004 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:09.000+0000 7f28c6000640 1 --2- 172.21.15.12:0/873438789 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563c4eaf4b00 0x563c4eaf5420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:13:09.004 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:09.000+0000 7f28c6000640 1 -- 172.21.15.12:0/873438789 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563c4eaf1ef0 con 0x563c4eaf20c0 2024-09-11T00:13:09.004 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:09.000+0000 7f28c6000640 1 --2- 172.21.15.12:0/873438789 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563c4eaf20c0 0x563c4eaf4ee0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f28b4103e80 tx=0x7f28b40b4040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e64203a656f6be4e server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:13:09.007 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:09.000+0000 7f28b3400640 1 -- 172.21.15.12:0/873438789 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28b4049070 con 0x563c4eaf20c0 2024-09-11T00:13:09.007 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:09.001+0000 7f28b3400640 1 -- 172.21.15.12:0/873438789 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f28b40cc650 con 0x563c4eaf20c0 2024-09-11T00:13:09.007 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:09.001+0000 7f28b3400640 1 -- 172.21.15.12:0/873438789 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28b40d60d0 con 0x563c4eaf20c0 2024-09-11T00:13:09.007 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:09.001+0000 7f28d0322740 1 -- 172.21.15.12:0/873438789 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563c4eaf20c0 msgr2=0x563c4eaf4ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:13:09.007 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:09.001+0000 7f28d0322740 1 --2- 172.21.15.12:0/873438789 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563c4eaf20c0 0x563c4eaf4ee0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f28b4103e80 tx=0x7f28b40b4040 comp rx=0 tx=0).stop 2024-09-11T00:13:09.007 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:09.001+0000 7f28d0322740 1 -- 172.21.15.12:0/873438789 shutdown_connections 2024-09-11T00:13:09.008 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:09.001+0000 7f28d0322740 1 --2- 172.21.15.12:0/873438789 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563c4eaf4b00 0x563c4eaf5420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:13:09.008 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:09.001+0000 7f28d0322740 1 --2- 172.21.15.12:0/873438789 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563c4eaf20c0 0x563c4eaf4ee0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:13:09.008 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:09.001+0000 7f28d0322740 1 --2- 172.21.15.12:0/873438789 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563c4eaef790 0x563c4eaf8be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:13:09.008 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:09.001+0000 7f28d0322740 1 -- 172.21.15.12:0/873438789 >> 172.21.15.12:0/873438789 conn(0x563c4eaea8e0 msgr2=0x563c4eaf8030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:13:09.008 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:09.001+0000 7f28d0322740 1 -- 172.21.15.12:0/873438789 shutdown_connections 2024-09-11T00:13:09.008 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:09.001+0000 7f28d0322740 1 -- 172.21.15.12:0/873438789 wait complete. 2024-09-11T00:13:09.008 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:09.002+0000 7f28d0322740 1 -- start start 2024-09-11T00:13:09.008 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:09.002+0000 7f28d0322740 1 Processor -- start 2024-09-11T00:13:09.008 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:09.003+0000 7f28d0322740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563c4eaef790 0x563c4eb005f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:13:09.008 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:09.003+0000 7f28d0322740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563c4eaf20c0 0x563c4eaff2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:13:09.008 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:09.003+0000 7f28d0322740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563c4eaf4b00 0x563c4eaff830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:13:09.008 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:09.003+0000 7f28d0322740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f28c81aedd0 con 0x563c4eaf4b00 2024-09-11T00:13:09.009 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:09.003+0000 7f28d0322740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f28c81ae700 con 0x563c4eaef790 2024-09-11T00:13:09.009 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:09.003+0000 7f28d0322740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f28c81ab8e0 con 0x563c4eaf20c0 2024-09-11T00:13:09.009 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:09.003+0000 7f28d0322740 10 client.0 _ll_register_callbacks cb 0x7f28b801c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:13:09.009 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:09.003+0000 7f28c6a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563c4eaef790 0x563c4eb005f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:13:09.009 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:09.003+0000 7f28c6a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563c4eaef790 0x563c4eb005f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.12:35632/0 (socket says 172.21.15.12:35632) 2024-09-11T00:13:09.009 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:09.003+0000 7f28c6a00640 1 -- 172.21.15.12:0/3917824770 learned_addr learned my addr 172.21.15.12:0/3917824770 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:13:09.009 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:09.003+0000 7f28c7400640 1 --2- 172.21.15.12:0/3917824770 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563c4eaf4b00 0x563c4eaff830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:13:09.009 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:09.003+0000 7f28c6000640 1 --2- 172.21.15.12:0/3917824770 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563c4eaf20c0 0x563c4eaff2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:13:09.009 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:09.003+0000 7f28c6a00640 1 -- 172.21.15.12:0/3917824770 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563c4eaf20c0 msgr2=0x563c4eaff2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:13:09.009 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:09.003+0000 7f28c6a00640 1 --2- 172.21.15.12:0/3917824770 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563c4eaf20c0 0x563c4eaff2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:13:09.009 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:09.003+0000 7f28c6a00640 1 -- 172.21.15.12:0/3917824770 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563c4eaf4b00 msgr2=0x563c4eaff830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:13:09.009 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:09.003+0000 7f28c6a00640 1 --2- 172.21.15.12:0/3917824770 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563c4eaf4b00 0x563c4eaff830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:13:09.010 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:09.003+0000 7f28c6a00640 1 -- 172.21.15.12:0/3917824770 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563c4eaffd70 con 0x563c4eaef790 2024-09-11T00:13:09.010 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:09.003+0000 7f28c6000640 1 --2- 172.21.15.12:0/3917824770 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563c4eaf20c0 0x563c4eaff2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T00:13:09.010 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:09.004+0000 7f28c6a00640 1 --2- 172.21.15.12:0/3917824770 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563c4eaef790 0x563c4eb005f0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f28b80059f0 tx=0x7f28b8003820 comp rx=0 tx=0).ready entity=mon.2 client_cookie=28f1258fbbedf746 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:13:09.010 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:09.004+0000 7f28b0c00640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-09-11T00:13:09.010 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:09.004+0000 7f28b0c00640 1 -- 172.21.15.12:0/3917824770 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28b801e040 con 0x563c4eaef790 2024-09-11T00:13:09.010 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:09.004+0000 7f28d0322740 1 -- 172.21.15.12:0/3917824770 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f28b40cc650 con 0x563c4eaef790 2024-09-11T00:13:09.010 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:09.004+0000 7f28b0c00640 1 -- 172.21.15.12:0/3917824770 <== mon.2 v2:172.21.15.165:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9208+0+0 (secure 0 0 0) 0x7f28b8003a10 con 0x563c4eaef790 2024-09-11T00:13:09.010 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:09.004+0000 7f28b0c00640 1 -- 172.21.15.12:0/3917824770 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f28b80196d0 con 0x563c4eaef790 2024-09-11T00:13:09.010 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:09.004+0000 7f28d0322740 10 client.26216 fetch_fsmap learned FSMap version 39 2024-09-11T00:13:09.010 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:09.004+0000 7f28d0322740 1 -- 172.21.15.12:0/3917824770 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x563c4eafeb30 con 0x563c4eaef790 2024-09-11T00:13:09.010 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:09.005+0000 7f28b0c00640 1 -- 172.21.15.12:0/3917824770 <== mon.2 v2:172.21.15.165:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f28b40cc650 con 0x563c4eaef790 2024-09-11T00:13:09.010 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:09.005+0000 7f28b0c00640 1 -- 172.21.15.12:0/3917824770 <== mon.2 v2:172.21.15.165:3300/0 5 ==== fsmap(e 39) ==== 8378+0+0 (secure 0 0 0) 0x7f28b80044f0 con 0x563c4eaef790 2024-09-11T00:13:09.011 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:09.005+0000 7f28d0322740 10 client.26216 fetch_fsmap finished waiting for FSMap version 39 2024-09-11T00:13:09.011 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:09.005+0000 7f28d0322740 10 client.26216 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T00:13:09.011 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:09.005+0000 7f28d0322740 20 client.26216 populate_metadata read hostname 'smithi012' 2024-09-11T00:13:09.011 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:09.005+0000 7f28d0322740 1 --2- 172.21.15.12:0/3917824770 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x563c4eb71bf0 0x563c4eb74010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:13:09.011 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:09.005+0000 7f28d0322740 4 client.26216 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T00:13:09.011 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:09.005+0000 7f28d0322740 1 -- 172.21.15.12:0/3917824770 --> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] -- command(tid 0: {"prefix": "flush journal"}) -- 0x563c4eaf92c0 con 0x563c4eb71bf0 2024-09-11T00:13:09.011 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:09.005+0000 7f28c6000640 1 --2- 172.21.15.12:0/3917824770 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x563c4eb71bf0 0x563c4eb74010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:13:09.011 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:09.006+0000 7f28c6000640 1 --2- 172.21.15.12:0/3917824770 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x563c4eb71bf0 0x563c4eb74010 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a600c26cdfb87054 server_cookie=23364123f104a62d in_seq=0 out_seq=0 2024-09-11T00:13:09.012 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:09.006+0000 7f28b0c00640 10 client.26216 ms_handle_connect on v2:172.21.15.165:6838/501312984 2024-09-11T00:13:09.063 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:09.061+0000 7f28b0c00640 1 -- 172.21.15.12:0/3917824770 <== mds.0 v2:172.21.15.165:6838/501312984 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x563c4eaf92c0 con 0x563c4eb71bf0 2024-09-11T00:13:09.063 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:09.061+0000 7f28b0c00640 10 client.26216 handle_command_reply: tid=0 2024-09-11T00:13:09.063 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:09.061+0000 7f28d0322740 1 client.26216 shutdown 2024-09-11T00:13:09.064 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:09.062+0000 7f28d0322740 1 -- 172.21.15.12:0/3917824770 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563c4eaef790 msgr2=0x563c4eb005f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:13:09.065 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:09.062+0000 7f28d0322740 1 --2- 172.21.15.12:0/3917824770 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563c4eaef790 0x563c4eb005f0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f28b80059f0 tx=0x7f28b8003820 comp rx=0 tx=0).stop 2024-09-11T00:13:09.065 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:09.062+0000 7f28d0322740 1 -- 172.21.15.12:0/3917824770 shutdown_connections 2024-09-11T00:13:09.065 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:09.062+0000 7f28d0322740 1 --2- 172.21.15.12:0/3917824770 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x563c4eb71bf0 0x563c4eb74010 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:13:09.065 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:09.062+0000 7f28d0322740 1 --2- 172.21.15.12:0/3917824770 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563c4eaf4b00 0x563c4eaff830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:13:09.065 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:09.062+0000 7f28d0322740 1 --2- 172.21.15.12:0/3917824770 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563c4eaf20c0 0x563c4eaff2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:13:09.065 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:09.062+0000 7f28b0c00640 0 client.26216 ms_handle_reset on v2:172.21.15.165:6838/501312984 2024-09-11T00:13:09.065 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:09.062+0000 7f28d0322740 1 --2- 172.21.15.12:0/3917824770 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563c4eaef790 0x563c4eb005f0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:13:09.065 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:09.062+0000 7f28d0322740 1 -- 172.21.15.12:0/3917824770 >> 172.21.15.12:0/3917824770 conn(0x563c4eaea8e0 msgr2=0x563c4eaf8030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:13:09.065 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:09.062+0000 7f28d0322740 1 -- 172.21.15.12:0/3917824770 shutdown_connections 2024-09-11T00:13:09.065 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:09.062+0000 7f28d0322740 1 -- 172.21.15.12:0/3917824770 wait complete. 2024-09-11T00:13:09.065 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:09.062+0000 7f28d0322740 20 client.26216 trim_cache size 0 max 16384 2024-09-11T00:13:09.066 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:{ 2024-09-11T00:13:09.066 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "message": "", 2024-09-11T00:13:09.066 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "return_code": 0 2024-09-11T00:13:09.066 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:} 2024-09-11T00:13:09.066 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:09.063+0000 7f28cd800640 1 -- 172.21.15.12:0/322008537 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f28ac0b52e0 msgr2=0x7f28ac0b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:13:09.067 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:09.063+0000 7f28cd800640 1 --2- 172.21.15.12:0/322008537 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f28ac0b52e0 0x7f28ac0b77a0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f28c819bcc0 tx=0x7f28c0009500 comp rx=0 tx=0).stop 2024-09-11T00:13:09.067 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:09.063+0000 7f28cd800640 1 -- 172.21.15.12:0/322008537 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f28c8105ce0 msgr2=0x7f28c819b590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:13:09.067 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:09.063+0000 7f28cd800640 1 --2- 172.21.15.12:0/322008537 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f28c8105ce0 0x7f28c819b590 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f28b4009230 tx=0x7f28b40042f0 comp rx=0 tx=0).stop 2024-09-11T00:13:09.067 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:09.063+0000 7f28cd800640 1 -- 172.21.15.12:0/322008537 shutdown_connections 2024-09-11T00:13:09.067 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:09.063+0000 7f28cd800640 1 --2- 172.21.15.12:0/322008537 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f28ac0b52e0 0x7f28ac0b77a0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:13:09.067 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:09.063+0000 7f28cd800640 1 --2- 172.21.15.12:0/322008537 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f28c81066a0 0x7f28c819c1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:13:09.067 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:09.063+0000 7f28cd800640 1 --2- 172.21.15.12:0/322008537 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f28c8105ce0 0x7f28c819b590 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:13:09.067 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:09.063+0000 7f28cd800640 1 --2- 172.21.15.12:0/322008537 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f28c8104b90 0x7f28c819af40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:13:09.067 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:09.063+0000 7f28cd800640 1 -- 172.21.15.12:0/322008537 >> 172.21.15.12:0/322008537 conn(0x7f28c8100470 msgr2=0x7f28c81090d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:13:09.067 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:09.064+0000 7f28cd800640 1 -- 172.21.15.12:0/322008537 shutdown_connections 2024-09-11T00:13:09.067 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:09.064+0000 7f28cd800640 1 -- 172.21.15.12:0/322008537 wait complete. 2024-09-11T00:13:09.638 INFO:tasks.workunit.client.0.smithi012.stdout: 1 188329 2.67 MB/sec execute 186 sec latency 190.011 ms 2024-09-11T00:13:10.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:13:09 smithi165 ceph-mon[35356]: pgmap v581: 129 pgs: 129 active+clean; 51 MiB data, 905 MiB used, 1.0 TiB / 1.0 TiB avail; 52 MiB/s rd, 781 KiB/s wr, 409 op/s 2024-09-11T00:13:10.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:13:09 smithi012 ceph-mon[30333]: pgmap v581: 129 pgs: 129 active+clean; 51 MiB data, 905 MiB used, 1.0 TiB / 1.0 TiB avail; 52 MiB/s rd, 781 KiB/s wr, 409 op/s 2024-09-11T00:13:10.330 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:13:09 smithi178 ceph-mon[35919]: pgmap v581: 129 pgs: 129 active+clean; 51 MiB data, 905 MiB used, 1.0 TiB / 1.0 TiB avail; 52 MiB/s rd, 781 KiB/s wr, 409 op/s 2024-09-11T00:13:10.638 INFO:tasks.workunit.client.0.smithi012.stdout: 1 190384 2.69 MB/sec execute 187 sec latency 13.083 ms 2024-09-11T00:13:11.638 INFO:tasks.workunit.client.0.smithi012.stdout: 1 191525 2.73 MB/sec execute 188 sec latency 463.139 ms 2024-09-11T00:13:12.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:13:11 smithi165 ceph-mon[35356]: pgmap v582: 129 pgs: 129 active+clean; 49 MiB data, 918 MiB used, 1.0 TiB / 1.0 TiB avail; 52 MiB/s rd, 871 KiB/s wr, 432 op/s 2024-09-11T00:13:12.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:13:11 smithi012 ceph-mon[30333]: pgmap v582: 129 pgs: 129 active+clean; 49 MiB data, 918 MiB used, 1.0 TiB / 1.0 TiB avail; 52 MiB/s rd, 871 KiB/s wr, 432 op/s 2024-09-11T00:13:12.330 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:13:11 smithi178 ceph-mon[35919]: pgmap v582: 129 pgs: 129 active+clean; 49 MiB data, 918 MiB used, 1.0 TiB / 1.0 TiB avail; 52 MiB/s rd, 871 KiB/s wr, 432 op/s 2024-09-11T00:13:12.638 INFO:tasks.workunit.client.0.smithi012.stdout: 1 192450 2.73 MB/sec execute 189 sec latency 28.842 ms 2024-09-11T00:13:13.639 INFO:tasks.workunit.client.0.smithi012.stdout: 1 194754 2.78 MB/sec execute 190 sec latency 18.282 ms 2024-09-11T00:13:14.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:13:13 smithi165 ceph-mon[35356]: pgmap v583: 129 pgs: 129 active+clean; 51 MiB data, 919 MiB used, 1.0 TiB / 1.0 TiB avail; 43 MiB/s rd, 898 KiB/s wr, 411 op/s 2024-09-11T00:13:14.312 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:13:13 smithi012 ceph-mon[30333]: pgmap v583: 129 pgs: 129 active+clean; 51 MiB data, 919 MiB used, 1.0 TiB / 1.0 TiB avail; 43 MiB/s rd, 898 KiB/s wr, 411 op/s 2024-09-11T00:13:14.330 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:13:13 smithi178 ceph-mon[35919]: pgmap v583: 129 pgs: 129 active+clean; 51 MiB data, 919 MiB used, 1.0 TiB / 1.0 TiB avail; 43 MiB/s rd, 898 KiB/s wr, 411 op/s 2024-09-11T00:13:14.638 INFO:tasks.workunit.client.0.smithi012.stdout: 1 195654 2.80 MB/sec execute 191 sec latency 169.216 ms 2024-09-11T00:13:15.639 INFO:tasks.workunit.client.0.smithi012.stdout: 1 197872 2.84 MB/sec execute 192 sec latency 13.154 ms 2024-09-11T00:13:16.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:13:15 smithi165 ceph-mon[35356]: pgmap v584: 129 pgs: 129 active+clean; 51 MiB data, 921 MiB used, 1.0 TiB / 1.0 TiB avail; 32 MiB/s rd, 743 KiB/s wr, 322 op/s 2024-09-11T00:13:16.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:13:15 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T00:13:16.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:13:15 smithi012 ceph-mon[30333]: pgmap v584: 129 pgs: 129 active+clean; 51 MiB data, 921 MiB used, 1.0 TiB / 1.0 TiB avail; 32 MiB/s rd, 743 KiB/s wr, 322 op/s 2024-09-11T00:13:16.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:13:15 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T00:13:16.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:13:15 smithi178 ceph-mon[35919]: pgmap v584: 129 pgs: 129 active+clean; 51 MiB data, 921 MiB used, 1.0 TiB / 1.0 TiB avail; 32 MiB/s rd, 743 KiB/s wr, 322 op/s 2024-09-11T00:13:16.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:13:15 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T00:13:16.638 INFO:tasks.workunit.client.0.smithi012.stdout: 1 199081 2.87 MB/sec execute 193 sec latency 30.804 ms 2024-09-11T00:13:17.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:13:16 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-11T00:13:17.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:13:16 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-11T00:13:17.283 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:13:16 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-11T00:13:17.283 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:13:16 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-11T00:13:17.283 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:13:16 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-09-11T00:13:17.283 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:13:16 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"} : dispatch 2024-09-11T00:13:17.283 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:13:16 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"} : dispatch 2024-09-11T00:13:17.283 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:13:16 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"} : dispatch 2024-09-11T00:13:17.283 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:13:16 smithi165 ceph-mon[35356]: Adjusting osd_memory_target on smithi165 to 176.0M 2024-09-11T00:13:17.283 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:13:16 smithi165 ceph-mon[35356]: Adjusting osd_memory_target on smithi178 to 176.0M 2024-09-11T00:13:17.283 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:13:16 smithi165 ceph-mon[35356]: Unable to set osd_memory_target on smithi165 to 184589619: error parsing value: Value '184589619' is below minimum 939524096 2024-09-11T00:13:17.283 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:13:16 smithi165 ceph-mon[35356]: Unable to set osd_memory_target on smithi178 to 184560230: error parsing value: Value '184560230' is below minimum 939524096 2024-09-11T00:13:17.283 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:13:16 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-11T00:13:17.284 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:13:16 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-11T00:13:17.284 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:13:16 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-09-11T00:13:17.284 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:13:16 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"} : dispatch 2024-09-11T00:13:17.284 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:13:16 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:13:17.284 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:13:16 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T00:13:17.284 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:13:16 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:13:17.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:13:16 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-11T00:13:17.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:13:16 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-11T00:13:17.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:13:16 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-11T00:13:17.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:13:16 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-11T00:13:17.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:13:16 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-09-11T00:13:17.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:13:16 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"} : dispatch 2024-09-11T00:13:17.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:13:16 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"} : dispatch 2024-09-11T00:13:17.314 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:13:16 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"} : dispatch 2024-09-11T00:13:17.314 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:13:16 smithi012 ceph-mon[30333]: Adjusting osd_memory_target on smithi165 to 176.0M 2024-09-11T00:13:17.314 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:13:16 smithi012 ceph-mon[30333]: Adjusting osd_memory_target on smithi178 to 176.0M 2024-09-11T00:13:17.314 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:13:16 smithi012 ceph-mon[30333]: Unable to set osd_memory_target on smithi165 to 184589619: error parsing value: Value '184589619' is below minimum 939524096 2024-09-11T00:13:17.314 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:13:16 smithi012 ceph-mon[30333]: Unable to set osd_memory_target on smithi178 to 184560230: error parsing value: Value '184560230' is below minimum 939524096 2024-09-11T00:13:17.314 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:13:16 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-11T00:13:17.314 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:13:16 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-11T00:13:17.314 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:13:16 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-09-11T00:13:17.314 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:13:16 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"} : dispatch 2024-09-11T00:13:17.314 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:13:16 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:13:17.314 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:13:16 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T00:13:17.314 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:13:16 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:13:17.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:13:16 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-11T00:13:17.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:13:16 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-11T00:13:17.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:13:16 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-11T00:13:17.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:13:16 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-11T00:13:17.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:13:16 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-09-11T00:13:17.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:13:16 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"} : dispatch 2024-09-11T00:13:17.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:13:16 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"} : dispatch 2024-09-11T00:13:17.332 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:13:16 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"} : dispatch 2024-09-11T00:13:17.332 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:13:16 smithi178 ceph-mon[35919]: Adjusting osd_memory_target on smithi165 to 176.0M 2024-09-11T00:13:17.332 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:13:16 smithi178 ceph-mon[35919]: Adjusting osd_memory_target on smithi178 to 176.0M 2024-09-11T00:13:17.332 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:13:16 smithi178 ceph-mon[35919]: Unable to set osd_memory_target on smithi165 to 184589619: error parsing value: Value '184589619' is below minimum 939524096 2024-09-11T00:13:17.332 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:13:16 smithi178 ceph-mon[35919]: Unable to set osd_memory_target on smithi178 to 184560230: error parsing value: Value '184560230' is below minimum 939524096 2024-09-11T00:13:17.332 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:13:16 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-11T00:13:17.332 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:13:16 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-11T00:13:17.332 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:13:16 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-09-11T00:13:17.332 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:13:16 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"} : dispatch 2024-09-11T00:13:17.332 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:13:16 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:13:17.332 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:13:16 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T00:13:17.332 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:13:16 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:13:17.639 INFO:tasks.workunit.client.0.smithi012.stdout: 1 201235 2.90 MB/sec execute 194 sec latency 10.282 ms 2024-09-11T00:13:18.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:13:17 smithi165 ceph-mon[35356]: pgmap v585: 129 pgs: 129 active+clean; 54 MiB data, 940 MiB used, 1.0 TiB / 1.0 TiB avail; 32 MiB/s rd, 1.0 MiB/s wr, 395 op/s 2024-09-11T00:13:18.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:13:17 smithi165 ceph-mon[35356]: Adjusting osd_memory_target on smithi012 to 177.5M 2024-09-11T00:13:18.283 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:13:17 smithi165 ceph-mon[35356]: Unable to set osd_memory_target on smithi012 to 186147225: error parsing value: Value '186147225' is below minimum 939524096 2024-09-11T00:13:18.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:13:17 smithi012 ceph-mon[30333]: pgmap v585: 129 pgs: 129 active+clean; 54 MiB data, 940 MiB used, 1.0 TiB / 1.0 TiB avail; 32 MiB/s rd, 1.0 MiB/s wr, 395 op/s 2024-09-11T00:13:18.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:13:17 smithi012 ceph-mon[30333]: Adjusting osd_memory_target on smithi012 to 177.5M 2024-09-11T00:13:18.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:13:17 smithi012 ceph-mon[30333]: Unable to set osd_memory_target on smithi012 to 186147225: error parsing value: Value '186147225' is below minimum 939524096 2024-09-11T00:13:18.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:13:17 smithi178 ceph-mon[35919]: pgmap v585: 129 pgs: 129 active+clean; 54 MiB data, 940 MiB used, 1.0 TiB / 1.0 TiB avail; 32 MiB/s rd, 1.0 MiB/s wr, 395 op/s 2024-09-11T00:13:18.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:13:17 smithi178 ceph-mon[35919]: Adjusting osd_memory_target on smithi012 to 177.5M 2024-09-11T00:13:18.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:13:17 smithi178 ceph-mon[35919]: Unable to set osd_memory_target on smithi012 to 186147225: error parsing value: Value '186147225' is below minimum 939524096 2024-09-11T00:13:18.639 INFO:tasks.workunit.client.0.smithi012.stdout: 1 202428 2.94 MB/sec execute 195 sec latency 238.440 ms 2024-09-11T00:13:19.639 INFO:tasks.workunit.client.0.smithi012.stdout: 1 203745 2.95 MB/sec execute 196 sec latency 19.693 ms 2024-09-11T00:13:20.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:13:19 smithi165 ceph-mon[35356]: pgmap v586: 129 pgs: 129 active+clean; 55 MiB data, 940 MiB used, 1.0 TiB / 1.0 TiB avail; 28 MiB/s rd, 874 KiB/s wr, 336 op/s 2024-09-11T00:13:20.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:13:19 smithi012 ceph-mon[30333]: pgmap v586: 129 pgs: 129 active+clean; 55 MiB data, 940 MiB used, 1.0 TiB / 1.0 TiB avail; 28 MiB/s rd, 874 KiB/s wr, 336 op/s 2024-09-11T00:13:20.330 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:13:19 smithi178 ceph-mon[35919]: pgmap v586: 129 pgs: 129 active+clean; 55 MiB data, 940 MiB used, 1.0 TiB / 1.0 TiB avail; 28 MiB/s rd, 874 KiB/s wr, 336 op/s 2024-09-11T00:13:20.639 INFO:tasks.workunit.client.0.smithi012.stdout: 1 205687 3.00 MB/sec execute 197 sec latency 286.569 ms 2024-09-11T00:13:21.640 INFO:tasks.workunit.client.0.smithi012.stdout: 1 205863 2.98 MB/sec execute 198 sec latency 747.815 ms 2024-09-11T00:13:22.193 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:22.191+0000 7fcb66200640 1 -- 172.21.15.12:0/1466140065 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcb601066a0 msgr2=0x7fcb6010d020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:13:22.193 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:22.191+0000 7fcb66200640 1 --2- 172.21.15.12:0/1466140065 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcb601066a0 0x7fcb6010d020 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fcb58009cd0 tx=0x7fcb5802a8e0 comp rx=0 tx=0).stop 2024-09-11T00:13:22.193 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:22.191+0000 7fcb66200640 1 -- 172.21.15.12:0/1466140065 shutdown_connections 2024-09-11T00:13:22.193 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:22.191+0000 7fcb66200640 1 --2- 172.21.15.12:0/1466140065 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcb601066a0 0x7fcb6010d020 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:13:22.193 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:22.191+0000 7fcb66200640 1 --2- 172.21.15.12:0/1466140065 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcb60105ce0 0x7fcb60106160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:13:22.193 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:22.191+0000 7fcb66200640 1 --2- 172.21.15.12:0/1466140065 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcb60104b90 0x7fcb60104f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:13:22.193 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:22.191+0000 7fcb66200640 1 -- 172.21.15.12:0/1466140065 >> 172.21.15.12:0/1466140065 conn(0x7fcb60100470 msgr2=0x7fcb60102890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:13:22.193 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:22.191+0000 7fcb66200640 1 -- 172.21.15.12:0/1466140065 shutdown_connections 2024-09-11T00:13:22.193 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:22.191+0000 7fcb66200640 1 -- 172.21.15.12:0/1466140065 wait complete. 2024-09-11T00:13:22.195 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:22.192+0000 7fcb66200640 1 Processor -- start 2024-09-11T00:13:22.195 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:22.192+0000 7fcb66200640 1 -- start start 2024-09-11T00:13:22.195 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:22.192+0000 7fcb66200640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcb60104b90 0x7fcb6019af20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:13:22.196 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:22.192+0000 7fcb66200640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcb60105ce0 0x7fcb6019b570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:13:22.196 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:22.192+0000 7fcb66200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcb601066a0 0x7fcb6019c1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:13:22.196 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:22.193+0000 7fcb66200640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fcb60110800 con 0x7fcb60104b90 2024-09-11T00:13:22.196 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:22.193+0000 7fcb66200640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fcb60110680 con 0x7fcb601066a0 2024-09-11T00:13:22.196 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:22.193+0000 7fcb66200640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fcb60110980 con 0x7fcb60105ce0 2024-09-11T00:13:22.196 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:22.193+0000 7fcb5ea00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcb60105ce0 0x7fcb6019b570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:13:22.196 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:22.193+0000 7fcb5f400640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcb60104b90 0x7fcb6019af20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:13:22.196 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:22.193+0000 7fcb5ea00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcb60105ce0 0x7fcb6019b570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.12:54288/0 (socket says 172.21.15.12:54288) 2024-09-11T00:13:22.196 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:22.193+0000 7fcb5ea00640 1 -- 172.21.15.12:0/2920529558 learned_addr learned my addr 172.21.15.12:0/2920529558 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:13:22.196 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:22.193+0000 7fcb5fe00640 1 --2- 172.21.15.12:0/2920529558 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcb601066a0 0x7fcb6019c1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:13:22.196 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:22.193+0000 7fcb5ea00640 1 -- 172.21.15.12:0/2920529558 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcb601066a0 msgr2=0x7fcb6019c1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:13:22.196 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:22.193+0000 7fcb5ea00640 1 --2- 172.21.15.12:0/2920529558 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcb601066a0 0x7fcb6019c1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:13:22.197 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:22.193+0000 7fcb5ea00640 1 -- 172.21.15.12:0/2920529558 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcb60104b90 msgr2=0x7fcb6019af20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:13:22.197 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:22.193+0000 7fcb5ea00640 1 --2- 172.21.15.12:0/2920529558 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcb60104b90 0x7fcb6019af20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:13:22.197 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:22.193+0000 7fcb5ea00640 1 -- 172.21.15.12:0/2920529558 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb6019ca60 con 0x7fcb60105ce0 2024-09-11T00:13:22.197 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:22.193+0000 7fcb5ea00640 1 --2- 172.21.15.12:0/2920529558 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcb60105ce0 0x7fcb6019b570 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fcb4c00d950 tx=0x7fcb4c00de10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:13:22.197 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:22.194+0000 7fcb57e00640 1 -- 172.21.15.12:0/2920529558 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb4c017030 con 0x7fcb60105ce0 2024-09-11T00:13:22.197 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:22.194+0000 7fcb66200640 1 -- 172.21.15.12:0/2920529558 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcb6019cd50 con 0x7fcb60105ce0 2024-09-11T00:13:22.197 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:22.194+0000 7fcb57e00640 1 -- 172.21.15.12:0/2920529558 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcb4c004020 con 0x7fcb60105ce0 2024-09-11T00:13:22.197 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:22.194+0000 7fcb57e00640 1 -- 172.21.15.12:0/2920529558 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb4c0047a0 con 0x7fcb60105ce0 2024-09-11T00:13:22.197 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:22.194+0000 7fcb66200640 1 -- 172.21.15.12:0/2920529558 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcb601a3420 con 0x7fcb60105ce0 2024-09-11T00:13:22.198 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:22.197+0000 7fcb69247740 1 Processor -- start 2024-09-11T00:13:22.198 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:22.197+0000 7fcb69247740 1 -- start start 2024-09-11T00:13:22.199 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:22.197+0000 7fcb69247740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d3192a9790 0x55d3192abb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:13:22.199 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:22.197+0000 7fcb69247740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d3192ac0c0 0x55d3192ae4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:13:22.199 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:22.197+0000 7fcb69247740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d3192aeb00 0x55d3192b0fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:13:22.199 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:22.197+0000 7fcb69247740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55d31921c6b0 con 0x55d3192a9790 2024-09-11T00:13:22.199 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:22.197+0000 7fcb69247740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55d31920bae0 con 0x55d3192ac0c0 2024-09-11T00:13:22.199 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:22.197+0000 7fcb69247740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55d31920f8f0 con 0x55d3192aeb00 2024-09-11T00:13:22.199 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:22.197+0000 7fcb5fe00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d3192a9790 0x55d3192abb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:13:22.199 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:22.198+0000 7fcb5f400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d3192ac0c0 0x55d3192ae4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:13:22.199 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:22.198+0000 7fcb5f400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d3192ac0c0 0x55d3192ae4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.12:59598/0 (socket says 172.21.15.12:59598) 2024-09-11T00:13:22.199 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:22.198+0000 7fcb5f400640 1 -- 172.21.15.12:0/1740148509 learned_addr learned my addr 172.21.15.12:0/1740148509 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:13:22.201 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:22.198+0000 7fcb5fe00640 1 -- 172.21.15.12:0/1740148509 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d3192aeb00 msgr2=0x55d3192b0fc0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-11T00:13:22.201 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:22.198+0000 7fcb5fe00640 1 --2- 172.21.15.12:0/1740148509 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d3192aeb00 0x55d3192b0fc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:13:22.201 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:22.198+0000 7fcb5fe00640 1 -- 172.21.15.12:0/1740148509 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d3192ac0c0 msgr2=0x55d3192ae4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:13:22.201 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:22.198+0000 7fcb5fe00640 1 --2- 172.21.15.12:0/1740148509 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d3192ac0c0 0x55d3192ae4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:13:22.201 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:22.198+0000 7fcb5fe00640 1 -- 172.21.15.12:0/1740148509 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d3191d8780 con 0x55d3192a9790 2024-09-11T00:13:22.202 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:22.198+0000 7fcb5f400640 1 --2- 172.21.15.12:0/1740148509 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d3192ac0c0 0x55d3192ae4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:13:22.202 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:22.198+0000 7fcb57e00640 1 -- 172.21.15.12:0/2920529558 <== mon.1 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7fcb4c009a30 con 0x7fcb60105ce0 2024-09-11T00:13:22.202 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:22.198+0000 7fcb5fe00640 1 --2- 172.21.15.12:0/1740148509 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d3192a9790 0x55d3192abb80 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fcb58009ca0 tx=0x7fcb58009a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bbc735b6b814f9b4 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:13:22.202 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:22.199+0000 7fcb4be00640 1 -- 172.21.15.12:0/1740148509 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb580314f0 con 0x55d3192a9790 2024-09-11T00:13:22.202 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:22.199+0000 7fcb4be00640 1 -- 172.21.15.12:0/1740148509 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcb58036040 con 0x55d3192a9790 2024-09-11T00:13:22.202 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:22.199+0000 7fcb4be00640 1 -- 172.21.15.12:0/1740148509 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb580042a0 con 0x55d3192a9790 2024-09-11T00:13:22.202 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:22.199+0000 7fcb69247740 1 -- 172.21.15.12:0/1740148509 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d3192a9790 msgr2=0x55d3192abb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:13:22.202 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:22.199+0000 7fcb69247740 1 --2- 172.21.15.12:0/1740148509 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d3192a9790 0x55d3192abb80 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fcb58009ca0 tx=0x7fcb58009a20 comp rx=0 tx=0).stop 2024-09-11T00:13:22.202 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:22.199+0000 7fcb57e00640 1 --2- 172.21.15.12:0/2920529558 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fcb440b52e0 0x7fcb440b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:13:22.202 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:22.199+0000 7fcb69247740 1 -- 172.21.15.12:0/1740148509 shutdown_connections 2024-09-11T00:13:22.203 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:22.199+0000 7fcb69247740 1 --2- 172.21.15.12:0/1740148509 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d3192aeb00 0x55d3192b0fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:13:22.203 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:22.199+0000 7fcb69247740 1 --2- 172.21.15.12:0/1740148509 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d3192ac0c0 0x55d3192ae4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:13:22.203 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:22.199+0000 7fcb69247740 1 --2- 172.21.15.12:0/1740148509 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d3192a9790 0x55d3192abb80 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:13:22.203 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:22.199+0000 7fcb69247740 1 -- 172.21.15.12:0/1740148509 >> 172.21.15.12:0/1740148509 conn(0x55d3192a48e0 msgr2=0x55d3192a5670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:13:22.203 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:22.200+0000 7fcb57e00640 1 -- 172.21.15.12:0/2920529558 <== mon.1 v2:172.21.15.178:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9208+0+0 (secure 0 0 0) 0x7fcb4c0e5980 con 0x7fcb60105ce0 2024-09-11T00:13:22.203 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:22.200+0000 7fcb69247740 1 -- 172.21.15.12:0/1740148509 shutdown_connections 2024-09-11T00:13:22.203 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:22.200+0000 7fcb69247740 1 -- 172.21.15.12:0/1740148509 wait complete. 2024-09-11T00:13:22.203 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:22.200+0000 7fcb5fe00640 1 --2- 172.21.15.12:0/2920529558 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fcb440b52e0 0x7fcb440b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:13:22.203 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:22.200+0000 7fcb69247740 1 -- start start 2024-09-11T00:13:22.206 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:22.201+0000 7fcb5fe00640 1 --2- 172.21.15.12:0/2920529558 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fcb440b52e0 0x7fcb440b77a0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fcb58009960 tx=0x7fcb5802a8e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:13:22.206 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:22.201+0000 7fcb69247740 1 Processor -- start 2024-09-11T00:13:22.206 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:22.201+0000 7fcb69247740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d3192a9790 0x55d3192b2e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:13:22.207 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:22.201+0000 7fcb69247740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d3192ac0c0 0x55d3192b3500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:13:22.207 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:22.201+0000 7fcb69247740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d3192aeb00 0x55d3192b40d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:13:22.207 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:22.201+0000 7fcb69247740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55d319143ab0 con 0x55d3192a9790 2024-09-11T00:13:22.207 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:22.201+0000 7fcb69247740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55d3191b2570 con 0x55d3192aeb00 2024-09-11T00:13:22.207 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:22.201+0000 7fcb69247740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55d3192b1a20 con 0x55d3192ac0c0 2024-09-11T00:13:22.207 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:22.201+0000 7fcb69247740 10 client.0 _ll_register_callbacks cb 0x55d3191c1100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:13:22.207 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:22.201+0000 7fcb5fe00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d3192aeb00 0x55d3192b40d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:13:22.207 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:22.201+0000 7fcb5ea00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d3192ac0c0 0x55d3192b3500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:13:22.207 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:22.201+0000 7fcb5f400640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d3192a9790 0x55d3192b2e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:13:22.207 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:22.201+0000 7fcb5fe00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d3192aeb00 0x55d3192b40d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.12:59600/0 (socket says 172.21.15.12:59600) 2024-09-11T00:13:22.207 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:22.201+0000 7fcb5fe00640 1 -- 172.21.15.12:0/3217989510 learned_addr learned my addr 172.21.15.12:0/3217989510 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:13:22.207 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:22.202+0000 7fcb5ea00640 1 -- 172.21.15.12:0/3217989510 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d3192aeb00 msgr2=0x55d3192b40d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:13:22.207 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:22.202+0000 7fcb5ea00640 1 --2- 172.21.15.12:0/3217989510 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d3192aeb00 0x55d3192b40d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:13:22.208 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:22.202+0000 7fcb5ea00640 1 -- 172.21.15.12:0/3217989510 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d3192a9790 msgr2=0x55d3192b2e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:13:22.208 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:22.202+0000 7fcb5ea00640 1 --2- 172.21.15.12:0/3217989510 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d3192a9790 0x55d3192b2e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:13:22.208 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:22.202+0000 7fcb5ea00640 1 -- 172.21.15.12:0/3217989510 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d318f6d790 con 0x55d3192ac0c0 2024-09-11T00:13:22.208 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:22.202+0000 7fcb5ea00640 1 --2- 172.21.15.12:0/3217989510 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d3192ac0c0 0x55d3192b3500 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fcb4c0a82e0 tx=0x7fcb4c0ab820 comp rx=0 tx=0).ready entity=mon.1 client_cookie=48346612af624982 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:13:22.208 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:22.202+0000 7fcb48c00640 10 client.0 ms_handle_connect on v2:172.21.15.178:3300/0 2024-09-11T00:13:22.208 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:22.202+0000 7fcb48c00640 1 -- 172.21.15.12:0/3217989510 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb4c0b07f0 con 0x55d3192ac0c0 2024-09-11T00:13:22.208 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:22.202+0000 7fcb48c00640 1 -- 172.21.15.12:0/3217989510 <== mon.1 v2:172.21.15.178:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9208+0+0 (secure 0 0 0) 0x7fcb4c0041d0 con 0x55d3192ac0c0 2024-09-11T00:13:22.208 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:22.202+0000 7fcb69247740 1 -- 172.21.15.12:0/3217989510 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcb58036040 con 0x55d3192ac0c0 2024-09-11T00:13:22.208 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:22.202+0000 7fcb48c00640 1 -- 172.21.15.12:0/3217989510 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fcb4c0b52a0 con 0x55d3192ac0c0 2024-09-11T00:13:22.208 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:22.202+0000 7fcb69247740 10 client.26221 fetch_fsmap learned FSMap version 39 2024-09-11T00:13:22.208 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:22.202+0000 7fcb69247740 1 -- 172.21.15.12:0/3217989510 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55d318fe6620 con 0x55d3192ac0c0 2024-09-11T00:13:22.208 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:22.203+0000 7fcb48c00640 1 -- 172.21.15.12:0/3217989510 <== mon.1 v2:172.21.15.178:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcb58036040 con 0x55d3192ac0c0 2024-09-11T00:13:22.208 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:22.203+0000 7fcb48c00640 1 -- 172.21.15.12:0/3217989510 <== mon.1 v2:172.21.15.178:3300/0 5 ==== fsmap(e 39) ==== 8378+0+0 (secure 0 0 0) 0x7fcb4c0aaf70 con 0x55d3192ac0c0 2024-09-11T00:13:22.209 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:22.203+0000 7fcb69247740 10 client.26221 fetch_fsmap finished waiting for FSMap version 39 2024-09-11T00:13:22.209 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:22.203+0000 7fcb69247740 10 client.26221 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T00:13:22.209 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:22.203+0000 7fcb69247740 20 client.26221 populate_metadata read hostname 'smithi012' 2024-09-11T00:13:22.209 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:22.203+0000 7fcb69247740 1 --2- 172.21.15.12:0/3217989510 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55d3192be790 0x55d3192c0b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:13:22.209 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:22.203+0000 7fcb69247740 4 client.26221 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T00:13:22.209 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:22.203+0000 7fcb69247740 1 -- 172.21.15.12:0/3217989510 --> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d31921e600 con 0x55d3192be790 2024-09-11T00:13:22.209 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:22.203+0000 7fcb5f400640 1 --2- 172.21.15.12:0/3217989510 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55d3192be790 0x55d3192c0b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:13:22.209 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:22.204+0000 7fcb5f400640 1 --2- 172.21.15.12:0/3217989510 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55d3192be790 0x55d3192c0b80 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=44aa30c37027700b server_cookie=e87475e2ca00bb8c in_seq=0 out_seq=0 2024-09-11T00:13:22.209 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:22.204+0000 7fcb48c00640 10 client.26221 ms_handle_connect on v2:172.21.15.165:6838/501312984 2024-09-11T00:13:22.209 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:22.204+0000 7fcb48c00640 1 -- 172.21.15.12:0/3217989510 <== mds.0 v2:172.21.15.165:6838/501312984 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55d31921e600 con 0x55d3192be790 2024-09-11T00:13:22.209 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:22.204+0000 7fcb48c00640 10 client.26221 handle_command_reply: tid=0 2024-09-11T00:13:22.209 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:22.205+0000 7fcb69247740 1 client.26221 shutdown 2024-09-11T00:13:22.209 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:22.205+0000 7fcb69247740 1 -- 172.21.15.12:0/3217989510 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d3192ac0c0 msgr2=0x55d3192b3500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:13:22.210 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:22.205+0000 7fcb69247740 1 --2- 172.21.15.12:0/3217989510 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d3192ac0c0 0x55d3192b3500 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fcb4c0a82e0 tx=0x7fcb4c0ab820 comp rx=0 tx=0).stop 2024-09-11T00:13:22.210 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:22.205+0000 7fcb69247740 1 -- 172.21.15.12:0/3217989510 shutdown_connections 2024-09-11T00:13:22.210 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:22.205+0000 7fcb69247740 1 --2- 172.21.15.12:0/3217989510 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55d3192be790 0x55d3192c0b80 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:13:22.210 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:22.205+0000 7fcb69247740 1 --2- 172.21.15.12:0/3217989510 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d3192aeb00 0x55d3192b40d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:13:22.210 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:22.205+0000 7fcb69247740 1 --2- 172.21.15.12:0/3217989510 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d3192ac0c0 0x55d3192b3500 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:13:22.210 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:22.205+0000 7fcb48c00640 0 client.26221 ms_handle_reset on v2:172.21.15.165:6838/501312984 2024-09-11T00:13:22.210 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:22.205+0000 7fcb69247740 1 --2- 172.21.15.12:0/3217989510 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d3192a9790 0x55d3192b2e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:13:22.210 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:22.205+0000 7fcb69247740 1 -- 172.21.15.12:0/3217989510 >> 172.21.15.12:0/3217989510 conn(0x55d3192a48e0 msgr2=0x55d3192a3a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:13:22.210 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:22.205+0000 7fcb69247740 1 -- 172.21.15.12:0/3217989510 shutdown_connections 2024-09-11T00:13:22.210 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:22.205+0000 7fcb69247740 1 -- 172.21.15.12:0/3217989510 wait complete. 2024-09-11T00:13:22.210 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:22.205+0000 7fcb69247740 20 client.26221 trim_cache size 0 max 16384 2024-09-11T00:13:22.223 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:22.221+0000 7fcb69247740 1 Processor -- start 2024-09-11T00:13:22.223 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:22.221+0000 7fcb69247740 1 -- start start 2024-09-11T00:13:22.223 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:22.222+0000 7fcb69247740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d3192a9790 0x55d3192a6710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:13:22.223 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:22.222+0000 7fcb69247740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d3192ac0c0 0x55d3192a6c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:13:22.223 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:22.222+0000 7fcb69247740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d3192aeb00 0x55d3192a7190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:13:22.224 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:22.222+0000 7fcb69247740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fcb601ad7d0 con 0x55d3192aeb00 2024-09-11T00:13:22.224 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:22.222+0000 7fcb69247740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fcb601af370 con 0x55d3192ac0c0 2024-09-11T00:13:22.224 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:22.222+0000 7fcb69247740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fcb601adea0 con 0x55d3192a9790 2024-09-11T00:13:22.224 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:22.222+0000 7fcb5f400640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d3192a9790 0x55d3192a6710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:13:22.224 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:22.222+0000 7fcb5f400640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d3192a9790 0x55d3192a6710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.12:54314/0 (socket says 172.21.15.12:54314) 2024-09-11T00:13:22.224 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:22.222+0000 7fcb5ea00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d3192ac0c0 0x55d3192a6c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:13:22.224 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:22.222+0000 7fcb5f400640 1 -- 172.21.15.12:0/2603026252 learned_addr learned my addr 172.21.15.12:0/2603026252 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:13:22.224 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:22.222+0000 7fcb5fe00640 1 --2- 172.21.15.12:0/2603026252 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d3192aeb00 0x55d3192a7190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:13:22.225 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:22.222+0000 7fcb5f400640 1 -- 172.21.15.12:0/2603026252 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d3192ac0c0 msgr2=0x55d3192a6c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:13:22.225 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:22.222+0000 7fcb5f400640 1 --2- 172.21.15.12:0/2603026252 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d3192ac0c0 0x55d3192a6c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:13:22.226 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:22.222+0000 7fcb5f400640 1 -- 172.21.15.12:0/2603026252 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d3192aeb00 msgr2=0x55d3192a7190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:13:22.226 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:22.222+0000 7fcb5f400640 1 --2- 172.21.15.12:0/2603026252 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d3192aeb00 0x55d3192a7190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:13:22.226 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:22.222+0000 7fcb5f400640 1 -- 172.21.15.12:0/2603026252 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d3192abef0 con 0x55d3192a9790 2024-09-11T00:13:22.226 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:22.222+0000 7fcb5fe00640 1 --2- 172.21.15.12:0/2603026252 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d3192aeb00 0x55d3192a7190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T00:13:22.226 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:22.223+0000 7fcb5f400640 1 --2- 172.21.15.12:0/2603026252 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d3192a9790 0x55d3192a6710 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x55d3192c1380 tx=0x7fcb50005f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c4dc0c0b01c87699 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:13:22.226 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:22.223+0000 7fcb4b400640 1 -- 172.21.15.12:0/2603026252 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb50005220 con 0x55d3192a9790 2024-09-11T00:13:22.226 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:22.223+0000 7fcb4b400640 1 -- 172.21.15.12:0/2603026252 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcb50004c50 con 0x55d3192a9790 2024-09-11T00:13:22.226 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:22.223+0000 7fcb4b400640 1 -- 172.21.15.12:0/2603026252 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb500258b0 con 0x55d3192a9790 2024-09-11T00:13:22.226 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:22.224+0000 7fcb69247740 1 -- 172.21.15.12:0/2603026252 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d3192a9790 msgr2=0x55d3192a6710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:13:22.226 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:22.224+0000 7fcb69247740 1 --2- 172.21.15.12:0/2603026252 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d3192a9790 0x55d3192a6710 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x55d3192c1380 tx=0x7fcb50005f90 comp rx=0 tx=0).stop 2024-09-11T00:13:22.226 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:22.224+0000 7fcb69247740 1 -- 172.21.15.12:0/2603026252 shutdown_connections 2024-09-11T00:13:22.226 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:22.224+0000 7fcb69247740 1 --2- 172.21.15.12:0/2603026252 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d3192aeb00 0x55d3192a7190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:13:22.226 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:22.224+0000 7fcb69247740 1 --2- 172.21.15.12:0/2603026252 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d3192ac0c0 0x55d3192a6c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:13:22.227 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:22.224+0000 7fcb69247740 1 --2- 172.21.15.12:0/2603026252 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d3192a9790 0x55d3192a6710 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:13:22.227 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:22.224+0000 7fcb69247740 1 -- 172.21.15.12:0/2603026252 >> 172.21.15.12:0/2603026252 conn(0x55d3192a48e0 msgr2=0x55d3192aef70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:13:22.227 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:22.224+0000 7fcb69247740 1 -- 172.21.15.12:0/2603026252 shutdown_connections 2024-09-11T00:13:22.227 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:22.224+0000 7fcb69247740 1 -- 172.21.15.12:0/2603026252 wait complete. 2024-09-11T00:13:22.227 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:22.225+0000 7fcb69247740 1 -- start start 2024-09-11T00:13:22.229 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:22.225+0000 7fcb69247740 1 Processor -- start 2024-09-11T00:13:22.229 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:22.225+0000 7fcb69247740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d3192a9790 0x55d3192b8f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:13:22.230 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:22.225+0000 7fcb69247740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d3192ac0c0 0x55d3192b95a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:13:22.230 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:22.225+0000 7fcb69247740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d3192aeb00 0x55d3192ba170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:13:22.230 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:22.225+0000 7fcb69247740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fcb601aeca0 con 0x55d3192ac0c0 2024-09-11T00:13:22.230 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:22.225+0000 7fcb69247740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fcb601ae5d0 con 0x55d3192a9790 2024-09-11T00:13:22.230 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:22.225+0000 7fcb69247740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fcb601ab7b0 con 0x55d3192aeb00 2024-09-11T00:13:22.230 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:22.225+0000 7fcb69247740 10 client.0 _ll_register_callbacks cb 0x7fcb4c0ab660 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:13:22.231 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:22.226+0000 7fcb5f400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d3192a9790 0x55d3192b8f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:13:22.231 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:22.226+0000 7fcb5f400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d3192a9790 0x55d3192b8f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.12:59612/0 (socket says 172.21.15.12:59612) 2024-09-11T00:13:22.231 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:22.226+0000 7fcb5f400640 1 -- 172.21.15.12:0/672976524 learned_addr learned my addr 172.21.15.12:0/672976524 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:13:22.231 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:22.226+0000 7fcb5fe00640 1 --2- 172.21.15.12:0/672976524 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d3192aeb00 0x55d3192ba170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:13:22.231 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:22.226+0000 7fcb5ea00640 1 --2- 172.21.15.12:0/672976524 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d3192ac0c0 0x55d3192b95a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:13:22.231 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:22.226+0000 7fcb5f400640 1 -- 172.21.15.12:0/672976524 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d3192aeb00 msgr2=0x55d3192ba170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:13:22.231 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:22.226+0000 7fcb5f400640 1 --2- 172.21.15.12:0/672976524 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d3192aeb00 0x55d3192ba170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:13:22.231 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:22.226+0000 7fcb5f400640 1 -- 172.21.15.12:0/672976524 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d3192ac0c0 msgr2=0x55d3192b95a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:13:22.231 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:22.226+0000 7fcb5f400640 1 --2- 172.21.15.12:0/672976524 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d3192ac0c0 0x55d3192b95a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:13:22.231 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:22.226+0000 7fcb5f400640 1 -- 172.21.15.12:0/672976524 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d3192ba9a0 con 0x55d3192a9790 2024-09-11T00:13:22.231 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:22.226+0000 7fcb5f400640 1 --2- 172.21.15.12:0/672976524 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d3192a9790 0x55d3192b8f20 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fcb50010fa0 tx=0x7fcb50009f20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7eb4a1038e9d615f server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:13:22.231 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:22.226+0000 7fcb48c00640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-09-11T00:13:22.231 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:22.226+0000 7fcb48c00640 1 -- 172.21.15.12:0/672976524 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb50004740 con 0x55d3192a9790 2024-09-11T00:13:22.232 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:22.227+0000 7fcb69247740 1 -- 172.21.15.12:0/672976524 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcb50004c50 con 0x55d3192a9790 2024-09-11T00:13:22.232 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:22.227+0000 7fcb48c00640 1 -- 172.21.15.12:0/672976524 <== mon.2 v2:172.21.15.165:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9208+0+0 (secure 0 0 0) 0x7fcb500048e0 con 0x55d3192a9790 2024-09-11T00:13:22.232 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:22.227+0000 7fcb48c00640 1 -- 172.21.15.12:0/672976524 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fcb50005020 con 0x55d3192a9790 2024-09-11T00:13:22.232 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:22.227+0000 7fcb69247740 10 client.26234 fetch_fsmap learned FSMap version 39 2024-09-11T00:13:22.232 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:22.227+0000 7fcb69247740 1 -- 172.21.15.12:0/672976524 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55d31932c210 con 0x55d3192a9790 2024-09-11T00:13:22.232 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:22.227+0000 7fcb48c00640 1 -- 172.21.15.12:0/672976524 <== mon.2 v2:172.21.15.165:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcb50004c50 con 0x55d3192a9790 2024-09-11T00:13:22.232 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:22.227+0000 7fcb48c00640 1 -- 172.21.15.12:0/672976524 <== mon.2 v2:172.21.15.165:3300/0 5 ==== fsmap(e 39) ==== 8378+0+0 (secure 0 0 0) 0x7fcb5001e420 con 0x55d3192a9790 2024-09-11T00:13:22.232 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:22.227+0000 7fcb69247740 10 client.26234 fetch_fsmap finished waiting for FSMap version 39 2024-09-11T00:13:22.232 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:22.227+0000 7fcb69247740 10 client.26234 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T00:13:22.232 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:22.227+0000 7fcb69247740 20 client.26234 populate_metadata read hostname 'smithi012' 2024-09-11T00:13:22.232 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:22.227+0000 7fcb69247740 1 --2- 172.21.15.12:0/672976524 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55d31932cba0 0x55d31932efc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:13:22.232 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:22.227+0000 7fcb69247740 4 client.26234 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T00:13:22.232 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:22.227+0000 7fcb69247740 1 -- 172.21.15.12:0/672976524 --> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55d31921a0b0 con 0x55d31932cba0 2024-09-11T00:13:22.233 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:22.228+0000 7fcb5ea00640 1 --2- 172.21.15.12:0/672976524 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55d31932cba0 0x55d31932efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:13:22.233 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:22.228+0000 7fcb5ea00640 1 --2- 172.21.15.12:0/672976524 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55d31932cba0 0x55d31932efc0 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=21c1cc3bed88219f server_cookie=c26cba7856dca69b in_seq=0 out_seq=0 2024-09-11T00:13:22.233 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:22.228+0000 7fcb48c00640 10 client.26234 ms_handle_connect on v2:172.21.15.165:6838/501312984 2024-09-11T00:13:22.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:13:21 smithi165 ceph-mon[35356]: pgmap v587: 129 pgs: 129 active+clean; 56 MiB data, 959 MiB used, 1.0 TiB / 1.0 TiB avail; 18 MiB/s rd, 1006 KiB/s wr, 306 op/s 2024-09-11T00:13:22.286 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:22.285+0000 7fcb48c00640 1 -- 172.21.15.12:0/672976524 <== mds.0 v2:172.21.15.165:6838/501312984 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55d31921a0b0 con 0x55d31932cba0 2024-09-11T00:13:22.286 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:22.285+0000 7fcb48c00640 10 client.26234 handle_command_reply: tid=0 2024-09-11T00:13:22.286 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:22.285+0000 7fcb69247740 1 client.26234 shutdown 2024-09-11T00:13:22.287 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:22.286+0000 7fcb69247740 1 -- 172.21.15.12:0/672976524 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d3192a9790 msgr2=0x55d3192b8f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:13:22.287 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:22.286+0000 7fcb69247740 1 --2- 172.21.15.12:0/672976524 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d3192a9790 0x55d3192b8f20 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fcb50010fa0 tx=0x7fcb50009f20 comp rx=0 tx=0).stop 2024-09-11T00:13:22.287 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:22.286+0000 7fcb69247740 1 -- 172.21.15.12:0/672976524 shutdown_connections 2024-09-11T00:13:22.287 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:22.286+0000 7fcb69247740 1 --2- 172.21.15.12:0/672976524 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55d31932cba0 0x55d31932efc0 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:13:22.288 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:22.286+0000 7fcb69247740 1 --2- 172.21.15.12:0/672976524 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d3192aeb00 0x55d3192ba170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:13:22.288 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:22.286+0000 7fcb69247740 1 --2- 172.21.15.12:0/672976524 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d3192ac0c0 0x55d3192b95a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:13:22.288 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:22.286+0000 7fcb69247740 1 --2- 172.21.15.12:0/672976524 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d3192a9790 0x55d3192b8f20 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:13:22.288 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:22.286+0000 7fcb69247740 1 -- 172.21.15.12:0/672976524 >> 172.21.15.12:0/672976524 conn(0x55d3192a48e0 msgr2=0x55d3192a3240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:13:22.288 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:22.286+0000 7fcb48c00640 0 client.26234 ms_handle_reset on v2:172.21.15.165:6838/501312984 2024-09-11T00:13:22.288 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:22.286+0000 7fcb69247740 1 -- 172.21.15.12:0/672976524 shutdown_connections 2024-09-11T00:13:22.288 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:22.286+0000 7fcb69247740 1 -- 172.21.15.12:0/672976524 wait complete. 2024-09-11T00:13:22.288 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:22.286+0000 7fcb69247740 20 client.26234 trim_cache size 0 max 16384 2024-09-11T00:13:22.289 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:{ 2024-09-11T00:13:22.289 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "message": "", 2024-09-11T00:13:22.289 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "return_code": 0 2024-09-11T00:13:22.289 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:} 2024-09-11T00:13:22.289 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:22.287+0000 7fcb66200640 1 -- 172.21.15.12:0/2920529558 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fcb440b52e0 msgr2=0x7fcb440b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:13:22.290 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:22.287+0000 7fcb66200640 1 --2- 172.21.15.12:0/2920529558 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fcb440b52e0 0x7fcb440b77a0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fcb58009960 tx=0x7fcb5802a8e0 comp rx=0 tx=0).stop 2024-09-11T00:13:22.290 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:22.287+0000 7fcb66200640 1 -- 172.21.15.12:0/2920529558 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcb60105ce0 msgr2=0x7fcb6019b570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:13:22.290 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:22.287+0000 7fcb66200640 1 --2- 172.21.15.12:0/2920529558 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcb60105ce0 0x7fcb6019b570 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fcb4c00d950 tx=0x7fcb4c00de10 comp rx=0 tx=0).stop 2024-09-11T00:13:22.290 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:22.287+0000 7fcb66200640 1 -- 172.21.15.12:0/2920529558 shutdown_connections 2024-09-11T00:13:22.290 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:22.287+0000 7fcb66200640 1 --2- 172.21.15.12:0/2920529558 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fcb440b52e0 0x7fcb440b77a0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:13:22.290 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:22.287+0000 7fcb66200640 1 --2- 172.21.15.12:0/2920529558 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcb601066a0 0x7fcb6019c1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:13:22.290 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:22.287+0000 7fcb66200640 1 --2- 172.21.15.12:0/2920529558 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcb60105ce0 0x7fcb6019b570 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:13:22.290 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:22.287+0000 7fcb66200640 1 --2- 172.21.15.12:0/2920529558 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcb60104b90 0x7fcb6019af20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:13:22.290 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:22.287+0000 7fcb66200640 1 -- 172.21.15.12:0/2920529558 >> 172.21.15.12:0/2920529558 conn(0x7fcb60100470 msgr2=0x7fcb601090b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:13:22.290 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:22.288+0000 7fcb66200640 1 -- 172.21.15.12:0/2920529558 shutdown_connections 2024-09-11T00:13:22.290 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:22.288+0000 7fcb66200640 1 -- 172.21.15.12:0/2920529558 wait complete. 2024-09-11T00:13:22.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:13:21 smithi012 ceph-mon[30333]: pgmap v587: 129 pgs: 129 active+clean; 56 MiB data, 959 MiB used, 1.0 TiB / 1.0 TiB avail; 18 MiB/s rd, 1006 KiB/s wr, 306 op/s 2024-09-11T00:13:22.330 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:13:21 smithi178 ceph-mon[35919]: pgmap v587: 129 pgs: 129 active+clean; 56 MiB data, 959 MiB used, 1.0 TiB / 1.0 TiB avail; 18 MiB/s rd, 1006 KiB/s wr, 306 op/s 2024-09-11T00:13:22.639 INFO:tasks.workunit.client.0.smithi012.stdout: 1 206865 2.99 MB/sec execute 199 sec latency 77.715 ms 2024-09-11T00:13:23.639 INFO:tasks.workunit.client.0.smithi012.stdout: 1 209250 3.05 MB/sec execute 200 sec latency 9.333 ms 2024-09-11T00:13:24.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:13:23 smithi165 ceph-mon[35356]: pgmap v588: 129 pgs: 129 active+clean; 60 MiB data, 972 MiB used, 1.0 TiB / 1.0 TiB avail; 28 MiB/s rd, 1.2 MiB/s wr, 364 op/s 2024-09-11T00:13:24.312 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:13:23 smithi012 ceph-mon[30333]: pgmap v588: 129 pgs: 129 active+clean; 60 MiB data, 972 MiB used, 1.0 TiB / 1.0 TiB avail; 28 MiB/s rd, 1.2 MiB/s wr, 364 op/s 2024-09-11T00:13:24.330 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:13:23 smithi178 ceph-mon[35919]: pgmap v588: 129 pgs: 129 active+clean; 60 MiB data, 972 MiB used, 1.0 TiB / 1.0 TiB avail; 28 MiB/s rd, 1.2 MiB/s wr, 364 op/s 2024-09-11T00:13:24.639 INFO:tasks.workunit.client.0.smithi012.stdout: 1 210084 3.04 MB/sec execute 201 sec latency 71.920 ms 2024-09-11T00:13:25.639 INFO:tasks.workunit.client.0.smithi012.stdout: 1 212472 3.09 MB/sec execute 202 sec latency 10.777 ms 2024-09-11T00:13:25.945 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:13:25 smithi012 ceph-mon[30333]: pgmap v589: 129 pgs: 129 active+clean; 60 MiB data, 986 MiB used, 1.0 TiB / 1.0 TiB avail; 18 MiB/s rd, 1.1 MiB/s wr, 257 op/s 2024-09-11T00:13:26.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:13:25 smithi165 ceph-mon[35356]: pgmap v589: 129 pgs: 129 active+clean; 60 MiB data, 986 MiB used, 1.0 TiB / 1.0 TiB avail; 18 MiB/s rd, 1.1 MiB/s wr, 257 op/s 2024-09-11T00:13:26.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:13:25 smithi178 ceph-mon[35919]: pgmap v589: 129 pgs: 129 active+clean; 60 MiB data, 986 MiB used, 1.0 TiB / 1.0 TiB avail; 18 MiB/s rd, 1.1 MiB/s wr, 257 op/s 2024-09-11T00:13:26.639 INFO:tasks.workunit.client.0.smithi012.stdout: 1 212545 3.08 MB/sec execute 203 sec latency 858.772 ms 2024-09-11T00:13:27.639 INFO:tasks.workunit.client.0.smithi012.stdout: 1 213138 3.09 MB/sec execute 204 sec latency 1146.091 ms 2024-09-11T00:13:28.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:13:27 smithi165 ceph-mon[35356]: pgmap v590: 129 pgs: 129 active+clean; 58 MiB data, 983 MiB used, 1.0 TiB / 1.0 TiB avail; 22 MiB/s rd, 1.6 MiB/s wr, 375 op/s 2024-09-11T00:13:28.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:13:27 smithi012 ceph-mon[30333]: pgmap v590: 129 pgs: 129 active+clean; 58 MiB data, 983 MiB used, 1.0 TiB / 1.0 TiB avail; 22 MiB/s rd, 1.6 MiB/s wr, 375 op/s 2024-09-11T00:13:28.330 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:13:27 smithi178 ceph-mon[35919]: pgmap v590: 129 pgs: 129 active+clean; 58 MiB data, 983 MiB used, 1.0 TiB / 1.0 TiB avail; 22 MiB/s rd, 1.6 MiB/s wr, 375 op/s 2024-09-11T00:13:28.639 INFO:tasks.workunit.client.0.smithi012.stdout: 1 214753 3.10 MB/sec execute 205 sec latency 13.125 ms 2024-09-11T00:13:29.639 INFO:tasks.workunit.client.0.smithi012.stdout: 1 216389 3.15 MB/sec execute 206 sec latency 325.825 ms 2024-09-11T00:13:30.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:13:29 smithi165 ceph-mon[35356]: pgmap v591: 129 pgs: 129 active+clean; 59 MiB data, 1002 MiB used, 1.0 TiB / 1.0 TiB avail; 22 MiB/s rd, 1.4 MiB/s wr, 323 op/s 2024-09-11T00:13:30.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:13:29 smithi012 ceph-mon[30333]: pgmap v591: 129 pgs: 129 active+clean; 59 MiB data, 1002 MiB used, 1.0 TiB / 1.0 TiB avail; 22 MiB/s rd, 1.4 MiB/s wr, 323 op/s 2024-09-11T00:13:30.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:13:29 smithi178 ceph-mon[35919]: pgmap v591: 129 pgs: 129 active+clean; 59 MiB data, 1002 MiB used, 1.0 TiB / 1.0 TiB avail; 22 MiB/s rd, 1.4 MiB/s wr, 323 op/s 2024-09-11T00:13:30.639 INFO:tasks.workunit.client.0.smithi012.stdout: 1 217460 3.15 MB/sec execute 207 sec latency 26.540 ms 2024-09-11T00:13:31.639 INFO:tasks.workunit.client.0.smithi012.stdout: 1 217685 3.13 MB/sec execute 208 sec latency 795.736 ms 2024-09-11T00:13:32.183 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:13:31 smithi012 ceph-mon[30333]: pgmap v592: 129 pgs: 129 active+clean; 59 MiB data, 1015 MiB used, 1.0 TiB / 1.0 TiB avail; 19 MiB/s rd, 1.9 MiB/s wr, 332 op/s 2024-09-11T00:13:32.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:13:31 smithi165 ceph-mon[35356]: pgmap v592: 129 pgs: 129 active+clean; 59 MiB data, 1015 MiB used, 1.0 TiB / 1.0 TiB avail; 19 MiB/s rd, 1.9 MiB/s wr, 332 op/s 2024-09-11T00:13:32.330 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:13:31 smithi178 ceph-mon[35919]: pgmap v592: 129 pgs: 129 active+clean; 59 MiB data, 1015 MiB used, 1.0 TiB / 1.0 TiB avail; 19 MiB/s rd, 1.9 MiB/s wr, 332 op/s 2024-09-11T00:13:32.639 INFO:tasks.workunit.client.0.smithi012.stdout: 1 218853 3.14 MB/sec execute 209 sec latency 449.882 ms 2024-09-11T00:13:33.640 INFO:tasks.workunit.client.0.smithi012.stdout: 1 220335 3.18 MB/sec execute 210 sec latency 83.774 ms 2024-09-11T00:13:34.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:13:33 smithi165 ceph-mon[35356]: pgmap v593: 129 pgs: 129 active+clean; 60 MiB data, 1014 MiB used, 1.0 TiB / 1.0 TiB avail; 26 MiB/s rd, 2.0 MiB/s wr, 389 op/s 2024-09-11T00:13:34.312 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:13:33 smithi012 ceph-mon[30333]: pgmap v593: 129 pgs: 129 active+clean; 60 MiB data, 1014 MiB used, 1.0 TiB / 1.0 TiB avail; 26 MiB/s rd, 2.0 MiB/s wr, 389 op/s 2024-09-11T00:13:34.330 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:13:33 smithi178 ceph-mon[35919]: pgmap v593: 129 pgs: 129 active+clean; 60 MiB data, 1014 MiB used, 1.0 TiB / 1.0 TiB avail; 26 MiB/s rd, 2.0 MiB/s wr, 389 op/s 2024-09-11T00:13:34.640 INFO:tasks.workunit.client.0.smithi012.stdout: 1 222166 3.20 MB/sec execute 211 sec latency 11.676 ms 2024-09-11T00:13:35.417 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:35.416+0000 7f9b84200640 1 -- 172.21.15.12:0/35623038 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9b7c104ba0 msgr2=0x7f9b7c104fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:13:35.417 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:35.416+0000 7f9b84200640 1 --2- 172.21.15.12:0/35623038 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9b7c104ba0 0x7f9b7c104fa0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f9b70009930 tx=0x7f9b7002a420 comp rx=0 tx=0).stop 2024-09-11T00:13:35.417 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:35.416+0000 7f9b84200640 1 -- 172.21.15.12:0/35623038 shutdown_connections 2024-09-11T00:13:35.417 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:35.416+0000 7f9b84200640 1 --2- 172.21.15.12:0/35623038 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9b7c1066b0 0x7f9b7c10d030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:13:35.418 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:35.416+0000 7f9b84200640 1 --2- 172.21.15.12:0/35623038 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9b7c105cf0 0x7f9b7c106170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:13:35.418 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:35.416+0000 7f9b84200640 1 --2- 172.21.15.12:0/35623038 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9b7c104ba0 0x7f9b7c104fa0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:13:35.418 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:35.416+0000 7f9b84200640 1 -- 172.21.15.12:0/35623038 >> 172.21.15.12:0/35623038 conn(0x7f9b7c100460 msgr2=0x7f9b7c102880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:13:35.418 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:35.416+0000 7f9b84200640 1 -- 172.21.15.12:0/35623038 shutdown_connections 2024-09-11T00:13:35.418 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:35.416+0000 7f9b84200640 1 -- 172.21.15.12:0/35623038 wait complete. 2024-09-11T00:13:35.420 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:35.417+0000 7f9b84200640 1 Processor -- start 2024-09-11T00:13:35.420 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:35.417+0000 7f9b84200640 1 -- start start 2024-09-11T00:13:35.420 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:35.417+0000 7f9b84200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9b7c104ba0 0x7f9b7c19afa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:13:35.420 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:35.417+0000 7f9b84200640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9b7c105cf0 0x7f9b7c19b5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:13:35.420 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:35.417+0000 7f9b84200640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9b7c1066b0 0x7f9b7c19c220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:13:35.420 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:35.417+0000 7f9b84200640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f9b7c110810 con 0x7f9b7c1066b0 2024-09-11T00:13:35.420 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:35.417+0000 7f9b84200640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f9b7c110690 con 0x7f9b7c104ba0 2024-09-11T00:13:35.421 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:35.417+0000 7f9b84200640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f9b7c110990 con 0x7f9b7c105cf0 2024-09-11T00:13:35.421 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:35.418+0000 7f9b81800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9b7c104ba0 0x7f9b7c19afa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:13:35.421 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:35.418+0000 7f9b81800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9b7c104ba0 0x7f9b7c19afa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.12:56438/0 (socket says 172.21.15.12:56438) 2024-09-11T00:13:35.421 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:35.418+0000 7f9b81800640 1 -- 172.21.15.12:0/3423830406 learned_addr learned my addr 172.21.15.12:0/3423830406 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:13:35.421 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:35.418+0000 7f9b80e00640 1 --2- 172.21.15.12:0/3423830406 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9b7c105cf0 0x7f9b7c19b5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:13:35.421 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:35.418+0000 7f9b82200640 1 --2- 172.21.15.12:0/3423830406 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9b7c1066b0 0x7f9b7c19c220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:13:35.421 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:35.418+0000 7f9b81800640 1 -- 172.21.15.12:0/3423830406 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9b7c105cf0 msgr2=0x7f9b7c19b5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:13:35.421 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:35.418+0000 7f9b81800640 1 --2- 172.21.15.12:0/3423830406 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9b7c105cf0 0x7f9b7c19b5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:13:35.421 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:35.418+0000 7f9b81800640 1 -- 172.21.15.12:0/3423830406 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9b7c1066b0 msgr2=0x7f9b7c19c220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:13:35.421 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:35.418+0000 7f9b81800640 1 --2- 172.21.15.12:0/3423830406 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9b7c1066b0 0x7f9b7c19c220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:13:35.421 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:35.418+0000 7f9b81800640 1 -- 172.21.15.12:0/3423830406 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b7c19cae0 con 0x7f9b7c104ba0 2024-09-11T00:13:35.421 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:35.418+0000 7f9b81800640 1 --2- 172.21.15.12:0/3423830406 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9b7c104ba0 0x7f9b7c19afa0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f9b70009a50 tx=0x7f9b700042f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:13:35.422 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:35.419+0000 7f9b7a000640 1 -- 172.21.15.12:0/3423830406 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b7003f030 con 0x7f9b7c104ba0 2024-09-11T00:13:35.422 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:35.419+0000 7f9b7a000640 1 -- 172.21.15.12:0/3423830406 <== mon.2 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9b70031910 con 0x7f9b7c104ba0 2024-09-11T00:13:35.422 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:35.419+0000 7f9b84200640 1 -- 172.21.15.12:0/3423830406 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9b7c19cd70 con 0x7f9b7c104ba0 2024-09-11T00:13:35.422 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:35.419+0000 7f9b7a000640 1 -- 172.21.15.12:0/3423830406 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b70035070 con 0x7f9b7c104ba0 2024-09-11T00:13:35.422 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:35.419+0000 7f9b84200640 1 -- 172.21.15.12:0/3423830406 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9b7c1a34a0 con 0x7f9b7c104ba0 2024-09-11T00:13:35.423 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:35.422+0000 7f9b871eb740 1 Processor -- start 2024-09-11T00:13:35.423 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:35.422+0000 7f9b871eb740 1 -- start start 2024-09-11T00:13:35.425 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:35.422+0000 7f9b871eb740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ce52f87790 0x55ce52f89b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:13:35.425 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:35.422+0000 7f9b871eb740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ce52f8a0c0 0x55ce52f8c4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:13:35.425 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:35.422+0000 7f9b871eb740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ce52f8cb00 0x55ce52f8efc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:13:35.425 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:35.422+0000 7f9b871eb740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55ce52efa6b0 con 0x55ce52f8cb00 2024-09-11T00:13:35.425 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:35.422+0000 7f9b871eb740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55ce52ee9ae0 con 0x55ce52f8a0c0 2024-09-11T00:13:35.425 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:35.422+0000 7f9b871eb740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55ce52eed8f0 con 0x55ce52f87790 2024-09-11T00:13:35.425 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:35.422+0000 7f9b7a000640 1 -- 172.21.15.12:0/3423830406 <== mon.2 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7f9b7003ba60 con 0x7f9b7c104ba0 2024-09-11T00:13:35.425 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:35.422+0000 7f9b82200640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ce52f87790 0x55ce52f89b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:13:35.426 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:35.422+0000 7f9b82200640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ce52f87790 0x55ce52f89b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.12:60204/0 (socket says 172.21.15.12:60204) 2024-09-11T00:13:35.426 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:35.423+0000 7f9b82200640 1 -- 172.21.15.12:0/2167400108 learned_addr learned my addr 172.21.15.12:0/2167400108 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:13:35.426 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:35.423+0000 7f9b80e00640 1 --2- 172.21.15.12:0/2167400108 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ce52f8cb00 0x55ce52f8efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:13:35.426 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:35.423+0000 7f9b81800640 1 --2- 172.21.15.12:0/2167400108 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ce52f8a0c0 0x55ce52f8c4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:13:35.426 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:35.423+0000 7f9b82200640 1 -- 172.21.15.12:0/2167400108 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ce52f8a0c0 msgr2=0x55ce52f8c4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:13:35.427 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:35.423+0000 7f9b82200640 1 --2- 172.21.15.12:0/2167400108 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ce52f8a0c0 0x55ce52f8c4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:13:35.427 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:35.423+0000 7f9b82200640 1 -- 172.21.15.12:0/2167400108 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ce52f8cb00 msgr2=0x55ce52f8efc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:13:35.427 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:35.423+0000 7f9b82200640 1 --2- 172.21.15.12:0/2167400108 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ce52f8cb00 0x55ce52f8efc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:13:35.427 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:35.423+0000 7f9b82200640 1 -- 172.21.15.12:0/2167400108 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ce52eb6780 con 0x55ce52f87790 2024-09-11T00:13:35.427 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:35.423+0000 7f9b81800640 1 --2- 172.21.15.12:0/2167400108 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ce52f8a0c0 0x55ce52f8c4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T00:13:35.427 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:35.423+0000 7f9b82200640 1 --2- 172.21.15.12:0/2167400108 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ce52f87790 0x55ce52f89b80 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f9b74009f90 tx=0x7f9b740118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e7a61f72f820affe server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:13:35.427 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:35.423+0000 7f9b6ea00640 1 -- 172.21.15.12:0/2167400108 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b74004d20 con 0x55ce52f87790 2024-09-11T00:13:35.427 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:35.423+0000 7f9b6ea00640 1 -- 172.21.15.12:0/2167400108 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9b74004050 con 0x55ce52f87790 2024-09-11T00:13:35.427 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:35.423+0000 7f9b6ea00640 1 -- 172.21.15.12:0/2167400108 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b74005280 con 0x55ce52f87790 2024-09-11T00:13:35.432 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:35.424+0000 7f9b871eb740 1 -- 172.21.15.12:0/2167400108 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ce52f87790 msgr2=0x55ce52f89b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:13:35.433 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:35.424+0000 7f9b871eb740 1 --2- 172.21.15.12:0/2167400108 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ce52f87790 0x55ce52f89b80 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f9b74009f90 tx=0x7f9b740118d0 comp rx=0 tx=0).stop 2024-09-11T00:13:35.433 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:35.424+0000 7f9b871eb740 1 -- 172.21.15.12:0/2167400108 shutdown_connections 2024-09-11T00:13:35.433 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:35.424+0000 7f9b871eb740 1 --2- 172.21.15.12:0/2167400108 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ce52f8cb00 0x55ce52f8efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:13:35.433 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:35.424+0000 7f9b871eb740 1 --2- 172.21.15.12:0/2167400108 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ce52f8a0c0 0x55ce52f8c4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:13:35.433 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:35.424+0000 7f9b871eb740 1 --2- 172.21.15.12:0/2167400108 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ce52f87790 0x55ce52f89b80 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:13:35.433 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:35.424+0000 7f9b871eb740 1 -- 172.21.15.12:0/2167400108 >> 172.21.15.12:0/2167400108 conn(0x55ce52f828e0 msgr2=0x55ce52f83670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:13:35.433 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:35.424+0000 7f9b7a000640 1 --2- 172.21.15.12:0/3423830406 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f9b640b5290 0x7f9b640b7750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:13:35.433 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:35.424+0000 7f9b871eb740 1 -- 172.21.15.12:0/2167400108 shutdown_connections 2024-09-11T00:13:35.433 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:35.425+0000 7f9b871eb740 1 -- 172.21.15.12:0/2167400108 wait complete. 2024-09-11T00:13:35.433 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:35.425+0000 7f9b7a000640 1 -- 172.21.15.12:0/3423830406 <== mon.2 v2:172.21.15.165:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9208+0+0 (secure 0 0 0) 0x7f9b70106850 con 0x7f9b7c104ba0 2024-09-11T00:13:35.433 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:35.425+0000 7f9b80e00640 1 --2- 172.21.15.12:0/3423830406 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f9b640b5290 0x7f9b640b7750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:13:35.434 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:35.425+0000 7f9b871eb740 1 -- start start 2024-09-11T00:13:35.434 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:35.425+0000 7f9b871eb740 1 Processor -- start 2024-09-11T00:13:35.434 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:35.425+0000 7f9b80e00640 1 --2- 172.21.15.12:0/3423830406 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f9b640b5290 0x7f9b640b7750 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f9b68009cc0 tx=0x7f9b68005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:13:35.434 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:35.426+0000 7f9b871eb740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ce52f87790 0x55ce52f90e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:13:35.434 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:35.426+0000 7f9b871eb740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ce52f8a0c0 0x55ce52f91530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:13:35.434 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:35.426+0000 7f9b871eb740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ce52f8cb00 0x55ce52f92100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:13:35.434 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:35.426+0000 7f9b871eb740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55ce52e21ab0 con 0x55ce52f8a0c0 2024-09-11T00:13:35.434 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:35.426+0000 7f9b871eb740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55ce52e90570 con 0x55ce52f8cb00 2024-09-11T00:13:35.434 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:35.426+0000 7f9b871eb740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55ce52f8fae0 con 0x55ce52f87790 2024-09-11T00:13:35.434 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:35.426+0000 7f9b871eb740 10 client.0 _ll_register_callbacks cb 0x55ce52e9f100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:13:35.434 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:35.426+0000 7f9b82200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ce52f8cb00 0x55ce52f92100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:13:35.434 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:35.426+0000 7f9b82200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ce52f8cb00 0x55ce52f92100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.12:56466/0 (socket says 172.21.15.12:56466) 2024-09-11T00:13:35.435 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:35.426+0000 7f9b82200640 1 -- 172.21.15.12:0/3525422038 learned_addr learned my addr 172.21.15.12:0/3525422038 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:13:35.435 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:35.426+0000 7f9b80e00640 1 --2- 172.21.15.12:0/3525422038 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ce52f8a0c0 0x55ce52f91530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:13:35.435 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:35.426+0000 7f9b81800640 1 --2- 172.21.15.12:0/3525422038 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ce52f87790 0x55ce52f90e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:13:35.435 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:35.427+0000 7f9b82200640 1 -- 172.21.15.12:0/3525422038 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ce52f87790 msgr2=0x55ce52f90e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:13:35.435 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:35.427+0000 7f9b82200640 1 --2- 172.21.15.12:0/3525422038 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ce52f87790 0x55ce52f90e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:13:35.435 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:35.427+0000 7f9b82200640 1 -- 172.21.15.12:0/3525422038 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ce52f8a0c0 msgr2=0x55ce52f91530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:13:35.435 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:35.427+0000 7f9b82200640 1 --2- 172.21.15.12:0/3525422038 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ce52f8a0c0 0x55ce52f91530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:13:35.435 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:35.427+0000 7f9b82200640 1 -- 172.21.15.12:0/3525422038 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ce52c4b790 con 0x55ce52f8cb00 2024-09-11T00:13:35.435 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:35.427+0000 7f9b82200640 1 --2- 172.21.15.12:0/3525422038 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ce52f8cb00 0x55ce52f92100 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f9b74005c40 tx=0x7f9b74004860 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8e20f1bc713bb352 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:13:35.435 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:35.427+0000 7f9b63400640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-09-11T00:13:35.435 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:35.427+0000 7f9b63400640 1 -- 172.21.15.12:0/3525422038 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b74028070 con 0x55ce52f8cb00 2024-09-11T00:13:35.435 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:35.427+0000 7f9b871eb740 1 -- 172.21.15.12:0/3525422038 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9b74004050 con 0x55ce52f8cb00 2024-09-11T00:13:35.435 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:35.427+0000 7f9b63400640 1 -- 172.21.15.12:0/3525422038 <== mon.2 v2:172.21.15.165:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9208+0+0 (secure 0 0 0) 0x7f9b74005040 con 0x55ce52f8cb00 2024-09-11T00:13:35.436 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:35.427+0000 7f9b63400640 1 -- 172.21.15.12:0/3525422038 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f9b74034050 con 0x55ce52f8cb00 2024-09-11T00:13:35.436 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:35.427+0000 7f9b871eb740 10 client.26246 fetch_fsmap learned FSMap version 39 2024-09-11T00:13:35.436 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:35.427+0000 7f9b871eb740 1 -- 172.21.15.12:0/3525422038 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55ce52cc4620 con 0x55ce52f8cb00 2024-09-11T00:13:35.436 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:35.428+0000 7f9b63400640 1 -- 172.21.15.12:0/3525422038 <== mon.2 v2:172.21.15.165:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9b74004050 con 0x55ce52f8cb00 2024-09-11T00:13:35.436 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:35.428+0000 7f9b63400640 1 -- 172.21.15.12:0/3525422038 <== mon.2 v2:172.21.15.165:3300/0 5 ==== fsmap(e 39) ==== 8378+0+0 (secure 0 0 0) 0x7f9b7400f4e0 con 0x55ce52f8cb00 2024-09-11T00:13:35.436 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:35.428+0000 7f9b871eb740 10 client.26246 fetch_fsmap finished waiting for FSMap version 39 2024-09-11T00:13:35.436 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:35.428+0000 7f9b871eb740 10 client.26246 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T00:13:35.436 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:35.428+0000 7f9b871eb740 20 client.26246 populate_metadata read hostname 'smithi012' 2024-09-11T00:13:35.436 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:35.428+0000 7f9b871eb740 1 --2- 172.21.15.12:0/3525422038 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55ce52f9c560 0x55ce52f9e950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:13:35.436 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:35.428+0000 7f9b871eb740 4 client.26246 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T00:13:35.436 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:35.428+0000 7f9b871eb740 1 -- 172.21.15.12:0/3525422038 --> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ce52efc600 con 0x55ce52f9c560 2024-09-11T00:13:35.436 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:35.428+0000 7f9b81800640 1 --2- 172.21.15.12:0/3525422038 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55ce52f9c560 0x55ce52f9e950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:13:35.437 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:35.429+0000 7f9b81800640 1 --2- 172.21.15.12:0/3525422038 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55ce52f9c560 0x55ce52f9e950 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c9a3075c35e793e9 server_cookie=20855e95a5f5416 in_seq=0 out_seq=0 2024-09-11T00:13:35.437 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:35.429+0000 7f9b63400640 10 client.26246 ms_handle_connect on v2:172.21.15.165:6838/501312984 2024-09-11T00:13:35.437 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:35.429+0000 7f9b63400640 1 -- 172.21.15.12:0/3525422038 <== mds.0 v2:172.21.15.165:6838/501312984 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55ce52efc600 con 0x55ce52f9c560 2024-09-11T00:13:35.437 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:35.429+0000 7f9b63400640 10 client.26246 handle_command_reply: tid=0 2024-09-11T00:13:35.437 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:35.430+0000 7f9b871eb740 1 client.26246 shutdown 2024-09-11T00:13:35.437 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:35.430+0000 7f9b871eb740 1 -- 172.21.15.12:0/3525422038 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ce52f8cb00 msgr2=0x55ce52f92100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:13:35.437 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:35.430+0000 7f9b871eb740 1 --2- 172.21.15.12:0/3525422038 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ce52f8cb00 0x55ce52f92100 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f9b74005c40 tx=0x7f9b74004860 comp rx=0 tx=0).stop 2024-09-11T00:13:35.437 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:35.430+0000 7f9b871eb740 1 -- 172.21.15.12:0/3525422038 shutdown_connections 2024-09-11T00:13:35.437 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:35.430+0000 7f9b871eb740 1 --2- 172.21.15.12:0/3525422038 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55ce52f9c560 0x55ce52f9e950 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:13:35.437 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:35.430+0000 7f9b871eb740 1 --2- 172.21.15.12:0/3525422038 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ce52f8cb00 0x55ce52f92100 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:13:35.437 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:35.430+0000 7f9b871eb740 1 --2- 172.21.15.12:0/3525422038 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ce52f8a0c0 0x55ce52f91530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:13:35.437 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:35.430+0000 7f9b63400640 0 client.26246 ms_handle_reset on v2:172.21.15.165:6838/501312984 2024-09-11T00:13:35.437 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:35.430+0000 7f9b871eb740 1 --2- 172.21.15.12:0/3525422038 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ce52f87790 0x55ce52f90e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:13:35.438 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:35.430+0000 7f9b871eb740 1 -- 172.21.15.12:0/3525422038 >> 172.21.15.12:0/3525422038 conn(0x55ce52f828e0 msgr2=0x55ce52f81a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:13:35.438 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:35.430+0000 7f9b871eb740 1 -- 172.21.15.12:0/3525422038 shutdown_connections 2024-09-11T00:13:35.438 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:35.430+0000 7f9b871eb740 1 -- 172.21.15.12:0/3525422038 wait complete. 2024-09-11T00:13:35.438 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:35.430+0000 7f9b871eb740 20 client.26246 trim_cache size 0 max 16384 2024-09-11T00:13:35.451 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:35.447+0000 7f9b871eb740 1 Processor -- start 2024-09-11T00:13:35.451 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:35.447+0000 7f9b871eb740 1 -- start start 2024-09-11T00:13:35.451 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:35.447+0000 7f9b871eb740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ce52f87790 0x55ce52f86c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:13:35.451 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:35.447+0000 7f9b871eb740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ce52f8a0c0 0x55ce52f87160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:13:35.451 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:35.447+0000 7f9b871eb740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ce52f8cb00 0x55ce52f8cee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:13:35.451 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:35.447+0000 7f9b871eb740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f9b7c1ad940 con 0x55ce52f8cb00 2024-09-11T00:13:35.451 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:35.447+0000 7f9b871eb740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f9b7c1af4e0 con 0x55ce52f8a0c0 2024-09-11T00:13:35.451 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:35.447+0000 7f9b871eb740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f9b7c1ae010 con 0x55ce52f87790 2024-09-11T00:13:35.451 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:35.448+0000 7f9b80e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ce52f8a0c0 0x55ce52f87160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:13:35.452 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:35.448+0000 7f9b80e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ce52f8a0c0 0x55ce52f87160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.12:56470/0 (socket says 172.21.15.12:56470) 2024-09-11T00:13:35.452 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:35.448+0000 7f9b80e00640 1 -- 172.21.15.12:0/3355187316 learned_addr learned my addr 172.21.15.12:0/3355187316 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:13:35.452 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:35.448+0000 7f9b82200640 1 --2- 172.21.15.12:0/3355187316 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ce52f8cb00 0x55ce52f8cee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:13:35.452 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:35.448+0000 7f9b81800640 1 --2- 172.21.15.12:0/3355187316 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ce52f87790 0x55ce52f86c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:13:35.452 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:35.448+0000 7f9b80e00640 1 -- 172.21.15.12:0/3355187316 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ce52f87790 msgr2=0x55ce52f86c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:13:35.452 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:35.448+0000 7f9b80e00640 1 --2- 172.21.15.12:0/3355187316 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ce52f87790 0x55ce52f86c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:13:35.452 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:35.448+0000 7f9b80e00640 1 -- 172.21.15.12:0/3355187316 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ce52f8cb00 msgr2=0x55ce52f8cee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:13:35.452 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:35.448+0000 7f9b80e00640 1 --2- 172.21.15.12:0/3355187316 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ce52f8cb00 0x55ce52f8cee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:13:35.452 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:35.448+0000 7f9b80e00640 1 -- 172.21.15.12:0/3355187316 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ce52f83fe0 con 0x55ce52f8a0c0 2024-09-11T00:13:35.452 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:35.448+0000 7f9b80e00640 1 --2- 172.21.15.12:0/3355187316 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ce52f8a0c0 0x55ce52f87160 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f9b680043c0 tx=0x7f9b6801b000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=388bbca0d9eba6d7 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:13:35.452 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:35.449+0000 7f9b6e000640 1 -- 172.21.15.12:0/3355187316 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b680044b0 con 0x55ce52f8a0c0 2024-09-11T00:13:35.452 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:35.449+0000 7f9b6e000640 1 -- 172.21.15.12:0/3355187316 <== mon.2 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9b6802a8e0 con 0x55ce52f8a0c0 2024-09-11T00:13:35.453 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:35.449+0000 7f9b6e000640 1 -- 172.21.15.12:0/3355187316 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b6801f9d0 con 0x55ce52f8a0c0 2024-09-11T00:13:35.453 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:35.450+0000 7f9b871eb740 1 -- 172.21.15.12:0/3355187316 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ce52f8a0c0 msgr2=0x55ce52f87160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:13:35.453 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:35.450+0000 7f9b871eb740 1 --2- 172.21.15.12:0/3355187316 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ce52f8a0c0 0x55ce52f87160 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f9b680043c0 tx=0x7f9b6801b000 comp rx=0 tx=0).stop 2024-09-11T00:13:35.453 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:35.450+0000 7f9b871eb740 1 -- 172.21.15.12:0/3355187316 shutdown_connections 2024-09-11T00:13:35.453 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:35.450+0000 7f9b871eb740 1 --2- 172.21.15.12:0/3355187316 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ce52f8cb00 0x55ce52f8cee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:13:35.453 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:35.450+0000 7f9b871eb740 1 --2- 172.21.15.12:0/3355187316 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ce52f8a0c0 0x55ce52f87160 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:13:35.453 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:35.450+0000 7f9b871eb740 1 --2- 172.21.15.12:0/3355187316 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ce52f87790 0x55ce52f86c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:13:35.454 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:35.450+0000 7f9b871eb740 1 -- 172.21.15.12:0/3355187316 >> 172.21.15.12:0/3355187316 conn(0x55ce52f828e0 msgr2=0x55ce52f835a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:13:35.454 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:35.450+0000 7f9b871eb740 1 -- 172.21.15.12:0/3355187316 shutdown_connections 2024-09-11T00:13:35.454 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:35.450+0000 7f9b871eb740 1 -- 172.21.15.12:0/3355187316 wait complete. 2024-09-11T00:13:35.456 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:35.450+0000 7f9b871eb740 1 -- start start 2024-09-11T00:13:35.456 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:35.450+0000 7f9b871eb740 1 Processor -- start 2024-09-11T00:13:35.456 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:35.451+0000 7f9b871eb740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ce52f87790 0x55ce52f98820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:13:35.457 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:35.451+0000 7f9b871eb740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ce52f8a0c0 0x55ce52f97520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:13:35.457 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:35.451+0000 7f9b871eb740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ce52f8cb00 0x55ce52f97a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:13:35.457 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:35.451+0000 7f9b871eb740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f9b7c1aee10 con 0x55ce52f8cb00 2024-09-11T00:13:35.457 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:35.451+0000 7f9b871eb740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f9b7c1ae740 con 0x55ce52f87790 2024-09-11T00:13:35.457 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:35.451+0000 7f9b871eb740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f9b7c1ab920 con 0x55ce52f8a0c0 2024-09-11T00:13:35.457 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:35.451+0000 7f9b871eb740 10 client.0 _ll_register_callbacks cb 0x7f9b680097a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:13:35.457 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:35.451+0000 7f9b81800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ce52f87790 0x55ce52f98820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:13:35.457 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:35.451+0000 7f9b81800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ce52f87790 0x55ce52f98820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.12:56486/0 (socket says 172.21.15.12:56486) 2024-09-11T00:13:35.457 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:35.451+0000 7f9b81800640 1 -- 172.21.15.12:0/3477227476 learned_addr learned my addr 172.21.15.12:0/3477227476 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:13:35.457 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:35.451+0000 7f9b80e00640 1 --2- 172.21.15.12:0/3477227476 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ce52f8a0c0 0x55ce52f97520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:13:35.458 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:35.451+0000 7f9b82200640 1 --2- 172.21.15.12:0/3477227476 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ce52f8cb00 0x55ce52f97a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:13:35.458 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:35.452+0000 7f9b81800640 1 -- 172.21.15.12:0/3477227476 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ce52f8a0c0 msgr2=0x55ce52f97520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:13:35.458 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:35.452+0000 7f9b81800640 1 --2- 172.21.15.12:0/3477227476 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ce52f8a0c0 0x55ce52f97520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:13:35.458 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:35.452+0000 7f9b81800640 1 -- 172.21.15.12:0/3477227476 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ce52f8cb00 msgr2=0x55ce52f97a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:13:35.458 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:35.452+0000 7f9b81800640 1 --2- 172.21.15.12:0/3477227476 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ce52f8cb00 0x55ce52f97a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:13:35.458 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:35.452+0000 7f9b81800640 1 -- 172.21.15.12:0/3477227476 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ce52f98120 con 0x55ce52f87790 2024-09-11T00:13:35.458 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:35.452+0000 7f9b82200640 1 --2- 172.21.15.12:0/3477227476 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ce52f8cb00 0x55ce52f97a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T00:13:35.458 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:35.452+0000 7f9b80e00640 1 --2- 172.21.15.12:0/3477227476 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ce52f8a0c0 0x55ce52f97520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:13:35.458 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:35.452+0000 7f9b81800640 1 --2- 172.21.15.12:0/3477227476 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ce52f87790 0x55ce52f98820 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f9b700c9f00 tx=0x7f9b7002a420 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1012d3adc7c38c70 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:13:35.458 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:35.452+0000 7f9b63400640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-09-11T00:13:35.459 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:35.452+0000 7f9b63400640 1 -- 172.21.15.12:0/3477227476 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b70049040 con 0x55ce52f87790 2024-09-11T00:13:35.459 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:35.452+0000 7f9b871eb740 1 -- 172.21.15.12:0/3477227476 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9b6802a8e0 con 0x55ce52f87790 2024-09-11T00:13:35.459 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:35.452+0000 7f9b63400640 1 -- 172.21.15.12:0/3477227476 <== mon.2 v2:172.21.15.165:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9208+0+0 (secure 0 0 0) 0x7f9b700c8f00 con 0x55ce52f87790 2024-09-11T00:13:35.459 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:35.452+0000 7f9b63400640 1 -- 172.21.15.12:0/3477227476 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f9b700cc7a0 con 0x55ce52f87790 2024-09-11T00:13:35.459 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:35.452+0000 7f9b871eb740 10 client.26252 fetch_fsmap learned FSMap version 39 2024-09-11T00:13:35.459 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:35.452+0000 7f9b871eb740 1 -- 172.21.15.12:0/3477227476 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55ce53009420 con 0x55ce52f87790 2024-09-11T00:13:35.459 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:35.453+0000 7f9b63400640 1 -- 172.21.15.12:0/3477227476 <== mon.2 v2:172.21.15.165:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9b6802a8e0 con 0x55ce52f87790 2024-09-11T00:13:35.459 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:35.453+0000 7f9b63400640 1 -- 172.21.15.12:0/3477227476 <== mon.2 v2:172.21.15.165:3300/0 5 ==== fsmap(e 39) ==== 8378+0+0 (secure 0 0 0) 0x7f9b700d5ce0 con 0x55ce52f87790 2024-09-11T00:13:35.459 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:35.454+0000 7f9b871eb740 10 client.26252 fetch_fsmap finished waiting for FSMap version 39 2024-09-11T00:13:35.459 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:35.454+0000 7f9b871eb740 10 client.26252 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T00:13:35.459 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:35.454+0000 7f9b871eb740 20 client.26252 populate_metadata read hostname 'smithi012' 2024-09-11T00:13:35.460 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:35.454+0000 7f9b871eb740 1 --2- 172.21.15.12:0/3477227476 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55ce53009d80 0x55ce5300c1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:13:35.460 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:35.454+0000 7f9b871eb740 4 client.26252 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T00:13:35.460 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:35.454+0000 7f9b871eb740 1 -- 172.21.15.12:0/3477227476 --> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55ce52ef80b0 con 0x55ce53009d80 2024-09-11T00:13:35.460 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:35.455+0000 7f9b80e00640 1 --2- 172.21.15.12:0/3477227476 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55ce53009d80 0x55ce5300c1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:13:35.460 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:35.456+0000 7f9b63400640 10 client.26252 ms_handle_connect on v2:172.21.15.165:6838/501312984 2024-09-11T00:13:35.460 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:35.456+0000 7f9b80e00640 1 --2- 172.21.15.12:0/3477227476 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55ce53009d80 0x55ce5300c1a0 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4b9ca30682b4c789 server_cookie=b66c42cca17cbcec in_seq=0 out_seq=0 2024-09-11T00:13:35.527 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:35.525+0000 7f9b63400640 1 -- 172.21.15.12:0/3477227476 <== mds.0 v2:172.21.15.165:6838/501312984 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55ce52ef80b0 con 0x55ce53009d80 2024-09-11T00:13:35.527 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:35.525+0000 7f9b63400640 10 client.26252 handle_command_reply: tid=0 2024-09-11T00:13:35.527 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:35.525+0000 7f9b871eb740 1 client.26252 shutdown 2024-09-11T00:13:35.528 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:35.525+0000 7f9b871eb740 1 -- 172.21.15.12:0/3477227476 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ce52f87790 msgr2=0x55ce52f98820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:13:35.528 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:35.525+0000 7f9b871eb740 1 --2- 172.21.15.12:0/3477227476 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ce52f87790 0x55ce52f98820 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f9b700c9f00 tx=0x7f9b7002a420 comp rx=0 tx=0).stop 2024-09-11T00:13:35.528 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:35.526+0000 7f9b871eb740 1 -- 172.21.15.12:0/3477227476 shutdown_connections 2024-09-11T00:13:35.528 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:35.526+0000 7f9b871eb740 1 --2- 172.21.15.12:0/3477227476 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55ce53009d80 0x55ce5300c1a0 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:13:35.528 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:35.526+0000 7f9b871eb740 1 --2- 172.21.15.12:0/3477227476 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ce52f8cb00 0x55ce52f97a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:13:35.528 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:35.526+0000 7f9b871eb740 1 --2- 172.21.15.12:0/3477227476 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ce52f8a0c0 0x55ce52f97520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:13:35.528 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:35.526+0000 7f9b871eb740 1 --2- 172.21.15.12:0/3477227476 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ce52f87790 0x55ce52f98820 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:13:35.528 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:35.526+0000 7f9b871eb740 1 -- 172.21.15.12:0/3477227476 >> 172.21.15.12:0/3477227476 conn(0x55ce52f828e0 msgr2=0x55ce52f8cee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:13:35.528 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:35.526+0000 7f9b63400640 0 client.26252 ms_handle_reset on v2:172.21.15.165:6838/501312984 2024-09-11T00:13:35.528 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:35.526+0000 7f9b871eb740 1 -- 172.21.15.12:0/3477227476 shutdown_connections 2024-09-11T00:13:35.528 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:35.526+0000 7f9b871eb740 1 -- 172.21.15.12:0/3477227476 wait complete. 2024-09-11T00:13:35.528 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:35.526+0000 7f9b871eb740 20 client.26252 trim_cache size 0 max 16384 2024-09-11T00:13:35.529 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:{ 2024-09-11T00:13:35.529 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "message": "", 2024-09-11T00:13:35.529 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "return_code": 0 2024-09-11T00:13:35.529 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:} 2024-09-11T00:13:35.530 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:35.528+0000 7f9b84200640 1 -- 172.21.15.12:0/3423830406 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f9b640b5290 msgr2=0x7f9b640b7750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:13:35.530 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:35.528+0000 7f9b84200640 1 --2- 172.21.15.12:0/3423830406 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f9b640b5290 0x7f9b640b7750 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f9b68009cc0 tx=0x7f9b68005f90 comp rx=0 tx=0).stop 2024-09-11T00:13:35.530 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:35.528+0000 7f9b84200640 1 -- 172.21.15.12:0/3423830406 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9b7c104ba0 msgr2=0x7f9b7c19afa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:13:35.530 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:35.528+0000 7f9b84200640 1 --2- 172.21.15.12:0/3423830406 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9b7c104ba0 0x7f9b7c19afa0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f9b70009a50 tx=0x7f9b700042f0 comp rx=0 tx=0).stop 2024-09-11T00:13:35.530 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:35.529+0000 7f9b84200640 1 -- 172.21.15.12:0/3423830406 shutdown_connections 2024-09-11T00:13:35.530 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:35.529+0000 7f9b84200640 1 --2- 172.21.15.12:0/3423830406 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f9b640b5290 0x7f9b640b7750 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:13:35.530 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:35.529+0000 7f9b84200640 1 --2- 172.21.15.12:0/3423830406 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9b7c1066b0 0x7f9b7c19c220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:13:35.530 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:35.529+0000 7f9b84200640 1 --2- 172.21.15.12:0/3423830406 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9b7c105cf0 0x7f9b7c19b5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:13:35.530 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:35.529+0000 7f9b84200640 1 --2- 172.21.15.12:0/3423830406 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9b7c104ba0 0x7f9b7c19afa0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:13:35.530 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:35.529+0000 7f9b84200640 1 -- 172.21.15.12:0/3423830406 >> 172.21.15.12:0/3423830406 conn(0x7f9b7c100460 msgr2=0x7f9b7c1090e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:13:35.531 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:35.529+0000 7f9b84200640 1 -- 172.21.15.12:0/3423830406 shutdown_connections 2024-09-11T00:13:35.531 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:35.529+0000 7f9b84200640 1 -- 172.21.15.12:0/3423830406 wait complete. 2024-09-11T00:13:35.640 INFO:tasks.workunit.client.0.smithi012.stdout: 1 223766 3.24 MB/sec execute 212 sec latency 49.733 ms 2024-09-11T00:13:36.190 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:13:35 smithi012 ceph-mon[30333]: pgmap v594: 129 pgs: 129 active+clean; 60 MiB data, 985 MiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 1.6 MiB/s wr, 278 op/s 2024-09-11T00:13:36.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:13:35 smithi165 ceph-mon[35356]: pgmap v594: 129 pgs: 129 active+clean; 60 MiB data, 985 MiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 1.6 MiB/s wr, 278 op/s 2024-09-11T00:13:36.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:13:35 smithi178 ceph-mon[35919]: pgmap v594: 129 pgs: 129 active+clean; 60 MiB data, 985 MiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 1.6 MiB/s wr, 278 op/s 2024-09-11T00:13:36.650 INFO:tasks.workunit.client.0.smithi012.stdout: 1 223968 3.23 MB/sec execute 213 sec latency 694.782 ms 2024-09-11T00:13:37.650 INFO:tasks.workunit.client.0.smithi012.stdout: 1 225653 3.24 MB/sec execute 214 sec latency 743.642 ms 2024-09-11T00:13:38.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:13:37 smithi165 ceph-mon[35356]: pgmap v595: 129 pgs: 129 active+clean; 62 MiB data, 1015 MiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 2.0 MiB/s wr, 360 op/s 2024-09-11T00:13:38.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:13:37 smithi012 ceph-mon[30333]: pgmap v595: 129 pgs: 129 active+clean; 62 MiB data, 1015 MiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 2.0 MiB/s wr, 360 op/s 2024-09-11T00:13:38.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:13:37 smithi178 ceph-mon[35919]: pgmap v595: 129 pgs: 129 active+clean; 62 MiB data, 1015 MiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 2.0 MiB/s wr, 360 op/s 2024-09-11T00:13:38.651 INFO:tasks.workunit.client.0.smithi012.stdout: 1 227256 3.29 MB/sec execute 215 sec latency 177.786 ms 2024-09-11T00:13:39.651 INFO:tasks.workunit.client.0.smithi012.stdout: 1 228544 3.30 MB/sec execute 216 sec latency 13.094 ms 2024-09-11T00:13:40.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:13:39 smithi165 ceph-mon[35356]: pgmap v596: 129 pgs: 129 active+clean; 61 MiB data, 1019 MiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 1.5 MiB/s wr, 260 op/s 2024-09-11T00:13:40.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:13:39 smithi012 ceph-mon[30333]: pgmap v596: 129 pgs: 129 active+clean; 61 MiB data, 1019 MiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 1.5 MiB/s wr, 260 op/s 2024-09-11T00:13:40.330 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:13:39 smithi178 ceph-mon[35919]: pgmap v596: 129 pgs: 129 active+clean; 61 MiB data, 1019 MiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 1.5 MiB/s wr, 260 op/s 2024-09-11T00:13:40.651 INFO:tasks.workunit.client.0.smithi012.stdout: 1 230633 3.34 MB/sec execute 217 sec latency 164.246 ms 2024-09-11T00:13:41.651 INFO:tasks.workunit.client.0.smithi012.stdout: 1 230928 3.33 MB/sec execute 218 sec latency 583.462 ms 2024-09-11T00:13:42.208 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:13:41 smithi012 ceph-mon[30333]: pgmap v597: 129 pgs: 129 active+clean; 61 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 1.7 MiB/s wr, 288 op/s 2024-09-11T00:13:42.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:13:41 smithi165 ceph-mon[35356]: pgmap v597: 129 pgs: 129 active+clean; 61 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 1.7 MiB/s wr, 288 op/s 2024-09-11T00:13:42.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:13:41 smithi178 ceph-mon[35919]: pgmap v597: 129 pgs: 129 active+clean; 61 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 1.7 MiB/s wr, 288 op/s 2024-09-11T00:13:42.651 INFO:tasks.workunit.client.0.smithi012.stdout: 1 230983 3.32 MB/sec execute 219 sec latency 1453.178 ms 2024-09-11T00:13:43.651 INFO:tasks.workunit.client.0.smithi012.stdout: 1 232940 3.33 MB/sec execute 220 sec latency 13.537 ms 2024-09-11T00:13:44.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:13:43 smithi165 ceph-mon[35356]: pgmap v598: 129 pgs: 129 active+clean; 64 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 1.7 MiB/s wr, 324 op/s 2024-09-11T00:13:44.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:13:43 smithi012 ceph-mon[30333]: pgmap v598: 129 pgs: 129 active+clean; 64 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 1.7 MiB/s wr, 324 op/s 2024-09-11T00:13:44.330 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:13:43 smithi178 ceph-mon[35919]: pgmap v598: 129 pgs: 129 active+clean; 64 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 1.7 MiB/s wr, 324 op/s 2024-09-11T00:13:44.651 INFO:tasks.workunit.client.0.smithi012.stdout: 1 234236 3.37 MB/sec execute 221 sec latency 275.640 ms 2024-09-11T00:13:45.651 INFO:tasks.workunit.client.0.smithi012.stdout: 1 234699 3.36 MB/sec execute 222 sec latency 592.197 ms 2024-09-11T00:13:46.208 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:13:45 smithi012 ceph-mon[30333]: pgmap v599: 129 pgs: 129 active+clean; 64 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 1.5 MiB/s wr, 244 op/s 2024-09-11T00:13:46.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:13:45 smithi165 ceph-mon[35356]: pgmap v599: 129 pgs: 129 active+clean; 64 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 1.5 MiB/s wr, 244 op/s 2024-09-11T00:13:46.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:13:45 smithi178 ceph-mon[35919]: pgmap v599: 129 pgs: 129 active+clean; 64 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 1.5 MiB/s wr, 244 op/s 2024-09-11T00:13:46.651 INFO:tasks.workunit.client.0.smithi012.stdout: 1 235368 3.35 MB/sec execute 223 sec latency 514.054 ms 2024-09-11T00:13:47.651 INFO:tasks.workunit.client.0.smithi012.stdout: 1 235840 3.35 MB/sec execute 224 sec latency 1310.469 ms 2024-09-11T00:13:48.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:13:47 smithi165 ceph-mon[35356]: pgmap v600: 129 pgs: 129 active+clean; 65 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 2.2 MiB/s wr, 357 op/s 2024-09-11T00:13:48.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:13:47 smithi012 ceph-mon[30333]: pgmap v600: 129 pgs: 129 active+clean; 65 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 2.2 MiB/s wr, 357 op/s 2024-09-11T00:13:48.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:13:47 smithi178 ceph-mon[35919]: pgmap v600: 129 pgs: 129 active+clean; 65 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 2.2 MiB/s wr, 357 op/s 2024-09-11T00:13:48.651 INFO:tasks.workunit.client.0.smithi012.stdout: 1 237858 3.39 MB/sec execute 225 sec latency 82.268 ms 2024-09-11T00:13:48.678 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:48.677+0000 7f2873400640 1 -- 172.21.15.12:0/2685959625 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f286c102990 msgr2=0x7f286c102d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:13:48.679 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:48.677+0000 7f2873400640 1 --2- 172.21.15.12:0/2685959625 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f286c102990 0x7f286c102d90 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f2860009900 tx=0x7f286002a3f0 comp rx=0 tx=0).stop 2024-09-11T00:13:48.679 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:48.677+0000 7f2873400640 1 -- 172.21.15.12:0/2685959625 shutdown_connections 2024-09-11T00:13:48.679 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:48.677+0000 7f2873400640 1 --2- 172.21.15.12:0/2685959625 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f286c1044a0 0x7f286c10ae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:13:48.679 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:48.677+0000 7f2873400640 1 --2- 172.21.15.12:0/2685959625 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f286c103ae0 0x7f286c103f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:13:48.679 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:48.677+0000 7f2873400640 1 --2- 172.21.15.12:0/2685959625 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f286c102990 0x7f286c102d90 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:13:48.679 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:48.677+0000 7f2873400640 1 -- 172.21.15.12:0/2685959625 >> 172.21.15.12:0/2685959625 conn(0x7f286c0fe250 msgr2=0x7f286c100670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:13:48.679 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:48.677+0000 7f2873400640 1 -- 172.21.15.12:0/2685959625 shutdown_connections 2024-09-11T00:13:48.679 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:48.677+0000 7f2873400640 1 -- 172.21.15.12:0/2685959625 wait complete. 2024-09-11T00:13:48.681 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:48.678+0000 7f2873400640 1 Processor -- start 2024-09-11T00:13:48.681 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:48.678+0000 7f2873400640 1 -- start start 2024-09-11T00:13:48.681 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:48.679+0000 7f2873400640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f286c102990 0x7f286c198d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:13:48.681 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:48.679+0000 7f2873400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f286c103ae0 0x7f286c1993c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:13:48.681 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:48.679+0000 7f2873400640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f286c1044a0 0x7f286c199ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:13:48.681 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:48.679+0000 7f2873400640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f286c10e650 con 0x7f286c102990 2024-09-11T00:13:48.681 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:48.679+0000 7f2873400640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f286c10e4d0 con 0x7f286c103ae0 2024-09-11T00:13:48.681 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:48.679+0000 7f2873400640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f286c10e7d0 con 0x7f286c1044a0 2024-09-11T00:13:48.681 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:48.679+0000 7f2870a00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f286c102990 0x7f286c198d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:13:48.681 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:48.679+0000 7f2870a00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f286c102990 0x7f286c198d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:57532/0 (socket says 172.21.15.12:57532) 2024-09-11T00:13:48.682 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:48.679+0000 7f2870a00640 1 -- 172.21.15.12:0/1634858843 learned_addr learned my addr 172.21.15.12:0/1634858843 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:13:48.682 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:48.679+0000 7f286be00640 1 --2- 172.21.15.12:0/1634858843 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f286c103ae0 0x7f286c1993c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:13:48.682 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:48.680+0000 7f2871400640 1 --2- 172.21.15.12:0/1634858843 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f286c1044a0 0x7f286c199ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:13:48.682 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:48.680+0000 7f286be00640 1 -- 172.21.15.12:0/1634858843 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f286c1044a0 msgr2=0x7f286c199ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:13:48.682 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:48.680+0000 7f286be00640 1 --2- 172.21.15.12:0/1634858843 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f286c1044a0 0x7f286c199ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:13:48.682 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:48.680+0000 7f286be00640 1 -- 172.21.15.12:0/1634858843 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f286c102990 msgr2=0x7f286c198d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:13:48.682 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:48.680+0000 7f286be00640 1 --2- 172.21.15.12:0/1634858843 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f286c102990 0x7f286c198d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:13:48.682 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:48.680+0000 7f286be00640 1 -- 172.21.15.12:0/1634858843 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f286c19a8b0 con 0x7f286c103ae0 2024-09-11T00:13:48.682 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:48.680+0000 7f2870a00640 1 --2- 172.21.15.12:0/1634858843 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f286c102990 0x7f286c198d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:13:48.683 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:48.680+0000 7f286be00640 1 --2- 172.21.15.12:0/1634858843 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f286c103ae0 0x7f286c1993c0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f285800e9f0 tx=0x7f285800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:13:48.683 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:48.680+0000 7f2869600640 1 -- 172.21.15.12:0/1634858843 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f285800ccc0 con 0x7f286c103ae0 2024-09-11T00:13:48.683 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:48.680+0000 7f2869600640 1 -- 172.21.15.12:0/1634858843 <== mon.2 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f28580108e0 con 0x7f286c103ae0 2024-09-11T00:13:48.683 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:48.681+0000 7f2873400640 1 -- 172.21.15.12:0/1634858843 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f286c19aba0 con 0x7f286c103ae0 2024-09-11T00:13:48.683 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:48.681+0000 7f2873400640 1 -- 172.21.15.12:0/1634858843 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f286c1a1270 con 0x7f286c103ae0 2024-09-11T00:13:48.683 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:48.681+0000 7f2869600640 1 -- 172.21.15.12:0/1634858843 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2858018880 con 0x7f286c103ae0 2024-09-11T00:13:48.685 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:48.684+0000 7f287628a740 1 Processor -- start 2024-09-11T00:13:48.685 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:48.684+0000 7f287628a740 1 -- start start 2024-09-11T00:13:48.686 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:48.684+0000 7f287628a740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5559d235f790 0x5559d2361b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:13:48.686 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:48.684+0000 7f287628a740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5559d23620c0 0x5559d23644b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:13:48.686 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:48.684+0000 7f287628a740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5559d2364b00 0x5559d2366fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:13:48.686 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:48.684+0000 7f287628a740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x5559d22d26b0 con 0x5559d235f790 2024-09-11T00:13:48.687 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:48.684+0000 7f287628a740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x5559d22c1ae0 con 0x5559d23620c0 2024-09-11T00:13:48.687 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:48.684+0000 7f287628a740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5559d22c58f0 con 0x5559d2364b00 2024-09-11T00:13:48.687 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:48.685+0000 7f2870a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5559d23620c0 0x5559d23644b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:13:48.687 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:48.685+0000 7f2870a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5559d23620c0 0x5559d23644b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.12:53876/0 (socket says 172.21.15.12:53876) 2024-09-11T00:13:48.687 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:48.685+0000 7f2871400640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5559d235f790 0x5559d2361b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:13:48.687 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:48.685+0000 7f2870a00640 1 -- 172.21.15.12:0/988243618 learned_addr learned my addr 172.21.15.12:0/988243618 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:13:48.687 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:48.685+0000 7f2870a00640 1 -- 172.21.15.12:0/988243618 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5559d2364b00 msgr2=0x5559d2366fc0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-11T00:13:48.687 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:48.685+0000 7f2870a00640 1 --2- 172.21.15.12:0/988243618 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5559d2364b00 0x5559d2366fc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:13:48.687 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:48.685+0000 7f2870a00640 1 -- 172.21.15.12:0/988243618 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5559d235f790 msgr2=0x5559d2361b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:13:48.687 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:48.685+0000 7f2870a00640 1 --2- 172.21.15.12:0/988243618 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5559d235f790 0x5559d2361b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:13:48.687 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:48.685+0000 7f2870a00640 1 -- 172.21.15.12:0/988243618 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5559d228e780 con 0x5559d23620c0 2024-09-11T00:13:48.687 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:48.685+0000 7f2869600640 1 -- 172.21.15.12:0/1634858843 <== mon.2 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7f2858010430 con 0x7f286c103ae0 2024-09-11T00:13:48.687 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:48.685+0000 7f2870a00640 1 --2- 172.21.15.12:0/988243618 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5559d23620c0 0x5559d23644b0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f2860005ab0 tx=0x7f2860004470 comp rx=0 tx=0).ready entity=mon.2 client_cookie=786e69a8cffdc9a1 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:13:48.689 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:48.686+0000 7f285dc00640 1 -- 172.21.15.12:0/988243618 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2860035030 con 0x5559d23620c0 2024-09-11T00:13:48.689 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:48.686+0000 7f285dc00640 1 -- 172.21.15.12:0/988243618 <== mon.2 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2860040070 con 0x5559d23620c0 2024-09-11T00:13:48.689 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:48.686+0000 7f285dc00640 1 -- 172.21.15.12:0/988243618 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2860031d30 con 0x5559d23620c0 2024-09-11T00:13:48.689 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:48.687+0000 7f287628a740 1 -- 172.21.15.12:0/988243618 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5559d23620c0 msgr2=0x5559d23644b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:13:48.689 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:48.687+0000 7f287628a740 1 --2- 172.21.15.12:0/988243618 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5559d23620c0 0x5559d23644b0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f2860005ab0 tx=0x7f2860004470 comp rx=0 tx=0).stop 2024-09-11T00:13:48.689 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:48.687+0000 7f287628a740 1 -- 172.21.15.12:0/988243618 shutdown_connections 2024-09-11T00:13:48.689 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:48.687+0000 7f287628a740 1 --2- 172.21.15.12:0/988243618 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5559d2364b00 0x5559d2366fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:13:48.689 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:48.687+0000 7f287628a740 1 --2- 172.21.15.12:0/988243618 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5559d23620c0 0x5559d23644b0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:13:48.689 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:48.687+0000 7f287628a740 1 --2- 172.21.15.12:0/988243618 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5559d235f790 0x5559d2361b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:13:48.690 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:48.687+0000 7f287628a740 1 -- 172.21.15.12:0/988243618 >> 172.21.15.12:0/988243618 conn(0x5559d235a8e0 msgr2=0x5559d235b670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:13:48.690 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:48.687+0000 7f287628a740 1 -- 172.21.15.12:0/988243618 shutdown_connections 2024-09-11T00:13:48.690 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:48.687+0000 7f287628a740 1 -- 172.21.15.12:0/988243618 wait complete. 2024-09-11T00:13:48.690 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:48.688+0000 7f2869600640 1 --2- 172.21.15.12:0/1634858843 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f28540b5280 0x7f28540b7740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:13:48.690 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:48.688+0000 7f287628a740 1 -- start start 2024-09-11T00:13:48.690 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:48.688+0000 7f2871400640 1 --2- 172.21.15.12:0/1634858843 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f28540b5280 0x7f28540b7740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:13:48.690 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:48.688+0000 7f2869600640 1 -- 172.21.15.12:0/1634858843 <== mon.2 v2:172.21.15.165:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9208+0+0 (secure 0 0 0) 0x7f28580e61c0 con 0x7f286c103ae0 2024-09-11T00:13:48.690 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:48.688+0000 7f287628a740 1 Processor -- start 2024-09-11T00:13:48.691 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:48.689+0000 7f287628a740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5559d235f790 0x5559d2368ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:13:48.691 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:48.689+0000 7f287628a740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5559d23620c0 0x5559d2369540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:13:48.692 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:48.689+0000 7f287628a740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5559d2364b00 0x5559d236a110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:13:48.692 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:48.689+0000 7f287628a740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x5559d21f9ab0 con 0x5559d23620c0 2024-09-11T00:13:48.692 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:48.689+0000 7f287628a740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x5559d2268570 con 0x5559d2364b00 2024-09-11T00:13:48.692 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:48.689+0000 7f287628a740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5559d2367a80 con 0x5559d235f790 2024-09-11T00:13:48.692 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:48.689+0000 7f2871400640 1 --2- 172.21.15.12:0/1634858843 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f28540b5280 0x7f28540b7740 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f286c199af0 tx=0x7f2864009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:13:48.692 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:48.689+0000 7f287628a740 10 client.0 _ll_register_callbacks cb 0x5559d2277100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:13:48.692 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:48.689+0000 7f2870a00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5559d235f790 0x5559d2368ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:13:48.692 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:48.689+0000 7f2871400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5559d2364b00 0x5559d236a110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:13:48.692 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:48.689+0000 7f2870a00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5559d235f790 0x5559d2368ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.12:48626/0 (socket says 172.21.15.12:48626) 2024-09-11T00:13:48.692 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:48.689+0000 7f2870a00640 1 -- 172.21.15.12:0/1353924051 learned_addr learned my addr 172.21.15.12:0/1353924051 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:13:48.692 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:48.689+0000 7f286be00640 1 --2- 172.21.15.12:0/1353924051 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5559d23620c0 0x5559d2369540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:13:48.692 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:48.689+0000 7f2870a00640 1 -- 172.21.15.12:0/1353924051 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5559d2364b00 msgr2=0x5559d236a110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:13:48.693 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:48.689+0000 7f2870a00640 1 --2- 172.21.15.12:0/1353924051 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5559d2364b00 0x5559d236a110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:13:48.693 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:48.689+0000 7f2870a00640 1 -- 172.21.15.12:0/1353924051 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5559d23620c0 msgr2=0x5559d2369540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:13:48.693 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:48.689+0000 7f2870a00640 1 --2- 172.21.15.12:0/1353924051 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5559d23620c0 0x5559d2369540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:13:48.693 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:48.690+0000 7f2870a00640 1 -- 172.21.15.12:0/1353924051 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5559d2023790 con 0x5559d235f790 2024-09-11T00:13:48.693 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:48.690+0000 7f2871400640 1 --2- 172.21.15.12:0/1353924051 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5559d2364b00 0x5559d236a110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:13:48.693 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:48.690+0000 7f286be00640 1 --2- 172.21.15.12:0/1353924051 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5559d23620c0 0x5559d2369540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T00:13:48.693 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:48.690+0000 7f2870a00640 1 --2- 172.21.15.12:0/1353924051 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5559d235f790 0x5559d2368ea0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f2860000c00 tx=0x7f2860031760 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b2a23319a30b964b server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:13:48.693 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:48.690+0000 7f2852000640 10 client.0 ms_handle_connect on v2:172.21.15.178:3300/0 2024-09-11T00:13:48.693 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:48.690+0000 7f2852000640 1 -- 172.21.15.12:0/1353924051 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2860053060 con 0x5559d235f790 2024-09-11T00:13:48.693 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:48.690+0000 7f287628a740 1 -- 172.21.15.12:0/1353924051 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2860040070 con 0x5559d235f790 2024-09-11T00:13:48.693 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:48.690+0000 7f2852000640 1 -- 172.21.15.12:0/1353924051 <== mon.1 v2:172.21.15.178:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9208+0+0 (secure 0 0 0) 0x7f28600036b0 con 0x5559d235f790 2024-09-11T00:13:48.693 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:48.691+0000 7f2852000640 1 -- 172.21.15.12:0/1353924051 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f2860035070 con 0x5559d235f790 2024-09-11T00:13:48.693 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:48.691+0000 7f287628a740 10 client.26254 fetch_fsmap learned FSMap version 39 2024-09-11T00:13:48.693 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:48.691+0000 7f287628a740 1 -- 172.21.15.12:0/1353924051 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x5559d209c620 con 0x5559d235f790 2024-09-11T00:13:48.697 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:48.691+0000 7f2852000640 1 -- 172.21.15.12:0/1353924051 <== mon.1 v2:172.21.15.178:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2860040070 con 0x5559d235f790 2024-09-11T00:13:48.697 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:48.691+0000 7f2852000640 1 -- 172.21.15.12:0/1353924051 <== mon.1 v2:172.21.15.178:3300/0 5 ==== fsmap(e 39) ==== 8378+0+0 (secure 0 0 0) 0x7f286004d030 con 0x5559d235f790 2024-09-11T00:13:48.697 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:48.691+0000 7f287628a740 10 client.26254 fetch_fsmap finished waiting for FSMap version 39 2024-09-11T00:13:48.697 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:48.691+0000 7f287628a740 10 client.26254 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T00:13:48.697 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:48.691+0000 7f287628a740 20 client.26254 populate_metadata read hostname 'smithi012' 2024-09-11T00:13:48.697 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:48.691+0000 7f287628a740 1 --2- 172.21.15.12:0/1353924051 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x5559d2374340 0x5559d2376760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:13:48.697 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:48.691+0000 7f287628a740 4 client.26254 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T00:13:48.697 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:48.691+0000 7f287628a740 1 -- 172.21.15.12:0/1353924051 --> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5559d22d4600 con 0x5559d2374340 2024-09-11T00:13:48.697 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:48.692+0000 7f286be00640 1 --2- 172.21.15.12:0/1353924051 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x5559d2374340 0x5559d2376760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:13:48.698 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:48.692+0000 7f286be00640 1 --2- 172.21.15.12:0/1353924051 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x5559d2374340 0x5559d2376760 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7ecdf65c0c9c568e server_cookie=53d5d7d7a9450762 in_seq=0 out_seq=0 2024-09-11T00:13:48.698 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:48.692+0000 7f2852000640 10 client.26254 ms_handle_connect on v2:172.21.15.165:6838/501312984 2024-09-11T00:13:48.698 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:48.693+0000 7f2852000640 1 -- 172.21.15.12:0/1353924051 <== mds.0 v2:172.21.15.165:6838/501312984 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5559d22d4600 con 0x5559d2374340 2024-09-11T00:13:48.698 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:48.693+0000 7f2852000640 10 client.26254 handle_command_reply: tid=0 2024-09-11T00:13:48.698 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:48.693+0000 7f287628a740 1 client.26254 shutdown 2024-09-11T00:13:48.698 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:48.693+0000 7f287628a740 1 -- 172.21.15.12:0/1353924051 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5559d235f790 msgr2=0x5559d2368ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:13:48.698 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:48.693+0000 7f287628a740 1 --2- 172.21.15.12:0/1353924051 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5559d235f790 0x5559d2368ea0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f2860000c00 tx=0x7f2860031760 comp rx=0 tx=0).stop 2024-09-11T00:13:48.698 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:48.693+0000 7f287628a740 1 -- 172.21.15.12:0/1353924051 shutdown_connections 2024-09-11T00:13:48.698 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:48.693+0000 7f287628a740 1 --2- 172.21.15.12:0/1353924051 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x5559d2374340 0x5559d2376760 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:13:48.698 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:48.693+0000 7f287628a740 1 --2- 172.21.15.12:0/1353924051 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5559d2364b00 0x5559d236a110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:13:48.698 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:48.693+0000 7f287628a740 1 --2- 172.21.15.12:0/1353924051 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5559d23620c0 0x5559d2369540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:13:48.698 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:48.693+0000 7f2852000640 0 client.26254 ms_handle_reset on v2:172.21.15.165:6838/501312984 2024-09-11T00:13:48.698 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:48.693+0000 7f287628a740 1 --2- 172.21.15.12:0/1353924051 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5559d235f790 0x5559d2368ea0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:13:48.699 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:48.693+0000 7f287628a740 1 -- 172.21.15.12:0/1353924051 >> 172.21.15.12:0/1353924051 conn(0x5559d235a8e0 msgr2=0x5559d2359a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:13:48.699 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:48.694+0000 7f287628a740 1 -- 172.21.15.12:0/1353924051 shutdown_connections 2024-09-11T00:13:48.699 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:48.694+0000 7f287628a740 1 -- 172.21.15.12:0/1353924051 wait complete. 2024-09-11T00:13:48.699 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:48.694+0000 7f287628a740 20 client.26254 trim_cache size 0 max 16384 2024-09-11T00:13:48.712 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:48.710+0000 7f287628a740 1 Processor -- start 2024-09-11T00:13:48.712 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:48.710+0000 7f287628a740 1 -- start start 2024-09-11T00:13:48.712 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:48.710+0000 7f287628a740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5559d235f790 0x5559d235d930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:13:48.712 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:48.710+0000 7f287628a740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5559d23620c0 0x5559d235de70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:13:48.712 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:48.710+0000 7f287628a740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5559d2364b00 0x5559d235b140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:13:48.712 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:48.710+0000 7f287628a740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f284c00fd20 con 0x5559d23620c0 2024-09-11T00:13:48.712 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:48.710+0000 7f287628a740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f284c0118c0 con 0x5559d235f790 2024-09-11T00:13:48.712 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:48.710+0000 7f287628a740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f284c0103f0 con 0x5559d2364b00 2024-09-11T00:13:48.713 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:48.710+0000 7f2870a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5559d235f790 0x5559d235d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:13:48.713 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:48.711+0000 7f2870a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5559d235f790 0x5559d235d930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.12:53902/0 (socket says 172.21.15.12:53902) 2024-09-11T00:13:48.713 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:48.711+0000 7f2870a00640 1 -- 172.21.15.12:0/2732133828 learned_addr learned my addr 172.21.15.12:0/2732133828 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:13:48.713 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:48.711+0000 7f286be00640 1 --2- 172.21.15.12:0/2732133828 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5559d23620c0 0x5559d235de70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:13:48.713 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:48.711+0000 7f2871400640 1 --2- 172.21.15.12:0/2732133828 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5559d2364b00 0x5559d235b140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:13:48.713 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:48.711+0000 7f2870a00640 1 -- 172.21.15.12:0/2732133828 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5559d2364b00 msgr2=0x5559d235b140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:13:48.713 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:48.711+0000 7f2870a00640 1 --2- 172.21.15.12:0/2732133828 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5559d2364b00 0x5559d235b140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:13:48.713 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:48.711+0000 7f2870a00640 1 -- 172.21.15.12:0/2732133828 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5559d23620c0 msgr2=0x5559d235de70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:13:48.713 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:48.711+0000 7f2870a00640 1 --2- 172.21.15.12:0/2732133828 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5559d23620c0 0x5559d235de70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:13:48.713 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:48.711+0000 7f2870a00640 1 -- 172.21.15.12:0/2732133828 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5559d2361ef0 con 0x5559d235f790 2024-09-11T00:13:48.715 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:48.711+0000 7f2870a00640 1 --2- 172.21.15.12:0/2732133828 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5559d235f790 0x5559d235d930 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f286002a430 tx=0x7f2860005e80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c20d1a224c2c4b5d server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:13:48.715 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:48.711+0000 7f285d200640 1 -- 172.21.15.12:0/2732133828 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2860053040 con 0x5559d235f790 2024-09-11T00:13:48.716 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:48.712+0000 7f285d200640 1 -- 172.21.15.12:0/2732133828 <== mon.2 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2860040040 con 0x5559d235f790 2024-09-11T00:13:48.716 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:48.712+0000 7f285d200640 1 -- 172.21.15.12:0/2732133828 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f286003c530 con 0x5559d235f790 2024-09-11T00:13:48.716 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:48.712+0000 7f287628a740 1 -- 172.21.15.12:0/2732133828 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5559d235f790 msgr2=0x5559d235d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:13:48.716 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:48.712+0000 7f287628a740 1 --2- 172.21.15.12:0/2732133828 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5559d235f790 0x5559d235d930 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f286002a430 tx=0x7f2860005e80 comp rx=0 tx=0).stop 2024-09-11T00:13:48.716 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:48.713+0000 7f287628a740 1 -- 172.21.15.12:0/2732133828 shutdown_connections 2024-09-11T00:13:48.716 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:48.713+0000 7f287628a740 1 --2- 172.21.15.12:0/2732133828 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5559d2364b00 0x5559d235b140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:13:48.716 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:48.713+0000 7f287628a740 1 --2- 172.21.15.12:0/2732133828 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5559d23620c0 0x5559d235de70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:13:48.716 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:48.713+0000 7f287628a740 1 --2- 172.21.15.12:0/2732133828 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5559d235f790 0x5559d235d930 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:13:48.716 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:48.713+0000 7f287628a740 1 -- 172.21.15.12:0/2732133828 >> 172.21.15.12:0/2732133828 conn(0x5559d235a8e0 msgr2=0x5559d235ec50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:13:48.716 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:48.713+0000 7f287628a740 1 -- 172.21.15.12:0/2732133828 shutdown_connections 2024-09-11T00:13:48.716 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:48.713+0000 7f287628a740 1 -- 172.21.15.12:0/2732133828 wait complete. 2024-09-11T00:13:48.716 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:48.713+0000 7f287628a740 1 -- start start 2024-09-11T00:13:48.717 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:48.713+0000 7f287628a740 1 Processor -- start 2024-09-11T00:13:48.717 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:48.714+0000 7f287628a740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5559d235f790 0x5559d236fc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:13:48.717 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:48.714+0000 7f287628a740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5559d23620c0 0x5559d2370140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:13:48.717 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:48.714+0000 7f287628a740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5559d2364b00 0x5559d23706a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:13:48.717 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:48.714+0000 7f287628a740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f284c0111f0 con 0x5559d23620c0 2024-09-11T00:13:48.717 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:48.714+0000 7f287628a740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f284c010b20 con 0x5559d2364b00 2024-09-11T00:13:48.717 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:48.714+0000 7f287628a740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f284c00dd00 con 0x5559d235f790 2024-09-11T00:13:48.717 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:48.714+0000 7f287628a740 10 client.0 _ll_register_callbacks cb 0x7f2864009b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:13:48.717 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:48.714+0000 7f2870a00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5559d235f790 0x5559d236fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:13:48.717 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:48.714+0000 7f2871400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5559d2364b00 0x5559d23706a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:13:48.717 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:48.714+0000 7f2870a00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5559d235f790 0x5559d236fc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.12:48656/0 (socket says 172.21.15.12:48656) 2024-09-11T00:13:48.717 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:48.714+0000 7f2870a00640 1 -- 172.21.15.12:0/2403192106 learned_addr learned my addr 172.21.15.12:0/2403192106 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:13:48.717 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:48.714+0000 7f286be00640 1 --2- 172.21.15.12:0/2403192106 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5559d23620c0 0x5559d2370140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:13:48.718 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:48.714+0000 7f2870a00640 1 -- 172.21.15.12:0/2403192106 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5559d2364b00 msgr2=0x5559d23706a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:13:48.718 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:48.714+0000 7f2870a00640 1 --2- 172.21.15.12:0/2403192106 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5559d2364b00 0x5559d23706a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:13:48.718 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:48.714+0000 7f2870a00640 1 -- 172.21.15.12:0/2403192106 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5559d23620c0 msgr2=0x5559d2370140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:13:48.718 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:48.714+0000 7f2870a00640 1 --2- 172.21.15.12:0/2403192106 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5559d23620c0 0x5559d2370140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:13:48.718 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:48.714+0000 7f2870a00640 1 -- 172.21.15.12:0/2403192106 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5559d236ea30 con 0x5559d235f790 2024-09-11T00:13:48.718 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:48.714+0000 7f286be00640 1 --2- 172.21.15.12:0/2403192106 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5559d23620c0 0x5559d2370140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T00:13:48.719 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:48.715+0000 7f2870a00640 1 --2- 172.21.15.12:0/2403192106 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5559d235f790 0x5559d236fc00 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f2860006010 tx=0x7f2860003f50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cde8e6753343e085 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:13:48.719 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:48.715+0000 7f2852000640 10 client.0 ms_handle_connect on v2:172.21.15.178:3300/0 2024-09-11T00:13:48.719 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:48.715+0000 7f2852000640 1 -- 172.21.15.12:0/2403192106 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f286006d060 con 0x5559d235f790 2024-09-11T00:13:48.719 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:48.715+0000 7f287628a740 1 -- 172.21.15.12:0/2403192106 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2860040040 con 0x5559d235f790 2024-09-11T00:13:48.720 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:48.715+0000 7f2852000640 1 -- 172.21.15.12:0/2403192106 <== mon.1 v2:172.21.15.178:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9208+0+0 (secure 0 0 0) 0x7f2860045680 con 0x5559d235f790 2024-09-11T00:13:48.720 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:48.716+0000 7f2852000640 1 -- 172.21.15.12:0/2403192106 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f2860074050 con 0x5559d235f790 2024-09-11T00:13:48.720 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:48.716+0000 7f287628a740 10 client.26260 fetch_fsmap learned FSMap version 39 2024-09-11T00:13:48.720 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:48.716+0000 7f287628a740 1 -- 172.21.15.12:0/2403192106 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x5559d23e5bf0 con 0x5559d235f790 2024-09-11T00:13:48.720 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:48.716+0000 7f2852000640 1 -- 172.21.15.12:0/2403192106 <== mon.1 v2:172.21.15.178:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2860040040 con 0x5559d235f790 2024-09-11T00:13:48.720 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:48.716+0000 7f2852000640 1 -- 172.21.15.12:0/2403192106 <== mon.1 v2:172.21.15.178:3300/0 5 ==== fsmap(e 39) ==== 8378+0+0 (secure 0 0 0) 0x7f2860067070 con 0x5559d235f790 2024-09-11T00:13:48.720 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:48.716+0000 7f287628a740 10 client.26260 fetch_fsmap finished waiting for FSMap version 39 2024-09-11T00:13:48.720 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:48.716+0000 7f287628a740 10 client.26260 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T00:13:48.720 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:48.716+0000 7f287628a740 20 client.26260 populate_metadata read hostname 'smithi012' 2024-09-11T00:13:48.720 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:48.716+0000 7f287628a740 1 --2- 172.21.15.12:0/2403192106 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x5559d23e6350 0x5559d23e8770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:13:48.720 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:48.716+0000 7f287628a740 4 client.26260 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T00:13:48.720 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:48.716+0000 7f287628a740 1 -- 172.21.15.12:0/2403192106 --> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5559d22d00b0 con 0x5559d23e6350 2024-09-11T00:13:48.720 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:48.717+0000 7f286be00640 1 --2- 172.21.15.12:0/2403192106 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x5559d23e6350 0x5559d23e8770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:13:48.721 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:48.717+0000 7f286be00640 1 --2- 172.21.15.12:0/2403192106 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x5559d23e6350 0x5559d23e8770 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=31141129f49629c8 server_cookie=d91d482a3e680afb in_seq=0 out_seq=0 2024-09-11T00:13:48.721 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:48.717+0000 7f2852000640 10 client.26260 ms_handle_connect on v2:172.21.15.165:6838/501312984 2024-09-11T00:13:48.778 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:48.775+0000 7f2852000640 1 -- 172.21.15.12:0/2403192106 <== mds.0 v2:172.21.15.165:6838/501312984 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5559d22d00b0 con 0x5559d23e6350 2024-09-11T00:13:48.778 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:48.775+0000 7f2852000640 10 client.26260 handle_command_reply: tid=0 2024-09-11T00:13:48.778 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:48.775+0000 7f287628a740 1 client.26260 shutdown 2024-09-11T00:13:48.778 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:48.776+0000 7f287628a740 1 -- 172.21.15.12:0/2403192106 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5559d235f790 msgr2=0x5559d236fc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:13:48.778 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:48.776+0000 7f287628a740 1 --2- 172.21.15.12:0/2403192106 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5559d235f790 0x5559d236fc00 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f2860006010 tx=0x7f2860003f50 comp rx=0 tx=0).stop 2024-09-11T00:13:48.778 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:48.776+0000 7f287628a740 1 -- 172.21.15.12:0/2403192106 shutdown_connections 2024-09-11T00:13:48.778 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:48.776+0000 7f287628a740 1 --2- 172.21.15.12:0/2403192106 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x5559d23e6350 0x5559d23e8770 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:13:48.778 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:48.776+0000 7f287628a740 1 --2- 172.21.15.12:0/2403192106 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5559d2364b00 0x5559d23706a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:13:48.778 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:48.776+0000 7f287628a740 1 --2- 172.21.15.12:0/2403192106 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5559d23620c0 0x5559d2370140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:13:48.778 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:48.776+0000 7f2852000640 0 client.26260 ms_handle_reset on v2:172.21.15.165:6838/501312984 2024-09-11T00:13:48.779 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:48.776+0000 7f287628a740 1 --2- 172.21.15.12:0/2403192106 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5559d235f790 0x5559d236fc00 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:13:48.779 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:48.776+0000 7f287628a740 1 -- 172.21.15.12:0/2403192106 >> 172.21.15.12:0/2403192106 conn(0x5559d235a8e0 msgr2=0x5559d235b140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:13:48.779 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:48.776+0000 7f287628a740 1 -- 172.21.15.12:0/2403192106 shutdown_connections 2024-09-11T00:13:48.779 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:48.776+0000 7f287628a740 1 -- 172.21.15.12:0/2403192106 wait complete. 2024-09-11T00:13:48.779 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:48.776+0000 7f287628a740 20 client.26260 trim_cache size 0 max 16384 2024-09-11T00:13:48.779 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:{ 2024-09-11T00:13:48.779 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "message": "", 2024-09-11T00:13:48.779 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "return_code": 0 2024-09-11T00:13:48.779 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:} 2024-09-11T00:13:48.780 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:48.777+0000 7f2873400640 1 -- 172.21.15.12:0/1634858843 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f28540b5280 msgr2=0x7f28540b7740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:13:48.780 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:48.777+0000 7f2873400640 1 --2- 172.21.15.12:0/1634858843 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f28540b5280 0x7f28540b7740 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f286c199af0 tx=0x7f2864009500 comp rx=0 tx=0).stop 2024-09-11T00:13:48.780 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:48.777+0000 7f2873400640 1 -- 172.21.15.12:0/1634858843 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f286c103ae0 msgr2=0x7f286c1993c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:13:48.780 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:48.777+0000 7f2873400640 1 --2- 172.21.15.12:0/1634858843 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f286c103ae0 0x7f286c1993c0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f285800e9f0 tx=0x7f285800eeb0 comp rx=0 tx=0).stop 2024-09-11T00:13:48.781 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:48.777+0000 7f2873400640 1 -- 172.21.15.12:0/1634858843 shutdown_connections 2024-09-11T00:13:48.781 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:48.777+0000 7f2873400640 1 --2- 172.21.15.12:0/1634858843 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f28540b5280 0x7f28540b7740 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:13:48.781 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:48.777+0000 7f2873400640 1 --2- 172.21.15.12:0/1634858843 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f286c1044a0 0x7f286c199ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:13:48.781 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:48.777+0000 7f2873400640 1 --2- 172.21.15.12:0/1634858843 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f286c103ae0 0x7f286c1993c0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:13:48.781 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:48.777+0000 7f2873400640 1 --2- 172.21.15.12:0/1634858843 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f286c102990 0x7f286c198d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:13:48.781 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:48.777+0000 7f2873400640 1 -- 172.21.15.12:0/1634858843 >> 172.21.15.12:0/1634858843 conn(0x7f286c0fe250 msgr2=0x7f286c106f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:13:48.781 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:48.777+0000 7f2873400640 1 -- 172.21.15.12:0/1634858843 shutdown_connections 2024-09-11T00:13:48.781 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:13:48.777+0000 7f2873400640 1 -- 172.21.15.12:0/1634858843 wait complete. 2024-09-11T00:13:49.651 INFO:tasks.workunit.client.0.smithi012.stdout: 1 238860 3.39 MB/sec execute 226 sec latency 168.956 ms 2024-09-11T00:13:50.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:13:49 smithi165 ceph-mon[35356]: pgmap v601: 129 pgs: 129 active+clean; 65 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 17 MiB/s rd, 1.8 MiB/s wr, 279 op/s 2024-09-11T00:13:50.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:13:49 smithi012 ceph-mon[30333]: pgmap v601: 129 pgs: 129 active+clean; 65 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 17 MiB/s rd, 1.8 MiB/s wr, 279 op/s 2024-09-11T00:13:50.332 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 26.8 2024-09-11T00:13:50.333 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid ff3ad74a-6fcf-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T00:13:50.334 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:13:49 smithi178 ceph-mon[35919]: pgmap v601: 129 pgs: 129 active+clean; 65 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 17 MiB/s rd, 1.8 MiB/s wr, 279 op/s 2024-09-11T00:13:50.651 INFO:tasks.workunit.client.0.smithi012.stdout: 1 241195 3.44 MB/sec execute 227 sec latency 13.211 ms 2024-09-11T00:13:50.743 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:13:51.652 INFO:tasks.workunit.client.0.smithi012.stdout: 1 241397 3.43 MB/sec execute 228 sec latency 363.447 ms 2024-09-11T00:13:52.206 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:13:51 smithi012 ceph-mon[30333]: pgmap v602: 129 pgs: 129 active+clean; 65 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 17 MiB/s rd, 2.3 MiB/s wr, 338 op/s 2024-09-11T00:13:52.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:13:51 smithi165 ceph-mon[35356]: pgmap v602: 129 pgs: 129 active+clean; 65 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 17 MiB/s rd, 2.3 MiB/s wr, 338 op/s 2024-09-11T00:13:52.330 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:13:51 smithi178 ceph-mon[35919]: pgmap v602: 129 pgs: 129 active+clean; 65 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 17 MiB/s rd, 2.3 MiB/s wr, 338 op/s 2024-09-11T00:13:52.502 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:13:52.500+0000 7fd43b200640 1 -- 172.21.15.12:0/4035860348 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd434103b10 msgr2=0x7fd434103ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:13:52.502 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:13:52.500+0000 7fd43b200640 1 --2- 172.21.15.12:0/4035860348 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd434103b10 0x7fd434103ef0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fd428009860 tx=0x7fd42802a340 comp rx=0 tx=0).stop 2024-09-11T00:13:52.502 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:13:52.500+0000 7fd43b200640 1 -- 172.21.15.12:0/4035860348 shutdown_connections 2024-09-11T00:13:52.502 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:13:52.500+0000 7fd43b200640 1 --2- 172.21.15.12:0/4035860348 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd43410cf70 0x7fd43410f3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:13:52.503 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:13:52.500+0000 7fd43b200640 1 --2- 172.21.15.12:0/4035860348 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd434104430 0x7fd43410c920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:13:52.503 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:13:52.500+0000 7fd43b200640 1 --2- 172.21.15.12:0/4035860348 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd434103b10 0x7fd434103ef0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:13:52.503 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:13:52.500+0000 7fd43b200640 1 -- 172.21.15.12:0/4035860348 >> 172.21.15.12:0/4035860348 conn(0x7fd4340fd5b0 msgr2=0x7fd4340ff9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:13:52.503 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:13:52.501+0000 7fd43b200640 1 -- 172.21.15.12:0/4035860348 shutdown_connections 2024-09-11T00:13:52.503 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:13:52.501+0000 7fd43b200640 1 -- 172.21.15.12:0/4035860348 wait complete. 2024-09-11T00:13:52.504 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:13:52.502+0000 7fd43b200640 1 Processor -- start 2024-09-11T00:13:52.504 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:13:52.502+0000 7fd43b200640 1 -- start start 2024-09-11T00:13:52.504 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:13:52.502+0000 7fd43b200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd434103b10 0x7fd434199070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:13:52.504 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:13:52.502+0000 7fd43b200640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd434104430 0x7fd4341995b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:13:52.504 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:13:52.502+0000 7fd43b200640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd43410cf70 0x7fd43419e950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:13:52.504 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:13:52.502+0000 7fd43b200640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fd434112be0 con 0x7fd434104430 2024-09-11T00:13:52.504 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:13:52.502+0000 7fd43b200640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fd434112a60 con 0x7fd434103b10 2024-09-11T00:13:52.504 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:13:52.502+0000 7fd43b200640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fd434112d60 con 0x7fd43410cf70 2024-09-11T00:13:52.504 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:13:52.502+0000 7fd439e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd434103b10 0x7fd434199070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:13:52.504 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:13:52.502+0000 7fd439e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd434103b10 0x7fd434199070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.12:34504/0 (socket says 172.21.15.12:34504) 2024-09-11T00:13:52.504 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:13:52.502+0000 7fd439e00640 1 -- 172.21.15.12:0/586349752 learned_addr learned my addr 172.21.15.12:0/586349752 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:13:52.505 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:13:52.503+0000 7fd439400640 1 --2- 172.21.15.12:0/586349752 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd434104430 0x7fd4341995b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:13:52.505 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:13:52.503+0000 7fd43a800640 1 --2- 172.21.15.12:0/586349752 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd43410cf70 0x7fd43419e950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:13:52.505 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:13:52.503+0000 7fd439e00640 1 -- 172.21.15.12:0/586349752 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd43410cf70 msgr2=0x7fd43419e950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:13:52.506 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:13:52.503+0000 7fd439e00640 1 --2- 172.21.15.12:0/586349752 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd43410cf70 0x7fd43419e950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:13:52.506 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:13:52.503+0000 7fd439e00640 1 -- 172.21.15.12:0/586349752 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd434104430 msgr2=0x7fd4341995b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:13:52.506 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:13:52.503+0000 7fd439e00640 1 --2- 172.21.15.12:0/586349752 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd434104430 0x7fd4341995b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:13:52.506 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:13:52.503+0000 7fd439e00640 1 -- 172.21.15.12:0/586349752 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd43419f120 con 0x7fd434103b10 2024-09-11T00:13:52.506 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:13:52.503+0000 7fd439e00640 1 --2- 172.21.15.12:0/586349752 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd434103b10 0x7fd434199070 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fd428035040 tx=0x7fd428032d50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:13:52.506 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:13:52.504+0000 7fd42ea00640 1 -- 172.21.15.12:0/586349752 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd428037040 con 0x7fd434103b10 2024-09-11T00:13:52.506 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:13:52.504+0000 7fd42ea00640 1 -- 172.21.15.12:0/586349752 <== mon.2 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd428041070 con 0x7fd434103b10 2024-09-11T00:13:52.506 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:13:52.504+0000 7fd42ea00640 1 -- 172.21.15.12:0/586349752 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd428003850 con 0x7fd434103b10 2024-09-11T00:13:52.507 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:13:52.504+0000 7fd43b200640 1 -- 172.21.15.12:0/586349752 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd43419f3b0 con 0x7fd434103b10 2024-09-11T00:13:52.507 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:13:52.504+0000 7fd43b200640 1 -- 172.21.15.12:0/586349752 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4341a5ad0 con 0x7fd434103b10 2024-09-11T00:13:52.512 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:13:52.505+0000 7fd43b200640 1 -- 172.21.15.12:0/586349752 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd434103ef0 con 0x7fd434103b10 2024-09-11T00:13:52.512 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:13:52.508+0000 7fd42ea00640 1 -- 172.21.15.12:0/586349752 <== mon.2 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7fd428004050 con 0x7fd434103b10 2024-09-11T00:13:52.512 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:13:52.510+0000 7fd42ea00640 1 --2- 172.21.15.12:0/586349752 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fd41c0b5410 0x7fd41c0b78d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:13:52.512 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:13:52.510+0000 7fd42ea00640 1 -- 172.21.15.12:0/586349752 <== mon.2 v2:172.21.15.165:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9208+0+0 (secure 0 0 0) 0x7fd4281062a0 con 0x7fd434103b10 2024-09-11T00:13:52.512 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:13:52.510+0000 7fd439400640 1 --2- 172.21.15.12:0/586349752 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fd41c0b5410 0x7fd41c0b78d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:13:52.515 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:13:52.511+0000 7fd439400640 1 --2- 172.21.15.12:0/586349752 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fd41c0b5410 0x7fd41c0b78d0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fd43410f630 tx=0x7fd424005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:13:52.515 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:13:52.511+0000 7fd42ea00640 1 -- 172.21.15.12:0/586349752 <== mon.2 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7fd4280ccd30 con 0x7fd434103b10 2024-09-11T00:13:52.652 INFO:tasks.workunit.client.0.smithi012.stdout: 1 241609 3.42 MB/sec execute 229 sec latency 846.130 ms 2024-09-11T00:13:52.774 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:13:52.773+0000 7fd43b200640 1 -- 172.21.15.12:0/586349752 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd4341a5d60 con 0x7fd434103b10 2024-09-11T00:13:52.774 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:13:52.773+0000 7fd42ea00640 1 -- 172.21.15.12:0/586349752 <== mon.2 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 39 v39) ==== 94+0+11968 (secure 0 0 0) 0x7fd4280e5ed0 con 0x7fd434103b10 2024-09-11T00:13:52.775 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-11T00:13:52.775 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":39,"btime":"2024-09-11T00:12:30:781929+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14577,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6835/447171591","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":447171591},{"type":"v1","addr":"172.21.15.12:6835","nonce":447171591}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14589,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/901486040","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":901486040},{"type":"v1","addr":"172.21.15.12:6837","nonce":901486040}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":39,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T00:04:04.739881+0000","modified":"2024-09-11T00:12:30.583169+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24674,"mds_1":24643,"mds_2":24638,"mds_3":24637,"mds_4":24614},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14631":{"gid":14631,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.12:6841/3069919437","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6840","nonce":3069919437},{"type":"v1","addr":"172.21.15.12:6841","nonce":3069919437}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24562":{"gid":24562,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6833/1141636112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":1141636112},{"type":"v1","addr":"172.21.15.178:6833","nonce":1141636112}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24581":{"gid":24581,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6833/3111980970","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6832","nonce":3111980970},{"type":"v1","addr":"172.21.15.165:6833","nonce":3111980970}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24607":{"gid":24607,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6835/4084868287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":4084868287},{"type":"v1","addr":"172.21.15.178:6835","nonce":4084868287}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24614":{"gid":24614,"name":"e","rank":4,"incarnation":26,"state":"up:active","state_seq":21,"addr":"172.21.15.165:6835/2425395274","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":2425395274},{"type":"v1","addr":"172.21.15.165:6835","nonce":2425395274}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24637":{"gid":24637,"name":"i","rank":3,"incarnation":24,"state":"up:active","state_seq":18,"addr":"172.21.15.178:6839/70628097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6838","nonce":70628097},{"type":"v1","addr":"172.21.15.178:6839","nonce":70628097}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24638":{"gid":24638,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":18,"addr":"172.21.15.165:6837/6215088","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6836","nonce":6215088},{"type":"v1","addr":"172.21.15.165:6837","nonce":6215088}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24643":{"gid":24643,"name":"g","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.12:6839/3124093009","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6838","nonce":3124093009},{"type":"v1","addr":"172.21.15.12:6839","nonce":3124093009}]},"join_fscid":-1,"export_targets":[0,4],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24674":{"gid":24674,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.165:6839/501312984","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6838","nonce":501312984},{"type":"v1","addr":"172.21.15.165:6839","nonce":501312984}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25672":{"gid":25672,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6837/308601741","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":308601741},{"type":"v1","addr":"172.21.15.178:6837","nonce":308601741}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24674,"qdb_cluster":[24614,24637,24638,24643,24674]},"id":1}]} 2024-09-11T00:13:52.775 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 39 2024-09-11T00:13:52.777 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:13:52.776+0000 7fd43b200640 1 -- 172.21.15.12:0/586349752 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fd41c0b5410 msgr2=0x7fd41c0b78d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:13:52.777 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:13:52.776+0000 7fd43b200640 1 --2- 172.21.15.12:0/586349752 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fd41c0b5410 0x7fd41c0b78d0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fd43410f630 tx=0x7fd424005f90 comp rx=0 tx=0).stop 2024-09-11T00:13:52.777 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:13:52.776+0000 7fd43b200640 1 -- 172.21.15.12:0/586349752 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd434103b10 msgr2=0x7fd434199070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:13:52.778 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:13:52.776+0000 7fd43b200640 1 --2- 172.21.15.12:0/586349752 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd434103b10 0x7fd434199070 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fd428035040 tx=0x7fd428032d50 comp rx=0 tx=0).stop 2024-09-11T00:13:52.778 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:13:52.776+0000 7fd43b200640 1 -- 172.21.15.12:0/586349752 shutdown_connections 2024-09-11T00:13:52.778 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:13:52.776+0000 7fd43b200640 1 --2- 172.21.15.12:0/586349752 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fd41c0b5410 0x7fd41c0b78d0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:13:52.778 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:13:52.776+0000 7fd43b200640 1 --2- 172.21.15.12:0/586349752 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd43410cf70 0x7fd43419e950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:13:52.778 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:13:52.776+0000 7fd43b200640 1 --2- 172.21.15.12:0/586349752 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd434104430 0x7fd4341995b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:13:52.778 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:13:52.776+0000 7fd43b200640 1 --2- 172.21.15.12:0/586349752 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd434103b10 0x7fd434199070 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:13:52.778 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:13:52.776+0000 7fd43b200640 1 -- 172.21.15.12:0/586349752 >> 172.21.15.12:0/586349752 conn(0x7fd4340fd5b0 msgr2=0x7fd4340fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:13:52.778 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:13:52.776+0000 7fd43b200640 1 -- 172.21.15.12:0/586349752 shutdown_connections 2024-09-11T00:13:52.778 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:13:52.776+0000 7fd43b200640 1 -- 172.21.15.12:0/586349752 wait complete. 2024-09-11T00:13:53.063 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:13:52 smithi012 ceph-mon[30333]: from='client.? 172.21.15.12:0/586349752' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T00:13:53.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:13:52 smithi165 ceph-mon[35356]: from='client.? 172.21.15.12:0/586349752' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T00:13:53.330 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:13:52 smithi178 ceph-mon[35919]: from='client.? 172.21.15.12:0/586349752' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T00:13:53.622 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24674 quiesce db / --timeout 90 --expiration 146.8 --await-for 45.0' 2024-09-11T00:13:53.622 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid ff3ad74a-6fcf-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24674 quiesce db / --timeout 90 --expiration 146.8 --await-for 45.0 2024-09-11T00:13:53.652 INFO:tasks.workunit.client.0.smithi012.stdout: 1 242888 3.43 MB/sec execute 230 sec latency 41.040 ms 2024-09-11T00:13:54.055 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:13:54.229 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:13:53 smithi012 ceph-mon[30333]: pgmap v603: 129 pgs: 129 active+clean; 62 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 20 MiB/s rd, 2.1 MiB/s wr, 337 op/s 2024-09-11T00:13:54.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:13:53 smithi165 ceph-mon[35356]: pgmap v603: 129 pgs: 129 active+clean; 62 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 20 MiB/s rd, 2.1 MiB/s wr, 337 op/s 2024-09-11T00:13:54.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:13:53 smithi178 ceph-mon[35919]: pgmap v603: 129 pgs: 129 active+clean; 62 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 20 MiB/s rd, 2.1 MiB/s wr, 337 op/s 2024-09-11T00:13:54.652 INFO:tasks.workunit.client.0.smithi012.stdout: 1 244767 3.47 MB/sec execute 231 sec latency 170.224 ms 2024-09-11T00:13:54.678 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:13:54.676+0000 7f6758400640 1 -- 172.21.15.12:0/3542609799 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6750075e00 msgr2=0x7f675010ee90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:13:54.678 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:13:54.676+0000 7f6758400640 1 --2- 172.21.15.12:0/3542609799 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6750075e00 0x7f675010ee90 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f673c009900 tx=0x7f673c02a3f0 comp rx=0 tx=0).stop 2024-09-11T00:13:54.678 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:13:54.676+0000 7f6758400640 1 -- 172.21.15.12:0/3542609799 shutdown_connections 2024-09-11T00:13:54.678 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:13:54.676+0000 7f6758400640 1 --2- 172.21.15.12:0/3542609799 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f675010f3d0 0x7f67501117c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:13:54.678 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:13:54.676+0000 7f6758400640 1 --2- 172.21.15.12:0/3542609799 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6750075e00 0x7f675010ee90 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:13:54.678 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:13:54.676+0000 7f6758400640 1 --2- 172.21.15.12:0/3542609799 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f67500754e0 0x7f67500758c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:13:54.678 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:13:54.676+0000 7f6758400640 1 -- 172.21.15.12:0/3542609799 >> 172.21.15.12:0/3542609799 conn(0x7f67500fe830 msgr2=0x7f6750100c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:13:54.678 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:13:54.676+0000 7f6758400640 1 -- 172.21.15.12:0/3542609799 shutdown_connections 2024-09-11T00:13:54.678 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:13:54.676+0000 7f6758400640 1 -- 172.21.15.12:0/3542609799 wait complete. 2024-09-11T00:13:54.679 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:13:54.677+0000 7f6758400640 1 Processor -- start 2024-09-11T00:13:54.679 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:13:54.678+0000 7f6758400640 1 -- start start 2024-09-11T00:13:54.679 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:13:54.678+0000 7f6758400640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f67500754e0 0x7f675019b4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:13:54.679 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:13:54.678+0000 7f6758400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6750075e00 0x7f675019b9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:13:54.679 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:13:54.678+0000 7f6758400640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f675010f3d0 0x7f67501a0d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:13:54.680 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:13:54.678+0000 7f6758400640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f6750114f80 con 0x7f675010f3d0 2024-09-11T00:13:54.680 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:13:54.678+0000 7f6758400640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f6750114e00 con 0x7f6750075e00 2024-09-11T00:13:54.680 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:13:54.678+0000 7f6758400640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f6750115100 con 0x7f67500754e0 2024-09-11T00:13:54.680 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:13:54.678+0000 7f6756400640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f675010f3d0 0x7f67501a0d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:13:54.680 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:13:54.678+0000 7f6756400640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f675010f3d0 0x7f67501a0d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:44532/0 (socket says 172.21.15.12:44532) 2024-09-11T00:13:54.680 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:13:54.678+0000 7f6756400640 1 -- 172.21.15.12:0/2830084817 learned_addr learned my addr 172.21.15.12:0/2830084817 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:13:54.680 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:13:54.678+0000 7f6755000640 1 --2- 172.21.15.12:0/2830084817 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6750075e00 0x7f675019b9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:13:54.681 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:13:54.679+0000 7f6755a00640 1 --2- 172.21.15.12:0/2830084817 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f67500754e0 0x7f675019b4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:13:54.681 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:13:54.679+0000 7f6756400640 1 -- 172.21.15.12:0/2830084817 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f67500754e0 msgr2=0x7f675019b4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:13:54.681 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:13:54.679+0000 7f6756400640 1 --2- 172.21.15.12:0/2830084817 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f67500754e0 0x7f675019b4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:13:54.681 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:13:54.679+0000 7f6756400640 1 -- 172.21.15.12:0/2830084817 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6750075e00 msgr2=0x7f675019b9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:13:54.681 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:13:54.679+0000 7f6756400640 1 --2- 172.21.15.12:0/2830084817 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6750075e00 0x7f675019b9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:13:54.681 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:13:54.679+0000 7f6756400640 1 -- 172.21.15.12:0/2830084817 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67501a1560 con 0x7f675010f3d0 2024-09-11T00:13:54.681 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:13:54.679+0000 7f6755a00640 1 --2- 172.21.15.12:0/2830084817 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f67500754e0 0x7f675019b4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:13:54.681 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:13:54.679+0000 7f6755000640 1 --2- 172.21.15.12:0/2830084817 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6750075e00 0x7f675019b9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:13:54.682 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:13:54.680+0000 7f6756400640 1 --2- 172.21.15.12:0/2830084817 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f675010f3d0 0x7f67501a0d90 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f674800eb40 tx=0x7f674800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:13:54.682 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:13:54.680+0000 7f674e000640 1 -- 172.21.15.12:0/2830084817 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6748004b20 con 0x7f675010f3d0 2024-09-11T00:13:54.682 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:13:54.680+0000 7f674e000640 1 -- 172.21.15.12:0/2830084817 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6748004cc0 con 0x7f675010f3d0 2024-09-11T00:13:54.682 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:13:54.680+0000 7f6758400640 1 -- 172.21.15.12:0/2830084817 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f67501a1850 con 0x7f675010f3d0 2024-09-11T00:13:54.682 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:13:54.680+0000 7f674e000640 1 -- 172.21.15.12:0/2830084817 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6748010720 con 0x7f675010f3d0 2024-09-11T00:13:54.682 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:13:54.680+0000 7f6758400640 1 -- 172.21.15.12:0/2830084817 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67501a7f10 con 0x7f675010f3d0 2024-09-11T00:13:54.686 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:13:54.684+0000 7f674e000640 1 -- 172.21.15.12:0/2830084817 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7f6748005560 con 0x7f675010f3d0 2024-09-11T00:13:54.686 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:13:54.684+0000 7f675adc5740 1 Processor -- start 2024-09-11T00:13:54.686 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:13:54.685+0000 7f675adc5740 1 -- start start 2024-09-11T00:13:54.686 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:13:54.685+0000 7f675adc5740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c8e45ec180 0x55c8e45ec560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:13:54.687 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:13:54.685+0000 7f675adc5740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c8e45ecaa0 0x55c8e45f84f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:13:54.687 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:13:54.685+0000 7f675adc5740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c8e45f8a30 0x55c8e45fae20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:13:54.687 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:13:54.685+0000 7f675adc5740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55c8e4561ca0 con 0x55c8e45ecaa0 2024-09-11T00:13:54.687 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:13:54.685+0000 7f675adc5740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55c8e45510d0 con 0x55c8e45f8a30 2024-09-11T00:13:54.687 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:13:54.685+0000 7f675adc5740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55c8e4554ee0 con 0x55c8e45ec180 2024-09-11T00:13:54.688 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:13:54.686+0000 7f6755a00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c8e45ecaa0 0x55c8e45f84f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:13:54.688 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:13:54.686+0000 7f6755000640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c8e45ec180 0x55c8e45ec560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:13:54.688 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:13:54.686+0000 7f6755a00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c8e45ecaa0 0x55c8e45f84f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:44536/0 (socket says 172.21.15.12:44536) 2024-09-11T00:13:54.688 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:13:54.686+0000 7f6755a00640 1 -- 172.21.15.12:0/2774568615 learned_addr learned my addr 172.21.15.12:0/2774568615 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:13:54.688 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:13:54.686+0000 7f6755000640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c8e45ec180 0x55c8e45ec560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.12:41250/0 (socket says 172.21.15.12:41250) 2024-09-11T00:13:54.688 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:13:54.686+0000 7f6755000640 1 --2- 172.21.15.12:0/2774568615 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c8e45f8a30 0x55c8e45fae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:13:54.689 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:13:54.686+0000 7f6755000640 1 -- 172.21.15.12:0/2774568615 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c8e45f8a30 msgr2=0x55c8e45fae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:13:54.689 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:13:54.686+0000 7f6755000640 1 --2- 172.21.15.12:0/2774568615 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c8e45f8a30 0x55c8e45fae20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:13:54.689 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:13:54.686+0000 7f6755000640 1 -- 172.21.15.12:0/2774568615 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c8e45ecaa0 msgr2=0x55c8e45f84f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:13:54.689 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:13:54.686+0000 7f6755000640 1 --2- 172.21.15.12:0/2774568615 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c8e45ecaa0 0x55c8e45f84f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:13:54.689 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:13:54.686+0000 7f6755000640 1 -- 172.21.15.12:0/2774568615 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c8e4503400 con 0x55c8e45ec180 2024-09-11T00:13:54.689 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:13:54.687+0000 7f6755a00640 1 --2- 172.21.15.12:0/2774568615 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c8e45ecaa0 0x55c8e45f84f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:13:54.689 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:13:54.687+0000 7f674e000640 1 --2- 172.21.15.12:0/2830084817 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f67380b5360 0x7f67380b7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:13:54.689 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:13:54.687+0000 7f6755000640 1 --2- 172.21.15.12:0/2774568615 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c8e45ec180 0x55c8e45ec560 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f673c009230 tx=0x7f673c005780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=81044b3b7dbf8195 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:13:54.690 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:13:54.687+0000 7f674e000640 1 -- 172.21.15.12:0/2830084817 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9208+0+0 (secure 0 0 0) 0x7f67480e66d0 con 0x7f675010f3d0 2024-09-11T00:13:54.690 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:13:54.687+0000 7f6755a00640 1 --2- 172.21.15.12:0/2830084817 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f67380b5360 0x7f67380b7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:13:54.690 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:13:54.687+0000 7f6742a00640 1 -- 172.21.15.12:0/2774568615 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f673c035070 con 0x55c8e45ec180 2024-09-11T00:13:54.690 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:13:54.687+0000 7f6742a00640 1 -- 172.21.15.12:0/2774568615 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f673c004230 con 0x55c8e45ec180 2024-09-11T00:13:54.690 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:13:54.687+0000 7f6742a00640 1 -- 172.21.15.12:0/2774568615 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f673c03d450 con 0x55c8e45ec180 2024-09-11T00:13:54.691 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:13:54.688+0000 7f6755a00640 1 --2- 172.21.15.12:0/2830084817 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f67380b5360 0x7f67380b7820 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x55c8e45eae90 tx=0x7f6744005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:13:54.691 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:13:54.688+0000 7f675adc5740 1 -- 172.21.15.12:0/2774568615 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c8e45ec180 msgr2=0x55c8e45ec560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:13:54.691 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:13:54.688+0000 7f675adc5740 1 --2- 172.21.15.12:0/2774568615 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c8e45ec180 0x55c8e45ec560 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f673c009230 tx=0x7f673c005780 comp rx=0 tx=0).stop 2024-09-11T00:13:54.691 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:13:54.689+0000 7f675adc5740 1 -- 172.21.15.12:0/2774568615 shutdown_connections 2024-09-11T00:13:54.691 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:13:54.689+0000 7f675adc5740 1 --2- 172.21.15.12:0/2774568615 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c8e45f8a30 0x55c8e45fae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:13:54.691 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:13:54.689+0000 7f675adc5740 1 --2- 172.21.15.12:0/2774568615 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c8e45ecaa0 0x55c8e45f84f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:13:54.691 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:13:54.689+0000 7f675adc5740 1 --2- 172.21.15.12:0/2774568615 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c8e45ec180 0x55c8e45ec560 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:13:54.691 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:13:54.689+0000 7f675adc5740 1 -- 172.21.15.12:0/2774568615 >> 172.21.15.12:0/2774568615 conn(0x55c8e45e9fb0 msgr2=0x55c8e45eb150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:13:54.691 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:13:54.689+0000 7f675adc5740 1 -- 172.21.15.12:0/2774568615 shutdown_connections 2024-09-11T00:13:54.691 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:13:54.689+0000 7f675adc5740 1 -- 172.21.15.12:0/2774568615 wait complete. 2024-09-11T00:13:54.692 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:13:54.690+0000 7f675adc5740 1 -- start start 2024-09-11T00:13:54.693 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:13:54.690+0000 7f675adc5740 1 Processor -- start 2024-09-11T00:13:54.693 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:13:54.690+0000 7f675adc5740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c8e45ec180 0x55c8e45f83a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:13:54.693 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:13:54.691+0000 7f675adc5740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c8e45ecaa0 0x55c8e45f57f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:13:54.693 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:13:54.691+0000 7f675adc5740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c8e45f8a30 0x55c8e45f5e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:13:54.693 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:13:54.691+0000 7f675adc5740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55c8e43b27a0 con 0x55c8e45ecaa0 2024-09-11T00:13:54.693 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:13:54.691+0000 7f675adc5740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55c8e44f6a10 con 0x55c8e45ec180 2024-09-11T00:13:54.693 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:13:54.691+0000 7f675adc5740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55c8e45fb650 con 0x55c8e45f8a30 2024-09-11T00:13:54.693 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:13:54.691+0000 7f675adc5740 10 client.0 _ll_register_callbacks cb 0x55c8e4506be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:13:54.693 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:13:54.691+0000 7f6755a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c8e45ec180 0x55c8e45f83a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:13:54.693 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:13:54.691+0000 7f6755a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c8e45ec180 0x55c8e45f83a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.12:34540/0 (socket says 172.21.15.12:34540) 2024-09-11T00:13:54.693 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:13:54.691+0000 7f6755a00640 1 -- 172.21.15.12:0/573122187 learned_addr learned my addr 172.21.15.12:0/573122187 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:13:54.693 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:13:54.691+0000 7f6756400640 1 --2- 172.21.15.12:0/573122187 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c8e45f8a30 0x55c8e45f5e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:13:54.694 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:13:54.691+0000 7f6755000640 1 --2- 172.21.15.12:0/573122187 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c8e45ecaa0 0x55c8e45f57f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:13:54.694 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:13:54.691+0000 7f6755a00640 1 -- 172.21.15.12:0/573122187 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c8e45f8a30 msgr2=0x55c8e45f5e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:13:54.694 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:13:54.691+0000 7f6755a00640 1 --2- 172.21.15.12:0/573122187 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c8e45f8a30 0x55c8e45f5e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:13:54.694 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:13:54.691+0000 7f6755a00640 1 -- 172.21.15.12:0/573122187 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c8e45ecaa0 msgr2=0x55c8e45f57f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:13:54.694 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:13:54.691+0000 7f6755a00640 1 --2- 172.21.15.12:0/573122187 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c8e45ecaa0 0x55c8e45f57f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:13:54.694 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:13:54.691+0000 7f6755a00640 1 -- 172.21.15.12:0/573122187 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c8e42beb90 con 0x55c8e45ec180 2024-09-11T00:13:54.694 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:13:54.691+0000 7f6756400640 1 --2- 172.21.15.12:0/573122187 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c8e45f8a30 0x55c8e45f5e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T00:13:54.694 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:13:54.691+0000 7f6755000640 1 --2- 172.21.15.12:0/573122187 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c8e45ecaa0 0x55c8e45f57f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T00:13:54.694 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:13:54.692+0000 7f6755a00640 1 --2- 172.21.15.12:0/573122187 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c8e45ec180 0x55c8e45f83a0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f6744004620 tx=0x7f6744004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7e8e04c6c1810095 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:13:54.694 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:13:54.692+0000 7f6737400640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-09-11T00:13:54.696 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:13:54.692+0000 7f6737400640 1 -- 172.21.15.12:0/573122187 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6744009c40 con 0x55c8e45ec180 2024-09-11T00:13:54.696 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:13:54.692+0000 7f675adc5740 1 -- 172.21.15.12:0/573122187 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f673c004230 con 0x55c8e45ec180 2024-09-11T00:13:54.696 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:13:54.692+0000 7f6737400640 1 -- 172.21.15.12:0/573122187 <== mon.2 v2:172.21.15.165:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9208+0+0 (secure 0 0 0) 0x7f6744003af0 con 0x55c8e45ec180 2024-09-11T00:13:54.697 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:13:54.692+0000 7f6737400640 1 -- 172.21.15.12:0/573122187 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f6744003d30 con 0x55c8e45ec180 2024-09-11T00:13:54.697 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:13:54.692+0000 7f675adc5740 10 client.26288 fetch_fsmap learned FSMap version 39 2024-09-11T00:13:54.697 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:13:54.692+0000 7f675adc5740 1 -- 172.21.15.12:0/573122187 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55c8e46070b0 con 0x55c8e45ec180 2024-09-11T00:13:54.697 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:13:54.693+0000 7f6737400640 1 -- 172.21.15.12:0/573122187 <== mon.2 v2:172.21.15.165:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f673c004230 con 0x55c8e45ec180 2024-09-11T00:13:54.697 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:13:54.693+0000 7f6737400640 1 -- 172.21.15.12:0/573122187 <== mon.2 v2:172.21.15.165:3300/0 5 ==== fsmap(e 39) ==== 8378+0+0 (secure 0 0 0) 0x7f6744027070 con 0x55c8e45ec180 2024-09-11T00:13:54.697 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:13:54.693+0000 7f675adc5740 10 client.26288 fetch_fsmap finished waiting for FSMap version 39 2024-09-11T00:13:54.697 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:13:54.693+0000 7f675adc5740 10 client.26288 resolve_mds: validated gid 24674 aka daemon mds.b 2024-09-11T00:13:54.697 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:13:54.693+0000 7f675adc5740 20 client.26288 populate_metadata read hostname 'smithi012' 2024-09-11T00:13:54.697 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:13:54.693+0000 7f675adc5740 1 --2- 172.21.15.12:0/573122187 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55c8e4608510 0x55c8e460a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:13:54.697 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:13:54.693+0000 7f675adc5740 4 client.26288 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T00:13:54.697 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:13:54.693+0000 7f675adc5740 1 -- 172.21.15.12:0/573122187 --> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c8e4563bf0 con 0x55c8e4608510 2024-09-11T00:13:54.697 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:13:54.693+0000 7f6755000640 1 --2- 172.21.15.12:0/573122187 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55c8e4608510 0x55c8e460a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:13:54.697 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:13:54.694+0000 7f6755000640 1 --2- 172.21.15.12:0/573122187 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55c8e4608510 0x55c8e460a9b0 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4ab777307b0798c2 server_cookie=c4541d6ecb636750 in_seq=0 out_seq=0 2024-09-11T00:13:54.698 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:13:54.694+0000 7f6737400640 10 client.26288 ms_handle_connect on v2:172.21.15.165:6838/501312984 2024-09-11T00:13:54.698 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:13:54.694+0000 7f6737400640 1 -- 172.21.15.12:0/573122187 <== mds.0 v2:172.21.15.165:6838/501312984 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55c8e4563bf0 con 0x55c8e4608510 2024-09-11T00:13:54.698 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:13:54.694+0000 7f6737400640 10 client.26288 handle_command_reply: tid=0 2024-09-11T00:13:54.698 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:13:54.695+0000 7f675adc5740 1 client.26288 shutdown 2024-09-11T00:13:54.698 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:13:54.695+0000 7f675adc5740 1 -- 172.21.15.12:0/573122187 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c8e45ec180 msgr2=0x55c8e45f83a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:13:54.698 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:13:54.695+0000 7f675adc5740 1 --2- 172.21.15.12:0/573122187 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c8e45ec180 0x55c8e45f83a0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f6744004620 tx=0x7f6744004730 comp rx=0 tx=0).stop 2024-09-11T00:13:54.698 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:13:54.695+0000 7f675adc5740 1 -- 172.21.15.12:0/573122187 shutdown_connections 2024-09-11T00:13:54.698 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:13:54.695+0000 7f675adc5740 1 --2- 172.21.15.12:0/573122187 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55c8e4608510 0x55c8e460a9b0 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:13:54.698 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:13:54.695+0000 7f675adc5740 1 --2- 172.21.15.12:0/573122187 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c8e45f8a30 0x55c8e45f5e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:13:54.698 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:13:54.695+0000 7f675adc5740 1 --2- 172.21.15.12:0/573122187 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c8e45ecaa0 0x55c8e45f57f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:13:54.698 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:13:54.695+0000 7f675adc5740 1 --2- 172.21.15.12:0/573122187 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c8e45ec180 0x55c8e45f83a0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:13:54.698 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:13:54.695+0000 7f6737400640 0 client.26288 ms_handle_reset on v2:172.21.15.165:6838/501312984 2024-09-11T00:13:54.698 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:13:54.695+0000 7f675adc5740 1 -- 172.21.15.12:0/573122187 >> 172.21.15.12:0/573122187 conn(0x55c8e45e9fb0 msgr2=0x55c8e45efb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:13:54.698 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:13:54.695+0000 7f675adc5740 1 -- 172.21.15.12:0/573122187 shutdown_connections 2024-09-11T00:13:54.699 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:13:54.695+0000 7f675adc5740 1 -- 172.21.15.12:0/573122187 wait complete. 2024-09-11T00:13:54.699 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:13:54.695+0000 7f675adc5740 20 client.26288 trim_cache size 0 max 16384 2024-09-11T00:13:54.717 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:13:54.716+0000 7f675adc5740 1 Processor -- start 2024-09-11T00:13:54.717 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:13:54.716+0000 7f675adc5740 1 -- start start 2024-09-11T00:13:54.717 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:13:54.716+0000 7f675adc5740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c8e45ec180 0x55c8e4604880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:13:54.718 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:13:54.716+0000 7f675adc5740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c8e45ecaa0 0x55c8e45f9e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:13:54.718 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:13:54.716+0000 7f675adc5740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c8e45f8a30 0x55c8e45fa360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:13:54.718 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:13:54.716+0000 7f675adc5740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f67501b1e20 con 0x55c8e45ec180 2024-09-11T00:13:54.718 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:13:54.716+0000 7f675adc5740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f67501b39c0 con 0x55c8e45f8a30 2024-09-11T00:13:54.718 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:13:54.716+0000 7f675adc5740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f67501b24f0 con 0x55c8e45ecaa0 2024-09-11T00:13:54.718 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:13:54.716+0000 7f6755000640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c8e45ecaa0 0x55c8e45f9e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:13:54.718 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:13:54.716+0000 7f6755000640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c8e45ecaa0 0x55c8e45f9e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.12:41254/0 (socket says 172.21.15.12:41254) 2024-09-11T00:13:54.718 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:13:54.716+0000 7f6755000640 1 -- 172.21.15.12:0/1022244806 learned_addr learned my addr 172.21.15.12:0/1022244806 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:13:54.718 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:13:54.716+0000 7f6756400640 1 --2- 172.21.15.12:0/1022244806 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c8e45f8a30 0x55c8e45fa360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:13:54.718 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:13:54.716+0000 7f6755a00640 1 --2- 172.21.15.12:0/1022244806 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c8e45ec180 0x55c8e4604880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:13:54.719 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:13:54.717+0000 7f6755000640 1 -- 172.21.15.12:0/1022244806 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c8e45f8a30 msgr2=0x55c8e45fa360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:13:54.719 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:13:54.717+0000 7f6755000640 1 --2- 172.21.15.12:0/1022244806 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c8e45f8a30 0x55c8e45fa360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:13:54.720 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:13:54.717+0000 7f6755000640 1 -- 172.21.15.12:0/1022244806 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c8e45ec180 msgr2=0x55c8e4604880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:13:54.720 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:13:54.717+0000 7f6755000640 1 --2- 172.21.15.12:0/1022244806 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c8e45ec180 0x55c8e4604880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:13:54.720 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:13:54.717+0000 7f6755000640 1 -- 172.21.15.12:0/1022244806 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c8e45f1120 con 0x55c8e45ecaa0 2024-09-11T00:13:54.720 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:13:54.717+0000 7f6755000640 1 --2- 172.21.15.12:0/1022244806 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c8e45ecaa0 0x55c8e45f9e20 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f673c031870 tx=0x7f673c02a3f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=771f882a91ccde38 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:13:54.720 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:13:54.717+0000 7f6742000640 1 -- 172.21.15.12:0/1022244806 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f673c035040 con 0x55c8e45ecaa0 2024-09-11T00:13:54.720 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:13:54.717+0000 7f6742000640 1 -- 172.21.15.12:0/1022244806 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f673c041040 con 0x55c8e45ecaa0 2024-09-11T00:13:54.720 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:13:54.717+0000 7f6742000640 1 -- 172.21.15.12:0/1022244806 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f673c0039f0 con 0x55c8e45ecaa0 2024-09-11T00:13:54.720 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:13:54.718+0000 7f675adc5740 1 -- 172.21.15.12:0/1022244806 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c8e45ecaa0 msgr2=0x55c8e45f9e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:13:54.720 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:13:54.718+0000 7f675adc5740 1 --2- 172.21.15.12:0/1022244806 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c8e45ecaa0 0x55c8e45f9e20 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f673c031870 tx=0x7f673c02a3f0 comp rx=0 tx=0).stop 2024-09-11T00:13:54.720 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:13:54.718+0000 7f675adc5740 1 -- 172.21.15.12:0/1022244806 shutdown_connections 2024-09-11T00:13:54.720 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:13:54.718+0000 7f675adc5740 1 --2- 172.21.15.12:0/1022244806 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c8e45f8a30 0x55c8e45fa360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:13:54.720 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:13:54.718+0000 7f675adc5740 1 --2- 172.21.15.12:0/1022244806 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c8e45ecaa0 0x55c8e45f9e20 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:13:54.721 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:13:54.718+0000 7f675adc5740 1 --2- 172.21.15.12:0/1022244806 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c8e45ec180 0x55c8e4604880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:13:54.721 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:13:54.718+0000 7f675adc5740 1 -- 172.21.15.12:0/1022244806 >> 172.21.15.12:0/1022244806 conn(0x55c8e45e9fb0 msgr2=0x55c8e45f4070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:13:54.721 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:13:54.718+0000 7f675adc5740 1 -- 172.21.15.12:0/1022244806 shutdown_connections 2024-09-11T00:13:54.721 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:13:54.718+0000 7f675adc5740 1 -- 172.21.15.12:0/1022244806 wait complete. 2024-09-11T00:13:54.723 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:13:54.719+0000 7f675adc5740 1 -- start start 2024-09-11T00:13:54.723 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:13:54.719+0000 7f675adc5740 1 Processor -- start 2024-09-11T00:13:54.723 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:13:54.720+0000 7f675adc5740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c8e45ec180 0x55c8e4601090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:13:54.723 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:13:54.720+0000 7f675adc5740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c8e45ecaa0 0x55c8e45fb8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:13:54.723 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:13:54.720+0000 7f675adc5740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c8e45f8a30 0x55c8e45fbde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:13:54.724 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:13:54.720+0000 7f675adc5740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f67501b32f0 con 0x55c8e45ecaa0 2024-09-11T00:13:54.724 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:13:54.720+0000 7f675adc5740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f67501b2c20 con 0x55c8e45f8a30 2024-09-11T00:13:54.724 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:13:54.720+0000 7f675adc5740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f67501afe00 con 0x55c8e45ec180 2024-09-11T00:13:54.724 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:13:54.720+0000 7f675adc5740 10 client.0 _ll_register_callbacks cb 0x7f6744012ed0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:13:54.724 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:13:54.720+0000 7f6755a00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c8e45ec180 0x55c8e4601090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:13:54.724 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:13:54.720+0000 7f6755a00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c8e45ec180 0x55c8e4601090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.12:41256/0 (socket says 172.21.15.12:41256) 2024-09-11T00:13:54.724 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:13:54.720+0000 7f6755000640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c8e45ecaa0 0x55c8e45fb8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:13:54.724 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:13:54.720+0000 7f6755a00640 1 -- 172.21.15.12:0/2575651819 learned_addr learned my addr 172.21.15.12:0/2575651819 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:13:54.724 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:13:54.720+0000 7f6756400640 1 --2- 172.21.15.12:0/2575651819 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c8e45f8a30 0x55c8e45fbde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:13:54.724 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:13:54.720+0000 7f6756400640 1 -- 172.21.15.12:0/2575651819 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c8e45ec180 msgr2=0x55c8e4601090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:13:54.724 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:13:54.720+0000 7f6756400640 1 --2- 172.21.15.12:0/2575651819 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c8e45ec180 0x55c8e4601090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:13:54.725 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:13:54.720+0000 7f6756400640 1 -- 172.21.15.12:0/2575651819 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c8e45ecaa0 msgr2=0x55c8e45fb8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:13:54.725 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:13:54.720+0000 7f6756400640 1 --2- 172.21.15.12:0/2575651819 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c8e45ecaa0 0x55c8e45fb8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:13:54.725 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:13:54.720+0000 7f6756400640 1 -- 172.21.15.12:0/2575651819 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c8e45fc610 con 0x55c8e45f8a30 2024-09-11T00:13:54.725 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:13:54.720+0000 7f6755a00640 1 --2- 172.21.15.12:0/2575651819 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c8e45ec180 0x55c8e4601090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:13:54.725 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:13:54.721+0000 7f6756400640 1 --2- 172.21.15.12:0/2575651819 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c8e45f8a30 0x55c8e45fbde0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f67480e3e00 tx=0x7f67480a8230 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c3bfeb464ae22d83 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:13:54.725 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:13:54.721+0000 7f6737400640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-09-11T00:13:54.725 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:13:54.721+0000 7f6737400640 1 -- 172.21.15.12:0/2575651819 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67480d3070 con 0x55c8e45f8a30 2024-09-11T00:13:54.725 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:13:54.721+0000 7f675adc5740 1 -- 172.21.15.12:0/2575651819 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f673c041040 con 0x55c8e45f8a30 2024-09-11T00:13:54.725 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:13:54.721+0000 7f6737400640 1 -- 172.21.15.12:0/2575651819 <== mon.2 v2:172.21.15.165:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9208+0+0 (secure 0 0 0) 0x7f67480b5bc0 con 0x55c8e45f8a30 2024-09-11T00:13:54.725 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:13:54.721+0000 7f6737400640 1 -- 172.21.15.12:0/2575651819 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f67480acac0 con 0x55c8e45f8a30 2024-09-11T00:13:54.725 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:13:54.721+0000 7f675adc5740 10 client.26294 fetch_fsmap learned FSMap version 39 2024-09-11T00:13:54.725 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:13:54.721+0000 7f675adc5740 1 -- 172.21.15.12:0/2575651819 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55c8e46765b0 con 0x55c8e45f8a30 2024-09-11T00:13:54.725 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:13:54.722+0000 7f6737400640 1 -- 172.21.15.12:0/2575651819 <== mon.2 v2:172.21.15.165:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f673c041040 con 0x55c8e45f8a30 2024-09-11T00:13:54.725 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:13:54.722+0000 7f6737400640 1 -- 172.21.15.12:0/2575651819 <== mon.2 v2:172.21.15.165:3300/0 5 ==== fsmap(e 39) ==== 8378+0+0 (secure 0 0 0) 0x7f67480a8410 con 0x55c8e45f8a30 2024-09-11T00:13:54.726 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:13:54.722+0000 7f675adc5740 10 client.26294 fetch_fsmap finished waiting for FSMap version 39 2024-09-11T00:13:54.726 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:13:54.722+0000 7f675adc5740 10 client.26294 resolve_mds: validated gid 24674 aka daemon mds.b 2024-09-11T00:13:54.726 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:13:54.722+0000 7f675adc5740 20 client.26294 populate_metadata read hostname 'smithi012' 2024-09-11T00:13:54.726 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:13:54.722+0000 7f675adc5740 1 --2- 172.21.15.12:0/2575651819 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55c8e4676ca0 0x55c8e4679090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:13:54.726 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:13:54.722+0000 7f675adc5740 4 client.26294 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 146.8, "await_for": 45.0}] 2024-09-11T00:13:54.726 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:13:54.722+0000 7f675adc5740 1 -- 172.21.15.12:0/2575651819 --> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 146.8, "await_for": 45.0}) -- 0x55c8e455f6a0 con 0x55c8e4676ca0 2024-09-11T00:13:54.726 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:13:54.722+0000 7f6755a00640 1 --2- 172.21.15.12:0/2575651819 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55c8e4676ca0 0x55c8e4679090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:13:54.726 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:13:54.723+0000 7f6755a00640 1 --2- 172.21.15.12:0/2575651819 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55c8e4676ca0 0x55c8e4679090 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9e3d34958f9c6338 server_cookie=60ffe05c79c79a55 in_seq=0 out_seq=0 2024-09-11T00:13:54.726 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:13:54.723+0000 7f6737400640 10 client.26294 ms_handle_connect on v2:172.21.15.165:6838/501312984 2024-09-11T00:13:54.957 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:13:54.955+0000 7f6737400640 1 -- 172.21.15.12:0/2575651819 <== mds.0 v2:172.21.15.165:6838/501312984 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55c8e455f6a0 con 0x55c8e4676ca0 2024-09-11T00:13:54.957 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:13:54.955+0000 7f6737400640 10 client.26294 handle_command_reply: tid=0 2024-09-11T00:13:54.957 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:13:54.955+0000 7f675adc5740 1 client.26294 shutdown 2024-09-11T00:13:54.957 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:13:54.955+0000 7f675adc5740 1 -- 172.21.15.12:0/2575651819 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c8e45f8a30 msgr2=0x55c8e45fbde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:13:54.957 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:13:54.955+0000 7f675adc5740 1 --2- 172.21.15.12:0/2575651819 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c8e45f8a30 0x55c8e45fbde0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f67480e3e00 tx=0x7f67480a8230 comp rx=0 tx=0).stop 2024-09-11T00:13:54.957 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:13:54.955+0000 7f675adc5740 1 -- 172.21.15.12:0/2575651819 shutdown_connections 2024-09-11T00:13:54.957 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:13:54.955+0000 7f675adc5740 1 --2- 172.21.15.12:0/2575651819 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55c8e4676ca0 0x55c8e4679090 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:13:54.957 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:13:54.955+0000 7f675adc5740 1 --2- 172.21.15.12:0/2575651819 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c8e45f8a30 0x55c8e45fbde0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:13:54.957 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:13:54.955+0000 7f675adc5740 1 --2- 172.21.15.12:0/2575651819 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c8e45ecaa0 0x55c8e45fb8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:13:54.957 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:13:54.955+0000 7f675adc5740 1 --2- 172.21.15.12:0/2575651819 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c8e45ec180 0x55c8e4601090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:13:54.958 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:13:54.955+0000 7f6737400640 0 client.26294 ms_handle_reset on v2:172.21.15.165:6838/501312984 2024-09-11T00:13:54.958 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:13:54.955+0000 7f675adc5740 1 -- 172.21.15.12:0/2575651819 >> 172.21.15.12:0/2575651819 conn(0x55c8e45e9fb0 msgr2=0x55c8e45f6e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:13:54.958 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:13:54.955+0000 7f675adc5740 1 -- 172.21.15.12:0/2575651819 shutdown_connections 2024-09-11T00:13:54.958 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:13:54.955+0000 7f675adc5740 1 -- 172.21.15.12:0/2575651819 wait complete. 2024-09-11T00:13:54.958 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:13:54.955+0000 7f675adc5740 20 client.26294 trim_cache size 0 max 16384 2024-09-11T00:13:54.958 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-09-11T00:13:54.958 INFO:teuthology.orchestra.run.smithi012.stdout: "epoch": 39, 2024-09-11T00:13:54.958 INFO:teuthology.orchestra.run.smithi012.stdout: "leader": 24674, 2024-09-11T00:13:54.959 INFO:teuthology.orchestra.run.smithi012.stdout: "set_version": 22, 2024-09-11T00:13:54.959 INFO:teuthology.orchestra.run.smithi012.stdout: "sets": { 2024-09-11T00:13:54.959 INFO:teuthology.orchestra.run.smithi012.stdout: "dd3a477f": { 2024-09-11T00:13:54.959 INFO:teuthology.orchestra.run.smithi012.stdout: "version": 22, 2024-09-11T00:13:54.959 INFO:teuthology.orchestra.run.smithi012.stdout: "age_ref": 0.0, 2024-09-11T00:13:54.959 INFO:teuthology.orchestra.run.smithi012.stdout: "state": { 2024-09-11T00:13:54.959 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "QUIESCED", 2024-09-11T00:13:54.959 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 0.0 2024-09-11T00:13:54.959 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:13:54.959 INFO:teuthology.orchestra.run.smithi012.stdout: "timeout": 90.0, 2024-09-11T00:13:54.959 INFO:teuthology.orchestra.run.smithi012.stdout: "expiration": 146.8, 2024-09-11T00:13:54.959 INFO:teuthology.orchestra.run.smithi012.stdout: "members": { 2024-09-11T00:13:54.959 INFO:teuthology.orchestra.run.smithi012.stdout: "file:/": { 2024-09-11T00:13:54.960 INFO:teuthology.orchestra.run.smithi012.stdout: "excluded": false, 2024-09-11T00:13:54.960 INFO:teuthology.orchestra.run.smithi012.stdout: "state": { 2024-09-11T00:13:54.960 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "QUIESCED", 2024-09-11T00:13:54.960 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 0.0 2024-09-11T00:13:54.960 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:13:54.960 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:13:54.960 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:13:54.960 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:13:54.960 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:13:54.960 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-09-11T00:13:54.961 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:13:54.956+0000 7f6758400640 1 -- 172.21.15.12:0/2830084817 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f67380b5360 msgr2=0x7f67380b7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:13:54.961 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:13:54.956+0000 7f6758400640 1 --2- 172.21.15.12:0/2830084817 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f67380b5360 0x7f67380b7820 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x55c8e45eae90 tx=0x7f6744005f90 comp rx=0 tx=0).stop 2024-09-11T00:13:54.961 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:13:54.956+0000 7f6758400640 1 -- 172.21.15.12:0/2830084817 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f675010f3d0 msgr2=0x7f67501a0d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:13:54.961 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:13:54.956+0000 7f6758400640 1 --2- 172.21.15.12:0/2830084817 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f675010f3d0 0x7f67501a0d90 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f674800eb40 tx=0x7f674800c2d0 comp rx=0 tx=0).stop 2024-09-11T00:13:54.961 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:13:54.957+0000 7f6758400640 1 -- 172.21.15.12:0/2830084817 shutdown_connections 2024-09-11T00:13:54.962 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:13:54.957+0000 7f6758400640 1 --2- 172.21.15.12:0/2830084817 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f67380b5360 0x7f67380b7820 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:13:54.962 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:13:54.957+0000 7f6758400640 1 --2- 172.21.15.12:0/2830084817 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f675010f3d0 0x7f67501a0d90 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:13:54.962 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:13:54.957+0000 7f6758400640 1 --2- 172.21.15.12:0/2830084817 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6750075e00 0x7f675019b9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:13:54.962 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:13:54.957+0000 7f6758400640 1 --2- 172.21.15.12:0/2830084817 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f67500754e0 0x7f675019b4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:13:54.962 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:13:54.957+0000 7f6758400640 1 -- 172.21.15.12:0/2830084817 >> 172.21.15.12:0/2830084817 conn(0x7f67500fe830 msgr2=0x7f67500ff430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:13:54.962 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:13:54.957+0000 7f6758400640 1 -- 172.21.15.12:0/2830084817 shutdown_connections 2024-09-11T00:13:54.962 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:13:54.957+0000 7f6758400640 1 -- 172.21.15.12:0/2830084817 wait complete. 2024-09-11T00:13:55.652 INFO:tasks.workunit.client.0.smithi012.stdout: 1 244855 3.45 MB/sec execute 232 sec latency 861.689 ms 2024-09-11T00:13:55.807 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'dd3a477f', quiesce took 5.5 seconds. Will release after: 21.3 2024-09-11T00:13:56.063 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:13:55 smithi012 ceph-mon[30333]: pgmap v604: 129 pgs: 129 active+clean; 62 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 1.7 MiB/s wr, 263 op/s 2024-09-11T00:13:56.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:13:55 smithi165 ceph-mon[35356]: pgmap v604: 129 pgs: 129 active+clean; 62 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 1.7 MiB/s wr, 263 op/s 2024-09-11T00:13:56.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:13:55 smithi178 ceph-mon[35919]: pgmap v604: 129 pgs: 129 active+clean; 62 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 1.7 MiB/s wr, 263 op/s 2024-09-11T00:13:56.652 INFO:tasks.workunit.client.0.smithi012.stdout: 1 244855 3.44 MB/sec execute 233 sec latency 1861.748 ms 2024-09-11T00:13:57.652 INFO:tasks.workunit.client.0.smithi012.stdout: 1 244855 3.42 MB/sec execute 234 sec latency 2861.863 ms 2024-09-11T00:13:58.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:13:57 smithi165 ceph-mon[35356]: pgmap v605: 129 pgs: 129 active+clean; 64 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 21 MiB/s rd, 2.2 MiB/s wr, 358 op/s 2024-09-11T00:13:58.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:13:57 smithi012 ceph-mon[30333]: pgmap v605: 129 pgs: 129 active+clean; 64 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 21 MiB/s rd, 2.2 MiB/s wr, 358 op/s 2024-09-11T00:13:58.330 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:13:57 smithi178 ceph-mon[35919]: pgmap v605: 129 pgs: 129 active+clean; 64 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 21 MiB/s rd, 2.2 MiB/s wr, 358 op/s 2024-09-11T00:13:58.652 INFO:tasks.workunit.client.0.smithi012.stdout: 1 244855 3.41 MB/sec execute 235 sec latency 3862.006 ms 2024-09-11T00:13:59.653 INFO:tasks.workunit.client.0.smithi012.stdout: 1 244855 3.39 MB/sec execute 236 sec latency 4862.149 ms 2024-09-11T00:14:00.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:13:59 smithi165 ceph-mon[35356]: pgmap v606: 129 pgs: 129 active+clean; 64 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 1.6 MiB/s wr, 255 op/s 2024-09-11T00:14:00.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:13:59 smithi012 ceph-mon[30333]: pgmap v606: 129 pgs: 129 active+clean; 64 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 1.6 MiB/s wr, 255 op/s 2024-09-11T00:14:00.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:13:59 smithi178 ceph-mon[35919]: pgmap v606: 129 pgs: 129 active+clean; 64 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 1.6 MiB/s wr, 255 op/s 2024-09-11T00:14:00.653 INFO:tasks.workunit.client.0.smithi012.stdout: 1 244855 3.38 MB/sec execute 237 sec latency 5862.240 ms 2024-09-11T00:14:01.653 INFO:tasks.workunit.client.0.smithi012.stdout: 1 244855 3.37 MB/sec execute 238 sec latency 6862.330 ms 2024-09-11T00:14:01.896 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:01.894+0000 7f81fe800640 1 -- 172.21.15.12:0/3094582688 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f81f81066a0 msgr2=0x7f81f810d020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:14:01.897 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:01.894+0000 7f81fe800640 1 --2- 172.21.15.12:0/3094582688 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f81f81066a0 0x7f81f810d020 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f81f0009cd0 tx=0x7f81f002a910 comp rx=0 tx=0).stop 2024-09-11T00:14:01.897 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:01.894+0000 7f81fe800640 1 -- 172.21.15.12:0/3094582688 shutdown_connections 2024-09-11T00:14:01.897 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:01.894+0000 7f81fe800640 1 --2- 172.21.15.12:0/3094582688 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f81f81066a0 0x7f81f810d020 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:14:01.897 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:01.894+0000 7f81fe800640 1 --2- 172.21.15.12:0/3094582688 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f81f8105ce0 0x7f81f8106160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:14:01.897 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:01.894+0000 7f81fe800640 1 --2- 172.21.15.12:0/3094582688 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f81f8104b90 0x7f81f8104f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:14:01.897 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:01.894+0000 7f81fe800640 1 -- 172.21.15.12:0/3094582688 >> 172.21.15.12:0/3094582688 conn(0x7f81f8100470 msgr2=0x7f81f8102890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:14:01.897 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:01.894+0000 7f81fe800640 1 -- 172.21.15.12:0/3094582688 shutdown_connections 2024-09-11T00:14:01.897 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:01.894+0000 7f81fe800640 1 -- 172.21.15.12:0/3094582688 wait complete. 2024-09-11T00:14:01.899 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:01.895+0000 7f81fe800640 1 Processor -- start 2024-09-11T00:14:01.899 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:01.896+0000 7f81fe800640 1 -- start start 2024-09-11T00:14:01.899 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:01.896+0000 7f81fe800640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f81f8104b90 0x7f81f819afc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:14:01.899 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:01.896+0000 7f81fe800640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f81f8105ce0 0x7f81f819b610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:14:01.900 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:01.896+0000 7f81fe800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f81f81066a0 0x7f81f819c240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:14:01.900 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:01.896+0000 7f81fe800640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f81f81108a0 con 0x7f81f8105ce0 2024-09-11T00:14:01.900 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:01.896+0000 7f81fe800640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f81f8110720 con 0x7f81f81066a0 2024-09-11T00:14:01.900 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:01.896+0000 7f81fe800640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f81f8110a20 con 0x7f81f8104b90 2024-09-11T00:14:01.900 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:01.896+0000 7f81f6a00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f81f8105ce0 0x7f81f819b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:14:01.900 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:01.896+0000 7f81f6a00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f81f8105ce0 0x7f81f819b610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:33056/0 (socket says 172.21.15.12:33056) 2024-09-11T00:14:01.900 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:01.896+0000 7f81f7400640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f81f8104b90 0x7f81f819afc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:14:01.900 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:01.896+0000 7f81f6a00640 1 -- 172.21.15.12:0/2105756777 learned_addr learned my addr 172.21.15.12:0/2105756777 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:14:01.900 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:01.896+0000 7f81f7e00640 1 --2- 172.21.15.12:0/2105756777 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f81f81066a0 0x7f81f819c240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:14:01.900 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:01.897+0000 7f81f6a00640 1 -- 172.21.15.12:0/2105756777 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f81f8104b90 msgr2=0x7f81f819afc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:14:01.900 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:01.897+0000 7f81f6a00640 1 --2- 172.21.15.12:0/2105756777 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f81f8104b90 0x7f81f819afc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:14:01.900 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:01.897+0000 7f81f6a00640 1 -- 172.21.15.12:0/2105756777 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f81f81066a0 msgr2=0x7f81f819c240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:14:01.900 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:01.897+0000 7f81f6a00640 1 --2- 172.21.15.12:0/2105756777 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f81f81066a0 0x7f81f819c240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:14:01.901 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:01.897+0000 7f81f6a00640 1 -- 172.21.15.12:0/2105756777 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81f819cb00 con 0x7f81f8105ce0 2024-09-11T00:14:01.901 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:01.897+0000 7f81f7400640 1 --2- 172.21.15.12:0/2105756777 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f81f8104b90 0x7f81f819afc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:14:01.901 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:01.897+0000 7f81f7e00640 1 --2- 172.21.15.12:0/2105756777 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f81f81066a0 0x7f81f819c240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T00:14:01.901 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:01.897+0000 7f81f6a00640 1 --2- 172.21.15.12:0/2105756777 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f81f8105ce0 0x7f81f819b610 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f81e400b450 tx=0x7f81e400b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:14:01.901 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:01.897+0000 7f81efe00640 1 -- 172.21.15.12:0/2105756777 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81e4015020 con 0x7f81f8105ce0 2024-09-11T00:14:01.901 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:01.897+0000 7f81efe00640 1 -- 172.21.15.12:0/2105756777 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f81e4011d40 con 0x7f81f8105ce0 2024-09-11T00:14:01.901 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:01.897+0000 7f81fe800640 1 -- 172.21.15.12:0/2105756777 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f81f819cdf0 con 0x7f81f8105ce0 2024-09-11T00:14:01.901 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:01.897+0000 7f81efe00640 1 -- 172.21.15.12:0/2105756777 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81e400f340 con 0x7f81f8105ce0 2024-09-11T00:14:01.901 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:01.897+0000 7f81fe800640 1 -- 172.21.15.12:0/2105756777 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81f81a34c0 con 0x7f81f8105ce0 2024-09-11T00:14:01.902 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:01.900+0000 7f8201634740 1 Processor -- start 2024-09-11T00:14:01.902 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:01.901+0000 7f8201634740 1 -- start start 2024-09-11T00:14:01.904 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:01.901+0000 7f81efe00640 1 -- 172.21.15.12:0/2105756777 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7f81e4011890 con 0x7f81f8105ce0 2024-09-11T00:14:01.904 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:01.901+0000 7f8201634740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564142177790 0x564142179b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:14:01.904 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:01.901+0000 7f8201634740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56414217a0c0 0x56414217c4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:14:01.904 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:01.901+0000 7f8201634740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56414217cb00 0x56414217efc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:14:01.904 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:01.901+0000 7f8201634740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x5641420ea6b0 con 0x56414217a0c0 2024-09-11T00:14:01.905 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:01.901+0000 7f8201634740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x5641420d9ae0 con 0x564142177790 2024-09-11T00:14:01.905 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:01.901+0000 7f8201634740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5641420dd8f0 con 0x56414217cb00 2024-09-11T00:14:01.905 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:01.901+0000 7f81f7400640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56414217a0c0 0x56414217c4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:14:01.905 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:01.901+0000 7f81f7e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564142177790 0x564142179b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:14:01.905 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:01.901+0000 7f81f7400640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56414217a0c0 0x56414217c4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:33070/0 (socket says 172.21.15.12:33070) 2024-09-11T00:14:01.905 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:01.901+0000 7f81f7400640 1 -- 172.21.15.12:0/1425428459 learned_addr learned my addr 172.21.15.12:0/1425428459 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:14:01.905 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:01.901+0000 7f81f6a00640 1 --2- 172.21.15.12:0/1425428459 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56414217cb00 0x56414217efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:14:01.905 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:01.902+0000 7f81f7400640 1 -- 172.21.15.12:0/1425428459 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56414217cb00 msgr2=0x56414217efc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:14:01.905 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:01.902+0000 7f81f7400640 1 --2- 172.21.15.12:0/1425428459 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56414217cb00 0x56414217efc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:14:01.906 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:01.902+0000 7f81f7400640 1 -- 172.21.15.12:0/1425428459 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564142177790 msgr2=0x564142179b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:14:01.906 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:01.902+0000 7f81f7400640 1 --2- 172.21.15.12:0/1425428459 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564142177790 0x564142179b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:14:01.906 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:01.902+0000 7f81f7400640 1 -- 172.21.15.12:0/1425428459 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5641420a6780 con 0x56414217a0c0 2024-09-11T00:14:01.906 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:01.902+0000 7f81f7400640 1 --2- 172.21.15.12:0/1425428459 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56414217a0c0 0x56414217c4b0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f81e80115c0 tx=0x7f81e8011a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1c0849625827d00 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:14:01.906 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:01.902+0000 7f81ecc00640 1 -- 172.21.15.12:0/1425428459 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81e800f640 con 0x56414217a0c0 2024-09-11T00:14:01.906 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:01.902+0000 7f81ecc00640 1 -- 172.21.15.12:0/1425428459 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f81e800fc90 con 0x56414217a0c0 2024-09-11T00:14:01.906 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:01.902+0000 7f81ecc00640 1 -- 172.21.15.12:0/1425428459 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81e8019c60 con 0x56414217a0c0 2024-09-11T00:14:01.908 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:01.903+0000 7f81efe00640 1 --2- 172.21.15.12:0/2105756777 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f81dc0b5360 0x7f81dc0b7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:14:01.908 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:01.903+0000 7f8201634740 1 -- 172.21.15.12:0/1425428459 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56414217a0c0 msgr2=0x56414217c4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:14:01.908 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:01.903+0000 7f8201634740 1 --2- 172.21.15.12:0/1425428459 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56414217a0c0 0x56414217c4b0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f81e80115c0 tx=0x7f81e8011a80 comp rx=0 tx=0).stop 2024-09-11T00:14:01.908 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:01.903+0000 7f81efe00640 1 -- 172.21.15.12:0/2105756777 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9208+0+0 (secure 0 0 0) 0x7f81e40e5c60 con 0x7f81f8105ce0 2024-09-11T00:14:01.908 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:01.903+0000 7f81f7e00640 1 --2- 172.21.15.12:0/2105756777 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f81dc0b5360 0x7f81dc0b7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:14:01.909 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:01.903+0000 7f8201634740 1 -- 172.21.15.12:0/1425428459 shutdown_connections 2024-09-11T00:14:01.909 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:01.903+0000 7f8201634740 1 --2- 172.21.15.12:0/1425428459 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56414217cb00 0x56414217efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:14:01.909 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:01.903+0000 7f8201634740 1 --2- 172.21.15.12:0/1425428459 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56414217a0c0 0x56414217c4b0 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:14:01.909 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:01.903+0000 7f8201634740 1 --2- 172.21.15.12:0/1425428459 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564142177790 0x564142179b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:14:01.909 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:01.903+0000 7f8201634740 1 -- 172.21.15.12:0/1425428459 >> 172.21.15.12:0/1425428459 conn(0x5641421728e0 msgr2=0x564142173670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:14:01.909 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:01.903+0000 7f8201634740 1 -- 172.21.15.12:0/1425428459 shutdown_connections 2024-09-11T00:14:01.909 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:01.903+0000 7f8201634740 1 -- 172.21.15.12:0/1425428459 wait complete. 2024-09-11T00:14:01.909 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:01.904+0000 7f8201634740 1 -- start start 2024-09-11T00:14:01.909 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:01.904+0000 7f81f7e00640 1 --2- 172.21.15.12:0/2105756777 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f81dc0b5360 0x7f81dc0b7820 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f81f819bd40 tx=0x7f81f0031b20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:14:01.909 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:01.904+0000 7f8201634740 1 Processor -- start 2024-09-11T00:14:01.909 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:01.904+0000 7f8201634740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564142177790 0x564142180e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:14:01.909 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:01.904+0000 7f8201634740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56414217a0c0 0x564142181530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:14:01.910 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:01.905+0000 7f8201634740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56414217cb00 0x564142182100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:14:01.910 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:01.905+0000 7f8201634740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x564142011ab0 con 0x56414217a0c0 2024-09-11T00:14:01.910 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:01.905+0000 7f8201634740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x564142080570 con 0x56414217cb00 2024-09-11T00:14:01.910 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:01.905+0000 7f8201634740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x56414217fae0 con 0x564142177790 2024-09-11T00:14:01.910 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:01.905+0000 7f8201634740 10 client.0 _ll_register_callbacks cb 0x56414208f100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:14:01.910 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:01.905+0000 7f81f6a00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56414217a0c0 0x564142181530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:14:01.910 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:01.905+0000 7f81f7400640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564142177790 0x564142180e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:14:01.910 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:01.905+0000 7f81f7400640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564142177790 0x564142180e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.12:58698/0 (socket says 172.21.15.12:58698) 2024-09-11T00:14:01.910 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:01.905+0000 7f81f7400640 1 -- 172.21.15.12:0/966776098 learned_addr learned my addr 172.21.15.12:0/966776098 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:14:01.910 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:01.905+0000 7f81f7e00640 1 --2- 172.21.15.12:0/966776098 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56414217cb00 0x564142182100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:14:01.910 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:01.905+0000 7f81f7400640 1 -- 172.21.15.12:0/966776098 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56414217cb00 msgr2=0x564142182100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:14:01.910 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:01.905+0000 7f81f7400640 1 --2- 172.21.15.12:0/966776098 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56414217cb00 0x564142182100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:14:01.910 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:01.905+0000 7f81f7400640 1 -- 172.21.15.12:0/966776098 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56414217a0c0 msgr2=0x564142181530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:14:01.910 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:01.905+0000 7f81f7400640 1 --2- 172.21.15.12:0/966776098 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56414217a0c0 0x564142181530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:14:01.913 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:01.905+0000 7f81f7400640 1 -- 172.21.15.12:0/966776098 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564141e3b790 con 0x564142177790 2024-09-11T00:14:01.913 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:01.906+0000 7f81f7400640 1 --2- 172.21.15.12:0/966776098 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564142177790 0x564142180e90 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f81e800a010 tx=0x7f81e8004270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=20b8b0af77cd4472 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:14:01.913 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:01.906+0000 7f81e1600640 10 client.0 ms_handle_connect on v2:172.21.15.178:3300/0 2024-09-11T00:14:01.913 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:01.906+0000 7f81e1600640 1 -- 172.21.15.12:0/966776098 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81e8026020 con 0x564142177790 2024-09-11T00:14:01.913 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:01.906+0000 7f8201634740 1 -- 172.21.15.12:0/966776098 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f81e800fc90 con 0x564142177790 2024-09-11T00:14:01.913 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:01.906+0000 7f81e1600640 1 -- 172.21.15.12:0/966776098 <== mon.1 v2:172.21.15.178:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9208+0+0 (secure 0 0 0) 0x7f81e800fe40 con 0x564142177790 2024-09-11T00:14:01.913 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:01.906+0000 7f81e1600640 1 -- 172.21.15.12:0/966776098 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f81e8031050 con 0x564142177790 2024-09-11T00:14:01.913 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:01.906+0000 7f8201634740 10 client.26296 fetch_fsmap learned FSMap version 39 2024-09-11T00:14:01.913 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:01.906+0000 7f8201634740 1 -- 172.21.15.12:0/966776098 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x564141eb4620 con 0x564142177790 2024-09-11T00:14:01.914 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:01.907+0000 7f81e1600640 1 -- 172.21.15.12:0/966776098 <== mon.1 v2:172.21.15.178:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f81e800fc90 con 0x564142177790 2024-09-11T00:14:01.914 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:01.907+0000 7f81e1600640 1 -- 172.21.15.12:0/966776098 <== mon.1 v2:172.21.15.178:3300/0 5 ==== fsmap(e 39) ==== 8378+0+0 (secure 0 0 0) 0x7f81e8003680 con 0x564142177790 2024-09-11T00:14:01.914 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:01.907+0000 7f8201634740 10 client.26296 fetch_fsmap finished waiting for FSMap version 39 2024-09-11T00:14:01.914 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:01.907+0000 7f8201634740 10 client.26296 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T00:14:01.914 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:01.907+0000 7f8201634740 20 client.26296 populate_metadata read hostname 'smithi012' 2024-09-11T00:14:01.914 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:01.907+0000 7f8201634740 1 --2- 172.21.15.12:0/966776098 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x56414218c560 0x56414218e950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:14:01.914 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:01.907+0000 7f8201634740 4 client.26296 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T00:14:01.914 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:01.907+0000 7f8201634740 1 -- 172.21.15.12:0/966776098 --> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5641420ec600 con 0x56414218c560 2024-09-11T00:14:01.914 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:01.907+0000 7f81f6a00640 1 --2- 172.21.15.12:0/966776098 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x56414218c560 0x56414218e950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:14:01.914 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:01.908+0000 7f81f6a00640 1 --2- 172.21.15.12:0/966776098 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x56414218c560 0x56414218e950 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ae691d39b4599f36 server_cookie=407d472ebfa4a7b0 in_seq=0 out_seq=0 2024-09-11T00:14:01.914 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:01.908+0000 7f81e1600640 10 client.26296 ms_handle_connect on v2:172.21.15.165:6838/501312984 2024-09-11T00:14:01.914 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:01.909+0000 7f81e1600640 1 -- 172.21.15.12:0/966776098 <== mds.0 v2:172.21.15.165:6838/501312984 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5641420ec600 con 0x56414218c560 2024-09-11T00:14:01.914 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:01.909+0000 7f81e1600640 10 client.26296 handle_command_reply: tid=0 2024-09-11T00:14:01.915 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:01.909+0000 7f8201634740 1 client.26296 shutdown 2024-09-11T00:14:01.915 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:01.909+0000 7f8201634740 1 -- 172.21.15.12:0/966776098 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564142177790 msgr2=0x564142180e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:14:01.915 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:01.909+0000 7f8201634740 1 --2- 172.21.15.12:0/966776098 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564142177790 0x564142180e90 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f81e800a010 tx=0x7f81e8004270 comp rx=0 tx=0).stop 2024-09-11T00:14:01.915 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:01.909+0000 7f8201634740 1 -- 172.21.15.12:0/966776098 shutdown_connections 2024-09-11T00:14:01.915 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:01.909+0000 7f8201634740 1 --2- 172.21.15.12:0/966776098 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x56414218c560 0x56414218e950 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:14:01.915 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:01.909+0000 7f8201634740 1 --2- 172.21.15.12:0/966776098 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56414217cb00 0x564142182100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:14:01.915 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:01.909+0000 7f81e1600640 0 client.26296 ms_handle_reset on v2:172.21.15.165:6838/501312984 2024-09-11T00:14:01.915 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:01.909+0000 7f8201634740 1 --2- 172.21.15.12:0/966776098 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56414217a0c0 0x564142181530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:14:01.915 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:01.909+0000 7f8201634740 1 --2- 172.21.15.12:0/966776098 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564142177790 0x564142180e90 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:14:01.915 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:01.909+0000 7f8201634740 1 -- 172.21.15.12:0/966776098 >> 172.21.15.12:0/966776098 conn(0x5641421728e0 msgr2=0x564142171a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:14:01.915 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:01.909+0000 7f8201634740 1 -- 172.21.15.12:0/966776098 shutdown_connections 2024-09-11T00:14:01.915 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:01.909+0000 7f8201634740 1 -- 172.21.15.12:0/966776098 wait complete. 2024-09-11T00:14:01.916 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:01.909+0000 7f8201634740 20 client.26296 trim_cache size 0 max 16384 2024-09-11T00:14:01.928 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:01.925+0000 7f8201634740 1 Processor -- start 2024-09-11T00:14:01.928 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:01.925+0000 7f8201634740 1 -- start start 2024-09-11T00:14:01.928 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:01.925+0000 7f8201634740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564142177790 0x5641421805f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:14:01.928 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:01.925+0000 7f8201634740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56414217a0c0 0x564142173140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:14:01.928 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:01.925+0000 7f8201634740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56414217cb00 0x564142173680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:14:01.928 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:01.925+0000 7f8201634740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f81f81ada30 con 0x56414217cb00 2024-09-11T00:14:01.928 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:01.925+0000 7f8201634740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f81f81af5d0 con 0x564142177790 2024-09-11T00:14:01.928 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:01.925+0000 7f8201634740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f81f81ae100 con 0x56414217a0c0 2024-09-11T00:14:01.929 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:01.926+0000 7f81f7e00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56414217cb00 0x564142173680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:14:01.929 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:01.926+0000 7f81f7e00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56414217cb00 0x564142173680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:33096/0 (socket says 172.21.15.12:33096) 2024-09-11T00:14:01.929 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:01.926+0000 7f81f7e00640 1 -- 172.21.15.12:0/3191407850 learned_addr learned my addr 172.21.15.12:0/3191407850 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:14:01.929 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:01.926+0000 7f81f7400640 1 --2- 172.21.15.12:0/3191407850 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564142177790 0x5641421805f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:14:01.929 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:01.926+0000 7f81f6a00640 1 --2- 172.21.15.12:0/3191407850 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56414217a0c0 0x564142173140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:14:01.929 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:01.926+0000 7f81f7e00640 1 -- 172.21.15.12:0/3191407850 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56414217a0c0 msgr2=0x564142173140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:14:01.929 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:01.926+0000 7f81f7e00640 1 --2- 172.21.15.12:0/3191407850 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56414217a0c0 0x564142173140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:14:01.929 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:01.926+0000 7f81f7e00640 1 -- 172.21.15.12:0/3191407850 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564142177790 msgr2=0x5641421805f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:14:01.929 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:01.926+0000 7f81f7e00640 1 --2- 172.21.15.12:0/3191407850 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564142177790 0x5641421805f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:14:01.929 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:01.926+0000 7f81f7e00640 1 -- 172.21.15.12:0/3191407850 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56414218ecc0 con 0x56414217cb00 2024-09-11T00:14:01.929 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:01.926+0000 7f81f7400640 1 --2- 172.21.15.12:0/3191407850 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564142177790 0x5641421805f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:14:01.933 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:01.926+0000 7f81f7e00640 1 --2- 172.21.15.12:0/3191407850 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56414217cb00 0x564142173680 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f81f0009cd0 tx=0x7f81f0041000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de5578ec01992cc4 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:14:01.933 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:01.927+0000 7f81e3e00640 1 -- 172.21.15.12:0/3191407850 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81f00316e0 con 0x56414217cb00 2024-09-11T00:14:01.933 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:01.927+0000 7f81e3e00640 1 -- 172.21.15.12:0/3191407850 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f81f0036040 con 0x56414217cb00 2024-09-11T00:14:01.933 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:01.927+0000 7f81e3e00640 1 -- 172.21.15.12:0/3191407850 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81f0004630 con 0x56414217cb00 2024-09-11T00:14:01.933 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:01.927+0000 7f8201634740 1 -- 172.21.15.12:0/3191407850 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56414217cb00 msgr2=0x564142173680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:14:01.933 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:01.927+0000 7f8201634740 1 --2- 172.21.15.12:0/3191407850 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56414217cb00 0x564142173680 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f81f0009cd0 tx=0x7f81f0041000 comp rx=0 tx=0).stop 2024-09-11T00:14:01.933 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:01.927+0000 7f8201634740 1 -- 172.21.15.12:0/3191407850 shutdown_connections 2024-09-11T00:14:01.934 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:01.927+0000 7f8201634740 1 --2- 172.21.15.12:0/3191407850 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56414217cb00 0x564142173680 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:14:01.934 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:01.927+0000 7f8201634740 1 --2- 172.21.15.12:0/3191407850 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56414217a0c0 0x564142173140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:14:01.934 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:01.927+0000 7f8201634740 1 --2- 172.21.15.12:0/3191407850 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564142177790 0x5641421805f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:14:01.934 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:01.927+0000 7f8201634740 1 -- 172.21.15.12:0/3191407850 >> 172.21.15.12:0/3191407850 conn(0x5641421728e0 msgr2=0x56414217fee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:14:01.934 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:01.927+0000 7f8201634740 1 -- 172.21.15.12:0/3191407850 shutdown_connections 2024-09-11T00:14:01.934 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:01.928+0000 7f8201634740 1 -- 172.21.15.12:0/3191407850 wait complete. 2024-09-11T00:14:01.934 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:01.928+0000 7f8201634740 1 -- start start 2024-09-11T00:14:01.934 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:01.928+0000 7f8201634740 1 Processor -- start 2024-09-11T00:14:01.934 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:01.929+0000 7f8201634740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564142177790 0x564142187f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:14:01.934 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:01.929+0000 7f8201634740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56414217a0c0 0x5641421885b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:14:01.934 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:01.929+0000 7f8201634740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56414217cb00 0x564142186a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:14:01.934 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:01.929+0000 7f8201634740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f81f81aef00 con 0x56414217cb00 2024-09-11T00:14:01.935 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:01.929+0000 7f8201634740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f81f81ae830 con 0x564142177790 2024-09-11T00:14:01.935 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:01.929+0000 7f8201634740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f81f81aba10 con 0x56414217a0c0 2024-09-11T00:14:01.935 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:01.929+0000 7f8201634740 10 client.0 _ll_register_callbacks cb 0x7f81e8005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:14:01.935 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:01.929+0000 7f81f7400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564142177790 0x564142187f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:14:01.935 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:01.929+0000 7f81f6a00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56414217a0c0 0x5641421885b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:14:01.935 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:01.929+0000 7f81f7e00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56414217cb00 0x564142186a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:14:01.935 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:01.929+0000 7f81f7400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564142177790 0x564142187f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.12:45994/0 (socket says 172.21.15.12:45994) 2024-09-11T00:14:01.935 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:01.929+0000 7f81f6a00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56414217a0c0 0x5641421885b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.12:58712/0 (socket says 172.21.15.12:58712) 2024-09-11T00:14:01.935 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:01.929+0000 7f81f7400640 1 -- 172.21.15.12:0/4231916915 learned_addr learned my addr 172.21.15.12:0/4231916915 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:14:01.936 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:01.929+0000 7f81f6a00640 1 -- 172.21.15.12:0/4231916915 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564142177790 msgr2=0x564142187f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:14:01.936 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:01.929+0000 7f81f6a00640 1 --2- 172.21.15.12:0/4231916915 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564142177790 0x564142187f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:14:01.936 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:01.929+0000 7f81f6a00640 1 -- 172.21.15.12:0/4231916915 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56414217cb00 msgr2=0x564142186a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:14:01.936 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:01.929+0000 7f81f6a00640 1 --2- 172.21.15.12:0/4231916915 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56414217cb00 0x564142186a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:14:01.936 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:01.929+0000 7f81f6a00640 1 -- 172.21.15.12:0/4231916915 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564142179ef0 con 0x56414217a0c0 2024-09-11T00:14:01.936 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:01.929+0000 7f81f7e00640 1 --2- 172.21.15.12:0/4231916915 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56414217cb00 0x564142186a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:14:01.936 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:01.930+0000 7f81f6a00640 1 --2- 172.21.15.12:0/4231916915 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56414217a0c0 0x5641421885b0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f81e400be10 tx=0x7f81e40108d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=12bb6d31530e39ad server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:14:01.936 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:01.930+0000 7f81e1600640 10 client.0 ms_handle_connect on v2:172.21.15.178:3300/0 2024-09-11T00:14:01.936 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:01.930+0000 7f81e1600640 1 -- 172.21.15.12:0/4231916915 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81e40ab760 con 0x56414217a0c0 2024-09-11T00:14:01.936 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:01.930+0000 7f81e1600640 1 -- 172.21.15.12:0/4231916915 <== mon.1 v2:172.21.15.178:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9208+0+0 (secure 0 0 0) 0x7f81e40a8260 con 0x56414217a0c0 2024-09-11T00:14:01.936 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:01.930+0000 7f8201634740 1 -- 172.21.15.12:0/4231916915 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f81f0036040 con 0x56414217a0c0 2024-09-11T00:14:01.936 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:01.930+0000 7f81e1600640 1 -- 172.21.15.12:0/4231916915 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f81e4010ed0 con 0x56414217a0c0 2024-09-11T00:14:01.937 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:01.930+0000 7f8201634740 10 client.26302 fetch_fsmap learned FSMap version 39 2024-09-11T00:14:01.937 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:01.930+0000 7f8201634740 1 -- 172.21.15.12:0/4231916915 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x5641421f9c00 con 0x56414217a0c0 2024-09-11T00:14:01.937 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:01.930+0000 7f81e1600640 1 -- 172.21.15.12:0/4231916915 <== mon.1 v2:172.21.15.178:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f81f0036040 con 0x56414217a0c0 2024-09-11T00:14:01.937 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:01.931+0000 7f81e1600640 1 -- 172.21.15.12:0/4231916915 <== mon.1 v2:172.21.15.178:3300/0 5 ==== fsmap(e 39) ==== 8378+0+0 (secure 0 0 0) 0x7f81e40a7940 con 0x56414217a0c0 2024-09-11T00:14:01.937 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:01.931+0000 7f8201634740 10 client.26302 fetch_fsmap finished waiting for FSMap version 39 2024-09-11T00:14:01.937 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:01.931+0000 7f8201634740 10 client.26302 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T00:14:01.937 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:01.931+0000 7f8201634740 20 client.26302 populate_metadata read hostname 'smithi012' 2024-09-11T00:14:01.937 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:01.931+0000 7f8201634740 1 --2- 172.21.15.12:0/4231916915 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x5641421fa600 0x5641421fca20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:14:01.937 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:01.931+0000 7f8201634740 4 client.26302 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T00:14:01.937 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:01.931+0000 7f8201634740 1 -- 172.21.15.12:0/4231916915 --> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5641420e80b0 con 0x5641421fa600 2024-09-11T00:14:01.937 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:01.931+0000 7f81f7400640 1 --2- 172.21.15.12:0/4231916915 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x5641421fa600 0x5641421fca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:14:01.938 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:01.932+0000 7f81f7400640 1 --2- 172.21.15.12:0/4231916915 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x5641421fa600 0x5641421fca20 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=20a10bc3d794e16 server_cookie=da6bedabddd52841 in_seq=0 out_seq=0 2024-09-11T00:14:01.938 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:01.932+0000 7f81e1600640 10 client.26302 ms_handle_connect on v2:172.21.15.165:6838/501312984 2024-09-11T00:14:01.990 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:01.988+0000 7f81e1600640 1 -- 172.21.15.12:0/4231916915 <== mds.0 v2:172.21.15.165:6838/501312984 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5641420e80b0 con 0x5641421fa600 2024-09-11T00:14:01.991 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:01.988+0000 7f81e1600640 10 client.26302 handle_command_reply: tid=0 2024-09-11T00:14:01.991 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:01.988+0000 7f8201634740 1 client.26302 shutdown 2024-09-11T00:14:01.991 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:01.988+0000 7f8201634740 1 -- 172.21.15.12:0/4231916915 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56414217a0c0 msgr2=0x5641421885b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:14:01.991 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:01.988+0000 7f8201634740 1 --2- 172.21.15.12:0/4231916915 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56414217a0c0 0x5641421885b0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f81e400be10 tx=0x7f81e40108d0 comp rx=0 tx=0).stop 2024-09-11T00:14:01.991 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:01.988+0000 7f8201634740 1 -- 172.21.15.12:0/4231916915 shutdown_connections 2024-09-11T00:14:01.991 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:01.988+0000 7f8201634740 1 --2- 172.21.15.12:0/4231916915 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x5641421fa600 0x5641421fca20 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:14:01.991 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:01.988+0000 7f8201634740 1 --2- 172.21.15.12:0/4231916915 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56414217cb00 0x564142186a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:14:01.991 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:01.988+0000 7f81e1600640 0 client.26302 ms_handle_reset on v2:172.21.15.165:6838/501312984 2024-09-11T00:14:01.991 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:01.988+0000 7f8201634740 1 --2- 172.21.15.12:0/4231916915 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56414217a0c0 0x5641421885b0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:14:01.991 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:01.988+0000 7f8201634740 1 --2- 172.21.15.12:0/4231916915 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564142177790 0x564142187f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:14:01.991 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:01.988+0000 7f8201634740 1 -- 172.21.15.12:0/4231916915 >> 172.21.15.12:0/4231916915 conn(0x5641421728e0 msgr2=0x564142173140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:14:01.991 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:01.988+0000 7f8201634740 1 -- 172.21.15.12:0/4231916915 shutdown_connections 2024-09-11T00:14:01.992 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:01.988+0000 7f8201634740 1 -- 172.21.15.12:0/4231916915 wait complete. 2024-09-11T00:14:01.992 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:01.988+0000 7f8201634740 20 client.26302 trim_cache size 0 max 16384 2024-09-11T00:14:01.992 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:{ 2024-09-11T00:14:01.992 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "message": "", 2024-09-11T00:14:01.992 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "return_code": 0 2024-09-11T00:14:01.992 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:} 2024-09-11T00:14:01.993 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:01.989+0000 7f81fe800640 1 -- 172.21.15.12:0/2105756777 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f81dc0b5360 msgr2=0x7f81dc0b7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:14:01.993 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:01.989+0000 7f81fe800640 1 --2- 172.21.15.12:0/2105756777 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f81dc0b5360 0x7f81dc0b7820 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f81f819bd40 tx=0x7f81f0031b20 comp rx=0 tx=0).stop 2024-09-11T00:14:01.993 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:01.989+0000 7f81fe800640 1 -- 172.21.15.12:0/2105756777 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f81f8105ce0 msgr2=0x7f81f819b610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:14:01.993 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:01.989+0000 7f81fe800640 1 --2- 172.21.15.12:0/2105756777 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f81f8105ce0 0x7f81f819b610 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f81e400b450 tx=0x7f81e400b910 comp rx=0 tx=0).stop 2024-09-11T00:14:01.994 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:01.989+0000 7f81fe800640 1 -- 172.21.15.12:0/2105756777 shutdown_connections 2024-09-11T00:14:01.994 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:01.989+0000 7f81fe800640 1 --2- 172.21.15.12:0/2105756777 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f81dc0b5360 0x7f81dc0b7820 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:14:01.994 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:01.989+0000 7f81fe800640 1 --2- 172.21.15.12:0/2105756777 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f81f81066a0 0x7f81f819c240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:14:01.994 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:01.989+0000 7f81fe800640 1 --2- 172.21.15.12:0/2105756777 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f81f8105ce0 0x7f81f819b610 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:14:01.994 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:01.989+0000 7f81fe800640 1 --2- 172.21.15.12:0/2105756777 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f81f8104b90 0x7f81f819afc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:14:01.994 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:01.989+0000 7f81fe800640 1 -- 172.21.15.12:0/2105756777 >> 172.21.15.12:0/2105756777 conn(0x7f81f8100470 msgr2=0x7f81f81090b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:14:01.994 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:01.989+0000 7f81fe800640 1 -- 172.21.15.12:0/2105756777 shutdown_connections 2024-09-11T00:14:01.994 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:01.989+0000 7f81fe800640 1 -- 172.21.15.12:0/2105756777 wait complete. 2024-09-11T00:14:02.244 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:14:01 smithi012 ceph-mon[30333]: pgmap v607: 129 pgs: 129 active+clean; 66 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 1.9 MiB/s wr, 268 op/s 2024-09-11T00:14:02.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:14:01 smithi165 ceph-mon[35356]: pgmap v607: 129 pgs: 129 active+clean; 66 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 1.9 MiB/s wr, 268 op/s 2024-09-11T00:14:02.330 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:14:01 smithi178 ceph-mon[35919]: pgmap v607: 129 pgs: 129 active+clean; 66 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 1.9 MiB/s wr, 268 op/s 2024-09-11T00:14:02.653 INFO:tasks.workunit.client.0.smithi012.stdout: 1 244855 3.35 MB/sec execute 239 sec latency 7862.397 ms 2024-09-11T00:14:03.653 INFO:tasks.workunit.client.0.smithi012.stdout: 1 244855 3.34 MB/sec execute 240 sec latency 8862.484 ms 2024-09-11T00:14:04.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:14:03 smithi165 ceph-mon[35356]: pgmap v608: 129 pgs: 129 active+clean; 65 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 1.5 MiB/s wr, 202 op/s 2024-09-11T00:14:04.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:14:03 smithi012 ceph-mon[30333]: pgmap v608: 129 pgs: 129 active+clean; 65 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 1.5 MiB/s wr, 202 op/s 2024-09-11T00:14:04.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:14:03 smithi178 ceph-mon[35919]: pgmap v608: 129 pgs: 129 active+clean; 65 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 1.5 MiB/s wr, 202 op/s 2024-09-11T00:14:04.653 INFO:tasks.workunit.client.0.smithi012.stdout: 1 244855 3.32 MB/sec execute 241 sec latency 9862.580 ms 2024-09-11T00:14:05.653 INFO:tasks.workunit.client.0.smithi012.stdout: 1 244855 3.31 MB/sec execute 242 sec latency 10862.671 ms 2024-09-11T00:14:06.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:14:05 smithi165 ceph-mon[35356]: pgmap v609: 129 pgs: 129 active+clean; 65 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 MiB/s rd, 1.3 MiB/s wr, 154 op/s 2024-09-11T00:14:06.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:14:05 smithi012 ceph-mon[30333]: pgmap v609: 129 pgs: 129 active+clean; 65 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 MiB/s rd, 1.3 MiB/s wr, 154 op/s 2024-09-11T00:14:06.330 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:14:05 smithi178 ceph-mon[35919]: pgmap v609: 129 pgs: 129 active+clean; 65 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 MiB/s rd, 1.3 MiB/s wr, 154 op/s 2024-09-11T00:14:06.653 INFO:tasks.workunit.client.0.smithi012.stdout: 1 244855 3.30 MB/sec execute 243 sec latency 11862.767 ms 2024-09-11T00:14:07.653 INFO:tasks.workunit.client.0.smithi012.stdout: 1 244855 3.28 MB/sec execute 244 sec latency 12862.847 ms 2024-09-11T00:14:08.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:14:08 smithi165 ceph-mon[35356]: pgmap v610: 129 pgs: 129 active+clean; 64 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 MiB/s rd, 1.5 MiB/s wr, 182 op/s 2024-09-11T00:14:08.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:14:08 smithi012 ceph-mon[30333]: pgmap v610: 129 pgs: 129 active+clean; 64 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 MiB/s rd, 1.5 MiB/s wr, 182 op/s 2024-09-11T00:14:08.330 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:14:08 smithi178 ceph-mon[35919]: pgmap v610: 129 pgs: 129 active+clean; 64 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 MiB/s rd, 1.5 MiB/s wr, 182 op/s 2024-09-11T00:14:08.653 INFO:tasks.workunit.client.0.smithi012.stdout: 1 244855 3.27 MB/sec execute 245 sec latency 13862.909 ms 2024-09-11T00:14:09.653 INFO:tasks.workunit.client.0.smithi012.stdout: 1 244855 3.26 MB/sec execute 246 sec latency 14863.024 ms 2024-09-11T00:14:10.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:14:10 smithi165 ceph-mon[35356]: pgmap v611: 129 pgs: 129 active+clean; 64 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 910 KiB/s wr, 85 op/s 2024-09-11T00:14:10.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:14:10 smithi012 ceph-mon[30333]: pgmap v611: 129 pgs: 129 active+clean; 64 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 910 KiB/s wr, 85 op/s 2024-09-11T00:14:10.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:14:10 smithi178 ceph-mon[35919]: pgmap v611: 129 pgs: 129 active+clean; 64 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 910 KiB/s wr, 85 op/s 2024-09-11T00:14:10.653 INFO:tasks.workunit.client.0.smithi012.stdout: 1 244855 3.24 MB/sec execute 247 sec latency 15863.167 ms 2024-09-11T00:14:11.654 INFO:tasks.workunit.client.0.smithi012.stdout: 1 244855 3.23 MB/sec execute 248 sec latency 16863.312 ms 2024-09-11T00:14:12.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:14:12 smithi165 ceph-mon[35356]: pgmap v612: 129 pgs: 129 active+clean; 64 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 1.1 MiB/s wr, 89 op/s 2024-09-11T00:14:12.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:14:12 smithi012 ceph-mon[30333]: pgmap v612: 129 pgs: 129 active+clean; 64 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 1.1 MiB/s wr, 89 op/s 2024-09-11T00:14:12.330 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:14:12 smithi178 ceph-mon[35919]: pgmap v612: 129 pgs: 129 active+clean; 64 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 1.1 MiB/s wr, 89 op/s 2024-09-11T00:14:12.654 INFO:tasks.workunit.client.0.smithi012.stdout: 1 244855 3.22 MB/sec execute 249 sec latency 17863.380 ms 2024-09-11T00:14:13.654 INFO:tasks.workunit.client.0.smithi012.stdout: 1 244855 3.20 MB/sec execute 250 sec latency 18863.489 ms 2024-09-11T00:14:14.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:14:14 smithi165 ceph-mon[35356]: pgmap v613: 129 pgs: 129 active+clean; 63 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 919 KiB/s wr, 74 op/s 2024-09-11T00:14:14.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:14:14 smithi012 ceph-mon[30333]: pgmap v613: 129 pgs: 129 active+clean; 63 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 919 KiB/s wr, 74 op/s 2024-09-11T00:14:14.330 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:14:14 smithi178 ceph-mon[35919]: pgmap v613: 129 pgs: 129 active+clean; 63 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 919 KiB/s wr, 74 op/s 2024-09-11T00:14:14.654 INFO:tasks.workunit.client.0.smithi012.stdout: 1 244855 3.19 MB/sec execute 251 sec latency 19863.596 ms 2024-09-11T00:14:15.121 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:15.119+0000 7ff723a00640 1 -- 172.21.15.12:0/3518447814 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff71c102990 msgr2=0x7ff71c102d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:14:15.122 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:15.119+0000 7ff723a00640 1 --2- 172.21.15.12:0/3518447814 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff71c102990 0x7ff71c102d90 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7ff710009930 tx=0x7ff71002a440 comp rx=0 tx=0).stop 2024-09-11T00:14:15.122 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:15.119+0000 7ff723a00640 1 -- 172.21.15.12:0/3518447814 shutdown_connections 2024-09-11T00:14:15.122 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:15.119+0000 7ff723a00640 1 --2- 172.21.15.12:0/3518447814 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff71c1044a0 0x7ff71c10ae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:14:15.122 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:15.119+0000 7ff723a00640 1 --2- 172.21.15.12:0/3518447814 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff71c103ae0 0x7ff71c103f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:14:15.122 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:15.119+0000 7ff723a00640 1 --2- 172.21.15.12:0/3518447814 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff71c102990 0x7ff71c102d90 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:14:15.122 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:15.119+0000 7ff723a00640 1 -- 172.21.15.12:0/3518447814 >> 172.21.15.12:0/3518447814 conn(0x7ff71c0fe250 msgr2=0x7ff71c100670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:14:15.122 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:15.119+0000 7ff723a00640 1 -- 172.21.15.12:0/3518447814 shutdown_connections 2024-09-11T00:14:15.122 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:15.119+0000 7ff723a00640 1 -- 172.21.15.12:0/3518447814 wait complete. 2024-09-11T00:14:15.124 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:15.120+0000 7ff723a00640 1 Processor -- start 2024-09-11T00:14:15.124 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:15.120+0000 7ff723a00640 1 -- start start 2024-09-11T00:14:15.124 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:15.120+0000 7ff723a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff71c102990 0x7ff71c19afa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:14:15.124 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:15.120+0000 7ff723a00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff71c103ae0 0x7ff71c19b5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:14:15.124 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:15.120+0000 7ff723a00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff71c1044a0 0x7ff71c19c220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:14:15.124 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:15.120+0000 7ff723a00640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7ff71c10e650 con 0x7ff71c1044a0 2024-09-11T00:14:15.124 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:15.120+0000 7ff723a00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7ff71c10e4d0 con 0x7ff71c102990 2024-09-11T00:14:15.125 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:15.120+0000 7ff723a00640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7ff71c10e7d0 con 0x7ff71c103ae0 2024-09-11T00:14:15.125 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:15.121+0000 7ff721000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff71c102990 0x7ff71c19afa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:14:15.125 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:15.121+0000 7ff721a00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff71c1044a0 0x7ff71c19c220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:14:15.125 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:15.121+0000 7ff721000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff71c102990 0x7ff71c19afa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.12:53854/0 (socket says 172.21.15.12:53854) 2024-09-11T00:14:15.125 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:15.121+0000 7ff721000640 1 -- 172.21.15.12:0/169732308 learned_addr learned my addr 172.21.15.12:0/169732308 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:14:15.125 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:15.121+0000 7ff71be00640 1 --2- 172.21.15.12:0/169732308 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff71c103ae0 0x7ff71c19b5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:14:15.125 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:15.121+0000 7ff721000640 1 -- 172.21.15.12:0/169732308 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff71c103ae0 msgr2=0x7ff71c19b5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:14:15.125 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:15.121+0000 7ff721000640 1 --2- 172.21.15.12:0/169732308 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff71c103ae0 0x7ff71c19b5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:14:15.125 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:15.121+0000 7ff721000640 1 -- 172.21.15.12:0/169732308 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff71c1044a0 msgr2=0x7ff71c19c220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:14:15.125 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:15.121+0000 7ff721000640 1 --2- 172.21.15.12:0/169732308 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff71c1044a0 0x7ff71c19c220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:14:15.125 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:15.121+0000 7ff721000640 1 -- 172.21.15.12:0/169732308 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff71c19cae0 con 0x7ff71c102990 2024-09-11T00:14:15.125 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:15.121+0000 7ff721000640 1 --2- 172.21.15.12:0/169732308 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff71c102990 0x7ff71c19afa0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7ff710031500 tx=0x7ff710003c10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:14:15.125 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:15.122+0000 7ff719600640 1 -- 172.21.15.12:0/169732308 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff71003f070 con 0x7ff71c102990 2024-09-11T00:14:15.126 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:15.122+0000 7ff719600640 1 -- 172.21.15.12:0/169732308 <== mon.2 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff710031c50 con 0x7ff71c102990 2024-09-11T00:14:15.126 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:15.122+0000 7ff723a00640 1 -- 172.21.15.12:0/169732308 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff71c19cd70 con 0x7ff71c102990 2024-09-11T00:14:15.126 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:15.122+0000 7ff719600640 1 -- 172.21.15.12:0/169732308 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff710035030 con 0x7ff71c102990 2024-09-11T00:14:15.126 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:15.122+0000 7ff723a00640 1 -- 172.21.15.12:0/169732308 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff71c1a34a0 con 0x7ff71c102990 2024-09-11T00:14:15.127 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:15.125+0000 7ff7269c8740 1 Processor -- start 2024-09-11T00:14:15.127 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:15.125+0000 7ff7269c8740 1 -- start start 2024-09-11T00:14:15.128 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:15.125+0000 7ff719600640 1 -- 172.21.15.12:0/169732308 <== mon.2 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7ff71003b460 con 0x7ff71c102990 2024-09-11T00:14:15.128 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:15.125+0000 7ff7269c8740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564785173790 0x564785175b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:14:15.128 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:15.125+0000 7ff7269c8740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5647851760c0 0x5647851784b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:14:15.128 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:15.125+0000 7ff7269c8740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564785178b00 0x56478517afc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:14:15.128 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:15.125+0000 7ff7269c8740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x5647850e66b0 con 0x564785178b00 2024-09-11T00:14:15.128 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:15.125+0000 7ff7269c8740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x5647850d5ae0 con 0x5647851760c0 2024-09-11T00:14:15.128 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:15.125+0000 7ff7269c8740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5647850d98f0 con 0x564785173790 2024-09-11T00:14:15.129 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:15.126+0000 7ff721a00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564785173790 0x564785175b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:14:15.129 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:15.126+0000 7ff721000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5647851760c0 0x5647851784b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:14:15.129 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:15.126+0000 7ff721a00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564785173790 0x564785175b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.12:57180/0 (socket says 172.21.15.12:57180) 2024-09-11T00:14:15.129 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:15.126+0000 7ff721a00640 1 -- 172.21.15.12:0/223032570 learned_addr learned my addr 172.21.15.12:0/223032570 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:14:15.129 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:15.126+0000 7ff71be00640 1 --2- 172.21.15.12:0/223032570 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564785178b00 0x56478517afc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:14:15.129 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:15.126+0000 7ff721000640 1 -- 172.21.15.12:0/223032570 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564785173790 msgr2=0x564785175b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:14:15.129 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:15.126+0000 7ff721000640 1 --2- 172.21.15.12:0/223032570 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564785173790 0x564785175b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:14:15.129 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:15.126+0000 7ff721000640 1 -- 172.21.15.12:0/223032570 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564785178b00 msgr2=0x56478517afc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:14:15.129 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:15.126+0000 7ff721000640 1 --2- 172.21.15.12:0/223032570 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564785178b00 0x56478517afc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:14:15.129 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:15.126+0000 7ff721000640 1 -- 172.21.15.12:0/223032570 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5647850a2780 con 0x5647851760c0 2024-09-11T00:14:15.129 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:15.126+0000 7ff71be00640 1 --2- 172.21.15.12:0/223032570 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564785178b00 0x56478517afc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:14:15.134 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:15.126+0000 7ff721000640 1 --2- 172.21.15.12:0/223032570 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5647851760c0 0x5647851784b0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7ff710104100 tx=0x7ff71010ccb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=798643ddd58bfd12 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:14:15.134 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:15.127+0000 7ff70e000640 1 -- 172.21.15.12:0/223032570 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff710115070 con 0x5647851760c0 2024-09-11T00:14:15.134 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:15.127+0000 7ff70e000640 1 -- 172.21.15.12:0/223032570 <== mon.2 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff71010ec60 con 0x5647851760c0 2024-09-11T00:14:15.134 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:15.127+0000 7ff70e000640 1 -- 172.21.15.12:0/223032570 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff710110380 con 0x5647851760c0 2024-09-11T00:14:15.134 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:15.127+0000 7ff719600640 1 --2- 172.21.15.12:0/169732308 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7ff7040b52e0 0x7ff7040b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:14:15.134 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:15.127+0000 7ff719600640 1 -- 172.21.15.12:0/169732308 <== mon.2 v2:172.21.15.165:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9208+0+0 (secure 0 0 0) 0x7ff710106a90 con 0x7ff71c102990 2024-09-11T00:14:15.134 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:15.127+0000 7ff7269c8740 1 -- 172.21.15.12:0/223032570 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5647851760c0 msgr2=0x5647851784b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:14:15.135 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:15.127+0000 7ff7269c8740 1 --2- 172.21.15.12:0/223032570 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5647851760c0 0x5647851784b0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7ff710104100 tx=0x7ff71010ccb0 comp rx=0 tx=0).stop 2024-09-11T00:14:15.135 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:15.127+0000 7ff721a00640 1 --2- 172.21.15.12:0/169732308 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7ff7040b52e0 0x7ff7040b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:14:15.135 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:15.127+0000 7ff7269c8740 1 -- 172.21.15.12:0/223032570 shutdown_connections 2024-09-11T00:14:15.135 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:15.127+0000 7ff7269c8740 1 --2- 172.21.15.12:0/223032570 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564785178b00 0x56478517afc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:14:15.135 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:15.127+0000 7ff7269c8740 1 --2- 172.21.15.12:0/223032570 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5647851760c0 0x5647851784b0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:14:15.135 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:15.127+0000 7ff7269c8740 1 --2- 172.21.15.12:0/223032570 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564785173790 0x564785175b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:14:15.135 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:15.127+0000 7ff7269c8740 1 -- 172.21.15.12:0/223032570 >> 172.21.15.12:0/223032570 conn(0x56478516e8e0 msgr2=0x56478516f670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:14:15.135 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:15.128+0000 7ff7269c8740 1 -- 172.21.15.12:0/223032570 shutdown_connections 2024-09-11T00:14:15.136 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:15.128+0000 7ff7269c8740 1 -- 172.21.15.12:0/223032570 wait complete. 2024-09-11T00:14:15.136 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:15.128+0000 7ff721a00640 1 --2- 172.21.15.12:0/169732308 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7ff7040b52e0 0x7ff7040b77a0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7ff714005fd0 tx=0x7ff714009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:14:15.136 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:15.128+0000 7ff7269c8740 1 -- start start 2024-09-11T00:14:15.136 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:15.128+0000 7ff7269c8740 1 Processor -- start 2024-09-11T00:14:15.136 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:15.129+0000 7ff7269c8740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564785173790 0x56478517cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:14:15.136 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:15.129+0000 7ff7269c8740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5647851760c0 0x56478517d280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:14:15.136 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:15.129+0000 7ff7269c8740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564785178b00 0x56478517de50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:14:15.136 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:15.129+0000 7ff7269c8740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x56478500dab0 con 0x5647851760c0 2024-09-11T00:14:15.136 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:15.129+0000 7ff7269c8740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x56478507c570 con 0x564785178b00 2024-09-11T00:14:15.136 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:15.129+0000 7ff7269c8740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x56478517b8a0 con 0x564785173790 2024-09-11T00:14:15.136 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:15.129+0000 7ff7269c8740 10 client.0 _ll_register_callbacks cb 0x56478508b100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:14:15.136 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:15.129+0000 7ff721000640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564785173790 0x56478517cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:14:15.136 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:15.129+0000 7ff721000640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564785173790 0x56478517cbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.12:57182/0 (socket says 172.21.15.12:57182) 2024-09-11T00:14:15.137 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:15.129+0000 7ff721000640 1 -- 172.21.15.12:0/2917964460 learned_addr learned my addr 172.21.15.12:0/2917964460 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:14:15.137 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:15.129+0000 7ff71be00640 1 --2- 172.21.15.12:0/2917964460 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5647851760c0 0x56478517d280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:14:15.137 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:15.129+0000 7ff721a00640 1 --2- 172.21.15.12:0/2917964460 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564785178b00 0x56478517de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:14:15.137 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:15.129+0000 7ff721000640 1 -- 172.21.15.12:0/2917964460 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564785178b00 msgr2=0x56478517de50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:14:15.137 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:15.129+0000 7ff721000640 1 --2- 172.21.15.12:0/2917964460 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564785178b00 0x56478517de50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:14:15.137 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:15.129+0000 7ff721000640 1 -- 172.21.15.12:0/2917964460 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5647851760c0 msgr2=0x56478517d280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:14:15.137 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:15.129+0000 7ff721000640 1 --2- 172.21.15.12:0/2917964460 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5647851760c0 0x56478517d280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:14:15.137 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:15.129+0000 7ff721000640 1 -- 172.21.15.12:0/2917964460 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564784e37790 con 0x564785173790 2024-09-11T00:14:15.137 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:15.130+0000 7ff721000640 1 --2- 172.21.15.12:0/2917964460 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564785173790 0x56478517cbe0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7ff710005ab0 tx=0x7ff710005e10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4691153ca3cf67f8 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:14:15.137 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:15.130+0000 7ff702a00640 10 client.0 ms_handle_connect on v2:172.21.15.178:3300/0 2024-09-11T00:14:15.137 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:15.130+0000 7ff702a00640 1 -- 172.21.15.12:0/2917964460 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff710049070 con 0x564785173790 2024-09-11T00:14:15.137 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:15.130+0000 7ff7269c8740 1 -- 172.21.15.12:0/2917964460 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff71010ec60 con 0x564785173790 2024-09-11T00:14:15.138 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:15.130+0000 7ff702a00640 1 -- 172.21.15.12:0/2917964460 <== mon.1 v2:172.21.15.178:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9208+0+0 (secure 0 0 0) 0x7ff710104270 con 0x564785173790 2024-09-11T00:14:15.138 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:15.130+0000 7ff702a00640 1 -- 172.21.15.12:0/2917964460 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7ff7100f1020 con 0x564785173790 2024-09-11T00:14:15.138 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:15.130+0000 7ff7269c8740 10 client.26314 fetch_fsmap learned FSMap version 39 2024-09-11T00:14:15.138 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:15.130+0000 7ff7269c8740 1 -- 172.21.15.12:0/2917964460 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x564784eb0620 con 0x564785173790 2024-09-11T00:14:15.138 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:15.130+0000 7ff702a00640 1 -- 172.21.15.12:0/2917964460 <== mon.1 v2:172.21.15.178:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff71010ec60 con 0x564785173790 2024-09-11T00:14:15.138 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:15.131+0000 7ff702a00640 1 -- 172.21.15.12:0/2917964460 <== mon.1 v2:172.21.15.178:3300/0 5 ==== fsmap(e 39) ==== 8378+0+0 (secure 0 0 0) 0x7ff710078ce0 con 0x564785173790 2024-09-11T00:14:15.138 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:15.131+0000 7ff7269c8740 10 client.26314 fetch_fsmap finished waiting for FSMap version 39 2024-09-11T00:14:15.138 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:15.131+0000 7ff7269c8740 10 client.26314 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T00:14:15.138 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:15.131+0000 7ff7269c8740 20 client.26314 populate_metadata read hostname 'smithi012' 2024-09-11T00:14:15.138 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:15.131+0000 7ff7269c8740 1 --2- 172.21.15.12:0/2917964460 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x564785188220 0x56478518a610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:14:15.138 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:15.131+0000 7ff7269c8740 4 client.26314 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T00:14:15.138 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:15.131+0000 7ff7269c8740 1 -- 172.21.15.12:0/2917964460 --> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5647850e8600 con 0x564785188220 2024-09-11T00:14:15.138 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:15.131+0000 7ff71be00640 1 --2- 172.21.15.12:0/2917964460 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x564785188220 0x56478518a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:14:15.139 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:15.132+0000 7ff71be00640 1 --2- 172.21.15.12:0/2917964460 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x564785188220 0x56478518a610 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=768c2a327e1dcbf1 server_cookie=d7c0c68983552e0b in_seq=0 out_seq=0 2024-09-11T00:14:15.139 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:15.132+0000 7ff702a00640 10 client.26314 ms_handle_connect on v2:172.21.15.165:6838/501312984 2024-09-11T00:14:15.140 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:15.132+0000 7ff702a00640 1 -- 172.21.15.12:0/2917964460 <== mds.0 v2:172.21.15.165:6838/501312984 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5647850e8600 con 0x564785188220 2024-09-11T00:14:15.140 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:15.132+0000 7ff702a00640 10 client.26314 handle_command_reply: tid=0 2024-09-11T00:14:15.140 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:15.132+0000 7ff7269c8740 1 client.26314 shutdown 2024-09-11T00:14:15.140 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:15.133+0000 7ff7269c8740 1 -- 172.21.15.12:0/2917964460 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564785173790 msgr2=0x56478517cbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:14:15.140 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:15.133+0000 7ff7269c8740 1 --2- 172.21.15.12:0/2917964460 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564785173790 0x56478517cbe0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7ff710005ab0 tx=0x7ff710005e10 comp rx=0 tx=0).stop 2024-09-11T00:14:15.140 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:15.133+0000 7ff7269c8740 1 -- 172.21.15.12:0/2917964460 shutdown_connections 2024-09-11T00:14:15.140 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:15.133+0000 7ff7269c8740 1 --2- 172.21.15.12:0/2917964460 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x564785188220 0x56478518a610 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:14:15.141 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:15.133+0000 7ff7269c8740 1 --2- 172.21.15.12:0/2917964460 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564785178b00 0x56478517de50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:14:15.141 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:15.133+0000 7ff7269c8740 1 --2- 172.21.15.12:0/2917964460 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5647851760c0 0x56478517d280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:14:15.141 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:15.133+0000 7ff702a00640 0 client.26314 ms_handle_reset on v2:172.21.15.165:6838/501312984 2024-09-11T00:14:15.141 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:15.133+0000 7ff7269c8740 1 --2- 172.21.15.12:0/2917964460 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564785173790 0x56478517cbe0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:14:15.141 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:15.133+0000 7ff7269c8740 1 -- 172.21.15.12:0/2917964460 >> 172.21.15.12:0/2917964460 conn(0x56478516e8e0 msgr2=0x56478516da60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:14:15.141 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:15.133+0000 7ff7269c8740 1 -- 172.21.15.12:0/2917964460 shutdown_connections 2024-09-11T00:14:15.141 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:15.133+0000 7ff7269c8740 1 -- 172.21.15.12:0/2917964460 wait complete. 2024-09-11T00:14:15.141 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:15.133+0000 7ff7269c8740 20 client.26314 trim_cache size 0 max 16384 2024-09-11T00:14:15.151 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:15.148+0000 7ff7269c8740 1 Processor -- start 2024-09-11T00:14:15.151 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:15.148+0000 7ff7269c8740 1 -- start start 2024-09-11T00:14:15.151 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:15.149+0000 7ff7269c8740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564785173790 0x564785169960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:14:15.152 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:15.149+0000 7ff7269c8740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5647851760c0 0x564785171930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:14:15.152 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:15.149+0000 7ff7269c8740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564785178b00 0x564785171e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:14:15.152 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:15.149+0000 7ff7269c8740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7ff71c1ad980 con 0x5647851760c0 2024-09-11T00:14:15.152 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:15.149+0000 7ff7269c8740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7ff71c1af520 con 0x564785173790 2024-09-11T00:14:15.152 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:15.149+0000 7ff7269c8740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7ff71c1ae050 con 0x564785178b00 2024-09-11T00:14:15.152 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:15.149+0000 7ff71be00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5647851760c0 0x564785171930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:14:15.152 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:15.149+0000 7ff71be00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5647851760c0 0x564785171930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:43184/0 (socket says 172.21.15.12:43184) 2024-09-11T00:14:15.152 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:15.149+0000 7ff71be00640 1 -- 172.21.15.12:0/3999196505 learned_addr learned my addr 172.21.15.12:0/3999196505 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:14:15.152 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:15.149+0000 7ff721000640 1 --2- 172.21.15.12:0/3999196505 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564785173790 0x564785169960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:14:15.152 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:15.149+0000 7ff721a00640 1 --2- 172.21.15.12:0/3999196505 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564785178b00 0x564785171e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:14:15.152 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:15.149+0000 7ff71be00640 1 -- 172.21.15.12:0/3999196505 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564785178b00 msgr2=0x564785171e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:14:15.152 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:15.149+0000 7ff71be00640 1 --2- 172.21.15.12:0/3999196505 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564785178b00 0x564785171e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:14:15.152 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:15.149+0000 7ff71be00640 1 -- 172.21.15.12:0/3999196505 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564785173790 msgr2=0x564785169960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:14:15.153 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:15.149+0000 7ff71be00640 1 --2- 172.21.15.12:0/3999196505 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564785173790 0x564785169960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:14:15.153 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:15.149+0000 7ff71be00640 1 -- 172.21.15.12:0/3999196505 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564785178820 con 0x5647851760c0 2024-09-11T00:14:15.153 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:15.149+0000 7ff721000640 1 --2- 172.21.15.12:0/3999196505 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564785173790 0x564785169960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:14:15.157 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:15.150+0000 7ff71be00640 1 --2- 172.21.15.12:0/3999196505 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5647851760c0 0x564785171930 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7ff708009fd0 tx=0x7ff708005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ca51dd4c5fefd9b server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:14:15.157 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:15.150+0000 7ff70d600640 1 -- 172.21.15.12:0/3999196505 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff708029070 con 0x5647851760c0 2024-09-11T00:14:15.157 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:15.150+0000 7ff70d600640 1 -- 172.21.15.12:0/3999196505 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff70801ead0 con 0x5647851760c0 2024-09-11T00:14:15.157 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:15.150+0000 7ff70d600640 1 -- 172.21.15.12:0/3999196505 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff708011ce0 con 0x5647851760c0 2024-09-11T00:14:15.157 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:15.151+0000 7ff7269c8740 1 -- 172.21.15.12:0/3999196505 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5647851760c0 msgr2=0x564785171930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:14:15.157 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:15.151+0000 7ff7269c8740 1 --2- 172.21.15.12:0/3999196505 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5647851760c0 0x564785171930 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7ff708009fd0 tx=0x7ff708005f90 comp rx=0 tx=0).stop 2024-09-11T00:14:15.158 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:15.151+0000 7ff7269c8740 1 -- 172.21.15.12:0/3999196505 shutdown_connections 2024-09-11T00:14:15.158 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:15.151+0000 7ff7269c8740 1 --2- 172.21.15.12:0/3999196505 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564785178b00 0x564785171e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:14:15.158 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:15.151+0000 7ff7269c8740 1 --2- 172.21.15.12:0/3999196505 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5647851760c0 0x564785171930 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:14:15.158 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:15.151+0000 7ff7269c8740 1 --2- 172.21.15.12:0/3999196505 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564785173790 0x564785169960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:14:15.158 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:15.151+0000 7ff7269c8740 1 -- 172.21.15.12:0/3999196505 >> 172.21.15.12:0/3999196505 conn(0x56478516e8e0 msgr2=0x564785172c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:14:15.158 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:15.151+0000 7ff7269c8740 1 -- 172.21.15.12:0/3999196505 shutdown_connections 2024-09-11T00:14:15.158 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:15.151+0000 7ff7269c8740 1 -- 172.21.15.12:0/3999196505 wait complete. 2024-09-11T00:14:15.158 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:15.151+0000 7ff7269c8740 1 -- start start 2024-09-11T00:14:15.158 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:15.151+0000 7ff7269c8740 1 Processor -- start 2024-09-11T00:14:15.158 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:15.152+0000 7ff7269c8740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564785173790 0x5647851846c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:14:15.158 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:15.152+0000 7ff7269c8740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5647851760c0 0x5647851833c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:14:15.158 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:15.152+0000 7ff7269c8740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564785178b00 0x564785183900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:14:15.159 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:15.152+0000 7ff7269c8740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7ff71c1aee50 con 0x564785173790 2024-09-11T00:14:15.159 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:15.152+0000 7ff7269c8740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7ff71c1ae780 con 0x564785178b00 2024-09-11T00:14:15.159 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:15.152+0000 7ff7269c8740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7ff71c1ab960 con 0x5647851760c0 2024-09-11T00:14:15.159 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:15.152+0000 7ff7269c8740 10 client.0 _ll_register_callbacks cb 0x7ff708009590 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:14:15.159 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:15.152+0000 7ff721000640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564785173790 0x5647851846c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:14:15.159 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:15.152+0000 7ff721000640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564785173790 0x5647851846c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:43192/0 (socket says 172.21.15.12:43192) 2024-09-11T00:14:15.159 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:15.152+0000 7ff721000640 1 -- 172.21.15.12:0/842970131 learned_addr learned my addr 172.21.15.12:0/842970131 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:14:15.159 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:15.152+0000 7ff71be00640 1 --2- 172.21.15.12:0/842970131 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5647851760c0 0x5647851833c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:14:15.159 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:15.152+0000 7ff721a00640 1 --2- 172.21.15.12:0/842970131 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564785178b00 0x564785183900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:14:15.159 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:15.152+0000 7ff721000640 1 -- 172.21.15.12:0/842970131 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5647851760c0 msgr2=0x5647851833c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:14:15.159 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:15.152+0000 7ff721000640 1 --2- 172.21.15.12:0/842970131 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5647851760c0 0x5647851833c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:14:15.159 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:15.152+0000 7ff721000640 1 -- 172.21.15.12:0/842970131 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564785178b00 msgr2=0x564785183900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:14:15.160 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:15.152+0000 7ff721000640 1 --2- 172.21.15.12:0/842970131 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564785178b00 0x564785183900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:14:15.160 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:15.152+0000 7ff721000640 1 -- 172.21.15.12:0/842970131 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564785184110 con 0x564785173790 2024-09-11T00:14:15.160 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:15.152+0000 7ff721a00640 1 --2- 172.21.15.12:0/842970131 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564785178b00 0x564785183900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T00:14:15.160 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:15.153+0000 7ff721000640 1 --2- 172.21.15.12:0/842970131 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564785173790 0x5647851846c0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7ff71002aa00 tx=0x7ff71010ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7422149b0267052c server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:14:15.160 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:15.153+0000 7ff702a00640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-09-11T00:14:15.160 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:15.153+0000 7ff702a00640 1 -- 172.21.15.12:0/842970131 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff710049070 con 0x564785173790 2024-09-11T00:14:15.160 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:15.153+0000 7ff702a00640 1 -- 172.21.15.12:0/842970131 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9208+0+0 (secure 0 0 0) 0x7ff7100b8080 con 0x564785173790 2024-09-11T00:14:15.160 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:15.153+0000 7ff7269c8740 1 -- 172.21.15.12:0/842970131 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff70801ead0 con 0x564785173790 2024-09-11T00:14:15.160 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:15.153+0000 7ff702a00640 1 -- 172.21.15.12:0/842970131 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7ff710115030 con 0x564785173790 2024-09-11T00:14:15.160 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:15.153+0000 7ff7269c8740 10 client.16320 fetch_fsmap learned FSMap version 39 2024-09-11T00:14:15.160 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:15.153+0000 7ff7269c8740 1 -- 172.21.15.12:0/842970131 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x564785184110 con 0x564785173790 2024-09-11T00:14:15.160 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:15.153+0000 7ff702a00640 1 -- 172.21.15.12:0/842970131 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff70801ead0 con 0x564785173790 2024-09-11T00:14:15.160 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:15.154+0000 7ff702a00640 1 -- 172.21.15.12:0/842970131 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 39) ==== 8378+0+0 (secure 0 0 0) 0x7ff7101041e0 con 0x564785173790 2024-09-11T00:14:15.161 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:15.154+0000 7ff7269c8740 10 client.16320 fetch_fsmap finished waiting for FSMap version 39 2024-09-11T00:14:15.161 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:15.154+0000 7ff7269c8740 10 client.16320 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T00:14:15.161 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:15.154+0000 7ff7269c8740 20 client.16320 populate_metadata read hostname 'smithi012' 2024-09-11T00:14:15.161 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:15.154+0000 7ff7269c8740 1 --2- 172.21.15.12:0/842970131 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x5647851f5780 0x5647851f7ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:14:15.161 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:15.154+0000 7ff7269c8740 4 client.16320 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T00:14:15.161 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:15.154+0000 7ff7269c8740 1 -- 172.21.15.12:0/842970131 --> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5647850e40b0 con 0x5647851f5780 2024-09-11T00:14:15.161 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:15.154+0000 7ff71be00640 1 --2- 172.21.15.12:0/842970131 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x5647851f5780 0x5647851f7ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:14:15.161 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:15.154+0000 7ff71be00640 1 --2- 172.21.15.12:0/842970131 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x5647851f5780 0x5647851f7ba0 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4fadc503740d15d7 server_cookie=2ab3baaa5079a23d in_seq=0 out_seq=0 2024-09-11T00:14:15.161 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:15.154+0000 7ff702a00640 10 client.16320 ms_handle_connect on v2:172.21.15.165:6838/501312984 2024-09-11T00:14:15.186 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:15.183+0000 7ff702a00640 1 -- 172.21.15.12:0/842970131 <== mds.0 v2:172.21.15.165:6838/501312984 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5647850e40b0 con 0x5647851f5780 2024-09-11T00:14:15.186 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:15.183+0000 7ff702a00640 10 client.16320 handle_command_reply: tid=0 2024-09-11T00:14:15.186 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:15.183+0000 7ff7269c8740 1 client.16320 shutdown 2024-09-11T00:14:15.186 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:15.183+0000 7ff7269c8740 1 -- 172.21.15.12:0/842970131 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564785173790 msgr2=0x5647851846c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:14:15.186 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:15.183+0000 7ff7269c8740 1 --2- 172.21.15.12:0/842970131 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564785173790 0x5647851846c0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7ff71002aa00 tx=0x7ff71010ccb0 comp rx=0 tx=0).stop 2024-09-11T00:14:15.186 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:15.183+0000 7ff7269c8740 1 -- 172.21.15.12:0/842970131 shutdown_connections 2024-09-11T00:14:15.186 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:15.183+0000 7ff7269c8740 1 --2- 172.21.15.12:0/842970131 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x5647851f5780 0x5647851f7ba0 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:14:15.186 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:15.183+0000 7ff7269c8740 1 --2- 172.21.15.12:0/842970131 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564785178b00 0x564785183900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:14:15.186 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:15.183+0000 7ff7269c8740 1 --2- 172.21.15.12:0/842970131 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5647851760c0 0x5647851833c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:14:15.187 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:15.183+0000 7ff702a00640 0 client.16320 ms_handle_reset on v2:172.21.15.165:6838/501312984 2024-09-11T00:14:15.187 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:15.183+0000 7ff7269c8740 1 --2- 172.21.15.12:0/842970131 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564785173790 0x5647851846c0 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:14:15.187 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:15.183+0000 7ff7269c8740 1 -- 172.21.15.12:0/842970131 >> 172.21.15.12:0/842970131 conn(0x56478516e8e0 msgr2=0x564785171930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:14:15.187 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:15.183+0000 7ff7269c8740 1 -- 172.21.15.12:0/842970131 shutdown_connections 2024-09-11T00:14:15.187 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:15.183+0000 7ff7269c8740 1 -- 172.21.15.12:0/842970131 wait complete. 2024-09-11T00:14:15.187 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:15.184+0000 7ff7269c8740 20 client.16320 trim_cache size 0 max 16384 2024-09-11T00:14:15.187 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:{ 2024-09-11T00:14:15.187 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "message": "", 2024-09-11T00:14:15.187 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "return_code": 0 2024-09-11T00:14:15.187 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:} 2024-09-11T00:14:15.188 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:15.184+0000 7ff723a00640 1 -- 172.21.15.12:0/169732308 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7ff7040b52e0 msgr2=0x7ff7040b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:14:15.188 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:15.184+0000 7ff723a00640 1 --2- 172.21.15.12:0/169732308 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7ff7040b52e0 0x7ff7040b77a0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7ff714005fd0 tx=0x7ff714009500 comp rx=0 tx=0).stop 2024-09-11T00:14:15.189 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:15.184+0000 7ff723a00640 1 -- 172.21.15.12:0/169732308 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff71c102990 msgr2=0x7ff71c19afa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:14:15.189 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:15.184+0000 7ff723a00640 1 --2- 172.21.15.12:0/169732308 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff71c102990 0x7ff71c19afa0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7ff710031500 tx=0x7ff710003c10 comp rx=0 tx=0).stop 2024-09-11T00:14:15.189 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:15.184+0000 7ff723a00640 1 -- 172.21.15.12:0/169732308 shutdown_connections 2024-09-11T00:14:15.189 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:15.184+0000 7ff723a00640 1 --2- 172.21.15.12:0/169732308 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7ff7040b52e0 0x7ff7040b77a0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:14:15.189 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:15.184+0000 7ff723a00640 1 --2- 172.21.15.12:0/169732308 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff71c1044a0 0x7ff71c19c220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:14:15.189 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:15.184+0000 7ff723a00640 1 --2- 172.21.15.12:0/169732308 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff71c103ae0 0x7ff71c19b5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:14:15.189 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:15.184+0000 7ff723a00640 1 --2- 172.21.15.12:0/169732308 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff71c102990 0x7ff71c19afa0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:14:15.189 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:15.184+0000 7ff723a00640 1 -- 172.21.15.12:0/169732308 >> 172.21.15.12:0/169732308 conn(0x7ff71c0fe250 msgr2=0x7ff71c109200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:14:15.189 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:15.184+0000 7ff723a00640 1 -- 172.21.15.12:0/169732308 shutdown_connections 2024-09-11T00:14:15.189 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:15.185+0000 7ff723a00640 1 -- 172.21.15.12:0/169732308 wait complete. 2024-09-11T00:14:15.654 INFO:tasks.workunit.client.0.smithi012.stdout: 1 244855 3.18 MB/sec execute 252 sec latency 20863.706 ms 2024-09-11T00:14:16.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:14:16 smithi165 ceph-mon[35356]: pgmap v614: 129 pgs: 129 active+clean; 63 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 806 KiB/s wr, 65 op/s 2024-09-11T00:14:16.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:14:16 smithi012 ceph-mon[30333]: pgmap v614: 129 pgs: 129 active+clean; 63 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 806 KiB/s wr, 65 op/s 2024-09-11T00:14:16.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:14:16 smithi178 ceph-mon[35919]: pgmap v614: 129 pgs: 129 active+clean; 63 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 806 KiB/s wr, 65 op/s 2024-09-11T00:14:16.654 INFO:tasks.workunit.client.0.smithi012.stdout: 1 244855 3.17 MB/sec execute 253 sec latency 21863.790 ms 2024-09-11T00:14:17.108 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid ff3ad74a-6fcf-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T00:14:17.165 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:14:17 smithi012 ceph-mon[30333]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T00:14:17.166 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:14:17 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T00:14:17.532 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:14:17 smithi165 ceph-mon[35356]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T00:14:17.533 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:14:17 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T00:14:17.581 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:14:17 smithi178 ceph-mon[35919]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T00:14:17.581 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:14:17 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T00:14:17.582 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:14:17.654 INFO:tasks.workunit.client.0.smithi012.stdout: 1 244855 3.15 MB/sec execute 254 sec latency 22863.830 ms 2024-09-11T00:14:18.263 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:14:18.260+0000 7f7e0ec00640 1 -- 172.21.15.12:0/2079429537 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7e08102c10 msgr2=0x7f7e08102ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:14:18.263 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:14:18.260+0000 7f7e0ec00640 1 --2- 172.21.15.12:0/2079429537 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7e08102c10 0x7f7e08102ff0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f7dfc009ca0 tx=0x7f7dfc02a8c0 comp rx=0 tx=0).stop 2024-09-11T00:14:18.263 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:14:18.260+0000 7f7e0ec00640 1 -- 172.21.15.12:0/2079429537 shutdown_connections 2024-09-11T00:14:18.263 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:14:18.260+0000 7f7e0ec00640 1 --2- 172.21.15.12:0/2079429537 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7e08108fa0 0x7f7e0810f7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:14:18.263 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:14:18.260+0000 7f7e0ec00640 1 --2- 172.21.15.12:0/2079429537 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7e08108620 0x7f7e08108a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:14:18.263 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:14:18.260+0000 7f7e0ec00640 1 --2- 172.21.15.12:0/2079429537 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7e08102c10 0x7f7e08102ff0 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:14:18.263 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:14:18.260+0000 7f7e0ec00640 1 -- 172.21.15.12:0/2079429537 >> 172.21.15.12:0/2079429537 conn(0x7f7e080fe6c0 msgr2=0x7f7e08100ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:14:18.263 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:14:18.260+0000 7f7e0ec00640 1 -- 172.21.15.12:0/2079429537 shutdown_connections 2024-09-11T00:14:18.263 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:14:18.260+0000 7f7e0ec00640 1 -- 172.21.15.12:0/2079429537 wait complete. 2024-09-11T00:14:18.265 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:14:18.261+0000 7f7e0ec00640 1 Processor -- start 2024-09-11T00:14:18.265 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:14:18.261+0000 7f7e0ec00640 1 -- start start 2024-09-11T00:14:18.265 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:14:18.262+0000 7f7e0ec00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7e08102c10 0x7f7e08076f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:14:18.265 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:14:18.262+0000 7f7e0ec00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7e08108620 0x7f7e080754a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:14:18.265 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:14:18.262+0000 7f7e0ec00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7e08108fa0 0x7f7e08075ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:14:18.265 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:14:18.262+0000 7f7e0ec00640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f7e08112cb0 con 0x7f7e08102c10 2024-09-11T00:14:18.265 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:14:18.262+0000 7f7e0ec00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f7e08112b30 con 0x7f7e08108620 2024-09-11T00:14:18.266 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:14:18.262+0000 7f7e0ec00640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f7e08112e30 con 0x7f7e08108fa0 2024-09-11T00:14:18.266 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:14:18.262+0000 7f7e0d800640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7e08102c10 0x7f7e08076f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:14:18.266 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:14:18.262+0000 7f7e0d800640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7e08102c10 0x7f7e08076f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:43202/0 (socket says 172.21.15.12:43202) 2024-09-11T00:14:18.266 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:14:18.262+0000 7f7e0d800640 1 -- 172.21.15.12:0/1780795584 learned_addr learned my addr 172.21.15.12:0/1780795584 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:14:18.266 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:14:18.262+0000 7f7e0e200640 1 --2- 172.21.15.12:0/1780795584 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7e08108fa0 0x7f7e08075ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:14:18.266 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:14:18.262+0000 7f7e0ce00640 1 --2- 172.21.15.12:0/1780795584 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7e08108620 0x7f7e080754a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:14:18.266 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:14:18.262+0000 7f7e0d800640 1 -- 172.21.15.12:0/1780795584 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7e08108fa0 msgr2=0x7f7e08075ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:14:18.266 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:14:18.262+0000 7f7e0d800640 1 --2- 172.21.15.12:0/1780795584 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7e08108fa0 0x7f7e08075ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:14:18.266 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:14:18.263+0000 7f7e0d800640 1 -- 172.21.15.12:0/1780795584 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7e08108620 msgr2=0x7f7e080754a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:14:18.266 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:14:18.263+0000 7f7e0d800640 1 --2- 172.21.15.12:0/1780795584 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7e08108620 0x7f7e080754a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:14:18.266 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:14:18.263+0000 7f7e0d800640 1 -- 172.21.15.12:0/1780795584 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e081ab9b0 con 0x7f7e08102c10 2024-09-11T00:14:18.266 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:14:18.263+0000 7f7e0e200640 1 --2- 172.21.15.12:0/1780795584 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7e08108fa0 0x7f7e08075ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:14:18.266 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:14:18.263+0000 7f7e0ce00640 1 --2- 172.21.15.12:0/1780795584 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7e08108620 0x7f7e080754a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:14:18.267 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:14:18.263+0000 7f7e0d800640 1 --2- 172.21.15.12:0/1780795584 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7e08102c10 0x7f7e08076f50 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f7dfc005e30 tx=0x7f7dfc031840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:14:18.267 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:14:18.263+0000 7f7e06000640 1 -- 172.21.15.12:0/1780795584 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7dfc003b70 con 0x7f7e08102c10 2024-09-11T00:14:18.267 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:14:18.263+0000 7f7e06000640 1 -- 172.21.15.12:0/1780795584 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7dfc003d10 con 0x7f7e08102c10 2024-09-11T00:14:18.267 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:14:18.263+0000 7f7e0ec00640 1 -- 172.21.15.12:0/1780795584 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7e081abb80 con 0x7f7e08102c10 2024-09-11T00:14:18.267 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:14:18.263+0000 7f7e06000640 1 -- 172.21.15.12:0/1780795584 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7dfc02fa50 con 0x7f7e08102c10 2024-09-11T00:14:18.267 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:14:18.263+0000 7f7e0ec00640 1 -- 172.21.15.12:0/1780795584 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7e081ac080 con 0x7f7e08102c10 2024-09-11T00:14:18.268 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:14:18.265+0000 7f7e0ec00640 1 -- 172.21.15.12:0/1780795584 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7e08102ff0 con 0x7f7e08102c10 2024-09-11T00:14:18.268 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:14:18.267+0000 7f7e06000640 1 -- 172.21.15.12:0/1780795584 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7f7dfc02fbf0 con 0x7f7e08102c10 2024-09-11T00:14:18.270 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:14:18.268+0000 7f7e06000640 1 --2- 172.21.15.12:0/1780795584 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f7df00b5410 0x7f7df00b78d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:14:18.270 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:14:18.268+0000 7f7e06000640 1 -- 172.21.15.12:0/1780795584 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9208+0+0 (secure 0 0 0) 0x7f7dfc107110 con 0x7f7e08102c10 2024-09-11T00:14:18.270 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:14:18.268+0000 7f7e0ce00640 1 --2- 172.21.15.12:0/1780795584 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f7df00b5410 0x7f7df00b78d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:14:18.271 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:14:18.269+0000 7f7e0ce00640 1 --2- 172.21.15.12:0/1780795584 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f7df00b5410 0x7f7df00b78d0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f7e0819cf20 tx=0x7f7df4005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:14:18.274 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:14:18.272+0000 7f7e06000640 1 -- 172.21.15.12:0/1780795584 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f7dfc0cdba0 con 0x7f7e08102c10 2024-09-11T00:14:18.507 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:14:18.505+0000 7f7e0ec00640 1 -- 172.21.15.12:0/1780795584 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7e081ac250 con 0x7f7e08102c10 2024-09-11T00:14:18.507 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:14:18 smithi012 ceph-mon[30333]: pgmap v615: 129 pgs: 129 active+clean; 63 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 1.2 MiB/s wr, 88 op/s 2024-09-11T00:14:18.507 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:14:18 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:14:18.508 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:14:18 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T00:14:18.508 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:14:18 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:14:18.508 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:14:18.505+0000 7f7e06000640 1 -- 172.21.15.12:0/1780795584 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 40 v40) ==== 94+0+11966 (secure 0 0 0) 0x7f7dfc0cd4a0 con 0x7f7e08102c10 2024-09-11T00:14:18.508 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-11T00:14:18.509 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":40,"btime":"2024-09-11T00:14:16:015763+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14577,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6835/447171591","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":447171591},{"type":"v1","addr":"172.21.15.12:6835","nonce":447171591}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14589,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/901486040","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":901486040},{"type":"v1","addr":"172.21.15.12:6837","nonce":901486040}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":40,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T00:04:04.739881+0000","modified":"2024-09-11T00:14:15.594189+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24674,"mds_1":24643,"mds_2":24638,"mds_3":24637,"mds_4":24614},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14631":{"gid":14631,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.12:6841/3069919437","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6840","nonce":3069919437},{"type":"v1","addr":"172.21.15.12:6841","nonce":3069919437}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24562":{"gid":24562,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6833/1141636112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":1141636112},{"type":"v1","addr":"172.21.15.178:6833","nonce":1141636112}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24581":{"gid":24581,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6833/3111980970","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6832","nonce":3111980970},{"type":"v1","addr":"172.21.15.165:6833","nonce":3111980970}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24607":{"gid":24607,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6835/4084868287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":4084868287},{"type":"v1","addr":"172.21.15.178:6835","nonce":4084868287}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24614":{"gid":24614,"name":"e","rank":4,"incarnation":26,"state":"up:active","state_seq":21,"addr":"172.21.15.165:6835/2425395274","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":2425395274},{"type":"v1","addr":"172.21.15.165:6835","nonce":2425395274}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24637":{"gid":24637,"name":"i","rank":3,"incarnation":24,"state":"up:active","state_seq":18,"addr":"172.21.15.178:6839/70628097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6838","nonce":70628097},{"type":"v1","addr":"172.21.15.178:6839","nonce":70628097}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24638":{"gid":24638,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":18,"addr":"172.21.15.165:6837/6215088","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6836","nonce":6215088},{"type":"v1","addr":"172.21.15.165:6837","nonce":6215088}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24643":{"gid":24643,"name":"g","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.12:6839/3124093009","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6838","nonce":3124093009},{"type":"v1","addr":"172.21.15.12:6839","nonce":3124093009}]},"join_fscid":-1,"export_targets":[4],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24674":{"gid":24674,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.165:6839/501312984","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6838","nonce":501312984},{"type":"v1","addr":"172.21.15.165:6839","nonce":501312984}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25672":{"gid":25672,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6837/308601741","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":308601741},{"type":"v1","addr":"172.21.15.178:6837","nonce":308601741}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24674,"qdb_cluster":[24674,24643,24638,24637,24614]},"id":1}]} 2024-09-11T00:14:18.509 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 40 2024-09-11T00:14:18.510 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:14:18.508+0000 7f7e0ec00640 1 -- 172.21.15.12:0/1780795584 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f7df00b5410 msgr2=0x7f7df00b78d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:14:18.511 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:14:18.508+0000 7f7e0ec00640 1 --2- 172.21.15.12:0/1780795584 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f7df00b5410 0x7f7df00b78d0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f7e0819cf20 tx=0x7f7df4005f90 comp rx=0 tx=0).stop 2024-09-11T00:14:18.511 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:14:18.508+0000 7f7e0ec00640 1 -- 172.21.15.12:0/1780795584 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7e08102c10 msgr2=0x7f7e08076f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:14:18.511 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:14:18.508+0000 7f7e0ec00640 1 --2- 172.21.15.12:0/1780795584 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7e08102c10 0x7f7e08076f50 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f7dfc005e30 tx=0x7f7dfc031840 comp rx=0 tx=0).stop 2024-09-11T00:14:18.511 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:14:18.508+0000 7f7e0ec00640 1 -- 172.21.15.12:0/1780795584 shutdown_connections 2024-09-11T00:14:18.511 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:14:18.508+0000 7f7e0ec00640 1 --2- 172.21.15.12:0/1780795584 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f7df00b5410 0x7f7df00b78d0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:14:18.511 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:14:18.508+0000 7f7e0ec00640 1 --2- 172.21.15.12:0/1780795584 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7e08108fa0 0x7f7e08075ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:14:18.511 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:14:18.508+0000 7f7e0ec00640 1 --2- 172.21.15.12:0/1780795584 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7e08108620 0x7f7e080754a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:14:18.511 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:14:18.508+0000 7f7e0ec00640 1 --2- 172.21.15.12:0/1780795584 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7e08102c10 0x7f7e08076f50 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:14:18.511 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:14:18.508+0000 7f7e0ec00640 1 -- 172.21.15.12:0/1780795584 >> 172.21.15.12:0/1780795584 conn(0x7f7e080fe6c0 msgr2=0x7f7e080ffed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:14:18.511 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:14:18.508+0000 7f7e0ec00640 1 -- 172.21.15.12:0/1780795584 shutdown_connections 2024-09-11T00:14:18.511 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:14:18.509+0000 7f7e0ec00640 1 -- 172.21.15.12:0/1780795584 wait complete. 2024-09-11T00:14:18.532 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:14:18 smithi165 ceph-mon[35356]: pgmap v615: 129 pgs: 129 active+clean; 63 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 1.2 MiB/s wr, 88 op/s 2024-09-11T00:14:18.532 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:14:18 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:14:18.532 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:14:18 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T00:14:18.532 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:14:18 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:14:18.581 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:14:18 smithi178 ceph-mon[35919]: pgmap v615: 129 pgs: 129 active+clean; 63 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 1.2 MiB/s wr, 88 op/s 2024-09-11T00:14:18.581 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:14:18 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:14:18.581 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:14:18 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T00:14:18.581 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:14:18 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:14:18.654 INFO:tasks.workunit.client.0.smithi012.stdout: 1 244855 3.14 MB/sec execute 255 sec latency 23863.932 ms 2024-09-11T00:14:19.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:14:19 smithi012 ceph-mon[30333]: pgmap v616: 129 pgs: 129 active+clean; 63 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 968 KiB/s wr, 60 op/s 2024-09-11T00:14:19.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:14:19 smithi012 ceph-mon[30333]: from='client.? 172.21.15.12:0/1780795584' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T00:14:19.483 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24674 quiesce db --set-id dd3a477f --release --await' 2024-09-11T00:14:19.483 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid ff3ad74a-6fcf-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24674 quiesce db --set-id dd3a477f --release --await 2024-09-11T00:14:19.532 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:14:19 smithi165 ceph-mon[35356]: pgmap v616: 129 pgs: 129 active+clean; 63 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 968 KiB/s wr, 60 op/s 2024-09-11T00:14:19.532 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:14:19 smithi165 ceph-mon[35356]: from='client.? 172.21.15.12:0/1780795584' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T00:14:19.581 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:14:19 smithi178 ceph-mon[35919]: pgmap v616: 129 pgs: 129 active+clean; 63 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 968 KiB/s wr, 60 op/s 2024-09-11T00:14:19.581 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:14:19 smithi178 ceph-mon[35919]: from='client.? 172.21.15.12:0/1780795584' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T00:14:19.654 INFO:tasks.workunit.client.0.smithi012.stdout: 1 244855 3.13 MB/sec execute 256 sec latency 24864.018 ms 2024-09-11T00:14:19.911 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:14:20.551 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:14:20.548+0000 7f963a600640 1 -- 172.21.15.12:0/2522305654 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f96341087d0 msgr2=0x7f9634108c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:14:20.551 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:14:20.548+0000 7f963a600640 1 --2- 172.21.15.12:0/2522305654 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f96341087d0 0x7f9634108c10 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f9620009900 tx=0x7f962002a410 comp rx=0 tx=0).stop 2024-09-11T00:14:20.551 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:14:20.548+0000 7f963a600640 1 -- 172.21.15.12:0/2522305654 shutdown_connections 2024-09-11T00:14:20.551 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:14:20.548+0000 7f963a600640 1 --2- 172.21.15.12:0/2522305654 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9634109150 0x7f963410f950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:14:20.551 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:14:20.548+0000 7f963a600640 1 --2- 172.21.15.12:0/2522305654 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f96341087d0 0x7f9634108c10 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:14:20.551 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:14:20.548+0000 7f963a600640 1 --2- 172.21.15.12:0/2522305654 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9634102dc0 0x7f96341031a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:14:20.552 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:14:20.548+0000 7f963a600640 1 -- 172.21.15.12:0/2522305654 >> 172.21.15.12:0/2522305654 conn(0x7f96340fe830 msgr2=0x7f9634100c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:14:20.552 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:14:20.549+0000 7f963a600640 1 -- 172.21.15.12:0/2522305654 shutdown_connections 2024-09-11T00:14:20.552 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:14:20.549+0000 7f963a600640 1 -- 172.21.15.12:0/2522305654 wait complete. 2024-09-11T00:14:20.554 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:14:20.549+0000 7f963a600640 1 Processor -- start 2024-09-11T00:14:20.554 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:14:20.550+0000 7f963a600640 1 -- start start 2024-09-11T00:14:20.554 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:14:20.550+0000 7f963a600640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9634102dc0 0x7f96340754a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:14:20.554 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:14:20.550+0000 7f963a600640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f96341087d0 0x7f96340759e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:14:20.554 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:14:20.550+0000 7f963a600640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9634109150 0x7f9634075f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:14:20.554 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:14:20.550+0000 7f963a600640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f9634112e60 con 0x7f9634102dc0 2024-09-11T00:14:20.554 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:14:20.550+0000 7f963a600640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f9634112ce0 con 0x7f96341087d0 2024-09-11T00:14:20.555 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:14:20.550+0000 7f963a600640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f9634112fe0 con 0x7f9634109150 2024-09-11T00:14:20.555 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:14:20.550+0000 7f9633400640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9634102dc0 0x7f96340754a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:14:20.555 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:14:20.550+0000 7f9633400640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9634102dc0 0x7f96340754a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:60738/0 (socket says 172.21.15.12:60738) 2024-09-11T00:14:20.555 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:14:20.550+0000 7f9633400640 1 -- 172.21.15.12:0/2262085105 learned_addr learned my addr 172.21.15.12:0/2262085105 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:14:20.555 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:14:20.550+0000 7f9632a00640 1 --2- 172.21.15.12:0/2262085105 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f96341087d0 0x7f96340759e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:14:20.555 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:14:20.551+0000 7f9633e00640 1 --2- 172.21.15.12:0/2262085105 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9634109150 0x7f9634075f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:14:20.555 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:14:20.551+0000 7f9633400640 1 -- 172.21.15.12:0/2262085105 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9634109150 msgr2=0x7f9634075f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:14:20.555 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:14:20.551+0000 7f9633400640 1 --2- 172.21.15.12:0/2262085105 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9634109150 0x7f9634075f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:14:20.555 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:14:20.551+0000 7f9633400640 1 -- 172.21.15.12:0/2262085105 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f96341087d0 msgr2=0x7f96340759e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:14:20.555 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:14:20.551+0000 7f9633400640 1 --2- 172.21.15.12:0/2262085105 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f96341087d0 0x7f96340759e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:14:20.555 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:14:20.551+0000 7f9633400640 1 -- 172.21.15.12:0/2262085105 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96341a76b0 con 0x7f9634102dc0 2024-09-11T00:14:20.556 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:14:20.551+0000 7f9632a00640 1 --2- 172.21.15.12:0/2262085105 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f96341087d0 0x7f96340759e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:14:20.556 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:14:20.551+0000 7f9633400640 1 --2- 172.21.15.12:0/2262085105 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9634102dc0 0x7f96340754a0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f962400e9f0 tx=0x7f962400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:14:20.556 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:14:20.552+0000 7f962be00640 1 -- 172.21.15.12:0/2262085105 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f962400cd80 con 0x7f9634102dc0 2024-09-11T00:14:20.556 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:14:20.552+0000 7f962be00640 1 -- 172.21.15.12:0/2262085105 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f96240108e0 con 0x7f9634102dc0 2024-09-11T00:14:20.556 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:14:20.552+0000 7f963a600640 1 -- 172.21.15.12:0/2262085105 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96341a79a0 con 0x7f9634102dc0 2024-09-11T00:14:20.556 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:14:20.552+0000 7f962be00640 1 -- 172.21.15.12:0/2262085105 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9624018980 con 0x7f9634102dc0 2024-09-11T00:14:20.557 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:14:20.552+0000 7f963a600640 1 -- 172.21.15.12:0/2262085105 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96341a7f00 con 0x7f9634102dc0 2024-09-11T00:14:20.557 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:14:20.555+0000 7f963d0c1740 1 Processor -- start 2024-09-11T00:14:20.558 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:14:20.555+0000 7f963d0c1740 1 -- start start 2024-09-11T00:14:20.565 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:14:20.556+0000 7f963d0c1740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ab4159dff0 0x55ab4159e3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:14:20.566 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:14:20.556+0000 7f962be00640 1 -- 172.21.15.12:0/2262085105 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7f9624010430 con 0x7f9634102dc0 2024-09-11T00:14:20.566 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:14:20.556+0000 7f963d0c1740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ab4159e910 0x55ab415aa360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:14:20.566 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:14:20.556+0000 7f963d0c1740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ab415aa8a0 0x55ab415acc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:14:20.566 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:14:20.556+0000 7f963d0c1740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55ab41515290 con 0x55ab4159e910 2024-09-11T00:14:20.566 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:14:20.556+0000 7f963d0c1740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55ab415046c0 con 0x55ab415aa8a0 2024-09-11T00:14:20.566 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:14:20.556+0000 7f963d0c1740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55ab415084d0 con 0x55ab4159dff0 2024-09-11T00:14:20.567 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:14:20.556+0000 7f9633400640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ab4159e910 0x55ab415aa360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:14:20.567 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:14:20.556+0000 7f9633e00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ab4159dff0 0x55ab4159e3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:14:20.567 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:14:20.556+0000 7f9633e00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ab4159dff0 0x55ab4159e3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.12:47750/0 (socket says 172.21.15.12:47750) 2024-09-11T00:14:20.567 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:14:20.556+0000 7f9633e00640 1 -- 172.21.15.12:0/632225824 learned_addr learned my addr 172.21.15.12:0/632225824 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:14:20.567 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:14:20.556+0000 7f9632a00640 1 --2- 172.21.15.12:0/632225824 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ab415aa8a0 0x55ab415acc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:14:20.567 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:14:20.556+0000 7f9633e00640 1 -- 172.21.15.12:0/632225824 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ab415aa8a0 msgr2=0x55ab415acc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:14:20.568 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:14:20.556+0000 7f9633e00640 1 --2- 172.21.15.12:0/632225824 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ab415aa8a0 0x55ab415acc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:14:20.568 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:14:20.556+0000 7f9633e00640 1 -- 172.21.15.12:0/632225824 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ab4159e910 msgr2=0x55ab415aa360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:14:20.568 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:14:20.556+0000 7f9633e00640 1 --2- 172.21.15.12:0/632225824 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ab4159e910 0x55ab415aa360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:14:20.568 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:14:20.556+0000 7f9633e00640 1 -- 172.21.15.12:0/632225824 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ab41491460 con 0x55ab4159dff0 2024-09-11T00:14:20.568 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:14:20.557+0000 7f9633e00640 1 --2- 172.21.15.12:0/632225824 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ab4159dff0 0x55ab4159e3d0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f962c0039a0 tx=0x7f962c003e60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fdca9044846497c3 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:14:20.568 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:14:20.557+0000 7f9628c00640 1 -- 172.21.15.12:0/632225824 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f962c01a470 con 0x55ab4159dff0 2024-09-11T00:14:20.569 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:14:20.557+0000 7f9628c00640 1 -- 172.21.15.12:0/632225824 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f962c015070 con 0x55ab4159dff0 2024-09-11T00:14:20.569 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:14:20.557+0000 7f9628c00640 1 -- 172.21.15.12:0/632225824 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f962c0119b0 con 0x55ab4159dff0 2024-09-11T00:14:20.569 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:14:20.558+0000 7f963d0c1740 1 -- 172.21.15.12:0/632225824 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ab4159dff0 msgr2=0x55ab4159e3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:14:20.569 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:14:20.558+0000 7f963d0c1740 1 --2- 172.21.15.12:0/632225824 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ab4159dff0 0x55ab4159e3d0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f962c0039a0 tx=0x7f962c003e60 comp rx=0 tx=0).stop 2024-09-11T00:14:20.569 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:14:20.558+0000 7f962be00640 1 --2- 172.21.15.12:0/2262085105 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f96180b5360 0x7f96180b7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:14:20.570 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:14:20.558+0000 7f963d0c1740 1 -- 172.21.15.12:0/632225824 shutdown_connections 2024-09-11T00:14:20.570 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:14:20.558+0000 7f963d0c1740 1 --2- 172.21.15.12:0/632225824 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ab415aa8a0 0x55ab415acc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:14:20.570 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:14:20.558+0000 7f963d0c1740 1 --2- 172.21.15.12:0/632225824 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ab4159e910 0x55ab415aa360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:14:20.570 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:14:20.558+0000 7f963d0c1740 1 --2- 172.21.15.12:0/632225824 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ab4159dff0 0x55ab4159e3d0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:14:20.570 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:14:20.558+0000 7f963d0c1740 1 -- 172.21.15.12:0/632225824 >> 172.21.15.12:0/632225824 conn(0x55ab4159be20 msgr2=0x55ab4159cfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:14:20.570 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:14:20.558+0000 7f962be00640 1 -- 172.21.15.12:0/2262085105 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9208+0+0 (secure 0 0 0) 0x7f96240e6200 con 0x7f9634102dc0 2024-09-11T00:14:20.570 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:14:20.558+0000 7f9633400640 1 --2- 172.21.15.12:0/2262085105 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f96180b5360 0x7f96180b7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:14:20.571 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:14:20.558+0000 7f963d0c1740 1 -- 172.21.15.12:0/632225824 shutdown_connections 2024-09-11T00:14:20.571 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:14:20.558+0000 7f963d0c1740 1 -- 172.21.15.12:0/632225824 wait complete. 2024-09-11T00:14:20.571 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:14:20.559+0000 7f963d0c1740 1 -- start start 2024-09-11T00:14:20.571 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:14:20.559+0000 7f9633400640 1 --2- 172.21.15.12:0/2262085105 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f96180b5360 0x7f96180b7820 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f96240e3ca0 tx=0x7f96240e9ce0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:14:20.571 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:14:20.559+0000 7f963d0c1740 1 Processor -- start 2024-09-11T00:14:20.571 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:14:20.559+0000 7f963d0c1740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ab4159dff0 0x55ab415a78a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:14:20.571 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:14:20.560+0000 7f963d0c1740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ab4159e910 0x55ab415a7de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:14:20.572 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:14:20.560+0000 7f963d0c1740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ab415aa8a0 0x55ab415a8320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:14:20.572 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:14:20.560+0000 7f963d0c1740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55ab41365bb0 con 0x55ab4159e910 2024-09-11T00:14:20.572 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:14:20.560+0000 7f963d0c1740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55ab4149f6c0 con 0x55ab415aa8a0 2024-09-11T00:14:20.572 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:14:20.560+0000 7f963d0c1740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55ab415ad550 con 0x55ab4159dff0 2024-09-11T00:14:20.572 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:14:20.560+0000 7f963d0c1740 10 client.0 _ll_register_callbacks cb 0x55ab414ba490 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:14:20.573 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:14:20.560+0000 7f9633e00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ab4159dff0 0x55ab415a78a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:14:20.573 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:14:20.560+0000 7f9632a00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ab4159e910 0x55ab415a7de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:14:20.573 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:14:20.560+0000 7f9633e00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ab4159dff0 0x55ab415a78a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.12:47756/0 (socket says 172.21.15.12:47756) 2024-09-11T00:14:20.573 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:14:20.560+0000 7f9633e00640 1 -- 172.21.15.12:0/1857714802 learned_addr learned my addr 172.21.15.12:0/1857714802 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:14:20.573 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:14:20.560+0000 7f9633e00640 1 --2- 172.21.15.12:0/1857714802 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ab415aa8a0 0x55ab415a8320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:14:20.573 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:14:20.560+0000 7f9633e00640 1 -- 172.21.15.12:0/1857714802 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ab415aa8a0 msgr2=0x55ab415a8320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:14:20.573 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:14:20.560+0000 7f9633e00640 1 --2- 172.21.15.12:0/1857714802 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ab415aa8a0 0x55ab415a8320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:14:20.573 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:14:20.560+0000 7f9633e00640 1 -- 172.21.15.12:0/1857714802 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ab4159e910 msgr2=0x55ab415a7de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:14:20.574 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:14:20.560+0000 7f9633e00640 1 --2- 172.21.15.12:0/1857714802 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ab4159e910 0x55ab415a7de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:14:20.574 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:14:20.560+0000 7f9633e00640 1 -- 172.21.15.12:0/1857714802 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ab41262b00 con 0x55ab4159dff0 2024-09-11T00:14:20.574 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:14:20.561+0000 7f9633e00640 1 --2- 172.21.15.12:0/1857714802 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ab4159dff0 0x55ab415a78a0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f962c000f80 tx=0x7f962c011550 comp rx=0 tx=0).ready entity=mon.1 client_cookie=27a1f48ad4f43b7c server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:14:20.574 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:14:20.561+0000 7f961d600640 10 client.0 ms_handle_connect on v2:172.21.15.178:3300/0 2024-09-11T00:14:20.574 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:14:20.561+0000 7f961d600640 1 -- 172.21.15.12:0/1857714802 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f962c023040 con 0x55ab4159dff0 2024-09-11T00:14:20.574 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:14:20.561+0000 7f963d0c1740 1 -- 172.21.15.12:0/1857714802 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f962c015070 con 0x55ab4159dff0 2024-09-11T00:14:20.574 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:14:20.561+0000 7f961d600640 1 -- 172.21.15.12:0/1857714802 <== mon.1 v2:172.21.15.178:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9208+0+0 (secure 0 0 0) 0x7f962c011d30 con 0x55ab4159dff0 2024-09-11T00:14:20.574 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:14:20.561+0000 7f961d600640 1 -- 172.21.15.12:0/1857714802 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f962c030050 con 0x55ab4159dff0 2024-09-11T00:14:20.574 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:14:20.561+0000 7f963d0c1740 10 client.26338 fetch_fsmap learned FSMap version 40 2024-09-11T00:14:20.574 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:14:20.561+0000 7f963d0c1740 1 -- 172.21.15.12:0/1857714802 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x55ab412d9960 con 0x55ab4159dff0 2024-09-11T00:14:20.575 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:14:20.562+0000 7f961d600640 1 -- 172.21.15.12:0/1857714802 <== mon.1 v2:172.21.15.178:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f962c015070 con 0x55ab4159dff0 2024-09-11T00:14:20.575 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:14:20.562+0000 7f961d600640 1 -- 172.21.15.12:0/1857714802 <== mon.1 v2:172.21.15.178:3300/0 5 ==== fsmap(e 40) ==== 8374+0+0 (secure 0 0 0) 0x7f962c0055d0 con 0x55ab4159dff0 2024-09-11T00:14:20.575 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:14:20.562+0000 7f963d0c1740 10 client.26338 fetch_fsmap finished waiting for FSMap version 40 2024-09-11T00:14:20.575 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:14:20.562+0000 7f963d0c1740 10 client.26338 resolve_mds: validated gid 24674 aka daemon mds.b 2024-09-11T00:14:20.575 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:14:20.562+0000 7f963d0c1740 20 client.26338 populate_metadata read hostname 'smithi012' 2024-09-11T00:14:20.575 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:14:20.562+0000 7f963d0c1740 1 --2- 172.21.15.12:0/1857714802 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55ab415ba380 0x55ab415bc770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:14:20.575 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:14:20.562+0000 7f963d0c1740 4 client.26338 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T00:14:20.575 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:14:20.562+0000 7f963d0c1740 1 -- 172.21.15.12:0/1857714802 --> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ab415171e0 con 0x55ab415ba380 2024-09-11T00:14:20.575 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:14:20.563+0000 7f9632a00640 1 --2- 172.21.15.12:0/1857714802 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55ab415ba380 0x55ab415bc770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:14:20.575 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:14:20.563+0000 7f9632a00640 1 --2- 172.21.15.12:0/1857714802 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55ab415ba380 0x55ab415bc770 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=138a45a3e79b631a server_cookie=e445318b645e2f76 in_seq=0 out_seq=0 2024-09-11T00:14:20.575 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:14:20.563+0000 7f961d600640 10 client.26338 ms_handle_connect on v2:172.21.15.165:6838/501312984 2024-09-11T00:14:20.576 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:14:20.564+0000 7f961d600640 1 -- 172.21.15.12:0/1857714802 <== mds.0 v2:172.21.15.165:6838/501312984 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55ab415171e0 con 0x55ab415ba380 2024-09-11T00:14:20.576 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:14:20.564+0000 7f961d600640 10 client.26338 handle_command_reply: tid=0 2024-09-11T00:14:20.577 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:14:20.564+0000 7f963d0c1740 1 client.26338 shutdown 2024-09-11T00:14:20.577 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:14:20.564+0000 7f963d0c1740 1 -- 172.21.15.12:0/1857714802 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ab4159dff0 msgr2=0x55ab415a78a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:14:20.577 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:14:20.564+0000 7f963d0c1740 1 --2- 172.21.15.12:0/1857714802 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ab4159dff0 0x55ab415a78a0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f962c000f80 tx=0x7f962c011550 comp rx=0 tx=0).stop 2024-09-11T00:14:20.577 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:14:20.564+0000 7f963d0c1740 1 -- 172.21.15.12:0/1857714802 shutdown_connections 2024-09-11T00:14:20.577 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:14:20.564+0000 7f963d0c1740 1 --2- 172.21.15.12:0/1857714802 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55ab415ba380 0x55ab415bc770 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:14:20.577 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:14:20.564+0000 7f963d0c1740 1 --2- 172.21.15.12:0/1857714802 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ab415aa8a0 0x55ab415a8320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:14:20.578 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:14:20.564+0000 7f963d0c1740 1 --2- 172.21.15.12:0/1857714802 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ab4159e910 0x55ab415a7de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:14:20.578 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:14:20.564+0000 7f963d0c1740 1 --2- 172.21.15.12:0/1857714802 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ab4159dff0 0x55ab415a78a0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:14:20.578 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:14:20.564+0000 7f961d600640 0 client.26338 ms_handle_reset on v2:172.21.15.165:6838/501312984 2024-09-11T00:14:20.578 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:14:20.564+0000 7f963d0c1740 1 -- 172.21.15.12:0/1857714802 >> 172.21.15.12:0/1857714802 conn(0x55ab4159be20 msgr2=0x55ab415a19c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:14:20.578 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:14:20.564+0000 7f963d0c1740 1 -- 172.21.15.12:0/1857714802 shutdown_connections 2024-09-11T00:14:20.578 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:14:20.564+0000 7f963d0c1740 1 -- 172.21.15.12:0/1857714802 wait complete. 2024-09-11T00:14:20.578 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:14:20.565+0000 7f963d0c1740 20 client.26338 trim_cache size 0 max 16384 2024-09-11T00:14:20.586 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:14:20.582+0000 7f963d0c1740 1 Processor -- start 2024-09-11T00:14:20.586 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:14:20.583+0000 7f963d0c1740 1 -- start start 2024-09-11T00:14:20.586 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:14:20.583+0000 7f963d0c1740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ab4159dff0 0x55ab4159ac00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:14:20.587 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:14:20.583+0000 7f963d0c1740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ab4159e910 0x55ab4159ce70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:14:20.587 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:14:20.583+0000 7f963d0c1740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ab415aa8a0 0x55ab415a4ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:14:20.587 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:14:20.583+0000 7f963d0c1740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f96341b1e10 con 0x55ab415aa8a0 2024-09-11T00:14:20.587 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:14:20.583+0000 7f963d0c1740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f96341b3980 con 0x55ab4159dff0 2024-09-11T00:14:20.587 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:14:20.583+0000 7f963d0c1740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f96341b24e0 con 0x55ab4159e910 2024-09-11T00:14:20.587 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:14:20.583+0000 7f9633e00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ab415aa8a0 0x55ab415a4ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:14:20.587 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:14:20.583+0000 7f9633e00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ab415aa8a0 0x55ab415a4ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:60766/0 (socket says 172.21.15.12:60766) 2024-09-11T00:14:20.587 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:14:20.583+0000 7f9633e00640 1 -- 172.21.15.12:0/4151553719 learned_addr learned my addr 172.21.15.12:0/4151553719 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:14:20.587 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:14:20.583+0000 7f9632a00640 1 --2- 172.21.15.12:0/4151553719 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ab4159e910 0x55ab4159ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:14:20.587 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:14:20.583+0000 7f9633e00640 1 --2- 172.21.15.12:0/4151553719 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ab4159dff0 0x55ab4159ac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:14:20.587 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:14:20.584+0000 7f9632a00640 1 -- 172.21.15.12:0/4151553719 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ab4159dff0 msgr2=0x55ab4159ac00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:14:20.587 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:14:20.584+0000 7f9632a00640 1 --2- 172.21.15.12:0/4151553719 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ab4159dff0 0x55ab4159ac00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:14:20.588 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:14:20.584+0000 7f9632a00640 1 -- 172.21.15.12:0/4151553719 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ab415aa8a0 msgr2=0x55ab415a4ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:14:20.588 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:14:20.584+0000 7f9632a00640 1 --2- 172.21.15.12:0/4151553719 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ab415aa8a0 0x55ab415a4ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:14:20.588 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:14:20.584+0000 7f9632a00640 1 -- 172.21.15.12:0/4151553719 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ab4159c8a0 con 0x55ab4159e910 2024-09-11T00:14:20.588 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:14:20.584+0000 7f9633e00640 1 --2- 172.21.15.12:0/4151553719 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ab415aa8a0 0x55ab415a4ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:14:20.588 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:14:20.584+0000 7f9632a00640 1 --2- 172.21.15.12:0/4151553719 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ab4159e910 0x55ab4159ce70 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f962002aa00 tx=0x7f9620004510 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f0d31b03a3dc62ac server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:14:20.588 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:14:20.584+0000 7f961fe00640 1 -- 172.21.15.12:0/4151553719 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9620035040 con 0x55ab4159e910 2024-09-11T00:14:20.588 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:14:20.584+0000 7f961fe00640 1 -- 172.21.15.12:0/4151553719 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9620004770 con 0x55ab4159e910 2024-09-11T00:14:20.588 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:14:20.584+0000 7f961fe00640 1 -- 172.21.15.12:0/4151553719 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9620003850 con 0x55ab4159e910 2024-09-11T00:14:20.591 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:14:20.585+0000 7f963d0c1740 1 -- 172.21.15.12:0/4151553719 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ab4159e910 msgr2=0x55ab4159ce70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:14:20.591 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:14:20.585+0000 7f963d0c1740 1 --2- 172.21.15.12:0/4151553719 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ab4159e910 0x55ab4159ce70 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f962002aa00 tx=0x7f9620004510 comp rx=0 tx=0).stop 2024-09-11T00:14:20.591 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:14:20.585+0000 7f963d0c1740 1 -- 172.21.15.12:0/4151553719 shutdown_connections 2024-09-11T00:14:20.591 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:14:20.585+0000 7f963d0c1740 1 --2- 172.21.15.12:0/4151553719 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ab415aa8a0 0x55ab415a4ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:14:20.591 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:14:20.585+0000 7f963d0c1740 1 --2- 172.21.15.12:0/4151553719 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ab4159e910 0x55ab4159ce70 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:14:20.591 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:14:20.585+0000 7f963d0c1740 1 --2- 172.21.15.12:0/4151553719 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ab4159dff0 0x55ab4159ac00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:14:20.591 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:14:20.585+0000 7f963d0c1740 1 -- 172.21.15.12:0/4151553719 >> 172.21.15.12:0/4151553719 conn(0x55ab4159be20 msgr2=0x55ab415a96f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:14:20.591 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:14:20.585+0000 7f963d0c1740 1 -- 172.21.15.12:0/4151553719 shutdown_connections 2024-09-11T00:14:20.591 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:14:20.585+0000 7f963d0c1740 1 -- 172.21.15.12:0/4151553719 wait complete. 2024-09-11T00:14:20.591 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:14:20.586+0000 7f963d0c1740 1 -- start start 2024-09-11T00:14:20.591 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:14:20.586+0000 7f963d0c1740 1 Processor -- start 2024-09-11T00:14:20.591 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:14:20.586+0000 7f963d0c1740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ab4159dff0 0x55ab415b2cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:14:20.592 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:14:20.586+0000 7f963d0c1740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ab4159e910 0x55ab415af220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:14:20.592 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:14:20.586+0000 7f963d0c1740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ab415aa8a0 0x55ab415af760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:14:20.592 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:14:20.586+0000 7f963d0c1740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f96341b32b0 con 0x55ab415aa8a0 2024-09-11T00:14:20.592 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:14:20.586+0000 7f963d0c1740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f96341b2be0 con 0x55ab4159dff0 2024-09-11T00:14:20.592 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:14:20.586+0000 7f963d0c1740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f96341afdf0 con 0x55ab4159e910 2024-09-11T00:14:20.592 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:14:20.587+0000 7f963d0c1740 10 client.0 _ll_register_callbacks cb 0x7f96240e9bb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:14:20.592 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:14:20.587+0000 7f9632a00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ab4159e910 0x55ab415af220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:14:20.592 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:14:20.587+0000 7f9633e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ab4159dff0 0x55ab415b2cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:14:20.592 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:14:20.587+0000 7f9632a00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ab4159e910 0x55ab415af220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.12:47776/0 (socket says 172.21.15.12:47776) 2024-09-11T00:14:20.592 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:14:20.587+0000 7f9633e00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ab415aa8a0 0x55ab415af760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:14:20.592 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:14:20.587+0000 7f9632a00640 1 -- 172.21.15.12:0/880770800 learned_addr learned my addr 172.21.15.12:0/880770800 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:14:20.592 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:14:20.587+0000 7f9632a00640 1 -- 172.21.15.12:0/880770800 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ab4159dff0 msgr2=0x55ab415b2cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:14:20.593 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:14:20.587+0000 7f9632a00640 1 --2- 172.21.15.12:0/880770800 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ab4159dff0 0x55ab415b2cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:14:20.593 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:14:20.587+0000 7f9632a00640 1 -- 172.21.15.12:0/880770800 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ab415aa8a0 msgr2=0x55ab415af760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:14:20.593 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:14:20.587+0000 7f9632a00640 1 --2- 172.21.15.12:0/880770800 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ab415aa8a0 0x55ab415af760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:14:20.593 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:14:20.587+0000 7f9632a00640 1 -- 172.21.15.12:0/880770800 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ab415afcf0 con 0x55ab4159e910 2024-09-11T00:14:20.593 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:14:20.587+0000 7f9632a00640 1 --2- 172.21.15.12:0/880770800 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ab4159e910 0x55ab415af220 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f962002a9d0 tx=0x7f9620005360 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3ce3dc75b3aeddde server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:14:20.593 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:14:20.587+0000 7f961d600640 10 client.0 ms_handle_connect on v2:172.21.15.178:3300/0 2024-09-11T00:14:20.593 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:14:20.588+0000 7f961d600640 1 -- 172.21.15.12:0/880770800 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9620035070 con 0x55ab4159e910 2024-09-11T00:14:20.593 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:14:20.588+0000 7f963d0c1740 1 -- 172.21.15.12:0/880770800 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9620004770 con 0x55ab4159e910 2024-09-11T00:14:20.593 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:14:20.588+0000 7f961d600640 1 -- 172.21.15.12:0/880770800 <== mon.1 v2:172.21.15.178:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9208+0+0 (secure 0 0 0) 0x7f9620005550 con 0x55ab4159e910 2024-09-11T00:14:20.593 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:14:20.588+0000 7f961d600640 1 -- 172.21.15.12:0/880770800 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f9620060050 con 0x55ab4159e910 2024-09-11T00:14:20.594 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:14:20.588+0000 7f963d0c1740 10 client.26344 fetch_fsmap learned FSMap version 40 2024-09-11T00:14:20.594 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:14:20.588+0000 7f963d0c1740 1 -- 172.21.15.12:0/880770800 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x55ab41627f20 con 0x55ab4159e910 2024-09-11T00:14:20.594 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:14:20.588+0000 7f961d600640 1 -- 172.21.15.12:0/880770800 <== mon.1 v2:172.21.15.178:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9620004770 con 0x55ab4159e910 2024-09-11T00:14:20.594 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:14:20.589+0000 7f961d600640 1 -- 172.21.15.12:0/880770800 <== mon.1 v2:172.21.15.178:3300/0 5 ==== fsmap(e 40) ==== 8374+0+0 (secure 0 0 0) 0x7f9620003c10 con 0x55ab4159e910 2024-09-11T00:14:20.594 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:14:20.589+0000 7f963d0c1740 10 client.26344 fetch_fsmap finished waiting for FSMap version 40 2024-09-11T00:14:20.594 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:14:20.589+0000 7f963d0c1740 10 client.26344 resolve_mds: validated gid 24674 aka daemon mds.b 2024-09-11T00:14:20.594 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:14:20.589+0000 7f963d0c1740 20 client.26344 populate_metadata read hostname 'smithi012' 2024-09-11T00:14:20.594 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:14:20.589+0000 7f963d0c1740 1 --2- 172.21.15.12:0/880770800 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55ab41628680 0x55ab4162aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:14:20.594 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:14:20.589+0000 7f963d0c1740 4 client.26344 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "dd3a477f", "release": true, "await": true}] 2024-09-11T00:14:20.594 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:14:20.589+0000 7f963d0c1740 1 -- 172.21.15.12:0/880770800 --> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] -- command(tid 0: {"prefix": "quiesce db", "set_id": "dd3a477f", "release": true, "await": true}) -- 0x55ab41512c90 con 0x55ab41628680 2024-09-11T00:14:20.595 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:14:20.589+0000 7f9633e00640 1 --2- 172.21.15.12:0/880770800 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55ab41628680 0x55ab4162aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:14:20.595 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:14:20.590+0000 7f9633e00640 1 --2- 172.21.15.12:0/880770800 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55ab41628680 0x55ab4162aaa0 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b404fad015cbb5f8 server_cookie=c6bb0dd0aa270c1b in_seq=0 out_seq=0 2024-09-11T00:14:20.595 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:14:20.590+0000 7f961d600640 10 client.26344 ms_handle_connect on v2:172.21.15.165:6838/501312984 2024-09-11T00:14:20.605 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:14:20.602+0000 7f961d600640 1 -- 172.21.15.12:0/880770800 <== mds.0 v2:172.21.15.165:6838/501312984 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55ab41512c90 con 0x55ab41628680 2024-09-11T00:14:20.605 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:14:20.602+0000 7f961d600640 10 client.26344 handle_command_reply: tid=0 2024-09-11T00:14:20.605 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:14:20.602+0000 7f963d0c1740 1 client.26344 shutdown 2024-09-11T00:14:20.605 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:14:20.602+0000 7f963d0c1740 1 -- 172.21.15.12:0/880770800 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ab4159e910 msgr2=0x55ab415af220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:14:20.605 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:14:20.602+0000 7f963d0c1740 1 --2- 172.21.15.12:0/880770800 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ab4159e910 0x55ab415af220 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f962002a9d0 tx=0x7f9620005360 comp rx=0 tx=0).stop 2024-09-11T00:14:20.605 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:14:20.602+0000 7f963d0c1740 1 -- 172.21.15.12:0/880770800 shutdown_connections 2024-09-11T00:14:20.605 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:14:20.602+0000 7f963d0c1740 1 --2- 172.21.15.12:0/880770800 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55ab41628680 0x55ab4162aaa0 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:14:20.606 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:14:20.602+0000 7f963d0c1740 1 --2- 172.21.15.12:0/880770800 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ab415aa8a0 0x55ab415af760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:14:20.606 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:14:20.602+0000 7f963d0c1740 1 --2- 172.21.15.12:0/880770800 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ab4159e910 0x55ab415af220 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:14:20.606 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:14:20.602+0000 7f963d0c1740 1 --2- 172.21.15.12:0/880770800 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ab4159dff0 0x55ab415b2cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:14:20.606 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:14:20.602+0000 7f961d600640 0 client.26344 ms_handle_reset on v2:172.21.15.165:6838/501312984 2024-09-11T00:14:20.606 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:14:20.602+0000 7f963d0c1740 1 -- 172.21.15.12:0/880770800 >> 172.21.15.12:0/880770800 conn(0x55ab4159be20 msgr2=0x55ab415a7810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:14:20.606 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:14:20.603+0000 7f963d0c1740 1 -- 172.21.15.12:0/880770800 shutdown_connections 2024-09-11T00:14:20.606 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:14:20.603+0000 7f963d0c1740 1 -- 172.21.15.12:0/880770800 wait complete. 2024-09-11T00:14:20.606 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:14:20.603+0000 7f963d0c1740 20 client.26344 trim_cache size 0 max 16384 2024-09-11T00:14:20.607 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-09-11T00:14:20.607 INFO:teuthology.orchestra.run.smithi012.stdout: "epoch": 40, 2024-09-11T00:14:20.607 INFO:teuthology.orchestra.run.smithi012.stdout: "leader": 24674, 2024-09-11T00:14:20.607 INFO:teuthology.orchestra.run.smithi012.stdout: "set_version": 24, 2024-09-11T00:14:20.607 INFO:teuthology.orchestra.run.smithi012.stdout: "sets": { 2024-09-11T00:14:20.607 INFO:teuthology.orchestra.run.smithi012.stdout: "dd3a477f": { 2024-09-11T00:14:20.607 INFO:teuthology.orchestra.run.smithi012.stdout: "version": 24, 2024-09-11T00:14:20.607 INFO:teuthology.orchestra.run.smithi012.stdout: "age_ref": 0.0, 2024-09-11T00:14:20.607 INFO:teuthology.orchestra.run.smithi012.stdout: "state": { 2024-09-11T00:14:20.608 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "RELEASED", 2024-09-11T00:14:20.608 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 0.0 2024-09-11T00:14:20.608 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:14:20.608 INFO:teuthology.orchestra.run.smithi012.stdout: "timeout": 90.0, 2024-09-11T00:14:20.608 INFO:teuthology.orchestra.run.smithi012.stdout: "expiration": 146.8, 2024-09-11T00:14:20.608 INFO:teuthology.orchestra.run.smithi012.stdout: "members": { 2024-09-11T00:14:20.608 INFO:teuthology.orchestra.run.smithi012.stdout: "file:/": { 2024-09-11T00:14:20.608 INFO:teuthology.orchestra.run.smithi012.stdout: "excluded": false, 2024-09-11T00:14:20.608 INFO:teuthology.orchestra.run.smithi012.stdout: "state": { 2024-09-11T00:14:20.609 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "RELEASED", 2024-09-11T00:14:20.609 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 0.0 2024-09-11T00:14:20.609 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:14:20.609 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:14:20.609 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:14:20.609 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:14:20.609 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:14:20.609 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-09-11T00:14:20.610 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:14:20.604+0000 7f963a600640 1 -- 172.21.15.12:0/2262085105 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f96180b5360 msgr2=0x7f96180b7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:14:20.610 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:14:20.604+0000 7f963a600640 1 --2- 172.21.15.12:0/2262085105 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f96180b5360 0x7f96180b7820 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f96240e3ca0 tx=0x7f96240e9ce0 comp rx=0 tx=0).stop 2024-09-11T00:14:20.610 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:14:20.604+0000 7f963a600640 1 -- 172.21.15.12:0/2262085105 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9634102dc0 msgr2=0x7f96340754a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:14:20.610 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:14:20.604+0000 7f963a600640 1 --2- 172.21.15.12:0/2262085105 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9634102dc0 0x7f96340754a0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f962400e9f0 tx=0x7f962400eeb0 comp rx=0 tx=0).stop 2024-09-11T00:14:20.610 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:14:20.604+0000 7f963a600640 1 -- 172.21.15.12:0/2262085105 shutdown_connections 2024-09-11T00:14:20.610 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:14:20.604+0000 7f963a600640 1 --2- 172.21.15.12:0/2262085105 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f96180b5360 0x7f96180b7820 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:14:20.610 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:14:20.604+0000 7f963a600640 1 --2- 172.21.15.12:0/2262085105 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9634109150 0x7f9634075f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:14:20.610 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:14:20.604+0000 7f963a600640 1 --2- 172.21.15.12:0/2262085105 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f96341087d0 0x7f96340759e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:14:20.610 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:14:20.604+0000 7f963a600640 1 --2- 172.21.15.12:0/2262085105 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9634102dc0 0x7f96340754a0 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:14:20.610 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:14:20.604+0000 7f963a600640 1 -- 172.21.15.12:0/2262085105 >> 172.21.15.12:0/2262085105 conn(0x7f96340fe830 msgr2=0x7f9634100c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:14:20.611 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:14:20.604+0000 7f963a600640 1 -- 172.21.15.12:0/2262085105 shutdown_connections 2024-09-11T00:14:20.611 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:14:20.604+0000 7f963a600640 1 -- 172.21.15.12:0/2262085105 wait complete. 2024-09-11T00:14:20.654 INFO:tasks.workunit.client.0.smithi012.stdout: 1 244880 3.12 MB/sec execute 257 sec latency 25816.542 ms 2024-09-11T00:14:21.375 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'dd3a477f', total seconds elapsed: 31.0 2024-09-11T00:14:21.654 INFO:tasks.workunit.client.0.smithi012.stdout: 1 245950 3.12 MB/sec execute 258 sec latency 11.661 ms 2024-09-11T00:14:21.782 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:14:21 smithi165 ceph-mon[35356]: pgmap v617: 129 pgs: 129 active+clean; 62 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 1.1 MiB/s wr, 73 op/s 2024-09-11T00:14:21.812 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:14:21 smithi012 ceph-mon[30333]: pgmap v617: 129 pgs: 129 active+clean; 62 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 1.1 MiB/s wr, 73 op/s 2024-09-11T00:14:21.830 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:14:21 smithi178 ceph-mon[35919]: pgmap v617: 129 pgs: 129 active+clean; 62 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 1.1 MiB/s wr, 73 op/s 2024-09-11T00:14:22.655 INFO:tasks.workunit.client.0.smithi012.stdout: 1 246878 3.12 MB/sec execute 259 sec latency 609.803 ms 2024-09-11T00:14:23.563 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:14:23 smithi012 ceph-mon[30333]: pgmap v618: 129 pgs: 129 active+clean; 62 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 275 KiB/s rd, 949 KiB/s wr, 66 op/s 2024-09-11T00:14:23.654 INFO:tasks.workunit.client.0.smithi012.stdout: 1 247974 3.14 MB/sec execute 260 sec latency 910.381 ms 2024-09-11T00:14:23.782 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:14:23 smithi165 ceph-mon[35356]: pgmap v618: 129 pgs: 129 active+clean; 62 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 275 KiB/s rd, 949 KiB/s wr, 66 op/s 2024-09-11T00:14:23.830 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:14:23 smithi178 ceph-mon[35919]: pgmap v618: 129 pgs: 129 active+clean; 62 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 275 KiB/s rd, 949 KiB/s wr, 66 op/s 2024-09-11T00:14:24.655 INFO:tasks.workunit.client.0.smithi012.stdout: 1 248932 3.15 MB/sec execute 261 sec latency 79.921 ms 2024-09-11T00:14:25.655 INFO:tasks.workunit.client.0.smithi012.stdout: 1 251207 3.18 MB/sec execute 262 sec latency 11.036 ms 2024-09-11T00:14:25.782 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:14:25 smithi165 ceph-mon[35356]: pgmap v619: 129 pgs: 129 active+clean; 62 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 374 KiB/s rd, 862 KiB/s wr, 61 op/s 2024-09-11T00:14:25.812 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:14:25 smithi012 ceph-mon[30333]: pgmap v619: 129 pgs: 129 active+clean; 62 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 374 KiB/s rd, 862 KiB/s wr, 61 op/s 2024-09-11T00:14:25.831 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:14:25 smithi178 ceph-mon[35919]: pgmap v619: 129 pgs: 129 active+clean; 62 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 374 KiB/s rd, 862 KiB/s wr, 61 op/s 2024-09-11T00:14:26.655 INFO:tasks.workunit.client.0.smithi012.stdout: 1 252042 3.19 MB/sec execute 263 sec latency 245.013 ms 2024-09-11T00:14:27.404 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:14:27 smithi012 ceph-mon[30333]: pgmap v620: 129 pgs: 129 active+clean; 63 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.5 MiB/s wr, 149 op/s 2024-09-11T00:14:27.655 INFO:tasks.workunit.client.0.smithi012.stdout: 1 252687 3.19 MB/sec execute 264 sec latency 300.611 ms 2024-09-11T00:14:27.782 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:14:27 smithi165 ceph-mon[35356]: pgmap v620: 129 pgs: 129 active+clean; 63 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.5 MiB/s wr, 149 op/s 2024-09-11T00:14:27.831 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:14:27 smithi178 ceph-mon[35919]: pgmap v620: 129 pgs: 129 active+clean; 63 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.5 MiB/s wr, 149 op/s 2024-09-11T00:14:28.315 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:28.312+0000 7f739e000640 1 -- 172.21.15.12:0/1515857759 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7398104b90 msgr2=0x7f7398104f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:14:28.315 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:28.312+0000 7f739e000640 1 --2- 172.21.15.12:0/1515857759 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7398104b90 0x7f7398104f90 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f7388009900 tx=0x7f738802a410 comp rx=0 tx=0).stop 2024-09-11T00:14:28.315 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:28.312+0000 7f739e000640 1 -- 172.21.15.12:0/1515857759 shutdown_connections 2024-09-11T00:14:28.315 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:28.312+0000 7f739e000640 1 --2- 172.21.15.12:0/1515857759 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f73981066a0 0x7f739810d020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:14:28.315 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:28.312+0000 7f739e000640 1 --2- 172.21.15.12:0/1515857759 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7398105ce0 0x7f7398106160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:14:28.315 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:28.312+0000 7f739e000640 1 --2- 172.21.15.12:0/1515857759 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7398104b90 0x7f7398104f90 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:14:28.316 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:28.312+0000 7f739e000640 1 -- 172.21.15.12:0/1515857759 >> 172.21.15.12:0/1515857759 conn(0x7f7398100470 msgr2=0x7f7398102890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:14:28.316 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:28.312+0000 7f739e000640 1 -- 172.21.15.12:0/1515857759 shutdown_connections 2024-09-11T00:14:28.316 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:28.312+0000 7f739e000640 1 -- 172.21.15.12:0/1515857759 wait complete. 2024-09-11T00:14:28.317 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:28.313+0000 7f739e000640 1 Processor -- start 2024-09-11T00:14:28.318 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:28.314+0000 7f739e000640 1 -- start start 2024-09-11T00:14:28.318 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:28.314+0000 7f739e000640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7398104b90 0x7f739819af60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:14:28.318 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:28.314+0000 7f739e000640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7398105ce0 0x7f739819b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:14:28.318 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:28.314+0000 7f739e000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f73981066a0 0x7f739819c1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:14:28.318 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:28.314+0000 7f739e000640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f73981107e0 con 0x7f7398104b90 2024-09-11T00:14:28.318 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:28.314+0000 7f739e000640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f7398110660 con 0x7f73981066a0 2024-09-11T00:14:28.318 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:28.314+0000 7f739e000640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f7398110960 con 0x7f7398105ce0 2024-09-11T00:14:28.318 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:28.314+0000 7f7397400640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7398104b90 0x7f739819af60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:14:28.318 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:28.314+0000 7f7397400640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7398104b90 0x7f739819af60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:60790/0 (socket says 172.21.15.12:60790) 2024-09-11T00:14:28.318 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:28.314+0000 7f7397400640 1 -- 172.21.15.12:0/1528562074 learned_addr learned my addr 172.21.15.12:0/1528562074 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:14:28.318 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:28.314+0000 7f7397e00640 1 --2- 172.21.15.12:0/1528562074 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f73981066a0 0x7f739819c1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:14:28.318 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:28.314+0000 7f738ea00640 1 --2- 172.21.15.12:0/1528562074 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7398105ce0 0x7f739819b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:14:28.318 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:28.314+0000 7f7397400640 1 -- 172.21.15.12:0/1528562074 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7398105ce0 msgr2=0x7f739819b5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:14:28.319 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:28.314+0000 7f7397400640 1 --2- 172.21.15.12:0/1528562074 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7398105ce0 0x7f739819b5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:14:28.319 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:28.314+0000 7f7397400640 1 -- 172.21.15.12:0/1528562074 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f73981066a0 msgr2=0x7f739819c1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:14:28.319 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:28.314+0000 7f7397400640 1 --2- 172.21.15.12:0/1528562074 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f73981066a0 0x7f739819c1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:14:28.319 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:28.314+0000 7f7397400640 1 -- 172.21.15.12:0/1528562074 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f739819caa0 con 0x7f7398104b90 2024-09-11T00:14:28.319 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:28.315+0000 7f7397400640 1 --2- 172.21.15.12:0/1528562074 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7398104b90 0x7f739819af60 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f7388009230 tx=0x7f73880042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:14:28.319 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:28.315+0000 7f7394c00640 1 -- 172.21.15.12:0/1528562074 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f738803f030 con 0x7f7398104b90 2024-09-11T00:14:28.319 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:28.315+0000 7f7394c00640 1 -- 172.21.15.12:0/1528562074 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7388032e40 con 0x7f7398104b90 2024-09-11T00:14:28.319 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:28.315+0000 7f7394c00640 1 -- 172.21.15.12:0/1528562074 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7388035070 con 0x7f7398104b90 2024-09-11T00:14:28.319 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:28.315+0000 7f739e000640 1 -- 172.21.15.12:0/1528562074 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f739819cd30 con 0x7f7398104b90 2024-09-11T00:14:28.319 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:28.315+0000 7f739e000640 1 -- 172.21.15.12:0/1528562074 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73981a3280 con 0x7f7398104b90 2024-09-11T00:14:28.321 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:28.319+0000 7f73a0df4740 1 Processor -- start 2024-09-11T00:14:28.321 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:28.319+0000 7f7394c00640 1 -- 172.21.15.12:0/1528562074 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7f738803bad0 con 0x7f7398104b90 2024-09-11T00:14:28.321 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:28.319+0000 7f73a0df4740 1 -- start start 2024-09-11T00:14:28.323 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:28.319+0000 7f73a0df4740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a6f1136790 0x55a6f1138b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:14:28.323 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:28.319+0000 7f73a0df4740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a6f11390c0 0x55a6f113b4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:14:28.323 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:28.319+0000 7f73a0df4740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a6f113bb00 0x55a6f113dfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:14:28.323 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:28.319+0000 7f73a0df4740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55a6f10a96b0 con 0x55a6f11390c0 2024-09-11T00:14:28.323 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:28.319+0000 7f73a0df4740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55a6f1098ae0 con 0x55a6f1136790 2024-09-11T00:14:28.323 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:28.319+0000 7f73a0df4740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55a6f109c8f0 con 0x55a6f113bb00 2024-09-11T00:14:28.323 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:28.320+0000 7f7397e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a6f1136790 0x55a6f1138b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:14:28.324 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:28.320+0000 7f7397e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a6f1136790 0x55a6f1138b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.12:35970/0 (socket says 172.21.15.12:35970) 2024-09-11T00:14:28.324 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:28.320+0000 7f7397e00640 1 -- 172.21.15.12:0/224825779 learned_addr learned my addr 172.21.15.12:0/224825779 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:14:28.324 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:28.320+0000 7f7397400640 1 --2- 172.21.15.12:0/224825779 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a6f11390c0 0x55a6f113b4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:14:28.324 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:28.320+0000 7f738ea00640 1 --2- 172.21.15.12:0/224825779 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a6f113bb00 0x55a6f113dfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:14:28.324 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:28.320+0000 7f7397e00640 1 -- 172.21.15.12:0/224825779 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a6f113bb00 msgr2=0x55a6f113dfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:14:28.324 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:28.320+0000 7f7397e00640 1 --2- 172.21.15.12:0/224825779 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a6f113bb00 0x55a6f113dfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:14:28.324 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:28.320+0000 7f7397e00640 1 -- 172.21.15.12:0/224825779 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a6f11390c0 msgr2=0x55a6f113b4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:14:28.324 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:28.320+0000 7f7397e00640 1 --2- 172.21.15.12:0/224825779 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a6f11390c0 0x55a6f113b4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:14:28.324 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:28.320+0000 7f7397e00640 1 -- 172.21.15.12:0/224825779 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a6f1065780 con 0x55a6f1136790 2024-09-11T00:14:28.325 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:28.320+0000 7f7397400640 1 --2- 172.21.15.12:0/224825779 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a6f11390c0 0x55a6f113b4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T00:14:28.325 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:28.320+0000 7f7397e00640 1 --2- 172.21.15.12:0/224825779 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a6f1136790 0x55a6f1138b80 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f7390009f90 tx=0x7f73900118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4591729589af2f6 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:14:28.325 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:28.321+0000 7f7394c00640 1 --2- 172.21.15.12:0/1528562074 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f737c0b52e0 0x7f737c0b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:14:28.325 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:28.321+0000 7f7387e00640 1 -- 172.21.15.12:0/224825779 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7390004d20 con 0x55a6f1136790 2024-09-11T00:14:28.325 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:28.321+0000 7f7387e00640 1 -- 172.21.15.12:0/224825779 <== mon.2 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7390004050 con 0x55a6f1136790 2024-09-11T00:14:28.325 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:28.321+0000 7f7387e00640 1 -- 172.21.15.12:0/224825779 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7390005280 con 0x55a6f1136790 2024-09-11T00:14:28.325 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:28.321+0000 7f7394c00640 1 -- 172.21.15.12:0/1528562074 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9208+0+0 (secure 0 0 0) 0x7f73881067e0 con 0x7f7398104b90 2024-09-11T00:14:28.325 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:28.321+0000 7f7397400640 1 --2- 172.21.15.12:0/1528562074 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f737c0b52e0 0x7f737c0b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:14:28.330 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:28.322+0000 7f73a0df4740 1 -- 172.21.15.12:0/224825779 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a6f1136790 msgr2=0x55a6f1138b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:14:28.330 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:28.322+0000 7f73a0df4740 1 --2- 172.21.15.12:0/224825779 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a6f1136790 0x55a6f1138b80 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f7390009f90 tx=0x7f73900118d0 comp rx=0 tx=0).stop 2024-09-11T00:14:28.330 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:28.322+0000 7f7397400640 1 --2- 172.21.15.12:0/1528562074 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f737c0b52e0 0x7f737c0b77a0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f73880c91f0 tx=0x7f7388009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:14:28.330 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:28.322+0000 7f73a0df4740 1 -- 172.21.15.12:0/224825779 shutdown_connections 2024-09-11T00:14:28.330 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:28.322+0000 7f73a0df4740 1 --2- 172.21.15.12:0/224825779 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a6f113bb00 0x55a6f113dfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:14:28.330 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:28.322+0000 7f73a0df4740 1 --2- 172.21.15.12:0/224825779 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a6f11390c0 0x55a6f113b4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:14:28.330 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:28.322+0000 7f73a0df4740 1 --2- 172.21.15.12:0/224825779 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a6f1136790 0x55a6f1138b80 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:14:28.330 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:28.322+0000 7f73a0df4740 1 -- 172.21.15.12:0/224825779 >> 172.21.15.12:0/224825779 conn(0x55a6f11318e0 msgr2=0x55a6f1132670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:14:28.330 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:28.322+0000 7f73a0df4740 1 -- 172.21.15.12:0/224825779 shutdown_connections 2024-09-11T00:14:28.330 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:28.322+0000 7f73a0df4740 1 -- 172.21.15.12:0/224825779 wait complete. 2024-09-11T00:14:28.330 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:28.322+0000 7f73a0df4740 1 -- start start 2024-09-11T00:14:28.330 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:28.322+0000 7f73a0df4740 1 Processor -- start 2024-09-11T00:14:28.331 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:28.323+0000 7f73a0df4740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a6f1136790 0x55a6f113fea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:14:28.331 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:28.323+0000 7f73a0df4740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a6f11390c0 0x55a6f1140540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:14:28.331 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:28.323+0000 7f73a0df4740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a6f113bb00 0x55a6f1141110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:14:28.331 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:28.323+0000 7f73a0df4740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55a6f0fd0ab0 con 0x55a6f11390c0 2024-09-11T00:14:28.331 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:28.323+0000 7f73a0df4740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55a6f103f570 con 0x55a6f1136790 2024-09-11T00:14:28.331 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:28.323+0000 7f73a0df4740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55a6f113eae0 con 0x55a6f113bb00 2024-09-11T00:14:28.331 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:28.323+0000 7f73a0df4740 10 client.0 _ll_register_callbacks cb 0x55a6f104e100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:14:28.331 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:28.323+0000 7f7397e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a6f1136790 0x55a6f113fea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:14:28.331 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:28.323+0000 7f7397e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a6f1136790 0x55a6f113fea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.12:35980/0 (socket says 172.21.15.12:35980) 2024-09-11T00:14:28.331 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:28.323+0000 7f7397e00640 1 -- 172.21.15.12:0/3581645584 learned_addr learned my addr 172.21.15.12:0/3581645584 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:14:28.331 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:28.323+0000 7f738ea00640 1 --2- 172.21.15.12:0/3581645584 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a6f11390c0 0x55a6f1140540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:14:28.331 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:28.323+0000 7f7397e00640 1 --2- 172.21.15.12:0/3581645584 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a6f113bb00 0x55a6f1141110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:14:28.331 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:28.324+0000 7f738ea00640 1 -- 172.21.15.12:0/3581645584 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a6f113bb00 msgr2=0x55a6f1141110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:14:28.332 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:28.324+0000 7f738ea00640 1 --2- 172.21.15.12:0/3581645584 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a6f113bb00 0x55a6f1141110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:14:28.332 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:28.324+0000 7f738ea00640 1 -- 172.21.15.12:0/3581645584 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a6f1136790 msgr2=0x55a6f113fea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:14:28.332 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:28.324+0000 7f738ea00640 1 --2- 172.21.15.12:0/3581645584 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a6f1136790 0x55a6f113fea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:14:28.332 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:28.324+0000 7f738ea00640 1 -- 172.21.15.12:0/3581645584 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a6f0dfa790 con 0x55a6f11390c0 2024-09-11T00:14:28.332 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:28.324+0000 7f7397e00640 1 --2- 172.21.15.12:0/3581645584 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a6f1136790 0x55a6f113fea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:14:28.332 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:28.324+0000 7f738ea00640 1 --2- 172.21.15.12:0/3581645584 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a6f11390c0 0x55a6f1140540 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f7380012700 tx=0x7f7380012bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bfea74857368d123 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:14:28.332 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:28.324+0000 7f7384c00640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-09-11T00:14:28.332 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:28.324+0000 7f7384c00640 1 -- 172.21.15.12:0/3581645584 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f738000f920 con 0x55a6f11390c0 2024-09-11T00:14:28.332 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:28.324+0000 7f73a0df4740 1 -- 172.21.15.12:0/3581645584 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7390004050 con 0x55a6f11390c0 2024-09-11T00:14:28.332 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:28.324+0000 7f7384c00640 1 -- 172.21.15.12:0/3581645584 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9208+0+0 (secure 0 0 0) 0x7f738000fac0 con 0x55a6f11390c0 2024-09-11T00:14:28.332 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:28.325+0000 7f7384c00640 1 -- 172.21.15.12:0/3581645584 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f738000fcc0 con 0x55a6f11390c0 2024-09-11T00:14:28.332 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:28.325+0000 7f73a0df4740 10 client.16356 fetch_fsmap learned FSMap version 40 2024-09-11T00:14:28.333 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:28.325+0000 7f73a0df4740 1 -- 172.21.15.12:0/3581645584 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x55a6f0e73620 con 0x55a6f11390c0 2024-09-11T00:14:28.333 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:28.325+0000 7f7384c00640 1 -- 172.21.15.12:0/3581645584 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7390004050 con 0x55a6f11390c0 2024-09-11T00:14:28.333 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:28.325+0000 7f7384c00640 1 -- 172.21.15.12:0/3581645584 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 40) ==== 8374+0+0 (secure 0 0 0) 0x7f738001c420 con 0x55a6f11390c0 2024-09-11T00:14:28.333 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:28.325+0000 7f73a0df4740 10 client.16356 fetch_fsmap finished waiting for FSMap version 40 2024-09-11T00:14:28.333 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:28.325+0000 7f73a0df4740 10 client.16356 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T00:14:28.333 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:28.325+0000 7f73a0df4740 20 client.16356 populate_metadata read hostname 'smithi012' 2024-09-11T00:14:28.333 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:28.325+0000 7f73a0df4740 1 --2- 172.21.15.12:0/3581645584 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55a6f114b8a0 0x55a6f114dcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:14:28.333 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:28.325+0000 7f73a0df4740 4 client.16356 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T00:14:28.333 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:28.325+0000 7f73a0df4740 1 -- 172.21.15.12:0/3581645584 --> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a6f10ab600 con 0x55a6f114b8a0 2024-09-11T00:14:28.333 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:28.326+0000 7f7397e00640 1 --2- 172.21.15.12:0/3581645584 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55a6f114b8a0 0x55a6f114dcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:14:28.333 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:28.326+0000 7f7397e00640 1 --2- 172.21.15.12:0/3581645584 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55a6f114b8a0 0x55a6f114dcd0 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6b6243fd5334162c server_cookie=a79eb4c4aa63876d in_seq=0 out_seq=0 2024-09-11T00:14:28.333 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:28.326+0000 7f7384c00640 10 client.16356 ms_handle_connect on v2:172.21.15.165:6838/501312984 2024-09-11T00:14:28.334 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:28.327+0000 7f7384c00640 1 -- 172.21.15.12:0/3581645584 <== mds.0 v2:172.21.15.165:6838/501312984 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55a6f10ab600 con 0x55a6f114b8a0 2024-09-11T00:14:28.334 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:28.327+0000 7f7384c00640 10 client.16356 handle_command_reply: tid=0 2024-09-11T00:14:28.334 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:28.327+0000 7f73a0df4740 1 client.16356 shutdown 2024-09-11T00:14:28.334 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:28.327+0000 7f73a0df4740 1 -- 172.21.15.12:0/3581645584 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a6f11390c0 msgr2=0x55a6f1140540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:14:28.334 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:28.327+0000 7f73a0df4740 1 --2- 172.21.15.12:0/3581645584 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a6f11390c0 0x55a6f1140540 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f7380012700 tx=0x7f7380012bc0 comp rx=0 tx=0).stop 2024-09-11T00:14:28.334 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:28.327+0000 7f73a0df4740 1 -- 172.21.15.12:0/3581645584 shutdown_connections 2024-09-11T00:14:28.334 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:28.327+0000 7f73a0df4740 1 --2- 172.21.15.12:0/3581645584 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55a6f114b8a0 0x55a6f114dcd0 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:14:28.334 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:28.327+0000 7f73a0df4740 1 --2- 172.21.15.12:0/3581645584 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a6f113bb00 0x55a6f1141110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:14:28.334 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:28.327+0000 7f73a0df4740 1 --2- 172.21.15.12:0/3581645584 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a6f11390c0 0x55a6f1140540 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:14:28.334 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:28.327+0000 7f73a0df4740 1 --2- 172.21.15.12:0/3581645584 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a6f1136790 0x55a6f113fea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:14:28.334 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:28.327+0000 7f7384c00640 0 client.16356 ms_handle_reset on v2:172.21.15.165:6838/501312984 2024-09-11T00:14:28.334 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:28.327+0000 7f73a0df4740 1 -- 172.21.15.12:0/3581645584 >> 172.21.15.12:0/3581645584 conn(0x55a6f11318e0 msgr2=0x55a6f1130a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:14:28.335 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:28.327+0000 7f73a0df4740 1 -- 172.21.15.12:0/3581645584 shutdown_connections 2024-09-11T00:14:28.335 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:28.327+0000 7f73a0df4740 1 -- 172.21.15.12:0/3581645584 wait complete. 2024-09-11T00:14:28.335 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:28.327+0000 7f73a0df4740 20 client.16356 trim_cache size 0 max 16384 2024-09-11T00:14:28.355 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:28.344+0000 7f73a0df4740 1 Processor -- start 2024-09-11T00:14:28.355 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:28.344+0000 7f73a0df4740 1 -- start start 2024-09-11T00:14:28.355 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:28.344+0000 7f73a0df4740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a6f1136790 0x55a6f113bee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:14:28.355 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:28.344+0000 7f73a0df4740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a6f11390c0 0x55a6f113c420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:14:28.355 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:28.344+0000 7f73a0df4740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a6f113bb00 0x55a6f113fea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:14:28.355 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:28.344+0000 7f73a0df4740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f73981ad9b0 con 0x55a6f113bb00 2024-09-11T00:14:28.355 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:28.344+0000 7f73a0df4740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f73981af520 con 0x55a6f1136790 2024-09-11T00:14:28.355 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:28.344+0000 7f73a0df4740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f73981ae080 con 0x55a6f11390c0 2024-09-11T00:14:28.355 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:28.344+0000 7f7397e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a6f1136790 0x55a6f113bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:14:28.356 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:28.344+0000 7f738ea00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a6f11390c0 0x55a6f113c420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:14:28.356 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:28.345+0000 7f7397e00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a6f113bb00 0x55a6f113fea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:14:28.356 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:28.345+0000 7f738ea00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a6f11390c0 0x55a6f113c420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.12:47822/0 (socket says 172.21.15.12:47822) 2024-09-11T00:14:28.356 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:28.345+0000 7f738ea00640 1 -- 172.21.15.12:0/424129507 learned_addr learned my addr 172.21.15.12:0/424129507 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:14:28.356 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:28.345+0000 7f7397e00640 1 -- 172.21.15.12:0/424129507 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a6f11390c0 msgr2=0x55a6f113c420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:14:28.356 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:28.345+0000 7f7397e00640 1 --2- 172.21.15.12:0/424129507 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a6f11390c0 0x55a6f113c420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:14:28.356 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:28.345+0000 7f7397e00640 1 -- 172.21.15.12:0/424129507 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a6f113bb00 msgr2=0x55a6f113fea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:14:28.356 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:28.345+0000 7f7397e00640 1 --2- 172.21.15.12:0/424129507 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a6f113bb00 0x55a6f113fea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:14:28.356 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:28.345+0000 7f7397e00640 1 -- 172.21.15.12:0/424129507 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a6f113f9a0 con 0x55a6f1136790 2024-09-11T00:14:28.357 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:28.345+0000 7f738ea00640 1 --2- 172.21.15.12:0/424129507 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a6f11390c0 0x55a6f113c420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:14:28.357 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:28.345+0000 7f7397e00640 1 --2- 172.21.15.12:0/424129507 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a6f1136790 0x55a6f113bee0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x55a6f1140bf0 tx=0x7f7390009a50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ba25d5de2b6379f3 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:14:28.357 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:28.348+0000 7f7387400640 1 -- 172.21.15.12:0/424129507 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73900053a0 con 0x55a6f1136790 2024-09-11T00:14:28.357 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:28.348+0000 7f7387400640 1 -- 172.21.15.12:0/424129507 <== mon.2 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7390004050 con 0x55a6f1136790 2024-09-11T00:14:28.357 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:28.348+0000 7f7387400640 1 -- 172.21.15.12:0/424129507 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f739000fb50 con 0x55a6f1136790 2024-09-11T00:14:28.357 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:28.349+0000 7f73a0df4740 1 -- 172.21.15.12:0/424129507 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a6f1136790 msgr2=0x55a6f113bee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:14:28.357 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:28.349+0000 7f73a0df4740 1 --2- 172.21.15.12:0/424129507 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a6f1136790 0x55a6f113bee0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x55a6f1140bf0 tx=0x7f7390009a50 comp rx=0 tx=0).stop 2024-09-11T00:14:28.357 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:28.349+0000 7f73a0df4740 1 -- 172.21.15.12:0/424129507 shutdown_connections 2024-09-11T00:14:28.357 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:28.349+0000 7f73a0df4740 1 --2- 172.21.15.12:0/424129507 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a6f113bb00 0x55a6f113fea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:14:28.357 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:28.349+0000 7f73a0df4740 1 --2- 172.21.15.12:0/424129507 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a6f11390c0 0x55a6f113c420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:14:28.357 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:28.349+0000 7f73a0df4740 1 --2- 172.21.15.12:0/424129507 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a6f1136790 0x55a6f113bee0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:14:28.357 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:28.349+0000 7f73a0df4740 1 -- 172.21.15.12:0/424129507 >> 172.21.15.12:0/424129507 conn(0x55a6f11318e0 msgr2=0x55a6f1132330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:14:28.357 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:28.349+0000 7f73a0df4740 1 -- 172.21.15.12:0/424129507 shutdown_connections 2024-09-11T00:14:28.358 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:28.349+0000 7f73a0df4740 1 -- 172.21.15.12:0/424129507 wait complete. 2024-09-11T00:14:28.358 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:28.349+0000 7f73a0df4740 1 -- start start 2024-09-11T00:14:28.358 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:28.349+0000 7f73a0df4740 1 Processor -- start 2024-09-11T00:14:28.358 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:28.350+0000 7f73a0df4740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a6f1136790 0x55a6f1146d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:14:28.358 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:28.350+0000 7f73a0df4740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a6f11390c0 0x55a6f1147260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:14:28.358 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:28.350+0000 7f73a0df4740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a6f113bb00 0x55a6f1145840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:14:28.358 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:28.350+0000 7f73a0df4740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f73981aee50 con 0x55a6f1136790 2024-09-11T00:14:28.358 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:28.350+0000 7f73a0df4740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f73981ae780 con 0x55a6f11390c0 2024-09-11T00:14:28.358 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:28.350+0000 7f73a0df4740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f73981ab990 con 0x55a6f113bb00 2024-09-11T00:14:28.358 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:28.350+0000 7f73a0df4740 10 client.0 _ll_register_callbacks cb 0x7f7380009930 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:14:28.358 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:28.350+0000 7f738ea00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a6f11390c0 0x55a6f1147260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:14:28.358 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:28.350+0000 7f7397e00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a6f1136790 0x55a6f1146d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:14:28.358 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:28.350+0000 7f738ea00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a6f11390c0 0x55a6f1147260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.12:36004/0 (socket says 172.21.15.12:36004) 2024-09-11T00:14:28.359 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:28.350+0000 7f738ea00640 1 -- 172.21.15.12:0/2696216550 learned_addr learned my addr 172.21.15.12:0/2696216550 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:14:28.359 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:28.350+0000 7f7397e00640 1 --2- 172.21.15.12:0/2696216550 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a6f113bb00 0x55a6f1145840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:14:28.359 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:28.351+0000 7f738ea00640 1 -- 172.21.15.12:0/2696216550 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a6f113bb00 msgr2=0x55a6f1145840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:14:28.359 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:28.351+0000 7f738ea00640 1 --2- 172.21.15.12:0/2696216550 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a6f113bb00 0x55a6f1145840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:14:28.359 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:28.351+0000 7f738ea00640 1 -- 172.21.15.12:0/2696216550 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a6f1136790 msgr2=0x55a6f1146d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:14:28.359 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:28.351+0000 7f738ea00640 1 --2- 172.21.15.12:0/2696216550 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a6f1136790 0x55a6f1146d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:14:28.359 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:28.351+0000 7f738ea00640 1 -- 172.21.15.12:0/2696216550 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a6f113c960 con 0x55a6f11390c0 2024-09-11T00:14:28.359 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:28.351+0000 7f7397e00640 1 --2- 172.21.15.12:0/2696216550 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a6f113bb00 0x55a6f1145840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T00:14:28.359 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:28.351+0000 7f738ea00640 1 --2- 172.21.15.12:0/2696216550 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a6f11390c0 0x55a6f1147260 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f7380009e00 tx=0x7f738001cd30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=97a462baf40ca225 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:14:28.359 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:28.351+0000 7f7384c00640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-09-11T00:14:28.359 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:28.351+0000 7f7384c00640 1 -- 172.21.15.12:0/2696216550 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7380009e30 con 0x55a6f11390c0 2024-09-11T00:14:28.359 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:28.351+0000 7f73a0df4740 1 -- 172.21.15.12:0/2696216550 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7390004050 con 0x55a6f11390c0 2024-09-11T00:14:28.359 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:28.351+0000 7f7384c00640 1 -- 172.21.15.12:0/2696216550 <== mon.2 v2:172.21.15.165:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9208+0+0 (secure 0 0 0) 0x7f738001c630 con 0x55a6f11390c0 2024-09-11T00:14:28.360 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:28.351+0000 7f7384c00640 1 -- 172.21.15.12:0/2696216550 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f73800050d0 con 0x55a6f11390c0 2024-09-11T00:14:28.360 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:28.351+0000 7f73a0df4740 10 client.26369 fetch_fsmap learned FSMap version 40 2024-09-11T00:14:28.360 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:28.351+0000 7f73a0df4740 1 -- 172.21.15.12:0/2696216550 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x55a6f11b8d00 con 0x55a6f11390c0 2024-09-11T00:14:28.360 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:28.352+0000 7f7384c00640 1 -- 172.21.15.12:0/2696216550 <== mon.2 v2:172.21.15.165:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7390004050 con 0x55a6f11390c0 2024-09-11T00:14:28.360 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:28.352+0000 7f7384c00640 1 -- 172.21.15.12:0/2696216550 <== mon.2 v2:172.21.15.165:3300/0 5 ==== fsmap(e 40) ==== 8374+0+0 (secure 0 0 0) 0x7f7380003c40 con 0x55a6f11390c0 2024-09-11T00:14:28.360 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:28.352+0000 7f73a0df4740 10 client.26369 fetch_fsmap finished waiting for FSMap version 40 2024-09-11T00:14:28.360 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:28.352+0000 7f73a0df4740 10 client.26369 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T00:14:28.360 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:28.352+0000 7f73a0df4740 20 client.26369 populate_metadata read hostname 'smithi012' 2024-09-11T00:14:28.360 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:28.352+0000 7f73a0df4740 1 --2- 172.21.15.12:0/2696216550 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55a6f11b96d0 0x55a6f11bbaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:14:28.360 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:28.352+0000 7f73a0df4740 4 client.26369 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T00:14:28.360 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:28.352+0000 7f73a0df4740 1 -- 172.21.15.12:0/2696216550 --> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55a6f10a70b0 con 0x55a6f11b96d0 2024-09-11T00:14:28.360 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:28.352+0000 7f7397e00640 1 --2- 172.21.15.12:0/2696216550 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55a6f11b96d0 0x55a6f11bbaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:14:28.360 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:28.353+0000 7f7397e00640 1 --2- 172.21.15.12:0/2696216550 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55a6f11b96d0 0x55a6f11bbaf0 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=57c338ac447007fc server_cookie=f8fcf7dfebfb1849 in_seq=0 out_seq=0 2024-09-11T00:14:28.361 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:28.353+0000 7f7384c00640 10 client.26369 ms_handle_connect on v2:172.21.15.165:6838/501312984 2024-09-11T00:14:28.399 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:28.397+0000 7f7384c00640 1 -- 172.21.15.12:0/2696216550 <== mds.0 v2:172.21.15.165:6838/501312984 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55a6f10a70b0 con 0x55a6f11b96d0 2024-09-11T00:14:28.400 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:28.397+0000 7f7384c00640 10 client.26369 handle_command_reply: tid=0 2024-09-11T00:14:28.400 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:28.397+0000 7f73a0df4740 1 client.26369 shutdown 2024-09-11T00:14:28.400 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:28.397+0000 7f73a0df4740 1 -- 172.21.15.12:0/2696216550 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a6f11390c0 msgr2=0x55a6f1147260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:14:28.400 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:28.397+0000 7f73a0df4740 1 --2- 172.21.15.12:0/2696216550 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a6f11390c0 0x55a6f1147260 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f7380009e00 tx=0x7f738001cd30 comp rx=0 tx=0).stop 2024-09-11T00:14:28.400 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:28.397+0000 7f73a0df4740 1 -- 172.21.15.12:0/2696216550 shutdown_connections 2024-09-11T00:14:28.400 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:28.397+0000 7f73a0df4740 1 --2- 172.21.15.12:0/2696216550 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55a6f11b96d0 0x55a6f11bbaf0 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:14:28.400 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:28.397+0000 7f73a0df4740 1 --2- 172.21.15.12:0/2696216550 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a6f113bb00 0x55a6f1145840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:14:28.400 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:28.397+0000 7f73a0df4740 1 --2- 172.21.15.12:0/2696216550 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a6f11390c0 0x55a6f1147260 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:14:28.400 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:28.397+0000 7f73a0df4740 1 --2- 172.21.15.12:0/2696216550 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a6f1136790 0x55a6f1146d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:14:28.400 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:28.397+0000 7f7384c00640 0 client.26369 ms_handle_reset on v2:172.21.15.165:6838/501312984 2024-09-11T00:14:28.400 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:28.397+0000 7f73a0df4740 1 -- 172.21.15.12:0/2696216550 >> 172.21.15.12:0/2696216550 conn(0x55a6f11318e0 msgr2=0x55a6f1135af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:14:28.400 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:28.397+0000 7f73a0df4740 1 -- 172.21.15.12:0/2696216550 shutdown_connections 2024-09-11T00:14:28.401 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:28.397+0000 7f73a0df4740 1 -- 172.21.15.12:0/2696216550 wait complete. 2024-09-11T00:14:28.401 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:28.397+0000 7f73a0df4740 20 client.26369 trim_cache size 0 max 16384 2024-09-11T00:14:28.401 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:{ 2024-09-11T00:14:28.401 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "message": "", 2024-09-11T00:14:28.401 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "return_code": 0 2024-09-11T00:14:28.401 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:} 2024-09-11T00:14:28.402 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:28.398+0000 7f739e000640 1 -- 172.21.15.12:0/1528562074 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f737c0b52e0 msgr2=0x7f737c0b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:14:28.402 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:28.398+0000 7f739e000640 1 --2- 172.21.15.12:0/1528562074 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f737c0b52e0 0x7f737c0b77a0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f73880c91f0 tx=0x7f7388009260 comp rx=0 tx=0).stop 2024-09-11T00:14:28.402 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:28.398+0000 7f739e000640 1 -- 172.21.15.12:0/1528562074 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7398104b90 msgr2=0x7f739819af60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:14:28.402 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:28.398+0000 7f739e000640 1 --2- 172.21.15.12:0/1528562074 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7398104b90 0x7f739819af60 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f7388009230 tx=0x7f73880042f0 comp rx=0 tx=0).stop 2024-09-11T00:14:28.402 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:28.398+0000 7f739e000640 1 -- 172.21.15.12:0/1528562074 shutdown_connections 2024-09-11T00:14:28.402 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:28.398+0000 7f739e000640 1 --2- 172.21.15.12:0/1528562074 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f737c0b52e0 0x7f737c0b77a0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:14:28.402 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:28.398+0000 7f739e000640 1 --2- 172.21.15.12:0/1528562074 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f73981066a0 0x7f739819c1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:14:28.402 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:28.398+0000 7f739e000640 1 --2- 172.21.15.12:0/1528562074 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7398105ce0 0x7f739819b5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:14:28.402 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:28.398+0000 7f739e000640 1 --2- 172.21.15.12:0/1528562074 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7398104b90 0x7f739819af60 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:14:28.403 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:28.398+0000 7f739e000640 1 -- 172.21.15.12:0/1528562074 >> 172.21.15.12:0/1528562074 conn(0x7f7398100470 msgr2=0x7f73981090d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:14:28.403 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:28.398+0000 7f739e000640 1 -- 172.21.15.12:0/1528562074 shutdown_connections 2024-09-11T00:14:28.403 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:28.398+0000 7f739e000640 1 -- 172.21.15.12:0/1528562074 wait complete. 2024-09-11T00:14:28.655 INFO:tasks.workunit.client.0.smithi012.stdout: 1 255065 3.22 MB/sec execute 265 sec latency 22.684 ms 2024-09-11T00:14:29.655 INFO:tasks.workunit.client.0.smithi012.stdout: 1 255990 3.23 MB/sec execute 266 sec latency 105.456 ms 2024-09-11T00:14:29.782 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:14:29 smithi165 ceph-mon[35356]: pgmap v621: 129 pgs: 129 active+clean; 62 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 1.0 MiB/s wr, 154 op/s 2024-09-11T00:14:29.813 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:14:29 smithi012 ceph-mon[30333]: pgmap v621: 129 pgs: 129 active+clean; 62 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 1.0 MiB/s wr, 154 op/s 2024-09-11T00:14:29.830 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:14:29 smithi178 ceph-mon[35919]: pgmap v621: 129 pgs: 129 active+clean; 62 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 1.0 MiB/s wr, 154 op/s 2024-09-11T00:14:30.655 INFO:tasks.workunit.client.0.smithi012.stdout: 1 258181 3.26 MB/sec execute 267 sec latency 10.461 ms 2024-09-11T00:14:31.655 INFO:tasks.workunit.client.0.smithi012.stdout: 1 259416 3.28 MB/sec execute 268 sec latency 80.295 ms 2024-09-11T00:14:31.782 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:14:31 smithi165 ceph-mon[35356]: pgmap v622: 129 pgs: 129 active+clean; 59 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 1.2 MiB/s wr, 214 op/s 2024-09-11T00:14:31.813 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:14:31 smithi012 ceph-mon[30333]: pgmap v622: 129 pgs: 129 active+clean; 59 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 1.2 MiB/s wr, 214 op/s 2024-09-11T00:14:31.831 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:14:31 smithi178 ceph-mon[35919]: pgmap v622: 129 pgs: 129 active+clean; 59 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 1.2 MiB/s wr, 214 op/s 2024-09-11T00:14:32.655 INFO:tasks.workunit.client.0.smithi012.stdout: 1 261454 3.30 MB/sec execute 269 sec latency 12.336 ms 2024-09-11T00:14:32.782 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:14:32 smithi165 ceph-mon[35356]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T00:14:32.813 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:14:32 smithi012 ceph-mon[30333]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T00:14:32.830 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:14:32 smithi178 ceph-mon[35919]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T00:14:33.655 INFO:tasks.workunit.client.0.smithi012.stdout: 1 262767 3.33 MB/sec execute 270 sec latency 277.750 ms 2024-09-11T00:14:33.782 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:14:33 smithi165 ceph-mon[35356]: pgmap v623: 129 pgs: 129 active+clean; 61 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 1.5 MiB/s wr, 288 op/s 2024-09-11T00:14:33.812 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:14:33 smithi012 ceph-mon[30333]: pgmap v623: 129 pgs: 129 active+clean; 61 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 1.5 MiB/s wr, 288 op/s 2024-09-11T00:14:33.831 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:14:33 smithi178 ceph-mon[35919]: pgmap v623: 129 pgs: 129 active+clean; 61 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 1.5 MiB/s wr, 288 op/s 2024-09-11T00:14:34.655 INFO:tasks.workunit.client.0.smithi012.stdout: 1 264207 3.33 MB/sec execute 271 sec latency 10.574 ms 2024-09-11T00:14:35.655 INFO:tasks.workunit.client.0.smithi012.stdout: 1 266119 3.37 MB/sec execute 272 sec latency 230.631 ms 2024-09-11T00:14:35.782 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:14:35 smithi165 ceph-mon[35356]: pgmap v624: 129 pgs: 129 active+clean; 61 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 1.4 MiB/s wr, 280 op/s 2024-09-11T00:14:35.812 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:14:35 smithi012 ceph-mon[30333]: pgmap v624: 129 pgs: 129 active+clean; 61 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 1.4 MiB/s wr, 280 op/s 2024-09-11T00:14:35.830 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:14:35 smithi178 ceph-mon[35919]: pgmap v624: 129 pgs: 129 active+clean; 61 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 1.4 MiB/s wr, 280 op/s 2024-09-11T00:14:36.655 INFO:tasks.workunit.client.0.smithi012.stdout: 1 266935 3.36 MB/sec execute 273 sec latency 126.783 ms 2024-09-11T00:14:37.656 INFO:tasks.workunit.client.0.smithi012.stdout: 1 269328 3.40 MB/sec execute 274 sec latency 13.080 ms 2024-09-11T00:14:37.782 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:14:37 smithi165 ceph-mon[35356]: pgmap v625: 129 pgs: 129 active+clean; 65 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 21 MiB/s rd, 1.7 MiB/s wr, 380 op/s 2024-09-11T00:14:37.813 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:14:37 smithi012 ceph-mon[30333]: pgmap v625: 129 pgs: 129 active+clean; 65 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 21 MiB/s rd, 1.7 MiB/s wr, 380 op/s 2024-09-11T00:14:37.830 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:14:37 smithi178 ceph-mon[35919]: pgmap v625: 129 pgs: 129 active+clean; 65 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 21 MiB/s rd, 1.7 MiB/s wr, 380 op/s 2024-09-11T00:14:38.655 INFO:tasks.workunit.client.0.smithi012.stdout: 1 270422 3.41 MB/sec execute 275 sec latency 11.305 ms 2024-09-11T00:14:39.631 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:14:39 smithi178 ceph-mon[35919]: pgmap v626: 129 pgs: 129 active+clean; 65 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 25 MiB/s rd, 1.0 MiB/s wr, 313 op/s 2024-09-11T00:14:39.656 INFO:tasks.workunit.client.0.smithi012.stdout: 1 272789 3.44 MB/sec execute 276 sec latency 11.002 ms 2024-09-11T00:14:39.782 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:14:39 smithi165 ceph-mon[35356]: pgmap v626: 129 pgs: 129 active+clean; 65 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 25 MiB/s rd, 1.0 MiB/s wr, 313 op/s 2024-09-11T00:14:39.813 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:14:39 smithi012 ceph-mon[30333]: pgmap v626: 129 pgs: 129 active+clean; 65 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 25 MiB/s rd, 1.0 MiB/s wr, 313 op/s 2024-09-11T00:14:40.656 INFO:tasks.workunit.client.0.smithi012.stdout: 1 273313 3.44 MB/sec execute 277 sec latency 354.073 ms 2024-09-11T00:14:41.523 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:41.520+0000 7f2bf9c00640 1 -- 172.21.15.12:0/2624280983 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2bf4102970 msgr2=0x7f2bf4102d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:14:41.523 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:41.520+0000 7f2bf9c00640 1 --2- 172.21.15.12:0/2624280983 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2bf4102970 0x7f2bf4102d70 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f2be4009900 tx=0x7f2be402a3f0 comp rx=0 tx=0).stop 2024-09-11T00:14:41.523 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:41.520+0000 7f2bf9c00640 1 -- 172.21.15.12:0/2624280983 shutdown_connections 2024-09-11T00:14:41.523 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:41.520+0000 7f2bf9c00640 1 --2- 172.21.15.12:0/2624280983 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2bf4104480 0x7f2bf410ae00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:14:41.523 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:41.520+0000 7f2bf9c00640 1 --2- 172.21.15.12:0/2624280983 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2bf4103ac0 0x7f2bf4103f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:14:41.523 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:41.520+0000 7f2bf9c00640 1 --2- 172.21.15.12:0/2624280983 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2bf4102970 0x7f2bf4102d70 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:14:41.523 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:41.520+0000 7f2bf9c00640 1 -- 172.21.15.12:0/2624280983 >> 172.21.15.12:0/2624280983 conn(0x7f2bf40fe250 msgr2=0x7f2bf4100670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:14:41.524 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:41.520+0000 7f2bf9c00640 1 -- 172.21.15.12:0/2624280983 shutdown_connections 2024-09-11T00:14:41.524 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:41.521+0000 7f2bf9c00640 1 -- 172.21.15.12:0/2624280983 wait complete. 2024-09-11T00:14:41.526 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:41.521+0000 7f2bf9c00640 1 Processor -- start 2024-09-11T00:14:41.526 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:41.522+0000 7f2bf9c00640 1 -- start start 2024-09-11T00:14:41.526 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:41.522+0000 7f2bf9c00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2bf4102970 0x7f2bf4076ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:14:41.526 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:41.522+0000 7f2bf9c00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2bf4103ac0 0x7f2bf4075540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:14:41.526 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:41.522+0000 7f2bf9c00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2bf4104480 0x7f2bf4075a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:14:41.526 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:41.522+0000 7f2bf9c00640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f2bf410e630 con 0x7f2bf4104480 2024-09-11T00:14:41.526 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:41.522+0000 7f2bf9c00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f2bf410e4b0 con 0x7f2bf4102970 2024-09-11T00:14:41.526 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:41.522+0000 7f2bf9c00640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f2bf410e7b0 con 0x7f2bf4103ac0 2024-09-11T00:14:41.526 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:41.522+0000 7f2bf2a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2bf4102970 0x7f2bf4076ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:14:41.526 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:41.522+0000 7f2bf2a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2bf4102970 0x7f2bf4076ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.12:36156/0 (socket says 172.21.15.12:36156) 2024-09-11T00:14:41.526 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:41.522+0000 7f2bf2a00640 1 -- 172.21.15.12:0/2387011845 learned_addr learned my addr 172.21.15.12:0/2387011845 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:14:41.527 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:41.522+0000 7f2bf2000640 1 --2- 172.21.15.12:0/2387011845 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2bf4103ac0 0x7f2bf4075540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:14:41.527 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:41.522+0000 7f2bf3400640 1 --2- 172.21.15.12:0/2387011845 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2bf4104480 0x7f2bf4075a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:14:41.527 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:41.523+0000 7f2bf2a00640 1 -- 172.21.15.12:0/2387011845 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2bf4103ac0 msgr2=0x7f2bf4075540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:14:41.527 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:41.523+0000 7f2bf2a00640 1 --2- 172.21.15.12:0/2387011845 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2bf4103ac0 0x7f2bf4075540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:14:41.527 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:41.523+0000 7f2bf2a00640 1 -- 172.21.15.12:0/2387011845 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2bf4104480 msgr2=0x7f2bf4075a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:14:41.527 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:41.523+0000 7f2bf2a00640 1 --2- 172.21.15.12:0/2387011845 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2bf4104480 0x7f2bf4075a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:14:41.527 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:41.523+0000 7f2bf2a00640 1 -- 172.21.15.12:0/2387011845 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2bf40762f0 con 0x7f2bf4102970 2024-09-11T00:14:41.527 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:41.523+0000 7f2bf3400640 1 --2- 172.21.15.12:0/2387011845 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2bf4104480 0x7f2bf4075a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T00:14:41.527 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:41.523+0000 7f2bf2000640 1 --2- 172.21.15.12:0/2387011845 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2bf4103ac0 0x7f2bf4075540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:14:41.527 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:41.523+0000 7f2bf2a00640 1 --2- 172.21.15.12:0/2387011845 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2bf4102970 0x7f2bf4076ce0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f2be4031490 tx=0x7f2be4003c10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:14:41.527 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:41.523+0000 7f2beb400640 1 -- 172.21.15.12:0/2387011845 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2be403f070 con 0x7f2bf4102970 2024-09-11T00:14:41.528 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:41.523+0000 7f2beb400640 1 -- 172.21.15.12:0/2387011845 <== mon.2 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2be4031bf0 con 0x7f2bf4102970 2024-09-11T00:14:41.528 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:41.523+0000 7f2bf9c00640 1 -- 172.21.15.12:0/2387011845 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2bf41a2e30 con 0x7f2bf4102970 2024-09-11T00:14:41.528 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:41.523+0000 7f2beb400640 1 -- 172.21.15.12:0/2387011845 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2be4035030 con 0x7f2bf4102970 2024-09-11T00:14:41.528 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:41.524+0000 7f2bf9c00640 1 -- 172.21.15.12:0/2387011845 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2bf41a3240 con 0x7f2bf4102970 2024-09-11T00:14:41.528 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:41.527+0000 7f2bfcab5740 1 Processor -- start 2024-09-11T00:14:41.529 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:41.527+0000 7f2bfcab5740 1 -- start start 2024-09-11T00:14:41.529 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:41.527+0000 7f2bfcab5740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561ebd4e2790 0x561ebd4e4b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:14:41.529 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:41.527+0000 7f2bfcab5740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561ebd4e50c0 0x561ebd4e74b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:14:41.530 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:41.527+0000 7f2bfcab5740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561ebd4e7b00 0x561ebd4e9fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:14:41.530 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:41.527+0000 7f2bfcab5740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x561ebd4556b0 con 0x561ebd4e50c0 2024-09-11T00:14:41.530 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:41.527+0000 7f2bfcab5740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x561ebd444ae0 con 0x561ebd4e2790 2024-09-11T00:14:41.530 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:41.527+0000 7f2bfcab5740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x561ebd4488f0 con 0x561ebd4e7b00 2024-09-11T00:14:41.531 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:41.528+0000 7f2beb400640 1 -- 172.21.15.12:0/2387011845 <== mon.2 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7f2be403b460 con 0x7f2bf4102970 2024-09-11T00:14:41.531 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:41.528+0000 7f2bf3400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561ebd4e2790 0x561ebd4e4b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:14:41.531 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:41.528+0000 7f2bf2000640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561ebd4e7b00 0x561ebd4e9fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:14:41.531 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:41.528+0000 7f2bf3400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561ebd4e2790 0x561ebd4e4b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.12:36168/0 (socket says 172.21.15.12:36168) 2024-09-11T00:14:41.531 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:41.528+0000 7f2bf3400640 1 -- 172.21.15.12:0/1667560050 learned_addr learned my addr 172.21.15.12:0/1667560050 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:14:41.531 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:41.528+0000 7f2bf2a00640 1 --2- 172.21.15.12:0/1667560050 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561ebd4e50c0 0x561ebd4e74b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:14:41.532 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:41.528+0000 7f2bf2000640 1 -- 172.21.15.12:0/1667560050 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561ebd4e2790 msgr2=0x561ebd4e4b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:14:41.532 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:41.528+0000 7f2bf2000640 1 --2- 172.21.15.12:0/1667560050 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561ebd4e2790 0x561ebd4e4b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:14:41.532 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:41.528+0000 7f2bf2000640 1 -- 172.21.15.12:0/1667560050 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561ebd4e50c0 msgr2=0x561ebd4e74b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:14:41.532 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:41.528+0000 7f2bf2000640 1 --2- 172.21.15.12:0/1667560050 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561ebd4e50c0 0x561ebd4e74b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:14:41.532 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:41.528+0000 7f2bf2000640 1 -- 172.21.15.12:0/1667560050 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561ebd411780 con 0x561ebd4e7b00 2024-09-11T00:14:41.532 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:41.528+0000 7f2bf2a00640 1 --2- 172.21.15.12:0/1667560050 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561ebd4e50c0 0x561ebd4e74b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T00:14:41.532 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:41.528+0000 7f2bf2000640 1 --2- 172.21.15.12:0/1667560050 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561ebd4e7b00 0x561ebd4e9fc0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f2be0009fd0 tx=0x7f2be00118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2b290ba57ad3bde9 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:14:41.532 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:41.529+0000 7f2bdfe00640 1 -- 172.21.15.12:0/1667560050 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2be0004310 con 0x561ebd4e7b00 2024-09-11T00:14:41.532 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:41.529+0000 7f2bdfe00640 1 -- 172.21.15.12:0/1667560050 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2be00044b0 con 0x561ebd4e7b00 2024-09-11T00:14:41.532 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:41.529+0000 7f2bdfe00640 1 -- 172.21.15.12:0/1667560050 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2be00035f0 con 0x561ebd4e7b00 2024-09-11T00:14:41.532 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:41.529+0000 7f2bfcab5740 1 -- 172.21.15.12:0/1667560050 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561ebd4e7b00 msgr2=0x561ebd4e9fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:14:41.532 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:41.529+0000 7f2bfcab5740 1 --2- 172.21.15.12:0/1667560050 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561ebd4e7b00 0x561ebd4e9fc0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f2be0009fd0 tx=0x7f2be00118d0 comp rx=0 tx=0).stop 2024-09-11T00:14:41.535 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:41.530+0000 7f2bfcab5740 1 -- 172.21.15.12:0/1667560050 shutdown_connections 2024-09-11T00:14:41.535 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:41.530+0000 7f2beb400640 1 --2- 172.21.15.12:0/2387011845 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f2bd80b5360 0x7f2bd80b7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:14:41.535 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:41.530+0000 7f2bfcab5740 1 --2- 172.21.15.12:0/1667560050 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561ebd4e7b00 0x561ebd4e9fc0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:14:41.535 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:41.530+0000 7f2bfcab5740 1 --2- 172.21.15.12:0/1667560050 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561ebd4e50c0 0x561ebd4e74b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:14:41.535 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:41.530+0000 7f2bfcab5740 1 --2- 172.21.15.12:0/1667560050 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561ebd4e2790 0x561ebd4e4b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:14:41.535 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:41.530+0000 7f2bfcab5740 1 -- 172.21.15.12:0/1667560050 >> 172.21.15.12:0/1667560050 conn(0x561ebd4dd8e0 msgr2=0x561ebd4de670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:14:41.535 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:41.530+0000 7f2beb400640 1 -- 172.21.15.12:0/2387011845 <== mon.2 v2:172.21.15.165:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9208+0+0 (secure 0 0 0) 0x7f2be4106a70 con 0x7f2bf4102970 2024-09-11T00:14:41.535 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:41.530+0000 7f2bfcab5740 1 -- 172.21.15.12:0/1667560050 shutdown_connections 2024-09-11T00:14:41.535 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:41.530+0000 7f2bf3400640 1 --2- 172.21.15.12:0/2387011845 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f2bd80b5360 0x7f2bd80b7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:14:41.536 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:41.530+0000 7f2bfcab5740 1 -- 172.21.15.12:0/1667560050 wait complete. 2024-09-11T00:14:41.536 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:41.531+0000 7f2bfcab5740 1 -- start start 2024-09-11T00:14:41.536 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:41.531+0000 7f2bf3400640 1 --2- 172.21.15.12:0/2387011845 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f2bd80b5360 0x7f2bd80b7820 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f2bec005f10 tx=0x7f2bec009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:14:41.536 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:41.531+0000 7f2bfcab5740 1 Processor -- start 2024-09-11T00:14:41.536 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:41.531+0000 7f2bfcab5740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561ebd4e2790 0x561ebd4ebe30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:14:41.536 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:41.531+0000 7f2bfcab5740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561ebd4e50c0 0x561ebd4ec4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:14:41.536 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:41.531+0000 7f2bfcab5740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561ebd4e7b00 0x561ebd4ed0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:14:41.536 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:41.531+0000 7f2bfcab5740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x561ebd37cab0 con 0x561ebd4e7b00 2024-09-11T00:14:41.536 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:41.532+0000 7f2bfcab5740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x561ebd3eb570 con 0x561ebd4e50c0 2024-09-11T00:14:41.536 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:41.532+0000 7f2bfcab5740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x561ebd4eaae0 con 0x561ebd4e2790 2024-09-11T00:14:41.536 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:41.532+0000 7f2bfcab5740 10 client.0 _ll_register_callbacks cb 0x561ebd3fa100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:14:41.536 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:41.532+0000 7f2bf2000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561ebd4e50c0 0x561ebd4ec4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:14:41.537 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:41.532+0000 7f2bf3400640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561ebd4e7b00 0x561ebd4ed0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:14:41.537 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:41.532+0000 7f2bf2000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561ebd4e50c0 0x561ebd4ec4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.12:36172/0 (socket says 172.21.15.12:36172) 2024-09-11T00:14:41.537 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:41.532+0000 7f2bf2000640 1 -- 172.21.15.12:0/2992652882 learned_addr learned my addr 172.21.15.12:0/2992652882 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:14:41.537 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:41.532+0000 7f2bf2a00640 1 --2- 172.21.15.12:0/2992652882 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561ebd4e2790 0x561ebd4ebe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:14:41.537 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:41.532+0000 7f2bf2000640 1 -- 172.21.15.12:0/2992652882 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561ebd4e2790 msgr2=0x561ebd4ebe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:14:41.537 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:41.532+0000 7f2bf2000640 1 --2- 172.21.15.12:0/2992652882 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561ebd4e2790 0x561ebd4ebe30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:14:41.537 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:41.532+0000 7f2bf2000640 1 -- 172.21.15.12:0/2992652882 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561ebd4e7b00 msgr2=0x561ebd4ed0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:14:41.537 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:41.532+0000 7f2bf2000640 1 --2- 172.21.15.12:0/2992652882 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561ebd4e7b00 0x561ebd4ed0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:14:41.537 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:41.532+0000 7f2bf2000640 1 -- 172.21.15.12:0/2992652882 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561ebd1a6790 con 0x561ebd4e50c0 2024-09-11T00:14:41.537 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:41.533+0000 7f2bf2000640 1 --2- 172.21.15.12:0/2992652882 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561ebd4e50c0 0x561ebd4ec4d0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f2be0004410 tx=0x7f2be0004440 comp rx=0 tx=0).ready entity=mon.2 client_cookie=71572d4edf644257 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:14:41.537 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:41.533+0000 7f2bdcc00640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-09-11T00:14:41.537 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:41.533+0000 7f2bdcc00640 1 -- 172.21.15.12:0/2992652882 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2be0029070 con 0x561ebd4e50c0 2024-09-11T00:14:41.538 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:41.533+0000 7f2bfcab5740 1 -- 172.21.15.12:0/2992652882 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2be00044b0 con 0x561ebd4e50c0 2024-09-11T00:14:41.538 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:41.533+0000 7f2bdcc00640 1 -- 172.21.15.12:0/2992652882 <== mon.2 v2:172.21.15.165:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9208+0+0 (secure 0 0 0) 0x7f2be0003bb0 con 0x561ebd4e50c0 2024-09-11T00:14:41.540 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:41.533+0000 7f2bdcc00640 1 -- 172.21.15.12:0/2992652882 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f2be0035050 con 0x561ebd4e50c0 2024-09-11T00:14:41.540 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:41.533+0000 7f2bfcab5740 10 client.26381 fetch_fsmap learned FSMap version 41 2024-09-11T00:14:41.540 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:41.533+0000 7f2bfcab5740 1 -- 172.21.15.12:0/2992652882 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x561ebd21f620 con 0x561ebd4e50c0 2024-09-11T00:14:41.540 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:41.534+0000 7f2bdcc00640 1 -- 172.21.15.12:0/2992652882 <== mon.2 v2:172.21.15.165:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2be00044b0 con 0x561ebd4e50c0 2024-09-11T00:14:41.540 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:41.534+0000 7f2bdcc00640 1 -- 172.21.15.12:0/2992652882 <== mon.2 v2:172.21.15.165:3300/0 5 ==== fsmap(e 41) ==== 8370+0+0 (secure 0 0 0) 0x7f2be0004c40 con 0x561ebd4e50c0 2024-09-11T00:14:41.540 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:41.534+0000 7f2bfcab5740 10 client.26381 fetch_fsmap finished waiting for FSMap version 41 2024-09-11T00:14:41.540 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:41.534+0000 7f2bfcab5740 10 client.26381 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T00:14:41.540 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:41.534+0000 7f2bfcab5740 20 client.26381 populate_metadata read hostname 'smithi012' 2024-09-11T00:14:41.540 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:41.534+0000 7f2bfcab5740 1 --2- 172.21.15.12:0/2992652882 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x561ebd4f7450 0x561ebd4f9860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:14:41.540 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:41.534+0000 7f2bfcab5740 4 client.26381 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T00:14:41.541 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:41.534+0000 7f2bfcab5740 1 -- 172.21.15.12:0/2992652882 --> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561ebd457600 con 0x561ebd4f7450 2024-09-11T00:14:41.541 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:41.535+0000 7f2bf2a00640 1 --2- 172.21.15.12:0/2992652882 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x561ebd4f7450 0x561ebd4f9860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:14:41.541 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:41.535+0000 7f2bf2a00640 1 --2- 172.21.15.12:0/2992652882 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x561ebd4f7450 0x561ebd4f9860 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5f6af473d0036e01 server_cookie=c62dd7553e741957 in_seq=0 out_seq=0 2024-09-11T00:14:41.541 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:41.535+0000 7f2bdcc00640 10 client.26381 ms_handle_connect on v2:172.21.15.165:6838/501312984 2024-09-11T00:14:41.541 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:41.536+0000 7f2bdcc00640 1 -- 172.21.15.12:0/2992652882 <== mds.0 v2:172.21.15.165:6838/501312984 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x561ebd457600 con 0x561ebd4f7450 2024-09-11T00:14:41.541 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:41.536+0000 7f2bdcc00640 10 client.26381 handle_command_reply: tid=0 2024-09-11T00:14:41.541 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:41.536+0000 7f2bfcab5740 1 client.26381 shutdown 2024-09-11T00:14:41.541 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:41.536+0000 7f2bfcab5740 1 -- 172.21.15.12:0/2992652882 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561ebd4e50c0 msgr2=0x561ebd4ec4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:14:41.541 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:41.536+0000 7f2bfcab5740 1 --2- 172.21.15.12:0/2992652882 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561ebd4e50c0 0x561ebd4ec4d0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f2be0004410 tx=0x7f2be0004440 comp rx=0 tx=0).stop 2024-09-11T00:14:41.541 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:41.536+0000 7f2bfcab5740 1 -- 172.21.15.12:0/2992652882 shutdown_connections 2024-09-11T00:14:41.541 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:41.536+0000 7f2bfcab5740 1 --2- 172.21.15.12:0/2992652882 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x561ebd4f7450 0x561ebd4f9860 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:14:41.541 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:41.536+0000 7f2bfcab5740 1 --2- 172.21.15.12:0/2992652882 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561ebd4e7b00 0x561ebd4ed0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:14:41.541 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:41.536+0000 7f2bfcab5740 1 --2- 172.21.15.12:0/2992652882 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561ebd4e50c0 0x561ebd4ec4d0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:14:41.542 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:41.536+0000 7f2bfcab5740 1 --2- 172.21.15.12:0/2992652882 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561ebd4e2790 0x561ebd4ebe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:14:41.542 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:41.536+0000 7f2bfcab5740 1 -- 172.21.15.12:0/2992652882 >> 172.21.15.12:0/2992652882 conn(0x561ebd4dd8e0 msgr2=0x561ebd4dca60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:14:41.542 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:41.536+0000 7f2bdcc00640 0 client.26381 ms_handle_reset on v2:172.21.15.165:6838/501312984 2024-09-11T00:14:41.542 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:41.536+0000 7f2bfcab5740 1 -- 172.21.15.12:0/2992652882 shutdown_connections 2024-09-11T00:14:41.542 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:41.536+0000 7f2bfcab5740 1 -- 172.21.15.12:0/2992652882 wait complete. 2024-09-11T00:14:41.542 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:41.537+0000 7f2bfcab5740 20 client.26381 trim_cache size 0 max 16384 2024-09-11T00:14:41.555 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:41.552+0000 7f2bfcab5740 1 Processor -- start 2024-09-11T00:14:41.555 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:41.553+0000 7f2bfcab5740 1 -- start start 2024-09-11T00:14:41.555 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:41.553+0000 7f2bfcab5740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561ebd4e2790 0x561ebd4ec540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:14:41.555 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:41.553+0000 7f2bfcab5740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561ebd4e50c0 0x561ebd4eaee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:14:41.555 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:41.553+0000 7f2bfcab5740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561ebd4e7b00 0x561ebd4eb420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:14:41.555 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:41.553+0000 7f2bfcab5740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f2bf41ad910 con 0x561ebd4e2790 2024-09-11T00:14:41.555 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:41.553+0000 7f2bfcab5740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f2bf41af450 con 0x561ebd4e7b00 2024-09-11T00:14:41.555 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:41.553+0000 7f2bfcab5740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f2bf41adfe0 con 0x561ebd4e50c0 2024-09-11T00:14:41.556 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:41.553+0000 7f2bf2a00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561ebd4e2790 0x561ebd4ec540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:14:41.557 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:41.553+0000 7f2bf3400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561ebd4e7b00 0x561ebd4eb420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:14:41.557 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:41.553+0000 7f2bf2000640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561ebd4e50c0 0x561ebd4eaee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:14:41.557 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:41.553+0000 7f2bf3400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561ebd4e7b00 0x561ebd4eb420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.12:36186/0 (socket says 172.21.15.12:36186) 2024-09-11T00:14:41.557 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:41.553+0000 7f2bf3400640 1 -- 172.21.15.12:0/1739131506 learned_addr learned my addr 172.21.15.12:0/1739131506 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:14:41.557 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:41.554+0000 7f2bf3400640 1 -- 172.21.15.12:0/1739131506 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561ebd4e50c0 msgr2=0x561ebd4eaee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:14:41.557 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:41.554+0000 7f2bf3400640 1 --2- 172.21.15.12:0/1739131506 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561ebd4e50c0 0x561ebd4eaee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:14:41.557 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:41.554+0000 7f2bf3400640 1 -- 172.21.15.12:0/1739131506 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561ebd4e2790 msgr2=0x561ebd4ec540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:14:41.557 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:41.554+0000 7f2bf3400640 1 --2- 172.21.15.12:0/1739131506 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561ebd4e2790 0x561ebd4ec540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:14:41.557 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:41.554+0000 7f2bf3400640 1 -- 172.21.15.12:0/1739131506 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561ebd4e4ef0 con 0x561ebd4e7b00 2024-09-11T00:14:41.557 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:41.554+0000 7f2bf2000640 1 --2- 172.21.15.12:0/1739131506 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561ebd4e50c0 0x561ebd4eaee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:14:41.557 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:41.554+0000 7f2bf3400640 1 --2- 172.21.15.12:0/1739131506 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561ebd4e7b00 0x561ebd4eb420 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f2bec010a90 tx=0x7f2bec019000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=90e873ded67efba5 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:14:41.558 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:41.554+0000 7f2bdf400640 1 -- 172.21.15.12:0/1739131506 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bec01e9a0 con 0x561ebd4e7b00 2024-09-11T00:14:41.558 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:41.554+0000 7f2bdf400640 1 -- 172.21.15.12:0/1739131506 <== mon.2 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2bec003970 con 0x561ebd4e7b00 2024-09-11T00:14:41.558 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:41.554+0000 7f2bdf400640 1 -- 172.21.15.12:0/1739131506 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bec004630 con 0x561ebd4e7b00 2024-09-11T00:14:41.561 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:41.555+0000 7f2bfcab5740 1 -- 172.21.15.12:0/1739131506 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561ebd4e7b00 msgr2=0x561ebd4eb420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:14:41.561 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:41.555+0000 7f2bfcab5740 1 --2- 172.21.15.12:0/1739131506 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561ebd4e7b00 0x561ebd4eb420 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f2bec010a90 tx=0x7f2bec019000 comp rx=0 tx=0).stop 2024-09-11T00:14:41.561 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:41.555+0000 7f2bfcab5740 1 -- 172.21.15.12:0/1739131506 shutdown_connections 2024-09-11T00:14:41.561 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:41.555+0000 7f2bfcab5740 1 --2- 172.21.15.12:0/1739131506 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561ebd4e7b00 0x561ebd4eb420 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:14:41.561 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:41.555+0000 7f2bfcab5740 1 --2- 172.21.15.12:0/1739131506 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561ebd4e50c0 0x561ebd4eaee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:14:41.561 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:41.555+0000 7f2bfcab5740 1 --2- 172.21.15.12:0/1739131506 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561ebd4e2790 0x561ebd4ec540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:14:41.561 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:41.555+0000 7f2bfcab5740 1 -- 172.21.15.12:0/1739131506 >> 172.21.15.12:0/1739131506 conn(0x561ebd4dd8e0 msgr2=0x561ebd4ebe30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:14:41.562 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:41.555+0000 7f2bfcab5740 1 -- 172.21.15.12:0/1739131506 shutdown_connections 2024-09-11T00:14:41.562 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:41.555+0000 7f2bfcab5740 1 -- 172.21.15.12:0/1739131506 wait complete. 2024-09-11T00:14:41.562 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:41.555+0000 7f2bfcab5740 1 -- start start 2024-09-11T00:14:41.562 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:41.556+0000 7f2bfcab5740 1 Processor -- start 2024-09-11T00:14:41.562 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:41.556+0000 7f2bfcab5740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561ebd4e2790 0x561ebd4f2a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:14:41.562 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:41.556+0000 7f2bfcab5740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561ebd4e50c0 0x561ebd4f2fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:14:41.562 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:41.556+0000 7f2bfcab5740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561ebd4e7b00 0x561ebd4f34e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:14:41.562 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:41.556+0000 7f2bfcab5740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f2bf41aed80 con 0x561ebd4e50c0 2024-09-11T00:14:41.562 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:41.556+0000 7f2bfcab5740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f2bf41ae6b0 con 0x561ebd4e2790 2024-09-11T00:14:41.562 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:41.556+0000 7f2bfcab5740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f2bf41ab8f0 con 0x561ebd4e7b00 2024-09-11T00:14:41.562 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:41.556+0000 7f2bfcab5740 10 client.0 _ll_register_callbacks cb 0x7f2bec009ab0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:14:41.562 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:41.556+0000 7f2bf2000640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561ebd4e50c0 0x561ebd4f2fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:14:41.563 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:41.556+0000 7f2bf3400640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561ebd4e7b00 0x561ebd4f34e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:14:41.563 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:41.556+0000 7f2bf2000640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561ebd4e50c0 0x561ebd4f2fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:47816/0 (socket says 172.21.15.12:47816) 2024-09-11T00:14:41.563 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:41.556+0000 7f2bf2000640 1 -- 172.21.15.12:0/3428637027 learned_addr learned my addr 172.21.15.12:0/3428637027 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:14:41.563 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:41.556+0000 7f2bf2a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561ebd4e2790 0x561ebd4f2a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:14:41.563 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:41.557+0000 7f2bf2000640 1 -- 172.21.15.12:0/3428637027 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561ebd4e7b00 msgr2=0x561ebd4f34e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:14:41.563 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:41.557+0000 7f2bf2000640 1 --2- 172.21.15.12:0/3428637027 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561ebd4e7b00 0x561ebd4f34e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:14:41.563 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:41.557+0000 7f2bf2000640 1 -- 172.21.15.12:0/3428637027 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561ebd4e2790 msgr2=0x561ebd4f2a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:14:41.563 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:41.557+0000 7f2bf2000640 1 --2- 172.21.15.12:0/3428637027 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561ebd4e2790 0x561ebd4f2a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:14:41.563 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:41.557+0000 7f2bf2000640 1 -- 172.21.15.12:0/3428637027 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561ebd4f17a0 con 0x561ebd4e50c0 2024-09-11T00:14:41.563 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:41.557+0000 7f2bf3400640 1 --2- 172.21.15.12:0/3428637027 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561ebd4e7b00 0x561ebd4f34e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:14:41.563 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:41.557+0000 7f2bf2a00640 1 --2- 172.21.15.12:0/3428637027 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561ebd4e2790 0x561ebd4f2a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:14:41.563 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:41.557+0000 7f2bf2000640 1 --2- 172.21.15.12:0/3428637027 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561ebd4e50c0 0x561ebd4f2fa0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f2be0000c00 tx=0x7f2be00056e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38a3add5a1457f42 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:14:41.564 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:41.557+0000 7f2bdcc00640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-09-11T00:14:41.564 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:41.557+0000 7f2bdcc00640 1 -- 172.21.15.12:0/3428637027 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2be0029050 con 0x561ebd4e50c0 2024-09-11T00:14:41.564 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:41.557+0000 7f2bdcc00640 1 -- 172.21.15.12:0/3428637027 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9208+0+0 (secure 0 0 0) 0x7f2be00049d0 con 0x561ebd4e50c0 2024-09-11T00:14:41.564 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:41.557+0000 7f2bfcab5740 1 -- 172.21.15.12:0/3428637027 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2bec003970 con 0x561ebd4e50c0 2024-09-11T00:14:41.564 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:41.557+0000 7f2bdcc00640 1 -- 172.21.15.12:0/3428637027 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f2be00035f0 con 0x561ebd4e50c0 2024-09-11T00:14:41.564 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:41.557+0000 7f2bfcab5740 10 client.16377 fetch_fsmap learned FSMap version 41 2024-09-11T00:14:41.564 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:41.557+0000 7f2bfcab5740 1 -- 172.21.15.12:0/3428637027 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x561ebd564810 con 0x561ebd4e50c0 2024-09-11T00:14:41.564 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:41.558+0000 7f2bdcc00640 1 -- 172.21.15.12:0/3428637027 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2bec003970 con 0x561ebd4e50c0 2024-09-11T00:14:41.564 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:41.558+0000 7f2bdcc00640 1 -- 172.21.15.12:0/3428637027 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 41) ==== 8370+0+0 (secure 0 0 0) 0x7f2be000f4e0 con 0x561ebd4e50c0 2024-09-11T00:14:41.564 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:41.558+0000 7f2bfcab5740 10 client.16377 fetch_fsmap finished waiting for FSMap version 41 2024-09-11T00:14:41.564 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:41.558+0000 7f2bfcab5740 10 client.16377 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T00:14:41.564 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:41.558+0000 7f2bfcab5740 20 client.16377 populate_metadata read hostname 'smithi012' 2024-09-11T00:14:41.564 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:41.558+0000 7f2bfcab5740 1 --2- 172.21.15.12:0/3428637027 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x561ebd565240 0x561ebd567660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:14:41.565 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:41.558+0000 7f2bfcab5740 4 client.16377 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T00:14:41.565 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:41.558+0000 7f2bfcab5740 1 -- 172.21.15.12:0/3428637027 --> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] -- command(tid 0: {"prefix": "flush journal"}) -- 0x561ebd4530b0 con 0x561ebd565240 2024-09-11T00:14:41.565 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:41.558+0000 7f2bf2a00640 1 --2- 172.21.15.12:0/3428637027 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x561ebd565240 0x561ebd567660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:14:41.565 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:41.559+0000 7f2bf2a00640 1 --2- 172.21.15.12:0/3428637027 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x561ebd565240 0x561ebd567660 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e562ffc077812c93 server_cookie=f20805c09146f58b in_seq=0 out_seq=0 2024-09-11T00:14:41.565 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:41.559+0000 7f2bdcc00640 10 client.16377 ms_handle_connect on v2:172.21.15.165:6838/501312984 2024-09-11T00:14:41.615 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:41.612+0000 7f2bdcc00640 1 -- 172.21.15.12:0/3428637027 <== mds.0 v2:172.21.15.165:6838/501312984 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x561ebd4530b0 con 0x561ebd565240 2024-09-11T00:14:41.615 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:41.612+0000 7f2bdcc00640 10 client.16377 handle_command_reply: tid=0 2024-09-11T00:14:41.615 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:41.612+0000 7f2bfcab5740 1 client.16377 shutdown 2024-09-11T00:14:41.616 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:41.613+0000 7f2bfcab5740 1 -- 172.21.15.12:0/3428637027 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561ebd4e50c0 msgr2=0x561ebd4f2fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:14:41.616 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:41.613+0000 7f2bfcab5740 1 --2- 172.21.15.12:0/3428637027 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561ebd4e50c0 0x561ebd4f2fa0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f2be0000c00 tx=0x7f2be00056e0 comp rx=0 tx=0).stop 2024-09-11T00:14:41.616 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:41.613+0000 7f2bfcab5740 1 -- 172.21.15.12:0/3428637027 shutdown_connections 2024-09-11T00:14:41.616 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:41.613+0000 7f2bfcab5740 1 --2- 172.21.15.12:0/3428637027 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x561ebd565240 0x561ebd567660 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:14:41.616 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:41.613+0000 7f2bfcab5740 1 --2- 172.21.15.12:0/3428637027 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561ebd4e7b00 0x561ebd4f34e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:14:41.616 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:41.613+0000 7f2bfcab5740 1 --2- 172.21.15.12:0/3428637027 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561ebd4e50c0 0x561ebd4f2fa0 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:14:41.616 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:41.613+0000 7f2bfcab5740 1 --2- 172.21.15.12:0/3428637027 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561ebd4e2790 0x561ebd4f2a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:14:41.616 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:41.613+0000 7f2bdcc00640 0 client.16377 ms_handle_reset on v2:172.21.15.165:6838/501312984 2024-09-11T00:14:41.616 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:41.613+0000 7f2bfcab5740 1 -- 172.21.15.12:0/3428637027 >> 172.21.15.12:0/3428637027 conn(0x561ebd4dd8e0 msgr2=0x561ebd4eaee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:14:41.616 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:41.613+0000 7f2bfcab5740 1 -- 172.21.15.12:0/3428637027 shutdown_connections 2024-09-11T00:14:41.616 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:41.613+0000 7f2bfcab5740 1 -- 172.21.15.12:0/3428637027 wait complete. 2024-09-11T00:14:41.616 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:41.613+0000 7f2bfcab5740 20 client.16377 trim_cache size 0 max 16384 2024-09-11T00:14:41.617 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:{ 2024-09-11T00:14:41.617 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "message": "", 2024-09-11T00:14:41.617 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "return_code": 0 2024-09-11T00:14:41.617 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:} 2024-09-11T00:14:41.618 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:41.614+0000 7f2bf9c00640 1 -- 172.21.15.12:0/2387011845 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f2bd80b5360 msgr2=0x7f2bd80b7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:14:41.618 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:41.614+0000 7f2bf9c00640 1 --2- 172.21.15.12:0/2387011845 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f2bd80b5360 0x7f2bd80b7820 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f2bec005f10 tx=0x7f2bec009500 comp rx=0 tx=0).stop 2024-09-11T00:14:41.618 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:41.614+0000 7f2bf9c00640 1 -- 172.21.15.12:0/2387011845 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2bf4102970 msgr2=0x7f2bf4076ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:14:41.618 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:41.614+0000 7f2bf9c00640 1 --2- 172.21.15.12:0/2387011845 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2bf4102970 0x7f2bf4076ce0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f2be4031490 tx=0x7f2be4003c10 comp rx=0 tx=0).stop 2024-09-11T00:14:41.618 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:41.614+0000 7f2bf9c00640 1 -- 172.21.15.12:0/2387011845 shutdown_connections 2024-09-11T00:14:41.618 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:41.614+0000 7f2bf9c00640 1 --2- 172.21.15.12:0/2387011845 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f2bd80b5360 0x7f2bd80b7820 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:14:41.619 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:41.614+0000 7f2bf9c00640 1 --2- 172.21.15.12:0/2387011845 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2bf4104480 0x7f2bf4075a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:14:41.619 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:41.614+0000 7f2bf9c00640 1 --2- 172.21.15.12:0/2387011845 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2bf4103ac0 0x7f2bf4075540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:14:41.619 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:41.614+0000 7f2bf9c00640 1 --2- 172.21.15.12:0/2387011845 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2bf4102970 0x7f2bf4076ce0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:14:41.619 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:41.614+0000 7f2bf9c00640 1 -- 172.21.15.12:0/2387011845 >> 172.21.15.12:0/2387011845 conn(0x7f2bf40fe250 msgr2=0x7f2bf4106eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:14:41.619 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:41.614+0000 7f2bf9c00640 1 -- 172.21.15.12:0/2387011845 shutdown_connections 2024-09-11T00:14:41.619 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:41.614+0000 7f2bf9c00640 1 -- 172.21.15.12:0/2387011845 wait complete. 2024-09-11T00:14:41.656 INFO:tasks.workunit.client.0.smithi012.stdout: 1 273745 3.44 MB/sec execute 278 sec latency 428.234 ms 2024-09-11T00:14:41.782 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:14:41 smithi165 ceph-mon[35356]: pgmap v627: 129 pgs: 129 active+clean; 66 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 20 MiB/s rd, 1.1 MiB/s wr, 303 op/s 2024-09-11T00:14:41.813 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:14:41 smithi012 ceph-mon[30333]: pgmap v627: 129 pgs: 129 active+clean; 66 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 20 MiB/s rd, 1.1 MiB/s wr, 303 op/s 2024-09-11T00:14:41.831 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:14:41 smithi178 ceph-mon[35919]: pgmap v627: 129 pgs: 129 active+clean; 66 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 20 MiB/s rd, 1.1 MiB/s wr, 303 op/s 2024-09-11T00:14:42.656 INFO:tasks.workunit.client.0.smithi012.stdout: 1 275869 3.46 MB/sec execute 279 sec latency 12.054 ms 2024-09-11T00:14:43.656 INFO:tasks.workunit.client.0.smithi012.stdout: 1 276733 3.48 MB/sec execute 280 sec latency 523.908 ms 2024-09-11T00:14:43.783 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:14:43 smithi165 ceph-mon[35356]: pgmap v628: 129 pgs: 129 active+clean; 69 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 25 MiB/s rd, 1.2 MiB/s wr, 353 op/s 2024-09-11T00:14:43.813 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:14:43 smithi012 ceph-mon[30333]: pgmap v628: 129 pgs: 129 active+clean; 69 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 25 MiB/s rd, 1.2 MiB/s wr, 353 op/s 2024-09-11T00:14:43.831 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:14:43 smithi178 ceph-mon[35919]: pgmap v628: 129 pgs: 129 active+clean; 69 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 25 MiB/s rd, 1.2 MiB/s wr, 353 op/s 2024-09-11T00:14:44.656 INFO:tasks.workunit.client.0.smithi012.stdout: 1 277069 3.47 MB/sec execute 281 sec latency 377.085 ms 2024-09-11T00:14:45.656 INFO:tasks.workunit.client.0.smithi012.stdout: 1 279053 3.48 MB/sec execute 282 sec latency 11.994 ms 2024-09-11T00:14:45.782 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:14:45 smithi165 ceph-mon[35356]: pgmap v629: 129 pgs: 129 active+clean; 69 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 22 MiB/s rd, 836 KiB/s wr, 267 op/s 2024-09-11T00:14:45.813 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:14:45 smithi012 ceph-mon[30333]: pgmap v629: 129 pgs: 129 active+clean; 69 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 22 MiB/s rd, 836 KiB/s wr, 267 op/s 2024-09-11T00:14:45.830 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:14:45 smithi178 ceph-mon[35919]: pgmap v629: 129 pgs: 129 active+clean; 69 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 22 MiB/s rd, 836 KiB/s wr, 267 op/s 2024-09-11T00:14:46.656 INFO:tasks.workunit.client.0.smithi012.stdout: 1 280215 3.51 MB/sec execute 283 sec latency 325.944 ms 2024-09-11T00:14:47.656 INFO:tasks.workunit.client.0.smithi012.stdout: 1 280542 3.50 MB/sec execute 284 sec latency 630.432 ms 2024-09-11T00:14:47.737 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:14:47 smithi012 ceph-mon[30333]: pgmap v630: 129 pgs: 129 active+clean; 71 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 32 MiB/s rd, 1.0 MiB/s wr, 392 op/s 2024-09-11T00:14:47.782 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:14:47 smithi165 ceph-mon[35356]: pgmap v630: 129 pgs: 129 active+clean; 71 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 32 MiB/s rd, 1.0 MiB/s wr, 392 op/s 2024-09-11T00:14:47.831 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:14:47 smithi178 ceph-mon[35919]: pgmap v630: 129 pgs: 129 active+clean; 71 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 32 MiB/s rd, 1.0 MiB/s wr, 392 op/s 2024-09-11T00:14:48.656 INFO:tasks.workunit.client.0.smithi012.stdout: 1 282302 3.51 MB/sec execute 285 sec latency 15.839 ms 2024-09-11T00:14:49.656 INFO:tasks.workunit.client.0.smithi012.stdout: 1 283950 3.54 MB/sec execute 286 sec latency 195.884 ms 2024-09-11T00:14:49.782 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:14:49 smithi165 ceph-mon[35356]: pgmap v631: 129 pgs: 129 active+clean; 71 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 41 MiB/s rd, 698 KiB/s wr, 351 op/s 2024-09-11T00:14:49.813 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:14:49 smithi012 ceph-mon[30333]: pgmap v631: 129 pgs: 129 active+clean; 71 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 41 MiB/s rd, 698 KiB/s wr, 351 op/s 2024-09-11T00:14:49.831 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:14:49 smithi178 ceph-mon[35919]: pgmap v631: 129 pgs: 129 active+clean; 71 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 41 MiB/s rd, 698 KiB/s wr, 351 op/s 2024-09-11T00:14:50.656 INFO:tasks.workunit.client.0.smithi012.stdout: 1 284380 3.53 MB/sec execute 287 sec latency 472.504 ms 2024-09-11T00:14:51.656 INFO:tasks.workunit.client.0.smithi012.stdout: 1 286558 3.56 MB/sec execute 288 sec latency 12.220 ms 2024-09-11T00:14:51.782 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:14:51 smithi165 ceph-mon[35356]: pgmap v632: 129 pgs: 129 active+clean; 73 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 49 MiB/s rd, 870 KiB/s wr, 417 op/s 2024-09-11T00:14:51.813 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:14:51 smithi012 ceph-mon[30333]: pgmap v632: 129 pgs: 129 active+clean; 73 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 49 MiB/s rd, 870 KiB/s wr, 417 op/s 2024-09-11T00:14:51.830 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:14:51 smithi178 ceph-mon[35919]: pgmap v632: 129 pgs: 129 active+clean; 73 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 49 MiB/s rd, 870 KiB/s wr, 417 op/s 2024-09-11T00:14:52.656 INFO:tasks.workunit.client.0.smithi012.stdout: 1 287767 3.57 MB/sec execute 289 sec latency 108.692 ms 2024-09-11T00:14:53.657 INFO:tasks.workunit.client.0.smithi012.stdout: 1 288801 3.58 MB/sec execute 290 sec latency 272.162 ms 2024-09-11T00:14:53.782 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:14:53 smithi165 ceph-mon[35356]: pgmap v633: 129 pgs: 129 active+clean; 74 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 59 MiB/s rd, 872 KiB/s wr, 471 op/s 2024-09-11T00:14:53.813 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:14:53 smithi012 ceph-mon[30333]: pgmap v633: 129 pgs: 129 active+clean; 74 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 59 MiB/s rd, 872 KiB/s wr, 471 op/s 2024-09-11T00:14:53.831 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:14:53 smithi178 ceph-mon[35919]: pgmap v633: 129 pgs: 129 active+clean; 74 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 59 MiB/s rd, 872 KiB/s wr, 471 op/s 2024-09-11T00:14:54.656 INFO:tasks.workunit.client.0.smithi012.stdout: 1 290915 3.61 MB/sec execute 291 sec latency 79.277 ms 2024-09-11T00:14:54.747 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:54.744+0000 7f4cc9200640 1 -- 172.21.15.12:0/2346598349 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4cc41031a0 msgr2=0x7f4cc4109b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:14:54.747 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:54.744+0000 7f4cc9200640 1 --2- 172.21.15.12:0/2346598349 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4cc41031a0 0x7f4cc4109b20 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f4cbc009cd0 tx=0x7f4cbc02a8e0 comp rx=0 tx=0).stop 2024-09-11T00:14:54.747 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:54.744+0000 7f4cc9200640 1 -- 172.21.15.12:0/2346598349 shutdown_connections 2024-09-11T00:14:54.747 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:54.744+0000 7f4cc9200640 1 --2- 172.21.15.12:0/2346598349 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4cc41031a0 0x7f4cc4109b20 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:14:54.747 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:54.744+0000 7f4cc9200640 1 --2- 172.21.15.12:0/2346598349 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4cc41027e0 0x7f4cc4102c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:14:54.747 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:54.744+0000 7f4cc9200640 1 --2- 172.21.15.12:0/2346598349 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4cc4101690 0x7f4cc4101a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:14:54.747 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:54.744+0000 7f4cc9200640 1 -- 172.21.15.12:0/2346598349 >> 172.21.15.12:0/2346598349 conn(0x7f4cc40fcf30 msgr2=0x7f4cc40ff370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:14:54.747 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:54.744+0000 7f4cc9200640 1 -- 172.21.15.12:0/2346598349 shutdown_connections 2024-09-11T00:14:54.747 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:54.744+0000 7f4cc9200640 1 -- 172.21.15.12:0/2346598349 wait complete. 2024-09-11T00:14:54.750 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:54.745+0000 7f4cc9200640 1 Processor -- start 2024-09-11T00:14:54.750 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:54.745+0000 7f4cc9200640 1 -- start start 2024-09-11T00:14:54.750 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:54.746+0000 7f4cc9200640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4cc4101690 0x7f4cc419ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:14:54.750 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:54.746+0000 7f4cc9200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4cc41027e0 0x7f4cc419b390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:14:54.750 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:54.746+0000 7f4cc9200640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4cc41031a0 0x7f4cc419bfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:14:54.750 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:54.746+0000 7f4cc9200640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f4cc410d300 con 0x7f4cc41031a0 2024-09-11T00:14:54.750 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:54.746+0000 7f4cc9200640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f4cc410d180 con 0x7f4cc41027e0 2024-09-11T00:14:54.750 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:54.746+0000 7f4cc9200640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f4cc410d480 con 0x7f4cc4101690 2024-09-11T00:14:54.750 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:54.746+0000 7f4cc2a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4cc41027e0 0x7f4cc419b390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:14:54.751 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:54.746+0000 7f4cc3e00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4cc41031a0 0x7f4cc419bfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:14:54.751 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:54.746+0000 7f4cc2a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4cc41027e0 0x7f4cc419b390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.12:56864/0 (socket says 172.21.15.12:56864) 2024-09-11T00:14:54.751 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:54.746+0000 7f4cc2a00640 1 -- 172.21.15.12:0/3530812176 learned_addr learned my addr 172.21.15.12:0/3530812176 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:14:54.751 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:54.746+0000 7f4cc3400640 1 --2- 172.21.15.12:0/3530812176 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4cc4101690 0x7f4cc419ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:14:54.751 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:54.747+0000 7f4cc3400640 1 -- 172.21.15.12:0/3530812176 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4cc41027e0 msgr2=0x7f4cc419b390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:14:54.751 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:54.747+0000 7f4cc3400640 1 --2- 172.21.15.12:0/3530812176 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4cc41027e0 0x7f4cc419b390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:14:54.751 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:54.747+0000 7f4cc3400640 1 -- 172.21.15.12:0/3530812176 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4cc41031a0 msgr2=0x7f4cc419bfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:14:54.751 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:54.747+0000 7f4cc3400640 1 --2- 172.21.15.12:0/3530812176 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4cc41031a0 0x7f4cc419bfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:14:54.751 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:54.747+0000 7f4cc3400640 1 -- 172.21.15.12:0/3530812176 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4cc419c880 con 0x7f4cc4101690 2024-09-11T00:14:54.751 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:54.747+0000 7f4cc3400640 1 --2- 172.21.15.12:0/3530812176 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4cc4101690 0x7f4cc419ad40 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f4cb400e9f0 tx=0x7f4cb400eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:14:54.751 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:54.747+0000 7f4cbbe00640 1 -- 172.21.15.12:0/3530812176 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cb400ccc0 con 0x7f4cc4101690 2024-09-11T00:14:54.751 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:54.747+0000 7f4cbbe00640 1 -- 172.21.15.12:0/3530812176 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4cb4005020 con 0x7f4cc4101690 2024-09-11T00:14:54.751 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:54.747+0000 7f4cbbe00640 1 -- 172.21.15.12:0/3530812176 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cb4004a00 con 0x7f4cc4101690 2024-09-11T00:14:54.752 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:54.747+0000 7f4cc9200640 1 -- 172.21.15.12:0/3530812176 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4cc419cb70 con 0x7f4cc4101690 2024-09-11T00:14:54.752 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:54.748+0000 7f4cc9200640 1 -- 172.21.15.12:0/3530812176 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4cc41a3240 con 0x7f4cc4101690 2024-09-11T00:14:54.753 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:54.751+0000 7f4ccd84f740 1 Processor -- start 2024-09-11T00:14:54.753 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:54.751+0000 7f4ccd84f740 1 -- start start 2024-09-11T00:14:54.753 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:54.751+0000 7f4ccd84f740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555bf9d78180 0x555bf9d7a570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:14:54.753 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:54.751+0000 7f4ccd84f740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555bf9d7aab0 0x555bf9d7cea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:14:54.754 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:54.751+0000 7f4ccd84f740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555bf9d7d4f0 0x555bf9d7f9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:14:54.754 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:54.751+0000 7f4ccd84f740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x555bf9ce26b0 con 0x555bf9d78180 2024-09-11T00:14:54.754 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:54.751+0000 7f4ccd84f740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x555bf9cd1ae0 con 0x555bf9d7d4f0 2024-09-11T00:14:54.754 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:54.751+0000 7f4ccd84f740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x555bf9cd58f0 con 0x555bf9d7aab0 2024-09-11T00:14:54.755 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:54.752+0000 7f4cc3e00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555bf9d78180 0x555bf9d7a570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:14:54.755 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:54.752+0000 7f4cc2a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555bf9d7d4f0 0x555bf9d7f9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:14:54.755 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:54.752+0000 7f4cc3e00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555bf9d78180 0x555bf9d7a570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:47958/0 (socket says 172.21.15.12:47958) 2024-09-11T00:14:54.755 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:54.752+0000 7f4cc2a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555bf9d7d4f0 0x555bf9d7f9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.12:56868/0 (socket says 172.21.15.12:56868) 2024-09-11T00:14:54.755 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:54.752+0000 7f4cc3e00640 1 -- 172.21.15.12:0/2549559284 learned_addr learned my addr 172.21.15.12:0/2549559284 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:14:54.755 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:54.752+0000 7f4cc2a00640 1 -- 172.21.15.12:0/2549559284 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555bf9d7aab0 msgr2=0x555bf9d7cea0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-11T00:14:54.755 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:54.752+0000 7f4cc2a00640 1 --2- 172.21.15.12:0/2549559284 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555bf9d7aab0 0x555bf9d7cea0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:14:54.755 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:54.752+0000 7f4cc2a00640 1 -- 172.21.15.12:0/2549559284 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555bf9d78180 msgr2=0x555bf9d7a570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:14:54.755 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:54.752+0000 7f4cc2a00640 1 --2- 172.21.15.12:0/2549559284 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555bf9d78180 0x555bf9d7a570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:14:54.755 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:54.752+0000 7f4cc2a00640 1 -- 172.21.15.12:0/2549559284 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555bf9c9e780 con 0x555bf9d7d4f0 2024-09-11T00:14:54.755 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:54.752+0000 7f4cbbe00640 1 -- 172.21.15.12:0/3530812176 <== mon.1 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7f4cb4004ba0 con 0x7f4cc4101690 2024-09-11T00:14:54.756 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:54.752+0000 7f4cc2a00640 1 --2- 172.21.15.12:0/2549559284 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555bf9d7d4f0 0x555bf9d7f9b0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f4cb0004590 tx=0x7f4cb000bbb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b2d12354b58af6f server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:14:54.756 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:54.753+0000 7f4cb8c00640 1 -- 172.21.15.12:0/2549559284 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cb0003bd0 con 0x555bf9d7d4f0 2024-09-11T00:14:54.756 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:54.753+0000 7f4cb8c00640 1 -- 172.21.15.12:0/2549559284 <== mon.2 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4cb0015070 con 0x555bf9d7d4f0 2024-09-11T00:14:54.756 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:54.753+0000 7f4cb8c00640 1 -- 172.21.15.12:0/2549559284 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cb0004e00 con 0x555bf9d7d4f0 2024-09-11T00:14:54.756 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:54.754+0000 7f4ccd84f740 1 -- 172.21.15.12:0/2549559284 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555bf9d7d4f0 msgr2=0x555bf9d7f9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:14:54.757 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:54.754+0000 7f4ccd84f740 1 --2- 172.21.15.12:0/2549559284 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555bf9d7d4f0 0x555bf9d7f9b0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f4cb0004590 tx=0x7f4cb000bbb0 comp rx=0 tx=0).stop 2024-09-11T00:14:54.757 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:54.754+0000 7f4cbbe00640 1 --2- 172.21.15.12:0/3530812176 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f4ca80b5360 0x7f4ca80b7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:14:54.757 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:54.754+0000 7f4cbbe00640 1 -- 172.21.15.12:0/3530812176 <== mon.1 v2:172.21.15.178:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9208+0+0 (secure 0 0 0) 0x7f4cb40e5f30 con 0x7f4cc4101690 2024-09-11T00:14:54.757 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:54.754+0000 7f4ccd84f740 1 -- 172.21.15.12:0/2549559284 shutdown_connections 2024-09-11T00:14:54.757 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:54.754+0000 7f4ccd84f740 1 --2- 172.21.15.12:0/2549559284 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555bf9d7d4f0 0x555bf9d7f9b0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:14:54.757 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:54.754+0000 7f4ccd84f740 1 --2- 172.21.15.12:0/2549559284 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555bf9d7aab0 0x555bf9d7cea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:14:54.757 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:54.754+0000 7f4ccd84f740 1 --2- 172.21.15.12:0/2549559284 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555bf9d78180 0x555bf9d7a570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:14:54.757 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:54.754+0000 7f4ccd84f740 1 -- 172.21.15.12:0/2549559284 >> 172.21.15.12:0/2549559284 conn(0x555bf9d6aac0 msgr2=0x555bf9d6b850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:14:54.757 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:54.754+0000 7f4cc3e00640 1 --2- 172.21.15.12:0/3530812176 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f4ca80b5360 0x7f4ca80b7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:14:54.757 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:54.754+0000 7f4ccd84f740 1 -- 172.21.15.12:0/2549559284 shutdown_connections 2024-09-11T00:14:54.758 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:54.754+0000 7f4ccd84f740 1 -- 172.21.15.12:0/2549559284 wait complete. 2024-09-11T00:14:54.760 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:54.755+0000 7f4ccd84f740 1 -- start start 2024-09-11T00:14:54.760 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:54.755+0000 7f4cc3e00640 1 --2- 172.21.15.12:0/3530812176 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f4ca80b5360 0x7f4ca80b7820 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f4cbc006010 tx=0x7f4cbc03d040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:14:54.760 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:54.755+0000 7f4ccd84f740 1 Processor -- start 2024-09-11T00:14:54.760 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:54.756+0000 7f4ccd84f740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555bf9d78180 0x555bf9d81970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:14:54.760 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:54.756+0000 7f4ccd84f740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555bf9d7aab0 0x555bf9d82010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:14:54.760 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:54.756+0000 7f4ccd84f740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555bf9d7d4f0 0x555bf9d82b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:14:54.760 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:54.756+0000 7f4ccd84f740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x555bf9c09ab0 con 0x555bf9d78180 2024-09-11T00:14:54.760 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:54.756+0000 7f4ccd84f740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x555bf9c78570 con 0x555bf9d7aab0 2024-09-11T00:14:54.760 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:54.756+0000 7f4ccd84f740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x555bf9d804d0 con 0x555bf9d7d4f0 2024-09-11T00:14:54.761 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:54.756+0000 7f4ccd84f740 10 client.0 _ll_register_callbacks cb 0x555bf9c87100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:14:54.761 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:54.756+0000 7f4cc2a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555bf9d7aab0 0x555bf9d82010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:14:54.761 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:54.756+0000 7f4cc2a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555bf9d7aab0 0x555bf9d82010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.12:56878/0 (socket says 172.21.15.12:56878) 2024-09-11T00:14:54.761 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:54.756+0000 7f4cc2a00640 1 -- 172.21.15.12:0/2094531549 learned_addr learned my addr 172.21.15.12:0/2094531549 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:14:54.761 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:54.756+0000 7f4cc3e00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555bf9d7d4f0 0x555bf9d82b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:14:54.761 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:54.756+0000 7f4cc3400640 1 --2- 172.21.15.12:0/2094531549 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555bf9d78180 0x555bf9d81970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:14:54.761 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:54.757+0000 7f4cc2a00640 1 -- 172.21.15.12:0/2094531549 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555bf9d7d4f0 msgr2=0x555bf9d82b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:14:54.761 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:54.757+0000 7f4cc2a00640 1 --2- 172.21.15.12:0/2094531549 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555bf9d7d4f0 0x555bf9d82b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:14:54.761 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:54.757+0000 7f4cc2a00640 1 -- 172.21.15.12:0/2094531549 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555bf9d78180 msgr2=0x555bf9d81970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:14:54.761 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:54.757+0000 7f4cc2a00640 1 --2- 172.21.15.12:0/2094531549 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555bf9d78180 0x555bf9d81970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:14:54.761 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:54.757+0000 7f4cc2a00640 1 -- 172.21.15.12:0/2094531549 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555bf9a33790 con 0x555bf9d7aab0 2024-09-11T00:14:54.761 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:54.757+0000 7f4cc2a00640 1 --2- 172.21.15.12:0/2094531549 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555bf9d7aab0 0x555bf9d82010 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f4cb0000c00 tx=0x7f4cb00046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a66b3df97bb94e84 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:14:54.762 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:54.757+0000 7f4cad600640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-09-11T00:14:54.762 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:54.757+0000 7f4cad600640 1 -- 172.21.15.12:0/2094531549 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cb0003de0 con 0x555bf9d7aab0 2024-09-11T00:14:54.762 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:54.757+0000 7f4ccd84f740 1 -- 172.21.15.12:0/2094531549 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4cb0015070 con 0x555bf9d7aab0 2024-09-11T00:14:54.762 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:54.757+0000 7f4cad600640 1 -- 172.21.15.12:0/2094531549 <== mon.2 v2:172.21.15.165:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9208+0+0 (secure 0 0 0) 0x7f4cb0011440 con 0x555bf9d7aab0 2024-09-11T00:14:54.762 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:54.758+0000 7f4cad600640 1 -- 172.21.15.12:0/2094531549 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f4cb0030050 con 0x555bf9d7aab0 2024-09-11T00:14:54.762 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:54.758+0000 7f4ccd84f740 10 client.26399 fetch_fsmap learned FSMap version 41 2024-09-11T00:14:54.762 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:54.758+0000 7f4ccd84f740 1 -- 172.21.15.12:0/2094531549 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x555bf9aac620 con 0x555bf9d7aab0 2024-09-11T00:14:54.764 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:54.758+0000 7f4cad600640 1 -- 172.21.15.12:0/2094531549 <== mon.2 v2:172.21.15.165:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4cb0015070 con 0x555bf9d7aab0 2024-09-11T00:14:54.764 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:54.758+0000 7f4cad600640 1 -- 172.21.15.12:0/2094531549 <== mon.2 v2:172.21.15.165:3300/0 5 ==== fsmap(e 41) ==== 8370+0+0 (secure 0 0 0) 0x7f4cb00257a0 con 0x555bf9d7aab0 2024-09-11T00:14:54.764 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:54.758+0000 7f4ccd84f740 10 client.26399 fetch_fsmap finished waiting for FSMap version 41 2024-09-11T00:14:54.764 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:54.758+0000 7f4ccd84f740 10 client.26399 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T00:14:54.765 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:54.759+0000 7f4ccd84f740 20 client.26399 populate_metadata read hostname 'smithi012' 2024-09-11T00:14:54.765 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:54.759+0000 7f4ccd84f740 1 --2- 172.21.15.12:0/2094531549 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x555bf9d8cfc0 0x555bf9d8f3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:14:54.765 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:54.759+0000 7f4ccd84f740 4 client.26399 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T00:14:54.765 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:54.759+0000 7f4ccd84f740 1 -- 172.21.15.12:0/2094531549 --> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555bf9ce4600 con 0x555bf9d8cfc0 2024-09-11T00:14:54.765 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:54.759+0000 7f4cc3400640 1 --2- 172.21.15.12:0/2094531549 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x555bf9d8cfc0 0x555bf9d8f3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:14:54.765 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:54.759+0000 7f4cc3400640 1 --2- 172.21.15.12:0/2094531549 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x555bf9d8cfc0 0x555bf9d8f3b0 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b9aca9a706bc4aeb server_cookie=f63995b7a445fc3e in_seq=0 out_seq=0 2024-09-11T00:14:54.765 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:54.759+0000 7f4cad600640 10 client.26399 ms_handle_connect on v2:172.21.15.165:6838/501312984 2024-09-11T00:14:54.765 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:54.760+0000 7f4cad600640 1 -- 172.21.15.12:0/2094531549 <== mds.0 v2:172.21.15.165:6838/501312984 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x555bf9ce4600 con 0x555bf9d8cfc0 2024-09-11T00:14:54.765 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:54.760+0000 7f4cad600640 10 client.26399 handle_command_reply: tid=0 2024-09-11T00:14:54.765 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:54.760+0000 7f4ccd84f740 1 client.26399 shutdown 2024-09-11T00:14:54.765 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:54.760+0000 7f4ccd84f740 1 -- 172.21.15.12:0/2094531549 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555bf9d7aab0 msgr2=0x555bf9d82010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:14:54.766 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:54.760+0000 7f4ccd84f740 1 --2- 172.21.15.12:0/2094531549 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555bf9d7aab0 0x555bf9d82010 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f4cb0000c00 tx=0x7f4cb00046c0 comp rx=0 tx=0).stop 2024-09-11T00:14:54.766 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:54.761+0000 7f4ccd84f740 1 -- 172.21.15.12:0/2094531549 shutdown_connections 2024-09-11T00:14:54.766 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:54.761+0000 7f4ccd84f740 1 --2- 172.21.15.12:0/2094531549 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x555bf9d8cfc0 0x555bf9d8f3b0 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:14:54.766 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:54.761+0000 7f4ccd84f740 1 --2- 172.21.15.12:0/2094531549 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555bf9d7d4f0 0x555bf9d82b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:14:54.766 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:54.761+0000 7f4ccd84f740 1 --2- 172.21.15.12:0/2094531549 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555bf9d7aab0 0x555bf9d82010 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:14:54.766 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:54.761+0000 7f4ccd84f740 1 --2- 172.21.15.12:0/2094531549 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555bf9d78180 0x555bf9d81970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:14:54.766 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:54.761+0000 7f4cad600640 0 client.26399 ms_handle_reset on v2:172.21.15.165:6838/501312984 2024-09-11T00:14:54.766 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:54.761+0000 7f4ccd84f740 1 -- 172.21.15.12:0/2094531549 >> 172.21.15.12:0/2094531549 conn(0x555bf9d6aac0 msgr2=0x555bf9d69bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:14:54.766 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:54.761+0000 7f4ccd84f740 1 -- 172.21.15.12:0/2094531549 shutdown_connections 2024-09-11T00:14:54.766 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:54.761+0000 7f4ccd84f740 1 -- 172.21.15.12:0/2094531549 wait complete. 2024-09-11T00:14:54.766 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:54.761+0000 7f4ccd84f740 20 client.26399 trim_cache size 0 max 16384 2024-09-11T00:14:54.781 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:54.779+0000 7f4ccd84f740 1 Processor -- start 2024-09-11T00:14:54.781 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:54.779+0000 7f4ccd84f740 1 -- start start 2024-09-11T00:14:54.781 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:54.779+0000 7f4ccd84f740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555bf9d78180 0x555bf9d7dfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:14:54.782 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:54.779+0000 7f4ccd84f740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555bf9d7aab0 0x555bf9d65bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:14:54.782 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:54.779+0000 7f4ccd84f740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555bf9d7d4f0 0x555bf9d81970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:14:54.782 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:54.779+0000 7f4ccd84f740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f4cc41a9380 con 0x555bf9d78180 2024-09-11T00:14:54.782 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:54.779+0000 7f4ccd84f740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f4cc41aaec0 con 0x555bf9d7d4f0 2024-09-11T00:14:54.782 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:54.779+0000 7f4ccd84f740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f4cc41a9a50 con 0x555bf9d7aab0 2024-09-11T00:14:54.782 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:54.779+0000 7f4cc3e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555bf9d7d4f0 0x555bf9d81970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:14:54.782 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:54.779+0000 7f4cc3400640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555bf9d78180 0x555bf9d7dfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:14:54.782 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:54.779+0000 7f4cc3e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555bf9d7d4f0 0x555bf9d81970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.12:56882/0 (socket says 172.21.15.12:56882) 2024-09-11T00:14:54.782 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:54.779+0000 7f4cc3e00640 1 -- 172.21.15.12:0/3571246111 learned_addr learned my addr 172.21.15.12:0/3571246111 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:14:54.782 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:54.779+0000 7f4cc2a00640 1 --2- 172.21.15.12:0/3571246111 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555bf9d7aab0 0x555bf9d65bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:14:54.787 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:54.780+0000 7f4cc2a00640 1 -- 172.21.15.12:0/3571246111 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555bf9d7d4f0 msgr2=0x555bf9d81970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:14:54.788 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:54.780+0000 7f4cc2a00640 1 --2- 172.21.15.12:0/3571246111 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555bf9d7d4f0 0x555bf9d81970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:14:54.788 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:54.780+0000 7f4cc2a00640 1 -- 172.21.15.12:0/3571246111 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555bf9d78180 msgr2=0x555bf9d7dfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:14:54.788 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:54.780+0000 7f4cc2a00640 1 --2- 172.21.15.12:0/3571246111 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555bf9d78180 0x555bf9d7dfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:14:54.788 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:54.780+0000 7f4cc2a00640 1 -- 172.21.15.12:0/3571246111 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555bf9d8f720 con 0x555bf9d7aab0 2024-09-11T00:14:54.788 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:54.780+0000 7f4cc3e00640 1 --2- 172.21.15.12:0/3571246111 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555bf9d7d4f0 0x555bf9d81970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:14:54.788 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:54.780+0000 7f4cc3400640 1 --2- 172.21.15.12:0/3571246111 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555bf9d78180 0x555bf9d7dfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:14:54.788 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:54.780+0000 7f4cc2a00640 1 --2- 172.21.15.12:0/3571246111 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555bf9d7aab0 0x555bf9d65bc0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f4cb0001eb0 tx=0x7f4cb0004220 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7152c1839af3f6ff server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:14:54.788 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:54.780+0000 7f4cafe00640 1 -- 172.21.15.12:0/3571246111 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cb0038020 con 0x555bf9d7aab0 2024-09-11T00:14:54.788 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:54.780+0000 7f4cafe00640 1 -- 172.21.15.12:0/3571246111 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4cb0015070 con 0x555bf9d7aab0 2024-09-11T00:14:54.788 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:54.780+0000 7f4cafe00640 1 -- 172.21.15.12:0/3571246111 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cb0005570 con 0x555bf9d7aab0 2024-09-11T00:14:54.788 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:54.781+0000 7f4ccd84f740 1 -- 172.21.15.12:0/3571246111 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555bf9d7aab0 msgr2=0x555bf9d65bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:14:54.789 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:54.781+0000 7f4ccd84f740 1 --2- 172.21.15.12:0/3571246111 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555bf9d7aab0 0x555bf9d65bc0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f4cb0001eb0 tx=0x7f4cb0004220 comp rx=0 tx=0).stop 2024-09-11T00:14:54.789 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:54.781+0000 7f4ccd84f740 1 -- 172.21.15.12:0/3571246111 shutdown_connections 2024-09-11T00:14:54.789 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:54.781+0000 7f4ccd84f740 1 --2- 172.21.15.12:0/3571246111 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555bf9d7d4f0 0x555bf9d81970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:14:54.789 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:54.781+0000 7f4ccd84f740 1 --2- 172.21.15.12:0/3571246111 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555bf9d7aab0 0x555bf9d65bc0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:14:54.789 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:54.781+0000 7f4ccd84f740 1 --2- 172.21.15.12:0/3571246111 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555bf9d78180 0x555bf9d7dfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:14:54.789 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:54.781+0000 7f4ccd84f740 1 -- 172.21.15.12:0/3571246111 >> 172.21.15.12:0/3571246111 conn(0x555bf9d6aac0 msgr2=0x555bf9d7d8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:14:54.789 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:54.781+0000 7f4ccd84f740 1 -- 172.21.15.12:0/3571246111 shutdown_connections 2024-09-11T00:14:54.789 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:54.781+0000 7f4ccd84f740 1 -- 172.21.15.12:0/3571246111 wait complete. 2024-09-11T00:14:54.789 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:54.782+0000 7f4ccd84f740 1 -- start start 2024-09-11T00:14:54.790 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:54.782+0000 7f4ccd84f740 1 Processor -- start 2024-09-11T00:14:54.790 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:54.782+0000 7f4ccd84f740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555bf9d78180 0x555bf9d89600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:14:54.790 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:54.782+0000 7f4ccd84f740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555bf9d7aab0 0x555bf9d88360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:14:54.790 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:54.782+0000 7f4ccd84f740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555bf9d7d4f0 0x555bf9d88a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:14:54.790 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:54.782+0000 7f4ccd84f740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f4cc41aa7f0 con 0x555bf9d7d4f0 2024-09-11T00:14:54.790 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:54.782+0000 7f4ccd84f740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f4cc41aa120 con 0x555bf9d78180 2024-09-11T00:14:54.790 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:54.782+0000 7f4ccd84f740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f4cc41a73a0 con 0x555bf9d7aab0 2024-09-11T00:14:54.790 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:54.782+0000 7f4ccd84f740 10 client.0 _ll_register_callbacks cb 0x7f4cbc009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:14:54.790 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:54.783+0000 7f4cc3400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555bf9d78180 0x555bf9d89600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:14:54.790 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:54.783+0000 7f4cc3400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555bf9d78180 0x555bf9d89600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.12:56890/0 (socket says 172.21.15.12:56890) 2024-09-11T00:14:54.791 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:54.783+0000 7f4cc3400640 1 -- 172.21.15.12:0/2256311981 learned_addr learned my addr 172.21.15.12:0/2256311981 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:14:54.791 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:54.783+0000 7f4cc2a00640 1 --2- 172.21.15.12:0/2256311981 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555bf9d7aab0 0x555bf9d88360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:14:54.791 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:54.783+0000 7f4cc3e00640 1 --2- 172.21.15.12:0/2256311981 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555bf9d7d4f0 0x555bf9d88a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:14:54.791 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:54.783+0000 7f4cc3400640 1 -- 172.21.15.12:0/2256311981 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555bf9d7aab0 msgr2=0x555bf9d88360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:14:54.791 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:54.783+0000 7f4cc3400640 1 --2- 172.21.15.12:0/2256311981 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555bf9d7aab0 0x555bf9d88360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:14:54.791 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:54.783+0000 7f4cc3400640 1 -- 172.21.15.12:0/2256311981 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555bf9d7d4f0 msgr2=0x555bf9d88a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:14:54.791 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:54.783+0000 7f4cc3400640 1 --2- 172.21.15.12:0/2256311981 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555bf9d7d4f0 0x555bf9d88a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:14:54.791 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:54.783+0000 7f4cc3400640 1 -- 172.21.15.12:0/2256311981 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555bf9d86e90 con 0x555bf9d78180 2024-09-11T00:14:54.791 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:54.783+0000 7f4cc3400640 1 --2- 172.21.15.12:0/2256311981 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555bf9d78180 0x555bf9d89600 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f4cb40ac250 tx=0x7f4cb40b5190 comp rx=0 tx=0).ready entity=mon.2 client_cookie=da29257b9e16ba1d server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:14:54.791 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:54.783+0000 7f4cad600640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-09-11T00:14:54.791 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:54.783+0000 7f4cad600640 1 -- 172.21.15.12:0/2256311981 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cb401c040 con 0x555bf9d78180 2024-09-11T00:14:54.791 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:54.784+0000 7f4cad600640 1 -- 172.21.15.12:0/2256311981 <== mon.2 v2:172.21.15.165:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9208+0+0 (secure 0 0 0) 0x7f4cb40054f0 con 0x555bf9d78180 2024-09-11T00:14:54.791 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:54.784+0000 7f4ccd84f740 1 -- 172.21.15.12:0/2256311981 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4cb0015070 con 0x555bf9d78180 2024-09-11T00:14:54.792 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:54.784+0000 7f4cad600640 1 -- 172.21.15.12:0/2256311981 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f4cb40ac2c0 con 0x555bf9d78180 2024-09-11T00:14:54.792 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:54.784+0000 7f4ccd84f740 10 client.26405 fetch_fsmap learned FSMap version 41 2024-09-11T00:14:54.792 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:54.784+0000 7f4ccd84f740 1 -- 172.21.15.12:0/2256311981 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x555bf9dfa160 con 0x555bf9d78180 2024-09-11T00:14:54.792 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:54.784+0000 7f4cad600640 1 -- 172.21.15.12:0/2256311981 <== mon.2 v2:172.21.15.165:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4cb0015070 con 0x555bf9d78180 2024-09-11T00:14:54.792 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:54.784+0000 7f4cad600640 1 -- 172.21.15.12:0/2256311981 <== mon.2 v2:172.21.15.165:3300/0 5 ==== fsmap(e 41) ==== 8370+0+0 (secure 0 0 0) 0x7f4cb40cf0d0 con 0x555bf9d78180 2024-09-11T00:14:54.792 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:54.784+0000 7f4ccd84f740 10 client.26405 fetch_fsmap finished waiting for FSMap version 41 2024-09-11T00:14:54.792 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:54.784+0000 7f4ccd84f740 10 client.26405 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T00:14:54.792 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:54.784+0000 7f4ccd84f740 20 client.26405 populate_metadata read hostname 'smithi012' 2024-09-11T00:14:54.792 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:54.784+0000 7f4ccd84f740 1 --2- 172.21.15.12:0/2256311981 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x555bf9dfaae0 0x555bf9dfcf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:14:54.792 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:54.784+0000 7f4ccd84f740 4 client.26405 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T00:14:54.792 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:54.784+0000 7f4ccd84f740 1 -- 172.21.15.12:0/2256311981 --> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] -- command(tid 0: {"prefix": "flush journal"}) -- 0x555bf9ce00b0 con 0x555bf9dfaae0 2024-09-11T00:14:54.792 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:54.785+0000 7f4cc2a00640 1 --2- 172.21.15.12:0/2256311981 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x555bf9dfaae0 0x555bf9dfcf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:14:54.793 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:54.785+0000 7f4cc2a00640 1 --2- 172.21.15.12:0/2256311981 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x555bf9dfaae0 0x555bf9dfcf00 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9596c10e62609ead server_cookie=17eb9183758eb4b3 in_seq=0 out_seq=0 2024-09-11T00:14:54.793 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:54.785+0000 7f4cad600640 10 client.26405 ms_handle_connect on v2:172.21.15.165:6838/501312984 2024-09-11T00:14:54.853 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:54.850+0000 7f4cad600640 1 -- 172.21.15.12:0/2256311981 <== mds.0 v2:172.21.15.165:6838/501312984 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x555bf9ce00b0 con 0x555bf9dfaae0 2024-09-11T00:14:54.853 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:54.850+0000 7f4cad600640 10 client.26405 handle_command_reply: tid=0 2024-09-11T00:14:54.854 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:54.850+0000 7f4ccd84f740 1 client.26405 shutdown 2024-09-11T00:14:54.854 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:54.850+0000 7f4ccd84f740 1 -- 172.21.15.12:0/2256311981 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555bf9d78180 msgr2=0x555bf9d89600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:14:54.854 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:54.850+0000 7f4ccd84f740 1 --2- 172.21.15.12:0/2256311981 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555bf9d78180 0x555bf9d89600 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f4cb40ac250 tx=0x7f4cb40b5190 comp rx=0 tx=0).stop 2024-09-11T00:14:54.854 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:54.850+0000 7f4ccd84f740 1 -- 172.21.15.12:0/2256311981 shutdown_connections 2024-09-11T00:14:54.854 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:54.850+0000 7f4ccd84f740 1 --2- 172.21.15.12:0/2256311981 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x555bf9dfaae0 0x555bf9dfcf00 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:14:54.854 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:54.850+0000 7f4ccd84f740 1 --2- 172.21.15.12:0/2256311981 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555bf9d7d4f0 0x555bf9d88a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:14:54.854 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:54.850+0000 7f4ccd84f740 1 --2- 172.21.15.12:0/2256311981 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555bf9d7aab0 0x555bf9d88360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:14:54.854 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:54.850+0000 7f4ccd84f740 1 --2- 172.21.15.12:0/2256311981 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555bf9d78180 0x555bf9d89600 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:14:54.854 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:54.850+0000 7f4cad600640 0 client.26405 ms_handle_reset on v2:172.21.15.165:6838/501312984 2024-09-11T00:14:54.855 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:54.850+0000 7f4ccd84f740 1 -- 172.21.15.12:0/2256311981 >> 172.21.15.12:0/2256311981 conn(0x555bf9d6aac0 msgr2=0x555bf9d7d8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:14:54.855 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:54.850+0000 7f4ccd84f740 1 -- 172.21.15.12:0/2256311981 shutdown_connections 2024-09-11T00:14:54.855 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:54.850+0000 7f4ccd84f740 1 -- 172.21.15.12:0/2256311981 wait complete. 2024-09-11T00:14:54.855 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:54.851+0000 7f4ccd84f740 20 client.26405 trim_cache size 0 max 16384 2024-09-11T00:14:54.855 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:{ 2024-09-11T00:14:54.855 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "message": "", 2024-09-11T00:14:54.855 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "return_code": 0 2024-09-11T00:14:54.856 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:} 2024-09-11T00:14:54.856 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:54.851+0000 7f4cc9200640 1 -- 172.21.15.12:0/3530812176 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f4ca80b5360 msgr2=0x7f4ca80b7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:14:54.856 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:54.851+0000 7f4cc9200640 1 --2- 172.21.15.12:0/3530812176 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f4ca80b5360 0x7f4ca80b7820 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f4cbc006010 tx=0x7f4cbc03d040 comp rx=0 tx=0).stop 2024-09-11T00:14:54.856 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:54.851+0000 7f4cc9200640 1 -- 172.21.15.12:0/3530812176 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4cc4101690 msgr2=0x7f4cc419ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:14:54.856 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:54.851+0000 7f4cc9200640 1 --2- 172.21.15.12:0/3530812176 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4cc4101690 0x7f4cc419ad40 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f4cb400e9f0 tx=0x7f4cb400eeb0 comp rx=0 tx=0).stop 2024-09-11T00:14:54.856 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:54.851+0000 7f4cc9200640 1 -- 172.21.15.12:0/3530812176 shutdown_connections 2024-09-11T00:14:54.857 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:54.851+0000 7f4cc9200640 1 --2- 172.21.15.12:0/3530812176 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f4ca80b5360 0x7f4ca80b7820 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:14:54.857 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:54.851+0000 7f4cc9200640 1 --2- 172.21.15.12:0/3530812176 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4cc41031a0 0x7f4cc419bfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:14:54.857 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:54.851+0000 7f4cc9200640 1 --2- 172.21.15.12:0/3530812176 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4cc41027e0 0x7f4cc419b390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:14:54.857 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:54.851+0000 7f4cc9200640 1 --2- 172.21.15.12:0/3530812176 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4cc4101690 0x7f4cc419ad40 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:14:54.857 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:54.851+0000 7f4cc9200640 1 -- 172.21.15.12:0/3530812176 >> 172.21.15.12:0/3530812176 conn(0x7f4cc40fcf30 msgr2=0x7f4cc40fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:14:54.857 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:54.851+0000 7f4cc9200640 1 -- 172.21.15.12:0/3530812176 shutdown_connections 2024-09-11T00:14:54.857 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:14:54.852+0000 7f4cc9200640 1 -- 172.21.15.12:0/3530812176 wait complete. 2024-09-11T00:14:55.656 INFO:tasks.workunit.client.0.smithi012.stdout: 1 291337 3.60 MB/sec execute 292 sec latency 238.176 ms 2024-09-11T00:14:55.782 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:14:55 smithi165 ceph-mon[35356]: pgmap v634: 129 pgs: 129 active+clean; 75 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 48 MiB/s rd, 705 KiB/s wr, 388 op/s 2024-09-11T00:14:55.813 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:14:55 smithi012 ceph-mon[30333]: pgmap v634: 129 pgs: 129 active+clean; 75 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 48 MiB/s rd, 705 KiB/s wr, 388 op/s 2024-09-11T00:14:55.830 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:14:55 smithi178 ceph-mon[35919]: pgmap v634: 129 pgs: 129 active+clean; 75 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 48 MiB/s rd, 705 KiB/s wr, 388 op/s 2024-09-11T00:14:56.656 INFO:tasks.workunit.client.0.smithi012.stdout: 1 293365 3.61 MB/sec execute 293 sec latency 10.553 ms 2024-09-11T00:14:57.657 INFO:tasks.workunit.client.0.smithi012.stdout: 1 294585 3.64 MB/sec execute 294 sec latency 359.570 ms 2024-09-11T00:14:57.782 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:14:57 smithi165 ceph-mon[35356]: pgmap v635: 129 pgs: 129 active+clean; 73 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 50 MiB/s rd, 1.0 MiB/s wr, 453 op/s 2024-09-11T00:14:57.813 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:14:57 smithi012 ceph-mon[30333]: pgmap v635: 129 pgs: 129 active+clean; 73 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 50 MiB/s rd, 1.0 MiB/s wr, 453 op/s 2024-09-11T00:14:57.831 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:14:57 smithi178 ceph-mon[35919]: pgmap v635: 129 pgs: 129 active+clean; 73 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 50 MiB/s rd, 1.0 MiB/s wr, 453 op/s 2024-09-11T00:14:58.657 INFO:tasks.workunit.client.0.smithi012.stdout: 1 294655 3.63 MB/sec execute 295 sec latency 717.757 ms 2024-09-11T00:14:59.657 INFO:tasks.workunit.client.0.smithi012.stdout: 1 295231 3.62 MB/sec execute 296 sec latency 1082.283 ms 2024-09-11T00:14:59.782 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:14:59 smithi165 ceph-mon[35356]: pgmap v636: 129 pgs: 129 active+clean; 73 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 45 MiB/s rd, 815 KiB/s wr, 356 op/s 2024-09-11T00:14:59.813 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:14:59 smithi012 ceph-mon[30333]: pgmap v636: 129 pgs: 129 active+clean; 73 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 45 MiB/s rd, 815 KiB/s wr, 356 op/s 2024-09-11T00:14:59.830 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:14:59 smithi178 ceph-mon[35919]: pgmap v636: 129 pgs: 129 active+clean; 73 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 45 MiB/s rd, 815 KiB/s wr, 356 op/s 2024-09-11T00:15:00.657 INFO:tasks.workunit.client.0.smithi012.stdout: 1 297605 3.65 MB/sec execute 297 sec latency 19.976 ms 2024-09-11T00:15:01.657 INFO:tasks.workunit.client.0.smithi012.stdout: 1 298641 3.66 MB/sec execute 298 sec latency 81.307 ms 2024-09-11T00:15:01.782 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:15:01 smithi165 ceph-mon[35356]: pgmap v637: 129 pgs: 129 active+clean; 77 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 33 MiB/s rd, 1.4 MiB/s wr, 376 op/s 2024-09-11T00:15:01.813 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:15:01 smithi012 ceph-mon[30333]: pgmap v637: 129 pgs: 129 active+clean; 77 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 33 MiB/s rd, 1.4 MiB/s wr, 376 op/s 2024-09-11T00:15:01.830 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:15:01 smithi178 ceph-mon[35919]: pgmap v637: 129 pgs: 129 active+clean; 77 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 33 MiB/s rd, 1.4 MiB/s wr, 376 op/s 2024-09-11T00:15:02.657 INFO:tasks.workunit.client.0.smithi012.stdout: 1 300903 3.69 MB/sec execute 299 sec latency 15.421 ms 2024-09-11T00:15:03.514 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:15:03 smithi012 ceph-mon[30333]: pgmap v638: 129 pgs: 129 active+clean; 78 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 29 MiB/s rd, 1.4 MiB/s wr, 343 op/s 2024-09-11T00:15:03.657 INFO:tasks.workunit.client.0.smithi012.stdout: 1 301582 3.69 MB/sec execute 300 sec latency 620.005 ms 2024-09-11T00:15:03.782 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:15:03 smithi165 ceph-mon[35356]: pgmap v638: 129 pgs: 129 active+clean; 78 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 29 MiB/s rd, 1.4 MiB/s wr, 343 op/s 2024-09-11T00:15:03.831 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:15:03 smithi178 ceph-mon[35919]: pgmap v638: 129 pgs: 129 active+clean; 78 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 29 MiB/s rd, 1.4 MiB/s wr, 343 op/s 2024-09-11T00:15:04.657 INFO:tasks.workunit.client.0.smithi012.stdout: 1 301685 3.68 MB/sec execute 301 sec latency 1291.798 ms 2024-09-11T00:15:05.657 INFO:tasks.workunit.client.0.smithi012.stdout: 1 302501 3.67 MB/sec execute 302 sec latency 219.185 ms 2024-09-11T00:15:05.782 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:15:05 smithi165 ceph-mon[35356]: pgmap v639: 129 pgs: 129 active+clean; 78 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 19 MiB/s rd, 1.3 MiB/s wr, 302 op/s 2024-09-11T00:15:05.813 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:15:05 smithi012 ceph-mon[30333]: pgmap v639: 129 pgs: 129 active+clean; 78 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 19 MiB/s rd, 1.3 MiB/s wr, 302 op/s 2024-09-11T00:15:05.830 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:15:05 smithi178 ceph-mon[35919]: pgmap v639: 129 pgs: 129 active+clean; 78 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 19 MiB/s rd, 1.3 MiB/s wr, 302 op/s 2024-09-11T00:15:06.657 INFO:tasks.workunit.client.0.smithi012.stdout: 1 305022 3.72 MB/sec execute 303 sec latency 19.450 ms 2024-09-11T00:15:07.657 INFO:tasks.workunit.client.0.smithi012.stdout: 1 305685 3.71 MB/sec execute 304 sec latency 160.935 ms 2024-09-11T00:15:07.782 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:15:07 smithi165 ceph-mon[35356]: pgmap v640: 129 pgs: 129 active+clean; 84 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 20 MiB/s rd, 2.0 MiB/s wr, 332 op/s 2024-09-11T00:15:07.813 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:15:07 smithi012 ceph-mon[30333]: pgmap v640: 129 pgs: 129 active+clean; 84 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 20 MiB/s rd, 2.0 MiB/s wr, 332 op/s 2024-09-11T00:15:07.830 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:15:07 smithi178 ceph-mon[35919]: pgmap v640: 129 pgs: 129 active+clean; 84 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 20 MiB/s rd, 2.0 MiB/s wr, 332 op/s 2024-09-11T00:15:07.983 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:07.981+0000 7f7f8b600640 1 -- 172.21.15.12:0/2445877818 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7f841064a0 msgr2=0x7f7f84111180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:15:07.984 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:07.981+0000 7f7f8b600640 1 --2- 172.21.15.12:0/2445877818 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7f841064a0 0x7f7f84111180 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f7f7c009ca0 tx=0x7f7f7c02a8c0 comp rx=0 tx=0).stop 2024-09-11T00:15:07.984 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:07.981+0000 7f7f8b600640 1 -- 172.21.15.12:0/2445877818 shutdown_connections 2024-09-11T00:15:07.984 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:07.981+0000 7f7f8b600640 1 --2- 172.21.15.12:0/2445877818 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7f841064a0 0x7f7f84111180 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:15:07.984 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:07.981+0000 7f7f8b600640 1 --2- 172.21.15.12:0/2445877818 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7f84075b20 0x7f7f84075fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:15:07.984 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:07.981+0000 7f7f8b600640 1 --2- 172.21.15.12:0/2445877818 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7f84076f90 0x7f7f840755e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:15:07.984 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:07.981+0000 7f7f8b600640 1 -- 172.21.15.12:0/2445877818 >> 172.21.15.12:0/2445877818 conn(0x7f7f840fe250 msgr2=0x7f7f84100670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:15:07.984 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:07.981+0000 7f7f8b600640 1 -- 172.21.15.12:0/2445877818 shutdown_connections 2024-09-11T00:15:07.984 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:07.981+0000 7f7f8b600640 1 -- 172.21.15.12:0/2445877818 wait complete. 2024-09-11T00:15:07.986 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:07.982+0000 7f7f8b600640 1 Processor -- start 2024-09-11T00:15:07.987 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:07.982+0000 7f7f8b600640 1 -- start start 2024-09-11T00:15:07.987 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:07.982+0000 7f7f8b600640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7f84075b20 0x7f7f8419f360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:15:07.987 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:07.982+0000 7f7f8b600640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7f84076f90 0x7f7f8419f9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:15:07.987 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:07.982+0000 7f7f8b600640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7f841a05e0 0x7f7f841a0a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:15:07.987 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:07.983+0000 7f7f8b600640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f7f84114c30 con 0x7f7f84076f90 2024-09-11T00:15:07.987 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:07.983+0000 7f7f8b600640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f7f84114ab0 con 0x7f7f841a05e0 2024-09-11T00:15:07.987 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:07.983+0000 7f7f8b600640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f7f84114db0 con 0x7f7f84075b20 2024-09-11T00:15:07.987 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:07.983+0000 7f7f88c00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7f84075b20 0x7f7f8419f360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:15:07.988 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:07.983+0000 7f7f88c00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7f84075b20 0x7f7f8419f360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.12:45956/0 (socket says 172.21.15.12:45956) 2024-09-11T00:15:07.988 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:07.983+0000 7f7f88c00640 1 -- 172.21.15.12:0/924142935 learned_addr learned my addr 172.21.15.12:0/924142935 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:15:07.988 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:07.983+0000 7f7f83e00640 1 --2- 172.21.15.12:0/924142935 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7f84076f90 0x7f7f8419f9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:15:07.988 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:07.983+0000 7f7f89600640 1 --2- 172.21.15.12:0/924142935 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7f841a05e0 0x7f7f841a0a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:15:07.988 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:07.983+0000 7f7f88c00640 1 -- 172.21.15.12:0/924142935 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7f841a05e0 msgr2=0x7f7f841a0a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:15:07.988 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:07.983+0000 7f7f88c00640 1 --2- 172.21.15.12:0/924142935 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7f841a05e0 0x7f7f841a0a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:15:07.988 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:07.983+0000 7f7f88c00640 1 -- 172.21.15.12:0/924142935 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7f84076f90 msgr2=0x7f7f8419f9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:15:07.988 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:07.983+0000 7f7f88c00640 1 --2- 172.21.15.12:0/924142935 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7f84076f90 0x7f7f8419f9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:15:07.988 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:07.983+0000 7f7f88c00640 1 -- 172.21.15.12:0/924142935 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f841a1290 con 0x7f7f84075b20 2024-09-11T00:15:07.988 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:07.983+0000 7f7f89600640 1 --2- 172.21.15.12:0/924142935 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7f841a05e0 0x7f7f841a0a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T00:15:07.988 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:07.983+0000 7f7f83e00640 1 --2- 172.21.15.12:0/924142935 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7f84076f90 0x7f7f8419f9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T00:15:07.988 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:07.983+0000 7f7f88c00640 1 --2- 172.21.15.12:0/924142935 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7f84075b20 0x7f7f8419f360 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f7f7800e9f0 tx=0x7f7f7800edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:15:07.988 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:07.984+0000 7f7f81600640 1 -- 172.21.15.12:0/924142935 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f7800cd80 con 0x7f7f84075b20 2024-09-11T00:15:07.989 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:07.984+0000 7f7f81600640 1 -- 172.21.15.12:0/924142935 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7f780054d0 con 0x7f7f84075b20 2024-09-11T00:15:07.989 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:07.984+0000 7f7f8b600640 1 -- 172.21.15.12:0/924142935 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7f841a7680 con 0x7f7f84075b20 2024-09-11T00:15:07.989 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:07.984+0000 7f7f81600640 1 -- 172.21.15.12:0/924142935 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f78010690 con 0x7f7f84075b20 2024-09-11T00:15:07.989 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:07.984+0000 7f7f8b600640 1 -- 172.21.15.12:0/924142935 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7f841a7b20 con 0x7f7f84075b20 2024-09-11T00:15:07.993 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:07.988+0000 7f7f8e438740 1 Processor -- start 2024-09-11T00:15:07.993 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:07.988+0000 7f7f8e438740 1 -- start start 2024-09-11T00:15:07.993 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:07.989+0000 7f7f8e438740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560aecafe790 0x560aecb00b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:15:07.993 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:07.989+0000 7f7f8e438740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560aecb010c0 0x560aecb034b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:15:07.993 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:07.989+0000 7f7f8e438740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560aecb03b00 0x560aecb05fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:15:07.993 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:07.989+0000 7f7f8e438740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x560aeca716b0 con 0x560aecafe790 2024-09-11T00:15:07.993 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:07.989+0000 7f7f8e438740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x560aeca60ae0 con 0x560aecb010c0 2024-09-11T00:15:07.993 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:07.989+0000 7f7f8e438740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x560aeca648f0 con 0x560aecb03b00 2024-09-11T00:15:07.993 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:07.989+0000 7f7f81600640 1 -- 172.21.15.12:0/924142935 <== mon.1 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7f7f78004e90 con 0x7f7f84075b20 2024-09-11T00:15:07.994 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:07.989+0000 7f7f83e00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560aecb03b00 0x560aecb05fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:15:07.994 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:07.989+0000 7f7f89600640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560aecafe790 0x560aecb00b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:15:07.994 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:07.989+0000 7f7f83e00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560aecb03b00 0x560aecb05fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.12:45966/0 (socket says 172.21.15.12:45966) 2024-09-11T00:15:07.994 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:07.989+0000 7f7f83e00640 1 -- 172.21.15.12:0/121390357 learned_addr learned my addr 172.21.15.12:0/121390357 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:15:07.994 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:07.989+0000 7f7f88c00640 1 --2- 172.21.15.12:0/121390357 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560aecb010c0 0x560aecb034b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:15:07.995 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:07.989+0000 7f7f83e00640 1 -- 172.21.15.12:0/121390357 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560aecb010c0 msgr2=0x560aecb034b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:15:07.995 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:07.989+0000 7f7f83e00640 1 --2- 172.21.15.12:0/121390357 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560aecb010c0 0x560aecb034b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:15:07.995 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:07.989+0000 7f7f83e00640 1 -- 172.21.15.12:0/121390357 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560aecafe790 msgr2=0x560aecb00b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:15:07.995 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:07.989+0000 7f7f83e00640 1 --2- 172.21.15.12:0/121390357 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560aecafe790 0x560aecb00b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:15:07.995 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:07.989+0000 7f7f83e00640 1 -- 172.21.15.12:0/121390357 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560aeca2d780 con 0x560aecb03b00 2024-09-11T00:15:07.995 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:07.989+0000 7f7f88c00640 1 --2- 172.21.15.12:0/121390357 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560aecb010c0 0x560aecb034b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T00:15:07.995 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:07.990+0000 7f7f83e00640 1 --2- 172.21.15.12:0/121390357 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560aecb03b00 0x560aecb05fc0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f7f70009d50 tx=0x7f7f700117b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fb94352c1010b6e9 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:15:07.995 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:07.990+0000 7f7f75c00640 1 -- 172.21.15.12:0/121390357 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f70012650 con 0x560aecb03b00 2024-09-11T00:15:07.995 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:07.990+0000 7f7f75c00640 1 -- 172.21.15.12:0/121390357 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7f70009e40 con 0x560aecb03b00 2024-09-11T00:15:07.995 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:07.990+0000 7f7f75c00640 1 -- 172.21.15.12:0/121390357 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f7000fbb0 con 0x560aecb03b00 2024-09-11T00:15:07.996 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:07.991+0000 7f7f8e438740 1 -- 172.21.15.12:0/121390357 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560aecb03b00 msgr2=0x560aecb05fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:15:07.996 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:07.991+0000 7f7f8e438740 1 --2- 172.21.15.12:0/121390357 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560aecb03b00 0x560aecb05fc0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f7f70009d50 tx=0x7f7f700117b0 comp rx=0 tx=0).stop 2024-09-11T00:15:07.996 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:07.991+0000 7f7f8e438740 1 -- 172.21.15.12:0/121390357 shutdown_connections 2024-09-11T00:15:07.996 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:07.991+0000 7f7f8e438740 1 --2- 172.21.15.12:0/121390357 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560aecb03b00 0x560aecb05fc0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:15:07.996 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:07.991+0000 7f7f8e438740 1 --2- 172.21.15.12:0/121390357 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560aecb010c0 0x560aecb034b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:15:07.996 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:07.991+0000 7f7f8e438740 1 --2- 172.21.15.12:0/121390357 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560aecafe790 0x560aecb00b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:15:07.996 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:07.991+0000 7f7f8e438740 1 -- 172.21.15.12:0/121390357 >> 172.21.15.12:0/121390357 conn(0x560aecaf98e0 msgr2=0x560aecafa670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:15:07.996 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:07.991+0000 7f7f8e438740 1 -- 172.21.15.12:0/121390357 shutdown_connections 2024-09-11T00:15:07.996 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:07.991+0000 7f7f8e438740 1 -- 172.21.15.12:0/121390357 wait complete. 2024-09-11T00:15:08.001 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:07.991+0000 7f7f81600640 1 --2- 172.21.15.12:0/924142935 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f7f6c0b5210 0x7f7f6c0b76d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:15:08.001 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:07.991+0000 7f7f8e438740 1 -- start start 2024-09-11T00:15:08.001 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:07.991+0000 7f7f81600640 1 -- 172.21.15.12:0/924142935 <== mon.1 v2:172.21.15.178:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9208+0+0 (secure 0 0 0) 0x7f7f780e6510 con 0x7f7f84075b20 2024-09-11T00:15:08.001 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:07.991+0000 7f7f89600640 1 --2- 172.21.15.12:0/924142935 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f7f6c0b5210 0x7f7f6c0b76d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:15:08.001 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:07.992+0000 7f7f8e438740 1 Processor -- start 2024-09-11T00:15:08.001 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:07.992+0000 7f7f8e438740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560aecafe790 0x560aecb07e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:15:08.001 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:07.992+0000 7f7f8e438740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560aecb010c0 0x560aecb084a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:15:08.001 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:07.992+0000 7f7f89600640 1 --2- 172.21.15.12:0/924142935 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f7f6c0b5210 0x7f7f6c0b76d0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f7f7c005e30 tx=0x7f7f7c031b10 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:15:08.001 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:07.992+0000 7f7f8e438740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560aecb03b00 0x560aecb09070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:15:08.001 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:07.992+0000 7f7f8e438740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x560aec998ab0 con 0x560aecb03b00 2024-09-11T00:15:08.001 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:07.992+0000 7f7f8e438740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x560aeca07570 con 0x560aecafe790 2024-09-11T00:15:08.002 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:07.992+0000 7f7f8e438740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x560aecb06ae0 con 0x560aecb010c0 2024-09-11T00:15:08.002 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:07.992+0000 7f7f8e438740 10 client.0 _ll_register_callbacks cb 0x560aeca16100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:15:08.002 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:07.993+0000 7f7f83e00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560aecb010c0 0x560aecb084a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:15:08.002 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:07.993+0000 7f7f83e00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560aecb010c0 0x560aecb084a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.12:45974/0 (socket says 172.21.15.12:45974) 2024-09-11T00:15:08.002 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:07.993+0000 7f7f83e00640 1 -- 172.21.15.12:0/598553459 learned_addr learned my addr 172.21.15.12:0/598553459 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:15:08.002 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:07.993+0000 7f7f89600640 1 --2- 172.21.15.12:0/598553459 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560aecb03b00 0x560aecb09070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:15:08.002 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:07.993+0000 7f7f88c00640 1 --2- 172.21.15.12:0/598553459 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560aecafe790 0x560aecb07e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:15:08.002 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:07.993+0000 7f7f83e00640 1 -- 172.21.15.12:0/598553459 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560aecafe790 msgr2=0x560aecb07e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:15:08.002 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:07.993+0000 7f7f83e00640 1 --2- 172.21.15.12:0/598553459 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560aecafe790 0x560aecb07e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:15:08.002 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:07.993+0000 7f7f83e00640 1 -- 172.21.15.12:0/598553459 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560aecb03b00 msgr2=0x560aecb09070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:15:08.002 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:07.993+0000 7f7f83e00640 1 --2- 172.21.15.12:0/598553459 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560aecb03b00 0x560aecb09070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:15:08.002 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:07.993+0000 7f7f83e00640 1 -- 172.21.15.12:0/598553459 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560aec7c2790 con 0x560aecb010c0 2024-09-11T00:15:08.002 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:07.993+0000 7f7f83e00640 1 --2- 172.21.15.12:0/598553459 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560aecb010c0 0x560aecb084a0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f7f70009d20 tx=0x7f7f70004540 comp rx=0 tx=0).ready entity=mon.1 client_cookie=61549a406d650294 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:15:08.003 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:07.993+0000 7f7f6a000640 10 client.0 ms_handle_connect on v2:172.21.15.178:3300/0 2024-09-11T00:15:08.003 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:07.994+0000 7f7f6a000640 1 -- 172.21.15.12:0/598553459 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f70027020 con 0x560aecb010c0 2024-09-11T00:15:08.003 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:07.994+0000 7f7f6a000640 1 -- 172.21.15.12:0/598553459 <== mon.1 v2:172.21.15.178:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9208+0+0 (secure 0 0 0) 0x7f7f7000fd80 con 0x560aecb010c0 2024-09-11T00:15:08.003 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:07.994+0000 7f7f8e438740 1 -- 172.21.15.12:0/598553459 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x560aeca73600 con 0x560aecb010c0 2024-09-11T00:15:08.003 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:07.994+0000 7f7f6a000640 1 -- 172.21.15.12:0/598553459 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f7f70009940 con 0x560aecb010c0 2024-09-11T00:15:08.003 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:07.994+0000 7f7f8e438740 10 client.26407 fetch_fsmap learned FSMap version 41 2024-09-11T00:15:08.003 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:07.994+0000 7f7f8e438740 1 -- 172.21.15.12:0/598553459 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x560aec83b620 con 0x560aecb010c0 2024-09-11T00:15:08.003 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:07.994+0000 7f7f6a000640 1 -- 172.21.15.12:0/598553459 <== mon.1 v2:172.21.15.178:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x560aeca73600 con 0x560aecb010c0 2024-09-11T00:15:08.003 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:07.994+0000 7f7f6a000640 1 -- 172.21.15.12:0/598553459 <== mon.1 v2:172.21.15.178:3300/0 5 ==== fsmap(e 41) ==== 8370+0+0 (secure 0 0 0) 0x7f7f70004ca0 con 0x560aecb010c0 2024-09-11T00:15:08.003 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:07.995+0000 7f7f8e438740 10 client.26407 fetch_fsmap finished waiting for FSMap version 41 2024-09-11T00:15:08.003 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:07.995+0000 7f7f8e438740 10 client.26407 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T00:15:08.003 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:07.995+0000 7f7f8e438740 20 client.26407 populate_metadata read hostname 'smithi012' 2024-09-11T00:15:08.003 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:07.995+0000 7f7f8e438740 1 --2- 172.21.15.12:0/598553459 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x560aecb13380 0x560aecb15770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:15:08.004 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:07.995+0000 7f7f8e438740 4 client.26407 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T00:15:08.004 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:07.995+0000 7f7f8e438740 1 -- 172.21.15.12:0/598553459 --> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560aeca6f0b0 con 0x560aecb13380 2024-09-11T00:15:08.004 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:07.995+0000 7f7f88c00640 1 --2- 172.21.15.12:0/598553459 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x560aecb13380 0x560aecb15770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:15:08.004 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:07.995+0000 7f7f88c00640 1 --2- 172.21.15.12:0/598553459 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x560aecb13380 0x560aecb15770 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=14480d4f3ba3ddf0 server_cookie=23e2091d0a01217d in_seq=0 out_seq=0 2024-09-11T00:15:08.004 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:07.995+0000 7f7f6a000640 10 client.26407 ms_handle_connect on v2:172.21.15.165:6838/501312984 2024-09-11T00:15:08.004 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:07.996+0000 7f7f6a000640 1 -- 172.21.15.12:0/598553459 <== mds.0 v2:172.21.15.165:6838/501312984 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x560aeca6f0b0 con 0x560aecb13380 2024-09-11T00:15:08.004 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:07.996+0000 7f7f6a000640 10 client.26407 handle_command_reply: tid=0 2024-09-11T00:15:08.004 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:07.996+0000 7f7f8e438740 1 client.26407 shutdown 2024-09-11T00:15:08.004 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:07.996+0000 7f7f8e438740 1 -- 172.21.15.12:0/598553459 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560aecb010c0 msgr2=0x560aecb084a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:15:08.004 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:07.996+0000 7f7f8e438740 1 --2- 172.21.15.12:0/598553459 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560aecb010c0 0x560aecb084a0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f7f70009d20 tx=0x7f7f70004540 comp rx=0 tx=0).stop 2024-09-11T00:15:08.004 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:07.996+0000 7f7f8e438740 1 -- 172.21.15.12:0/598553459 shutdown_connections 2024-09-11T00:15:08.004 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:07.996+0000 7f7f8e438740 1 --2- 172.21.15.12:0/598553459 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x560aecb13380 0x560aecb15770 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:15:08.004 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:07.996+0000 7f7f8e438740 1 --2- 172.21.15.12:0/598553459 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560aecb03b00 0x560aecb09070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:15:08.005 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:07.996+0000 7f7f8e438740 1 --2- 172.21.15.12:0/598553459 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560aecb010c0 0x560aecb084a0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:15:08.005 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:07.996+0000 7f7f6a000640 0 client.26407 ms_handle_reset on v2:172.21.15.165:6838/501312984 2024-09-11T00:15:08.005 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:07.996+0000 7f7f8e438740 1 --2- 172.21.15.12:0/598553459 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560aecafe790 0x560aecb07e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:15:08.005 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:07.996+0000 7f7f8e438740 1 -- 172.21.15.12:0/598553459 >> 172.21.15.12:0/598553459 conn(0x560aecaf98e0 msgr2=0x560aecaf8a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:15:08.005 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:07.997+0000 7f7f8e438740 1 -- 172.21.15.12:0/598553459 shutdown_connections 2024-09-11T00:15:08.005 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:07.997+0000 7f7f8e438740 1 -- 172.21.15.12:0/598553459 wait complete. 2024-09-11T00:15:08.005 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:07.997+0000 7f7f8e438740 20 client.26407 trim_cache size 0 max 16384 2024-09-11T00:15:08.017 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:08.014+0000 7f7f8e438740 1 Processor -- start 2024-09-11T00:15:08.017 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:08.015+0000 7f7f8e438740 1 -- start start 2024-09-11T00:15:08.018 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:08.015+0000 7f7f8e438740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560aecafe790 0x560aecafdb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:15:08.018 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:08.015+0000 7f7f8e438740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560aecb010c0 0x560aecafe0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:15:08.018 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:08.015+0000 7f7f8e438740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560aecb03b00 0x560aecb07e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:15:08.018 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:08.015+0000 7f7f8e438740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f7f841b1d90 con 0x560aecb010c0 2024-09-11T00:15:08.018 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:08.015+0000 7f7f8e438740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f7f841b38d0 con 0x560aecb03b00 2024-09-11T00:15:08.018 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:08.015+0000 7f7f8e438740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f7f841b2460 con 0x560aecafe790 2024-09-11T00:15:08.018 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:08.015+0000 7f7f88c00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560aecafe790 0x560aecafdb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:15:08.018 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:08.015+0000 7f7f89600640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560aecb03b00 0x560aecb07e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:15:08.018 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:08.015+0000 7f7f83e00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560aecb010c0 0x560aecafe0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:15:08.018 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:08.015+0000 7f7f88c00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560aecafe790 0x560aecafdb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.12:45988/0 (socket says 172.21.15.12:45988) 2024-09-11T00:15:08.018 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:08.015+0000 7f7f89600640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560aecb03b00 0x560aecb07e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.12:41594/0 (socket says 172.21.15.12:41594) 2024-09-11T00:15:08.018 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:08.015+0000 7f7f88c00640 1 -- 172.21.15.12:0/1037323274 learned_addr learned my addr 172.21.15.12:0/1037323274 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:15:08.019 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:08.015+0000 7f7f88c00640 1 -- 172.21.15.12:0/1037323274 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560aecb03b00 msgr2=0x560aecb07e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:15:08.019 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:08.015+0000 7f7f88c00640 1 --2- 172.21.15.12:0/1037323274 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560aecb03b00 0x560aecb07e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:15:08.019 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:08.015+0000 7f7f88c00640 1 -- 172.21.15.12:0/1037323274 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560aecb010c0 msgr2=0x560aecafe0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:15:08.019 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:08.016+0000 7f7f88c00640 1 --2- 172.21.15.12:0/1037323274 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560aecb010c0 0x560aecafe0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:15:08.019 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:08.016+0000 7f7f88c00640 1 -- 172.21.15.12:0/1037323274 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560aecb05cb0 con 0x560aecafe790 2024-09-11T00:15:08.022 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:08.016+0000 7f7f88c00640 1 --2- 172.21.15.12:0/1037323274 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560aecafe790 0x560aecafdb90 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f7f780a9b10 tx=0x7f7f78004a90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=87f15f704f9e3d5c server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:15:08.022 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:08.016+0000 7f7f75200640 1 -- 172.21.15.12:0/1037323274 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f780ac3b0 con 0x560aecafe790 2024-09-11T00:15:08.022 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:08.016+0000 7f7f75200640 1 -- 172.21.15.12:0/1037323274 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7f78005680 con 0x560aecafe790 2024-09-11T00:15:08.022 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:08.016+0000 7f7f75200640 1 -- 172.21.15.12:0/1037323274 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f780046c0 con 0x560aecafe790 2024-09-11T00:15:08.022 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:08.017+0000 7f7f8e438740 1 -- 172.21.15.12:0/1037323274 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560aecafe790 msgr2=0x560aecafdb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:15:08.022 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:08.017+0000 7f7f8e438740 1 --2- 172.21.15.12:0/1037323274 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560aecafe790 0x560aecafdb90 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f7f780a9b10 tx=0x7f7f78004a90 comp rx=0 tx=0).stop 2024-09-11T00:15:08.022 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:08.017+0000 7f7f8e438740 1 -- 172.21.15.12:0/1037323274 shutdown_connections 2024-09-11T00:15:08.022 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:08.017+0000 7f7f8e438740 1 --2- 172.21.15.12:0/1037323274 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560aecb03b00 0x560aecb07e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:15:08.022 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:08.017+0000 7f7f8e438740 1 --2- 172.21.15.12:0/1037323274 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560aecb010c0 0x560aecafe0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:15:08.023 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:08.017+0000 7f7f8e438740 1 --2- 172.21.15.12:0/1037323274 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560aecafe790 0x560aecafdb90 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:15:08.023 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:08.017+0000 7f7f8e438740 1 -- 172.21.15.12:0/1037323274 >> 172.21.15.12:0/1037323274 conn(0x560aecaf98e0 msgr2=0x560aecb03ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:15:08.023 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:08.017+0000 7f7f8e438740 1 -- 172.21.15.12:0/1037323274 shutdown_connections 2024-09-11T00:15:08.023 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:08.017+0000 7f7f8e438740 1 -- 172.21.15.12:0/1037323274 wait complete. 2024-09-11T00:15:08.023 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:08.017+0000 7f7f8e438740 1 -- start start 2024-09-11T00:15:08.023 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:08.017+0000 7f7f8e438740 1 Processor -- start 2024-09-11T00:15:08.023 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:08.018+0000 7f7f8e438740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560aecafe790 0x560aecb0fb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:15:08.023 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:08.018+0000 7f7f8e438740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560aecb010c0 0x560aecb0e980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:15:08.023 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:08.018+0000 7f7f8e438740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560aecb03b00 0x560aecb0f070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:15:08.023 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:08.018+0000 7f7f8e438740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f7f841b3200 con 0x560aecb010c0 2024-09-11T00:15:08.023 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:08.018+0000 7f7f8e438740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f7f841b2b30 con 0x560aecafe790 2024-09-11T00:15:08.023 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:08.018+0000 7f7f8e438740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f7f841afd70 con 0x560aecb03b00 2024-09-11T00:15:08.023 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:08.018+0000 7f7f8e438740 10 client.0 _ll_register_callbacks cb 0x7f7f70005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:15:08.024 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:08.018+0000 7f7f88c00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560aecafe790 0x560aecb0fb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:15:08.024 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:08.018+0000 7f7f89600640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560aecb03b00 0x560aecb0f070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:15:08.024 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:08.018+0000 7f7f88c00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560aecafe790 0x560aecb0fb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.12:41596/0 (socket says 172.21.15.12:41596) 2024-09-11T00:15:08.024 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:08.018+0000 7f7f88c00640 1 -- 172.21.15.12:0/2442858865 learned_addr learned my addr 172.21.15.12:0/2442858865 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:15:08.024 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:08.018+0000 7f7f83e00640 1 --2- 172.21.15.12:0/2442858865 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560aecb010c0 0x560aecb0e980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:15:08.024 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:08.019+0000 7f7f88c00640 1 -- 172.21.15.12:0/2442858865 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560aecb03b00 msgr2=0x560aecb0f070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:15:08.024 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:08.019+0000 7f7f88c00640 1 --2- 172.21.15.12:0/2442858865 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560aecb03b00 0x560aecb0f070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:15:08.024 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:08.019+0000 7f7f88c00640 1 -- 172.21.15.12:0/2442858865 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560aecb010c0 msgr2=0x560aecb0e980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:15:08.024 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:08.019+0000 7f7f88c00640 1 --2- 172.21.15.12:0/2442858865 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560aecb010c0 0x560aecb0e980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:15:08.024 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:08.019+0000 7f7f88c00640 1 -- 172.21.15.12:0/2442858865 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560aecb0d3b0 con 0x560aecafe790 2024-09-11T00:15:08.024 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:08.019+0000 7f7f89600640 1 --2- 172.21.15.12:0/2442858865 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560aecb03b00 0x560aecb0f070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:15:08.024 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:08.019+0000 7f7f88c00640 1 --2- 172.21.15.12:0/2442858865 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560aecafe790 0x560aecb0fb90 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f7f780ac550 tx=0x7f7f78009720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e972b0a5ab8f81ef server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:15:08.025 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:08.019+0000 7f7f6a000640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-09-11T00:15:08.025 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:08.019+0000 7f7f6a000640 1 -- 172.21.15.12:0/2442858865 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f78097020 con 0x560aecafe790 2024-09-11T00:15:08.025 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:08.019+0000 7f7f8e438740 1 -- 172.21.15.12:0/2442858865 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7f78005680 con 0x560aecafe790 2024-09-11T00:15:08.025 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:08.019+0000 7f7f6a000640 1 -- 172.21.15.12:0/2442858865 <== mon.2 v2:172.21.15.165:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9208+0+0 (secure 0 0 0) 0x7f7f780051b0 con 0x560aecafe790 2024-09-11T00:15:08.025 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:08.019+0000 7f7f6a000640 1 -- 172.21.15.12:0/2442858865 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f7f780b1380 con 0x560aecafe790 2024-09-11T00:15:08.025 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:08.019+0000 7f7f8e438740 10 client.26420 fetch_fsmap learned FSMap version 41 2024-09-11T00:15:08.025 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:08.019+0000 7f7f8e438740 1 -- 172.21.15.12:0/2442858865 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x560aecb805e0 con 0x560aecafe790 2024-09-11T00:15:08.025 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:08.020+0000 7f7f6a000640 1 -- 172.21.15.12:0/2442858865 <== mon.2 v2:172.21.15.165:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7f78005680 con 0x560aecafe790 2024-09-11T00:15:08.025 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:08.020+0000 7f7f6a000640 1 -- 172.21.15.12:0/2442858865 <== mon.2 v2:172.21.15.165:3300/0 5 ==== fsmap(e 41) ==== 8370+0+0 (secure 0 0 0) 0x7f7f780a8b10 con 0x560aecafe790 2024-09-11T00:15:08.025 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:08.020+0000 7f7f8e438740 10 client.26420 fetch_fsmap finished waiting for FSMap version 41 2024-09-11T00:15:08.025 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:08.020+0000 7f7f8e438740 10 client.26420 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T00:15:08.025 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:08.020+0000 7f7f8e438740 20 client.26420 populate_metadata read hostname 'smithi012' 2024-09-11T00:15:08.025 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:08.020+0000 7f7f8e438740 1 --2- 172.21.15.12:0/2442858865 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x560aecb80f80 0x560aecb833a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:15:08.026 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:08.020+0000 7f7f8e438740 4 client.26420 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T00:15:08.026 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:08.020+0000 7f7f8e438740 1 -- 172.21.15.12:0/2442858865 --> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] -- command(tid 0: {"prefix": "flush journal"}) -- 0x560aeca6c4e0 con 0x560aecb80f80 2024-09-11T00:15:08.026 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:08.020+0000 7f7f83e00640 1 --2- 172.21.15.12:0/2442858865 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x560aecb80f80 0x560aecb833a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:15:08.026 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:08.021+0000 7f7f83e00640 1 --2- 172.21.15.12:0/2442858865 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x560aecb80f80 0x560aecb833a0 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fff4838a1c01be38 server_cookie=2848e6e0f3388ca9 in_seq=0 out_seq=0 2024-09-11T00:15:08.026 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:08.021+0000 7f7f6a000640 10 client.26420 ms_handle_connect on v2:172.21.15.165:6838/501312984 2024-09-11T00:15:08.135 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:08.131+0000 7f7f6a000640 1 -- 172.21.15.12:0/2442858865 <== mds.0 v2:172.21.15.165:6838/501312984 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x560aeca6c4e0 con 0x560aecb80f80 2024-09-11T00:15:08.136 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:08.131+0000 7f7f6a000640 10 client.26420 handle_command_reply: tid=0 2024-09-11T00:15:08.136 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:08.131+0000 7f7f8e438740 1 client.26420 shutdown 2024-09-11T00:15:08.136 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:08.132+0000 7f7f8e438740 1 -- 172.21.15.12:0/2442858865 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560aecafe790 msgr2=0x560aecb0fb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:15:08.136 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:08.132+0000 7f7f8e438740 1 --2- 172.21.15.12:0/2442858865 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560aecafe790 0x560aecb0fb90 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f7f780ac550 tx=0x7f7f78009720 comp rx=0 tx=0).stop 2024-09-11T00:15:08.136 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:08.132+0000 7f7f8e438740 1 -- 172.21.15.12:0/2442858865 shutdown_connections 2024-09-11T00:15:08.136 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:08.132+0000 7f7f8e438740 1 --2- 172.21.15.12:0/2442858865 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x560aecb80f80 0x560aecb833a0 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:15:08.136 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:08.132+0000 7f7f8e438740 1 --2- 172.21.15.12:0/2442858865 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560aecb03b00 0x560aecb0f070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:15:08.136 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:08.132+0000 7f7f8e438740 1 --2- 172.21.15.12:0/2442858865 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560aecb010c0 0x560aecb0e980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:15:08.136 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:08.132+0000 7f7f8e438740 1 --2- 172.21.15.12:0/2442858865 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560aecafe790 0x560aecb0fb90 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:15:08.136 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:08.132+0000 7f7f8e438740 1 -- 172.21.15.12:0/2442858865 >> 172.21.15.12:0/2442858865 conn(0x560aecaf98e0 msgr2=0x560aecb07e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:15:08.136 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:08.132+0000 7f7f6a000640 0 client.26420 ms_handle_reset on v2:172.21.15.165:6838/501312984 2024-09-11T00:15:08.136 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:08.133+0000 7f7f8e438740 1 -- 172.21.15.12:0/2442858865 shutdown_connections 2024-09-11T00:15:08.137 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:08.133+0000 7f7f8e438740 1 -- 172.21.15.12:0/2442858865 wait complete. 2024-09-11T00:15:08.137 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:08.133+0000 7f7f8e438740 20 client.26420 trim_cache size 0 max 16384 2024-09-11T00:15:08.142 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:{ 2024-09-11T00:15:08.142 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "message": "", 2024-09-11T00:15:08.142 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "return_code": 0 2024-09-11T00:15:08.142 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:} 2024-09-11T00:15:08.143 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:08.140+0000 7f7f8b600640 1 -- 172.21.15.12:0/924142935 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f7f6c0b5210 msgr2=0x7f7f6c0b76d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:15:08.143 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:08.140+0000 7f7f8b600640 1 --2- 172.21.15.12:0/924142935 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f7f6c0b5210 0x7f7f6c0b76d0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f7f7c005e30 tx=0x7f7f7c031b10 comp rx=0 tx=0).stop 2024-09-11T00:15:08.143 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:08.140+0000 7f7f8b600640 1 -- 172.21.15.12:0/924142935 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7f84075b20 msgr2=0x7f7f8419f360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:15:08.143 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:08.141+0000 7f7f8b600640 1 --2- 172.21.15.12:0/924142935 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7f84075b20 0x7f7f8419f360 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f7f7800e9f0 tx=0x7f7f7800edf0 comp rx=0 tx=0).stop 2024-09-11T00:15:08.143 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:08.141+0000 7f7f8b600640 1 -- 172.21.15.12:0/924142935 shutdown_connections 2024-09-11T00:15:08.143 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:08.141+0000 7f7f8b600640 1 --2- 172.21.15.12:0/924142935 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f7f6c0b5210 0x7f7f6c0b76d0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:15:08.143 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:08.141+0000 7f7f8b600640 1 --2- 172.21.15.12:0/924142935 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7f841a05e0 0x7f7f841a0a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:15:08.143 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:08.141+0000 7f7f8b600640 1 --2- 172.21.15.12:0/924142935 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7f84076f90 0x7f7f8419f9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:15:08.143 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:08.141+0000 7f7f8b600640 1 --2- 172.21.15.12:0/924142935 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7f84075b20 0x7f7f8419f360 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:15:08.144 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:08.141+0000 7f7f8b600640 1 -- 172.21.15.12:0/924142935 >> 172.21.15.12:0/924142935 conn(0x7f7f840fe250 msgr2=0x7f7f84100640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:15:08.156 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:08.141+0000 7f7f8b600640 1 -- 172.21.15.12:0/924142935 shutdown_connections 2024-09-11T00:15:08.156 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:08.142+0000 7f7f8b600640 1 -- 172.21.15.12:0/924142935 wait complete. 2024-09-11T00:15:08.657 INFO:tasks.workunit.client.0.smithi012.stdout: 1 306466 3.71 MB/sec execute 305 sec latency 378.531 ms 2024-09-11T00:15:09.657 INFO:tasks.workunit.client.0.smithi012.stdout: 1 307412 3.71 MB/sec execute 306 sec latency 732.285 ms 2024-09-11T00:15:09.782 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:15:09 smithi165 ceph-mon[35356]: pgmap v641: 129 pgs: 129 active+clean; 84 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 25 MiB/s rd, 1.7 MiB/s wr, 297 op/s 2024-09-11T00:15:09.813 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:15:09 smithi012 ceph-mon[30333]: pgmap v641: 129 pgs: 129 active+clean; 84 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 25 MiB/s rd, 1.7 MiB/s wr, 297 op/s 2024-09-11T00:15:09.830 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:15:09 smithi178 ceph-mon[35919]: pgmap v641: 129 pgs: 129 active+clean; 84 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 25 MiB/s rd, 1.7 MiB/s wr, 297 op/s 2024-09-11T00:15:10.657 INFO:tasks.workunit.client.0.smithi012.stdout: 1 308889 3.73 MB/sec execute 307 sec latency 104.881 ms 2024-09-11T00:15:11.148 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 28.4 2024-09-11T00:15:11.149 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid ff3ad74a-6fcf-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T00:15:11.563 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:15:11 smithi012 ceph-mon[30333]: pgmap v642: 129 pgs: 129 active+clean; 77 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 20 MiB/s rd, 2.3 MiB/s wr, 356 op/s 2024-09-11T00:15:11.570 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:15:11.657 INFO:tasks.workunit.client.0.smithi012.stdout: 1 310119 3.74 MB/sec execute 308 sec latency 70.591 ms 2024-09-11T00:15:11.782 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:15:11 smithi165 ceph-mon[35356]: pgmap v642: 129 pgs: 129 active+clean; 77 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 20 MiB/s rd, 2.3 MiB/s wr, 356 op/s 2024-09-11T00:15:11.831 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:15:11 smithi178 ceph-mon[35919]: pgmap v642: 129 pgs: 129 active+clean; 77 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 20 MiB/s rd, 2.3 MiB/s wr, 356 op/s 2024-09-11T00:15:12.582 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:12.580+0000 7fc055a00640 1 -- 172.21.15.12:0/3897784446 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc05010b190 msgr2=0x7fc050111990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:15:12.582 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:12.580+0000 7fc055a00640 1 --2- 172.21.15.12:0/3897784446 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc05010b190 0x7fc050111990 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fc048009ca0 tx=0x7fc04802a8c0 comp rx=0 tx=0).stop 2024-09-11T00:15:12.582 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:12.580+0000 7fc055a00640 1 -- 172.21.15.12:0/3897784446 shutdown_connections 2024-09-11T00:15:12.582 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:12.580+0000 7fc055a00640 1 --2- 172.21.15.12:0/3897784446 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc05010b190 0x7fc050111990 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:15:12.582 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:12.580+0000 7fc055a00640 1 --2- 172.21.15.12:0/3897784446 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc05010a810 0x7fc05010ac50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:15:12.582 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:12.580+0000 7fc055a00640 1 --2- 172.21.15.12:0/3897784446 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc050104e00 0x7fc0501051e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:15:12.582 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:12.580+0000 7fc055a00640 1 -- 172.21.15.12:0/3897784446 >> 172.21.15.12:0/3897784446 conn(0x7fc050100870 msgr2=0x7fc050102c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:15:12.582 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:12.580+0000 7fc055a00640 1 -- 172.21.15.12:0/3897784446 shutdown_connections 2024-09-11T00:15:12.583 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:12.580+0000 7fc055a00640 1 -- 172.21.15.12:0/3897784446 wait complete. 2024-09-11T00:15:12.583 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:12.581+0000 7fc055a00640 1 Processor -- start 2024-09-11T00:15:12.583 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:12.581+0000 7fc055a00640 1 -- start start 2024-09-11T00:15:12.583 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:12.582+0000 7fc055a00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc050104e00 0x7fc0500776b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:15:12.584 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:12.582+0000 7fc055a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc05010a810 0x7fc050077bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:15:12.584 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:12.582+0000 7fc055a00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc05010b190 0x7fc050078130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:15:12.584 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:12.582+0000 7fc055a00640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fc050114e50 con 0x7fc05010b190 2024-09-11T00:15:12.584 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:12.582+0000 7fc055a00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fc050114cd0 con 0x7fc05010a810 2024-09-11T00:15:12.584 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:12.582+0000 7fc055a00640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fc050114fd0 con 0x7fc050104e00 2024-09-11T00:15:12.585 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:12.582+0000 7fc04fe00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc050104e00 0x7fc0500776b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:15:12.585 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:12.582+0000 7fc04f400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc05010a810 0x7fc050077bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:15:12.585 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:12.582+0000 7fc04fe00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc050104e00 0x7fc0500776b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.12:57184/0 (socket says 172.21.15.12:57184) 2024-09-11T00:15:12.585 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:12.582+0000 7fc04fe00640 1 -- 172.21.15.12:0/261513082 learned_addr learned my addr 172.21.15.12:0/261513082 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:15:12.585 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:12.583+0000 7fc04f400640 1 -- 172.21.15.12:0/261513082 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc050104e00 msgr2=0x7fc0500776b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:15:12.585 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:12.583+0000 7fc04f400640 1 --2- 172.21.15.12:0/261513082 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc050104e00 0x7fc0500776b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:15:12.585 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:12.583+0000 7fc04f400640 1 -- 172.21.15.12:0/261513082 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc05010b190 msgr2=0x7fc050078130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:15:12.585 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:12.583+0000 7fc04f400640 1 --2- 172.21.15.12:0/261513082 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc05010b190 0x7fc050078130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:15:12.585 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:12.583+0000 7fc04f400640 1 -- 172.21.15.12:0/261513082 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0501ab7b0 con 0x7fc05010a810 2024-09-11T00:15:12.586 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:12.583+0000 7fc04fe00640 1 --2- 172.21.15.12:0/261513082 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc050104e00 0x7fc0500776b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:15:12.586 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:12.583+0000 7fc04f400640 1 --2- 172.21.15.12:0/261513082 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc05010a810 0x7fc050077bf0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fc03c00e9f0 tx=0x7fc03c00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:15:12.586 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:12.583+0000 7fc04cc00640 1 -- 172.21.15.12:0/261513082 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc03c00ccc0 con 0x7fc05010a810 2024-09-11T00:15:12.586 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:12.583+0000 7fc04cc00640 1 -- 172.21.15.12:0/261513082 <== mon.2 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc03c0108e0 con 0x7fc05010a810 2024-09-11T00:15:12.586 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:12.583+0000 7fc04cc00640 1 -- 172.21.15.12:0/261513082 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc03c018880 con 0x7fc05010a810 2024-09-11T00:15:12.586 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:12.584+0000 7fc055a00640 1 -- 172.21.15.12:0/261513082 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc0501abaa0 con 0x7fc05010a810 2024-09-11T00:15:12.586 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:12.584+0000 7fc055a00640 1 -- 172.21.15.12:0/261513082 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0501abf90 con 0x7fc05010a810 2024-09-11T00:15:12.586 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:12.585+0000 7fc055a00640 1 -- 172.21.15.12:0/261513082 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc0501051e0 con 0x7fc05010a810 2024-09-11T00:15:12.589 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:12.587+0000 7fc04cc00640 1 -- 172.21.15.12:0/261513082 <== mon.2 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7fc03c010430 con 0x7fc05010a810 2024-09-11T00:15:12.591 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:12.590+0000 7fc04cc00640 1 --2- 172.21.15.12:0/261513082 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fc0340b5410 0x7fc0340b78d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:15:12.591 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:12.590+0000 7fc04cc00640 1 -- 172.21.15.12:0/261513082 <== mon.2 v2:172.21.15.165:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9208+0+0 (secure 0 0 0) 0x7fc03c0e6360 con 0x7fc05010a810 2024-09-11T00:15:12.591 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:12.590+0000 7fc04fe00640 1 --2- 172.21.15.12:0/261513082 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fc0340b5410 0x7fc0340b78d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:15:12.592 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:12.591+0000 7fc04fe00640 1 --2- 172.21.15.12:0/261513082 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fc0340b5410 0x7fc0340b78d0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fc040011500 tx=0x7fc040005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:15:12.593 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:12.593+0000 7fc04cc00640 1 -- 172.21.15.12:0/261513082 <== mon.2 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7fc03c0acdf0 con 0x7fc05010a810 2024-09-11T00:15:12.657 INFO:tasks.workunit.client.0.smithi012.stdout: 1 312113 3.77 MB/sec execute 309 sec latency 142.880 ms 2024-09-11T00:15:12.907 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:12.906+0000 7fc055a00640 1 -- 172.21.15.12:0/261513082 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc0501ac1c0 con 0x7fc05010a810 2024-09-11T00:15:12.908 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:12.907+0000 7fc04cc00640 1 -- 172.21.15.12:0/261513082 <== mon.2 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 41 v41) ==== 94+0+11965 (secure 0 0 0) 0x7fc03c0ac6f0 con 0x7fc05010a810 2024-09-11T00:15:12.908 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-11T00:15:12.909 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 41 2024-09-11T00:15:12.909 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":41,"btime":"2024-09-11T00:14:31:349726+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14577,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6835/447171591","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":447171591},{"type":"v1","addr":"172.21.15.12:6835","nonce":447171591}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14589,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/901486040","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":901486040},{"type":"v1","addr":"172.21.15.12:6837","nonce":901486040}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":41,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T00:04:04.739881+0000","modified":"2024-09-11T00:14:30.594242+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24674,"mds_1":24643,"mds_2":24638,"mds_3":24637,"mds_4":24614},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14631":{"gid":14631,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.12:6841/3069919437","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6840","nonce":3069919437},{"type":"v1","addr":"172.21.15.12:6841","nonce":3069919437}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24562":{"gid":24562,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6833/1141636112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":1141636112},{"type":"v1","addr":"172.21.15.178:6833","nonce":1141636112}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24581":{"gid":24581,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6833/3111980970","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6832","nonce":3111980970},{"type":"v1","addr":"172.21.15.165:6833","nonce":3111980970}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24607":{"gid":24607,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6835/4084868287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":4084868287},{"type":"v1","addr":"172.21.15.178:6835","nonce":4084868287}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24614":{"gid":24614,"name":"e","rank":4,"incarnation":26,"state":"up:active","state_seq":21,"addr":"172.21.15.165:6835/2425395274","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":2425395274},{"type":"v1","addr":"172.21.15.165:6835","nonce":2425395274}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24637":{"gid":24637,"name":"i","rank":3,"incarnation":24,"state":"up:active","state_seq":18,"addr":"172.21.15.178:6839/70628097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6838","nonce":70628097},{"type":"v1","addr":"172.21.15.178:6839","nonce":70628097}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24638":{"gid":24638,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":18,"addr":"172.21.15.165:6837/6215088","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6836","nonce":6215088},{"type":"v1","addr":"172.21.15.165:6837","nonce":6215088}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24643":{"gid":24643,"name":"g","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.12:6839/3124093009","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6838","nonce":3124093009},{"type":"v1","addr":"172.21.15.12:6839","nonce":3124093009}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24674":{"gid":24674,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.165:6839/501312984","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6838","nonce":501312984},{"type":"v1","addr":"172.21.15.165:6839","nonce":501312984}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25672":{"gid":25672,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6837/308601741","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":308601741},{"type":"v1","addr":"172.21.15.178:6837","nonce":308601741}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24674,"qdb_cluster":[24614,24637,24638,24643,24674]},"id":1}]} 2024-09-11T00:15:12.911 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:12.910+0000 7fc055a00640 1 -- 172.21.15.12:0/261513082 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fc0340b5410 msgr2=0x7fc0340b78d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:15:12.911 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:12.910+0000 7fc055a00640 1 --2- 172.21.15.12:0/261513082 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fc0340b5410 0x7fc0340b78d0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fc040011500 tx=0x7fc040005f90 comp rx=0 tx=0).stop 2024-09-11T00:15:12.911 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:12.910+0000 7fc055a00640 1 -- 172.21.15.12:0/261513082 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc05010a810 msgr2=0x7fc050077bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:15:12.911 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:12.910+0000 7fc055a00640 1 --2- 172.21.15.12:0/261513082 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc05010a810 0x7fc050077bf0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fc03c00e9f0 tx=0x7fc03c00eeb0 comp rx=0 tx=0).stop 2024-09-11T00:15:12.912 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:12.910+0000 7fc055a00640 1 -- 172.21.15.12:0/261513082 shutdown_connections 2024-09-11T00:15:12.912 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:12.910+0000 7fc055a00640 1 --2- 172.21.15.12:0/261513082 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fc0340b5410 0x7fc0340b78d0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:15:12.912 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:12.910+0000 7fc055a00640 1 --2- 172.21.15.12:0/261513082 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc05010b190 0x7fc050078130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:15:12.912 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:12.910+0000 7fc055a00640 1 --2- 172.21.15.12:0/261513082 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc05010a810 0x7fc050077bf0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:15:12.912 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:12.910+0000 7fc055a00640 1 --2- 172.21.15.12:0/261513082 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc050104e00 0x7fc0500776b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:15:12.912 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:12.910+0000 7fc055a00640 1 -- 172.21.15.12:0/261513082 >> 172.21.15.12:0/261513082 conn(0x7fc050100870 msgr2=0x7fc050103d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:15:12.912 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:12.910+0000 7fc055a00640 1 -- 172.21.15.12:0/261513082 shutdown_connections 2024-09-11T00:15:12.912 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:12.910+0000 7fc055a00640 1 -- 172.21.15.12:0/261513082 wait complete. 2024-09-11T00:15:13.463 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:15:13 smithi012 ceph-mon[30333]: pgmap v643: 129 pgs: 129 active+clean; 78 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 20 MiB/s rd, 2.0 MiB/s wr, 329 op/s 2024-09-11T00:15:13.463 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:15:13 smithi012 ceph-mon[30333]: from='client.? 172.21.15.12:0/261513082' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T00:15:13.662 INFO:tasks.workunit.client.0.smithi012.stdout: 1 312359 3.76 MB/sec execute 310 sec latency 417.071 ms 2024-09-11T00:15:13.782 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:15:13 smithi165 ceph-mon[35356]: pgmap v643: 129 pgs: 129 active+clean; 78 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 20 MiB/s rd, 2.0 MiB/s wr, 329 op/s 2024-09-11T00:15:13.782 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:15:13 smithi165 ceph-mon[35356]: from='client.? 172.21.15.12:0/261513082' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T00:15:13.788 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24674 quiesce db / --timeout 90 --expiration 148.4 --await-for 45.0' 2024-09-11T00:15:13.788 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid ff3ad74a-6fcf-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24674 quiesce db / --timeout 90 --expiration 148.4 --await-for 45.0 2024-09-11T00:15:13.830 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:15:13 smithi178 ceph-mon[35919]: pgmap v643: 129 pgs: 129 active+clean; 78 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 20 MiB/s rd, 2.0 MiB/s wr, 329 op/s 2024-09-11T00:15:13.831 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:15:13 smithi178 ceph-mon[35919]: from='client.? 172.21.15.12:0/261513082' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T00:15:14.191 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:15:14.662 INFO:tasks.workunit.client.0.smithi012.stdout: 1 312613 3.75 MB/sec execute 311 sec latency 639.670 ms 2024-09-11T00:15:14.974 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:14.770+0000 7f7bc4400640 1 -- 172.21.15.12:0/4224250937 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7bbc104fa0 msgr2=0x7f7bbc105380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:15:14.974 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:14.770+0000 7f7bc4400640 1 --2- 172.21.15.12:0/4224250937 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7bbc104fa0 0x7f7bbc105380 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f7bb0009ca0 tx=0x7f7bb002a8c0 comp rx=0 tx=0).stop 2024-09-11T00:15:14.974 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:14.770+0000 7f7bc4400640 1 -- 172.21.15.12:0/4224250937 shutdown_connections 2024-09-11T00:15:14.974 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:14.770+0000 7f7bc4400640 1 --2- 172.21.15.12:0/4224250937 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7bbc10b330 0x7f7bbc111b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:15:14.974 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:14.770+0000 7f7bc4400640 1 --2- 172.21.15.12:0/4224250937 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7bbc10a9b0 0x7f7bbc10adf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:15:14.974 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:14.770+0000 7f7bc4400640 1 --2- 172.21.15.12:0/4224250937 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7bbc104fa0 0x7f7bbc105380 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:15:14.974 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:14.770+0000 7f7bc4400640 1 -- 172.21.15.12:0/4224250937 >> 172.21.15.12:0/4224250937 conn(0x7f7bbc100a50 msgr2=0x7f7bbc102e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:15:14.975 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:14.770+0000 7f7bc4400640 1 -- 172.21.15.12:0/4224250937 shutdown_connections 2024-09-11T00:15:14.975 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:14.770+0000 7f7bc4400640 1 -- 172.21.15.12:0/4224250937 wait complete. 2024-09-11T00:15:14.975 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:14.771+0000 7f7bc4400640 1 Processor -- start 2024-09-11T00:15:14.975 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:14.771+0000 7f7bc4400640 1 -- start start 2024-09-11T00:15:14.975 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:14.772+0000 7f7bc4400640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7bbc104fa0 0x7f7bbc19b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:15:14.975 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:14.772+0000 7f7bc4400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7bbc10a9b0 0x7f7bbc19ba00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:15:14.975 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:14.772+0000 7f7bc4400640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7bbc10b330 0x7f7bbc1a0da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:15:14.975 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:14.772+0000 7f7bc4400640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f7bbc114fb0 con 0x7f7bbc10b330 2024-09-11T00:15:14.975 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:14.772+0000 7f7bc4400640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f7bbc114e30 con 0x7f7bbc10a9b0 2024-09-11T00:15:14.975 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:14.772+0000 7f7bc4400640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f7bbc115130 con 0x7f7bbc104fa0 2024-09-11T00:15:14.975 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:14.772+0000 7f7bc1000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7bbc10a9b0 0x7f7bbc19ba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:15:14.976 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:14.772+0000 7f7bc1000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7bbc10a9b0 0x7f7bbc19ba00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.12:36500/0 (socket says 172.21.15.12:36500) 2024-09-11T00:15:14.976 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:14.772+0000 7f7bc2400640 1 --2- 172.21.15.12:0/680712568 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7bbc10b330 0x7f7bbc1a0da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:15:14.976 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:14.772+0000 7f7bc1000640 1 -- 172.21.15.12:0/680712568 learned_addr learned my addr 172.21.15.12:0/680712568 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:15:14.976 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:14.772+0000 7f7bc1a00640 1 --2- 172.21.15.12:0/680712568 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7bbc104fa0 0x7f7bbc19b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:15:14.976 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:14.773+0000 7f7bc1000640 1 -- 172.21.15.12:0/680712568 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7bbc104fa0 msgr2=0x7f7bbc19b4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:15:14.976 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:14.773+0000 7f7bc1000640 1 --2- 172.21.15.12:0/680712568 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7bbc104fa0 0x7f7bbc19b4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:15:14.976 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:14.773+0000 7f7bc1000640 1 -- 172.21.15.12:0/680712568 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7bbc10b330 msgr2=0x7f7bbc1a0da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:15:14.976 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:14.773+0000 7f7bc1000640 1 --2- 172.21.15.12:0/680712568 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7bbc10b330 0x7f7bbc1a0da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:15:14.976 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:14.773+0000 7f7bc1000640 1 -- 172.21.15.12:0/680712568 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7bbc1a1570 con 0x7f7bbc10a9b0 2024-09-11T00:15:14.976 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:14.773+0000 7f7bc1000640 1 --2- 172.21.15.12:0/680712568 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7bbc10a9b0 0x7f7bbc19ba00 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f7ba800ea60 tx=0x7f7ba800ef20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:15:14.977 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:14.773+0000 7f7bba000640 1 -- 172.21.15.12:0/680712568 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ba800cde0 con 0x7f7bbc10a9b0 2024-09-11T00:15:14.977 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:14.773+0000 7f7bba000640 1 -- 172.21.15.12:0/680712568 <== mon.2 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7ba80035f0 con 0x7f7bbc10a9b0 2024-09-11T00:15:14.977 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:14.773+0000 7f7bba000640 1 -- 172.21.15.12:0/680712568 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ba8010690 con 0x7f7bbc10a9b0 2024-09-11T00:15:14.977 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:14.773+0000 7f7bc4400640 1 -- 172.21.15.12:0/680712568 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7bbc1a1860 con 0x7f7bbc10a9b0 2024-09-11T00:15:14.977 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:14.774+0000 7f7bc4400640 1 -- 172.21.15.12:0/680712568 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7bbc1a7f20 con 0x7f7bbc10a9b0 2024-09-11T00:15:14.977 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:14.777+0000 7f7bc7056740 1 Processor -- start 2024-09-11T00:15:14.977 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:14.778+0000 7f7bc7056740 1 -- start start 2024-09-11T00:15:14.977 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:14.778+0000 7f7bc7056740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ac0906b180 0x55ac0906b560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:15:14.977 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:14.778+0000 7f7bc7056740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ac0906baa0 0x55ac090774f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:15:14.977 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:14.778+0000 7f7bc7056740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ac09077a30 0x55ac09079e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:15:14.978 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:14.778+0000 7f7bc7056740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55ac08fe0ca0 con 0x55ac0906baa0 2024-09-11T00:15:14.978 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:14.778+0000 7f7bc7056740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55ac08fd00d0 con 0x55ac09077a30 2024-09-11T00:15:14.978 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:14.778+0000 7f7bc7056740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55ac08fd3ee0 con 0x55ac0906b180 2024-09-11T00:15:14.978 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:14.778+0000 7f7bba000640 1 -- 172.21.15.12:0/680712568 <== mon.2 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7f7ba8004000 con 0x7f7bbc10a9b0 2024-09-11T00:15:14.978 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:14.778+0000 7f7bc1a00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ac0906baa0 0x55ac090774f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:15:14.978 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:14.778+0000 7f7bc1a00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ac0906baa0 0x55ac090774f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:55760/0 (socket says 172.21.15.12:55760) 2024-09-11T00:15:14.978 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:14.778+0000 7f7bc1a00640 1 -- 172.21.15.12:0/2907715335 learned_addr learned my addr 172.21.15.12:0/2907715335 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:15:14.978 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:14.779+0000 7f7bc2400640 1 --2- 172.21.15.12:0/2907715335 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ac0906b180 0x55ac0906b560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:15:14.978 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:14.779+0000 7f7bc1000640 1 --2- 172.21.15.12:0/2907715335 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ac09077a30 0x55ac09079e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:15:14.979 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:14.779+0000 7f7bc1a00640 1 -- 172.21.15.12:0/2907715335 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ac0906b180 msgr2=0x55ac0906b560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:15:14.979 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:14.779+0000 7f7bc1a00640 1 --2- 172.21.15.12:0/2907715335 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ac0906b180 0x55ac0906b560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:15:14.979 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:14.779+0000 7f7bc1a00640 1 -- 172.21.15.12:0/2907715335 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ac09077a30 msgr2=0x55ac09079e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:15:14.979 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:14.779+0000 7f7bc1a00640 1 --2- 172.21.15.12:0/2907715335 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ac09077a30 0x55ac09079e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:15:14.979 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:14.779+0000 7f7bc1a00640 1 -- 172.21.15.12:0/2907715335 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ac08f82400 con 0x55ac0906baa0 2024-09-11T00:15:14.979 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:14.779+0000 7f7bc1000640 1 --2- 172.21.15.12:0/2907715335 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ac09077a30 0x55ac09079e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T00:15:14.979 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:14.779+0000 7f7bc1a00640 1 --2- 172.21.15.12:0/2907715335 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ac0906baa0 0x55ac090774f0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f7bb002ade0 tx=0x7f7bb0005fa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72f073af44b5fd60 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:15:14.979 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:14.780+0000 7f7baea00640 1 -- 172.21.15.12:0/2907715335 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bb0031cf0 con 0x55ac0906baa0 2024-09-11T00:15:14.979 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:14.780+0000 7f7baea00640 1 -- 172.21.15.12:0/2907715335 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7bb0036030 con 0x55ac0906baa0 2024-09-11T00:15:14.979 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:14.780+0000 7f7baea00640 1 -- 172.21.15.12:0/2907715335 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bb00042a0 con 0x55ac0906baa0 2024-09-11T00:15:14.980 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:14.780+0000 7f7bba000640 1 --2- 172.21.15.12:0/680712568 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f7ba40b5360 0x7f7ba40b7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:15:14.980 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:14.780+0000 7f7bc7056740 1 -- 172.21.15.12:0/2907715335 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ac0906baa0 msgr2=0x55ac090774f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:15:14.980 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:14.780+0000 7f7bc7056740 1 --2- 172.21.15.12:0/2907715335 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ac0906baa0 0x55ac090774f0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f7bb002ade0 tx=0x7f7bb0005fa0 comp rx=0 tx=0).stop 2024-09-11T00:15:14.980 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:14.781+0000 7f7bba000640 1 -- 172.21.15.12:0/680712568 <== mon.2 v2:172.21.15.165:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9208+0+0 (secure 0 0 0) 0x7f7ba80e6190 con 0x7f7bbc10a9b0 2024-09-11T00:15:14.980 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:14.781+0000 7f7bc7056740 1 -- 172.21.15.12:0/2907715335 shutdown_connections 2024-09-11T00:15:14.980 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:14.781+0000 7f7bc7056740 1 --2- 172.21.15.12:0/2907715335 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ac09077a30 0x55ac09079e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:15:14.980 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:14.781+0000 7f7bc7056740 1 --2- 172.21.15.12:0/2907715335 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ac0906baa0 0x55ac090774f0 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:15:14.980 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:14.781+0000 7f7bc7056740 1 --2- 172.21.15.12:0/2907715335 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ac0906b180 0x55ac0906b560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:15:14.980 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:14.781+0000 7f7bc7056740 1 -- 172.21.15.12:0/2907715335 >> 172.21.15.12:0/2907715335 conn(0x55ac09068fb0 msgr2=0x55ac0906a150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:15:14.980 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:14.781+0000 7f7bc2400640 1 --2- 172.21.15.12:0/680712568 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f7ba40b5360 0x7f7ba40b7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:15:14.981 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:14.781+0000 7f7bc7056740 1 -- 172.21.15.12:0/2907715335 shutdown_connections 2024-09-11T00:15:14.981 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:14.781+0000 7f7bc7056740 1 -- 172.21.15.12:0/2907715335 wait complete. 2024-09-11T00:15:14.981 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:14.781+0000 7f7bc2400640 1 --2- 172.21.15.12:0/680712568 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f7ba40b5360 0x7f7ba40b7820 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f7bb4009c80 tx=0x7f7bb4005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:15:14.981 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:14.782+0000 7f7bc7056740 1 -- start start 2024-09-11T00:15:14.981 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:14.782+0000 7f7bc7056740 1 Processor -- start 2024-09-11T00:15:14.981 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:14.782+0000 7f7bc7056740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ac0906b180 0x55ac090772c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:15:14.981 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:14.782+0000 7f7bc7056740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ac0906baa0 0x55ac09074710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:15:14.981 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:14.782+0000 7f7bc7056740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ac09077a30 0x55ac09074cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:15:14.981 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:14.782+0000 7f7bc7056740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55ac08e317a0 con 0x55ac0906baa0 2024-09-11T00:15:14.981 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:14.782+0000 7f7bc7056740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55ac08f75a10 con 0x55ac0906b180 2024-09-11T00:15:14.982 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:14.782+0000 7f7bc7056740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55ac0907a4d0 con 0x55ac09077a30 2024-09-11T00:15:14.982 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:14.782+0000 7f7bc7056740 10 client.0 _ll_register_callbacks cb 0x55ac08f85be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:15:14.982 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:14.783+0000 7f7bc1a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ac0906b180 0x55ac090772c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:15:14.982 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:14.783+0000 7f7bc1a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ac0906b180 0x55ac090772c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.12:36518/0 (socket says 172.21.15.12:36518) 2024-09-11T00:15:14.982 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:14.783+0000 7f7bc1a00640 1 -- 172.21.15.12:0/4290802797 learned_addr learned my addr 172.21.15.12:0/4290802797 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:15:14.982 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:14.783+0000 7f7bc2400640 1 --2- 172.21.15.12:0/4290802797 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ac09077a30 0x55ac09074cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:15:14.982 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:14.783+0000 7f7bc1000640 1 --2- 172.21.15.12:0/4290802797 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ac0906baa0 0x55ac09074710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:15:14.982 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:14.783+0000 7f7bc1a00640 1 -- 172.21.15.12:0/4290802797 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ac09077a30 msgr2=0x55ac09074cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:15:14.982 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:14.783+0000 7f7bc1a00640 1 --2- 172.21.15.12:0/4290802797 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ac09077a30 0x55ac09074cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:15:14.982 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:14.783+0000 7f7bc1a00640 1 -- 172.21.15.12:0/4290802797 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ac0906baa0 msgr2=0x55ac09074710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:15:14.983 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:14.783+0000 7f7bc1a00640 1 --2- 172.21.15.12:0/4290802797 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ac0906baa0 0x55ac09074710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:15:14.983 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:14.783+0000 7f7bc1a00640 1 -- 172.21.15.12:0/4290802797 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ac08d3db90 con 0x55ac0906b180 2024-09-11T00:15:14.983 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:14.783+0000 7f7bc1a00640 1 --2- 172.21.15.12:0/4290802797 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ac0906b180 0x55ac090772c0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f7bb0009890 tx=0x7f7bb0003c40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d5b6b812c7065a33 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:15:14.983 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:14.783+0000 7f7ba3400640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-09-11T00:15:14.983 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:14.784+0000 7f7ba3400640 1 -- 172.21.15.12:0/4290802797 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bb004a040 con 0x55ac0906b180 2024-09-11T00:15:14.983 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:14.784+0000 7f7bc7056740 1 -- 172.21.15.12:0/4290802797 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7bb0036030 con 0x55ac0906b180 2024-09-11T00:15:14.983 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:14.784+0000 7f7ba3400640 1 -- 172.21.15.12:0/4290802797 <== mon.2 v2:172.21.15.165:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9208+0+0 (secure 0 0 0) 0x7f7bb0031650 con 0x55ac0906b180 2024-09-11T00:15:14.983 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:14.784+0000 7f7ba3400640 1 -- 172.21.15.12:0/4290802797 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f7bb00098e0 con 0x55ac0906b180 2024-09-11T00:15:14.983 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:14.784+0000 7f7bc7056740 10 client.26438 fetch_fsmap learned FSMap version 41 2024-09-11T00:15:14.983 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:14.784+0000 7f7bc7056740 1 -- 172.21.15.12:0/4290802797 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x55ac0908e610 con 0x55ac0906b180 2024-09-11T00:15:14.984 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:14.784+0000 7f7ba3400640 1 -- 172.21.15.12:0/4290802797 <== mon.2 v2:172.21.15.165:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7bb0036030 con 0x55ac0906b180 2024-09-11T00:15:14.984 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:14.784+0000 7f7ba3400640 1 -- 172.21.15.12:0/4290802797 <== mon.2 v2:172.21.15.165:3300/0 5 ==== fsmap(e 41) ==== 8370+0+0 (secure 0 0 0) 0x7f7bb004d030 con 0x55ac0906b180 2024-09-11T00:15:14.984 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:14.785+0000 7f7bc7056740 10 client.26438 fetch_fsmap finished waiting for FSMap version 41 2024-09-11T00:15:14.984 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:14.785+0000 7f7bc7056740 10 client.26438 resolve_mds: validated gid 24674 aka daemon mds.b 2024-09-11T00:15:14.984 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:14.785+0000 7f7bc7056740 20 client.26438 populate_metadata read hostname 'smithi012' 2024-09-11T00:15:14.984 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:14.785+0000 7f7bc7056740 1 --2- 172.21.15.12:0/4290802797 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55ac0908f960 0x55ac09091d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:15:14.984 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:14.785+0000 7f7bc7056740 4 client.26438 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T00:15:14.984 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:14.785+0000 7f7bc7056740 1 -- 172.21.15.12:0/4290802797 --> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ac08fe2bf0 con 0x55ac0908f960 2024-09-11T00:15:14.984 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:14.785+0000 7f7bc1000640 1 --2- 172.21.15.12:0/4290802797 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55ac0908f960 0x55ac09091d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:15:14.984 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:14.785+0000 7f7bc1000640 1 --2- 172.21.15.12:0/4290802797 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55ac0908f960 0x55ac09091d50 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=11466a0a869e59e8 server_cookie=4aea09596a70d3d0 in_seq=0 out_seq=0 2024-09-11T00:15:14.985 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:14.785+0000 7f7ba3400640 10 client.26438 ms_handle_connect on v2:172.21.15.165:6838/501312984 2024-09-11T00:15:14.985 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:14.787+0000 7f7ba3400640 1 -- 172.21.15.12:0/4290802797 <== mds.0 v2:172.21.15.165:6838/501312984 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55ac08fe2bf0 con 0x55ac0908f960 2024-09-11T00:15:14.985 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:14.787+0000 7f7ba3400640 10 client.26438 handle_command_reply: tid=0 2024-09-11T00:15:14.985 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:14.787+0000 7f7bc7056740 1 client.26438 shutdown 2024-09-11T00:15:14.985 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:14.788+0000 7f7bc7056740 1 -- 172.21.15.12:0/4290802797 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ac0906b180 msgr2=0x55ac090772c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:15:14.985 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:14.788+0000 7f7bc7056740 1 --2- 172.21.15.12:0/4290802797 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ac0906b180 0x55ac090772c0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f7bb0009890 tx=0x7f7bb0003c40 comp rx=0 tx=0).stop 2024-09-11T00:15:14.985 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:14.788+0000 7f7bc7056740 1 -- 172.21.15.12:0/4290802797 shutdown_connections 2024-09-11T00:15:14.985 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:14.788+0000 7f7bc7056740 1 --2- 172.21.15.12:0/4290802797 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55ac0908f960 0x55ac09091d50 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:15:14.985 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:14.788+0000 7f7bc7056740 1 --2- 172.21.15.12:0/4290802797 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ac09077a30 0x55ac09074cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:15:14.985 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:14.788+0000 7f7bc7056740 1 --2- 172.21.15.12:0/4290802797 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ac0906baa0 0x55ac09074710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:15:14.985 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:14.788+0000 7f7bc7056740 1 --2- 172.21.15.12:0/4290802797 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ac0906b180 0x55ac090772c0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:15:14.986 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:14.788+0000 7f7ba3400640 0 client.26438 ms_handle_reset on v2:172.21.15.165:6838/501312984 2024-09-11T00:15:14.986 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:14.788+0000 7f7bc7056740 1 -- 172.21.15.12:0/4290802797 >> 172.21.15.12:0/4290802797 conn(0x55ac09068fb0 msgr2=0x55ac0906eb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:15:14.986 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:14.788+0000 7f7bc7056740 1 -- 172.21.15.12:0/4290802797 shutdown_connections 2024-09-11T00:15:14.986 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:14.788+0000 7f7bc7056740 1 -- 172.21.15.12:0/4290802797 wait complete. 2024-09-11T00:15:14.986 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:14.788+0000 7f7bc7056740 20 client.26438 trim_cache size 0 max 16384 2024-09-11T00:15:14.986 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:14.807+0000 7f7bc7056740 1 Processor -- start 2024-09-11T00:15:14.986 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:14.807+0000 7f7bc7056740 1 -- start start 2024-09-11T00:15:14.986 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:14.808+0000 7f7bc7056740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ac0906b180 0x55ac09073c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:15:14.986 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:14.808+0000 7f7bc7056740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ac0906baa0 0x55ac09074160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:15:14.986 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:14.808+0000 7f7bc7056740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ac09077a30 0x55ac09078e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:15:14.986 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:14.808+0000 7f7bc7056740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f7bbc1ad9f0 con 0x55ac09077a30 2024-09-11T00:15:14.987 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:14.808+0000 7f7bc7056740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f7bbc1af530 con 0x55ac0906b180 2024-09-11T00:15:14.987 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:14.808+0000 7f7bc7056740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f7bbc1ae0c0 con 0x55ac0906baa0 2024-09-11T00:15:14.987 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:14.808+0000 7f7bc1000640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ac0906baa0 0x55ac09074160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:15:14.987 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:14.808+0000 7f7bc1a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ac0906b180 0x55ac09073c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:15:14.987 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:14.808+0000 7f7bc1000640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ac0906baa0 0x55ac09074160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.12:57214/0 (socket says 172.21.15.12:57214) 2024-09-11T00:15:14.987 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:14.808+0000 7f7bc2400640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ac09077a30 0x55ac09078e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:15:14.987 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:14.808+0000 7f7bc1000640 1 -- 172.21.15.12:0/1365195694 learned_addr learned my addr 172.21.15.12:0/1365195694 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:15:14.987 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:14.808+0000 7f7bc1000640 1 -- 172.21.15.12:0/1365195694 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ac0906b180 msgr2=0x55ac09073c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:15:14.987 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:14.808+0000 7f7bc1000640 1 --2- 172.21.15.12:0/1365195694 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ac0906b180 0x55ac09073c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:15:14.987 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:14.809+0000 7f7bc1000640 1 -- 172.21.15.12:0/1365195694 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ac09077a30 msgr2=0x55ac09078e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:15:14.987 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:14.809+0000 7f7bc1000640 1 --2- 172.21.15.12:0/1365195694 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ac09077a30 0x55ac09078e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:15:14.988 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:14.809+0000 7f7bc1000640 1 -- 172.21.15.12:0/1365195694 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ac09069a30 con 0x55ac0906baa0 2024-09-11T00:15:14.988 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:14.809+0000 7f7bc1a00640 1 --2- 172.21.15.12:0/1365195694 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ac0906b180 0x55ac09073c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:15:14.988 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:14.809+0000 7f7bc1000640 1 --2- 172.21.15.12:0/1365195694 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ac0906baa0 0x55ac09074160 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f7ba8000c00 tx=0x7f7ba80cf040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a95a8a99491b1621 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:15:14.988 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:14.809+0000 7f7bae000640 1 -- 172.21.15.12:0/1365195694 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ba80ac090 con 0x55ac0906baa0 2024-09-11T00:15:14.988 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:14.809+0000 7f7bae000640 1 -- 172.21.15.12:0/1365195694 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7ba80acc80 con 0x55ac0906baa0 2024-09-11T00:15:14.988 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:14.809+0000 7f7bae000640 1 -- 172.21.15.12:0/1365195694 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ba8003a60 con 0x55ac0906baa0 2024-09-11T00:15:14.988 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:14.810+0000 7f7bc7056740 1 -- 172.21.15.12:0/1365195694 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ac0906baa0 msgr2=0x55ac09074160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:15:14.988 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:14.810+0000 7f7bc7056740 1 --2- 172.21.15.12:0/1365195694 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ac0906baa0 0x55ac09074160 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f7ba8000c00 tx=0x7f7ba80cf040 comp rx=0 tx=0).stop 2024-09-11T00:15:14.988 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:14.810+0000 7f7bc7056740 1 -- 172.21.15.12:0/1365195694 shutdown_connections 2024-09-11T00:15:14.988 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:14.810+0000 7f7bc7056740 1 --2- 172.21.15.12:0/1365195694 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ac09077a30 0x55ac09078e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:15:14.988 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:14.810+0000 7f7bc7056740 1 --2- 172.21.15.12:0/1365195694 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ac0906baa0 0x55ac09074160 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:15:14.989 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:14.810+0000 7f7bc7056740 1 --2- 172.21.15.12:0/1365195694 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ac0906b180 0x55ac09073c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:15:14.989 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:14.810+0000 7f7bc7056740 1 -- 172.21.15.12:0/1365195694 >> 172.21.15.12:0/1365195694 conn(0x55ac09068fb0 msgr2=0x55ac090680b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:15:14.989 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:14.810+0000 7f7bc7056740 1 -- 172.21.15.12:0/1365195694 shutdown_connections 2024-09-11T00:15:14.989 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:14.810+0000 7f7bc7056740 1 -- 172.21.15.12:0/1365195694 wait complete. 2024-09-11T00:15:14.989 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:14.810+0000 7f7bc7056740 1 -- start start 2024-09-11T00:15:14.989 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:14.811+0000 7f7bc7056740 1 Processor -- start 2024-09-11T00:15:14.989 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:14.811+0000 7f7bc7056740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ac0906b180 0x55ac09088840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:15:14.989 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:14.811+0000 7f7bc7056740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ac0906baa0 0x55ac09083010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:15:14.989 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:14.811+0000 7f7bc7056740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ac09077a30 0x55ac09083590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:15:14.989 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:14.811+0000 7f7bc7056740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f7bbc1aee60 con 0x55ac09077a30 2024-09-11T00:15:14.990 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:14.811+0000 7f7bc7056740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f7bbc1ae790 con 0x55ac0906baa0 2024-09-11T00:15:14.990 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:14.811+0000 7f7bc7056740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f7bbc1ab9d0 con 0x55ac0906b180 2024-09-11T00:15:14.990 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:14.811+0000 7f7bc7056740 10 client.0 _ll_register_callbacks cb 0x7f7bb4009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:15:14.990 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:14.811+0000 7f7bc1000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ac0906baa0 0x55ac09083010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:15:14.990 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:14.811+0000 7f7bc1a00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ac0906b180 0x55ac09088840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:15:14.990 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:14.811+0000 7f7bc2400640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ac09077a30 0x55ac09083590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:15:14.990 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:14.811+0000 7f7bc1000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ac0906baa0 0x55ac09083010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.12:36538/0 (socket says 172.21.15.12:36538) 2024-09-11T00:15:14.990 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:14.811+0000 7f7bc1000640 1 -- 172.21.15.12:0/1006174837 learned_addr learned my addr 172.21.15.12:0/1006174837 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:15:14.990 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:14.812+0000 7f7bc1a00640 1 -- 172.21.15.12:0/1006174837 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ac0906baa0 msgr2=0x55ac09083010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:15:14.991 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:14.812+0000 7f7bc1a00640 1 --2- 172.21.15.12:0/1006174837 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ac0906baa0 0x55ac09083010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:15:14.991 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:14.812+0000 7f7bc1a00640 1 -- 172.21.15.12:0/1006174837 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ac09077a30 msgr2=0x55ac09083590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:15:14.991 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:14.812+0000 7f7bc1a00640 1 --2- 172.21.15.12:0/1006174837 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ac09077a30 0x55ac09083590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:15:14.991 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:14.812+0000 7f7bc1a00640 1 -- 172.21.15.12:0/1006174837 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ac0906f490 con 0x55ac0906b180 2024-09-11T00:15:14.991 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:14.812+0000 7f7bc1000640 1 --2- 172.21.15.12:0/1006174837 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ac0906baa0 0x55ac09083010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:15:14.991 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:14.812+0000 7f7bc1a00640 1 --2- 172.21.15.12:0/1006174837 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ac0906b180 0x55ac09088840 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f7bb002ade0 tx=0x7f7bb0030e80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1a9361c299a16848 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:15:14.991 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:14.812+0000 7f7ba3400640 10 client.0 ms_handle_connect on v2:172.21.15.178:3300/0 2024-09-11T00:15:14.991 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:14.812+0000 7f7ba3400640 1 -- 172.21.15.12:0/1006174837 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bb0040020 con 0x55ac0906b180 2024-09-11T00:15:14.991 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:14.812+0000 7f7bc7056740 1 -- 172.21.15.12:0/1006174837 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7ba80acc80 con 0x55ac0906b180 2024-09-11T00:15:14.991 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:14.812+0000 7f7ba3400640 1 -- 172.21.15.12:0/1006174837 <== mon.1 v2:172.21.15.178:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9208+0+0 (secure 0 0 0) 0x7f7bb0031660 con 0x55ac0906b180 2024-09-11T00:15:14.992 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:14.813+0000 7f7ba3400640 1 -- 172.21.15.12:0/1006174837 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f7bb00098e0 con 0x55ac0906b180 2024-09-11T00:15:14.992 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:14.813+0000 7f7bc7056740 10 client.26437 fetch_fsmap learned FSMap version 41 2024-09-11T00:15:14.992 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:14.813+0000 7f7bc7056740 1 -- 172.21.15.12:0/1006174837 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x55ac090fd520 con 0x55ac0906b180 2024-09-11T00:15:14.992 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:14.813+0000 7f7ba3400640 1 -- 172.21.15.12:0/1006174837 <== mon.1 v2:172.21.15.178:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7ba80acc80 con 0x55ac0906b180 2024-09-11T00:15:14.992 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:14.813+0000 7f7ba3400640 1 -- 172.21.15.12:0/1006174837 <== mon.1 v2:172.21.15.178:3300/0 5 ==== fsmap(e 41) ==== 8370+0+0 (secure 0 0 0) 0x7f7bb004d040 con 0x55ac0906b180 2024-09-11T00:15:14.992 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:14.813+0000 7f7bc7056740 10 client.26437 fetch_fsmap finished waiting for FSMap version 41 2024-09-11T00:15:14.992 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:14.813+0000 7f7bc7056740 10 client.26437 resolve_mds: validated gid 24674 aka daemon mds.b 2024-09-11T00:15:14.992 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:14.813+0000 7f7bc7056740 20 client.26437 populate_metadata read hostname 'smithi012' 2024-09-11T00:15:14.992 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:14.813+0000 7f7bc7056740 1 --2- 172.21.15.12:0/1006174837 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55ac090fdb50 0x55ac090fff70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:15:14.992 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:14.813+0000 7f7bc7056740 4 client.26437 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 148.4, "await_for": 45.0}] 2024-09-11T00:15:14.993 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:14.813+0000 7f7bc7056740 1 -- 172.21.15.12:0/1006174837 --> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 148.4, "await_for": 45.0}) -- 0x55ac08fde6a0 con 0x55ac090fdb50 2024-09-11T00:15:14.993 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:14.814+0000 7f7bc1000640 1 --2- 172.21.15.12:0/1006174837 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55ac090fdb50 0x55ac090fff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:15:14.993 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:14.814+0000 7f7bc1000640 1 --2- 172.21.15.12:0/1006174837 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55ac090fdb50 0x55ac090fff70 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=261deb1bd3f5b2d8 server_cookie=b7aeef521d0aed4e in_seq=0 out_seq=0 2024-09-11T00:15:14.993 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:14.814+0000 7f7ba3400640 10 client.26437 ms_handle_connect on v2:172.21.15.165:6838/501312984 2024-09-11T00:15:14.993 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:14.870+0000 7f7ba3400640 1 -- 172.21.15.12:0/1006174837 <== mds.0 v2:172.21.15.165:6838/501312984 1 ==== command_reply(tid 0: 0 ) ==== 8+0+590 (crc 0 0 0) 0x55ac08fde6a0 con 0x55ac090fdb50 2024-09-11T00:15:14.993 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:14.870+0000 7f7ba3400640 10 client.26437 handle_command_reply: tid=0 2024-09-11T00:15:14.993 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:14.870+0000 7f7bc7056740 1 client.26437 shutdown 2024-09-11T00:15:14.993 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:14.870+0000 7f7bc7056740 1 -- 172.21.15.12:0/1006174837 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ac0906b180 msgr2=0x55ac09088840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:15:14.993 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:14.870+0000 7f7bc7056740 1 --2- 172.21.15.12:0/1006174837 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ac0906b180 0x55ac09088840 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f7bb002ade0 tx=0x7f7bb0030e80 comp rx=0 tx=0).stop 2024-09-11T00:15:14.993 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:14.870+0000 7f7bc7056740 1 -- 172.21.15.12:0/1006174837 shutdown_connections 2024-09-11T00:15:14.993 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:14.870+0000 7f7bc7056740 1 --2- 172.21.15.12:0/1006174837 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55ac090fdb50 0x55ac090fff70 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:15:14.994 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:14.870+0000 7f7bc7056740 1 --2- 172.21.15.12:0/1006174837 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ac09077a30 0x55ac09083590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:15:14.994 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:14.870+0000 7f7bc7056740 1 --2- 172.21.15.12:0/1006174837 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ac0906baa0 0x55ac09083010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:15:14.994 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:14.870+0000 7f7bc7056740 1 --2- 172.21.15.12:0/1006174837 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ac0906b180 0x55ac09088840 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:15:14.994 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:14.870+0000 7f7ba3400640 0 client.26437 ms_handle_reset on v2:172.21.15.165:6838/501312984 2024-09-11T00:15:14.994 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-09-11T00:15:14.994 INFO:teuthology.orchestra.run.smithi012.stdout: "epoch": 41, 2024-09-11T00:15:14.995 INFO:teuthology.orchestra.run.smithi012.stdout: "leader": 24674, 2024-09-11T00:15:14.995 INFO:teuthology.orchestra.run.smithi012.stdout: "set_version": 26, 2024-09-11T00:15:14.995 INFO:teuthology.orchestra.run.smithi012.stdout: "sets": { 2024-09-11T00:15:14.995 INFO:teuthology.orchestra.run.smithi012.stdout: "e3bc5e2": { 2024-09-11T00:15:14.995 INFO:teuthology.orchestra.run.smithi012.stdout: "version": 26, 2024-09-11T00:15:14.995 INFO:teuthology.orchestra.run.smithi012.stdout: "age_ref": 0.0, 2024-09-11T00:15:14.995 INFO:teuthology.orchestra.run.smithi012.stdout: "state": { 2024-09-11T00:15:14.995 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "QUIESCED", 2024-09-11T00:15:14.995 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 0.0 2024-09-11T00:15:14.995 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:15:14.996 INFO:teuthology.orchestra.run.smithi012.stdout: "timeout": 90.0, 2024-09-11T00:15:14.996 INFO:teuthology.orchestra.run.smithi012.stdout: "expiration": 148.4, 2024-09-11T00:15:14.996 INFO:teuthology.orchestra.run.smithi012.stdout: "members": { 2024-09-11T00:15:14.996 INFO:teuthology.orchestra.run.smithi012.stdout: "file:/": { 2024-09-11T00:15:14.996 INFO:teuthology.orchestra.run.smithi012.stdout: "excluded": false, 2024-09-11T00:15:14.996 INFO:teuthology.orchestra.run.smithi012.stdout: "state": { 2024-09-11T00:15:14.996 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "QUIESCED", 2024-09-11T00:15:14.996 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 0.0 2024-09-11T00:15:14.996 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:15:14.996 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:15:14.997 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:15:14.997 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:15:14.997 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:15:14.997 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-09-11T00:15:14.997 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:14.870+0000 7f7bc7056740 1 -- 172.21.15.12:0/1006174837 >> 172.21.15.12:0/1006174837 conn(0x55ac09068fb0 msgr2=0x55ac09075d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:15:14.997 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:14.870+0000 7f7bc7056740 1 -- 172.21.15.12:0/1006174837 shutdown_connections 2024-09-11T00:15:14.997 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:14.870+0000 7f7bc7056740 1 -- 172.21.15.12:0/1006174837 wait complete. 2024-09-11T00:15:14.998 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:14.870+0000 7f7bc7056740 20 client.26437 trim_cache size 0 max 16384 2024-09-11T00:15:14.998 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:14.871+0000 7f7bc4400640 1 -- 172.21.15.12:0/680712568 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f7ba40b5360 msgr2=0x7f7ba40b7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:15:14.998 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:14.871+0000 7f7bc4400640 1 --2- 172.21.15.12:0/680712568 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f7ba40b5360 0x7f7ba40b7820 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f7bb4009c80 tx=0x7f7bb4005f90 comp rx=0 tx=0).stop 2024-09-11T00:15:14.998 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:14.871+0000 7f7bc4400640 1 -- 172.21.15.12:0/680712568 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7bbc10a9b0 msgr2=0x7f7bbc19ba00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:15:14.998 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:14.871+0000 7f7bc4400640 1 --2- 172.21.15.12:0/680712568 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7bbc10a9b0 0x7f7bbc19ba00 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f7ba800ea60 tx=0x7f7ba800ef20 comp rx=0 tx=0).stop 2024-09-11T00:15:14.998 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:14.871+0000 7f7bc4400640 1 -- 172.21.15.12:0/680712568 shutdown_connections 2024-09-11T00:15:14.998 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:14.871+0000 7f7bc4400640 1 --2- 172.21.15.12:0/680712568 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f7ba40b5360 0x7f7ba40b7820 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:15:14.998 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:14.871+0000 7f7bc4400640 1 --2- 172.21.15.12:0/680712568 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7bbc10b330 0x7f7bbc1a0da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:15:14.998 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:14.871+0000 7f7bc4400640 1 --2- 172.21.15.12:0/680712568 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7bbc10a9b0 0x7f7bbc19ba00 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:15:14.999 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:14.871+0000 7f7bc4400640 1 --2- 172.21.15.12:0/680712568 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7bbc104fa0 0x7f7bbc19b4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:15:14.999 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:14.871+0000 7f7bc4400640 1 -- 172.21.15.12:0/680712568 >> 172.21.15.12:0/680712568 conn(0x7f7bbc100a50 msgr2=0x7f7bbc101330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:15:14.999 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:14.871+0000 7f7bc4400640 1 -- 172.21.15.12:0/680712568 shutdown_connections 2024-09-11T00:15:14.999 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:14.871+0000 7f7bc4400640 1 -- 172.21.15.12:0/680712568 wait complete. 2024-09-11T00:15:15.662 INFO:tasks.workunit.client.0.smithi012.stdout: 1 312804 3.74 MB/sec execute 312 sec latency 830.278 ms 2024-09-11T00:15:15.782 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:15:15 smithi165 ceph-mon[35356]: pgmap v644: 129 pgs: 129 active+clean; 78 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 1.9 MiB/s wr, 274 op/s 2024-09-11T00:15:15.813 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:15:15 smithi012 ceph-mon[30333]: pgmap v644: 129 pgs: 129 active+clean; 78 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 1.9 MiB/s wr, 274 op/s 2024-09-11T00:15:15.831 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:15:15 smithi178 ceph-mon[35919]: pgmap v644: 129 pgs: 129 active+clean; 78 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 1.9 MiB/s wr, 274 op/s 2024-09-11T00:15:15.968 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'e3bc5e2', quiesce took 4.8 seconds. Will release after: 23.599999999999998 2024-09-11T00:15:16.662 INFO:tasks.workunit.client.0.smithi012.stdout: 1 312804 3.73 MB/sec execute 313 sec latency 1830.401 ms 2024-09-11T00:15:17.665 INFO:tasks.workunit.client.0.smithi012.stdout: 1 312804 3.71 MB/sec execute 314 sec latency 2830.497 ms 2024-09-11T00:15:17.782 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:15:17 smithi165 ceph-mon[35356]: pgmap v645: 129 pgs: 129 active+clean; 80 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 2.4 MiB/s wr, 331 op/s 2024-09-11T00:15:17.813 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:15:17 smithi012 ceph-mon[30333]: pgmap v645: 129 pgs: 129 active+clean; 80 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 2.4 MiB/s wr, 331 op/s 2024-09-11T00:15:17.830 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:15:17 smithi178 ceph-mon[35919]: pgmap v645: 129 pgs: 129 active+clean; 80 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 2.4 MiB/s wr, 331 op/s 2024-09-11T00:15:18.616 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:15:18 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T00:15:18.616 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:15:18 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:15:18.616 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:15:18 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T00:15:18.616 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:15:18 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:15:18.662 INFO:tasks.workunit.client.0.smithi012.stdout: 1 312804 3.70 MB/sec execute 315 sec latency 3830.555 ms 2024-09-11T00:15:18.782 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:15:18 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T00:15:18.782 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:15:18 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:15:18.782 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:15:18 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T00:15:18.783 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:15:18 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:15:18.831 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:15:18 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T00:15:18.831 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:15:18 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:15:18.831 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:15:18 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T00:15:18.831 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:15:18 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:15:19.662 INFO:tasks.workunit.client.0.smithi012.stdout: 1 312804 3.69 MB/sec execute 316 sec latency 4830.621 ms 2024-09-11T00:15:19.782 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:15:19 smithi165 ceph-mon[35356]: pgmap v646: 129 pgs: 129 active+clean; 80 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 1.6 MiB/s wr, 284 op/s 2024-09-11T00:15:19.813 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:15:19 smithi012 ceph-mon[30333]: pgmap v646: 129 pgs: 129 active+clean; 80 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 1.6 MiB/s wr, 284 op/s 2024-09-11T00:15:19.830 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:15:19 smithi178 ceph-mon[35919]: pgmap v646: 129 pgs: 129 active+clean; 80 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 1.6 MiB/s wr, 284 op/s 2024-09-11T00:15:20.663 INFO:tasks.workunit.client.0.smithi012.stdout: 1 312804 3.68 MB/sec execute 317 sec latency 5830.688 ms 2024-09-11T00:15:21.273 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:21.271+0000 7f8b13400640 1 -- 172.21.15.12:0/2498433163 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8b0c1066a0 msgr2=0x7f8b0c10d020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:15:21.273 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:21.271+0000 7f8b13400640 1 --2- 172.21.15.12:0/2498433163 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8b0c1066a0 0x7f8b0c10d020 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f8b04009cd0 tx=0x7f8b0402a8e0 comp rx=0 tx=0).stop 2024-09-11T00:15:21.273 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:21.271+0000 7f8b13400640 1 -- 172.21.15.12:0/2498433163 shutdown_connections 2024-09-11T00:15:21.273 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:21.271+0000 7f8b13400640 1 --2- 172.21.15.12:0/2498433163 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8b0c1066a0 0x7f8b0c10d020 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:15:21.273 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:21.271+0000 7f8b13400640 1 --2- 172.21.15.12:0/2498433163 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8b0c105ce0 0x7f8b0c106160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:15:21.273 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:21.271+0000 7f8b13400640 1 --2- 172.21.15.12:0/2498433163 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8b0c104b90 0x7f8b0c104f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:15:21.273 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:21.271+0000 7f8b13400640 1 -- 172.21.15.12:0/2498433163 >> 172.21.15.12:0/2498433163 conn(0x7f8b0c100470 msgr2=0x7f8b0c102890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:15:21.273 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:21.271+0000 7f8b13400640 1 -- 172.21.15.12:0/2498433163 shutdown_connections 2024-09-11T00:15:21.273 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:21.271+0000 7f8b13400640 1 -- 172.21.15.12:0/2498433163 wait complete. 2024-09-11T00:15:21.276 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:21.272+0000 7f8b13400640 1 Processor -- start 2024-09-11T00:15:21.276 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:21.272+0000 7f8b13400640 1 -- start start 2024-09-11T00:15:21.276 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:21.273+0000 7f8b13400640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8b0c104b90 0x7f8b0c19af20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:15:21.276 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:21.273+0000 7f8b13400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8b0c105ce0 0x7f8b0c19b570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:15:21.276 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:21.273+0000 7f8b13400640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8b0c1066a0 0x7f8b0c19c1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:15:21.276 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:21.273+0000 7f8b13400640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f8b0c110830 con 0x7f8b0c1066a0 2024-09-11T00:15:21.276 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:21.273+0000 7f8b13400640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f8b0c1106b0 con 0x7f8b0c105ce0 2024-09-11T00:15:21.276 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:21.273+0000 7f8b13400640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f8b0c1109b0 con 0x7f8b0c104b90 2024-09-11T00:15:21.276 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:21.273+0000 7f8b11400640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8b0c1066a0 0x7f8b0c19c1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:15:21.276 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:21.273+0000 7f8b10a00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8b0c104b90 0x7f8b0c19af20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:15:21.277 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:21.273+0000 7f8b10a00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8b0c104b90 0x7f8b0c19af20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.12:49904/0 (socket says 172.21.15.12:49904) 2024-09-11T00:15:21.277 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:21.273+0000 7f8b11400640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8b0c1066a0 0x7f8b0c19c1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:56536/0 (socket says 172.21.15.12:56536) 2024-09-11T00:15:21.277 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:21.273+0000 7f8b03e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8b0c105ce0 0x7f8b0c19b570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:15:21.277 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:21.273+0000 7f8b10a00640 1 -- 172.21.15.12:0/2402925372 learned_addr learned my addr 172.21.15.12:0/2402925372 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:15:21.277 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:21.273+0000 7f8b10a00640 1 -- 172.21.15.12:0/2402925372 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8b0c105ce0 msgr2=0x7f8b0c19b570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:15:21.277 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:21.273+0000 7f8b10a00640 1 --2- 172.21.15.12:0/2402925372 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8b0c105ce0 0x7f8b0c19b570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:15:21.277 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:21.273+0000 7f8b10a00640 1 -- 172.21.15.12:0/2402925372 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8b0c1066a0 msgr2=0x7f8b0c19c1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:15:21.277 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:21.273+0000 7f8b10a00640 1 --2- 172.21.15.12:0/2402925372 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8b0c1066a0 0x7f8b0c19c1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:15:21.277 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:21.273+0000 7f8b10a00640 1 -- 172.21.15.12:0/2402925372 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b0c19ca60 con 0x7f8b0c104b90 2024-09-11T00:15:21.277 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:21.273+0000 7f8b11400640 1 --2- 172.21.15.12:0/2402925372 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8b0c1066a0 0x7f8b0c19c1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:15:21.277 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:21.274+0000 7f8b10a00640 1 --2- 172.21.15.12:0/2402925372 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8b0c104b90 0x7f8b0c19af20 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f8afc00e9f0 tx=0x7f8afc00eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:15:21.277 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:21.274+0000 7f8b0a000640 1 -- 172.21.15.12:0/2402925372 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8afc00cd80 con 0x7f8b0c104b90 2024-09-11T00:15:21.277 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:21.274+0000 7f8b0a000640 1 -- 172.21.15.12:0/2402925372 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8afc0108e0 con 0x7f8b0c104b90 2024-09-11T00:15:21.278 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:21.274+0000 7f8b13400640 1 -- 172.21.15.12:0/2402925372 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8b0c19cd50 con 0x7f8b0c104b90 2024-09-11T00:15:21.278 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:21.274+0000 7f8b0a000640 1 -- 172.21.15.12:0/2402925372 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8afc018980 con 0x7f8b0c104b90 2024-09-11T00:15:21.278 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:21.274+0000 7f8b13400640 1 -- 172.21.15.12:0/2402925372 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8b0c1a3420 con 0x7f8b0c104b90 2024-09-11T00:15:21.280 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:21.277+0000 7f8b163f0740 1 Processor -- start 2024-09-11T00:15:21.280 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:21.277+0000 7f8b163f0740 1 -- start start 2024-09-11T00:15:21.280 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:21.277+0000 7f8b163f0740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557faca9e790 0x557facaa0b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:15:21.280 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:21.277+0000 7f8b163f0740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557facaa10c0 0x557facaa34b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:15:21.280 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:21.277+0000 7f8b163f0740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557facaa3b00 0x557facaa5fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:15:21.280 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:21.277+0000 7f8b163f0740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x557faca116b0 con 0x557facaa3b00 2024-09-11T00:15:21.280 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:21.277+0000 7f8b163f0740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x557faca00ae0 con 0x557facaa10c0 2024-09-11T00:15:21.280 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:21.277+0000 7f8b163f0740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x557faca048f0 con 0x557faca9e790 2024-09-11T00:15:21.280 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:21.278+0000 7f8b03e00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557facaa3b00 0x557facaa5fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:15:21.280 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:21.278+0000 7f8b11400640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557faca9e790 0x557facaa0b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:15:21.280 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:21.278+0000 7f8b03e00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557facaa3b00 0x557facaa5fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:56548/0 (socket says 172.21.15.12:56548) 2024-09-11T00:15:21.281 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:21.278+0000 7f8b03e00640 1 -- 172.21.15.12:0/3269256635 learned_addr learned my addr 172.21.15.12:0/3269256635 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:15:21.281 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:21.278+0000 7f8b0a000640 1 -- 172.21.15.12:0/2402925372 <== mon.1 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7f8afc010430 con 0x7f8b0c104b90 2024-09-11T00:15:21.281 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:21.278+0000 7f8b03e00640 1 -- 172.21.15.12:0/3269256635 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557faca9e790 msgr2=0x557facaa0b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:15:21.281 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:21.278+0000 7f8b03e00640 1 --2- 172.21.15.12:0/3269256635 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557faca9e790 0x557facaa0b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:15:21.281 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:21.278+0000 7f8b03e00640 1 -- 172.21.15.12:0/3269256635 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557facaa10c0 msgr2=0x557facaa34b0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-11T00:15:21.281 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:21.278+0000 7f8b03e00640 1 --2- 172.21.15.12:0/3269256635 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557facaa10c0 0x557facaa34b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:15:21.281 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:21.278+0000 7f8b03e00640 1 -- 172.21.15.12:0/3269256635 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557fac9cd780 con 0x557facaa3b00 2024-09-11T00:15:21.281 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:21.278+0000 7f8b11400640 1 --2- 172.21.15.12:0/3269256635 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557faca9e790 0x557facaa0b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:15:21.281 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:21.278+0000 7f8b03e00640 1 --2- 172.21.15.12:0/3269256635 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557facaa3b00 0x557facaa5fc0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f8af400bd10 tx=0x7f8af40107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38a420105bda73ab server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:15:21.281 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:21.279+0000 7f8b01c00640 1 -- 172.21.15.12:0/3269256635 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8af4004030 con 0x557facaa3b00 2024-09-11T00:15:21.281 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:21.279+0000 7f8b01c00640 1 -- 172.21.15.12:0/3269256635 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8af40035f0 con 0x557facaa3b00 2024-09-11T00:15:21.281 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:21.279+0000 7f8b01c00640 1 -- 172.21.15.12:0/3269256635 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8af4011770 con 0x557facaa3b00 2024-09-11T00:15:21.285 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:21.279+0000 7f8b163f0740 1 -- 172.21.15.12:0/3269256635 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557facaa3b00 msgr2=0x557facaa5fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:15:21.285 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:21.279+0000 7f8b163f0740 1 --2- 172.21.15.12:0/3269256635 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557facaa3b00 0x557facaa5fc0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f8af400bd10 tx=0x7f8af40107a0 comp rx=0 tx=0).stop 2024-09-11T00:15:21.285 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:21.279+0000 7f8b163f0740 1 -- 172.21.15.12:0/3269256635 shutdown_connections 2024-09-11T00:15:21.285 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:21.279+0000 7f8b163f0740 1 --2- 172.21.15.12:0/3269256635 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557facaa3b00 0x557facaa5fc0 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:15:21.285 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:21.279+0000 7f8b163f0740 1 --2- 172.21.15.12:0/3269256635 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557facaa10c0 0x557facaa34b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:15:21.285 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:21.279+0000 7f8b163f0740 1 --2- 172.21.15.12:0/3269256635 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557faca9e790 0x557facaa0b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:15:21.285 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:21.279+0000 7f8b163f0740 1 -- 172.21.15.12:0/3269256635 >> 172.21.15.12:0/3269256635 conn(0x557faca998e0 msgr2=0x557faca9a670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:15:21.285 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:21.280+0000 7f8b163f0740 1 -- 172.21.15.12:0/3269256635 shutdown_connections 2024-09-11T00:15:21.286 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:21.280+0000 7f8b163f0740 1 -- 172.21.15.12:0/3269256635 wait complete. 2024-09-11T00:15:21.286 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:21.280+0000 7f8b0a000640 1 --2- 172.21.15.12:0/2402925372 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f8af00b51f0 0x7f8af00b76b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:15:21.286 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:21.280+0000 7f8b163f0740 1 -- start start 2024-09-11T00:15:21.286 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:21.280+0000 7f8b0a000640 1 -- 172.21.15.12:0/2402925372 <== mon.1 v2:172.21.15.178:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9208+0+0 (secure 0 0 0) 0x7f8afc0e62f0 con 0x7f8b0c104b90 2024-09-11T00:15:21.286 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:21.280+0000 7f8b11400640 1 --2- 172.21.15.12:0/2402925372 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f8af00b51f0 0x7f8af00b76b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:15:21.286 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:21.280+0000 7f8b163f0740 1 Processor -- start 2024-09-11T00:15:21.286 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:21.281+0000 7f8b163f0740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557faca9e790 0x557facaa7f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:15:21.286 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:21.281+0000 7f8b163f0740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557facaa10c0 0x557facaa8630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:15:21.286 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:21.281+0000 7f8b163f0740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557facaa3b00 0x557facaa9190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:15:21.286 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:21.281+0000 7f8b163f0740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x557fac938ab0 con 0x557facaa3b00 2024-09-11T00:15:21.286 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:21.281+0000 7f8b163f0740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x557fac9a7570 con 0x557facaa10c0 2024-09-11T00:15:21.286 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:21.281+0000 7f8b163f0740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x557facaa6ae0 con 0x557faca9e790 2024-09-11T00:15:21.286 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:21.281+0000 7f8b163f0740 10 client.0 _ll_register_callbacks cb 0x557fac9b6100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:15:21.287 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:21.281+0000 7f8b11400640 1 --2- 172.21.15.12:0/2402925372 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f8af00b51f0 0x7f8af00b76b0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f8b04006010 tx=0x7f8b04031850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:15:21.287 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:21.281+0000 7f8b11400640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557facaa3b00 0x557facaa9190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:15:21.287 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:21.281+0000 7f8b03e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557facaa10c0 0x557facaa8630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:15:21.287 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:21.281+0000 7f8b03e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557facaa10c0 0x557facaa8630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.12:52176/0 (socket says 172.21.15.12:52176) 2024-09-11T00:15:21.287 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:21.281+0000 7f8b03e00640 1 -- 172.21.15.12:0/1633395768 learned_addr learned my addr 172.21.15.12:0/1633395768 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:15:21.287 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:21.281+0000 7f8b10a00640 1 --2- 172.21.15.12:0/1633395768 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557faca9e790 0x557facaa7f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:15:21.287 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:21.282+0000 7f8b03e00640 1 -- 172.21.15.12:0/1633395768 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557faca9e790 msgr2=0x557facaa7f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:15:21.287 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:21.282+0000 7f8b03e00640 1 --2- 172.21.15.12:0/1633395768 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557faca9e790 0x557facaa7f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:15:21.287 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:21.282+0000 7f8b03e00640 1 -- 172.21.15.12:0/1633395768 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557facaa3b00 msgr2=0x557facaa9190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:15:21.287 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:21.282+0000 7f8b03e00640 1 --2- 172.21.15.12:0/1633395768 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557facaa3b00 0x557facaa9190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:15:21.287 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:21.282+0000 7f8b03e00640 1 -- 172.21.15.12:0/1633395768 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557fac762790 con 0x557facaa10c0 2024-09-11T00:15:21.287 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:21.282+0000 7f8b11400640 1 --2- 172.21.15.12:0/1633395768 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557facaa3b00 0x557facaa9190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:15:21.287 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:21.282+0000 7f8b10a00640 1 --2- 172.21.15.12:0/1633395768 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557faca9e790 0x557facaa7f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:15:21.288 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:21.282+0000 7f8b03e00640 1 --2- 172.21.15.12:0/1633395768 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557facaa10c0 0x557facaa8630 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f8af4011010 tx=0x7f8af4003dd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=af138e711d8dc070 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:15:21.288 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:21.282+0000 7f8afa000640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-09-11T00:15:21.288 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:21.282+0000 7f8afa000640 1 -- 172.21.15.12:0/1633395768 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8af4029060 con 0x557facaa10c0 2024-09-11T00:15:21.288 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:21.282+0000 7f8b163f0740 1 -- 172.21.15.12:0/1633395768 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8af40035f0 con 0x557facaa10c0 2024-09-11T00:15:21.288 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:21.282+0000 7f8afa000640 1 -- 172.21.15.12:0/1633395768 <== mon.2 v2:172.21.15.165:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9208+0+0 (secure 0 0 0) 0x7f8af4003ab0 con 0x557facaa10c0 2024-09-11T00:15:21.288 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:21.282+0000 7f8afa000640 1 -- 172.21.15.12:0/1633395768 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f8af4030050 con 0x557facaa10c0 2024-09-11T00:15:21.288 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:21.282+0000 7f8b163f0740 10 client.26453 fetch_fsmap learned FSMap version 41 2024-09-11T00:15:21.288 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:21.282+0000 7f8b163f0740 1 -- 172.21.15.12:0/1633395768 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x557fac7db620 con 0x557facaa10c0 2024-09-11T00:15:21.288 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:21.283+0000 7f8afa000640 1 -- 172.21.15.12:0/1633395768 <== mon.2 v2:172.21.15.165:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8af40035f0 con 0x557facaa10c0 2024-09-11T00:15:21.288 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:21.283+0000 7f8afa000640 1 -- 172.21.15.12:0/1633395768 <== mon.2 v2:172.21.15.165:3300/0 5 ==== fsmap(e 41) ==== 8370+0+0 (secure 0 0 0) 0x7f8af40046c0 con 0x557facaa10c0 2024-09-11T00:15:21.288 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:21.283+0000 7f8b163f0740 10 client.26453 fetch_fsmap finished waiting for FSMap version 41 2024-09-11T00:15:21.288 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:21.283+0000 7f8b163f0740 10 client.26453 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T00:15:21.289 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:21.283+0000 7f8b163f0740 20 client.26453 populate_metadata read hostname 'smithi012' 2024-09-11T00:15:21.289 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:21.283+0000 7f8b163f0740 1 --2- 172.21.15.12:0/1633395768 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x557facab34d0 0x557facab58c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:15:21.289 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:21.283+0000 7f8b163f0740 4 client.26453 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T00:15:21.289 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:21.283+0000 7f8b163f0740 1 -- 172.21.15.12:0/1633395768 --> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557faca13600 con 0x557facab34d0 2024-09-11T00:15:21.289 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:21.283+0000 7f8b10a00640 1 --2- 172.21.15.12:0/1633395768 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x557facab34d0 0x557facab58c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:15:21.289 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:21.284+0000 7f8b10a00640 1 --2- 172.21.15.12:0/1633395768 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x557facab34d0 0x557facab58c0 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=909db0de7f195bb3 server_cookie=5d5b6f894683009b in_seq=0 out_seq=0 2024-09-11T00:15:21.289 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:21.284+0000 7f8afa000640 10 client.26453 ms_handle_connect on v2:172.21.15.165:6838/501312984 2024-09-11T00:15:21.290 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:21.285+0000 7f8afa000640 1 -- 172.21.15.12:0/1633395768 <== mds.0 v2:172.21.15.165:6838/501312984 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x557faca13600 con 0x557facab34d0 2024-09-11T00:15:21.290 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:21.285+0000 7f8afa000640 10 client.26453 handle_command_reply: tid=0 2024-09-11T00:15:21.290 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:21.285+0000 7f8b163f0740 1 client.26453 shutdown 2024-09-11T00:15:21.290 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:21.285+0000 7f8b163f0740 1 -- 172.21.15.12:0/1633395768 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557facaa10c0 msgr2=0x557facaa8630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:15:21.290 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:21.285+0000 7f8b163f0740 1 --2- 172.21.15.12:0/1633395768 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557facaa10c0 0x557facaa8630 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f8af4011010 tx=0x7f8af4003dd0 comp rx=0 tx=0).stop 2024-09-11T00:15:21.291 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:21.285+0000 7f8b163f0740 1 -- 172.21.15.12:0/1633395768 shutdown_connections 2024-09-11T00:15:21.291 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:21.285+0000 7f8b163f0740 1 --2- 172.21.15.12:0/1633395768 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x557facab34d0 0x557facab58c0 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:15:21.291 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:21.285+0000 7f8b163f0740 1 --2- 172.21.15.12:0/1633395768 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557facaa3b00 0x557facaa9190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:15:21.291 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:21.285+0000 7f8b163f0740 1 --2- 172.21.15.12:0/1633395768 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557facaa10c0 0x557facaa8630 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:15:21.291 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:21.285+0000 7f8b163f0740 1 --2- 172.21.15.12:0/1633395768 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557faca9e790 0x557facaa7f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:15:21.291 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:21.285+0000 7f8afa000640 0 client.26453 ms_handle_reset on v2:172.21.15.165:6838/501312984 2024-09-11T00:15:21.291 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:21.285+0000 7f8b163f0740 1 -- 172.21.15.12:0/1633395768 >> 172.21.15.12:0/1633395768 conn(0x557faca998e0 msgr2=0x557faca98a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:15:21.291 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:21.285+0000 7f8b163f0740 1 -- 172.21.15.12:0/1633395768 shutdown_connections 2024-09-11T00:15:21.291 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:21.285+0000 7f8b163f0740 1 -- 172.21.15.12:0/1633395768 wait complete. 2024-09-11T00:15:21.291 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:21.285+0000 7f8b163f0740 20 client.26453 trim_cache size 0 max 16384 2024-09-11T00:15:21.302 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:21.300+0000 7f8b163f0740 1 Processor -- start 2024-09-11T00:15:21.302 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:21.300+0000 7f8b163f0740 1 -- start start 2024-09-11T00:15:21.302 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:21.301+0000 7f8b163f0740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557faca9e790 0x557facaa86a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:15:21.302 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:21.301+0000 7f8b163f0740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557facaa10c0 0x557facaa6eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:15:21.303 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:21.301+0000 7f8b163f0740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557facaa3b00 0x557facaa73f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:15:21.303 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:21.301+0000 7f8b163f0740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f8b0c1ada20 con 0x557faca9e790 2024-09-11T00:15:21.303 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:21.301+0000 7f8b163f0740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f8b0c1af560 con 0x557facaa3b00 2024-09-11T00:15:21.303 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:21.301+0000 7f8b163f0740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f8b0c1ae0f0 con 0x557facaa10c0 2024-09-11T00:15:21.303 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:21.301+0000 7f8b10a00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557faca9e790 0x557facaa86a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:15:21.303 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:21.301+0000 7f8b10a00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557faca9e790 0x557facaa86a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:56574/0 (socket says 172.21.15.12:56574) 2024-09-11T00:15:21.303 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:21.301+0000 7f8b10a00640 1 -- 172.21.15.12:0/86829426 learned_addr learned my addr 172.21.15.12:0/86829426 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:15:21.303 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:21.301+0000 7f8b03e00640 1 --2- 172.21.15.12:0/86829426 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557facaa10c0 0x557facaa6eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:15:21.303 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:21.301+0000 7f8b11400640 1 --2- 172.21.15.12:0/86829426 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557facaa3b00 0x557facaa73f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:15:21.303 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:21.301+0000 7f8b10a00640 1 -- 172.21.15.12:0/86829426 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557facaa10c0 msgr2=0x557facaa6eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:15:21.303 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:21.301+0000 7f8b10a00640 1 --2- 172.21.15.12:0/86829426 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557facaa10c0 0x557facaa6eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:15:21.303 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:21.301+0000 7f8b10a00640 1 -- 172.21.15.12:0/86829426 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557facaa3b00 msgr2=0x557facaa73f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:15:21.303 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:21.301+0000 7f8b10a00640 1 --2- 172.21.15.12:0/86829426 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557facaa3b00 0x557facaa73f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:15:21.304 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:21.301+0000 7f8b10a00640 1 -- 172.21.15.12:0/86829426 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557facaa9500 con 0x557faca9e790 2024-09-11T00:15:21.304 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:21.301+0000 7f8b10a00640 1 --2- 172.21.15.12:0/86829426 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557faca9e790 0x557facaa86a0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f8afc0e39c0 tx=0x7f8afc000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80c8e1d005a105c5 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:15:21.307 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:21.302+0000 7f8b01200640 1 -- 172.21.15.12:0/86829426 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8afc0b5e60 con 0x557faca9e790 2024-09-11T00:15:21.307 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:21.302+0000 7f8b01200640 1 -- 172.21.15.12:0/86829426 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8afc0b1160 con 0x557faca9e790 2024-09-11T00:15:21.307 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:21.302+0000 7f8b01200640 1 -- 172.21.15.12:0/86829426 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8afc018d80 con 0x557faca9e790 2024-09-11T00:15:21.307 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:21.302+0000 7f8b163f0740 1 -- 172.21.15.12:0/86829426 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557faca9e790 msgr2=0x557facaa86a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:15:21.307 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:21.302+0000 7f8b163f0740 1 --2- 172.21.15.12:0/86829426 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557faca9e790 0x557facaa86a0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f8afc0e39c0 tx=0x7f8afc000bc0 comp rx=0 tx=0).stop 2024-09-11T00:15:21.307 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:21.302+0000 7f8b163f0740 1 -- 172.21.15.12:0/86829426 shutdown_connections 2024-09-11T00:15:21.307 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:21.302+0000 7f8b163f0740 1 --2- 172.21.15.12:0/86829426 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557facaa3b00 0x557facaa73f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:15:21.307 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:21.302+0000 7f8b163f0740 1 --2- 172.21.15.12:0/86829426 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557facaa10c0 0x557facaa6eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:15:21.308 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:21.302+0000 7f8b163f0740 1 --2- 172.21.15.12:0/86829426 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557faca9e790 0x557facaa86a0 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:15:21.308 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:21.302+0000 7f8b163f0740 1 -- 172.21.15.12:0/86829426 >> 172.21.15.12:0/86829426 conn(0x557faca998e0 msgr2=0x557facaa7f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:15:21.308 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:21.302+0000 7f8b163f0740 1 -- 172.21.15.12:0/86829426 shutdown_connections 2024-09-11T00:15:21.308 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:21.302+0000 7f8b163f0740 1 -- 172.21.15.12:0/86829426 wait complete. 2024-09-11T00:15:21.308 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:21.303+0000 7f8b163f0740 1 -- start start 2024-09-11T00:15:21.308 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:21.303+0000 7f8b163f0740 1 Processor -- start 2024-09-11T00:15:21.308 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:21.303+0000 7f8b163f0740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557faca9e790 0x557facaaf7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:15:21.308 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:21.303+0000 7f8b163f0740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557facaa10c0 0x557facaae4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:15:21.308 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:21.303+0000 7f8b163f0740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557facaa3b00 0x557facaae9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:15:21.308 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:21.303+0000 7f8b163f0740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f8b0c1aee90 con 0x557faca9e790 2024-09-11T00:15:21.308 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:21.303+0000 7f8b163f0740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f8b0c1ae7c0 con 0x557facaa3b00 2024-09-11T00:15:21.308 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:21.303+0000 7f8b163f0740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f8b0c1aba00 con 0x557facaa10c0 2024-09-11T00:15:21.309 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:21.303+0000 7f8b163f0740 10 client.0 _ll_register_callbacks cb 0x7f8b04031f50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:15:21.309 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:21.304+0000 7f8b10a00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557faca9e790 0x557facaaf7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:15:21.309 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:21.304+0000 7f8b10a00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557faca9e790 0x557facaaf7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:56580/0 (socket says 172.21.15.12:56580) 2024-09-11T00:15:21.309 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:21.304+0000 7f8b10a00640 1 -- 172.21.15.12:0/1879279829 learned_addr learned my addr 172.21.15.12:0/1879279829 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:15:21.309 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:21.304+0000 7f8b03e00640 1 --2- 172.21.15.12:0/1879279829 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557facaa10c0 0x557facaae4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:15:21.309 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:21.304+0000 7f8b11400640 1 --2- 172.21.15.12:0/1879279829 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557facaa3b00 0x557facaae9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:15:21.309 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:21.304+0000 7f8b10a00640 1 -- 172.21.15.12:0/1879279829 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557facaa10c0 msgr2=0x557facaae4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:15:21.309 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:21.304+0000 7f8b10a00640 1 --2- 172.21.15.12:0/1879279829 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557facaa10c0 0x557facaae4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:15:21.309 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:21.304+0000 7f8b10a00640 1 -- 172.21.15.12:0/1879279829 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557facaa3b00 msgr2=0x557facaae9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:15:21.309 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:21.304+0000 7f8b10a00640 1 --2- 172.21.15.12:0/1879279829 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557facaa3b00 0x557facaae9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:15:21.309 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:21.304+0000 7f8b10a00640 1 -- 172.21.15.12:0/1879279829 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557facaaefa0 con 0x557faca9e790 2024-09-11T00:15:21.309 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:21.304+0000 7f8b11400640 1 --2- 172.21.15.12:0/1879279829 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557facaa3b00 0x557facaae9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T00:15:21.310 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:21.304+0000 7f8b10a00640 1 --2- 172.21.15.12:0/1879279829 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557faca9e790 0x557facaaf7a0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f8afc0ac430 tx=0x7f8afc00c840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6665f251c4b3e93 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:15:21.310 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:21.304+0000 7f8afa000640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-09-11T00:15:21.310 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:21.304+0000 7f8afa000640 1 -- 172.21.15.12:0/1879279829 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8afc018b20 con 0x557faca9e790 2024-09-11T00:15:21.310 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:21.304+0000 7f8afa000640 1 -- 172.21.15.12:0/1879279829 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9208+0+0 (secure 0 0 0) 0x7f8afc018cc0 con 0x557faca9e790 2024-09-11T00:15:21.310 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:21.304+0000 7f8b163f0740 1 -- 172.21.15.12:0/1879279829 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8afc0b1160 con 0x557faca9e790 2024-09-11T00:15:21.310 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:21.305+0000 7f8afa000640 1 -- 172.21.15.12:0/1879279829 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f8afc0a8080 con 0x557faca9e790 2024-09-11T00:15:21.310 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:21.305+0000 7f8b163f0740 10 client.16452 fetch_fsmap learned FSMap version 41 2024-09-11T00:15:21.310 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:21.305+0000 7f8b163f0740 1 -- 172.21.15.12:0/1879279829 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x557facaaddc0 con 0x557faca9e790 2024-09-11T00:15:21.310 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:21.305+0000 7f8afa000640 1 -- 172.21.15.12:0/1879279829 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8afc0b1160 con 0x557faca9e790 2024-09-11T00:15:21.310 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:21.305+0000 7f8afa000640 1 -- 172.21.15.12:0/1879279829 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 41) ==== 8370+0+0 (secure 0 0 0) 0x7f8afc0c5070 con 0x557faca9e790 2024-09-11T00:15:21.310 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:21.305+0000 7f8b163f0740 10 client.16452 fetch_fsmap finished waiting for FSMap version 41 2024-09-11T00:15:21.310 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:21.305+0000 7f8b163f0740 10 client.16452 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T00:15:21.310 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:21.305+0000 7f8b163f0740 20 client.16452 populate_metadata read hostname 'smithi012' 2024-09-11T00:15:21.311 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:21.305+0000 7f8b163f0740 1 --2- 172.21.15.12:0/1879279829 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x557facb20a90 0x557facb22eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:15:21.311 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:21.305+0000 7f8b163f0740 4 client.16452 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T00:15:21.311 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:21.305+0000 7f8b163f0740 1 -- 172.21.15.12:0/1879279829 --> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] -- command(tid 0: {"prefix": "flush journal"}) -- 0x557faca0f0b0 con 0x557facb20a90 2024-09-11T00:15:21.311 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:21.305+0000 7f8b03e00640 1 --2- 172.21.15.12:0/1879279829 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x557facb20a90 0x557facb22eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:15:21.311 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:21.306+0000 7f8b03e00640 1 --2- 172.21.15.12:0/1879279829 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x557facb20a90 0x557facb22eb0 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=98ecdf706ad1906d server_cookie=a4b651ae68825c70 in_seq=0 out_seq=0 2024-09-11T00:15:21.311 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:21.306+0000 7f8afa000640 10 client.16452 ms_handle_connect on v2:172.21.15.165:6838/501312984 2024-09-11T00:15:21.347 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:21.345+0000 7f8afa000640 1 -- 172.21.15.12:0/1879279829 <== mds.0 v2:172.21.15.165:6838/501312984 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x557faca0f0b0 con 0x557facb20a90 2024-09-11T00:15:21.347 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:21.345+0000 7f8afa000640 10 client.16452 handle_command_reply: tid=0 2024-09-11T00:15:21.347 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:21.345+0000 7f8b163f0740 1 client.16452 shutdown 2024-09-11T00:15:21.347 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:21.345+0000 7f8b163f0740 1 -- 172.21.15.12:0/1879279829 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557faca9e790 msgr2=0x557facaaf7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:15:21.347 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:21.345+0000 7f8b163f0740 1 --2- 172.21.15.12:0/1879279829 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557faca9e790 0x557facaaf7a0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f8afc0ac430 tx=0x7f8afc00c840 comp rx=0 tx=0).stop 2024-09-11T00:15:21.347 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:21.345+0000 7f8b163f0740 1 -- 172.21.15.12:0/1879279829 shutdown_connections 2024-09-11T00:15:21.347 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:21.345+0000 7f8b163f0740 1 --2- 172.21.15.12:0/1879279829 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x557facb20a90 0x557facb22eb0 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:15:21.347 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:21.345+0000 7f8b163f0740 1 --2- 172.21.15.12:0/1879279829 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557facaa3b00 0x557facaae9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:15:21.348 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:21.345+0000 7f8b163f0740 1 --2- 172.21.15.12:0/1879279829 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557facaa10c0 0x557facaae4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:15:21.348 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:21.345+0000 7f8b163f0740 1 --2- 172.21.15.12:0/1879279829 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557faca9e790 0x557facaaf7a0 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:15:21.348 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:21.345+0000 7f8afa000640 0 client.16452 ms_handle_reset on v2:172.21.15.165:6838/501312984 2024-09-11T00:15:21.348 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:21.345+0000 7f8b163f0740 1 -- 172.21.15.12:0/1879279829 >> 172.21.15.12:0/1879279829 conn(0x557faca998e0 msgr2=0x557facaa6eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:15:21.348 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:21.346+0000 7f8b163f0740 1 -- 172.21.15.12:0/1879279829 shutdown_connections 2024-09-11T00:15:21.348 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:21.346+0000 7f8b163f0740 1 -- 172.21.15.12:0/1879279829 wait complete. 2024-09-11T00:15:21.348 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:21.346+0000 7f8b163f0740 20 client.16452 trim_cache size 0 max 16384 2024-09-11T00:15:21.348 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:{ 2024-09-11T00:15:21.348 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "message": "", 2024-09-11T00:15:21.348 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "return_code": 0 2024-09-11T00:15:21.348 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:} 2024-09-11T00:15:21.349 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:21.346+0000 7f8b13400640 1 -- 172.21.15.12:0/2402925372 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f8af00b51f0 msgr2=0x7f8af00b76b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:15:21.349 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:21.346+0000 7f8b13400640 1 --2- 172.21.15.12:0/2402925372 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f8af00b51f0 0x7f8af00b76b0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f8b04006010 tx=0x7f8b04031850 comp rx=0 tx=0).stop 2024-09-11T00:15:21.349 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:21.346+0000 7f8b13400640 1 -- 172.21.15.12:0/2402925372 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8b0c104b90 msgr2=0x7f8b0c19af20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:15:21.349 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:21.346+0000 7f8b13400640 1 --2- 172.21.15.12:0/2402925372 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8b0c104b90 0x7f8b0c19af20 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f8afc00e9f0 tx=0x7f8afc00eeb0 comp rx=0 tx=0).stop 2024-09-11T00:15:21.350 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:21.347+0000 7f8b13400640 1 -- 172.21.15.12:0/2402925372 shutdown_connections 2024-09-11T00:15:21.350 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:21.347+0000 7f8b13400640 1 --2- 172.21.15.12:0/2402925372 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f8af00b51f0 0x7f8af00b76b0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:15:21.350 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:21.347+0000 7f8b13400640 1 --2- 172.21.15.12:0/2402925372 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8b0c1066a0 0x7f8b0c19c1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:15:21.350 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:21.347+0000 7f8b13400640 1 --2- 172.21.15.12:0/2402925372 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8b0c105ce0 0x7f8b0c19b570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:15:21.350 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:21.347+0000 7f8b13400640 1 --2- 172.21.15.12:0/2402925372 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8b0c104b90 0x7f8b0c19af20 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:15:21.350 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:21.347+0000 7f8b13400640 1 -- 172.21.15.12:0/2402925372 >> 172.21.15.12:0/2402925372 conn(0x7f8b0c100470 msgr2=0x7f8b0c109070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:15:21.350 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:21.347+0000 7f8b13400640 1 -- 172.21.15.12:0/2402925372 shutdown_connections 2024-09-11T00:15:21.350 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:21.347+0000 7f8b13400640 1 -- 172.21.15.12:0/2402925372 wait complete. 2024-09-11T00:15:21.663 INFO:tasks.workunit.client.0.smithi012.stdout: 1 312804 3.67 MB/sec execute 318 sec latency 6831.042 ms 2024-09-11T00:15:21.782 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:15:21 smithi165 ceph-mon[35356]: pgmap v647: 129 pgs: 129 active+clean; 79 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 1.8 MiB/s wr, 264 op/s 2024-09-11T00:15:21.813 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:15:21 smithi012 ceph-mon[30333]: pgmap v647: 129 pgs: 129 active+clean; 79 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 1.8 MiB/s wr, 264 op/s 2024-09-11T00:15:21.831 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:15:21 smithi178 ceph-mon[35919]: pgmap v647: 129 pgs: 129 active+clean; 79 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 1.8 MiB/s wr, 264 op/s 2024-09-11T00:15:22.663 INFO:tasks.workunit.client.0.smithi012.stdout: 1 312804 3.66 MB/sec execute 319 sec latency 7831.157 ms 2024-09-11T00:15:23.663 INFO:tasks.workunit.client.0.smithi012.stdout: 1 312804 3.64 MB/sec execute 320 sec latency 8831.210 ms 2024-09-11T00:15:23.705 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:15:23 smithi012 ceph-mon[30333]: pgmap v648: 129 pgs: 129 active+clean; 79 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 1.2 MiB/s wr, 186 op/s 2024-09-11T00:15:23.782 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:15:23 smithi165 ceph-mon[35356]: pgmap v648: 129 pgs: 129 active+clean; 79 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 1.2 MiB/s wr, 186 op/s 2024-09-11T00:15:23.831 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:15:23 smithi178 ceph-mon[35919]: pgmap v648: 129 pgs: 129 active+clean; 79 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 1.2 MiB/s wr, 186 op/s 2024-09-11T00:15:24.663 INFO:tasks.workunit.client.0.smithi012.stdout: 1 312804 3.63 MB/sec execute 321 sec latency 9831.322 ms 2024-09-11T00:15:25.664 INFO:tasks.workunit.client.0.smithi012.stdout: 1 312804 3.62 MB/sec execute 322 sec latency 10831.450 ms 2024-09-11T00:15:25.782 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:15:25 smithi165 ceph-mon[35356]: pgmap v649: 129 pgs: 129 active+clean; 79 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 1.1 MiB/s wr, 133 op/s 2024-09-11T00:15:25.813 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:15:25 smithi012 ceph-mon[30333]: pgmap v649: 129 pgs: 129 active+clean; 79 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 1.1 MiB/s wr, 133 op/s 2024-09-11T00:15:25.830 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:15:25 smithi178 ceph-mon[35919]: pgmap v649: 129 pgs: 129 active+clean; 79 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 1.1 MiB/s wr, 133 op/s 2024-09-11T00:15:26.664 INFO:tasks.workunit.client.0.smithi012.stdout: 1 312804 3.61 MB/sec execute 323 sec latency 11831.605 ms 2024-09-11T00:15:27.664 INFO:tasks.workunit.client.0.smithi012.stdout: 1 312804 3.60 MB/sec execute 324 sec latency 12831.792 ms 2024-09-11T00:15:27.782 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:15:27 smithi165 ceph-mon[35356]: pgmap v650: 129 pgs: 129 active+clean; 80 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 1.3 MiB/s wr, 159 op/s 2024-09-11T00:15:27.813 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:15:27 smithi012 ceph-mon[30333]: pgmap v650: 129 pgs: 129 active+clean; 80 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 1.3 MiB/s wr, 159 op/s 2024-09-11T00:15:27.831 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:15:27 smithi178 ceph-mon[35919]: pgmap v650: 129 pgs: 129 active+clean; 80 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 1.3 MiB/s wr, 159 op/s 2024-09-11T00:15:28.664 INFO:tasks.workunit.client.0.smithi012.stdout: 1 312804 3.59 MB/sec execute 325 sec latency 13831.859 ms 2024-09-11T00:15:29.664 INFO:tasks.workunit.client.0.smithi012.stdout: 1 312804 3.58 MB/sec execute 326 sec latency 14831.903 ms 2024-09-11T00:15:29.782 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:15:29 smithi165 ceph-mon[35356]: pgmap v651: 129 pgs: 129 active+clean; 80 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 805 KiB/s wr, 73 op/s 2024-09-11T00:15:29.813 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:15:29 smithi012 ceph-mon[30333]: pgmap v651: 129 pgs: 129 active+clean; 80 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 805 KiB/s wr, 73 op/s 2024-09-11T00:15:29.830 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:15:29 smithi178 ceph-mon[35919]: pgmap v651: 129 pgs: 129 active+clean; 80 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 805 KiB/s wr, 73 op/s 2024-09-11T00:15:30.664 INFO:tasks.workunit.client.0.smithi012.stdout: 1 312804 3.57 MB/sec execute 327 sec latency 15831.968 ms 2024-09-11T00:15:31.563 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:15:31 smithi012 ceph-mon[30333]: pgmap v652: 129 pgs: 129 active+clean; 78 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 1.2 MiB/s wr, 78 op/s 2024-09-11T00:15:31.664 INFO:tasks.workunit.client.0.smithi012.stdout: 1 312804 3.55 MB/sec execute 328 sec latency 16832.126 ms 2024-09-11T00:15:31.782 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:15:31 smithi165 ceph-mon[35356]: pgmap v652: 129 pgs: 129 active+clean; 78 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 1.2 MiB/s wr, 78 op/s 2024-09-11T00:15:31.831 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:15:31 smithi178 ceph-mon[35919]: pgmap v652: 129 pgs: 129 active+clean; 78 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 1.2 MiB/s wr, 78 op/s 2024-09-11T00:15:32.664 INFO:tasks.workunit.client.0.smithi012.stdout: 1 312804 3.54 MB/sec execute 329 sec latency 17832.250 ms 2024-09-11T00:15:33.665 INFO:tasks.workunit.client.0.smithi012.stdout: 1 312804 3.53 MB/sec execute 330 sec latency 18832.344 ms 2024-09-11T00:15:33.781 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:15:33 smithi012 ceph-mon[30333]: pgmap v653: 129 pgs: 129 active+clean; 78 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 1.0 MiB/s wr, 77 op/s 2024-09-11T00:15:33.782 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:15:33 smithi165 ceph-mon[35356]: pgmap v653: 129 pgs: 129 active+clean; 78 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 1.0 MiB/s wr, 77 op/s 2024-09-11T00:15:33.831 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:15:33 smithi178 ceph-mon[35919]: pgmap v653: 129 pgs: 129 active+clean; 78 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 1.0 MiB/s wr, 77 op/s 2024-09-11T00:15:34.465 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:34.464+0000 7fd725200640 1 -- 172.21.15.12:0/396071599 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd720104ba0 msgr2=0x7fd720104fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:15:34.466 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:34.464+0000 7fd725200640 1 --2- 172.21.15.12:0/396071599 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd720104ba0 0x7fd720104fa0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fd710009930 tx=0x7fd71002a420 comp rx=0 tx=0).stop 2024-09-11T00:15:34.466 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:34.464+0000 7fd725200640 1 -- 172.21.15.12:0/396071599 shutdown_connections 2024-09-11T00:15:34.466 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:34.464+0000 7fd725200640 1 --2- 172.21.15.12:0/396071599 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd7201066b0 0x7fd72010d030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:15:34.466 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:34.464+0000 7fd725200640 1 --2- 172.21.15.12:0/396071599 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd720105cf0 0x7fd720106170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:15:34.466 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:34.464+0000 7fd725200640 1 --2- 172.21.15.12:0/396071599 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd720104ba0 0x7fd720104fa0 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:15:34.466 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:34.464+0000 7fd725200640 1 -- 172.21.15.12:0/396071599 >> 172.21.15.12:0/396071599 conn(0x7fd720100460 msgr2=0x7fd720102880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:15:34.466 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:34.464+0000 7fd725200640 1 -- 172.21.15.12:0/396071599 shutdown_connections 2024-09-11T00:15:34.466 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:34.464+0000 7fd725200640 1 -- 172.21.15.12:0/396071599 wait complete. 2024-09-11T00:15:34.468 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:34.465+0000 7fd725200640 1 Processor -- start 2024-09-11T00:15:34.468 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:34.465+0000 7fd725200640 1 -- start start 2024-09-11T00:15:34.468 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:34.465+0000 7fd725200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd720104ba0 0x7fd72019afd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:15:34.468 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:34.465+0000 7fd725200640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd720105cf0 0x7fd72019b620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:15:34.468 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:34.465+0000 7fd725200640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd7201066b0 0x7fd72019c250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:15:34.468 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:34.465+0000 7fd725200640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fd720110810 con 0x7fd7201066b0 2024-09-11T00:15:34.468 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:34.465+0000 7fd725200640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fd720110690 con 0x7fd720104ba0 2024-09-11T00:15:34.469 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:34.465+0000 7fd725200640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fd720110990 con 0x7fd720105cf0 2024-09-11T00:15:34.469 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:34.466+0000 7fd71e800640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd7201066b0 0x7fd72019c250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:15:34.469 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:34.466+0000 7fd71e800640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd7201066b0 0x7fd72019c250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:50250/0 (socket says 172.21.15.12:50250) 2024-09-11T00:15:34.469 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:34.466+0000 7fd71e800640 1 -- 172.21.15.12:0/3524160410 learned_addr learned my addr 172.21.15.12:0/3524160410 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:15:34.469 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:34.466+0000 7fd71d400640 1 --2- 172.21.15.12:0/3524160410 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd720105cf0 0x7fd72019b620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:15:34.469 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:34.466+0000 7fd71de00640 1 --2- 172.21.15.12:0/3524160410 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd720104ba0 0x7fd72019afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:15:34.469 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:34.466+0000 7fd71e800640 1 -- 172.21.15.12:0/3524160410 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd720105cf0 msgr2=0x7fd72019b620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:15:34.469 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:34.466+0000 7fd71e800640 1 --2- 172.21.15.12:0/3524160410 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd720105cf0 0x7fd72019b620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:15:34.469 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:34.466+0000 7fd71e800640 1 -- 172.21.15.12:0/3524160410 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd720104ba0 msgr2=0x7fd72019afd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:15:34.469 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:34.466+0000 7fd71e800640 1 --2- 172.21.15.12:0/3524160410 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd720104ba0 0x7fd72019afd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:15:34.469 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:34.466+0000 7fd71e800640 1 -- 172.21.15.12:0/3524160410 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd72019cb10 con 0x7fd7201066b0 2024-09-11T00:15:34.469 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:34.466+0000 7fd71e800640 1 --2- 172.21.15.12:0/3524160410 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd7201066b0 0x7fd72019c250 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fd71800eaa0 tx=0x7fd71800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:15:34.470 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:34.467+0000 7fd716a00640 1 -- 172.21.15.12:0/3524160410 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd71800cdb0 con 0x7fd7201066b0 2024-09-11T00:15:34.470 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:34.467+0000 7fd716a00640 1 -- 172.21.15.12:0/3524160410 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd718005020 con 0x7fd7201066b0 2024-09-11T00:15:34.470 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:34.467+0000 7fd716a00640 1 -- 172.21.15.12:0/3524160410 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7180116d0 con 0x7fd7201066b0 2024-09-11T00:15:34.470 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:34.467+0000 7fd725200640 1 -- 172.21.15.12:0/3524160410 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd72019ce00 con 0x7fd7201066b0 2024-09-11T00:15:34.470 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:34.467+0000 7fd725200640 1 -- 172.21.15.12:0/3524160410 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd7201a34d0 con 0x7fd7201066b0 2024-09-11T00:15:34.471 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:34.470+0000 7fd7280ad740 1 Processor -- start 2024-09-11T00:15:34.471 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:34.470+0000 7fd716a00640 1 -- 172.21.15.12:0/3524160410 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7fd71800f040 con 0x7fd7201066b0 2024-09-11T00:15:34.471 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:34.471+0000 7fd7280ad740 1 -- start start 2024-09-11T00:15:34.472 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:34.471+0000 7fd7280ad740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560155d8b790 0x560155d8db80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:15:34.472 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:34.471+0000 7fd7280ad740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560155d8e0c0 0x560155d904b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:15:34.472 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:34.471+0000 7fd7280ad740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560155d90b00 0x560155d92fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:15:34.472 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:34.471+0000 7fd7280ad740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x560155cfe6b0 con 0x560155d8e0c0 2024-09-11T00:15:34.473 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:34.471+0000 7fd7280ad740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x560155cedae0 con 0x560155d90b00 2024-09-11T00:15:34.473 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:34.471+0000 7fd7280ad740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x560155cf18f0 con 0x560155d8b790 2024-09-11T00:15:34.474 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:34.471+0000 7fd71de00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560155d8e0c0 0x560155d904b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:15:34.476 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:34.471+0000 7fd71d400640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560155d8b790 0x560155d8db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:15:34.476 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:34.471+0000 7fd71de00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560155d8e0c0 0x560155d904b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:50254/0 (socket says 172.21.15.12:50254) 2024-09-11T00:15:34.476 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:34.471+0000 7fd71de00640 1 -- 172.21.15.12:0/3511357887 learned_addr learned my addr 172.21.15.12:0/3511357887 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:15:34.476 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:34.472+0000 7fd71d400640 1 --2- 172.21.15.12:0/3511357887 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560155d90b00 0x560155d92fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:15:34.476 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:34.472+0000 7fd71de00640 1 -- 172.21.15.12:0/3511357887 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560155d8b790 msgr2=0x560155d8db80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:15:34.477 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:34.472+0000 7fd71de00640 1 --2- 172.21.15.12:0/3511357887 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560155d8b790 0x560155d8db80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:15:34.477 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:34.472+0000 7fd71de00640 1 -- 172.21.15.12:0/3511357887 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560155d90b00 msgr2=0x560155d92fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:15:34.477 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:34.472+0000 7fd71de00640 1 --2- 172.21.15.12:0/3511357887 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560155d90b00 0x560155d92fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:15:34.477 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:34.472+0000 7fd71de00640 1 -- 172.21.15.12:0/3511357887 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560155cba780 con 0x560155d8e0c0 2024-09-11T00:15:34.477 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:34.472+0000 7fd71de00640 1 --2- 172.21.15.12:0/3511357887 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560155d8e0c0 0x560155d904b0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fd71002a940 tx=0x7fd710032b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b382cac9d1bdbc5 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:15:34.477 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:34.472+0000 7fd716a00640 1 --2- 172.21.15.12:0/3524160410 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fd7040b5360 0x7fd7040b7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:15:34.477 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:34.472+0000 7fd716a00640 1 -- 172.21.15.12:0/3524160410 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9208+0+0 (secure 0 0 0) 0x7fd7180e5e20 con 0x7fd7201066b0 2024-09-11T00:15:34.477 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:34.472+0000 7fd71d400640 1 --2- 172.21.15.12:0/3524160410 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fd7040b5360 0x7fd7040b7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:15:34.477 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:34.473+0000 7fd70b400640 1 -- 172.21.15.12:0/3511357887 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd71003f070 con 0x560155d8e0c0 2024-09-11T00:15:34.477 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:34.473+0000 7fd70b400640 1 -- 172.21.15.12:0/3511357887 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd710004040 con 0x560155d8e0c0 2024-09-11T00:15:34.478 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:34.473+0000 7fd70b400640 1 -- 172.21.15.12:0/3511357887 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd710035070 con 0x560155d8e0c0 2024-09-11T00:15:34.481 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:34.473+0000 7fd7280ad740 1 -- 172.21.15.12:0/3511357887 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560155d8e0c0 msgr2=0x560155d904b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:15:34.482 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:34.473+0000 7fd71d400640 1 --2- 172.21.15.12:0/3524160410 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fd7040b5360 0x7fd7040b7820 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fd70c009fd0 tx=0x7fd70c0046c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:15:34.482 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:34.473+0000 7fd7280ad740 1 --2- 172.21.15.12:0/3511357887 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560155d8e0c0 0x560155d904b0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fd71002a940 tx=0x7fd710032b20 comp rx=0 tx=0).stop 2024-09-11T00:15:34.482 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:34.474+0000 7fd7280ad740 1 -- 172.21.15.12:0/3511357887 shutdown_connections 2024-09-11T00:15:34.482 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:34.474+0000 7fd7280ad740 1 --2- 172.21.15.12:0/3511357887 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560155d90b00 0x560155d92fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:15:34.482 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:34.474+0000 7fd7280ad740 1 --2- 172.21.15.12:0/3511357887 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560155d8e0c0 0x560155d904b0 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:15:34.482 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:34.474+0000 7fd7280ad740 1 --2- 172.21.15.12:0/3511357887 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560155d8b790 0x560155d8db80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:15:34.482 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:34.474+0000 7fd7280ad740 1 -- 172.21.15.12:0/3511357887 >> 172.21.15.12:0/3511357887 conn(0x560155d868e0 msgr2=0x560155d87670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:15:34.482 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:34.474+0000 7fd7280ad740 1 -- 172.21.15.12:0/3511357887 shutdown_connections 2024-09-11T00:15:34.482 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:34.474+0000 7fd7280ad740 1 -- 172.21.15.12:0/3511357887 wait complete. 2024-09-11T00:15:34.482 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:34.474+0000 7fd7280ad740 1 -- start start 2024-09-11T00:15:34.482 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:34.474+0000 7fd7280ad740 1 Processor -- start 2024-09-11T00:15:34.482 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:34.475+0000 7fd7280ad740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560155d8b790 0x560155d94e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:15:34.483 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:34.475+0000 7fd7280ad740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560155d8e0c0 0x560155d95500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:15:34.483 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:34.475+0000 7fd7280ad740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560155d90b00 0x560155d960d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:15:34.483 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:34.475+0000 7fd7280ad740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x560155c25ab0 con 0x560155d8e0c0 2024-09-11T00:15:34.483 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:34.475+0000 7fd7280ad740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x560155c94570 con 0x560155d90b00 2024-09-11T00:15:34.483 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:34.475+0000 7fd7280ad740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x560155d93a80 con 0x560155d8b790 2024-09-11T00:15:34.483 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:34.475+0000 7fd7280ad740 10 client.0 _ll_register_callbacks cb 0x560155ca3100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:15:34.483 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:34.475+0000 7fd71d400640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560155d8e0c0 0x560155d95500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:15:34.483 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:34.475+0000 7fd71e800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560155d90b00 0x560155d960d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:15:34.483 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:34.475+0000 7fd71e800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560155d90b00 0x560155d960d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.12:44642/0 (socket says 172.21.15.12:44642) 2024-09-11T00:15:34.483 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:34.475+0000 7fd71e800640 1 -- 172.21.15.12:0/2179855121 learned_addr learned my addr 172.21.15.12:0/2179855121 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:15:34.484 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:34.475+0000 7fd71e800640 1 -- 172.21.15.12:0/2179855121 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560155d8b790 msgr2=0x560155d94e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:15:34.484 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:34.475+0000 7fd71e800640 1 --2- 172.21.15.12:0/2179855121 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560155d8b790 0x560155d94e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:15:34.484 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:34.475+0000 7fd71e800640 1 -- 172.21.15.12:0/2179855121 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560155d8e0c0 msgr2=0x560155d95500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:15:34.484 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:34.475+0000 7fd71e800640 1 --2- 172.21.15.12:0/2179855121 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560155d8e0c0 0x560155d95500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:15:34.484 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:34.475+0000 7fd71e800640 1 -- 172.21.15.12:0/2179855121 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560155a4f790 con 0x560155d90b00 2024-09-11T00:15:34.484 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:34.476+0000 7fd71e800640 1 --2- 172.21.15.12:0/2179855121 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560155d90b00 0x560155d960d0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fd7180a8780 tx=0x7fd7180abe00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=87ef4b0dbe95f161 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:15:34.484 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:34.476+0000 7fd6ffe00640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-09-11T00:15:34.484 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:34.476+0000 7fd6ffe00640 1 -- 172.21.15.12:0/2179855121 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd71801f040 con 0x560155d90b00 2024-09-11T00:15:34.484 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:34.476+0000 7fd7280ad740 1 -- 172.21.15.12:0/2179855121 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd710004040 con 0x560155d90b00 2024-09-11T00:15:34.484 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:34.476+0000 7fd6ffe00640 1 -- 172.21.15.12:0/2179855121 <== mon.2 v2:172.21.15.165:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9208+0+0 (secure 0 0 0) 0x7fd7180974c0 con 0x560155d90b00 2024-09-11T00:15:34.485 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:34.476+0000 7fd6ffe00640 1 -- 172.21.15.12:0/2179855121 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7fd7180b5740 con 0x560155d90b00 2024-09-11T00:15:34.485 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:34.476+0000 7fd7280ad740 10 client.26468 fetch_fsmap learned FSMap version 41 2024-09-11T00:15:34.485 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:34.476+0000 7fd7280ad740 1 -- 172.21.15.12:0/2179855121 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x560155ac8620 con 0x560155d90b00 2024-09-11T00:15:34.485 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:34.476+0000 7fd6ffe00640 1 -- 172.21.15.12:0/2179855121 <== mon.2 v2:172.21.15.165:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd710004040 con 0x560155d90b00 2024-09-11T00:15:34.485 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:34.477+0000 7fd6ffe00640 1 -- 172.21.15.12:0/2179855121 <== mon.2 v2:172.21.15.165:3300/0 5 ==== fsmap(e 41) ==== 8370+0+0 (secure 0 0 0) 0x7fd7180cdb90 con 0x560155d90b00 2024-09-11T00:15:34.485 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:34.477+0000 7fd7280ad740 10 client.26468 fetch_fsmap finished waiting for FSMap version 41 2024-09-11T00:15:34.485 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:34.477+0000 7fd7280ad740 10 client.26468 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T00:15:34.485 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:34.477+0000 7fd7280ad740 20 client.26468 populate_metadata read hostname 'smithi012' 2024-09-11T00:15:34.485 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:34.477+0000 7fd7280ad740 1 --2- 172.21.15.12:0/2179855121 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x560155da0780 0x560155da2bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:15:34.485 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:34.477+0000 7fd7280ad740 4 client.26468 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T00:15:34.485 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:34.477+0000 7fd7280ad740 1 -- 172.21.15.12:0/2179855121 --> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560155d00600 con 0x560155da0780 2024-09-11T00:15:34.485 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:34.477+0000 7fd71de00640 1 --2- 172.21.15.12:0/2179855121 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x560155da0780 0x560155da2bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:15:34.486 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:34.478+0000 7fd71de00640 1 --2- 172.21.15.12:0/2179855121 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x560155da0780 0x560155da2bf0 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6818d5094c32e29f server_cookie=8021f03967beea73 in_seq=0 out_seq=0 2024-09-11T00:15:34.486 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:34.478+0000 7fd6ffe00640 10 client.26468 ms_handle_connect on v2:172.21.15.165:6838/501312984 2024-09-11T00:15:34.486 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:34.478+0000 7fd6ffe00640 1 -- 172.21.15.12:0/2179855121 <== mds.0 v2:172.21.15.165:6838/501312984 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x560155d00600 con 0x560155da0780 2024-09-11T00:15:34.486 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:34.478+0000 7fd6ffe00640 10 client.26468 handle_command_reply: tid=0 2024-09-11T00:15:34.486 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:34.479+0000 7fd7280ad740 1 client.26468 shutdown 2024-09-11T00:15:34.486 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:34.479+0000 7fd7280ad740 1 -- 172.21.15.12:0/2179855121 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560155d90b00 msgr2=0x560155d960d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:15:34.486 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:34.479+0000 7fd7280ad740 1 --2- 172.21.15.12:0/2179855121 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560155d90b00 0x560155d960d0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fd7180a8780 tx=0x7fd7180abe00 comp rx=0 tx=0).stop 2024-09-11T00:15:34.486 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:34.479+0000 7fd7280ad740 1 -- 172.21.15.12:0/2179855121 shutdown_connections 2024-09-11T00:15:34.486 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:34.479+0000 7fd7280ad740 1 --2- 172.21.15.12:0/2179855121 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x560155da0780 0x560155da2bf0 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:15:34.486 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:34.479+0000 7fd7280ad740 1 --2- 172.21.15.12:0/2179855121 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560155d90b00 0x560155d960d0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:15:34.486 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:34.479+0000 7fd7280ad740 1 --2- 172.21.15.12:0/2179855121 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560155d8e0c0 0x560155d95500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:15:34.486 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:34.479+0000 7fd6ffe00640 0 client.26468 ms_handle_reset on v2:172.21.15.165:6838/501312984 2024-09-11T00:15:34.486 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:34.479+0000 7fd7280ad740 1 --2- 172.21.15.12:0/2179855121 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560155d8b790 0x560155d94e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:15:34.487 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:34.479+0000 7fd7280ad740 1 -- 172.21.15.12:0/2179855121 >> 172.21.15.12:0/2179855121 conn(0x560155d868e0 msgr2=0x560155d85a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:15:34.487 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:34.479+0000 7fd7280ad740 1 -- 172.21.15.12:0/2179855121 shutdown_connections 2024-09-11T00:15:34.487 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:34.479+0000 7fd7280ad740 1 -- 172.21.15.12:0/2179855121 wait complete. 2024-09-11T00:15:34.487 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:34.479+0000 7fd7280ad740 20 client.26468 trim_cache size 0 max 16384 2024-09-11T00:15:34.496 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:34.495+0000 7fd7280ad740 1 Processor -- start 2024-09-11T00:15:34.497 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:34.495+0000 7fd7280ad740 1 -- start start 2024-09-11T00:15:34.497 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:34.495+0000 7fd7280ad740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560155d8b790 0x560155d89930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:15:34.497 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:34.495+0000 7fd7280ad740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560155d8e0c0 0x560155d89e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:15:34.497 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:34.495+0000 7fd7280ad740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560155d90b00 0x560155d871e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:15:34.497 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:34.495+0000 7fd7280ad740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fd7201ad900 con 0x560155d8e0c0 2024-09-11T00:15:34.497 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:34.495+0000 7fd7280ad740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fd7201af440 con 0x560155d90b00 2024-09-11T00:15:34.497 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:34.495+0000 7fd7280ad740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fd7201adfd0 con 0x560155d8b790 2024-09-11T00:15:34.499 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:34.496+0000 7fd71d400640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560155d8e0c0 0x560155d89e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:15:34.499 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:34.496+0000 7fd71e800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560155d90b00 0x560155d871e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:15:34.499 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:34.496+0000 7fd71d400640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560155d8e0c0 0x560155d89e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:50272/0 (socket says 172.21.15.12:50272) 2024-09-11T00:15:34.499 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:34.496+0000 7fd71d400640 1 -- 172.21.15.12:0/3668772649 learned_addr learned my addr 172.21.15.12:0/3668772649 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:15:34.499 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:34.496+0000 7fd71de00640 1 --2- 172.21.15.12:0/3668772649 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560155d8b790 0x560155d89930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:15:34.499 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:34.496+0000 7fd71e800640 1 -- 172.21.15.12:0/3668772649 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560155d8b790 msgr2=0x560155d89930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:15:34.499 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:34.496+0000 7fd71e800640 1 --2- 172.21.15.12:0/3668772649 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560155d8b790 0x560155d89930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:15:34.499 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:34.496+0000 7fd71e800640 1 -- 172.21.15.12:0/3668772649 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560155d8e0c0 msgr2=0x560155d89e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:15:34.500 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:34.496+0000 7fd71e800640 1 --2- 172.21.15.12:0/3668772649 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560155d8e0c0 0x560155d89e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:15:34.500 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:34.496+0000 7fd71e800640 1 -- 172.21.15.12:0/3668772649 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560155da3270 con 0x560155d90b00 2024-09-11T00:15:34.500 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:34.496+0000 7fd71e800640 1 --2- 172.21.15.12:0/3668772649 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560155d90b00 0x560155d871e0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fd718009270 tx=0x7fd7180b0e10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3508bf41ce66c8c7 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:15:34.500 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:34.497+0000 7fd70aa00640 1 -- 172.21.15.12:0/3668772649 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd71801f070 con 0x560155d90b00 2024-09-11T00:15:34.500 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:34.497+0000 7fd70aa00640 1 -- 172.21.15.12:0/3668772649 <== mon.2 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd7180ac1b0 con 0x560155d90b00 2024-09-11T00:15:34.500 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:34.497+0000 7fd70aa00640 1 -- 172.21.15.12:0/3668772649 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7180cdb90 con 0x560155d90b00 2024-09-11T00:15:34.500 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:34.497+0000 7fd7280ad740 1 -- 172.21.15.12:0/3668772649 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560155d90b00 msgr2=0x560155d871e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:15:34.500 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:34.497+0000 7fd7280ad740 1 --2- 172.21.15.12:0/3668772649 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560155d90b00 0x560155d871e0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fd718009270 tx=0x7fd7180b0e10 comp rx=0 tx=0).stop 2024-09-11T00:15:34.500 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:34.497+0000 7fd7280ad740 1 -- 172.21.15.12:0/3668772649 shutdown_connections 2024-09-11T00:15:34.500 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:34.497+0000 7fd7280ad740 1 --2- 172.21.15.12:0/3668772649 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560155d90b00 0x560155d871e0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:15:34.500 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:34.497+0000 7fd7280ad740 1 --2- 172.21.15.12:0/3668772649 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560155d8e0c0 0x560155d89e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:15:34.500 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:34.497+0000 7fd7280ad740 1 --2- 172.21.15.12:0/3668772649 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560155d8b790 0x560155d89930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:15:34.501 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:34.497+0000 7fd7280ad740 1 -- 172.21.15.12:0/3668772649 >> 172.21.15.12:0/3668772649 conn(0x560155d868e0 msgr2=0x560155d8ac20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:15:34.501 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:34.497+0000 7fd7280ad740 1 -- 172.21.15.12:0/3668772649 shutdown_connections 2024-09-11T00:15:34.501 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:34.497+0000 7fd7280ad740 1 -- 172.21.15.12:0/3668772649 wait complete. 2024-09-11T00:15:34.501 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:34.498+0000 7fd7280ad740 1 -- start start 2024-09-11T00:15:34.501 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:34.498+0000 7fd7280ad740 1 Processor -- start 2024-09-11T00:15:34.503 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:34.498+0000 7fd7280ad740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560155d8b790 0x560155d9cb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:15:34.503 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:34.498+0000 7fd7280ad740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560155d8e0c0 0x560155d9b910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:15:34.503 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:34.498+0000 7fd7280ad740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560155d90b00 0x560155d9c000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:15:34.503 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:34.498+0000 7fd7280ad740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fd7201aed70 con 0x560155d8b790 2024-09-11T00:15:34.503 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:34.498+0000 7fd7280ad740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fd7201ae6a0 con 0x560155d90b00 2024-09-11T00:15:34.503 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:34.498+0000 7fd7280ad740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fd7201ab8e0 con 0x560155d8e0c0 2024-09-11T00:15:34.503 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:34.499+0000 7fd7280ad740 10 client.0 _ll_register_callbacks cb 0x7fd7180a88a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:15:34.503 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:34.499+0000 7fd71de00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560155d8b790 0x560155d9cb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:15:34.504 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:34.499+0000 7fd71de00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560155d8b790 0x560155d9cb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:50278/0 (socket says 172.21.15.12:50278) 2024-09-11T00:15:34.504 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:34.499+0000 7fd71de00640 1 -- 172.21.15.12:0/373427476 learned_addr learned my addr 172.21.15.12:0/373427476 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:15:34.504 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:34.499+0000 7fd71e800640 1 --2- 172.21.15.12:0/373427476 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560155d90b00 0x560155d9c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:15:34.504 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:34.499+0000 7fd71de00640 1 -- 172.21.15.12:0/373427476 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560155d8e0c0 msgr2=0x560155d9b910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:15:34.504 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:34.499+0000 7fd71de00640 1 --2- 172.21.15.12:0/373427476 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560155d8e0c0 0x560155d9b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:15:34.504 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:34.499+0000 7fd71de00640 1 -- 172.21.15.12:0/373427476 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560155d90b00 msgr2=0x560155d9c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:15:34.504 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:34.499+0000 7fd71de00640 1 --2- 172.21.15.12:0/373427476 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560155d90b00 0x560155d9c000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:15:34.504 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:34.499+0000 7fd71de00640 1 -- 172.21.15.12:0/373427476 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560155d9a3b0 con 0x560155d8b790 2024-09-11T00:15:34.504 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:34.500+0000 7fd71de00640 1 --2- 172.21.15.12:0/373427476 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560155d8b790 0x560155d9cb90 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x560155da34a0 tx=0x7fd710009670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7a634b8a309b8a4 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:15:34.504 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:34.500+0000 7fd6ffe00640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-09-11T00:15:34.504 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:34.500+0000 7fd6ffe00640 1 -- 172.21.15.12:0/373427476 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd710059060 con 0x560155d8b790 2024-09-11T00:15:34.504 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:34.500+0000 7fd6ffe00640 1 -- 172.21.15.12:0/373427476 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9208+0+0 (secure 0 0 0) 0x7fd7100315a0 con 0x560155d8b790 2024-09-11T00:15:34.505 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:34.500+0000 7fd7280ad740 1 -- 172.21.15.12:0/373427476 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd7180ac1b0 con 0x560155d8b790 2024-09-11T00:15:34.505 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:34.500+0000 7fd6ffe00640 1 -- 172.21.15.12:0/373427476 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7fd710035070 con 0x560155d8b790 2024-09-11T00:15:34.505 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:34.500+0000 7fd7280ad740 10 client.16470 fetch_fsmap learned FSMap version 41 2024-09-11T00:15:34.505 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:34.500+0000 7fd7280ad740 1 -- 172.21.15.12:0/373427476 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x560155e0da20 con 0x560155d8b790 2024-09-11T00:15:34.505 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:34.501+0000 7fd6ffe00640 1 -- 172.21.15.12:0/373427476 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd7180ac1b0 con 0x560155d8b790 2024-09-11T00:15:34.505 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:34.501+0000 7fd6ffe00640 1 -- 172.21.15.12:0/373427476 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 41) ==== 8370+0+0 (secure 0 0 0) 0x7fd71004c070 con 0x560155d8b790 2024-09-11T00:15:34.505 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:34.501+0000 7fd7280ad740 10 client.16470 fetch_fsmap finished waiting for FSMap version 41 2024-09-11T00:15:34.505 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:34.501+0000 7fd7280ad740 10 client.16470 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T00:15:34.505 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:34.501+0000 7fd7280ad740 20 client.16470 populate_metadata read hostname 'smithi012' 2024-09-11T00:15:34.505 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:34.501+0000 7fd7280ad740 1 --2- 172.21.15.12:0/373427476 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x560155e0e010 0x560155e10400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:15:34.505 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:34.501+0000 7fd7280ad740 4 client.16470 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T00:15:34.505 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:34.501+0000 7fd7280ad740 1 -- 172.21.15.12:0/373427476 --> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] -- command(tid 0: {"prefix": "flush journal"}) -- 0x560155cfc0b0 con 0x560155e0e010 2024-09-11T00:15:34.506 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:34.501+0000 7fd71d400640 1 --2- 172.21.15.12:0/373427476 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x560155e0e010 0x560155e10400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:15:34.506 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:34.502+0000 7fd71d400640 1 --2- 172.21.15.12:0/373427476 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x560155e0e010 0x560155e10400 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a6829d21782c482c server_cookie=e5cf8a8321e2d0bb in_seq=0 out_seq=0 2024-09-11T00:15:34.506 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:34.502+0000 7fd6ffe00640 10 client.16470 ms_handle_connect on v2:172.21.15.165:6838/501312984 2024-09-11T00:15:34.555 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:34.553+0000 7fd6ffe00640 1 -- 172.21.15.12:0/373427476 <== mds.0 v2:172.21.15.165:6838/501312984 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x560155cfc0b0 con 0x560155e0e010 2024-09-11T00:15:34.555 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:34.553+0000 7fd6ffe00640 10 client.16470 handle_command_reply: tid=0 2024-09-11T00:15:34.555 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:34.553+0000 7fd7280ad740 1 client.16470 shutdown 2024-09-11T00:15:34.555 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:34.553+0000 7fd7280ad740 1 -- 172.21.15.12:0/373427476 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560155d8b790 msgr2=0x560155d9cb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:15:34.555 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:34.553+0000 7fd7280ad740 1 --2- 172.21.15.12:0/373427476 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560155d8b790 0x560155d9cb90 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x560155da34a0 tx=0x7fd710009670 comp rx=0 tx=0).stop 2024-09-11T00:15:34.555 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:34.553+0000 7fd7280ad740 1 -- 172.21.15.12:0/373427476 shutdown_connections 2024-09-11T00:15:34.555 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:34.553+0000 7fd7280ad740 1 --2- 172.21.15.12:0/373427476 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x560155e0e010 0x560155e10400 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:15:34.555 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:34.553+0000 7fd7280ad740 1 --2- 172.21.15.12:0/373427476 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560155d90b00 0x560155d9c000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:15:34.555 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:34.553+0000 7fd6ffe00640 0 client.16470 ms_handle_reset on v2:172.21.15.165:6838/501312984 2024-09-11T00:15:34.556 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:34.553+0000 7fd7280ad740 1 --2- 172.21.15.12:0/373427476 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560155d8e0c0 0x560155d9b910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:15:34.556 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:34.553+0000 7fd7280ad740 1 --2- 172.21.15.12:0/373427476 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560155d8b790 0x560155d9cb90 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:15:34.556 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:34.553+0000 7fd7280ad740 1 -- 172.21.15.12:0/373427476 >> 172.21.15.12:0/373427476 conn(0x560155d868e0 msgr2=0x560155d8ab90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:15:34.556 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:34.553+0000 7fd7280ad740 1 -- 172.21.15.12:0/373427476 shutdown_connections 2024-09-11T00:15:34.556 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:34.554+0000 7fd7280ad740 1 -- 172.21.15.12:0/373427476 wait complete. 2024-09-11T00:15:34.556 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:34.554+0000 7fd7280ad740 20 client.16470 trim_cache size 0 max 16384 2024-09-11T00:15:34.556 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:{ 2024-09-11T00:15:34.557 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "message": "", 2024-09-11T00:15:34.557 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "return_code": 0 2024-09-11T00:15:34.557 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:} 2024-09-11T00:15:34.557 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:34.554+0000 7fd725200640 1 -- 172.21.15.12:0/3524160410 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fd7040b5360 msgr2=0x7fd7040b7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:15:34.558 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:34.554+0000 7fd725200640 1 --2- 172.21.15.12:0/3524160410 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fd7040b5360 0x7fd7040b7820 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fd70c009fd0 tx=0x7fd70c0046c0 comp rx=0 tx=0).stop 2024-09-11T00:15:34.558 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:34.554+0000 7fd725200640 1 -- 172.21.15.12:0/3524160410 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd7201066b0 msgr2=0x7fd72019c250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:15:34.558 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:34.554+0000 7fd725200640 1 --2- 172.21.15.12:0/3524160410 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd7201066b0 0x7fd72019c250 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fd71800eaa0 tx=0x7fd71800ef60 comp rx=0 tx=0).stop 2024-09-11T00:15:34.558 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:34.554+0000 7fd725200640 1 -- 172.21.15.12:0/3524160410 shutdown_connections 2024-09-11T00:15:34.559 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:34.554+0000 7fd725200640 1 --2- 172.21.15.12:0/3524160410 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fd7040b5360 0x7fd7040b7820 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:15:34.559 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:34.554+0000 7fd725200640 1 --2- 172.21.15.12:0/3524160410 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd7201066b0 0x7fd72019c250 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:15:34.559 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:34.554+0000 7fd725200640 1 --2- 172.21.15.12:0/3524160410 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd720105cf0 0x7fd72019b620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:15:34.559 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:34.554+0000 7fd725200640 1 --2- 172.21.15.12:0/3524160410 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd720104ba0 0x7fd72019afd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:15:34.559 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:34.554+0000 7fd725200640 1 -- 172.21.15.12:0/3524160410 >> 172.21.15.12:0/3524160410 conn(0x7fd720100460 msgr2=0x7fd7201090e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:15:34.559 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:34.555+0000 7fd725200640 1 -- 172.21.15.12:0/3524160410 shutdown_connections 2024-09-11T00:15:34.559 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:34.555+0000 7fd725200640 1 -- 172.21.15.12:0/3524160410 wait complete. 2024-09-11T00:15:34.665 INFO:tasks.workunit.client.0.smithi012.stdout: 1 312804 3.52 MB/sec execute 331 sec latency 19832.701 ms 2024-09-11T00:15:35.665 INFO:tasks.workunit.client.0.smithi012.stdout: 1 312804 3.51 MB/sec execute 332 sec latency 20832.812 ms 2024-09-11T00:15:35.782 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:15:35 smithi165 ceph-mon[35356]: pgmap v654: 129 pgs: 129 active+clean; 78 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 1.1 MiB/s wr, 69 op/s 2024-09-11T00:15:35.813 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:15:35 smithi012 ceph-mon[30333]: pgmap v654: 129 pgs: 129 active+clean; 78 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 1.1 MiB/s wr, 69 op/s 2024-09-11T00:15:35.831 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:15:35 smithi178 ceph-mon[35919]: pgmap v654: 129 pgs: 129 active+clean; 78 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 1.1 MiB/s wr, 69 op/s 2024-09-11T00:15:36.665 INFO:tasks.workunit.client.0.smithi012.stdout: 1 312804 3.50 MB/sec execute 333 sec latency 21832.969 ms 2024-09-11T00:15:37.665 INFO:tasks.workunit.client.0.smithi012.stdout: 1 312804 3.49 MB/sec execute 334 sec latency 22833.060 ms 2024-09-11T00:15:37.782 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:15:37 smithi165 ceph-mon[35356]: pgmap v655: 129 pgs: 129 active+clean; 77 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 1.4 MiB/s wr, 92 op/s 2024-09-11T00:15:37.813 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:15:37 smithi012 ceph-mon[30333]: pgmap v655: 129 pgs: 129 active+clean; 77 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 1.4 MiB/s wr, 92 op/s 2024-09-11T00:15:37.831 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:15:37 smithi178 ceph-mon[35919]: pgmap v655: 129 pgs: 129 active+clean; 77 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 1.4 MiB/s wr, 92 op/s 2024-09-11T00:15:38.665 INFO:tasks.workunit.client.0.smithi012.stdout: 1 312804 3.48 MB/sec execute 335 sec latency 23833.195 ms 2024-09-11T00:15:39.569 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid ff3ad74a-6fcf-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T00:15:39.665 INFO:tasks.workunit.client.0.smithi012.stdout: 1 312804 3.47 MB/sec execute 336 sec latency 24833.277 ms 2024-09-11T00:15:39.782 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:15:39 smithi165 ceph-mon[35356]: pgmap v656: 129 pgs: 129 active+clean; 76 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 1.1 MiB/s wr, 67 op/s 2024-09-11T00:15:39.813 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:15:39 smithi012 ceph-mon[30333]: pgmap v656: 129 pgs: 129 active+clean; 76 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 1.1 MiB/s wr, 67 op/s 2024-09-11T00:15:39.830 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:15:39 smithi178 ceph-mon[35919]: pgmap v656: 129 pgs: 129 active+clean; 76 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 1.1 MiB/s wr, 67 op/s 2024-09-11T00:15:39.954 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:15:40.665 INFO:tasks.workunit.client.0.smithi012.stdout: 1 312804 3.46 MB/sec execute 337 sec latency 25833.332 ms 2024-09-11T00:15:40.680 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:40.679+0000 7f61c1a00640 1 -- 172.21.15.12:0/617797903 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f61bc10f3e0 msgr2=0x7f61bc1117d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:15:40.680 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:40.679+0000 7f61c1a00640 1 --2- 172.21.15.12:0/617797903 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f61bc10f3e0 0x7f61bc1117d0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f61b4009900 tx=0x7f61b402a410 comp rx=0 tx=0).stop 2024-09-11T00:15:40.681 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:40.679+0000 7f61c1a00640 1 -- 172.21.15.12:0/617797903 shutdown_connections 2024-09-11T00:15:40.681 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:40.679+0000 7f61c1a00640 1 --2- 172.21.15.12:0/617797903 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f61bc10f3e0 0x7f61bc1117d0 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:15:40.681 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:40.679+0000 7f61c1a00640 1 --2- 172.21.15.12:0/617797903 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f61bc075f40 0x7f61bc10ee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:15:40.681 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:40.679+0000 7f61c1a00640 1 --2- 172.21.15.12:0/617797903 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f61bc075540 0x7f61bc075920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:15:40.681 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:40.679+0000 7f61c1a00640 1 -- 172.21.15.12:0/617797903 >> 172.21.15.12:0/617797903 conn(0x7f61bc0fe830 msgr2=0x7f61bc100c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:15:40.681 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:40.679+0000 7f61c1a00640 1 -- 172.21.15.12:0/617797903 shutdown_connections 2024-09-11T00:15:40.681 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:40.679+0000 7f61c1a00640 1 -- 172.21.15.12:0/617797903 wait complete. 2024-09-11T00:15:40.682 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:40.680+0000 7f61c1a00640 1 Processor -- start 2024-09-11T00:15:40.683 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:40.680+0000 7f61c1a00640 1 -- start start 2024-09-11T00:15:40.683 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:40.680+0000 7f61c1a00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f61bc075540 0x7f61bc19b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:15:40.683 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:40.680+0000 7f61c1a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f61bc075f40 0x7f61bc19b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:15:40.683 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:40.680+0000 7f61c1a00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f61bc10f3e0 0x7f61bc1a0d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:15:40.683 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:40.680+0000 7f61c1a00640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f61bc114fd0 con 0x7f61bc075540 2024-09-11T00:15:40.683 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:40.680+0000 7f61c1a00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f61bc114e50 con 0x7f61bc075f40 2024-09-11T00:15:40.683 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:40.680+0000 7f61c1a00640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f61bc115150 con 0x7f61bc10f3e0 2024-09-11T00:15:40.683 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:40.681+0000 7f61baa00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f61bc075540 0x7f61bc19b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:15:40.683 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:40.681+0000 7f61bb400640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f61bc10f3e0 0x7f61bc1a0d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:15:40.683 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:40.681+0000 7f61bb400640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f61bc10f3e0 0x7f61bc1a0d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.12:57906/0 (socket says 172.21.15.12:57906) 2024-09-11T00:15:40.683 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:40.681+0000 7f61bb400640 1 -- 172.21.15.12:0/506961500 learned_addr learned my addr 172.21.15.12:0/506961500 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:15:40.683 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:40.681+0000 7f61ba000640 1 --2- 172.21.15.12:0/506961500 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f61bc075f40 0x7f61bc19b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:15:40.683 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:40.681+0000 7f61bb400640 1 -- 172.21.15.12:0/506961500 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f61bc075f40 msgr2=0x7f61bc19b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:15:40.683 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:40.681+0000 7f61bb400640 1 --2- 172.21.15.12:0/506961500 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f61bc075f40 0x7f61bc19b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:15:40.684 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:40.681+0000 7f61bb400640 1 -- 172.21.15.12:0/506961500 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f61bc075540 msgr2=0x7f61bc19b480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:15:40.684 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:40.681+0000 7f61bb400640 1 --2- 172.21.15.12:0/506961500 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f61bc075540 0x7f61bc19b480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:15:40.684 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:40.681+0000 7f61bb400640 1 -- 172.21.15.12:0/506961500 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61bc1a1530 con 0x7f61bc10f3e0 2024-09-11T00:15:40.684 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:40.682+0000 7f61bb400640 1 --2- 172.21.15.12:0/506961500 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f61bc10f3e0 0x7f61bc1a0d60 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f61b402a930 tx=0x7f61b4003c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:15:40.685 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:40.682+0000 7f61b3400640 1 -- 172.21.15.12:0/506961500 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61b403f070 con 0x7f61bc10f3e0 2024-09-11T00:15:40.685 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:40.682+0000 7f61b3400640 1 -- 172.21.15.12:0/506961500 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f61b4031b50 con 0x7f61bc10f3e0 2024-09-11T00:15:40.685 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:40.682+0000 7f61c1a00640 1 -- 172.21.15.12:0/506961500 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f61bc1a17c0 con 0x7f61bc10f3e0 2024-09-11T00:15:40.685 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:40.682+0000 7f61b3400640 1 -- 172.21.15.12:0/506961500 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61b4035030 con 0x7f61bc10f3e0 2024-09-11T00:15:40.685 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:40.682+0000 7f61c1a00640 1 -- 172.21.15.12:0/506961500 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61bc1a7ee0 con 0x7f61bc10f3e0 2024-09-11T00:15:40.685 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:40.683+0000 7f61c1a00640 1 -- 172.21.15.12:0/506961500 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f61bc075920 con 0x7f61bc10f3e0 2024-09-11T00:15:40.686 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:40.685+0000 7f61b3400640 1 -- 172.21.15.12:0/506961500 <== mon.1 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7f61b403b460 con 0x7f61bc10f3e0 2024-09-11T00:15:40.688 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:40.687+0000 7f61b3400640 1 --2- 172.21.15.12:0/506961500 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f61a00b52e0 0x7f61a00b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:15:40.688 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:40.687+0000 7f61b3400640 1 -- 172.21.15.12:0/506961500 <== mon.1 v2:172.21.15.178:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9208+0+0 (secure 0 0 0) 0x7f61b4106c00 con 0x7f61bc10f3e0 2024-09-11T00:15:40.688 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:40.687+0000 7f61baa00640 1 --2- 172.21.15.12:0/506961500 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f61a00b52e0 0x7f61a00b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:15:40.689 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:40.688+0000 7f61baa00640 1 --2- 172.21.15.12:0/506961500 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f61a00b52e0 0x7f61a00b77a0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f61ac005fd0 tx=0x7f61ac009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:15:40.690 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:40.689+0000 7f61b3400640 1 -- 172.21.15.12:0/506961500 <== mon.1 v2:172.21.15.178:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f61b40cd040 con 0x7f61bc10f3e0 2024-09-11T00:15:40.927 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:40.926+0000 7f61c1a00640 1 -- 172.21.15.12:0/506961500 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f61bc1a8140 con 0x7f61bc10f3e0 2024-09-11T00:15:40.928 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:40.927+0000 7f61b3400640 1 -- 172.21.15.12:0/506961500 <== mon.1 v2:172.21.15.178:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 41 v41) ==== 94+0+11965 (secure 0 0 0) 0x7f61b40d1a70 con 0x7f61bc10f3e0 2024-09-11T00:15:40.928 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-11T00:15:40.928 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":41,"btime":"2024-09-11T00:14:31:349726+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14577,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6835/447171591","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":447171591},{"type":"v1","addr":"172.21.15.12:6835","nonce":447171591}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14589,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/901486040","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":901486040},{"type":"v1","addr":"172.21.15.12:6837","nonce":901486040}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":41,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T00:04:04.739881+0000","modified":"2024-09-11T00:14:30.594242+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24674,"mds_1":24643,"mds_2":24638,"mds_3":24637,"mds_4":24614},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14631":{"gid":14631,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.12:6841/3069919437","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6840","nonce":3069919437},{"type":"v1","addr":"172.21.15.12:6841","nonce":3069919437}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24562":{"gid":24562,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6833/1141636112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":1141636112},{"type":"v1","addr":"172.21.15.178:6833","nonce":1141636112}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24581":{"gid":24581,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6833/3111980970","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6832","nonce":3111980970},{"type":"v1","addr":"172.21.15.165:6833","nonce":3111980970}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24607":{"gid":24607,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6835/4084868287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":4084868287},{"type":"v1","addr":"172.21.15.178:6835","nonce":4084868287}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24614":{"gid":24614,"name":"e","rank":4,"incarnation":26,"state":"up:active","state_seq":21,"addr":"172.21.15.165:6835/2425395274","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":2425395274},{"type":"v1","addr":"172.21.15.165:6835","nonce":2425395274}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24637":{"gid":24637,"name":"i","rank":3,"incarnation":24,"state":"up:active","state_seq":18,"addr":"172.21.15.178:6839/70628097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6838","nonce":70628097},{"type":"v1","addr":"172.21.15.178:6839","nonce":70628097}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24638":{"gid":24638,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":18,"addr":"172.21.15.165:6837/6215088","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6836","nonce":6215088},{"type":"v1","addr":"172.21.15.165:6837","nonce":6215088}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24643":{"gid":24643,"name":"g","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.12:6839/3124093009","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6838","nonce":3124093009},{"type":"v1","addr":"172.21.15.12:6839","nonce":3124093009}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24674":{"gid":24674,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.165:6839/501312984","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6838","nonce":501312984},{"type":"v1","addr":"172.21.15.165:6839","nonce":501312984}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25672":{"gid":25672,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6837/308601741","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":308601741},{"type":"v1","addr":"172.21.15.178:6837","nonce":308601741}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24674,"qdb_cluster":[24614,24637,24638,24643,24674]},"id":1}]} 2024-09-11T00:15:40.929 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 41 2024-09-11T00:15:40.931 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:40.929+0000 7f61c1a00640 1 -- 172.21.15.12:0/506961500 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f61a00b52e0 msgr2=0x7f61a00b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:15:40.931 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:40.929+0000 7f61c1a00640 1 --2- 172.21.15.12:0/506961500 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f61a00b52e0 0x7f61a00b77a0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f61ac005fd0 tx=0x7f61ac009500 comp rx=0 tx=0).stop 2024-09-11T00:15:40.931 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:40.929+0000 7f61c1a00640 1 -- 172.21.15.12:0/506961500 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f61bc10f3e0 msgr2=0x7f61bc1a0d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:15:40.931 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:40.929+0000 7f61c1a00640 1 --2- 172.21.15.12:0/506961500 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f61bc10f3e0 0x7f61bc1a0d60 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f61b402a930 tx=0x7f61b4003c00 comp rx=0 tx=0).stop 2024-09-11T00:15:40.931 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:40.929+0000 7f61c1a00640 1 -- 172.21.15.12:0/506961500 shutdown_connections 2024-09-11T00:15:40.931 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:40.929+0000 7f61c1a00640 1 --2- 172.21.15.12:0/506961500 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f61a00b52e0 0x7f61a00b77a0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:15:40.931 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:40.929+0000 7f61c1a00640 1 --2- 172.21.15.12:0/506961500 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f61bc10f3e0 0x7f61bc1a0d60 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:15:40.931 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:40.929+0000 7f61c1a00640 1 --2- 172.21.15.12:0/506961500 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f61bc075f40 0x7f61bc19b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:15:40.931 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:40.929+0000 7f61c1a00640 1 --2- 172.21.15.12:0/506961500 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f61bc075540 0x7f61bc19b480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:15:40.932 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:40.929+0000 7f61c1a00640 1 -- 172.21.15.12:0/506961500 >> 172.21.15.12:0/506961500 conn(0x7f61bc0fe830 msgr2=0x7f61bc1115a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:15:40.932 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:40.930+0000 7f61c1a00640 1 -- 172.21.15.12:0/506961500 shutdown_connections 2024-09-11T00:15:40.932 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:40.930+0000 7f61c1a00640 1 -- 172.21.15.12:0/506961500 wait complete. 2024-09-11T00:15:41.665 INFO:tasks.workunit.client.0.smithi012.stdout: 1 312804 3.45 MB/sec execute 338 sec latency 26833.398 ms 2024-09-11T00:15:41.675 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24674 quiesce db --set-id e3bc5e2 --release --await' 2024-09-11T00:15:41.675 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid ff3ad74a-6fcf-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24674 quiesce db --set-id e3bc5e2 --release --await 2024-09-11T00:15:41.782 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:15:41 smithi165 ceph-mon[35356]: pgmap v657: 129 pgs: 129 active+clean; 74 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 1.2 MiB/s wr, 81 op/s 2024-09-11T00:15:41.782 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:15:41 smithi165 ceph-mon[35356]: from='client.? 172.21.15.12:0/506961500' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T00:15:41.813 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:15:41 smithi012 ceph-mon[30333]: pgmap v657: 129 pgs: 129 active+clean; 74 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 1.2 MiB/s wr, 81 op/s 2024-09-11T00:15:41.813 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:15:41 smithi012 ceph-mon[30333]: from='client.? 172.21.15.12:0/506961500' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T00:15:41.831 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:15:41 smithi178 ceph-mon[35919]: pgmap v657: 129 pgs: 129 active+clean; 74 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 1.2 MiB/s wr, 81 op/s 2024-09-11T00:15:41.831 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:15:41 smithi178 ceph-mon[35919]: from='client.? 172.21.15.12:0/506961500' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T00:15:42.054 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:15:42.666 INFO:tasks.workunit.client.0.smithi012.stdout: 1 312804 3.44 MB/sec execute 339 sec latency 27833.446 ms 2024-09-11T00:15:42.751 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:42.750+0000 7f3efb200640 1 -- 172.21.15.12:0/1269898427 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3efc108fa0 msgr2=0x7f3efc10f7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:15:42.751 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:42.750+0000 7f3efb200640 1 --2- 172.21.15.12:0/1269898427 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3efc108fa0 0x7f3efc10f7a0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f3ef4009ca0 tx=0x7f3ef402a8c0 comp rx=0 tx=0).stop 2024-09-11T00:15:42.752 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:42.750+0000 7f3efb200640 1 -- 172.21.15.12:0/1269898427 shutdown_connections 2024-09-11T00:15:42.752 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:42.750+0000 7f3efb200640 1 --2- 172.21.15.12:0/1269898427 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3efc108fa0 0x7f3efc10f7a0 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:15:42.752 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:42.750+0000 7f3efb200640 1 --2- 172.21.15.12:0/1269898427 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3efc108620 0x7f3efc108a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:15:42.752 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:42.750+0000 7f3efb200640 1 --2- 172.21.15.12:0/1269898427 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3efc102c10 0x7f3efc102ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:15:42.752 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:42.750+0000 7f3efb200640 1 -- 172.21.15.12:0/1269898427 >> 172.21.15.12:0/1269898427 conn(0x7f3efc0fe660 msgr2=0x7f3efc100a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:15:42.752 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:42.750+0000 7f3efb200640 1 -- 172.21.15.12:0/1269898427 shutdown_connections 2024-09-11T00:15:42.752 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:42.750+0000 7f3efb200640 1 -- 172.21.15.12:0/1269898427 wait complete. 2024-09-11T00:15:42.754 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:42.751+0000 7f3efb200640 1 Processor -- start 2024-09-11T00:15:42.754 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:42.751+0000 7f3efb200640 1 -- start start 2024-09-11T00:15:42.754 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:42.751+0000 7f3efb200640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3efc102c10 0x7f3efc19b1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:15:42.754 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:42.751+0000 7f3efb200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3efc108620 0x7f3efc19b6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:15:42.754 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:42.751+0000 7f3efb200640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3efc108fa0 0x7f3efc1a0a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:15:42.754 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:42.751+0000 7f3efb200640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f3efc112cb0 con 0x7f3efc102c10 2024-09-11T00:15:42.754 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:42.751+0000 7f3efb200640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f3efc112b30 con 0x7f3efc108620 2024-09-11T00:15:42.755 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:42.751+0000 7f3efb200640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f3efc112e30 con 0x7f3efc108fa0 2024-09-11T00:15:42.755 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:42.752+0000 7f3ef9e00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3efc102c10 0x7f3efc19b1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:15:42.755 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:42.752+0000 7f3ef9e00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3efc102c10 0x7f3efc19b1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:42846/0 (socket says 172.21.15.12:42846) 2024-09-11T00:15:42.755 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:42.752+0000 7f3ef9e00640 1 -- 172.21.15.12:0/3338626149 learned_addr learned my addr 172.21.15.12:0/3338626149 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:15:42.755 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:42.752+0000 7f3ef9400640 1 --2- 172.21.15.12:0/3338626149 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3efc108620 0x7f3efc19b6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:15:42.755 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:42.752+0000 7f3efa800640 1 --2- 172.21.15.12:0/3338626149 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3efc108fa0 0x7f3efc1a0a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:15:42.755 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:42.752+0000 7f3ef9e00640 1 -- 172.21.15.12:0/3338626149 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3efc108fa0 msgr2=0x7f3efc1a0a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:15:42.755 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:42.752+0000 7f3ef9e00640 1 --2- 172.21.15.12:0/3338626149 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3efc108fa0 0x7f3efc1a0a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:15:42.755 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:42.752+0000 7f3ef9e00640 1 -- 172.21.15.12:0/3338626149 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3efc108620 msgr2=0x7f3efc19b6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:15:42.755 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:42.752+0000 7f3ef9e00640 1 --2- 172.21.15.12:0/3338626149 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3efc108620 0x7f3efc19b6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:15:42.755 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:42.752+0000 7f3ef9e00640 1 -- 172.21.15.12:0/3338626149 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3efc1a1260 con 0x7f3efc102c10 2024-09-11T00:15:42.755 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:42.753+0000 7f3ef9e00640 1 --2- 172.21.15.12:0/3338626149 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3efc102c10 0x7f3efc19b1b0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f3eec00e9f0 tx=0x7f3eec00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:15:42.755 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:42.753+0000 7f3ef2a00640 1 -- 172.21.15.12:0/3338626149 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3eec00cd80 con 0x7f3efc102c10 2024-09-11T00:15:42.755 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:42.753+0000 7f3efb200640 1 -- 172.21.15.12:0/3338626149 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3efc1a1550 con 0x7f3efc102c10 2024-09-11T00:15:42.756 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:42.753+0000 7f3ef2a00640 1 -- 172.21.15.12:0/3338626149 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3eec0108e0 con 0x7f3efc102c10 2024-09-11T00:15:42.756 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:42.753+0000 7f3efb200640 1 -- 172.21.15.12:0/3338626149 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3efc1a7c10 con 0x7f3efc102c10 2024-09-11T00:15:42.756 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:42.753+0000 7f3ef2a00640 1 -- 172.21.15.12:0/3338626149 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3eec018980 con 0x7f3efc102c10 2024-09-11T00:15:42.757 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:42.756+0000 7f3f0366c740 1 Processor -- start 2024-09-11T00:15:42.757 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:42.756+0000 7f3f0366c740 1 -- start start 2024-09-11T00:15:42.758 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:42.757+0000 7f3f0366c740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557f7c4b61d0 0x557f7c4b65b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:15:42.758 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:42.757+0000 7f3ef2a00640 1 -- 172.21.15.12:0/3338626149 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7f3eec018b20 con 0x7f3efc102c10 2024-09-11T00:15:42.758 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:42.757+0000 7f3f0366c740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557f7c4b6af0 0x557f7c4c2540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:15:42.758 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:42.757+0000 7f3f0366c740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557f7c4c2a80 0x557f7c4c4e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:15:42.758 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:42.757+0000 7f3f0366c740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x557f7c42d290 con 0x557f7c4c2a80 2024-09-11T00:15:42.758 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:42.757+0000 7f3f0366c740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x557f7c41c6c0 con 0x557f7c4b61d0 2024-09-11T00:15:42.758 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:42.757+0000 7f3f0366c740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x557f7c4204d0 con 0x557f7c4b6af0 2024-09-11T00:15:42.759 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:42.757+0000 7f3efa800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557f7c4b61d0 0x557f7c4b65b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:15:42.760 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:42.757+0000 7f3ef9400640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557f7c4c2a80 0x557f7c4c4e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:15:42.760 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:42.757+0000 7f3efa800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557f7c4b61d0 0x557f7c4b65b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.12:44110/0 (socket says 172.21.15.12:44110) 2024-09-11T00:15:42.760 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:42.757+0000 7f3efa800640 1 -- 172.21.15.12:0/2210807171 learned_addr learned my addr 172.21.15.12:0/2210807171 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:15:42.760 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:42.757+0000 7f3ef9e00640 1 --2- 172.21.15.12:0/2210807171 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557f7c4b6af0 0x557f7c4c2540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:15:42.760 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:42.758+0000 7f3ef9400640 1 -- 172.21.15.12:0/2210807171 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557f7c4b6af0 msgr2=0x557f7c4c2540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:15:42.760 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:42.758+0000 7f3ef9400640 1 --2- 172.21.15.12:0/2210807171 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557f7c4b6af0 0x557f7c4c2540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:15:42.760 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:42.758+0000 7f3ef9400640 1 -- 172.21.15.12:0/2210807171 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557f7c4b61d0 msgr2=0x557f7c4b65b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:15:42.760 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:42.758+0000 7f3ef9400640 1 --2- 172.21.15.12:0/2210807171 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557f7c4b61d0 0x557f7c4b65b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:15:42.760 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:42.758+0000 7f3ef9400640 1 -- 172.21.15.12:0/2210807171 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557f7c3a9460 con 0x557f7c4c2a80 2024-09-11T00:15:42.760 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:42.758+0000 7f3ef9400640 1 --2- 172.21.15.12:0/2210807171 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557f7c4c2a80 0x557f7c4c4e70 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f3ee8005fd0 tx=0x7f3ee80107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb54838745bfdd8c server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:15:42.760 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:42.758+0000 7f3ee7000640 1 -- 172.21.15.12:0/2210807171 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ee801c480 con 0x557f7c4c2a80 2024-09-11T00:15:42.760 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:42.758+0000 7f3ee7000640 1 -- 172.21.15.12:0/2210807171 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3ee801cad0 con 0x557f7c4c2a80 2024-09-11T00:15:42.761 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:42.758+0000 7f3ee7000640 1 -- 172.21.15.12:0/2210807171 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ee8011bc0 con 0x557f7c4c2a80 2024-09-11T00:15:42.762 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:42.759+0000 7f3f0366c740 1 -- 172.21.15.12:0/2210807171 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557f7c4c2a80 msgr2=0x557f7c4c4e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:15:42.762 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:42.759+0000 7f3f0366c740 1 --2- 172.21.15.12:0/2210807171 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557f7c4c2a80 0x557f7c4c4e70 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f3ee8005fd0 tx=0x7f3ee80107a0 comp rx=0 tx=0).stop 2024-09-11T00:15:42.762 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:42.759+0000 7f3ef2a00640 1 --2- 172.21.15.12:0/3338626149 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f3ee00b5390 0x7f3ee00b7850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:15:42.762 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:42.759+0000 7f3f0366c740 1 -- 172.21.15.12:0/2210807171 shutdown_connections 2024-09-11T00:15:42.762 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:42.759+0000 7f3f0366c740 1 --2- 172.21.15.12:0/2210807171 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557f7c4c2a80 0x557f7c4c4e70 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:15:42.762 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:42.759+0000 7f3f0366c740 1 --2- 172.21.15.12:0/2210807171 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557f7c4b6af0 0x557f7c4c2540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:15:42.762 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:42.759+0000 7f3ef2a00640 1 -- 172.21.15.12:0/3338626149 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9208+0+0 (secure 0 0 0) 0x7f3eec0e6150 con 0x7f3efc102c10 2024-09-11T00:15:42.762 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:42.759+0000 7f3f0366c740 1 --2- 172.21.15.12:0/2210807171 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557f7c4b61d0 0x557f7c4b65b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:15:42.762 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:42.759+0000 7f3f0366c740 1 -- 172.21.15.12:0/2210807171 >> 172.21.15.12:0/2210807171 conn(0x557f7c4b4000 msgr2=0x557f7c4b51a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:15:42.762 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:42.760+0000 7f3efa800640 1 --2- 172.21.15.12:0/3338626149 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f3ee00b5390 0x7f3ee00b7850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:15:42.762 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:42.760+0000 7f3f0366c740 1 -- 172.21.15.12:0/2210807171 shutdown_connections 2024-09-11T00:15:42.763 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:42.760+0000 7f3f0366c740 1 -- 172.21.15.12:0/2210807171 wait complete. 2024-09-11T00:15:42.763 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:42.760+0000 7f3efa800640 1 --2- 172.21.15.12:0/3338626149 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f3ee00b5390 0x7f3ee00b7850 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f3ef402aeb0 tx=0x7f3ef4005d80 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:15:42.763 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:42.760+0000 7f3f0366c740 1 -- start start 2024-09-11T00:15:42.763 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:42.761+0000 7f3f0366c740 1 Processor -- start 2024-09-11T00:15:42.764 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:42.761+0000 7f3f0366c740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557f7c4b61d0 0x557f7c4bf9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:15:42.765 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:42.761+0000 7f3f0366c740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557f7c4b6af0 0x557f7c4bff10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:15:42.765 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:42.761+0000 7f3f0366c740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557f7c4c2a80 0x557f7c4c0450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:15:42.765 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:42.762+0000 7f3f0366c740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x557f7c27dbb0 con 0x557f7c4b6af0 2024-09-11T00:15:42.765 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:42.762+0000 7f3f0366c740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x557f7c3b76c0 con 0x557f7c4c2a80 2024-09-11T00:15:42.765 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:42.762+0000 7f3f0366c740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x557f7c4c5730 con 0x557f7c4b61d0 2024-09-11T00:15:42.765 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:42.762+0000 7f3f0366c740 10 client.0 _ll_register_callbacks cb 0x557f7c3d2490 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:15:42.765 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:42.762+0000 7f3ef9400640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557f7c4b6af0 0x557f7c4bff10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:15:42.765 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:42.762+0000 7f3ef9e00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557f7c4b61d0 0x557f7c4bf9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:15:42.765 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:42.762+0000 7f3ef9400640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557f7c4b6af0 0x557f7c4bff10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:42868/0 (socket says 172.21.15.12:42868) 2024-09-11T00:15:42.765 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:42.762+0000 7f3ef9400640 1 -- 172.21.15.12:0/2670040025 learned_addr learned my addr 172.21.15.12:0/2670040025 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:15:42.765 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:42.762+0000 7f3efa800640 1 --2- 172.21.15.12:0/2670040025 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557f7c4c2a80 0x557f7c4c0450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:15:42.765 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:42.762+0000 7f3ef9e00640 1 -- 172.21.15.12:0/2670040025 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557f7c4c2a80 msgr2=0x557f7c4c0450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:15:42.766 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:42.762+0000 7f3ef9e00640 1 --2- 172.21.15.12:0/2670040025 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557f7c4c2a80 0x557f7c4c0450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:15:42.766 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:42.762+0000 7f3ef9e00640 1 -- 172.21.15.12:0/2670040025 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557f7c4b6af0 msgr2=0x557f7c4bff10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:15:42.766 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:42.762+0000 7f3ef9e00640 1 --2- 172.21.15.12:0/2670040025 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557f7c4b6af0 0x557f7c4bff10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:15:42.766 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:42.762+0000 7f3ef9e00640 1 -- 172.21.15.12:0/2670040025 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557f7c17ab00 con 0x557f7c4b61d0 2024-09-11T00:15:42.766 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:42.763+0000 7f3ef9e00640 1 --2- 172.21.15.12:0/2670040025 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557f7c4b61d0 0x557f7c4bf9d0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x557f7c4b9350 tx=0x7f3eec0e9780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=93da23af5d53826f server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:15:42.766 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:42.763+0000 7f3edb400640 10 client.0 ms_handle_connect on v2:172.21.15.178:3300/0 2024-09-11T00:15:42.766 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:42.763+0000 7f3edb400640 1 -- 172.21.15.12:0/2670040025 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3eec0abee0 con 0x557f7c4b61d0 2024-09-11T00:15:42.766 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:42.763+0000 7f3edb400640 1 -- 172.21.15.12:0/2670040025 <== mon.1 v2:172.21.15.178:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9208+0+0 (secure 0 0 0) 0x7f3eec0e8310 con 0x557f7c4b61d0 2024-09-11T00:15:42.766 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:42.763+0000 7f3f0366c740 1 -- 172.21.15.12:0/2670040025 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3ee801cad0 con 0x557f7c4b61d0 2024-09-11T00:15:42.766 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:42.763+0000 7f3edb400640 1 -- 172.21.15.12:0/2670040025 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f3eec0abd00 con 0x557f7c4b61d0 2024-09-11T00:15:42.766 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:42.763+0000 7f3f0366c740 10 client.26485 fetch_fsmap learned FSMap version 41 2024-09-11T00:15:42.766 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:42.763+0000 7f3f0366c740 1 -- 172.21.15.12:0/2670040025 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x557f7c1f1960 con 0x557f7c4b61d0 2024-09-11T00:15:42.768 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:42.764+0000 7f3edb400640 1 -- 172.21.15.12:0/2670040025 <== mon.1 v2:172.21.15.178:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3ee801cad0 con 0x557f7c4b61d0 2024-09-11T00:15:42.768 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:42.764+0000 7f3edb400640 1 -- 172.21.15.12:0/2670040025 <== mon.1 v2:172.21.15.178:3300/0 5 ==== fsmap(e 41) ==== 8370+0+0 (secure 0 0 0) 0x7f3eec0e3ce0 con 0x557f7c4b61d0 2024-09-11T00:15:42.768 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:42.764+0000 7f3f0366c740 10 client.26485 fetch_fsmap finished waiting for FSMap version 41 2024-09-11T00:15:42.768 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:42.764+0000 7f3f0366c740 10 client.26485 resolve_mds: validated gid 24674 aka daemon mds.b 2024-09-11T00:15:42.768 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:42.764+0000 7f3f0366c740 20 client.26485 populate_metadata read hostname 'smithi012' 2024-09-11T00:15:42.768 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:42.764+0000 7f3f0366c740 1 --2- 172.21.15.12:0/2670040025 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x557f7c4d2520 0x557f7c4d49b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:15:42.768 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:42.764+0000 7f3f0366c740 4 client.26485 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T00:15:42.768 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:42.764+0000 7f3f0366c740 1 -- 172.21.15.12:0/2670040025 --> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557f7c42f1e0 con 0x557f7c4d2520 2024-09-11T00:15:42.769 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:42.765+0000 7f3ef9400640 1 --2- 172.21.15.12:0/2670040025 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x557f7c4d2520 0x557f7c4d49b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:15:42.769 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:42.765+0000 7f3ef9400640 1 --2- 172.21.15.12:0/2670040025 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x557f7c4d2520 0x557f7c4d49b0 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f8de81b54cb53061 server_cookie=633910a5e2052e5c in_seq=0 out_seq=0 2024-09-11T00:15:42.769 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:42.765+0000 7f3edb400640 10 client.26485 ms_handle_connect on v2:172.21.15.165:6838/501312984 2024-09-11T00:15:42.769 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:42.766+0000 7f3edb400640 1 -- 172.21.15.12:0/2670040025 <== mds.0 v2:172.21.15.165:6838/501312984 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x557f7c42f1e0 con 0x557f7c4d2520 2024-09-11T00:15:42.769 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:42.766+0000 7f3edb400640 10 client.26485 handle_command_reply: tid=0 2024-09-11T00:15:42.769 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:42.766+0000 7f3f0366c740 1 client.26485 shutdown 2024-09-11T00:15:42.769 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:42.766+0000 7f3f0366c740 1 -- 172.21.15.12:0/2670040025 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557f7c4b61d0 msgr2=0x557f7c4bf9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:15:42.769 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:42.766+0000 7f3f0366c740 1 --2- 172.21.15.12:0/2670040025 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557f7c4b61d0 0x557f7c4bf9d0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x557f7c4b9350 tx=0x7f3eec0e9780 comp rx=0 tx=0).stop 2024-09-11T00:15:42.769 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:42.766+0000 7f3f0366c740 1 -- 172.21.15.12:0/2670040025 shutdown_connections 2024-09-11T00:15:42.769 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:42.766+0000 7f3f0366c740 1 --2- 172.21.15.12:0/2670040025 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x557f7c4d2520 0x557f7c4d49b0 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:15:42.769 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:42.766+0000 7f3f0366c740 1 --2- 172.21.15.12:0/2670040025 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557f7c4c2a80 0x557f7c4c0450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:15:42.769 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:42.766+0000 7f3f0366c740 1 --2- 172.21.15.12:0/2670040025 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557f7c4b6af0 0x557f7c4bff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:15:42.769 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:42.766+0000 7f3edb400640 0 client.26485 ms_handle_reset on v2:172.21.15.165:6838/501312984 2024-09-11T00:15:42.770 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:42.766+0000 7f3f0366c740 1 --2- 172.21.15.12:0/2670040025 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557f7c4b61d0 0x557f7c4bf9d0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:15:42.770 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:42.766+0000 7f3f0366c740 1 -- 172.21.15.12:0/2670040025 >> 172.21.15.12:0/2670040025 conn(0x557f7c4b4000 msgr2=0x557f7c4b9ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:15:42.770 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:42.767+0000 7f3f0366c740 1 -- 172.21.15.12:0/2670040025 shutdown_connections 2024-09-11T00:15:42.770 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:42.767+0000 7f3f0366c740 1 -- 172.21.15.12:0/2670040025 wait complete. 2024-09-11T00:15:42.770 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:42.767+0000 7f3f0366c740 20 client.26485 trim_cache size 0 max 16384 2024-09-11T00:15:42.799 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:42.798+0000 7f3f0366c740 1 Processor -- start 2024-09-11T00:15:42.799 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:42.798+0000 7f3f0366c740 1 -- start start 2024-09-11T00:15:42.799 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:42.798+0000 7f3f0366c740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557f7c4b4000 0x557f7c4c1ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:15:42.800 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:42.798+0000 7f3f0366c740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557f7c4b61d0 0x557f7c4da2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:15:42.800 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:42.798+0000 7f3f0366c740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557f7c4b6af0 0x557f7c4da800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:15:42.800 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:42.798+0000 7f3f0366c740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f3edc00fc30 con 0x557f7c4b61d0 2024-09-11T00:15:42.800 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:42.798+0000 7f3f0366c740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f3edc011770 con 0x557f7c4b6af0 2024-09-11T00:15:42.800 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:42.798+0000 7f3f0366c740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f3edc010300 con 0x557f7c4b4000 2024-09-11T00:15:42.801 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:42.799+0000 7f3ef9400640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557f7c4b61d0 0x557f7c4da2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:15:42.801 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:42.799+0000 7f3ef9400640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557f7c4b61d0 0x557f7c4da2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:42880/0 (socket says 172.21.15.12:42880) 2024-09-11T00:15:42.801 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:42.799+0000 7f3ef9e00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557f7c4b4000 0x557f7c4c1ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:15:42.801 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:42.799+0000 7f3ef9400640 1 -- 172.21.15.12:0/1830834888 learned_addr learned my addr 172.21.15.12:0/1830834888 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:15:42.801 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:42.799+0000 7f3efa800640 1 --2- 172.21.15.12:0/1830834888 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557f7c4b6af0 0x557f7c4da800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:15:42.801 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:42.799+0000 7f3ef9400640 1 -- 172.21.15.12:0/1830834888 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557f7c4b4000 msgr2=0x557f7c4c1ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:15:42.801 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:42.799+0000 7f3ef9400640 1 --2- 172.21.15.12:0/1830834888 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557f7c4b4000 0x557f7c4c1ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:15:42.801 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:42.799+0000 7f3ef9400640 1 -- 172.21.15.12:0/1830834888 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557f7c4b6af0 msgr2=0x557f7c4da800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:15:42.802 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:42.799+0000 7f3ef9400640 1 --2- 172.21.15.12:0/1830834888 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557f7c4b6af0 0x557f7c4da800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:15:42.802 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:42.799+0000 7f3ef9400640 1 -- 172.21.15.12:0/1830834888 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557f7c4bded0 con 0x557f7c4b61d0 2024-09-11T00:15:42.802 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:42.800+0000 7f3ef9400640 1 --2- 172.21.15.12:0/1830834888 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557f7c4b61d0 0x557f7c4da2c0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f3ee80116f0 tx=0x7f3ee8011680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=868e1b321d00c804 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:15:42.802 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:42.800+0000 7f3ee6600640 1 -- 172.21.15.12:0/1830834888 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ee800bd20 con 0x557f7c4b61d0 2024-09-11T00:15:42.802 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:42.800+0000 7f3ee6600640 1 -- 172.21.15.12:0/1830834888 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3ee8005040 con 0x557f7c4b61d0 2024-09-11T00:15:42.802 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:42.800+0000 7f3ee6600640 1 -- 172.21.15.12:0/1830834888 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ee801c6e0 con 0x557f7c4b61d0 2024-09-11T00:15:42.803 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:42.801+0000 7f3f0366c740 1 -- 172.21.15.12:0/1830834888 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557f7c4b61d0 msgr2=0x557f7c4da2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:15:42.803 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:42.801+0000 7f3f0366c740 1 --2- 172.21.15.12:0/1830834888 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557f7c4b61d0 0x557f7c4da2c0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f3ee80116f0 tx=0x7f3ee8011680 comp rx=0 tx=0).stop 2024-09-11T00:15:42.803 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:42.801+0000 7f3f0366c740 1 -- 172.21.15.12:0/1830834888 shutdown_connections 2024-09-11T00:15:42.803 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:42.801+0000 7f3f0366c740 1 --2- 172.21.15.12:0/1830834888 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557f7c4b6af0 0x557f7c4da800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:15:42.803 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:42.801+0000 7f3f0366c740 1 --2- 172.21.15.12:0/1830834888 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557f7c4b61d0 0x557f7c4da2c0 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:15:42.803 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:42.801+0000 7f3f0366c740 1 --2- 172.21.15.12:0/1830834888 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557f7c4b4000 0x557f7c4c1ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:15:42.803 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:42.801+0000 7f3f0366c740 1 -- 172.21.15.12:0/1830834888 >> 172.21.15.12:0/1830834888 conn(0x557f7c4bd4d0 msgr2=0x557f7c4b4ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:15:42.803 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:42.801+0000 7f3f0366c740 1 -- 172.21.15.12:0/1830834888 shutdown_connections 2024-09-11T00:15:42.803 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:42.801+0000 7f3f0366c740 1 -- 172.21.15.12:0/1830834888 wait complete. 2024-09-11T00:15:42.804 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:42.802+0000 7f3f0366c740 1 -- start start 2024-09-11T00:15:42.804 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:42.802+0000 7f3f0366c740 1 Processor -- start 2024-09-11T00:15:42.805 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:42.803+0000 7f3f0366c740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557f7c4b4000 0x557f7c4cb3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:15:42.805 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:42.803+0000 7f3f0366c740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557f7c4b61d0 0x557f7c4c5a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:15:42.805 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:42.803+0000 7f3f0366c740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557f7c4b6af0 0x557f7c4c61a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:15:42.805 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:42.803+0000 7f3f0366c740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f3edc0110a0 con 0x557f7c4b6af0 2024-09-11T00:15:42.805 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:42.803+0000 7f3f0366c740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f3edc0109d0 con 0x557f7c4b4000 2024-09-11T00:15:42.805 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:42.803+0000 7f3f0366c740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f3edc00dc10 con 0x557f7c4b61d0 2024-09-11T00:15:42.805 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:42.803+0000 7f3f0366c740 10 client.0 _ll_register_callbacks cb 0x7f3eec0e9880 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:15:42.806 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:42.803+0000 7f3efa800640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557f7c4b6af0 0x557f7c4c61a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:15:42.806 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:42.803+0000 7f3ef9400640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557f7c4b61d0 0x557f7c4c5a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:15:42.806 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:42.803+0000 7f3ef9400640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557f7c4b61d0 0x557f7c4c5a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.12:57982/0 (socket says 172.21.15.12:57982) 2024-09-11T00:15:42.806 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:42.803+0000 7f3efa800640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557f7c4b6af0 0x557f7c4c61a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:42896/0 (socket says 172.21.15.12:42896) 2024-09-11T00:15:42.806 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:42.803+0000 7f3ef9400640 1 -- 172.21.15.12:0/1931695140 learned_addr learned my addr 172.21.15.12:0/1931695140 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:15:42.806 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:42.803+0000 7f3ef9e00640 1 --2- 172.21.15.12:0/1931695140 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557f7c4b4000 0x557f7c4cb3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:15:42.806 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:42.804+0000 7f3ef9e00640 1 -- 172.21.15.12:0/1931695140 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557f7c4b61d0 msgr2=0x557f7c4c5a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:15:42.806 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:42.804+0000 7f3ef9e00640 1 --2- 172.21.15.12:0/1931695140 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557f7c4b61d0 0x557f7c4c5a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:15:42.806 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:42.804+0000 7f3ef9e00640 1 -- 172.21.15.12:0/1931695140 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557f7c4b6af0 msgr2=0x557f7c4c61a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:15:42.806 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:42.804+0000 7f3ef9e00640 1 --2- 172.21.15.12:0/1931695140 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557f7c4b6af0 0x557f7c4c61a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:15:42.806 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:42.804+0000 7f3ef9e00640 1 -- 172.21.15.12:0/1931695140 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557f7c53f4c0 con 0x557f7c4b4000 2024-09-11T00:15:42.807 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:42.804+0000 7f3ef9400640 1 --2- 172.21.15.12:0/1931695140 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557f7c4b61d0 0x557f7c4c5a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:15:42.807 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:42.804+0000 7f3efa800640 1 --2- 172.21.15.12:0/1931695140 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557f7c4b6af0 0x557f7c4c61a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:15:42.807 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:42.804+0000 7f3ef9e00640 1 --2- 172.21.15.12:0/1931695140 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557f7c4b4000 0x557f7c4cb3f0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f3eec023040 tx=0x7f3eec010660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3c53b9ebc8584366 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:15:42.807 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:42.804+0000 7f3edb400640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-09-11T00:15:42.808 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:42.804+0000 7f3edb400640 1 -- 172.21.15.12:0/1931695140 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3eec0abee0 con 0x557f7c4b4000 2024-09-11T00:15:42.808 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:42.804+0000 7f3edb400640 1 -- 172.21.15.12:0/1931695140 <== mon.2 v2:172.21.15.165:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9208+0+0 (secure 0 0 0) 0x7f3eec0e9180 con 0x557f7c4b4000 2024-09-11T00:15:42.808 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:42.804+0000 7f3f0366c740 1 -- 172.21.15.12:0/1931695140 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3ee8005040 con 0x557f7c4b4000 2024-09-11T00:15:42.808 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:42.804+0000 7f3edb400640 1 -- 172.21.15.12:0/1931695140 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f3eec0a8be0 con 0x557f7c4b4000 2024-09-11T00:15:42.808 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:42.805+0000 7f3f0366c740 10 client.26498 fetch_fsmap learned FSMap version 41 2024-09-11T00:15:42.808 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:42.805+0000 7f3f0366c740 1 -- 172.21.15.12:0/1931695140 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x557f7c540750 con 0x557f7c4b4000 2024-09-11T00:15:42.809 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:42.805+0000 7f3edb400640 1 -- 172.21.15.12:0/1931695140 <== mon.2 v2:172.21.15.165:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3ee8005040 con 0x557f7c4b4000 2024-09-11T00:15:42.809 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:42.805+0000 7f3edb400640 1 -- 172.21.15.12:0/1931695140 <== mon.2 v2:172.21.15.165:3300/0 5 ==== fsmap(e 41) ==== 8370+0+0 (secure 0 0 0) 0x7f3eec0e3ce0 con 0x557f7c4b4000 2024-09-11T00:15:42.809 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:42.805+0000 7f3f0366c740 10 client.26498 fetch_fsmap finished waiting for FSMap version 41 2024-09-11T00:15:42.809 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:42.805+0000 7f3f0366c740 10 client.26498 resolve_mds: validated gid 24674 aka daemon mds.b 2024-09-11T00:15:42.809 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:42.805+0000 7f3f0366c740 20 client.26498 populate_metadata read hostname 'smithi012' 2024-09-11T00:15:42.809 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:42.805+0000 7f3f0366c740 1 --2- 172.21.15.12:0/1931695140 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x557f7c4c2a80 0x557f7c542d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:15:42.809 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:42.805+0000 7f3f0366c740 4 client.26498 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "e3bc5e2", "release": true, "await": true}] 2024-09-11T00:15:42.809 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:42.805+0000 7f3f0366c740 1 -- 172.21.15.12:0/1931695140 --> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] -- command(tid 0: {"prefix": "quiesce db", "set_id": "e3bc5e2", "release": true, "await": true}) -- 0x557f7c42ac90 con 0x557f7c4c2a80 2024-09-11T00:15:42.809 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:42.805+0000 7f3ef9400640 1 --2- 172.21.15.12:0/1931695140 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x557f7c4c2a80 0x557f7c542d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:15:42.809 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:42.806+0000 7f3ef9400640 1 --2- 172.21.15.12:0/1931695140 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x557f7c4c2a80 0x557f7c542d80 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bce81b5fe11cd8b6 server_cookie=65a12c765a58ec99 in_seq=0 out_seq=0 2024-09-11T00:15:42.809 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:42.806+0000 7f3edb400640 10 client.26498 ms_handle_connect on v2:172.21.15.165:6838/501312984 2024-09-11T00:15:42.816 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:42.814+0000 7f3edb400640 1 -- 172.21.15.12:0/1931695140 <== mds.0 v2:172.21.15.165:6838/501312984 1 ==== command_reply(tid 0: 0 ) ==== 8+0+590 (crc 0 0 0) 0x557f7c42ac90 con 0x557f7c4c2a80 2024-09-11T00:15:42.816 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:42.814+0000 7f3edb400640 10 client.26498 handle_command_reply: tid=0 2024-09-11T00:15:42.816 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:42.814+0000 7f3f0366c740 1 client.26498 shutdown 2024-09-11T00:15:42.816 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:42.815+0000 7f3f0366c740 1 -- 172.21.15.12:0/1931695140 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557f7c4b4000 msgr2=0x557f7c4cb3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:15:42.816 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:42.815+0000 7f3f0366c740 1 --2- 172.21.15.12:0/1931695140 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557f7c4b4000 0x557f7c4cb3f0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f3eec023040 tx=0x7f3eec010660 comp rx=0 tx=0).stop 2024-09-11T00:15:42.817 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:42.815+0000 7f3f0366c740 1 -- 172.21.15.12:0/1931695140 shutdown_connections 2024-09-11T00:15:42.817 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:42.815+0000 7f3f0366c740 1 --2- 172.21.15.12:0/1931695140 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x557f7c4c2a80 0x557f7c542d80 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:15:42.817 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:42.815+0000 7f3f0366c740 1 --2- 172.21.15.12:0/1931695140 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557f7c4b6af0 0x557f7c4c61a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:15:42.817 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:42.815+0000 7f3f0366c740 1 --2- 172.21.15.12:0/1931695140 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557f7c4b61d0 0x557f7c4c5a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:15:42.817 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:42.815+0000 7f3f0366c740 1 --2- 172.21.15.12:0/1931695140 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557f7c4b4000 0x557f7c4cb3f0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:15:42.817 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:42.815+0000 7f3edb400640 0 client.26498 ms_handle_reset on v2:172.21.15.165:6838/501312984 2024-09-11T00:15:42.817 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:42.815+0000 7f3f0366c740 1 -- 172.21.15.12:0/1931695140 >> 172.21.15.12:0/1931695140 conn(0x557f7c4bd4d0 msgr2=0x557f7c4b28a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:15:42.817 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:42.815+0000 7f3f0366c740 1 -- 172.21.15.12:0/1931695140 shutdown_connections 2024-09-11T00:15:42.817 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:42.815+0000 7f3f0366c740 1 -- 172.21.15.12:0/1931695140 wait complete. 2024-09-11T00:15:42.817 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:42.815+0000 7f3f0366c740 20 client.26498 trim_cache size 0 max 16384 2024-09-11T00:15:42.818 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-09-11T00:15:42.818 INFO:teuthology.orchestra.run.smithi012.stdout: "epoch": 41, 2024-09-11T00:15:42.818 INFO:teuthology.orchestra.run.smithi012.stdout: "leader": 24674, 2024-09-11T00:15:42.818 INFO:teuthology.orchestra.run.smithi012.stdout: "set_version": 28, 2024-09-11T00:15:42.818 INFO:teuthology.orchestra.run.smithi012.stdout: "sets": { 2024-09-11T00:15:42.818 INFO:teuthology.orchestra.run.smithi012.stdout: "e3bc5e2": { 2024-09-11T00:15:42.818 INFO:teuthology.orchestra.run.smithi012.stdout: "version": 28, 2024-09-11T00:15:42.818 INFO:teuthology.orchestra.run.smithi012.stdout: "age_ref": 0.0, 2024-09-11T00:15:42.818 INFO:teuthology.orchestra.run.smithi012.stdout: "state": { 2024-09-11T00:15:42.818 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "RELEASED", 2024-09-11T00:15:42.819 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 0.0 2024-09-11T00:15:42.819 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:15:42.819 INFO:teuthology.orchestra.run.smithi012.stdout: "timeout": 90.0, 2024-09-11T00:15:42.819 INFO:teuthology.orchestra.run.smithi012.stdout: "expiration": 148.4, 2024-09-11T00:15:42.819 INFO:teuthology.orchestra.run.smithi012.stdout: "members": { 2024-09-11T00:15:42.819 INFO:teuthology.orchestra.run.smithi012.stdout: "file:/": { 2024-09-11T00:15:42.819 INFO:teuthology.orchestra.run.smithi012.stdout: "excluded": false, 2024-09-11T00:15:42.819 INFO:teuthology.orchestra.run.smithi012.stdout: "state": { 2024-09-11T00:15:42.819 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "RELEASED", 2024-09-11T00:15:42.819 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 0.0 2024-09-11T00:15:42.819 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:15:42.819 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:15:42.819 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:15:42.820 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:15:42.820 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:15:42.820 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-09-11T00:15:42.820 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:42.816+0000 7f3efb200640 1 -- 172.21.15.12:0/3338626149 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f3ee00b5390 msgr2=0x7f3ee00b7850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:15:42.820 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:42.816+0000 7f3efb200640 1 --2- 172.21.15.12:0/3338626149 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f3ee00b5390 0x7f3ee00b7850 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f3ef402aeb0 tx=0x7f3ef4005d80 comp rx=0 tx=0).stop 2024-09-11T00:15:42.820 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:42.816+0000 7f3efb200640 1 -- 172.21.15.12:0/3338626149 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3efc102c10 msgr2=0x7f3efc19b1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:15:42.820 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:42.816+0000 7f3efb200640 1 --2- 172.21.15.12:0/3338626149 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3efc102c10 0x7f3efc19b1b0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f3eec00e9f0 tx=0x7f3eec00eeb0 comp rx=0 tx=0).stop 2024-09-11T00:15:42.821 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:42.816+0000 7f3efb200640 1 -- 172.21.15.12:0/3338626149 shutdown_connections 2024-09-11T00:15:42.821 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:42.816+0000 7f3efb200640 1 --2- 172.21.15.12:0/3338626149 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f3ee00b5390 0x7f3ee00b7850 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:15:42.821 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:42.816+0000 7f3efb200640 1 --2- 172.21.15.12:0/3338626149 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3efc108fa0 0x7f3efc1a0a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:15:42.821 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:42.816+0000 7f3efb200640 1 --2- 172.21.15.12:0/3338626149 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3efc108620 0x7f3efc19b6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:15:42.821 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:42.816+0000 7f3efb200640 1 --2- 172.21.15.12:0/3338626149 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3efc102c10 0x7f3efc19b1b0 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:15:42.821 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:42.816+0000 7f3efb200640 1 -- 172.21.15.12:0/3338626149 >> 172.21.15.12:0/3338626149 conn(0x7f3efc0fe660 msgr2=0x7f3efc075760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:15:42.821 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:42.816+0000 7f3efb200640 1 -- 172.21.15.12:0/3338626149 shutdown_connections 2024-09-11T00:15:42.821 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:15:42.816+0000 7f3efb200640 1 -- 172.21.15.12:0/3338626149 wait complete. 2024-09-11T00:15:43.666 INFO:tasks.workunit.client.0.smithi012.stdout: 1 314490 3.45 MB/sec execute 340 sec latency 27977.052 ms 2024-09-11T00:15:43.744 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:15:43 smithi012 ceph-mon[30333]: pgmap v658: 129 pgs: 129 active+clean; 74 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 930 KiB/s wr, 68 op/s 2024-09-11T00:15:43.747 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'e3bc5e2', total seconds elapsed: 32.6 2024-09-11T00:15:43.782 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:15:43 smithi165 ceph-mon[35356]: pgmap v658: 129 pgs: 129 active+clean; 74 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 930 KiB/s wr, 68 op/s 2024-09-11T00:15:43.830 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:15:43 smithi178 ceph-mon[35919]: pgmap v658: 129 pgs: 129 active+clean; 74 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 930 KiB/s wr, 68 op/s 2024-09-11T00:15:44.666 INFO:tasks.workunit.client.0.smithi012.stdout: 1 314763 3.44 MB/sec execute 341 sec latency 760.379 ms 2024-09-11T00:15:45.666 INFO:tasks.workunit.client.0.smithi012.stdout: 1 315968 3.46 MB/sec execute 342 sec latency 1006.842 ms 2024-09-11T00:15:45.813 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:15:45 smithi012 ceph-mon[30333]: pgmap v659: 129 pgs: 129 active+clean; 75 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 156 KiB/s rd, 859 KiB/s wr, 57 op/s 2024-09-11T00:15:45.831 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:15:45 smithi178 ceph-mon[35919]: pgmap v659: 129 pgs: 129 active+clean; 75 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 156 KiB/s rd, 859 KiB/s wr, 57 op/s 2024-09-11T00:15:46.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:15:45 smithi165 ceph-mon[35356]: pgmap v659: 129 pgs: 129 active+clean; 75 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 156 KiB/s rd, 859 KiB/s wr, 57 op/s 2024-09-11T00:15:46.666 INFO:tasks.workunit.client.0.smithi012.stdout: 1 317363 3.47 MB/sec execute 343 sec latency 12.108 ms 2024-09-11T00:15:47.666 INFO:tasks.workunit.client.0.smithi012.stdout: 1 319393 3.50 MB/sec execute 344 sec latency 27.349 ms 2024-09-11T00:15:47.686 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:47.685+0000 7f527b200640 1 -- 172.21.15.12:0/3365215467 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f527406b470 msgr2=0x7f5274105360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:15:47.686 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:47.685+0000 7f527b200640 1 --2- 172.21.15.12:0/3365215467 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f527406b470 0x7f5274105360 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f5268009900 tx=0x7f526802a3f0 comp rx=0 tx=0).stop 2024-09-11T00:15:47.687 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:47.685+0000 7f527b200640 1 -- 172.21.15.12:0/3365215467 shutdown_connections 2024-09-11T00:15:47.687 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:47.685+0000 7f527b200640 1 --2- 172.21.15.12:0/3365215467 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f52741084a0 0x7f527410a960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:15:47.687 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:47.685+0000 7f527b200640 1 --2- 172.21.15.12:0/3365215467 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f52741058f0 0x7f5274107d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:15:47.687 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:47.685+0000 7f527b200640 1 --2- 172.21.15.12:0/3365215467 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f527406b470 0x7f5274105360 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:15:47.687 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:47.685+0000 7f527b200640 1 -- 172.21.15.12:0/3365215467 >> 172.21.15.12:0/3365215467 conn(0x7f52740fcf30 msgr2=0x7f52740ff370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:15:47.687 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:47.685+0000 7f527b200640 1 -- 172.21.15.12:0/3365215467 shutdown_connections 2024-09-11T00:15:47.687 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:47.685+0000 7f527b200640 1 -- 172.21.15.12:0/3365215467 wait complete. 2024-09-11T00:15:47.689 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:47.686+0000 7f527b200640 1 Processor -- start 2024-09-11T00:15:47.689 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:47.686+0000 7f527b200640 1 -- start start 2024-09-11T00:15:47.690 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:47.686+0000 7f527b200640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f527406b470 0x7f527419ad60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:15:47.690 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:47.686+0000 7f527b200640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f52741058f0 0x7f527419b3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:15:47.690 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:47.686+0000 7f527b200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f52741084a0 0x7f527419bfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:15:47.690 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:47.686+0000 7f527b200640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f527410e420 con 0x7f52741058f0 2024-09-11T00:15:47.690 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:47.686+0000 7f527b200640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f527410e2a0 con 0x7f52741084a0 2024-09-11T00:15:47.690 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:47.686+0000 7f527b200640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f527410e5a0 con 0x7f527406b470 2024-09-11T00:15:47.690 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:47.686+0000 7f527a800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f52741084a0 0x7f527419bfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:15:47.690 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:47.686+0000 7f527a800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f52741084a0 0x7f527419bfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.12:44152/0 (socket says 172.21.15.12:44152) 2024-09-11T00:15:47.690 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:47.687+0000 7f527a800640 1 -- 172.21.15.12:0/3403242842 learned_addr learned my addr 172.21.15.12:0/3403242842 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:15:47.690 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:47.686+0000 7f5279400640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f52741058f0 0x7f527419b3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:15:47.691 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:47.687+0000 7f5279e00640 1 --2- 172.21.15.12:0/3403242842 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f527406b470 0x7f527419ad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:15:47.691 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:47.687+0000 7f527a800640 1 -- 172.21.15.12:0/3403242842 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f527406b470 msgr2=0x7f527419ad60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:15:47.691 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:47.687+0000 7f527a800640 1 --2- 172.21.15.12:0/3403242842 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f527406b470 0x7f527419ad60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:15:47.691 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:47.687+0000 7f527a800640 1 -- 172.21.15.12:0/3403242842 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f52741058f0 msgr2=0x7f527419b3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:15:47.691 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:47.687+0000 7f527a800640 1 --2- 172.21.15.12:0/3403242842 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f52741058f0 0x7f527419b3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:15:47.691 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:47.687+0000 7f527a800640 1 -- 172.21.15.12:0/3403242842 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f527419c8a0 con 0x7f52741084a0 2024-09-11T00:15:47.691 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:47.687+0000 7f527a800640 1 --2- 172.21.15.12:0/3403242842 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f52741084a0 0x7f527419bfe0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f527000eb10 tx=0x7f527000c2d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:15:47.691 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:47.687+0000 7f526ea00640 1 -- 172.21.15.12:0/3403242842 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5270004b20 con 0x7f52741084a0 2024-09-11T00:15:47.691 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:47.687+0000 7f526ea00640 1 -- 172.21.15.12:0/3403242842 <== mon.2 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5270004cc0 con 0x7f52741084a0 2024-09-11T00:15:47.691 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:47.687+0000 7f526ea00640 1 -- 172.21.15.12:0/3403242842 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5270010670 con 0x7f52741084a0 2024-09-11T00:15:47.691 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:47.687+0000 7f527b200640 1 -- 172.21.15.12:0/3403242842 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f527419cb90 con 0x7f52741084a0 2024-09-11T00:15:47.691 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:47.688+0000 7f527b200640 1 -- 172.21.15.12:0/3403242842 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52741a3260 con 0x7f52741084a0 2024-09-11T00:15:47.692 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:47.691+0000 7f527f845740 1 Processor -- start 2024-09-11T00:15:47.693 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:47.692+0000 7f526ea00640 1 -- 172.21.15.12:0/3403242842 <== mon.2 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7f5270005020 con 0x7f52741084a0 2024-09-11T00:15:47.693 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:47.692+0000 7f527f845740 1 -- start start 2024-09-11T00:15:47.693 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:47.692+0000 7f527f845740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56183df75970 0x56183df77d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:15:47.693 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:47.692+0000 7f527f845740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56183df782a0 0x56183df7a690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:15:47.694 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:47.692+0000 7f527f845740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56183df7ace0 0x56183df7d1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:15:47.694 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:47.692+0000 7f527f845740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x56183dee86b0 con 0x56183df782a0 2024-09-11T00:15:47.694 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:47.692+0000 7f527f845740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x56183ded7ae0 con 0x56183df75970 2024-09-11T00:15:47.694 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:47.692+0000 7f527f845740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x56183dedb8f0 con 0x56183df7ace0 2024-09-11T00:15:47.695 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:47.692+0000 7f5279400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56183df75970 0x56183df77d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:15:47.695 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:47.693+0000 7f5279400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56183df75970 0x56183df77d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.12:44168/0 (socket says 172.21.15.12:44168) 2024-09-11T00:15:47.695 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:47.693+0000 7f5279e00640 1 --2- 172.21.15.12:0/2690977951 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56183df782a0 0x56183df7a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:15:47.696 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:47.693+0000 7f5279400640 1 -- 172.21.15.12:0/2690977951 learned_addr learned my addr 172.21.15.12:0/2690977951 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:15:47.696 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:47.693+0000 7f5279400640 1 --2- 172.21.15.12:0/2690977951 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56183df7ace0 0x56183df7d1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:15:47.696 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:47.693+0000 7f5279e00640 1 -- 172.21.15.12:0/2690977951 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56183df7ace0 msgr2=0x56183df7d1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:15:47.696 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:47.693+0000 7f5279e00640 1 --2- 172.21.15.12:0/2690977951 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56183df7ace0 0x56183df7d1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:15:47.696 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:47.693+0000 7f5279e00640 1 -- 172.21.15.12:0/2690977951 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56183df75970 msgr2=0x56183df77d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:15:47.696 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:47.693+0000 7f5279e00640 1 --2- 172.21.15.12:0/2690977951 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56183df75970 0x56183df77d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:15:47.696 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:47.693+0000 7f5279e00640 1 -- 172.21.15.12:0/2690977951 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56183dea4780 con 0x56183df782a0 2024-09-11T00:15:47.697 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:47.693+0000 7f5279400640 1 --2- 172.21.15.12:0/2690977951 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56183df75970 0x56183df77d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:15:47.697 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:47.693+0000 7f5279e00640 1 --2- 172.21.15.12:0/2690977951 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56183df782a0 0x56183df7a690 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f526802a9e0 tx=0x7f5268005c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf3e010922774cab server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:15:47.697 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:47.694+0000 7f526ea00640 1 --2- 172.21.15.12:0/3403242842 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f525c0b52e0 0x7f525c0b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:15:47.697 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:47.694+0000 7f5263000640 1 -- 172.21.15.12:0/2690977951 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5268035030 con 0x56183df782a0 2024-09-11T00:15:47.697 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:47.694+0000 7f5263000640 1 -- 172.21.15.12:0/2690977951 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5268040070 con 0x56183df782a0 2024-09-11T00:15:47.697 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:47.694+0000 7f5263000640 1 -- 172.21.15.12:0/2690977951 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52680318d0 con 0x56183df782a0 2024-09-11T00:15:47.698 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:47.694+0000 7f526ea00640 1 -- 172.21.15.12:0/3403242842 <== mon.2 v2:172.21.15.165:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9208+0+0 (secure 0 0 0) 0x7f52700e6590 con 0x7f52741084a0 2024-09-11T00:15:47.698 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:47.694+0000 7f5279400640 1 --2- 172.21.15.12:0/3403242842 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f525c0b52e0 0x7f525c0b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:15:47.702 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:47.695+0000 7f527f845740 1 -- 172.21.15.12:0/2690977951 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56183df782a0 msgr2=0x56183df7a690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:15:47.702 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:47.695+0000 7f527f845740 1 --2- 172.21.15.12:0/2690977951 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56183df782a0 0x56183df7a690 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f526802a9e0 tx=0x7f5268005c00 comp rx=0 tx=0).stop 2024-09-11T00:15:47.702 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:47.695+0000 7f5279400640 1 --2- 172.21.15.12:0/3403242842 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f525c0b52e0 0x7f525c0b77a0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f5264005bc0 tx=0x7f5264005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:15:47.702 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:47.695+0000 7f527f845740 1 -- 172.21.15.12:0/2690977951 shutdown_connections 2024-09-11T00:15:47.702 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:47.695+0000 7f527f845740 1 --2- 172.21.15.12:0/2690977951 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56183df7ace0 0x56183df7d1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:15:47.702 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:47.695+0000 7f527f845740 1 --2- 172.21.15.12:0/2690977951 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56183df782a0 0x56183df7a690 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:15:47.702 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:47.695+0000 7f527f845740 1 --2- 172.21.15.12:0/2690977951 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56183df75970 0x56183df77d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:15:47.702 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:47.695+0000 7f527f845740 1 -- 172.21.15.12:0/2690977951 >> 172.21.15.12:0/2690977951 conn(0x56183df70ac0 msgr2=0x56183df71850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:15:47.702 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:47.695+0000 7f527f845740 1 -- 172.21.15.12:0/2690977951 shutdown_connections 2024-09-11T00:15:47.703 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:47.695+0000 7f527f845740 1 -- 172.21.15.12:0/2690977951 wait complete. 2024-09-11T00:15:47.703 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:47.696+0000 7f527f845740 1 -- start start 2024-09-11T00:15:47.703 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:47.696+0000 7f527f845740 1 Processor -- start 2024-09-11T00:15:47.703 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:47.696+0000 7f527f845740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56183df75970 0x56183df7f040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:15:47.703 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:47.696+0000 7f527f845740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56183df782a0 0x56183df7f6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:15:47.703 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:47.696+0000 7f527f845740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56183df7ace0 0x56183df802b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:15:47.704 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:47.696+0000 7f527f845740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x56183de0fab0 con 0x56183df75970 2024-09-11T00:15:47.704 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:47.696+0000 7f527f845740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x56183de7e570 con 0x56183df7ace0 2024-09-11T00:15:47.704 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:47.696+0000 7f527f845740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x56183df7dc60 con 0x56183df782a0 2024-09-11T00:15:47.704 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:47.696+0000 7f527f845740 10 client.0 _ll_register_callbacks cb 0x56183de8d100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:15:47.704 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:47.696+0000 7f5279400640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56183df782a0 0x56183df7f6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:15:47.705 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:47.697+0000 7f527a800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56183df7ace0 0x56183df802b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:15:47.705 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:47.697+0000 7f5279400640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56183df782a0 0x56183df7f6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.12:58024/0 (socket says 172.21.15.12:58024) 2024-09-11T00:15:47.705 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:47.697+0000 7f5279400640 1 -- 172.21.15.12:0/4079834770 learned_addr learned my addr 172.21.15.12:0/4079834770 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:15:47.705 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:47.697+0000 7f5279e00640 1 --2- 172.21.15.12:0/4079834770 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56183df75970 0x56183df7f040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:15:47.705 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:47.697+0000 7f527a800640 1 -- 172.21.15.12:0/4079834770 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56183df782a0 msgr2=0x56183df7f6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:15:47.705 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:47.697+0000 7f527a800640 1 --2- 172.21.15.12:0/4079834770 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56183df782a0 0x56183df7f6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:15:47.705 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:47.697+0000 7f527a800640 1 -- 172.21.15.12:0/4079834770 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56183df75970 msgr2=0x56183df7f040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:15:47.705 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:47.697+0000 7f527a800640 1 --2- 172.21.15.12:0/4079834770 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56183df75970 0x56183df7f040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:15:47.706 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:47.697+0000 7f527a800640 1 -- 172.21.15.12:0/4079834770 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56183dc39790 con 0x56183df7ace0 2024-09-11T00:15:47.706 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:47.697+0000 7f5279400640 1 --2- 172.21.15.12:0/4079834770 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56183df782a0 0x56183df7f6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:15:47.706 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:47.697+0000 7f527a800640 1 --2- 172.21.15.12:0/4079834770 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56183df7ace0 0x56183df802b0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f52700e3cc0 tx=0x7f52700ac810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bfa9f4695f3574c9 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:15:47.706 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:47.697+0000 7f5257400640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-09-11T00:15:47.706 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:47.697+0000 7f5257400640 1 -- 172.21.15.12:0/4079834770 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52700ac490 con 0x56183df7ace0 2024-09-11T00:15:47.706 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:47.698+0000 7f527f845740 1 -- 172.21.15.12:0/4079834770 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5268040070 con 0x56183df7ace0 2024-09-11T00:15:47.706 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:47.698+0000 7f5257400640 1 -- 172.21.15.12:0/4079834770 <== mon.2 v2:172.21.15.165:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9208+0+0 (secure 0 0 0) 0x7f5270005340 con 0x56183df7ace0 2024-09-11T00:15:47.706 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:47.698+0000 7f5257400640 1 -- 172.21.15.12:0/4079834770 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f5270005540 con 0x56183df7ace0 2024-09-11T00:15:47.707 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:47.698+0000 7f527f845740 10 client.26510 fetch_fsmap learned FSMap version 41 2024-09-11T00:15:47.707 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:47.698+0000 7f527f845740 1 -- 172.21.15.12:0/4079834770 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x56183dcb2620 con 0x56183df7ace0 2024-09-11T00:15:47.707 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:47.698+0000 7f5257400640 1 -- 172.21.15.12:0/4079834770 <== mon.2 v2:172.21.15.165:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5268040070 con 0x56183df7ace0 2024-09-11T00:15:47.707 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:47.698+0000 7f5257400640 1 -- 172.21.15.12:0/4079834770 <== mon.2 v2:172.21.15.165:3300/0 5 ==== fsmap(e 41) ==== 8370+0+0 (secure 0 0 0) 0x7f52700b5a80 con 0x56183df7ace0 2024-09-11T00:15:47.707 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:47.698+0000 7f527f845740 10 client.26510 fetch_fsmap finished waiting for FSMap version 41 2024-09-11T00:15:47.707 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:47.698+0000 7f527f845740 10 client.26510 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T00:15:47.707 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:47.698+0000 7f527f845740 20 client.26510 populate_metadata read hostname 'smithi012' 2024-09-11T00:15:47.708 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:47.699+0000 7f527f845740 1 --2- 172.21.15.12:0/4079834770 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x56183df8a930 0x56183df8cd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:15:47.708 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:47.699+0000 7f527f845740 4 client.26510 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T00:15:47.708 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:47.699+0000 7f527f845740 1 -- 172.21.15.12:0/4079834770 --> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56183deea600 con 0x56183df8a930 2024-09-11T00:15:47.708 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:47.699+0000 7f5279e00640 1 --2- 172.21.15.12:0/4079834770 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x56183df8a930 0x56183df8cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:15:47.708 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:47.699+0000 7f5279e00640 1 --2- 172.21.15.12:0/4079834770 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x56183df8a930 0x56183df8cd40 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f10b684583ab7dfa server_cookie=f66ddf2274f4f662 in_seq=0 out_seq=0 2024-09-11T00:15:47.708 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:47.699+0000 7f5257400640 10 client.26510 ms_handle_connect on v2:172.21.15.165:6838/501312984 2024-09-11T00:15:47.708 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:47.700+0000 7f5257400640 1 -- 172.21.15.12:0/4079834770 <== mds.0 v2:172.21.15.165:6838/501312984 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x56183deea600 con 0x56183df8a930 2024-09-11T00:15:47.709 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:47.700+0000 7f5257400640 10 client.26510 handle_command_reply: tid=0 2024-09-11T00:15:47.709 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:47.700+0000 7f527f845740 1 client.26510 shutdown 2024-09-11T00:15:47.709 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:47.700+0000 7f527f845740 1 -- 172.21.15.12:0/4079834770 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56183df7ace0 msgr2=0x56183df802b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:15:47.709 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:47.700+0000 7f527f845740 1 --2- 172.21.15.12:0/4079834770 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56183df7ace0 0x56183df802b0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f52700e3cc0 tx=0x7f52700ac810 comp rx=0 tx=0).stop 2024-09-11T00:15:47.709 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:47.701+0000 7f527f845740 1 -- 172.21.15.12:0/4079834770 shutdown_connections 2024-09-11T00:15:47.709 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:47.701+0000 7f527f845740 1 --2- 172.21.15.12:0/4079834770 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x56183df8a930 0x56183df8cd40 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:15:47.709 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:47.701+0000 7f527f845740 1 --2- 172.21.15.12:0/4079834770 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56183df7ace0 0x56183df802b0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:15:47.709 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:47.701+0000 7f527f845740 1 --2- 172.21.15.12:0/4079834770 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56183df782a0 0x56183df7f6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:15:47.709 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:47.701+0000 7f527f845740 1 --2- 172.21.15.12:0/4079834770 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56183df75970 0x56183df7f040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:15:47.710 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:47.701+0000 7f5257400640 0 client.26510 ms_handle_reset on v2:172.21.15.165:6838/501312984 2024-09-11T00:15:47.710 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:47.701+0000 7f527f845740 1 -- 172.21.15.12:0/4079834770 >> 172.21.15.12:0/4079834770 conn(0x56183df70ac0 msgr2=0x56183df6fbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:15:47.710 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:47.701+0000 7f527f845740 1 -- 172.21.15.12:0/4079834770 shutdown_connections 2024-09-11T00:15:47.710 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:47.701+0000 7f527f845740 1 -- 172.21.15.12:0/4079834770 wait complete. 2024-09-11T00:15:47.710 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:47.701+0000 7f527f845740 20 client.26510 trim_cache size 0 max 16384 2024-09-11T00:15:47.723 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:47.717+0000 7f527f845740 1 Processor -- start 2024-09-11T00:15:47.723 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:47.717+0000 7f527f845740 1 -- start start 2024-09-11T00:15:47.723 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:47.717+0000 7f527f845740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56183df75970 0x56183df71320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:15:47.723 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:47.717+0000 7f527f845740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56183df782a0 0x56183df71860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:15:47.723 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:47.718+0000 7f527f845740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56183df7ace0 0x56183df7b0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:15:47.723 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:47.718+0000 7f527f845740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f52741ad640 con 0x56183df7ace0 2024-09-11T00:15:47.723 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:47.718+0000 7f527f845740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f52741af180 con 0x56183df75970 2024-09-11T00:15:47.723 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:47.718+0000 7f527f845740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f52741add10 con 0x56183df782a0 2024-09-11T00:15:47.724 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:47.718+0000 7f5279e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56183df75970 0x56183df71320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:15:47.724 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:47.718+0000 7f5279400640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56183df782a0 0x56183df71860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:15:47.724 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:47.718+0000 7f5279e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56183df75970 0x56183df71320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.12:44178/0 (socket says 172.21.15.12:44178) 2024-09-11T00:15:47.724 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:47.718+0000 7f5279400640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56183df782a0 0x56183df71860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.12:58026/0 (socket says 172.21.15.12:58026) 2024-09-11T00:15:47.724 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:47.718+0000 7f5279e00640 1 -- 172.21.15.12:0/3482216292 learned_addr learned my addr 172.21.15.12:0/3482216292 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:15:47.724 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:47.718+0000 7f527a800640 1 --2- 172.21.15.12:0/3482216292 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56183df7ace0 0x56183df7b0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:15:47.724 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:47.718+0000 7f5279e00640 1 -- 172.21.15.12:0/3482216292 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56183df782a0 msgr2=0x56183df71860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:15:47.725 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:47.718+0000 7f5279e00640 1 --2- 172.21.15.12:0/3482216292 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56183df782a0 0x56183df71860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:15:47.725 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:47.718+0000 7f5279e00640 1 -- 172.21.15.12:0/3482216292 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56183df7ace0 msgr2=0x56183df7b0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:15:47.725 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:47.718+0000 7f5279e00640 1 --2- 172.21.15.12:0/3482216292 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56183df7ace0 0x56183df7b0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:15:47.725 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:47.718+0000 7f5279e00640 1 -- 172.21.15.12:0/3482216292 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56183df7fa50 con 0x56183df75970 2024-09-11T00:15:47.725 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:47.718+0000 7f5279400640 1 --2- 172.21.15.12:0/3482216292 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56183df782a0 0x56183df71860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:15:47.725 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:47.719+0000 7f5279e00640 1 --2- 172.21.15.12:0/3482216292 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56183df75970 0x56183df71320 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f5268009900 tx=0x7f5268009230 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3c69043bb386ae9c server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:15:47.726 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:47.719+0000 7f5262600640 1 -- 172.21.15.12:0/3482216292 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5268035040 con 0x56183df75970 2024-09-11T00:15:47.726 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:47.719+0000 7f5262600640 1 -- 172.21.15.12:0/3482216292 <== mon.2 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5268040070 con 0x56183df75970 2024-09-11T00:15:47.726 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:47.719+0000 7f5262600640 1 -- 172.21.15.12:0/3482216292 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5268031b00 con 0x56183df75970 2024-09-11T00:15:47.726 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:47.720+0000 7f527f845740 1 -- 172.21.15.12:0/3482216292 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56183df75970 msgr2=0x56183df71320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:15:47.726 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:47.720+0000 7f527f845740 1 --2- 172.21.15.12:0/3482216292 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56183df75970 0x56183df71320 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f5268009900 tx=0x7f5268009230 comp rx=0 tx=0).stop 2024-09-11T00:15:47.726 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:47.720+0000 7f527f845740 1 -- 172.21.15.12:0/3482216292 shutdown_connections 2024-09-11T00:15:47.726 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:47.720+0000 7f527f845740 1 --2- 172.21.15.12:0/3482216292 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56183df7ace0 0x56183df7b0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:15:47.726 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:47.720+0000 7f527f845740 1 --2- 172.21.15.12:0/3482216292 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56183df782a0 0x56183df71860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:15:47.726 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:47.720+0000 7f527f845740 1 --2- 172.21.15.12:0/3482216292 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56183df75970 0x56183df71320 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:15:47.727 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:47.720+0000 7f527f845740 1 -- 172.21.15.12:0/3482216292 >> 172.21.15.12:0/3482216292 conn(0x56183df70ac0 msgr2=0x56183df7e000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:15:47.727 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:47.720+0000 7f527f845740 1 -- 172.21.15.12:0/3482216292 shutdown_connections 2024-09-11T00:15:47.727 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:47.720+0000 7f527f845740 1 -- 172.21.15.12:0/3482216292 wait complete. 2024-09-11T00:15:47.727 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:47.720+0000 7f527f845740 1 -- start start 2024-09-11T00:15:47.727 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:47.720+0000 7f527f845740 1 Processor -- start 2024-09-11T00:15:47.727 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:47.721+0000 7f527f845740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56183df75970 0x56183df86910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:15:47.728 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:47.721+0000 7f527f845740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56183df782a0 0x56183df85610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:15:47.728 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:47.721+0000 7f527f845740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56183df7ace0 0x56183df85b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:15:47.728 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:47.721+0000 7f527f845740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f52741aeab0 con 0x56183df782a0 2024-09-11T00:15:47.728 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:47.721+0000 7f527f845740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f52741ae3e0 con 0x56183df75970 2024-09-11T00:15:47.728 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:47.721+0000 7f527f845740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f52741ab620 con 0x56183df7ace0 2024-09-11T00:15:47.728 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:47.721+0000 7f527f845740 10 client.0 _ll_register_callbacks cb 0x7f52700a8040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:15:47.729 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:47.721+0000 7f5279400640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56183df782a0 0x56183df85610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:15:47.729 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:47.721+0000 7f5279e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56183df75970 0x56183df86910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:15:47.729 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:47.721+0000 7f5279e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56183df75970 0x56183df86910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.12:44182/0 (socket says 172.21.15.12:44182) 2024-09-11T00:15:47.729 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:47.721+0000 7f5279e00640 1 -- 172.21.15.12:0/2008700530 learned_addr learned my addr 172.21.15.12:0/2008700530 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:15:47.729 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:47.721+0000 7f5279400640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56183df782a0 0x56183df85610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:42966/0 (socket says 172.21.15.12:42966) 2024-09-11T00:15:47.730 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:47.721+0000 7f527a800640 1 --2- 172.21.15.12:0/2008700530 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56183df7ace0 0x56183df85b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:15:47.730 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:47.722+0000 7f5279e00640 1 -- 172.21.15.12:0/2008700530 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56183df7ace0 msgr2=0x56183df85b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:15:47.730 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:47.722+0000 7f5279e00640 1 --2- 172.21.15.12:0/2008700530 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56183df7ace0 0x56183df85b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:15:47.730 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:47.722+0000 7f5279e00640 1 -- 172.21.15.12:0/2008700530 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56183df782a0 msgr2=0x56183df85610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:15:47.730 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:47.722+0000 7f5279e00640 1 --2- 172.21.15.12:0/2008700530 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56183df782a0 0x56183df85610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:15:47.731 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:47.722+0000 7f5279e00640 1 -- 172.21.15.12:0/2008700530 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56183df86190 con 0x56183df75970 2024-09-11T00:15:47.731 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:47.722+0000 7f5279e00640 1 --2- 172.21.15.12:0/2008700530 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56183df75970 0x56183df86910 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f5268000c00 tx=0x7f5268009520 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b24f76bdb4be0f5e server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:15:47.731 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:47.722+0000 7f5257400640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-09-11T00:15:47.733 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:47.722+0000 7f5257400640 1 -- 172.21.15.12:0/2008700530 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5268035040 con 0x56183df75970 2024-09-11T00:15:47.733 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:47.722+0000 7f5257400640 1 -- 172.21.15.12:0/2008700530 <== mon.2 v2:172.21.15.165:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9208+0+0 (secure 0 0 0) 0x7f5268031c90 con 0x56183df75970 2024-09-11T00:15:47.733 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:47.722+0000 7f527f845740 1 -- 172.21.15.12:0/2008700530 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5268040070 con 0x56183df75970 2024-09-11T00:15:47.734 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:47.723+0000 7f5257400640 1 -- 172.21.15.12:0/2008700530 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f526806a050 con 0x56183df75970 2024-09-11T00:15:47.734 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:47.723+0000 7f527f845740 10 client.26516 fetch_fsmap learned FSMap version 41 2024-09-11T00:15:47.734 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:47.723+0000 7f527f845740 1 -- 172.21.15.12:0/2008700530 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x56183df84f70 con 0x56183df75970 2024-09-11T00:15:47.734 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:47.723+0000 7f5257400640 1 -- 172.21.15.12:0/2008700530 <== mon.2 v2:172.21.15.165:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5268040070 con 0x56183df75970 2024-09-11T00:15:47.734 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:47.723+0000 7f5257400640 1 -- 172.21.15.12:0/2008700530 <== mon.2 v2:172.21.15.165:3300/0 5 ==== fsmap(e 41) ==== 8370+0+0 (secure 0 0 0) 0x7f5268061600 con 0x56183df75970 2024-09-11T00:15:47.734 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:47.723+0000 7f527f845740 10 client.26516 fetch_fsmap finished waiting for FSMap version 41 2024-09-11T00:15:47.734 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:47.723+0000 7f527f845740 10 client.26516 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T00:15:47.734 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:47.723+0000 7f527f845740 20 client.26516 populate_metadata read hostname 'smithi012' 2024-09-11T00:15:47.734 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:47.723+0000 7f527f845740 1 --2- 172.21.15.12:0/2008700530 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x56183dff7c10 0x56183dffa000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:15:47.734 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:47.723+0000 7f527f845740 4 client.26516 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T00:15:47.734 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:47.723+0000 7f527f845740 1 -- 172.21.15.12:0/2008700530 --> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56183dee60b0 con 0x56183dff7c10 2024-09-11T00:15:47.735 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:47.723+0000 7f5279400640 1 --2- 172.21.15.12:0/2008700530 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x56183dff7c10 0x56183dffa000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:15:47.735 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:47.724+0000 7f5279400640 1 --2- 172.21.15.12:0/2008700530 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x56183dff7c10 0x56183dffa000 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d702ea4ca4c3a22 server_cookie=74dd0e1e6621706f in_seq=0 out_seq=0 2024-09-11T00:15:47.735 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:47.724+0000 7f5257400640 10 client.26516 ms_handle_connect on v2:172.21.15.165:6838/501312984 2024-09-11T00:15:47.765 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:47.762+0000 7f5257400640 1 -- 172.21.15.12:0/2008700530 <== mds.0 v2:172.21.15.165:6838/501312984 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56183dee60b0 con 0x56183dff7c10 2024-09-11T00:15:47.765 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:47.762+0000 7f5257400640 10 client.26516 handle_command_reply: tid=0 2024-09-11T00:15:47.765 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:47.762+0000 7f527f845740 1 client.26516 shutdown 2024-09-11T00:15:47.765 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:47.763+0000 7f527f845740 1 -- 172.21.15.12:0/2008700530 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56183df75970 msgr2=0x56183df86910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:15:47.765 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:47.763+0000 7f527f845740 1 --2- 172.21.15.12:0/2008700530 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56183df75970 0x56183df86910 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f5268000c00 tx=0x7f5268009520 comp rx=0 tx=0).stop 2024-09-11T00:15:47.766 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:47.763+0000 7f527f845740 1 -- 172.21.15.12:0/2008700530 shutdown_connections 2024-09-11T00:15:47.766 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:47.763+0000 7f527f845740 1 --2- 172.21.15.12:0/2008700530 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x56183dff7c10 0x56183dffa000 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:15:47.766 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:47.763+0000 7f527f845740 1 --2- 172.21.15.12:0/2008700530 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56183df7ace0 0x56183df85b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:15:47.766 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:47.763+0000 7f527f845740 1 --2- 172.21.15.12:0/2008700530 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56183df782a0 0x56183df85610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:15:47.766 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:47.763+0000 7f5257400640 0 client.26516 ms_handle_reset on v2:172.21.15.165:6838/501312984 2024-09-11T00:15:47.767 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:47.763+0000 7f527f845740 1 --2- 172.21.15.12:0/2008700530 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56183df75970 0x56183df86910 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:15:47.767 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:47.763+0000 7f527f845740 1 -- 172.21.15.12:0/2008700530 >> 172.21.15.12:0/2008700530 conn(0x56183df70ac0 msgr2=0x56183df7e000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:15:47.767 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:47.763+0000 7f527f845740 1 -- 172.21.15.12:0/2008700530 shutdown_connections 2024-09-11T00:15:47.767 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:47.763+0000 7f527f845740 1 -- 172.21.15.12:0/2008700530 wait complete. 2024-09-11T00:15:47.768 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:47.763+0000 7f527f845740 20 client.26516 trim_cache size 0 max 16384 2024-09-11T00:15:47.768 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:{ 2024-09-11T00:15:47.768 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "message": "", 2024-09-11T00:15:47.768 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "return_code": 0 2024-09-11T00:15:47.768 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:} 2024-09-11T00:15:47.769 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:47.764+0000 7f527b200640 1 -- 172.21.15.12:0/3403242842 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f525c0b52e0 msgr2=0x7f525c0b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:15:47.769 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:47.764+0000 7f527b200640 1 --2- 172.21.15.12:0/3403242842 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f525c0b52e0 0x7f525c0b77a0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f5264005bc0 tx=0x7f5264005f90 comp rx=0 tx=0).stop 2024-09-11T00:15:47.769 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:47.764+0000 7f527b200640 1 -- 172.21.15.12:0/3403242842 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f52741084a0 msgr2=0x7f527419bfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:15:47.769 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:47.764+0000 7f527b200640 1 --2- 172.21.15.12:0/3403242842 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f52741084a0 0x7f527419bfe0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f527000eb10 tx=0x7f527000c2d0 comp rx=0 tx=0).stop 2024-09-11T00:15:47.769 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:47.764+0000 7f527b200640 1 -- 172.21.15.12:0/3403242842 shutdown_connections 2024-09-11T00:15:47.769 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:47.764+0000 7f527b200640 1 --2- 172.21.15.12:0/3403242842 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f525c0b52e0 0x7f525c0b77a0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:15:47.769 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:47.764+0000 7f527b200640 1 --2- 172.21.15.12:0/3403242842 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f52741084a0 0x7f527419bfe0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:15:47.770 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:47.764+0000 7f527b200640 1 --2- 172.21.15.12:0/3403242842 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f52741058f0 0x7f527419b3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:15:47.770 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:47.764+0000 7f527b200640 1 --2- 172.21.15.12:0/3403242842 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f527406b470 0x7f527419ad60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:15:47.770 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:47.764+0000 7f527b200640 1 -- 172.21.15.12:0/3403242842 >> 172.21.15.12:0/3403242842 conn(0x7f52740fcf30 msgr2=0x7f52740ff320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:15:47.770 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:47.764+0000 7f527b200640 1 -- 172.21.15.12:0/3403242842 shutdown_connections 2024-09-11T00:15:47.770 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:15:47.765+0000 7f527b200640 1 -- 172.21.15.12:0/3403242842 wait complete. 2024-09-11T00:15:47.813 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:15:47 smithi012 ceph-mon[30333]: pgmap v660: 129 pgs: 129 active+clean; 76 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 843 KiB/s rd, 1.3 MiB/s wr, 109 op/s 2024-09-11T00:15:47.830 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:15:47 smithi178 ceph-mon[35919]: pgmap v660: 129 pgs: 129 active+clean; 76 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 843 KiB/s rd, 1.3 MiB/s wr, 109 op/s 2024-09-11T00:15:48.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:15:47 smithi165 ceph-mon[35356]: pgmap v660: 129 pgs: 129 active+clean; 76 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 843 KiB/s rd, 1.3 MiB/s wr, 109 op/s 2024-09-11T00:15:48.666 INFO:tasks.workunit.client.0.smithi012.stdout: 1 320467 3.50 MB/sec execute 345 sec latency 30.631 ms 2024-09-11T00:15:49.666 INFO:tasks.workunit.client.0.smithi012.stdout: 1 322736 3.53 MB/sec execute 346 sec latency 15.307 ms 2024-09-11T00:15:49.813 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:15:49 smithi012 ceph-mon[30333]: pgmap v661: 129 pgs: 129 active+clean; 76 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 919 KiB/s rd, 921 KiB/s wr, 88 op/s 2024-09-11T00:15:49.830 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:15:49 smithi178 ceph-mon[35919]: pgmap v661: 129 pgs: 129 active+clean; 76 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 919 KiB/s rd, 921 KiB/s wr, 88 op/s 2024-09-11T00:15:50.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:15:49 smithi165 ceph-mon[35356]: pgmap v661: 129 pgs: 129 active+clean; 76 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 919 KiB/s rd, 921 KiB/s wr, 88 op/s 2024-09-11T00:15:50.666 INFO:tasks.workunit.client.0.smithi012.stdout: 1 323773 3.53 MB/sec execute 347 sec latency 68.931 ms 2024-09-11T00:15:51.666 INFO:tasks.workunit.client.0.smithi012.stdout: 1 325693 3.55 MB/sec execute 348 sec latency 159.540 ms 2024-09-11T00:15:51.813 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:15:51 smithi012 ceph-mon[30333]: pgmap v662: 129 pgs: 129 active+clean; 76 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.1 MiB/s wr, 160 op/s 2024-09-11T00:15:51.830 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:15:51 smithi178 ceph-mon[35919]: pgmap v662: 129 pgs: 129 active+clean; 76 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.1 MiB/s wr, 160 op/s 2024-09-11T00:15:52.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:15:51 smithi165 ceph-mon[35356]: pgmap v662: 129 pgs: 129 active+clean; 76 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.1 MiB/s wr, 160 op/s 2024-09-11T00:15:52.666 INFO:tasks.workunit.client.0.smithi012.stdout: 1 327004 3.56 MB/sec execute 349 sec latency 10.097 ms 2024-09-11T00:15:53.666 INFO:tasks.workunit.client.0.smithi012.stdout: 1 329080 3.58 MB/sec execute 350 sec latency 13.594 ms 2024-09-11T00:15:53.813 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:15:53 smithi012 ceph-mon[30333]: pgmap v663: 129 pgs: 129 active+clean; 77 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 1.1 MiB/s wr, 196 op/s 2024-09-11T00:15:53.830 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:15:53 smithi178 ceph-mon[35919]: pgmap v663: 129 pgs: 129 active+clean; 77 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 1.1 MiB/s wr, 196 op/s 2024-09-11T00:15:54.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:15:53 smithi165 ceph-mon[35356]: pgmap v663: 129 pgs: 129 active+clean; 77 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 1.1 MiB/s wr, 196 op/s 2024-09-11T00:15:54.666 INFO:tasks.workunit.client.0.smithi012.stdout: 1 330325 3.59 MB/sec execute 351 sec latency 15.177 ms 2024-09-11T00:15:55.666 INFO:tasks.workunit.client.0.smithi012.stdout: 1 332319 3.60 MB/sec execute 352 sec latency 13.945 ms 2024-09-11T00:15:55.813 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:15:55 smithi012 ceph-mon[30333]: pgmap v664: 129 pgs: 129 active+clean; 77 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 1.1 MiB/s wr, 191 op/s 2024-09-11T00:15:55.831 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:15:55 smithi178 ceph-mon[35919]: pgmap v664: 129 pgs: 129 active+clean; 77 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 1.1 MiB/s wr, 191 op/s 2024-09-11T00:15:56.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:15:55 smithi165 ceph-mon[35356]: pgmap v664: 129 pgs: 129 active+clean; 77 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 1.1 MiB/s wr, 191 op/s 2024-09-11T00:15:56.666 INFO:tasks.workunit.client.0.smithi012.stdout: 1 333811 3.63 MB/sec execute 353 sec latency 11.229 ms 2024-09-11T00:15:57.666 INFO:tasks.workunit.client.0.smithi012.stdout: 1 335492 3.63 MB/sec execute 354 sec latency 18.541 ms 2024-09-11T00:15:57.813 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:15:57 smithi012 ceph-mon[30333]: pgmap v665: 129 pgs: 129 active+clean; 81 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 1.4 MiB/s wr, 298 op/s 2024-09-11T00:15:57.831 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:15:57 smithi178 ceph-mon[35919]: pgmap v665: 129 pgs: 129 active+clean; 81 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 1.4 MiB/s wr, 298 op/s 2024-09-11T00:15:58.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:15:57 smithi165 ceph-mon[35356]: pgmap v665: 129 pgs: 129 active+clean; 81 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 1.4 MiB/s wr, 298 op/s 2024-09-11T00:15:58.666 INFO:tasks.workunit.client.0.smithi012.stdout: 1 337251 3.66 MB/sec execute 355 sec latency 10.023 ms 2024-09-11T00:15:59.667 INFO:tasks.workunit.client.0.smithi012.stdout: 1 338465 3.67 MB/sec execute 356 sec latency 13.650 ms 2024-09-11T00:15:59.813 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:15:59 smithi012 ceph-mon[30333]: pgmap v666: 129 pgs: 129 active+clean; 81 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 955 KiB/s wr, 247 op/s 2024-09-11T00:15:59.830 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:15:59 smithi178 ceph-mon[35919]: pgmap v666: 129 pgs: 129 active+clean; 81 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 955 KiB/s wr, 247 op/s 2024-09-11T00:16:00.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:15:59 smithi165 ceph-mon[35356]: pgmap v666: 129 pgs: 129 active+clean; 81 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 955 KiB/s wr, 247 op/s 2024-09-11T00:16:00.667 INFO:tasks.workunit.client.0.smithi012.stdout: 1 340520 3.69 MB/sec execute 357 sec latency 19.541 ms 2024-09-11T00:16:00.897 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:00.894+0000 7f8c8c200640 1 -- 172.21.15.12:0/3365597771 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8c841066b0 msgr2=0x7f8c8410d030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:16:00.897 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:00.894+0000 7f8c8c200640 1 --2- 172.21.15.12:0/3365597771 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8c841066b0 0x7f8c8410d030 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f8c7c009be0 tx=0x7f8c7c02a6d0 comp rx=0 tx=0).stop 2024-09-11T00:16:00.897 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:00.894+0000 7f8c8c200640 1 -- 172.21.15.12:0/3365597771 shutdown_connections 2024-09-11T00:16:00.898 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:00.894+0000 7f8c8c200640 1 --2- 172.21.15.12:0/3365597771 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8c841066b0 0x7f8c8410d030 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:16:00.898 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:00.894+0000 7f8c8c200640 1 --2- 172.21.15.12:0/3365597771 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8c84105cf0 0x7f8c84106170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:16:00.898 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:00.894+0000 7f8c8c200640 1 --2- 172.21.15.12:0/3365597771 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8c84104ba0 0x7f8c84104fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:16:00.898 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:00.894+0000 7f8c8c200640 1 -- 172.21.15.12:0/3365597771 >> 172.21.15.12:0/3365597771 conn(0x7f8c84100460 msgr2=0x7f8c84102880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:16:00.898 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:00.894+0000 7f8c8c200640 1 -- 172.21.15.12:0/3365597771 shutdown_connections 2024-09-11T00:16:00.898 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:00.894+0000 7f8c8c200640 1 -- 172.21.15.12:0/3365597771 wait complete. 2024-09-11T00:16:00.898 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:00.895+0000 7f8c8c200640 1 Processor -- start 2024-09-11T00:16:00.898 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:00.895+0000 7f8c8c200640 1 -- start start 2024-09-11T00:16:00.900 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:00.896+0000 7f8c8c200640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8c84104ba0 0x7f8c8419aeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:16:00.900 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:00.896+0000 7f8c8c200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8c84105cf0 0x7f8c8419b500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:16:00.900 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:00.896+0000 7f8c8c200640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8c841066b0 0x7f8c8419c130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:16:00.900 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:00.896+0000 7f8c8c200640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f8c841107f0 con 0x7f8c84104ba0 2024-09-11T00:16:00.900 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:00.896+0000 7f8c8c200640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f8c84110670 con 0x7f8c84105cf0 2024-09-11T00:16:00.900 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:00.896+0000 7f8c8c200640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f8c84110970 con 0x7f8c841066b0 2024-09-11T00:16:00.900 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:00.896+0000 7f8c88e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8c84105cf0 0x7f8c8419b500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:16:00.900 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:00.896+0000 7f8c88e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8c84105cf0 0x7f8c8419b500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.12:60482/0 (socket says 172.21.15.12:60482) 2024-09-11T00:16:00.901 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:00.896+0000 7f8c88e00640 1 -- 172.21.15.12:0/3343344591 learned_addr learned my addr 172.21.15.12:0/3343344591 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:16:00.901 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:00.896+0000 7f8c8a200640 1 --2- 172.21.15.12:0/3343344591 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8c841066b0 0x7f8c8419c130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:16:00.901 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:00.896+0000 7f8c89800640 1 --2- 172.21.15.12:0/3343344591 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8c84104ba0 0x7f8c8419aeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:16:00.901 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:00.896+0000 7f8c88e00640 1 -- 172.21.15.12:0/3343344591 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8c841066b0 msgr2=0x7f8c8419c130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:16:00.901 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:00.896+0000 7f8c88e00640 1 --2- 172.21.15.12:0/3343344591 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8c841066b0 0x7f8c8419c130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:16:00.901 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:00.896+0000 7f8c88e00640 1 -- 172.21.15.12:0/3343344591 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8c84104ba0 msgr2=0x7f8c8419aeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:16:00.901 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:00.896+0000 7f8c88e00640 1 --2- 172.21.15.12:0/3343344591 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8c84104ba0 0x7f8c8419aeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:16:00.901 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:00.896+0000 7f8c88e00640 1 -- 172.21.15.12:0/3343344591 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c8419c9f0 con 0x7f8c84105cf0 2024-09-11T00:16:00.901 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:00.896+0000 7f8c89800640 1 --2- 172.21.15.12:0/3343344591 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8c84104ba0 0x7f8c8419aeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T00:16:00.901 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:00.896+0000 7f8c8a200640 1 --2- 172.21.15.12:0/3343344591 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8c841066b0 0x7f8c8419c130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:16:00.901 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:00.897+0000 7f8c88e00640 1 --2- 172.21.15.12:0/3343344591 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8c84105cf0 0x7f8c8419b500 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f8c7000e9c0 tx=0x7f8c7000ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:16:00.901 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:00.897+0000 7f8c82000640 1 -- 172.21.15.12:0/3343344591 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c7000cde0 con 0x7f8c84105cf0 2024-09-11T00:16:00.902 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:00.897+0000 7f8c82000640 1 -- 172.21.15.12:0/3343344591 <== mon.2 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8c700035f0 con 0x7f8c84105cf0 2024-09-11T00:16:00.902 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:00.897+0000 7f8c82000640 1 -- 172.21.15.12:0/3343344591 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c70010690 con 0x7f8c84105cf0 2024-09-11T00:16:00.902 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:00.897+0000 7f8c8c200640 1 -- 172.21.15.12:0/3343344591 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8c8419cce0 con 0x7f8c84105cf0 2024-09-11T00:16:00.902 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:00.897+0000 7f8c8c200640 1 -- 172.21.15.12:0/3343344591 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8c841a33b0 con 0x7f8c84105cf0 2024-09-11T00:16:00.904 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:00.902+0000 7f8c8f09c740 1 Processor -- start 2024-09-11T00:16:00.904 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:00.902+0000 7f8c8f09c740 1 -- start start 2024-09-11T00:16:00.905 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:00.902+0000 7f8c8f09c740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56018728e790 0x560187290b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:16:00.905 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:00.902+0000 7f8c8f09c740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5601872910c0 0x5601872934b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:16:00.905 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:00.902+0000 7f8c8f09c740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560187293b00 0x560187295fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:16:00.905 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:00.902+0000 7f8c8f09c740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x5601872016b0 con 0x56018728e790 2024-09-11T00:16:00.905 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:00.902+0000 7f8c8f09c740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x5601871f0ae0 con 0x5601872910c0 2024-09-11T00:16:00.905 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:00.902+0000 7f8c8f09c740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5601871f48f0 con 0x560187293b00 2024-09-11T00:16:00.905 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:00.902+0000 7f8c82000640 1 -- 172.21.15.12:0/3343344591 <== mon.2 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7f8c70004000 con 0x7f8c84105cf0 2024-09-11T00:16:00.906 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:00.903+0000 7f8c89800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5601872910c0 0x5601872934b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:16:00.906 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:00.903+0000 7f8c8a200640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56018728e790 0x560187290b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:16:00.906 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:00.903+0000 7f8c89800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5601872910c0 0x5601872934b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.12:60498/0 (socket says 172.21.15.12:60498) 2024-09-11T00:16:00.906 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:00.903+0000 7f8c89800640 1 -- 172.21.15.12:0/2244088537 learned_addr learned my addr 172.21.15.12:0/2244088537 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:16:00.906 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:00.903+0000 7f8c89800640 1 -- 172.21.15.12:0/2244088537 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560187293b00 msgr2=0x560187295fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:16:00.906 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:00.903+0000 7f8c89800640 1 --2- 172.21.15.12:0/2244088537 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560187293b00 0x560187295fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:16:00.906 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:00.903+0000 7f8c89800640 1 -- 172.21.15.12:0/2244088537 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56018728e790 msgr2=0x560187290b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:16:00.906 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:00.903+0000 7f8c89800640 1 --2- 172.21.15.12:0/2244088537 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56018728e790 0x560187290b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:16:00.906 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:00.903+0000 7f8c89800640 1 -- 172.21.15.12:0/2244088537 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5601871bd780 con 0x5601872910c0 2024-09-11T00:16:00.907 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:00.903+0000 7f8c89800640 1 --2- 172.21.15.12:0/2244088537 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5601872910c0 0x5601872934b0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f8c78009d50 tx=0x7f8c780117b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9b096ddfa3f5cd53 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:16:00.907 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:00.904+0000 7f8c76a00640 1 -- 172.21.15.12:0/2244088537 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c78012650 con 0x5601872910c0 2024-09-11T00:16:00.907 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:00.904+0000 7f8c76a00640 1 -- 172.21.15.12:0/2244088537 <== mon.2 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8c78009e40 con 0x5601872910c0 2024-09-11T00:16:00.907 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:00.904+0000 7f8c76a00640 1 -- 172.21.15.12:0/2244088537 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c7800fbf0 con 0x5601872910c0 2024-09-11T00:16:00.907 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:00.905+0000 7f8c82000640 1 --2- 172.21.15.12:0/3343344591 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f8c6c0b52e0 0x7f8c6c0b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:16:00.907 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:00.905+0000 7f8c82000640 1 -- 172.21.15.12:0/3343344591 <== mon.2 v2:172.21.15.165:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9208+0+0 (secure 0 0 0) 0x7f8c700e6360 con 0x7f8c84105cf0 2024-09-11T00:16:00.908 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:00.905+0000 7f8c8f09c740 1 -- 172.21.15.12:0/2244088537 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5601872910c0 msgr2=0x5601872934b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:16:00.908 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:00.905+0000 7f8c8f09c740 1 --2- 172.21.15.12:0/2244088537 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5601872910c0 0x5601872934b0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f8c78009d50 tx=0x7f8c780117b0 comp rx=0 tx=0).stop 2024-09-11T00:16:00.908 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:00.905+0000 7f8c8a200640 1 --2- 172.21.15.12:0/3343344591 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f8c6c0b52e0 0x7f8c6c0b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:16:00.910 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:00.905+0000 7f8c8f09c740 1 -- 172.21.15.12:0/2244088537 shutdown_connections 2024-09-11T00:16:00.910 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:00.905+0000 7f8c8f09c740 1 --2- 172.21.15.12:0/2244088537 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560187293b00 0x560187295fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:16:00.910 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:00.905+0000 7f8c8f09c740 1 --2- 172.21.15.12:0/2244088537 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5601872910c0 0x5601872934b0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:16:00.910 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:00.905+0000 7f8c8f09c740 1 --2- 172.21.15.12:0/2244088537 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56018728e790 0x560187290b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:16:00.911 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:00.905+0000 7f8c8f09c740 1 -- 172.21.15.12:0/2244088537 >> 172.21.15.12:0/2244088537 conn(0x5601872898e0 msgr2=0x56018728a670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:16:00.911 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:00.906+0000 7f8c8f09c740 1 -- 172.21.15.12:0/2244088537 shutdown_connections 2024-09-11T00:16:00.911 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:00.906+0000 7f8c8f09c740 1 -- 172.21.15.12:0/2244088537 wait complete. 2024-09-11T00:16:00.911 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:00.906+0000 7f8c8a200640 1 --2- 172.21.15.12:0/3343344591 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f8c6c0b52e0 0x7f8c6c0b77a0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f8c7c006010 tx=0x7f8c7c030960 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:16:00.911 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:00.906+0000 7f8c8f09c740 1 -- start start 2024-09-11T00:16:00.911 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:00.907+0000 7f8c8f09c740 1 Processor -- start 2024-09-11T00:16:00.911 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:00.907+0000 7f8c8f09c740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56018728e790 0x560187297ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:16:00.911 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:00.907+0000 7f8c8f09c740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5601872910c0 0x560187298590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:16:00.911 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:00.907+0000 7f8c8f09c740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560187293b00 0x560187299160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:16:00.911 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:00.907+0000 7f8c8f09c740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x560187128ab0 con 0x5601872910c0 2024-09-11T00:16:00.911 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:00.907+0000 7f8c8f09c740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x560187197570 con 0x560187293b00 2024-09-11T00:16:00.912 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:00.907+0000 7f8c8f09c740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x560187296ae0 con 0x56018728e790 2024-09-11T00:16:00.912 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:00.907+0000 7f8c8f09c740 10 client.0 _ll_register_callbacks cb 0x5601871a6100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:16:00.912 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:00.907+0000 7f8c8a200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560187293b00 0x560187299160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:16:00.912 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:00.907+0000 7f8c8a200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560187293b00 0x560187299160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.12:60502/0 (socket says 172.21.15.12:60502) 2024-09-11T00:16:00.912 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:00.907+0000 7f8c8a200640 1 -- 172.21.15.12:0/3222157078 learned_addr learned my addr 172.21.15.12:0/3222157078 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:16:00.912 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:00.907+0000 7f8c88e00640 1 --2- 172.21.15.12:0/3222157078 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5601872910c0 0x560187298590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:16:00.912 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:00.907+0000 7f8c89800640 1 --2- 172.21.15.12:0/3222157078 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56018728e790 0x560187297ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:16:00.912 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:00.908+0000 7f8c8a200640 1 -- 172.21.15.12:0/3222157078 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56018728e790 msgr2=0x560187297ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:16:00.912 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:00.908+0000 7f8c8a200640 1 --2- 172.21.15.12:0/3222157078 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56018728e790 0x560187297ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:16:00.912 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:00.908+0000 7f8c8a200640 1 -- 172.21.15.12:0/3222157078 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5601872910c0 msgr2=0x560187298590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:16:00.912 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:00.908+0000 7f8c8a200640 1 --2- 172.21.15.12:0/3222157078 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5601872910c0 0x560187298590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:16:00.912 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:00.908+0000 7f8c8a200640 1 -- 172.21.15.12:0/3222157078 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560186f52790 con 0x560187293b00 2024-09-11T00:16:00.912 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:00.908+0000 7f8c89800640 1 --2- 172.21.15.12:0/3222157078 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56018728e790 0x560187297ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:16:00.913 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:00.908+0000 7f8c88e00640 1 --2- 172.21.15.12:0/3222157078 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5601872910c0 0x560187298590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:16:00.913 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:00.908+0000 7f8c8a200640 1 --2- 172.21.15.12:0/3222157078 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560187293b00 0x560187299160 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f8c7c009be0 tx=0x7f8c7c009500 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e937b5e1ec30fab0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:16:00.913 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:00.908+0000 7f8c6b400640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-09-11T00:16:00.913 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:00.908+0000 7f8c6b400640 1 -- 172.21.15.12:0/3222157078 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c7c04f070 con 0x560187293b00 2024-09-11T00:16:00.913 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:00.908+0000 7f8c6b400640 1 -- 172.21.15.12:0/3222157078 <== mon.2 v2:172.21.15.165:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9208+0+0 (secure 0 0 0) 0x7f8c7c004450 con 0x560187293b00 2024-09-11T00:16:00.913 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:00.908+0000 7f8c8f09c740 1 -- 172.21.15.12:0/3222157078 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x560187203600 con 0x560187293b00 2024-09-11T00:16:00.915 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:00.909+0000 7f8c6b400640 1 -- 172.21.15.12:0/3222157078 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f8c7c05b050 con 0x560187293b00 2024-09-11T00:16:00.915 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:00.909+0000 7f8c8f09c740 10 client.26528 fetch_fsmap learned FSMap version 41 2024-09-11T00:16:00.915 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:00.909+0000 7f8c8f09c740 1 -- 172.21.15.12:0/3222157078 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x560186fcb620 con 0x560187293b00 2024-09-11T00:16:00.915 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:00.909+0000 7f8c6b400640 1 -- 172.21.15.12:0/3222157078 <== mon.2 v2:172.21.15.165:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x560187203600 con 0x560187293b00 2024-09-11T00:16:00.915 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:00.909+0000 7f8c6b400640 1 -- 172.21.15.12:0/3222157078 <== mon.2 v2:172.21.15.165:3300/0 5 ==== fsmap(e 41) ==== 8370+0+0 (secure 0 0 0) 0x7f8c7c04bc70 con 0x560187293b00 2024-09-11T00:16:00.915 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:00.909+0000 7f8c8f09c740 10 client.26528 fetch_fsmap finished waiting for FSMap version 41 2024-09-11T00:16:00.915 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:00.909+0000 7f8c8f09c740 10 client.26528 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T00:16:00.915 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:00.909+0000 7f8c8f09c740 20 client.26528 populate_metadata read hostname 'smithi012' 2024-09-11T00:16:00.915 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:00.909+0000 7f8c8f09c740 1 --2- 172.21.15.12:0/3222157078 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x5601872a37f0 0x5601872a5be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:16:00.916 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:00.909+0000 7f8c8f09c740 4 client.26528 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T00:16:00.916 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:00.909+0000 7f8c8f09c740 1 -- 172.21.15.12:0/3222157078 --> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5601871ff0b0 con 0x5601872a37f0 2024-09-11T00:16:00.916 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:00.910+0000 7f8c89800640 1 --2- 172.21.15.12:0/3222157078 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x5601872a37f0 0x5601872a5be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:16:00.916 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:00.910+0000 7f8c89800640 1 --2- 172.21.15.12:0/3222157078 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x5601872a37f0 0x5601872a5be0 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=af948073beab3ee server_cookie=26c3a26ad0aa1d58 in_seq=0 out_seq=0 2024-09-11T00:16:00.916 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:00.910+0000 7f8c6b400640 10 client.26528 ms_handle_connect on v2:172.21.15.165:6838/501312984 2024-09-11T00:16:00.916 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:00.911+0000 7f8c6b400640 1 -- 172.21.15.12:0/3222157078 <== mds.0 v2:172.21.15.165:6838/501312984 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5601871ff0b0 con 0x5601872a37f0 2024-09-11T00:16:00.916 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:00.911+0000 7f8c6b400640 10 client.26528 handle_command_reply: tid=0 2024-09-11T00:16:00.916 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:00.911+0000 7f8c8f09c740 1 client.26528 shutdown 2024-09-11T00:16:00.916 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:00.912+0000 7f8c8f09c740 1 -- 172.21.15.12:0/3222157078 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560187293b00 msgr2=0x560187299160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:16:00.916 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:00.912+0000 7f8c8f09c740 1 --2- 172.21.15.12:0/3222157078 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560187293b00 0x560187299160 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f8c7c009be0 tx=0x7f8c7c009500 comp rx=0 tx=0).stop 2024-09-11T00:16:00.916 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:00.912+0000 7f8c8f09c740 1 -- 172.21.15.12:0/3222157078 shutdown_connections 2024-09-11T00:16:00.916 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:00.912+0000 7f8c8f09c740 1 --2- 172.21.15.12:0/3222157078 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x5601872a37f0 0x5601872a5be0 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:16:00.917 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:00.912+0000 7f8c8f09c740 1 --2- 172.21.15.12:0/3222157078 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560187293b00 0x560187299160 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:16:00.917 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:00.912+0000 7f8c8f09c740 1 --2- 172.21.15.12:0/3222157078 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5601872910c0 0x560187298590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:16:00.917 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:00.912+0000 7f8c8f09c740 1 --2- 172.21.15.12:0/3222157078 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56018728e790 0x560187297ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:16:00.917 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:00.912+0000 7f8c6b400640 0 client.26528 ms_handle_reset on v2:172.21.15.165:6838/501312984 2024-09-11T00:16:00.917 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:00.912+0000 7f8c8f09c740 1 -- 172.21.15.12:0/3222157078 >> 172.21.15.12:0/3222157078 conn(0x5601872898e0 msgr2=0x560187288a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:16:00.917 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:00.912+0000 7f8c8f09c740 1 -- 172.21.15.12:0/3222157078 shutdown_connections 2024-09-11T00:16:00.917 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:00.912+0000 7f8c8f09c740 1 -- 172.21.15.12:0/3222157078 wait complete. 2024-09-11T00:16:00.917 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:00.912+0000 7f8c8f09c740 20 client.26528 trim_cache size 0 max 16384 2024-09-11T00:16:00.932 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:00.930+0000 7f8c8f09c740 1 Processor -- start 2024-09-11T00:16:00.932 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:00.930+0000 7f8c8f09c740 1 -- start start 2024-09-11T00:16:00.934 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:00.931+0000 7f8c8f09c740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56018728e790 0x56018728a140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:16:00.934 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:00.931+0000 7f8c8f09c740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5601872910c0 0x56018728a680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:16:00.934 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:00.931+0000 7f8c8f09c740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560187293b00 0x56018728b710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:16:00.934 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:00.931+0000 7f8c8f09c740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f8c841ad8d0 con 0x560187293b00 2024-09-11T00:16:00.934 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:00.931+0000 7f8c8f09c740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f8c841af410 con 0x5601872910c0 2024-09-11T00:16:00.934 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:00.931+0000 7f8c8f09c740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f8c841adfa0 con 0x56018728e790 2024-09-11T00:16:00.934 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:00.931+0000 7f8c88e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5601872910c0 0x56018728a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:16:00.934 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:00.931+0000 7f8c88e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5601872910c0 0x56018728a680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.12:60512/0 (socket says 172.21.15.12:60512) 2024-09-11T00:16:00.934 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:00.931+0000 7f8c88e00640 1 -- 172.21.15.12:0/3856793776 learned_addr learned my addr 172.21.15.12:0/3856793776 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:16:00.935 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:00.931+0000 7f8c8a200640 1 --2- 172.21.15.12:0/3856793776 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560187293b00 0x56018728b710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:16:00.935 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:00.931+0000 7f8c89800640 1 --2- 172.21.15.12:0/3856793776 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56018728e790 0x56018728a140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:16:00.935 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:00.931+0000 7f8c88e00640 1 -- 172.21.15.12:0/3856793776 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56018728e790 msgr2=0x56018728a140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:16:00.935 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:00.931+0000 7f8c88e00640 1 --2- 172.21.15.12:0/3856793776 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56018728e790 0x56018728a140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:16:00.935 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:00.931+0000 7f8c88e00640 1 -- 172.21.15.12:0/3856793776 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560187293b00 msgr2=0x56018728b710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:16:00.935 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:00.931+0000 7f8c88e00640 1 --2- 172.21.15.12:0/3856793776 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560187293b00 0x56018728b710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:16:00.935 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:00.931+0000 7f8c88e00640 1 -- 172.21.15.12:0/3856793776 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560187293820 con 0x5601872910c0 2024-09-11T00:16:00.935 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:00.932+0000 7f8c88e00640 1 --2- 172.21.15.12:0/3856793776 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5601872910c0 0x56018728a680 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f8c70004770 tx=0x7f8c700abf80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ce5fbf5d269d4c0f server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:16:00.935 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:00.932+0000 7f8c76000640 1 -- 172.21.15.12:0/3856793776 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c700b1030 con 0x5601872910c0 2024-09-11T00:16:00.935 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:00.932+0000 7f8c76000640 1 -- 172.21.15.12:0/3856793776 <== mon.2 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8c700a6e50 con 0x5601872910c0 2024-09-11T00:16:00.935 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:00.932+0000 7f8c76000640 1 -- 172.21.15.12:0/3856793776 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c700037a0 con 0x5601872910c0 2024-09-11T00:16:00.939 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:00.933+0000 7f8c8f09c740 1 -- 172.21.15.12:0/3856793776 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5601872910c0 msgr2=0x56018728a680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:16:00.939 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:00.933+0000 7f8c8f09c740 1 --2- 172.21.15.12:0/3856793776 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5601872910c0 0x56018728a680 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f8c70004770 tx=0x7f8c700abf80 comp rx=0 tx=0).stop 2024-09-11T00:16:00.939 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:00.933+0000 7f8c8f09c740 1 -- 172.21.15.12:0/3856793776 shutdown_connections 2024-09-11T00:16:00.939 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:00.933+0000 7f8c8f09c740 1 --2- 172.21.15.12:0/3856793776 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560187293b00 0x56018728b710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:16:00.939 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:00.933+0000 7f8c8f09c740 1 --2- 172.21.15.12:0/3856793776 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5601872910c0 0x56018728a680 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:16:00.939 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:00.933+0000 7f8c8f09c740 1 --2- 172.21.15.12:0/3856793776 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56018728e790 0x56018728a140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:16:00.939 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:00.933+0000 7f8c8f09c740 1 -- 172.21.15.12:0/3856793776 >> 172.21.15.12:0/3856793776 conn(0x5601872898e0 msgr2=0x560187293f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:16:00.939 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:00.933+0000 7f8c8f09c740 1 -- 172.21.15.12:0/3856793776 shutdown_connections 2024-09-11T00:16:00.940 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:00.933+0000 7f8c8f09c740 1 -- 172.21.15.12:0/3856793776 wait complete. 2024-09-11T00:16:00.940 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:00.933+0000 7f8c8f09c740 1 -- start start 2024-09-11T00:16:00.940 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:00.933+0000 7f8c8f09c740 1 Processor -- start 2024-09-11T00:16:00.940 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:00.934+0000 7f8c8f09c740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56018728e790 0x56018729fc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:16:00.940 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:00.934+0000 7f8c8f09c740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5601872910c0 0x56018729ea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:16:00.940 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:00.934+0000 7f8c8f09c740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560187293b00 0x56018729f160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:16:00.940 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:00.934+0000 7f8c8f09c740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f8c841aed40 con 0x56018728e790 2024-09-11T00:16:00.940 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:00.934+0000 7f8c8f09c740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f8c841ae670 con 0x5601872910c0 2024-09-11T00:16:00.940 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:00.934+0000 7f8c8f09c740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f8c841ab8b0 con 0x560187293b00 2024-09-11T00:16:00.940 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:00.934+0000 7f8c8f09c740 10 client.0 _ll_register_callbacks cb 0x7f8c78005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:16:00.940 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:00.934+0000 7f8c88e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5601872910c0 0x56018729ea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:16:00.940 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:00.934+0000 7f8c88e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5601872910c0 0x56018729ea70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.12:60526/0 (socket says 172.21.15.12:60526) 2024-09-11T00:16:00.941 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:00.934+0000 7f8c88e00640 1 -- 172.21.15.12:0/3194302136 learned_addr learned my addr 172.21.15.12:0/3194302136 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:16:00.941 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:00.934+0000 7f8c8a200640 1 --2- 172.21.15.12:0/3194302136 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560187293b00 0x56018729f160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:16:00.941 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:00.934+0000 7f8c88e00640 1 -- 172.21.15.12:0/3194302136 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560187293b00 msgr2=0x56018729f160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:16:00.941 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:00.934+0000 7f8c88e00640 1 --2- 172.21.15.12:0/3194302136 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560187293b00 0x56018729f160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:16:00.941 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:00.934+0000 7f8c88e00640 1 -- 172.21.15.12:0/3194302136 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56018728e790 msgr2=0x56018729fc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:16:00.941 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:00.934+0000 7f8c88e00640 1 --2- 172.21.15.12:0/3194302136 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56018728e790 0x56018729fc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:16:00.941 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:00.934+0000 7f8c88e00640 1 -- 172.21.15.12:0/3194302136 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56018729d4a0 con 0x5601872910c0 2024-09-11T00:16:00.941 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:00.935+0000 7f8c8a200640 1 --2- 172.21.15.12:0/3194302136 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560187293b00 0x56018729f160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T00:16:00.941 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:00.935+0000 7f8c88e00640 1 --2- 172.21.15.12:0/3194302136 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5601872910c0 0x56018729ea70 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f8c700a8030 tx=0x7f8c70003cd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6542f7a50a16b4cf server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:16:00.941 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:00.935+0000 7f8c6b400640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-09-11T00:16:00.941 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:00.935+0000 7f8c6b400640 1 -- 172.21.15.12:0/3194302136 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c700b1030 con 0x5601872910c0 2024-09-11T00:16:00.941 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:00.935+0000 7f8c8f09c740 1 -- 172.21.15.12:0/3194302136 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8c700a6e50 con 0x5601872910c0 2024-09-11T00:16:00.941 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:00.935+0000 7f8c6b400640 1 -- 172.21.15.12:0/3194302136 <== mon.2 v2:172.21.15.165:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9208+0+0 (secure 0 0 0) 0x7f8c700a89c0 con 0x5601872910c0 2024-09-11T00:16:00.942 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:00.935+0000 7f8c6b400640 1 -- 172.21.15.12:0/3194302136 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f8c700ac750 con 0x5601872910c0 2024-09-11T00:16:00.942 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:00.935+0000 7f8c8f09c740 10 client.26534 fetch_fsmap learned FSMap version 41 2024-09-11T00:16:00.942 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:00.936+0000 7f8c8f09c740 1 -- 172.21.15.12:0/3194302136 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x560187310a50 con 0x5601872910c0 2024-09-11T00:16:00.942 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:00.936+0000 7f8c6b400640 1 -- 172.21.15.12:0/3194302136 <== mon.2 v2:172.21.15.165:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8c700a6e50 con 0x5601872910c0 2024-09-11T00:16:00.942 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:00.936+0000 7f8c6b400640 1 -- 172.21.15.12:0/3194302136 <== mon.2 v2:172.21.15.165:3300/0 5 ==== fsmap(e 41) ==== 8370+0+0 (secure 0 0 0) 0x7f8c700a80a0 con 0x5601872910c0 2024-09-11T00:16:00.942 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:00.936+0000 7f8c8f09c740 10 client.26534 fetch_fsmap finished waiting for FSMap version 41 2024-09-11T00:16:00.942 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:00.936+0000 7f8c8f09c740 10 client.26534 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T00:16:00.942 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:00.936+0000 7f8c8f09c740 20 client.26534 populate_metadata read hostname 'smithi012' 2024-09-11T00:16:00.942 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:00.936+0000 7f8c8f09c740 1 --2- 172.21.15.12:0/3194302136 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x560187311300 0x560187313720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:16:00.942 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:00.936+0000 7f8c8f09c740 4 client.26534 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T00:16:00.942 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:00.936+0000 7f8c8f09c740 1 -- 172.21.15.12:0/3194302136 --> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5601871fc4e0 con 0x560187311300 2024-09-11T00:16:00.942 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:00.937+0000 7f8c89800640 1 --2- 172.21.15.12:0/3194302136 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x560187311300 0x560187313720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:16:00.943 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:00.937+0000 7f8c89800640 1 --2- 172.21.15.12:0/3194302136 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x560187311300 0x560187313720 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a2065a2715fab7a9 server_cookie=5517c0f8409e4f37 in_seq=0 out_seq=0 2024-09-11T00:16:00.943 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:00.937+0000 7f8c6b400640 10 client.26534 ms_handle_connect on v2:172.21.15.165:6838/501312984 2024-09-11T00:16:01.013 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:01.010+0000 7f8c6b400640 1 -- 172.21.15.12:0/3194302136 <== mds.0 v2:172.21.15.165:6838/501312984 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5601871fc4e0 con 0x560187311300 2024-09-11T00:16:01.013 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:01.010+0000 7f8c6b400640 10 client.26534 handle_command_reply: tid=0 2024-09-11T00:16:01.013 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:01.010+0000 7f8c8f09c740 1 client.26534 shutdown 2024-09-11T00:16:01.013 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:01.010+0000 7f8c8f09c740 1 -- 172.21.15.12:0/3194302136 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5601872910c0 msgr2=0x56018729ea70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:16:01.013 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:01.010+0000 7f8c8f09c740 1 --2- 172.21.15.12:0/3194302136 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5601872910c0 0x56018729ea70 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f8c700a8030 tx=0x7f8c70003cd0 comp rx=0 tx=0).stop 2024-09-11T00:16:01.014 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:01.010+0000 7f8c8f09c740 1 -- 172.21.15.12:0/3194302136 shutdown_connections 2024-09-11T00:16:01.014 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:01.010+0000 7f8c8f09c740 1 --2- 172.21.15.12:0/3194302136 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x560187311300 0x560187313720 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:16:01.014 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:01.010+0000 7f8c8f09c740 1 --2- 172.21.15.12:0/3194302136 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560187293b00 0x56018729f160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:16:01.014 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:01.010+0000 7f8c8f09c740 1 --2- 172.21.15.12:0/3194302136 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5601872910c0 0x56018729ea70 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:16:01.014 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:01.010+0000 7f8c6b400640 0 client.26534 ms_handle_reset on v2:172.21.15.165:6838/501312984 2024-09-11T00:16:01.014 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:01.010+0000 7f8c8f09c740 1 --2- 172.21.15.12:0/3194302136 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56018728e790 0x56018729fc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:16:01.014 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:01.011+0000 7f8c8f09c740 1 -- 172.21.15.12:0/3194302136 >> 172.21.15.12:0/3194302136 conn(0x5601872898e0 msgr2=0x560187293ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:16:01.014 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:01.011+0000 7f8c8f09c740 1 -- 172.21.15.12:0/3194302136 shutdown_connections 2024-09-11T00:16:01.014 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:01.011+0000 7f8c8f09c740 1 -- 172.21.15.12:0/3194302136 wait complete. 2024-09-11T00:16:01.014 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:01.011+0000 7f8c8f09c740 20 client.26534 trim_cache size 0 max 16384 2024-09-11T00:16:01.015 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:{ 2024-09-11T00:16:01.015 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "message": "", 2024-09-11T00:16:01.015 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "return_code": 0 2024-09-11T00:16:01.015 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:} 2024-09-11T00:16:01.016 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:01.012+0000 7f8c8c200640 1 -- 172.21.15.12:0/3343344591 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f8c6c0b52e0 msgr2=0x7f8c6c0b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:16:01.016 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:01.012+0000 7f8c8c200640 1 --2- 172.21.15.12:0/3343344591 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f8c6c0b52e0 0x7f8c6c0b77a0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f8c7c006010 tx=0x7f8c7c030960 comp rx=0 tx=0).stop 2024-09-11T00:16:01.016 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:01.012+0000 7f8c8c200640 1 -- 172.21.15.12:0/3343344591 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8c84105cf0 msgr2=0x7f8c8419b500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:16:01.016 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:01.012+0000 7f8c8c200640 1 --2- 172.21.15.12:0/3343344591 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8c84105cf0 0x7f8c8419b500 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f8c7000e9c0 tx=0x7f8c7000ee80 comp rx=0 tx=0).stop 2024-09-11T00:16:01.016 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:01.012+0000 7f8c8c200640 1 -- 172.21.15.12:0/3343344591 shutdown_connections 2024-09-11T00:16:01.016 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:01.012+0000 7f8c8c200640 1 --2- 172.21.15.12:0/3343344591 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f8c6c0b52e0 0x7f8c6c0b77a0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:16:01.016 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:01.012+0000 7f8c8c200640 1 --2- 172.21.15.12:0/3343344591 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8c841066b0 0x7f8c8419c130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:16:01.016 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:01.012+0000 7f8c8c200640 1 --2- 172.21.15.12:0/3343344591 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8c84105cf0 0x7f8c8419b500 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:16:01.017 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:01.012+0000 7f8c8c200640 1 --2- 172.21.15.12:0/3343344591 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8c84104ba0 0x7f8c8419aeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:16:01.017 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:01.012+0000 7f8c8c200640 1 -- 172.21.15.12:0/3343344591 >> 172.21.15.12:0/3343344591 conn(0x7f8c84100460 msgr2=0x7f8c84109010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:16:01.017 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:01.012+0000 7f8c8c200640 1 -- 172.21.15.12:0/3343344591 shutdown_connections 2024-09-11T00:16:01.017 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:01.012+0000 7f8c8c200640 1 -- 172.21.15.12:0/3343344591 wait complete. 2024-09-11T00:16:01.667 INFO:tasks.workunit.client.0.smithi012.stdout: 1 341431 3.69 MB/sec execute 358 sec latency 38.119 ms 2024-09-11T00:16:01.813 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:16:01 smithi012 ceph-mon[30333]: pgmap v667: 129 pgs: 129 active+clean; 84 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 1.1 MiB/s wr, 301 op/s 2024-09-11T00:16:01.830 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:16:01 smithi178 ceph-mon[35919]: pgmap v667: 129 pgs: 129 active+clean; 84 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 1.1 MiB/s wr, 301 op/s 2024-09-11T00:16:02.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:16:01 smithi165 ceph-mon[35356]: pgmap v667: 129 pgs: 129 active+clean; 84 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 1.1 MiB/s wr, 301 op/s 2024-09-11T00:16:02.667 INFO:tasks.workunit.client.0.smithi012.stdout: 1 343243 3.70 MB/sec execute 359 sec latency 195.227 ms 2024-09-11T00:16:03.667 INFO:tasks.workunit.client.0.smithi012.stdout: 1 344336 3.72 MB/sec execute 360 sec latency 486.024 ms 2024-09-11T00:16:03.813 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:16:03 smithi012 ceph-mon[30333]: pgmap v668: 129 pgs: 129 active+clean; 85 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 1.0 MiB/s wr, 278 op/s 2024-09-11T00:16:03.831 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:16:03 smithi178 ceph-mon[35919]: pgmap v668: 129 pgs: 129 active+clean; 85 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 1.0 MiB/s wr, 278 op/s 2024-09-11T00:16:04.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:16:03 smithi165 ceph-mon[35356]: pgmap v668: 129 pgs: 129 active+clean; 85 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 1.0 MiB/s wr, 278 op/s 2024-09-11T00:16:04.667 INFO:tasks.workunit.client.0.smithi012.stdout: 1 345523 3.72 MB/sec execute 361 sec latency 17.372 ms 2024-09-11T00:16:05.667 INFO:tasks.workunit.client.0.smithi012.stdout: 1 347584 3.74 MB/sec execute 362 sec latency 67.638 ms 2024-09-11T00:16:05.813 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:16:05 smithi012 ceph-mon[30333]: pgmap v669: 129 pgs: 129 active+clean; 85 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 889 KiB/s wr, 230 op/s 2024-09-11T00:16:05.831 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:16:05 smithi178 ceph-mon[35919]: pgmap v669: 129 pgs: 129 active+clean; 85 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 889 KiB/s wr, 230 op/s 2024-09-11T00:16:06.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:16:05 smithi165 ceph-mon[35356]: pgmap v669: 129 pgs: 129 active+clean; 85 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 889 KiB/s wr, 230 op/s 2024-09-11T00:16:06.667 INFO:tasks.workunit.client.0.smithi012.stdout: 1 348554 3.74 MB/sec execute 363 sec latency 16.262 ms 2024-09-11T00:16:07.667 INFO:tasks.workunit.client.0.smithi012.stdout: 1 350582 3.76 MB/sec execute 364 sec latency 11.712 ms 2024-09-11T00:16:07.831 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:16:07 smithi178 ceph-mon[35919]: pgmap v670: 129 pgs: 129 active+clean; 85 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 1.2 MiB/s wr, 365 op/s 2024-09-11T00:16:08.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:16:07 smithi165 ceph-mon[35356]: pgmap v670: 129 pgs: 129 active+clean; 85 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 1.2 MiB/s wr, 365 op/s 2024-09-11T00:16:08.063 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:16:07 smithi012 ceph-mon[30333]: pgmap v670: 129 pgs: 129 active+clean; 85 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 1.2 MiB/s wr, 365 op/s 2024-09-11T00:16:08.667 INFO:tasks.workunit.client.0.smithi012.stdout: 1 351762 3.77 MB/sec execute 365 sec latency 11.786 ms 2024-09-11T00:16:09.667 INFO:tasks.workunit.client.0.smithi012.stdout: 1 353716 3.78 MB/sec execute 366 sec latency 16.963 ms 2024-09-11T00:16:09.831 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:16:09 smithi178 ceph-mon[35919]: pgmap v671: 129 pgs: 129 active+clean; 85 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 823 KiB/s wr, 259 op/s 2024-09-11T00:16:09.833 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:16:09 smithi165 ceph-mon[35356]: pgmap v671: 129 pgs: 129 active+clean; 85 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 823 KiB/s wr, 259 op/s 2024-09-11T00:16:10.063 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:16:09 smithi012 ceph-mon[30333]: pgmap v671: 129 pgs: 129 active+clean; 85 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 823 KiB/s wr, 259 op/s 2024-09-11T00:16:10.667 INFO:tasks.workunit.client.0.smithi012.stdout: 1 355096 3.80 MB/sec execute 367 sec latency 11.565 ms 2024-09-11T00:16:11.667 INFO:tasks.workunit.client.0.smithi012.stdout: 1 356996 3.81 MB/sec execute 368 sec latency 16.682 ms 2024-09-11T00:16:11.831 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:16:11 smithi178 ceph-mon[35919]: pgmap v672: 129 pgs: 129 active+clean; 87 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 MiB/s rd, 1.1 MiB/s wr, 316 op/s 2024-09-11T00:16:11.864 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:16:11 smithi012 ceph-mon[30333]: pgmap v672: 129 pgs: 129 active+clean; 87 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 MiB/s rd, 1.1 MiB/s wr, 316 op/s 2024-09-11T00:16:12.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:16:11 smithi165 ceph-mon[35356]: pgmap v672: 129 pgs: 129 active+clean; 87 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 MiB/s rd, 1.1 MiB/s wr, 316 op/s 2024-09-11T00:16:12.667 INFO:tasks.workunit.client.0.smithi012.stdout: 1 358604 3.83 MB/sec execute 369 sec latency 13.168 ms 2024-09-11T00:16:13.667 INFO:tasks.workunit.client.0.smithi012.stdout: 1 360229 3.84 MB/sec execute 370 sec latency 10.806 ms 2024-09-11T00:16:13.831 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:16:13 smithi178 ceph-mon[35919]: pgmap v673: 129 pgs: 129 active+clean; 88 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 MiB/s rd, 931 KiB/s wr, 301 op/s 2024-09-11T00:16:14.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:16:13 smithi165 ceph-mon[35356]: pgmap v673: 129 pgs: 129 active+clean; 88 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 MiB/s rd, 931 KiB/s wr, 301 op/s 2024-09-11T00:16:14.063 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:16:13 smithi012 ceph-mon[30333]: pgmap v673: 129 pgs: 129 active+clean; 88 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 MiB/s rd, 931 KiB/s wr, 301 op/s 2024-09-11T00:16:14.139 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:14.137+0000 7f27aee00640 1 -- 172.21.15.12:0/936661314 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f27a81027e0 msgr2=0x7f27a8102c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:16:14.140 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:14.137+0000 7f27aee00640 1 --2- 172.21.15.12:0/936661314 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f27a81027e0 0x7f27a8102c60 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f2794009900 tx=0x7f279402a3f0 comp rx=0 tx=0).stop 2024-09-11T00:16:14.140 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:14.137+0000 7f27aee00640 1 -- 172.21.15.12:0/936661314 shutdown_connections 2024-09-11T00:16:14.140 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:14.137+0000 7f27aee00640 1 --2- 172.21.15.12:0/936661314 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f27a81031a0 0x7f27a8109b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:16:14.140 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:14.137+0000 7f27aee00640 1 --2- 172.21.15.12:0/936661314 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f27a81027e0 0x7f27a8102c60 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:16:14.140 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:14.137+0000 7f27aee00640 1 --2- 172.21.15.12:0/936661314 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f27a8101690 0x7f27a8101a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:16:14.140 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:14.137+0000 7f27aee00640 1 -- 172.21.15.12:0/936661314 >> 172.21.15.12:0/936661314 conn(0x7f27a80fcf30 msgr2=0x7f27a80ff370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:16:14.140 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:14.137+0000 7f27aee00640 1 -- 172.21.15.12:0/936661314 shutdown_connections 2024-09-11T00:16:14.140 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:14.137+0000 7f27aee00640 1 -- 172.21.15.12:0/936661314 wait complete. 2024-09-11T00:16:14.143 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:14.138+0000 7f27aee00640 1 Processor -- start 2024-09-11T00:16:14.143 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:14.138+0000 7f27aee00640 1 -- start start 2024-09-11T00:16:14.143 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:14.138+0000 7f27aee00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f27a8101690 0x7f27a8198b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:16:14.143 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:14.138+0000 7f27aee00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f27a81027e0 0x7f27a81991a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:16:14.143 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:14.138+0000 7f27aee00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f27a81031a0 0x7f27a8199dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:16:14.143 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:14.138+0000 7f27aee00640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f27a810e470 con 0x7f27a81027e0 2024-09-11T00:16:14.143 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:14.138+0000 7f27aee00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f27a810e2f0 con 0x7f27a81031a0 2024-09-11T00:16:14.143 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:14.138+0000 7f27aee00640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f27a810e5f0 con 0x7f27a8101690 2024-09-11T00:16:14.144 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:14.139+0000 7f27ada00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f27a8101690 0x7f27a8198b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:16:14.144 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:14.139+0000 7f27ad000640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f27a81027e0 0x7f27a81991a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:16:14.144 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:14.139+0000 7f27ae400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f27a81031a0 0x7f27a8199dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:16:14.144 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:14.139+0000 7f27ae400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f27a81031a0 0x7f27a8199dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.12:39528/0 (socket says 172.21.15.12:39528) 2024-09-11T00:16:14.144 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:14.139+0000 7f27ae400640 1 -- 172.21.15.12:0/3475730974 learned_addr learned my addr 172.21.15.12:0/3475730974 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:16:14.144 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:14.139+0000 7f27ad000640 1 -- 172.21.15.12:0/3475730974 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f27a8101690 msgr2=0x7f27a8198b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:16:14.144 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:14.139+0000 7f27ad000640 1 --2- 172.21.15.12:0/3475730974 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f27a8101690 0x7f27a8198b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:16:14.144 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:14.139+0000 7f27ad000640 1 -- 172.21.15.12:0/3475730974 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f27a81031a0 msgr2=0x7f27a8199dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:16:14.144 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:14.139+0000 7f27ad000640 1 --2- 172.21.15.12:0/3475730974 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f27a81031a0 0x7f27a8199dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:16:14.144 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:14.139+0000 7f27ad000640 1 -- 172.21.15.12:0/3475730974 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27a819a690 con 0x7f27a81027e0 2024-09-11T00:16:14.144 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:14.139+0000 7f27ae400640 1 --2- 172.21.15.12:0/3475730974 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f27a81031a0 0x7f27a8199dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:16:14.144 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:14.140+0000 7f27ad000640 1 --2- 172.21.15.12:0/3475730974 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f27a81027e0 0x7f27a81991a0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f279402a910 tx=0x7f2794003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:16:14.144 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:14.140+0000 7f27a6000640 1 -- 172.21.15.12:0/3475730974 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f279403f070 con 0x7f27a81027e0 2024-09-11T00:16:14.144 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:14.140+0000 7f27a6000640 1 -- 172.21.15.12:0/3475730974 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2794031bf0 con 0x7f27a81027e0 2024-09-11T00:16:14.145 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:14.140+0000 7f27aee00640 1 -- 172.21.15.12:0/3475730974 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f27a819a920 con 0x7f27a81027e0 2024-09-11T00:16:14.145 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:14.140+0000 7f27a6000640 1 -- 172.21.15.12:0/3475730974 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2794035030 con 0x7f27a81027e0 2024-09-11T00:16:14.145 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:14.140+0000 7f27aee00640 1 -- 172.21.15.12:0/3475730974 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27a81a0e70 con 0x7f27a81027e0 2024-09-11T00:16:14.146 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:14.144+0000 7f27b3436740 1 Processor -- start 2024-09-11T00:16:14.148 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:14.144+0000 7f27b3436740 1 -- start start 2024-09-11T00:16:14.148 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:14.145+0000 7f27b3436740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e951cb1970 0x55e951cb3d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:16:14.148 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:14.145+0000 7f27b3436740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e951cb42a0 0x55e951cb6690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:16:14.148 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:14.145+0000 7f27b3436740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e951cb6ce0 0x55e951cb91a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:16:14.148 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:14.145+0000 7f27b3436740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55e951c246b0 con 0x55e951cb1970 2024-09-11T00:16:14.149 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:14.145+0000 7f27b3436740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55e951c13ae0 con 0x55e951cb42a0 2024-09-11T00:16:14.149 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:14.145+0000 7f27b3436740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55e951c178f0 con 0x55e951cb6ce0 2024-09-11T00:16:14.149 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:14.145+0000 7f27ada00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e951cb42a0 0x55e951cb6690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:16:14.149 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:14.145+0000 7f27ada00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e951cb42a0 0x55e951cb6690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.12:39530/0 (socket says 172.21.15.12:39530) 2024-09-11T00:16:14.149 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:14.145+0000 7f27ada00640 1 -- 172.21.15.12:0/824956594 learned_addr learned my addr 172.21.15.12:0/824956594 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:16:14.149 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:14.145+0000 7f27ae400640 1 --2- 172.21.15.12:0/824956594 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e951cb1970 0x55e951cb3d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:16:14.149 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:14.145+0000 7f27a6000640 1 -- 172.21.15.12:0/3475730974 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7f2794031740 con 0x7f27a81027e0 2024-09-11T00:16:14.149 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:14.145+0000 7f27ada00640 1 -- 172.21.15.12:0/824956594 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e951cb6ce0 msgr2=0x55e951cb91a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:16:14.149 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:14.145+0000 7f27ad000640 1 --2- 172.21.15.12:0/824956594 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e951cb6ce0 0x55e951cb91a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:16:14.149 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:14.146+0000 7f27ada00640 1 --2- 172.21.15.12:0/824956594 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e951cb6ce0 0x55e951cb91a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:16:14.150 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:14.146+0000 7f27ada00640 1 -- 172.21.15.12:0/824956594 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e951cb1970 msgr2=0x55e951cb3d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:16:14.150 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:14.146+0000 7f27ada00640 1 --2- 172.21.15.12:0/824956594 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e951cb1970 0x55e951cb3d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:16:14.150 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:14.146+0000 7f27ada00640 1 -- 172.21.15.12:0/824956594 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e951be0780 con 0x55e951cb42a0 2024-09-11T00:16:14.150 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:14.146+0000 7f27ae400640 1 --2- 172.21.15.12:0/824956594 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e951cb1970 0x55e951cb3d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:16:14.150 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:14.146+0000 7f27ad000640 1 --2- 172.21.15.12:0/824956594 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e951cb6ce0 0x55e951cb91a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T00:16:14.150 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:14.146+0000 7f27ada00640 1 --2- 172.21.15.12:0/824956594 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e951cb42a0 0x55e951cb6690 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f279c009cf0 tx=0x7f279c011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=260206cc06110541 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:16:14.150 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:14.146+0000 7f279aa00640 1 -- 172.21.15.12:0/824956594 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f279c004030 con 0x55e951cb42a0 2024-09-11T00:16:14.150 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:14.146+0000 7f279aa00640 1 -- 172.21.15.12:0/824956594 <== mon.2 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f279c0041d0 con 0x55e951cb42a0 2024-09-11T00:16:14.150 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:14.146+0000 7f279aa00640 1 -- 172.21.15.12:0/824956594 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f279c019710 con 0x55e951cb42a0 2024-09-11T00:16:14.155 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:14.147+0000 7f27b3436740 1 -- 172.21.15.12:0/824956594 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e951cb42a0 msgr2=0x55e951cb6690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:16:14.155 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:14.147+0000 7f27b3436740 1 --2- 172.21.15.12:0/824956594 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e951cb42a0 0x55e951cb6690 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f279c009cf0 tx=0x7f279c011810 comp rx=0 tx=0).stop 2024-09-11T00:16:14.155 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:14.147+0000 7f27b3436740 1 -- 172.21.15.12:0/824956594 shutdown_connections 2024-09-11T00:16:14.155 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:14.147+0000 7f27b3436740 1 --2- 172.21.15.12:0/824956594 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e951cb6ce0 0x55e951cb91a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:16:14.155 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:14.147+0000 7f27b3436740 1 --2- 172.21.15.12:0/824956594 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e951cb42a0 0x55e951cb6690 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:16:14.156 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:14.147+0000 7f27b3436740 1 --2- 172.21.15.12:0/824956594 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e951cb1970 0x55e951cb3d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:16:14.156 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:14.147+0000 7f27b3436740 1 -- 172.21.15.12:0/824956594 >> 172.21.15.12:0/824956594 conn(0x55e951cacac0 msgr2=0x55e951cad850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:16:14.156 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:14.147+0000 7f27b3436740 1 -- 172.21.15.12:0/824956594 shutdown_connections 2024-09-11T00:16:14.156 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:14.147+0000 7f27b3436740 1 -- 172.21.15.12:0/824956594 wait complete. 2024-09-11T00:16:14.156 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:14.147+0000 7f27a6000640 1 --2- 172.21.15.12:0/3475730974 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f27900b51f0 0x7f27900b76b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:16:14.156 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:14.147+0000 7f27a6000640 1 -- 172.21.15.12:0/3475730974 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9208+0+0 (secure 0 0 0) 0x7f2794106910 con 0x7f27a81027e0 2024-09-11T00:16:14.156 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:14.148+0000 7f27b3436740 1 -- start start 2024-09-11T00:16:14.156 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:14.148+0000 7f27ada00640 1 --2- 172.21.15.12:0/3475730974 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f27900b51f0 0x7f27900b76b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:16:14.156 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:14.148+0000 7f27b3436740 1 Processor -- start 2024-09-11T00:16:14.156 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:14.148+0000 7f27b3436740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e951cb1970 0x55e951cbb000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:16:14.156 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:14.148+0000 7f27b3436740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e951cb42a0 0x55e951cbb6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:16:14.157 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:14.148+0000 7f27b3436740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e951cb6ce0 0x55e951cbc270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:16:14.157 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:14.148+0000 7f27b3436740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55e951b4bab0 con 0x55e951cb6ce0 2024-09-11T00:16:14.157 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:14.148+0000 7f27b3436740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55e951bba570 con 0x55e951cb1970 2024-09-11T00:16:14.157 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:14.148+0000 7f27b3436740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55e951cb9cc0 con 0x55e951cb42a0 2024-09-11T00:16:14.157 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:14.148+0000 7f27b3436740 10 client.0 _ll_register_callbacks cb 0x55e951bc9100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:16:14.157 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:14.148+0000 7f27ada00640 1 --2- 172.21.15.12:0/3475730974 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f27900b51f0 0x7f27900b76b0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f279c009cf0 tx=0x7f279c000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:16:14.157 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:14.149+0000 7f27ada00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e951cb1970 0x55e951cbb000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:16:14.157 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:14.149+0000 7f27ad000640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e951cb42a0 0x55e951cbb6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:16:14.157 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:14.149+0000 7f27ada00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e951cb1970 0x55e951cbb000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.12:39542/0 (socket says 172.21.15.12:39542) 2024-09-11T00:16:14.157 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:14.149+0000 7f27ada00640 1 -- 172.21.15.12:0/3231529141 learned_addr learned my addr 172.21.15.12:0/3231529141 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:16:14.157 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:14.149+0000 7f27ae400640 1 --2- 172.21.15.12:0/3231529141 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e951cb6ce0 0x55e951cbc270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:16:14.157 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:14.149+0000 7f27ad000640 1 -- 172.21.15.12:0/3231529141 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e951cb1970 msgr2=0x55e951cbb000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:16:14.157 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:14.149+0000 7f27ad000640 1 --2- 172.21.15.12:0/3231529141 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e951cb1970 0x55e951cbb000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:16:14.158 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:14.149+0000 7f27ad000640 1 -- 172.21.15.12:0/3231529141 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e951cb6ce0 msgr2=0x55e951cbc270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:16:14.158 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:14.149+0000 7f27ad000640 1 --2- 172.21.15.12:0/3231529141 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e951cb6ce0 0x55e951cbc270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:16:14.158 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:14.149+0000 7f27ad000640 1 -- 172.21.15.12:0/3231529141 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e951975790 con 0x55e951cb42a0 2024-09-11T00:16:14.158 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:14.149+0000 7f27ae400640 1 --2- 172.21.15.12:0/3231529141 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e951cb6ce0 0x55e951cbc270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T00:16:14.158 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:14.149+0000 7f27ad000640 1 --2- 172.21.15.12:0/3231529141 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e951cb42a0 0x55e951cbb6a0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f27940c9fc0 tx=0x7f2794031430 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e37c66356a4665cc server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:16:14.158 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:14.149+0000 7f278f400640 10 client.0 ms_handle_connect on v2:172.21.15.178:3300/0 2024-09-11T00:16:14.158 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:14.149+0000 7f278f400640 1 -- 172.21.15.12:0/3231529141 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2794049070 con 0x55e951cb42a0 2024-09-11T00:16:14.158 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:14.149+0000 7f27b3436740 1 -- 172.21.15.12:0/3231529141 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f279c0041d0 con 0x55e951cb42a0 2024-09-11T00:16:14.158 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:14.149+0000 7f278f400640 1 -- 172.21.15.12:0/3231529141 <== mon.1 v2:172.21.15.178:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9208+0+0 (secure 0 0 0) 0x7f279403b460 con 0x55e951cb42a0 2024-09-11T00:16:14.158 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:14.150+0000 7f278f400640 1 -- 172.21.15.12:0/3231529141 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f27940cc4c0 con 0x55e951cb42a0 2024-09-11T00:16:14.158 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:14.150+0000 7f27b3436740 10 client.26533 fetch_fsmap learned FSMap version 41 2024-09-11T00:16:14.158 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:14.150+0000 7f27b3436740 1 -- 172.21.15.12:0/3231529141 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x55e9519ee620 con 0x55e951cb42a0 2024-09-11T00:16:14.158 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:14.150+0000 7f278f400640 1 -- 172.21.15.12:0/3231529141 <== mon.1 v2:172.21.15.178:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f279c0041d0 con 0x55e951cb42a0 2024-09-11T00:16:14.159 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:14.150+0000 7f278f400640 1 -- 172.21.15.12:0/3231529141 <== mon.1 v2:172.21.15.178:3300/0 5 ==== fsmap(e 41) ==== 8370+0+0 (secure 0 0 0) 0x7f27940c8fc0 con 0x55e951cb42a0 2024-09-11T00:16:14.159 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:14.150+0000 7f27b3436740 10 client.26533 fetch_fsmap finished waiting for FSMap version 41 2024-09-11T00:16:14.159 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:14.150+0000 7f27b3436740 10 client.26533 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T00:16:14.159 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:14.150+0000 7f27b3436740 20 client.26533 populate_metadata read hostname 'smithi012' 2024-09-11T00:16:14.159 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:14.150+0000 7f27b3436740 1 --2- 172.21.15.12:0/3231529141 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55e951cc6800 0x55e951cc8c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:16:14.159 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:14.150+0000 7f27b3436740 4 client.26533 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T00:16:14.159 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:14.150+0000 7f27b3436740 1 -- 172.21.15.12:0/3231529141 --> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e951c26600 con 0x55e951cc6800 2024-09-11T00:16:14.159 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:14.151+0000 7f27ada00640 1 --2- 172.21.15.12:0/3231529141 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55e951cc6800 0x55e951cc8c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:16:14.159 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:14.151+0000 7f27ada00640 1 --2- 172.21.15.12:0/3231529141 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55e951cc6800 0x55e951cc8c40 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=48805cb1a48af0c6 server_cookie=3a284def6f88d4af in_seq=0 out_seq=0 2024-09-11T00:16:14.159 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:14.151+0000 7f278f400640 10 client.26533 ms_handle_connect on v2:172.21.15.165:6838/501312984 2024-09-11T00:16:14.159 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:14.152+0000 7f278f400640 1 -- 172.21.15.12:0/3231529141 <== mds.0 v2:172.21.15.165:6838/501312984 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55e951c26600 con 0x55e951cc6800 2024-09-11T00:16:14.159 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:14.152+0000 7f278f400640 10 client.26533 handle_command_reply: tid=0 2024-09-11T00:16:14.159 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:14.152+0000 7f27b3436740 1 client.26533 shutdown 2024-09-11T00:16:14.159 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:14.152+0000 7f27b3436740 1 -- 172.21.15.12:0/3231529141 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e951cb42a0 msgr2=0x55e951cbb6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:16:14.159 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:14.152+0000 7f27b3436740 1 --2- 172.21.15.12:0/3231529141 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e951cb42a0 0x55e951cbb6a0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f27940c9fc0 tx=0x7f2794031430 comp rx=0 tx=0).stop 2024-09-11T00:16:14.160 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:14.152+0000 7f27b3436740 1 -- 172.21.15.12:0/3231529141 shutdown_connections 2024-09-11T00:16:14.160 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:14.152+0000 7f27b3436740 1 --2- 172.21.15.12:0/3231529141 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55e951cc6800 0x55e951cc8c40 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:16:14.160 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:14.152+0000 7f27b3436740 1 --2- 172.21.15.12:0/3231529141 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e951cb6ce0 0x55e951cbc270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:16:14.160 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:14.152+0000 7f27b3436740 1 --2- 172.21.15.12:0/3231529141 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e951cb42a0 0x55e951cbb6a0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:16:14.160 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:14.152+0000 7f278f400640 0 client.26533 ms_handle_reset on v2:172.21.15.165:6838/501312984 2024-09-11T00:16:14.160 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:14.152+0000 7f27b3436740 1 --2- 172.21.15.12:0/3231529141 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e951cb1970 0x55e951cbb000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:16:14.160 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:14.152+0000 7f27b3436740 1 -- 172.21.15.12:0/3231529141 >> 172.21.15.12:0/3231529141 conn(0x55e951cacac0 msgr2=0x55e951cabbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:16:14.160 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:14.152+0000 7f27b3436740 1 -- 172.21.15.12:0/3231529141 shutdown_connections 2024-09-11T00:16:14.160 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:14.152+0000 7f27b3436740 1 -- 172.21.15.12:0/3231529141 wait complete. 2024-09-11T00:16:14.160 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:14.153+0000 7f27b3436740 20 client.26533 trim_cache size 0 max 16384 2024-09-11T00:16:14.174 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:14.171+0000 7f27b3436740 1 Processor -- start 2024-09-11T00:16:14.174 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:14.171+0000 7f27b3436740 1 -- start start 2024-09-11T00:16:14.174 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:14.171+0000 7f27b3436740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e951cb1970 0x55e951ca7ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:16:14.174 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:14.171+0000 7f27b3436740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e951cb42a0 0x55e951cb0dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:16:14.174 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:14.171+0000 7f27b3436740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e951cb6ce0 0x55e951cb1310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:16:14.174 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:14.171+0000 7f27b3436740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f27a81ab450 con 0x55e951cb1970 2024-09-11T00:16:14.174 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:14.171+0000 7f27b3436740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f27a81acf90 con 0x55e951cb42a0 2024-09-11T00:16:14.174 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:14.171+0000 7f27b3436740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f27a81abb20 con 0x55e951cb6ce0 2024-09-11T00:16:14.174 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:14.171+0000 7f27ad000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e951cb42a0 0x55e951cb0dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:16:14.174 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:14.171+0000 7f27ae400640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e951cb6ce0 0x55e951cb1310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:16:14.174 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:14.171+0000 7f27ad000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e951cb42a0 0x55e951cb0dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.12:39546/0 (socket says 172.21.15.12:39546) 2024-09-11T00:16:14.175 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:14.171+0000 7f27ad000640 1 -- 172.21.15.12:0/551330604 learned_addr learned my addr 172.21.15.12:0/551330604 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:16:14.175 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:14.172+0000 7f27ada00640 1 --2- 172.21.15.12:0/551330604 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e951cb1970 0x55e951ca7ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:16:14.179 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:14.172+0000 7f27ad000640 1 -- 172.21.15.12:0/551330604 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e951cb6ce0 msgr2=0x55e951cb1310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:16:14.179 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:14.172+0000 7f27ad000640 1 --2- 172.21.15.12:0/551330604 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e951cb6ce0 0x55e951cb1310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:16:14.179 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:14.172+0000 7f27ad000640 1 -- 172.21.15.12:0/551330604 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e951cb1970 msgr2=0x55e951ca7ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:16:14.179 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:14.172+0000 7f27ad000640 1 --2- 172.21.15.12:0/551330604 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e951cb1970 0x55e951ca7ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:16:14.179 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:14.172+0000 7f27ad000640 1 -- 172.21.15.12:0/551330604 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e951cb9510 con 0x55e951cb42a0 2024-09-11T00:16:14.179 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:14.172+0000 7f27ae400640 1 --2- 172.21.15.12:0/551330604 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e951cb6ce0 0x55e951cb1310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:16:14.179 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:14.172+0000 7f27ad000640 1 --2- 172.21.15.12:0/551330604 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e951cb42a0 0x55e951cb0dd0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f2794005c40 tx=0x7f27940b42f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=256c13b81f5584df server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:16:14.179 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:14.172+0000 7f279a000640 1 -- 172.21.15.12:0/551330604 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2794049070 con 0x55e951cb42a0 2024-09-11T00:16:14.179 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:14.172+0000 7f279a000640 1 -- 172.21.15.12:0/551330604 <== mon.2 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f27940cc860 con 0x55e951cb42a0 2024-09-11T00:16:14.179 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:14.172+0000 7f279a000640 1 -- 172.21.15.12:0/551330604 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27940b2b70 con 0x55e951cb42a0 2024-09-11T00:16:14.179 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:14.173+0000 7f27b3436740 1 -- 172.21.15.12:0/551330604 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e951cb42a0 msgr2=0x55e951cb0dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:16:14.179 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:14.173+0000 7f27b3436740 1 --2- 172.21.15.12:0/551330604 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e951cb42a0 0x55e951cb0dd0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f2794005c40 tx=0x7f27940b42f0 comp rx=0 tx=0).stop 2024-09-11T00:16:14.180 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:14.173+0000 7f27b3436740 1 -- 172.21.15.12:0/551330604 shutdown_connections 2024-09-11T00:16:14.180 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:14.173+0000 7f27b3436740 1 --2- 172.21.15.12:0/551330604 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e951cb6ce0 0x55e951cb1310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:16:14.180 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:14.173+0000 7f27b3436740 1 --2- 172.21.15.12:0/551330604 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e951cb42a0 0x55e951cb0dd0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:16:14.180 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:14.173+0000 7f27b3436740 1 --2- 172.21.15.12:0/551330604 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e951cb1970 0x55e951ca7ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:16:14.180 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:14.173+0000 7f27b3436740 1 -- 172.21.15.12:0/551330604 >> 172.21.15.12:0/551330604 conn(0x55e951cacac0 msgr2=0x55e951cb70c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:16:14.180 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:14.173+0000 7f27b3436740 1 -- 172.21.15.12:0/551330604 shutdown_connections 2024-09-11T00:16:14.180 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:14.173+0000 7f27b3436740 1 -- 172.21.15.12:0/551330604 wait complete. 2024-09-11T00:16:14.180 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:14.174+0000 7f27b3436740 1 -- start start 2024-09-11T00:16:14.180 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:14.174+0000 7f27b3436740 1 Processor -- start 2024-09-11T00:16:14.180 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:14.174+0000 7f27b3436740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e951cb1970 0x55e951cc2420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:16:14.180 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:14.174+0000 7f27b3436740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e951cb42a0 0x55e951cc2980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:16:14.180 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:14.174+0000 7f27b3436740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e951cb6ce0 0x55e951cc0f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:16:14.180 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:14.174+0000 7f27b3436740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f27a81ac8c0 con 0x55e951cb1970 2024-09-11T00:16:14.181 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:14.174+0000 7f27b3436740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f27a81ac1f0 con 0x55e951cb42a0 2024-09-11T00:16:14.181 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:14.174+0000 7f27b3436740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f27a81a9430 con 0x55e951cb6ce0 2024-09-11T00:16:14.181 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:14.174+0000 7f27b3436740 10 client.0 _ll_register_callbacks cb 0x7f2794005b70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:16:14.181 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:14.175+0000 7f27ada00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e951cb1970 0x55e951cc2420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:16:14.181 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:14.175+0000 7f27ad000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e951cb42a0 0x55e951cc2980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:16:14.181 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:14.175+0000 7f27ada00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e951cb1970 0x55e951cc2420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:60132/0 (socket says 172.21.15.12:60132) 2024-09-11T00:16:14.181 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:14.175+0000 7f27ada00640 1 -- 172.21.15.12:0/4017843933 learned_addr learned my addr 172.21.15.12:0/4017843933 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:16:14.181 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:14.175+0000 7f27ae400640 1 --2- 172.21.15.12:0/4017843933 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e951cb6ce0 0x55e951cc0f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:16:14.181 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:14.175+0000 7f27ada00640 1 -- 172.21.15.12:0/4017843933 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e951cb6ce0 msgr2=0x55e951cc0f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:16:14.181 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:14.175+0000 7f27ada00640 1 --2- 172.21.15.12:0/4017843933 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e951cb6ce0 0x55e951cc0f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:16:14.181 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:14.175+0000 7f27ada00640 1 -- 172.21.15.12:0/4017843933 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e951cb42a0 msgr2=0x55e951cc2980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:16:14.181 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:14.175+0000 7f27ada00640 1 --2- 172.21.15.12:0/4017843933 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e951cb42a0 0x55e951cc2980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:16:14.181 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:14.175+0000 7f27ada00640 1 -- 172.21.15.12:0/4017843933 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e951cc1480 con 0x55e951cb1970 2024-09-11T00:16:14.182 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:14.175+0000 7f27ad000640 1 --2- 172.21.15.12:0/4017843933 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e951cb42a0 0x55e951cc2980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:16:14.182 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:14.175+0000 7f27ada00640 1 --2- 172.21.15.12:0/4017843933 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e951cb1970 0x55e951cc2420 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f279c009250 tx=0x7f279c022000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=873fb43431a3094e server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:16:14.182 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:14.175+0000 7f278f400640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-09-11T00:16:14.182 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:14.176+0000 7f278f400640 1 -- 172.21.15.12:0/4017843933 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f279c004610 con 0x55e951cb1970 2024-09-11T00:16:14.182 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:14.176+0000 7f278f400640 1 -- 172.21.15.12:0/4017843933 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9208+0+0 (secure 0 0 0) 0x7f279c0047b0 con 0x55e951cb1970 2024-09-11T00:16:14.182 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:14.176+0000 7f27b3436740 1 -- 172.21.15.12:0/4017843933 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f27940cc860 con 0x55e951cb1970 2024-09-11T00:16:14.182 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:14.176+0000 7f278f400640 1 -- 172.21.15.12:0/4017843933 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f279c047050 con 0x55e951cb1970 2024-09-11T00:16:14.182 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:14.176+0000 7f27b3436740 10 client.16545 fetch_fsmap learned FSMap version 41 2024-09-11T00:16:14.182 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:14.176+0000 7f27b3436740 1 -- 172.21.15.12:0/4017843933 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x55e951d33f30 con 0x55e951cb1970 2024-09-11T00:16:14.182 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:14.176+0000 7f278f400640 1 -- 172.21.15.12:0/4017843933 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f27940cc860 con 0x55e951cb1970 2024-09-11T00:16:14.182 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:14.176+0000 7f278f400640 1 -- 172.21.15.12:0/4017843933 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 41) ==== 8370+0+0 (secure 0 0 0) 0x7f279c031980 con 0x55e951cb1970 2024-09-11T00:16:14.183 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:14.176+0000 7f27b3436740 10 client.16545 fetch_fsmap finished waiting for FSMap version 41 2024-09-11T00:16:14.183 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:14.176+0000 7f27b3436740 10 client.16545 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T00:16:14.183 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:14.176+0000 7f27b3436740 20 client.16545 populate_metadata read hostname 'smithi012' 2024-09-11T00:16:14.183 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:14.176+0000 7f27b3436740 1 --2- 172.21.15.12:0/4017843933 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55e951d34730 0x55e951d36b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:16:14.183 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:14.176+0000 7f27b3436740 4 client.16545 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T00:16:14.183 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:14.176+0000 7f27b3436740 1 -- 172.21.15.12:0/4017843933 --> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55e951c220b0 con 0x55e951d34730 2024-09-11T00:16:14.183 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:14.177+0000 7f27ad000640 1 --2- 172.21.15.12:0/4017843933 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55e951d34730 0x55e951d36b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:16:14.183 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:14.177+0000 7f27ad000640 1 --2- 172.21.15.12:0/4017843933 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55e951d34730 0x55e951d36b50 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3fc297de01f78685 server_cookie=7364d575f462d4b8 in_seq=0 out_seq=0 2024-09-11T00:16:14.183 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:14.177+0000 7f278f400640 10 client.16545 ms_handle_connect on v2:172.21.15.165:6838/501312984 2024-09-11T00:16:14.244 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:14.242+0000 7f278f400640 1 -- 172.21.15.12:0/4017843933 <== mds.0 v2:172.21.15.165:6838/501312984 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55e951c220b0 con 0x55e951d34730 2024-09-11T00:16:14.245 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:14.242+0000 7f278f400640 10 client.16545 handle_command_reply: tid=0 2024-09-11T00:16:14.245 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:14.242+0000 7f27b3436740 1 client.16545 shutdown 2024-09-11T00:16:14.245 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:14.242+0000 7f27b3436740 1 -- 172.21.15.12:0/4017843933 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e951cb1970 msgr2=0x55e951cc2420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:16:14.245 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:14.242+0000 7f27b3436740 1 --2- 172.21.15.12:0/4017843933 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e951cb1970 0x55e951cc2420 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f279c009250 tx=0x7f279c022000 comp rx=0 tx=0).stop 2024-09-11T00:16:14.245 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:14.242+0000 7f27b3436740 1 -- 172.21.15.12:0/4017843933 shutdown_connections 2024-09-11T00:16:14.245 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:14.242+0000 7f27b3436740 1 --2- 172.21.15.12:0/4017843933 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55e951d34730 0x55e951d36b50 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:16:14.245 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:14.242+0000 7f27b3436740 1 --2- 172.21.15.12:0/4017843933 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e951cb6ce0 0x55e951cc0f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:16:14.245 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:14.242+0000 7f27b3436740 1 --2- 172.21.15.12:0/4017843933 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e951cb42a0 0x55e951cc2980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:16:14.245 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:14.242+0000 7f278f400640 0 client.16545 ms_handle_reset on v2:172.21.15.165:6838/501312984 2024-09-11T00:16:14.245 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:14.242+0000 7f27b3436740 1 --2- 172.21.15.12:0/4017843933 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e951cb1970 0x55e951cc2420 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:16:14.245 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:14.242+0000 7f27b3436740 1 -- 172.21.15.12:0/4017843933 >> 172.21.15.12:0/4017843933 conn(0x55e951cacac0 msgr2=0x55e951cb09a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:16:14.246 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:14.242+0000 7f27b3436740 1 -- 172.21.15.12:0/4017843933 shutdown_connections 2024-09-11T00:16:14.246 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:14.242+0000 7f27b3436740 1 -- 172.21.15.12:0/4017843933 wait complete. 2024-09-11T00:16:14.246 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:14.242+0000 7f27b3436740 20 client.16545 trim_cache size 0 max 16384 2024-09-11T00:16:14.246 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:{ 2024-09-11T00:16:14.246 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "message": "", 2024-09-11T00:16:14.246 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "return_code": 0 2024-09-11T00:16:14.246 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:} 2024-09-11T00:16:14.247 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:14.243+0000 7f27aee00640 1 -- 172.21.15.12:0/3475730974 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f27900b51f0 msgr2=0x7f27900b76b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:16:14.247 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:14.243+0000 7f27aee00640 1 --2- 172.21.15.12:0/3475730974 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f27900b51f0 0x7f27900b76b0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f279c009cf0 tx=0x7f279c000bc0 comp rx=0 tx=0).stop 2024-09-11T00:16:14.247 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:14.243+0000 7f27aee00640 1 -- 172.21.15.12:0/3475730974 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f27a81027e0 msgr2=0x7f27a81991a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:16:14.247 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:14.243+0000 7f27aee00640 1 --2- 172.21.15.12:0/3475730974 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f27a81027e0 0x7f27a81991a0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f279402a910 tx=0x7f2794003c10 comp rx=0 tx=0).stop 2024-09-11T00:16:14.247 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:14.243+0000 7f27aee00640 1 -- 172.21.15.12:0/3475730974 shutdown_connections 2024-09-11T00:16:14.247 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:14.243+0000 7f27aee00640 1 --2- 172.21.15.12:0/3475730974 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f27900b51f0 0x7f27900b76b0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:16:14.247 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:14.243+0000 7f27aee00640 1 --2- 172.21.15.12:0/3475730974 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f27a81031a0 0x7f27a8199dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:16:14.247 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:14.243+0000 7f27aee00640 1 --2- 172.21.15.12:0/3475730974 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f27a81027e0 0x7f27a81991a0 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:16:14.248 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:14.243+0000 7f27aee00640 1 --2- 172.21.15.12:0/3475730974 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f27a8101690 0x7f27a8198b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:16:14.248 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:14.243+0000 7f27aee00640 1 -- 172.21.15.12:0/3475730974 >> 172.21.15.12:0/3475730974 conn(0x7f27a80fcf30 msgr2=0x7f27a8105bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:16:14.248 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:14.243+0000 7f27aee00640 1 -- 172.21.15.12:0/3475730974 shutdown_connections 2024-09-11T00:16:14.248 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:14.243+0000 7f27aee00640 1 -- 172.21.15.12:0/3475730974 wait complete. 2024-09-11T00:16:14.667 INFO:tasks.workunit.client.0.smithi012.stdout: 1 362027 3.86 MB/sec execute 371 sec latency 50.751 ms 2024-09-11T00:16:15.667 INFO:tasks.workunit.client.0.smithi012.stdout: 1 363193 3.87 MB/sec execute 372 sec latency 63.516 ms 2024-09-11T00:16:15.831 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:16:15 smithi178 ceph-mon[35919]: pgmap v674: 129 pgs: 129 active+clean; 89 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 811 KiB/s wr, 251 op/s 2024-09-11T00:16:16.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:16:15 smithi165 ceph-mon[35356]: pgmap v674: 129 pgs: 129 active+clean; 89 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 811 KiB/s wr, 251 op/s 2024-09-11T00:16:16.063 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:16:15 smithi012 ceph-mon[30333]: pgmap v674: 129 pgs: 129 active+clean; 89 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 811 KiB/s wr, 251 op/s 2024-09-11T00:16:16.667 INFO:tasks.workunit.client.0.smithi012.stdout: 1 365369 3.89 MB/sec execute 373 sec latency 14.208 ms 2024-09-11T00:16:17.667 INFO:tasks.workunit.client.0.smithi012.stdout: 1 366524 3.89 MB/sec execute 374 sec latency 14.023 ms 2024-09-11T00:16:17.831 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:16:17 smithi178 ceph-mon[35919]: pgmap v675: 129 pgs: 129 active+clean; 88 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 MiB/s rd, 1.3 MiB/s wr, 385 op/s 2024-09-11T00:16:18.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:16:17 smithi165 ceph-mon[35356]: pgmap v675: 129 pgs: 129 active+clean; 88 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 MiB/s rd, 1.3 MiB/s wr, 385 op/s 2024-09-11T00:16:18.063 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:16:17 smithi012 ceph-mon[30333]: pgmap v675: 129 pgs: 129 active+clean; 88 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 MiB/s rd, 1.3 MiB/s wr, 385 op/s 2024-09-11T00:16:18.668 INFO:tasks.workunit.client.0.smithi012.stdout: 1 368891 3.92 MB/sec execute 375 sec latency 10.930 ms 2024-09-11T00:16:19.668 INFO:tasks.workunit.client.0.smithi012.stdout: 1 369857 3.92 MB/sec execute 376 sec latency 12.301 ms 2024-09-11T00:16:19.843 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:16:19 smithi012 ceph-mon[30333]: pgmap v676: 129 pgs: 129 active+clean; 88 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 1002 KiB/s wr, 256 op/s 2024-09-11T00:16:19.843 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:16:19 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T00:16:20.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:16:19 smithi165 ceph-mon[35356]: pgmap v676: 129 pgs: 129 active+clean; 88 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 1002 KiB/s wr, 256 op/s 2024-09-11T00:16:20.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:16:19 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T00:16:20.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:16:19 smithi178 ceph-mon[35919]: pgmap v676: 129 pgs: 129 active+clean; 88 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 1002 KiB/s wr, 256 op/s 2024-09-11T00:16:20.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:16:19 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T00:16:20.668 INFO:tasks.workunit.client.0.smithi012.stdout: 1 369976 3.91 MB/sec execute 377 sec latency 719.798 ms 2024-09-11T00:16:21.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:16:21 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:16:21.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:16:21 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:16:21.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:16:21 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:16:21.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:16:21 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:16:21.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:16:21 smithi012 ceph-mon[30333]: pgmap v677: 129 pgs: 129 active+clean; 93 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 1.5 MiB/s wr, 314 op/s 2024-09-11T00:16:21.532 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:16:21 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:16:21.532 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:16:21 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:16:21.533 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:16:21 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:16:21.533 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:16:21 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:16:21.533 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:16:21 smithi165 ceph-mon[35356]: pgmap v677: 129 pgs: 129 active+clean; 93 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 1.5 MiB/s wr, 314 op/s 2024-09-11T00:16:21.581 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:16:21 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:16:21.581 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:16:21 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:16:21.581 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:16:21 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:16:21.581 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:16:21 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:16:21.581 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:16:21 smithi178 ceph-mon[35919]: pgmap v677: 129 pgs: 129 active+clean; 93 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 1.5 MiB/s wr, 314 op/s 2024-09-11T00:16:21.668 INFO:tasks.workunit.client.0.smithi012.stdout: 1 371239 3.91 MB/sec execute 378 sec latency 1216.099 ms 2024-09-11T00:16:22.668 INFO:tasks.workunit.client.0.smithi012.stdout: 1 372707 3.93 MB/sec execute 379 sec latency 34.064 ms 2024-09-11T00:16:23.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:16:22 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:16:23.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:16:22 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:16:23.033 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:16:22 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:16:23.033 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:16:22 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T00:16:23.033 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:16:22 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:16:23.063 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:16:22 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:16:23.063 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:16:22 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:16:23.063 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:16:22 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:16:23.063 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:16:22 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T00:16:23.063 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:16:22 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:16:23.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:16:22 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:16:23.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:16:22 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:16:23.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:16:22 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:16:23.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:16:22 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T00:16:23.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:16:22 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:16:23.668 INFO:tasks.workunit.client.0.smithi012.stdout: 1 374099 3.94 MB/sec execute 380 sec latency 30.530 ms 2024-09-11T00:16:24.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:16:23 smithi165 ceph-mon[35356]: pgmap v678: 129 pgs: 129 active+clean; 96 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 1.6 MiB/s wr, 304 op/s 2024-09-11T00:16:24.063 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:16:23 smithi012 ceph-mon[30333]: pgmap v678: 129 pgs: 129 active+clean; 96 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 1.6 MiB/s wr, 304 op/s 2024-09-11T00:16:24.080 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:16:23 smithi178 ceph-mon[35919]: pgmap v678: 129 pgs: 129 active+clean; 96 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 1.6 MiB/s wr, 304 op/s 2024-09-11T00:16:24.668 INFO:tasks.workunit.client.0.smithi012.stdout: 1 375419 3.95 MB/sec execute 381 sec latency 424.915 ms 2024-09-11T00:16:25.668 INFO:tasks.workunit.client.0.smithi012.stdout: 1 375920 3.95 MB/sec execute 382 sec latency 786.355 ms 2024-09-11T00:16:25.813 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:16:25 smithi012 ceph-mon[30333]: pgmap v679: 129 pgs: 129 active+clean; 96 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.4 MiB/s wr, 262 op/s 2024-09-11T00:16:26.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:16:25 smithi165 ceph-mon[35356]: pgmap v679: 129 pgs: 129 active+clean; 96 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.4 MiB/s wr, 262 op/s 2024-09-11T00:16:26.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:16:25 smithi178 ceph-mon[35919]: pgmap v679: 129 pgs: 129 active+clean; 96 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.4 MiB/s wr, 262 op/s 2024-09-11T00:16:26.668 INFO:tasks.workunit.client.0.smithi012.stdout: 1 376427 3.94 MB/sec execute 383 sec latency 1202.136 ms 2024-09-11T00:16:27.373 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:27.370+0000 7fbb84600640 1 -- 172.21.15.12:0/2814278359 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbb7c104ba0 msgr2=0x7fbb7c104fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:16:27.373 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:27.370+0000 7fbb84600640 1 --2- 172.21.15.12:0/2814278359 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbb7c104ba0 0x7fbb7c104fa0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fbb70009900 tx=0x7fbb7002a3f0 comp rx=0 tx=0).stop 2024-09-11T00:16:27.373 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:27.371+0000 7fbb84600640 1 -- 172.21.15.12:0/2814278359 shutdown_connections 2024-09-11T00:16:27.373 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:27.371+0000 7fbb84600640 1 --2- 172.21.15.12:0/2814278359 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbb7c1066b0 0x7fbb7c10d030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:16:27.373 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:27.371+0000 7fbb84600640 1 --2- 172.21.15.12:0/2814278359 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbb7c105cf0 0x7fbb7c106170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:16:27.373 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:27.371+0000 7fbb84600640 1 --2- 172.21.15.12:0/2814278359 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbb7c104ba0 0x7fbb7c104fa0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:16:27.373 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:27.371+0000 7fbb84600640 1 -- 172.21.15.12:0/2814278359 >> 172.21.15.12:0/2814278359 conn(0x7fbb7c100440 msgr2=0x7fbb7c102880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:16:27.374 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:27.371+0000 7fbb84600640 1 -- 172.21.15.12:0/2814278359 shutdown_connections 2024-09-11T00:16:27.374 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:27.371+0000 7fbb84600640 1 -- 172.21.15.12:0/2814278359 wait complete. 2024-09-11T00:16:27.375 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:27.372+0000 7fbb84600640 1 Processor -- start 2024-09-11T00:16:27.375 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:27.372+0000 7fbb84600640 1 -- start start 2024-09-11T00:16:27.375 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:27.372+0000 7fbb84600640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbb7c104ba0 0x7fbb7c19af70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:16:27.375 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:27.372+0000 7fbb84600640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbb7c105cf0 0x7fbb7c19b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:16:27.375 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:27.372+0000 7fbb84600640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbb7c1066b0 0x7fbb7c19c1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:16:27.375 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:27.372+0000 7fbb84600640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fbb7c1107f0 con 0x7fbb7c1066b0 2024-09-11T00:16:27.375 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:27.372+0000 7fbb84600640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fbb7c110670 con 0x7fbb7c105cf0 2024-09-11T00:16:27.375 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:27.372+0000 7fbb84600640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fbb7c110970 con 0x7fbb7c104ba0 2024-09-11T00:16:27.375 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:27.373+0000 7fbb81c00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbb7c104ba0 0x7fbb7c19af70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:16:27.375 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:27.373+0000 7fbb81200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbb7c105cf0 0x7fbb7c19b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:16:27.375 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:27.373+0000 7fbb81c00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbb7c104ba0 0x7fbb7c19af70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.12:51480/0 (socket says 172.21.15.12:51480) 2024-09-11T00:16:27.376 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:27.373+0000 7fbb82600640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbb7c1066b0 0x7fbb7c19c1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:16:27.376 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:27.373+0000 7fbb81c00640 1 -- 172.21.15.12:0/1856767109 learned_addr learned my addr 172.21.15.12:0/1856767109 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:16:27.377 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:27.373+0000 7fbb81200640 1 -- 172.21.15.12:0/1856767109 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbb7c104ba0 msgr2=0x7fbb7c19af70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:16:27.377 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:27.373+0000 7fbb81200640 1 --2- 172.21.15.12:0/1856767109 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbb7c104ba0 0x7fbb7c19af70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:16:27.377 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:27.373+0000 7fbb81200640 1 -- 172.21.15.12:0/1856767109 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbb7c1066b0 msgr2=0x7fbb7c19c1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:16:27.377 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:27.373+0000 7fbb81200640 1 --2- 172.21.15.12:0/1856767109 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbb7c1066b0 0x7fbb7c19c1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:16:27.377 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:27.373+0000 7fbb81200640 1 -- 172.21.15.12:0/1856767109 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb7c19cab0 con 0x7fbb7c105cf0 2024-09-11T00:16:27.377 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:27.373+0000 7fbb82600640 1 --2- 172.21.15.12:0/1856767109 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbb7c1066b0 0x7fbb7c19c1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:16:27.377 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:27.373+0000 7fbb81c00640 1 --2- 172.21.15.12:0/1856767109 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbb7c104ba0 0x7fbb7c19af70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:16:27.377 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:27.373+0000 7fbb81200640 1 --2- 172.21.15.12:0/1856767109 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbb7c105cf0 0x7fbb7c19b5c0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fbb6800e9f0 tx=0x7fbb6800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:16:27.377 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:27.374+0000 7fbb7a000640 1 -- 172.21.15.12:0/1856767109 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb6800cd80 con 0x7fbb7c105cf0 2024-09-11T00:16:27.378 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:27.374+0000 7fbb84600640 1 -- 172.21.15.12:0/1856767109 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbb7c19cda0 con 0x7fbb7c105cf0 2024-09-11T00:16:27.378 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:27.374+0000 7fbb7a000640 1 -- 172.21.15.12:0/1856767109 <== mon.2 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbb680035f0 con 0x7fbb7c105cf0 2024-09-11T00:16:27.378 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:27.374+0000 7fbb7a000640 1 -- 172.21.15.12:0/1856767109 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb68010670 con 0x7fbb7c105cf0 2024-09-11T00:16:27.378 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:27.374+0000 7fbb84600640 1 -- 172.21.15.12:0/1856767109 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbb7c1a3470 con 0x7fbb7c105cf0 2024-09-11T00:16:27.379 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:27.377+0000 7fbb87490740 1 Processor -- start 2024-09-11T00:16:27.379 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:27.377+0000 7fbb87490740 1 -- start start 2024-09-11T00:16:27.381 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:27.377+0000 7fbb87490740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5589be28a790 0x5589be28cb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:16:27.381 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:27.377+0000 7fbb87490740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5589be28d0c0 0x5589be28f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:16:27.381 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:27.377+0000 7fbb87490740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5589be28fb00 0x5589be291fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:16:27.381 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:27.377+0000 7fbb87490740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x5589be1fd6b0 con 0x5589be28a790 2024-09-11T00:16:27.381 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:27.377+0000 7fbb87490740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x5589be1ecae0 con 0x5589be28d0c0 2024-09-11T00:16:27.381 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:27.377+0000 7fbb87490740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5589be1f08f0 con 0x5589be28fb00 2024-09-11T00:16:27.381 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:27.378+0000 7fbb82600640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5589be28a790 0x5589be28cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:16:27.381 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:27.378+0000 7fbb81c00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5589be28d0c0 0x5589be28f4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:16:27.381 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:27.378+0000 7fbb81c00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5589be28d0c0 0x5589be28f4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.12:59518/0 (socket says 172.21.15.12:59518) 2024-09-11T00:16:27.381 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:27.378+0000 7fbb81c00640 1 -- 172.21.15.12:0/2115628308 learned_addr learned my addr 172.21.15.12:0/2115628308 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:16:27.382 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:27.378+0000 7fbb82600640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5589be28a790 0x5589be28cb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:33142/0 (socket says 172.21.15.12:33142) 2024-09-11T00:16:27.382 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:27.378+0000 7fbb81c00640 1 -- 172.21.15.12:0/2115628308 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5589be28fb00 msgr2=0x5589be291fc0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-11T00:16:27.382 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:27.378+0000 7fbb81c00640 1 --2- 172.21.15.12:0/2115628308 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5589be28fb00 0x5589be291fc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:16:27.382 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:27.378+0000 7fbb81c00640 1 -- 172.21.15.12:0/2115628308 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5589be28a790 msgr2=0x5589be28cb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:16:27.382 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:27.378+0000 7fbb81c00640 1 --2- 172.21.15.12:0/2115628308 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5589be28a790 0x5589be28cb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:16:27.382 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:27.378+0000 7fbb81c00640 1 -- 172.21.15.12:0/2115628308 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5589be1b9780 con 0x5589be28d0c0 2024-09-11T00:16:27.382 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:27.378+0000 7fbb82600640 1 --2- 172.21.15.12:0/2115628308 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5589be28a790 0x5589be28cb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:16:27.382 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:27.378+0000 7fbb7a000640 1 -- 172.21.15.12:0/1856767109 <== mon.2 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7fbb68004000 con 0x7fbb7c105cf0 2024-09-11T00:16:27.382 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:27.378+0000 7fbb81c00640 1 --2- 172.21.15.12:0/2115628308 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5589be28d0c0 0x5589be28f4b0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fbb70006010 tx=0x7fbb70005e80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d74561bdbb15e83a server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:16:27.382 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:27.379+0000 7fbb6ea00640 1 -- 172.21.15.12:0/2115628308 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb70035030 con 0x5589be28d0c0 2024-09-11T00:16:27.382 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:27.379+0000 7fbb6ea00640 1 -- 172.21.15.12:0/2115628308 <== mon.2 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbb70040070 con 0x5589be28d0c0 2024-09-11T00:16:27.382 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:27.379+0000 7fbb6ea00640 1 -- 172.21.15.12:0/2115628308 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb70003da0 con 0x5589be28d0c0 2024-09-11T00:16:27.386 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:27.379+0000 7fbb87490740 1 -- 172.21.15.12:0/2115628308 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5589be28d0c0 msgr2=0x5589be28f4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:16:27.386 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:27.379+0000 7fbb87490740 1 --2- 172.21.15.12:0/2115628308 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5589be28d0c0 0x5589be28f4b0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fbb70006010 tx=0x7fbb70005e80 comp rx=0 tx=0).stop 2024-09-11T00:16:27.386 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:27.380+0000 7fbb87490740 1 -- 172.21.15.12:0/2115628308 shutdown_connections 2024-09-11T00:16:27.386 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:27.380+0000 7fbb87490740 1 --2- 172.21.15.12:0/2115628308 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5589be28fb00 0x5589be291fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:16:27.386 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:27.380+0000 7fbb87490740 1 --2- 172.21.15.12:0/2115628308 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5589be28d0c0 0x5589be28f4b0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:16:27.386 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:27.380+0000 7fbb87490740 1 --2- 172.21.15.12:0/2115628308 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5589be28a790 0x5589be28cb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:16:27.386 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:27.380+0000 7fbb87490740 1 -- 172.21.15.12:0/2115628308 >> 172.21.15.12:0/2115628308 conn(0x5589be2858e0 msgr2=0x5589be286670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:16:27.386 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:27.380+0000 7fbb7a000640 1 --2- 172.21.15.12:0/1856767109 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fbb640b52e0 0x7fbb640b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:16:27.386 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:27.380+0000 7fbb87490740 1 -- 172.21.15.12:0/2115628308 shutdown_connections 2024-09-11T00:16:27.386 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:27.380+0000 7fbb87490740 1 -- 172.21.15.12:0/2115628308 wait complete. 2024-09-11T00:16:27.387 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:27.380+0000 7fbb7a000640 1 -- 172.21.15.12:0/1856767109 <== mon.2 v2:172.21.15.165:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9208+0+0 (secure 0 0 0) 0x7fbb680e6240 con 0x7fbb7c105cf0 2024-09-11T00:16:27.387 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:27.380+0000 7fbb82600640 1 --2- 172.21.15.12:0/1856767109 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fbb640b52e0 0x7fbb640b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:16:27.387 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:27.380+0000 7fbb87490740 1 -- start start 2024-09-11T00:16:27.387 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:27.381+0000 7fbb87490740 1 Processor -- start 2024-09-11T00:16:27.387 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:27.381+0000 7fbb82600640 1 --2- 172.21.15.12:0/1856767109 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fbb640b52e0 0x7fbb640b77a0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fbb74009fd0 tx=0x7fbb74009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:16:27.387 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:27.381+0000 7fbb87490740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5589be28a790 0x5589be293ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:16:27.387 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:27.381+0000 7fbb87490740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5589be28d0c0 0x5589be294540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:16:27.387 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:27.381+0000 7fbb87490740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5589be28fb00 0x5589be295110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:16:27.387 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:27.381+0000 7fbb87490740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x5589be124ab0 con 0x5589be28d0c0 2024-09-11T00:16:27.387 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:27.381+0000 7fbb87490740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x5589be193570 con 0x5589be28a790 2024-09-11T00:16:27.387 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:27.381+0000 7fbb87490740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5589be292a80 con 0x5589be28fb00 2024-09-11T00:16:27.387 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:27.381+0000 7fbb87490740 10 client.0 _ll_register_callbacks cb 0x5589be1a2100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:16:27.387 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:27.381+0000 7fbb81c00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5589be28a790 0x5589be293ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:16:27.388 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:27.381+0000 7fbb81200640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5589be28d0c0 0x5589be294540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:16:27.388 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:27.381+0000 7fbb81c00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5589be28a790 0x5589be293ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.12:59528/0 (socket says 172.21.15.12:59528) 2024-09-11T00:16:27.388 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:27.381+0000 7fbb81c00640 1 -- 172.21.15.12:0/3686090181 learned_addr learned my addr 172.21.15.12:0/3686090181 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:16:27.388 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:27.381+0000 7fbb82600640 1 --2- 172.21.15.12:0/3686090181 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5589be28fb00 0x5589be295110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:16:27.388 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:27.382+0000 7fbb81c00640 1 -- 172.21.15.12:0/3686090181 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5589be28fb00 msgr2=0x5589be295110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:16:27.388 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:27.382+0000 7fbb81c00640 1 --2- 172.21.15.12:0/3686090181 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5589be28fb00 0x5589be295110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:16:27.388 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:27.382+0000 7fbb81c00640 1 -- 172.21.15.12:0/3686090181 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5589be28d0c0 msgr2=0x5589be294540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:16:27.388 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:27.382+0000 7fbb81c00640 1 --2- 172.21.15.12:0/3686090181 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5589be28d0c0 0x5589be294540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:16:27.388 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:27.382+0000 7fbb81c00640 1 -- 172.21.15.12:0/3686090181 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5589bdf4e790 con 0x5589be28a790 2024-09-11T00:16:27.388 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:27.382+0000 7fbb81c00640 1 --2- 172.21.15.12:0/3686090181 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5589be28a790 0x5589be293ea0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fbb70005940 tx=0x7fbb70031e90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=912922e982c3f470 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:16:27.388 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:27.382+0000 7fbb63400640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-09-11T00:16:27.388 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:27.382+0000 7fbb63400640 1 -- 172.21.15.12:0/3686090181 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb70035050 con 0x5589be28a790 2024-09-11T00:16:27.388 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:27.382+0000 7fbb87490740 1 -- 172.21.15.12:0/3686090181 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbb70040070 con 0x5589be28a790 2024-09-11T00:16:27.388 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:27.382+0000 7fbb63400640 1 -- 172.21.15.12:0/3686090181 <== mon.2 v2:172.21.15.165:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9208+0+0 (secure 0 0 0) 0x7fbb70003ef0 con 0x5589be28a790 2024-09-11T00:16:27.389 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:27.382+0000 7fbb63400640 1 -- 172.21.15.12:0/3686090181 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7fbb700040f0 con 0x5589be28a790 2024-09-11T00:16:27.389 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:27.382+0000 7fbb87490740 10 client.26564 fetch_fsmap learned FSMap version 41 2024-09-11T00:16:27.389 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:27.382+0000 7fbb87490740 1 -- 172.21.15.12:0/3686090181 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x5589bdfc7620 con 0x5589be28a790 2024-09-11T00:16:27.389 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:27.383+0000 7fbb63400640 1 -- 172.21.15.12:0/3686090181 <== mon.2 v2:172.21.15.165:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbb70040070 con 0x5589be28a790 2024-09-11T00:16:27.389 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:27.383+0000 7fbb63400640 1 -- 172.21.15.12:0/3686090181 <== mon.2 v2:172.21.15.165:3300/0 5 ==== fsmap(e 41) ==== 8370+0+0 (secure 0 0 0) 0x7fbb7003cc90 con 0x5589be28a790 2024-09-11T00:16:27.389 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:27.383+0000 7fbb87490740 10 client.26564 fetch_fsmap finished waiting for FSMap version 41 2024-09-11T00:16:27.389 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:27.383+0000 7fbb87490740 10 client.26564 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T00:16:27.389 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:27.383+0000 7fbb87490740 20 client.26564 populate_metadata read hostname 'smithi012' 2024-09-11T00:16:27.389 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:27.383+0000 7fbb87490740 1 --2- 172.21.15.12:0/3686090181 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x5589be29f340 0x5589be2a1730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:16:27.389 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:27.383+0000 7fbb87490740 4 client.26564 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T00:16:27.389 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:27.383+0000 7fbb87490740 1 -- 172.21.15.12:0/3686090181 --> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5589be1ff600 con 0x5589be29f340 2024-09-11T00:16:27.389 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:27.383+0000 7fbb81200640 1 --2- 172.21.15.12:0/3686090181 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x5589be29f340 0x5589be2a1730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:16:27.389 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:27.384+0000 7fbb81200640 1 --2- 172.21.15.12:0/3686090181 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x5589be29f340 0x5589be2a1730 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=273059df91f8049a server_cookie=303a9f3a706dab2e in_seq=0 out_seq=0 2024-09-11T00:16:27.389 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:27.384+0000 7fbb63400640 10 client.26564 ms_handle_connect on v2:172.21.15.165:6838/501312984 2024-09-11T00:16:27.390 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:27.385+0000 7fbb63400640 1 -- 172.21.15.12:0/3686090181 <== mds.0 v2:172.21.15.165:6838/501312984 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5589be1ff600 con 0x5589be29f340 2024-09-11T00:16:27.390 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:27.385+0000 7fbb63400640 10 client.26564 handle_command_reply: tid=0 2024-09-11T00:16:27.391 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:27.385+0000 7fbb87490740 1 client.26564 shutdown 2024-09-11T00:16:27.391 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:27.386+0000 7fbb87490740 1 -- 172.21.15.12:0/3686090181 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5589be28a790 msgr2=0x5589be293ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:16:27.391 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:27.386+0000 7fbb87490740 1 --2- 172.21.15.12:0/3686090181 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5589be28a790 0x5589be293ea0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fbb70005940 tx=0x7fbb70031e90 comp rx=0 tx=0).stop 2024-09-11T00:16:27.391 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:27.386+0000 7fbb87490740 1 -- 172.21.15.12:0/3686090181 shutdown_connections 2024-09-11T00:16:27.391 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:27.386+0000 7fbb87490740 1 --2- 172.21.15.12:0/3686090181 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x5589be29f340 0x5589be2a1730 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:16:27.391 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:27.386+0000 7fbb87490740 1 --2- 172.21.15.12:0/3686090181 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5589be28fb00 0x5589be295110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:16:27.391 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:27.386+0000 7fbb87490740 1 --2- 172.21.15.12:0/3686090181 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5589be28d0c0 0x5589be294540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:16:27.391 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:27.386+0000 7fbb87490740 1 --2- 172.21.15.12:0/3686090181 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5589be28a790 0x5589be293ea0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:16:27.391 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:27.386+0000 7fbb63400640 0 client.26564 ms_handle_reset on v2:172.21.15.165:6838/501312984 2024-09-11T00:16:27.391 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:27.386+0000 7fbb87490740 1 -- 172.21.15.12:0/3686090181 >> 172.21.15.12:0/3686090181 conn(0x5589be2858e0 msgr2=0x5589be284a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:16:27.391 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:27.386+0000 7fbb87490740 1 -- 172.21.15.12:0/3686090181 shutdown_connections 2024-09-11T00:16:27.392 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:27.386+0000 7fbb87490740 1 -- 172.21.15.12:0/3686090181 wait complete. 2024-09-11T00:16:27.392 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:27.386+0000 7fbb87490740 20 client.26564 trim_cache size 0 max 16384 2024-09-11T00:16:27.405 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:27.403+0000 7fbb87490740 1 Processor -- start 2024-09-11T00:16:27.405 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:27.403+0000 7fbb87490740 1 -- start start 2024-09-11T00:16:27.405 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:27.403+0000 7fbb87490740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5589be28a790 0x5589be287710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:16:27.405 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:27.403+0000 7fbb87490740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5589be28d0c0 0x5589be287c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:16:27.405 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:27.403+0000 7fbb87490740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5589be28fb00 0x5589be288190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:16:27.406 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:27.403+0000 7fbb87490740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fbb7c1ad9b0 con 0x5589be28fb00 2024-09-11T00:16:27.406 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:27.403+0000 7fbb87490740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fbb7c1af4f0 con 0x5589be28d0c0 2024-09-11T00:16:27.406 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:27.403+0000 7fbb87490740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fbb7c1ae080 con 0x5589be28a790 2024-09-11T00:16:27.407 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:27.403+0000 7fbb81c00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5589be28a790 0x5589be287710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:16:27.407 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:27.403+0000 7fbb81200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5589be28d0c0 0x5589be287c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:16:27.407 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:27.403+0000 7fbb81c00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5589be28a790 0x5589be287710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.12:51500/0 (socket says 172.21.15.12:51500) 2024-09-11T00:16:27.407 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:27.403+0000 7fbb81c00640 1 -- 172.21.15.12:0/3264977259 learned_addr learned my addr 172.21.15.12:0/3264977259 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:16:27.407 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:27.404+0000 7fbb82600640 1 --2- 172.21.15.12:0/3264977259 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5589be28fb00 0x5589be288190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:16:27.407 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:27.404+0000 7fbb81c00640 1 -- 172.21.15.12:0/3264977259 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5589be28d0c0 msgr2=0x5589be287c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:16:27.407 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:27.404+0000 7fbb81c00640 1 --2- 172.21.15.12:0/3264977259 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5589be28d0c0 0x5589be287c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:16:27.407 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:27.404+0000 7fbb81c00640 1 -- 172.21.15.12:0/3264977259 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5589be28fb00 msgr2=0x5589be288190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:16:27.408 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:27.404+0000 7fbb81c00640 1 --2- 172.21.15.12:0/3264977259 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5589be28fb00 0x5589be288190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:16:27.408 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:27.404+0000 7fbb81c00640 1 -- 172.21.15.12:0/3264977259 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5589be28f820 con 0x5589be28a790 2024-09-11T00:16:27.408 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:27.404+0000 7fbb81c00640 1 --2- 172.21.15.12:0/3264977259 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5589be28a790 0x5589be287710 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fbb70004520 tx=0x7fbb700046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4d37e0d3ea34cf16 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:16:27.408 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:27.404+0000 7fbb6e000640 1 -- 172.21.15.12:0/3264977259 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb70035040 con 0x5589be28a790 2024-09-11T00:16:27.408 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:27.404+0000 7fbb6e000640 1 -- 172.21.15.12:0/3264977259 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbb70040070 con 0x5589be28a790 2024-09-11T00:16:27.408 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:27.404+0000 7fbb6e000640 1 -- 172.21.15.12:0/3264977259 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb70003940 con 0x5589be28a790 2024-09-11T00:16:27.408 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:27.405+0000 7fbb87490740 1 -- 172.21.15.12:0/3264977259 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5589be28a790 msgr2=0x5589be287710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:16:27.408 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:27.405+0000 7fbb87490740 1 --2- 172.21.15.12:0/3264977259 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5589be28a790 0x5589be287710 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fbb70004520 tx=0x7fbb700046c0 comp rx=0 tx=0).stop 2024-09-11T00:16:27.408 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:27.405+0000 7fbb87490740 1 -- 172.21.15.12:0/3264977259 shutdown_connections 2024-09-11T00:16:27.408 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:27.405+0000 7fbb87490740 1 --2- 172.21.15.12:0/3264977259 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5589be28fb00 0x5589be288190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:16:27.408 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:27.405+0000 7fbb87490740 1 --2- 172.21.15.12:0/3264977259 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5589be28d0c0 0x5589be287c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:16:27.408 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:27.405+0000 7fbb87490740 1 --2- 172.21.15.12:0/3264977259 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5589be28a790 0x5589be287710 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:16:27.408 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:27.405+0000 7fbb87490740 1 -- 172.21.15.12:0/3264977259 >> 172.21.15.12:0/3264977259 conn(0x5589be2858e0 msgr2=0x5589be28ff70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:16:27.408 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:27.405+0000 7fbb87490740 1 -- 172.21.15.12:0/3264977259 shutdown_connections 2024-09-11T00:16:27.409 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:27.405+0000 7fbb87490740 1 -- 172.21.15.12:0/3264977259 wait complete. 2024-09-11T00:16:27.411 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:27.405+0000 7fbb87490740 1 -- start start 2024-09-11T00:16:27.411 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:27.406+0000 7fbb87490740 1 Processor -- start 2024-09-11T00:16:27.411 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:27.406+0000 7fbb87490740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5589be28a790 0x5589be29b440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:16:27.411 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:27.406+0000 7fbb87490740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5589be28d0c0 0x5589be29bae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:16:27.412 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:27.406+0000 7fbb87490740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5589be28fb00 0x5589be299dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:16:27.412 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:27.406+0000 7fbb87490740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fbb7c1aee20 con 0x5589be28d0c0 2024-09-11T00:16:27.412 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:27.406+0000 7fbb87490740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fbb7c1ae750 con 0x5589be28a790 2024-09-11T00:16:27.412 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:27.406+0000 7fbb87490740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fbb7c1ab990 con 0x5589be28fb00 2024-09-11T00:16:27.412 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:27.406+0000 7fbb87490740 10 client.0 _ll_register_callbacks cb 0x7fbb74009ee0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:16:27.412 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:27.406+0000 7fbb81c00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5589be28a790 0x5589be29b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:16:27.412 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:27.406+0000 7fbb81c00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5589be28a790 0x5589be29b440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.12:59532/0 (socket says 172.21.15.12:59532) 2024-09-11T00:16:27.412 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:27.406+0000 7fbb81c00640 1 -- 172.21.15.12:0/3176848762 learned_addr learned my addr 172.21.15.12:0/3176848762 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:16:27.412 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:27.406+0000 7fbb82600640 1 --2- 172.21.15.12:0/3176848762 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5589be28fb00 0x5589be299dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:16:27.412 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:27.406+0000 7fbb81200640 1 --2- 172.21.15.12:0/3176848762 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5589be28d0c0 0x5589be29bae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:16:27.412 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:27.407+0000 7fbb81c00640 1 -- 172.21.15.12:0/3176848762 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5589be28fb00 msgr2=0x5589be299dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:16:27.412 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:27.407+0000 7fbb81c00640 1 --2- 172.21.15.12:0/3176848762 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5589be28fb00 0x5589be299dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:16:27.412 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:27.407+0000 7fbb81c00640 1 -- 172.21.15.12:0/3176848762 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5589be28d0c0 msgr2=0x5589be29bae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:16:27.413 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:27.407+0000 7fbb81c00640 1 --2- 172.21.15.12:0/3176848762 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5589be28d0c0 0x5589be29bae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:16:27.413 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:27.407+0000 7fbb81c00640 1 -- 172.21.15.12:0/3176848762 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5589be29a5d0 con 0x5589be28a790 2024-09-11T00:16:27.413 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:27.407+0000 7fbb81200640 1 --2- 172.21.15.12:0/3176848762 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5589be28d0c0 0x5589be29bae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T00:16:27.413 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:27.407+0000 7fbb81c00640 1 --2- 172.21.15.12:0/3176848762 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5589be28a790 0x5589be29b440 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fbb70006010 tx=0x7fbb70044800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4c32516146317dcb server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:16:27.413 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:27.407+0000 7fbb63400640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-09-11T00:16:27.413 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:27.407+0000 7fbb63400640 1 -- 172.21.15.12:0/3176848762 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb70035030 con 0x5589be28a790 2024-09-11T00:16:27.413 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:27.407+0000 7fbb87490740 1 -- 172.21.15.12:0/3176848762 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbb70040070 con 0x5589be28a790 2024-09-11T00:16:27.413 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:27.407+0000 7fbb63400640 1 -- 172.21.15.12:0/3176848762 <== mon.2 v2:172.21.15.165:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9208+0+0 (secure 0 0 0) 0x7fbb70044e00 con 0x5589be28a790 2024-09-11T00:16:27.413 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:27.407+0000 7fbb63400640 1 -- 172.21.15.12:0/3176848762 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7fbb70003920 con 0x5589be28a790 2024-09-11T00:16:27.413 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:27.408+0000 7fbb87490740 10 client.26570 fetch_fsmap learned FSMap version 41 2024-09-11T00:16:27.413 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:27.408+0000 7fbb87490740 1 -- 172.21.15.12:0/3176848762 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x5589be30cd80 con 0x5589be28a790 2024-09-11T00:16:27.413 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:27.408+0000 7fbb63400640 1 -- 172.21.15.12:0/3176848762 <== mon.2 v2:172.21.15.165:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbb70040070 con 0x5589be28a790 2024-09-11T00:16:27.413 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:27.408+0000 7fbb63400640 1 -- 172.21.15.12:0/3176848762 <== mon.2 v2:172.21.15.165:3300/0 5 ==== fsmap(e 41) ==== 8370+0+0 (secure 0 0 0) 0x7fbb7003cab0 con 0x5589be28a790 2024-09-11T00:16:27.413 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:27.408+0000 7fbb87490740 10 client.26570 fetch_fsmap finished waiting for FSMap version 41 2024-09-11T00:16:27.413 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:27.408+0000 7fbb87490740 10 client.26570 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T00:16:27.414 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:27.408+0000 7fbb87490740 20 client.26570 populate_metadata read hostname 'smithi012' 2024-09-11T00:16:27.414 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:27.408+0000 7fbb87490740 1 --2- 172.21.15.12:0/3176848762 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x5589be30d580 0x5589be30f9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:16:27.414 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:27.408+0000 7fbb87490740 4 client.26570 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T00:16:27.414 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:27.408+0000 7fbb87490740 1 -- 172.21.15.12:0/3176848762 --> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5589be1fb0b0 con 0x5589be30d580 2024-09-11T00:16:27.414 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:27.408+0000 7fbb81200640 1 --2- 172.21.15.12:0/3176848762 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x5589be30d580 0x5589be30f9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:16:27.414 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:27.409+0000 7fbb81200640 1 --2- 172.21.15.12:0/3176848762 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x5589be30d580 0x5589be30f9a0 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bc2711476125635b server_cookie=ca69fcbebefe399 in_seq=0 out_seq=0 2024-09-11T00:16:27.414 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:27.409+0000 7fbb63400640 10 client.26570 ms_handle_connect on v2:172.21.15.165:6838/501312984 2024-09-11T00:16:27.474 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:27.471+0000 7fbb63400640 1 -- 172.21.15.12:0/3176848762 <== mds.0 v2:172.21.15.165:6838/501312984 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5589be1fb0b0 con 0x5589be30d580 2024-09-11T00:16:27.474 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:27.471+0000 7fbb63400640 10 client.26570 handle_command_reply: tid=0 2024-09-11T00:16:27.474 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:27.471+0000 7fbb87490740 1 client.26570 shutdown 2024-09-11T00:16:27.474 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:27.472+0000 7fbb87490740 1 -- 172.21.15.12:0/3176848762 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5589be28a790 msgr2=0x5589be29b440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:16:27.474 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:27.472+0000 7fbb87490740 1 --2- 172.21.15.12:0/3176848762 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5589be28a790 0x5589be29b440 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fbb70006010 tx=0x7fbb70044800 comp rx=0 tx=0).stop 2024-09-11T00:16:27.474 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:27.472+0000 7fbb87490740 1 -- 172.21.15.12:0/3176848762 shutdown_connections 2024-09-11T00:16:27.474 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:27.472+0000 7fbb87490740 1 --2- 172.21.15.12:0/3176848762 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x5589be30d580 0x5589be30f9a0 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:16:27.474 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:27.472+0000 7fbb87490740 1 --2- 172.21.15.12:0/3176848762 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5589be28fb00 0x5589be299dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:16:27.475 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:27.472+0000 7fbb63400640 0 client.26570 ms_handle_reset on v2:172.21.15.165:6838/501312984 2024-09-11T00:16:27.475 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:27.472+0000 7fbb87490740 1 --2- 172.21.15.12:0/3176848762 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5589be28d0c0 0x5589be29bae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:16:27.475 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:27.472+0000 7fbb87490740 1 --2- 172.21.15.12:0/3176848762 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5589be28a790 0x5589be29b440 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:16:27.475 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:27.472+0000 7fbb87490740 1 -- 172.21.15.12:0/3176848762 >> 172.21.15.12:0/3176848762 conn(0x5589be2858e0 msgr2=0x5589be284240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:16:27.475 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:27.472+0000 7fbb87490740 1 -- 172.21.15.12:0/3176848762 shutdown_connections 2024-09-11T00:16:27.475 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:27.472+0000 7fbb87490740 1 -- 172.21.15.12:0/3176848762 wait complete. 2024-09-11T00:16:27.475 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:27.472+0000 7fbb87490740 20 client.26570 trim_cache size 0 max 16384 2024-09-11T00:16:27.475 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:{ 2024-09-11T00:16:27.475 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "message": "", 2024-09-11T00:16:27.476 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "return_code": 0 2024-09-11T00:16:27.476 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:} 2024-09-11T00:16:27.476 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:27.473+0000 7fbb84600640 1 -- 172.21.15.12:0/1856767109 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fbb640b52e0 msgr2=0x7fbb640b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:16:27.476 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:27.473+0000 7fbb84600640 1 --2- 172.21.15.12:0/1856767109 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fbb640b52e0 0x7fbb640b77a0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fbb74009fd0 tx=0x7fbb74009500 comp rx=0 tx=0).stop 2024-09-11T00:16:27.476 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:27.473+0000 7fbb84600640 1 -- 172.21.15.12:0/1856767109 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbb7c105cf0 msgr2=0x7fbb7c19b5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:16:27.477 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:27.473+0000 7fbb84600640 1 --2- 172.21.15.12:0/1856767109 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbb7c105cf0 0x7fbb7c19b5c0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fbb6800e9f0 tx=0x7fbb6800eeb0 comp rx=0 tx=0).stop 2024-09-11T00:16:27.477 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:27.473+0000 7fbb84600640 1 -- 172.21.15.12:0/1856767109 shutdown_connections 2024-09-11T00:16:27.477 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:27.473+0000 7fbb84600640 1 --2- 172.21.15.12:0/1856767109 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fbb640b52e0 0x7fbb640b77a0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:16:27.477 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:27.473+0000 7fbb84600640 1 --2- 172.21.15.12:0/1856767109 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbb7c1066b0 0x7fbb7c19c1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:16:27.477 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:27.473+0000 7fbb84600640 1 --2- 172.21.15.12:0/1856767109 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbb7c105cf0 0x7fbb7c19b5c0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:16:27.477 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:27.473+0000 7fbb84600640 1 --2- 172.21.15.12:0/1856767109 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbb7c104ba0 0x7fbb7c19af70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:16:27.477 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:27.473+0000 7fbb84600640 1 -- 172.21.15.12:0/1856767109 >> 172.21.15.12:0/1856767109 conn(0x7fbb7c100440 msgr2=0x7fbb7c1090e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:16:27.477 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:27.473+0000 7fbb84600640 1 -- 172.21.15.12:0/1856767109 shutdown_connections 2024-09-11T00:16:27.477 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:27.473+0000 7fbb84600640 1 -- 172.21.15.12:0/1856767109 wait complete. 2024-09-11T00:16:27.668 INFO:tasks.workunit.client.0.smithi012.stdout: 1 378415 3.95 MB/sec execute 384 sec latency 22.202 ms 2024-09-11T00:16:28.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:16:27 smithi165 ceph-mon[35356]: pgmap v680: 129 pgs: 129 active+clean; 97 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 2.1 MiB/s wr, 374 op/s 2024-09-11T00:16:28.063 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:16:27 smithi012 ceph-mon[30333]: pgmap v680: 129 pgs: 129 active+clean; 97 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 2.1 MiB/s wr, 374 op/s 2024-09-11T00:16:28.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:16:27 smithi178 ceph-mon[35919]: pgmap v680: 129 pgs: 129 active+clean; 97 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 2.1 MiB/s wr, 374 op/s 2024-09-11T00:16:28.668 INFO:tasks.workunit.client.0.smithi012.stdout: 1 379946 3.97 MB/sec execute 385 sec latency 11.562 ms 2024-09-11T00:16:29.668 INFO:tasks.workunit.client.0.smithi012.stdout: 1 381903 3.98 MB/sec execute 386 sec latency 17.802 ms 2024-09-11T00:16:30.049 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:16:29 smithi012 ceph-mon[30333]: pgmap v681: 129 pgs: 129 active+clean; 97 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 1.7 MiB/s wr, 241 op/s 2024-09-11T00:16:30.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:16:29 smithi178 ceph-mon[35919]: pgmap v681: 129 pgs: 129 active+clean; 97 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 1.7 MiB/s wr, 241 op/s 2024-09-11T00:16:30.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:16:29 smithi165 ceph-mon[35356]: pgmap v681: 129 pgs: 129 active+clean; 97 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 1.7 MiB/s wr, 241 op/s 2024-09-11T00:16:30.668 INFO:tasks.workunit.client.0.smithi012.stdout: 1 382310 3.98 MB/sec execute 387 sec latency 591.849 ms 2024-09-11T00:16:31.668 INFO:tasks.workunit.client.0.smithi012.stdout: 1 382477 3.97 MB/sec execute 388 sec latency 1441.446 ms 2024-09-11T00:16:32.063 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:16:31 smithi012 ceph-mon[30333]: pgmap v682: 129 pgs: 129 active+clean; 96 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 2.0 MiB/s wr, 302 op/s 2024-09-11T00:16:32.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:16:31 smithi178 ceph-mon[35919]: pgmap v682: 129 pgs: 129 active+clean; 96 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 2.0 MiB/s wr, 302 op/s 2024-09-11T00:16:32.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:16:31 smithi165 ceph-mon[35356]: pgmap v682: 129 pgs: 129 active+clean; 96 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 2.0 MiB/s wr, 302 op/s 2024-09-11T00:16:32.668 INFO:tasks.workunit.client.0.smithi012.stdout: 1 383781 3.98 MB/sec execute 389 sec latency 14.474 ms 2024-09-11T00:16:33.668 INFO:tasks.workunit.client.0.smithi012.stdout: 1 385940 4.00 MB/sec execute 390 sec latency 11.582 ms 2024-09-11T00:16:34.063 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:16:33 smithi012 ceph-mon[30333]: pgmap v683: 129 pgs: 129 active+clean; 97 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 1.8 MiB/s wr, 307 op/s 2024-09-11T00:16:34.080 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:16:33 smithi178 ceph-mon[35919]: pgmap v683: 129 pgs: 129 active+clean; 97 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 1.8 MiB/s wr, 307 op/s 2024-09-11T00:16:34.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:16:33 smithi165 ceph-mon[35356]: pgmap v683: 129 pgs: 129 active+clean; 97 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 1.8 MiB/s wr, 307 op/s 2024-09-11T00:16:34.668 INFO:tasks.workunit.client.0.smithi012.stdout: 1 387233 4.01 MB/sec execute 391 sec latency 12.147 ms 2024-09-11T00:16:35.669 INFO:tasks.workunit.client.0.smithi012.stdout: 1 387864 4.00 MB/sec execute 392 sec latency 560.725 ms 2024-09-11T00:16:36.054 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:16:35 smithi012 ceph-mon[30333]: pgmap v684: 129 pgs: 129 active+clean; 97 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 1.5 MiB/s wr, 260 op/s 2024-09-11T00:16:36.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:16:35 smithi178 ceph-mon[35919]: pgmap v684: 129 pgs: 129 active+clean; 97 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 1.5 MiB/s wr, 260 op/s 2024-09-11T00:16:36.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:16:35 smithi165 ceph-mon[35356]: pgmap v684: 129 pgs: 129 active+clean; 97 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 1.5 MiB/s wr, 260 op/s 2024-09-11T00:16:36.491 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 33.4 2024-09-11T00:16:36.492 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid ff3ad74a-6fcf-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T00:16:36.669 INFO:tasks.workunit.client.0.smithi012.stdout: 1 388285 4.00 MB/sec execute 393 sec latency 1253.901 ms 2024-09-11T00:16:36.935 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:16:37.538 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:16:37.534+0000 7f2a97e00640 1 -- 172.21.15.12:0/1934009950 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2a98108fa0 msgr2=0x7f2a9810f7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:16:37.538 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:16:37.534+0000 7f2a97e00640 1 --2- 172.21.15.12:0/1934009950 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2a98108fa0 0x7f2a9810f7a0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f2a90009ca0 tx=0x7f2a9002a8c0 comp rx=0 tx=0).stop 2024-09-11T00:16:37.538 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:16:37.534+0000 7f2a97e00640 1 -- 172.21.15.12:0/1934009950 shutdown_connections 2024-09-11T00:16:37.538 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:16:37.534+0000 7f2a97e00640 1 --2- 172.21.15.12:0/1934009950 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2a98108fa0 0x7f2a9810f7a0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:16:37.539 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:16:37.534+0000 7f2a97e00640 1 --2- 172.21.15.12:0/1934009950 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2a98108620 0x7f2a98108a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:16:37.539 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:16:37.534+0000 7f2a97e00640 1 --2- 172.21.15.12:0/1934009950 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2a98102c10 0x7f2a98102ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:16:37.539 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:16:37.534+0000 7f2a97e00640 1 -- 172.21.15.12:0/1934009950 >> 172.21.15.12:0/1934009950 conn(0x7f2a980fe660 msgr2=0x7f2a98100a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:16:37.539 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:16:37.534+0000 7f2a97e00640 1 -- 172.21.15.12:0/1934009950 shutdown_connections 2024-09-11T00:16:37.539 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:16:37.534+0000 7f2a97e00640 1 -- 172.21.15.12:0/1934009950 wait complete. 2024-09-11T00:16:37.539 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:16:37.536+0000 7f2a97e00640 1 Processor -- start 2024-09-11T00:16:37.539 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:16:37.536+0000 7f2a97e00640 1 -- start start 2024-09-11T00:16:37.541 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:16:37.536+0000 7f2a97e00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2a98102c10 0x7f2a9819b230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:16:37.541 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:16:37.536+0000 7f2a97e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2a98108620 0x7f2a9819b770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:16:37.541 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:16:37.536+0000 7f2a97e00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2a98108fa0 0x7f2a981a0b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:16:37.541 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:16:37.536+0000 7f2a97e00640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f2a98112c60 con 0x7f2a98102c10 2024-09-11T00:16:37.542 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:16:37.536+0000 7f2a97e00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f2a98112ae0 con 0x7f2a98108620 2024-09-11T00:16:37.542 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:16:37.536+0000 7f2a97e00640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f2a98112de0 con 0x7f2a98108fa0 2024-09-11T00:16:37.542 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:16:37.537+0000 7f2a96a00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2a98102c10 0x7f2a9819b230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:16:37.542 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:16:37.537+0000 7f2a96a00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2a98102c10 0x7f2a9819b230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:58834/0 (socket says 172.21.15.12:58834) 2024-09-11T00:16:37.542 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:16:37.537+0000 7f2a96a00640 1 -- 172.21.15.12:0/282037918 learned_addr learned my addr 172.21.15.12:0/282037918 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:16:37.542 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:16:37.537+0000 7f2a97400640 1 --2- 172.21.15.12:0/282037918 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2a98108fa0 0x7f2a981a0b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:16:37.542 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:16:37.537+0000 7f2a96000640 1 --2- 172.21.15.12:0/282037918 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2a98108620 0x7f2a9819b770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:16:37.542 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:16:37.538+0000 7f2a96000640 1 -- 172.21.15.12:0/282037918 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2a98108fa0 msgr2=0x7f2a981a0b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:16:37.543 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:16:37.538+0000 7f2a96000640 1 --2- 172.21.15.12:0/282037918 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2a98108fa0 0x7f2a981a0b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:16:37.543 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:16:37.538+0000 7f2a96000640 1 -- 172.21.15.12:0/282037918 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2a98102c10 msgr2=0x7f2a9819b230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:16:37.543 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:16:37.538+0000 7f2a96000640 1 --2- 172.21.15.12:0/282037918 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2a98102c10 0x7f2a9819b230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:16:37.543 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:16:37.539+0000 7f2a96000640 1 -- 172.21.15.12:0/282037918 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a981a12e0 con 0x7f2a98108620 2024-09-11T00:16:37.543 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:16:37.539+0000 7f2a96000640 1 --2- 172.21.15.12:0/282037918 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2a98108620 0x7f2a9819b770 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f2a8400e9f0 tx=0x7f2a8400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:16:37.543 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:16:37.539+0000 7f2a8f400640 1 -- 172.21.15.12:0/282037918 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a8400ccc0 con 0x7f2a98108620 2024-09-11T00:16:37.544 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:16:37.539+0000 7f2a97e00640 1 -- 172.21.15.12:0/282037918 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2a981a15d0 con 0x7f2a98108620 2024-09-11T00:16:37.544 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:16:37.540+0000 7f2a8f400640 1 -- 172.21.15.12:0/282037918 <== mon.2 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2a84005020 con 0x7f2a98108620 2024-09-11T00:16:37.544 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:16:37.540+0000 7f2a8f400640 1 -- 172.21.15.12:0/282037918 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a84004a00 con 0x7f2a98108620 2024-09-11T00:16:37.544 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:16:37.541+0000 7f2a97e00640 1 -- 172.21.15.12:0/282037918 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2a981a7c90 con 0x7f2a98108620 2024-09-11T00:16:37.548 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:16:37.545+0000 7f2a97e00640 1 -- 172.21.15.12:0/282037918 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2a98102ff0 con 0x7f2a98108620 2024-09-11T00:16:37.549 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:16:37.547+0000 7f2a8f400640 1 -- 172.21.15.12:0/282037918 <== mon.2 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7f2a84004ba0 con 0x7f2a98108620 2024-09-11T00:16:37.551 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:16:37.549+0000 7f2a8f400640 1 --2- 172.21.15.12:0/282037918 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f2a7c0b5390 0x7f2a7c0b7850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:16:37.551 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:16:37.549+0000 7f2a8f400640 1 -- 172.21.15.12:0/282037918 <== mon.2 v2:172.21.15.165:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9208+0+0 (secure 0 0 0) 0x7f2a840e5fd0 con 0x7f2a98108620 2024-09-11T00:16:37.551 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:16:37.549+0000 7f2a96a00640 1 --2- 172.21.15.12:0/282037918 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f2a7c0b5390 0x7f2a7c0b7850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:16:37.552 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:16:37.550+0000 7f2a96a00640 1 --2- 172.21.15.12:0/282037918 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f2a7c0b5390 0x7f2a7c0b7850 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f2a88009c90 tx=0x7f2a88005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:16:37.554 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:16:37.552+0000 7f2a8f400640 1 -- 172.21.15.12:0/282037918 <== mon.2 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f2a840acac0 con 0x7f2a98108620 2024-09-11T00:16:37.669 INFO:tasks.workunit.client.0.smithi012.stdout: 1 390370 4.02 MB/sec execute 394 sec latency 13.960 ms 2024-09-11T00:16:37.825 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:16:37.823+0000 7f2a97e00640 1 -- 172.21.15.12:0/282037918 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2a9819c360 con 0x7f2a98108620 2024-09-11T00:16:37.825 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:16:37.823+0000 7f2a8f400640 1 -- 172.21.15.12:0/282037918 <== mon.2 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 41 v41) ==== 94+0+11965 (secure 0 0 0) 0x7f2a840ac3c0 con 0x7f2a98108620 2024-09-11T00:16:37.826 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-11T00:16:37.826 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":41,"btime":"2024-09-11T00:14:31:349726+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14577,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6835/447171591","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":447171591},{"type":"v1","addr":"172.21.15.12:6835","nonce":447171591}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14589,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/901486040","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":901486040},{"type":"v1","addr":"172.21.15.12:6837","nonce":901486040}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":41,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T00:04:04.739881+0000","modified":"2024-09-11T00:14:30.594242+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24674,"mds_1":24643,"mds_2":24638,"mds_3":24637,"mds_4":24614},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14631":{"gid":14631,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.12:6841/3069919437","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6840","nonce":3069919437},{"type":"v1","addr":"172.21.15.12:6841","nonce":3069919437}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24562":{"gid":24562,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6833/1141636112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":1141636112},{"type":"v1","addr":"172.21.15.178:6833","nonce":1141636112}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24581":{"gid":24581,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6833/3111980970","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6832","nonce":3111980970},{"type":"v1","addr":"172.21.15.165:6833","nonce":3111980970}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24607":{"gid":24607,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6835/4084868287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":4084868287},{"type":"v1","addr":"172.21.15.178:6835","nonce":4084868287}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24614":{"gid":24614,"name":"e","rank":4,"incarnation":26,"state":"up:active","state_seq":21,"addr":"172.21.15.165:6835/2425395274","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":2425395274},{"type":"v1","addr":"172.21.15.165:6835","nonce":2425395274}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24637":{"gid":24637,"name":"i","rank":3,"incarnation":24,"state":"up:active","state_seq":18,"addr":"172.21.15.178:6839/70628097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6838","nonce":70628097},{"type":"v1","addr":"172.21.15.178:6839","nonce":70628097}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24638":{"gid":24638,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":18,"addr":"172.21.15.165:6837/6215088","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6836","nonce":6215088},{"type":"v1","addr":"172.21.15.165:6837","nonce":6215088}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24643":{"gid":24643,"name":"g","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.12:6839/3124093009","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6838","nonce":3124093009},{"type":"v1","addr":"172.21.15.12:6839","nonce":3124093009}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24674":{"gid":24674,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.165:6839/501312984","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6838","nonce":501312984},{"type":"v1","addr":"172.21.15.165:6839","nonce":501312984}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25672":{"gid":25672,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6837/308601741","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":308601741},{"type":"v1","addr":"172.21.15.178:6837","nonce":308601741}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24674,"qdb_cluster":[24614,24637,24638,24643,24674]},"id":1}]} 2024-09-11T00:16:37.827 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 41 2024-09-11T00:16:37.829 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:16:37.826+0000 7f2a97e00640 1 -- 172.21.15.12:0/282037918 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f2a7c0b5390 msgr2=0x7f2a7c0b7850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:16:37.829 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:16:37.826+0000 7f2a97e00640 1 --2- 172.21.15.12:0/282037918 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f2a7c0b5390 0x7f2a7c0b7850 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f2a88009c90 tx=0x7f2a88005f90 comp rx=0 tx=0).stop 2024-09-11T00:16:37.829 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:16:37.826+0000 7f2a97e00640 1 -- 172.21.15.12:0/282037918 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2a98108620 msgr2=0x7f2a9819b770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:16:37.829 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:16:37.826+0000 7f2a97e00640 1 --2- 172.21.15.12:0/282037918 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2a98108620 0x7f2a9819b770 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f2a8400e9f0 tx=0x7f2a8400eeb0 comp rx=0 tx=0).stop 2024-09-11T00:16:37.829 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:16:37.826+0000 7f2a97e00640 1 -- 172.21.15.12:0/282037918 shutdown_connections 2024-09-11T00:16:37.829 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:16:37.826+0000 7f2a97e00640 1 --2- 172.21.15.12:0/282037918 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f2a7c0b5390 0x7f2a7c0b7850 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:16:37.829 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:16:37.827+0000 7f2a97e00640 1 --2- 172.21.15.12:0/282037918 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2a98108fa0 0x7f2a981a0b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:16:37.829 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:16:37.827+0000 7f2a97e00640 1 --2- 172.21.15.12:0/282037918 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2a98108620 0x7f2a9819b770 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:16:37.830 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:16:37.827+0000 7f2a97e00640 1 --2- 172.21.15.12:0/282037918 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2a98102c10 0x7f2a9819b230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:16:37.830 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:16:37.827+0000 7f2a97e00640 1 -- 172.21.15.12:0/282037918 >> 172.21.15.12:0/282037918 conn(0x7f2a980fe660 msgr2=0x7f2a98104bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:16:37.830 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:16:37.827+0000 7f2a97e00640 1 -- 172.21.15.12:0/282037918 shutdown_connections 2024-09-11T00:16:37.830 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:16:37.827+0000 7f2a97e00640 1 -- 172.21.15.12:0/282037918 wait complete. 2024-09-11T00:16:38.063 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:16:37 smithi012 ceph-mon[30333]: pgmap v685: 129 pgs: 129 active+clean; 98 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 MiB/s rd, 2.1 MiB/s wr, 359 op/s 2024-09-11T00:16:38.080 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:16:37 smithi178 ceph-mon[35919]: pgmap v685: 129 pgs: 129 active+clean; 98 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 MiB/s rd, 2.1 MiB/s wr, 359 op/s 2024-09-11T00:16:38.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:16:37 smithi165 ceph-mon[35356]: pgmap v685: 129 pgs: 129 active+clean; 98 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 MiB/s rd, 2.1 MiB/s wr, 359 op/s 2024-09-11T00:16:38.669 INFO:tasks.workunit.client.0.smithi012.stdout: 1 391441 4.02 MB/sec execute 395 sec latency 11.807 ms 2024-09-11T00:16:38.714 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24674 quiesce db / --timeout 90 --expiration 153.4 --await-for 45.0' 2024-09-11T00:16:38.714 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid ff3ad74a-6fcf-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24674 quiesce db / --timeout 90 --expiration 153.4 --await-for 45.0 2024-09-11T00:16:39.063 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:16:38 smithi012 ceph-mon[30333]: from='client.? 172.21.15.12:0/282037918' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T00:16:39.080 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:16:38 smithi178 ceph-mon[35919]: from='client.? 172.21.15.12:0/282037918' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T00:16:39.152 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:16:39.183 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:16:38 smithi165 ceph-mon[35356]: from='client.? 172.21.15.12:0/282037918' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T00:16:39.669 INFO:tasks.workunit.client.0.smithi012.stdout: 1 393688 4.05 MB/sec execute 396 sec latency 12.575 ms 2024-09-11T00:16:39.767 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:16:39.764+0000 7feac3c00640 1 -- 172.21.15.12:0/4183105091 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7feabc10a9c0 msgr2=0x7feabc10ae00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:16:39.767 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:16:39.764+0000 7feac3c00640 1 --2- 172.21.15.12:0/4183105091 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7feabc10a9c0 0x7feabc10ae00 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7feaa8009900 tx=0x7feaa802a3f0 comp rx=0 tx=0).stop 2024-09-11T00:16:39.767 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:16:39.765+0000 7feac3c00640 1 -- 172.21.15.12:0/4183105091 shutdown_connections 2024-09-11T00:16:39.768 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:16:39.765+0000 7feac3c00640 1 --2- 172.21.15.12:0/4183105091 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7feabc10b340 0x7feabc111b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:16:39.768 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:16:39.765+0000 7feac3c00640 1 --2- 172.21.15.12:0/4183105091 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7feabc10a9c0 0x7feabc10ae00 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:16:39.768 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:16:39.765+0000 7feac3c00640 1 --2- 172.21.15.12:0/4183105091 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7feabc104fb0 0x7feabc105390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:16:39.768 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:16:39.765+0000 7feac3c00640 1 -- 172.21.15.12:0/4183105091 >> 172.21.15.12:0/4183105091 conn(0x7feabc100a20 msgr2=0x7feabc102e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:16:39.768 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:16:39.765+0000 7feac3c00640 1 -- 172.21.15.12:0/4183105091 shutdown_connections 2024-09-11T00:16:39.768 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:16:39.765+0000 7feac3c00640 1 -- 172.21.15.12:0/4183105091 wait complete. 2024-09-11T00:16:39.768 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:16:39.766+0000 7feac3c00640 1 Processor -- start 2024-09-11T00:16:39.769 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:16:39.766+0000 7feac3c00640 1 -- start start 2024-09-11T00:16:39.769 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:16:39.767+0000 7feac3c00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7feabc104fb0 0x7feabc19b4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:16:39.769 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:16:39.767+0000 7feac3c00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7feabc10a9c0 0x7feabc19b9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:16:39.769 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:16:39.767+0000 7feac3c00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7feabc10b340 0x7feabc1a0d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:16:39.769 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:16:39.767+0000 7feac3c00640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7feabc114fc0 con 0x7feabc10a9c0 2024-09-11T00:16:39.769 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:16:39.767+0000 7feac3c00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7feabc114e40 con 0x7feabc104fb0 2024-09-11T00:16:39.770 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:16:39.767+0000 7feac3c00640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7feabc115140 con 0x7feabc10b340 2024-09-11T00:16:39.770 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:16:39.767+0000 7feac1200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7feabc104fb0 0x7feabc19b4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:16:39.770 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:16:39.767+0000 7feac1c00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7feabc10b340 0x7feabc1a0d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:16:39.770 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:16:39.767+0000 7feac1200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7feabc104fb0 0x7feabc19b4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.12:43350/0 (socket says 172.21.15.12:43350) 2024-09-11T00:16:39.770 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:16:39.767+0000 7feac1200640 1 -- 172.21.15.12:0/2709363618 learned_addr learned my addr 172.21.15.12:0/2709363618 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:16:39.771 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:16:39.767+0000 7feabbe00640 1 --2- 172.21.15.12:0/2709363618 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7feabc10a9c0 0x7feabc19b9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:16:39.771 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:16:39.768+0000 7feac1200640 1 -- 172.21.15.12:0/2709363618 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7feabc10b340 msgr2=0x7feabc1a0d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:16:39.771 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:16:39.768+0000 7feac1200640 1 --2- 172.21.15.12:0/2709363618 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7feabc10b340 0x7feabc1a0d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:16:39.771 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:16:39.768+0000 7feac1200640 1 -- 172.21.15.12:0/2709363618 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7feabc10a9c0 msgr2=0x7feabc19b9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:16:39.771 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:16:39.768+0000 7feac1200640 1 --2- 172.21.15.12:0/2709363618 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7feabc10a9c0 0x7feabc19b9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:16:39.771 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:16:39.768+0000 7feac1200640 1 -- 172.21.15.12:0/2709363618 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feabc1a1560 con 0x7feabc104fb0 2024-09-11T00:16:39.771 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:16:39.768+0000 7feac1c00640 1 --2- 172.21.15.12:0/2709363618 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7feabc10b340 0x7feabc1a0d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:16:39.772 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:16:39.768+0000 7feac1200640 1 --2- 172.21.15.12:0/2709363618 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7feabc104fb0 0x7feabc19b4b0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7feab000ea60 tx=0x7feab000ef20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:16:39.772 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:16:39.768+0000 7feab9600640 1 -- 172.21.15.12:0/2709363618 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feab000cde0 con 0x7feabc104fb0 2024-09-11T00:16:39.772 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:16:39.768+0000 7feab9600640 1 -- 172.21.15.12:0/2709363618 <== mon.2 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feab00035f0 con 0x7feabc104fb0 2024-09-11T00:16:39.772 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:16:39.768+0000 7feab9600640 1 -- 172.21.15.12:0/2709363618 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feab0010690 con 0x7feabc104fb0 2024-09-11T00:16:39.772 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:16:39.768+0000 7feac3c00640 1 -- 172.21.15.12:0/2709363618 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feabc1a1850 con 0x7feabc104fb0 2024-09-11T00:16:39.772 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:16:39.769+0000 7feac3c00640 1 -- 172.21.15.12:0/2709363618 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feabc1a7f10 con 0x7feabc104fb0 2024-09-11T00:16:39.774 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:16:39.772+0000 7feac6688740 1 Processor -- start 2024-09-11T00:16:39.774 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:16:39.772+0000 7feac6688740 1 -- start start 2024-09-11T00:16:39.775 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:16:39.773+0000 7feab9600640 1 -- 172.21.15.12:0/2709363618 <== mon.2 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7feab0004000 con 0x7feabc104fb0 2024-09-11T00:16:39.775 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:16:39.773+0000 7feac6688740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5644e813e180 0x5644e813e560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:16:39.775 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:16:39.773+0000 7feac6688740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5644e813eaa0 0x5644e814a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:16:39.775 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:16:39.773+0000 7feac6688740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5644e814aa30 0x5644e814ce20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:16:39.776 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:16:39.773+0000 7feac6688740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x5644e80b3ca0 con 0x5644e814aa30 2024-09-11T00:16:39.776 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:16:39.773+0000 7feac6688740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x5644e80a30d0 con 0x5644e813eaa0 2024-09-11T00:16:39.776 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:16:39.773+0000 7feac6688740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5644e80a6ee0 con 0x5644e813e180 2024-09-11T00:16:39.776 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:16:39.773+0000 7feac1c00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5644e813e180 0x5644e813e560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:16:39.776 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:16:39.773+0000 7feac1c00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5644e813e180 0x5644e813e560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.12:40654/0 (socket says 172.21.15.12:40654) 2024-09-11T00:16:39.776 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:16:39.773+0000 7feac1c00640 1 -- 172.21.15.12:0/3419096687 learned_addr learned my addr 172.21.15.12:0/3419096687 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:16:39.776 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:16:39.773+0000 7feac1200640 1 --2- 172.21.15.12:0/3419096687 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5644e813eaa0 0x5644e814a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:16:39.776 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:16:39.773+0000 7feabbe00640 1 --2- 172.21.15.12:0/3419096687 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5644e814aa30 0x5644e814ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:16:39.777 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:16:39.774+0000 7feac1c00640 1 -- 172.21.15.12:0/3419096687 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5644e813eaa0 msgr2=0x5644e814a4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:16:39.777 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:16:39.774+0000 7feac1c00640 1 --2- 172.21.15.12:0/3419096687 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5644e813eaa0 0x5644e814a4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:16:39.777 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:16:39.774+0000 7feac1c00640 1 -- 172.21.15.12:0/3419096687 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5644e814aa30 msgr2=0x5644e814ce20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:16:39.777 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:16:39.774+0000 7feac1c00640 1 --2- 172.21.15.12:0/3419096687 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5644e814aa30 0x5644e814ce20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:16:39.778 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:16:39.774+0000 7feac1c00640 1 -- 172.21.15.12:0/3419096687 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5644e8055400 con 0x5644e813e180 2024-09-11T00:16:39.778 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:16:39.774+0000 7feac1200640 1 --2- 172.21.15.12:0/3419096687 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5644e813eaa0 0x5644e814a4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:16:39.778 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:16:39.774+0000 7feac1c00640 1 --2- 172.21.15.12:0/3419096687 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5644e813e180 0x5644e813e560 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7feab40117e0 tx=0x7feab4011ca0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=473c24888cff server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:16:39.778 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:16:39.774+0000 7feaae000640 1 -- 172.21.15.12:0/3419096687 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feab4011310 con 0x5644e813e180 2024-09-11T00:16:39.778 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:16:39.775+0000 7feaae000640 1 -- 172.21.15.12:0/3419096687 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feab4003e20 con 0x5644e813e180 2024-09-11T00:16:39.778 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:16:39.775+0000 7feaae000640 1 -- 172.21.15.12:0/3419096687 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feab40044f0 con 0x5644e813e180 2024-09-11T00:16:39.778 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:16:39.775+0000 7feab9600640 1 --2- 172.21.15.12:0/2709363618 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7feaa40b5360 0x7feaa40b7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:16:39.778 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:16:39.775+0000 7feab9600640 1 -- 172.21.15.12:0/2709363618 <== mon.2 v2:172.21.15.165:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9208+0+0 (secure 0 0 0) 0x7feab00e62b0 con 0x7feabc104fb0 2024-09-11T00:16:39.778 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:16:39.775+0000 7feac1200640 1 --2- 172.21.15.12:0/2709363618 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7feaa40b5360 0x7feaa40b7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:16:39.778 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:16:39.775+0000 7feac6688740 1 -- 172.21.15.12:0/3419096687 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5644e813e180 msgr2=0x5644e813e560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:16:39.778 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:16:39.775+0000 7feac6688740 1 --2- 172.21.15.12:0/3419096687 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5644e813e180 0x5644e813e560 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7feab40117e0 tx=0x7feab4011ca0 comp rx=0 tx=0).stop 2024-09-11T00:16:39.780 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:16:39.775+0000 7feac6688740 1 -- 172.21.15.12:0/3419096687 shutdown_connections 2024-09-11T00:16:39.780 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:16:39.775+0000 7feac6688740 1 --2- 172.21.15.12:0/3419096687 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5644e814aa30 0x5644e814ce20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:16:39.780 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:16:39.775+0000 7feac6688740 1 --2- 172.21.15.12:0/3419096687 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5644e813eaa0 0x5644e814a4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:16:39.780 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:16:39.775+0000 7feac6688740 1 --2- 172.21.15.12:0/3419096687 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5644e813e180 0x5644e813e560 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:16:39.780 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:16:39.776+0000 7feac6688740 1 -- 172.21.15.12:0/3419096687 >> 172.21.15.12:0/3419096687 conn(0x5644e813bfb0 msgr2=0x5644e813d150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:16:39.780 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:16:39.776+0000 7feac6688740 1 -- 172.21.15.12:0/3419096687 shutdown_connections 2024-09-11T00:16:39.780 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:16:39.776+0000 7feac6688740 1 -- 172.21.15.12:0/3419096687 wait complete. 2024-09-11T00:16:39.780 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:16:39.776+0000 7feac1200640 1 --2- 172.21.15.12:0/2709363618 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7feaa40b5360 0x7feaa40b7820 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7feab00ac340 tx=0x7feab00e9cf0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:16:39.780 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:16:39.776+0000 7feac6688740 1 -- start start 2024-09-11T00:16:39.780 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:16:39.777+0000 7feac6688740 1 Processor -- start 2024-09-11T00:16:39.780 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:16:39.777+0000 7feac6688740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5644e813e180 0x5644e814a340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:16:39.780 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:16:39.777+0000 7feac6688740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5644e813eaa0 0x5644e8147790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:16:39.781 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:16:39.777+0000 7feac6688740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5644e814aa30 0x5644e8147dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:16:39.781 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:16:39.777+0000 7feac6688740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x5644e7f047a0 con 0x5644e813eaa0 2024-09-11T00:16:39.781 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:16:39.777+0000 7feac6688740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x5644e8048a10 con 0x5644e813e180 2024-09-11T00:16:39.781 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:16:39.777+0000 7feac6688740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5644e814d6b0 con 0x5644e814aa30 2024-09-11T00:16:39.781 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:16:39.777+0000 7feac6688740 10 client.0 _ll_register_callbacks cb 0x5644e8058be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:16:39.781 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:16:39.777+0000 7feac1c00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5644e813e180 0x5644e814a340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:16:39.781 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:16:39.777+0000 7feabbe00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5644e813eaa0 0x5644e8147790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:16:39.781 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:16:39.777+0000 7feac1c00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5644e813e180 0x5644e814a340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.12:43356/0 (socket says 172.21.15.12:43356) 2024-09-11T00:16:39.781 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:16:39.778+0000 7feac1c00640 1 -- 172.21.15.12:0/993266685 learned_addr learned my addr 172.21.15.12:0/993266685 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:16:39.781 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:16:39.778+0000 7feac1c00640 1 --2- 172.21.15.12:0/993266685 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5644e814aa30 0x5644e8147dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:16:39.783 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:16:39.778+0000 7feabbe00640 1 -- 172.21.15.12:0/993266685 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5644e814aa30 msgr2=0x5644e8147dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:16:39.783 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:16:39.778+0000 7feabbe00640 1 --2- 172.21.15.12:0/993266685 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5644e814aa30 0x5644e8147dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:16:39.783 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:16:39.778+0000 7feabbe00640 1 -- 172.21.15.12:0/993266685 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5644e813e180 msgr2=0x5644e814a340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:16:39.783 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:16:39.778+0000 7feabbe00640 1 --2- 172.21.15.12:0/993266685 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5644e813e180 0x5644e814a340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:16:39.783 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:16:39.778+0000 7feabbe00640 1 -- 172.21.15.12:0/993266685 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5644e7e10b90 con 0x5644e813eaa0 2024-09-11T00:16:39.783 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:16:39.778+0000 7feabbe00640 1 --2- 172.21.15.12:0/993266685 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5644e813eaa0 0x5644e8147790 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7feaa802a9e0 tx=0x7feaa8005270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bde3bdf3ee0b9480 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:16:39.783 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:16:39.778+0000 7feaa2a00640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-09-11T00:16:39.783 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:16:39.779+0000 7feaa2a00640 1 -- 172.21.15.12:0/993266685 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feaa8035040 con 0x5644e813eaa0 2024-09-11T00:16:39.783 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:16:39.779+0000 7feaa2a00640 1 -- 172.21.15.12:0/993266685 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9208+0+0 (secure 0 0 0) 0x7feaa80048c0 con 0x5644e813eaa0 2024-09-11T00:16:39.783 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:16:39.779+0000 7feac6688740 1 -- 172.21.15.12:0/993266685 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feab4003e20 con 0x5644e813eaa0 2024-09-11T00:16:39.783 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:16:39.779+0000 7feaa2a00640 1 -- 172.21.15.12:0/993266685 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7feaa804e050 con 0x5644e813eaa0 2024-09-11T00:16:39.784 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:16:39.779+0000 7feac6688740 10 client.16581 fetch_fsmap learned FSMap version 41 2024-09-11T00:16:39.784 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:16:39.779+0000 7feac6688740 1 -- 172.21.15.12:0/993266685 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x5644e8159040 con 0x5644e813eaa0 2024-09-11T00:16:39.784 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:16:39.779+0000 7feaa2a00640 1 -- 172.21.15.12:0/993266685 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feab4003e20 con 0x5644e813eaa0 2024-09-11T00:16:39.784 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:16:39.779+0000 7feaa2a00640 1 -- 172.21.15.12:0/993266685 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 41) ==== 8370+0+0 (secure 0 0 0) 0x7feaa8003b20 con 0x5644e813eaa0 2024-09-11T00:16:39.784 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:16:39.780+0000 7feac6688740 10 client.16581 fetch_fsmap finished waiting for FSMap version 41 2024-09-11T00:16:39.784 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:16:39.780+0000 7feac6688740 10 client.16581 resolve_mds: validated gid 24674 aka daemon mds.b 2024-09-11T00:16:39.784 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:16:39.780+0000 7feac6688740 20 client.16581 populate_metadata read hostname 'smithi012' 2024-09-11T00:16:39.784 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:16:39.780+0000 7feac6688740 1 --2- 172.21.15.12:0/993266685 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x5644e815a500 0x5644e815c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:16:39.784 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:16:39.780+0000 7feac6688740 4 client.16581 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T00:16:39.784 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:16:39.780+0000 7feac6688740 1 -- 172.21.15.12:0/993266685 --> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5644e80b5bf0 con 0x5644e815a500 2024-09-11T00:16:39.784 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:16:39.780+0000 7feac1c00640 1 --2- 172.21.15.12:0/993266685 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x5644e815a500 0x5644e815c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:16:39.784 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:16:39.780+0000 7feac1c00640 1 --2- 172.21.15.12:0/993266685 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x5644e815a500 0x5644e815c8f0 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7e9470b58c9780b2 server_cookie=2159797eed097f92 in_seq=0 out_seq=0 2024-09-11T00:16:39.784 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:16:39.780+0000 7feaa2a00640 10 client.16581 ms_handle_connect on v2:172.21.15.165:6838/501312984 2024-09-11T00:16:39.786 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:16:39.781+0000 7feaa2a00640 1 -- 172.21.15.12:0/993266685 <== mds.0 v2:172.21.15.165:6838/501312984 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5644e80b5bf0 con 0x5644e815a500 2024-09-11T00:16:39.786 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:16:39.781+0000 7feaa2a00640 10 client.16581 handle_command_reply: tid=0 2024-09-11T00:16:39.786 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:16:39.781+0000 7feac6688740 1 client.16581 shutdown 2024-09-11T00:16:39.786 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:16:39.781+0000 7feac6688740 1 -- 172.21.15.12:0/993266685 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5644e813eaa0 msgr2=0x5644e8147790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:16:39.786 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:16:39.781+0000 7feac6688740 1 --2- 172.21.15.12:0/993266685 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5644e813eaa0 0x5644e8147790 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7feaa802a9e0 tx=0x7feaa8005270 comp rx=0 tx=0).stop 2024-09-11T00:16:39.786 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:16:39.781+0000 7feac6688740 1 -- 172.21.15.12:0/993266685 shutdown_connections 2024-09-11T00:16:39.786 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:16:39.781+0000 7feac6688740 1 --2- 172.21.15.12:0/993266685 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x5644e815a500 0x5644e815c8f0 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:16:39.786 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:16:39.781+0000 7feac6688740 1 --2- 172.21.15.12:0/993266685 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5644e814aa30 0x5644e8147dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:16:39.786 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:16:39.781+0000 7feac6688740 1 --2- 172.21.15.12:0/993266685 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5644e813eaa0 0x5644e8147790 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:16:39.786 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:16:39.781+0000 7feac6688740 1 --2- 172.21.15.12:0/993266685 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5644e813e180 0x5644e814a340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:16:39.787 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:16:39.782+0000 7feac6688740 1 -- 172.21.15.12:0/993266685 >> 172.21.15.12:0/993266685 conn(0x5644e813bfb0 msgr2=0x5644e8141b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:16:39.787 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:16:39.782+0000 7feaa2a00640 0 client.16581 ms_handle_reset on v2:172.21.15.165:6838/501312984 2024-09-11T00:16:39.787 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:16:39.782+0000 7feac6688740 1 -- 172.21.15.12:0/993266685 shutdown_connections 2024-09-11T00:16:39.787 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:16:39.782+0000 7feac6688740 1 -- 172.21.15.12:0/993266685 wait complete. 2024-09-11T00:16:39.787 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:16:39.782+0000 7feac6688740 20 client.16581 trim_cache size 0 max 16384 2024-09-11T00:16:39.805 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:16:39.803+0000 7feac6688740 1 Processor -- start 2024-09-11T00:16:39.805 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:16:39.803+0000 7feac6688740 1 -- start start 2024-09-11T00:16:39.806 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:16:39.803+0000 7feac6688740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5644e813e180 0x5644e8156910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:16:39.806 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:16:39.803+0000 7feac6688740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5644e813eaa0 0x5644e813d780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:16:39.806 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:16:39.803+0000 7feac6688740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5644e814aa30 0x5644e81449a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:16:39.806 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:16:39.803+0000 7feac6688740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7feabc1b1e20 con 0x5644e813e180 2024-09-11T00:16:39.806 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:16:39.803+0000 7feac6688740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7feabc1b3960 con 0x5644e814aa30 2024-09-11T00:16:39.806 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:16:39.803+0000 7feac6688740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7feabc1b24f0 con 0x5644e813eaa0 2024-09-11T00:16:39.807 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:16:39.804+0000 7feabbe00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5644e813eaa0 0x5644e813d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:16:39.807 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:16:39.804+0000 7feac1c00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5644e813e180 0x5644e8156910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:16:39.807 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:16:39.804+0000 7feabbe00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5644e813eaa0 0x5644e813d780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.12:40674/0 (socket says 172.21.15.12:40674) 2024-09-11T00:16:39.807 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:16:39.804+0000 7feabbe00640 1 -- 172.21.15.12:0/2539810856 learned_addr learned my addr 172.21.15.12:0/2539810856 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:16:39.807 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:16:39.804+0000 7feac1c00640 1 --2- 172.21.15.12:0/2539810856 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5644e814aa30 0x5644e81449a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:16:39.807 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:16:39.804+0000 7feabbe00640 1 -- 172.21.15.12:0/2539810856 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5644e814aa30 msgr2=0x5644e81449a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:16:39.808 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:16:39.804+0000 7feabbe00640 1 --2- 172.21.15.12:0/2539810856 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5644e814aa30 0x5644e81449a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:16:39.808 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:16:39.804+0000 7feabbe00640 1 -- 172.21.15.12:0/2539810856 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5644e813e180 msgr2=0x5644e8156910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:16:39.808 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:16:39.804+0000 7feabbe00640 1 --2- 172.21.15.12:0/2539810856 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5644e813e180 0x5644e8156910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:16:39.808 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:16:39.804+0000 7feabbe00640 1 -- 172.21.15.12:0/2539810856 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5644e81466f0 con 0x5644e813eaa0 2024-09-11T00:16:39.808 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:16:39.804+0000 7feac1c00640 1 --2- 172.21.15.12:0/2539810856 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5644e813e180 0x5644e8156910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:16:39.808 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:16:39.804+0000 7feabbe00640 1 --2- 172.21.15.12:0/2539810856 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5644e813eaa0 0x5644e813d780 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7feaa8006010 tx=0x7feaa802a3f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8715968798472fc6 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:16:39.808 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:16:39.805+0000 7feaad600640 1 -- 172.21.15.12:0/2539810856 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feaa8057070 con 0x5644e813eaa0 2024-09-11T00:16:39.808 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:16:39.805+0000 7feaad600640 1 -- 172.21.15.12:0/2539810856 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feaa8040070 con 0x5644e813eaa0 2024-09-11T00:16:39.808 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:16:39.805+0000 7feaad600640 1 -- 172.21.15.12:0/2539810856 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feaa8035070 con 0x5644e813eaa0 2024-09-11T00:16:39.809 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:16:39.806+0000 7feac6688740 1 -- 172.21.15.12:0/2539810856 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5644e813eaa0 msgr2=0x5644e813d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:16:39.810 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:16:39.806+0000 7feac6688740 1 --2- 172.21.15.12:0/2539810856 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5644e813eaa0 0x5644e813d780 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7feaa8006010 tx=0x7feaa802a3f0 comp rx=0 tx=0).stop 2024-09-11T00:16:39.810 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:16:39.806+0000 7feac6688740 1 -- 172.21.15.12:0/2539810856 shutdown_connections 2024-09-11T00:16:39.810 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:16:39.806+0000 7feac6688740 1 --2- 172.21.15.12:0/2539810856 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5644e814aa30 0x5644e81449a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:16:39.810 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:16:39.806+0000 7feac6688740 1 --2- 172.21.15.12:0/2539810856 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5644e813eaa0 0x5644e813d780 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:16:39.810 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:16:39.806+0000 7feac6688740 1 --2- 172.21.15.12:0/2539810856 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5644e813e180 0x5644e8156910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:16:39.810 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:16:39.806+0000 7feac6688740 1 -- 172.21.15.12:0/2539810856 >> 172.21.15.12:0/2539810856 conn(0x5644e813bfb0 msgr2=0x5644e813d070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:16:39.810 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:16:39.806+0000 7feac6688740 1 -- 172.21.15.12:0/2539810856 shutdown_connections 2024-09-11T00:16:39.810 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:16:39.806+0000 7feac6688740 1 -- 172.21.15.12:0/2539810856 wait complete. 2024-09-11T00:16:39.810 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:16:39.806+0000 7feac6688740 1 -- start start 2024-09-11T00:16:39.810 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:16:39.807+0000 7feac6688740 1 Processor -- start 2024-09-11T00:16:39.810 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:16:39.807+0000 7feac6688740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5644e813e180 0x5644e8152e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:16:39.810 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:16:39.807+0000 7feac6688740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5644e813eaa0 0x5644e81519b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:16:39.811 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:16:39.807+0000 7feac6688740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5644e814aa30 0x5644e8151ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:16:39.811 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:16:39.807+0000 7feac6688740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7feabc1b3290 con 0x5644e814aa30 2024-09-11T00:16:39.811 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:16:39.807+0000 7feac6688740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7feabc1b2bc0 con 0x5644e813eaa0 2024-09-11T00:16:39.811 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:16:39.807+0000 7feac6688740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7feabc1afe00 con 0x5644e813e180 2024-09-11T00:16:39.811 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:16:39.807+0000 7feac6688740 10 client.0 _ll_register_callbacks cb 0x7feab40093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:16:39.811 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:16:39.807+0000 7feac1c00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5644e813e180 0x5644e8152e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:16:39.811 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:16:39.807+0000 7feac1c00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5644e814aa30 0x5644e8151ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:16:39.811 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:16:39.807+0000 7feabbe00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5644e813eaa0 0x5644e81519b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:16:39.811 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:16:39.807+0000 7feac1c00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5644e814aa30 0x5644e8151ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:39934/0 (socket says 172.21.15.12:39934) 2024-09-11T00:16:39.811 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:16:39.807+0000 7feac1c00640 1 -- 172.21.15.12:0/1826788310 learned_addr learned my addr 172.21.15.12:0/1826788310 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:16:39.811 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:16:39.807+0000 7feabbe00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5644e813eaa0 0x5644e81519b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.12:43378/0 (socket says 172.21.15.12:43378) 2024-09-11T00:16:39.813 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:16:39.808+0000 7feabbe00640 1 -- 172.21.15.12:0/1826788310 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5644e813e180 msgr2=0x5644e8152e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:16:39.813 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:16:39.808+0000 7feabbe00640 1 --2- 172.21.15.12:0/1826788310 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5644e813e180 0x5644e8152e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:16:39.813 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:16:39.808+0000 7feabbe00640 1 -- 172.21.15.12:0/1826788310 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5644e814aa30 msgr2=0x5644e8151ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:16:39.813 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:16:39.808+0000 7feabbe00640 1 --2- 172.21.15.12:0/1826788310 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5644e814aa30 0x5644e8151ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:16:39.813 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:16:39.808+0000 7feabbe00640 1 -- 172.21.15.12:0/1826788310 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5644e81524b0 con 0x5644e813eaa0 2024-09-11T00:16:39.813 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:16:39.808+0000 7feac1c00640 1 --2- 172.21.15.12:0/1826788310 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5644e814aa30 0x5644e8151ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:16:39.813 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:16:39.808+0000 7feabbe00640 1 --2- 172.21.15.12:0/1826788310 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5644e813eaa0 0x5644e81519b0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7feaa8003ef0 tx=0x7feaa8031b10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3e5ef6d49fc4228 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:16:39.813 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:16:39.808+0000 7feaa2a00640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-09-11T00:16:39.813 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:16:39.808+0000 7feaa2a00640 1 -- 172.21.15.12:0/1826788310 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feaa8035070 con 0x5644e813eaa0 2024-09-11T00:16:39.814 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:16:39.808+0000 7feac6688740 1 -- 172.21.15.12:0/1826788310 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feaa8040070 con 0x5644e813eaa0 2024-09-11T00:16:39.814 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:16:39.808+0000 7feaa2a00640 1 -- 172.21.15.12:0/1826788310 <== mon.2 v2:172.21.15.165:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9208+0+0 (secure 0 0 0) 0x7feaa8031e00 con 0x5644e813eaa0 2024-09-11T00:16:39.814 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:16:39.809+0000 7feaa2a00640 1 -- 172.21.15.12:0/1826788310 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7feaa8057070 con 0x5644e813eaa0 2024-09-11T00:16:39.814 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:16:39.809+0000 7feac6688740 10 client.26594 fetch_fsmap learned FSMap version 41 2024-09-11T00:16:39.814 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:16:39.809+0000 7feac6688740 1 -- 172.21.15.12:0/1826788310 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x5644e81c8120 con 0x5644e813eaa0 2024-09-11T00:16:39.814 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:16:39.809+0000 7feaa2a00640 1 -- 172.21.15.12:0/1826788310 <== mon.2 v2:172.21.15.165:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feaa8040070 con 0x5644e813eaa0 2024-09-11T00:16:39.814 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:16:39.809+0000 7feaa2a00640 1 -- 172.21.15.12:0/1826788310 <== mon.2 v2:172.21.15.165:3300/0 5 ==== fsmap(e 41) ==== 8370+0+0 (secure 0 0 0) 0x7feaa80046c0 con 0x5644e813eaa0 2024-09-11T00:16:39.814 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:16:39.809+0000 7feac6688740 10 client.26594 fetch_fsmap finished waiting for FSMap version 41 2024-09-11T00:16:39.814 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:16:39.809+0000 7feac6688740 10 client.26594 resolve_mds: validated gid 24674 aka daemon mds.b 2024-09-11T00:16:39.814 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:16:39.809+0000 7feac6688740 20 client.26594 populate_metadata read hostname 'smithi012' 2024-09-11T00:16:39.814 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:16:39.809+0000 7feac6688740 1 --2- 172.21.15.12:0/1826788310 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x5644e81c87e0 0x5644e81cabd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:16:39.814 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:16:39.809+0000 7feac6688740 4 client.26594 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 153.4, "await_for": 45.0}] 2024-09-11T00:16:39.814 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:16:39.809+0000 7feac6688740 1 -- 172.21.15.12:0/1826788310 --> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 153.4, "await_for": 45.0}) -- 0x5644e80b16a0 con 0x5644e81c87e0 2024-09-11T00:16:39.815 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:16:39.810+0000 7feac1c00640 1 --2- 172.21.15.12:0/1826788310 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x5644e81c87e0 0x5644e81cabd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:16:39.815 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:16:39.810+0000 7feac1c00640 1 --2- 172.21.15.12:0/1826788310 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x5644e81c87e0 0x5644e81cabd0 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=efafa29028f8759e server_cookie=1675d3da4978655a in_seq=0 out_seq=0 2024-09-11T00:16:39.815 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:16:39.810+0000 7feaa2a00640 10 client.26594 ms_handle_connect on v2:172.21.15.165:6838/501312984 2024-09-11T00:16:39.885 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:16:39 smithi012 ceph-mon[30333]: pgmap v686: 129 pgs: 129 active+clean; 98 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 1.5 MiB/s wr, 250 op/s 2024-09-11T00:16:40.023 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:16:40.020+0000 7feaa2a00640 1 -- 172.21.15.12:0/1826788310 <== mds.0 v2:172.21.15.165:6838/501312984 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x5644e80b16a0 con 0x5644e81c87e0 2024-09-11T00:16:40.024 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:16:40.020+0000 7feaa2a00640 10 client.26594 handle_command_reply: tid=0 2024-09-11T00:16:40.024 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:16:40.021+0000 7feac6688740 1 client.26594 shutdown 2024-09-11T00:16:40.024 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:16:40.021+0000 7feac6688740 1 -- 172.21.15.12:0/1826788310 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5644e813eaa0 msgr2=0x5644e81519b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:16:40.024 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:16:40.021+0000 7feac6688740 1 --2- 172.21.15.12:0/1826788310 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5644e813eaa0 0x5644e81519b0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7feaa8003ef0 tx=0x7feaa8031b10 comp rx=0 tx=0).stop 2024-09-11T00:16:40.024 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:16:40.021+0000 7feac6688740 1 -- 172.21.15.12:0/1826788310 shutdown_connections 2024-09-11T00:16:40.024 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:16:40.021+0000 7feac6688740 1 --2- 172.21.15.12:0/1826788310 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x5644e81c87e0 0x5644e81cabd0 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:16:40.024 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:16:40.021+0000 7feac6688740 1 --2- 172.21.15.12:0/1826788310 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5644e814aa30 0x5644e8151ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:16:40.024 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:16:40.021+0000 7feac6688740 1 --2- 172.21.15.12:0/1826788310 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5644e813eaa0 0x5644e81519b0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:16:40.024 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:16:40.021+0000 7feac6688740 1 --2- 172.21.15.12:0/1826788310 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5644e813e180 0x5644e8152e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:16:40.024 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:16:40.021+0000 7feaa2a00640 0 client.26594 ms_handle_reset on v2:172.21.15.165:6838/501312984 2024-09-11T00:16:40.024 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:16:40.021+0000 7feac6688740 1 -- 172.21.15.12:0/1826788310 >> 172.21.15.12:0/1826788310 conn(0x5644e813bfb0 msgr2=0x5644e813a890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:16:40.024 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:16:40.021+0000 7feac6688740 1 -- 172.21.15.12:0/1826788310 shutdown_connections 2024-09-11T00:16:40.025 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:16:40.021+0000 7feac6688740 1 -- 172.21.15.12:0/1826788310 wait complete. 2024-09-11T00:16:40.025 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:16:40.021+0000 7feac6688740 20 client.26594 trim_cache size 0 max 16384 2024-09-11T00:16:40.025 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-09-11T00:16:40.025 INFO:teuthology.orchestra.run.smithi012.stdout: "epoch": 41, 2024-09-11T00:16:40.025 INFO:teuthology.orchestra.run.smithi012.stdout: "leader": 24674, 2024-09-11T00:16:40.025 INFO:teuthology.orchestra.run.smithi012.stdout: "set_version": 30, 2024-09-11T00:16:40.025 INFO:teuthology.orchestra.run.smithi012.stdout: "sets": { 2024-09-11T00:16:40.025 INFO:teuthology.orchestra.run.smithi012.stdout: "bfba9d53": { 2024-09-11T00:16:40.026 INFO:teuthology.orchestra.run.smithi012.stdout: "version": 30, 2024-09-11T00:16:40.026 INFO:teuthology.orchestra.run.smithi012.stdout: "age_ref": 0.0, 2024-09-11T00:16:40.026 INFO:teuthology.orchestra.run.smithi012.stdout: "state": { 2024-09-11T00:16:40.026 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "QUIESCED", 2024-09-11T00:16:40.026 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 0.0 2024-09-11T00:16:40.026 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:16:40.026 INFO:teuthology.orchestra.run.smithi012.stdout: "timeout": 90.0, 2024-09-11T00:16:40.026 INFO:teuthology.orchestra.run.smithi012.stdout: "expiration": 153.4, 2024-09-11T00:16:40.026 INFO:teuthology.orchestra.run.smithi012.stdout: "members": { 2024-09-11T00:16:40.026 INFO:teuthology.orchestra.run.smithi012.stdout: "file:/": { 2024-09-11T00:16:40.026 INFO:teuthology.orchestra.run.smithi012.stdout: "excluded": false, 2024-09-11T00:16:40.026 INFO:teuthology.orchestra.run.smithi012.stdout: "state": { 2024-09-11T00:16:40.026 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "QUIESCED", 2024-09-11T00:16:40.027 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 0.0 2024-09-11T00:16:40.027 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:16:40.027 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:16:40.027 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:16:40.027 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:16:40.027 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:16:40.027 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-09-11T00:16:40.028 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:16:40.022+0000 7feac3c00640 1 -- 172.21.15.12:0/2709363618 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7feaa40b5360 msgr2=0x7feaa40b7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:16:40.028 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:16:40.022+0000 7feac3c00640 1 --2- 172.21.15.12:0/2709363618 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7feaa40b5360 0x7feaa40b7820 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7feab00ac340 tx=0x7feab00e9cf0 comp rx=0 tx=0).stop 2024-09-11T00:16:40.028 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:16:40.022+0000 7feac3c00640 1 -- 172.21.15.12:0/2709363618 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7feabc104fb0 msgr2=0x7feabc19b4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:16:40.028 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:16:40.022+0000 7feac3c00640 1 --2- 172.21.15.12:0/2709363618 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7feabc104fb0 0x7feabc19b4b0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7feab000ea60 tx=0x7feab000ef20 comp rx=0 tx=0).stop 2024-09-11T00:16:40.028 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:16:40.022+0000 7feac3c00640 1 -- 172.21.15.12:0/2709363618 shutdown_connections 2024-09-11T00:16:40.028 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:16:40.022+0000 7feac3c00640 1 --2- 172.21.15.12:0/2709363618 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7feaa40b5360 0x7feaa40b7820 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:16:40.028 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:16:40.022+0000 7feac3c00640 1 --2- 172.21.15.12:0/2709363618 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7feabc10b340 0x7feabc1a0d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:16:40.028 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:16:40.022+0000 7feac3c00640 1 --2- 172.21.15.12:0/2709363618 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7feabc10a9c0 0x7feabc19b9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:16:40.028 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:16:40.022+0000 7feac3c00640 1 --2- 172.21.15.12:0/2709363618 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7feabc104fb0 0x7feabc19b4b0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:16:40.028 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:16:40.022+0000 7feac3c00640 1 -- 172.21.15.12:0/2709363618 >> 172.21.15.12:0/2709363618 conn(0x7feabc100a20 msgr2=0x7feabc102e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:16:40.029 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:16:40.022+0000 7feac3c00640 1 -- 172.21.15.12:0/2709363618 shutdown_connections 2024-09-11T00:16:40.029 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:16:40.022+0000 7feac3c00640 1 -- 172.21.15.12:0/2709363618 wait complete. 2024-09-11T00:16:40.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:16:39 smithi178 ceph-mon[35919]: pgmap v686: 129 pgs: 129 active+clean; 98 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 1.5 MiB/s wr, 250 op/s 2024-09-11T00:16:40.283 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:16:39 smithi165 ceph-mon[35356]: pgmap v686: 129 pgs: 129 active+clean; 98 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 1.5 MiB/s wr, 250 op/s 2024-09-11T00:16:40.604 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:40.601+0000 7ffa0d600640 1 -- 172.21.15.12:0/1515352579 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ffa08103900 msgr2=0x7ffa08103d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:16:40.604 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:40.601+0000 7ffa0d600640 1 --2- 172.21.15.12:0/1515352579 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ffa08103900 0x7ffa08103d80 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7ff9f8009930 tx=0x7ff9f802a460 comp rx=0 tx=0).stop 2024-09-11T00:16:40.605 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:40.602+0000 7ffa0d600640 1 -- 172.21.15.12:0/1515352579 shutdown_connections 2024-09-11T00:16:40.605 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:40.602+0000 7ffa0d600640 1 --2- 172.21.15.12:0/1515352579 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ffa081042c0 0x7ffa0810ac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:16:40.605 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:40.602+0000 7ffa0d600640 1 --2- 172.21.15.12:0/1515352579 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ffa08103900 0x7ffa08103d80 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:16:40.605 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:40.602+0000 7ffa0d600640 1 --2- 172.21.15.12:0/1515352579 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ffa081027b0 0x7ffa08102bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:16:40.605 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:40.602+0000 7ffa0d600640 1 -- 172.21.15.12:0/1515352579 >> 172.21.15.12:0/1515352579 conn(0x7ffa080fe050 msgr2=0x7ffa08100490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:16:40.605 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:40.602+0000 7ffa0d600640 1 -- 172.21.15.12:0/1515352579 shutdown_connections 2024-09-11T00:16:40.605 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:40.602+0000 7ffa0d600640 1 -- 172.21.15.12:0/1515352579 wait complete. 2024-09-11T00:16:40.606 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:40.604+0000 7ffa0d600640 1 Processor -- start 2024-09-11T00:16:40.606 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:40.604+0000 7ffa0d600640 1 -- start start 2024-09-11T00:16:40.607 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:40.604+0000 7ffa0d600640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ffa081027b0 0x7ffa08113140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:16:40.607 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:40.604+0000 7ffa0d600640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ffa08103900 0x7ffa08113680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:16:40.607 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:40.604+0000 7ffa0d600640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ffa081042c0 0x7ffa08111a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:16:40.607 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:40.604+0000 7ffa0d600640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7ffa0810c260 con 0x7ffa081042c0 2024-09-11T00:16:40.607 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:40.604+0000 7ffa0d600640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7ffa0810c0e0 con 0x7ffa08103900 2024-09-11T00:16:40.607 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:40.604+0000 7ffa0d600640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7ffa0810c3e0 con 0x7ffa081027b0 2024-09-11T00:16:40.607 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:40.604+0000 7ffa07e00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ffa081027b0 0x7ffa08113140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:16:40.608 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:40.604+0000 7ffa07e00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ffa081027b0 0x7ffa08113140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.12:40702/0 (socket says 172.21.15.12:40702) 2024-09-11T00:16:40.608 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:40.604+0000 7ffa07e00640 1 -- 172.21.15.12:0/3780633356 learned_addr learned my addr 172.21.15.12:0/3780633356 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:16:40.608 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:40.605+0000 7ffa0cc00640 1 --2- 172.21.15.12:0/3780633356 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ffa081042c0 0x7ffa08111a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:16:40.608 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:40.605+0000 7ffa07400640 1 --2- 172.21.15.12:0/3780633356 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ffa08103900 0x7ffa08113680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:16:40.609 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:40.605+0000 7ffa07e00640 1 -- 172.21.15.12:0/3780633356 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ffa08103900 msgr2=0x7ffa08113680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:16:40.609 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:40.605+0000 7ffa07e00640 1 --2- 172.21.15.12:0/3780633356 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ffa08103900 0x7ffa08113680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:16:40.609 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:40.605+0000 7ffa07e00640 1 -- 172.21.15.12:0/3780633356 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ffa081042c0 msgr2=0x7ffa08111a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:16:40.609 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:40.605+0000 7ffa07e00640 1 --2- 172.21.15.12:0/3780633356 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ffa081042c0 0x7ffa08111a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:16:40.609 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:40.605+0000 7ffa07e00640 1 -- 172.21.15.12:0/3780633356 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa08112170 con 0x7ffa081027b0 2024-09-11T00:16:40.609 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:40.605+0000 7ffa0cc00640 1 --2- 172.21.15.12:0/3780633356 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ffa081042c0 0x7ffa08111a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T00:16:40.609 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:40.606+0000 7ffa07e00640 1 --2- 172.21.15.12:0/3780633356 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ffa081027b0 0x7ffa08113140 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7ff9f000b450 tx=0x7ff9f000b910 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:16:40.610 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:40.606+0000 7ffa04c00640 1 -- 172.21.15.12:0/3780633356 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9f0015020 con 0x7ffa081027b0 2024-09-11T00:16:40.610 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:40.607+0000 7ffa04c00640 1 -- 172.21.15.12:0/3780633356 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff9f0011d40 con 0x7ffa081027b0 2024-09-11T00:16:40.610 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:40.607+0000 7ffa04c00640 1 -- 172.21.15.12:0/3780633356 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9f000f340 con 0x7ffa081027b0 2024-09-11T00:16:40.610 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:40.607+0000 7ffa0d600640 1 -- 172.21.15.12:0/3780633356 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa08112400 con 0x7ffa081027b0 2024-09-11T00:16:40.610 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:40.607+0000 7ffa0d600640 1 -- 172.21.15.12:0/3780633356 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa081aff40 con 0x7ffa081027b0 2024-09-11T00:16:40.614 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:40.611+0000 7ffa11aa4740 1 Processor -- start 2024-09-11T00:16:40.614 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:40.612+0000 7ffa11aa4740 1 -- start start 2024-09-11T00:16:40.614 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:40.612+0000 7ffa11aa4740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55830ab12970 0x55830ab14d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:16:40.614 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:40.612+0000 7ffa11aa4740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55830ab152a0 0x55830ab17690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:16:40.614 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:40.612+0000 7ffa11aa4740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55830ab17ce0 0x55830ab1a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:16:40.614 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:40.612+0000 7ffa11aa4740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55830aa856b0 con 0x55830ab17ce0 2024-09-11T00:16:40.614 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:40.612+0000 7ffa11aa4740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55830aa74ae0 con 0x55830ab12970 2024-09-11T00:16:40.614 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:40.612+0000 7ffa11aa4740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55830aa788f0 con 0x55830ab152a0 2024-09-11T00:16:40.615 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:40.613+0000 7ffa07400640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55830ab17ce0 0x55830ab1a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:16:40.615 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:40.613+0000 7ffa07400640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55830ab17ce0 0x55830ab1a1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:39960/0 (socket says 172.21.15.12:39960) 2024-09-11T00:16:40.615 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:40.613+0000 7ffa07400640 1 -- 172.21.15.12:0/248273352 learned_addr learned my addr 172.21.15.12:0/248273352 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:16:40.615 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:40.613+0000 7ffa07400640 1 -- 172.21.15.12:0/248273352 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55830ab152a0 msgr2=0x55830ab17690 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-11T00:16:40.616 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:40.613+0000 7ffa04c00640 1 -- 172.21.15.12:0/3780633356 <== mon.1 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7ff9f0011890 con 0x7ffa081027b0 2024-09-11T00:16:40.616 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:40.614+0000 7ffa07400640 1 --2- 172.21.15.12:0/248273352 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55830ab152a0 0x55830ab17690 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:16:40.617 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:40.614+0000 7ffa07400640 1 -- 172.21.15.12:0/248273352 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55830ab12970 msgr2=0x55830ab14d60 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-11T00:16:40.617 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:40.614+0000 7ffa07400640 1 --2- 172.21.15.12:0/248273352 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55830ab12970 0x55830ab14d60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:16:40.617 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:40.614+0000 7ffa07400640 1 -- 172.21.15.12:0/248273352 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55830aa41780 con 0x55830ab17ce0 2024-09-11T00:16:40.617 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:40.615+0000 7ffa07400640 1 --2- 172.21.15.12:0/248273352 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55830ab17ce0 0x55830ab1a1a0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7ff9f802aa50 tx=0x7ff9f8004200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dbbfb4149e943ef8 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:16:40.617 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:40.615+0000 7ff9fd200640 1 -- 172.21.15.12:0/248273352 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9f8035030 con 0x55830ab17ce0 2024-09-11T00:16:40.618 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:40.615+0000 7ff9fd200640 1 -- 172.21.15.12:0/248273352 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff9f8040070 con 0x55830ab17ce0 2024-09-11T00:16:40.618 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:40.615+0000 7ff9fd200640 1 -- 172.21.15.12:0/248273352 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9f8031b60 con 0x55830ab17ce0 2024-09-11T00:16:40.618 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:40.616+0000 7ffa04c00640 1 --2- 172.21.15.12:0/3780633356 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7ff9ec0b5420 0x7ff9ec0b78e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:16:40.618 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:40.616+0000 7ffa04c00640 1 -- 172.21.15.12:0/3780633356 <== mon.1 v2:172.21.15.178:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9208+0+0 (secure 0 0 0) 0x7ff9f00e5ba0 con 0x7ffa081027b0 2024-09-11T00:16:40.619 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:40.617+0000 7ffa11aa4740 1 -- 172.21.15.12:0/248273352 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55830ab17ce0 msgr2=0x55830ab1a1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:16:40.619 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:40.617+0000 7ffa11aa4740 1 --2- 172.21.15.12:0/248273352 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55830ab17ce0 0x55830ab1a1a0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7ff9f802aa50 tx=0x7ff9f8004200 comp rx=0 tx=0).stop 2024-09-11T00:16:40.621 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:40.617+0000 7ffa0cc00640 1 --2- 172.21.15.12:0/3780633356 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7ff9ec0b5420 0x7ff9ec0b78e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:16:40.621 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:40.619+0000 7ffa11aa4740 1 -- 172.21.15.12:0/248273352 shutdown_connections 2024-09-11T00:16:40.621 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:40.619+0000 7ffa11aa4740 1 --2- 172.21.15.12:0/248273352 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55830ab17ce0 0x55830ab1a1a0 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:16:40.621 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:40.619+0000 7ffa11aa4740 1 --2- 172.21.15.12:0/248273352 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55830ab152a0 0x55830ab17690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:16:40.621 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:40.619+0000 7ffa11aa4740 1 --2- 172.21.15.12:0/248273352 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55830ab12970 0x55830ab14d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:16:40.621 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:40.619+0000 7ffa11aa4740 1 -- 172.21.15.12:0/248273352 >> 172.21.15.12:0/248273352 conn(0x55830ab0dac0 msgr2=0x55830ab0e850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:16:40.621 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:40.619+0000 7ffa11aa4740 1 -- 172.21.15.12:0/248273352 shutdown_connections 2024-09-11T00:16:40.621 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:40.619+0000 7ffa11aa4740 1 -- 172.21.15.12:0/248273352 wait complete. 2024-09-11T00:16:40.625 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:40.620+0000 7ffa11aa4740 1 -- start start 2024-09-11T00:16:40.625 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:40.621+0000 7ffa11aa4740 1 Processor -- start 2024-09-11T00:16:40.625 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:40.623+0000 7ffa0cc00640 1 --2- 172.21.15.12:0/3780633356 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7ff9ec0b5420 0x7ff9ec0b78e0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7ffa00005fd0 tx=0x7ffa000094a0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:16:40.626 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:40.624+0000 7ffa11aa4740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55830ab12970 0x55830ab1bf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:16:40.626 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:40.624+0000 7ffa11aa4740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55830ab152a0 0x55830ab1c630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:16:40.626 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:40.624+0000 7ffa11aa4740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55830ab17ce0 0x55830ab1d200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:16:40.627 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:40.624+0000 7ffa11aa4740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55830a9acab0 con 0x55830ab17ce0 2024-09-11T00:16:40.627 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:40.624+0000 7ffa11aa4740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55830aa1b570 con 0x55830ab152a0 2024-09-11T00:16:40.627 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:40.624+0000 7ffa11aa4740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55830ab1ac60 con 0x55830ab12970 2024-09-11T00:16:40.627 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:40.624+0000 7ffa11aa4740 10 client.0 _ll_register_callbacks cb 0x55830aa2a100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:16:40.627 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:40.625+0000 7ffa07e00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55830ab12970 0x55830ab1bf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:16:40.629 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:40.625+0000 7ffa07e00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55830ab12970 0x55830ab1bf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.12:40720/0 (socket says 172.21.15.12:40720) 2024-09-11T00:16:40.629 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:40.625+0000 7ffa07e00640 1 -- 172.21.15.12:0/2856052323 learned_addr learned my addr 172.21.15.12:0/2856052323 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:16:40.629 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:40.625+0000 7ffa0cc00640 1 --2- 172.21.15.12:0/2856052323 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55830ab17ce0 0x55830ab1d200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:16:40.629 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:40.625+0000 7ffa07e00640 1 -- 172.21.15.12:0/2856052323 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55830ab152a0 msgr2=0x55830ab1c630 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-11T00:16:40.629 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:40.625+0000 7ffa07e00640 1 --2- 172.21.15.12:0/2856052323 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55830ab152a0 0x55830ab1c630 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:16:40.629 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:40.625+0000 7ffa07e00640 1 -- 172.21.15.12:0/2856052323 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55830ab17ce0 msgr2=0x55830ab1d200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:16:40.629 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:40.625+0000 7ffa07e00640 1 --2- 172.21.15.12:0/2856052323 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55830ab17ce0 0x55830ab1d200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:16:40.630 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:40.625+0000 7ffa07e00640 1 -- 172.21.15.12:0/2856052323 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55830a7d6790 con 0x55830ab12970 2024-09-11T00:16:40.630 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:40.626+0000 7ffa07e00640 1 --2- 172.21.15.12:0/2856052323 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55830ab12970 0x55830ab1bf90 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7ff9f00abec0 tx=0x7ff9f00a7e90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=29df0e26beb878e0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:16:40.630 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:40.626+0000 7ff9f5600640 10 client.0 ms_handle_connect on v2:172.21.15.178:3300/0 2024-09-11T00:16:40.630 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:40.626+0000 7ff9f5600640 1 -- 172.21.15.12:0/2856052323 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9f00ab390 con 0x55830ab12970 2024-09-11T00:16:40.630 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:40.626+0000 7ff9f5600640 1 -- 172.21.15.12:0/2856052323 <== mon.1 v2:172.21.15.178:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9208+0+0 (secure 0 0 0) 0x7ff9f00a7880 con 0x55830ab12970 2024-09-11T00:16:40.630 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:40.626+0000 7ffa11aa4740 1 -- 172.21.15.12:0/2856052323 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff9f8040070 con 0x55830ab12970 2024-09-11T00:16:40.630 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:40.627+0000 7ff9f5600640 1 -- 172.21.15.12:0/2856052323 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7ff9f00aba40 con 0x55830ab12970 2024-09-11T00:16:40.630 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:40.627+0000 7ffa11aa4740 10 client.26584 fetch_fsmap learned FSMap version 41 2024-09-11T00:16:40.630 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:40.627+0000 7ffa11aa4740 1 -- 172.21.15.12:0/2856052323 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x55830a84f620 con 0x55830ab12970 2024-09-11T00:16:40.630 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:40.627+0000 7ff9f5600640 1 -- 172.21.15.12:0/2856052323 <== mon.1 v2:172.21.15.178:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff9f8040070 con 0x55830ab12970 2024-09-11T00:16:40.631 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:40.627+0000 7ff9f5600640 1 -- 172.21.15.12:0/2856052323 <== mon.1 v2:172.21.15.178:3300/0 5 ==== fsmap(e 41) ==== 8370+0+0 (secure 0 0 0) 0x7ff9f00a7a80 con 0x55830ab12970 2024-09-11T00:16:40.631 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:40.627+0000 7ffa11aa4740 10 client.26584 fetch_fsmap finished waiting for FSMap version 41 2024-09-11T00:16:40.631 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:40.627+0000 7ffa11aa4740 10 client.26584 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T00:16:40.631 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:40.627+0000 7ffa11aa4740 20 client.26584 populate_metadata read hostname 'smithi012' 2024-09-11T00:16:40.631 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:40.627+0000 7ffa11aa4740 1 --2- 172.21.15.12:0/2856052323 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55830ab278c0 0x55830ab29cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:16:40.631 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:40.627+0000 7ffa11aa4740 4 client.26584 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T00:16:40.631 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:40.627+0000 7ffa11aa4740 1 -- 172.21.15.12:0/2856052323 --> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55830aa87600 con 0x55830ab278c0 2024-09-11T00:16:40.631 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:40.628+0000 7ffa07400640 1 --2- 172.21.15.12:0/2856052323 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55830ab278c0 0x55830ab29cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:16:40.666 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:40.664+0000 7ffa07400640 1 --2- 172.21.15.12:0/2856052323 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55830ab278c0 0x55830ab29cb0 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=77223decf597eff4 server_cookie=96bd524bf3d91422 in_seq=0 out_seq=0 2024-09-11T00:16:40.666 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:40.664+0000 7ff9f5600640 10 client.26584 ms_handle_connect on v2:172.21.15.165:6838/501312984 2024-09-11T00:16:40.669 INFO:tasks.workunit.client.0.smithi012.stdout: 1 393902 4.04 MB/sec execute 397 sec latency 836.420 ms 2024-09-11T00:16:40.690 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:40.688+0000 7ff9f5600640 1 -- 172.21.15.12:0/2856052323 <== mds.0 v2:172.21.15.165:6838/501312984 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55830aa87600 con 0x55830ab278c0 2024-09-11T00:16:40.691 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:40.688+0000 7ff9f5600640 10 client.26584 handle_command_reply: tid=0 2024-09-11T00:16:40.691 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:40.688+0000 7ffa11aa4740 1 client.26584 shutdown 2024-09-11T00:16:40.691 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:40.688+0000 7ffa11aa4740 1 -- 172.21.15.12:0/2856052323 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55830ab12970 msgr2=0x55830ab1bf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:16:40.691 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:40.688+0000 7ffa11aa4740 1 --2- 172.21.15.12:0/2856052323 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55830ab12970 0x55830ab1bf90 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7ff9f00abec0 tx=0x7ff9f00a7e90 comp rx=0 tx=0).stop 2024-09-11T00:16:40.691 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:40.688+0000 7ffa11aa4740 1 -- 172.21.15.12:0/2856052323 shutdown_connections 2024-09-11T00:16:40.691 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:40.688+0000 7ffa11aa4740 1 --2- 172.21.15.12:0/2856052323 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55830ab278c0 0x55830ab29cb0 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:16:40.691 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:40.688+0000 7ffa11aa4740 1 --2- 172.21.15.12:0/2856052323 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55830ab17ce0 0x55830ab1d200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:16:40.691 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:40.688+0000 7ffa11aa4740 1 --2- 172.21.15.12:0/2856052323 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55830ab152a0 0x55830ab1c630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:16:40.691 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:40.688+0000 7ff9f5600640 0 client.26584 ms_handle_reset on v2:172.21.15.165:6838/501312984 2024-09-11T00:16:40.691 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:40.688+0000 7ffa11aa4740 1 --2- 172.21.15.12:0/2856052323 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55830ab12970 0x55830ab1bf90 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:16:40.691 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:40.688+0000 7ffa11aa4740 1 -- 172.21.15.12:0/2856052323 >> 172.21.15.12:0/2856052323 conn(0x55830ab0dac0 msgr2=0x55830ab0cbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:16:40.692 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:40.688+0000 7ffa11aa4740 1 -- 172.21.15.12:0/2856052323 shutdown_connections 2024-09-11T00:16:40.692 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:40.688+0000 7ffa11aa4740 1 -- 172.21.15.12:0/2856052323 wait complete. 2024-09-11T00:16:40.692 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:40.689+0000 7ffa11aa4740 20 client.26584 trim_cache size 0 max 16384 2024-09-11T00:16:40.707 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:40.704+0000 7ffa11aa4740 1 Processor -- start 2024-09-11T00:16:40.707 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:40.704+0000 7ffa11aa4740 1 -- start start 2024-09-11T00:16:40.707 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:40.704+0000 7ffa11aa4740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55830ab12970 0x55830ab0ea30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:16:40.708 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:40.704+0000 7ffa11aa4740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55830ab152a0 0x55830ab2e590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:16:40.708 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:40.704+0000 7ffa11aa4740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55830ab17ce0 0x55830ab2ead0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:16:40.708 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:40.705+0000 7ffa11aa4740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7ffa081ba4c0 con 0x55830ab12970 2024-09-11T00:16:40.708 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:40.705+0000 7ffa11aa4740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7ffa081bc000 con 0x55830ab17ce0 2024-09-11T00:16:40.708 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:40.705+0000 7ffa11aa4740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7ffa081bab90 con 0x55830ab152a0 2024-09-11T00:16:40.708 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:40.705+0000 7ffa07e00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55830ab12970 0x55830ab0ea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:16:40.708 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:40.705+0000 7ffa07e00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55830ab12970 0x55830ab0ea30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:39984/0 (socket says 172.21.15.12:39984) 2024-09-11T00:16:40.708 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:40.705+0000 7ffa07e00640 1 -- 172.21.15.12:0/365576644 learned_addr learned my addr 172.21.15.12:0/365576644 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:16:40.710 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:40.705+0000 7ffa0cc00640 1 --2- 172.21.15.12:0/365576644 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55830ab17ce0 0x55830ab2ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:16:40.710 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:40.705+0000 7ffa07e00640 1 -- 172.21.15.12:0/365576644 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55830ab152a0 msgr2=0x55830ab2e590 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-11T00:16:40.710 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:40.705+0000 7ffa07e00640 1 --2- 172.21.15.12:0/365576644 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55830ab152a0 0x55830ab2e590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:16:40.710 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:40.705+0000 7ffa07e00640 1 -- 172.21.15.12:0/365576644 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55830ab17ce0 msgr2=0x55830ab2ead0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:16:40.710 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:40.705+0000 7ffa07e00640 1 --2- 172.21.15.12:0/365576644 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55830ab17ce0 0x55830ab2ead0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:16:40.710 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:40.705+0000 7ffa07e00640 1 -- 172.21.15.12:0/365576644 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55830ab1ba90 con 0x55830ab12970 2024-09-11T00:16:40.710 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:40.706+0000 7ffa07e00640 1 --2- 172.21.15.12:0/365576644 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55830ab12970 0x55830ab0ea30 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7ff9f00a84b0 tx=0x7ff9f00a7f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1868baf9a5f454f5 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:16:40.710 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:40.706+0000 7ff9f7e00640 1 -- 172.21.15.12:0/365576644 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9f00ac630 con 0x55830ab12970 2024-09-11T00:16:40.710 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:40.706+0000 7ff9f7e00640 1 -- 172.21.15.12:0/365576644 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff9f00abf30 con 0x55830ab12970 2024-09-11T00:16:40.711 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:40.706+0000 7ff9f7e00640 1 -- 172.21.15.12:0/365576644 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9f00ab3f0 con 0x55830ab12970 2024-09-11T00:16:40.711 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:40.707+0000 7ffa11aa4740 1 -- 172.21.15.12:0/365576644 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55830ab12970 msgr2=0x55830ab0ea30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:16:40.711 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:40.707+0000 7ffa11aa4740 1 --2- 172.21.15.12:0/365576644 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55830ab12970 0x55830ab0ea30 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7ff9f00a84b0 tx=0x7ff9f00a7f00 comp rx=0 tx=0).stop 2024-09-11T00:16:40.711 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:40.707+0000 7ffa11aa4740 1 -- 172.21.15.12:0/365576644 shutdown_connections 2024-09-11T00:16:40.711 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:40.707+0000 7ffa11aa4740 1 --2- 172.21.15.12:0/365576644 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55830ab17ce0 0x55830ab2ead0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:16:40.711 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:40.707+0000 7ffa11aa4740 1 --2- 172.21.15.12:0/365576644 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55830ab152a0 0x55830ab2e590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:16:40.711 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:40.707+0000 7ffa11aa4740 1 --2- 172.21.15.12:0/365576644 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55830ab12970 0x55830ab0ea30 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:16:40.711 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:40.707+0000 7ffa11aa4740 1 -- 172.21.15.12:0/365576644 >> 172.21.15.12:0/365576644 conn(0x55830ab0dac0 msgr2=0x55830ab0e320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:16:40.711 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:40.707+0000 7ffa11aa4740 1 -- 172.21.15.12:0/365576644 shutdown_connections 2024-09-11T00:16:40.711 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:40.707+0000 7ffa11aa4740 1 -- 172.21.15.12:0/365576644 wait complete. 2024-09-11T00:16:40.714 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:40.707+0000 7ffa11aa4740 1 -- start start 2024-09-11T00:16:40.714 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:40.708+0000 7ffa11aa4740 1 Processor -- start 2024-09-11T00:16:40.714 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:40.708+0000 7ffa11aa4740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55830ab12970 0x55830ab233a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:16:40.714 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:40.708+0000 7ffa11aa4740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55830ab152a0 0x55830ab23960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:16:40.714 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:40.708+0000 7ffa11aa4740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55830ab17ce0 0x55830ab21c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:16:40.714 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:40.708+0000 7ffa11aa4740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7ffa081bb930 con 0x55830ab152a0 2024-09-11T00:16:40.714 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:40.708+0000 7ffa11aa4740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7ffa081bb260 con 0x55830ab17ce0 2024-09-11T00:16:40.714 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:40.708+0000 7ffa11aa4740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7ffa081b84a0 con 0x55830ab12970 2024-09-11T00:16:40.714 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:40.708+0000 7ffa11aa4740 10 client.0 _ll_register_callbacks cb 0x7ff9f00a8580 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:16:40.714 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:40.709+0000 7ffa07400640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55830ab152a0 0x55830ab23960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:16:40.714 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:40.709+0000 7ffa07e00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55830ab12970 0x55830ab233a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:16:40.714 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:40.709+0000 7ffa07e00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55830ab12970 0x55830ab233a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.12:40742/0 (socket says 172.21.15.12:40742) 2024-09-11T00:16:40.715 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:40.709+0000 7ffa07e00640 1 -- 172.21.15.12:0/1213200307 learned_addr learned my addr 172.21.15.12:0/1213200307 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:16:40.715 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:40.709+0000 7ffa07400640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55830ab152a0 0x55830ab23960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:39990/0 (socket says 172.21.15.12:39990) 2024-09-11T00:16:40.715 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:40.709+0000 7ffa0cc00640 1 --2- 172.21.15.12:0/1213200307 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55830ab17ce0 0x55830ab21c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:16:40.715 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:40.709+0000 7ffa07400640 1 -- 172.21.15.12:0/1213200307 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55830ab12970 msgr2=0x55830ab233a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:16:40.715 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:40.709+0000 7ffa07400640 1 --2- 172.21.15.12:0/1213200307 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55830ab12970 0x55830ab233a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:16:40.715 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:40.709+0000 7ffa07400640 1 -- 172.21.15.12:0/1213200307 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55830ab17ce0 msgr2=0x55830ab21c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:16:40.715 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:40.709+0000 7ffa07400640 1 --2- 172.21.15.12:0/1213200307 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55830ab17ce0 0x55830ab21c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:16:40.715 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:40.709+0000 7ffa07400640 1 -- 172.21.15.12:0/1213200307 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55830ab2a310 con 0x55830ab152a0 2024-09-11T00:16:40.715 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:40.709+0000 7ffa07e00640 1 --2- 172.21.15.12:0/1213200307 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55830ab12970 0x55830ab233a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:16:40.715 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:40.709+0000 7ffa0cc00640 1 --2- 172.21.15.12:0/1213200307 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55830ab17ce0 0x55830ab21c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T00:16:40.715 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:40.709+0000 7ffa07400640 1 --2- 172.21.15.12:0/1213200307 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55830ab152a0 0x55830ab23960 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x55830ab2a510 tx=0x7ff9f8032860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95448d493a1fd867 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:16:40.715 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:40.709+0000 7ff9f5600640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-09-11T00:16:40.715 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:40.710+0000 7ff9f5600640 1 -- 172.21.15.12:0/1213200307 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9f8052070 con 0x55830ab152a0 2024-09-11T00:16:40.715 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:40.710+0000 7ffa11aa4740 1 -- 172.21.15.12:0/1213200307 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff9f00abf30 con 0x55830ab152a0 2024-09-11T00:16:40.716 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:40.710+0000 7ff9f5600640 1 -- 172.21.15.12:0/1213200307 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9208+0+0 (secure 0 0 0) 0x7ff9f8032d60 con 0x55830ab152a0 2024-09-11T00:16:40.716 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:40.710+0000 7ff9f5600640 1 -- 172.21.15.12:0/1213200307 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7ff9f8060050 con 0x55830ab152a0 2024-09-11T00:16:40.716 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:40.710+0000 7ffa11aa4740 10 client.16605 fetch_fsmap learned FSMap version 41 2024-09-11T00:16:40.716 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:40.710+0000 7ffa11aa4740 1 -- 172.21.15.12:0/1213200307 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x55830ab950c0 con 0x55830ab152a0 2024-09-11T00:16:40.716 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:40.710+0000 7ff9f5600640 1 -- 172.21.15.12:0/1213200307 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff9f00abf30 con 0x55830ab152a0 2024-09-11T00:16:40.716 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:40.710+0000 7ff9f5600640 1 -- 172.21.15.12:0/1213200307 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 41) ==== 8370+0+0 (secure 0 0 0) 0x7ff9f80505d0 con 0x55830ab152a0 2024-09-11T00:16:40.716 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:40.711+0000 7ffa11aa4740 10 client.16605 fetch_fsmap finished waiting for FSMap version 41 2024-09-11T00:16:40.716 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:40.711+0000 7ffa11aa4740 10 client.16605 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T00:16:40.716 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:40.711+0000 7ffa11aa4740 20 client.16605 populate_metadata read hostname 'smithi012' 2024-09-11T00:16:40.716 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:40.711+0000 7ffa11aa4740 1 --2- 172.21.15.12:0/1213200307 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55830ab95680 0x55830ab97a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:16:40.716 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:40.711+0000 7ffa11aa4740 4 client.16605 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T00:16:40.717 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:40.711+0000 7ffa11aa4740 1 -- 172.21.15.12:0/1213200307 --> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55830ab0d890 con 0x55830ab95680 2024-09-11T00:16:40.717 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:40.712+0000 7ffa07e00640 1 --2- 172.21.15.12:0/1213200307 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55830ab95680 0x55830ab97a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:16:40.737 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:40.735+0000 7ffa07e00640 1 --2- 172.21.15.12:0/1213200307 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55830ab95680 0x55830ab97a70 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8e429c37b533ec02 server_cookie=d282fb52e07b876a in_seq=0 out_seq=0 2024-09-11T00:16:40.737 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:40.735+0000 7ff9f5600640 10 client.16605 ms_handle_connect on v2:172.21.15.165:6838/501312984 2024-09-11T00:16:40.816 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:40.814+0000 7ff9f5600640 1 -- 172.21.15.12:0/1213200307 <== mds.0 v2:172.21.15.165:6838/501312984 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55830ab0d890 con 0x55830ab95680 2024-09-11T00:16:40.816 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:40.814+0000 7ff9f5600640 10 client.16605 handle_command_reply: tid=0 2024-09-11T00:16:40.816 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:40.814+0000 7ffa11aa4740 1 client.16605 shutdown 2024-09-11T00:16:40.816 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:40.814+0000 7ffa11aa4740 1 -- 172.21.15.12:0/1213200307 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55830ab152a0 msgr2=0x55830ab23960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:16:40.816 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:40.814+0000 7ffa11aa4740 1 --2- 172.21.15.12:0/1213200307 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55830ab152a0 0x55830ab23960 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x55830ab2a510 tx=0x7ff9f8032860 comp rx=0 tx=0).stop 2024-09-11T00:16:40.817 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:40.814+0000 7ffa11aa4740 1 -- 172.21.15.12:0/1213200307 shutdown_connections 2024-09-11T00:16:40.817 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:40.814+0000 7ffa11aa4740 1 --2- 172.21.15.12:0/1213200307 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55830ab95680 0x55830ab97a70 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:16:40.817 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:40.814+0000 7ffa11aa4740 1 --2- 172.21.15.12:0/1213200307 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55830ab17ce0 0x55830ab21c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:16:40.818 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:40.814+0000 7ffa11aa4740 1 --2- 172.21.15.12:0/1213200307 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55830ab152a0 0x55830ab23960 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:16:40.818 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:40.814+0000 7ffa11aa4740 1 --2- 172.21.15.12:0/1213200307 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55830ab12970 0x55830ab233a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:16:40.818 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:40.814+0000 7ffa11aa4740 1 -- 172.21.15.12:0/1213200307 >> 172.21.15.12:0/1213200307 conn(0x55830ab0dac0 msgr2=0x55830ab0c3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:16:40.818 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:40.814+0000 7ff9f5600640 0 client.16605 ms_handle_reset on v2:172.21.15.165:6838/501312984 2024-09-11T00:16:40.818 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:40.814+0000 7ffa11aa4740 1 -- 172.21.15.12:0/1213200307 shutdown_connections 2024-09-11T00:16:40.818 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:40.815+0000 7ffa11aa4740 1 -- 172.21.15.12:0/1213200307 wait complete. 2024-09-11T00:16:40.818 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:40.815+0000 7ffa11aa4740 20 client.16605 trim_cache size 0 max 16384 2024-09-11T00:16:40.818 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:{ 2024-09-11T00:16:40.819 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "message": "", 2024-09-11T00:16:40.819 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "return_code": 0 2024-09-11T00:16:40.819 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:} 2024-09-11T00:16:40.820 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:40.815+0000 7ffa0d600640 1 -- 172.21.15.12:0/3780633356 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7ff9ec0b5420 msgr2=0x7ff9ec0b78e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:16:40.820 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:40.815+0000 7ffa0d600640 1 --2- 172.21.15.12:0/3780633356 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7ff9ec0b5420 0x7ff9ec0b78e0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7ffa00005fd0 tx=0x7ffa000094a0 comp rx=0 tx=0).stop 2024-09-11T00:16:40.820 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:40.815+0000 7ffa0d600640 1 -- 172.21.15.12:0/3780633356 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ffa081027b0 msgr2=0x7ffa08113140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:16:40.820 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:40.815+0000 7ffa0d600640 1 --2- 172.21.15.12:0/3780633356 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ffa081027b0 0x7ffa08113140 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7ff9f000b450 tx=0x7ff9f000b910 comp rx=0 tx=0).stop 2024-09-11T00:16:40.821 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:40.816+0000 7ffa0d600640 1 -- 172.21.15.12:0/3780633356 shutdown_connections 2024-09-11T00:16:40.821 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:40.816+0000 7ffa0d600640 1 --2- 172.21.15.12:0/3780633356 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7ff9ec0b5420 0x7ff9ec0b78e0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:16:40.821 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:40.816+0000 7ffa0d600640 1 --2- 172.21.15.12:0/3780633356 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ffa081042c0 0x7ffa08111a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:16:40.821 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:40.816+0000 7ffa0d600640 1 --2- 172.21.15.12:0/3780633356 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ffa08103900 0x7ffa08113680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:16:40.821 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:40.816+0000 7ffa0d600640 1 --2- 172.21.15.12:0/3780633356 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ffa081027b0 0x7ffa08113140 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:16:40.821 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:40.816+0000 7ffa0d600640 1 -- 172.21.15.12:0/3780633356 >> 172.21.15.12:0/3780633356 conn(0x7ffa080fe050 msgr2=0x7ffa08106cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:16:40.821 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:40.816+0000 7ffa0d600640 1 -- 172.21.15.12:0/3780633356 shutdown_connections 2024-09-11T00:16:40.821 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:40.816+0000 7ffa0d600640 1 -- 172.21.15.12:0/3780633356 wait complete. 2024-09-11T00:16:41.060 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'bfba9d53', quiesce took 4.6 seconds. Will release after: 28.799999999999997 2024-09-11T00:16:41.670 INFO:tasks.workunit.client.0.smithi012.stdout: 1 393902 4.03 MB/sec execute 398 sec latency 1836.532 ms 2024-09-11T00:16:42.063 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:16:41 smithi012 ceph-mon[30333]: pgmap v687: 129 pgs: 129 active+clean; 99 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.1 MiB/s wr, 331 op/s 2024-09-11T00:16:42.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:16:41 smithi178 ceph-mon[35919]: pgmap v687: 129 pgs: 129 active+clean; 99 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.1 MiB/s wr, 331 op/s 2024-09-11T00:16:42.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:16:41 smithi165 ceph-mon[35356]: pgmap v687: 129 pgs: 129 active+clean; 99 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.1 MiB/s wr, 331 op/s 2024-09-11T00:16:42.670 INFO:tasks.workunit.client.0.smithi012.stdout: 1 393902 4.02 MB/sec execute 399 sec latency 2836.577 ms 2024-09-11T00:16:43.670 INFO:tasks.workunit.client.0.smithi012.stdout: 1 393902 4.01 MB/sec execute 400 sec latency 3836.640 ms 2024-09-11T00:16:44.063 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:16:43 smithi012 ceph-mon[30333]: pgmap v688: 129 pgs: 129 active+clean; 97 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.0 MiB/s wr, 294 op/s 2024-09-11T00:16:44.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:16:43 smithi178 ceph-mon[35919]: pgmap v688: 129 pgs: 129 active+clean; 97 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.0 MiB/s wr, 294 op/s 2024-09-11T00:16:44.283 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:16:43 smithi165 ceph-mon[35356]: pgmap v688: 129 pgs: 129 active+clean; 97 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.0 MiB/s wr, 294 op/s 2024-09-11T00:16:44.670 INFO:tasks.workunit.client.0.smithi012.stdout: 1 393902 4.00 MB/sec execute 401 sec latency 4836.771 ms 2024-09-11T00:16:45.671 INFO:tasks.workunit.client.0.smithi012.stdout: 1 393902 3.99 MB/sec execute 402 sec latency 5837.994 ms 2024-09-11T00:16:45.919 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:16:45 smithi012 ceph-mon[30333]: pgmap v689: 129 pgs: 129 active+clean; 96 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 1.6 MiB/s wr, 233 op/s 2024-09-11T00:16:46.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:16:45 smithi178 ceph-mon[35919]: pgmap v689: 129 pgs: 129 active+clean; 96 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 1.6 MiB/s wr, 233 op/s 2024-09-11T00:16:46.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:16:45 smithi165 ceph-mon[35356]: pgmap v689: 129 pgs: 129 active+clean; 96 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 1.6 MiB/s wr, 233 op/s 2024-09-11T00:16:46.671 INFO:tasks.workunit.client.0.smithi012.stdout: 1 393902 3.98 MB/sec execute 403 sec latency 6838.034 ms 2024-09-11T00:16:47.671 INFO:tasks.workunit.client.0.smithi012.stdout: 1 393902 3.97 MB/sec execute 404 sec latency 7838.079 ms 2024-09-11T00:16:48.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:16:47 smithi178 ceph-mon[35919]: pgmap v690: 129 pgs: 129 active+clean; 97 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.1 MiB/s wr, 274 op/s 2024-09-11T00:16:48.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:16:47 smithi165 ceph-mon[35356]: pgmap v690: 129 pgs: 129 active+clean; 97 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.1 MiB/s wr, 274 op/s 2024-09-11T00:16:48.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:16:47 smithi012 ceph-mon[30333]: pgmap v690: 129 pgs: 129 active+clean; 97 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.1 MiB/s wr, 274 op/s 2024-09-11T00:16:48.671 INFO:tasks.workunit.client.0.smithi012.stdout: 1 393902 3.96 MB/sec execute 405 sec latency 8838.187 ms 2024-09-11T00:16:49.671 INFO:tasks.workunit.client.0.smithi012.stdout: 1 393902 3.95 MB/sec execute 406 sec latency 9838.310 ms 2024-09-11T00:16:50.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:16:49 smithi178 ceph-mon[35919]: pgmap v691: 129 pgs: 129 active+clean; 97 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 952 KiB/s rd, 1.6 MiB/s wr, 176 op/s 2024-09-11T00:16:50.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:16:49 smithi165 ceph-mon[35356]: pgmap v691: 129 pgs: 129 active+clean; 97 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 952 KiB/s rd, 1.6 MiB/s wr, 176 op/s 2024-09-11T00:16:50.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:16:49 smithi012 ceph-mon[30333]: pgmap v691: 129 pgs: 129 active+clean; 97 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 952 KiB/s rd, 1.6 MiB/s wr, 176 op/s 2024-09-11T00:16:50.680 INFO:tasks.workunit.client.0.smithi012.stdout: 1 393902 3.94 MB/sec execute 407 sec latency 10847.333 ms 2024-09-11T00:16:51.683 INFO:tasks.workunit.client.0.smithi012.stdout: 1 393902 3.93 MB/sec execute 408 sec latency 11849.967 ms 2024-09-11T00:16:52.063 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:16:51 smithi012 ceph-mon[30333]: pgmap v692: 129 pgs: 129 active+clean; 95 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 956 KiB/s rd, 1.9 MiB/s wr, 190 op/s 2024-09-11T00:16:52.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:16:51 smithi165 ceph-mon[35356]: pgmap v692: 129 pgs: 129 active+clean; 95 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 956 KiB/s rd, 1.9 MiB/s wr, 190 op/s 2024-09-11T00:16:52.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:16:51 smithi178 ceph-mon[35919]: pgmap v692: 129 pgs: 129 active+clean; 95 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 956 KiB/s rd, 1.9 MiB/s wr, 190 op/s 2024-09-11T00:16:52.683 INFO:tasks.workunit.client.0.smithi012.stdout: 1 393902 3.92 MB/sec execute 409 sec latency 12850.030 ms 2024-09-11T00:16:53.683 INFO:tasks.workunit.client.0.smithi012.stdout: 1 393902 3.91 MB/sec execute 410 sec latency 13850.169 ms 2024-09-11T00:16:53.948 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:53.946+0000 7f8738000640 1 -- 172.21.15.12:0/2534375719 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f87301044a0 msgr2=0x7f873010ae20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:16:53.949 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:53.946+0000 7f8738000640 1 --2- 172.21.15.12:0/2534375719 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f87301044a0 0x7f873010ae20 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f8728009cd0 tx=0x7f872802a8e0 comp rx=0 tx=0).stop 2024-09-11T00:16:53.949 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:53.946+0000 7f8738000640 1 -- 172.21.15.12:0/2534375719 shutdown_connections 2024-09-11T00:16:53.949 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:53.946+0000 7f8738000640 1 --2- 172.21.15.12:0/2534375719 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f87301044a0 0x7f873010ae20 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:16:53.949 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:53.946+0000 7f8738000640 1 --2- 172.21.15.12:0/2534375719 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8730103ae0 0x7f8730103f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:16:53.949 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:53.946+0000 7f8738000640 1 --2- 172.21.15.12:0/2534375719 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8730102990 0x7f8730102d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:16:53.949 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:53.946+0000 7f8738000640 1 -- 172.21.15.12:0/2534375719 >> 172.21.15.12:0/2534375719 conn(0x7f87300fe250 msgr2=0x7f8730100670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:16:53.949 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:53.946+0000 7f8738000640 1 -- 172.21.15.12:0/2534375719 shutdown_connections 2024-09-11T00:16:53.950 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:53.946+0000 7f8738000640 1 -- 172.21.15.12:0/2534375719 wait complete. 2024-09-11T00:16:53.952 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:53.947+0000 7f8738000640 1 Processor -- start 2024-09-11T00:16:53.952 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:53.947+0000 7f8738000640 1 -- start start 2024-09-11T00:16:53.952 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:53.948+0000 7f8738000640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8730102990 0x7f873019af40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:16:53.952 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:53.948+0000 7f8738000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8730103ae0 0x7f873019b590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:16:53.952 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:53.948+0000 7f8738000640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f87301044a0 0x7f873019c1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:16:53.952 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:53.948+0000 7f8738000640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f873010e600 con 0x7f8730102990 2024-09-11T00:16:53.952 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:53.948+0000 7f8738000640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f873010e480 con 0x7f8730103ae0 2024-09-11T00:16:53.952 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:53.948+0000 7f8738000640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f873010e780 con 0x7f87301044a0 2024-09-11T00:16:53.953 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:53.948+0000 7f8735600640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8730102990 0x7f873019af40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:16:53.953 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:53.948+0000 7f8735600640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8730102990 0x7f873019af40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:40124/0 (socket says 172.21.15.12:40124) 2024-09-11T00:16:53.953 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:53.948+0000 7f8735600640 1 -- 172.21.15.12:0/358213554 learned_addr learned my addr 172.21.15.12:0/358213554 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:16:53.953 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:53.948+0000 7f8736000640 1 --2- 172.21.15.12:0/358213554 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f87301044a0 0x7f873019c1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:16:53.953 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:53.948+0000 7f8734c00640 1 --2- 172.21.15.12:0/358213554 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8730103ae0 0x7f873019b590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:16:53.953 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:53.948+0000 7f8735600640 1 -- 172.21.15.12:0/358213554 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f87301044a0 msgr2=0x7f873019c1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:16:53.953 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:53.948+0000 7f8735600640 1 --2- 172.21.15.12:0/358213554 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f87301044a0 0x7f873019c1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:16:53.953 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:53.948+0000 7f8735600640 1 -- 172.21.15.12:0/358213554 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8730103ae0 msgr2=0x7f873019b590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:16:53.953 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:53.948+0000 7f8735600640 1 --2- 172.21.15.12:0/358213554 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8730103ae0 0x7f873019b590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:16:53.954 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:53.948+0000 7f8735600640 1 -- 172.21.15.12:0/358213554 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f873019ca80 con 0x7f8730102990 2024-09-11T00:16:53.954 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:53.949+0000 7f8735600640 1 --2- 172.21.15.12:0/358213554 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8730102990 0x7f873019af40 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f872400e9f0 tx=0x7f872400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:16:53.954 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:53.949+0000 7f872e000640 1 -- 172.21.15.12:0/358213554 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f872400cd80 con 0x7f8730102990 2024-09-11T00:16:53.954 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:53.949+0000 7f872e000640 1 -- 172.21.15.12:0/358213554 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f87240054d0 con 0x7f8730102990 2024-09-11T00:16:53.954 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:53.949+0000 7f872e000640 1 -- 172.21.15.12:0/358213554 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8724010690 con 0x7f8730102990 2024-09-11T00:16:53.954 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:53.949+0000 7f8738000640 1 -- 172.21.15.12:0/358213554 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f873019cd70 con 0x7f8730102990 2024-09-11T00:16:53.954 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:53.949+0000 7f8738000640 1 -- 172.21.15.12:0/358213554 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87301a3440 con 0x7f8730102990 2024-09-11T00:16:53.957 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:53.952+0000 7f873adef740 1 Processor -- start 2024-09-11T00:16:53.958 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:53.952+0000 7f873adef740 1 -- start start 2024-09-11T00:16:53.958 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:53.952+0000 7f873adef740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559c726ea790 0x559c726ecb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:16:53.958 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:53.952+0000 7f873adef740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559c726ed0c0 0x559c726ef4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:16:53.958 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:53.952+0000 7f873adef740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559c726efb00 0x559c726f1fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:16:53.958 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:53.952+0000 7f873adef740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x559c7265d6b0 con 0x559c726ed0c0 2024-09-11T00:16:53.958 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:53.952+0000 7f873adef740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x559c7264cae0 con 0x559c726efb00 2024-09-11T00:16:53.958 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:53.953+0000 7f873adef740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x559c726508f0 con 0x559c726ea790 2024-09-11T00:16:53.958 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:53.953+0000 7f872e000640 1 -- 172.21.15.12:0/358213554 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7f8724004e90 con 0x7f8730102990 2024-09-11T00:16:53.958 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:53.953+0000 7f8736000640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559c726ea790 0x559c726ecb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:16:53.959 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:53.953+0000 7f8736000640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559c726ea790 0x559c726ecb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.12:38346/0 (socket says 172.21.15.12:38346) 2024-09-11T00:16:53.959 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:53.953+0000 7f8736000640 1 -- 172.21.15.12:0/2551504201 learned_addr learned my addr 172.21.15.12:0/2551504201 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:16:53.959 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:53.953+0000 7f8734c00640 1 --2- 172.21.15.12:0/2551504201 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559c726efb00 0x559c726f1fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:16:53.959 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:53.953+0000 7f8735600640 1 --2- 172.21.15.12:0/2551504201 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559c726ed0c0 0x559c726ef4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:16:53.959 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:53.953+0000 7f8736000640 1 -- 172.21.15.12:0/2551504201 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559c726efb00 msgr2=0x559c726f1fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:16:53.959 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:53.953+0000 7f8736000640 1 --2- 172.21.15.12:0/2551504201 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559c726efb00 0x559c726f1fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:16:53.959 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:53.953+0000 7f8736000640 1 -- 172.21.15.12:0/2551504201 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559c726ed0c0 msgr2=0x559c726ef4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:16:53.959 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:53.953+0000 7f8736000640 1 --2- 172.21.15.12:0/2551504201 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559c726ed0c0 0x559c726ef4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:16:53.959 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:53.953+0000 7f8736000640 1 -- 172.21.15.12:0/2551504201 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559c72619780 con 0x559c726ea790 2024-09-11T00:16:53.959 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:53.954+0000 7f8736000640 1 --2- 172.21.15.12:0/2551504201 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559c726ea790 0x559c726ecb80 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f872802ae00 tx=0x7f8728009a20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d9a5ccd8c1c3b178 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:16:53.959 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:53.954+0000 7f8722600640 1 -- 172.21.15.12:0/2551504201 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8728031430 con 0x559c726ea790 2024-09-11T00:16:53.960 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:53.954+0000 7f8722600640 1 -- 172.21.15.12:0/2551504201 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8728036040 con 0x559c726ea790 2024-09-11T00:16:53.960 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:53.954+0000 7f8722600640 1 -- 172.21.15.12:0/2551504201 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87280042d0 con 0x559c726ea790 2024-09-11T00:16:53.960 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:53.954+0000 7f872e000640 1 --2- 172.21.15.12:0/358213554 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f87180b52e0 0x7f87180b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:16:53.960 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:53.955+0000 7f873adef740 1 -- 172.21.15.12:0/2551504201 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559c726ea790 msgr2=0x559c726ecb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:16:53.960 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:53.955+0000 7f873adef740 1 --2- 172.21.15.12:0/2551504201 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559c726ea790 0x559c726ecb80 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f872802ae00 tx=0x7f8728009a20 comp rx=0 tx=0).stop 2024-09-11T00:16:53.960 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:53.955+0000 7f872e000640 1 -- 172.21.15.12:0/358213554 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9208+0+0 (secure 0 0 0) 0x7f87240e6410 con 0x7f8730102990 2024-09-11T00:16:53.960 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:53.955+0000 7f8735600640 1 --2- 172.21.15.12:0/358213554 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f87180b52e0 0x7f87180b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:16:53.960 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:53.955+0000 7f873adef740 1 -- 172.21.15.12:0/2551504201 shutdown_connections 2024-09-11T00:16:53.960 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:53.955+0000 7f873adef740 1 --2- 172.21.15.12:0/2551504201 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559c726efb00 0x559c726f1fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:16:53.960 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:53.955+0000 7f873adef740 1 --2- 172.21.15.12:0/2551504201 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559c726ed0c0 0x559c726ef4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:16:53.961 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:53.955+0000 7f873adef740 1 --2- 172.21.15.12:0/2551504201 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559c726ea790 0x559c726ecb80 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:16:53.961 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:53.955+0000 7f873adef740 1 -- 172.21.15.12:0/2551504201 >> 172.21.15.12:0/2551504201 conn(0x559c726e58e0 msgr2=0x559c726e6670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:16:53.961 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:53.955+0000 7f873adef740 1 -- 172.21.15.12:0/2551504201 shutdown_connections 2024-09-11T00:16:53.961 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:53.955+0000 7f873adef740 1 -- 172.21.15.12:0/2551504201 wait complete. 2024-09-11T00:16:53.961 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:53.955+0000 7f8735600640 1 --2- 172.21.15.12:0/358213554 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f87180b52e0 0x7f87180b77a0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f87240ac730 tx=0x7f87240a8040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:16:53.961 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:53.955+0000 7f873adef740 1 -- start start 2024-09-11T00:16:53.961 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:53.956+0000 7f873adef740 1 Processor -- start 2024-09-11T00:16:53.964 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:53.956+0000 7f873adef740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559c726ea790 0x559c726f3d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:16:53.965 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:53.956+0000 7f873adef740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559c726ed0c0 0x559c726f4410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:16:53.965 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:53.956+0000 7f873adef740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559c726efb00 0x559c726f4fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:16:53.965 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:53.956+0000 7f873adef740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x559c72584ab0 con 0x559c726ed0c0 2024-09-11T00:16:53.965 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:53.956+0000 7f873adef740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x559c725f3570 con 0x559c726efb00 2024-09-11T00:16:53.965 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:53.956+0000 7f873adef740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x559c726f2a20 con 0x559c726ea790 2024-09-11T00:16:53.965 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:53.956+0000 7f873adef740 10 client.0 _ll_register_callbacks cb 0x559c72602100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:16:53.965 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:53.956+0000 7f8736000640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559c726ea790 0x559c726f3d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:16:53.965 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:53.956+0000 7f8736000640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559c726ea790 0x559c726f3d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.12:38356/0 (socket says 172.21.15.12:38356) 2024-09-11T00:16:53.965 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:53.956+0000 7f8736000640 1 -- 172.21.15.12:0/1466609954 learned_addr learned my addr 172.21.15.12:0/1466609954 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:16:53.965 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:53.956+0000 7f8734c00640 1 --2- 172.21.15.12:0/1466609954 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559c726ed0c0 0x559c726f4410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:16:53.966 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:53.957+0000 7f8736000640 1 --2- 172.21.15.12:0/1466609954 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559c726efb00 0x559c726f4fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:16:53.966 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:53.957+0000 7f8736000640 1 -- 172.21.15.12:0/1466609954 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559c726efb00 msgr2=0x559c726f4fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:16:53.966 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:53.957+0000 7f8736000640 1 --2- 172.21.15.12:0/1466609954 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559c726efb00 0x559c726f4fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:16:53.966 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:53.957+0000 7f8736000640 1 -- 172.21.15.12:0/1466609954 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559c726ed0c0 msgr2=0x559c726f4410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:16:53.966 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:53.957+0000 7f8736000640 1 --2- 172.21.15.12:0/1466609954 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559c726ed0c0 0x559c726f4410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:16:53.966 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:53.957+0000 7f8736000640 1 -- 172.21.15.12:0/1466609954 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559c723ae790 con 0x559c726ea790 2024-09-11T00:16:53.966 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:53.957+0000 7f8736000640 1 --2- 172.21.15.12:0/1466609954 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559c726ea790 0x559c726f3d70 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f8728046040 tx=0x7f8728005fb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b10beb82c6f8dfaa server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:16:53.966 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:53.957+0000 7f8716a00640 10 client.0 ms_handle_connect on v2:172.21.15.178:3300/0 2024-09-11T00:16:53.966 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:53.957+0000 7f8716a00640 1 -- 172.21.15.12:0/1466609954 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8728003c00 con 0x559c726ea790 2024-09-11T00:16:53.966 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:53.957+0000 7f873adef740 1 -- 172.21.15.12:0/1466609954 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8728036040 con 0x559c726ea790 2024-09-11T00:16:53.967 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:53.957+0000 7f8716a00640 1 -- 172.21.15.12:0/1466609954 <== mon.1 v2:172.21.15.178:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9208+0+0 (secure 0 0 0) 0x7f872804a040 con 0x559c726ea790 2024-09-11T00:16:53.967 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:53.958+0000 7f8716a00640 1 -- 172.21.15.12:0/1466609954 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f8728004b40 con 0x559c726ea790 2024-09-11T00:16:53.967 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:53.958+0000 7f873adef740 10 client.26599 fetch_fsmap learned FSMap version 41 2024-09-11T00:16:53.967 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:53.958+0000 7f873adef740 1 -- 172.21.15.12:0/1466609954 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x559c72427620 con 0x559c726ea790 2024-09-11T00:16:53.967 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:53.958+0000 7f8716a00640 1 -- 172.21.15.12:0/1466609954 <== mon.1 v2:172.21.15.178:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8728036040 con 0x559c726ea790 2024-09-11T00:16:53.967 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:53.959+0000 7f8716a00640 1 -- 172.21.15.12:0/1466609954 <== mon.1 v2:172.21.15.178:3300/0 5 ==== fsmap(e 41) ==== 8370+0+0 (secure 0 0 0) 0x7f8728030070 con 0x559c726ea790 2024-09-11T00:16:53.967 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:53.959+0000 7f873adef740 10 client.26599 fetch_fsmap finished waiting for FSMap version 41 2024-09-11T00:16:53.967 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:53.959+0000 7f873adef740 10 client.26599 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T00:16:53.967 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:53.959+0000 7f873adef740 20 client.26599 populate_metadata read hostname 'smithi012' 2024-09-11T00:16:53.967 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:53.959+0000 7f873adef740 1 --2- 172.21.15.12:0/1466609954 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x559c726ff210 0x559c72701600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:16:53.967 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:53.959+0000 7f873adef740 4 client.26599 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T00:16:53.967 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:53.959+0000 7f873adef740 1 -- 172.21.15.12:0/1466609954 --> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559c7265f600 con 0x559c726ff210 2024-09-11T00:16:53.967 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:53.959+0000 7f8734c00640 1 --2- 172.21.15.12:0/1466609954 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x559c726ff210 0x559c72701600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:16:53.968 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:53.960+0000 7f8734c00640 1 --2- 172.21.15.12:0/1466609954 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x559c726ff210 0x559c72701600 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fc6d519ce860c372 server_cookie=7cde7d9ff29af894 in_seq=0 out_seq=0 2024-09-11T00:16:53.968 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:53.960+0000 7f8716a00640 10 client.26599 ms_handle_connect on v2:172.21.15.165:6838/501312984 2024-09-11T00:16:53.968 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:53.961+0000 7f8716a00640 1 -- 172.21.15.12:0/1466609954 <== mds.0 v2:172.21.15.165:6838/501312984 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x559c7265f600 con 0x559c726ff210 2024-09-11T00:16:53.968 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:53.961+0000 7f8716a00640 10 client.26599 handle_command_reply: tid=0 2024-09-11T00:16:53.968 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:53.961+0000 7f873adef740 1 client.26599 shutdown 2024-09-11T00:16:53.968 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:53.961+0000 7f873adef740 1 -- 172.21.15.12:0/1466609954 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559c726ea790 msgr2=0x559c726f3d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:16:53.968 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:53.961+0000 7f873adef740 1 --2- 172.21.15.12:0/1466609954 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559c726ea790 0x559c726f3d70 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f8728046040 tx=0x7f8728005fb0 comp rx=0 tx=0).stop 2024-09-11T00:16:53.968 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:53.961+0000 7f873adef740 1 -- 172.21.15.12:0/1466609954 shutdown_connections 2024-09-11T00:16:53.968 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:53.961+0000 7f873adef740 1 --2- 172.21.15.12:0/1466609954 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x559c726ff210 0x559c72701600 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:16:53.968 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:53.961+0000 7f873adef740 1 --2- 172.21.15.12:0/1466609954 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559c726efb00 0x559c726f4fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:16:53.968 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:53.961+0000 7f873adef740 1 --2- 172.21.15.12:0/1466609954 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559c726ed0c0 0x559c726f4410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:16:53.968 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:53.961+0000 7f873adef740 1 --2- 172.21.15.12:0/1466609954 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559c726ea790 0x559c726f3d70 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:16:53.968 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:53.961+0000 7f8716a00640 0 client.26599 ms_handle_reset on v2:172.21.15.165:6838/501312984 2024-09-11T00:16:53.969 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:53.961+0000 7f873adef740 1 -- 172.21.15.12:0/1466609954 >> 172.21.15.12:0/1466609954 conn(0x559c726e58e0 msgr2=0x559c726e4a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:16:53.969 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:53.961+0000 7f873adef740 1 -- 172.21.15.12:0/1466609954 shutdown_connections 2024-09-11T00:16:53.969 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:53.961+0000 7f873adef740 1 -- 172.21.15.12:0/1466609954 wait complete. 2024-09-11T00:16:53.969 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:53.961+0000 7f873adef740 20 client.26599 trim_cache size 0 max 16384 2024-09-11T00:16:53.980 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:53.977+0000 7f873adef740 1 Processor -- start 2024-09-11T00:16:53.980 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:53.977+0000 7f873adef740 1 -- start start 2024-09-11T00:16:53.980 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:53.977+0000 7f873adef740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559c726ea790 0x559c726efee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:16:53.980 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:53.977+0000 7f873adef740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559c726ed0c0 0x559c726f0420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:16:53.980 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:53.977+0000 7f873adef740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559c726efb00 0x559c726e9b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:16:53.980 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:53.977+0000 7f873adef740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f87301ad840 con 0x559c726efb00 2024-09-11T00:16:53.980 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:53.977+0000 7f873adef740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f87301af380 con 0x559c726ea790 2024-09-11T00:16:53.981 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:53.977+0000 7f873adef740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f87301adf10 con 0x559c726ed0c0 2024-09-11T00:16:53.981 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:53.977+0000 7f8736000640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559c726efb00 0x559c726e9b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:16:53.981 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:53.977+0000 7f8736000640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559c726efb00 0x559c726e9b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:40146/0 (socket says 172.21.15.12:40146) 2024-09-11T00:16:53.981 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:53.977+0000 7f8736000640 1 -- 172.21.15.12:0/2680356698 learned_addr learned my addr 172.21.15.12:0/2680356698 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:16:53.981 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:53.977+0000 7f8734c00640 1 --2- 172.21.15.12:0/2680356698 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559c726ed0c0 0x559c726f0420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:16:53.981 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:53.977+0000 7f8736000640 1 --2- 172.21.15.12:0/2680356698 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559c726ea790 0x559c726efee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:16:53.981 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:53.978+0000 7f8736000640 1 -- 172.21.15.12:0/2680356698 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559c726ed0c0 msgr2=0x559c726f0420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:16:53.982 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:53.978+0000 7f8736000640 1 --2- 172.21.15.12:0/2680356698 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559c726ed0c0 0x559c726f0420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:16:53.982 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:53.978+0000 7f8736000640 1 -- 172.21.15.12:0/2680356698 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559c726ea790 msgr2=0x559c726efee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:16:53.982 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:53.978+0000 7f8736000640 1 --2- 172.21.15.12:0/2680356698 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559c726ea790 0x559c726efee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:16:53.982 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:53.978+0000 7f8736000640 1 -- 172.21.15.12:0/2680356698 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559c726ef820 con 0x559c726efb00 2024-09-11T00:16:53.982 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:53.978+0000 7f8734c00640 1 --2- 172.21.15.12:0/2680356698 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559c726ed0c0 0x559c726f0420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:16:53.982 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:53.978+0000 7f8736000640 1 --2- 172.21.15.12:0/2680356698 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559c726efb00 0x559c726e9b90 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f8728009cd0 tx=0x7f8728005d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=723de72e33331ab9 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:16:53.989 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:53.978+0000 7f8721c00640 1 -- 172.21.15.12:0/2680356698 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8728031950 con 0x559c726efb00 2024-09-11T00:16:53.989 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:53.978+0000 7f8721c00640 1 -- 172.21.15.12:0/2680356698 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8728036030 con 0x559c726efb00 2024-09-11T00:16:53.989 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:53.978+0000 7f8721c00640 1 -- 172.21.15.12:0/2680356698 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8728030a30 con 0x559c726efb00 2024-09-11T00:16:53.989 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:53.979+0000 7f873adef740 1 -- 172.21.15.12:0/2680356698 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559c726efb00 msgr2=0x559c726e9b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:16:53.990 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:53.979+0000 7f873adef740 1 --2- 172.21.15.12:0/2680356698 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559c726efb00 0x559c726e9b90 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f8728009cd0 tx=0x7f8728005d80 comp rx=0 tx=0).stop 2024-09-11T00:16:53.990 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:53.979+0000 7f873adef740 1 -- 172.21.15.12:0/2680356698 shutdown_connections 2024-09-11T00:16:53.990 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:53.979+0000 7f873adef740 1 --2- 172.21.15.12:0/2680356698 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559c726efb00 0x559c726e9b90 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:16:53.990 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:53.979+0000 7f873adef740 1 --2- 172.21.15.12:0/2680356698 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559c726ed0c0 0x559c726f0420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:16:53.990 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:53.979+0000 7f873adef740 1 --2- 172.21.15.12:0/2680356698 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559c726ea790 0x559c726efee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:16:53.990 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:53.979+0000 7f873adef740 1 -- 172.21.15.12:0/2680356698 >> 172.21.15.12:0/2680356698 conn(0x559c726e58e0 msgr2=0x559c726e6380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:16:53.991 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:53.979+0000 7f873adef740 1 -- 172.21.15.12:0/2680356698 shutdown_connections 2024-09-11T00:16:53.991 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:53.979+0000 7f873adef740 1 -- 172.21.15.12:0/2680356698 wait complete. 2024-09-11T00:16:53.991 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:53.979+0000 7f873adef740 1 -- start start 2024-09-11T00:16:53.991 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:53.979+0000 7f873adef740 1 Processor -- start 2024-09-11T00:16:53.991 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:53.980+0000 7f873adef740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559c726ea790 0x559c726fb8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:16:53.991 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:53.980+0000 7f873adef740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559c726ed0c0 0x559c726fa5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:16:53.992 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:53.980+0000 7f873adef740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559c726efb00 0x559c726fab30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:16:53.992 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:53.980+0000 7f873adef740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f87301aecb0 con 0x559c726ed0c0 2024-09-11T00:16:53.992 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:53.980+0000 7f873adef740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f87301ae5e0 con 0x559c726ea790 2024-09-11T00:16:53.992 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:53.980+0000 7f873adef740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f87301ab820 con 0x559c726efb00 2024-09-11T00:16:53.992 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:53.980+0000 7f873adef740 10 client.0 _ll_register_callbacks cb 0x7f871c0097a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:16:53.993 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:53.980+0000 7f8734c00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559c726ed0c0 0x559c726fa5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:16:53.993 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:53.980+0000 7f8734c00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559c726ed0c0 0x559c726fa5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:40152/0 (socket says 172.21.15.12:40152) 2024-09-11T00:16:53.993 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:53.980+0000 7f8734c00640 1 -- 172.21.15.12:0/264332497 learned_addr learned my addr 172.21.15.12:0/264332497 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:16:53.993 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:53.980+0000 7f8736000640 1 --2- 172.21.15.12:0/264332497 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559c726ea790 0x559c726fb8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:16:53.993 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:53.980+0000 7f8736000640 1 --2- 172.21.15.12:0/264332497 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559c726efb00 0x559c726fab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:16:53.994 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:53.981+0000 7f8734c00640 1 -- 172.21.15.12:0/264332497 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559c726efb00 msgr2=0x559c726fab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:16:53.994 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:53.981+0000 7f8734c00640 1 --2- 172.21.15.12:0/264332497 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559c726efb00 0x559c726fab30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:16:53.994 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:53.981+0000 7f8734c00640 1 -- 172.21.15.12:0/264332497 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559c726ea790 msgr2=0x559c726fb8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:16:53.994 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:53.981+0000 7f8734c00640 1 --2- 172.21.15.12:0/264332497 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559c726ea790 0x559c726fb8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:16:53.994 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:53.981+0000 7f8734c00640 1 -- 172.21.15.12:0/264332497 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559c726fb360 con 0x559c726ed0c0 2024-09-11T00:16:53.995 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:53.981+0000 7f8736000640 1 --2- 172.21.15.12:0/264332497 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559c726efb00 0x559c726fab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T00:16:53.995 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:53.981+0000 7f8734c00640 1 --2- 172.21.15.12:0/264332497 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559c726ed0c0 0x559c726fa5b0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f871c004700 tx=0x7f871c010b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6251ba9c3e1f7230 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:16:53.995 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:53.981+0000 7f8716a00640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-09-11T00:16:53.995 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:53.981+0000 7f8716a00640 1 -- 172.21.15.12:0/264332497 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f871c028020 con 0x559c726ed0c0 2024-09-11T00:16:53.995 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:53.981+0000 7f8716a00640 1 -- 172.21.15.12:0/264332497 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9208+0+0 (secure 0 0 0) 0x7f871c005020 con 0x559c726ed0c0 2024-09-11T00:16:53.995 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:53.981+0000 7f873adef740 1 -- 172.21.15.12:0/264332497 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8728036030 con 0x559c726ed0c0 2024-09-11T00:16:53.995 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:53.981+0000 7f8716a00640 1 -- 172.21.15.12:0/264332497 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f871c030050 con 0x559c726ed0c0 2024-09-11T00:16:53.996 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:53.981+0000 7f873adef740 10 client.16623 fetch_fsmap learned FSMap version 41 2024-09-11T00:16:53.996 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:53.982+0000 7f873adef740 1 -- 172.21.15.12:0/264332497 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x559c7276c240 con 0x559c726ed0c0 2024-09-11T00:16:53.996 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:53.982+0000 7f8716a00640 1 -- 172.21.15.12:0/264332497 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8728036030 con 0x559c726ed0c0 2024-09-11T00:16:53.996 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:53.982+0000 7f8716a00640 1 -- 172.21.15.12:0/264332497 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 41) ==== 8370+0+0 (secure 0 0 0) 0x7f871c004830 con 0x559c726ed0c0 2024-09-11T00:16:53.996 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:53.982+0000 7f873adef740 10 client.16623 fetch_fsmap finished waiting for FSMap version 41 2024-09-11T00:16:53.996 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:53.982+0000 7f873adef740 10 client.16623 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T00:16:53.996 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:53.982+0000 7f873adef740 20 client.16623 populate_metadata read hostname 'smithi012' 2024-09-11T00:16:53.996 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:53.982+0000 7f873adef740 1 --2- 172.21.15.12:0/264332497 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x559c7276cc40 0x559c7276f060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:16:53.997 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:53.982+0000 7f873adef740 4 client.16623 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T00:16:53.997 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:53.982+0000 7f873adef740 1 -- 172.21.15.12:0/264332497 --> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] -- command(tid 0: {"prefix": "flush journal"}) -- 0x559c7265b0b0 con 0x559c7276cc40 2024-09-11T00:16:53.997 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:53.982+0000 7f8736000640 1 --2- 172.21.15.12:0/264332497 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x559c7276cc40 0x559c7276f060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:16:53.997 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:53.983+0000 7f8736000640 1 --2- 172.21.15.12:0/264332497 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x559c7276cc40 0x559c7276f060 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8962939d280a5ad9 server_cookie=2d6c28f37a3922c7 in_seq=0 out_seq=0 2024-09-11T00:16:53.997 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:53.983+0000 7f8716a00640 10 client.16623 ms_handle_connect on v2:172.21.15.165:6838/501312984 2024-09-11T00:16:54.011 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:54.008+0000 7f8716a00640 1 -- 172.21.15.12:0/264332497 <== mds.0 v2:172.21.15.165:6838/501312984 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x559c7265b0b0 con 0x559c7276cc40 2024-09-11T00:16:54.011 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:54.008+0000 7f8716a00640 10 client.16623 handle_command_reply: tid=0 2024-09-11T00:16:54.012 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:54.008+0000 7f873adef740 1 client.16623 shutdown 2024-09-11T00:16:54.012 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:54.008+0000 7f873adef740 1 -- 172.21.15.12:0/264332497 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559c726ed0c0 msgr2=0x559c726fa5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:16:54.012 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:54.008+0000 7f873adef740 1 --2- 172.21.15.12:0/264332497 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559c726ed0c0 0x559c726fa5b0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f871c004700 tx=0x7f871c010b10 comp rx=0 tx=0).stop 2024-09-11T00:16:54.012 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:54.009+0000 7f873adef740 1 -- 172.21.15.12:0/264332497 shutdown_connections 2024-09-11T00:16:54.012 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:54.009+0000 7f873adef740 1 --2- 172.21.15.12:0/264332497 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x559c7276cc40 0x559c7276f060 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:16:54.012 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:54.009+0000 7f873adef740 1 --2- 172.21.15.12:0/264332497 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559c726efb00 0x559c726fab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:16:54.012 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:54.009+0000 7f873adef740 1 --2- 172.21.15.12:0/264332497 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559c726ed0c0 0x559c726fa5b0 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:16:54.012 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:54.009+0000 7f8716a00640 0 client.16623 ms_handle_reset on v2:172.21.15.165:6838/501312984 2024-09-11T00:16:54.012 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:54.009+0000 7f873adef740 1 --2- 172.21.15.12:0/264332497 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559c726ea790 0x559c726fb8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:16:54.012 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:54.009+0000 7f873adef740 1 -- 172.21.15.12:0/264332497 >> 172.21.15.12:0/264332497 conn(0x559c726e58e0 msgr2=0x559c726efee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:16:54.012 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:54.009+0000 7f873adef740 1 -- 172.21.15.12:0/264332497 shutdown_connections 2024-09-11T00:16:54.012 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:54.009+0000 7f873adef740 1 -- 172.21.15.12:0/264332497 wait complete. 2024-09-11T00:16:54.013 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:54.009+0000 7f873adef740 20 client.16623 trim_cache size 0 max 16384 2024-09-11T00:16:54.013 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:{ 2024-09-11T00:16:54.013 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "message": "", 2024-09-11T00:16:54.013 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "return_code": 0 2024-09-11T00:16:54.013 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:} 2024-09-11T00:16:54.014 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:54.010+0000 7f8738000640 1 -- 172.21.15.12:0/358213554 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f87180b52e0 msgr2=0x7f87180b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:16:54.014 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:54.010+0000 7f8738000640 1 --2- 172.21.15.12:0/358213554 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f87180b52e0 0x7f87180b77a0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f87240ac730 tx=0x7f87240a8040 comp rx=0 tx=0).stop 2024-09-11T00:16:54.014 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:54.010+0000 7f8738000640 1 -- 172.21.15.12:0/358213554 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8730102990 msgr2=0x7f873019af40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:16:54.014 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:54.010+0000 7f8738000640 1 --2- 172.21.15.12:0/358213554 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8730102990 0x7f873019af40 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f872400e9f0 tx=0x7f872400edf0 comp rx=0 tx=0).stop 2024-09-11T00:16:54.014 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:54.010+0000 7f8738000640 1 -- 172.21.15.12:0/358213554 shutdown_connections 2024-09-11T00:16:54.014 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:54.010+0000 7f8738000640 1 --2- 172.21.15.12:0/358213554 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f87180b52e0 0x7f87180b77a0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:16:54.014 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:54.010+0000 7f8738000640 1 --2- 172.21.15.12:0/358213554 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f87301044a0 0x7f873019c1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:16:54.014 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:54.010+0000 7f8738000640 1 --2- 172.21.15.12:0/358213554 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8730103ae0 0x7f873019b590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:16:54.014 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:54.010+0000 7f8738000640 1 --2- 172.21.15.12:0/358213554 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8730102990 0x7f873019af40 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:16:54.015 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:54.010+0000 7f8738000640 1 -- 172.21.15.12:0/358213554 >> 172.21.15.12:0/358213554 conn(0x7f87300fe250 msgr2=0x7f87301091e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:16:54.015 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:54.010+0000 7f8738000640 1 -- 172.21.15.12:0/358213554 shutdown_connections 2024-09-11T00:16:54.015 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:16:54.010+0000 7f8738000640 1 -- 172.21.15.12:0/358213554 wait complete. 2024-09-11T00:16:54.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:16:53 smithi165 ceph-mon[35356]: pgmap v693: 129 pgs: 129 active+clean; 95 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 334 KiB/s rd, 1.4 MiB/s wr, 115 op/s 2024-09-11T00:16:54.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:16:53 smithi012 ceph-mon[30333]: pgmap v693: 129 pgs: 129 active+clean; 95 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 334 KiB/s rd, 1.4 MiB/s wr, 115 op/s 2024-09-11T00:16:54.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:16:53 smithi178 ceph-mon[35919]: pgmap v693: 129 pgs: 129 active+clean; 95 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 334 KiB/s rd, 1.4 MiB/s wr, 115 op/s 2024-09-11T00:16:54.683 INFO:tasks.workunit.client.0.smithi012.stdout: 1 393902 3.90 MB/sec execute 411 sec latency 14850.286 ms 2024-09-11T00:16:55.683 INFO:tasks.workunit.client.0.smithi012.stdout: 1 393902 3.89 MB/sec execute 412 sec latency 15850.346 ms 2024-09-11T00:16:55.859 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:16:55 smithi012 ceph-mon[30333]: pgmap v694: 129 pgs: 129 active+clean; 95 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 1.2 MiB/s wr, 87 op/s 2024-09-11T00:16:56.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:16:55 smithi165 ceph-mon[35356]: pgmap v694: 129 pgs: 129 active+clean; 95 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 1.2 MiB/s wr, 87 op/s 2024-09-11T00:16:56.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:16:55 smithi178 ceph-mon[35919]: pgmap v694: 129 pgs: 129 active+clean; 95 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 1.2 MiB/s wr, 87 op/s 2024-09-11T00:16:56.684 INFO:tasks.workunit.client.0.smithi012.stdout: 1 393902 3.88 MB/sec execute 413 sec latency 16850.478 ms 2024-09-11T00:16:57.684 INFO:tasks.workunit.client.0.smithi012.stdout: 1 393902 3.88 MB/sec execute 414 sec latency 17850.608 ms 2024-09-11T00:16:58.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:16:57 smithi165 ceph-mon[35356]: pgmap v695: 129 pgs: 129 active+clean; 94 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 1.6 MiB/s wr, 112 op/s 2024-09-11T00:16:58.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:16:57 smithi012 ceph-mon[30333]: pgmap v695: 129 pgs: 129 active+clean; 94 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 1.6 MiB/s wr, 112 op/s 2024-09-11T00:16:58.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:16:57 smithi178 ceph-mon[35919]: pgmap v695: 129 pgs: 129 active+clean; 94 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 1.6 MiB/s wr, 112 op/s 2024-09-11T00:16:58.684 INFO:tasks.workunit.client.0.smithi012.stdout: 1 393902 3.87 MB/sec execute 415 sec latency 18850.746 ms 2024-09-11T00:16:59.684 INFO:tasks.workunit.client.0.smithi012.stdout: 1 393902 3.86 MB/sec execute 416 sec latency 19850.862 ms 2024-09-11T00:17:00.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:16:59 smithi165 ceph-mon[35356]: pgmap v696: 129 pgs: 129 active+clean; 94 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 1.1 MiB/s wr, 68 op/s 2024-09-11T00:17:00.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:16:59 smithi012 ceph-mon[30333]: pgmap v696: 129 pgs: 129 active+clean; 94 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 1.1 MiB/s wr, 68 op/s 2024-09-11T00:17:00.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:16:59 smithi178 ceph-mon[35919]: pgmap v696: 129 pgs: 129 active+clean; 94 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 1.1 MiB/s wr, 68 op/s 2024-09-11T00:17:00.684 INFO:tasks.workunit.client.0.smithi012.stdout: 1 393902 3.85 MB/sec execute 417 sec latency 20851.022 ms 2024-09-11T00:17:01.684 INFO:tasks.workunit.client.0.smithi012.stdout: 1 393902 3.84 MB/sec execute 418 sec latency 21851.104 ms 2024-09-11T00:17:02.063 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:17:01 smithi012 ceph-mon[30333]: pgmap v697: 129 pgs: 129 active+clean; 93 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 1.4 MiB/s wr, 83 op/s 2024-09-11T00:17:02.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:17:01 smithi165 ceph-mon[35356]: pgmap v697: 129 pgs: 129 active+clean; 93 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 1.4 MiB/s wr, 83 op/s 2024-09-11T00:17:02.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:17:01 smithi178 ceph-mon[35919]: pgmap v697: 129 pgs: 129 active+clean; 93 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 1.4 MiB/s wr, 83 op/s 2024-09-11T00:17:02.684 INFO:tasks.workunit.client.0.smithi012.stdout: 1 393902 3.83 MB/sec execute 419 sec latency 22851.194 ms 2024-09-11T00:17:03.684 INFO:tasks.workunit.client.0.smithi012.stdout: 1 393902 3.82 MB/sec execute 420 sec latency 23851.337 ms 2024-09-11T00:17:04.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:17:03 smithi165 ceph-mon[35356]: pgmap v698: 129 pgs: 129 active+clean; 94 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 1.1 MiB/s wr, 71 op/s 2024-09-11T00:17:04.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:17:03 smithi012 ceph-mon[30333]: pgmap v698: 129 pgs: 129 active+clean; 94 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 1.1 MiB/s wr, 71 op/s 2024-09-11T00:17:04.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:17:03 smithi178 ceph-mon[35919]: pgmap v698: 129 pgs: 129 active+clean; 94 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 1.1 MiB/s wr, 71 op/s 2024-09-11T00:17:04.684 INFO:tasks.workunit.client.0.smithi012.stdout: 1 393902 3.81 MB/sec execute 421 sec latency 24851.444 ms 2024-09-11T00:17:05.685 INFO:tasks.workunit.client.0.smithi012.stdout: 1 393902 3.80 MB/sec execute 422 sec latency 25851.591 ms 2024-09-11T00:17:06.099 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:17:05 smithi012 ceph-mon[30333]: pgmap v699: 129 pgs: 129 active+clean; 93 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 932 KiB/s wr, 63 op/s 2024-09-11T00:17:06.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:17:05 smithi165 ceph-mon[35356]: pgmap v699: 129 pgs: 129 active+clean; 93 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 932 KiB/s wr, 63 op/s 2024-09-11T00:17:06.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:17:05 smithi178 ceph-mon[35919]: pgmap v699: 129 pgs: 129 active+clean; 93 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 932 KiB/s wr, 63 op/s 2024-09-11T00:17:06.685 INFO:tasks.workunit.client.0.smithi012.stdout: 1 393902 3.79 MB/sec execute 423 sec latency 26851.712 ms 2024-09-11T00:17:07.131 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:07.129+0000 7f8712800640 1 -- 172.21.15.12:0/749157449 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f870c105ce0 msgr2=0x7f870c106160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:17:07.131 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:07.129+0000 7f8712800640 1 --2- 172.21.15.12:0/749157449 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f870c105ce0 0x7f870c106160 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f86f8009930 tx=0x7f86f802a420 comp rx=0 tx=0).stop 2024-09-11T00:17:07.131 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:07.129+0000 7f8712800640 1 -- 172.21.15.12:0/749157449 shutdown_connections 2024-09-11T00:17:07.131 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:07.129+0000 7f8712800640 1 --2- 172.21.15.12:0/749157449 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f870c1066a0 0x7f870c10d020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:17:07.131 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:07.129+0000 7f8712800640 1 --2- 172.21.15.12:0/749157449 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f870c105ce0 0x7f870c106160 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:17:07.131 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:07.129+0000 7f8712800640 1 --2- 172.21.15.12:0/749157449 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f870c104b90 0x7f870c104f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:17:07.131 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:07.129+0000 7f8712800640 1 -- 172.21.15.12:0/749157449 >> 172.21.15.12:0/749157449 conn(0x7f870c100470 msgr2=0x7f870c102890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:17:07.132 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:07.130+0000 7f8712800640 1 -- 172.21.15.12:0/749157449 shutdown_connections 2024-09-11T00:17:07.132 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:07.130+0000 7f8712800640 1 -- 172.21.15.12:0/749157449 wait complete. 2024-09-11T00:17:07.134 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:07.130+0000 7f8712800640 1 Processor -- start 2024-09-11T00:17:07.134 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:07.131+0000 7f8712800640 1 -- start start 2024-09-11T00:17:07.134 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:07.131+0000 7f8712800640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f870c104b90 0x7f870c19afb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:17:07.134 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:07.131+0000 7f8712800640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f870c105ce0 0x7f870c19b600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:17:07.134 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:07.131+0000 7f8712800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f870c1066a0 0x7f870c19c230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:17:07.134 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:07.131+0000 7f8712800640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f870c110850 con 0x7f870c104b90 2024-09-11T00:17:07.134 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:07.131+0000 7f8712800640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f870c1106d0 con 0x7f870c1066a0 2024-09-11T00:17:07.134 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:07.131+0000 7f8712800640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f870c1109d0 con 0x7f870c105ce0 2024-09-11T00:17:07.134 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:07.131+0000 7f870b400640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f870c104b90 0x7f870c19afb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:17:07.134 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:07.131+0000 7f870b400640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f870c104b90 0x7f870c19afb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:50594/0 (socket says 172.21.15.12:50594) 2024-09-11T00:17:07.134 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:07.131+0000 7f870b400640 1 -- 172.21.15.12:0/151184904 learned_addr learned my addr 172.21.15.12:0/151184904 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:17:07.135 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:07.131+0000 7f870be00640 1 --2- 172.21.15.12:0/151184904 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f870c1066a0 0x7f870c19c230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:17:07.135 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:07.131+0000 7f870aa00640 1 --2- 172.21.15.12:0/151184904 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f870c105ce0 0x7f870c19b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:17:07.135 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:07.131+0000 7f870b400640 1 -- 172.21.15.12:0/151184904 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f870c105ce0 msgr2=0x7f870c19b600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:17:07.135 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:07.131+0000 7f870b400640 1 --2- 172.21.15.12:0/151184904 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f870c105ce0 0x7f870c19b600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:17:07.135 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:07.131+0000 7f870b400640 1 -- 172.21.15.12:0/151184904 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f870c1066a0 msgr2=0x7f870c19c230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:17:07.135 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:07.131+0000 7f870b400640 1 --2- 172.21.15.12:0/151184904 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f870c1066a0 0x7f870c19c230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:17:07.135 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:07.131+0000 7f870b400640 1 -- 172.21.15.12:0/151184904 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f870c19caf0 con 0x7f870c104b90 2024-09-11T00:17:07.135 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:07.132+0000 7f870b400640 1 --2- 172.21.15.12:0/151184904 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f870c104b90 0x7f870c19afb0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f86fc00e9f0 tx=0x7f86fc00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:17:07.135 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:07.132+0000 7f8703e00640 1 -- 172.21.15.12:0/151184904 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86fc00cd80 con 0x7f870c104b90 2024-09-11T00:17:07.135 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:07.132+0000 7f8703e00640 1 -- 172.21.15.12:0/151184904 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f86fc0054d0 con 0x7f870c104b90 2024-09-11T00:17:07.135 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:07.132+0000 7f8703e00640 1 -- 172.21.15.12:0/151184904 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86fc010690 con 0x7f870c104b90 2024-09-11T00:17:07.135 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:07.132+0000 7f8712800640 1 -- 172.21.15.12:0/151184904 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f870c19cde0 con 0x7f870c104b90 2024-09-11T00:17:07.135 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:07.132+0000 7f8712800640 1 -- 172.21.15.12:0/151184904 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f870c1a34b0 con 0x7f870c104b90 2024-09-11T00:17:07.136 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:07.135+0000 7f87157de740 1 Processor -- start 2024-09-11T00:17:07.136 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:07.135+0000 7f87157de740 1 -- start start 2024-09-11T00:17:07.138 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:07.135+0000 7f87157de740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55870028d790 0x55870028fb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:17:07.138 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:07.135+0000 7f87157de740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5587002900c0 0x5587002924b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:17:07.138 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:07.135+0000 7f87157de740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558700292b00 0x558700294fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:17:07.138 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:07.135+0000 7f87157de740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x5587002006b0 con 0x55870028d790 2024-09-11T00:17:07.138 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:07.135+0000 7f87157de740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x5587001efae0 con 0x5587002900c0 2024-09-11T00:17:07.138 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:07.135+0000 7f87157de740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5587001f38f0 con 0x558700292b00 2024-09-11T00:17:07.138 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:07.135+0000 7f8703e00640 1 -- 172.21.15.12:0/151184904 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7f86fc004e90 con 0x7f870c104b90 2024-09-11T00:17:07.138 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:07.136+0000 7f870be00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55870028d790 0x55870028fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:17:07.138 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:07.136+0000 7f870be00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55870028d790 0x55870028fb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:50604/0 (socket says 172.21.15.12:50604) 2024-09-11T00:17:07.138 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:07.136+0000 7f870be00640 1 -- 172.21.15.12:0/438469044 learned_addr learned my addr 172.21.15.12:0/438469044 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:17:07.139 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:07.136+0000 7f870aa00640 1 --2- 172.21.15.12:0/438469044 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558700292b00 0x558700294fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:17:07.139 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:07.136+0000 7f870b400640 1 --2- 172.21.15.12:0/438469044 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5587002900c0 0x5587002924b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:17:07.139 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:07.136+0000 7f870be00640 1 -- 172.21.15.12:0/438469044 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558700292b00 msgr2=0x558700294fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:17:07.139 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:07.136+0000 7f870be00640 1 --2- 172.21.15.12:0/438469044 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558700292b00 0x558700294fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:17:07.139 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:07.136+0000 7f870be00640 1 -- 172.21.15.12:0/438469044 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5587002900c0 msgr2=0x5587002924b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:17:07.139 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:07.136+0000 7f870be00640 1 --2- 172.21.15.12:0/438469044 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5587002900c0 0x5587002924b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:17:07.139 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:07.136+0000 7f870be00640 1 -- 172.21.15.12:0/438469044 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5587001bc780 con 0x55870028d790 2024-09-11T00:17:07.139 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:07.136+0000 7f870be00640 1 --2- 172.21.15.12:0/438469044 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55870028d790 0x55870028fb80 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f8704009f90 tx=0x7f87040118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d36029934f2c37e server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:17:07.139 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:07.137+0000 7f8700c00640 1 -- 172.21.15.12:0/438469044 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8704004d20 con 0x55870028d790 2024-09-11T00:17:07.139 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:07.137+0000 7f8700c00640 1 -- 172.21.15.12:0/438469044 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8704004050 con 0x55870028d790 2024-09-11T00:17:07.139 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:07.137+0000 7f8700c00640 1 -- 172.21.15.12:0/438469044 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8704005280 con 0x55870028d790 2024-09-11T00:17:07.143 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:07.137+0000 7f8703e00640 1 --2- 172.21.15.12:0/151184904 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f86f00b5360 0x7f86f00b7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:17:07.143 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:07.137+0000 7f87157de740 1 -- 172.21.15.12:0/438469044 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55870028d790 msgr2=0x55870028fb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:17:07.143 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:07.137+0000 7f87157de740 1 --2- 172.21.15.12:0/438469044 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55870028d790 0x55870028fb80 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f8704009f90 tx=0x7f87040118d0 comp rx=0 tx=0).stop 2024-09-11T00:17:07.144 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:07.137+0000 7f87157de740 1 -- 172.21.15.12:0/438469044 shutdown_connections 2024-09-11T00:17:07.144 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:07.137+0000 7f87157de740 1 --2- 172.21.15.12:0/438469044 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558700292b00 0x558700294fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:17:07.144 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:07.137+0000 7f87157de740 1 --2- 172.21.15.12:0/438469044 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5587002900c0 0x5587002924b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:17:07.144 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:07.138+0000 7f87157de740 1 --2- 172.21.15.12:0/438469044 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55870028d790 0x55870028fb80 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:17:07.144 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:07.138+0000 7f87157de740 1 -- 172.21.15.12:0/438469044 >> 172.21.15.12:0/438469044 conn(0x5587002888e0 msgr2=0x558700289670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:17:07.144 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:07.138+0000 7f870b400640 1 --2- 172.21.15.12:0/151184904 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f86f00b5360 0x7f86f00b7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:17:07.144 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:07.138+0000 7f8703e00640 1 -- 172.21.15.12:0/151184904 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9208+0+0 (secure 0 0 0) 0x7f86fc0e6430 con 0x7f870c104b90 2024-09-11T00:17:07.144 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:07.138+0000 7f87157de740 1 -- 172.21.15.12:0/438469044 shutdown_connections 2024-09-11T00:17:07.144 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:07.138+0000 7f87157de740 1 -- 172.21.15.12:0/438469044 wait complete. 2024-09-11T00:17:07.144 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:07.138+0000 7f87157de740 1 -- start start 2024-09-11T00:17:07.144 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:07.138+0000 7f870b400640 1 --2- 172.21.15.12:0/151184904 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f86f00b5360 0x7f86f00b7820 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f86fc00c6a0 tx=0x7f86fc004a90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:17:07.145 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:07.138+0000 7f87157de740 1 Processor -- start 2024-09-11T00:17:07.145 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:07.139+0000 7f87157de740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55870028d790 0x558700296ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:17:07.145 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:07.139+0000 7f87157de740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5587002900c0 0x558700297560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:17:07.145 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:07.139+0000 7f87157de740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558700292b00 0x558700298130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:17:07.145 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:07.139+0000 7f87157de740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x558700127ab0 con 0x5587002900c0 2024-09-11T00:17:07.145 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:07.139+0000 7f87157de740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x558700196570 con 0x558700292b00 2024-09-11T00:17:07.145 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:07.139+0000 7f87157de740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x558700295ae0 con 0x55870028d790 2024-09-11T00:17:07.145 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:07.139+0000 7f87157de740 10 client.0 _ll_register_callbacks cb 0x5587001a5100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:17:07.145 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:07.139+0000 7f870be00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55870028d790 0x558700296ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:17:07.145 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:07.139+0000 7f870aa00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5587002900c0 0x558700297560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:17:07.145 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:07.139+0000 7f870be00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55870028d790 0x558700296ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.12:37962/0 (socket says 172.21.15.12:37962) 2024-09-11T00:17:07.145 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:07.139+0000 7f870be00640 1 -- 172.21.15.12:0/1472516001 learned_addr learned my addr 172.21.15.12:0/1472516001 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:17:07.146 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:07.139+0000 7f870be00640 1 --2- 172.21.15.12:0/1472516001 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558700292b00 0x558700298130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:17:07.146 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:07.140+0000 7f870aa00640 1 -- 172.21.15.12:0/1472516001 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55870028d790 msgr2=0x558700296ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:17:07.146 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:07.140+0000 7f870aa00640 1 --2- 172.21.15.12:0/1472516001 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55870028d790 0x558700296ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:17:07.146 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:07.140+0000 7f870aa00640 1 -- 172.21.15.12:0/1472516001 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558700292b00 msgr2=0x558700298130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:17:07.146 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:07.140+0000 7f870aa00640 1 --2- 172.21.15.12:0/1472516001 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558700292b00 0x558700298130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:17:07.146 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:07.140+0000 7f870aa00640 1 -- 172.21.15.12:0/1472516001 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5586fff51790 con 0x5587002900c0 2024-09-11T00:17:07.146 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:07.140+0000 7f870aa00640 1 --2- 172.21.15.12:0/1472516001 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5587002900c0 0x558700297560 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f86f802a940 tx=0x7f86f8005270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49889037234726ec server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:17:07.146 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:07.140+0000 7f86f5600640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-09-11T00:17:07.146 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:07.140+0000 7f86f5600640 1 -- 172.21.15.12:0/1472516001 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86f8035030 con 0x5587002900c0 2024-09-11T00:17:07.146 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:07.140+0000 7f86f5600640 1 -- 172.21.15.12:0/1472516001 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9208+0+0 (secure 0 0 0) 0x7f86f80048c0 con 0x5587002900c0 2024-09-11T00:17:07.146 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:07.140+0000 7f87157de740 1 -- 172.21.15.12:0/1472516001 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8704004050 con 0x5587002900c0 2024-09-11T00:17:07.146 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:07.140+0000 7f86f5600640 1 -- 172.21.15.12:0/1472516001 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f86f8005d50 con 0x5587002900c0 2024-09-11T00:17:07.146 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:07.140+0000 7f87157de740 10 client.16635 fetch_fsmap learned FSMap version 41 2024-09-11T00:17:07.146 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:07.140+0000 7f87157de740 1 -- 172.21.15.12:0/1472516001 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x5586fffca620 con 0x5587002900c0 2024-09-11T00:17:07.147 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:07.141+0000 7f86f5600640 1 -- 172.21.15.12:0/1472516001 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8704004050 con 0x5587002900c0 2024-09-11T00:17:07.147 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:07.141+0000 7f86f5600640 1 -- 172.21.15.12:0/1472516001 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 41) ==== 8370+0+0 (secure 0 0 0) 0x7f86f8003b10 con 0x5587002900c0 2024-09-11T00:17:07.147 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:07.141+0000 7f87157de740 10 client.16635 fetch_fsmap finished waiting for FSMap version 41 2024-09-11T00:17:07.147 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:07.141+0000 7f87157de740 10 client.16635 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T00:17:07.147 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:07.141+0000 7f87157de740 20 client.16635 populate_metadata read hostname 'smithi012' 2024-09-11T00:17:07.147 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:07.141+0000 7f87157de740 1 --2- 172.21.15.12:0/1472516001 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x5587002a27c0 0x5587002a4bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:17:07.147 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:07.141+0000 7f87157de740 4 client.16635 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T00:17:07.147 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:07.141+0000 7f87157de740 1 -- 172.21.15.12:0/1472516001 --> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558700202600 con 0x5587002a27c0 2024-09-11T00:17:07.147 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:07.142+0000 7f870be00640 1 --2- 172.21.15.12:0/1472516001 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x5587002a27c0 0x5587002a4bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:17:07.147 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:07.142+0000 7f870be00640 1 --2- 172.21.15.12:0/1472516001 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x5587002a27c0 0x5587002a4bb0 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=806781cb1558b4ef server_cookie=454cc9eed388b81f in_seq=0 out_seq=0 2024-09-11T00:17:07.147 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:07.142+0000 7f86f5600640 10 client.16635 ms_handle_connect on v2:172.21.15.165:6838/501312984 2024-09-11T00:17:07.148 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:07.143+0000 7f86f5600640 1 -- 172.21.15.12:0/1472516001 <== mds.0 v2:172.21.15.165:6838/501312984 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x558700202600 con 0x5587002a27c0 2024-09-11T00:17:07.148 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:07.143+0000 7f86f5600640 10 client.16635 handle_command_reply: tid=0 2024-09-11T00:17:07.149 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:07.143+0000 7f87157de740 1 client.16635 shutdown 2024-09-11T00:17:07.149 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:07.143+0000 7f87157de740 1 -- 172.21.15.12:0/1472516001 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5587002900c0 msgr2=0x558700297560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:17:07.149 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:07.143+0000 7f87157de740 1 --2- 172.21.15.12:0/1472516001 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5587002900c0 0x558700297560 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f86f802a940 tx=0x7f86f8005270 comp rx=0 tx=0).stop 2024-09-11T00:17:07.149 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:07.143+0000 7f87157de740 1 -- 172.21.15.12:0/1472516001 shutdown_connections 2024-09-11T00:17:07.149 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:07.143+0000 7f87157de740 1 --2- 172.21.15.12:0/1472516001 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x5587002a27c0 0x5587002a4bb0 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:17:07.149 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:07.143+0000 7f87157de740 1 --2- 172.21.15.12:0/1472516001 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558700292b00 0x558700298130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:17:07.149 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:07.143+0000 7f87157de740 1 --2- 172.21.15.12:0/1472516001 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5587002900c0 0x558700297560 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:17:07.149 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:07.143+0000 7f86f5600640 0 client.16635 ms_handle_reset on v2:172.21.15.165:6838/501312984 2024-09-11T00:17:07.149 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:07.143+0000 7f87157de740 1 --2- 172.21.15.12:0/1472516001 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55870028d790 0x558700296ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:17:07.149 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:07.143+0000 7f87157de740 1 -- 172.21.15.12:0/1472516001 >> 172.21.15.12:0/1472516001 conn(0x5587002888e0 msgr2=0x558700287a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:17:07.149 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:07.144+0000 7f87157de740 1 -- 172.21.15.12:0/1472516001 shutdown_connections 2024-09-11T00:17:07.149 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:07.144+0000 7f87157de740 1 -- 172.21.15.12:0/1472516001 wait complete. 2024-09-11T00:17:07.150 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:07.144+0000 7f87157de740 20 client.16635 trim_cache size 0 max 16384 2024-09-11T00:17:07.161 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:07.159+0000 7f87157de740 1 Processor -- start 2024-09-11T00:17:07.161 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:07.159+0000 7f87157de740 1 -- start start 2024-09-11T00:17:07.161 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:07.159+0000 7f87157de740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55870028d790 0x558700296ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:17:07.161 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:07.159+0000 7f87157de740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5587002900c0 0x558700297400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:17:07.161 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:07.159+0000 7f87157de740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558700292b00 0x558700289140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:17:07.161 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:07.159+0000 7f87157de740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f870c1ad900 con 0x55870028d790 2024-09-11T00:17:07.161 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:07.159+0000 7f87157de740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f870c1af440 con 0x5587002900c0 2024-09-11T00:17:07.161 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:07.159+0000 7f87157de740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f870c1adfd0 con 0x558700292b00 2024-09-11T00:17:07.161 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:07.160+0000 7f870be00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55870028d790 0x558700296ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:17:07.161 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:07.160+0000 7f870be00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55870028d790 0x558700296ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:50620/0 (socket says 172.21.15.12:50620) 2024-09-11T00:17:07.161 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:07.160+0000 7f870be00640 1 -- 172.21.15.12:0/83466329 learned_addr learned my addr 172.21.15.12:0/83466329 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:17:07.162 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:07.160+0000 7f870aa00640 1 --2- 172.21.15.12:0/83466329 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5587002900c0 0x558700297400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:17:07.164 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:07.160+0000 7f870be00640 1 -- 172.21.15.12:0/83466329 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558700292b00 msgr2=0x558700289140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:17:07.165 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:07.160+0000 7f870be00640 1 --2- 172.21.15.12:0/83466329 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558700292b00 0x558700289140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:17:07.165 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:07.160+0000 7f870be00640 1 -- 172.21.15.12:0/83466329 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5587002900c0 msgr2=0x558700297400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:17:07.165 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:07.160+0000 7f870be00640 1 --2- 172.21.15.12:0/83466329 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5587002900c0 0x558700297400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:17:07.165 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:07.160+0000 7f870be00640 1 -- 172.21.15.12:0/83466329 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558700292820 con 0x55870028d790 2024-09-11T00:17:07.165 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:07.160+0000 7f870be00640 1 --2- 172.21.15.12:0/83466329 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55870028d790 0x558700296ec0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f8704009f30 tx=0x7f8704000f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f25bc7050562e12e server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:17:07.165 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:07.160+0000 7f86f7e00640 1 -- 172.21.15.12:0/83466329 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8704003d80 con 0x55870028d790 2024-09-11T00:17:07.165 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:07.160+0000 7f86f7e00640 1 -- 172.21.15.12:0/83466329 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8704004320 con 0x55870028d790 2024-09-11T00:17:07.165 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:07.160+0000 7f86f7e00640 1 -- 172.21.15.12:0/83466329 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8704028450 con 0x55870028d790 2024-09-11T00:17:07.165 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:07.161+0000 7f87157de740 1 -- 172.21.15.12:0/83466329 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55870028d790 msgr2=0x558700296ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:17:07.165 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:07.161+0000 7f87157de740 1 --2- 172.21.15.12:0/83466329 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55870028d790 0x558700296ec0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f8704009f30 tx=0x7f8704000f40 comp rx=0 tx=0).stop 2024-09-11T00:17:07.165 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:07.161+0000 7f87157de740 1 -- 172.21.15.12:0/83466329 shutdown_connections 2024-09-11T00:17:07.165 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:07.161+0000 7f87157de740 1 --2- 172.21.15.12:0/83466329 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558700292b00 0x558700289140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:17:07.166 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:07.161+0000 7f87157de740 1 --2- 172.21.15.12:0/83466329 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5587002900c0 0x558700297400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:17:07.166 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:07.161+0000 7f87157de740 1 --2- 172.21.15.12:0/83466329 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55870028d790 0x558700296ec0 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:17:07.166 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:07.161+0000 7f87157de740 1 -- 172.21.15.12:0/83466329 >> 172.21.15.12:0/83466329 conn(0x5587002888e0 msgr2=0x55870028cc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:17:07.166 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:07.161+0000 7f87157de740 1 -- 172.21.15.12:0/83466329 shutdown_connections 2024-09-11T00:17:07.166 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:07.161+0000 7f87157de740 1 -- 172.21.15.12:0/83466329 wait complete. 2024-09-11T00:17:07.166 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:07.161+0000 7f87157de740 1 -- start start 2024-09-11T00:17:07.166 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:07.162+0000 7f87157de740 1 Processor -- start 2024-09-11T00:17:07.166 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:07.162+0000 7f87157de740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55870028d790 0x55870029daf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:17:07.166 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:07.162+0000 7f87157de740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5587002900c0 0x55870029e030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:17:07.166 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:07.162+0000 7f87157de740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558700292b00 0x55870029e570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:17:07.166 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:07.162+0000 7f87157de740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f870c1aed70 con 0x5587002900c0 2024-09-11T00:17:07.166 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:07.162+0000 7f87157de740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f870c1ae6a0 con 0x558700292b00 2024-09-11T00:17:07.166 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:07.162+0000 7f87157de740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f870c1ab8e0 con 0x55870028d790 2024-09-11T00:17:07.167 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:07.162+0000 7f87157de740 10 client.0 _ll_register_callbacks cb 0x7f87040093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:17:07.167 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:07.162+0000 7f870aa00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5587002900c0 0x55870029e030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:17:07.167 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:07.162+0000 7f870be00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558700292b00 0x55870029e570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:17:07.167 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:07.162+0000 7f870be00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558700292b00 0x55870029e570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.12:60948/0 (socket says 172.21.15.12:60948) 2024-09-11T00:17:07.167 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:07.162+0000 7f870be00640 1 -- 172.21.15.12:0/321404604 learned_addr learned my addr 172.21.15.12:0/321404604 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:17:07.167 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:07.162+0000 7f870aa00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5587002900c0 0x55870029e030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:50622/0 (socket says 172.21.15.12:50622) 2024-09-11T00:17:07.167 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:07.162+0000 7f870be00640 1 --2- 172.21.15.12:0/321404604 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55870028d790 0x55870029daf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:17:07.167 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:07.163+0000 7f870be00640 1 -- 172.21.15.12:0/321404604 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55870028d790 msgr2=0x55870029daf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:17:07.167 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:07.163+0000 7f870be00640 1 --2- 172.21.15.12:0/321404604 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55870028d790 0x55870029daf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:17:07.167 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:07.163+0000 7f870be00640 1 -- 172.21.15.12:0/321404604 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5587002900c0 msgr2=0x55870029e030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:17:07.167 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:07.163+0000 7f870be00640 1 --2- 172.21.15.12:0/321404604 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5587002900c0 0x55870029e030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:17:07.167 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:07.163+0000 7f870be00640 1 -- 172.21.15.12:0/321404604 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55870029c8b0 con 0x558700292b00 2024-09-11T00:17:07.167 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:07.163+0000 7f870be00640 1 --2- 172.21.15.12:0/321404604 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558700292b00 0x55870029e570 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f8704011e50 tx=0x7f87040096c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e229a2ad2e4b4721 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:17:07.168 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:07.163+0000 7f86f5600640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-09-11T00:17:07.168 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:07.163+0000 7f86f5600640 1 -- 172.21.15.12:0/321404604 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8704004d30 con 0x558700292b00 2024-09-11T00:17:07.168 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:07.163+0000 7f87157de740 1 -- 172.21.15.12:0/321404604 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8704004320 con 0x558700292b00 2024-09-11T00:17:07.168 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:07.163+0000 7f86f5600640 1 -- 172.21.15.12:0/321404604 <== mon.2 v2:172.21.15.165:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9208+0+0 (secure 0 0 0) 0x7f870400f4e0 con 0x558700292b00 2024-09-11T00:17:07.168 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:07.164+0000 7f86f5600640 1 -- 172.21.15.12:0/321404604 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f8704044050 con 0x558700292b00 2024-09-11T00:17:07.168 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:07.164+0000 7f87157de740 10 client.26639 fetch_fsmap learned FSMap version 41 2024-09-11T00:17:07.168 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:07.164+0000 7f87157de740 1 -- 172.21.15.12:0/321404604 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x55870030fb90 con 0x558700292b00 2024-09-11T00:17:07.169 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:07.164+0000 7f86f5600640 1 -- 172.21.15.12:0/321404604 <== mon.2 v2:172.21.15.165:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8704004320 con 0x558700292b00 2024-09-11T00:17:07.169 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:07.164+0000 7f86f5600640 1 -- 172.21.15.12:0/321404604 <== mon.2 v2:172.21.15.165:3300/0 5 ==== fsmap(e 41) ==== 8370+0+0 (secure 0 0 0) 0x7f8704028ad0 con 0x558700292b00 2024-09-11T00:17:07.169 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:07.164+0000 7f87157de740 10 client.26639 fetch_fsmap finished waiting for FSMap version 41 2024-09-11T00:17:07.169 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:07.164+0000 7f87157de740 10 client.26639 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T00:17:07.169 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:07.164+0000 7f87157de740 20 client.26639 populate_metadata read hostname 'smithi012' 2024-09-11T00:17:07.169 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:07.164+0000 7f87157de740 1 --2- 172.21.15.12:0/321404604 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x558700310360 0x558700312780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:17:07.170 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:07.164+0000 7f87157de740 4 client.26639 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T00:17:07.170 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:07.164+0000 7f87157de740 1 -- 172.21.15.12:0/321404604 --> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5587001fe0b0 con 0x558700310360 2024-09-11T00:17:07.170 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:07.165+0000 7f870aa00640 1 --2- 172.21.15.12:0/321404604 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x558700310360 0x558700312780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:17:07.170 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:07.165+0000 7f870aa00640 1 --2- 172.21.15.12:0/321404604 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x558700310360 0x558700312780 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8f46ab35940c3c79 server_cookie=150d4bade392418b in_seq=0 out_seq=0 2024-09-11T00:17:07.170 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:07.165+0000 7f86f5600640 10 client.26639 ms_handle_connect on v2:172.21.15.165:6838/501312984 2024-09-11T00:17:07.186 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:07.185+0000 7f86f5600640 1 -- 172.21.15.12:0/321404604 <== mds.0 v2:172.21.15.165:6838/501312984 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5587001fe0b0 con 0x558700310360 2024-09-11T00:17:07.186 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:07.185+0000 7f86f5600640 10 client.26639 handle_command_reply: tid=0 2024-09-11T00:17:07.186 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:07.185+0000 7f87157de740 1 client.26639 shutdown 2024-09-11T00:17:07.187 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:07.185+0000 7f87157de740 1 -- 172.21.15.12:0/321404604 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558700292b00 msgr2=0x55870029e570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:17:07.187 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:07.185+0000 7f87157de740 1 --2- 172.21.15.12:0/321404604 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558700292b00 0x55870029e570 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f8704011e50 tx=0x7f87040096c0 comp rx=0 tx=0).stop 2024-09-11T00:17:07.187 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:07.185+0000 7f87157de740 1 -- 172.21.15.12:0/321404604 shutdown_connections 2024-09-11T00:17:07.187 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:07.185+0000 7f87157de740 1 --2- 172.21.15.12:0/321404604 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x558700310360 0x558700312780 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:17:07.187 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:07.185+0000 7f87157de740 1 --2- 172.21.15.12:0/321404604 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558700292b00 0x55870029e570 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:17:07.187 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:07.185+0000 7f87157de740 1 --2- 172.21.15.12:0/321404604 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5587002900c0 0x55870029e030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:17:07.187 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:07.185+0000 7f86f5600640 0 client.26639 ms_handle_reset on v2:172.21.15.165:6838/501312984 2024-09-11T00:17:07.187 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:07.185+0000 7f87157de740 1 --2- 172.21.15.12:0/321404604 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55870028d790 0x55870029daf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:17:07.187 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:07.185+0000 7f87157de740 1 -- 172.21.15.12:0/321404604 >> 172.21.15.12:0/321404604 conn(0x5587002888e0 msgr2=0x558700289140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:17:07.187 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:07.185+0000 7f87157de740 1 -- 172.21.15.12:0/321404604 shutdown_connections 2024-09-11T00:17:07.187 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:07.185+0000 7f87157de740 1 -- 172.21.15.12:0/321404604 wait complete. 2024-09-11T00:17:07.187 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:07.185+0000 7f87157de740 20 client.26639 trim_cache size 0 max 16384 2024-09-11T00:17:07.188 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:{ 2024-09-11T00:17:07.188 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "message": "", 2024-09-11T00:17:07.188 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "return_code": 0 2024-09-11T00:17:07.188 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:} 2024-09-11T00:17:07.189 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:07.186+0000 7f8712800640 1 -- 172.21.15.12:0/151184904 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f86f00b5360 msgr2=0x7f86f00b7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:17:07.189 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:07.186+0000 7f8712800640 1 --2- 172.21.15.12:0/151184904 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f86f00b5360 0x7f86f00b7820 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f86fc00c6a0 tx=0x7f86fc004a90 comp rx=0 tx=0).stop 2024-09-11T00:17:07.189 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:07.186+0000 7f8712800640 1 -- 172.21.15.12:0/151184904 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f870c104b90 msgr2=0x7f870c19afb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:17:07.189 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:07.186+0000 7f8712800640 1 --2- 172.21.15.12:0/151184904 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f870c104b90 0x7f870c19afb0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f86fc00e9f0 tx=0x7f86fc00edf0 comp rx=0 tx=0).stop 2024-09-11T00:17:07.189 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:07.186+0000 7f8712800640 1 -- 172.21.15.12:0/151184904 shutdown_connections 2024-09-11T00:17:07.189 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:07.186+0000 7f8712800640 1 --2- 172.21.15.12:0/151184904 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f86f00b5360 0x7f86f00b7820 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:17:07.189 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:07.186+0000 7f8712800640 1 --2- 172.21.15.12:0/151184904 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f870c1066a0 0x7f870c19c230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:17:07.189 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:07.186+0000 7f8712800640 1 --2- 172.21.15.12:0/151184904 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f870c105ce0 0x7f870c19b600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:17:07.189 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:07.186+0000 7f8712800640 1 --2- 172.21.15.12:0/151184904 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f870c104b90 0x7f870c19afb0 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:17:07.189 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:07.186+0000 7f8712800640 1 -- 172.21.15.12:0/151184904 >> 172.21.15.12:0/151184904 conn(0x7f870c100470 msgr2=0x7f870c1090b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:17:07.189 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:07.186+0000 7f8712800640 1 -- 172.21.15.12:0/151184904 shutdown_connections 2024-09-11T00:17:07.189 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:07.186+0000 7f8712800640 1 -- 172.21.15.12:0/151184904 wait complete. 2024-09-11T00:17:07.685 INFO:tasks.workunit.client.0.smithi012.stdout: 1 393902 3.78 MB/sec execute 424 sec latency 27851.851 ms 2024-09-11T00:17:08.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:17:07 smithi165 ceph-mon[35356]: pgmap v700: 129 pgs: 129 active+clean; 92 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 1.2 MiB/s wr, 83 op/s 2024-09-11T00:17:08.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:17:07 smithi012 ceph-mon[30333]: pgmap v700: 129 pgs: 129 active+clean; 92 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 1.2 MiB/s wr, 83 op/s 2024-09-11T00:17:08.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:17:07 smithi178 ceph-mon[35919]: pgmap v700: 129 pgs: 129 active+clean; 92 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 1.2 MiB/s wr, 83 op/s 2024-09-11T00:17:08.685 INFO:tasks.workunit.client.0.smithi012.stdout: 1 393902 3.77 MB/sec execute 425 sec latency 28851.914 ms 2024-09-11T00:17:09.685 INFO:tasks.workunit.client.0.smithi012.stdout: 1 393902 3.77 MB/sec execute 426 sec latency 29852.028 ms 2024-09-11T00:17:09.861 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid ff3ad74a-6fcf-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T00:17:10.258 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:17:10.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:17:09 smithi165 ceph-mon[35356]: pgmap v701: 129 pgs: 129 active+clean; 92 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 832 KiB/s wr, 53 op/s 2024-09-11T00:17:10.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:17:09 smithi012 ceph-mon[30333]: pgmap v701: 129 pgs: 129 active+clean; 92 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 832 KiB/s wr, 53 op/s 2024-09-11T00:17:10.330 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:17:09 smithi178 ceph-mon[35919]: pgmap v701: 129 pgs: 129 active+clean; 92 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 832 KiB/s wr, 53 op/s 2024-09-11T00:17:10.685 INFO:tasks.workunit.client.0.smithi012.stdout: 1 393902 3.76 MB/sec execute 427 sec latency 30852.073 ms 2024-09-11T00:17:10.816 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:17:10.795+0000 7fca51800640 1 -- 172.21.15.12:0/1910650423 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fca4c104fb0 msgr2=0x7fca4c105390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:17:10.816 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:17:10.795+0000 7fca51800640 1 --2- 172.21.15.12:0/1910650423 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fca4c104fb0 0x7fca4c105390 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7fca3c009900 tx=0x7fca3c02a410 comp rx=0 tx=0).stop 2024-09-11T00:17:10.817 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:17:10.795+0000 7fca51800640 1 -- 172.21.15.12:0/1910650423 shutdown_connections 2024-09-11T00:17:10.817 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:17:10.795+0000 7fca51800640 1 --2- 172.21.15.12:0/1910650423 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fca4c10b340 0x7fca4c111b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:17:10.817 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:17:10.795+0000 7fca51800640 1 --2- 172.21.15.12:0/1910650423 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fca4c10a9c0 0x7fca4c10ae00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:17:10.817 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:17:10.795+0000 7fca51800640 1 --2- 172.21.15.12:0/1910650423 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fca4c104fb0 0x7fca4c105390 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:17:10.817 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:17:10.795+0000 7fca51800640 1 -- 172.21.15.12:0/1910650423 >> 172.21.15.12:0/1910650423 conn(0x7fca4c100a20 msgr2=0x7fca4c102e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:17:10.817 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:17:10.796+0000 7fca51800640 1 -- 172.21.15.12:0/1910650423 shutdown_connections 2024-09-11T00:17:10.817 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:17:10.796+0000 7fca51800640 1 -- 172.21.15.12:0/1910650423 wait complete. 2024-09-11T00:17:10.817 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:17:10.796+0000 7fca51800640 1 Processor -- start 2024-09-11T00:17:10.817 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:17:10.796+0000 7fca51800640 1 -- start start 2024-09-11T00:17:10.817 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:17:10.797+0000 7fca51800640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fca4c104fb0 0x7fca4c19b450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:17:10.817 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:17:10.797+0000 7fca51800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fca4c10a9c0 0x7fca4c19b990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:17:10.818 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:17:10.797+0000 7fca51800640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fca4c10b340 0x7fca4c1a0d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:17:10.818 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:17:10.797+0000 7fca51800640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fca4c114fa0 con 0x7fca4c10b340 2024-09-11T00:17:10.818 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:17:10.797+0000 7fca51800640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fca4c114e20 con 0x7fca4c10a9c0 2024-09-11T00:17:10.818 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:17:10.797+0000 7fca51800640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fca4c115120 con 0x7fca4c104fb0 2024-09-11T00:17:10.818 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:17:10.797+0000 7fca4a000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fca4c10a9c0 0x7fca4c19b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:17:10.818 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:17:10.797+0000 7fca4b400640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fca4c10b340 0x7fca4c1a0d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:17:10.818 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:17:10.797+0000 7fca4a000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fca4c10a9c0 0x7fca4c19b990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.12:46504/0 (socket says 172.21.15.12:46504) 2024-09-11T00:17:10.818 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:17:10.797+0000 7fca4a000640 1 -- 172.21.15.12:0/4164743014 learned_addr learned my addr 172.21.15.12:0/4164743014 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:17:10.818 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:17:10.797+0000 7fca4aa00640 1 --2- 172.21.15.12:0/4164743014 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fca4c104fb0 0x7fca4c19b450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:17:10.818 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:17:10.798+0000 7fca4a000640 1 -- 172.21.15.12:0/4164743014 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fca4c104fb0 msgr2=0x7fca4c19b450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:17:10.818 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:17:10.798+0000 7fca4a000640 1 --2- 172.21.15.12:0/4164743014 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fca4c104fb0 0x7fca4c19b450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:17:10.818 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:17:10.798+0000 7fca4a000640 1 -- 172.21.15.12:0/4164743014 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fca4c10b340 msgr2=0x7fca4c1a0d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:17:10.818 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:17:10.798+0000 7fca4a000640 1 --2- 172.21.15.12:0/4164743014 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fca4c10b340 0x7fca4c1a0d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:17:10.819 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:17:10.798+0000 7fca4a000640 1 -- 172.21.15.12:0/4164743014 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca4c1a1500 con 0x7fca4c10a9c0 2024-09-11T00:17:10.819 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:17:10.798+0000 7fca4a000640 1 --2- 172.21.15.12:0/4164743014 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fca4c10a9c0 0x7fca4c19b990 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fca3800d950 tx=0x7fca3800de10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:17:10.819 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:17:10.798+0000 7fca43400640 1 -- 172.21.15.12:0/4164743014 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca38017070 con 0x7fca4c10a9c0 2024-09-11T00:17:10.819 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:17:10.798+0000 7fca43400640 1 -- 172.21.15.12:0/4164743014 <== mon.2 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fca38009c30 con 0x7fca4c10a9c0 2024-09-11T00:17:10.819 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:17:10.798+0000 7fca43400640 1 -- 172.21.15.12:0/4164743014 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca3800f620 con 0x7fca4c10a9c0 2024-09-11T00:17:10.819 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:17:10.798+0000 7fca51800640 1 -- 172.21.15.12:0/4164743014 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fca4c1a17f0 con 0x7fca4c10a9c0 2024-09-11T00:17:10.819 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:17:10.799+0000 7fca51800640 1 -- 172.21.15.12:0/4164743014 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fca4c1a7eb0 con 0x7fca4c10a9c0 2024-09-11T00:17:10.819 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:17:10.800+0000 7fca51800640 1 -- 172.21.15.12:0/4164743014 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fca4c105390 con 0x7fca4c10a9c0 2024-09-11T00:17:10.819 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:17:10.803+0000 7fca43400640 1 -- 172.21.15.12:0/4164743014 <== mon.2 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7fca38010460 con 0x7fca4c10a9c0 2024-09-11T00:17:10.819 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:17:10.804+0000 7fca43400640 1 --2- 172.21.15.12:0/4164743014 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fca300b5360 0x7fca300b7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:17:10.819 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:17:10.804+0000 7fca43400640 1 -- 172.21.15.12:0/4164743014 <== mon.2 v2:172.21.15.165:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9208+0+0 (secure 0 0 0) 0x7fca380e5b10 con 0x7fca4c10a9c0 2024-09-11T00:17:10.819 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:17:10.805+0000 7fca4aa00640 1 --2- 172.21.15.12:0/4164743014 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fca300b5360 0x7fca300b7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:17:10.820 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:17:10.805+0000 7fca4aa00640 1 --2- 172.21.15.12:0/4164743014 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fca300b5360 0x7fca300b7820 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fca3c005ab0 tx=0x7fca3c005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:17:10.820 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:17:10.805+0000 7fca43400640 1 -- 172.21.15.12:0/4164743014 <== mon.2 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7fca38009de0 con 0x7fca4c10a9c0 2024-09-11T00:17:11.094 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:17:11.092+0000 7fca51800640 1 -- 172.21.15.12:0/4164743014 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fca4c19c580 con 0x7fca4c10a9c0 2024-09-11T00:17:11.094 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:17:11.093+0000 7fca43400640 1 -- 172.21.15.12:0/4164743014 <== mon.2 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 41 v41) ==== 94+0+11965 (secure 0 0 0) 0x7fca4c19c580 con 0x7fca4c10a9c0 2024-09-11T00:17:11.095 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-11T00:17:11.095 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":41,"btime":"2024-09-11T00:14:31:349726+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14577,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6835/447171591","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":447171591},{"type":"v1","addr":"172.21.15.12:6835","nonce":447171591}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14589,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/901486040","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":901486040},{"type":"v1","addr":"172.21.15.12:6837","nonce":901486040}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":41,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T00:04:04.739881+0000","modified":"2024-09-11T00:14:30.594242+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24674,"mds_1":24643,"mds_2":24638,"mds_3":24637,"mds_4":24614},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14631":{"gid":14631,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.12:6841/3069919437","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6840","nonce":3069919437},{"type":"v1","addr":"172.21.15.12:6841","nonce":3069919437}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24562":{"gid":24562,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6833/1141636112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":1141636112},{"type":"v1","addr":"172.21.15.178:6833","nonce":1141636112}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24581":{"gid":24581,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6833/3111980970","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6832","nonce":3111980970},{"type":"v1","addr":"172.21.15.165:6833","nonce":3111980970}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24607":{"gid":24607,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6835/4084868287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":4084868287},{"type":"v1","addr":"172.21.15.178:6835","nonce":4084868287}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24614":{"gid":24614,"name":"e","rank":4,"incarnation":26,"state":"up:active","state_seq":21,"addr":"172.21.15.165:6835/2425395274","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":2425395274},{"type":"v1","addr":"172.21.15.165:6835","nonce":2425395274}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24637":{"gid":24637,"name":"i","rank":3,"incarnation":24,"state":"up:active","state_seq":18,"addr":"172.21.15.178:6839/70628097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6838","nonce":70628097},{"type":"v1","addr":"172.21.15.178:6839","nonce":70628097}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24638":{"gid":24638,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":18,"addr":"172.21.15.165:6837/6215088","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6836","nonce":6215088},{"type":"v1","addr":"172.21.15.165:6837","nonce":6215088}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24643":{"gid":24643,"name":"g","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.12:6839/3124093009","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6838","nonce":3124093009},{"type":"v1","addr":"172.21.15.12:6839","nonce":3124093009}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24674":{"gid":24674,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.165:6839/501312984","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6838","nonce":501312984},{"type":"v1","addr":"172.21.15.165:6839","nonce":501312984}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25672":{"gid":25672,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6837/308601741","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":308601741},{"type":"v1","addr":"172.21.15.178:6837","nonce":308601741}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24674,"qdb_cluster":[24614,24637,24638,24643,24674]},"id":1}]} 2024-09-11T00:17:11.095 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 41 2024-09-11T00:17:11.097 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:17:11.095+0000 7fca51800640 1 -- 172.21.15.12:0/4164743014 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fca300b5360 msgr2=0x7fca300b7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:17:11.097 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:17:11.095+0000 7fca51800640 1 --2- 172.21.15.12:0/4164743014 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fca300b5360 0x7fca300b7820 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fca3c005ab0 tx=0x7fca3c005a20 comp rx=0 tx=0).stop 2024-09-11T00:17:11.097 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:17:11.095+0000 7fca51800640 1 -- 172.21.15.12:0/4164743014 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fca4c10a9c0 msgr2=0x7fca4c19b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:17:11.097 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:17:11.095+0000 7fca51800640 1 --2- 172.21.15.12:0/4164743014 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fca4c10a9c0 0x7fca4c19b990 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fca3800d950 tx=0x7fca3800de10 comp rx=0 tx=0).stop 2024-09-11T00:17:11.098 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:17:11.096+0000 7fca51800640 1 -- 172.21.15.12:0/4164743014 shutdown_connections 2024-09-11T00:17:11.098 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:17:11.096+0000 7fca51800640 1 --2- 172.21.15.12:0/4164743014 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fca300b5360 0x7fca300b7820 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:17:11.098 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:17:11.096+0000 7fca51800640 1 --2- 172.21.15.12:0/4164743014 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fca4c10b340 0x7fca4c1a0d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:17:11.098 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:17:11.096+0000 7fca51800640 1 --2- 172.21.15.12:0/4164743014 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fca4c10a9c0 0x7fca4c19b990 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:17:11.098 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:17:11.096+0000 7fca51800640 1 --2- 172.21.15.12:0/4164743014 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fca4c104fb0 0x7fca4c19b450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:17:11.098 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:17:11.096+0000 7fca51800640 1 -- 172.21.15.12:0/4164743014 >> 172.21.15.12:0/4164743014 conn(0x7fca4c100a20 msgr2=0x7fca4c1012c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:17:11.098 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:17:11.096+0000 7fca51800640 1 -- 172.21.15.12:0/4164743014 shutdown_connections 2024-09-11T00:17:11.098 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:17:11.096+0000 7fca51800640 1 -- 172.21.15.12:0/4164743014 wait complete. 2024-09-11T00:17:11.685 INFO:tasks.workunit.client.0.smithi012.stdout: 1 393902 3.75 MB/sec execute 428 sec latency 31852.315 ms 2024-09-11T00:17:12.063 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:17:11 smithi012 ceph-mon[30333]: pgmap v702: 129 pgs: 129 active+clean; 92 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 1.2 MiB/s wr, 70 op/s 2024-09-11T00:17:12.063 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:17:11 smithi012 ceph-mon[30333]: from='client.? 172.21.15.12:0/4164743014' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T00:17:12.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:17:11 smithi165 ceph-mon[35356]: pgmap v702: 129 pgs: 129 active+clean; 92 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 1.2 MiB/s wr, 70 op/s 2024-09-11T00:17:12.283 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:17:11 smithi165 ceph-mon[35356]: from='client.? 172.21.15.12:0/4164743014' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T00:17:12.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:17:11 smithi178 ceph-mon[35919]: pgmap v702: 129 pgs: 129 active+clean; 92 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 1.2 MiB/s wr, 70 op/s 2024-09-11T00:17:12.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:17:11 smithi178 ceph-mon[35919]: from='client.? 172.21.15.12:0/4164743014' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T00:17:12.685 INFO:tasks.workunit.client.0.smithi012.stdout: 1 393902 3.74 MB/sec execute 429 sec latency 32852.376 ms 2024-09-11T00:17:12.743 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24674 quiesce db --set-id bfba9d53 --release --await' 2024-09-11T00:17:12.744 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid ff3ad74a-6fcf-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24674 quiesce db --set-id bfba9d53 --release --await 2024-09-11T00:17:13.125 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:17:13.671 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:17:13.669+0000 7f59ebc00640 1 -- 172.21.15.12:0/2943804903 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f59e4075e00 msgr2=0x7f59e410ee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:17:13.671 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:17:13.669+0000 7f59ebc00640 1 --2- 172.21.15.12:0/2943804903 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f59e4075e00 0x7f59e410ee70 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f59d0009900 tx=0x7f59d002a3f0 comp rx=0 tx=0).stop 2024-09-11T00:17:13.671 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:17:13.669+0000 7f59ebc00640 1 -- 172.21.15.12:0/2943804903 shutdown_connections 2024-09-11T00:17:13.671 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:17:13.669+0000 7f59ebc00640 1 --2- 172.21.15.12:0/2943804903 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f59e410f3b0 0x7f59e41117a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:17:13.671 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:17:13.669+0000 7f59ebc00640 1 --2- 172.21.15.12:0/2943804903 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f59e4075e00 0x7f59e410ee70 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:17:13.671 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:17:13.669+0000 7f59ebc00640 1 --2- 172.21.15.12:0/2943804903 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f59e40754e0 0x7f59e40758c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:17:13.671 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:17:13.669+0000 7f59ebc00640 1 -- 172.21.15.12:0/2943804903 >> 172.21.15.12:0/2943804903 conn(0x7f59e40fe830 msgr2=0x7f59e4100c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:17:13.671 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:17:13.669+0000 7f59ebc00640 1 -- 172.21.15.12:0/2943804903 shutdown_connections 2024-09-11T00:17:13.671 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:17:13.669+0000 7f59ebc00640 1 -- 172.21.15.12:0/2943804903 wait complete. 2024-09-11T00:17:13.673 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:17:13.670+0000 7f59ebc00640 1 Processor -- start 2024-09-11T00:17:13.673 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:17:13.670+0000 7f59ebc00640 1 -- start start 2024-09-11T00:17:13.673 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:17:13.671+0000 7f59ebc00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f59e40754e0 0x7f59e419b510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:17:13.673 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:17:13.671+0000 7f59ebc00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f59e4075e00 0x7f59e419ba50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:17:13.673 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:17:13.671+0000 7f59ebc00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f59e410f3b0 0x7f59e41a0df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:17:13.673 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:17:13.671+0000 7f59ebc00640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f59e4114f60 con 0x7f59e410f3b0 2024-09-11T00:17:13.673 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:17:13.671+0000 7f59ebc00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f59e4114de0 con 0x7f59e4075e00 2024-09-11T00:17:13.673 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:17:13.671+0000 7f59ebc00640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f59e41150e0 con 0x7f59e40754e0 2024-09-11T00:17:13.673 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:17:13.671+0000 7f59e9c00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f59e410f3b0 0x7f59e41a0df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:17:13.674 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:17:13.671+0000 7f59e9c00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f59e410f3b0 0x7f59e41a0df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:39804/0 (socket says 172.21.15.12:39804) 2024-09-11T00:17:13.674 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:17:13.671+0000 7f59e9c00640 1 -- 172.21.15.12:0/3505034821 learned_addr learned my addr 172.21.15.12:0/3505034821 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:17:13.674 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:17:13.671+0000 7f59e9200640 1 --2- 172.21.15.12:0/3505034821 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f59e40754e0 0x7f59e419b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:17:13.674 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:17:13.671+0000 7f59e3e00640 1 --2- 172.21.15.12:0/3505034821 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f59e4075e00 0x7f59e419ba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:17:13.674 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:17:13.671+0000 7f59e9c00640 1 -- 172.21.15.12:0/3505034821 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f59e40754e0 msgr2=0x7f59e419b510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:17:13.674 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:17:13.671+0000 7f59e9c00640 1 --2- 172.21.15.12:0/3505034821 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f59e40754e0 0x7f59e419b510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:17:13.674 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:17:13.671+0000 7f59e9c00640 1 -- 172.21.15.12:0/3505034821 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f59e4075e00 msgr2=0x7f59e419ba50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:17:13.674 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:17:13.671+0000 7f59e9c00640 1 --2- 172.21.15.12:0/3505034821 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f59e4075e00 0x7f59e419ba50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:17:13.674 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:17:13.671+0000 7f59e9c00640 1 -- 172.21.15.12:0/3505034821 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59e41a15c0 con 0x7f59e410f3b0 2024-09-11T00:17:13.674 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:17:13.671+0000 7f59e9200640 1 --2- 172.21.15.12:0/3505034821 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f59e40754e0 0x7f59e419b510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:17:13.674 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:17:13.672+0000 7f59e9c00640 1 --2- 172.21.15.12:0/3505034821 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f59e410f3b0 0x7f59e41a0df0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f59dc00e9d0 tx=0x7f59dc00ee90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:17:13.675 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:17:13.672+0000 7f59e1600640 1 -- 172.21.15.12:0/3505034821 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59dc00cd80 con 0x7f59e410f3b0 2024-09-11T00:17:13.675 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:17:13.672+0000 7f59e1600640 1 -- 172.21.15.12:0/3505034821 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f59dc005020 con 0x7f59e410f3b0 2024-09-11T00:17:13.675 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:17:13.672+0000 7f59e1600640 1 -- 172.21.15.12:0/3505034821 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59dc0116d0 con 0x7f59e410f3b0 2024-09-11T00:17:13.675 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:17:13.672+0000 7f59ebc00640 1 -- 172.21.15.12:0/3505034821 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f59e41a18b0 con 0x7f59e410f3b0 2024-09-11T00:17:13.675 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:17:13.672+0000 7f59ebc00640 1 -- 172.21.15.12:0/3505034821 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59e41a7f70 con 0x7f59e410f3b0 2024-09-11T00:17:13.676 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:17:13.676+0000 7f59ee83c740 1 Processor -- start 2024-09-11T00:17:13.677 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:17:13.676+0000 7f59e1600640 1 -- 172.21.15.12:0/3505034821 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7f59dc00f040 con 0x7f59e410f3b0 2024-09-11T00:17:13.677 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:17:13.676+0000 7f59ee83c740 1 -- start start 2024-09-11T00:17:13.678 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:17:13.676+0000 7f59ee83c740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561e0f223ff0 0x561e0f2243d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:17:13.678 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:17:13.676+0000 7f59ee83c740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561e0f224910 0x561e0f230360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:17:13.678 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:17:13.676+0000 7f59ee83c740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561e0f2308a0 0x561e0f232c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:17:13.678 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:17:13.676+0000 7f59ee83c740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x561e0f19b290 con 0x561e0f2308a0 2024-09-11T00:17:13.678 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:17:13.676+0000 7f59ee83c740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x561e0f18a6c0 con 0x561e0f223ff0 2024-09-11T00:17:13.678 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:17:13.676+0000 7f59ee83c740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x561e0f18e4d0 con 0x561e0f224910 2024-09-11T00:17:13.678 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:17:13.677+0000 7f59e3e00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561e0f2308a0 0x561e0f232c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:17:13.678 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:17:13.677+0000 7f59e3e00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561e0f2308a0 0x561e0f232c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:39808/0 (socket says 172.21.15.12:39808) 2024-09-11T00:17:13.678 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:17:13.677+0000 7f59e3e00640 1 -- 172.21.15.12:0/1733692939 learned_addr learned my addr 172.21.15.12:0/1733692939 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:17:13.679 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:17:13.677+0000 7f59e9200640 1 --2- 172.21.15.12:0/1733692939 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561e0f224910 0x561e0f230360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:17:13.679 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:17:13.677+0000 7f59e3e00640 1 --2- 172.21.15.12:0/1733692939 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561e0f223ff0 0x561e0f2243d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:17:13.679 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:17:13.677+0000 7f59e3e00640 1 -- 172.21.15.12:0/1733692939 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561e0f224910 msgr2=0x561e0f230360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:17:13.680 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:17:13.677+0000 7f59e3e00640 1 --2- 172.21.15.12:0/1733692939 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561e0f224910 0x561e0f230360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:17:13.680 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:17:13.677+0000 7f59e3e00640 1 -- 172.21.15.12:0/1733692939 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561e0f223ff0 msgr2=0x561e0f2243d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:17:13.680 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:17:13.677+0000 7f59e3e00640 1 --2- 172.21.15.12:0/1733692939 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561e0f223ff0 0x561e0f2243d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:17:13.680 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:17:13.677+0000 7f59e3e00640 1 -- 172.21.15.12:0/1733692939 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561e0f117460 con 0x561e0f2308a0 2024-09-11T00:17:13.680 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:17:13.677+0000 7f59e9200640 1 --2- 172.21.15.12:0/1733692939 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561e0f224910 0x561e0f230360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:17:13.680 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:17:13.678+0000 7f59e3e00640 1 --2- 172.21.15.12:0/1733692939 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561e0f2308a0 0x561e0f232c90 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f59d00096c0 tx=0x7f59d0005b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=deea7e406e66f54a server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:17:13.680 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:17:13.678+0000 7f59e1600640 1 --2- 172.21.15.12:0/3505034821 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f59cc0b5360 0x7f59cc0b7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:17:13.680 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:17:13.678+0000 7f59e1600640 1 -- 172.21.15.12:0/3505034821 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9208+0+0 (secure 0 0 0) 0x7f59dc0e5d70 con 0x7f59e410f3b0 2024-09-11T00:17:13.680 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:17:13.678+0000 7f59d6000640 1 -- 172.21.15.12:0/1733692939 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59d0035070 con 0x561e0f2308a0 2024-09-11T00:17:13.680 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:17:13.678+0000 7f59e9200640 1 --2- 172.21.15.12:0/3505034821 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f59cc0b5360 0x7f59cc0b7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:17:13.680 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:17:13.678+0000 7f59d6000640 1 -- 172.21.15.12:0/1733692939 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f59d00035f0 con 0x561e0f2308a0 2024-09-11T00:17:13.681 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:17:13.678+0000 7f59d6000640 1 -- 172.21.15.12:0/1733692939 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59d00044e0 con 0x561e0f2308a0 2024-09-11T00:17:13.682 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:17:13.679+0000 7f59e9200640 1 --2- 172.21.15.12:0/3505034821 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f59cc0b5360 0x7f59cc0b7820 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x561e0f227170 tx=0x7f59d800b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:17:13.682 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:17:13.679+0000 7f59ee83c740 1 -- 172.21.15.12:0/1733692939 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561e0f2308a0 msgr2=0x561e0f232c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:17:13.682 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:17:13.679+0000 7f59ee83c740 1 --2- 172.21.15.12:0/1733692939 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561e0f2308a0 0x561e0f232c90 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f59d00096c0 tx=0x7f59d0005b50 comp rx=0 tx=0).stop 2024-09-11T00:17:13.682 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:17:13.679+0000 7f59ee83c740 1 -- 172.21.15.12:0/1733692939 shutdown_connections 2024-09-11T00:17:13.682 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:17:13.679+0000 7f59ee83c740 1 --2- 172.21.15.12:0/1733692939 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561e0f2308a0 0x561e0f232c90 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:17:13.683 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:17:13.679+0000 7f59ee83c740 1 --2- 172.21.15.12:0/1733692939 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561e0f224910 0x561e0f230360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:17:13.683 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:17:13.679+0000 7f59ee83c740 1 --2- 172.21.15.12:0/1733692939 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561e0f223ff0 0x561e0f2243d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:17:13.683 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:17:13.679+0000 7f59ee83c740 1 -- 172.21.15.12:0/1733692939 >> 172.21.15.12:0/1733692939 conn(0x561e0f221e20 msgr2=0x561e0f222fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:17:13.683 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:17:13.679+0000 7f59ee83c740 1 -- 172.21.15.12:0/1733692939 shutdown_connections 2024-09-11T00:17:13.683 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:17:13.679+0000 7f59ee83c740 1 -- 172.21.15.12:0/1733692939 wait complete. 2024-09-11T00:17:13.683 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:17:13.680+0000 7f59ee83c740 1 -- start start 2024-09-11T00:17:13.683 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:17:13.680+0000 7f59ee83c740 1 Processor -- start 2024-09-11T00:17:13.683 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:17:13.680+0000 7f59ee83c740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561e0f223ff0 0x561e0f230360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:17:13.683 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:17:13.680+0000 7f59ee83c740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561e0f224910 0x561e0f22d7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:17:13.683 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:17:13.680+0000 7f59ee83c740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561e0f2308a0 0x561e0f22df80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:17:13.683 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:17:13.680+0000 7f59ee83c740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x561e0efebbb0 con 0x561e0f224910 2024-09-11T00:17:13.683 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:17:13.680+0000 7f59ee83c740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x561e0f1256c0 con 0x561e0f2308a0 2024-09-11T00:17:13.683 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:17:13.680+0000 7f59ee83c740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x561e0f2334f0 con 0x561e0f223ff0 2024-09-11T00:17:13.684 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:17:13.681+0000 7f59ee83c740 10 client.0 _ll_register_callbacks cb 0x561e0f140490 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:17:13.684 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:17:13.681+0000 7f59e3e00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561e0f224910 0x561e0f22d7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:17:13.684 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:17:13.681+0000 7f59e3e00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561e0f224910 0x561e0f22d7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:39818/0 (socket says 172.21.15.12:39818) 2024-09-11T00:17:13.684 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:17:13.681+0000 7f59e3e00640 1 -- 172.21.15.12:0/2634355041 learned_addr learned my addr 172.21.15.12:0/2634355041 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:17:13.684 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:17:13.681+0000 7f59e9200640 1 --2- 172.21.15.12:0/2634355041 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561e0f223ff0 0x561e0f230360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:17:13.684 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:17:13.681+0000 7f59e9c00640 1 --2- 172.21.15.12:0/2634355041 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561e0f2308a0 0x561e0f22df80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:17:13.686 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:17:13.681+0000 7f59e3e00640 1 -- 172.21.15.12:0/2634355041 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561e0f223ff0 msgr2=0x561e0f230360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:17:13.686 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:17:13.681+0000 7f59e3e00640 1 --2- 172.21.15.12:0/2634355041 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561e0f223ff0 0x561e0f230360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:17:13.686 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:17:13.681+0000 7f59e3e00640 1 -- 172.21.15.12:0/2634355041 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561e0f2308a0 msgr2=0x561e0f22df80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:17:13.686 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:17:13.681+0000 7f59e3e00640 1 --2- 172.21.15.12:0/2634355041 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561e0f2308a0 0x561e0f22df80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:17:13.686 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:17:13.681+0000 7f59e3e00640 1 -- 172.21.15.12:0/2634355041 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561e0eee8b00 con 0x561e0f224910 2024-09-11T00:17:13.686 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:17:13.682+0000 7f59e3e00640 1 --2- 172.21.15.12:0/2634355041 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561e0f224910 0x561e0f22d7f0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f59d0031a40 tx=0x7f59d0005890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9e2b6353ec9de6c server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:17:13.686 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:17:13.682+0000 7f59caa00640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-09-11T00:17:13.686 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:17:13.682+0000 7f59caa00640 1 -- 172.21.15.12:0/2634355041 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59d0035040 con 0x561e0f224910 2024-09-11T00:17:13.686 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:17:13.682+0000 7f59ee83c740 1 -- 172.21.15.12:0/2634355041 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f59d00035f0 con 0x561e0f224910 2024-09-11T00:17:13.686 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:17:13.682+0000 7f59caa00640 1 -- 172.21.15.12:0/2634355041 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9208+0+0 (secure 0 0 0) 0x7f59d0004000 con 0x561e0f224910 2024-09-11T00:17:13.686 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:17:13.682+0000 7f59caa00640 1 -- 172.21.15.12:0/2634355041 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f59d0058050 con 0x561e0f224910 2024-09-11T00:17:13.686 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:17:13.682+0000 7f59ee83c740 10 client.16659 fetch_fsmap learned FSMap version 41 2024-09-11T00:17:13.686 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:17:13.682+0000 7f59ee83c740 1 -- 172.21.15.12:0/2634355041 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x561e0ef5f960 con 0x561e0f224910 2024-09-11T00:17:13.687 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:17:13.683+0000 7f59caa00640 1 -- 172.21.15.12:0/2634355041 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f59d00035f0 con 0x561e0f224910 2024-09-11T00:17:13.687 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:17:13.683+0000 7f59caa00640 1 -- 172.21.15.12:0/2634355041 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 41) ==== 8370+0+0 (secure 0 0 0) 0x7f59d00037a0 con 0x561e0f224910 2024-09-11T00:17:13.687 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:17:13.683+0000 7f59ee83c740 10 client.16659 fetch_fsmap finished waiting for FSMap version 41 2024-09-11T00:17:13.687 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:17:13.683+0000 7f59ee83c740 10 client.16659 resolve_mds: validated gid 24674 aka daemon mds.b 2024-09-11T00:17:13.687 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:17:13.683+0000 7f59ee83c740 20 client.16659 populate_metadata read hostname 'smithi012' 2024-09-11T00:17:13.687 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:17:13.683+0000 7f59ee83c740 1 --2- 172.21.15.12:0/2634355041 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x561e0f2401e0 0x561e0f2425d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:17:13.687 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:17:13.683+0000 7f59ee83c740 4 client.16659 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T00:17:13.687 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:17:13.683+0000 7f59ee83c740 1 -- 172.21.15.12:0/2634355041 --> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561e0f19d1e0 con 0x561e0f2401e0 2024-09-11T00:17:13.687 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:17:13.683+0000 7f59e9200640 1 --2- 172.21.15.12:0/2634355041 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x561e0f2401e0 0x561e0f2425d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:17:13.687 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:17:13.684+0000 7f59e9200640 1 --2- 172.21.15.12:0/2634355041 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x561e0f2401e0 0x561e0f2425d0 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=493a55f584dacb2d server_cookie=20c4131c4e757f00 in_seq=0 out_seq=0 2024-09-11T00:17:13.687 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:17:13.684+0000 7f59caa00640 10 client.16659 ms_handle_connect on v2:172.21.15.165:6838/501312984 2024-09-11T00:17:13.688 INFO:tasks.workunit.client.0.smithi012.stdout: 1 393902 3.73 MB/sec execute 430 sec latency 33852.418 ms 2024-09-11T00:17:13.688 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:17:13.685+0000 7f59caa00640 1 -- 172.21.15.12:0/2634355041 <== mds.0 v2:172.21.15.165:6838/501312984 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x561e0f19d1e0 con 0x561e0f2401e0 2024-09-11T00:17:13.688 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:17:13.685+0000 7f59caa00640 10 client.16659 handle_command_reply: tid=0 2024-09-11T00:17:13.689 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:17:13.685+0000 7f59ee83c740 1 client.16659 shutdown 2024-09-11T00:17:13.689 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:17:13.685+0000 7f59ee83c740 1 -- 172.21.15.12:0/2634355041 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561e0f224910 msgr2=0x561e0f22d7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:17:13.689 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:17:13.685+0000 7f59ee83c740 1 --2- 172.21.15.12:0/2634355041 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561e0f224910 0x561e0f22d7f0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f59d0031a40 tx=0x7f59d0005890 comp rx=0 tx=0).stop 2024-09-11T00:17:13.689 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:17:13.685+0000 7f59ee83c740 1 -- 172.21.15.12:0/2634355041 shutdown_connections 2024-09-11T00:17:13.689 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:17:13.685+0000 7f59ee83c740 1 --2- 172.21.15.12:0/2634355041 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x561e0f2401e0 0x561e0f2425d0 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:17:13.689 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:17:13.685+0000 7f59ee83c740 1 --2- 172.21.15.12:0/2634355041 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561e0f2308a0 0x561e0f22df80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:17:13.689 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:17:13.685+0000 7f59ee83c740 1 --2- 172.21.15.12:0/2634355041 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561e0f224910 0x561e0f22d7f0 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:17:13.689 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:17:13.685+0000 7f59ee83c740 1 --2- 172.21.15.12:0/2634355041 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561e0f223ff0 0x561e0f230360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:17:13.689 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:17:13.685+0000 7f59caa00640 0 client.16659 ms_handle_reset on v2:172.21.15.165:6838/501312984 2024-09-11T00:17:13.689 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:17:13.685+0000 7f59ee83c740 1 -- 172.21.15.12:0/2634355041 >> 172.21.15.12:0/2634355041 conn(0x561e0f221e20 msgr2=0x561e0f2279c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:17:13.689 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:17:13.685+0000 7f59ee83c740 1 -- 172.21.15.12:0/2634355041 shutdown_connections 2024-09-11T00:17:13.689 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:17:13.685+0000 7f59ee83c740 1 -- 172.21.15.12:0/2634355041 wait complete. 2024-09-11T00:17:13.689 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:17:13.685+0000 7f59ee83c740 20 client.16659 trim_cache size 0 max 16384 2024-09-11T00:17:13.705 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:17:13.703+0000 7f59ee83c740 1 Processor -- start 2024-09-11T00:17:13.705 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:17:13.704+0000 7f59ee83c740 1 -- start start 2024-09-11T00:17:13.705 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:17:13.704+0000 7f59ee83c740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561e0f223ff0 0x561e0f22b2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:17:13.705 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:17:13.704+0000 7f59ee83c740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561e0f224910 0x561e0f21ca40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:17:13.705 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:17:13.704+0000 7f59ee83c740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561e0f2308a0 0x561e0f2455f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:17:13.705 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:17:13.704+0000 7f59ee83c740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f59e41b1dd0 con 0x561e0f2308a0 2024-09-11T00:17:13.705 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:17:13.704+0000 7f59ee83c740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f59e41b3910 con 0x561e0f224910 2024-09-11T00:17:13.705 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:17:13.704+0000 7f59ee83c740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f59e41b24a0 con 0x561e0f223ff0 2024-09-11T00:17:13.706 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:17:13.704+0000 7f59e9c00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561e0f2308a0 0x561e0f2455f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:17:13.706 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:17:13.704+0000 7f59e9c00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561e0f2308a0 0x561e0f2455f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:39832/0 (socket says 172.21.15.12:39832) 2024-09-11T00:17:13.707 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:17:13.704+0000 7f59e9c00640 1 -- 172.21.15.12:0/209702762 learned_addr learned my addr 172.21.15.12:0/209702762 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:17:13.707 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:17:13.704+0000 7f59e9200640 1 --2- 172.21.15.12:0/209702762 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561e0f223ff0 0x561e0f22b2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:17:13.707 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:17:13.704+0000 7f59e3e00640 1 --2- 172.21.15.12:0/209702762 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561e0f224910 0x561e0f21ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:17:13.707 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:17:13.705+0000 7f59e9200640 1 -- 172.21.15.12:0/209702762 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561e0f224910 msgr2=0x561e0f21ca40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:17:13.707 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:17:13.705+0000 7f59e9200640 1 --2- 172.21.15.12:0/209702762 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561e0f224910 0x561e0f21ca40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:17:13.707 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:17:13.705+0000 7f59e9200640 1 -- 172.21.15.12:0/209702762 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561e0f2308a0 msgr2=0x561e0f2455f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:17:13.707 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:17:13.705+0000 7f59e9200640 1 --2- 172.21.15.12:0/209702762 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561e0f2308a0 0x561e0f2455f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:17:13.707 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:17:13.705+0000 7f59e9200640 1 -- 172.21.15.12:0/209702762 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561e0f223610 con 0x561e0f223ff0 2024-09-11T00:17:13.707 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:17:13.705+0000 7f59e3e00640 1 --2- 172.21.15.12:0/209702762 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561e0f224910 0x561e0f21ca40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:17:13.707 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:17:13.705+0000 7f59e9c00640 1 --2- 172.21.15.12:0/209702762 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561e0f2308a0 0x561e0f2455f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:17:13.707 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:17:13.705+0000 7f59e9200640 1 --2- 172.21.15.12:0/209702762 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561e0f223ff0 0x561e0f22b2e0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x561e0f19b350 tx=0x7f59d8019000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b202b8e35968a00c server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:17:13.707 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:17:13.705+0000 7f59d5600640 1 -- 172.21.15.12:0/209702762 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59d8004020 con 0x561e0f223ff0 2024-09-11T00:17:13.708 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:17:13.705+0000 7f59d5600640 1 -- 172.21.15.12:0/209702762 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f59d80041c0 con 0x561e0f223ff0 2024-09-11T00:17:13.708 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:17:13.705+0000 7f59d5600640 1 -- 172.21.15.12:0/209702762 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59d801e890 con 0x561e0f223ff0 2024-09-11T00:17:13.710 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:17:13.706+0000 7f59ee83c740 1 -- 172.21.15.12:0/209702762 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561e0f223ff0 msgr2=0x561e0f22b2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:17:13.710 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:17:13.706+0000 7f59ee83c740 1 --2- 172.21.15.12:0/209702762 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561e0f223ff0 0x561e0f22b2e0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x561e0f19b350 tx=0x7f59d8019000 comp rx=0 tx=0).stop 2024-09-11T00:17:13.710 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:17:13.706+0000 7f59ee83c740 1 -- 172.21.15.12:0/209702762 shutdown_connections 2024-09-11T00:17:13.710 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:17:13.706+0000 7f59ee83c740 1 --2- 172.21.15.12:0/209702762 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561e0f2308a0 0x561e0f2455f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:17:13.710 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:17:13.706+0000 7f59ee83c740 1 --2- 172.21.15.12:0/209702762 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561e0f224910 0x561e0f21ca40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:17:13.710 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:17:13.706+0000 7f59ee83c740 1 --2- 172.21.15.12:0/209702762 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561e0f223ff0 0x561e0f22b2e0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:17:13.710 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:17:13.706+0000 7f59ee83c740 1 -- 172.21.15.12:0/209702762 >> 172.21.15.12:0/209702762 conn(0x561e0f221e20 msgr2=0x561e0f22abd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:17:13.710 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:17:13.706+0000 7f59ee83c740 1 -- 172.21.15.12:0/209702762 shutdown_connections 2024-09-11T00:17:13.711 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:17:13.706+0000 7f59ee83c740 1 -- 172.21.15.12:0/209702762 wait complete. 2024-09-11T00:17:13.711 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:17:13.706+0000 7f59ee83c740 1 -- start start 2024-09-11T00:17:13.711 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:17:13.707+0000 7f59ee83c740 1 Processor -- start 2024-09-11T00:17:13.711 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:17:13.707+0000 7f59ee83c740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561e0f223ff0 0x561e0f245aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:17:13.711 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:17:13.707+0000 7f59ee83c740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561e0f224910 0x561e0f246020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:17:13.711 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:17:13.707+0000 7f59ee83c740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561e0f2308a0 0x561e0f2421b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:17:13.711 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:17:13.707+0000 7f59ee83c740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f59e41b3240 con 0x561e0f2308a0 2024-09-11T00:17:13.711 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:17:13.707+0000 7f59ee83c740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f59e41b2b70 con 0x561e0f223ff0 2024-09-11T00:17:13.711 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:17:13.707+0000 7f59ee83c740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f59e41afdb0 con 0x561e0f224910 2024-09-11T00:17:13.711 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:17:13.707+0000 7f59ee83c740 10 client.0 _ll_register_callbacks cb 0x7f59d0032d50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:17:13.711 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:17:13.707+0000 7f59e9c00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561e0f2308a0 0x561e0f2421b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:17:13.711 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:17:13.707+0000 7f59e9200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561e0f223ff0 0x561e0f245aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:17:13.711 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:17:13.708+0000 7f59e9c00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561e0f2308a0 0x561e0f2421b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:39836/0 (socket says 172.21.15.12:39836) 2024-09-11T00:17:13.712 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:17:13.708+0000 7f59e9c00640 1 -- 172.21.15.12:0/1148877407 learned_addr learned my addr 172.21.15.12:0/1148877407 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:17:13.712 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:17:13.708+0000 7f59e3e00640 1 --2- 172.21.15.12:0/1148877407 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561e0f224910 0x561e0f246020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:17:13.712 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:17:13.708+0000 7f59e3e00640 1 -- 172.21.15.12:0/1148877407 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561e0f223ff0 msgr2=0x561e0f245aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:17:13.712 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:17:13.708+0000 7f59e3e00640 1 --2- 172.21.15.12:0/1148877407 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561e0f223ff0 0x561e0f245aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:17:13.712 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:17:13.708+0000 7f59e3e00640 1 -- 172.21.15.12:0/1148877407 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561e0f2308a0 msgr2=0x561e0f2421b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:17:13.712 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:17:13.708+0000 7f59e3e00640 1 --2- 172.21.15.12:0/1148877407 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561e0f2308a0 0x561e0f2421b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:17:13.712 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:17:13.708+0000 7f59e3e00640 1 -- 172.21.15.12:0/1148877407 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561e0f22e2f0 con 0x561e0f224910 2024-09-11T00:17:13.712 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:17:13.708+0000 7f59e9c00640 1 --2- 172.21.15.12:0/1148877407 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561e0f2308a0 0x561e0f2421b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:17:13.712 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:17:13.708+0000 7f59e3e00640 1 --2- 172.21.15.12:0/1148877407 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561e0f224910 0x561e0f246020 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f59d0009850 tx=0x7f59d0003820 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bb8a998622a301f1 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:17:13.712 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:17:13.708+0000 7f59caa00640 10 client.0 ms_handle_connect on v2:172.21.15.178:3300/0 2024-09-11T00:17:13.712 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:17:13.708+0000 7f59caa00640 1 -- 172.21.15.12:0/1148877407 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59d0035040 con 0x561e0f224910 2024-09-11T00:17:13.712 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:17:13.709+0000 7f59ee83c740 1 -- 172.21.15.12:0/1148877407 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f59d80041c0 con 0x561e0f224910 2024-09-11T00:17:13.713 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:17:13.709+0000 7f59caa00640 1 -- 172.21.15.12:0/1148877407 <== mon.1 v2:172.21.15.178:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9208+0+0 (secure 0 0 0) 0x7f59d0004310 con 0x561e0f224910 2024-09-11T00:17:13.713 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:17:13.709+0000 7f59caa00640 1 -- 172.21.15.12:0/1148877407 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f59d006a050 con 0x561e0f224910 2024-09-11T00:17:13.713 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:17:13.709+0000 7f59ee83c740 10 client.26644 fetch_fsmap learned FSMap version 41 2024-09-11T00:17:13.713 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:17:13.709+0000 7f59ee83c740 1 -- 172.21.15.12:0/1148877407 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x561e0f2ad910 con 0x561e0f224910 2024-09-11T00:17:13.713 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:17:13.709+0000 7f59caa00640 1 -- 172.21.15.12:0/1148877407 <== mon.1 v2:172.21.15.178:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f59d80041c0 con 0x561e0f224910 2024-09-11T00:17:13.714 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:17:13.709+0000 7f59caa00640 1 -- 172.21.15.12:0/1148877407 <== mon.1 v2:172.21.15.178:3300/0 5 ==== fsmap(e 41) ==== 8370+0+0 (secure 0 0 0) 0x7f59d00046c0 con 0x561e0f224910 2024-09-11T00:17:13.714 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:17:13.709+0000 7f59ee83c740 10 client.26644 fetch_fsmap finished waiting for FSMap version 41 2024-09-11T00:17:13.714 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:17:13.709+0000 7f59ee83c740 10 client.26644 resolve_mds: validated gid 24674 aka daemon mds.b 2024-09-11T00:17:13.714 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:17:13.709+0000 7f59ee83c740 20 client.26644 populate_metadata read hostname 'smithi012' 2024-09-11T00:17:13.714 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:17:13.709+0000 7f59ee83c740 1 --2- 172.21.15.12:0/1148877407 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x561e0f2ae080 0x561e0f2b04a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:17:13.714 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:17:13.709+0000 7f59ee83c740 4 client.26644 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "bfba9d53", "release": true, "await": true}] 2024-09-11T00:17:13.714 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:17:13.709+0000 7f59ee83c740 1 -- 172.21.15.12:0/1148877407 --> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] -- command(tid 0: {"prefix": "quiesce db", "set_id": "bfba9d53", "release": true, "await": true}) -- 0x561e0f198c90 con 0x561e0f2ae080 2024-09-11T00:17:13.714 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:17:13.710+0000 7f59e9200640 1 --2- 172.21.15.12:0/1148877407 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x561e0f2ae080 0x561e0f2b04a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:17:13.714 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:17:13.710+0000 7f59e9200640 1 --2- 172.21.15.12:0/1148877407 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x561e0f2ae080 0x561e0f2b04a0 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=71fcb8aec590acc1 server_cookie=6588c27a550e55a8 in_seq=0 out_seq=0 2024-09-11T00:17:13.714 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:17:13.710+0000 7f59caa00640 10 client.26644 ms_handle_connect on v2:172.21.15.165:6838/501312984 2024-09-11T00:17:13.718 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:17:13.717+0000 7f59caa00640 1 -- 172.21.15.12:0/1148877407 <== mds.0 v2:172.21.15.165:6838/501312984 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x561e0f198c90 con 0x561e0f2ae080 2024-09-11T00:17:13.718 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:17:13.717+0000 7f59caa00640 10 client.26644 handle_command_reply: tid=0 2024-09-11T00:17:13.718 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:17:13.717+0000 7f59ee83c740 1 client.26644 shutdown 2024-09-11T00:17:13.718 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:17:13.717+0000 7f59ee83c740 1 -- 172.21.15.12:0/1148877407 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561e0f224910 msgr2=0x561e0f246020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:17:13.718 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:17:13.717+0000 7f59ee83c740 1 --2- 172.21.15.12:0/1148877407 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561e0f224910 0x561e0f246020 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f59d0009850 tx=0x7f59d0003820 comp rx=0 tx=0).stop 2024-09-11T00:17:13.719 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:17:13.717+0000 7f59ee83c740 1 -- 172.21.15.12:0/1148877407 shutdown_connections 2024-09-11T00:17:13.719 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:17:13.717+0000 7f59ee83c740 1 --2- 172.21.15.12:0/1148877407 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x561e0f2ae080 0x561e0f2b04a0 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:17:13.719 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:17:13.717+0000 7f59ee83c740 1 --2- 172.21.15.12:0/1148877407 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561e0f2308a0 0x561e0f2421b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:17:13.719 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:17:13.717+0000 7f59ee83c740 1 --2- 172.21.15.12:0/1148877407 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561e0f224910 0x561e0f246020 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:17:13.720 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:17:13.717+0000 7f59ee83c740 1 --2- 172.21.15.12:0/1148877407 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561e0f223ff0 0x561e0f245aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:17:13.720 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:17:13.717+0000 7f59ee83c740 1 -- 172.21.15.12:0/1148877407 >> 172.21.15.12:0/1148877407 conn(0x561e0f221e20 msgr2=0x561e0f220590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:17:13.720 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:17:13.717+0000 7f59caa00640 0 client.26644 ms_handle_reset on v2:172.21.15.165:6838/501312984 2024-09-11T00:17:13.720 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:17:13.718+0000 7f59ee83c740 1 -- 172.21.15.12:0/1148877407 shutdown_connections 2024-09-11T00:17:13.720 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:17:13.718+0000 7f59ee83c740 1 -- 172.21.15.12:0/1148877407 wait complete. 2024-09-11T00:17:13.720 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:17:13.718+0000 7f59ee83c740 20 client.26644 trim_cache size 0 max 16384 2024-09-11T00:17:13.720 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-09-11T00:17:13.720 INFO:teuthology.orchestra.run.smithi012.stdout: "epoch": 41, 2024-09-11T00:17:13.720 INFO:teuthology.orchestra.run.smithi012.stdout: "leader": 24674, 2024-09-11T00:17:13.721 INFO:teuthology.orchestra.run.smithi012.stdout: "set_version": 32, 2024-09-11T00:17:13.721 INFO:teuthology.orchestra.run.smithi012.stdout: "sets": { 2024-09-11T00:17:13.721 INFO:teuthology.orchestra.run.smithi012.stdout: "bfba9d53": { 2024-09-11T00:17:13.721 INFO:teuthology.orchestra.run.smithi012.stdout: "version": 32, 2024-09-11T00:17:13.721 INFO:teuthology.orchestra.run.smithi012.stdout: "age_ref": 0.0, 2024-09-11T00:17:13.721 INFO:teuthology.orchestra.run.smithi012.stdout: "state": { 2024-09-11T00:17:13.721 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "RELEASED", 2024-09-11T00:17:13.721 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 0.0 2024-09-11T00:17:13.721 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:17:13.721 INFO:teuthology.orchestra.run.smithi012.stdout: "timeout": 90.0, 2024-09-11T00:17:13.721 INFO:teuthology.orchestra.run.smithi012.stdout: "expiration": 153.4, 2024-09-11T00:17:13.721 INFO:teuthology.orchestra.run.smithi012.stdout: "members": { 2024-09-11T00:17:13.721 INFO:teuthology.orchestra.run.smithi012.stdout: "file:/": { 2024-09-11T00:17:13.722 INFO:teuthology.orchestra.run.smithi012.stdout: "excluded": false, 2024-09-11T00:17:13.722 INFO:teuthology.orchestra.run.smithi012.stdout: "state": { 2024-09-11T00:17:13.722 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "RELEASED", 2024-09-11T00:17:13.722 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 0.0 2024-09-11T00:17:13.722 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:17:13.722 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:17:13.722 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:17:13.722 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:17:13.722 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:17:13.722 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-09-11T00:17:13.723 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:17:13.719+0000 7f59ebc00640 1 -- 172.21.15.12:0/3505034821 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f59cc0b5360 msgr2=0x7f59cc0b7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:17:13.723 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:17:13.719+0000 7f59ebc00640 1 --2- 172.21.15.12:0/3505034821 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f59cc0b5360 0x7f59cc0b7820 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x561e0f227170 tx=0x7f59d800b410 comp rx=0 tx=0).stop 2024-09-11T00:17:13.723 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:17:13.719+0000 7f59ebc00640 1 -- 172.21.15.12:0/3505034821 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f59e410f3b0 msgr2=0x7f59e41a0df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:17:13.723 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:17:13.719+0000 7f59ebc00640 1 --2- 172.21.15.12:0/3505034821 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f59e410f3b0 0x7f59e41a0df0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f59dc00e9d0 tx=0x7f59dc00ee90 comp rx=0 tx=0).stop 2024-09-11T00:17:13.723 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:17:13.719+0000 7f59ebc00640 1 -- 172.21.15.12:0/3505034821 shutdown_connections 2024-09-11T00:17:13.723 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:17:13.719+0000 7f59ebc00640 1 --2- 172.21.15.12:0/3505034821 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f59cc0b5360 0x7f59cc0b7820 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:17:13.723 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:17:13.719+0000 7f59ebc00640 1 --2- 172.21.15.12:0/3505034821 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f59e410f3b0 0x7f59e41a0df0 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:17:13.723 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:17:13.719+0000 7f59ebc00640 1 --2- 172.21.15.12:0/3505034821 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f59e4075e00 0x7f59e419ba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:17:13.723 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:17:13.719+0000 7f59ebc00640 1 --2- 172.21.15.12:0/3505034821 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f59e40754e0 0x7f59e419b510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:17:13.723 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:17:13.719+0000 7f59ebc00640 1 -- 172.21.15.12:0/3505034821 >> 172.21.15.12:0/3505034821 conn(0x7f59e40fe830 msgr2=0x7f59e40ff430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:17:13.723 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:17:13.719+0000 7f59ebc00640 1 -- 172.21.15.12:0/3505034821 shutdown_connections 2024-09-11T00:17:13.724 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:17:13.719+0000 7f59ebc00640 1 -- 172.21.15.12:0/3505034821 wait complete. 2024-09-11T00:17:14.234 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:17:13 smithi012 ceph-mon[30333]: pgmap v703: 129 pgs: 129 active+clean; 93 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 908 KiB/s wr, 62 op/s 2024-09-11T00:17:14.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:17:13 smithi165 ceph-mon[35356]: pgmap v703: 129 pgs: 129 active+clean; 93 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 908 KiB/s wr, 62 op/s 2024-09-11T00:17:14.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:17:13 smithi178 ceph-mon[35919]: pgmap v703: 129 pgs: 129 active+clean; 93 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 908 KiB/s wr, 62 op/s 2024-09-11T00:17:14.661 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'bfba9d53', total seconds elapsed: 38.2 2024-09-11T00:17:14.685 INFO:tasks.workunit.client.0.smithi012.stdout: 1 394680 3.73 MB/sec execute 431 sec latency 33888.688 ms 2024-09-11T00:17:15.685 INFO:tasks.workunit.client.0.smithi012.stdout: 1 396962 3.75 MB/sec execute 432 sec latency 9.983 ms 2024-09-11T00:17:16.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:17:15 smithi165 ceph-mon[35356]: pgmap v704: 129 pgs: 129 active+clean; 93 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 836 KiB/s wr, 55 op/s 2024-09-11T00:17:16.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:17:15 smithi012 ceph-mon[30333]: pgmap v704: 129 pgs: 129 active+clean; 93 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 836 KiB/s wr, 55 op/s 2024-09-11T00:17:16.330 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:17:15 smithi178 ceph-mon[35919]: pgmap v704: 129 pgs: 129 active+clean; 93 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 836 KiB/s wr, 55 op/s 2024-09-11T00:17:16.686 INFO:tasks.workunit.client.0.smithi012.stdout: 1 398122 3.75 MB/sec execute 433 sec latency 11.900 ms 2024-09-11T00:17:17.685 INFO:tasks.workunit.client.0.smithi012.stdout: 1 400322 3.77 MB/sec execute 434 sec latency 13.371 ms 2024-09-11T00:17:18.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:17:17 smithi165 ceph-mon[35356]: pgmap v705: 129 pgs: 129 active+clean; 95 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 765 KiB/s rd, 1.2 MiB/s wr, 90 op/s 2024-09-11T00:17:18.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:17:17 smithi012 ceph-mon[30333]: pgmap v705: 129 pgs: 129 active+clean; 95 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 765 KiB/s rd, 1.2 MiB/s wr, 90 op/s 2024-09-11T00:17:18.330 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:17:17 smithi178 ceph-mon[35919]: pgmap v705: 129 pgs: 129 active+clean; 95 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 765 KiB/s rd, 1.2 MiB/s wr, 90 op/s 2024-09-11T00:17:18.686 INFO:tasks.workunit.client.0.smithi012.stdout: 1 401598 3.78 MB/sec execute 435 sec latency 13.426 ms 2024-09-11T00:17:19.685 INFO:tasks.workunit.client.0.smithi012.stdout: 1 403845 3.80 MB/sec execute 436 sec latency 10.039 ms 2024-09-11T00:17:20.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:17:19 smithi165 ceph-mon[35356]: pgmap v706: 129 pgs: 129 active+clean; 94 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 784 KiB/s rd, 922 KiB/s wr, 68 op/s 2024-09-11T00:17:20.313 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:20.311+0000 7fb57fe00640 1 -- 172.21.15.12:0/3462199510 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb5801058f0 msgr2=0x7fb580107d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:17:20.313 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:20.311+0000 7fb57fe00640 1 --2- 172.21.15.12:0/3462199510 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb5801058f0 0x7fb580107d60 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fb568009900 tx=0x7fb56802a3f0 comp rx=0 tx=0).stop 2024-09-11T00:17:20.313 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:20.311+0000 7fb57fe00640 1 -- 172.21.15.12:0/3462199510 shutdown_connections 2024-09-11T00:17:20.313 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:20.311+0000 7fb57fe00640 1 --2- 172.21.15.12:0/3462199510 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb5801084a0 0x7fb58010a960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:17:20.313 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:20.311+0000 7fb57fe00640 1 --2- 172.21.15.12:0/3462199510 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb5801058f0 0x7fb580107d60 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:17:20.313 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:20.311+0000 7fb57fe00640 1 --2- 172.21.15.12:0/3462199510 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb58006b470 0x7fb580105360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:17:20.313 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:20.311+0000 7fb57fe00640 1 -- 172.21.15.12:0/3462199510 >> 172.21.15.12:0/3462199510 conn(0x7fb5800fcf30 msgr2=0x7fb5800ff370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:17:20.313 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:20.311+0000 7fb57fe00640 1 -- 172.21.15.12:0/3462199510 shutdown_connections 2024-09-11T00:17:20.314 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:20.311+0000 7fb57fe00640 1 -- 172.21.15.12:0/3462199510 wait complete. 2024-09-11T00:17:20.314 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:17:19 smithi012 ceph-mon[30333]: pgmap v706: 129 pgs: 129 active+clean; 94 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 784 KiB/s rd, 922 KiB/s wr, 68 op/s 2024-09-11T00:17:20.316 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:20.312+0000 7fb57fe00640 1 Processor -- start 2024-09-11T00:17:20.316 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:20.312+0000 7fb57fe00640 1 -- start start 2024-09-11T00:17:20.316 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:20.313+0000 7fb57fe00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb58006b470 0x7fb580102aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:17:20.316 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:20.313+0000 7fb57fe00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb5801058f0 0x7fb580101300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:17:20.316 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:20.313+0000 7fb57fe00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb5801084a0 0x7fb580101840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:17:20.316 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:20.313+0000 7fb57fe00640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fb58010e3f0 con 0x7fb58006b470 2024-09-11T00:17:20.317 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:20.313+0000 7fb57fe00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fb58010e270 con 0x7fb5801058f0 2024-09-11T00:17:20.317 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:20.313+0000 7fb57fe00640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fb58010e570 con 0x7fb5801084a0 2024-09-11T00:17:20.317 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:20.313+0000 7fb57f400640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb5801084a0 0x7fb580101840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:17:20.317 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:20.313+0000 7fb577e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb5801058f0 0x7fb580101300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:17:20.317 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:20.313+0000 7fb57ea00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb58006b470 0x7fb580102aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:17:20.317 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:20.313+0000 7fb57f400640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb5801084a0 0x7fb580101840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.12:35580/0 (socket says 172.21.15.12:35580) 2024-09-11T00:17:20.317 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:20.313+0000 7fb577e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb5801058f0 0x7fb580101300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.12:34634/0 (socket says 172.21.15.12:34634) 2024-09-11T00:17:20.317 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:20.313+0000 7fb57f400640 1 -- 172.21.15.12:0/2506867204 learned_addr learned my addr 172.21.15.12:0/2506867204 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:17:20.317 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:20.313+0000 7fb57f400640 1 -- 172.21.15.12:0/2506867204 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb5801058f0 msgr2=0x7fb580101300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:17:20.317 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:20.313+0000 7fb57f400640 1 --2- 172.21.15.12:0/2506867204 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb5801058f0 0x7fb580101300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:17:20.317 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:20.314+0000 7fb57f400640 1 -- 172.21.15.12:0/2506867204 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb58006b470 msgr2=0x7fb580102aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:17:20.318 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:20.314+0000 7fb57f400640 1 --2- 172.21.15.12:0/2506867204 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb58006b470 0x7fb580102aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:17:20.318 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:20.314+0000 7fb57f400640 1 -- 172.21.15.12:0/2506867204 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5801020b0 con 0x7fb5801084a0 2024-09-11T00:17:20.318 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:20.314+0000 7fb57f400640 1 --2- 172.21.15.12:0/2506867204 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb5801084a0 0x7fb580101840 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fb57800eb10 tx=0x7fb57800c2d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:17:20.318 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:20.314+0000 7fb577400640 1 -- 172.21.15.12:0/2506867204 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb578004b20 con 0x7fb5801084a0 2024-09-11T00:17:20.318 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:20.314+0000 7fb577400640 1 -- 172.21.15.12:0/2506867204 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb578004cc0 con 0x7fb5801084a0 2024-09-11T00:17:20.318 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:20.314+0000 7fb57fe00640 1 -- 172.21.15.12:0/2506867204 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb5801a2c20 con 0x7fb5801084a0 2024-09-11T00:17:20.318 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:20.314+0000 7fb577400640 1 -- 172.21.15.12:0/2506867204 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb578010670 con 0x7fb5801084a0 2024-09-11T00:17:20.318 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:20.314+0000 7fb57fe00640 1 -- 172.21.15.12:0/2506867204 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5801a3190 con 0x7fb5801084a0 2024-09-11T00:17:20.320 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:20.317+0000 7fb5887c3740 1 Processor -- start 2024-09-11T00:17:20.320 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:20.318+0000 7fb577400640 1 -- 172.21.15.12:0/2506867204 <== mon.1 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7fb578005020 con 0x7fb5801084a0 2024-09-11T00:17:20.320 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:20.318+0000 7fb5887c3740 1 -- start start 2024-09-11T00:17:20.320 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:20.318+0000 7fb5887c3740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56360171c970 0x56360171ed60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:17:20.320 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:20.318+0000 7fb5887c3740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56360171f2a0 0x563601721690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:17:20.321 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:20.318+0000 7fb5887c3740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563601721ce0 0x5636017241a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:17:20.321 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:20.318+0000 7fb5887c3740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x56360168f6b0 con 0x56360171f2a0 2024-09-11T00:17:20.321 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:20.318+0000 7fb5887c3740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x56360167eae0 con 0x56360171c970 2024-09-11T00:17:20.321 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:20.318+0000 7fb5887c3740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5636016828f0 con 0x563601721ce0 2024-09-11T00:17:20.321 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:20.318+0000 7fb57ea00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56360171f2a0 0x563601721690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:17:20.321 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:20.318+0000 7fb57ea00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56360171f2a0 0x563601721690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:58888/0 (socket says 172.21.15.12:58888) 2024-09-11T00:17:20.321 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:20.318+0000 7fb57ea00640 1 -- 172.21.15.12:0/2643771071 learned_addr learned my addr 172.21.15.12:0/2643771071 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:17:20.321 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:20.319+0000 7fb577e00640 1 --2- 172.21.15.12:0/2643771071 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56360171c970 0x56360171ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:17:20.321 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:20.319+0000 7fb577e00640 1 --2- 172.21.15.12:0/2643771071 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563601721ce0 0x5636017241a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:17:20.321 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:20.319+0000 7fb57ea00640 1 -- 172.21.15.12:0/2643771071 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563601721ce0 msgr2=0x5636017241a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:17:20.321 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:20.319+0000 7fb57ea00640 1 --2- 172.21.15.12:0/2643771071 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563601721ce0 0x5636017241a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:17:20.322 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:20.319+0000 7fb57ea00640 1 -- 172.21.15.12:0/2643771071 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56360171c970 msgr2=0x56360171ed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:17:20.322 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:20.319+0000 7fb57ea00640 1 --2- 172.21.15.12:0/2643771071 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56360171c970 0x56360171ed60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:17:20.322 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:20.319+0000 7fb57ea00640 1 -- 172.21.15.12:0/2643771071 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56360164b780 con 0x56360171f2a0 2024-09-11T00:17:20.322 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:20.319+0000 7fb577e00640 1 --2- 172.21.15.12:0/2643771071 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56360171c970 0x56360171ed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T00:17:20.326 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:20.319+0000 7fb57ea00640 1 --2- 172.21.15.12:0/2643771071 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56360171f2a0 0x563601721690 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fb570009cf0 tx=0x7fb570011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe7f37d5aa5c7d0c server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:17:20.326 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:20.320+0000 7fb577400640 1 --2- 172.21.15.12:0/2506867204 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fb5640b5360 0x7fb5640b7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:17:20.327 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:20.320+0000 7fb56fe00640 1 -- 172.21.15.12:0/2643771071 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb570004230 con 0x56360171f2a0 2024-09-11T00:17:20.327 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:20.320+0000 7fb577400640 1 -- 172.21.15.12:0/2506867204 <== mon.1 v2:172.21.15.178:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9208+0+0 (secure 0 0 0) 0x7fb5780e64e0 con 0x7fb5801084a0 2024-09-11T00:17:20.327 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:20.320+0000 7fb56fe00640 1 -- 172.21.15.12:0/2643771071 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb5700043d0 con 0x56360171f2a0 2024-09-11T00:17:20.327 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:20.320+0000 7fb56fe00640 1 -- 172.21.15.12:0/2643771071 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb570019880 con 0x56360171f2a0 2024-09-11T00:17:20.327 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:20.320+0000 7fb577e00640 1 --2- 172.21.15.12:0/2506867204 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fb5640b5360 0x7fb5640b7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:17:20.327 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:20.321+0000 7fb5887c3740 1 -- 172.21.15.12:0/2643771071 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56360171f2a0 msgr2=0x563601721690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:17:20.327 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:20.321+0000 7fb5887c3740 1 --2- 172.21.15.12:0/2643771071 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56360171f2a0 0x563601721690 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fb570009cf0 tx=0x7fb570011810 comp rx=0 tx=0).stop 2024-09-11T00:17:20.327 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:20.321+0000 7fb577e00640 1 --2- 172.21.15.12:0/2506867204 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fb5640b5360 0x7fb5640b7820 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fb568000c00 tx=0x7fb56802a3f0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:17:20.327 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:20.321+0000 7fb5887c3740 1 -- 172.21.15.12:0/2643771071 shutdown_connections 2024-09-11T00:17:20.328 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:20.321+0000 7fb5887c3740 1 --2- 172.21.15.12:0/2643771071 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563601721ce0 0x5636017241a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:17:20.328 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:20.321+0000 7fb5887c3740 1 --2- 172.21.15.12:0/2643771071 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56360171f2a0 0x563601721690 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:17:20.328 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:20.321+0000 7fb5887c3740 1 --2- 172.21.15.12:0/2643771071 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56360171c970 0x56360171ed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:17:20.328 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:20.321+0000 7fb5887c3740 1 -- 172.21.15.12:0/2643771071 >> 172.21.15.12:0/2643771071 conn(0x563601717ac0 msgr2=0x563601718850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:17:20.328 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:20.321+0000 7fb5887c3740 1 -- 172.21.15.12:0/2643771071 shutdown_connections 2024-09-11T00:17:20.328 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:20.321+0000 7fb5887c3740 1 -- 172.21.15.12:0/2643771071 wait complete. 2024-09-11T00:17:20.328 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:20.322+0000 7fb5887c3740 1 -- start start 2024-09-11T00:17:20.328 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:20.322+0000 7fb5887c3740 1 Processor -- start 2024-09-11T00:17:20.328 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:20.322+0000 7fb5887c3740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56360171c970 0x563601726070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:17:20.328 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:20.322+0000 7fb5887c3740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56360171f2a0 0x563601726710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:17:20.329 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:20.322+0000 7fb5887c3740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563601721ce0 0x5636017272e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:17:20.329 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:20.322+0000 7fb5887c3740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x5636015b6ab0 con 0x563601721ce0 2024-09-11T00:17:20.329 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:20.322+0000 7fb5887c3740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x563601625570 con 0x56360171c970 2024-09-11T00:17:20.329 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:20.322+0000 7fb5887c3740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x563601724cc0 con 0x56360171f2a0 2024-09-11T00:17:20.329 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:20.322+0000 7fb5887c3740 10 client.0 _ll_register_callbacks cb 0x563601634100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:17:20.329 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:20.323+0000 7fb57ea00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56360171c970 0x563601726070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:17:20.329 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:20.323+0000 7fb577e00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56360171f2a0 0x563601726710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:17:20.329 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:20.323+0000 7fb57ea00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56360171c970 0x563601726070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.12:34650/0 (socket says 172.21.15.12:34650) 2024-09-11T00:17:20.329 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:20.323+0000 7fb57ea00640 1 -- 172.21.15.12:0/1114993948 learned_addr learned my addr 172.21.15.12:0/1114993948 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:17:20.329 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:20.323+0000 7fb577e00640 1 --2- 172.21.15.12:0/1114993948 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56360171f2a0 0x563601726710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.12:35608/0 (socket says 172.21.15.12:35608) 2024-09-11T00:17:20.329 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:20.323+0000 7fb57f400640 1 --2- 172.21.15.12:0/1114993948 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563601721ce0 0x5636017272e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:17:20.329 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:20.323+0000 7fb57ea00640 1 -- 172.21.15.12:0/1114993948 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56360171f2a0 msgr2=0x563601726710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:17:20.329 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:20.323+0000 7fb57ea00640 1 --2- 172.21.15.12:0/1114993948 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56360171f2a0 0x563601726710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:17:20.330 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:20.323+0000 7fb57ea00640 1 -- 172.21.15.12:0/1114993948 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563601721ce0 msgr2=0x5636017272e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:17:20.330 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:20.323+0000 7fb57ea00640 1 --2- 172.21.15.12:0/1114993948 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563601721ce0 0x5636017272e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:17:20.330 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:20.323+0000 7fb57ea00640 1 -- 172.21.15.12:0/1114993948 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5636013e0790 con 0x56360171c970 2024-09-11T00:17:20.330 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:20.323+0000 7fb57ea00640 1 --2- 172.21.15.12:0/1114993948 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56360171c970 0x563601726070 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fb570000c00 tx=0x7fb570003820 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5c4e8046da56cece server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:17:20.330 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:20.323+0000 7fb56cc00640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-09-11T00:17:20.330 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:20.324+0000 7fb56cc00640 1 -- 172.21.15.12:0/1114993948 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb570027070 con 0x56360171c970 2024-09-11T00:17:20.330 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:20.324+0000 7fb5887c3740 1 -- 172.21.15.12:0/1114993948 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb5700043d0 con 0x56360171c970 2024-09-11T00:17:20.330 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:20.324+0000 7fb56cc00640 1 -- 172.21.15.12:0/1114993948 <== mon.2 v2:172.21.15.165:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9208+0+0 (secure 0 0 0) 0x7fb570005240 con 0x56360171c970 2024-09-11T00:17:20.330 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:20.324+0000 7fb56cc00640 1 -- 172.21.15.12:0/1114993948 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7fb570033050 con 0x56360171c970 2024-09-11T00:17:20.330 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:20.324+0000 7fb5887c3740 10 client.26675 fetch_fsmap learned FSMap version 41 2024-09-11T00:17:20.330 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:20.324+0000 7fb5887c3740 1 -- 172.21.15.12:0/1114993948 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x563601459620 con 0x56360171c970 2024-09-11T00:17:20.331 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:20.324+0000 7fb56cc00640 1 -- 172.21.15.12:0/1114993948 <== mon.2 v2:172.21.15.165:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb5700043d0 con 0x56360171c970 2024-09-11T00:17:20.331 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:20.324+0000 7fb56cc00640 1 -- 172.21.15.12:0/1114993948 <== mon.2 v2:172.21.15.165:3300/0 5 ==== fsmap(e 41) ==== 8370+0+0 (secure 0 0 0) 0x7fb570004c00 con 0x56360171c970 2024-09-11T00:17:20.331 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:20.325+0000 7fb5887c3740 10 client.26675 fetch_fsmap finished waiting for FSMap version 41 2024-09-11T00:17:20.331 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:20.325+0000 7fb5887c3740 10 client.26675 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T00:17:20.331 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:20.325+0000 7fb5887c3740 20 client.26675 populate_metadata read hostname 'smithi012' 2024-09-11T00:17:20.331 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:20.325+0000 7fb5887c3740 1 --2- 172.21.15.12:0/1114993948 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x563601731680 0x563601733a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:17:20.331 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:20.325+0000 7fb5887c3740 4 client.26675 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T00:17:20.331 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:20.325+0000 7fb5887c3740 1 -- 172.21.15.12:0/1114993948 --> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563601691600 con 0x563601731680 2024-09-11T00:17:20.331 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:20.325+0000 7fb577e00640 1 --2- 172.21.15.12:0/1114993948 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x563601731680 0x563601733a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:17:20.331 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:20.326+0000 7fb577e00640 1 --2- 172.21.15.12:0/1114993948 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x563601731680 0x563601733a70 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e035bb754e78c6f0 server_cookie=37a79d02dbceb58 in_seq=0 out_seq=0 2024-09-11T00:17:20.331 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:20.326+0000 7fb56cc00640 10 client.26675 ms_handle_connect on v2:172.21.15.165:6838/501312984 2024-09-11T00:17:20.332 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:17:19 smithi178 ceph-mon[35919]: pgmap v706: 129 pgs: 129 active+clean; 94 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 784 KiB/s rd, 922 KiB/s wr, 68 op/s 2024-09-11T00:17:20.333 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:20.326+0000 7fb56cc00640 1 -- 172.21.15.12:0/1114993948 <== mds.0 v2:172.21.15.165:6838/501312984 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x563601691600 con 0x563601731680 2024-09-11T00:17:20.333 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:20.326+0000 7fb56cc00640 10 client.26675 handle_command_reply: tid=0 2024-09-11T00:17:20.333 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:20.326+0000 7fb5887c3740 1 client.26675 shutdown 2024-09-11T00:17:20.333 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:20.327+0000 7fb5887c3740 1 -- 172.21.15.12:0/1114993948 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56360171c970 msgr2=0x563601726070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:17:20.333 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:20.327+0000 7fb5887c3740 1 --2- 172.21.15.12:0/1114993948 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56360171c970 0x563601726070 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fb570000c00 tx=0x7fb570003820 comp rx=0 tx=0).stop 2024-09-11T00:17:20.333 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:20.327+0000 7fb5887c3740 1 -- 172.21.15.12:0/1114993948 shutdown_connections 2024-09-11T00:17:20.333 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:20.327+0000 7fb5887c3740 1 --2- 172.21.15.12:0/1114993948 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x563601731680 0x563601733a70 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:17:20.333 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:20.327+0000 7fb5887c3740 1 --2- 172.21.15.12:0/1114993948 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563601721ce0 0x5636017272e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:17:20.333 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:20.327+0000 7fb5887c3740 1 --2- 172.21.15.12:0/1114993948 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56360171f2a0 0x563601726710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:17:20.333 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:20.327+0000 7fb5887c3740 1 --2- 172.21.15.12:0/1114993948 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56360171c970 0x563601726070 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:17:20.333 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:20.327+0000 7fb56cc00640 0 client.26675 ms_handle_reset on v2:172.21.15.165:6838/501312984 2024-09-11T00:17:20.334 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:20.327+0000 7fb5887c3740 1 -- 172.21.15.12:0/1114993948 >> 172.21.15.12:0/1114993948 conn(0x563601717ac0 msgr2=0x563601716bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:17:20.334 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:20.327+0000 7fb5887c3740 1 -- 172.21.15.12:0/1114993948 shutdown_connections 2024-09-11T00:17:20.334 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:20.327+0000 7fb5887c3740 1 -- 172.21.15.12:0/1114993948 wait complete. 2024-09-11T00:17:20.334 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:20.327+0000 7fb5887c3740 20 client.26675 trim_cache size 0 max 16384 2024-09-11T00:17:20.347 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:20.345+0000 7fb5887c3740 1 Processor -- start 2024-09-11T00:17:20.348 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:20.345+0000 7fb5887c3740 1 -- start start 2024-09-11T00:17:20.348 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:20.345+0000 7fb5887c3740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56360171c970 0x5636017220c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:17:20.348 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:20.345+0000 7fb5887c3740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56360171f2a0 0x563601722600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:17:20.348 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:20.345+0000 7fb5887c3740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563601721ce0 0x56360171a900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:17:20.348 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:20.345+0000 7fb5887c3740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fb5801ad800 con 0x563601721ce0 2024-09-11T00:17:20.348 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:20.345+0000 7fb5887c3740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fb5801af340 con 0x56360171c970 2024-09-11T00:17:20.348 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:20.345+0000 7fb5887c3740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fb5801aded0 con 0x56360171f2a0 2024-09-11T00:17:20.348 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:20.346+0000 7fb577e00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56360171f2a0 0x563601722600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:17:20.348 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:20.346+0000 7fb577e00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56360171f2a0 0x563601722600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.12:35624/0 (socket says 172.21.15.12:35624) 2024-09-11T00:17:20.349 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:20.346+0000 7fb57ea00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56360171c970 0x5636017220c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:17:20.349 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:20.346+0000 7fb577e00640 1 -- 172.21.15.12:0/709090549 learned_addr learned my addr 172.21.15.12:0/709090549 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:17:20.349 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:20.346+0000 7fb57f400640 1 --2- 172.21.15.12:0/709090549 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563601721ce0 0x56360171a900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:17:20.349 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:20.346+0000 7fb577e00640 1 -- 172.21.15.12:0/709090549 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56360171c970 msgr2=0x5636017220c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:17:20.349 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:20.346+0000 7fb577e00640 1 --2- 172.21.15.12:0/709090549 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56360171c970 0x5636017220c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:17:20.349 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:20.346+0000 7fb577e00640 1 -- 172.21.15.12:0/709090549 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563601721ce0 msgr2=0x56360171a900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:17:20.349 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:20.346+0000 7fb577e00640 1 --2- 172.21.15.12:0/709090549 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563601721ce0 0x56360171a900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:17:20.349 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:20.346+0000 7fb577e00640 1 -- 172.21.15.12:0/709090549 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563601718320 con 0x56360171f2a0 2024-09-11T00:17:20.349 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:20.346+0000 7fb577e00640 1 --2- 172.21.15.12:0/709090549 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56360171f2a0 0x563601722600 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fb5680096c0 tx=0x7fb568040000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4cb11da1cd5dcc18 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:17:20.349 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:20.346+0000 7fb56f400640 1 -- 172.21.15.12:0/709090549 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb568059070 con 0x56360171f2a0 2024-09-11T00:17:20.350 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:20.347+0000 7fb56f400640 1 -- 172.21.15.12:0/709090549 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb568045070 con 0x56360171f2a0 2024-09-11T00:17:20.350 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:20.347+0000 7fb56f400640 1 -- 172.21.15.12:0/709090549 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb568035040 con 0x56360171f2a0 2024-09-11T00:17:20.353 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:20.347+0000 7fb5887c3740 1 -- 172.21.15.12:0/709090549 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56360171f2a0 msgr2=0x563601722600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:17:20.353 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:20.347+0000 7fb5887c3740 1 --2- 172.21.15.12:0/709090549 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56360171f2a0 0x563601722600 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fb5680096c0 tx=0x7fb568040000 comp rx=0 tx=0).stop 2024-09-11T00:17:20.353 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:20.347+0000 7fb5887c3740 1 -- 172.21.15.12:0/709090549 shutdown_connections 2024-09-11T00:17:20.353 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:20.347+0000 7fb5887c3740 1 --2- 172.21.15.12:0/709090549 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563601721ce0 0x56360171a900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:17:20.354 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:20.347+0000 7fb5887c3740 1 --2- 172.21.15.12:0/709090549 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56360171f2a0 0x563601722600 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:17:20.354 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:20.347+0000 7fb5887c3740 1 --2- 172.21.15.12:0/709090549 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56360171c970 0x5636017220c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:17:20.354 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:20.347+0000 7fb5887c3740 1 -- 172.21.15.12:0/709090549 >> 172.21.15.12:0/709090549 conn(0x563601717ac0 msgr2=0x563601718510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:17:20.354 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:20.348+0000 7fb5887c3740 1 -- 172.21.15.12:0/709090549 shutdown_connections 2024-09-11T00:17:20.354 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:20.348+0000 7fb5887c3740 1 -- 172.21.15.12:0/709090549 wait complete. 2024-09-11T00:17:20.354 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:20.348+0000 7fb5887c3740 1 -- start start 2024-09-11T00:17:20.354 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:20.348+0000 7fb5887c3740 1 Processor -- start 2024-09-11T00:17:20.354 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:20.349+0000 7fb5887c3740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56360171c970 0x56360172d970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:17:20.354 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:20.349+0000 7fb5887c3740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56360171f2a0 0x56360172c670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:17:20.354 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:20.349+0000 7fb5887c3740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563601721ce0 0x56360172cbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:17:20.354 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:20.349+0000 7fb5887c3740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fb5801aec70 con 0x56360171f2a0 2024-09-11T00:17:20.354 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:20.349+0000 7fb5887c3740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fb5801ae5a0 con 0x56360171c970 2024-09-11T00:17:20.354 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:20.349+0000 7fb5887c3740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fb5801ab7e0 con 0x563601721ce0 2024-09-11T00:17:20.355 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:20.349+0000 7fb5887c3740 10 client.0 _ll_register_callbacks cb 0x7fb570009b70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:17:20.355 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:20.349+0000 7fb57ea00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56360171c970 0x56360172d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:17:20.355 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:20.349+0000 7fb57ea00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56360171c970 0x56360172d970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.12:34658/0 (socket says 172.21.15.12:34658) 2024-09-11T00:17:20.355 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:20.349+0000 7fb57ea00640 1 -- 172.21.15.12:0/2644384208 learned_addr learned my addr 172.21.15.12:0/2644384208 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:17:20.355 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:20.349+0000 7fb577e00640 1 --2- 172.21.15.12:0/2644384208 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56360171f2a0 0x56360172c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:17:20.355 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:20.349+0000 7fb57f400640 1 --2- 172.21.15.12:0/2644384208 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563601721ce0 0x56360172cbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:17:20.355 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:20.349+0000 7fb57ea00640 1 -- 172.21.15.12:0/2644384208 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563601721ce0 msgr2=0x56360172cbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:17:20.355 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:20.349+0000 7fb57ea00640 1 --2- 172.21.15.12:0/2644384208 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563601721ce0 0x56360172cbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:17:20.355 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:20.349+0000 7fb57ea00640 1 -- 172.21.15.12:0/2644384208 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56360171f2a0 msgr2=0x56360172c670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:17:20.355 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:20.349+0000 7fb57ea00640 1 --2- 172.21.15.12:0/2644384208 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56360171f2a0 0x56360172c670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:17:20.356 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:20.349+0000 7fb57ea00640 1 -- 172.21.15.12:0/2644384208 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56360172d1f0 con 0x56360171c970 2024-09-11T00:17:20.356 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:20.350+0000 7fb57ea00640 1 --2- 172.21.15.12:0/2644384208 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56360171c970 0x56360172d970 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fb570011850 tx=0x7fb570019800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9c18176472f19c35 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:17:20.356 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:20.350+0000 7fb56cc00640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-09-11T00:17:20.356 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:20.350+0000 7fb56cc00640 1 -- 172.21.15.12:0/2644384208 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb57003c060 con 0x56360171c970 2024-09-11T00:17:20.356 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:20.350+0000 7fb56cc00640 1 -- 172.21.15.12:0/2644384208 <== mon.2 v2:172.21.15.165:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9208+0+0 (secure 0 0 0) 0x7fb570003c20 con 0x56360171c970 2024-09-11T00:17:20.356 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:20.350+0000 7fb5887c3740 1 -- 172.21.15.12:0/2644384208 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb568045070 con 0x56360171c970 2024-09-11T00:17:20.356 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:20.350+0000 7fb56cc00640 1 -- 172.21.15.12:0/2644384208 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7fb570043050 con 0x56360171c970 2024-09-11T00:17:20.356 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:20.350+0000 7fb5887c3740 10 client.26681 fetch_fsmap learned FSMap version 41 2024-09-11T00:17:20.356 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:20.350+0000 7fb5887c3740 1 -- 172.21.15.12:0/2644384208 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x5636017a6d90 con 0x56360171c970 2024-09-11T00:17:20.356 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:20.350+0000 7fb56cc00640 1 -- 172.21.15.12:0/2644384208 <== mon.2 v2:172.21.15.165:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb568045070 con 0x56360171c970 2024-09-11T00:17:20.356 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:20.351+0000 7fb56cc00640 1 -- 172.21.15.12:0/2644384208 <== mon.2 v2:172.21.15.165:3300/0 5 ==== fsmap(e 41) ==== 8370+0+0 (secure 0 0 0) 0x7fb570004fa0 con 0x56360171c970 2024-09-11T00:17:20.356 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:20.351+0000 7fb5887c3740 10 client.26681 fetch_fsmap finished waiting for FSMap version 41 2024-09-11T00:17:20.356 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:20.351+0000 7fb5887c3740 10 client.26681 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T00:17:20.357 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:20.351+0000 7fb5887c3740 20 client.26681 populate_metadata read hostname 'smithi012' 2024-09-11T00:17:20.357 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:20.351+0000 7fb5887c3740 1 --2- 172.21.15.12:0/2644384208 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x5636017a7650 0x5636017a9a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:17:20.357 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:20.351+0000 7fb5887c3740 4 client.26681 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T00:17:20.357 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:20.351+0000 7fb5887c3740 1 -- 172.21.15.12:0/2644384208 --> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56360168d0b0 con 0x5636017a7650 2024-09-11T00:17:20.357 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:20.351+0000 7fb577e00640 1 --2- 172.21.15.12:0/2644384208 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x5636017a7650 0x5636017a9a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:17:20.357 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:20.352+0000 7fb577e00640 1 --2- 172.21.15.12:0/2644384208 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x5636017a7650 0x5636017a9a70 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a18a677904f786de server_cookie=8355b0e1504a964c in_seq=0 out_seq=0 2024-09-11T00:17:20.357 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:20.352+0000 7fb56cc00640 10 client.26681 ms_handle_connect on v2:172.21.15.165:6838/501312984 2024-09-11T00:17:20.397 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:20.395+0000 7fb56cc00640 1 -- 172.21.15.12:0/2644384208 <== mds.0 v2:172.21.15.165:6838/501312984 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56360168d0b0 con 0x5636017a7650 2024-09-11T00:17:20.397 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:20.395+0000 7fb56cc00640 10 client.26681 handle_command_reply: tid=0 2024-09-11T00:17:20.398 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:20.395+0000 7fb5887c3740 1 client.26681 shutdown 2024-09-11T00:17:20.398 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:20.396+0000 7fb5887c3740 1 -- 172.21.15.12:0/2644384208 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56360171c970 msgr2=0x56360172d970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:17:20.398 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:20.396+0000 7fb5887c3740 1 --2- 172.21.15.12:0/2644384208 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56360171c970 0x56360172d970 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fb570011850 tx=0x7fb570019800 comp rx=0 tx=0).stop 2024-09-11T00:17:20.398 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:20.396+0000 7fb5887c3740 1 -- 172.21.15.12:0/2644384208 shutdown_connections 2024-09-11T00:17:20.398 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:20.396+0000 7fb5887c3740 1 --2- 172.21.15.12:0/2644384208 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x5636017a7650 0x5636017a9a70 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:17:20.398 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:20.396+0000 7fb5887c3740 1 --2- 172.21.15.12:0/2644384208 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563601721ce0 0x56360172cbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:17:20.398 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:20.396+0000 7fb5887c3740 1 --2- 172.21.15.12:0/2644384208 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56360171f2a0 0x56360172c670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:17:20.398 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:20.396+0000 7fb5887c3740 1 --2- 172.21.15.12:0/2644384208 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56360171c970 0x56360172d970 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:17:20.398 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:20.396+0000 7fb5887c3740 1 -- 172.21.15.12:0/2644384208 >> 172.21.15.12:0/2644384208 conn(0x563601717ac0 msgr2=0x5636017220c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:17:20.398 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:20.396+0000 7fb56cc00640 0 client.26681 ms_handle_reset on v2:172.21.15.165:6838/501312984 2024-09-11T00:17:20.398 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:20.396+0000 7fb5887c3740 1 -- 172.21.15.12:0/2644384208 shutdown_connections 2024-09-11T00:17:20.398 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:20.396+0000 7fb5887c3740 1 -- 172.21.15.12:0/2644384208 wait complete. 2024-09-11T00:17:20.399 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:20.396+0000 7fb5887c3740 20 client.26681 trim_cache size 0 max 16384 2024-09-11T00:17:20.399 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:{ 2024-09-11T00:17:20.399 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "message": "", 2024-09-11T00:17:20.399 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "return_code": 0 2024-09-11T00:17:20.399 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:} 2024-09-11T00:17:20.400 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:20.397+0000 7fb57fe00640 1 -- 172.21.15.12:0/2506867204 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fb5640b5360 msgr2=0x7fb5640b7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:17:20.400 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:20.397+0000 7fb57fe00640 1 --2- 172.21.15.12:0/2506867204 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fb5640b5360 0x7fb5640b7820 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fb568000c00 tx=0x7fb56802a3f0 comp rx=0 tx=0).stop 2024-09-11T00:17:20.400 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:20.397+0000 7fb57fe00640 1 -- 172.21.15.12:0/2506867204 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb5801084a0 msgr2=0x7fb580101840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:17:20.400 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:20.397+0000 7fb57fe00640 1 --2- 172.21.15.12:0/2506867204 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb5801084a0 0x7fb580101840 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fb57800eb10 tx=0x7fb57800c2d0 comp rx=0 tx=0).stop 2024-09-11T00:17:20.400 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:20.398+0000 7fb57fe00640 1 -- 172.21.15.12:0/2506867204 shutdown_connections 2024-09-11T00:17:20.400 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:20.398+0000 7fb57fe00640 1 --2- 172.21.15.12:0/2506867204 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fb5640b5360 0x7fb5640b7820 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:17:20.400 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:20.398+0000 7fb57fe00640 1 --2- 172.21.15.12:0/2506867204 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb5801084a0 0x7fb580101840 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:17:20.400 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:20.398+0000 7fb57fe00640 1 --2- 172.21.15.12:0/2506867204 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb5801058f0 0x7fb580101300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:17:20.401 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:20.398+0000 7fb57fe00640 1 --2- 172.21.15.12:0/2506867204 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb58006b470 0x7fb580102aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:17:20.401 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:20.398+0000 7fb57fe00640 1 -- 172.21.15.12:0/2506867204 >> 172.21.15.12:0/2506867204 conn(0x7fb5800fcf30 msgr2=0x7fb580103b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:17:20.401 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:20.398+0000 7fb57fe00640 1 -- 172.21.15.12:0/2506867204 shutdown_connections 2024-09-11T00:17:20.401 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:20.398+0000 7fb57fe00640 1 -- 172.21.15.12:0/2506867204 wait complete. 2024-09-11T00:17:20.686 INFO:tasks.workunit.client.0.smithi012.stdout: 1 405084 3.81 MB/sec execute 437 sec latency 28.443 ms 2024-09-11T00:17:21.686 INFO:tasks.workunit.client.0.smithi012.stdout: 1 407300 3.82 MB/sec execute 438 sec latency 13.793 ms 2024-09-11T00:17:22.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:17:21 smithi165 ceph-mon[35356]: pgmap v707: 129 pgs: 129 active+clean; 95 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.3 MiB/s wr, 148 op/s 2024-09-11T00:17:22.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:17:21 smithi012 ceph-mon[30333]: pgmap v707: 129 pgs: 129 active+clean; 95 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.3 MiB/s wr, 148 op/s 2024-09-11T00:17:22.330 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:17:21 smithi178 ceph-mon[35919]: pgmap v707: 129 pgs: 129 active+clean; 95 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.3 MiB/s wr, 148 op/s 2024-09-11T00:17:22.686 INFO:tasks.workunit.client.0.smithi012.stdout: 1 408551 3.83 MB/sec execute 439 sec latency 12.847 ms 2024-09-11T00:17:23.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:17:22 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T00:17:23.312 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:17:22 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T00:17:23.330 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:17:22 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T00:17:23.686 INFO:tasks.workunit.client.0.smithi012.stdout: 1 410708 3.85 MB/sec execute 440 sec latency 11.316 ms 2024-09-11T00:17:24.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:17:23 smithi165 ceph-mon[35356]: pgmap v708: 129 pgs: 129 active+clean; 95 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 1.1 MiB/s wr, 174 op/s 2024-09-11T00:17:24.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:17:23 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:17:24.283 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:17:23 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:17:24.283 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:17:23 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:17:24.283 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:17:23 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:17:24.283 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:17:23 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:17:24.283 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:17:23 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:17:24.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:17:23 smithi012 ceph-mon[30333]: pgmap v708: 129 pgs: 129 active+clean; 95 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 1.1 MiB/s wr, 174 op/s 2024-09-11T00:17:24.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:17:23 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:17:24.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:17:23 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:17:24.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:17:23 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:17:24.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:17:23 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:17:24.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:17:23 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:17:24.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:17:23 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:17:24.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:17:23 smithi178 ceph-mon[35919]: pgmap v708: 129 pgs: 129 active+clean; 95 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 1.1 MiB/s wr, 174 op/s 2024-09-11T00:17:24.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:17:23 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:17:24.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:17:23 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:17:24.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:17:23 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:17:24.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:17:23 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:17:24.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:17:23 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:17:24.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:17:23 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:17:24.686 INFO:tasks.workunit.client.0.smithi012.stdout: 1 412153 3.86 MB/sec execute 441 sec latency 12.914 ms 2024-09-11T00:17:25.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:17:24 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:17:25.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:17:24 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T00:17:25.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:17:24 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:17:25.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:17:24 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:17:25.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:17:24 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T00:17:25.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:17:24 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:17:25.330 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:17:24 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:17:25.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:17:24 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T00:17:25.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:17:24 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:17:25.686 INFO:tasks.workunit.client.0.smithi012.stdout: 1 414277 3.87 MB/sec execute 442 sec latency 13.224 ms 2024-09-11T00:17:26.164 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:17:25 smithi012 ceph-mon[30333]: pgmap v709: 129 pgs: 129 active+clean; 96 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 985 KiB/s wr, 168 op/s 2024-09-11T00:17:26.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:17:25 smithi165 ceph-mon[35356]: pgmap v709: 129 pgs: 129 active+clean; 96 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 985 KiB/s wr, 168 op/s 2024-09-11T00:17:26.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:17:25 smithi178 ceph-mon[35919]: pgmap v709: 129 pgs: 129 active+clean; 96 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 985 KiB/s wr, 168 op/s 2024-09-11T00:17:26.686 INFO:tasks.workunit.client.0.smithi012.stdout: 1 415635 3.88 MB/sec execute 443 sec latency 38.184 ms 2024-09-11T00:17:27.686 INFO:tasks.workunit.client.0.smithi012.stdout: 1 417940 3.90 MB/sec execute 444 sec latency 11.840 ms 2024-09-11T00:17:28.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:17:27 smithi165 ceph-mon[35356]: pgmap v710: 129 pgs: 129 active+clean; 96 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 1.4 MiB/s wr, 274 op/s 2024-09-11T00:17:28.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:17:27 smithi012 ceph-mon[30333]: pgmap v710: 129 pgs: 129 active+clean; 96 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 1.4 MiB/s wr, 274 op/s 2024-09-11T00:17:28.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:17:27 smithi178 ceph-mon[35919]: pgmap v710: 129 pgs: 129 active+clean; 96 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 1.4 MiB/s wr, 274 op/s 2024-09-11T00:17:28.686 INFO:tasks.workunit.client.0.smithi012.stdout: 1 419181 3.91 MB/sec execute 445 sec latency 62.174 ms 2024-09-11T00:17:29.687 INFO:tasks.workunit.client.0.smithi012.stdout: 1 421456 3.93 MB/sec execute 446 sec latency 11.873 ms 2024-09-11T00:17:30.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:17:29 smithi165 ceph-mon[35356]: pgmap v711: 129 pgs: 129 active+clean; 96 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 989 KiB/s wr, 238 op/s 2024-09-11T00:17:30.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:17:29 smithi012 ceph-mon[30333]: pgmap v711: 129 pgs: 129 active+clean; 96 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 989 KiB/s wr, 238 op/s 2024-09-11T00:17:30.330 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:17:29 smithi178 ceph-mon[35919]: pgmap v711: 129 pgs: 129 active+clean; 96 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 989 KiB/s wr, 238 op/s 2024-09-11T00:17:30.686 INFO:tasks.workunit.client.0.smithi012.stdout: 1 422725 3.93 MB/sec execute 447 sec latency 11.942 ms 2024-09-11T00:17:31.687 INFO:tasks.workunit.client.0.smithi012.stdout: 1 424894 3.95 MB/sec execute 448 sec latency 10.351 ms 2024-09-11T00:17:32.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:17:31 smithi165 ceph-mon[35356]: pgmap v712: 129 pgs: 129 active+clean; 97 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 1.1 MiB/s wr, 257 op/s 2024-09-11T00:17:32.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:17:31 smithi012 ceph-mon[30333]: pgmap v712: 129 pgs: 129 active+clean; 97 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 1.1 MiB/s wr, 257 op/s 2024-09-11T00:17:32.330 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:17:31 smithi178 ceph-mon[35919]: pgmap v712: 129 pgs: 129 active+clean; 97 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 1.1 MiB/s wr, 257 op/s 2024-09-11T00:17:32.687 INFO:tasks.workunit.client.0.smithi012.stdout: 1 426275 3.96 MB/sec execute 449 sec latency 10.383 ms 2024-09-11T00:17:33.524 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:33.522+0000 7fb631c00640 1 -- 172.21.15.12:0/2323612735 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb62c1058f0 msgr2=0x7fb62c107d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:17:33.524 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:33.522+0000 7fb631c00640 1 --2- 172.21.15.12:0/2323612735 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb62c1058f0 0x7fb62c107d60 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fb618009860 tx=0x7fb61802a340 comp rx=0 tx=0).stop 2024-09-11T00:17:33.524 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:33.522+0000 7fb631c00640 1 -- 172.21.15.12:0/2323612735 shutdown_connections 2024-09-11T00:17:33.524 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:33.522+0000 7fb631c00640 1 --2- 172.21.15.12:0/2323612735 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb62c1084a0 0x7fb62c10a960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:17:33.524 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:33.522+0000 7fb631c00640 1 --2- 172.21.15.12:0/2323612735 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb62c1058f0 0x7fb62c107d60 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:17:33.524 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:33.522+0000 7fb631c00640 1 --2- 172.21.15.12:0/2323612735 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb62c06b470 0x7fb62c105360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:17:33.525 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:33.522+0000 7fb631c00640 1 -- 172.21.15.12:0/2323612735 >> 172.21.15.12:0/2323612735 conn(0x7fb62c0fcf30 msgr2=0x7fb62c0ff370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:17:33.525 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:33.523+0000 7fb631c00640 1 -- 172.21.15.12:0/2323612735 shutdown_connections 2024-09-11T00:17:33.525 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:33.523+0000 7fb631c00640 1 -- 172.21.15.12:0/2323612735 wait complete. 2024-09-11T00:17:33.527 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:33.523+0000 7fb631c00640 1 Processor -- start 2024-09-11T00:17:33.527 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:33.524+0000 7fb631c00640 1 -- start start 2024-09-11T00:17:33.527 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:33.524+0000 7fb631c00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb62c06b470 0x7fb62c19ada0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:17:33.527 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:33.524+0000 7fb631c00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb62c1058f0 0x7fb62c19b3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:17:33.527 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:33.524+0000 7fb631c00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb62c1084a0 0x7fb62c19c020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:17:33.527 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:33.524+0000 7fb631c00640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fb62c10e3d0 con 0x7fb62c06b470 2024-09-11T00:17:33.527 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:33.524+0000 7fb631c00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fb62c10e250 con 0x7fb62c1058f0 2024-09-11T00:17:33.527 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:33.524+0000 7fb631c00640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fb62c10e550 con 0x7fb62c1084a0 2024-09-11T00:17:33.528 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:33.524+0000 7fb62b400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb62c1058f0 0x7fb62c19b3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:17:33.528 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:33.524+0000 7fb631200640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb62c1084a0 0x7fb62c19c020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:17:33.528 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:33.524+0000 7fb62b400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb62c1058f0 0x7fb62c19b3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.12:33088/0 (socket says 172.21.15.12:33088) 2024-09-11T00:17:33.528 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:33.524+0000 7fb62b400640 1 -- 172.21.15.12:0/3811669051 learned_addr learned my addr 172.21.15.12:0/3811669051 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:17:33.528 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:33.524+0000 7fb62be00640 1 --2- 172.21.15.12:0/3811669051 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb62c06b470 0x7fb62c19ada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:17:33.528 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:33.525+0000 7fb62b400640 1 -- 172.21.15.12:0/3811669051 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb62c1084a0 msgr2=0x7fb62c19c020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:17:33.528 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:33.525+0000 7fb62b400640 1 --2- 172.21.15.12:0/3811669051 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb62c1084a0 0x7fb62c19c020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:17:33.528 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:33.525+0000 7fb62b400640 1 -- 172.21.15.12:0/3811669051 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb62c06b470 msgr2=0x7fb62c19ada0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:17:33.528 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:33.525+0000 7fb62b400640 1 --2- 172.21.15.12:0/3811669051 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb62c06b470 0x7fb62c19ada0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:17:33.528 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:33.525+0000 7fb62b400640 1 -- 172.21.15.12:0/3811669051 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb62c19c8e0 con 0x7fb62c1058f0 2024-09-11T00:17:33.528 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:33.525+0000 7fb631200640 1 --2- 172.21.15.12:0/3811669051 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb62c1084a0 0x7fb62c19c020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:17:33.528 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:33.525+0000 7fb62b400640 1 --2- 172.21.15.12:0/3811669051 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb62c1058f0 0x7fb62c19b3f0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fb618005940 tx=0x7fb61802ae20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:17:33.529 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:33.525+0000 7fb628c00640 1 -- 172.21.15.12:0/3811669051 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb618040020 con 0x7fb62c1058f0 2024-09-11T00:17:33.529 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:33.525+0000 7fb631c00640 1 -- 172.21.15.12:0/3811669051 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb62c19cb70 con 0x7fb62c1058f0 2024-09-11T00:17:33.529 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:33.525+0000 7fb628c00640 1 -- 172.21.15.12:0/3811669051 <== mon.2 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb618030e50 con 0x7fb62c1058f0 2024-09-11T00:17:33.529 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:33.525+0000 7fb628c00640 1 -- 172.21.15.12:0/3811669051 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb618037040 con 0x7fb62c1058f0 2024-09-11T00:17:33.529 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:33.525+0000 7fb631c00640 1 -- 172.21.15.12:0/3811669051 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb62c1a32a0 con 0x7fb62c1058f0 2024-09-11T00:17:33.530 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:33.528+0000 7fb636241740 1 Processor -- start 2024-09-11T00:17:33.531 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:33.529+0000 7fb636241740 1 -- start start 2024-09-11T00:17:33.531 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:33.529+0000 7fb628c00640 1 -- 172.21.15.12:0/3811669051 <== mon.2 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7fb618004160 con 0x7fb62c1058f0 2024-09-11T00:17:33.531 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:33.529+0000 7fb636241740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e84d188970 0x55e84d18ad60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:17:33.531 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:33.529+0000 7fb636241740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e84d18b2a0 0x55e84d18d690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:17:33.531 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:33.529+0000 7fb636241740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e84d18dce0 0x55e84d1901a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:17:33.531 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:33.529+0000 7fb636241740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55e84d0fb6b0 con 0x55e84d188970 2024-09-11T00:17:33.531 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:33.529+0000 7fb636241740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55e84d0eaae0 con 0x55e84d18b2a0 2024-09-11T00:17:33.531 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:33.529+0000 7fb636241740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55e84d0ee8f0 con 0x55e84d18dce0 2024-09-11T00:17:33.531 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:33.529+0000 7fb631200640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e84d188970 0x55e84d18ad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:17:33.531 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:33.529+0000 7fb62be00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e84d18b2a0 0x55e84d18d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:17:33.531 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:33.529+0000 7fb62be00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e84d18b2a0 0x55e84d18d690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.12:33098/0 (socket says 172.21.15.12:33098) 2024-09-11T00:17:33.531 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:33.529+0000 7fb62b400640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e84d18dce0 0x55e84d1901a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:17:33.531 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:33.529+0000 7fb631200640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e84d188970 0x55e84d18ad60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:47466/0 (socket says 172.21.15.12:47466) 2024-09-11T00:17:33.532 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:33.529+0000 7fb62be00640 1 -- 172.21.15.12:0/3118136293 learned_addr learned my addr 172.21.15.12:0/3118136293 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:17:33.532 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:33.530+0000 7fb62be00640 1 -- 172.21.15.12:0/3118136293 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e84d18dce0 msgr2=0x55e84d1901a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:17:33.532 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:33.530+0000 7fb62be00640 1 --2- 172.21.15.12:0/3118136293 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e84d18dce0 0x55e84d1901a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:17:33.532 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:33.530+0000 7fb62be00640 1 -- 172.21.15.12:0/3118136293 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e84d188970 msgr2=0x55e84d18ad60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:17:33.532 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:33.530+0000 7fb62be00640 1 --2- 172.21.15.12:0/3118136293 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e84d188970 0x55e84d18ad60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:17:33.532 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:33.530+0000 7fb62be00640 1 -- 172.21.15.12:0/3118136293 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e84d0b7780 con 0x55e84d18b2a0 2024-09-11T00:17:33.532 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:33.530+0000 7fb631200640 1 --2- 172.21.15.12:0/3118136293 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e84d188970 0x55e84d18ad60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:17:33.538 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:33.530+0000 7fb62be00640 1 --2- 172.21.15.12:0/3118136293 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e84d18b2a0 0x55e84d18d690 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fb61c009cc0 tx=0x7fb61c011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c4bfc26b7fee8fe1 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:17:33.538 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:33.530+0000 7fb621600640 1 -- 172.21.15.12:0/3118136293 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb61c005220 con 0x55e84d18b2a0 2024-09-11T00:17:33.538 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:33.530+0000 7fb621600640 1 -- 172.21.15.12:0/3118136293 <== mon.2 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb61c004c50 con 0x55e84d18b2a0 2024-09-11T00:17:33.538 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:33.530+0000 7fb621600640 1 -- 172.21.15.12:0/3118136293 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb61c0197f0 con 0x55e84d18b2a0 2024-09-11T00:17:33.538 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:33.530+0000 7fb628c00640 1 --2- 172.21.15.12:0/3811669051 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fb6100b52e0 0x7fb6100b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:17:33.538 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:33.531+0000 7fb628c00640 1 -- 172.21.15.12:0/3811669051 <== mon.2 v2:172.21.15.165:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9208+0+0 (secure 0 0 0) 0x7fb618105f30 con 0x7fb62c1058f0 2024-09-11T00:17:33.538 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:33.531+0000 7fb631200640 1 --2- 172.21.15.12:0/3811669051 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fb6100b52e0 0x7fb6100b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:17:33.538 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:33.531+0000 7fb636241740 1 -- 172.21.15.12:0/3118136293 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e84d18b2a0 msgr2=0x55e84d18d690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:17:33.538 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:33.531+0000 7fb636241740 1 --2- 172.21.15.12:0/3118136293 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e84d18b2a0 0x55e84d18d690 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fb61c009cc0 tx=0x7fb61c011810 comp rx=0 tx=0).stop 2024-09-11T00:17:33.539 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:33.531+0000 7fb636241740 1 -- 172.21.15.12:0/3118136293 shutdown_connections 2024-09-11T00:17:33.539 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:33.531+0000 7fb636241740 1 --2- 172.21.15.12:0/3118136293 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e84d18dce0 0x55e84d1901a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:17:33.539 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:33.531+0000 7fb636241740 1 --2- 172.21.15.12:0/3118136293 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e84d18b2a0 0x55e84d18d690 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:17:33.539 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:33.531+0000 7fb636241740 1 --2- 172.21.15.12:0/3118136293 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e84d188970 0x55e84d18ad60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:17:33.539 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:33.531+0000 7fb636241740 1 -- 172.21.15.12:0/3118136293 >> 172.21.15.12:0/3118136293 conn(0x55e84d183ac0 msgr2=0x55e84d184850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:17:33.539 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:33.531+0000 7fb636241740 1 -- 172.21.15.12:0/3118136293 shutdown_connections 2024-09-11T00:17:33.539 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:33.531+0000 7fb636241740 1 -- 172.21.15.12:0/3118136293 wait complete. 2024-09-11T00:17:33.539 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:33.531+0000 7fb631200640 1 --2- 172.21.15.12:0/3811669051 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fb6100b52e0 0x7fb6100b77a0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fb624009fd0 tx=0x7fb624009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:17:33.539 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:33.532+0000 7fb636241740 1 -- start start 2024-09-11T00:17:33.539 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:33.532+0000 7fb636241740 1 Processor -- start 2024-09-11T00:17:33.539 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:33.532+0000 7fb636241740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e84d188970 0x55e84d192010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:17:33.539 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:33.532+0000 7fb636241740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e84d18b2a0 0x55e84d1926b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:17:33.539 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:33.532+0000 7fb636241740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e84d18dce0 0x55e84d193280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:17:33.539 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:33.532+0000 7fb636241740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55e84d022ab0 con 0x55e84d18b2a0 2024-09-11T00:17:33.540 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:33.532+0000 7fb636241740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55e84d091570 con 0x55e84d18dce0 2024-09-11T00:17:33.540 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:33.532+0000 7fb636241740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55e84d190cc0 con 0x55e84d188970 2024-09-11T00:17:33.540 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:33.532+0000 7fb636241740 10 client.0 _ll_register_callbacks cb 0x55e84d0a0100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:17:33.540 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:33.533+0000 7fb62be00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e84d188970 0x55e84d192010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:17:33.540 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:33.533+0000 7fb631200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e84d18dce0 0x55e84d193280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:17:33.540 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:33.533+0000 7fb62b400640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e84d18b2a0 0x55e84d1926b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:17:33.540 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:33.533+0000 7fb62be00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e84d188970 0x55e84d192010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.12:45622/0 (socket says 172.21.15.12:45622) 2024-09-11T00:17:33.540 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:33.533+0000 7fb62be00640 1 -- 172.21.15.12:0/3660866240 learned_addr learned my addr 172.21.15.12:0/3660866240 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:17:33.540 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:33.533+0000 7fb631200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e84d18dce0 0x55e84d193280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.12:33112/0 (socket says 172.21.15.12:33112) 2024-09-11T00:17:33.540 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:33.533+0000 7fb62be00640 1 -- 172.21.15.12:0/3660866240 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e84d18dce0 msgr2=0x55e84d193280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:17:33.540 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:33.533+0000 7fb62be00640 1 --2- 172.21.15.12:0/3660866240 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e84d18dce0 0x55e84d193280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:17:33.540 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:33.533+0000 7fb62be00640 1 -- 172.21.15.12:0/3660866240 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e84d18b2a0 msgr2=0x55e84d1926b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:17:33.541 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:33.533+0000 7fb62be00640 1 --2- 172.21.15.12:0/3660866240 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e84d18b2a0 0x55e84d1926b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:17:33.541 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:33.533+0000 7fb62be00640 1 -- 172.21.15.12:0/3660866240 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e84ce4c790 con 0x55e84d188970 2024-09-11T00:17:33.541 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:33.533+0000 7fb62b400640 1 --2- 172.21.15.12:0/3660866240 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e84d18b2a0 0x55e84d1926b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:17:33.541 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:33.533+0000 7fb62be00640 1 --2- 172.21.15.12:0/3660866240 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e84d188970 0x55e84d192010 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fb61c009230 tx=0x7fb61c005030 comp rx=0 tx=0).ready entity=mon.1 client_cookie=734c452ddb7447cf server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:17:33.541 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:33.533+0000 7fb616000640 10 client.0 ms_handle_connect on v2:172.21.15.178:3300/0 2024-09-11T00:17:33.541 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:33.534+0000 7fb616000640 1 -- 172.21.15.12:0/3660866240 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb61c029070 con 0x55e84d188970 2024-09-11T00:17:33.541 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:33.534+0000 7fb616000640 1 -- 172.21.15.12:0/3660866240 <== mon.1 v2:172.21.15.178:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9208+0+0 (secure 0 0 0) 0x7fb61c019710 con 0x55e84d188970 2024-09-11T00:17:33.541 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:33.534+0000 7fb636241740 1 -- 172.21.15.12:0/3660866240 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb61c004c50 con 0x55e84d188970 2024-09-11T00:17:33.541 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:33.534+0000 7fb616000640 1 -- 172.21.15.12:0/3660866240 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7fb61c035050 con 0x55e84d188970 2024-09-11T00:17:33.541 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:33.534+0000 7fb636241740 10 client.26674 fetch_fsmap learned FSMap version 41 2024-09-11T00:17:33.541 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:33.534+0000 7fb636241740 1 -- 172.21.15.12:0/3660866240 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x55e84cec5620 con 0x55e84d188970 2024-09-11T00:17:33.541 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:33.534+0000 7fb616000640 1 -- 172.21.15.12:0/3660866240 <== mon.1 v2:172.21.15.178:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb61c004c50 con 0x55e84d188970 2024-09-11T00:17:33.542 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:33.534+0000 7fb616000640 1 -- 172.21.15.12:0/3660866240 <== mon.1 v2:172.21.15.178:3300/0 5 ==== fsmap(e 41) ==== 8370+0+0 (secure 0 0 0) 0x7fb61c0224f0 con 0x55e84d188970 2024-09-11T00:17:33.542 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:33.535+0000 7fb636241740 10 client.26674 fetch_fsmap finished waiting for FSMap version 41 2024-09-11T00:17:33.542 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:33.535+0000 7fb636241740 10 client.26674 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T00:17:33.542 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:33.535+0000 7fb636241740 20 client.26674 populate_metadata read hostname 'smithi012' 2024-09-11T00:17:33.542 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:33.535+0000 7fb636241740 1 --2- 172.21.15.12:0/3660866240 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55e84d19d6f0 0x55e84d19fae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:17:33.542 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:33.535+0000 7fb636241740 4 client.26674 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T00:17:33.542 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:33.535+0000 7fb636241740 1 -- 172.21.15.12:0/3660866240 --> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e84d0fd600 con 0x55e84d19d6f0 2024-09-11T00:17:33.542 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:33.535+0000 7fb62b400640 1 --2- 172.21.15.12:0/3660866240 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55e84d19d6f0 0x55e84d19fae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:17:33.542 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:33.535+0000 7fb62b400640 1 --2- 172.21.15.12:0/3660866240 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55e84d19d6f0 0x55e84d19fae0 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=54364e5ea1ce5fcb server_cookie=2d10b2381206ddb6 in_seq=0 out_seq=0 2024-09-11T00:17:33.542 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:33.535+0000 7fb616000640 10 client.26674 ms_handle_connect on v2:172.21.15.165:6838/501312984 2024-09-11T00:17:33.542 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:33.536+0000 7fb616000640 1 -- 172.21.15.12:0/3660866240 <== mds.0 v2:172.21.15.165:6838/501312984 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55e84d0fd600 con 0x55e84d19d6f0 2024-09-11T00:17:33.542 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:33.536+0000 7fb616000640 10 client.26674 handle_command_reply: tid=0 2024-09-11T00:17:33.542 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:33.536+0000 7fb636241740 1 client.26674 shutdown 2024-09-11T00:17:33.543 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:33.536+0000 7fb636241740 1 -- 172.21.15.12:0/3660866240 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e84d188970 msgr2=0x55e84d192010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:17:33.543 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:33.536+0000 7fb636241740 1 --2- 172.21.15.12:0/3660866240 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e84d188970 0x55e84d192010 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fb61c009230 tx=0x7fb61c005030 comp rx=0 tx=0).stop 2024-09-11T00:17:33.543 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:33.536+0000 7fb636241740 1 -- 172.21.15.12:0/3660866240 shutdown_connections 2024-09-11T00:17:33.543 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:33.536+0000 7fb636241740 1 --2- 172.21.15.12:0/3660866240 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55e84d19d6f0 0x55e84d19fae0 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:17:33.543 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:33.536+0000 7fb636241740 1 --2- 172.21.15.12:0/3660866240 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e84d18dce0 0x55e84d193280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:17:33.543 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:33.536+0000 7fb636241740 1 --2- 172.21.15.12:0/3660866240 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e84d18b2a0 0x55e84d1926b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:17:33.543 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:33.536+0000 7fb616000640 0 client.26674 ms_handle_reset on v2:172.21.15.165:6838/501312984 2024-09-11T00:17:33.543 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:33.536+0000 7fb636241740 1 --2- 172.21.15.12:0/3660866240 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e84d188970 0x55e84d192010 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:17:33.543 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:33.536+0000 7fb636241740 1 -- 172.21.15.12:0/3660866240 >> 172.21.15.12:0/3660866240 conn(0x55e84d183ac0 msgr2=0x55e84d182bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:17:33.543 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:33.537+0000 7fb636241740 1 -- 172.21.15.12:0/3660866240 shutdown_connections 2024-09-11T00:17:33.543 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:33.537+0000 7fb636241740 1 -- 172.21.15.12:0/3660866240 wait complete. 2024-09-11T00:17:33.543 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:33.537+0000 7fb636241740 20 client.26674 trim_cache size 0 max 16384 2024-09-11T00:17:33.556 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:33.554+0000 7fb636241740 1 Processor -- start 2024-09-11T00:17:33.557 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:33.554+0000 7fb636241740 1 -- start start 2024-09-11T00:17:33.557 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:33.554+0000 7fb636241740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e84d188970 0x55e84d191d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:17:33.557 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:33.554+0000 7fb636241740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e84d18b2a0 0x55e84d18e0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:17:33.557 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:33.554+0000 7fb636241740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e84d18dce0 0x55e84d18e600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:17:33.557 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:33.554+0000 7fb636241740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fb62c1ad7e0 con 0x55e84d18b2a0 2024-09-11T00:17:33.557 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:33.554+0000 7fb636241740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fb62c1af320 con 0x55e84d188970 2024-09-11T00:17:33.557 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:33.554+0000 7fb636241740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fb62c1adeb0 con 0x55e84d18dce0 2024-09-11T00:17:33.557 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:33.555+0000 7fb62be00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e84d188970 0x55e84d191d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:17:33.557 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:33.555+0000 7fb62be00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e84d188970 0x55e84d191d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.12:33118/0 (socket says 172.21.15.12:33118) 2024-09-11T00:17:33.558 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:33.555+0000 7fb62be00640 1 -- 172.21.15.12:0/1119491542 learned_addr learned my addr 172.21.15.12:0/1119491542 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:17:33.558 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:33.555+0000 7fb631200640 1 --2- 172.21.15.12:0/1119491542 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e84d18dce0 0x55e84d18e600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:17:33.558 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:33.555+0000 7fb62b400640 1 --2- 172.21.15.12:0/1119491542 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e84d18b2a0 0x55e84d18e0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:17:33.558 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:33.555+0000 7fb62be00640 1 -- 172.21.15.12:0/1119491542 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e84d18dce0 msgr2=0x55e84d18e600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:17:33.558 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:33.555+0000 7fb62be00640 1 --2- 172.21.15.12:0/1119491542 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e84d18dce0 0x55e84d18e600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:17:33.558 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:33.555+0000 7fb62be00640 1 -- 172.21.15.12:0/1119491542 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e84d18b2a0 msgr2=0x55e84d18e0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:17:33.558 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:33.555+0000 7fb62be00640 1 --2- 172.21.15.12:0/1119491542 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e84d18b2a0 0x55e84d18e0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:17:33.558 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:33.555+0000 7fb62be00640 1 -- 172.21.15.12:0/1119491542 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e84d18fe90 con 0x55e84d188970 2024-09-11T00:17:33.558 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:33.555+0000 7fb62be00640 1 --2- 172.21.15.12:0/1119491542 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e84d188970 0x55e84d191d80 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fb61c0092a0 tx=0x7fb61c005c50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cce336a57ea7defe server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:17:33.558 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:33.555+0000 7fb620c00640 1 -- 172.21.15.12:0/1119491542 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb61c029040 con 0x55e84d188970 2024-09-11T00:17:33.559 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:33.555+0000 7fb620c00640 1 -- 172.21.15.12:0/1119491542 <== mon.2 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb61c026070 con 0x55e84d188970 2024-09-11T00:17:33.559 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:33.555+0000 7fb620c00640 1 -- 172.21.15.12:0/1119491542 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb61c022420 con 0x55e84d188970 2024-09-11T00:17:33.562 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:33.556+0000 7fb636241740 1 -- 172.21.15.12:0/1119491542 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e84d188970 msgr2=0x55e84d191d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:17:33.562 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:33.556+0000 7fb636241740 1 --2- 172.21.15.12:0/1119491542 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e84d188970 0x55e84d191d80 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fb61c0092a0 tx=0x7fb61c005c50 comp rx=0 tx=0).stop 2024-09-11T00:17:33.562 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:33.556+0000 7fb636241740 1 -- 172.21.15.12:0/1119491542 shutdown_connections 2024-09-11T00:17:33.562 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:33.556+0000 7fb636241740 1 --2- 172.21.15.12:0/1119491542 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e84d18dce0 0x55e84d18e600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:17:33.562 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:33.556+0000 7fb636241740 1 --2- 172.21.15.12:0/1119491542 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e84d18b2a0 0x55e84d18e0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:17:33.562 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:33.556+0000 7fb636241740 1 --2- 172.21.15.12:0/1119491542 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e84d188970 0x55e84d191d80 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:17:33.562 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:33.556+0000 7fb636241740 1 -- 172.21.15.12:0/1119491542 >> 172.21.15.12:0/1119491542 conn(0x55e84d183ac0 msgr2=0x55e84d1911b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:17:33.562 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:33.556+0000 7fb636241740 1 -- 172.21.15.12:0/1119491542 shutdown_connections 2024-09-11T00:17:33.562 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:33.556+0000 7fb636241740 1 -- 172.21.15.12:0/1119491542 wait complete. 2024-09-11T00:17:33.562 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:33.556+0000 7fb636241740 1 -- start start 2024-09-11T00:17:33.562 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:33.557+0000 7fb636241740 1 Processor -- start 2024-09-11T00:17:33.563 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:33.557+0000 7fb636241740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e84d188970 0x55e84d199990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:17:33.563 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:33.557+0000 7fb636241740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e84d18b2a0 0x55e84d198690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:17:33.563 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:33.557+0000 7fb636241740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e84d18dce0 0x55e84d198bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:17:33.563 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:33.557+0000 7fb636241740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fb62c1aec50 con 0x55e84d18b2a0 2024-09-11T00:17:33.563 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:33.557+0000 7fb636241740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fb62c1ae580 con 0x55e84d18dce0 2024-09-11T00:17:33.563 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:33.557+0000 7fb636241740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fb62c1ab7c0 con 0x55e84d188970 2024-09-11T00:17:33.563 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:33.557+0000 7fb636241740 10 client.0 _ll_register_callbacks cb 0x7fb624009ee0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:17:33.563 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:33.557+0000 7fb62be00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e84d188970 0x55e84d199990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:17:33.563 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:33.557+0000 7fb62be00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e84d188970 0x55e84d199990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.12:45638/0 (socket says 172.21.15.12:45638) 2024-09-11T00:17:33.563 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:33.557+0000 7fb62be00640 1 -- 172.21.15.12:0/1094138550 learned_addr learned my addr 172.21.15.12:0/1094138550 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:17:33.563 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:33.557+0000 7fb631200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e84d18dce0 0x55e84d198bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:17:33.563 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:33.557+0000 7fb62b400640 1 --2- 172.21.15.12:0/1094138550 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e84d18b2a0 0x55e84d198690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:17:33.563 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:33.558+0000 7fb62be00640 1 -- 172.21.15.12:0/1094138550 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e84d18dce0 msgr2=0x55e84d198bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:17:33.564 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:33.558+0000 7fb62be00640 1 --2- 172.21.15.12:0/1094138550 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e84d18dce0 0x55e84d198bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:17:33.564 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:33.558+0000 7fb62be00640 1 -- 172.21.15.12:0/1094138550 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e84d18b2a0 msgr2=0x55e84d198690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:17:33.564 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:33.558+0000 7fb62be00640 1 --2- 172.21.15.12:0/1094138550 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e84d18b2a0 0x55e84d198690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:17:33.564 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:33.558+0000 7fb62be00640 1 -- 172.21.15.12:0/1094138550 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e84d1992c0 con 0x55e84d188970 2024-09-11T00:17:33.564 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:33.558+0000 7fb62b400640 1 --2- 172.21.15.12:0/1094138550 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e84d18b2a0 0x55e84d198690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T00:17:33.564 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:33.558+0000 7fb62be00640 1 --2- 172.21.15.12:0/1094138550 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e84d188970 0x55e84d199990 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fb61c011d10 tx=0x7fb61c0054e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a8eb92fde05331e2 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:17:33.564 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:33.558+0000 7fb616000640 10 client.0 ms_handle_connect on v2:172.21.15.178:3300/0 2024-09-11T00:17:33.564 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:33.558+0000 7fb616000640 1 -- 172.21.15.12:0/1094138550 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb61c029040 con 0x55e84d188970 2024-09-11T00:17:33.564 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:33.558+0000 7fb636241740 1 -- 172.21.15.12:0/1094138550 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb61c026070 con 0x55e84d188970 2024-09-11T00:17:33.564 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:33.558+0000 7fb616000640 1 -- 172.21.15.12:0/1094138550 <== mon.1 v2:172.21.15.178:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9208+0+0 (secure 0 0 0) 0x7fb61c022b30 con 0x55e84d188970 2024-09-11T00:17:33.564 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:33.558+0000 7fb616000640 1 -- 172.21.15.12:0/1094138550 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7fb61c009510 con 0x55e84d188970 2024-09-11T00:17:33.564 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:33.558+0000 7fb636241740 10 client.26680 fetch_fsmap learned FSMap version 41 2024-09-11T00:17:33.564 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:33.558+0000 7fb636241740 1 -- 172.21.15.12:0/1094138550 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x55e84d20a5b0 con 0x55e84d188970 2024-09-11T00:17:33.565 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:33.559+0000 7fb616000640 1 -- 172.21.15.12:0/1094138550 <== mon.1 v2:172.21.15.178:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb61c026070 con 0x55e84d188970 2024-09-11T00:17:33.565 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:33.559+0000 7fb616000640 1 -- 172.21.15.12:0/1094138550 <== mon.1 v2:172.21.15.178:3300/0 5 ==== fsmap(e 41) ==== 8370+0+0 (secure 0 0 0) 0x7fb61c0035f0 con 0x55e84d188970 2024-09-11T00:17:33.565 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:33.559+0000 7fb636241740 10 client.26680 fetch_fsmap finished waiting for FSMap version 41 2024-09-11T00:17:33.565 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:33.559+0000 7fb636241740 10 client.26680 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T00:17:33.565 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:33.559+0000 7fb636241740 20 client.26680 populate_metadata read hostname 'smithi012' 2024-09-11T00:17:33.565 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:33.559+0000 7fb636241740 1 --2- 172.21.15.12:0/1094138550 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55e84d20add0 0x55e84d20d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:17:33.565 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:33.559+0000 7fb636241740 4 client.26680 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T00:17:33.565 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:33.559+0000 7fb636241740 1 -- 172.21.15.12:0/1094138550 --> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55e84d192460 con 0x55e84d20add0 2024-09-11T00:17:33.565 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:33.559+0000 7fb62b400640 1 --2- 172.21.15.12:0/1094138550 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55e84d20add0 0x55e84d20d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:17:33.565 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:33.560+0000 7fb62b400640 1 --2- 172.21.15.12:0/1094138550 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55e84d20add0 0x55e84d20d1c0 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d54e57ea3fa5d493 server_cookie=13b18eb50d5a352c in_seq=0 out_seq=0 2024-09-11T00:17:33.565 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:33.560+0000 7fb616000640 10 client.26680 ms_handle_connect on v2:172.21.15.165:6838/501312984 2024-09-11T00:17:33.635 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:33.634+0000 7fb616000640 1 -- 172.21.15.12:0/1094138550 <== mds.0 v2:172.21.15.165:6838/501312984 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55e84d192460 con 0x55e84d20add0 2024-09-11T00:17:33.636 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:33.634+0000 7fb616000640 10 client.26680 handle_command_reply: tid=0 2024-09-11T00:17:33.636 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:33.634+0000 7fb636241740 1 client.26680 shutdown 2024-09-11T00:17:33.636 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:33.634+0000 7fb636241740 1 -- 172.21.15.12:0/1094138550 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e84d188970 msgr2=0x55e84d199990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:17:33.636 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:33.634+0000 7fb636241740 1 --2- 172.21.15.12:0/1094138550 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e84d188970 0x55e84d199990 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fb61c011d10 tx=0x7fb61c0054e0 comp rx=0 tx=0).stop 2024-09-11T00:17:33.636 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:33.634+0000 7fb636241740 1 -- 172.21.15.12:0/1094138550 shutdown_connections 2024-09-11T00:17:33.636 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:33.634+0000 7fb636241740 1 --2- 172.21.15.12:0/1094138550 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55e84d20add0 0x55e84d20d1c0 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:17:33.636 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:33.634+0000 7fb636241740 1 --2- 172.21.15.12:0/1094138550 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e84d18dce0 0x55e84d198bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:17:33.636 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:33.634+0000 7fb636241740 1 --2- 172.21.15.12:0/1094138550 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e84d18b2a0 0x55e84d198690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:17:33.636 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:33.634+0000 7fb636241740 1 --2- 172.21.15.12:0/1094138550 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e84d188970 0x55e84d199990 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:17:33.637 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:33.634+0000 7fb616000640 0 client.26680 ms_handle_reset on v2:172.21.15.165:6838/501312984 2024-09-11T00:17:33.637 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:33.634+0000 7fb636241740 1 -- 172.21.15.12:0/1094138550 >> 172.21.15.12:0/1094138550 conn(0x55e84d183ac0 msgr2=0x55e84d18e0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:17:33.637 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:33.634+0000 7fb636241740 1 -- 172.21.15.12:0/1094138550 shutdown_connections 2024-09-11T00:17:33.637 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:33.634+0000 7fb636241740 1 -- 172.21.15.12:0/1094138550 wait complete. 2024-09-11T00:17:33.637 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:33.634+0000 7fb636241740 20 client.26680 trim_cache size 0 max 16384 2024-09-11T00:17:33.637 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:{ 2024-09-11T00:17:33.637 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "message": "", 2024-09-11T00:17:33.637 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "return_code": 0 2024-09-11T00:17:33.637 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:} 2024-09-11T00:17:33.638 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:33.635+0000 7fb631c00640 1 -- 172.21.15.12:0/3811669051 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fb6100b52e0 msgr2=0x7fb6100b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:17:33.638 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:33.635+0000 7fb631c00640 1 --2- 172.21.15.12:0/3811669051 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fb6100b52e0 0x7fb6100b77a0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fb624009fd0 tx=0x7fb624009500 comp rx=0 tx=0).stop 2024-09-11T00:17:33.638 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:33.635+0000 7fb631c00640 1 -- 172.21.15.12:0/3811669051 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb62c1058f0 msgr2=0x7fb62c19b3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:17:33.638 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:33.635+0000 7fb631c00640 1 --2- 172.21.15.12:0/3811669051 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb62c1058f0 0x7fb62c19b3f0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fb618005940 tx=0x7fb61802ae20 comp rx=0 tx=0).stop 2024-09-11T00:17:33.638 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:33.635+0000 7fb631c00640 1 -- 172.21.15.12:0/3811669051 shutdown_connections 2024-09-11T00:17:33.638 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:33.635+0000 7fb631c00640 1 --2- 172.21.15.12:0/3811669051 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fb6100b52e0 0x7fb6100b77a0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:17:33.638 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:33.635+0000 7fb631c00640 1 --2- 172.21.15.12:0/3811669051 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb62c1084a0 0x7fb62c19c020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:17:33.638 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:33.635+0000 7fb631c00640 1 --2- 172.21.15.12:0/3811669051 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb62c1058f0 0x7fb62c19b3f0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:17:33.639 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:33.635+0000 7fb631c00640 1 --2- 172.21.15.12:0/3811669051 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb62c06b470 0x7fb62c19ada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:17:33.639 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:33.635+0000 7fb631c00640 1 -- 172.21.15.12:0/3811669051 >> 172.21.15.12:0/3811669051 conn(0x7fb62c0fcf30 msgr2=0x7fb62c103b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:17:33.639 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:33.635+0000 7fb631c00640 1 -- 172.21.15.12:0/3811669051 shutdown_connections 2024-09-11T00:17:33.639 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:33.635+0000 7fb631c00640 1 -- 172.21.15.12:0/3811669051 wait complete. 2024-09-11T00:17:33.686 INFO:tasks.workunit.client.0.smithi012.stdout: 1 428441 3.97 MB/sec execute 450 sec latency 56.095 ms 2024-09-11T00:17:34.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:17:33 smithi165 ceph-mon[35356]: pgmap v713: 129 pgs: 129 active+clean; 100 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 1.1 MiB/s wr, 263 op/s 2024-09-11T00:17:34.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:17:33 smithi012 ceph-mon[30333]: pgmap v713: 129 pgs: 129 active+clean; 100 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 1.1 MiB/s wr, 263 op/s 2024-09-11T00:17:34.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:17:33 smithi178 ceph-mon[35919]: pgmap v713: 129 pgs: 129 active+clean; 100 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 1.1 MiB/s wr, 263 op/s 2024-09-11T00:17:34.687 INFO:tasks.workunit.client.0.smithi012.stdout: 1 429825 3.98 MB/sec execute 451 sec latency 12.784 ms 2024-09-11T00:17:35.687 INFO:tasks.workunit.client.0.smithi012.stdout: 1 432000 4.00 MB/sec execute 452 sec latency 15.443 ms 2024-09-11T00:17:36.284 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:17:35 smithi165 ceph-mon[35356]: pgmap v714: 129 pgs: 129 active+clean; 100 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 968 KiB/s wr, 225 op/s 2024-09-11T00:17:36.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:17:35 smithi012 ceph-mon[30333]: pgmap v714: 129 pgs: 129 active+clean; 100 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 968 KiB/s wr, 225 op/s 2024-09-11T00:17:36.330 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:17:35 smithi178 ceph-mon[35919]: pgmap v714: 129 pgs: 129 active+clean; 100 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 968 KiB/s wr, 225 op/s 2024-09-11T00:17:36.687 INFO:tasks.workunit.client.0.smithi012.stdout: 1 433430 4.01 MB/sec execute 453 sec latency 12.833 ms 2024-09-11T00:17:37.687 INFO:tasks.workunit.client.0.smithi012.stdout: 1 435538 4.02 MB/sec execute 454 sec latency 10.271 ms 2024-09-11T00:17:38.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:17:37 smithi165 ceph-mon[35356]: pgmap v715: 129 pgs: 129 active+clean; 104 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 1.3 MiB/s wr, 345 op/s 2024-09-11T00:17:38.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:17:37 smithi012 ceph-mon[30333]: pgmap v715: 129 pgs: 129 active+clean; 104 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 1.3 MiB/s wr, 345 op/s 2024-09-11T00:17:38.332 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:17:37 smithi178 ceph-mon[35919]: pgmap v715: 129 pgs: 129 active+clean; 104 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 1.3 MiB/s wr, 345 op/s 2024-09-11T00:17:38.687 INFO:tasks.workunit.client.0.smithi012.stdout: 1 436914 4.03 MB/sec execute 455 sec latency 13.010 ms 2024-09-11T00:17:39.687 INFO:tasks.workunit.client.0.smithi012.stdout: 1 439073 4.05 MB/sec execute 456 sec latency 13.855 ms 2024-09-11T00:17:40.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:17:39 smithi165 ceph-mon[35356]: pgmap v716: 129 pgs: 129 active+clean; 104 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 966 KiB/s wr, 241 op/s 2024-09-11T00:17:40.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:17:39 smithi012 ceph-mon[30333]: pgmap v716: 129 pgs: 129 active+clean; 104 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 966 KiB/s wr, 241 op/s 2024-09-11T00:17:40.330 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:17:39 smithi178 ceph-mon[35919]: pgmap v716: 129 pgs: 129 active+clean; 104 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 966 KiB/s wr, 241 op/s 2024-09-11T00:17:40.687 INFO:tasks.workunit.client.0.smithi012.stdout: 1 440431 4.06 MB/sec execute 457 sec latency 13.028 ms 2024-09-11T00:17:41.687 INFO:tasks.workunit.client.0.smithi012.stdout: 1 442457 4.07 MB/sec execute 458 sec latency 12.410 ms 2024-09-11T00:17:42.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:17:41 smithi165 ceph-mon[35356]: pgmap v717: 129 pgs: 129 active+clean; 106 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 1.1 MiB/s wr, 256 op/s 2024-09-11T00:17:42.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:17:41 smithi012 ceph-mon[30333]: pgmap v717: 129 pgs: 129 active+clean; 106 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 1.1 MiB/s wr, 256 op/s 2024-09-11T00:17:42.330 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:17:41 smithi178 ceph-mon[35919]: pgmap v717: 129 pgs: 129 active+clean; 106 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 1.1 MiB/s wr, 256 op/s 2024-09-11T00:17:42.687 INFO:tasks.workunit.client.0.smithi012.stdout: 1 443938 4.08 MB/sec execute 459 sec latency 11.502 ms 2024-09-11T00:17:43.687 INFO:tasks.workunit.client.0.smithi012.stdout: 1 445963 4.09 MB/sec execute 460 sec latency 10.350 ms 2024-09-11T00:17:44.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:17:43 smithi165 ceph-mon[35356]: pgmap v718: 129 pgs: 129 active+clean; 109 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 1.3 MiB/s wr, 321 op/s 2024-09-11T00:17:44.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:17:43 smithi012 ceph-mon[30333]: pgmap v718: 129 pgs: 129 active+clean; 109 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 1.3 MiB/s wr, 321 op/s 2024-09-11T00:17:44.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:17:43 smithi178 ceph-mon[35919]: pgmap v718: 129 pgs: 129 active+clean; 109 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 1.3 MiB/s wr, 321 op/s 2024-09-11T00:17:44.687 INFO:tasks.workunit.client.0.smithi012.stdout: 1 447404 4.10 MB/sec execute 461 sec latency 15.408 ms 2024-09-11T00:17:45.688 INFO:tasks.workunit.client.0.smithi012.stdout: 1 449347 4.11 MB/sec execute 462 sec latency 29.699 ms 2024-09-11T00:17:46.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:17:45 smithi165 ceph-mon[35356]: pgmap v719: 129 pgs: 129 active+clean; 109 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 988 KiB/s wr, 237 op/s 2024-09-11T00:17:46.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:17:45 smithi012 ceph-mon[30333]: pgmap v719: 129 pgs: 129 active+clean; 109 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 988 KiB/s wr, 237 op/s 2024-09-11T00:17:46.330 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:17:45 smithi178 ceph-mon[35919]: pgmap v719: 129 pgs: 129 active+clean; 109 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 988 KiB/s wr, 237 op/s 2024-09-11T00:17:46.688 INFO:tasks.workunit.client.0.smithi012.stdout: 1 450883 4.13 MB/sec execute 463 sec latency 11.597 ms 2024-09-11T00:17:46.764 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:46.759+0000 7f41b1800640 1 -- 172.21.15.12:0/3533016019 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f41ac1049d0 msgr2=0x7f41ac104dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:17:46.764 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:46.759+0000 7f41b1800640 1 --2- 172.21.15.12:0/3533016019 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f41ac1049d0 0x7f41ac104dd0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f419c009900 tx=0x7f419c02a3f0 comp rx=0 tx=0).stop 2024-09-11T00:17:46.765 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:46.759+0000 7f41b1800640 1 -- 172.21.15.12:0/3533016019 shutdown_connections 2024-09-11T00:17:46.765 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:46.759+0000 7f41b1800640 1 --2- 172.21.15.12:0/3533016019 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f41ac1064e0 0x7f41ac10ce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:17:46.765 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:46.759+0000 7f41b1800640 1 --2- 172.21.15.12:0/3533016019 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f41ac105b20 0x7f41ac105fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:17:46.765 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:46.759+0000 7f41b1800640 1 --2- 172.21.15.12:0/3533016019 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f41ac1049d0 0x7f41ac104dd0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:17:46.765 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:46.759+0000 7f41b1800640 1 -- 172.21.15.12:0/3533016019 >> 172.21.15.12:0/3533016019 conn(0x7f41ac100270 msgr2=0x7f41ac1026b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:17:46.766 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:46.759+0000 7f41b1800640 1 -- 172.21.15.12:0/3533016019 shutdown_connections 2024-09-11T00:17:46.766 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:46.759+0000 7f41b1800640 1 -- 172.21.15.12:0/3533016019 wait complete. 2024-09-11T00:17:46.766 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:46.760+0000 7f41b1800640 1 Processor -- start 2024-09-11T00:17:46.766 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:46.760+0000 7f41b1800640 1 -- start start 2024-09-11T00:17:46.766 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:46.760+0000 7f41b1800640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f41ac1049d0 0x7f41ac19ade0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:17:46.766 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:46.760+0000 7f41b1800640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f41ac105b20 0x7f41ac19b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:17:46.767 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:46.760+0000 7f41b1800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f41ac1064e0 0x7f41ac19c060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:17:46.767 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:46.760+0000 7f41b1800640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f41ac110690 con 0x7f41ac1049d0 2024-09-11T00:17:46.767 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:46.760+0000 7f41b1800640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f41ac110510 con 0x7f41ac1064e0 2024-09-11T00:17:46.767 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:46.760+0000 7f41b1800640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f41ac110810 con 0x7f41ac105b20 2024-09-11T00:17:46.767 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:46.761+0000 7f41ab400640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f41ac105b20 0x7f41ac19b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:17:46.768 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:46.761+0000 7f41b0e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f41ac1064e0 0x7f41ac19c060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:17:46.768 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:46.761+0000 7f41ab400640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f41ac105b20 0x7f41ac19b430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.12:48456/0 (socket says 172.21.15.12:48456) 2024-09-11T00:17:46.768 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:46.761+0000 7f41b0e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f41ac1064e0 0x7f41ac19c060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.12:33466/0 (socket says 172.21.15.12:33466) 2024-09-11T00:17:46.768 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:46.761+0000 7f41ab400640 1 -- 172.21.15.12:0/1671245056 learned_addr learned my addr 172.21.15.12:0/1671245056 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:17:46.768 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:46.761+0000 7f41abe00640 1 --2- 172.21.15.12:0/1671245056 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f41ac1049d0 0x7f41ac19ade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:17:46.768 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:46.761+0000 7f41ab400640 1 -- 172.21.15.12:0/1671245056 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f41ac1064e0 msgr2=0x7f41ac19c060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:17:46.769 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:46.761+0000 7f41ab400640 1 --2- 172.21.15.12:0/1671245056 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f41ac1064e0 0x7f41ac19c060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:17:46.769 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:46.761+0000 7f41ab400640 1 -- 172.21.15.12:0/1671245056 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f41ac1049d0 msgr2=0x7f41ac19ade0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:17:46.769 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:46.761+0000 7f41ab400640 1 --2- 172.21.15.12:0/1671245056 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f41ac1049d0 0x7f41ac19ade0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:17:46.769 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:46.761+0000 7f41ab400640 1 -- 172.21.15.12:0/1671245056 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41ac19c920 con 0x7f41ac105b20 2024-09-11T00:17:46.769 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:46.761+0000 7f41ab400640 1 --2- 172.21.15.12:0/1671245056 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f41ac105b20 0x7f41ac19b430 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f419800e9f0 tx=0x7f419800edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:17:46.769 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:46.762+0000 7f41a8c00640 1 -- 172.21.15.12:0/1671245056 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f419800cd80 con 0x7f41ac105b20 2024-09-11T00:17:46.770 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:46.762+0000 7f41a8c00640 1 -- 172.21.15.12:0/1671245056 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f41980054d0 con 0x7f41ac105b20 2024-09-11T00:17:46.770 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:46.762+0000 7f41b1800640 1 -- 172.21.15.12:0/1671245056 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f41ac19cc10 con 0x7f41ac105b20 2024-09-11T00:17:46.770 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:46.762+0000 7f41a8c00640 1 -- 172.21.15.12:0/1671245056 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4198010690 con 0x7f41ac105b20 2024-09-11T00:17:46.770 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:46.762+0000 7f41b1800640 1 -- 172.21.15.12:0/1671245056 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41ac1a32e0 con 0x7f41ac105b20 2024-09-11T00:17:46.782 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:46.765+0000 7f41b5c92740 1 Processor -- start 2024-09-11T00:17:46.782 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:46.765+0000 7f41b5c92740 1 -- start start 2024-09-11T00:17:46.782 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:46.765+0000 7f41b5c92740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5615fc41e970 0x5615fc420d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:17:46.782 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:46.765+0000 7f41b5c92740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5615fc4212a0 0x5615fc423690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:17:46.782 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:46.765+0000 7f41b5c92740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5615fc423ce0 0x5615fc4261a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:17:46.783 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:46.765+0000 7f41b5c92740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x5615fc3916b0 con 0x5615fc423ce0 2024-09-11T00:17:46.783 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:46.765+0000 7f41b5c92740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x5615fc380ae0 con 0x5615fc41e970 2024-09-11T00:17:46.783 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:46.765+0000 7f41b5c92740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5615fc3848f0 con 0x5615fc4212a0 2024-09-11T00:17:46.783 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:46.765+0000 7f41b0e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5615fc41e970 0x5615fc420d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:17:46.783 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:46.765+0000 7f41b0e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5615fc41e970 0x5615fc420d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.12:33468/0 (socket says 172.21.15.12:33468) 2024-09-11T00:17:46.783 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:46.765+0000 7f41b0e00640 1 -- 172.21.15.12:0/4271752498 learned_addr learned my addr 172.21.15.12:0/4271752498 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:17:46.784 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:46.765+0000 7f41abe00640 1 --2- 172.21.15.12:0/4271752498 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5615fc4212a0 0x5615fc423690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:17:46.784 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:46.766+0000 7f41a8c00640 1 -- 172.21.15.12:0/1671245056 <== mon.1 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7f4198004e90 con 0x7f41ac105b20 2024-09-11T00:17:46.784 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:46.766+0000 7f41b0e00640 1 -- 172.21.15.12:0/4271752498 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5615fc4212a0 msgr2=0x5615fc423690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:17:46.784 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:46.766+0000 7f41b0e00640 1 --2- 172.21.15.12:0/4271752498 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5615fc4212a0 0x5615fc423690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:17:46.784 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:46.766+0000 7f41b0e00640 1 -- 172.21.15.12:0/4271752498 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5615fc423ce0 msgr2=0x5615fc4261a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-11T00:17:46.784 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:46.766+0000 7f41b0e00640 1 --2- 172.21.15.12:0/4271752498 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5615fc423ce0 0x5615fc4261a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:17:46.784 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:46.766+0000 7f41b0e00640 1 -- 172.21.15.12:0/4271752498 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5615fc34d780 con 0x5615fc41e970 2024-09-11T00:17:46.784 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:46.766+0000 7f41abe00640 1 --2- 172.21.15.12:0/4271752498 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5615fc4212a0 0x5615fc423690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:17:46.785 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:46.766+0000 7f41b0e00640 1 --2- 172.21.15.12:0/4271752498 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5615fc41e970 0x5615fc420d60 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f41a4003970 tx=0x7f41a4003e30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=56ee27edff47b490 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:17:46.785 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:46.767+0000 7f41a1600640 1 -- 172.21.15.12:0/4271752498 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41a4011440 con 0x5615fc41e970 2024-09-11T00:17:46.785 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:46.767+0000 7f41a1600640 1 -- 172.21.15.12:0/4271752498 <== mon.2 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f41a4015030 con 0x5615fc41e970 2024-09-11T00:17:46.785 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:46.767+0000 7f41a1600640 1 -- 172.21.15.12:0/4271752498 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41a4010980 con 0x5615fc41e970 2024-09-11T00:17:46.785 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:46.768+0000 7f41b5c92740 1 -- 172.21.15.12:0/4271752498 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5615fc41e970 msgr2=0x5615fc420d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:17:46.785 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:46.768+0000 7f41b5c92740 1 --2- 172.21.15.12:0/4271752498 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5615fc41e970 0x5615fc420d60 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f41a4003970 tx=0x7f41a4003e30 comp rx=0 tx=0).stop 2024-09-11T00:17:46.785 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:46.768+0000 7f41b5c92740 1 -- 172.21.15.12:0/4271752498 shutdown_connections 2024-09-11T00:17:46.786 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:46.768+0000 7f41b5c92740 1 --2- 172.21.15.12:0/4271752498 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5615fc423ce0 0x5615fc4261a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:17:46.786 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:46.768+0000 7f41b5c92740 1 --2- 172.21.15.12:0/4271752498 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5615fc4212a0 0x5615fc423690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:17:46.786 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:46.768+0000 7f41b5c92740 1 --2- 172.21.15.12:0/4271752498 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5615fc41e970 0x5615fc420d60 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:17:46.786 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:46.768+0000 7f41b5c92740 1 -- 172.21.15.12:0/4271752498 >> 172.21.15.12:0/4271752498 conn(0x5615fc419ac0 msgr2=0x5615fc41a850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:17:46.786 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:46.768+0000 7f41b5c92740 1 -- 172.21.15.12:0/4271752498 shutdown_connections 2024-09-11T00:17:46.787 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:46.768+0000 7f41b5c92740 1 -- 172.21.15.12:0/4271752498 wait complete. 2024-09-11T00:17:46.787 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:46.768+0000 7f41a8c00640 1 --2- 172.21.15.12:0/1671245056 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f41900b5270 0x7f41900b7730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:17:46.787 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:46.768+0000 7f41a8c00640 1 -- 172.21.15.12:0/1671245056 <== mon.1 v2:172.21.15.178:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9208+0+0 (secure 0 0 0) 0x7f41980e63e0 con 0x7f41ac105b20 2024-09-11T00:17:46.788 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:46.768+0000 7f41b5c92740 1 -- start start 2024-09-11T00:17:46.788 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:46.769+0000 7f41b0e00640 1 --2- 172.21.15.12:0/1671245056 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f41900b5270 0x7f41900b7730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:17:46.788 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:46.769+0000 7f41b5c92740 1 Processor -- start 2024-09-11T00:17:46.788 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:46.769+0000 7f41b5c92740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5615fc41e970 0x5615fc428110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:17:46.788 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:46.769+0000 7f41b5c92740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5615fc4212a0 0x5615fc4287b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:17:46.789 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:46.769+0000 7f41b0e00640 1 --2- 172.21.15.12:0/1671245056 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f41900b5270 0x7f41900b7730 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f41a4009d90 tx=0x7f41a4000f40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:17:46.789 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:46.769+0000 7f41b5c92740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5615fc423ce0 0x5615fc429380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:17:46.789 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:46.769+0000 7f41b5c92740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x5615fc2b8ab0 con 0x5615fc41e970 2024-09-11T00:17:46.789 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:46.769+0000 7f41b5c92740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x5615fc327570 con 0x5615fc4212a0 2024-09-11T00:17:46.789 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:46.769+0000 7f41b5c92740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5615fc426cc0 con 0x5615fc423ce0 2024-09-11T00:17:46.789 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:46.769+0000 7f41b5c92740 10 client.0 _ll_register_callbacks cb 0x5615fc336100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:17:46.789 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:46.770+0000 7f41abe00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5615fc41e970 0x5615fc428110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:17:46.789 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:46.770+0000 7f41ab400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5615fc4212a0 0x5615fc4287b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:17:46.789 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:46.770+0000 7f41abe00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5615fc41e970 0x5615fc428110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:58688/0 (socket says 172.21.15.12:58688) 2024-09-11T00:17:46.789 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:46.770+0000 7f41abe00640 1 -- 172.21.15.12:0/643151555 learned_addr learned my addr 172.21.15.12:0/643151555 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:17:46.790 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:46.770+0000 7f41b0e00640 1 --2- 172.21.15.12:0/643151555 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5615fc423ce0 0x5615fc429380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:17:46.790 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:46.770+0000 7f41ab400640 1 -- 172.21.15.12:0/643151555 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5615fc423ce0 msgr2=0x5615fc429380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:17:46.790 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:46.770+0000 7f41ab400640 1 --2- 172.21.15.12:0/643151555 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5615fc423ce0 0x5615fc429380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:17:46.790 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:46.770+0000 7f41ab400640 1 -- 172.21.15.12:0/643151555 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5615fc41e970 msgr2=0x5615fc428110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:17:46.790 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:46.770+0000 7f41ab400640 1 --2- 172.21.15.12:0/643151555 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5615fc41e970 0x5615fc428110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:17:46.790 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:46.770+0000 7f41ab400640 1 -- 172.21.15.12:0/643151555 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5615fc0e2790 con 0x5615fc4212a0 2024-09-11T00:17:46.790 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:46.770+0000 7f41ab400640 1 --2- 172.21.15.12:0/643151555 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5615fc4212a0 0x5615fc4287b0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f419800c6a0 tx=0x7f41980ac0c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1e238971e59b7185 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:17:46.790 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:46.770+0000 7f4196000640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-09-11T00:17:46.790 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:46.771+0000 7f4196000640 1 -- 172.21.15.12:0/643151555 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41980ace70 con 0x5615fc4212a0 2024-09-11T00:17:46.791 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:46.771+0000 7f41b5c92740 1 -- 172.21.15.12:0/643151555 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f41a4015030 con 0x5615fc4212a0 2024-09-11T00:17:46.791 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:46.771+0000 7f4196000640 1 -- 172.21.15.12:0/643151555 <== mon.2 v2:172.21.15.165:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9208+0+0 (secure 0 0 0) 0x7f41980a89e0 con 0x5615fc4212a0 2024-09-11T00:17:46.791 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:46.771+0000 7f4196000640 1 -- 172.21.15.12:0/643151555 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f41980ac770 con 0x5615fc4212a0 2024-09-11T00:17:46.791 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:46.771+0000 7f41b5c92740 10 client.26711 fetch_fsmap learned FSMap version 41 2024-09-11T00:17:46.791 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:46.771+0000 7f41b5c92740 1 -- 172.21.15.12:0/643151555 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x5615fc15b620 con 0x5615fc4212a0 2024-09-11T00:17:46.791 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:46.771+0000 7f4196000640 1 -- 172.21.15.12:0/643151555 <== mon.2 v2:172.21.15.165:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f41a4015030 con 0x5615fc4212a0 2024-09-11T00:17:46.792 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:46.772+0000 7f4196000640 1 -- 172.21.15.12:0/643151555 <== mon.2 v2:172.21.15.165:3300/0 5 ==== fsmap(e 41) ==== 8370+0+0 (secure 0 0 0) 0x7f41980c3080 con 0x5615fc4212a0 2024-09-11T00:17:46.792 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:46.772+0000 7f41b5c92740 10 client.26711 fetch_fsmap finished waiting for FSMap version 41 2024-09-11T00:17:46.792 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:46.772+0000 7f41b5c92740 10 client.26711 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T00:17:46.792 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:46.772+0000 7f41b5c92740 20 client.26711 populate_metadata read hostname 'smithi012' 2024-09-11T00:17:46.792 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:46.772+0000 7f41b5c92740 1 --2- 172.21.15.12:0/643151555 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x5615fc433930 0x5615fc435d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:17:46.793 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:46.772+0000 7f41b5c92740 4 client.26711 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T00:17:46.793 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:46.772+0000 7f41abe00640 1 --2- 172.21.15.12:0/643151555 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x5615fc433930 0x5615fc435d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:17:46.793 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:46.772+0000 7f41b5c92740 1 -- 172.21.15.12:0/643151555 --> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5615fc393600 con 0x5615fc433930 2024-09-11T00:17:46.793 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:46.773+0000 7f41abe00640 1 --2- 172.21.15.12:0/643151555 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x5615fc433930 0x5615fc435d20 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fe1ed587bc32281d server_cookie=e003d74cb84c1988 in_seq=0 out_seq=0 2024-09-11T00:17:46.793 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:46.773+0000 7f4196000640 10 client.26711 ms_handle_connect on v2:172.21.15.165:6838/501312984 2024-09-11T00:17:46.794 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:46.773+0000 7f4196000640 1 -- 172.21.15.12:0/643151555 <== mds.0 v2:172.21.15.165:6838/501312984 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5615fc393600 con 0x5615fc433930 2024-09-11T00:17:46.794 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:46.773+0000 7f4196000640 10 client.26711 handle_command_reply: tid=0 2024-09-11T00:17:46.794 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:46.773+0000 7f41b5c92740 1 client.26711 shutdown 2024-09-11T00:17:46.794 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:46.774+0000 7f41b5c92740 1 -- 172.21.15.12:0/643151555 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5615fc4212a0 msgr2=0x5615fc4287b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:17:46.794 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:46.774+0000 7f41b5c92740 1 --2- 172.21.15.12:0/643151555 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5615fc4212a0 0x5615fc4287b0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f419800c6a0 tx=0x7f41980ac0c0 comp rx=0 tx=0).stop 2024-09-11T00:17:46.794 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:46.774+0000 7f41b5c92740 1 -- 172.21.15.12:0/643151555 shutdown_connections 2024-09-11T00:17:46.794 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:46.774+0000 7f41b5c92740 1 --2- 172.21.15.12:0/643151555 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x5615fc433930 0x5615fc435d20 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:17:46.795 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:46.774+0000 7f41b5c92740 1 --2- 172.21.15.12:0/643151555 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5615fc423ce0 0x5615fc429380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:17:46.795 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:46.774+0000 7f41b5c92740 1 --2- 172.21.15.12:0/643151555 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5615fc4212a0 0x5615fc4287b0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:17:46.795 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:46.774+0000 7f4196000640 0 client.26711 ms_handle_reset on v2:172.21.15.165:6838/501312984 2024-09-11T00:17:46.795 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:46.774+0000 7f41b5c92740 1 --2- 172.21.15.12:0/643151555 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5615fc41e970 0x5615fc428110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:17:46.795 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:46.774+0000 7f41b5c92740 1 -- 172.21.15.12:0/643151555 >> 172.21.15.12:0/643151555 conn(0x5615fc419ac0 msgr2=0x5615fc418bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:17:46.795 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:46.774+0000 7f41b5c92740 1 -- 172.21.15.12:0/643151555 shutdown_connections 2024-09-11T00:17:46.795 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:46.774+0000 7f41b5c92740 1 -- 172.21.15.12:0/643151555 wait complete. 2024-09-11T00:17:46.796 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:46.774+0000 7f41b5c92740 20 client.26711 trim_cache size 0 max 16384 2024-09-11T00:17:46.801 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:46.789+0000 7f41b5c92740 1 Processor -- start 2024-09-11T00:17:46.801 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:46.790+0000 7f41b5c92740 1 -- start start 2024-09-11T00:17:46.801 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:46.790+0000 7f41b5c92740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5615fc41e970 0x5615fc424170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:17:46.801 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:46.790+0000 7f41b5c92740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5615fc4212a0 0x5615fc4303e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:17:46.802 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:46.790+0000 7f41b5c92740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5615fc423ce0 0x5615fc4246b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:17:46.802 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:46.790+0000 7f41b5c92740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f41ac1ad760 con 0x5615fc41e970 2024-09-11T00:17:46.802 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:46.790+0000 7f41b5c92740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f41ac1af2a0 con 0x5615fc4212a0 2024-09-11T00:17:46.802 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:46.790+0000 7f41b5c92740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f41ac1ade30 con 0x5615fc423ce0 2024-09-11T00:17:46.802 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:46.790+0000 7f41abe00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5615fc41e970 0x5615fc424170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:17:46.802 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:46.790+0000 7f41abe00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5615fc41e970 0x5615fc424170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:58704/0 (socket says 172.21.15.12:58704) 2024-09-11T00:17:46.802 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:46.790+0000 7f41abe00640 1 -- 172.21.15.12:0/717798998 learned_addr learned my addr 172.21.15.12:0/717798998 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:17:46.802 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:46.790+0000 7f41b0e00640 1 --2- 172.21.15.12:0/717798998 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5615fc423ce0 0x5615fc4246b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:17:46.802 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:46.790+0000 7f41ab400640 1 --2- 172.21.15.12:0/717798998 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5615fc4212a0 0x5615fc4303e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:17:46.802 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:46.790+0000 7f41abe00640 1 -- 172.21.15.12:0/717798998 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5615fc423ce0 msgr2=0x5615fc4246b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:17:46.802 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:46.790+0000 7f41abe00640 1 --2- 172.21.15.12:0/717798998 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5615fc423ce0 0x5615fc4246b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:17:46.803 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:46.790+0000 7f41abe00640 1 -- 172.21.15.12:0/717798998 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5615fc4212a0 msgr2=0x5615fc4303e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:17:46.803 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:46.790+0000 7f41abe00640 1 --2- 172.21.15.12:0/717798998 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5615fc4212a0 0x5615fc4303e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:17:46.803 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:46.790+0000 7f41abe00640 1 -- 172.21.15.12:0/717798998 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5615fc436090 con 0x5615fc41e970 2024-09-11T00:17:46.803 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:46.790+0000 7f41ab400640 1 --2- 172.21.15.12:0/717798998 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5615fc4212a0 0x5615fc4303e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T00:17:46.803 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:46.791+0000 7f41abe00640 1 --2- 172.21.15.12:0/717798998 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5615fc41e970 0x5615fc424170 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f419c005740 tx=0x7f419c02ab70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7530ca5e57aa9b45 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:17:46.803 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:46.791+0000 7f41a0c00640 1 -- 172.21.15.12:0/717798998 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f419c035040 con 0x5615fc41e970 2024-09-11T00:17:46.803 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:46.791+0000 7f41a0c00640 1 -- 172.21.15.12:0/717798998 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f419c003d60 con 0x5615fc41e970 2024-09-11T00:17:46.803 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:46.791+0000 7f41a0c00640 1 -- 172.21.15.12:0/717798998 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f419c031df0 con 0x5615fc41e970 2024-09-11T00:17:46.803 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:46.792+0000 7f41b5c92740 1 -- 172.21.15.12:0/717798998 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5615fc41e970 msgr2=0x5615fc424170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:17:46.803 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:46.792+0000 7f41b5c92740 1 --2- 172.21.15.12:0/717798998 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5615fc41e970 0x5615fc424170 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f419c005740 tx=0x7f419c02ab70 comp rx=0 tx=0).stop 2024-09-11T00:17:46.803 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:46.792+0000 7f41b5c92740 1 -- 172.21.15.12:0/717798998 shutdown_connections 2024-09-11T00:17:46.803 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:46.792+0000 7f41b5c92740 1 --2- 172.21.15.12:0/717798998 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5615fc423ce0 0x5615fc4246b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:17:46.803 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:46.792+0000 7f41b5c92740 1 --2- 172.21.15.12:0/717798998 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5615fc4212a0 0x5615fc4303e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:17:46.804 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:46.792+0000 7f41b5c92740 1 --2- 172.21.15.12:0/717798998 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5615fc41e970 0x5615fc424170 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:17:46.804 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:46.792+0000 7f41b5c92740 1 -- 172.21.15.12:0/717798998 >> 172.21.15.12:0/717798998 conn(0x5615fc419ac0 msgr2=0x5615fc4272d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:17:46.804 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:46.792+0000 7f41b5c92740 1 -- 172.21.15.12:0/717798998 shutdown_connections 2024-09-11T00:17:46.804 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:46.792+0000 7f41b5c92740 1 -- 172.21.15.12:0/717798998 wait complete. 2024-09-11T00:17:46.804 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:46.792+0000 7f41b5c92740 1 -- start start 2024-09-11T00:17:46.804 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:46.793+0000 7f41b5c92740 1 Processor -- start 2024-09-11T00:17:46.804 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:46.793+0000 7f41b5c92740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5615fc41e970 0x5615fc4301f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:17:46.804 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:46.793+0000 7f41b5c92740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5615fc4212a0 0x5615fc42eef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:17:46.804 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:46.793+0000 7f41b5c92740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5615fc423ce0 0x5615fc42f550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:17:46.804 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:46.793+0000 7f41b5c92740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f41ac1aebd0 con 0x5615fc4212a0 2024-09-11T00:17:46.804 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:46.793+0000 7f41b5c92740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f41ac1ae500 con 0x5615fc423ce0 2024-09-11T00:17:46.804 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:46.793+0000 7f41b5c92740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f41ac1ab740 con 0x5615fc41e970 2024-09-11T00:17:46.805 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:46.793+0000 7f41b5c92740 10 client.0 _ll_register_callbacks cb 0x7f419800cf20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:17:46.805 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:46.793+0000 7f41abe00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5615fc41e970 0x5615fc4301f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:17:46.805 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:46.793+0000 7f41abe00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5615fc41e970 0x5615fc4301f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.12:48478/0 (socket says 172.21.15.12:48478) 2024-09-11T00:17:46.805 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:46.793+0000 7f41abe00640 1 -- 172.21.15.12:0/408396885 learned_addr learned my addr 172.21.15.12:0/408396885 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:17:46.805 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:46.793+0000 7f41b0e00640 1 --2- 172.21.15.12:0/408396885 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5615fc423ce0 0x5615fc42f550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:17:46.805 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:46.793+0000 7f41ab400640 1 --2- 172.21.15.12:0/408396885 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5615fc4212a0 0x5615fc42eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:17:46.805 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:46.794+0000 7f41abe00640 1 -- 172.21.15.12:0/408396885 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5615fc423ce0 msgr2=0x5615fc42f550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:17:46.805 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:46.794+0000 7f41abe00640 1 --2- 172.21.15.12:0/408396885 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5615fc423ce0 0x5615fc42f550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:17:46.805 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:46.794+0000 7f41abe00640 1 -- 172.21.15.12:0/408396885 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5615fc4212a0 msgr2=0x5615fc42eef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:17:46.805 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:46.794+0000 7f41abe00640 1 --2- 172.21.15.12:0/408396885 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5615fc4212a0 0x5615fc42eef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:17:46.805 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:46.794+0000 7f41abe00640 1 -- 172.21.15.12:0/408396885 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5615fc42da10 con 0x5615fc41e970 2024-09-11T00:17:46.806 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:46.794+0000 7f41ab400640 1 --2- 172.21.15.12:0/408396885 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5615fc4212a0 0x5615fc42eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T00:17:46.806 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:46.794+0000 7f41abe00640 1 --2- 172.21.15.12:0/408396885 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5615fc41e970 0x5615fc4301f0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f419c000c00 tx=0x7f419c0317e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8e63a8a0a81b3d7e server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:17:46.806 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:46.794+0000 7f4196000640 10 client.0 ms_handle_connect on v2:172.21.15.178:3300/0 2024-09-11T00:17:46.806 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:46.794+0000 7f4196000640 1 -- 172.21.15.12:0/408396885 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f419c056020 con 0x5615fc41e970 2024-09-11T00:17:46.806 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:46.794+0000 7f41b5c92740 1 -- 172.21.15.12:0/408396885 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f419c003d60 con 0x5615fc41e970 2024-09-11T00:17:46.806 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:46.794+0000 7f4196000640 1 -- 172.21.15.12:0/408396885 <== mon.1 v2:172.21.15.178:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9208+0+0 (secure 0 0 0) 0x7f419c0036b0 con 0x5615fc41e970 2024-09-11T00:17:46.806 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:46.795+0000 7f4196000640 1 -- 172.21.15.12:0/408396885 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f419c061050 con 0x5615fc41e970 2024-09-11T00:17:46.806 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:46.795+0000 7f41b5c92740 10 client.26698 fetch_fsmap learned FSMap version 41 2024-09-11T00:17:46.806 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:46.795+0000 7f41b5c92740 1 -- 172.21.15.12:0/408396885 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x5615fc4a09e0 con 0x5615fc41e970 2024-09-11T00:17:46.806 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:46.795+0000 7f4196000640 1 -- 172.21.15.12:0/408396885 <== mon.1 v2:172.21.15.178:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f419c003d60 con 0x5615fc41e970 2024-09-11T00:17:46.806 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:46.795+0000 7f4196000640 1 -- 172.21.15.12:0/408396885 <== mon.1 v2:172.21.15.178:3300/0 5 ==== fsmap(e 41) ==== 8370+0+0 (secure 0 0 0) 0x7f41980051b0 con 0x5615fc41e970 2024-09-11T00:17:46.806 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:46.795+0000 7f41b5c92740 10 client.26698 fetch_fsmap finished waiting for FSMap version 41 2024-09-11T00:17:46.807 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:46.795+0000 7f41b5c92740 10 client.26698 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T00:17:46.807 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:46.795+0000 7f41b5c92740 20 client.26698 populate_metadata read hostname 'smithi012' 2024-09-11T00:17:46.807 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:46.795+0000 7f41b5c92740 1 --2- 172.21.15.12:0/408396885 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x5615fc4a13c0 0x5615fc4a37e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:17:46.807 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:46.795+0000 7f41b5c92740 4 client.26698 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T00:17:46.807 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:46.795+0000 7f41b5c92740 1 -- 172.21.15.12:0/408396885 --> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5615fc38f0b0 con 0x5615fc4a13c0 2024-09-11T00:17:46.807 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:46.796+0000 7f41ab400640 1 --2- 172.21.15.12:0/408396885 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x5615fc4a13c0 0x5615fc4a37e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:17:46.807 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:46.796+0000 7f41ab400640 1 --2- 172.21.15.12:0/408396885 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x5615fc4a13c0 0x5615fc4a37e0 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3e45bb782f179d82 server_cookie=f6ea02a29f01b9e1 in_seq=0 out_seq=0 2024-09-11T00:17:46.807 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:46.796+0000 7f4196000640 10 client.26698 ms_handle_connect on v2:172.21.15.165:6838/501312984 2024-09-11T00:17:46.866 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:46.864+0000 7f4196000640 1 -- 172.21.15.12:0/408396885 <== mds.0 v2:172.21.15.165:6838/501312984 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5615fc38f0b0 con 0x5615fc4a13c0 2024-09-11T00:17:46.866 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:46.864+0000 7f4196000640 10 client.26698 handle_command_reply: tid=0 2024-09-11T00:17:46.866 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:46.864+0000 7f41b5c92740 1 client.26698 shutdown 2024-09-11T00:17:46.866 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:46.864+0000 7f41b5c92740 1 -- 172.21.15.12:0/408396885 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5615fc41e970 msgr2=0x5615fc4301f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:17:46.866 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:46.864+0000 7f41b5c92740 1 --2- 172.21.15.12:0/408396885 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5615fc41e970 0x5615fc4301f0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f419c000c00 tx=0x7f419c0317e0 comp rx=0 tx=0).stop 2024-09-11T00:17:46.866 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:46.865+0000 7f41b5c92740 1 -- 172.21.15.12:0/408396885 shutdown_connections 2024-09-11T00:17:46.867 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:46.865+0000 7f41b5c92740 1 --2- 172.21.15.12:0/408396885 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x5615fc4a13c0 0x5615fc4a37e0 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:17:46.867 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:46.865+0000 7f41b5c92740 1 --2- 172.21.15.12:0/408396885 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5615fc423ce0 0x5615fc42f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:17:46.867 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:46.865+0000 7f41b5c92740 1 --2- 172.21.15.12:0/408396885 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5615fc4212a0 0x5615fc42eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:17:46.867 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:46.865+0000 7f4196000640 0 client.26698 ms_handle_reset on v2:172.21.15.165:6838/501312984 2024-09-11T00:17:46.867 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:46.865+0000 7f41b5c92740 1 --2- 172.21.15.12:0/408396885 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5615fc41e970 0x5615fc4301f0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:17:46.867 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:46.865+0000 7f41b5c92740 1 -- 172.21.15.12:0/408396885 >> 172.21.15.12:0/408396885 conn(0x5615fc419ac0 msgr2=0x5615fc4272d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:17:46.867 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:46.865+0000 7f41b5c92740 1 -- 172.21.15.12:0/408396885 shutdown_connections 2024-09-11T00:17:46.867 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:46.865+0000 7f41b5c92740 1 -- 172.21.15.12:0/408396885 wait complete. 2024-09-11T00:17:46.867 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:46.865+0000 7f41b5c92740 20 client.26698 trim_cache size 0 max 16384 2024-09-11T00:17:46.868 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:{ 2024-09-11T00:17:46.868 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "message": "", 2024-09-11T00:17:46.868 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "return_code": 0 2024-09-11T00:17:46.868 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:} 2024-09-11T00:17:46.868 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:46.866+0000 7f41b1800640 1 -- 172.21.15.12:0/1671245056 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f41900b5270 msgr2=0x7f41900b7730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:17:46.869 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:46.866+0000 7f41b1800640 1 --2- 172.21.15.12:0/1671245056 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f41900b5270 0x7f41900b7730 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f41a4009d90 tx=0x7f41a4000f40 comp rx=0 tx=0).stop 2024-09-11T00:17:46.869 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:46.866+0000 7f41b1800640 1 -- 172.21.15.12:0/1671245056 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f41ac105b20 msgr2=0x7f41ac19b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:17:46.869 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:46.866+0000 7f41b1800640 1 --2- 172.21.15.12:0/1671245056 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f41ac105b20 0x7f41ac19b430 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f419800e9f0 tx=0x7f419800edf0 comp rx=0 tx=0).stop 2024-09-11T00:17:46.869 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:46.866+0000 7f41b1800640 1 -- 172.21.15.12:0/1671245056 shutdown_connections 2024-09-11T00:17:46.869 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:46.866+0000 7f41b1800640 1 --2- 172.21.15.12:0/1671245056 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f41900b5270 0x7f41900b7730 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:17:46.869 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:46.866+0000 7f41b1800640 1 --2- 172.21.15.12:0/1671245056 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f41ac1064e0 0x7f41ac19c060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:17:46.869 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:46.866+0000 7f41b1800640 1 --2- 172.21.15.12:0/1671245056 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f41ac105b20 0x7f41ac19b430 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:17:46.869 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:46.866+0000 7f41b1800640 1 --2- 172.21.15.12:0/1671245056 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f41ac1049d0 0x7f41ac19ade0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:17:46.869 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:46.866+0000 7f41b1800640 1 -- 172.21.15.12:0/1671245056 >> 172.21.15.12:0/1671245056 conn(0x7f41ac100270 msgr2=0x7f41ac108f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:17:46.869 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:46.866+0000 7f41b1800640 1 -- 172.21.15.12:0/1671245056 shutdown_connections 2024-09-11T00:17:46.869 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:46.866+0000 7f41b1800640 1 -- 172.21.15.12:0/1671245056 wait complete. 2024-09-11T00:17:47.688 INFO:tasks.workunit.client.0.smithi012.stdout: 1 452356 4.13 MB/sec execute 464 sec latency 61.206 ms 2024-09-11T00:17:48.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:17:47 smithi165 ceph-mon[35356]: pgmap v720: 129 pgs: 129 active+clean; 113 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 1.4 MiB/s wr, 326 op/s 2024-09-11T00:17:48.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:17:47 smithi012 ceph-mon[30333]: pgmap v720: 129 pgs: 129 active+clean; 113 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 1.4 MiB/s wr, 326 op/s 2024-09-11T00:17:48.330 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:17:47 smithi178 ceph-mon[35919]: pgmap v720: 129 pgs: 129 active+clean; 113 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 1.4 MiB/s wr, 326 op/s 2024-09-11T00:17:48.688 INFO:tasks.workunit.client.0.smithi012.stdout: 1 454308 4.15 MB/sec execute 465 sec latency 11.565 ms 2024-09-11T00:17:49.688 INFO:tasks.workunit.client.0.smithi012.stdout: 1 455606 4.15 MB/sec execute 466 sec latency 12.338 ms 2024-09-11T00:17:50.148 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:17:49 smithi012 ceph-mon[30333]: pgmap v721: 129 pgs: 129 active+clean; 113 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1015 KiB/s wr, 205 op/s 2024-09-11T00:17:50.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:17:49 smithi165 ceph-mon[35356]: pgmap v721: 129 pgs: 129 active+clean; 113 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1015 KiB/s wr, 205 op/s 2024-09-11T00:17:50.330 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:17:49 smithi178 ceph-mon[35919]: pgmap v721: 129 pgs: 129 active+clean; 113 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1015 KiB/s wr, 205 op/s 2024-09-11T00:17:50.688 INFO:tasks.workunit.client.0.smithi012.stdout: 1 456239 4.15 MB/sec execute 467 sec latency 373.024 ms 2024-09-11T00:17:51.688 INFO:tasks.workunit.client.0.smithi012.stdout: 1 456468 4.14 MB/sec execute 468 sec latency 1219.719 ms 2024-09-11T00:17:52.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:17:51 smithi165 ceph-mon[35356]: pgmap v722: 129 pgs: 129 active+clean; 117 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 1.5 MiB/s wr, 240 op/s 2024-09-11T00:17:52.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:17:51 smithi012 ceph-mon[30333]: pgmap v722: 129 pgs: 129 active+clean; 117 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 1.5 MiB/s wr, 240 op/s 2024-09-11T00:17:52.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:17:51 smithi178 ceph-mon[35919]: pgmap v722: 129 pgs: 129 active+clean; 117 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 1.5 MiB/s wr, 240 op/s 2024-09-11T00:17:52.688 INFO:tasks.workunit.client.0.smithi012.stdout: 1 457970 4.16 MB/sec execute 469 sec latency 23.168 ms 2024-09-11T00:17:53.688 INFO:tasks.workunit.client.0.smithi012.stdout: 1 458797 4.18 MB/sec execute 470 sec latency 60.204 ms 2024-09-11T00:17:54.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:17:53 smithi165 ceph-mon[35356]: pgmap v723: 129 pgs: 129 active+clean; 117 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 1.6 MiB/s wr, 296 op/s 2024-09-11T00:17:54.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:17:53 smithi012 ceph-mon[30333]: pgmap v723: 129 pgs: 129 active+clean; 117 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 1.6 MiB/s wr, 296 op/s 2024-09-11T00:17:54.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:17:53 smithi178 ceph-mon[35919]: pgmap v723: 129 pgs: 129 active+clean; 117 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 1.6 MiB/s wr, 296 op/s 2024-09-11T00:17:54.688 INFO:tasks.workunit.client.0.smithi012.stdout: 1 461224 4.21 MB/sec execute 471 sec latency 12.981 ms 2024-09-11T00:17:55.688 INFO:tasks.workunit.client.0.smithi012.stdout: 1 461838 4.21 MB/sec execute 472 sec latency 665.872 ms 2024-09-11T00:17:56.223 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:17:55 smithi012 ceph-mon[30333]: pgmap v724: 129 pgs: 129 active+clean; 117 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.4 MiB/s wr, 213 op/s 2024-09-11T00:17:56.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:17:55 smithi165 ceph-mon[35356]: pgmap v724: 129 pgs: 129 active+clean; 117 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.4 MiB/s wr, 213 op/s 2024-09-11T00:17:56.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:17:55 smithi178 ceph-mon[35919]: pgmap v724: 129 pgs: 129 active+clean; 117 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.4 MiB/s wr, 213 op/s 2024-09-11T00:17:56.689 INFO:tasks.workunit.client.0.smithi012.stdout: 1 462271 4.21 MB/sec execute 473 sec latency 987.247 ms 2024-09-11T00:17:57.689 INFO:tasks.workunit.client.0.smithi012.stdout: 1 463412 4.21 MB/sec execute 474 sec latency 211.947 ms 2024-09-11T00:17:58.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:17:57 smithi165 ceph-mon[35356]: pgmap v725: 129 pgs: 129 active+clean; 112 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.1 MiB/s wr, 302 op/s 2024-09-11T00:17:58.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:17:57 smithi012 ceph-mon[30333]: pgmap v725: 129 pgs: 129 active+clean; 112 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.1 MiB/s wr, 302 op/s 2024-09-11T00:17:58.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:17:57 smithi178 ceph-mon[35919]: pgmap v725: 129 pgs: 129 active+clean; 112 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.1 MiB/s wr, 302 op/s 2024-09-11T00:17:58.689 INFO:tasks.workunit.client.0.smithi012.stdout: 1 465251 4.22 MB/sec execute 475 sec latency 303.537 ms 2024-09-11T00:17:59.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:17:58 smithi165 ceph-mon[35356]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T00:17:59.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:17:58 smithi012 ceph-mon[30333]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T00:17:59.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:17:58 smithi178 ceph-mon[35919]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T00:17:59.689 INFO:tasks.workunit.client.0.smithi012.stdout: 1 465264 4.21 MB/sec execute 476 sec latency 1156.825 ms 2024-09-11T00:17:59.996 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:59.993+0000 7f9516a00640 1 -- 172.21.15.12:0/2566848213 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9510104b90 msgr2=0x7f9510104f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:17:59.996 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:59.993+0000 7f9516a00640 1 --2- 172.21.15.12:0/2566848213 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9510104b90 0x7f9510104f90 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f9500009ca0 tx=0x7f950002a8c0 comp rx=0 tx=0).stop 2024-09-11T00:17:59.996 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:59.993+0000 7f9516a00640 1 -- 172.21.15.12:0/2566848213 shutdown_connections 2024-09-11T00:17:59.996 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:59.993+0000 7f9516a00640 1 --2- 172.21.15.12:0/2566848213 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f95101066a0 0x7f951010d020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:17:59.996 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:59.993+0000 7f9516a00640 1 --2- 172.21.15.12:0/2566848213 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9510105ce0 0x7f9510106160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:17:59.996 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:59.993+0000 7f9516a00640 1 --2- 172.21.15.12:0/2566848213 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9510104b90 0x7f9510104f90 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:17:59.996 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:59.993+0000 7f9516a00640 1 -- 172.21.15.12:0/2566848213 >> 172.21.15.12:0/2566848213 conn(0x7f9510100470 msgr2=0x7f9510102890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:17:59.996 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:59.993+0000 7f9516a00640 1 -- 172.21.15.12:0/2566848213 shutdown_connections 2024-09-11T00:17:59.997 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:59.993+0000 7f9516a00640 1 -- 172.21.15.12:0/2566848213 wait complete. 2024-09-11T00:17:59.998 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:59.994+0000 7f9516a00640 1 Processor -- start 2024-09-11T00:17:59.998 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:59.994+0000 7f9516a00640 1 -- start start 2024-09-11T00:17:59.999 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:59.995+0000 7f9516a00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9510104b90 0x7f951019afb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:17:59.999 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:59.995+0000 7f9516a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9510105ce0 0x7f951019b600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:17:59.999 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:59.995+0000 7f9516a00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f95101066a0 0x7f951019c230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:17:59.999 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:59.995+0000 7f9516a00640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f9510110830 con 0x7f9510104b90 2024-09-11T00:17:59.999 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:59.995+0000 7f9516a00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f95101106b0 con 0x7f9510105ce0 2024-09-11T00:17:59.999 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:59.995+0000 7f9516a00640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f95101109b0 con 0x7f95101066a0 2024-09-11T00:17:59.999 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:59.995+0000 7f950fe00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9510104b90 0x7f951019afb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:17:59.999 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:59.995+0000 7f950fe00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9510104b90 0x7f951019afb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:54906/0 (socket says 172.21.15.12:54906) 2024-09-11T00:17:59.999 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:59.995+0000 7f950fe00640 1 -- 172.21.15.12:0/1498839943 learned_addr learned my addr 172.21.15.12:0/1498839943 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:17:59.999 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:59.995+0000 7f9507400640 1 --2- 172.21.15.12:0/1498839943 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9510105ce0 0x7f951019b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:17:59.999 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:59.995+0000 7f9514a00640 1 --2- 172.21.15.12:0/1498839943 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f95101066a0 0x7f951019c230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:18:00.000 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:59.995+0000 7f950fe00640 1 -- 172.21.15.12:0/1498839943 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f95101066a0 msgr2=0x7f951019c230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:18:00.000 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:59.995+0000 7f950fe00640 1 --2- 172.21.15.12:0/1498839943 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f95101066a0 0x7f951019c230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:18:00.000 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:59.995+0000 7f950fe00640 1 -- 172.21.15.12:0/1498839943 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9510105ce0 msgr2=0x7f951019b600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:18:00.000 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:59.995+0000 7f950fe00640 1 --2- 172.21.15.12:0/1498839943 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9510105ce0 0x7f951019b600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:18:00.000 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:59.995+0000 7f950fe00640 1 -- 172.21.15.12:0/1498839943 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f951019caf0 con 0x7f9510104b90 2024-09-11T00:18:00.000 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:59.996+0000 7f950fe00640 1 --2- 172.21.15.12:0/1498839943 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9510104b90 0x7f951019afb0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f9500000f80 tx=0x7f95000317c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:18:00.000 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:59.996+0000 7f950d600640 1 -- 172.21.15.12:0/1498839943 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9500003bf0 con 0x7f9510104b90 2024-09-11T00:18:00.000 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:59.996+0000 7f950d600640 1 -- 172.21.15.12:0/1498839943 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9500003d90 con 0x7f9510104b90 2024-09-11T00:18:00.000 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:59.996+0000 7f9516a00640 1 -- 172.21.15.12:0/1498839943 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f951019cd80 con 0x7f9510104b90 2024-09-11T00:18:00.000 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:59.996+0000 7f950d600640 1 -- 172.21.15.12:0/1498839943 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f950002fb40 con 0x7f9510104b90 2024-09-11T00:18:00.000 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:17:59.996+0000 7f9516a00640 1 -- 172.21.15.12:0/1498839943 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95101a34b0 con 0x7f9510104b90 2024-09-11T00:18:00.002 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:00.000+0000 7f9519880740 1 Processor -- start 2024-09-11T00:18:00.003 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:00.000+0000 7f9519880740 1 -- start start 2024-09-11T00:18:00.003 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:00.000+0000 7f950d600640 1 -- 172.21.15.12:0/1498839943 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7f9500004610 con 0x7f9510104b90 2024-09-11T00:18:00.003 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:00.000+0000 7f9519880740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556beda5e790 0x556beda60b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:18:00.003 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:00.000+0000 7f9519880740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556beda610c0 0x556beda634b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:18:00.003 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:00.000+0000 7f9519880740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556beda63b00 0x556beda65fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:18:00.003 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:00.000+0000 7f9519880740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x556bed9d16b0 con 0x556beda5e790 2024-09-11T00:18:00.003 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:00.000+0000 7f9519880740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x556bed9c0ae0 con 0x556beda63b00 2024-09-11T00:18:00.003 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:00.000+0000 7f9519880740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x556bed9c48f0 con 0x556beda610c0 2024-09-11T00:18:00.003 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:00.000+0000 7f9514a00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556beda5e790 0x556beda60b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:18:00.003 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:00.000+0000 7f9507400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556beda63b00 0x556beda65fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:18:00.003 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:00.000+0000 7f9514a00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556beda5e790 0x556beda60b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:54920/0 (socket says 172.21.15.12:54920) 2024-09-11T00:18:00.004 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:00.001+0000 7f9514a00640 1 -- 172.21.15.12:0/503029714 learned_addr learned my addr 172.21.15.12:0/503029714 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:18:00.004 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:00.000+0000 7f9507400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556beda63b00 0x556beda65fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.12:53576/0 (socket says 172.21.15.12:53576) 2024-09-11T00:18:00.007 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:00.001+0000 7f9514a00640 1 -- 172.21.15.12:0/503029714 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556beda610c0 msgr2=0x556beda634b0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-11T00:18:00.007 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:00.001+0000 7f9514a00640 1 --2- 172.21.15.12:0/503029714 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556beda610c0 0x556beda634b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:18:00.007 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:00.001+0000 7f9514a00640 1 -- 172.21.15.12:0/503029714 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556beda63b00 msgr2=0x556beda65fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:18:00.007 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:00.001+0000 7f9514a00640 1 --2- 172.21.15.12:0/503029714 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556beda63b00 0x556beda65fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:18:00.007 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:00.001+0000 7f9514a00640 1 -- 172.21.15.12:0/503029714 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556bed98d780 con 0x556beda5e790 2024-09-11T00:18:00.007 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:00.001+0000 7f9514a00640 1 --2- 172.21.15.12:0/503029714 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556beda5e790 0x556beda60b80 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f9508009c30 tx=0x7f9508011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d81f0402581eb12 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:18:00.007 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:00.002+0000 7f9505200640 1 -- 172.21.15.12:0/503029714 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9508004230 con 0x556beda5e790 2024-09-11T00:18:00.007 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:00.002+0000 7f9505200640 1 -- 172.21.15.12:0/503029714 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f95080043d0 con 0x556beda5e790 2024-09-11T00:18:00.007 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:00.002+0000 7f9505200640 1 -- 172.21.15.12:0/503029714 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9508019640 con 0x556beda5e790 2024-09-11T00:18:00.008 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:00.002+0000 7f950d600640 1 --2- 172.21.15.12:0/1498839943 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f94f40b52e0 0x7f94f40b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:18:00.008 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:00.002+0000 7f950d600640 1 -- 172.21.15.12:0/1498839943 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9208+0+0 (secure 0 0 0) 0x7f9500107310 con 0x7f9510104b90 2024-09-11T00:18:00.008 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:00.002+0000 7f950fe00640 1 --2- 172.21.15.12:0/1498839943 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f94f40b52e0 0x7f94f40b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:18:00.008 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:00.002+0000 7f9519880740 1 -- 172.21.15.12:0/503029714 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556beda5e790 msgr2=0x556beda60b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:18:00.008 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:00.002+0000 7f9519880740 1 --2- 172.21.15.12:0/503029714 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556beda5e790 0x556beda60b80 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f9508009c30 tx=0x7f9508011810 comp rx=0 tx=0).stop 2024-09-11T00:18:00.008 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:00.003+0000 7f9519880740 1 -- 172.21.15.12:0/503029714 shutdown_connections 2024-09-11T00:18:00.008 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:00.003+0000 7f9519880740 1 --2- 172.21.15.12:0/503029714 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556beda63b00 0x556beda65fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:18:00.008 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:00.003+0000 7f9519880740 1 --2- 172.21.15.12:0/503029714 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556beda610c0 0x556beda634b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:18:00.008 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:00.003+0000 7f9519880740 1 --2- 172.21.15.12:0/503029714 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556beda5e790 0x556beda60b80 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:18:00.008 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:00.003+0000 7f9519880740 1 -- 172.21.15.12:0/503029714 >> 172.21.15.12:0/503029714 conn(0x556beda598e0 msgr2=0x556beda5a670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:18:00.009 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:00.003+0000 7f9519880740 1 -- 172.21.15.12:0/503029714 shutdown_connections 2024-09-11T00:18:00.009 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:00.003+0000 7f9519880740 1 -- 172.21.15.12:0/503029714 wait complete. 2024-09-11T00:18:00.009 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:00.003+0000 7f950fe00640 1 --2- 172.21.15.12:0/1498839943 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f94f40b52e0 0x7f94f40b77a0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f950002ae80 tx=0x7f9500003f40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:18:00.009 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:00.003+0000 7f9519880740 1 -- start start 2024-09-11T00:18:00.009 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:00.004+0000 7f9519880740 1 Processor -- start 2024-09-11T00:18:00.009 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:00.004+0000 7f9519880740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556beda5e790 0x556beda67f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:18:00.009 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:00.004+0000 7f9519880740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556beda610c0 0x556beda685b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:18:00.009 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:00.004+0000 7f9519880740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556beda63b00 0x556beda69180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:18:00.009 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:00.004+0000 7f9519880740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x556bed8f8ab0 con 0x556beda63b00 2024-09-11T00:18:00.009 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:00.004+0000 7f9519880740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x556bed967570 con 0x556beda610c0 2024-09-11T00:18:00.009 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:00.004+0000 7f9519880740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x556beda66b30 con 0x556beda5e790 2024-09-11T00:18:00.009 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:00.004+0000 7f9519880740 10 client.0 _ll_register_callbacks cb 0x556bed976100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:18:00.009 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:00.004+0000 7f9514a00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556beda63b00 0x556beda69180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:18:00.010 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:00.004+0000 7f9514a00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556beda63b00 0x556beda69180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:54922/0 (socket says 172.21.15.12:54922) 2024-09-11T00:18:00.010 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:00.004+0000 7f9514a00640 1 -- 172.21.15.12:0/2931401294 learned_addr learned my addr 172.21.15.12:0/2931401294 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:18:00.010 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:00.004+0000 7f9514a00640 1 --2- 172.21.15.12:0/2931401294 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556beda5e790 0x556beda67f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:18:00.010 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:00.005+0000 7f9507400640 1 --2- 172.21.15.12:0/2931401294 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556beda610c0 0x556beda685b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:18:00.010 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:00.005+0000 7f9514a00640 1 -- 172.21.15.12:0/2931401294 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556beda5e790 msgr2=0x556beda67f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:18:00.010 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:00.005+0000 7f9514a00640 1 --2- 172.21.15.12:0/2931401294 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556beda5e790 0x556beda67f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:18:00.010 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:00.005+0000 7f9514a00640 1 -- 172.21.15.12:0/2931401294 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556beda610c0 msgr2=0x556beda685b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:18:00.010 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:00.005+0000 7f9514a00640 1 --2- 172.21.15.12:0/2931401294 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556beda610c0 0x556beda685b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:18:00.010 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:00.005+0000 7f9514a00640 1 -- 172.21.15.12:0/2931401294 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556bed722790 con 0x556beda63b00 2024-09-11T00:18:00.013 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:00.005+0000 7f9514a00640 1 --2- 172.21.15.12:0/2931401294 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556beda63b00 0x556beda69180 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f9508005bc0 tx=0x7f9508009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34a0f1709213154d server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:18:00.013 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:00.005+0000 7f94fd600640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-09-11T00:18:00.013 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:00.006+0000 7f94fd600640 1 -- 172.21.15.12:0/2931401294 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9508003b80 con 0x556beda63b00 2024-09-11T00:18:00.013 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:00.006+0000 7f9519880740 1 -- 172.21.15.12:0/2931401294 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f95080043d0 con 0x556beda63b00 2024-09-11T00:18:00.013 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:00.006+0000 7f94fd600640 1 -- 172.21.15.12:0/2931401294 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9208+0+0 (secure 0 0 0) 0x7f9508005020 con 0x556beda63b00 2024-09-11T00:18:00.013 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:00.006+0000 7f94fd600640 1 -- 172.21.15.12:0/2931401294 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f9508005220 con 0x556beda63b00 2024-09-11T00:18:00.013 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:00.006+0000 7f9519880740 10 client.16728 fetch_fsmap learned FSMap version 42 2024-09-11T00:18:00.013 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:00.006+0000 7f9519880740 1 -- 172.21.15.12:0/2931401294 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x556bed79b620 con 0x556beda63b00 2024-09-11T00:18:00.014 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:00.006+0000 7f94fd600640 1 -- 172.21.15.12:0/2931401294 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f95080043d0 con 0x556beda63b00 2024-09-11T00:18:00.014 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:00.007+0000 7f94fd600640 1 -- 172.21.15.12:0/2931401294 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 42) ==== 8374+0+0 (secure 0 0 0) 0x7f9508018070 con 0x556beda63b00 2024-09-11T00:18:00.014 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:00.007+0000 7f9519880740 10 client.16728 fetch_fsmap finished waiting for FSMap version 42 2024-09-11T00:18:00.014 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:00.007+0000 7f9519880740 10 client.16728 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T00:18:00.014 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:00.007+0000 7f9519880740 20 client.16728 populate_metadata read hostname 'smithi012' 2024-09-11T00:18:00.014 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:00.007+0000 7f9519880740 1 --2- 172.21.15.12:0/2931401294 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x556beda73590 0x556beda75980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:18:00.014 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:00.007+0000 7f9519880740 4 client.16728 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T00:18:00.014 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:00.007+0000 7f9519880740 1 -- 172.21.15.12:0/2931401294 --> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556bed9d3600 con 0x556beda73590 2024-09-11T00:18:00.015 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:00.007+0000 7f9507400640 1 --2- 172.21.15.12:0/2931401294 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x556beda73590 0x556beda75980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:18:00.015 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:00.008+0000 7f9507400640 1 --2- 172.21.15.12:0/2931401294 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x556beda73590 0x556beda75980 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a90d7b4843fe3568 server_cookie=66cef2611ddb64bd in_seq=0 out_seq=0 2024-09-11T00:18:00.015 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:00.008+0000 7f94fd600640 10 client.16728 ms_handle_connect on v2:172.21.15.165:6838/501312984 2024-09-11T00:18:00.015 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:00.008+0000 7f94fd600640 1 -- 172.21.15.12:0/2931401294 <== mds.0 v2:172.21.15.165:6838/501312984 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x556bed9d3600 con 0x556beda73590 2024-09-11T00:18:00.015 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:00.008+0000 7f94fd600640 10 client.16728 handle_command_reply: tid=0 2024-09-11T00:18:00.015 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:00.008+0000 7f9519880740 1 client.16728 shutdown 2024-09-11T00:18:00.015 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:00.009+0000 7f9519880740 1 -- 172.21.15.12:0/2931401294 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556beda63b00 msgr2=0x556beda69180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:18:00.015 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:00.009+0000 7f9519880740 1 --2- 172.21.15.12:0/2931401294 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556beda63b00 0x556beda69180 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f9508005bc0 tx=0x7f9508009840 comp rx=0 tx=0).stop 2024-09-11T00:18:00.015 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:00.009+0000 7f9519880740 1 -- 172.21.15.12:0/2931401294 shutdown_connections 2024-09-11T00:18:00.015 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:00.009+0000 7f9519880740 1 --2- 172.21.15.12:0/2931401294 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x556beda73590 0x556beda75980 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:18:00.015 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:00.009+0000 7f9519880740 1 --2- 172.21.15.12:0/2931401294 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556beda63b00 0x556beda69180 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:18:00.015 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:00.009+0000 7f9519880740 1 --2- 172.21.15.12:0/2931401294 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556beda610c0 0x556beda685b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:18:00.016 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:00.009+0000 7f94fd600640 0 client.16728 ms_handle_reset on v2:172.21.15.165:6838/501312984 2024-09-11T00:18:00.016 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:00.009+0000 7f9519880740 1 --2- 172.21.15.12:0/2931401294 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556beda5e790 0x556beda67f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:18:00.016 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:00.009+0000 7f9519880740 1 -- 172.21.15.12:0/2931401294 >> 172.21.15.12:0/2931401294 conn(0x556beda598e0 msgr2=0x556beda58a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:18:00.016 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:00.009+0000 7f9519880740 1 -- 172.21.15.12:0/2931401294 shutdown_connections 2024-09-11T00:18:00.016 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:00.009+0000 7f9519880740 1 -- 172.21.15.12:0/2931401294 wait complete. 2024-09-11T00:18:00.016 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:00.009+0000 7f9519880740 20 client.16728 trim_cache size 0 max 16384 2024-09-11T00:18:00.028 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:00.025+0000 7f9519880740 1 Processor -- start 2024-09-11T00:18:00.028 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:00.025+0000 7f9519880740 1 -- start start 2024-09-11T00:18:00.028 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:00.025+0000 7f9519880740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556beda5e790 0x556beda63ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:18:00.028 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:00.025+0000 7f9519880740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556beda610c0 0x556beda64420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:18:00.028 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:00.025+0000 7f9519880740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556beda63b00 0x556beda5b710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:18:00.028 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:00.025+0000 7f9519880740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f95101ad980 con 0x556beda5e790 2024-09-11T00:18:00.028 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:00.025+0000 7f9519880740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f95101af4f0 con 0x556beda63b00 2024-09-11T00:18:00.028 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:00.025+0000 7f9519880740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f95101ae080 con 0x556beda610c0 2024-09-11T00:18:00.028 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:00.025+0000 7f9514a00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556beda5e790 0x556beda63ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:18:00.028 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:00.025+0000 7f9514a00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556beda5e790 0x556beda63ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:54934/0 (socket says 172.21.15.12:54934) 2024-09-11T00:18:00.029 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:00.025+0000 7f9514a00640 1 -- 172.21.15.12:0/3559095122 learned_addr learned my addr 172.21.15.12:0/3559095122 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:18:00.029 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:00.025+0000 7f9507400640 1 --2- 172.21.15.12:0/3559095122 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556beda610c0 0x556beda64420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:18:00.029 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:00.026+0000 7f9514a00640 1 --2- 172.21.15.12:0/3559095122 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556beda63b00 0x556beda5b710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:18:00.029 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:00.026+0000 7f9507400640 1 -- 172.21.15.12:0/3559095122 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556beda63b00 msgr2=0x556beda5b710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:18:00.029 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:00.026+0000 7f9507400640 1 --2- 172.21.15.12:0/3559095122 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556beda63b00 0x556beda5b710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:18:00.029 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:00.026+0000 7f9507400640 1 -- 172.21.15.12:0/3559095122 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556beda5e790 msgr2=0x556beda63ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:18:00.029 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:00.026+0000 7f9507400640 1 --2- 172.21.15.12:0/3559095122 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556beda5e790 0x556beda63ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:18:00.029 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:00.026+0000 7f9507400640 1 -- 172.21.15.12:0/3559095122 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556beda63820 con 0x556beda610c0 2024-09-11T00:18:00.029 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:00.026+0000 7f9514a00640 1 --2- 172.21.15.12:0/3559095122 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556beda5e790 0x556beda63ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:18:00.032 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:00.026+0000 7f9507400640 1 --2- 172.21.15.12:0/3559095122 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556beda610c0 0x556beda64420 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x556beda75f60 tx=0x7f94f8005f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a6530be8931c2281 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:18:00.032 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:00.026+0000 7f94ffe00640 1 -- 172.21.15.12:0/3559095122 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94f8020040 con 0x556beda610c0 2024-09-11T00:18:00.032 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:00.026+0000 7f94ffe00640 1 -- 172.21.15.12:0/3559095122 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f94f80054d0 con 0x556beda610c0 2024-09-11T00:18:00.032 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:00.026+0000 7f94ffe00640 1 -- 172.21.15.12:0/3559095122 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94f8004da0 con 0x556beda610c0 2024-09-11T00:18:00.033 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:00.027+0000 7f9519880740 1 -- 172.21.15.12:0/3559095122 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556beda610c0 msgr2=0x556beda64420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:18:00.033 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:00.027+0000 7f9519880740 1 --2- 172.21.15.12:0/3559095122 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556beda610c0 0x556beda64420 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x556beda75f60 tx=0x7f94f8005f90 comp rx=0 tx=0).stop 2024-09-11T00:18:00.033 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:00.027+0000 7f9519880740 1 -- 172.21.15.12:0/3559095122 shutdown_connections 2024-09-11T00:18:00.033 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:00.027+0000 7f9519880740 1 --2- 172.21.15.12:0/3559095122 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556beda63b00 0x556beda5b710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:18:00.033 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:00.027+0000 7f9519880740 1 --2- 172.21.15.12:0/3559095122 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556beda610c0 0x556beda64420 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:18:00.033 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:00.027+0000 7f9519880740 1 --2- 172.21.15.12:0/3559095122 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556beda5e790 0x556beda63ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:18:00.033 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:00.027+0000 7f9519880740 1 -- 172.21.15.12:0/3559095122 >> 172.21.15.12:0/3559095122 conn(0x556beda598e0 msgr2=0x556beda5a300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:18:00.033 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:00.027+0000 7f9519880740 1 -- 172.21.15.12:0/3559095122 shutdown_connections 2024-09-11T00:18:00.033 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:00.027+0000 7f9519880740 1 -- 172.21.15.12:0/3559095122 wait complete. 2024-09-11T00:18:00.033 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:00.027+0000 7f9519880740 1 -- start start 2024-09-11T00:18:00.033 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:00.028+0000 7f9519880740 1 Processor -- start 2024-09-11T00:18:00.033 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:00.028+0000 7f9519880740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556beda5e790 0x556beda6ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:18:00.034 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:00.028+0000 7f9519880740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556beda610c0 0x556beda6f200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:18:00.034 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:00.028+0000 7f9519880740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556beda63b00 0x556beda6d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:18:00.034 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:00.028+0000 7f9519880740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f95101aee20 con 0x556beda63b00 2024-09-11T00:18:00.034 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:00.028+0000 7f9519880740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f95101ae750 con 0x556beda5e790 2024-09-11T00:18:00.034 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:00.028+0000 7f9519880740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f95101ab960 con 0x556beda610c0 2024-09-11T00:18:00.034 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:00.028+0000 7f9519880740 10 client.0 _ll_register_callbacks cb 0x7f94f80097a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:18:00.034 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:00.028+0000 7f9514a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556beda5e790 0x556beda6ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:18:00.034 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:00.028+0000 7f9514a00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556beda63b00 0x556beda6d7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:18:00.034 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:00.028+0000 7f9514a00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556beda63b00 0x556beda6d7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:54942/0 (socket says 172.21.15.12:54942) 2024-09-11T00:18:00.034 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:00.028+0000 7f9514a00640 1 -- 172.21.15.12:0/150157537 learned_addr learned my addr 172.21.15.12:0/150157537 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:18:00.034 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:00.028+0000 7f9507400640 1 --2- 172.21.15.12:0/150157537 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556beda610c0 0x556beda6f200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:18:00.034 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:00.029+0000 7f9514a00640 1 -- 172.21.15.12:0/150157537 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556beda610c0 msgr2=0x556beda6f200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:18:00.035 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:00.029+0000 7f9514a00640 1 --2- 172.21.15.12:0/150157537 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556beda610c0 0x556beda6f200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:18:00.035 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:00.029+0000 7f9514a00640 1 -- 172.21.15.12:0/150157537 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556beda5e790 msgr2=0x556beda6ecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:18:00.035 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:00.029+0000 7f9514a00640 1 --2- 172.21.15.12:0/150157537 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556beda5e790 0x556beda6ecc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:18:00.035 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:00.029+0000 7f9514a00640 1 -- 172.21.15.12:0/150157537 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556beda6fab0 con 0x556beda63b00 2024-09-11T00:18:00.035 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:00.029+0000 7f9514a00640 1 --2- 172.21.15.12:0/150157537 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556beda63b00 0x556beda6d7e0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f9508000c00 tx=0x7f9508012f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14c677d890b2c50b server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:18:00.035 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:00.029+0000 7f94fd600640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-09-11T00:18:00.035 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:00.029+0000 7f94fd600640 1 -- 172.21.15.12:0/150157537 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95080055f0 con 0x556beda63b00 2024-09-11T00:18:00.035 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:00.029+0000 7f94fd600640 1 -- 172.21.15.12:0/150157537 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9208+0+0 (secure 0 0 0) 0x7f9508003bc0 con 0x556beda63b00 2024-09-11T00:18:00.035 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:00.029+0000 7f9519880740 1 -- 172.21.15.12:0/150157537 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f94f80054d0 con 0x556beda63b00 2024-09-11T00:18:00.035 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:00.030+0000 7f94fd600640 1 -- 172.21.15.12:0/150157537 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f9508003de0 con 0x556beda63b00 2024-09-11T00:18:00.035 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:00.030+0000 7f9519880740 10 client.16734 fetch_fsmap learned FSMap version 42 2024-09-11T00:18:00.035 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:00.030+0000 7f9519880740 1 -- 172.21.15.12:0/150157537 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x556bedae0a90 con 0x556beda63b00 2024-09-11T00:18:00.036 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:00.030+0000 7f94fd600640 1 -- 172.21.15.12:0/150157537 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f94f80054d0 con 0x556beda63b00 2024-09-11T00:18:00.036 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:00.030+0000 7f94fd600640 1 -- 172.21.15.12:0/150157537 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 42) ==== 8374+0+0 (secure 0 0 0) 0x7f9508043030 con 0x556beda63b00 2024-09-11T00:18:00.036 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:00.030+0000 7f9519880740 10 client.16734 fetch_fsmap finished waiting for FSMap version 42 2024-09-11T00:18:00.036 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:00.030+0000 7f9519880740 10 client.16734 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T00:18:00.036 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:00.030+0000 7f9519880740 20 client.16734 populate_metadata read hostname 'smithi012' 2024-09-11T00:18:00.036 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:00.030+0000 7f9519880740 1 --2- 172.21.15.12:0/150157537 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x556bedae1380 0x556bedae37a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:18:00.036 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:00.030+0000 7f9519880740 4 client.16734 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T00:18:00.036 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:00.030+0000 7f9519880740 1 -- 172.21.15.12:0/150157537 --> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] -- command(tid 0: {"prefix": "flush journal"}) -- 0x556bed9cf0b0 con 0x556bedae1380 2024-09-11T00:18:00.037 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:00.031+0000 7f9507400640 1 --2- 172.21.15.12:0/150157537 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x556bedae1380 0x556bedae37a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:18:00.037 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:00.033+0000 7f9507400640 1 --2- 172.21.15.12:0/150157537 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x556bedae1380 0x556bedae37a0 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8650a48197802740 server_cookie=53158cd30b4c09d0 in_seq=0 out_seq=0 2024-09-11T00:18:00.037 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:00.033+0000 7f94fd600640 10 client.16734 ms_handle_connect on v2:172.21.15.165:6838/501312984 2024-09-11T00:18:00.148 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:00.145+0000 7f94fd600640 1 -- 172.21.15.12:0/150157537 <== mds.0 v2:172.21.15.165:6838/501312984 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x556bed9cf0b0 con 0x556bedae1380 2024-09-11T00:18:00.148 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:00.145+0000 7f94fd600640 10 client.16734 handle_command_reply: tid=0 2024-09-11T00:18:00.148 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:00.145+0000 7f9519880740 1 client.16734 shutdown 2024-09-11T00:18:00.151 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:00.148+0000 7f9519880740 1 -- 172.21.15.12:0/150157537 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556beda63b00 msgr2=0x556beda6d7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:18:00.151 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:00.148+0000 7f9519880740 1 --2- 172.21.15.12:0/150157537 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556beda63b00 0x556beda6d7e0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f9508000c00 tx=0x7f9508012f90 comp rx=0 tx=0).stop 2024-09-11T00:18:00.152 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:00.150+0000 7f9519880740 1 -- 172.21.15.12:0/150157537 shutdown_connections 2024-09-11T00:18:00.152 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:00.150+0000 7f9519880740 1 --2- 172.21.15.12:0/150157537 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x556bedae1380 0x556bedae37a0 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:18:00.152 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:00.150+0000 7f9519880740 1 --2- 172.21.15.12:0/150157537 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556beda63b00 0x556beda6d7e0 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:18:00.152 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:00.150+0000 7f9519880740 1 --2- 172.21.15.12:0/150157537 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556beda610c0 0x556beda6f200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:18:00.152 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:00.150+0000 7f9519880740 1 --2- 172.21.15.12:0/150157537 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556beda5e790 0x556beda6ecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:18:00.152 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:00.150+0000 7f9519880740 1 -- 172.21.15.12:0/150157537 >> 172.21.15.12:0/150157537 conn(0x556beda598e0 msgr2=0x556beda5d7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:18:00.152 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:00.150+0000 7f94fd600640 0 client.16734 ms_handle_reset on v2:172.21.15.165:6838/501312984 2024-09-11T00:18:00.153 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:00.151+0000 7f9519880740 1 -- 172.21.15.12:0/150157537 shutdown_connections 2024-09-11T00:18:00.153 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:00.151+0000 7f9519880740 1 -- 172.21.15.12:0/150157537 wait complete. 2024-09-11T00:18:00.155 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:00.153+0000 7f9519880740 20 client.16734 trim_cache size 0 max 16384 2024-09-11T00:18:00.156 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:{ 2024-09-11T00:18:00.156 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "message": "", 2024-09-11T00:18:00.156 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "return_code": 0 2024-09-11T00:18:00.156 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:} 2024-09-11T00:18:00.157 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:00.154+0000 7f9516a00640 1 -- 172.21.15.12:0/1498839943 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f94f40b52e0 msgr2=0x7f94f40b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:18:00.157 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:00.154+0000 7f9516a00640 1 --2- 172.21.15.12:0/1498839943 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f94f40b52e0 0x7f94f40b77a0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f950002ae80 tx=0x7f9500003f40 comp rx=0 tx=0).stop 2024-09-11T00:18:00.157 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:00.154+0000 7f9516a00640 1 -- 172.21.15.12:0/1498839943 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9510104b90 msgr2=0x7f951019afb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:18:00.157 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:00.154+0000 7f9516a00640 1 --2- 172.21.15.12:0/1498839943 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9510104b90 0x7f951019afb0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f9500000f80 tx=0x7f95000317c0 comp rx=0 tx=0).stop 2024-09-11T00:18:00.157 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:00.154+0000 7f9516a00640 1 -- 172.21.15.12:0/1498839943 shutdown_connections 2024-09-11T00:18:00.157 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:00.154+0000 7f9516a00640 1 --2- 172.21.15.12:0/1498839943 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f94f40b52e0 0x7f94f40b77a0 secure :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f950002ae80 tx=0x7f9500003f40 comp rx=0 tx=0).stop 2024-09-11T00:18:00.157 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:00.154+0000 7f9516a00640 1 --2- 172.21.15.12:0/1498839943 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f95101066a0 0x7f951019c230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:18:00.158 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:00.154+0000 7f9516a00640 1 --2- 172.21.15.12:0/1498839943 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9510105ce0 0x7f951019b600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:18:00.158 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:00.154+0000 7f9516a00640 1 --2- 172.21.15.12:0/1498839943 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9510104b90 0x7f951019afb0 secure :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f9500000f80 tx=0x7f95000317c0 comp rx=0 tx=0).stop 2024-09-11T00:18:00.158 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:00.154+0000 7f9516a00640 1 -- 172.21.15.12:0/1498839943 >> 172.21.15.12:0/1498839943 conn(0x7f9510100470 msgr2=0x7f95101090d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:18:00.158 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:00.154+0000 7f9516a00640 1 -- 172.21.15.12:0/1498839943 shutdown_connections 2024-09-11T00:18:00.158 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:00.155+0000 7f9516a00640 1 -- 172.21.15.12:0/1498839943 wait complete. 2024-09-11T00:18:00.239 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:17:59 smithi012 ceph-mon[30333]: pgmap v726: 129 pgs: 129 active+clean; 111 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.6 MiB/s wr, 209 op/s 2024-09-11T00:18:00.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:17:59 smithi165 ceph-mon[35356]: pgmap v726: 129 pgs: 129 active+clean; 111 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.6 MiB/s wr, 209 op/s 2024-09-11T00:18:00.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:17:59 smithi178 ceph-mon[35919]: pgmap v726: 129 pgs: 129 active+clean; 111 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.6 MiB/s wr, 209 op/s 2024-09-11T00:18:00.689 INFO:tasks.workunit.client.0.smithi012.stdout: 1 465273 4.20 MB/sec execute 477 sec latency 542.470 ms 2024-09-11T00:18:01.689 INFO:tasks.workunit.client.0.smithi012.stdout: 1 465273 4.19 MB/sec execute 478 sec latency 1542.538 ms 2024-09-11T00:18:02.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:18:01 smithi165 ceph-mon[35356]: pgmap v727: 129 pgs: 129 active+clean; 99 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.9 MiB/s wr, 234 op/s 2024-09-11T00:18:02.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:18:01 smithi012 ceph-mon[30333]: pgmap v727: 129 pgs: 129 active+clean; 99 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.9 MiB/s wr, 234 op/s 2024-09-11T00:18:02.330 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:18:01 smithi178 ceph-mon[35919]: pgmap v727: 129 pgs: 129 active+clean; 99 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.9 MiB/s wr, 234 op/s 2024-09-11T00:18:02.689 INFO:tasks.workunit.client.0.smithi012.stdout: 1 466124 4.19 MB/sec execute 479 sec latency 1684.526 ms 2024-09-11T00:18:03.689 INFO:tasks.workunit.client.0.smithi012.stdout: 1 468365 4.21 MB/sec execute 480 sec latency 15.223 ms 2024-09-11T00:18:04.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:18:03 smithi165 ceph-mon[35356]: pgmap v728: 129 pgs: 129 active+clean; 96 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 796 KiB/s rd, 1.7 MiB/s wr, 253 op/s 2024-09-11T00:18:04.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:18:03 smithi012 ceph-mon[30333]: pgmap v728: 129 pgs: 129 active+clean; 96 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 796 KiB/s rd, 1.7 MiB/s wr, 253 op/s 2024-09-11T00:18:04.330 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:18:03 smithi178 ceph-mon[35919]: pgmap v728: 129 pgs: 129 active+clean; 96 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 796 KiB/s rd, 1.7 MiB/s wr, 253 op/s 2024-09-11T00:18:04.689 INFO:tasks.workunit.client.0.smithi012.stdout: 1 469802 4.22 MB/sec execute 481 sec latency 15.355 ms 2024-09-11T00:18:05.689 INFO:tasks.workunit.client.0.smithi012.stdout: 1 471325 4.22 MB/sec execute 482 sec latency 457.538 ms 2024-09-11T00:18:06.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:18:05 smithi165 ceph-mon[35356]: pgmap v729: 129 pgs: 129 active+clean; 96 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 405 KiB/s rd, 1.4 MiB/s wr, 189 op/s 2024-09-11T00:18:06.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:18:05 smithi012 ceph-mon[30333]: pgmap v729: 129 pgs: 129 active+clean; 96 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 405 KiB/s rd, 1.4 MiB/s wr, 189 op/s 2024-09-11T00:18:06.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:18:05 smithi178 ceph-mon[35919]: pgmap v729: 129 pgs: 129 active+clean; 96 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 405 KiB/s rd, 1.4 MiB/s wr, 189 op/s 2024-09-11T00:18:06.689 INFO:tasks.workunit.client.0.smithi012.stdout: 1 471387 4.21 MB/sec execute 483 sec latency 1379.085 ms 2024-09-11T00:18:07.690 INFO:tasks.workunit.client.0.smithi012.stdout: 1 472958 4.23 MB/sec execute 484 sec latency 31.014 ms 2024-09-11T00:18:08.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:18:07 smithi165 ceph-mon[35356]: pgmap v730: 129 pgs: 129 active+clean; 98 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 589 KiB/s rd, 2.2 MiB/s wr, 288 op/s 2024-09-11T00:18:08.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:18:07 smithi012 ceph-mon[30333]: pgmap v730: 129 pgs: 129 active+clean; 98 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 589 KiB/s rd, 2.2 MiB/s wr, 288 op/s 2024-09-11T00:18:08.330 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:18:07 smithi178 ceph-mon[35919]: pgmap v730: 129 pgs: 129 active+clean; 98 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 589 KiB/s rd, 2.2 MiB/s wr, 288 op/s 2024-09-11T00:18:08.690 INFO:tasks.workunit.client.0.smithi012.stdout: 1 474993 4.24 MB/sec execute 485 sec latency 17.455 ms 2024-09-11T00:18:09.690 INFO:tasks.workunit.client.0.smithi012.stdout: 1 476624 4.25 MB/sec execute 486 sec latency 9.924 ms 2024-09-11T00:18:10.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:18:09 smithi165 ceph-mon[35356]: pgmap v731: 129 pgs: 129 active+clean; 97 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 320 KiB/s rd, 1.6 MiB/s wr, 200 op/s 2024-09-11T00:18:10.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:18:09 smithi012 ceph-mon[30333]: pgmap v731: 129 pgs: 129 active+clean; 97 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 320 KiB/s rd, 1.6 MiB/s wr, 200 op/s 2024-09-11T00:18:10.330 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:18:09 smithi178 ceph-mon[35919]: pgmap v731: 129 pgs: 129 active+clean; 97 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 320 KiB/s rd, 1.6 MiB/s wr, 200 op/s 2024-09-11T00:18:10.690 INFO:tasks.workunit.client.0.smithi012.stdout: 1 477783 4.25 MB/sec execute 487 sec latency 356.618 ms 2024-09-11T00:18:11.690 INFO:tasks.workunit.client.0.smithi012.stdout: 1 477783 4.25 MB/sec execute 488 sec latency 1356.669 ms 2024-09-11T00:18:12.090 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:18:12 smithi012 ceph-mon[30333]: pgmap v732: 129 pgs: 129 active+clean; 97 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 324 KiB/s rd, 2.0 MiB/s wr, 226 op/s 2024-09-11T00:18:12.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:18:12 smithi165 ceph-mon[35356]: pgmap v732: 129 pgs: 129 active+clean; 97 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 324 KiB/s rd, 2.0 MiB/s wr, 226 op/s 2024-09-11T00:18:12.330 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:18:12 smithi178 ceph-mon[35919]: pgmap v732: 129 pgs: 129 active+clean; 97 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 324 KiB/s rd, 2.0 MiB/s wr, 226 op/s 2024-09-11T00:18:12.691 INFO:tasks.workunit.client.0.smithi012.stdout: 1 478895 4.25 MB/sec execute 489 sec latency 1950.164 ms 2024-09-11T00:18:13.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:18:13 smithi165 ceph-mon[35356]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T00:18:13.295 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:13.293+0000 7f7653800640 1 -- 172.21.15.12:0/374117940 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f764c1066a0 msgr2=0x7f764c10d020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:18:13.296 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:13.293+0000 7f7653800640 1 --2- 172.21.15.12:0/374117940 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f764c1066a0 0x7f764c10d020 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f7644009ca0 tx=0x7f764402a8c0 comp rx=0 tx=0).stop 2024-09-11T00:18:13.296 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:13.293+0000 7f7653800640 1 -- 172.21.15.12:0/374117940 shutdown_connections 2024-09-11T00:18:13.296 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:13.293+0000 7f7653800640 1 --2- 172.21.15.12:0/374117940 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f764c1066a0 0x7f764c10d020 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:18:13.296 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:13.293+0000 7f7653800640 1 --2- 172.21.15.12:0/374117940 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f764c105ce0 0x7f764c106160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:18:13.296 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:13.293+0000 7f7653800640 1 --2- 172.21.15.12:0/374117940 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f764c104b90 0x7f764c104f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:18:13.296 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:13.293+0000 7f7653800640 1 -- 172.21.15.12:0/374117940 >> 172.21.15.12:0/374117940 conn(0x7f764c100470 msgr2=0x7f764c102890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:18:13.296 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:13.293+0000 7f7653800640 1 -- 172.21.15.12:0/374117940 shutdown_connections 2024-09-11T00:18:13.296 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:13.293+0000 7f7653800640 1 -- 172.21.15.12:0/374117940 wait complete. 2024-09-11T00:18:13.298 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:13.294+0000 7f7653800640 1 Processor -- start 2024-09-11T00:18:13.298 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:13.294+0000 7f7653800640 1 -- start start 2024-09-11T00:18:13.298 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:13.295+0000 7f7653800640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f764c104b90 0x7f764c19aec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:18:13.298 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:13.295+0000 7f7653800640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f764c105ce0 0x7f764c19b510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:18:13.298 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:13.295+0000 7f7653800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f764c1066a0 0x7f764c19c140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:18:13.299 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:13.295+0000 7f7653800640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f764c110850 con 0x7f764c105ce0 2024-09-11T00:18:13.299 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:13.295+0000 7f7653800640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f764c1106d0 con 0x7f764c1066a0 2024-09-11T00:18:13.299 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:13.295+0000 7f7653800640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f764c1109d0 con 0x7f764c104b90 2024-09-11T00:18:13.299 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:13.295+0000 7f7650e00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f764c104b90 0x7f764c19aec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:18:13.299 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:13.295+0000 7f764be00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f764c105ce0 0x7f764c19b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:18:13.299 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:13.295+0000 7f7650e00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f764c104b90 0x7f764c19aec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.12:55794/0 (socket says 172.21.15.12:55794) 2024-09-11T00:18:13.299 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:13.295+0000 7f7651800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f764c1066a0 0x7f764c19c140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:18:13.299 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:13.295+0000 7f7650e00640 1 -- 172.21.15.12:0/1797522826 learned_addr learned my addr 172.21.15.12:0/1797522826 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:18:13.299 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:13.295+0000 7f7650e00640 1 -- 172.21.15.12:0/1797522826 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f764c1066a0 msgr2=0x7f764c19c140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:18:13.299 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:13.295+0000 7f7650e00640 1 --2- 172.21.15.12:0/1797522826 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f764c1066a0 0x7f764c19c140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:18:13.299 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:13.295+0000 7f7650e00640 1 -- 172.21.15.12:0/1797522826 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f764c105ce0 msgr2=0x7f764c19b510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:18:13.299 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:13.295+0000 7f7650e00640 1 --2- 172.21.15.12:0/1797522826 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f764c105ce0 0x7f764c19b510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:18:13.299 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:13.295+0000 7f7650e00640 1 -- 172.21.15.12:0/1797522826 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f764c19ca00 con 0x7f764c104b90 2024-09-11T00:18:13.300 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:13.295+0000 7f764be00640 1 --2- 172.21.15.12:0/1797522826 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f764c105ce0 0x7f764c19b510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:18:13.300 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:13.296+0000 7f7650e00640 1 --2- 172.21.15.12:0/1797522826 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f764c104b90 0x7f764c19aec0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f763c00e9f0 tx=0x7f763c00eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:18:13.300 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:13.296+0000 7f7649600640 1 -- 172.21.15.12:0/1797522826 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f763c00cd80 con 0x7f764c104b90 2024-09-11T00:18:13.300 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:13.296+0000 7f7649600640 1 -- 172.21.15.12:0/1797522826 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f763c0108e0 con 0x7f764c104b90 2024-09-11T00:18:13.300 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:13.296+0000 7f7653800640 1 -- 172.21.15.12:0/1797522826 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f764c19ccf0 con 0x7f764c104b90 2024-09-11T00:18:13.300 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:13.296+0000 7f7649600640 1 -- 172.21.15.12:0/1797522826 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f763c018980 con 0x7f764c104b90 2024-09-11T00:18:13.300 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:13.296+0000 7f7653800640 1 -- 172.21.15.12:0/1797522826 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f764c1a33c0 con 0x7f764c104b90 2024-09-11T00:18:13.301 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:13.299+0000 7f765680f740 1 Processor -- start 2024-09-11T00:18:13.301 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:13.299+0000 7f765680f740 1 -- start start 2024-09-11T00:18:13.301 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:13.299+0000 7f765680f740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561c9813a790 0x561c9813cb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:18:13.301 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:13.299+0000 7f765680f740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561c9813d0c0 0x561c9813f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:18:13.301 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:13.299+0000 7f765680f740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561c9813fb00 0x561c98141fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:18:13.301 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:13.299+0000 7f765680f740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x561c980ad6b0 con 0x561c9813fb00 2024-09-11T00:18:13.302 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:13.299+0000 7f765680f740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x561c9809cae0 con 0x561c9813a790 2024-09-11T00:18:13.302 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:13.299+0000 7f765680f740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x561c980a08f0 con 0x561c9813d0c0 2024-09-11T00:18:13.303 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:13.299+0000 7f7651800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561c9813a790 0x561c9813cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:18:13.303 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:13.299+0000 7f764be00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561c9813fb00 0x561c98141fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:18:13.304 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:13.299+0000 7f7651800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561c9813a790 0x561c9813cb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.12:32932/0 (socket says 172.21.15.12:32932) 2024-09-11T00:18:13.304 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:13.299+0000 7f7651800640 1 -- 172.21.15.12:0/2506302342 learned_addr learned my addr 172.21.15.12:0/2506302342 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:18:13.304 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:13.300+0000 7f7649600640 1 -- 172.21.15.12:0/1797522826 <== mon.1 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7f763c010430 con 0x7f764c104b90 2024-09-11T00:18:13.304 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:13.300+0000 7f7651800640 1 -- 172.21.15.12:0/2506302342 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561c9813d0c0 msgr2=0x561c9813f4b0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-11T00:18:13.304 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:13.300+0000 7f7651800640 1 --2- 172.21.15.12:0/2506302342 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561c9813d0c0 0x561c9813f4b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:18:13.304 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:13.300+0000 7f7651800640 1 -- 172.21.15.12:0/2506302342 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561c9813fb00 msgr2=0x561c98141fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:18:13.304 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:13.300+0000 7f7651800640 1 --2- 172.21.15.12:0/2506302342 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561c9813fb00 0x561c98141fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:18:13.304 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:13.300+0000 7f7651800640 1 -- 172.21.15.12:0/2506302342 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561c98069780 con 0x561c9813a790 2024-09-11T00:18:13.304 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:13.300+0000 7f764be00640 1 --2- 172.21.15.12:0/2506302342 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561c9813fb00 0x561c98141fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:18:13.304 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:13.300+0000 7f7651800640 1 --2- 172.21.15.12:0/2506302342 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561c9813a790 0x561c9813cb80 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f7644009ca0 tx=0x7f76440093c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4d8d845dd5a5b36 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:18:13.304 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:13.300+0000 7f7642000640 1 -- 172.21.15.12:0/2506302342 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7644031de0 con 0x561c9813a790 2024-09-11T00:18:13.304 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:13.300+0000 7f7642000640 1 -- 172.21.15.12:0/2506302342 <== mon.2 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7644036030 con 0x561c9813a790 2024-09-11T00:18:13.304 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:13.300+0000 7f7642000640 1 -- 172.21.15.12:0/2506302342 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76440042a0 con 0x561c9813a790 2024-09-11T00:18:13.305 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:13.301+0000 7f765680f740 1 -- 172.21.15.12:0/2506302342 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561c9813a790 msgr2=0x561c9813cb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:18:13.305 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:13.301+0000 7f765680f740 1 --2- 172.21.15.12:0/2506302342 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561c9813a790 0x561c9813cb80 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f7644009ca0 tx=0x7f76440093c0 comp rx=0 tx=0).stop 2024-09-11T00:18:13.305 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:13.301+0000 7f765680f740 1 -- 172.21.15.12:0/2506302342 shutdown_connections 2024-09-11T00:18:13.305 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:13.301+0000 7f765680f740 1 --2- 172.21.15.12:0/2506302342 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561c9813fb00 0x561c98141fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:18:13.305 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:13.301+0000 7f765680f740 1 --2- 172.21.15.12:0/2506302342 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561c9813d0c0 0x561c9813f4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:18:13.305 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:13.301+0000 7f765680f740 1 --2- 172.21.15.12:0/2506302342 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561c9813a790 0x561c9813cb80 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:18:13.305 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:13.301+0000 7f765680f740 1 -- 172.21.15.12:0/2506302342 >> 172.21.15.12:0/2506302342 conn(0x561c981358e0 msgr2=0x561c98136670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:18:13.305 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:13.301+0000 7f765680f740 1 -- 172.21.15.12:0/2506302342 shutdown_connections 2024-09-11T00:18:13.305 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:13.301+0000 7f765680f740 1 -- 172.21.15.12:0/2506302342 wait complete. 2024-09-11T00:18:13.309 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:13.302+0000 7f7649600640 1 --2- 172.21.15.12:0/1797522826 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f76300b5270 0x7f76300b7730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:18:13.309 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:13.302+0000 7f765680f740 1 -- start start 2024-09-11T00:18:13.309 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:13.302+0000 7f7649600640 1 -- 172.21.15.12:0/1797522826 <== mon.1 v2:172.21.15.178:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9208+0+0 (secure 0 0 0) 0x7f763c0e6260 con 0x7f764c104b90 2024-09-11T00:18:13.310 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:13.302+0000 7f7651800640 1 --2- 172.21.15.12:0/1797522826 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f76300b5270 0x7f76300b7730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:18:13.310 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:13.302+0000 7f765680f740 1 Processor -- start 2024-09-11T00:18:13.310 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:13.303+0000 7f765680f740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561c9813a790 0x561c98143cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:18:13.310 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:13.303+0000 7f765680f740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561c9813d0c0 0x561c98144360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:18:13.310 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:13.303+0000 7f7651800640 1 --2- 172.21.15.12:0/1797522826 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f76300b5270 0x7f76300b7730 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f764402f470 tx=0x7f764402a8c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:18:13.310 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:13.303+0000 7f765680f740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561c9813fb00 0x561c98144f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:18:13.310 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:13.303+0000 7f765680f740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x561c97fd4ab0 con 0x561c9813a790 2024-09-11T00:18:13.310 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:13.303+0000 7f765680f740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x561c98043570 con 0x561c9813d0c0 2024-09-11T00:18:13.310 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:13.303+0000 7f765680f740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x561c98142900 con 0x561c9813fb00 2024-09-11T00:18:13.310 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:13.303+0000 7f765680f740 10 client.0 _ll_register_callbacks cb 0x561c98052100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:18:13.310 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:13.303+0000 7f7650e00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561c9813a790 0x561c98143cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:18:13.310 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:13.303+0000 7f764be00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561c9813d0c0 0x561c98144360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:18:13.310 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:13.303+0000 7f764be00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561c9813d0c0 0x561c98144360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.12:32934/0 (socket says 172.21.15.12:32934) 2024-09-11T00:18:13.311 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:13.303+0000 7f7650e00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561c9813a790 0x561c98143cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:46454/0 (socket says 172.21.15.12:46454) 2024-09-11T00:18:13.311 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:13.303+0000 7f764be00640 1 -- 172.21.15.12:0/191175730 learned_addr learned my addr 172.21.15.12:0/191175730 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:18:13.311 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:13.303+0000 7f7651800640 1 --2- 172.21.15.12:0/191175730 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561c9813fb00 0x561c98144f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:18:13.311 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:13.303+0000 7f764be00640 1 -- 172.21.15.12:0/191175730 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561c9813fb00 msgr2=0x561c98144f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:18:13.311 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:13.304+0000 7f764be00640 1 --2- 172.21.15.12:0/191175730 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561c9813fb00 0x561c98144f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:18:13.311 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:13.304+0000 7f764be00640 1 -- 172.21.15.12:0/191175730 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561c9813a790 msgr2=0x561c98143cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:18:13.311 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:13.304+0000 7f764be00640 1 --2- 172.21.15.12:0/191175730 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561c9813a790 0x561c98143cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:18:13.311 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:13.304+0000 7f764be00640 1 -- 172.21.15.12:0/191175730 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561c97dfe790 con 0x561c9813d0c0 2024-09-11T00:18:13.311 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:13.304+0000 7f764be00640 1 --2- 172.21.15.12:0/191175730 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561c9813d0c0 0x561c98144360 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f7638012d20 tx=0x7f763800f790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e4694658e5715b82 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:18:13.311 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:13.304+0000 7f7636a00640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-09-11T00:18:13.311 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:13.304+0000 7f7636a00640 1 -- 172.21.15.12:0/191175730 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76380120d0 con 0x561c9813d0c0 2024-09-11T00:18:13.311 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:13.304+0000 7f765680f740 1 -- 172.21.15.12:0/191175730 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7644036030 con 0x561c9813d0c0 2024-09-11T00:18:13.311 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:13.304+0000 7f7636a00640 1 -- 172.21.15.12:0/191175730 <== mon.2 v2:172.21.15.165:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9208+0+0 (secure 0 0 0) 0x7f7638004020 con 0x561c9813d0c0 2024-09-11T00:18:13.311 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:13.304+0000 7f7636a00640 1 -- 172.21.15.12:0/191175730 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f7638025050 con 0x561c9813d0c0 2024-09-11T00:18:13.311 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:13.304+0000 7f765680f740 10 client.26747 fetch_fsmap learned FSMap version 43 2024-09-11T00:18:13.312 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:13.304+0000 7f765680f740 1 -- 172.21.15.12:0/191175730 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x561c97e77620 con 0x561c9813d0c0 2024-09-11T00:18:13.312 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:13.305+0000 7f7636a00640 1 -- 172.21.15.12:0/191175730 <== mon.2 v2:172.21.15.165:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7644036030 con 0x561c9813d0c0 2024-09-11T00:18:13.312 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:13.305+0000 7f7636a00640 1 -- 172.21.15.12:0/191175730 <== mon.2 v2:172.21.15.165:3300/0 5 ==== fsmap(e 43) ==== 8378+0+0 (secure 0 0 0) 0x7f763801c470 con 0x561c9813d0c0 2024-09-11T00:18:13.312 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:13.305+0000 7f765680f740 10 client.26747 fetch_fsmap finished waiting for FSMap version 43 2024-09-11T00:18:13.312 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:13.305+0000 7f765680f740 10 client.26747 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T00:18:13.312 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:13.305+0000 7f765680f740 20 client.26747 populate_metadata read hostname 'smithi012' 2024-09-11T00:18:13.312 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:13.305+0000 7f765680f740 1 --2- 172.21.15.12:0/191175730 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x561c9814f680 0x561c98151ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:18:13.312 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:13.305+0000 7f765680f740 4 client.26747 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T00:18:13.312 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:13.305+0000 7f765680f740 1 -- 172.21.15.12:0/191175730 --> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561c980af600 con 0x561c9814f680 2024-09-11T00:18:13.312 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:13.305+0000 7f7650e00640 1 --2- 172.21.15.12:0/191175730 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x561c9814f680 0x561c98151ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:18:13.312 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:13.306+0000 7f7650e00640 1 --2- 172.21.15.12:0/191175730 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x561c9814f680 0x561c98151ac0 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=388ea6d4d89bf5e2 server_cookie=f2eddf26f1f74198 in_seq=0 out_seq=0 2024-09-11T00:18:13.312 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:13.306+0000 7f7636a00640 10 client.26747 ms_handle_connect on v2:172.21.15.165:6838/501312984 2024-09-11T00:18:13.313 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:13.306+0000 7f7636a00640 1 -- 172.21.15.12:0/191175730 <== mds.0 v2:172.21.15.165:6838/501312984 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x561c980af600 con 0x561c9814f680 2024-09-11T00:18:13.313 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:13.306+0000 7f7636a00640 10 client.26747 handle_command_reply: tid=0 2024-09-11T00:18:13.313 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:13.306+0000 7f765680f740 1 client.26747 shutdown 2024-09-11T00:18:13.313 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:13.307+0000 7f765680f740 1 -- 172.21.15.12:0/191175730 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561c9813d0c0 msgr2=0x561c98144360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:18:13.313 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:13.307+0000 7f765680f740 1 --2- 172.21.15.12:0/191175730 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561c9813d0c0 0x561c98144360 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f7638012d20 tx=0x7f763800f790 comp rx=0 tx=0).stop 2024-09-11T00:18:13.313 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:13.307+0000 7f765680f740 1 -- 172.21.15.12:0/191175730 shutdown_connections 2024-09-11T00:18:13.313 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:13.307+0000 7f765680f740 1 --2- 172.21.15.12:0/191175730 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x561c9814f680 0x561c98151ac0 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:18:13.313 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:13.307+0000 7f765680f740 1 --2- 172.21.15.12:0/191175730 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561c9813fb00 0x561c98144f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:18:13.313 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:13.307+0000 7f765680f740 1 --2- 172.21.15.12:0/191175730 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561c9813d0c0 0x561c98144360 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:18:13.313 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:13.307+0000 7f765680f740 1 --2- 172.21.15.12:0/191175730 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561c9813a790 0x561c98143cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:18:13.313 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:13.307+0000 7f7636a00640 0 client.26747 ms_handle_reset on v2:172.21.15.165:6838/501312984 2024-09-11T00:18:13.313 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:13.307+0000 7f765680f740 1 -- 172.21.15.12:0/191175730 >> 172.21.15.12:0/191175730 conn(0x561c981358e0 msgr2=0x561c98134a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:18:13.313 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:13.307+0000 7f765680f740 1 -- 172.21.15.12:0/191175730 shutdown_connections 2024-09-11T00:18:13.313 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:13.307+0000 7f765680f740 1 -- 172.21.15.12:0/191175730 wait complete. 2024-09-11T00:18:13.314 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:13.307+0000 7f765680f740 20 client.26747 trim_cache size 0 max 16384 2024-09-11T00:18:13.314 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:18:13 smithi012 ceph-mon[30333]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T00:18:13.325 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:13.322+0000 7f765680f740 1 Processor -- start 2024-09-11T00:18:13.326 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:13.322+0000 7f765680f740 1 -- start start 2024-09-11T00:18:13.326 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:13.322+0000 7f765680f740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561c9813a790 0x561c98136380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:18:13.326 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:13.322+0000 7f765680f740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561c9813d0c0 0x561c98143cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:18:13.326 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:13.322+0000 7f765680f740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561c9813fb00 0x561c98144200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:18:13.326 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:13.322+0000 7f765680f740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f764c1ad810 con 0x561c9813a790 2024-09-11T00:18:13.326 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:13.322+0000 7f765680f740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f764c1af3b0 con 0x561c9813fb00 2024-09-11T00:18:13.326 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:13.322+0000 7f765680f740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f764c1adf10 con 0x561c9813d0c0 2024-09-11T00:18:13.326 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:13.323+0000 7f7650e00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561c9813a790 0x561c98136380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:18:13.326 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:13.323+0000 7f7650e00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561c9813a790 0x561c98136380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:46462/0 (socket says 172.21.15.12:46462) 2024-09-11T00:18:13.326 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:13.323+0000 7f7650e00640 1 -- 172.21.15.12:0/2806579361 learned_addr learned my addr 172.21.15.12:0/2806579361 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:18:13.326 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:13.323+0000 7f764be00640 1 --2- 172.21.15.12:0/2806579361 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561c9813d0c0 0x561c98143cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:18:13.326 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:13.323+0000 7f7651800640 1 --2- 172.21.15.12:0/2806579361 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561c9813fb00 0x561c98144200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:18:13.326 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:13.323+0000 7f7650e00640 1 -- 172.21.15.12:0/2806579361 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561c9813d0c0 msgr2=0x561c98143cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:18:13.326 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:13.323+0000 7f7650e00640 1 --2- 172.21.15.12:0/2806579361 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561c9813d0c0 0x561c98143cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:18:13.327 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:13.323+0000 7f7650e00640 1 -- 172.21.15.12:0/2806579361 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561c9813fb00 msgr2=0x561c98144200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:18:13.327 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:13.323+0000 7f7650e00640 1 --2- 172.21.15.12:0/2806579361 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561c9813fb00 0x561c98144200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:18:13.327 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:13.323+0000 7f7650e00640 1 -- 172.21.15.12:0/2806579361 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561c9813cef0 con 0x561c9813a790 2024-09-11T00:18:13.327 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:13.323+0000 7f764be00640 1 --2- 172.21.15.12:0/2806579361 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561c9813d0c0 0x561c98143cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T00:18:13.327 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:13.323+0000 7f7650e00640 1 --2- 172.21.15.12:0/2806579361 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561c9813a790 0x561c98136380 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f763c0a9910 tx=0x7f763c000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46eacf07a7d1fe02 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:18:13.327 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:13.324+0000 7f7641600640 1 -- 172.21.15.12:0/2806579361 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f763c0b10d0 con 0x561c9813a790 2024-09-11T00:18:13.327 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:13.324+0000 7f7641600640 1 -- 172.21.15.12:0/2806579361 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f763c0b5b80 con 0x561c9813a790 2024-09-11T00:18:13.327 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:13.324+0000 7f7641600640 1 -- 172.21.15.12:0/2806579361 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f763c018e10 con 0x561c9813a790 2024-09-11T00:18:13.330 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:13.324+0000 7f765680f740 1 -- 172.21.15.12:0/2806579361 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561c9813a790 msgr2=0x561c98136380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:18:13.330 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:13.324+0000 7f765680f740 1 --2- 172.21.15.12:0/2806579361 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561c9813a790 0x561c98136380 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f763c0a9910 tx=0x7f763c000bc0 comp rx=0 tx=0).stop 2024-09-11T00:18:13.330 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:13.324+0000 7f765680f740 1 -- 172.21.15.12:0/2806579361 shutdown_connections 2024-09-11T00:18:13.330 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:13.324+0000 7f765680f740 1 --2- 172.21.15.12:0/2806579361 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561c9813fb00 0x561c98144200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:18:13.330 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:13.324+0000 7f765680f740 1 --2- 172.21.15.12:0/2806579361 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561c9813d0c0 0x561c98143cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:18:13.330 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:13.324+0000 7f765680f740 1 --2- 172.21.15.12:0/2806579361 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561c9813a790 0x561c98136380 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:18:13.331 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:13.324+0000 7f765680f740 1 -- 172.21.15.12:0/2806579361 >> 172.21.15.12:0/2806579361 conn(0x561c981358e0 msgr2=0x561c9813fee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:18:13.331 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:13.324+0000 7f765680f740 1 -- 172.21.15.12:0/2806579361 shutdown_connections 2024-09-11T00:18:13.331 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:13.324+0000 7f765680f740 1 -- 172.21.15.12:0/2806579361 wait complete. 2024-09-11T00:18:13.331 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:13.325+0000 7f765680f740 1 -- start start 2024-09-11T00:18:13.331 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:13.325+0000 7f765680f740 1 Processor -- start 2024-09-11T00:18:13.331 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:13.325+0000 7f765680f740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561c9813a790 0x561c9814b0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:18:13.331 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:13.325+0000 7f765680f740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561c9813d0c0 0x561c9814b720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:18:13.331 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:13.325+0000 7f765680f740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561c9813fb00 0x561c981499e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:18:13.331 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:13.325+0000 7f765680f740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f764c1aece0 con 0x561c9813d0c0 2024-09-11T00:18:13.331 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:13.325+0000 7f765680f740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f764c1ae5e0 con 0x561c9813a790 2024-09-11T00:18:13.331 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:13.325+0000 7f765680f740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f764c1ab7f0 con 0x561c9813fb00 2024-09-11T00:18:13.331 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:13.325+0000 7f765680f740 10 client.0 _ll_register_callbacks cb 0x7f7638009be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:18:13.331 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:13.325+0000 7f7650e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561c9813a790 0x561c9814b0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:18:13.331 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:13.325+0000 7f7650e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561c9813a790 0x561c9814b0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.12:32950/0 (socket says 172.21.15.12:32950) 2024-09-11T00:18:13.331 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:13.325+0000 7f7650e00640 1 -- 172.21.15.12:0/2292480691 learned_addr learned my addr 172.21.15.12:0/2292480691 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:18:13.332 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:13.325+0000 7f764be00640 1 --2- 172.21.15.12:0/2292480691 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561c9813d0c0 0x561c9814b720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:18:13.332 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:13.326+0000 7f7651800640 1 --2- 172.21.15.12:0/2292480691 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561c9813fb00 0x561c981499e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:18:13.332 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:13.326+0000 7f7650e00640 1 -- 172.21.15.12:0/2292480691 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561c9813fb00 msgr2=0x561c981499e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:18:13.332 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:13.326+0000 7f7650e00640 1 --2- 172.21.15.12:0/2292480691 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561c9813fb00 0x561c981499e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:18:13.332 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:13.326+0000 7f7650e00640 1 -- 172.21.15.12:0/2292480691 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561c9813d0c0 msgr2=0x561c9814b720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:18:13.332 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:13.326+0000 7f7650e00640 1 --2- 172.21.15.12:0/2292480691 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561c9813d0c0 0x561c9814b720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:18:13.332 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:13.326+0000 7f7650e00640 1 -- 172.21.15.12:0/2292480691 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561c9814a210 con 0x561c9813a790 2024-09-11T00:18:13.332 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:13.326+0000 7f7650e00640 1 --2- 172.21.15.12:0/2292480691 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561c9813a790 0x561c9814b0c0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f763c0ac3a0 tx=0x7f763c018420 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cf7fffe7faf71665 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:18:13.332 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:13.326+0000 7f7636a00640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-09-11T00:18:13.332 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:13.326+0000 7f7636a00640 1 -- 172.21.15.12:0/2292480691 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f763c018b90 con 0x561c9813a790 2024-09-11T00:18:13.332 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:13.326+0000 7f765680f740 1 -- 172.21.15.12:0/2292480691 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f763c0b5b80 con 0x561c9813a790 2024-09-11T00:18:13.332 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:13.326+0000 7f7636a00640 1 -- 172.21.15.12:0/2292480691 <== mon.2 v2:172.21.15.165:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9208+0+0 (secure 0 0 0) 0x7f763c018d30 con 0x561c9813a790 2024-09-11T00:18:13.332 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:13.326+0000 7f7636a00640 1 -- 172.21.15.12:0/2292480691 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f763c0ab900 con 0x561c9813a790 2024-09-11T00:18:13.332 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:13.326+0000 7f765680f740 10 client.26753 fetch_fsmap learned FSMap version 43 2024-09-11T00:18:13.333 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:13.326+0000 7f765680f740 1 -- 172.21.15.12:0/2292480691 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x561c981bd0d0 con 0x561c9813a790 2024-09-11T00:18:13.333 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:13.327+0000 7f7636a00640 1 -- 172.21.15.12:0/2292480691 <== mon.2 v2:172.21.15.165:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f763c0b5b80 con 0x561c9813a790 2024-09-11T00:18:13.333 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:13.327+0000 7f7636a00640 1 -- 172.21.15.12:0/2292480691 <== mon.2 v2:172.21.15.165:3300/0 5 ==== fsmap(e 43) ==== 8378+0+0 (secure 0 0 0) 0x7f763c0ab370 con 0x561c9813a790 2024-09-11T00:18:13.333 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:13.327+0000 7f765680f740 10 client.26753 fetch_fsmap finished waiting for FSMap version 43 2024-09-11T00:18:13.333 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:13.327+0000 7f765680f740 10 client.26753 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T00:18:13.333 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:13.327+0000 7f765680f740 20 client.26753 populate_metadata read hostname 'smithi012' 2024-09-11T00:18:13.333 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:13.327+0000 7f765680f740 1 --2- 172.21.15.12:0/2292480691 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x561c981bda60 0x561c981bfe80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:18:13.333 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:13.327+0000 7f765680f740 4 client.26753 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T00:18:13.333 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:13.327+0000 7f765680f740 1 -- 172.21.15.12:0/2292480691 --> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] -- command(tid 0: {"prefix": "flush journal"}) -- 0x561c980ab0b0 con 0x561c981bda60 2024-09-11T00:18:13.333 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:13.327+0000 7f764be00640 1 --2- 172.21.15.12:0/2292480691 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x561c981bda60 0x561c981bfe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:18:13.333 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:13.328+0000 7f764be00640 1 --2- 172.21.15.12:0/2292480691 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x561c981bda60 0x561c981bfe80 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f862482517579a9c server_cookie=6df2923868f2e6db in_seq=0 out_seq=0 2024-09-11T00:18:13.333 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:13.328+0000 7f7636a00640 10 client.26753 ms_handle_connect on v2:172.21.15.165:6838/501312984 2024-09-11T00:18:13.334 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:18:13 smithi178 ceph-mon[35919]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T00:18:13.396 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:13.393+0000 7f7636a00640 1 -- 172.21.15.12:0/2292480691 <== mds.0 v2:172.21.15.165:6838/501312984 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x561c980ab0b0 con 0x561c981bda60 2024-09-11T00:18:13.396 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:13.393+0000 7f7636a00640 10 client.26753 handle_command_reply: tid=0 2024-09-11T00:18:13.396 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:13.393+0000 7f765680f740 1 client.26753 shutdown 2024-09-11T00:18:13.396 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:13.394+0000 7f765680f740 1 -- 172.21.15.12:0/2292480691 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561c9813a790 msgr2=0x561c9814b0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:18:13.396 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:13.394+0000 7f765680f740 1 --2- 172.21.15.12:0/2292480691 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561c9813a790 0x561c9814b0c0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f763c0ac3a0 tx=0x7f763c018420 comp rx=0 tx=0).stop 2024-09-11T00:18:13.397 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:13.394+0000 7f765680f740 1 -- 172.21.15.12:0/2292480691 shutdown_connections 2024-09-11T00:18:13.397 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:13.394+0000 7f765680f740 1 --2- 172.21.15.12:0/2292480691 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x561c981bda60 0x561c981bfe80 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:18:13.397 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:13.394+0000 7f765680f740 1 --2- 172.21.15.12:0/2292480691 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561c9813fb00 0x561c981499e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:18:13.397 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:13.394+0000 7f765680f740 1 --2- 172.21.15.12:0/2292480691 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561c9813d0c0 0x561c9814b720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:18:13.397 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:13.394+0000 7f7636a00640 0 client.26753 ms_handle_reset on v2:172.21.15.165:6838/501312984 2024-09-11T00:18:13.397 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:13.394+0000 7f765680f740 1 --2- 172.21.15.12:0/2292480691 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561c9813a790 0x561c9814b0c0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:18:13.397 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:13.394+0000 7f765680f740 1 -- 172.21.15.12:0/2292480691 >> 172.21.15.12:0/2292480691 conn(0x561c981358e0 msgr2=0x561c98139810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:18:13.397 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:13.394+0000 7f765680f740 1 -- 172.21.15.12:0/2292480691 shutdown_connections 2024-09-11T00:18:13.397 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:13.394+0000 7f765680f740 1 -- 172.21.15.12:0/2292480691 wait complete. 2024-09-11T00:18:13.397 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:13.394+0000 7f765680f740 20 client.26753 trim_cache size 0 max 16384 2024-09-11T00:18:13.397 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:{ 2024-09-11T00:18:13.398 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "message": "", 2024-09-11T00:18:13.398 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "return_code": 0 2024-09-11T00:18:13.398 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:} 2024-09-11T00:18:13.398 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:13.395+0000 7f7653800640 1 -- 172.21.15.12:0/1797522826 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f76300b5270 msgr2=0x7f76300b7730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:18:13.398 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:13.395+0000 7f7653800640 1 --2- 172.21.15.12:0/1797522826 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f76300b5270 0x7f76300b7730 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f764402f470 tx=0x7f764402a8c0 comp rx=0 tx=0).stop 2024-09-11T00:18:13.398 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:13.395+0000 7f7653800640 1 -- 172.21.15.12:0/1797522826 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f764c104b90 msgr2=0x7f764c19aec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:18:13.399 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:13.395+0000 7f7653800640 1 --2- 172.21.15.12:0/1797522826 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f764c104b90 0x7f764c19aec0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f763c00e9f0 tx=0x7f763c00eeb0 comp rx=0 tx=0).stop 2024-09-11T00:18:13.399 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:13.395+0000 7f7653800640 1 -- 172.21.15.12:0/1797522826 shutdown_connections 2024-09-11T00:18:13.399 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:13.395+0000 7f7653800640 1 --2- 172.21.15.12:0/1797522826 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f76300b5270 0x7f76300b7730 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:18:13.399 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:13.395+0000 7f7653800640 1 --2- 172.21.15.12:0/1797522826 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f764c1066a0 0x7f764c19c140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:18:13.399 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:13.395+0000 7f7653800640 1 --2- 172.21.15.12:0/1797522826 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f764c105ce0 0x7f764c19b510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:18:13.399 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:13.395+0000 7f7653800640 1 --2- 172.21.15.12:0/1797522826 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f764c104b90 0x7f764c19aec0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:18:13.399 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:13.395+0000 7f7653800640 1 -- 172.21.15.12:0/1797522826 >> 172.21.15.12:0/1797522826 conn(0x7f764c100470 msgr2=0x7f764c109000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:18:13.399 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:13.395+0000 7f7653800640 1 -- 172.21.15.12:0/1797522826 shutdown_connections 2024-09-11T00:18:13.399 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:13.395+0000 7f7653800640 1 -- 172.21.15.12:0/1797522826 wait complete. 2024-09-11T00:18:13.690 INFO:tasks.workunit.client.0.smithi012.stdout: 1 479475 4.25 MB/sec execute 490 sec latency 747.714 ms 2024-09-11T00:18:14.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:18:14 smithi165 ceph-mon[35356]: pgmap v733: 129 pgs: 129 active+clean; 100 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 520 KiB/s rd, 2.1 MiB/s wr, 270 op/s 2024-09-11T00:18:14.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:18:14 smithi012 ceph-mon[30333]: pgmap v733: 129 pgs: 129 active+clean; 100 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 520 KiB/s rd, 2.1 MiB/s wr, 270 op/s 2024-09-11T00:18:14.330 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:18:14 smithi178 ceph-mon[35919]: pgmap v733: 129 pgs: 129 active+clean; 100 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 520 KiB/s rd, 2.1 MiB/s wr, 270 op/s 2024-09-11T00:18:14.690 INFO:tasks.workunit.client.0.smithi012.stdout: 1 479475 4.24 MB/sec execute 491 sec latency 1747.786 ms 2024-09-11T00:18:15.690 INFO:tasks.workunit.client.0.smithi012.stdout: 1 479475 4.23 MB/sec execute 492 sec latency 2747.854 ms 2024-09-11T00:18:16.089 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:18:16 smithi012 ceph-mon[30333]: pgmap v734: 129 pgs: 129 active+clean; 99 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 386 KiB/s rd, 1.9 MiB/s wr, 219 op/s 2024-09-11T00:18:16.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:18:16 smithi165 ceph-mon[35356]: pgmap v734: 129 pgs: 129 active+clean; 99 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 386 KiB/s rd, 1.9 MiB/s wr, 219 op/s 2024-09-11T00:18:16.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:18:16 smithi178 ceph-mon[35919]: pgmap v734: 129 pgs: 129 active+clean; 99 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 386 KiB/s rd, 1.9 MiB/s wr, 219 op/s 2024-09-11T00:18:16.690 INFO:tasks.quiescer.fs.[cephfs]:Total duration (50.8) is longer than `split_if_longer` (35.0), will split into two consecutive quiesces 2024-09-11T00:18:16.691 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 25.4 2024-09-11T00:18:16.691 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid ff3ad74a-6fcf-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T00:18:16.692 INFO:tasks.workunit.client.0.smithi012.stdout: 1 479476 4.22 MB/sec execute 493 sec latency 3667.330 ms 2024-09-11T00:18:17.106 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:18:17.691 INFO:tasks.workunit.client.0.smithi012.stdout: 1 479501 4.21 MB/sec execute 494 sec latency 657.480 ms 2024-09-11T00:18:17.723 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:17.720+0000 7fd11f200640 1 -- 172.21.15.12:0/284331737 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd11810f760 msgr2=0x7fd118115f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:18:17.724 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:17.721+0000 7fd11f200640 1 --2- 172.21.15.12:0/284331737 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd11810f760 0x7fd118115f60 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fd110009ca0 tx=0x7fd11002a8f0 comp rx=0 tx=0).stop 2024-09-11T00:18:17.724 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:17.722+0000 7fd11f200640 1 -- 172.21.15.12:0/284331737 shutdown_connections 2024-09-11T00:18:17.725 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:17.722+0000 7fd11f200640 1 --2- 172.21.15.12:0/284331737 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd11810f760 0x7fd118115f60 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:18:17.725 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:17.722+0000 7fd11f200640 1 --2- 172.21.15.12:0/284331737 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd11810ede0 0x7fd11810f220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:18:17.725 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:17.722+0000 7fd11f200640 1 --2- 172.21.15.12:0/284331737 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd1181093d0 0x7fd1181097b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:18:17.725 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:17.722+0000 7fd11f200640 1 -- 172.21.15.12:0/284331737 >> 172.21.15.12:0/284331737 conn(0x7fd118078a40 msgr2=0x7fd11807ae60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:18:17.726 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:17.724+0000 7fd11f200640 1 -- 172.21.15.12:0/284331737 shutdown_connections 2024-09-11T00:18:17.726 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:17.724+0000 7fd11f200640 1 -- 172.21.15.12:0/284331737 wait complete. 2024-09-11T00:18:17.727 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:17.725+0000 7fd11f200640 1 Processor -- start 2024-09-11T00:18:17.728 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:17.726+0000 7fd11f200640 1 -- start start 2024-09-11T00:18:17.729 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:17.726+0000 7fd11f200640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd1181093d0 0x7fd11819f800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:18:17.729 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:17.726+0000 7fd11f200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd11810ede0 0x7fd11819fd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:18:17.729 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:17.726+0000 7fd11f200640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd11810f760 0x7fd1181a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:18:17.730 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:17.726+0000 7fd11f200640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fd118119390 con 0x7fd1181093d0 2024-09-11T00:18:17.731 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:17.726+0000 7fd11f200640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fd118119210 con 0x7fd11810ede0 2024-09-11T00:18:17.731 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:17.726+0000 7fd11f200640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fd118119510 con 0x7fd11810f760 2024-09-11T00:18:17.731 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:17.727+0000 7fd11d200640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd11810f760 0x7fd1181a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:18:17.731 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:17.727+0000 7fd11d200640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd11810f760 0x7fd1181a50e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.12:55854/0 (socket says 172.21.15.12:55854) 2024-09-11T00:18:17.731 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:17.727+0000 7fd11d200640 1 -- 172.21.15.12:0/3655921449 learned_addr learned my addr 172.21.15.12:0/3655921449 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:18:17.731 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:17.727+0000 7fd117400640 1 --2- 172.21.15.12:0/3655921449 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd11810ede0 0x7fd11819fd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:18:17.731 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:17.727+0000 7fd117e00640 1 --2- 172.21.15.12:0/3655921449 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd1181093d0 0x7fd11819f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:18:17.731 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:17.727+0000 7fd11d200640 1 -- 172.21.15.12:0/3655921449 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd11810ede0 msgr2=0x7fd11819fd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:18:17.731 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:17.727+0000 7fd11d200640 1 --2- 172.21.15.12:0/3655921449 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd11810ede0 0x7fd11819fd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:18:17.731 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:17.727+0000 7fd11d200640 1 -- 172.21.15.12:0/3655921449 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd1181093d0 msgr2=0x7fd11819f800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:18:17.731 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:17.727+0000 7fd11d200640 1 --2- 172.21.15.12:0/3655921449 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd1181093d0 0x7fd11819f800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:18:17.731 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:17.727+0000 7fd11d200640 1 -- 172.21.15.12:0/3655921449 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1181a58b0 con 0x7fd11810f760 2024-09-11T00:18:17.732 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:17.728+0000 7fd11d200640 1 --2- 172.21.15.12:0/3655921449 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd11810f760 0x7fd1181a50e0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fd110031f80 tx=0x7fd1100305f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:18:17.732 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:17.728+0000 7fd114c00640 1 -- 172.21.15.12:0/3655921449 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1100309d0 con 0x7fd11810f760 2024-09-11T00:18:17.732 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:17.728+0000 7fd114c00640 1 -- 172.21.15.12:0/3655921449 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd11002d070 con 0x7fd11810f760 2024-09-11T00:18:17.732 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:17.728+0000 7fd114c00640 1 -- 172.21.15.12:0/3655921449 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd11003a420 con 0x7fd11810f760 2024-09-11T00:18:17.732 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:17.729+0000 7fd11f200640 1 -- 172.21.15.12:0/3655921449 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd1181a5b40 con 0x7fd11810f760 2024-09-11T00:18:17.732 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:17.729+0000 7fd11f200640 1 -- 172.21.15.12:0/3655921449 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd11806cb70 con 0x7fd11810f760 2024-09-11T00:18:17.733 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:17.731+0000 7fd11f200640 1 -- 172.21.15.12:0/3655921449 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd1181097b0 con 0x7fd11810f760 2024-09-11T00:18:17.736 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:17.734+0000 7fd114c00640 1 -- 172.21.15.12:0/3655921449 <== mon.1 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7fd11003b620 con 0x7fd11810f760 2024-09-11T00:18:17.738 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:17.736+0000 7fd114c00640 1 --2- 172.21.15.12:0/3655921449 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fd0fc0b53a0 0x7fd0fc0b7860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:18:17.738 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:17.736+0000 7fd114c00640 1 -- 172.21.15.12:0/3655921449 <== mon.1 v2:172.21.15.178:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9208+0+0 (secure 0 0 0) 0x7fd1101070e0 con 0x7fd11810f760 2024-09-11T00:18:17.739 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:17.736+0000 7fd117e00640 1 --2- 172.21.15.12:0/3655921449 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fd0fc0b53a0 0x7fd0fc0b7860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:18:17.739 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:17.737+0000 7fd117e00640 1 --2- 172.21.15.12:0/3655921449 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fd0fc0b53a0 0x7fd0fc0b7860 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fd108005d00 tx=0x7fd108005c50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:18:17.740 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:17.738+0000 7fd114c00640 1 -- 172.21.15.12:0/3655921449 <== mon.1 v2:172.21.15.178:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7fd1100433d0 con 0x7fd11810f760 2024-09-11T00:18:17.974 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:17.972+0000 7fd11f200640 1 -- 172.21.15.12:0/3655921449 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd11806cf70 con 0x7fd11810f760 2024-09-11T00:18:17.975 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:17.973+0000 7fd114c00640 1 -- 172.21.15.12:0/3655921449 <== mon.1 v2:172.21.15.178:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 44 v44) ==== 94+0+11969 (secure 0 0 0) 0x7fd1100cdbd0 con 0x7fd11810f760 2024-09-11T00:18:17.976 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-11T00:18:17.976 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":44,"btime":"2024-09-11T00:18:17:022132+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14577,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6835/447171591","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":447171591},{"type":"v1","addr":"172.21.15.12:6835","nonce":447171591}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14589,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/901486040","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":901486040},{"type":"v1","addr":"172.21.15.12:6837","nonce":901486040}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":44,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T00:04:04.739881+0000","modified":"2024-09-11T00:18:16.957829+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24674,"mds_1":24643,"mds_2":24638,"mds_3":24637,"mds_4":24614},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14631":{"gid":14631,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.12:6841/3069919437","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6840","nonce":3069919437},{"type":"v1","addr":"172.21.15.12:6841","nonce":3069919437}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24562":{"gid":24562,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6833/1141636112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":1141636112},{"type":"v1","addr":"172.21.15.178:6833","nonce":1141636112}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24581":{"gid":24581,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6833/3111980970","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6832","nonce":3111980970},{"type":"v1","addr":"172.21.15.165:6833","nonce":3111980970}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24607":{"gid":24607,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6835/4084868287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":4084868287},{"type":"v1","addr":"172.21.15.178:6835","nonce":4084868287}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24614":{"gid":24614,"name":"e","rank":4,"incarnation":26,"state":"up:active","state_seq":21,"addr":"172.21.15.165:6835/2425395274","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":2425395274},{"type":"v1","addr":"172.21.15.165:6835","nonce":2425395274}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24637":{"gid":24637,"name":"i","rank":3,"incarnation":24,"state":"up:active","state_seq":18,"addr":"172.21.15.178:6839/70628097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6838","nonce":70628097},{"type":"v1","addr":"172.21.15.178:6839","nonce":70628097}]},"join_fscid":-1,"export_targets":[0],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24638":{"gid":24638,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":18,"addr":"172.21.15.165:6837/6215088","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6836","nonce":6215088},{"type":"v1","addr":"172.21.15.165:6837","nonce":6215088}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24643":{"gid":24643,"name":"g","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.12:6839/3124093009","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6838","nonce":3124093009},{"type":"v1","addr":"172.21.15.12:6839","nonce":3124093009}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24674":{"gid":24674,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.165:6839/501312984","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6838","nonce":501312984},{"type":"v1","addr":"172.21.15.165:6839","nonce":501312984}]},"join_fscid":-1,"export_targets":[1,3],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25672":{"gid":25672,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6837/308601741","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":308601741},{"type":"v1","addr":"172.21.15.178:6837","nonce":308601741}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24674,"qdb_cluster":[24674,24643,24638,24637,24614]},"id":1}]} 2024-09-11T00:18:17.976 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 44 2024-09-11T00:18:17.978 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:17.976+0000 7fd11f200640 1 -- 172.21.15.12:0/3655921449 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fd0fc0b53a0 msgr2=0x7fd0fc0b7860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:18:17.978 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:17.976+0000 7fd11f200640 1 --2- 172.21.15.12:0/3655921449 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fd0fc0b53a0 0x7fd0fc0b7860 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fd108005d00 tx=0x7fd108005c50 comp rx=0 tx=0).stop 2024-09-11T00:18:17.978 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:17.976+0000 7fd11f200640 1 -- 172.21.15.12:0/3655921449 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd11810f760 msgr2=0x7fd1181a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:18:17.979 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:17.976+0000 7fd11f200640 1 --2- 172.21.15.12:0/3655921449 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd11810f760 0x7fd1181a50e0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fd110031f80 tx=0x7fd1100305f0 comp rx=0 tx=0).stop 2024-09-11T00:18:17.979 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:17.976+0000 7fd11f200640 1 -- 172.21.15.12:0/3655921449 shutdown_connections 2024-09-11T00:18:17.979 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:17.976+0000 7fd11f200640 1 --2- 172.21.15.12:0/3655921449 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fd0fc0b53a0 0x7fd0fc0b7860 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:18:17.979 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:17.976+0000 7fd11f200640 1 --2- 172.21.15.12:0/3655921449 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd11810f760 0x7fd1181a50e0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:18:17.979 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:17.976+0000 7fd11f200640 1 --2- 172.21.15.12:0/3655921449 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd11810ede0 0x7fd11819fd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:18:17.979 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:17.976+0000 7fd11f200640 1 --2- 172.21.15.12:0/3655921449 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd1181093d0 0x7fd11819f800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:18:17.979 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:17.976+0000 7fd11f200640 1 -- 172.21.15.12:0/3655921449 >> 172.21.15.12:0/3655921449 conn(0x7fd118078a40 msgr2=0x7fd11807ae30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:18:17.979 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:17.976+0000 7fd11f200640 1 -- 172.21.15.12:0/3655921449 shutdown_connections 2024-09-11T00:18:17.979 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:17.976+0000 7fd11f200640 1 -- 172.21.15.12:0/3655921449 wait complete. 2024-09-11T00:18:18.285 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:18:18 smithi165 ceph-mon[35356]: pgmap v735: 129 pgs: 129 active+clean; 99 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 465 KiB/s rd, 2.6 MiB/s wr, 281 op/s 2024-09-11T00:18:18.285 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:18:18 smithi165 ceph-mon[35356]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T00:18:18.286 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:18:18 smithi165 ceph-mon[35356]: from='client.? 172.21.15.12:0/3655921449' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T00:18:18.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:18:18 smithi012 ceph-mon[30333]: pgmap v735: 129 pgs: 129 active+clean; 99 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 465 KiB/s rd, 2.6 MiB/s wr, 281 op/s 2024-09-11T00:18:18.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:18:18 smithi012 ceph-mon[30333]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T00:18:18.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:18:18 smithi012 ceph-mon[30333]: from='client.? 172.21.15.12:0/3655921449' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T00:18:18.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:18:18 smithi178 ceph-mon[35919]: pgmap v735: 129 pgs: 129 active+clean; 99 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 465 KiB/s rd, 2.6 MiB/s wr, 281 op/s 2024-09-11T00:18:18.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:18:18 smithi178 ceph-mon[35919]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T00:18:18.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:18:18 smithi178 ceph-mon[35919]: from='client.? 172.21.15.12:0/3655921449' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T00:18:18.692 INFO:tasks.workunit.client.0.smithi012.stdout: 1 479501 4.20 MB/sec execute 495 sec latency 1657.532 ms 2024-09-11T00:18:19.074 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24674 quiesce db / --timeout 90 --expiration 145.4 --await-for 45.0' 2024-09-11T00:18:19.075 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid ff3ad74a-6fcf-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24674 quiesce db / --timeout 90 --expiration 145.4 --await-for 45.0 2024-09-11T00:18:19.461 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:18:19.691 INFO:tasks.workunit.client.0.smithi012.stdout: 1 479501 4.20 MB/sec execute 496 sec latency 2657.613 ms 2024-09-11T00:18:20.233 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:18:20 smithi012 ceph-mon[30333]: pgmap v736: 129 pgs: 129 active+clean; 100 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 282 KiB/s rd, 1.8 MiB/s wr, 183 op/s 2024-09-11T00:18:20.252 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:20.249+0000 7fdd57a00640 1 -- 172.21.15.12:0/176215294 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdd5010a9b0 msgr2=0x7fdd5010adf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:18:20.252 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:20.249+0000 7fdd57a00640 1 --2- 172.21.15.12:0/176215294 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdd5010a9b0 0x7fdd5010adf0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7fdd3c009900 tx=0x7fdd3c02a410 comp rx=0 tx=0).stop 2024-09-11T00:18:20.252 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:20.249+0000 7fdd57a00640 1 -- 172.21.15.12:0/176215294 shutdown_connections 2024-09-11T00:18:20.252 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:20.249+0000 7fdd57a00640 1 --2- 172.21.15.12:0/176215294 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fdd5010b330 0x7fdd50111b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:18:20.252 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:20.249+0000 7fdd57a00640 1 --2- 172.21.15.12:0/176215294 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdd5010a9b0 0x7fdd5010adf0 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:18:20.252 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:20.249+0000 7fdd57a00640 1 --2- 172.21.15.12:0/176215294 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdd50104fa0 0x7fdd50105380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:18:20.252 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:20.250+0000 7fdd57a00640 1 -- 172.21.15.12:0/176215294 >> 172.21.15.12:0/176215294 conn(0x7fdd50100a50 msgr2=0x7fdd50102e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:18:20.253 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:20.250+0000 7fdd57a00640 1 -- 172.21.15.12:0/176215294 shutdown_connections 2024-09-11T00:18:20.253 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:20.250+0000 7fdd57a00640 1 -- 172.21.15.12:0/176215294 wait complete. 2024-09-11T00:18:20.254 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:20.251+0000 7fdd57a00640 1 Processor -- start 2024-09-11T00:18:20.254 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:20.251+0000 7fdd57a00640 1 -- start start 2024-09-11T00:18:20.254 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:20.251+0000 7fdd57a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fdd50104fa0 0x7fdd5019b470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:18:20.254 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:20.251+0000 7fdd57a00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdd5010a9b0 0x7fdd5019b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:18:20.254 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:20.251+0000 7fdd57a00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdd5010b330 0x7fdd501a0d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:18:20.254 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:20.251+0000 7fdd57a00640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fdd50114f50 con 0x7fdd5010b330 2024-09-11T00:18:20.254 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:20.251+0000 7fdd57a00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fdd50114dd0 con 0x7fdd50104fa0 2024-09-11T00:18:20.254 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:20.251+0000 7fdd57a00640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fdd501150d0 con 0x7fdd5010a9b0 2024-09-11T00:18:20.254 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:20.252+0000 7fdd55a00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdd5010b330 0x7fdd501a0d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:18:20.254 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:20.252+0000 7fdd55000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fdd50104fa0 0x7fdd5019b470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:18:20.254 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:20.252+0000 7fdd4fe00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdd5010a9b0 0x7fdd5019b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:18:20.255 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:20.252+0000 7fdd55a00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdd5010b330 0x7fdd501a0d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:41784/0 (socket says 172.21.15.12:41784) 2024-09-11T00:18:20.255 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:20.252+0000 7fdd55000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fdd50104fa0 0x7fdd5019b470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.12:56794/0 (socket says 172.21.15.12:56794) 2024-09-11T00:18:20.255 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:20.252+0000 7fdd55a00640 1 -- 172.21.15.12:0/998430815 learned_addr learned my addr 172.21.15.12:0/998430815 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:18:20.255 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:20.252+0000 7fdd4fe00640 1 -- 172.21.15.12:0/998430815 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fdd50104fa0 msgr2=0x7fdd5019b470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:18:20.256 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:20.252+0000 7fdd4fe00640 1 --2- 172.21.15.12:0/998430815 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fdd50104fa0 0x7fdd5019b470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:18:20.256 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:20.252+0000 7fdd4fe00640 1 -- 172.21.15.12:0/998430815 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdd5010b330 msgr2=0x7fdd501a0d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:18:20.256 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:20.252+0000 7fdd4fe00640 1 --2- 172.21.15.12:0/998430815 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdd5010b330 0x7fdd501a0d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:18:20.256 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:20.252+0000 7fdd4fe00640 1 -- 172.21.15.12:0/998430815 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd501a1520 con 0x7fdd5010a9b0 2024-09-11T00:18:20.256 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:20.252+0000 7fdd55000640 1 --2- 172.21.15.12:0/998430815 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fdd50104fa0 0x7fdd5019b470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:18:20.256 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:20.252+0000 7fdd55a00640 1 --2- 172.21.15.12:0/998430815 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdd5010b330 0x7fdd501a0d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:18:20.256 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:20.253+0000 7fdd4fe00640 1 --2- 172.21.15.12:0/998430815 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdd5010a9b0 0x7fdd5019b9b0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fdd3c02a930 tx=0x7fdd3c031a20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:18:20.256 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:20.253+0000 7fdd4d600640 1 -- 172.21.15.12:0/998430815 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd3c03f030 con 0x7fdd5010a9b0 2024-09-11T00:18:20.256 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:20.253+0000 7fdd57a00640 1 -- 172.21.15.12:0/998430815 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdd501a17b0 con 0x7fdd5010a9b0 2024-09-11T00:18:20.256 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:20.253+0000 7fdd4d600640 1 -- 172.21.15.12:0/998430815 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdd3c004490 con 0x7fdd5010a9b0 2024-09-11T00:18:20.256 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:20.253+0000 7fdd4d600640 1 -- 172.21.15.12:0/998430815 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd3c035070 con 0x7fdd5010a9b0 2024-09-11T00:18:20.256 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:20.253+0000 7fdd57a00640 1 -- 172.21.15.12:0/998430815 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdd501a7ed0 con 0x7fdd5010a9b0 2024-09-11T00:18:20.258 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:20.256+0000 7fdd5a676740 1 Processor -- start 2024-09-11T00:18:20.259 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:20.256+0000 7fdd5a676740 1 -- start start 2024-09-11T00:18:20.259 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:20.257+0000 7fdd4d600640 1 -- 172.21.15.12:0/998430815 <== mon.1 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7fdd3c03ba30 con 0x7fdd5010a9b0 2024-09-11T00:18:20.259 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:20.257+0000 7fdd5a676740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fdda03b180 0x55fdda03b560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:18:20.259 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:20.257+0000 7fdd5a676740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fdda03baa0 0x55fdda0474f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:18:20.259 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:20.257+0000 7fdd5a676740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fdda047a30 0x55fdda049e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:18:20.260 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:20.257+0000 7fdd5a676740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55fdd9fb0ca0 con 0x55fdda03baa0 2024-09-11T00:18:20.260 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:20.257+0000 7fdd5a676740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55fdd9fa00d0 con 0x55fdda03b180 2024-09-11T00:18:20.260 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:20.257+0000 7fdd5a676740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55fdd9fa3ee0 con 0x55fdda047a30 2024-09-11T00:18:20.260 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:20.257+0000 7fdd55000640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fdda03baa0 0x55fdda0474f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:18:20.260 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:20.257+0000 7fdd55000640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fdda03baa0 0x55fdda0474f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:41788/0 (socket says 172.21.15.12:41788) 2024-09-11T00:18:20.260 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:20.257+0000 7fdd55000640 1 -- 172.21.15.12:0/2882222904 learned_addr learned my addr 172.21.15.12:0/2882222904 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:18:20.260 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:20.257+0000 7fdd55a00640 1 --2- 172.21.15.12:0/2882222904 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fdda03b180 0x55fdda03b560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:18:20.260 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:20.257+0000 7fdd4fe00640 1 --2- 172.21.15.12:0/2882222904 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fdda047a30 0x55fdda049e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:18:20.261 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:20.258+0000 7fdd55000640 1 -- 172.21.15.12:0/2882222904 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fdda047a30 msgr2=0x55fdda049e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:18:20.262 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:20.258+0000 7fdd55000640 1 --2- 172.21.15.12:0/2882222904 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fdda047a30 0x55fdda049e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:18:20.262 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:20.258+0000 7fdd55000640 1 -- 172.21.15.12:0/2882222904 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fdda03b180 msgr2=0x55fdda03b560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:18:20.262 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:20.258+0000 7fdd55000640 1 --2- 172.21.15.12:0/2882222904 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fdda03b180 0x55fdda03b560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:18:20.262 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:20.258+0000 7fdd55000640 1 -- 172.21.15.12:0/2882222904 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fdd9f52400 con 0x55fdda03baa0 2024-09-11T00:18:20.262 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:20.258+0000 7fdd4fe00640 1 --2- 172.21.15.12:0/2882222904 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fdda047a30 0x55fdda049e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T00:18:20.262 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:20.258+0000 7fdd55000640 1 --2- 172.21.15.12:0/2882222904 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fdda03baa0 0x55fdda0474f0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7fdd400115c0 tx=0x7fdd400119c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b773c44944eb231d server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:18:20.262 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:20.258+0000 7fdd46000640 1 -- 172.21.15.12:0/2882222904 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd4001e070 con 0x55fdda03baa0 2024-09-11T00:18:20.262 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:20.258+0000 7fdd46000640 1 -- 172.21.15.12:0/2882222904 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdd4000f9f0 con 0x55fdda03baa0 2024-09-11T00:18:20.262 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:20.258+0000 7fdd46000640 1 -- 172.21.15.12:0/2882222904 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd40019a20 con 0x55fdda03baa0 2024-09-11T00:18:20.262 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:20.258+0000 7fdd4d600640 1 --2- 172.21.15.12:0/998430815 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fdd340b52e0 0x7fdd340b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:18:20.262 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:20.259+0000 7fdd4d600640 1 -- 172.21.15.12:0/998430815 <== mon.1 v2:172.21.15.178:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9208+0+0 (secure 0 0 0) 0x7fdd3c106550 con 0x7fdd5010a9b0 2024-09-11T00:18:20.262 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:20.259+0000 7fdd55a00640 1 --2- 172.21.15.12:0/998430815 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fdd340b52e0 0x7fdd340b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:18:20.263 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:20.259+0000 7fdd5a676740 1 -- 172.21.15.12:0/2882222904 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fdda03baa0 msgr2=0x55fdda0474f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:18:20.263 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:20.259+0000 7fdd5a676740 1 --2- 172.21.15.12:0/2882222904 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fdda03baa0 0x55fdda0474f0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7fdd400115c0 tx=0x7fdd400119c0 comp rx=0 tx=0).stop 2024-09-11T00:18:20.263 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:20.259+0000 7fdd5a676740 1 -- 172.21.15.12:0/2882222904 shutdown_connections 2024-09-11T00:18:20.263 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:20.259+0000 7fdd5a676740 1 --2- 172.21.15.12:0/2882222904 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fdda047a30 0x55fdda049e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:18:20.263 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:20.259+0000 7fdd5a676740 1 --2- 172.21.15.12:0/2882222904 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fdda03baa0 0x55fdda0474f0 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:18:20.263 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:20.259+0000 7fdd5a676740 1 --2- 172.21.15.12:0/2882222904 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fdda03b180 0x55fdda03b560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:18:20.263 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:20.259+0000 7fdd5a676740 1 -- 172.21.15.12:0/2882222904 >> 172.21.15.12:0/2882222904 conn(0x55fdda038fb0 msgr2=0x55fdda03a150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:18:20.263 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:20.259+0000 7fdd5a676740 1 -- 172.21.15.12:0/2882222904 shutdown_connections 2024-09-11T00:18:20.263 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:20.259+0000 7fdd5a676740 1 -- 172.21.15.12:0/2882222904 wait complete. 2024-09-11T00:18:20.265 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:20.260+0000 7fdd5a676740 1 -- start start 2024-09-11T00:18:20.265 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:20.260+0000 7fdd5a676740 1 Processor -- start 2024-09-11T00:18:20.265 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:20.260+0000 7fdd55a00640 1 --2- 172.21.15.12:0/998430815 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fdd340b52e0 0x7fdd340b77a0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fdd480049d0 tx=0x7fdd48009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:18:20.266 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:20.261+0000 7fdd5a676740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fdda03b180 0x55fdda0473b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:18:20.266 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:20.261+0000 7fdd5a676740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fdda03baa0 0x55fdda044800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:18:20.266 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:20.261+0000 7fdd5a676740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fdda047a30 0x55fdda044ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:18:20.266 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:20.261+0000 7fdd5a676740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55fdd9e017a0 con 0x55fdda03baa0 2024-09-11T00:18:20.266 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:20.261+0000 7fdd5a676740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55fdd9f45a10 con 0x55fdda03b180 2024-09-11T00:18:20.266 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:20.261+0000 7fdd5a676740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55fdda04a6b0 con 0x55fdda047a30 2024-09-11T00:18:20.266 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:20.261+0000 7fdd5a676740 10 client.0 _ll_register_callbacks cb 0x55fdd9f55be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:18:20.266 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:20.261+0000 7fdd4fe00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fdda03baa0 0x55fdda044800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:18:20.266 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:20.261+0000 7fdd55000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fdda03b180 0x55fdda0473b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:18:20.266 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:20.261+0000 7fdd4fe00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fdda03baa0 0x55fdda044800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:41794/0 (socket says 172.21.15.12:41794) 2024-09-11T00:18:20.266 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:20.261+0000 7fdd4fe00640 1 -- 172.21.15.12:0/3657520660 learned_addr learned my addr 172.21.15.12:0/3657520660 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:18:20.266 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:20.261+0000 7fdd55000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fdda03b180 0x55fdda0473b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.12:56808/0 (socket says 172.21.15.12:56808) 2024-09-11T00:18:20.267 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:20.261+0000 7fdd55a00640 1 --2- 172.21.15.12:0/3657520660 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fdda047a30 0x55fdda044ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:18:20.267 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:20.262+0000 7fdd55000640 1 -- 172.21.15.12:0/3657520660 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fdda047a30 msgr2=0x55fdda044ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:18:20.267 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:20.262+0000 7fdd55000640 1 --2- 172.21.15.12:0/3657520660 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fdda047a30 0x55fdda044ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:18:20.267 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:20.262+0000 7fdd55000640 1 -- 172.21.15.12:0/3657520660 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fdda03baa0 msgr2=0x55fdda044800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:18:20.267 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:20.262+0000 7fdd55000640 1 --2- 172.21.15.12:0/3657520660 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fdda03baa0 0x55fdda044800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:18:20.267 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:20.262+0000 7fdd55000640 1 -- 172.21.15.12:0/3657520660 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fdd9d0db90 con 0x55fdda03b180 2024-09-11T00:18:20.267 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:20.262+0000 7fdd55a00640 1 --2- 172.21.15.12:0/3657520660 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fdda047a30 0x55fdda044ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T00:18:20.267 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:20.262+0000 7fdd55000640 1 --2- 172.21.15.12:0/3657520660 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fdda03b180 0x55fdda0473b0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fdd40014040 tx=0x7fdd4000f7f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b081f4490594ba58 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:18:20.267 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:20.262+0000 7fdd3aa00640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-09-11T00:18:20.267 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:20.262+0000 7fdd3aa00640 1 -- 172.21.15.12:0/3657520660 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd4001e040 con 0x55fdda03b180 2024-09-11T00:18:20.267 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:20.262+0000 7fdd5a676740 1 -- 172.21.15.12:0/3657520660 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdd4000f9f0 con 0x55fdda03b180 2024-09-11T00:18:20.267 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:20.262+0000 7fdd3aa00640 1 -- 172.21.15.12:0/3657520660 <== mon.2 v2:172.21.15.165:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9208+0+0 (secure 0 0 0) 0x7fdd40004040 con 0x55fdda03b180 2024-09-11T00:18:20.267 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:20.262+0000 7fdd3aa00640 1 -- 172.21.15.12:0/3657520660 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7fdd4002f050 con 0x55fdda03b180 2024-09-11T00:18:20.268 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:20.262+0000 7fdd5a676740 10 client.26771 fetch_fsmap learned FSMap version 44 2024-09-11T00:18:20.268 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:20.262+0000 7fdd5a676740 1 -- 172.21.15.12:0/3657520660 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x55fdda0560d0 con 0x55fdda03b180 2024-09-11T00:18:20.268 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:20.263+0000 7fdd3aa00640 1 -- 172.21.15.12:0/3657520660 <== mon.2 v2:172.21.15.165:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdd4000f9f0 con 0x55fdda03b180 2024-09-11T00:18:20.268 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:20.263+0000 7fdd3aa00640 1 -- 172.21.15.12:0/3657520660 <== mon.2 v2:172.21.15.165:3300/0 5 ==== fsmap(e 44) ==== 8382+0+0 (secure 0 0 0) 0x7fdd4002a030 con 0x55fdda03b180 2024-09-11T00:18:20.268 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:20.263+0000 7fdd5a676740 10 client.26771 fetch_fsmap finished waiting for FSMap version 44 2024-09-11T00:18:20.268 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:20.263+0000 7fdd5a676740 10 client.26771 resolve_mds: validated gid 24674 aka daemon mds.b 2024-09-11T00:18:20.268 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:20.263+0000 7fdd5a676740 20 client.26771 populate_metadata read hostname 'smithi012' 2024-09-11T00:18:20.268 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:20.263+0000 7fdd5a676740 1 --2- 172.21.15.12:0/3657520660 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55fdda057480 0x55fdda059870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:18:20.268 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:20.263+0000 7fdd5a676740 4 client.26771 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T00:18:20.268 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:20.263+0000 7fdd5a676740 1 -- 172.21.15.12:0/3657520660 --> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fdd9fb2bf0 con 0x55fdda057480 2024-09-11T00:18:20.269 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:20.264+0000 7fdd4fe00640 1 --2- 172.21.15.12:0/3657520660 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55fdda057480 0x55fdda059870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:18:20.270 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:20.264+0000 7fdd4fe00640 1 --2- 172.21.15.12:0/3657520660 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55fdda057480 0x55fdda059870 crc :-1 s=READY pgs=172 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fa6097f7cdb3a8a6 server_cookie=e81e4aee191d3307 in_seq=0 out_seq=0 2024-09-11T00:18:20.270 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:20.264+0000 7fdd3aa00640 10 client.26771 ms_handle_connect on v2:172.21.15.165:6838/501312984 2024-09-11T00:18:20.270 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:20.265+0000 7fdd3aa00640 1 -- 172.21.15.12:0/3657520660 <== mds.0 v2:172.21.15.165:6838/501312984 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55fdd9fb2bf0 con 0x55fdda057480 2024-09-11T00:18:20.270 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:20.265+0000 7fdd3aa00640 10 client.26771 handle_command_reply: tid=0 2024-09-11T00:18:20.270 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:20.265+0000 7fdd5a676740 1 client.26771 shutdown 2024-09-11T00:18:20.270 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:20.265+0000 7fdd5a676740 1 -- 172.21.15.12:0/3657520660 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fdda03b180 msgr2=0x55fdda0473b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:18:20.270 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:20.265+0000 7fdd5a676740 1 --2- 172.21.15.12:0/3657520660 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fdda03b180 0x55fdda0473b0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fdd40014040 tx=0x7fdd4000f7f0 comp rx=0 tx=0).stop 2024-09-11T00:18:20.270 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:20.265+0000 7fdd5a676740 1 -- 172.21.15.12:0/3657520660 shutdown_connections 2024-09-11T00:18:20.270 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:20.265+0000 7fdd5a676740 1 --2- 172.21.15.12:0/3657520660 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55fdda057480 0x55fdda059870 crc :-1 s=READY pgs=172 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:18:20.270 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:20.265+0000 7fdd5a676740 1 --2- 172.21.15.12:0/3657520660 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fdda047a30 0x55fdda044ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:18:20.270 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:20.265+0000 7fdd5a676740 1 --2- 172.21.15.12:0/3657520660 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fdda03baa0 0x55fdda044800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:18:20.270 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:20.265+0000 7fdd5a676740 1 --2- 172.21.15.12:0/3657520660 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fdda03b180 0x55fdda0473b0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:18:20.270 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:20.265+0000 7fdd5a676740 1 -- 172.21.15.12:0/3657520660 >> 172.21.15.12:0/3657520660 conn(0x55fdda038fb0 msgr2=0x55fdda03eb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:18:20.271 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:20.265+0000 7fdd3aa00640 0 client.26771 ms_handle_reset on v2:172.21.15.165:6838/501312984 2024-09-11T00:18:20.271 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:20.265+0000 7fdd5a676740 1 -- 172.21.15.12:0/3657520660 shutdown_connections 2024-09-11T00:18:20.271 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:20.265+0000 7fdd5a676740 1 -- 172.21.15.12:0/3657520660 wait complete. 2024-09-11T00:18:20.271 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:20.266+0000 7fdd5a676740 20 client.26771 trim_cache size 0 max 16384 2024-09-11T00:18:20.286 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:20.284+0000 7fdd5a676740 1 Processor -- start 2024-09-11T00:18:20.286 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:20.284+0000 7fdd5a676740 1 -- start start 2024-09-11T00:18:20.287 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:20.284+0000 7fdd5a676740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fdda03b180 0x55fdda0538e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:18:20.287 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:20.284+0000 7fdd5a676740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fdda03baa0 0x55fdda044090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:18:20.288 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:20.284+0000 7fdd5a676740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fdda047a30 0x55fdda048e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:18:20.288 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:20.284+0000 7fdd5a676740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fdd501b1d20 con 0x55fdda03baa0 2024-09-11T00:18:20.288 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:20.284+0000 7fdd5a676740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fdd501b38f0 con 0x55fdda047a30 2024-09-11T00:18:20.288 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:20.284+0000 7fdd5a676740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fdd501b2420 con 0x55fdda03b180 2024-09-11T00:18:20.288 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:20.285+0000 7fdd4fe00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fdda03baa0 0x55fdda044090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:18:20.288 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:20.285+0000 7fdd4fe00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fdda03baa0 0x55fdda044090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:41810/0 (socket says 172.21.15.12:41810) 2024-09-11T00:18:20.288 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:20.285+0000 7fdd4fe00640 1 -- 172.21.15.12:0/1003894092 learned_addr learned my addr 172.21.15.12:0/1003894092 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:18:20.288 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:20.285+0000 7fdd55000640 1 --2- 172.21.15.12:0/1003894092 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fdda03b180 0x55fdda0538e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:18:20.288 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:20.285+0000 7fdd55a00640 1 --2- 172.21.15.12:0/1003894092 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fdda047a30 0x55fdda048e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:18:20.288 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:20.285+0000 7fdd4fe00640 1 -- 172.21.15.12:0/1003894092 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fdda03b180 msgr2=0x55fdda0538e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:18:20.288 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:20.285+0000 7fdd4fe00640 1 --2- 172.21.15.12:0/1003894092 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fdda03b180 0x55fdda0538e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:18:20.288 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:20.285+0000 7fdd4fe00640 1 -- 172.21.15.12:0/1003894092 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fdda047a30 msgr2=0x55fdda048e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:18:20.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:20.285+0000 7fdd4fe00640 1 --2- 172.21.15.12:0/1003894092 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fdda047a30 0x55fdda048e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:18:20.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:20.285+0000 7fdd4fe00640 1 -- 172.21.15.12:0/1003894092 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fdda0445d0 con 0x55fdda03baa0 2024-09-11T00:18:20.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:20.286+0000 7fdd4fe00640 1 --2- 172.21.15.12:0/1003894092 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fdda03baa0 0x55fdda044090 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7fdd3c0c8f60 tx=0x7fdd3c02a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9da826c19e72e757 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:18:20.290 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:20.286+0000 7fdd45600640 1 -- 172.21.15.12:0/1003894092 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd3c049040 con 0x55fdda03baa0 2024-09-11T00:18:20.290 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:20.286+0000 7fdd45600640 1 -- 172.21.15.12:0/1003894092 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdd3c106050 con 0x55fdda03baa0 2024-09-11T00:18:20.290 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:20.286+0000 7fdd45600640 1 -- 172.21.15.12:0/1003894092 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd3c0cc4a0 con 0x55fdda03baa0 2024-09-11T00:18:20.290 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:20.286+0000 7fdd5a676740 1 -- 172.21.15.12:0/1003894092 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fdda03baa0 msgr2=0x55fdda044090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:18:20.290 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:20.286+0000 7fdd5a676740 1 --2- 172.21.15.12:0/1003894092 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fdda03baa0 0x55fdda044090 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7fdd3c0c8f60 tx=0x7fdd3c02a410 comp rx=0 tx=0).stop 2024-09-11T00:18:20.290 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:20.287+0000 7fdd5a676740 1 -- 172.21.15.12:0/1003894092 shutdown_connections 2024-09-11T00:18:20.290 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:20.287+0000 7fdd5a676740 1 --2- 172.21.15.12:0/1003894092 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fdda047a30 0x55fdda048e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:18:20.290 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:20.287+0000 7fdd5a676740 1 --2- 172.21.15.12:0/1003894092 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fdda03baa0 0x55fdda044090 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:18:20.290 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:20.287+0000 7fdd5a676740 1 --2- 172.21.15.12:0/1003894092 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fdda03b180 0x55fdda0538e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:18:20.290 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:20.287+0000 7fdd5a676740 1 -- 172.21.15.12:0/1003894092 >> 172.21.15.12:0/1003894092 conn(0x55fdda038fb0 msgr2=0x55fdda046650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:18:20.290 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:20.287+0000 7fdd5a676740 1 -- 172.21.15.12:0/1003894092 shutdown_connections 2024-09-11T00:18:20.290 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:20.287+0000 7fdd5a676740 1 -- 172.21.15.12:0/1003894092 wait complete. 2024-09-11T00:18:20.291 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:20.287+0000 7fdd5a676740 1 -- start start 2024-09-11T00:18:20.291 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:20.287+0000 7fdd5a676740 1 Processor -- start 2024-09-11T00:18:20.292 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:20.288+0000 7fdd5a676740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fdda03b180 0x55fdda04fb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:18:20.292 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:20.288+0000 7fdd5a676740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fdda03baa0 0x55fdda050070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:18:20.293 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:20.288+0000 7fdd5a676740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fdda047a30 0x55fdda04a900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:18:20.293 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:20.288+0000 7fdd5a676740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fdd501b3220 con 0x55fdda047a30 2024-09-11T00:18:20.293 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:20.288+0000 7fdd5a676740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fdd501b2af0 con 0x55fdda03b180 2024-09-11T00:18:20.293 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:20.288+0000 7fdd5a676740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fdd501afd00 con 0x55fdda03baa0 2024-09-11T00:18:20.293 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:20.288+0000 7fdd5a676740 10 client.0 _ll_register_callbacks cb 0x7fdd4001c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:18:20.293 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:20.288+0000 7fdd55000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fdda03b180 0x55fdda04fb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:18:20.293 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:20.288+0000 7fdd55000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fdda03b180 0x55fdda04fb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.12:56826/0 (socket says 172.21.15.12:56826) 2024-09-11T00:18:20.293 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:20.288+0000 7fdd55000640 1 -- 172.21.15.12:0/74952404 learned_addr learned my addr 172.21.15.12:0/74952404 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:18:20.293 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:20.288+0000 7fdd55a00640 1 --2- 172.21.15.12:0/74952404 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fdda047a30 0x55fdda04a900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:18:20.293 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:20.288+0000 7fdd4fe00640 1 --2- 172.21.15.12:0/74952404 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fdda03baa0 0x55fdda050070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:18:20.293 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:20.289+0000 7fdd55000640 1 -- 172.21.15.12:0/74952404 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fdda03baa0 msgr2=0x55fdda050070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:18:20.293 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:20.289+0000 7fdd55000640 1 --2- 172.21.15.12:0/74952404 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fdda03baa0 0x55fdda050070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:18:20.294 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:20.289+0000 7fdd55000640 1 -- 172.21.15.12:0/74952404 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fdda047a30 msgr2=0x55fdda04a900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:18:20.294 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:20.289+0000 7fdd55000640 1 --2- 172.21.15.12:0/74952404 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fdda047a30 0x55fdda04a900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:18:20.294 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:20.289+0000 7fdd55000640 1 -- 172.21.15.12:0/74952404 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fdda04afa0 con 0x55fdda03b180 2024-09-11T00:18:20.294 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:20.289+0000 7fdd55000640 1 --2- 172.21.15.12:0/74952404 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fdda03b180 0x55fdda04fb30 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fdd40009250 tx=0x7fdd4000fb10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7585adfc60ae4148 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:18:20.294 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:20.289+0000 7fdd3aa00640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-09-11T00:18:20.294 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:20.289+0000 7fdd3aa00640 1 -- 172.21.15.12:0/74952404 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd4001e040 con 0x55fdda03b180 2024-09-11T00:18:20.294 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:20.289+0000 7fdd5a676740 1 -- 172.21.15.12:0/74952404 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdd3c106050 con 0x55fdda03b180 2024-09-11T00:18:20.294 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:20.289+0000 7fdd3aa00640 1 -- 172.21.15.12:0/74952404 <== mon.2 v2:172.21.15.165:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9208+0+0 (secure 0 0 0) 0x7fdd40005030 con 0x55fdda03b180 2024-09-11T00:18:20.294 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:20.289+0000 7fdd3aa00640 1 -- 172.21.15.12:0/74952404 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7fdd40005c50 con 0x55fdda03b180 2024-09-11T00:18:20.294 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:20.289+0000 7fdd5a676740 10 client.26777 fetch_fsmap learned FSMap version 44 2024-09-11T00:18:20.294 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:20.289+0000 7fdd5a676740 1 -- 172.21.15.12:0/74952404 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x55fdda0c4c10 con 0x55fdda03b180 2024-09-11T00:18:20.294 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:20.290+0000 7fdd3aa00640 1 -- 172.21.15.12:0/74952404 <== mon.2 v2:172.21.15.165:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdd3c106050 con 0x55fdda03b180 2024-09-11T00:18:20.294 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:20.290+0000 7fdd3aa00640 1 -- 172.21.15.12:0/74952404 <== mon.2 v2:172.21.15.165:3300/0 5 ==== fsmap(e 44) ==== 8382+0+0 (secure 0 0 0) 0x7fdd4002a070 con 0x55fdda03b180 2024-09-11T00:18:20.295 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:20.290+0000 7fdd5a676740 10 client.26777 fetch_fsmap finished waiting for FSMap version 44 2024-09-11T00:18:20.295 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:20.290+0000 7fdd5a676740 10 client.26777 resolve_mds: validated gid 24674 aka daemon mds.b 2024-09-11T00:18:20.295 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:20.290+0000 7fdd5a676740 20 client.26777 populate_metadata read hostname 'smithi012' 2024-09-11T00:18:20.295 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:20.290+0000 7fdd5a676740 1 --2- 172.21.15.12:0/74952404 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55fdda0c5370 0x55fdda0c7790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:18:20.295 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:20.290+0000 7fdd5a676740 4 client.26777 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 145.4, "await_for": 45.0}] 2024-09-11T00:18:20.295 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:20.290+0000 7fdd5a676740 1 -- 172.21.15.12:0/74952404 --> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 145.4, "await_for": 45.0}) -- 0x55fdd9fae6a0 con 0x55fdda0c5370 2024-09-11T00:18:20.295 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:20.290+0000 7fdd4fe00640 1 --2- 172.21.15.12:0/74952404 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55fdda0c5370 0x55fdda0c7790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:18:20.295 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:20.291+0000 7fdd4fe00640 1 --2- 172.21.15.12:0/74952404 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55fdda0c5370 0x55fdda0c7790 crc :-1 s=READY pgs=173 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4678e2c4bafd692f server_cookie=af7d6f5b6a40c426 in_seq=0 out_seq=0 2024-09-11T00:18:20.295 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:20.291+0000 7fdd3aa00640 10 client.26777 ms_handle_connect on v2:172.21.15.165:6838/501312984 2024-09-11T00:18:20.330 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:18:20 smithi178 ceph-mon[35919]: pgmap v736: 129 pgs: 129 active+clean; 100 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 282 KiB/s rd, 1.8 MiB/s wr, 183 op/s 2024-09-11T00:18:20.429 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:20.425+0000 7fdd3aa00640 1 -- 172.21.15.12:0/74952404 <== mds.0 v2:172.21.15.165:6838/501312984 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55fdd9fae6a0 con 0x55fdda0c5370 2024-09-11T00:18:20.429 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:20.425+0000 7fdd3aa00640 10 client.26777 handle_command_reply: tid=0 2024-09-11T00:18:20.429 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:20.425+0000 7fdd5a676740 1 client.26777 shutdown 2024-09-11T00:18:20.429 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:20.426+0000 7fdd5a676740 1 -- 172.21.15.12:0/74952404 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fdda03b180 msgr2=0x55fdda04fb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:18:20.429 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:20.426+0000 7fdd5a676740 1 --2- 172.21.15.12:0/74952404 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fdda03b180 0x55fdda04fb30 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fdd40009250 tx=0x7fdd4000fb10 comp rx=0 tx=0).stop 2024-09-11T00:18:20.430 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:20.427+0000 7fdd5a676740 1 -- 172.21.15.12:0/74952404 shutdown_connections 2024-09-11T00:18:20.430 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:20.427+0000 7fdd5a676740 1 --2- 172.21.15.12:0/74952404 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55fdda0c5370 0x55fdda0c7790 crc :-1 s=READY pgs=173 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:18:20.430 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:20.427+0000 7fdd5a676740 1 --2- 172.21.15.12:0/74952404 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fdda047a30 0x55fdda04a900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:18:20.430 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:20.427+0000 7fdd5a676740 1 --2- 172.21.15.12:0/74952404 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fdda03baa0 0x55fdda050070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:18:20.430 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:20.427+0000 7fdd5a676740 1 --2- 172.21.15.12:0/74952404 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fdda03b180 0x55fdda04fb30 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:18:20.430 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:20.427+0000 7fdd3aa00640 0 client.26777 ms_handle_reset on v2:172.21.15.165:6838/501312984 2024-09-11T00:18:20.430 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:20.427+0000 7fdd5a676740 1 -- 172.21.15.12:0/74952404 >> 172.21.15.12:0/74952404 conn(0x55fdda038fb0 msgr2=0x55fdda0528d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:18:20.431 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:20.427+0000 7fdd5a676740 1 -- 172.21.15.12:0/74952404 shutdown_connections 2024-09-11T00:18:20.431 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:20.427+0000 7fdd5a676740 1 -- 172.21.15.12:0/74952404 wait complete. 2024-09-11T00:18:20.431 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:20.427+0000 7fdd5a676740 20 client.26777 trim_cache size 0 max 16384 2024-09-11T00:18:20.432 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-09-11T00:18:20.432 INFO:teuthology.orchestra.run.smithi012.stdout: "epoch": 44, 2024-09-11T00:18:20.432 INFO:teuthology.orchestra.run.smithi012.stdout: "leader": 24674, 2024-09-11T00:18:20.432 INFO:teuthology.orchestra.run.smithi012.stdout: "set_version": 34, 2024-09-11T00:18:20.432 INFO:teuthology.orchestra.run.smithi012.stdout: "sets": { 2024-09-11T00:18:20.432 INFO:teuthology.orchestra.run.smithi012.stdout: "d9d39e24": { 2024-09-11T00:18:20.432 INFO:teuthology.orchestra.run.smithi012.stdout: "version": 34, 2024-09-11T00:18:20.432 INFO:teuthology.orchestra.run.smithi012.stdout: "age_ref": 0.0, 2024-09-11T00:18:20.433 INFO:teuthology.orchestra.run.smithi012.stdout: "state": { 2024-09-11T00:18:20.433 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "QUIESCED", 2024-09-11T00:18:20.433 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 0.0 2024-09-11T00:18:20.433 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:18:20.433 INFO:teuthology.orchestra.run.smithi012.stdout: "timeout": 90.0, 2024-09-11T00:18:20.433 INFO:teuthology.orchestra.run.smithi012.stdout: "expiration": 145.4, 2024-09-11T00:18:20.433 INFO:teuthology.orchestra.run.smithi012.stdout: "members": { 2024-09-11T00:18:20.433 INFO:teuthology.orchestra.run.smithi012.stdout: "file:/": { 2024-09-11T00:18:20.433 INFO:teuthology.orchestra.run.smithi012.stdout: "excluded": false, 2024-09-11T00:18:20.433 INFO:teuthology.orchestra.run.smithi012.stdout: "state": { 2024-09-11T00:18:20.433 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "QUIESCED", 2024-09-11T00:18:20.433 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 0.0 2024-09-11T00:18:20.433 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:18:20.434 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:18:20.434 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:18:20.434 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:18:20.434 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:18:20.434 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-09-11T00:18:20.434 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:20.430+0000 7fdd57a00640 1 -- 172.21.15.12:0/998430815 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fdd340b52e0 msgr2=0x7fdd340b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:18:20.435 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:20.430+0000 7fdd57a00640 1 --2- 172.21.15.12:0/998430815 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fdd340b52e0 0x7fdd340b77a0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fdd480049d0 tx=0x7fdd48009500 comp rx=0 tx=0).stop 2024-09-11T00:18:20.435 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:20.430+0000 7fdd57a00640 1 -- 172.21.15.12:0/998430815 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdd5010a9b0 msgr2=0x7fdd5019b9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:18:20.435 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:20.430+0000 7fdd57a00640 1 --2- 172.21.15.12:0/998430815 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdd5010a9b0 0x7fdd5019b9b0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fdd3c02a930 tx=0x7fdd3c031a20 comp rx=0 tx=0).stop 2024-09-11T00:18:20.435 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:20.430+0000 7fdd57a00640 1 -- 172.21.15.12:0/998430815 shutdown_connections 2024-09-11T00:18:20.435 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:20.430+0000 7fdd57a00640 1 --2- 172.21.15.12:0/998430815 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fdd340b52e0 0x7fdd340b77a0 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:18:20.435 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:20.430+0000 7fdd57a00640 1 --2- 172.21.15.12:0/998430815 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdd5010b330 0x7fdd501a0d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:18:20.435 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:20.430+0000 7fdd57a00640 1 --2- 172.21.15.12:0/998430815 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdd5010a9b0 0x7fdd5019b9b0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:18:20.435 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:20.430+0000 7fdd57a00640 1 --2- 172.21.15.12:0/998430815 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fdd50104fa0 0x7fdd5019b470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:18:20.435 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:20.430+0000 7fdd57a00640 1 -- 172.21.15.12:0/998430815 >> 172.21.15.12:0/998430815 conn(0x7fdd50100a50 msgr2=0x7fdd50102e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:18:20.435 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:20.431+0000 7fdd57a00640 1 -- 172.21.15.12:0/998430815 shutdown_connections 2024-09-11T00:18:20.435 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:20.432+0000 7fdd57a00640 1 -- 172.21.15.12:0/998430815 wait complete. 2024-09-11T00:18:20.532 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:18:20 smithi165 ceph-mon[35356]: pgmap v736: 129 pgs: 129 active+clean; 100 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 282 KiB/s rd, 1.8 MiB/s wr, 183 op/s 2024-09-11T00:18:20.691 INFO:tasks.workunit.client.0.smithi012.stdout: 1 479502 4.19 MB/sec execute 497 sec latency 3379.218 ms 2024-09-11T00:18:21.691 INFO:tasks.workunit.client.0.smithi012.stdout: 1 479502 4.18 MB/sec execute 498 sec latency 1278.507 ms 2024-09-11T00:18:22.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:18:22 smithi012 ceph-mon[30333]: pgmap v737: 129 pgs: 129 active+clean; 99 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 284 KiB/s rd, 2.1 MiB/s wr, 201 op/s 2024-09-11T00:18:22.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:18:22 smithi178 ceph-mon[35919]: pgmap v737: 129 pgs: 129 active+clean; 99 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 284 KiB/s rd, 2.1 MiB/s wr, 201 op/s 2024-09-11T00:18:22.429 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'd9d39e24', quiesce took 5.7 seconds. Will release after: 19.7 2024-09-11T00:18:22.532 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:18:22 smithi165 ceph-mon[35356]: pgmap v737: 129 pgs: 129 active+clean; 99 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 284 KiB/s rd, 2.1 MiB/s wr, 201 op/s 2024-09-11T00:18:22.691 INFO:tasks.workunit.client.0.smithi012.stdout: 1 479502 4.17 MB/sec execute 499 sec latency 2278.568 ms 2024-09-11T00:18:23.691 INFO:tasks.workunit.client.0.smithi012.stdout: 1 479502 4.16 MB/sec execute 500 sec latency 3278.703 ms 2024-09-11T00:18:24.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:18:24 smithi012 ceph-mon[30333]: pgmap v738: 129 pgs: 129 active+clean; 102 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 325 KiB/s rd, 2.2 MiB/s wr, 195 op/s 2024-09-11T00:18:24.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:18:24 smithi178 ceph-mon[35919]: pgmap v738: 129 pgs: 129 active+clean; 102 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 325 KiB/s rd, 2.2 MiB/s wr, 195 op/s 2024-09-11T00:18:24.532 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:18:24 smithi165 ceph-mon[35356]: pgmap v738: 129 pgs: 129 active+clean; 102 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 325 KiB/s rd, 2.2 MiB/s wr, 195 op/s 2024-09-11T00:18:24.691 INFO:tasks.workunit.client.0.smithi012.stdout: 1 479502 4.15 MB/sec execute 501 sec latency 4278.770 ms 2024-09-11T00:18:25.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:18:25 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T00:18:25.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:18:25 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:18:25.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:18:25 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T00:18:25.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:18:25 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:18:25.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:18:25 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T00:18:25.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:18:25 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:18:25.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:18:25 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T00:18:25.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:18:25 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:18:25.532 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:18:25 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T00:18:25.532 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:18:25 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:18:25.533 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:18:25 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T00:18:25.533 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:18:25 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:18:25.691 INFO:tasks.workunit.client.0.smithi012.stdout: 1 479502 4.15 MB/sec execute 502 sec latency 5278.817 ms 2024-09-11T00:18:26.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:18:26 smithi012 ceph-mon[30333]: pgmap v739: 129 pgs: 129 active+clean; 103 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 2.1 MiB/s wr, 125 op/s 2024-09-11T00:18:26.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:18:26 smithi178 ceph-mon[35919]: pgmap v739: 129 pgs: 129 active+clean; 103 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 2.1 MiB/s wr, 125 op/s 2024-09-11T00:18:26.513 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:26.510+0000 7f93b5a00640 1 -- 172.21.15.12:0/861378921 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f93b0104b90 msgr2=0x7f93b0104f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:18:26.513 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:26.510+0000 7f93b5a00640 1 --2- 172.21.15.12:0/861378921 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f93b0104b90 0x7f93b0104f90 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f93a0009930 tx=0x7f93a002a440 comp rx=0 tx=0).stop 2024-09-11T00:18:26.513 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:26.510+0000 7f93b5a00640 1 -- 172.21.15.12:0/861378921 shutdown_connections 2024-09-11T00:18:26.513 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:26.510+0000 7f93b5a00640 1 --2- 172.21.15.12:0/861378921 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f93b01066a0 0x7f93b010d020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:18:26.514 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:26.511+0000 7f93b5a00640 1 --2- 172.21.15.12:0/861378921 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f93b0105ce0 0x7f93b0106160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:18:26.514 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:26.511+0000 7f93b5a00640 1 --2- 172.21.15.12:0/861378921 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f93b0104b90 0x7f93b0104f90 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:18:26.514 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:26.511+0000 7f93b5a00640 1 -- 172.21.15.12:0/861378921 >> 172.21.15.12:0/861378921 conn(0x7f93b0100470 msgr2=0x7f93b0102890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:18:26.514 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:26.511+0000 7f93b5a00640 1 -- 172.21.15.12:0/861378921 shutdown_connections 2024-09-11T00:18:26.514 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:26.511+0000 7f93b5a00640 1 -- 172.21.15.12:0/861378921 wait complete. 2024-09-11T00:18:26.516 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:26.511+0000 7f93b5a00640 1 Processor -- start 2024-09-11T00:18:26.516 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:26.512+0000 7f93b5a00640 1 -- start start 2024-09-11T00:18:26.516 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:26.512+0000 7f93b5a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f93b0104b90 0x7f93b019b050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:18:26.516 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:26.512+0000 7f93b5a00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f93b0105ce0 0x7f93b019b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:18:26.516 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:26.512+0000 7f93b5a00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f93b01066a0 0x7f93b019c2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:18:26.516 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:26.512+0000 7f93b5a00640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f93b01108d0 con 0x7f93b01066a0 2024-09-11T00:18:26.516 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:26.512+0000 7f93b5a00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f93b0110750 con 0x7f93b0104b90 2024-09-11T00:18:26.516 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:26.512+0000 7f93b5a00640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f93b0110a50 con 0x7f93b0105ce0 2024-09-11T00:18:26.516 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:26.512+0000 7f93aea00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f93b0104b90 0x7f93b019b050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:18:26.516 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:26.512+0000 7f93aea00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f93b0104b90 0x7f93b019b050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.12:56830/0 (socket says 172.21.15.12:56830) 2024-09-11T00:18:26.516 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:26.512+0000 7f93aea00640 1 -- 172.21.15.12:0/1766426709 learned_addr learned my addr 172.21.15.12:0/1766426709 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:18:26.517 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:26.512+0000 7f93ae000640 1 --2- 172.21.15.12:0/1766426709 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f93b0105ce0 0x7f93b019b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:18:26.517 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:26.512+0000 7f93af400640 1 --2- 172.21.15.12:0/1766426709 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f93b01066a0 0x7f93b019c2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:18:26.517 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:26.513+0000 7f93aea00640 1 -- 172.21.15.12:0/1766426709 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f93b0105ce0 msgr2=0x7f93b019b6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:18:26.517 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:26.513+0000 7f93aea00640 1 --2- 172.21.15.12:0/1766426709 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f93b0105ce0 0x7f93b019b6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:18:26.517 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:26.513+0000 7f93aea00640 1 -- 172.21.15.12:0/1766426709 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f93b01066a0 msgr2=0x7f93b019c2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:18:26.517 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:26.513+0000 7f93aea00640 1 --2- 172.21.15.12:0/1766426709 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f93b01066a0 0x7f93b019c2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:18:26.517 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:26.513+0000 7f93aea00640 1 -- 172.21.15.12:0/1766426709 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93b019cb90 con 0x7f93b0104b90 2024-09-11T00:18:26.517 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:26.513+0000 7f93ae000640 1 --2- 172.21.15.12:0/1766426709 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f93b0105ce0 0x7f93b019b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:18:26.517 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:26.513+0000 7f93aea00640 1 --2- 172.21.15.12:0/1766426709 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f93b0104b90 0x7f93b019b050 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f93a0004770 tx=0x7f93a0003c30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:18:26.518 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:26.513+0000 7f93a7400640 1 -- 172.21.15.12:0/1766426709 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93a003f070 con 0x7f93b0104b90 2024-09-11T00:18:26.518 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:26.513+0000 7f93b5a00640 1 -- 172.21.15.12:0/1766426709 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f93b019ce20 con 0x7f93b0104b90 2024-09-11T00:18:26.518 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:26.513+0000 7f93a7400640 1 -- 172.21.15.12:0/1766426709 <== mon.2 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f93a0031c50 con 0x7f93b0104b90 2024-09-11T00:18:26.518 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:26.513+0000 7f93a7400640 1 -- 172.21.15.12:0/1766426709 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93a0035030 con 0x7f93b0104b90 2024-09-11T00:18:26.519 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:26.513+0000 7f93b5a00640 1 -- 172.21.15.12:0/1766426709 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f93b01a3550 con 0x7f93b0104b90 2024-09-11T00:18:26.519 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:26.516+0000 7f93b89f7740 1 Processor -- start 2024-09-11T00:18:26.519 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:26.516+0000 7f93b89f7740 1 -- start start 2024-09-11T00:18:26.520 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:26.517+0000 7f93b89f7740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5578a9ef9790 0x5578a9efbb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:18:26.520 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:26.517+0000 7f93b89f7740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5578a9efc0c0 0x5578a9efe4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:18:26.520 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:26.517+0000 7f93b89f7740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5578a9efeb00 0x5578a9f00fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:18:26.521 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:26.517+0000 7f93b89f7740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x5578a9e6c6b0 con 0x5578a9efc0c0 2024-09-11T00:18:26.521 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:26.517+0000 7f93b89f7740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x5578a9e5bae0 con 0x5578a9ef9790 2024-09-11T00:18:26.521 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:26.517+0000 7f93b89f7740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5578a9e5f8f0 con 0x5578a9efeb00 2024-09-11T00:18:26.521 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:26.517+0000 7f93af400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5578a9ef9790 0x5578a9efbb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:18:26.521 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:26.517+0000 7f93af400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5578a9ef9790 0x5578a9efbb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.12:56838/0 (socket says 172.21.15.12:56838) 2024-09-11T00:18:26.521 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:26.517+0000 7f93af400640 1 -- 172.21.15.12:0/154486734 learned_addr learned my addr 172.21.15.12:0/154486734 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:18:26.521 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:26.517+0000 7f93ae000640 1 --2- 172.21.15.12:0/154486734 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5578a9efeb00 0x5578a9f00fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:18:26.521 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:26.517+0000 7f93a7400640 1 -- 172.21.15.12:0/1766426709 <== mon.2 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7f93a003b580 con 0x7f93b0104b90 2024-09-11T00:18:26.521 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:26.517+0000 7f93af400640 1 -- 172.21.15.12:0/154486734 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5578a9efeb00 msgr2=0x5578a9f00fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:18:26.521 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:26.517+0000 7f93af400640 1 --2- 172.21.15.12:0/154486734 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5578a9efeb00 0x5578a9f00fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:18:26.521 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:26.517+0000 7f93af400640 1 -- 172.21.15.12:0/154486734 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5578a9efc0c0 msgr2=0x5578a9efe4b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-11T00:18:26.521 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:26.518+0000 7f93af400640 1 --2- 172.21.15.12:0/154486734 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5578a9efc0c0 0x5578a9efe4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:18:26.521 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:26.518+0000 7f93af400640 1 -- 172.21.15.12:0/154486734 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5578a9e28780 con 0x5578a9ef9790 2024-09-11T00:18:26.522 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:26.518+0000 7f93af400640 1 --2- 172.21.15.12:0/154486734 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5578a9ef9790 0x5578a9efbb80 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f93a800a4b0 tx=0x7f93a8009650 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3520d93c2aa93923 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:18:26.522 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:26.518+0000 7f939be00640 1 -- 172.21.15.12:0/154486734 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93a8013790 con 0x5578a9ef9790 2024-09-11T00:18:26.522 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:26.518+0000 7f939be00640 1 -- 172.21.15.12:0/154486734 <== mon.2 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f93a8019070 con 0x5578a9ef9790 2024-09-11T00:18:26.522 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:26.518+0000 7f939be00640 1 -- 172.21.15.12:0/154486734 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93a8014cf0 con 0x5578a9ef9790 2024-09-11T00:18:26.526 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:26.519+0000 7f93b89f7740 1 -- 172.21.15.12:0/154486734 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5578a9ef9790 msgr2=0x5578a9efbb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:18:26.526 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:26.519+0000 7f93b89f7740 1 --2- 172.21.15.12:0/154486734 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5578a9ef9790 0x5578a9efbb80 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f93a800a4b0 tx=0x7f93a8009650 comp rx=0 tx=0).stop 2024-09-11T00:18:26.526 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:26.519+0000 7f93b89f7740 1 -- 172.21.15.12:0/154486734 shutdown_connections 2024-09-11T00:18:26.526 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:26.519+0000 7f93b89f7740 1 --2- 172.21.15.12:0/154486734 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5578a9efeb00 0x5578a9f00fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:18:26.526 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:26.519+0000 7f93b89f7740 1 --2- 172.21.15.12:0/154486734 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5578a9efc0c0 0x5578a9efe4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:18:26.526 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:26.519+0000 7f93b89f7740 1 --2- 172.21.15.12:0/154486734 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5578a9ef9790 0x5578a9efbb80 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:18:26.526 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:26.519+0000 7f93b89f7740 1 -- 172.21.15.12:0/154486734 >> 172.21.15.12:0/154486734 conn(0x5578a9ef48e0 msgr2=0x5578a9ef5670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:18:26.526 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:26.519+0000 7f93b89f7740 1 -- 172.21.15.12:0/154486734 shutdown_connections 2024-09-11T00:18:26.526 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:26.519+0000 7f93a7400640 1 --2- 172.21.15.12:0/1766426709 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f93940b5270 0x7f93940b7730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:18:26.526 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:26.519+0000 7f93b89f7740 1 -- 172.21.15.12:0/154486734 wait complete. 2024-09-11T00:18:26.526 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:26.519+0000 7f93a7400640 1 -- 172.21.15.12:0/1766426709 <== mon.2 v2:172.21.15.165:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9208+0+0 (secure 0 0 0) 0x7f93a0106c80 con 0x7f93b0104b90 2024-09-11T00:18:26.527 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:26.519+0000 7f93ae000640 1 --2- 172.21.15.12:0/1766426709 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f93940b5270 0x7f93940b7730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:18:26.527 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:26.519+0000 7f93b89f7740 1 -- start start 2024-09-11T00:18:26.527 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:26.520+0000 7f93b89f7740 1 Processor -- start 2024-09-11T00:18:26.527 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:26.520+0000 7f93ae000640 1 --2- 172.21.15.12:0/1766426709 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f93940b5270 0x7f93940b7730 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f939c005d00 tx=0x7f939c00b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:18:26.527 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:26.520+0000 7f93b89f7740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5578a9ef9790 0x5578a9f02f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:18:26.527 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:26.520+0000 7f93b89f7740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5578a9efc0c0 0x5578a9f035c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:18:26.527 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:26.520+0000 7f93b89f7740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5578a9efeb00 0x5578a9f04190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:18:26.527 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:26.520+0000 7f93b89f7740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x5578a9d93ab0 con 0x5578a9efc0c0 2024-09-11T00:18:26.527 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:26.520+0000 7f93b89f7740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x5578a9e02570 con 0x5578a9efeb00 2024-09-11T00:18:26.527 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:26.520+0000 7f93b89f7740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5578a9f01ae0 con 0x5578a9ef9790 2024-09-11T00:18:26.527 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:26.520+0000 7f93b89f7740 10 client.0 _ll_register_callbacks cb 0x5578a9e11100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:18:26.527 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:26.520+0000 7f93ae000640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5578a9efc0c0 0x5578a9f035c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:18:26.527 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:26.520+0000 7f93ae000640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5578a9efc0c0 0x5578a9f035c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:41844/0 (socket says 172.21.15.12:41844) 2024-09-11T00:18:26.528 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:26.520+0000 7f93ae000640 1 -- 172.21.15.12:0/3523809752 learned_addr learned my addr 172.21.15.12:0/3523809752 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:18:26.528 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:26.520+0000 7f93aea00640 1 --2- 172.21.15.12:0/3523809752 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5578a9ef9790 0x5578a9f02f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:18:26.528 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:26.521+0000 7f93af400640 1 --2- 172.21.15.12:0/3523809752 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5578a9efeb00 0x5578a9f04190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:18:26.528 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:26.521+0000 7f93ae000640 1 -- 172.21.15.12:0/3523809752 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5578a9ef9790 msgr2=0x5578a9f02f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:18:26.528 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:26.521+0000 7f93ae000640 1 --2- 172.21.15.12:0/3523809752 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5578a9ef9790 0x5578a9f02f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:18:26.528 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:26.521+0000 7f93ae000640 1 -- 172.21.15.12:0/3523809752 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5578a9efeb00 msgr2=0x5578a9f04190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:18:26.528 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:26.521+0000 7f93ae000640 1 --2- 172.21.15.12:0/3523809752 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5578a9efeb00 0x5578a9f04190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:18:26.528 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:26.521+0000 7f93ae000640 1 -- 172.21.15.12:0/3523809752 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5578a9bbd790 con 0x5578a9efc0c0 2024-09-11T00:18:26.528 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:26.521+0000 7f93ae000640 1 --2- 172.21.15.12:0/3523809752 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5578a9efc0c0 0x5578a9f035c0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f939c010830 tx=0x7f939c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=145a09bef503e4bd server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:18:26.528 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:26.521+0000 7f9398c00640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-09-11T00:18:26.528 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:26.521+0000 7f9398c00640 1 -- 172.21.15.12:0/3523809752 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f939c02c060 con 0x5578a9efc0c0 2024-09-11T00:18:26.528 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:26.521+0000 7f93b89f7740 1 -- 172.21.15.12:0/3523809752 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f93a8019070 con 0x5578a9efc0c0 2024-09-11T00:18:26.529 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:26.521+0000 7f9398c00640 1 -- 172.21.15.12:0/3523809752 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9208+0+0 (secure 0 0 0) 0x7f939c017030 con 0x5578a9efc0c0 2024-09-11T00:18:26.529 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:26.522+0000 7f9398c00640 1 -- 172.21.15.12:0/3523809752 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f939c033050 con 0x5578a9efc0c0 2024-09-11T00:18:26.529 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:26.522+0000 7f93b89f7740 10 client.16785 fetch_fsmap learned FSMap version 44 2024-09-11T00:18:26.529 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:26.522+0000 7f93b89f7740 1 -- 172.21.15.12:0/3523809752 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x5578a9c36620 con 0x5578a9efc0c0 2024-09-11T00:18:26.529 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:26.522+0000 7f9398c00640 1 -- 172.21.15.12:0/3523809752 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f93a8019070 con 0x5578a9efc0c0 2024-09-11T00:18:26.529 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:26.522+0000 7f9398c00640 1 -- 172.21.15.12:0/3523809752 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 44) ==== 8382+0+0 (secure 0 0 0) 0x7f939c01cb90 con 0x5578a9efc0c0 2024-09-11T00:18:26.529 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:26.522+0000 7f93b89f7740 10 client.16785 fetch_fsmap finished waiting for FSMap version 44 2024-09-11T00:18:26.529 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:26.522+0000 7f93b89f7740 10 client.16785 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T00:18:26.529 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:26.522+0000 7f93b89f7740 20 client.16785 populate_metadata read hostname 'smithi012' 2024-09-11T00:18:26.529 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:26.522+0000 7f93b89f7740 1 --2- 172.21.15.12:0/3523809752 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x5578a9f0e950 0x5578a9f10d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:18:26.529 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:26.522+0000 7f93b89f7740 4 client.16785 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T00:18:26.529 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:26.522+0000 7f93b89f7740 1 -- 172.21.15.12:0/3523809752 --> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5578a9e6e600 con 0x5578a9f0e950 2024-09-11T00:18:26.529 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:26.523+0000 7f93aea00640 1 --2- 172.21.15.12:0/3523809752 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x5578a9f0e950 0x5578a9f10d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:18:26.530 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:26.523+0000 7f93aea00640 1 --2- 172.21.15.12:0/3523809752 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x5578a9f0e950 0x5578a9f10d90 crc :-1 s=READY pgs=174 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dc4cabd66a1be1f7 server_cookie=3a05f7c9d386bd35 in_seq=0 out_seq=0 2024-09-11T00:18:26.530 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:26.523+0000 7f9398c00640 10 client.16785 ms_handle_connect on v2:172.21.15.165:6838/501312984 2024-09-11T00:18:26.531 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:26.524+0000 7f9398c00640 1 -- 172.21.15.12:0/3523809752 <== mds.0 v2:172.21.15.165:6838/501312984 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5578a9e6e600 con 0x5578a9f0e950 2024-09-11T00:18:26.531 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:26.524+0000 7f9398c00640 10 client.16785 handle_command_reply: tid=0 2024-09-11T00:18:26.531 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:26.524+0000 7f93b89f7740 1 client.16785 shutdown 2024-09-11T00:18:26.531 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:26.524+0000 7f93b89f7740 1 -- 172.21.15.12:0/3523809752 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5578a9efc0c0 msgr2=0x5578a9f035c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:18:26.531 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:26.524+0000 7f93b89f7740 1 --2- 172.21.15.12:0/3523809752 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5578a9efc0c0 0x5578a9f035c0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f939c010830 tx=0x7f939c004730 comp rx=0 tx=0).stop 2024-09-11T00:18:26.531 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:26.524+0000 7f93b89f7740 1 -- 172.21.15.12:0/3523809752 shutdown_connections 2024-09-11T00:18:26.531 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:26.524+0000 7f93b89f7740 1 --2- 172.21.15.12:0/3523809752 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x5578a9f0e950 0x5578a9f10d90 crc :-1 s=READY pgs=174 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:18:26.531 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:26.524+0000 7f93b89f7740 1 --2- 172.21.15.12:0/3523809752 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5578a9efeb00 0x5578a9f04190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:18:26.531 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:26.524+0000 7f93b89f7740 1 --2- 172.21.15.12:0/3523809752 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5578a9efc0c0 0x5578a9f035c0 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:18:26.531 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:26.524+0000 7f9398c00640 0 client.16785 ms_handle_reset on v2:172.21.15.165:6838/501312984 2024-09-11T00:18:26.531 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:26.524+0000 7f93b89f7740 1 --2- 172.21.15.12:0/3523809752 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5578a9ef9790 0x5578a9f02f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:18:26.532 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:26.524+0000 7f93b89f7740 1 -- 172.21.15.12:0/3523809752 >> 172.21.15.12:0/3523809752 conn(0x5578a9ef48e0 msgr2=0x5578a9ef3a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:18:26.532 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:26.525+0000 7f93b89f7740 1 -- 172.21.15.12:0/3523809752 shutdown_connections 2024-09-11T00:18:26.532 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:26.525+0000 7f93b89f7740 1 -- 172.21.15.12:0/3523809752 wait complete. 2024-09-11T00:18:26.532 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:26.525+0000 7f93b89f7740 20 client.16785 trim_cache size 0 max 16384 2024-09-11T00:18:26.532 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:18:26 smithi165 ceph-mon[35356]: pgmap v739: 129 pgs: 129 active+clean; 103 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 2.1 MiB/s wr, 125 op/s 2024-09-11T00:18:26.544 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:26.541+0000 7f93b89f7740 1 Processor -- start 2024-09-11T00:18:26.544 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:26.541+0000 7f93b89f7740 1 -- start start 2024-09-11T00:18:26.544 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:26.542+0000 7f93b89f7740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5578a9ef9790 0x5578a9eef960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:18:26.544 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:26.542+0000 7f93b89f7740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5578a9efc0c0 0x5578a9f0b1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:18:26.544 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:26.542+0000 7f93b89f7740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5578a9efeb00 0x5578a9efeee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:18:26.544 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:26.542+0000 7f93b89f7740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f93b01ad8c0 con 0x5578a9efc0c0 2024-09-11T00:18:26.544 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:26.542+0000 7f93b89f7740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f93b01af490 con 0x5578a9efeb00 2024-09-11T00:18:26.544 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:26.542+0000 7f93b89f7740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f93b01adfc0 con 0x5578a9ef9790 2024-09-11T00:18:26.545 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:26.542+0000 7f93ae000640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5578a9efc0c0 0x5578a9f0b1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:18:26.545 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:26.542+0000 7f93ae000640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5578a9efc0c0 0x5578a9f0b1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:41850/0 (socket says 172.21.15.12:41850) 2024-09-11T00:18:26.545 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:26.542+0000 7f93ae000640 1 -- 172.21.15.12:0/3178980611 learned_addr learned my addr 172.21.15.12:0/3178980611 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:18:26.546 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:26.542+0000 7f93af400640 1 --2- 172.21.15.12:0/3178980611 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5578a9efeb00 0x5578a9efeee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:18:26.546 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:26.542+0000 7f93aea00640 1 --2- 172.21.15.12:0/3178980611 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5578a9ef9790 0x5578a9eef960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:18:26.546 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:26.542+0000 7f93ae000640 1 -- 172.21.15.12:0/3178980611 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5578a9ef9790 msgr2=0x5578a9eef960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:18:26.546 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:26.542+0000 7f93ae000640 1 --2- 172.21.15.12:0/3178980611 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5578a9ef9790 0x5578a9eef960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:18:26.546 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:26.542+0000 7f93ae000640 1 -- 172.21.15.12:0/3178980611 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5578a9efeb00 msgr2=0x5578a9efeee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:18:26.546 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:26.542+0000 7f93ae000640 1 --2- 172.21.15.12:0/3178980611 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5578a9efeb00 0x5578a9efeee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:18:26.546 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:26.542+0000 7f93ae000640 1 -- 172.21.15.12:0/3178980611 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5578a9efe820 con 0x5578a9efc0c0 2024-09-11T00:18:26.546 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:26.543+0000 7f93ae000640 1 --2- 172.21.15.12:0/3178980611 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5578a9efc0c0 0x5578a9f0b1d0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f939c000c00 tx=0x7f939c011440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36cbd0b8a2b97a56 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:18:26.546 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:26.543+0000 7f939b400640 1 -- 172.21.15.12:0/3178980611 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f939c01fac0 con 0x5578a9efc0c0 2024-09-11T00:18:26.546 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:26.543+0000 7f939b400640 1 -- 172.21.15.12:0/3178980611 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f939c03c070 con 0x5578a9efc0c0 2024-09-11T00:18:26.546 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:26.543+0000 7f939b400640 1 -- 172.21.15.12:0/3178980611 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f939c01c650 con 0x5578a9efc0c0 2024-09-11T00:18:26.548 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:26.544+0000 7f93b89f7740 1 -- 172.21.15.12:0/3178980611 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5578a9efc0c0 msgr2=0x5578a9f0b1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:18:26.549 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:26.544+0000 7f93b89f7740 1 --2- 172.21.15.12:0/3178980611 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5578a9efc0c0 0x5578a9f0b1d0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f939c000c00 tx=0x7f939c011440 comp rx=0 tx=0).stop 2024-09-11T00:18:26.549 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:26.544+0000 7f93b89f7740 1 -- 172.21.15.12:0/3178980611 shutdown_connections 2024-09-11T00:18:26.549 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:26.544+0000 7f93b89f7740 1 --2- 172.21.15.12:0/3178980611 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5578a9efeb00 0x5578a9efeee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:18:26.549 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:26.544+0000 7f93b89f7740 1 --2- 172.21.15.12:0/3178980611 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5578a9efc0c0 0x5578a9f0b1d0 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:18:26.549 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:26.544+0000 7f93b89f7740 1 --2- 172.21.15.12:0/3178980611 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5578a9ef9790 0x5578a9eef960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:18:26.549 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:26.544+0000 7f93b89f7740 1 -- 172.21.15.12:0/3178980611 >> 172.21.15.12:0/3178980611 conn(0x5578a9ef48e0 msgr2=0x5578a9f020c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:18:26.549 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:26.544+0000 7f93b89f7740 1 -- 172.21.15.12:0/3178980611 shutdown_connections 2024-09-11T00:18:26.549 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:26.544+0000 7f93b89f7740 1 -- 172.21.15.12:0/3178980611 wait complete. 2024-09-11T00:18:26.549 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:26.544+0000 7f93b89f7740 1 -- start start 2024-09-11T00:18:26.549 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:26.545+0000 7f93b89f7740 1 Processor -- start 2024-09-11T00:18:26.549 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:26.545+0000 7f93b89f7740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5578a9ef9790 0x5578a9f0a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:18:26.549 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:26.545+0000 7f93b89f7740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5578a9efc0c0 0x5578a9f0adc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:18:26.550 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:26.545+0000 7f93b89f7740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5578a9efeb00 0x5578a9f090b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:18:26.550 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:26.545+0000 7f93b89f7740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f93b01aedc0 con 0x5578a9efeb00 2024-09-11T00:18:26.550 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:26.545+0000 7f93b89f7740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f93b01ae690 con 0x5578a9ef9790 2024-09-11T00:18:26.550 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:26.545+0000 7f93b89f7740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f93b01ab8a0 con 0x5578a9efc0c0 2024-09-11T00:18:26.550 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:26.545+0000 7f93b89f7740 10 client.0 _ll_register_callbacks cb 0x7f939c00bdb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:18:26.550 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:26.545+0000 7f93af400640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5578a9efeb00 0x5578a9f090b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:18:26.550 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:26.545+0000 7f93af400640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5578a9efeb00 0x5578a9f090b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:41862/0 (socket says 172.21.15.12:41862) 2024-09-11T00:18:26.550 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:26.545+0000 7f93af400640 1 -- 172.21.15.12:0/3777756017 learned_addr learned my addr 172.21.15.12:0/3777756017 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:18:26.550 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:26.545+0000 7f93aea00640 1 --2- 172.21.15.12:0/3777756017 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5578a9ef9790 0x5578a9f0a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:18:26.550 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:26.546+0000 7f93ae000640 1 --2- 172.21.15.12:0/3777756017 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5578a9efc0c0 0x5578a9f0adc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:18:26.550 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:26.546+0000 7f93af400640 1 -- 172.21.15.12:0/3777756017 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5578a9efc0c0 msgr2=0x5578a9f0adc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:18:26.550 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:26.546+0000 7f93af400640 1 --2- 172.21.15.12:0/3777756017 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5578a9efc0c0 0x5578a9f0adc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:18:26.550 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:26.546+0000 7f93af400640 1 -- 172.21.15.12:0/3777756017 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5578a9ef9790 msgr2=0x5578a9f0a7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:18:26.551 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:26.546+0000 7f93af400640 1 --2- 172.21.15.12:0/3777756017 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5578a9ef9790 0x5578a9f0a7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:18:26.551 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:26.546+0000 7f93af400640 1 -- 172.21.15.12:0/3777756017 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5578a9efbef0 con 0x5578a9efeb00 2024-09-11T00:18:26.551 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:26.546+0000 7f93af400640 1 --2- 172.21.15.12:0/3777756017 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5578a9efeb00 0x5578a9f090b0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f93a8009f70 tx=0x7f93a8003cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6cb9db50891fe03 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:18:26.551 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:26.546+0000 7f9398c00640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-09-11T00:18:26.551 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:26.546+0000 7f9398c00640 1 -- 172.21.15.12:0/3777756017 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93a80139a0 con 0x5578a9efeb00 2024-09-11T00:18:26.551 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:26.546+0000 7f9398c00640 1 -- 172.21.15.12:0/3777756017 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9208+0+0 (secure 0 0 0) 0x7f93a8013b40 con 0x5578a9efeb00 2024-09-11T00:18:26.551 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:26.546+0000 7f93b89f7740 1 -- 172.21.15.12:0/3777756017 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f939c03c070 con 0x5578a9efeb00 2024-09-11T00:18:26.552 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:26.547+0000 7f9398c00640 1 -- 172.21.15.12:0/3777756017 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f93a8003e20 con 0x5578a9efeb00 2024-09-11T00:18:26.552 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:26.547+0000 7f93b89f7740 10 client.16791 fetch_fsmap learned FSMap version 44 2024-09-11T00:18:26.552 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:26.547+0000 7f93b89f7740 1 -- 172.21.15.12:0/3777756017 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x5578a9f04500 con 0x5578a9efeb00 2024-09-11T00:18:26.552 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:26.547+0000 7f9398c00640 1 -- 172.21.15.12:0/3777756017 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f939c03c070 con 0x5578a9efeb00 2024-09-11T00:18:26.552 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:26.547+0000 7f9398c00640 1 -- 172.21.15.12:0/3777756017 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 44) ==== 8382+0+0 (secure 0 0 0) 0x7f93a8014420 con 0x5578a9efeb00 2024-09-11T00:18:26.553 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:26.547+0000 7f93b89f7740 10 client.16791 fetch_fsmap finished waiting for FSMap version 44 2024-09-11T00:18:26.553 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:26.547+0000 7f93b89f7740 10 client.16791 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T00:18:26.553 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:26.547+0000 7f93b89f7740 20 client.16791 populate_metadata read hostname 'smithi012' 2024-09-11T00:18:26.553 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:26.547+0000 7f93b89f7740 1 --2- 172.21.15.12:0/3777756017 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x5578a9f7c790 0x5578a9f7ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:18:26.553 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:26.547+0000 7f93b89f7740 4 client.16791 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T00:18:26.553 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:26.547+0000 7f93b89f7740 1 -- 172.21.15.12:0/3777756017 --> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5578a9e6a0b0 con 0x5578a9f7c790 2024-09-11T00:18:26.553 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:26.548+0000 7f93aea00640 1 --2- 172.21.15.12:0/3777756017 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x5578a9f7c790 0x5578a9f7ebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:18:26.553 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:26.548+0000 7f93aea00640 1 --2- 172.21.15.12:0/3777756017 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x5578a9f7c790 0x5578a9f7ebb0 crc :-1 s=READY pgs=175 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6f9ed54d8b120ba4 server_cookie=5f802bcb317733e2 in_seq=0 out_seq=0 2024-09-11T00:18:26.553 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:26.548+0000 7f9398c00640 10 client.16791 ms_handle_connect on v2:172.21.15.165:6838/501312984 2024-09-11T00:18:26.581 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:26.579+0000 7f9398c00640 1 -- 172.21.15.12:0/3777756017 <== mds.0 v2:172.21.15.165:6838/501312984 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5578a9e6a0b0 con 0x5578a9f7c790 2024-09-11T00:18:26.582 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:26.579+0000 7f9398c00640 10 client.16791 handle_command_reply: tid=0 2024-09-11T00:18:26.582 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:26.579+0000 7f93b89f7740 1 client.16791 shutdown 2024-09-11T00:18:26.582 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:26.579+0000 7f93b89f7740 1 -- 172.21.15.12:0/3777756017 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5578a9efeb00 msgr2=0x5578a9f090b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:18:26.582 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:26.579+0000 7f93b89f7740 1 --2- 172.21.15.12:0/3777756017 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5578a9efeb00 0x5578a9f090b0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f93a8009f70 tx=0x7f93a8003cb0 comp rx=0 tx=0).stop 2024-09-11T00:18:26.582 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:26.579+0000 7f93b89f7740 1 -- 172.21.15.12:0/3777756017 shutdown_connections 2024-09-11T00:18:26.582 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:26.579+0000 7f93b89f7740 1 --2- 172.21.15.12:0/3777756017 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x5578a9f7c790 0x5578a9f7ebb0 crc :-1 s=READY pgs=175 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:18:26.582 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:26.579+0000 7f93b89f7740 1 --2- 172.21.15.12:0/3777756017 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5578a9efeb00 0x5578a9f090b0 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:18:26.582 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:26.579+0000 7f93b89f7740 1 --2- 172.21.15.12:0/3777756017 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5578a9efc0c0 0x5578a9f0adc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:18:26.582 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:26.579+0000 7f93b89f7740 1 --2- 172.21.15.12:0/3777756017 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5578a9ef9790 0x5578a9f0a7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:18:26.582 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:26.579+0000 7f93b89f7740 1 -- 172.21.15.12:0/3777756017 >> 172.21.15.12:0/3777756017 conn(0x5578a9ef48e0 msgr2=0x5578a9efeee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:18:26.582 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:26.579+0000 7f9398c00640 0 client.16791 ms_handle_reset on v2:172.21.15.165:6838/501312984 2024-09-11T00:18:26.583 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:26.579+0000 7f93b89f7740 1 -- 172.21.15.12:0/3777756017 shutdown_connections 2024-09-11T00:18:26.583 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:26.579+0000 7f93b89f7740 1 -- 172.21.15.12:0/3777756017 wait complete. 2024-09-11T00:18:26.583 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:26.579+0000 7f93b89f7740 20 client.16791 trim_cache size 0 max 16384 2024-09-11T00:18:26.583 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:{ 2024-09-11T00:18:26.583 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "message": "", 2024-09-11T00:18:26.583 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "return_code": 0 2024-09-11T00:18:26.583 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:} 2024-09-11T00:18:26.584 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:26.580+0000 7f93b5a00640 1 -- 172.21.15.12:0/1766426709 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f93940b5270 msgr2=0x7f93940b7730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:18:26.584 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:26.580+0000 7f93b5a00640 1 --2- 172.21.15.12:0/1766426709 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f93940b5270 0x7f93940b7730 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f939c005d00 tx=0x7f939c00b410 comp rx=0 tx=0).stop 2024-09-11T00:18:26.584 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:26.580+0000 7f93b5a00640 1 -- 172.21.15.12:0/1766426709 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f93b0104b90 msgr2=0x7f93b019b050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:18:26.584 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:26.580+0000 7f93b5a00640 1 --2- 172.21.15.12:0/1766426709 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f93b0104b90 0x7f93b019b050 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f93a0004770 tx=0x7f93a0003c30 comp rx=0 tx=0).stop 2024-09-11T00:18:26.584 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:26.580+0000 7f93b5a00640 1 -- 172.21.15.12:0/1766426709 shutdown_connections 2024-09-11T00:18:26.584 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:26.580+0000 7f93b5a00640 1 --2- 172.21.15.12:0/1766426709 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f93940b5270 0x7f93940b7730 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:18:26.585 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:26.580+0000 7f93b5a00640 1 --2- 172.21.15.12:0/1766426709 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f93b01066a0 0x7f93b019c2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:18:26.585 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:26.580+0000 7f93b5a00640 1 --2- 172.21.15.12:0/1766426709 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f93b0105ce0 0x7f93b019b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:18:26.585 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:26.580+0000 7f93b5a00640 1 --2- 172.21.15.12:0/1766426709 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f93b0104b90 0x7f93b019b050 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:18:26.585 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:26.580+0000 7f93b5a00640 1 -- 172.21.15.12:0/1766426709 >> 172.21.15.12:0/1766426709 conn(0x7f93b0100470 msgr2=0x7f93b01090d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:18:26.585 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:26.580+0000 7f93b5a00640 1 -- 172.21.15.12:0/1766426709 shutdown_connections 2024-09-11T00:18:26.585 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:26.580+0000 7f93b5a00640 1 -- 172.21.15.12:0/1766426709 wait complete. 2024-09-11T00:18:26.691 INFO:tasks.workunit.client.0.smithi012.stdout: 1 479502 4.14 MB/sec execute 503 sec latency 6278.886 ms 2024-09-11T00:18:27.691 INFO:tasks.workunit.client.0.smithi012.stdout: 1 479502 4.13 MB/sec execute 504 sec latency 7278.933 ms 2024-09-11T00:18:28.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:18:28 smithi012 ceph-mon[30333]: pgmap v740: 129 pgs: 129 active+clean; 119 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 168 KiB/s rd, 3.7 MiB/s wr, 152 op/s 2024-09-11T00:18:28.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:18:28 smithi178 ceph-mon[35919]: pgmap v740: 129 pgs: 129 active+clean; 119 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 168 KiB/s rd, 3.7 MiB/s wr, 152 op/s 2024-09-11T00:18:28.532 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:18:28 smithi165 ceph-mon[35356]: pgmap v740: 129 pgs: 129 active+clean; 119 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 168 KiB/s rd, 3.7 MiB/s wr, 152 op/s 2024-09-11T00:18:28.691 INFO:tasks.workunit.client.0.smithi012.stdout: 1 479502 4.12 MB/sec execute 505 sec latency 8279.053 ms 2024-09-11T00:18:29.692 INFO:tasks.workunit.client.0.smithi012.stdout: 1 479502 4.11 MB/sec execute 506 sec latency 9279.215 ms 2024-09-11T00:18:30.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:18:30 smithi012 ceph-mon[30333]: pgmap v741: 129 pgs: 129 active+clean; 118 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 2.9 MiB/s wr, 83 op/s 2024-09-11T00:18:30.330 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:18:30 smithi178 ceph-mon[35919]: pgmap v741: 129 pgs: 129 active+clean; 118 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 2.9 MiB/s wr, 83 op/s 2024-09-11T00:18:30.532 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:18:30 smithi165 ceph-mon[35356]: pgmap v741: 129 pgs: 129 active+clean; 118 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 2.9 MiB/s wr, 83 op/s 2024-09-11T00:18:30.692 INFO:tasks.workunit.client.0.smithi012.stdout: 1 479502 4.10 MB/sec execute 507 sec latency 10279.358 ms 2024-09-11T00:18:31.692 INFO:tasks.workunit.client.0.smithi012.stdout: 1 479502 4.10 MB/sec execute 508 sec latency 11279.507 ms 2024-09-11T00:18:32.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:18:32 smithi178 ceph-mon[35919]: pgmap v742: 129 pgs: 129 active+clean; 116 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 3.2 MiB/s wr, 94 op/s 2024-09-11T00:18:32.532 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:18:32 smithi165 ceph-mon[35356]: pgmap v742: 129 pgs: 129 active+clean; 116 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 3.2 MiB/s wr, 94 op/s 2024-09-11T00:18:32.563 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:18:32 smithi012 ceph-mon[30333]: pgmap v742: 129 pgs: 129 active+clean; 116 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 3.2 MiB/s wr, 94 op/s 2024-09-11T00:18:32.692 INFO:tasks.workunit.client.0.smithi012.stdout: 1 479502 4.09 MB/sec execute 509 sec latency 12279.650 ms 2024-09-11T00:18:33.692 INFO:tasks.workunit.client.0.smithi012.stdout: 1 479502 4.08 MB/sec execute 510 sec latency 13279.789 ms 2024-09-11T00:18:34.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:18:34 smithi178 ceph-mon[35919]: pgmap v743: 129 pgs: 129 active+clean; 114 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 115 KiB/s rd, 2.8 MiB/s wr, 86 op/s 2024-09-11T00:18:34.532 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:18:34 smithi165 ceph-mon[35356]: pgmap v743: 129 pgs: 129 active+clean; 114 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 115 KiB/s rd, 2.8 MiB/s wr, 86 op/s 2024-09-11T00:18:34.563 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:18:34 smithi012 ceph-mon[30333]: pgmap v743: 129 pgs: 129 active+clean; 114 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 115 KiB/s rd, 2.8 MiB/s wr, 86 op/s 2024-09-11T00:18:34.692 INFO:tasks.workunit.client.0.smithi012.stdout: 1 479502 4.07 MB/sec execute 511 sec latency 14279.949 ms 2024-09-11T00:18:35.692 INFO:tasks.workunit.client.0.smithi012.stdout: 1 479502 4.06 MB/sec execute 512 sec latency 15280.104 ms 2024-09-11T00:18:36.221 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:18:36 smithi012 ceph-mon[30333]: pgmap v744: 129 pgs: 129 active+clean; 113 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 2.3 MiB/s wr, 65 op/s 2024-09-11T00:18:36.532 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:18:36 smithi165 ceph-mon[35356]: pgmap v744: 129 pgs: 129 active+clean; 113 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 2.3 MiB/s wr, 65 op/s 2024-09-11T00:18:36.581 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:18:36 smithi178 ceph-mon[35919]: pgmap v744: 129 pgs: 129 active+clean; 113 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 2.3 MiB/s wr, 65 op/s 2024-09-11T00:18:36.692 INFO:tasks.workunit.client.0.smithi012.stdout: 1 479502 4.06 MB/sec execute 513 sec latency 16280.189 ms 2024-09-11T00:18:37.316 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:18:37 smithi012 ceph-mon[30333]: pgmap v745: 129 pgs: 129 active+clean; 114 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 2.3 MiB/s wr, 78 op/s 2024-09-11T00:18:37.532 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:18:37 smithi165 ceph-mon[35356]: pgmap v745: 129 pgs: 129 active+clean; 114 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 2.3 MiB/s wr, 78 op/s 2024-09-11T00:18:37.581 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:18:37 smithi178 ceph-mon[35919]: pgmap v745: 129 pgs: 129 active+clean; 114 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 2.3 MiB/s wr, 78 op/s 2024-09-11T00:18:37.693 INFO:tasks.workunit.client.0.smithi012.stdout: 1 479502 4.05 MB/sec execute 514 sec latency 17280.299 ms 2024-09-11T00:18:38.693 INFO:tasks.workunit.client.0.smithi012.stdout: 1 479502 4.04 MB/sec execute 515 sec latency 18280.443 ms 2024-09-11T00:18:39.693 INFO:tasks.workunit.client.0.smithi012.stdout: 1 479502 4.03 MB/sec execute 516 sec latency 19280.530 ms 2024-09-11T00:18:39.717 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:39.715+0000 7f4aeb000640 1 -- 172.21.15.12:0/2418344950 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4ae4105cf0 msgr2=0x7f4ae4106170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:18:39.718 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:39.715+0000 7f4aeb000640 1 --2- 172.21.15.12:0/2418344950 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4ae4105cf0 0x7f4ae4106170 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f4acc009930 tx=0x7f4acc02a440 comp rx=0 tx=0).stop 2024-09-11T00:18:39.718 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:39.715+0000 7f4aeb000640 1 -- 172.21.15.12:0/2418344950 shutdown_connections 2024-09-11T00:18:39.718 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:39.715+0000 7f4aeb000640 1 --2- 172.21.15.12:0/2418344950 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4ae41066b0 0x7f4ae410d030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:18:39.718 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:39.715+0000 7f4aeb000640 1 --2- 172.21.15.12:0/2418344950 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4ae4105cf0 0x7f4ae4106170 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:18:39.718 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:39.715+0000 7f4aeb000640 1 --2- 172.21.15.12:0/2418344950 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4ae4104ba0 0x7f4ae4104fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:18:39.718 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:39.715+0000 7f4aeb000640 1 -- 172.21.15.12:0/2418344950 >> 172.21.15.12:0/2418344950 conn(0x7f4ae4100460 msgr2=0x7f4ae4102880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:18:39.718 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:39.715+0000 7f4aeb000640 1 -- 172.21.15.12:0/2418344950 shutdown_connections 2024-09-11T00:18:39.718 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:39.715+0000 7f4aeb000640 1 -- 172.21.15.12:0/2418344950 wait complete. 2024-09-11T00:18:39.720 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:39.716+0000 7f4aeb000640 1 Processor -- start 2024-09-11T00:18:39.720 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:39.716+0000 7f4aeb000640 1 -- start start 2024-09-11T00:18:39.720 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:39.717+0000 7f4aeb000640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4ae4104ba0 0x7f4ae419af70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:18:39.721 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:39.717+0000 7f4aeb000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4ae4105cf0 0x7f4ae419b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:18:39.721 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:39.717+0000 7f4aeb000640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4ae41066b0 0x7f4ae419c1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:18:39.721 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:39.717+0000 7f4aeb000640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f4ae4110840 con 0x7f4ae41066b0 2024-09-11T00:18:39.721 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:39.717+0000 7f4aeb000640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f4ae41106c0 con 0x7f4ae4105cf0 2024-09-11T00:18:39.721 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:39.717+0000 7f4aeb000640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f4ae41109c0 con 0x7f4ae4104ba0 2024-09-11T00:18:39.721 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:39.717+0000 7f4ae9000640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4ae41066b0 0x7f4ae419c1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:18:39.721 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:39.717+0000 7f4adb400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4ae4105cf0 0x7f4ae419b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:18:39.721 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:39.717+0000 7f4adb400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4ae4105cf0 0x7f4ae419b5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.12:40946/0 (socket says 172.21.15.12:40946) 2024-09-11T00:18:39.721 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:39.717+0000 7f4ae9000640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4ae41066b0 0x7f4ae419c1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:49128/0 (socket says 172.21.15.12:49128) 2024-09-11T00:18:39.721 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:39.717+0000 7f4ae3e00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4ae4104ba0 0x7f4ae419af70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:18:39.722 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:39.717+0000 7f4ae9000640 1 -- 172.21.15.12:0/3414175996 learned_addr learned my addr 172.21.15.12:0/3414175996 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:18:39.722 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:39.717+0000 7f4ae9000640 1 -- 172.21.15.12:0/3414175996 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4ae4104ba0 msgr2=0x7f4ae419af70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:18:39.722 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:39.717+0000 7f4ae9000640 1 --2- 172.21.15.12:0/3414175996 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4ae4104ba0 0x7f4ae419af70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:18:39.722 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:39.717+0000 7f4ae9000640 1 -- 172.21.15.12:0/3414175996 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4ae4105cf0 msgr2=0x7f4ae419b5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:18:39.722 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:39.717+0000 7f4ae9000640 1 --2- 172.21.15.12:0/3414175996 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4ae4105cf0 0x7f4ae419b5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:18:39.722 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:39.717+0000 7f4ae9000640 1 -- 172.21.15.12:0/3414175996 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ae419cab0 con 0x7f4ae41066b0 2024-09-11T00:18:39.722 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:39.717+0000 7f4ae3e00640 1 --2- 172.21.15.12:0/3414175996 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4ae4104ba0 0x7f4ae419af70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:18:39.723 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:39.717+0000 7f4adb400640 1 --2- 172.21.15.12:0/3414175996 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4ae4105cf0 0x7f4ae419b5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:18:39.723 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:39.718+0000 7f4ae9000640 1 --2- 172.21.15.12:0/3414175996 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4ae41066b0 0x7f4ae419c1f0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f4adc00e970 tx=0x7f4adc00ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:18:39.723 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:39.718+0000 7f4ae1600640 1 -- 172.21.15.12:0/3414175996 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4adc00ccc0 con 0x7f4ae41066b0 2024-09-11T00:18:39.723 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:39.718+0000 7f4ae1600640 1 -- 172.21.15.12:0/3414175996 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4adc00ce60 con 0x7f4ae41066b0 2024-09-11T00:18:39.723 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:39.718+0000 7f4ae1600640 1 -- 172.21.15.12:0/3414175996 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4adc010430 con 0x7f4ae41066b0 2024-09-11T00:18:39.723 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:39.718+0000 7f4aeb000640 1 -- 172.21.15.12:0/3414175996 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4ae419cda0 con 0x7f4ae41066b0 2024-09-11T00:18:39.723 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:39.718+0000 7f4aeb000640 1 -- 172.21.15.12:0/3414175996 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4ae41a3470 con 0x7f4ae41066b0 2024-09-11T00:18:39.724 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:39.721+0000 7f4aedfd2740 1 Processor -- start 2024-09-11T00:18:39.724 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:39.722+0000 7f4ae1600640 1 -- 172.21.15.12:0/3414175996 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7f4adc004ff0 con 0x7f4ae41066b0 2024-09-11T00:18:39.724 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:39.722+0000 7f4aedfd2740 1 -- start start 2024-09-11T00:18:39.727 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:39.722+0000 7f4aedfd2740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560bc2883790 0x560bc2885b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:18:39.727 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:39.722+0000 7f4aedfd2740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560bc28860c0 0x560bc28884b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:18:39.727 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:39.722+0000 7f4aedfd2740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560bc2888b00 0x560bc288afc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:18:39.727 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:39.722+0000 7f4aedfd2740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x560bc27f66b0 con 0x560bc28860c0 2024-09-11T00:18:39.727 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:39.722+0000 7f4aedfd2740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x560bc27e5ae0 con 0x560bc2883790 2024-09-11T00:18:39.727 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:39.722+0000 7f4aedfd2740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x560bc27e98f0 con 0x560bc2888b00 2024-09-11T00:18:39.727 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:39.722+0000 7f4ae3e00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560bc28860c0 0x560bc28884b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:18:39.727 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:39.722+0000 7f4ae3e00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560bc28860c0 0x560bc28884b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:49132/0 (socket says 172.21.15.12:49132) 2024-09-11T00:18:39.727 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:39.722+0000 7f4ae3e00640 1 -- 172.21.15.12:0/2749594539 learned_addr learned my addr 172.21.15.12:0/2749594539 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:18:39.727 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:39.722+0000 7f4adb400640 1 --2- 172.21.15.12:0/2749594539 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560bc2883790 0x560bc2885b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:18:39.727 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:39.722+0000 7f4adb400640 1 --2- 172.21.15.12:0/2749594539 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560bc2888b00 0x560bc288afc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:18:39.727 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:39.723+0000 7f4ae3e00640 1 -- 172.21.15.12:0/2749594539 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560bc2888b00 msgr2=0x560bc288afc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:18:39.728 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:39.723+0000 7f4ae3e00640 1 --2- 172.21.15.12:0/2749594539 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560bc2888b00 0x560bc288afc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:18:39.728 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:39.723+0000 7f4ae3e00640 1 -- 172.21.15.12:0/2749594539 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560bc2883790 msgr2=0x560bc2885b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:18:39.728 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:39.723+0000 7f4ae3e00640 1 --2- 172.21.15.12:0/2749594539 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560bc2883790 0x560bc2885b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:18:39.728 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:39.723+0000 7f4ae3e00640 1 -- 172.21.15.12:0/2749594539 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560bc27b2780 con 0x560bc28860c0 2024-09-11T00:18:39.728 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:39.723+0000 7f4adb400640 1 --2- 172.21.15.12:0/2749594539 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560bc2888b00 0x560bc288afc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T00:18:39.728 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:39.723+0000 7f4ae3e00640 1 --2- 172.21.15.12:0/2749594539 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560bc28860c0 0x560bc28884b0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f4ad4005f40 tx=0x7f4ad4011d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a435ded008998dc1 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:18:39.728 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:39.723+0000 7f4ad9600640 1 -- 172.21.15.12:0/2749594539 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ad4011330 con 0x560bc28860c0 2024-09-11T00:18:39.728 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:39.723+0000 7f4ad9600640 1 -- 172.21.15.12:0/2749594539 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4ad400fe50 con 0x560bc28860c0 2024-09-11T00:18:39.728 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:39.723+0000 7f4ad9600640 1 -- 172.21.15.12:0/2749594539 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ad40193e0 con 0x560bc28860c0 2024-09-11T00:18:39.728 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:39.724+0000 7f4ae1600640 1 --2- 172.21.15.12:0/3414175996 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f4ac80b52e0 0x7f4ac80b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:18:39.729 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:39.724+0000 7f4ae1600640 1 -- 172.21.15.12:0/3414175996 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9208+0+0 (secure 0 0 0) 0x7f4adc0e63e0 con 0x7f4ae41066b0 2024-09-11T00:18:39.729 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:39.724+0000 7f4aedfd2740 1 -- 172.21.15.12:0/2749594539 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560bc28860c0 msgr2=0x560bc28884b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:18:39.729 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:39.724+0000 7f4aedfd2740 1 --2- 172.21.15.12:0/2749594539 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560bc28860c0 0x560bc28884b0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f4ad4005f40 tx=0x7f4ad4011d30 comp rx=0 tx=0).stop 2024-09-11T00:18:39.729 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:39.724+0000 7f4adb400640 1 --2- 172.21.15.12:0/3414175996 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f4ac80b52e0 0x7f4ac80b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:18:39.729 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:39.724+0000 7f4aedfd2740 1 -- 172.21.15.12:0/2749594539 shutdown_connections 2024-09-11T00:18:39.729 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:39.724+0000 7f4aedfd2740 1 --2- 172.21.15.12:0/2749594539 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560bc2888b00 0x560bc288afc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:18:39.729 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:39.724+0000 7f4aedfd2740 1 --2- 172.21.15.12:0/2749594539 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560bc28860c0 0x560bc28884b0 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:18:39.729 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:39.724+0000 7f4aedfd2740 1 --2- 172.21.15.12:0/2749594539 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560bc2883790 0x560bc2885b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:18:39.729 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:39.724+0000 7f4aedfd2740 1 -- 172.21.15.12:0/2749594539 >> 172.21.15.12:0/2749594539 conn(0x560bc287e8e0 msgr2=0x560bc287f670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:18:39.729 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:39.724+0000 7f4aedfd2740 1 -- 172.21.15.12:0/2749594539 shutdown_connections 2024-09-11T00:18:39.729 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:39.725+0000 7f4aedfd2740 1 -- 172.21.15.12:0/2749594539 wait complete. 2024-09-11T00:18:39.729 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:39.725+0000 7f4adb400640 1 --2- 172.21.15.12:0/3414175996 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f4ac80b52e0 0x7f4ac80b77a0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f4acc005ab0 tx=0x7f4acc005e10 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:18:39.733 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:39.725+0000 7f4aedfd2740 1 -- start start 2024-09-11T00:18:39.733 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:39.725+0000 7f4aedfd2740 1 Processor -- start 2024-09-11T00:18:39.733 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:39.726+0000 7f4aedfd2740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560bc2883790 0x560bc288ce60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:18:39.733 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:39.726+0000 7f4aedfd2740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560bc28860c0 0x560bc288d500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:18:39.733 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:39.726+0000 7f4aedfd2740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560bc2888b00 0x560bc288e0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:18:39.733 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:39.726+0000 7f4aedfd2740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x560bc271dab0 con 0x560bc2888b00 2024-09-11T00:18:39.734 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:39.726+0000 7f4aedfd2740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x560bc278c570 con 0x560bc28860c0 2024-09-11T00:18:39.734 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:39.726+0000 7f4aedfd2740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x560bc288bb30 con 0x560bc2883790 2024-09-11T00:18:39.734 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:39.726+0000 7f4aedfd2740 10 client.0 _ll_register_callbacks cb 0x560bc279b100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:18:39.734 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:39.726+0000 7f4adb400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560bc28860c0 0x560bc288d500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:18:39.734 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:39.726+0000 7f4adb400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560bc28860c0 0x560bc288d500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.12:40952/0 (socket says 172.21.15.12:40952) 2024-09-11T00:18:39.734 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:39.726+0000 7f4adb400640 1 -- 172.21.15.12:0/3177277208 learned_addr learned my addr 172.21.15.12:0/3177277208 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:18:39.734 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:39.726+0000 7f4ae9000640 1 --2- 172.21.15.12:0/3177277208 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560bc2888b00 0x560bc288e0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:18:39.734 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:39.726+0000 7f4ae3e00640 1 --2- 172.21.15.12:0/3177277208 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560bc2883790 0x560bc288ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:18:39.734 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:39.726+0000 7f4adb400640 1 -- 172.21.15.12:0/3177277208 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560bc2883790 msgr2=0x560bc288ce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:18:39.734 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:39.726+0000 7f4adb400640 1 --2- 172.21.15.12:0/3177277208 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560bc2883790 0x560bc288ce60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:18:39.734 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:39.726+0000 7f4adb400640 1 -- 172.21.15.12:0/3177277208 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560bc2888b00 msgr2=0x560bc288e0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:18:39.735 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:39.726+0000 7f4adb400640 1 --2- 172.21.15.12:0/3177277208 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560bc2888b00 0x560bc288e0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:18:39.735 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:39.726+0000 7f4adb400640 1 -- 172.21.15.12:0/3177277208 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560bc2547790 con 0x560bc28860c0 2024-09-11T00:18:39.735 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:39.726+0000 7f4ae9000640 1 --2- 172.21.15.12:0/3177277208 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560bc2888b00 0x560bc288e0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:18:39.735 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:39.727+0000 7f4adb400640 1 --2- 172.21.15.12:0/3177277208 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560bc28860c0 0x560bc288d500 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f4acc009930 tx=0x7f4acc02a440 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d6121704a30560df server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:18:39.735 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:39.727+0000 7f4ad2000640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-09-11T00:18:39.735 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:39.727+0000 7f4ad2000640 1 -- 172.21.15.12:0/3177277208 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4acc035040 con 0x560bc28860c0 2024-09-11T00:18:39.735 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:39.727+0000 7f4aedfd2740 1 -- 172.21.15.12:0/3177277208 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4ad400fe50 con 0x560bc28860c0 2024-09-11T00:18:39.735 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:39.727+0000 7f4ad2000640 1 -- 172.21.15.12:0/3177277208 <== mon.2 v2:172.21.15.165:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9208+0+0 (secure 0 0 0) 0x7f4acc031c30 con 0x560bc28860c0 2024-09-11T00:18:39.735 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:39.727+0000 7f4ad2000640 1 -- 172.21.15.12:0/3177277208 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f4acc05d050 con 0x560bc28860c0 2024-09-11T00:18:39.735 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:39.727+0000 7f4aedfd2740 10 client.26807 fetch_fsmap learned FSMap version 44 2024-09-11T00:18:39.736 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:39.727+0000 7f4aedfd2740 1 -- 172.21.15.12:0/3177277208 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x560bc25c0620 con 0x560bc28860c0 2024-09-11T00:18:39.736 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:39.728+0000 7f4ad2000640 1 -- 172.21.15.12:0/3177277208 <== mon.2 v2:172.21.15.165:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4ad400fe50 con 0x560bc28860c0 2024-09-11T00:18:39.736 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:39.728+0000 7f4ad2000640 1 -- 172.21.15.12:0/3177277208 <== mon.2 v2:172.21.15.165:3300/0 5 ==== fsmap(e 44) ==== 8382+0+0 (secure 0 0 0) 0x7f4acc03d480 con 0x560bc28860c0 2024-09-11T00:18:39.736 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:39.728+0000 7f4aedfd2740 10 client.26807 fetch_fsmap finished waiting for FSMap version 44 2024-09-11T00:18:39.736 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:39.728+0000 7f4aedfd2740 10 client.26807 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T00:18:39.736 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:39.728+0000 7f4aedfd2740 20 client.26807 populate_metadata read hostname 'smithi012' 2024-09-11T00:18:39.736 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:39.728+0000 7f4aedfd2740 1 --2- 172.21.15.12:0/3177277208 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x560bc2898820 0x560bc289ac70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:18:39.736 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:39.728+0000 7f4aedfd2740 4 client.26807 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T00:18:39.736 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:39.728+0000 7f4aedfd2740 1 -- 172.21.15.12:0/3177277208 --> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560bc27f8600 con 0x560bc2898820 2024-09-11T00:18:39.736 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:39.728+0000 7f4ae3e00640 1 --2- 172.21.15.12:0/3177277208 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x560bc2898820 0x560bc289ac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:18:39.736 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:39.729+0000 7f4ae3e00640 1 --2- 172.21.15.12:0/3177277208 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x560bc2898820 0x560bc289ac70 crc :-1 s=READY pgs=176 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d669140dbfa8f30 server_cookie=899cd41707df6c59 in_seq=0 out_seq=0 2024-09-11T00:18:39.736 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:39.729+0000 7f4ad2000640 10 client.26807 ms_handle_connect on v2:172.21.15.165:6838/501312984 2024-09-11T00:18:39.737 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:39.730+0000 7f4ad2000640 1 -- 172.21.15.12:0/3177277208 <== mds.0 v2:172.21.15.165:6838/501312984 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x560bc27f8600 con 0x560bc2898820 2024-09-11T00:18:39.737 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:39.730+0000 7f4ad2000640 10 client.26807 handle_command_reply: tid=0 2024-09-11T00:18:39.737 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:39.730+0000 7f4aedfd2740 1 client.26807 shutdown 2024-09-11T00:18:39.737 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:39.730+0000 7f4aedfd2740 1 -- 172.21.15.12:0/3177277208 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560bc28860c0 msgr2=0x560bc288d500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:18:39.737 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:39.730+0000 7f4aedfd2740 1 --2- 172.21.15.12:0/3177277208 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560bc28860c0 0x560bc288d500 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f4acc009930 tx=0x7f4acc02a440 comp rx=0 tx=0).stop 2024-09-11T00:18:39.737 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:39.730+0000 7f4aedfd2740 1 -- 172.21.15.12:0/3177277208 shutdown_connections 2024-09-11T00:18:39.737 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:39.730+0000 7f4aedfd2740 1 --2- 172.21.15.12:0/3177277208 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x560bc2898820 0x560bc289ac70 crc :-1 s=READY pgs=176 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:18:39.737 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:39.730+0000 7f4aedfd2740 1 --2- 172.21.15.12:0/3177277208 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560bc2888b00 0x560bc288e0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:18:39.737 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:39.730+0000 7f4aedfd2740 1 --2- 172.21.15.12:0/3177277208 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560bc28860c0 0x560bc288d500 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:18:39.737 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:39.730+0000 7f4ad2000640 0 client.26807 ms_handle_reset on v2:172.21.15.165:6838/501312984 2024-09-11T00:18:39.737 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:39.730+0000 7f4aedfd2740 1 --2- 172.21.15.12:0/3177277208 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560bc2883790 0x560bc288ce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:18:39.737 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:39.730+0000 7f4aedfd2740 1 -- 172.21.15.12:0/3177277208 >> 172.21.15.12:0/3177277208 conn(0x560bc287e8e0 msgr2=0x560bc287da60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:18:39.738 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:39.730+0000 7f4aedfd2740 1 -- 172.21.15.12:0/3177277208 shutdown_connections 2024-09-11T00:18:39.738 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:39.730+0000 7f4aedfd2740 1 -- 172.21.15.12:0/3177277208 wait complete. 2024-09-11T00:18:39.738 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:39.730+0000 7f4aedfd2740 20 client.26807 trim_cache size 0 max 16384 2024-09-11T00:18:39.750 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:39.745+0000 7f4aedfd2740 1 Processor -- start 2024-09-11T00:18:39.750 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:39.746+0000 7f4aedfd2740 1 -- start start 2024-09-11T00:18:39.750 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:39.746+0000 7f4aedfd2740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560bc2883790 0x560bc2882b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:18:39.750 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:39.746+0000 7f4aedfd2740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560bc28860c0 0x560bc28830d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:18:39.750 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:39.746+0000 7f4aedfd2740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560bc2888b00 0x560bc2880710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:18:39.750 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:39.746+0000 7f4aedfd2740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f4ae41ad930 con 0x560bc28860c0 2024-09-11T00:18:39.750 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:39.746+0000 7f4aedfd2740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f4ae41af500 con 0x560bc2883790 2024-09-11T00:18:39.750 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:39.746+0000 7f4aedfd2740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f4ae41ae030 con 0x560bc2888b00 2024-09-11T00:18:39.750 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:39.746+0000 7f4adb400640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560bc28860c0 0x560bc28830d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:18:39.751 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:39.746+0000 7f4adb400640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560bc28860c0 0x560bc28830d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:49150/0 (socket says 172.21.15.12:49150) 2024-09-11T00:18:39.751 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:39.746+0000 7f4adb400640 1 -- 172.21.15.12:0/781878290 learned_addr learned my addr 172.21.15.12:0/781878290 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:18:39.751 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:39.746+0000 7f4ae9000640 1 --2- 172.21.15.12:0/781878290 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560bc2888b00 0x560bc2880710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:18:39.751 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:39.746+0000 7f4ae3e00640 1 --2- 172.21.15.12:0/781878290 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560bc2883790 0x560bc2882b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:18:39.751 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:39.746+0000 7f4adb400640 1 -- 172.21.15.12:0/781878290 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560bc2888b00 msgr2=0x560bc2880710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:18:39.751 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:39.746+0000 7f4adb400640 1 --2- 172.21.15.12:0/781878290 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560bc2888b00 0x560bc2880710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:18:39.751 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:39.746+0000 7f4adb400640 1 -- 172.21.15.12:0/781878290 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560bc2883790 msgr2=0x560bc2882b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:18:39.751 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:39.746+0000 7f4adb400640 1 --2- 172.21.15.12:0/781878290 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560bc2883790 0x560bc2882b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:18:39.751 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:39.746+0000 7f4adb400640 1 -- 172.21.15.12:0/781878290 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560bc28954c0 con 0x560bc28860c0 2024-09-11T00:18:39.751 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:39.747+0000 7f4adb400640 1 --2- 172.21.15.12:0/781878290 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560bc28860c0 0x560bc28830d0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f4acc009230 tx=0x7f4acc0058d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28d882339936aa95 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:18:39.751 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:39.747+0000 7f4ad8c00640 1 -- 172.21.15.12:0/781878290 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4acc035040 con 0x560bc28860c0 2024-09-11T00:18:39.751 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:39.747+0000 7f4ad8c00640 1 -- 172.21.15.12:0/781878290 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4acc03dd10 con 0x560bc28860c0 2024-09-11T00:18:39.752 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:39.747+0000 7f4ad8c00640 1 -- 172.21.15.12:0/781878290 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4acc005500 con 0x560bc28860c0 2024-09-11T00:18:39.752 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:39.748+0000 7f4aedfd2740 1 -- 172.21.15.12:0/781878290 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560bc28860c0 msgr2=0x560bc28830d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:18:39.752 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:39.748+0000 7f4aedfd2740 1 --2- 172.21.15.12:0/781878290 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560bc28860c0 0x560bc28830d0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f4acc009230 tx=0x7f4acc0058d0 comp rx=0 tx=0).stop 2024-09-11T00:18:39.752 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:39.748+0000 7f4aedfd2740 1 -- 172.21.15.12:0/781878290 shutdown_connections 2024-09-11T00:18:39.752 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:39.748+0000 7f4aedfd2740 1 --2- 172.21.15.12:0/781878290 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560bc2888b00 0x560bc2880710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:18:39.752 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:39.748+0000 7f4aedfd2740 1 --2- 172.21.15.12:0/781878290 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560bc28860c0 0x560bc28830d0 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:18:39.752 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:39.748+0000 7f4aedfd2740 1 --2- 172.21.15.12:0/781878290 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560bc2883790 0x560bc2882b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:18:39.752 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:39.748+0000 7f4aedfd2740 1 -- 172.21.15.12:0/781878290 >> 172.21.15.12:0/781878290 conn(0x560bc287e8e0 msgr2=0x560bc2888ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:18:39.752 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:39.748+0000 7f4aedfd2740 1 -- 172.21.15.12:0/781878290 shutdown_connections 2024-09-11T00:18:39.753 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:39.748+0000 7f4aedfd2740 1 -- 172.21.15.12:0/781878290 wait complete. 2024-09-11T00:18:39.755 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:39.748+0000 7f4aedfd2740 1 -- start start 2024-09-11T00:18:39.755 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:39.748+0000 7f4aedfd2740 1 Processor -- start 2024-09-11T00:18:39.755 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:39.749+0000 7f4aedfd2740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560bc2883790 0x560bc2893d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:18:39.756 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:39.749+0000 7f4aedfd2740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560bc28860c0 0x560bc2894270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:18:39.756 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:39.749+0000 7f4aedfd2740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560bc2888b00 0x560bc2892850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:18:39.756 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:39.749+0000 7f4aedfd2740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f4ae41aee30 con 0x560bc2883790 2024-09-11T00:18:39.756 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:39.749+0000 7f4aedfd2740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f4ae41ae700 con 0x560bc2888b00 2024-09-11T00:18:39.756 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:39.749+0000 7f4aedfd2740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f4ae41ab910 con 0x560bc28860c0 2024-09-11T00:18:39.756 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:39.749+0000 7f4aedfd2740 10 client.0 _ll_register_callbacks cb 0x7f4acc009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:18:39.756 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:39.749+0000 7f4ae3e00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560bc2883790 0x560bc2893d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:18:39.756 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:39.749+0000 7f4ae3e00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560bc2883790 0x560bc2893d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:49164/0 (socket says 172.21.15.12:49164) 2024-09-11T00:18:39.756 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:39.749+0000 7f4ae3e00640 1 -- 172.21.15.12:0/1501348110 learned_addr learned my addr 172.21.15.12:0/1501348110 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:18:39.756 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:39.749+0000 7f4ae9000640 1 --2- 172.21.15.12:0/1501348110 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560bc2888b00 0x560bc2892850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:18:39.756 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:39.749+0000 7f4adb400640 1 --2- 172.21.15.12:0/1501348110 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560bc28860c0 0x560bc2894270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:18:39.756 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:39.749+0000 7f4ae3e00640 1 -- 172.21.15.12:0/1501348110 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560bc28860c0 msgr2=0x560bc2894270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:18:39.757 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:39.749+0000 7f4ae3e00640 1 --2- 172.21.15.12:0/1501348110 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560bc28860c0 0x560bc2894270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:18:39.757 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:39.749+0000 7f4ae3e00640 1 -- 172.21.15.12:0/1501348110 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560bc2888b00 msgr2=0x560bc2892850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:18:39.757 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:39.749+0000 7f4ae3e00640 1 --2- 172.21.15.12:0/1501348110 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560bc2888b00 0x560bc2892850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:18:39.757 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:39.749+0000 7f4ae3e00640 1 -- 172.21.15.12:0/1501348110 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560bc2892d90 con 0x560bc2883790 2024-09-11T00:18:39.757 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:39.749+0000 7f4ae9000640 1 --2- 172.21.15.12:0/1501348110 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560bc2888b00 0x560bc2892850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:18:39.757 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:39.750+0000 7f4ae3e00640 1 --2- 172.21.15.12:0/1501348110 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560bc2883790 0x560bc2893d30 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f4ad4006010 tx=0x7f4ad40113f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81c9e9e2d70ed3a5 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:18:39.757 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:39.750+0000 7f4ad2000640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-09-11T00:18:39.757 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:39.750+0000 7f4ad2000640 1 -- 172.21.15.12:0/1501348110 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ad401f040 con 0x560bc2883790 2024-09-11T00:18:39.757 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:39.750+0000 7f4ad2000640 1 -- 172.21.15.12:0/1501348110 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9208+0+0 (secure 0 0 0) 0x7f4ad400f9a0 con 0x560bc2883790 2024-09-11T00:18:39.757 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:39.750+0000 7f4aedfd2740 1 -- 172.21.15.12:0/1501348110 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4acc03dd10 con 0x560bc2883790 2024-09-11T00:18:39.757 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:39.750+0000 7f4ad2000640 1 -- 172.21.15.12:0/1501348110 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f4ad4011130 con 0x560bc2883790 2024-09-11T00:18:39.757 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:39.750+0000 7f4aedfd2740 10 client.16809 fetch_fsmap learned FSMap version 44 2024-09-11T00:18:39.758 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:39.750+0000 7f4aedfd2740 1 -- 172.21.15.12:0/1501348110 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x560bc2905f30 con 0x560bc2883790 2024-09-11T00:18:39.758 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:39.751+0000 7f4ad2000640 1 -- 172.21.15.12:0/1501348110 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4acc03dd10 con 0x560bc2883790 2024-09-11T00:18:39.758 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:39.751+0000 7f4ad2000640 1 -- 172.21.15.12:0/1501348110 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 44) ==== 8382+0+0 (secure 0 0 0) 0x7f4ad4018070 con 0x560bc2883790 2024-09-11T00:18:39.758 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:39.751+0000 7f4aedfd2740 10 client.16809 fetch_fsmap finished waiting for FSMap version 44 2024-09-11T00:18:39.758 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:39.751+0000 7f4aedfd2740 10 client.16809 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T00:18:39.758 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:39.751+0000 7f4aedfd2740 20 client.16809 populate_metadata read hostname 'smithi012' 2024-09-11T00:18:39.758 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:39.751+0000 7f4aedfd2740 1 --2- 172.21.15.12:0/1501348110 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x560bc29067f0 0x560bc2908c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:18:39.758 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:39.751+0000 7f4aedfd2740 4 client.16809 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T00:18:39.758 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:39.751+0000 7f4aedfd2740 1 -- 172.21.15.12:0/1501348110 --> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] -- command(tid 0: {"prefix": "flush journal"}) -- 0x560bc27f40b0 con 0x560bc29067f0 2024-09-11T00:18:39.758 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:39.751+0000 7f4adb400640 1 --2- 172.21.15.12:0/1501348110 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x560bc29067f0 0x560bc2908c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:18:39.758 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:39.752+0000 7f4adb400640 1 --2- 172.21.15.12:0/1501348110 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x560bc29067f0 0x560bc2908c10 crc :-1 s=READY pgs=177 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5b161698a1f43742 server_cookie=bb5e2f90db93c04c in_seq=0 out_seq=0 2024-09-11T00:18:39.758 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:39.752+0000 7f4ad2000640 10 client.16809 ms_handle_connect on v2:172.21.15.165:6838/501312984 2024-09-11T00:18:39.778 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:39.775+0000 7f4ad2000640 1 -- 172.21.15.12:0/1501348110 <== mds.0 v2:172.21.15.165:6838/501312984 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x560bc27f40b0 con 0x560bc29067f0 2024-09-11T00:18:39.778 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:39.775+0000 7f4ad2000640 10 client.16809 handle_command_reply: tid=0 2024-09-11T00:18:39.779 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:39.775+0000 7f4aedfd2740 1 client.16809 shutdown 2024-09-11T00:18:39.779 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:39.775+0000 7f4aedfd2740 1 -- 172.21.15.12:0/1501348110 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560bc2883790 msgr2=0x560bc2893d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:18:39.779 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:39.775+0000 7f4aedfd2740 1 --2- 172.21.15.12:0/1501348110 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560bc2883790 0x560bc2893d30 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f4ad4006010 tx=0x7f4ad40113f0 comp rx=0 tx=0).stop 2024-09-11T00:18:39.779 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:39.775+0000 7f4aedfd2740 1 -- 172.21.15.12:0/1501348110 shutdown_connections 2024-09-11T00:18:39.779 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:39.775+0000 7f4aedfd2740 1 --2- 172.21.15.12:0/1501348110 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x560bc29067f0 0x560bc2908c10 crc :-1 s=READY pgs=177 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:18:39.779 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:39.775+0000 7f4aedfd2740 1 --2- 172.21.15.12:0/1501348110 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560bc2888b00 0x560bc2892850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:18:39.779 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:39.775+0000 7f4ad2000640 0 client.16809 ms_handle_reset on v2:172.21.15.165:6838/501312984 2024-09-11T00:18:39.779 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:39.775+0000 7f4aedfd2740 1 --2- 172.21.15.12:0/1501348110 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560bc28860c0 0x560bc2894270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:18:39.779 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:39.775+0000 7f4aedfd2740 1 --2- 172.21.15.12:0/1501348110 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560bc2883790 0x560bc2893d30 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:18:39.779 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:39.775+0000 7f4aedfd2740 1 -- 172.21.15.12:0/1501348110 >> 172.21.15.12:0/1501348110 conn(0x560bc287e8e0 msgr2=0x560bc2882760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:18:39.780 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:39.775+0000 7f4aedfd2740 1 -- 172.21.15.12:0/1501348110 shutdown_connections 2024-09-11T00:18:39.780 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:39.775+0000 7f4aedfd2740 1 -- 172.21.15.12:0/1501348110 wait complete. 2024-09-11T00:18:39.780 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:39.775+0000 7f4aedfd2740 20 client.16809 trim_cache size 0 max 16384 2024-09-11T00:18:39.780 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:{ 2024-09-11T00:18:39.780 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "message": "", 2024-09-11T00:18:39.780 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "return_code": 0 2024-09-11T00:18:39.780 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:} 2024-09-11T00:18:39.781 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:39.776+0000 7f4aeb000640 1 -- 172.21.15.12:0/3414175996 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f4ac80b52e0 msgr2=0x7f4ac80b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:18:39.781 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:39.776+0000 7f4aeb000640 1 --2- 172.21.15.12:0/3414175996 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f4ac80b52e0 0x7f4ac80b77a0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f4acc005ab0 tx=0x7f4acc005e10 comp rx=0 tx=0).stop 2024-09-11T00:18:39.781 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:39.776+0000 7f4aeb000640 1 -- 172.21.15.12:0/3414175996 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4ae41066b0 msgr2=0x7f4ae419c1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:18:39.782 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:39.776+0000 7f4aeb000640 1 --2- 172.21.15.12:0/3414175996 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4ae41066b0 0x7f4ae419c1f0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f4adc00e970 tx=0x7f4adc00ed70 comp rx=0 tx=0).stop 2024-09-11T00:18:39.782 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:39.776+0000 7f4aeb000640 1 -- 172.21.15.12:0/3414175996 shutdown_connections 2024-09-11T00:18:39.782 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:39.776+0000 7f4aeb000640 1 --2- 172.21.15.12:0/3414175996 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f4ac80b52e0 0x7f4ac80b77a0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:18:39.782 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:39.776+0000 7f4aeb000640 1 --2- 172.21.15.12:0/3414175996 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4ae41066b0 0x7f4ae419c1f0 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:18:39.782 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:39.776+0000 7f4aeb000640 1 --2- 172.21.15.12:0/3414175996 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4ae4105cf0 0x7f4ae419b5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:18:39.782 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:39.776+0000 7f4aeb000640 1 --2- 172.21.15.12:0/3414175996 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4ae4104ba0 0x7f4ae419af70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:18:39.782 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:39.776+0000 7f4aeb000640 1 -- 172.21.15.12:0/3414175996 >> 172.21.15.12:0/3414175996 conn(0x7f4ae4100460 msgr2=0x7f4ae4109080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:18:39.782 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:39.776+0000 7f4aeb000640 1 -- 172.21.15.12:0/3414175996 shutdown_connections 2024-09-11T00:18:39.782 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:39.776+0000 7f4aeb000640 1 -- 172.21.15.12:0/3414175996 wait complete. 2024-09-11T00:18:39.783 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:18:39 smithi165 ceph-mon[35356]: pgmap v746: 129 pgs: 129 active+clean; 114 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 595 KiB/s wr, 47 op/s 2024-09-11T00:18:39.813 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:18:39 smithi012 ceph-mon[30333]: pgmap v746: 129 pgs: 129 active+clean; 114 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 595 KiB/s wr, 47 op/s 2024-09-11T00:18:39.831 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:18:39 smithi178 ceph-mon[35919]: pgmap v746: 129 pgs: 129 active+clean; 114 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 595 KiB/s wr, 47 op/s 2024-09-11T00:18:40.693 INFO:tasks.workunit.client.0.smithi012.stdout: 1 479502 4.03 MB/sec execute 517 sec latency 20280.628 ms 2024-09-11T00:18:41.693 INFO:tasks.workunit.client.0.smithi012.stdout: 1 479502 4.02 MB/sec execute 518 sec latency 21280.701 ms 2024-09-11T00:18:41.782 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:18:41 smithi165 ceph-mon[35356]: pgmap v747: 129 pgs: 129 active+clean; 113 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 816 KiB/s wr, 61 op/s 2024-09-11T00:18:41.813 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:18:41 smithi012 ceph-mon[30333]: pgmap v747: 129 pgs: 129 active+clean; 113 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 816 KiB/s wr, 61 op/s 2024-09-11T00:18:41.831 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:18:41 smithi178 ceph-mon[35919]: pgmap v747: 129 pgs: 129 active+clean; 113 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 816 KiB/s wr, 61 op/s 2024-09-11T00:18:42.129 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid ff3ad74a-6fcf-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T00:18:42.537 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:18:42.693 INFO:tasks.workunit.client.0.smithi012.stdout: 1 479502 4.01 MB/sec execute 519 sec latency 22280.743 ms 2024-09-11T00:18:43.298 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:43.295+0000 7fc309a00640 1 -- 172.21.15.12:0/3762922979 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc304104e10 msgr2=0x7fc3041051f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:18:43.298 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:43.295+0000 7fc309a00640 1 --2- 172.21.15.12:0/3762922979 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc304104e10 0x7fc3041051f0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fc2f4009900 tx=0x7fc2f402a3f0 comp rx=0 tx=0).stop 2024-09-11T00:18:43.298 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:43.295+0000 7fc309a00640 1 -- 172.21.15.12:0/3762922979 shutdown_connections 2024-09-11T00:18:43.299 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:43.295+0000 7fc309a00640 1 --2- 172.21.15.12:0/3762922979 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc30410b1a0 0x7fc3041119a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:18:43.299 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:43.295+0000 7fc309a00640 1 --2- 172.21.15.12:0/3762922979 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc30410a820 0x7fc30410ac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:18:43.299 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:43.295+0000 7fc309a00640 1 --2- 172.21.15.12:0/3762922979 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc304104e10 0x7fc3041051f0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:18:43.299 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:43.295+0000 7fc309a00640 1 -- 172.21.15.12:0/3762922979 >> 172.21.15.12:0/3762922979 conn(0x7fc304100840 msgr2=0x7fc304102c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:18:43.299 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:43.295+0000 7fc309a00640 1 -- 172.21.15.12:0/3762922979 shutdown_connections 2024-09-11T00:18:43.299 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:43.295+0000 7fc309a00640 1 -- 172.21.15.12:0/3762922979 wait complete. 2024-09-11T00:18:43.300 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:43.296+0000 7fc309a00640 1 Processor -- start 2024-09-11T00:18:43.300 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:43.297+0000 7fc309a00640 1 -- start start 2024-09-11T00:18:43.300 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:43.297+0000 7fc309a00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc304104e10 0x7fc30419b200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:18:43.300 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:43.297+0000 7fc309a00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc30410a820 0x7fc30419b740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:18:43.300 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:43.297+0000 7fc309a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc30410b1a0 0x7fc3041a0ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:18:43.300 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:43.297+0000 7fc309a00640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fc304114e70 con 0x7fc30410a820 2024-09-11T00:18:43.300 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:43.297+0000 7fc309a00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fc304114cf0 con 0x7fc30410b1a0 2024-09-11T00:18:43.301 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:43.297+0000 7fc309a00640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fc304114ff0 con 0x7fc304104e10 2024-09-11T00:18:43.301 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:43.297+0000 7fc303e00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc304104e10 0x7fc30419b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:18:43.301 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:43.297+0000 7fc303400640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc30410a820 0x7fc30419b740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:18:43.301 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:43.297+0000 7fc303e00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc304104e10 0x7fc30419b200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.12:47388/0 (socket says 172.21.15.12:47388) 2024-09-11T00:18:43.301 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:43.297+0000 7fc303e00640 1 -- 172.21.15.12:0/1879317465 learned_addr learned my addr 172.21.15.12:0/1879317465 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:18:43.301 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:43.298+0000 7fc309000640 1 --2- 172.21.15.12:0/1879317465 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc30410b1a0 0x7fc3041a0ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:18:43.302 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:43.298+0000 7fc303e00640 1 -- 172.21.15.12:0/1879317465 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc30410b1a0 msgr2=0x7fc3041a0ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:18:43.302 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:43.298+0000 7fc303e00640 1 --2- 172.21.15.12:0/1879317465 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc30410b1a0 0x7fc3041a0ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:18:43.302 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:43.298+0000 7fc303e00640 1 -- 172.21.15.12:0/1879317465 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc30410a820 msgr2=0x7fc30419b740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:18:43.302 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:43.298+0000 7fc303e00640 1 --2- 172.21.15.12:0/1879317465 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc30410a820 0x7fc30419b740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:18:43.302 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:43.298+0000 7fc303e00640 1 -- 172.21.15.12:0/1879317465 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3041a12b0 con 0x7fc304104e10 2024-09-11T00:18:43.302 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:43.298+0000 7fc309000640 1 --2- 172.21.15.12:0/1879317465 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc30410b1a0 0x7fc3041a0ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:18:43.302 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:43.298+0000 7fc303400640 1 --2- 172.21.15.12:0/1879317465 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc30410a820 0x7fc30419b740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:18:43.302 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:43.298+0000 7fc303e00640 1 --2- 172.21.15.12:0/1879317465 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc304104e10 0x7fc30419b200 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fc2f4005ab0 tx=0x7fc2f4003c10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:18:43.302 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:43.299+0000 7fc300c00640 1 -- 172.21.15.12:0/1879317465 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2f403f070 con 0x7fc304104e10 2024-09-11T00:18:43.303 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:43.299+0000 7fc300c00640 1 -- 172.21.15.12:0/1879317465 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc2f4031b60 con 0x7fc304104e10 2024-09-11T00:18:43.303 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:43.299+0000 7fc309a00640 1 -- 172.21.15.12:0/1879317465 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc3041a1540 con 0x7fc304104e10 2024-09-11T00:18:43.303 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:43.299+0000 7fc300c00640 1 -- 172.21.15.12:0/1879317465 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2f4035030 con 0x7fc304104e10 2024-09-11T00:18:43.303 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:43.299+0000 7fc309a00640 1 -- 172.21.15.12:0/1879317465 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3041a7c60 con 0x7fc304104e10 2024-09-11T00:18:43.303 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:43.300+0000 7fc309a00640 1 -- 172.21.15.12:0/1879317465 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc3041051f0 con 0x7fc304104e10 2024-09-11T00:18:43.305 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:43.303+0000 7fc300c00640 1 -- 172.21.15.12:0/1879317465 <== mon.1 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7fc2f403b460 con 0x7fc304104e10 2024-09-11T00:18:43.306 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:43.304+0000 7fc300c00640 1 --2- 172.21.15.12:0/1879317465 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fc2e80b5410 0x7fc2e80b78d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:18:43.306 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:43.304+0000 7fc300c00640 1 -- 172.21.15.12:0/1879317465 <== mon.1 v2:172.21.15.178:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9208+0+0 (secure 0 0 0) 0x7fc2f41070c0 con 0x7fc304104e10 2024-09-11T00:18:43.306 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:43.304+0000 7fc303400640 1 --2- 172.21.15.12:0/1879317465 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fc2e80b5410 0x7fc2e80b78d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:18:43.307 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:43.305+0000 7fc303400640 1 --2- 172.21.15.12:0/1879317465 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fc2e80b5410 0x7fc2e80b78d0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fc30410a3a0 tx=0x7fc2f0005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:18:43.307 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:43.305+0000 7fc300c00640 1 -- 172.21.15.12:0/1879317465 <== mon.1 v2:172.21.15.178:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7fc2f40cd500 con 0x7fc304104e10 2024-09-11T00:18:43.546 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:43.543+0000 7fc309a00640 1 -- 172.21.15.12:0/1879317465 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc30419c330 con 0x7fc304104e10 2024-09-11T00:18:43.546 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:18:43 smithi012 ceph-mon[30333]: pgmap v748: 129 pgs: 129 active+clean; 113 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 611 KiB/s wr, 60 op/s 2024-09-11T00:18:43.546 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:43.544+0000 7fc300c00640 1 -- 172.21.15.12:0/1879317465 <== mon.1 v2:172.21.15.178:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 44 v44) ==== 94+0+11969 (secure 0 0 0) 0x7fc2f40d1f30 con 0x7fc304104e10 2024-09-11T00:18:43.547 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-11T00:18:43.547 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":44,"btime":"2024-09-11T00:18:17:022132+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14577,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6835/447171591","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":447171591},{"type":"v1","addr":"172.21.15.12:6835","nonce":447171591}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14589,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/901486040","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":901486040},{"type":"v1","addr":"172.21.15.12:6837","nonce":901486040}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":44,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T00:04:04.739881+0000","modified":"2024-09-11T00:18:16.957829+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24674,"mds_1":24643,"mds_2":24638,"mds_3":24637,"mds_4":24614},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14631":{"gid":14631,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.12:6841/3069919437","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6840","nonce":3069919437},{"type":"v1","addr":"172.21.15.12:6841","nonce":3069919437}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24562":{"gid":24562,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6833/1141636112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":1141636112},{"type":"v1","addr":"172.21.15.178:6833","nonce":1141636112}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24581":{"gid":24581,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6833/3111980970","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6832","nonce":3111980970},{"type":"v1","addr":"172.21.15.165:6833","nonce":3111980970}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24607":{"gid":24607,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6835/4084868287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":4084868287},{"type":"v1","addr":"172.21.15.178:6835","nonce":4084868287}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24614":{"gid":24614,"name":"e","rank":4,"incarnation":26,"state":"up:active","state_seq":21,"addr":"172.21.15.165:6835/2425395274","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":2425395274},{"type":"v1","addr":"172.21.15.165:6835","nonce":2425395274}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24637":{"gid":24637,"name":"i","rank":3,"incarnation":24,"state":"up:active","state_seq":18,"addr":"172.21.15.178:6839/70628097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6838","nonce":70628097},{"type":"v1","addr":"172.21.15.178:6839","nonce":70628097}]},"join_fscid":-1,"export_targets":[0],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24638":{"gid":24638,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":18,"addr":"172.21.15.165:6837/6215088","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6836","nonce":6215088},{"type":"v1","addr":"172.21.15.165:6837","nonce":6215088}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24643":{"gid":24643,"name":"g","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.12:6839/3124093009","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6838","nonce":3124093009},{"type":"v1","addr":"172.21.15.12:6839","nonce":3124093009}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24674":{"gid":24674,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.165:6839/501312984","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6838","nonce":501312984},{"type":"v1","addr":"172.21.15.165:6839","nonce":501312984}]},"join_fscid":-1,"export_targets":[1,3],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25672":{"gid":25672,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6837/308601741","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":308601741},{"type":"v1","addr":"172.21.15.178:6837","nonce":308601741}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24674,"qdb_cluster":[24674,24643,24638,24637,24614]},"id":1}]} 2024-09-11T00:18:43.547 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 44 2024-09-11T00:18:43.549 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:43.547+0000 7fc309a00640 1 -- 172.21.15.12:0/1879317465 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fc2e80b5410 msgr2=0x7fc2e80b78d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:18:43.549 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:43.547+0000 7fc309a00640 1 --2- 172.21.15.12:0/1879317465 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fc2e80b5410 0x7fc2e80b78d0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fc30410a3a0 tx=0x7fc2f0005f90 comp rx=0 tx=0).stop 2024-09-11T00:18:43.550 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:43.547+0000 7fc309a00640 1 -- 172.21.15.12:0/1879317465 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc304104e10 msgr2=0x7fc30419b200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:18:43.550 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:43.547+0000 7fc309a00640 1 --2- 172.21.15.12:0/1879317465 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc304104e10 0x7fc30419b200 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fc2f4005ab0 tx=0x7fc2f4003c10 comp rx=0 tx=0).stop 2024-09-11T00:18:43.550 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:43.547+0000 7fc309a00640 1 -- 172.21.15.12:0/1879317465 shutdown_connections 2024-09-11T00:18:43.550 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:43.547+0000 7fc309a00640 1 --2- 172.21.15.12:0/1879317465 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fc2e80b5410 0x7fc2e80b78d0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:18:43.550 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:43.547+0000 7fc309a00640 1 --2- 172.21.15.12:0/1879317465 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc30410b1a0 0x7fc3041a0ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:18:43.550 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:43.547+0000 7fc309a00640 1 --2- 172.21.15.12:0/1879317465 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc30410a820 0x7fc30419b740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:18:43.550 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:43.547+0000 7fc309a00640 1 --2- 172.21.15.12:0/1879317465 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc304104e10 0x7fc30419b200 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:18:43.550 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:43.547+0000 7fc309a00640 1 -- 172.21.15.12:0/1879317465 >> 172.21.15.12:0/1879317465 conn(0x7fc304100840 msgr2=0x7fc304102c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:18:43.550 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:43.547+0000 7fc309a00640 1 -- 172.21.15.12:0/1879317465 shutdown_connections 2024-09-11T00:18:43.550 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:43.547+0000 7fc309a00640 1 -- 172.21.15.12:0/1879317465 wait complete. 2024-09-11T00:18:43.693 INFO:tasks.workunit.client.0.smithi012.stdout: 1 479502 4.00 MB/sec execute 520 sec latency 23280.781 ms 2024-09-11T00:18:43.782 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:18:43 smithi165 ceph-mon[35356]: pgmap v748: 129 pgs: 129 active+clean; 113 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 611 KiB/s wr, 60 op/s 2024-09-11T00:18:43.831 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:18:43 smithi178 ceph-mon[35919]: pgmap v748: 129 pgs: 129 active+clean; 113 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 611 KiB/s wr, 60 op/s 2024-09-11T00:18:44.380 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24674 quiesce db --set-id d9d39e24 --release --await' 2024-09-11T00:18:44.381 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid ff3ad74a-6fcf-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24674 quiesce db --set-id d9d39e24 --release --await 2024-09-11T00:18:44.693 INFO:tasks.workunit.client.0.smithi012.stdout: 1 479502 3.99 MB/sec execute 521 sec latency 24280.823 ms 2024-09-11T00:18:44.778 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:18:44.782 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:18:44 smithi165 ceph-mon[35356]: from='client.? 172.21.15.12:0/1879317465' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T00:18:44.813 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:18:44 smithi012 ceph-mon[30333]: from='client.? 172.21.15.12:0/1879317465' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T00:18:44.831 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:18:44 smithi178 ceph-mon[35919]: from='client.? 172.21.15.12:0/1879317465' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T00:18:45.458 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:18:45 smithi012 ceph-mon[30333]: pgmap v749: 129 pgs: 129 active+clean; 114 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 698 KiB/s wr, 49 op/s 2024-09-11T00:18:45.458 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:45.455+0000 7f2b0fa00640 1 -- 172.21.15.12:0/3331789056 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2b0810f760 msgr2=0x7f2b08115f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:18:45.458 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:45.455+0000 7f2b0fa00640 1 --2- 172.21.15.12:0/3331789056 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2b0810f760 0x7f2b08115f60 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f2b00009ca0 tx=0x7f2b0002a8c0 comp rx=0 tx=0).stop 2024-09-11T00:18:45.458 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:45.455+0000 7f2b0fa00640 1 -- 172.21.15.12:0/3331789056 shutdown_connections 2024-09-11T00:18:45.459 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:45.455+0000 7f2b0fa00640 1 --2- 172.21.15.12:0/3331789056 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2b0810f760 0x7f2b08115f60 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:18:45.459 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:45.455+0000 7f2b0fa00640 1 --2- 172.21.15.12:0/3331789056 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2b0810ede0 0x7f2b0810f220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:18:45.459 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:45.455+0000 7f2b0fa00640 1 --2- 172.21.15.12:0/3331789056 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2b081093d0 0x7f2b081097b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:18:45.459 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:45.455+0000 7f2b0fa00640 1 -- 172.21.15.12:0/3331789056 >> 172.21.15.12:0/3331789056 conn(0x7f2b08078a40 msgr2=0x7f2b0807ae60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:18:45.459 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:45.456+0000 7f2b0fa00640 1 -- 172.21.15.12:0/3331789056 shutdown_connections 2024-09-11T00:18:45.459 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:45.456+0000 7f2b0fa00640 1 -- 172.21.15.12:0/3331789056 wait complete. 2024-09-11T00:18:45.460 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:45.457+0000 7f2b0fa00640 1 Processor -- start 2024-09-11T00:18:45.460 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:45.457+0000 7f2b0fa00640 1 -- start start 2024-09-11T00:18:45.460 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:45.457+0000 7f2b0fa00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2b081093d0 0x7f2b0819f850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:18:45.460 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:45.457+0000 7f2b0fa00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2b0810ede0 0x7f2b0819fd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:18:45.460 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:45.457+0000 7f2b0fa00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2b0810f760 0x7f2b081a5130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:18:45.461 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:45.457+0000 7f2b0fa00640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f2b081193e0 con 0x7f2b081093d0 2024-09-11T00:18:45.461 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:45.457+0000 7f2b0fa00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f2b08119260 con 0x7f2b0810f760 2024-09-11T00:18:45.461 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:45.457+0000 7f2b0fa00640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f2b08119560 con 0x7f2b0810ede0 2024-09-11T00:18:45.461 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:45.457+0000 7f2b0d000640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2b081093d0 0x7f2b0819f850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:18:45.461 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:45.458+0000 7f2b0d000640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2b081093d0 0x7f2b0819f850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:49192/0 (socket says 172.21.15.12:49192) 2024-09-11T00:18:45.461 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:45.458+0000 7f2b0d000640 1 -- 172.21.15.12:0/3888418095 learned_addr learned my addr 172.21.15.12:0/3888418095 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:18:45.461 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:45.458+0000 7f2b07e00640 1 --2- 172.21.15.12:0/3888418095 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2b0810ede0 0x7f2b0819fd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:18:45.461 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:45.458+0000 7f2b0da00640 1 --2- 172.21.15.12:0/3888418095 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2b0810f760 0x7f2b081a5130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:18:45.461 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:45.458+0000 7f2b0d000640 1 -- 172.21.15.12:0/3888418095 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2b0810ede0 msgr2=0x7f2b0819fd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:18:45.461 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:45.458+0000 7f2b0d000640 1 --2- 172.21.15.12:0/3888418095 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2b0810ede0 0x7f2b0819fd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:18:45.461 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:45.458+0000 7f2b0d000640 1 -- 172.21.15.12:0/3888418095 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2b0810f760 msgr2=0x7f2b081a5130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:18:45.461 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:45.458+0000 7f2b0d000640 1 --2- 172.21.15.12:0/3888418095 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2b0810f760 0x7f2b081a5130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:18:45.461 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:45.458+0000 7f2b0d000640 1 -- 172.21.15.12:0/3888418095 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b081a5900 con 0x7f2b081093d0 2024-09-11T00:18:45.462 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:45.458+0000 7f2b0d000640 1 --2- 172.21.15.12:0/3888418095 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2b081093d0 0x7f2b0819f850 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f2afc00e9f0 tx=0x7f2afc00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:18:45.462 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:45.459+0000 7f2b05600640 1 -- 172.21.15.12:0/3888418095 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2afc00cd80 con 0x7f2b081093d0 2024-09-11T00:18:45.462 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:45.459+0000 7f2b05600640 1 -- 172.21.15.12:0/3888418095 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2afc0054d0 con 0x7f2b081093d0 2024-09-11T00:18:45.462 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:45.459+0000 7f2b0fa00640 1 -- 172.21.15.12:0/3888418095 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2b081a5bf0 con 0x7f2b081093d0 2024-09-11T00:18:45.462 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:45.459+0000 7f2b05600640 1 -- 172.21.15.12:0/3888418095 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2afc010690 con 0x7f2b081093d0 2024-09-11T00:18:45.462 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:45.459+0000 7f2b0fa00640 1 -- 172.21.15.12:0/3888418095 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2b081ac2b0 con 0x7f2b081093d0 2024-09-11T00:18:45.464 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:45.462+0000 7f2b1265c740 1 Processor -- start 2024-09-11T00:18:45.464 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:45.462+0000 7f2b1265c740 1 -- start start 2024-09-11T00:18:45.465 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:45.462+0000 7f2b1265c740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c7082f3ff0 0x55c7082f43d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:18:45.465 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:45.462+0000 7f2b1265c740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c7082f4910 0x55c708300360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:18:45.465 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:45.462+0000 7f2b1265c740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c7083008a0 0x55c708302c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:18:45.465 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:45.462+0000 7f2b1265c740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55c70826b290 con 0x55c7083008a0 2024-09-11T00:18:45.465 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:45.462+0000 7f2b1265c740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55c70825a6c0 con 0x55c7082f4910 2024-09-11T00:18:45.465 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:45.462+0000 7f2b1265c740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55c70825e4d0 con 0x55c7082f3ff0 2024-09-11T00:18:45.466 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:45.463+0000 7f2b07e00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c7083008a0 0x55c708302c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:18:45.466 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:45.463+0000 7f2b0da00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c7082f3ff0 0x55c7082f43d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:18:45.466 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:45.463+0000 7f2b07e00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c7083008a0 0x55c708302c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:49202/0 (socket says 172.21.15.12:49202) 2024-09-11T00:18:45.466 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:45.463+0000 7f2b0da00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c7082f3ff0 0x55c7082f43d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.12:47418/0 (socket says 172.21.15.12:47418) 2024-09-11T00:18:45.466 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:45.463+0000 7f2b07e00640 1 -- 172.21.15.12:0/2507807279 learned_addr learned my addr 172.21.15.12:0/2507807279 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:18:45.467 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:45.463+0000 7f2b05600640 1 -- 172.21.15.12:0/3888418095 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7f2afc004e90 con 0x7f2b081093d0 2024-09-11T00:18:45.467 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:45.463+0000 7f2b0da00640 1 -- 172.21.15.12:0/2507807279 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c7082f4910 msgr2=0x55c708300360 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-11T00:18:45.467 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:45.463+0000 7f2b0da00640 1 --2- 172.21.15.12:0/2507807279 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c7082f4910 0x55c708300360 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:18:45.467 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:45.463+0000 7f2b0da00640 1 -- 172.21.15.12:0/2507807279 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c7083008a0 msgr2=0x55c708302c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:18:45.467 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:45.463+0000 7f2b0da00640 1 --2- 172.21.15.12:0/2507807279 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c7083008a0 0x55c708302c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:18:45.467 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:45.463+0000 7f2b0da00640 1 -- 172.21.15.12:0/2507807279 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c7081e7460 con 0x55c7082f3ff0 2024-09-11T00:18:45.467 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:45.464+0000 7f2b0da00640 1 --2- 172.21.15.12:0/2507807279 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c7082f3ff0 0x55c7082f43d0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f2b00005e30 tx=0x7f2b0002f3c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=802bd2657956cf35 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:18:45.467 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:45.464+0000 7f2afa000640 1 -- 172.21.15.12:0/2507807279 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b00031db0 con 0x55c7082f3ff0 2024-09-11T00:18:45.467 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:45.464+0000 7f2afa000640 1 -- 172.21.15.12:0/2507807279 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2b00036070 con 0x55c7082f3ff0 2024-09-11T00:18:45.467 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:45.464+0000 7f2afa000640 1 -- 172.21.15.12:0/2507807279 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b000042a0 con 0x55c7082f3ff0 2024-09-11T00:18:45.468 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:45.465+0000 7f2b1265c740 1 -- 172.21.15.12:0/2507807279 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c7082f3ff0 msgr2=0x55c7082f43d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:18:45.468 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:45.465+0000 7f2b1265c740 1 --2- 172.21.15.12:0/2507807279 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c7082f3ff0 0x55c7082f43d0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f2b00005e30 tx=0x7f2b0002f3c0 comp rx=0 tx=0).stop 2024-09-11T00:18:45.469 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:45.465+0000 7f2b05600640 1 --2- 172.21.15.12:0/3888418095 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f2af00b52e0 0x7f2af00b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:18:45.469 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:45.465+0000 7f2b1265c740 1 -- 172.21.15.12:0/2507807279 shutdown_connections 2024-09-11T00:18:45.469 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:45.465+0000 7f2b1265c740 1 --2- 172.21.15.12:0/2507807279 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c7083008a0 0x55c708302c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:18:45.469 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:45.465+0000 7f2b1265c740 1 --2- 172.21.15.12:0/2507807279 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c7082f4910 0x55c708300360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:18:45.469 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:45.465+0000 7f2b05600640 1 -- 172.21.15.12:0/3888418095 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9208+0+0 (secure 0 0 0) 0x7f2afc0e6410 con 0x7f2b081093d0 2024-09-11T00:18:45.469 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:45.465+0000 7f2b1265c740 1 --2- 172.21.15.12:0/2507807279 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c7082f3ff0 0x55c7082f43d0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:18:45.469 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:45.465+0000 7f2b1265c740 1 -- 172.21.15.12:0/2507807279 >> 172.21.15.12:0/2507807279 conn(0x55c7082f1e20 msgr2=0x55c7082f2fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:18:45.469 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:45.465+0000 7f2b1265c740 1 -- 172.21.15.12:0/2507807279 shutdown_connections 2024-09-11T00:18:45.469 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:45.465+0000 7f2b0da00640 1 --2- 172.21.15.12:0/3888418095 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f2af00b52e0 0x7f2af00b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:18:45.469 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:45.465+0000 7f2b1265c740 1 -- 172.21.15.12:0/2507807279 wait complete. 2024-09-11T00:18:45.469 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:45.466+0000 7f2b1265c740 1 -- start start 2024-09-11T00:18:45.469 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:45.466+0000 7f2b0da00640 1 --2- 172.21.15.12:0/3888418095 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f2af00b52e0 0x7f2af00b77a0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f2b00003e90 tx=0x7f2b0003e040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:18:45.469 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:45.466+0000 7f2b1265c740 1 Processor -- start 2024-09-11T00:18:45.471 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:45.467+0000 7f2b1265c740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c7082f3ff0 0x55c7082fd8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:18:45.471 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:45.467+0000 7f2b1265c740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c7082f4910 0x55c7082fde10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:18:45.472 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:45.467+0000 7f2b1265c740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c7083008a0 0x55c7082fe350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:18:45.472 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:45.467+0000 7f2b1265c740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55c7080bbbb0 con 0x55c7083008a0 2024-09-11T00:18:45.472 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:45.467+0000 7f2b1265c740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55c7081f56c0 con 0x55c7082f4910 2024-09-11T00:18:45.472 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:45.467+0000 7f2b1265c740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55c708303370 con 0x55c7082f3ff0 2024-09-11T00:18:45.472 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:45.467+0000 7f2b1265c740 10 client.0 _ll_register_callbacks cb 0x55c708210490 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:18:45.472 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:45.467+0000 7f2b0da00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c7083008a0 0x55c7082fe350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:18:45.472 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:45.467+0000 7f2b0da00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c7083008a0 0x55c7082fe350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:49218/0 (socket says 172.21.15.12:49218) 2024-09-11T00:18:45.472 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:45.467+0000 7f2b0da00640 1 -- 172.21.15.12:0/1112853209 learned_addr learned my addr 172.21.15.12:0/1112853209 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:18:45.472 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:45.467+0000 7f2b0d000640 1 --2- 172.21.15.12:0/1112853209 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c7082f3ff0 0x55c7082fd8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:18:45.472 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:45.467+0000 7f2b07e00640 1 --2- 172.21.15.12:0/1112853209 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c7082f4910 0x55c7082fde10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:18:45.472 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:45.467+0000 7f2b0d000640 1 -- 172.21.15.12:0/1112853209 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c7082f4910 msgr2=0x55c7082fde10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:18:45.473 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:45.467+0000 7f2b0d000640 1 --2- 172.21.15.12:0/1112853209 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c7082f4910 0x55c7082fde10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:18:45.473 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:45.467+0000 7f2b0d000640 1 -- 172.21.15.12:0/1112853209 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c7083008a0 msgr2=0x55c7082fe350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:18:45.473 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:45.467+0000 7f2b0d000640 1 --2- 172.21.15.12:0/1112853209 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c7083008a0 0x55c7082fe350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:18:45.473 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:45.467+0000 7f2b0d000640 1 -- 172.21.15.12:0/1112853209 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c707fb8b00 con 0x55c7082f3ff0 2024-09-11T00:18:45.473 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:45.468+0000 7f2b0d000640 1 --2- 172.21.15.12:0/1112853209 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c7082f3ff0 0x55c7082fd8d0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x55c7082f7170 tx=0x7f2afc0ac570 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5caa597e750affb7 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:18:45.473 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:45.468+0000 7f2aeea00640 10 client.0 ms_handle_connect on v2:172.21.15.178:3300/0 2024-09-11T00:18:45.473 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:45.468+0000 7f2aeea00640 1 -- 172.21.15.12:0/1112853209 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2afc0ac2b0 con 0x55c7082f3ff0 2024-09-11T00:18:45.473 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:45.468+0000 7f2b1265c740 1 -- 172.21.15.12:0/1112853209 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2b00036070 con 0x55c7082f3ff0 2024-09-11T00:18:45.473 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:45.468+0000 7f2aeea00640 1 -- 172.21.15.12:0/1112853209 <== mon.1 v2:172.21.15.178:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9208+0+0 (secure 0 0 0) 0x7f2afc0a8a10 con 0x55c7082f3ff0 2024-09-11T00:18:45.473 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:45.468+0000 7f2aeea00640 1 -- 172.21.15.12:0/1112853209 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f2afc0acea0 con 0x55c7082f3ff0 2024-09-11T00:18:45.473 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:45.468+0000 7f2b1265c740 10 client.26803 fetch_fsmap learned FSMap version 44 2024-09-11T00:18:45.473 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:45.468+0000 7f2b1265c740 1 -- 172.21.15.12:0/1112853209 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x55c70802f960 con 0x55c7082f3ff0 2024-09-11T00:18:45.473 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:45.468+0000 7f2aeea00640 1 -- 172.21.15.12:0/1112853209 <== mon.1 v2:172.21.15.178:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2b00036070 con 0x55c7082f3ff0 2024-09-11T00:18:45.473 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:45.469+0000 7f2aeea00640 1 -- 172.21.15.12:0/1112853209 <== mon.1 v2:172.21.15.178:3300/0 5 ==== fsmap(e 44) ==== 8382+0+0 (secure 0 0 0) 0x7f2afc0b5930 con 0x55c7082f3ff0 2024-09-11T00:18:45.474 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:45.469+0000 7f2b1265c740 10 client.26803 fetch_fsmap finished waiting for FSMap version 44 2024-09-11T00:18:45.474 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:45.469+0000 7f2b1265c740 10 client.26803 resolve_mds: validated gid 24674 aka daemon mds.b 2024-09-11T00:18:45.474 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:45.469+0000 7f2b1265c740 20 client.26803 populate_metadata read hostname 'smithi012' 2024-09-11T00:18:45.474 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:45.469+0000 7f2b1265c740 1 --2- 172.21.15.12:0/1112853209 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55c708310270 0x55c7083126b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:18:45.474 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:45.469+0000 7f2b1265c740 4 client.26803 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T00:18:45.474 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:45.469+0000 7f2b1265c740 1 -- 172.21.15.12:0/1112853209 --> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c70826d1e0 con 0x55c708310270 2024-09-11T00:18:45.474 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:45.469+0000 7f2b07e00640 1 --2- 172.21.15.12:0/1112853209 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55c708310270 0x55c7083126b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:18:45.475 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:45.470+0000 7f2b07e00640 1 --2- 172.21.15.12:0/1112853209 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55c708310270 0x55c7083126b0 crc :-1 s=READY pgs=178 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=35376e3a3f6e864c server_cookie=cbe4f2a3985b2f65 in_seq=0 out_seq=0 2024-09-11T00:18:45.475 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:45.470+0000 7f2aeea00640 10 client.26803 ms_handle_connect on v2:172.21.15.165:6838/501312984 2024-09-11T00:18:45.475 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:45.470+0000 7f2aeea00640 1 -- 172.21.15.12:0/1112853209 <== mds.0 v2:172.21.15.165:6838/501312984 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55c70826d1e0 con 0x55c708310270 2024-09-11T00:18:45.475 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:45.470+0000 7f2aeea00640 10 client.26803 handle_command_reply: tid=0 2024-09-11T00:18:45.475 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:45.470+0000 7f2b1265c740 1 client.26803 shutdown 2024-09-11T00:18:45.475 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:45.471+0000 7f2b1265c740 1 -- 172.21.15.12:0/1112853209 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c7082f3ff0 msgr2=0x55c7082fd8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:18:45.476 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:45.471+0000 7f2b1265c740 1 --2- 172.21.15.12:0/1112853209 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c7082f3ff0 0x55c7082fd8d0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x55c7082f7170 tx=0x7f2afc0ac570 comp rx=0 tx=0).stop 2024-09-11T00:18:45.476 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:45.471+0000 7f2b1265c740 1 -- 172.21.15.12:0/1112853209 shutdown_connections 2024-09-11T00:18:45.476 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:45.471+0000 7f2b1265c740 1 --2- 172.21.15.12:0/1112853209 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55c708310270 0x55c7083126b0 crc :-1 s=READY pgs=178 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:18:45.476 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:45.471+0000 7f2b1265c740 1 --2- 172.21.15.12:0/1112853209 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c7083008a0 0x55c7082fe350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:18:45.476 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:45.471+0000 7f2b1265c740 1 --2- 172.21.15.12:0/1112853209 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c7082f4910 0x55c7082fde10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:18:45.476 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:45.471+0000 7f2b1265c740 1 --2- 172.21.15.12:0/1112853209 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c7082f3ff0 0x55c7082fd8d0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:18:45.476 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:45.471+0000 7f2b1265c740 1 -- 172.21.15.12:0/1112853209 >> 172.21.15.12:0/1112853209 conn(0x55c7082f1e20 msgr2=0x55c7082f79c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:18:45.476 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:45.471+0000 7f2aeea00640 0 client.26803 ms_handle_reset on v2:172.21.15.165:6838/501312984 2024-09-11T00:18:45.476 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:45.471+0000 7f2b1265c740 1 -- 172.21.15.12:0/1112853209 shutdown_connections 2024-09-11T00:18:45.476 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:45.471+0000 7f2b1265c740 1 -- 172.21.15.12:0/1112853209 wait complete. 2024-09-11T00:18:45.476 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:45.471+0000 7f2b1265c740 20 client.26803 trim_cache size 0 max 16384 2024-09-11T00:18:45.492 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:45.489+0000 7f2b1265c740 1 Processor -- start 2024-09-11T00:18:45.492 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:45.489+0000 7f2b1265c740 1 -- start start 2024-09-11T00:18:45.492 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:45.490+0000 7f2b1265c740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c7082f3ff0 0x55c7082ff570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:18:45.492 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:45.490+0000 7f2b1265c740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c7082f4910 0x55c7082f0fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:18:45.492 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:45.490+0000 7f2b1265c740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c7083008a0 0x55c70830bca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:18:45.492 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:45.490+0000 7f2b1265c740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f2b081b6130 con 0x55c7082f4910 2024-09-11T00:18:45.492 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:45.490+0000 7f2b1265c740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f2b081b7d00 con 0x55c7083008a0 2024-09-11T00:18:45.492 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:45.490+0000 7f2b1265c740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f2b081b6830 con 0x55c7082f3ff0 2024-09-11T00:18:45.493 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:45.490+0000 7f2b07e00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c7082f4910 0x55c7082f0fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:18:45.493 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:45.490+0000 7f2b0da00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c7083008a0 0x55c70830bca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:18:45.493 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:45.490+0000 7f2b07e00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c7082f4910 0x55c7082f0fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:49226/0 (socket says 172.21.15.12:49226) 2024-09-11T00:18:45.493 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:45.490+0000 7f2b07e00640 1 -- 172.21.15.12:0/4217114874 learned_addr learned my addr 172.21.15.12:0/4217114874 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:18:45.493 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:45.490+0000 7f2b0d000640 1 --2- 172.21.15.12:0/4217114874 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c7082f3ff0 0x55c7082ff570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:18:45.494 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:45.490+0000 7f2b0da00640 1 -- 172.21.15.12:0/4217114874 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c7082f3ff0 msgr2=0x55c7082ff570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:18:45.494 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:45.490+0000 7f2b0da00640 1 --2- 172.21.15.12:0/4217114874 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c7082f3ff0 0x55c7082ff570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:18:45.494 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:45.491+0000 7f2b0da00640 1 -- 172.21.15.12:0/4217114874 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c7082f4910 msgr2=0x55c7082f0fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:18:45.494 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:45.491+0000 7f2b0da00640 1 --2- 172.21.15.12:0/4217114874 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c7082f4910 0x55c7082f0fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:18:45.494 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:45.491+0000 7f2b0da00640 1 -- 172.21.15.12:0/4217114874 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c7082f4740 con 0x55c7083008a0 2024-09-11T00:18:45.494 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:45.491+0000 7f2b0da00640 1 --2- 172.21.15.12:0/4217114874 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c7083008a0 0x55c70830bca0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f2b00005e30 tx=0x7f2b00005d80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=659560e9cdc73070 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:18:45.494 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:45.491+0000 7f2af9600640 1 -- 172.21.15.12:0/4217114874 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b00031450 con 0x55c7083008a0 2024-09-11T00:18:45.494 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:45.491+0000 7f2af9600640 1 -- 172.21.15.12:0/4217114874 <== mon.2 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2b00036050 con 0x55c7083008a0 2024-09-11T00:18:45.494 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:45.491+0000 7f2af9600640 1 -- 172.21.15.12:0/4217114874 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b0002fe90 con 0x55c7083008a0 2024-09-11T00:18:45.496 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:45.492+0000 7f2b1265c740 1 -- 172.21.15.12:0/4217114874 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c7083008a0 msgr2=0x55c70830bca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:18:45.497 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:45.492+0000 7f2b1265c740 1 --2- 172.21.15.12:0/4217114874 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c7083008a0 0x55c70830bca0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f2b00005e30 tx=0x7f2b00005d80 comp rx=0 tx=0).stop 2024-09-11T00:18:45.497 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:45.492+0000 7f2b1265c740 1 -- 172.21.15.12:0/4217114874 shutdown_connections 2024-09-11T00:18:45.497 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:45.492+0000 7f2b1265c740 1 --2- 172.21.15.12:0/4217114874 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c7083008a0 0x55c70830bca0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:18:45.497 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:45.492+0000 7f2b1265c740 1 --2- 172.21.15.12:0/4217114874 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c7082f4910 0x55c7082f0fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:18:45.497 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:45.492+0000 7f2b1265c740 1 --2- 172.21.15.12:0/4217114874 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c7082f3ff0 0x55c7082ff570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:18:45.497 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:45.492+0000 7f2b1265c740 1 -- 172.21.15.12:0/4217114874 >> 172.21.15.12:0/4217114874 conn(0x55c7082f1e20 msgr2=0x55c70830b590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:18:45.497 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:45.492+0000 7f2b1265c740 1 -- 172.21.15.12:0/4217114874 shutdown_connections 2024-09-11T00:18:45.497 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:45.492+0000 7f2b1265c740 1 -- 172.21.15.12:0/4217114874 wait complete. 2024-09-11T00:18:45.497 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:45.492+0000 7f2b1265c740 1 -- start start 2024-09-11T00:18:45.497 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:45.493+0000 7f2b1265c740 1 Processor -- start 2024-09-11T00:18:45.497 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:45.493+0000 7f2b1265c740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c7082f3ff0 0x55c708308e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:18:45.497 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:45.493+0000 7f2b1265c740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c7082f4910 0x55c7083053c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:18:45.497 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:45.493+0000 7f2b1265c740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c7083008a0 0x55c7083059f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:18:45.498 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:45.493+0000 7f2b1265c740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f2b081b7630 con 0x55c7083008a0 2024-09-11T00:18:45.498 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:45.493+0000 7f2b1265c740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f2b081b6f00 con 0x55c7082f4910 2024-09-11T00:18:45.498 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:45.493+0000 7f2b1265c740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f2b081b4110 con 0x55c7082f3ff0 2024-09-11T00:18:45.498 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:45.493+0000 7f2b1265c740 10 client.0 _ll_register_callbacks cb 0x7f2afc00cf20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:18:45.498 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:45.493+0000 7f2b0d000640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c7082f3ff0 0x55c708308e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:18:45.498 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:45.493+0000 7f2b07e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c7082f4910 0x55c7083053c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:18:45.498 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:45.493+0000 7f2b0da00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c7083008a0 0x55c7083059f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:18:45.498 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:45.493+0000 7f2b0d000640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c7082f3ff0 0x55c708308e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.12:47448/0 (socket says 172.21.15.12:47448) 2024-09-11T00:18:45.498 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:45.493+0000 7f2b07e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c7082f4910 0x55c7083053c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.12:41048/0 (socket says 172.21.15.12:41048) 2024-09-11T00:18:45.498 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:45.493+0000 7f2b0d000640 1 -- 172.21.15.12:0/171402836 learned_addr learned my addr 172.21.15.12:0/171402836 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:18:45.498 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:45.494+0000 7f2b0d000640 1 -- 172.21.15.12:0/171402836 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c7082f4910 msgr2=0x55c7083053c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:18:45.498 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:45.494+0000 7f2b0d000640 1 --2- 172.21.15.12:0/171402836 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c7082f4910 0x55c7083053c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:18:45.498 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:45.494+0000 7f2b0d000640 1 -- 172.21.15.12:0/171402836 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c7083008a0 msgr2=0x55c7083059f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:18:45.498 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:45.494+0000 7f2b0d000640 1 --2- 172.21.15.12:0/171402836 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c7083008a0 0x55c7083059f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:18:45.499 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:45.494+0000 7f2b0d000640 1 -- 172.21.15.12:0/171402836 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c70837d190 con 0x55c7082f3ff0 2024-09-11T00:18:45.499 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:45.494+0000 7f2b07e00640 1 --2- 172.21.15.12:0/171402836 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c7082f4910 0x55c7083053c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:18:45.499 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:45.494+0000 7f2b0da00640 1 --2- 172.21.15.12:0/171402836 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c7083008a0 0x55c7083059f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:18:45.499 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:45.494+0000 7f2b0d000640 1 --2- 172.21.15.12:0/171402836 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c7082f3ff0 0x55c708308e90 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f2afc0e6380 tx=0x7f2afc0048f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ad22ee91ec306afd server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:18:45.499 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:45.494+0000 7f2aeea00640 10 client.0 ms_handle_connect on v2:172.21.15.178:3300/0 2024-09-11T00:18:45.499 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:45.494+0000 7f2aeea00640 1 -- 172.21.15.12:0/171402836 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2afc0a8640 con 0x55c7082f3ff0 2024-09-11T00:18:45.499 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:45.495+0000 7f2b1265c740 1 -- 172.21.15.12:0/171402836 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2b00036050 con 0x55c7082f3ff0 2024-09-11T00:18:45.499 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:45.495+0000 7f2aeea00640 1 -- 172.21.15.12:0/171402836 <== mon.1 v2:172.21.15.178:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9208+0+0 (secure 0 0 0) 0x7f2afc0aba00 con 0x55c7082f3ff0 2024-09-11T00:18:45.500 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:45.495+0000 7f2aeea00640 1 -- 172.21.15.12:0/171402836 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f2afc0b5ab0 con 0x55c7082f3ff0 2024-09-11T00:18:45.500 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:45.495+0000 7f2b1265c740 10 client.26809 fetch_fsmap learned FSMap version 44 2024-09-11T00:18:45.500 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:45.495+0000 7f2b1265c740 1 -- 172.21.15.12:0/171402836 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x55c70837e320 con 0x55c7082f3ff0 2024-09-11T00:18:45.500 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:45.495+0000 7f2aeea00640 1 -- 172.21.15.12:0/171402836 <== mon.1 v2:172.21.15.178:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2b00036050 con 0x55c7082f3ff0 2024-09-11T00:18:45.500 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:45.495+0000 7f2aeea00640 1 -- 172.21.15.12:0/171402836 <== mon.1 v2:172.21.15.178:3300/0 5 ==== fsmap(e 44) ==== 8382+0+0 (secure 0 0 0) 0x7f2afc0adad0 con 0x55c7082f3ff0 2024-09-11T00:18:45.500 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:45.495+0000 7f2b1265c740 10 client.26809 fetch_fsmap finished waiting for FSMap version 44 2024-09-11T00:18:45.500 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:45.495+0000 7f2b1265c740 10 client.26809 resolve_mds: validated gid 24674 aka daemon mds.b 2024-09-11T00:18:45.500 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:45.495+0000 7f2b1265c740 20 client.26809 populate_metadata read hostname 'smithi012' 2024-09-11T00:18:45.501 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:45.495+0000 7f2b1265c740 1 --2- 172.21.15.12:0/171402836 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55c70837ea70 0x55c708380e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:18:45.501 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:45.495+0000 7f2b1265c740 4 client.26809 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "d9d39e24", "release": true, "await": true}] 2024-09-11T00:18:45.501 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:45.495+0000 7f2b1265c740 1 -- 172.21.15.12:0/171402836 --> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] -- command(tid 0: {"prefix": "quiesce db", "set_id": "d9d39e24", "release": true, "await": true}) -- 0x55c708268c90 con 0x55c70837ea70 2024-09-11T00:18:45.501 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:45.496+0000 7f2b07e00640 1 --2- 172.21.15.12:0/171402836 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55c70837ea70 0x55c708380e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:18:45.501 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:45.496+0000 7f2b07e00640 1 --2- 172.21.15.12:0/171402836 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55c70837ea70 0x55c708380e90 crc :-1 s=READY pgs=179 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7362647b33ddfccd server_cookie=689eb178f2b1eb89 in_seq=0 out_seq=0 2024-09-11T00:18:45.501 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:45.496+0000 7f2aeea00640 10 client.26809 ms_handle_connect on v2:172.21.15.165:6838/501312984 2024-09-11T00:18:45.506 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:45.504+0000 7f2aeea00640 1 -- 172.21.15.12:0/171402836 <== mds.0 v2:172.21.15.165:6838/501312984 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55c708268c90 con 0x55c70837ea70 2024-09-11T00:18:45.506 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:45.504+0000 7f2aeea00640 10 client.26809 handle_command_reply: tid=0 2024-09-11T00:18:45.506 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:45.504+0000 7f2b1265c740 1 client.26809 shutdown 2024-09-11T00:18:45.506 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:45.504+0000 7f2b1265c740 1 -- 172.21.15.12:0/171402836 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c7082f3ff0 msgr2=0x55c708308e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:18:45.506 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:45.504+0000 7f2b1265c740 1 --2- 172.21.15.12:0/171402836 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c7082f3ff0 0x55c708308e90 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f2afc0e6380 tx=0x7f2afc0048f0 comp rx=0 tx=0).stop 2024-09-11T00:18:45.506 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:45.504+0000 7f2b1265c740 1 -- 172.21.15.12:0/171402836 shutdown_connections 2024-09-11T00:18:45.506 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:45.504+0000 7f2b1265c740 1 --2- 172.21.15.12:0/171402836 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55c70837ea70 0x55c708380e90 crc :-1 s=READY pgs=179 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:18:45.507 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:45.504+0000 7f2b1265c740 1 --2- 172.21.15.12:0/171402836 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c7083008a0 0x55c7083059f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:18:45.507 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:45.504+0000 7f2b1265c740 1 --2- 172.21.15.12:0/171402836 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c7082f4910 0x55c7083053c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:18:45.507 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:45.504+0000 7f2b1265c740 1 --2- 172.21.15.12:0/171402836 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c7082f3ff0 0x55c708308e90 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:18:45.507 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:45.504+0000 7f2aeea00640 0 client.26809 ms_handle_reset on v2:172.21.15.165:6838/501312984 2024-09-11T00:18:45.507 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:45.504+0000 7f2b1265c740 1 -- 172.21.15.12:0/171402836 >> 172.21.15.12:0/171402836 conn(0x55c7082f1e20 msgr2=0x55c70830bdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:18:45.507 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:45.504+0000 7f2b1265c740 1 -- 172.21.15.12:0/171402836 shutdown_connections 2024-09-11T00:18:45.507 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:45.504+0000 7f2b1265c740 1 -- 172.21.15.12:0/171402836 wait complete. 2024-09-11T00:18:45.507 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:45.504+0000 7f2b1265c740 20 client.26809 trim_cache size 0 max 16384 2024-09-11T00:18:45.508 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-09-11T00:18:45.508 INFO:teuthology.orchestra.run.smithi012.stdout: "epoch": 44, 2024-09-11T00:18:45.508 INFO:teuthology.orchestra.run.smithi012.stdout: "leader": 24674, 2024-09-11T00:18:45.508 INFO:teuthology.orchestra.run.smithi012.stdout: "set_version": 36, 2024-09-11T00:18:45.508 INFO:teuthology.orchestra.run.smithi012.stdout: "sets": { 2024-09-11T00:18:45.508 INFO:teuthology.orchestra.run.smithi012.stdout: "d9d39e24": { 2024-09-11T00:18:45.508 INFO:teuthology.orchestra.run.smithi012.stdout: "version": 36, 2024-09-11T00:18:45.508 INFO:teuthology.orchestra.run.smithi012.stdout: "age_ref": 0.0, 2024-09-11T00:18:45.509 INFO:teuthology.orchestra.run.smithi012.stdout: "state": { 2024-09-11T00:18:45.509 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "RELEASED", 2024-09-11T00:18:45.509 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 0.0 2024-09-11T00:18:45.509 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:18:45.509 INFO:teuthology.orchestra.run.smithi012.stdout: "timeout": 90.0, 2024-09-11T00:18:45.509 INFO:teuthology.orchestra.run.smithi012.stdout: "expiration": 145.4, 2024-09-11T00:18:45.509 INFO:teuthology.orchestra.run.smithi012.stdout: "members": { 2024-09-11T00:18:45.509 INFO:teuthology.orchestra.run.smithi012.stdout: "file:/": { 2024-09-11T00:18:45.509 INFO:teuthology.orchestra.run.smithi012.stdout: "excluded": false, 2024-09-11T00:18:45.509 INFO:teuthology.orchestra.run.smithi012.stdout: "state": { 2024-09-11T00:18:45.509 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "RELEASED", 2024-09-11T00:18:45.509 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 0.0 2024-09-11T00:18:45.509 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:18:45.510 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:18:45.510 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:18:45.510 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:18:45.510 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:18:45.510 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-09-11T00:18:45.510 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:45.505+0000 7f2b0fa00640 1 -- 172.21.15.12:0/3888418095 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f2af00b52e0 msgr2=0x7f2af00b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:18:45.510 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:45.505+0000 7f2b0fa00640 1 --2- 172.21.15.12:0/3888418095 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f2af00b52e0 0x7f2af00b77a0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f2b00003e90 tx=0x7f2b0003e040 comp rx=0 tx=0).stop 2024-09-11T00:18:45.510 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:45.505+0000 7f2b0fa00640 1 -- 172.21.15.12:0/3888418095 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2b081093d0 msgr2=0x7f2b0819f850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:18:45.511 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:45.505+0000 7f2b0fa00640 1 --2- 172.21.15.12:0/3888418095 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2b081093d0 0x7f2b0819f850 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f2afc00e9f0 tx=0x7f2afc00edf0 comp rx=0 tx=0).stop 2024-09-11T00:18:45.511 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:45.506+0000 7f2b0fa00640 1 -- 172.21.15.12:0/3888418095 shutdown_connections 2024-09-11T00:18:45.511 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:45.506+0000 7f2b0fa00640 1 --2- 172.21.15.12:0/3888418095 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f2af00b52e0 0x7f2af00b77a0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:18:45.511 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:45.506+0000 7f2b0fa00640 1 --2- 172.21.15.12:0/3888418095 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2b0810f760 0x7f2b081a5130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:18:45.511 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:45.506+0000 7f2b0fa00640 1 --2- 172.21.15.12:0/3888418095 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2b0810ede0 0x7f2b0819fd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:18:45.511 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:45.506+0000 7f2b0fa00640 1 --2- 172.21.15.12:0/3888418095 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2b081093d0 0x7f2b0819f850 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:18:45.511 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:45.506+0000 7f2b0fa00640 1 -- 172.21.15.12:0/3888418095 >> 172.21.15.12:0/3888418095 conn(0x7f2b08078a40 msgr2=0x7f2b0807ae30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:18:45.511 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:45.506+0000 7f2b0fa00640 1 -- 172.21.15.12:0/3888418095 shutdown_connections 2024-09-11T00:18:45.511 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:45.506+0000 7f2b0fa00640 1 -- 172.21.15.12:0/3888418095 wait complete. 2024-09-11T00:18:45.696 INFO:tasks.workunit.client.0.smithi012.stdout: 1 479732 3.99 MB/sec execute 522 sec latency 25095.850 ms 2024-09-11T00:18:45.782 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:18:45 smithi165 ceph-mon[35356]: pgmap v749: 129 pgs: 129 active+clean; 114 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 698 KiB/s wr, 49 op/s 2024-09-11T00:18:45.831 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:18:45 smithi178 ceph-mon[35919]: pgmap v749: 129 pgs: 129 active+clean; 114 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 698 KiB/s wr, 49 op/s 2024-09-11T00:18:46.462 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'd9d39e24', total seconds elapsed: 29.8 2024-09-11T00:18:46.463 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 25.4 2024-09-11T00:18:46.463 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid ff3ad74a-6fcf-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T00:18:46.693 INFO:tasks.workunit.client.0.smithi012.stdout: 1 480390 3.99 MB/sec execute 523 sec latency 126.862 ms 2024-09-11T00:18:46.877 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:18:47.464 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:47.462+0000 7f5571200640 1 -- 172.21.15.12:0/1292582720 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f556c102040 msgr2=0x7f556c10ed90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:18:47.465 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:47.462+0000 7f5571200640 1 --2- 172.21.15.12:0/1292582720 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f556c102040 0x7f556c10ed90 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f5558009900 tx=0x7f555802a480 comp rx=0 tx=0).stop 2024-09-11T00:18:47.465 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:47.462+0000 7f5571200640 1 -- 172.21.15.12:0/1292582720 shutdown_connections 2024-09-11T00:18:47.465 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:47.462+0000 7f5571200640 1 --2- 172.21.15.12:0/1292582720 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f556c10f2d0 0x7f556c1116c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:18:47.465 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:47.462+0000 7f5571200640 1 --2- 172.21.15.12:0/1292582720 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f556c102040 0x7f556c10ed90 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:18:47.465 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:47.462+0000 7f5571200640 1 --2- 172.21.15.12:0/1292582720 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f556c101720 0x7f556c101b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:18:47.465 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:47.462+0000 7f5571200640 1 -- 172.21.15.12:0/1292582720 >> 172.21.15.12:0/1292582720 conn(0x7f556c0fd5b0 msgr2=0x7f556c0ff9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:18:47.465 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:47.462+0000 7f5571200640 1 -- 172.21.15.12:0/1292582720 shutdown_connections 2024-09-11T00:18:47.465 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:47.462+0000 7f5571200640 1 -- 172.21.15.12:0/1292582720 wait complete. 2024-09-11T00:18:47.466 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:47.463+0000 7f5571200640 1 Processor -- start 2024-09-11T00:18:47.466 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:47.463+0000 7f5571200640 1 -- start start 2024-09-11T00:18:47.466 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:47.463+0000 7f5571200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f556c101720 0x7f556c19b210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:18:47.466 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:47.463+0000 7f5571200640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f556c102040 0x7f556c19b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:18:47.466 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:47.463+0000 7f5571200640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f556c10f2d0 0x7f556c1a0af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:18:47.466 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:47.463+0000 7f5571200640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f556c114de0 con 0x7f556c10f2d0 2024-09-11T00:18:47.466 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:47.463+0000 7f5571200640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f556c114c60 con 0x7f556c101720 2024-09-11T00:18:47.466 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:47.463+0000 7f5571200640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f556c114f60 con 0x7f556c102040 2024-09-11T00:18:47.466 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:47.464+0000 7f556b400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f556c101720 0x7f556c19b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:18:47.467 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:47.464+0000 7f556aa00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f556c102040 0x7f556c19b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:18:47.467 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:47.464+0000 7f556be00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f556c10f2d0 0x7f556c1a0af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:18:47.467 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:47.464+0000 7f556b400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f556c101720 0x7f556c19b210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.12:41066/0 (socket says 172.21.15.12:41066) 2024-09-11T00:18:47.467 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:47.464+0000 7f556b400640 1 -- 172.21.15.12:0/210714917 learned_addr learned my addr 172.21.15.12:0/210714917 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:18:47.468 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:47.464+0000 7f556b400640 1 -- 172.21.15.12:0/210714917 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f556c102040 msgr2=0x7f556c19b750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:18:47.468 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:47.464+0000 7f556b400640 1 --2- 172.21.15.12:0/210714917 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f556c102040 0x7f556c19b750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:18:47.468 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:47.464+0000 7f556b400640 1 -- 172.21.15.12:0/210714917 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f556c10f2d0 msgr2=0x7f556c1a0af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:18:47.468 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:47.464+0000 7f556b400640 1 --2- 172.21.15.12:0/210714917 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f556c10f2d0 0x7f556c1a0af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:18:47.468 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:47.464+0000 7f556b400640 1 -- 172.21.15.12:0/210714917 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f556c1a12c0 con 0x7f556c101720 2024-09-11T00:18:47.468 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:47.464+0000 7f556be00640 1 --2- 172.21.15.12:0/210714917 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f556c10f2d0 0x7f556c1a0af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:18:47.468 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:47.465+0000 7f556b400640 1 --2- 172.21.15.12:0/210714917 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f556c101720 0x7f556c19b210 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f555c00e9f0 tx=0x7f555c00edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:18:47.468 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:47.465+0000 7f5563e00640 1 -- 172.21.15.12:0/210714917 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f555c00cd80 con 0x7f556c101720 2024-09-11T00:18:47.468 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:47.465+0000 7f5563e00640 1 -- 172.21.15.12:0/210714917 <== mon.2 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f555c0054d0 con 0x7f556c101720 2024-09-11T00:18:47.469 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:47.465+0000 7f5563e00640 1 -- 172.21.15.12:0/210714917 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f555c010690 con 0x7f556c101720 2024-09-11T00:18:47.469 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:47.465+0000 7f5571200640 1 -- 172.21.15.12:0/210714917 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f556c1a15b0 con 0x7f556c101720 2024-09-11T00:18:47.469 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:47.465+0000 7f5571200640 1 -- 172.21.15.12:0/210714917 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f556c1a7c70 con 0x7f556c101720 2024-09-11T00:18:47.469 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:47.466+0000 7f5571200640 1 -- 172.21.15.12:0/210714917 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f556c101b00 con 0x7f556c101720 2024-09-11T00:18:47.471 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:47.468+0000 7f5563e00640 1 -- 172.21.15.12:0/210714917 <== mon.2 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7f555c004e90 con 0x7f556c101720 2024-09-11T00:18:47.472 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:47.470+0000 7f5563e00640 1 --2- 172.21.15.12:0/210714917 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f55500b5410 0x7f55500b78d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:18:47.472 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:47.470+0000 7f5563e00640 1 -- 172.21.15.12:0/210714917 <== mon.2 v2:172.21.15.165:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9208+0+0 (secure 0 0 0) 0x7f555c0e6390 con 0x7f556c101720 2024-09-11T00:18:47.472 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:47.470+0000 7f556aa00640 1 --2- 172.21.15.12:0/210714917 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f55500b5410 0x7f55500b78d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:18:47.473 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:47.471+0000 7f556aa00640 1 --2- 172.21.15.12:0/210714917 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f55500b5410 0x7f55500b78d0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f5558031440 tx=0x7f5558005b50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:18:47.476 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:47.474+0000 7f5563e00640 1 -- 172.21.15.12:0/210714917 <== mon.2 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f555c0ace20 con 0x7f556c101720 2024-09-11T00:18:47.693 INFO:tasks.workunit.client.0.smithi012.stdout: 1 481290 3.99 MB/sec execute 524 sec latency 369.148 ms 2024-09-11T00:18:47.743 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:47.740+0000 7f5571200640 1 -- 172.21.15.12:0/210714917 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f556c1a7fe0 con 0x7f556c101720 2024-09-11T00:18:47.743 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:18:47 smithi012 ceph-mon[30333]: pgmap v750: 129 pgs: 129 active+clean; 114 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 987 KiB/s wr, 74 op/s 2024-09-11T00:18:47.744 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:47.741+0000 7f5563e00640 1 -- 172.21.15.12:0/210714917 <== mon.2 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 44 v44) ==== 94+0+11969 (secure 0 0 0) 0x7f555c0a6e20 con 0x7f556c101720 2024-09-11T00:18:47.744 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 44 2024-09-11T00:18:47.744 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-11T00:18:47.744 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":44,"btime":"2024-09-11T00:18:17:022132+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14577,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6835/447171591","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":447171591},{"type":"v1","addr":"172.21.15.12:6835","nonce":447171591}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14589,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/901486040","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":901486040},{"type":"v1","addr":"172.21.15.12:6837","nonce":901486040}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":44,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T00:04:04.739881+0000","modified":"2024-09-11T00:18:16.957829+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24674,"mds_1":24643,"mds_2":24638,"mds_3":24637,"mds_4":24614},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14631":{"gid":14631,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.12:6841/3069919437","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6840","nonce":3069919437},{"type":"v1","addr":"172.21.15.12:6841","nonce":3069919437}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24562":{"gid":24562,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6833/1141636112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":1141636112},{"type":"v1","addr":"172.21.15.178:6833","nonce":1141636112}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24581":{"gid":24581,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6833/3111980970","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6832","nonce":3111980970},{"type":"v1","addr":"172.21.15.165:6833","nonce":3111980970}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24607":{"gid":24607,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6835/4084868287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":4084868287},{"type":"v1","addr":"172.21.15.178:6835","nonce":4084868287}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24614":{"gid":24614,"name":"e","rank":4,"incarnation":26,"state":"up:active","state_seq":21,"addr":"172.21.15.165:6835/2425395274","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":2425395274},{"type":"v1","addr":"172.21.15.165:6835","nonce":2425395274}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24637":{"gid":24637,"name":"i","rank":3,"incarnation":24,"state":"up:active","state_seq":18,"addr":"172.21.15.178:6839/70628097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6838","nonce":70628097},{"type":"v1","addr":"172.21.15.178:6839","nonce":70628097}]},"join_fscid":-1,"export_targets":[0],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24638":{"gid":24638,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":18,"addr":"172.21.15.165:6837/6215088","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6836","nonce":6215088},{"type":"v1","addr":"172.21.15.165:6837","nonce":6215088}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24643":{"gid":24643,"name":"g","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.12:6839/3124093009","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6838","nonce":3124093009},{"type":"v1","addr":"172.21.15.12:6839","nonce":3124093009}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24674":{"gid":24674,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.165:6839/501312984","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6838","nonce":501312984},{"type":"v1","addr":"172.21.15.165:6839","nonce":501312984}]},"join_fscid":-1,"export_targets":[1,3],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25672":{"gid":25672,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6837/308601741","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":308601741},{"type":"v1","addr":"172.21.15.178:6837","nonce":308601741}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24674,"qdb_cluster":[24674,24643,24638,24637,24614]},"id":1}]} 2024-09-11T00:18:47.747 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:47.744+0000 7f5571200640 1 -- 172.21.15.12:0/210714917 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f55500b5410 msgr2=0x7f55500b78d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:18:47.747 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:47.744+0000 7f5571200640 1 --2- 172.21.15.12:0/210714917 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f55500b5410 0x7f55500b78d0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f5558031440 tx=0x7f5558005b50 comp rx=0 tx=0).stop 2024-09-11T00:18:47.747 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:47.744+0000 7f5571200640 1 -- 172.21.15.12:0/210714917 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f556c101720 msgr2=0x7f556c19b210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:18:47.747 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:47.744+0000 7f5571200640 1 --2- 172.21.15.12:0/210714917 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f556c101720 0x7f556c19b210 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f555c00e9f0 tx=0x7f555c00edf0 comp rx=0 tx=0).stop 2024-09-11T00:18:47.747 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:47.744+0000 7f5571200640 1 -- 172.21.15.12:0/210714917 shutdown_connections 2024-09-11T00:18:47.747 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:47.744+0000 7f5571200640 1 --2- 172.21.15.12:0/210714917 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f55500b5410 0x7f55500b78d0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:18:47.747 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:47.744+0000 7f5571200640 1 --2- 172.21.15.12:0/210714917 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f556c10f2d0 0x7f556c1a0af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:18:47.747 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:47.744+0000 7f5571200640 1 --2- 172.21.15.12:0/210714917 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f556c102040 0x7f556c19b750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:18:47.747 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:47.744+0000 7f5571200640 1 --2- 172.21.15.12:0/210714917 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f556c101720 0x7f556c19b210 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:18:47.747 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:47.744+0000 7f5571200640 1 -- 172.21.15.12:0/210714917 >> 172.21.15.12:0/210714917 conn(0x7f556c0fd5b0 msgr2=0x7f556c111500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:18:47.747 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:47.744+0000 7f5571200640 1 -- 172.21.15.12:0/210714917 shutdown_connections 2024-09-11T00:18:47.748 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:47.744+0000 7f5571200640 1 -- 172.21.15.12:0/210714917 wait complete. 2024-09-11T00:18:47.782 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:18:47 smithi165 ceph-mon[35356]: pgmap v750: 129 pgs: 129 active+clean; 114 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 987 KiB/s wr, 74 op/s 2024-09-11T00:18:47.831 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:18:47 smithi178 ceph-mon[35919]: pgmap v750: 129 pgs: 129 active+clean; 114 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 987 KiB/s wr, 74 op/s 2024-09-11T00:18:48.566 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24674 quiesce db / --timeout 90 --expiration 145.4 --await-for 45.0' 2024-09-11T00:18:48.566 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid ff3ad74a-6fcf-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24674 quiesce db / --timeout 90 --expiration 145.4 --await-for 45.0 2024-09-11T00:18:48.693 INFO:tasks.workunit.client.0.smithi012.stdout: 1 482325 3.99 MB/sec execute 525 sec latency 516.443 ms 2024-09-11T00:18:48.782 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:18:48 smithi165 ceph-mon[35356]: from='client.? 172.21.15.12:0/210714917' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T00:18:48.813 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:18:48 smithi012 ceph-mon[30333]: from='client.? 172.21.15.12:0/210714917' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T00:18:48.831 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:18:48 smithi178 ceph-mon[35919]: from='client.? 172.21.15.12:0/210714917' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T00:18:49.008 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:18:49.693 INFO:tasks.workunit.client.0.smithi012.stdout: 1 483323 4.00 MB/sec execute 526 sec latency 751.719 ms 2024-09-11T00:18:49.710 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:18:49 smithi012 ceph-mon[30333]: pgmap v751: 129 pgs: 129 active+clean; 114 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 793 KiB/s wr, 59 op/s 2024-09-11T00:18:49.733 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:49.634+0000 7fc01fe00640 1 -- 172.21.15.12:0/715595176 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc020103b00 msgr2=0x7fc020103ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:18:49.733 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:49.634+0000 7fc01fe00640 1 --2- 172.21.15.12:0/715595176 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc020103b00 0x7fc020103ee0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fc010009900 tx=0x7fc01002a3f0 comp rx=0 tx=0).stop 2024-09-11T00:18:49.734 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:49.635+0000 7fc01fe00640 1 -- 172.21.15.12:0/715595176 shutdown_connections 2024-09-11T00:18:49.734 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:49.635+0000 7fc01fe00640 1 --2- 172.21.15.12:0/715595176 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc02010cf60 0x7fc02010f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:18:49.734 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:49.635+0000 7fc01fe00640 1 --2- 172.21.15.12:0/715595176 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc020104420 0x7fc02010c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:18:49.734 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:49.635+0000 7fc01fe00640 1 --2- 172.21.15.12:0/715595176 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc020103b00 0x7fc020103ee0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:18:49.734 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:49.635+0000 7fc01fe00640 1 -- 172.21.15.12:0/715595176 >> 172.21.15.12:0/715595176 conn(0x7fc0200fd560 msgr2=0x7fc0200ff980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:18:49.734 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:49.635+0000 7fc01fe00640 1 -- 172.21.15.12:0/715595176 shutdown_connections 2024-09-11T00:18:49.734 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:49.635+0000 7fc01fe00640 1 -- 172.21.15.12:0/715595176 wait complete. 2024-09-11T00:18:49.734 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:49.636+0000 7fc01fe00640 1 Processor -- start 2024-09-11T00:18:49.734 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:49.636+0000 7fc01fe00640 1 -- start start 2024-09-11T00:18:49.734 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:49.636+0000 7fc01fe00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc020103b00 0x7fc02019b2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:18:49.734 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:49.636+0000 7fc01fe00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc020104420 0x7fc02019b810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:18:49.734 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:49.636+0000 7fc01fe00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc02010cf60 0x7fc0201a0bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:18:49.735 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:49.636+0000 7fc01fe00640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fc020112c20 con 0x7fc02010cf60 2024-09-11T00:18:49.735 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:49.636+0000 7fc01fe00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fc020112aa0 con 0x7fc020103b00 2024-09-11T00:18:49.735 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:49.636+0000 7fc01fe00640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fc020112da0 con 0x7fc020104420 2024-09-11T00:18:49.735 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:49.637+0000 7fc01e000640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc020104420 0x7fc02019b810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:18:49.735 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:49.637+0000 7fc01ea00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc020103b00 0x7fc02019b2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:18:49.735 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:49.637+0000 7fc01e000640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc020104420 0x7fc02019b810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.12:51550/0 (socket says 172.21.15.12:51550) 2024-09-11T00:18:49.735 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:49.637+0000 7fc01e000640 1 -- 172.21.15.12:0/24654977 learned_addr learned my addr 172.21.15.12:0/24654977 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:18:49.735 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:49.637+0000 7fc01f400640 1 --2- 172.21.15.12:0/24654977 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc02010cf60 0x7fc0201a0bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:18:49.735 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:49.637+0000 7fc01e000640 1 -- 172.21.15.12:0/24654977 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc020103b00 msgr2=0x7fc02019b2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:18:49.735 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:49.637+0000 7fc01e000640 1 --2- 172.21.15.12:0/24654977 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc020103b00 0x7fc02019b2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:18:49.735 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:49.637+0000 7fc01e000640 1 -- 172.21.15.12:0/24654977 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc02010cf60 msgr2=0x7fc0201a0bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:18:49.735 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:49.637+0000 7fc01e000640 1 --2- 172.21.15.12:0/24654977 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc02010cf60 0x7fc0201a0bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:18:49.736 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:49.637+0000 7fc01e000640 1 -- 172.21.15.12:0/24654977 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0201a1380 con 0x7fc020104420 2024-09-11T00:18:49.736 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:49.637+0000 7fc01e000640 1 --2- 172.21.15.12:0/24654977 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc020104420 0x7fc02019b810 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fc00c00d740 tx=0x7fc00c00dc00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:18:49.736 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:49.638+0000 7fc017400640 1 -- 172.21.15.12:0/24654977 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc00c017020 con 0x7fc020104420 2024-09-11T00:18:49.736 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:49.638+0000 7fc017400640 1 -- 172.21.15.12:0/24654977 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc00c00de50 con 0x7fc020104420 2024-09-11T00:18:49.736 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:49.638+0000 7fc017400640 1 -- 172.21.15.12:0/24654977 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc00c00f620 con 0x7fc020104420 2024-09-11T00:18:49.736 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:49.638+0000 7fc01fe00640 1 -- 172.21.15.12:0/24654977 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc0201a1670 con 0x7fc020104420 2024-09-11T00:18:49.736 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:49.638+0000 7fc01fe00640 1 -- 172.21.15.12:0/24654977 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0201a7d30 con 0x7fc020104420 2024-09-11T00:18:49.736 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:49.641+0000 7fc028649740 1 Processor -- start 2024-09-11T00:18:49.736 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:49.641+0000 7fc028649740 1 -- start start 2024-09-11T00:18:49.736 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:49.642+0000 7fc028649740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ac9b2aa2f0 0x55ac9b2aa6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:18:49.736 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:49.642+0000 7fc028649740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ac9b2aac10 0x55ac9b2b6660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:18:49.736 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:49.642+0000 7fc028649740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ac9b2b6ba0 0x55ac9b2b8f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:18:49.737 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:49.642+0000 7fc028649740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55ac9b21fca0 con 0x55ac9b2aa2f0 2024-09-11T00:18:49.737 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:49.642+0000 7fc028649740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55ac9b20f0d0 con 0x55ac9b2b6ba0 2024-09-11T00:18:49.737 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:49.642+0000 7fc028649740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55ac9b212ee0 con 0x55ac9b2aac10 2024-09-11T00:18:49.737 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:49.642+0000 7fc01f400640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ac9b2aa2f0 0x55ac9b2aa6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:18:49.737 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:49.642+0000 7fc01ea00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ac9b2aac10 0x55ac9b2b6660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:18:49.737 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:49.642+0000 7fc01f400640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ac9b2aa2f0 0x55ac9b2aa6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:46084/0 (socket says 172.21.15.12:46084) 2024-09-11T00:18:49.737 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:49.642+0000 7fc017400640 1 -- 172.21.15.12:0/24654977 <== mon.1 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7fc00c010460 con 0x7fc020104420 2024-09-11T00:18:49.738 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:49.642+0000 7fc01f400640 1 -- 172.21.15.12:0/4205736223 learned_addr learned my addr 172.21.15.12:0/4205736223 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:18:49.738 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:49.643+0000 7fc01e000640 1 --2- 172.21.15.12:0/4205736223 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ac9b2b6ba0 0x55ac9b2b8f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:18:49.738 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:49.643+0000 7fc01f400640 1 -- 172.21.15.12:0/4205736223 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ac9b2aac10 msgr2=0x55ac9b2b6660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:18:49.738 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:49.643+0000 7fc01f400640 1 --2- 172.21.15.12:0/4205736223 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ac9b2aac10 0x55ac9b2b6660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:18:49.738 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:49.643+0000 7fc01f400640 1 -- 172.21.15.12:0/4205736223 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ac9b2b6ba0 msgr2=0x55ac9b2b8f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:18:49.738 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:49.643+0000 7fc01f400640 1 --2- 172.21.15.12:0/4205736223 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ac9b2b6ba0 0x55ac9b2b8f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:18:49.738 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:49.643+0000 7fc01f400640 1 -- 172.21.15.12:0/4205736223 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ac9b1c1400 con 0x55ac9b2aa2f0 2024-09-11T00:18:49.738 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:49.643+0000 7fc01f400640 1 --2- 172.21.15.12:0/4205736223 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ac9b2aa2f0 0x55ac9b2aa6d0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7fc018009f50 tx=0x7fc0180118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4b49ed70981abe1 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:18:49.738 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:49.643+0000 7fc00be00640 1 -- 172.21.15.12:0/4205736223 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc018004d20 con 0x55ac9b2aa2f0 2024-09-11T00:18:49.738 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:49.643+0000 7fc00be00640 1 -- 172.21.15.12:0/4205736223 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc018004050 con 0x55ac9b2aa2f0 2024-09-11T00:18:49.738 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:49.643+0000 7fc00be00640 1 -- 172.21.15.12:0/4205736223 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc018005280 con 0x55ac9b2aa2f0 2024-09-11T00:18:49.738 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:49.644+0000 7fc017400640 1 --2- 172.21.15.12:0/24654977 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fc0040b5410 0x7fc0040b78d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:18:49.739 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:49.644+0000 7fc028649740 1 -- 172.21.15.12:0/4205736223 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ac9b2aa2f0 msgr2=0x55ac9b2aa6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:18:49.739 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:49.644+0000 7fc028649740 1 --2- 172.21.15.12:0/4205736223 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ac9b2aa2f0 0x55ac9b2aa6d0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7fc018009f50 tx=0x7fc0180118d0 comp rx=0 tx=0).stop 2024-09-11T00:18:49.739 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:49.644+0000 7fc017400640 1 -- 172.21.15.12:0/24654977 <== mon.1 v2:172.21.15.178:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9208+0+0 (secure 0 0 0) 0x7fc00c0e5f30 con 0x7fc020104420 2024-09-11T00:18:49.739 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:49.644+0000 7fc028649740 1 -- 172.21.15.12:0/4205736223 shutdown_connections 2024-09-11T00:18:49.739 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:49.644+0000 7fc028649740 1 --2- 172.21.15.12:0/4205736223 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ac9b2b6ba0 0x55ac9b2b8f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:18:49.739 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:49.644+0000 7fc028649740 1 --2- 172.21.15.12:0/4205736223 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ac9b2aac10 0x55ac9b2b6660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:18:49.739 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:49.644+0000 7fc028649740 1 --2- 172.21.15.12:0/4205736223 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ac9b2aa2f0 0x55ac9b2aa6d0 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:18:49.739 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:49.644+0000 7fc028649740 1 -- 172.21.15.12:0/4205736223 >> 172.21.15.12:0/4205736223 conn(0x55ac9b2a8120 msgr2=0x55ac9b2a92c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:18:49.739 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:49.644+0000 7fc01ea00640 1 --2- 172.21.15.12:0/24654977 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fc0040b5410 0x7fc0040b78d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:18:49.739 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:49.645+0000 7fc028649740 1 -- 172.21.15.12:0/4205736223 shutdown_connections 2024-09-11T00:18:49.739 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:49.645+0000 7fc028649740 1 -- 172.21.15.12:0/4205736223 wait complete. 2024-09-11T00:18:49.739 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:49.645+0000 7fc01ea00640 1 --2- 172.21.15.12:0/24654977 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fc0040b5410 0x7fc0040b78d0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fc010005ab0 tx=0x7fc010005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:18:49.740 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:49.645+0000 7fc028649740 1 -- start start 2024-09-11T00:18:49.740 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:49.645+0000 7fc028649740 1 Processor -- start 2024-09-11T00:18:49.740 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:49.646+0000 7fc028649740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ac9b2aa2f0 0x55ac9b2b6440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:18:49.740 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:49.646+0000 7fc028649740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ac9b2aac10 0x55ac9b2b3890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:18:49.740 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:49.646+0000 7fc028649740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ac9b2b6ba0 0x55ac9b2b3e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:18:49.740 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:49.646+0000 7fc028649740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55ac9b0707a0 con 0x55ac9b2aa2f0 2024-09-11T00:18:49.740 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:49.646+0000 7fc028649740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55ac9b1b4a10 con 0x55ac9b2aac10 2024-09-11T00:18:49.740 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:49.646+0000 7fc028649740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55ac9b2b9820 con 0x55ac9b2b6ba0 2024-09-11T00:18:49.740 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:49.646+0000 7fc028649740 10 client.0 _ll_register_callbacks cb 0x55ac9b1c4be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:18:49.740 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:49.646+0000 7fc01ea00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ac9b2aa2f0 0x55ac9b2b6440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:18:49.741 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:49.646+0000 7fc01e000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ac9b2aac10 0x55ac9b2b3890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:18:49.741 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:49.646+0000 7fc01e000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ac9b2aac10 0x55ac9b2b3890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.12:36576/0 (socket says 172.21.15.12:36576) 2024-09-11T00:18:49.741 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:49.646+0000 7fc01e000640 1 -- 172.21.15.12:0/3774314094 learned_addr learned my addr 172.21.15.12:0/3774314094 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:18:49.741 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:49.646+0000 7fc01f400640 1 --2- 172.21.15.12:0/3774314094 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ac9b2b6ba0 0x55ac9b2b3e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:18:49.741 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:49.647+0000 7fc01e000640 1 -- 172.21.15.12:0/3774314094 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ac9b2b6ba0 msgr2=0x55ac9b2b3e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:18:49.741 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:49.647+0000 7fc01e000640 1 --2- 172.21.15.12:0/3774314094 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ac9b2b6ba0 0x55ac9b2b3e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:18:49.741 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:49.647+0000 7fc01e000640 1 -- 172.21.15.12:0/3774314094 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ac9b2aa2f0 msgr2=0x55ac9b2b6440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:18:49.741 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:49.647+0000 7fc01e000640 1 --2- 172.21.15.12:0/3774314094 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ac9b2aa2f0 0x55ac9b2b6440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:18:49.741 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:49.647+0000 7fc01e000640 1 -- 172.21.15.12:0/3774314094 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ac9af7cb90 con 0x55ac9b2aac10 2024-09-11T00:18:49.741 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:49.647+0000 7fc01e000640 1 --2- 172.21.15.12:0/3774314094 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ac9b2aac10 0x55ac9b2b3890 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fc00c005fd0 tx=0x7fc00c0a8970 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2d60f62c3c005e90 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:18:49.741 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:49.647+0000 7fc008c00640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-09-11T00:18:49.741 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:49.647+0000 7fc008c00640 1 -- 172.21.15.12:0/3774314094 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc00c0abc70 con 0x55ac9b2aac10 2024-09-11T00:18:49.742 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:49.647+0000 7fc028649740 1 -- 172.21.15.12:0/3774314094 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc018004050 con 0x55ac9b2aac10 2024-09-11T00:18:49.742 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:49.647+0000 7fc008c00640 1 -- 172.21.15.12:0/3774314094 <== mon.2 v2:172.21.15.165:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9208+0+0 (secure 0 0 0) 0x7fc00c0cf0f0 con 0x55ac9b2aac10 2024-09-11T00:18:49.742 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:49.647+0000 7fc008c00640 1 -- 172.21.15.12:0/3774314094 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7fc00c0aba90 con 0x55ac9b2aac10 2024-09-11T00:18:49.742 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:49.647+0000 7fc028649740 10 client.26852 fetch_fsmap learned FSMap version 44 2024-09-11T00:18:49.742 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:49.647+0000 7fc028649740 1 -- 172.21.15.12:0/3774314094 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x55ac9b2c5160 con 0x55ac9b2aac10 2024-09-11T00:18:49.742 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:49.648+0000 7fc008c00640 1 -- 172.21.15.12:0/3774314094 <== mon.2 v2:172.21.15.165:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc018004050 con 0x55ac9b2aac10 2024-09-11T00:18:49.742 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:49.648+0000 7fc008c00640 1 -- 172.21.15.12:0/3774314094 <== mon.2 v2:172.21.15.165:3300/0 5 ==== fsmap(e 44) ==== 8382+0+0 (secure 0 0 0) 0x7fc00c0e3b20 con 0x55ac9b2aac10 2024-09-11T00:18:49.742 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:49.648+0000 7fc028649740 10 client.26852 fetch_fsmap finished waiting for FSMap version 44 2024-09-11T00:18:49.742 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:49.648+0000 7fc028649740 10 client.26852 resolve_mds: validated gid 24674 aka daemon mds.b 2024-09-11T00:18:49.742 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:49.648+0000 7fc028649740 20 client.26852 populate_metadata read hostname 'smithi012' 2024-09-11T00:18:49.742 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:49.648+0000 7fc028649740 1 --2- 172.21.15.12:0/3774314094 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55ac9b2c64b0 0x55ac9b2c88a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:18:49.742 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:49.648+0000 7fc028649740 4 client.26852 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T00:18:49.742 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:49.648+0000 7fc028649740 1 -- 172.21.15.12:0/3774314094 --> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ac9b221bf0 con 0x55ac9b2c64b0 2024-09-11T00:18:49.743 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:49.648+0000 7fc01ea00640 1 --2- 172.21.15.12:0/3774314094 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55ac9b2c64b0 0x55ac9b2c88a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:18:49.743 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:49.649+0000 7fc01ea00640 1 --2- 172.21.15.12:0/3774314094 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55ac9b2c64b0 0x55ac9b2c88a0 crc :-1 s=READY pgs=180 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c27740d73caa1b1c server_cookie=4c1910f252ec5241 in_seq=0 out_seq=0 2024-09-11T00:18:49.743 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:49.649+0000 7fc008c00640 10 client.26852 ms_handle_connect on v2:172.21.15.165:6838/501312984 2024-09-11T00:18:49.743 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:49.650+0000 7fc008c00640 1 -- 172.21.15.12:0/3774314094 <== mds.0 v2:172.21.15.165:6838/501312984 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55ac9b221bf0 con 0x55ac9b2c64b0 2024-09-11T00:18:49.743 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:49.650+0000 7fc008c00640 10 client.26852 handle_command_reply: tid=0 2024-09-11T00:18:49.743 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:49.650+0000 7fc028649740 1 client.26852 shutdown 2024-09-11T00:18:49.743 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:49.650+0000 7fc028649740 1 -- 172.21.15.12:0/3774314094 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ac9b2aac10 msgr2=0x55ac9b2b3890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:18:49.743 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:49.650+0000 7fc028649740 1 --2- 172.21.15.12:0/3774314094 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ac9b2aac10 0x55ac9b2b3890 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fc00c005fd0 tx=0x7fc00c0a8970 comp rx=0 tx=0).stop 2024-09-11T00:18:49.743 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:49.650+0000 7fc028649740 1 -- 172.21.15.12:0/3774314094 shutdown_connections 2024-09-11T00:18:49.743 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:49.650+0000 7fc028649740 1 --2- 172.21.15.12:0/3774314094 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55ac9b2c64b0 0x55ac9b2c88a0 crc :-1 s=READY pgs=180 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:18:49.744 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:49.650+0000 7fc028649740 1 --2- 172.21.15.12:0/3774314094 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ac9b2b6ba0 0x55ac9b2b3e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:18:49.744 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:49.650+0000 7fc028649740 1 --2- 172.21.15.12:0/3774314094 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ac9b2aac10 0x55ac9b2b3890 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:18:49.744 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:49.650+0000 7fc028649740 1 --2- 172.21.15.12:0/3774314094 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ac9b2aa2f0 0x55ac9b2b6440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:18:49.744 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:49.650+0000 7fc028649740 1 -- 172.21.15.12:0/3774314094 >> 172.21.15.12:0/3774314094 conn(0x55ac9b2a8120 msgr2=0x55ac9b2ad4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:18:49.744 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:49.651+0000 7fc008c00640 0 client.26852 ms_handle_reset on v2:172.21.15.165:6838/501312984 2024-09-11T00:18:49.744 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:49.651+0000 7fc028649740 1 -- 172.21.15.12:0/3774314094 shutdown_connections 2024-09-11T00:18:49.744 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:49.651+0000 7fc028649740 1 -- 172.21.15.12:0/3774314094 wait complete. 2024-09-11T00:18:49.744 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:49.651+0000 7fc028649740 20 client.26852 trim_cache size 0 max 16384 2024-09-11T00:18:49.744 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:49.681+0000 7fc028649740 1 Processor -- start 2024-09-11T00:18:49.744 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:49.681+0000 7fc028649740 1 -- start start 2024-09-11T00:18:49.744 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:49.681+0000 7fc028649740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ac9b2aa2f0 0x55ac9b2c2a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:18:49.744 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:49.681+0000 7fc028649740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ac9b2aac10 0x55ac9b2b2780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:18:49.744 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:49.681+0000 7fc028649740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ac9b2b6ba0 0x55ac9b2a71f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:18:49.745 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:49.681+0000 7fc028649740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fc0201b1ca0 con 0x55ac9b2aa2f0 2024-09-11T00:18:49.745 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:49.681+0000 7fc028649740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fc0201b3870 con 0x55ac9b2b6ba0 2024-09-11T00:18:49.745 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:49.681+0000 7fc028649740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fc0201b23a0 con 0x55ac9b2aac10 2024-09-11T00:18:49.745 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:49.682+0000 7fc01e000640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ac9b2aac10 0x55ac9b2b2780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:18:49.745 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:49.682+0000 7fc01e000640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ac9b2aac10 0x55ac9b2b2780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.12:51574/0 (socket says 172.21.15.12:51574) 2024-09-11T00:18:49.745 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:49.682+0000 7fc01e000640 1 -- 172.21.15.12:0/1707913000 learned_addr learned my addr 172.21.15.12:0/1707913000 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:18:49.745 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:49.682+0000 7fc01f400640 1 --2- 172.21.15.12:0/1707913000 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ac9b2b6ba0 0x55ac9b2a71f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:18:49.745 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:49.682+0000 7fc01ea00640 1 --2- 172.21.15.12:0/1707913000 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ac9b2aa2f0 0x55ac9b2c2a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:18:49.745 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:49.682+0000 7fc01f400640 1 -- 172.21.15.12:0/1707913000 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ac9b2aac10 msgr2=0x55ac9b2b2780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:18:49.745 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:49.682+0000 7fc01f400640 1 --2- 172.21.15.12:0/1707913000 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ac9b2aac10 0x55ac9b2b2780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:18:49.745 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:49.682+0000 7fc01f400640 1 -- 172.21.15.12:0/1707913000 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ac9b2aa2f0 msgr2=0x55ac9b2c2a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:18:49.745 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:49.682+0000 7fc01f400640 1 --2- 172.21.15.12:0/1707913000 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ac9b2aa2f0 0x55ac9b2c2a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:18:49.746 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:49.682+0000 7fc01f400640 1 -- 172.21.15.12:0/1707913000 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ac9b2c8f20 con 0x55ac9b2b6ba0 2024-09-11T00:18:49.746 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:49.682+0000 7fc01ea00640 1 --2- 172.21.15.12:0/1707913000 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ac9b2aa2f0 0x55ac9b2c2a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T00:18:49.746 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:49.683+0000 7fc01e000640 1 --2- 172.21.15.12:0/1707913000 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ac9b2aac10 0x55ac9b2b2780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T00:18:49.746 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:49.684+0000 7fc01f400640 1 --2- 172.21.15.12:0/1707913000 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ac9b2b6ba0 0x55ac9b2a71f0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fc018012fd0 tx=0x7fc018005bd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8b6c3d2c30998d46 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:18:49.746 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:49.685+0000 7fc00b400640 1 -- 172.21.15.12:0/1707913000 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc018003d30 con 0x55ac9b2b6ba0 2024-09-11T00:18:49.746 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:49.685+0000 7fc00b400640 1 -- 172.21.15.12:0/1707913000 <== mon.2 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc018004e40 con 0x55ac9b2b6ba0 2024-09-11T00:18:49.746 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:49.685+0000 7fc00b400640 1 -- 172.21.15.12:0/1707913000 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc018005050 con 0x55ac9b2b6ba0 2024-09-11T00:18:49.746 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:49.686+0000 7fc028649740 1 -- 172.21.15.12:0/1707913000 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ac9b2b6ba0 msgr2=0x55ac9b2a71f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:18:49.746 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:49.686+0000 7fc028649740 1 --2- 172.21.15.12:0/1707913000 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ac9b2b6ba0 0x55ac9b2a71f0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fc018012fd0 tx=0x7fc018005bd0 comp rx=0 tx=0).stop 2024-09-11T00:18:49.746 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:49.686+0000 7fc028649740 1 -- 172.21.15.12:0/1707913000 shutdown_connections 2024-09-11T00:18:49.746 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:49.686+0000 7fc028649740 1 --2- 172.21.15.12:0/1707913000 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ac9b2b6ba0 0x55ac9b2a71f0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:18:49.746 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:49.686+0000 7fc028649740 1 --2- 172.21.15.12:0/1707913000 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ac9b2aac10 0x55ac9b2b2780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:18:49.746 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:49.686+0000 7fc028649740 1 --2- 172.21.15.12:0/1707913000 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ac9b2aa2f0 0x55ac9b2c2a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:18:49.747 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:49.686+0000 7fc028649740 1 -- 172.21.15.12:0/1707913000 >> 172.21.15.12:0/1707913000 conn(0x55ac9b2a8120 msgr2=0x55ac9b2a6ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:18:49.747 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:49.686+0000 7fc028649740 1 -- 172.21.15.12:0/1707913000 shutdown_connections 2024-09-11T00:18:49.747 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:49.686+0000 7fc028649740 1 -- 172.21.15.12:0/1707913000 wait complete. 2024-09-11T00:18:49.747 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:49.687+0000 7fc028649740 1 -- start start 2024-09-11T00:18:49.747 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:49.688+0000 7fc028649740 1 Processor -- start 2024-09-11T00:18:49.747 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:49.688+0000 7fc028649740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ac9b2aa2f0 0x55ac9b2beab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:18:49.747 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:49.689+0000 7fc028649740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ac9b2aac10 0x55ac9b2beff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:18:49.747 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:49.689+0000 7fc028649740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ac9b2b6ba0 0x55ac9b2afb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:18:49.747 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:49.689+0000 7fc028649740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fc0201b31a0 con 0x55ac9b2aa2f0 2024-09-11T00:18:49.747 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:49.689+0000 7fc028649740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fc0201b2a70 con 0x55ac9b2aac10 2024-09-11T00:18:49.747 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:49.689+0000 7fc028649740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fc0201afc80 con 0x55ac9b2b6ba0 2024-09-11T00:18:49.748 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:49.689+0000 7fc028649740 10 client.0 _ll_register_callbacks cb 0x7fc00c010b70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:18:49.748 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:49.689+0000 7fc01e000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ac9b2aac10 0x55ac9b2beff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:18:49.748 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:49.689+0000 7fc01e000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ac9b2aac10 0x55ac9b2beff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.12:36592/0 (socket says 172.21.15.12:36592) 2024-09-11T00:18:49.748 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:49.689+0000 7fc01e000640 1 -- 172.21.15.12:0/2817511328 learned_addr learned my addr 172.21.15.12:0/2817511328 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:18:49.748 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:49.689+0000 7fc01ea00640 1 --2- 172.21.15.12:0/2817511328 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ac9b2aa2f0 0x55ac9b2beab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:18:49.748 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:49.690+0000 7fc01f400640 1 --2- 172.21.15.12:0/2817511328 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ac9b2b6ba0 0x55ac9b2afb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:18:49.748 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:49.690+0000 7fc01e000640 1 -- 172.21.15.12:0/2817511328 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ac9b2b6ba0 msgr2=0x55ac9b2afb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:18:49.748 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:49.690+0000 7fc01e000640 1 --2- 172.21.15.12:0/2817511328 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ac9b2b6ba0 0x55ac9b2afb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:18:49.748 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:49.690+0000 7fc01e000640 1 -- 172.21.15.12:0/2817511328 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ac9b2aa2f0 msgr2=0x55ac9b2beab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:18:49.748 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:49.690+0000 7fc01e000640 1 --2- 172.21.15.12:0/2817511328 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ac9b2aa2f0 0x55ac9b2beab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:18:49.748 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:49.690+0000 7fc01e000640 1 -- 172.21.15.12:0/2817511328 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ac9b2bf630 con 0x55ac9b2aac10 2024-09-11T00:18:49.749 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:49.690+0000 7fc01f400640 1 --2- 172.21.15.12:0/2817511328 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ac9b2b6ba0 0x55ac9b2afb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T00:18:49.749 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:49.690+0000 7fc01ea00640 1 --2- 172.21.15.12:0/2817511328 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ac9b2aa2f0 0x55ac9b2beab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T00:18:49.749 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:49.690+0000 7fc01e000640 1 --2- 172.21.15.12:0/2817511328 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ac9b2aac10 0x55ac9b2beff0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fc00c0a88f0 tx=0x7fc00c010a30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=60491d613f3b6b server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:18:49.749 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:49.690+0000 7fc008c00640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-09-11T00:18:49.749 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:49.691+0000 7fc008c00640 1 -- 172.21.15.12:0/2817511328 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc00c0abc70 con 0x55ac9b2aac10 2024-09-11T00:18:49.749 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:49.691+0000 7fc028649740 1 -- 172.21.15.12:0/2817511328 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55ac9b21d6a0 con 0x55ac9b2aac10 2024-09-11T00:18:49.749 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:49.691+0000 7fc008c00640 1 -- 172.21.15.12:0/2817511328 <== mon.2 v2:172.21.15.165:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9208+0+0 (secure 0 0 0) 0x7fc00c0d30b0 con 0x55ac9b2aac10 2024-09-11T00:18:49.750 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:49.691+0000 7fc008c00640 1 -- 172.21.15.12:0/2817511328 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7fc00c0e9640 con 0x55ac9b2aac10 2024-09-11T00:18:49.750 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:49.691+0000 7fc028649740 10 client.26858 fetch_fsmap learned FSMap version 44 2024-09-11T00:18:49.750 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:49.691+0000 7fc028649740 1 -- 172.21.15.12:0/2817511328 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x55ac9b3347d0 con 0x55ac9b2aac10 2024-09-11T00:18:49.750 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:49.692+0000 7fc008c00640 1 -- 172.21.15.12:0/2817511328 <== mon.2 v2:172.21.15.165:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55ac9b21d6a0 con 0x55ac9b2aac10 2024-09-11T00:18:49.750 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:49.692+0000 7fc008c00640 1 -- 172.21.15.12:0/2817511328 <== mon.2 v2:172.21.15.165:3300/0 5 ==== fsmap(e 44) ==== 8382+0+0 (secure 0 0 0) 0x7fc00c0e90b0 con 0x55ac9b2aac10 2024-09-11T00:18:49.750 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:49.692+0000 7fc028649740 10 client.26858 fetch_fsmap finished waiting for FSMap version 44 2024-09-11T00:18:49.750 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:49.692+0000 7fc028649740 10 client.26858 resolve_mds: validated gid 24674 aka daemon mds.b 2024-09-11T00:18:49.751 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:49.692+0000 7fc028649740 20 client.26858 populate_metadata read hostname 'smithi012' 2024-09-11T00:18:49.751 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:49.692+0000 7fc028649740 1 --2- 172.21.15.12:0/2817511328 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55ac9b334fd0 0x55ac9b3373f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:18:49.751 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:49.692+0000 7fc028649740 4 client.26858 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 145.4, "await_for": 45.0}] 2024-09-11T00:18:49.751 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:49.692+0000 7fc028649740 1 -- 172.21.15.12:0/2817511328 --> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 145.4, "await_for": 45.0}) -- 0x55ac9b21aad0 con 0x55ac9b334fd0 2024-09-11T00:18:49.751 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:49.692+0000 7fc01ea00640 1 --2- 172.21.15.12:0/2817511328 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55ac9b334fd0 0x55ac9b3373f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:18:49.751 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:49.693+0000 7fc01ea00640 1 --2- 172.21.15.12:0/2817511328 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55ac9b334fd0 0x55ac9b3373f0 crc :-1 s=READY pgs=181 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e6de48337b36f856 server_cookie=60eb248e57aee6b4 in_seq=0 out_seq=0 2024-09-11T00:18:49.751 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:49.693+0000 7fc008c00640 10 client.26858 ms_handle_connect on v2:172.21.15.165:6838/501312984 2024-09-11T00:18:49.782 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:18:49 smithi165 ceph-mon[35356]: pgmap v751: 129 pgs: 129 active+clean; 114 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 793 KiB/s wr, 59 op/s 2024-09-11T00:18:49.831 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:18:49 smithi178 ceph-mon[35919]: pgmap v751: 129 pgs: 129 active+clean; 114 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 793 KiB/s wr, 59 op/s 2024-09-11T00:18:49.903 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:49.900+0000 7fc008c00640 1 -- 172.21.15.12:0/2817511328 <== mds.0 v2:172.21.15.165:6838/501312984 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55ac9b21aad0 con 0x55ac9b334fd0 2024-09-11T00:18:49.903 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:49.900+0000 7fc008c00640 10 client.26858 handle_command_reply: tid=0 2024-09-11T00:18:49.903 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:49.900+0000 7fc028649740 1 client.26858 shutdown 2024-09-11T00:18:49.903 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:49.900+0000 7fc028649740 1 -- 172.21.15.12:0/2817511328 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ac9b2aac10 msgr2=0x55ac9b2beff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:18:49.903 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:49.900+0000 7fc028649740 1 --2- 172.21.15.12:0/2817511328 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ac9b2aac10 0x55ac9b2beff0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fc00c0a88f0 tx=0x7fc00c010a30 comp rx=0 tx=0).stop 2024-09-11T00:18:49.903 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:49.900+0000 7fc028649740 1 -- 172.21.15.12:0/2817511328 shutdown_connections 2024-09-11T00:18:49.903 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:49.900+0000 7fc028649740 1 --2- 172.21.15.12:0/2817511328 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55ac9b334fd0 0x55ac9b3373f0 crc :-1 s=READY pgs=181 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:18:49.903 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:49.900+0000 7fc028649740 1 --2- 172.21.15.12:0/2817511328 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ac9b2b6ba0 0x55ac9b2afb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:18:49.904 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:49.900+0000 7fc028649740 1 --2- 172.21.15.12:0/2817511328 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ac9b2aac10 0x55ac9b2beff0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:18:49.904 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:49.900+0000 7fc028649740 1 --2- 172.21.15.12:0/2817511328 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ac9b2aa2f0 0x55ac9b2beab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:18:49.904 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:49.900+0000 7fc008c00640 0 client.26858 ms_handle_reset on v2:172.21.15.165:6838/501312984 2024-09-11T00:18:49.904 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:49.900+0000 7fc028649740 1 -- 172.21.15.12:0/2817511328 >> 172.21.15.12:0/2817511328 conn(0x55ac9b2a8120 msgr2=0x55ac9b2ad220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:18:49.904 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:49.900+0000 7fc028649740 1 -- 172.21.15.12:0/2817511328 shutdown_connections 2024-09-11T00:18:49.904 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:49.900+0000 7fc028649740 1 -- 172.21.15.12:0/2817511328 wait complete. 2024-09-11T00:18:49.904 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:49.901+0000 7fc028649740 20 client.26858 trim_cache size 0 max 16384 2024-09-11T00:18:49.904 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-09-11T00:18:49.905 INFO:teuthology.orchestra.run.smithi012.stdout: "epoch": 44, 2024-09-11T00:18:49.905 INFO:teuthology.orchestra.run.smithi012.stdout: "leader": 24674, 2024-09-11T00:18:49.905 INFO:teuthology.orchestra.run.smithi012.stdout: "set_version": 38, 2024-09-11T00:18:49.905 INFO:teuthology.orchestra.run.smithi012.stdout: "sets": { 2024-09-11T00:18:49.905 INFO:teuthology.orchestra.run.smithi012.stdout: "bd2e38fb": { 2024-09-11T00:18:49.905 INFO:teuthology.orchestra.run.smithi012.stdout: "version": 38, 2024-09-11T00:18:49.905 INFO:teuthology.orchestra.run.smithi012.stdout: "age_ref": 0.0, 2024-09-11T00:18:49.905 INFO:teuthology.orchestra.run.smithi012.stdout: "state": { 2024-09-11T00:18:49.905 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "QUIESCED", 2024-09-11T00:18:49.905 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 0.0 2024-09-11T00:18:49.905 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:18:49.906 INFO:teuthology.orchestra.run.smithi012.stdout: "timeout": 90.0, 2024-09-11T00:18:49.906 INFO:teuthology.orchestra.run.smithi012.stdout: "expiration": 145.4, 2024-09-11T00:18:49.906 INFO:teuthology.orchestra.run.smithi012.stdout: "members": { 2024-09-11T00:18:49.906 INFO:teuthology.orchestra.run.smithi012.stdout: "file:/": { 2024-09-11T00:18:49.906 INFO:teuthology.orchestra.run.smithi012.stdout: "excluded": false, 2024-09-11T00:18:49.906 INFO:teuthology.orchestra.run.smithi012.stdout: "state": { 2024-09-11T00:18:49.906 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "QUIESCED", 2024-09-11T00:18:49.906 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 0.0 2024-09-11T00:18:49.906 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:18:49.906 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:18:49.906 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:18:49.907 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:18:49.907 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:18:49.907 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-09-11T00:18:49.907 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:49.901+0000 7fc01fe00640 1 -- 172.21.15.12:0/24654977 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fc0040b5410 msgr2=0x7fc0040b78d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:18:49.907 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:49.901+0000 7fc01fe00640 1 --2- 172.21.15.12:0/24654977 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fc0040b5410 0x7fc0040b78d0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fc010005ab0 tx=0x7fc010005a20 comp rx=0 tx=0).stop 2024-09-11T00:18:49.908 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:49.901+0000 7fc01fe00640 1 -- 172.21.15.12:0/24654977 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc020104420 msgr2=0x7fc02019b810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:18:49.908 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:49.901+0000 7fc01fe00640 1 --2- 172.21.15.12:0/24654977 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc020104420 0x7fc02019b810 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fc00c00d740 tx=0x7fc00c00dc00 comp rx=0 tx=0).stop 2024-09-11T00:18:49.908 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:49.902+0000 7fc01fe00640 1 -- 172.21.15.12:0/24654977 shutdown_connections 2024-09-11T00:18:49.908 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:49.902+0000 7fc01fe00640 1 --2- 172.21.15.12:0/24654977 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fc0040b5410 0x7fc0040b78d0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:18:49.908 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:49.902+0000 7fc01fe00640 1 --2- 172.21.15.12:0/24654977 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc02010cf60 0x7fc0201a0bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:18:49.908 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:49.902+0000 7fc01fe00640 1 --2- 172.21.15.12:0/24654977 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc020104420 0x7fc02019b810 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:18:49.908 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:49.902+0000 7fc01fe00640 1 --2- 172.21.15.12:0/24654977 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc020103b00 0x7fc02019b2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:18:49.908 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:49.902+0000 7fc01fe00640 1 -- 172.21.15.12:0/24654977 >> 172.21.15.12:0/24654977 conn(0x7fc0200fd560 msgr2=0x7fc0200fd940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:18:49.908 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:49.902+0000 7fc01fe00640 1 -- 172.21.15.12:0/24654977 shutdown_connections 2024-09-11T00:18:49.908 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:18:49.902+0000 7fc01fe00640 1 -- 172.21.15.12:0/24654977 wait complete. 2024-09-11T00:18:50.658 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'bd2e38fb', quiesce took 4.2 seconds. Will release after: 21.2 2024-09-11T00:18:50.693 INFO:tasks.workunit.client.0.smithi012.stdout: 1 483326 3.99 MB/sec execute 527 sec latency 979.968 ms 2024-09-11T00:18:51.694 INFO:tasks.workunit.client.0.smithi012.stdout: 1 483326 3.98 MB/sec execute 528 sec latency 1980.110 ms 2024-09-11T00:18:51.782 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:18:51 smithi165 ceph-mon[35356]: pgmap v752: 129 pgs: 129 active+clean; 112 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 28 MiB/s rd, 1.1 MiB/s wr, 139 op/s 2024-09-11T00:18:51.813 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:18:51 smithi012 ceph-mon[30333]: pgmap v752: 129 pgs: 129 active+clean; 112 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 28 MiB/s rd, 1.1 MiB/s wr, 139 op/s 2024-09-11T00:18:51.831 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:18:51 smithi178 ceph-mon[35919]: pgmap v752: 129 pgs: 129 active+clean; 112 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 28 MiB/s rd, 1.1 MiB/s wr, 139 op/s 2024-09-11T00:18:52.694 INFO:tasks.workunit.client.0.smithi012.stdout: 1 483326 3.98 MB/sec execute 529 sec latency 2980.258 ms 2024-09-11T00:18:52.907 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:52.905+0000 7f8315800640 1 -- 172.21.15.12:0/742743240 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f83101066a0 msgr2=0x7f831010d020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:18:52.907 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:52.905+0000 7f8315800640 1 --2- 172.21.15.12:0/742743240 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f83101066a0 0x7f831010d020 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f8308009c10 tx=0x7f830802a6e0 comp rx=0 tx=0).stop 2024-09-11T00:18:52.908 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:52.905+0000 7f8315800640 1 -- 172.21.15.12:0/742743240 shutdown_connections 2024-09-11T00:18:52.908 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:52.905+0000 7f8315800640 1 --2- 172.21.15.12:0/742743240 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f83101066a0 0x7f831010d020 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:18:52.908 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:52.905+0000 7f8315800640 1 --2- 172.21.15.12:0/742743240 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8310105ce0 0x7f8310106160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:18:52.908 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:52.905+0000 7f8315800640 1 --2- 172.21.15.12:0/742743240 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8310104b90 0x7f8310104f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:18:52.908 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:52.905+0000 7f8315800640 1 -- 172.21.15.12:0/742743240 >> 172.21.15.12:0/742743240 conn(0x7f8310100470 msgr2=0x7f8310102890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:18:52.908 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:52.905+0000 7f8315800640 1 -- 172.21.15.12:0/742743240 shutdown_connections 2024-09-11T00:18:52.908 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:52.905+0000 7f8315800640 1 -- 172.21.15.12:0/742743240 wait complete. 2024-09-11T00:18:52.910 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:52.906+0000 7f8315800640 1 Processor -- start 2024-09-11T00:18:52.910 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:52.906+0000 7f8315800640 1 -- start start 2024-09-11T00:18:52.910 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:52.906+0000 7f8315800640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8310104b90 0x7f831019af60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:18:52.910 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:52.906+0000 7f8315800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8310105ce0 0x7f831019b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:18:52.911 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:52.906+0000 7f8315800640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f83101066a0 0x7f831019c1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:18:52.911 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:52.906+0000 7f8315800640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f83101107e0 con 0x7f83101066a0 2024-09-11T00:18:52.911 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:52.906+0000 7f8315800640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f8310110660 con 0x7f8310105ce0 2024-09-11T00:18:52.911 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:52.906+0000 7f8315800640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f8310110960 con 0x7f8310104b90 2024-09-11T00:18:52.911 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:52.907+0000 7f830f400640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f83101066a0 0x7f831019c1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:18:52.911 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:52.907+0000 7f830f400640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f83101066a0 0x7f831019c1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:46152/0 (socket says 172.21.15.12:46152) 2024-09-11T00:18:52.911 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:52.907+0000 7f830f400640 1 -- 172.21.15.12:0/2985767385 learned_addr learned my addr 172.21.15.12:0/2985767385 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:18:52.911 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:52.907+0000 7f830ea00640 1 --2- 172.21.15.12:0/2985767385 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8310104b90 0x7f831019af60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:18:52.911 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:52.907+0000 7f8307e00640 1 --2- 172.21.15.12:0/2985767385 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8310105ce0 0x7f831019b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:18:52.911 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:52.907+0000 7f830f400640 1 -- 172.21.15.12:0/2985767385 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8310104b90 msgr2=0x7f831019af60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:18:52.911 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:52.907+0000 7f830f400640 1 --2- 172.21.15.12:0/2985767385 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8310104b90 0x7f831019af60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:18:52.911 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:52.907+0000 7f830f400640 1 -- 172.21.15.12:0/2985767385 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8310105ce0 msgr2=0x7f831019b5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:18:52.911 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:52.907+0000 7f830f400640 1 --2- 172.21.15.12:0/2985767385 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8310105ce0 0x7f831019b5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:18:52.911 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:52.907+0000 7f830f400640 1 -- 172.21.15.12:0/2985767385 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f831019caa0 con 0x7f83101066a0 2024-09-11T00:18:52.912 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:52.907+0000 7f830f400640 1 --2- 172.21.15.12:0/2985767385 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f83101066a0 0x7f831019c1e0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f830802ac00 tx=0x7f8308030d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:18:52.912 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:52.908+0000 7f8307400640 1 -- 172.21.15.12:0/2985767385 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8308041020 con 0x7f83101066a0 2024-09-11T00:18:52.912 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:52.908+0000 7f8307400640 1 -- 172.21.15.12:0/2985767385 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f83080318f0 con 0x7f83101066a0 2024-09-11T00:18:52.912 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:52.908+0000 7f8315800640 1 -- 172.21.15.12:0/2985767385 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f831019cd30 con 0x7f83101066a0 2024-09-11T00:18:52.912 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:52.908+0000 7f8307400640 1 -- 172.21.15.12:0/2985767385 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8308036040 con 0x7f83101066a0 2024-09-11T00:18:52.912 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:52.908+0000 7f8315800640 1 -- 172.21.15.12:0/2985767385 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83101a3460 con 0x7f83101066a0 2024-09-11T00:18:52.913 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:52.911+0000 7f8318671740 1 Processor -- start 2024-09-11T00:18:52.914 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:52.911+0000 7f8307400640 1 -- 172.21.15.12:0/2985767385 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7f8308003c00 con 0x7f83101066a0 2024-09-11T00:18:52.914 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:52.911+0000 7f8318671740 1 -- start start 2024-09-11T00:18:52.915 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:52.912+0000 7f8318671740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556a63334790 0x556a63336b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:18:52.915 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:52.912+0000 7f8318671740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556a633370c0 0x556a633394b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:18:52.915 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:52.912+0000 7f8318671740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556a63339b00 0x556a6333bfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:18:52.915 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:52.912+0000 7f8318671740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x556a632a76b0 con 0x556a633370c0 2024-09-11T00:18:52.915 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:52.912+0000 7f8318671740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x556a63296ae0 con 0x556a63339b00 2024-09-11T00:18:52.915 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:52.912+0000 7f8318671740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x556a6329a8f0 con 0x556a63334790 2024-09-11T00:18:52.915 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:52.912+0000 7f830ea00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556a633370c0 0x556a633394b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:18:52.915 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:52.912+0000 7f8307e00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556a63334790 0x556a63336b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:18:52.915 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:52.912+0000 7f830ea00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556a633370c0 0x556a633394b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:46158/0 (socket says 172.21.15.12:46158) 2024-09-11T00:18:52.915 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:52.912+0000 7f830ea00640 1 -- 172.21.15.12:0/1656778945 learned_addr learned my addr 172.21.15.12:0/1656778945 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:18:52.916 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:52.912+0000 7f8307e00640 1 --2- 172.21.15.12:0/1656778945 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556a63339b00 0x556a6333bfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:18:52.921 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:52.913+0000 7f830ea00640 1 -- 172.21.15.12:0/1656778945 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556a63334790 msgr2=0x556a63336b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:18:52.921 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:52.913+0000 7f830ea00640 1 --2- 172.21.15.12:0/1656778945 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556a63334790 0x556a63336b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:18:52.921 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:52.913+0000 7f830ea00640 1 -- 172.21.15.12:0/1656778945 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556a63339b00 msgr2=0x556a6333bfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:18:52.921 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:52.913+0000 7f830ea00640 1 --2- 172.21.15.12:0/1656778945 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556a63339b00 0x556a6333bfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:18:52.921 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:52.913+0000 7f830ea00640 1 -- 172.21.15.12:0/1656778945 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556a63263780 con 0x556a633370c0 2024-09-11T00:18:52.921 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:52.913+0000 7f830ea00640 1 --2- 172.21.15.12:0/1656778945 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556a633370c0 0x556a633394b0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f8300009cf0 tx=0x7f8300011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b911801cf3a0aa68 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:18:52.921 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:52.913+0000 7f8307400640 1 --2- 172.21.15.12:0/2985767385 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f82f40b52e0 0x7f82f40b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:18:52.921 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:52.913+0000 7f82ffe00640 1 -- 172.21.15.12:0/1656778945 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8300004230 con 0x556a633370c0 2024-09-11T00:18:52.922 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:52.913+0000 7f82ffe00640 1 -- 172.21.15.12:0/1656778945 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f83000043d0 con 0x556a633370c0 2024-09-11T00:18:52.922 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:52.913+0000 7f82ffe00640 1 -- 172.21.15.12:0/1656778945 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8300019820 con 0x556a633370c0 2024-09-11T00:18:52.922 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:52.913+0000 7f8307400640 1 -- 172.21.15.12:0/2985767385 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9208+0+0 (secure 0 0 0) 0x7f8308106ff0 con 0x7f83101066a0 2024-09-11T00:18:52.922 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:52.913+0000 7f8307e00640 1 --2- 172.21.15.12:0/2985767385 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f82f40b52e0 0x7f82f40b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:18:52.922 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:52.914+0000 7f8318671740 1 -- 172.21.15.12:0/1656778945 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556a633370c0 msgr2=0x556a633394b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:18:52.922 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:52.914+0000 7f8318671740 1 --2- 172.21.15.12:0/1656778945 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556a633370c0 0x556a633394b0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f8300009cf0 tx=0x7f8300011810 comp rx=0 tx=0).stop 2024-09-11T00:18:52.922 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:52.914+0000 7f8307e00640 1 --2- 172.21.15.12:0/2985767385 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f82f40b52e0 0x7f82f40b77a0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f82f8009fd0 tx=0x7f82f8009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:18:52.922 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:52.914+0000 7f8318671740 1 -- 172.21.15.12:0/1656778945 shutdown_connections 2024-09-11T00:18:52.922 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:52.914+0000 7f8318671740 1 --2- 172.21.15.12:0/1656778945 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556a63339b00 0x556a6333bfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:18:52.923 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:52.914+0000 7f8318671740 1 --2- 172.21.15.12:0/1656778945 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556a633370c0 0x556a633394b0 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:18:52.923 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:52.914+0000 7f8318671740 1 --2- 172.21.15.12:0/1656778945 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556a63334790 0x556a63336b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:18:52.923 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:52.914+0000 7f8318671740 1 -- 172.21.15.12:0/1656778945 >> 172.21.15.12:0/1656778945 conn(0x556a6332f8e0 msgr2=0x556a63330670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:18:52.923 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:52.914+0000 7f8318671740 1 -- 172.21.15.12:0/1656778945 shutdown_connections 2024-09-11T00:18:52.923 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:52.914+0000 7f8318671740 1 -- 172.21.15.12:0/1656778945 wait complete. 2024-09-11T00:18:52.923 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:52.915+0000 7f8318671740 1 -- start start 2024-09-11T00:18:52.923 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:52.915+0000 7f8318671740 1 Processor -- start 2024-09-11T00:18:52.923 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:52.916+0000 7f8318671740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556a63334790 0x556a6333ded0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:18:52.923 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:52.916+0000 7f8318671740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556a633370c0 0x556a6333e570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:18:52.923 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:52.916+0000 7f8318671740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556a63339b00 0x556a6333f140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:18:52.923 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:52.916+0000 7f8318671740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x556a631ceab0 con 0x556a633370c0 2024-09-11T00:18:52.923 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:52.916+0000 7f8318671740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x556a6323d570 con 0x556a63334790 2024-09-11T00:18:52.924 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:52.916+0000 7f8318671740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x556a6333cae0 con 0x556a63339b00 2024-09-11T00:18:52.924 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:52.916+0000 7f8318671740 10 client.0 _ll_register_callbacks cb 0x556a6324c100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:18:52.924 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:52.916+0000 7f830ea00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556a63334790 0x556a6333ded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:18:52.924 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:52.916+0000 7f830ea00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556a63334790 0x556a6333ded0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.12:36624/0 (socket says 172.21.15.12:36624) 2024-09-11T00:18:52.924 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:52.916+0000 7f830ea00640 1 -- 172.21.15.12:0/2367118490 learned_addr learned my addr 172.21.15.12:0/2367118490 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:18:52.924 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:52.916+0000 7f830f400640 1 --2- 172.21.15.12:0/2367118490 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556a63339b00 0x556a6333f140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:18:52.924 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:52.916+0000 7f8307e00640 1 --2- 172.21.15.12:0/2367118490 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556a633370c0 0x556a6333e570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:18:52.924 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:52.916+0000 7f830f400640 1 -- 172.21.15.12:0/2367118490 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556a63334790 msgr2=0x556a6333ded0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:18:52.924 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:52.916+0000 7f830f400640 1 --2- 172.21.15.12:0/2367118490 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556a63334790 0x556a6333ded0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:18:52.924 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:52.916+0000 7f830f400640 1 -- 172.21.15.12:0/2367118490 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556a633370c0 msgr2=0x556a6333e570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:18:52.924 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:52.916+0000 7f830f400640 1 --2- 172.21.15.12:0/2367118490 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556a633370c0 0x556a6333e570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:18:52.924 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:52.916+0000 7f830f400640 1 -- 172.21.15.12:0/2367118490 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556a62ff8790 con 0x556a63339b00 2024-09-11T00:18:52.925 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:52.916+0000 7f8307e00640 1 --2- 172.21.15.12:0/2367118490 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556a633370c0 0x556a6333e570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T00:18:52.925 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:52.917+0000 7f830f400640 1 --2- 172.21.15.12:0/2367118490 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556a63339b00 0x556a6333f140 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f83080cd0c0 tx=0x7f83080ccd10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=eed9c2b679997364 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:18:52.925 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:52.917+0000 7f82fcc00640 10 client.0 ms_handle_connect on v2:172.21.15.178:3300/0 2024-09-11T00:18:52.925 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:52.917+0000 7f82fcc00640 1 -- 172.21.15.12:0/2367118490 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83080cced0 con 0x556a63339b00 2024-09-11T00:18:52.925 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:52.917+0000 7f8318671740 1 -- 172.21.15.12:0/2367118490 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f83000043d0 con 0x556a63339b00 2024-09-11T00:18:52.925 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:52.917+0000 7f82fcc00640 1 -- 172.21.15.12:0/2367118490 <== mon.1 v2:172.21.15.178:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9208+0+0 (secure 0 0 0) 0x7f8308104bc0 con 0x556a63339b00 2024-09-11T00:18:52.925 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:52.917+0000 7f82fcc00640 1 -- 172.21.15.12:0/2367118490 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f83080cdac0 con 0x556a63339b00 2024-09-11T00:18:52.925 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:52.917+0000 7f8318671740 10 client.26845 fetch_fsmap learned FSMap version 44 2024-09-11T00:18:52.925 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:52.917+0000 7f8318671740 1 -- 172.21.15.12:0/2367118490 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x556a63071620 con 0x556a63339b00 2024-09-11T00:18:52.925 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:52.917+0000 7f82fcc00640 1 -- 172.21.15.12:0/2367118490 <== mon.1 v2:172.21.15.178:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f83000043d0 con 0x556a63339b00 2024-09-11T00:18:52.925 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:52.918+0000 7f82fcc00640 1 -- 172.21.15.12:0/2367118490 <== mon.1 v2:172.21.15.178:3300/0 5 ==== fsmap(e 44) ==== 8382+0+0 (secure 0 0 0) 0x7f8308004c60 con 0x556a63339b00 2024-09-11T00:18:52.925 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:52.918+0000 7f8318671740 10 client.26845 fetch_fsmap finished waiting for FSMap version 44 2024-09-11T00:18:52.925 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:52.918+0000 7f8318671740 10 client.26845 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T00:18:52.926 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:52.918+0000 7f8318671740 20 client.26845 populate_metadata read hostname 'smithi012' 2024-09-11T00:18:52.926 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:52.918+0000 7f8318671740 1 --2- 172.21.15.12:0/2367118490 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x556a633497a0 0x556a6334bc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:18:52.926 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:52.918+0000 7f8318671740 4 client.26845 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T00:18:52.926 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:52.918+0000 7f8318671740 1 -- 172.21.15.12:0/2367118490 --> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556a632a9600 con 0x556a633497a0 2024-09-11T00:18:52.926 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:52.918+0000 7f830ea00640 1 --2- 172.21.15.12:0/2367118490 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x556a633497a0 0x556a6334bc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:18:52.926 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:52.919+0000 7f830ea00640 1 --2- 172.21.15.12:0/2367118490 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x556a633497a0 0x556a6334bc20 crc :-1 s=READY pgs=182 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ee06338f7a311c45 server_cookie=c06a02f1de29c42a in_seq=0 out_seq=0 2024-09-11T00:18:52.926 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:52.919+0000 7f82fcc00640 10 client.26845 ms_handle_connect on v2:172.21.15.165:6838/501312984 2024-09-11T00:18:52.927 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:52.919+0000 7f82fcc00640 1 -- 172.21.15.12:0/2367118490 <== mds.0 v2:172.21.15.165:6838/501312984 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x556a632a9600 con 0x556a633497a0 2024-09-11T00:18:52.927 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:52.919+0000 7f82fcc00640 10 client.26845 handle_command_reply: tid=0 2024-09-11T00:18:52.927 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:52.920+0000 7f8318671740 1 client.26845 shutdown 2024-09-11T00:18:52.927 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:52.920+0000 7f8318671740 1 -- 172.21.15.12:0/2367118490 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556a63339b00 msgr2=0x556a6333f140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:18:52.928 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:52.920+0000 7f8318671740 1 --2- 172.21.15.12:0/2367118490 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556a63339b00 0x556a6333f140 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f83080cd0c0 tx=0x7f83080ccd10 comp rx=0 tx=0).stop 2024-09-11T00:18:52.928 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:52.920+0000 7f8318671740 1 -- 172.21.15.12:0/2367118490 shutdown_connections 2024-09-11T00:18:52.928 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:52.920+0000 7f8318671740 1 --2- 172.21.15.12:0/2367118490 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x556a633497a0 0x556a6334bc20 crc :-1 s=READY pgs=182 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:18:52.928 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:52.920+0000 7f8318671740 1 --2- 172.21.15.12:0/2367118490 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556a63339b00 0x556a6333f140 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:18:52.928 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:52.920+0000 7f8318671740 1 --2- 172.21.15.12:0/2367118490 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556a633370c0 0x556a6333e570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:18:52.928 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:52.920+0000 7f82fcc00640 0 client.26845 ms_handle_reset on v2:172.21.15.165:6838/501312984 2024-09-11T00:18:52.928 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:52.920+0000 7f8318671740 1 --2- 172.21.15.12:0/2367118490 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556a63334790 0x556a6333ded0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:18:52.928 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:52.920+0000 7f8318671740 1 -- 172.21.15.12:0/2367118490 >> 172.21.15.12:0/2367118490 conn(0x556a6332f8e0 msgr2=0x556a6332ea60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:18:52.928 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:52.920+0000 7f8318671740 1 -- 172.21.15.12:0/2367118490 shutdown_connections 2024-09-11T00:18:52.928 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:52.920+0000 7f8318671740 1 -- 172.21.15.12:0/2367118490 wait complete. 2024-09-11T00:18:52.928 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:52.920+0000 7f8318671740 20 client.26845 trim_cache size 0 max 16384 2024-09-11T00:18:52.939 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:52.936+0000 7f8318671740 1 Processor -- start 2024-09-11T00:18:52.939 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:52.936+0000 7f8318671740 1 -- start start 2024-09-11T00:18:52.940 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:52.936+0000 7f8318671740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556a63334790 0x556a63332930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:18:52.940 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:52.936+0000 7f8318671740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556a633370c0 0x556a63332e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:18:52.940 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:52.936+0000 7f8318671740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556a63339b00 0x556a63330140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:18:52.940 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:52.936+0000 7f8318671740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f83101ad980 con 0x556a633370c0 2024-09-11T00:18:52.940 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:52.936+0000 7f8318671740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f83101af550 con 0x556a63334790 2024-09-11T00:18:52.940 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:52.936+0000 7f8318671740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f83101ae080 con 0x556a63339b00 2024-09-11T00:18:52.940 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:52.937+0000 7f8307e00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556a633370c0 0x556a63332e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:18:52.940 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:52.937+0000 7f830ea00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556a63334790 0x556a63332930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:18:52.940 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:52.937+0000 7f8307e00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556a633370c0 0x556a63332e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:46180/0 (socket says 172.21.15.12:46180) 2024-09-11T00:18:52.940 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:52.937+0000 7f8307e00640 1 -- 172.21.15.12:0/465774483 learned_addr learned my addr 172.21.15.12:0/465774483 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:18:52.940 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:52.937+0000 7f830f400640 1 --2- 172.21.15.12:0/465774483 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556a63339b00 0x556a63330140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:18:52.940 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:52.937+0000 7f830ea00640 1 -- 172.21.15.12:0/465774483 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556a63339b00 msgr2=0x556a63330140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:18:52.941 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:52.937+0000 7f830ea00640 1 --2- 172.21.15.12:0/465774483 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556a63339b00 0x556a63330140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:18:52.941 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:52.937+0000 7f830ea00640 1 -- 172.21.15.12:0/465774483 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556a633370c0 msgr2=0x556a63332e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:18:52.941 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:52.937+0000 7f830ea00640 1 --2- 172.21.15.12:0/465774483 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556a633370c0 0x556a63332e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:18:52.941 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:52.937+0000 7f830ea00640 1 -- 172.21.15.12:0/465774483 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556a6333e8e0 con 0x556a63334790 2024-09-11T00:18:52.941 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:52.937+0000 7f8307e00640 1 --2- 172.21.15.12:0/465774483 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556a633370c0 0x556a63332e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:18:52.941 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:52.937+0000 7f830ea00640 1 --2- 172.21.15.12:0/465774483 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556a63334790 0x556a63332930 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x556a6334c4a0 tx=0x7f83000096a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=326148d0586274bf server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:18:52.944 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:52.938+0000 7f82ff400640 1 -- 172.21.15.12:0/465774483 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8300031070 con 0x556a63334790 2024-09-11T00:18:52.944 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:52.938+0000 7f82ff400640 1 -- 172.21.15.12:0/465774483 <== mon.2 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f83000054f0 con 0x556a63334790 2024-09-11T00:18:52.945 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:52.938+0000 7f82ff400640 1 -- 172.21.15.12:0/465774483 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8300003ca0 con 0x556a63334790 2024-09-11T00:18:52.945 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:52.938+0000 7f8318671740 1 -- 172.21.15.12:0/465774483 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556a63334790 msgr2=0x556a63332930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:18:52.945 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:52.938+0000 7f8318671740 1 --2- 172.21.15.12:0/465774483 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556a63334790 0x556a63332930 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x556a6334c4a0 tx=0x7f83000096a0 comp rx=0 tx=0).stop 2024-09-11T00:18:52.945 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:52.938+0000 7f8318671740 1 -- 172.21.15.12:0/465774483 shutdown_connections 2024-09-11T00:18:52.945 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:52.938+0000 7f8318671740 1 --2- 172.21.15.12:0/465774483 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556a63339b00 0x556a63330140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:18:52.945 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:52.938+0000 7f8318671740 1 --2- 172.21.15.12:0/465774483 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556a633370c0 0x556a63332e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:18:52.945 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:52.938+0000 7f8318671740 1 --2- 172.21.15.12:0/465774483 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556a63334790 0x556a63332930 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:18:52.945 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:52.938+0000 7f8318671740 1 -- 172.21.15.12:0/465774483 >> 172.21.15.12:0/465774483 conn(0x556a6332f8e0 msgr2=0x556a63333c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:18:52.945 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:52.939+0000 7f8318671740 1 -- 172.21.15.12:0/465774483 shutdown_connections 2024-09-11T00:18:52.945 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:52.939+0000 7f8318671740 1 -- 172.21.15.12:0/465774483 wait complete. 2024-09-11T00:18:52.945 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:52.939+0000 7f8318671740 1 -- start start 2024-09-11T00:18:52.945 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:52.939+0000 7f8318671740 1 Processor -- start 2024-09-11T00:18:52.946 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:52.939+0000 7f8318671740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556a63334790 0x556a633458b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:18:52.946 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:52.939+0000 7f8318671740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556a633370c0 0x556a633445b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:18:52.946 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:52.939+0000 7f8318671740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556a63339b00 0x556a63344af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:18:52.946 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:52.939+0000 7f8318671740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f83101aee80 con 0x556a633370c0 2024-09-11T00:18:52.946 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:52.939+0000 7f8318671740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f83101ae750 con 0x556a63339b00 2024-09-11T00:18:52.946 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:52.939+0000 7f8318671740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f83101ab960 con 0x556a63334790 2024-09-11T00:18:52.946 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:52.939+0000 7f8318671740 10 client.0 _ll_register_callbacks cb 0x7f8308009500 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:18:52.946 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:52.940+0000 7f8307e00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556a633370c0 0x556a633445b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:18:52.946 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:52.940+0000 7f8307e00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556a633370c0 0x556a633445b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:46192/0 (socket says 172.21.15.12:46192) 2024-09-11T00:18:52.946 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:52.940+0000 7f8307e00640 1 -- 172.21.15.12:0/3691450346 learned_addr learned my addr 172.21.15.12:0/3691450346 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:18:52.946 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:52.940+0000 7f830f400640 1 --2- 172.21.15.12:0/3691450346 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556a63339b00 0x556a63344af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:18:52.946 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:52.940+0000 7f830ea00640 1 --2- 172.21.15.12:0/3691450346 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556a63334790 0x556a633458b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:18:52.946 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:52.940+0000 7f8307e00640 1 -- 172.21.15.12:0/3691450346 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556a63334790 msgr2=0x556a633458b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:18:52.947 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:52.940+0000 7f8307e00640 1 --2- 172.21.15.12:0/3691450346 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556a63334790 0x556a633458b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:18:52.947 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:52.940+0000 7f8307e00640 1 -- 172.21.15.12:0/3691450346 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556a63339b00 msgr2=0x556a63344af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:18:52.947 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:52.940+0000 7f8307e00640 1 --2- 172.21.15.12:0/3691450346 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556a63339b00 0x556a63344af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:18:52.947 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:52.940+0000 7f8307e00640 1 -- 172.21.15.12:0/3691450346 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556a63345200 con 0x556a633370c0 2024-09-11T00:18:52.947 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:52.941+0000 7f8307e00640 1 --2- 172.21.15.12:0/3691450346 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556a633370c0 0x556a633445b0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f82f800fdf0 tx=0x7f82f80097b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=111665e5e69e79aa server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:18:52.947 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:52.941+0000 7f82fcc00640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-09-11T00:18:52.947 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:52.941+0000 7f82fcc00640 1 -- 172.21.15.12:0/3691450346 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82f8032060 con 0x556a633370c0 2024-09-11T00:18:52.947 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:52.941+0000 7f8318671740 1 -- 172.21.15.12:0/3691450346 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f83000054f0 con 0x556a633370c0 2024-09-11T00:18:52.947 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:52.941+0000 7f82fcc00640 1 -- 172.21.15.12:0/3691450346 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9208+0+0 (secure 0 0 0) 0x7f82f8020ba0 con 0x556a633370c0 2024-09-11T00:18:52.947 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:52.941+0000 7f82fcc00640 1 -- 172.21.15.12:0/3691450346 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f82f8020dc0 con 0x556a633370c0 2024-09-11T00:18:52.947 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:52.941+0000 7f8318671740 10 client.16872 fetch_fsmap learned FSMap version 44 2024-09-11T00:18:52.947 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:52.941+0000 7f8318671740 1 -- 172.21.15.12:0/3691450346 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x556a633b67c0 con 0x556a633370c0 2024-09-11T00:18:52.948 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:52.942+0000 7f82fcc00640 1 -- 172.21.15.12:0/3691450346 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f83000054f0 con 0x556a633370c0 2024-09-11T00:18:52.948 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:52.942+0000 7f82fcc00640 1 -- 172.21.15.12:0/3691450346 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 44) ==== 8382+0+0 (secure 0 0 0) 0x7f82f800e3b0 con 0x556a633370c0 2024-09-11T00:18:52.948 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:52.942+0000 7f8318671740 10 client.16872 fetch_fsmap finished waiting for FSMap version 44 2024-09-11T00:18:52.948 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:52.942+0000 7f8318671740 10 client.16872 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T00:18:52.948 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:52.942+0000 7f8318671740 20 client.16872 populate_metadata read hostname 'smithi012' 2024-09-11T00:18:52.948 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:52.942+0000 7f8318671740 1 --2- 172.21.15.12:0/3691450346 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x556a633b71b0 0x556a633b95d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:18:52.948 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:52.942+0000 7f8318671740 4 client.16872 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T00:18:52.948 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:52.942+0000 7f8318671740 1 -- 172.21.15.12:0/3691450346 --> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] -- command(tid 0: {"prefix": "flush journal"}) -- 0x556a632a50b0 con 0x556a633b71b0 2024-09-11T00:18:52.948 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:52.942+0000 7f830ea00640 1 --2- 172.21.15.12:0/3691450346 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x556a633b71b0 0x556a633b95d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:18:52.949 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:52.943+0000 7f830ea00640 1 --2- 172.21.15.12:0/3691450346 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x556a633b71b0 0x556a633b95d0 crc :-1 s=READY pgs=184 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4eb0a042a111b209 server_cookie=b3733e8bd8547dbb in_seq=0 out_seq=0 2024-09-11T00:18:52.949 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:52.943+0000 7f82fcc00640 10 client.16872 ms_handle_connect on v2:172.21.15.165:6838/501312984 2024-09-11T00:18:52.979 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:52.976+0000 7f82fcc00640 1 -- 172.21.15.12:0/3691450346 <== mds.0 v2:172.21.15.165:6838/501312984 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x556a632a50b0 con 0x556a633b71b0 2024-09-11T00:18:52.979 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:52.976+0000 7f82fcc00640 10 client.16872 handle_command_reply: tid=0 2024-09-11T00:18:52.979 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:52.976+0000 7f8318671740 1 client.16872 shutdown 2024-09-11T00:18:52.979 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:52.976+0000 7f8318671740 1 -- 172.21.15.12:0/3691450346 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556a633370c0 msgr2=0x556a633445b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:18:52.979 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:52.976+0000 7f8318671740 1 --2- 172.21.15.12:0/3691450346 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556a633370c0 0x556a633445b0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f82f800fdf0 tx=0x7f82f80097b0 comp rx=0 tx=0).stop 2024-09-11T00:18:52.979 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:52.976+0000 7f8318671740 1 -- 172.21.15.12:0/3691450346 shutdown_connections 2024-09-11T00:18:52.979 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:52.976+0000 7f8318671740 1 --2- 172.21.15.12:0/3691450346 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x556a633b71b0 0x556a633b95d0 crc :-1 s=READY pgs=184 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:18:52.979 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:52.976+0000 7f8318671740 1 --2- 172.21.15.12:0/3691450346 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556a63339b00 0x556a63344af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:18:52.979 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:52.976+0000 7f82fcc00640 0 client.16872 ms_handle_reset on v2:172.21.15.165:6838/501312984 2024-09-11T00:18:52.979 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:52.976+0000 7f8318671740 1 --2- 172.21.15.12:0/3691450346 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556a633370c0 0x556a633445b0 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:18:52.980 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:52.976+0000 7f8318671740 1 --2- 172.21.15.12:0/3691450346 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556a63334790 0x556a633458b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:18:52.980 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:52.976+0000 7f8318671740 1 -- 172.21.15.12:0/3691450346 >> 172.21.15.12:0/3691450346 conn(0x556a6332f8e0 msgr2=0x556a63330140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:18:52.980 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:52.976+0000 7f8318671740 1 -- 172.21.15.12:0/3691450346 shutdown_connections 2024-09-11T00:18:52.980 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:52.976+0000 7f8318671740 1 -- 172.21.15.12:0/3691450346 wait complete. 2024-09-11T00:18:52.980 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:52.977+0000 7f8318671740 20 client.16872 trim_cache size 0 max 16384 2024-09-11T00:18:52.980 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:{ 2024-09-11T00:18:52.980 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "message": "", 2024-09-11T00:18:52.980 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "return_code": 0 2024-09-11T00:18:52.980 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:} 2024-09-11T00:18:52.981 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:52.977+0000 7f8315800640 1 -- 172.21.15.12:0/2985767385 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f82f40b52e0 msgr2=0x7f82f40b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:18:52.981 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:52.977+0000 7f8315800640 1 --2- 172.21.15.12:0/2985767385 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f82f40b52e0 0x7f82f40b77a0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f82f8009fd0 tx=0x7f82f8009260 comp rx=0 tx=0).stop 2024-09-11T00:18:52.981 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:52.977+0000 7f8315800640 1 -- 172.21.15.12:0/2985767385 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f83101066a0 msgr2=0x7f831019c1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:18:52.981 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:52.977+0000 7f8315800640 1 --2- 172.21.15.12:0/2985767385 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f83101066a0 0x7f831019c1e0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f830802ac00 tx=0x7f8308030d20 comp rx=0 tx=0).stop 2024-09-11T00:18:52.982 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:52.977+0000 7f8315800640 1 -- 172.21.15.12:0/2985767385 shutdown_connections 2024-09-11T00:18:52.982 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:52.977+0000 7f8315800640 1 --2- 172.21.15.12:0/2985767385 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f82f40b52e0 0x7f82f40b77a0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:18:52.982 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:52.977+0000 7f8315800640 1 --2- 172.21.15.12:0/2985767385 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f83101066a0 0x7f831019c1e0 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:18:52.982 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:52.977+0000 7f8315800640 1 --2- 172.21.15.12:0/2985767385 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8310105ce0 0x7f831019b5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:18:52.982 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:52.977+0000 7f8315800640 1 --2- 172.21.15.12:0/2985767385 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8310104b90 0x7f831019af60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:18:52.982 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:52.977+0000 7f8315800640 1 -- 172.21.15.12:0/2985767385 >> 172.21.15.12:0/2985767385 conn(0x7f8310100470 msgr2=0x7f8310109070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:18:52.982 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:52.978+0000 7f8315800640 1 -- 172.21.15.12:0/2985767385 shutdown_connections 2024-09-11T00:18:52.982 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:18:52.978+0000 7f8315800640 1 -- 172.21.15.12:0/2985767385 wait complete. 2024-09-11T00:18:53.694 INFO:tasks.workunit.client.0.smithi012.stdout: 1 483326 3.97 MB/sec execute 530 sec latency 3980.381 ms 2024-09-11T00:18:53.782 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:18:53 smithi165 ceph-mon[35356]: pgmap v753: 129 pgs: 129 active+clean; 112 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 28 MiB/s rd, 937 KiB/s wr, 144 op/s 2024-09-11T00:18:53.813 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:18:53 smithi012 ceph-mon[30333]: pgmap v753: 129 pgs: 129 active+clean; 112 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 28 MiB/s rd, 937 KiB/s wr, 144 op/s 2024-09-11T00:18:53.831 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:18:53 smithi178 ceph-mon[35919]: pgmap v753: 129 pgs: 129 active+clean; 112 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 28 MiB/s rd, 937 KiB/s wr, 144 op/s 2024-09-11T00:18:54.694 INFO:tasks.workunit.client.0.smithi012.stdout: 1 483326 3.96 MB/sec execute 531 sec latency 4980.499 ms 2024-09-11T00:18:55.694 INFO:tasks.workunit.client.0.smithi012.stdout: 1 483326 3.95 MB/sec execute 532 sec latency 5980.641 ms 2024-09-11T00:18:55.782 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:18:55 smithi165 ceph-mon[35356]: pgmap v754: 129 pgs: 129 active+clean; 112 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 28 MiB/s rd, 834 KiB/s wr, 131 op/s 2024-09-11T00:18:55.813 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:18:55 smithi012 ceph-mon[30333]: pgmap v754: 129 pgs: 129 active+clean; 112 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 28 MiB/s rd, 834 KiB/s wr, 131 op/s 2024-09-11T00:18:55.831 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:18:55 smithi178 ceph-mon[35919]: pgmap v754: 129 pgs: 129 active+clean; 112 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 28 MiB/s rd, 834 KiB/s wr, 131 op/s 2024-09-11T00:18:56.694 INFO:tasks.workunit.client.0.smithi012.stdout: 1 483326 3.95 MB/sec execute 533 sec latency 6980.782 ms 2024-09-11T00:18:57.694 INFO:tasks.workunit.client.0.smithi012.stdout: 1 483326 3.94 MB/sec execute 534 sec latency 7980.933 ms 2024-09-11T00:18:57.782 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:18:57 smithi165 ceph-mon[35356]: pgmap v755: 129 pgs: 129 active+clean; 107 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 28 MiB/s rd, 738 KiB/s wr, 150 op/s 2024-09-11T00:18:57.813 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:18:57 smithi012 ceph-mon[30333]: pgmap v755: 129 pgs: 129 active+clean; 107 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 28 MiB/s rd, 738 KiB/s wr, 150 op/s 2024-09-11T00:18:57.831 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:18:57 smithi178 ceph-mon[35919]: pgmap v755: 129 pgs: 129 active+clean; 107 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 28 MiB/s rd, 738 KiB/s wr, 150 op/s 2024-09-11T00:18:58.695 INFO:tasks.workunit.client.0.smithi012.stdout: 1 483326 3.93 MB/sec execute 535 sec latency 8981.007 ms 2024-09-11T00:18:59.695 INFO:tasks.workunit.client.0.smithi012.stdout: 1 483326 3.92 MB/sec execute 536 sec latency 9981.131 ms 2024-09-11T00:18:59.782 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:18:59 smithi165 ceph-mon[35356]: pgmap v756: 129 pgs: 129 active+clean; 107 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 28 MiB/s rd, 405 KiB/s wr, 124 op/s 2024-09-11T00:18:59.813 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:18:59 smithi012 ceph-mon[30333]: pgmap v756: 129 pgs: 129 active+clean; 107 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 28 MiB/s rd, 405 KiB/s wr, 124 op/s 2024-09-11T00:18:59.831 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:18:59 smithi178 ceph-mon[35919]: pgmap v756: 129 pgs: 129 active+clean; 107 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 28 MiB/s rd, 405 KiB/s wr, 124 op/s 2024-09-11T00:19:00.695 INFO:tasks.workunit.client.0.smithi012.stdout: 1 483326 3.92 MB/sec execute 537 sec latency 10981.251 ms 2024-09-11T00:19:01.695 INFO:tasks.workunit.client.0.smithi012.stdout: 1 483326 3.91 MB/sec execute 538 sec latency 11981.321 ms 2024-09-11T00:19:01.722 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:19:01 smithi012 ceph-mon[30333]: pgmap v757: 129 pgs: 129 active+clean; 107 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 28 MiB/s rd, 408 KiB/s wr, 126 op/s 2024-09-11T00:19:01.782 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:19:01 smithi165 ceph-mon[35356]: pgmap v757: 129 pgs: 129 active+clean; 107 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 28 MiB/s rd, 408 KiB/s wr, 126 op/s 2024-09-11T00:19:01.831 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:19:01 smithi178 ceph-mon[35919]: pgmap v757: 129 pgs: 129 active+clean; 107 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 28 MiB/s rd, 408 KiB/s wr, 126 op/s 2024-09-11T00:19:02.695 INFO:tasks.workunit.client.0.smithi012.stdout: 1 483326 3.90 MB/sec execute 539 sec latency 12981.386 ms 2024-09-11T00:19:03.563 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:19:03 smithi012 ceph-mon[30333]: pgmap v758: 129 pgs: 129 active+clean; 107 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 265 KiB/s rd, 110 KiB/s wr, 48 op/s 2024-09-11T00:19:03.695 INFO:tasks.workunit.client.0.smithi012.stdout: 1 483326 3.90 MB/sec execute 540 sec latency 13981.444 ms 2024-09-11T00:19:03.782 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:19:03 smithi165 ceph-mon[35356]: pgmap v758: 129 pgs: 129 active+clean; 107 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 265 KiB/s rd, 110 KiB/s wr, 48 op/s 2024-09-11T00:19:03.831 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:19:03 smithi178 ceph-mon[35919]: pgmap v758: 129 pgs: 129 active+clean; 107 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 265 KiB/s rd, 110 KiB/s wr, 48 op/s 2024-09-11T00:19:04.695 INFO:tasks.workunit.client.0.smithi012.stdout: 1 483326 3.89 MB/sec execute 541 sec latency 14981.756 ms 2024-09-11T00:19:05.695 INFO:tasks.workunit.client.0.smithi012.stdout: 1 483326 3.88 MB/sec execute 542 sec latency 15981.872 ms 2024-09-11T00:19:05.782 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:19:05 smithi165 ceph-mon[35356]: pgmap v759: 129 pgs: 129 active+clean; 107 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 51 KiB/s wr, 29 op/s 2024-09-11T00:19:05.813 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:19:05 smithi012 ceph-mon[30333]: pgmap v759: 129 pgs: 129 active+clean; 107 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 51 KiB/s wr, 29 op/s 2024-09-11T00:19:05.831 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:19:05 smithi178 ceph-mon[35919]: pgmap v759: 129 pgs: 129 active+clean; 107 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 51 KiB/s wr, 29 op/s 2024-09-11T00:19:06.107 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:06.105+0000 7fa2f0400640 1 -- 172.21.15.12:0/3066434400 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa2e8077a20 msgr2=0x7fa2e8077e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:19:06.107 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:06.105+0000 7fa2f0400640 1 --2- 172.21.15.12:0/3066434400 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa2e8077a20 0x7fa2e8077e20 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7fa2dc009ca0 tx=0x7fa2dc02a8c0 comp rx=0 tx=0).stop 2024-09-11T00:19:06.108 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:06.105+0000 7fa2f0400640 1 -- 172.21.15.12:0/3066434400 shutdown_connections 2024-09-11T00:19:06.108 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:06.105+0000 7fa2f0400640 1 --2- 172.21.15.12:0/3066434400 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa2e8079530 0x7fa2e8079e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:19:06.108 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:06.105+0000 7fa2f0400640 1 --2- 172.21.15.12:0/3066434400 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa2e8078b70 0x7fa2e8078ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:19:06.108 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:06.105+0000 7fa2f0400640 1 --2- 172.21.15.12:0/3066434400 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa2e8077a20 0x7fa2e8077e20 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:19:06.108 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:06.105+0000 7fa2f0400640 1 -- 172.21.15.12:0/3066434400 >> 172.21.15.12:0/3066434400 conn(0x7fa2e8100470 msgr2=0x7fa2e8102890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:19:06.108 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:06.105+0000 7fa2f0400640 1 -- 172.21.15.12:0/3066434400 shutdown_connections 2024-09-11T00:19:06.108 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:06.105+0000 7fa2f0400640 1 -- 172.21.15.12:0/3066434400 wait complete. 2024-09-11T00:19:06.108 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:06.106+0000 7fa2f0400640 1 Processor -- start 2024-09-11T00:19:06.108 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:06.106+0000 7fa2f0400640 1 -- start start 2024-09-11T00:19:06.110 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:06.106+0000 7fa2f0400640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa2e8077a20 0x7fa2e819f3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:19:06.110 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:06.106+0000 7fa2f0400640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa2e8078b70 0x7fa2e819fa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:19:06.111 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:06.107+0000 7fa2f0400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa2e8079530 0x7fa2e81a0630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:19:06.111 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:06.107+0000 7fa2f0400640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fa2e8114c50 con 0x7fa2e8077a20 2024-09-11T00:19:06.111 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:06.107+0000 7fa2f0400640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fa2e8114ad0 con 0x7fa2e8079530 2024-09-11T00:19:06.111 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:06.107+0000 7fa2f0400640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fa2e8114dd0 con 0x7fa2e8078b70 2024-09-11T00:19:06.111 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:06.107+0000 7fa2eda00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa2e8077a20 0x7fa2e819f3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:19:06.111 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:06.107+0000 7fa2eda00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa2e8077a20 0x7fa2e819f3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:44636/0 (socket says 172.21.15.12:44636) 2024-09-11T00:19:06.111 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:06.107+0000 7fa2eda00640 1 -- 172.21.15.12:0/3821473160 learned_addr learned my addr 172.21.15.12:0/3821473160 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:19:06.111 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:06.107+0000 7fa2ed000640 1 --2- 172.21.15.12:0/3821473160 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa2e8078b70 0x7fa2e819fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:19:06.111 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:06.107+0000 7fa2ee400640 1 --2- 172.21.15.12:0/3821473160 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa2e8079530 0x7fa2e81a0630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:19:06.111 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:06.107+0000 7fa2eda00640 1 -- 172.21.15.12:0/3821473160 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa2e8078b70 msgr2=0x7fa2e819fa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:19:06.111 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:06.107+0000 7fa2eda00640 1 --2- 172.21.15.12:0/3821473160 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa2e8078b70 0x7fa2e819fa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:19:06.112 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:06.107+0000 7fa2eda00640 1 -- 172.21.15.12:0/3821473160 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa2e8079530 msgr2=0x7fa2e81a0630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:19:06.112 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:06.107+0000 7fa2eda00640 1 --2- 172.21.15.12:0/3821473160 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa2e8079530 0x7fa2e81a0630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:19:06.112 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:06.107+0000 7fa2eda00640 1 -- 172.21.15.12:0/3821473160 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2e81a0ef0 con 0x7fa2e8077a20 2024-09-11T00:19:06.112 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:06.107+0000 7fa2ee400640 1 --2- 172.21.15.12:0/3821473160 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa2e8079530 0x7fa2e81a0630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T00:19:06.112 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:06.108+0000 7fa2eda00640 1 --2- 172.21.15.12:0/3821473160 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa2e8077a20 0x7fa2e819f3b0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7fa2dc000f80 tx=0x7fa2dc003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:19:06.112 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:06.108+0000 7fa2e6000640 1 -- 172.21.15.12:0/3821473160 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2dc0317c0 con 0x7fa2e8077a20 2024-09-11T00:19:06.112 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:06.108+0000 7fa2e6000640 1 -- 172.21.15.12:0/3821473160 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa2dc0044f0 con 0x7fa2e8077a20 2024-09-11T00:19:06.112 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:06.108+0000 7fa2e6000640 1 -- 172.21.15.12:0/3821473160 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2dc02fcf0 con 0x7fa2e8077a20 2024-09-11T00:19:06.112 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:06.108+0000 7fa2f0400640 1 -- 172.21.15.12:0/3821473160 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa2e81a1180 con 0x7fa2e8077a20 2024-09-11T00:19:06.112 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:06.108+0000 7fa2f0400640 1 -- 172.21.15.12:0/3821473160 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2e81a78b0 con 0x7fa2e8077a20 2024-09-11T00:19:06.116 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:06.111+0000 7fa2f3444740 1 Processor -- start 2024-09-11T00:19:06.116 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:06.111+0000 7fa2f3444740 1 -- start start 2024-09-11T00:19:06.116 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:06.111+0000 7fa2e6000640 1 -- 172.21.15.12:0/3821473160 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7fa2dc0046a0 con 0x7fa2e8077a20 2024-09-11T00:19:06.116 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:06.112+0000 7fa2f3444740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e80cea0790 0x55e80cea2b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:19:06.116 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:06.112+0000 7fa2f3444740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e80cea30c0 0x55e80cea54b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:19:06.116 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:06.112+0000 7fa2f3444740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e80cea5b00 0x55e80cea7fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:19:06.116 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:06.112+0000 7fa2f3444740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55e80ce136b0 con 0x55e80cea0790 2024-09-11T00:19:06.116 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:06.112+0000 7fa2f3444740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55e80ce02ae0 con 0x55e80cea30c0 2024-09-11T00:19:06.117 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:06.112+0000 7fa2f3444740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55e80ce068f0 con 0x55e80cea5b00 2024-09-11T00:19:06.117 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:06.112+0000 7fa2ee400640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e80cea0790 0x55e80cea2b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:19:06.117 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:06.112+0000 7fa2ee400640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e80cea0790 0x55e80cea2b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:44640/0 (socket says 172.21.15.12:44640) 2024-09-11T00:19:06.117 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:06.112+0000 7fa2ee400640 1 -- 172.21.15.12:0/2714305044 learned_addr learned my addr 172.21.15.12:0/2714305044 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:19:06.117 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:06.112+0000 7fa2eda00640 1 --2- 172.21.15.12:0/2714305044 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e80cea30c0 0x55e80cea54b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:19:06.117 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:06.112+0000 7fa2ed000640 1 --2- 172.21.15.12:0/2714305044 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e80cea5b00 0x55e80cea7fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:19:06.117 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:06.112+0000 7fa2ee400640 1 -- 172.21.15.12:0/2714305044 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e80cea5b00 msgr2=0x55e80cea7fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:19:06.117 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:06.112+0000 7fa2ee400640 1 --2- 172.21.15.12:0/2714305044 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e80cea5b00 0x55e80cea7fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:19:06.117 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:06.112+0000 7fa2ee400640 1 -- 172.21.15.12:0/2714305044 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e80cea30c0 msgr2=0x55e80cea54b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:19:06.117 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:06.112+0000 7fa2ee400640 1 --2- 172.21.15.12:0/2714305044 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e80cea30c0 0x55e80cea54b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:19:06.118 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:06.112+0000 7fa2ee400640 1 -- 172.21.15.12:0/2714305044 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e80cdcf780 con 0x55e80cea0790 2024-09-11T00:19:06.118 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:06.112+0000 7fa2ed000640 1 --2- 172.21.15.12:0/2714305044 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e80cea5b00 0x55e80cea7fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:19:06.118 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:06.113+0000 7fa2ee400640 1 --2- 172.21.15.12:0/2714305044 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e80cea0790 0x55e80cea2b80 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7fa2e000dc40 tx=0x7fa2e000f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9931c80ac1faff5 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:19:06.118 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:06.113+0000 7fa2e6000640 1 --2- 172.21.15.12:0/3821473160 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fa2d00b52e0 0x7fa2d00b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:19:06.118 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:06.113+0000 7fa2daa00640 1 -- 172.21.15.12:0/2714305044 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2e0015070 con 0x55e80cea0790 2024-09-11T00:19:06.118 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:06.113+0000 7fa2daa00640 1 -- 172.21.15.12:0/2714305044 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa2e00035f0 con 0x55e80cea0790 2024-09-11T00:19:06.118 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:06.113+0000 7fa2daa00640 1 -- 172.21.15.12:0/2714305044 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2e00116a0 con 0x55e80cea0790 2024-09-11T00:19:06.118 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:06.113+0000 7fa2e6000640 1 -- 172.21.15.12:0/3821473160 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9208+0+0 (secure 0 0 0) 0x7fa2dc1073e0 con 0x7fa2e8077a20 2024-09-11T00:19:06.118 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:06.113+0000 7fa2eda00640 1 --2- 172.21.15.12:0/3821473160 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fa2d00b52e0 0x7fa2d00b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:19:06.118 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:06.114+0000 7fa2f3444740 1 -- 172.21.15.12:0/2714305044 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e80cea0790 msgr2=0x55e80cea2b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:19:06.118 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:06.114+0000 7fa2f3444740 1 --2- 172.21.15.12:0/2714305044 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e80cea0790 0x55e80cea2b80 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7fa2e000dc40 tx=0x7fa2e000f730 comp rx=0 tx=0).stop 2024-09-11T00:19:06.118 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:06.114+0000 7fa2f3444740 1 -- 172.21.15.12:0/2714305044 shutdown_connections 2024-09-11T00:19:06.119 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:06.114+0000 7fa2f3444740 1 --2- 172.21.15.12:0/2714305044 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e80cea5b00 0x55e80cea7fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:19:06.119 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:06.114+0000 7fa2f3444740 1 --2- 172.21.15.12:0/2714305044 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e80cea30c0 0x55e80cea54b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:19:06.119 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:06.114+0000 7fa2f3444740 1 --2- 172.21.15.12:0/2714305044 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e80cea0790 0x55e80cea2b80 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:19:06.119 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:06.114+0000 7fa2f3444740 1 -- 172.21.15.12:0/2714305044 >> 172.21.15.12:0/2714305044 conn(0x55e80ce9b8e0 msgr2=0x55e80ce9c670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:19:06.119 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:06.114+0000 7fa2eda00640 1 --2- 172.21.15.12:0/3821473160 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fa2d00b52e0 0x7fa2d00b77a0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fa2dc0cd700 tx=0x7fa2dc10ae20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:19:06.119 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:06.114+0000 7fa2f3444740 1 -- 172.21.15.12:0/2714305044 shutdown_connections 2024-09-11T00:19:06.119 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:06.114+0000 7fa2f3444740 1 -- 172.21.15.12:0/2714305044 wait complete. 2024-09-11T00:19:06.119 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:06.115+0000 7fa2f3444740 1 -- start start 2024-09-11T00:19:06.119 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:06.115+0000 7fa2f3444740 1 Processor -- start 2024-09-11T00:19:06.123 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:06.115+0000 7fa2f3444740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e80cea0790 0x55e80cea9f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:19:06.123 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:06.115+0000 7fa2f3444740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e80cea30c0 0x55e80ceaa5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:19:06.123 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:06.115+0000 7fa2f3444740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e80cea5b00 0x55e80ceab170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:19:06.123 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:06.115+0000 7fa2f3444740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55e80cd3aab0 con 0x55e80cea5b00 2024-09-11T00:19:06.123 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:06.115+0000 7fa2f3444740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55e80cda9570 con 0x55e80cea30c0 2024-09-11T00:19:06.123 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:06.115+0000 7fa2f3444740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55e80cea8ae0 con 0x55e80cea0790 2024-09-11T00:19:06.123 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:06.115+0000 7fa2f3444740 10 client.0 _ll_register_callbacks cb 0x55e80cdb8100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:19:06.123 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:06.116+0000 7fa2ee400640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e80cea5b00 0x55e80ceab170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:19:06.123 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:06.116+0000 7fa2ed000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e80cea30c0 0x55e80ceaa5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:19:06.123 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:06.116+0000 7fa2ee400640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e80cea0790 0x55e80cea9f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:19:06.123 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:06.116+0000 7fa2ed000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e80cea30c0 0x55e80ceaa5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.12:34728/0 (socket says 172.21.15.12:34728) 2024-09-11T00:19:06.123 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:06.116+0000 7fa2ed000640 1 -- 172.21.15.12:0/3808759620 learned_addr learned my addr 172.21.15.12:0/3808759620 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:19:06.123 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:06.116+0000 7fa2ee400640 1 -- 172.21.15.12:0/3808759620 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e80cea30c0 msgr2=0x55e80ceaa5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:19:06.124 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:06.116+0000 7fa2ee400640 1 --2- 172.21.15.12:0/3808759620 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e80cea30c0 0x55e80ceaa5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:19:06.124 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:06.116+0000 7fa2ee400640 1 -- 172.21.15.12:0/3808759620 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e80cea5b00 msgr2=0x55e80ceab170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:19:06.124 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:06.116+0000 7fa2ee400640 1 --2- 172.21.15.12:0/3808759620 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e80cea5b00 0x55e80ceab170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:19:06.124 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:06.116+0000 7fa2ee400640 1 -- 172.21.15.12:0/3808759620 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e80cb64790 con 0x55e80cea0790 2024-09-11T00:19:06.124 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:06.116+0000 7fa2ee400640 1 --2- 172.21.15.12:0/3808759620 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e80cea0790 0x55e80cea9f00 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fa2e0005fd0 tx=0x7fa2e0000bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=753d392237f0ebfa server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:19:06.124 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:06.116+0000 7fa2cf400640 10 client.0 ms_handle_connect on v2:172.21.15.178:3300/0 2024-09-11T00:19:06.124 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:06.117+0000 7fa2cf400640 1 -- 172.21.15.12:0/3808759620 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2e0015040 con 0x55e80cea0790 2024-09-11T00:19:06.124 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:06.117+0000 7fa2f3444740 1 -- 172.21.15.12:0/3808759620 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa2e00035f0 con 0x55e80cea0790 2024-09-11T00:19:06.124 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:06.117+0000 7fa2cf400640 1 -- 172.21.15.12:0/3808759620 <== mon.1 v2:172.21.15.178:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9208+0+0 (secure 0 0 0) 0x7fa2e0004210 con 0x55e80cea0790 2024-09-11T00:19:06.124 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:06.117+0000 7fa2cf400640 1 -- 172.21.15.12:0/3808759620 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7fa2e0004410 con 0x55e80cea0790 2024-09-11T00:19:06.124 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:06.117+0000 7fa2f3444740 10 client.26863 fetch_fsmap learned FSMap version 44 2024-09-11T00:19:06.124 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:06.117+0000 7fa2f3444740 1 -- 172.21.15.12:0/3808759620 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x55e80cbdd620 con 0x55e80cea0790 2024-09-11T00:19:06.124 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:06.117+0000 7fa2cf400640 1 -- 172.21.15.12:0/3808759620 <== mon.1 v2:172.21.15.178:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa2e00035f0 con 0x55e80cea0790 2024-09-11T00:19:06.125 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:06.117+0000 7fa2cf400640 1 -- 172.21.15.12:0/3808759620 <== mon.1 v2:172.21.15.178:3300/0 5 ==== fsmap(e 44) ==== 8382+0+0 (secure 0 0 0) 0x7fa2e0027040 con 0x55e80cea0790 2024-09-11T00:19:06.125 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:06.118+0000 7fa2f3444740 10 client.26863 fetch_fsmap finished waiting for FSMap version 44 2024-09-11T00:19:06.125 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:06.118+0000 7fa2f3444740 10 client.26863 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T00:19:06.125 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:06.118+0000 7fa2f3444740 20 client.26863 populate_metadata read hostname 'smithi012' 2024-09-11T00:19:06.125 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:06.118+0000 7fa2f3444740 1 --2- 172.21.15.12:0/3808759620 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55e80ceb54c0 0x55e80ceb78b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:19:06.125 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:06.118+0000 7fa2f3444740 4 client.26863 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T00:19:06.125 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:06.118+0000 7fa2f3444740 1 -- 172.21.15.12:0/3808759620 --> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e80ce15600 con 0x55e80ceb54c0 2024-09-11T00:19:06.125 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:06.118+0000 7fa2ed000640 1 --2- 172.21.15.12:0/3808759620 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55e80ceb54c0 0x55e80ceb78b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:19:06.125 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:06.118+0000 7fa2ed000640 1 --2- 172.21.15.12:0/3808759620 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55e80ceb54c0 0x55e80ceb78b0 crc :-1 s=READY pgs=185 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bbe65ec333183cf1 server_cookie=cb40b80cdee73bbc in_seq=0 out_seq=0 2024-09-11T00:19:06.125 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:06.119+0000 7fa2cf400640 10 client.26863 ms_handle_connect on v2:172.21.15.165:6838/501312984 2024-09-11T00:19:06.125 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:06.119+0000 7fa2cf400640 1 -- 172.21.15.12:0/3808759620 <== mds.0 v2:172.21.15.165:6838/501312984 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55e80ce15600 con 0x55e80ceb54c0 2024-09-11T00:19:06.125 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:06.119+0000 7fa2cf400640 10 client.26863 handle_command_reply: tid=0 2024-09-11T00:19:06.125 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:06.119+0000 7fa2f3444740 1 client.26863 shutdown 2024-09-11T00:19:06.125 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:06.120+0000 7fa2f3444740 1 -- 172.21.15.12:0/3808759620 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e80cea0790 msgr2=0x55e80cea9f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:19:06.126 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:06.120+0000 7fa2f3444740 1 --2- 172.21.15.12:0/3808759620 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e80cea0790 0x55e80cea9f00 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fa2e0005fd0 tx=0x7fa2e0000bc0 comp rx=0 tx=0).stop 2024-09-11T00:19:06.126 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:06.120+0000 7fa2f3444740 1 -- 172.21.15.12:0/3808759620 shutdown_connections 2024-09-11T00:19:06.126 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:06.120+0000 7fa2f3444740 1 --2- 172.21.15.12:0/3808759620 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55e80ceb54c0 0x55e80ceb78b0 crc :-1 s=READY pgs=185 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:19:06.126 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:06.120+0000 7fa2f3444740 1 --2- 172.21.15.12:0/3808759620 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e80cea5b00 0x55e80ceab170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:19:06.126 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:06.120+0000 7fa2f3444740 1 --2- 172.21.15.12:0/3808759620 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e80cea30c0 0x55e80ceaa5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:19:06.126 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:06.120+0000 7fa2cf400640 0 client.26863 ms_handle_reset on v2:172.21.15.165:6838/501312984 2024-09-11T00:19:06.126 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:06.120+0000 7fa2f3444740 1 --2- 172.21.15.12:0/3808759620 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e80cea0790 0x55e80cea9f00 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:19:06.126 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:06.120+0000 7fa2f3444740 1 -- 172.21.15.12:0/3808759620 >> 172.21.15.12:0/3808759620 conn(0x55e80ce9b8e0 msgr2=0x55e80ce9aa60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:19:06.126 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:06.120+0000 7fa2f3444740 1 -- 172.21.15.12:0/3808759620 shutdown_connections 2024-09-11T00:19:06.126 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:06.120+0000 7fa2f3444740 1 -- 172.21.15.12:0/3808759620 wait complete. 2024-09-11T00:19:06.126 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:06.120+0000 7fa2f3444740 20 client.26863 trim_cache size 0 max 16384 2024-09-11T00:19:06.138 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:06.135+0000 7fa2f3444740 1 Processor -- start 2024-09-11T00:19:06.138 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:06.136+0000 7fa2f3444740 1 -- start start 2024-09-11T00:19:06.139 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:06.136+0000 7fa2f3444740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e80cea0790 0x55e80ce96960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:19:06.139 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:06.136+0000 7fa2f3444740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e80cea30c0 0x55e80ce9c2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:19:06.139 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:06.136+0000 7fa2f3444740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e80cea5b00 0x55e80ce9c800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:19:06.139 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:06.136+0000 7fa2f3444740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fa2e81b1da0 con 0x55e80cea30c0 2024-09-11T00:19:06.139 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:06.136+0000 7fa2f3444740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fa2e81b3970 con 0x55e80cea5b00 2024-09-11T00:19:06.139 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:06.136+0000 7fa2f3444740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fa2e81b24a0 con 0x55e80cea0790 2024-09-11T00:19:06.139 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:06.136+0000 7fa2ed000640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e80cea30c0 0x55e80ce9c2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:19:06.139 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:06.136+0000 7fa2ed000640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e80cea30c0 0x55e80ce9c2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:44670/0 (socket says 172.21.15.12:44670) 2024-09-11T00:19:06.139 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:06.136+0000 7fa2ed000640 1 -- 172.21.15.12:0/3364497756 learned_addr learned my addr 172.21.15.12:0/3364497756 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:19:06.139 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:06.136+0000 7fa2ee400640 1 --2- 172.21.15.12:0/3364497756 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e80cea0790 0x55e80ce96960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:19:06.139 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:06.136+0000 7fa2ee400640 1 --2- 172.21.15.12:0/3364497756 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e80cea5b00 0x55e80ce9c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:19:06.139 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:06.137+0000 7fa2ed000640 1 -- 172.21.15.12:0/3364497756 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e80cea0790 msgr2=0x55e80ce96960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:19:06.139 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:06.137+0000 7fa2ed000640 1 --2- 172.21.15.12:0/3364497756 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e80cea0790 0x55e80ce96960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:19:06.140 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:06.137+0000 7fa2ed000640 1 -- 172.21.15.12:0/3364497756 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e80cea5b00 msgr2=0x55e80ce9c800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:19:06.140 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:06.137+0000 7fa2ed000640 1 --2- 172.21.15.12:0/3364497756 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e80cea5b00 0x55e80ce9c800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:19:06.140 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:06.137+0000 7fa2ed000640 1 -- 172.21.15.12:0/3364497756 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e80ceb7c20 con 0x55e80cea30c0 2024-09-11T00:19:06.140 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:06.137+0000 7fa2ed000640 1 --2- 172.21.15.12:0/3364497756 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e80cea30c0 0x55e80ce9c2c0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x55e80ceb8040 tx=0x7fa2d400b410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac81638913589ebd server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:19:06.140 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:06.137+0000 7fa2da000640 1 -- 172.21.15.12:0/3364497756 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2d4004220 con 0x55e80cea30c0 2024-09-11T00:19:06.140 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:06.137+0000 7fa2da000640 1 -- 172.21.15.12:0/3364497756 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa2d40043c0 con 0x55e80cea30c0 2024-09-11T00:19:06.140 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:06.137+0000 7fa2da000640 1 -- 172.21.15.12:0/3364497756 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2d401da40 con 0x55e80cea30c0 2024-09-11T00:19:06.143 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:06.138+0000 7fa2f3444740 1 -- 172.21.15.12:0/3364497756 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e80cea30c0 msgr2=0x55e80ce9c2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:19:06.143 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:06.138+0000 7fa2f3444740 1 --2- 172.21.15.12:0/3364497756 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e80cea30c0 0x55e80ce9c2c0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x55e80ceb8040 tx=0x7fa2d400b410 comp rx=0 tx=0).stop 2024-09-11T00:19:06.143 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:06.138+0000 7fa2f3444740 1 -- 172.21.15.12:0/3364497756 shutdown_connections 2024-09-11T00:19:06.143 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:06.138+0000 7fa2f3444740 1 --2- 172.21.15.12:0/3364497756 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e80cea5b00 0x55e80ce9c800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:19:06.143 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:06.138+0000 7fa2f3444740 1 --2- 172.21.15.12:0/3364497756 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e80cea30c0 0x55e80ce9c2c0 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:19:06.143 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:06.138+0000 7fa2f3444740 1 --2- 172.21.15.12:0/3364497756 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e80cea0790 0x55e80ce96960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:19:06.143 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:06.138+0000 7fa2f3444740 1 -- 172.21.15.12:0/3364497756 >> 172.21.15.12:0/3364497756 conn(0x55e80ce9b8e0 msgr2=0x55e80cea5ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:19:06.143 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:06.138+0000 7fa2f3444740 1 -- 172.21.15.12:0/3364497756 shutdown_connections 2024-09-11T00:19:06.143 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:06.138+0000 7fa2f3444740 1 -- 172.21.15.12:0/3364497756 wait complete. 2024-09-11T00:19:06.144 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:06.138+0000 7fa2f3444740 1 -- start start 2024-09-11T00:19:06.144 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:06.138+0000 7fa2f3444740 1 Processor -- start 2024-09-11T00:19:06.144 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:06.139+0000 7fa2f3444740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e80cea0790 0x55e80ceb1760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:19:06.144 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:06.139+0000 7fa2f3444740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e80cea30c0 0x55e80ceb0460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:19:06.144 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:06.139+0000 7fa2f3444740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e80cea5b00 0x55e80ceb09a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:19:06.144 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:06.139+0000 7fa2f3444740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fa2e81b32a0 con 0x55e80cea5b00 2024-09-11T00:19:06.144 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:06.139+0000 7fa2f3444740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fa2e81b2b70 con 0x55e80cea30c0 2024-09-11T00:19:06.144 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:06.139+0000 7fa2f3444740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fa2e81afd80 con 0x55e80cea0790 2024-09-11T00:19:06.144 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:06.139+0000 7fa2f3444740 10 client.0 _ll_register_callbacks cb 0x7fa2dc10acf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:19:06.144 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:06.139+0000 7fa2ee400640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e80cea5b00 0x55e80ceb09a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:19:06.144 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:06.139+0000 7fa2ee400640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e80cea5b00 0x55e80ceb09a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:44686/0 (socket says 172.21.15.12:44686) 2024-09-11T00:19:06.144 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:06.139+0000 7fa2ee400640 1 -- 172.21.15.12:0/764949731 learned_addr learned my addr 172.21.15.12:0/764949731 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:19:06.144 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:06.139+0000 7fa2ed000640 1 --2- 172.21.15.12:0/764949731 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e80cea30c0 0x55e80ceb0460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:19:06.145 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:06.139+0000 7fa2ee400640 1 --2- 172.21.15.12:0/764949731 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e80cea0790 0x55e80ceb1760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:19:06.145 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:06.140+0000 7fa2ee400640 1 -- 172.21.15.12:0/764949731 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e80cea0790 msgr2=0x55e80ceb1760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:19:06.145 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:06.140+0000 7fa2ee400640 1 --2- 172.21.15.12:0/764949731 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e80cea0790 0x55e80ceb1760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:19:06.145 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:06.140+0000 7fa2ee400640 1 -- 172.21.15.12:0/764949731 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e80cea30c0 msgr2=0x55e80ceb0460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:19:06.145 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:06.140+0000 7fa2ee400640 1 --2- 172.21.15.12:0/764949731 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e80cea30c0 0x55e80ceb0460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:19:06.145 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:06.140+0000 7fa2ee400640 1 -- 172.21.15.12:0/764949731 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e80ceb0f30 con 0x55e80cea5b00 2024-09-11T00:19:06.145 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:06.140+0000 7fa2ee400640 1 --2- 172.21.15.12:0/764949731 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e80cea5b00 0x55e80ceb09a0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7fa2e0029de0 tx=0x7fa2e002b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb8a014cd49194d1 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:19:06.145 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:06.140+0000 7fa2cf400640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-09-11T00:19:06.145 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:06.140+0000 7fa2cf400640 1 -- 172.21.15.12:0/764949731 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2e00035f0 con 0x55e80cea5b00 2024-09-11T00:19:06.145 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:06.140+0000 7fa2cf400640 1 -- 172.21.15.12:0/764949731 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9208+0+0 (secure 0 0 0) 0x7fa2e0003c20 con 0x55e80cea5b00 2024-09-11T00:19:06.145 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:06.140+0000 7fa2f3444740 1 -- 172.21.15.12:0/764949731 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa2d40043c0 con 0x55e80cea5b00 2024-09-11T00:19:06.145 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:06.140+0000 7fa2cf400640 1 -- 172.21.15.12:0/764949731 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7fa2e0003e20 con 0x55e80cea5b00 2024-09-11T00:19:06.145 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:06.140+0000 7fa2f3444740 10 client.16890 fetch_fsmap learned FSMap version 44 2024-09-11T00:19:06.145 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:06.140+0000 7fa2f3444740 1 -- 172.21.15.12:0/764949731 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x55e80ceafcc0 con 0x55e80cea5b00 2024-09-11T00:19:06.145 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:06.141+0000 7fa2cf400640 1 -- 172.21.15.12:0/764949731 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa2d40043c0 con 0x55e80cea5b00 2024-09-11T00:19:06.146 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:06.141+0000 7fa2cf400640 1 -- 172.21.15.12:0/764949731 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 44) ==== 8382+0+0 (secure 0 0 0) 0x7fa2e00052e0 con 0x55e80cea5b00 2024-09-11T00:19:06.146 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:06.141+0000 7fa2f3444740 10 client.16890 fetch_fsmap finished waiting for FSMap version 44 2024-09-11T00:19:06.146 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:06.141+0000 7fa2f3444740 10 client.16890 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T00:19:06.146 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:06.141+0000 7fa2f3444740 20 client.16890 populate_metadata read hostname 'smithi012' 2024-09-11T00:19:06.146 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:06.141+0000 7fa2f3444740 1 --2- 172.21.15.12:0/764949731 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55e80cf22910 0x55e80cf24d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:19:06.146 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:06.141+0000 7fa2f3444740 4 client.16890 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T00:19:06.146 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:06.141+0000 7fa2f3444740 1 -- 172.21.15.12:0/764949731 --> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55e80ce110b0 con 0x55e80cf22910 2024-09-11T00:19:06.146 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:06.141+0000 7fa2ed000640 1 --2- 172.21.15.12:0/764949731 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55e80cf22910 0x55e80cf24d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:19:06.146 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:06.142+0000 7fa2ed000640 1 --2- 172.21.15.12:0/764949731 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55e80cf22910 0x55e80cf24d30 crc :-1 s=READY pgs=186 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=951d56b89ddddf54 server_cookie=b3fa524a1e7a6d31 in_seq=0 out_seq=0 2024-09-11T00:19:06.146 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:06.142+0000 7fa2cf400640 10 client.16890 ms_handle_connect on v2:172.21.15.165:6838/501312984 2024-09-11T00:19:06.167 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:06.165+0000 7fa2cf400640 1 -- 172.21.15.12:0/764949731 <== mds.0 v2:172.21.15.165:6838/501312984 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55e80ce110b0 con 0x55e80cf22910 2024-09-11T00:19:06.168 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:06.165+0000 7fa2cf400640 10 client.16890 handle_command_reply: tid=0 2024-09-11T00:19:06.168 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:06.165+0000 7fa2f3444740 1 client.16890 shutdown 2024-09-11T00:19:06.168 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:06.166+0000 7fa2f3444740 1 -- 172.21.15.12:0/764949731 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e80cea5b00 msgr2=0x55e80ceb09a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:19:06.168 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:06.166+0000 7fa2f3444740 1 --2- 172.21.15.12:0/764949731 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e80cea5b00 0x55e80ceb09a0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7fa2e0029de0 tx=0x7fa2e002b040 comp rx=0 tx=0).stop 2024-09-11T00:19:06.168 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:06.166+0000 7fa2f3444740 1 -- 172.21.15.12:0/764949731 shutdown_connections 2024-09-11T00:19:06.168 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:06.166+0000 7fa2f3444740 1 --2- 172.21.15.12:0/764949731 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55e80cf22910 0x55e80cf24d30 crc :-1 s=READY pgs=186 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:19:06.168 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:06.166+0000 7fa2f3444740 1 --2- 172.21.15.12:0/764949731 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e80cea5b00 0x55e80ceb09a0 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:19:06.168 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:06.166+0000 7fa2f3444740 1 --2- 172.21.15.12:0/764949731 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e80cea30c0 0x55e80ceb0460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:19:06.168 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:06.166+0000 7fa2f3444740 1 --2- 172.21.15.12:0/764949731 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e80cea0790 0x55e80ceb1760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:19:06.168 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:06.166+0000 7fa2cf400640 0 client.16890 ms_handle_reset on v2:172.21.15.165:6838/501312984 2024-09-11T00:19:06.168 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:06.166+0000 7fa2f3444740 1 -- 172.21.15.12:0/764949731 >> 172.21.15.12:0/764949731 conn(0x55e80ce9b8e0 msgr2=0x55e80ce9c220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:19:06.168 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:06.166+0000 7fa2f3444740 1 -- 172.21.15.12:0/764949731 shutdown_connections 2024-09-11T00:19:06.169 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:06.166+0000 7fa2f3444740 1 -- 172.21.15.12:0/764949731 wait complete. 2024-09-11T00:19:06.169 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:06.166+0000 7fa2f3444740 20 client.16890 trim_cache size 0 max 16384 2024-09-11T00:19:06.169 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:{ 2024-09-11T00:19:06.169 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "message": "", 2024-09-11T00:19:06.169 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "return_code": 0 2024-09-11T00:19:06.170 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:} 2024-09-11T00:19:06.170 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:06.167+0000 7fa2f0400640 1 -- 172.21.15.12:0/3821473160 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fa2d00b52e0 msgr2=0x7fa2d00b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:19:06.170 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:06.167+0000 7fa2f0400640 1 --2- 172.21.15.12:0/3821473160 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fa2d00b52e0 0x7fa2d00b77a0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fa2dc0cd700 tx=0x7fa2dc10ae20 comp rx=0 tx=0).stop 2024-09-11T00:19:06.170 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:06.167+0000 7fa2f0400640 1 -- 172.21.15.12:0/3821473160 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa2e8077a20 msgr2=0x7fa2e819f3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:19:06.170 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:06.167+0000 7fa2f0400640 1 --2- 172.21.15.12:0/3821473160 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa2e8077a20 0x7fa2e819f3b0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7fa2dc000f80 tx=0x7fa2dc003ba0 comp rx=0 tx=0).stop 2024-09-11T00:19:06.170 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:06.167+0000 7fa2f0400640 1 -- 172.21.15.12:0/3821473160 shutdown_connections 2024-09-11T00:19:06.171 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:06.167+0000 7fa2f0400640 1 --2- 172.21.15.12:0/3821473160 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fa2d00b52e0 0x7fa2d00b77a0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:19:06.171 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:06.167+0000 7fa2f0400640 1 --2- 172.21.15.12:0/3821473160 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa2e8079530 0x7fa2e81a0630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:19:06.171 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:06.167+0000 7fa2f0400640 1 --2- 172.21.15.12:0/3821473160 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa2e8078b70 0x7fa2e819fa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:19:06.171 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:06.167+0000 7fa2f0400640 1 --2- 172.21.15.12:0/3821473160 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa2e8077a20 0x7fa2e819f3b0 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:19:06.171 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:06.167+0000 7fa2f0400640 1 -- 172.21.15.12:0/3821473160 >> 172.21.15.12:0/3821473160 conn(0x7fa2e8100470 msgr2=0x7fa2e8101f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:19:06.171 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:06.167+0000 7fa2f0400640 1 -- 172.21.15.12:0/3821473160 shutdown_connections 2024-09-11T00:19:06.171 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:06.167+0000 7fa2f0400640 1 -- 172.21.15.12:0/3821473160 wait complete. 2024-09-11T00:19:06.696 INFO:tasks.workunit.client.0.smithi012.stdout: 1 483326 3.87 MB/sec execute 543 sec latency 16981.995 ms 2024-09-11T00:19:07.696 INFO:tasks.workunit.client.0.smithi012.stdout: 1 483326 3.87 MB/sec execute 544 sec latency 17982.128 ms 2024-09-11T00:19:07.782 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:19:07 smithi165 ceph-mon[35356]: pgmap v760: 129 pgs: 129 active+clean; 109 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 324 KiB/s wr, 45 op/s 2024-09-11T00:19:07.813 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:19:07 smithi012 ceph-mon[30333]: pgmap v760: 129 pgs: 129 active+clean; 109 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 324 KiB/s wr, 45 op/s 2024-09-11T00:19:07.831 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:19:07 smithi178 ceph-mon[35919]: pgmap v760: 129 pgs: 129 active+clean; 109 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 324 KiB/s wr, 45 op/s 2024-09-11T00:19:08.696 INFO:tasks.workunit.client.0.smithi012.stdout: 1 483326 3.86 MB/sec execute 545 sec latency 18982.269 ms 2024-09-11T00:19:09.696 INFO:tasks.workunit.client.0.smithi012.stdout: 1 483326 3.85 MB/sec execute 546 sec latency 19982.413 ms 2024-09-11T00:19:09.782 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:19:09 smithi165 ceph-mon[35356]: pgmap v761: 129 pgs: 129 active+clean; 109 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 298 KiB/s wr, 25 op/s 2024-09-11T00:19:09.813 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:19:09 smithi012 ceph-mon[30333]: pgmap v761: 129 pgs: 129 active+clean; 109 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 298 KiB/s wr, 25 op/s 2024-09-11T00:19:09.831 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:19:09 smithi178 ceph-mon[35919]: pgmap v761: 129 pgs: 129 active+clean; 109 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 298 KiB/s wr, 25 op/s 2024-09-11T00:19:10.696 INFO:tasks.workunit.client.0.smithi012.stdout: 1 483326 3.85 MB/sec execute 547 sec latency 20982.574 ms 2024-09-11T00:19:11.696 INFO:tasks.workunit.client.0.smithi012.stdout: 1 483326 3.84 MB/sec execute 548 sec latency 21982.722 ms 2024-09-11T00:19:11.782 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:19:11 smithi165 ceph-mon[35356]: pgmap v762: 129 pgs: 129 active+clean; 109 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 303 KiB/s wr, 31 op/s 2024-09-11T00:19:11.813 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:19:11 smithi012 ceph-mon[30333]: pgmap v762: 129 pgs: 129 active+clean; 109 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 303 KiB/s wr, 31 op/s 2024-09-11T00:19:11.831 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:19:11 smithi178 ceph-mon[35919]: pgmap v762: 129 pgs: 129 active+clean; 109 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 303 KiB/s wr, 31 op/s 2024-09-11T00:19:11.858 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid ff3ad74a-6fcf-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T00:19:12.252 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:19:12.697 INFO:tasks.workunit.client.0.smithi012.stdout: 1 483326 3.83 MB/sec execute 549 sec latency 22982.847 ms 2024-09-11T00:19:12.828 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:19:12.824+0000 7f8b1c200640 1 -- 172.21.15.12:0/1627862846 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8b141087b0 msgr2=0x7f8b14108bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:19:12.828 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:19:12.824+0000 7f8b1c200640 1 --2- 172.21.15.12:0/1627862846 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8b141087b0 0x7f8b14108bf0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f8b00009860 tx=0x7f8b0002a340 comp rx=0 tx=0).stop 2024-09-11T00:19:12.828 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:19:12.824+0000 7f8b1c200640 1 -- 172.21.15.12:0/1627862846 shutdown_connections 2024-09-11T00:19:12.828 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:19:12.824+0000 7f8b1c200640 1 --2- 172.21.15.12:0/1627862846 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8b14109130 0x7f8b1410f930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:19:12.829 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:19:12.824+0000 7f8b1c200640 1 --2- 172.21.15.12:0/1627862846 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8b141087b0 0x7f8b14108bf0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:19:12.829 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:19:12.824+0000 7f8b1c200640 1 --2- 172.21.15.12:0/1627862846 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8b14102da0 0x7f8b14103180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:19:12.829 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:19:12.824+0000 7f8b1c200640 1 -- 172.21.15.12:0/1627862846 >> 172.21.15.12:0/1627862846 conn(0x7f8b140fe810 msgr2=0x7f8b14100c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:19:12.829 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:19:12.824+0000 7f8b1c200640 1 -- 172.21.15.12:0/1627862846 shutdown_connections 2024-09-11T00:19:12.829 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:19:12.824+0000 7f8b1c200640 1 -- 172.21.15.12:0/1627862846 wait complete. 2024-09-11T00:19:12.829 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:19:12.825+0000 7f8b1c200640 1 Processor -- start 2024-09-11T00:19:12.829 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:19:12.825+0000 7f8b1c200640 1 -- start start 2024-09-11T00:19:12.829 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:19:12.825+0000 7f8b1c200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8b14102da0 0x7f8b1419b460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:19:12.829 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:19:12.825+0000 7f8b1c200640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8b141087b0 0x7f8b1419b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:19:12.829 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:19:12.825+0000 7f8b1c200640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8b14109130 0x7f8b141a0d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:19:12.829 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:19:12.825+0000 7f8b1c200640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f8b14112df0 con 0x7f8b141087b0 2024-09-11T00:19:12.829 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:19:12.825+0000 7f8b1c200640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f8b14112c70 con 0x7f8b14102da0 2024-09-11T00:19:12.830 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:19:12.825+0000 7f8b1c200640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f8b14112f70 con 0x7f8b14109130 2024-09-11T00:19:12.830 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:19:12.826+0000 7f8b18e00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8b141087b0 0x7f8b1419b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:19:12.830 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:19:12.826+0000 7f8b18e00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8b141087b0 0x7f8b1419b9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:40496/0 (socket says 172.21.15.12:40496) 2024-09-11T00:19:12.830 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:19:12.826+0000 7f8b18e00640 1 -- 172.21.15.12:0/3715024558 learned_addr learned my addr 172.21.15.12:0/3715024558 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:19:12.830 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:19:12.826+0000 7f8b1a200640 1 --2- 172.21.15.12:0/3715024558 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8b14109130 0x7f8b141a0d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:19:12.830 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:19:12.826+0000 7f8b19800640 1 --2- 172.21.15.12:0/3715024558 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8b14102da0 0x7f8b1419b460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:19:12.830 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:19:12.826+0000 7f8b1a200640 1 -- 172.21.15.12:0/3715024558 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8b14102da0 msgr2=0x7f8b1419b460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:19:12.830 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:19:12.826+0000 7f8b1a200640 1 --2- 172.21.15.12:0/3715024558 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8b14102da0 0x7f8b1419b460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:19:12.830 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:19:12.826+0000 7f8b1a200640 1 -- 172.21.15.12:0/3715024558 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8b141087b0 msgr2=0x7f8b1419b9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:19:12.830 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:19:12.826+0000 7f8b1a200640 1 --2- 172.21.15.12:0/3715024558 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8b141087b0 0x7f8b1419b9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:19:12.830 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:19:12.826+0000 7f8b1a200640 1 -- 172.21.15.12:0/3715024558 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b141a1510 con 0x7f8b14109130 2024-09-11T00:19:12.830 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:19:12.826+0000 7f8b18e00640 1 --2- 172.21.15.12:0/3715024558 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8b141087b0 0x7f8b1419b9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:19:12.830 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:19:12.827+0000 7f8b1a200640 1 --2- 172.21.15.12:0/3715024558 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8b14109130 0x7f8b141a0d40 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f8b0c00ef00 tx=0x7f8b0c00c740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:19:12.830 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:19:12.827+0000 7f8b12000640 1 -- 172.21.15.12:0/3715024558 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b0c004b20 con 0x7f8b14109130 2024-09-11T00:19:12.830 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:19:12.827+0000 7f8b12000640 1 -- 172.21.15.12:0/3715024558 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8b0c003e20 con 0x7f8b14109130 2024-09-11T00:19:12.831 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:19:12.827+0000 7f8b1c200640 1 -- 172.21.15.12:0/3715024558 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8b141a1800 con 0x7f8b14109130 2024-09-11T00:19:12.831 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:19:12.827+0000 7f8b12000640 1 -- 172.21.15.12:0/3715024558 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b0c00b570 con 0x7f8b14109130 2024-09-11T00:19:12.831 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:19:12.827+0000 7f8b1c200640 1 -- 172.21.15.12:0/3715024558 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8b141a7ec0 con 0x7f8b14109130 2024-09-11T00:19:12.831 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:19:12.828+0000 7f8b1c200640 1 -- 172.21.15.12:0/3715024558 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8b14103180 con 0x7f8b14109130 2024-09-11T00:19:12.832 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:19:12.830+0000 7f8b12000640 1 -- 172.21.15.12:0/3715024558 <== mon.1 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7f8b0c00b040 con 0x7f8b14109130 2024-09-11T00:19:12.833 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:19:12.832+0000 7f8b12000640 1 --2- 172.21.15.12:0/3715024558 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f8afc0b5360 0x7f8afc0b7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:19:12.833 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:19:12.832+0000 7f8b12000640 1 -- 172.21.15.12:0/3715024558 <== mon.1 v2:172.21.15.178:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9208+0+0 (secure 0 0 0) 0x7f8b0c0e5f80 con 0x7f8b14109130 2024-09-11T00:19:12.833 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:19:12.832+0000 7f8b19800640 1 --2- 172.21.15.12:0/3715024558 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f8afc0b5360 0x7f8afc0b7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:19:12.834 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:19:12.833+0000 7f8b19800640 1 --2- 172.21.15.12:0/3715024558 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f8afc0b5360 0x7f8afc0b7820 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f8b08005d00 tx=0x7f8b0800b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:19:12.835 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:19:12.833+0000 7f8b12000640 1 -- 172.21.15.12:0/3715024558 <== mon.1 v2:172.21.15.178:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f8b0c004cc0 con 0x7f8b14109130 2024-09-11T00:19:13.074 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:19:13.072+0000 7f8b1c200640 1 -- 172.21.15.12:0/3715024558 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8b1419c590 con 0x7f8b14109130 2024-09-11T00:19:13.075 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:19:13.073+0000 7f8b12000640 1 -- 172.21.15.12:0/3715024558 <== mon.1 v2:172.21.15.178:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 44 v44) ==== 94+0+11969 (secure 0 0 0) 0x7f8b1419c590 con 0x7f8b14109130 2024-09-11T00:19:13.075 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-11T00:19:13.075 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":44,"btime":"2024-09-11T00:18:17:022132+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14577,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6835/447171591","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":447171591},{"type":"v1","addr":"172.21.15.12:6835","nonce":447171591}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14589,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/901486040","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":901486040},{"type":"v1","addr":"172.21.15.12:6837","nonce":901486040}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":44,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T00:04:04.739881+0000","modified":"2024-09-11T00:18:16.957829+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24674,"mds_1":24643,"mds_2":24638,"mds_3":24637,"mds_4":24614},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14631":{"gid":14631,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.12:6841/3069919437","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6840","nonce":3069919437},{"type":"v1","addr":"172.21.15.12:6841","nonce":3069919437}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24562":{"gid":24562,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6833/1141636112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":1141636112},{"type":"v1","addr":"172.21.15.178:6833","nonce":1141636112}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24581":{"gid":24581,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6833/3111980970","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6832","nonce":3111980970},{"type":"v1","addr":"172.21.15.165:6833","nonce":3111980970}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24607":{"gid":24607,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6835/4084868287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":4084868287},{"type":"v1","addr":"172.21.15.178:6835","nonce":4084868287}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24614":{"gid":24614,"name":"e","rank":4,"incarnation":26,"state":"up:active","state_seq":21,"addr":"172.21.15.165:6835/2425395274","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":2425395274},{"type":"v1","addr":"172.21.15.165:6835","nonce":2425395274}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24637":{"gid":24637,"name":"i","rank":3,"incarnation":24,"state":"up:active","state_seq":18,"addr":"172.21.15.178:6839/70628097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6838","nonce":70628097},{"type":"v1","addr":"172.21.15.178:6839","nonce":70628097}]},"join_fscid":-1,"export_targets":[0],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24638":{"gid":24638,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":18,"addr":"172.21.15.165:6837/6215088","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6836","nonce":6215088},{"type":"v1","addr":"172.21.15.165:6837","nonce":6215088}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24643":{"gid":24643,"name":"g","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.12:6839/3124093009","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6838","nonce":3124093009},{"type":"v1","addr":"172.21.15.12:6839","nonce":3124093009}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24674":{"gid":24674,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.165:6839/501312984","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6838","nonce":501312984},{"type":"v1","addr":"172.21.15.165:6839","nonce":501312984}]},"join_fscid":-1,"export_targets":[1,3],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25672":{"gid":25672,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6837/308601741","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":308601741},{"type":"v1","addr":"172.21.15.178:6837","nonce":308601741}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24674,"qdb_cluster":[24674,24643,24638,24637,24614]},"id":1}]} 2024-09-11T00:19:13.075 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 44 2024-09-11T00:19:13.077 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:19:13.076+0000 7f8b1c200640 1 -- 172.21.15.12:0/3715024558 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f8afc0b5360 msgr2=0x7f8afc0b7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:19:13.078 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:19:13.076+0000 7f8b1c200640 1 --2- 172.21.15.12:0/3715024558 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f8afc0b5360 0x7f8afc0b7820 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f8b08005d00 tx=0x7f8b0800b410 comp rx=0 tx=0).stop 2024-09-11T00:19:13.078 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:19:13.076+0000 7f8b1c200640 1 -- 172.21.15.12:0/3715024558 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8b14109130 msgr2=0x7f8b141a0d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:19:13.078 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:19:13.076+0000 7f8b1c200640 1 --2- 172.21.15.12:0/3715024558 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8b14109130 0x7f8b141a0d40 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f8b0c00ef00 tx=0x7f8b0c00c740 comp rx=0 tx=0).stop 2024-09-11T00:19:13.078 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:19:13.076+0000 7f8b1c200640 1 -- 172.21.15.12:0/3715024558 shutdown_connections 2024-09-11T00:19:13.078 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:19:13.076+0000 7f8b1c200640 1 --2- 172.21.15.12:0/3715024558 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f8afc0b5360 0x7f8afc0b7820 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:19:13.078 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:19:13.076+0000 7f8b1c200640 1 --2- 172.21.15.12:0/3715024558 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8b14109130 0x7f8b141a0d40 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:19:13.078 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:19:13.076+0000 7f8b1c200640 1 --2- 172.21.15.12:0/3715024558 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8b141087b0 0x7f8b1419b9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:19:13.078 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:19:13.076+0000 7f8b1c200640 1 --2- 172.21.15.12:0/3715024558 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8b14102da0 0x7f8b1419b460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:19:13.078 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:19:13.076+0000 7f8b1c200640 1 -- 172.21.15.12:0/3715024558 >> 172.21.15.12:0/3715024558 conn(0x7f8b140fe810 msgr2=0x7f8b14104d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:19:13.078 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:19:13.076+0000 7f8b1c200640 1 -- 172.21.15.12:0/3715024558 shutdown_connections 2024-09-11T00:19:13.078 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:19:13.076+0000 7f8b1c200640 1 -- 172.21.15.12:0/3715024558 wait complete. 2024-09-11T00:19:13.696 INFO:tasks.workunit.client.0.smithi012.stdout: 1 483326 3.82 MB/sec execute 550 sec latency 23982.975 ms 2024-09-11T00:19:13.782 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:19:13 smithi165 ceph-mon[35356]: pgmap v763: 129 pgs: 129 active+clean; 109 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 302 KiB/s wr, 31 op/s 2024-09-11T00:19:13.782 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:19:13 smithi165 ceph-mon[35356]: from='client.? 172.21.15.12:0/3715024558' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T00:19:13.813 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:19:13 smithi012 ceph-mon[30333]: pgmap v763: 129 pgs: 129 active+clean; 109 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 302 KiB/s wr, 31 op/s 2024-09-11T00:19:13.813 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:19:13 smithi012 ceph-mon[30333]: from='client.? 172.21.15.12:0/3715024558' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T00:19:13.831 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:19:13 smithi178 ceph-mon[35919]: pgmap v763: 129 pgs: 129 active+clean; 109 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 302 KiB/s wr, 31 op/s 2024-09-11T00:19:13.831 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:19:13 smithi178 ceph-mon[35919]: from='client.? 172.21.15.12:0/3715024558' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T00:19:13.928 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24674 quiesce db --set-id bd2e38fb --release --await' 2024-09-11T00:19:13.929 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid ff3ad74a-6fcf-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24674 quiesce db --set-id bd2e38fb --release --await 2024-09-11T00:19:14.320 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:19:14.697 INFO:tasks.workunit.client.0.smithi012.stdout: 1 483326 3.82 MB/sec execute 551 sec latency 24983.111 ms 2024-09-11T00:19:14.884 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:19:14.882+0000 7fc06ca00640 1 -- 172.21.15.12:0/3218537025 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc06810b330 msgr2=0x7fc068111b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:19:14.884 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:19:14.882+0000 7fc06ca00640 1 --2- 172.21.15.12:0/3218537025 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc06810b330 0x7fc068111b30 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7fc060009900 tx=0x7fc06002a410 comp rx=0 tx=0).stop 2024-09-11T00:19:14.884 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:19:14.882+0000 7fc06ca00640 1 -- 172.21.15.12:0/3218537025 shutdown_connections 2024-09-11T00:19:14.884 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:19:14.882+0000 7fc06ca00640 1 --2- 172.21.15.12:0/3218537025 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc06810b330 0x7fc068111b30 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:19:14.884 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:19:14.882+0000 7fc06ca00640 1 --2- 172.21.15.12:0/3218537025 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc06810a9b0 0x7fc06810adf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:19:14.884 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:19:14.882+0000 7fc06ca00640 1 --2- 172.21.15.12:0/3218537025 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc068104fa0 0x7fc068105380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:19:14.884 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:19:14.882+0000 7fc06ca00640 1 -- 172.21.15.12:0/3218537025 >> 172.21.15.12:0/3218537025 conn(0x7fc068100a50 msgr2=0x7fc068102e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:19:14.884 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:19:14.882+0000 7fc06ca00640 1 -- 172.21.15.12:0/3218537025 shutdown_connections 2024-09-11T00:19:14.885 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:19:14.882+0000 7fc06ca00640 1 -- 172.21.15.12:0/3218537025 wait complete. 2024-09-11T00:19:14.885 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:19:14.883+0000 7fc06ca00640 1 Processor -- start 2024-09-11T00:19:14.886 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:19:14.884+0000 7fc06ca00640 1 -- start start 2024-09-11T00:19:14.886 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:19:14.884+0000 7fc06ca00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc068104fa0 0x7fc06819b4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:19:14.886 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:19:14.884+0000 7fc06ca00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc06810a9b0 0x7fc06819b9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:19:14.886 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:19:14.884+0000 7fc06ca00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc06810b330 0x7fc0681a0d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:19:14.886 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:19:14.884+0000 7fc06ca00640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fc068114f60 con 0x7fc06810b330 2024-09-11T00:19:14.886 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:19:14.884+0000 7fc06ca00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fc068114de0 con 0x7fc06810a9b0 2024-09-11T00:19:14.886 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:19:14.884+0000 7fc06ca00640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fc0681150e0 con 0x7fc068104fa0 2024-09-11T00:19:14.886 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:19:14.884+0000 7fc05d400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc06810a9b0 0x7fc06819b9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:19:14.886 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:19:14.884+0000 7fc05d400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc06810a9b0 0x7fc06819b9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.12:46284/0 (socket says 172.21.15.12:46284) 2024-09-11T00:19:14.886 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:19:14.884+0000 7fc05d400640 1 -- 172.21.15.12:0/1464763288 learned_addr learned my addr 172.21.15.12:0/1464763288 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:19:14.886 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:19:14.884+0000 7fc066800640 1 --2- 172.21.15.12:0/1464763288 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc06810b330 0x7fc0681a0d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:19:14.886 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:19:14.884+0000 7fc065e00640 1 --2- 172.21.15.12:0/1464763288 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc068104fa0 0x7fc06819b4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:19:14.887 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:19:14.885+0000 7fc066800640 1 -- 172.21.15.12:0/1464763288 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc068104fa0 msgr2=0x7fc06819b4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:19:14.887 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:19:14.885+0000 7fc066800640 1 --2- 172.21.15.12:0/1464763288 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc068104fa0 0x7fc06819b4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:19:14.887 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:19:14.885+0000 7fc066800640 1 -- 172.21.15.12:0/1464763288 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc06810a9b0 msgr2=0x7fc06819b9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:19:14.887 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:19:14.885+0000 7fc066800640 1 --2- 172.21.15.12:0/1464763288 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc06810a9b0 0x7fc06819b9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:19:14.888 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:19:14.885+0000 7fc066800640 1 -- 172.21.15.12:0/1464763288 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0681a1560 con 0x7fc06810b330 2024-09-11T00:19:14.888 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:19:14.885+0000 7fc05d400640 1 --2- 172.21.15.12:0/1464763288 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc06810a9b0 0x7fc06819b9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:19:14.888 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:19:14.885+0000 7fc065e00640 1 --2- 172.21.15.12:0/1464763288 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc068104fa0 0x7fc06819b4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:19:14.888 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:19:14.885+0000 7fc066800640 1 --2- 172.21.15.12:0/1464763288 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc06810b330 0x7fc0681a0d90 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7fc060005940 tx=0x7fc060004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:19:14.888 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:19:14.886+0000 7fc05f400640 1 -- 172.21.15.12:0/1464763288 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc06003f030 con 0x7fc06810b330 2024-09-11T00:19:14.888 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:19:14.886+0000 7fc06ca00640 1 -- 172.21.15.12:0/1464763288 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc0681a17f0 con 0x7fc06810b330 2024-09-11T00:19:14.888 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:19:14.886+0000 7fc05f400640 1 -- 172.21.15.12:0/1464763288 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc060004440 con 0x7fc06810b330 2024-09-11T00:19:14.888 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:19:14.886+0000 7fc05f400640 1 -- 172.21.15.12:0/1464763288 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc060035070 con 0x7fc06810b330 2024-09-11T00:19:14.888 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:19:14.886+0000 7fc06ca00640 1 -- 172.21.15.12:0/1464763288 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0681a7f10 con 0x7fc06810b330 2024-09-11T00:19:14.891 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:19:14.890+0000 7fc06f61b740 1 Processor -- start 2024-09-11T00:19:14.892 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:19:14.890+0000 7fc05f400640 1 -- 172.21.15.12:0/1464763288 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7fc06004c060 con 0x7fc06810b330 2024-09-11T00:19:14.892 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:19:14.891+0000 7fc06f61b740 1 -- start start 2024-09-11T00:19:14.892 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:19:14.891+0000 7fc06f61b740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5604f7d5eff0 0x5604f7d5f3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:19:14.892 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:19:14.891+0000 7fc06f61b740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5604f7d5f910 0x5604f7d6b360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:19:14.892 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:19:14.891+0000 7fc06f61b740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5604f7d6b8a0 0x5604f7d6dc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:19:14.893 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:19:14.891+0000 7fc06f61b740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x5604f7cd6290 con 0x5604f7d6b8a0 2024-09-11T00:19:14.893 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:19:14.891+0000 7fc06f61b740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x5604f7cc56c0 con 0x5604f7d5f910 2024-09-11T00:19:14.893 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:19:14.891+0000 7fc06f61b740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5604f7cc94d0 con 0x5604f7d5eff0 2024-09-11T00:19:14.893 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:19:14.891+0000 7fc05d400640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5604f7d5eff0 0x5604f7d5f3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:19:14.893 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:19:14.891+0000 7fc05d400640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5604f7d5eff0 0x5604f7d5f3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.12:53442/0 (socket says 172.21.15.12:53442) 2024-09-11T00:19:14.893 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:19:14.891+0000 7fc065e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5604f7d5f910 0x5604f7d6b360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:19:14.893 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:19:14.891+0000 7fc05d400640 1 -- 172.21.15.12:0/1220141317 learned_addr learned my addr 172.21.15.12:0/1220141317 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:19:14.893 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:19:14.891+0000 7fc05d400640 1 --2- 172.21.15.12:0/1220141317 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5604f7d6b8a0 0x5604f7d6dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:19:14.894 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:19:14.892+0000 7fc065e00640 1 -- 172.21.15.12:0/1220141317 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5604f7d5eff0 msgr2=0x5604f7d5f3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:19:14.894 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:19:14.892+0000 7fc065e00640 1 --2- 172.21.15.12:0/1220141317 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5604f7d5eff0 0x5604f7d5f3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:19:14.894 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:19:14.892+0000 7fc065e00640 1 -- 172.21.15.12:0/1220141317 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5604f7d6b8a0 msgr2=0x5604f7d6dc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:19:14.894 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:19:14.892+0000 7fc065e00640 1 --2- 172.21.15.12:0/1220141317 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5604f7d6b8a0 0x5604f7d6dc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:19:14.894 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:19:14.892+0000 7fc065e00640 1 -- 172.21.15.12:0/1220141317 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5604f7c52460 con 0x5604f7d5f910 2024-09-11T00:19:14.895 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:19:14.892+0000 7fc05d400640 1 --2- 172.21.15.12:0/1220141317 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5604f7d5eff0 0x5604f7d5f3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:19:14.895 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:19:14.892+0000 7fc065e00640 1 --2- 172.21.15.12:0/1220141317 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5604f7d5f910 0x5604f7d6b360 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fc058010770 tx=0x7fc058010c30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c01b0ba55ecd361 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:19:14.895 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:19:14.892+0000 7fc05f400640 1 --2- 172.21.15.12:0/1464763288 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fc04c0b52e0 0x7fc04c0b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:19:14.895 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:19:14.892+0000 7fc057000640 1 -- 172.21.15.12:0/1220141317 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc058004d20 con 0x5604f7d5f910 2024-09-11T00:19:14.895 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:19:14.892+0000 7fc05f400640 1 -- 172.21.15.12:0/1464763288 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9208+0+0 (secure 0 0 0) 0x7fc060105ef0 con 0x7fc06810b330 2024-09-11T00:19:14.895 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:19:14.893+0000 7fc057000640 1 -- 172.21.15.12:0/1220141317 <== mon.2 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc05800ede0 con 0x5604f7d5f910 2024-09-11T00:19:14.896 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:19:14.893+0000 7fc057000640 1 -- 172.21.15.12:0/1220141317 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0580057f0 con 0x5604f7d5f910 2024-09-11T00:19:14.896 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:19:14.893+0000 7fc05d400640 1 --2- 172.21.15.12:0/1464763288 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fc04c0b52e0 0x7fc04c0b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:19:14.898 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:19:14.893+0000 7fc06f61b740 1 -- 172.21.15.12:0/1220141317 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5604f7d5f910 msgr2=0x5604f7d6b360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:19:14.898 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:19:14.893+0000 7fc05d400640 1 --2- 172.21.15.12:0/1464763288 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fc04c0b52e0 0x7fc04c0b77a0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x5604f7d5efa0 tx=0x7fc050005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:19:14.898 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:19:14.893+0000 7fc06f61b740 1 --2- 172.21.15.12:0/1220141317 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5604f7d5f910 0x5604f7d6b360 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fc058010770 tx=0x7fc058010c30 comp rx=0 tx=0).stop 2024-09-11T00:19:14.898 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:19:14.893+0000 7fc06f61b740 1 -- 172.21.15.12:0/1220141317 shutdown_connections 2024-09-11T00:19:14.898 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:19:14.893+0000 7fc06f61b740 1 --2- 172.21.15.12:0/1220141317 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5604f7d6b8a0 0x5604f7d6dc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:19:14.898 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:19:14.893+0000 7fc06f61b740 1 --2- 172.21.15.12:0/1220141317 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5604f7d5f910 0x5604f7d6b360 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:19:14.898 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:19:14.893+0000 7fc06f61b740 1 --2- 172.21.15.12:0/1220141317 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5604f7d5eff0 0x5604f7d5f3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:19:14.898 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:19:14.893+0000 7fc06f61b740 1 -- 172.21.15.12:0/1220141317 >> 172.21.15.12:0/1220141317 conn(0x5604f7d5ce20 msgr2=0x5604f7d5dfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:19:14.898 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:19:14.894+0000 7fc06f61b740 1 -- 172.21.15.12:0/1220141317 shutdown_connections 2024-09-11T00:19:14.898 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:19:14.894+0000 7fc06f61b740 1 -- 172.21.15.12:0/1220141317 wait complete. 2024-09-11T00:19:14.899 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:19:14.894+0000 7fc06f61b740 1 -- start start 2024-09-11T00:19:14.899 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:19:14.894+0000 7fc06f61b740 1 Processor -- start 2024-09-11T00:19:14.899 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:19:14.895+0000 7fc06f61b740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5604f7d5eff0 0x5604f7d687d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:19:14.899 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:19:14.895+0000 7fc06f61b740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5604f7d5f910 0x5604f7d68d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:19:14.899 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:19:14.895+0000 7fc06f61b740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5604f7d6b8a0 0x5604f7d69250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:19:14.899 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:19:14.895+0000 7fc06f61b740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x5604f7b26bb0 con 0x5604f7d6b8a0 2024-09-11T00:19:14.899 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:19:14.895+0000 7fc06f61b740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x5604f7c606c0 con 0x5604f7d5f910 2024-09-11T00:19:14.899 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:19:14.895+0000 7fc06f61b740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5604f7d6e550 con 0x5604f7d5eff0 2024-09-11T00:19:14.899 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:19:14.895+0000 7fc06f61b740 10 client.0 _ll_register_callbacks cb 0x5604f7c7b490 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:19:14.899 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:19:14.895+0000 7fc05d400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5604f7d5f910 0x5604f7d68d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:19:14.899 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:19:14.895+0000 7fc066800640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5604f7d6b8a0 0x5604f7d69250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:19:14.899 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:19:14.895+0000 7fc065e00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5604f7d5eff0 0x5604f7d687d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:19:14.899 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:19:14.895+0000 7fc05d400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5604f7d5f910 0x5604f7d68d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.12:46306/0 (socket says 172.21.15.12:46306) 2024-09-11T00:19:14.900 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:19:14.895+0000 7fc05d400640 1 -- 172.21.15.12:0/2956618255 learned_addr learned my addr 172.21.15.12:0/2956618255 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:19:14.900 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:19:14.896+0000 7fc065e00640 1 -- 172.21.15.12:0/2956618255 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5604f7d5f910 msgr2=0x5604f7d68d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:19:14.900 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:19:14.896+0000 7fc065e00640 1 --2- 172.21.15.12:0/2956618255 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5604f7d5f910 0x5604f7d68d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:19:14.900 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:19:14.896+0000 7fc065e00640 1 -- 172.21.15.12:0/2956618255 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5604f7d6b8a0 msgr2=0x5604f7d69250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:19:14.900 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:19:14.896+0000 7fc065e00640 1 --2- 172.21.15.12:0/2956618255 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5604f7d6b8a0 0x5604f7d69250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:19:14.900 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:19:14.896+0000 7fc065e00640 1 -- 172.21.15.12:0/2956618255 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5604f7a23b00 con 0x5604f7d5eff0 2024-09-11T00:19:14.900 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:19:14.896+0000 7fc065e00640 1 --2- 172.21.15.12:0/2956618255 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5604f7d5eff0 0x5604f7d687d0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fc058010770 tx=0x7fc058009520 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6cd453579c0795ab server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:19:14.900 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:19:14.896+0000 7fc04b400640 10 client.0 ms_handle_connect on v2:172.21.15.178:3300/0 2024-09-11T00:19:14.900 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:19:14.896+0000 7fc04b400640 1 -- 172.21.15.12:0/2956618255 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc058027020 con 0x5604f7d5eff0 2024-09-11T00:19:14.900 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:19:14.896+0000 7fc06f61b740 1 -- 172.21.15.12:0/2956618255 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc05800ede0 con 0x5604f7d5eff0 2024-09-11T00:19:14.900 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:19:14.896+0000 7fc04b400640 1 -- 172.21.15.12:0/2956618255 <== mon.1 v2:172.21.15.178:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9208+0+0 (secure 0 0 0) 0x7fc0580054f0 con 0x5604f7d5eff0 2024-09-11T00:19:14.900 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:19:14.897+0000 7fc04b400640 1 -- 172.21.15.12:0/2956618255 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7fc0580056f0 con 0x5604f7d5eff0 2024-09-11T00:19:14.900 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:19:14.897+0000 7fc06f61b740 10 client.26887 fetch_fsmap learned FSMap version 44 2024-09-11T00:19:14.901 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:19:14.897+0000 7fc06f61b740 1 -- 172.21.15.12:0/2956618255 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x5604f7a9a960 con 0x5604f7d5eff0 2024-09-11T00:19:14.902 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:19:14.897+0000 7fc04b400640 1 -- 172.21.15.12:0/2956618255 <== mon.1 v2:172.21.15.178:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc05800ede0 con 0x5604f7d5eff0 2024-09-11T00:19:14.902 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:19:14.897+0000 7fc04b400640 1 -- 172.21.15.12:0/2956618255 <== mon.1 v2:172.21.15.178:3300/0 5 ==== fsmap(e 44) ==== 8382+0+0 (secure 0 0 0) 0x7fc05801a6b0 con 0x5604f7d5eff0 2024-09-11T00:19:14.902 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:19:14.897+0000 7fc06f61b740 10 client.26887 fetch_fsmap finished waiting for FSMap version 44 2024-09-11T00:19:14.902 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:19:14.897+0000 7fc06f61b740 10 client.26887 resolve_mds: validated gid 24674 aka daemon mds.b 2024-09-11T00:19:14.902 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:19:14.897+0000 7fc06f61b740 20 client.26887 populate_metadata read hostname 'smithi012' 2024-09-11T00:19:14.903 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:19:14.897+0000 7fc06f61b740 1 --2- 172.21.15.12:0/2956618255 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x5604f7d7b330 0x5604f7d7d720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:19:14.903 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:19:14.897+0000 7fc06f61b740 4 client.26887 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T00:19:14.903 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:19:14.897+0000 7fc06f61b740 1 -- 172.21.15.12:0/2956618255 --> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5604f7cd81e0 con 0x5604f7d7b330 2024-09-11T00:19:14.903 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:19:14.898+0000 7fc05d400640 1 --2- 172.21.15.12:0/2956618255 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x5604f7d7b330 0x5604f7d7d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:19:14.903 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:19:14.898+0000 7fc05d400640 1 --2- 172.21.15.12:0/2956618255 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x5604f7d7b330 0x5604f7d7d720 crc :-1 s=READY pgs=187 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2128adb566a67aa1 server_cookie=dc3c2cbdf44d9b51 in_seq=0 out_seq=0 2024-09-11T00:19:14.903 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:19:14.898+0000 7fc04b400640 10 client.26887 ms_handle_connect on v2:172.21.15.165:6838/501312984 2024-09-11T00:19:14.903 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:19:14.899+0000 7fc04b400640 1 -- 172.21.15.12:0/2956618255 <== mds.0 v2:172.21.15.165:6838/501312984 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5604f7cd81e0 con 0x5604f7d7b330 2024-09-11T00:19:14.903 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:19:14.899+0000 7fc04b400640 10 client.26887 handle_command_reply: tid=0 2024-09-11T00:19:14.903 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:19:14.899+0000 7fc06f61b740 1 client.26887 shutdown 2024-09-11T00:19:14.903 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:19:14.900+0000 7fc06f61b740 1 -- 172.21.15.12:0/2956618255 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5604f7d5eff0 msgr2=0x5604f7d687d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:19:14.903 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:19:14.900+0000 7fc06f61b740 1 --2- 172.21.15.12:0/2956618255 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5604f7d5eff0 0x5604f7d687d0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fc058010770 tx=0x7fc058009520 comp rx=0 tx=0).stop 2024-09-11T00:19:14.903 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:19:14.900+0000 7fc06f61b740 1 -- 172.21.15.12:0/2956618255 shutdown_connections 2024-09-11T00:19:14.903 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:19:14.900+0000 7fc06f61b740 1 --2- 172.21.15.12:0/2956618255 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x5604f7d7b330 0x5604f7d7d720 crc :-1 s=READY pgs=187 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:19:14.903 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:19:14.900+0000 7fc06f61b740 1 --2- 172.21.15.12:0/2956618255 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5604f7d6b8a0 0x5604f7d69250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:19:14.904 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:19:14.900+0000 7fc06f61b740 1 --2- 172.21.15.12:0/2956618255 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5604f7d5f910 0x5604f7d68d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:19:14.904 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:19:14.900+0000 7fc06f61b740 1 --2- 172.21.15.12:0/2956618255 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5604f7d5eff0 0x5604f7d687d0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:19:14.904 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:19:14.900+0000 7fc04b400640 0 client.26887 ms_handle_reset on v2:172.21.15.165:6838/501312984 2024-09-11T00:19:14.904 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:19:14.900+0000 7fc06f61b740 1 -- 172.21.15.12:0/2956618255 >> 172.21.15.12:0/2956618255 conn(0x5604f7d5ce20 msgr2=0x5604f7d629c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:19:14.904 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:19:14.900+0000 7fc06f61b740 1 -- 172.21.15.12:0/2956618255 shutdown_connections 2024-09-11T00:19:14.904 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:19:14.900+0000 7fc06f61b740 1 -- 172.21.15.12:0/2956618255 wait complete. 2024-09-11T00:19:14.904 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:19:14.900+0000 7fc06f61b740 20 client.26887 trim_cache size 0 max 16384 2024-09-11T00:19:14.919 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:19:14.918+0000 7fc06f61b740 1 Processor -- start 2024-09-11T00:19:14.919 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:19:14.918+0000 7fc06f61b740 1 -- start start 2024-09-11T00:19:14.920 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:19:14.918+0000 7fc06f61b740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5604f7d5eff0 0x5604f7d630f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:19:14.920 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:19:14.918+0000 7fc06f61b740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5604f7d5f910 0x5604f7d65a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:19:14.920 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:19:14.918+0000 7fc06f61b740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5604f7d6b8a0 0x5604f7d65f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:19:14.920 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:19:14.918+0000 7fc06f61b740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fc0681b1e20 con 0x5604f7d5eff0 2024-09-11T00:19:14.920 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:19:14.918+0000 7fc06f61b740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fc0681b39f0 con 0x5604f7d5f910 2024-09-11T00:19:14.920 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:19:14.918+0000 7fc06f61b740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fc0681b2520 con 0x5604f7d6b8a0 2024-09-11T00:19:14.920 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:19:14.919+0000 7fc065e00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5604f7d5eff0 0x5604f7d630f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:19:14.921 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:19:14.919+0000 7fc065e00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5604f7d5eff0 0x5604f7d630f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:40550/0 (socket says 172.21.15.12:40550) 2024-09-11T00:19:14.921 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:19:14.919+0000 7fc065e00640 1 -- 172.21.15.12:0/4102071617 learned_addr learned my addr 172.21.15.12:0/4102071617 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:19:14.921 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:19:14.919+0000 7fc05d400640 1 --2- 172.21.15.12:0/4102071617 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5604f7d5f910 0x5604f7d65a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:19:14.921 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:19:14.919+0000 7fc066800640 1 --2- 172.21.15.12:0/4102071617 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5604f7d6b8a0 0x5604f7d65f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:19:14.922 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:19:14.919+0000 7fc065e00640 1 -- 172.21.15.12:0/4102071617 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5604f7d6b8a0 msgr2=0x5604f7d65f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:19:14.922 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:19:14.919+0000 7fc065e00640 1 --2- 172.21.15.12:0/4102071617 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5604f7d6b8a0 0x5604f7d65f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:19:14.922 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:19:14.919+0000 7fc065e00640 1 -- 172.21.15.12:0/4102071617 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5604f7d5f910 msgr2=0x5604f7d65a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:19:14.922 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:19:14.919+0000 7fc065e00640 1 --2- 172.21.15.12:0/4102071617 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5604f7d5f910 0x5604f7d65a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:19:14.923 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:19:14.919+0000 7fc065e00640 1 -- 172.21.15.12:0/4102071617 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5604f7d5d8a0 con 0x5604f7d5eff0 2024-09-11T00:19:14.923 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:19:14.919+0000 7fc05d400640 1 --2- 172.21.15.12:0/4102071617 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5604f7d5f910 0x5604f7d65a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:19:14.923 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:19:14.920+0000 7fc065e00640 1 --2- 172.21.15.12:0/4102071617 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5604f7d5eff0 0x5604f7d630f0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7fc058006010 tx=0x7fc058005ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8cc087dd355b92b server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:19:14.923 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:19:14.920+0000 7fc056600640 1 -- 172.21.15.12:0/4102071617 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc058027070 con 0x5604f7d5eff0 2024-09-11T00:19:14.923 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:19:14.920+0000 7fc056600640 1 -- 172.21.15.12:0/4102071617 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc05801a480 con 0x5604f7d5eff0 2024-09-11T00:19:14.923 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:19:14.920+0000 7fc056600640 1 -- 172.21.15.12:0/4102071617 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc058003bd0 con 0x5604f7d5eff0 2024-09-11T00:19:14.923 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:19:14.920+0000 7fc06f61b740 1 -- 172.21.15.12:0/4102071617 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5604f7d5eff0 msgr2=0x5604f7d630f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:19:14.923 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:19:14.920+0000 7fc06f61b740 1 --2- 172.21.15.12:0/4102071617 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5604f7d5eff0 0x5604f7d630f0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7fc058006010 tx=0x7fc058005ae0 comp rx=0 tx=0).stop 2024-09-11T00:19:14.923 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:19:14.921+0000 7fc06f61b740 1 -- 172.21.15.12:0/4102071617 shutdown_connections 2024-09-11T00:19:14.923 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:19:14.921+0000 7fc06f61b740 1 --2- 172.21.15.12:0/4102071617 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5604f7d6b8a0 0x5604f7d65f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:19:14.923 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:19:14.921+0000 7fc06f61b740 1 --2- 172.21.15.12:0/4102071617 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5604f7d5f910 0x5604f7d65a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:19:14.923 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:19:14.921+0000 7fc06f61b740 1 --2- 172.21.15.12:0/4102071617 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5604f7d5eff0 0x5604f7d630f0 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:19:14.923 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:19:14.921+0000 7fc06f61b740 1 -- 172.21.15.12:0/4102071617 >> 172.21.15.12:0/4102071617 conn(0x5604f7d5ce20 msgr2=0x5604f7d69e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:19:14.924 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:19:14.921+0000 7fc06f61b740 1 -- 172.21.15.12:0/4102071617 shutdown_connections 2024-09-11T00:19:14.924 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:19:14.921+0000 7fc06f61b740 1 -- 172.21.15.12:0/4102071617 wait complete. 2024-09-11T00:19:14.924 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:19:14.921+0000 7fc06f61b740 1 -- start start 2024-09-11T00:19:14.924 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:19:14.921+0000 7fc06f61b740 1 Processor -- start 2024-09-11T00:19:14.926 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:19:14.922+0000 7fc06f61b740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5604f7d5eff0 0x5604f7d73aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:19:14.926 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:19:14.922+0000 7fc06f61b740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5604f7d5f910 0x5604f7d74040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:19:14.926 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:19:14.922+0000 7fc06f61b740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5604f7d6b8a0 0x5604f7d70820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:19:14.926 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:19:14.922+0000 7fc06f61b740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fc0681b3320 con 0x5604f7d5eff0 2024-09-11T00:19:14.926 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:19:14.922+0000 7fc06f61b740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fc0681b2bf0 con 0x5604f7d6b8a0 2024-09-11T00:19:14.927 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:19:14.922+0000 7fc06f61b740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fc0681afe00 con 0x5604f7d5f910 2024-09-11T00:19:14.927 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:19:14.922+0000 7fc06f61b740 10 client.0 _ll_register_callbacks cb 0x7fc050011710 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:19:14.927 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:19:14.922+0000 7fc065e00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5604f7d5eff0 0x5604f7d73aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:19:14.927 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:19:14.922+0000 7fc065e00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5604f7d5eff0 0x5604f7d73aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:40562/0 (socket says 172.21.15.12:40562) 2024-09-11T00:19:14.927 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:19:14.922+0000 7fc066800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5604f7d6b8a0 0x5604f7d70820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:19:14.927 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:19:14.922+0000 7fc065e00640 1 -- 172.21.15.12:0/329397502 learned_addr learned my addr 172.21.15.12:0/329397502 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:19:14.927 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:19:14.922+0000 7fc05d400640 1 --2- 172.21.15.12:0/329397502 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5604f7d5f910 0x5604f7d74040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:19:14.927 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:19:14.923+0000 7fc066800640 1 -- 172.21.15.12:0/329397502 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5604f7d5f910 msgr2=0x5604f7d74040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:19:14.927 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:19:14.923+0000 7fc066800640 1 --2- 172.21.15.12:0/329397502 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5604f7d5f910 0x5604f7d74040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:19:14.927 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:19:14.923+0000 7fc066800640 1 -- 172.21.15.12:0/329397502 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5604f7d5eff0 msgr2=0x5604f7d73aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:19:14.927 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:19:14.923+0000 7fc066800640 1 --2- 172.21.15.12:0/329397502 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5604f7d5eff0 0x5604f7d73aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:19:14.927 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:19:14.923+0000 7fc066800640 1 -- 172.21.15.12:0/329397502 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5604f7d71050 con 0x5604f7d6b8a0 2024-09-11T00:19:14.927 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:19:14.923+0000 7fc065e00640 1 --2- 172.21.15.12:0/329397502 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5604f7d5eff0 0x5604f7d73aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:19:14.927 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:19:14.923+0000 7fc066800640 1 --2- 172.21.15.12:0/329397502 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5604f7d6b8a0 0x5604f7d70820 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fc0601035c0 tx=0x7fc0600cbc80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a298c2c3abd50afc server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:19:14.928 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:19:14.923+0000 7fc04b400640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-09-11T00:19:14.928 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:19:14.923+0000 7fc04b400640 1 -- 172.21.15.12:0/329397502 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc060049040 con 0x5604f7d6b8a0 2024-09-11T00:19:14.928 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:19:14.923+0000 7fc06f61b740 1 -- 172.21.15.12:0/329397502 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc05801a480 con 0x5604f7d6b8a0 2024-09-11T00:19:14.928 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:19:14.923+0000 7fc04b400640 1 -- 172.21.15.12:0/329397502 <== mon.2 v2:172.21.15.165:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9208+0+0 (secure 0 0 0) 0x7fc0600cc590 con 0x5604f7d6b8a0 2024-09-11T00:19:14.928 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:19:14.923+0000 7fc04b400640 1 -- 172.21.15.12:0/329397502 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7fc0600cbe40 con 0x5604f7d6b8a0 2024-09-11T00:19:14.928 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:19:14.923+0000 7fc06f61b740 10 client.26918 fetch_fsmap learned FSMap version 44 2024-09-11T00:19:14.928 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:19:14.923+0000 7fc06f61b740 1 -- 172.21.15.12:0/329397502 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x5604f7de8d10 con 0x5604f7d6b8a0 2024-09-11T00:19:14.928 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:19:14.924+0000 7fc04b400640 1 -- 172.21.15.12:0/329397502 <== mon.2 v2:172.21.15.165:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc05801a480 con 0x5604f7d6b8a0 2024-09-11T00:19:14.928 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:19:14.924+0000 7fc04b400640 1 -- 172.21.15.12:0/329397502 <== mon.2 v2:172.21.15.165:3300/0 5 ==== fsmap(e 44) ==== 8382+0+0 (secure 0 0 0) 0x7fc0600d5380 con 0x5604f7d6b8a0 2024-09-11T00:19:14.928 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:19:14.924+0000 7fc06f61b740 10 client.26918 fetch_fsmap finished waiting for FSMap version 44 2024-09-11T00:19:14.928 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:19:14.924+0000 7fc06f61b740 10 client.26918 resolve_mds: validated gid 24674 aka daemon mds.b 2024-09-11T00:19:14.928 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:19:14.924+0000 7fc06f61b740 20 client.26918 populate_metadata read hostname 'smithi012' 2024-09-11T00:19:14.928 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:19:14.924+0000 7fc06f61b740 1 --2- 172.21.15.12:0/329397502 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x5604f7de94a0 0x5604f7deb8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:19:14.929 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:19:14.924+0000 7fc06f61b740 4 client.26918 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "bd2e38fb", "release": true, "await": true}] 2024-09-11T00:19:14.929 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:19:14.924+0000 7fc06f61b740 1 -- 172.21.15.12:0/329397502 --> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] -- command(tid 0: {"prefix": "quiesce db", "set_id": "bd2e38fb", "release": true, "await": true}) -- 0x5604f7cd3c90 con 0x5604f7de94a0 2024-09-11T00:19:14.929 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:19:14.924+0000 7fc065e00640 1 --2- 172.21.15.12:0/329397502 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x5604f7de94a0 0x5604f7deb8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:19:14.929 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:19:14.925+0000 7fc065e00640 1 --2- 172.21.15.12:0/329397502 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x5604f7de94a0 0x5604f7deb8c0 crc :-1 s=READY pgs=188 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4c7ac425eddd7a08 server_cookie=e2e31e45ce280b74 in_seq=0 out_seq=0 2024-09-11T00:19:14.929 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:19:14.925+0000 7fc04b400640 10 client.26918 ms_handle_connect on v2:172.21.15.165:6838/501312984 2024-09-11T00:19:14.933 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:19:14.931+0000 7fc04b400640 1 -- 172.21.15.12:0/329397502 <== mds.0 v2:172.21.15.165:6838/501312984 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x5604f7cd3c90 con 0x5604f7de94a0 2024-09-11T00:19:14.933 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:19:14.931+0000 7fc04b400640 10 client.26918 handle_command_reply: tid=0 2024-09-11T00:19:14.933 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:19:14.931+0000 7fc06f61b740 1 client.26918 shutdown 2024-09-11T00:19:14.933 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:19:14.931+0000 7fc06f61b740 1 -- 172.21.15.12:0/329397502 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5604f7d6b8a0 msgr2=0x5604f7d70820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:19:14.933 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:19:14.932+0000 7fc06f61b740 1 --2- 172.21.15.12:0/329397502 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5604f7d6b8a0 0x5604f7d70820 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fc0601035c0 tx=0x7fc0600cbc80 comp rx=0 tx=0).stop 2024-09-11T00:19:14.933 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:19:14.932+0000 7fc06f61b740 1 -- 172.21.15.12:0/329397502 shutdown_connections 2024-09-11T00:19:14.933 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:19:14.932+0000 7fc06f61b740 1 --2- 172.21.15.12:0/329397502 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x5604f7de94a0 0x5604f7deb8c0 crc :-1 s=READY pgs=188 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:19:14.934 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:19:14.932+0000 7fc06f61b740 1 --2- 172.21.15.12:0/329397502 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5604f7d6b8a0 0x5604f7d70820 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:19:14.934 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:19:14.932+0000 7fc06f61b740 1 --2- 172.21.15.12:0/329397502 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5604f7d5f910 0x5604f7d74040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:19:14.934 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:19:14.932+0000 7fc06f61b740 1 --2- 172.21.15.12:0/329397502 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5604f7d5eff0 0x5604f7d73aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:19:14.934 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:19:14.932+0000 7fc06f61b740 1 -- 172.21.15.12:0/329397502 >> 172.21.15.12:0/329397502 conn(0x5604f7d5ce20 msgr2=0x5604f7d67050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:19:14.934 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:19:14.932+0000 7fc04b400640 0 client.26918 ms_handle_reset on v2:172.21.15.165:6838/501312984 2024-09-11T00:19:14.934 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:19:14.932+0000 7fc06f61b740 1 -- 172.21.15.12:0/329397502 shutdown_connections 2024-09-11T00:19:14.934 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:19:14.932+0000 7fc06f61b740 1 -- 172.21.15.12:0/329397502 wait complete. 2024-09-11T00:19:14.934 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:19:14.932+0000 7fc06f61b740 20 client.26918 trim_cache size 0 max 16384 2024-09-11T00:19:14.935 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-09-11T00:19:14.935 INFO:teuthology.orchestra.run.smithi012.stdout: "epoch": 44, 2024-09-11T00:19:14.935 INFO:teuthology.orchestra.run.smithi012.stdout: "leader": 24674, 2024-09-11T00:19:14.935 INFO:teuthology.orchestra.run.smithi012.stdout: "set_version": 40, 2024-09-11T00:19:14.935 INFO:teuthology.orchestra.run.smithi012.stdout: "sets": { 2024-09-11T00:19:14.935 INFO:teuthology.orchestra.run.smithi012.stdout: "bd2e38fb": { 2024-09-11T00:19:14.935 INFO:teuthology.orchestra.run.smithi012.stdout: "version": 40, 2024-09-11T00:19:14.935 INFO:teuthology.orchestra.run.smithi012.stdout: "age_ref": 0.0, 2024-09-11T00:19:14.935 INFO:teuthology.orchestra.run.smithi012.stdout: "state": { 2024-09-11T00:19:14.935 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "RELEASED", 2024-09-11T00:19:14.935 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 0.0 2024-09-11T00:19:14.935 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:19:14.935 INFO:teuthology.orchestra.run.smithi012.stdout: "timeout": 90.0, 2024-09-11T00:19:14.936 INFO:teuthology.orchestra.run.smithi012.stdout: "expiration": 145.4, 2024-09-11T00:19:14.936 INFO:teuthology.orchestra.run.smithi012.stdout: "members": { 2024-09-11T00:19:14.936 INFO:teuthology.orchestra.run.smithi012.stdout: "file:/": { 2024-09-11T00:19:14.936 INFO:teuthology.orchestra.run.smithi012.stdout: "excluded": false, 2024-09-11T00:19:14.936 INFO:teuthology.orchestra.run.smithi012.stdout: "state": { 2024-09-11T00:19:14.936 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "RELEASED", 2024-09-11T00:19:14.936 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 0.0 2024-09-11T00:19:14.936 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:19:14.936 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:19:14.936 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:19:14.936 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:19:14.936 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:19:14.936 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-09-11T00:19:14.937 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:19:14.933+0000 7fc06ca00640 1 -- 172.21.15.12:0/1464763288 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fc04c0b52e0 msgr2=0x7fc04c0b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:19:14.937 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:19:14.933+0000 7fc06ca00640 1 --2- 172.21.15.12:0/1464763288 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fc04c0b52e0 0x7fc04c0b77a0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x5604f7d5efa0 tx=0x7fc050005f90 comp rx=0 tx=0).stop 2024-09-11T00:19:14.937 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:19:14.933+0000 7fc06ca00640 1 -- 172.21.15.12:0/1464763288 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc06810b330 msgr2=0x7fc0681a0d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:19:14.937 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:19:14.933+0000 7fc06ca00640 1 --2- 172.21.15.12:0/1464763288 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc06810b330 0x7fc0681a0d90 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7fc060005940 tx=0x7fc060004040 comp rx=0 tx=0).stop 2024-09-11T00:19:14.937 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:19:14.933+0000 7fc06ca00640 1 -- 172.21.15.12:0/1464763288 shutdown_connections 2024-09-11T00:19:14.937 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:19:14.933+0000 7fc06ca00640 1 --2- 172.21.15.12:0/1464763288 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fc04c0b52e0 0x7fc04c0b77a0 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:19:14.937 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:19:14.933+0000 7fc06ca00640 1 --2- 172.21.15.12:0/1464763288 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc06810b330 0x7fc0681a0d90 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:19:14.937 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:19:14.933+0000 7fc06ca00640 1 --2- 172.21.15.12:0/1464763288 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc06810a9b0 0x7fc06819b9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:19:14.938 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:19:14.933+0000 7fc06ca00640 1 --2- 172.21.15.12:0/1464763288 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc068104fa0 0x7fc06819b4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:19:14.938 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:19:14.933+0000 7fc06ca00640 1 -- 172.21.15.12:0/1464763288 >> 172.21.15.12:0/1464763288 conn(0x7fc068100a50 msgr2=0x7fc068102e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:19:14.938 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:19:14.933+0000 7fc06ca00640 1 -- 172.21.15.12:0/1464763288 shutdown_connections 2024-09-11T00:19:14.938 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:19:14.933+0000 7fc06ca00640 1 -- 172.21.15.12:0/1464763288 wait complete. 2024-09-11T00:19:15.563 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:19:15 smithi012 ceph-mon[30333]: pgmap v764: 129 pgs: 129 active+clean; 109 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 300 KiB/s wr, 28 op/s 2024-09-11T00:19:15.696 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'bd2e38fb', total seconds elapsed: 29.2 2024-09-11T00:19:15.696 INFO:tasks.workunit.client.0.smithi012.stdout: 1 483912 3.81 MB/sec execute 552 sec latency 25224.651 ms 2024-09-11T00:19:15.782 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:19:15 smithi165 ceph-mon[35356]: pgmap v764: 129 pgs: 129 active+clean; 109 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 300 KiB/s wr, 28 op/s 2024-09-11T00:19:15.831 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:19:15 smithi178 ceph-mon[35919]: pgmap v764: 129 pgs: 129 active+clean; 109 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 300 KiB/s wr, 28 op/s 2024-09-11T00:19:16.697 INFO:tasks.workunit.client.0.smithi012.stdout: 1 485507 3.82 MB/sec execute 553 sec latency 213.379 ms 2024-09-11T00:19:17.697 INFO:tasks.workunit.client.0.smithi012.stdout: 1 486701 3.83 MB/sec execute 554 sec latency 215.275 ms 2024-09-11T00:19:17.782 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:19:17 smithi165 ceph-mon[35356]: pgmap v765: 129 pgs: 129 active+clean; 109 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 207 KiB/s rd, 318 KiB/s wr, 38 op/s 2024-09-11T00:19:17.813 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:19:17 smithi012 ceph-mon[30333]: pgmap v765: 129 pgs: 129 active+clean; 109 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 207 KiB/s rd, 318 KiB/s wr, 38 op/s 2024-09-11T00:19:17.830 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:19:17 smithi178 ceph-mon[35919]: pgmap v765: 129 pgs: 129 active+clean; 109 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 207 KiB/s rd, 318 KiB/s wr, 38 op/s 2024-09-11T00:19:18.697 INFO:tasks.workunit.client.0.smithi012.stdout: 1 487561 3.83 MB/sec execute 555 sec latency 38.108 ms 2024-09-11T00:19:19.297 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:19.295+0000 7f33a7800640 1 -- 172.21.15.12:0/324640936 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f33a0075b20 msgr2=0x7f33a0075fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:19:19.297 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:19.295+0000 7f33a7800640 1 --2- 172.21.15.12:0/324640936 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f33a0075b20 0x7f33a0075fa0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f3388009900 tx=0x7f338802a3f0 comp rx=0 tx=0).stop 2024-09-11T00:19:19.298 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:19.296+0000 7f33a7800640 1 -- 172.21.15.12:0/324640936 shutdown_connections 2024-09-11T00:19:19.298 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:19.296+0000 7f33a7800640 1 --2- 172.21.15.12:0/324640936 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f33a01064a0 0x7f33a0111180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:19:19.298 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:19.296+0000 7f33a7800640 1 --2- 172.21.15.12:0/324640936 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f33a0075b20 0x7f33a0075fa0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:19:19.298 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:19.296+0000 7f33a7800640 1 --2- 172.21.15.12:0/324640936 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f33a0076f90 0x7f33a00755e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:19:19.298 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:19.296+0000 7f33a7800640 1 -- 172.21.15.12:0/324640936 >> 172.21.15.12:0/324640936 conn(0x7f33a00fe250 msgr2=0x7f33a0100670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:19:19.298 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:19.296+0000 7f33a7800640 1 -- 172.21.15.12:0/324640936 shutdown_connections 2024-09-11T00:19:19.298 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:19.296+0000 7f33a7800640 1 -- 172.21.15.12:0/324640936 wait complete. 2024-09-11T00:19:19.300 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:19.297+0000 7f33a7800640 1 Processor -- start 2024-09-11T00:19:19.300 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:19.297+0000 7f33a7800640 1 -- start start 2024-09-11T00:19:19.300 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:19.297+0000 7f33a7800640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f33a0075b20 0x7f33a019f340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:19:19.300 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:19.297+0000 7f33a7800640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f33a0076f90 0x7f33a019f990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:19:19.300 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:19.297+0000 7f33a7800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f33a01a05c0 0x7f33a01a0a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:19:19.301 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:19.297+0000 7f33a7800640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f33a0114c30 con 0x7f33a0076f90 2024-09-11T00:19:19.301 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:19.297+0000 7f33a7800640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f33a0114ab0 con 0x7f33a01a05c0 2024-09-11T00:19:19.301 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:19.297+0000 7f33a7800640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f33a0114db0 con 0x7f33a0075b20 2024-09-11T00:19:19.301 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:19.297+0000 7f33a5800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f33a01a05c0 0x7f33a01a0a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:19:19.301 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:19.297+0000 7f33a4e00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f33a0075b20 0x7f33a019f340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:19:19.301 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:19.297+0000 7f3397e00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f33a0076f90 0x7f33a019f990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:19:19.301 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:19.297+0000 7f33a5800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f33a01a05c0 0x7f33a01a0a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.12:49196/0 (socket says 172.21.15.12:49196) 2024-09-11T00:19:19.301 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:19.297+0000 7f33a5800640 1 -- 172.21.15.12:0/1144107385 learned_addr learned my addr 172.21.15.12:0/1144107385 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:19:19.301 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:19.297+0000 7f33a4e00640 1 --2- 172.21.15.12:0/1144107385 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f33a0075b20 0x7f33a019f340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.12:41246/0 (socket says 172.21.15.12:41246) 2024-09-11T00:19:19.301 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:19.298+0000 7f33a5800640 1 -- 172.21.15.12:0/1144107385 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f33a0075b20 msgr2=0x7f33a019f340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:19:19.301 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:19.298+0000 7f33a5800640 1 --2- 172.21.15.12:0/1144107385 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f33a0075b20 0x7f33a019f340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:19:19.302 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:19.298+0000 7f33a5800640 1 -- 172.21.15.12:0/1144107385 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f33a0076f90 msgr2=0x7f33a019f990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:19:19.302 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:19.298+0000 7f33a5800640 1 --2- 172.21.15.12:0/1144107385 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f33a0076f90 0x7f33a019f990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:19:19.302 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:19.298+0000 7f33a5800640 1 -- 172.21.15.12:0/1144107385 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33a01a1270 con 0x7f33a01a05c0 2024-09-11T00:19:19.302 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:19.298+0000 7f33a4e00640 1 --2- 172.21.15.12:0/1144107385 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f33a0075b20 0x7f33a019f340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:19:19.302 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:19.298+0000 7f33a5800640 1 --2- 172.21.15.12:0/1144107385 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f33a01a05c0 0x7f33a01a0a70 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f339800eb10 tx=0x7f339800c2d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:19:19.302 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:19.298+0000 7f339e000640 1 -- 172.21.15.12:0/1144107385 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3398004b20 con 0x7f33a01a05c0 2024-09-11T00:19:19.302 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:19.298+0000 7f33a7800640 1 -- 172.21.15.12:0/1144107385 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f33a01a7660 con 0x7f33a01a05c0 2024-09-11T00:19:19.302 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:19.298+0000 7f339e000640 1 -- 172.21.15.12:0/1144107385 <== mon.2 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3398004cc0 con 0x7f33a01a05c0 2024-09-11T00:19:19.302 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:19.298+0000 7f339e000640 1 -- 172.21.15.12:0/1144107385 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3398010670 con 0x7f33a01a05c0 2024-09-11T00:19:19.302 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:19.299+0000 7f33a7800640 1 -- 172.21.15.12:0/1144107385 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f33a01a7b50 con 0x7f33a01a05c0 2024-09-11T00:19:19.303 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:19.302+0000 7f33aa7d7740 1 Processor -- start 2024-09-11T00:19:19.303 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:19.302+0000 7f339e000640 1 -- 172.21.15.12:0/1144107385 <== mon.2 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7f3398005020 con 0x7f33a01a05c0 2024-09-11T00:19:19.304 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:19.303+0000 7f33aa7d7740 1 -- start start 2024-09-11T00:19:19.305 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:19.303+0000 7f33aa7d7740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560007989790 0x56000798bb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:19:19.305 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:19.303+0000 7f33aa7d7740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56000798c0c0 0x56000798e4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:19:19.305 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:19.303+0000 7f33aa7d7740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56000798eb00 0x560007990fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:19:19.305 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:19.303+0000 7f33aa7d7740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x5600078fc6b0 con 0x56000798eb00 2024-09-11T00:19:19.305 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:19.303+0000 7f33aa7d7740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x5600078ebae0 con 0x560007989790 2024-09-11T00:19:19.305 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:19.303+0000 7f33aa7d7740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5600078ef8f0 con 0x56000798c0c0 2024-09-11T00:19:19.306 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:19.303+0000 7f3397e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560007989790 0x56000798bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:19:19.306 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:19.303+0000 7f3397e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560007989790 0x56000798bb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.12:49206/0 (socket says 172.21.15.12:49206) 2024-09-11T00:19:19.306 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:19.303+0000 7f3397e00640 1 -- 172.21.15.12:0/1430346275 learned_addr learned my addr 172.21.15.12:0/1430346275 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:19:19.306 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:19.303+0000 7f33a4e00640 1 --2- 172.21.15.12:0/1430346275 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56000798c0c0 0x56000798e4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:19:19.306 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:19.303+0000 7f3397e00640 1 --2- 172.21.15.12:0/1430346275 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56000798eb00 0x560007990fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:19:19.306 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:19.304+0000 7f33a4e00640 1 -- 172.21.15.12:0/1430346275 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560007989790 msgr2=0x56000798bb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:19:19.306 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:19.304+0000 7f33a4e00640 1 --2- 172.21.15.12:0/1430346275 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560007989790 0x56000798bb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:19:19.306 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:19.304+0000 7f33a4e00640 1 -- 172.21.15.12:0/1430346275 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56000798eb00 msgr2=0x560007990fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:19:19.306 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:19.304+0000 7f33a4e00640 1 --2- 172.21.15.12:0/1430346275 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56000798eb00 0x560007990fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:19:19.306 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:19.304+0000 7f33a4e00640 1 -- 172.21.15.12:0/1430346275 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5600078b8780 con 0x56000798c0c0 2024-09-11T00:19:19.306 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:19.304+0000 7f3397e00640 1 --2- 172.21.15.12:0/1430346275 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560007989790 0x56000798bb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:19:19.307 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:19.304+0000 7f33a4e00640 1 --2- 172.21.15.12:0/1430346275 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56000798c0c0 0x56000798e4b0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f3390009fd0 tx=0x7f33900118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=95468f4bc1bbb4a2 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:19:19.311 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:19.304+0000 7f3396000640 1 -- 172.21.15.12:0/1430346275 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3390004020 con 0x56000798c0c0 2024-09-11T00:19:19.311 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:19.305+0000 7f3396000640 1 -- 172.21.15.12:0/1430346275 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f33900054d0 con 0x56000798c0c0 2024-09-11T00:19:19.311 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:19.305+0000 7f3396000640 1 -- 172.21.15.12:0/1430346275 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3390004ba0 con 0x56000798c0c0 2024-09-11T00:19:19.311 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:19.305+0000 7f339e000640 1 --2- 172.21.15.12:0/1144107385 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f33840b52e0 0x7f33840b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:19:19.312 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:19.305+0000 7f339e000640 1 -- 172.21.15.12:0/1144107385 <== mon.2 v2:172.21.15.165:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9208+0+0 (secure 0 0 0) 0x7f33980e6520 con 0x7f33a01a05c0 2024-09-11T00:19:19.312 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:19.305+0000 7f3397e00640 1 --2- 172.21.15.12:0/1144107385 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f33840b52e0 0x7f33840b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:19:19.312 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:19.305+0000 7f33aa7d7740 1 -- 172.21.15.12:0/1430346275 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56000798c0c0 msgr2=0x56000798e4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:19:19.312 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:19.305+0000 7f33aa7d7740 1 --2- 172.21.15.12:0/1430346275 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56000798c0c0 0x56000798e4b0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f3390009fd0 tx=0x7f33900118d0 comp rx=0 tx=0).stop 2024-09-11T00:19:19.312 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:19.305+0000 7f33aa7d7740 1 -- 172.21.15.12:0/1430346275 shutdown_connections 2024-09-11T00:19:19.312 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:19.305+0000 7f33aa7d7740 1 --2- 172.21.15.12:0/1430346275 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56000798eb00 0x560007990fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:19:19.312 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:19.305+0000 7f33aa7d7740 1 --2- 172.21.15.12:0/1430346275 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56000798c0c0 0x56000798e4b0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:19:19.312 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:19.305+0000 7f33aa7d7740 1 --2- 172.21.15.12:0/1430346275 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560007989790 0x56000798bb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:19:19.312 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:19.305+0000 7f33aa7d7740 1 -- 172.21.15.12:0/1430346275 >> 172.21.15.12:0/1430346275 conn(0x5600079848e0 msgr2=0x560007985670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:19:19.312 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:19.306+0000 7f33aa7d7740 1 -- 172.21.15.12:0/1430346275 shutdown_connections 2024-09-11T00:19:19.313 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:19.306+0000 7f33aa7d7740 1 -- 172.21.15.12:0/1430346275 wait complete. 2024-09-11T00:19:19.313 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:19.306+0000 7f3397e00640 1 --2- 172.21.15.12:0/1144107385 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f33840b52e0 0x7f33840b77a0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f33880096c0 tx=0x7f338802a3f0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:19:19.313 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:19.306+0000 7f33aa7d7740 1 -- start start 2024-09-11T00:19:19.313 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:19.306+0000 7f33aa7d7740 1 Processor -- start 2024-09-11T00:19:19.313 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:19.307+0000 7f33aa7d7740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560007989790 0x560007992e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:19:19.313 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:19.307+0000 7f33aa7d7740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56000798c0c0 0x5600079934d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:19:19.313 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:19.307+0000 7f33aa7d7740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56000798eb00 0x5600079940a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:19:19.313 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:19.307+0000 7f33aa7d7740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x560007823ab0 con 0x560007989790 2024-09-11T00:19:19.313 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:19.307+0000 7f33aa7d7740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x560007892570 con 0x56000798eb00 2024-09-11T00:19:19.313 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:19.307+0000 7f33aa7d7740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x560007991ae0 con 0x56000798c0c0 2024-09-11T00:19:19.313 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:19.307+0000 7f33aa7d7740 10 client.0 _ll_register_callbacks cb 0x5600078a1100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:19:19.314 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:19.307+0000 7f3397e00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56000798c0c0 0x5600079934d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:19:19.314 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:19.307+0000 7f33a4e00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560007989790 0x560007992e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:19:19.314 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:19.307+0000 7f3397e00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56000798c0c0 0x5600079934d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.12:41262/0 (socket says 172.21.15.12:41262) 2024-09-11T00:19:19.314 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:19.307+0000 7f3397e00640 1 -- 172.21.15.12:0/844747259 learned_addr learned my addr 172.21.15.12:0/844747259 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:19:19.314 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:19.307+0000 7f33a5800640 1 --2- 172.21.15.12:0/844747259 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56000798eb00 0x5600079940a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:19:19.314 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:19.307+0000 7f3397e00640 1 -- 172.21.15.12:0/844747259 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56000798eb00 msgr2=0x5600079940a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:19:19.314 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:19.308+0000 7f3397e00640 1 --2- 172.21.15.12:0/844747259 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56000798eb00 0x5600079940a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:19:19.314 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:19.308+0000 7f3397e00640 1 -- 172.21.15.12:0/844747259 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560007989790 msgr2=0x560007992e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:19:19.314 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:19.308+0000 7f3397e00640 1 --2- 172.21.15.12:0/844747259 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560007989790 0x560007992e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:19:19.314 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:19.308+0000 7f3397e00640 1 -- 172.21.15.12:0/844747259 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56000764d790 con 0x56000798c0c0 2024-09-11T00:19:19.314 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:19.308+0000 7f33a5800640 1 --2- 172.21.15.12:0/844747259 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56000798eb00 0x5600079940a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:19:19.315 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:19.308+0000 7f33a4e00640 1 --2- 172.21.15.12:0/844747259 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560007989790 0x560007992e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:19:19.315 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:19.308+0000 7f3397e00640 1 --2- 172.21.15.12:0/844747259 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56000798c0c0 0x5600079934d0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f3388009230 tx=0x7f3388009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=90d7663f17982b53 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:19:19.315 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:19.308+0000 7f338ea00640 10 client.0 ms_handle_connect on v2:172.21.15.178:3300/0 2024-09-11T00:19:19.315 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:19.308+0000 7f338ea00640 1 -- 172.21.15.12:0/844747259 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3388035040 con 0x56000798c0c0 2024-09-11T00:19:19.315 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:19.308+0000 7f33aa7d7740 1 -- 172.21.15.12:0/844747259 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f33900054d0 con 0x56000798c0c0 2024-09-11T00:19:19.315 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:19.308+0000 7f338ea00640 1 -- 172.21.15.12:0/844747259 <== mon.1 v2:172.21.15.178:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9208+0+0 (secure 0 0 0) 0x7f3388003800 con 0x56000798c0c0 2024-09-11T00:19:19.315 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:19.308+0000 7f338ea00640 1 -- 172.21.15.12:0/844747259 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f338805d050 con 0x56000798c0c0 2024-09-11T00:19:19.315 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:19.308+0000 7f33aa7d7740 10 client.26905 fetch_fsmap learned FSMap version 44 2024-09-11T00:19:19.315 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:19.308+0000 7f33aa7d7740 1 -- 172.21.15.12:0/844747259 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x5600076c6620 con 0x56000798c0c0 2024-09-11T00:19:19.315 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:19.309+0000 7f338ea00640 1 -- 172.21.15.12:0/844747259 <== mon.1 v2:172.21.15.178:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f33900054d0 con 0x56000798c0c0 2024-09-11T00:19:19.315 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:19.309+0000 7f338ea00640 1 -- 172.21.15.12:0/844747259 <== mon.1 v2:172.21.15.178:3300/0 5 ==== fsmap(e 44) ==== 8382+0+0 (secure 0 0 0) 0x7f338803dad0 con 0x56000798c0c0 2024-09-11T00:19:19.315 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:19.309+0000 7f33aa7d7740 10 client.26905 fetch_fsmap finished waiting for FSMap version 44 2024-09-11T00:19:19.316 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:19.309+0000 7f33aa7d7740 10 client.26905 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T00:19:19.316 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:19.309+0000 7f33aa7d7740 20 client.26905 populate_metadata read hostname 'smithi012' 2024-09-11T00:19:19.316 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:19.309+0000 7f33aa7d7740 1 --2- 172.21.15.12:0/844747259 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x56000799e770 0x5600079a0bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:19:19.316 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:19.309+0000 7f33aa7d7740 4 client.26905 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T00:19:19.316 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:19.309+0000 7f33aa7d7740 1 -- 172.21.15.12:0/844747259 --> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5600078fe600 con 0x56000799e770 2024-09-11T00:19:19.316 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:19.309+0000 7f33a4e00640 1 --2- 172.21.15.12:0/844747259 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x56000799e770 0x5600079a0bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:19:19.316 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:19.310+0000 7f33a4e00640 1 --2- 172.21.15.12:0/844747259 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x56000799e770 0x5600079a0bf0 crc :-1 s=READY pgs=189 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a3b616e4805c40ab server_cookie=ddebfb5ce7dab402 in_seq=0 out_seq=0 2024-09-11T00:19:19.316 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:19.310+0000 7f338ea00640 10 client.26905 ms_handle_connect on v2:172.21.15.165:6838/501312984 2024-09-11T00:19:19.317 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:19.311+0000 7f338ea00640 1 -- 172.21.15.12:0/844747259 <== mds.0 v2:172.21.15.165:6838/501312984 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5600078fe600 con 0x56000799e770 2024-09-11T00:19:19.317 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:19.311+0000 7f338ea00640 10 client.26905 handle_command_reply: tid=0 2024-09-11T00:19:19.318 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:19.311+0000 7f33aa7d7740 1 client.26905 shutdown 2024-09-11T00:19:19.318 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:19.311+0000 7f33aa7d7740 1 -- 172.21.15.12:0/844747259 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56000798c0c0 msgr2=0x5600079934d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:19:19.318 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:19.311+0000 7f33aa7d7740 1 --2- 172.21.15.12:0/844747259 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56000798c0c0 0x5600079934d0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f3388009230 tx=0x7f3388009260 comp rx=0 tx=0).stop 2024-09-11T00:19:19.318 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:19.311+0000 7f33aa7d7740 1 -- 172.21.15.12:0/844747259 shutdown_connections 2024-09-11T00:19:19.318 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:19.311+0000 7f33aa7d7740 1 --2- 172.21.15.12:0/844747259 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x56000799e770 0x5600079a0bf0 crc :-1 s=READY pgs=189 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:19:19.318 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:19.311+0000 7f33aa7d7740 1 --2- 172.21.15.12:0/844747259 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56000798eb00 0x5600079940a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:19:19.318 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:19.311+0000 7f33aa7d7740 1 --2- 172.21.15.12:0/844747259 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56000798c0c0 0x5600079934d0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:19:19.318 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:19.311+0000 7f33aa7d7740 1 --2- 172.21.15.12:0/844747259 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560007989790 0x560007992e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:19:19.318 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:19.311+0000 7f33aa7d7740 1 -- 172.21.15.12:0/844747259 >> 172.21.15.12:0/844747259 conn(0x5600079848e0 msgr2=0x560007983a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:19:19.318 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:19.311+0000 7f338ea00640 0 client.26905 ms_handle_reset on v2:172.21.15.165:6838/501312984 2024-09-11T00:19:19.318 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:19.311+0000 7f33aa7d7740 1 -- 172.21.15.12:0/844747259 shutdown_connections 2024-09-11T00:19:19.319 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:19.311+0000 7f33aa7d7740 1 -- 172.21.15.12:0/844747259 wait complete. 2024-09-11T00:19:19.319 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:19.311+0000 7f33aa7d7740 20 client.26905 trim_cache size 0 max 16384 2024-09-11T00:19:19.329 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:19.327+0000 7f33aa7d7740 1 Processor -- start 2024-09-11T00:19:19.329 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:19.327+0000 7f33aa7d7740 1 -- start start 2024-09-11T00:19:19.329 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:19.328+0000 7f33aa7d7740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560007989790 0x56000797f960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:19:19.329 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:19.328+0000 7f33aa7d7740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56000798c0c0 0x560007988bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:19:19.329 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:19.328+0000 7f33aa7d7740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56000798eb00 0x560007989100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:19:19.330 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:19.328+0000 7f33aa7d7740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f33a01b1d60 con 0x56000798c0c0 2024-09-11T00:19:19.330 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:19.328+0000 7f33aa7d7740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f33a01b3930 con 0x560007989790 2024-09-11T00:19:19.330 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:19.328+0000 7f33aa7d7740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f33a01b2460 con 0x56000798eb00 2024-09-11T00:19:19.330 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:19.328+0000 7f33a4e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560007989790 0x56000797f960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:19:19.330 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:19.328+0000 7f3397e00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56000798c0c0 0x560007988bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:19:19.330 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:19.328+0000 7f33a4e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560007989790 0x56000797f960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.12:49214/0 (socket says 172.21.15.12:49214) 2024-09-11T00:19:19.330 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:19.328+0000 7f33a4e00640 1 -- 172.21.15.12:0/3169656550 learned_addr learned my addr 172.21.15.12:0/3169656550 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:19:19.330 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:19.328+0000 7f33a5800640 1 --2- 172.21.15.12:0/3169656550 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56000798eb00 0x560007989100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:19:19.334 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:19.328+0000 7f33a4e00640 1 -- 172.21.15.12:0/3169656550 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56000798eb00 msgr2=0x560007989100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:19:19.334 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:19.328+0000 7f33a4e00640 1 --2- 172.21.15.12:0/3169656550 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56000798eb00 0x560007989100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:19:19.334 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:19.328+0000 7f33a4e00640 1 -- 172.21.15.12:0/3169656550 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56000798c0c0 msgr2=0x560007988bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:19:19.334 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:19.328+0000 7f33a4e00640 1 --2- 172.21.15.12:0/3169656550 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56000798c0c0 0x560007988bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:19:19.334 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:19.328+0000 7f33a4e00640 1 -- 172.21.15.12:0/3169656550 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560007994410 con 0x560007989790 2024-09-11T00:19:19.334 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:19.328+0000 7f3397e00640 1 --2- 172.21.15.12:0/3169656550 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56000798c0c0 0x560007988bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:19:19.334 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:19.329+0000 7f33a4e00640 1 --2- 172.21.15.12:0/3169656550 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560007989790 0x56000797f960 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f3390011dd0 tx=0x7f33900094b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=110f7c015aeef016 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:19:19.334 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:19.329+0000 7f3395600640 1 -- 172.21.15.12:0/3169656550 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f339001d040 con 0x560007989790 2024-09-11T00:19:19.334 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:19.329+0000 7f3395600640 1 -- 172.21.15.12:0/3169656550 <== mon.2 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3390004020 con 0x560007989790 2024-09-11T00:19:19.334 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:19.329+0000 7f3395600640 1 -- 172.21.15.12:0/3169656550 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33900037c0 con 0x560007989790 2024-09-11T00:19:19.335 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:19.330+0000 7f33aa7d7740 1 -- 172.21.15.12:0/3169656550 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560007989790 msgr2=0x56000797f960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:19:19.335 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:19.330+0000 7f33aa7d7740 1 --2- 172.21.15.12:0/3169656550 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560007989790 0x56000797f960 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f3390011dd0 tx=0x7f33900094b0 comp rx=0 tx=0).stop 2024-09-11T00:19:19.335 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:19.330+0000 7f33aa7d7740 1 -- 172.21.15.12:0/3169656550 shutdown_connections 2024-09-11T00:19:19.335 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:19.330+0000 7f33aa7d7740 1 --2- 172.21.15.12:0/3169656550 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56000798eb00 0x560007989100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:19:19.335 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:19.330+0000 7f33aa7d7740 1 --2- 172.21.15.12:0/3169656550 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56000798c0c0 0x560007988bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:19:19.335 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:19.330+0000 7f33aa7d7740 1 --2- 172.21.15.12:0/3169656550 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560007989790 0x56000797f960 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:19:19.335 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:19.330+0000 7f33aa7d7740 1 -- 172.21.15.12:0/3169656550 >> 172.21.15.12:0/3169656550 conn(0x5600079848e0 msgr2=0x560007987930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:19:19.335 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:19.330+0000 7f33aa7d7740 1 -- 172.21.15.12:0/3169656550 shutdown_connections 2024-09-11T00:19:19.335 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:19.330+0000 7f33aa7d7740 1 -- 172.21.15.12:0/3169656550 wait complete. 2024-09-11T00:19:19.335 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:19.330+0000 7f33aa7d7740 1 -- start start 2024-09-11T00:19:19.335 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:19.331+0000 7f33aa7d7740 1 Processor -- start 2024-09-11T00:19:19.335 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:19.331+0000 7f33aa7d7740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560007989790 0x56000799abc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:19:19.336 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:19.331+0000 7f33aa7d7740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56000798c0c0 0x5600079999b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:19:19.336 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:19.331+0000 7f33aa7d7740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56000798eb00 0x56000799a0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:19:19.336 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:19.331+0000 7f33aa7d7740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f33a01b3260 con 0x560007989790 2024-09-11T00:19:19.336 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:19.331+0000 7f33aa7d7740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f33a01b2b30 con 0x56000798c0c0 2024-09-11T00:19:19.336 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:19.331+0000 7f33aa7d7740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f33a01afd40 con 0x56000798eb00 2024-09-11T00:19:19.336 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:19.331+0000 7f33aa7d7740 10 client.0 _ll_register_callbacks cb 0x7f33880315a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:19:19.336 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:19.331+0000 7f3397e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56000798c0c0 0x5600079999b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:19:19.336 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:19.331+0000 7f33a4e00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560007989790 0x56000799abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:19:19.336 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:19.331+0000 7f3397e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56000798c0c0 0x5600079999b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.12:49220/0 (socket says 172.21.15.12:49220) 2024-09-11T00:19:19.336 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:19.331+0000 7f3397e00640 1 -- 172.21.15.12:0/3935033838 learned_addr learned my addr 172.21.15.12:0/3935033838 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:19:19.336 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:19.331+0000 7f33a5800640 1 --2- 172.21.15.12:0/3935033838 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56000798eb00 0x56000799a0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:19:19.337 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:19.332+0000 7f3397e00640 1 -- 172.21.15.12:0/3935033838 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56000798eb00 msgr2=0x56000799a0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:19:19.337 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:19.332+0000 7f3397e00640 1 --2- 172.21.15.12:0/3935033838 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56000798eb00 0x56000799a0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:19:19.337 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:19.332+0000 7f3397e00640 1 -- 172.21.15.12:0/3935033838 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560007989790 msgr2=0x56000799abc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:19:19.337 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:19.332+0000 7f3397e00640 1 --2- 172.21.15.12:0/3935033838 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560007989790 0x56000799abc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:19:19.337 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:19.332+0000 7f3397e00640 1 -- 172.21.15.12:0/3935033838 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560007998430 con 0x56000798c0c0 2024-09-11T00:19:19.337 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:19.332+0000 7f33a5800640 1 --2- 172.21.15.12:0/3935033838 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56000798eb00 0x56000799a0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T00:19:19.337 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:19.332+0000 7f3397e00640 1 --2- 172.21.15.12:0/3935033838 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56000798c0c0 0x5600079999b0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f3388006010 tx=0x7f3388003a30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a05f3a4e165f763d server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:19:19.337 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:19.332+0000 7f338ea00640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-09-11T00:19:19.337 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:19.332+0000 7f338ea00640 1 -- 172.21.15.12:0/3935033838 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3388066060 con 0x56000798c0c0 2024-09-11T00:19:19.337 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:19.332+0000 7f338ea00640 1 -- 172.21.15.12:0/3935033838 <== mon.2 v2:172.21.15.165:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9208+0+0 (secure 0 0 0) 0x7f338803d650 con 0x56000798c0c0 2024-09-11T00:19:19.338 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:19.332+0000 7f33aa7d7740 1 -- 172.21.15.12:0/3935033838 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3390004020 con 0x56000798c0c0 2024-09-11T00:19:19.338 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:19.333+0000 7f338ea00640 1 -- 172.21.15.12:0/3935033838 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f3388041050 con 0x56000798c0c0 2024-09-11T00:19:19.338 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:19.333+0000 7f33aa7d7740 10 client.26936 fetch_fsmap learned FSMap version 44 2024-09-11T00:19:19.338 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:19.333+0000 7f33aa7d7740 1 -- 172.21.15.12:0/3935033838 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x560007a0ba60 con 0x56000798c0c0 2024-09-11T00:19:19.339 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:19.333+0000 7f338ea00640 1 -- 172.21.15.12:0/3935033838 <== mon.2 v2:172.21.15.165:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3390004020 con 0x56000798c0c0 2024-09-11T00:19:19.339 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:19.333+0000 7f338ea00640 1 -- 172.21.15.12:0/3935033838 <== mon.2 v2:172.21.15.165:3300/0 5 ==== fsmap(e 44) ==== 8382+0+0 (secure 0 0 0) 0x7f3388060030 con 0x56000798c0c0 2024-09-11T00:19:19.339 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:19.333+0000 7f33aa7d7740 10 client.26936 fetch_fsmap finished waiting for FSMap version 44 2024-09-11T00:19:19.339 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:19.333+0000 7f33aa7d7740 10 client.26936 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T00:19:19.339 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:19.333+0000 7f33aa7d7740 20 client.26936 populate_metadata read hostname 'smithi012' 2024-09-11T00:19:19.339 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:19.333+0000 7f33aa7d7740 1 --2- 172.21.15.12:0/3935033838 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x560007a0c4a0 0x560007a0e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:19:19.340 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:19.333+0000 7f33aa7d7740 4 client.26936 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T00:19:19.340 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:19.333+0000 7f33aa7d7740 1 -- 172.21.15.12:0/3935033838 --> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5600078fa0b0 con 0x560007a0c4a0 2024-09-11T00:19:19.340 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:19.333+0000 7f33a4e00640 1 --2- 172.21.15.12:0/3935033838 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x560007a0c4a0 0x560007a0e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:19:19.340 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:19.334+0000 7f33a4e00640 1 --2- 172.21.15.12:0/3935033838 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x560007a0c4a0 0x560007a0e8c0 crc :-1 s=READY pgs=190 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=277ae2924b76dfff server_cookie=4ab5dab0a77dadf in_seq=0 out_seq=0 2024-09-11T00:19:19.340 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:19.334+0000 7f338ea00640 10 client.26936 ms_handle_connect on v2:172.21.15.165:6838/501312984 2024-09-11T00:19:19.374 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:19.372+0000 7f338ea00640 1 -- 172.21.15.12:0/3935033838 <== mds.0 v2:172.21.15.165:6838/501312984 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5600078fa0b0 con 0x560007a0c4a0 2024-09-11T00:19:19.374 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:19.372+0000 7f338ea00640 10 client.26936 handle_command_reply: tid=0 2024-09-11T00:19:19.374 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:19.372+0000 7f33aa7d7740 1 client.26936 shutdown 2024-09-11T00:19:19.374 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:19.372+0000 7f33aa7d7740 1 -- 172.21.15.12:0/3935033838 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56000798c0c0 msgr2=0x5600079999b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:19:19.374 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:19.372+0000 7f33aa7d7740 1 --2- 172.21.15.12:0/3935033838 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56000798c0c0 0x5600079999b0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f3388006010 tx=0x7f3388003a30 comp rx=0 tx=0).stop 2024-09-11T00:19:19.374 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:19.372+0000 7f33aa7d7740 1 -- 172.21.15.12:0/3935033838 shutdown_connections 2024-09-11T00:19:19.374 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:19.372+0000 7f33aa7d7740 1 --2- 172.21.15.12:0/3935033838 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x560007a0c4a0 0x560007a0e8c0 crc :-1 s=READY pgs=190 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:19:19.375 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:19.372+0000 7f33aa7d7740 1 --2- 172.21.15.12:0/3935033838 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56000798eb00 0x56000799a0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:19:19.375 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:19.372+0000 7f33aa7d7740 1 --2- 172.21.15.12:0/3935033838 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56000798c0c0 0x5600079999b0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:19:19.375 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:19.372+0000 7f33aa7d7740 1 --2- 172.21.15.12:0/3935033838 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560007989790 0x56000799abc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:19:19.375 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:19.372+0000 7f338ea00640 0 client.26936 ms_handle_reset on v2:172.21.15.165:6838/501312984 2024-09-11T00:19:19.375 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:19.372+0000 7f33aa7d7740 1 -- 172.21.15.12:0/3935033838 >> 172.21.15.12:0/3935033838 conn(0x5600079848e0 msgr2=0x560007987930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:19:19.375 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:19.372+0000 7f33aa7d7740 1 -- 172.21.15.12:0/3935033838 shutdown_connections 2024-09-11T00:19:19.375 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:19.372+0000 7f33aa7d7740 1 -- 172.21.15.12:0/3935033838 wait complete. 2024-09-11T00:19:19.375 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:19.373+0000 7f33aa7d7740 20 client.26936 trim_cache size 0 max 16384 2024-09-11T00:19:19.375 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:{ 2024-09-11T00:19:19.375 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "message": "", 2024-09-11T00:19:19.376 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "return_code": 0 2024-09-11T00:19:19.376 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:} 2024-09-11T00:19:19.376 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:19.373+0000 7f33a7800640 1 -- 172.21.15.12:0/1144107385 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f33840b52e0 msgr2=0x7f33840b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:19:19.376 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:19.373+0000 7f33a7800640 1 --2- 172.21.15.12:0/1144107385 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f33840b52e0 0x7f33840b77a0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f33880096c0 tx=0x7f338802a3f0 comp rx=0 tx=0).stop 2024-09-11T00:19:19.376 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:19.373+0000 7f33a7800640 1 -- 172.21.15.12:0/1144107385 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f33a01a05c0 msgr2=0x7f33a01a0a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:19:19.377 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:19.373+0000 7f33a7800640 1 --2- 172.21.15.12:0/1144107385 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f33a01a05c0 0x7f33a01a0a70 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f339800eb10 tx=0x7f339800c2d0 comp rx=0 tx=0).stop 2024-09-11T00:19:19.377 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:19.373+0000 7f33a7800640 1 -- 172.21.15.12:0/1144107385 shutdown_connections 2024-09-11T00:19:19.377 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:19.373+0000 7f33a7800640 1 --2- 172.21.15.12:0/1144107385 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f33840b52e0 0x7f33840b77a0 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:19:19.377 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:19.373+0000 7f33a7800640 1 --2- 172.21.15.12:0/1144107385 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f33a01a05c0 0x7f33a01a0a70 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:19:19.377 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:19.373+0000 7f33a7800640 1 --2- 172.21.15.12:0/1144107385 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f33a0076f90 0x7f33a019f990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:19:19.377 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:19.373+0000 7f33a7800640 1 --2- 172.21.15.12:0/1144107385 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f33a0075b20 0x7f33a019f340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:19:19.377 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:19.373+0000 7f33a7800640 1 -- 172.21.15.12:0/1144107385 >> 172.21.15.12:0/1144107385 conn(0x7f33a00fe250 msgr2=0x7f33a0102440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:19:19.377 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:19.374+0000 7f33a7800640 1 -- 172.21.15.12:0/1144107385 shutdown_connections 2024-09-11T00:19:19.377 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:19.374+0000 7f33a7800640 1 -- 172.21.15.12:0/1144107385 wait complete. 2024-09-11T00:19:19.697 INFO:tasks.workunit.client.0.smithi012.stdout: 1 489097 3.83 MB/sec execute 556 sec latency 187.731 ms 2024-09-11T00:19:19.782 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:19:19 smithi165 ceph-mon[35356]: pgmap v766: 129 pgs: 129 active+clean; 109 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 214 KiB/s rd, 59 KiB/s wr, 22 op/s 2024-09-11T00:19:19.813 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:19:19 smithi012 ceph-mon[30333]: pgmap v766: 129 pgs: 129 active+clean; 109 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 214 KiB/s rd, 59 KiB/s wr, 22 op/s 2024-09-11T00:19:19.830 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:19:19 smithi178 ceph-mon[35919]: pgmap v766: 129 pgs: 129 active+clean; 109 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 214 KiB/s rd, 59 KiB/s wr, 22 op/s 2024-09-11T00:19:20.697 INFO:tasks.workunit.client.0.smithi012.stdout: 1 490191 3.84 MB/sec execute 557 sec latency 250.557 ms 2024-09-11T00:19:21.697 INFO:tasks.workunit.client.0.smithi012.stdout: 1 490663 3.84 MB/sec execute 558 sec latency 388.876 ms 2024-09-11T00:19:21.782 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:19:21 smithi165 ceph-mon[35356]: pgmap v767: 129 pgs: 129 active+clean; 110 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 21 MiB/s rd, 564 KiB/s wr, 117 op/s 2024-09-11T00:19:21.813 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:19:21 smithi012 ceph-mon[30333]: pgmap v767: 129 pgs: 129 active+clean; 110 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 21 MiB/s rd, 564 KiB/s wr, 117 op/s 2024-09-11T00:19:21.831 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:19:21 smithi178 ceph-mon[35919]: pgmap v767: 129 pgs: 129 active+clean; 110 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 21 MiB/s rd, 564 KiB/s wr, 117 op/s 2024-09-11T00:19:22.697 INFO:tasks.workunit.client.0.smithi012.stdout: 1 491764 3.84 MB/sec execute 559 sec latency 106.432 ms 2024-09-11T00:19:23.697 INFO:tasks.workunit.client.0.smithi012.stdout: 1 493636 3.85 MB/sec execute 560 sec latency 86.571 ms 2024-09-11T00:19:23.782 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:19:23 smithi165 ceph-mon[35356]: pgmap v768: 129 pgs: 129 active+clean; 111 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 21 MiB/s rd, 655 KiB/s wr, 137 op/s 2024-09-11T00:19:23.813 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:19:23 smithi012 ceph-mon[30333]: pgmap v768: 129 pgs: 129 active+clean; 111 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 21 MiB/s rd, 655 KiB/s wr, 137 op/s 2024-09-11T00:19:23.831 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:19:23 smithi178 ceph-mon[35919]: pgmap v768: 129 pgs: 129 active+clean; 111 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 21 MiB/s rd, 655 KiB/s wr, 137 op/s 2024-09-11T00:19:24.698 INFO:tasks.workunit.client.0.smithi012.stdout: 1 494239 3.85 MB/sec execute 561 sec latency 287.175 ms 2024-09-11T00:19:25.697 INFO:tasks.workunit.client.0.smithi012.stdout: 1 495200 3.85 MB/sec execute 562 sec latency 187.290 ms 2024-09-11T00:19:25.782 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:19:25 smithi165 ceph-mon[35356]: pgmap v769: 129 pgs: 129 active+clean; 111 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 21 MiB/s rd, 660 KiB/s wr, 135 op/s 2024-09-11T00:19:25.782 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:19:25 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T00:19:25.813 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:19:25 smithi012 ceph-mon[30333]: pgmap v769: 129 pgs: 129 active+clean; 111 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 21 MiB/s rd, 660 KiB/s wr, 135 op/s 2024-09-11T00:19:25.813 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:19:25 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T00:19:25.830 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:19:25 smithi178 ceph-mon[35919]: pgmap v769: 129 pgs: 129 active+clean; 111 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 21 MiB/s rd, 660 KiB/s wr, 135 op/s 2024-09-11T00:19:25.831 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:19:25 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T00:19:26.698 INFO:tasks.workunit.client.0.smithi012.stdout: 1 496823 3.86 MB/sec execute 563 sec latency 196.230 ms 2024-09-11T00:19:26.782 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:19:26 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:19:26.783 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:19:26 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T00:19:26.783 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:19:26 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:19:26.813 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:19:26 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:19:26.813 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:19:26 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T00:19:26.813 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:19:26 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:19:26.831 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:19:26 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:19:26.831 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:19:26 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T00:19:26.831 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:19:26 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:19:27.698 INFO:tasks.workunit.client.0.smithi012.stdout: 1 497396 3.86 MB/sec execute 564 sec latency 572.388 ms 2024-09-11T00:19:28.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:19:28 smithi012 ceph-mon[30333]: pgmap v770: 129 pgs: 129 active+clean; 112 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 49 MiB/s rd, 1.2 MiB/s wr, 282 op/s 2024-09-11T00:19:28.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:19:28 smithi178 ceph-mon[35919]: pgmap v770: 129 pgs: 129 active+clean; 112 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 49 MiB/s rd, 1.2 MiB/s wr, 282 op/s 2024-09-11T00:19:28.532 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:19:28 smithi165 ceph-mon[35356]: pgmap v770: 129 pgs: 129 active+clean; 112 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 49 MiB/s rd, 1.2 MiB/s wr, 282 op/s 2024-09-11T00:19:28.698 INFO:tasks.workunit.client.0.smithi012.stdout: 1 498371 3.86 MB/sec execute 565 sec latency 10.753 ms 2024-09-11T00:19:29.698 INFO:tasks.workunit.client.0.smithi012.stdout: 1 498930 3.86 MB/sec execute 566 sec latency 607.994 ms 2024-09-11T00:19:30.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:19:30 smithi012 ceph-mon[30333]: pgmap v771: 129 pgs: 129 active+clean; 112 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 49 MiB/s rd, 1.1 MiB/s wr, 271 op/s 2024-09-11T00:19:30.330 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:19:30 smithi178 ceph-mon[35919]: pgmap v771: 129 pgs: 129 active+clean; 112 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 49 MiB/s rd, 1.1 MiB/s wr, 271 op/s 2024-09-11T00:19:30.533 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:19:30 smithi165 ceph-mon[35356]: pgmap v771: 129 pgs: 129 active+clean; 112 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 49 MiB/s rd, 1.1 MiB/s wr, 271 op/s 2024-09-11T00:19:30.698 INFO:tasks.workunit.client.0.smithi012.stdout: 1 500057 3.86 MB/sec execute 567 sec latency 469.676 ms 2024-09-11T00:19:31.698 INFO:tasks.workunit.client.0.smithi012.stdout: 1 501254 3.87 MB/sec execute 568 sec latency 513.538 ms 2024-09-11T00:19:32.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:19:32 smithi012 ceph-mon[30333]: pgmap v772: 129 pgs: 129 active+clean; 114 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 87 MiB/s rd, 1.5 MiB/s wr, 364 op/s 2024-09-11T00:19:32.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:19:32 smithi178 ceph-mon[35919]: pgmap v772: 129 pgs: 129 active+clean; 114 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 87 MiB/s rd, 1.5 MiB/s wr, 364 op/s 2024-09-11T00:19:32.499 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:32.498+0000 7f9548e00640 1 -- 172.21.15.12:0/2451880314 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9544104ba0 msgr2=0x7f9544104fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:19:32.499 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:32.498+0000 7f9548e00640 1 --2- 172.21.15.12:0/2451880314 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9544104ba0 0x7f9544104fa0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f9534009930 tx=0x7f953402a440 comp rx=0 tx=0).stop 2024-09-11T00:19:32.499 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:32.498+0000 7f9548e00640 1 -- 172.21.15.12:0/2451880314 shutdown_connections 2024-09-11T00:19:32.500 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:32.498+0000 7f9548e00640 1 --2- 172.21.15.12:0/2451880314 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f95441066b0 0x7f954410d030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:19:32.500 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:32.498+0000 7f9548e00640 1 --2- 172.21.15.12:0/2451880314 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9544105cf0 0x7f9544106170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:19:32.500 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:32.498+0000 7f9548e00640 1 --2- 172.21.15.12:0/2451880314 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9544104ba0 0x7f9544104fa0 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:19:32.500 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:32.498+0000 7f9548e00640 1 -- 172.21.15.12:0/2451880314 >> 172.21.15.12:0/2451880314 conn(0x7f9544100460 msgr2=0x7f9544102880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:19:32.500 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:32.498+0000 7f9548e00640 1 -- 172.21.15.12:0/2451880314 shutdown_connections 2024-09-11T00:19:32.500 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:32.498+0000 7f9548e00640 1 -- 172.21.15.12:0/2451880314 wait complete. 2024-09-11T00:19:32.502 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:32.499+0000 7f9548e00640 1 Processor -- start 2024-09-11T00:19:32.502 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:32.499+0000 7f9548e00640 1 -- start start 2024-09-11T00:19:32.502 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:32.499+0000 7f9548e00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9544104ba0 0x7f954419af90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:19:32.502 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:32.499+0000 7f9548e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9544105cf0 0x7f954419b5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:19:32.502 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:32.499+0000 7f9548e00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f95441066b0 0x7f954419c210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:19:32.502 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:32.499+0000 7f9548e00640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f9544110810 con 0x7f95441066b0 2024-09-11T00:19:32.503 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:32.499+0000 7f9548e00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f9544110690 con 0x7f9544105cf0 2024-09-11T00:19:32.503 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:32.499+0000 7f9548e00640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f9544110990 con 0x7f9544104ba0 2024-09-11T00:19:32.503 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:32.499+0000 7f9539400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9544105cf0 0x7f954419b5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:19:32.503 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:32.499+0000 7f9539400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9544105cf0 0x7f954419b5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.12:38210/0 (socket says 172.21.15.12:38210) 2024-09-11T00:19:32.503 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:32.499+0000 7f9539400640 1 -- 172.21.15.12:0/2431850419 learned_addr learned my addr 172.21.15.12:0/2431850419 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:19:32.503 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:32.499+0000 7f9541e00640 1 --2- 172.21.15.12:0/2431850419 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9544104ba0 0x7f954419af90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:19:32.503 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:32.499+0000 7f9542800640 1 --2- 172.21.15.12:0/2431850419 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f95441066b0 0x7f954419c210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:19:32.503 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:32.500+0000 7f9539400640 1 -- 172.21.15.12:0/2431850419 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9544104ba0 msgr2=0x7f954419af90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:19:32.503 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:32.500+0000 7f9539400640 1 --2- 172.21.15.12:0/2431850419 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9544104ba0 0x7f954419af90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:19:32.503 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:32.500+0000 7f9539400640 1 -- 172.21.15.12:0/2431850419 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f95441066b0 msgr2=0x7f954419c210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:19:32.503 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:32.500+0000 7f9539400640 1 --2- 172.21.15.12:0/2431850419 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f95441066b0 0x7f954419c210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:19:32.503 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:32.500+0000 7f9539400640 1 -- 172.21.15.12:0/2431850419 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f954419cad0 con 0x7f9544105cf0 2024-09-11T00:19:32.503 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:32.500+0000 7f9539400640 1 --2- 172.21.15.12:0/2431850419 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9544105cf0 0x7f954419b5e0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f952c00e9f0 tx=0x7f952c00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:19:32.504 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:32.500+0000 7f953b400640 1 -- 172.21.15.12:0/2431850419 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f952c00cd80 con 0x7f9544105cf0 2024-09-11T00:19:32.504 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:32.500+0000 7f953b400640 1 -- 172.21.15.12:0/2431850419 <== mon.2 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f952c0108e0 con 0x7f9544105cf0 2024-09-11T00:19:32.504 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:32.500+0000 7f9548e00640 1 -- 172.21.15.12:0/2431850419 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f954419cdc0 con 0x7f9544105cf0 2024-09-11T00:19:32.504 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:32.500+0000 7f953b400640 1 -- 172.21.15.12:0/2431850419 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f952c018980 con 0x7f9544105cf0 2024-09-11T00:19:32.504 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:32.500+0000 7f9548e00640 1 -- 172.21.15.12:0/2431850419 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95441a3490 con 0x7f9544105cf0 2024-09-11T00:19:32.505 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:32.504+0000 7f954bcc1740 1 Processor -- start 2024-09-11T00:19:32.506 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:32.505+0000 7f954bcc1740 1 -- start start 2024-09-11T00:19:32.507 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:32.505+0000 7f954bcc1740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557916e5e790 0x557916e60b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:19:32.507 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:32.505+0000 7f954bcc1740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557916e610c0 0x557916e634b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:19:32.507 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:32.505+0000 7f954bcc1740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557916e63b00 0x557916e65fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:19:32.507 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:32.505+0000 7f954bcc1740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x557916dd16b0 con 0x557916e63b00 2024-09-11T00:19:32.507 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:32.505+0000 7f954bcc1740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x557916dc0ae0 con 0x557916e5e790 2024-09-11T00:19:32.507 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:32.505+0000 7f954bcc1740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x557916dc48f0 con 0x557916e610c0 2024-09-11T00:19:32.507 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:32.505+0000 7f9542800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557916e5e790 0x557916e60b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:19:32.507 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:32.505+0000 7f9541e00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557916e610c0 0x557916e634b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:19:32.507 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:32.505+0000 7f9542800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557916e5e790 0x557916e60b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.12:38220/0 (socket says 172.21.15.12:38220) 2024-09-11T00:19:32.507 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:32.505+0000 7f9542800640 1 -- 172.21.15.12:0/3829164161 learned_addr learned my addr 172.21.15.12:0/3829164161 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:19:32.508 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:32.505+0000 7f953b400640 1 -- 172.21.15.12:0/2431850419 <== mon.2 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7f952c010430 con 0x7f9544105cf0 2024-09-11T00:19:32.508 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:32.506+0000 7f9542800640 1 -- 172.21.15.12:0/3829164161 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557916e610c0 msgr2=0x557916e634b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:19:32.508 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:32.506+0000 7f9542800640 1 --2- 172.21.15.12:0/3829164161 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557916e610c0 0x557916e634b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:19:32.508 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:32.506+0000 7f9542800640 1 -- 172.21.15.12:0/3829164161 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557916e63b00 msgr2=0x557916e65fc0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-11T00:19:32.508 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:32.506+0000 7f9542800640 1 --2- 172.21.15.12:0/3829164161 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557916e63b00 0x557916e65fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:19:32.508 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:32.506+0000 7f9542800640 1 -- 172.21.15.12:0/3829164161 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557916d8d780 con 0x557916e5e790 2024-09-11T00:19:32.509 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:32.506+0000 7f9542800640 1 --2- 172.21.15.12:0/3829164161 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557916e5e790 0x557916e60b80 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f953c009f90 tx=0x7f953c0118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e7a7ca09d2e70300 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:19:32.510 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:32.506+0000 7f9533400640 1 -- 172.21.15.12:0/3829164161 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f953c004d20 con 0x557916e5e790 2024-09-11T00:19:32.510 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:32.506+0000 7f9533400640 1 -- 172.21.15.12:0/3829164161 <== mon.2 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f953c004050 con 0x557916e5e790 2024-09-11T00:19:32.510 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:32.506+0000 7f9533400640 1 -- 172.21.15.12:0/3829164161 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f953c005280 con 0x557916e5e790 2024-09-11T00:19:32.510 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:32.507+0000 7f954bcc1740 1 -- 172.21.15.12:0/3829164161 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557916e5e790 msgr2=0x557916e60b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:19:32.510 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:32.507+0000 7f954bcc1740 1 --2- 172.21.15.12:0/3829164161 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557916e5e790 0x557916e60b80 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f953c009f90 tx=0x7f953c0118d0 comp rx=0 tx=0).stop 2024-09-11T00:19:32.510 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:32.507+0000 7f954bcc1740 1 -- 172.21.15.12:0/3829164161 shutdown_connections 2024-09-11T00:19:32.510 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:32.507+0000 7f954bcc1740 1 --2- 172.21.15.12:0/3829164161 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557916e63b00 0x557916e65fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:19:32.510 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:32.507+0000 7f954bcc1740 1 --2- 172.21.15.12:0/3829164161 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557916e610c0 0x557916e634b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:19:32.510 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:32.507+0000 7f954bcc1740 1 --2- 172.21.15.12:0/3829164161 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557916e5e790 0x557916e60b80 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:19:32.510 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:32.507+0000 7f954bcc1740 1 -- 172.21.15.12:0/3829164161 >> 172.21.15.12:0/3829164161 conn(0x557916e598e0 msgr2=0x557916e5a670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:19:32.510 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:32.507+0000 7f953b400640 1 --2- 172.21.15.12:0/2431850419 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f95280b5270 0x7f95280b7730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:19:32.511 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:32.507+0000 7f954bcc1740 1 -- 172.21.15.12:0/3829164161 shutdown_connections 2024-09-11T00:19:32.511 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:32.508+0000 7f953b400640 1 -- 172.21.15.12:0/2431850419 <== mon.2 v2:172.21.15.165:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9208+0+0 (secure 0 0 0) 0x7f952c0e6260 con 0x7f9544105cf0 2024-09-11T00:19:32.511 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:32.508+0000 7f954bcc1740 1 -- 172.21.15.12:0/3829164161 wait complete. 2024-09-11T00:19:32.511 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:32.508+0000 7f9541e00640 1 --2- 172.21.15.12:0/2431850419 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f95280b5270 0x7f95280b7730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:19:32.511 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:32.508+0000 7f954bcc1740 1 -- start start 2024-09-11T00:19:32.511 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:32.508+0000 7f954bcc1740 1 Processor -- start 2024-09-11T00:19:32.511 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:32.509+0000 7f9541e00640 1 --2- 172.21.15.12:0/2431850419 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f95280b5270 0x7f95280b7730 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f9534006010 tx=0x7f95340314a0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:19:32.517 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:32.509+0000 7f954bcc1740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557916e5e790 0x557916e67f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:19:32.517 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:32.509+0000 7f954bcc1740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557916e610c0 0x557916e685a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:19:32.517 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:32.509+0000 7f954bcc1740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557916e63b00 0x557916e69170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:19:32.517 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:32.509+0000 7f954bcc1740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x557916cf8ab0 con 0x557916e610c0 2024-09-11T00:19:32.517 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:32.509+0000 7f954bcc1740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x557916d67570 con 0x557916e63b00 2024-09-11T00:19:32.517 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:32.509+0000 7f954bcc1740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x557916e66ae0 con 0x557916e5e790 2024-09-11T00:19:32.518 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:32.509+0000 7f954bcc1740 10 client.0 _ll_register_callbacks cb 0x557916d76100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:19:32.518 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:32.509+0000 7f9541e00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557916e5e790 0x557916e67f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:19:32.518 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:32.509+0000 7f9539400640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557916e610c0 0x557916e685a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:19:32.518 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:32.509+0000 7f9541e00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557916e5e790 0x557916e67f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.12:38242/0 (socket says 172.21.15.12:38242) 2024-09-11T00:19:32.518 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:32.509+0000 7f9541e00640 1 -- 172.21.15.12:0/67870309 learned_addr learned my addr 172.21.15.12:0/67870309 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:19:32.518 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:32.509+0000 7f9542800640 1 --2- 172.21.15.12:0/67870309 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557916e63b00 0x557916e69170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:19:32.518 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:32.510+0000 7f9541e00640 1 -- 172.21.15.12:0/67870309 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557916e63b00 msgr2=0x557916e69170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:19:32.518 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:32.510+0000 7f9541e00640 1 --2- 172.21.15.12:0/67870309 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557916e63b00 0x557916e69170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:19:32.518 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:32.510+0000 7f9541e00640 1 -- 172.21.15.12:0/67870309 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557916e610c0 msgr2=0x557916e685a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:19:32.518 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:32.510+0000 7f9541e00640 1 --2- 172.21.15.12:0/67870309 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557916e610c0 0x557916e685a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:19:32.518 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:32.510+0000 7f9541e00640 1 -- 172.21.15.12:0/67870309 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557916b22790 con 0x557916e5e790 2024-09-11T00:19:32.518 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:32.510+0000 7f9542800640 1 --2- 172.21.15.12:0/67870309 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557916e63b00 0x557916e69170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:19:32.518 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:32.510+0000 7f9541e00640 1 --2- 172.21.15.12:0/67870309 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557916e5e790 0x557916e67f00 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f9534009930 tx=0x7f9534009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1736fc11d8e26cab server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:19:32.519 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:32.510+0000 7f9527e00640 10 client.0 ms_handle_connect on v2:172.21.15.178:3300/0 2024-09-11T00:19:32.519 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:32.510+0000 7f9527e00640 1 -- 172.21.15.12:0/67870309 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9534035040 con 0x557916e5e790 2024-09-11T00:19:32.519 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:32.510+0000 7f954bcc1740 1 -- 172.21.15.12:0/67870309 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f953c004050 con 0x557916e5e790 2024-09-11T00:19:32.519 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:32.510+0000 7f9527e00640 1 -- 172.21.15.12:0/67870309 <== mon.1 v2:172.21.15.178:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9208+0+0 (secure 0 0 0) 0x7f95340037f0 con 0x557916e5e790 2024-09-11T00:19:32.519 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:32.510+0000 7f9527e00640 1 -- 172.21.15.12:0/67870309 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f95340318e0 con 0x557916e5e790 2024-09-11T00:19:32.519 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:32.510+0000 7f954bcc1740 10 client.26923 fetch_fsmap learned FSMap version 44 2024-09-11T00:19:32.519 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:32.510+0000 7f954bcc1740 1 -- 172.21.15.12:0/67870309 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x557916b9b620 con 0x557916e5e790 2024-09-11T00:19:32.519 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:32.511+0000 7f9527e00640 1 -- 172.21.15.12:0/67870309 <== mon.1 v2:172.21.15.178:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f953c004050 con 0x557916e5e790 2024-09-11T00:19:32.519 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:32.511+0000 7f9527e00640 1 -- 172.21.15.12:0/67870309 <== mon.1 v2:172.21.15.178:3300/0 5 ==== fsmap(e 44) ==== 8382+0+0 (secure 0 0 0) 0x7f9534044070 con 0x557916e5e790 2024-09-11T00:19:32.519 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:32.511+0000 7f954bcc1740 10 client.26923 fetch_fsmap finished waiting for FSMap version 44 2024-09-11T00:19:32.519 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:32.511+0000 7f954bcc1740 10 client.26923 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T00:19:32.519 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:32.511+0000 7f954bcc1740 20 client.26923 populate_metadata read hostname 'smithi012' 2024-09-11T00:19:32.519 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:32.511+0000 7f954bcc1740 1 --2- 172.21.15.12:0/67870309 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x557916e73810 0x557916e75c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:19:32.520 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:32.511+0000 7f954bcc1740 4 client.26923 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T00:19:32.520 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:32.511+0000 7f954bcc1740 1 -- 172.21.15.12:0/67870309 --> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557916dd3600 con 0x557916e73810 2024-09-11T00:19:32.520 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:32.511+0000 7f9539400640 1 --2- 172.21.15.12:0/67870309 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x557916e73810 0x557916e75c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:19:32.520 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:32.512+0000 7f9539400640 1 --2- 172.21.15.12:0/67870309 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x557916e73810 0x557916e75c70 crc :-1 s=READY pgs=191 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=29a5a2cd4c30a446 server_cookie=9d4b83a09c31dc6e in_seq=0 out_seq=0 2024-09-11T00:19:32.520 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:32.512+0000 7f9527e00640 10 client.26923 ms_handle_connect on v2:172.21.15.165:6838/501312984 2024-09-11T00:19:32.520 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:32.512+0000 7f9527e00640 1 -- 172.21.15.12:0/67870309 <== mds.0 v2:172.21.15.165:6838/501312984 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x557916dd3600 con 0x557916e73810 2024-09-11T00:19:32.520 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:32.512+0000 7f9527e00640 10 client.26923 handle_command_reply: tid=0 2024-09-11T00:19:32.520 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:32.513+0000 7f954bcc1740 1 client.26923 shutdown 2024-09-11T00:19:32.520 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:32.513+0000 7f954bcc1740 1 -- 172.21.15.12:0/67870309 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557916e5e790 msgr2=0x557916e67f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:19:32.520 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:32.513+0000 7f954bcc1740 1 --2- 172.21.15.12:0/67870309 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557916e5e790 0x557916e67f00 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f9534009930 tx=0x7f9534009260 comp rx=0 tx=0).stop 2024-09-11T00:19:32.520 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:32.513+0000 7f954bcc1740 1 -- 172.21.15.12:0/67870309 shutdown_connections 2024-09-11T00:19:32.520 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:32.513+0000 7f954bcc1740 1 --2- 172.21.15.12:0/67870309 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x557916e73810 0x557916e75c70 crc :-1 s=READY pgs=191 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:19:32.521 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:32.513+0000 7f9527e00640 0 client.26923 ms_handle_reset on v2:172.21.15.165:6838/501312984 2024-09-11T00:19:32.521 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:32.513+0000 7f954bcc1740 1 --2- 172.21.15.12:0/67870309 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557916e63b00 0x557916e69170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:19:32.521 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:32.513+0000 7f954bcc1740 1 --2- 172.21.15.12:0/67870309 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557916e610c0 0x557916e685a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:19:32.521 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:32.513+0000 7f954bcc1740 1 --2- 172.21.15.12:0/67870309 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557916e5e790 0x557916e67f00 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:19:32.521 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:32.513+0000 7f954bcc1740 1 -- 172.21.15.12:0/67870309 >> 172.21.15.12:0/67870309 conn(0x557916e598e0 msgr2=0x557916e58a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:19:32.521 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:32.513+0000 7f954bcc1740 1 -- 172.21.15.12:0/67870309 shutdown_connections 2024-09-11T00:19:32.521 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:32.513+0000 7f954bcc1740 1 -- 172.21.15.12:0/67870309 wait complete. 2024-09-11T00:19:32.521 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:32.513+0000 7f954bcc1740 20 client.26923 trim_cache size 0 max 16384 2024-09-11T00:19:32.530 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:32.529+0000 7f954bcc1740 1 Processor -- start 2024-09-11T00:19:32.530 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:32.529+0000 7f954bcc1740 1 -- start start 2024-09-11T00:19:32.530 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:32.529+0000 7f954bcc1740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557916e5e790 0x557916e54960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:19:32.530 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:32.529+0000 7f954bcc1740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557916e610c0 0x557916e5a140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:19:32.530 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:32.529+0000 7f954bcc1740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557916e63b00 0x557916e5a680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:19:32.531 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:32.529+0000 7f954bcc1740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f95441ad9d0 con 0x557916e610c0 2024-09-11T00:19:32.531 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:32.529+0000 7f954bcc1740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f95441af5a0 con 0x557916e63b00 2024-09-11T00:19:32.531 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:32.529+0000 7f954bcc1740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f95441ae0d0 con 0x557916e5e790 2024-09-11T00:19:32.532 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:32.529+0000 7f9541e00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557916e5e790 0x557916e54960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:19:32.532 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:32.529+0000 7f9541e00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557916e5e790 0x557916e54960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.12:38244/0 (socket says 172.21.15.12:38244) 2024-09-11T00:19:32.533 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:32.529+0000 7f9541e00640 1 -- 172.21.15.12:0/2026829744 learned_addr learned my addr 172.21.15.12:0/2026829744 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:19:32.533 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:32.529+0000 7f9539400640 1 --2- 172.21.15.12:0/2026829744 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557916e610c0 0x557916e5a140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:19:32.533 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:32.530+0000 7f9542800640 1 --2- 172.21.15.12:0/2026829744 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557916e63b00 0x557916e5a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:19:32.533 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:32.530+0000 7f9541e00640 1 -- 172.21.15.12:0/2026829744 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557916e63b00 msgr2=0x557916e5a680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:19:32.533 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:32.530+0000 7f9541e00640 1 --2- 172.21.15.12:0/2026829744 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557916e63b00 0x557916e5a680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:19:32.533 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:32.530+0000 7f9541e00640 1 -- 172.21.15.12:0/2026829744 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557916e610c0 msgr2=0x557916e5a140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:19:32.533 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:32.530+0000 7f9541e00640 1 --2- 172.21.15.12:0/2026829744 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557916e610c0 0x557916e5a140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:19:32.533 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:32.530+0000 7f9541e00640 1 -- 172.21.15.12:0/2026829744 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557916e60ef0 con 0x557916e5e790 2024-09-11T00:19:32.533 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:32.530+0000 7f9541e00640 1 --2- 172.21.15.12:0/2026829744 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557916e5e790 0x557916e54960 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f9534031b30 tx=0x7f9534032e30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6920f4d6c0d23d95 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:19:32.533 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:32.530+0000 7f9532a00640 1 -- 172.21.15.12:0/2026829744 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9534035040 con 0x557916e5e790 2024-09-11T00:19:32.533 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:32.530+0000 7f9532a00640 1 -- 172.21.15.12:0/2026829744 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f953403d900 con 0x557916e5e790 2024-09-11T00:19:32.533 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:32.530+0000 7f9532a00640 1 -- 172.21.15.12:0/2026829744 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9534003af0 con 0x557916e5e790 2024-09-11T00:19:32.533 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:32.531+0000 7f954bcc1740 1 -- 172.21.15.12:0/2026829744 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557916e5e790 msgr2=0x557916e54960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:19:32.534 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:32.531+0000 7f954bcc1740 1 --2- 172.21.15.12:0/2026829744 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557916e5e790 0x557916e54960 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f9534031b30 tx=0x7f9534032e30 comp rx=0 tx=0).stop 2024-09-11T00:19:32.534 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:32.531+0000 7f954bcc1740 1 -- 172.21.15.12:0/2026829744 shutdown_connections 2024-09-11T00:19:32.534 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:32.531+0000 7f954bcc1740 1 --2- 172.21.15.12:0/2026829744 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557916e63b00 0x557916e5a680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:19:32.534 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:32.531+0000 7f954bcc1740 1 --2- 172.21.15.12:0/2026829744 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557916e610c0 0x557916e5a140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:19:32.534 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:32.531+0000 7f954bcc1740 1 --2- 172.21.15.12:0/2026829744 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557916e5e790 0x557916e54960 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:19:32.534 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:32.531+0000 7f954bcc1740 1 -- 172.21.15.12:0/2026829744 >> 172.21.15.12:0/2026829744 conn(0x557916e598e0 msgr2=0x557916e63f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:19:32.534 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:32.531+0000 7f954bcc1740 1 -- 172.21.15.12:0/2026829744 shutdown_connections 2024-09-11T00:19:32.534 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:32.531+0000 7f954bcc1740 1 -- 172.21.15.12:0/2026829744 wait complete. 2024-09-11T00:19:32.535 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:19:32 smithi165 ceph-mon[35356]: pgmap v772: 129 pgs: 129 active+clean; 114 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 87 MiB/s rd, 1.5 MiB/s wr, 364 op/s 2024-09-11T00:19:32.537 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:32.532+0000 7f954bcc1740 1 -- start start 2024-09-11T00:19:32.537 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:32.532+0000 7f954bcc1740 1 Processor -- start 2024-09-11T00:19:32.537 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:32.532+0000 7f954bcc1740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557916e5e790 0x557916e6fbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:19:32.537 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:32.532+0000 7f954bcc1740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557916e610c0 0x557916e6e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:19:32.537 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:32.532+0000 7f954bcc1740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557916e63b00 0x557916e6efe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:19:32.537 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:32.532+0000 7f954bcc1740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f95441aeed0 con 0x557916e63b00 2024-09-11T00:19:32.537 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:32.532+0000 7f954bcc1740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f95441ae7a0 con 0x557916e5e790 2024-09-11T00:19:32.538 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:32.532+0000 7f954bcc1740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f95441ab9b0 con 0x557916e610c0 2024-09-11T00:19:32.538 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:32.532+0000 7f954bcc1740 10 client.0 _ll_register_callbacks cb 0x7f9534009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:19:32.538 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:32.532+0000 7f9541e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557916e5e790 0x557916e6fbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:19:32.538 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:32.533+0000 7f9539400640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557916e610c0 0x557916e6e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:19:32.538 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:32.533+0000 7f9541e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557916e5e790 0x557916e6fbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.12:38236/0 (socket says 172.21.15.12:38236) 2024-09-11T00:19:32.538 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:32.533+0000 7f9541e00640 1 -- 172.21.15.12:0/892315712 learned_addr learned my addr 172.21.15.12:0/892315712 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:19:32.538 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:32.533+0000 7f9542800640 1 --2- 172.21.15.12:0/892315712 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557916e63b00 0x557916e6efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:19:32.538 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:32.533+0000 7f9541e00640 1 -- 172.21.15.12:0/892315712 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557916e610c0 msgr2=0x557916e6e8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:19:32.538 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:32.533+0000 7f9541e00640 1 --2- 172.21.15.12:0/892315712 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557916e610c0 0x557916e6e8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:19:32.538 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:32.533+0000 7f9541e00640 1 -- 172.21.15.12:0/892315712 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557916e63b00 msgr2=0x557916e6efe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:19:32.538 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:32.533+0000 7f9541e00640 1 --2- 172.21.15.12:0/892315712 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557916e63b00 0x557916e6efe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:19:32.538 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:32.533+0000 7f9541e00640 1 -- 172.21.15.12:0/892315712 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557916e6d3f0 con 0x557916e5e790 2024-09-11T00:19:32.538 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:32.533+0000 7f9542800640 1 --2- 172.21.15.12:0/892315712 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557916e63b00 0x557916e6efe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T00:19:32.539 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:32.533+0000 7f9541e00640 1 --2- 172.21.15.12:0/892315712 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557916e5e790 0x557916e6fbd0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f95340096c0 tx=0x7f9534004520 comp rx=0 tx=0).ready entity=mon.2 client_cookie=aac9d1f624ca4f7c server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:19:32.539 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:32.533+0000 7f9527e00640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-09-11T00:19:32.539 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:32.533+0000 7f9527e00640 1 -- 172.21.15.12:0/892315712 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9534035070 con 0x557916e5e790 2024-09-11T00:19:32.539 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:32.533+0000 7f9527e00640 1 -- 172.21.15.12:0/892315712 <== mon.2 v2:172.21.15.165:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9208+0+0 (secure 0 0 0) 0x7f9534003870 con 0x557916e5e790 2024-09-11T00:19:32.539 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:32.533+0000 7f954bcc1740 1 -- 172.21.15.12:0/892315712 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f953403d900 con 0x557916e5e790 2024-09-11T00:19:32.539 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:32.534+0000 7f9527e00640 1 -- 172.21.15.12:0/892315712 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f9534070050 con 0x557916e5e790 2024-09-11T00:19:32.539 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:32.534+0000 7f954bcc1740 10 client.26954 fetch_fsmap learned FSMap version 44 2024-09-11T00:19:32.539 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:32.534+0000 7f954bcc1740 1 -- 172.21.15.12:0/892315712 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x557916ee0980 con 0x557916e5e790 2024-09-11T00:19:32.539 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:32.534+0000 7f9527e00640 1 -- 172.21.15.12:0/892315712 <== mon.2 v2:172.21.15.165:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f953403d900 con 0x557916e5e790 2024-09-11T00:19:32.539 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:32.534+0000 7f9527e00640 1 -- 172.21.15.12:0/892315712 <== mon.2 v2:172.21.15.165:3300/0 5 ==== fsmap(e 44) ==== 8382+0+0 (secure 0 0 0) 0x7f9534065020 con 0x557916e5e790 2024-09-11T00:19:32.539 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:32.534+0000 7f954bcc1740 10 client.26954 fetch_fsmap finished waiting for FSMap version 44 2024-09-11T00:19:32.539 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:32.534+0000 7f954bcc1740 10 client.26954 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T00:19:32.539 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:32.534+0000 7f954bcc1740 20 client.26954 populate_metadata read hostname 'smithi012' 2024-09-11T00:19:32.540 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:32.534+0000 7f954bcc1740 1 --2- 172.21.15.12:0/892315712 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x557916e73810 0x557916ee3350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:19:32.540 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:32.534+0000 7f954bcc1740 4 client.26954 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T00:19:32.540 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:32.534+0000 7f954bcc1740 1 -- 172.21.15.12:0/892315712 --> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] -- command(tid 0: {"prefix": "flush journal"}) -- 0x557916dcf0b0 con 0x557916e73810 2024-09-11T00:19:32.540 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:32.535+0000 7f9539400640 1 --2- 172.21.15.12:0/892315712 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x557916e73810 0x557916ee3350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:19:32.540 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:32.535+0000 7f9539400640 1 --2- 172.21.15.12:0/892315712 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x557916e73810 0x557916ee3350 crc :-1 s=READY pgs=192 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9adf6b12986216e6 server_cookie=52e6f588a6692e97 in_seq=0 out_seq=0 2024-09-11T00:19:32.540 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:32.535+0000 7f9527e00640 10 client.26954 ms_handle_connect on v2:172.21.15.165:6838/501312984 2024-09-11T00:19:32.602 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:32.600+0000 7f9527e00640 1 -- 172.21.15.12:0/892315712 <== mds.0 v2:172.21.15.165:6838/501312984 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x557916dcf0b0 con 0x557916e73810 2024-09-11T00:19:32.602 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:32.600+0000 7f9527e00640 10 client.26954 handle_command_reply: tid=0 2024-09-11T00:19:32.602 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:32.600+0000 7f954bcc1740 1 client.26954 shutdown 2024-09-11T00:19:32.602 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:32.600+0000 7f954bcc1740 1 -- 172.21.15.12:0/892315712 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557916e5e790 msgr2=0x557916e6fbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:19:32.602 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:32.600+0000 7f954bcc1740 1 --2- 172.21.15.12:0/892315712 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557916e5e790 0x557916e6fbd0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f95340096c0 tx=0x7f9534004520 comp rx=0 tx=0).stop 2024-09-11T00:19:32.602 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:32.600+0000 7f954bcc1740 1 -- 172.21.15.12:0/892315712 shutdown_connections 2024-09-11T00:19:32.602 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:32.600+0000 7f954bcc1740 1 --2- 172.21.15.12:0/892315712 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x557916e73810 0x557916ee3350 crc :-1 s=READY pgs=192 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:19:32.603 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:32.600+0000 7f954bcc1740 1 --2- 172.21.15.12:0/892315712 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557916e63b00 0x557916e6efe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:19:32.603 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:32.600+0000 7f954bcc1740 1 --2- 172.21.15.12:0/892315712 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557916e610c0 0x557916e6e8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:19:32.603 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:32.600+0000 7f9527e00640 0 client.26954 ms_handle_reset on v2:172.21.15.165:6838/501312984 2024-09-11T00:19:32.603 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:32.600+0000 7f954bcc1740 1 --2- 172.21.15.12:0/892315712 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557916e5e790 0x557916e6fbd0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:19:32.603 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:32.600+0000 7f954bcc1740 1 -- 172.21.15.12:0/892315712 >> 172.21.15.12:0/892315712 conn(0x557916e598e0 msgr2=0x557916e5a140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:19:32.603 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:32.601+0000 7f954bcc1740 1 -- 172.21.15.12:0/892315712 shutdown_connections 2024-09-11T00:19:32.603 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:32.601+0000 7f954bcc1740 1 -- 172.21.15.12:0/892315712 wait complete. 2024-09-11T00:19:32.603 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:32.601+0000 7f954bcc1740 20 client.26954 trim_cache size 0 max 16384 2024-09-11T00:19:32.603 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:{ 2024-09-11T00:19:32.604 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "message": "", 2024-09-11T00:19:32.604 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "return_code": 0 2024-09-11T00:19:32.604 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:} 2024-09-11T00:19:32.604 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:32.601+0000 7f9548e00640 1 -- 172.21.15.12:0/2431850419 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f95280b5270 msgr2=0x7f95280b7730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:19:32.605 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:32.601+0000 7f9548e00640 1 --2- 172.21.15.12:0/2431850419 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f95280b5270 0x7f95280b7730 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f9534006010 tx=0x7f95340314a0 comp rx=0 tx=0).stop 2024-09-11T00:19:32.605 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:32.601+0000 7f9548e00640 1 -- 172.21.15.12:0/2431850419 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9544105cf0 msgr2=0x7f954419b5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:19:32.605 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:32.601+0000 7f9548e00640 1 --2- 172.21.15.12:0/2431850419 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9544105cf0 0x7f954419b5e0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f952c00e9f0 tx=0x7f952c00eeb0 comp rx=0 tx=0).stop 2024-09-11T00:19:32.605 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:32.602+0000 7f9548e00640 1 -- 172.21.15.12:0/2431850419 shutdown_connections 2024-09-11T00:19:32.605 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:32.602+0000 7f9548e00640 1 --2- 172.21.15.12:0/2431850419 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f95280b5270 0x7f95280b7730 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:19:32.605 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:32.602+0000 7f9548e00640 1 --2- 172.21.15.12:0/2431850419 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f95441066b0 0x7f954419c210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:19:32.605 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:32.602+0000 7f9548e00640 1 --2- 172.21.15.12:0/2431850419 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9544105cf0 0x7f954419b5e0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:19:32.605 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:32.602+0000 7f9548e00640 1 --2- 172.21.15.12:0/2431850419 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9544104ba0 0x7f954419af90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:19:32.605 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:32.602+0000 7f9548e00640 1 -- 172.21.15.12:0/2431850419 >> 172.21.15.12:0/2431850419 conn(0x7f9544100460 msgr2=0x7f95441090e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:19:32.605 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:32.602+0000 7f9548e00640 1 -- 172.21.15.12:0/2431850419 shutdown_connections 2024-09-11T00:19:32.605 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:32.602+0000 7f9548e00640 1 -- 172.21.15.12:0/2431850419 wait complete. 2024-09-11T00:19:32.698 INFO:tasks.workunit.client.0.smithi012.stdout: 1 502267 3.87 MB/sec execute 569 sec latency 59.841 ms 2024-09-11T00:19:33.699 INFO:tasks.workunit.client.0.smithi012.stdout: 1 502267 3.87 MB/sec execute 570 sec latency 1058.269 ms 2024-09-11T00:19:34.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:19:34 smithi012 ceph-mon[30333]: pgmap v773: 129 pgs: 129 active+clean; 115 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 66 MiB/s rd, 1.1 MiB/s wr, 287 op/s 2024-09-11T00:19:34.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:19:34 smithi178 ceph-mon[35919]: pgmap v773: 129 pgs: 129 active+clean; 115 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 66 MiB/s rd, 1.1 MiB/s wr, 287 op/s 2024-09-11T00:19:34.532 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:19:34 smithi165 ceph-mon[35356]: pgmap v773: 129 pgs: 129 active+clean; 115 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 66 MiB/s rd, 1.1 MiB/s wr, 287 op/s 2024-09-11T00:19:34.698 INFO:tasks.workunit.client.0.smithi012.stdout: 1 502267 3.86 MB/sec execute 571 sec latency 2058.329 ms 2024-09-11T00:19:35.699 INFO:tasks.workunit.client.0.smithi012.stdout: 1 503653 3.87 MB/sec execute 572 sec latency 2281.928 ms 2024-09-11T00:19:36.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:19:36 smithi012 ceph-mon[30333]: pgmap v774: 129 pgs: 129 active+clean; 115 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 66 MiB/s rd, 1.0 MiB/s wr, 263 op/s 2024-09-11T00:19:36.330 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:19:36 smithi178 ceph-mon[35919]: pgmap v774: 129 pgs: 129 active+clean; 115 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 66 MiB/s rd, 1.0 MiB/s wr, 263 op/s 2024-09-11T00:19:36.532 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:19:36 smithi165 ceph-mon[35356]: pgmap v774: 129 pgs: 129 active+clean; 115 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 66 MiB/s rd, 1.0 MiB/s wr, 263 op/s 2024-09-11T00:19:36.699 INFO:tasks.workunit.client.0.smithi012.stdout: 1 504671 3.87 MB/sec execute 573 sec latency 206.863 ms 2024-09-11T00:19:37.699 INFO:tasks.workunit.client.0.smithi012.stdout: 1 505749 3.87 MB/sec execute 574 sec latency 12.119 ms 2024-09-11T00:19:38.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:19:38 smithi012 ceph-mon[30333]: pgmap v775: 129 pgs: 129 active+clean; 116 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 132 MiB/s rd, 1.3 MiB/s wr, 381 op/s 2024-09-11T00:19:38.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:19:38 smithi178 ceph-mon[35919]: pgmap v775: 129 pgs: 129 active+clean; 116 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 132 MiB/s rd, 1.3 MiB/s wr, 381 op/s 2024-09-11T00:19:38.532 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:19:38 smithi165 ceph-mon[35356]: pgmap v775: 129 pgs: 129 active+clean; 116 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 132 MiB/s rd, 1.3 MiB/s wr, 381 op/s 2024-09-11T00:19:38.699 INFO:tasks.workunit.client.0.smithi012.stdout: 1 507146 3.88 MB/sec execute 575 sec latency 240.208 ms 2024-09-11T00:19:39.699 INFO:tasks.workunit.client.0.smithi012.stdout: 1 507146 3.87 MB/sec execute 576 sec latency 1240.271 ms 2024-09-11T00:19:40.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:19:40 smithi012 ceph-mon[30333]: pgmap v776: 129 pgs: 129 active+clean; 116 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 104 MiB/s rd, 836 KiB/s wr, 236 op/s 2024-09-11T00:19:40.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:19:40 smithi178 ceph-mon[35919]: pgmap v776: 129 pgs: 129 active+clean; 116 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 104 MiB/s rd, 836 KiB/s wr, 236 op/s 2024-09-11T00:19:40.532 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:19:40 smithi165 ceph-mon[35356]: pgmap v776: 129 pgs: 129 active+clean; 116 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 104 MiB/s rd, 836 KiB/s wr, 236 op/s 2024-09-11T00:19:40.699 INFO:tasks.workunit.client.0.smithi012.stdout: 1 507845 3.87 MB/sec execute 577 sec latency 1885.901 ms 2024-09-11T00:19:41.699 INFO:tasks.workunit.client.0.smithi012.stdout: 1 508283 3.87 MB/sec execute 578 sec latency 521.273 ms 2024-09-11T00:19:42.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:19:42 smithi012 ceph-mon[30333]: pgmap v777: 129 pgs: 129 active+clean; 118 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 155 MiB/s rd, 1.1 MiB/s wr, 319 op/s 2024-09-11T00:19:42.330 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:19:42 smithi178 ceph-mon[35919]: pgmap v777: 129 pgs: 129 active+clean; 118 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 155 MiB/s rd, 1.1 MiB/s wr, 319 op/s 2024-09-11T00:19:42.334 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:19:42 smithi165 ceph-mon[35356]: pgmap v777: 129 pgs: 129 active+clean; 118 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 155 MiB/s rd, 1.1 MiB/s wr, 319 op/s 2024-09-11T00:19:42.699 INFO:tasks.workunit.client.0.smithi012.stdout: 1 509360 3.87 MB/sec execute 579 sec latency 97.515 ms 2024-09-11T00:19:43.699 INFO:tasks.workunit.client.0.smithi012.stdout: 1 510948 3.88 MB/sec execute 580 sec latency 236.740 ms 2024-09-11T00:19:44.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:19:44 smithi012 ceph-mon[30333]: pgmap v778: 129 pgs: 129 active+clean; 119 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 117 MiB/s rd, 880 KiB/s wr, 257 op/s 2024-09-11T00:19:44.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:19:44 smithi178 ceph-mon[35919]: pgmap v778: 129 pgs: 129 active+clean; 119 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 117 MiB/s rd, 880 KiB/s wr, 257 op/s 2024-09-11T00:19:44.532 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:19:44 smithi165 ceph-mon[35356]: pgmap v778: 129 pgs: 129 active+clean; 119 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 117 MiB/s rd, 880 KiB/s wr, 257 op/s 2024-09-11T00:19:44.699 INFO:tasks.workunit.client.0.smithi012.stdout: 1 511576 3.88 MB/sec execute 581 sec latency 625.509 ms 2024-09-11T00:19:45.699 INFO:tasks.workunit.client.0.smithi012.stdout: 1 512560 3.88 MB/sec execute 582 sec latency 15.691 ms 2024-09-11T00:19:45.732 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:45.731+0000 7fe6dfe00640 1 -- 172.21.15.12:0/109786055 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe6e0076f30 msgr2=0x7fe6e0075580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:19:45.732 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:45.731+0000 7fe6dfe00640 1 --2- 172.21.15.12:0/109786055 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe6e0076f30 0x7fe6e0075580 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7fe6d0009860 tx=0x7fe6d002a340 comp rx=0 tx=0).stop 2024-09-11T00:19:45.732 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:45.731+0000 7fe6dfe00640 1 -- 172.21.15.12:0/109786055 shutdown_connections 2024-09-11T00:19:45.732 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:45.731+0000 7fe6dfe00640 1 --2- 172.21.15.12:0/109786055 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe6e010a710 0x7fe6e010cb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:19:45.733 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:45.731+0000 7fe6dfe00640 1 --2- 172.21.15.12:0/109786055 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe6e0075ac0 0x7fe6e0075f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:19:45.733 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:45.731+0000 7fe6dfe00640 1 --2- 172.21.15.12:0/109786055 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe6e0076f30 0x7fe6e0075580 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:19:45.733 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:45.731+0000 7fe6dfe00640 1 -- 172.21.15.12:0/109786055 >> 172.21.15.12:0/109786055 conn(0x7fe6e00fe030 msgr2=0x7fe6e0100470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:19:45.733 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:45.731+0000 7fe6dfe00640 1 -- 172.21.15.12:0/109786055 shutdown_connections 2024-09-11T00:19:45.733 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:45.731+0000 7fe6dfe00640 1 -- 172.21.15.12:0/109786055 wait complete. 2024-09-11T00:19:45.735 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:45.732+0000 7fe6dfe00640 1 Processor -- start 2024-09-11T00:19:45.735 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:45.732+0000 7fe6dfe00640 1 -- start start 2024-09-11T00:19:45.735 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:45.732+0000 7fe6dfe00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe6e0075ac0 0x7fe6e019ae00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:19:45.735 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:45.732+0000 7fe6dfe00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe6e0076f30 0x7fe6e019b450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:19:45.735 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:45.732+0000 7fe6dfe00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe6e010a710 0x7fe6e019c010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:19:45.736 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:45.732+0000 7fe6dfe00640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fe6e0110600 con 0x7fe6e0075ac0 2024-09-11T00:19:45.736 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:45.732+0000 7fe6dfe00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fe6e0110480 con 0x7fe6e0076f30 2024-09-11T00:19:45.736 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:45.732+0000 7fe6dfe00640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fe6e0110780 con 0x7fe6e010a710 2024-09-11T00:19:45.736 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:45.733+0000 7fe6de000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe6e0076f30 0x7fe6e019b450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:19:45.736 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:45.733+0000 7fe6dea00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe6e0075ac0 0x7fe6e019ae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:19:45.736 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:45.733+0000 7fe6de000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe6e0076f30 0x7fe6e019b450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.12:42694/0 (socket says 172.21.15.12:42694) 2024-09-11T00:19:45.736 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:45.733+0000 7fe6de000640 1 -- 172.21.15.12:0/2797648032 learned_addr learned my addr 172.21.15.12:0/2797648032 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:19:45.736 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:45.733+0000 7fe6df400640 1 --2- 172.21.15.12:0/2797648032 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe6e010a710 0x7fe6e019c010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:19:45.736 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:45.733+0000 7fe6de000640 1 -- 172.21.15.12:0/2797648032 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe6e010a710 msgr2=0x7fe6e019c010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:19:45.736 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:45.733+0000 7fe6de000640 1 --2- 172.21.15.12:0/2797648032 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe6e010a710 0x7fe6e019c010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:19:45.736 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:45.733+0000 7fe6de000640 1 -- 172.21.15.12:0/2797648032 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe6e0075ac0 msgr2=0x7fe6e019ae00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:19:45.736 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:45.733+0000 7fe6de000640 1 --2- 172.21.15.12:0/2797648032 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe6e0075ac0 0x7fe6e019ae00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:19:45.737 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:45.733+0000 7fe6de000640 1 -- 172.21.15.12:0/2797648032 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6e019c8d0 con 0x7fe6e0076f30 2024-09-11T00:19:45.737 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:45.733+0000 7fe6de000640 1 --2- 172.21.15.12:0/2797648032 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe6e0076f30 0x7fe6e019b450 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fe6c800e9c0 tx=0x7fe6c800ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:19:45.737 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:45.734+0000 7fe6d7400640 1 -- 172.21.15.12:0/2797648032 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6c800ccc0 con 0x7fe6e0076f30 2024-09-11T00:19:45.737 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:45.734+0000 7fe6d7400640 1 -- 172.21.15.12:0/2797648032 <== mon.2 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe6c80054d0 con 0x7fe6e0076f30 2024-09-11T00:19:45.737 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:45.734+0000 7fe6d7400640 1 -- 172.21.15.12:0/2797648032 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6c8010430 con 0x7fe6e0076f30 2024-09-11T00:19:45.737 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:45.734+0000 7fe6dfe00640 1 -- 172.21.15.12:0/2797648032 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe6e019cbc0 con 0x7fe6e0076f30 2024-09-11T00:19:45.737 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:45.734+0000 7fe6dfe00640 1 -- 172.21.15.12:0/2797648032 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6e01a3290 con 0x7fe6e0076f30 2024-09-11T00:19:45.738 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:45.737+0000 7fe6e8a82740 1 Processor -- start 2024-09-11T00:19:45.738 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:45.737+0000 7fe6e8a82740 1 -- start start 2024-09-11T00:19:45.738 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:45.737+0000 7fe6e8a82740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5572803cb970 0x5572803cdd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:19:45.738 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:45.737+0000 7fe6e8a82740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5572803ce2a0 0x5572803d0690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:19:45.738 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:45.737+0000 7fe6e8a82740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5572803d0ce0 0x5572803d31a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:19:45.739 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:45.737+0000 7fe6e8a82740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55728033e6b0 con 0x5572803cb970 2024-09-11T00:19:45.739 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:45.737+0000 7fe6e8a82740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55728032dae0 con 0x5572803d0ce0 2024-09-11T00:19:45.739 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:45.737+0000 7fe6e8a82740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5572803318f0 con 0x5572803ce2a0 2024-09-11T00:19:45.740 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:45.738+0000 7fe6dea00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5572803ce2a0 0x5572803d0690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:19:45.740 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:45.738+0000 7fe6df400640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5572803cb970 0x5572803cdd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:19:45.740 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:45.738+0000 7fe6dea00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5572803ce2a0 0x5572803d0690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.12:58148/0 (socket says 172.21.15.12:58148) 2024-09-11T00:19:45.740 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:45.738+0000 7fe6dea00640 1 -- 172.21.15.12:0/78692309 learned_addr learned my addr 172.21.15.12:0/78692309 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:19:45.740 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:45.738+0000 7fe6d7400640 1 -- 172.21.15.12:0/2797648032 <== mon.2 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7fe6c8004f80 con 0x7fe6e0076f30 2024-09-11T00:19:45.741 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:45.738+0000 7fe6dea00640 1 -- 172.21.15.12:0/78692309 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5572803d0ce0 msgr2=0x5572803d31a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-11T00:19:45.741 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:45.738+0000 7fe6dea00640 1 --2- 172.21.15.12:0/78692309 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5572803d0ce0 0x5572803d31a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:19:45.741 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:45.738+0000 7fe6dea00640 1 -- 172.21.15.12:0/78692309 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5572803cb970 msgr2=0x5572803cdd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:19:45.741 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:45.738+0000 7fe6dea00640 1 --2- 172.21.15.12:0/78692309 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5572803cb970 0x5572803cdd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:19:45.741 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:45.738+0000 7fe6dea00640 1 -- 172.21.15.12:0/78692309 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5572802fa780 con 0x5572803ce2a0 2024-09-11T00:19:45.741 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:45.738+0000 7fe6df400640 1 --2- 172.21.15.12:0/78692309 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5572803cb970 0x5572803cdd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:19:45.741 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:45.739+0000 7fe6dea00640 1 --2- 172.21.15.12:0/78692309 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5572803ce2a0 0x5572803d0690 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fe6d00057c0 tx=0x7fe6d0005b20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=54b9684fddfa74a1 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:19:45.741 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:45.739+0000 7fe6cfe00640 1 -- 172.21.15.12:0/78692309 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6d0041020 con 0x5572803ce2a0 2024-09-11T00:19:45.741 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:45.739+0000 7fe6cfe00640 1 -- 172.21.15.12:0/78692309 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe6d0030dd0 con 0x5572803ce2a0 2024-09-11T00:19:45.741 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:45.739+0000 7fe6cfe00640 1 -- 172.21.15.12:0/78692309 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6d0037040 con 0x5572803ce2a0 2024-09-11T00:19:45.744 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:45.740+0000 7fe6e8a82740 1 -- 172.21.15.12:0/78692309 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5572803ce2a0 msgr2=0x5572803d0690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:19:45.744 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:45.740+0000 7fe6e8a82740 1 --2- 172.21.15.12:0/78692309 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5572803ce2a0 0x5572803d0690 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fe6d00057c0 tx=0x7fe6d0005b20 comp rx=0 tx=0).stop 2024-09-11T00:19:45.744 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:45.740+0000 7fe6e8a82740 1 -- 172.21.15.12:0/78692309 shutdown_connections 2024-09-11T00:19:45.744 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:45.740+0000 7fe6e8a82740 1 --2- 172.21.15.12:0/78692309 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5572803d0ce0 0x5572803d31a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:19:45.744 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:45.740+0000 7fe6e8a82740 1 --2- 172.21.15.12:0/78692309 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5572803ce2a0 0x5572803d0690 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:19:45.744 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:45.740+0000 7fe6e8a82740 1 --2- 172.21.15.12:0/78692309 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5572803cb970 0x5572803cdd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:19:45.744 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:45.740+0000 7fe6e8a82740 1 -- 172.21.15.12:0/78692309 >> 172.21.15.12:0/78692309 conn(0x5572803c6ac0 msgr2=0x5572803c7850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:19:45.744 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:45.740+0000 7fe6e8a82740 1 -- 172.21.15.12:0/78692309 shutdown_connections 2024-09-11T00:19:45.744 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:45.740+0000 7fe6e8a82740 1 -- 172.21.15.12:0/78692309 wait complete. 2024-09-11T00:19:45.744 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:45.740+0000 7fe6d7400640 1 --2- 172.21.15.12:0/2797648032 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fe6c40b5200 0x7fe6c40b76c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:19:45.744 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:45.741+0000 7fe6dea00640 1 --2- 172.21.15.12:0/2797648032 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fe6c40b5200 0x7fe6c40b76c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:19:45.745 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:45.741+0000 7fe6d7400640 1 -- 172.21.15.12:0/2797648032 <== mon.2 v2:172.21.15.165:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9208+0+0 (secure 0 0 0) 0x7fe6c80e6390 con 0x7fe6e0076f30 2024-09-11T00:19:45.745 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:45.741+0000 7fe6e8a82740 1 -- start start 2024-09-11T00:19:45.745 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:45.741+0000 7fe6e8a82740 1 Processor -- start 2024-09-11T00:19:45.745 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:45.742+0000 7fe6dea00640 1 --2- 172.21.15.12:0/2797648032 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fe6c40b5200 0x7fe6c40b76c0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fe6d0006010 tx=0x7fe6d0000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:19:45.745 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:45.742+0000 7fe6e8a82740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5572803cb970 0x5572803d50e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:19:45.745 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:45.742+0000 7fe6e8a82740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5572803ce2a0 0x5572803d5780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:19:45.745 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:45.742+0000 7fe6e8a82740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5572803d0ce0 0x5572803d62e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:19:45.745 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:45.742+0000 7fe6e8a82740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x557280265ab0 con 0x5572803ce2a0 2024-09-11T00:19:45.745 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:45.742+0000 7fe6e8a82740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x5572802d4570 con 0x5572803cb970 2024-09-11T00:19:45.745 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:45.742+0000 7fe6e8a82740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5572803d3c60 con 0x5572803d0ce0 2024-09-11T00:19:45.745 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:45.742+0000 7fe6e8a82740 10 client.0 _ll_register_callbacks cb 0x5572802e3100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:19:45.745 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:45.742+0000 7fe6dea00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5572803cb970 0x5572803d50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:19:45.745 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:45.742+0000 7fe6dea00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5572803cb970 0x5572803d50e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.12:42696/0 (socket says 172.21.15.12:42696) 2024-09-11T00:19:45.746 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:45.742+0000 7fe6dea00640 1 -- 172.21.15.12:0/1597389411 learned_addr learned my addr 172.21.15.12:0/1597389411 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:19:45.746 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:45.742+0000 7fe6de000640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5572803ce2a0 0x5572803d5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:19:45.746 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:45.742+0000 7fe6df400640 1 --2- 172.21.15.12:0/1597389411 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5572803d0ce0 0x5572803d62e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:19:45.746 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:45.742+0000 7fe6dea00640 1 -- 172.21.15.12:0/1597389411 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5572803d0ce0 msgr2=0x5572803d62e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:19:45.746 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:45.743+0000 7fe6dea00640 1 --2- 172.21.15.12:0/1597389411 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5572803d0ce0 0x5572803d62e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:19:45.746 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:45.743+0000 7fe6dea00640 1 -- 172.21.15.12:0/1597389411 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5572803ce2a0 msgr2=0x5572803d5780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:19:45.746 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:45.743+0000 7fe6dea00640 1 --2- 172.21.15.12:0/1597389411 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5572803ce2a0 0x5572803d5780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:19:45.746 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:45.743+0000 7fe6dea00640 1 -- 172.21.15.12:0/1597389411 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55728008f790 con 0x5572803cb970 2024-09-11T00:19:45.748 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:45.743+0000 7fe6dea00640 1 --2- 172.21.15.12:0/1597389411 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5572803cb970 0x5572803d50e0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fe6d00044e0 tx=0x7fe6d0005b20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6b7e8fc1d6278a96 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:19:45.748 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:45.743+0000 7fe6ccc00640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-09-11T00:19:45.749 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:45.743+0000 7fe6ccc00640 1 -- 172.21.15.12:0/1597389411 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6d0056030 con 0x5572803cb970 2024-09-11T00:19:45.749 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:45.743+0000 7fe6e8a82740 1 -- 172.21.15.12:0/1597389411 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe6d0030dd0 con 0x5572803cb970 2024-09-11T00:19:45.749 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:45.743+0000 7fe6ccc00640 1 -- 172.21.15.12:0/1597389411 <== mon.2 v2:172.21.15.165:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9208+0+0 (secure 0 0 0) 0x7fe6d0003ab0 con 0x5572803cb970 2024-09-11T00:19:45.749 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:45.744+0000 7fe6ccc00640 1 -- 172.21.15.12:0/1597389411 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7fe6d0041040 con 0x5572803cb970 2024-09-11T00:19:45.749 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:45.744+0000 7fe6e8a82740 10 client.26963 fetch_fsmap learned FSMap version 44 2024-09-11T00:19:45.749 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:45.744+0000 7fe6e8a82740 1 -- 172.21.15.12:0/1597389411 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x557280108620 con 0x5572803cb970 2024-09-11T00:19:45.749 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:45.744+0000 7fe6ccc00640 1 -- 172.21.15.12:0/1597389411 <== mon.2 v2:172.21.15.165:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe6d0030dd0 con 0x5572803cb970 2024-09-11T00:19:45.749 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:45.744+0000 7fe6ccc00640 1 -- 172.21.15.12:0/1597389411 <== mon.2 v2:172.21.15.165:3300/0 5 ==== fsmap(e 44) ==== 8382+0+0 (secure 0 0 0) 0x7fe6d005d4e0 con 0x5572803cb970 2024-09-11T00:19:45.749 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:45.744+0000 7fe6e8a82740 10 client.26963 fetch_fsmap finished waiting for FSMap version 44 2024-09-11T00:19:45.749 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:45.744+0000 7fe6e8a82740 10 client.26963 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T00:19:45.749 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:45.744+0000 7fe6e8a82740 20 client.26963 populate_metadata read hostname 'smithi012' 2024-09-11T00:19:45.749 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:45.744+0000 7fe6e8a82740 1 --2- 172.21.15.12:0/1597389411 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x5572803e0510 0x5572803e2900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:19:45.749 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:45.744+0000 7fe6e8a82740 4 client.26963 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T00:19:45.750 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:45.744+0000 7fe6e8a82740 1 -- 172.21.15.12:0/1597389411 --> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557280340600 con 0x5572803e0510 2024-09-11T00:19:45.750 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:45.745+0000 7fe6de000640 1 --2- 172.21.15.12:0/1597389411 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x5572803e0510 0x5572803e2900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:19:45.750 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:45.745+0000 7fe6ccc00640 10 client.26963 ms_handle_connect on v2:172.21.15.165:6838/501312984 2024-09-11T00:19:45.750 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:45.745+0000 7fe6de000640 1 --2- 172.21.15.12:0/1597389411 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x5572803e0510 0x5572803e2900 crc :-1 s=READY pgs=193 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=86597a3a9f6b99c3 server_cookie=6764deb1811ee61c in_seq=0 out_seq=0 2024-09-11T00:19:45.750 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:45.746+0000 7fe6ccc00640 1 -- 172.21.15.12:0/1597389411 <== mds.0 v2:172.21.15.165:6838/501312984 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x557280340600 con 0x5572803e0510 2024-09-11T00:19:45.750 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:45.746+0000 7fe6ccc00640 10 client.26963 handle_command_reply: tid=0 2024-09-11T00:19:45.750 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:45.746+0000 7fe6e8a82740 1 client.26963 shutdown 2024-09-11T00:19:45.750 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:45.746+0000 7fe6e8a82740 1 -- 172.21.15.12:0/1597389411 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5572803cb970 msgr2=0x5572803d50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:19:45.750 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:45.746+0000 7fe6e8a82740 1 --2- 172.21.15.12:0/1597389411 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5572803cb970 0x5572803d50e0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fe6d00044e0 tx=0x7fe6d0005b20 comp rx=0 tx=0).stop 2024-09-11T00:19:45.750 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:45.746+0000 7fe6e8a82740 1 -- 172.21.15.12:0/1597389411 shutdown_connections 2024-09-11T00:19:45.750 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:45.746+0000 7fe6e8a82740 1 --2- 172.21.15.12:0/1597389411 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x5572803e0510 0x5572803e2900 crc :-1 s=READY pgs=193 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:19:45.750 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:45.746+0000 7fe6e8a82740 1 --2- 172.21.15.12:0/1597389411 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5572803d0ce0 0x5572803d62e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:19:45.750 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:45.746+0000 7fe6e8a82740 1 --2- 172.21.15.12:0/1597389411 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5572803ce2a0 0x5572803d5780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:19:45.751 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:45.746+0000 7fe6e8a82740 1 --2- 172.21.15.12:0/1597389411 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5572803cb970 0x5572803d50e0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:19:45.751 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:45.746+0000 7fe6e8a82740 1 -- 172.21.15.12:0/1597389411 >> 172.21.15.12:0/1597389411 conn(0x5572803c6ac0 msgr2=0x5572803c5bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:19:45.751 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:45.746+0000 7fe6ccc00640 0 client.26963 ms_handle_reset on v2:172.21.15.165:6838/501312984 2024-09-11T00:19:45.751 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:45.747+0000 7fe6e8a82740 1 -- 172.21.15.12:0/1597389411 shutdown_connections 2024-09-11T00:19:45.751 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:45.747+0000 7fe6e8a82740 1 -- 172.21.15.12:0/1597389411 wait complete. 2024-09-11T00:19:45.751 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:45.747+0000 7fe6e8a82740 20 client.26963 trim_cache size 0 max 16384 2024-09-11T00:19:45.765 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:45.762+0000 7fe6e8a82740 1 Processor -- start 2024-09-11T00:19:45.765 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:45.762+0000 7fe6e8a82740 1 -- start start 2024-09-11T00:19:45.765 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:45.763+0000 7fe6e8a82740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5572803cb970 0x5572803d1150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:19:45.765 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:45.763+0000 7fe6e8a82740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5572803ce2a0 0x5572803d1690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:19:45.765 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:45.763+0000 7fe6e8a82740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5572803d0ce0 0x5572803c88f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:19:45.765 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:45.763+0000 7fe6e8a82740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fe6e01ad7f0 con 0x5572803cb970 2024-09-11T00:19:45.765 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:45.763+0000 7fe6e8a82740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fe6e01af3c0 con 0x5572803d0ce0 2024-09-11T00:19:45.765 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:45.763+0000 7fe6e8a82740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fe6e01adef0 con 0x5572803ce2a0 2024-09-11T00:19:45.765 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:45.763+0000 7fe6de000640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5572803ce2a0 0x5572803d1690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:19:45.765 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:45.763+0000 7fe6de000640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5572803ce2a0 0x5572803d1690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.12:58162/0 (socket says 172.21.15.12:58162) 2024-09-11T00:19:45.766 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:45.763+0000 7fe6de000640 1 -- 172.21.15.12:0/602051548 learned_addr learned my addr 172.21.15.12:0/602051548 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:19:45.766 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:45.763+0000 7fe6dea00640 1 --2- 172.21.15.12:0/602051548 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5572803cb970 0x5572803d1150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:19:45.766 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:45.763+0000 7fe6df400640 1 --2- 172.21.15.12:0/602051548 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5572803d0ce0 0x5572803c88f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:19:45.766 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:45.763+0000 7fe6de000640 1 -- 172.21.15.12:0/602051548 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5572803d0ce0 msgr2=0x5572803c88f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:19:45.766 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:45.763+0000 7fe6de000640 1 --2- 172.21.15.12:0/602051548 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5572803d0ce0 0x5572803c88f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:19:45.766 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:45.763+0000 7fe6de000640 1 -- 172.21.15.12:0/602051548 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5572803cb970 msgr2=0x5572803d1150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:19:45.766 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:45.763+0000 7fe6de000640 1 --2- 172.21.15.12:0/602051548 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5572803cb970 0x5572803d1150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:19:45.766 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:45.763+0000 7fe6de000640 1 -- 172.21.15.12:0/602051548 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5572803e2c70 con 0x5572803ce2a0 2024-09-11T00:19:45.766 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:45.764+0000 7fe6de000640 1 --2- 172.21.15.12:0/602051548 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5572803ce2a0 0x5572803d1690 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fe6c80a8d50 tx=0x7fe6c80094c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6adf10fe8a2edaef server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:19:45.766 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:45.764+0000 7fe6cf400640 1 -- 172.21.15.12:0/602051548 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6c80ace80 con 0x5572803ce2a0 2024-09-11T00:19:45.766 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:45.764+0000 7fe6cf400640 1 -- 172.21.15.12:0/602051548 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe6c80ac780 con 0x5572803ce2a0 2024-09-11T00:19:45.766 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:45.764+0000 7fe6cf400640 1 -- 172.21.15.12:0/602051548 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6c80052a0 con 0x5572803ce2a0 2024-09-11T00:19:45.770 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:45.765+0000 7fe6e8a82740 1 -- 172.21.15.12:0/602051548 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5572803ce2a0 msgr2=0x5572803d1690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:19:45.770 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:45.765+0000 7fe6e8a82740 1 --2- 172.21.15.12:0/602051548 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5572803ce2a0 0x5572803d1690 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fe6c80a8d50 tx=0x7fe6c80094c0 comp rx=0 tx=0).stop 2024-09-11T00:19:45.770 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:45.765+0000 7fe6e8a82740 1 -- 172.21.15.12:0/602051548 shutdown_connections 2024-09-11T00:19:45.770 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:45.765+0000 7fe6e8a82740 1 --2- 172.21.15.12:0/602051548 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5572803d0ce0 0x5572803c88f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:19:45.770 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:45.765+0000 7fe6e8a82740 1 --2- 172.21.15.12:0/602051548 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5572803ce2a0 0x5572803d1690 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:19:45.770 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:45.765+0000 7fe6e8a82740 1 --2- 172.21.15.12:0/602051548 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5572803cb970 0x5572803d1150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:19:45.770 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:45.765+0000 7fe6e8a82740 1 -- 172.21.15.12:0/602051548 >> 172.21.15.12:0/602051548 conn(0x5572803c6ac0 msgr2=0x5572803c7320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:19:45.770 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:45.765+0000 7fe6e8a82740 1 -- 172.21.15.12:0/602051548 shutdown_connections 2024-09-11T00:19:45.770 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:45.765+0000 7fe6e8a82740 1 -- 172.21.15.12:0/602051548 wait complete. 2024-09-11T00:19:45.770 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:45.765+0000 7fe6e8a82740 1 -- start start 2024-09-11T00:19:45.770 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:45.765+0000 7fe6e8a82740 1 Processor -- start 2024-09-11T00:19:45.770 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:45.766+0000 7fe6e8a82740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5572803cb970 0x5572803dbc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:19:45.771 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:45.766+0000 7fe6e8a82740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5572803ce2a0 0x5572803dc140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:19:45.771 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:45.766+0000 7fe6e8a82740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5572803d0ce0 0x5572803dc680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:19:45.771 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:45.766+0000 7fe6e8a82740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fe6e01aecf0 con 0x5572803cb970 2024-09-11T00:19:45.771 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:45.766+0000 7fe6e8a82740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fe6e01ae5c0 con 0x5572803ce2a0 2024-09-11T00:19:45.771 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:45.766+0000 7fe6e8a82740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fe6e01ab7d0 con 0x5572803d0ce0 2024-09-11T00:19:45.771 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:45.766+0000 7fe6e8a82740 10 client.0 _ll_register_callbacks cb 0x7fe6d8009b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:19:45.771 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:45.766+0000 7fe6de000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5572803ce2a0 0x5572803dc140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:19:45.771 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:45.766+0000 7fe6dea00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5572803cb970 0x5572803dbc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:19:45.771 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:45.766+0000 7fe6df400640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5572803d0ce0 0x5572803dc680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:19:45.771 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:45.766+0000 7fe6de000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5572803ce2a0 0x5572803dc140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.12:42716/0 (socket says 172.21.15.12:42716) 2024-09-11T00:19:45.771 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:45.766+0000 7fe6de000640 1 -- 172.21.15.12:0/319874553 learned_addr learned my addr 172.21.15.12:0/319874553 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:19:45.771 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:45.766+0000 7fe6de000640 1 -- 172.21.15.12:0/319874553 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5572803d0ce0 msgr2=0x5572803dc680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:19:45.771 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:45.766+0000 7fe6de000640 1 --2- 172.21.15.12:0/319874553 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5572803d0ce0 0x5572803dc680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:19:45.771 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:45.766+0000 7fe6de000640 1 -- 172.21.15.12:0/319874553 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5572803cb970 msgr2=0x5572803dbc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:19:45.772 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:45.766+0000 7fe6de000640 1 --2- 172.21.15.12:0/319874553 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5572803cb970 0x5572803dbc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:19:45.772 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:45.766+0000 7fe6de000640 1 -- 172.21.15.12:0/319874553 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5572803da8d0 con 0x5572803ce2a0 2024-09-11T00:19:45.772 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:45.766+0000 7fe6dea00640 1 --2- 172.21.15.12:0/319874553 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5572803cb970 0x5572803dbc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:19:45.772 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:45.767+0000 7fe6de000640 1 --2- 172.21.15.12:0/319874553 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5572803ce2a0 0x5572803dc140 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fe6c80ac480 tx=0x7fe6c800efb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=355e02d608a10de6 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:19:45.772 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:45.767+0000 7fe6ccc00640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-09-11T00:19:45.772 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:45.767+0000 7fe6ccc00640 1 -- 172.21.15.12:0/319874553 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6c80b1260 con 0x5572803ce2a0 2024-09-11T00:19:45.772 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:45.767+0000 7fe6ccc00640 1 -- 172.21.15.12:0/319874553 <== mon.2 v2:172.21.15.165:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9208+0+0 (secure 0 0 0) 0x7fe6c80a89f0 con 0x5572803ce2a0 2024-09-11T00:19:45.772 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:45.767+0000 7fe6e8a82740 1 -- 172.21.15.12:0/319874553 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe6c80ac780 con 0x5572803ce2a0 2024-09-11T00:19:45.772 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:45.767+0000 7fe6ccc00640 1 -- 172.21.15.12:0/319874553 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7fe6c80b1020 con 0x5572803ce2a0 2024-09-11T00:19:45.772 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:45.767+0000 7fe6e8a82740 10 client.26969 fetch_fsmap learned FSMap version 44 2024-09-11T00:19:45.772 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:45.767+0000 7fe6e8a82740 1 -- 172.21.15.12:0/319874553 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x55728044d8b0 con 0x5572803ce2a0 2024-09-11T00:19:45.772 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:45.768+0000 7fe6ccc00640 1 -- 172.21.15.12:0/319874553 <== mon.2 v2:172.21.15.165:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe6c80ac780 con 0x5572803ce2a0 2024-09-11T00:19:45.772 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:45.768+0000 7fe6ccc00640 1 -- 172.21.15.12:0/319874553 <== mon.2 v2:172.21.15.165:3300/0 5 ==== fsmap(e 44) ==== 8382+0+0 (secure 0 0 0) 0x7fe6c80cfad0 con 0x5572803ce2a0 2024-09-11T00:19:45.772 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:45.768+0000 7fe6e8a82740 10 client.26969 fetch_fsmap finished waiting for FSMap version 44 2024-09-11T00:19:45.773 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:45.768+0000 7fe6e8a82740 10 client.26969 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T00:19:45.773 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:45.768+0000 7fe6e8a82740 20 client.26969 populate_metadata read hostname 'smithi012' 2024-09-11T00:19:45.773 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:45.768+0000 7fe6e8a82740 1 --2- 172.21.15.12:0/319874553 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55728044e2f0 0x557280450710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:19:45.773 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:45.768+0000 7fe6e8a82740 4 client.26969 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T00:19:45.773 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:45.768+0000 7fe6e8a82740 1 -- 172.21.15.12:0/319874553 --> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55728033c0b0 con 0x55728044e2f0 2024-09-11T00:19:45.773 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:45.768+0000 7fe6dea00640 1 --2- 172.21.15.12:0/319874553 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55728044e2f0 0x557280450710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:19:45.773 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:45.769+0000 7fe6dea00640 1 --2- 172.21.15.12:0/319874553 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55728044e2f0 0x557280450710 crc :-1 s=READY pgs=194 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=991e759758fcece6 server_cookie=eaedbb698fa2ec1c in_seq=0 out_seq=0 2024-09-11T00:19:45.773 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:45.769+0000 7fe6ccc00640 10 client.26969 ms_handle_connect on v2:172.21.15.165:6838/501312984 2024-09-11T00:19:45.818 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:45.816+0000 7fe6ccc00640 1 -- 172.21.15.12:0/319874553 <== mds.0 v2:172.21.15.165:6838/501312984 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55728033c0b0 con 0x55728044e2f0 2024-09-11T00:19:45.818 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:45.816+0000 7fe6ccc00640 10 client.26969 handle_command_reply: tid=0 2024-09-11T00:19:45.818 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:45.816+0000 7fe6e8a82740 1 client.26969 shutdown 2024-09-11T00:19:45.818 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:45.816+0000 7fe6e8a82740 1 -- 172.21.15.12:0/319874553 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5572803ce2a0 msgr2=0x5572803dc140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:19:45.818 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:45.816+0000 7fe6e8a82740 1 --2- 172.21.15.12:0/319874553 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5572803ce2a0 0x5572803dc140 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fe6c80ac480 tx=0x7fe6c800efb0 comp rx=0 tx=0).stop 2024-09-11T00:19:45.818 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:45.816+0000 7fe6e8a82740 1 -- 172.21.15.12:0/319874553 shutdown_connections 2024-09-11T00:19:45.818 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:45.816+0000 7fe6e8a82740 1 --2- 172.21.15.12:0/319874553 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55728044e2f0 0x557280450710 crc :-1 s=READY pgs=194 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:19:45.818 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:45.816+0000 7fe6e8a82740 1 --2- 172.21.15.12:0/319874553 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5572803d0ce0 0x5572803dc680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:19:45.818 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:45.816+0000 7fe6e8a82740 1 --2- 172.21.15.12:0/319874553 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5572803ce2a0 0x5572803dc140 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:19:45.818 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:45.816+0000 7fe6e8a82740 1 --2- 172.21.15.12:0/319874553 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5572803cb970 0x5572803dbc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:19:45.819 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:45.816+0000 7fe6ccc00640 0 client.26969 ms_handle_reset on v2:172.21.15.165:6838/501312984 2024-09-11T00:19:45.819 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:45.816+0000 7fe6e8a82740 1 -- 172.21.15.12:0/319874553 >> 172.21.15.12:0/319874553 conn(0x5572803c6ac0 msgr2=0x5572803c7320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:19:45.819 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:45.817+0000 7fe6e8a82740 1 -- 172.21.15.12:0/319874553 shutdown_connections 2024-09-11T00:19:45.819 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:45.817+0000 7fe6e8a82740 1 -- 172.21.15.12:0/319874553 wait complete. 2024-09-11T00:19:45.819 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:45.817+0000 7fe6e8a82740 20 client.26969 trim_cache size 0 max 16384 2024-09-11T00:19:45.819 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:{ 2024-09-11T00:19:45.819 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "message": "", 2024-09-11T00:19:45.819 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "return_code": 0 2024-09-11T00:19:45.819 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:} 2024-09-11T00:19:45.820 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:45.817+0000 7fe6dfe00640 1 -- 172.21.15.12:0/2797648032 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fe6c40b5200 msgr2=0x7fe6c40b76c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:19:45.820 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:45.817+0000 7fe6dfe00640 1 --2- 172.21.15.12:0/2797648032 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fe6c40b5200 0x7fe6c40b76c0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fe6d0006010 tx=0x7fe6d0000bc0 comp rx=0 tx=0).stop 2024-09-11T00:19:45.820 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:45.817+0000 7fe6dfe00640 1 -- 172.21.15.12:0/2797648032 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe6e0076f30 msgr2=0x7fe6e019b450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:19:45.820 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:45.817+0000 7fe6dfe00640 1 --2- 172.21.15.12:0/2797648032 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe6e0076f30 0x7fe6e019b450 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fe6c800e9c0 tx=0x7fe6c800ee80 comp rx=0 tx=0).stop 2024-09-11T00:19:45.820 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:45.818+0000 7fe6dfe00640 1 -- 172.21.15.12:0/2797648032 shutdown_connections 2024-09-11T00:19:45.820 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:45.818+0000 7fe6dfe00640 1 --2- 172.21.15.12:0/2797648032 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fe6c40b5200 0x7fe6c40b76c0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:19:45.820 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:45.818+0000 7fe6dfe00640 1 --2- 172.21.15.12:0/2797648032 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe6e010a710 0x7fe6e019c010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:19:45.820 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:45.818+0000 7fe6dfe00640 1 --2- 172.21.15.12:0/2797648032 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe6e0076f30 0x7fe6e019b450 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:19:45.820 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:45.818+0000 7fe6dfe00640 1 --2- 172.21.15.12:0/2797648032 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe6e0075ac0 0x7fe6e019ae00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:19:45.821 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:45.818+0000 7fe6dfe00640 1 -- 172.21.15.12:0/2797648032 >> 172.21.15.12:0/2797648032 conn(0x7fe6e00fe030 msgr2=0x7fe6e00fe680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:19:45.821 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:45.818+0000 7fe6dfe00640 1 -- 172.21.15.12:0/2797648032 shutdown_connections 2024-09-11T00:19:45.821 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:45.818+0000 7fe6dfe00640 1 -- 172.21.15.12:0/2797648032 wait complete. 2024-09-11T00:19:46.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:19:46 smithi012 ceph-mon[30333]: pgmap v779: 129 pgs: 129 active+clean; 119 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 117 MiB/s rd, 784 KiB/s wr, 238 op/s 2024-09-11T00:19:46.330 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:19:46 smithi178 ceph-mon[35919]: pgmap v779: 129 pgs: 129 active+clean; 119 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 117 MiB/s rd, 784 KiB/s wr, 238 op/s 2024-09-11T00:19:46.532 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:19:46 smithi165 ceph-mon[35356]: pgmap v779: 129 pgs: 129 active+clean; 119 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 117 MiB/s rd, 784 KiB/s wr, 238 op/s 2024-09-11T00:19:46.699 INFO:tasks.workunit.client.0.smithi012.stdout: 1 514022 3.88 MB/sec execute 583 sec latency 283.907 ms 2024-09-11T00:19:47.699 INFO:tasks.workunit.client.0.smithi012.stdout: 1 514956 3.89 MB/sec execute 584 sec latency 455.670 ms 2024-09-11T00:19:48.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:19:48 smithi178 ceph-mon[35919]: pgmap v780: 129 pgs: 129 active+clean; 121 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 162 MiB/s rd, 1.2 MiB/s wr, 360 op/s 2024-09-11T00:19:48.532 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:19:48 smithi165 ceph-mon[35356]: pgmap v780: 129 pgs: 129 active+clean; 121 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 162 MiB/s rd, 1.2 MiB/s wr, 360 op/s 2024-09-11T00:19:48.562 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:19:48 smithi012 ceph-mon[30333]: pgmap v780: 129 pgs: 129 active+clean; 121 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 162 MiB/s rd, 1.2 MiB/s wr, 360 op/s 2024-09-11T00:19:48.699 INFO:tasks.workunit.client.0.smithi012.stdout: 1 515348 3.89 MB/sec execute 585 sec latency 610.455 ms 2024-09-11T00:19:49.699 INFO:tasks.workunit.client.0.smithi012.stdout: 1 516450 3.89 MB/sec execute 586 sec latency 67.545 ms 2024-09-11T00:19:50.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:19:50 smithi178 ceph-mon[35919]: pgmap v781: 129 pgs: 129 active+clean; 121 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 96 MiB/s rd, 970 KiB/s wr, 243 op/s 2024-09-11T00:19:50.532 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:19:50 smithi165 ceph-mon[35356]: pgmap v781: 129 pgs: 129 active+clean; 121 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 96 MiB/s rd, 970 KiB/s wr, 243 op/s 2024-09-11T00:19:50.563 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:19:50 smithi012 ceph-mon[30333]: pgmap v781: 129 pgs: 129 active+clean; 121 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 96 MiB/s rd, 970 KiB/s wr, 243 op/s 2024-09-11T00:19:50.699 INFO:tasks.workunit.client.0.smithi012.stdout: 1 517781 3.89 MB/sec execute 587 sec latency 217.338 ms 2024-09-11T00:19:51.699 INFO:tasks.workunit.client.0.smithi012.stdout: 1 517841 3.89 MB/sec execute 588 sec latency 106.204 ms 2024-09-11T00:19:52.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:19:52 smithi012 ceph-mon[30333]: pgmap v782: 129 pgs: 129 active+clean; 119 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 134 MiB/s rd, 1.4 MiB/s wr, 329 op/s 2024-09-11T00:19:52.532 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:19:52 smithi165 ceph-mon[35356]: pgmap v782: 129 pgs: 129 active+clean; 119 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 134 MiB/s rd, 1.4 MiB/s wr, 329 op/s 2024-09-11T00:19:52.581 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:19:52 smithi178 ceph-mon[35919]: pgmap v782: 129 pgs: 129 active+clean; 119 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 134 MiB/s rd, 1.4 MiB/s wr, 329 op/s 2024-09-11T00:19:52.699 INFO:tasks.workunit.client.0.smithi012.stdout: 1 518628 3.90 MB/sec execute 589 sec latency 242.636 ms 2024-09-11T00:19:53.700 INFO:tasks.workunit.client.0.smithi012.stdout: 1 519616 3.89 MB/sec execute 590 sec latency 19.590 ms 2024-09-11T00:19:54.532 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:19:54 smithi165 ceph-mon[35356]: pgmap v783: 129 pgs: 129 active+clean; 121 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 84 MiB/s rd, 1.3 MiB/s wr, 285 op/s 2024-09-11T00:19:54.563 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:19:54 smithi012 ceph-mon[30333]: pgmap v783: 129 pgs: 129 active+clean; 121 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 84 MiB/s rd, 1.3 MiB/s wr, 285 op/s 2024-09-11T00:19:54.581 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:19:54 smithi178 ceph-mon[35919]: pgmap v783: 129 pgs: 129 active+clean; 121 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 84 MiB/s rd, 1.3 MiB/s wr, 285 op/s 2024-09-11T00:19:54.700 INFO:tasks.workunit.client.0.smithi012.stdout: 1 521269 3.90 MB/sec execute 591 sec latency 141.846 ms 2024-09-11T00:19:55.700 INFO:tasks.workunit.client.0.smithi012.stdout: 1 522101 3.90 MB/sec execute 592 sec latency 393.925 ms 2024-09-11T00:19:56.532 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:19:56 smithi165 ceph-mon[35356]: pgmap v784: 129 pgs: 129 active+clean; 121 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 83 MiB/s rd, 1.2 MiB/s wr, 258 op/s 2024-09-11T00:19:56.563 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:19:56 smithi012 ceph-mon[30333]: pgmap v784: 129 pgs: 129 active+clean; 121 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 83 MiB/s rd, 1.2 MiB/s wr, 258 op/s 2024-09-11T00:19:56.581 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:19:56 smithi178 ceph-mon[35919]: pgmap v784: 129 pgs: 129 active+clean; 121 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 83 MiB/s rd, 1.2 MiB/s wr, 258 op/s 2024-09-11T00:19:56.700 INFO:tasks.workunit.client.0.smithi012.stdout: 1 522310 3.90 MB/sec execute 593 sec latency 324.570 ms 2024-09-11T00:19:57.700 INFO:tasks.workunit.client.0.smithi012.stdout: 1 523203 3.90 MB/sec execute 594 sec latency 16.581 ms 2024-09-11T00:19:58.532 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:19:58 smithi165 ceph-mon[35356]: pgmap v785: 129 pgs: 129 active+clean; 122 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 109 MiB/s rd, 2.0 MiB/s wr, 370 op/s 2024-09-11T00:19:58.563 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:19:58 smithi012 ceph-mon[30333]: pgmap v785: 129 pgs: 129 active+clean; 122 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 109 MiB/s rd, 2.0 MiB/s wr, 370 op/s 2024-09-11T00:19:58.580 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:19:58 smithi178 ceph-mon[35919]: pgmap v785: 129 pgs: 129 active+clean; 122 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 109 MiB/s rd, 2.0 MiB/s wr, 370 op/s 2024-09-11T00:19:58.700 INFO:tasks.workunit.client.0.smithi012.stdout: 1 524822 3.90 MB/sec execute 595 sec latency 166.600 ms 2024-09-11T00:19:58.939 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:58.937+0000 7f3039400640 1 -- 172.21.15.12:0/3575779776 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f30341036e0 msgr2=0x7f3034105b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:19:58.940 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:58.937+0000 7f3039400640 1 --2- 172.21.15.12:0/3575779776 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f30341036e0 0x7f3034105b50 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f3020009900 tx=0x7f302002a410 comp rx=0 tx=0).stop 2024-09-11T00:19:58.940 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:58.937+0000 7f3039400640 1 -- 172.21.15.12:0/3575779776 shutdown_connections 2024-09-11T00:19:58.940 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:58.937+0000 7f3039400640 1 --2- 172.21.15.12:0/3575779776 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3034106290 0x7f3034106740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:19:58.940 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:58.937+0000 7f3039400640 1 --2- 172.21.15.12:0/3575779776 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f30341036e0 0x7f3034105b50 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:19:58.940 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:58.937+0000 7f3039400640 1 --2- 172.21.15.12:0/3575779776 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f303406b470 0x7f3034103150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:19:58.940 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:58.937+0000 7f3039400640 1 -- 172.21.15.12:0/3575779776 >> 172.21.15.12:0/3575779776 conn(0x7f30340fcf30 msgr2=0x7f30340ff370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:19:58.940 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:58.937+0000 7f3039400640 1 -- 172.21.15.12:0/3575779776 shutdown_connections 2024-09-11T00:19:58.940 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:58.937+0000 7f3039400640 1 -- 172.21.15.12:0/3575779776 wait complete. 2024-09-11T00:19:58.942 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:58.938+0000 7f3039400640 1 Processor -- start 2024-09-11T00:19:58.942 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:58.938+0000 7f3039400640 1 -- start start 2024-09-11T00:19:58.942 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:58.938+0000 7f3039400640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f303406b470 0x7f303419f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:19:58.942 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:58.938+0000 7f3039400640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f30341036e0 0x7f303419f800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:19:58.942 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:58.938+0000 7f3039400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3034106290 0x7f30341a0430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:19:58.943 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:58.938+0000 7f3039400640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f30341149f0 con 0x7f303406b470 2024-09-11T00:19:58.943 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:58.938+0000 7f3039400640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f3034114870 con 0x7f3034106290 2024-09-11T00:19:58.943 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:58.938+0000 7f3039400640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f3034114b70 con 0x7f30341036e0 2024-09-11T00:19:58.943 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:58.939+0000 7f3033e00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f303406b470 0x7f303419f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:19:58.943 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:58.939+0000 7f3038a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3034106290 0x7f30341a0430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:19:58.943 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:58.939+0000 7f3038a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3034106290 0x7f30341a0430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.12:56398/0 (socket says 172.21.15.12:56398) 2024-09-11T00:19:58.943 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:58.939+0000 7f3033e00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f303406b470 0x7f303419f1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:60934/0 (socket says 172.21.15.12:60934) 2024-09-11T00:19:58.943 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:58.939+0000 7f3038a00640 1 -- 172.21.15.12:0/878071510 learned_addr learned my addr 172.21.15.12:0/878071510 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:19:58.943 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:58.939+0000 7f3033400640 1 --2- 172.21.15.12:0/878071510 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f30341036e0 0x7f303419f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:19:58.943 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:58.939+0000 7f3033e00640 1 -- 172.21.15.12:0/878071510 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f30341036e0 msgr2=0x7f303419f800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:19:58.943 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:58.939+0000 7f3033e00640 1 --2- 172.21.15.12:0/878071510 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f30341036e0 0x7f303419f800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:19:58.943 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:58.939+0000 7f3033e00640 1 -- 172.21.15.12:0/878071510 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3034106290 msgr2=0x7f30341a0430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:19:58.944 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:58.939+0000 7f3033e00640 1 --2- 172.21.15.12:0/878071510 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3034106290 0x7f30341a0430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:19:58.944 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:58.939+0000 7f3033e00640 1 -- 172.21.15.12:0/878071510 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30341a0cf0 con 0x7f303406b470 2024-09-11T00:19:58.944 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:58.939+0000 7f3038a00640 1 --2- 172.21.15.12:0/878071510 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3034106290 0x7f30341a0430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:19:58.944 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:58.939+0000 7f3033400640 1 --2- 172.21.15.12:0/878071510 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f30341036e0 0x7f303419f800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T00:19:58.944 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:58.940+0000 7f3033e00640 1 --2- 172.21.15.12:0/878071510 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f303406b470 0x7f303419f1b0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f302400e9f0 tx=0x7f302400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:19:58.944 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:58.940+0000 7f3030c00640 1 -- 172.21.15.12:0/878071510 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f302400ccc0 con 0x7f303406b470 2024-09-11T00:19:58.944 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:58.940+0000 7f3039400640 1 -- 172.21.15.12:0/878071510 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f30341a0fe0 con 0x7f303406b470 2024-09-11T00:19:58.944 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:58.940+0000 7f3030c00640 1 -- 172.21.15.12:0/878071510 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3024005020 con 0x7f303406b470 2024-09-11T00:19:58.944 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:58.940+0000 7f3030c00640 1 -- 172.21.15.12:0/878071510 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3024004a00 con 0x7f303406b470 2024-09-11T00:19:58.944 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:58.940+0000 7f3039400640 1 -- 172.21.15.12:0/878071510 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f30341a76b0 con 0x7f303406b470 2024-09-11T00:19:58.945 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:58.943+0000 7f303d812740 1 Processor -- start 2024-09-11T00:19:58.945 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:58.943+0000 7f303d812740 1 -- start start 2024-09-11T00:19:58.946 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:58.943+0000 7f303d812740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5612b00c4970 0x5612b00c6d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:19:58.946 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:58.943+0000 7f303d812740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5612b00c72a0 0x5612b00c9690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:19:58.946 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:58.944+0000 7f303d812740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5612b00c9ce0 0x5612b00cc1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:19:58.947 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:58.944+0000 7f303d812740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x5612b00376b0 con 0x5612b00c72a0 2024-09-11T00:19:58.947 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:58.944+0000 7f303d812740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x5612b0026ae0 con 0x5612b00c9ce0 2024-09-11T00:19:58.947 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:58.944+0000 7f303d812740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5612b002a8f0 con 0x5612b00c4970 2024-09-11T00:19:58.948 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:58.944+0000 7f3033400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5612b00c9ce0 0x5612b00cc1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:19:58.948 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:58.944+0000 7f3033400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5612b00c9ce0 0x5612b00cc1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.12:56414/0 (socket says 172.21.15.12:56414) 2024-09-11T00:19:58.948 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:58.944+0000 7f3038a00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5612b00c4970 0x5612b00c6d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:19:58.948 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:58.944+0000 7f3033400640 1 -- 172.21.15.12:0/2041079980 learned_addr learned my addr 172.21.15.12:0/2041079980 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:19:58.949 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:58.944+0000 7f3038a00640 1 -- 172.21.15.12:0/2041079980 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5612b00c9ce0 msgr2=0x5612b00cc1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:19:58.949 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:58.944+0000 7f3038a00640 1 --2- 172.21.15.12:0/2041079980 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5612b00c9ce0 0x5612b00cc1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:19:58.949 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:58.944+0000 7f3038a00640 1 -- 172.21.15.12:0/2041079980 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5612b00c72a0 msgr2=0x5612b00c9690 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-11T00:19:58.949 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:58.944+0000 7f3038a00640 1 --2- 172.21.15.12:0/2041079980 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5612b00c72a0 0x5612b00c9690 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:19:58.949 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:58.944+0000 7f3038a00640 1 -- 172.21.15.12:0/2041079980 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5612afff3780 con 0x5612b00c4970 2024-09-11T00:19:58.952 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:58.945+0000 7f3038a00640 1 --2- 172.21.15.12:0/2041079980 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5612b00c4970 0x5612b00c6d60 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f302c005c40 tx=0x7f302c010c60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=32c158471c1090e2 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:19:58.952 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:58.945+0000 7f3029200640 1 -- 172.21.15.12:0/2041079980 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f302c004d20 con 0x5612b00c4970 2024-09-11T00:19:58.952 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:58.945+0000 7f3029200640 1 -- 172.21.15.12:0/2041079980 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f302c015070 con 0x5612b00c4970 2024-09-11T00:19:58.952 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:58.945+0000 7f3029200640 1 -- 172.21.15.12:0/2041079980 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f302c0056a0 con 0x5612b00c4970 2024-09-11T00:19:58.953 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:58.945+0000 7f3030c00640 1 -- 172.21.15.12:0/878071510 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7f3024004ba0 con 0x7f303406b470 2024-09-11T00:19:58.953 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:58.946+0000 7f303d812740 1 -- 172.21.15.12:0/2041079980 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5612b00c4970 msgr2=0x5612b00c6d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:19:58.953 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:58.946+0000 7f303d812740 1 --2- 172.21.15.12:0/2041079980 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5612b00c4970 0x5612b00c6d60 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f302c005c40 tx=0x7f302c010c60 comp rx=0 tx=0).stop 2024-09-11T00:19:58.953 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:58.946+0000 7f303d812740 1 -- 172.21.15.12:0/2041079980 shutdown_connections 2024-09-11T00:19:58.953 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:58.946+0000 7f303d812740 1 --2- 172.21.15.12:0/2041079980 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5612b00c9ce0 0x5612b00cc1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:19:58.953 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:58.946+0000 7f303d812740 1 --2- 172.21.15.12:0/2041079980 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5612b00c72a0 0x5612b00c9690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:19:58.953 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:58.946+0000 7f303d812740 1 --2- 172.21.15.12:0/2041079980 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5612b00c4970 0x5612b00c6d60 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:19:58.953 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:58.946+0000 7f303d812740 1 -- 172.21.15.12:0/2041079980 >> 172.21.15.12:0/2041079980 conn(0x5612b00bfac0 msgr2=0x5612b00c0850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:19:58.953 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:58.946+0000 7f303d812740 1 -- 172.21.15.12:0/2041079980 shutdown_connections 2024-09-11T00:19:58.954 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:58.946+0000 7f303d812740 1 -- 172.21.15.12:0/2041079980 wait complete. 2024-09-11T00:19:58.954 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:58.946+0000 7f303d812740 1 -- start start 2024-09-11T00:19:58.954 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:58.946+0000 7f303d812740 1 Processor -- start 2024-09-11T00:19:58.954 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:58.947+0000 7f303d812740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5612b00c4970 0x5612b00ce130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:19:58.954 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:58.947+0000 7f303d812740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5612b00c72a0 0x5612b00ce7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:19:58.954 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:58.947+0000 7f303d812740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5612b00c9ce0 0x5612b00cf3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:19:58.954 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:58.947+0000 7f303d812740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x5612aff5eab0 con 0x5612b00c72a0 2024-09-11T00:19:58.954 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:58.947+0000 7f303d812740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x5612affcd570 con 0x5612b00c9ce0 2024-09-11T00:19:58.954 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:58.947+0000 7f303d812740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5612b00cccc0 con 0x5612b00c4970 2024-09-11T00:19:58.954 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:58.947+0000 7f303d812740 10 client.0 _ll_register_callbacks cb 0x5612affdc100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:19:58.954 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:58.947+0000 7f3038a00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5612b00c4970 0x5612b00ce130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:19:58.954 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:58.947+0000 7f3033e00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5612b00c72a0 0x5612b00ce7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:19:58.954 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:58.947+0000 7f3038a00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5612b00c4970 0x5612b00ce130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.12:42314/0 (socket says 172.21.15.12:42314) 2024-09-11T00:19:58.955 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:58.947+0000 7f3038a00640 1 -- 172.21.15.12:0/3011891635 learned_addr learned my addr 172.21.15.12:0/3011891635 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:19:58.955 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:58.947+0000 7f3030c00640 1 --2- 172.21.15.12:0/878071510 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f30180b51f0 0x7f30180b76b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:19:58.955 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:58.947+0000 7f3033400640 1 --2- 172.21.15.12:0/3011891635 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5612b00c9ce0 0x5612b00cf3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:19:58.955 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:58.948+0000 7f3030c00640 1 -- 172.21.15.12:0/878071510 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9208+0+0 (secure 0 0 0) 0x7f30240e6000 con 0x7f303406b470 2024-09-11T00:19:58.955 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:58.948+0000 7f3038a00640 1 --2- 172.21.15.12:0/878071510 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f30180b51f0 0x7f30180b76b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:19:58.955 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:58.948+0000 7f3038a00640 1 -- 172.21.15.12:0/3011891635 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5612b00c9ce0 msgr2=0x5612b00cf3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:19:58.955 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:58.949+0000 7f3038a00640 1 --2- 172.21.15.12:0/3011891635 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5612b00c9ce0 0x5612b00cf3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:19:58.955 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:58.949+0000 7f3038a00640 1 -- 172.21.15.12:0/3011891635 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5612b00c72a0 msgr2=0x5612b00ce7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:19:58.955 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:58.949+0000 7f3038a00640 1 --2- 172.21.15.12:0/3011891635 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5612b00c72a0 0x5612b00ce7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:19:58.955 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:58.949+0000 7f3038a00640 1 -- 172.21.15.12:0/3011891635 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5612afd88790 con 0x5612b00c4970 2024-09-11T00:19:58.955 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:58.949+0000 7f3033e00640 1 --2- 172.21.15.12:0/3011891635 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5612b00c72a0 0x5612b00ce7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:19:58.955 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:58.949+0000 7f3033400640 1 --2- 172.21.15.12:0/3011891635 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5612b00c9ce0 0x5612b00cf3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:19:58.955 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:58.949+0000 7f3038a00640 1 --2- 172.21.15.12:0/3011891635 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5612b00c4970 0x5612b00ce130 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f302c009a20 tx=0x7f302c005750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=89074b5c57ae34b1 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:19:58.956 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:58.949+0000 7f301d600640 10 client.0 ms_handle_connect on v2:172.21.15.178:3300/0 2024-09-11T00:19:58.956 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:58.950+0000 7f3038a00640 1 --2- 172.21.15.12:0/878071510 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f30180b51f0 0x7f30180b76b0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f302c028000 tx=0x7f302c029000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:19:58.956 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:58.950+0000 7f301d600640 1 -- 172.21.15.12:0/3011891635 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f302c022040 con 0x5612b00c4970 2024-09-11T00:19:58.956 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:58.950+0000 7f303d812740 1 -- 172.21.15.12:0/3011891635 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f302c015070 con 0x5612b00c4970 2024-09-11T00:19:58.956 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:58.950+0000 7f301d600640 1 -- 172.21.15.12:0/3011891635 <== mon.1 v2:172.21.15.178:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9208+0+0 (secure 0 0 0) 0x7f302c029850 con 0x5612b00c4970 2024-09-11T00:19:58.958 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:58.951+0000 7f301d600640 1 -- 172.21.15.12:0/3011891635 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f302c03a050 con 0x5612b00c4970 2024-09-11T00:19:58.958 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:58.951+0000 7f303d812740 10 client.26959 fetch_fsmap learned FSMap version 44 2024-09-11T00:19:58.958 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:58.951+0000 7f303d812740 1 -- 172.21.15.12:0/3011891635 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x5612afe01620 con 0x5612b00c4970 2024-09-11T00:19:58.958 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:58.951+0000 7f301d600640 1 -- 172.21.15.12:0/3011891635 <== mon.1 v2:172.21.15.178:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f302c015070 con 0x5612b00c4970 2024-09-11T00:19:58.958 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:58.951+0000 7f301d600640 1 -- 172.21.15.12:0/3011891635 <== mon.1 v2:172.21.15.178:3300/0 5 ==== fsmap(e 44) ==== 8382+0+0 (secure 0 0 0) 0x7f302c011820 con 0x5612b00c4970 2024-09-11T00:19:58.958 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:58.951+0000 7f303d812740 10 client.26959 fetch_fsmap finished waiting for FSMap version 44 2024-09-11T00:19:58.958 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:58.951+0000 7f303d812740 10 client.26959 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T00:19:58.958 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:58.951+0000 7f303d812740 20 client.26959 populate_metadata read hostname 'smithi012' 2024-09-11T00:19:58.958 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:58.951+0000 7f303d812740 1 --2- 172.21.15.12:0/3011891635 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x5612b00d97e0 0x5612b00dbbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:19:58.958 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:58.951+0000 7f303d812740 4 client.26959 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T00:19:58.958 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:58.951+0000 7f303d812740 1 -- 172.21.15.12:0/3011891635 --> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5612b0039600 con 0x5612b00d97e0 2024-09-11T00:19:58.958 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:58.952+0000 7f3033e00640 1 --2- 172.21.15.12:0/3011891635 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x5612b00d97e0 0x5612b00dbbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:19:58.959 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:58.953+0000 7f3033e00640 1 --2- 172.21.15.12:0/3011891635 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x5612b00d97e0 0x5612b00dbbd0 crc :-1 s=READY pgs=195 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6e548278a33319bf server_cookie=2b4edec9e0ce741d in_seq=0 out_seq=0 2024-09-11T00:19:58.959 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:58.953+0000 7f301d600640 10 client.26959 ms_handle_connect on v2:172.21.15.165:6838/501312984 2024-09-11T00:19:58.959 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:58.953+0000 7f301d600640 1 -- 172.21.15.12:0/3011891635 <== mds.0 v2:172.21.15.165:6838/501312984 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5612b0039600 con 0x5612b00d97e0 2024-09-11T00:19:58.959 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:58.953+0000 7f301d600640 10 client.26959 handle_command_reply: tid=0 2024-09-11T00:19:58.959 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:58.953+0000 7f303d812740 1 client.26959 shutdown 2024-09-11T00:19:58.959 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:58.954+0000 7f303d812740 1 -- 172.21.15.12:0/3011891635 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5612b00c4970 msgr2=0x5612b00ce130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:19:58.959 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:58.954+0000 7f303d812740 1 --2- 172.21.15.12:0/3011891635 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5612b00c4970 0x5612b00ce130 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f302c009a20 tx=0x7f302c005750 comp rx=0 tx=0).stop 2024-09-11T00:19:58.959 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:58.954+0000 7f303d812740 1 -- 172.21.15.12:0/3011891635 shutdown_connections 2024-09-11T00:19:58.959 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:58.954+0000 7f303d812740 1 --2- 172.21.15.12:0/3011891635 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x5612b00d97e0 0x5612b00dbbd0 crc :-1 s=READY pgs=195 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:19:58.959 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:58.954+0000 7f303d812740 1 --2- 172.21.15.12:0/3011891635 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5612b00c9ce0 0x5612b00cf3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:19:58.959 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:58.954+0000 7f303d812740 1 --2- 172.21.15.12:0/3011891635 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5612b00c72a0 0x5612b00ce7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:19:58.959 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:58.954+0000 7f301d600640 0 client.26959 ms_handle_reset on v2:172.21.15.165:6838/501312984 2024-09-11T00:19:58.959 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:58.954+0000 7f303d812740 1 --2- 172.21.15.12:0/3011891635 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5612b00c4970 0x5612b00ce130 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:19:58.960 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:58.954+0000 7f303d812740 1 -- 172.21.15.12:0/3011891635 >> 172.21.15.12:0/3011891635 conn(0x5612b00bfac0 msgr2=0x5612b00bebc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:19:58.960 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:58.954+0000 7f303d812740 1 -- 172.21.15.12:0/3011891635 shutdown_connections 2024-09-11T00:19:58.960 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:58.954+0000 7f303d812740 1 -- 172.21.15.12:0/3011891635 wait complete. 2024-09-11T00:19:58.960 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:58.954+0000 7f303d812740 20 client.26959 trim_cache size 0 max 16384 2024-09-11T00:19:58.972 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:58.970+0000 7f303d812740 1 Processor -- start 2024-09-11T00:19:58.972 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:58.970+0000 7f303d812740 1 -- start start 2024-09-11T00:19:58.973 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:58.970+0000 7f303d812740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5612b00c4970 0x5612b00babc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:19:58.973 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:58.970+0000 7f303d812740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5612b00c72a0 0x5612b00bf3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:19:58.973 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:58.970+0000 7f303d812740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5612b00c9ce0 0x5612b00d5ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:19:58.973 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:58.970+0000 7f303d812740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f30341b1bc0 con 0x5612b00c9ce0 2024-09-11T00:19:58.974 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:58.970+0000 7f303d812740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f30341b3790 con 0x5612b00c72a0 2024-09-11T00:19:58.974 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:58.970+0000 7f303d812740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f30341b22c0 con 0x5612b00c4970 2024-09-11T00:19:58.974 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:58.971+0000 7f3038a00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5612b00c9ce0 0x5612b00d5ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:19:58.974 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:58.971+0000 7f3038a00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5612b00c9ce0 0x5612b00d5ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:60942/0 (socket says 172.21.15.12:60942) 2024-09-11T00:19:58.974 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:58.971+0000 7f3038a00640 1 -- 172.21.15.12:0/2605835153 learned_addr learned my addr 172.21.15.12:0/2605835153 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:19:58.974 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:58.971+0000 7f3033400640 1 --2- 172.21.15.12:0/2605835153 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5612b00c72a0 0x5612b00bf3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:19:58.974 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:58.971+0000 7f3033e00640 1 --2- 172.21.15.12:0/2605835153 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5612b00c4970 0x5612b00babc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:19:58.974 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:58.971+0000 7f3038a00640 1 -- 172.21.15.12:0/2605835153 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5612b00c4970 msgr2=0x5612b00babc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:19:58.974 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:58.971+0000 7f3038a00640 1 --2- 172.21.15.12:0/2605835153 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5612b00c4970 0x5612b00babc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:19:58.974 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:58.971+0000 7f3038a00640 1 -- 172.21.15.12:0/2605835153 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5612b00c72a0 msgr2=0x5612b00bf3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:19:58.974 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:58.971+0000 7f3038a00640 1 --2- 172.21.15.12:0/2605835153 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5612b00c72a0 0x5612b00bf3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:19:58.974 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:58.971+0000 7f3038a00640 1 -- 172.21.15.12:0/2605835153 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5612b00cdc30 con 0x5612b00c9ce0 2024-09-11T00:19:58.974 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:58.971+0000 7f3038a00640 1 --2- 172.21.15.12:0/2605835153 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5612b00c9ce0 0x5612b00d5ca0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f302c010ca0 tx=0x7f302c010fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=982024a26c0c260f server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:19:58.977 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:58.972+0000 7f301fe00640 1 -- 172.21.15.12:0/2605835153 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f302c044070 con 0x5612b00c9ce0 2024-09-11T00:19:58.977 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:58.972+0000 7f301fe00640 1 -- 172.21.15.12:0/2605835153 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f302c015070 con 0x5612b00c9ce0 2024-09-11T00:19:58.978 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:58.972+0000 7f301fe00640 1 -- 172.21.15.12:0/2605835153 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f302c029b40 con 0x5612b00c9ce0 2024-09-11T00:19:58.978 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:58.972+0000 7f303d812740 1 -- 172.21.15.12:0/2605835153 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5612b00c9ce0 msgr2=0x5612b00d5ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:19:58.978 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:58.972+0000 7f303d812740 1 --2- 172.21.15.12:0/2605835153 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5612b00c9ce0 0x5612b00d5ca0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f302c010ca0 tx=0x7f302c010fd0 comp rx=0 tx=0).stop 2024-09-11T00:19:58.978 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:58.973+0000 7f303d812740 1 -- 172.21.15.12:0/2605835153 shutdown_connections 2024-09-11T00:19:58.978 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:58.973+0000 7f303d812740 1 --2- 172.21.15.12:0/2605835153 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5612b00c9ce0 0x5612b00d5ca0 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:19:58.978 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:58.973+0000 7f303d812740 1 --2- 172.21.15.12:0/2605835153 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5612b00c72a0 0x5612b00bf3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:19:58.978 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:58.973+0000 7f303d812740 1 --2- 172.21.15.12:0/2605835153 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5612b00c4970 0x5612b00babc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:19:58.978 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:58.973+0000 7f303d812740 1 -- 172.21.15.12:0/2605835153 >> 172.21.15.12:0/2605835153 conn(0x5612b00bfac0 msgr2=0x5612b00c3e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:19:58.978 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:58.973+0000 7f303d812740 1 -- 172.21.15.12:0/2605835153 shutdown_connections 2024-09-11T00:19:58.978 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:58.973+0000 7f303d812740 1 -- 172.21.15.12:0/2605835153 wait complete. 2024-09-11T00:19:58.978 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:58.973+0000 7f303d812740 1 -- start start 2024-09-11T00:19:58.979 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:58.973+0000 7f303d812740 1 Processor -- start 2024-09-11T00:19:58.979 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:58.974+0000 7f303d812740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5612b00c4970 0x5612b00d5be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:19:58.979 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:58.974+0000 7f303d812740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5612b00c72a0 0x5612b00d48e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:19:58.979 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:58.974+0000 7f303d812740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5612b00c9ce0 0x5612b00d4e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:19:58.979 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:58.974+0000 7f303d812740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f30341b30c0 con 0x5612b00c9ce0 2024-09-11T00:19:58.979 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:58.974+0000 7f303d812740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f30341b2990 con 0x5612b00c4970 2024-09-11T00:19:58.979 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:58.974+0000 7f303d812740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f30341afba0 con 0x5612b00c72a0 2024-09-11T00:19:58.979 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:58.974+0000 7f303d812740 10 client.0 _ll_register_callbacks cb 0x7f3024005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:19:58.979 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:58.974+0000 7f3033e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5612b00c4970 0x5612b00d5be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:19:58.979 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:58.974+0000 7f3033e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5612b00c4970 0x5612b00d5be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.12:56442/0 (socket says 172.21.15.12:56442) 2024-09-11T00:19:58.979 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:58.974+0000 7f3033e00640 1 -- 172.21.15.12:0/172811137 learned_addr learned my addr 172.21.15.12:0/172811137 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:19:58.979 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:58.974+0000 7f3038a00640 1 --2- 172.21.15.12:0/172811137 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5612b00c9ce0 0x5612b00d4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:19:58.979 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:58.974+0000 7f3033400640 1 --2- 172.21.15.12:0/172811137 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5612b00c72a0 0x5612b00d48e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:19:58.979 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:58.974+0000 7f3033e00640 1 -- 172.21.15.12:0/172811137 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5612b00c72a0 msgr2=0x5612b00d48e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:19:58.980 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:58.974+0000 7f3033e00640 1 --2- 172.21.15.12:0/172811137 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5612b00c72a0 0x5612b00d48e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:19:58.980 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:58.974+0000 7f3033e00640 1 -- 172.21.15.12:0/172811137 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5612b00c9ce0 msgr2=0x5612b00d4e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:19:58.980 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:58.974+0000 7f3033e00640 1 --2- 172.21.15.12:0/172811137 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5612b00c9ce0 0x5612b00d4e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:19:58.980 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:58.974+0000 7f3033e00640 1 -- 172.21.15.12:0/172811137 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5612b00bf8f0 con 0x5612b00c4970 2024-09-11T00:19:58.980 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:58.975+0000 7f3033e00640 1 --2- 172.21.15.12:0/172811137 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5612b00c4970 0x5612b00d5be0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f30240a89c0 tx=0x7f3024009f60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1bc62007e9a9b874 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:19:58.980 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:58.975+0000 7f301d600640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-09-11T00:19:58.980 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:58.975+0000 7f301d600640 1 -- 172.21.15.12:0/172811137 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f302401c040 con 0x5612b00c4970 2024-09-11T00:19:58.980 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:58.975+0000 7f303d812740 1 -- 172.21.15.12:0/172811137 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f302c015070 con 0x5612b00c4970 2024-09-11T00:19:58.980 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:58.975+0000 7f301d600640 1 -- 172.21.15.12:0/172811137 <== mon.2 v2:172.21.15.165:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9208+0+0 (secure 0 0 0) 0x7f30240b54f0 con 0x5612b00c4970 2024-09-11T00:19:58.980 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:58.975+0000 7f301d600640 1 -- 172.21.15.12:0/172811137 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f30240abf00 con 0x5612b00c4970 2024-09-11T00:19:58.980 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:58.975+0000 7f303d812740 10 client.26987 fetch_fsmap learned FSMap version 44 2024-09-11T00:19:58.980 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:58.975+0000 7f303d812740 1 -- 172.21.15.12:0/172811137 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x5612b00cd0c0 con 0x5612b00c4970 2024-09-11T00:19:58.980 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:58.975+0000 7f301d600640 1 -- 172.21.15.12:0/172811137 <== mon.2 v2:172.21.15.165:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f302c015070 con 0x5612b00c4970 2024-09-11T00:19:58.981 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:58.975+0000 7f301d600640 1 -- 172.21.15.12:0/172811137 <== mon.2 v2:172.21.15.165:3300/0 5 ==== fsmap(e 44) ==== 8382+0+0 (secure 0 0 0) 0x7f3024093b80 con 0x5612b00c4970 2024-09-11T00:19:58.981 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:58.975+0000 7f303d812740 10 client.26987 fetch_fsmap finished waiting for FSMap version 44 2024-09-11T00:19:58.981 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:58.975+0000 7f303d812740 10 client.26987 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T00:19:58.981 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:58.975+0000 7f303d812740 20 client.26987 populate_metadata read hostname 'smithi012' 2024-09-11T00:19:58.981 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:58.976+0000 7f303d812740 1 --2- 172.21.15.12:0/172811137 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x5612b0146d00 0x5612b0149120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:19:58.981 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:58.976+0000 7f303d812740 4 client.26987 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T00:19:58.981 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:58.976+0000 7f303d812740 1 -- 172.21.15.12:0/172811137 --> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5612b00350b0 con 0x5612b0146d00 2024-09-11T00:19:58.981 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:58.976+0000 7f3033400640 1 --2- 172.21.15.12:0/172811137 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x5612b0146d00 0x5612b0149120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:19:58.982 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:58.976+0000 7f3033400640 1 --2- 172.21.15.12:0/172811137 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x5612b0146d00 0x5612b0149120 crc :-1 s=READY pgs=196 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=40dbd5b63de931bc server_cookie=77618a65ec05f62f in_seq=0 out_seq=0 2024-09-11T00:19:58.982 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:58.976+0000 7f301d600640 10 client.26987 ms_handle_connect on v2:172.21.15.165:6838/501312984 2024-09-11T00:19:59.042 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:59.039+0000 7f301d600640 1 -- 172.21.15.12:0/172811137 <== mds.0 v2:172.21.15.165:6838/501312984 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5612b00350b0 con 0x5612b0146d00 2024-09-11T00:19:59.042 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:59.039+0000 7f301d600640 10 client.26987 handle_command_reply: tid=0 2024-09-11T00:19:59.042 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:59.039+0000 7f303d812740 1 client.26987 shutdown 2024-09-11T00:19:59.042 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:59.040+0000 7f303d812740 1 -- 172.21.15.12:0/172811137 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5612b00c4970 msgr2=0x5612b00d5be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:19:59.043 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:59.040+0000 7f303d812740 1 --2- 172.21.15.12:0/172811137 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5612b00c4970 0x5612b00d5be0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f30240a89c0 tx=0x7f3024009f60 comp rx=0 tx=0).stop 2024-09-11T00:19:59.043 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:59.040+0000 7f303d812740 1 -- 172.21.15.12:0/172811137 shutdown_connections 2024-09-11T00:19:59.043 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:59.040+0000 7f303d812740 1 --2- 172.21.15.12:0/172811137 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x5612b0146d00 0x5612b0149120 crc :-1 s=READY pgs=196 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:19:59.043 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:59.040+0000 7f303d812740 1 --2- 172.21.15.12:0/172811137 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5612b00c9ce0 0x5612b00d4e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:19:59.043 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:59.040+0000 7f303d812740 1 --2- 172.21.15.12:0/172811137 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5612b00c72a0 0x5612b00d48e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:19:59.043 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:59.040+0000 7f303d812740 1 --2- 172.21.15.12:0/172811137 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5612b00c4970 0x5612b00d5be0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:19:59.043 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:59.040+0000 7f301d600640 0 client.26987 ms_handle_reset on v2:172.21.15.165:6838/501312984 2024-09-11T00:19:59.043 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:59.040+0000 7f303d812740 1 -- 172.21.15.12:0/172811137 >> 172.21.15.12:0/172811137 conn(0x5612b00bfac0 msgr2=0x5612b00c3a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:19:59.043 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:59.040+0000 7f303d812740 1 -- 172.21.15.12:0/172811137 shutdown_connections 2024-09-11T00:19:59.043 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:59.040+0000 7f303d812740 1 -- 172.21.15.12:0/172811137 wait complete. 2024-09-11T00:19:59.043 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:59.040+0000 7f303d812740 20 client.26987 trim_cache size 0 max 16384 2024-09-11T00:19:59.044 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:{ 2024-09-11T00:19:59.044 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "message": "", 2024-09-11T00:19:59.044 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "return_code": 0 2024-09-11T00:19:59.044 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:} 2024-09-11T00:19:59.045 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:59.041+0000 7f3039400640 1 -- 172.21.15.12:0/878071510 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f30180b51f0 msgr2=0x7f30180b76b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:19:59.045 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:59.041+0000 7f3039400640 1 --2- 172.21.15.12:0/878071510 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f30180b51f0 0x7f30180b76b0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f302c028000 tx=0x7f302c029000 comp rx=0 tx=0).stop 2024-09-11T00:19:59.045 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:59.041+0000 7f3039400640 1 -- 172.21.15.12:0/878071510 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f303406b470 msgr2=0x7f303419f1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:19:59.045 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:59.041+0000 7f3039400640 1 --2- 172.21.15.12:0/878071510 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f303406b470 0x7f303419f1b0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f302400e9f0 tx=0x7f302400eeb0 comp rx=0 tx=0).stop 2024-09-11T00:19:59.045 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:59.041+0000 7f3039400640 1 -- 172.21.15.12:0/878071510 shutdown_connections 2024-09-11T00:19:59.045 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:59.041+0000 7f3039400640 1 --2- 172.21.15.12:0/878071510 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f30180b51f0 0x7f30180b76b0 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:19:59.045 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:59.041+0000 7f3039400640 1 --2- 172.21.15.12:0/878071510 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3034106290 0x7f30341a0430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:19:59.045 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:59.041+0000 7f3039400640 1 --2- 172.21.15.12:0/878071510 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f30341036e0 0x7f303419f800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:19:59.045 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:59.041+0000 7f3039400640 1 --2- 172.21.15.12:0/878071510 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f303406b470 0x7f303419f1b0 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:19:59.045 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:59.041+0000 7f3039400640 1 -- 172.21.15.12:0/878071510 >> 172.21.15.12:0/878071510 conn(0x7f30340fcf30 msgr2=0x7f3034101a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:19:59.046 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:59.041+0000 7f3039400640 1 -- 172.21.15.12:0/878071510 shutdown_connections 2024-09-11T00:19:59.046 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:19:59.041+0000 7f3039400640 1 -- 172.21.15.12:0/878071510 wait complete. 2024-09-11T00:19:59.700 INFO:tasks.workunit.client.0.smithi012.stdout: 1 525720 3.91 MB/sec execute 596 sec latency 322.425 ms 2024-09-11T00:20:00.532 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:20:00 smithi165 ceph-mon[35356]: pgmap v786: 129 pgs: 129 active+clean; 121 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 64 MiB/s rd, 1.5 MiB/s wr, 246 op/s 2024-09-11T00:20:00.532 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:20:00 smithi165 ceph-mon[35356]: overall HEALTH_OK 2024-09-11T00:20:00.563 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:20:00 smithi012 ceph-mon[30333]: pgmap v786: 129 pgs: 129 active+clean; 121 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 64 MiB/s rd, 1.5 MiB/s wr, 246 op/s 2024-09-11T00:20:00.563 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:20:00 smithi012 ceph-mon[30333]: overall HEALTH_OK 2024-09-11T00:20:00.580 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:20:00 smithi178 ceph-mon[35919]: pgmap v786: 129 pgs: 129 active+clean; 121 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 64 MiB/s rd, 1.5 MiB/s wr, 246 op/s 2024-09-11T00:20:00.581 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:20:00 smithi178 ceph-mon[35919]: overall HEALTH_OK 2024-09-11T00:20:00.700 INFO:tasks.workunit.client.0.smithi012.stdout: 1 526384 3.91 MB/sec execute 597 sec latency 276.703 ms 2024-09-11T00:20:01.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:20:01 smithi012 ceph-mon[30333]: pgmap v787: 129 pgs: 129 active+clean; 122 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 93 MiB/s rd, 2.0 MiB/s wr, 358 op/s 2024-09-11T00:20:01.532 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:20:01 smithi165 ceph-mon[35356]: pgmap v787: 129 pgs: 129 active+clean; 122 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 93 MiB/s rd, 2.0 MiB/s wr, 358 op/s 2024-09-11T00:20:01.581 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:20:01 smithi178 ceph-mon[35919]: pgmap v787: 129 pgs: 129 active+clean; 122 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 93 MiB/s rd, 2.0 MiB/s wr, 358 op/s 2024-09-11T00:20:01.701 INFO:tasks.workunit.client.0.smithi012.stdout: 1 526709 3.90 MB/sec execute 598 sec latency 602.952 ms 2024-09-11T00:20:02.701 INFO:tasks.workunit.client.0.smithi012.stdout: 1 528193 3.91 MB/sec execute 599 sec latency 779.093 ms 2024-09-11T00:20:03.701 INFO:tasks.workunit.client.0.smithi012.stdout: 1 cleanup 600 sec 2024-09-11T00:20:03.747 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:20:03 smithi012 ceph-mon[30333]: pgmap v788: 129 pgs: 129 active+clean; 123 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 55 MiB/s rd, 1.8 MiB/s wr, 316 op/s 2024-09-11T00:20:03.782 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:20:03 smithi165 ceph-mon[35356]: pgmap v788: 129 pgs: 129 active+clean; 123 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 55 MiB/s rd, 1.8 MiB/s wr, 316 op/s 2024-09-11T00:20:03.831 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:20:03 smithi178 ceph-mon[35919]: pgmap v788: 129 pgs: 129 active+clean; 123 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 55 MiB/s rd, 1.8 MiB/s wr, 316 op/s 2024-09-11T00:20:03.971 INFO:tasks.workunit.client.0.smithi012.stdout: 0 cleanup 600 sec 2024-09-11T00:20:03.971 INFO:tasks.workunit.client.0.smithi012.stdout: 2024-09-11T00:20:03.972 INFO:tasks.workunit.client.0.smithi012.stdout: Operation Count AvgLat MaxLat 2024-09-11T00:20:03.972 INFO:tasks.workunit.client.0.smithi012.stdout: ---------------------------------------- 2024-09-11T00:20:03.972 INFO:tasks.workunit.client.0.smithi012.stdout: NTCreateX 74700 3.858 29939.073 2024-09-11T00:20:03.972 INFO:tasks.workunit.client.0.smithi012.stdout: Close 54904 0.005 0.273 2024-09-11T00:20:03.972 INFO:tasks.workunit.client.0.smithi012.stdout: Rename 3167 25.072 33888.678 2024-09-11T00:20:03.972 INFO:tasks.workunit.client.0.smithi012.stdout: Unlink 15053 1.506 136.444 2024-09-11T00:20:03.972 INFO:tasks.workunit.client.0.smithi012.stdout: Deltree 2 106.020 151.843 2024-09-11T00:20:03.972 INFO:tasks.workunit.client.0.smithi012.stdout: Mkdir 1 0.013 0.013 2024-09-11T00:20:03.972 INFO:tasks.workunit.client.0.smithi012.stdout: Qpathinfo 67747 1.446 6105.620 2024-09-11T00:20:03.972 INFO:tasks.workunit.client.0.smithi012.stdout: Qfileinfo 11894 0.006 1.594 2024-09-11T00:20:03.972 INFO:tasks.workunit.client.0.smithi012.stdout: Qfsinfo 12406 0.317 211.939 2024-09-11T00:20:03.972 INFO:tasks.workunit.client.0.smithi012.stdout: Sfileinfo 6097 4.627 27977.044 2024-09-11T00:20:03.973 INFO:tasks.workunit.client.0.smithi012.stdout: Find 26185 0.229 29.694 2024-09-11T00:20:03.973 INFO:tasks.workunit.client.0.smithi012.stdout: WriteX 37314 0.070 16.924 2024-09-11T00:20:03.973 INFO:tasks.workunit.client.0.smithi012.stdout: ReadX 117256 0.452 2281.918 2024-09-11T00:20:03.973 INFO:tasks.workunit.client.0.smithi012.stdout: LockX 244 0.569 2.227 2024-09-11T00:20:03.973 INFO:tasks.workunit.client.0.smithi012.stdout: UnlockX 244 0.548 1.222 2024-09-11T00:20:03.973 INFO:tasks.workunit.client.0.smithi012.stdout: Flush 5237 3.801 68.923 2024-09-11T00:20:03.973 INFO:tasks.workunit.client.0.smithi012.stdout: 2024-09-11T00:20:03.973 INFO:tasks.workunit.client.0.smithi012.stdout:Throughput 3.90888 MB/sec 1 clients 1 procs max_latency=33888.688 ms 2024-09-11T00:20:03.974 INFO:tasks.workunit.client.0.smithi012.stderr:+ dbench 10 2024-09-11T00:20:03.975 INFO:tasks.workunit.client.0.smithi012.stdout:dbench version 4.00 - Copyright Andrew Tridgell 1999-2004 2024-09-11T00:20:03.975 INFO:tasks.workunit.client.0.smithi012.stdout: 2024-09-11T00:20:03.975 INFO:tasks.workunit.client.0.smithi012.stdout:Running for 600 seconds with load '/usr/share/dbench/client.txt' and minimum warmup 120 secs 2024-09-11T00:20:03.976 INFO:tasks.workunit.client.0.smithi012.stdout:8 of 10 processes prepared for launch 0 sec 2024-09-11T00:20:04.076 INFO:tasks.workunit.client.0.smithi012.stdout:10 of 10 processes prepared for launch 0 sec 2024-09-11T00:20:04.076 INFO:tasks.workunit.client.0.smithi012.stdout:releasing clients 2024-09-11T00:20:05.076 INFO:tasks.workunit.client.0.smithi012.stdout: 10 443 155.90 MB/sec warmup 1 sec latency 50.908 ms 2024-09-11T00:20:05.782 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:20:05 smithi165 ceph-mon[35356]: pgmap v789: 129 pgs: 129 active+clean; 123 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 54 MiB/s rd, 1.6 MiB/s wr, 280 op/s 2024-09-11T00:20:05.813 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:20:05 smithi012 ceph-mon[30333]: pgmap v789: 129 pgs: 129 active+clean; 123 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 54 MiB/s rd, 1.6 MiB/s wr, 280 op/s 2024-09-11T00:20:05.830 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:20:05 smithi178 ceph-mon[35919]: pgmap v789: 129 pgs: 129 active+clean; 123 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 54 MiB/s rd, 1.6 MiB/s wr, 280 op/s 2024-09-11T00:20:06.077 INFO:tasks.workunit.client.0.smithi012.stdout: 10 983 111.44 MB/sec warmup 2 sec latency 79.252 ms 2024-09-11T00:20:07.076 INFO:tasks.workunit.client.0.smithi012.stdout: 10 1192 77.27 MB/sec warmup 3 sec latency 649.319 ms 2024-09-11T00:20:07.782 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:20:07 smithi165 ceph-mon[35356]: pgmap v790: 129 pgs: 129 active+clean; 125 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 70 MiB/s rd, 2.6 MiB/s wr, 400 op/s 2024-09-11T00:20:07.782 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:20:07 smithi165 ceph-mon[35356]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T00:20:07.813 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:20:07 smithi012 ceph-mon[30333]: pgmap v790: 129 pgs: 129 active+clean; 125 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 70 MiB/s rd, 2.6 MiB/s wr, 400 op/s 2024-09-11T00:20:07.813 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:20:07 smithi012 ceph-mon[30333]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T00:20:07.831 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:20:07 smithi178 ceph-mon[35919]: pgmap v790: 129 pgs: 129 active+clean; 125 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 70 MiB/s rd, 2.6 MiB/s wr, 400 op/s 2024-09-11T00:20:07.831 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:20:07 smithi178 ceph-mon[35919]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T00:20:08.076 INFO:tasks.workunit.client.0.smithi012.stdout: 10 2595 75.11 MB/sec warmup 4 sec latency 59.237 ms 2024-09-11T00:20:08.782 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:20:08 smithi165 ceph-mon[35356]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T00:20:08.813 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:20:08 smithi012 ceph-mon[30333]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T00:20:08.830 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:20:08 smithi178 ceph-mon[35919]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T00:20:09.076 INFO:tasks.workunit.client.0.smithi012.stdout: 10 3514 72.61 MB/sec warmup 5 sec latency 42.979 ms 2024-09-11T00:20:09.782 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:20:09 smithi165 ceph-mon[35356]: pgmap v791: 129 pgs: 129 active+clean; 124 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 44 MiB/s rd, 1.9 MiB/s wr, 289 op/s 2024-09-11T00:20:09.813 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:20:09 smithi012 ceph-mon[30333]: pgmap v791: 129 pgs: 129 active+clean; 124 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 44 MiB/s rd, 1.9 MiB/s wr, 289 op/s 2024-09-11T00:20:09.831 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:20:09 smithi178 ceph-mon[35919]: pgmap v791: 129 pgs: 129 active+clean; 124 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 44 MiB/s rd, 1.9 MiB/s wr, 289 op/s 2024-09-11T00:20:10.076 INFO:tasks.workunit.client.0.smithi012.stdout: 10 4420 68.58 MB/sec warmup 6 sec latency 65.127 ms 2024-09-11T00:20:11.076 INFO:tasks.workunit.client.0.smithi012.stdout: 10 5058 64.56 MB/sec warmup 7 sec latency 548.035 ms 2024-09-11T00:20:11.564 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:20:11 smithi012 ceph-mon[30333]: pgmap v792: 129 pgs: 129 active+clean; 126 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 44 MiB/s rd, 3.0 MiB/s wr, 376 op/s 2024-09-11T00:20:11.564 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:20:11 smithi012 ceph-mon[30333]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T00:20:11.782 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:20:11 smithi165 ceph-mon[35356]: pgmap v792: 129 pgs: 129 active+clean; 126 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 44 MiB/s rd, 3.0 MiB/s wr, 376 op/s 2024-09-11T00:20:11.782 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:20:11 smithi165 ceph-mon[35356]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T00:20:11.831 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:20:11 smithi178 ceph-mon[35919]: pgmap v792: 129 pgs: 129 active+clean; 126 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 44 MiB/s rd, 3.0 MiB/s wr, 376 op/s 2024-09-11T00:20:11.831 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:20:11 smithi178 ceph-mon[35919]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T00:20:12.076 INFO:tasks.workunit.client.0.smithi012.stdout: 10 5288 57.04 MB/sec warmup 8 sec latency 1548.098 ms 2024-09-11T00:20:12.237 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:12.234+0000 7fe84c200640 1 -- 172.21.15.12:0/2090313543 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe844106480 msgr2=0x7fe844111160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:20:12.237 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:12.234+0000 7fe84c200640 1 --2- 172.21.15.12:0/2090313543 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe844106480 0x7fe844111160 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fe83c009ca0 tx=0x7fe83c02a8d0 comp rx=0 tx=0).stop 2024-09-11T00:20:12.238 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:12.235+0000 7fe84c200640 1 -- 172.21.15.12:0/2090313543 shutdown_connections 2024-09-11T00:20:12.238 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:12.235+0000 7fe84c200640 1 --2- 172.21.15.12:0/2090313543 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe844106480 0x7fe844111160 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:20:12.238 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:12.235+0000 7fe84c200640 1 --2- 172.21.15.12:0/2090313543 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe844075b20 0x7fe844075fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:20:12.238 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:12.235+0000 7fe84c200640 1 --2- 172.21.15.12:0/2090313543 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe844076f90 0x7fe8440755e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:20:12.238 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:12.235+0000 7fe84c200640 1 -- 172.21.15.12:0/2090313543 >> 172.21.15.12:0/2090313543 conn(0x7fe8440fe250 msgr2=0x7fe844100670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:20:12.238 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:12.235+0000 7fe84c200640 1 -- 172.21.15.12:0/2090313543 shutdown_connections 2024-09-11T00:20:12.238 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:12.235+0000 7fe84c200640 1 -- 172.21.15.12:0/2090313543 wait complete. 2024-09-11T00:20:12.239 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:12.237+0000 7fe84c200640 1 Processor -- start 2024-09-11T00:20:12.239 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:12.237+0000 7fe84c200640 1 -- start start 2024-09-11T00:20:12.240 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:12.238+0000 7fe84c200640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe844075b20 0x7fe84419f380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:20:12.240 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:12.238+0000 7fe84c200640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe844076f90 0x7fe84419f9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:20:12.240 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:12.238+0000 7fe849800640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe844075b20 0x7fe84419f380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:20:12.241 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:12.238+0000 7fe849800640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe844075b20 0x7fe84419f380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.12:43450/0 (socket says 172.21.15.12:43450) 2024-09-11T00:20:12.241 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:12.238+0000 7fe84c200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe8441a0600 0x7fe8441a0ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:20:12.241 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:12.239+0000 7fe84c200640 1 -- 172.21.15.12:0/3989851074 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fe844114cc0 con 0x7fe844076f90 2024-09-11T00:20:12.241 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:12.239+0000 7fe849800640 1 -- 172.21.15.12:0/3989851074 learned_addr learned my addr 172.21.15.12:0/3989851074 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:20:12.241 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:12.239+0000 7fe84c200640 1 -- 172.21.15.12:0/3989851074 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fe844114b40 con 0x7fe8441a0600 2024-09-11T00:20:12.241 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:12.239+0000 7fe84c200640 1 -- 172.21.15.12:0/3989851074 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fe844114e40 con 0x7fe844075b20 2024-09-11T00:20:12.241 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:12.239+0000 7fe848e00640 1 --2- 172.21.15.12:0/3989851074 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe844076f90 0x7fe84419f9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:20:12.242 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:12.239+0000 7fe84a200640 1 --2- 172.21.15.12:0/3989851074 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe8441a0600 0x7fe8441a0ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:20:12.242 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:12.240+0000 7fe849800640 1 -- 172.21.15.12:0/3989851074 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe8441a0600 msgr2=0x7fe8441a0ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:20:12.242 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:12.240+0000 7fe849800640 1 --2- 172.21.15.12:0/3989851074 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe8441a0600 0x7fe8441a0ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:20:12.242 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:12.240+0000 7fe849800640 1 -- 172.21.15.12:0/3989851074 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe844076f90 msgr2=0x7fe84419f9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:20:12.242 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:12.240+0000 7fe849800640 1 --2- 172.21.15.12:0/3989851074 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe844076f90 0x7fe84419f9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:20:12.242 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:12.240+0000 7fe849800640 1 -- 172.21.15.12:0/3989851074 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8441a12b0 con 0x7fe844075b20 2024-09-11T00:20:12.243 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:12.240+0000 7fe849800640 1 --2- 172.21.15.12:0/3989851074 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe844075b20 0x7fe84419f380 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fe83800b450 tx=0x7fe83800b910 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:20:12.243 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:12.241+0000 7fe842000640 1 -- 172.21.15.12:0/3989851074 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe838015020 con 0x7fe844075b20 2024-09-11T00:20:12.243 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:12.241+0000 7fe842000640 1 -- 172.21.15.12:0/3989851074 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe838011d40 con 0x7fe844075b20 2024-09-11T00:20:12.244 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:12.241+0000 7fe842000640 1 -- 172.21.15.12:0/3989851074 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe83800f340 con 0x7fe844075b20 2024-09-11T00:20:12.244 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:12.242+0000 7fe84c200640 1 -- 172.21.15.12:0/3989851074 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe8441a76a0 con 0x7fe844075b20 2024-09-11T00:20:12.244 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:12.242+0000 7fe84c200640 1 -- 172.21.15.12:0/3989851074 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8441a7bc0 con 0x7fe844075b20 2024-09-11T00:20:12.248 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:12.246+0000 7fe84f23f740 1 Processor -- start 2024-09-11T00:20:12.248 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:12.246+0000 7fe84f23f740 1 -- start start 2024-09-11T00:20:12.248 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:12.246+0000 7fe84f23f740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f6b1c37790 0x55f6b1c39b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:20:12.248 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:12.246+0000 7fe84f23f740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f6b1c3a0c0 0x55f6b1c3c4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:20:12.248 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:12.246+0000 7fe84f23f740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f6b1c3cb00 0x55f6b1c3efc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:20:12.249 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:12.246+0000 7fe84f23f740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55f6b1baa6b0 con 0x55f6b1c37790 2024-09-11T00:20:12.249 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:12.246+0000 7fe84f23f740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55f6b1b99ae0 con 0x55f6b1c3a0c0 2024-09-11T00:20:12.249 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:12.246+0000 7fe84f23f740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55f6b1b9d8f0 con 0x55f6b1c3cb00 2024-09-11T00:20:12.251 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:12.248+0000 7fe842000640 1 -- 172.21.15.12:0/3989851074 <== mon.1 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7fe838011890 con 0x7fe844075b20 2024-09-11T00:20:12.251 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:12.248+0000 7fe848e00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f6b1c3cb00 0x55f6b1c3efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:20:12.251 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:12.248+0000 7fe84a200640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f6b1c37790 0x55f6b1c39b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:20:12.251 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:12.248+0000 7fe848e00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f6b1c3cb00 0x55f6b1c3efc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.12:43452/0 (socket says 172.21.15.12:43452) 2024-09-11T00:20:12.251 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:12.248+0000 7fe848e00640 1 -- 172.21.15.12:0/40912498 learned_addr learned my addr 172.21.15.12:0/40912498 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:20:12.252 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:12.249+0000 7fe849800640 1 --2- 172.21.15.12:0/40912498 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f6b1c3a0c0 0x55f6b1c3c4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:20:12.252 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:12.249+0000 7fe848e00640 1 -- 172.21.15.12:0/40912498 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f6b1c3a0c0 msgr2=0x55f6b1c3c4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:20:12.252 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:12.249+0000 7fe848e00640 1 --2- 172.21.15.12:0/40912498 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f6b1c3a0c0 0x55f6b1c3c4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:20:12.252 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:12.249+0000 7fe848e00640 1 -- 172.21.15.12:0/40912498 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f6b1c37790 msgr2=0x55f6b1c39b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:20:12.252 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:12.249+0000 7fe848e00640 1 --2- 172.21.15.12:0/40912498 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f6b1c37790 0x55f6b1c39b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:20:12.252 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:12.249+0000 7fe848e00640 1 -- 172.21.15.12:0/40912498 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f6b1b66780 con 0x55f6b1c3cb00 2024-09-11T00:20:12.253 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:12.249+0000 7fe849800640 1 --2- 172.21.15.12:0/40912498 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f6b1c3a0c0 0x55f6b1c3c4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T00:20:12.253 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:12.249+0000 7fe84a200640 1 --2- 172.21.15.12:0/40912498 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f6b1c37790 0x55f6b1c39b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T00:20:12.253 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:12.250+0000 7fe848e00640 1 --2- 172.21.15.12:0/40912498 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f6b1c3cb00 0x55f6b1c3efc0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fe83000bd10 tx=0x7fe8300107a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=149a4a56e8a458ef server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:20:12.253 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:12.250+0000 7fe836a00640 1 -- 172.21.15.12:0/40912498 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe830004030 con 0x55f6b1c3cb00 2024-09-11T00:20:12.253 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:12.250+0000 7fe836a00640 1 -- 172.21.15.12:0/40912498 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe8300035f0 con 0x55f6b1c3cb00 2024-09-11T00:20:12.253 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:12.250+0000 7fe836a00640 1 -- 172.21.15.12:0/40912498 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe830011770 con 0x55f6b1c3cb00 2024-09-11T00:20:12.253 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:12.251+0000 7fe842000640 1 --2- 172.21.15.12:0/3989851074 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fe82c0b5430 0x7fe82c0b78f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:20:12.254 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:12.251+0000 7fe842000640 1 -- 172.21.15.12:0/3989851074 <== mon.1 v2:172.21.15.178:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9208+0+0 (secure 0 0 0) 0x7fe8380e5b10 con 0x7fe844075b20 2024-09-11T00:20:12.254 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:12.251+0000 7fe84f23f740 1 -- 172.21.15.12:0/40912498 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f6b1c3cb00 msgr2=0x55f6b1c3efc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:20:12.254 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:12.251+0000 7fe84f23f740 1 --2- 172.21.15.12:0/40912498 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f6b1c3cb00 0x55f6b1c3efc0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fe83000bd10 tx=0x7fe8300107a0 comp rx=0 tx=0).stop 2024-09-11T00:20:12.254 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:12.252+0000 7fe84a200640 1 --2- 172.21.15.12:0/3989851074 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fe82c0b5430 0x7fe82c0b78f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:20:12.254 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:12.252+0000 7fe84f23f740 1 -- 172.21.15.12:0/40912498 shutdown_connections 2024-09-11T00:20:12.255 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:12.252+0000 7fe84f23f740 1 --2- 172.21.15.12:0/40912498 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f6b1c3cb00 0x55f6b1c3efc0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:20:12.255 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:12.252+0000 7fe84f23f740 1 --2- 172.21.15.12:0/40912498 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f6b1c3a0c0 0x55f6b1c3c4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:20:12.255 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:12.252+0000 7fe84f23f740 1 --2- 172.21.15.12:0/40912498 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f6b1c37790 0x55f6b1c39b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:20:12.255 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:12.252+0000 7fe84f23f740 1 -- 172.21.15.12:0/40912498 >> 172.21.15.12:0/40912498 conn(0x55f6b1c328e0 msgr2=0x55f6b1c33670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:20:12.255 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:12.253+0000 7fe84f23f740 1 -- 172.21.15.12:0/40912498 shutdown_connections 2024-09-11T00:20:12.255 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:12.254+0000 7fe84f23f740 1 -- 172.21.15.12:0/40912498 wait complete. 2024-09-11T00:20:12.256 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:12.254+0000 7fe84f23f740 1 -- start start 2024-09-11T00:20:12.256 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:12.254+0000 7fe84f23f740 1 Processor -- start 2024-09-11T00:20:12.257 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:12.255+0000 7fe84a200640 1 --2- 172.21.15.12:0/3989851074 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fe82c0b5430 0x7fe82c0b78f0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fe83c02f550 tx=0x7fe83c031b00 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:20:12.258 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:12.256+0000 7fe84f23f740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f6b1c37790 0x55f6b1c40ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:20:12.258 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:12.256+0000 7fe84f23f740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f6b1c3a0c0 0x55f6b1c41540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:20:12.259 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:12.256+0000 7fe84f23f740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f6b1c3cb00 0x55f6b1c420a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:20:12.259 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:12.257+0000 7fe84f23f740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55f6b1ad1ab0 con 0x55f6b1c3a0c0 2024-09-11T00:20:12.259 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:12.257+0000 7fe84f23f740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55f6b1b40570 con 0x55f6b1c3cb00 2024-09-11T00:20:12.259 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:12.257+0000 7fe84f23f740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55f6b1c3fae0 con 0x55f6b1c37790 2024-09-11T00:20:12.259 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:12.257+0000 7fe849800640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f6b1c37790 0x55f6b1c40ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:20:12.259 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:12.257+0000 7fe84a200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f6b1c3cb00 0x55f6b1c420a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:20:12.260 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:12.257+0000 7fe849800640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f6b1c37790 0x55f6b1c40ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.12:43454/0 (socket says 172.21.15.12:43454) 2024-09-11T00:20:12.260 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:12.257+0000 7fe849800640 1 -- 172.21.15.12:0/3601416915 learned_addr learned my addr 172.21.15.12:0/3601416915 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:20:12.260 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:12.257+0000 7fe84f23f740 10 client.0 _ll_register_callbacks cb 0x55f6b1b4f100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:20:12.260 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:12.257+0000 7fe848e00640 1 --2- 172.21.15.12:0/3601416915 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f6b1c3a0c0 0x55f6b1c41540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:20:12.261 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:12.258+0000 7fe84a200640 1 -- 172.21.15.12:0/3601416915 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f6b1c37790 msgr2=0x55f6b1c40ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:20:12.261 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:12.258+0000 7fe84a200640 1 --2- 172.21.15.12:0/3601416915 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f6b1c37790 0x55f6b1c40ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:20:12.261 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:12.258+0000 7fe84a200640 1 -- 172.21.15.12:0/3601416915 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f6b1c3a0c0 msgr2=0x55f6b1c41540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:20:12.261 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:12.258+0000 7fe84a200640 1 --2- 172.21.15.12:0/3601416915 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f6b1c3a0c0 0x55f6b1c41540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:20:12.261 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:12.258+0000 7fe84a200640 1 -- 172.21.15.12:0/3601416915 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f6b18fb790 con 0x55f6b1c3cb00 2024-09-11T00:20:12.261 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:12.258+0000 7fe848e00640 1 --2- 172.21.15.12:0/3601416915 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f6b1c3a0c0 0x55f6b1c41540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:20:12.261 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:12.259+0000 7fe84a200640 1 --2- 172.21.15.12:0/3601416915 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f6b1c3cb00 0x55f6b1c420a0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fe83c02adf0 tx=0x7fe83c0095c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7c0b8a6bc32d0eb4 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:20:12.261 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:12.259+0000 7fe82b400640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-09-11T00:20:12.263 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:12.259+0000 7fe82b400640 1 -- 172.21.15.12:0/3601416915 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe83c004790 con 0x55f6b1c3cb00 2024-09-11T00:20:12.263 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:12.259+0000 7fe84f23f740 1 -- 172.21.15.12:0/3601416915 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe8300035f0 con 0x55f6b1c3cb00 2024-09-11T00:20:12.263 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:12.259+0000 7fe82b400640 1 -- 172.21.15.12:0/3601416915 <== mon.2 v2:172.21.15.165:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9208+0+0 (secure 0 0 0) 0x7fe83c02f620 con 0x55f6b1c3cb00 2024-09-11T00:20:12.263 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:12.260+0000 7fe82b400640 1 -- 172.21.15.12:0/3601416915 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7fe83c02f820 con 0x55f6b1c3cb00 2024-09-11T00:20:12.263 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:12.260+0000 7fe84f23f740 10 client.26996 fetch_fsmap learned FSMap version 48 2024-09-11T00:20:12.263 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:12.260+0000 7fe84f23f740 1 -- 172.21.15.12:0/3601416915 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x55f6b1974620 con 0x55f6b1c3cb00 2024-09-11T00:20:12.263 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:12.260+0000 7fe82b400640 1 -- 172.21.15.12:0/3601416915 <== mon.2 v2:172.21.15.165:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe8300035f0 con 0x55f6b1c3cb00 2024-09-11T00:20:12.263 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:12.260+0000 7fe82b400640 1 -- 172.21.15.12:0/3601416915 <== mon.2 v2:172.21.15.165:3300/0 5 ==== fsmap(e 48) ==== 8398+0+0 (secure 0 0 0) 0x7fe83c04f070 con 0x55f6b1c3cb00 2024-09-11T00:20:12.263 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:12.260+0000 7fe84f23f740 10 client.26996 fetch_fsmap finished waiting for FSMap version 48 2024-09-11T00:20:12.263 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:12.260+0000 7fe84f23f740 10 client.26996 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T00:20:12.263 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:12.260+0000 7fe84f23f740 20 client.26996 populate_metadata read hostname 'smithi012' 2024-09-11T00:20:12.264 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:12.261+0000 7fe84f23f740 1 --2- 172.21.15.12:0/3601416915 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55f6b1c4c760 0x55f6b1c4ebc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:20:12.264 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:12.261+0000 7fe84f23f740 4 client.26996 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T00:20:12.264 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:12.261+0000 7fe84f23f740 1 -- 172.21.15.12:0/3601416915 --> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f6b1bac600 con 0x55f6b1c4c760 2024-09-11T00:20:12.264 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:12.261+0000 7fe849800640 1 --2- 172.21.15.12:0/3601416915 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55f6b1c4c760 0x55f6b1c4ebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:20:12.265 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:12.262+0000 7fe849800640 1 --2- 172.21.15.12:0/3601416915 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55f6b1c4c760 0x55f6b1c4ebc0 crc :-1 s=READY pgs=197 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6903d571515639a0 server_cookie=35947f0b79f05615 in_seq=0 out_seq=0 2024-09-11T00:20:12.265 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:12.262+0000 7fe82b400640 10 client.26996 ms_handle_connect on v2:172.21.15.165:6838/501312984 2024-09-11T00:20:12.266 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:12.264+0000 7fe82b400640 1 -- 172.21.15.12:0/3601416915 <== mds.0 v2:172.21.15.165:6838/501312984 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55f6b1bac600 con 0x55f6b1c4c760 2024-09-11T00:20:12.266 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:12.264+0000 7fe82b400640 10 client.26996 handle_command_reply: tid=0 2024-09-11T00:20:12.268 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:12.264+0000 7fe84f23f740 1 client.26996 shutdown 2024-09-11T00:20:12.268 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:12.265+0000 7fe84f23f740 1 -- 172.21.15.12:0/3601416915 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f6b1c3cb00 msgr2=0x55f6b1c420a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:20:12.268 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:12.265+0000 7fe84f23f740 1 --2- 172.21.15.12:0/3601416915 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f6b1c3cb00 0x55f6b1c420a0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fe83c02adf0 tx=0x7fe83c0095c0 comp rx=0 tx=0).stop 2024-09-11T00:20:12.268 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:12.265+0000 7fe84f23f740 1 -- 172.21.15.12:0/3601416915 shutdown_connections 2024-09-11T00:20:12.268 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:12.265+0000 7fe84f23f740 1 --2- 172.21.15.12:0/3601416915 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55f6b1c4c760 0x55f6b1c4ebc0 crc :-1 s=READY pgs=197 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:20:12.268 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:12.265+0000 7fe84f23f740 1 --2- 172.21.15.12:0/3601416915 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f6b1c3cb00 0x55f6b1c420a0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:20:12.268 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:12.265+0000 7fe84f23f740 1 --2- 172.21.15.12:0/3601416915 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f6b1c3a0c0 0x55f6b1c41540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:20:12.268 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:12.265+0000 7fe84f23f740 1 --2- 172.21.15.12:0/3601416915 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f6b1c37790 0x55f6b1c40ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:20:12.268 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:12.265+0000 7fe84f23f740 1 -- 172.21.15.12:0/3601416915 >> 172.21.15.12:0/3601416915 conn(0x55f6b1c328e0 msgr2=0x55f6b1c31a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:20:12.268 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:12.265+0000 7fe82b400640 0 client.26996 ms_handle_reset on v2:172.21.15.165:6838/501312984 2024-09-11T00:20:12.269 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:12.265+0000 7fe84f23f740 1 -- 172.21.15.12:0/3601416915 shutdown_connections 2024-09-11T00:20:12.269 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:12.266+0000 7fe84f23f740 1 -- 172.21.15.12:0/3601416915 wait complete. 2024-09-11T00:20:12.269 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:12.266+0000 7fe84f23f740 20 client.26996 trim_cache size 0 max 16384 2024-09-11T00:20:12.300 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:12.298+0000 7fe84f23f740 1 Processor -- start 2024-09-11T00:20:12.301 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:12.299+0000 7fe84f23f740 1 -- start start 2024-09-11T00:20:12.301 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:12.299+0000 7fe84f23f740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f6b1c37790 0x55f6b1c3d5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:20:12.301 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:12.299+0000 7fe84f23f740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f6b1c3a0c0 0x55f6b1c3feb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:20:12.301 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:12.299+0000 7fe84f23f740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f6b1c3cb00 0x55f6b1c403f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:20:12.302 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:12.300+0000 7fe84f23f740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fe8441b1da0 con 0x55f6b1c37790 2024-09-11T00:20:12.302 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:12.300+0000 7fe84f23f740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fe8441b39d0 con 0x55f6b1c3cb00 2024-09-11T00:20:12.302 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:12.300+0000 7fe84f23f740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fe8441b24a0 con 0x55f6b1c3a0c0 2024-09-11T00:20:12.303 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:12.300+0000 7fe848e00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f6b1c3a0c0 0x55f6b1c3feb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:20:12.303 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:12.300+0000 7fe848e00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f6b1c3a0c0 0x55f6b1c3feb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.12:43458/0 (socket says 172.21.15.12:43458) 2024-09-11T00:20:12.303 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:12.300+0000 7fe848e00640 1 -- 172.21.15.12:0/178536995 learned_addr learned my addr 172.21.15.12:0/178536995 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:20:12.303 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:12.301+0000 7fe84a200640 1 --2- 172.21.15.12:0/178536995 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f6b1c3cb00 0x55f6b1c403f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:20:12.303 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:12.301+0000 7fe849800640 1 --2- 172.21.15.12:0/178536995 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f6b1c37790 0x55f6b1c3d5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:20:12.303 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:12.301+0000 7fe84a200640 1 -- 172.21.15.12:0/178536995 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f6b1c3a0c0 msgr2=0x55f6b1c3feb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:20:12.303 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:12.301+0000 7fe84a200640 1 --2- 172.21.15.12:0/178536995 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f6b1c3a0c0 0x55f6b1c3feb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:20:12.303 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:12.301+0000 7fe84a200640 1 -- 172.21.15.12:0/178536995 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f6b1c37790 msgr2=0x55f6b1c3d5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:20:12.304 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:12.301+0000 7fe84a200640 1 --2- 172.21.15.12:0/178536995 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f6b1c37790 0x55f6b1c3d5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:20:12.304 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:12.301+0000 7fe84a200640 1 -- 172.21.15.12:0/178536995 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f6b1c3fc20 con 0x55f6b1c3cb00 2024-09-11T00:20:12.304 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:12.301+0000 7fe849800640 1 --2- 172.21.15.12:0/178536995 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f6b1c37790 0x55f6b1c3d5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T00:20:12.305 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:12.303+0000 7fe84a200640 1 --2- 172.21.15.12:0/178536995 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f6b1c3cb00 0x55f6b1c403f0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fe83c02aec0 tx=0x7fe83c009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6ca66a29106f628b server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:20:12.305 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:12.303+0000 7fe836000640 1 -- 172.21.15.12:0/178536995 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe83c031800 con 0x55f6b1c3cb00 2024-09-11T00:20:12.306 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:12.303+0000 7fe836000640 1 -- 172.21.15.12:0/178536995 <== mon.2 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe83c036040 con 0x55f6b1c3cb00 2024-09-11T00:20:12.306 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:12.303+0000 7fe836000640 1 -- 172.21.15.12:0/178536995 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe83c004670 con 0x55f6b1c3cb00 2024-09-11T00:20:12.306 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:12.304+0000 7fe84f23f740 1 -- 172.21.15.12:0/178536995 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f6b1c3cb00 msgr2=0x55f6b1c403f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:20:12.306 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:12.304+0000 7fe84f23f740 1 --2- 172.21.15.12:0/178536995 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f6b1c3cb00 0x55f6b1c403f0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fe83c02aec0 tx=0x7fe83c009780 comp rx=0 tx=0).stop 2024-09-11T00:20:12.307 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:12.305+0000 7fe84f23f740 1 -- 172.21.15.12:0/178536995 shutdown_connections 2024-09-11T00:20:12.307 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:12.305+0000 7fe84f23f740 1 --2- 172.21.15.12:0/178536995 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f6b1c3cb00 0x55f6b1c403f0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:20:12.307 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:12.305+0000 7fe84f23f740 1 --2- 172.21.15.12:0/178536995 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f6b1c3a0c0 0x55f6b1c3feb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:20:12.307 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:12.305+0000 7fe84f23f740 1 --2- 172.21.15.12:0/178536995 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f6b1c37790 0x55f6b1c3d5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:20:12.307 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:12.305+0000 7fe84f23f740 1 -- 172.21.15.12:0/178536995 >> 172.21.15.12:0/178536995 conn(0x55f6b1c328e0 msgr2=0x55f6b1c3cee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:20:12.309 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:12.307+0000 7fe84f23f740 1 -- 172.21.15.12:0/178536995 shutdown_connections 2024-09-11T00:20:12.309 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:12.307+0000 7fe84f23f740 1 -- 172.21.15.12:0/178536995 wait complete. 2024-09-11T00:20:12.309 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:12.307+0000 7fe84f23f740 1 -- start start 2024-09-11T00:20:12.310 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:12.308+0000 7fe84f23f740 1 Processor -- start 2024-09-11T00:20:12.311 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:12.309+0000 7fe84f23f740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f6b1c37790 0x55f6b1c47980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:20:12.311 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:12.309+0000 7fe84f23f740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f6b1c3a0c0 0x55f6b1c47ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:20:12.311 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:12.309+0000 7fe84f23f740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f6b1c3cb00 0x55f6b1c48400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:20:12.311 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:12.309+0000 7fe84f23f740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fe8441b3300 con 0x55f6b1c3a0c0 2024-09-11T00:20:12.311 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:12.309+0000 7fe84f23f740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fe8441b2bd0 con 0x55f6b1c3cb00 2024-09-11T00:20:12.311 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:12.309+0000 7fe84f23f740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fe8441afd80 con 0x55f6b1c37790 2024-09-11T00:20:12.311 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:12.309+0000 7fe84f23f740 10 client.0 _ll_register_callbacks cb 0x7fe83c009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:20:12.312 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:12.309+0000 7fe849800640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f6b1c37790 0x55f6b1c47980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:20:12.312 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:12.309+0000 7fe848e00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f6b1c3a0c0 0x55f6b1c47ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:20:12.312 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:12.310+0000 7fe848e00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f6b1c3a0c0 0x55f6b1c47ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:58354/0 (socket says 172.21.15.12:58354) 2024-09-11T00:20:12.312 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:12.310+0000 7fe848e00640 1 -- 172.21.15.12:0/2259654304 learned_addr learned my addr 172.21.15.12:0/2259654304 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:20:12.314 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:12.310+0000 7fe84a200640 1 --2- 172.21.15.12:0/2259654304 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f6b1c3cb00 0x55f6b1c48400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:20:12.314 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:12.310+0000 7fe849800640 1 -- 172.21.15.12:0/2259654304 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f6b1c3cb00 msgr2=0x55f6b1c48400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:20:12.314 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:12.310+0000 7fe849800640 1 --2- 172.21.15.12:0/2259654304 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f6b1c3cb00 0x55f6b1c48400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:20:12.314 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:12.310+0000 7fe849800640 1 -- 172.21.15.12:0/2259654304 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f6b1c3a0c0 msgr2=0x55f6b1c47ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:20:12.315 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:12.311+0000 7fe849800640 1 --2- 172.21.15.12:0/2259654304 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f6b1c3a0c0 0x55f6b1c47ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:20:12.315 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:12.311+0000 7fe849800640 1 -- 172.21.15.12:0/2259654304 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f6b1c46640 con 0x55f6b1c37790 2024-09-11T00:20:12.315 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:12.311+0000 7fe848e00640 1 --2- 172.21.15.12:0/2259654304 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f6b1c3a0c0 0x55f6b1c47ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:20:12.315 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:12.313+0000 7fe849800640 1 --2- 172.21.15.12:0/2259654304 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f6b1c37790 0x55f6b1c47980 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fe8380a84d0 tx=0x7fe8380e8020 comp rx=0 tx=0).ready entity=mon.1 client_cookie=487693979fce8138 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:20:12.315 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:12.313+0000 7fe82b400640 10 client.0 ms_handle_connect on v2:172.21.15.178:3300/0 2024-09-11T00:20:12.316 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:12.313+0000 7fe82b400640 1 -- 172.21.15.12:0/2259654304 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8380aba10 con 0x55f6b1c37790 2024-09-11T00:20:12.316 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:12.313+0000 7fe84f23f740 1 -- 172.21.15.12:0/2259654304 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe83c036040 con 0x55f6b1c37790 2024-09-11T00:20:12.316 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:12.314+0000 7fe82b400640 1 -- 172.21.15.12:0/2259654304 <== mon.1 v2:172.21.15.178:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9208+0+0 (secure 0 0 0) 0x7fe8380ab160 con 0x55f6b1c37790 2024-09-11T00:20:12.317 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:12.314+0000 7fe82b400640 1 -- 172.21.15.12:0/2259654304 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7fe8380ac600 con 0x55f6b1c37790 2024-09-11T00:20:12.317 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:12.315+0000 7fe84f23f740 10 client.26980 fetch_fsmap learned FSMap version 48 2024-09-11T00:20:12.317 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:12.315+0000 7fe84f23f740 1 -- 172.21.15.12:0/2259654304 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x55f6b1cb9a80 con 0x55f6b1c37790 2024-09-11T00:20:12.318 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:12.316+0000 7fe82b400640 1 -- 172.21.15.12:0/2259654304 <== mon.1 v2:172.21.15.178:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe83c036040 con 0x55f6b1c37790 2024-09-11T00:20:12.318 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:12.316+0000 7fe82b400640 1 -- 172.21.15.12:0/2259654304 <== mon.1 v2:172.21.15.178:3300/0 5 ==== fsmap(e 48) ==== 8398+0+0 (secure 0 0 0) 0x7fe8380d1020 con 0x55f6b1c37790 2024-09-11T00:20:12.318 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:12.316+0000 7fe84f23f740 10 client.26980 fetch_fsmap finished waiting for FSMap version 48 2024-09-11T00:20:12.318 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:12.316+0000 7fe84f23f740 10 client.26980 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T00:20:12.319 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:12.316+0000 7fe84f23f740 20 client.26980 populate_metadata read hostname 'smithi012' 2024-09-11T00:20:12.319 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:12.316+0000 7fe84f23f740 1 --2- 172.21.15.12:0/2259654304 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55f6b1cba440 0x55f6b1cbc860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:20:12.319 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:12.316+0000 7fe84f23f740 4 client.26980 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T00:20:12.319 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:12.316+0000 7fe84f23f740 1 -- 172.21.15.12:0/2259654304 --> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55f6b1ba80b0 con 0x55f6b1cba440 2024-09-11T00:20:12.319 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:12.317+0000 7fe848e00640 1 --2- 172.21.15.12:0/2259654304 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55f6b1cba440 0x55f6b1cbc860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:20:12.320 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:12.318+0000 7fe82b400640 10 client.26980 ms_handle_connect on v2:172.21.15.165:6838/501312984 2024-09-11T00:20:12.321 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:12.319+0000 7fe848e00640 1 --2- 172.21.15.12:0/2259654304 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55f6b1cba440 0x55f6b1cbc860 crc :-1 s=READY pgs=198 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f969c183369c0c7c server_cookie=5aa7d98718d69b42 in_seq=0 out_seq=0 2024-09-11T00:20:12.782 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:20:12 smithi165 ceph-mon[35356]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T00:20:12.813 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:20:12 smithi012 ceph-mon[30333]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T00:20:12.830 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:20:12 smithi178 ceph-mon[35919]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T00:20:12.917 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:12.914+0000 7fe82b400640 1 -- 172.21.15.12:0/2259654304 <== mds.0 v2:172.21.15.165:6838/501312984 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55f6b1ba80b0 con 0x55f6b1cba440 2024-09-11T00:20:12.917 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:12.914+0000 7fe82b400640 10 client.26980 handle_command_reply: tid=0 2024-09-11T00:20:12.917 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:12.914+0000 7fe84f23f740 1 client.26980 shutdown 2024-09-11T00:20:12.917 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:12.915+0000 7fe84f23f740 1 -- 172.21.15.12:0/2259654304 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f6b1c37790 msgr2=0x55f6b1c47980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:20:12.917 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:12.915+0000 7fe84f23f740 1 --2- 172.21.15.12:0/2259654304 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f6b1c37790 0x55f6b1c47980 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fe8380a84d0 tx=0x7fe8380e8020 comp rx=0 tx=0).stop 2024-09-11T00:20:12.918 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:12.915+0000 7fe84f23f740 1 -- 172.21.15.12:0/2259654304 shutdown_connections 2024-09-11T00:20:12.918 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:12.915+0000 7fe84f23f740 1 --2- 172.21.15.12:0/2259654304 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55f6b1cba440 0x55f6b1cbc860 crc :-1 s=READY pgs=198 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:20:12.918 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:12.915+0000 7fe84f23f740 1 --2- 172.21.15.12:0/2259654304 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f6b1c3cb00 0x55f6b1c48400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:20:12.918 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:12.915+0000 7fe84f23f740 1 --2- 172.21.15.12:0/2259654304 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f6b1c3a0c0 0x55f6b1c47ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:20:12.918 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:12.915+0000 7fe84f23f740 1 --2- 172.21.15.12:0/2259654304 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f6b1c37790 0x55f6b1c47980 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:20:12.918 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:12.915+0000 7fe84f23f740 1 -- 172.21.15.12:0/2259654304 >> 172.21.15.12:0/2259654304 conn(0x55f6b1c328e0 msgr2=0x55f6b1c3feb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:20:12.918 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:12.915+0000 7fe82b400640 0 client.26980 ms_handle_reset on v2:172.21.15.165:6838/501312984 2024-09-11T00:20:12.919 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:12.917+0000 7fe84f23f740 1 -- 172.21.15.12:0/2259654304 shutdown_connections 2024-09-11T00:20:12.919 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:12.917+0000 7fe84f23f740 1 -- 172.21.15.12:0/2259654304 wait complete. 2024-09-11T00:20:12.919 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:12.917+0000 7fe84f23f740 20 client.26980 trim_cache size 0 max 16384 2024-09-11T00:20:12.920 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:{ 2024-09-11T00:20:12.920 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "message": "", 2024-09-11T00:20:12.920 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "return_code": 0 2024-09-11T00:20:12.920 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:} 2024-09-11T00:20:12.921 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:12.918+0000 7fe84c200640 1 -- 172.21.15.12:0/3989851074 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fe82c0b5430 msgr2=0x7fe82c0b78f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:20:12.921 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:12.918+0000 7fe84c200640 1 --2- 172.21.15.12:0/3989851074 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fe82c0b5430 0x7fe82c0b78f0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fe83c02f550 tx=0x7fe83c031b00 comp rx=0 tx=0).stop 2024-09-11T00:20:12.921 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:12.918+0000 7fe84c200640 1 -- 172.21.15.12:0/3989851074 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe844075b20 msgr2=0x7fe84419f380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:20:12.921 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:12.918+0000 7fe84c200640 1 --2- 172.21.15.12:0/3989851074 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe844075b20 0x7fe84419f380 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fe83800b450 tx=0x7fe83800b910 comp rx=0 tx=0).stop 2024-09-11T00:20:12.921 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:12.918+0000 7fe84c200640 1 -- 172.21.15.12:0/3989851074 shutdown_connections 2024-09-11T00:20:12.921 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:12.918+0000 7fe84c200640 1 --2- 172.21.15.12:0/3989851074 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fe82c0b5430 0x7fe82c0b78f0 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:20:12.921 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:12.918+0000 7fe84c200640 1 --2- 172.21.15.12:0/3989851074 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe8441a0600 0x7fe8441a0ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:20:12.921 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:12.918+0000 7fe84c200640 1 --2- 172.21.15.12:0/3989851074 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe844076f90 0x7fe84419f9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:20:12.922 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:12.918+0000 7fe84c200640 1 --2- 172.21.15.12:0/3989851074 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe844075b20 0x7fe84419f380 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:20:12.922 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:12.918+0000 7fe84c200640 1 -- 172.21.15.12:0/3989851074 >> 172.21.15.12:0/3989851074 conn(0x7fe8440fe250 msgr2=0x7fe844102420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:20:12.922 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:12.919+0000 7fe84c200640 1 -- 172.21.15.12:0/3989851074 shutdown_connections 2024-09-11T00:20:12.922 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:12.919+0000 7fe84c200640 1 -- 172.21.15.12:0/3989851074 wait complete. 2024-09-11T00:20:13.077 INFO:tasks.workunit.client.0.smithi012.stdout: 10 5615 52.39 MB/sec warmup 9 sec latency 2510.814 ms 2024-09-11T00:20:13.782 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:20:13 smithi165 ceph-mon[35356]: pgmap v793: 129 pgs: 129 active+clean; 131 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 17 MiB/s rd, 3.4 MiB/s wr, 358 op/s 2024-09-11T00:20:13.783 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:20:13 smithi165 ceph-mon[35356]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T00:20:13.785 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:20:13 smithi012 ceph-mon[30333]: pgmap v793: 129 pgs: 129 active+clean; 131 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 17 MiB/s rd, 3.4 MiB/s wr, 358 op/s 2024-09-11T00:20:13.785 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:20:13 smithi012 ceph-mon[30333]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T00:20:13.831 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:20:13 smithi178 ceph-mon[35919]: pgmap v793: 129 pgs: 129 active+clean; 131 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 17 MiB/s rd, 3.4 MiB/s wr, 358 op/s 2024-09-11T00:20:13.831 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:20:13 smithi178 ceph-mon[35919]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T00:20:14.077 INFO:tasks.workunit.client.0.smithi012.stdout: 10 6987 53.74 MB/sec warmup 10 sec latency 46.086 ms 2024-09-11T00:20:15.078 INFO:tasks.workunit.client.0.smithi012.stdout: 10 8001 56.07 MB/sec warmup 11 sec latency 38.619 ms 2024-09-11T00:20:15.813 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:20:15 smithi012 ceph-mon[30333]: pgmap v794: 129 pgs: 129 active+clean; 132 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 3.4 MiB/s wr, 342 op/s 2024-09-11T00:20:15.831 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:20:15 smithi178 ceph-mon[35919]: pgmap v794: 129 pgs: 129 active+clean; 132 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 3.4 MiB/s wr, 342 op/s 2024-09-11T00:20:16.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:20:15 smithi165 ceph-mon[35356]: pgmap v794: 129 pgs: 129 active+clean; 132 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 3.4 MiB/s wr, 342 op/s 2024-09-11T00:20:16.078 INFO:tasks.workunit.client.0.smithi012.stdout: 10 8622 53.45 MB/sec warmup 12 sec latency 56.156 ms 2024-09-11T00:20:17.124 INFO:tasks.workunit.client.0.smithi012.stdout: 10 8794 49.51 MB/sec warmup 13 sec latency 805.976 ms 2024-09-11T00:20:17.553 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:20:17 smithi012 ceph-mon[30333]: pgmap v795: 129 pgs: 129 active+clean; 131 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 17 MiB/s rd, 4.9 MiB/s wr, 687 op/s 2024-09-11T00:20:17.831 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:20:17 smithi178 ceph-mon[35919]: pgmap v795: 129 pgs: 129 active+clean; 131 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 17 MiB/s rd, 4.9 MiB/s wr, 687 op/s 2024-09-11T00:20:18.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:20:17 smithi165 ceph-mon[35356]: pgmap v795: 129 pgs: 129 active+clean; 131 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 17 MiB/s rd, 4.9 MiB/s wr, 687 op/s 2024-09-11T00:20:18.132 INFO:tasks.workunit.client.0.smithi012.stdout: 10 8939 46.76 MB/sec warmup 14 sec latency 1761.430 ms 2024-09-11T00:20:18.773 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:20:18 smithi012 ceph-mon[30333]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T00:20:19.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:20:18 smithi165 ceph-mon[35356]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T00:20:19.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:20:18 smithi178 ceph-mon[35919]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T00:20:19.132 INFO:tasks.workunit.client.0.smithi012.stdout: 10 9048 44.13 MB/sec warmup 15 sec latency 2761.522 ms 2024-09-11T00:20:19.627 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:20:19 smithi012 ceph-mon[30333]: pgmap v796: 129 pgs: 129 active+clean; 138 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 4.5 MiB/s wr, 606 op/s 2024-09-11T00:20:20.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:20:19 smithi165 ceph-mon[35356]: pgmap v796: 129 pgs: 129 active+clean; 138 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 4.5 MiB/s wr, 606 op/s 2024-09-11T00:20:20.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:20:19 smithi178 ceph-mon[35919]: pgmap v796: 129 pgs: 129 active+clean; 138 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 4.5 MiB/s wr, 606 op/s 2024-09-11T00:20:20.132 INFO:tasks.workunit.client.0.smithi012.stdout: 10 9230 42.27 MB/sec warmup 16 sec latency 3761.572 ms 2024-09-11T00:20:21.132 INFO:tasks.workunit.client.0.smithi012.stdout: 10 9409 39.92 MB/sec warmup 17 sec latency 4547.765 ms 2024-09-11T00:20:22.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:20:21 smithi165 ceph-mon[35356]: pgmap v797: 129 pgs: 129 active+clean; 186 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 9.6 MiB/s wr, 695 op/s 2024-09-11T00:20:22.064 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:20:21 smithi012 ceph-mon[30333]: pgmap v797: 129 pgs: 129 active+clean; 186 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 9.6 MiB/s wr, 695 op/s 2024-09-11T00:20:22.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:20:21 smithi178 ceph-mon[35919]: pgmap v797: 129 pgs: 129 active+clean; 186 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 9.6 MiB/s wr, 695 op/s 2024-09-11T00:20:22.133 INFO:tasks.workunit.client.0.smithi012.stdout: 10 9663 37.97 MB/sec warmup 18 sec latency 441.556 ms 2024-09-11T00:20:23.141 INFO:tasks.workunit.client.0.smithi012.stdout: 10 9684 36.01 MB/sec warmup 19 sec latency 989.037 ms 2024-09-11T00:20:23.785 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:20:23 smithi012 ceph-mon[30333]: pgmap v798: 129 pgs: 129 active+clean; 208 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 11 MiB/s wr, 648 op/s 2024-09-11T00:20:23.785 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:20:23 smithi012 ceph-mon[30333]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T00:20:24.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:20:23 smithi165 ceph-mon[35356]: pgmap v798: 129 pgs: 129 active+clean; 208 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 11 MiB/s wr, 648 op/s 2024-09-11T00:20:24.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:20:23 smithi165 ceph-mon[35356]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T00:20:24.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:20:23 smithi178 ceph-mon[35919]: pgmap v798: 129 pgs: 129 active+clean; 208 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 11 MiB/s wr, 648 op/s 2024-09-11T00:20:24.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:20:23 smithi178 ceph-mon[35919]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T00:20:24.141 INFO:tasks.workunit.client.0.smithi012.stdout: 10 9770 34.46 MB/sec warmup 20 sec latency 1886.008 ms 2024-09-11T00:20:25.141 INFO:tasks.workunit.client.0.smithi012.stdout: 10 9973 33.46 MB/sec warmup 21 sec latency 2864.878 ms 2024-09-11T00:20:25.694 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:20:25 smithi012 ceph-mon[30333]: pgmap v799: 129 pgs: 129 active+clean; 223 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 11 MiB/s wr, 570 op/s 2024-09-11T00:20:26.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:20:25 smithi165 ceph-mon[35356]: pgmap v799: 129 pgs: 129 active+clean; 223 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 11 MiB/s wr, 570 op/s 2024-09-11T00:20:26.048 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:26.044+0000 7fa168400640 1 -- 172.21.15.12:0/284663651 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa160103ad0 msgr2=0x7fa160103f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:20:26.048 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:26.044+0000 7fa168400640 1 --2- 172.21.15.12:0/284663651 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa160103ad0 0x7fa160103f50 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fa14c009930 tx=0x7fa14c02a420 comp rx=0 tx=0).stop 2024-09-11T00:20:26.049 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:26.044+0000 7fa168400640 1 -- 172.21.15.12:0/284663651 shutdown_connections 2024-09-11T00:20:26.049 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:26.044+0000 7fa168400640 1 --2- 172.21.15.12:0/284663651 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa160104490 0x7fa16010ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:20:26.049 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:26.044+0000 7fa168400640 1 --2- 172.21.15.12:0/284663651 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa160103ad0 0x7fa160103f50 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:20:26.049 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:26.044+0000 7fa168400640 1 --2- 172.21.15.12:0/284663651 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa160102980 0x7fa160102d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:20:26.050 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:26.044+0000 7fa168400640 1 -- 172.21.15.12:0/284663651 >> 172.21.15.12:0/284663651 conn(0x7fa1600fe240 msgr2=0x7fa160100660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:20:26.050 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:26.044+0000 7fa168400640 1 -- 172.21.15.12:0/284663651 shutdown_connections 2024-09-11T00:20:26.050 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:26.044+0000 7fa168400640 1 -- 172.21.15.12:0/284663651 wait complete. 2024-09-11T00:20:26.051 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:26.044+0000 7fa168400640 1 Processor -- start 2024-09-11T00:20:26.051 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:26.044+0000 7fa168400640 1 -- start start 2024-09-11T00:20:26.051 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:26.048+0000 7fa168400640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa160102980 0x7fa160076ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:20:26.051 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:26.048+0000 7fa168400640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa160103ad0 0x7fa160075540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:20:26.051 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:26.048+0000 7fa168400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa160104490 0x7fa160075a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:20:26.051 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:26.048+0000 7fa168400640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fa16010e620 con 0x7fa160102980 2024-09-11T00:20:26.051 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:26.048+0000 7fa168400640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fa16010e4a0 con 0x7fa160104490 2024-09-11T00:20:26.051 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:26.048+0000 7fa168400640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fa16010e7a0 con 0x7fa160103ad0 2024-09-11T00:20:26.052 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:26.048+0000 7fa165a00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa160102980 0x7fa160076ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:20:26.052 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:26.048+0000 7fa165a00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa160102980 0x7fa160076ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:45168/0 (socket says 172.21.15.12:45168) 2024-09-11T00:20:26.052 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:26.048+0000 7fa165a00640 1 -- 172.21.15.12:0/2619626940 learned_addr learned my addr 172.21.15.12:0/2619626940 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:20:26.052 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:26.048+0000 7fa165000640 1 --2- 172.21.15.12:0/2619626940 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa160103ad0 0x7fa160075540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:20:26.052 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:26.048+0000 7fa166400640 1 --2- 172.21.15.12:0/2619626940 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa160104490 0x7fa160075a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:20:26.052 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:26.048+0000 7fa165000640 1 -- 172.21.15.12:0/2619626940 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa160104490 msgr2=0x7fa160075a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:20:26.052 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:26.048+0000 7fa165000640 1 --2- 172.21.15.12:0/2619626940 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa160104490 0x7fa160075a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:20:26.053 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:26.048+0000 7fa165000640 1 -- 172.21.15.12:0/2619626940 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa160102980 msgr2=0x7fa160076ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:20:26.053 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:26.048+0000 7fa165000640 1 --2- 172.21.15.12:0/2619626940 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa160102980 0x7fa160076ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:20:26.057 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:26.048+0000 7fa165000640 1 -- 172.21.15.12:0/2619626940 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1600762f0 con 0x7fa160103ad0 2024-09-11T00:20:26.057 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:26.048+0000 7fa165a00640 1 --2- 172.21.15.12:0/2619626940 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa160102980 0x7fa160076ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:20:26.057 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:26.048+0000 7fa165000640 1 --2- 172.21.15.12:0/2619626940 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa160103ad0 0x7fa160075540 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fa14c02a940 tx=0x7fa14c003c80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:20:26.057 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:26.048+0000 7fa15e000640 1 -- 172.21.15.12:0/2619626940 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa14c035030 con 0x7fa160103ad0 2024-09-11T00:20:26.058 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:26.048+0000 7fa168400640 1 -- 172.21.15.12:0/2619626940 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1601989c0 con 0x7fa160103ad0 2024-09-11T00:20:26.058 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:26.050+0000 7fa15e000640 1 -- 172.21.15.12:0/2619626940 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa14c045070 con 0x7fa160103ad0 2024-09-11T00:20:26.058 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:26.050+0000 7fa15e000640 1 -- 172.21.15.12:0/2619626940 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa14c030720 con 0x7fa160103ad0 2024-09-11T00:20:26.058 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:26.050+0000 7fa168400640 1 -- 172.21.15.12:0/2619626940 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa160198ed0 con 0x7fa160103ad0 2024-09-11T00:20:26.058 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:26.053+0000 7fa16b220740 1 Processor -- start 2024-09-11T00:20:26.058 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:26.053+0000 7fa16b220740 1 -- start start 2024-09-11T00:20:26.058 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:26.053+0000 7fa15e000640 1 -- 172.21.15.12:0/2619626940 <== mon.1 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7fa14c0308c0 con 0x7fa160103ad0 2024-09-11T00:20:26.058 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:26.053+0000 7fa16b220740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d0c2120790 0x55d0c2122b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:20:26.058 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:26.053+0000 7fa16b220740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d0c21230c0 0x55d0c21254b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:20:26.058 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:26.053+0000 7fa16b220740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d0c2125b00 0x55d0c2127fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:20:26.058 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:26.053+0000 7fa16b220740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55d0c20936b0 con 0x55d0c21230c0 2024-09-11T00:20:26.058 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:26.053+0000 7fa16b220740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55d0c2082ae0 con 0x55d0c2120790 2024-09-11T00:20:26.059 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:26.053+0000 7fa16b220740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55d0c20868f0 con 0x55d0c2125b00 2024-09-11T00:20:26.059 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:26.053+0000 7fa165a00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d0c21230c0 0x55d0c21254b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:20:26.059 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:26.053+0000 7fa165000640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d0c2125b00 0x55d0c2127fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:20:26.059 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:26.053+0000 7fa165a00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d0c21230c0 0x55d0c21254b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:45178/0 (socket says 172.21.15.12:45178) 2024-09-11T00:20:26.059 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:26.053+0000 7fa165a00640 1 -- 172.21.15.12:0/4020557377 learned_addr learned my addr 172.21.15.12:0/4020557377 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:20:26.059 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:26.053+0000 7fa166400640 1 --2- 172.21.15.12:0/4020557377 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d0c2120790 0x55d0c2122b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:20:26.065 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:26.053+0000 7fa165a00640 1 -- 172.21.15.12:0/4020557377 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d0c2125b00 msgr2=0x55d0c2127fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:20:26.066 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:26.053+0000 7fa165a00640 1 --2- 172.21.15.12:0/4020557377 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d0c2125b00 0x55d0c2127fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:20:26.066 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:26.053+0000 7fa165a00640 1 -- 172.21.15.12:0/4020557377 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d0c2120790 msgr2=0x55d0c2122b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:20:26.066 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:26.053+0000 7fa165a00640 1 --2- 172.21.15.12:0/4020557377 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d0c2120790 0x55d0c2122b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:20:26.066 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:26.053+0000 7fa165a00640 1 -- 172.21.15.12:0/4020557377 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d0c204f780 con 0x55d0c21230c0 2024-09-11T00:20:26.066 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:26.053+0000 7fa165a00640 1 --2- 172.21.15.12:0/4020557377 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d0c21230c0 0x55d0c21254b0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fa1540107a0 tx=0x7fa154010c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9aae393c5da8eb32 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:20:26.066 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:26.053+0000 7fa152a00640 1 -- 172.21.15.12:0/4020557377 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa15400e930 con 0x55d0c21230c0 2024-09-11T00:20:26.066 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:26.053+0000 7fa152a00640 1 -- 172.21.15.12:0/4020557377 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa154011440 con 0x55d0c21230c0 2024-09-11T00:20:26.066 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:26.053+0000 7fa152a00640 1 -- 172.21.15.12:0/4020557377 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa154019630 con 0x55d0c21230c0 2024-09-11T00:20:26.066 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:26.053+0000 7fa15e000640 1 --2- 172.21.15.12:0/2619626940 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fa1480b52e0 0x7fa1480b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:20:26.066 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:26.053+0000 7fa15e000640 1 -- 172.21.15.12:0/2619626940 <== mon.1 v2:172.21.15.178:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9208+0+0 (secure 0 0 0) 0x7fa14c10eef0 con 0x7fa160103ad0 2024-09-11T00:20:26.066 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:26.053+0000 7fa166400640 1 --2- 172.21.15.12:0/2619626940 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fa1480b52e0 0x7fa1480b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:20:26.067 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:26.053+0000 7fa16b220740 1 -- 172.21.15.12:0/4020557377 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d0c21230c0 msgr2=0x55d0c21254b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:20:26.067 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:26.053+0000 7fa16b220740 1 --2- 172.21.15.12:0/4020557377 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d0c21230c0 0x55d0c21254b0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fa1540107a0 tx=0x7fa154010c60 comp rx=0 tx=0).stop 2024-09-11T00:20:26.067 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:26.053+0000 7fa16b220740 1 -- 172.21.15.12:0/4020557377 shutdown_connections 2024-09-11T00:20:26.067 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:26.053+0000 7fa16b220740 1 --2- 172.21.15.12:0/4020557377 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d0c2125b00 0x55d0c2127fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:20:26.067 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:26.053+0000 7fa16b220740 1 --2- 172.21.15.12:0/4020557377 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d0c21230c0 0x55d0c21254b0 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:20:26.067 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:26.053+0000 7fa16b220740 1 --2- 172.21.15.12:0/4020557377 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d0c2120790 0x55d0c2122b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:20:26.067 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:26.053+0000 7fa16b220740 1 -- 172.21.15.12:0/4020557377 >> 172.21.15.12:0/4020557377 conn(0x55d0c211b8e0 msgr2=0x55d0c211c670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:20:26.067 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:26.053+0000 7fa16b220740 1 -- 172.21.15.12:0/4020557377 shutdown_connections 2024-09-11T00:20:26.067 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:26.053+0000 7fa166400640 1 --2- 172.21.15.12:0/2619626940 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fa1480b52e0 0x7fa1480b77a0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fa158005fd0 tx=0x7fa1580096a0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:20:26.067 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:26.053+0000 7fa16b220740 1 -- 172.21.15.12:0/4020557377 wait complete. 2024-09-11T00:20:26.067 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:26.053+0000 7fa16b220740 1 -- start start 2024-09-11T00:20:26.067 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:26.053+0000 7fa16b220740 1 Processor -- start 2024-09-11T00:20:26.067 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:26.053+0000 7fa16b220740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d0c2120790 0x55d0c2129ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:20:26.068 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:26.053+0000 7fa16b220740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d0c21230c0 0x55d0c212a540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:20:26.068 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:26.053+0000 7fa16b220740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d0c2125b00 0x55d0c212b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:20:26.068 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:26.053+0000 7fa16b220740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55d0c1fbaab0 con 0x55d0c21230c0 2024-09-11T00:20:26.068 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:26.053+0000 7fa16b220740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55d0c2029570 con 0x55d0c2120790 2024-09-11T00:20:26.068 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:26.053+0000 7fa16b220740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55d0c2128ae0 con 0x55d0c2125b00 2024-09-11T00:20:26.068 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:26.053+0000 7fa16b220740 10 client.0 _ll_register_callbacks cb 0x55d0c2038100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:20:26.068 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:26.053+0000 7fa165000640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d0c21230c0 0x55d0c212a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:20:26.068 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:26.053+0000 7fa166400640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d0c2125b00 0x55d0c212b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:20:26.068 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:26.053+0000 7fa165a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d0c2120790 0x55d0c2129ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:20:26.068 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:26.053+0000 7fa165000640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d0c21230c0 0x55d0c212a540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:45192/0 (socket says 172.21.15.12:45192) 2024-09-11T00:20:26.068 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:26.053+0000 7fa166400640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d0c2125b00 0x55d0c212b0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.12:45544/0 (socket says 172.21.15.12:45544) 2024-09-11T00:20:26.068 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:26.053+0000 7fa165000640 1 -- 172.21.15.12:0/3641379478 learned_addr learned my addr 172.21.15.12:0/3641379478 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:20:26.068 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:26.053+0000 7fa166400640 1 -- 172.21.15.12:0/3641379478 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d0c2120790 msgr2=0x55d0c2129ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:20:26.069 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:26.053+0000 7fa166400640 1 --2- 172.21.15.12:0/3641379478 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d0c2120790 0x55d0c2129ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:20:26.069 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:26.053+0000 7fa166400640 1 -- 172.21.15.12:0/3641379478 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d0c21230c0 msgr2=0x55d0c212a540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:20:26.069 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:26.053+0000 7fa166400640 1 --2- 172.21.15.12:0/3641379478 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d0c21230c0 0x55d0c212a540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:20:26.069 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:26.053+0000 7fa166400640 1 -- 172.21.15.12:0/3641379478 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d0c1de4790 con 0x55d0c2125b00 2024-09-11T00:20:26.069 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:26.053+0000 7fa166400640 1 --2- 172.21.15.12:0/3641379478 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d0c2125b00 0x55d0c212b0a0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fa158011db0 tx=0x7fa158004ab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=47143872e4bc79c0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:20:26.069 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:26.053+0000 7fa147400640 10 client.0 ms_handle_connect on v2:172.21.15.178:3300/0 2024-09-11T00:20:26.069 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:26.053+0000 7fa147400640 1 -- 172.21.15.12:0/3641379478 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa158003c40 con 0x55d0c2125b00 2024-09-11T00:20:26.069 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:26.053+0000 7fa16b220740 1 -- 172.21.15.12:0/3641379478 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa154011440 con 0x55d0c2125b00 2024-09-11T00:20:26.069 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:26.053+0000 7fa147400640 1 -- 172.21.15.12:0/3641379478 <== mon.1 v2:172.21.15.178:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9208+0+0 (secure 0 0 0) 0x7fa158017030 con 0x55d0c2125b00 2024-09-11T00:20:26.069 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:26.053+0000 7fa147400640 1 -- 172.21.15.12:0/3641379478 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) ==== 24+0+0 (secure 0 0 0) 0x7fa158003de0 con 0x55d0c2125b00 2024-09-11T00:20:26.069 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:26.053+0000 7fa16b220740 10 client.26992 fetch_fsmap learned FSMap version 51 2024-09-11T00:20:26.069 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:26.053+0000 7fa16b220740 1 -- 172.21.15.12:0/3641379478 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=4+}) -- 0x55d0c1e5d620 con 0x55d0c2125b00 2024-09-11T00:20:26.070 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:26.053+0000 7fa147400640 1 -- 172.21.15.12:0/3641379478 <== mon.1 v2:172.21.15.178:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa154011440 con 0x55d0c2125b00 2024-09-11T00:20:26.070 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:26.059+0000 7fa147400640 1 -- 172.21.15.12:0/3641379478 <== mon.1 v2:172.21.15.178:3300/0 5 ==== fsmap(e 51) ==== 8406+0+0 (secure 0 0 0) 0x7fa15801f960 con 0x55d0c2125b00 2024-09-11T00:20:26.070 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:26.059+0000 7fa16b220740 10 client.26992 fetch_fsmap finished waiting for FSMap version 51 2024-09-11T00:20:26.070 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:26.059+0000 7fa16b220740 10 client.26992 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T00:20:26.070 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:26.059+0000 7fa16b220740 20 client.26992 populate_metadata read hostname 'smithi012' 2024-09-11T00:20:26.070 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:26.059+0000 7fa16b220740 1 --2- 172.21.15.12:0/3641379478 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55d0c21358d0 0x55d0c2137d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:20:26.070 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:26.059+0000 7fa16b220740 4 client.26992 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T00:20:26.070 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:26.059+0000 7fa16b220740 1 -- 172.21.15.12:0/3641379478 --> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d0c2095600 con 0x55d0c21358d0 2024-09-11T00:20:26.070 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:26.059+0000 7fa165a00640 1 --2- 172.21.15.12:0/3641379478 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55d0c21358d0 0x55d0c2137d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:20:26.070 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:26.060+0000 7fa165a00640 1 --2- 172.21.15.12:0/3641379478 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55d0c21358d0 0x55d0c2137d30 crc :-1 s=READY pgs=199 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ae3b9f0ddfa2e3f5 server_cookie=b43a4113e826d1d7 in_seq=0 out_seq=0 2024-09-11T00:20:26.070 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:26.060+0000 7fa147400640 10 client.26992 ms_handle_connect on v2:172.21.15.165:6838/501312984 2024-09-11T00:20:26.070 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:26.060+0000 7fa147400640 1 -- 172.21.15.12:0/3641379478 <== mds.0 v2:172.21.15.165:6838/501312984 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55d0c2095600 con 0x55d0c21358d0 2024-09-11T00:20:26.070 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:26.060+0000 7fa147400640 10 client.26992 handle_command_reply: tid=0 2024-09-11T00:20:26.070 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:26.060+0000 7fa16b220740 1 client.26992 shutdown 2024-09-11T00:20:26.071 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:26.060+0000 7fa16b220740 1 -- 172.21.15.12:0/3641379478 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d0c2125b00 msgr2=0x55d0c212b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:20:26.071 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:26.060+0000 7fa16b220740 1 --2- 172.21.15.12:0/3641379478 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d0c2125b00 0x55d0c212b0a0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fa158011db0 tx=0x7fa158004ab0 comp rx=0 tx=0).stop 2024-09-11T00:20:26.071 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:26.060+0000 7fa16b220740 1 -- 172.21.15.12:0/3641379478 shutdown_connections 2024-09-11T00:20:26.071 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:26.060+0000 7fa16b220740 1 --2- 172.21.15.12:0/3641379478 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55d0c21358d0 0x55d0c2137d30 crc :-1 s=READY pgs=199 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:20:26.071 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:26.060+0000 7fa16b220740 1 --2- 172.21.15.12:0/3641379478 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d0c2125b00 0x55d0c212b0a0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:20:26.071 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:26.060+0000 7fa16b220740 1 --2- 172.21.15.12:0/3641379478 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d0c21230c0 0x55d0c212a540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:20:26.071 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:26.060+0000 7fa16b220740 1 --2- 172.21.15.12:0/3641379478 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d0c2120790 0x55d0c2129ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:20:26.071 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:26.060+0000 7fa147400640 0 client.26992 ms_handle_reset on v2:172.21.15.165:6838/501312984 2024-09-11T00:20:26.071 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:26.060+0000 7fa16b220740 1 -- 172.21.15.12:0/3641379478 >> 172.21.15.12:0/3641379478 conn(0x55d0c211b8e0 msgr2=0x55d0c211aa60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:20:26.071 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:26.060+0000 7fa16b220740 1 -- 172.21.15.12:0/3641379478 shutdown_connections 2024-09-11T00:20:26.071 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:26.060+0000 7fa16b220740 1 -- 172.21.15.12:0/3641379478 wait complete. 2024-09-11T00:20:26.071 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:26.060+0000 7fa16b220740 20 client.26992 trim_cache size 0 max 16384 2024-09-11T00:20:26.078 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:26.072+0000 7fa16b220740 1 Processor -- start 2024-09-11T00:20:26.078 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:26.072+0000 7fa16b220740 1 -- start start 2024-09-11T00:20:26.079 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:26.072+0000 7fa16b220740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d0c2120790 0x55d0c211fc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:20:26.079 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:26.072+0000 7fa16b220740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d0c21230c0 0x55d0c2120160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:20:26.079 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:26.072+0000 7fa16b220740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d0c2125b00 0x55d0c211d710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:20:26.079 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:26.072+0000 7fa16b220740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fa1601a94d0 con 0x55d0c2120790 2024-09-11T00:20:26.079 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:26.072+0000 7fa16b220740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fa1601ab130 con 0x55d0c21230c0 2024-09-11T00:20:26.079 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:26.072+0000 7fa16b220740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fa1601a9ba0 con 0x55d0c2125b00 2024-09-11T00:20:26.081 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:26.077+0000 7fa165a00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d0c2120790 0x55d0c211fc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:20:26.081 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:26.077+0000 7fa165a00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d0c2120790 0x55d0c211fc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:45202/0 (socket says 172.21.15.12:45202) 2024-09-11T00:20:26.081 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:26.077+0000 7fa165a00640 1 -- 172.21.15.12:0/1291258155 learned_addr learned my addr 172.21.15.12:0/1291258155 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:20:26.081 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:26.077+0000 7fa165000640 1 --2- 172.21.15.12:0/1291258155 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d0c21230c0 0x55d0c2120160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:20:26.081 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:26.077+0000 7fa166400640 1 --2- 172.21.15.12:0/1291258155 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d0c2125b00 0x55d0c211d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:20:26.081 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:26.077+0000 7fa165a00640 1 -- 172.21.15.12:0/1291258155 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d0c2125b00 msgr2=0x55d0c211d710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:20:26.082 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:26.077+0000 7fa165a00640 1 --2- 172.21.15.12:0/1291258155 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d0c2125b00 0x55d0c211d710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:20:26.082 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:26.077+0000 7fa165a00640 1 -- 172.21.15.12:0/1291258155 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d0c21230c0 msgr2=0x55d0c2120160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:20:26.082 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:26.077+0000 7fa165a00640 1 --2- 172.21.15.12:0/1291258155 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d0c21230c0 0x55d0c2120160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:20:26.082 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:26.077+0000 7fa165a00640 1 -- 172.21.15.12:0/1291258155 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d0c2122ef0 con 0x55d0c2120790 2024-09-11T00:20:26.082 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:26.077+0000 7fa165000640 1 --2- 172.21.15.12:0/1291258155 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d0c21230c0 0x55d0c2120160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:20:26.082 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:26.077+0000 7fa165a00640 1 --2- 172.21.15.12:0/1291258155 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d0c2120790 0x55d0c211fc20 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fa154000c00 tx=0x7fa15400b4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b54de76945ff38f server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:20:26.082 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:26.077+0000 7fa152000640 1 -- 172.21.15.12:0/1291258155 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa154010d90 con 0x55d0c2120790 2024-09-11T00:20:26.082 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:26.077+0000 7fa152000640 1 -- 172.21.15.12:0/1291258155 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa154029020 con 0x55d0c2120790 2024-09-11T00:20:26.082 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:26.077+0000 7fa152000640 1 -- 172.21.15.12:0/1291258155 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa154011950 con 0x55d0c2120790 2024-09-11T00:20:26.082 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:26.077+0000 7fa16b220740 1 -- 172.21.15.12:0/1291258155 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d0c2120790 msgr2=0x55d0c211fc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:20:26.082 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:26.077+0000 7fa16b220740 1 --2- 172.21.15.12:0/1291258155 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d0c2120790 0x55d0c211fc20 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fa154000c00 tx=0x7fa15400b4a0 comp rx=0 tx=0).stop 2024-09-11T00:20:26.082 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:26.077+0000 7fa16b220740 1 -- 172.21.15.12:0/1291258155 shutdown_connections 2024-09-11T00:20:26.082 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:26.077+0000 7fa16b220740 1 --2- 172.21.15.12:0/1291258155 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d0c2125b00 0x55d0c211d710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:20:26.083 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:26.077+0000 7fa16b220740 1 --2- 172.21.15.12:0/1291258155 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d0c21230c0 0x55d0c2120160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:20:26.083 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:26.077+0000 7fa16b220740 1 --2- 172.21.15.12:0/1291258155 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d0c2120790 0x55d0c211fc20 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:20:26.083 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:26.077+0000 7fa16b220740 1 -- 172.21.15.12:0/1291258155 >> 172.21.15.12:0/1291258155 conn(0x55d0c211b8e0 msgr2=0x55d0c2125ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:20:26.083 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:26.077+0000 7fa16b220740 1 -- 172.21.15.12:0/1291258155 shutdown_connections 2024-09-11T00:20:26.083 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:26.077+0000 7fa16b220740 1 -- 172.21.15.12:0/1291258155 wait complete. 2024-09-11T00:20:26.083 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:26.077+0000 7fa16b220740 1 -- start start 2024-09-11T00:20:26.083 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:26.077+0000 7fa16b220740 1 Processor -- start 2024-09-11T00:20:26.084 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:20:25 smithi178 ceph-mon[35919]: pgmap v799: 129 pgs: 129 active+clean; 223 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 11 MiB/s wr, 570 op/s 2024-09-11T00:20:26.086 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:26.077+0000 7fa16b220740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d0c2120790 0x55d0c21319a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:20:26.086 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:26.077+0000 7fa16b220740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d0c21230c0 0x55d0c21306a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:20:26.086 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:26.077+0000 7fa16b220740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d0c2125b00 0x55d0c2130c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:20:26.086 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:26.077+0000 7fa16b220740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fa1601aaa60 con 0x55d0c21230c0 2024-09-11T00:20:26.086 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:26.077+0000 7fa16b220740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fa1601aa2d0 con 0x55d0c2125b00 2024-09-11T00:20:26.086 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:26.077+0000 7fa16b220740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fa1601a74b0 con 0x55d0c2120790 2024-09-11T00:20:26.086 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:26.077+0000 7fa16b220740 10 client.0 _ll_register_callbacks cb 0x7fa158009cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:20:26.086 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:26.077+0000 7fa165000640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d0c21230c0 0x55d0c21306a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:20:26.087 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:26.077+0000 7fa165000640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d0c21230c0 0x55d0c21306a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:45216/0 (socket says 172.21.15.12:45216) 2024-09-11T00:20:26.087 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:26.077+0000 7fa165000640 1 -- 172.21.15.12:0/2718052436 learned_addr learned my addr 172.21.15.12:0/2718052436 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:20:26.087 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:26.077+0000 7fa166400640 1 --2- 172.21.15.12:0/2718052436 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d0c2125b00 0x55d0c2130c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:20:26.087 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:26.077+0000 7fa165000640 1 -- 172.21.15.12:0/2718052436 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d0c2120790 msgr2=0x55d0c21319a0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-11T00:20:26.087 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:26.077+0000 7fa165000640 1 --2- 172.21.15.12:0/2718052436 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d0c2120790 0x55d0c21319a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:20:26.087 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:26.077+0000 7fa165000640 1 -- 172.21.15.12:0/2718052436 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d0c2125b00 msgr2=0x55d0c2130c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:20:26.087 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:26.077+0000 7fa165000640 1 --2- 172.21.15.12:0/2718052436 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d0c2125b00 0x55d0c2130c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:20:26.087 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:26.077+0000 7fa165000640 1 -- 172.21.15.12:0/2718052436 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d0c2131490 con 0x55d0c21230c0 2024-09-11T00:20:26.087 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:26.077+0000 7fa166400640 1 --2- 172.21.15.12:0/2718052436 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d0c2125b00 0x55d0c2130c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T00:20:26.087 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:26.077+0000 7fa165000640 1 --2- 172.21.15.12:0/2718052436 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d0c21230c0 0x55d0c21306a0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fa14c1126d0 tx=0x7fa14c041f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c13571f632c9ba5 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:20:26.087 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:26.077+0000 7fa147400640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-09-11T00:20:26.087 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:26.077+0000 7fa147400640 1 -- 172.21.15.12:0/2718052436 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa14c0d1120 con 0x55d0c21230c0 2024-09-11T00:20:26.087 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:26.077+0000 7fa147400640 1 -- 172.21.15.12:0/2718052436 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9208+0+0 (secure 0 0 0) 0x7fa14c0de380 con 0x55d0c21230c0 2024-09-11T00:20:26.088 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:26.077+0000 7fa16b220740 1 -- 172.21.15.12:0/2718052436 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa154029020 con 0x55d0c21230c0 2024-09-11T00:20:26.088 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:26.077+0000 7fa147400640 1 -- 172.21.15.12:0/2718052436 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) ==== 24+0+0 (secure 0 0 0) 0x7fa14c0d49f0 con 0x55d0c21230c0 2024-09-11T00:20:26.088 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:26.077+0000 7fa16b220740 10 client.17013 fetch_fsmap learned FSMap version 51 2024-09-11T00:20:26.088 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:26.077+0000 7fa16b220740 1 -- 172.21.15.12:0/2718052436 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=4+}) -- 0x55d0c21a29c0 con 0x55d0c21230c0 2024-09-11T00:20:26.088 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:26.077+0000 7fa147400640 1 -- 172.21.15.12:0/2718052436 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa154029020 con 0x55d0c21230c0 2024-09-11T00:20:26.088 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:26.077+0000 7fa147400640 1 -- 172.21.15.12:0/2718052436 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 51) ==== 8406+0+0 (secure 0 0 0) 0x7fa14c0f8070 con 0x55d0c21230c0 2024-09-11T00:20:26.088 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:26.077+0000 7fa16b220740 10 client.17013 fetch_fsmap finished waiting for FSMap version 51 2024-09-11T00:20:26.088 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:26.077+0000 7fa16b220740 10 client.17013 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T00:20:26.088 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:26.077+0000 7fa16b220740 20 client.17013 populate_metadata read hostname 'smithi012' 2024-09-11T00:20:26.088 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:26.077+0000 7fa16b220740 1 --2- 172.21.15.12:0/2718052436 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55d0c21a3460 0x55d0c21a5880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:20:26.088 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:26.077+0000 7fa16b220740 4 client.17013 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T00:20:26.088 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:26.077+0000 7fa16b220740 1 -- 172.21.15.12:0/2718052436 --> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55d0c20910b0 con 0x55d0c21a3460 2024-09-11T00:20:26.097 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:26.096+0000 7fa165a00640 1 --2- 172.21.15.12:0/2718052436 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55d0c21a3460 0x55d0c21a5880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:20:26.121 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:26.120+0000 7fa147400640 10 client.17013 ms_handle_connect on v2:172.21.15.165:6838/501312984 2024-09-11T00:20:26.121 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:26.120+0000 7fa165a00640 1 --2- 172.21.15.12:0/2718052436 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55d0c21a3460 0x55d0c21a5880 crc :-1 s=READY pgs=200 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4505fe0484b90833 server_cookie=b9739cce44abca9c in_seq=0 out_seq=0 2024-09-11T00:20:26.141 INFO:tasks.workunit.client.0.smithi012.stdout: 10 10001 31.96 MB/sec warmup 22 sec latency 3864.939 ms 2024-09-11T00:20:26.319 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:26.316+0000 7fa147400640 1 -- 172.21.15.12:0/2718052436 <== mds.0 v2:172.21.15.165:6838/501312984 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55d0c20910b0 con 0x55d0c21a3460 2024-09-11T00:20:26.319 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:26.316+0000 7fa147400640 10 client.17013 handle_command_reply: tid=0 2024-09-11T00:20:26.319 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:26.316+0000 7fa16b220740 1 client.17013 shutdown 2024-09-11T00:20:26.319 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:26.316+0000 7fa16b220740 1 -- 172.21.15.12:0/2718052436 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d0c21230c0 msgr2=0x55d0c21306a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:20:26.319 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:26.316+0000 7fa16b220740 1 --2- 172.21.15.12:0/2718052436 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d0c21230c0 0x55d0c21306a0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fa14c1126d0 tx=0x7fa14c041f70 comp rx=0 tx=0).stop 2024-09-11T00:20:26.320 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:26.316+0000 7fa16b220740 1 -- 172.21.15.12:0/2718052436 shutdown_connections 2024-09-11T00:20:26.320 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:26.316+0000 7fa16b220740 1 --2- 172.21.15.12:0/2718052436 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55d0c21a3460 0x55d0c21a5880 crc :-1 s=READY pgs=200 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:20:26.320 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:26.316+0000 7fa16b220740 1 --2- 172.21.15.12:0/2718052436 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d0c2125b00 0x55d0c2130c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:20:26.320 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:26.316+0000 7fa16b220740 1 --2- 172.21.15.12:0/2718052436 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d0c21230c0 0x55d0c21306a0 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:20:26.320 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:26.316+0000 7fa16b220740 1 --2- 172.21.15.12:0/2718052436 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d0c2120790 0x55d0c21319a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:20:26.320 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:26.316+0000 7fa147400640 0 client.17013 ms_handle_reset on v2:172.21.15.165:6838/501312984 2024-09-11T00:20:26.320 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:26.316+0000 7fa16b220740 1 -- 172.21.15.12:0/2718052436 >> 172.21.15.12:0/2718052436 conn(0x55d0c211b8e0 msgr2=0x55d0c211f7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:20:26.321 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:26.316+0000 7fa16b220740 1 -- 172.21.15.12:0/2718052436 shutdown_connections 2024-09-11T00:20:26.321 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:26.316+0000 7fa16b220740 1 -- 172.21.15.12:0/2718052436 wait complete. 2024-09-11T00:20:26.321 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:26.316+0000 7fa16b220740 20 client.17013 trim_cache size 0 max 16384 2024-09-11T00:20:26.322 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:{ 2024-09-11T00:20:26.322 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "message": "", 2024-09-11T00:20:26.322 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "return_code": 0 2024-09-11T00:20:26.322 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:} 2024-09-11T00:20:26.322 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:26.316+0000 7fa168400640 1 -- 172.21.15.12:0/2619626940 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fa1480b52e0 msgr2=0x7fa1480b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:20:26.322 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:26.316+0000 7fa168400640 1 --2- 172.21.15.12:0/2619626940 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fa1480b52e0 0x7fa1480b77a0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fa158005fd0 tx=0x7fa1580096a0 comp rx=0 tx=0).stop 2024-09-11T00:20:26.323 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:26.316+0000 7fa168400640 1 -- 172.21.15.12:0/2619626940 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa160103ad0 msgr2=0x7fa160075540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:20:26.323 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:26.316+0000 7fa168400640 1 --2- 172.21.15.12:0/2619626940 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa160103ad0 0x7fa160075540 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fa14c02a940 tx=0x7fa14c003c80 comp rx=0 tx=0).stop 2024-09-11T00:20:26.323 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:26.321+0000 7fa168400640 1 -- 172.21.15.12:0/2619626940 shutdown_connections 2024-09-11T00:20:26.323 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:26.321+0000 7fa168400640 1 --2- 172.21.15.12:0/2619626940 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fa1480b52e0 0x7fa1480b77a0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:20:26.323 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:26.321+0000 7fa168400640 1 --2- 172.21.15.12:0/2619626940 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa160104490 0x7fa160075a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:20:26.324 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:26.321+0000 7fa168400640 1 --2- 172.21.15.12:0/2619626940 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa160103ad0 0x7fa160075540 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:20:26.324 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:26.321+0000 7fa168400640 1 --2- 172.21.15.12:0/2619626940 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa160102980 0x7fa160076ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:20:26.324 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:26.321+0000 7fa168400640 1 -- 172.21.15.12:0/2619626940 >> 172.21.15.12:0/2619626940 conn(0x7fa1600fe240 msgr2=0x7fa160106ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:20:26.324 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:26.321+0000 7fa168400640 1 -- 172.21.15.12:0/2619626940 shutdown_connections 2024-09-11T00:20:26.324 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:26.321+0000 7fa168400640 1 -- 172.21.15.12:0/2619626940 wait complete. 2024-09-11T00:20:26.815 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:20:26 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T00:20:27.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:20:26 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T00:20:27.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:20:26 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T00:20:27.141 INFO:tasks.workunit.client.0.smithi012.stdout: 10 10006 30.58 MB/sec warmup 23 sec latency 4865.009 ms 2024-09-11T00:20:27.922 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:20:27 smithi012 ceph-mon[30333]: pgmap v800: 129 pgs: 129 active+clean; 378 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 26 MiB/s wr, 751 op/s 2024-09-11T00:20:27.922 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:20:27 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:20:27.922 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:20:27 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T00:20:27.922 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:20:27 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:20:28.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:20:27 smithi165 ceph-mon[35356]: pgmap v800: 129 pgs: 129 active+clean; 378 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 26 MiB/s wr, 751 op/s 2024-09-11T00:20:28.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:20:27 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:20:28.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:20:27 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T00:20:28.033 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:20:27 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:20:28.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:20:27 smithi178 ceph-mon[35919]: pgmap v800: 129 pgs: 129 active+clean; 378 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 26 MiB/s wr, 751 op/s 2024-09-11T00:20:28.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:20:27 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:20:28.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:20:27 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T00:20:28.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:20:27 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:20:28.141 INFO:tasks.workunit.client.0.smithi012.stdout: 10 10038 29.41 MB/sec warmup 24 sec latency 5865.050 ms 2024-09-11T00:20:29.141 INFO:tasks.workunit.client.0.smithi012.stdout: 10 10076 28.37 MB/sec warmup 25 sec latency 6730.190 ms 2024-09-11T00:20:29.872 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:20:29 smithi012 ceph-mon[30333]: pgmap v801: 129 pgs: 129 active+clean; 381 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1021 KiB/s rd, 25 MiB/s wr, 423 op/s 2024-09-11T00:20:30.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:20:29 smithi165 ceph-mon[35356]: pgmap v801: 129 pgs: 129 active+clean; 381 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1021 KiB/s rd, 25 MiB/s wr, 423 op/s 2024-09-11T00:20:30.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:20:29 smithi178 ceph-mon[35919]: pgmap v801: 129 pgs: 129 active+clean; 381 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1021 KiB/s rd, 25 MiB/s wr, 423 op/s 2024-09-11T00:20:30.141 INFO:tasks.workunit.client.0.smithi012.stdout: 10 10076 27.28 MB/sec warmup 26 sec latency 7730.255 ms 2024-09-11T00:20:31.141 INFO:tasks.workunit.client.0.smithi012.stdout: 10 10076 26.27 MB/sec warmup 27 sec latency 8730.297 ms 2024-09-11T00:20:31.778 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:20:31 smithi012 ceph-mon[30333]: pgmap v802: 129 pgs: 129 active+clean; 350 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 972 KiB/s rd, 24 MiB/s wr, 480 op/s 2024-09-11T00:20:32.033 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:20:31 smithi165 ceph-mon[35356]: pgmap v802: 129 pgs: 129 active+clean; 350 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 972 KiB/s rd, 24 MiB/s wr, 480 op/s 2024-09-11T00:20:32.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:20:31 smithi178 ceph-mon[35919]: pgmap v802: 129 pgs: 129 active+clean; 350 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 972 KiB/s rd, 24 MiB/s wr, 480 op/s 2024-09-11T00:20:32.142 INFO:tasks.workunit.client.0.smithi012.stdout: 10 10076 25.33 MB/sec warmup 28 sec latency 9730.356 ms 2024-09-11T00:20:33.142 INFO:tasks.workunit.client.0.smithi012.stdout: 10 10076 24.46 MB/sec warmup 29 sec latency 10730.415 ms 2024-09-11T00:20:33.809 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:20:33 smithi012 ceph-mon[30333]: pgmap v803: 129 pgs: 129 active+clean; 341 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 815 KiB/s rd, 19 MiB/s wr, 438 op/s 2024-09-11T00:20:34.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:20:33 smithi165 ceph-mon[35356]: pgmap v803: 129 pgs: 129 active+clean; 341 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 815 KiB/s rd, 19 MiB/s wr, 438 op/s 2024-09-11T00:20:34.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:20:33 smithi178 ceph-mon[35919]: pgmap v803: 129 pgs: 129 active+clean; 341 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 815 KiB/s rd, 19 MiB/s wr, 438 op/s 2024-09-11T00:20:34.142 INFO:tasks.workunit.client.0.smithi012.stdout: 10 10076 23.65 MB/sec warmup 30 sec latency 11730.475 ms 2024-09-11T00:20:35.142 INFO:tasks.workunit.client.0.smithi012.stdout: 10 10446 23.69 MB/sec warmup 31 sec latency 11848.330 ms 2024-09-11T00:20:35.813 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:20:35 smithi012 ceph-mon[30333]: pgmap v804: 129 pgs: 129 active+clean; 339 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 683 KiB/s rd, 17 MiB/s wr, 408 op/s 2024-09-11T00:20:36.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:20:35 smithi165 ceph-mon[35356]: pgmap v804: 129 pgs: 129 active+clean; 339 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 683 KiB/s rd, 17 MiB/s wr, 408 op/s 2024-09-11T00:20:36.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:20:35 smithi178 ceph-mon[35919]: pgmap v804: 129 pgs: 129 active+clean; 339 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 683 KiB/s rd, 17 MiB/s wr, 408 op/s 2024-09-11T00:20:36.142 INFO:tasks.workunit.client.0.smithi012.stdout: 10 11005 23.83 MB/sec warmup 32 sec latency 12524.092 ms 2024-09-11T00:20:37.142 INFO:tasks.workunit.client.0.smithi012.stdout: 10 12520 25.69 MB/sec warmup 33 sec latency 86.375 ms 2024-09-11T00:20:38.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:20:37 smithi165 ceph-mon[35356]: pgmap v805: 129 pgs: 129 active+clean; 322 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 763 KiB/s rd, 15 MiB/s wr, 452 op/s 2024-09-11T00:20:38.063 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:20:37 smithi012 ceph-mon[30333]: pgmap v805: 129 pgs: 129 active+clean; 322 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 763 KiB/s rd, 15 MiB/s wr, 452 op/s 2024-09-11T00:20:38.080 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:20:37 smithi178 ceph-mon[35919]: pgmap v805: 129 pgs: 129 active+clean; 322 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 763 KiB/s rd, 15 MiB/s wr, 452 op/s 2024-09-11T00:20:38.142 INFO:tasks.workunit.client.0.smithi012.stdout: 10 13510 26.62 MB/sec warmup 34 sec latency 41.663 ms 2024-09-11T00:20:39.143 INFO:tasks.workunit.client.0.smithi012.stdout: 10 14472 27.11 MB/sec warmup 35 sec latency 52.597 ms 2024-09-11T00:20:39.534 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:39.532+0000 7f5852400640 1 -- 172.21.15.12:0/3527737433 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f584c1036e0 msgr2=0x7f584c105b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:20:39.534 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:39.532+0000 7f5852400640 1 --2- 172.21.15.12:0/3527737433 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f584c1036e0 0x7f584c105b50 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f583c009860 tx=0x7f583c02a360 comp rx=0 tx=0).stop 2024-09-11T00:20:39.535 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:39.533+0000 7f5852400640 1 -- 172.21.15.12:0/3527737433 shutdown_connections 2024-09-11T00:20:39.535 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:39.533+0000 7f5852400640 1 --2- 172.21.15.12:0/3527737433 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f584c106290 0x7f584c106740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:20:39.535 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:39.533+0000 7f5852400640 1 --2- 172.21.15.12:0/3527737433 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f584c1036e0 0x7f584c105b50 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:20:39.535 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:39.533+0000 7f5852400640 1 --2- 172.21.15.12:0/3527737433 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f584c06b470 0x7f584c103150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:20:39.535 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:39.533+0000 7f5852400640 1 -- 172.21.15.12:0/3527737433 >> 172.21.15.12:0/3527737433 conn(0x7f584c0fcf30 msgr2=0x7f584c0ff370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:20:39.535 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:39.533+0000 7f5852400640 1 -- 172.21.15.12:0/3527737433 shutdown_connections 2024-09-11T00:20:39.535 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:39.533+0000 7f5852400640 1 -- 172.21.15.12:0/3527737433 wait complete. 2024-09-11T00:20:39.537 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:39.535+0000 7f5852400640 1 Processor -- start 2024-09-11T00:20:39.537 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:39.535+0000 7f5852400640 1 -- start start 2024-09-11T00:20:39.537 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:39.535+0000 7f5852400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f584c06b470 0x7f584c110fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:20:39.537 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:39.535+0000 7f5852400640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f584c1036e0 0x7f584c1114e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:20:39.537 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:39.535+0000 7f5852400640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f584c106290 0x7f584c10f8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:20:39.537 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:39.535+0000 7f5852400640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f584c108060 con 0x7f584c106290 2024-09-11T00:20:39.537 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:39.535+0000 7f5852400640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f584c107ee0 con 0x7f584c06b470 2024-09-11T00:20:39.538 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:39.535+0000 7f5852400640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f584c1081e0 con 0x7f584c1036e0 2024-09-11T00:20:39.538 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:39.535+0000 7f5851000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f584c06b470 0x7f584c110fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:20:39.538 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:39.536+0000 7f584be00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f584c1036e0 0x7f584c1114e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:20:39.538 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:39.536+0000 7f5851000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f584c06b470 0x7f584c110fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.12:46076/0 (socket says 172.21.15.12:46076) 2024-09-11T00:20:39.538 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:39.536+0000 7f5851000640 1 -- 172.21.15.12:0/1415241451 learned_addr learned my addr 172.21.15.12:0/1415241451 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:20:39.539 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:39.536+0000 7f5851a00640 1 --2- 172.21.15.12:0/1415241451 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f584c106290 0x7f584c10f8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:20:39.539 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:39.536+0000 7f584be00640 1 -- 172.21.15.12:0/1415241451 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f584c06b470 msgr2=0x7f584c110fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:20:39.539 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:39.536+0000 7f584be00640 1 --2- 172.21.15.12:0/1415241451 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f584c06b470 0x7f584c110fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:20:39.539 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:39.536+0000 7f584be00640 1 -- 172.21.15.12:0/1415241451 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f584c106290 msgr2=0x7f584c10f8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:20:39.539 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:39.536+0000 7f584be00640 1 --2- 172.21.15.12:0/1415241451 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f584c106290 0x7f584c10f8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:20:39.539 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:39.536+0000 7f584be00640 1 -- 172.21.15.12:0/1415241451 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f584c10ffd0 con 0x7f584c1036e0 2024-09-11T00:20:39.540 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:39.537+0000 7f584be00640 1 --2- 172.21.15.12:0/1415241451 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f584c1036e0 0x7f584c1114e0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f583c02a880 tx=0x7f583c032d50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:20:39.540 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:39.537+0000 7f5849600640 1 -- 172.21.15.12:0/1415241451 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f583c037050 con 0x7f584c1036e0 2024-09-11T00:20:39.540 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:39.537+0000 7f5849600640 1 -- 172.21.15.12:0/1415241451 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f583c038040 con 0x7f584c1036e0 2024-09-11T00:20:39.540 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:39.537+0000 7f5849600640 1 -- 172.21.15.12:0/1415241451 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f583c003b00 con 0x7f584c1036e0 2024-09-11T00:20:39.540 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:39.537+0000 7f5852400640 1 -- 172.21.15.12:0/1415241451 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f584c110260 con 0x7f584c1036e0 2024-09-11T00:20:39.540 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:39.537+0000 7f5852400640 1 -- 172.21.15.12:0/1415241451 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f584c1afe20 con 0x7f584c1036e0 2024-09-11T00:20:39.543 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:39.541+0000 7f5856a41740 1 Processor -- start 2024-09-11T00:20:39.545 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:39.543+0000 7f5856a41740 1 -- start start 2024-09-11T00:20:39.546 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:39.544+0000 7f5856a41740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5638e5efb970 0x5638e5efdd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:20:39.546 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:39.544+0000 7f5849600640 1 -- 172.21.15.12:0/1415241451 <== mon.1 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7f583c003ca0 con 0x7f584c1036e0 2024-09-11T00:20:39.546 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:39.544+0000 7f5851a00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5638e5efb970 0x5638e5efdd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:20:39.548 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:39.545+0000 7f5851a00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5638e5efb970 0x5638e5efdd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:38778/0 (socket says 172.21.15.12:38778) 2024-09-11T00:20:39.548 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:39.545+0000 7f5851a00640 1 -- 172.21.15.12:0/1358033544 learned_addr learned my addr 172.21.15.12:0/1358033544 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:20:39.548 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:39.545+0000 7f5856a41740 1 --2- 172.21.15.12:0/1358033544 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5638e5efe2a0 0x5638e5f00690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:20:39.548 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:39.545+0000 7f5856a41740 1 --2- 172.21.15.12:0/1358033544 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5638e5f00ce0 0x5638e5f031a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:20:39.548 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:39.545+0000 7f5856a41740 1 -- 172.21.15.12:0/1358033544 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x5638e5e6e6b0 con 0x5638e5efb970 2024-09-11T00:20:39.548 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:39.545+0000 7f5856a41740 1 -- 172.21.15.12:0/1358033544 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x5638e5e5dae0 con 0x5638e5f00ce0 2024-09-11T00:20:39.548 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:39.545+0000 7f5856a41740 1 -- 172.21.15.12:0/1358033544 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5638e5e618f0 con 0x5638e5efe2a0 2024-09-11T00:20:39.548 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:39.545+0000 7f5851000640 1 --2- 172.21.15.12:0/1358033544 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5638e5efe2a0 0x5638e5f00690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:20:39.548 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:39.545+0000 7f5851a00640 1 -- 172.21.15.12:0/1358033544 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5638e5efe2a0 msgr2=0x5638e5f00690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:20:39.548 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:39.545+0000 7f5851a00640 1 --2- 172.21.15.12:0/1358033544 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5638e5efe2a0 0x5638e5f00690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:20:39.548 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:39.545+0000 7f5851a00640 1 -- 172.21.15.12:0/1358033544 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5638e5f00ce0 msgr2=0x5638e5f031a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:20:39.549 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:39.545+0000 7f5851a00640 1 --2- 172.21.15.12:0/1358033544 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5638e5f00ce0 0x5638e5f031a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:20:39.549 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:39.545+0000 7f5851a00640 1 -- 172.21.15.12:0/1358033544 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5638e5e2a780 con 0x5638e5efb970 2024-09-11T00:20:39.549 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:39.545+0000 7f5851000640 1 --2- 172.21.15.12:0/1358033544 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5638e5efe2a0 0x5638e5f00690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:20:39.549 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:39.546+0000 7f5851a00640 1 --2- 172.21.15.12:0/1358033544 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5638e5efb970 0x5638e5efdd60 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f584400e7e0 tx=0x7f58440095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=351dfe9633f354ca server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:20:39.549 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:39.547+0000 7f583a000640 1 -- 172.21.15.12:0/1358033544 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5844003970 con 0x5638e5efb970 2024-09-11T00:20:39.550 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:39.547+0000 7f583a000640 1 -- 172.21.15.12:0/1358033544 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f584400d940 con 0x5638e5efb970 2024-09-11T00:20:39.550 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:39.547+0000 7f583a000640 1 -- 172.21.15.12:0/1358033544 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5844004570 con 0x5638e5efb970 2024-09-11T00:20:39.550 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:39.547+0000 7f5849600640 1 --2- 172.21.15.12:0/1415241451 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f58340b52e0 0x7f58340b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:20:39.550 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:39.547+0000 7f5849600640 1 -- 172.21.15.12:0/1415241451 <== mon.1 v2:172.21.15.178:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9208+0+0 (secure 0 0 0) 0x7f583c105e00 con 0x7f584c1036e0 2024-09-11T00:20:39.551 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:39.548+0000 7f5851000640 1 --2- 172.21.15.12:0/1415241451 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f58340b52e0 0x7f58340b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:20:39.551 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:39.548+0000 7f5856a41740 1 -- 172.21.15.12:0/1358033544 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5638e5efb970 msgr2=0x5638e5efdd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:20:39.551 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:39.548+0000 7f5856a41740 1 --2- 172.21.15.12:0/1358033544 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5638e5efb970 0x5638e5efdd60 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f584400e7e0 tx=0x7f58440095c0 comp rx=0 tx=0).stop 2024-09-11T00:20:39.552 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:39.550+0000 7f5851000640 1 --2- 172.21.15.12:0/1415241451 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f58340b52e0 0x7f58340b77a0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f5840005fd0 tx=0x7f584000b480 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:20:39.552 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:39.550+0000 7f5856a41740 1 -- 172.21.15.12:0/1358033544 shutdown_connections 2024-09-11T00:20:39.552 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:39.550+0000 7f5856a41740 1 --2- 172.21.15.12:0/1358033544 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5638e5f00ce0 0x5638e5f031a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:20:39.552 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:39.550+0000 7f5856a41740 1 --2- 172.21.15.12:0/1358033544 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5638e5efe2a0 0x5638e5f00690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:20:39.553 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:39.550+0000 7f5856a41740 1 --2- 172.21.15.12:0/1358033544 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5638e5efb970 0x5638e5efdd60 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:20:39.553 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:39.550+0000 7f5856a41740 1 -- 172.21.15.12:0/1358033544 >> 172.21.15.12:0/1358033544 conn(0x5638e5ef6ac0 msgr2=0x5638e5ef7850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:20:39.553 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:39.550+0000 7f5856a41740 1 -- 172.21.15.12:0/1358033544 shutdown_connections 2024-09-11T00:20:39.553 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:39.551+0000 7f5856a41740 1 -- 172.21.15.12:0/1358033544 wait complete. 2024-09-11T00:20:39.555 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:39.552+0000 7f5856a41740 1 -- start start 2024-09-11T00:20:39.555 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:39.553+0000 7f5856a41740 1 Processor -- start 2024-09-11T00:20:39.559 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:39.556+0000 7f5856a41740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5638e5efb970 0x5638e5f05190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:20:39.559 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:39.556+0000 7f5856a41740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5638e5efe2a0 0x5638e5f05830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:20:39.559 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:39.556+0000 7f5856a41740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5638e5f00ce0 0x5638e5f06400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:20:39.559 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:39.556+0000 7f5856a41740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x5638e5d95ab0 con 0x5638e5f00ce0 2024-09-11T00:20:39.559 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:39.556+0000 7f5856a41740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x5638e5e04570 con 0x5638e5efb970 2024-09-11T00:20:39.560 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:39.556+0000 7f5856a41740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5638e5f03cc0 con 0x5638e5efe2a0 2024-09-11T00:20:39.560 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:39.556+0000 7f5856a41740 10 client.0 _ll_register_callbacks cb 0x5638e5e13100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:20:39.560 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:39.557+0000 7f5851000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5638e5efb970 0x5638e5f05190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:20:39.560 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:39.557+0000 7f584be00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5638e5efe2a0 0x5638e5f05830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:20:39.560 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:39.557+0000 7f584be00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5638e5efe2a0 0x5638e5f05830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.12:46980/0 (socket says 172.21.15.12:46980) 2024-09-11T00:20:39.560 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:39.557+0000 7f584be00640 1 -- 172.21.15.12:0/2264676544 learned_addr learned my addr 172.21.15.12:0/2264676544 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:20:39.560 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:39.557+0000 7f584be00640 1 -- 172.21.15.12:0/2264676544 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5638e5efb970 msgr2=0x5638e5f05190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:20:39.560 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:39.557+0000 7f584be00640 1 --2- 172.21.15.12:0/2264676544 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5638e5efb970 0x5638e5f05190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:20:39.560 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:39.557+0000 7f584be00640 1 -- 172.21.15.12:0/2264676544 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5638e5f00ce0 msgr2=0x5638e5f06400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:20:39.560 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:39.557+0000 7f584be00640 1 --2- 172.21.15.12:0/2264676544 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5638e5f00ce0 0x5638e5f06400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:20:39.560 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:39.557+0000 7f584be00640 1 -- 172.21.15.12:0/2264676544 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5638e5bbf790 con 0x5638e5efe2a0 2024-09-11T00:20:39.560 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:39.557+0000 7f5851000640 1 --2- 172.21.15.12:0/2264676544 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5638e5efb970 0x5638e5f05190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T00:20:39.562 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:39.557+0000 7f584be00640 1 --2- 172.21.15.12:0/2264676544 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5638e5efe2a0 0x5638e5f05830 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f583c035040 tx=0x7f583c0c81f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a5257e6dec8b8c63 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:20:39.562 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:39.557+0000 7f582ea00640 10 client.0 ms_handle_connect on v2:172.21.15.178:3300/0 2024-09-11T00:20:39.562 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:39.558+0000 7f582ea00640 1 -- 172.21.15.12:0/2264676544 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f583c0d5060 con 0x5638e5efe2a0 2024-09-11T00:20:39.562 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:39.558+0000 7f582ea00640 1 -- 172.21.15.12:0/2264676544 <== mon.1 v2:172.21.15.178:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9208+0+0 (secure 0 0 0) 0x7f583c0d5290 con 0x5638e5efe2a0 2024-09-11T00:20:39.563 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:39.559+0000 7f5856a41740 1 -- 172.21.15.12:0/2264676544 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f584400d940 con 0x5638e5efe2a0 2024-09-11T00:20:39.563 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:39.559+0000 7f582ea00640 1 -- 172.21.15.12:0/2264676544 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) ==== 24+0+0 (secure 0 0 0) 0x7f583c0cbd30 con 0x5638e5efe2a0 2024-09-11T00:20:39.563 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:39.559+0000 7f5856a41740 10 client.27007 fetch_fsmap learned FSMap version 51 2024-09-11T00:20:39.563 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:39.559+0000 7f5856a41740 1 -- 172.21.15.12:0/2264676544 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=4+}) -- 0x5638e5c38620 con 0x5638e5efe2a0 2024-09-11T00:20:39.563 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:39.559+0000 7f582ea00640 1 -- 172.21.15.12:0/2264676544 <== mon.1 v2:172.21.15.178:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f584400d940 con 0x5638e5efe2a0 2024-09-11T00:20:39.563 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:39.560+0000 7f582ea00640 1 -- 172.21.15.12:0/2264676544 <== mon.1 v2:172.21.15.178:3300/0 5 ==== fsmap(e 51) ==== 8406+0+0 (secure 0 0 0) 0x7f583c0edb70 con 0x5638e5efe2a0 2024-09-11T00:20:39.563 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:39.560+0000 7f5856a41740 10 client.27007 fetch_fsmap finished waiting for FSMap version 51 2024-09-11T00:20:39.563 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:39.560+0000 7f5856a41740 10 client.27007 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T00:20:39.563 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:39.560+0000 7f5856a41740 20 client.27007 populate_metadata read hostname 'smithi012' 2024-09-11T00:20:39.563 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:39.560+0000 7f5856a41740 1 --2- 172.21.15.12:0/2264676544 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x5638e5f21a30 0x5638e5f23e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:20:39.563 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:39.560+0000 7f5856a41740 4 client.27007 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T00:20:39.563 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:39.560+0000 7f5856a41740 1 -- 172.21.15.12:0/2264676544 --> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5638e5e70600 con 0x5638e5f21a30 2024-09-11T00:20:39.564 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:39.561+0000 7f5851000640 1 --2- 172.21.15.12:0/2264676544 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x5638e5f21a30 0x5638e5f23e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:20:39.565 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:39.562+0000 7f5851000640 1 --2- 172.21.15.12:0/2264676544 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x5638e5f21a30 0x5638e5f23e90 crc :-1 s=READY pgs=201 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=75b54d27cce5868c server_cookie=b6f7afc0228ad65e in_seq=0 out_seq=0 2024-09-11T00:20:39.565 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:39.562+0000 7f582ea00640 10 client.27007 ms_handle_connect on v2:172.21.15.165:6838/501312984 2024-09-11T00:20:39.567 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:39.564+0000 7f582ea00640 1 -- 172.21.15.12:0/2264676544 <== mds.0 v2:172.21.15.165:6838/501312984 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5638e5e70600 con 0x5638e5f21a30 2024-09-11T00:20:39.567 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:39.564+0000 7f582ea00640 10 client.27007 handle_command_reply: tid=0 2024-09-11T00:20:39.567 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:39.564+0000 7f5856a41740 1 client.27007 shutdown 2024-09-11T00:20:39.568 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:39.566+0000 7f5856a41740 1 -- 172.21.15.12:0/2264676544 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5638e5efe2a0 msgr2=0x5638e5f05830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:20:39.568 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:39.566+0000 7f5856a41740 1 --2- 172.21.15.12:0/2264676544 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5638e5efe2a0 0x5638e5f05830 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f583c035040 tx=0x7f583c0c81f0 comp rx=0 tx=0).stop 2024-09-11T00:20:39.569 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:39.567+0000 7f5856a41740 1 -- 172.21.15.12:0/2264676544 shutdown_connections 2024-09-11T00:20:39.569 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:39.567+0000 7f5856a41740 1 --2- 172.21.15.12:0/2264676544 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x5638e5f21a30 0x5638e5f23e90 crc :-1 s=READY pgs=201 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:20:39.570 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:39.567+0000 7f5856a41740 1 --2- 172.21.15.12:0/2264676544 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5638e5f00ce0 0x5638e5f06400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:20:39.570 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:39.567+0000 7f5856a41740 1 --2- 172.21.15.12:0/2264676544 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5638e5efe2a0 0x5638e5f05830 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:20:39.570 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:39.567+0000 7f5856a41740 1 --2- 172.21.15.12:0/2264676544 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5638e5efb970 0x5638e5f05190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:20:39.570 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:39.567+0000 7f582ea00640 0 client.27007 ms_handle_reset on v2:172.21.15.165:6838/501312984 2024-09-11T00:20:39.570 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:39.567+0000 7f5856a41740 1 -- 172.21.15.12:0/2264676544 >> 172.21.15.12:0/2264676544 conn(0x5638e5ef6ac0 msgr2=0x5638e5ef5bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:20:39.570 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:39.568+0000 7f5856a41740 1 -- 172.21.15.12:0/2264676544 shutdown_connections 2024-09-11T00:20:39.571 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:39.568+0000 7f5856a41740 1 -- 172.21.15.12:0/2264676544 wait complete. 2024-09-11T00:20:39.571 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:39.569+0000 7f5856a41740 20 client.27007 trim_cache size 0 max 16384 2024-09-11T00:20:39.609 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:39.606+0000 7f5856a41740 1 Processor -- start 2024-09-11T00:20:39.609 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:39.606+0000 7f5856a41740 1 -- start start 2024-09-11T00:20:39.609 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:39.607+0000 7f5856a41740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5638e5efb970 0x5638e5f05190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:20:39.609 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:39.607+0000 7f5856a41740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5638e5efe2a0 0x5638e5ef7320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:20:39.609 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:39.607+0000 7f5856a41740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5638e5f00ce0 0x5638e5ef7860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:20:39.609 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:39.607+0000 7f5856a41740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f584c1b1ad0 con 0x5638e5efe2a0 2024-09-11T00:20:39.609 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:39.607+0000 7f5856a41740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f584c1b3730 con 0x5638e5f00ce0 2024-09-11T00:20:39.610 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:39.607+0000 7f5856a41740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f584c1b21a0 con 0x5638e5efb970 2024-09-11T00:20:39.610 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:39.607+0000 7f5851000640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5638e5efb970 0x5638e5f05190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:20:39.610 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:39.607+0000 7f5851000640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5638e5efb970 0x5638e5f05190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.12:46984/0 (socket says 172.21.15.12:46984) 2024-09-11T00:20:39.610 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:39.607+0000 7f5851000640 1 -- 172.21.15.12:0/2926106482 learned_addr learned my addr 172.21.15.12:0/2926106482 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:20:39.610 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:39.607+0000 7f5851a00640 1 --2- 172.21.15.12:0/2926106482 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5638e5f00ce0 0x5638e5ef7860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:20:39.611 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:39.608+0000 7f5851a00640 1 -- 172.21.15.12:0/2926106482 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5638e5efb970 msgr2=0x5638e5f05190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:20:39.611 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:39.608+0000 7f5851a00640 1 --2- 172.21.15.12:0/2926106482 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5638e5efb970 0x5638e5f05190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:20:39.611 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:39.608+0000 7f5851a00640 1 -- 172.21.15.12:0/2926106482 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5638e5efe2a0 msgr2=0x5638e5ef7320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:20:39.611 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:39.608+0000 7f5851a00640 1 --2- 172.21.15.12:0/2926106482 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5638e5efe2a0 0x5638e5ef7320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:20:39.611 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:39.608+0000 7f5851a00640 1 -- 172.21.15.12:0/2926106482 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5638e5f06770 con 0x5638e5f00ce0 2024-09-11T00:20:39.611 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:39.608+0000 7f5851a00640 1 --2- 172.21.15.12:0/2926106482 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5638e5f00ce0 0x5638e5ef7860 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f584400d7b0 tx=0x7f5844009da0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a933e47476ea61ed server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:20:39.611 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:39.609+0000 7f5839600640 1 -- 172.21.15.12:0/2926106482 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5844026020 con 0x5638e5f00ce0 2024-09-11T00:20:39.611 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:39.609+0000 7f5839600640 1 -- 172.21.15.12:0/2926106482 <== mon.2 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5844003e20 con 0x5638e5f00ce0 2024-09-11T00:20:39.611 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:39.609+0000 7f5839600640 1 -- 172.21.15.12:0/2926106482 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5844005320 con 0x5638e5f00ce0 2024-09-11T00:20:39.614 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:39.610+0000 7f5856a41740 1 -- 172.21.15.12:0/2926106482 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5638e5f00ce0 msgr2=0x5638e5ef7860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:20:39.614 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:39.610+0000 7f5856a41740 1 --2- 172.21.15.12:0/2926106482 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5638e5f00ce0 0x5638e5ef7860 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f584400d7b0 tx=0x7f5844009da0 comp rx=0 tx=0).stop 2024-09-11T00:20:39.614 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:39.610+0000 7f5856a41740 1 -- 172.21.15.12:0/2926106482 shutdown_connections 2024-09-11T00:20:39.614 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:39.610+0000 7f5856a41740 1 --2- 172.21.15.12:0/2926106482 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5638e5f00ce0 0x5638e5ef7860 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:20:39.614 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:39.610+0000 7f5856a41740 1 --2- 172.21.15.12:0/2926106482 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5638e5efe2a0 0x5638e5ef7320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:20:39.614 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:39.610+0000 7f5856a41740 1 --2- 172.21.15.12:0/2926106482 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5638e5efb970 0x5638e5f05190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:20:39.614 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:39.610+0000 7f5856a41740 1 -- 172.21.15.12:0/2926106482 >> 172.21.15.12:0/2926106482 conn(0x5638e5ef6ac0 msgr2=0x5638e5f04330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:20:39.614 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:39.611+0000 7f5856a41740 1 -- 172.21.15.12:0/2926106482 shutdown_connections 2024-09-11T00:20:39.614 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:39.612+0000 7f5856a41740 1 -- 172.21.15.12:0/2926106482 wait complete. 2024-09-11T00:20:39.615 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:39.612+0000 7f5856a41740 1 -- start start 2024-09-11T00:20:39.615 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:39.612+0000 7f5856a41740 1 Processor -- start 2024-09-11T00:20:39.616 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:39.613+0000 7f5856a41740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5638e5efb970 0x5638e5f0ca90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:20:39.616 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:39.614+0000 7f5856a41740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5638e5efe2a0 0x5638e5f0b790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:20:39.616 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:39.614+0000 7f5856a41740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5638e5f00ce0 0x5638e5f0bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:20:39.616 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:39.614+0000 7f5856a41740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f584c1b3060 con 0x5638e5efb970 2024-09-11T00:20:39.617 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:39.614+0000 7f5856a41740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f584c1b28d0 con 0x5638e5f00ce0 2024-09-11T00:20:39.617 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:39.614+0000 7f5856a41740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f584c074e80 con 0x5638e5efe2a0 2024-09-11T00:20:39.617 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:39.614+0000 7f5856a41740 10 client.0 _ll_register_callbacks cb 0x7f583c005760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:20:39.617 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:39.614+0000 7f584be00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5638e5efe2a0 0x5638e5f0b790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:20:39.617 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:39.614+0000 7f584be00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5638e5efe2a0 0x5638e5f0b790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.12:46986/0 (socket says 172.21.15.12:46986) 2024-09-11T00:20:39.617 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:39.614+0000 7f584be00640 1 -- 172.21.15.12:0/3560339594 learned_addr learned my addr 172.21.15.12:0/3560339594 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:20:39.617 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:39.614+0000 7f5851a00640 1 --2- 172.21.15.12:0/3560339594 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5638e5f00ce0 0x5638e5f0bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:20:39.617 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:39.615+0000 7f5851000640 1 --2- 172.21.15.12:0/3560339594 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5638e5efb970 0x5638e5f0ca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:20:39.618 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:39.615+0000 7f584be00640 1 -- 172.21.15.12:0/3560339594 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5638e5f00ce0 msgr2=0x5638e5f0bcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:20:39.618 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:39.615+0000 7f584be00640 1 --2- 172.21.15.12:0/3560339594 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5638e5f00ce0 0x5638e5f0bcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:20:39.618 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:39.615+0000 7f584be00640 1 -- 172.21.15.12:0/3560339594 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5638e5efb970 msgr2=0x5638e5f0ca90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:20:39.618 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:39.615+0000 7f584be00640 1 --2- 172.21.15.12:0/3560339594 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5638e5efb970 0x5638e5f0ca90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:20:39.618 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:39.615+0000 7f584be00640 1 -- 172.21.15.12:0/3560339594 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5638e5f0c370 con 0x5638e5efe2a0 2024-09-11T00:20:39.618 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:39.615+0000 7f5851000640 1 --2- 172.21.15.12:0/3560339594 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5638e5efb970 0x5638e5f0ca90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T00:20:39.619 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:39.616+0000 7f584be00640 1 --2- 172.21.15.12:0/3560339594 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5638e5efe2a0 0x5638e5f0b790 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f583c0cc1b0 tx=0x7f583c0f3260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=44bc2a07a7e749c5 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:20:39.619 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:39.616+0000 7f582ea00640 10 client.0 ms_handle_connect on v2:172.21.15.178:3300/0 2024-09-11T00:20:39.619 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:39.616+0000 7f582ea00640 1 -- 172.21.15.12:0/3560339594 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f583c0c8870 con 0x5638e5efe2a0 2024-09-11T00:20:39.619 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:39.616+0000 7f582ea00640 1 -- 172.21.15.12:0/3560339594 <== mon.1 v2:172.21.15.178:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9208+0+0 (secure 0 0 0) 0x7f583c0cb480 con 0x5638e5efe2a0 2024-09-11T00:20:39.619 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:39.616+0000 7f5856a41740 1 -- 172.21.15.12:0/3560339594 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5844003e20 con 0x5638e5efe2a0 2024-09-11T00:20:39.619 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:39.617+0000 7f582ea00640 1 -- 172.21.15.12:0/3560339594 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) ==== 24+0+0 (secure 0 0 0) 0x7f583c0cbb70 con 0x5638e5efe2a0 2024-09-11T00:20:39.619 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:39.617+0000 7f5856a41740 10 client.27013 fetch_fsmap learned FSMap version 51 2024-09-11T00:20:39.620 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:39.617+0000 7f5856a41740 1 -- 172.21.15.12:0/3560339594 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=4+}) -- 0x5638e5f8e960 con 0x5638e5efe2a0 2024-09-11T00:20:39.620 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:39.617+0000 7f582ea00640 1 -- 172.21.15.12:0/3560339594 <== mon.1 v2:172.21.15.178:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5844003e20 con 0x5638e5efe2a0 2024-09-11T00:20:39.620 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:39.618+0000 7f582ea00640 1 -- 172.21.15.12:0/3560339594 <== mon.1 v2:172.21.15.178:3300/0 5 ==== fsmap(e 51) ==== 8406+0+0 (secure 0 0 0) 0x7f583c0caef0 con 0x5638e5efe2a0 2024-09-11T00:20:39.620 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:39.618+0000 7f5856a41740 10 client.27013 fetch_fsmap finished waiting for FSMap version 51 2024-09-11T00:20:39.620 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:39.618+0000 7f5856a41740 10 client.27013 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T00:20:39.621 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:39.618+0000 7f5856a41740 20 client.27013 populate_metadata read hostname 'smithi012' 2024-09-11T00:20:39.621 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:39.618+0000 7f5856a41740 1 --2- 172.21.15.12:0/3560339594 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x5638e5f8f0d0 0x5638e5f914f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:20:39.621 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:39.618+0000 7f5856a41740 4 client.27013 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T00:20:39.621 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:39.618+0000 7f5856a41740 1 -- 172.21.15.12:0/3560339594 --> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5638e5ef2260 con 0x5638e5f8f0d0 2024-09-11T00:20:39.621 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:39.619+0000 7f5851000640 1 --2- 172.21.15.12:0/3560339594 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x5638e5f8f0d0 0x5638e5f914f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:20:39.621 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:39.619+0000 7f5851000640 1 --2- 172.21.15.12:0/3560339594 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x5638e5f8f0d0 0x5638e5f914f0 crc :-1 s=READY pgs=202 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e81f70beafcbd2cd server_cookie=b167fe22ff1e2248 in_seq=0 out_seq=0 2024-09-11T00:20:39.621 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:39.619+0000 7f582ea00640 10 client.27013 ms_handle_connect on v2:172.21.15.165:6838/501312984 2024-09-11T00:20:39.887 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:39.885+0000 7f582ea00640 1 -- 172.21.15.12:0/3560339594 <== mds.0 v2:172.21.15.165:6838/501312984 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5638e5ef2260 con 0x5638e5f8f0d0 2024-09-11T00:20:39.887 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:39.885+0000 7f582ea00640 10 client.27013 handle_command_reply: tid=0 2024-09-11T00:20:39.887 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:39.885+0000 7f5856a41740 1 client.27013 shutdown 2024-09-11T00:20:39.887 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:39.885+0000 7f5856a41740 1 -- 172.21.15.12:0/3560339594 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5638e5efe2a0 msgr2=0x5638e5f0b790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:20:39.888 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:39.885+0000 7f5856a41740 1 --2- 172.21.15.12:0/3560339594 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5638e5efe2a0 0x5638e5f0b790 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f583c0cc1b0 tx=0x7f583c0f3260 comp rx=0 tx=0).stop 2024-09-11T00:20:39.888 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:39.885+0000 7f5856a41740 1 -- 172.21.15.12:0/3560339594 shutdown_connections 2024-09-11T00:20:39.888 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:39.885+0000 7f5856a41740 1 --2- 172.21.15.12:0/3560339594 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x5638e5f8f0d0 0x5638e5f914f0 crc :-1 s=READY pgs=202 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:20:39.888 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:39.885+0000 7f5856a41740 1 --2- 172.21.15.12:0/3560339594 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5638e5f00ce0 0x5638e5f0bcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:20:39.888 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:39.885+0000 7f5856a41740 1 --2- 172.21.15.12:0/3560339594 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5638e5efe2a0 0x5638e5f0b790 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:20:39.888 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:39.885+0000 7f582ea00640 0 client.27013 ms_handle_reset on v2:172.21.15.165:6838/501312984 2024-09-11T00:20:39.888 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:39.885+0000 7f5856a41740 1 --2- 172.21.15.12:0/3560339594 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5638e5efb970 0x5638e5f0ca90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:20:39.888 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:39.885+0000 7f5856a41740 1 -- 172.21.15.12:0/3560339594 >> 172.21.15.12:0/3560339594 conn(0x5638e5ef6ac0 msgr2=0x5638e5f04330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:20:39.889 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:39.886+0000 7f5856a41740 1 -- 172.21.15.12:0/3560339594 shutdown_connections 2024-09-11T00:20:39.889 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:39.886+0000 7f5856a41740 1 -- 172.21.15.12:0/3560339594 wait complete. 2024-09-11T00:20:39.889 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:39.886+0000 7f5856a41740 20 client.27013 trim_cache size 0 max 16384 2024-09-11T00:20:39.889 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:{ 2024-09-11T00:20:39.889 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "message": "", 2024-09-11T00:20:39.889 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "return_code": 0 2024-09-11T00:20:39.890 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:} 2024-09-11T00:20:39.890 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:39.887+0000 7f5852400640 1 -- 172.21.15.12:0/1415241451 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f58340b52e0 msgr2=0x7f58340b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:20:39.890 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:39.887+0000 7f5852400640 1 --2- 172.21.15.12:0/1415241451 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f58340b52e0 0x7f58340b77a0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f5840005fd0 tx=0x7f584000b480 comp rx=0 tx=0).stop 2024-09-11T00:20:39.890 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:39.887+0000 7f5852400640 1 -- 172.21.15.12:0/1415241451 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f584c1036e0 msgr2=0x7f584c1114e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:20:39.890 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:39.887+0000 7f5852400640 1 --2- 172.21.15.12:0/1415241451 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f584c1036e0 0x7f584c1114e0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f583c02a880 tx=0x7f583c032d50 comp rx=0 tx=0).stop 2024-09-11T00:20:39.890 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:39.887+0000 7f5852400640 1 -- 172.21.15.12:0/1415241451 shutdown_connections 2024-09-11T00:20:39.891 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:39.887+0000 7f5852400640 1 --2- 172.21.15.12:0/1415241451 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f58340b52e0 0x7f58340b77a0 secure :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f5840005fd0 tx=0x7f584000b480 comp rx=0 tx=0).stop 2024-09-11T00:20:39.891 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:39.887+0000 7f5852400640 1 --2- 172.21.15.12:0/1415241451 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f584c106290 0x7f584c10f8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:20:39.891 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:39.887+0000 7f5852400640 1 --2- 172.21.15.12:0/1415241451 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f584c1036e0 0x7f584c1114e0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:20:39.891 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:39.887+0000 7f5852400640 1 --2- 172.21.15.12:0/1415241451 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f584c06b470 0x7f584c110fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:20:39.891 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:39.887+0000 7f5852400640 1 -- 172.21.15.12:0/1415241451 >> 172.21.15.12:0/1415241451 conn(0x7f584c0fcf30 msgr2=0x7f584c0ff320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:20:39.891 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:39.887+0000 7f5852400640 1 -- 172.21.15.12:0/1415241451 shutdown_connections 2024-09-11T00:20:39.891 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:39.887+0000 7f5852400640 1 -- 172.21.15.12:0/1415241451 wait complete. 2024-09-11T00:20:40.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:20:39 smithi165 ceph-mon[35356]: pgmap v806: 129 pgs: 129 active+clean; 323 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 426 KiB/s rd, 938 KiB/s wr, 245 op/s 2024-09-11T00:20:40.063 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:20:39 smithi012 ceph-mon[30333]: pgmap v806: 129 pgs: 129 active+clean; 323 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 426 KiB/s rd, 938 KiB/s wr, 245 op/s 2024-09-11T00:20:40.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:20:39 smithi178 ceph-mon[35919]: pgmap v806: 129 pgs: 129 active+clean; 323 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 426 KiB/s rd, 938 KiB/s wr, 245 op/s 2024-09-11T00:20:40.143 INFO:tasks.workunit.client.0.smithi012.stdout: 10 15783 28.46 MB/sec warmup 36 sec latency 121.515 ms 2024-09-11T00:20:41.143 INFO:tasks.workunit.client.0.smithi012.stdout: 10 16600 28.81 MB/sec warmup 37 sec latency 71.703 ms 2024-09-11T00:20:42.033 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:20:41 smithi165 ceph-mon[35356]: pgmap v807: 129 pgs: 129 active+clean; 335 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.7 MiB/s wr, 369 op/s 2024-09-11T00:20:42.033 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:20:41 smithi165 ceph-mon[35356]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T00:20:42.063 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:20:41 smithi012 ceph-mon[30333]: pgmap v807: 129 pgs: 129 active+clean; 335 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.7 MiB/s wr, 369 op/s 2024-09-11T00:20:42.063 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:20:41 smithi012 ceph-mon[30333]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T00:20:42.080 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:20:41 smithi178 ceph-mon[35919]: pgmap v807: 129 pgs: 129 active+clean; 335 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.7 MiB/s wr, 369 op/s 2024-09-11T00:20:42.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:20:41 smithi178 ceph-mon[35919]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T00:20:42.143 INFO:tasks.workunit.client.0.smithi012.stdout: 10 17614 29.46 MB/sec warmup 38 sec latency 78.112 ms 2024-09-11T00:20:43.143 INFO:tasks.workunit.client.0.smithi012.stdout: 10 18865 30.38 MB/sec warmup 39 sec latency 42.767 ms 2024-09-11T00:20:44.010 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:20:43 smithi012 ceph-mon[30333]: pgmap v808: 129 pgs: 129 active+clean; 345 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 3.0 MiB/s wr, 495 op/s 2024-09-11T00:20:44.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:20:43 smithi165 ceph-mon[35356]: pgmap v808: 129 pgs: 129 active+clean; 345 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 3.0 MiB/s wr, 495 op/s 2024-09-11T00:20:44.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:20:43 smithi178 ceph-mon[35919]: pgmap v808: 129 pgs: 129 active+clean; 345 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 3.0 MiB/s wr, 495 op/s 2024-09-11T00:20:44.143 INFO:tasks.workunit.client.0.smithi012.stdout: 10 19587 30.75 MB/sec warmup 40 sec latency 355.116 ms 2024-09-11T00:20:45.143 INFO:tasks.workunit.client.0.smithi012.stdout: 10 20456 31.04 MB/sec warmup 41 sec latency 61.654 ms 2024-09-11T00:20:46.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:20:45 smithi165 ceph-mon[35356]: pgmap v809: 129 pgs: 129 active+clean; 348 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 3.1 MiB/s wr, 489 op/s 2024-09-11T00:20:46.063 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:20:45 smithi012 ceph-mon[30333]: pgmap v809: 129 pgs: 129 active+clean; 348 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 3.1 MiB/s wr, 489 op/s 2024-09-11T00:20:46.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:20:45 smithi178 ceph-mon[35919]: pgmap v809: 129 pgs: 129 active+clean; 348 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 3.1 MiB/s wr, 489 op/s 2024-09-11T00:20:46.143 INFO:tasks.workunit.client.0.smithi012.stdout: 10 21598 31.81 MB/sec warmup 42 sec latency 62.904 ms 2024-09-11T00:20:47.143 INFO:tasks.workunit.client.0.smithi012.stdout: 10 22593 32.26 MB/sec warmup 43 sec latency 60.334 ms 2024-09-11T00:20:47.999 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:20:47 smithi012 ceph-mon[30333]: pgmap v810: 129 pgs: 129 active+clean; 364 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 4.9 MiB/s wr, 731 op/s 2024-09-11T00:20:48.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:20:47 smithi165 ceph-mon[35356]: pgmap v810: 129 pgs: 129 active+clean; 364 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 4.9 MiB/s wr, 731 op/s 2024-09-11T00:20:48.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:20:47 smithi178 ceph-mon[35919]: pgmap v810: 129 pgs: 129 active+clean; 364 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 4.9 MiB/s wr, 731 op/s 2024-09-11T00:20:48.143 INFO:tasks.workunit.client.0.smithi012.stdout: 10 23437 32.56 MB/sec warmup 44 sec latency 100.416 ms 2024-09-11T00:20:49.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:20:48 smithi165 ceph-mon[35356]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T00:20:49.063 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:20:48 smithi012 ceph-mon[30333]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T00:20:49.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:20:48 smithi178 ceph-mon[35919]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T00:20:49.143 INFO:tasks.workunit.client.0.smithi012.stdout: 10 24183 32.79 MB/sec warmup 45 sec latency 323.364 ms 2024-09-11T00:20:50.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:20:49 smithi165 ceph-mon[35356]: pgmap v811: 129 pgs: 129 active+clean; 365 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 4.8 MiB/s wr, 674 op/s 2024-09-11T00:20:50.040 INFO:tasks.quiescer.fs.[cephfs]:Total duration (45.5) is longer than `split_if_longer` (35.0), will split into two consecutive quiesces 2024-09-11T00:20:50.040 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 22.75 2024-09-11T00:20:50.041 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid ff3ad74a-6fcf-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T00:20:50.064 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:20:49 smithi012 ceph-mon[30333]: pgmap v811: 129 pgs: 129 active+clean; 365 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 4.8 MiB/s wr, 674 op/s 2024-09-11T00:20:50.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:20:49 smithi178 ceph-mon[35919]: pgmap v811: 129 pgs: 129 active+clean; 365 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 4.8 MiB/s wr, 674 op/s 2024-09-11T00:20:50.143 INFO:tasks.workunit.client.0.smithi012.stdout: 10 25297 33.29 MB/sec warmup 46 sec latency 327.341 ms 2024-09-11T00:20:50.688 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:20:51.143 INFO:tasks.workunit.client.0.smithi012.stdout: 10 26361 34.04 MB/sec warmup 47 sec latency 40.081 ms 2024-09-11T00:20:51.508 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:20:51.505+0000 7fa78f600640 1 -- 172.21.15.12:0/1880495102 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa788104f90 msgr2=0x7fa788105370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:20:51.508 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:20:51.505+0000 7fa78f600640 1 --2- 172.21.15.12:0/1880495102 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa788104f90 0x7fa788105370 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fa774009900 tx=0x7fa77402a410 comp rx=0 tx=0).stop 2024-09-11T00:20:51.508 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:20:51.505+0000 7fa78f600640 1 -- 172.21.15.12:0/1880495102 shutdown_connections 2024-09-11T00:20:51.508 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:20:51.505+0000 7fa78f600640 1 --2- 172.21.15.12:0/1880495102 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa78810b320 0x7fa788111b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:20:51.508 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:20:51.505+0000 7fa78f600640 1 --2- 172.21.15.12:0/1880495102 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa78810a9a0 0x7fa78810ade0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:20:51.508 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:20:51.505+0000 7fa78f600640 1 --2- 172.21.15.12:0/1880495102 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa788104f90 0x7fa788105370 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:20:51.509 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:20:51.505+0000 7fa78f600640 1 -- 172.21.15.12:0/1880495102 >> 172.21.15.12:0/1880495102 conn(0x7fa788100a00 msgr2=0x7fa788102e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:20:51.509 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:20:51.506+0000 7fa78f600640 1 -- 172.21.15.12:0/1880495102 shutdown_connections 2024-09-11T00:20:51.509 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:20:51.506+0000 7fa78f600640 1 -- 172.21.15.12:0/1880495102 wait complete. 2024-09-11T00:20:51.510 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:20:51.508+0000 7fa78f600640 1 Processor -- start 2024-09-11T00:20:51.510 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:20:51.508+0000 7fa78f600640 1 -- start start 2024-09-11T00:20:51.511 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:20:51.508+0000 7fa78f600640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa788104f90 0x7fa788074780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:20:51.511 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:20:51.508+0000 7fa78f600640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa78810a9a0 0x7fa788074cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:20:51.511 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:20:51.508+0000 7fa78f600640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa78810b320 0x7fa78806eaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:20:51.511 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:20:51.508+0000 7fa78f600640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fa78806d470 con 0x7fa78810b320 2024-09-11T00:20:51.511 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:20:51.508+0000 7fa78f600640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fa78806d2f0 con 0x7fa78810a9a0 2024-09-11T00:20:51.511 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:20:51.508+0000 7fa78f600640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fa78806d5f0 con 0x7fa788104f90 2024-09-11T00:20:51.512 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:20:51.509+0000 7fa787e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa78810a9a0 0x7fa788074cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:20:51.512 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:20:51.509+0000 7fa787e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa78810a9a0 0x7fa788074cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.12:47710/0 (socket says 172.21.15.12:47710) 2024-09-11T00:20:51.512 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:20:51.509+0000 7fa787e00640 1 -- 172.21.15.12:0/1882996674 learned_addr learned my addr 172.21.15.12:0/1882996674 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:20:51.513 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:20:51.509+0000 7fa787e00640 1 -- 172.21.15.12:0/1882996674 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa788104f90 msgr2=0x7fa788074780 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-11T00:20:51.513 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:20:51.509+0000 7fa787e00640 1 --2- 172.21.15.12:0/1882996674 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa788104f90 0x7fa788074780 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:20:51.513 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:20:51.509+0000 7fa787e00640 1 -- 172.21.15.12:0/1882996674 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa78810b320 msgr2=0x7fa78806eaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:20:51.513 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:20:51.509+0000 7fa787e00640 1 --2- 172.21.15.12:0/1882996674 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa78810b320 0x7fa78806eaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:20:51.513 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:20:51.509+0000 7fa787e00640 1 -- 172.21.15.12:0/1882996674 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa78806f360 con 0x7fa78810a9a0 2024-09-11T00:20:51.513 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:20:51.510+0000 7fa787e00640 1 --2- 172.21.15.12:0/1882996674 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa78810a9a0 0x7fa788074cc0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fa77800c920 tx=0x7fa77800cde0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:20:51.513 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:20:51.510+0000 7fa785600640 1 -- 172.21.15.12:0/1882996674 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa778007c70 con 0x7fa78810a9a0 2024-09-11T00:20:51.513 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:20:51.510+0000 7fa78f600640 1 -- 172.21.15.12:0/1882996674 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa78806f650 con 0x7fa78810a9a0 2024-09-11T00:20:51.514 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:20:51.511+0000 7fa785600640 1 -- 172.21.15.12:0/1882996674 <== mon.2 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa778007e10 con 0x7fa78810a9a0 2024-09-11T00:20:51.514 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:20:51.511+0000 7fa785600640 1 -- 172.21.15.12:0/1882996674 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa778018820 con 0x7fa78810a9a0 2024-09-11T00:20:51.514 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:20:51.511+0000 7fa78f600640 1 -- 172.21.15.12:0/1882996674 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7880777c0 con 0x7fa78810a9a0 2024-09-11T00:20:51.516 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:20:51.514+0000 7fa78f600640 1 -- 172.21.15.12:0/1882996674 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa788105370 con 0x7fa78810a9a0 2024-09-11T00:20:51.518 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:20:51.515+0000 7fa785600640 1 -- 172.21.15.12:0/1882996674 <== mon.2 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7fa77800f450 con 0x7fa78810a9a0 2024-09-11T00:20:51.520 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:20:51.517+0000 7fa785600640 1 --2- 172.21.15.12:0/1882996674 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fa76c0b5360 0x7fa76c0b7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:20:51.520 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:20:51.517+0000 7fa785600640 1 -- 172.21.15.12:0/1882996674 <== mon.2 v2:172.21.15.165:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9208+0+0 (secure 0 0 0) 0x7fa7780e61c0 con 0x7fa78810a9a0 2024-09-11T00:20:51.520 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:20:51.518+0000 7fa78cc00640 1 --2- 172.21.15.12:0/1882996674 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fa76c0b5360 0x7fa76c0b7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:20:51.521 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:20:51.519+0000 7fa78cc00640 1 --2- 172.21.15.12:0/1882996674 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fa76c0b5360 0x7fa76c0b7820 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fa7740096c0 tx=0x7fa774005fd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:20:51.526 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:20:51.523+0000 7fa785600640 1 -- 172.21.15.12:0/1882996674 <== mon.2 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7fa778014030 con 0x7fa78810a9a0 2024-09-11T00:20:51.970 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:20:51.967+0000 7fa78f600640 1 -- 172.21.15.12:0/1882996674 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa788077c50 con 0x7fa78810a9a0 2024-09-11T00:20:51.971 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:20:51.969+0000 7fa785600640 1 -- 172.21.15.12:0/1882996674 <== mon.2 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 53 v53) ==== 94+0+11984 (secure 0 0 0) 0x7fa7780acc50 con 0x7fa78810a9a0 2024-09-11T00:20:51.976 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-11T00:20:51.976 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":53,"btime":"2024-09-11T00:20:47:743860+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14577,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6835/447171591","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":447171591},{"type":"v1","addr":"172.21.15.12:6835","nonce":447171591}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14589,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/901486040","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":901486040},{"type":"v1","addr":"172.21.15.12:6837","nonce":901486040}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":53,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T00:04:04.739881+0000","modified":"2024-09-11T00:20:47.231733+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24674,"mds_1":24643,"mds_2":24638,"mds_3":24637,"mds_4":24614},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14631":{"gid":14631,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.12:6841/3069919437","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6840","nonce":3069919437},{"type":"v1","addr":"172.21.15.12:6841","nonce":3069919437}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24562":{"gid":24562,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6833/1141636112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":1141636112},{"type":"v1","addr":"172.21.15.178:6833","nonce":1141636112}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24581":{"gid":24581,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6833/3111980970","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6832","nonce":3111980970},{"type":"v1","addr":"172.21.15.165:6833","nonce":3111980970}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24607":{"gid":24607,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6835/4084868287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":4084868287},{"type":"v1","addr":"172.21.15.178:6835","nonce":4084868287}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24614":{"gid":24614,"name":"e","rank":4,"incarnation":26,"state":"up:active","state_seq":21,"addr":"172.21.15.165:6835/2425395274","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":2425395274},{"type":"v1","addr":"172.21.15.165:6835","nonce":2425395274}]},"join_fscid":-1,"export_targets":[0,1],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24637":{"gid":24637,"name":"i","rank":3,"incarnation":24,"state":"up:active","state_seq":18,"addr":"172.21.15.178:6839/70628097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6838","nonce":70628097},{"type":"v1","addr":"172.21.15.178:6839","nonce":70628097}]},"join_fscid":-1,"export_targets":[0],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24638":{"gid":24638,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":18,"addr":"172.21.15.165:6837/6215088","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6836","nonce":6215088},{"type":"v1","addr":"172.21.15.165:6837","nonce":6215088}]},"join_fscid":-1,"export_targets":[0],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24643":{"gid":24643,"name":"g","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.12:6839/3124093009","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6838","nonce":3124093009},{"type":"v1","addr":"172.21.15.12:6839","nonce":3124093009}]},"join_fscid":-1,"export_targets":[0,2,3,4],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24674":{"gid":24674,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.165:6839/501312984","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6838","nonce":501312984},{"type":"v1","addr":"172.21.15.165:6839","nonce":501312984}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25672":{"gid":25672,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6837/308601741","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":308601741},{"type":"v1","addr":"172.21.15.178:6837","nonce":308601741}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24674,"qdb_cluster":[24614,24637,24638,24643,24674]},"id":1}]} 2024-09-11T00:20:51.978 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 53 2024-09-11T00:20:51.978 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:20:51.972+0000 7fa78f600640 1 -- 172.21.15.12:0/1882996674 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fa76c0b5360 msgr2=0x7fa76c0b7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:20:51.978 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:20:51.972+0000 7fa78f600640 1 --2- 172.21.15.12:0/1882996674 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fa76c0b5360 0x7fa76c0b7820 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fa7740096c0 tx=0x7fa774005fd0 comp rx=0 tx=0).stop 2024-09-11T00:20:51.978 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:20:51.972+0000 7fa78f600640 1 -- 172.21.15.12:0/1882996674 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa78810a9a0 msgr2=0x7fa788074cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:20:51.978 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:20:51.972+0000 7fa78f600640 1 --2- 172.21.15.12:0/1882996674 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa78810a9a0 0x7fa788074cc0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fa77800c920 tx=0x7fa77800cde0 comp rx=0 tx=0).stop 2024-09-11T00:20:51.978 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:20:51.973+0000 7fa78f600640 1 -- 172.21.15.12:0/1882996674 shutdown_connections 2024-09-11T00:20:51.978 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:20:51.973+0000 7fa78f600640 1 --2- 172.21.15.12:0/1882996674 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fa76c0b5360 0x7fa76c0b7820 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:20:51.978 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:20:51.973+0000 7fa78f600640 1 --2- 172.21.15.12:0/1882996674 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa78810b320 0x7fa78806eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:20:51.978 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:20:51.973+0000 7fa78f600640 1 --2- 172.21.15.12:0/1882996674 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa78810a9a0 0x7fa788074cc0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:20:51.978 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:20:51.973+0000 7fa78f600640 1 --2- 172.21.15.12:0/1882996674 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa788104f90 0x7fa788074780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:20:51.979 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:20:51.973+0000 7fa78f600640 1 -- 172.21.15.12:0/1882996674 >> 172.21.15.12:0/1882996674 conn(0x7fa788100a00 msgr2=0x7fa788101130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:20:51.979 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:20:51.974+0000 7fa78f600640 1 -- 172.21.15.12:0/1882996674 shutdown_connections 2024-09-11T00:20:51.979 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:20:51.975+0000 7fa78f600640 1 -- 172.21.15.12:0/1882996674 wait complete. 2024-09-11T00:20:52.063 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:20:51 smithi012 ceph-mon[30333]: pgmap v812: 129 pgs: 129 active+clean; 376 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 6.2 MiB/s wr, 810 op/s 2024-09-11T00:20:52.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:20:51 smithi178 ceph-mon[35919]: pgmap v812: 129 pgs: 129 active+clean; 376 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 6.2 MiB/s wr, 810 op/s 2024-09-11T00:20:52.143 INFO:tasks.workunit.client.0.smithi012.stdout: 10 27199 33.99 MB/sec warmup 48 sec latency 86.149 ms 2024-09-11T00:20:52.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:20:51 smithi165 ceph-mon[35356]: pgmap v812: 129 pgs: 129 active+clean; 376 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 6.2 MiB/s wr, 810 op/s 2024-09-11T00:20:53.046 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:20:52 smithi012 ceph-mon[30333]: from='client.? 172.21.15.12:0/1882996674' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T00:20:53.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:20:52 smithi178 ceph-mon[35919]: from='client.? 172.21.15.12:0/1882996674' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T00:20:53.103 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:53.101+0000 7f01d6000640 1 -- 172.21.15.12:0/1569395451 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f01c8095540 msgr2=0x7f01c8095940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:20:53.103 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:53.101+0000 7f01d6000640 1 --2- 172.21.15.12:0/1569395451 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f01c8095540 0x7f01c8095940 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f01bc009910 tx=0x7f01bc02a3f0 comp rx=0 tx=0).stop 2024-09-11T00:20:53.106 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:53.104+0000 7f01d6000640 1 -- 172.21.15.12:0/1569395451 shutdown_connections 2024-09-11T00:20:53.106 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:53.104+0000 7f01d6000640 1 --2- 172.21.15.12:0/1569395451 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f01c8097050 0x7f01c809d9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:20:53.106 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:53.104+0000 7f01d6000640 1 --2- 172.21.15.12:0/1569395451 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f01c8096690 0x7f01c8096b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:20:53.106 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:53.104+0000 7f01d6000640 1 --2- 172.21.15.12:0/1569395451 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f01c8095540 0x7f01c8095940 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:20:53.106 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:53.104+0000 7f01d6000640 1 -- 172.21.15.12:0/1569395451 >> 172.21.15.12:0/1569395451 conn(0x7f01c8090de0 msgr2=0x7f01c8093220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:20:53.107 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:53.104+0000 7f01d6000640 1 -- 172.21.15.12:0/1569395451 shutdown_connections 2024-09-11T00:20:53.107 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:53.105+0000 7f01d6000640 1 -- 172.21.15.12:0/1569395451 wait complete. 2024-09-11T00:20:53.110 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:53.108+0000 7f01d6000640 1 Processor -- start 2024-09-11T00:20:53.110 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:53.109+0000 7f01d6000640 1 -- start start 2024-09-11T00:20:53.111 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:53.109+0000 7f01d6000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f01c8095540 0x7f01c812da60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:20:53.112 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:53.109+0000 7f01d6000640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f01c8096690 0x7f01c812e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:20:53.112 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:53.109+0000 7f01d6000640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f01c8097050 0x7f01c812ec70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:20:53.112 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:53.109+0000 7f01d6000640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f01c80a1210 con 0x7f01c8097050 2024-09-11T00:20:53.112 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:53.109+0000 7f01d6000640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f01c80a1090 con 0x7f01c8095540 2024-09-11T00:20:53.112 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:53.109+0000 7f01d6000640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f01c80a1390 con 0x7f01c8096690 2024-09-11T00:20:53.112 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:53.109+0000 7f01d4c00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f01c8095540 0x7f01c812da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:20:53.112 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:53.109+0000 7f01cfe00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f01c8096690 0x7f01c812e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:20:53.112 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:53.109+0000 7f01cfe00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f01c8096690 0x7f01c812e0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.12:53884/0 (socket says 172.21.15.12:53884) 2024-09-11T00:20:53.112 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:53.109+0000 7f01d4c00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f01c8095540 0x7f01c812da60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.12:47732/0 (socket says 172.21.15.12:47732) 2024-09-11T00:20:53.112 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:53.109+0000 7f01cfe00640 1 -- 172.21.15.12:0/2394524212 learned_addr learned my addr 172.21.15.12:0/2394524212 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:20:53.113 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:53.110+0000 7f01cfe00640 1 -- 172.21.15.12:0/2394524212 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f01c8095540 msgr2=0x7f01c812da60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:20:53.113 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:53.110+0000 7f01cfe00640 1 --2- 172.21.15.12:0/2394524212 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f01c8095540 0x7f01c812da60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:20:53.113 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:53.110+0000 7f01d5600640 1 --2- 172.21.15.12:0/2394524212 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f01c8097050 0x7f01c812ec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:20:53.113 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:53.110+0000 7f01cfe00640 1 -- 172.21.15.12:0/2394524212 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f01c8097050 msgr2=0x7f01c812ec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:20:53.113 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:53.111+0000 7f01cfe00640 1 --2- 172.21.15.12:0/2394524212 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f01c8097050 0x7f01c812ec70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:20:53.114 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:53.111+0000 7f01cfe00640 1 -- 172.21.15.12:0/2394524212 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01c812f530 con 0x7f01c8096690 2024-09-11T00:20:53.114 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:53.111+0000 7f01d5600640 1 --2- 172.21.15.12:0/2394524212 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f01c8097050 0x7f01c812ec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T00:20:53.114 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:53.112+0000 7f01cfe00640 1 --2- 172.21.15.12:0/2394524212 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f01c8096690 0x7f01c812e0b0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f01c400c920 tx=0x7f01c400cde0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:20:53.114 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:53.112+0000 7f01cd600640 1 -- 172.21.15.12:0/2394524212 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01c4007c70 con 0x7f01c8096690 2024-09-11T00:20:53.114 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:53.112+0000 7f01cd600640 1 -- 172.21.15.12:0/2394524212 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f01c4007e10 con 0x7f01c8096690 2024-09-11T00:20:53.115 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:53.112+0000 7f01cd600640 1 -- 172.21.15.12:0/2394524212 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01c4018820 con 0x7f01c8096690 2024-09-11T00:20:53.115 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:53.112+0000 7f01d6000640 1 -- 172.21.15.12:0/2394524212 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01c812f820 con 0x7f01c8096690 2024-09-11T00:20:53.115 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:53.112+0000 7f01d6000640 1 -- 172.21.15.12:0/2394524212 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01c8097b80 con 0x7f01c8096690 2024-09-11T00:20:53.124 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:53.122+0000 7f01cd600640 1 -- 172.21.15.12:0/2394524212 <== mon.1 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7f01c400f450 con 0x7f01c8096690 2024-09-11T00:20:53.127 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:53.124+0000 7f01cd600640 1 --2- 172.21.15.12:0/2394524212 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f01b80b5360 0x7f01b80b7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:20:53.127 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:53.125+0000 7f01da5c4740 1 Processor -- start 2024-09-11T00:20:53.128 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:53.125+0000 7f01cd600640 1 -- 172.21.15.12:0/2394524212 <== mon.1 v2:172.21.15.178:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9208+0+0 (secure 0 0 0) 0x7f01c40e6060 con 0x7f01c8096690 2024-09-11T00:20:53.129 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:53.125+0000 7f01da5c4740 1 -- start start 2024-09-11T00:20:53.129 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:53.125+0000 7f01da5c4740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a2f6e2a970 0x55a2f6e2cd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:20:53.130 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:53.125+0000 7f01da5c4740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a2f6e2d2a0 0x55a2f6e2f690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:20:53.130 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:53.125+0000 7f01da5c4740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a2f6e2fce0 0x55a2f6e321a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:20:53.130 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:53.126+0000 7f01d4c00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a2f6e2a970 0x55a2f6e2cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:20:53.130 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:53.126+0000 7f01cfe00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a2f6e2d2a0 0x55a2f6e2f690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:20:53.130 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:53.126+0000 7f01cfe00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a2f6e2d2a0 0x55a2f6e2f690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:53842/0 (socket says 172.21.15.12:53842) 2024-09-11T00:20:53.130 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:53.126+0000 7f01d5600640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a2f6e2fce0 0x55a2f6e321a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:20:53.130 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:53.126+0000 7f01d5600640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a2f6e2fce0 0x55a2f6e321a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.12:53888/0 (socket says 172.21.15.12:53888) 2024-09-11T00:20:53.130 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:53.127+0000 7f01da5c4740 1 -- 172.21.15.12:0/344836924 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55a2f6d9d6b0 con 0x55a2f6e2d2a0 2024-09-11T00:20:53.130 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:53.127+0000 7f01cfe00640 1 -- 172.21.15.12:0/344836924 learned_addr learned my addr 172.21.15.12:0/344836924 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:20:53.130 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:53.127+0000 7f01da5c4740 1 -- 172.21.15.12:0/344836924 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55a2f6d8cae0 con 0x55a2f6e2a970 2024-09-11T00:20:53.130 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:53.127+0000 7f01da5c4740 1 -- 172.21.15.12:0/344836924 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55a2f6d908f0 con 0x55a2f6e2fce0 2024-09-11T00:20:53.130 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:53.127+0000 7f01d4c00640 1 --2- 172.21.15.12:0/2394524212 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f01b80b5360 0x7f01b80b7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:20:53.131 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:53.127+0000 7f01d5600640 1 -- 172.21.15.12:0/344836924 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a2f6e2a970 msgr2=0x55a2f6e2cd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:20:53.131 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:53.127+0000 7f01d5600640 1 --2- 172.21.15.12:0/344836924 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a2f6e2a970 0x55a2f6e2cd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:20:53.131 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:53.127+0000 7f01d5600640 1 -- 172.21.15.12:0/344836924 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a2f6e2d2a0 msgr2=0x55a2f6e2f690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:20:53.131 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:53.127+0000 7f01d5600640 1 --2- 172.21.15.12:0/344836924 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a2f6e2d2a0 0x55a2f6e2f690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:20:53.131 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:53.127+0000 7f01d5600640 1 -- 172.21.15.12:0/344836924 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a2f6d59780 con 0x55a2f6e2fce0 2024-09-11T00:20:53.131 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:53.128+0000 7f01d4c00640 1 --2- 172.21.15.12:0/2394524212 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f01b80b5360 0x7f01b80b7820 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f01bc02a9e0 tx=0x7f01bc009580 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:20:53.131 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:53.129+0000 7f01d5600640 1 --2- 172.21.15.12:0/344836924 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a2f6e2fce0 0x55a2f6e321a0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f01d002eb50 tx=0x7f01d006cf50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7eda8b6b808619f0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:20:53.131 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:53.129+0000 7f01c1c00640 1 -- 172.21.15.12:0/344836924 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01d0069d50 con 0x55a2f6e2fce0 2024-09-11T00:20:53.132 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:53.129+0000 7f01c1c00640 1 -- 172.21.15.12:0/344836924 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f01d006c5c0 con 0x55a2f6e2fce0 2024-09-11T00:20:53.132 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:53.129+0000 7f01c1c00640 1 -- 172.21.15.12:0/344836924 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01d00716f0 con 0x55a2f6e2fce0 2024-09-11T00:20:53.133 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:53.131+0000 7f01da5c4740 1 -- 172.21.15.12:0/344836924 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a2f6e2fce0 msgr2=0x55a2f6e321a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:20:53.133 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:53.131+0000 7f01da5c4740 1 --2- 172.21.15.12:0/344836924 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a2f6e2fce0 0x55a2f6e321a0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f01d002eb50 tx=0x7f01d006cf50 comp rx=0 tx=0).stop 2024-09-11T00:20:53.134 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:53.131+0000 7f01da5c4740 1 -- 172.21.15.12:0/344836924 shutdown_connections 2024-09-11T00:20:53.134 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:53.131+0000 7f01da5c4740 1 --2- 172.21.15.12:0/344836924 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a2f6e2fce0 0x55a2f6e321a0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:20:53.134 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:53.131+0000 7f01da5c4740 1 --2- 172.21.15.12:0/344836924 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a2f6e2d2a0 0x55a2f6e2f690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:20:53.134 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:53.131+0000 7f01da5c4740 1 --2- 172.21.15.12:0/344836924 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a2f6e2a970 0x55a2f6e2cd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:20:53.134 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:53.131+0000 7f01da5c4740 1 -- 172.21.15.12:0/344836924 >> 172.21.15.12:0/344836924 conn(0x55a2f6e25ac0 msgr2=0x55a2f6e26850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:20:53.135 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:53.132+0000 7f01da5c4740 1 -- 172.21.15.12:0/344836924 shutdown_connections 2024-09-11T00:20:53.136 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:53.133+0000 7f01da5c4740 1 -- 172.21.15.12:0/344836924 wait complete. 2024-09-11T00:20:53.136 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:53.134+0000 7f01da5c4740 1 -- start start 2024-09-11T00:20:53.137 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:53.135+0000 7f01da5c4740 1 Processor -- start 2024-09-11T00:20:53.139 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:53.136+0000 7f01da5c4740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a2f6e2a970 0x55a2f6e34070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:20:53.139 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:53.136+0000 7f01da5c4740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a2f6e2d2a0 0x55a2f6e34710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:20:53.139 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:53.136+0000 7f01da5c4740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a2f6e2fce0 0x55a2f6e352e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:20:53.139 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:53.136+0000 7f01da5c4740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55a2f6cc4ab0 con 0x55a2f6e2a970 2024-09-11T00:20:53.139 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:53.136+0000 7f01da5c4740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55a2f6d33570 con 0x55a2f6e2d2a0 2024-09-11T00:20:53.139 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:53.136+0000 7f01da5c4740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55a2f6e32cc0 con 0x55a2f6e2fce0 2024-09-11T00:20:53.140 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:53.136+0000 7f01da5c4740 10 client.0 _ll_register_callbacks cb 0x55a2f6d42100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:20:53.140 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:53.137+0000 7f01cfe00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a2f6e2d2a0 0x55a2f6e34710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:20:53.140 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:53.137+0000 7f01cfe00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a2f6e2d2a0 0x55a2f6e34710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.12:47746/0 (socket says 172.21.15.12:47746) 2024-09-11T00:20:53.140 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:53.137+0000 7f01cfe00640 1 -- 172.21.15.12:0/2809249771 learned_addr learned my addr 172.21.15.12:0/2809249771 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:20:53.141 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:53.138+0000 7f01cfe00640 1 -- 172.21.15.12:0/2809249771 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a2f6e2fce0 msgr2=0x55a2f6e352e0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-11T00:20:53.141 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:53.138+0000 7f01cfe00640 1 --2- 172.21.15.12:0/2809249771 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a2f6e2fce0 0x55a2f6e352e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:20:53.141 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:53.138+0000 7f01cfe00640 1 -- 172.21.15.12:0/2809249771 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a2f6e2a970 msgr2=0x55a2f6e34070 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-11T00:20:53.141 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:53.138+0000 7f01cfe00640 1 --2- 172.21.15.12:0/2809249771 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a2f6e2a970 0x55a2f6e34070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:20:53.141 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:53.138+0000 7f01cfe00640 1 -- 172.21.15.12:0/2809249771 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a2f6aee790 con 0x55a2f6e2d2a0 2024-09-11T00:20:53.141 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:53.139+0000 7f01cfe00640 1 --2- 172.21.15.12:0/2809249771 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a2f6e2d2a0 0x55a2f6e34710 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f01c40a8460 tx=0x7f01c40abf60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5187ab10fdc28024 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:20:53.142 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:53.139+0000 7f01b6000640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-09-11T00:20:53.142 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:53.139+0000 7f01b6000640 1 -- 172.21.15.12:0/2809249771 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01c4097070 con 0x55a2f6e2d2a0 2024-09-11T00:20:53.142 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:53.139+0000 7f01b6000640 1 -- 172.21.15.12:0/2809249771 <== mon.2 v2:172.21.15.165:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9208+0+0 (secure 0 0 0) 0x7f01c40d31b0 con 0x55a2f6e2d2a0 2024-09-11T00:20:53.142 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:53.139+0000 7f01da5c4740 1 -- 172.21.15.12:0/2809249771 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f01d006c5c0 con 0x55a2f6e2d2a0 2024-09-11T00:20:53.142 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:53.139+0000 7f01b6000640 1 -- 172.21.15.12:0/2809249771 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7f01c40d33b0 con 0x55a2f6e2d2a0 2024-09-11T00:20:53.143 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:53.140+0000 7f01da5c4740 10 client.27050 fetch_fsmap learned FSMap version 54 2024-09-11T00:20:53.143 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:53.140+0000 7f01da5c4740 1 -- 172.21.15.12:0/2809249771 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) -- 0x55a2f6b67620 con 0x55a2f6e2d2a0 2024-09-11T00:20:53.143 INFO:tasks.workunit.client.0.smithi012.stdout: 10 27808 34.07 MB/sec warmup 49 sec latency 103.814 ms 2024-09-11T00:20:53.145 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:53.143+0000 7f01b6000640 1 -- 172.21.15.12:0/2809249771 <== mon.2 v2:172.21.15.165:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f01d006c5c0 con 0x55a2f6e2d2a0 2024-09-11T00:20:53.145 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:53.143+0000 7f01b6000640 1 -- 172.21.15.12:0/2809249771 <== mon.2 v2:172.21.15.165:3300/0 5 ==== fsmap(e 54) ==== 8422+0+0 (secure 0 0 0) 0x7f01c40b5550 con 0x55a2f6e2d2a0 2024-09-11T00:20:53.145 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:53.143+0000 7f01da5c4740 10 client.27050 fetch_fsmap finished waiting for FSMap version 54 2024-09-11T00:20:53.145 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:53.143+0000 7f01da5c4740 10 client.27050 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T00:20:53.146 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:53.143+0000 7f01da5c4740 20 client.27050 populate_metadata read hostname 'smithi012' 2024-09-11T00:20:53.146 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:53.143+0000 7f01da5c4740 1 --2- 172.21.15.12:0/2809249771 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55a2f6e3fa90 0x55a2f6e41f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:20:53.147 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:53.144+0000 7f01da5c4740 4 client.27050 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T00:20:53.147 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:53.144+0000 7f01da5c4740 1 -- 172.21.15.12:0/2809249771 --> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a2f6d9f600 con 0x55a2f6e3fa90 2024-09-11T00:20:53.148 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:53.146+0000 7f01d4c00640 1 --2- 172.21.15.12:0/2809249771 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55a2f6e3fa90 0x55a2f6e41f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:20:53.150 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:53.148+0000 7f01d4c00640 1 --2- 172.21.15.12:0/2809249771 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55a2f6e3fa90 0x55a2f6e41f50 crc :-1 s=READY pgs=203 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=df61c87e9be06756 server_cookie=53d633e8587c574d in_seq=0 out_seq=0 2024-09-11T00:20:53.151 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:53.149+0000 7f01b6000640 10 client.27050 ms_handle_connect on v2:172.21.15.165:6838/501312984 2024-09-11T00:20:53.153 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:53.151+0000 7f01b6000640 1 -- 172.21.15.12:0/2809249771 <== mds.0 v2:172.21.15.165:6838/501312984 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55a2f6d9f600 con 0x55a2f6e3fa90 2024-09-11T00:20:53.153 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:53.151+0000 7f01b6000640 10 client.27050 handle_command_reply: tid=0 2024-09-11T00:20:53.154 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:53.152+0000 7f01da5c4740 1 client.27050 shutdown 2024-09-11T00:20:53.156 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:53.154+0000 7f01da5c4740 1 -- 172.21.15.12:0/2809249771 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a2f6e2d2a0 msgr2=0x55a2f6e34710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:20:53.156 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:53.154+0000 7f01da5c4740 1 --2- 172.21.15.12:0/2809249771 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a2f6e2d2a0 0x55a2f6e34710 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f01c40a8460 tx=0x7f01c40abf60 comp rx=0 tx=0).stop 2024-09-11T00:20:53.157 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:53.154+0000 7f01da5c4740 1 -- 172.21.15.12:0/2809249771 shutdown_connections 2024-09-11T00:20:53.157 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:53.154+0000 7f01da5c4740 1 --2- 172.21.15.12:0/2809249771 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55a2f6e3fa90 0x55a2f6e41f50 crc :-1 s=READY pgs=203 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:20:53.157 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:53.154+0000 7f01da5c4740 1 --2- 172.21.15.12:0/2809249771 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a2f6e2fce0 0x55a2f6e352e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:20:53.157 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:53.154+0000 7f01da5c4740 1 --2- 172.21.15.12:0/2809249771 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a2f6e2d2a0 0x55a2f6e34710 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:20:53.157 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:53.154+0000 7f01da5c4740 1 --2- 172.21.15.12:0/2809249771 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a2f6e2a970 0x55a2f6e34070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:20:53.157 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:53.155+0000 7f01da5c4740 1 -- 172.21.15.12:0/2809249771 >> 172.21.15.12:0/2809249771 conn(0x55a2f6e25ac0 msgr2=0x55a2f6e24bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:20:53.158 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:53.156+0000 7f01b6000640 0 client.27050 ms_handle_reset on v2:172.21.15.165:6838/501312984 2024-09-11T00:20:53.158 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:53.156+0000 7f01da5c4740 1 -- 172.21.15.12:0/2809249771 shutdown_connections 2024-09-11T00:20:53.158 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:53.156+0000 7f01da5c4740 1 -- 172.21.15.12:0/2809249771 wait complete. 2024-09-11T00:20:53.159 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:53.156+0000 7f01da5c4740 20 client.27050 trim_cache size 0 max 16384 2024-09-11T00:20:53.214 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:53.208+0000 7f01da5c4740 1 Processor -- start 2024-09-11T00:20:53.214 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:53.208+0000 7f01da5c4740 1 -- start start 2024-09-11T00:20:53.214 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:53.208+0000 7f01da5c4740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a2f6e2a970 0x55a2f6e300c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:20:53.215 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:53.208+0000 7f01da5c4740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a2f6e2d2a0 0x55a2f6e30600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:20:53.215 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:53.208+0000 7f01da5c4740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a2f6e2fce0 0x55a2f6e29dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:20:53.215 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:53.208+0000 7f01da5c4740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f01c8144830 con 0x55a2f6e2a970 2024-09-11T00:20:53.215 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:53.208+0000 7f01da5c4740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f01c8146550 con 0x55a2f6e2fce0 2024-09-11T00:20:53.215 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:53.208+0000 7f01da5c4740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f01c8144f60 con 0x55a2f6e2d2a0 2024-09-11T00:20:53.216 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:53.213+0000 7f01cfe00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a2f6e2d2a0 0x55a2f6e30600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:20:53.216 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:53.213+0000 7f01cfe00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a2f6e2d2a0 0x55a2f6e30600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.12:53898/0 (socket says 172.21.15.12:53898) 2024-09-11T00:20:53.216 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:53.213+0000 7f01cfe00640 1 -- 172.21.15.12:0/210748259 learned_addr learned my addr 172.21.15.12:0/210748259 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:20:53.220 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:53.213+0000 7f01cfe00640 1 -- 172.21.15.12:0/210748259 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a2f6e2fce0 msgr2=0x55a2f6e29dd0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-11T00:20:53.221 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:53.213+0000 7f01cfe00640 1 --2- 172.21.15.12:0/210748259 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a2f6e2fce0 0x55a2f6e29dd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:20:53.221 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:53.213+0000 7f01cfe00640 1 -- 172.21.15.12:0/210748259 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a2f6e2a970 msgr2=0x55a2f6e300c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:20:53.221 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:53.213+0000 7f01cfe00640 1 --2- 172.21.15.12:0/210748259 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a2f6e2a970 0x55a2f6e300c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:20:53.221 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:53.213+0000 7f01cfe00640 1 -- 172.21.15.12:0/210748259 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a2f6e358c0 con 0x55a2f6e2d2a0 2024-09-11T00:20:53.221 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:53.218+0000 7f01cfe00640 1 --2- 172.21.15.12:0/210748259 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a2f6e2d2a0 0x55a2f6e30600 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f01c40a96c0 tx=0x7f01c400cf90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4af771de4accd7aa server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:20:53.222 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:53.218+0000 7f01c1200640 1 -- 172.21.15.12:0/210748259 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01c4097040 con 0x55a2f6e2d2a0 2024-09-11T00:20:53.223 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:53.218+0000 7f01c1200640 1 -- 172.21.15.12:0/210748259 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f01c40b0f30 con 0x55a2f6e2d2a0 2024-09-11T00:20:53.223 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:53.218+0000 7f01c1200640 1 -- 172.21.15.12:0/210748259 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01c40b5550 con 0x55a2f6e2d2a0 2024-09-11T00:20:53.226 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:53.223+0000 7f01da5c4740 1 -- 172.21.15.12:0/210748259 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a2f6e2d2a0 msgr2=0x55a2f6e30600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:20:53.227 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:53.223+0000 7f01da5c4740 1 --2- 172.21.15.12:0/210748259 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a2f6e2d2a0 0x55a2f6e30600 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f01c40a96c0 tx=0x7f01c400cf90 comp rx=0 tx=0).stop 2024-09-11T00:20:53.227 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:53.223+0000 7f01da5c4740 1 -- 172.21.15.12:0/210748259 shutdown_connections 2024-09-11T00:20:53.227 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:53.225+0000 7f01da5c4740 1 --2- 172.21.15.12:0/210748259 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a2f6e2fce0 0x55a2f6e29dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:20:53.227 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:53.225+0000 7f01da5c4740 1 --2- 172.21.15.12:0/210748259 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a2f6e2d2a0 0x55a2f6e30600 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:20:53.227 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:53.225+0000 7f01da5c4740 1 --2- 172.21.15.12:0/210748259 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a2f6e2a970 0x55a2f6e300c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:20:53.227 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:53.225+0000 7f01da5c4740 1 -- 172.21.15.12:0/210748259 >> 172.21.15.12:0/210748259 conn(0x55a2f6e25ac0 msgr2=0x55a2f6e26560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:20:53.228 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:53.225+0000 7f01da5c4740 1 -- 172.21.15.12:0/210748259 shutdown_connections 2024-09-11T00:20:53.228 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:53.225+0000 7f01da5c4740 1 -- 172.21.15.12:0/210748259 wait complete. 2024-09-11T00:20:53.228 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:53.225+0000 7f01da5c4740 1 -- start start 2024-09-11T00:20:53.228 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:53.225+0000 7f01da5c4740 1 Processor -- start 2024-09-11T00:20:53.230 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:53.227+0000 7f01da5c4740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a2f6e2a970 0x55a2f6e36fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:20:53.233 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:53.227+0000 7f01da5c4740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a2f6e2d2a0 0x55a2f6e37640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:20:53.233 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:53.227+0000 7f01d4c00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a2f6e2a970 0x55a2f6e36fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:20:53.233 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:53.227+0000 7f01d4c00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a2f6e2a970 0x55a2f6e36fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.12:53904/0 (socket says 172.21.15.12:53904) 2024-09-11T00:20:53.233 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:53.227+0000 7f01da5c4740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a2f6e2fce0 0x55a2f6e38210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:20:53.233 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:53.227+0000 7f01da5c4740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f01c8145e80 con 0x55a2f6e2d2a0 2024-09-11T00:20:53.233 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:53.227+0000 7f01da5c4740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f01c81456f0 con 0x55a2f6e2fce0 2024-09-11T00:20:53.233 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:53.227+0000 7f01da5c4740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f01c8142810 con 0x55a2f6e2a970 2024-09-11T00:20:53.233 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:53.227+0000 7f01da5c4740 10 client.0 _ll_register_callbacks cb 0x7f01c40abe70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:20:53.234 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:53.227+0000 7f01d4c00640 1 -- 172.21.15.12:0/1561871388 learned_addr learned my addr 172.21.15.12:0/1561871388 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:20:53.234 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:53.227+0000 7f01d5600640 1 --2- 172.21.15.12:0/1561871388 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a2f6e2fce0 0x55a2f6e38210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:20:53.235 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:53.232+0000 7f01d5600640 1 -- 172.21.15.12:0/1561871388 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a2f6e2a970 msgr2=0x55a2f6e36fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:20:53.235 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:53.232+0000 7f01d5600640 1 --2- 172.21.15.12:0/1561871388 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a2f6e2a970 0x55a2f6e36fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:20:53.235 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:53.232+0000 7f01d5600640 1 -- 172.21.15.12:0/1561871388 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a2f6e2d2a0 msgr2=0x55a2f6e37640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:20:53.235 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:53.232+0000 7f01cfe00640 1 --2- 172.21.15.12:0/1561871388 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a2f6e2d2a0 0x55a2f6e37640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:20:53.235 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:53.232+0000 7f01d5600640 1 --2- 172.21.15.12:0/1561871388 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a2f6e2d2a0 0x55a2f6e37640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:20:53.235 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:53.232+0000 7f01d5600640 1 -- 172.21.15.12:0/1561871388 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a2f6e38a40 con 0x55a2f6e2fce0 2024-09-11T00:20:53.236 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:53.232+0000 7f01d5600640 1 --2- 172.21.15.12:0/1561871388 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a2f6e2fce0 0x55a2f6e38210 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f01d0069fd0 tx=0x7f01d00690f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5dd3337e326804e8 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:20:53.236 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:53.232+0000 7f01b6000640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-09-11T00:20:53.236 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:53.232+0000 7f01b6000640 1 -- 172.21.15.12:0/1561871388 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01d0086060 con 0x55a2f6e2fce0 2024-09-11T00:20:53.236 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:53.232+0000 7f01b6000640 1 -- 172.21.15.12:0/1561871388 <== mon.2 v2:172.21.15.165:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9208+0+0 (secure 0 0 0) 0x7f01d006c5e0 con 0x55a2f6e2fce0 2024-09-11T00:20:53.236 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:53.232+0000 7f01da5c4740 1 -- 172.21.15.12:0/1561871388 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f01c40b0f30 con 0x55a2f6e2fce0 2024-09-11T00:20:53.236 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:53.232+0000 7f01b6000640 1 -- 172.21.15.12:0/1561871388 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7f01d00517b0 con 0x55a2f6e2fce0 2024-09-11T00:20:53.237 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:53.232+0000 7f01da5c4740 10 client.27053 fetch_fsmap learned FSMap version 54 2024-09-11T00:20:53.237 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:53.232+0000 7f01da5c4740 1 -- 172.21.15.12:0/1561871388 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) -- 0x55a2f6e3bdc0 con 0x55a2f6e2fce0 2024-09-11T00:20:53.237 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:53.232+0000 7f01b6000640 1 -- 172.21.15.12:0/1561871388 <== mon.2 v2:172.21.15.165:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f01c40b0f30 con 0x55a2f6e2fce0 2024-09-11T00:20:53.238 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:53.232+0000 7f01b6000640 1 -- 172.21.15.12:0/1561871388 <== mon.2 v2:172.21.15.165:3300/0 5 ==== fsmap(e 54) ==== 8422+0+0 (secure 0 0 0) 0x7f01d0080070 con 0x55a2f6e2fce0 2024-09-11T00:20:53.238 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:53.232+0000 7f01da5c4740 10 client.27053 fetch_fsmap finished waiting for FSMap version 54 2024-09-11T00:20:53.239 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:53.232+0000 7f01da5c4740 10 client.27053 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T00:20:53.239 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:53.232+0000 7f01da5c4740 20 client.27053 populate_metadata read hostname 'smithi012' 2024-09-11T00:20:53.239 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:53.232+0000 7f01da5c4740 1 --2- 172.21.15.12:0/1561871388 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55a2f6ebdcf0 0x55a2f6ec0110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:20:53.239 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:53.232+0000 7f01da5c4740 4 client.27053 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T00:20:53.239 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:53.232+0000 7f01da5c4740 1 -- 172.21.15.12:0/1561871388 --> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55a2f6d9b0b0 con 0x55a2f6ebdcf0 2024-09-11T00:20:53.274 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:53.237+0000 7f01d4c00640 1 --2- 172.21.15.12:0/1561871388 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55a2f6ebdcf0 0x55a2f6ec0110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:20:53.274 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:53.259+0000 7f01b6000640 10 client.27053 ms_handle_connect on v2:172.21.15.165:6838/501312984 2024-09-11T00:20:53.274 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:53.259+0000 7f01d4c00640 1 --2- 172.21.15.12:0/1561871388 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55a2f6ebdcf0 0x55a2f6ec0110 crc :-1 s=READY pgs=204 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cbc189b46f6c8d2b server_cookie=476e042871ef3d13 in_seq=0 out_seq=0 2024-09-11T00:20:53.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:20:52 smithi165 ceph-mon[35356]: from='client.? 172.21.15.12:0/1882996674' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T00:20:53.977 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:20:53 smithi012 ceph-mon[30333]: pgmap v813: 129 pgs: 129 active+clean; 387 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 6.1 MiB/s wr, 833 op/s 2024-09-11T00:20:53.977 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:20:53 smithi012 ceph-mon[30333]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T00:20:54.000 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24674 quiesce db / --timeout 90 --expiration 142.75 --await-for 45.0' 2024-09-11T00:20:54.000 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid ff3ad74a-6fcf-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24674 quiesce db / --timeout 90 --expiration 142.75 --await-for 45.0 2024-09-11T00:20:54.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:20:53 smithi178 ceph-mon[35919]: pgmap v813: 129 pgs: 129 active+clean; 387 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 6.1 MiB/s wr, 833 op/s 2024-09-11T00:20:54.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:20:53 smithi178 ceph-mon[35919]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T00:20:54.144 INFO:tasks.workunit.client.0.smithi012.stdout: 10 27840 33.40 MB/sec warmup 50 sec latency 971.065 ms 2024-09-11T00:20:54.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:20:53 smithi165 ceph-mon[35356]: pgmap v813: 129 pgs: 129 active+clean; 387 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 6.1 MiB/s wr, 833 op/s 2024-09-11T00:20:54.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:20:53 smithi165 ceph-mon[35356]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T00:20:54.593 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:20:54.938 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:54.936+0000 7f01b6000640 1 -- 172.21.15.12:0/1561871388 <== mds.0 v2:172.21.15.165:6838/501312984 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55a2f6d9b0b0 con 0x55a2f6ebdcf0 2024-09-11T00:20:54.938 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:54.936+0000 7f01b6000640 10 client.27053 handle_command_reply: tid=0 2024-09-11T00:20:54.938 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:54.936+0000 7f01da5c4740 1 client.27053 shutdown 2024-09-11T00:20:54.938 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:54.937+0000 7f01da5c4740 1 -- 172.21.15.12:0/1561871388 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a2f6e2fce0 msgr2=0x55a2f6e38210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:20:54.939 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:54.937+0000 7f01da5c4740 1 --2- 172.21.15.12:0/1561871388 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a2f6e2fce0 0x55a2f6e38210 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f01d0069fd0 tx=0x7f01d00690f0 comp rx=0 tx=0).stop 2024-09-11T00:20:54.940 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:54.937+0000 7f01da5c4740 1 -- 172.21.15.12:0/1561871388 shutdown_connections 2024-09-11T00:20:54.940 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:54.937+0000 7f01da5c4740 1 --2- 172.21.15.12:0/1561871388 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55a2f6ebdcf0 0x55a2f6ec0110 crc :-1 s=READY pgs=204 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:20:54.941 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:54.937+0000 7f01da5c4740 1 --2- 172.21.15.12:0/1561871388 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a2f6e2fce0 0x55a2f6e38210 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:20:54.941 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:54.937+0000 7f01da5c4740 1 --2- 172.21.15.12:0/1561871388 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a2f6e2d2a0 0x55a2f6e37640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:20:54.941 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:54.937+0000 7f01da5c4740 1 --2- 172.21.15.12:0/1561871388 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a2f6e2a970 0x55a2f6e36fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:20:54.941 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:54.937+0000 7f01da5c4740 1 -- 172.21.15.12:0/1561871388 >> 172.21.15.12:0/1561871388 conn(0x55a2f6e25ac0 msgr2=0x55a2f6e26560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:20:54.941 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:54.937+0000 7f01b6000640 0 client.27053 ms_handle_reset on v2:172.21.15.165:6838/501312984 2024-09-11T00:20:54.941 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:54.937+0000 7f01da5c4740 1 -- 172.21.15.12:0/1561871388 shutdown_connections 2024-09-11T00:20:54.941 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:54.937+0000 7f01da5c4740 1 -- 172.21.15.12:0/1561871388 wait complete. 2024-09-11T00:20:54.941 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:54.937+0000 7f01da5c4740 20 client.27053 trim_cache size 0 max 16384 2024-09-11T00:20:54.949 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:{ 2024-09-11T00:20:54.949 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "message": "", 2024-09-11T00:20:54.949 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "return_code": 0 2024-09-11T00:20:54.949 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:} 2024-09-11T00:20:54.949 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:54.937+0000 7f01d6000640 1 -- 172.21.15.12:0/2394524212 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f01b80b5360 msgr2=0x7f01b80b7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:20:54.949 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:54.937+0000 7f01d6000640 1 --2- 172.21.15.12:0/2394524212 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f01b80b5360 0x7f01b80b7820 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f01bc02a9e0 tx=0x7f01bc009580 comp rx=0 tx=0).stop 2024-09-11T00:20:54.950 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:54.937+0000 7f01d6000640 1 -- 172.21.15.12:0/2394524212 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f01c8096690 msgr2=0x7f01c812e0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:20:54.950 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:54.937+0000 7f01d6000640 1 --2- 172.21.15.12:0/2394524212 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f01c8096690 0x7f01c812e0b0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f01c400c920 tx=0x7f01c400cde0 comp rx=0 tx=0).stop 2024-09-11T00:20:54.950 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:54.937+0000 7f01d6000640 1 -- 172.21.15.12:0/2394524212 shutdown_connections 2024-09-11T00:20:54.950 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:54.937+0000 7f01d6000640 1 --2- 172.21.15.12:0/2394524212 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f01b80b5360 0x7f01b80b7820 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:20:54.950 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:54.937+0000 7f01d6000640 1 --2- 172.21.15.12:0/2394524212 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f01c8097050 0x7f01c812ec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:20:54.950 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:54.937+0000 7f01d6000640 1 --2- 172.21.15.12:0/2394524212 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f01c8096690 0x7f01c812e0b0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:20:54.950 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:54.937+0000 7f01d6000640 1 --2- 172.21.15.12:0/2394524212 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f01c8095540 0x7f01c812da60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:20:54.950 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:54.937+0000 7f01d6000640 1 -- 172.21.15.12:0/2394524212 >> 172.21.15.12:0/2394524212 conn(0x7f01c8090de0 msgr2=0x7f01c8092d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:20:54.950 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:54.937+0000 7f01d6000640 1 -- 172.21.15.12:0/2394524212 shutdown_connections 2024-09-11T00:20:54.950 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:20:54.937+0000 7f01d6000640 1 -- 172.21.15.12:0/2394524212 wait complete. 2024-09-11T00:20:55.144 INFO:tasks.workunit.client.0.smithi012.stdout: 10 27841 32.75 MB/sec warmup 51 sec latency 1971.122 ms 2024-09-11T00:20:55.946 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:20:55 smithi012 ceph-mon[30333]: pgmap v814: 129 pgs: 129 active+clean; 397 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 5.8 MiB/s wr, 639 op/s 2024-09-11T00:20:56.144 INFO:tasks.workunit.client.0.smithi012.stdout: 10 27889 32.22 MB/sec warmup 52 sec latency 2551.904 ms 2024-09-11T00:20:56.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:20:55 smithi165 ceph-mon[35356]: pgmap v814: 129 pgs: 129 active+clean; 397 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 5.8 MiB/s wr, 639 op/s 2024-09-11T00:20:56.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:20:55 smithi178 ceph-mon[35919]: pgmap v814: 129 pgs: 129 active+clean; 397 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 5.8 MiB/s wr, 639 op/s 2024-09-11T00:20:56.498 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:20:56.463+0000 7fe103200640 1 -- 172.21.15.12:0/2024572950 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe104108620 msgr2=0x7fe104108a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:20:56.498 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:20:56.463+0000 7fe103200640 1 --2- 172.21.15.12:0/2024572950 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe104108620 0x7fe104108a60 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7fe0f0009900 tx=0x7fe0f002a410 comp rx=0 tx=0).stop 2024-09-11T00:20:56.498 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:20:56.463+0000 7fe103200640 1 -- 172.21.15.12:0/2024572950 shutdown_connections 2024-09-11T00:20:56.498 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:20:56.463+0000 7fe103200640 1 --2- 172.21.15.12:0/2024572950 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe104108fa0 0x7fe10410f7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:20:56.498 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:20:56.463+0000 7fe103200640 1 --2- 172.21.15.12:0/2024572950 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe104108620 0x7fe104108a60 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:20:56.498 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:20:56.463+0000 7fe103200640 1 --2- 172.21.15.12:0/2024572950 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe104102c10 0x7fe104102ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:20:56.498 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:20:56.463+0000 7fe103200640 1 -- 172.21.15.12:0/2024572950 >> 172.21.15.12:0/2024572950 conn(0x7fe1040fe660 msgr2=0x7fe104100a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:20:56.498 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:20:56.463+0000 7fe103200640 1 -- 172.21.15.12:0/2024572950 shutdown_connections 2024-09-11T00:20:56.499 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:20:56.463+0000 7fe103200640 1 -- 172.21.15.12:0/2024572950 wait complete. 2024-09-11T00:20:56.499 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:20:56.468+0000 7fe103200640 1 Processor -- start 2024-09-11T00:20:56.499 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:20:56.468+0000 7fe103200640 1 -- start start 2024-09-11T00:20:56.499 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:20:56.468+0000 7fe103200640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe104102c10 0x7fe10419b260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:20:56.499 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:20:56.468+0000 7fe103200640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe104108620 0x7fe10419b7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:20:56.499 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:20:56.468+0000 7fe103200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe104108fa0 0x7fe1041a0b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:20:56.499 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:20:56.468+0000 7fe103200640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fe104112c30 con 0x7fe104102c10 2024-09-11T00:20:56.499 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:20:56.468+0000 7fe103200640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fe104112ab0 con 0x7fe104108fa0 2024-09-11T00:20:56.499 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:20:56.468+0000 7fe103200640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fe104112db0 con 0x7fe104108620 2024-09-11T00:20:56.499 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:20:56.468+0000 7fe101e00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe104102c10 0x7fe10419b260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:20:56.500 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:20:56.468+0000 7fe101e00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe104102c10 0x7fe10419b260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:53882/0 (socket says 172.21.15.12:53882) 2024-09-11T00:20:56.500 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:20:56.468+0000 7fe101e00640 1 -- 172.21.15.12:0/1317748335 learned_addr learned my addr 172.21.15.12:0/1317748335 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:20:56.500 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:20:56.468+0000 7fe102800640 1 --2- 172.21.15.12:0/1317748335 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe104108fa0 0x7fe1041a0b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:20:56.500 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:20:56.468+0000 7fe101400640 1 --2- 172.21.15.12:0/1317748335 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe104108620 0x7fe10419b7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:20:56.500 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:20:56.468+0000 7fe101e00640 1 -- 172.21.15.12:0/1317748335 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe104108620 msgr2=0x7fe10419b7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:20:56.500 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:20:56.468+0000 7fe101e00640 1 --2- 172.21.15.12:0/1317748335 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe104108620 0x7fe10419b7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:20:56.500 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:20:56.468+0000 7fe101e00640 1 -- 172.21.15.12:0/1317748335 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe104108fa0 msgr2=0x7fe1041a0b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:20:56.500 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:20:56.468+0000 7fe101e00640 1 --2- 172.21.15.12:0/1317748335 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe104108fa0 0x7fe1041a0b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:20:56.500 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:20:56.468+0000 7fe101e00640 1 -- 172.21.15.12:0/1317748335 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1041a1310 con 0x7fe104102c10 2024-09-11T00:20:56.500 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:20:56.468+0000 7fe101e00640 1 --2- 172.21.15.12:0/1317748335 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe104102c10 0x7fe10419b260 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7fe0f400e9f0 tx=0x7fe0f400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:20:56.500 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:20:56.468+0000 7fe0faa00640 1 -- 172.21.15.12:0/1317748335 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0f400cd80 con 0x7fe104102c10 2024-09-11T00:20:56.500 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:20:56.468+0000 7fe0faa00640 1 -- 172.21.15.12:0/1317748335 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe0f40108e0 con 0x7fe104102c10 2024-09-11T00:20:56.501 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:20:56.468+0000 7fe0faa00640 1 -- 172.21.15.12:0/1317748335 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0f4018980 con 0x7fe104102c10 2024-09-11T00:20:56.501 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:20:56.468+0000 7fe103200640 1 -- 172.21.15.12:0/1317748335 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe1041a1600 con 0x7fe104102c10 2024-09-11T00:20:56.501 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:20:56.468+0000 7fe103200640 1 -- 172.21.15.12:0/1317748335 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1041a7cc0 con 0x7fe104102c10 2024-09-11T00:20:56.501 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:20:56.473+0000 7fe10b68e740 1 Processor -- start 2024-09-11T00:20:56.501 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:20:56.473+0000 7fe10b68e740 1 -- start start 2024-09-11T00:20:56.501 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:20:56.473+0000 7fe0faa00640 1 -- 172.21.15.12:0/1317748335 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7fe0f4010430 con 0x7fe104102c10 2024-09-11T00:20:56.501 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:20:56.473+0000 7fe10b68e740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560720ea7e10 0x560720ea6370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:20:56.501 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:20:56.473+0000 7fe10b68e740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560720ea68b0 0x560720ea6cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:20:56.501 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:20:56.473+0000 7fe10b68e740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560720eb26e0 0x560720eb4ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:20:56.501 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:20:56.473+0000 7fe10b68e740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x560720e1ba90 con 0x560720ea68b0 2024-09-11T00:20:56.502 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:20:56.473+0000 7fe10b68e740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x560720e0aec0 con 0x560720ea7e10 2024-09-11T00:20:56.502 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:20:56.473+0000 7fe10b68e740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x560720e0ecd0 con 0x560720eb26e0 2024-09-11T00:20:56.502 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:20:56.473+0000 7fe101e00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560720ea68b0 0x560720ea6cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:20:56.502 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:20:56.473+0000 7fe101e00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560720ea68b0 0x560720ea6cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:53898/0 (socket says 172.21.15.12:53898) 2024-09-11T00:20:56.502 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:20:56.473+0000 7fe101e00640 1 -- 172.21.15.12:0/3367441738 learned_addr learned my addr 172.21.15.12:0/3367441738 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:20:56.502 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:20:56.473+0000 7fe101e00640 1 -- 172.21.15.12:0/3367441738 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560720eb26e0 msgr2=0x560720eb4ad0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-11T00:20:56.502 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:20:56.473+0000 7fe101e00640 1 --2- 172.21.15.12:0/3367441738 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560720eb26e0 0x560720eb4ad0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:20:56.503 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:20:56.473+0000 7fe101e00640 1 -- 172.21.15.12:0/3367441738 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560720ea7e10 msgr2=0x560720ea6370 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-11T00:20:56.503 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:20:56.473+0000 7fe101e00640 1 --2- 172.21.15.12:0/3367441738 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560720ea7e10 0x560720ea6370 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:20:56.503 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:20:56.473+0000 7fe101e00640 1 -- 172.21.15.12:0/3367441738 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560720db09d0 con 0x560720ea68b0 2024-09-11T00:20:56.503 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:20:56.473+0000 7fe101e00640 1 --2- 172.21.15.12:0/3367441738 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560720ea68b0 0x560720ea6cd0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7fe0f40ecc10 tx=0x7fe0f40ee690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b17aaeac94db869a server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:20:56.503 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:20:56.473+0000 7fe0ef000640 1 -- 172.21.15.12:0/3367441738 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0f40f5070 con 0x560720ea68b0 2024-09-11T00:20:56.503 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:20:56.473+0000 7fe0ef000640 1 -- 172.21.15.12:0/3367441738 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe0f40e6cb0 con 0x560720ea68b0 2024-09-11T00:20:56.503 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:20:56.473+0000 7fe0ef000640 1 -- 172.21.15.12:0/3367441738 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0f40f17e0 con 0x560720ea68b0 2024-09-11T00:20:56.503 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:20:56.473+0000 7fe0faa00640 1 --2- 172.21.15.12:0/1317748335 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fe0e80b5410 0x7fe0e80b78d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:20:56.503 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:20:56.473+0000 7fe0faa00640 1 -- 172.21.15.12:0/1317748335 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9208+0+0 (secure 0 0 0) 0x7fe0f40e6600 con 0x7fe104102c10 2024-09-11T00:20:56.507 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:20:56.473+0000 7fe102800640 1 --2- 172.21.15.12:0/1317748335 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fe0e80b5410 0x7fe0e80b78d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:20:56.507 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:20:56.473+0000 7fe10b68e740 1 -- 172.21.15.12:0/3367441738 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560720ea68b0 msgr2=0x560720ea6cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:20:56.507 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:20:56.473+0000 7fe10b68e740 1 --2- 172.21.15.12:0/3367441738 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560720ea68b0 0x560720ea6cd0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7fe0f40ecc10 tx=0x7fe0f40ee690 comp rx=0 tx=0).stop 2024-09-11T00:20:56.507 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:20:56.473+0000 7fe10b68e740 1 -- 172.21.15.12:0/3367441738 shutdown_connections 2024-09-11T00:20:56.507 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:20:56.473+0000 7fe10b68e740 1 --2- 172.21.15.12:0/3367441738 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560720eb26e0 0x560720eb4ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:20:56.508 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:20:56.473+0000 7fe10b68e740 1 --2- 172.21.15.12:0/3367441738 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560720ea68b0 0x560720ea6cd0 secure :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7fe0f40ecc10 tx=0x7fe0f40ee690 comp rx=0 tx=0).stop 2024-09-11T00:20:56.508 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:20:56.473+0000 7fe10b68e740 1 --2- 172.21.15.12:0/3367441738 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560720ea7e10 0x560720ea6370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:20:56.508 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:20:56.473+0000 7fe10b68e740 1 -- 172.21.15.12:0/3367441738 >> 172.21.15.12:0/3367441738 conn(0x560720e706f0 msgr2=0x560720ea41a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:20:56.508 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:20:56.473+0000 7fe102800640 1 --2- 172.21.15.12:0/1317748335 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fe0e80b5410 0x7fe0e80b78d0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fe0fc005fd0 tx=0x7fe0fc009600 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:20:56.508 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:20:56.473+0000 7fe10b68e740 1 -- 172.21.15.12:0/3367441738 shutdown_connections 2024-09-11T00:20:56.508 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:20:56.473+0000 7fe10b68e740 1 -- 172.21.15.12:0/3367441738 wait complete. 2024-09-11T00:20:56.508 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:20:56.478+0000 7fe10b68e740 1 -- start start 2024-09-11T00:20:56.508 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:20:56.478+0000 7fe10b68e740 1 Processor -- start 2024-09-11T00:20:56.508 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:20:56.478+0000 7fe10b68e740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560720ea7e10 0x560720eaf810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:20:56.509 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:20:56.478+0000 7fe10b68e740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560720eb26e0 0x560720eafd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:20:56.509 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:20:56.478+0000 7fe10b68e740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560720eb0290 0x560720ebe890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:20:56.509 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:20:56.478+0000 7fe10b68e740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x560720c6c7b0 con 0x560720eb0290 2024-09-11T00:20:56.509 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:20:56.478+0000 7fe10b68e740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x560720daf650 con 0x560720eb26e0 2024-09-11T00:20:56.509 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:20:56.478+0000 7fe10b68e740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x560720eb5500 con 0x560720ea7e10 2024-09-11T00:20:56.509 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:20:56.478+0000 7fe10b68e740 10 client.0 _ll_register_callbacks cb 0x560720b160a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:20:56.509 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:20:56.478+0000 7fe101400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560720eb26e0 0x560720eafd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:20:56.510 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:20:56.478+0000 7fe101400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560720eb26e0 0x560720eafd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.12:47778/0 (socket says 172.21.15.12:47778) 2024-09-11T00:20:56.510 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:20:56.478+0000 7fe101400640 1 -- 172.21.15.12:0/2989450527 learned_addr learned my addr 172.21.15.12:0/2989450527 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:20:56.510 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:20:56.478+0000 7fe101e00640 1 --2- 172.21.15.12:0/2989450527 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560720ea7e10 0x560720eaf810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:20:56.510 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:20:56.478+0000 7fe102800640 1 --2- 172.21.15.12:0/2989450527 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560720eb0290 0x560720ebe890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:20:56.510 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:20:56.478+0000 7fe101400640 1 -- 172.21.15.12:0/2989450527 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560720ea7e10 msgr2=0x560720eaf810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:20:56.510 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:20:56.478+0000 7fe101400640 1 --2- 172.21.15.12:0/2989450527 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560720ea7e10 0x560720eaf810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:20:56.510 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:20:56.478+0000 7fe101400640 1 -- 172.21.15.12:0/2989450527 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560720eb0290 msgr2=0x560720ebe890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:20:56.510 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:20:56.478+0000 7fe101400640 1 --2- 172.21.15.12:0/2989450527 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560720eb0290 0x560720ebe890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:20:56.510 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:20:56.478+0000 7fe101400640 1 -- 172.21.15.12:0/2989450527 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560720b78ba0 con 0x560720eb26e0 2024-09-11T00:20:56.511 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:20:56.478+0000 7fe102800640 1 --2- 172.21.15.12:0/2989450527 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560720eb0290 0x560720ebe890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T00:20:56.511 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:20:56.478+0000 7fe101400640 1 --2- 172.21.15.12:0/2989450527 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560720eb26e0 0x560720eafd50 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fe0f002a930 tx=0x7fe0f00316d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1234f779ddf34b98 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:20:56.511 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:20:56.478+0000 7fe0e3400640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-09-11T00:20:56.511 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:20:56.478+0000 7fe0e3400640 1 -- 172.21.15.12:0/2989450527 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0f003f070 con 0x560720eb26e0 2024-09-11T00:20:56.511 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:20:56.478+0000 7fe0e3400640 1 -- 172.21.15.12:0/2989450527 <== mon.2 v2:172.21.15.165:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9208+0+0 (secure 0 0 0) 0x7fe0f0004240 con 0x560720eb26e0 2024-09-11T00:20:56.511 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:20:56.478+0000 7fe10b68e740 1 -- 172.21.15.12:0/2989450527 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe0f40e6cb0 con 0x560720eb26e0 2024-09-11T00:20:56.511 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:20:56.478+0000 7fe0e3400640 1 -- 172.21.15.12:0/2989450527 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7fe0f0035070 con 0x560720eb26e0 2024-09-11T00:20:56.511 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:20:56.478+0000 7fe10b68e740 10 client.27062 fetch_fsmap learned FSMap version 54 2024-09-11T00:20:56.512 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:20:56.478+0000 7fe10b68e740 1 -- 172.21.15.12:0/2989450527 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) -- 0x560720ec1660 con 0x560720eb26e0 2024-09-11T00:20:56.512 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:20:56.478+0000 7fe0e3400640 1 -- 172.21.15.12:0/2989450527 <== mon.2 v2:172.21.15.165:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe0f40e6cb0 con 0x560720eb26e0 2024-09-11T00:20:56.512 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:20:56.478+0000 7fe0e3400640 1 -- 172.21.15.12:0/2989450527 <== mon.2 v2:172.21.15.165:3300/0 5 ==== fsmap(e 54) ==== 8422+0+0 (secure 0 0 0) 0x7fe0f0004920 con 0x560720eb26e0 2024-09-11T00:20:56.512 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:20:56.478+0000 7fe10b68e740 10 client.27062 fetch_fsmap finished waiting for FSMap version 54 2024-09-11T00:20:56.512 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:20:56.478+0000 7fe10b68e740 10 client.27062 resolve_mds: validated gid 24674 aka daemon mds.b 2024-09-11T00:20:56.512 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:20:56.478+0000 7fe10b68e740 20 client.27062 populate_metadata read hostname 'smithi012' 2024-09-11T00:20:56.512 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:20:56.478+0000 7fe10b68e740 1 --2- 172.21.15.12:0/2989450527 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x560720ec2c40 0x560720ec5060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:20:56.513 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:20:56.478+0000 7fe10b68e740 4 client.27062 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T00:20:56.513 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:20:56.478+0000 7fe10b68e740 1 -- 172.21.15.12:0/2989450527 --> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560720e1d9e0 con 0x560720ec2c40 2024-09-11T00:20:56.513 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:20:56.478+0000 7fe101e00640 1 --2- 172.21.15.12:0/2989450527 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x560720ec2c40 0x560720ec5060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:20:56.513 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:20:56.478+0000 7fe101e00640 1 --2- 172.21.15.12:0/2989450527 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x560720ec2c40 0x560720ec5060 crc :-1 s=READY pgs=205 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2d9a1e340c02091b server_cookie=576898e2ca79bae0 in_seq=0 out_seq=0 2024-09-11T00:20:56.513 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:20:56.478+0000 7fe0e3400640 10 client.27062 ms_handle_connect on v2:172.21.15.165:6838/501312984 2024-09-11T00:20:56.513 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:20:56.483+0000 7fe0e3400640 1 -- 172.21.15.12:0/2989450527 <== mds.0 v2:172.21.15.165:6838/501312984 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x560720e1d9e0 con 0x560720ec2c40 2024-09-11T00:20:56.514 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:20:56.483+0000 7fe0e3400640 10 client.27062 handle_command_reply: tid=0 2024-09-11T00:20:56.514 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:20:56.483+0000 7fe10b68e740 1 client.27062 shutdown 2024-09-11T00:20:56.514 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:20:56.483+0000 7fe10b68e740 1 -- 172.21.15.12:0/2989450527 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560720eb26e0 msgr2=0x560720eafd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:20:56.514 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:20:56.483+0000 7fe10b68e740 1 --2- 172.21.15.12:0/2989450527 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560720eb26e0 0x560720eafd50 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fe0f002a930 tx=0x7fe0f00316d0 comp rx=0 tx=0).stop 2024-09-11T00:20:56.514 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:20:56.483+0000 7fe10b68e740 1 -- 172.21.15.12:0/2989450527 shutdown_connections 2024-09-11T00:20:56.514 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:20:56.483+0000 7fe10b68e740 1 --2- 172.21.15.12:0/2989450527 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x560720ec2c40 0x560720ec5060 crc :-1 s=READY pgs=205 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:20:56.514 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:20:56.483+0000 7fe10b68e740 1 --2- 172.21.15.12:0/2989450527 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560720eb0290 0x560720ebe890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:20:56.514 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:20:56.483+0000 7fe10b68e740 1 --2- 172.21.15.12:0/2989450527 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560720eb26e0 0x560720eafd50 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:20:56.514 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:20:56.483+0000 7fe0e3400640 0 client.27062 ms_handle_reset on v2:172.21.15.165:6838/501312984 2024-09-11T00:20:56.515 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:20:56.483+0000 7fe10b68e740 1 --2- 172.21.15.12:0/2989450527 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560720ea7e10 0x560720eaf810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:20:56.515 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:20:56.483+0000 7fe10b68e740 1 -- 172.21.15.12:0/2989450527 >> 172.21.15.12:0/2989450527 conn(0x560720e706f0 msgr2=0x560720ea9d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:20:56.515 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:20:56.483+0000 7fe10b68e740 1 -- 172.21.15.12:0/2989450527 shutdown_connections 2024-09-11T00:20:56.515 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:20:56.483+0000 7fe10b68e740 1 -- 172.21.15.12:0/2989450527 wait complete. 2024-09-11T00:20:56.515 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:20:56.483+0000 7fe10b68e740 20 client.27062 trim_cache size 0 max 16384 2024-09-11T00:20:56.515 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:20:56.503+0000 7fe10b68e740 1 Processor -- start 2024-09-11T00:20:56.515 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:20:56.503+0000 7fe10b68e740 1 -- start start 2024-09-11T00:20:56.515 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:20:56.503+0000 7fe10b68e740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560720e706f0 0x560720ebe790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:20:56.515 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:20:56.503+0000 7fe10b68e740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560720ea7e10 0x560720eaca20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:20:56.515 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:20:56.503+0000 7fe10b68e740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560720eb0290 0x560720eacf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:20:56.515 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:20:56.503+0000 7fe10b68e740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fe1041b1bf0 con 0x560720ea7e10 2024-09-11T00:20:56.515 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:20:56.503+0000 7fe10b68e740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fe1041b3910 con 0x560720e706f0 2024-09-11T00:20:56.515 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:20:56.503+0000 7fe10b68e740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fe1041b2320 con 0x560720eb0290 2024-09-11T00:20:56.516 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:20:56.503+0000 7fe101400640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560720ea7e10 0x560720eaca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:20:56.516 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:20:56.503+0000 7fe101400640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560720ea7e10 0x560720eaca20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:53916/0 (socket says 172.21.15.12:53916) 2024-09-11T00:20:56.516 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:20:56.503+0000 7fe101400640 1 -- 172.21.15.12:0/180160260 learned_addr learned my addr 172.21.15.12:0/180160260 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:20:56.516 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:20:56.503+0000 7fe101e00640 1 --2- 172.21.15.12:0/180160260 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560720e706f0 0x560720ebe790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:20:56.516 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:20:56.503+0000 7fe102800640 1 --2- 172.21.15.12:0/180160260 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560720eb0290 0x560720eacf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:20:56.517 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:20:56.503+0000 7fe101400640 1 -- 172.21.15.12:0/180160260 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560720eb0290 msgr2=0x560720eacf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:20:56.517 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:20:56.503+0000 7fe101400640 1 --2- 172.21.15.12:0/180160260 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560720eb0290 0x560720eacf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:20:56.518 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:20:56.503+0000 7fe101400640 1 -- 172.21.15.12:0/180160260 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560720e706f0 msgr2=0x560720ebe790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:20:56.518 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:20:56.503+0000 7fe101400640 1 --2- 172.21.15.12:0/180160260 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560720e706f0 0x560720ebe790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:20:56.518 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:20:56.503+0000 7fe101400640 1 -- 172.21.15.12:0/180160260 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560720e958d0 con 0x560720ea7e10 2024-09-11T00:20:56.518 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:20:56.503+0000 7fe101400640 1 --2- 172.21.15.12:0/180160260 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560720ea7e10 0x560720eaca20 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7fe0f0004d70 tx=0x7fe0f0001e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4afb386eba802ef3 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:20:56.518 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:20:56.503+0000 7fe0ee600640 1 -- 172.21.15.12:0/180160260 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0f003f070 con 0x560720ea7e10 2024-09-11T00:20:56.518 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:20:56.503+0000 7fe0ee600640 1 -- 172.21.15.12:0/180160260 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe0f00046c0 con 0x560720ea7e10 2024-09-11T00:20:56.518 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:20:56.503+0000 7fe0ee600640 1 -- 172.21.15.12:0/180160260 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0f00052c0 con 0x560720ea7e10 2024-09-11T00:20:56.518 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:20:56.503+0000 7fe10b68e740 1 -- 172.21.15.12:0/180160260 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560720ea7e10 msgr2=0x560720eaca20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:20:56.518 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:20:56.503+0000 7fe10b68e740 1 --2- 172.21.15.12:0/180160260 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560720ea7e10 0x560720eaca20 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7fe0f0004d70 tx=0x7fe0f0001e70 comp rx=0 tx=0).stop 2024-09-11T00:20:56.518 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:20:56.503+0000 7fe10b68e740 1 -- 172.21.15.12:0/180160260 shutdown_connections 2024-09-11T00:20:56.518 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:20:56.503+0000 7fe10b68e740 1 --2- 172.21.15.12:0/180160260 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560720eb0290 0x560720eacf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:20:56.518 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:20:56.503+0000 7fe10b68e740 1 --2- 172.21.15.12:0/180160260 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560720ea7e10 0x560720eaca20 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:20:56.519 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:20:56.503+0000 7fe10b68e740 1 --2- 172.21.15.12:0/180160260 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560720e706f0 0x560720ebe790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:20:56.519 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:20:56.503+0000 7fe10b68e740 1 -- 172.21.15.12:0/180160260 >> 172.21.15.12:0/180160260 conn(0x560720ea42c0 msgr2=0x560720ea32c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:20:56.519 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:20:56.503+0000 7fe10b68e740 1 -- 172.21.15.12:0/180160260 shutdown_connections 2024-09-11T00:20:56.519 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:20:56.503+0000 7fe10b68e740 1 -- 172.21.15.12:0/180160260 wait complete. 2024-09-11T00:20:56.519 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:20:56.503+0000 7fe10b68e740 1 -- start start 2024-09-11T00:20:56.519 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:20:56.503+0000 7fe10b68e740 1 Processor -- start 2024-09-11T00:20:56.519 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:20:56.503+0000 7fe10b68e740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560720e706f0 0x560720ebae60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:20:56.519 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:20:56.503+0000 7fe10b68e740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560720ea7e10 0x560720eb57b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:20:56.524 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:20:56.503+0000 7fe10b68e740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560720eb0290 0x560720eb5cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:20:56.524 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:20:56.503+0000 7fe10b68e740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fe1041b3240 con 0x560720eb0290 2024-09-11T00:20:56.524 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:20:56.503+0000 7fe10b68e740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fe1041b2ab0 con 0x560720e706f0 2024-09-11T00:20:56.524 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:20:56.503+0000 7fe10b68e740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fe1041afbd0 con 0x560720ea7e10 2024-09-11T00:20:56.525 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:20:56.503+0000 7fe10b68e740 10 client.0 _ll_register_callbacks cb 0x7fe0f4005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:20:56.525 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:20:56.508+0000 7fe101e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560720e706f0 0x560720ebae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:20:56.525 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:20:56.508+0000 7fe101e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560720e706f0 0x560720ebae60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.12:47796/0 (socket says 172.21.15.12:47796) 2024-09-11T00:20:56.525 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:20:56.508+0000 7fe101e00640 1 -- 172.21.15.12:0/1654440787 learned_addr learned my addr 172.21.15.12:0/1654440787 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:20:56.525 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:20:56.508+0000 7fe102800640 1 --2- 172.21.15.12:0/1654440787 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560720eb0290 0x560720eb5cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:20:56.525 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:20:56.508+0000 7fe101400640 1 --2- 172.21.15.12:0/1654440787 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560720ea7e10 0x560720eb57b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:20:56.525 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:20:56.508+0000 7fe102800640 1 -- 172.21.15.12:0/1654440787 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560720ea7e10 msgr2=0x560720eb57b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:20:56.525 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:20:56.508+0000 7fe102800640 1 --2- 172.21.15.12:0/1654440787 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560720ea7e10 0x560720eb57b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:20:56.525 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:20:56.508+0000 7fe102800640 1 -- 172.21.15.12:0/1654440787 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560720e706f0 msgr2=0x560720ebae60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:20:56.526 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:20:56.508+0000 7fe102800640 1 --2- 172.21.15.12:0/1654440787 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560720e706f0 0x560720ebae60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:20:56.526 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:20:56.508+0000 7fe102800640 1 -- 172.21.15.12:0/1654440787 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560720eb63e0 con 0x560720eb0290 2024-09-11T00:20:56.526 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:20:56.508+0000 7fe102800640 1 --2- 172.21.15.12:0/1654440787 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560720eb0290 0x560720eb5cf0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7fe0fc009fd0 tx=0x7fe0fc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e595bb8c7c44d9bf server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:20:56.526 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:20:56.508+0000 7fe0e3400640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-09-11T00:20:56.526 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:20:56.510+0000 7fe0e3400640 1 -- 172.21.15.12:0/1654440787 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0fc020e50 con 0x560720eb0290 2024-09-11T00:20:56.526 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:20:56.510+0000 7fe10b68e740 1 -- 172.21.15.12:0/1654440787 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe0f00046c0 con 0x560720eb0290 2024-09-11T00:20:56.526 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:20:56.510+0000 7fe0e3400640 1 -- 172.21.15.12:0/1654440787 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9208+0+0 (secure 0 0 0) 0x7fe0fc0283e0 con 0x560720eb0290 2024-09-11T00:20:56.526 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:20:56.510+0000 7fe0e3400640 1 -- 172.21.15.12:0/1654440787 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7fe0fc0285e0 con 0x560720eb0290 2024-09-11T00:20:56.526 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:20:56.510+0000 7fe10b68e740 10 client.17043 fetch_fsmap learned FSMap version 54 2024-09-11T00:20:56.526 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:20:56.510+0000 7fe10b68e740 1 -- 172.21.15.12:0/1654440787 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) -- 0x560720f392f0 con 0x560720eb0290 2024-09-11T00:20:56.526 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:20:56.510+0000 7fe0e3400640 1 -- 172.21.15.12:0/1654440787 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe0f00046c0 con 0x560720eb0290 2024-09-11T00:20:56.526 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:20:56.510+0000 7fe0e3400640 1 -- 172.21.15.12:0/1654440787 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 54) ==== 8422+0+0 (secure 0 0 0) 0x7fe0fc027070 con 0x560720eb0290 2024-09-11T00:20:56.526 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:20:56.510+0000 7fe10b68e740 10 client.17043 fetch_fsmap finished waiting for FSMap version 54 2024-09-11T00:20:56.526 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:20:56.510+0000 7fe10b68e740 10 client.17043 resolve_mds: validated gid 24674 aka daemon mds.b 2024-09-11T00:20:56.527 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:20:56.510+0000 7fe10b68e740 20 client.17043 populate_metadata read hostname 'smithi012' 2024-09-11T00:20:56.527 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:20:56.510+0000 7fe10b68e740 1 --2- 172.21.15.12:0/1654440787 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x560720eb26e0 0x560720f3bb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:20:56.527 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:20:56.510+0000 7fe10b68e740 4 client.17043 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 142.75, "await_for": 45.0}] 2024-09-11T00:20:56.527 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:20:56.510+0000 7fe10b68e740 1 -- 172.21.15.12:0/1654440787 --> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 142.75, "await_for": 45.0}) -- 0x560720e19490 con 0x560720eb26e0 2024-09-11T00:20:56.527 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:20:56.510+0000 7fe101e00640 1 --2- 172.21.15.12:0/1654440787 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x560720eb26e0 0x560720f3bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:20:56.527 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:20:56.510+0000 7fe101e00640 1 --2- 172.21.15.12:0/1654440787 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x560720eb26e0 0x560720f3bb80 crc :-1 s=READY pgs=206 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=295ae32d7686a15f server_cookie=cc225656141e58bd in_seq=0 out_seq=0 2024-09-11T00:20:56.527 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:20:56.510+0000 7fe0e3400640 10 client.17043 ms_handle_connect on v2:172.21.15.165:6838/501312984 2024-09-11T00:20:57.144 INFO:tasks.workunit.client.0.smithi012.stdout: 10 27889 31.61 MB/sec warmup 53 sec latency 3551.950 ms 2024-09-11T00:20:58.144 INFO:tasks.workunit.client.0.smithi012.stdout: 10 27889 31.03 MB/sec warmup 54 sec latency 4552.014 ms 2024-09-11T00:20:58.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:20:57 smithi165 ceph-mon[35356]: pgmap v815: 129 pgs: 129 active+clean; 415 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 MiB/s rd, 9.6 MiB/s wr, 849 op/s 2024-09-11T00:20:58.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:20:57 smithi012 ceph-mon[30333]: pgmap v815: 129 pgs: 129 active+clean; 415 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 MiB/s rd, 9.6 MiB/s wr, 849 op/s 2024-09-11T00:20:58.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:20:57 smithi178 ceph-mon[35919]: pgmap v815: 129 pgs: 129 active+clean; 415 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 MiB/s rd, 9.6 MiB/s wr, 849 op/s 2024-09-11T00:20:59.144 INFO:tasks.workunit.client.0.smithi012.stdout: 10 27889 30.46 MB/sec warmup 55 sec latency 5552.062 ms 2024-09-11T00:21:00.144 INFO:tasks.workunit.client.0.smithi012.stdout: 10 27889 29.92 MB/sec warmup 56 sec latency 6552.112 ms 2024-09-11T00:21:00.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:20:59 smithi165 ceph-mon[35356]: pgmap v816: 129 pgs: 129 active+clean; 416 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 8.1 MiB/s wr, 621 op/s 2024-09-11T00:21:00.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:20:59 smithi012 ceph-mon[30333]: pgmap v816: 129 pgs: 129 active+clean; 416 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 8.1 MiB/s wr, 621 op/s 2024-09-11T00:21:00.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:20:59 smithi178 ceph-mon[35919]: pgmap v816: 129 pgs: 129 active+clean; 416 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 8.1 MiB/s wr, 621 op/s 2024-09-11T00:21:01.144 INFO:tasks.workunit.client.0.smithi012.stdout: 10 27889 29.39 MB/sec warmup 57 sec latency 7552.165 ms 2024-09-11T00:21:02.072 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:21:01 smithi012 ceph-mon[30333]: pgmap v817: 129 pgs: 129 active+clean; 430 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 11 MiB/s wr, 742 op/s 2024-09-11T00:21:02.144 INFO:tasks.workunit.client.0.smithi012.stdout: 10 27890 28.89 MB/sec warmup 58 sec latency 8545.471 ms 2024-09-11T00:21:02.285 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:21:01 smithi165 ceph-mon[35356]: pgmap v817: 129 pgs: 129 active+clean; 430 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 11 MiB/s wr, 742 op/s 2024-09-11T00:21:02.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:21:01 smithi178 ceph-mon[35919]: pgmap v817: 129 pgs: 129 active+clean; 430 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 11 MiB/s wr, 742 op/s 2024-09-11T00:21:03.144 INFO:tasks.workunit.client.0.smithi012.stdout: 10 27890 28.40 MB/sec warmup 59 sec latency 9545.521 ms 2024-09-11T00:21:04.062 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:21:03 smithi012 ceph-mon[30333]: pgmap v818: 129 pgs: 129 active+clean; 442 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 10 MiB/s wr, 633 op/s 2024-09-11T00:21:04.144 INFO:tasks.workunit.client.0.smithi012.stdout: 10 27890 27.93 MB/sec warmup 60 sec latency 10545.572 ms 2024-09-11T00:21:04.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:21:03 smithi165 ceph-mon[35356]: pgmap v818: 129 pgs: 129 active+clean; 442 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 10 MiB/s wr, 633 op/s 2024-09-11T00:21:04.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:21:03 smithi178 ceph-mon[35919]: pgmap v818: 129 pgs: 129 active+clean; 442 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 10 MiB/s wr, 633 op/s 2024-09-11T00:21:05.144 INFO:tasks.workunit.client.0.smithi012.stdout: 10 27890 27.47 MB/sec warmup 61 sec latency 11545.621 ms 2024-09-11T00:21:06.144 INFO:tasks.workunit.client.0.smithi012.stdout: 10 27890 27.03 MB/sec warmup 62 sec latency 12545.674 ms 2024-09-11T00:21:06.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:21:05 smithi165 ceph-mon[35356]: pgmap v819: 129 pgs: 129 active+clean; 444 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 9.5 MiB/s wr, 471 op/s 2024-09-11T00:21:06.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:21:05 smithi012 ceph-mon[30333]: pgmap v819: 129 pgs: 129 active+clean; 444 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 9.5 MiB/s wr, 471 op/s 2024-09-11T00:21:06.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:21:05 smithi178 ceph-mon[35919]: pgmap v819: 129 pgs: 129 active+clean; 444 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 9.5 MiB/s wr, 471 op/s 2024-09-11T00:21:07.144 INFO:tasks.workunit.client.0.smithi012.stdout: 10 27890 26.60 MB/sec warmup 63 sec latency 13545.728 ms 2024-09-11T00:21:08.067 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:08.066+0000 7f3b51800640 1 -- 172.21.15.12:0/1238207425 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3b4c102990 msgr2=0x7f3b4c102d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:21:08.068 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:08.066+0000 7f3b51800640 1 --2- 172.21.15.12:0/1238207425 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3b4c102990 0x7f3b4c102d90 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f3b3c009900 tx=0x7f3b3c02a400 comp rx=0 tx=0).stop 2024-09-11T00:21:08.068 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:08.066+0000 7f3b51800640 1 -- 172.21.15.12:0/1238207425 shutdown_connections 2024-09-11T00:21:08.068 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:08.066+0000 7f3b51800640 1 --2- 172.21.15.12:0/1238207425 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3b4c1044a0 0x7f3b4c10ae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:21:08.068 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:08.066+0000 7f3b51800640 1 --2- 172.21.15.12:0/1238207425 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3b4c103ae0 0x7f3b4c103f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:21:08.068 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:08.066+0000 7f3b51800640 1 --2- 172.21.15.12:0/1238207425 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3b4c102990 0x7f3b4c102d90 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:21:08.068 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:08.066+0000 7f3b51800640 1 -- 172.21.15.12:0/1238207425 >> 172.21.15.12:0/1238207425 conn(0x7f3b4c0fe250 msgr2=0x7f3b4c100670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:21:08.068 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:08.066+0000 7f3b51800640 1 -- 172.21.15.12:0/1238207425 shutdown_connections 2024-09-11T00:21:08.068 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:08.066+0000 7f3b51800640 1 -- 172.21.15.12:0/1238207425 wait complete. 2024-09-11T00:21:08.069 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:08.066+0000 7f3b51800640 1 Processor -- start 2024-09-11T00:21:08.069 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:08.066+0000 7f3b51800640 1 -- start start 2024-09-11T00:21:08.069 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:08.066+0000 7f3b51800640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3b4c102990 0x7f3b4c076ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:21:08.069 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:08.066+0000 7f3b51800640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3b4c103ae0 0x7f3b4c075540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:21:08.069 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:08.066+0000 7f3b51800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3b4c1044a0 0x7f3b4c075a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:21:08.069 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:08.066+0000 7f3b51800640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f3b4c10e600 con 0x7f3b4c102990 2024-09-11T00:21:08.082 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:08.066+0000 7f3b51800640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f3b4c10e480 con 0x7f3b4c1044a0 2024-09-11T00:21:08.082 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:08.066+0000 7f3b51800640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f3b4c10e780 con 0x7f3b4c103ae0 2024-09-11T00:21:08.082 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:08.066+0000 7f3b4aa00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3b4c102990 0x7f3b4c076ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:21:08.082 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:08.066+0000 7f3b43e00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3b4c103ae0 0x7f3b4c075540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:21:08.082 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:08.066+0000 7f3b43e00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3b4c103ae0 0x7f3b4c075540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.12:37206/0 (socket says 172.21.15.12:37206) 2024-09-11T00:21:08.082 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:08.066+0000 7f3b4b400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3b4c1044a0 0x7f3b4c075a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:21:08.082 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:08.066+0000 7f3b43e00640 1 -- 172.21.15.12:0/3196611564 learned_addr learned my addr 172.21.15.12:0/3196611564 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:21:08.083 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:08.066+0000 7f3b43e00640 1 -- 172.21.15.12:0/3196611564 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3b4c1044a0 msgr2=0x7f3b4c075a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:21:08.083 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:08.066+0000 7f3b43e00640 1 --2- 172.21.15.12:0/3196611564 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3b4c1044a0 0x7f3b4c075a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:21:08.083 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:08.066+0000 7f3b43e00640 1 -- 172.21.15.12:0/3196611564 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3b4c102990 msgr2=0x7f3b4c076ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:21:08.083 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:08.066+0000 7f3b43e00640 1 --2- 172.21.15.12:0/3196611564 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3b4c102990 0x7f3b4c076ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:21:08.083 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:08.066+0000 7f3b43e00640 1 -- 172.21.15.12:0/3196611564 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b4c0762f0 con 0x7f3b4c103ae0 2024-09-11T00:21:08.083 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:08.066+0000 7f3b43e00640 1 --2- 172.21.15.12:0/3196611564 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3b4c103ae0 0x7f3b4c075540 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f3b3400e9f0 tx=0x7f3b3400eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:21:08.083 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:08.066+0000 7f3b43400640 1 -- 172.21.15.12:0/3196611564 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b3400cd80 con 0x7f3b4c103ae0 2024-09-11T00:21:08.083 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:08.066+0000 7f3b43400640 1 -- 172.21.15.12:0/3196611564 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3b340108e0 con 0x7f3b4c103ae0 2024-09-11T00:21:08.083 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:08.066+0000 7f3b43400640 1 -- 172.21.15.12:0/3196611564 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b34018980 con 0x7f3b4c103ae0 2024-09-11T00:21:08.083 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:08.066+0000 7f3b51800640 1 -- 172.21.15.12:0/3196611564 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3b4c1a2e60 con 0x7f3b4c103ae0 2024-09-11T00:21:08.083 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:08.066+0000 7f3b51800640 1 -- 172.21.15.12:0/3196611564 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3b4c1a3320 con 0x7f3b4c103ae0 2024-09-11T00:21:08.083 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:08.073+0000 7f3b5462d740 1 Processor -- start 2024-09-11T00:21:08.084 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:08.073+0000 7f3b5462d740 1 -- start start 2024-09-11T00:21:08.084 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:08.073+0000 7f3b5462d740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e9f670f790 0x55e9f6711b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:21:08.084 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:08.073+0000 7f3b5462d740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e9f67120c0 0x55e9f67144b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:21:08.084 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:08.073+0000 7f3b5462d740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e9f6714b00 0x55e9f6716fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:21:08.084 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:08.073+0000 7f3b5462d740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55e9f66826b0 con 0x55e9f670f790 2024-09-11T00:21:08.084 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:08.073+0000 7f3b5462d740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55e9f6671ae0 con 0x55e9f67120c0 2024-09-11T00:21:08.084 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:08.073+0000 7f3b5462d740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55e9f66758f0 con 0x55e9f6714b00 2024-09-11T00:21:08.084 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:08.073+0000 7f3b43400640 1 -- 172.21.15.12:0/3196611564 <== mon.1 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7f3b34010430 con 0x7f3b4c103ae0 2024-09-11T00:21:08.084 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:08.073+0000 7f3b4b400640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e9f670f790 0x55e9f6711b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:21:08.084 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:08.073+0000 7f3b4b400640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e9f670f790 0x55e9f6711b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:32982/0 (socket says 172.21.15.12:32982) 2024-09-11T00:21:08.084 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:08.073+0000 7f3b4b400640 1 -- 172.21.15.12:0/1178039566 learned_addr learned my addr 172.21.15.12:0/1178039566 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:21:08.084 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:08.073+0000 7f3b43e00640 1 --2- 172.21.15.12:0/1178039566 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e9f6714b00 0x55e9f6716fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:21:08.085 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:08.073+0000 7f3b4aa00640 1 --2- 172.21.15.12:0/1178039566 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e9f67120c0 0x55e9f67144b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:21:08.085 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:08.073+0000 7f3b4b400640 1 -- 172.21.15.12:0/1178039566 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e9f6714b00 msgr2=0x55e9f6716fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:21:08.085 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:08.073+0000 7f3b4b400640 1 --2- 172.21.15.12:0/1178039566 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e9f6714b00 0x55e9f6716fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:21:08.085 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:08.073+0000 7f3b4b400640 1 -- 172.21.15.12:0/1178039566 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e9f67120c0 msgr2=0x55e9f67144b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:21:08.085 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:08.073+0000 7f3b4b400640 1 --2- 172.21.15.12:0/1178039566 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e9f67120c0 0x55e9f67144b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:21:08.085 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:08.073+0000 7f3b4b400640 1 -- 172.21.15.12:0/1178039566 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e9f663e780 con 0x55e9f670f790 2024-09-11T00:21:08.085 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:08.073+0000 7f3b4b400640 1 --2- 172.21.15.12:0/1178039566 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e9f670f790 0x55e9f6711b80 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f3b44009fd0 tx=0x7f3b440118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fcf3638eaa50c9f0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:21:08.085 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:08.073+0000 7f3b3be00640 1 -- 172.21.15.12:0/1178039566 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b44004d20 con 0x55e9f670f790 2024-09-11T00:21:08.085 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:08.073+0000 7f3b3be00640 1 -- 172.21.15.12:0/1178039566 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3b44004050 con 0x55e9f670f790 2024-09-11T00:21:08.085 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:08.073+0000 7f3b3be00640 1 -- 172.21.15.12:0/1178039566 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b44005240 con 0x55e9f670f790 2024-09-11T00:21:08.085 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:08.073+0000 7f3b5462d740 1 -- 172.21.15.12:0/1178039566 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e9f670f790 msgr2=0x55e9f6711b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:21:08.085 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:08.073+0000 7f3b5462d740 1 --2- 172.21.15.12:0/1178039566 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e9f670f790 0x55e9f6711b80 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f3b44009fd0 tx=0x7f3b440118d0 comp rx=0 tx=0).stop 2024-09-11T00:21:08.085 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:08.073+0000 7f3b43400640 1 --2- 172.21.15.12:0/3196611564 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f3b300b52e0 0x7f3b300b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:21:08.086 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:08.076+0000 7f3b5462d740 1 -- 172.21.15.12:0/1178039566 shutdown_connections 2024-09-11T00:21:08.086 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:08.076+0000 7f3b5462d740 1 --2- 172.21.15.12:0/1178039566 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e9f6714b00 0x55e9f6716fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:21:08.086 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:08.076+0000 7f3b5462d740 1 --2- 172.21.15.12:0/1178039566 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e9f67120c0 0x55e9f67144b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:21:08.086 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:08.076+0000 7f3b5462d740 1 --2- 172.21.15.12:0/1178039566 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e9f670f790 0x55e9f6711b80 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:21:08.086 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:08.076+0000 7f3b5462d740 1 -- 172.21.15.12:0/1178039566 >> 172.21.15.12:0/1178039566 conn(0x55e9f670a8e0 msgr2=0x55e9f670b670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:21:08.086 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:08.076+0000 7f3b43400640 1 -- 172.21.15.12:0/3196611564 <== mon.1 v2:172.21.15.178:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9208+0+0 (secure 0 0 0) 0x7f3b340e6350 con 0x7f3b4c103ae0 2024-09-11T00:21:08.086 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:08.076+0000 7f3b4aa00640 1 --2- 172.21.15.12:0/3196611564 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f3b300b52e0 0x7f3b300b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:21:08.086 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:08.077+0000 7f3b5462d740 1 -- 172.21.15.12:0/1178039566 shutdown_connections 2024-09-11T00:21:08.086 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:08.077+0000 7f3b5462d740 1 -- 172.21.15.12:0/1178039566 wait complete. 2024-09-11T00:21:08.086 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:08.077+0000 7f3b5462d740 1 -- start start 2024-09-11T00:21:08.086 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:08.077+0000 7f3b4aa00640 1 --2- 172.21.15.12:0/3196611564 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f3b300b52e0 0x7f3b300b77a0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f3b3c006010 tx=0x7f3b3c031430 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:21:08.086 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:08.077+0000 7f3b5462d740 1 Processor -- start 2024-09-11T00:21:08.086 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:08.077+0000 7f3b5462d740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e9f670f790 0x55e9f6721690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:21:08.086 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:08.077+0000 7f3b5462d740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e9f67120c0 0x55e9f6721d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:21:08.087 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:08.077+0000 7f3b5462d740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e9f6714b00 0x55e9f6722900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:21:08.087 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:08.078+0000 7f3b5462d740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55e9f65a9ab0 con 0x55e9f67120c0 2024-09-11T00:21:08.087 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:08.078+0000 7f3b5462d740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55e9f6618570 con 0x55e9f670f790 2024-09-11T00:21:08.087 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:08.078+0000 7f3b5462d740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55e9f6717ae0 con 0x55e9f6714b00 2024-09-11T00:21:08.087 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:08.078+0000 7f3b5462d740 10 client.0 _ll_register_callbacks cb 0x55e9f6627100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:21:08.087 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:08.078+0000 7f3b4aa00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e9f670f790 0x55e9f6721690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:21:08.087 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:08.078+0000 7f3b4aa00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e9f670f790 0x55e9f6721690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.12:38026/0 (socket says 172.21.15.12:38026) 2024-09-11T00:21:08.087 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:08.078+0000 7f3b4aa00640 1 -- 172.21.15.12:0/627699119 learned_addr learned my addr 172.21.15.12:0/627699119 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:21:08.087 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:08.078+0000 7f3b43e00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e9f67120c0 0x55e9f6721d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:21:08.087 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:08.078+0000 7f3b43e00640 1 -- 172.21.15.12:0/627699119 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e9f6714b00 msgr2=0x55e9f6722900 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-11T00:21:08.087 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:08.078+0000 7f3b43e00640 1 --2- 172.21.15.12:0/627699119 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e9f6714b00 0x55e9f6722900 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:21:08.087 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:08.079+0000 7f3b43e00640 1 -- 172.21.15.12:0/627699119 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e9f670f790 msgr2=0x55e9f6721690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:21:08.087 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:08.079+0000 7f3b43e00640 1 --2- 172.21.15.12:0/627699119 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e9f670f790 0x55e9f6721690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:21:08.088 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:08.079+0000 7f3b43e00640 1 -- 172.21.15.12:0/627699119 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e9f63d3790 con 0x55e9f67120c0 2024-09-11T00:21:08.088 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:08.079+0000 7f3b4aa00640 1 --2- 172.21.15.12:0/627699119 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e9f670f790 0x55e9f6721690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:21:08.088 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:08.079+0000 7f3b43e00640 1 --2- 172.21.15.12:0/627699119 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e9f67120c0 0x55e9f6721d30 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f3b3400e9c0 tx=0x7f3b34018420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb5b794f1787deeb server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:21:08.088 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:08.079+0000 7f3b38c00640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-09-11T00:21:08.088 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:08.079+0000 7f3b38c00640 1 -- 172.21.15.12:0/627699119 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b340ac560 con 0x55e9f67120c0 2024-09-11T00:21:08.090 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:08.079+0000 7f3b38c00640 1 -- 172.21.15.12:0/627699119 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9208+0+0 (secure 0 0 0) 0x7f3b340a80e0 con 0x55e9f67120c0 2024-09-11T00:21:08.091 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:08.079+0000 7f3b5462d740 1 -- 172.21.15.12:0/627699119 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3b44004050 con 0x55e9f67120c0 2024-09-11T00:21:08.091 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:08.079+0000 7f3b38c00640 1 -- 172.21.15.12:0/627699119 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7f3b340abf00 con 0x55e9f67120c0 2024-09-11T00:21:08.091 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:08.079+0000 7f3b5462d740 10 client.17055 fetch_fsmap learned FSMap version 54 2024-09-11T00:21:08.091 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:08.079+0000 7f3b5462d740 1 -- 172.21.15.12:0/627699119 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) -- 0x55e9f644c620 con 0x55e9f67120c0 2024-09-11T00:21:08.091 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:08.079+0000 7f3b38c00640 1 -- 172.21.15.12:0/627699119 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3b44004050 con 0x55e9f67120c0 2024-09-11T00:21:08.091 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:08.079+0000 7f3b38c00640 1 -- 172.21.15.12:0/627699119 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 54) ==== 8422+0+0 (secure 0 0 0) 0x7f3b340a82e0 con 0x55e9f67120c0 2024-09-11T00:21:08.091 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:08.079+0000 7f3b5462d740 10 client.17055 fetch_fsmap finished waiting for FSMap version 54 2024-09-11T00:21:08.091 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:08.079+0000 7f3b5462d740 10 client.17055 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T00:21:08.091 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:08.079+0000 7f3b5462d740 20 client.17055 populate_metadata read hostname 'smithi012' 2024-09-11T00:21:08.091 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:08.079+0000 7f3b5462d740 1 --2- 172.21.15.12:0/627699119 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55e9f672d050 0x55e9f672f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:21:08.091 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:08.079+0000 7f3b5462d740 4 client.17055 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T00:21:08.092 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:08.079+0000 7f3b5462d740 1 -- 172.21.15.12:0/627699119 --> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e9f6684600 con 0x55e9f672d050 2024-09-11T00:21:08.102 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:08.101+0000 7f3b4aa00640 1 --2- 172.21.15.12:0/627699119 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55e9f672d050 0x55e9f672f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:21:08.102 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:21:07 smithi012 ceph-mon[30333]: pgmap v820: 129 pgs: 129 active+clean; 453 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 9.5 MiB/s wr, 475 op/s 2024-09-11T00:21:08.144 INFO:tasks.workunit.client.0.smithi012.stdout: 10 27890 26.18 MB/sec warmup 64 sec latency 14545.781 ms 2024-09-11T00:21:08.161 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:08.160+0000 7f3b4aa00640 1 --2- 172.21.15.12:0/627699119 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55e9f672d050 0x55e9f672f4c0 crc :-1 s=READY pgs=207 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=853577775d98c194 server_cookie=a19c65c4bf4bab43 in_seq=0 out_seq=0 2024-09-11T00:21:08.161 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:08.160+0000 7f3b38c00640 10 client.17055 ms_handle_connect on v2:172.21.15.165:6838/501312984 2024-09-11T00:21:08.186 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:08.184+0000 7f3b38c00640 1 -- 172.21.15.12:0/627699119 <== mds.0 v2:172.21.15.165:6838/501312984 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55e9f6684600 con 0x55e9f672d050 2024-09-11T00:21:08.186 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:08.184+0000 7f3b38c00640 10 client.17055 handle_command_reply: tid=0 2024-09-11T00:21:08.186 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:08.184+0000 7f3b5462d740 1 client.17055 shutdown 2024-09-11T00:21:08.186 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:08.184+0000 7f3b5462d740 1 -- 172.21.15.12:0/627699119 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e9f67120c0 msgr2=0x55e9f6721d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:21:08.186 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:08.184+0000 7f3b5462d740 1 --2- 172.21.15.12:0/627699119 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e9f67120c0 0x55e9f6721d30 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f3b3400e9c0 tx=0x7f3b34018420 comp rx=0 tx=0).stop 2024-09-11T00:21:08.186 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:08.184+0000 7f3b5462d740 1 -- 172.21.15.12:0/627699119 shutdown_connections 2024-09-11T00:21:08.186 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:08.184+0000 7f3b5462d740 1 --2- 172.21.15.12:0/627699119 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55e9f672d050 0x55e9f672f4c0 crc :-1 s=READY pgs=207 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:21:08.186 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:08.184+0000 7f3b5462d740 1 --2- 172.21.15.12:0/627699119 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e9f6714b00 0x55e9f6722900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:21:08.186 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:08.184+0000 7f3b5462d740 1 --2- 172.21.15.12:0/627699119 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e9f67120c0 0x55e9f6721d30 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:21:08.186 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:08.184+0000 7f3b38c00640 0 client.17055 ms_handle_reset on v2:172.21.15.165:6838/501312984 2024-09-11T00:21:08.186 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:08.184+0000 7f3b5462d740 1 --2- 172.21.15.12:0/627699119 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e9f670f790 0x55e9f6721690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:21:08.186 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:08.184+0000 7f3b5462d740 1 -- 172.21.15.12:0/627699119 >> 172.21.15.12:0/627699119 conn(0x55e9f670a8e0 msgr2=0x55e9f67124a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:21:08.187 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:08.184+0000 7f3b5462d740 1 -- 172.21.15.12:0/627699119 shutdown_connections 2024-09-11T00:21:08.187 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:08.184+0000 7f3b5462d740 1 -- 172.21.15.12:0/627699119 wait complete. 2024-09-11T00:21:08.187 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:08.184+0000 7f3b5462d740 20 client.17055 trim_cache size 0 max 16384 2024-09-11T00:21:08.202 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:08.200+0000 7f3b5462d740 1 Processor -- start 2024-09-11T00:21:08.202 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:08.200+0000 7f3b5462d740 1 -- start start 2024-09-11T00:21:08.202 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:08.200+0000 7f3b5462d740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e9f670f790 0x55e9f67134b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:21:08.202 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:08.200+0000 7f3b5462d740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e9f67120c0 0x55e9f67139f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:21:08.202 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:08.200+0000 7f3b5462d740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e9f6714b00 0x55e9f670c730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:21:08.203 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:08.200+0000 7f3b5462d740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f3b4c1a9510 con 0x55e9f67120c0 2024-09-11T00:21:08.203 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:08.200+0000 7f3b5462d740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f3b4c1ab230 con 0x55e9f670f790 2024-09-11T00:21:08.203 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:08.200+0000 7f3b5462d740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f3b4c1a9c40 con 0x55e9f6714b00 2024-09-11T00:21:08.203 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:08.200+0000 7f3b43e00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e9f67120c0 0x55e9f67139f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:21:08.203 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:08.200+0000 7f3b43e00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e9f67120c0 0x55e9f67139f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:32998/0 (socket says 172.21.15.12:32998) 2024-09-11T00:21:08.203 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:08.200+0000 7f3b43e00640 1 -- 172.21.15.12:0/3263963768 learned_addr learned my addr 172.21.15.12:0/3263963768 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:21:08.203 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:08.200+0000 7f3b4aa00640 1 --2- 172.21.15.12:0/3263963768 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e9f670f790 0x55e9f67134b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:21:08.203 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:08.200+0000 7f3b4b400640 1 --2- 172.21.15.12:0/3263963768 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e9f6714b00 0x55e9f670c730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:21:08.203 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:08.200+0000 7f3b43e00640 1 -- 172.21.15.12:0/3263963768 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e9f6714b00 msgr2=0x55e9f670c730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:21:08.203 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:08.200+0000 7f3b43e00640 1 --2- 172.21.15.12:0/3263963768 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e9f6714b00 0x55e9f670c730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:21:08.203 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:08.200+0000 7f3b43e00640 1 -- 172.21.15.12:0/3263963768 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e9f670f790 msgr2=0x55e9f67134b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:21:08.203 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:08.200+0000 7f3b43e00640 1 --2- 172.21.15.12:0/3263963768 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e9f670f790 0x55e9f67134b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:21:08.203 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:08.200+0000 7f3b43e00640 1 -- 172.21.15.12:0/3263963768 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e9f6711ef0 con 0x55e9f67120c0 2024-09-11T00:21:08.204 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:08.200+0000 7f3b4aa00640 1 --2- 172.21.15.12:0/3263963768 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e9f670f790 0x55e9f67134b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:21:08.207 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:08.200+0000 7f3b43e00640 1 --2- 172.21.15.12:0/3263963768 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e9f67120c0 0x55e9f67139f0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f3b340a8070 tx=0x7f3b340ab930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b947380480434a0a server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:21:08.207 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:08.200+0000 7f3b3b400640 1 -- 172.21.15.12:0/3263963768 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b340ac0e0 con 0x55e9f67120c0 2024-09-11T00:21:08.207 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:08.200+0000 7f3b3b400640 1 -- 172.21.15.12:0/3263963768 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3b340ac2a0 con 0x55e9f67120c0 2024-09-11T00:21:08.208 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:08.200+0000 7f3b3b400640 1 -- 172.21.15.12:0/3263963768 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b340b25e0 con 0x55e9f67120c0 2024-09-11T00:21:08.208 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:08.200+0000 7f3b5462d740 1 -- 172.21.15.12:0/3263963768 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e9f67120c0 msgr2=0x55e9f67139f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:21:08.208 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:08.200+0000 7f3b5462d740 1 --2- 172.21.15.12:0/3263963768 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e9f67120c0 0x55e9f67139f0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f3b340a8070 tx=0x7f3b340ab930 comp rx=0 tx=0).stop 2024-09-11T00:21:08.208 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:08.200+0000 7f3b5462d740 1 -- 172.21.15.12:0/3263963768 shutdown_connections 2024-09-11T00:21:08.208 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:08.200+0000 7f3b5462d740 1 --2- 172.21.15.12:0/3263963768 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e9f6714b00 0x55e9f670c730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:21:08.208 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:08.200+0000 7f3b5462d740 1 --2- 172.21.15.12:0/3263963768 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e9f67120c0 0x55e9f67139f0 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:21:08.208 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:08.200+0000 7f3b5462d740 1 --2- 172.21.15.12:0/3263963768 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e9f670f790 0x55e9f67134b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:21:08.208 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:08.200+0000 7f3b5462d740 1 -- 172.21.15.12:0/3263963768 >> 172.21.15.12:0/3263963768 conn(0x55e9f670a8e0 msgr2=0x55e9f670ec40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:21:08.208 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:08.200+0000 7f3b5462d740 1 -- 172.21.15.12:0/3263963768 shutdown_connections 2024-09-11T00:21:08.208 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:08.200+0000 7f3b5462d740 1 -- 172.21.15.12:0/3263963768 wait complete. 2024-09-11T00:21:08.208 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:08.200+0000 7f3b5462d740 1 -- start start 2024-09-11T00:21:08.208 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:08.200+0000 7f3b5462d740 1 Processor -- start 2024-09-11T00:21:08.209 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:08.200+0000 7f3b5462d740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e9f670f790 0x55e9f6728440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:21:08.209 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:08.200+0000 7f3b5462d740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e9f67120c0 0x55e9f6728980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:21:08.209 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:08.200+0000 7f3b5462d740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e9f6714b00 0x55e9f6726fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:21:08.209 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:08.200+0000 7f3b5462d740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f3b4c1aab60 con 0x55e9f670f790 2024-09-11T00:21:08.209 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:08.200+0000 7f3b5462d740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f3b4c1aa3d0 con 0x55e9f67120c0 2024-09-11T00:21:08.209 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:08.200+0000 7f3b5462d740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f3b4c1a7510 con 0x55e9f6714b00 2024-09-11T00:21:08.209 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:08.200+0000 7f3b5462d740 10 client.0 _ll_register_callbacks cb 0x7f3b34009700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:21:08.209 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:08.200+0000 7f3b4aa00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e9f670f790 0x55e9f6728440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:21:08.209 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:08.200+0000 7f3b4aa00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e9f670f790 0x55e9f6728440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:33008/0 (socket says 172.21.15.12:33008) 2024-09-11T00:21:08.209 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:08.200+0000 7f3b4aa00640 1 -- 172.21.15.12:0/2236269309 learned_addr learned my addr 172.21.15.12:0/2236269309 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:21:08.209 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:08.200+0000 7f3b43e00640 1 --2- 172.21.15.12:0/2236269309 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e9f67120c0 0x55e9f6728980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:21:08.209 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:08.200+0000 7f3b4b400640 1 --2- 172.21.15.12:0/2236269309 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e9f6714b00 0x55e9f6726fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:21:08.210 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:08.200+0000 7f3b4aa00640 1 -- 172.21.15.12:0/2236269309 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e9f6714b00 msgr2=0x55e9f6726fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:21:08.210 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:08.200+0000 7f3b4aa00640 1 --2- 172.21.15.12:0/2236269309 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e9f6714b00 0x55e9f6726fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:21:08.210 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:08.200+0000 7f3b4aa00640 1 -- 172.21.15.12:0/2236269309 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e9f67120c0 msgr2=0x55e9f6728980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:21:08.210 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:08.200+0000 7f3b4aa00640 1 --2- 172.21.15.12:0/2236269309 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e9f67120c0 0x55e9f6728980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:21:08.210 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:08.200+0000 7f3b4aa00640 1 -- 172.21.15.12:0/2236269309 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e9f6729230 con 0x55e9f670f790 2024-09-11T00:21:08.210 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:08.205+0000 7f3b4aa00640 1 --2- 172.21.15.12:0/2236269309 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e9f670f790 0x55e9f6728440 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f3b3c005ab0 tx=0x7f3b3c009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8b2437fdc1571c4 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:21:08.210 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:08.205+0000 7f3b38c00640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-09-11T00:21:08.210 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:08.205+0000 7f3b38c00640 1 -- 172.21.15.12:0/2236269309 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b3c035040 con 0x55e9f670f790 2024-09-11T00:21:08.210 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:08.205+0000 7f3b38c00640 1 -- 172.21.15.12:0/2236269309 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9208+0+0 (secure 0 0 0) 0x7f3b3c003c00 con 0x55e9f670f790 2024-09-11T00:21:08.210 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:08.205+0000 7f3b5462d740 1 -- 172.21.15.12:0/2236269309 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3b340ac2a0 con 0x55e9f670f790 2024-09-11T00:21:08.210 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:08.205+0000 7f3b38c00640 1 -- 172.21.15.12:0/2236269309 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7f3b3c062050 con 0x55e9f670f790 2024-09-11T00:21:08.210 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:08.205+0000 7f3b5462d740 10 client.17061 fetch_fsmap learned FSMap version 54 2024-09-11T00:21:08.211 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:08.205+0000 7f3b5462d740 1 -- 172.21.15.12:0/2236269309 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) -- 0x55e9f679a680 con 0x55e9f670f790 2024-09-11T00:21:08.211 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:08.205+0000 7f3b38c00640 1 -- 172.21.15.12:0/2236269309 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3b340ac2a0 con 0x55e9f670f790 2024-09-11T00:21:08.211 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:08.205+0000 7f3b38c00640 1 -- 172.21.15.12:0/2236269309 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 54) ==== 8422+0+0 (secure 0 0 0) 0x7f3b3c044040 con 0x55e9f670f790 2024-09-11T00:21:08.211 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:08.205+0000 7f3b5462d740 10 client.17061 fetch_fsmap finished waiting for FSMap version 54 2024-09-11T00:21:08.211 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:08.205+0000 7f3b5462d740 10 client.17061 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T00:21:08.211 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:08.205+0000 7f3b5462d740 20 client.17061 populate_metadata read hostname 'smithi012' 2024-09-11T00:21:08.211 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:08.205+0000 7f3b5462d740 1 --2- 172.21.15.12:0/2236269309 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55e9f679b0a0 0x55e9f679d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:21:08.211 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:08.205+0000 7f3b5462d740 4 client.17061 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T00:21:08.211 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:08.205+0000 7f3b5462d740 1 -- 172.21.15.12:0/2236269309 --> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55e9f66800b0 con 0x55e9f679b0a0 2024-09-11T00:21:08.211 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:08.205+0000 7f3b43e00640 1 --2- 172.21.15.12:0/2236269309 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55e9f679b0a0 0x55e9f679d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:21:08.212 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:08.205+0000 7f3b43e00640 1 --2- 172.21.15.12:0/2236269309 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55e9f679b0a0 0x55e9f679d4c0 crc :-1 s=READY pgs=208 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8c1f83e7219db10c server_cookie=3e65b11f9309568e in_seq=0 out_seq=0 2024-09-11T00:21:08.212 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:08.205+0000 7f3b38c00640 10 client.17061 ms_handle_connect on v2:172.21.15.165:6838/501312984 2024-09-11T00:21:08.247 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:08.243+0000 7f3b38c00640 1 -- 172.21.15.12:0/2236269309 <== mds.0 v2:172.21.15.165:6838/501312984 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55e9f66800b0 con 0x55e9f679b0a0 2024-09-11T00:21:08.247 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:08.243+0000 7f3b38c00640 10 client.17061 handle_command_reply: tid=0 2024-09-11T00:21:08.247 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:08.243+0000 7f3b5462d740 1 client.17061 shutdown 2024-09-11T00:21:08.248 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:08.243+0000 7f3b5462d740 1 -- 172.21.15.12:0/2236269309 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e9f670f790 msgr2=0x55e9f6728440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:21:08.248 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:08.243+0000 7f3b5462d740 1 --2- 172.21.15.12:0/2236269309 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e9f670f790 0x55e9f6728440 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f3b3c005ab0 tx=0x7f3b3c009260 comp rx=0 tx=0).stop 2024-09-11T00:21:08.248 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:08.246+0000 7f3b5462d740 1 -- 172.21.15.12:0/2236269309 shutdown_connections 2024-09-11T00:21:08.248 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:08.246+0000 7f3b5462d740 1 --2- 172.21.15.12:0/2236269309 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55e9f679b0a0 0x55e9f679d4c0 crc :-1 s=READY pgs=208 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:21:08.248 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:08.246+0000 7f3b5462d740 1 --2- 172.21.15.12:0/2236269309 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e9f6714b00 0x55e9f6726fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:21:08.248 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:08.246+0000 7f3b5462d740 1 --2- 172.21.15.12:0/2236269309 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e9f67120c0 0x55e9f6728980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:21:08.248 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:08.246+0000 7f3b5462d740 1 --2- 172.21.15.12:0/2236269309 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e9f670f790 0x55e9f6728440 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:21:08.248 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:08.246+0000 7f3b38c00640 0 client.17061 ms_handle_reset on v2:172.21.15.165:6838/501312984 2024-09-11T00:21:08.248 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:08.246+0000 7f3b5462d740 1 -- 172.21.15.12:0/2236269309 >> 172.21.15.12:0/2236269309 conn(0x55e9f670a8e0 msgr2=0x55e9f670f030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:21:08.248 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:08.246+0000 7f3b5462d740 1 -- 172.21.15.12:0/2236269309 shutdown_connections 2024-09-11T00:21:08.248 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:08.246+0000 7f3b5462d740 1 -- 172.21.15.12:0/2236269309 wait complete. 2024-09-11T00:21:08.248 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:08.246+0000 7f3b5462d740 20 client.17061 trim_cache size 0 max 16384 2024-09-11T00:21:08.249 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:{ 2024-09-11T00:21:08.249 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "message": "", 2024-09-11T00:21:08.249 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "return_code": 0 2024-09-11T00:21:08.249 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:} 2024-09-11T00:21:08.250 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:08.246+0000 7f3b51800640 1 -- 172.21.15.12:0/3196611564 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f3b300b52e0 msgr2=0x7f3b300b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:21:08.250 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:08.246+0000 7f3b51800640 1 --2- 172.21.15.12:0/3196611564 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f3b300b52e0 0x7f3b300b77a0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f3b3c006010 tx=0x7f3b3c031430 comp rx=0 tx=0).stop 2024-09-11T00:21:08.250 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:08.246+0000 7f3b51800640 1 -- 172.21.15.12:0/3196611564 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3b4c103ae0 msgr2=0x7f3b4c075540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:21:08.250 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:08.246+0000 7f3b51800640 1 --2- 172.21.15.12:0/3196611564 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3b4c103ae0 0x7f3b4c075540 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f3b3400e9f0 tx=0x7f3b3400eeb0 comp rx=0 tx=0).stop 2024-09-11T00:21:08.250 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:08.246+0000 7f3b51800640 1 -- 172.21.15.12:0/3196611564 shutdown_connections 2024-09-11T00:21:08.250 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:08.246+0000 7f3b51800640 1 --2- 172.21.15.12:0/3196611564 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f3b300b52e0 0x7f3b300b77a0 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:21:08.250 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:08.246+0000 7f3b51800640 1 --2- 172.21.15.12:0/3196611564 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3b4c1044a0 0x7f3b4c075a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:21:08.250 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:08.246+0000 7f3b51800640 1 --2- 172.21.15.12:0/3196611564 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3b4c103ae0 0x7f3b4c075540 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:21:08.250 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:08.246+0000 7f3b51800640 1 --2- 172.21.15.12:0/3196611564 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3b4c102990 0x7f3b4c076ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:21:08.251 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:08.246+0000 7f3b51800640 1 -- 172.21.15.12:0/3196611564 >> 172.21.15.12:0/3196611564 conn(0x7f3b4c0fe250 msgr2=0x7f3b4c106ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:21:08.251 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:08.246+0000 7f3b51800640 1 -- 172.21.15.12:0/3196611564 shutdown_connections 2024-09-11T00:21:08.251 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:08.246+0000 7f3b51800640 1 -- 172.21.15.12:0/3196611564 wait complete. 2024-09-11T00:21:08.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:21:07 smithi165 ceph-mon[35356]: pgmap v820: 129 pgs: 129 active+clean; 453 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 9.5 MiB/s wr, 475 op/s 2024-09-11T00:21:08.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:21:07 smithi178 ceph-mon[35919]: pgmap v820: 129 pgs: 129 active+clean; 453 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 9.5 MiB/s wr, 475 op/s 2024-09-11T00:21:09.144 INFO:tasks.workunit.client.0.smithi012.stdout: 10 27890 25.78 MB/sec warmup 65 sec latency 15545.829 ms 2024-09-11T00:21:10.092 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:21:09 smithi012 ceph-mon[30333]: pgmap v821: 129 pgs: 129 active+clean; 451 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 447 KiB/s rd, 5.4 MiB/s wr, 225 op/s 2024-09-11T00:21:10.144 INFO:tasks.workunit.client.0.smithi012.stdout: 10 27890 25.39 MB/sec warmup 66 sec latency 16545.881 ms 2024-09-11T00:21:10.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:21:09 smithi165 ceph-mon[35356]: pgmap v821: 129 pgs: 129 active+clean; 451 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 447 KiB/s rd, 5.4 MiB/s wr, 225 op/s 2024-09-11T00:21:10.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:21:09 smithi178 ceph-mon[35919]: pgmap v821: 129 pgs: 129 active+clean; 451 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 447 KiB/s rd, 5.4 MiB/s wr, 225 op/s 2024-09-11T00:21:11.144 INFO:tasks.workunit.client.0.smithi012.stdout: 10 27890 25.01 MB/sec warmup 67 sec latency 17545.931 ms 2024-09-11T00:21:12.096 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:21:11 smithi012 ceph-mon[30333]: pgmap v822: 129 pgs: 129 active+clean; 457 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 309 KiB/s rd, 7.0 MiB/s wr, 262 op/s 2024-09-11T00:21:12.145 INFO:tasks.workunit.client.0.smithi012.stdout: 10 27890 24.64 MB/sec warmup 68 sec latency 18545.979 ms 2024-09-11T00:21:12.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:21:11 smithi165 ceph-mon[35356]: pgmap v822: 129 pgs: 129 active+clean; 457 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 309 KiB/s rd, 7.0 MiB/s wr, 262 op/s 2024-09-11T00:21:12.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:21:11 smithi178 ceph-mon[35919]: pgmap v822: 129 pgs: 129 active+clean; 457 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 309 KiB/s rd, 7.0 MiB/s wr, 262 op/s 2024-09-11T00:21:13.144 INFO:tasks.workunit.client.0.smithi012.stdout: 10 27890 24.29 MB/sec warmup 69 sec latency 19546.029 ms 2024-09-11T00:21:14.050 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:21:13 smithi012 ceph-mon[30333]: pgmap v823: 129 pgs: 129 active+clean; 465 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 210 KiB/s rd, 5.2 MiB/s wr, 163 op/s 2024-09-11T00:21:14.145 INFO:tasks.workunit.client.0.smithi012.stdout: 10 27890 23.94 MB/sec warmup 70 sec latency 20546.078 ms 2024-09-11T00:21:14.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:21:13 smithi165 ceph-mon[35356]: pgmap v823: 129 pgs: 129 active+clean; 465 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 210 KiB/s rd, 5.2 MiB/s wr, 163 op/s 2024-09-11T00:21:14.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:21:13 smithi178 ceph-mon[35919]: pgmap v823: 129 pgs: 129 active+clean; 465 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 210 KiB/s rd, 5.2 MiB/s wr, 163 op/s 2024-09-11T00:21:15.145 INFO:tasks.workunit.client.0.smithi012.stdout: 10 27890 23.60 MB/sec warmup 71 sec latency 21546.127 ms 2024-09-11T00:21:16.033 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:21:15 smithi012 ceph-mon[30333]: pgmap v824: 129 pgs: 129 active+clean; 465 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 160 KiB/s rd, 4.2 MiB/s wr, 132 op/s 2024-09-11T00:21:16.145 INFO:tasks.workunit.client.0.smithi012.stdout: 10 27890 23.28 MB/sec warmup 72 sec latency 22546.178 ms 2024-09-11T00:21:16.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:21:15 smithi165 ceph-mon[35356]: pgmap v824: 129 pgs: 129 active+clean; 465 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 160 KiB/s rd, 4.2 MiB/s wr, 132 op/s 2024-09-11T00:21:16.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:21:15 smithi178 ceph-mon[35919]: pgmap v824: 129 pgs: 129 active+clean; 465 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 160 KiB/s rd, 4.2 MiB/s wr, 132 op/s 2024-09-11T00:21:17.145 INFO:tasks.workunit.client.0.smithi012.stdout: 10 27890 22.96 MB/sec warmup 73 sec latency 23546.227 ms 2024-09-11T00:21:17.987 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:21:17 smithi012 ceph-mon[30333]: pgmap v825: 129 pgs: 129 active+clean; 477 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 189 KiB/s rd, 4.9 MiB/s wr, 149 op/s 2024-09-11T00:21:18.151 INFO:tasks.workunit.client.0.smithi012.stdout: 10 27890 22.65 MB/sec warmup 74 sec latency 24552.299 ms 2024-09-11T00:21:18.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:21:17 smithi165 ceph-mon[35356]: pgmap v825: 129 pgs: 129 active+clean; 477 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 189 KiB/s rd, 4.9 MiB/s wr, 149 op/s 2024-09-11T00:21:18.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:21:17 smithi178 ceph-mon[35919]: pgmap v825: 129 pgs: 129 active+clean; 477 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 189 KiB/s rd, 4.9 MiB/s wr, 149 op/s 2024-09-11T00:21:19.151 INFO:tasks.workunit.client.0.smithi012.stdout: 10 27891 22.35 MB/sec warmup 75 sec latency 25552.367 ms 2024-09-11T00:21:20.151 INFO:tasks.workunit.client.0.smithi012.stdout: 10 27891 22.05 MB/sec warmup 76 sec latency 26552.410 ms 2024-09-11T00:21:20.211 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:21:19 smithi012 ceph-mon[30333]: pgmap v826: 129 pgs: 129 active+clean; 477 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 3.9 MiB/s wr, 115 op/s 2024-09-11T00:21:20.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:21:19 smithi165 ceph-mon[35356]: pgmap v826: 129 pgs: 129 active+clean; 477 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 3.9 MiB/s wr, 115 op/s 2024-09-11T00:21:20.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:21:19 smithi178 ceph-mon[35919]: pgmap v826: 129 pgs: 129 active+clean; 477 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 3.9 MiB/s wr, 115 op/s 2024-09-11T00:21:21.151 INFO:tasks.workunit.client.0.smithi012.stdout: 10 27891 21.77 MB/sec warmup 77 sec latency 27552.452 ms 2024-09-11T00:21:21.369 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:21.366+0000 7f6b2a600640 1 -- 172.21.15.12:0/1145165089 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6b241058f0 msgr2=0x7f6b24107d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:21:21.369 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:21.366+0000 7f6b2a600640 1 --2- 172.21.15.12:0/1145165089 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6b241058f0 0x7f6b24107d60 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f6b10009900 tx=0x7f6b1002a440 comp rx=0 tx=0).stop 2024-09-11T00:21:21.370 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:21.366+0000 7f6b2a600640 1 -- 172.21.15.12:0/1145165089 shutdown_connections 2024-09-11T00:21:21.370 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:21.366+0000 7f6b2a600640 1 --2- 172.21.15.12:0/1145165089 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6b241084a0 0x7f6b2410a960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:21:21.370 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:21.366+0000 7f6b2a600640 1 --2- 172.21.15.12:0/1145165089 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6b241058f0 0x7f6b24107d60 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:21:21.370 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:21.366+0000 7f6b2a600640 1 --2- 172.21.15.12:0/1145165089 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6b2406b470 0x7f6b24105360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:21:21.370 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:21.366+0000 7f6b2a600640 1 -- 172.21.15.12:0/1145165089 >> 172.21.15.12:0/1145165089 conn(0x7f6b240fcf30 msgr2=0x7f6b240ff370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:21:21.370 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:21.366+0000 7f6b2a600640 1 -- 172.21.15.12:0/1145165089 shutdown_connections 2024-09-11T00:21:21.370 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:21.366+0000 7f6b2a600640 1 -- 172.21.15.12:0/1145165089 wait complete. 2024-09-11T00:21:21.371 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:21.366+0000 7f6b2a600640 1 Processor -- start 2024-09-11T00:21:21.371 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:21.366+0000 7f6b2a600640 1 -- start start 2024-09-11T00:21:21.372 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:21.366+0000 7f6b2a600640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6b2406b470 0x7f6b2419af30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:21:21.372 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:21.366+0000 7f6b2a600640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6b241058f0 0x7f6b2419b580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:21:21.372 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:21.366+0000 7f6b2a600640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6b241084a0 0x7f6b2419c1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:21:21.372 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:21.366+0000 7f6b2a600640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f6b2410e560 con 0x7f6b241058f0 2024-09-11T00:21:21.372 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:21.366+0000 7f6b2a600640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f6b2410e3e0 con 0x7f6b2406b470 2024-09-11T00:21:21.372 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:21.366+0000 7f6b2a600640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f6b2410e6e0 con 0x7f6b241084a0 2024-09-11T00:21:21.372 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:21.366+0000 7f6b23e00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6b241058f0 0x7f6b2419b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:21:21.372 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:21.366+0000 7f6b23e00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6b241058f0 0x7f6b2419b580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:45160/0 (socket says 172.21.15.12:45160) 2024-09-11T00:21:21.372 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:21.366+0000 7f6b23e00640 1 -- 172.21.15.12:0/2758965840 learned_addr learned my addr 172.21.15.12:0/2758965840 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:21:21.372 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:21.366+0000 7f6b29200640 1 --2- 172.21.15.12:0/2758965840 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6b2406b470 0x7f6b2419af30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:21:21.372 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:21.366+0000 7f6b23e00640 1 -- 172.21.15.12:0/2758965840 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6b241084a0 msgr2=0x7f6b2419c1b0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-11T00:21:21.372 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:21.366+0000 7f6b23e00640 1 --2- 172.21.15.12:0/2758965840 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6b241084a0 0x7f6b2419c1b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:21:21.372 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:21.366+0000 7f6b23e00640 1 -- 172.21.15.12:0/2758965840 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6b2406b470 msgr2=0x7f6b2419af30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:21:21.373 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:21.366+0000 7f6b23e00640 1 --2- 172.21.15.12:0/2758965840 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6b2406b470 0x7f6b2419af30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:21:21.373 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:21.366+0000 7f6b23e00640 1 -- 172.21.15.12:0/2758965840 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b2419ca70 con 0x7f6b241058f0 2024-09-11T00:21:21.373 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:21.366+0000 7f6b23e00640 1 --2- 172.21.15.12:0/2758965840 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6b241058f0 0x7f6b2419b580 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f6b1002aa30 tx=0x7f6b10003d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:21:21.373 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:21.366+0000 7f6b21600640 1 -- 172.21.15.12:0/2758965840 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b1003f070 con 0x7f6b241058f0 2024-09-11T00:21:21.373 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:21.366+0000 7f6b21600640 1 -- 172.21.15.12:0/2758965840 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6b1003b460 con 0x7f6b241058f0 2024-09-11T00:21:21.374 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:21.366+0000 7f6b21600640 1 -- 172.21.15.12:0/2758965840 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b10035030 con 0x7f6b241058f0 2024-09-11T00:21:21.374 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:21.366+0000 7f6b2a600640 1 -- 172.21.15.12:0/2758965840 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6b2419cd00 con 0x7f6b241058f0 2024-09-11T00:21:21.374 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:21.366+0000 7f6b2a600640 1 -- 172.21.15.12:0/2758965840 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6b241a32d0 con 0x7f6b241058f0 2024-09-11T00:21:21.376 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:21.372+0000 7f6b2ec7b740 1 Processor -- start 2024-09-11T00:21:21.376 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:21.372+0000 7f6b2ec7b740 1 -- start start 2024-09-11T00:21:21.377 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:21.372+0000 7f6b21600640 1 -- 172.21.15.12:0/2758965840 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7f6b1003b930 con 0x7f6b241058f0 2024-09-11T00:21:21.377 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:21.372+0000 7f6b2ec7b740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562f7e4b7970 0x562f7e4b9d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:21:21.377 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:21.372+0000 7f6b2ec7b740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562f7e4ba2a0 0x562f7e4bc690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:21:21.377 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:21.372+0000 7f6b2ec7b740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562f7e4bcce0 0x562f7e4bf1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:21:21.377 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:21.372+0000 7f6b2ec7b740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x562f7e42a6b0 con 0x562f7e4ba2a0 2024-09-11T00:21:21.377 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:21.372+0000 7f6b2ec7b740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x562f7e419ae0 con 0x562f7e4bcce0 2024-09-11T00:21:21.377 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:21.372+0000 7f6b2ec7b740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x562f7e41d8f0 con 0x562f7e4b7970 2024-09-11T00:21:21.377 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:21.372+0000 7f6b29200640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562f7e4ba2a0 0x562f7e4bc690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:21:21.377 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:21.372+0000 7f6b29200640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562f7e4ba2a0 0x562f7e4bc690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:45164/0 (socket says 172.21.15.12:45164) 2024-09-11T00:21:21.377 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:21.372+0000 7f6b29200640 1 -- 172.21.15.12:0/3425012000 learned_addr learned my addr 172.21.15.12:0/3425012000 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:21:21.379 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:21.372+0000 7f6b23e00640 1 --2- 172.21.15.12:0/3425012000 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562f7e4bcce0 0x562f7e4bf1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:21:21.380 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:21.372+0000 7f6b29200640 1 -- 172.21.15.12:0/3425012000 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562f7e4b7970 msgr2=0x562f7e4b9d60 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-11T00:21:21.380 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:21.372+0000 7f6b29200640 1 --2- 172.21.15.12:0/3425012000 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562f7e4b7970 0x562f7e4b9d60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:21:21.380 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:21.372+0000 7f6b29200640 1 -- 172.21.15.12:0/3425012000 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562f7e4bcce0 msgr2=0x562f7e4bf1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:21:21.380 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:21.372+0000 7f6b29200640 1 --2- 172.21.15.12:0/3425012000 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562f7e4bcce0 0x562f7e4bf1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:21:21.380 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:21.372+0000 7f6b29200640 1 -- 172.21.15.12:0/3425012000 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562f7e3e6780 con 0x562f7e4ba2a0 2024-09-11T00:21:21.380 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:21.376+0000 7f6b29200640 1 --2- 172.21.15.12:0/3425012000 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562f7e4ba2a0 0x562f7e4bc690 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f6b18009a40 tx=0x7f6b18009f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3945bd733e1b9bef server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:21:21.380 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:21.376+0000 7f6b16000640 1 -- 172.21.15.12:0/3425012000 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b18017070 con 0x562f7e4ba2a0 2024-09-11T00:21:21.380 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:21.376+0000 7f6b16000640 1 -- 172.21.15.12:0/3425012000 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6b18013470 con 0x562f7e4ba2a0 2024-09-11T00:21:21.380 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:21.376+0000 7f6b16000640 1 -- 172.21.15.12:0/3425012000 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b18012630 con 0x562f7e4ba2a0 2024-09-11T00:21:21.380 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:21.376+0000 7f6b21600640 1 --2- 172.21.15.12:0/2758965840 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f6b0c0b5360 0x7f6b0c0b7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:21:21.380 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:21.376+0000 7f6b21600640 1 -- 172.21.15.12:0/2758965840 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9208+0+0 (secure 0 0 0) 0x7f6b10106dc0 con 0x7f6b241058f0 2024-09-11T00:21:21.381 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:21.376+0000 7f6b29c00640 1 --2- 172.21.15.12:0/2758965840 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f6b0c0b5360 0x7f6b0c0b7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:21:21.381 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:21.376+0000 7f6b2ec7b740 1 -- 172.21.15.12:0/3425012000 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562f7e4ba2a0 msgr2=0x562f7e4bc690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:21:21.381 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:21.376+0000 7f6b2ec7b740 1 --2- 172.21.15.12:0/3425012000 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562f7e4ba2a0 0x562f7e4bc690 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f6b18009a40 tx=0x7f6b18009f00 comp rx=0 tx=0).stop 2024-09-11T00:21:21.381 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:21.376+0000 7f6b2ec7b740 1 -- 172.21.15.12:0/3425012000 shutdown_connections 2024-09-11T00:21:21.381 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:21.376+0000 7f6b2ec7b740 1 --2- 172.21.15.12:0/3425012000 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562f7e4bcce0 0x562f7e4bf1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:21:21.381 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:21.376+0000 7f6b2ec7b740 1 --2- 172.21.15.12:0/3425012000 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562f7e4ba2a0 0x562f7e4bc690 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:21:21.381 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:21.376+0000 7f6b2ec7b740 1 --2- 172.21.15.12:0/3425012000 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562f7e4b7970 0x562f7e4b9d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:21:21.381 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:21.376+0000 7f6b2ec7b740 1 -- 172.21.15.12:0/3425012000 >> 172.21.15.12:0/3425012000 conn(0x562f7e4b2ac0 msgr2=0x562f7e4b3850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:21:21.381 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:21.376+0000 7f6b2ec7b740 1 -- 172.21.15.12:0/3425012000 shutdown_connections 2024-09-11T00:21:21.381 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:21.376+0000 7f6b29c00640 1 --2- 172.21.15.12:0/2758965840 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f6b0c0b5360 0x7f6b0c0b7820 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f6b2419bcb0 tx=0x7f6b1c009250 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:21:21.381 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:21.376+0000 7f6b2ec7b740 1 -- 172.21.15.12:0/3425012000 wait complete. 2024-09-11T00:21:21.384 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:21.376+0000 7f6b2ec7b740 1 -- start start 2024-09-11T00:21:21.384 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:21.376+0000 7f6b2ec7b740 1 Processor -- start 2024-09-11T00:21:21.384 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:21.376+0000 7f6b2ec7b740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562f7e4b7970 0x562f7e4c1160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:21:21.384 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:21.376+0000 7f6b2ec7b740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562f7e4ba2a0 0x562f7e4c1800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:21:21.384 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:21.376+0000 7f6b2ec7b740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562f7e4bcce0 0x562f7e4c2360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:21:21.384 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:21.376+0000 7f6b2ec7b740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x562f7e351ab0 con 0x562f7e4bcce0 2024-09-11T00:21:21.385 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:21.376+0000 7f6b2ec7b740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x562f7e3c0570 con 0x562f7e4b7970 2024-09-11T00:21:21.385 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:21.376+0000 7f6b2ec7b740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x562f7e4bfcc0 con 0x562f7e4ba2a0 2024-09-11T00:21:21.385 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:21.376+0000 7f6b2ec7b740 10 client.0 _ll_register_callbacks cb 0x562f7e3cf100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:21:21.385 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:21.376+0000 7f6b29c00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562f7e4bcce0 0x562f7e4c2360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:21:21.385 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:21.376+0000 7f6b29200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562f7e4b7970 0x562f7e4c1160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:21:21.385 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:21.376+0000 7f6b29c00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562f7e4bcce0 0x562f7e4c2360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:45170/0 (socket says 172.21.15.12:45170) 2024-09-11T00:21:21.385 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:21.376+0000 7f6b29200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562f7e4b7970 0x562f7e4c1160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.12:54860/0 (socket says 172.21.15.12:54860) 2024-09-11T00:21:21.385 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:21.376+0000 7f6b29c00640 1 -- 172.21.15.12:0/1318893986 learned_addr learned my addr 172.21.15.12:0/1318893986 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:21:21.385 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:21.376+0000 7f6b29200640 1 -- 172.21.15.12:0/1318893986 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562f7e4ba2a0 msgr2=0x562f7e4c1800 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-11T00:21:21.385 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:21.376+0000 7f6b29200640 1 --2- 172.21.15.12:0/1318893986 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562f7e4ba2a0 0x562f7e4c1800 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:21:21.385 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:21.376+0000 7f6b29200640 1 -- 172.21.15.12:0/1318893986 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562f7e4bcce0 msgr2=0x562f7e4c2360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:21:21.385 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:21.376+0000 7f6b29200640 1 --2- 172.21.15.12:0/1318893986 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562f7e4bcce0 0x562f7e4c2360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:21:21.386 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:21.376+0000 7f6b29200640 1 -- 172.21.15.12:0/1318893986 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562f7e17b790 con 0x562f7e4b7970 2024-09-11T00:21:21.386 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:21.376+0000 7f6b29c00640 1 --2- 172.21.15.12:0/1318893986 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562f7e4bcce0 0x562f7e4c2360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:21:21.386 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:21.376+0000 7f6b29200640 1 --2- 172.21.15.12:0/1318893986 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562f7e4b7970 0x562f7e4c1160 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f6b18004790 tx=0x7f6b18013620 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1d168a9df1734b29 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:21:21.386 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:21.376+0000 7f6b0aa00640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-09-11T00:21:21.386 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:21.376+0000 7f6b0aa00640 1 -- 172.21.15.12:0/1318893986 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b18017050 con 0x562f7e4b7970 2024-09-11T00:21:21.386 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:21.376+0000 7f6b2ec7b740 1 -- 172.21.15.12:0/1318893986 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6b18013470 con 0x562f7e4b7970 2024-09-11T00:21:21.386 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:21.376+0000 7f6b0aa00640 1 -- 172.21.15.12:0/1318893986 <== mon.2 v2:172.21.15.165:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9208+0+0 (secure 0 0 0) 0x7f6b1801c450 con 0x562f7e4b7970 2024-09-11T00:21:21.386 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:21.376+0000 7f6b0aa00640 1 -- 172.21.15.12:0/1318893986 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7f6b1802f050 con 0x562f7e4b7970 2024-09-11T00:21:21.386 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:21.376+0000 7f6b2ec7b740 10 client.27095 fetch_fsmap learned FSMap version 54 2024-09-11T00:21:21.386 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:21.376+0000 7f6b2ec7b740 1 -- 172.21.15.12:0/1318893986 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) -- 0x562f7e1f4620 con 0x562f7e4b7970 2024-09-11T00:21:21.387 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:21.382+0000 7f6b0aa00640 1 -- 172.21.15.12:0/1318893986 <== mon.2 v2:172.21.15.165:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6b18013470 con 0x562f7e4b7970 2024-09-11T00:21:21.387 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:21.382+0000 7f6b0aa00640 1 -- 172.21.15.12:0/1318893986 <== mon.2 v2:172.21.15.165:3300/0 5 ==== fsmap(e 54) ==== 8422+0+0 (secure 0 0 0) 0x7f6b180266b0 con 0x562f7e4b7970 2024-09-11T00:21:21.387 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:21.382+0000 7f6b2ec7b740 10 client.27095 fetch_fsmap finished waiting for FSMap version 54 2024-09-11T00:21:21.387 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:21.382+0000 7f6b2ec7b740 10 client.27095 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T00:21:21.388 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:21.382+0000 7f6b2ec7b740 20 client.27095 populate_metadata read hostname 'smithi012' 2024-09-11T00:21:21.388 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:21.382+0000 7f6b2ec7b740 1 --2- 172.21.15.12:0/1318893986 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x562f7e4cc6a0 0x562f7e4cead0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:21:21.388 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:21.382+0000 7f6b2ec7b740 4 client.27095 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T00:21:21.388 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:21.382+0000 7f6b2ec7b740 1 -- 172.21.15.12:0/1318893986 --> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562f7e42c600 con 0x562f7e4cc6a0 2024-09-11T00:21:21.389 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:21.388+0000 7f6b23e00640 1 --2- 172.21.15.12:0/1318893986 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x562f7e4cc6a0 0x562f7e4cead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:21:21.425 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:21.423+0000 7f6b23e00640 1 --2- 172.21.15.12:0/1318893986 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x562f7e4cc6a0 0x562f7e4cead0 crc :-1 s=READY pgs=209 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=81ddd97f906f6e1c server_cookie=e4f21f70941764ff in_seq=0 out_seq=0 2024-09-11T00:21:21.425 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:21.423+0000 7f6b0aa00640 10 client.27095 ms_handle_connect on v2:172.21.15.165:6838/501312984 2024-09-11T00:21:21.426 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:21.423+0000 7f6b0aa00640 1 -- 172.21.15.12:0/1318893986 <== mds.0 v2:172.21.15.165:6838/501312984 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x562f7e42c600 con 0x562f7e4cc6a0 2024-09-11T00:21:21.426 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:21.423+0000 7f6b0aa00640 10 client.27095 handle_command_reply: tid=0 2024-09-11T00:21:21.426 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:21.423+0000 7f6b2ec7b740 1 client.27095 shutdown 2024-09-11T00:21:21.426 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:21.423+0000 7f6b2ec7b740 1 -- 172.21.15.12:0/1318893986 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562f7e4b7970 msgr2=0x562f7e4c1160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:21:21.427 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:21.423+0000 7f6b2ec7b740 1 --2- 172.21.15.12:0/1318893986 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562f7e4b7970 0x562f7e4c1160 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f6b18004790 tx=0x7f6b18013620 comp rx=0 tx=0).stop 2024-09-11T00:21:21.427 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:21.423+0000 7f6b2ec7b740 1 -- 172.21.15.12:0/1318893986 shutdown_connections 2024-09-11T00:21:21.427 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:21.423+0000 7f6b2ec7b740 1 --2- 172.21.15.12:0/1318893986 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x562f7e4cc6a0 0x562f7e4cead0 crc :-1 s=READY pgs=209 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:21:21.427 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:21.423+0000 7f6b2ec7b740 1 --2- 172.21.15.12:0/1318893986 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562f7e4bcce0 0x562f7e4c2360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:21:21.427 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:21.423+0000 7f6b2ec7b740 1 --2- 172.21.15.12:0/1318893986 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562f7e4ba2a0 0x562f7e4c1800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:21:21.427 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:21.423+0000 7f6b2ec7b740 1 --2- 172.21.15.12:0/1318893986 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562f7e4b7970 0x562f7e4c1160 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:21:21.427 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:21.423+0000 7f6b2ec7b740 1 -- 172.21.15.12:0/1318893986 >> 172.21.15.12:0/1318893986 conn(0x562f7e4b2ac0 msgr2=0x562f7e4b1bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:21:21.427 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:21.423+0000 7f6b0aa00640 0 client.27095 ms_handle_reset on v2:172.21.15.165:6838/501312984 2024-09-11T00:21:21.427 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:21.423+0000 7f6b2ec7b740 1 -- 172.21.15.12:0/1318893986 shutdown_connections 2024-09-11T00:21:21.427 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:21.423+0000 7f6b2ec7b740 1 -- 172.21.15.12:0/1318893986 wait complete. 2024-09-11T00:21:21.427 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:21.423+0000 7f6b2ec7b740 20 client.27095 trim_cache size 0 max 16384 2024-09-11T00:21:21.444 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:21.435+0000 7f6b2ec7b740 1 Processor -- start 2024-09-11T00:21:21.444 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:21.435+0000 7f6b2ec7b740 1 -- start start 2024-09-11T00:21:21.445 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:21.435+0000 7f6b2ec7b740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562f7e4b7970 0x562f7e4c00c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:21:21.445 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:21.435+0000 7f6b2ec7b740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562f7e4ba2a0 0x562f7e4c0600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:21:21.445 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:21.441+0000 7f6b2ec7b740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562f7e4bcce0 0x562f7e4b5900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:21:21.445 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:21.441+0000 7f6b2ec7b740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f6b241ad940 con 0x562f7e4bcce0 2024-09-11T00:21:21.445 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:21.441+0000 7f6b2ec7b740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f6b241af660 con 0x562f7e4ba2a0 2024-09-11T00:21:21.446 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:21.441+0000 7f6b2ec7b740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f6b241ae070 con 0x562f7e4b7970 2024-09-11T00:21:21.446 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:21.441+0000 7f6b29c00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562f7e4bcce0 0x562f7e4b5900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:21:21.446 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:21.441+0000 7f6b29c00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562f7e4bcce0 0x562f7e4b5900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:45180/0 (socket says 172.21.15.12:45180) 2024-09-11T00:21:21.446 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:21.441+0000 7f6b29c00640 1 -- 172.21.15.12:0/4077087496 learned_addr learned my addr 172.21.15.12:0/4077087496 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:21:21.446 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:21.441+0000 7f6b29200640 1 --2- 172.21.15.12:0/4077087496 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562f7e4b7970 0x562f7e4c00c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:21:21.446 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:21.441+0000 7f6b29c00640 1 -- 172.21.15.12:0/4077087496 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562f7e4b7970 msgr2=0x562f7e4c00c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:21:21.447 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:21.441+0000 7f6b29c00640 1 --2- 172.21.15.12:0/4077087496 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562f7e4b7970 0x562f7e4c00c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:21:21.447 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:21.441+0000 7f6b29c00640 1 -- 172.21.15.12:0/4077087496 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562f7e4ba2a0 msgr2=0x562f7e4c0600 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-11T00:21:21.447 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:21.441+0000 7f6b29c00640 1 --2- 172.21.15.12:0/4077087496 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562f7e4ba2a0 0x562f7e4c0600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:21:21.447 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:21.441+0000 7f6b29c00640 1 -- 172.21.15.12:0/4077087496 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562f7e4ba0d0 con 0x562f7e4bcce0 2024-09-11T00:21:21.447 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:21.441+0000 7f6b29c00640 1 --2- 172.21.15.12:0/4077087496 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562f7e4bcce0 0x562f7e4b5900 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f6b1c00d8b0 tx=0x7f6b1c00f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed49d346de782d0f server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:21:21.447 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:21.441+0000 7f6b15600640 1 -- 172.21.15.12:0/4077087496 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b1c01bd40 con 0x562f7e4bcce0 2024-09-11T00:21:21.447 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:21.441+0000 7f6b15600640 1 -- 172.21.15.12:0/4077087496 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6b1c023890 con 0x562f7e4bcce0 2024-09-11T00:21:21.448 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:21.441+0000 7f6b15600640 1 -- 172.21.15.12:0/4077087496 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b1c0166a0 con 0x562f7e4bcce0 2024-09-11T00:21:21.448 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:21.441+0000 7f6b2ec7b740 1 -- 172.21.15.12:0/4077087496 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562f7e4bcce0 msgr2=0x562f7e4b5900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:21:21.448 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:21.441+0000 7f6b2ec7b740 1 --2- 172.21.15.12:0/4077087496 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562f7e4bcce0 0x562f7e4b5900 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f6b1c00d8b0 tx=0x7f6b1c00f000 comp rx=0 tx=0).stop 2024-09-11T00:21:21.448 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:21.441+0000 7f6b2ec7b740 1 -- 172.21.15.12:0/4077087496 shutdown_connections 2024-09-11T00:21:21.448 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:21.441+0000 7f6b2ec7b740 1 --2- 172.21.15.12:0/4077087496 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562f7e4bcce0 0x562f7e4b5900 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:21:21.448 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:21.441+0000 7f6b2ec7b740 1 --2- 172.21.15.12:0/4077087496 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562f7e4ba2a0 0x562f7e4c0600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:21:21.448 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:21.441+0000 7f6b2ec7b740 1 --2- 172.21.15.12:0/4077087496 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562f7e4b7970 0x562f7e4c00c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:21:21.448 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:21.441+0000 7f6b2ec7b740 1 -- 172.21.15.12:0/4077087496 >> 172.21.15.12:0/4077087496 conn(0x562f7e4b2ac0 msgr2=0x562f7e4bd0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:21:21.448 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:21.441+0000 7f6b2ec7b740 1 -- 172.21.15.12:0/4077087496 shutdown_connections 2024-09-11T00:21:21.449 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:21.441+0000 7f6b2ec7b740 1 -- 172.21.15.12:0/4077087496 wait complete. 2024-09-11T00:21:21.449 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:21.441+0000 7f6b2ec7b740 1 -- start start 2024-09-11T00:21:21.449 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:21.441+0000 7f6b2ec7b740 1 Processor -- start 2024-09-11T00:21:21.452 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:21.441+0000 7f6b2ec7b740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562f7e4b7970 0x562f7e4c8d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:21:21.452 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:21.441+0000 7f6b2ec7b740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562f7e4ba2a0 0x562f7e4c7a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:21:21.452 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:21.441+0000 7f6b2ec7b740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562f7e4bcce0 0x562f7e4c8180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:21:21.452 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:21.441+0000 7f6b2ec7b740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f6b241aef90 con 0x562f7e4b7970 2024-09-11T00:21:21.452 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:21.441+0000 7f6b2ec7b740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f6b241ae800 con 0x562f7e4bcce0 2024-09-11T00:21:21.452 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:21.441+0000 7f6b2ec7b740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f6b241ab920 con 0x562f7e4ba2a0 2024-09-11T00:21:21.452 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:21.441+0000 7f6b2ec7b740 10 client.0 _ll_register_callbacks cb 0x7f6b180098c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:21:21.452 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:21.441+0000 7f6b29200640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562f7e4b7970 0x562f7e4c8d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:21:21.452 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:21.441+0000 7f6b29200640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562f7e4b7970 0x562f7e4c8d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:45194/0 (socket says 172.21.15.12:45194) 2024-09-11T00:21:21.452 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:21.441+0000 7f6b29200640 1 -- 172.21.15.12:0/434323986 learned_addr learned my addr 172.21.15.12:0/434323986 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:21:21.453 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:21.441+0000 7f6b23e00640 1 --2- 172.21.15.12:0/434323986 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562f7e4ba2a0 0x562f7e4c7a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:21:21.453 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:21.441+0000 7f6b29200640 1 -- 172.21.15.12:0/434323986 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562f7e4ba2a0 msgr2=0x562f7e4c7a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:21:21.453 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:21.441+0000 7f6b29200640 1 --2- 172.21.15.12:0/434323986 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562f7e4ba2a0 0x562f7e4c7a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:21:21.453 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:21.441+0000 7f6b29200640 1 -- 172.21.15.12:0/434323986 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562f7e4bcce0 msgr2=0x562f7e4c8180 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-11T00:21:21.453 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:21.441+0000 7f6b29200640 1 --2- 172.21.15.12:0/434323986 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562f7e4bcce0 0x562f7e4c8180 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:21:21.453 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:21.441+0000 7f6b29200640 1 -- 172.21.15.12:0/434323986 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562f7e4c65b0 con 0x562f7e4b7970 2024-09-11T00:21:21.453 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:21.441+0000 7f6b29200640 1 --2- 172.21.15.12:0/434323986 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562f7e4b7970 0x562f7e4c8d90 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f6b18009f40 tx=0x7f6b1800ebf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2f72db237d95875 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:21:21.453 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:21.441+0000 7f6b0aa00640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-09-11T00:21:21.453 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:21.441+0000 7f6b0aa00640 1 -- 172.21.15.12:0/434323986 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b18038060 con 0x562f7e4b7970 2024-09-11T00:21:21.453 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:21.441+0000 7f6b0aa00640 1 -- 172.21.15.12:0/434323986 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9208+0+0 (secure 0 0 0) 0x7f6b18013c20 con 0x562f7e4b7970 2024-09-11T00:21:21.454 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:21.441+0000 7f6b2ec7b740 1 -- 172.21.15.12:0/434323986 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6b1c023890 con 0x562f7e4b7970 2024-09-11T00:21:21.454 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:21.441+0000 7f6b0aa00640 1 -- 172.21.15.12:0/434323986 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7f6b1803f050 con 0x562f7e4b7970 2024-09-11T00:21:21.454 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:21.441+0000 7f6b2ec7b740 10 client.17079 fetch_fsmap learned FSMap version 54 2024-09-11T00:21:21.454 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:21.441+0000 7f6b2ec7b740 1 -- 172.21.15.12:0/434323986 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) -- 0x562f7e5397b0 con 0x562f7e4b7970 2024-09-11T00:21:21.454 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:21.441+0000 7f6b0aa00640 1 -- 172.21.15.12:0/434323986 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6b1c023890 con 0x562f7e4b7970 2024-09-11T00:21:21.454 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:21.446+0000 7f6b0aa00640 1 -- 172.21.15.12:0/434323986 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 54) ==== 8422+0+0 (secure 0 0 0) 0x7f6b180124e0 con 0x562f7e4b7970 2024-09-11T00:21:21.454 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:21.446+0000 7f6b2ec7b740 10 client.17079 fetch_fsmap finished waiting for FSMap version 54 2024-09-11T00:21:21.454 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:21.446+0000 7f6b2ec7b740 10 client.17079 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T00:21:21.454 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:21.446+0000 7f6b2ec7b740 20 client.17079 populate_metadata read hostname 'smithi012' 2024-09-11T00:21:21.454 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:21.446+0000 7f6b2ec7b740 1 --2- 172.21.15.12:0/434323986 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x562f7e53a220 0x562f7e53c640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:21:21.455 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:21.446+0000 7f6b2ec7b740 4 client.17079 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T00:21:21.455 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:21.446+0000 7f6b2ec7b740 1 -- 172.21.15.12:0/434323986 --> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] -- command(tid 0: {"prefix": "flush journal"}) -- 0x562f7e4280b0 con 0x562f7e53a220 2024-09-11T00:21:21.471 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:21.470+0000 7f6b23e00640 1 --2- 172.21.15.12:0/434323986 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x562f7e53a220 0x562f7e53c640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:21:21.532 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:21.529+0000 7f6b23e00640 1 --2- 172.21.15.12:0/434323986 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x562f7e53a220 0x562f7e53c640 crc :-1 s=READY pgs=210 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3d6f2139a0210b15 server_cookie=e8b811eaf6642fc7 in_seq=0 out_seq=0 2024-09-11T00:21:21.532 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:21.529+0000 7f6b0aa00640 10 client.17079 ms_handle_connect on v2:172.21.15.165:6838/501312984 2024-09-11T00:21:21.637 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:21.636+0000 7f6b0aa00640 1 -- 172.21.15.12:0/434323986 <== mds.0 v2:172.21.15.165:6838/501312984 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x562f7e4280b0 con 0x562f7e53a220 2024-09-11T00:21:21.638 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:21.636+0000 7f6b0aa00640 10 client.17079 handle_command_reply: tid=0 2024-09-11T00:21:21.638 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:21.636+0000 7f6b2ec7b740 1 client.17079 shutdown 2024-09-11T00:21:21.638 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:21.636+0000 7f6b2ec7b740 1 -- 172.21.15.12:0/434323986 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562f7e4b7970 msgr2=0x562f7e4c8d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:21:21.638 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:21.636+0000 7f6b2ec7b740 1 --2- 172.21.15.12:0/434323986 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562f7e4b7970 0x562f7e4c8d90 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f6b18009f40 tx=0x7f6b1800ebf0 comp rx=0 tx=0).stop 2024-09-11T00:21:21.638 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:21.636+0000 7f6b2ec7b740 1 -- 172.21.15.12:0/434323986 shutdown_connections 2024-09-11T00:21:21.638 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:21.636+0000 7f6b2ec7b740 1 --2- 172.21.15.12:0/434323986 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x562f7e53a220 0x562f7e53c640 crc :-1 s=READY pgs=210 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:21:21.638 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:21.636+0000 7f6b0aa00640 0 client.17079 ms_handle_reset on v2:172.21.15.165:6838/501312984 2024-09-11T00:21:21.638 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:21.636+0000 7f6b2ec7b740 1 --2- 172.21.15.12:0/434323986 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562f7e4bcce0 0x562f7e4c8180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:21:21.638 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:21.636+0000 7f6b2ec7b740 1 --2- 172.21.15.12:0/434323986 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562f7e4ba2a0 0x562f7e4c7a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:21:21.638 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:21.636+0000 7f6b2ec7b740 1 --2- 172.21.15.12:0/434323986 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562f7e4b7970 0x562f7e4c8d90 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:21:21.638 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:21.636+0000 7f6b2ec7b740 1 -- 172.21.15.12:0/434323986 >> 172.21.15.12:0/434323986 conn(0x562f7e4b2ac0 msgr2=0x562f7e4c00c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:21:21.638 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:21.636+0000 7f6b2ec7b740 1 -- 172.21.15.12:0/434323986 shutdown_connections 2024-09-11T00:21:21.638 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:21.636+0000 7f6b2ec7b740 1 -- 172.21.15.12:0/434323986 wait complete. 2024-09-11T00:21:21.639 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:21.636+0000 7f6b2ec7b740 20 client.17079 trim_cache size 0 max 16384 2024-09-11T00:21:21.639 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:{ 2024-09-11T00:21:21.639 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "message": "", 2024-09-11T00:21:21.639 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "return_code": 0 2024-09-11T00:21:21.639 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:} 2024-09-11T00:21:21.640 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:21.636+0000 7f6b2a600640 1 -- 172.21.15.12:0/2758965840 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f6b0c0b5360 msgr2=0x7f6b0c0b7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:21:21.640 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:21.636+0000 7f6b2a600640 1 --2- 172.21.15.12:0/2758965840 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f6b0c0b5360 0x7f6b0c0b7820 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f6b2419bcb0 tx=0x7f6b1c009250 comp rx=0 tx=0).stop 2024-09-11T00:21:21.640 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:21.636+0000 7f6b2a600640 1 -- 172.21.15.12:0/2758965840 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6b241058f0 msgr2=0x7f6b2419b580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:21:21.640 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:21.636+0000 7f6b2a600640 1 --2- 172.21.15.12:0/2758965840 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6b241058f0 0x7f6b2419b580 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f6b1002aa30 tx=0x7f6b10003d70 comp rx=0 tx=0).stop 2024-09-11T00:21:21.640 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:21.636+0000 7f6b2a600640 1 -- 172.21.15.12:0/2758965840 shutdown_connections 2024-09-11T00:21:21.640 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:21.636+0000 7f6b2a600640 1 --2- 172.21.15.12:0/2758965840 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f6b0c0b5360 0x7f6b0c0b7820 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:21:21.640 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:21.636+0000 7f6b2a600640 1 --2- 172.21.15.12:0/2758965840 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6b241084a0 0x7f6b2419c1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:21:21.640 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:21.636+0000 7f6b2a600640 1 --2- 172.21.15.12:0/2758965840 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6b241058f0 0x7f6b2419b580 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:21:21.640 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:21.636+0000 7f6b2a600640 1 --2- 172.21.15.12:0/2758965840 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6b2406b470 0x7f6b2419af30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:21:21.640 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:21.636+0000 7f6b2a600640 1 -- 172.21.15.12:0/2758965840 >> 172.21.15.12:0/2758965840 conn(0x7f6b240fcf30 msgr2=0x7f6b240ff320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:21:21.640 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:21.636+0000 7f6b2a600640 1 -- 172.21.15.12:0/2758965840 shutdown_connections 2024-09-11T00:21:21.641 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:21.636+0000 7f6b2a600640 1 -- 172.21.15.12:0/2758965840 wait complete. 2024-09-11T00:21:22.145 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:21:21 smithi012 ceph-mon[30333]: pgmap v827: 129 pgs: 129 active+clean; 476 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 134 KiB/s rd, 3.9 MiB/s wr, 118 op/s 2024-09-11T00:21:22.151 INFO:tasks.workunit.client.0.smithi012.stdout: 10 27891 21.49 MB/sec warmup 78 sec latency 28552.492 ms 2024-09-11T00:21:22.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:21:21 smithi165 ceph-mon[35356]: pgmap v827: 129 pgs: 129 active+clean; 476 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 134 KiB/s rd, 3.9 MiB/s wr, 118 op/s 2024-09-11T00:21:22.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:21:21 smithi178 ceph-mon[35919]: pgmap v827: 129 pgs: 129 active+clean; 476 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 134 KiB/s rd, 3.9 MiB/s wr, 118 op/s 2024-09-11T00:21:23.151 INFO:tasks.workunit.client.0.smithi012.stdout: 10 27891 21.22 MB/sec warmup 79 sec latency 29552.544 ms 2024-09-11T00:21:24.063 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:21:23 smithi012 ceph-mon[30333]: pgmap v828: 129 pgs: 129 active+clean; 477 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 254 KiB/s rd, 2.2 MiB/s wr, 64 op/s 2024-09-11T00:21:24.151 INFO:tasks.workunit.client.0.smithi012.stdout: 10 27891 20.95 MB/sec warmup 80 sec latency 30552.602 ms 2024-09-11T00:21:24.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:21:23 smithi165 ceph-mon[35356]: pgmap v828: 129 pgs: 129 active+clean; 477 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 254 KiB/s rd, 2.2 MiB/s wr, 64 op/s 2024-09-11T00:21:24.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:21:23 smithi178 ceph-mon[35919]: pgmap v828: 129 pgs: 129 active+clean; 477 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 254 KiB/s rd, 2.2 MiB/s wr, 64 op/s 2024-09-11T00:21:25.151 INFO:tasks.workunit.client.0.smithi012.stdout: 10 27891 20.69 MB/sec warmup 81 sec latency 31552.648 ms 2024-09-11T00:21:26.018 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:21:25 smithi012 ceph-mon[30333]: pgmap v829: 129 pgs: 129 active+clean; 467 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 202 KiB/s rd, 1.1 MiB/s wr, 53 op/s 2024-09-11T00:21:26.151 INFO:tasks.workunit.client.0.smithi012.stdout: 10 27891 20.44 MB/sec warmup 82 sec latency 32552.709 ms 2024-09-11T00:21:26.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:21:25 smithi165 ceph-mon[35356]: pgmap v829: 129 pgs: 129 active+clean; 467 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 202 KiB/s rd, 1.1 MiB/s wr, 53 op/s 2024-09-11T00:21:26.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:21:25 smithi178 ceph-mon[35919]: pgmap v829: 129 pgs: 129 active+clean; 467 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 202 KiB/s rd, 1.1 MiB/s wr, 53 op/s 2024-09-11T00:21:27.151 INFO:tasks.workunit.client.0.smithi012.stdout: 10 27891 20.19 MB/sec warmup 83 sec latency 33552.768 ms 2024-09-11T00:21:28.063 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:21:27 smithi012 ceph-mon[30333]: pgmap v830: 129 pgs: 129 active+clean; 421 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 268 KiB/s rd, 1.2 MiB/s wr, 122 op/s 2024-09-11T00:21:28.063 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:21:27 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T00:21:28.151 INFO:tasks.workunit.client.0.smithi012.stdout: 10 27891 19.95 MB/sec warmup 84 sec latency 34552.828 ms 2024-09-11T00:21:28.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:21:28 smithi165 ceph-mon[35356]: pgmap v830: 129 pgs: 129 active+clean; 421 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 268 KiB/s rd, 1.2 MiB/s wr, 122 op/s 2024-09-11T00:21:28.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:21:28 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T00:21:28.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:21:27 smithi178 ceph-mon[35919]: pgmap v830: 129 pgs: 129 active+clean; 421 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 268 KiB/s rd, 1.2 MiB/s wr, 122 op/s 2024-09-11T00:21:28.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:21:27 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T00:21:29.151 INFO:tasks.workunit.client.0.smithi012.stdout: 10 27891 19.72 MB/sec warmup 85 sec latency 35552.878 ms 2024-09-11T00:21:30.114 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:21:30 smithi012 ceph-mon[30333]: pgmap v831: 129 pgs: 129 active+clean; 418 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 262 KiB/s rd, 270 KiB/s wr, 106 op/s 2024-09-11T00:21:30.114 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:21:30 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:21:30.114 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:21:30 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T00:21:30.114 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:21:30 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:21:30.151 INFO:tasks.workunit.client.0.smithi012.stdout: 10 27891 19.49 MB/sec warmup 86 sec latency 36552.953 ms 2024-09-11T00:21:30.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:21:30 smithi165 ceph-mon[35356]: pgmap v831: 129 pgs: 129 active+clean; 418 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 262 KiB/s rd, 270 KiB/s wr, 106 op/s 2024-09-11T00:21:30.283 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:21:30 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:21:30.283 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:21:30 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T00:21:30.283 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:21:30 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:21:30.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:21:30 smithi178 ceph-mon[35919]: pgmap v831: 129 pgs: 129 active+clean; 418 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 262 KiB/s rd, 270 KiB/s wr, 106 op/s 2024-09-11T00:21:30.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:21:30 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:21:30.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:21:30 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T00:21:30.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:21:30 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:21:31.151 INFO:tasks.workunit.client.0.smithi012.stdout: 10 27891 19.27 MB/sec warmup 87 sec latency 37553.015 ms 2024-09-11T00:21:32.063 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:21:32 smithi012 ceph-mon[30333]: pgmap v832: 129 pgs: 129 active+clean; 407 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 278 KiB/s rd, 401 KiB/s wr, 119 op/s 2024-09-11T00:21:32.151 INFO:tasks.workunit.client.0.smithi012.stdout: 10 27891 19.05 MB/sec warmup 88 sec latency 38553.079 ms 2024-09-11T00:21:32.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:21:32 smithi178 ceph-mon[35919]: pgmap v832: 129 pgs: 129 active+clean; 407 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 278 KiB/s rd, 401 KiB/s wr, 119 op/s 2024-09-11T00:21:32.533 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:21:32 smithi165 ceph-mon[35356]: pgmap v832: 129 pgs: 129 active+clean; 407 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 278 KiB/s rd, 401 KiB/s wr, 119 op/s 2024-09-11T00:21:33.152 INFO:tasks.workunit.client.0.smithi012.stdout: 10 27891 18.83 MB/sec warmup 89 sec latency 39553.130 ms 2024-09-11T00:21:34.152 INFO:tasks.workunit.client.0.smithi012.stdout: 10 27891 18.62 MB/sec warmup 90 sec latency 40553.190 ms 2024-09-11T00:21:34.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:21:34 smithi012 ceph-mon[30333]: pgmap v833: 129 pgs: 129 active+clean; 408 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 277 KiB/s rd, 435 KiB/s wr, 124 op/s 2024-09-11T00:21:34.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:21:34 smithi178 ceph-mon[35919]: pgmap v833: 129 pgs: 129 active+clean; 408 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 277 KiB/s rd, 435 KiB/s wr, 124 op/s 2024-09-11T00:21:34.532 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:21:34 smithi165 ceph-mon[35356]: pgmap v833: 129 pgs: 129 active+clean; 408 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 277 KiB/s rd, 435 KiB/s wr, 124 op/s 2024-09-11T00:21:34.760 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:34.756+0000 7fd311400640 1 -- 172.21.15.12:0/4075242436 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd30c104b90 msgr2=0x7fd30c104f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:21:34.760 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:34.756+0000 7fd311400640 1 --2- 172.21.15.12:0/4075242436 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd30c104b90 0x7fd30c104f90 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fd2fc009900 tx=0x7fd2fc02a3f0 comp rx=0 tx=0).stop 2024-09-11T00:21:34.760 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:34.756+0000 7fd311400640 1 -- 172.21.15.12:0/4075242436 shutdown_connections 2024-09-11T00:21:34.760 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:34.756+0000 7fd311400640 1 --2- 172.21.15.12:0/4075242436 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd30c1066a0 0x7fd30c10d020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:21:34.761 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:34.756+0000 7fd311400640 1 --2- 172.21.15.12:0/4075242436 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd30c105ce0 0x7fd30c106160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:21:34.761 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:34.756+0000 7fd311400640 1 --2- 172.21.15.12:0/4075242436 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd30c104b90 0x7fd30c104f90 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:21:34.761 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:34.756+0000 7fd311400640 1 -- 172.21.15.12:0/4075242436 >> 172.21.15.12:0/4075242436 conn(0x7fd30c100470 msgr2=0x7fd30c102890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:21:34.761 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:34.756+0000 7fd311400640 1 -- 172.21.15.12:0/4075242436 shutdown_connections 2024-09-11T00:21:34.761 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:34.756+0000 7fd311400640 1 -- 172.21.15.12:0/4075242436 wait complete. 2024-09-11T00:21:34.762 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:34.756+0000 7fd311400640 1 Processor -- start 2024-09-11T00:21:34.762 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:34.756+0000 7fd311400640 1 -- start start 2024-09-11T00:21:34.762 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:34.756+0000 7fd311400640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd30c104b90 0x7fd30c19b010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:21:34.762 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:34.756+0000 7fd311400640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd30c105ce0 0x7fd30c19b660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:21:34.762 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:34.756+0000 7fd311400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd30c1066a0 0x7fd30c19c290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:21:34.763 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:34.756+0000 7fd311400640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fd30c110800 con 0x7fd30c104b90 2024-09-11T00:21:34.763 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:34.756+0000 7fd311400640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fd30c110680 con 0x7fd30c1066a0 2024-09-11T00:21:34.763 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:34.756+0000 7fd311400640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fd30c110980 con 0x7fd30c105ce0 2024-09-11T00:21:34.763 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:34.756+0000 7fd30a800640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd30c104b90 0x7fd30c19b010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:21:34.763 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:34.756+0000 7fd30a800640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd30c104b90 0x7fd30c19b010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:39884/0 (socket says 172.21.15.12:39884) 2024-09-11T00:21:34.763 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:34.756+0000 7fd30a800640 1 -- 172.21.15.12:0/1309613083 learned_addr learned my addr 172.21.15.12:0/1309613083 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:21:34.763 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:34.756+0000 7fd30a800640 1 -- 172.21.15.12:0/1309613083 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd30c105ce0 msgr2=0x7fd30c19b660 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-11T00:21:34.763 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:34.756+0000 7fd30a800640 1 --2- 172.21.15.12:0/1309613083 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd30c105ce0 0x7fd30c19b660 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:21:34.763 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:34.756+0000 7fd30a800640 1 -- 172.21.15.12:0/1309613083 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd30c1066a0 msgr2=0x7fd30c19c290 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-11T00:21:34.763 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:34.756+0000 7fd30a800640 1 --2- 172.21.15.12:0/1309613083 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd30c1066a0 0x7fd30c19c290 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:21:34.763 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:34.756+0000 7fd30a800640 1 -- 172.21.15.12:0/1309613083 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd30c19cb50 con 0x7fd30c104b90 2024-09-11T00:21:34.763 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:34.756+0000 7fd30a800640 1 --2- 172.21.15.12:0/1309613083 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd30c104b90 0x7fd30c19b010 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7fd2fc005940 tx=0x7fd2fc0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:21:34.764 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:34.756+0000 7fd303e00640 1 -- 172.21.15.12:0/1309613083 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2fc03f030 con 0x7fd30c104b90 2024-09-11T00:21:34.764 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:34.756+0000 7fd303e00640 1 -- 172.21.15.12:0/1309613083 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd2fc003cb0 con 0x7fd30c104b90 2024-09-11T00:21:34.764 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:34.756+0000 7fd311400640 1 -- 172.21.15.12:0/1309613083 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd30c19cde0 con 0x7fd30c104b90 2024-09-11T00:21:34.764 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:34.762+0000 7fd311400640 1 -- 172.21.15.12:0/1309613083 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd30c1a3510 con 0x7fd30c104b90 2024-09-11T00:21:34.764 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:34.762+0000 7fd303e00640 1 -- 172.21.15.12:0/1309613083 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2fc035070 con 0x7fd30c104b90 2024-09-11T00:21:34.766 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:34.762+0000 7fd314277740 1 Processor -- start 2024-09-11T00:21:34.767 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:34.762+0000 7fd314277740 1 -- start start 2024-09-11T00:21:34.767 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:34.762+0000 7fd314277740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c6daf11790 0x55c6daf13b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:21:34.768 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:34.766+0000 7fd314277740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c6daf140c0 0x55c6daf164b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:21:34.768 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:34.766+0000 7fd314277740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c6daf16b00 0x55c6daf18fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:21:34.768 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:34.766+0000 7fd314277740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55c6dae846b0 con 0x55c6daf11790 2024-09-11T00:21:34.768 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:34.766+0000 7fd314277740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55c6dae73ae0 con 0x55c6daf16b00 2024-09-11T00:21:34.768 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:34.766+0000 7fd314277740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55c6dae778f0 con 0x55c6daf140c0 2024-09-11T00:21:34.768 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:34.766+0000 7fd30b200640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c6daf11790 0x55c6daf13b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:21:34.768 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:34.766+0000 7fd301e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c6daf16b00 0x55c6daf18fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:21:34.768 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:34.766+0000 7fd30b200640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c6daf11790 0x55c6daf13b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:39900/0 (socket says 172.21.15.12:39900) 2024-09-11T00:21:34.768 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:34.766+0000 7fd30b200640 1 -- 172.21.15.12:0/1319948965 learned_addr learned my addr 172.21.15.12:0/1319948965 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:21:34.768 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:34.766+0000 7fd301e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c6daf16b00 0x55c6daf18fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.12:46594/0 (socket says 172.21.15.12:46594) 2024-09-11T00:21:34.769 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:34.766+0000 7fd301e00640 1 -- 172.21.15.12:0/1319948965 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c6daf140c0 msgr2=0x55c6daf164b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-11T00:21:34.769 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:34.766+0000 7fd301e00640 1 --2- 172.21.15.12:0/1319948965 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c6daf140c0 0x55c6daf164b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:21:34.769 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:34.766+0000 7fd301e00640 1 -- 172.21.15.12:0/1319948965 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c6daf11790 msgr2=0x55c6daf13b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:21:34.769 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:34.766+0000 7fd301e00640 1 --2- 172.21.15.12:0/1319948965 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c6daf11790 0x55c6daf13b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:21:34.769 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:34.766+0000 7fd301e00640 1 -- 172.21.15.12:0/1319948965 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c6dae40780 con 0x55c6daf16b00 2024-09-11T00:21:34.769 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:34.766+0000 7fd30b200640 1 --2- 172.21.15.12:0/1319948965 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c6daf11790 0x55c6daf13b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:21:34.770 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:34.766+0000 7fd303e00640 1 -- 172.21.15.12:0/1309613083 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7fd2fc03bbf0 con 0x7fd30c104b90 2024-09-11T00:21:34.771 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:34.766+0000 7fd301e00640 1 --2- 172.21.15.12:0/1319948965 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c6daf16b00 0x55c6daf18fc0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fd2f400b9d0 tx=0x7fd2f400be90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2238b2d4e7e85da server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:21:34.771 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:34.766+0000 7fd2fb400640 1 -- 172.21.15.12:0/1319948965 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2f4016070 con 0x55c6daf16b00 2024-09-11T00:21:34.771 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:34.766+0000 7fd2fb400640 1 -- 172.21.15.12:0/1319948965 <== mon.2 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd2f40054d0 con 0x55c6daf16b00 2024-09-11T00:21:34.771 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:34.766+0000 7fd2fb400640 1 -- 172.21.15.12:0/1319948965 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2f4010460 con 0x55c6daf16b00 2024-09-11T00:21:34.771 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:34.766+0000 7fd314277740 1 -- 172.21.15.12:0/1319948965 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c6daf16b00 msgr2=0x55c6daf18fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:21:34.771 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:34.766+0000 7fd314277740 1 --2- 172.21.15.12:0/1319948965 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c6daf16b00 0x55c6daf18fc0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fd2f400b9d0 tx=0x7fd2f400be90 comp rx=0 tx=0).stop 2024-09-11T00:21:34.771 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:34.766+0000 7fd314277740 1 -- 172.21.15.12:0/1319948965 shutdown_connections 2024-09-11T00:21:34.771 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:34.766+0000 7fd314277740 1 --2- 172.21.15.12:0/1319948965 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c6daf16b00 0x55c6daf18fc0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:21:34.771 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:34.766+0000 7fd314277740 1 --2- 172.21.15.12:0/1319948965 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c6daf140c0 0x55c6daf164b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:21:34.771 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:34.766+0000 7fd314277740 1 --2- 172.21.15.12:0/1319948965 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c6daf11790 0x55c6daf13b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:21:34.771 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:34.766+0000 7fd314277740 1 -- 172.21.15.12:0/1319948965 >> 172.21.15.12:0/1319948965 conn(0x55c6daf0c8e0 msgr2=0x55c6daf0d670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:21:34.771 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:34.766+0000 7fd303e00640 1 --2- 172.21.15.12:0/1309613083 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fd2f00b52e0 0x7fd2f00b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:21:34.771 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:34.766+0000 7fd314277740 1 -- 172.21.15.12:0/1319948965 shutdown_connections 2024-09-11T00:21:34.772 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:34.766+0000 7fd314277740 1 -- 172.21.15.12:0/1319948965 wait complete. 2024-09-11T00:21:34.772 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:34.766+0000 7fd303e00640 1 -- 172.21.15.12:0/1309613083 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9208+0+0 (secure 0 0 0) 0x7fd2fc106890 con 0x7fd30c104b90 2024-09-11T00:21:34.772 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:34.766+0000 7fd301e00640 1 --2- 172.21.15.12:0/1309613083 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fd2f00b52e0 0x7fd2f00b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:21:34.772 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:34.766+0000 7fd314277740 1 -- start start 2024-09-11T00:21:34.772 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:34.766+0000 7fd314277740 1 Processor -- start 2024-09-11T00:21:34.772 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:34.766+0000 7fd301e00640 1 --2- 172.21.15.12:0/1309613083 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fd2f00b52e0 0x7fd2f00b77a0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fd2f40057c0 tx=0x7fd2f4004740 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:21:34.774 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:34.766+0000 7fd314277740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c6daf11790 0x55c6daf23650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:21:34.774 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:34.766+0000 7fd314277740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c6daf140c0 0x55c6daf23cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:21:34.774 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:34.766+0000 7fd314277740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c6daf16b00 0x55c6daf248c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:21:34.774 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:34.766+0000 7fd314277740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55c6dadabab0 con 0x55c6daf11790 2024-09-11T00:21:34.775 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:34.766+0000 7fd314277740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55c6dae1a570 con 0x55c6daf140c0 2024-09-11T00:21:34.775 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:34.766+0000 7fd314277740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55c6daf22260 con 0x55c6daf16b00 2024-09-11T00:21:34.775 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:34.766+0000 7fd314277740 10 client.0 _ll_register_callbacks cb 0x55c6dae29100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:21:34.775 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:34.766+0000 7fd30a800640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c6daf11790 0x55c6daf23650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:21:34.775 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:34.766+0000 7fd301e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c6daf140c0 0x55c6daf23cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:21:34.775 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:34.766+0000 7fd30b200640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c6daf16b00 0x55c6daf248c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:21:34.775 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:34.766+0000 7fd30b200640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c6daf16b00 0x55c6daf248c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.12:33412/0 (socket says 172.21.15.12:33412) 2024-09-11T00:21:34.775 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:34.766+0000 7fd30b200640 1 -- 172.21.15.12:0/3105677344 learned_addr learned my addr 172.21.15.12:0/3105677344 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:21:34.775 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:34.766+0000 7fd30a800640 1 -- 172.21.15.12:0/3105677344 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c6daf16b00 msgr2=0x55c6daf248c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:21:34.775 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:34.766+0000 7fd30a800640 1 --2- 172.21.15.12:0/3105677344 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c6daf16b00 0x55c6daf248c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:21:34.776 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:34.766+0000 7fd30a800640 1 -- 172.21.15.12:0/3105677344 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c6daf140c0 msgr2=0x55c6daf23cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:21:34.776 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:34.766+0000 7fd30a800640 1 --2- 172.21.15.12:0/3105677344 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c6daf140c0 0x55c6daf23cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:21:34.776 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:34.766+0000 7fd30a800640 1 -- 172.21.15.12:0/3105677344 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c6dabd5790 con 0x55c6daf11790 2024-09-11T00:21:34.776 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:34.766+0000 7fd30a800640 1 --2- 172.21.15.12:0/3105677344 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c6daf11790 0x55c6daf23650 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7fd2fc006010 tx=0x7fd2fc03b690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7fbf27e6beae890e server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:21:34.776 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:34.766+0000 7fd2efe00640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-09-11T00:21:34.776 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:34.766+0000 7fd2efe00640 1 -- 172.21.15.12:0/3105677344 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2fc0cc390 con 0x55c6daf11790 2024-09-11T00:21:34.776 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:34.766+0000 7fd2efe00640 1 -- 172.21.15.12:0/3105677344 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9208+0+0 (secure 0 0 0) 0x7fd2fc0d5f20 con 0x55c6daf11790 2024-09-11T00:21:34.776 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:34.766+0000 7fd314277740 1 -- 172.21.15.12:0/3105677344 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd2f40054d0 con 0x55c6daf11790 2024-09-11T00:21:34.776 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:34.766+0000 7fd2efe00640 1 -- 172.21.15.12:0/3105677344 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7fd2fc003e60 con 0x55c6daf11790 2024-09-11T00:21:34.776 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:34.766+0000 7fd314277740 10 client.17091 fetch_fsmap learned FSMap version 54 2024-09-11T00:21:34.776 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:34.766+0000 7fd314277740 1 -- 172.21.15.12:0/3105677344 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) -- 0x55c6dac4e620 con 0x55c6daf11790 2024-09-11T00:21:34.777 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:34.766+0000 7fd2efe00640 1 -- 172.21.15.12:0/3105677344 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd2f40054d0 con 0x55c6daf11790 2024-09-11T00:21:34.777 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:34.772+0000 7fd2efe00640 1 -- 172.21.15.12:0/3105677344 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 54) ==== 8422+0+0 (secure 0 0 0) 0x7fd2fc0cbe80 con 0x55c6daf11790 2024-09-11T00:21:34.777 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:34.772+0000 7fd314277740 10 client.17091 fetch_fsmap finished waiting for FSMap version 54 2024-09-11T00:21:34.777 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:34.772+0000 7fd314277740 10 client.17091 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T00:21:34.777 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:34.772+0000 7fd314277740 20 client.17091 populate_metadata read hostname 'smithi012' 2024-09-11T00:21:34.777 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:34.772+0000 7fd314277740 1 --2- 172.21.15.12:0/3105677344 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55c6daf2eee0 0x55c6daf31380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:21:34.777 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:34.772+0000 7fd314277740 4 client.17091 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T00:21:34.777 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:34.772+0000 7fd314277740 1 -- 172.21.15.12:0/3105677344 --> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c6dae86600 con 0x55c6daf2eee0 2024-09-11T00:21:34.777 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:34.772+0000 7fd301e00640 1 --2- 172.21.15.12:0/3105677344 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55c6daf2eee0 0x55c6daf31380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:21:34.777 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:34.772+0000 7fd301e00640 1 --2- 172.21.15.12:0/3105677344 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55c6daf2eee0 0x55c6daf31380 crc :-1 s=READY pgs=211 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=694fe5e653bedac0 server_cookie=e2d849b6c0416d13 in_seq=0 out_seq=0 2024-09-11T00:21:34.777 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:34.772+0000 7fd2efe00640 10 client.17091 ms_handle_connect on v2:172.21.15.165:6838/501312984 2024-09-11T00:21:34.777 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:34.772+0000 7fd2efe00640 1 -- 172.21.15.12:0/3105677344 <== mds.0 v2:172.21.15.165:6838/501312984 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55c6dae86600 con 0x55c6daf2eee0 2024-09-11T00:21:34.777 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:34.772+0000 7fd2efe00640 10 client.17091 handle_command_reply: tid=0 2024-09-11T00:21:34.778 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:34.772+0000 7fd314277740 1 client.17091 shutdown 2024-09-11T00:21:34.778 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:34.772+0000 7fd314277740 1 -- 172.21.15.12:0/3105677344 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c6daf11790 msgr2=0x55c6daf23650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:21:34.779 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:34.772+0000 7fd314277740 1 --2- 172.21.15.12:0/3105677344 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c6daf11790 0x55c6daf23650 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7fd2fc006010 tx=0x7fd2fc03b690 comp rx=0 tx=0).stop 2024-09-11T00:21:34.779 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:34.772+0000 7fd314277740 1 -- 172.21.15.12:0/3105677344 shutdown_connections 2024-09-11T00:21:34.779 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:34.772+0000 7fd314277740 1 --2- 172.21.15.12:0/3105677344 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55c6daf2eee0 0x55c6daf31380 crc :-1 s=READY pgs=211 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:21:34.779 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:34.772+0000 7fd314277740 1 --2- 172.21.15.12:0/3105677344 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c6daf16b00 0x55c6daf248c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:21:34.779 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:34.772+0000 7fd314277740 1 --2- 172.21.15.12:0/3105677344 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c6daf140c0 0x55c6daf23cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:21:34.779 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:34.772+0000 7fd314277740 1 --2- 172.21.15.12:0/3105677344 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c6daf11790 0x55c6daf23650 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:21:34.779 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:34.772+0000 7fd314277740 1 -- 172.21.15.12:0/3105677344 >> 172.21.15.12:0/3105677344 conn(0x55c6daf0c8e0 msgr2=0x55c6daf0ba60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:21:34.779 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:34.772+0000 7fd2efe00640 0 client.17091 ms_handle_reset on v2:172.21.15.165:6838/501312984 2024-09-11T00:21:34.779 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:34.772+0000 7fd314277740 1 -- 172.21.15.12:0/3105677344 shutdown_connections 2024-09-11T00:21:34.779 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:34.772+0000 7fd314277740 1 -- 172.21.15.12:0/3105677344 wait complete. 2024-09-11T00:21:34.779 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:34.772+0000 7fd314277740 20 client.17091 trim_cache size 0 max 16384 2024-09-11T00:21:34.791 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:34.789+0000 7fd314277740 1 Processor -- start 2024-09-11T00:21:34.792 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:34.789+0000 7fd314277740 1 -- start start 2024-09-11T00:21:34.792 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:34.790+0000 7fd314277740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c6daf11790 0x55c6daf16ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:21:34.792 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:34.790+0000 7fd314277740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c6daf140c0 0x55c6daf17420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:21:34.792 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:34.790+0000 7fd314277740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c6daf16b00 0x55c6daf0d140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:21:34.792 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:34.790+0000 7fd314277740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fd30c1a95c0 con 0x55c6daf11790 2024-09-11T00:21:34.792 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:34.790+0000 7fd314277740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fd30c1ab2e0 con 0x55c6daf16b00 2024-09-11T00:21:34.792 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:34.790+0000 7fd314277740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fd30c1a9cf0 con 0x55c6daf140c0 2024-09-11T00:21:34.792 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:34.790+0000 7fd30a800640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c6daf11790 0x55c6daf16ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:21:34.792 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:34.790+0000 7fd30a800640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c6daf11790 0x55c6daf16ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:39916/0 (socket says 172.21.15.12:39916) 2024-09-11T00:21:34.792 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:34.790+0000 7fd30a800640 1 -- 172.21.15.12:0/122943213 learned_addr learned my addr 172.21.15.12:0/122943213 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:21:34.792 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:34.790+0000 7fd301e00640 1 --2- 172.21.15.12:0/122943213 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c6daf140c0 0x55c6daf17420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:21:34.792 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:34.790+0000 7fd30a800640 1 -- 172.21.15.12:0/122943213 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c6daf140c0 msgr2=0x55c6daf17420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:21:34.793 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:34.790+0000 7fd30a800640 1 --2- 172.21.15.12:0/122943213 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c6daf140c0 0x55c6daf17420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:21:34.793 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:34.790+0000 7fd30a800640 1 -- 172.21.15.12:0/122943213 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c6daf16b00 msgr2=0x55c6daf0d140 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-11T00:21:34.793 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:34.790+0000 7fd30a800640 1 --2- 172.21.15.12:0/122943213 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c6daf16b00 0x55c6daf0d140 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:21:34.793 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:34.790+0000 7fd30a800640 1 -- 172.21.15.12:0/122943213 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c6daf239c0 con 0x55c6daf11790 2024-09-11T00:21:34.793 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:34.790+0000 7fd301e00640 1 --2- 172.21.15.12:0/122943213 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c6daf140c0 0x55c6daf17420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T00:21:34.793 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:34.790+0000 7fd30a800640 1 --2- 172.21.15.12:0/122943213 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c6daf11790 0x55c6daf16ee0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7fd2fc0d1720 tx=0x7fd2fc005e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eac5ad6f04e7b20a server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:21:34.794 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:34.790+0000 7fd2faa00640 1 -- 172.21.15.12:0/122943213 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2fc0cc390 con 0x55c6daf11790 2024-09-11T00:21:34.794 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:34.790+0000 7fd2faa00640 1 -- 172.21.15.12:0/122943213 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd2fc0b40e0 con 0x55c6daf11790 2024-09-11T00:21:34.794 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:34.790+0000 7fd2faa00640 1 -- 172.21.15.12:0/122943213 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2fc0d6200 con 0x55c6daf11790 2024-09-11T00:21:34.794 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:34.790+0000 7fd314277740 1 -- 172.21.15.12:0/122943213 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c6daf11790 msgr2=0x55c6daf16ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:21:34.794 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:34.790+0000 7fd314277740 1 --2- 172.21.15.12:0/122943213 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c6daf11790 0x55c6daf16ee0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7fd2fc0d1720 tx=0x7fd2fc005e80 comp rx=0 tx=0).stop 2024-09-11T00:21:34.794 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:34.790+0000 7fd314277740 1 -- 172.21.15.12:0/122943213 shutdown_connections 2024-09-11T00:21:34.794 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:34.790+0000 7fd314277740 1 --2- 172.21.15.12:0/122943213 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c6daf16b00 0x55c6daf0d140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:21:34.795 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:34.790+0000 7fd314277740 1 --2- 172.21.15.12:0/122943213 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c6daf140c0 0x55c6daf17420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:21:34.795 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:34.790+0000 7fd314277740 1 --2- 172.21.15.12:0/122943213 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c6daf11790 0x55c6daf16ee0 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:21:34.795 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:34.790+0000 7fd314277740 1 -- 172.21.15.12:0/122943213 >> 172.21.15.12:0/122943213 conn(0x55c6daf0c8e0 msgr2=0x55c6daf10c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:21:34.795 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:34.790+0000 7fd314277740 1 -- 172.21.15.12:0/122943213 shutdown_connections 2024-09-11T00:21:34.795 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:34.790+0000 7fd314277740 1 -- 172.21.15.12:0/122943213 wait complete. 2024-09-11T00:21:34.797 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:34.790+0000 7fd314277740 1 -- start start 2024-09-11T00:21:34.797 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:34.790+0000 7fd314277740 1 Processor -- start 2024-09-11T00:21:34.797 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:34.790+0000 7fd314277740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c6daf11790 0x55c6daf2a580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:21:34.797 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:34.790+0000 7fd314277740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c6daf140c0 0x55c6daf2ab00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:21:34.797 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:34.790+0000 7fd314277740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c6daf16b00 0x55c6daf290f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:21:34.797 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:34.790+0000 7fd314277740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fd30c1aac10 con 0x55c6daf16b00 2024-09-11T00:21:34.798 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:34.790+0000 7fd314277740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fd30c1aa480 con 0x55c6daf140c0 2024-09-11T00:21:34.798 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:34.790+0000 7fd314277740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fd30c1a7600 con 0x55c6daf11790 2024-09-11T00:21:34.798 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:34.790+0000 7fd314277740 10 client.0 _ll_register_callbacks cb 0x7fd2fc004570 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:21:34.798 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:34.790+0000 7fd30b200640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c6daf16b00 0x55c6daf290f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:21:34.798 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:34.790+0000 7fd301e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c6daf140c0 0x55c6daf2ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:21:34.798 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:34.790+0000 7fd30b200640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c6daf16b00 0x55c6daf290f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:39930/0 (socket says 172.21.15.12:39930) 2024-09-11T00:21:34.798 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:34.790+0000 7fd30b200640 1 -- 172.21.15.12:0/3729057611 learned_addr learned my addr 172.21.15.12:0/3729057611 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:21:34.798 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:34.790+0000 7fd30a800640 1 --2- 172.21.15.12:0/3729057611 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c6daf11790 0x55c6daf2a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:21:34.798 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:34.790+0000 7fd30b200640 1 -- 172.21.15.12:0/3729057611 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c6daf11790 msgr2=0x55c6daf2a580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:21:34.798 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:34.790+0000 7fd30b200640 1 --2- 172.21.15.12:0/3729057611 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c6daf11790 0x55c6daf2a580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:21:34.798 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:34.790+0000 7fd30b200640 1 -- 172.21.15.12:0/3729057611 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c6daf140c0 msgr2=0x55c6daf2ab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:21:34.798 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:34.790+0000 7fd30b200640 1 --2- 172.21.15.12:0/3729057611 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c6daf140c0 0x55c6daf2ab00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:21:34.799 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:34.790+0000 7fd30b200640 1 -- 172.21.15.12:0/3729057611 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c6daf13ef0 con 0x55c6daf16b00 2024-09-11T00:21:34.799 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:34.790+0000 7fd301e00640 1 --2- 172.21.15.12:0/3729057611 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c6daf140c0 0x55c6daf2ab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:21:34.799 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:34.790+0000 7fd30a800640 1 --2- 172.21.15.12:0/3729057611 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c6daf11790 0x55c6daf2a580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:21:34.799 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:34.790+0000 7fd30b200640 1 --2- 172.21.15.12:0/3729057611 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c6daf16b00 0x55c6daf290f0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7fd3040119d0 tx=0x7fd304011e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79ac7db770330298 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:21:34.799 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:34.790+0000 7fd2efe00640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-09-11T00:21:34.799 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:34.790+0000 7fd2efe00640 1 -- 172.21.15.12:0/3729057611 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd304003c40 con 0x55c6daf16b00 2024-09-11T00:21:34.799 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:34.790+0000 7fd2efe00640 1 -- 172.21.15.12:0/3729057611 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9208+0+0 (secure 0 0 0) 0x7fd304003de0 con 0x55c6daf16b00 2024-09-11T00:21:34.799 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:34.790+0000 7fd314277740 1 -- 172.21.15.12:0/3729057611 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd2fc0b40e0 con 0x55c6daf16b00 2024-09-11T00:21:34.799 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:34.790+0000 7fd2efe00640 1 -- 172.21.15.12:0/3729057611 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7fd30400f960 con 0x55c6daf16b00 2024-09-11T00:21:34.799 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:34.790+0000 7fd314277740 10 client.17097 fetch_fsmap learned FSMap version 54 2024-09-11T00:21:34.799 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:34.790+0000 7fd314277740 1 -- 172.21.15.12:0/3729057611 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) -- 0x55c6daf9c520 con 0x55c6daf16b00 2024-09-11T00:21:34.799 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:34.790+0000 7fd2efe00640 1 -- 172.21.15.12:0/3729057611 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd2fc0b40e0 con 0x55c6daf16b00 2024-09-11T00:21:34.800 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:34.790+0000 7fd2efe00640 1 -- 172.21.15.12:0/3729057611 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 54) ==== 8422+0+0 (secure 0 0 0) 0x7fd30401b420 con 0x55c6daf16b00 2024-09-11T00:21:34.800 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:34.790+0000 7fd314277740 10 client.17097 fetch_fsmap finished waiting for FSMap version 54 2024-09-11T00:21:34.800 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:34.790+0000 7fd314277740 10 client.17097 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T00:21:34.800 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:34.790+0000 7fd314277740 20 client.17097 populate_metadata read hostname 'smithi012' 2024-09-11T00:21:34.800 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:34.790+0000 7fd314277740 1 --2- 172.21.15.12:0/3729057611 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55c6daf9d040 0x55c6daf9f460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:21:34.800 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:34.790+0000 7fd314277740 4 client.17097 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T00:21:34.800 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:34.790+0000 7fd314277740 1 -- 172.21.15.12:0/3729057611 --> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55c6dae820b0 con 0x55c6daf9d040 2024-09-11T00:21:34.800 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:34.790+0000 7fd30a800640 1 --2- 172.21.15.12:0/3729057611 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55c6daf9d040 0x55c6daf9f460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:21:34.800 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:34.796+0000 7fd30a800640 1 --2- 172.21.15.12:0/3729057611 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55c6daf9d040 0x55c6daf9f460 crc :-1 s=READY pgs=212 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=95bd9a734eda2507 server_cookie=7bac8b338b1d5e7d in_seq=0 out_seq=0 2024-09-11T00:21:34.800 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:34.796+0000 7fd2efe00640 10 client.17097 ms_handle_connect on v2:172.21.15.165:6838/501312984 2024-09-11T00:21:34.842 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:34.837+0000 7fd2efe00640 1 -- 172.21.15.12:0/3729057611 <== mds.0 v2:172.21.15.165:6838/501312984 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55c6dae820b0 con 0x55c6daf9d040 2024-09-11T00:21:34.842 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:34.837+0000 7fd2efe00640 10 client.17097 handle_command_reply: tid=0 2024-09-11T00:21:34.842 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:34.837+0000 7fd314277740 1 client.17097 shutdown 2024-09-11T00:21:34.842 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:34.837+0000 7fd314277740 1 -- 172.21.15.12:0/3729057611 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c6daf16b00 msgr2=0x55c6daf290f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:21:34.842 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:34.837+0000 7fd314277740 1 --2- 172.21.15.12:0/3729057611 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c6daf16b00 0x55c6daf290f0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7fd3040119d0 tx=0x7fd304011e90 comp rx=0 tx=0).stop 2024-09-11T00:21:34.842 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:34.837+0000 7fd314277740 1 -- 172.21.15.12:0/3729057611 shutdown_connections 2024-09-11T00:21:34.842 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:34.837+0000 7fd314277740 1 --2- 172.21.15.12:0/3729057611 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55c6daf9d040 0x55c6daf9f460 crc :-1 s=READY pgs=212 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:21:34.842 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:34.837+0000 7fd314277740 1 --2- 172.21.15.12:0/3729057611 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c6daf16b00 0x55c6daf290f0 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:21:34.842 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:34.837+0000 7fd314277740 1 --2- 172.21.15.12:0/3729057611 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c6daf140c0 0x55c6daf2ab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:21:34.843 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:34.837+0000 7fd314277740 1 --2- 172.21.15.12:0/3729057611 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55c6daf11790 0x55c6daf2a580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:21:34.843 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:34.837+0000 7fd314277740 1 -- 172.21.15.12:0/3729057611 >> 172.21.15.12:0/3729057611 conn(0x55c6daf0c8e0 msgr2=0x55c6daf10af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:21:34.843 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:34.837+0000 7fd2efe00640 0 client.17097 ms_handle_reset on v2:172.21.15.165:6838/501312984 2024-09-11T00:21:34.843 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:34.837+0000 7fd314277740 1 -- 172.21.15.12:0/3729057611 shutdown_connections 2024-09-11T00:21:34.843 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:34.837+0000 7fd314277740 1 -- 172.21.15.12:0/3729057611 wait complete. 2024-09-11T00:21:34.843 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:34.837+0000 7fd314277740 20 client.17097 trim_cache size 0 max 16384 2024-09-11T00:21:34.843 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:{ 2024-09-11T00:21:34.843 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "message": "", 2024-09-11T00:21:34.843 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "return_code": 0 2024-09-11T00:21:34.844 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:} 2024-09-11T00:21:34.844 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:34.837+0000 7fd311400640 1 -- 172.21.15.12:0/1309613083 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fd2f00b52e0 msgr2=0x7fd2f00b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:21:34.844 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:34.837+0000 7fd311400640 1 --2- 172.21.15.12:0/1309613083 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fd2f00b52e0 0x7fd2f00b77a0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fd2f40057c0 tx=0x7fd2f4004740 comp rx=0 tx=0).stop 2024-09-11T00:21:34.844 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:34.837+0000 7fd311400640 1 -- 172.21.15.12:0/1309613083 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd30c104b90 msgr2=0x7fd30c19b010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:21:34.845 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:34.837+0000 7fd311400640 1 --2- 172.21.15.12:0/1309613083 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd30c104b90 0x7fd30c19b010 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7fd2fc005940 tx=0x7fd2fc0042f0 comp rx=0 tx=0).stop 2024-09-11T00:21:34.845 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:34.837+0000 7fd311400640 1 -- 172.21.15.12:0/1309613083 shutdown_connections 2024-09-11T00:21:34.845 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:34.837+0000 7fd311400640 1 --2- 172.21.15.12:0/1309613083 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fd2f00b52e0 0x7fd2f00b77a0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:21:34.845 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:34.837+0000 7fd311400640 1 --2- 172.21.15.12:0/1309613083 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd30c1066a0 0x7fd30c19c290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:21:34.845 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:34.837+0000 7fd311400640 1 --2- 172.21.15.12:0/1309613083 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd30c105ce0 0x7fd30c19b660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:21:34.845 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:34.837+0000 7fd311400640 1 --2- 172.21.15.12:0/1309613083 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd30c104b90 0x7fd30c19b010 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:21:34.845 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:34.837+0000 7fd311400640 1 -- 172.21.15.12:0/1309613083 >> 172.21.15.12:0/1309613083 conn(0x7fd30c100470 msgr2=0x7fd30c107060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:21:34.845 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:34.837+0000 7fd311400640 1 -- 172.21.15.12:0/1309613083 shutdown_connections 2024-09-11T00:21:34.845 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:34.837+0000 7fd311400640 1 -- 172.21.15.12:0/1309613083 wait complete. 2024-09-11T00:21:35.152 INFO:tasks.workunit.client.0.smithi012.stdout: 10 27905 18.43 MB/sec warmup 91 sec latency 41553.264 ms 2024-09-11T00:21:35.602 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:35.600+0000 7fe0e3400640 1 -- 172.21.15.12:0/1654440787 <== mds.0 v2:172.21.15.165:6838/501312984 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x560720e19490 con 0x560720eb26e0 2024-09-11T00:21:35.602 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:35.600+0000 7fe0e3400640 10 client.17043 handle_command_reply: tid=0 2024-09-11T00:21:35.603 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:35.600+0000 7fe10b68e740 1 client.17043 shutdown 2024-09-11T00:21:35.603 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:35.600+0000 7fe10b68e740 1 -- 172.21.15.12:0/1654440787 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560720eb0290 msgr2=0x560720eb5cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:21:35.603 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:35.600+0000 7fe10b68e740 1 --2- 172.21.15.12:0/1654440787 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560720eb0290 0x560720eb5cf0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7fe0fc009fd0 tx=0x7fe0fc004a40 comp rx=0 tx=0).stop 2024-09-11T00:21:35.603 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:35.600+0000 7fe10b68e740 1 -- 172.21.15.12:0/1654440787 shutdown_connections 2024-09-11T00:21:35.603 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:35.600+0000 7fe10b68e740 1 --2- 172.21.15.12:0/1654440787 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x560720eb26e0 0x560720f3bb80 crc :-1 s=READY pgs=206 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:21:35.603 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:35.600+0000 7fe10b68e740 1 --2- 172.21.15.12:0/1654440787 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560720eb0290 0x560720eb5cf0 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:21:35.603 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:35.600+0000 7fe10b68e740 1 --2- 172.21.15.12:0/1654440787 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560720ea7e10 0x560720eb57b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:21:35.603 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:35.600+0000 7fe10b68e740 1 --2- 172.21.15.12:0/1654440787 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560720e706f0 0x560720ebae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:21:35.603 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:35.600+0000 7fe0e3400640 0 client.17043 ms_handle_reset on v2:172.21.15.165:6838/501312984 2024-09-11T00:21:35.603 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:35.600+0000 7fe10b68e740 1 -- 172.21.15.12:0/1654440787 >> 172.21.15.12:0/1654440787 conn(0x560720ea42c0 msgr2=0x560720ea2aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:21:35.603 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:35.601+0000 7fe10b68e740 1 -- 172.21.15.12:0/1654440787 shutdown_connections 2024-09-11T00:21:35.603 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:35.601+0000 7fe10b68e740 1 -- 172.21.15.12:0/1654440787 wait complete. 2024-09-11T00:21:35.603 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:35.601+0000 7fe10b68e740 20 client.17043 trim_cache size 0 max 16384 2024-09-11T00:21:35.604 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-09-11T00:21:35.604 INFO:teuthology.orchestra.run.smithi012.stdout: "epoch": 54, 2024-09-11T00:21:35.604 INFO:teuthology.orchestra.run.smithi012.stdout: "leader": 24674, 2024-09-11T00:21:35.604 INFO:teuthology.orchestra.run.smithi012.stdout: "set_version": 42, 2024-09-11T00:21:35.604 INFO:teuthology.orchestra.run.smithi012.stdout: "sets": { 2024-09-11T00:21:35.604 INFO:teuthology.orchestra.run.smithi012.stdout: "280397bf": { 2024-09-11T00:21:35.604 INFO:teuthology.orchestra.run.smithi012.stdout: "version": 42, 2024-09-11T00:21:35.605 INFO:teuthology.orchestra.run.smithi012.stdout: "age_ref": 0.0, 2024-09-11T00:21:35.605 INFO:teuthology.orchestra.run.smithi012.stdout: "state": { 2024-09-11T00:21:35.605 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "QUIESCED", 2024-09-11T00:21:35.605 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 0.0 2024-09-11T00:21:35.605 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:21:35.605 INFO:teuthology.orchestra.run.smithi012.stdout: "timeout": 90.0, 2024-09-11T00:21:35.605 INFO:teuthology.orchestra.run.smithi012.stdout: "expiration": 142.8, 2024-09-11T00:21:35.605 INFO:teuthology.orchestra.run.smithi012.stdout: "members": { 2024-09-11T00:21:35.605 INFO:teuthology.orchestra.run.smithi012.stdout: "file:/": { 2024-09-11T00:21:35.605 INFO:teuthology.orchestra.run.smithi012.stdout: "excluded": false, 2024-09-11T00:21:35.605 INFO:teuthology.orchestra.run.smithi012.stdout: "state": { 2024-09-11T00:21:35.605 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "QUIESCED", 2024-09-11T00:21:35.605 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 0.0 2024-09-11T00:21:35.605 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:21:35.605 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:21:35.606 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:21:35.606 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:21:35.606 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:21:35.606 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-09-11T00:21:35.606 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:35.602+0000 7fe103200640 1 -- 172.21.15.12:0/1317748335 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fe0e80b5410 msgr2=0x7fe0e80b78d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:21:35.606 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:35.602+0000 7fe103200640 1 --2- 172.21.15.12:0/1317748335 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fe0e80b5410 0x7fe0e80b78d0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fe0fc005fd0 tx=0x7fe0fc009600 comp rx=0 tx=0).stop 2024-09-11T00:21:35.606 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:35.602+0000 7fe103200640 1 -- 172.21.15.12:0/1317748335 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe104102c10 msgr2=0x7fe10419b260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:21:35.606 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:35.602+0000 7fe103200640 1 --2- 172.21.15.12:0/1317748335 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe104102c10 0x7fe10419b260 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7fe0f400e9f0 tx=0x7fe0f400eeb0 comp rx=0 tx=0).stop 2024-09-11T00:21:35.606 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:35.602+0000 7fe103200640 1 -- 172.21.15.12:0/1317748335 shutdown_connections 2024-09-11T00:21:35.606 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:35.602+0000 7fe103200640 1 --2- 172.21.15.12:0/1317748335 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fe0e80b5410 0x7fe0e80b78d0 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:21:35.607 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:35.602+0000 7fe103200640 1 --2- 172.21.15.12:0/1317748335 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe104108fa0 0x7fe1041a0b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:21:35.607 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:35.602+0000 7fe103200640 1 --2- 172.21.15.12:0/1317748335 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe104108620 0x7fe10419b7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:21:35.607 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:35.602+0000 7fe103200640 1 --2- 172.21.15.12:0/1317748335 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe104102c10 0x7fe10419b260 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:21:35.607 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:35.602+0000 7fe103200640 1 -- 172.21.15.12:0/1317748335 >> 172.21.15.12:0/1317748335 conn(0x7fe1040fe660 msgr2=0x7fe104105c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:21:35.607 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:35.602+0000 7fe103200640 1 -- 172.21.15.12:0/1317748335 shutdown_connections 2024-09-11T00:21:35.607 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:35.602+0000 7fe103200640 1 -- 172.21.15.12:0/1317748335 wait complete. 2024-09-11T00:21:36.152 INFO:tasks.workunit.client.0.smithi012.stdout: 10 27912 18.23 MB/sec warmup 92 sec latency 41988.563 ms 2024-09-11T00:21:36.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:21:36 smithi012 ceph-mon[30333]: pgmap v834: 129 pgs: 129 active+clean; 407 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 132 KiB/s rd, 305 KiB/s wr, 118 op/s 2024-09-11T00:21:36.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:21:36 smithi178 ceph-mon[35919]: pgmap v834: 129 pgs: 129 active+clean; 407 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 132 KiB/s rd, 305 KiB/s wr, 118 op/s 2024-09-11T00:21:36.532 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:21:36 smithi165 ceph-mon[35356]: pgmap v834: 129 pgs: 129 active+clean; 407 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 132 KiB/s rd, 305 KiB/s wr, 118 op/s 2024-09-11T00:21:36.537 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '280397bf', quiesce took 46.5 seconds. Will release after: -23.75 2024-09-11T00:21:36.537 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid ff3ad74a-6fcf-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T00:21:36.957 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:21:37.152 INFO:tasks.workunit.client.0.smithi012.stdout: 10 27912 18.03 MB/sec warmup 93 sec latency 2397.080 ms 2024-09-11T00:21:37.550 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:37.548+0000 7fa70c400640 1 -- 172.21.15.12:0/1273600395 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa70410b330 msgr2=0x7fa704111b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:21:37.550 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:37.548+0000 7fa70c400640 1 --2- 172.21.15.12:0/1273600395 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa70410b330 0x7fa704111b30 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7fa6fc009ca0 tx=0x7fa6fc02a8c0 comp rx=0 tx=0).stop 2024-09-11T00:21:37.551 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:37.549+0000 7fa70c400640 1 -- 172.21.15.12:0/1273600395 shutdown_connections 2024-09-11T00:21:37.551 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:37.549+0000 7fa70c400640 1 --2- 172.21.15.12:0/1273600395 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa70410b330 0x7fa704111b30 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:21:37.551 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:37.549+0000 7fa70c400640 1 --2- 172.21.15.12:0/1273600395 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa70410a9b0 0x7fa70410adf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:21:37.551 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:37.549+0000 7fa70c400640 1 --2- 172.21.15.12:0/1273600395 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa704104fa0 0x7fa704105380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:21:37.551 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:37.549+0000 7fa70c400640 1 -- 172.21.15.12:0/1273600395 >> 172.21.15.12:0/1273600395 conn(0x7fa704100a50 msgr2=0x7fa704102e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:21:37.551 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:37.549+0000 7fa70c400640 1 -- 172.21.15.12:0/1273600395 shutdown_connections 2024-09-11T00:21:37.551 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:37.549+0000 7fa70c400640 1 -- 172.21.15.12:0/1273600395 wait complete. 2024-09-11T00:21:37.552 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:37.550+0000 7fa70c400640 1 Processor -- start 2024-09-11T00:21:37.552 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:37.550+0000 7fa70c400640 1 -- start start 2024-09-11T00:21:37.552 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:37.550+0000 7fa70c400640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa704104fa0 0x7fa70419b440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:21:37.552 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:37.550+0000 7fa70c400640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa70410a9b0 0x7fa70419b980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:21:37.553 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:37.550+0000 7fa70c400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa70410b330 0x7fa7041a0d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:21:37.553 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:37.550+0000 7fa70c400640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fa704114fb0 con 0x7fa70410a9b0 2024-09-11T00:21:37.553 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:37.550+0000 7fa70c400640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fa704114e30 con 0x7fa70410b330 2024-09-11T00:21:37.553 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:37.550+0000 7fa70c400640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fa704115130 con 0x7fa704104fa0 2024-09-11T00:21:37.553 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:37.550+0000 7fa709000640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa70410a9b0 0x7fa70419b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:21:37.553 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:37.551+0000 7fa709a00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa704104fa0 0x7fa70419b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:21:37.553 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:37.551+0000 7fa709000640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa70410a9b0 0x7fa70419b980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:39942/0 (socket says 172.21.15.12:39942) 2024-09-11T00:21:37.553 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:37.551+0000 7fa709000640 1 -- 172.21.15.12:0/500840597 learned_addr learned my addr 172.21.15.12:0/500840597 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:21:37.553 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:37.551+0000 7fa70a400640 1 --2- 172.21.15.12:0/500840597 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa70410b330 0x7fa7041a0d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:21:37.553 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:37.551+0000 7fa709000640 1 -- 172.21.15.12:0/500840597 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa704104fa0 msgr2=0x7fa70419b440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:21:37.553 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:37.551+0000 7fa709000640 1 --2- 172.21.15.12:0/500840597 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa704104fa0 0x7fa70419b440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:21:37.553 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:37.551+0000 7fa709000640 1 -- 172.21.15.12:0/500840597 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa70410b330 msgr2=0x7fa7041a0d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:21:37.553 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:37.551+0000 7fa709000640 1 --2- 172.21.15.12:0/500840597 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa70410b330 0x7fa7041a0d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:21:37.554 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:37.551+0000 7fa709000640 1 -- 172.21.15.12:0/500840597 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7041a14f0 con 0x7fa70410a9b0 2024-09-11T00:21:37.554 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:37.551+0000 7fa709a00640 1 --2- 172.21.15.12:0/500840597 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa704104fa0 0x7fa70419b440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:21:37.554 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:37.551+0000 7fa709000640 1 --2- 172.21.15.12:0/500840597 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa70410a9b0 0x7fa70419b980 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7fa6f000ea60 tx=0x7fa6f000ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:21:37.554 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:37.552+0000 7fa702000640 1 -- 172.21.15.12:0/500840597 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6f000cde0 con 0x7fa70410a9b0 2024-09-11T00:21:37.555 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:37.552+0000 7fa702000640 1 -- 172.21.15.12:0/500840597 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa6f00035f0 con 0x7fa70410a9b0 2024-09-11T00:21:37.555 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:37.552+0000 7fa70c400640 1 -- 172.21.15.12:0/500840597 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa7041a17e0 con 0x7fa70410a9b0 2024-09-11T00:21:37.555 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:37.552+0000 7fa702000640 1 -- 172.21.15.12:0/500840597 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6f0010690 con 0x7fa70410a9b0 2024-09-11T00:21:37.555 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:37.552+0000 7fa70c400640 1 -- 172.21.15.12:0/500840597 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7041a7ea0 con 0x7fa70410a9b0 2024-09-11T00:21:37.555 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:37.553+0000 7fa70c400640 1 -- 172.21.15.12:0/500840597 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa704105380 con 0x7fa70410a9b0 2024-09-11T00:21:37.556 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:37.555+0000 7fa702000640 1 -- 172.21.15.12:0/500840597 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7fa6f0004000 con 0x7fa70410a9b0 2024-09-11T00:21:37.558 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:37.556+0000 7fa702000640 1 --2- 172.21.15.12:0/500840597 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fa6ec0b5360 0x7fa6ec0b7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:21:37.558 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:37.557+0000 7fa702000640 1 -- 172.21.15.12:0/500840597 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9208+0+0 (secure 0 0 0) 0x7fa6f00e6210 con 0x7fa70410a9b0 2024-09-11T00:21:37.558 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:37.557+0000 7fa709a00640 1 --2- 172.21.15.12:0/500840597 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fa6ec0b5360 0x7fa6ec0b7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:21:37.559 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:37.557+0000 7fa709a00640 1 --2- 172.21.15.12:0/500840597 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fa6ec0b5360 0x7fa6ec0b7820 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fa6f8009fc0 tx=0x7fa6f8005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:21:37.562 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:37.561+0000 7fa702000640 1 -- 172.21.15.12:0/500840597 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7fa6f00acca0 con 0x7fa70410a9b0 2024-09-11T00:21:37.795 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:37.793+0000 7fa70c400640 1 -- 172.21.15.12:0/500840597 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa70402e5c0 con 0x7fa70410a9b0 2024-09-11T00:21:37.795 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:37.794+0000 7fa702000640 1 -- 172.21.15.12:0/500840597 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 54 v54) ==== 94+0+11986 (secure 0 0 0) 0x7fa6f00ac5a0 con 0x7fa70410a9b0 2024-09-11T00:21:37.796 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-11T00:21:37.796 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":54,"btime":"2024-09-11T00:20:52:794956+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14577,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6835/447171591","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":447171591},{"type":"v1","addr":"172.21.15.12:6835","nonce":447171591}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14589,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/901486040","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":901486040},{"type":"v1","addr":"172.21.15.12:6837","nonce":901486040}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":54,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T00:04:04.739881+0000","modified":"2024-09-11T00:20:52.232549+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24674,"mds_1":24643,"mds_2":24638,"mds_3":24637,"mds_4":24614},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14631":{"gid":14631,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.12:6841/3069919437","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6840","nonce":3069919437},{"type":"v1","addr":"172.21.15.12:6841","nonce":3069919437}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24562":{"gid":24562,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6833/1141636112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":1141636112},{"type":"v1","addr":"172.21.15.178:6833","nonce":1141636112}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24581":{"gid":24581,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6833/3111980970","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6832","nonce":3111980970},{"type":"v1","addr":"172.21.15.165:6833","nonce":3111980970}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24607":{"gid":24607,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6835/4084868287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":4084868287},{"type":"v1","addr":"172.21.15.178:6835","nonce":4084868287}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24614":{"gid":24614,"name":"e","rank":4,"incarnation":26,"state":"up:active","state_seq":21,"addr":"172.21.15.165:6835/2425395274","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":2425395274},{"type":"v1","addr":"172.21.15.165:6835","nonce":2425395274}]},"join_fscid":-1,"export_targets":[0,1],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24637":{"gid":24637,"name":"i","rank":3,"incarnation":24,"state":"up:active","state_seq":18,"addr":"172.21.15.178:6839/70628097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6838","nonce":70628097},{"type":"v1","addr":"172.21.15.178:6839","nonce":70628097}]},"join_fscid":-1,"export_targets":[0,1],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24638":{"gid":24638,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":18,"addr":"172.21.15.165:6837/6215088","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6836","nonce":6215088},{"type":"v1","addr":"172.21.15.165:6837","nonce":6215088}]},"join_fscid":-1,"export_targets":[0],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24643":{"gid":24643,"name":"g","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.12:6839/3124093009","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6838","nonce":3124093009},{"type":"v1","addr":"172.21.15.12:6839","nonce":3124093009}]},"join_fscid":-1,"export_targets":[0,2,3,4],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24674":{"gid":24674,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.165:6839/501312984","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6838","nonce":501312984},{"type":"v1","addr":"172.21.15.165:6839","nonce":501312984}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25672":{"gid":25672,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6837/308601741","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":308601741},{"type":"v1","addr":"172.21.15.178:6837","nonce":308601741}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24674,"qdb_cluster":[24674,24643,24638,24637,24614]},"id":1}]} 2024-09-11T00:21:37.796 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 54 2024-09-11T00:21:37.799 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:37.797+0000 7fa70c400640 1 -- 172.21.15.12:0/500840597 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fa6ec0b5360 msgr2=0x7fa6ec0b7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:21:37.799 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:37.797+0000 7fa70c400640 1 --2- 172.21.15.12:0/500840597 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fa6ec0b5360 0x7fa6ec0b7820 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fa6f8009fc0 tx=0x7fa6f8005f90 comp rx=0 tx=0).stop 2024-09-11T00:21:37.800 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:37.798+0000 7fa70c400640 1 -- 172.21.15.12:0/500840597 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa70410a9b0 msgr2=0x7fa70419b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:21:37.800 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:37.798+0000 7fa70c400640 1 --2- 172.21.15.12:0/500840597 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa70410a9b0 0x7fa70419b980 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7fa6f000ea60 tx=0x7fa6f000ef20 comp rx=0 tx=0).stop 2024-09-11T00:21:37.800 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:37.798+0000 7fa70c400640 1 -- 172.21.15.12:0/500840597 shutdown_connections 2024-09-11T00:21:37.800 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:37.798+0000 7fa70c400640 1 --2- 172.21.15.12:0/500840597 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fa6ec0b5360 0x7fa6ec0b7820 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:21:37.800 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:37.798+0000 7fa70c400640 1 --2- 172.21.15.12:0/500840597 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa70410b330 0x7fa7041a0d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:21:37.800 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:37.798+0000 7fa70c400640 1 --2- 172.21.15.12:0/500840597 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa70410a9b0 0x7fa70419b980 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:21:37.800 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:37.798+0000 7fa70c400640 1 --2- 172.21.15.12:0/500840597 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa704104fa0 0x7fa70419b440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:21:37.800 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:37.798+0000 7fa70c400640 1 -- 172.21.15.12:0/500840597 >> 172.21.15.12:0/500840597 conn(0x7fa704100a50 msgr2=0x7fa704102e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:21:37.800 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:37.798+0000 7fa70c400640 1 -- 172.21.15.12:0/500840597 shutdown_connections 2024-09-11T00:21:37.800 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:37.798+0000 7fa70c400640 1 -- 172.21.15.12:0/500840597 wait complete. 2024-09-11T00:21:38.152 INFO:tasks.workunit.client.0.smithi012.stdout: 10 27912 17.84 MB/sec warmup 94 sec latency 3397.122 ms 2024-09-11T00:21:38.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:21:38 smithi012 ceph-mon[30333]: pgmap v835: 129 pgs: 129 active+clean; 389 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 192 KiB/s rd, 370 KiB/s wr, 145 op/s 2024-09-11T00:21:38.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:21:38 smithi012 ceph-mon[30333]: from='client.? 172.21.15.12:0/500840597' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T00:21:38.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:21:38 smithi178 ceph-mon[35919]: pgmap v835: 129 pgs: 129 active+clean; 389 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 192 KiB/s rd, 370 KiB/s wr, 145 op/s 2024-09-11T00:21:38.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:21:38 smithi178 ceph-mon[35919]: from='client.? 172.21.15.12:0/500840597' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T00:21:38.532 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:21:38 smithi165 ceph-mon[35356]: pgmap v835: 129 pgs: 129 active+clean; 389 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 192 KiB/s rd, 370 KiB/s wr, 145 op/s 2024-09-11T00:21:38.532 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:21:38 smithi165 ceph-mon[35356]: from='client.? 172.21.15.12:0/500840597' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T00:21:38.629 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24674 quiesce db --set-id 280397bf --release --await' 2024-09-11T00:21:38.630 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid ff3ad74a-6fcf-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24674 quiesce db --set-id 280397bf --release --await 2024-09-11T00:21:39.010 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:21:39.152 INFO:tasks.workunit.client.0.smithi012.stdout: 10 27912 17.65 MB/sec warmup 95 sec latency 4397.219 ms 2024-09-11T00:21:39.567 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:39.565+0000 7f8cb0200640 1 -- 172.21.15.12:0/2272762464 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8ca8104f90 msgr2=0x7f8ca8105370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:21:39.568 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:39.565+0000 7f8cb0200640 1 --2- 172.21.15.12:0/2272762464 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8ca8104f90 0x7f8ca8105370 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f8c9c009ca0 tx=0x7f8c9c02a8c0 comp rx=0 tx=0).stop 2024-09-11T00:21:39.568 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:39.566+0000 7f8cb0200640 1 -- 172.21.15.12:0/2272762464 shutdown_connections 2024-09-11T00:21:39.568 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:39.566+0000 7f8cb0200640 1 --2- 172.21.15.12:0/2272762464 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8ca810b320 0x7f8ca8111b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:21:39.568 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:39.566+0000 7f8cb0200640 1 --2- 172.21.15.12:0/2272762464 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8ca810a9a0 0x7f8ca810ade0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:21:39.568 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:39.566+0000 7f8cb0200640 1 --2- 172.21.15.12:0/2272762464 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8ca8104f90 0x7f8ca8105370 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:21:39.568 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:39.566+0000 7f8cb0200640 1 -- 172.21.15.12:0/2272762464 >> 172.21.15.12:0/2272762464 conn(0x7f8ca8100a00 msgr2=0x7f8ca8102e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:21:39.568 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:39.566+0000 7f8cb0200640 1 -- 172.21.15.12:0/2272762464 shutdown_connections 2024-09-11T00:21:39.568 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:39.566+0000 7f8cb0200640 1 -- 172.21.15.12:0/2272762464 wait complete. 2024-09-11T00:21:39.570 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:39.567+0000 7f8cb0200640 1 Processor -- start 2024-09-11T00:21:39.570 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:39.567+0000 7f8cb0200640 1 -- start start 2024-09-11T00:21:39.570 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:39.567+0000 7f8cb0200640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8ca8104f90 0x7f8ca819b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:21:39.570 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:39.567+0000 7f8cb0200640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8ca810a9a0 0x7f8ca819ba00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:21:39.570 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:39.567+0000 7f8cb0200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8ca810b320 0x7f8ca81a0da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:21:39.570 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:39.567+0000 7f8cb0200640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f8ca8114f30 con 0x7f8ca8104f90 2024-09-11T00:21:39.570 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:39.567+0000 7f8cb0200640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f8ca8114db0 con 0x7f8ca810b320 2024-09-11T00:21:39.571 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:39.567+0000 7f8cb0200640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f8ca81150b0 con 0x7f8ca810a9a0 2024-09-11T00:21:39.571 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:39.567+0000 7f8cad800640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8ca8104f90 0x7f8ca819b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:21:39.571 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:39.567+0000 7f8cad800640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8ca8104f90 0x7f8ca819b4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:53272/0 (socket says 172.21.15.12:53272) 2024-09-11T00:21:39.571 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:39.567+0000 7f8cad800640 1 -- 172.21.15.12:0/655032122 learned_addr learned my addr 172.21.15.12:0/655032122 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:21:39.571 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:39.567+0000 7f8cae200640 1 --2- 172.21.15.12:0/655032122 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8ca810b320 0x7f8ca81a0da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:21:39.571 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:39.567+0000 7f8cace00640 1 --2- 172.21.15.12:0/655032122 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8ca810a9a0 0x7f8ca819ba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:21:39.571 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:39.568+0000 7f8cad800640 1 -- 172.21.15.12:0/655032122 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8ca810a9a0 msgr2=0x7f8ca819ba00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:21:39.571 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:39.568+0000 7f8cad800640 1 --2- 172.21.15.12:0/655032122 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8ca810a9a0 0x7f8ca819ba00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:21:39.571 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:39.568+0000 7f8cad800640 1 -- 172.21.15.12:0/655032122 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8ca810b320 msgr2=0x7f8ca81a0da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:21:39.571 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:39.568+0000 7f8cad800640 1 --2- 172.21.15.12:0/655032122 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8ca810b320 0x7f8ca81a0da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:21:39.571 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:39.568+0000 7f8cad800640 1 -- 172.21.15.12:0/655032122 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ca81a1570 con 0x7f8ca8104f90 2024-09-11T00:21:39.571 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:39.568+0000 7f8cad800640 1 --2- 172.21.15.12:0/655032122 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8ca8104f90 0x7f8ca819b4c0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f8c9c006010 tx=0x7f8c9c0319f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:21:39.571 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:39.568+0000 7f8ca6000640 1 -- 172.21.15.12:0/655032122 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c9c003bf0 con 0x7f8ca8104f90 2024-09-11T00:21:39.572 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:39.569+0000 7f8ca6000640 1 -- 172.21.15.12:0/655032122 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8c9c003d90 con 0x7f8ca8104f90 2024-09-11T00:21:39.572 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:39.569+0000 7f8ca6000640 1 -- 172.21.15.12:0/655032122 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c9c02fc20 con 0x7f8ca8104f90 2024-09-11T00:21:39.572 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:39.569+0000 7f8cb0200640 1 -- 172.21.15.12:0/655032122 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ca81a1800 con 0x7f8ca8104f90 2024-09-11T00:21:39.572 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:39.569+0000 7f8cb0200640 1 -- 172.21.15.12:0/655032122 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ca81a7f20 con 0x7f8ca8104f90 2024-09-11T00:21:39.573 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:39.572+0000 7f8cb2cd5740 1 Processor -- start 2024-09-11T00:21:39.573 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:39.572+0000 7f8cb2cd5740 1 -- start start 2024-09-11T00:21:39.573 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:39.572+0000 7f8ca6000640 1 -- 172.21.15.12:0/655032122 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7f8c9c0046a0 con 0x7f8ca8104f90 2024-09-11T00:21:39.574 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:39.572+0000 7f8cb2cd5740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558aae4d7ff0 0x558aae4d83d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:21:39.574 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:39.572+0000 7f8cb2cd5740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558aae4d8910 0x558aae4e4360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:21:39.574 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:39.572+0000 7f8cb2cd5740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558aae4e48a0 0x558aae4e6c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:21:39.574 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:39.572+0000 7f8cb2cd5740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x558aae44f290 con 0x558aae4d7ff0 2024-09-11T00:21:39.574 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:39.572+0000 7f8cb2cd5740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x558aae43e6c0 con 0x558aae4d8910 2024-09-11T00:21:39.574 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:39.572+0000 7f8cb2cd5740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x558aae4424d0 con 0x558aae4e48a0 2024-09-11T00:21:39.574 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:39.573+0000 7f8cad800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558aae4d8910 0x558aae4e4360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:21:39.574 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:39.573+0000 7f8cae200640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558aae4d7ff0 0x558aae4d83d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:21:39.575 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:39.573+0000 7f8cae200640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558aae4d7ff0 0x558aae4d83d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:53282/0 (socket says 172.21.15.12:53282) 2024-09-11T00:21:39.575 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:39.573+0000 7f8cae200640 1 -- 172.21.15.12:0/1902948631 learned_addr learned my addr 172.21.15.12:0/1902948631 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:21:39.575 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:39.573+0000 7f8cace00640 1 --2- 172.21.15.12:0/1902948631 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558aae4e48a0 0x558aae4e6c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:21:39.576 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:39.573+0000 7f8cad800640 1 -- 172.21.15.12:0/1902948631 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558aae4e48a0 msgr2=0x558aae4e6c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:21:39.576 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:39.573+0000 7f8cad800640 1 --2- 172.21.15.12:0/1902948631 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558aae4e48a0 0x558aae4e6c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:21:39.576 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:39.573+0000 7f8cad800640 1 -- 172.21.15.12:0/1902948631 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558aae4d7ff0 msgr2=0x558aae4d83d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:21:39.576 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:39.573+0000 7f8cad800640 1 --2- 172.21.15.12:0/1902948631 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558aae4d7ff0 0x558aae4d83d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:21:39.576 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:39.573+0000 7f8cad800640 1 -- 172.21.15.12:0/1902948631 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558aae3cb460 con 0x558aae4d8910 2024-09-11T00:21:39.576 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:39.573+0000 7f8cad800640 1 --2- 172.21.15.12:0/1902948631 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558aae4d8910 0x558aae4e4360 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f8c9c1048a0 tx=0x7f8c9c10af90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=86e1a675f56ad602 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:21:39.576 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:39.574+0000 7f8c9aa00640 1 -- 172.21.15.12:0/1902948631 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c9c10f6e0 con 0x558aae4d8910 2024-09-11T00:21:39.577 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:39.574+0000 7f8c9aa00640 1 -- 172.21.15.12:0/1902948631 <== mon.2 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8c9c114030 con 0x558aae4d8910 2024-09-11T00:21:39.577 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:39.574+0000 7f8c9aa00640 1 -- 172.21.15.12:0/1902948631 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c9c107c90 con 0x558aae4d8910 2024-09-11T00:21:39.577 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:39.574+0000 7f8ca6000640 1 --2- 172.21.15.12:0/655032122 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f8c900b5360 0x7f8c900b7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:21:39.577 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:39.574+0000 7f8ca6000640 1 -- 172.21.15.12:0/655032122 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9208+0+0 (secure 0 0 0) 0x7f8c9c107230 con 0x7f8ca8104f90 2024-09-11T00:21:39.577 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:39.574+0000 7f8cae200640 1 --2- 172.21.15.12:0/655032122 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f8c900b5360 0x7f8c900b7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:21:39.577 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:39.575+0000 7f8cb2cd5740 1 -- 172.21.15.12:0/1902948631 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558aae4d8910 msgr2=0x558aae4e4360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:21:39.577 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:39.575+0000 7f8cb2cd5740 1 --2- 172.21.15.12:0/1902948631 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558aae4d8910 0x558aae4e4360 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f8c9c1048a0 tx=0x7f8c9c10af90 comp rx=0 tx=0).stop 2024-09-11T00:21:39.577 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:39.575+0000 7f8cb2cd5740 1 -- 172.21.15.12:0/1902948631 shutdown_connections 2024-09-11T00:21:39.577 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:39.575+0000 7f8cb2cd5740 1 --2- 172.21.15.12:0/1902948631 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558aae4e48a0 0x558aae4e6c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:21:39.577 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:39.575+0000 7f8cb2cd5740 1 --2- 172.21.15.12:0/1902948631 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558aae4d8910 0x558aae4e4360 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:21:39.577 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:39.575+0000 7f8cb2cd5740 1 --2- 172.21.15.12:0/1902948631 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558aae4d7ff0 0x558aae4d83d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:21:39.577 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:39.575+0000 7f8cb2cd5740 1 -- 172.21.15.12:0/1902948631 >> 172.21.15.12:0/1902948631 conn(0x558aae4d5e20 msgr2=0x558aae4d6fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:21:39.577 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:39.575+0000 7f8cb2cd5740 1 -- 172.21.15.12:0/1902948631 shutdown_connections 2024-09-11T00:21:39.577 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:39.575+0000 7f8cae200640 1 --2- 172.21.15.12:0/655032122 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f8c900b5360 0x7f8c900b7820 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f8ca0004650 tx=0x7f8ca000b2e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:21:39.578 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:39.575+0000 7f8cb2cd5740 1 -- 172.21.15.12:0/1902948631 wait complete. 2024-09-11T00:21:39.580 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:39.575+0000 7f8cb2cd5740 1 -- start start 2024-09-11T00:21:39.580 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:39.576+0000 7f8cb2cd5740 1 Processor -- start 2024-09-11T00:21:39.580 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:39.576+0000 7f8cb2cd5740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558aae4d7ff0 0x558aae4e1810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:21:39.580 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:39.576+0000 7f8cb2cd5740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558aae4d8910 0x558aae4e1d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:21:39.580 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:39.576+0000 7f8cb2cd5740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558aae4e48a0 0x558aae4e2290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:21:39.580 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:39.576+0000 7f8cb2cd5740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x558aae29fbb0 con 0x558aae4d8910 2024-09-11T00:21:39.580 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:39.576+0000 7f8cb2cd5740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x558aae3d96c0 con 0x558aae4e48a0 2024-09-11T00:21:39.580 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:39.576+0000 7f8cb2cd5740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x558aae43e6c0 con 0x558aae4d7ff0 2024-09-11T00:21:39.580 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:39.576+0000 7f8cb2cd5740 10 client.0 _ll_register_callbacks cb 0x558aae3f4490 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:21:39.580 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:39.576+0000 7f8cace00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558aae4d8910 0x558aae4e1d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:21:39.580 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:39.576+0000 7f8cace00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558aae4d8910 0x558aae4e1d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:53286/0 (socket says 172.21.15.12:53286) 2024-09-11T00:21:39.581 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:39.576+0000 7f8cace00640 1 -- 172.21.15.12:0/1962199764 learned_addr learned my addr 172.21.15.12:0/1962199764 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:21:39.581 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:39.576+0000 7f8cae200640 1 --2- 172.21.15.12:0/1962199764 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558aae4e48a0 0x558aae4e2290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:21:39.581 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:39.577+0000 7f8cad800640 1 --2- 172.21.15.12:0/1962199764 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558aae4d7ff0 0x558aae4e1810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:21:39.581 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:39.577+0000 7f8cace00640 1 -- 172.21.15.12:0/1962199764 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558aae4d7ff0 msgr2=0x558aae4e1810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:21:39.581 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:39.577+0000 7f8cace00640 1 --2- 172.21.15.12:0/1962199764 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558aae4d7ff0 0x558aae4e1810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:21:39.581 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:39.577+0000 7f8cace00640 1 -- 172.21.15.12:0/1962199764 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558aae4e48a0 msgr2=0x558aae4e2290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:21:39.581 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:39.577+0000 7f8cace00640 1 --2- 172.21.15.12:0/1962199764 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558aae4e48a0 0x558aae4e2290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:21:39.581 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:39.577+0000 7f8cace00640 1 -- 172.21.15.12:0/1962199764 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558aae19cb00 con 0x558aae4d8910 2024-09-11T00:21:39.581 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:39.577+0000 7f8cae200640 1 --2- 172.21.15.12:0/1962199764 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558aae4e48a0 0x558aae4e2290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:21:39.581 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:39.577+0000 7f8cace00640 1 --2- 172.21.15.12:0/1962199764 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558aae4d8910 0x558aae4e1d50 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f8c9400bf00 tx=0x7f8c94011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a64efcac7e28e68 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:21:39.581 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:39.577+0000 7f8c8f400640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-09-11T00:21:39.581 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:39.577+0000 7f8c8f400640 1 -- 172.21.15.12:0/1962199764 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c94004020 con 0x558aae4d8910 2024-09-11T00:21:39.582 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:39.578+0000 7f8c8f400640 1 -- 172.21.15.12:0/1962199764 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9208+0+0 (secure 0 0 0) 0x7f8c940041c0 con 0x558aae4d8910 2024-09-11T00:21:39.582 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:39.578+0000 7f8cb2cd5740 1 -- 172.21.15.12:0/1962199764 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8c9c114030 con 0x558aae4d8910 2024-09-11T00:21:39.582 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:39.578+0000 7f8c8f400640 1 -- 172.21.15.12:0/1962199764 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7f8c940043c0 con 0x558aae4d8910 2024-09-11T00:21:39.582 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:39.578+0000 7f8cb2cd5740 10 client.17115 fetch_fsmap learned FSMap version 54 2024-09-11T00:21:39.582 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:39.578+0000 7f8cb2cd5740 1 -- 172.21.15.12:0/1962199764 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) -- 0x558aae213960 con 0x558aae4d8910 2024-09-11T00:21:39.582 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:39.578+0000 7f8c8f400640 1 -- 172.21.15.12:0/1962199764 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8c9c114030 con 0x558aae4d8910 2024-09-11T00:21:39.582 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:39.578+0000 7f8c8f400640 1 -- 172.21.15.12:0/1962199764 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 54) ==== 8422+0+0 (secure 0 0 0) 0x7f8c940039c0 con 0x558aae4d8910 2024-09-11T00:21:39.582 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:39.578+0000 7f8cb2cd5740 10 client.17115 fetch_fsmap finished waiting for FSMap version 54 2024-09-11T00:21:39.582 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:39.578+0000 7f8cb2cd5740 10 client.17115 resolve_mds: validated gid 24674 aka daemon mds.b 2024-09-11T00:21:39.582 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:39.578+0000 7f8cb2cd5740 20 client.17115 populate_metadata read hostname 'smithi012' 2024-09-11T00:21:39.582 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:39.579+0000 7f8cb2cd5740 1 --2- 172.21.15.12:0/1962199764 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x558aae4f4010 0x558aae4f6480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:21:39.582 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:39.579+0000 7f8cb2cd5740 4 client.17115 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T00:21:39.582 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:39.579+0000 7f8cb2cd5740 1 -- 172.21.15.12:0/1962199764 --> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558aae4511e0 con 0x558aae4f4010 2024-09-11T00:21:39.584 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:39.579+0000 7f8cad800640 1 --2- 172.21.15.12:0/1962199764 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x558aae4f4010 0x558aae4f6480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:21:39.584 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:39.579+0000 7f8cad800640 1 --2- 172.21.15.12:0/1962199764 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x558aae4f4010 0x558aae4f6480 crc :-1 s=READY pgs=213 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=59fde81ff12edc28 server_cookie=d06c04a1859b6176 in_seq=0 out_seq=0 2024-09-11T00:21:39.584 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:39.579+0000 7f8c8f400640 10 client.17115 ms_handle_connect on v2:172.21.15.165:6838/501312984 2024-09-11T00:21:39.584 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:39.580+0000 7f8c8f400640 1 -- 172.21.15.12:0/1962199764 <== mds.0 v2:172.21.15.165:6838/501312984 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x558aae4511e0 con 0x558aae4f4010 2024-09-11T00:21:39.584 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:39.580+0000 7f8c8f400640 10 client.17115 handle_command_reply: tid=0 2024-09-11T00:21:39.584 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:39.580+0000 7f8cb2cd5740 1 client.17115 shutdown 2024-09-11T00:21:39.584 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:39.581+0000 7f8cb2cd5740 1 -- 172.21.15.12:0/1962199764 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558aae4d8910 msgr2=0x558aae4e1d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:21:39.584 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:39.581+0000 7f8cb2cd5740 1 --2- 172.21.15.12:0/1962199764 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558aae4d8910 0x558aae4e1d50 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f8c9400bf00 tx=0x7f8c94011810 comp rx=0 tx=0).stop 2024-09-11T00:21:39.584 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:39.581+0000 7f8cb2cd5740 1 -- 172.21.15.12:0/1962199764 shutdown_connections 2024-09-11T00:21:39.584 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:39.581+0000 7f8cb2cd5740 1 --2- 172.21.15.12:0/1962199764 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x558aae4f4010 0x558aae4f6480 crc :-1 s=READY pgs=213 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:21:39.584 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:39.581+0000 7f8cb2cd5740 1 --2- 172.21.15.12:0/1962199764 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558aae4e48a0 0x558aae4e2290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:21:39.585 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:39.581+0000 7f8cb2cd5740 1 --2- 172.21.15.12:0/1962199764 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558aae4d8910 0x558aae4e1d50 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:21:39.585 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:39.581+0000 7f8cb2cd5740 1 --2- 172.21.15.12:0/1962199764 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558aae4d7ff0 0x558aae4e1810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:21:39.585 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:39.581+0000 7f8cb2cd5740 1 -- 172.21.15.12:0/1962199764 >> 172.21.15.12:0/1962199764 conn(0x558aae4d5e20 msgr2=0x558aae4db9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:21:39.585 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:39.581+0000 7f8c8f400640 0 client.17115 ms_handle_reset on v2:172.21.15.165:6838/501312984 2024-09-11T00:21:39.585 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:39.581+0000 7f8cb2cd5740 1 -- 172.21.15.12:0/1962199764 shutdown_connections 2024-09-11T00:21:39.585 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:39.581+0000 7f8cb2cd5740 1 -- 172.21.15.12:0/1962199764 wait complete. 2024-09-11T00:21:39.585 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:39.581+0000 7f8cb2cd5740 20 client.17115 trim_cache size 0 max 16384 2024-09-11T00:21:39.601 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:39.599+0000 7f8cb2cd5740 1 Processor -- start 2024-09-11T00:21:39.602 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:39.599+0000 7f8cb2cd5740 1 -- start start 2024-09-11T00:21:39.602 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:39.599+0000 7f8cb2cd5740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558aae4d7ff0 0x558aae4d4ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:21:39.602 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:39.599+0000 7f8cb2cd5740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558aae4d8910 0x558aae4dea60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:21:39.602 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:39.599+0000 7f8cb2cd5740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558aae4e48a0 0x558aae4defa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:21:39.602 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:39.599+0000 7f8cb2cd5740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f8ca81b1e20 con 0x558aae4d7ff0 2024-09-11T00:21:39.602 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:39.599+0000 7f8cb2cd5740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f8ca81b3b40 con 0x558aae4d8910 2024-09-11T00:21:39.602 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:39.599+0000 7f8cb2cd5740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f8ca81b2550 con 0x558aae4e48a0 2024-09-11T00:21:39.602 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:39.600+0000 7f8cad800640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558aae4d7ff0 0x558aae4d4ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:21:39.602 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:39.600+0000 7f8cad800640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558aae4d7ff0 0x558aae4d4ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:53288/0 (socket says 172.21.15.12:53288) 2024-09-11T00:21:39.602 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:39.600+0000 7f8cad800640 1 -- 172.21.15.12:0/1190045952 learned_addr learned my addr 172.21.15.12:0/1190045952 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:21:39.602 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:39.600+0000 7f8cae200640 1 --2- 172.21.15.12:0/1190045952 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558aae4e48a0 0x558aae4defa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:21:39.602 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:39.600+0000 7f8cace00640 1 --2- 172.21.15.12:0/1190045952 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558aae4d8910 0x558aae4dea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:21:39.603 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:39.600+0000 7f8cad800640 1 -- 172.21.15.12:0/1190045952 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558aae4e48a0 msgr2=0x558aae4defa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:21:39.603 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:39.600+0000 7f8cad800640 1 --2- 172.21.15.12:0/1190045952 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558aae4e48a0 0x558aae4defa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:21:39.603 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:39.600+0000 7f8cad800640 1 -- 172.21.15.12:0/1190045952 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558aae4d8910 msgr2=0x558aae4dea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:21:39.603 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:39.600+0000 7f8cad800640 1 --2- 172.21.15.12:0/1190045952 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558aae4d8910 0x558aae4dea60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:21:39.603 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:39.600+0000 7f8cad800640 1 -- 172.21.15.12:0/1190045952 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558aae4d8740 con 0x558aae4d7ff0 2024-09-11T00:21:39.603 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:39.600+0000 7f8cae200640 1 --2- 172.21.15.12:0/1190045952 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558aae4e48a0 0x558aae4defa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T00:21:39.603 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:39.600+0000 7f8cad800640 1 --2- 172.21.15.12:0/1190045952 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558aae4d7ff0 0x558aae4d4ca0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f8c9c0c9b90 tx=0x7f8c9c1077f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=869722f6be96945a server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:21:39.605 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:39.601+0000 7f8c9a000640 1 -- 172.21.15.12:0/1190045952 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c9c0c9c10 con 0x558aae4d7ff0 2024-09-11T00:21:39.605 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:39.601+0000 7f8c9a000640 1 -- 172.21.15.12:0/1190045952 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8c9c114070 con 0x558aae4d7ff0 2024-09-11T00:21:39.605 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:39.601+0000 7f8c9a000640 1 -- 172.21.15.12:0/1190045952 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c9c0ccd30 con 0x558aae4d7ff0 2024-09-11T00:21:39.605 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:39.601+0000 7f8cb2cd5740 1 -- 172.21.15.12:0/1190045952 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558aae4d7ff0 msgr2=0x558aae4d4ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:21:39.606 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:39.601+0000 7f8cb2cd5740 1 --2- 172.21.15.12:0/1190045952 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558aae4d7ff0 0x558aae4d4ca0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f8c9c0c9b90 tx=0x7f8c9c1077f0 comp rx=0 tx=0).stop 2024-09-11T00:21:39.606 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:39.601+0000 7f8cb2cd5740 1 -- 172.21.15.12:0/1190045952 shutdown_connections 2024-09-11T00:21:39.606 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:39.601+0000 7f8cb2cd5740 1 --2- 172.21.15.12:0/1190045952 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558aae4e48a0 0x558aae4defa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:21:39.606 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:39.601+0000 7f8cb2cd5740 1 --2- 172.21.15.12:0/1190045952 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558aae4d8910 0x558aae4dea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:21:39.606 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:39.601+0000 7f8cb2cd5740 1 --2- 172.21.15.12:0/1190045952 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558aae4d7ff0 0x558aae4d4ca0 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:21:39.606 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:39.601+0000 7f8cb2cd5740 1 -- 172.21.15.12:0/1190045952 >> 172.21.15.12:0/1190045952 conn(0x558aae4d5e20 msgr2=0x558aae4d4590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:21:39.606 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:39.602+0000 7f8cb2cd5740 1 -- 172.21.15.12:0/1190045952 shutdown_connections 2024-09-11T00:21:39.606 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:39.602+0000 7f8cb2cd5740 1 -- 172.21.15.12:0/1190045952 wait complete. 2024-09-11T00:21:39.606 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:39.602+0000 7f8cb2cd5740 1 -- start start 2024-09-11T00:21:39.606 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:39.602+0000 7f8cb2cd5740 1 Processor -- start 2024-09-11T00:21:39.606 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:39.603+0000 7f8cb2cd5740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558aae4d7ff0 0x558aae4ec950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:21:39.606 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:39.603+0000 7f8cb2cd5740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558aae4d8910 0x558aae4e93b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:21:39.606 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:39.603+0000 7f8cb2cd5740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558aae4e48a0 0x558aae4e98f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:21:39.607 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:39.603+0000 7f8cb2cd5740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f8ca81b3470 con 0x558aae4d8910 2024-09-11T00:21:39.607 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:39.603+0000 7f8cb2cd5740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f8ca81b2ce0 con 0x558aae4d7ff0 2024-09-11T00:21:39.607 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:39.603+0000 7f8cb2cd5740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f8ca81afe00 con 0x558aae4e48a0 2024-09-11T00:21:39.607 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:39.603+0000 7f8cb2cd5740 10 client.0 _ll_register_callbacks cb 0x7f8c9400bb10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:21:39.607 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:39.603+0000 7f8cace00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558aae4d8910 0x558aae4e93b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:21:39.607 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:39.603+0000 7f8cace00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558aae4d8910 0x558aae4e93b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:53298/0 (socket says 172.21.15.12:53298) 2024-09-11T00:21:39.607 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:39.603+0000 7f8cace00640 1 -- 172.21.15.12:0/975436714 learned_addr learned my addr 172.21.15.12:0/975436714 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:21:39.607 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:39.603+0000 7f8cad800640 1 --2- 172.21.15.12:0/975436714 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558aae4d7ff0 0x558aae4ec950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:21:39.607 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:39.603+0000 7f8cae200640 1 --2- 172.21.15.12:0/975436714 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558aae4e48a0 0x558aae4e98f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:21:39.607 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:39.603+0000 7f8cad800640 1 -- 172.21.15.12:0/975436714 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558aae4e48a0 msgr2=0x558aae4e98f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:21:39.607 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:39.603+0000 7f8cad800640 1 --2- 172.21.15.12:0/975436714 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558aae4e48a0 0x558aae4e98f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:21:39.607 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:39.603+0000 7f8cad800640 1 -- 172.21.15.12:0/975436714 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558aae4d8910 msgr2=0x558aae4e93b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:21:39.608 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:39.603+0000 7f8cad800640 1 --2- 172.21.15.12:0/975436714 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558aae4d8910 0x558aae4e93b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:21:39.608 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:39.603+0000 7f8cad800640 1 -- 172.21.15.12:0/975436714 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558aae4e9ee0 con 0x558aae4d7ff0 2024-09-11T00:21:39.608 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:39.603+0000 7f8cace00640 1 --2- 172.21.15.12:0/975436714 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558aae4d8910 0x558aae4e93b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:21:39.608 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:39.604+0000 7f8cad800640 1 --2- 172.21.15.12:0/975436714 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558aae4d7ff0 0x558aae4ec950 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f8c9c02ade0 tx=0x7f8c9c0ccd30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=21307c17678448fd server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:21:39.608 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:39.604+0000 7f8c8f400640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-09-11T00:21:39.608 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:39.604+0000 7f8c8f400640 1 -- 172.21.15.12:0/975436714 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c9c0ccf10 con 0x558aae4d7ff0 2024-09-11T00:21:39.608 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:39.604+0000 7f8cb2cd5740 1 -- 172.21.15.12:0/975436714 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8c9c114070 con 0x558aae4d7ff0 2024-09-11T00:21:39.608 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:39.604+0000 7f8c8f400640 1 -- 172.21.15.12:0/975436714 <== mon.2 v2:172.21.15.165:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9208+0+0 (secure 0 0 0) 0x7f8c9c1092e0 con 0x558aae4d7ff0 2024-09-11T00:21:39.608 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:39.604+0000 7f8c8f400640 1 -- 172.21.15.12:0/975436714 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7f8c9c0cd390 con 0x558aae4d7ff0 2024-09-11T00:21:39.608 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:39.604+0000 7f8cb2cd5740 10 client.27131 fetch_fsmap learned FSMap version 54 2024-09-11T00:21:39.608 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:39.604+0000 7f8cb2cd5740 1 -- 172.21.15.12:0/975436714 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) -- 0x558aae561de0 con 0x558aae4d7ff0 2024-09-11T00:21:39.609 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:39.605+0000 7f8c8f400640 1 -- 172.21.15.12:0/975436714 <== mon.2 v2:172.21.15.165:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8c9c114070 con 0x558aae4d7ff0 2024-09-11T00:21:39.609 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:39.605+0000 7f8c8f400640 1 -- 172.21.15.12:0/975436714 <== mon.2 v2:172.21.15.165:3300/0 5 ==== fsmap(e 54) ==== 8422+0+0 (secure 0 0 0) 0x7f8c9c104980 con 0x558aae4d7ff0 2024-09-11T00:21:39.609 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:39.605+0000 7f8cb2cd5740 10 client.27131 fetch_fsmap finished waiting for FSMap version 54 2024-09-11T00:21:39.609 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:39.605+0000 7f8cb2cd5740 10 client.27131 resolve_mds: validated gid 24674 aka daemon mds.b 2024-09-11T00:21:39.609 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:39.605+0000 7f8cb2cd5740 20 client.27131 populate_metadata read hostname 'smithi012' 2024-09-11T00:21:39.610 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:39.605+0000 7f8cb2cd5740 1 --2- 172.21.15.12:0/975436714 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x558aae562330 0x558aae564750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:21:39.610 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:39.605+0000 7f8cb2cd5740 4 client.27131 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "280397bf", "release": true, "await": true}] 2024-09-11T00:21:39.610 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:39.605+0000 7f8cb2cd5740 1 -- 172.21.15.12:0/975436714 --> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] -- command(tid 0: {"prefix": "quiesce db", "set_id": "280397bf", "release": true, "await": true}) -- 0x558aae44cc90 con 0x558aae562330 2024-09-11T00:21:39.610 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:39.605+0000 7f8cace00640 1 --2- 172.21.15.12:0/975436714 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x558aae562330 0x558aae564750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:21:39.610 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:39.606+0000 7f8cace00640 1 --2- 172.21.15.12:0/975436714 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x558aae562330 0x558aae564750 crc :-1 s=READY pgs=214 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=799b2045885ca992 server_cookie=5f31667ecbbc05f4 in_seq=0 out_seq=0 2024-09-11T00:21:39.610 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:39.606+0000 7f8c8f400640 10 client.27131 ms_handle_connect on v2:172.21.15.165:6838/501312984 2024-09-11T00:21:39.634 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:39.633+0000 7f8c8f400640 1 -- 172.21.15.12:0/975436714 <== mds.0 v2:172.21.15.165:6838/501312984 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x558aae44cc90 con 0x558aae562330 2024-09-11T00:21:39.635 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:39.633+0000 7f8c8f400640 10 client.27131 handle_command_reply: tid=0 2024-09-11T00:21:39.635 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:39.633+0000 7f8cb2cd5740 1 client.27131 shutdown 2024-09-11T00:21:39.635 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:39.633+0000 7f8cb2cd5740 1 -- 172.21.15.12:0/975436714 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558aae4d7ff0 msgr2=0x558aae4ec950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:21:39.635 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:39.633+0000 7f8cb2cd5740 1 --2- 172.21.15.12:0/975436714 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558aae4d7ff0 0x558aae4ec950 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f8c9c02ade0 tx=0x7f8c9c0ccd30 comp rx=0 tx=0).stop 2024-09-11T00:21:39.635 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:39.634+0000 7f8cb2cd5740 1 -- 172.21.15.12:0/975436714 shutdown_connections 2024-09-11T00:21:39.635 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:39.634+0000 7f8cb2cd5740 1 --2- 172.21.15.12:0/975436714 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x558aae562330 0x558aae564750 crc :-1 s=READY pgs=214 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:21:39.636 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:39.634+0000 7f8cb2cd5740 1 --2- 172.21.15.12:0/975436714 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558aae4e48a0 0x558aae4e98f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:21:39.636 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:39.634+0000 7f8cb2cd5740 1 --2- 172.21.15.12:0/975436714 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558aae4d8910 0x558aae4e93b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:21:39.636 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:39.634+0000 7f8cb2cd5740 1 --2- 172.21.15.12:0/975436714 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558aae4d7ff0 0x558aae4ec950 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:21:39.636 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:39.634+0000 7f8cb2cd5740 1 -- 172.21.15.12:0/975436714 >> 172.21.15.12:0/975436714 conn(0x558aae4d5e20 msgr2=0x558aae4d4db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:21:39.636 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:39.634+0000 7f8c8f400640 0 client.27131 ms_handle_reset on v2:172.21.15.165:6838/501312984 2024-09-11T00:21:39.636 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:39.634+0000 7f8cb2cd5740 1 -- 172.21.15.12:0/975436714 shutdown_connections 2024-09-11T00:21:39.636 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:39.635+0000 7f8cb2cd5740 1 -- 172.21.15.12:0/975436714 wait complete. 2024-09-11T00:21:39.637 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:39.635+0000 7f8cb2cd5740 20 client.27131 trim_cache size 0 max 16384 2024-09-11T00:21:39.637 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-09-11T00:21:39.638 INFO:teuthology.orchestra.run.smithi012.stdout: "epoch": 54, 2024-09-11T00:21:39.638 INFO:teuthology.orchestra.run.smithi012.stdout: "leader": 24674, 2024-09-11T00:21:39.638 INFO:teuthology.orchestra.run.smithi012.stdout: "set_version": 44, 2024-09-11T00:21:39.638 INFO:teuthology.orchestra.run.smithi012.stdout: "sets": { 2024-09-11T00:21:39.638 INFO:teuthology.orchestra.run.smithi012.stdout: "280397bf": { 2024-09-11T00:21:39.638 INFO:teuthology.orchestra.run.smithi012.stdout: "version": 44, 2024-09-11T00:21:39.638 INFO:teuthology.orchestra.run.smithi012.stdout: "age_ref": 0.0, 2024-09-11T00:21:39.638 INFO:teuthology.orchestra.run.smithi012.stdout: "state": { 2024-09-11T00:21:39.638 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "RELEASED", 2024-09-11T00:21:39.638 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 0.0 2024-09-11T00:21:39.638 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:21:39.639 INFO:teuthology.orchestra.run.smithi012.stdout: "timeout": 90.0, 2024-09-11T00:21:39.639 INFO:teuthology.orchestra.run.smithi012.stdout: "expiration": 142.8, 2024-09-11T00:21:39.639 INFO:teuthology.orchestra.run.smithi012.stdout: "members": { 2024-09-11T00:21:39.639 INFO:teuthology.orchestra.run.smithi012.stdout: "file:/": { 2024-09-11T00:21:39.639 INFO:teuthology.orchestra.run.smithi012.stdout: "excluded": false, 2024-09-11T00:21:39.639 INFO:teuthology.orchestra.run.smithi012.stdout: "state": { 2024-09-11T00:21:39.639 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "RELEASED", 2024-09-11T00:21:39.639 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 0.0 2024-09-11T00:21:39.639 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:21:39.639 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:21:39.639 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:21:39.639 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:21:39.639 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:21:39.640 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-09-11T00:21:39.641 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:39.638+0000 7f8cb0200640 1 -- 172.21.15.12:0/655032122 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f8c900b5360 msgr2=0x7f8c900b7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:21:39.641 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:39.638+0000 7f8cb0200640 1 --2- 172.21.15.12:0/655032122 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f8c900b5360 0x7f8c900b7820 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f8ca0004650 tx=0x7f8ca000b2e0 comp rx=0 tx=0).stop 2024-09-11T00:21:39.641 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:39.638+0000 7f8cb0200640 1 -- 172.21.15.12:0/655032122 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8ca8104f90 msgr2=0x7f8ca819b4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:21:39.641 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:39.638+0000 7f8cb0200640 1 --2- 172.21.15.12:0/655032122 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8ca8104f90 0x7f8ca819b4c0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f8c9c006010 tx=0x7f8c9c0319f0 comp rx=0 tx=0).stop 2024-09-11T00:21:39.641 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:39.638+0000 7f8cb0200640 1 -- 172.21.15.12:0/655032122 shutdown_connections 2024-09-11T00:21:39.641 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:39.638+0000 7f8cb0200640 1 --2- 172.21.15.12:0/655032122 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f8c900b5360 0x7f8c900b7820 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:21:39.641 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:39.638+0000 7f8cb0200640 1 --2- 172.21.15.12:0/655032122 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8ca810b320 0x7f8ca81a0da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:21:39.641 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:39.638+0000 7f8cb0200640 1 --2- 172.21.15.12:0/655032122 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8ca810a9a0 0x7f8ca819ba00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:21:39.641 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:39.638+0000 7f8cb0200640 1 --2- 172.21.15.12:0/655032122 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8ca8104f90 0x7f8ca819b4c0 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:21:39.641 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:39.638+0000 7f8cb0200640 1 -- 172.21.15.12:0/655032122 >> 172.21.15.12:0/655032122 conn(0x7f8ca8100a00 msgr2=0x7f8ca8101370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:21:39.641 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:39.638+0000 7f8cb0200640 1 -- 172.21.15.12:0/655032122 shutdown_connections 2024-09-11T00:21:39.642 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:39.639+0000 7f8cb0200640 1 -- 172.21.15.12:0/655032122 wait complete. 2024-09-11T00:21:40.152 INFO:tasks.workunit.client.0.smithi012.stdout: 10 28407 17.70 MB/sec warmup 96 sec latency 4868.547 ms 2024-09-11T00:21:40.314 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:21:40 smithi012 ceph-mon[30333]: pgmap v836: 129 pgs: 129 active+clean; 389 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 281 KiB/s wr, 80 op/s 2024-09-11T00:21:40.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:21:40 smithi178 ceph-mon[35919]: pgmap v836: 129 pgs: 129 active+clean; 389 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 281 KiB/s wr, 80 op/s 2024-09-11T00:21:40.532 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:21:40 smithi165 ceph-mon[35356]: pgmap v836: 129 pgs: 129 active+clean; 389 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 281 KiB/s wr, 80 op/s 2024-09-11T00:21:40.976 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '280397bf', total seconds elapsed: 50.9 2024-09-11T00:21:40.976 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 22.75 2024-09-11T00:21:40.976 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid ff3ad74a-6fcf-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T00:21:41.152 INFO:tasks.workunit.client.0.smithi012.stdout: 10 29532 18.16 MB/sec warmup 97 sec latency 47.105 ms 2024-09-11T00:21:41.643 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:21:42.062 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:21:42 smithi012 ceph-mon[30333]: pgmap v837: 129 pgs: 129 active+clean; 375 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 278 KiB/s wr, 121 op/s 2024-09-11T00:21:42.152 INFO:tasks.workunit.client.0.smithi012.stdout: 10 30504 18.60 MB/sec warmup 98 sec latency 35.050 ms 2024-09-11T00:21:42.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:21:42 smithi178 ceph-mon[35919]: pgmap v837: 129 pgs: 129 active+clean; 375 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 278 KiB/s wr, 121 op/s 2024-09-11T00:21:42.347 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:42.345+0000 7f4e86a00640 1 -- 172.21.15.12:0/3615269825 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4e8010a9b0 msgr2=0x7f4e8010adf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:21:42.348 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:42.345+0000 7f4e86a00640 1 --2- 172.21.15.12:0/3615269825 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4e8010a9b0 0x7f4e8010adf0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f4e6c009860 tx=0x7f4e6c02a340 comp rx=0 tx=0).stop 2024-09-11T00:21:42.348 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:42.346+0000 7f4e86a00640 1 -- 172.21.15.12:0/3615269825 shutdown_connections 2024-09-11T00:21:42.350 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:42.346+0000 7f4e86a00640 1 --2- 172.21.15.12:0/3615269825 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4e8010b330 0x7f4e80111b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:21:42.350 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:42.346+0000 7f4e86a00640 1 --2- 172.21.15.12:0/3615269825 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4e8010a9b0 0x7f4e8010adf0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:21:42.350 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:42.346+0000 7f4e86a00640 1 --2- 172.21.15.12:0/3615269825 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4e80104fa0 0x7f4e80105380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:21:42.350 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:42.346+0000 7f4e86a00640 1 -- 172.21.15.12:0/3615269825 >> 172.21.15.12:0/3615269825 conn(0x7f4e80100a50 msgr2=0x7f4e80102e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:21:42.350 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:42.349+0000 7f4e86a00640 1 -- 172.21.15.12:0/3615269825 shutdown_connections 2024-09-11T00:21:42.352 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:42.351+0000 7f4e86a00640 1 -- 172.21.15.12:0/3615269825 wait complete. 2024-09-11T00:21:42.353 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:42.352+0000 7f4e86a00640 1 Processor -- start 2024-09-11T00:21:42.355 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:42.353+0000 7f4e86a00640 1 -- start start 2024-09-11T00:21:42.355 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:42.354+0000 7f4e86a00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4e80104fa0 0x7f4e8019b460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:21:42.355 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:42.354+0000 7f4e86a00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4e8010a9b0 0x7f4e8019b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:21:42.356 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:42.354+0000 7f4e86a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4e8010b330 0x7f4e801a0d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:21:42.356 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:42.354+0000 7f4e86a00640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f4e80114f30 con 0x7f4e80104fa0 2024-09-11T00:21:42.356 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:42.354+0000 7f4e86a00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f4e80114db0 con 0x7f4e8010b330 2024-09-11T00:21:42.356 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:42.354+0000 7f4e86a00640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f4e801150b0 con 0x7f4e8010a9b0 2024-09-11T00:21:42.356 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:42.354+0000 7f4e84a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4e8010b330 0x7f4e801a0d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:21:42.356 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:42.354+0000 7f4e84a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4e8010b330 0x7f4e801a0d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.12:55582/0 (socket says 172.21.15.12:55582) 2024-09-11T00:21:42.356 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:42.354+0000 7f4e7fe00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4e80104fa0 0x7f4e8019b460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:21:42.356 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:42.354+0000 7f4e84a00640 1 -- 172.21.15.12:0/3975896831 learned_addr learned my addr 172.21.15.12:0/3975896831 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:21:42.356 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:42.354+0000 7f4e7f400640 1 --2- 172.21.15.12:0/3975896831 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4e8010a9b0 0x7f4e8019b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:21:42.357 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:42.355+0000 7f4e84a00640 1 -- 172.21.15.12:0/3975896831 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4e8010a9b0 msgr2=0x7f4e8019b9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:21:42.357 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:42.355+0000 7f4e84a00640 1 --2- 172.21.15.12:0/3975896831 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4e8010a9b0 0x7f4e8019b9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:21:42.357 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:42.355+0000 7f4e84a00640 1 -- 172.21.15.12:0/3975896831 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4e80104fa0 msgr2=0x7f4e8019b460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:21:42.357 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:42.355+0000 7f4e84a00640 1 --2- 172.21.15.12:0/3975896831 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4e80104fa0 0x7f4e8019b460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:21:42.357 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:42.355+0000 7f4e84a00640 1 -- 172.21.15.12:0/3975896831 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e801a1510 con 0x7f4e8010b330 2024-09-11T00:21:42.357 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:42.355+0000 7f4e84a00640 1 --2- 172.21.15.12:0/3975896831 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4e8010b330 0x7f4e801a0d40 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f4e7800eb10 tx=0x7f4e7800c2d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:21:42.357 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:42.356+0000 7f4e7cc00640 1 -- 172.21.15.12:0/3975896831 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e78004b20 con 0x7f4e8010b330 2024-09-11T00:21:42.358 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:42.356+0000 7f4e7cc00640 1 -- 172.21.15.12:0/3975896831 <== mon.2 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4e78004cc0 con 0x7f4e8010b330 2024-09-11T00:21:42.358 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:42.356+0000 7f4e7cc00640 1 -- 172.21.15.12:0/3975896831 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e78010670 con 0x7f4e8010b330 2024-09-11T00:21:42.358 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:42.356+0000 7f4e86a00640 1 -- 172.21.15.12:0/3975896831 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4e801a1800 con 0x7f4e8010b330 2024-09-11T00:21:42.359 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:42.356+0000 7f4e86a00640 1 -- 172.21.15.12:0/3975896831 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4e801a7ec0 con 0x7f4e8010b330 2024-09-11T00:21:42.360 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:42.358+0000 7f4e76000640 1 -- 172.21.15.12:0/3975896831 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4e58005290 con 0x7f4e8010b330 2024-09-11T00:21:42.366 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:42.364+0000 7f4e7cc00640 1 -- 172.21.15.12:0/3975896831 <== mon.2 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7f4e78005000 con 0x7f4e8010b330 2024-09-11T00:21:42.369 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:42.368+0000 7f4e7cc00640 1 --2- 172.21.15.12:0/3975896831 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f4e640b5430 0x7f4e640b78f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:21:42.369 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:42.368+0000 7f4e7cc00640 1 -- 172.21.15.12:0/3975896831 <== mon.2 v2:172.21.15.165:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9208+0+0 (secure 0 0 0) 0x7f4e780e65e0 con 0x7f4e8010b330 2024-09-11T00:21:42.370 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:42.369+0000 7f4e7fe00640 1 --2- 172.21.15.12:0/3975896831 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f4e640b5430 0x7f4e640b78f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:21:42.370 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:42.369+0000 7f4e7cc00640 1 -- 172.21.15.12:0/3975896831 <== mon.2 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f4e780a8cc0 con 0x7f4e8010b330 2024-09-11T00:21:42.373 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:42.372+0000 7f4e7fe00640 1 --2- 172.21.15.12:0/3975896831 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f4e640b5430 0x7f4e640b78f0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f4e700045f0 tx=0x7f4e7000b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:21:42.532 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:21:42 smithi165 ceph-mon[35356]: pgmap v837: 129 pgs: 129 active+clean; 375 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 278 KiB/s wr, 121 op/s 2024-09-11T00:21:42.848 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:42.847+0000 7f4e76000640 1 -- 172.21.15.12:0/3975896831 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4e58005580 con 0x7f4e8010b330 2024-09-11T00:21:42.850 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:42.848+0000 7f4e7cc00640 1 -- 172.21.15.12:0/3975896831 <== mon.2 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 54 v54) ==== 94+0+11986 (secure 0 0 0) 0x7f4e780ad150 con 0x7f4e8010b330 2024-09-11T00:21:42.851 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-11T00:21:42.851 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 54 2024-09-11T00:21:42.852 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":54,"btime":"2024-09-11T00:20:52:794956+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14577,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6835/447171591","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":447171591},{"type":"v1","addr":"172.21.15.12:6835","nonce":447171591}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14589,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/901486040","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":901486040},{"type":"v1","addr":"172.21.15.12:6837","nonce":901486040}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":54,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T00:04:04.739881+0000","modified":"2024-09-11T00:20:52.232549+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24674,"mds_1":24643,"mds_2":24638,"mds_3":24637,"mds_4":24614},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14631":{"gid":14631,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.12:6841/3069919437","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6840","nonce":3069919437},{"type":"v1","addr":"172.21.15.12:6841","nonce":3069919437}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24562":{"gid":24562,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6833/1141636112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":1141636112},{"type":"v1","addr":"172.21.15.178:6833","nonce":1141636112}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24581":{"gid":24581,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6833/3111980970","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6832","nonce":3111980970},{"type":"v1","addr":"172.21.15.165:6833","nonce":3111980970}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24607":{"gid":24607,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6835/4084868287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":4084868287},{"type":"v1","addr":"172.21.15.178:6835","nonce":4084868287}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24614":{"gid":24614,"name":"e","rank":4,"incarnation":26,"state":"up:active","state_seq":21,"addr":"172.21.15.165:6835/2425395274","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":2425395274},{"type":"v1","addr":"172.21.15.165:6835","nonce":2425395274}]},"join_fscid":-1,"export_targets":[0,1],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24637":{"gid":24637,"name":"i","rank":3,"incarnation":24,"state":"up:active","state_seq":18,"addr":"172.21.15.178:6839/70628097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6838","nonce":70628097},{"type":"v1","addr":"172.21.15.178:6839","nonce":70628097}]},"join_fscid":-1,"export_targets":[0,1],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24638":{"gid":24638,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":18,"addr":"172.21.15.165:6837/6215088","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6836","nonce":6215088},{"type":"v1","addr":"172.21.15.165:6837","nonce":6215088}]},"join_fscid":-1,"export_targets":[0],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24643":{"gid":24643,"name":"g","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.12:6839/3124093009","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6838","nonce":3124093009},{"type":"v1","addr":"172.21.15.12:6839","nonce":3124093009}]},"join_fscid":-1,"export_targets":[0,2,3,4],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24674":{"gid":24674,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.165:6839/501312984","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6838","nonce":501312984},{"type":"v1","addr":"172.21.15.165:6839","nonce":501312984}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25672":{"gid":25672,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6837/308601741","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":308601741},{"type":"v1","addr":"172.21.15.178:6837","nonce":308601741}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24674,"qdb_cluster":[24674,24643,24638,24637,24614]},"id":1}]} 2024-09-11T00:21:42.855 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:42.853+0000 7f4e76000640 1 -- 172.21.15.12:0/3975896831 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f4e640b5430 msgr2=0x7f4e640b78f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:21:42.855 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:42.853+0000 7f4e76000640 1 --2- 172.21.15.12:0/3975896831 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f4e640b5430 0x7f4e640b78f0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f4e700045f0 tx=0x7f4e7000b390 comp rx=0 tx=0).stop 2024-09-11T00:21:42.855 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:42.853+0000 7f4e76000640 1 -- 172.21.15.12:0/3975896831 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4e8010b330 msgr2=0x7f4e801a0d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:21:42.855 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:42.853+0000 7f4e76000640 1 --2- 172.21.15.12:0/3975896831 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4e8010b330 0x7f4e801a0d40 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f4e7800eb10 tx=0x7f4e7800c2d0 comp rx=0 tx=0).stop 2024-09-11T00:21:42.856 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:42.854+0000 7f4e76000640 1 -- 172.21.15.12:0/3975896831 shutdown_connections 2024-09-11T00:21:42.856 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:42.854+0000 7f4e76000640 1 --2- 172.21.15.12:0/3975896831 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f4e640b5430 0x7f4e640b78f0 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:21:42.856 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:42.854+0000 7f4e76000640 1 --2- 172.21.15.12:0/3975896831 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4e8010b330 0x7f4e801a0d40 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:21:42.856 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:42.854+0000 7f4e76000640 1 --2- 172.21.15.12:0/3975896831 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4e8010a9b0 0x7f4e8019b9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:21:42.856 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:42.854+0000 7f4e76000640 1 --2- 172.21.15.12:0/3975896831 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4e80104fa0 0x7f4e8019b460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:21:42.856 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:42.854+0000 7f4e76000640 1 -- 172.21.15.12:0/3975896831 >> 172.21.15.12:0/3975896831 conn(0x7f4e80100a50 msgr2=0x7f4e80102e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:21:42.856 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:42.854+0000 7f4e76000640 1 -- 172.21.15.12:0/3975896831 shutdown_connections 2024-09-11T00:21:42.856 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:42.854+0000 7f4e76000640 1 -- 172.21.15.12:0/3975896831 wait complete. 2024-09-11T00:21:43.152 INFO:tasks.workunit.client.0.smithi012.stdout: 10 31122 18.72 MB/sec warmup 99 sec latency 57.222 ms 2024-09-11T00:21:43.218 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:21:43 smithi012 ceph-mon[30333]: from='client.? 172.21.15.12:0/3975896831' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T00:21:43.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:21:43 smithi178 ceph-mon[35919]: from='client.? 172.21.15.12:0/3975896831' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T00:21:43.532 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:21:43 smithi165 ceph-mon[35356]: from='client.? 172.21.15.12:0/3975896831' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T00:21:43.670 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24674 quiesce db / --timeout 90 --expiration 142.75 --await-for 45.0' 2024-09-11T00:21:43.670 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid ff3ad74a-6fcf-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24674 quiesce db / --timeout 90 --expiration 142.75 --await-for 45.0 2024-09-11T00:21:44.152 INFO:tasks.workunit.client.0.smithi012.stdout: 10 32095 18.94 MB/sec warmup 100 sec latency 35.125 ms 2024-09-11T00:21:44.319 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:21:44.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:21:44 smithi178 ceph-mon[35919]: pgmap v838: 129 pgs: 129 active+clean; 376 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 456 KiB/s wr, 168 op/s 2024-09-11T00:21:44.433 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:21:44 smithi165 ceph-mon[35356]: pgmap v838: 129 pgs: 129 active+clean; 376 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 456 KiB/s wr, 168 op/s 2024-09-11T00:21:44.462 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:21:44 smithi012 ceph-mon[30333]: pgmap v838: 129 pgs: 129 active+clean; 376 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 456 KiB/s wr, 168 op/s 2024-09-11T00:21:45.083 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:45.081+0000 7ff76ca00640 1 -- 172.21.15.12:0/1791894724 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff768106d40 msgr2=0x7ff768107190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:21:45.084 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:45.081+0000 7ff76ca00640 1 --2- 172.21.15.12:0/1791894724 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff768106d40 0x7ff768107190 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7ff760009900 tx=0x7ff76002a3f0 comp rx=0 tx=0).stop 2024-09-11T00:21:45.085 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:45.082+0000 7ff76ca00640 1 -- 172.21.15.12:0/1791894724 shutdown_connections 2024-09-11T00:21:45.085 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:45.082+0000 7ff76ca00640 1 --2- 172.21.15.12:0/1791894724 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff768106d40 0x7ff768107190 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:21:45.085 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:45.082+0000 7ff76ca00640 1 --2- 172.21.15.12:0/1791894724 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff768102220 0x7ff7681066f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:21:45.085 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:45.082+0000 7ff76ca00640 1 --2- 172.21.15.12:0/1791894724 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff768101900 0x7ff768101ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:21:45.085 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:45.082+0000 7ff76ca00640 1 -- 172.21.15.12:0/1791894724 >> 172.21.15.12:0/1791894724 conn(0x7ff7680fd5b0 msgr2=0x7ff7680ff9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:21:45.085 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:45.083+0000 7ff76ca00640 1 -- 172.21.15.12:0/1791894724 shutdown_connections 2024-09-11T00:21:45.085 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:45.083+0000 7ff76ca00640 1 -- 172.21.15.12:0/1791894724 wait complete. 2024-09-11T00:21:45.087 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:45.086+0000 7ff76ca00640 1 Processor -- start 2024-09-11T00:21:45.088 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:45.087+0000 7ff76ca00640 1 -- start start 2024-09-11T00:21:45.088 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:45.087+0000 7ff76ca00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff768101900 0x7ff76819f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:21:45.090 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:45.087+0000 7ff76ca00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff768102220 0x7ff76819fb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:21:45.090 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:45.087+0000 7ff76ca00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff768106d40 0x7ff7681a4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:21:45.090 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:45.087+0000 7ff76ca00640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7ff7681191c0 con 0x7ff768102220 2024-09-11T00:21:45.090 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:45.087+0000 7ff76ca00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7ff768119040 con 0x7ff768101900 2024-09-11T00:21:45.090 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:45.087+0000 7ff76ca00640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7ff768119340 con 0x7ff768106d40 2024-09-11T00:21:45.090 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:45.088+0000 7ff767200640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff768106d40 0x7ff7681a4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:21:45.090 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:45.088+0000 7ff767200640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff768106d40 0x7ff7681a4eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.12:50776/0 (socket says 172.21.15.12:50776) 2024-09-11T00:21:45.090 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:45.088+0000 7ff767200640 1 -- 172.21.15.12:0/1051351159 learned_addr learned my addr 172.21.15.12:0/1051351159 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:21:45.090 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:45.088+0000 7ff766800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff768101900 0x7ff76819f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:21:45.090 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:45.088+0000 7ff765e00640 1 --2- 172.21.15.12:0/1051351159 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff768102220 0x7ff76819fb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:21:45.090 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:45.088+0000 7ff767200640 1 -- 172.21.15.12:0/1051351159 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff768101900 msgr2=0x7ff76819f5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:21:45.090 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:45.088+0000 7ff767200640 1 --2- 172.21.15.12:0/1051351159 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff768101900 0x7ff76819f5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:21:45.091 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:45.088+0000 7ff767200640 1 -- 172.21.15.12:0/1051351159 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff768102220 msgr2=0x7ff76819fb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:21:45.091 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:45.088+0000 7ff767200640 1 --2- 172.21.15.12:0/1051351159 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff768102220 0x7ff76819fb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:21:45.091 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:45.088+0000 7ff767200640 1 -- 172.21.15.12:0/1051351159 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7681a5680 con 0x7ff768106d40 2024-09-11T00:21:45.091 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:45.089+0000 7ff767200640 1 --2- 172.21.15.12:0/1051351159 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff768106d40 0x7ff7681a4eb0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7ff76002a960 tx=0x7ff760003d60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:21:45.091 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:45.089+0000 7ff75f400640 1 -- 172.21.15.12:0/1051351159 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff76003f070 con 0x7ff768106d40 2024-09-11T00:21:45.091 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:45.089+0000 7ff75f400640 1 -- 172.21.15.12:0/1051351159 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff760031d60 con 0x7ff768106d40 2024-09-11T00:21:45.091 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:45.089+0000 7ff75f400640 1 -- 172.21.15.12:0/1051351159 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff760035030 con 0x7ff768106d40 2024-09-11T00:21:45.092 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:45.089+0000 7ff76ca00640 1 -- 172.21.15.12:0/1051351159 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff7681a5910 con 0x7ff768106d40 2024-09-11T00:21:45.093 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:45.090+0000 7ff76ca00640 1 -- 172.21.15.12:0/1051351159 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff76806cb70 con 0x7ff768106d40 2024-09-11T00:21:45.098 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:45.097+0000 7ff76fe0c740 1 Processor -- start 2024-09-11T00:21:45.099 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:45.097+0000 7ff75f400640 1 -- 172.21.15.12:0/1051351159 <== mon.1 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7ff76003b4f0 con 0x7ff768106d40 2024-09-11T00:21:45.099 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:45.097+0000 7ff76fe0c740 1 -- start start 2024-09-11T00:21:45.099 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:45.098+0000 7ff76fe0c740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5590e62d0e10 0x5590e62cf370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:21:45.101 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:45.098+0000 7ff76fe0c740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5590e62cf8b0 0x5590e62cfcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:21:45.101 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:45.098+0000 7ff76fe0c740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5590e62db6e0 0x5590e62ddad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:21:45.102 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:45.098+0000 7ff76fe0c740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x5590e6244a90 con 0x5590e62d0e10 2024-09-11T00:21:45.102 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:45.098+0000 7ff76fe0c740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x5590e6233ec0 con 0x5590e62db6e0 2024-09-11T00:21:45.102 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:45.098+0000 7ff76fe0c740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5590e6237cd0 con 0x5590e62cf8b0 2024-09-11T00:21:45.102 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:45.100+0000 7ff766800640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5590e62cf8b0 0x5590e62cfcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:21:45.102 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:45.100+0000 7ff766800640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5590e62cf8b0 0x5590e62cfcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.12:50790/0 (socket says 172.21.15.12:50790) 2024-09-11T00:21:45.102 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:45.099+0000 7ff765e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5590e62db6e0 0x5590e62ddad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:21:45.102 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:45.100+0000 7ff765e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5590e62db6e0 0x5590e62ddad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.12:55604/0 (socket says 172.21.15.12:55604) 2024-09-11T00:21:45.102 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:45.100+0000 7ff765e00640 1 -- 172.21.15.12:0/768739179 learned_addr learned my addr 172.21.15.12:0/768739179 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:21:45.102 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:45.100+0000 7ff765e00640 1 --2- 172.21.15.12:0/768739179 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5590e62d0e10 0x5590e62cf370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:21:45.104 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:45.101+0000 7ff765e00640 1 -- 172.21.15.12:0/768739179 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5590e62cf8b0 msgr2=0x5590e62cfcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:21:45.104 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:45.101+0000 7ff765e00640 1 --2- 172.21.15.12:0/768739179 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5590e62cf8b0 0x5590e62cfcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:21:45.104 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:45.101+0000 7ff765e00640 1 -- 172.21.15.12:0/768739179 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5590e62d0e10 msgr2=0x5590e62cf370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:21:45.104 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:45.101+0000 7ff765e00640 1 --2- 172.21.15.12:0/768739179 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5590e62d0e10 0x5590e62cf370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:21:45.104 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:45.101+0000 7ff765e00640 1 -- 172.21.15.12:0/768739179 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5590e61d99d0 con 0x5590e62db6e0 2024-09-11T00:21:45.105 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:45.102+0000 7ff766800640 1 --2- 172.21.15.12:0/768739179 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5590e62cf8b0 0x5590e62cfcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T00:21:45.105 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:45.102+0000 7ff75f400640 1 --2- 172.21.15.12:0/1051351159 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7ff7480b54e0 0x7ff7480b79a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:21:45.105 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:45.102+0000 7ff75f400640 1 -- 172.21.15.12:0/1051351159 <== mon.1 v2:172.21.15.178:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9208+0+0 (secure 0 0 0) 0x7ff760106c30 con 0x7ff768106d40 2024-09-11T00:21:45.105 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:45.103+0000 7ff765e00640 1 --2- 172.21.15.12:0/768739179 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5590e62db6e0 0x5590e62ddad0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7ff750024b60 tx=0x7ff75001b700 comp rx=0 tx=0).ready entity=mon.2 client_cookie=350035f3edac2e32 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:21:45.105 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:45.103+0000 7ff766800640 1 --2- 172.21.15.12:0/1051351159 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7ff7480b54e0 0x7ff7480b79a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:21:45.105 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:45.104+0000 7ff757400640 1 -- 172.21.15.12:0/768739179 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7500242d0 con 0x5590e62db6e0 2024-09-11T00:21:45.106 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:45.104+0000 7ff757400640 1 -- 172.21.15.12:0/768739179 <== mon.2 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff750003e20 con 0x5590e62db6e0 2024-09-11T00:21:45.106 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:45.104+0000 7ff757400640 1 -- 172.21.15.12:0/768739179 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff750004490 con 0x5590e62db6e0 2024-09-11T00:21:45.106 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:45.105+0000 7ff76fe0c740 1 -- 172.21.15.12:0/768739179 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5590e62db6e0 msgr2=0x5590e62ddad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:21:45.106 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:45.105+0000 7ff76fe0c740 1 --2- 172.21.15.12:0/768739179 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5590e62db6e0 0x5590e62ddad0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7ff750024b60 tx=0x7ff75001b700 comp rx=0 tx=0).stop 2024-09-11T00:21:45.107 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:45.106+0000 7ff766800640 1 --2- 172.21.15.12:0/1051351159 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7ff7480b54e0 0x7ff7480b79a0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x5590e62d24f0 tx=0x7ff75800a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:21:45.107 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:45.106+0000 7ff76fe0c740 1 -- 172.21.15.12:0/768739179 shutdown_connections 2024-09-11T00:21:45.107 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:45.106+0000 7ff76fe0c740 1 --2- 172.21.15.12:0/768739179 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5590e62db6e0 0x5590e62ddad0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:21:45.107 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:45.106+0000 7ff76fe0c740 1 --2- 172.21.15.12:0/768739179 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5590e62cf8b0 0x5590e62cfcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:21:45.107 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:45.106+0000 7ff76fe0c740 1 --2- 172.21.15.12:0/768739179 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5590e62d0e10 0x5590e62cf370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:21:45.107 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:45.106+0000 7ff76fe0c740 1 -- 172.21.15.12:0/768739179 >> 172.21.15.12:0/768739179 conn(0x5590e62996f0 msgr2=0x5590e62cd1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:21:45.108 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:45.106+0000 7ff76fe0c740 1 -- 172.21.15.12:0/768739179 shutdown_connections 2024-09-11T00:21:45.108 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:45.106+0000 7ff76fe0c740 1 -- 172.21.15.12:0/768739179 wait complete. 2024-09-11T00:21:45.108 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:45.107+0000 7ff76fe0c740 1 -- start start 2024-09-11T00:21:45.109 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:45.107+0000 7ff76fe0c740 1 Processor -- start 2024-09-11T00:21:45.110 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:45.108+0000 7ff76fe0c740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5590e62cf8b0 0x5590e62d86d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:21:45.110 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:45.108+0000 7ff76fe0c740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5590e62d0e10 0x5590e62d8c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:21:45.110 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:45.108+0000 7ff76fe0c740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5590e62db6e0 0x5590e62d9150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:21:45.110 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:45.108+0000 7ff76fe0c740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x5590e60957b0 con 0x5590e62cf8b0 2024-09-11T00:21:45.110 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:45.108+0000 7ff76fe0c740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x5590e61d8650 con 0x5590e62d0e10 2024-09-11T00:21:45.110 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:45.108+0000 7ff76fe0c740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5590e62de4a0 con 0x5590e62db6e0 2024-09-11T00:21:45.111 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:45.108+0000 7ff76fe0c740 10 client.0 _ll_register_callbacks cb 0x5590e5f3f0a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:21:45.111 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:45.108+0000 7ff766800640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5590e62cf8b0 0x5590e62d86d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:21:45.111 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:45.108+0000 7ff766800640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5590e62cf8b0 0x5590e62d86d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:53362/0 (socket says 172.21.15.12:53362) 2024-09-11T00:21:45.111 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:45.108+0000 7ff766800640 1 -- 172.21.15.12:0/2693403443 learned_addr learned my addr 172.21.15.12:0/2693403443 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:21:45.111 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:45.108+0000 7ff767200640 1 --2- 172.21.15.12:0/2693403443 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5590e62db6e0 0x5590e62d9150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:21:45.111 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:45.109+0000 7ff767200640 1 -- 172.21.15.12:0/2693403443 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5590e62d0e10 msgr2=0x5590e62d8c10 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-11T00:21:45.111 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:45.109+0000 7ff767200640 1 --2- 172.21.15.12:0/2693403443 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5590e62d0e10 0x5590e62d8c10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:21:45.111 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:45.109+0000 7ff767200640 1 -- 172.21.15.12:0/2693403443 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5590e62cf8b0 msgr2=0x5590e62d86d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:21:45.111 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:45.109+0000 7ff767200640 1 --2- 172.21.15.12:0/2693403443 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5590e62cf8b0 0x5590e62d86d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:21:45.112 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:45.109+0000 7ff767200640 1 -- 172.21.15.12:0/2693403443 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5590e5fa1ba0 con 0x5590e62db6e0 2024-09-11T00:21:45.112 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:45.109+0000 7ff766800640 1 --2- 172.21.15.12:0/2693403443 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5590e62cf8b0 0x5590e62d86d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:21:45.113 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:45.110+0000 7ff74fe00640 10 client.0 ms_handle_connect on v2:172.21.15.178:3300/0 2024-09-11T00:21:45.113 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:45.110+0000 7ff767200640 1 --2- 172.21.15.12:0/2693403443 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5590e62db6e0 0x5590e62d9150 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7ff760104300 tx=0x7ff7600ccd90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=977f008147fad0e0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:21:45.113 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:45.110+0000 7ff74fe00640 1 -- 172.21.15.12:0/2693403443 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff760049070 con 0x5590e62db6e0 2024-09-11T00:21:45.113 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:45.111+0000 7ff74fe00640 1 -- 172.21.15.12:0/2693403443 <== mon.1 v2:172.21.15.178:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9208+0+0 (secure 0 0 0) 0x7ff7600d62c0 con 0x5590e62db6e0 2024-09-11T00:21:45.113 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:45.111+0000 7ff76fe0c740 1 -- 172.21.15.12:0/2693403443 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff750003e20 con 0x5590e62db6e0 2024-09-11T00:21:45.113 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:45.111+0000 7ff74fe00640 1 -- 172.21.15.12:0/2693403443 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7ff7600ccad0 con 0x5590e62db6e0 2024-09-11T00:21:45.113 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:45.111+0000 7ff76fe0c740 10 client.27112 fetch_fsmap learned FSMap version 54 2024-09-11T00:21:45.113 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:45.111+0000 7ff76fe0c740 1 -- 172.21.15.12:0/2693403443 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) -- 0x5590e62e9ff0 con 0x5590e62db6e0 2024-09-11T00:21:45.113 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:45.112+0000 7ff74fe00640 1 -- 172.21.15.12:0/2693403443 <== mon.1 v2:172.21.15.178:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff750003e20 con 0x5590e62db6e0 2024-09-11T00:21:45.114 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:45.113+0000 7ff74fe00640 1 -- 172.21.15.12:0/2693403443 <== mon.1 v2:172.21.15.178:3300/0 5 ==== fsmap(e 54) ==== 8422+0+0 (secure 0 0 0) 0x7ff7600cf030 con 0x5590e62db6e0 2024-09-11T00:21:45.114 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:45.113+0000 7ff76fe0c740 10 client.27112 fetch_fsmap finished waiting for FSMap version 54 2024-09-11T00:21:45.114 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:45.113+0000 7ff76fe0c740 10 client.27112 resolve_mds: validated gid 24674 aka daemon mds.b 2024-09-11T00:21:45.114 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:45.113+0000 7ff76fe0c740 20 client.27112 populate_metadata read hostname 'smithi012' 2024-09-11T00:21:45.115 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:45.113+0000 7ff76fe0c740 1 --2- 172.21.15.12:0/2693403443 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x5590e62eb4a0 0x5590e62ed940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:21:45.115 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:45.113+0000 7ff76fe0c740 4 client.27112 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T00:21:45.115 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:45.113+0000 7ff76fe0c740 1 -- 172.21.15.12:0/2693403443 --> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5590e62469e0 con 0x5590e62eb4a0 2024-09-11T00:21:45.115 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:45.113+0000 7ff766800640 1 --2- 172.21.15.12:0/2693403443 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x5590e62eb4a0 0x5590e62ed940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:21:45.115 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:45.114+0000 7ff766800640 1 --2- 172.21.15.12:0/2693403443 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x5590e62eb4a0 0x5590e62ed940 crc :-1 s=READY pgs=215 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7078bb0f4aa0bc98 server_cookie=8fe33561dc00f377 in_seq=0 out_seq=0 2024-09-11T00:21:45.116 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:45.114+0000 7ff74fe00640 10 client.27112 ms_handle_connect on v2:172.21.15.165:6838/501312984 2024-09-11T00:21:45.116 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:45.115+0000 7ff74fe00640 1 -- 172.21.15.12:0/2693403443 <== mds.0 v2:172.21.15.165:6838/501312984 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5590e62469e0 con 0x5590e62eb4a0 2024-09-11T00:21:45.117 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:45.115+0000 7ff74fe00640 10 client.27112 handle_command_reply: tid=0 2024-09-11T00:21:45.117 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:45.116+0000 7ff76fe0c740 1 client.27112 shutdown 2024-09-11T00:21:45.117 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:45.116+0000 7ff76fe0c740 1 -- 172.21.15.12:0/2693403443 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5590e62db6e0 msgr2=0x5590e62d9150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:21:45.118 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:45.116+0000 7ff76fe0c740 1 --2- 172.21.15.12:0/2693403443 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5590e62db6e0 0x5590e62d9150 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7ff760104300 tx=0x7ff7600ccd90 comp rx=0 tx=0).stop 2024-09-11T00:21:45.118 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:45.117+0000 7ff76fe0c740 1 -- 172.21.15.12:0/2693403443 shutdown_connections 2024-09-11T00:21:45.118 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:45.117+0000 7ff76fe0c740 1 --2- 172.21.15.12:0/2693403443 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x5590e62eb4a0 0x5590e62ed940 crc :-1 s=READY pgs=215 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:21:45.118 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:45.117+0000 7ff76fe0c740 1 --2- 172.21.15.12:0/2693403443 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5590e62db6e0 0x5590e62d9150 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:21:45.118 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:45.117+0000 7ff76fe0c740 1 --2- 172.21.15.12:0/2693403443 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5590e62d0e10 0x5590e62d8c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:21:45.119 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:45.117+0000 7ff76fe0c740 1 --2- 172.21.15.12:0/2693403443 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5590e62cf8b0 0x5590e62d86d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:21:45.119 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:45.117+0000 7ff76fe0c740 1 -- 172.21.15.12:0/2693403443 >> 172.21.15.12:0/2693403443 conn(0x5590e62996f0 msgr2=0x5590e62d2d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:21:45.119 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:45.117+0000 7ff74fe00640 0 client.27112 ms_handle_reset on v2:172.21.15.165:6838/501312984 2024-09-11T00:21:45.119 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:45.118+0000 7ff76fe0c740 1 -- 172.21.15.12:0/2693403443 shutdown_connections 2024-09-11T00:21:45.119 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:45.118+0000 7ff76fe0c740 1 -- 172.21.15.12:0/2693403443 wait complete. 2024-09-11T00:21:45.119 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:45.118+0000 7ff76fe0c740 20 client.27112 trim_cache size 0 max 16384 2024-09-11T00:21:45.153 INFO:tasks.workunit.client.0.smithi012.stdout: 10 33093 19.34 MB/sec warmup 101 sec latency 73.073 ms 2024-09-11T00:21:45.159 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:45.158+0000 7ff76fe0c740 1 Processor -- start 2024-09-11T00:21:45.160 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:45.159+0000 7ff76fe0c740 1 -- start start 2024-09-11T00:21:45.161 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:45.159+0000 7ff76fe0c740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5590e62cf8b0 0x5590e62ccab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:21:45.161 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:45.159+0000 7ff76fe0c740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5590e62d0e10 0x5590e62ccff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:21:45.161 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:45.159+0000 7ff76fe0c740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5590e62db6e0 0x5590e62d5900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:21:45.161 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:45.159+0000 7ff76fe0c740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7ff7681ad680 con 0x5590e62db6e0 2024-09-11T00:21:45.161 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:45.159+0000 7ff76fe0c740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7ff7681af3a0 con 0x5590e62cf8b0 2024-09-11T00:21:45.161 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:45.159+0000 7ff76fe0c740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7ff7681addb0 con 0x5590e62d0e10 2024-09-11T00:21:45.161 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:45.159+0000 7ff766800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5590e62cf8b0 0x5590e62ccab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:21:45.162 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:45.159+0000 7ff766800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5590e62cf8b0 0x5590e62ccab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.12:55612/0 (socket says 172.21.15.12:55612) 2024-09-11T00:21:45.162 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:45.159+0000 7ff766800640 1 -- 172.21.15.12:0/80677249 learned_addr learned my addr 172.21.15.12:0/80677249 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:21:45.162 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:45.159+0000 7ff765e00640 1 --2- 172.21.15.12:0/80677249 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5590e62d0e10 0x5590e62ccff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:21:45.162 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:45.160+0000 7ff765e00640 1 -- 172.21.15.12:0/80677249 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5590e62cf8b0 msgr2=0x5590e62ccab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:21:45.162 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:45.160+0000 7ff765e00640 1 --2- 172.21.15.12:0/80677249 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5590e62cf8b0 0x5590e62ccab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:21:45.162 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:45.160+0000 7ff767200640 1 --2- 172.21.15.12:0/80677249 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5590e62db6e0 0x5590e62d5900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:21:45.162 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:45.160+0000 7ff765e00640 1 -- 172.21.15.12:0/80677249 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5590e62db6e0 msgr2=0x5590e62d5900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:21:45.162 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:45.160+0000 7ff765e00640 1 --2- 172.21.15.12:0/80677249 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5590e62db6e0 0x5590e62d5900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:21:45.162 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:45.160+0000 7ff765e00640 1 -- 172.21.15.12:0/80677249 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5590e62d6bd0 con 0x5590e62d0e10 2024-09-11T00:21:45.163 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:45.160+0000 7ff767200640 1 --2- 172.21.15.12:0/80677249 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5590e62db6e0 0x5590e62d5900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T00:21:45.163 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:45.161+0000 7ff765e00640 1 --2- 172.21.15.12:0/80677249 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5590e62d0e10 0x5590e62ccff0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7ff750024b60 tx=0x7ff7500248c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7175abcc02eec998 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:21:45.163 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:45.161+0000 7ff756a00640 1 -- 172.21.15.12:0/80677249 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7500242d0 con 0x5590e62d0e10 2024-09-11T00:21:45.163 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:45.161+0000 7ff756a00640 1 -- 172.21.15.12:0/80677249 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff750033030 con 0x5590e62d0e10 2024-09-11T00:21:45.163 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:45.161+0000 7ff756a00640 1 -- 172.21.15.12:0/80677249 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff750003bd0 con 0x5590e62d0e10 2024-09-11T00:21:45.164 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:45.162+0000 7ff76fe0c740 1 -- 172.21.15.12:0/80677249 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5590e62d0e10 msgr2=0x5590e62ccff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:21:45.164 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:45.162+0000 7ff76fe0c740 1 --2- 172.21.15.12:0/80677249 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5590e62d0e10 0x5590e62ccff0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7ff750024b60 tx=0x7ff7500248c0 comp rx=0 tx=0).stop 2024-09-11T00:21:45.164 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:45.162+0000 7ff76fe0c740 1 -- 172.21.15.12:0/80677249 shutdown_connections 2024-09-11T00:21:45.164 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:45.162+0000 7ff76fe0c740 1 --2- 172.21.15.12:0/80677249 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5590e62db6e0 0x5590e62d5900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:21:45.164 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:45.162+0000 7ff76fe0c740 1 --2- 172.21.15.12:0/80677249 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5590e62d0e10 0x5590e62ccff0 secure :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7ff750024b60 tx=0x7ff7500248c0 comp rx=0 tx=0).stop 2024-09-11T00:21:45.164 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:45.162+0000 7ff76fe0c740 1 --2- 172.21.15.12:0/80677249 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5590e62cf8b0 0x5590e62ccab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:21:45.164 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:45.162+0000 7ff76fe0c740 1 -- 172.21.15.12:0/80677249 >> 172.21.15.12:0/80677249 conn(0x5590e62996f0 msgr2=0x5590e62da520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:21:45.165 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:45.163+0000 7ff76fe0c740 1 -- 172.21.15.12:0/80677249 shutdown_connections 2024-09-11T00:21:45.165 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:45.163+0000 7ff76fe0c740 1 -- 172.21.15.12:0/80677249 wait complete. 2024-09-11T00:21:45.165 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:45.163+0000 7ff76fe0c740 1 -- start start 2024-09-11T00:21:45.165 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:45.164+0000 7ff76fe0c740 1 Processor -- start 2024-09-11T00:21:45.167 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:45.165+0000 7ff76fe0c740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5590e62cf8b0 0x5590e62e39c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:21:45.167 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:45.165+0000 7ff76fe0c740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5590e62db6e0 0x5590e62e3f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:21:45.167 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:45.165+0000 7ff76fe0c740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5590e62d48f0 0x5590e62d4d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:21:45.168 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:45.165+0000 7ff76fe0c740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7ff7681aecd0 con 0x5590e62d48f0 2024-09-11T00:21:45.169 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:45.165+0000 7ff76fe0c740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7ff7681ae540 con 0x5590e62db6e0 2024-09-11T00:21:45.169 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:45.165+0000 7ff76fe0c740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7ff7681a0700 con 0x5590e62cf8b0 2024-09-11T00:21:45.169 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:45.165+0000 7ff76fe0c740 10 client.0 _ll_register_callbacks cb 0x7ff760005b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:21:45.169 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:45.165+0000 7ff765e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5590e62db6e0 0x5590e62e3f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:21:45.169 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:45.165+0000 7ff765e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5590e62db6e0 0x5590e62e3f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.12:55620/0 (socket says 172.21.15.12:55620) 2024-09-11T00:21:45.169 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:45.165+0000 7ff765e00640 1 -- 172.21.15.12:0/1984287826 learned_addr learned my addr 172.21.15.12:0/1984287826 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:21:45.169 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:45.166+0000 7ff766800640 1 --2- 172.21.15.12:0/1984287826 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5590e62cf8b0 0x5590e62e39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:21:45.169 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:45.166+0000 7ff766800640 1 -- 172.21.15.12:0/1984287826 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5590e62db6e0 msgr2=0x5590e62e3f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:21:45.169 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:45.166+0000 7ff766800640 1 --2- 172.21.15.12:0/1984287826 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5590e62db6e0 0x5590e62e3f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:21:45.169 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:45.166+0000 7ff766800640 1 -- 172.21.15.12:0/1984287826 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5590e62d48f0 msgr2=0x5590e62d4d70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-11T00:21:45.169 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:45.167+0000 7ff766800640 1 --2- 172.21.15.12:0/1984287826 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5590e62d48f0 0x5590e62d4d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:21:45.169 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:45.167+0000 7ff766800640 1 -- 172.21.15.12:0/1984287826 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5590e62d54b0 con 0x5590e62cf8b0 2024-09-11T00:21:45.169 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:45.167+0000 7ff765e00640 1 --2- 172.21.15.12:0/1984287826 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5590e62db6e0 0x5590e62e3f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:21:45.170 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:45.168+0000 7ff766800640 1 --2- 172.21.15.12:0/1984287826 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5590e62cf8b0 0x5590e62e39c0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7ff758000c00 tx=0x7ff758014000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=862d54c2fb7c9e8b server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:21:45.170 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:45.168+0000 7ff74fe00640 10 client.0 ms_handle_connect on v2:172.21.15.178:3300/0 2024-09-11T00:21:45.170 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:45.168+0000 7ff74fe00640 1 -- 172.21.15.12:0/1984287826 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff758005020 con 0x5590e62cf8b0 2024-09-11T00:21:45.171 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:45.169+0000 7ff74fe00640 1 -- 172.21.15.12:0/1984287826 <== mon.1 v2:172.21.15.178:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9208+0+0 (secure 0 0 0) 0x7ff7580051c0 con 0x5590e62cf8b0 2024-09-11T00:21:45.171 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:45.169+0000 7ff76fe0c740 1 -- 172.21.15.12:0/1984287826 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff750033030 con 0x5590e62cf8b0 2024-09-11T00:21:45.171 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:45.169+0000 7ff74fe00640 1 -- 172.21.15.12:0/1984287826 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7ff758038050 con 0x5590e62cf8b0 2024-09-11T00:21:45.171 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:45.169+0000 7ff76fe0c740 10 client.27118 fetch_fsmap learned FSMap version 54 2024-09-11T00:21:45.171 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:45.169+0000 7ff76fe0c740 1 -- 172.21.15.12:0/1984287826 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) -- 0x5590e6359420 con 0x5590e62cf8b0 2024-09-11T00:21:45.171 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:45.170+0000 7ff74fe00640 1 -- 172.21.15.12:0/1984287826 <== mon.1 v2:172.21.15.178:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff750033030 con 0x5590e62cf8b0 2024-09-11T00:21:45.171 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:45.170+0000 7ff74fe00640 1 -- 172.21.15.12:0/1984287826 <== mon.1 v2:172.21.15.178:3300/0 5 ==== fsmap(e 54) ==== 8422+0+0 (secure 0 0 0) 0x7ff75801f420 con 0x5590e62cf8b0 2024-09-11T00:21:45.172 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:45.170+0000 7ff76fe0c740 10 client.27118 fetch_fsmap finished waiting for FSMap version 54 2024-09-11T00:21:45.172 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:45.170+0000 7ff76fe0c740 10 client.27118 resolve_mds: validated gid 24674 aka daemon mds.b 2024-09-11T00:21:45.172 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:45.170+0000 7ff76fe0c740 20 client.27118 populate_metadata read hostname 'smithi012' 2024-09-11T00:21:45.172 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:45.171+0000 7ff76fe0c740 1 --2- 172.21.15.12:0/1984287826 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x5590e6359bb0 0x5590e635bfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:21:45.172 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:45.171+0000 7ff76fe0c740 4 client.27118 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 142.75, "await_for": 45.0}] 2024-09-11T00:21:45.172 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:45.171+0000 7ff76fe0c740 1 -- 172.21.15.12:0/1984287826 --> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 142.75, "await_for": 45.0}) -- 0x5590e6242490 con 0x5590e6359bb0 2024-09-11T00:21:45.173 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:45.171+0000 7ff765e00640 1 --2- 172.21.15.12:0/1984287826 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x5590e6359bb0 0x5590e635bfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:21:45.174 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:45.173+0000 7ff74fe00640 10 client.27118 ms_handle_connect on v2:172.21.15.165:6838/501312984 2024-09-11T00:21:45.174 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:45.173+0000 7ff765e00640 1 --2- 172.21.15.12:0/1984287826 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x5590e6359bb0 0x5590e635bfd0 crc :-1 s=READY pgs=216 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6eeaaf68279aa614 server_cookie=37a2c3ab5f2e0e71 in_seq=0 out_seq=0 2024-09-11T00:21:45.627 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:45.625+0000 7ff74fe00640 1 -- 172.21.15.12:0/1984287826 <== mds.0 v2:172.21.15.165:6838/501312984 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x5590e6242490 con 0x5590e6359bb0 2024-09-11T00:21:45.627 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:45.625+0000 7ff74fe00640 10 client.27118 handle_command_reply: tid=0 2024-09-11T00:21:45.627 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:45.626+0000 7ff76fe0c740 1 client.27118 shutdown 2024-09-11T00:21:45.628 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:45.626+0000 7ff76fe0c740 1 -- 172.21.15.12:0/1984287826 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5590e62cf8b0 msgr2=0x5590e62e39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:21:45.628 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:45.626+0000 7ff76fe0c740 1 --2- 172.21.15.12:0/1984287826 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5590e62cf8b0 0x5590e62e39c0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7ff758000c00 tx=0x7ff758014000 comp rx=0 tx=0).stop 2024-09-11T00:21:45.629 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:45.626+0000 7ff76fe0c740 1 -- 172.21.15.12:0/1984287826 shutdown_connections 2024-09-11T00:21:45.629 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:45.626+0000 7ff76fe0c740 1 --2- 172.21.15.12:0/1984287826 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x5590e6359bb0 0x5590e635bfd0 crc :-1 s=READY pgs=216 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:21:45.629 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:45.626+0000 7ff76fe0c740 1 --2- 172.21.15.12:0/1984287826 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5590e62d48f0 0x5590e62d4d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:21:45.629 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:45.626+0000 7ff76fe0c740 1 --2- 172.21.15.12:0/1984287826 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5590e62db6e0 0x5590e62e3f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:21:45.629 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:45.626+0000 7ff74fe00640 0 client.27118 ms_handle_reset on v2:172.21.15.165:6838/501312984 2024-09-11T00:21:45.629 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:45.626+0000 7ff76fe0c740 1 --2- 172.21.15.12:0/1984287826 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5590e62cf8b0 0x5590e62e39c0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:21:45.629 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:45.627+0000 7ff76fe0c740 1 -- 172.21.15.12:0/1984287826 >> 172.21.15.12:0/1984287826 conn(0x5590e62996f0 msgr2=0x5590e62d86a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:21:45.629 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:45.627+0000 7ff76fe0c740 1 -- 172.21.15.12:0/1984287826 shutdown_connections 2024-09-11T00:21:45.629 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:45.627+0000 7ff76fe0c740 1 -- 172.21.15.12:0/1984287826 wait complete. 2024-09-11T00:21:45.629 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:45.627+0000 7ff76fe0c740 20 client.27118 trim_cache size 0 max 16384 2024-09-11T00:21:45.630 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-09-11T00:21:45.630 INFO:teuthology.orchestra.run.smithi012.stdout: "epoch": 54, 2024-09-11T00:21:45.630 INFO:teuthology.orchestra.run.smithi012.stdout: "leader": 24674, 2024-09-11T00:21:45.630 INFO:teuthology.orchestra.run.smithi012.stdout: "set_version": 46, 2024-09-11T00:21:45.630 INFO:teuthology.orchestra.run.smithi012.stdout: "sets": { 2024-09-11T00:21:45.630 INFO:teuthology.orchestra.run.smithi012.stdout: "7dde0ef8": { 2024-09-11T00:21:45.630 INFO:teuthology.orchestra.run.smithi012.stdout: "version": 46, 2024-09-11T00:21:45.630 INFO:teuthology.orchestra.run.smithi012.stdout: "age_ref": 0.0, 2024-09-11T00:21:45.630 INFO:teuthology.orchestra.run.smithi012.stdout: "state": { 2024-09-11T00:21:45.630 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "QUIESCED", 2024-09-11T00:21:45.630 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 0.0 2024-09-11T00:21:45.630 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:21:45.631 INFO:teuthology.orchestra.run.smithi012.stdout: "timeout": 90.0, 2024-09-11T00:21:45.631 INFO:teuthology.orchestra.run.smithi012.stdout: "expiration": 142.8, 2024-09-11T00:21:45.631 INFO:teuthology.orchestra.run.smithi012.stdout: "members": { 2024-09-11T00:21:45.631 INFO:teuthology.orchestra.run.smithi012.stdout: "file:/": { 2024-09-11T00:21:45.631 INFO:teuthology.orchestra.run.smithi012.stdout: "excluded": false, 2024-09-11T00:21:45.631 INFO:teuthology.orchestra.run.smithi012.stdout: "state": { 2024-09-11T00:21:45.631 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "QUIESCED", 2024-09-11T00:21:45.631 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 0.0 2024-09-11T00:21:45.631 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:21:45.631 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:21:45.631 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:21:45.631 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:21:45.631 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:21:45.632 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-09-11T00:21:45.632 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:45.628+0000 7ff76ca00640 1 -- 172.21.15.12:0/1051351159 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7ff7480b54e0 msgr2=0x7ff7480b79a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:21:45.632 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:45.628+0000 7ff76ca00640 1 --2- 172.21.15.12:0/1051351159 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7ff7480b54e0 0x7ff7480b79a0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x5590e62d24f0 tx=0x7ff75800a2d0 comp rx=0 tx=0).stop 2024-09-11T00:21:45.632 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:45.628+0000 7ff76ca00640 1 -- 172.21.15.12:0/1051351159 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff768106d40 msgr2=0x7ff7681a4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:21:45.632 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:45.628+0000 7ff76ca00640 1 --2- 172.21.15.12:0/1051351159 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff768106d40 0x7ff7681a4eb0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7ff76002a960 tx=0x7ff760003d60 comp rx=0 tx=0).stop 2024-09-11T00:21:45.632 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:45.628+0000 7ff76ca00640 1 -- 172.21.15.12:0/1051351159 shutdown_connections 2024-09-11T00:21:45.633 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:45.628+0000 7ff76ca00640 1 --2- 172.21.15.12:0/1051351159 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7ff7480b54e0 0x7ff7480b79a0 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:21:45.633 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:45.628+0000 7ff76ca00640 1 --2- 172.21.15.12:0/1051351159 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff768106d40 0x7ff7681a4eb0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:21:45.633 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:45.628+0000 7ff76ca00640 1 --2- 172.21.15.12:0/1051351159 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff768102220 0x7ff76819fb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:21:45.633 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:45.628+0000 7ff76ca00640 1 --2- 172.21.15.12:0/1051351159 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff768101900 0x7ff76819f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:21:45.633 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:45.628+0000 7ff76ca00640 1 -- 172.21.15.12:0/1051351159 >> 172.21.15.12:0/1051351159 conn(0x7ff7680fd5b0 msgr2=0x7ff768104670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:21:45.633 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:45.628+0000 7ff76ca00640 1 -- 172.21.15.12:0/1051351159 shutdown_connections 2024-09-11T00:21:45.633 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:21:45.628+0000 7ff76ca00640 1 -- 172.21.15.12:0/1051351159 wait complete. 2024-09-11T00:21:46.153 INFO:tasks.workunit.client.0.smithi012.stdout: 10 33135 19.17 MB/sec warmup 102 sec latency 967.519 ms 2024-09-11T00:21:46.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:21:46 smithi178 ceph-mon[35919]: pgmap v839: 129 pgs: 129 active+clean; 377 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 604 KiB/s wr, 177 op/s 2024-09-11T00:21:46.532 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:21:46 smithi165 ceph-mon[35356]: pgmap v839: 129 pgs: 129 active+clean; 377 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 604 KiB/s wr, 177 op/s 2024-09-11T00:21:46.563 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:21:46 smithi012 ceph-mon[30333]: pgmap v839: 129 pgs: 129 active+clean; 377 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 604 KiB/s wr, 177 op/s 2024-09-11T00:21:47.153 INFO:tasks.workunit.client.0.smithi012.stdout: 10 33135 18.98 MB/sec warmup 103 sec latency 1967.681 ms 2024-09-11T00:21:47.579 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '7dde0ef8', quiesce took 6.6 seconds. Will release after: 16.15 2024-09-11T00:21:47.977 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:47.976+0000 7f3f1ea00640 1 -- 172.21.15.12:0/1514854446 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3f18105cd0 msgr2=0x7f3f18106150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:21:47.978 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:47.976+0000 7f3f1ea00640 1 --2- 172.21.15.12:0/1514854446 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3f18105cd0 0x7f3f18106150 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f3f00009930 tx=0x7f3f0002a420 comp rx=0 tx=0).stop 2024-09-11T00:21:47.978 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:47.976+0000 7f3f1ea00640 1 -- 172.21.15.12:0/1514854446 shutdown_connections 2024-09-11T00:21:47.978 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:47.976+0000 7f3f1ea00640 1 --2- 172.21.15.12:0/1514854446 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3f18106690 0x7f3f1810d010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:21:47.978 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:47.976+0000 7f3f1ea00640 1 --2- 172.21.15.12:0/1514854446 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3f18105cd0 0x7f3f18106150 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:21:47.978 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:47.976+0000 7f3f1ea00640 1 --2- 172.21.15.12:0/1514854446 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3f18104b80 0x7f3f18104f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:21:47.978 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:47.976+0000 7f3f1ea00640 1 -- 172.21.15.12:0/1514854446 >> 172.21.15.12:0/1514854446 conn(0x7f3f18100440 msgr2=0x7f3f18102860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:21:47.979 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:47.976+0000 7f3f1ea00640 1 -- 172.21.15.12:0/1514854446 shutdown_connections 2024-09-11T00:21:47.979 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:47.976+0000 7f3f1ea00640 1 -- 172.21.15.12:0/1514854446 wait complete. 2024-09-11T00:21:47.981 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:47.977+0000 7f3f1ea00640 1 Processor -- start 2024-09-11T00:21:47.981 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:47.977+0000 7f3f1ea00640 1 -- start start 2024-09-11T00:21:47.981 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:47.977+0000 7f3f1ea00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3f18104b80 0x7f3f1819b020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:21:47.981 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:47.977+0000 7f3f1ea00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3f18105cd0 0x7f3f1819b670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:21:47.981 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:47.977+0000 7f3f1ea00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3f18106690 0x7f3f1819c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:21:47.981 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:47.977+0000 7f3f1ea00640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f3f18110840 con 0x7f3f18105cd0 2024-09-11T00:21:47.982 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:47.977+0000 7f3f1ea00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f3f181106c0 con 0x7f3f18104b80 2024-09-11T00:21:47.982 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:47.977+0000 7f3f1ea00640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f3f181109c0 con 0x7f3f18106690 2024-09-11T00:21:47.982 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:47.978+0000 7f3f0f400640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3f18105cd0 0x7f3f1819b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:21:47.982 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:47.978+0000 7f3f0f400640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3f18105cd0 0x7f3f1819b670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:53394/0 (socket says 172.21.15.12:53394) 2024-09-11T00:21:47.982 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:47.978+0000 7f3f0f400640 1 -- 172.21.15.12:0/1034212229 learned_addr learned my addr 172.21.15.12:0/1034212229 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:21:47.982 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:47.978+0000 7f3f17e00640 1 --2- 172.21.15.12:0/1034212229 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3f18104b80 0x7f3f1819b020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:21:47.982 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:47.978+0000 7f3f1ca00640 1 --2- 172.21.15.12:0/1034212229 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3f18106690 0x7f3f1819c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:21:47.982 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:47.978+0000 7f3f17e00640 1 -- 172.21.15.12:0/1034212229 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3f18106690 msgr2=0x7f3f1819c2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:21:47.982 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:47.978+0000 7f3f17e00640 1 --2- 172.21.15.12:0/1034212229 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3f18106690 0x7f3f1819c2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:21:47.982 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:47.978+0000 7f3f17e00640 1 -- 172.21.15.12:0/1034212229 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3f18105cd0 msgr2=0x7f3f1819b670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:21:47.982 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:47.978+0000 7f3f17e00640 1 --2- 172.21.15.12:0/1034212229 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3f18105cd0 0x7f3f1819b670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:21:47.982 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:47.978+0000 7f3f17e00640 1 -- 172.21.15.12:0/1034212229 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f1819cb60 con 0x7f3f18104b80 2024-09-11T00:21:47.983 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:47.978+0000 7f3f17e00640 1 --2- 172.21.15.12:0/1034212229 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3f18104b80 0x7f3f1819b020 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f3f0800e9f0 tx=0x7f3f0800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:21:47.983 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:47.979+0000 7f3f15600640 1 -- 172.21.15.12:0/1034212229 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f0800ccc0 con 0x7f3f18104b80 2024-09-11T00:21:47.983 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:47.979+0000 7f3f15600640 1 -- 172.21.15.12:0/1034212229 <== mon.2 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3f080108e0 con 0x7f3f18104b80 2024-09-11T00:21:47.983 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:47.979+0000 7f3f1ea00640 1 -- 172.21.15.12:0/1034212229 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3f1819ce50 con 0x7f3f18104b80 2024-09-11T00:21:47.983 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:47.979+0000 7f3f15600640 1 -- 172.21.15.12:0/1034212229 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f08018880 con 0x7f3f18104b80 2024-09-11T00:21:47.983 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:47.979+0000 7f3f1ea00640 1 -- 172.21.15.12:0/1034212229 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3f181a3520 con 0x7f3f18104b80 2024-09-11T00:21:47.986 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:47.982+0000 7f3f21a41740 1 Processor -- start 2024-09-11T00:21:47.987 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:47.982+0000 7f3f21a41740 1 -- start start 2024-09-11T00:21:47.987 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:47.982+0000 7f3f21a41740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555acd34e790 0x555acd350b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:21:47.987 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:47.982+0000 7f3f21a41740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555acd3510c0 0x555acd3534b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:21:47.987 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:47.982+0000 7f3f21a41740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555acd353b00 0x555acd355fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:21:47.987 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:47.982+0000 7f3f21a41740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x555acd2c16b0 con 0x555acd34e790 2024-09-11T00:21:47.987 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:47.982+0000 7f3f21a41740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x555acd2b0ae0 con 0x555acd353b00 2024-09-11T00:21:47.987 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:47.982+0000 7f3f21a41740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x555acd2b48f0 con 0x555acd3510c0 2024-09-11T00:21:47.987 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:47.982+0000 7f3f15600640 1 -- 172.21.15.12:0/1034212229 <== mon.2 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7f3f08010430 con 0x7f3f18104b80 2024-09-11T00:21:47.987 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:47.983+0000 7f3f1ca00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555acd34e790 0x555acd350b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:21:47.987 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:47.983+0000 7f3f1ca00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555acd34e790 0x555acd350b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:53400/0 (socket says 172.21.15.12:53400) 2024-09-11T00:21:47.988 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:47.983+0000 7f3f1ca00640 1 -- 172.21.15.12:0/4234633110 learned_addr learned my addr 172.21.15.12:0/4234633110 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:21:47.988 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:47.983+0000 7f3f0f400640 1 --2- 172.21.15.12:0/4234633110 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555acd353b00 0x555acd355fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:21:47.988 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:47.983+0000 7f3f17e00640 1 --2- 172.21.15.12:0/4234633110 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555acd3510c0 0x555acd3534b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:21:47.988 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:47.983+0000 7f3f1ca00640 1 -- 172.21.15.12:0/4234633110 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555acd3510c0 msgr2=0x555acd3534b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:21:47.988 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:47.983+0000 7f3f1ca00640 1 --2- 172.21.15.12:0/4234633110 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555acd3510c0 0x555acd3534b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:21:47.988 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:47.983+0000 7f3f1ca00640 1 -- 172.21.15.12:0/4234633110 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555acd353b00 msgr2=0x555acd355fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:21:47.988 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:47.983+0000 7f3f1ca00640 1 --2- 172.21.15.12:0/4234633110 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555acd353b00 0x555acd355fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:21:47.988 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:47.983+0000 7f3f1ca00640 1 -- 172.21.15.12:0/4234633110 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555acd27d780 con 0x555acd34e790 2024-09-11T00:21:47.988 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:47.983+0000 7f3f1ca00640 1 --2- 172.21.15.12:0/4234633110 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555acd34e790 0x555acd350b80 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f3f10009f90 tx=0x7f3f100118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35ade3617f9f590b server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:21:47.988 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:47.984+0000 7f3f0d200640 1 -- 172.21.15.12:0/4234633110 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f10004d20 con 0x555acd34e790 2024-09-11T00:21:47.988 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:47.984+0000 7f3f0d200640 1 -- 172.21.15.12:0/4234633110 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3f10004050 con 0x555acd34e790 2024-09-11T00:21:47.988 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:47.984+0000 7f3f0d200640 1 -- 172.21.15.12:0/4234633110 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f10005260 con 0x555acd34e790 2024-09-11T00:21:47.989 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:47.984+0000 7f3f15600640 1 --2- 172.21.15.12:0/1034212229 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f3efc0b52e0 0x7f3efc0b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:21:47.989 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:47.984+0000 7f3f21a41740 1 -- 172.21.15.12:0/4234633110 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555acd34e790 msgr2=0x555acd350b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:21:47.989 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:47.984+0000 7f3f21a41740 1 --2- 172.21.15.12:0/4234633110 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555acd34e790 0x555acd350b80 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f3f10009f90 tx=0x7f3f100118d0 comp rx=0 tx=0).stop 2024-09-11T00:21:47.989 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:47.984+0000 7f3f15600640 1 -- 172.21.15.12:0/1034212229 <== mon.2 v2:172.21.15.165:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9208+0+0 (secure 0 0 0) 0x7f3f080e6320 con 0x7f3f18104b80 2024-09-11T00:21:47.989 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:47.984+0000 7f3f21a41740 1 -- 172.21.15.12:0/4234633110 shutdown_connections 2024-09-11T00:21:47.989 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:47.984+0000 7f3f21a41740 1 --2- 172.21.15.12:0/4234633110 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555acd353b00 0x555acd355fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:21:47.989 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:47.984+0000 7f3f21a41740 1 --2- 172.21.15.12:0/4234633110 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555acd3510c0 0x555acd3534b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:21:47.989 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:47.984+0000 7f3f21a41740 1 --2- 172.21.15.12:0/4234633110 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555acd34e790 0x555acd350b80 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:21:47.989 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:47.984+0000 7f3f21a41740 1 -- 172.21.15.12:0/4234633110 >> 172.21.15.12:0/4234633110 conn(0x555acd3498e0 msgr2=0x555acd34a670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:21:47.989 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:47.984+0000 7f3f17e00640 1 --2- 172.21.15.12:0/1034212229 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f3efc0b52e0 0x7f3efc0b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:21:47.989 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:47.985+0000 7f3f21a41740 1 -- 172.21.15.12:0/4234633110 shutdown_connections 2024-09-11T00:21:47.990 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:47.985+0000 7f3f21a41740 1 -- 172.21.15.12:0/4234633110 wait complete. 2024-09-11T00:21:47.990 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:47.985+0000 7f3f17e00640 1 --2- 172.21.15.12:0/1034212229 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f3efc0b52e0 0x7f3efc0b77a0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f3f08000c00 tx=0x7f3f08009700 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:21:47.990 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:47.985+0000 7f3f21a41740 1 -- start start 2024-09-11T00:21:47.990 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:47.985+0000 7f3f21a41740 1 Processor -- start 2024-09-11T00:21:47.993 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:47.986+0000 7f3f21a41740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555acd34e790 0x555acd357ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:21:47.993 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:47.986+0000 7f3f21a41740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555acd3510c0 0x555acd358580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:21:47.993 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:47.986+0000 7f3f21a41740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555acd353b00 0x555acd359150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:21:47.993 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:47.986+0000 7f3f21a41740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x555acd1e8ab0 con 0x555acd34e790 2024-09-11T00:21:47.994 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:47.986+0000 7f3f21a41740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x555acd257570 con 0x555acd353b00 2024-09-11T00:21:47.994 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:47.986+0000 7f3f21a41740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x555acd356b30 con 0x555acd3510c0 2024-09-11T00:21:47.994 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:47.986+0000 7f3f21a41740 10 client.0 _ll_register_callbacks cb 0x555acd266100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:21:47.994 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:47.986+0000 7f3f1ca00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555acd34e790 0x555acd357ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:21:47.994 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:47.986+0000 7f3f1ca00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555acd34e790 0x555acd357ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:53406/0 (socket says 172.21.15.12:53406) 2024-09-11T00:21:47.994 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:47.986+0000 7f3f1ca00640 1 -- 172.21.15.12:0/413368310 learned_addr learned my addr 172.21.15.12:0/413368310 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:21:47.994 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:47.986+0000 7f3f0f400640 1 --2- 172.21.15.12:0/413368310 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555acd3510c0 0x555acd358580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:21:47.994 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:47.986+0000 7f3f1ca00640 1 --2- 172.21.15.12:0/413368310 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555acd353b00 0x555acd359150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:21:47.994 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:47.987+0000 7f3f1ca00640 1 -- 172.21.15.12:0/413368310 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555acd3510c0 msgr2=0x555acd358580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:21:47.994 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:47.987+0000 7f3f1ca00640 1 --2- 172.21.15.12:0/413368310 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555acd3510c0 0x555acd358580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:21:47.994 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:47.987+0000 7f3f1ca00640 1 -- 172.21.15.12:0/413368310 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555acd353b00 msgr2=0x555acd359150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:21:47.994 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:47.987+0000 7f3f1ca00640 1 --2- 172.21.15.12:0/413368310 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555acd353b00 0x555acd359150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:21:47.994 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:47.987+0000 7f3f1ca00640 1 -- 172.21.15.12:0/413368310 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555acd012790 con 0x555acd34e790 2024-09-11T00:21:47.995 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:47.987+0000 7f3f0f400640 1 --2- 172.21.15.12:0/413368310 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555acd3510c0 0x555acd358580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:21:47.995 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:47.987+0000 7f3f1ca00640 1 --2- 172.21.15.12:0/413368310 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555acd34e790 0x555acd357ee0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f3f10009da0 tx=0x7f3f10009a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b37e8fcda631d4b server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:21:47.995 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:47.987+0000 7f3f05600640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-09-11T00:21:47.995 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:47.987+0000 7f3f05600640 1 -- 172.21.15.12:0/413368310 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f1002c020 con 0x555acd34e790 2024-09-11T00:21:47.995 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:47.987+0000 7f3f21a41740 1 -- 172.21.15.12:0/413368310 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3f10004050 con 0x555acd34e790 2024-09-11T00:21:47.995 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:47.987+0000 7f3f05600640 1 -- 172.21.15.12:0/413368310 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9208+0+0 (secure 0 0 0) 0x7f3f1000f4e0 con 0x555acd34e790 2024-09-11T00:21:47.995 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:47.987+0000 7f3f05600640 1 -- 172.21.15.12:0/413368310 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7f3f10037050 con 0x555acd34e790 2024-09-11T00:21:47.995 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:47.987+0000 7f3f21a41740 10 client.17148 fetch_fsmap learned FSMap version 54 2024-09-11T00:21:47.995 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:47.987+0000 7f3f21a41740 1 -- 172.21.15.12:0/413368310 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) -- 0x555acd08b620 con 0x555acd34e790 2024-09-11T00:21:47.995 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:47.988+0000 7f3f05600640 1 -- 172.21.15.12:0/413368310 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3f10004050 con 0x555acd34e790 2024-09-11T00:21:47.995 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:47.988+0000 7f3f05600640 1 -- 172.21.15.12:0/413368310 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 54) ==== 8422+0+0 (secure 0 0 0) 0x7f3f100050f0 con 0x555acd34e790 2024-09-11T00:21:47.995 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:47.988+0000 7f3f21a41740 10 client.17148 fetch_fsmap finished waiting for FSMap version 54 2024-09-11T00:21:47.995 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:47.988+0000 7f3f21a41740 10 client.17148 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T00:21:47.995 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:47.988+0000 7f3f21a41740 20 client.17148 populate_metadata read hostname 'smithi012' 2024-09-11T00:21:47.996 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:47.988+0000 7f3f21a41740 1 --2- 172.21.15.12:0/413368310 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x555acd363670 0x555acd365a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:21:47.996 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:47.988+0000 7f3f21a41740 4 client.17148 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T00:21:47.996 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:47.988+0000 7f3f21a41740 1 -- 172.21.15.12:0/413368310 --> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555acd2c3600 con 0x555acd363670 2024-09-11T00:21:47.996 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:47.988+0000 7f3f0f400640 1 --2- 172.21.15.12:0/413368310 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x555acd363670 0x555acd365a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:21:47.996 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:47.989+0000 7f3f0f400640 1 --2- 172.21.15.12:0/413368310 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x555acd363670 0x555acd365a60 crc :-1 s=READY pgs=217 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=370b005c0a5dea5a server_cookie=bec92bc9179195a3 in_seq=0 out_seq=0 2024-09-11T00:21:47.996 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:47.989+0000 7f3f05600640 10 client.17148 ms_handle_connect on v2:172.21.15.165:6838/501312984 2024-09-11T00:21:47.996 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:47.990+0000 7f3f05600640 1 -- 172.21.15.12:0/413368310 <== mds.0 v2:172.21.15.165:6838/501312984 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x555acd2c3600 con 0x555acd363670 2024-09-11T00:21:47.996 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:47.990+0000 7f3f05600640 10 client.17148 handle_command_reply: tid=0 2024-09-11T00:21:47.996 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:47.990+0000 7f3f21a41740 1 client.17148 shutdown 2024-09-11T00:21:47.996 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:47.990+0000 7f3f21a41740 1 -- 172.21.15.12:0/413368310 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555acd34e790 msgr2=0x555acd357ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:21:47.996 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:47.990+0000 7f3f21a41740 1 --2- 172.21.15.12:0/413368310 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555acd34e790 0x555acd357ee0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f3f10009da0 tx=0x7f3f10009a50 comp rx=0 tx=0).stop 2024-09-11T00:21:47.996 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:47.991+0000 7f3f21a41740 1 -- 172.21.15.12:0/413368310 shutdown_connections 2024-09-11T00:21:47.996 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:47.991+0000 7f3f21a41740 1 --2- 172.21.15.12:0/413368310 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x555acd363670 0x555acd365a60 crc :-1 s=READY pgs=217 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:21:47.997 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:47.991+0000 7f3f21a41740 1 --2- 172.21.15.12:0/413368310 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555acd353b00 0x555acd359150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:21:47.997 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:47.991+0000 7f3f21a41740 1 --2- 172.21.15.12:0/413368310 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555acd3510c0 0x555acd358580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:21:47.997 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:47.991+0000 7f3f05600640 0 client.17148 ms_handle_reset on v2:172.21.15.165:6838/501312984 2024-09-11T00:21:47.997 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:47.991+0000 7f3f21a41740 1 --2- 172.21.15.12:0/413368310 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555acd34e790 0x555acd357ee0 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:21:47.997 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:47.991+0000 7f3f21a41740 1 -- 172.21.15.12:0/413368310 >> 172.21.15.12:0/413368310 conn(0x555acd3498e0 msgr2=0x555acd348a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:21:47.997 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:47.991+0000 7f3f21a41740 1 -- 172.21.15.12:0/413368310 shutdown_connections 2024-09-11T00:21:47.997 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:47.991+0000 7f3f21a41740 1 -- 172.21.15.12:0/413368310 wait complete. 2024-09-11T00:21:47.997 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:47.991+0000 7f3f21a41740 20 client.17148 trim_cache size 0 max 16384 2024-09-11T00:21:48.009 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:48.006+0000 7f3f21a41740 1 Processor -- start 2024-09-11T00:21:48.009 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:48.006+0000 7f3f21a41740 1 -- start start 2024-09-11T00:21:48.009 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:48.006+0000 7f3f21a41740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555acd34e790 0x555acd3545f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:21:48.009 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:48.006+0000 7f3f21a41740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555acd3510c0 0x555acd34a140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:21:48.009 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:48.006+0000 7f3f21a41740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555acd353b00 0x555acd34a680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:21:48.009 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:48.007+0000 7f3f21a41740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f3f181ad9e0 con 0x555acd34e790 2024-09-11T00:21:48.009 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:48.007+0000 7f3f21a41740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f3f181af700 con 0x555acd353b00 2024-09-11T00:21:48.010 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:48.007+0000 7f3f21a41740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f3f181ae110 con 0x555acd3510c0 2024-09-11T00:21:48.010 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:48.007+0000 7f3f1ca00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555acd34e790 0x555acd3545f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:21:48.010 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:48.007+0000 7f3f1ca00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555acd34e790 0x555acd3545f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:53420/0 (socket says 172.21.15.12:53420) 2024-09-11T00:21:48.010 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:48.007+0000 7f3f1ca00640 1 -- 172.21.15.12:0/3235641873 learned_addr learned my addr 172.21.15.12:0/3235641873 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:21:48.010 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:48.007+0000 7f3f0f400640 1 --2- 172.21.15.12:0/3235641873 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555acd3510c0 0x555acd34a140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:21:48.010 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:48.007+0000 7f3f1ca00640 1 --2- 172.21.15.12:0/3235641873 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555acd353b00 0x555acd34a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:21:48.010 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:48.007+0000 7f3f1ca00640 1 -- 172.21.15.12:0/3235641873 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555acd3510c0 msgr2=0x555acd34a140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:21:48.010 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:48.007+0000 7f3f1ca00640 1 --2- 172.21.15.12:0/3235641873 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555acd3510c0 0x555acd34a140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:21:48.010 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:48.007+0000 7f3f1ca00640 1 -- 172.21.15.12:0/3235641873 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555acd353b00 msgr2=0x555acd34a680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:21:48.010 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:48.007+0000 7f3f1ca00640 1 --2- 172.21.15.12:0/3235641873 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555acd353b00 0x555acd34a680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:21:48.010 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:48.007+0000 7f3f1ca00640 1 -- 172.21.15.12:0/3235641873 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555acd365dd0 con 0x555acd34e790 2024-09-11T00:21:48.011 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:48.008+0000 7f3f1ca00640 1 --2- 172.21.15.12:0/3235641873 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555acd34e790 0x555acd3545f0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f3f10003b30 tx=0x7f3f10003aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=927bb9caa7d8b5d5 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:21:48.011 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:48.008+0000 7f3f07e00640 1 -- 172.21.15.12:0/3235641873 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f1002c030 con 0x555acd34e790 2024-09-11T00:21:48.011 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:48.008+0000 7f3f07e00640 1 -- 172.21.15.12:0/3235641873 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3f10004890 con 0x555acd34e790 2024-09-11T00:21:48.011 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:48.008+0000 7f3f07e00640 1 -- 172.21.15.12:0/3235641873 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f1000f4e0 con 0x555acd34e790 2024-09-11T00:21:48.014 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:48.008+0000 7f3f21a41740 1 -- 172.21.15.12:0/3235641873 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555acd34e790 msgr2=0x555acd3545f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:21:48.014 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:48.008+0000 7f3f21a41740 1 --2- 172.21.15.12:0/3235641873 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555acd34e790 0x555acd3545f0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f3f10003b30 tx=0x7f3f10003aa0 comp rx=0 tx=0).stop 2024-09-11T00:21:48.014 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:48.008+0000 7f3f21a41740 1 -- 172.21.15.12:0/3235641873 shutdown_connections 2024-09-11T00:21:48.014 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:48.009+0000 7f3f21a41740 1 --2- 172.21.15.12:0/3235641873 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555acd353b00 0x555acd34a680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:21:48.014 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:48.009+0000 7f3f21a41740 1 --2- 172.21.15.12:0/3235641873 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555acd3510c0 0x555acd34a140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:21:48.014 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:48.009+0000 7f3f21a41740 1 --2- 172.21.15.12:0/3235641873 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555acd34e790 0x555acd3545f0 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:21:48.015 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:48.009+0000 7f3f21a41740 1 -- 172.21.15.12:0/3235641873 >> 172.21.15.12:0/3235641873 conn(0x555acd3498e0 msgr2=0x555acd353ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:21:48.015 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:48.009+0000 7f3f21a41740 1 -- 172.21.15.12:0/3235641873 shutdown_connections 2024-09-11T00:21:48.015 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:48.009+0000 7f3f21a41740 1 -- 172.21.15.12:0/3235641873 wait complete. 2024-09-11T00:21:48.015 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:48.009+0000 7f3f21a41740 1 -- start start 2024-09-11T00:21:48.015 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:48.009+0000 7f3f21a41740 1 Processor -- start 2024-09-11T00:21:48.015 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:48.009+0000 7f3f21a41740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555acd34e790 0x555acd35fb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:21:48.015 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:48.009+0000 7f3f21a41740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555acd3510c0 0x555acd35e830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:21:48.015 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:48.009+0000 7f3f21a41740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555acd353b00 0x555acd35ef00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:21:48.015 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:48.009+0000 7f3f21a41740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f3f181af030 con 0x555acd353b00 2024-09-11T00:21:48.015 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:48.009+0000 7f3f21a41740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f3f181ae8a0 con 0x555acd3510c0 2024-09-11T00:21:48.015 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:48.009+0000 7f3f21a41740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f3f181ab9c0 con 0x555acd34e790 2024-09-11T00:21:48.015 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:48.010+0000 7f3f21a41740 10 client.0 _ll_register_callbacks cb 0x7f3f100093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:21:48.016 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:48.010+0000 7f3f1ca00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555acd353b00 0x555acd35ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:21:48.016 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:48.010+0000 7f3f1ca00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555acd353b00 0x555acd35ef00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:53430/0 (socket says 172.21.15.12:53430) 2024-09-11T00:21:48.016 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:48.010+0000 7f3f1ca00640 1 -- 172.21.15.12:0/2022582419 learned_addr learned my addr 172.21.15.12:0/2022582419 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:21:48.016 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:48.010+0000 7f3f1ca00640 1 --2- 172.21.15.12:0/2022582419 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555acd34e790 0x555acd35fb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:21:48.016 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:48.010+0000 7f3f0f400640 1 --2- 172.21.15.12:0/2022582419 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555acd3510c0 0x555acd35e830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:21:48.016 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:48.010+0000 7f3f0f400640 1 -- 172.21.15.12:0/2022582419 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555acd34e790 msgr2=0x555acd35fb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:21:48.016 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:48.010+0000 7f3f0f400640 1 --2- 172.21.15.12:0/2022582419 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555acd34e790 0x555acd35fb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:21:48.016 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:48.010+0000 7f3f0f400640 1 -- 172.21.15.12:0/2022582419 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555acd353b00 msgr2=0x555acd35ef00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:21:48.016 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:48.010+0000 7f3f0f400640 1 --2- 172.21.15.12:0/2022582419 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555acd353b00 0x555acd35ef00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:21:48.016 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:48.010+0000 7f3f0f400640 1 -- 172.21.15.12:0/2022582419 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555acd35d350 con 0x555acd3510c0 2024-09-11T00:21:48.016 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:48.010+0000 7f3f1ca00640 1 --2- 172.21.15.12:0/2022582419 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555acd353b00 0x555acd35ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:21:48.016 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:48.011+0000 7f3f0f400640 1 --2- 172.21.15.12:0/2022582419 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555acd3510c0 0x555acd35e830 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f3f00006010 tx=0x7f3f00004380 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b190df1bf60e956c server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:21:48.017 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:48.011+0000 7f3f05600640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-09-11T00:21:48.017 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:48.011+0000 7f3f05600640 1 -- 172.21.15.12:0/2022582419 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f00035040 con 0x555acd3510c0 2024-09-11T00:21:48.017 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:48.011+0000 7f3f21a41740 1 -- 172.21.15.12:0/2022582419 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3f10004890 con 0x555acd3510c0 2024-09-11T00:21:48.017 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:48.011+0000 7f3f05600640 1 -- 172.21.15.12:0/2022582419 <== mon.2 v2:172.21.15.165:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9208+0+0 (secure 0 0 0) 0x7f3f00003800 con 0x555acd3510c0 2024-09-11T00:21:48.017 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:48.011+0000 7f3f05600640 1 -- 172.21.15.12:0/2022582419 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7f3f0005a050 con 0x555acd3510c0 2024-09-11T00:21:48.017 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:48.011+0000 7f3f21a41740 10 client.27170 fetch_fsmap learned FSMap version 54 2024-09-11T00:21:48.017 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:48.011+0000 7f3f21a41740 1 -- 172.21.15.12:0/2022582419 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) -- 0x555acd3d07c0 con 0x555acd3510c0 2024-09-11T00:21:48.017 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:48.012+0000 7f3f05600640 1 -- 172.21.15.12:0/2022582419 <== mon.2 v2:172.21.15.165:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3f10004890 con 0x555acd3510c0 2024-09-11T00:21:48.017 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:48.012+0000 7f3f05600640 1 -- 172.21.15.12:0/2022582419 <== mon.2 v2:172.21.15.165:3300/0 5 ==== fsmap(e 54) ==== 8422+0+0 (secure 0 0 0) 0x7f3f00055030 con 0x555acd3510c0 2024-09-11T00:21:48.017 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:48.012+0000 7f3f21a41740 10 client.27170 fetch_fsmap finished waiting for FSMap version 54 2024-09-11T00:21:48.017 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:48.012+0000 7f3f21a41740 10 client.27170 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T00:21:48.018 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:48.012+0000 7f3f21a41740 20 client.27170 populate_metadata read hostname 'smithi012' 2024-09-11T00:21:48.018 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:48.012+0000 7f3f21a41740 1 --2- 172.21.15.12:0/2022582419 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x555acd3d1260 0x555acd3d3680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:21:48.018 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:48.012+0000 7f3f21a41740 4 client.27170 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T00:21:48.018 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:48.012+0000 7f3f21a41740 1 -- 172.21.15.12:0/2022582419 --> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] -- command(tid 0: {"prefix": "flush journal"}) -- 0x555acd2bf0b0 con 0x555acd3d1260 2024-09-11T00:21:48.018 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:48.012+0000 7f3f1ca00640 1 --2- 172.21.15.12:0/2022582419 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x555acd3d1260 0x555acd3d3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:21:48.018 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:48.013+0000 7f3f1ca00640 1 --2- 172.21.15.12:0/2022582419 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x555acd3d1260 0x555acd3d3680 crc :-1 s=READY pgs=218 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=144911a5412720cd server_cookie=8e729ea4545266b6 in_seq=0 out_seq=0 2024-09-11T00:21:48.018 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:48.013+0000 7f3f05600640 10 client.27170 ms_handle_connect on v2:172.21.15.165:6838/501312984 2024-09-11T00:21:48.153 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:48.151+0000 7f3f05600640 1 -- 172.21.15.12:0/2022582419 <== mds.0 v2:172.21.15.165:6838/501312984 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x555acd2bf0b0 con 0x555acd3d1260 2024-09-11T00:21:48.153 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:48.151+0000 7f3f05600640 10 client.27170 handle_command_reply: tid=0 2024-09-11T00:21:48.153 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:48.151+0000 7f3f21a41740 1 client.27170 shutdown 2024-09-11T00:21:48.153 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:48.151+0000 7f3f21a41740 1 -- 172.21.15.12:0/2022582419 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555acd3510c0 msgr2=0x555acd35e830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:21:48.153 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:48.151+0000 7f3f21a41740 1 --2- 172.21.15.12:0/2022582419 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555acd3510c0 0x555acd35e830 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f3f00006010 tx=0x7f3f00004380 comp rx=0 tx=0).stop 2024-09-11T00:21:48.153 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:48.152+0000 7f3f21a41740 1 -- 172.21.15.12:0/2022582419 shutdown_connections 2024-09-11T00:21:48.153 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:48.152+0000 7f3f21a41740 1 --2- 172.21.15.12:0/2022582419 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x555acd3d1260 0x555acd3d3680 crc :-1 s=READY pgs=218 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:21:48.154 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:48.152+0000 7f3f21a41740 1 --2- 172.21.15.12:0/2022582419 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555acd353b00 0x555acd35ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:21:48.154 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:48.152+0000 7f3f21a41740 1 --2- 172.21.15.12:0/2022582419 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555acd3510c0 0x555acd35e830 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:21:48.154 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:48.152+0000 7f3f21a41740 1 --2- 172.21.15.12:0/2022582419 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555acd34e790 0x555acd35fb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:21:48.154 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:48.152+0000 7f3f05600640 0 client.27170 ms_handle_reset on v2:172.21.15.165:6838/501312984 2024-09-11T00:21:48.154 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:48.152+0000 7f3f21a41740 1 -- 172.21.15.12:0/2022582419 >> 172.21.15.12:0/2022582419 conn(0x555acd3498e0 msgr2=0x555acd34d7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:21:48.154 INFO:tasks.workunit.client.0.smithi012.stdout: 10 33135 18.80 MB/sec warmup 104 sec latency 2967.749 ms 2024-09-11T00:21:48.155 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:48.152+0000 7f3f21a41740 1 -- 172.21.15.12:0/2022582419 shutdown_connections 2024-09-11T00:21:48.155 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:48.152+0000 7f3f21a41740 1 -- 172.21.15.12:0/2022582419 wait complete. 2024-09-11T00:21:48.155 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:48.152+0000 7f3f21a41740 20 client.27170 trim_cache size 0 max 16384 2024-09-11T00:21:48.155 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:{ 2024-09-11T00:21:48.155 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "message": "", 2024-09-11T00:21:48.155 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "return_code": 0 2024-09-11T00:21:48.155 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:} 2024-09-11T00:21:48.156 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:48.152+0000 7f3f1ea00640 1 -- 172.21.15.12:0/1034212229 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f3efc0b52e0 msgr2=0x7f3efc0b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:21:48.156 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:48.152+0000 7f3f1ea00640 1 --2- 172.21.15.12:0/1034212229 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f3efc0b52e0 0x7f3efc0b77a0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f3f08000c00 tx=0x7f3f08009700 comp rx=0 tx=0).stop 2024-09-11T00:21:48.156 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:48.152+0000 7f3f1ea00640 1 -- 172.21.15.12:0/1034212229 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3f18104b80 msgr2=0x7f3f1819b020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:21:48.156 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:48.152+0000 7f3f1ea00640 1 --2- 172.21.15.12:0/1034212229 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3f18104b80 0x7f3f1819b020 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f3f0800e9f0 tx=0x7f3f0800eeb0 comp rx=0 tx=0).stop 2024-09-11T00:21:48.156 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:48.153+0000 7f3f1ea00640 1 -- 172.21.15.12:0/1034212229 shutdown_connections 2024-09-11T00:21:48.156 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:48.153+0000 7f3f1ea00640 1 --2- 172.21.15.12:0/1034212229 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f3efc0b52e0 0x7f3efc0b77a0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:21:48.157 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:48.153+0000 7f3f1ea00640 1 --2- 172.21.15.12:0/1034212229 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3f18106690 0x7f3f1819c2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:21:48.157 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:48.153+0000 7f3f1ea00640 1 --2- 172.21.15.12:0/1034212229 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3f18105cd0 0x7f3f1819b670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:21:48.157 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:48.153+0000 7f3f1ea00640 1 --2- 172.21.15.12:0/1034212229 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3f18104b80 0x7f3f1819b020 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:21:48.157 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:48.153+0000 7f3f1ea00640 1 -- 172.21.15.12:0/1034212229 >> 172.21.15.12:0/1034212229 conn(0x7f3f18100440 msgr2=0x7f3f18109060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:21:48.157 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:48.153+0000 7f3f1ea00640 1 -- 172.21.15.12:0/1034212229 shutdown_connections 2024-09-11T00:21:48.157 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:21:48.153+0000 7f3f1ea00640 1 -- 172.21.15.12:0/1034212229 wait complete. 2024-09-11T00:21:48.532 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:21:48 smithi165 ceph-mon[35356]: pgmap v840: 129 pgs: 129 active+clean; 390 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 2.5 MiB/s wr, 440 op/s 2024-09-11T00:21:48.563 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:21:48 smithi012 ceph-mon[30333]: pgmap v840: 129 pgs: 129 active+clean; 390 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 2.5 MiB/s wr, 440 op/s 2024-09-11T00:21:48.581 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:21:48 smithi178 ceph-mon[35919]: pgmap v840: 129 pgs: 129 active+clean; 390 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 2.5 MiB/s wr, 440 op/s 2024-09-11T00:21:49.153 INFO:tasks.workunit.client.0.smithi012.stdout: 10 33135 18.62 MB/sec warmup 105 sec latency 3967.823 ms 2024-09-11T00:21:50.153 INFO:tasks.workunit.client.0.smithi012.stdout: 10 33135 18.44 MB/sec warmup 106 sec latency 4968.008 ms 2024-09-11T00:21:50.265 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:21:50 smithi012 ceph-mon[30333]: pgmap v841: 129 pgs: 129 active+clean; 390 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 2.4 MiB/s wr, 402 op/s 2024-09-11T00:21:50.532 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:21:50 smithi165 ceph-mon[35356]: pgmap v841: 129 pgs: 129 active+clean; 390 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 2.4 MiB/s wr, 402 op/s 2024-09-11T00:21:50.582 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:21:50 smithi178 ceph-mon[35919]: pgmap v841: 129 pgs: 129 active+clean; 390 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 2.4 MiB/s wr, 402 op/s 2024-09-11T00:21:51.153 INFO:tasks.workunit.client.0.smithi012.stdout: 10 33135 18.27 MB/sec warmup 107 sec latency 5968.068 ms 2024-09-11T00:21:52.153 INFO:tasks.workunit.client.0.smithi012.stdout: 10 33135 18.10 MB/sec warmup 108 sec latency 6968.119 ms 2024-09-11T00:21:52.532 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:21:52 smithi165 ceph-mon[35356]: pgmap v842: 129 pgs: 129 active+clean; 392 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 2.7 MiB/s wr, 481 op/s 2024-09-11T00:21:52.563 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:21:52 smithi012 ceph-mon[30333]: pgmap v842: 129 pgs: 129 active+clean; 392 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 2.7 MiB/s wr, 481 op/s 2024-09-11T00:21:52.581 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:21:52 smithi178 ceph-mon[35919]: pgmap v842: 129 pgs: 129 active+clean; 392 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 2.7 MiB/s wr, 481 op/s 2024-09-11T00:21:53.153 INFO:tasks.workunit.client.0.smithi012.stdout: 10 33135 17.94 MB/sec warmup 109 sec latency 7968.189 ms 2024-09-11T00:21:54.153 INFO:tasks.workunit.client.0.smithi012.stdout: 10 33135 17.77 MB/sec warmup 110 sec latency 8968.248 ms 2024-09-11T00:21:54.223 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:21:54 smithi012 ceph-mon[30333]: pgmap v843: 129 pgs: 129 active+clean; 391 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 3.2 MiB/s wr, 474 op/s 2024-09-11T00:21:54.532 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:21:54 smithi165 ceph-mon[35356]: pgmap v843: 129 pgs: 129 active+clean; 391 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 3.2 MiB/s wr, 474 op/s 2024-09-11T00:21:54.581 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:21:54 smithi178 ceph-mon[35919]: pgmap v843: 129 pgs: 129 active+clean; 391 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 3.2 MiB/s wr, 474 op/s 2024-09-11T00:21:55.163 INFO:tasks.workunit.client.0.smithi012.stdout: 10 33135 17.61 MB/sec warmup 111 sec latency 9977.697 ms 2024-09-11T00:21:55.532 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:21:55 smithi165 ceph-mon[35356]: pgmap v844: 129 pgs: 129 active+clean; 392 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 3.5 MiB/s wr, 429 op/s 2024-09-11T00:21:55.563 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:21:55 smithi012 ceph-mon[30333]: pgmap v844: 129 pgs: 129 active+clean; 392 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 3.5 MiB/s wr, 429 op/s 2024-09-11T00:21:55.581 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:21:55 smithi178 ceph-mon[35919]: pgmap v844: 129 pgs: 129 active+clean; 392 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 3.5 MiB/s wr, 429 op/s 2024-09-11T00:21:56.163 INFO:tasks.workunit.client.0.smithi012.stdout: 10 33135 17.45 MB/sec warmup 112 sec latency 10977.753 ms 2024-09-11T00:21:57.163 INFO:tasks.workunit.client.0.smithi012.stdout: 10 33135 17.30 MB/sec warmup 113 sec latency 11977.796 ms 2024-09-11T00:21:57.557 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:21:57 smithi012 ceph-mon[30333]: pgmap v845: 129 pgs: 129 active+clean; 395 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 5.5 MiB/s wr, 477 op/s 2024-09-11T00:21:57.782 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:21:57 smithi165 ceph-mon[35356]: pgmap v845: 129 pgs: 129 active+clean; 395 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 5.5 MiB/s wr, 477 op/s 2024-09-11T00:21:57.831 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:21:57 smithi178 ceph-mon[35919]: pgmap v845: 129 pgs: 129 active+clean; 395 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 5.5 MiB/s wr, 477 op/s 2024-09-11T00:21:58.163 INFO:tasks.workunit.client.0.smithi012.stdout: 10 33135 17.15 MB/sec warmup 114 sec latency 12977.858 ms 2024-09-11T00:21:59.163 INFO:tasks.workunit.client.0.smithi012.stdout: 10 33135 17.00 MB/sec warmup 115 sec latency 13977.991 ms 2024-09-11T00:21:59.782 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:21:59 smithi165 ceph-mon[35356]: pgmap v846: 129 pgs: 129 active+clean; 395 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.7 MiB/s wr, 216 op/s 2024-09-11T00:21:59.813 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:21:59 smithi012 ceph-mon[30333]: pgmap v846: 129 pgs: 129 active+clean; 395 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.7 MiB/s wr, 216 op/s 2024-09-11T00:21:59.831 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:21:59 smithi178 ceph-mon[35919]: pgmap v846: 129 pgs: 129 active+clean; 395 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.7 MiB/s wr, 216 op/s 2024-09-11T00:22:00.163 INFO:tasks.workunit.client.0.smithi012.stdout: 10 33135 16.85 MB/sec warmup 116 sec latency 14978.152 ms 2024-09-11T00:22:01.163 INFO:tasks.workunit.client.0.smithi012.stdout: 10 33135 16.71 MB/sec warmup 117 sec latency 15978.310 ms 2024-09-11T00:22:01.282 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:01.279+0000 7f2170400640 1 -- 172.21.15.12:0/2097304779 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2168102970 msgr2=0x7f2168102d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:22:01.282 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:01.279+0000 7f2170400640 1 --2- 172.21.15.12:0/2097304779 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2168102970 0x7f2168102d70 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f215c009930 tx=0x7f215c02a420 comp rx=0 tx=0).stop 2024-09-11T00:22:01.282 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:01.279+0000 7f2170400640 1 -- 172.21.15.12:0/2097304779 shutdown_connections 2024-09-11T00:22:01.282 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:01.279+0000 7f2170400640 1 --2- 172.21.15.12:0/2097304779 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2168104480 0x7f216810ae00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:01.282 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:01.279+0000 7f2170400640 1 --2- 172.21.15.12:0/2097304779 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2168103ac0 0x7f2168103f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:01.282 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:01.279+0000 7f2170400640 1 --2- 172.21.15.12:0/2097304779 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2168102970 0x7f2168102d70 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:01.282 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:01.279+0000 7f2170400640 1 -- 172.21.15.12:0/2097304779 >> 172.21.15.12:0/2097304779 conn(0x7f21680fe250 msgr2=0x7f2168100670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:22:01.282 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:01.279+0000 7f2170400640 1 -- 172.21.15.12:0/2097304779 shutdown_connections 2024-09-11T00:22:01.283 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:01.279+0000 7f2170400640 1 -- 172.21.15.12:0/2097304779 wait complete. 2024-09-11T00:22:01.285 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:01.280+0000 7f2170400640 1 Processor -- start 2024-09-11T00:22:01.285 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:01.280+0000 7f2170400640 1 -- start start 2024-09-11T00:22:01.285 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:01.281+0000 7f2170400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2168102970 0x7f2168076ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:22:01.285 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:01.281+0000 7f2170400640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2168103ac0 0x7f2168075540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:22:01.285 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:01.281+0000 7f2170400640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2168104480 0x7f2168075a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:22:01.285 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:01.281+0000 7f2170400640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f216810e5e0 con 0x7f2168103ac0 2024-09-11T00:22:01.285 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:01.281+0000 7f2170400640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f216810e460 con 0x7f2168102970 2024-09-11T00:22:01.285 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:01.281+0000 7f2170400640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f216810e760 con 0x7f2168104480 2024-09-11T00:22:01.286 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:01.281+0000 7f216d000640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2168103ac0 0x7f2168075540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:22:01.286 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:01.281+0000 7f216d000640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2168103ac0 0x7f2168075540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:34646/0 (socket says 172.21.15.12:34646) 2024-09-11T00:22:01.286 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:01.281+0000 7f216d000640 1 -- 172.21.15.12:0/22896465 learned_addr learned my addr 172.21.15.12:0/22896465 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:22:01.286 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:01.281+0000 7f216da00640 1 --2- 172.21.15.12:0/22896465 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2168102970 0x7f2168076ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:22:01.286 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:01.281+0000 7f216e400640 1 --2- 172.21.15.12:0/22896465 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2168104480 0x7f2168075a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:22:01.286 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:01.282+0000 7f216d000640 1 -- 172.21.15.12:0/22896465 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2168104480 msgr2=0x7f2168075a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:22:01.286 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:01.282+0000 7f216d000640 1 --2- 172.21.15.12:0/22896465 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2168104480 0x7f2168075a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:01.286 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:01.282+0000 7f216d000640 1 -- 172.21.15.12:0/22896465 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2168102970 msgr2=0x7f2168076ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:22:01.286 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:01.282+0000 7f216d000640 1 --2- 172.21.15.12:0/22896465 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2168102970 0x7f2168076ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:01.286 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:01.282+0000 7f216d000640 1 -- 172.21.15.12:0/22896465 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21680762f0 con 0x7f2168103ac0 2024-09-11T00:22:01.287 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:01.282+0000 7f216da00640 1 --2- 172.21.15.12:0/22896465 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2168102970 0x7f2168076ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:22:01.287 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:01.282+0000 7f216d000640 1 --2- 172.21.15.12:0/22896465 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2168103ac0 0x7f2168075540 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f215400e9f0 tx=0x7f215400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:22:01.287 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:01.282+0000 7f2166000640 1 -- 172.21.15.12:0/22896465 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f215400cd80 con 0x7f2168103ac0 2024-09-11T00:22:01.287 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:01.282+0000 7f2166000640 1 -- 172.21.15.12:0/22896465 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f21540054d0 con 0x7f2168103ac0 2024-09-11T00:22:01.287 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:01.282+0000 7f2166000640 1 -- 172.21.15.12:0/22896465 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2154010690 con 0x7f2168103ac0 2024-09-11T00:22:01.287 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:01.282+0000 7f2170400640 1 -- 172.21.15.12:0/22896465 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f21681a2ea0 con 0x7f2168103ac0 2024-09-11T00:22:01.287 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:01.282+0000 7f2170400640 1 -- 172.21.15.12:0/22896465 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f21681a3310 con 0x7f2168103ac0 2024-09-11T00:22:01.288 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:01.286+0000 7f2173203740 1 Processor -- start 2024-09-11T00:22:01.289 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:01.287+0000 7f2173203740 1 -- start start 2024-09-11T00:22:01.290 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:01.287+0000 7f2173203740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b758a53790 0x55b758a55b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:22:01.290 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:01.287+0000 7f2173203740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b758a560c0 0x55b758a584b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:22:01.290 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:01.287+0000 7f2173203740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b758a58b00 0x55b758a5afc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:22:01.291 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:01.287+0000 7f2173203740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55b7589c66b0 con 0x55b758a58b00 2024-09-11T00:22:01.291 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:01.287+0000 7f2173203740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55b7589b5ae0 con 0x55b758a560c0 2024-09-11T00:22:01.291 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:01.287+0000 7f2173203740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55b7589b98f0 con 0x55b758a53790 2024-09-11T00:22:01.291 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:01.287+0000 7f216e400640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b758a53790 0x55b758a55b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:22:01.291 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:01.287+0000 7f216e400640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b758a53790 0x55b758a55b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.12:38852/0 (socket says 172.21.15.12:38852) 2024-09-11T00:22:01.291 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:01.287+0000 7f216e400640 1 -- 172.21.15.12:0/3294502562 learned_addr learned my addr 172.21.15.12:0/3294502562 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:22:01.291 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:01.287+0000 7f216da00640 1 --2- 172.21.15.12:0/3294502562 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b758a560c0 0x55b758a584b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:22:01.291 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:01.288+0000 7f216e400640 1 -- 172.21.15.12:0/3294502562 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b758a560c0 msgr2=0x55b758a584b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:22:01.291 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:01.288+0000 7f216e400640 1 --2- 172.21.15.12:0/3294502562 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b758a560c0 0x55b758a584b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:01.291 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:01.288+0000 7f216e400640 1 -- 172.21.15.12:0/3294502562 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b758a58b00 msgr2=0x55b758a5afc0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-11T00:22:01.291 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:01.288+0000 7f216e400640 1 --2- 172.21.15.12:0/3294502562 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b758a58b00 0x55b758a5afc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:01.291 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:01.288+0000 7f216e400640 1 -- 172.21.15.12:0/3294502562 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b758982780 con 0x55b758a53790 2024-09-11T00:22:01.292 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:01.288+0000 7f2166000640 1 -- 172.21.15.12:0/22896465 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7f2154004e90 con 0x7f2168103ac0 2024-09-11T00:22:01.292 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:01.288+0000 7f216e400640 1 --2- 172.21.15.12:0/3294502562 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b758a53790 0x55b758a55b80 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f2160009fd0 tx=0x7f21600118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9232450afbdef7af server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:22:01.292 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:01.288+0000 7f215aa00640 1 -- 172.21.15.12:0/3294502562 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2160004d20 con 0x55b758a53790 2024-09-11T00:22:01.292 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:01.288+0000 7f215aa00640 1 -- 172.21.15.12:0/3294502562 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2160004050 con 0x55b758a53790 2024-09-11T00:22:01.292 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:01.288+0000 7f215aa00640 1 -- 172.21.15.12:0/3294502562 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2160005260 con 0x55b758a53790 2024-09-11T00:22:01.295 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:01.289+0000 7f2173203740 1 -- 172.21.15.12:0/3294502562 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b758a53790 msgr2=0x55b758a55b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:22:01.296 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:01.289+0000 7f2173203740 1 --2- 172.21.15.12:0/3294502562 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b758a53790 0x55b758a55b80 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f2160009fd0 tx=0x7f21600118d0 comp rx=0 tx=0).stop 2024-09-11T00:22:01.296 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:01.289+0000 7f2173203740 1 -- 172.21.15.12:0/3294502562 shutdown_connections 2024-09-11T00:22:01.296 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:01.289+0000 7f2173203740 1 --2- 172.21.15.12:0/3294502562 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b758a58b00 0x55b758a5afc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:01.296 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:01.289+0000 7f2173203740 1 --2- 172.21.15.12:0/3294502562 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b758a560c0 0x55b758a584b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:01.296 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:01.289+0000 7f2173203740 1 --2- 172.21.15.12:0/3294502562 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b758a53790 0x55b758a55b80 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:01.296 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:01.289+0000 7f2173203740 1 -- 172.21.15.12:0/3294502562 >> 172.21.15.12:0/3294502562 conn(0x55b758a4e8e0 msgr2=0x55b758a4f670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:22:01.296 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:01.289+0000 7f2173203740 1 -- 172.21.15.12:0/3294502562 shutdown_connections 2024-09-11T00:22:01.296 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:01.289+0000 7f2173203740 1 -- 172.21.15.12:0/3294502562 wait complete. 2024-09-11T00:22:01.296 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:01.290+0000 7f2173203740 1 -- start start 2024-09-11T00:22:01.296 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:01.290+0000 7f2166000640 1 --2- 172.21.15.12:0/22896465 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f21500b51f0 0x7f21500b76b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:22:01.297 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:01.290+0000 7f2173203740 1 Processor -- start 2024-09-11T00:22:01.297 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:01.290+0000 7f2166000640 1 -- 172.21.15.12:0/22896465 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9208+0+0 (secure 0 0 0) 0x7f21540e63e0 con 0x7f2168103ac0 2024-09-11T00:22:01.297 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:01.290+0000 7f216e400640 1 --2- 172.21.15.12:0/22896465 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f21500b51f0 0x7f21500b76b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:22:01.297 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:01.290+0000 7f2173203740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b758a53790 0x55b758a5ceb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:22:01.297 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:01.290+0000 7f2173203740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b758a560c0 0x55b758a5d550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:22:01.297 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:01.290+0000 7f2173203740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b758a58b00 0x55b758a5e120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:22:01.297 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:01.290+0000 7f2173203740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55b7588edab0 con 0x55b758a53790 2024-09-11T00:22:01.297 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:01.290+0000 7f2173203740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55b75895c570 con 0x55b758a58b00 2024-09-11T00:22:01.297 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:01.290+0000 7f2173203740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55b758a5bae0 con 0x55b758a560c0 2024-09-11T00:22:01.297 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:01.290+0000 7f2173203740 10 client.0 _ll_register_callbacks cb 0x55b75896b100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:22:01.297 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:01.291+0000 7f216d000640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b758a560c0 0x55b758a5d550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:22:01.297 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:01.291+0000 7f216d000640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b758a560c0 0x55b758a5d550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.12:38868/0 (socket says 172.21.15.12:38868) 2024-09-11T00:22:01.298 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:01.291+0000 7f216e400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b758a58b00 0x55b758a5e120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:22:01.298 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:01.291+0000 7f216d000640 1 -- 172.21.15.12:0/3838691131 learned_addr learned my addr 172.21.15.12:0/3838691131 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:22:01.298 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:01.291+0000 7f216da00640 1 --2- 172.21.15.12:0/3838691131 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b758a53790 0x55b758a5ceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:22:01.298 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:01.291+0000 7f216e400640 1 --2- 172.21.15.12:0/22896465 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f21500b51f0 0x7f21500b76b0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f2160011dd0 tx=0x7f2160005fd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:22:01.298 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:01.291+0000 7f216d000640 1 -- 172.21.15.12:0/3838691131 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b758a58b00 msgr2=0x55b758a5e120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:22:01.298 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:01.291+0000 7f216d000640 1 --2- 172.21.15.12:0/3838691131 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b758a58b00 0x55b758a5e120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:01.298 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:01.291+0000 7f216d000640 1 -- 172.21.15.12:0/3838691131 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b758a53790 msgr2=0x55b758a5ceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:22:01.298 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:01.291+0000 7f216d000640 1 --2- 172.21.15.12:0/3838691131 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b758a53790 0x55b758a5ceb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:01.298 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:01.291+0000 7f216d000640 1 -- 172.21.15.12:0/3838691131 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b758717790 con 0x55b758a560c0 2024-09-11T00:22:01.298 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:01.291+0000 7f216d000640 1 --2- 172.21.15.12:0/3838691131 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b758a560c0 0x55b758a5d550 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f2154009380 tx=0x7f21540ac600 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7a51106556d4738d server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:22:01.298 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:01.291+0000 7f214f400640 10 client.0 ms_handle_connect on v2:172.21.15.178:3300/0 2024-09-11T00:22:01.298 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:01.292+0000 7f214f400640 1 -- 172.21.15.12:0/3838691131 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21540b1020 con 0x55b758a560c0 2024-09-11T00:22:01.299 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:01.292+0000 7f2173203740 1 -- 172.21.15.12:0/3838691131 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2160004050 con 0x55b758a560c0 2024-09-11T00:22:01.299 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:01.292+0000 7f214f400640 1 -- 172.21.15.12:0/3838691131 <== mon.1 v2:172.21.15.178:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9208+0+0 (secure 0 0 0) 0x7f21540b5870 con 0x55b758a560c0 2024-09-11T00:22:01.299 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:01.292+0000 7f214f400640 1 -- 172.21.15.12:0/3838691131 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7f21540ace70 con 0x55b758a560c0 2024-09-11T00:22:01.299 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:01.292+0000 7f2173203740 10 client.27148 fetch_fsmap learned FSMap version 54 2024-09-11T00:22:01.299 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:01.292+0000 7f2173203740 1 -- 172.21.15.12:0/3838691131 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) -- 0x55b758790620 con 0x55b758a560c0 2024-09-11T00:22:01.299 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:01.292+0000 7f214f400640 1 -- 172.21.15.12:0/3838691131 <== mon.1 v2:172.21.15.178:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2160004050 con 0x55b758a560c0 2024-09-11T00:22:01.299 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:01.292+0000 7f214f400640 1 -- 172.21.15.12:0/3838691131 <== mon.1 v2:172.21.15.178:3300/0 5 ==== fsmap(e 54) ==== 8422+0+0 (secure 0 0 0) 0x7f21540c3070 con 0x55b758a560c0 2024-09-11T00:22:01.299 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:01.292+0000 7f2173203740 10 client.27148 fetch_fsmap finished waiting for FSMap version 54 2024-09-11T00:22:01.299 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:01.292+0000 7f2173203740 10 client.27148 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T00:22:01.299 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:01.292+0000 7f2173203740 20 client.27148 populate_metadata read hostname 'smithi012' 2024-09-11T00:22:01.299 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:01.292+0000 7f2173203740 1 --2- 172.21.15.12:0/3838691131 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55b758a687c0 0x55b758a6abe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:22:01.299 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:01.292+0000 7f2173203740 4 client.27148 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T00:22:01.300 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:01.292+0000 7f2173203740 1 -- 172.21.15.12:0/3838691131 --> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b7589c8600 con 0x55b758a687c0 2024-09-11T00:22:01.300 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:01.293+0000 7f216da00640 1 --2- 172.21.15.12:0/3838691131 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55b758a687c0 0x55b758a6abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:22:01.300 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:01.293+0000 7f216da00640 1 --2- 172.21.15.12:0/3838691131 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55b758a687c0 0x55b758a6abe0 crc :-1 s=READY pgs=219 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d60872afb124a2e9 server_cookie=177bbba09ac7b7a2 in_seq=0 out_seq=0 2024-09-11T00:22:01.300 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:01.293+0000 7f214f400640 10 client.27148 ms_handle_connect on v2:172.21.15.165:6838/501312984 2024-09-11T00:22:01.301 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:01.294+0000 7f214f400640 1 -- 172.21.15.12:0/3838691131 <== mds.0 v2:172.21.15.165:6838/501312984 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55b7589c8600 con 0x55b758a687c0 2024-09-11T00:22:01.301 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:01.294+0000 7f214f400640 10 client.27148 handle_command_reply: tid=0 2024-09-11T00:22:01.301 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:01.294+0000 7f2173203740 1 client.27148 shutdown 2024-09-11T00:22:01.301 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:01.294+0000 7f2173203740 1 -- 172.21.15.12:0/3838691131 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b758a560c0 msgr2=0x55b758a5d550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:22:01.302 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:01.294+0000 7f2173203740 1 --2- 172.21.15.12:0/3838691131 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b758a560c0 0x55b758a5d550 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f2154009380 tx=0x7f21540ac600 comp rx=0 tx=0).stop 2024-09-11T00:22:01.302 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:01.295+0000 7f2173203740 1 -- 172.21.15.12:0/3838691131 shutdown_connections 2024-09-11T00:22:01.302 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:01.295+0000 7f2173203740 1 --2- 172.21.15.12:0/3838691131 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55b758a687c0 0x55b758a6abe0 crc :-1 s=READY pgs=219 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:01.302 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:01.295+0000 7f2173203740 1 --2- 172.21.15.12:0/3838691131 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b758a58b00 0x55b758a5e120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:01.302 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:01.295+0000 7f2173203740 1 --2- 172.21.15.12:0/3838691131 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b758a560c0 0x55b758a5d550 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:01.302 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:01.295+0000 7f2173203740 1 --2- 172.21.15.12:0/3838691131 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b758a53790 0x55b758a5ceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:01.302 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:01.295+0000 7f214f400640 0 client.27148 ms_handle_reset on v2:172.21.15.165:6838/501312984 2024-09-11T00:22:01.302 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:01.295+0000 7f2173203740 1 -- 172.21.15.12:0/3838691131 >> 172.21.15.12:0/3838691131 conn(0x55b758a4e8e0 msgr2=0x55b758a4da60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:22:01.302 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:01.295+0000 7f2173203740 1 -- 172.21.15.12:0/3838691131 shutdown_connections 2024-09-11T00:22:01.302 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:01.295+0000 7f2173203740 1 -- 172.21.15.12:0/3838691131 wait complete. 2024-09-11T00:22:01.302 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:01.295+0000 7f2173203740 20 client.27148 trim_cache size 0 max 16384 2024-09-11T00:22:01.314 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:01.311+0000 7f2173203740 1 Processor -- start 2024-09-11T00:22:01.314 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:01.311+0000 7f2173203740 1 -- start start 2024-09-11T00:22:01.315 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:01.311+0000 7f2173203740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b758a53790 0x55b758a49960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:22:01.315 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:01.311+0000 7f2173203740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b758a560c0 0x55b758a51930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:22:01.315 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:01.311+0000 7f2173203740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b758a58b00 0x55b758a51e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:22:01.315 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:01.311+0000 7f2173203740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f21681ad9b0 con 0x55b758a58b00 2024-09-11T00:22:01.315 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:01.311+0000 7f2173203740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f21681af6d0 con 0x55b758a53790 2024-09-11T00:22:01.315 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:01.311+0000 7f2173203740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f21681ae0e0 con 0x55b758a560c0 2024-09-11T00:22:01.315 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:01.311+0000 7f216e400640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b758a58b00 0x55b758a51e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:22:01.315 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:01.311+0000 7f216e400640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b758a58b00 0x55b758a51e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:34666/0 (socket says 172.21.15.12:34666) 2024-09-11T00:22:01.315 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:01.311+0000 7f216e400640 1 -- 172.21.15.12:0/1244800784 learned_addr learned my addr 172.21.15.12:0/1244800784 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:22:01.315 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:01.311+0000 7f216da00640 1 --2- 172.21.15.12:0/1244800784 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b758a53790 0x55b758a49960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:22:01.315 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:01.311+0000 7f216d000640 1 --2- 172.21.15.12:0/1244800784 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b758a560c0 0x55b758a51930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:22:01.316 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:01.312+0000 7f216e400640 1 -- 172.21.15.12:0/1244800784 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b758a560c0 msgr2=0x55b758a51930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:22:01.316 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:01.312+0000 7f216e400640 1 --2- 172.21.15.12:0/1244800784 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b758a560c0 0x55b758a51930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:01.316 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:01.312+0000 7f216e400640 1 -- 172.21.15.12:0/1244800784 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b758a53790 msgr2=0x55b758a49960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:22:01.316 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:01.312+0000 7f216e400640 1 --2- 172.21.15.12:0/1244800784 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b758a53790 0x55b758a49960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:01.316 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:01.312+0000 7f216e400640 1 -- 172.21.15.12:0/1244800784 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b758a58820 con 0x55b758a58b00 2024-09-11T00:22:01.316 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:01.312+0000 7f216da00640 1 --2- 172.21.15.12:0/1244800784 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b758a53790 0x55b758a49960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:22:01.316 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:01.312+0000 7f216e400640 1 --2- 172.21.15.12:0/1244800784 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b758a58b00 0x55b758a51e70 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f2160005c10 tx=0x7f2160021000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d1937d1b0dbacbd server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:22:01.316 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:01.312+0000 7f215a000640 1 -- 172.21.15.12:0/1244800784 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f216002b040 con 0x55b758a58b00 2024-09-11T00:22:01.316 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:01.312+0000 7f215a000640 1 -- 172.21.15.12:0/1244800784 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2160009b30 con 0x55b758a58b00 2024-09-11T00:22:01.316 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:01.312+0000 7f215a000640 1 -- 172.21.15.12:0/1244800784 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f216000f4e0 con 0x55b758a58b00 2024-09-11T00:22:01.319 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:01.313+0000 7f2173203740 1 -- 172.21.15.12:0/1244800784 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b758a58b00 msgr2=0x55b758a51e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:22:01.319 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:01.313+0000 7f2173203740 1 --2- 172.21.15.12:0/1244800784 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b758a58b00 0x55b758a51e70 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f2160005c10 tx=0x7f2160021000 comp rx=0 tx=0).stop 2024-09-11T00:22:01.319 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:01.313+0000 7f2173203740 1 -- 172.21.15.12:0/1244800784 shutdown_connections 2024-09-11T00:22:01.319 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:01.313+0000 7f2173203740 1 --2- 172.21.15.12:0/1244800784 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b758a58b00 0x55b758a51e70 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:01.319 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:01.313+0000 7f2173203740 1 --2- 172.21.15.12:0/1244800784 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b758a560c0 0x55b758a51930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:01.319 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:01.313+0000 7f2173203740 1 --2- 172.21.15.12:0/1244800784 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b758a53790 0x55b758a49960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:01.320 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:01.313+0000 7f2173203740 1 -- 172.21.15.12:0/1244800784 >> 172.21.15.12:0/1244800784 conn(0x55b758a4e8e0 msgr2=0x55b758a52c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:22:01.320 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:01.313+0000 7f2173203740 1 -- 172.21.15.12:0/1244800784 shutdown_connections 2024-09-11T00:22:01.320 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:01.313+0000 7f2173203740 1 -- 172.21.15.12:0/1244800784 wait complete. 2024-09-11T00:22:01.320 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:01.314+0000 7f2173203740 1 -- start start 2024-09-11T00:22:01.320 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:01.314+0000 7f2173203740 1 Processor -- start 2024-09-11T00:22:01.320 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:01.314+0000 7f2173203740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b758a53790 0x55b758a63c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:22:01.320 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:01.314+0000 7f2173203740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b758a560c0 0x55b758a64170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:22:01.320 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:01.314+0000 7f2173203740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b758a58b00 0x55b758a62750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:22:01.320 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:01.314+0000 7f2173203740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f21681af000 con 0x55b758a53790 2024-09-11T00:22:01.320 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:01.314+0000 7f2173203740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f21681ae870 con 0x55b758a560c0 2024-09-11T00:22:01.320 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:01.314+0000 7f2173203740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f21681ab990 con 0x55b758a58b00 2024-09-11T00:22:01.320 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:01.314+0000 7f2173203740 10 client.0 _ll_register_callbacks cb 0x7f2154009720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:22:01.321 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:01.315+0000 7f216da00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b758a53790 0x55b758a63c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:22:01.321 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:01.315+0000 7f216d000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b758a560c0 0x55b758a64170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:22:01.321 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:01.315+0000 7f216d000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b758a560c0 0x55b758a64170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.12:36294/0 (socket says 172.21.15.12:36294) 2024-09-11T00:22:01.321 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:01.315+0000 7f216da00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b758a53790 0x55b758a63c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:34682/0 (socket says 172.21.15.12:34682) 2024-09-11T00:22:01.321 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:01.315+0000 7f216d000640 1 -- 172.21.15.12:0/3768077680 learned_addr learned my addr 172.21.15.12:0/3768077680 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:22:01.321 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:01.315+0000 7f216e400640 1 --2- 172.21.15.12:0/3768077680 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b758a58b00 0x55b758a62750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:22:01.321 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:01.315+0000 7f216d000640 1 -- 172.21.15.12:0/3768077680 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b758a58b00 msgr2=0x55b758a62750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:22:01.321 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:01.315+0000 7f216d000640 1 --2- 172.21.15.12:0/3768077680 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b758a58b00 0x55b758a62750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:01.321 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:01.315+0000 7f216d000640 1 -- 172.21.15.12:0/3768077680 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b758a53790 msgr2=0x55b758a63c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:22:01.321 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:01.315+0000 7f216d000640 1 --2- 172.21.15.12:0/3768077680 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b758a53790 0x55b758a63c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:01.321 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:01.315+0000 7f216d000640 1 -- 172.21.15.12:0/3768077680 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b758a62c90 con 0x55b758a560c0 2024-09-11T00:22:01.321 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:01.315+0000 7f216e400640 1 --2- 172.21.15.12:0/3768077680 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b758a58b00 0x55b758a62750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T00:22:01.322 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:01.315+0000 7f216d000640 1 --2- 172.21.15.12:0/3768077680 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b758a560c0 0x55b758a64170 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f21540a99e0 tx=0x7f21540a8510 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bb093d47cb0a5bdb server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:22:01.322 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:01.315+0000 7f214f400640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-09-11T00:22:01.322 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:01.316+0000 7f214f400640 1 -- 172.21.15.12:0/3768077680 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21540abee0 con 0x55b758a560c0 2024-09-11T00:22:01.322 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:01.316+0000 7f214f400640 1 -- 172.21.15.12:0/3768077680 <== mon.2 v2:172.21.15.165:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9208+0+0 (secure 0 0 0) 0x7f21540abbd0 con 0x55b758a560c0 2024-09-11T00:22:01.322 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:01.316+0000 7f2173203740 1 -- 172.21.15.12:0/3768077680 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2160009b30 con 0x55b758a560c0 2024-09-11T00:22:01.322 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:01.316+0000 7f214f400640 1 -- 172.21.15.12:0/3768077680 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7f21540a8610 con 0x55b758a560c0 2024-09-11T00:22:01.322 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:01.316+0000 7f2173203740 10 client.27188 fetch_fsmap learned FSMap version 54 2024-09-11T00:22:01.322 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:01.316+0000 7f2173203740 1 -- 172.21.15.12:0/3768077680 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) -- 0x55b758ad5dd0 con 0x55b758a560c0 2024-09-11T00:22:01.322 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:01.316+0000 7f214f400640 1 -- 172.21.15.12:0/3768077680 <== mon.2 v2:172.21.15.165:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2160009b30 con 0x55b758a560c0 2024-09-11T00:22:01.322 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:01.316+0000 7f214f400640 1 -- 172.21.15.12:0/3768077680 <== mon.2 v2:172.21.15.165:3300/0 5 ==== fsmap(e 54) ==== 8422+0+0 (secure 0 0 0) 0x7f21540c3030 con 0x55b758a560c0 2024-09-11T00:22:01.322 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:01.317+0000 7f2173203740 10 client.27188 fetch_fsmap finished waiting for FSMap version 54 2024-09-11T00:22:01.322 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:01.317+0000 7f2173203740 10 client.27188 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T00:22:01.323 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:01.317+0000 7f2173203740 20 client.27188 populate_metadata read hostname 'smithi012' 2024-09-11T00:22:01.323 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:01.317+0000 7f2173203740 1 --2- 172.21.15.12:0/3768077680 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55b758ad6820 0x55b758ad8c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:22:01.323 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:01.317+0000 7f2173203740 4 client.27188 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T00:22:01.323 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:01.317+0000 7f2173203740 1 -- 172.21.15.12:0/3768077680 --> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55b7589c40b0 con 0x55b758ad6820 2024-09-11T00:22:01.323 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:01.317+0000 7f216da00640 1 --2- 172.21.15.12:0/3768077680 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55b758ad6820 0x55b758ad8c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:22:01.323 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:01.317+0000 7f216da00640 1 --2- 172.21.15.12:0/3768077680 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55b758ad6820 0x55b758ad8c40 crc :-1 s=READY pgs=220 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ebc297f6ba06d4b8 server_cookie=5bc7b6cb3a465e11 in_seq=0 out_seq=0 2024-09-11T00:22:01.323 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:01.317+0000 7f214f400640 10 client.27188 ms_handle_connect on v2:172.21.15.165:6838/501312984 2024-09-11T00:22:01.367 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:01.364+0000 7f214f400640 1 -- 172.21.15.12:0/3768077680 <== mds.0 v2:172.21.15.165:6838/501312984 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55b7589c40b0 con 0x55b758ad6820 2024-09-11T00:22:01.367 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:01.364+0000 7f214f400640 10 client.27188 handle_command_reply: tid=0 2024-09-11T00:22:01.367 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:01.364+0000 7f2173203740 1 client.27188 shutdown 2024-09-11T00:22:01.367 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:01.364+0000 7f2173203740 1 -- 172.21.15.12:0/3768077680 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b758a560c0 msgr2=0x55b758a64170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:22:01.367 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:01.364+0000 7f2173203740 1 --2- 172.21.15.12:0/3768077680 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b758a560c0 0x55b758a64170 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f21540a99e0 tx=0x7f21540a8510 comp rx=0 tx=0).stop 2024-09-11T00:22:01.367 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:01.364+0000 7f2173203740 1 -- 172.21.15.12:0/3768077680 shutdown_connections 2024-09-11T00:22:01.367 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:01.364+0000 7f2173203740 1 --2- 172.21.15.12:0/3768077680 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55b758ad6820 0x55b758ad8c40 crc :-1 s=READY pgs=220 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:01.367 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:01.364+0000 7f2173203740 1 --2- 172.21.15.12:0/3768077680 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b758a58b00 0x55b758a62750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:01.367 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:01.364+0000 7f2173203740 1 --2- 172.21.15.12:0/3768077680 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b758a560c0 0x55b758a64170 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:01.367 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:01.364+0000 7f2173203740 1 --2- 172.21.15.12:0/3768077680 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b758a53790 0x55b758a63c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:01.368 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:01.364+0000 7f214f400640 0 client.27188 ms_handle_reset on v2:172.21.15.165:6838/501312984 2024-09-11T00:22:01.368 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:01.364+0000 7f2173203740 1 -- 172.21.15.12:0/3768077680 >> 172.21.15.12:0/3768077680 conn(0x55b758a4e8e0 msgr2=0x55b758a52850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:22:01.368 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:01.364+0000 7f2173203740 1 -- 172.21.15.12:0/3768077680 shutdown_connections 2024-09-11T00:22:01.368 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:01.364+0000 7f2173203740 1 -- 172.21.15.12:0/3768077680 wait complete. 2024-09-11T00:22:01.368 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:01.364+0000 7f2173203740 20 client.27188 trim_cache size 0 max 16384 2024-09-11T00:22:01.368 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:{ 2024-09-11T00:22:01.368 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "message": "", 2024-09-11T00:22:01.369 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "return_code": 0 2024-09-11T00:22:01.369 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:} 2024-09-11T00:22:01.369 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:01.365+0000 7f2170400640 1 -- 172.21.15.12:0/22896465 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f21500b51f0 msgr2=0x7f21500b76b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:22:01.369 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:01.365+0000 7f2170400640 1 --2- 172.21.15.12:0/22896465 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f21500b51f0 0x7f21500b76b0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f2160011dd0 tx=0x7f2160005fd0 comp rx=0 tx=0).stop 2024-09-11T00:22:01.370 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:01.365+0000 7f2170400640 1 -- 172.21.15.12:0/22896465 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2168103ac0 msgr2=0x7f2168075540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:22:01.370 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:01.365+0000 7f2170400640 1 --2- 172.21.15.12:0/22896465 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2168103ac0 0x7f2168075540 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f215400e9f0 tx=0x7f215400edf0 comp rx=0 tx=0).stop 2024-09-11T00:22:01.370 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:01.365+0000 7f2170400640 1 -- 172.21.15.12:0/22896465 shutdown_connections 2024-09-11T00:22:01.370 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:01.365+0000 7f2170400640 1 --2- 172.21.15.12:0/22896465 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f21500b51f0 0x7f21500b76b0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:01.370 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:01.365+0000 7f2170400640 1 --2- 172.21.15.12:0/22896465 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2168104480 0x7f2168075a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:01.370 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:01.365+0000 7f2170400640 1 --2- 172.21.15.12:0/22896465 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2168103ac0 0x7f2168075540 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:01.370 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:01.365+0000 7f2170400640 1 --2- 172.21.15.12:0/22896465 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2168102970 0x7f2168076ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:01.370 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:01.365+0000 7f2170400640 1 -- 172.21.15.12:0/22896465 >> 172.21.15.12:0/22896465 conn(0x7f21680fe250 msgr2=0x7f2168106f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:22:01.370 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:01.365+0000 7f2170400640 1 -- 172.21.15.12:0/22896465 shutdown_connections 2024-09-11T00:22:01.370 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:01.365+0000 7f2170400640 1 -- 172.21.15.12:0/22896465 wait complete. 2024-09-11T00:22:01.782 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:22:01 smithi165 ceph-mon[35356]: pgmap v847: 129 pgs: 129 active+clean; 396 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 835 KiB/s rd, 4.6 MiB/s wr, 258 op/s 2024-09-11T00:22:01.813 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:22:01 smithi012 ceph-mon[30333]: pgmap v847: 129 pgs: 129 active+clean; 396 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 835 KiB/s rd, 4.6 MiB/s wr, 258 op/s 2024-09-11T00:22:01.831 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:22:01 smithi178 ceph-mon[35919]: pgmap v847: 129 pgs: 129 active+clean; 396 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 835 KiB/s rd, 4.6 MiB/s wr, 258 op/s 2024-09-11T00:22:02.164 INFO:tasks.workunit.client.0.smithi012.stdout: 10 33135 16.57 MB/sec warmup 118 sec latency 16978.429 ms 2024-09-11T00:22:03.164 INFO:tasks.workunit.client.0.smithi012.stdout: 10 33135 16.43 MB/sec warmup 119 sec latency 17978.573 ms 2024-09-11T00:22:03.730 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid ff3ad74a-6fcf-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T00:22:03.782 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:22:03 smithi165 ceph-mon[35356]: pgmap v848: 129 pgs: 129 active+clean; 394 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 188 KiB/s rd, 4.3 MiB/s wr, 187 op/s 2024-09-11T00:22:03.782 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:22:03 smithi165 ceph-mon[35356]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T00:22:03.786 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:22:03 smithi012 ceph-mon[30333]: pgmap v848: 129 pgs: 129 active+clean; 394 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 188 KiB/s rd, 4.3 MiB/s wr, 187 op/s 2024-09-11T00:22:03.786 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:22:03 smithi012 ceph-mon[30333]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T00:22:03.831 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:22:03 smithi178 ceph-mon[35919]: pgmap v848: 129 pgs: 129 active+clean; 394 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 188 KiB/s rd, 4.3 MiB/s wr, 187 op/s 2024-09-11T00:22:03.831 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:22:03 smithi178 ceph-mon[35919]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T00:22:04.129 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:22:04.710 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:22:04.684+0000 7f66c6a00640 1 -- 172.21.15.12:0/3795943832 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f66c0102dc0 msgr2=0x7f66c01031a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:22:04.710 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:22:04.684+0000 7f66c6a00640 1 --2- 172.21.15.12:0/3795943832 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f66c0102dc0 0x7f66c01031a0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f66b0009900 tx=0x7f66b002a3f0 comp rx=0 tx=0).stop 2024-09-11T00:22:04.710 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:22:04.684+0000 7f66c6a00640 1 -- 172.21.15.12:0/3795943832 shutdown_connections 2024-09-11T00:22:04.710 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:22:04.684+0000 7f66c6a00640 1 --2- 172.21.15.12:0/3795943832 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f66c0109150 0x7f66c010f950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:04.710 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:22:04.684+0000 7f66c6a00640 1 --2- 172.21.15.12:0/3795943832 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f66c01087d0 0x7f66c0108c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:04.710 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:22:04.684+0000 7f66c6a00640 1 --2- 172.21.15.12:0/3795943832 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f66c0102dc0 0x7f66c01031a0 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:04.710 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:22:04.684+0000 7f66c6a00640 1 -- 172.21.15.12:0/3795943832 >> 172.21.15.12:0/3795943832 conn(0x7f66c00fe830 msgr2=0x7f66c0100c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:22:04.711 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:22:04.684+0000 7f66c6a00640 1 -- 172.21.15.12:0/3795943832 shutdown_connections 2024-09-11T00:22:04.711 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:22:04.684+0000 7f66c6a00640 1 -- 172.21.15.12:0/3795943832 wait complete. 2024-09-11T00:22:04.711 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:22:04.685+0000 7f66c6a00640 1 Processor -- start 2024-09-11T00:22:04.711 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:22:04.686+0000 7f66c6a00640 1 -- start start 2024-09-11T00:22:04.711 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:22:04.686+0000 7f66c6a00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f66c0102dc0 0x7f66c01992d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:22:04.711 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:22:04.686+0000 7f66c6a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f66c01087d0 0x7f66c0199810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:22:04.711 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:22:04.686+0000 7f66c6a00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f66c0109150 0x7f66c019ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:22:04.711 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:22:04.686+0000 7f66c6a00640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f66c0112e20 con 0x7f66c0102dc0 2024-09-11T00:22:04.711 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:22:04.686+0000 7f66c6a00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f66c0112ca0 con 0x7f66c01087d0 2024-09-11T00:22:04.711 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:22:04.686+0000 7f66c6a00640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f66c0112fa0 con 0x7f66c0109150 2024-09-11T00:22:04.711 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:22:04.687+0000 7f66bfe00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f66c0102dc0 0x7f66c01992d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:22:04.711 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:22:04.687+0000 7f66bfe00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f66c0102dc0 0x7f66c01992d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:34698/0 (socket says 172.21.15.12:34698) 2024-09-11T00:22:04.711 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:22:04.687+0000 7f66c4a00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f66c0109150 0x7f66c019ebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:22:04.712 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:22:04.687+0000 7f66bfe00640 1 -- 172.21.15.12:0/1982925899 learned_addr learned my addr 172.21.15.12:0/1982925899 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:22:04.712 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:22:04.687+0000 7f66bf400640 1 --2- 172.21.15.12:0/1982925899 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f66c01087d0 0x7f66c0199810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:22:04.712 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:22:04.687+0000 7f66bfe00640 1 -- 172.21.15.12:0/1982925899 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f66c0109150 msgr2=0x7f66c019ebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:22:04.712 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:22:04.687+0000 7f66bfe00640 1 --2- 172.21.15.12:0/1982925899 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f66c0109150 0x7f66c019ebb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:04.712 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:22:04.687+0000 7f66bfe00640 1 -- 172.21.15.12:0/1982925899 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f66c01087d0 msgr2=0x7f66c0199810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:22:04.712 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:22:04.687+0000 7f66bfe00640 1 --2- 172.21.15.12:0/1982925899 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f66c01087d0 0x7f66c0199810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:04.712 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:22:04.687+0000 7f66bfe00640 1 -- 172.21.15.12:0/1982925899 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66c019f380 con 0x7f66c0102dc0 2024-09-11T00:22:04.712 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:22:04.687+0000 7f66c4a00640 1 --2- 172.21.15.12:0/1982925899 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f66c0109150 0x7f66c019ebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:22:04.712 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:22:04.688+0000 7f66bfe00640 1 --2- 172.21.15.12:0/1982925899 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f66c0102dc0 0x7f66c01992d0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f66b0005ab0 tx=0x7f66b0003c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:22:04.712 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:22:04.688+0000 7f66bcc00640 1 -- 172.21.15.12:0/1982925899 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66b003f070 con 0x7f66c0102dc0 2024-09-11T00:22:04.712 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:22:04.688+0000 7f66bcc00640 1 -- 172.21.15.12:0/1982925899 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f66b0031d60 con 0x7f66c0102dc0 2024-09-11T00:22:04.713 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:22:04.688+0000 7f66c6a00640 1 -- 172.21.15.12:0/1982925899 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f66c019f610 con 0x7f66c0102dc0 2024-09-11T00:22:04.713 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:22:04.688+0000 7f66bcc00640 1 -- 172.21.15.12:0/1982925899 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66b0035030 con 0x7f66c0102dc0 2024-09-11T00:22:04.713 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:22:04.688+0000 7f66c6a00640 1 -- 172.21.15.12:0/1982925899 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66c01a5d30 con 0x7f66c0102dc0 2024-09-11T00:22:04.713 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:22:04.689+0000 7f66c6a00640 1 -- 172.21.15.12:0/1982925899 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f66c01031a0 con 0x7f66c0102dc0 2024-09-11T00:22:04.713 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:22:04.693+0000 7f66bcc00640 1 -- 172.21.15.12:0/1982925899 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7f66b004c060 con 0x7f66c0102dc0 2024-09-11T00:22:04.713 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:22:04.694+0000 7f66bcc00640 1 --2- 172.21.15.12:0/1982925899 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f66a40b52e0 0x7f66a40b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:22:04.713 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:22:04.694+0000 7f66bcc00640 1 -- 172.21.15.12:0/1982925899 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9208+0+0 (secure 0 0 0) 0x7f66b01069f0 con 0x7f66c0102dc0 2024-09-11T00:22:04.713 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:22:04.695+0000 7f66bf400640 1 --2- 172.21.15.12:0/1982925899 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f66a40b52e0 0x7f66a40b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:22:04.713 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:22:04.695+0000 7f66bf400640 1 --2- 172.21.15.12:0/1982925899 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f66a40b52e0 0x7f66a40b77a0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f66c0107e30 tx=0x7f66ac00b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:22:04.713 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:22:04.696+0000 7f66bcc00640 1 -- 172.21.15.12:0/1982925899 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f66b00cce30 con 0x7f66c0102dc0 2024-09-11T00:22:04.931 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:22:04.929+0000 7f66c6a00640 1 -- 172.21.15.12:0/1982925899 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f66c010af30 con 0x7f66c0102dc0 2024-09-11T00:22:04.932 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:22:04.930+0000 7f66bcc00640 1 -- 172.21.15.12:0/1982925899 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 55 v55) ==== 94+0+11984 (secure 0 0 0) 0x7f66b00d1860 con 0x7f66c0102dc0 2024-09-11T00:22:04.932 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-11T00:22:04.933 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":55,"btime":"2024-09-11T00:22:02:517467+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14577,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6835/447171591","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":447171591},{"type":"v1","addr":"172.21.15.12:6835","nonce":447171591}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14589,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/901486040","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":901486040},{"type":"v1","addr":"172.21.15.12:6837","nonce":901486040}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":55,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T00:04:04.739881+0000","modified":"2024-09-11T00:22:01.725251+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24674,"mds_1":24643,"mds_2":24638,"mds_3":24637,"mds_4":24614},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14631":{"gid":14631,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.12:6841/3069919437","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6840","nonce":3069919437},{"type":"v1","addr":"172.21.15.12:6841","nonce":3069919437}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24562":{"gid":24562,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6833/1141636112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":1141636112},{"type":"v1","addr":"172.21.15.178:6833","nonce":1141636112}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24581":{"gid":24581,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6833/3111980970","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6832","nonce":3111980970},{"type":"v1","addr":"172.21.15.165:6833","nonce":3111980970}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24607":{"gid":24607,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6835/4084868287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":4084868287},{"type":"v1","addr":"172.21.15.178:6835","nonce":4084868287}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24614":{"gid":24614,"name":"e","rank":4,"incarnation":26,"state":"up:active","state_seq":21,"addr":"172.21.15.165:6835/2425395274","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":2425395274},{"type":"v1","addr":"172.21.15.165:6835","nonce":2425395274}]},"join_fscid":-1,"export_targets":[0],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24637":{"gid":24637,"name":"i","rank":3,"incarnation":24,"state":"up:active","state_seq":18,"addr":"172.21.15.178:6839/70628097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6838","nonce":70628097},{"type":"v1","addr":"172.21.15.178:6839","nonce":70628097}]},"join_fscid":-1,"export_targets":[0,1],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24638":{"gid":24638,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":18,"addr":"172.21.15.165:6837/6215088","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6836","nonce":6215088},{"type":"v1","addr":"172.21.15.165:6837","nonce":6215088}]},"join_fscid":-1,"export_targets":[0],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24643":{"gid":24643,"name":"g","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.12:6839/3124093009","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6838","nonce":3124093009},{"type":"v1","addr":"172.21.15.12:6839","nonce":3124093009}]},"join_fscid":-1,"export_targets":[0,2,3,4],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24674":{"gid":24674,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.165:6839/501312984","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6838","nonce":501312984},{"type":"v1","addr":"172.21.15.165:6839","nonce":501312984}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25672":{"gid":25672,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6837/308601741","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":308601741},{"type":"v1","addr":"172.21.15.178:6837","nonce":308601741}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24674,"qdb_cluster":[24614,24637,24638,24643,24674]},"id":1}]} 2024-09-11T00:22:04.933 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 55 2024-09-11T00:22:04.935 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:22:04.933+0000 7f66c6a00640 1 -- 172.21.15.12:0/1982925899 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f66a40b52e0 msgr2=0x7f66a40b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:22:04.935 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:22:04.933+0000 7f66c6a00640 1 --2- 172.21.15.12:0/1982925899 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f66a40b52e0 0x7f66a40b77a0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f66c0107e30 tx=0x7f66ac00b390 comp rx=0 tx=0).stop 2024-09-11T00:22:04.936 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:22:04.933+0000 7f66c6a00640 1 -- 172.21.15.12:0/1982925899 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f66c0102dc0 msgr2=0x7f66c01992d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:22:04.936 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:22:04.933+0000 7f66c6a00640 1 --2- 172.21.15.12:0/1982925899 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f66c0102dc0 0x7f66c01992d0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f66b0005ab0 tx=0x7f66b0003c80 comp rx=0 tx=0).stop 2024-09-11T00:22:04.936 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:22:04.933+0000 7f66c6a00640 1 -- 172.21.15.12:0/1982925899 shutdown_connections 2024-09-11T00:22:04.936 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:22:04.933+0000 7f66c6a00640 1 --2- 172.21.15.12:0/1982925899 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f66a40b52e0 0x7f66a40b77a0 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:04.936 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:22:04.933+0000 7f66c6a00640 1 --2- 172.21.15.12:0/1982925899 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f66c0109150 0x7f66c019ebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:04.936 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:22:04.933+0000 7f66c6a00640 1 --2- 172.21.15.12:0/1982925899 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f66c01087d0 0x7f66c0199810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:04.936 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:22:04.933+0000 7f66c6a00640 1 --2- 172.21.15.12:0/1982925899 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f66c0102dc0 0x7f66c01992d0 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:04.936 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:22:04.933+0000 7f66c6a00640 1 -- 172.21.15.12:0/1982925899 >> 172.21.15.12:0/1982925899 conn(0x7f66c00fe830 msgr2=0x7f66c0100c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:22:04.936 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:22:04.933+0000 7f66c6a00640 1 -- 172.21.15.12:0/1982925899 shutdown_connections 2024-09-11T00:22:04.936 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:22:04.933+0000 7f66c6a00640 1 -- 172.21.15.12:0/1982925899 wait complete. 2024-09-11T00:22:05.164 INFO:tasks.workunit.client.0.smithi012.stdout: 10 33135 0.00 MB/sec execute 1 sec latency 19978.695 ms 2024-09-11T00:22:05.563 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:22:05 smithi012 ceph-mon[30333]: pgmap v849: 129 pgs: 129 active+clean; 393 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 186 KiB/s rd, 3.8 MiB/s wr, 152 op/s 2024-09-11T00:22:05.563 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:22:05 smithi012 ceph-mon[30333]: from='client.? 172.21.15.12:0/1982925899' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T00:22:05.681 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24674 quiesce db --set-id 7dde0ef8 --release --await' 2024-09-11T00:22:05.681 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid ff3ad74a-6fcf-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24674 quiesce db --set-id 7dde0ef8 --release --await 2024-09-11T00:22:05.782 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:22:05 smithi165 ceph-mon[35356]: pgmap v849: 129 pgs: 129 active+clean; 393 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 186 KiB/s rd, 3.8 MiB/s wr, 152 op/s 2024-09-11T00:22:05.783 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:22:05 smithi165 ceph-mon[35356]: from='client.? 172.21.15.12:0/1982925899' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T00:22:05.831 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:22:05 smithi178 ceph-mon[35919]: pgmap v849: 129 pgs: 129 active+clean; 393 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 186 KiB/s rd, 3.8 MiB/s wr, 152 op/s 2024-09-11T00:22:05.831 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:22:05 smithi178 ceph-mon[35919]: from='client.? 172.21.15.12:0/1982925899' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T00:22:06.060 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:22:06.164 INFO:tasks.workunit.client.0.smithi012.stdout: 10 33135 0.00 MB/sec execute 2 sec latency 20978.740 ms 2024-09-11T00:22:06.648 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:22:06.645+0000 7ff1b9400640 1 -- 172.21.15.12:0/1658258327 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff1b4102da0 msgr2=0x7ff1b4103180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:22:06.648 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:22:06.645+0000 7ff1b9400640 1 --2- 172.21.15.12:0/1658258327 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff1b4102da0 0x7ff1b4103180 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7ff1a4009900 tx=0x7ff1a402a410 comp rx=0 tx=0).stop 2024-09-11T00:22:06.648 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:22:06.645+0000 7ff1b9400640 1 -- 172.21.15.12:0/1658258327 shutdown_connections 2024-09-11T00:22:06.648 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:22:06.645+0000 7ff1b9400640 1 --2- 172.21.15.12:0/1658258327 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff1b4109130 0x7ff1b410f930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:06.649 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:22:06.645+0000 7ff1b9400640 1 --2- 172.21.15.12:0/1658258327 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff1b41087b0 0x7ff1b4108bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:06.649 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:22:06.645+0000 7ff1b9400640 1 --2- 172.21.15.12:0/1658258327 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff1b4102da0 0x7ff1b4103180 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:06.649 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:22:06.645+0000 7ff1b9400640 1 -- 172.21.15.12:0/1658258327 >> 172.21.15.12:0/1658258327 conn(0x7ff1b40fe810 msgr2=0x7ff1b4100c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:22:06.649 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:22:06.645+0000 7ff1b9400640 1 -- 172.21.15.12:0/1658258327 shutdown_connections 2024-09-11T00:22:06.649 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:22:06.645+0000 7ff1b9400640 1 -- 172.21.15.12:0/1658258327 wait complete. 2024-09-11T00:22:06.649 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:22:06.646+0000 7ff1b9400640 1 Processor -- start 2024-09-11T00:22:06.650 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:22:06.647+0000 7ff1b9400640 1 -- start start 2024-09-11T00:22:06.650 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:22:06.647+0000 7ff1b9400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff1b4102da0 0x7ff1b4199260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:22:06.650 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:22:06.647+0000 7ff1b9400640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff1b41087b0 0x7ff1b41997a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:22:06.650 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:22:06.647+0000 7ff1b9400640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff1b4109130 0x7ff1b419eb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:22:06.650 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:22:06.647+0000 7ff1b9400640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7ff1b4112e40 con 0x7ff1b4109130 2024-09-11T00:22:06.651 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:22:06.647+0000 7ff1b9400640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7ff1b4112cc0 con 0x7ff1b4102da0 2024-09-11T00:22:06.651 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:22:06.647+0000 7ff1b9400640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7ff1b4112fc0 con 0x7ff1b41087b0 2024-09-11T00:22:06.651 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:22:06.648+0000 7ff1b1e00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff1b41087b0 0x7ff1b41997a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:22:06.651 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:22:06.648+0000 7ff1b1e00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff1b41087b0 0x7ff1b41997a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.12:38910/0 (socket says 172.21.15.12:38910) 2024-09-11T00:22:06.651 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:22:06.648+0000 7ff1b1e00640 1 -- 172.21.15.12:0/996095619 learned_addr learned my addr 172.21.15.12:0/996095619 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:22:06.651 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:22:06.648+0000 7ff1b2800640 1 --2- 172.21.15.12:0/996095619 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff1b4102da0 0x7ff1b4199260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:22:06.651 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:22:06.648+0000 7ff1b3200640 1 --2- 172.21.15.12:0/996095619 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff1b4109130 0x7ff1b419eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:22:06.652 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:22:06.648+0000 7ff1b1e00640 1 -- 172.21.15.12:0/996095619 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff1b4102da0 msgr2=0x7ff1b4199260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:22:06.652 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:22:06.648+0000 7ff1b1e00640 1 --2- 172.21.15.12:0/996095619 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff1b4102da0 0x7ff1b4199260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:06.652 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:22:06.648+0000 7ff1b1e00640 1 -- 172.21.15.12:0/996095619 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff1b4109130 msgr2=0x7ff1b419eb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:22:06.652 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:22:06.648+0000 7ff1b1e00640 1 --2- 172.21.15.12:0/996095619 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff1b4109130 0x7ff1b419eb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:06.652 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:22:06.648+0000 7ff1b1e00640 1 -- 172.21.15.12:0/996095619 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1b419f310 con 0x7ff1b41087b0 2024-09-11T00:22:06.652 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:22:06.648+0000 7ff1b1e00640 1 --2- 172.21.15.12:0/996095619 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff1b41087b0 0x7ff1b41997a0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7ff1a000d950 tx=0x7ff1a000de10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:22:06.652 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:22:06.649+0000 7ff1ab400640 1 -- 172.21.15.12:0/996095619 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1a0017070 con 0x7ff1b41087b0 2024-09-11T00:22:06.652 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:22:06.649+0000 7ff1b9400640 1 -- 172.21.15.12:0/996095619 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff1b419f600 con 0x7ff1b41087b0 2024-09-11T00:22:06.652 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:22:06.649+0000 7ff1ab400640 1 -- 172.21.15.12:0/996095619 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff1a0009c30 con 0x7ff1b41087b0 2024-09-11T00:22:06.652 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:22:06.649+0000 7ff1ab400640 1 -- 172.21.15.12:0/996095619 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1a000f620 con 0x7ff1b41087b0 2024-09-11T00:22:06.653 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:22:06.649+0000 7ff1b9400640 1 -- 172.21.15.12:0/996095619 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1b4075380 con 0x7ff1b41087b0 2024-09-11T00:22:06.654 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:22:06.652+0000 7ff1bbff1740 1 Processor -- start 2024-09-11T00:22:06.655 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:22:06.652+0000 7ff1bbff1740 1 -- start start 2024-09-11T00:22:06.655 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:22:06.652+0000 7ff1ab400640 1 -- 172.21.15.12:0/996095619 <== mon.1 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7ff1a0010460 con 0x7ff1b41087b0 2024-09-11T00:22:06.655 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:22:06.652+0000 7ff1bbff1740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561f82cf9ff0 0x561f82cfa3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:22:06.655 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:22:06.652+0000 7ff1bbff1740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561f82cfa910 0x561f82d06360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:22:06.655 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:22:06.652+0000 7ff1bbff1740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561f82d068a0 0x561f82d08c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:22:06.655 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:22:06.652+0000 7ff1bbff1740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x561f82c71290 con 0x561f82cfa910 2024-09-11T00:22:06.656 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:22:06.652+0000 7ff1bbff1740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x561f82c606c0 con 0x561f82d068a0 2024-09-11T00:22:06.656 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:22:06.652+0000 7ff1bbff1740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x561f82c644d0 con 0x561f82cf9ff0 2024-09-11T00:22:06.656 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:22:06.653+0000 7ff1b3200640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561f82cf9ff0 0x561f82cfa3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:22:06.656 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:22:06.653+0000 7ff1b2800640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561f82cfa910 0x561f82d06360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:22:06.656 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:22:06.653+0000 7ff1b3200640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561f82cf9ff0 0x561f82cfa3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.12:38918/0 (socket says 172.21.15.12:38918) 2024-09-11T00:22:06.656 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:22:06.653+0000 7ff1b3200640 1 -- 172.21.15.12:0/3032014257 learned_addr learned my addr 172.21.15.12:0/3032014257 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:22:06.658 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:22:06.653+0000 7ff1b1e00640 1 --2- 172.21.15.12:0/3032014257 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561f82d068a0 0x561f82d08c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:22:06.658 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:22:06.653+0000 7ff1b2800640 1 -- 172.21.15.12:0/3032014257 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561f82cf9ff0 msgr2=0x561f82cfa3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:22:06.658 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:22:06.653+0000 7ff1b2800640 1 --2- 172.21.15.12:0/3032014257 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561f82cf9ff0 0x561f82cfa3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:06.658 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:22:06.653+0000 7ff1b2800640 1 -- 172.21.15.12:0/3032014257 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561f82d068a0 msgr2=0x561f82d08c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:22:06.658 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:22:06.653+0000 7ff1b2800640 1 --2- 172.21.15.12:0/3032014257 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561f82d068a0 0x561f82d08c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:06.658 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:22:06.653+0000 7ff1b2800640 1 -- 172.21.15.12:0/3032014257 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561f82bed460 con 0x561f82cfa910 2024-09-11T00:22:06.658 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:22:06.653+0000 7ff1b1e00640 1 --2- 172.21.15.12:0/3032014257 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561f82d068a0 0x561f82d08c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T00:22:06.658 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:22:06.654+0000 7ff1b2800640 1 --2- 172.21.15.12:0/3032014257 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561f82cfa910 0x561f82d06360 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7ff1a402a930 tx=0x7ff1a4005a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aeae7689c519609d server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:22:06.658 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:22:06.654+0000 7ff19f400640 1 -- 172.21.15.12:0/3032014257 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1a4049070 con 0x561f82cfa910 2024-09-11T00:22:06.658 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:22:06.654+0000 7ff19f400640 1 -- 172.21.15.12:0/3032014257 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff1a4005500 con 0x561f82cfa910 2024-09-11T00:22:06.658 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:22:06.654+0000 7ff19f400640 1 -- 172.21.15.12:0/3032014257 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1a4035030 con 0x561f82cfa910 2024-09-11T00:22:06.659 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:22:06.654+0000 7ff1ab400640 1 --2- 172.21.15.12:0/996095619 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7ff1980b52e0 0x7ff1980b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:22:06.659 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:22:06.654+0000 7ff1ab400640 1 -- 172.21.15.12:0/996095619 <== mon.1 v2:172.21.15.178:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9208+0+0 (secure 0 0 0) 0x7ff1a00e5bf0 con 0x7ff1b41087b0 2024-09-11T00:22:06.659 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:22:06.654+0000 7ff1b3200640 1 --2- 172.21.15.12:0/996095619 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7ff1980b52e0 0x7ff1980b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:22:06.659 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:22:06.655+0000 7ff1bbff1740 1 -- 172.21.15.12:0/3032014257 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561f82cfa910 msgr2=0x561f82d06360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:22:06.659 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:22:06.655+0000 7ff1bbff1740 1 --2- 172.21.15.12:0/3032014257 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561f82cfa910 0x561f82d06360 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7ff1a402a930 tx=0x7ff1a4005a20 comp rx=0 tx=0).stop 2024-09-11T00:22:06.659 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:22:06.655+0000 7ff1bbff1740 1 -- 172.21.15.12:0/3032014257 shutdown_connections 2024-09-11T00:22:06.659 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:22:06.655+0000 7ff1bbff1740 1 --2- 172.21.15.12:0/3032014257 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561f82d068a0 0x561f82d08c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:06.659 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:22:06.655+0000 7ff1bbff1740 1 --2- 172.21.15.12:0/3032014257 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561f82cfa910 0x561f82d06360 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:06.659 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:22:06.655+0000 7ff1bbff1740 1 --2- 172.21.15.12:0/3032014257 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561f82cf9ff0 0x561f82cfa3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:06.659 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:22:06.655+0000 7ff1bbff1740 1 -- 172.21.15.12:0/3032014257 >> 172.21.15.12:0/3032014257 conn(0x561f82cf7e20 msgr2=0x561f82cf8fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:22:06.659 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:22:06.655+0000 7ff1bbff1740 1 -- 172.21.15.12:0/3032014257 shutdown_connections 2024-09-11T00:22:06.660 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:22:06.655+0000 7ff1bbff1740 1 -- 172.21.15.12:0/3032014257 wait complete. 2024-09-11T00:22:06.660 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:22:06.655+0000 7ff1b3200640 1 --2- 172.21.15.12:0/996095619 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7ff1980b52e0 0x7ff1980b77a0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7ff1ac0116b0 tx=0x7ff1ac009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:22:06.662 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:22:06.656+0000 7ff1bbff1740 1 -- start start 2024-09-11T00:22:06.662 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:22:06.656+0000 7ff1bbff1740 1 Processor -- start 2024-09-11T00:22:06.662 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:22:06.656+0000 7ff1bbff1740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561f82cf9ff0 0x561f82d038d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:22:06.662 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:22:06.656+0000 7ff1bbff1740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561f82cfa910 0x561f82d03e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:22:06.662 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:22:06.656+0000 7ff1bbff1740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561f82d068a0 0x561f82d04350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:22:06.662 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:22:06.656+0000 7ff1bbff1740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x561f82ac1bb0 con 0x561f82d068a0 2024-09-11T00:22:06.662 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:22:06.656+0000 7ff1bbff1740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x561f82bfb6c0 con 0x561f82cfa910 2024-09-11T00:22:06.662 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:22:06.656+0000 7ff1bbff1740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x561f82d094f0 con 0x561f82cf9ff0 2024-09-11T00:22:06.662 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:22:06.657+0000 7ff1bbff1740 10 client.0 _ll_register_callbacks cb 0x561f82c16490 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:22:06.662 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:22:06.657+0000 7ff1b3200640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561f82d068a0 0x561f82d04350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:22:06.663 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:22:06.657+0000 7ff1b3200640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561f82d068a0 0x561f82d04350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:34722/0 (socket says 172.21.15.12:34722) 2024-09-11T00:22:06.663 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:22:06.657+0000 7ff1b3200640 1 -- 172.21.15.12:0/486060261 learned_addr learned my addr 172.21.15.12:0/486060261 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:22:06.663 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:22:06.657+0000 7ff1b2800640 1 --2- 172.21.15.12:0/486060261 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561f82cf9ff0 0x561f82d038d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:22:06.663 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:22:06.657+0000 7ff1b1e00640 1 --2- 172.21.15.12:0/486060261 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561f82cfa910 0x561f82d03e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:22:06.663 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:22:06.657+0000 7ff1b3200640 1 -- 172.21.15.12:0/486060261 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561f82cf9ff0 msgr2=0x561f82d038d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:22:06.663 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:22:06.657+0000 7ff1b3200640 1 --2- 172.21.15.12:0/486060261 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561f82cf9ff0 0x561f82d038d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:06.663 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:22:06.657+0000 7ff1b3200640 1 -- 172.21.15.12:0/486060261 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561f82cfa910 msgr2=0x561f82d03e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:22:06.663 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:22:06.657+0000 7ff1b3200640 1 --2- 172.21.15.12:0/486060261 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561f82cfa910 0x561f82d03e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:06.663 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:22:06.657+0000 7ff1b3200640 1 -- 172.21.15.12:0/486060261 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561f829beb00 con 0x561f82d068a0 2024-09-11T00:22:06.663 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:22:06.658+0000 7ff1b3200640 1 --2- 172.21.15.12:0/486060261 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561f82d068a0 0x561f82d04350 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7ff1ac00f730 tx=0x7ff1ac004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=772ef592fb3ef7fd server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:22:06.663 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:22:06.658+0000 7ff193e00640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-09-11T00:22:06.663 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:22:06.658+0000 7ff193e00640 1 -- 172.21.15.12:0/486060261 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1ac003c40 con 0x561f82d068a0 2024-09-11T00:22:06.663 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:22:06.658+0000 7ff1bbff1740 1 -- 172.21.15.12:0/486060261 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff1a4005500 con 0x561f82d068a0 2024-09-11T00:22:06.664 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:22:06.658+0000 7ff193e00640 1 -- 172.21.15.12:0/486060261 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9208+0+0 (secure 0 0 0) 0x7ff1ac003de0 con 0x561f82d068a0 2024-09-11T00:22:06.664 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:22:06.658+0000 7ff193e00640 1 -- 172.21.15.12:0/486060261 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7ff1ac031050 con 0x561f82d068a0 2024-09-11T00:22:06.664 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:22:06.658+0000 7ff1bbff1740 10 client.17187 fetch_fsmap learned FSMap version 55 2024-09-11T00:22:06.664 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:22:06.658+0000 7ff1bbff1740 1 -- 172.21.15.12:0/486060261 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) -- 0x561f82a35960 con 0x561f82d068a0 2024-09-11T00:22:06.664 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:22:06.658+0000 7ff193e00640 1 -- 172.21.15.12:0/486060261 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff1a4005500 con 0x561f82d068a0 2024-09-11T00:22:06.664 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:22:06.659+0000 7ff193e00640 1 -- 172.21.15.12:0/486060261 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 55) ==== 8418+0+0 (secure 0 0 0) 0x7ff1ac026020 con 0x561f82d068a0 2024-09-11T00:22:06.664 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:22:06.659+0000 7ff1bbff1740 10 client.17187 fetch_fsmap finished waiting for FSMap version 55 2024-09-11T00:22:06.664 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:22:06.659+0000 7ff1bbff1740 10 client.17187 resolve_mds: validated gid 24674 aka daemon mds.b 2024-09-11T00:22:06.664 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:22:06.659+0000 7ff1bbff1740 20 client.17187 populate_metadata read hostname 'smithi012' 2024-09-11T00:22:06.664 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:22:06.659+0000 7ff1bbff1740 1 --2- 172.21.15.12:0/486060261 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x561f82d164f0 0x561f82d18930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:22:06.664 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:22:06.659+0000 7ff1bbff1740 4 client.17187 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T00:22:06.664 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:22:06.659+0000 7ff1bbff1740 1 -- 172.21.15.12:0/486060261 --> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561f82c731e0 con 0x561f82d164f0 2024-09-11T00:22:06.664 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:22:06.659+0000 7ff1b2800640 1 --2- 172.21.15.12:0/486060261 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x561f82d164f0 0x561f82d18930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:22:06.666 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:22:06.660+0000 7ff1b2800640 1 --2- 172.21.15.12:0/486060261 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x561f82d164f0 0x561f82d18930 crc :-1 s=READY pgs=221 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=70d1a237629d0683 server_cookie=bd5c02bdd202d459 in_seq=0 out_seq=0 2024-09-11T00:22:06.666 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:22:06.660+0000 7ff193e00640 10 client.17187 ms_handle_connect on v2:172.21.15.165:6838/501312984 2024-09-11T00:22:06.667 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:22:06.661+0000 7ff193e00640 1 -- 172.21.15.12:0/486060261 <== mds.0 v2:172.21.15.165:6838/501312984 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x561f82c731e0 con 0x561f82d164f0 2024-09-11T00:22:06.667 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:22:06.661+0000 7ff193e00640 10 client.17187 handle_command_reply: tid=0 2024-09-11T00:22:06.667 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:22:06.661+0000 7ff1bbff1740 1 client.17187 shutdown 2024-09-11T00:22:06.667 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:22:06.661+0000 7ff1bbff1740 1 -- 172.21.15.12:0/486060261 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561f82d068a0 msgr2=0x561f82d04350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:22:06.667 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:22:06.661+0000 7ff1bbff1740 1 --2- 172.21.15.12:0/486060261 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561f82d068a0 0x561f82d04350 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7ff1ac00f730 tx=0x7ff1ac004ab0 comp rx=0 tx=0).stop 2024-09-11T00:22:06.667 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:22:06.661+0000 7ff1bbff1740 1 -- 172.21.15.12:0/486060261 shutdown_connections 2024-09-11T00:22:06.667 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:22:06.661+0000 7ff1bbff1740 1 --2- 172.21.15.12:0/486060261 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x561f82d164f0 0x561f82d18930 crc :-1 s=READY pgs=221 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:06.667 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:22:06.661+0000 7ff1bbff1740 1 --2- 172.21.15.12:0/486060261 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561f82d068a0 0x561f82d04350 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:06.667 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:22:06.661+0000 7ff1bbff1740 1 --2- 172.21.15.12:0/486060261 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561f82cfa910 0x561f82d03e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:06.667 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:22:06.661+0000 7ff1bbff1740 1 --2- 172.21.15.12:0/486060261 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561f82cf9ff0 0x561f82d038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:06.667 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:22:06.661+0000 7ff1bbff1740 1 -- 172.21.15.12:0/486060261 >> 172.21.15.12:0/486060261 conn(0x561f82cf7e20 msgr2=0x561f82cfd9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:22:06.668 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:22:06.661+0000 7ff193e00640 0 client.17187 ms_handle_reset on v2:172.21.15.165:6838/501312984 2024-09-11T00:22:06.668 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:22:06.661+0000 7ff1bbff1740 1 -- 172.21.15.12:0/486060261 shutdown_connections 2024-09-11T00:22:06.668 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:22:06.661+0000 7ff1bbff1740 1 -- 172.21.15.12:0/486060261 wait complete. 2024-09-11T00:22:06.668 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:22:06.662+0000 7ff1bbff1740 20 client.17187 trim_cache size 0 max 16384 2024-09-11T00:22:06.683 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:22:06.680+0000 7ff1bbff1740 1 Processor -- start 2024-09-11T00:22:06.683 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:22:06.680+0000 7ff1bbff1740 1 -- start start 2024-09-11T00:22:06.683 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:22:06.680+0000 7ff1bbff1740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561f82cf9ff0 0x561f82cf94e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:22:06.683 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:22:06.680+0000 7ff1bbff1740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561f82cfa910 0x561f82d05610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:22:06.683 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:22:06.680+0000 7ff1bbff1740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561f82d068a0 0x561f82d00ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:22:06.683 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:22:06.680+0000 7ff1bbff1740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7ff1b41ada30 con 0x561f82d068a0 2024-09-11T00:22:06.683 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:22:06.680+0000 7ff1bbff1740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7ff1b41af750 con 0x561f82cfa910 2024-09-11T00:22:06.683 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:22:06.680+0000 7ff1bbff1740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7ff1b41ae160 con 0x561f82cf9ff0 2024-09-11T00:22:06.684 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:22:06.680+0000 7ff1b2800640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561f82cf9ff0 0x561f82cf94e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:22:06.684 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:22:06.680+0000 7ff1b3200640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561f82d068a0 0x561f82d00ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:22:06.684 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:22:06.680+0000 7ff1b2800640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561f82cf9ff0 0x561f82cf94e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.12:38924/0 (socket says 172.21.15.12:38924) 2024-09-11T00:22:06.684 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:22:06.680+0000 7ff1b2800640 1 -- 172.21.15.12:0/2010132098 learned_addr learned my addr 172.21.15.12:0/2010132098 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:22:06.684 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:22:06.680+0000 7ff1b1e00640 1 --2- 172.21.15.12:0/2010132098 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561f82cfa910 0x561f82d05610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:22:06.684 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:22:06.681+0000 7ff1b2800640 1 -- 172.21.15.12:0/2010132098 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561f82cfa910 msgr2=0x561f82d05610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:22:06.684 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:22:06.681+0000 7ff1b2800640 1 --2- 172.21.15.12:0/2010132098 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561f82cfa910 0x561f82d05610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:06.684 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:22:06.681+0000 7ff1b2800640 1 -- 172.21.15.12:0/2010132098 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561f82d068a0 msgr2=0x561f82d00ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:22:06.684 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:22:06.681+0000 7ff1b2800640 1 --2- 172.21.15.12:0/2010132098 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561f82d068a0 0x561f82d00ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:06.684 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:22:06.681+0000 7ff1b2800640 1 -- 172.21.15.12:0/2010132098 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561f82cf88a0 con 0x561f82cf9ff0 2024-09-11T00:22:06.684 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:22:06.681+0000 7ff1b1e00640 1 --2- 172.21.15.12:0/2010132098 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561f82cfa910 0x561f82d05610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T00:22:06.684 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:22:06.681+0000 7ff1b2800640 1 --2- 172.21.15.12:0/2010132098 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561f82cf9ff0 0x561f82cf94e0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7ff1a402a930 tx=0x7ff1a402a410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=352bb864bdffb609 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:22:06.688 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:22:06.681+0000 7ff19ea00640 1 -- 172.21.15.12:0/2010132098 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1a4049040 con 0x561f82cf9ff0 2024-09-11T00:22:06.689 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:22:06.681+0000 7ff19ea00640 1 -- 172.21.15.12:0/2010132098 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff1a404a040 con 0x561f82cf9ff0 2024-09-11T00:22:06.689 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:22:06.681+0000 7ff19ea00640 1 -- 172.21.15.12:0/2010132098 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1a4035040 con 0x561f82cf9ff0 2024-09-11T00:22:06.689 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:22:06.682+0000 7ff1bbff1740 1 -- 172.21.15.12:0/2010132098 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561f82cf9ff0 msgr2=0x561f82cf94e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:22:06.689 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:22:06.682+0000 7ff1bbff1740 1 --2- 172.21.15.12:0/2010132098 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561f82cf9ff0 0x561f82cf94e0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7ff1a402a930 tx=0x7ff1a402a410 comp rx=0 tx=0).stop 2024-09-11T00:22:06.689 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:22:06.682+0000 7ff1bbff1740 1 -- 172.21.15.12:0/2010132098 shutdown_connections 2024-09-11T00:22:06.689 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:22:06.682+0000 7ff1bbff1740 1 --2- 172.21.15.12:0/2010132098 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561f82d068a0 0x561f82d00ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:06.689 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:22:06.682+0000 7ff1bbff1740 1 --2- 172.21.15.12:0/2010132098 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561f82cfa910 0x561f82d05610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:06.689 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:22:06.682+0000 7ff1bbff1740 1 --2- 172.21.15.12:0/2010132098 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561f82cf9ff0 0x561f82cf94e0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:06.689 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:22:06.682+0000 7ff1bbff1740 1 -- 172.21.15.12:0/2010132098 >> 172.21.15.12:0/2010132098 conn(0x561f82cf7e20 msgr2=0x561f82d04f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:22:06.689 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:22:06.682+0000 7ff1bbff1740 1 -- 172.21.15.12:0/2010132098 shutdown_connections 2024-09-11T00:22:06.689 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:22:06.682+0000 7ff1bbff1740 1 -- 172.21.15.12:0/2010132098 wait complete. 2024-09-11T00:22:06.690 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:22:06.683+0000 7ff1bbff1740 1 -- start start 2024-09-11T00:22:06.690 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:22:06.683+0000 7ff1bbff1740 1 Processor -- start 2024-09-11T00:22:06.690 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:22:06.683+0000 7ff1bbff1740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561f82cf9ff0 0x561f82d0b230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:22:06.690 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:22:06.683+0000 7ff1bbff1740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561f82cfa910 0x561f82d0b7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:22:06.690 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:22:06.683+0000 7ff1bbff1740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561f82d068a0 0x561f82d7c520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:22:06.690 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:22:06.683+0000 7ff1bbff1740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7ff1b41af080 con 0x561f82d068a0 2024-09-11T00:22:06.690 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:22:06.683+0000 7ff1bbff1740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7ff1b41ae8f0 con 0x561f82cf9ff0 2024-09-11T00:22:06.690 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:22:06.683+0000 7ff1bbff1740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7ff1b41aba10 con 0x561f82cfa910 2024-09-11T00:22:06.690 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:22:06.683+0000 7ff1bbff1740 10 client.0 _ll_register_callbacks cb 0x7ff1ac009c40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:22:06.690 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:22:06.684+0000 7ff1b2800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561f82cf9ff0 0x561f82d0b230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:22:06.690 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:22:06.684+0000 7ff1b1e00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561f82cfa910 0x561f82d0b7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:22:06.690 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:22:06.684+0000 7ff1b2800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561f82cf9ff0 0x561f82d0b230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.12:36368/0 (socket says 172.21.15.12:36368) 2024-09-11T00:22:06.690 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:22:06.684+0000 7ff1b3200640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561f82d068a0 0x561f82d7c520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:22:06.691 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:22:06.684+0000 7ff1b2800640 1 -- 172.21.15.12:0/1687581762 learned_addr learned my addr 172.21.15.12:0/1687581762 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:22:06.691 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:22:06.684+0000 7ff1b2800640 1 -- 172.21.15.12:0/1687581762 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561f82cfa910 msgr2=0x561f82d0b7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:22:06.691 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:22:06.684+0000 7ff1b2800640 1 --2- 172.21.15.12:0/1687581762 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561f82cfa910 0x561f82d0b7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:06.691 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:22:06.684+0000 7ff1b2800640 1 -- 172.21.15.12:0/1687581762 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561f82d068a0 msgr2=0x561f82d7c520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:22:06.691 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:22:06.684+0000 7ff1b2800640 1 --2- 172.21.15.12:0/1687581762 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561f82d068a0 0x561f82d7c520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:06.691 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:22:06.684+0000 7ff1b2800640 1 -- 172.21.15.12:0/1687581762 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561f82d7cc30 con 0x561f82cf9ff0 2024-09-11T00:22:06.691 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:22:06.684+0000 7ff1b1e00640 1 --2- 172.21.15.12:0/1687581762 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561f82cfa910 0x561f82d0b7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:22:06.691 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:22:06.684+0000 7ff1b3200640 1 --2- 172.21.15.12:0/1687581762 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561f82d068a0 0x561f82d7c520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:22:06.691 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:22:06.684+0000 7ff1b2800640 1 --2- 172.21.15.12:0/1687581762 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561f82cf9ff0 0x561f82d0b230 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7ff1a4009900 tx=0x7ff1a4003680 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b3754f40e77b7b0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:22:06.691 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:22:06.684+0000 7ff193e00640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-09-11T00:22:06.691 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:22:06.685+0000 7ff193e00640 1 -- 172.21.15.12:0/1687581762 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1a4035040 con 0x561f82cf9ff0 2024-09-11T00:22:06.691 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:22:06.685+0000 7ff1bbff1740 1 -- 172.21.15.12:0/1687581762 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff1a404a040 con 0x561f82cf9ff0 2024-09-11T00:22:06.692 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:22:06.685+0000 7ff193e00640 1 -- 172.21.15.12:0/1687581762 <== mon.2 v2:172.21.15.165:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9208+0+0 (secure 0 0 0) 0x7ff1a4003960 con 0x561f82cf9ff0 2024-09-11T00:22:06.692 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:22:06.685+0000 7ff193e00640 1 -- 172.21.15.12:0/1687581762 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7ff1a406c050 con 0x561f82cf9ff0 2024-09-11T00:22:06.692 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:22:06.685+0000 7ff1bbff1740 10 client.27212 fetch_fsmap learned FSMap version 55 2024-09-11T00:22:06.692 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:22:06.685+0000 7ff1bbff1740 1 -- 172.21.15.12:0/1687581762 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) -- 0x561f82d84100 con 0x561f82cf9ff0 2024-09-11T00:22:06.692 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:22:06.685+0000 7ff193e00640 1 -- 172.21.15.12:0/1687581762 <== mon.2 v2:172.21.15.165:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff1a404a040 con 0x561f82cf9ff0 2024-09-11T00:22:06.692 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:22:06.685+0000 7ff193e00640 1 -- 172.21.15.12:0/1687581762 <== mon.2 v2:172.21.15.165:3300/0 5 ==== fsmap(e 55) ==== 8418+0+0 (secure 0 0 0) 0x7ff1a404f040 con 0x561f82cf9ff0 2024-09-11T00:22:06.692 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:22:06.686+0000 7ff1bbff1740 10 client.27212 fetch_fsmap finished waiting for FSMap version 55 2024-09-11T00:22:06.692 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:22:06.686+0000 7ff1bbff1740 10 client.27212 resolve_mds: validated gid 24674 aka daemon mds.b 2024-09-11T00:22:06.693 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:22:06.686+0000 7ff1bbff1740 20 client.27212 populate_metadata read hostname 'smithi012' 2024-09-11T00:22:06.693 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:22:06.686+0000 7ff1bbff1740 1 --2- 172.21.15.12:0/1687581762 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x561f82d84550 0x561f82d86970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:22:06.693 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:22:06.686+0000 7ff1bbff1740 4 client.27212 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "7dde0ef8", "release": true, "await": true}] 2024-09-11T00:22:06.693 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:22:06.686+0000 7ff1bbff1740 1 -- 172.21.15.12:0/1687581762 --> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] -- command(tid 0: {"prefix": "quiesce db", "set_id": "7dde0ef8", "release": true, "await": true}) -- 0x561f82c6ec90 con 0x561f82d84550 2024-09-11T00:22:06.693 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:22:06.686+0000 7ff1b1e00640 1 --2- 172.21.15.12:0/1687581762 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x561f82d84550 0x561f82d86970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:22:06.693 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:22:06.686+0000 7ff1b1e00640 1 --2- 172.21.15.12:0/1687581762 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x561f82d84550 0x561f82d86970 crc :-1 s=READY pgs=222 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e1b42c7270d6e89a server_cookie=9c9b69c04c0c7f54 in_seq=0 out_seq=0 2024-09-11T00:22:06.693 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:22:06.686+0000 7ff193e00640 10 client.27212 ms_handle_connect on v2:172.21.15.165:6838/501312984 2024-09-11T00:22:06.723 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:22:06.721+0000 7ff193e00640 1 -- 172.21.15.12:0/1687581762 <== mds.0 v2:172.21.15.165:6838/501312984 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x561f82c6ec90 con 0x561f82d84550 2024-09-11T00:22:06.723 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:22:06.721+0000 7ff193e00640 10 client.27212 handle_command_reply: tid=0 2024-09-11T00:22:06.724 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:22:06.721+0000 7ff1bbff1740 1 client.27212 shutdown 2024-09-11T00:22:06.724 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:22:06.721+0000 7ff1bbff1740 1 -- 172.21.15.12:0/1687581762 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561f82cf9ff0 msgr2=0x561f82d0b230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:22:06.724 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:22:06.721+0000 7ff1bbff1740 1 --2- 172.21.15.12:0/1687581762 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561f82cf9ff0 0x561f82d0b230 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7ff1a4009900 tx=0x7ff1a4003680 comp rx=0 tx=0).stop 2024-09-11T00:22:06.725 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:22:06.721+0000 7ff1bbff1740 1 -- 172.21.15.12:0/1687581762 shutdown_connections 2024-09-11T00:22:06.725 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:22:06.721+0000 7ff1bbff1740 1 --2- 172.21.15.12:0/1687581762 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x561f82d84550 0x561f82d86970 crc :-1 s=READY pgs=222 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:06.725 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:22:06.722+0000 7ff1bbff1740 1 --2- 172.21.15.12:0/1687581762 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561f82d068a0 0x561f82d7c520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:06.725 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:22:06.722+0000 7ff1bbff1740 1 --2- 172.21.15.12:0/1687581762 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561f82cfa910 0x561f82d0b7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:06.725 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:22:06.722+0000 7ff1bbff1740 1 --2- 172.21.15.12:0/1687581762 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561f82cf9ff0 0x561f82d0b230 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:06.725 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:22:06.722+0000 7ff1bbff1740 1 -- 172.21.15.12:0/1687581762 >> 172.21.15.12:0/1687581762 conn(0x561f82cf7e20 msgr2=0x561f82d11f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:22:06.725 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:22:06.722+0000 7ff193e00640 0 client.27212 ms_handle_reset on v2:172.21.15.165:6838/501312984 2024-09-11T00:22:06.725 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:22:06.722+0000 7ff1bbff1740 1 -- 172.21.15.12:0/1687581762 shutdown_connections 2024-09-11T00:22:06.725 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:22:06.722+0000 7ff1bbff1740 1 -- 172.21.15.12:0/1687581762 wait complete. 2024-09-11T00:22:06.725 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:22:06.722+0000 7ff1bbff1740 20 client.27212 trim_cache size 0 max 16384 2024-09-11T00:22:06.726 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-09-11T00:22:06.726 INFO:teuthology.orchestra.run.smithi012.stdout: "epoch": 55, 2024-09-11T00:22:06.726 INFO:teuthology.orchestra.run.smithi012.stdout: "leader": 24674, 2024-09-11T00:22:06.726 INFO:teuthology.orchestra.run.smithi012.stdout: "set_version": 48, 2024-09-11T00:22:06.726 INFO:teuthology.orchestra.run.smithi012.stdout: "sets": { 2024-09-11T00:22:06.726 INFO:teuthology.orchestra.run.smithi012.stdout: "7dde0ef8": { 2024-09-11T00:22:06.726 INFO:teuthology.orchestra.run.smithi012.stdout: "version": 48, 2024-09-11T00:22:06.726 INFO:teuthology.orchestra.run.smithi012.stdout: "age_ref": 0.0, 2024-09-11T00:22:06.726 INFO:teuthology.orchestra.run.smithi012.stdout: "state": { 2024-09-11T00:22:06.726 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "RELEASED", 2024-09-11T00:22:06.727 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 0.0 2024-09-11T00:22:06.727 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:22:06.727 INFO:teuthology.orchestra.run.smithi012.stdout: "timeout": 90.0, 2024-09-11T00:22:06.727 INFO:teuthology.orchestra.run.smithi012.stdout: "expiration": 142.8, 2024-09-11T00:22:06.727 INFO:teuthology.orchestra.run.smithi012.stdout: "members": { 2024-09-11T00:22:06.727 INFO:teuthology.orchestra.run.smithi012.stdout: "file:/": { 2024-09-11T00:22:06.727 INFO:teuthology.orchestra.run.smithi012.stdout: "excluded": false, 2024-09-11T00:22:06.727 INFO:teuthology.orchestra.run.smithi012.stdout: "state": { 2024-09-11T00:22:06.727 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "RELEASED", 2024-09-11T00:22:06.727 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 0.0 2024-09-11T00:22:06.727 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:22:06.727 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:22:06.727 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:22:06.728 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:22:06.728 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:22:06.728 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-09-11T00:22:06.728 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:22:06.723+0000 7ff1b9400640 1 -- 172.21.15.12:0/996095619 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7ff1980b52e0 msgr2=0x7ff1980b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:22:06.729 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:22:06.723+0000 7ff1b9400640 1 --2- 172.21.15.12:0/996095619 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7ff1980b52e0 0x7ff1980b77a0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7ff1ac0116b0 tx=0x7ff1ac009500 comp rx=0 tx=0).stop 2024-09-11T00:22:06.729 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:22:06.723+0000 7ff1b9400640 1 -- 172.21.15.12:0/996095619 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff1b41087b0 msgr2=0x7ff1b41997a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:22:06.729 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:22:06.723+0000 7ff1b9400640 1 --2- 172.21.15.12:0/996095619 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff1b41087b0 0x7ff1b41997a0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7ff1a000d950 tx=0x7ff1a000de10 comp rx=0 tx=0).stop 2024-09-11T00:22:06.729 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:22:06.724+0000 7ff1b9400640 1 -- 172.21.15.12:0/996095619 shutdown_connections 2024-09-11T00:22:06.729 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:22:06.724+0000 7ff1b9400640 1 --2- 172.21.15.12:0/996095619 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7ff1980b52e0 0x7ff1980b77a0 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:06.729 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:22:06.724+0000 7ff1b9400640 1 --2- 172.21.15.12:0/996095619 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff1b4109130 0x7ff1b419eb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:06.729 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:22:06.724+0000 7ff1b9400640 1 --2- 172.21.15.12:0/996095619 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff1b41087b0 0x7ff1b41997a0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:06.729 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:22:06.724+0000 7ff1b9400640 1 --2- 172.21.15.12:0/996095619 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff1b4102da0 0x7ff1b4199260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:06.729 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:22:06.724+0000 7ff1b9400640 1 -- 172.21.15.12:0/996095619 >> 172.21.15.12:0/996095619 conn(0x7ff1b40fe810 msgr2=0x7ff1b4100c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:22:06.729 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:22:06.724+0000 7ff1b9400640 1 -- 172.21.15.12:0/996095619 shutdown_connections 2024-09-11T00:22:06.729 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:22:06.724+0000 7ff1b9400640 1 -- 172.21.15.12:0/996095619 wait complete. 2024-09-11T00:22:07.164 INFO:tasks.workunit.client.0.smithi012.stdout: 10 33677 10.79 MB/sec execute 3 sec latency 21520.793 ms 2024-09-11T00:22:07.570 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '7dde0ef8', total seconds elapsed: 26.6 2024-09-11T00:22:07.813 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:22:07 smithi012 ceph-mon[30333]: pgmap v850: 129 pgs: 129 active+clean; 393 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 218 KiB/s rd, 3.2 MiB/s wr, 150 op/s 2024-09-11T00:22:07.831 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:22:07 smithi178 ceph-mon[35919]: pgmap v850: 129 pgs: 129 active+clean; 393 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 218 KiB/s rd, 3.2 MiB/s wr, 150 op/s 2024-09-11T00:22:08.033 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:22:07 smithi165 ceph-mon[35356]: pgmap v850: 129 pgs: 129 active+clean; 393 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 218 KiB/s rd, 3.2 MiB/s wr, 150 op/s 2024-09-11T00:22:08.164 INFO:tasks.workunit.client.0.smithi012.stdout: 10 34761 23.66 MB/sec execute 4 sec latency 46.575 ms 2024-09-11T00:22:09.164 INFO:tasks.workunit.client.0.smithi012.stdout: 10 35655 26.93 MB/sec execute 5 sec latency 59.414 ms 2024-09-11T00:22:09.813 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:22:09 smithi012 ceph-mon[30333]: pgmap v851: 129 pgs: 129 active+clean; 393 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 1.0 MiB/s wr, 82 op/s 2024-09-11T00:22:09.831 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:22:09 smithi178 ceph-mon[35919]: pgmap v851: 129 pgs: 129 active+clean; 393 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 1.0 MiB/s wr, 82 op/s 2024-09-11T00:22:10.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:22:09 smithi165 ceph-mon[35356]: pgmap v851: 129 pgs: 129 active+clean; 393 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 1.0 MiB/s wr, 82 op/s 2024-09-11T00:22:10.164 INFO:tasks.workunit.client.0.smithi012.stdout: 10 36793 33.61 MB/sec execute 6 sec latency 39.524 ms 2024-09-11T00:22:11.165 INFO:tasks.workunit.client.0.smithi012.stdout: 10 37765 36.55 MB/sec execute 7 sec latency 49.892 ms 2024-09-11T00:22:11.813 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:22:11 smithi012 ceph-mon[30333]: pgmap v852: 129 pgs: 129 active+clean; 396 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.9 MiB/s wr, 180 op/s 2024-09-11T00:22:11.831 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:22:11 smithi178 ceph-mon[35919]: pgmap v852: 129 pgs: 129 active+clean; 396 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.9 MiB/s wr, 180 op/s 2024-09-11T00:22:12.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:22:11 smithi165 ceph-mon[35356]: pgmap v852: 129 pgs: 129 active+clean; 396 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.9 MiB/s wr, 180 op/s 2024-09-11T00:22:12.165 INFO:tasks.workunit.client.0.smithi012.stdout: 10 38986 39.90 MB/sec execute 8 sec latency 36.412 ms 2024-09-11T00:22:13.165 INFO:tasks.workunit.client.0.smithi012.stdout: 10 39989 41.08 MB/sec execute 9 sec latency 49.903 ms 2024-09-11T00:22:13.813 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:22:13 smithi012 ceph-mon[30333]: pgmap v853: 129 pgs: 129 active+clean; 405 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 2.0 MiB/s wr, 282 op/s 2024-09-11T00:22:13.831 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:22:13 smithi178 ceph-mon[35919]: pgmap v853: 129 pgs: 129 active+clean; 405 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 2.0 MiB/s wr, 282 op/s 2024-09-11T00:22:14.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:22:13 smithi165 ceph-mon[35356]: pgmap v853: 129 pgs: 129 active+clean; 405 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 2.0 MiB/s wr, 282 op/s 2024-09-11T00:22:14.167 INFO:tasks.workunit.client.0.smithi012.stdout: 10 40929 42.61 MB/sec execute 10 sec latency 43.065 ms 2024-09-11T00:22:14.551 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:14.547+0000 7fb9d5400640 1 -- 172.21.15.12:0/3058113036 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb9d0105cf0 msgr2=0x7fb9d0106170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:22:14.551 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:14.547+0000 7fb9d5400640 1 --2- 172.21.15.12:0/3058113036 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb9d0105cf0 0x7fb9d0106170 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7fb9b8009930 tx=0x7fb9b802a420 comp rx=0 tx=0).stop 2024-09-11T00:22:14.551 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:14.548+0000 7fb9d5400640 1 -- 172.21.15.12:0/3058113036 shutdown_connections 2024-09-11T00:22:14.551 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:14.548+0000 7fb9d5400640 1 --2- 172.21.15.12:0/3058113036 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb9d01066b0 0x7fb9d010d030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:14.551 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:14.548+0000 7fb9d5400640 1 --2- 172.21.15.12:0/3058113036 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb9d0105cf0 0x7fb9d0106170 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:14.551 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:14.548+0000 7fb9d5400640 1 --2- 172.21.15.12:0/3058113036 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb9d0104ba0 0x7fb9d0104fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:14.551 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:14.548+0000 7fb9d5400640 1 -- 172.21.15.12:0/3058113036 >> 172.21.15.12:0/3058113036 conn(0x7fb9d0100440 msgr2=0x7fb9d0102880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:22:14.551 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:14.548+0000 7fb9d5400640 1 -- 172.21.15.12:0/3058113036 shutdown_connections 2024-09-11T00:22:14.551 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:14.548+0000 7fb9d5400640 1 -- 172.21.15.12:0/3058113036 wait complete. 2024-09-11T00:22:14.553 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:14.550+0000 7fb9d5400640 1 Processor -- start 2024-09-11T00:22:14.553 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:14.550+0000 7fb9d5400640 1 -- start start 2024-09-11T00:22:14.553 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:14.550+0000 7fb9d5400640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb9d0104ba0 0x7fb9d00779b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:22:14.553 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:14.550+0000 7fb9d5400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb9d0105cf0 0x7fb9d0078000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:22:14.553 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:14.550+0000 7fb9d5400640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb9d01066b0 0x7fb9d0078c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:22:14.554 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:14.550+0000 7fb9d5400640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fb9d0110840 con 0x7fb9d01066b0 2024-09-11T00:22:14.554 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:14.550+0000 7fb9d5400640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fb9d01106c0 con 0x7fb9d0105cf0 2024-09-11T00:22:14.554 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:14.550+0000 7fb9d5400640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fb9d01109c0 con 0x7fb9d0104ba0 2024-09-11T00:22:14.554 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:14.551+0000 7fb9cf200640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb9d01066b0 0x7fb9d0078c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:22:14.554 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:14.551+0000 7fb9cf200640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb9d01066b0 0x7fb9d0078c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:55808/0 (socket says 172.21.15.12:55808) 2024-09-11T00:22:14.554 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:14.551+0000 7fb9cf200640 1 -- 172.21.15.12:0/3406712208 learned_addr learned my addr 172.21.15.12:0/3406712208 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:22:14.555 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:14.551+0000 7fb9c7e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb9d0105cf0 0x7fb9d0078000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:22:14.555 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:14.552+0000 7fb9cf200640 1 -- 172.21.15.12:0/3406712208 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb9d0104ba0 msgr2=0x7fb9d00779b0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-11T00:22:14.555 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:14.552+0000 7fb9cf200640 1 --2- 172.21.15.12:0/3406712208 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb9d0104ba0 0x7fb9d00779b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:14.555 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:14.552+0000 7fb9cf200640 1 -- 172.21.15.12:0/3406712208 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb9d0105cf0 msgr2=0x7fb9d0078000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:22:14.555 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:14.552+0000 7fb9cf200640 1 --2- 172.21.15.12:0/3406712208 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb9d0105cf0 0x7fb9d0078000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:14.555 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:14.552+0000 7fb9cf200640 1 -- 172.21.15.12:0/3406712208 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9d00794f0 con 0x7fb9d01066b0 2024-09-11T00:22:14.555 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:14.553+0000 7fb9cf200640 1 --2- 172.21.15.12:0/3406712208 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb9d01066b0 0x7fb9d0078c30 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7fb9c800eb10 tx=0x7fb9c800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:22:14.556 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:14.553+0000 7fb9c7400640 1 -- 172.21.15.12:0/3406712208 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9c8004b20 con 0x7fb9d01066b0 2024-09-11T00:22:14.556 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:14.553+0000 7fb9c7400640 1 -- 172.21.15.12:0/3406712208 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb9c8004cc0 con 0x7fb9d01066b0 2024-09-11T00:22:14.556 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:14.553+0000 7fb9c7400640 1 -- 172.21.15.12:0/3406712208 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9c8010720 con 0x7fb9d01066b0 2024-09-11T00:22:14.556 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:14.553+0000 7fb9d5400640 1 -- 172.21.15.12:0/3406712208 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb9d00797e0 con 0x7fb9d01066b0 2024-09-11T00:22:14.556 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:14.553+0000 7fb9d5400640 1 -- 172.21.15.12:0/3406712208 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9d0079e80 con 0x7fb9d01066b0 2024-09-11T00:22:14.560 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:14.558+0000 7fb9d83eb740 1 Processor -- start 2024-09-11T00:22:14.560 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:14.558+0000 7fb9d83eb740 1 -- start start 2024-09-11T00:22:14.561 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:14.559+0000 7fb9d83eb740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563a31f8b790 0x563a31f8db80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:22:14.561 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:14.559+0000 7fb9d83eb740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563a31f8e0c0 0x563a31f904b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:22:14.561 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:14.559+0000 7fb9d83eb740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563a31f90b00 0x563a31f92fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:22:14.561 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:14.559+0000 7fb9d83eb740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x563a31efe6b0 con 0x563a31f90b00 2024-09-11T00:22:14.561 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:14.559+0000 7fb9d83eb740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x563a31eedae0 con 0x563a31f8b790 2024-09-11T00:22:14.561 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:14.559+0000 7fb9d83eb740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x563a31ef18f0 con 0x563a31f8e0c0 2024-09-11T00:22:14.562 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:14.559+0000 7fb9c7e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563a31f8b790 0x563a31f8db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:22:14.562 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:14.560+0000 7fb9c7e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563a31f8b790 0x563a31f8db80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.12:49100/0 (socket says 172.21.15.12:49100) 2024-09-11T00:22:14.562 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:14.560+0000 7fb9c7e00640 1 -- 172.21.15.12:0/555795219 learned_addr learned my addr 172.21.15.12:0/555795219 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:22:14.562 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:14.560+0000 7fb9c7e00640 1 -- 172.21.15.12:0/555795219 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563a31f8e0c0 msgr2=0x563a31f904b0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-11T00:22:14.562 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:14.560+0000 7fb9c7e00640 1 --2- 172.21.15.12:0/555795219 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563a31f8e0c0 0x563a31f904b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:14.562 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:14.560+0000 7fb9c7e00640 1 -- 172.21.15.12:0/555795219 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563a31f90b00 msgr2=0x563a31f92fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:22:14.563 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:14.560+0000 7fb9c7e00640 1 --2- 172.21.15.12:0/555795219 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563a31f90b00 0x563a31f92fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:14.563 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:14.560+0000 7fb9c7e00640 1 -- 172.21.15.12:0/555795219 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563a31eba780 con 0x563a31f8b790 2024-09-11T00:22:14.563 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:14.560+0000 7fb9c7e00640 1 --2- 172.21.15.12:0/555795219 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563a31f8b790 0x563a31f8db80 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fb9b8000c00 tx=0x7fb9b8005bf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dc68c28c51d994b0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:22:14.563 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:14.561+0000 7fb9bf400640 1 -- 172.21.15.12:0/555795219 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9b8035040 con 0x563a31f8b790 2024-09-11T00:22:14.563 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:14.561+0000 7fb9bf400640 1 -- 172.21.15.12:0/555795219 <== mon.2 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb9b8003aa0 con 0x563a31f8b790 2024-09-11T00:22:14.563 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:14.561+0000 7fb9bf400640 1 -- 172.21.15.12:0/555795219 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9b8031c50 con 0x563a31f8b790 2024-09-11T00:22:14.564 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:14.562+0000 7fb9d83eb740 1 -- 172.21.15.12:0/555795219 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563a31f8b790 msgr2=0x563a31f8db80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:22:14.564 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:14.562+0000 7fb9d83eb740 1 --2- 172.21.15.12:0/555795219 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563a31f8b790 0x563a31f8db80 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fb9b8000c00 tx=0x7fb9b8005bf0 comp rx=0 tx=0).stop 2024-09-11T00:22:14.568 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:14.566+0000 7fb9c7400640 1 -- 172.21.15.12:0/3406712208 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7fb9c80108c0 con 0x7fb9d01066b0 2024-09-11T00:22:14.568 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:14.566+0000 7fb9d83eb740 1 -- 172.21.15.12:0/555795219 shutdown_connections 2024-09-11T00:22:14.568 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:14.566+0000 7fb9d83eb740 1 --2- 172.21.15.12:0/555795219 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563a31f90b00 0x563a31f92fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:14.569 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:14.566+0000 7fb9d83eb740 1 --2- 172.21.15.12:0/555795219 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563a31f8e0c0 0x563a31f904b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:14.569 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:14.566+0000 7fb9d83eb740 1 --2- 172.21.15.12:0/555795219 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563a31f8b790 0x563a31f8db80 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:14.569 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:14.566+0000 7fb9d83eb740 1 -- 172.21.15.12:0/555795219 >> 172.21.15.12:0/555795219 conn(0x563a31f868e0 msgr2=0x563a31f87670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:22:14.569 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:14.567+0000 7fb9d83eb740 1 -- 172.21.15.12:0/555795219 shutdown_connections 2024-09-11T00:22:14.569 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:14.567+0000 7fb9d83eb740 1 -- 172.21.15.12:0/555795219 wait complete. 2024-09-11T00:22:14.570 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:14.568+0000 7fb9c7400640 1 --2- 172.21.15.12:0/3406712208 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fb9b40b5270 0x7fb9b40b7730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:22:14.570 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:14.568+0000 7fb9d83eb740 1 -- start start 2024-09-11T00:22:14.571 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:14.569+0000 7fb9c7400640 1 -- 172.21.15.12:0/3406712208 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9208+0+0 (secure 0 0 0) 0x7fb9c80e64d0 con 0x7fb9d01066b0 2024-09-11T00:22:14.571 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:14.569+0000 7fb9d83eb740 1 Processor -- start 2024-09-11T00:22:14.571 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:14.569+0000 7fb9d83eb740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563a31f8b790 0x563a31f94f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:22:14.571 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:14.569+0000 7fb9d83eb740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563a31f8e0c0 0x563a31f955f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:22:14.572 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:14.569+0000 7fb9d83eb740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563a31f90b00 0x563a31f961c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:22:14.572 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:14.569+0000 7fb9c7e00640 1 --2- 172.21.15.12:0/3406712208 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fb9b40b5270 0x7fb9b40b7730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:22:14.572 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:14.570+0000 7fb9d83eb740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x563a31e25ab0 con 0x563a31f90b00 2024-09-11T00:22:14.573 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:14.570+0000 7fb9d83eb740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x563a31e94570 con 0x563a31f8e0c0 2024-09-11T00:22:14.573 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:14.570+0000 7fb9d83eb740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x563a31f93a80 con 0x563a31f8b790 2024-09-11T00:22:14.573 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:14.570+0000 7fb9d83eb740 10 client.0 _ll_register_callbacks cb 0x563a31ea3100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:22:14.573 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:14.570+0000 7fb9cf200640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563a31f90b00 0x563a31f961c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:22:14.573 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:14.570+0000 7fb9ce800640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563a31f8b790 0x563a31f94f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:22:14.573 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:14.570+0000 7fb9cf200640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563a31f90b00 0x563a31f961c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:55816/0 (socket says 172.21.15.12:55816) 2024-09-11T00:22:14.573 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:14.570+0000 7fb9ce800640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563a31f8b790 0x563a31f94f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.12:36422/0 (socket says 172.21.15.12:36422) 2024-09-11T00:22:14.573 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:14.570+0000 7fb9cf200640 1 -- 172.21.15.12:0/4184327961 learned_addr learned my addr 172.21.15.12:0/4184327961 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:22:14.573 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:14.570+0000 7fb9c7e00640 1 --2- 172.21.15.12:0/4184327961 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563a31f8e0c0 0x563a31f955f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:22:14.574 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:14.571+0000 7fb9ce800640 1 -- 172.21.15.12:0/4184327961 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563a31f8e0c0 msgr2=0x563a31f955f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:22:14.574 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:14.571+0000 7fb9ce800640 1 --2- 172.21.15.12:0/4184327961 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563a31f8e0c0 0x563a31f955f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:14.574 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:14.571+0000 7fb9ce800640 1 -- 172.21.15.12:0/4184327961 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563a31f90b00 msgr2=0x563a31f961c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:22:14.574 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:14.571+0000 7fb9ce800640 1 --2- 172.21.15.12:0/4184327961 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563a31f90b00 0x563a31f961c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:14.574 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:14.571+0000 7fb9ce800640 1 -- 172.21.15.12:0/4184327961 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563a31c4f790 con 0x563a31f8b790 2024-09-11T00:22:14.574 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:14.571+0000 7fb9cf200640 1 --2- 172.21.15.12:0/4184327961 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563a31f90b00 0x563a31f961c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:22:14.575 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:14.572+0000 7fb9ce800640 1 --2- 172.21.15.12:0/4184327961 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563a31f8b790 0x563a31f94f50 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fb9c000bb40 tx=0x7fb9c000bf40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b1a4df28b7dcfe9d server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:22:14.575 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:14.572+0000 7fb9b3e00640 10 client.0 ms_handle_connect on v2:172.21.15.178:3300/0 2024-09-11T00:22:14.576 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:14.573+0000 7fb9b3e00640 1 -- 172.21.15.12:0/4184327961 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9c0016070 con 0x563a31f8b790 2024-09-11T00:22:14.576 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:14.573+0000 7fb9c7e00640 1 --2- 172.21.15.12:0/3406712208 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fb9b40b5270 0x7fb9b40b7730 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fb9b802a940 tx=0x7fb9b8009600 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:22:14.576 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:14.573+0000 7fb9b3e00640 1 -- 172.21.15.12:0/4184327961 <== mon.1 v2:172.21.15.178:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9208+0+0 (secure 0 0 0) 0x7fb9c0005020 con 0x563a31f8b790 2024-09-11T00:22:14.576 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:14.574+0000 7fb9d83eb740 1 -- 172.21.15.12:0/4184327961 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb9b8003aa0 con 0x563a31f8b790 2024-09-11T00:22:14.576 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:14.574+0000 7fb9b3e00640 1 -- 172.21.15.12:0/4184327961 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7fb9c0005220 con 0x563a31f8b790 2024-09-11T00:22:14.576 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:14.574+0000 7fb9d83eb740 10 client.27184 fetch_fsmap learned FSMap version 55 2024-09-11T00:22:14.577 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:14.574+0000 7fb9d83eb740 1 -- 172.21.15.12:0/4184327961 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) -- 0x563a31cc8620 con 0x563a31f8b790 2024-09-11T00:22:14.577 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:14.574+0000 7fb9b3e00640 1 -- 172.21.15.12:0/4184327961 <== mon.1 v2:172.21.15.178:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb9b8003aa0 con 0x563a31f8b790 2024-09-11T00:22:14.578 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:14.575+0000 7fb9b3e00640 1 -- 172.21.15.12:0/4184327961 <== mon.1 v2:172.21.15.178:3300/0 5 ==== fsmap(e 55) ==== 8418+0+0 (secure 0 0 0) 0x7fb9c0010460 con 0x563a31f8b790 2024-09-11T00:22:14.578 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:14.575+0000 7fb9d83eb740 10 client.27184 fetch_fsmap finished waiting for FSMap version 55 2024-09-11T00:22:14.578 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:14.575+0000 7fb9d83eb740 10 client.27184 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T00:22:14.578 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:14.575+0000 7fb9d83eb740 20 client.27184 populate_metadata read hostname 'smithi012' 2024-09-11T00:22:14.578 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:14.575+0000 7fb9d83eb740 1 --2- 172.21.15.12:0/4184327961 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x563a31fa0900 0x563a31fa2d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:22:14.578 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:14.575+0000 7fb9d83eb740 4 client.27184 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T00:22:14.579 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:14.575+0000 7fb9d83eb740 1 -- 172.21.15.12:0/4184327961 --> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563a31f00600 con 0x563a31fa0900 2024-09-11T00:22:14.580 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:14.577+0000 7fb9c7e00640 1 --2- 172.21.15.12:0/4184327961 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x563a31fa0900 0x563a31fa2d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:22:14.582 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:14.580+0000 7fb9b3e00640 10 client.27184 ms_handle_connect on v2:172.21.15.165:6838/501312984 2024-09-11T00:22:14.582 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:14.580+0000 7fb9c7e00640 1 --2- 172.21.15.12:0/4184327961 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x563a31fa0900 0x563a31fa2d30 crc :-1 s=READY pgs=223 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=441408aa6e3acb91 server_cookie=bc3da01df2c270f3 in_seq=0 out_seq=0 2024-09-11T00:22:14.583 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:14.581+0000 7fb9b3e00640 1 -- 172.21.15.12:0/4184327961 <== mds.0 v2:172.21.15.165:6838/501312984 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x563a31f00600 con 0x563a31fa0900 2024-09-11T00:22:14.583 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:14.581+0000 7fb9b3e00640 10 client.27184 handle_command_reply: tid=0 2024-09-11T00:22:14.583 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:14.581+0000 7fb9d83eb740 1 client.27184 shutdown 2024-09-11T00:22:14.583 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:14.582+0000 7fb9d83eb740 1 -- 172.21.15.12:0/4184327961 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563a31f8b790 msgr2=0x563a31f94f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:22:14.584 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:14.582+0000 7fb9d83eb740 1 --2- 172.21.15.12:0/4184327961 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563a31f8b790 0x563a31f94f50 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fb9c000bb40 tx=0x7fb9c000bf40 comp rx=0 tx=0).stop 2024-09-11T00:22:14.584 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:14.582+0000 7fb9d83eb740 1 -- 172.21.15.12:0/4184327961 shutdown_connections 2024-09-11T00:22:14.584 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:14.582+0000 7fb9d83eb740 1 --2- 172.21.15.12:0/4184327961 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x563a31fa0900 0x563a31fa2d30 crc :-1 s=READY pgs=223 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:14.584 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:14.582+0000 7fb9d83eb740 1 --2- 172.21.15.12:0/4184327961 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563a31f90b00 0x563a31f961c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:14.585 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:14.582+0000 7fb9d83eb740 1 --2- 172.21.15.12:0/4184327961 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563a31f8e0c0 0x563a31f955f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:14.585 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:14.582+0000 7fb9b3e00640 0 client.27184 ms_handle_reset on v2:172.21.15.165:6838/501312984 2024-09-11T00:22:14.585 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:14.582+0000 7fb9d83eb740 1 --2- 172.21.15.12:0/4184327961 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563a31f8b790 0x563a31f94f50 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:14.585 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:14.582+0000 7fb9d83eb740 1 -- 172.21.15.12:0/4184327961 >> 172.21.15.12:0/4184327961 conn(0x563a31f868e0 msgr2=0x563a31f85a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:22:14.585 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:14.582+0000 7fb9d83eb740 1 -- 172.21.15.12:0/4184327961 shutdown_connections 2024-09-11T00:22:14.586 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:14.582+0000 7fb9d83eb740 1 -- 172.21.15.12:0/4184327961 wait complete. 2024-09-11T00:22:14.586 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:14.583+0000 7fb9d83eb740 20 client.27184 trim_cache size 0 max 16384 2024-09-11T00:22:14.624 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:14.622+0000 7fb9d83eb740 1 Processor -- start 2024-09-11T00:22:14.624 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:14.622+0000 7fb9d83eb740 1 -- start start 2024-09-11T00:22:14.625 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:14.622+0000 7fb9d83eb740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563a31f8b790 0x563a31f90ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:22:14.625 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:14.622+0000 7fb9d83eb740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563a31f8e0c0 0x563a31f91420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:22:14.625 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:14.623+0000 7fb9d83eb740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563a31f90b00 0x563a31f93e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:22:14.626 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:14.623+0000 7fb9d83eb740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fb9d01ba540 con 0x563a31f8e0c0 2024-09-11T00:22:14.626 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:14.623+0000 7fb9d83eb740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fb9d01bc260 con 0x563a31f8b790 2024-09-11T00:22:14.626 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:14.623+0000 7fb9d83eb740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fb9d01bac70 con 0x563a31f90b00 2024-09-11T00:22:14.626 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:14.623+0000 7fb9ce800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563a31f8b790 0x563a31f90ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:22:14.626 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:14.623+0000 7fb9cf200640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563a31f90b00 0x563a31f93e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:22:14.626 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:14.623+0000 7fb9cf200640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563a31f90b00 0x563a31f93e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.12:36430/0 (socket says 172.21.15.12:36430) 2024-09-11T00:22:14.626 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:14.623+0000 7fb9cf200640 1 -- 172.21.15.12:0/3976826567 learned_addr learned my addr 172.21.15.12:0/3976826567 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:22:14.626 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:14.623+0000 7fb9c7e00640 1 --2- 172.21.15.12:0/3976826567 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563a31f8e0c0 0x563a31f91420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:22:14.627 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:14.624+0000 7fb9cf200640 1 -- 172.21.15.12:0/3976826567 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563a31f8b790 msgr2=0x563a31f90ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:22:14.627 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:14.624+0000 7fb9cf200640 1 --2- 172.21.15.12:0/3976826567 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563a31f8b790 0x563a31f90ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:14.627 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:14.624+0000 7fb9cf200640 1 -- 172.21.15.12:0/3976826567 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563a31f8e0c0 msgr2=0x563a31f91420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:22:14.627 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:14.624+0000 7fb9cf200640 1 --2- 172.21.15.12:0/3976826567 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563a31f8e0c0 0x563a31f91420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:14.627 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:14.624+0000 7fb9cf200640 1 -- 172.21.15.12:0/3976826567 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563a31f8def0 con 0x563a31f90b00 2024-09-11T00:22:14.627 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:14.624+0000 7fb9cf200640 1 --2- 172.21.15.12:0/3976826567 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563a31f90b00 0x563a31f93e50 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fb9c80e3c50 tx=0x7fb9c80058a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=56b0fec9a18c2244 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:22:14.628 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:14.624+0000 7fb9bea00640 1 -- 172.21.15.12:0/3976826567 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9c80ac030 con 0x563a31f90b00 2024-09-11T00:22:14.628 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:14.624+0000 7fb9bea00640 1 -- 172.21.15.12:0/3976826567 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb9c80b1070 con 0x563a31f90b00 2024-09-11T00:22:14.628 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:14.624+0000 7fb9bea00640 1 -- 172.21.15.12:0/3976826567 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9c80b5e50 con 0x563a31f90b00 2024-09-11T00:22:14.628 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:14.625+0000 7fb9d83eb740 1 -- 172.21.15.12:0/3976826567 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563a31f90b00 msgr2=0x563a31f93e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:22:14.629 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:14.625+0000 7fb9d83eb740 1 --2- 172.21.15.12:0/3976826567 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563a31f90b00 0x563a31f93e50 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fb9c80e3c50 tx=0x7fb9c80058a0 comp rx=0 tx=0).stop 2024-09-11T00:22:14.629 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:14.625+0000 7fb9d83eb740 1 -- 172.21.15.12:0/3976826567 shutdown_connections 2024-09-11T00:22:14.629 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:14.625+0000 7fb9d83eb740 1 --2- 172.21.15.12:0/3976826567 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563a31f90b00 0x563a31f93e50 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:14.629 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:14.625+0000 7fb9d83eb740 1 --2- 172.21.15.12:0/3976826567 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563a31f8e0c0 0x563a31f91420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:14.629 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:14.625+0000 7fb9d83eb740 1 --2- 172.21.15.12:0/3976826567 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563a31f8b790 0x563a31f90ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:14.629 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:14.625+0000 7fb9d83eb740 1 -- 172.21.15.12:0/3976826567 >> 172.21.15.12:0/3976826567 conn(0x563a31f868e0 msgr2=0x563a31f87140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:22:14.629 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:14.626+0000 7fb9d83eb740 1 -- 172.21.15.12:0/3976826567 shutdown_connections 2024-09-11T00:22:14.629 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:14.626+0000 7fb9d83eb740 1 -- 172.21.15.12:0/3976826567 wait complete. 2024-09-11T00:22:14.629 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:14.626+0000 7fb9d83eb740 1 -- start start 2024-09-11T00:22:14.629 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:14.626+0000 7fb9d83eb740 1 Processor -- start 2024-09-11T00:22:14.631 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:14.627+0000 7fb9d83eb740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563a31f8b790 0x563a31f9cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:22:14.631 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:14.627+0000 7fb9d83eb740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563a31f8e0c0 0x563a31f9b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:22:14.631 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:14.627+0000 7fb9d83eb740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563a31f90b00 0x563a31f9c0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:22:14.631 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:14.627+0000 7fb9d83eb740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fb9d01bbb90 con 0x563a31f8b790 2024-09-11T00:22:14.631 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:14.627+0000 7fb9d83eb740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fb9d01bb400 con 0x563a31f8e0c0 2024-09-11T00:22:14.632 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:14.627+0000 7fb9d83eb740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fb9d01a14d0 con 0x563a31f90b00 2024-09-11T00:22:14.632 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:14.627+0000 7fb9d83eb740 10 client.0 _ll_register_callbacks cb 0x7fb9c0005f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:22:14.632 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:14.627+0000 7fb9cf200640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563a31f90b00 0x563a31f9c0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:22:14.632 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:14.627+0000 7fb9cf200640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563a31f90b00 0x563a31f9c0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.12:36440/0 (socket says 172.21.15.12:36440) 2024-09-11T00:22:14.632 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:14.627+0000 7fb9cf200640 1 -- 172.21.15.12:0/2443241678 learned_addr learned my addr 172.21.15.12:0/2443241678 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:22:14.632 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:14.628+0000 7fb9ce800640 1 --2- 172.21.15.12:0/2443241678 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563a31f8b790 0x563a31f9cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:22:14.632 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:14.628+0000 7fb9cf200640 1 -- 172.21.15.12:0/2443241678 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563a31f8e0c0 msgr2=0x563a31f9b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:22:14.632 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:14.628+0000 7fb9cf200640 1 --2- 172.21.15.12:0/2443241678 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563a31f8e0c0 0x563a31f9b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:14.632 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:14.628+0000 7fb9cf200640 1 -- 172.21.15.12:0/2443241678 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563a31f8b790 msgr2=0x563a31f9cbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:22:14.632 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:14.628+0000 7fb9cf200640 1 --2- 172.21.15.12:0/2443241678 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563a31f8b790 0x563a31f9cbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:14.632 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:14.628+0000 7fb9cf200640 1 -- 172.21.15.12:0/2443241678 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563a31f9a450 con 0x563a31f90b00 2024-09-11T00:22:14.632 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:14.628+0000 7fb9cf200640 1 --2- 172.21.15.12:0/2443241678 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563a31f90b00 0x563a31f9c0a0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fb9c80aca80 tx=0x7fb9c8021000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=40758004427a55df server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:22:14.632 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:14.628+0000 7fb9b3e00640 10 client.0 ms_handle_connect on v2:172.21.15.178:3300/0 2024-09-11T00:22:14.633 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:14.629+0000 7fb9b3e00640 1 -- 172.21.15.12:0/2443241678 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9c80cc060 con 0x563a31f90b00 2024-09-11T00:22:14.633 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:14.629+0000 7fb9b3e00640 1 -- 172.21.15.12:0/2443241678 <== mon.1 v2:172.21.15.178:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9208+0+0 (secure 0 0 0) 0x7fb9c80abb20 con 0x563a31f90b00 2024-09-11T00:22:14.633 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:14.629+0000 7fb9d83eb740 1 -- 172.21.15.12:0/2443241678 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb9c80b1070 con 0x563a31f90b00 2024-09-11T00:22:14.633 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:14.630+0000 7fb9b3e00640 1 -- 172.21.15.12:0/2443241678 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7fb9c80ac030 con 0x563a31f90b00 2024-09-11T00:22:14.634 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:14.630+0000 7fb9d83eb740 10 client.27190 fetch_fsmap learned FSMap version 55 2024-09-11T00:22:14.634 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:14.630+0000 7fb9d83eb740 1 -- 172.21.15.12:0/2443241678 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) -- 0x563a3200dae0 con 0x563a31f90b00 2024-09-11T00:22:14.634 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:14.630+0000 7fb9b3e00640 1 -- 172.21.15.12:0/2443241678 <== mon.1 v2:172.21.15.178:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb9c80b1070 con 0x563a31f90b00 2024-09-11T00:22:14.634 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:14.632+0000 7fb9b3e00640 1 -- 172.21.15.12:0/2443241678 <== mon.1 v2:172.21.15.178:3300/0 5 ==== fsmap(e 55) ==== 8418+0+0 (secure 0 0 0) 0x7fb9c80ab590 con 0x563a31f90b00 2024-09-11T00:22:14.635 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:14.633+0000 7fb9d83eb740 10 client.27190 fetch_fsmap finished waiting for FSMap version 55 2024-09-11T00:22:14.635 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:14.633+0000 7fb9d83eb740 10 client.27190 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T00:22:14.635 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:14.633+0000 7fb9d83eb740 20 client.27190 populate_metadata read hostname 'smithi012' 2024-09-11T00:22:14.635 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:14.633+0000 7fb9d83eb740 1 --2- 172.21.15.12:0/2443241678 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x563a3200e460 0x563a32010880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:22:14.635 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:14.633+0000 7fb9d83eb740 4 client.27190 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T00:22:14.635 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:14.633+0000 7fb9d83eb740 1 -- 172.21.15.12:0/2443241678 --> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] -- command(tid 0: {"prefix": "flush journal"}) -- 0x563a31efc0b0 con 0x563a3200e460 2024-09-11T00:22:14.636 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:14.634+0000 7fb9ce800640 1 --2- 172.21.15.12:0/2443241678 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x563a3200e460 0x563a32010880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:22:14.638 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:14.635+0000 7fb9ce800640 1 --2- 172.21.15.12:0/2443241678 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x563a3200e460 0x563a32010880 crc :-1 s=READY pgs=224 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1bd5236e1cbe27a5 server_cookie=3e5deb3796846976 in_seq=0 out_seq=0 2024-09-11T00:22:14.638 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:14.635+0000 7fb9b3e00640 10 client.27190 ms_handle_connect on v2:172.21.15.165:6838/501312984 2024-09-11T00:22:14.888 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:14.886+0000 7fb9b3e00640 1 -- 172.21.15.12:0/2443241678 <== mds.0 v2:172.21.15.165:6838/501312984 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x563a31efc0b0 con 0x563a3200e460 2024-09-11T00:22:14.889 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:14.886+0000 7fb9b3e00640 10 client.27190 handle_command_reply: tid=0 2024-09-11T00:22:14.889 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:14.886+0000 7fb9d83eb740 1 client.27190 shutdown 2024-09-11T00:22:14.889 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:14.886+0000 7fb9d83eb740 1 -- 172.21.15.12:0/2443241678 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563a31f90b00 msgr2=0x563a31f9c0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:22:14.889 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:14.886+0000 7fb9d83eb740 1 --2- 172.21.15.12:0/2443241678 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563a31f90b00 0x563a31f9c0a0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fb9c80aca80 tx=0x7fb9c8021000 comp rx=0 tx=0).stop 2024-09-11T00:22:14.889 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:14.887+0000 7fb9d83eb740 1 -- 172.21.15.12:0/2443241678 shutdown_connections 2024-09-11T00:22:14.890 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:14.887+0000 7fb9d83eb740 1 --2- 172.21.15.12:0/2443241678 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x563a3200e460 0x563a32010880 crc :-1 s=READY pgs=224 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:14.890 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:14.887+0000 7fb9d83eb740 1 --2- 172.21.15.12:0/2443241678 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563a31f90b00 0x563a31f9c0a0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:14.890 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:14.887+0000 7fb9d83eb740 1 --2- 172.21.15.12:0/2443241678 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563a31f8e0c0 0x563a31f9b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:14.890 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:14.887+0000 7fb9d83eb740 1 --2- 172.21.15.12:0/2443241678 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563a31f8b790 0x563a31f9cbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:14.890 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:14.887+0000 7fb9d83eb740 1 -- 172.21.15.12:0/2443241678 >> 172.21.15.12:0/2443241678 conn(0x563a31f868e0 msgr2=0x563a31f93e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:22:14.890 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:14.887+0000 7fb9b3e00640 0 client.27190 ms_handle_reset on v2:172.21.15.165:6838/501312984 2024-09-11T00:22:14.890 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:14.888+0000 7fb9d83eb740 1 -- 172.21.15.12:0/2443241678 shutdown_connections 2024-09-11T00:22:14.891 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:14.888+0000 7fb9d83eb740 1 -- 172.21.15.12:0/2443241678 wait complete. 2024-09-11T00:22:14.891 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:14.888+0000 7fb9d83eb740 20 client.27190 trim_cache size 0 max 16384 2024-09-11T00:22:14.891 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:{ 2024-09-11T00:22:14.891 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "message": "", 2024-09-11T00:22:14.891 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "return_code": 0 2024-09-11T00:22:14.891 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:} 2024-09-11T00:22:14.892 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:14.889+0000 7fb9d5400640 1 -- 172.21.15.12:0/3406712208 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fb9b40b5270 msgr2=0x7fb9b40b7730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:22:14.892 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:14.889+0000 7fb9d5400640 1 --2- 172.21.15.12:0/3406712208 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fb9b40b5270 0x7fb9b40b7730 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fb9b802a940 tx=0x7fb9b8009600 comp rx=0 tx=0).stop 2024-09-11T00:22:14.893 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:14.889+0000 7fb9d5400640 1 -- 172.21.15.12:0/3406712208 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb9d01066b0 msgr2=0x7fb9d0078c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:22:14.893 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:14.889+0000 7fb9d5400640 1 --2- 172.21.15.12:0/3406712208 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb9d01066b0 0x7fb9d0078c30 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7fb9c800eb10 tx=0x7fb9c800c2d0 comp rx=0 tx=0).stop 2024-09-11T00:22:14.893 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:14.890+0000 7fb9d5400640 1 -- 172.21.15.12:0/3406712208 shutdown_connections 2024-09-11T00:22:14.893 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:14.890+0000 7fb9d5400640 1 --2- 172.21.15.12:0/3406712208 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fb9b40b5270 0x7fb9b40b7730 secure :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fb9b802a940 tx=0x7fb9b8009600 comp rx=0 tx=0).stop 2024-09-11T00:22:14.893 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:14.890+0000 7fb9d5400640 1 --2- 172.21.15.12:0/3406712208 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb9d01066b0 0x7fb9d0078c30 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:14.894 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:14.890+0000 7fb9d5400640 1 --2- 172.21.15.12:0/3406712208 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb9d0105cf0 0x7fb9d0078000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:14.894 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:14.890+0000 7fb9d5400640 1 --2- 172.21.15.12:0/3406712208 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb9d0104ba0 0x7fb9d00779b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:14.894 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:14.890+0000 7fb9d5400640 1 -- 172.21.15.12:0/3406712208 >> 172.21.15.12:0/3406712208 conn(0x7fb9d0100440 msgr2=0x7fb9d0109080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:22:14.894 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:14.890+0000 7fb9d5400640 1 -- 172.21.15.12:0/3406712208 shutdown_connections 2024-09-11T00:22:14.894 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:14.891+0000 7fb9d5400640 1 -- 172.21.15.12:0/3406712208 wait complete. 2024-09-11T00:22:15.167 INFO:tasks.workunit.client.0.smithi012.stdout: 10 41716 42.26 MB/sec execute 11 sec latency 159.612 ms 2024-09-11T00:22:15.813 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:22:15 smithi012 ceph-mon[30333]: pgmap v854: 129 pgs: 129 active+clean; 408 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 2.5 MiB/s wr, 332 op/s 2024-09-11T00:22:15.831 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:22:15 smithi178 ceph-mon[35919]: pgmap v854: 129 pgs: 129 active+clean; 408 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 2.5 MiB/s wr, 332 op/s 2024-09-11T00:22:15.834 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:22:15 smithi165 ceph-mon[35356]: pgmap v854: 129 pgs: 129 active+clean; 408 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 2.5 MiB/s wr, 332 op/s 2024-09-11T00:22:16.167 INFO:tasks.workunit.client.0.smithi012.stdout: 10 42822 43.21 MB/sec execute 12 sec latency 68.545 ms 2024-09-11T00:22:17.167 INFO:tasks.workunit.client.0.smithi012.stdout: 10 43987 45.43 MB/sec execute 13 sec latency 50.347 ms 2024-09-11T00:22:17.814 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:22:17 smithi012 ceph-mon[30333]: pgmap v855: 129 pgs: 129 active+clean; 420 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 4.1 MiB/s wr, 572 op/s 2024-09-11T00:22:17.831 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:22:17 smithi178 ceph-mon[35919]: pgmap v855: 129 pgs: 129 active+clean; 420 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 4.1 MiB/s wr, 572 op/s 2024-09-11T00:22:18.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:22:17 smithi165 ceph-mon[35356]: pgmap v855: 129 pgs: 129 active+clean; 420 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 4.1 MiB/s wr, 572 op/s 2024-09-11T00:22:18.167 INFO:tasks.workunit.client.0.smithi012.stdout: 10 44671 45.16 MB/sec execute 14 sec latency 143.218 ms 2024-09-11T00:22:18.813 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:22:18 smithi012 ceph-mon[30333]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T00:22:18.831 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:22:18 smithi178 ceph-mon[35919]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T00:22:19.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:22:18 smithi165 ceph-mon[35356]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T00:22:19.167 INFO:tasks.workunit.client.0.smithi012.stdout: 10 45379 44.07 MB/sec execute 15 sec latency 67.173 ms 2024-09-11T00:22:19.831 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:22:19 smithi178 ceph-mon[35919]: pgmap v856: 129 pgs: 129 active+clean; 424 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 4.4 MiB/s wr, 605 op/s 2024-09-11T00:22:20.036 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:22:19 smithi165 ceph-mon[35356]: pgmap v856: 129 pgs: 129 active+clean; 424 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 4.4 MiB/s wr, 605 op/s 2024-09-11T00:22:20.064 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:22:19 smithi012 ceph-mon[30333]: pgmap v856: 129 pgs: 129 active+clean; 424 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 4.4 MiB/s wr, 605 op/s 2024-09-11T00:22:20.168 INFO:tasks.workunit.client.0.smithi012.stdout: 10 46512 44.84 MB/sec execute 16 sec latency 54.644 ms 2024-09-11T00:22:21.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:22:20 smithi165 ceph-mon[35356]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T00:22:21.063 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:22:20 smithi012 ceph-mon[30333]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T00:22:21.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:22:20 smithi178 ceph-mon[35919]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T00:22:21.167 INFO:tasks.workunit.client.0.smithi012.stdout: 10 47567 45.89 MB/sec execute 17 sec latency 47.600 ms 2024-09-11T00:22:22.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:22:21 smithi165 ceph-mon[35356]: pgmap v857: 129 pgs: 129 active+clean; 430 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 5.6 MiB/s wr, 750 op/s 2024-09-11T00:22:22.063 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:22:21 smithi012 ceph-mon[30333]: pgmap v857: 129 pgs: 129 active+clean; 430 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 5.6 MiB/s wr, 750 op/s 2024-09-11T00:22:22.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:22:21 smithi178 ceph-mon[35919]: pgmap v857: 129 pgs: 129 active+clean; 430 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 5.6 MiB/s wr, 750 op/s 2024-09-11T00:22:22.167 INFO:tasks.workunit.client.0.smithi012.stdout: 10 48424 46.05 MB/sec execute 18 sec latency 76.784 ms 2024-09-11T00:22:23.168 INFO:tasks.workunit.client.0.smithi012.stdout: 10 48584 44.15 MB/sec execute 19 sec latency 864.147 ms 2024-09-11T00:22:24.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:22:23 smithi165 ceph-mon[35356]: pgmap v858: 129 pgs: 129 active+clean; 440 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 5.7 MiB/s wr, 788 op/s 2024-09-11T00:22:24.063 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:22:23 smithi012 ceph-mon[30333]: pgmap v858: 129 pgs: 129 active+clean; 440 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 5.7 MiB/s wr, 788 op/s 2024-09-11T00:22:24.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:22:23 smithi178 ceph-mon[35919]: pgmap v858: 129 pgs: 129 active+clean; 440 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 5.7 MiB/s wr, 788 op/s 2024-09-11T00:22:24.168 INFO:tasks.workunit.client.0.smithi012.stdout: 10 48584 41.94 MB/sec execute 20 sec latency 1864.216 ms 2024-09-11T00:22:25.168 INFO:tasks.workunit.client.0.smithi012.stdout: 10 48742 40.45 MB/sec execute 21 sec latency 2607.825 ms 2024-09-11T00:22:26.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:22:25 smithi165 ceph-mon[35356]: pgmap v859: 129 pgs: 129 active+clean; 441 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 5.2 MiB/s wr, 670 op/s 2024-09-11T00:22:26.042 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:22:25 smithi012 ceph-mon[30333]: pgmap v859: 129 pgs: 129 active+clean; 441 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 5.2 MiB/s wr, 670 op/s 2024-09-11T00:22:26.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:22:25 smithi178 ceph-mon[35919]: pgmap v859: 129 pgs: 129 active+clean; 441 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 5.2 MiB/s wr, 670 op/s 2024-09-11T00:22:26.168 INFO:tasks.workunit.client.0.smithi012.stdout: 10 49770 40.67 MB/sec execute 22 sec latency 2632.660 ms 2024-09-11T00:22:27.168 INFO:tasks.workunit.client.0.smithi012.stdout: 10 50940 41.93 MB/sec execute 23 sec latency 60.875 ms 2024-09-11T00:22:27.917 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:22:27 smithi012 ceph-mon[30333]: pgmap v860: 129 pgs: 129 active+clean; 449 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 7.1 MiB/s wr, 781 op/s 2024-09-11T00:22:28.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:22:27 smithi165 ceph-mon[35356]: pgmap v860: 129 pgs: 129 active+clean; 449 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 7.1 MiB/s wr, 781 op/s 2024-09-11T00:22:28.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:22:27 smithi178 ceph-mon[35919]: pgmap v860: 129 pgs: 129 active+clean; 449 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 7.1 MiB/s wr, 781 op/s 2024-09-11T00:22:28.122 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:28.118+0000 7f29eda00640 1 -- 172.21.15.12:0/2101310184 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f29e8103ae0 msgr2=0x7f29e8103f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:22:28.122 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:28.118+0000 7f29eda00640 1 --2- 172.21.15.12:0/2101310184 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f29e8103ae0 0x7f29e8103f60 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f29d4009890 tx=0x7f29d402a360 comp rx=0 tx=0).stop 2024-09-11T00:22:28.122 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:28.118+0000 7f29eda00640 1 -- 172.21.15.12:0/2101310184 shutdown_connections 2024-09-11T00:22:28.122 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:28.118+0000 7f29eda00640 1 --2- 172.21.15.12:0/2101310184 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f29e81044a0 0x7f29e810ae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:28.122 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:28.118+0000 7f29eda00640 1 --2- 172.21.15.12:0/2101310184 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f29e8103ae0 0x7f29e8103f60 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:28.123 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:28.118+0000 7f29eda00640 1 --2- 172.21.15.12:0/2101310184 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f29e8102990 0x7f29e8102d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:28.123 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:28.118+0000 7f29eda00640 1 -- 172.21.15.12:0/2101310184 >> 172.21.15.12:0/2101310184 conn(0x7f29e80fe250 msgr2=0x7f29e8100670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:22:28.123 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:28.118+0000 7f29eda00640 1 -- 172.21.15.12:0/2101310184 shutdown_connections 2024-09-11T00:22:28.123 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:28.118+0000 7f29eda00640 1 -- 172.21.15.12:0/2101310184 wait complete. 2024-09-11T00:22:28.123 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:28.118+0000 7f29eda00640 1 Processor -- start 2024-09-11T00:22:28.124 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:28.118+0000 7f29eda00640 1 -- start start 2024-09-11T00:22:28.124 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:28.118+0000 7f29eda00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f29e8102990 0x7f29e8076ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:22:28.124 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:28.118+0000 7f29eda00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f29e8103ae0 0x7f29e8075540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:22:28.124 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:28.118+0000 7f29eda00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f29e81044a0 0x7f29e8075a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:22:28.125 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:28.118+0000 7f29eda00640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f29e810e660 con 0x7f29e8103ae0 2024-09-11T00:22:28.125 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:28.118+0000 7f29eda00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f29e810e4e0 con 0x7f29e8102990 2024-09-11T00:22:28.125 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:28.118+0000 7f29eda00640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f29e810e7e0 con 0x7f29e81044a0 2024-09-11T00:22:28.125 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:28.123+0000 7f29e6000640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f29e8103ae0 0x7f29e8075540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:22:28.125 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:28.123+0000 7f29e7400640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f29e81044a0 0x7f29e8075a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:22:28.125 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:28.123+0000 7f29e7400640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f29e81044a0 0x7f29e8075a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.12:58416/0 (socket says 172.21.15.12:58416) 2024-09-11T00:22:28.125 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:28.123+0000 7f29e7400640 1 -- 172.21.15.12:0/2366127360 learned_addr learned my addr 172.21.15.12:0/2366127360 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:22:28.128 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:28.123+0000 7f29e7400640 1 -- 172.21.15.12:0/2366127360 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f29e8102990 msgr2=0x7f29e8076ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:22:28.128 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:28.123+0000 7f29e7400640 1 --2- 172.21.15.12:0/2366127360 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f29e8102990 0x7f29e8076ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:28.128 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:28.123+0000 7f29e7400640 1 -- 172.21.15.12:0/2366127360 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f29e8103ae0 msgr2=0x7f29e8075540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:22:28.128 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:28.123+0000 7f29e7400640 1 --2- 172.21.15.12:0/2366127360 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f29e8103ae0 0x7f29e8075540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:28.128 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:28.123+0000 7f29e7400640 1 -- 172.21.15.12:0/2366127360 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29e80762f0 con 0x7f29e81044a0 2024-09-11T00:22:28.128 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:28.123+0000 7f29e7400640 1 --2- 172.21.15.12:0/2366127360 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f29e81044a0 0x7f29e8075a80 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f29e000b970 tx=0x7f29e000be30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:22:28.128 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:28.123+0000 7f29df400640 1 -- 172.21.15.12:0/2366127360 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29e0019070 con 0x7f29e81044a0 2024-09-11T00:22:28.128 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:28.123+0000 7f29df400640 1 -- 172.21.15.12:0/2366127360 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f29e0014440 con 0x7f29e81044a0 2024-09-11T00:22:28.128 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:28.123+0000 7f29df400640 1 -- 172.21.15.12:0/2366127360 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29e00075b0 con 0x7f29e81044a0 2024-09-11T00:22:28.129 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:28.123+0000 7f29eda00640 1 -- 172.21.15.12:0/2366127360 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f29e81a2ea0 con 0x7f29e81044a0 2024-09-11T00:22:28.129 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:28.123+0000 7f29eda00640 1 -- 172.21.15.12:0/2366127360 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29e81a3370 con 0x7f29e81044a0 2024-09-11T00:22:28.135 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:28.133+0000 7f29f07f9740 1 Processor -- start 2024-09-11T00:22:28.148 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:28.146+0000 7f29df400640 1 -- 172.21.15.12:0/2366127360 <== mon.1 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7f29e00077d0 con 0x7f29e81044a0 2024-09-11T00:22:28.150 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:28.148+0000 7f29f07f9740 1 -- start start 2024-09-11T00:22:28.154 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:28.148+0000 7f29f07f9740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e836baa790 0x55e836bacb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:22:28.154 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:28.148+0000 7f29f07f9740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e836bad0c0 0x55e836baf4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:22:28.154 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:28.148+0000 7f29f07f9740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e836bafb00 0x55e836bb1fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:22:28.154 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:28.148+0000 7f29f07f9740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55e836b1d6b0 con 0x55e836bad0c0 2024-09-11T00:22:28.154 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:28.148+0000 7f29f07f9740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55e836b0cae0 con 0x55e836bafb00 2024-09-11T00:22:28.154 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:28.148+0000 7f29f07f9740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55e836b108f0 con 0x55e836baa790 2024-09-11T00:22:28.155 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:28.148+0000 7f29e6a00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e836bad0c0 0x55e836baf4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:22:28.155 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:28.148+0000 7f29e6000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e836bafb00 0x55e836bb1fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:22:28.155 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:28.148+0000 7f29e6000640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e836baa790 0x55e836bacb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:22:28.155 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:28.148+0000 7f29e6000640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e836baa790 0x55e836bacb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.12:58430/0 (socket says 172.21.15.12:58430) 2024-09-11T00:22:28.155 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:28.148+0000 7f29e6000640 1 -- 172.21.15.12:0/1659145475 learned_addr learned my addr 172.21.15.12:0/1659145475 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:22:28.155 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:28.148+0000 7f29df400640 1 --2- 172.21.15.12:0/2366127360 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f29cc0b5360 0x7f29cc0b7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:22:28.155 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:28.148+0000 7f29df400640 1 -- 172.21.15.12:0/2366127360 <== mon.1 v2:172.21.15.178:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9208+0+0 (secure 0 0 0) 0x7f29e00e5700 con 0x7f29e81044a0 2024-09-11T00:22:28.155 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:28.148+0000 7f29e7400640 1 --2- 172.21.15.12:0/2366127360 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f29cc0b5360 0x7f29cc0b7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:22:28.155 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:28.148+0000 7f29e6000640 1 -- 172.21.15.12:0/1659145475 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e836bafb00 msgr2=0x55e836bb1fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:22:28.156 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:28.148+0000 7f29e6000640 1 --2- 172.21.15.12:0/1659145475 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e836bafb00 0x55e836bb1fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:28.156 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:28.148+0000 7f29e6000640 1 -- 172.21.15.12:0/1659145475 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e836bad0c0 msgr2=0x55e836baf4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:22:28.156 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:28.148+0000 7f29e6000640 1 --2- 172.21.15.12:0/1659145475 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e836bad0c0 0x55e836baf4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:28.156 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:28.148+0000 7f29e6000640 1 -- 172.21.15.12:0/1659145475 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e836ad9780 con 0x55e836baa790 2024-09-11T00:22:28.156 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:28.148+0000 7f29e7400640 1 --2- 172.21.15.12:0/2366127360 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f29cc0b5360 0x7f29cc0b7820 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f29e00ab7f0 tx=0x7f29e0004a40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:22:28.156 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:28.148+0000 7f29e6a00640 1 --2- 172.21.15.12:0/1659145475 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e836bad0c0 0x55e836baf4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:22:28.157 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:28.153+0000 7f29e6000640 1 --2- 172.21.15.12:0/1659145475 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e836baa790 0x55e836bacb80 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f29d4009890 tx=0x7f29d402a360 comp rx=0 tx=0).ready entity=mon.1 client_cookie=77c88f3ec1486853 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:22:28.157 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:28.153+0000 7f29d3e00640 1 -- 172.21.15.12:0/1659145475 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29d4043070 con 0x55e836baa790 2024-09-11T00:22:28.158 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:28.153+0000 7f29d3e00640 1 -- 172.21.15.12:0/1659145475 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f29d4030d20 con 0x55e836baa790 2024-09-11T00:22:28.158 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:28.153+0000 7f29f07f9740 1 -- 172.21.15.12:0/1659145475 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e836baa790 msgr2=0x55e836bacb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:22:28.158 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:28.153+0000 7f29f07f9740 1 --2- 172.21.15.12:0/1659145475 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e836baa790 0x55e836bacb80 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f29d4009890 tx=0x7f29d402a360 comp rx=0 tx=0).stop 2024-09-11T00:22:28.158 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:28.153+0000 7f29f07f9740 1 -- 172.21.15.12:0/1659145475 shutdown_connections 2024-09-11T00:22:28.158 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:28.153+0000 7f29f07f9740 1 --2- 172.21.15.12:0/1659145475 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e836bafb00 0x55e836bb1fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:28.158 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:28.153+0000 7f29f07f9740 1 --2- 172.21.15.12:0/1659145475 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e836bad0c0 0x55e836baf4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:28.158 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:28.153+0000 7f29f07f9740 1 --2- 172.21.15.12:0/1659145475 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e836baa790 0x55e836bacb80 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:28.158 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:28.153+0000 7f29f07f9740 1 -- 172.21.15.12:0/1659145475 >> 172.21.15.12:0/1659145475 conn(0x55e836ba58e0 msgr2=0x55e836ba6670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:22:28.159 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:28.153+0000 7f29f07f9740 1 -- 172.21.15.12:0/1659145475 shutdown_connections 2024-09-11T00:22:28.159 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:28.153+0000 7f29f07f9740 1 -- 172.21.15.12:0/1659145475 wait complete. 2024-09-11T00:22:28.160 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:28.158+0000 7f29f07f9740 1 -- start start 2024-09-11T00:22:28.161 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:28.158+0000 7f29f07f9740 1 Processor -- start 2024-09-11T00:22:28.163 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:28.160+0000 7f29f07f9740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e836baa790 0x55e836bb3f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:22:28.163 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:28.160+0000 7f29f07f9740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e836bad0c0 0x55e836bb45c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:22:28.163 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:28.160+0000 7f29f07f9740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e836bafb00 0x55e836bb5190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:22:28.163 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:28.160+0000 7f29f07f9740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55e836a44ab0 con 0x55e836baa790 2024-09-11T00:22:28.163 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:28.160+0000 7f29f07f9740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55e836ab3570 con 0x55e836bad0c0 2024-09-11T00:22:28.163 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:28.160+0000 7f29f07f9740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55e836bb2b20 con 0x55e836bafb00 2024-09-11T00:22:28.164 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:28.160+0000 7f29f07f9740 10 client.0 _ll_register_callbacks cb 0x55e836ac2100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:22:28.164 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:28.160+0000 7f29e6a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e836bad0c0 0x55e836bb45c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:22:28.164 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:28.160+0000 7f29e6a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e836bad0c0 0x55e836bb45c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.12:56236/0 (socket says 172.21.15.12:56236) 2024-09-11T00:22:28.164 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:28.160+0000 7f29e6a00640 1 -- 172.21.15.12:0/1872018649 learned_addr learned my addr 172.21.15.12:0/1872018649 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:22:28.165 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:28.160+0000 7f29e6000640 1 --2- 172.21.15.12:0/1872018649 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e836baa790 0x55e836bb3f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:22:28.165 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:28.160+0000 7f29e6000640 1 --2- 172.21.15.12:0/1872018649 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e836bafb00 0x55e836bb5190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:22:28.167 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:28.160+0000 7f29e6a00640 1 -- 172.21.15.12:0/1872018649 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e836bafb00 msgr2=0x55e836bb5190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:22:28.167 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:28.160+0000 7f29e6a00640 1 --2- 172.21.15.12:0/1872018649 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e836bafb00 0x55e836bb5190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:28.167 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:28.160+0000 7f29e6a00640 1 -- 172.21.15.12:0/1872018649 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e836baa790 msgr2=0x55e836bb3f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:22:28.167 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:28.160+0000 7f29e6a00640 1 --2- 172.21.15.12:0/1872018649 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e836baa790 0x55e836bb3f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:28.167 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:28.160+0000 7f29e6a00640 1 -- 172.21.15.12:0/1872018649 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e83686e790 con 0x55e836bad0c0 2024-09-11T00:22:28.167 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:28.165+0000 7f29e6000640 1 --2- 172.21.15.12:0/1872018649 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e836bafb00 0x55e836bb5190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T00:22:28.168 INFO:tasks.workunit.client.0.smithi012.stdout: 10 51178 40.75 MB/sec execute 24 sec latency 771.051 ms 2024-09-11T00:22:28.168 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:28.165+0000 7f29e6a00640 1 --2- 172.21.15.12:0/1872018649 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e836bad0c0 0x55e836bb45c0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f29d8009490 tx=0x7f29d8011b00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2c5943107f1f850f server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:22:28.168 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:28.165+0000 7f29d0c00640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-09-11T00:22:28.170 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:28.165+0000 7f29d0c00640 1 -- 172.21.15.12:0/1872018649 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29d801e060 con 0x55e836bad0c0 2024-09-11T00:22:28.170 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:28.165+0000 7f29d0c00640 1 -- 172.21.15.12:0/1872018649 <== mon.2 v2:172.21.15.165:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9208+0+0 (secure 0 0 0) 0x7f29d800f880 con 0x55e836bad0c0 2024-09-11T00:22:28.170 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:28.165+0000 7f29f07f9740 1 -- 172.21.15.12:0/1872018649 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f29d4030d20 con 0x55e836bad0c0 2024-09-11T00:22:28.171 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:28.165+0000 7f29d0c00640 1 -- 172.21.15.12:0/1872018649 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=57) ==== 24+0+0 (secure 0 0 0) 0x7f29d800fa80 con 0x55e836bad0c0 2024-09-11T00:22:28.171 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:28.165+0000 7f29f07f9740 10 client.27236 fetch_fsmap learned FSMap version 57 2024-09-11T00:22:28.171 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:28.165+0000 7f29f07f9740 1 -- 172.21.15.12:0/1872018649 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=57,monmap=4+}) -- 0x55e8368e7620 con 0x55e836bad0c0 2024-09-11T00:22:28.171 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:28.165+0000 7f29d0c00640 1 -- 172.21.15.12:0/1872018649 <== mon.2 v2:172.21.15.165:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f29d4030d20 con 0x55e836bad0c0 2024-09-11T00:22:28.172 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:28.165+0000 7f29d0c00640 1 -- 172.21.15.12:0/1872018649 <== mon.2 v2:172.21.15.165:3300/0 5 ==== fsmap(e 57) ==== 8410+0+0 (secure 0 0 0) 0x7f29d800e3b0 con 0x55e836bad0c0 2024-09-11T00:22:28.172 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:28.170+0000 7f29f07f9740 10 client.27236 fetch_fsmap finished waiting for FSMap version 57 2024-09-11T00:22:28.172 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:28.170+0000 7f29f07f9740 10 client.27236 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T00:22:28.172 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:28.170+0000 7f29f07f9740 20 client.27236 populate_metadata read hostname 'smithi012' 2024-09-11T00:22:28.172 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:28.170+0000 7f29f07f9740 1 --2- 172.21.15.12:0/1872018649 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55e836bbf8f0 0x55e836bc1d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:22:28.172 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:28.170+0000 7f29f07f9740 4 client.27236 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T00:22:28.172 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:28.170+0000 7f29f07f9740 1 -- 172.21.15.12:0/1872018649 --> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e836b1f600 con 0x55e836bbf8f0 2024-09-11T00:22:28.173 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:28.171+0000 7f29e6000640 1 --2- 172.21.15.12:0/1872018649 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55e836bbf8f0 0x55e836bc1d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:22:28.175 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:28.171+0000 7f29e6000640 1 --2- 172.21.15.12:0/1872018649 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55e836bbf8f0 0x55e836bc1d00 crc :-1 s=READY pgs=225 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9b963e6fdb63c783 server_cookie=28c432622e7f6371 in_seq=0 out_seq=0 2024-09-11T00:22:28.175 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:28.171+0000 7f29d0c00640 10 client.27236 ms_handle_connect on v2:172.21.15.165:6838/501312984 2024-09-11T00:22:28.177 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:28.171+0000 7f29d0c00640 1 -- 172.21.15.12:0/1872018649 <== mds.0 v2:172.21.15.165:6838/501312984 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55e836b1f600 con 0x55e836bbf8f0 2024-09-11T00:22:28.177 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:28.171+0000 7f29d0c00640 10 client.27236 handle_command_reply: tid=0 2024-09-11T00:22:28.179 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:28.171+0000 7f29f07f9740 1 client.27236 shutdown 2024-09-11T00:22:28.199 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:28.195+0000 7f29f07f9740 1 -- 172.21.15.12:0/1872018649 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e836bad0c0 msgr2=0x55e836bb45c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:22:28.199 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:28.195+0000 7f29f07f9740 1 --2- 172.21.15.12:0/1872018649 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e836bad0c0 0x55e836bb45c0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f29d8009490 tx=0x7f29d8011b00 comp rx=0 tx=0).stop 2024-09-11T00:22:28.199 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:28.195+0000 7f29f07f9740 1 -- 172.21.15.12:0/1872018649 shutdown_connections 2024-09-11T00:22:28.199 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:28.195+0000 7f29f07f9740 1 --2- 172.21.15.12:0/1872018649 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55e836bbf8f0 0x55e836bc1d00 crc :-1 s=READY pgs=225 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:28.199 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:28.195+0000 7f29f07f9740 1 --2- 172.21.15.12:0/1872018649 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e836bafb00 0x55e836bb5190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:28.199 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:28.195+0000 7f29f07f9740 1 --2- 172.21.15.12:0/1872018649 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e836bad0c0 0x55e836bb45c0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:28.199 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:28.195+0000 7f29f07f9740 1 --2- 172.21.15.12:0/1872018649 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e836baa790 0x55e836bb3f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:28.199 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:28.195+0000 7f29f07f9740 1 -- 172.21.15.12:0/1872018649 >> 172.21.15.12:0/1872018649 conn(0x55e836ba58e0 msgr2=0x55e836ba4a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:22:28.199 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:28.195+0000 7f29d0c00640 0 client.27236 ms_handle_reset on v2:172.21.15.165:6838/501312984 2024-09-11T00:22:28.213 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:28.207+0000 7f29f07f9740 1 -- 172.21.15.12:0/1872018649 shutdown_connections 2024-09-11T00:22:28.213 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:28.207+0000 7f29f07f9740 1 -- 172.21.15.12:0/1872018649 wait complete. 2024-09-11T00:22:28.214 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:28.212+0000 7f29f07f9740 20 client.27236 trim_cache size 0 max 16384 2024-09-11T00:22:28.254 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:28.248+0000 7f29f07f9740 1 Processor -- start 2024-09-11T00:22:28.256 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:28.253+0000 7f29f07f9740 1 -- start start 2024-09-11T00:22:28.256 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:28.253+0000 7f29f07f9740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e836baa790 0x55e836bc1900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:22:28.256 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:28.254+0000 7f29f07f9740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e836bad0c0 0x55e836bafee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:22:28.257 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:28.254+0000 7f29f07f9740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e836bafb00 0x55e836bb0420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:22:28.257 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:28.254+0000 7f29f07f9740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f29c800fd10 con 0x55e836bad0c0 2024-09-11T00:22:28.257 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:28.254+0000 7f29f07f9740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f29c8011a30 con 0x55e836baa790 2024-09-11T00:22:28.257 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:28.254+0000 7f29f07f9740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f29c8010440 con 0x55e836bafb00 2024-09-11T00:22:28.258 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:28.254+0000 7f29e6000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e836baa790 0x55e836bc1900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:22:28.258 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:28.254+0000 7f29e6000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e836baa790 0x55e836bc1900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.12:56238/0 (socket says 172.21.15.12:56238) 2024-09-11T00:22:28.258 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:28.254+0000 7f29e6000640 1 -- 172.21.15.12:0/4126486837 learned_addr learned my addr 172.21.15.12:0/4126486837 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:22:28.258 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:28.254+0000 7f29e6a00640 1 --2- 172.21.15.12:0/4126486837 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e836bad0c0 0x55e836bafee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:22:28.258 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:28.254+0000 7f29e6000640 1 --2- 172.21.15.12:0/4126486837 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e836bafb00 0x55e836bb0420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:22:28.259 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:28.254+0000 7f29e6000640 1 -- 172.21.15.12:0/4126486837 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e836bafb00 msgr2=0x55e836bb0420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:22:28.259 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:28.254+0000 7f29e6000640 1 --2- 172.21.15.12:0/4126486837 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e836bafb00 0x55e836bb0420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:28.259 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:28.254+0000 7f29e6000640 1 -- 172.21.15.12:0/4126486837 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e836bad0c0 msgr2=0x55e836bafee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:22:28.259 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:28.254+0000 7f29e6000640 1 --2- 172.21.15.12:0/4126486837 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e836bad0c0 0x55e836bafee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:28.259 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:28.254+0000 7f29e6000640 1 -- 172.21.15.12:0/4126486837 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e836bb5500 con 0x55e836baa790 2024-09-11T00:22:28.260 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:28.254+0000 7f29e6000640 1 --2- 172.21.15.12:0/4126486837 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e836baa790 0x55e836bc1900 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f29d4041040 tx=0x7f29d4009580 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cb61a3c4410a03c3 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:22:28.260 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:28.254+0000 7f29d3400640 1 -- 172.21.15.12:0/4126486837 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29d405d040 con 0x55e836baa790 2024-09-11T00:22:28.262 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:28.260+0000 7f29d3400640 1 -- 172.21.15.12:0/4126486837 <== mon.2 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f29d4030d20 con 0x55e836baa790 2024-09-11T00:22:28.262 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:28.260+0000 7f29d3400640 1 -- 172.21.15.12:0/4126486837 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29d4043040 con 0x55e836baa790 2024-09-11T00:22:28.263 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:28.260+0000 7f29f07f9740 1 -- 172.21.15.12:0/4126486837 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e836baa790 msgr2=0x55e836bc1900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:22:28.263 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:28.260+0000 7f29f07f9740 1 --2- 172.21.15.12:0/4126486837 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e836baa790 0x55e836bc1900 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f29d4041040 tx=0x7f29d4009580 comp rx=0 tx=0).stop 2024-09-11T00:22:28.264 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:28.260+0000 7f29f07f9740 1 -- 172.21.15.12:0/4126486837 shutdown_connections 2024-09-11T00:22:28.264 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:28.260+0000 7f29f07f9740 1 --2- 172.21.15.12:0/4126486837 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e836bafb00 0x55e836bb0420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:28.264 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:28.260+0000 7f29f07f9740 1 --2- 172.21.15.12:0/4126486837 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e836bad0c0 0x55e836bafee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:28.264 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:28.260+0000 7f29f07f9740 1 --2- 172.21.15.12:0/4126486837 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e836baa790 0x55e836bc1900 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:28.265 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:28.260+0000 7f29f07f9740 1 -- 172.21.15.12:0/4126486837 >> 172.21.15.12:0/4126486837 conn(0x55e836ba58e0 msgr2=0x55e836ba9bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:22:28.265 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:28.260+0000 7f29f07f9740 1 -- 172.21.15.12:0/4126486837 shutdown_connections 2024-09-11T00:22:28.265 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:28.260+0000 7f29f07f9740 1 -- 172.21.15.12:0/4126486837 wait complete. 2024-09-11T00:22:28.265 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:28.260+0000 7f29f07f9740 1 -- start start 2024-09-11T00:22:28.265 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:28.260+0000 7f29f07f9740 1 Processor -- start 2024-09-11T00:22:28.266 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:28.260+0000 7f29f07f9740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e836baa790 0x55e836bbbaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:22:28.267 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:28.265+0000 7f29f07f9740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e836bad0c0 0x55e836bba7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:22:28.267 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:28.265+0000 7f29f07f9740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e836bafb00 0x55e836bbaec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:22:28.267 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:28.265+0000 7f29f07f9740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f29c8011360 con 0x55e836bad0c0 2024-09-11T00:22:28.267 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:28.265+0000 7f29f07f9740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f29c8010bd0 con 0x55e836bafb00 2024-09-11T00:22:28.267 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:28.265+0000 7f29f07f9740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f29c800dcf0 con 0x55e836baa790 2024-09-11T00:22:28.268 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:28.265+0000 7f29f07f9740 10 client.0 _ll_register_callbacks cb 0x7f29d8009740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:22:28.268 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:28.265+0000 7f29e6000640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e836baa790 0x55e836bbbaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:22:28.268 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:28.265+0000 7f29e6a00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e836bad0c0 0x55e836bba7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:22:28.268 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:28.265+0000 7f29e6000640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e836baa790 0x55e836bbbaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.12:58466/0 (socket says 172.21.15.12:58466) 2024-09-11T00:22:28.268 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:28.265+0000 7f29e6000640 1 -- 172.21.15.12:0/3732611322 learned_addr learned my addr 172.21.15.12:0/3732611322 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:22:28.268 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:28.266+0000 7f29e6000640 1 --2- 172.21.15.12:0/3732611322 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e836bafb00 0x55e836bbaec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:22:28.269 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:28.266+0000 7f29e6000640 1 -- 172.21.15.12:0/3732611322 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e836bafb00 msgr2=0x55e836bbaec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:22:28.269 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:28.266+0000 7f29e6000640 1 --2- 172.21.15.12:0/3732611322 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e836bafb00 0x55e836bbaec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:28.269 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:28.266+0000 7f29e6000640 1 -- 172.21.15.12:0/3732611322 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e836bad0c0 msgr2=0x55e836bba7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:22:28.269 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:28.266+0000 7f29e6000640 1 --2- 172.21.15.12:0/3732611322 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e836bad0c0 0x55e836bba7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:28.269 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:28.266+0000 7f29e6000640 1 -- 172.21.15.12:0/3732611322 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e836bacef0 con 0x55e836baa790 2024-09-11T00:22:28.270 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:28.266+0000 7f29d0c00640 10 client.0 ms_handle_connect on v2:172.21.15.178:3300/0 2024-09-11T00:22:28.270 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:28.266+0000 7f29e6000640 1 --2- 172.21.15.12:0/3732611322 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e836baa790 0x55e836bbbaf0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f29d40094c0 tx=0x7f29d4009450 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cccac9a3f7d097f6 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:22:28.270 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:28.266+0000 7f29d0c00640 1 -- 172.21.15.12:0/3732611322 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29d4043040 con 0x55e836baa790 2024-09-11T00:22:28.270 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:28.266+0000 7f29d0c00640 1 -- 172.21.15.12:0/3732611322 <== mon.1 v2:172.21.15.178:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9208+0+0 (secure 0 0 0) 0x7f29d4030520 con 0x55e836baa790 2024-09-11T00:22:28.270 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:28.266+0000 7f29f07f9740 1 -- 172.21.15.12:0/3732611322 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f29d4030d20 con 0x55e836baa790 2024-09-11T00:22:28.271 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:28.266+0000 7f29d0c00640 1 -- 172.21.15.12:0/3732611322 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=57) ==== 24+0+0 (secure 0 0 0) 0x7f29d4030720 con 0x55e836baa790 2024-09-11T00:22:28.271 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:28.266+0000 7f29f07f9740 10 client.27205 fetch_fsmap learned FSMap version 57 2024-09-11T00:22:28.271 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:28.266+0000 7f29f07f9740 1 -- 172.21.15.12:0/3732611322 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,fsmap=57,monmap=4+}) -- 0x55e836c2c810 con 0x55e836baa790 2024-09-11T00:22:28.273 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:28.266+0000 7f29d0c00640 1 -- 172.21.15.12:0/3732611322 <== mon.1 v2:172.21.15.178:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f29d4030d20 con 0x55e836baa790 2024-09-11T00:22:28.274 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:28.266+0000 7f29d0c00640 1 -- 172.21.15.12:0/3732611322 <== mon.1 v2:172.21.15.178:3300/0 5 ==== fsmap(e 57) ==== 8410+0+0 (secure 0 0 0) 0x7f29d4044260 con 0x55e836baa790 2024-09-11T00:22:28.274 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:28.272+0000 7f29f07f9740 10 client.27205 fetch_fsmap finished waiting for FSMap version 57 2024-09-11T00:22:28.274 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:28.272+0000 7f29f07f9740 10 client.27205 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T00:22:28.274 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:28.272+0000 7f29f07f9740 20 client.27205 populate_metadata read hostname 'smithi012' 2024-09-11T00:22:28.274 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:28.272+0000 7f29f07f9740 1 --2- 172.21.15.12:0/3732611322 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55e836c2cc60 0x55e836c2f050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:22:28.274 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:28.272+0000 7f29f07f9740 4 client.27205 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T00:22:28.275 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:28.272+0000 7f29f07f9740 1 -- 172.21.15.12:0/3732611322 --> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55e836b1b0b0 con 0x55e836c2cc60 2024-09-11T00:22:28.275 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:28.272+0000 7f29e6a00640 1 --2- 172.21.15.12:0/3732611322 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55e836c2cc60 0x55e836c2f050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:22:28.277 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:28.272+0000 7f29d0c00640 10 client.27205 ms_handle_connect on v2:172.21.15.165:6838/501312984 2024-09-11T00:22:28.277 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:28.272+0000 7f29e6a00640 1 --2- 172.21.15.12:0/3732611322 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55e836c2cc60 0x55e836c2f050 crc :-1 s=READY pgs=226 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f3beb19c8a0638e5 server_cookie=e3bbdf3af78d73ca in_seq=0 out_seq=0 2024-09-11T00:22:29.168 INFO:tasks.workunit.client.0.smithi012.stdout: 10 51200 39.12 MB/sec execute 25 sec latency 975.980 ms 2024-09-11T00:22:29.811 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:22:29 smithi012 ceph-mon[30333]: pgmap v861: 129 pgs: 129 active+clean; 449 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 5.7 MiB/s wr, 544 op/s 2024-09-11T00:22:29.811 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:22:29 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T00:22:29.981 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:29.974+0000 7f29d0c00640 1 -- 172.21.15.12:0/3732611322 <== mds.0 v2:172.21.15.165:6838/501312984 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55e836b1b0b0 con 0x55e836c2cc60 2024-09-11T00:22:29.983 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:29.974+0000 7f29d0c00640 10 client.27205 handle_command_reply: tid=0 2024-09-11T00:22:29.983 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:29.980+0000 7f29f07f9740 1 client.27205 shutdown 2024-09-11T00:22:29.983 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:29.980+0000 7f29f07f9740 1 -- 172.21.15.12:0/3732611322 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e836baa790 msgr2=0x55e836bbbaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:22:29.983 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:29.980+0000 7f29f07f9740 1 --2- 172.21.15.12:0/3732611322 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e836baa790 0x55e836bbbaf0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f29d40094c0 tx=0x7f29d4009450 comp rx=0 tx=0).stop 2024-09-11T00:22:29.987 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:29.980+0000 7f29f07f9740 1 -- 172.21.15.12:0/3732611322 shutdown_connections 2024-09-11T00:22:29.987 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:29.980+0000 7f29f07f9740 1 --2- 172.21.15.12:0/3732611322 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55e836c2cc60 0x55e836c2f050 crc :-1 s=READY pgs=226 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:29.987 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:29.980+0000 7f29d0c00640 0 client.27205 ms_handle_reset on v2:172.21.15.165:6838/501312984 2024-09-11T00:22:29.987 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:29.980+0000 7f29f07f9740 1 --2- 172.21.15.12:0/3732611322 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e836bafb00 0x55e836bbaec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:29.987 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:29.980+0000 7f29f07f9740 1 --2- 172.21.15.12:0/3732611322 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e836bad0c0 0x55e836bba7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:29.987 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:29.980+0000 7f29f07f9740 1 --2- 172.21.15.12:0/3732611322 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e836baa790 0x55e836bbbaf0 secure :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f29d40094c0 tx=0x7f29d4009450 comp rx=0 tx=0).stop 2024-09-11T00:22:29.987 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:29.980+0000 7f29f07f9740 1 -- 172.21.15.12:0/3732611322 >> 172.21.15.12:0/3732611322 conn(0x55e836ba58e0 msgr2=0x55e836bb3ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:22:29.999 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:29.986+0000 7f29f07f9740 1 -- 172.21.15.12:0/3732611322 shutdown_connections 2024-09-11T00:22:29.999 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:29.986+0000 7f29f07f9740 1 -- 172.21.15.12:0/3732611322 wait complete. 2024-09-11T00:22:29.999 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:29.986+0000 7f29f07f9740 20 client.27205 trim_cache size 0 max 16384 2024-09-11T00:22:30.000 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:{ 2024-09-11T00:22:30.000 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "message": "", 2024-09-11T00:22:30.000 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "return_code": 0 2024-09-11T00:22:30.000 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:} 2024-09-11T00:22:30.001 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:29.986+0000 7f29eda00640 1 -- 172.21.15.12:0/2366127360 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f29cc0b5360 msgr2=0x7f29cc0b7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:22:30.001 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:29.986+0000 7f29eda00640 1 --2- 172.21.15.12:0/2366127360 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f29cc0b5360 0x7f29cc0b7820 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f29e00ab7f0 tx=0x7f29e0004a40 comp rx=0 tx=0).stop 2024-09-11T00:22:30.001 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:29.991+0000 7f29eda00640 1 -- 172.21.15.12:0/2366127360 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f29e81044a0 msgr2=0x7f29e8075a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:22:30.001 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:29.991+0000 7f29eda00640 1 --2- 172.21.15.12:0/2366127360 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f29e81044a0 0x7f29e8075a80 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f29e000b970 tx=0x7f29e000be30 comp rx=0 tx=0).stop 2024-09-11T00:22:30.002 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:29.991+0000 7f29eda00640 1 -- 172.21.15.12:0/2366127360 shutdown_connections 2024-09-11T00:22:30.002 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:29.991+0000 7f29eda00640 1 --2- 172.21.15.12:0/2366127360 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f29cc0b5360 0x7f29cc0b7820 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:30.002 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:29.993+0000 7f29eda00640 1 --2- 172.21.15.12:0/2366127360 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f29e81044a0 0x7f29e8075a80 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:30.002 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:29.993+0000 7f29eda00640 1 --2- 172.21.15.12:0/2366127360 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f29e8103ae0 0x7f29e8075540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:30.002 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:29.993+0000 7f29eda00640 1 --2- 172.21.15.12:0/2366127360 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f29e8102990 0x7f29e8076ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:30.002 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:29.993+0000 7f29eda00640 1 -- 172.21.15.12:0/2366127360 >> 172.21.15.12:0/2366127360 conn(0x7f29e80fe250 msgr2=0x7f29e8106e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:22:30.002 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:29.993+0000 7f29eda00640 1 -- 172.21.15.12:0/2366127360 shutdown_connections 2024-09-11T00:22:30.002 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:29.994+0000 7f29eda00640 1 -- 172.21.15.12:0/2366127360 wait complete. 2024-09-11T00:22:30.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:22:29 smithi165 ceph-mon[35356]: pgmap v861: 129 pgs: 129 active+clean; 449 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 5.7 MiB/s wr, 544 op/s 2024-09-11T00:22:30.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:22:29 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T00:22:30.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:22:29 smithi178 ceph-mon[35919]: pgmap v861: 129 pgs: 129 active+clean; 449 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 5.7 MiB/s wr, 544 op/s 2024-09-11T00:22:30.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:22:29 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T00:22:30.169 INFO:tasks.workunit.client.0.smithi012.stdout: 10 51204 37.62 MB/sec execute 26 sec latency 1735.905 ms 2024-09-11T00:22:31.169 INFO:tasks.workunit.client.0.smithi012.stdout: 10 51235 36.27 MB/sec execute 27 sec latency 2736.341 ms 2024-09-11T00:22:32.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:22:31 smithi165 ceph-mon[35356]: pgmap v862: 129 pgs: 129 active+clean; 522 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 MiB/s rd, 13 MiB/s wr, 778 op/s 2024-09-11T00:22:32.063 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:22:31 smithi012 ceph-mon[30333]: pgmap v862: 129 pgs: 129 active+clean; 522 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 MiB/s rd, 13 MiB/s wr, 778 op/s 2024-09-11T00:22:32.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:22:31 smithi178 ceph-mon[35919]: pgmap v862: 129 pgs: 129 active+clean; 522 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 MiB/s rd, 13 MiB/s wr, 778 op/s 2024-09-11T00:22:32.169 INFO:tasks.workunit.client.0.smithi012.stdout: 10 51266 34.98 MB/sec execute 28 sec latency 3736.394 ms 2024-09-11T00:22:33.169 INFO:tasks.workunit.client.0.smithi012.stdout: 10 51266 33.78 MB/sec execute 29 sec latency 4736.459 ms 2024-09-11T00:22:34.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:22:33 smithi165 ceph-mon[35356]: pgmap v863: 129 pgs: 129 active+clean; 519 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 14 MiB/s wr, 763 op/s 2024-09-11T00:22:34.063 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:22:33 smithi012 ceph-mon[30333]: pgmap v863: 129 pgs: 129 active+clean; 519 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 14 MiB/s wr, 763 op/s 2024-09-11T00:22:34.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:22:33 smithi178 ceph-mon[35919]: pgmap v863: 129 pgs: 129 active+clean; 519 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 14 MiB/s wr, 763 op/s 2024-09-11T00:22:34.169 INFO:tasks.workunit.client.0.smithi012.stdout: 10 51266 32.65 MB/sec execute 30 sec latency 5736.522 ms 2024-09-11T00:22:35.169 INFO:tasks.workunit.client.0.smithi012.stdout: 10 51266 31.60 MB/sec execute 31 sec latency 6736.591 ms 2024-09-11T00:22:36.033 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:22:35 smithi165 ceph-mon[35356]: pgmap v864: 129 pgs: 129 active+clean; 508 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 13 MiB/s wr, 657 op/s 2024-09-11T00:22:36.063 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:22:35 smithi012 ceph-mon[30333]: pgmap v864: 129 pgs: 129 active+clean; 508 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 13 MiB/s wr, 657 op/s 2024-09-11T00:22:36.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:22:35 smithi178 ceph-mon[35919]: pgmap v864: 129 pgs: 129 active+clean; 508 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 13 MiB/s wr, 657 op/s 2024-09-11T00:22:36.169 INFO:tasks.workunit.client.0.smithi012.stdout: 10 51266 30.61 MB/sec execute 32 sec latency 7736.634 ms 2024-09-11T00:22:37.169 INFO:tasks.workunit.client.0.smithi012.stdout: 10 51266 29.68 MB/sec execute 33 sec latency 8736.676 ms 2024-09-11T00:22:38.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:22:37 smithi165 ceph-mon[35356]: pgmap v865: 129 pgs: 129 active+clean; 459 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 13 MiB/s wr, 723 op/s 2024-09-11T00:22:38.063 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:22:37 smithi012 ceph-mon[30333]: pgmap v865: 129 pgs: 129 active+clean; 459 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 13 MiB/s wr, 723 op/s 2024-09-11T00:22:38.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:22:37 smithi178 ceph-mon[35919]: pgmap v865: 129 pgs: 129 active+clean; 459 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 13 MiB/s wr, 723 op/s 2024-09-11T00:22:38.169 INFO:tasks.workunit.client.0.smithi012.stdout: 10 51266 28.81 MB/sec execute 34 sec latency 9736.720 ms 2024-09-11T00:22:39.169 INFO:tasks.workunit.client.0.smithi012.stdout: 10 51266 27.99 MB/sec execute 35 sec latency 10736.760 ms 2024-09-11T00:22:40.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:22:39 smithi165 ceph-mon[35356]: pgmap v866: 129 pgs: 129 active+clean; 459 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 10 MiB/s wr, 548 op/s 2024-09-11T00:22:40.063 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:22:39 smithi012 ceph-mon[30333]: pgmap v866: 129 pgs: 129 active+clean; 459 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 10 MiB/s wr, 548 op/s 2024-09-11T00:22:40.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:22:39 smithi178 ceph-mon[35919]: pgmap v866: 129 pgs: 129 active+clean; 459 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 10 MiB/s wr, 548 op/s 2024-09-11T00:22:40.169 INFO:tasks.workunit.client.0.smithi012.stdout: 10 51266 27.21 MB/sec execute 36 sec latency 11736.804 ms 2024-09-11T00:22:41.169 INFO:tasks.workunit.client.0.smithi012.stdout: 10 51266 26.47 MB/sec execute 37 sec latency 12736.847 ms 2024-09-11T00:22:42.063 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:22:41 smithi012 ceph-mon[30333]: pgmap v867: 129 pgs: 129 active+clean; 460 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 10 MiB/s wr, 547 op/s 2024-09-11T00:22:42.063 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:22:41 smithi012 ceph-mon[30333]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T00:22:42.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:22:41 smithi178 ceph-mon[35919]: pgmap v867: 129 pgs: 129 active+clean; 460 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 10 MiB/s wr, 547 op/s 2024-09-11T00:22:42.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:22:41 smithi178 ceph-mon[35919]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T00:22:42.169 INFO:tasks.workunit.client.0.smithi012.stdout: 10 51266 25.78 MB/sec execute 38 sec latency 13736.890 ms 2024-09-11T00:22:42.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:22:41 smithi165 ceph-mon[35356]: pgmap v867: 129 pgs: 129 active+clean; 460 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 10 MiB/s wr, 547 op/s 2024-09-11T00:22:42.283 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:22:41 smithi165 ceph-mon[35356]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T00:22:43.148 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:43.145+0000 7fc142800640 1 -- 172.21.15.12:0/2027596620 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc13c102990 msgr2=0x7fc13c102d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:22:43.148 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:43.145+0000 7fc142800640 1 --2- 172.21.15.12:0/2027596620 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc13c102990 0x7fc13c102d90 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7fc12c009930 tx=0x7fc12c02a440 comp rx=0 tx=0).stop 2024-09-11T00:22:43.148 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:43.145+0000 7fc142800640 1 -- 172.21.15.12:0/2027596620 shutdown_connections 2024-09-11T00:22:43.148 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:43.145+0000 7fc142800640 1 --2- 172.21.15.12:0/2027596620 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc13c1044a0 0x7fc13c10ae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:43.148 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:43.145+0000 7fc142800640 1 --2- 172.21.15.12:0/2027596620 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc13c103ae0 0x7fc13c103f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:43.148 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:43.145+0000 7fc142800640 1 --2- 172.21.15.12:0/2027596620 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc13c102990 0x7fc13c102d90 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:43.148 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:43.145+0000 7fc142800640 1 -- 172.21.15.12:0/2027596620 >> 172.21.15.12:0/2027596620 conn(0x7fc13c0fe250 msgr2=0x7fc13c100670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:22:43.148 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:43.145+0000 7fc142800640 1 -- 172.21.15.12:0/2027596620 shutdown_connections 2024-09-11T00:22:43.149 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:43.145+0000 7fc142800640 1 -- 172.21.15.12:0/2027596620 wait complete. 2024-09-11T00:22:43.150 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:43.146+0000 7fc142800640 1 Processor -- start 2024-09-11T00:22:43.151 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:43.146+0000 7fc142800640 1 -- start start 2024-09-11T00:22:43.151 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:43.147+0000 7fc142800640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc13c102990 0x7fc13c19b0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:22:43.151 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:43.147+0000 7fc142800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc13c103ae0 0x7fc13c19b710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:22:43.151 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:43.147+0000 7fc142800640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc13c1044a0 0x7fc13c19c340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:22:43.151 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:43.147+0000 7fc142800640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fc13c10e6a0 con 0x7fc13c1044a0 2024-09-11T00:22:43.151 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:43.147+0000 7fc142800640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fc13c10e520 con 0x7fc13c103ae0 2024-09-11T00:22:43.151 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:43.147+0000 7fc142800640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fc13c10e820 con 0x7fc13c102990 2024-09-11T00:22:43.151 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:43.147+0000 7fc13be00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc13c1044a0 0x7fc13c19c340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:22:43.151 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:43.147+0000 7fc13be00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc13c1044a0 0x7fc13c19c340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:60462/0 (socket says 172.21.15.12:60462) 2024-09-11T00:22:43.151 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:43.147+0000 7fc13be00640 1 -- 172.21.15.12:0/2023602702 learned_addr learned my addr 172.21.15.12:0/2023602702 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:22:43.151 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:43.147+0000 7fc13b400640 1 --2- 172.21.15.12:0/2023602702 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc13c102990 0x7fc13c19b0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:22:43.151 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:43.147+0000 7fc13aa00640 1 --2- 172.21.15.12:0/2023602702 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc13c103ae0 0x7fc13c19b710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:22:43.151 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:43.147+0000 7fc13be00640 1 -- 172.21.15.12:0/2023602702 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc13c102990 msgr2=0x7fc13c19b0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:22:43.152 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:43.147+0000 7fc13be00640 1 --2- 172.21.15.12:0/2023602702 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc13c102990 0x7fc13c19b0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:43.152 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:43.147+0000 7fc13be00640 1 -- 172.21.15.12:0/2023602702 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc13c103ae0 msgr2=0x7fc13c19b710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:22:43.152 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:43.147+0000 7fc13be00640 1 --2- 172.21.15.12:0/2023602702 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc13c103ae0 0x7fc13c19b710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:43.152 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:43.147+0000 7fc13be00640 1 -- 172.21.15.12:0/2023602702 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc13c19cc00 con 0x7fc13c1044a0 2024-09-11T00:22:43.152 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:43.148+0000 7fc13be00640 1 --2- 172.21.15.12:0/2023602702 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc13c1044a0 0x7fc13c19c340 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7fc13400b990 tx=0x7fc13400be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:22:43.152 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:43.148+0000 7fc133e00640 1 -- 172.21.15.12:0/2023602702 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc134019070 con 0x7fc13c1044a0 2024-09-11T00:22:43.152 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:43.148+0000 7fc133e00640 1 -- 172.21.15.12:0/2023602702 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc13400cba0 con 0x7fc13c1044a0 2024-09-11T00:22:43.152 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:43.148+0000 7fc142800640 1 -- 172.21.15.12:0/2023602702 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc13c19cef0 con 0x7fc13c1044a0 2024-09-11T00:22:43.152 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:43.148+0000 7fc133e00640 1 -- 172.21.15.12:0/2023602702 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1340075d0 con 0x7fc13c1044a0 2024-09-11T00:22:43.152 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:43.148+0000 7fc142800640 1 -- 172.21.15.12:0/2023602702 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc13c1a35c0 con 0x7fc13c1044a0 2024-09-11T00:22:43.154 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:43.151+0000 7fc1457ea740 1 Processor -- start 2024-09-11T00:22:43.154 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:43.152+0000 7fc133e00640 1 -- 172.21.15.12:0/2023602702 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7fc134014440 con 0x7fc13c1044a0 2024-09-11T00:22:43.155 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:43.152+0000 7fc1457ea740 1 -- start start 2024-09-11T00:22:43.155 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:43.152+0000 7fc1457ea740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55dfe1046790 0x55dfe1048b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:22:43.155 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:43.152+0000 7fc1457ea740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55dfe10490c0 0x55dfe104b4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:22:43.155 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:43.152+0000 7fc1457ea740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55dfe104bb00 0x55dfe104dfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:22:43.155 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:43.152+0000 7fc1457ea740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55dfe0fb96b0 con 0x55dfe10490c0 2024-09-11T00:22:43.156 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:43.152+0000 7fc1457ea740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55dfe0fa8ae0 con 0x55dfe104bb00 2024-09-11T00:22:43.156 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:43.152+0000 7fc1457ea740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55dfe0fac8f0 con 0x55dfe1046790 2024-09-11T00:22:43.156 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:43.153+0000 7fc13b400640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55dfe10490c0 0x55dfe104b4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:22:43.156 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:43.153+0000 7fc13b400640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55dfe10490c0 0x55dfe104b4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:60466/0 (socket says 172.21.15.12:60466) 2024-09-11T00:22:43.156 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:43.153+0000 7fc13b400640 1 -- 172.21.15.12:0/3897990061 learned_addr learned my addr 172.21.15.12:0/3897990061 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:22:43.156 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:43.153+0000 7fc13aa00640 1 --2- 172.21.15.12:0/3897990061 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55dfe1046790 0x55dfe1048b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:22:43.156 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:43.153+0000 7fc13aa00640 1 --2- 172.21.15.12:0/3897990061 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55dfe104bb00 0x55dfe104dfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:22:43.156 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:43.153+0000 7fc13b400640 1 -- 172.21.15.12:0/3897990061 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55dfe1046790 msgr2=0x55dfe1048b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:22:43.156 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:43.153+0000 7fc13b400640 1 --2- 172.21.15.12:0/3897990061 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55dfe1046790 0x55dfe1048b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:43.157 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:43.153+0000 7fc13b400640 1 -- 172.21.15.12:0/3897990061 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55dfe104bb00 msgr2=0x55dfe104dfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:22:43.157 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:43.153+0000 7fc13b400640 1 --2- 172.21.15.12:0/3897990061 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55dfe104bb00 0x55dfe104dfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:43.157 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:43.153+0000 7fc13b400640 1 -- 172.21.15.12:0/3897990061 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dfe0f75780 con 0x55dfe10490c0 2024-09-11T00:22:43.159 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:43.153+0000 7fc13b400640 1 --2- 172.21.15.12:0/3897990061 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55dfe10490c0 0x55dfe104b4b0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7fc12c02a960 tx=0x7fc12c031b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f471af89d5b428c4 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:22:43.159 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:43.154+0000 7fc133e00640 1 --2- 172.21.15.12:0/2023602702 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fc1200b5360 0x7fc1200b7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:22:43.159 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:43.154+0000 7fc130c00640 1 -- 172.21.15.12:0/3897990061 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc12c035030 con 0x55dfe10490c0 2024-09-11T00:22:43.159 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:43.154+0000 7fc133e00640 1 -- 172.21.15.12:0/2023602702 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9208+0+0 (secure 0 0 0) 0x7fc1340e56c0 con 0x7fc13c1044a0 2024-09-11T00:22:43.159 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:43.154+0000 7fc130c00640 1 -- 172.21.15.12:0/3897990061 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc12c040070 con 0x55dfe10490c0 2024-09-11T00:22:43.159 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:43.154+0000 7fc13aa00640 1 --2- 172.21.15.12:0/2023602702 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fc1200b5360 0x7fc1200b7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:22:43.160 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:43.154+0000 7fc130c00640 1 -- 172.21.15.12:0/3897990061 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc12c003920 con 0x55dfe10490c0 2024-09-11T00:22:43.160 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:43.155+0000 7fc1457ea740 1 -- 172.21.15.12:0/3897990061 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55dfe10490c0 msgr2=0x55dfe104b4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:22:43.160 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:43.155+0000 7fc1457ea740 1 --2- 172.21.15.12:0/3897990061 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55dfe10490c0 0x55dfe104b4b0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7fc12c02a960 tx=0x7fc12c031b90 comp rx=0 tx=0).stop 2024-09-11T00:22:43.160 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:43.155+0000 7fc13aa00640 1 --2- 172.21.15.12:0/2023602702 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fc1200b5360 0x7fc1200b7820 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fc1280105d0 tx=0x7fc128013040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:22:43.160 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:43.155+0000 7fc1457ea740 1 -- 172.21.15.12:0/3897990061 shutdown_connections 2024-09-11T00:22:43.160 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:43.155+0000 7fc1457ea740 1 --2- 172.21.15.12:0/3897990061 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55dfe104bb00 0x55dfe104dfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:43.160 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:43.155+0000 7fc1457ea740 1 --2- 172.21.15.12:0/3897990061 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55dfe10490c0 0x55dfe104b4b0 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:43.160 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:43.155+0000 7fc1457ea740 1 --2- 172.21.15.12:0/3897990061 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55dfe1046790 0x55dfe1048b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:43.160 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:43.155+0000 7fc1457ea740 1 -- 172.21.15.12:0/3897990061 >> 172.21.15.12:0/3897990061 conn(0x55dfe10418e0 msgr2=0x55dfe1042670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:22:43.160 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:43.155+0000 7fc1457ea740 1 -- 172.21.15.12:0/3897990061 shutdown_connections 2024-09-11T00:22:43.160 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:43.155+0000 7fc1457ea740 1 -- 172.21.15.12:0/3897990061 wait complete. 2024-09-11T00:22:43.161 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:43.155+0000 7fc1457ea740 1 -- start start 2024-09-11T00:22:43.161 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:43.156+0000 7fc1457ea740 1 Processor -- start 2024-09-11T00:22:43.161 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:43.156+0000 7fc1457ea740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55dfe1046790 0x55dfe104fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:22:43.161 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:43.156+0000 7fc1457ea740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55dfe10490c0 0x55dfe1050510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:22:43.161 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:43.156+0000 7fc1457ea740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55dfe104bb00 0x55dfe10510e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:22:43.161 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:43.156+0000 7fc1457ea740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55dfe0ee0ab0 con 0x55dfe1046790 2024-09-11T00:22:43.161 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:43.156+0000 7fc1457ea740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55dfe0f4f570 con 0x55dfe104bb00 2024-09-11T00:22:43.161 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:43.156+0000 7fc1457ea740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55dfe104ea80 con 0x55dfe10490c0 2024-09-11T00:22:43.161 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:43.156+0000 7fc1457ea740 10 client.0 _ll_register_callbacks cb 0x55dfe0f5e100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:22:43.161 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:43.156+0000 7fc13b400640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55dfe1046790 0x55dfe104fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:22:43.161 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:43.156+0000 7fc13b400640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55dfe1046790 0x55dfe104fe70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:60476/0 (socket says 172.21.15.12:60476) 2024-09-11T00:22:43.161 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:43.156+0000 7fc13b400640 1 -- 172.21.15.12:0/4005609063 learned_addr learned my addr 172.21.15.12:0/4005609063 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:22:43.161 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:43.156+0000 7fc13aa00640 1 --2- 172.21.15.12:0/4005609063 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55dfe10490c0 0x55dfe1050510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:22:43.164 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:43.157+0000 7fc13b400640 1 -- 172.21.15.12:0/4005609063 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55dfe10490c0 msgr2=0x55dfe1050510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:22:43.164 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:43.157+0000 7fc13b400640 1 --2- 172.21.15.12:0/4005609063 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55dfe10490c0 0x55dfe1050510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:43.164 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:43.157+0000 7fc13b400640 1 -- 172.21.15.12:0/4005609063 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55dfe104bb00 msgr2=0x55dfe10510e0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-11T00:22:43.164 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:43.157+0000 7fc13b400640 1 --2- 172.21.15.12:0/4005609063 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55dfe104bb00 0x55dfe10510e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:43.164 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:43.157+0000 7fc13b400640 1 -- 172.21.15.12:0/4005609063 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dfe0d0a790 con 0x55dfe1046790 2024-09-11T00:22:43.164 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:43.157+0000 7fc13b400640 1 --2- 172.21.15.12:0/4005609063 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55dfe1046790 0x55dfe104fe70 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7fc12c006010 tx=0x7fc12c004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae47844a79611cfe server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:22:43.164 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:43.157+0000 7fc125600640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-09-11T00:22:43.165 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:43.157+0000 7fc125600640 1 -- 172.21.15.12:0/4005609063 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc12c035050 con 0x55dfe1046790 2024-09-11T00:22:43.165 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:43.157+0000 7fc1457ea740 1 -- 172.21.15.12:0/4005609063 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc12c040070 con 0x55dfe1046790 2024-09-11T00:22:43.165 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:43.157+0000 7fc125600640 1 -- 172.21.15.12:0/4005609063 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9208+0+0 (secure 0 0 0) 0x7fc12c0317b0 con 0x55dfe1046790 2024-09-11T00:22:43.165 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:43.158+0000 7fc125600640 1 -- 172.21.15.12:0/4005609063 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7fc12c05c050 con 0x55dfe1046790 2024-09-11T00:22:43.165 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:43.158+0000 7fc1457ea740 10 client.17232 fetch_fsmap learned FSMap version 58 2024-09-11T00:22:43.165 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:43.158+0000 7fc1457ea740 1 -- 172.21.15.12:0/4005609063 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) -- 0x55dfe0d83620 con 0x55dfe1046790 2024-09-11T00:22:43.165 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:43.158+0000 7fc125600640 1 -- 172.21.15.12:0/4005609063 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc12c040070 con 0x55dfe1046790 2024-09-11T00:22:43.165 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:43.158+0000 7fc125600640 1 -- 172.21.15.12:0/4005609063 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 58) ==== 8398+0+0 (secure 0 0 0) 0x7fc12c045590 con 0x55dfe1046790 2024-09-11T00:22:43.165 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:43.158+0000 7fc1457ea740 10 client.17232 fetch_fsmap finished waiting for FSMap version 58 2024-09-11T00:22:43.165 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:43.158+0000 7fc1457ea740 10 client.17232 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T00:22:43.165 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:43.158+0000 7fc1457ea740 20 client.17232 populate_metadata read hostname 'smithi012' 2024-09-11T00:22:43.165 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:43.158+0000 7fc1457ea740 1 --2- 172.21.15.12:0/4005609063 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55dfe105b310 0x55dfe105d790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:22:43.166 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:43.158+0000 7fc1457ea740 4 client.17232 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T00:22:43.166 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:43.158+0000 7fc1457ea740 1 -- 172.21.15.12:0/4005609063 --> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55dfe0fbb600 con 0x55dfe105b310 2024-09-11T00:22:43.166 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:43.159+0000 7fc13aa00640 1 --2- 172.21.15.12:0/4005609063 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55dfe105b310 0x55dfe105d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:22:43.166 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:43.159+0000 7fc13aa00640 1 --2- 172.21.15.12:0/4005609063 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55dfe105b310 0x55dfe105d790 crc :-1 s=READY pgs=227 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=72e93e1f9237d1b8 server_cookie=8ab3d4521da4f7d in_seq=0 out_seq=0 2024-09-11T00:22:43.166 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:43.159+0000 7fc125600640 10 client.17232 ms_handle_connect on v2:172.21.15.165:6838/501312984 2024-09-11T00:22:43.166 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:43.160+0000 7fc125600640 1 -- 172.21.15.12:0/4005609063 <== mds.0 v2:172.21.15.165:6838/501312984 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55dfe0fbb600 con 0x55dfe105b310 2024-09-11T00:22:43.166 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:43.160+0000 7fc125600640 10 client.17232 handle_command_reply: tid=0 2024-09-11T00:22:43.166 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:43.160+0000 7fc1457ea740 1 client.17232 shutdown 2024-09-11T00:22:43.167 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:43.160+0000 7fc1457ea740 1 -- 172.21.15.12:0/4005609063 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55dfe1046790 msgr2=0x55dfe104fe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:22:43.167 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:43.160+0000 7fc1457ea740 1 --2- 172.21.15.12:0/4005609063 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55dfe1046790 0x55dfe104fe70 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7fc12c006010 tx=0x7fc12c004270 comp rx=0 tx=0).stop 2024-09-11T00:22:43.167 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:43.160+0000 7fc1457ea740 1 -- 172.21.15.12:0/4005609063 shutdown_connections 2024-09-11T00:22:43.167 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:43.160+0000 7fc1457ea740 1 --2- 172.21.15.12:0/4005609063 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55dfe105b310 0x55dfe105d790 crc :-1 s=READY pgs=227 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:43.167 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:43.160+0000 7fc1457ea740 1 --2- 172.21.15.12:0/4005609063 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55dfe104bb00 0x55dfe10510e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:43.167 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:43.160+0000 7fc1457ea740 1 --2- 172.21.15.12:0/4005609063 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55dfe10490c0 0x55dfe1050510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:43.167 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:43.160+0000 7fc1457ea740 1 --2- 172.21.15.12:0/4005609063 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55dfe1046790 0x55dfe104fe70 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:43.168 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:43.160+0000 7fc125600640 0 client.17232 ms_handle_reset on v2:172.21.15.165:6838/501312984 2024-09-11T00:22:43.168 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:43.160+0000 7fc1457ea740 1 -- 172.21.15.12:0/4005609063 >> 172.21.15.12:0/4005609063 conn(0x55dfe10418e0 msgr2=0x55dfe1040a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:22:43.168 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:43.161+0000 7fc1457ea740 1 -- 172.21.15.12:0/4005609063 shutdown_connections 2024-09-11T00:22:43.168 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:43.161+0000 7fc1457ea740 1 -- 172.21.15.12:0/4005609063 wait complete. 2024-09-11T00:22:43.168 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:43.161+0000 7fc1457ea740 20 client.17232 trim_cache size 0 max 16384 2024-09-11T00:22:43.169 INFO:tasks.workunit.client.0.smithi012.stdout: 10 51266 25.12 MB/sec execute 39 sec latency 14736.945 ms 2024-09-11T00:22:43.179 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:43.176+0000 7fc1457ea740 1 Processor -- start 2024-09-11T00:22:43.179 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:43.176+0000 7fc1457ea740 1 -- start start 2024-09-11T00:22:43.179 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:43.176+0000 7fc1457ea740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55dfe1046790 0x55dfe1042140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:22:43.179 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:43.176+0000 7fc1457ea740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55dfe10490c0 0x55dfe1042680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:22:43.179 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:43.176+0000 7fc1457ea740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55dfe104bb00 0x55dfe1062050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:22:43.179 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:43.176+0000 7fc1457ea740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fc13c1ad9e0 con 0x55dfe104bb00 2024-09-11T00:22:43.179 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:43.176+0000 7fc1457ea740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fc13c1af670 con 0x55dfe10490c0 2024-09-11T00:22:43.179 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:43.176+0000 7fc1457ea740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fc13c1ae110 con 0x55dfe1046790 2024-09-11T00:22:43.180 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:43.177+0000 7fc13b400640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55dfe1046790 0x55dfe1042140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:22:43.180 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:43.177+0000 7fc13be00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55dfe104bb00 0x55dfe1062050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:22:43.180 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:43.177+0000 7fc13b400640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55dfe1046790 0x55dfe1042140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.12:49288/0 (socket says 172.21.15.12:49288) 2024-09-11T00:22:43.180 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:43.177+0000 7fc13b400640 1 -- 172.21.15.12:0/3367675886 learned_addr learned my addr 172.21.15.12:0/3367675886 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:22:43.180 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:43.177+0000 7fc13aa00640 1 --2- 172.21.15.12:0/3367675886 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55dfe10490c0 0x55dfe1042680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:22:43.180 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:43.177+0000 7fc13b400640 1 -- 172.21.15.12:0/3367675886 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55dfe10490c0 msgr2=0x55dfe1042680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:22:43.180 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:43.177+0000 7fc13b400640 1 --2- 172.21.15.12:0/3367675886 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55dfe10490c0 0x55dfe1042680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:43.180 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:43.177+0000 7fc13b400640 1 -- 172.21.15.12:0/3367675886 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55dfe104bb00 msgr2=0x55dfe1062050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:22:43.180 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:43.177+0000 7fc13b400640 1 --2- 172.21.15.12:0/3367675886 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55dfe104bb00 0x55dfe1062050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:43.180 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:43.177+0000 7fc13b400640 1 -- 172.21.15.12:0/3367675886 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dfe104ec00 con 0x55dfe1046790 2024-09-11T00:22:43.200 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:43.198+0000 7fc13b400640 1 --2- 172.21.15.12:0/3367675886 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55dfe1046790 0x55dfe1042140 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fc12c004750 tx=0x7fc12c000bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e48a307ac6c46f78 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:22:43.201 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:43.198+0000 7fc127e00640 1 -- 172.21.15.12:0/3367675886 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc12c035040 con 0x55dfe1046790 2024-09-11T00:22:43.201 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:43.198+0000 7fc127e00640 1 -- 172.21.15.12:0/3367675886 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc12c040030 con 0x55dfe1046790 2024-09-11T00:22:43.201 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:43.198+0000 7fc127e00640 1 -- 172.21.15.12:0/3367675886 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc12c0456d0 con 0x55dfe1046790 2024-09-11T00:22:43.202 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:43.199+0000 7fc1457ea740 1 -- 172.21.15.12:0/3367675886 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55dfe1046790 msgr2=0x55dfe1042140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:22:43.202 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:43.199+0000 7fc1457ea740 1 --2- 172.21.15.12:0/3367675886 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55dfe1046790 0x55dfe1042140 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fc12c004750 tx=0x7fc12c000bc0 comp rx=0 tx=0).stop 2024-09-11T00:22:43.202 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:43.199+0000 7fc1457ea740 1 -- 172.21.15.12:0/3367675886 shutdown_connections 2024-09-11T00:22:43.202 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:43.199+0000 7fc1457ea740 1 --2- 172.21.15.12:0/3367675886 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55dfe104bb00 0x55dfe1062050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:43.202 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:43.199+0000 7fc1457ea740 1 --2- 172.21.15.12:0/3367675886 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55dfe10490c0 0x55dfe1042680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:43.202 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:43.199+0000 7fc1457ea740 1 --2- 172.21.15.12:0/3367675886 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55dfe1046790 0x55dfe1042140 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:43.202 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:43.199+0000 7fc1457ea740 1 -- 172.21.15.12:0/3367675886 >> 172.21.15.12:0/3367675886 conn(0x55dfe10418e0 msgr2=0x55dfe1044930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:22:43.202 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:43.199+0000 7fc1457ea740 1 -- 172.21.15.12:0/3367675886 shutdown_connections 2024-09-11T00:22:43.202 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:43.199+0000 7fc1457ea740 1 -- 172.21.15.12:0/3367675886 wait complete. 2024-09-11T00:22:43.202 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:43.200+0000 7fc1457ea740 1 -- start start 2024-09-11T00:22:43.203 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:43.200+0000 7fc1457ea740 1 Processor -- start 2024-09-11T00:22:43.205 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:43.200+0000 7fc1457ea740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55dfe1046790 0x55dfe1057a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:22:43.205 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:43.200+0000 7fc1457ea740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55dfe10490c0 0x55dfe1056750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:22:43.205 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:43.200+0000 7fc1457ea740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55dfe104bb00 0x55dfe1056d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:22:43.206 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:43.200+0000 7fc1457ea740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fc13c1aefa0 con 0x55dfe1046790 2024-09-11T00:22:43.206 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:43.200+0000 7fc1457ea740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fc13c1ae810 con 0x55dfe10490c0 2024-09-11T00:22:43.206 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:43.200+0000 7fc1457ea740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fc13c1ab9c0 con 0x55dfe104bb00 2024-09-11T00:22:43.206 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:43.200+0000 7fc1457ea740 10 client.0 _ll_register_callbacks cb 0x7fc128009a10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:22:43.206 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:43.200+0000 7fc13b400640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55dfe1046790 0x55dfe1057a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:22:43.206 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:43.200+0000 7fc13b400640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55dfe1046790 0x55dfe1057a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:60492/0 (socket says 172.21.15.12:60492) 2024-09-11T00:22:43.206 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:43.200+0000 7fc13b400640 1 -- 172.21.15.12:0/4050111833 learned_addr learned my addr 172.21.15.12:0/4050111833 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:22:43.206 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:43.201+0000 7fc13be00640 1 --2- 172.21.15.12:0/4050111833 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55dfe104bb00 0x55dfe1056d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:22:43.206 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:43.201+0000 7fc13aa00640 1 --2- 172.21.15.12:0/4050111833 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55dfe10490c0 0x55dfe1056750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:22:43.206 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:43.201+0000 7fc13b400640 1 -- 172.21.15.12:0/4050111833 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55dfe104bb00 msgr2=0x55dfe1056d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:22:43.206 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:43.201+0000 7fc13b400640 1 --2- 172.21.15.12:0/4050111833 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55dfe104bb00 0x55dfe1056d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:43.207 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:43.201+0000 7fc13b400640 1 -- 172.21.15.12:0/4050111833 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55dfe10490c0 msgr2=0x55dfe1056750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:22:43.207 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:43.201+0000 7fc13b400640 1 --2- 172.21.15.12:0/4050111833 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55dfe10490c0 0x55dfe1056750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:43.207 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:43.201+0000 7fc13b400640 1 -- 172.21.15.12:0/4050111833 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dfe1055310 con 0x55dfe1046790 2024-09-11T00:22:43.207 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:43.201+0000 7fc13b400640 1 --2- 172.21.15.12:0/4050111833 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55dfe1046790 0x55dfe1057a50 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7fc12c006010 tx=0x7fc12c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e4d5bb4b84b3697 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:22:43.207 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:43.201+0000 7fc125600640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-09-11T00:22:43.207 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:43.201+0000 7fc125600640 1 -- 172.21.15.12:0/4050111833 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc12c035040 con 0x55dfe1046790 2024-09-11T00:22:43.207 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:43.202+0000 7fc125600640 1 -- 172.21.15.12:0/4050111833 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9208+0+0 (secure 0 0 0) 0x7fc12c045470 con 0x55dfe1046790 2024-09-11T00:22:43.207 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:43.202+0000 7fc1457ea740 1 -- 172.21.15.12:0/4050111833 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc12c040030 con 0x55dfe1046790 2024-09-11T00:22:43.207 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:43.202+0000 7fc125600640 1 -- 172.21.15.12:0/4050111833 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7fc12c077050 con 0x55dfe1046790 2024-09-11T00:22:43.207 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:43.202+0000 7fc1457ea740 10 client.17238 fetch_fsmap learned FSMap version 58 2024-09-11T00:22:43.207 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:43.202+0000 7fc1457ea740 1 -- 172.21.15.12:0/4050111833 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) -- 0x55dfe10c83b0 con 0x55dfe1046790 2024-09-11T00:22:43.207 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:43.202+0000 7fc125600640 1 -- 172.21.15.12:0/4050111833 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc12c040030 con 0x55dfe1046790 2024-09-11T00:22:43.207 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:43.202+0000 7fc125600640 1 -- 172.21.15.12:0/4050111833 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 58) ==== 8398+0+0 (secure 0 0 0) 0x7fc12c069450 con 0x55dfe1046790 2024-09-11T00:22:43.208 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:43.202+0000 7fc1457ea740 10 client.17238 fetch_fsmap finished waiting for FSMap version 58 2024-09-11T00:22:43.208 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:43.202+0000 7fc1457ea740 10 client.17238 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T00:22:43.208 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:43.202+0000 7fc1457ea740 20 client.17238 populate_metadata read hostname 'smithi012' 2024-09-11T00:22:43.208 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:43.202+0000 7fc1457ea740 1 --2- 172.21.15.12:0/4050111833 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55dfe10c8d90 0x55dfe10cb1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:22:43.208 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:43.202+0000 7fc1457ea740 4 client.17238 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T00:22:43.208 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:43.202+0000 7fc1457ea740 1 -- 172.21.15.12:0/4050111833 --> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55dfe0fb70b0 con 0x55dfe10c8d90 2024-09-11T00:22:43.208 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:43.203+0000 7fc13aa00640 1 --2- 172.21.15.12:0/4050111833 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55dfe10c8d90 0x55dfe10cb1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:22:43.208 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:43.203+0000 7fc13aa00640 1 --2- 172.21.15.12:0/4050111833 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55dfe10c8d90 0x55dfe10cb1b0 crc :-1 s=READY pgs=228 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c3c3142469b597b2 server_cookie=98d006369f552b6 in_seq=0 out_seq=0 2024-09-11T00:22:43.208 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:43.203+0000 7fc125600640 10 client.17238 ms_handle_connect on v2:172.21.15.165:6838/501312984 2024-09-11T00:22:43.263 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:43.261+0000 7fc125600640 1 -- 172.21.15.12:0/4050111833 <== mds.0 v2:172.21.15.165:6838/501312984 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55dfe0fb70b0 con 0x55dfe10c8d90 2024-09-11T00:22:43.264 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:43.261+0000 7fc125600640 10 client.17238 handle_command_reply: tid=0 2024-09-11T00:22:43.264 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:43.261+0000 7fc1457ea740 1 client.17238 shutdown 2024-09-11T00:22:43.264 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:43.261+0000 7fc1457ea740 1 -- 172.21.15.12:0/4050111833 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55dfe1046790 msgr2=0x55dfe1057a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:22:43.264 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:43.261+0000 7fc1457ea740 1 --2- 172.21.15.12:0/4050111833 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55dfe1046790 0x55dfe1057a50 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7fc12c006010 tx=0x7fc12c0046c0 comp rx=0 tx=0).stop 2024-09-11T00:22:43.264 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:43.261+0000 7fc1457ea740 1 -- 172.21.15.12:0/4050111833 shutdown_connections 2024-09-11T00:22:43.264 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:43.261+0000 7fc1457ea740 1 --2- 172.21.15.12:0/4050111833 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55dfe10c8d90 0x55dfe10cb1b0 crc :-1 s=READY pgs=228 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:43.264 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:43.261+0000 7fc1457ea740 1 --2- 172.21.15.12:0/4050111833 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55dfe104bb00 0x55dfe1056d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:43.264 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:43.261+0000 7fc125600640 0 client.17238 ms_handle_reset on v2:172.21.15.165:6838/501312984 2024-09-11T00:22:43.264 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:43.261+0000 7fc1457ea740 1 --2- 172.21.15.12:0/4050111833 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55dfe10490c0 0x55dfe1056750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:43.264 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:43.261+0000 7fc1457ea740 1 --2- 172.21.15.12:0/4050111833 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55dfe1046790 0x55dfe1057a50 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:43.264 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:43.261+0000 7fc1457ea740 1 -- 172.21.15.12:0/4050111833 >> 172.21.15.12:0/4050111833 conn(0x55dfe10418e0 msgr2=0x55dfe1042140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:22:43.265 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:43.261+0000 7fc1457ea740 1 -- 172.21.15.12:0/4050111833 shutdown_connections 2024-09-11T00:22:43.265 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:43.261+0000 7fc1457ea740 1 -- 172.21.15.12:0/4050111833 wait complete. 2024-09-11T00:22:43.265 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:43.261+0000 7fc1457ea740 20 client.17238 trim_cache size 0 max 16384 2024-09-11T00:22:43.265 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:{ 2024-09-11T00:22:43.265 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "message": "", 2024-09-11T00:22:43.265 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "return_code": 0 2024-09-11T00:22:43.265 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:} 2024-09-11T00:22:43.266 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:43.262+0000 7fc142800640 1 -- 172.21.15.12:0/2023602702 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fc1200b5360 msgr2=0x7fc1200b7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:22:43.266 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:43.262+0000 7fc142800640 1 --2- 172.21.15.12:0/2023602702 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fc1200b5360 0x7fc1200b7820 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fc1280105d0 tx=0x7fc128013040 comp rx=0 tx=0).stop 2024-09-11T00:22:43.266 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:43.262+0000 7fc142800640 1 -- 172.21.15.12:0/2023602702 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc13c1044a0 msgr2=0x7fc13c19c340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:22:43.266 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:43.262+0000 7fc142800640 1 --2- 172.21.15.12:0/2023602702 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc13c1044a0 0x7fc13c19c340 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7fc13400b990 tx=0x7fc13400be50 comp rx=0 tx=0).stop 2024-09-11T00:22:43.266 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:43.262+0000 7fc142800640 1 -- 172.21.15.12:0/2023602702 shutdown_connections 2024-09-11T00:22:43.266 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:43.262+0000 7fc142800640 1 --2- 172.21.15.12:0/2023602702 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fc1200b5360 0x7fc1200b7820 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:43.266 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:43.262+0000 7fc142800640 1 --2- 172.21.15.12:0/2023602702 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc13c1044a0 0x7fc13c19c340 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:43.266 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:43.262+0000 7fc142800640 1 --2- 172.21.15.12:0/2023602702 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc13c103ae0 0x7fc13c19b710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:43.267 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:43.262+0000 7fc142800640 1 --2- 172.21.15.12:0/2023602702 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc13c102990 0x7fc13c19b0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:43.267 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:43.262+0000 7fc142800640 1 -- 172.21.15.12:0/2023602702 >> 172.21.15.12:0/2023602702 conn(0x7fc13c0fe250 msgr2=0x7fc13c1092d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:22:43.267 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:43.262+0000 7fc142800640 1 -- 172.21.15.12:0/2023602702 shutdown_connections 2024-09-11T00:22:43.267 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:43.262+0000 7fc142800640 1 -- 172.21.15.12:0/2023602702 wait complete. 2024-09-11T00:22:44.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:22:43 smithi178 ceph-mon[35919]: pgmap v868: 129 pgs: 129 active+clean; 460 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 604 KiB/s rd, 2.7 MiB/s wr, 267 op/s 2024-09-11T00:22:44.169 INFO:tasks.workunit.client.0.smithi012.stdout: 10 51266 24.49 MB/sec execute 40 sec latency 15737.000 ms 2024-09-11T00:22:44.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:22:43 smithi165 ceph-mon[35356]: pgmap v868: 129 pgs: 129 active+clean; 460 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 604 KiB/s rd, 2.7 MiB/s wr, 267 op/s 2024-09-11T00:22:44.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:22:43 smithi012 ceph-mon[30333]: pgmap v868: 129 pgs: 129 active+clean; 460 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 604 KiB/s rd, 2.7 MiB/s wr, 267 op/s 2024-09-11T00:22:45.169 INFO:tasks.workunit.client.0.smithi012.stdout: 10 51266 23.89 MB/sec execute 41 sec latency 16737.059 ms 2024-09-11T00:22:45.910 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:22:45 smithi012 ceph-mon[30333]: pgmap v869: 129 pgs: 129 active+clean; 460 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 213 KiB/s rd, 174 KiB/s wr, 137 op/s 2024-09-11T00:22:46.169 INFO:tasks.workunit.client.0.smithi012.stdout: 10 51266 23.32 MB/sec execute 42 sec latency 17737.102 ms 2024-09-11T00:22:46.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:22:45 smithi165 ceph-mon[35356]: pgmap v869: 129 pgs: 129 active+clean; 460 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 213 KiB/s rd, 174 KiB/s wr, 137 op/s 2024-09-11T00:22:46.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:22:45 smithi178 ceph-mon[35919]: pgmap v869: 129 pgs: 129 active+clean; 460 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 213 KiB/s rd, 174 KiB/s wr, 137 op/s 2024-09-11T00:22:47.170 INFO:tasks.workunit.client.0.smithi012.stdout: 10 51273 22.78 MB/sec execute 43 sec latency 18737.147 ms 2024-09-11T00:22:48.102 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:22:47 smithi012 ceph-mon[30333]: pgmap v870: 129 pgs: 129 active+clean; 460 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 153 KiB/s rd, 113 KiB/s wr, 115 op/s 2024-09-11T00:22:48.102 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:22:47 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:22:48.102 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:22:47 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T00:22:48.102 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:22:47 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:22:48.169 INFO:tasks.workunit.client.0.smithi012.stdout: 10 51376 22.45 MB/sec execute 44 sec latency 19737.206 ms 2024-09-11T00:22:48.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:22:47 smithi165 ceph-mon[35356]: pgmap v870: 129 pgs: 129 active+clean; 460 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 153 KiB/s rd, 113 KiB/s wr, 115 op/s 2024-09-11T00:22:48.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:22:47 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:22:48.283 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:22:47 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T00:22:48.283 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:22:47 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:22:48.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:22:47 smithi178 ceph-mon[35919]: pgmap v870: 129 pgs: 129 active+clean; 460 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 153 KiB/s rd, 113 KiB/s wr, 115 op/s 2024-09-11T00:22:48.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:22:47 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:22:48.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:22:47 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T00:22:48.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:22:47 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:22:48.933 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:22:48 smithi012 ceph-mon[30333]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T00:22:49.169 INFO:tasks.workunit.client.0.smithi012.stdout: 10 51406 21.96 MB/sec execute 45 sec latency 20737.256 ms 2024-09-11T00:22:49.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:22:48 smithi165 ceph-mon[35356]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T00:22:49.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:22:48 smithi178 ceph-mon[35919]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T00:22:50.035 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:22:49 smithi012 ceph-mon[30333]: pgmap v871: 129 pgs: 129 active+clean; 460 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 25 KiB/s wr, 19 op/s 2024-09-11T00:22:50.169 INFO:tasks.workunit.client.0.smithi012.stdout: 10 51406 21.48 MB/sec execute 46 sec latency 21737.308 ms 2024-09-11T00:22:50.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:22:49 smithi165 ceph-mon[35356]: pgmap v871: 129 pgs: 129 active+clean; 460 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 25 KiB/s wr, 19 op/s 2024-09-11T00:22:50.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:22:49 smithi178 ceph-mon[35919]: pgmap v871: 129 pgs: 129 active+clean; 460 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 25 KiB/s wr, 19 op/s 2024-09-11T00:22:51.169 INFO:tasks.workunit.client.0.smithi012.stdout: 10 51823 21.52 MB/sec execute 47 sec latency 22737.360 ms 2024-09-11T00:22:52.113 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:22:51 smithi012 ceph-mon[30333]: pgmap v872: 129 pgs: 129 active+clean; 446 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 163 KiB/s rd, 146 KiB/s wr, 60 op/s 2024-09-11T00:22:52.170 INFO:tasks.workunit.client.0.smithi012.stdout: 10 52849 22.14 MB/sec execute 48 sec latency 23002.851 ms 2024-09-11T00:22:52.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:22:51 smithi165 ceph-mon[35356]: pgmap v872: 129 pgs: 129 active+clean; 446 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 163 KiB/s rd, 146 KiB/s wr, 60 op/s 2024-09-11T00:22:52.395 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:22:51 smithi178 ceph-mon[35919]: pgmap v872: 129 pgs: 129 active+clean; 446 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 163 KiB/s rd, 146 KiB/s wr, 60 op/s 2024-09-11T00:22:53.170 INFO:tasks.workunit.client.0.smithi012.stdout: 10 53672 22.66 MB/sec execute 49 sec latency 60.185 ms 2024-09-11T00:22:54.171 INFO:tasks.workunit.client.0.smithi012.stdout: 10 54442 23.05 MB/sec execute 50 sec latency 138.657 ms 2024-09-11T00:22:54.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:22:53 smithi165 ceph-mon[35356]: pgmap v873: 129 pgs: 129 active+clean; 444 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 407 KiB/s rd, 328 KiB/s wr, 88 op/s 2024-09-11T00:22:54.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:22:53 smithi012 ceph-mon[30333]: pgmap v873: 129 pgs: 129 active+clean; 444 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 407 KiB/s rd, 328 KiB/s wr, 88 op/s 2024-09-11T00:22:54.330 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:22:53 smithi178 ceph-mon[35919]: pgmap v873: 129 pgs: 129 active+clean; 444 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 407 KiB/s rd, 328 KiB/s wr, 88 op/s 2024-09-11T00:22:55.171 INFO:tasks.workunit.client.0.smithi012.stdout: 10 55326 23.57 MB/sec execute 51 sec latency 51.775 ms 2024-09-11T00:22:56.063 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:22:55 smithi012 ceph-mon[30333]: pgmap v874: 129 pgs: 129 active+clean; 447 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 831 KiB/s wr, 156 op/s 2024-09-11T00:22:56.172 INFO:tasks.workunit.client.0.smithi012.stdout: 10 56191 23.97 MB/sec execute 52 sec latency 68.381 ms 2024-09-11T00:22:56.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:22:55 smithi165 ceph-mon[35356]: pgmap v874: 129 pgs: 129 active+clean; 447 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 831 KiB/s wr, 156 op/s 2024-09-11T00:22:56.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:22:55 smithi178 ceph-mon[35919]: pgmap v874: 129 pgs: 129 active+clean; 447 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 831 KiB/s wr, 156 op/s 2024-09-11T00:22:56.473 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:56.470+0000 7f6a95e00640 1 -- 172.21.15.12:0/2302660656 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6a90074470 msgr2=0x7f6a90074870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:22:56.473 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:56.470+0000 7f6a95e00640 1 --2- 172.21.15.12:0/2302660656 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6a90074470 0x7f6a90074870 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f6a88009ca0 tx=0x7f6a8802a8b0 comp rx=0 tx=0).stop 2024-09-11T00:22:56.473 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:56.470+0000 7f6a95e00640 1 -- 172.21.15.12:0/2302660656 shutdown_connections 2024-09-11T00:22:56.473 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:56.470+0000 7f6a95e00640 1 --2- 172.21.15.12:0/2302660656 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6a9010ab10 0x7f6a901136b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:56.473 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:56.470+0000 7f6a95e00640 1 --2- 172.21.15.12:0/2302660656 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6a9010a150 0x7f6a9010a5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:56.473 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:56.470+0000 7f6a95e00640 1 --2- 172.21.15.12:0/2302660656 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6a90074470 0x7f6a90074870 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:56.473 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:56.470+0000 7f6a95e00640 1 -- 172.21.15.12:0/2302660656 >> 172.21.15.12:0/2302660656 conn(0x7f6a9006fa40 msgr2=0x7f6a90071e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:22:56.474 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:56.470+0000 7f6a95e00640 1 -- 172.21.15.12:0/2302660656 shutdown_connections 2024-09-11T00:22:56.474 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:56.470+0000 7f6a95e00640 1 -- 172.21.15.12:0/2302660656 wait complete. 2024-09-11T00:22:56.474 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:56.471+0000 7f6a95e00640 1 Processor -- start 2024-09-11T00:22:56.474 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:56.471+0000 7f6a95e00640 1 -- start start 2024-09-11T00:22:56.475 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:56.472+0000 7f6a95e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6a90074470 0x7f6a901a15a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:22:56.475 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:56.472+0000 7f6a95e00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6a9010a150 0x7f6a901a1bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:22:56.475 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:56.472+0000 7f6a95e00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6a9010ab10 0x7f6a901a2820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:22:56.475 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:56.472+0000 7f6a95e00640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f6a90116e20 con 0x7f6a9010ab10 2024-09-11T00:22:56.475 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:56.472+0000 7f6a95e00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f6a90116ca0 con 0x7f6a90074470 2024-09-11T00:22:56.475 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:56.472+0000 7f6a95e00640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f6a90116fa0 con 0x7f6a9010a150 2024-09-11T00:22:56.475 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:56.472+0000 7f6a8ea00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6a90074470 0x7f6a901a15a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:22:56.475 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:56.472+0000 7f6a8ea00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6a90074470 0x7f6a901a15a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.12:59000/0 (socket says 172.21.15.12:59000) 2024-09-11T00:22:56.475 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:56.472+0000 7f6a8ea00640 1 -- 172.21.15.12:0/2897115727 learned_addr learned my addr 172.21.15.12:0/2897115727 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:22:56.476 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:56.473+0000 7f6a8f400640 1 --2- 172.21.15.12:0/2897115727 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6a9010ab10 0x7f6a901a2820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:22:56.476 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:56.473+0000 7f6a8e000640 1 --2- 172.21.15.12:0/2897115727 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6a9010a150 0x7f6a901a1bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:22:56.476 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:56.474+0000 7f6a8ea00640 1 -- 172.21.15.12:0/2897115727 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6a9010a150 msgr2=0x7f6a901a1bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:22:56.477 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:56.474+0000 7f6a8ea00640 1 --2- 172.21.15.12:0/2897115727 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6a9010a150 0x7f6a901a1bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:56.477 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:56.474+0000 7f6a8ea00640 1 -- 172.21.15.12:0/2897115727 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6a9010ab10 msgr2=0x7f6a901a2820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:22:56.477 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:56.474+0000 7f6a8ea00640 1 --2- 172.21.15.12:0/2897115727 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6a9010ab10 0x7f6a901a2820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:56.477 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:56.474+0000 7f6a8ea00640 1 -- 172.21.15.12:0/2897115727 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a901a30e0 con 0x7f6a90074470 2024-09-11T00:22:56.477 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:56.474+0000 7f6a8e000640 1 --2- 172.21.15.12:0/2897115727 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6a9010a150 0x7f6a901a1bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T00:22:56.477 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:56.475+0000 7f6a8ea00640 1 --2- 172.21.15.12:0/2897115727 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6a90074470 0x7f6a901a15a0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f6a8802aea0 tx=0x7f6a88003c20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:22:56.477 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:56.475+0000 7f6a94a00640 1 -- 172.21.15.12:0/2897115727 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a88004450 con 0x7f6a90074470 2024-09-11T00:22:56.478 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:56.475+0000 7f6a95e00640 1 -- 172.21.15.12:0/2897115727 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6a901a3370 con 0x7f6a90074470 2024-09-11T00:22:56.478 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:56.475+0000 7f6a95e00640 1 -- 172.21.15.12:0/2897115727 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6a901a9aa0 con 0x7f6a90074470 2024-09-11T00:22:56.478 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:56.475+0000 7f6a94a00640 1 -- 172.21.15.12:0/2897115727 <== mon.2 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6a88040070 con 0x7f6a90074470 2024-09-11T00:22:56.478 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:56.475+0000 7f6a94a00640 1 -- 172.21.15.12:0/2897115727 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a88045c10 con 0x7f6a90074470 2024-09-11T00:22:56.483 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:56.480+0000 7f6a98c30740 1 Processor -- start 2024-09-11T00:22:56.483 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:56.480+0000 7f6a98c30740 1 -- start start 2024-09-11T00:22:56.483 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:56.480+0000 7f6a98c30740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a5b3880ba0 0x55a5b3882f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:22:56.483 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:56.480+0000 7f6a98c30740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a5b38834d0 0x55a5b38858c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:22:56.483 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:56.480+0000 7f6a98c30740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a5b3885f10 0x55a5b38883d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:22:56.484 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:56.480+0000 7f6a98c30740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55a5b37ef6b0 con 0x55a5b3880ba0 2024-09-11T00:22:56.484 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:56.480+0000 7f6a98c30740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55a5b37deae0 con 0x55a5b38834d0 2024-09-11T00:22:56.484 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:56.480+0000 7f6a98c30740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55a5b37e28f0 con 0x55a5b3885f10 2024-09-11T00:22:56.484 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:56.481+0000 7f6a94a00640 1 -- 172.21.15.12:0/2897115727 <== mon.2 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7f6a88045470 con 0x7f6a90074470 2024-09-11T00:22:56.484 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:56.481+0000 7f6a8e000640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a5b3885f10 0x55a5b38883d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:22:56.485 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:56.481+0000 7f6a8e000640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a5b3885f10 0x55a5b38883d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.12:36936/0 (socket says 172.21.15.12:36936) 2024-09-11T00:22:56.485 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:56.481+0000 7f6a8e000640 1 -- 172.21.15.12:0/3395593029 learned_addr learned my addr 172.21.15.12:0/3395593029 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:22:56.485 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:56.482+0000 7f6a8e000640 1 -- 172.21.15.12:0/3395593029 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a5b38834d0 msgr2=0x55a5b38858c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-11T00:22:56.485 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:56.482+0000 7f6a8e000640 1 --2- 172.21.15.12:0/3395593029 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a5b38834d0 0x55a5b38858c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:56.485 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:56.482+0000 7f6a8e000640 1 -- 172.21.15.12:0/3395593029 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a5b3880ba0 msgr2=0x55a5b3882f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:22:56.485 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:56.482+0000 7f6a8e000640 1 --2- 172.21.15.12:0/3395593029 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a5b3880ba0 0x55a5b3882f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:56.485 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:56.482+0000 7f6a8e000640 1 -- 172.21.15.12:0/3395593029 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a5b37ab780 con 0x55a5b3885f10 2024-09-11T00:22:56.486 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:56.482+0000 7f6a8e000640 1 --2- 172.21.15.12:0/3395593029 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a5b3885f10 0x55a5b38883d0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f6a7c00cca0 tx=0x7f6a7c00e720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9b4319c9b79a3d32 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:22:56.486 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:56.482+0000 7f6a7be00640 1 -- 172.21.15.12:0/3395593029 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a7c005220 con 0x55a5b3885f10 2024-09-11T00:22:56.486 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:56.483+0000 7f6a7be00640 1 -- 172.21.15.12:0/3395593029 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6a7c004c50 con 0x55a5b3885f10 2024-09-11T00:22:56.486 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:56.483+0000 7f6a7be00640 1 -- 172.21.15.12:0/3395593029 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a7c010830 con 0x55a5b3885f10 2024-09-11T00:22:56.486 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:56.483+0000 7f6a94a00640 1 --2- 172.21.15.12:0/2897115727 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f6a740b53e0 0x7f6a740b78a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:22:56.487 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:56.485+0000 7f6a8f400640 1 --2- 172.21.15.12:0/2897115727 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f6a740b53e0 0x7f6a740b78a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:22:56.488 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:56.486+0000 7f6a94a00640 1 -- 172.21.15.12:0/2897115727 <== mon.2 v2:172.21.15.165:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9208+0+0 (secure 0 0 0) 0x7f6a88107530 con 0x7f6a90074470 2024-09-11T00:22:56.490 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:56.488+0000 7f6a98c30740 1 -- 172.21.15.12:0/3395593029 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a5b3885f10 msgr2=0x55a5b38883d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:22:56.490 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:56.488+0000 7f6a98c30740 1 --2- 172.21.15.12:0/3395593029 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a5b3885f10 0x55a5b38883d0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f6a7c00cca0 tx=0x7f6a7c00e720 comp rx=0 tx=0).stop 2024-09-11T00:22:56.490 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:56.488+0000 7f6a8f400640 1 --2- 172.21.15.12:0/2897115727 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f6a740b53e0 0x7f6a740b78a0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f6a901a2320 tx=0x7f6a80009370 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:22:56.491 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:56.489+0000 7f6a98c30740 1 -- 172.21.15.12:0/3395593029 shutdown_connections 2024-09-11T00:22:56.491 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:56.489+0000 7f6a98c30740 1 --2- 172.21.15.12:0/3395593029 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a5b3885f10 0x55a5b38883d0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:56.491 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:56.489+0000 7f6a98c30740 1 --2- 172.21.15.12:0/3395593029 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a5b38834d0 0x55a5b38858c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:56.492 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:56.489+0000 7f6a98c30740 1 --2- 172.21.15.12:0/3395593029 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a5b3880ba0 0x55a5b3882f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:56.492 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:56.489+0000 7f6a98c30740 1 -- 172.21.15.12:0/3395593029 >> 172.21.15.12:0/3395593029 conn(0x55a5b387bcf0 msgr2=0x55a5b387ca80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:22:56.492 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:56.490+0000 7f6a98c30740 1 -- 172.21.15.12:0/3395593029 shutdown_connections 2024-09-11T00:22:56.493 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:56.490+0000 7f6a98c30740 1 -- 172.21.15.12:0/3395593029 wait complete. 2024-09-11T00:22:56.493 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:56.491+0000 7f6a98c30740 1 -- start start 2024-09-11T00:22:56.494 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:56.492+0000 7f6a98c30740 1 Processor -- start 2024-09-11T00:22:56.496 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:56.493+0000 7f6a98c30740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a5b3880ba0 0x55a5b387aec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:22:56.496 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:56.493+0000 7f6a98c30740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a5b38834d0 0x55a5b3879bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:22:56.496 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:56.493+0000 7f6a98c30740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a5b3885f10 0x55a5b387a100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:22:56.496 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:56.493+0000 7f6a98c30740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55a5b3716ab0 con 0x55a5b3885f10 2024-09-11T00:22:56.496 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:56.493+0000 7f6a98c30740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55a5b3785570 con 0x55a5b38834d0 2024-09-11T00:22:56.496 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:56.493+0000 7f6a98c30740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55a5b38773d0 con 0x55a5b3880ba0 2024-09-11T00:22:56.496 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:56.493+0000 7f6a98c30740 10 client.0 _ll_register_callbacks cb 0x55a5b3794100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:22:56.496 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:56.493+0000 7f6a8e000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a5b38834d0 0x55a5b3879bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:22:56.497 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:56.493+0000 7f6a8e000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a5b38834d0 0x55a5b3879bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.12:59012/0 (socket says 172.21.15.12:59012) 2024-09-11T00:22:56.497 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:56.493+0000 7f6a8e000640 1 -- 172.21.15.12:0/3026009032 learned_addr learned my addr 172.21.15.12:0/3026009032 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:22:56.497 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:56.494+0000 7f6a8e000640 1 -- 172.21.15.12:0/3026009032 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a5b3880ba0 msgr2=0x55a5b387aec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:22:56.497 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:56.494+0000 7f6a8ea00640 1 --2- 172.21.15.12:0/3026009032 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a5b3880ba0 0x55a5b387aec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:22:56.497 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:56.494+0000 7f6a8f400640 1 --2- 172.21.15.12:0/3026009032 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a5b3885f10 0x55a5b387a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:22:56.497 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:56.494+0000 7f6a8e000640 1 --2- 172.21.15.12:0/3026009032 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a5b3880ba0 0x55a5b387aec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:56.497 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:56.494+0000 7f6a8e000640 1 -- 172.21.15.12:0/3026009032 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a5b3885f10 msgr2=0x55a5b387a100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:22:56.498 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:56.494+0000 7f6a8e000640 1 --2- 172.21.15.12:0/3026009032 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a5b3885f10 0x55a5b387a100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:56.498 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:56.494+0000 7f6a8e000640 1 -- 172.21.15.12:0/3026009032 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a5b3540790 con 0x55a5b38834d0 2024-09-11T00:22:56.498 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:56.495+0000 7f6a8ea00640 1 --2- 172.21.15.12:0/3026009032 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a5b3880ba0 0x55a5b387aec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T00:22:56.498 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:56.495+0000 7f6a8f400640 1 --2- 172.21.15.12:0/3026009032 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a5b3885f10 0x55a5b387a100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T00:22:56.498 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:56.496+0000 7f6a8e000640 1 --2- 172.21.15.12:0/3026009032 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a5b38834d0 0x55a5b3879bc0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f6a7c004c10 tx=0x7f6a7c0055b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e73525b007a9b42c server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:22:56.499 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:56.497+0000 7f6a78c00640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-09-11T00:22:56.499 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:56.497+0000 7f6a78c00640 1 -- 172.21.15.12:0/3026009032 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a7c02a060 con 0x55a5b38834d0 2024-09-11T00:22:56.499 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:56.497+0000 7f6a78c00640 1 -- 172.21.15.12:0/3026009032 <== mon.2 v2:172.21.15.165:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9208+0+0 (secure 0 0 0) 0x7f6a7c0050d0 con 0x55a5b38834d0 2024-09-11T00:22:56.500 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:56.497+0000 7f6a98c30740 1 -- 172.21.15.12:0/3026009032 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6a7c004c50 con 0x55a5b38834d0 2024-09-11T00:22:56.500 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:56.498+0000 7f6a78c00640 1 -- 172.21.15.12:0/3026009032 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=59) ==== 24+0+0 (secure 0 0 0) 0x7f6a7c0052d0 con 0x55a5b38834d0 2024-09-11T00:22:56.500 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:56.498+0000 7f6a98c30740 10 client.27263 fetch_fsmap learned FSMap version 59 2024-09-11T00:22:56.500 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:56.498+0000 7f6a98c30740 1 -- 172.21.15.12:0/3026009032 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=59,monmap=4+}) -- 0x55a5b35b9620 con 0x55a5b38834d0 2024-09-11T00:22:56.501 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:56.498+0000 7f6a78c00640 1 -- 172.21.15.12:0/3026009032 <== mon.2 v2:172.21.15.165:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6a7c004c50 con 0x55a5b38834d0 2024-09-11T00:22:56.501 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:56.498+0000 7f6a78c00640 1 -- 172.21.15.12:0/3026009032 <== mon.2 v2:172.21.15.165:3300/0 5 ==== fsmap(e 59) ==== 8394+0+0 (secure 0 0 0) 0x7f6a7c025600 con 0x55a5b38834d0 2024-09-11T00:22:56.501 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:56.499+0000 7f6a98c30740 10 client.27263 fetch_fsmap finished waiting for FSMap version 59 2024-09-11T00:22:56.501 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:56.499+0000 7f6a98c30740 10 client.27263 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T00:22:56.501 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:56.499+0000 7f6a98c30740 20 client.27263 populate_metadata read hostname 'smithi012' 2024-09-11T00:22:56.502 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:56.499+0000 7f6a98c30740 1 --2- 172.21.15.12:0/3026009032 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55a5b38a24c0 0x55a5b38a48b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:22:56.502 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:56.499+0000 7f6a98c30740 4 client.27263 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T00:22:56.502 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:56.499+0000 7f6a98c30740 1 -- 172.21.15.12:0/3026009032 --> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a5b37f1600 con 0x55a5b38a24c0 2024-09-11T00:22:56.502 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:56.499+0000 7f6a8ea00640 1 --2- 172.21.15.12:0/3026009032 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55a5b38a24c0 0x55a5b38a48b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:22:56.502 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:56.500+0000 7f6a8ea00640 1 --2- 172.21.15.12:0/3026009032 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55a5b38a24c0 0x55a5b38a48b0 crc :-1 s=READY pgs=229 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b781173b9a36f31e server_cookie=237bf98a32886796 in_seq=0 out_seq=0 2024-09-11T00:22:56.504 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:56.501+0000 7f6a78c00640 10 client.27263 ms_handle_connect on v2:172.21.15.165:6838/501312984 2024-09-11T00:22:56.504 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:56.501+0000 7f6a78c00640 1 -- 172.21.15.12:0/3026009032 <== mds.0 v2:172.21.15.165:6838/501312984 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55a5b37f1600 con 0x55a5b38a24c0 2024-09-11T00:22:56.504 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:56.501+0000 7f6a78c00640 10 client.27263 handle_command_reply: tid=0 2024-09-11T00:22:56.504 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:56.502+0000 7f6a98c30740 1 client.27263 shutdown 2024-09-11T00:22:56.505 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:56.503+0000 7f6a98c30740 1 -- 172.21.15.12:0/3026009032 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a5b38834d0 msgr2=0x55a5b3879bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:22:56.505 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:56.503+0000 7f6a98c30740 1 --2- 172.21.15.12:0/3026009032 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a5b38834d0 0x55a5b3879bc0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f6a7c004c10 tx=0x7f6a7c0055b0 comp rx=0 tx=0).stop 2024-09-11T00:22:56.505 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:56.503+0000 7f6a98c30740 1 -- 172.21.15.12:0/3026009032 shutdown_connections 2024-09-11T00:22:56.505 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:56.503+0000 7f6a98c30740 1 --2- 172.21.15.12:0/3026009032 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55a5b38a24c0 0x55a5b38a48b0 crc :-1 s=READY pgs=229 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:56.505 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:56.503+0000 7f6a98c30740 1 --2- 172.21.15.12:0/3026009032 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a5b3885f10 0x55a5b387a100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:56.506 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:56.503+0000 7f6a98c30740 1 --2- 172.21.15.12:0/3026009032 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a5b38834d0 0x55a5b3879bc0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:56.506 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:56.503+0000 7f6a98c30740 1 --2- 172.21.15.12:0/3026009032 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a5b3880ba0 0x55a5b387aec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:56.506 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:56.503+0000 7f6a98c30740 1 -- 172.21.15.12:0/3026009032 >> 172.21.15.12:0/3026009032 conn(0x55a5b387bcf0 msgr2=0x55a5b3876a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:22:56.506 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:56.503+0000 7f6a78c00640 0 client.27263 ms_handle_reset on v2:172.21.15.165:6838/501312984 2024-09-11T00:22:56.506 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:56.504+0000 7f6a98c30740 1 -- 172.21.15.12:0/3026009032 shutdown_connections 2024-09-11T00:22:56.506 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:56.504+0000 7f6a98c30740 1 -- 172.21.15.12:0/3026009032 wait complete. 2024-09-11T00:22:56.508 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:56.505+0000 7f6a98c30740 20 client.27263 trim_cache size 0 max 16384 2024-09-11T00:22:56.553 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:56.551+0000 7f6a98c30740 1 Processor -- start 2024-09-11T00:22:56.554 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:56.551+0000 7f6a98c30740 1 -- start start 2024-09-11T00:22:56.555 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:56.553+0000 7f6a98c30740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a5b3880ba0 0x55a5b38801f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:22:56.555 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:56.553+0000 7f6a98c30740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a5b38834d0 0x55a5b38862f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:22:56.555 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:56.553+0000 7f6a98c30740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a5b3885f10 0x55a5b3886830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:22:56.555 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:56.553+0000 7f6a98c30740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f6a901b1ca0 con 0x55a5b3885f10 2024-09-11T00:22:56.556 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:56.553+0000 7f6a98c30740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f6a901b3900 con 0x55a5b38834d0 2024-09-11T00:22:56.556 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:56.554+0000 7f6a98c30740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f6a901b23d0 con 0x55a5b3880ba0 2024-09-11T00:22:56.556 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:56.554+0000 7f6a8e000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a5b38834d0 0x55a5b38862f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:22:56.556 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:56.554+0000 7f6a8e000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a5b38834d0 0x55a5b38862f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.12:59022/0 (socket says 172.21.15.12:59022) 2024-09-11T00:22:56.556 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:56.554+0000 7f6a8e000640 1 -- 172.21.15.12:0/3072780276 learned_addr learned my addr 172.21.15.12:0/3072780276 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:22:56.557 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:56.554+0000 7f6a8f400640 1 --2- 172.21.15.12:0/3072780276 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a5b3885f10 0x55a5b3886830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:22:56.557 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:56.554+0000 7f6a8ea00640 1 --2- 172.21.15.12:0/3072780276 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a5b3880ba0 0x55a5b38801f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:22:56.558 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:56.555+0000 7f6a8ea00640 1 -- 172.21.15.12:0/3072780276 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a5b38834d0 msgr2=0x55a5b38862f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:22:56.558 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:56.555+0000 7f6a8ea00640 1 --2- 172.21.15.12:0/3072780276 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a5b38834d0 0x55a5b38862f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:56.558 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:56.555+0000 7f6a8ea00640 1 -- 172.21.15.12:0/3072780276 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a5b3885f10 msgr2=0x55a5b3886830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:22:56.558 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:56.555+0000 7f6a8ea00640 1 --2- 172.21.15.12:0/3072780276 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a5b3885f10 0x55a5b3886830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:56.558 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:56.555+0000 7f6a8ea00640 1 -- 172.21.15.12:0/3072780276 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a5b3883300 con 0x55a5b3880ba0 2024-09-11T00:22:56.559 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:56.556+0000 7f6a8f400640 1 --2- 172.21.15.12:0/3072780276 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a5b3885f10 0x55a5b3886830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T00:22:56.559 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:56.556+0000 7f6a8ea00640 1 --2- 172.21.15.12:0/3072780276 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a5b3880ba0 0x55a5b38801f0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f6a88104bd0 tx=0x7f6a8803e040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f946e35fd4dbab95 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:22:56.559 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:56.557+0000 7f6a7b400640 1 -- 172.21.15.12:0/3072780276 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a880cd3d0 con 0x55a5b3880ba0 2024-09-11T00:22:56.560 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:56.558+0000 7f6a7b400640 1 -- 172.21.15.12:0/3072780276 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6a880f4070 con 0x55a5b3880ba0 2024-09-11T00:22:56.560 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:56.558+0000 7f6a7b400640 1 -- 172.21.15.12:0/3072780276 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a881071a0 con 0x55a5b3880ba0 2024-09-11T00:22:56.561 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:56.559+0000 7f6a98c30740 1 -- 172.21.15.12:0/3072780276 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a5b3880ba0 msgr2=0x55a5b38801f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:22:56.561 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:56.559+0000 7f6a98c30740 1 --2- 172.21.15.12:0/3072780276 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a5b3880ba0 0x55a5b38801f0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f6a88104bd0 tx=0x7f6a8803e040 comp rx=0 tx=0).stop 2024-09-11T00:22:56.562 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:56.559+0000 7f6a98c30740 1 -- 172.21.15.12:0/3072780276 shutdown_connections 2024-09-11T00:22:56.562 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:56.559+0000 7f6a98c30740 1 --2- 172.21.15.12:0/3072780276 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a5b3885f10 0x55a5b3886830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:56.562 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:56.559+0000 7f6a98c30740 1 --2- 172.21.15.12:0/3072780276 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a5b38834d0 0x55a5b38862f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:56.562 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:56.559+0000 7f6a98c30740 1 --2- 172.21.15.12:0/3072780276 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a5b3880ba0 0x55a5b38801f0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:56.562 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:56.559+0000 7f6a98c30740 1 -- 172.21.15.12:0/3072780276 >> 172.21.15.12:0/3072780276 conn(0x55a5b387bcf0 msgr2=0x55a5b38a44d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:22:56.562 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:56.560+0000 7f6a98c30740 1 -- 172.21.15.12:0/3072780276 shutdown_connections 2024-09-11T00:22:56.562 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:56.561+0000 7f6a98c30740 1 -- 172.21.15.12:0/3072780276 wait complete. 2024-09-11T00:22:56.563 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:56.561+0000 7f6a98c30740 1 -- start start 2024-09-11T00:22:56.563 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:56.561+0000 7f6a98c30740 1 Processor -- start 2024-09-11T00:22:56.564 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:56.562+0000 7f6a98c30740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a5b3880ba0 0x55a5b388d6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:22:56.564 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:56.562+0000 7f6a98c30740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a5b38834d0 0x55a5b388dc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:22:56.564 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:56.562+0000 7f6a98c30740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a5b3885f10 0x55a5b388e160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:22:56.564 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:56.562+0000 7f6a98c30740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f6a901b3230 con 0x55a5b3880ba0 2024-09-11T00:22:56.565 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:56.562+0000 7f6a98c30740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f6a901b2ad0 con 0x55a5b3885f10 2024-09-11T00:22:56.565 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:56.562+0000 7f6a98c30740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f6a901afc80 con 0x55a5b38834d0 2024-09-11T00:22:56.565 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:56.562+0000 7f6a98c30740 10 client.0 _ll_register_callbacks cb 0x7f6a7c014070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:22:56.565 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:56.562+0000 7f6a8ea00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a5b3880ba0 0x55a5b388d6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:22:56.565 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:56.563+0000 7f6a8ea00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a5b3880ba0 0x55a5b388d6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:49646/0 (socket says 172.21.15.12:49646) 2024-09-11T00:22:56.565 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:56.563+0000 7f6a8ea00640 1 -- 172.21.15.12:0/3625068178 learned_addr learned my addr 172.21.15.12:0/3625068178 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:22:56.565 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:56.563+0000 7f6a8e000640 1 --2- 172.21.15.12:0/3625068178 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a5b38834d0 0x55a5b388dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:22:56.566 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:56.563+0000 7f6a8ea00640 1 -- 172.21.15.12:0/3625068178 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a5b38834d0 msgr2=0x55a5b388dc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:22:56.566 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:56.563+0000 7f6a8ea00640 1 --2- 172.21.15.12:0/3625068178 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a5b38834d0 0x55a5b388dc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:56.566 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:56.563+0000 7f6a8ea00640 1 -- 172.21.15.12:0/3625068178 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a5b3885f10 msgr2=0x55a5b388e160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:22:56.566 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:56.563+0000 7f6a8ea00640 1 --2- 172.21.15.12:0/3625068178 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a5b3885f10 0x55a5b388e160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:56.566 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:56.563+0000 7f6a8ea00640 1 -- 172.21.15.12:0/3625068178 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a5b388c5e0 con 0x55a5b3880ba0 2024-09-11T00:22:56.566 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:56.564+0000 7f6a8ea00640 1 --2- 172.21.15.12:0/3625068178 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a5b3880ba0 0x55a5b388d6e0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f6a880d7010 tx=0x7f6a88045df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1fbf16eff5fdac6 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:22:56.566 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:56.564+0000 7f6a78c00640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-09-11T00:22:56.568 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:56.566+0000 7f6a78c00640 1 -- 172.21.15.12:0/3625068178 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a880ec020 con 0x55a5b3880ba0 2024-09-11T00:22:56.568 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:56.566+0000 7f6a78c00640 1 -- 172.21.15.12:0/3625068178 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9208+0+0 (secure 0 0 0) 0x7f6a880c9b30 con 0x55a5b3880ba0 2024-09-11T00:22:56.569 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:56.567+0000 7f6a98c30740 1 -- 172.21.15.12:0/3625068178 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6a880f4070 con 0x55a5b3880ba0 2024-09-11T00:22:56.572 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:56.570+0000 7f6a78c00640 1 -- 172.21.15.12:0/3625068178 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=59) ==== 24+0+0 (secure 0 0 0) 0x7f6a88003e00 con 0x55a5b3880ba0 2024-09-11T00:22:56.572 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:56.570+0000 7f6a98c30740 10 client.17250 fetch_fsmap learned FSMap version 59 2024-09-11T00:22:56.572 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:56.570+0000 7f6a98c30740 1 -- 172.21.15.12:0/3625068178 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=59,monmap=4+}) -- 0x55a5b390f820 con 0x55a5b3880ba0 2024-09-11T00:22:56.573 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:56.570+0000 7f6a78c00640 1 -- 172.21.15.12:0/3625068178 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6a880f4070 con 0x55a5b3880ba0 2024-09-11T00:22:56.574 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:56.571+0000 7f6a78c00640 1 -- 172.21.15.12:0/3625068178 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 59) ==== 8394+0+0 (secure 0 0 0) 0x7f6a880cdb20 con 0x55a5b3880ba0 2024-09-11T00:22:56.575 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:56.573+0000 7f6a98c30740 10 client.17250 fetch_fsmap finished waiting for FSMap version 59 2024-09-11T00:22:56.575 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:56.573+0000 7f6a98c30740 10 client.17250 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T00:22:56.575 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:56.573+0000 7f6a98c30740 20 client.17250 populate_metadata read hostname 'smithi012' 2024-09-11T00:22:56.575 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:56.573+0000 7f6a98c30740 1 --2- 172.21.15.12:0/3625068178 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55a5b39101c0 0x55a5b39125e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:22:56.576 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:56.573+0000 7f6a98c30740 4 client.17250 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T00:22:56.576 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:56.574+0000 7f6a98c30740 1 -- 172.21.15.12:0/3625068178 --> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55a5b37ed0b0 con 0x55a5b39101c0 2024-09-11T00:22:56.577 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:56.574+0000 7f6a8e000640 1 --2- 172.21.15.12:0/3625068178 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55a5b39101c0 0x55a5b39125e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:22:56.578 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:56.575+0000 7f6a8e000640 1 --2- 172.21.15.12:0/3625068178 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55a5b39101c0 0x55a5b39125e0 crc :-1 s=READY pgs=230 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=528306b6e650f122 server_cookie=7cecd1214b7a6092 in_seq=0 out_seq=0 2024-09-11T00:22:56.578 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:56.575+0000 7f6a78c00640 10 client.17250 ms_handle_connect on v2:172.21.15.165:6838/501312984 2024-09-11T00:22:56.805 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:56.803+0000 7f6a78c00640 1 -- 172.21.15.12:0/3625068178 <== mds.0 v2:172.21.15.165:6838/501312984 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55a5b37ed0b0 con 0x55a5b39101c0 2024-09-11T00:22:56.805 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:56.803+0000 7f6a78c00640 10 client.17250 handle_command_reply: tid=0 2024-09-11T00:22:56.807 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:56.805+0000 7f6a98c30740 1 client.17250 shutdown 2024-09-11T00:22:56.807 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:56.805+0000 7f6a98c30740 1 -- 172.21.15.12:0/3625068178 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a5b3880ba0 msgr2=0x55a5b388d6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:22:56.807 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:56.805+0000 7f6a98c30740 1 --2- 172.21.15.12:0/3625068178 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a5b3880ba0 0x55a5b388d6e0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f6a880d7010 tx=0x7f6a88045df0 comp rx=0 tx=0).stop 2024-09-11T00:22:56.808 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:56.806+0000 7f6a98c30740 1 -- 172.21.15.12:0/3625068178 shutdown_connections 2024-09-11T00:22:56.808 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:56.806+0000 7f6a98c30740 1 --2- 172.21.15.12:0/3625068178 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55a5b39101c0 0x55a5b39125e0 crc :-1 s=READY pgs=230 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:56.808 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:56.806+0000 7f6a98c30740 1 --2- 172.21.15.12:0/3625068178 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a5b3885f10 0x55a5b388e160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:56.808 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:56.806+0000 7f6a98c30740 1 --2- 172.21.15.12:0/3625068178 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a5b38834d0 0x55a5b388dc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:56.808 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:56.806+0000 7f6a98c30740 1 --2- 172.21.15.12:0/3625068178 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a5b3880ba0 0x55a5b388d6e0 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:56.808 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:56.806+0000 7f6a78c00640 0 client.17250 ms_handle_reset on v2:172.21.15.165:6838/501312984 2024-09-11T00:22:56.808 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:56.806+0000 7f6a98c30740 1 -- 172.21.15.12:0/3625068178 >> 172.21.15.12:0/3625068178 conn(0x55a5b387bcf0 msgr2=0x55a5b38a44d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:22:56.809 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:56.806+0000 7f6a98c30740 1 -- 172.21.15.12:0/3625068178 shutdown_connections 2024-09-11T00:22:56.809 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:56.806+0000 7f6a98c30740 1 -- 172.21.15.12:0/3625068178 wait complete. 2024-09-11T00:22:56.809 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:{ 2024-09-11T00:22:56.809 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "message": "", 2024-09-11T00:22:56.810 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "return_code": 0 2024-09-11T00:22:56.810 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:} 2024-09-11T00:22:56.810 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:56.806+0000 7f6a98c30740 20 client.17250 trim_cache size 0 max 16384 2024-09-11T00:22:56.810 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:56.808+0000 7f6a95e00640 1 -- 172.21.15.12:0/2897115727 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f6a740b53e0 msgr2=0x7f6a740b78a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:22:56.810 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:56.808+0000 7f6a95e00640 1 --2- 172.21.15.12:0/2897115727 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f6a740b53e0 0x7f6a740b78a0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f6a901a2320 tx=0x7f6a80009370 comp rx=0 tx=0).stop 2024-09-11T00:22:56.810 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:56.808+0000 7f6a95e00640 1 -- 172.21.15.12:0/2897115727 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6a90074470 msgr2=0x7f6a901a15a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:22:56.811 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:56.808+0000 7f6a95e00640 1 --2- 172.21.15.12:0/2897115727 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6a90074470 0x7f6a901a15a0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f6a8802aea0 tx=0x7f6a88003c20 comp rx=0 tx=0).stop 2024-09-11T00:22:56.811 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:56.808+0000 7f6a95e00640 1 -- 172.21.15.12:0/2897115727 shutdown_connections 2024-09-11T00:22:56.811 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:56.808+0000 7f6a95e00640 1 --2- 172.21.15.12:0/2897115727 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f6a740b53e0 0x7f6a740b78a0 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:56.811 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:56.808+0000 7f6a95e00640 1 --2- 172.21.15.12:0/2897115727 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6a9010ab10 0x7f6a901a2820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:56.811 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:56.808+0000 7f6a95e00640 1 --2- 172.21.15.12:0/2897115727 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6a9010a150 0x7f6a901a1bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:56.811 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:56.808+0000 7f6a95e00640 1 --2- 172.21.15.12:0/2897115727 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6a90074470 0x7f6a901a15a0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:56.811 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:56.808+0000 7f6a95e00640 1 -- 172.21.15.12:0/2897115727 >> 172.21.15.12:0/2897115727 conn(0x7f6a9006fa40 msgr2=0x7f6a9010f760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:22:56.814 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:56.812+0000 7f6a95e00640 1 -- 172.21.15.12:0/2897115727 shutdown_connections 2024-09-11T00:22:56.814 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:22:56.812+0000 7f6a95e00640 1 -- 172.21.15.12:0/2897115727 wait complete. 2024-09-11T00:22:57.171 INFO:tasks.workunit.client.0.smithi012.stdout: 10 56888 24.13 MB/sec execute 53 sec latency 145.561 ms 2024-09-11T00:22:58.171 INFO:tasks.workunit.client.0.smithi012.stdout: 10 57880 24.62 MB/sec execute 54 sec latency 55.201 ms 2024-09-11T00:22:58.257 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:22:57 smithi012 ceph-mon[30333]: pgmap v875: 129 pgs: 129 active+clean; 455 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 1.9 MiB/s wr, 293 op/s 2024-09-11T00:22:58.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:22:57 smithi165 ceph-mon[35356]: pgmap v875: 129 pgs: 129 active+clean; 455 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 1.9 MiB/s wr, 293 op/s 2024-09-11T00:22:58.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:22:57 smithi178 ceph-mon[35919]: pgmap v875: 129 pgs: 129 active+clean; 455 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 1.9 MiB/s wr, 293 op/s 2024-09-11T00:22:59.170 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:22:58 smithi012 ceph-mon[30333]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T00:22:59.171 INFO:tasks.workunit.client.0.smithi012.stdout: 10 58398 24.82 MB/sec execute 55 sec latency 524.760 ms 2024-09-11T00:22:59.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:22:58 smithi165 ceph-mon[35356]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T00:22:59.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:22:58 smithi178 ceph-mon[35919]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T00:23:00.173 INFO:tasks.workunit.client.0.smithi012.stdout: 10 58485 24.48 MB/sec execute 56 sec latency 1526.310 ms 2024-09-11T00:23:00.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:22:59 smithi165 ceph-mon[35356]: pgmap v876: 129 pgs: 129 active+clean; 457 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 2.2 MiB/s wr, 325 op/s 2024-09-11T00:23:00.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:22:59 smithi012 ceph-mon[30333]: pgmap v876: 129 pgs: 129 active+clean; 457 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 2.2 MiB/s wr, 325 op/s 2024-09-11T00:23:00.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:22:59 smithi178 ceph-mon[35919]: pgmap v876: 129 pgs: 129 active+clean; 457 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 2.2 MiB/s wr, 325 op/s 2024-09-11T00:23:01.173 INFO:tasks.workunit.client.0.smithi012.stdout: 10 58512 24.06 MB/sec execute 57 sec latency 2526.377 ms 2024-09-11T00:23:02.173 INFO:tasks.workunit.client.0.smithi012.stdout: 10 58867 24.07 MB/sec execute 58 sec latency 3196.138 ms 2024-09-11T00:23:02.191 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:23:01 smithi012 ceph-mon[30333]: pgmap v877: 129 pgs: 129 active+clean; 476 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 5.6 MiB/s wr, 559 op/s 2024-09-11T00:23:02.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:23:01 smithi165 ceph-mon[35356]: pgmap v877: 129 pgs: 129 active+clean; 476 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 5.6 MiB/s wr, 559 op/s 2024-09-11T00:23:02.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:23:01 smithi178 ceph-mon[35919]: pgmap v877: 129 pgs: 129 active+clean; 476 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 5.6 MiB/s wr, 559 op/s 2024-09-11T00:23:03.173 INFO:tasks.workunit.client.0.smithi012.stdout: 10 59725 24.30 MB/sec execute 59 sec latency 65.230 ms 2024-09-11T00:23:04.064 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:23:03 smithi012 ceph-mon[30333]: pgmap v878: 129 pgs: 129 active+clean; 480 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 MiB/s rd, 6.3 MiB/s wr, 609 op/s 2024-09-11T00:23:04.173 INFO:tasks.workunit.client.0.smithi012.stdout: 10 60857 24.94 MB/sec execute 60 sec latency 39.690 ms 2024-09-11T00:23:04.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:23:03 smithi165 ceph-mon[35356]: pgmap v878: 129 pgs: 129 active+clean; 480 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 MiB/s rd, 6.3 MiB/s wr, 609 op/s 2024-09-11T00:23:04.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:23:03 smithi178 ceph-mon[35919]: pgmap v878: 129 pgs: 129 active+clean; 480 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 MiB/s rd, 6.3 MiB/s wr, 609 op/s 2024-09-11T00:23:05.173 INFO:tasks.workunit.client.0.smithi012.stdout: 10 61904 25.57 MB/sec execute 61 sec latency 107.749 ms 2024-09-11T00:23:06.173 INFO:tasks.workunit.client.0.smithi012.stdout: 10 62621 25.73 MB/sec execute 62 sec latency 110.107 ms 2024-09-11T00:23:06.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:23:05 smithi165 ceph-mon[35356]: pgmap v879: 129 pgs: 129 active+clean; 475 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 MiB/s rd, 6.3 MiB/s wr, 609 op/s 2024-09-11T00:23:06.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:23:05 smithi012 ceph-mon[30333]: pgmap v879: 129 pgs: 129 active+clean; 475 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 MiB/s rd, 6.3 MiB/s wr, 609 op/s 2024-09-11T00:23:06.330 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:23:05 smithi178 ceph-mon[35919]: pgmap v879: 129 pgs: 129 active+clean; 475 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 MiB/s rd, 6.3 MiB/s wr, 609 op/s 2024-09-11T00:23:07.173 INFO:tasks.workunit.client.0.smithi012.stdout: 10 63500 25.87 MB/sec execute 63 sec latency 68.238 ms 2024-09-11T00:23:08.173 INFO:tasks.workunit.client.0.smithi012.stdout: 10 64531 26.41 MB/sec execute 64 sec latency 153.286 ms 2024-09-11T00:23:08.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:23:07 smithi165 ceph-mon[35356]: pgmap v880: 129 pgs: 129 active+clean; 480 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 7.3 MiB/s wr, 738 op/s 2024-09-11T00:23:08.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:23:07 smithi012 ceph-mon[30333]: pgmap v880: 129 pgs: 129 active+clean; 480 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 7.3 MiB/s wr, 738 op/s 2024-09-11T00:23:08.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:23:07 smithi178 ceph-mon[35919]: pgmap v880: 129 pgs: 129 active+clean; 480 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 7.3 MiB/s wr, 738 op/s 2024-09-11T00:23:09.173 INFO:tasks.workunit.client.0.smithi012.stdout: 10 65530 27.10 MB/sec execute 65 sec latency 168.917 ms 2024-09-11T00:23:09.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:23:08 smithi165 ceph-mon[35356]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T00:23:09.308 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:23:08 smithi012 ceph-mon[30333]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T00:23:09.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:23:08 smithi178 ceph-mon[35919]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T00:23:09.946 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:09.944+0000 7fa0e6600640 1 -- 172.21.15.12:0/4002039593 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa0e01066a0 msgr2=0x7fa0e010d020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:23:09.946 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:09.944+0000 7fa0e6600640 1 --2- 172.21.15.12:0/4002039593 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa0e01066a0 0x7fa0e010d020 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7fa0d8009cd0 tx=0x7fa0d802a8f0 comp rx=0 tx=0).stop 2024-09-11T00:23:09.946 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:09.944+0000 7fa0e6600640 1 -- 172.21.15.12:0/4002039593 shutdown_connections 2024-09-11T00:23:09.946 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:09.944+0000 7fa0e6600640 1 --2- 172.21.15.12:0/4002039593 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa0e01066a0 0x7fa0e010d020 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:09.946 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:09.944+0000 7fa0e6600640 1 --2- 172.21.15.12:0/4002039593 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa0e0105ce0 0x7fa0e0106160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:09.946 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:09.944+0000 7fa0e6600640 1 --2- 172.21.15.12:0/4002039593 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa0e0104b90 0x7fa0e0104f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:09.947 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:09.944+0000 7fa0e6600640 1 -- 172.21.15.12:0/4002039593 >> 172.21.15.12:0/4002039593 conn(0x7fa0e0100470 msgr2=0x7fa0e0102890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:23:09.947 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:09.945+0000 7fa0e6600640 1 -- 172.21.15.12:0/4002039593 shutdown_connections 2024-09-11T00:23:09.947 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:09.945+0000 7fa0e6600640 1 -- 172.21.15.12:0/4002039593 wait complete. 2024-09-11T00:23:09.949 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:09.946+0000 7fa0e6600640 1 Processor -- start 2024-09-11T00:23:09.949 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:09.946+0000 7fa0e6600640 1 -- start start 2024-09-11T00:23:09.949 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:09.946+0000 7fa0e6600640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa0e0104b90 0x7fa0e019af20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:23:09.949 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:09.946+0000 7fa0e6600640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa0e0105ce0 0x7fa0e019b570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:23:09.949 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:09.946+0000 7fa0e6600640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa0e01066a0 0x7fa0e019c1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:23:09.949 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:09.946+0000 7fa0e6600640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fa0e0110850 con 0x7fa0e01066a0 2024-09-11T00:23:09.949 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:09.946+0000 7fa0e6600640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fa0e01106d0 con 0x7fa0e0105ce0 2024-09-11T00:23:09.950 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:09.946+0000 7fa0e6600640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fa0e01109d0 con 0x7fa0e0104b90 2024-09-11T00:23:09.950 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:09.946+0000 7fa0dfe00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa0e01066a0 0x7fa0e019c1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:23:09.950 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:09.946+0000 7fa0dfe00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa0e01066a0 0x7fa0e019c1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:48298/0 (socket says 172.21.15.12:48298) 2024-09-11T00:23:09.950 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:09.946+0000 7fa0dfe00640 1 -- 172.21.15.12:0/3901564161 learned_addr learned my addr 172.21.15.12:0/3901564161 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:23:09.950 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:09.946+0000 7fa0df400640 1 --2- 172.21.15.12:0/3901564161 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa0e0104b90 0x7fa0e019af20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:23:09.950 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:09.946+0000 7fa0d6a00640 1 --2- 172.21.15.12:0/3901564161 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa0e0105ce0 0x7fa0e019b570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:23:09.950 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:09.947+0000 7fa0df400640 1 -- 172.21.15.12:0/3901564161 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa0e0105ce0 msgr2=0x7fa0e019b570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:23:09.950 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:09.947+0000 7fa0df400640 1 --2- 172.21.15.12:0/3901564161 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa0e0105ce0 0x7fa0e019b570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:09.950 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:09.947+0000 7fa0df400640 1 -- 172.21.15.12:0/3901564161 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa0e01066a0 msgr2=0x7fa0e019c1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:23:09.950 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:09.947+0000 7fa0df400640 1 --2- 172.21.15.12:0/3901564161 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa0e01066a0 0x7fa0e019c1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:09.950 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:09.947+0000 7fa0df400640 1 -- 172.21.15.12:0/3901564161 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0e019ca60 con 0x7fa0e0104b90 2024-09-11T00:23:09.950 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:09.947+0000 7fa0dfe00640 1 --2- 172.21.15.12:0/3901564161 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa0e01066a0 0x7fa0e019c1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:23:09.950 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:09.947+0000 7fa0df400640 1 --2- 172.21.15.12:0/3901564161 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa0e0104b90 0x7fa0e019af20 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fa0d000e9f0 tx=0x7fa0d000eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:23:09.951 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:09.947+0000 7fa0dcc00640 1 -- 172.21.15.12:0/3901564161 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0d000cd80 con 0x7fa0e0104b90 2024-09-11T00:23:09.951 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:09.947+0000 7fa0e6600640 1 -- 172.21.15.12:0/3901564161 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa0e019cd50 con 0x7fa0e0104b90 2024-09-11T00:23:09.951 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:09.947+0000 7fa0dcc00640 1 -- 172.21.15.12:0/3901564161 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa0d00035f0 con 0x7fa0e0104b90 2024-09-11T00:23:09.951 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:09.947+0000 7fa0dcc00640 1 -- 172.21.15.12:0/3901564161 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0d0010690 con 0x7fa0e0104b90 2024-09-11T00:23:09.951 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:09.948+0000 7fa0e6600640 1 -- 172.21.15.12:0/3901564161 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0e01a3420 con 0x7fa0e0104b90 2024-09-11T00:23:09.952 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:09.951+0000 7fa0e9609740 1 Processor -- start 2024-09-11T00:23:09.953 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:09.952+0000 7fa0e9609740 1 -- start start 2024-09-11T00:23:09.954 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:09.952+0000 7fa0e9609740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55618625a790 0x55618625cb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:23:09.954 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:09.952+0000 7fa0e9609740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55618625d0c0 0x55618625f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:23:09.954 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:09.952+0000 7fa0e9609740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55618625fb00 0x556186261fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:23:09.954 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:09.952+0000 7fa0e9609740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x5561861cd6b0 con 0x55618625a790 2024-09-11T00:23:09.954 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:09.952+0000 7fa0e9609740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x5561861bcae0 con 0x55618625fb00 2024-09-11T00:23:09.954 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:09.952+0000 7fa0e9609740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5561861c08f0 con 0x55618625d0c0 2024-09-11T00:23:09.954 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:09.952+0000 7fa0dfe00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55618625a790 0x55618625cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:23:09.954 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:09.952+0000 7fa0d6a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55618625fb00 0x556186261fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:23:09.955 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:09.952+0000 7fa0d6a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55618625fb00 0x556186261fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.12:51688/0 (socket says 172.21.15.12:51688) 2024-09-11T00:23:09.955 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:09.952+0000 7fa0dfe00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55618625a790 0x55618625cb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:48314/0 (socket says 172.21.15.12:48314) 2024-09-11T00:23:09.955 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:09.952+0000 7fa0d6a00640 1 -- 172.21.15.12:0/3563854208 learned_addr learned my addr 172.21.15.12:0/3563854208 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:23:09.955 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:09.953+0000 7fa0d6a00640 1 -- 172.21.15.12:0/3563854208 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55618625d0c0 msgr2=0x55618625f4b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-11T00:23:09.955 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:09.953+0000 7fa0d6a00640 1 --2- 172.21.15.12:0/3563854208 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55618625d0c0 0x55618625f4b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:09.955 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:09.953+0000 7fa0d6a00640 1 -- 172.21.15.12:0/3563854208 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55618625a790 msgr2=0x55618625cb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:23:09.955 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:09.953+0000 7fa0d6a00640 1 --2- 172.21.15.12:0/3563854208 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55618625a790 0x55618625cb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:09.955 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:09.953+0000 7fa0d6a00640 1 -- 172.21.15.12:0/3563854208 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556186189780 con 0x55618625fb00 2024-09-11T00:23:09.955 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:09.953+0000 7fa0dcc00640 1 -- 172.21.15.12:0/3901564161 <== mon.1 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7fa0d0004000 con 0x7fa0e0104b90 2024-09-11T00:23:09.955 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:09.953+0000 7fa0d6a00640 1 --2- 172.21.15.12:0/3563854208 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55618625fb00 0x556186261fc0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fa0c8009d20 tx=0x7fa0c8011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d179b785561bc473 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:23:09.959 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:09.953+0000 7fa0d4c00640 1 -- 172.21.15.12:0/3563854208 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0c801e070 con 0x55618625fb00 2024-09-11T00:23:09.959 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:09.953+0000 7fa0d4c00640 1 -- 172.21.15.12:0/3563854208 <== mon.2 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa0c8019ae0 con 0x55618625fb00 2024-09-11T00:23:09.959 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:09.953+0000 7fa0d4c00640 1 -- 172.21.15.12:0/3563854208 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0c800ea60 con 0x55618625fb00 2024-09-11T00:23:09.959 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:09.954+0000 7fa0e9609740 1 -- 172.21.15.12:0/3563854208 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55618625fb00 msgr2=0x556186261fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:23:09.959 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:09.954+0000 7fa0e9609740 1 --2- 172.21.15.12:0/3563854208 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55618625fb00 0x556186261fc0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fa0c8009d20 tx=0x7fa0c8011810 comp rx=0 tx=0).stop 2024-09-11T00:23:09.959 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:09.954+0000 7fa0e9609740 1 -- 172.21.15.12:0/3563854208 shutdown_connections 2024-09-11T00:23:09.959 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:09.954+0000 7fa0e9609740 1 --2- 172.21.15.12:0/3563854208 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55618625fb00 0x556186261fc0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:09.960 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:09.954+0000 7fa0e9609740 1 --2- 172.21.15.12:0/3563854208 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55618625d0c0 0x55618625f4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:09.960 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:09.954+0000 7fa0e9609740 1 --2- 172.21.15.12:0/3563854208 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55618625a790 0x55618625cb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:09.960 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:09.954+0000 7fa0e9609740 1 -- 172.21.15.12:0/3563854208 >> 172.21.15.12:0/3563854208 conn(0x5561862558e0 msgr2=0x556186256670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:23:09.960 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:09.954+0000 7fa0e9609740 1 -- 172.21.15.12:0/3563854208 shutdown_connections 2024-09-11T00:23:09.960 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:09.954+0000 7fa0e9609740 1 -- 172.21.15.12:0/3563854208 wait complete. 2024-09-11T00:23:09.960 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:09.954+0000 7fa0dcc00640 1 --2- 172.21.15.12:0/3901564161 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fa0c00b5270 0x7fa0c00b7730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:23:09.960 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:09.954+0000 7fa0e9609740 1 -- start start 2024-09-11T00:23:09.960 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:09.954+0000 7fa0dcc00640 1 -- 172.21.15.12:0/3901564161 <== mon.1 v2:172.21.15.178:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9208+0+0 (secure 0 0 0) 0x7fa0d00e6200 con 0x7fa0e0104b90 2024-09-11T00:23:09.960 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:09.955+0000 7fa0dfe00640 1 --2- 172.21.15.12:0/3901564161 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fa0c00b5270 0x7fa0c00b7730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:23:09.960 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:09.955+0000 7fa0e9609740 1 Processor -- start 2024-09-11T00:23:09.960 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:09.955+0000 7fa0e9609740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55618625a790 0x556186263f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:23:09.961 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:09.955+0000 7fa0e9609740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55618625d0c0 0x5561862645e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:23:09.961 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:09.955+0000 7fa0e9609740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55618625fb00 0x556186265140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:23:09.961 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:09.955+0000 7fa0e9609740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x5561860f4ab0 con 0x55618625fb00 2024-09-11T00:23:09.961 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:09.955+0000 7fa0e9609740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x556186163570 con 0x55618625a790 2024-09-11T00:23:09.961 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:09.955+0000 7fa0e9609740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x556186262ae0 con 0x55618625d0c0 2024-09-11T00:23:09.961 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:09.955+0000 7fa0e9609740 10 client.0 _ll_register_callbacks cb 0x556186172100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:23:09.961 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:09.955+0000 7fa0dfe00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55618625fb00 0x556186265140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:23:09.961 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:09.955+0000 7fa0d6a00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55618625d0c0 0x5561862645e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:23:09.961 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:09.956+0000 7fa0df400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55618625a790 0x556186263f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:23:09.961 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:09.956+0000 7fa0dfe00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55618625fb00 0x556186265140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:48326/0 (socket says 172.21.15.12:48326) 2024-09-11T00:23:09.961 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:09.956+0000 7fa0d6a00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55618625d0c0 0x5561862645e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.12:45976/0 (socket says 172.21.15.12:45976) 2024-09-11T00:23:09.961 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:09.956+0000 7fa0dfe00640 1 -- 172.21.15.12:0/1675695285 learned_addr learned my addr 172.21.15.12:0/1675695285 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:23:09.961 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:09.956+0000 7fa0dfe00640 1 --2- 172.21.15.12:0/3901564161 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fa0c00b5270 0x7fa0c00b7730 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fa0e019bca0 tx=0x7fa0d8030800 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:23:09.962 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:09.956+0000 7fa0d6a00640 1 -- 172.21.15.12:0/1675695285 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55618625a790 msgr2=0x556186263f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:23:09.962 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:09.956+0000 7fa0d6a00640 1 --2- 172.21.15.12:0/1675695285 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55618625a790 0x556186263f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:09.962 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:09.956+0000 7fa0d6a00640 1 -- 172.21.15.12:0/1675695285 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55618625fb00 msgr2=0x556186265140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:23:09.962 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:09.956+0000 7fa0d6a00640 1 --2- 172.21.15.12:0/1675695285 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55618625fb00 0x556186265140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:09.962 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:09.956+0000 7fa0d6a00640 1 -- 172.21.15.12:0/1675695285 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556185f1e790 con 0x55618625d0c0 2024-09-11T00:23:09.962 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:09.956+0000 7fa0df400640 1 --2- 172.21.15.12:0/1675695285 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55618625a790 0x556186263f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:23:09.962 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:09.956+0000 7fa0d6a00640 1 --2- 172.21.15.12:0/1675695285 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55618625d0c0 0x5561862645e0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fa0c8005890 tx=0x7fa0c8004270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a2511554e95d1b9c server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:23:09.962 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:09.956+0000 7fa0cd600640 10 client.0 ms_handle_connect on v2:172.21.15.178:3300/0 2024-09-11T00:23:09.962 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:09.957+0000 7fa0cd600640 1 -- 172.21.15.12:0/1675695285 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0c8027020 con 0x55618625d0c0 2024-09-11T00:23:09.962 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:09.957+0000 7fa0cd600640 1 -- 172.21.15.12:0/1675695285 <== mon.1 v2:172.21.15.178:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9208+0+0 (secure 0 0 0) 0x7fa0c8019630 con 0x55618625d0c0 2024-09-11T00:23:09.962 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:09.957+0000 7fa0e9609740 1 -- 172.21.15.12:0/1675695285 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa0c8019ae0 con 0x55618625d0c0 2024-09-11T00:23:09.962 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:09.957+0000 7fa0cd600640 1 -- 172.21.15.12:0/1675695285 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) ==== 24+0+0 (secure 0 0 0) 0x7fa0c8032050 con 0x55618625d0c0 2024-09-11T00:23:09.962 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:09.957+0000 7fa0e9609740 10 client.27244 fetch_fsmap learned FSMap version 61 2024-09-11T00:23:09.962 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:09.957+0000 7fa0e9609740 1 -- 172.21.15.12:0/1675695285 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=4+}) -- 0x556185f97620 con 0x55618625d0c0 2024-09-11T00:23:09.963 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:09.957+0000 7fa0cd600640 1 -- 172.21.15.12:0/1675695285 <== mon.1 v2:172.21.15.178:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa0c8019ae0 con 0x55618625d0c0 2024-09-11T00:23:09.963 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:09.957+0000 7fa0cd600640 1 -- 172.21.15.12:0/1675695285 <== mon.1 v2:172.21.15.178:3300/0 5 ==== fsmap(e 61) ==== 8386+0+0 (secure 0 0 0) 0x7fa0c800e820 con 0x55618625d0c0 2024-09-11T00:23:09.963 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:09.957+0000 7fa0e9609740 10 client.27244 fetch_fsmap finished waiting for FSMap version 61 2024-09-11T00:23:09.963 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:09.957+0000 7fa0e9609740 10 client.27244 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T00:23:09.963 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:09.957+0000 7fa0e9609740 20 client.27244 populate_metadata read hostname 'smithi012' 2024-09-11T00:23:09.963 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:09.958+0000 7fa0e9609740 1 --2- 172.21.15.12:0/1675695285 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55618626f5e0 0x556186271a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:23:09.963 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:09.958+0000 7fa0e9609740 4 client.27244 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T00:23:09.963 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:09.958+0000 7fa0e9609740 1 -- 172.21.15.12:0/1675695285 --> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5561861cf600 con 0x55618626f5e0 2024-09-11T00:23:09.963 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:09.958+0000 7fa0df400640 1 --2- 172.21.15.12:0/1675695285 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55618626f5e0 0x556186271a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:23:09.964 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:09.958+0000 7fa0df400640 1 --2- 172.21.15.12:0/1675695285 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55618626f5e0 0x556186271a00 crc :-1 s=READY pgs=231 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b5349ae64c7e536 server_cookie=e062579daf46a453 in_seq=0 out_seq=0 2024-09-11T00:23:09.964 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:09.958+0000 7fa0cd600640 10 client.27244 ms_handle_connect on v2:172.21.15.165:6838/501312984 2024-09-11T00:23:09.964 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:09.959+0000 7fa0cd600640 1 -- 172.21.15.12:0/1675695285 <== mds.0 v2:172.21.15.165:6838/501312984 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5561861cf600 con 0x55618626f5e0 2024-09-11T00:23:09.965 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:09.959+0000 7fa0cd600640 10 client.27244 handle_command_reply: tid=0 2024-09-11T00:23:09.965 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:09.959+0000 7fa0e9609740 1 client.27244 shutdown 2024-09-11T00:23:09.965 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:09.959+0000 7fa0e9609740 1 -- 172.21.15.12:0/1675695285 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55618625d0c0 msgr2=0x5561862645e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:23:09.965 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:09.959+0000 7fa0e9609740 1 --2- 172.21.15.12:0/1675695285 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55618625d0c0 0x5561862645e0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fa0c8005890 tx=0x7fa0c8004270 comp rx=0 tx=0).stop 2024-09-11T00:23:09.965 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:09.959+0000 7fa0e9609740 1 -- 172.21.15.12:0/1675695285 shutdown_connections 2024-09-11T00:23:09.965 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:09.959+0000 7fa0e9609740 1 --2- 172.21.15.12:0/1675695285 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55618626f5e0 0x556186271a00 crc :-1 s=READY pgs=231 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:09.965 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:09.959+0000 7fa0e9609740 1 --2- 172.21.15.12:0/1675695285 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55618625fb00 0x556186265140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:09.965 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:09.959+0000 7fa0cd600640 0 client.27244 ms_handle_reset on v2:172.21.15.165:6838/501312984 2024-09-11T00:23:09.965 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:09.959+0000 7fa0e9609740 1 --2- 172.21.15.12:0/1675695285 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55618625d0c0 0x5561862645e0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:09.965 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:09.959+0000 7fa0e9609740 1 --2- 172.21.15.12:0/1675695285 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55618625a790 0x556186263f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:09.965 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:09.959+0000 7fa0e9609740 1 -- 172.21.15.12:0/1675695285 >> 172.21.15.12:0/1675695285 conn(0x5561862558e0 msgr2=0x556186254a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:23:09.965 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:09.960+0000 7fa0e9609740 1 -- 172.21.15.12:0/1675695285 shutdown_connections 2024-09-11T00:23:09.965 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:09.960+0000 7fa0e9609740 1 -- 172.21.15.12:0/1675695285 wait complete. 2024-09-11T00:23:09.966 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:09.960+0000 7fa0e9609740 20 client.27244 trim_cache size 0 max 16384 2024-09-11T00:23:09.977 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:09.975+0000 7fa0e9609740 1 Processor -- start 2024-09-11T00:23:09.977 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:09.975+0000 7fa0e9609740 1 -- start start 2024-09-11T00:23:09.978 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:09.975+0000 7fa0e9609740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55618625a790 0x556186250960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:23:09.978 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:09.975+0000 7fa0e9609740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55618625d0c0 0x55618625fee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:23:09.978 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:09.975+0000 7fa0e9609740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55618625fb00 0x556186260420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:23:09.978 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:09.975+0000 7fa0e9609740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fa0e01ad970 con 0x55618625d0c0 2024-09-11T00:23:09.978 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:09.976+0000 7fa0e9609740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fa0e01af570 con 0x55618625a790 2024-09-11T00:23:09.978 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:09.976+0000 7fa0e9609740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fa0e01ae0a0 con 0x55618625fb00 2024-09-11T00:23:09.978 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:09.976+0000 7fa0d6a00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55618625d0c0 0x55618625fee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:23:09.978 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:09.976+0000 7fa0d6a00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55618625d0c0 0x55618625fee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:48334/0 (socket says 172.21.15.12:48334) 2024-09-11T00:23:09.978 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:09.976+0000 7fa0d6a00640 1 -- 172.21.15.12:0/2506263329 learned_addr learned my addr 172.21.15.12:0/2506263329 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:23:09.978 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:09.976+0000 7fa0df400640 1 --2- 172.21.15.12:0/2506263329 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55618625a790 0x556186250960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:23:09.978 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:09.976+0000 7fa0dfe00640 1 --2- 172.21.15.12:0/2506263329 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55618625fb00 0x556186260420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:23:09.978 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:09.976+0000 7fa0d6a00640 1 -- 172.21.15.12:0/2506263329 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55618625fb00 msgr2=0x556186260420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:23:09.979 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:09.976+0000 7fa0d6a00640 1 --2- 172.21.15.12:0/2506263329 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55618625fb00 0x556186260420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:09.979 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:09.976+0000 7fa0d6a00640 1 -- 172.21.15.12:0/2506263329 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55618625a790 msgr2=0x556186250960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:23:09.979 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:09.976+0000 7fa0d6a00640 1 --2- 172.21.15.12:0/2506263329 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55618625a790 0x556186250960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:09.979 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:09.976+0000 7fa0d6a00640 1 -- 172.21.15.12:0/2506263329 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55618625cef0 con 0x55618625d0c0 2024-09-11T00:23:09.979 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:09.976+0000 7fa0d6a00640 1 --2- 172.21.15.12:0/2506263329 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55618625d0c0 0x55618625fee0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7fa0c800ebc0 tx=0x7fa0c8009b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d23780932af05cb server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:23:09.981 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:09.977+0000 7fa0cfe00640 1 -- 172.21.15.12:0/2506263329 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0c8027070 con 0x55618625d0c0 2024-09-11T00:23:09.981 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:09.977+0000 7fa0cfe00640 1 -- 172.21.15.12:0/2506263329 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa0c801f040 con 0x55618625d0c0 2024-09-11T00:23:09.981 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:09.977+0000 7fa0cfe00640 1 -- 172.21.15.12:0/2506263329 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0c801e070 con 0x55618625d0c0 2024-09-11T00:23:09.981 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:09.977+0000 7fa0e9609740 1 -- 172.21.15.12:0/2506263329 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55618625d0c0 msgr2=0x55618625fee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:23:09.982 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:09.977+0000 7fa0e9609740 1 --2- 172.21.15.12:0/2506263329 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55618625d0c0 0x55618625fee0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7fa0c800ebc0 tx=0x7fa0c8009b70 comp rx=0 tx=0).stop 2024-09-11T00:23:09.982 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:09.977+0000 7fa0e9609740 1 -- 172.21.15.12:0/2506263329 shutdown_connections 2024-09-11T00:23:09.982 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:09.977+0000 7fa0e9609740 1 --2- 172.21.15.12:0/2506263329 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55618625fb00 0x556186260420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:09.982 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:09.977+0000 7fa0e9609740 1 --2- 172.21.15.12:0/2506263329 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55618625d0c0 0x55618625fee0 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:09.982 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:09.977+0000 7fa0e9609740 1 --2- 172.21.15.12:0/2506263329 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55618625a790 0x556186250960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:09.982 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:09.977+0000 7fa0e9609740 1 -- 172.21.15.12:0/2506263329 >> 172.21.15.12:0/2506263329 conn(0x5561862558e0 msgr2=0x556186256380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:23:09.982 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:09.977+0000 7fa0e9609740 1 -- 172.21.15.12:0/2506263329 shutdown_connections 2024-09-11T00:23:09.982 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:09.977+0000 7fa0e9609740 1 -- 172.21.15.12:0/2506263329 wait complete. 2024-09-11T00:23:09.982 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:09.978+0000 7fa0e9609740 1 -- start start 2024-09-11T00:23:09.982 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:09.978+0000 7fa0e9609740 1 Processor -- start 2024-09-11T00:23:09.982 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:09.978+0000 7fa0e9609740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55618625a790 0x55618626ac40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:23:09.982 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:09.978+0000 7fa0e9609740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55618625d0c0 0x55618626b180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:23:09.982 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:09.978+0000 7fa0e9609740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55618625fb00 0x5561862697b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:23:09.983 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:09.978+0000 7fa0e9609740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fa0e01aeea0 con 0x55618625d0c0 2024-09-11T00:23:09.983 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:09.978+0000 7fa0e9609740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fa0e01ae7a0 con 0x55618625a790 2024-09-11T00:23:09.983 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:09.978+0000 7fa0e9609740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fa0e01ab950 con 0x55618625fb00 2024-09-11T00:23:09.983 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:09.978+0000 7fa0e9609740 10 client.0 _ll_register_callbacks cb 0x7fa0c801c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:23:09.983 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:09.979+0000 7fa0df400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55618625a790 0x55618626ac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:23:09.983 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:09.979+0000 7fa0dfe00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55618625fb00 0x5561862697b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:23:09.983 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:09.979+0000 7fa0d6a00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55618625d0c0 0x55618626b180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:23:09.983 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:09.979+0000 7fa0df400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55618625a790 0x55618626ac40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.12:51712/0 (socket says 172.21.15.12:51712) 2024-09-11T00:23:09.983 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:09.979+0000 7fa0dfe00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55618625fb00 0x5561862697b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.12:45992/0 (socket says 172.21.15.12:45992) 2024-09-11T00:23:09.983 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:09.979+0000 7fa0df400640 1 -- 172.21.15.12:0/902919958 learned_addr learned my addr 172.21.15.12:0/902919958 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:23:09.983 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:09.979+0000 7fa0df400640 1 -- 172.21.15.12:0/902919958 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55618625fb00 msgr2=0x5561862697b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:23:09.983 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:09.979+0000 7fa0df400640 1 --2- 172.21.15.12:0/902919958 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55618625fb00 0x5561862697b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:09.983 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:09.979+0000 7fa0df400640 1 -- 172.21.15.12:0/902919958 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55618625d0c0 msgr2=0x55618626b180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:23:09.984 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:09.979+0000 7fa0df400640 1 --2- 172.21.15.12:0/902919958 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55618625d0c0 0x55618626b180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:09.984 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:09.979+0000 7fa0df400640 1 -- 172.21.15.12:0/902919958 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556186269cf0 con 0x55618625a790 2024-09-11T00:23:09.984 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:09.979+0000 7fa0d6a00640 1 --2- 172.21.15.12:0/902919958 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55618625d0c0 0x55618626b180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:23:09.984 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:09.980+0000 7fa0df400640 1 --2- 172.21.15.12:0/902919958 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55618625a790 0x55618626ac40 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fa0d00e3980 tx=0x7fa0d00e3800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8421e39689442015 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:23:09.984 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:09.980+0000 7fa0cd600640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-09-11T00:23:09.984 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:09.982+0000 7fa0cd600640 1 -- 172.21.15.12:0/902919958 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0d00c3070 con 0x55618625a790 2024-09-11T00:23:09.984 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:09.982+0000 7fa0cd600640 1 -- 172.21.15.12:0/902919958 <== mon.2 v2:172.21.15.165:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9208+0+0 (secure 0 0 0) 0x7fa0d00037a0 con 0x55618625a790 2024-09-11T00:23:09.984 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:09.982+0000 7fa0e9609740 1 -- 172.21.15.12:0/902919958 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa0c801f040 con 0x55618625a790 2024-09-11T00:23:10.053 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:10.049+0000 7fa0cd600640 1 -- 172.21.15.12:0/902919958 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) ==== 24+0+0 (secure 0 0 0) 0x7fa0d0097030 con 0x55618625a790 2024-09-11T00:23:10.053 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:10.049+0000 7fa0e9609740 10 client.27284 fetch_fsmap learned FSMap version 61 2024-09-11T00:23:10.053 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:10.049+0000 7fa0e9609740 1 -- 172.21.15.12:0/902919958 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=4+}) -- 0x5561862dcc90 con 0x55618625a790 2024-09-11T00:23:10.053 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:10.050+0000 7fa0cd600640 1 -- 172.21.15.12:0/902919958 <== mon.2 v2:172.21.15.165:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa0c801f040 con 0x55618625a790 2024-09-11T00:23:10.053 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:10.050+0000 7fa0cd600640 1 -- 172.21.15.12:0/902919958 <== mon.2 v2:172.21.15.165:3300/0 5 ==== fsmap(e 61) ==== 8386+0+0 (secure 0 0 0) 0x7fa0d00c6020 con 0x55618625a790 2024-09-11T00:23:10.054 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:10.050+0000 7fa0e9609740 10 client.27284 fetch_fsmap finished waiting for FSMap version 61 2024-09-11T00:23:10.054 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:10.050+0000 7fa0e9609740 10 client.27284 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T00:23:10.054 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:10.050+0000 7fa0e9609740 20 client.27284 populate_metadata read hostname 'smithi012' 2024-09-11T00:23:10.054 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:10.050+0000 7fa0e9609740 1 --2- 172.21.15.12:0/902919958 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x5561862dd5d0 0x5561862df9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:23:10.054 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:10.050+0000 7fa0e9609740 4 client.27284 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T00:23:10.054 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:10.050+0000 7fa0e9609740 1 -- 172.21.15.12:0/902919958 --> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5561861cb0b0 con 0x5561862dd5d0 2024-09-11T00:23:10.055 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:10.051+0000 7fa0d6a00640 1 --2- 172.21.15.12:0/902919958 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x5561862dd5d0 0x5561862df9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:23:10.055 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:10.052+0000 7fa0d6a00640 1 --2- 172.21.15.12:0/902919958 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x5561862dd5d0 0x5561862df9f0 crc :-1 s=READY pgs=232 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=221a2e724156693b server_cookie=754e8a7d1993bfb4 in_seq=0 out_seq=0 2024-09-11T00:23:10.055 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:10.052+0000 7fa0cd600640 10 client.27284 ms_handle_connect on v2:172.21.15.165:6838/501312984 2024-09-11T00:23:10.174 INFO:tasks.workunit.client.0.smithi012.stdout: 10 65890 26.85 MB/sec execute 66 sec latency 415.813 ms 2024-09-11T00:23:10.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:23:09 smithi165 ceph-mon[35356]: pgmap v881: 129 pgs: 129 active+clean; 481 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 6.2 MiB/s wr, 605 op/s 2024-09-11T00:23:10.306 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:23:09 smithi012 ceph-mon[30333]: pgmap v881: 129 pgs: 129 active+clean; 481 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 6.2 MiB/s wr, 605 op/s 2024-09-11T00:23:10.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:23:09 smithi178 ceph-mon[35919]: pgmap v881: 129 pgs: 129 active+clean; 481 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 6.2 MiB/s wr, 605 op/s 2024-09-11T00:23:10.512 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:10.511+0000 7fa0cd600640 1 -- 172.21.15.12:0/902919958 <== mds.0 v2:172.21.15.165:6838/501312984 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5561861cb0b0 con 0x5561862dd5d0 2024-09-11T00:23:10.512 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:10.511+0000 7fa0cd600640 10 client.27284 handle_command_reply: tid=0 2024-09-11T00:23:10.512 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:10.511+0000 7fa0e9609740 1 client.27284 shutdown 2024-09-11T00:23:10.515 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:10.511+0000 7fa0e9609740 1 -- 172.21.15.12:0/902919958 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55618625a790 msgr2=0x55618626ac40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:23:10.515 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:10.513+0000 7fa0e9609740 1 --2- 172.21.15.12:0/902919958 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55618625a790 0x55618626ac40 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fa0d00e3980 tx=0x7fa0d00e3800 comp rx=0 tx=0).stop 2024-09-11T00:23:10.515 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:10.513+0000 7fa0e9609740 1 -- 172.21.15.12:0/902919958 shutdown_connections 2024-09-11T00:23:10.515 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:10.513+0000 7fa0e9609740 1 --2- 172.21.15.12:0/902919958 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x5561862dd5d0 0x5561862df9f0 crc :-1 s=READY pgs=232 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:10.515 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:10.513+0000 7fa0e9609740 1 --2- 172.21.15.12:0/902919958 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55618625fb00 0x5561862697b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:10.515 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:10.513+0000 7fa0cd600640 0 client.27284 ms_handle_reset on v2:172.21.15.165:6838/501312984 2024-09-11T00:23:10.515 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:10.514+0000 7fa0e9609740 1 --2- 172.21.15.12:0/902919958 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55618625d0c0 0x55618626b180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:10.515 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:10.514+0000 7fa0e9609740 1 --2- 172.21.15.12:0/902919958 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55618625a790 0x55618626ac40 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:10.515 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:10.514+0000 7fa0e9609740 1 -- 172.21.15.12:0/902919958 >> 172.21.15.12:0/902919958 conn(0x5561862558e0 msgr2=0x55618625fee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:23:10.516 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:10.514+0000 7fa0e9609740 1 -- 172.21.15.12:0/902919958 shutdown_connections 2024-09-11T00:23:10.516 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:10.514+0000 7fa0e9609740 1 -- 172.21.15.12:0/902919958 wait complete. 2024-09-11T00:23:10.516 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:10.514+0000 7fa0e9609740 20 client.27284 trim_cache size 0 max 16384 2024-09-11T00:23:10.516 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:{ 2024-09-11T00:23:10.517 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "message": "", 2024-09-11T00:23:10.517 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "return_code": 0 2024-09-11T00:23:10.517 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:} 2024-09-11T00:23:10.517 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:10.516+0000 7fa0e6600640 1 -- 172.21.15.12:0/3901564161 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fa0c00b5270 msgr2=0x7fa0c00b7730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:23:10.518 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:10.516+0000 7fa0e6600640 1 --2- 172.21.15.12:0/3901564161 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fa0c00b5270 0x7fa0c00b7730 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fa0e019bca0 tx=0x7fa0d8030800 comp rx=0 tx=0).stop 2024-09-11T00:23:10.518 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:10.516+0000 7fa0e6600640 1 -- 172.21.15.12:0/3901564161 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa0e0104b90 msgr2=0x7fa0e019af20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:23:10.518 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:10.516+0000 7fa0e6600640 1 --2- 172.21.15.12:0/3901564161 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa0e0104b90 0x7fa0e019af20 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fa0d000e9f0 tx=0x7fa0d000eeb0 comp rx=0 tx=0).stop 2024-09-11T00:23:10.518 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:10.516+0000 7fa0e6600640 1 -- 172.21.15.12:0/3901564161 shutdown_connections 2024-09-11T00:23:10.518 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:10.516+0000 7fa0e6600640 1 --2- 172.21.15.12:0/3901564161 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fa0c00b5270 0x7fa0c00b7730 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:10.518 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:10.516+0000 7fa0e6600640 1 --2- 172.21.15.12:0/3901564161 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa0e01066a0 0x7fa0e019c1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:10.518 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:10.516+0000 7fa0e6600640 1 --2- 172.21.15.12:0/3901564161 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa0e0105ce0 0x7fa0e019b570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:10.518 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:10.516+0000 7fa0e6600640 1 --2- 172.21.15.12:0/3901564161 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa0e0104b90 0x7fa0e019af20 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:10.518 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:10.516+0000 7fa0e6600640 1 -- 172.21.15.12:0/3901564161 >> 172.21.15.12:0/3901564161 conn(0x7fa0e0100470 msgr2=0x7fa0e0109070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:23:10.518 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:10.516+0000 7fa0e6600640 1 -- 172.21.15.12:0/3901564161 shutdown_connections 2024-09-11T00:23:10.518 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:10.516+0000 7fa0e6600640 1 -- 172.21.15.12:0/3901564161 wait complete. 2024-09-11T00:23:11.174 INFO:tasks.workunit.client.0.smithi012.stdout: 10 66323 26.70 MB/sec execute 67 sec latency 242.793 ms 2024-09-11T00:23:12.174 INFO:tasks.workunit.client.0.smithi012.stdout: 10 67170 26.82 MB/sec execute 68 sec latency 149.766 ms 2024-09-11T00:23:12.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:23:11 smithi165 ceph-mon[35356]: pgmap v882: 129 pgs: 129 active+clean; 492 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 7.3 MiB/s wr, 712 op/s 2024-09-11T00:23:12.283 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:23:11 smithi165 ceph-mon[35356]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T00:23:12.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:23:11 smithi012 ceph-mon[30333]: pgmap v882: 129 pgs: 129 active+clean; 492 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 7.3 MiB/s wr, 712 op/s 2024-09-11T00:23:12.314 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:23:11 smithi012 ceph-mon[30333]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T00:23:12.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:23:11 smithi178 ceph-mon[35919]: pgmap v882: 129 pgs: 129 active+clean; 492 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 7.3 MiB/s wr, 712 op/s 2024-09-11T00:23:12.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:23:11 smithi178 ceph-mon[35919]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T00:23:13.174 INFO:tasks.workunit.client.0.smithi012.stdout: 10 68237 27.52 MB/sec execute 69 sec latency 167.980 ms 2024-09-11T00:23:14.174 INFO:tasks.workunit.client.0.smithi012.stdout: 10 68983 27.82 MB/sec execute 70 sec latency 203.886 ms 2024-09-11T00:23:14.273 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:23:14 smithi012 ceph-mon[30333]: pgmap v883: 129 pgs: 129 active+clean; 496 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 MiB/s rd, 4.7 MiB/s wr, 627 op/s 2024-09-11T00:23:14.273 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:23:14 smithi012 ceph-mon[30333]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T00:23:14.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:23:14 smithi165 ceph-mon[35356]: pgmap v883: 129 pgs: 129 active+clean; 496 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 MiB/s rd, 4.7 MiB/s wr, 627 op/s 2024-09-11T00:23:14.283 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:23:14 smithi165 ceph-mon[35356]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T00:23:14.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:23:14 smithi178 ceph-mon[35919]: pgmap v883: 129 pgs: 129 active+clean; 496 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 MiB/s rd, 4.7 MiB/s wr, 627 op/s 2024-09-11T00:23:14.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:23:14 smithi178 ceph-mon[35919]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T00:23:15.174 INFO:tasks.workunit.client.0.smithi012.stdout: 10 68994 27.42 MB/sec execute 71 sec latency 702.394 ms 2024-09-11T00:23:16.174 INFO:tasks.workunit.client.0.smithi012.stdout: 10 68995 27.04 MB/sec execute 72 sec latency 1483.064 ms 2024-09-11T00:23:16.216 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:23:16 smithi012 ceph-mon[30333]: pgmap v884: 129 pgs: 129 active+clean; 499 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 MiB/s rd, 4.0 MiB/s wr, 549 op/s 2024-09-11T00:23:16.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:23:16 smithi165 ceph-mon[35356]: pgmap v884: 129 pgs: 129 active+clean; 499 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 MiB/s rd, 4.0 MiB/s wr, 549 op/s 2024-09-11T00:23:16.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:23:16 smithi178 ceph-mon[35919]: pgmap v884: 129 pgs: 129 active+clean; 499 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 MiB/s rd, 4.0 MiB/s wr, 549 op/s 2024-09-11T00:23:17.174 INFO:tasks.workunit.client.0.smithi012.stdout: 10 68995 26.67 MB/sec execute 73 sec latency 2483.123 ms 2024-09-11T00:23:18.113 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:23:18 smithi012 ceph-mon[30333]: pgmap v885: 129 pgs: 129 active+clean; 565 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 11 MiB/s wr, 870 op/s 2024-09-11T00:23:18.174 INFO:tasks.workunit.client.0.smithi012.stdout: 10 68995 26.31 MB/sec execute 74 sec latency 3483.188 ms 2024-09-11T00:23:18.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:23:18 smithi165 ceph-mon[35356]: pgmap v885: 129 pgs: 129 active+clean; 565 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 11 MiB/s wr, 870 op/s 2024-09-11T00:23:18.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:23:18 smithi178 ceph-mon[35919]: pgmap v885: 129 pgs: 129 active+clean; 565 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 11 MiB/s wr, 870 op/s 2024-09-11T00:23:19.174 INFO:tasks.workunit.client.0.smithi012.stdout: 10 68996 25.96 MB/sec execute 75 sec latency 4470.300 ms 2024-09-11T00:23:20.129 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:23:20 smithi012 ceph-mon[30333]: pgmap v886: 129 pgs: 129 active+clean; 568 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 9.8 MiB/s wr, 690 op/s 2024-09-11T00:23:20.174 INFO:tasks.workunit.client.0.smithi012.stdout: 10 68996 25.62 MB/sec execute 76 sec latency 5470.361 ms 2024-09-11T00:23:20.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:23:20 smithi178 ceph-mon[35919]: pgmap v886: 129 pgs: 129 active+clean; 568 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 9.8 MiB/s wr, 690 op/s 2024-09-11T00:23:20.532 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:23:20 smithi165 ceph-mon[35356]: pgmap v886: 129 pgs: 129 active+clean; 568 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 9.8 MiB/s wr, 690 op/s 2024-09-11T00:23:21.174 INFO:tasks.workunit.client.0.smithi012.stdout: 10 68997 25.29 MB/sec execute 77 sec latency 6470.402 ms 2024-09-11T00:23:22.175 INFO:tasks.workunit.client.0.smithi012.stdout: 10 68997 24.96 MB/sec execute 78 sec latency 7470.451 ms 2024-09-11T00:23:22.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:23:22 smithi178 ceph-mon[35919]: pgmap v887: 129 pgs: 129 active+clean; 584 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 14 MiB/s wr, 817 op/s 2024-09-11T00:23:22.334 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:23:22 smithi012 ceph-mon[30333]: pgmap v887: 129 pgs: 129 active+clean; 584 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 14 MiB/s wr, 817 op/s 2024-09-11T00:23:22.532 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:23:22 smithi165 ceph-mon[35356]: pgmap v887: 129 pgs: 129 active+clean; 584 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 14 MiB/s wr, 817 op/s 2024-09-11T00:23:23.175 INFO:tasks.workunit.client.0.smithi012.stdout: 10 68997 24.65 MB/sec execute 79 sec latency 8470.512 ms 2024-09-11T00:23:23.646 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:23.643+0000 7f9b15e00640 1 -- 172.21.15.12:0/1418709504 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9b10105cf0 msgr2=0x7f9b10106170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:23:23.646 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:23.643+0000 7f9b15e00640 1 --2- 172.21.15.12:0/1418709504 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9b10105cf0 0x7f9b10106170 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f9afc009900 tx=0x7f9afc02a3f0 comp rx=0 tx=0).stop 2024-09-11T00:23:23.646 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:23.644+0000 7f9b15e00640 1 -- 172.21.15.12:0/1418709504 shutdown_connections 2024-09-11T00:23:23.646 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:23.644+0000 7f9b15e00640 1 --2- 172.21.15.12:0/1418709504 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9b101066b0 0x7f9b1010d030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:23.646 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:23.644+0000 7f9b15e00640 1 --2- 172.21.15.12:0/1418709504 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9b10105cf0 0x7f9b10106170 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:23.646 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:23.644+0000 7f9b15e00640 1 --2- 172.21.15.12:0/1418709504 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9b10104ba0 0x7f9b10104fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:23.646 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:23.644+0000 7f9b15e00640 1 -- 172.21.15.12:0/1418709504 >> 172.21.15.12:0/1418709504 conn(0x7f9b10100460 msgr2=0x7f9b10102880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:23:23.647 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:23.644+0000 7f9b15e00640 1 -- 172.21.15.12:0/1418709504 shutdown_connections 2024-09-11T00:23:23.647 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:23.644+0000 7f9b15e00640 1 -- 172.21.15.12:0/1418709504 wait complete. 2024-09-11T00:23:23.648 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:23.645+0000 7f9b15e00640 1 Processor -- start 2024-09-11T00:23:23.648 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:23.645+0000 7f9b15e00640 1 -- start start 2024-09-11T00:23:23.648 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:23.645+0000 7f9b15e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9b10104ba0 0x7f9b1019af90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:23:23.648 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:23.645+0000 7f9b15e00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9b10105cf0 0x7f9b1019b5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:23:23.648 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:23.645+0000 7f9b15e00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9b101066b0 0x7f9b1019c1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:23:23.648 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:23.645+0000 7f9b15e00640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f9b10110810 con 0x7f9b101066b0 2024-09-11T00:23:23.648 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:23.645+0000 7f9b15e00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f9b10110690 con 0x7f9b10104ba0 2024-09-11T00:23:23.648 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:23.645+0000 7f9b15e00640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f9b10110990 con 0x7f9b10105cf0 2024-09-11T00:23:23.648 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:23.645+0000 7f9b0f400640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9b101066b0 0x7f9b1019c1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:23:23.648 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:23.645+0000 7f9b0e000640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9b10105cf0 0x7f9b1019b5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:23:23.649 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:23.645+0000 7f9b0f400640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9b101066b0 0x7f9b1019c1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:45570/0 (socket says 172.21.15.12:45570) 2024-09-11T00:23:23.649 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:23.645+0000 7f9b0f400640 1 -- 172.21.15.12:0/2775840098 learned_addr learned my addr 172.21.15.12:0/2775840098 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:23:23.650 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:23.646+0000 7f9b0f400640 1 -- 172.21.15.12:0/2775840098 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9b10105cf0 msgr2=0x7f9b1019b5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:23:23.650 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:23.646+0000 7f9b0f400640 1 --2- 172.21.15.12:0/2775840098 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9b10105cf0 0x7f9b1019b5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:23.650 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:23.646+0000 7f9b0f400640 1 -- 172.21.15.12:0/2775840098 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9b10104ba0 msgr2=0x7f9b1019af90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:23:23.650 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:23.646+0000 7f9b0f400640 1 --2- 172.21.15.12:0/2775840098 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9b10104ba0 0x7f9b1019af90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:23.650 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:23.646+0000 7f9b0f400640 1 -- 172.21.15.12:0/2775840098 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b1019ca60 con 0x7f9b101066b0 2024-09-11T00:23:23.650 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:23.646+0000 7f9b0f400640 1 --2- 172.21.15.12:0/2775840098 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9b101066b0 0x7f9b1019c1a0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f9b0800eaa0 tx=0x7f9b0800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:23:23.650 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:23.646+0000 7f9b14a00640 1 -- 172.21.15.12:0/2775840098 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b0800cdb0 con 0x7f9b101066b0 2024-09-11T00:23:23.650 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:23.646+0000 7f9b15e00640 1 -- 172.21.15.12:0/2775840098 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9b1019cd50 con 0x7f9b101066b0 2024-09-11T00:23:23.650 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:23.646+0000 7f9b14a00640 1 -- 172.21.15.12:0/2775840098 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9b08005020 con 0x7f9b101066b0 2024-09-11T00:23:23.650 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:23.646+0000 7f9b14a00640 1 -- 172.21.15.12:0/2775840098 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b080116d0 con 0x7f9b101066b0 2024-09-11T00:23:23.650 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:23.646+0000 7f9b15e00640 1 -- 172.21.15.12:0/2775840098 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9b101a3420 con 0x7f9b101066b0 2024-09-11T00:23:23.652 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:23.646+0000 7f9b189fd740 1 Processor -- start 2024-09-11T00:23:23.652 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:23.646+0000 7f9b14a00640 1 -- 172.21.15.12:0/2775840098 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7f9b0800f040 con 0x7f9b101066b0 2024-09-11T00:23:23.653 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:23.646+0000 7f9b189fd740 1 -- start start 2024-09-11T00:23:23.653 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:23.651+0000 7f9b189fd740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557dccdcf790 0x557dccdd1b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:23:23.653 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:23.651+0000 7f9b189fd740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557dccdd20c0 0x557dccdd44b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:23:23.653 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:23.651+0000 7f9b189fd740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557dccdd4b00 0x557dccdd6fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:23:23.653 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:23.651+0000 7f9b189fd740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x557dccd426b0 con 0x557dccdd4b00 2024-09-11T00:23:23.653 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:23.651+0000 7f9b189fd740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x557dccd31ae0 con 0x557dccdd20c0 2024-09-11T00:23:23.653 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:23.651+0000 7f9b189fd740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x557dccd358f0 con 0x557dccdcf790 2024-09-11T00:23:23.655 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:23.651+0000 7f9b0ea00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557dccdd20c0 0x557dccdd44b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:23:23.655 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:23.651+0000 7f9b0ea00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557dccdd20c0 0x557dccdd44b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.12:37888/0 (socket says 172.21.15.12:37888) 2024-09-11T00:23:23.655 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:23.651+0000 7f9b0e000640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557dccdcf790 0x557dccdd1b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:23:23.655 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:23.651+0000 7f9b0ea00640 1 -- 172.21.15.12:0/2689579868 learned_addr learned my addr 172.21.15.12:0/2689579868 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:23:23.655 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:23.651+0000 7f9b0e000640 1 --2- 172.21.15.12:0/2689579868 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557dccdd4b00 0x557dccdd6fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:23:23.655 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:23.651+0000 7f9b0ea00640 1 -- 172.21.15.12:0/2689579868 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557dccdcf790 msgr2=0x557dccdd1b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:23:23.656 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:23.651+0000 7f9b0ea00640 1 --2- 172.21.15.12:0/2689579868 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557dccdcf790 0x557dccdd1b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:23.656 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:23.651+0000 7f9b0ea00640 1 -- 172.21.15.12:0/2689579868 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557dccdd4b00 msgr2=0x557dccdd6fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:23:23.656 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:23.651+0000 7f9b0ea00640 1 --2- 172.21.15.12:0/2689579868 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557dccdd4b00 0x557dccdd6fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:23.656 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:23.651+0000 7f9b0ea00640 1 -- 172.21.15.12:0/2689579868 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557dcccfe780 con 0x557dccdd20c0 2024-09-11T00:23:23.656 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:23.651+0000 7f9b0ea00640 1 --2- 172.21.15.12:0/2689579868 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557dccdd20c0 0x557dccdd44b0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f9b00009c40 tx=0x7f9b0000e720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ed60b19a18d8ea24 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:23:23.656 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:23.651+0000 7f9afbe00640 1 -- 172.21.15.12:0/2689579868 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b00005020 con 0x557dccdd20c0 2024-09-11T00:23:23.656 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:23.651+0000 7f9afbe00640 1 -- 172.21.15.12:0/2689579868 <== mon.2 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9b000047a0 con 0x557dccdd20c0 2024-09-11T00:23:23.656 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:23.651+0000 7f9afbe00640 1 -- 172.21.15.12:0/2689579868 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b00010650 con 0x557dccdd20c0 2024-09-11T00:23:23.656 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:23.651+0000 7f9b14a00640 1 --2- 172.21.15.12:0/2775840098 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f9af40b52e0 0x7f9af40b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:23:23.656 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:23.651+0000 7f9b14a00640 1 -- 172.21.15.12:0/2775840098 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9208+0+0 (secure 0 0 0) 0x7f9b080e5e70 con 0x7f9b101066b0 2024-09-11T00:23:23.656 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:23.651+0000 7f9b0e000640 1 --2- 172.21.15.12:0/2775840098 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f9af40b52e0 0x7f9af40b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:23:23.656 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:23.651+0000 7f9b189fd740 1 -- 172.21.15.12:0/2689579868 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557dccdd20c0 msgr2=0x557dccdd44b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:23:23.657 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:23.651+0000 7f9b189fd740 1 --2- 172.21.15.12:0/2689579868 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557dccdd20c0 0x557dccdd44b0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f9b00009c40 tx=0x7f9b0000e720 comp rx=0 tx=0).stop 2024-09-11T00:23:23.657 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:23.651+0000 7f9b189fd740 1 -- 172.21.15.12:0/2689579868 shutdown_connections 2024-09-11T00:23:23.657 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:23.651+0000 7f9b189fd740 1 --2- 172.21.15.12:0/2689579868 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557dccdd4b00 0x557dccdd6fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:23.657 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:23.651+0000 7f9b189fd740 1 --2- 172.21.15.12:0/2689579868 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557dccdd20c0 0x557dccdd44b0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:23.657 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:23.651+0000 7f9b189fd740 1 --2- 172.21.15.12:0/2689579868 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557dccdcf790 0x557dccdd1b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:23.657 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:23.651+0000 7f9b189fd740 1 -- 172.21.15.12:0/2689579868 >> 172.21.15.12:0/2689579868 conn(0x557dccdca8e0 msgr2=0x557dccdcb670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:23:23.657 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:23.654+0000 7f9b189fd740 1 -- 172.21.15.12:0/2689579868 shutdown_connections 2024-09-11T00:23:23.657 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:23.654+0000 7f9b189fd740 1 -- 172.21.15.12:0/2689579868 wait complete. 2024-09-11T00:23:23.657 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:23.654+0000 7f9b0e000640 1 --2- 172.21.15.12:0/2775840098 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f9af40b52e0 0x7f9af40b77a0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f9afc006010 tx=0x7f9afc031430 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:23:23.657 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:23.654+0000 7f9b189fd740 1 -- start start 2024-09-11T00:23:23.660 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:23.654+0000 7f9b189fd740 1 Processor -- start 2024-09-11T00:23:23.661 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:23.654+0000 7f9b189fd740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557dccdcf790 0x557dccdd8e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:23:23.661 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:23.654+0000 7f9b189fd740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557dccdd20c0 0x557dccdd94d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:23:23.661 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:23.654+0000 7f9b189fd740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557dccdd4b00 0x557dccdda0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:23:23.661 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:23.654+0000 7f9b189fd740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x557dccc69ab0 con 0x557dccdd4b00 2024-09-11T00:23:23.661 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:23.654+0000 7f9b189fd740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x557dcccd8570 con 0x557dccdd20c0 2024-09-11T00:23:23.661 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:23.654+0000 7f9b189fd740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x557dccdd7ae0 con 0x557dccdcf790 2024-09-11T00:23:23.661 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:23.654+0000 7f9b189fd740 10 client.0 _ll_register_callbacks cb 0x557dccce7100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:23:23.661 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:23.654+0000 7f9b0f400640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557dccdd4b00 0x557dccdda0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:23:23.661 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:23.654+0000 7f9b0f400640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557dccdd4b00 0x557dccdda0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:45600/0 (socket says 172.21.15.12:45600) 2024-09-11T00:23:23.661 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:23.654+0000 7f9b0f400640 1 -- 172.21.15.12:0/176605176 learned_addr learned my addr 172.21.15.12:0/176605176 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:23:23.661 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:23.654+0000 7f9b0f400640 1 -- 172.21.15.12:0/176605176 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557dccdcf790 msgr2=0x557dccdd8e30 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-11T00:23:23.662 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:23.654+0000 7f9b0f400640 1 --2- 172.21.15.12:0/176605176 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557dccdcf790 0x557dccdd8e30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:23.662 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:23.654+0000 7f9b0f400640 1 -- 172.21.15.12:0/176605176 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557dccdd20c0 msgr2=0x557dccdd94d0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-11T00:23:23.662 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:23.654+0000 7f9b0f400640 1 --2- 172.21.15.12:0/176605176 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557dccdd20c0 0x557dccdd94d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:23.662 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:23.654+0000 7f9b0f400640 1 -- 172.21.15.12:0/176605176 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557dcca93790 con 0x557dccdd4b00 2024-09-11T00:23:23.662 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:23.654+0000 7f9b0f400640 1 --2- 172.21.15.12:0/176605176 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557dccdd4b00 0x557dccdda0a0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f9b08009270 tx=0x7f9b080abaa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4bc0aa25fcc3f549 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:23:23.662 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:23.654+0000 7f9af8c00640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-09-11T00:23:23.662 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:23.654+0000 7f9af8c00640 1 -- 172.21.15.12:0/176605176 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b0801f030 con 0x557dccdd4b00 2024-09-11T00:23:23.662 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:23.654+0000 7f9af8c00640 1 -- 172.21.15.12:0/176605176 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9208+0+0 (secure 0 0 0) 0x7f9b080054a0 con 0x557dccdd4b00 2024-09-11T00:23:23.662 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:23.654+0000 7f9b189fd740 1 -- 172.21.15.12:0/176605176 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9b000047a0 con 0x557dccdd4b00 2024-09-11T00:23:23.662 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:23.654+0000 7f9af8c00640 1 -- 172.21.15.12:0/176605176 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=63) ==== 24+0+0 (secure 0 0 0) 0x7f9b080ac200 con 0x557dccdd4b00 2024-09-11T00:23:23.662 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:23.654+0000 7f9b189fd740 10 client.17280 fetch_fsmap learned FSMap version 63 2024-09-11T00:23:23.662 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:23.654+0000 7f9b189fd740 1 -- 172.21.15.12:0/176605176 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=63,monmap=4+}) -- 0x557dccb0c620 con 0x557dccdd4b00 2024-09-11T00:23:23.663 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:23.654+0000 7f9af8c00640 1 -- 172.21.15.12:0/176605176 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9b000047a0 con 0x557dccdd4b00 2024-09-11T00:23:23.663 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:23.654+0000 7f9af8c00640 1 -- 172.21.15.12:0/176605176 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 63) ==== 8378+0+0 (secure 0 0 0) 0x7f9b080cdbe0 con 0x557dccdd4b00 2024-09-11T00:23:23.663 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:23.654+0000 7f9b189fd740 10 client.17280 fetch_fsmap finished waiting for FSMap version 63 2024-09-11T00:23:23.663 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:23.654+0000 7f9b189fd740 10 client.17280 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T00:23:23.663 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:23.654+0000 7f9b189fd740 20 client.17280 populate_metadata read hostname 'smithi012' 2024-09-11T00:23:23.663 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:23.654+0000 7f9b189fd740 1 --2- 172.21.15.12:0/176605176 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x557dccde4750 0x557dccde6bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:23:23.663 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:23.654+0000 7f9b189fd740 4 client.17280 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T00:23:23.663 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:23.654+0000 7f9b189fd740 1 -- 172.21.15.12:0/176605176 --> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557dccd44600 con 0x557dccde4750 2024-09-11T00:23:23.663 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:23.654+0000 7f9b0ea00640 1 --2- 172.21.15.12:0/176605176 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x557dccde4750 0x557dccde6bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:23:23.663 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:23.654+0000 7f9b0ea00640 1 --2- 172.21.15.12:0/176605176 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x557dccde4750 0x557dccde6bc0 crc :-1 s=READY pgs=233 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eaaf602f94eddf18 server_cookie=1e68cca56b04392f in_seq=0 out_seq=0 2024-09-11T00:23:23.663 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:23.654+0000 7f9af8c00640 10 client.17280 ms_handle_connect on v2:172.21.15.165:6838/501312984 2024-09-11T00:23:23.663 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:23.654+0000 7f9af8c00640 1 -- 172.21.15.12:0/176605176 <== mds.0 v2:172.21.15.165:6838/501312984 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x557dccd44600 con 0x557dccde4750 2024-09-11T00:23:23.663 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:23.654+0000 7f9af8c00640 10 client.17280 handle_command_reply: tid=0 2024-09-11T00:23:23.664 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:23.654+0000 7f9b189fd740 1 client.17280 shutdown 2024-09-11T00:23:23.664 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:23.659+0000 7f9b189fd740 1 -- 172.21.15.12:0/176605176 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557dccdd4b00 msgr2=0x557dccdda0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:23:23.664 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:23.659+0000 7f9b189fd740 1 --2- 172.21.15.12:0/176605176 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557dccdd4b00 0x557dccdda0a0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f9b08009270 tx=0x7f9b080abaa0 comp rx=0 tx=0).stop 2024-09-11T00:23:23.664 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:23.659+0000 7f9b189fd740 1 -- 172.21.15.12:0/176605176 shutdown_connections 2024-09-11T00:23:23.664 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:23.659+0000 7f9b189fd740 1 --2- 172.21.15.12:0/176605176 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x557dccde4750 0x557dccde6bc0 crc :-1 s=READY pgs=233 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:23.664 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:23.659+0000 7f9b189fd740 1 --2- 172.21.15.12:0/176605176 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557dccdd4b00 0x557dccdda0a0 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:23.664 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:23.659+0000 7f9b189fd740 1 --2- 172.21.15.12:0/176605176 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557dccdd20c0 0x557dccdd94d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:23.664 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:23.659+0000 7f9b189fd740 1 --2- 172.21.15.12:0/176605176 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557dccdcf790 0x557dccdd8e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:23.664 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:23.659+0000 7f9b189fd740 1 -- 172.21.15.12:0/176605176 >> 172.21.15.12:0/176605176 conn(0x557dccdca8e0 msgr2=0x557dccdc9a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:23:23.664 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:23.659+0000 7f9af8c00640 0 client.17280 ms_handle_reset on v2:172.21.15.165:6838/501312984 2024-09-11T00:23:23.664 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:23.659+0000 7f9b189fd740 1 -- 172.21.15.12:0/176605176 shutdown_connections 2024-09-11T00:23:23.664 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:23.659+0000 7f9b189fd740 1 -- 172.21.15.12:0/176605176 wait complete. 2024-09-11T00:23:23.665 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:23.659+0000 7f9b189fd740 20 client.17280 trim_cache size 0 max 16384 2024-09-11T00:23:23.676 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:23.674+0000 7f9b189fd740 1 Processor -- start 2024-09-11T00:23:23.676 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:23.674+0000 7f9b189fd740 1 -- start start 2024-09-11T00:23:23.677 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:23.674+0000 7f9b189fd740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557dccdcf790 0x557dccdd4ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:23:23.677 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:23.674+0000 7f9b189fd740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557dccdd20c0 0x557dccdd5420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:23:23.677 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:23.674+0000 7f9b189fd740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557dccdd4b00 0x557dccde64c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:23:23.677 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:23.674+0000 7f9b189fd740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f9b101ada30 con 0x557dccdd4b00 2024-09-11T00:23:23.677 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:23.674+0000 7f9b189fd740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f9b101af5d0 con 0x557dccdd20c0 2024-09-11T00:23:23.677 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:23.674+0000 7f9b189fd740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f9b101ae130 con 0x557dccdcf790 2024-09-11T00:23:23.677 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:23.674+0000 7f9b0ea00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557dccdcf790 0x557dccdd4ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:23:23.677 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:23.674+0000 7f9b0f400640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557dccdd4b00 0x557dccde64c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:23:23.677 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:23.674+0000 7f9b0ea00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557dccdcf790 0x557dccdd4ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.12:44066/0 (socket says 172.21.15.12:44066) 2024-09-11T00:23:23.677 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:23.674+0000 7f9b0ea00640 1 -- 172.21.15.12:0/3574863328 learned_addr learned my addr 172.21.15.12:0/3574863328 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:23:23.679 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:23.674+0000 7f9b0ea00640 1 -- 172.21.15.12:0/3574863328 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557dccdd20c0 msgr2=0x557dccdd5420 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-11T00:23:23.679 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:23.674+0000 7f9b0ea00640 1 --2- 172.21.15.12:0/3574863328 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557dccdd20c0 0x557dccdd5420 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:23.679 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:23.674+0000 7f9b0ea00640 1 -- 172.21.15.12:0/3574863328 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557dccdd4b00 msgr2=0x557dccde64c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:23:23.679 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:23.674+0000 7f9b0ea00640 1 --2- 172.21.15.12:0/3574863328 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557dccdd4b00 0x557dccde64c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:23.679 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:23.674+0000 7f9b0ea00640 1 -- 172.21.15.12:0/3574863328 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557dccdd9840 con 0x557dccdcf790 2024-09-11T00:23:23.679 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:23.674+0000 7f9b0ea00640 1 --2- 172.21.15.12:0/3574863328 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557dccdcf790 0x557dccdd4ee0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f9b00006010 tx=0x7f9b00000bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=319139cca48f10c1 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:23:23.679 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:23.674+0000 7f9afb400640 1 -- 172.21.15.12:0/3574863328 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b0001d040 con 0x557dccdcf790 2024-09-11T00:23:23.679 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:23.674+0000 7f9afb400640 1 -- 172.21.15.12:0/3574863328 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9b00009d30 con 0x557dccdcf790 2024-09-11T00:23:23.679 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:23.674+0000 7f9afb400640 1 -- 172.21.15.12:0/3574863328 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b00004a20 con 0x557dccdcf790 2024-09-11T00:23:23.679 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:23.674+0000 7f9b189fd740 1 -- 172.21.15.12:0/3574863328 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557dccdcf790 msgr2=0x557dccdd4ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:23:23.679 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:23.674+0000 7f9b189fd740 1 --2- 172.21.15.12:0/3574863328 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557dccdcf790 0x557dccdd4ee0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f9b00006010 tx=0x7f9b00000bc0 comp rx=0 tx=0).stop 2024-09-11T00:23:23.680 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:23.674+0000 7f9b189fd740 1 -- 172.21.15.12:0/3574863328 shutdown_connections 2024-09-11T00:23:23.680 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:23.674+0000 7f9b189fd740 1 --2- 172.21.15.12:0/3574863328 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557dccdd4b00 0x557dccde64c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:23.680 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:23.674+0000 7f9b189fd740 1 --2- 172.21.15.12:0/3574863328 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557dccdd20c0 0x557dccdd5420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:23.680 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:23.674+0000 7f9b189fd740 1 --2- 172.21.15.12:0/3574863328 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557dccdcf790 0x557dccdd4ee0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:23.680 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:23.674+0000 7f9b189fd740 1 -- 172.21.15.12:0/3574863328 >> 172.21.15.12:0/3574863328 conn(0x557dccdca8e0 msgr2=0x557dccdceb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:23:23.680 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:23.674+0000 7f9b189fd740 1 -- 172.21.15.12:0/3574863328 shutdown_connections 2024-09-11T00:23:23.680 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:23.674+0000 7f9b189fd740 1 -- 172.21.15.12:0/3574863328 wait complete. 2024-09-11T00:23:23.680 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:23.674+0000 7f9b189fd740 1 -- start start 2024-09-11T00:23:23.680 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:23.674+0000 7f9b189fd740 1 Processor -- start 2024-09-11T00:23:23.681 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:23.674+0000 7f9b189fd740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557dccdcf790 0x557dccde0490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:23:23.681 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:23.674+0000 7f9b189fd740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557dccdd20c0 0x557dccde0b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:23:23.681 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:23.674+0000 7f9b189fd740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557dccdd4b00 0x557dccddf230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:23:23.681 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:23.674+0000 7f9b189fd740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f9b101aef00 con 0x557dccdd4b00 2024-09-11T00:23:23.681 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:23.674+0000 7f9b189fd740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f9b101ae800 con 0x557dccdcf790 2024-09-11T00:23:23.681 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:23.674+0000 7f9b189fd740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f9b101aba10 con 0x557dccdd20c0 2024-09-11T00:23:23.681 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:23.674+0000 7f9b189fd740 10 client.0 _ll_register_callbacks cb 0x7f9b0800cf50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:23:23.682 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:23.674+0000 7f9b0ea00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557dccdcf790 0x557dccde0490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:23:23.682 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:23.674+0000 7f9b0f400640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557dccdd4b00 0x557dccddf230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:23:23.682 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:23.674+0000 7f9b0ea00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557dccdcf790 0x557dccde0490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.12:37916/0 (socket says 172.21.15.12:37916) 2024-09-11T00:23:23.682 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:23.674+0000 7f9b0ea00640 1 -- 172.21.15.12:0/3103092843 learned_addr learned my addr 172.21.15.12:0/3103092843 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:23:23.682 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:23.679+0000 7f9b0ea00640 1 -- 172.21.15.12:0/3103092843 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557dccdd20c0 msgr2=0x557dccde0b30 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-11T00:23:23.682 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:23.679+0000 7f9b0ea00640 1 --2- 172.21.15.12:0/3103092843 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557dccdd20c0 0x557dccde0b30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:23.682 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:23.679+0000 7f9b0ea00640 1 -- 172.21.15.12:0/3103092843 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557dccdd4b00 msgr2=0x557dccddf230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:23:23.682 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:23.679+0000 7f9b0ea00640 1 --2- 172.21.15.12:0/3103092843 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557dccdd4b00 0x557dccddf230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:23.682 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:23.679+0000 7f9b0ea00640 1 -- 172.21.15.12:0/3103092843 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557dccdd1ef0 con 0x557dccdcf790 2024-09-11T00:23:23.682 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:23.679+0000 7f9b0f400640 1 --2- 172.21.15.12:0/3103092843 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557dccdd4b00 0x557dccddf230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:23:23.699 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:23.698+0000 7f9b0ea00640 1 --2- 172.21.15.12:0/3103092843 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557dccdcf790 0x557dccde0490 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f9b00009460 tx=0x7f9b00005b20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=689957e83450d499 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:23:23.699 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:23.698+0000 7f9af8c00640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-09-11T00:23:23.700 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:23.698+0000 7f9af8c00640 1 -- 172.21.15.12:0/3103092843 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b00037020 con 0x557dccdcf790 2024-09-11T00:23:23.700 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:23.698+0000 7f9b189fd740 1 -- 172.21.15.12:0/3103092843 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9b00009d30 con 0x557dccdcf790 2024-09-11T00:23:23.701 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:23.698+0000 7f9af8c00640 1 -- 172.21.15.12:0/3103092843 <== mon.2 v2:172.21.15.165:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9208+0+0 (secure 0 0 0) 0x7f9b000047a0 con 0x557dccdcf790 2024-09-11T00:23:23.701 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:23.698+0000 7f9af8c00640 1 -- 172.21.15.12:0/3103092843 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=63) ==== 24+0+0 (secure 0 0 0) 0x7f9b00042050 con 0x557dccdcf790 2024-09-11T00:23:23.701 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:23.698+0000 7f9b189fd740 10 client.27293 fetch_fsmap learned FSMap version 63 2024-09-11T00:23:23.701 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:23.698+0000 7f9b189fd740 1 -- 172.21.15.12:0/3103092843 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=63,monmap=4+}) -- 0x557dccdde700 con 0x557dccdcf790 2024-09-11T00:23:23.701 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:23.698+0000 7f9af8c00640 1 -- 172.21.15.12:0/3103092843 <== mon.2 v2:172.21.15.165:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9b00009d30 con 0x557dccdcf790 2024-09-11T00:23:23.702 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:23.698+0000 7f9af8c00640 1 -- 172.21.15.12:0/3103092843 <== mon.2 v2:172.21.15.165:3300/0 5 ==== fsmap(e 63) ==== 8378+0+0 (secure 0 0 0) 0x7f9b00010430 con 0x557dccdcf790 2024-09-11T00:23:23.702 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:23.698+0000 7f9b189fd740 10 client.27293 fetch_fsmap finished waiting for FSMap version 63 2024-09-11T00:23:23.702 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:23.698+0000 7f9b189fd740 10 client.27293 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T00:23:23.702 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:23.698+0000 7f9b189fd740 20 client.27293 populate_metadata read hostname 'smithi012' 2024-09-11T00:23:23.702 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:23.698+0000 7f9b189fd740 1 --2- 172.21.15.12:0/3103092843 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x557dcce519b0 0x557dcce53dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:23:23.702 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:23.698+0000 7f9b189fd740 4 client.27293 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T00:23:23.702 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:23.698+0000 7f9b189fd740 1 -- 172.21.15.12:0/3103092843 --> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] -- command(tid 0: {"prefix": "flush journal"}) -- 0x557dccd400b0 con 0x557dcce519b0 2024-09-11T00:23:23.734 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:23.733+0000 7f9b0e000640 1 --2- 172.21.15.12:0/3103092843 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x557dcce519b0 0x557dcce53dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:23:23.852 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:23.851+0000 7f9b0e000640 1 --2- 172.21.15.12:0/3103092843 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x557dcce519b0 0x557dcce53dd0 crc :-1 s=READY pgs=234 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f406f39898ad4007 server_cookie=51f93d14bc4ec1c1 in_seq=0 out_seq=0 2024-09-11T00:23:23.852 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:23.851+0000 7f9af8c00640 10 client.27293 ms_handle_connect on v2:172.21.15.165:6838/501312984 2024-09-11T00:23:24.018 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:24.016+0000 7f9af8c00640 1 -- 172.21.15.12:0/3103092843 <== mds.0 v2:172.21.15.165:6838/501312984 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x557dccd400b0 con 0x557dcce519b0 2024-09-11T00:23:24.018 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:24.016+0000 7f9af8c00640 10 client.27293 handle_command_reply: tid=0 2024-09-11T00:23:24.018 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:24.016+0000 7f9b189fd740 1 client.27293 shutdown 2024-09-11T00:23:24.018 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:24.017+0000 7f9b189fd740 1 -- 172.21.15.12:0/3103092843 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557dccdcf790 msgr2=0x557dccde0490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:23:24.018 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:24.017+0000 7f9b189fd740 1 --2- 172.21.15.12:0/3103092843 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557dccdcf790 0x557dccde0490 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f9b00009460 tx=0x7f9b00005b20 comp rx=0 tx=0).stop 2024-09-11T00:23:24.019 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:24.017+0000 7f9b189fd740 1 -- 172.21.15.12:0/3103092843 shutdown_connections 2024-09-11T00:23:24.019 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:24.017+0000 7f9b189fd740 1 --2- 172.21.15.12:0/3103092843 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x557dcce519b0 0x557dcce53dd0 crc :-1 s=READY pgs=234 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:24.019 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:24.017+0000 7f9b189fd740 1 --2- 172.21.15.12:0/3103092843 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557dccdd4b00 0x557dccddf230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:24.019 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:24.017+0000 7f9b189fd740 1 --2- 172.21.15.12:0/3103092843 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557dccdd20c0 0x557dccde0b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:24.019 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:24.017+0000 7f9b189fd740 1 --2- 172.21.15.12:0/3103092843 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557dccdcf790 0x557dccde0490 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:24.019 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:24.017+0000 7f9b189fd740 1 -- 172.21.15.12:0/3103092843 >> 172.21.15.12:0/3103092843 conn(0x557dccdca8e0 msgr2=0x557dccdd4ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:23:24.019 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:24.017+0000 7f9af8c00640 0 client.27293 ms_handle_reset on v2:172.21.15.165:6838/501312984 2024-09-11T00:23:24.020 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:24.017+0000 7f9b189fd740 1 -- 172.21.15.12:0/3103092843 shutdown_connections 2024-09-11T00:23:24.020 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:24.017+0000 7f9b189fd740 1 -- 172.21.15.12:0/3103092843 wait complete. 2024-09-11T00:23:24.020 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:24.017+0000 7f9b189fd740 20 client.27293 trim_cache size 0 max 16384 2024-09-11T00:23:24.020 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:{ 2024-09-11T00:23:24.020 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "message": "", 2024-09-11T00:23:24.020 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "return_code": 0 2024-09-11T00:23:24.020 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:} 2024-09-11T00:23:24.021 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:24.017+0000 7f9b15e00640 1 -- 172.21.15.12:0/2775840098 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f9af40b52e0 msgr2=0x7f9af40b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:23:24.021 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:24.017+0000 7f9b15e00640 1 --2- 172.21.15.12:0/2775840098 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f9af40b52e0 0x7f9af40b77a0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f9afc006010 tx=0x7f9afc031430 comp rx=0 tx=0).stop 2024-09-11T00:23:24.021 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:24.017+0000 7f9b15e00640 1 -- 172.21.15.12:0/2775840098 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9b101066b0 msgr2=0x7f9b1019c1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:23:24.021 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:24.017+0000 7f9b15e00640 1 --2- 172.21.15.12:0/2775840098 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9b101066b0 0x7f9b1019c1a0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f9b0800eaa0 tx=0x7f9b0800ef60 comp rx=0 tx=0).stop 2024-09-11T00:23:24.021 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:24.017+0000 7f9b15e00640 1 -- 172.21.15.12:0/2775840098 shutdown_connections 2024-09-11T00:23:24.021 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:24.017+0000 7f9b15e00640 1 --2- 172.21.15.12:0/2775840098 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f9af40b52e0 0x7f9af40b77a0 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:24.021 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:24.017+0000 7f9b15e00640 1 --2- 172.21.15.12:0/2775840098 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9b101066b0 0x7f9b1019c1a0 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:24.021 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:24.017+0000 7f9b15e00640 1 --2- 172.21.15.12:0/2775840098 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9b10105cf0 0x7f9b1019b5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:24.021 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:24.017+0000 7f9b15e00640 1 --2- 172.21.15.12:0/2775840098 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9b10104ba0 0x7f9b1019af90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:24.022 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:24.017+0000 7f9b15e00640 1 -- 172.21.15.12:0/2775840098 >> 172.21.15.12:0/2775840098 conn(0x7f9b10100460 msgr2=0x7f9b10109080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:23:24.024 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:24.017+0000 7f9b15e00640 1 -- 172.21.15.12:0/2775840098 shutdown_connections 2024-09-11T00:23:24.024 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:24.017+0000 7f9b15e00640 1 -- 172.21.15.12:0/2775840098 wait complete. 2024-09-11T00:23:24.175 INFO:tasks.workunit.client.0.smithi012.stdout: 10 68997 24.34 MB/sec execute 80 sec latency 9470.562 ms 2024-09-11T00:23:24.300 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:23:24 smithi012 ceph-mon[30333]: pgmap v888: 129 pgs: 129 active+clean; 579 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 14 MiB/s wr, 715 op/s 2024-09-11T00:23:24.532 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:23:24 smithi165 ceph-mon[35356]: pgmap v888: 129 pgs: 129 active+clean; 579 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 14 MiB/s wr, 715 op/s 2024-09-11T00:23:24.581 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:23:24 smithi178 ceph-mon[35919]: pgmap v888: 129 pgs: 129 active+clean; 579 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 14 MiB/s wr, 715 op/s 2024-09-11T00:23:25.175 INFO:tasks.workunit.client.0.smithi012.stdout: 10 68997 24.04 MB/sec execute 81 sec latency 10470.612 ms 2024-09-11T00:23:26.175 INFO:tasks.workunit.client.0.smithi012.stdout: 10 68997 23.75 MB/sec execute 82 sec latency 11470.667 ms 2024-09-11T00:23:26.302 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:23:26 smithi012 ceph-mon[30333]: pgmap v889: 129 pgs: 129 active+clean; 576 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 13 MiB/s wr, 577 op/s 2024-09-11T00:23:26.532 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:23:26 smithi165 ceph-mon[35356]: pgmap v889: 129 pgs: 129 active+clean; 576 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 13 MiB/s wr, 577 op/s 2024-09-11T00:23:26.581 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:23:26 smithi178 ceph-mon[35919]: pgmap v889: 129 pgs: 129 active+clean; 576 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 13 MiB/s wr, 577 op/s 2024-09-11T00:23:27.175 INFO:tasks.workunit.client.0.smithi012.stdout: 10 68997 23.46 MB/sec execute 83 sec latency 12470.716 ms 2024-09-11T00:23:28.175 INFO:tasks.workunit.client.0.smithi012.stdout: 10 68997 23.18 MB/sec execute 84 sec latency 13470.772 ms 2024-09-11T00:23:28.305 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:23:28 smithi012 ceph-mon[30333]: pgmap v890: 129 pgs: 129 active+clean; 537 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 14 MiB/s wr, 686 op/s 2024-09-11T00:23:28.532 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:23:28 smithi165 ceph-mon[35356]: pgmap v890: 129 pgs: 129 active+clean; 537 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 14 MiB/s wr, 686 op/s 2024-09-11T00:23:28.581 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:23:28 smithi178 ceph-mon[35919]: pgmap v890: 129 pgs: 129 active+clean; 537 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 14 MiB/s wr, 686 op/s 2024-09-11T00:23:29.175 INFO:tasks.workunit.client.0.smithi012.stdout: 10 68997 22.91 MB/sec execute 85 sec latency 14470.819 ms 2024-09-11T00:23:30.177 INFO:tasks.workunit.client.0.smithi012.stdout: 10 68997 22.64 MB/sec execute 86 sec latency 15470.862 ms 2024-09-11T00:23:30.296 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:23:30 smithi012 ceph-mon[30333]: pgmap v891: 129 pgs: 129 active+clean; 535 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 488 KiB/s rd, 6.8 MiB/s wr, 339 op/s 2024-09-11T00:23:30.532 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:23:30 smithi165 ceph-mon[35356]: pgmap v891: 129 pgs: 129 active+clean; 535 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 488 KiB/s rd, 6.8 MiB/s wr, 339 op/s 2024-09-11T00:23:30.581 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:23:30 smithi178 ceph-mon[35919]: pgmap v891: 129 pgs: 129 active+clean; 535 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 488 KiB/s rd, 6.8 MiB/s wr, 339 op/s 2024-09-11T00:23:31.175 INFO:tasks.workunit.client.0.smithi012.stdout: 10 68997 22.38 MB/sec execute 87 sec latency 16470.906 ms 2024-09-11T00:23:31.414 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:23:31 smithi012 ceph-mon[30333]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T00:23:31.532 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:23:31 smithi165 ceph-mon[35356]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T00:23:31.581 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:23:31 smithi178 ceph-mon[35919]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T00:23:32.071 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 34.8 2024-09-11T00:23:32.071 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid ff3ad74a-6fcf-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T00:23:32.175 INFO:tasks.workunit.client.0.smithi012.stdout: 10 68997 22.13 MB/sec execute 88 sec latency 17470.953 ms 2024-09-11T00:23:32.251 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:23:32 smithi012 ceph-mon[30333]: pgmap v892: 129 pgs: 129 active+clean; 526 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 427 KiB/s rd, 6.2 MiB/s wr, 342 op/s 2024-09-11T00:23:32.518 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:23:32.532 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:23:32 smithi165 ceph-mon[35356]: pgmap v892: 129 pgs: 129 active+clean; 526 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 427 KiB/s rd, 6.2 MiB/s wr, 342 op/s 2024-09-11T00:23:32.581 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:23:32 smithi178 ceph-mon[35919]: pgmap v892: 129 pgs: 129 active+clean; 526 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 427 KiB/s rd, 6.2 MiB/s wr, 342 op/s 2024-09-11T00:23:33.175 INFO:tasks.workunit.client.0.smithi012.stdout: 10 69015 21.88 MB/sec execute 89 sec latency 18471.007 ms 2024-09-11T00:23:33.532 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:23:33 smithi165 ceph-mon[35356]: pgmap v893: 129 pgs: 129 active+clean; 524 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 449 KiB/s rd, 2.0 MiB/s wr, 226 op/s 2024-09-11T00:23:33.563 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:23:33 smithi012 ceph-mon[30333]: pgmap v893: 129 pgs: 129 active+clean; 524 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 449 KiB/s rd, 2.0 MiB/s wr, 226 op/s 2024-09-11T00:23:33.581 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:23:33 smithi178 ceph-mon[35919]: pgmap v893: 129 pgs: 129 active+clean; 524 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 449 KiB/s rd, 2.0 MiB/s wr, 226 op/s 2024-09-11T00:23:34.175 INFO:tasks.workunit.client.0.smithi012.stdout: 10 69051 21.64 MB/sec execute 90 sec latency 19471.073 ms 2024-09-11T00:23:35.175 INFO:tasks.workunit.client.0.smithi012.stdout: 10 69051 21.40 MB/sec execute 91 sec latency 20471.130 ms 2024-09-11T00:23:35.813 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:23:35 smithi012 ceph-mon[30333]: pgmap v894: 129 pgs: 129 active+clean; 523 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 212 KiB/s rd, 1006 KiB/s wr, 180 op/s 2024-09-11T00:23:35.831 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:23:35 smithi178 ceph-mon[35919]: pgmap v894: 129 pgs: 129 active+clean; 523 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 212 KiB/s rd, 1006 KiB/s wr, 180 op/s 2024-09-11T00:23:36.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:23:35 smithi165 ceph-mon[35356]: pgmap v894: 129 pgs: 129 active+clean; 523 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 212 KiB/s rd, 1006 KiB/s wr, 180 op/s 2024-09-11T00:23:36.175 INFO:tasks.workunit.client.0.smithi012.stdout: 10 69051 21.17 MB/sec execute 92 sec latency 21471.189 ms 2024-09-11T00:23:37.137 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:37.134+0000 7f5653e00640 1 -- 172.21.15.12:0/41845449 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f564c075ac0 msgr2=0x7f564c075f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:23:37.137 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:37.134+0000 7f5653e00640 1 --2- 172.21.15.12:0/41845449 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f564c075ac0 0x7f564c075f40 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f5638009900 tx=0x7f563802a410 comp rx=0 tx=0).stop 2024-09-11T00:23:37.137 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:37.134+0000 7f5653e00640 1 -- 172.21.15.12:0/41845449 shutdown_connections 2024-09-11T00:23:37.137 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:37.134+0000 7f5653e00640 1 --2- 172.21.15.12:0/41845449 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f564c10a910 0x7f564c10cd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:37.138 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:37.134+0000 7f5653e00640 1 --2- 172.21.15.12:0/41845449 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f564c075ac0 0x7f564c075f40 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:37.138 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:37.134+0000 7f5653e00640 1 --2- 172.21.15.12:0/41845449 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f564c076f30 0x7f564c075580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:37.138 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:37.134+0000 7f5653e00640 1 -- 172.21.15.12:0/41845449 >> 172.21.15.12:0/41845449 conn(0x7f564c0fe250 msgr2=0x7f564c100670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:23:37.138 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:37.134+0000 7f5653e00640 1 -- 172.21.15.12:0/41845449 shutdown_connections 2024-09-11T00:23:37.138 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:37.134+0000 7f5653e00640 1 -- 172.21.15.12:0/41845449 wait complete. 2024-09-11T00:23:37.140 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:37.134+0000 7f5653e00640 1 Processor -- start 2024-09-11T00:23:37.140 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:37.134+0000 7f5653e00640 1 -- start start 2024-09-11T00:23:37.140 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:37.134+0000 7f5653e00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f564c075ac0 0x7f564c19afb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:23:37.140 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:37.134+0000 7f5653e00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f564c076f30 0x7f564c19b600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:23:37.140 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:37.134+0000 7f5653e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f564c10a910 0x7f564c19c230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:23:37.140 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:37.134+0000 7f5653e00640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f564c110850 con 0x7f564c075ac0 2024-09-11T00:23:37.140 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:37.134+0000 7f5653e00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f564c1106d0 con 0x7f564c10a910 2024-09-11T00:23:37.140 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:37.134+0000 7f5653e00640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f564c1109d0 con 0x7f564c076f30 2024-09-11T00:23:37.140 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:37.134+0000 7f5651400640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f564c075ac0 0x7f564c19afb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:23:37.140 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:37.134+0000 7f5651400640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f564c075ac0 0x7f564c19afb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:57722/0 (socket says 172.21.15.12:57722) 2024-09-11T00:23:37.141 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:37.134+0000 7f5651400640 1 -- 172.21.15.12:0/849472218 learned_addr learned my addr 172.21.15.12:0/849472218 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:23:37.141 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:37.134+0000 7f5651e00640 1 --2- 172.21.15.12:0/849472218 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f564c10a910 0x7f564c19c230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:23:37.141 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:37.134+0000 7f5650a00640 1 --2- 172.21.15.12:0/849472218 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f564c076f30 0x7f564c19b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:23:37.141 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:37.134+0000 7f5651400640 1 -- 172.21.15.12:0/849472218 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f564c076f30 msgr2=0x7f564c19b600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:23:37.141 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:37.134+0000 7f5651400640 1 --2- 172.21.15.12:0/849472218 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f564c076f30 0x7f564c19b600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:37.141 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:37.134+0000 7f5651400640 1 -- 172.21.15.12:0/849472218 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f564c10a910 msgr2=0x7f564c19c230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:23:37.141 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:37.134+0000 7f5651400640 1 --2- 172.21.15.12:0/849472218 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f564c10a910 0x7f564c19c230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:37.141 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:37.134+0000 7f5651400640 1 -- 172.21.15.12:0/849472218 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f564c19caf0 con 0x7f564c075ac0 2024-09-11T00:23:37.141 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:37.134+0000 7f5651400640 1 --2- 172.21.15.12:0/849472218 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f564c075ac0 0x7f564c19afb0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f564000e9f0 tx=0x7f564000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:23:37.141 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:37.134+0000 7f564a000640 1 -- 172.21.15.12:0/849472218 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f564000cd80 con 0x7f564c075ac0 2024-09-11T00:23:37.141 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:37.134+0000 7f564a000640 1 -- 172.21.15.12:0/849472218 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f56400054d0 con 0x7f564c075ac0 2024-09-11T00:23:37.141 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:37.134+0000 7f564a000640 1 -- 172.21.15.12:0/849472218 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5640010690 con 0x7f564c075ac0 2024-09-11T00:23:37.141 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:37.134+0000 7f5653e00640 1 -- 172.21.15.12:0/849472218 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f564c19cde0 con 0x7f564c075ac0 2024-09-11T00:23:37.142 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:37.134+0000 7f5653e00640 1 -- 172.21.15.12:0/849472218 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f564c1a34b0 con 0x7f564c075ac0 2024-09-11T00:23:37.143 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:37.140+0000 7f5656c2f740 1 Processor -- start 2024-09-11T00:23:37.143 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:37.140+0000 7f5656c2f740 1 -- start start 2024-09-11T00:23:37.143 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:37.140+0000 7f5656c2f740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558cfe9f3790 0x558cfe9f5b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:23:37.144 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:37.140+0000 7f5656c2f740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558cfe9f60c0 0x558cfe9f84b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:23:37.144 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:37.140+0000 7f5656c2f740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558cfe9f8b00 0x558cfe9fafc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:23:37.144 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:37.140+0000 7f5656c2f740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x558cfe9666b0 con 0x558cfe9f3790 2024-09-11T00:23:37.144 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:37.140+0000 7f5656c2f740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x558cfe955ae0 con 0x558cfe9f60c0 2024-09-11T00:23:37.144 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:37.140+0000 7f5656c2f740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x558cfe9598f0 con 0x558cfe9f8b00 2024-09-11T00:23:37.144 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:37.140+0000 7f564a000640 1 -- 172.21.15.12:0/849472218 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7f5640004e90 con 0x7f564c075ac0 2024-09-11T00:23:37.144 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:37.140+0000 7f5651e00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558cfe9f3790 0x558cfe9f5b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:23:37.144 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:37.140+0000 7f5650a00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558cfe9f8b00 0x558cfe9fafc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:23:37.144 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:37.140+0000 7f5651e00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558cfe9f3790 0x558cfe9f5b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:57734/0 (socket says 172.21.15.12:57734) 2024-09-11T00:23:37.144 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:37.140+0000 7f5651e00640 1 -- 172.21.15.12:0/1539545746 learned_addr learned my addr 172.21.15.12:0/1539545746 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:23:37.144 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:37.140+0000 7f5651400640 1 --2- 172.21.15.12:0/1539545746 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558cfe9f60c0 0x558cfe9f84b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:23:37.146 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:37.140+0000 7f5651e00640 1 -- 172.21.15.12:0/1539545746 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558cfe9f8b00 msgr2=0x558cfe9fafc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:23:37.146 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:37.140+0000 7f5651e00640 1 --2- 172.21.15.12:0/1539545746 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558cfe9f8b00 0x558cfe9fafc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:37.146 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:37.140+0000 7f5651e00640 1 -- 172.21.15.12:0/1539545746 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558cfe9f60c0 msgr2=0x558cfe9f84b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:23:37.146 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:37.140+0000 7f5651e00640 1 --2- 172.21.15.12:0/1539545746 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558cfe9f60c0 0x558cfe9f84b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:37.146 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:37.140+0000 7f5651e00640 1 -- 172.21.15.12:0/1539545746 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558cfe922780 con 0x558cfe9f3790 2024-09-11T00:23:37.147 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:37.140+0000 7f5650a00640 1 --2- 172.21.15.12:0/1539545746 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558cfe9f8b00 0x558cfe9fafc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:23:37.147 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:37.140+0000 7f5651400640 1 --2- 172.21.15.12:0/1539545746 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558cfe9f60c0 0x558cfe9f84b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T00:23:37.147 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:37.140+0000 7f5651e00640 1 --2- 172.21.15.12:0/1539545746 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558cfe9f3790 0x558cfe9f5b80 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f5644009f90 tx=0x7f56440118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5135f399954150c5 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:23:37.147 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:37.140+0000 7f563e600640 1 -- 172.21.15.12:0/1539545746 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5644004d20 con 0x558cfe9f3790 2024-09-11T00:23:37.147 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:37.140+0000 7f563e600640 1 -- 172.21.15.12:0/1539545746 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5644004050 con 0x558cfe9f3790 2024-09-11T00:23:37.147 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:37.140+0000 7f563e600640 1 -- 172.21.15.12:0/1539545746 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5644005240 con 0x558cfe9f3790 2024-09-11T00:23:37.147 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:37.140+0000 7f564a000640 1 --2- 172.21.15.12:0/849472218 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f56340b5360 0x7f56340b7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:23:37.147 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:37.140+0000 7f5656c2f740 1 -- 172.21.15.12:0/1539545746 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558cfe9f3790 msgr2=0x558cfe9f5b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:23:37.147 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:37.140+0000 7f5656c2f740 1 --2- 172.21.15.12:0/1539545746 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558cfe9f3790 0x558cfe9f5b80 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f5644009f90 tx=0x7f56440118d0 comp rx=0 tx=0).stop 2024-09-11T00:23:37.147 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:37.140+0000 7f5656c2f740 1 -- 172.21.15.12:0/1539545746 shutdown_connections 2024-09-11T00:23:37.147 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:37.140+0000 7f5656c2f740 1 --2- 172.21.15.12:0/1539545746 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558cfe9f8b00 0x558cfe9fafc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:37.147 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:37.140+0000 7f5656c2f740 1 --2- 172.21.15.12:0/1539545746 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558cfe9f60c0 0x558cfe9f84b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:37.147 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:37.140+0000 7f5656c2f740 1 --2- 172.21.15.12:0/1539545746 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558cfe9f3790 0x558cfe9f5b80 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:37.148 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:37.140+0000 7f5656c2f740 1 -- 172.21.15.12:0/1539545746 >> 172.21.15.12:0/1539545746 conn(0x558cfe9ee8e0 msgr2=0x558cfe9ef670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:23:37.148 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:37.140+0000 7f5651400640 1 --2- 172.21.15.12:0/849472218 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f56340b5360 0x7f56340b7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:23:37.148 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:37.140+0000 7f564a000640 1 -- 172.21.15.12:0/849472218 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9208+0+0 (secure 0 0 0) 0x7f56400e6380 con 0x7f564c075ac0 2024-09-11T00:23:37.148 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:37.140+0000 7f5656c2f740 1 -- 172.21.15.12:0/1539545746 shutdown_connections 2024-09-11T00:23:37.148 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:37.140+0000 7f5656c2f740 1 -- 172.21.15.12:0/1539545746 wait complete. 2024-09-11T00:23:37.148 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:37.145+0000 7f5656c2f740 1 -- start start 2024-09-11T00:23:37.151 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:37.145+0000 7f5651400640 1 --2- 172.21.15.12:0/849472218 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f56340b5360 0x7f56340b7820 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f56400ac470 tx=0x7f5640004a90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:23:37.151 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:37.145+0000 7f5656c2f740 1 Processor -- start 2024-09-11T00:23:37.151 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:37.145+0000 7f5656c2f740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558cfe9f3790 0x558cfe9fcec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:23:37.151 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:37.145+0000 7f5656c2f740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558cfe9f60c0 0x558cfe9fd560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:23:37.152 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:37.145+0000 7f5656c2f740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558cfe9f8b00 0x558cfe9fe130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:23:37.152 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:37.145+0000 7f5656c2f740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x558cfe88dab0 con 0x558cfe9f60c0 2024-09-11T00:23:37.152 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:37.145+0000 7f5656c2f740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x558cfe8fc570 con 0x558cfe9f8b00 2024-09-11T00:23:37.152 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:37.145+0000 7f5656c2f740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x558cfe9fbae0 con 0x558cfe9f3790 2024-09-11T00:23:37.152 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:37.145+0000 7f5656c2f740 10 client.0 _ll_register_callbacks cb 0x558cfe90b100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:23:37.152 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:37.145+0000 7f5650a00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558cfe9f60c0 0x558cfe9fd560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:23:37.152 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:37.145+0000 7f5650a00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558cfe9f60c0 0x558cfe9fd560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:57748/0 (socket says 172.21.15.12:57748) 2024-09-11T00:23:37.152 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:37.145+0000 7f5650a00640 1 -- 172.21.15.12:0/226854940 learned_addr learned my addr 172.21.15.12:0/226854940 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:23:37.152 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:37.145+0000 7f5651e00640 1 --2- 172.21.15.12:0/226854940 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558cfe9f8b00 0x558cfe9fe130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:23:37.152 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:37.146+0000 7f5651e00640 1 --2- 172.21.15.12:0/226854940 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558cfe9f3790 0x558cfe9fcec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:23:37.152 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:37.146+0000 7f5650a00640 1 -- 172.21.15.12:0/226854940 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558cfe9f3790 msgr2=0x558cfe9fcec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:23:37.152 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:37.146+0000 7f5650a00640 1 --2- 172.21.15.12:0/226854940 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558cfe9f3790 0x558cfe9fcec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:37.153 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:37.146+0000 7f5650a00640 1 -- 172.21.15.12:0/226854940 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558cfe9f8b00 msgr2=0x558cfe9fe130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:23:37.153 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:37.146+0000 7f5650a00640 1 --2- 172.21.15.12:0/226854940 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558cfe9f8b00 0x558cfe9fe130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:37.153 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:37.146+0000 7f5650a00640 1 -- 172.21.15.12:0/226854940 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558cfe6b7790 con 0x558cfe9f60c0 2024-09-11T00:23:37.153 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:37.146+0000 7f5650a00640 1 --2- 172.21.15.12:0/226854940 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558cfe9f60c0 0x558cfe9fd560 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f5638006010 tx=0x7f5638004340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae80bff8d376a92a server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:23:37.153 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:37.146+0000 7f5632a00640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-09-11T00:23:37.153 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:37.146+0000 7f5632a00640 1 -- 172.21.15.12:0/226854940 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5638035040 con 0x558cfe9f60c0 2024-09-11T00:23:37.153 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:37.146+0000 7f5656c2f740 1 -- 172.21.15.12:0/226854940 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5644004050 con 0x558cfe9f60c0 2024-09-11T00:23:37.153 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:37.146+0000 7f5632a00640 1 -- 172.21.15.12:0/226854940 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9208+0+0 (secure 0 0 0) 0x7f5638003a00 con 0x558cfe9f60c0 2024-09-11T00:23:37.153 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:37.146+0000 7f5632a00640 1 -- 172.21.15.12:0/226854940 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=64) ==== 24+0+0 (secure 0 0 0) 0x7f563804e050 con 0x558cfe9f60c0 2024-09-11T00:23:37.153 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:37.146+0000 7f5656c2f740 10 client.17298 fetch_fsmap learned FSMap version 64 2024-09-11T00:23:37.153 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:37.146+0000 7f5656c2f740 1 -- 172.21.15.12:0/226854940 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=64,monmap=4+}) -- 0x558cfe730620 con 0x558cfe9f60c0 2024-09-11T00:23:37.153 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:37.146+0000 7f5632a00640 1 -- 172.21.15.12:0/226854940 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5644004050 con 0x558cfe9f60c0 2024-09-11T00:23:37.153 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:37.146+0000 7f5632a00640 1 -- 172.21.15.12:0/226854940 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 64) ==== 8382+0+0 (secure 0 0 0) 0x7f563803c440 con 0x558cfe9f60c0 2024-09-11T00:23:37.154 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:37.146+0000 7f5656c2f740 10 client.17298 fetch_fsmap finished waiting for FSMap version 64 2024-09-11T00:23:37.154 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:37.146+0000 7f5656c2f740 10 client.17298 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T00:23:37.154 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:37.146+0000 7f5656c2f740 20 client.17298 populate_metadata read hostname 'smithi012' 2024-09-11T00:23:37.154 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:37.146+0000 7f5656c2f740 1 --2- 172.21.15.12:0/226854940 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x558cfea08810 0x558cfea0ac00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:23:37.154 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:37.146+0000 7f5656c2f740 4 client.17298 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T00:23:37.154 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:37.146+0000 7f5656c2f740 1 -- 172.21.15.12:0/226854940 --> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558cfe968600 con 0x558cfea08810 2024-09-11T00:23:37.154 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:37.146+0000 7f5651e00640 1 --2- 172.21.15.12:0/226854940 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x558cfea08810 0x558cfea0ac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:23:37.154 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:37.146+0000 7f5651e00640 1 --2- 172.21.15.12:0/226854940 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x558cfea08810 0x558cfea0ac00 crc :-1 s=READY pgs=235 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=599d58456ba8771c server_cookie=fab1cb507272ca0e in_seq=0 out_seq=0 2024-09-11T00:23:37.154 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:37.146+0000 7f5632a00640 10 client.17298 ms_handle_connect on v2:172.21.15.165:6838/501312984 2024-09-11T00:23:37.154 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:37.146+0000 7f5632a00640 1 -- 172.21.15.12:0/226854940 <== mds.0 v2:172.21.15.165:6838/501312984 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x558cfe968600 con 0x558cfea08810 2024-09-11T00:23:37.154 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:37.146+0000 7f5632a00640 10 client.17298 handle_command_reply: tid=0 2024-09-11T00:23:37.154 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:37.146+0000 7f5656c2f740 1 client.17298 shutdown 2024-09-11T00:23:37.154 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:37.146+0000 7f5656c2f740 1 -- 172.21.15.12:0/226854940 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558cfe9f60c0 msgr2=0x558cfe9fd560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:23:37.155 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:37.146+0000 7f5656c2f740 1 --2- 172.21.15.12:0/226854940 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558cfe9f60c0 0x558cfe9fd560 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f5638006010 tx=0x7f5638004340 comp rx=0 tx=0).stop 2024-09-11T00:23:37.155 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:37.146+0000 7f5656c2f740 1 -- 172.21.15.12:0/226854940 shutdown_connections 2024-09-11T00:23:37.155 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:37.146+0000 7f5656c2f740 1 --2- 172.21.15.12:0/226854940 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x558cfea08810 0x558cfea0ac00 crc :-1 s=READY pgs=235 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:37.155 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:37.146+0000 7f5656c2f740 1 --2- 172.21.15.12:0/226854940 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558cfe9f8b00 0x558cfe9fe130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:37.155 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:37.146+0000 7f5656c2f740 1 --2- 172.21.15.12:0/226854940 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558cfe9f60c0 0x558cfe9fd560 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:37.155 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:37.146+0000 7f5656c2f740 1 --2- 172.21.15.12:0/226854940 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558cfe9f3790 0x558cfe9fcec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:37.155 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:37.146+0000 7f5632a00640 0 client.17298 ms_handle_reset on v2:172.21.15.165:6838/501312984 2024-09-11T00:23:37.155 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:37.146+0000 7f5656c2f740 1 -- 172.21.15.12:0/226854940 >> 172.21.15.12:0/226854940 conn(0x558cfe9ee8e0 msgr2=0x558cfe9eda60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:23:37.155 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:37.146+0000 7f5656c2f740 1 -- 172.21.15.12:0/226854940 shutdown_connections 2024-09-11T00:23:37.156 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:37.146+0000 7f5656c2f740 1 -- 172.21.15.12:0/226854940 wait complete. 2024-09-11T00:23:37.156 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:37.146+0000 7f5656c2f740 20 client.17298 trim_cache size 0 max 16384 2024-09-11T00:23:37.168 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:37.164+0000 7f5656c2f740 1 Processor -- start 2024-09-11T00:23:37.168 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:37.164+0000 7f5656c2f740 1 -- start start 2024-09-11T00:23:37.168 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:37.164+0000 7f5656c2f740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558cfe9f3790 0x558cfe9f1930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:23:37.168 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:37.164+0000 7f5656c2f740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558cfe9f60c0 0x558cfe9f1e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:23:37.168 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:37.164+0000 7f5656c2f740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558cfe9f8b00 0x558cfe9fcec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:23:37.168 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:37.164+0000 7f5656c2f740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f564c1ad970 con 0x558cfe9f3790 2024-09-11T00:23:37.168 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:37.164+0000 7f5656c2f740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f564c1af510 con 0x558cfe9f60c0 2024-09-11T00:23:37.168 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:37.164+0000 7f5656c2f740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f564c1ae070 con 0x558cfe9f8b00 2024-09-11T00:23:37.168 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:37.164+0000 7f5651e00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558cfe9f3790 0x558cfe9f1930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:23:37.169 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:37.164+0000 7f5651e00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558cfe9f3790 0x558cfe9f1930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:57750/0 (socket says 172.21.15.12:57750) 2024-09-11T00:23:37.169 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:37.164+0000 7f5651e00640 1 -- 172.21.15.12:0/3213930472 learned_addr learned my addr 172.21.15.12:0/3213930472 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:23:37.169 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:37.164+0000 7f5650a00640 1 --2- 172.21.15.12:0/3213930472 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558cfe9f60c0 0x558cfe9f1e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:23:37.172 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:37.164+0000 7f5651e00640 1 --2- 172.21.15.12:0/3213930472 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558cfe9f8b00 0x558cfe9fcec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:23:37.172 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:37.164+0000 7f5651e00640 1 -- 172.21.15.12:0/3213930472 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558cfe9f8b00 msgr2=0x558cfe9fcec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:23:37.172 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:37.164+0000 7f5651e00640 1 --2- 172.21.15.12:0/3213930472 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558cfe9f8b00 0x558cfe9fcec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:37.172 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:37.164+0000 7f5651e00640 1 -- 172.21.15.12:0/3213930472 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558cfe9f60c0 msgr2=0x558cfe9f1e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:23:37.172 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:37.164+0000 7f5651e00640 1 --2- 172.21.15.12:0/3213930472 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558cfe9f60c0 0x558cfe9f1e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:37.172 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:37.164+0000 7f5651e00640 1 -- 172.21.15.12:0/3213930472 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558cfea0af70 con 0x558cfe9f3790 2024-09-11T00:23:37.172 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:37.164+0000 7f5650a00640 1 --2- 172.21.15.12:0/3213930472 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558cfe9f60c0 0x558cfe9f1e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:23:37.172 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:37.164+0000 7f5651e00640 1 --2- 172.21.15.12:0/3213930472 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558cfe9f3790 0x558cfe9f1930 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f5644006010 tx=0x7f5644009a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=776612b43f158865 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:23:37.172 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:37.164+0000 7f563dc00640 1 -- 172.21.15.12:0/3213930472 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5644005040 con 0x558cfe9f3790 2024-09-11T00:23:37.172 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:37.164+0000 7f563dc00640 1 -- 172.21.15.12:0/3213930472 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5644005690 con 0x558cfe9f3790 2024-09-11T00:23:37.173 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:37.164+0000 7f563dc00640 1 -- 172.21.15.12:0/3213930472 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f564400fb50 con 0x558cfe9f3790 2024-09-11T00:23:37.173 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:37.164+0000 7f5656c2f740 1 -- 172.21.15.12:0/3213930472 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558cfe9f3790 msgr2=0x558cfe9f1930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:23:37.173 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:37.164+0000 7f5656c2f740 1 --2- 172.21.15.12:0/3213930472 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558cfe9f3790 0x558cfe9f1930 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f5644006010 tx=0x7f5644009a50 comp rx=0 tx=0).stop 2024-09-11T00:23:37.173 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:37.164+0000 7f5656c2f740 1 -- 172.21.15.12:0/3213930472 shutdown_connections 2024-09-11T00:23:37.173 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:37.164+0000 7f5656c2f740 1 --2- 172.21.15.12:0/3213930472 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558cfe9f8b00 0x558cfe9fcec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:37.173 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:37.164+0000 7f5656c2f740 1 --2- 172.21.15.12:0/3213930472 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558cfe9f60c0 0x558cfe9f1e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:37.173 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:37.164+0000 7f5656c2f740 1 --2- 172.21.15.12:0/3213930472 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558cfe9f3790 0x558cfe9f1930 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:37.173 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:37.164+0000 7f5656c2f740 1 -- 172.21.15.12:0/3213930472 >> 172.21.15.12:0/3213930472 conn(0x558cfe9ee8e0 msgr2=0x558cfe9f2c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:23:37.173 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:37.164+0000 7f5656c2f740 1 -- 172.21.15.12:0/3213930472 shutdown_connections 2024-09-11T00:23:37.173 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:37.164+0000 7f5656c2f740 1 -- 172.21.15.12:0/3213930472 wait complete. 2024-09-11T00:23:37.173 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:37.169+0000 7f5656c2f740 1 -- start start 2024-09-11T00:23:37.173 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:37.169+0000 7f5656c2f740 1 Processor -- start 2024-09-11T00:23:37.173 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:37.169+0000 7f5656c2f740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558cfe9f3790 0x558cfea03b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:23:37.174 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:37.169+0000 7f5656c2f740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558cfe9f60c0 0x558cfea04060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:23:37.174 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:37.169+0000 7f5656c2f740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558cfe9f8b00 0x558cfea045a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:23:37.174 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:37.169+0000 7f5656c2f740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f564c1aee40 con 0x558cfe9f3790 2024-09-11T00:23:37.174 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:37.169+0000 7f5656c2f740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f564c1ae740 con 0x558cfe9f8b00 2024-09-11T00:23:37.174 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:37.169+0000 7f5656c2f740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f564c1ab950 con 0x558cfe9f60c0 2024-09-11T00:23:37.174 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:37.169+0000 7f5656c2f740 10 client.0 _ll_register_callbacks cb 0x7f5638009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:23:37.174 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:37.169+0000 7f5651e00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558cfe9f3790 0x558cfea03b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:23:37.174 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:37.169+0000 7f5651e00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558cfe9f3790 0x558cfea03b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:57758/0 (socket says 172.21.15.12:57758) 2024-09-11T00:23:37.174 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:37.169+0000 7f5651e00640 1 -- 172.21.15.12:0/3786341465 learned_addr learned my addr 172.21.15.12:0/3786341465 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:23:37.174 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:37.169+0000 7f5650a00640 1 --2- 172.21.15.12:0/3786341465 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558cfe9f60c0 0x558cfea04060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:23:37.174 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:37.169+0000 7f5651e00640 1 --2- 172.21.15.12:0/3786341465 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558cfe9f8b00 0x558cfea045a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:23:37.174 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:37.169+0000 7f5651e00640 1 -- 172.21.15.12:0/3786341465 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558cfe9f60c0 msgr2=0x558cfea04060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:23:37.175 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:37.169+0000 7f5651e00640 1 --2- 172.21.15.12:0/3786341465 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558cfe9f60c0 0x558cfea04060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:37.175 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:37.169+0000 7f5651e00640 1 -- 172.21.15.12:0/3786341465 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558cfe9f8b00 msgr2=0x558cfea045a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:23:37.175 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:37.169+0000 7f5651e00640 1 --2- 172.21.15.12:0/3786341465 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558cfe9f8b00 0x558cfea045a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:37.175 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:37.169+0000 7f5651e00640 1 -- 172.21.15.12:0/3786341465 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558cfe9f8820 con 0x558cfe9f3790 2024-09-11T00:23:37.175 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:37.169+0000 7f5651e00640 1 --2- 172.21.15.12:0/3786341465 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558cfe9f3790 0x558cfea03b20 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f5644000f80 tx=0x7f5644009500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21d2ded0aefd154 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:23:37.175 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:37.169+0000 7f5632a00640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-09-11T00:23:37.175 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:37.169+0000 7f5632a00640 1 -- 172.21.15.12:0/3786341465 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5644005240 con 0x558cfe9f3790 2024-09-11T00:23:37.175 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:37.169+0000 7f5656c2f740 1 -- 172.21.15.12:0/3786341465 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5644005690 con 0x558cfe9f3790 2024-09-11T00:23:37.175 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:37.169+0000 7f5632a00640 1 -- 172.21.15.12:0/3786341465 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9208+0+0 (secure 0 0 0) 0x7f56440053e0 con 0x558cfe9f3790 2024-09-11T00:23:37.176 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:37.169+0000 7f5632a00640 1 -- 172.21.15.12:0/3786341465 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=64) ==== 24+0+0 (secure 0 0 0) 0x7f5644046050 con 0x558cfe9f3790 2024-09-11T00:23:37.176 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:37.169+0000 7f5656c2f740 10 client.17304 fetch_fsmap learned FSMap version 64 2024-09-11T00:23:37.176 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:37.169+0000 7f5656c2f740 1 -- 172.21.15.12:0/3786341465 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=64,monmap=4+}) -- 0x558cfea75a80 con 0x558cfe9f3790 2024-09-11T00:23:37.177 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:37.169+0000 7f5632a00640 1 -- 172.21.15.12:0/3786341465 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5644005690 con 0x558cfe9f3790 2024-09-11T00:23:37.177 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:37.169+0000 7f5632a00640 1 -- 172.21.15.12:0/3786341465 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 64) ==== 8382+0+0 (secure 0 0 0) 0x7f5644028800 con 0x558cfe9f3790 2024-09-11T00:23:37.177 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:37.169+0000 7f5656c2f740 10 client.17304 fetch_fsmap finished waiting for FSMap version 64 2024-09-11T00:23:37.177 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:37.169+0000 7f5656c2f740 10 client.17304 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T00:23:37.177 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:37.169+0000 7f5656c2f740 20 client.17304 populate_metadata read hostname 'smithi012' 2024-09-11T00:23:37.177 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:37.169+0000 7f5656c2f740 1 --2- 172.21.15.12:0/3786341465 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x558cfea76270 0x558cfea78690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:23:37.177 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:37.169+0000 7f5656c2f740 4 client.17304 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T00:23:37.177 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:37.169+0000 7f5656c2f740 1 -- 172.21.15.12:0/3786341465 --> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] -- command(tid 0: {"prefix": "flush journal"}) -- 0x558cfe9640b0 con 0x558cfea76270 2024-09-11T00:23:37.177 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:37.169+0000 7f5650a00640 1 --2- 172.21.15.12:0/3786341465 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x558cfea76270 0x558cfea78690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:23:37.177 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:37.169+0000 7f5650a00640 1 --2- 172.21.15.12:0/3786341465 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x558cfea76270 0x558cfea78690 crc :-1 s=READY pgs=236 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cdd34a289e098cd2 server_cookie=d3a39201e103278 in_seq=0 out_seq=0 2024-09-11T00:23:37.177 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:37.169+0000 7f5632a00640 10 client.17304 ms_handle_connect on v2:172.21.15.165:6838/501312984 2024-09-11T00:23:37.178 INFO:tasks.workunit.client.0.smithi012.stdout: 10 69051 20.94 MB/sec execute 93 sec latency 22471.245 ms 2024-09-11T00:23:37.234 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:37.229+0000 7f5632a00640 1 -- 172.21.15.12:0/3786341465 <== mds.0 v2:172.21.15.165:6838/501312984 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x558cfe9640b0 con 0x558cfea76270 2024-09-11T00:23:37.235 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:37.229+0000 7f5632a00640 10 client.17304 handle_command_reply: tid=0 2024-09-11T00:23:37.235 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:37.229+0000 7f5656c2f740 1 client.17304 shutdown 2024-09-11T00:23:37.235 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:37.229+0000 7f5656c2f740 1 -- 172.21.15.12:0/3786341465 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558cfe9f3790 msgr2=0x558cfea03b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:23:37.235 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:37.229+0000 7f5656c2f740 1 --2- 172.21.15.12:0/3786341465 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558cfe9f3790 0x558cfea03b20 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f5644000f80 tx=0x7f5644009500 comp rx=0 tx=0).stop 2024-09-11T00:23:37.235 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:37.229+0000 7f5656c2f740 1 -- 172.21.15.12:0/3786341465 shutdown_connections 2024-09-11T00:23:37.235 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:37.229+0000 7f5656c2f740 1 --2- 172.21.15.12:0/3786341465 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x558cfea76270 0x558cfea78690 crc :-1 s=READY pgs=236 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:37.235 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:37.229+0000 7f5656c2f740 1 --2- 172.21.15.12:0/3786341465 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558cfe9f8b00 0x558cfea045a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:37.235 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:37.229+0000 7f5656c2f740 1 --2- 172.21.15.12:0/3786341465 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558cfe9f60c0 0x558cfea04060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:37.235 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:37.229+0000 7f5656c2f740 1 --2- 172.21.15.12:0/3786341465 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558cfe9f3790 0x558cfea03b20 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:37.235 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:37.229+0000 7f5656c2f740 1 -- 172.21.15.12:0/3786341465 >> 172.21.15.12:0/3786341465 conn(0x558cfe9ee8e0 msgr2=0x558cfe9fce60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:23:37.235 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:37.229+0000 7f5632a00640 0 client.17304 ms_handle_reset on v2:172.21.15.165:6838/501312984 2024-09-11T00:23:37.236 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:37.229+0000 7f5656c2f740 1 -- 172.21.15.12:0/3786341465 shutdown_connections 2024-09-11T00:23:37.236 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:37.229+0000 7f5656c2f740 1 -- 172.21.15.12:0/3786341465 wait complete. 2024-09-11T00:23:37.236 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:37.229+0000 7f5656c2f740 20 client.17304 trim_cache size 0 max 16384 2024-09-11T00:23:37.236 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:{ 2024-09-11T00:23:37.236 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "message": "", 2024-09-11T00:23:37.236 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "return_code": 0 2024-09-11T00:23:37.236 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:} 2024-09-11T00:23:37.237 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:37.229+0000 7f5653e00640 1 -- 172.21.15.12:0/849472218 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f56340b5360 msgr2=0x7f56340b7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:23:37.237 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:37.229+0000 7f5653e00640 1 --2- 172.21.15.12:0/849472218 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f56340b5360 0x7f56340b7820 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f56400ac470 tx=0x7f5640004a90 comp rx=0 tx=0).stop 2024-09-11T00:23:37.237 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:37.229+0000 7f5653e00640 1 -- 172.21.15.12:0/849472218 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f564c075ac0 msgr2=0x7f564c19afb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:23:37.237 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:37.229+0000 7f5653e00640 1 --2- 172.21.15.12:0/849472218 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f564c075ac0 0x7f564c19afb0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f564000e9f0 tx=0x7f564000edf0 comp rx=0 tx=0).stop 2024-09-11T00:23:37.237 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:37.229+0000 7f5653e00640 1 -- 172.21.15.12:0/849472218 shutdown_connections 2024-09-11T00:23:37.237 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:37.229+0000 7f5653e00640 1 --2- 172.21.15.12:0/849472218 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f56340b5360 0x7f56340b7820 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:37.238 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:37.229+0000 7f5653e00640 1 --2- 172.21.15.12:0/849472218 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f564c10a910 0x7f564c19c230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:37.238 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:37.229+0000 7f5653e00640 1 --2- 172.21.15.12:0/849472218 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f564c076f30 0x7f564c19b600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:37.238 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:37.229+0000 7f5653e00640 1 --2- 172.21.15.12:0/849472218 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f564c075ac0 0x7f564c19afb0 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:37.238 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:37.229+0000 7f5653e00640 1 -- 172.21.15.12:0/849472218 >> 172.21.15.12:0/849472218 conn(0x7f564c0fe250 msgr2=0x7f564c0fe870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:23:37.238 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:37.229+0000 7f5653e00640 1 -- 172.21.15.12:0/849472218 shutdown_connections 2024-09-11T00:23:37.238 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:37.229+0000 7f5653e00640 1 -- 172.21.15.12:0/849472218 wait complete. 2024-09-11T00:23:37.797 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:23:37 smithi012 ceph-mon[30333]: pgmap v895: 129 pgs: 129 active+clean; 516 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 284 KiB/s rd, 849 KiB/s wr, 192 op/s 2024-09-11T00:23:37.831 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:23:37 smithi178 ceph-mon[35919]: pgmap v895: 129 pgs: 129 active+clean; 516 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 284 KiB/s rd, 849 KiB/s wr, 192 op/s 2024-09-11T00:23:38.026 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:23:37.989+0000 7f7f57000640 1 -- 172.21.15.12:0/2126085182 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7f50077ff0 msgr2=0x7f7f50113450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:23:38.026 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:23:37.989+0000 7f7f57000640 1 --2- 172.21.15.12:0/2126085182 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7f50077ff0 0x7f7f50113450 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f7f3c009900 tx=0x7f7f3c02a3f0 comp rx=0 tx=0).stop 2024-09-11T00:23:38.026 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:23:37.989+0000 7f7f57000640 1 -- 172.21.15.12:0/2126085182 shutdown_connections 2024-09-11T00:23:38.027 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:23:37.989+0000 7f7f57000640 1 --2- 172.21.15.12:0/2126085182 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7f50113990 0x7f7f50115d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:38.027 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:23:37.989+0000 7f7f57000640 1 --2- 172.21.15.12:0/2126085182 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7f50077ff0 0x7f7f50113450 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:38.027 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:23:37.989+0000 7f7f57000640 1 --2- 172.21.15.12:0/2126085182 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7f500776d0 0x7f7f50077ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:38.027 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:23:37.989+0000 7f7f57000640 1 -- 172.21.15.12:0/2126085182 >> 172.21.15.12:0/2126085182 conn(0x7f7f50100a50 msgr2=0x7f7f50102e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:23:38.027 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:23:37.989+0000 7f7f57000640 1 -- 172.21.15.12:0/2126085182 shutdown_connections 2024-09-11T00:23:38.027 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:23:37.990+0000 7f7f57000640 1 -- 172.21.15.12:0/2126085182 wait complete. 2024-09-11T00:23:38.027 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:23:37.991+0000 7f7f57000640 1 Processor -- start 2024-09-11T00:23:38.027 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:23:37.991+0000 7f7f57000640 1 -- start start 2024-09-11T00:23:38.027 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:23:37.991+0000 7f7f57000640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7f500776d0 0x7f7f5019f890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:23:38.027 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:23:37.991+0000 7f7f57000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7f50077ff0 0x7f7f5019fdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:23:38.027 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:23:37.991+0000 7f7f57000640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7f50113990 0x7f7f501a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:23:38.027 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:23:37.991+0000 7f7f57000640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f7f501193e0 con 0x7f7f50113990 2024-09-11T00:23:38.027 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:23:37.991+0000 7f7f57000640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f7f50119260 con 0x7f7f50077ff0 2024-09-11T00:23:38.028 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:23:37.991+0000 7f7f57000640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f7f50119560 con 0x7f7f500776d0 2024-09-11T00:23:38.028 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:23:37.991+0000 7f7f4f400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7f50077ff0 0x7f7f5019fdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:23:38.028 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:23:37.991+0000 7f7f55000640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7f50113990 0x7f7f501a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:23:38.028 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:23:37.991+0000 7f7f4f400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7f50077ff0 0x7f7f5019fdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.12:44644/0 (socket says 172.21.15.12:44644) 2024-09-11T00:23:38.028 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:23:37.991+0000 7f7f4f400640 1 -- 172.21.15.12:0/403122088 learned_addr learned my addr 172.21.15.12:0/403122088 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:23:38.028 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:23:37.991+0000 7f7f4fe00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7f500776d0 0x7f7f5019f890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:23:38.028 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:23:37.991+0000 7f7f55000640 1 -- 172.21.15.12:0/403122088 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7f500776d0 msgr2=0x7f7f5019f890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:23:38.028 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:23:37.991+0000 7f7f55000640 1 --2- 172.21.15.12:0/403122088 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7f500776d0 0x7f7f5019f890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:38.028 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:23:37.991+0000 7f7f55000640 1 -- 172.21.15.12:0/403122088 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7f50077ff0 msgr2=0x7f7f5019fdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:23:38.028 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:23:37.991+0000 7f7f55000640 1 --2- 172.21.15.12:0/403122088 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7f50077ff0 0x7f7f5019fdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:38.028 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:23:37.991+0000 7f7f55000640 1 -- 172.21.15.12:0/403122088 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f501a5940 con 0x7f7f50113990 2024-09-11T00:23:38.028 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:23:37.991+0000 7f7f55000640 1 --2- 172.21.15.12:0/403122088 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7f50113990 0x7f7f501a5170 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f7f48005fd0 tx=0x7f7f4800c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:23:38.029 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:23:37.991+0000 7f7f4cc00640 1 -- 172.21.15.12:0/403122088 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f48003b70 con 0x7f7f50113990 2024-09-11T00:23:38.029 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:23:37.991+0000 7f7f4cc00640 1 -- 172.21.15.12:0/403122088 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7f48003d10 con 0x7f7f50113990 2024-09-11T00:23:38.029 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:23:37.991+0000 7f7f57000640 1 -- 172.21.15.12:0/403122088 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7f501a5c30 con 0x7f7f50113990 2024-09-11T00:23:38.029 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:23:37.991+0000 7f7f4cc00640 1 -- 172.21.15.12:0/403122088 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f480107a0 con 0x7f7f50113990 2024-09-11T00:23:38.029 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:23:37.991+0000 7f7f57000640 1 -- 172.21.15.12:0/403122088 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7f501ac2f0 con 0x7f7f50113990 2024-09-11T00:23:38.029 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:23:37.991+0000 7f7f57000640 1 -- 172.21.15.12:0/403122088 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7f50077ab0 con 0x7f7f50113990 2024-09-11T00:23:38.029 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:23:37.996+0000 7f7f4cc00640 1 -- 172.21.15.12:0/403122088 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7f7f48003fc0 con 0x7f7f50113990 2024-09-11T00:23:38.029 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:23:37.996+0000 7f7f4cc00640 1 --2- 172.21.15.12:0/403122088 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f7f340b5430 0x7f7f340b78f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:23:38.029 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:23:37.996+0000 7f7f4cc00640 1 -- 172.21.15.12:0/403122088 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9208+0+0 (secure 0 0 0) 0x7f7f480e6450 con 0x7f7f50113990 2024-09-11T00:23:38.029 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:23:37.996+0000 7f7f4fe00640 1 --2- 172.21.15.12:0/403122088 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f7f340b5430 0x7f7f340b78f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:23:38.029 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:23:37.996+0000 7f7f4fe00640 1 --2- 172.21.15.12:0/403122088 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f7f340b5430 0x7f7f340b78f0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f7f40009bd0 tx=0x7f7f40005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:23:38.029 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:23:38.002+0000 7f7f4cc00640 1 -- 172.21.15.12:0/403122088 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f7f480acee0 con 0x7f7f50113990 2024-09-11T00:23:38.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:23:37 smithi165 ceph-mon[35356]: pgmap v895: 129 pgs: 129 active+clean; 516 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 284 KiB/s rd, 849 KiB/s wr, 192 op/s 2024-09-11T00:23:38.175 INFO:tasks.workunit.client.0.smithi012.stdout: 10 69052 20.72 MB/sec execute 94 sec latency 23471.292 ms 2024-09-11T00:23:38.239 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:23:38.238+0000 7f7f57000640 1 -- 172.21.15.12:0/403122088 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7f501ac640 con 0x7f7f50113990 2024-09-11T00:23:38.239 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:23:38.238+0000 7f7f4cc00640 1 -- 172.21.15.12:0/403122088 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 64 v64) ==== 94+0+11968 (secure 0 0 0) 0x7f7f480ac7e0 con 0x7f7f50113990 2024-09-11T00:23:38.240 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-11T00:23:38.240 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":64,"btime":"2024-09-11T00:23:30:106469+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14577,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6835/447171591","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":447171591},{"type":"v1","addr":"172.21.15.12:6835","nonce":447171591}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14589,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/901486040","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":901486040},{"type":"v1","addr":"172.21.15.12:6837","nonce":901486040}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":64,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T00:04:04.739881+0000","modified":"2024-09-11T00:23:29.550496+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24674,"mds_1":24643,"mds_2":24638,"mds_3":24637,"mds_4":24614},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14631":{"gid":14631,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.12:6841/3069919437","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6840","nonce":3069919437},{"type":"v1","addr":"172.21.15.12:6841","nonce":3069919437}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24562":{"gid":24562,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6833/1141636112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":1141636112},{"type":"v1","addr":"172.21.15.178:6833","nonce":1141636112}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24581":{"gid":24581,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6833/3111980970","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6832","nonce":3111980970},{"type":"v1","addr":"172.21.15.165:6833","nonce":3111980970}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24607":{"gid":24607,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6835/4084868287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":4084868287},{"type":"v1","addr":"172.21.15.178:6835","nonce":4084868287}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24614":{"gid":24614,"name":"e","rank":4,"incarnation":26,"state":"up:active","state_seq":21,"addr":"172.21.15.165:6835/2425395274","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":2425395274},{"type":"v1","addr":"172.21.15.165:6835","nonce":2425395274}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24637":{"gid":24637,"name":"i","rank":3,"incarnation":24,"state":"up:active","state_seq":18,"addr":"172.21.15.178:6839/70628097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6838","nonce":70628097},{"type":"v1","addr":"172.21.15.178:6839","nonce":70628097}]},"join_fscid":-1,"export_targets":[1],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24638":{"gid":24638,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":18,"addr":"172.21.15.165:6837/6215088","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6836","nonce":6215088},{"type":"v1","addr":"172.21.15.165:6837","nonce":6215088}]},"join_fscid":-1,"export_targets":[3],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24643":{"gid":24643,"name":"g","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.12:6839/3124093009","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6838","nonce":3124093009},{"type":"v1","addr":"172.21.15.12:6839","nonce":3124093009}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24674":{"gid":24674,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.165:6839/501312984","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6838","nonce":501312984},{"type":"v1","addr":"172.21.15.165:6839","nonce":501312984}]},"join_fscid":-1,"export_targets":[1],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25672":{"gid":25672,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6837/308601741","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":308601741},{"type":"v1","addr":"172.21.15.178:6837","nonce":308601741}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24674,"qdb_cluster":[24674,24643,24638,24637,24614]},"id":1}]} 2024-09-11T00:23:38.241 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 64 2024-09-11T00:23:38.242 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:23:38.241+0000 7f7f57000640 1 -- 172.21.15.12:0/403122088 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f7f340b5430 msgr2=0x7f7f340b78f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:23:38.242 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:23:38.241+0000 7f7f57000640 1 --2- 172.21.15.12:0/403122088 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f7f340b5430 0x7f7f340b78f0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f7f40009bd0 tx=0x7f7f40005f90 comp rx=0 tx=0).stop 2024-09-11T00:23:38.242 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:23:38.241+0000 7f7f57000640 1 -- 172.21.15.12:0/403122088 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7f50113990 msgr2=0x7f7f501a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:23:38.243 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:23:38.241+0000 7f7f57000640 1 --2- 172.21.15.12:0/403122088 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7f50113990 0x7f7f501a5170 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f7f48005fd0 tx=0x7f7f4800c6a0 comp rx=0 tx=0).stop 2024-09-11T00:23:38.243 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:23:38.241+0000 7f7f57000640 1 -- 172.21.15.12:0/403122088 shutdown_connections 2024-09-11T00:23:38.243 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:23:38.241+0000 7f7f57000640 1 --2- 172.21.15.12:0/403122088 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f7f340b5430 0x7f7f340b78f0 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:38.243 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:23:38.241+0000 7f7f57000640 1 --2- 172.21.15.12:0/403122088 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7f50113990 0x7f7f501a5170 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:38.243 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:23:38.241+0000 7f7f57000640 1 --2- 172.21.15.12:0/403122088 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7f50077ff0 0x7f7f5019fdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:38.243 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:23:38.241+0000 7f7f57000640 1 --2- 172.21.15.12:0/403122088 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7f500776d0 0x7f7f5019f890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:38.244 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:23:38.241+0000 7f7f57000640 1 -- 172.21.15.12:0/403122088 >> 172.21.15.12:0/403122088 conn(0x7f7f50100a50 msgr2=0x7f7f50102e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:23:38.244 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:23:38.241+0000 7f7f57000640 1 -- 172.21.15.12:0/403122088 shutdown_connections 2024-09-11T00:23:38.244 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:23:38.241+0000 7f7f57000640 1 -- 172.21.15.12:0/403122088 wait complete. 2024-09-11T00:23:38.606 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:23:38 smithi012 ceph-mon[30333]: from='client.? 172.21.15.12:0/403122088' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T00:23:38.831 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:23:38 smithi178 ceph-mon[35919]: from='client.? 172.21.15.12:0/403122088' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T00:23:39.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:23:38 smithi165 ceph-mon[35356]: from='client.? 172.21.15.12:0/403122088' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T00:23:39.175 INFO:tasks.workunit.client.0.smithi012.stdout: 10 69052 20.50 MB/sec execute 95 sec latency 24471.354 ms 2024-09-11T00:23:39.402 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24674 quiesce db / --timeout 90 --expiration 154.8 --await-for 45.0' 2024-09-11T00:23:39.403 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid ff3ad74a-6fcf-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24674 quiesce db / --timeout 90 --expiration 154.8 --await-for 45.0 2024-09-11T00:23:39.563 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:23:39 smithi012 ceph-mon[30333]: pgmap v896: 129 pgs: 129 active+clean; 516 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 167 KiB/s rd, 157 KiB/s wr, 70 op/s 2024-09-11T00:23:39.831 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:23:39 smithi178 ceph-mon[35919]: pgmap v896: 129 pgs: 129 active+clean; 516 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 167 KiB/s rd, 157 KiB/s wr, 70 op/s 2024-09-11T00:23:39.836 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:23:40.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:23:39 smithi165 ceph-mon[35356]: pgmap v896: 129 pgs: 129 active+clean; 516 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 167 KiB/s rd, 157 KiB/s wr, 70 op/s 2024-09-11T00:23:40.176 INFO:tasks.workunit.client.0.smithi012.stdout: 10 69167 20.31 MB/sec execute 96 sec latency 25116.770 ms 2024-09-11T00:23:40.588 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:23:40.586+0000 7fb85a800640 1 -- 172.21.15.12:0/1466668092 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb8540754c0 msgr2=0x7fb8540758a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:23:40.588 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:23:40.586+0000 7fb85a800640 1 --2- 172.21.15.12:0/1466668092 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb8540754c0 0x7fb8540758a0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fb848009900 tx=0x7fb84802a420 comp rx=0 tx=0).stop 2024-09-11T00:23:40.588 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:23:40.586+0000 7fb85a800640 1 -- 172.21.15.12:0/1466668092 shutdown_connections 2024-09-11T00:23:40.588 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:23:40.586+0000 7fb85a800640 1 --2- 172.21.15.12:0/1466668092 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb85410f220 0x7fb854111610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:40.588 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:23:40.586+0000 7fb85a800640 1 --2- 172.21.15.12:0/1466668092 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb854075de0 0x7fb85410ece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:40.588 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:23:40.586+0000 7fb85a800640 1 --2- 172.21.15.12:0/1466668092 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb8540754c0 0x7fb8540758a0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:40.588 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:23:40.586+0000 7fb85a800640 1 -- 172.21.15.12:0/1466668092 >> 172.21.15.12:0/1466668092 conn(0x7fb8540fe660 msgr2=0x7fb854100a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:23:40.588 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:23:40.586+0000 7fb85a800640 1 -- 172.21.15.12:0/1466668092 shutdown_connections 2024-09-11T00:23:40.588 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:23:40.586+0000 7fb85a800640 1 -- 172.21.15.12:0/1466668092 wait complete. 2024-09-11T00:23:40.589 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:23:40.587+0000 7fb85a800640 1 Processor -- start 2024-09-11T00:23:40.589 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:23:40.587+0000 7fb85a800640 1 -- start start 2024-09-11T00:23:40.589 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:23:40.588+0000 7fb85a800640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb8540754c0 0x7fb85419b2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:23:40.589 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:23:40.588+0000 7fb85a800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb854075de0 0x7fb85419b810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:23:40.589 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:23:40.588+0000 7fb85a800640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb85410f220 0x7fb8541a0bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:23:40.590 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:23:40.588+0000 7fb85a800640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fb854114e80 con 0x7fb85410f220 2024-09-11T00:23:40.590 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:23:40.588+0000 7fb85a800640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fb854114d00 con 0x7fb854075de0 2024-09-11T00:23:40.590 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:23:40.588+0000 7fb85a800640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fb854115000 con 0x7fb8540754c0 2024-09-11T00:23:40.590 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:23:40.588+0000 7fb859400640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb8540754c0 0x7fb85419b2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:23:40.590 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:23:40.588+0000 7fb859400640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb8540754c0 0x7fb85419b2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.12:57196/0 (socket says 172.21.15.12:57196) 2024-09-11T00:23:40.590 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:23:40.588+0000 7fb859400640 1 -- 172.21.15.12:0/2679511716 learned_addr learned my addr 172.21.15.12:0/2679511716 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:23:40.590 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:23:40.588+0000 7fb858a00640 1 --2- 172.21.15.12:0/2679511716 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb854075de0 0x7fb85419b810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:23:40.590 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:23:40.588+0000 7fb859e00640 1 --2- 172.21.15.12:0/2679511716 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb85410f220 0x7fb8541a0bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:23:40.591 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:23:40.588+0000 7fb859400640 1 -- 172.21.15.12:0/2679511716 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb854075de0 msgr2=0x7fb85419b810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:23:40.591 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:23:40.588+0000 7fb859400640 1 --2- 172.21.15.12:0/2679511716 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb854075de0 0x7fb85419b810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:40.591 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:23:40.588+0000 7fb859400640 1 -- 172.21.15.12:0/2679511716 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb85410f220 msgr2=0x7fb8541a0bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:23:40.591 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:23:40.588+0000 7fb859400640 1 --2- 172.21.15.12:0/2679511716 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb85410f220 0x7fb8541a0bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:40.591 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:23:40.588+0000 7fb859400640 1 -- 172.21.15.12:0/2679511716 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8541a1380 con 0x7fb8540754c0 2024-09-11T00:23:40.591 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:23:40.588+0000 7fb858a00640 1 --2- 172.21.15.12:0/2679511716 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb854075de0 0x7fb85419b810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:23:40.591 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:23:40.588+0000 7fb859400640 1 --2- 172.21.15.12:0/2679511716 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb8540754c0 0x7fb85419b2d0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fb848005940 tx=0x7fb848004270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:23:40.592 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:23:40.588+0000 7fb852000640 1 -- 172.21.15.12:0/2679511716 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb84803f030 con 0x7fb8540754c0 2024-09-11T00:23:40.592 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:23:40.588+0000 7fb85a800640 1 -- 172.21.15.12:0/2679511716 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb8541a1610 con 0x7fb8540754c0 2024-09-11T00:23:40.592 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:23:40.588+0000 7fb852000640 1 -- 172.21.15.12:0/2679511716 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb848003aa0 con 0x7fb8540754c0 2024-09-11T00:23:40.592 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:23:40.588+0000 7fb852000640 1 -- 172.21.15.12:0/2679511716 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb848035070 con 0x7fb8540754c0 2024-09-11T00:23:40.592 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:23:40.588+0000 7fb85a800640 1 -- 172.21.15.12:0/2679511716 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8541a7d30 con 0x7fb8540754c0 2024-09-11T00:23:40.595 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:23:40.594+0000 7fb852000640 1 -- 172.21.15.12:0/2679511716 <== mon.1 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7fb8480035f0 con 0x7fb8540754c0 2024-09-11T00:23:40.596 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:23:40.594+0000 7fb85e9fc740 1 Processor -- start 2024-09-11T00:23:40.596 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:23:40.594+0000 7fb85e9fc740 1 -- start start 2024-09-11T00:23:40.597 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:23:40.594+0000 7fb85e9fc740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559b6357f2f0 0x559b6357f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:23:40.597 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:23:40.594+0000 7fb85e9fc740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559b6357fc10 0x559b6358b660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:23:40.597 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:23:40.594+0000 7fb85e9fc740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559b6358bba0 0x559b6358df90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:23:40.598 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:23:40.594+0000 7fb85e9fc740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x559b634f4ca0 con 0x559b6357fc10 2024-09-11T00:23:40.598 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:23:40.594+0000 7fb85e9fc740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x559b634e40d0 con 0x559b6358bba0 2024-09-11T00:23:40.598 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:23:40.594+0000 7fb85e9fc740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x559b634e7ee0 con 0x559b6357f2f0 2024-09-11T00:23:40.598 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:23:40.594+0000 7fb859400640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559b6357fc10 0x559b6358b660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:23:40.598 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:23:40.594+0000 7fb859e00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559b6357f2f0 0x559b6357f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:23:40.598 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:23:40.594+0000 7fb859400640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559b6357fc10 0x559b6358b660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:41798/0 (socket says 172.21.15.12:41798) 2024-09-11T00:23:40.598 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:23:40.594+0000 7fb859400640 1 -- 172.21.15.12:0/2183023332 learned_addr learned my addr 172.21.15.12:0/2183023332 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:23:40.598 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:23:40.594+0000 7fb858a00640 1 --2- 172.21.15.12:0/2183023332 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559b6358bba0 0x559b6358df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:23:40.598 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:23:40.594+0000 7fb858a00640 1 -- 172.21.15.12:0/2183023332 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559b6357f2f0 msgr2=0x559b6357f6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:23:40.599 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:23:40.594+0000 7fb858a00640 1 --2- 172.21.15.12:0/2183023332 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559b6357f2f0 0x559b6357f6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:40.599 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:23:40.594+0000 7fb858a00640 1 -- 172.21.15.12:0/2183023332 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559b6357fc10 msgr2=0x559b6358b660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:23:40.599 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:23:40.594+0000 7fb858a00640 1 --2- 172.21.15.12:0/2183023332 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559b6357fc10 0x559b6358b660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:40.599 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:23:40.594+0000 7fb852000640 1 --2- 172.21.15.12:0/2679511716 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fb83c0b5390 0x7fb83c0b7850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:23:40.599 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:23:40.594+0000 7fb858a00640 1 -- 172.21.15.12:0/2183023332 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559b63496400 con 0x559b6358bba0 2024-09-11T00:23:40.599 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:23:40.594+0000 7fb852000640 1 -- 172.21.15.12:0/2679511716 <== mon.1 v2:172.21.15.178:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9208+0+0 (secure 0 0 0) 0x7fb848106a00 con 0x7fb8540754c0 2024-09-11T00:23:40.599 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:23:40.594+0000 7fb858a00640 1 --2- 172.21.15.12:0/2183023332 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559b6358bba0 0x559b6358df90 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fb840006fd0 tx=0x7fb84000e5a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=95fc623801397194 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:23:40.599 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:23:40.594+0000 7fb859e00640 1 --2- 172.21.15.12:0/2679511716 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fb83c0b5390 0x7fb83c0b7850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:23:40.600 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:23:40.594+0000 7fb846600640 1 -- 172.21.15.12:0/2183023332 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb840017070 con 0x559b6358bba0 2024-09-11T00:23:40.600 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:23:40.594+0000 7fb846600640 1 -- 172.21.15.12:0/2183023332 <== mon.2 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb840013920 con 0x559b6358bba0 2024-09-11T00:23:40.600 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:23:40.594+0000 7fb846600640 1 -- 172.21.15.12:0/2183023332 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb840012790 con 0x559b6358bba0 2024-09-11T00:23:40.601 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:23:40.594+0000 7fb85e9fc740 1 -- 172.21.15.12:0/2183023332 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559b6358bba0 msgr2=0x559b6358df90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:23:40.601 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:23:40.594+0000 7fb859e00640 1 --2- 172.21.15.12:0/2679511716 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fb83c0b5390 0x7fb83c0b7850 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fb85406c220 tx=0x7fb84c009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:23:40.601 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:23:40.594+0000 7fb85e9fc740 1 --2- 172.21.15.12:0/2183023332 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559b6358bba0 0x559b6358df90 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fb840006fd0 tx=0x7fb84000e5a0 comp rx=0 tx=0).stop 2024-09-11T00:23:40.601 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:23:40.594+0000 7fb85e9fc740 1 -- 172.21.15.12:0/2183023332 shutdown_connections 2024-09-11T00:23:40.601 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:23:40.594+0000 7fb85e9fc740 1 --2- 172.21.15.12:0/2183023332 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559b6358bba0 0x559b6358df90 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:40.601 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:23:40.594+0000 7fb85e9fc740 1 --2- 172.21.15.12:0/2183023332 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559b6357fc10 0x559b6358b660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:40.601 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:23:40.594+0000 7fb85e9fc740 1 --2- 172.21.15.12:0/2183023332 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559b6357f2f0 0x559b6357f6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:40.601 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:23:40.594+0000 7fb85e9fc740 1 -- 172.21.15.12:0/2183023332 >> 172.21.15.12:0/2183023332 conn(0x559b6357d120 msgr2=0x559b6357e2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:23:40.601 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:23:40.594+0000 7fb85e9fc740 1 -- 172.21.15.12:0/2183023332 shutdown_connections 2024-09-11T00:23:40.601 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:23:40.594+0000 7fb85e9fc740 1 -- 172.21.15.12:0/2183023332 wait complete. 2024-09-11T00:23:40.601 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:23:40.599+0000 7fb85e9fc740 1 -- start start 2024-09-11T00:23:40.601 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:23:40.599+0000 7fb85e9fc740 1 Processor -- start 2024-09-11T00:23:40.602 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:23:40.599+0000 7fb85e9fc740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559b6357f2f0 0x559b6358b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:23:40.602 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:23:40.599+0000 7fb85e9fc740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559b6357fc10 0x559b63588910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:23:40.603 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:23:40.599+0000 7fb85e9fc740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559b6358bba0 0x559b63588f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:23:40.603 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:23:40.599+0000 7fb85e9fc740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x559b633457a0 con 0x559b6357f2f0 2024-09-11T00:23:40.603 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:23:40.599+0000 7fb85e9fc740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x559b63489a10 con 0x559b6357fc10 2024-09-11T00:23:40.603 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:23:40.599+0000 7fb85e9fc740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x559b63597030 con 0x559b6358bba0 2024-09-11T00:23:40.603 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:23:40.599+0000 7fb85e9fc740 10 client.0 _ll_register_callbacks cb 0x559b63499be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:23:40.603 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:23:40.599+0000 7fb859e00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559b6358bba0 0x559b63588f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:23:40.603 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:23:40.599+0000 7fb858a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559b6357fc10 0x559b63588910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:23:40.603 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:23:40.599+0000 7fb858a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559b6357fc10 0x559b63588910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.12:56036/0 (socket says 172.21.15.12:56036) 2024-09-11T00:23:40.603 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:23:40.599+0000 7fb858a00640 1 -- 172.21.15.12:0/1768402215 learned_addr learned my addr 172.21.15.12:0/1768402215 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:23:40.603 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:23:40.599+0000 7fb859400640 1 --2- 172.21.15.12:0/1768402215 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559b6357f2f0 0x559b6358b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:23:40.603 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:23:40.599+0000 7fb859400640 1 -- 172.21.15.12:0/1768402215 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559b6358bba0 msgr2=0x559b63588f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:23:40.603 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:23:40.599+0000 7fb859400640 1 --2- 172.21.15.12:0/1768402215 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559b6358bba0 0x559b63588f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:40.603 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:23:40.599+0000 7fb859400640 1 -- 172.21.15.12:0/1768402215 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559b6357fc10 msgr2=0x559b63588910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:23:40.604 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:23:40.599+0000 7fb859400640 1 --2- 172.21.15.12:0/1768402215 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559b6357fc10 0x559b63588910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:40.604 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:23:40.599+0000 7fb859400640 1 -- 172.21.15.12:0/1768402215 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559b63251b90 con 0x559b6357f2f0 2024-09-11T00:23:40.604 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:23:40.599+0000 7fb858a00640 1 --2- 172.21.15.12:0/1768402215 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559b6357fc10 0x559b63588910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:23:40.604 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:23:40.599+0000 7fb859400640 1 --2- 172.21.15.12:0/1768402215 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559b6357f2f0 0x559b6358b4c0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7fb8480c9360 tx=0x7fb8480092f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7f970d912dd17c0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:23:40.604 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:23:40.599+0000 7fb83aa00640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-09-11T00:23:40.604 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:23:40.599+0000 7fb83aa00640 1 -- 172.21.15.12:0/1768402215 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb848049030 con 0x559b6357f2f0 2024-09-11T00:23:40.604 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:23:40.599+0000 7fb83aa00640 1 -- 172.21.15.12:0/1768402215 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9208+0+0 (secure 0 0 0) 0x7fb8480d1560 con 0x559b6357f2f0 2024-09-11T00:23:40.604 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:23:40.599+0000 7fb85e9fc740 1 -- 172.21.15.12:0/1768402215 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb840013920 con 0x559b6357f2f0 2024-09-11T00:23:40.605 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:23:40.599+0000 7fb83aa00640 1 -- 172.21.15.12:0/1768402215 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=64) ==== 24+0+0 (secure 0 0 0) 0x7fb8480c93e0 con 0x559b6357f2f0 2024-09-11T00:23:40.605 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:23:40.599+0000 7fb85e9fc740 10 client.17322 fetch_fsmap learned FSMap version 64 2024-09-11T00:23:40.605 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:23:40.599+0000 7fb85e9fc740 1 -- 172.21.15.12:0/1768402215 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=64,monmap=4+}) -- 0x559b635a2990 con 0x559b6357f2f0 2024-09-11T00:23:40.605 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:23:40.599+0000 7fb83aa00640 1 -- 172.21.15.12:0/1768402215 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb840013920 con 0x559b6357f2f0 2024-09-11T00:23:40.606 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:23:40.599+0000 7fb83aa00640 1 -- 172.21.15.12:0/1768402215 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 64) ==== 8382+0+0 (secure 0 0 0) 0x7fb8480b40e0 con 0x559b6357f2f0 2024-09-11T00:23:40.606 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:23:40.599+0000 7fb85e9fc740 10 client.17322 fetch_fsmap finished waiting for FSMap version 64 2024-09-11T00:23:40.606 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:23:40.599+0000 7fb85e9fc740 10 client.17322 resolve_mds: validated gid 24674 aka daemon mds.b 2024-09-11T00:23:40.606 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:23:40.599+0000 7fb85e9fc740 20 client.17322 populate_metadata read hostname 'smithi012' 2024-09-11T00:23:40.606 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:23:40.599+0000 7fb85e9fc740 1 --2- 172.21.15.12:0/1768402215 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x559b635a3e90 0x559b635a6350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:23:40.606 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:23:40.599+0000 7fb85e9fc740 4 client.17322 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T00:23:40.606 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:23:40.599+0000 7fb85e9fc740 1 -- 172.21.15.12:0/1768402215 --> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559b634f6bf0 con 0x559b635a3e90 2024-09-11T00:23:40.606 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:23:40.599+0000 7fb858a00640 1 --2- 172.21.15.12:0/1768402215 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x559b635a3e90 0x559b635a6350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:23:40.606 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:23:40.604+0000 7fb858a00640 1 --2- 172.21.15.12:0/1768402215 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x559b635a3e90 0x559b635a6350 crc :-1 s=READY pgs=237 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b1e901c1a9434a1d server_cookie=e4f6687846b357ae in_seq=0 out_seq=0 2024-09-11T00:23:40.606 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:23:40.604+0000 7fb83aa00640 10 client.17322 ms_handle_connect on v2:172.21.15.165:6838/501312984 2024-09-11T00:23:40.607 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:23:40.604+0000 7fb83aa00640 1 -- 172.21.15.12:0/1768402215 <== mds.0 v2:172.21.15.165:6838/501312984 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x559b634f6bf0 con 0x559b635a3e90 2024-09-11T00:23:40.607 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:23:40.604+0000 7fb83aa00640 10 client.17322 handle_command_reply: tid=0 2024-09-11T00:23:40.607 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:23:40.604+0000 7fb85e9fc740 1 client.17322 shutdown 2024-09-11T00:23:40.607 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:23:40.604+0000 7fb85e9fc740 1 -- 172.21.15.12:0/1768402215 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559b6357f2f0 msgr2=0x559b6358b4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:23:40.607 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:23:40.604+0000 7fb85e9fc740 1 --2- 172.21.15.12:0/1768402215 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559b6357f2f0 0x559b6358b4c0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7fb8480c9360 tx=0x7fb8480092f0 comp rx=0 tx=0).stop 2024-09-11T00:23:40.608 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:23:40.604+0000 7fb85e9fc740 1 -- 172.21.15.12:0/1768402215 shutdown_connections 2024-09-11T00:23:40.608 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:23:40.604+0000 7fb85e9fc740 1 --2- 172.21.15.12:0/1768402215 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x559b635a3e90 0x559b635a6350 crc :-1 s=READY pgs=237 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:40.608 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:23:40.604+0000 7fb85e9fc740 1 --2- 172.21.15.12:0/1768402215 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559b6358bba0 0x559b63588f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:40.608 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:23:40.604+0000 7fb85e9fc740 1 --2- 172.21.15.12:0/1768402215 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559b6357fc10 0x559b63588910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:40.608 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:23:40.604+0000 7fb85e9fc740 1 --2- 172.21.15.12:0/1768402215 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559b6357f2f0 0x559b6358b4c0 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:40.608 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:23:40.604+0000 7fb85e9fc740 1 -- 172.21.15.12:0/1768402215 >> 172.21.15.12:0/1768402215 conn(0x559b6357d120 msgr2=0x559b63582cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:23:40.608 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:23:40.604+0000 7fb83aa00640 0 client.17322 ms_handle_reset on v2:172.21.15.165:6838/501312984 2024-09-11T00:23:40.609 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:23:40.604+0000 7fb85e9fc740 1 -- 172.21.15.12:0/1768402215 shutdown_connections 2024-09-11T00:23:40.609 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:23:40.604+0000 7fb85e9fc740 1 -- 172.21.15.12:0/1768402215 wait complete. 2024-09-11T00:23:40.609 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:23:40.604+0000 7fb85e9fc740 20 client.17322 trim_cache size 0 max 16384 2024-09-11T00:23:40.651 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:23:40.648+0000 7fb85e9fc740 1 Processor -- start 2024-09-11T00:23:40.651 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:23:40.648+0000 7fb85e9fc740 1 -- start start 2024-09-11T00:23:40.651 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:23:40.648+0000 7fb85e9fc740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559b6357f2f0 0x559b63587190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:23:40.652 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:23:40.648+0000 7fb85e9fc740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559b6357fc10 0x559b635876d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:23:40.652 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:23:40.648+0000 7fb85e9fc740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559b6358bba0 0x559b6358cf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:23:40.652 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:23:40.648+0000 7fb85e9fc740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fb8541ad800 con 0x559b6357f2f0 2024-09-11T00:23:40.652 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:23:40.648+0000 7fb85e9fc740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fb8541af3a0 con 0x559b6358bba0 2024-09-11T00:23:40.652 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:23:40.648+0000 7fb85e9fc740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fb8541adf00 con 0x559b6357fc10 2024-09-11T00:23:40.653 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:23:40.648+0000 7fb859e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559b6358bba0 0x559b6358cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:23:40.653 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:23:40.648+0000 7fb859400640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559b6357f2f0 0x559b63587190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:23:40.653 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:23:40.648+0000 7fb859400640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559b6357f2f0 0x559b63587190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:41816/0 (socket says 172.21.15.12:41816) 2024-09-11T00:23:40.653 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:23:40.648+0000 7fb859400640 1 -- 172.21.15.12:0/217766848 learned_addr learned my addr 172.21.15.12:0/217766848 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:23:40.653 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:23:40.648+0000 7fb858a00640 1 --2- 172.21.15.12:0/217766848 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559b6357fc10 0x559b635876d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:23:40.653 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:23:40.648+0000 7fb858a00640 1 -- 172.21.15.12:0/217766848 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559b6358bba0 msgr2=0x559b6358cf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:23:40.654 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:23:40.648+0000 7fb858a00640 1 --2- 172.21.15.12:0/217766848 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559b6358bba0 0x559b6358cf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:40.654 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:23:40.648+0000 7fb858a00640 1 -- 172.21.15.12:0/217766848 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559b6357f2f0 msgr2=0x559b63587190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:23:40.654 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:23:40.648+0000 7fb858a00640 1 --2- 172.21.15.12:0/217766848 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559b6357f2f0 0x559b63587190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:40.654 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:23:40.648+0000 7fb858a00640 1 -- 172.21.15.12:0/217766848 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559b635892b0 con 0x559b6357fc10 2024-09-11T00:23:40.654 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:23:40.648+0000 7fb858a00640 1 --2- 172.21.15.12:0/217766848 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559b6357fc10 0x559b635876d0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x559b635a6b60 tx=0x7fb840009890 comp rx=0 tx=0).ready entity=mon.1 client_cookie=752110aec71a1ffa server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:23:40.659 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:23:40.648+0000 7fb845c00640 1 -- 172.21.15.12:0/217766848 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb840017040 con 0x559b6357fc10 2024-09-11T00:23:40.659 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:23:40.648+0000 7fb845c00640 1 -- 172.21.15.12:0/217766848 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb840004500 con 0x559b6357fc10 2024-09-11T00:23:40.659 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:23:40.648+0000 7fb845c00640 1 -- 172.21.15.12:0/217766848 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb840003a00 con 0x559b6357fc10 2024-09-11T00:23:40.659 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:23:40.654+0000 7fb85e9fc740 1 -- 172.21.15.12:0/217766848 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559b6357fc10 msgr2=0x559b635876d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:23:40.659 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:23:40.654+0000 7fb85e9fc740 1 --2- 172.21.15.12:0/217766848 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559b6357fc10 0x559b635876d0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x559b635a6b60 tx=0x7fb840009890 comp rx=0 tx=0).stop 2024-09-11T00:23:40.659 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:23:40.654+0000 7fb85e9fc740 1 -- 172.21.15.12:0/217766848 shutdown_connections 2024-09-11T00:23:40.659 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:23:40.654+0000 7fb85e9fc740 1 --2- 172.21.15.12:0/217766848 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559b6358bba0 0x559b6358cf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:40.659 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:23:40.654+0000 7fb85e9fc740 1 --2- 172.21.15.12:0/217766848 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559b6357fc10 0x559b635876d0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:40.660 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:23:40.654+0000 7fb85e9fc740 1 --2- 172.21.15.12:0/217766848 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559b6357f2f0 0x559b63587190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:40.660 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:23:40.654+0000 7fb85e9fc740 1 -- 172.21.15.12:0/217766848 >> 172.21.15.12:0/217766848 conn(0x559b6357d120 msgr2=0x559b63583bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:23:40.660 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:23:40.654+0000 7fb85e9fc740 1 -- 172.21.15.12:0/217766848 shutdown_connections 2024-09-11T00:23:40.660 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:23:40.654+0000 7fb85e9fc740 1 -- 172.21.15.12:0/217766848 wait complete. 2024-09-11T00:23:40.660 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:23:40.654+0000 7fb85e9fc740 1 -- start start 2024-09-11T00:23:40.660 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:23:40.654+0000 7fb85e9fc740 1 Processor -- start 2024-09-11T00:23:40.660 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:23:40.654+0000 7fb85e9fc740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559b6357f2f0 0x559b6359c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:23:40.660 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:23:40.654+0000 7fb85e9fc740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559b6357fc10 0x559b6359cc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:23:40.660 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:23:40.654+0000 7fb85e9fc740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559b6358bba0 0x559b63597430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:23:40.660 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:23:40.654+0000 7fb85e9fc740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fb8541aecd0 con 0x559b6357fc10 2024-09-11T00:23:40.660 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:23:40.654+0000 7fb85e9fc740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fb8541ae5d0 con 0x559b6358bba0 2024-09-11T00:23:40.660 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:23:40.654+0000 7fb85e9fc740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fb8541ab820 con 0x559b6357f2f0 2024-09-11T00:23:40.660 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:23:40.654+0000 7fb85e9fc740 10 client.0 _ll_register_callbacks cb 0x7fb84810c220 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:23:40.660 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:23:40.654+0000 7fb859e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559b6358bba0 0x559b63597430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:23:40.661 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:23:40.654+0000 7fb859e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559b6358bba0 0x559b63597430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.12:56044/0 (socket says 172.21.15.12:56044) 2024-09-11T00:23:40.661 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:23:40.654+0000 7fb859e00640 1 -- 172.21.15.12:0/1745485229 learned_addr learned my addr 172.21.15.12:0/1745485229 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:23:40.661 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:23:40.657+0000 7fb859400640 1 --2- 172.21.15.12:0/1745485229 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559b6357f2f0 0x559b6359c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:23:40.662 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:23:40.657+0000 7fb858a00640 1 --2- 172.21.15.12:0/1745485229 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559b6357fc10 0x559b6359cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:23:40.662 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:23:40.657+0000 7fb859400640 1 -- 172.21.15.12:0/1745485229 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559b6358bba0 msgr2=0x559b63597430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:23:40.662 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:23:40.657+0000 7fb859400640 1 --2- 172.21.15.12:0/1745485229 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559b6358bba0 0x559b63597430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:40.662 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:23:40.657+0000 7fb859400640 1 -- 172.21.15.12:0/1745485229 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559b6357fc10 msgr2=0x559b6359cc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:23:40.662 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:23:40.657+0000 7fb859400640 1 --2- 172.21.15.12:0/1745485229 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559b6357fc10 0x559b6359cc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:40.662 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:23:40.657+0000 7fb859400640 1 -- 172.21.15.12:0/1745485229 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559b6358b830 con 0x559b6357f2f0 2024-09-11T00:23:40.662 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:23:40.657+0000 7fb858a00640 1 --2- 172.21.15.12:0/1745485229 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559b6357fc10 0x559b6359cc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T00:23:40.671 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:23:40.669+0000 7fb859400640 1 --2- 172.21.15.12:0/1745485229 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559b6357f2f0 0x559b6359c6f0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fb848106dc0 tx=0x7fb84810c4b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=db7137345dfd4442 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:23:40.671 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:23:40.669+0000 7fb83aa00640 10 client.0 ms_handle_connect on v2:172.21.15.178:3300/0 2024-09-11T00:23:40.672 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:23:40.669+0000 7fb83aa00640 1 -- 172.21.15.12:0/1745485229 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb848049070 con 0x559b6357f2f0 2024-09-11T00:23:40.672 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:23:40.669+0000 7fb83aa00640 1 -- 172.21.15.12:0/1745485229 <== mon.1 v2:172.21.15.178:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9208+0+0 (secure 0 0 0) 0x7fb8480d5e90 con 0x559b6357f2f0 2024-09-11T00:23:40.672 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:23:40.669+0000 7fb85e9fc740 1 -- 172.21.15.12:0/1745485229 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb840004500 con 0x559b6357f2f0 2024-09-11T00:23:40.672 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:23:40.669+0000 7fb83aa00640 1 -- 172.21.15.12:0/1745485229 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=64) ==== 24+0+0 (secure 0 0 0) 0x7fb8480cc500 con 0x559b6357f2f0 2024-09-11T00:23:40.673 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:23:40.669+0000 7fb85e9fc740 10 client.27301 fetch_fsmap learned FSMap version 64 2024-09-11T00:23:40.673 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:23:40.669+0000 7fb85e9fc740 1 -- 172.21.15.12:0/1745485229 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,fsmap=64,monmap=4+}) -- 0x559b63611900 con 0x559b6357f2f0 2024-09-11T00:23:40.673 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:23:40.669+0000 7fb83aa00640 1 -- 172.21.15.12:0/1745485229 <== mon.1 v2:172.21.15.178:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb840004500 con 0x559b6357f2f0 2024-09-11T00:23:40.673 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:23:40.669+0000 7fb83aa00640 1 -- 172.21.15.12:0/1745485229 <== mon.1 v2:172.21.15.178:3300/0 5 ==== fsmap(e 64) ==== 8382+0+0 (secure 0 0 0) 0x7fb8480b2bb0 con 0x559b6357f2f0 2024-09-11T00:23:40.673 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:23:40.669+0000 7fb85e9fc740 10 client.27301 fetch_fsmap finished waiting for FSMap version 64 2024-09-11T00:23:40.673 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:23:40.669+0000 7fb85e9fc740 10 client.27301 resolve_mds: validated gid 24674 aka daemon mds.b 2024-09-11T00:23:40.674 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:23:40.669+0000 7fb85e9fc740 20 client.27301 populate_metadata read hostname 'smithi012' 2024-09-11T00:23:40.674 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:23:40.669+0000 7fb85e9fc740 1 --2- 172.21.15.12:0/1745485229 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x559b63611f60 0x559b63614380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:23:40.674 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:23:40.669+0000 7fb85e9fc740 4 client.27301 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 154.8, "await_for": 45.0}] 2024-09-11T00:23:40.674 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:23:40.669+0000 7fb85e9fc740 1 -- 172.21.15.12:0/1745485229 --> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 154.8, "await_for": 45.0}) -- 0x559b634f26a0 con 0x559b63611f60 2024-09-11T00:23:40.674 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:23:40.669+0000 7fb858a00640 1 --2- 172.21.15.12:0/1745485229 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x559b63611f60 0x559b63614380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:23:40.675 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:23:40.669+0000 7fb83aa00640 10 client.27301 ms_handle_connect on v2:172.21.15.165:6838/501312984 2024-09-11T00:23:40.676 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:23:40.675+0000 7fb858a00640 1 --2- 172.21.15.12:0/1745485229 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x559b63611f60 0x559b63614380 crc :-1 s=READY pgs=238 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=97a1c8679e534b7f server_cookie=d1cac032e4f90f6d in_seq=0 out_seq=0 2024-09-11T00:23:41.176 INFO:tasks.workunit.client.0.smithi012.stdout: 10 69389 20.21 MB/sec execute 97 sec latency 22415.499 ms 2024-09-11T00:23:41.574 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:23:41 smithi012 ceph-mon[30333]: pgmap v897: 129 pgs: 129 active+clean; 511 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 169 KiB/s rd, 168 KiB/s wr, 73 op/s 2024-09-11T00:23:41.831 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:23:41 smithi178 ceph-mon[35919]: pgmap v897: 129 pgs: 129 active+clean; 511 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 169 KiB/s rd, 168 KiB/s wr, 73 op/s 2024-09-11T00:23:41.946 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:23:41.944+0000 7fb83aa00640 1 -- 172.21.15.12:0/1745485229 <== mds.0 v2:172.21.15.165:6838/501312984 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x559b634f26a0 con 0x559b63611f60 2024-09-11T00:23:41.946 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:23:41.944+0000 7fb83aa00640 10 client.27301 handle_command_reply: tid=0 2024-09-11T00:23:41.946 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:23:41.944+0000 7fb85e9fc740 1 client.27301 shutdown 2024-09-11T00:23:41.946 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:23:41.944+0000 7fb85e9fc740 1 -- 172.21.15.12:0/1745485229 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559b6357f2f0 msgr2=0x559b6359c6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:23:41.946 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:23:41.944+0000 7fb85e9fc740 1 --2- 172.21.15.12:0/1745485229 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559b6357f2f0 0x559b6359c6f0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fb848106dc0 tx=0x7fb84810c4b0 comp rx=0 tx=0).stop 2024-09-11T00:23:41.946 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:23:41.944+0000 7fb85e9fc740 1 -- 172.21.15.12:0/1745485229 shutdown_connections 2024-09-11T00:23:41.946 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:23:41.944+0000 7fb85e9fc740 1 --2- 172.21.15.12:0/1745485229 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x559b63611f60 0x559b63614380 crc :-1 s=READY pgs=238 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:41.946 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:23:41.944+0000 7fb85e9fc740 1 --2- 172.21.15.12:0/1745485229 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559b6358bba0 0x559b63597430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:41.946 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:23:41.944+0000 7fb83aa00640 0 client.27301 ms_handle_reset on v2:172.21.15.165:6838/501312984 2024-09-11T00:23:41.946 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:23:41.944+0000 7fb85e9fc740 1 --2- 172.21.15.12:0/1745485229 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559b6357fc10 0x559b6359cc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:41.947 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:23:41.944+0000 7fb85e9fc740 1 --2- 172.21.15.12:0/1745485229 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559b6357f2f0 0x559b6359c6f0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:41.947 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:23:41.944+0000 7fb85e9fc740 1 -- 172.21.15.12:0/1745485229 >> 172.21.15.12:0/1745485229 conn(0x559b6357d120 msgr2=0x559b63587150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:23:41.947 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:23:41.945+0000 7fb85e9fc740 1 -- 172.21.15.12:0/1745485229 shutdown_connections 2024-09-11T00:23:41.947 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:23:41.945+0000 7fb85e9fc740 1 -- 172.21.15.12:0/1745485229 wait complete. 2024-09-11T00:23:41.947 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:23:41.945+0000 7fb85e9fc740 20 client.27301 trim_cache size 0 max 16384 2024-09-11T00:23:41.947 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-09-11T00:23:41.948 INFO:teuthology.orchestra.run.smithi012.stdout: "epoch": 64, 2024-09-11T00:23:41.948 INFO:teuthology.orchestra.run.smithi012.stdout: "leader": 24674, 2024-09-11T00:23:41.948 INFO:teuthology.orchestra.run.smithi012.stdout: "set_version": 50, 2024-09-11T00:23:41.948 INFO:teuthology.orchestra.run.smithi012.stdout: "sets": { 2024-09-11T00:23:41.948 INFO:teuthology.orchestra.run.smithi012.stdout: "49e32b0e": { 2024-09-11T00:23:41.948 INFO:teuthology.orchestra.run.smithi012.stdout: "version": 50, 2024-09-11T00:23:41.948 INFO:teuthology.orchestra.run.smithi012.stdout: "age_ref": 0.0, 2024-09-11T00:23:41.948 INFO:teuthology.orchestra.run.smithi012.stdout: "state": { 2024-09-11T00:23:41.948 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "QUIESCED", 2024-09-11T00:23:41.948 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 0.0 2024-09-11T00:23:41.948 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:23:41.948 INFO:teuthology.orchestra.run.smithi012.stdout: "timeout": 90.0, 2024-09-11T00:23:41.949 INFO:teuthology.orchestra.run.smithi012.stdout: "expiration": 154.8, 2024-09-11T00:23:41.949 INFO:teuthology.orchestra.run.smithi012.stdout: "members": { 2024-09-11T00:23:41.949 INFO:teuthology.orchestra.run.smithi012.stdout: "file:/": { 2024-09-11T00:23:41.949 INFO:teuthology.orchestra.run.smithi012.stdout: "excluded": false, 2024-09-11T00:23:41.949 INFO:teuthology.orchestra.run.smithi012.stdout: "state": { 2024-09-11T00:23:41.949 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "QUIESCED", 2024-09-11T00:23:41.949 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 0.0 2024-09-11T00:23:41.949 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:23:41.949 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:23:41.949 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:23:41.949 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:23:41.949 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:23:41.950 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-09-11T00:23:41.950 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:23:41.946+0000 7fb85a800640 1 -- 172.21.15.12:0/2679511716 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fb83c0b5390 msgr2=0x7fb83c0b7850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:23:41.950 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:23:41.946+0000 7fb85a800640 1 --2- 172.21.15.12:0/2679511716 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fb83c0b5390 0x7fb83c0b7850 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fb85406c220 tx=0x7fb84c009500 comp rx=0 tx=0).stop 2024-09-11T00:23:41.950 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:23:41.946+0000 7fb85a800640 1 -- 172.21.15.12:0/2679511716 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb8540754c0 msgr2=0x7fb85419b2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:23:41.950 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:23:41.946+0000 7fb85a800640 1 --2- 172.21.15.12:0/2679511716 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb8540754c0 0x7fb85419b2d0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fb848005940 tx=0x7fb848004270 comp rx=0 tx=0).stop 2024-09-11T00:23:41.950 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:23:41.946+0000 7fb85a800640 1 -- 172.21.15.12:0/2679511716 shutdown_connections 2024-09-11T00:23:41.950 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:23:41.946+0000 7fb85a800640 1 --2- 172.21.15.12:0/2679511716 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fb83c0b5390 0x7fb83c0b7850 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:41.951 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:23:41.946+0000 7fb85a800640 1 --2- 172.21.15.12:0/2679511716 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb85410f220 0x7fb8541a0bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:41.951 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:23:41.946+0000 7fb85a800640 1 --2- 172.21.15.12:0/2679511716 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb854075de0 0x7fb85419b810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:41.951 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:23:41.946+0000 7fb85a800640 1 --2- 172.21.15.12:0/2679511716 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb8540754c0 0x7fb85419b2d0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:41.951 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:23:41.946+0000 7fb85a800640 1 -- 172.21.15.12:0/2679511716 >> 172.21.15.12:0/2679511716 conn(0x7fb8540fe660 msgr2=0x7fb854111390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:23:41.951 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:23:41.946+0000 7fb85a800640 1 -- 172.21.15.12:0/2679511716 shutdown_connections 2024-09-11T00:23:41.951 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:23:41.946+0000 7fb85a800640 1 -- 172.21.15.12:0/2679511716 wait complete. 2024-09-11T00:23:42.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:23:41 smithi165 ceph-mon[35356]: pgmap v897: 129 pgs: 129 active+clean; 511 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 169 KiB/s rd, 168 KiB/s wr, 73 op/s 2024-09-11T00:23:42.176 INFO:tasks.workunit.client.0.smithi012.stdout: 10 69399 20.01 MB/sec execute 98 sec latency 23015.758 ms 2024-09-11T00:23:42.857 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '49e32b0e', quiesce took 10.8 seconds. Will release after: 23.999999999999996 2024-09-11T00:23:43.176 INFO:tasks.workunit.client.0.smithi012.stdout: 10 69399 19.80 MB/sec execute 99 sec latency 2499.146 ms 2024-09-11T00:23:43.831 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:23:43 smithi178 ceph-mon[35919]: pgmap v898: 129 pgs: 129 active+clean; 509 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 336 KiB/s rd, 385 KiB/s wr, 95 op/s 2024-09-11T00:23:44.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:23:43 smithi165 ceph-mon[35356]: pgmap v898: 129 pgs: 129 active+clean; 509 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 336 KiB/s rd, 385 KiB/s wr, 95 op/s 2024-09-11T00:23:44.064 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:23:43 smithi012 ceph-mon[30333]: pgmap v898: 129 pgs: 129 active+clean; 509 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 336 KiB/s rd, 385 KiB/s wr, 95 op/s 2024-09-11T00:23:44.176 INFO:tasks.workunit.client.0.smithi012.stdout: 10 69399 19.61 MB/sec execute 100 sec latency 3499.285 ms 2024-09-11T00:23:45.176 INFO:tasks.workunit.client.0.smithi012.stdout: 10 69399 19.41 MB/sec execute 101 sec latency 4499.425 ms 2024-09-11T00:23:45.831 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:23:45 smithi178 ceph-mon[35919]: pgmap v899: 129 pgs: 129 active+clean; 507 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 243 KiB/s rd, 299 KiB/s wr, 82 op/s 2024-09-11T00:23:45.831 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:23:45 smithi178 ceph-mon[35919]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T00:23:46.033 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:23:45 smithi165 ceph-mon[35356]: pgmap v899: 129 pgs: 129 active+clean; 507 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 243 KiB/s rd, 299 KiB/s wr, 82 op/s 2024-09-11T00:23:46.033 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:23:45 smithi165 ceph-mon[35356]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T00:23:46.063 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:23:45 smithi012 ceph-mon[30333]: pgmap v899: 129 pgs: 129 active+clean; 507 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 243 KiB/s rd, 299 KiB/s wr, 82 op/s 2024-09-11T00:23:46.063 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:23:45 smithi012 ceph-mon[30333]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T00:23:46.176 INFO:tasks.workunit.client.0.smithi012.stdout: 10 69399 19.22 MB/sec execute 102 sec latency 5499.567 ms 2024-09-11T00:23:47.176 INFO:tasks.workunit.client.0.smithi012.stdout: 10 69399 19.03 MB/sec execute 103 sec latency 6499.693 ms 2024-09-11T00:23:47.831 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:23:47 smithi178 ceph-mon[35919]: pgmap v900: 129 pgs: 129 active+clean; 496 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 622 KiB/s rd, 466 KiB/s wr, 139 op/s 2024-09-11T00:23:47.831 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:23:47 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T00:23:47.834 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:23:47 smithi165 ceph-mon[35356]: pgmap v900: 129 pgs: 129 active+clean; 496 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 622 KiB/s rd, 466 KiB/s wr, 139 op/s 2024-09-11T00:23:47.834 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:23:47 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T00:23:47.866 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:23:47 smithi012 ceph-mon[30333]: pgmap v900: 129 pgs: 129 active+clean; 496 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 622 KiB/s rd, 466 KiB/s wr, 139 op/s 2024-09-11T00:23:47.866 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:23:47 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T00:23:48.176 INFO:tasks.workunit.client.0.smithi012.stdout: 10 69399 18.85 MB/sec execute 104 sec latency 7499.775 ms 2024-09-11T00:23:48.972 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:23:48 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-11T00:23:48.972 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:23:48 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-11T00:23:48.972 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:23:48 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"} : dispatch 2024-09-11T00:23:48.972 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:23:48 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"} : dispatch 2024-09-11T00:23:48.972 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:23:48 smithi012 ceph-mon[30333]: Adjusting osd_memory_target on smithi178 to 176.0M 2024-09-11T00:23:48.972 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:23:48 smithi012 ceph-mon[30333]: Unable to set osd_memory_target on smithi178 to 184560230: error parsing value: Value '184560230' is below minimum 939524096 2024-09-11T00:23:48.972 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:23:48 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-11T00:23:48.972 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:23:48 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-11T00:23:48.973 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:23:48 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-09-11T00:23:48.973 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:23:48 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"} : dispatch 2024-09-11T00:23:48.973 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:23:48 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-11T00:23:48.973 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:23:48 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-11T00:23:48.973 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:23:48 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-09-11T00:23:48.973 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:23:48 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"} : dispatch 2024-09-11T00:23:48.973 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:23:48 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:23:48.973 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:23:48 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T00:23:48.973 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:23:48 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:23:49.033 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:23:48 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-11T00:23:49.033 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:23:48 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-11T00:23:49.033 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:23:48 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"} : dispatch 2024-09-11T00:23:49.033 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:23:48 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"} : dispatch 2024-09-11T00:23:49.033 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:23:48 smithi165 ceph-mon[35356]: Adjusting osd_memory_target on smithi178 to 176.0M 2024-09-11T00:23:49.033 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:23:48 smithi165 ceph-mon[35356]: Unable to set osd_memory_target on smithi178 to 184560230: error parsing value: Value '184560230' is below minimum 939524096 2024-09-11T00:23:49.033 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:23:48 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-11T00:23:49.034 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:23:48 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-11T00:23:49.034 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:23:48 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-09-11T00:23:49.034 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:23:48 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"} : dispatch 2024-09-11T00:23:49.034 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:23:48 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-11T00:23:49.034 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:23:48 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-11T00:23:49.034 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:23:48 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-09-11T00:23:49.034 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:23:48 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"} : dispatch 2024-09-11T00:23:49.034 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:23:48 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:23:49.034 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:23:48 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T00:23:49.034 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:23:48 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:23:49.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:23:48 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-11T00:23:49.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:23:48 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-11T00:23:49.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:23:48 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"} : dispatch 2024-09-11T00:23:49.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:23:48 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"} : dispatch 2024-09-11T00:23:49.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:23:48 smithi178 ceph-mon[35919]: Adjusting osd_memory_target on smithi178 to 176.0M 2024-09-11T00:23:49.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:23:48 smithi178 ceph-mon[35919]: Unable to set osd_memory_target on smithi178 to 184560230: error parsing value: Value '184560230' is below minimum 939524096 2024-09-11T00:23:49.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:23:48 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-11T00:23:49.082 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:23:48 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-11T00:23:49.082 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:23:48 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-09-11T00:23:49.082 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:23:48 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"} : dispatch 2024-09-11T00:23:49.082 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:23:48 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-11T00:23:49.082 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:23:48 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-11T00:23:49.082 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:23:48 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-09-11T00:23:49.082 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:23:48 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"} : dispatch 2024-09-11T00:23:49.082 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:23:48 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:23:49.082 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:23:48 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T00:23:49.082 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:23:48 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:23:49.177 INFO:tasks.workunit.client.0.smithi012.stdout: 10 69399 18.67 MB/sec execute 105 sec latency 8499.836 ms 2024-09-11T00:23:49.847 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:23:49 smithi012 ceph-mon[30333]: Adjusting osd_memory_target on smithi165 to 176.0M 2024-09-11T00:23:49.847 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:23:49 smithi012 ceph-mon[30333]: Unable to set osd_memory_target on smithi165 to 184589619: error parsing value: Value '184589619' is below minimum 939524096 2024-09-11T00:23:49.847 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:23:49 smithi012 ceph-mon[30333]: Adjusting osd_memory_target on smithi012 to 177.5M 2024-09-11T00:23:49.847 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:23:49 smithi012 ceph-mon[30333]: Unable to set osd_memory_target on smithi012 to 186147225: error parsing value: Value '186147225' is below minimum 939524096 2024-09-11T00:23:49.847 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:23:49 smithi012 ceph-mon[30333]: pgmap v901: 129 pgs: 129 active+clean; 497 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 550 KiB/s rd, 438 KiB/s wr, 115 op/s 2024-09-11T00:23:50.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:23:49 smithi165 ceph-mon[35356]: Adjusting osd_memory_target on smithi165 to 176.0M 2024-09-11T00:23:50.033 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:23:49 smithi165 ceph-mon[35356]: Unable to set osd_memory_target on smithi165 to 184589619: error parsing value: Value '184589619' is below minimum 939524096 2024-09-11T00:23:50.033 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:23:49 smithi165 ceph-mon[35356]: Adjusting osd_memory_target on smithi012 to 177.5M 2024-09-11T00:23:50.033 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:23:49 smithi165 ceph-mon[35356]: Unable to set osd_memory_target on smithi012 to 186147225: error parsing value: Value '186147225' is below minimum 939524096 2024-09-11T00:23:50.033 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:23:49 smithi165 ceph-mon[35356]: pgmap v901: 129 pgs: 129 active+clean; 497 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 550 KiB/s rd, 438 KiB/s wr, 115 op/s 2024-09-11T00:23:50.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:23:49 smithi178 ceph-mon[35919]: Adjusting osd_memory_target on smithi165 to 176.0M 2024-09-11T00:23:50.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:23:49 smithi178 ceph-mon[35919]: Unable to set osd_memory_target on smithi165 to 184589619: error parsing value: Value '184589619' is below minimum 939524096 2024-09-11T00:23:50.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:23:49 smithi178 ceph-mon[35919]: Adjusting osd_memory_target on smithi012 to 177.5M 2024-09-11T00:23:50.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:23:49 smithi178 ceph-mon[35919]: Unable to set osd_memory_target on smithi012 to 186147225: error parsing value: Value '186147225' is below minimum 939524096 2024-09-11T00:23:50.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:23:49 smithi178 ceph-mon[35919]: pgmap v901: 129 pgs: 129 active+clean; 497 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 550 KiB/s rd, 438 KiB/s wr, 115 op/s 2024-09-11T00:23:50.177 INFO:tasks.workunit.client.0.smithi012.stdout: 10 69399 18.50 MB/sec execute 106 sec latency 9499.961 ms 2024-09-11T00:23:50.355 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:50.353+0000 7ffaf4600640 1 -- 172.21.15.12:0/2400307237 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ffaec1066a0 msgr2=0x7ffaec10d020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:23:50.355 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:50.353+0000 7ffaf4600640 1 --2- 172.21.15.12:0/2400307237 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ffaec1066a0 0x7ffaec10d020 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7ffae4009930 tx=0x7ffae402a420 comp rx=0 tx=0).stop 2024-09-11T00:23:50.355 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:50.353+0000 7ffaf4600640 1 -- 172.21.15.12:0/2400307237 shutdown_connections 2024-09-11T00:23:50.355 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:50.353+0000 7ffaf4600640 1 --2- 172.21.15.12:0/2400307237 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ffaec1066a0 0x7ffaec10d020 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:50.355 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:50.353+0000 7ffaf4600640 1 --2- 172.21.15.12:0/2400307237 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ffaec105ce0 0x7ffaec106160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:50.355 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:50.353+0000 7ffaf4600640 1 --2- 172.21.15.12:0/2400307237 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ffaec104b90 0x7ffaec104f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:50.355 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:50.353+0000 7ffaf4600640 1 -- 172.21.15.12:0/2400307237 >> 172.21.15.12:0/2400307237 conn(0x7ffaec100470 msgr2=0x7ffaec102890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:23:50.355 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:50.353+0000 7ffaf4600640 1 -- 172.21.15.12:0/2400307237 shutdown_connections 2024-09-11T00:23:50.355 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:50.353+0000 7ffaf4600640 1 -- 172.21.15.12:0/2400307237 wait complete. 2024-09-11T00:23:50.357 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:50.354+0000 7ffaf4600640 1 Processor -- start 2024-09-11T00:23:50.357 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:50.354+0000 7ffaf4600640 1 -- start start 2024-09-11T00:23:50.357 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:50.354+0000 7ffaf4600640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ffaec104b90 0x7ffaec19af30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:23:50.357 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:50.354+0000 7ffaf4600640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ffaec105ce0 0x7ffaec19b580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:23:50.358 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:50.354+0000 7ffaf4600640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ffaec1066a0 0x7ffaec19c1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:23:50.358 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:50.354+0000 7ffaf4600640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7ffaec110800 con 0x7ffaec1066a0 2024-09-11T00:23:50.358 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:50.354+0000 7ffaf4600640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7ffaec110680 con 0x7ffaec104b90 2024-09-11T00:23:50.358 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:50.354+0000 7ffaf4600640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7ffaec110980 con 0x7ffaec105ce0 2024-09-11T00:23:50.358 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:50.355+0000 7ffaf2600640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ffaec1066a0 0x7ffaec19c1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:23:50.358 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:50.355+0000 7ffaf2600640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ffaec1066a0 0x7ffaec19c1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:50314/0 (socket says 172.21.15.12:50314) 2024-09-11T00:23:50.358 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:50.355+0000 7ffaf2600640 1 -- 172.21.15.12:0/1817576800 learned_addr learned my addr 172.21.15.12:0/1817576800 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:23:50.358 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:50.355+0000 7ffaf1200640 1 --2- 172.21.15.12:0/1817576800 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ffaec105ce0 0x7ffaec19b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:23:50.358 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:50.355+0000 7ffaf1c00640 1 --2- 172.21.15.12:0/1817576800 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ffaec104b90 0x7ffaec19af30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:23:50.358 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:50.355+0000 7ffaf2600640 1 -- 172.21.15.12:0/1817576800 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ffaec105ce0 msgr2=0x7ffaec19b580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:23:50.359 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:50.355+0000 7ffaf2600640 1 --2- 172.21.15.12:0/1817576800 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ffaec105ce0 0x7ffaec19b580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:50.359 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:50.355+0000 7ffaf2600640 1 -- 172.21.15.12:0/1817576800 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ffaec104b90 msgr2=0x7ffaec19af30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:23:50.359 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:50.355+0000 7ffaf2600640 1 --2- 172.21.15.12:0/1817576800 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ffaec104b90 0x7ffaec19af30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:50.359 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:50.355+0000 7ffaf2600640 1 -- 172.21.15.12:0/1817576800 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffaec19ca70 con 0x7ffaec1066a0 2024-09-11T00:23:50.359 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:50.355+0000 7ffaf2600640 1 --2- 172.21.15.12:0/1817576800 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ffaec1066a0 0x7ffaec19c1b0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7ffae4031500 tx=0x7ffae40042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:23:50.359 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:50.356+0000 7ffaea000640 1 -- 172.21.15.12:0/1817576800 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffae403f030 con 0x7ffaec1066a0 2024-09-11T00:23:50.359 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:50.356+0000 7ffaea000640 1 -- 172.21.15.12:0/1817576800 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffae4003cb0 con 0x7ffaec1066a0 2024-09-11T00:23:50.359 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:50.356+0000 7ffaf4600640 1 -- 172.21.15.12:0/1817576800 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffaec19cd00 con 0x7ffaec1066a0 2024-09-11T00:23:50.359 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:50.356+0000 7ffaea000640 1 -- 172.21.15.12:0/1817576800 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffae4035070 con 0x7ffaec1066a0 2024-09-11T00:23:50.359 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:50.356+0000 7ffaf4600640 1 -- 172.21.15.12:0/1817576800 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffaec1a3430 con 0x7ffaec1066a0 2024-09-11T00:23:50.360 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:50.359+0000 7ffaf745b740 1 Processor -- start 2024-09-11T00:23:50.360 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:50.359+0000 7ffaea000640 1 -- 172.21.15.12:0/1817576800 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7ffae403bb70 con 0x7ffaec1066a0 2024-09-11T00:23:50.362 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:50.360+0000 7ffaf745b740 1 -- start start 2024-09-11T00:23:50.362 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:50.360+0000 7ffaf745b740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559dd4b6d790 0x559dd4b6fb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:23:50.362 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:50.360+0000 7ffaf745b740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559dd4b700c0 0x559dd4b724b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:23:50.362 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:50.360+0000 7ffaf745b740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559dd4b72b00 0x559dd4b74fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:23:50.362 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:50.360+0000 7ffaf745b740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x559dd4ae06b0 con 0x559dd4b72b00 2024-09-11T00:23:50.362 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:50.360+0000 7ffaf745b740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x559dd4acfae0 con 0x559dd4b6d790 2024-09-11T00:23:50.362 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:50.360+0000 7ffaf745b740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x559dd4ad38f0 con 0x559dd4b700c0 2024-09-11T00:23:50.362 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:50.360+0000 7ffaf1200640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559dd4b72b00 0x559dd4b74fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:23:50.362 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:50.360+0000 7ffaf1c00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559dd4b700c0 0x559dd4b724b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:23:50.362 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:50.360+0000 7ffaf1c00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559dd4b700c0 0x559dd4b724b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.12:49200/0 (socket says 172.21.15.12:49200) 2024-09-11T00:23:50.362 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:50.360+0000 7ffaf1c00640 1 -- 172.21.15.12:0/792666990 learned_addr learned my addr 172.21.15.12:0/792666990 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:23:50.363 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:50.360+0000 7ffaf1200640 1 --2- 172.21.15.12:0/792666990 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559dd4b6d790 0x559dd4b6fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:23:50.363 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:50.361+0000 7ffaf1c00640 1 -- 172.21.15.12:0/792666990 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559dd4b6d790 msgr2=0x559dd4b6fb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:23:50.363 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:50.361+0000 7ffaf1c00640 1 --2- 172.21.15.12:0/792666990 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559dd4b6d790 0x559dd4b6fb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:50.363 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:50.361+0000 7ffaf1c00640 1 -- 172.21.15.12:0/792666990 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559dd4b72b00 msgr2=0x559dd4b74fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:23:50.363 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:50.361+0000 7ffaf1c00640 1 --2- 172.21.15.12:0/792666990 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559dd4b72b00 0x559dd4b74fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:50.363 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:50.361+0000 7ffaf1c00640 1 -- 172.21.15.12:0/792666990 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559dd4a9c780 con 0x559dd4b700c0 2024-09-11T00:23:50.363 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:50.361+0000 7ffaf1200640 1 --2- 172.21.15.12:0/792666990 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559dd4b72b00 0x559dd4b74fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:23:50.366 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:50.361+0000 7ffaf1c00640 1 --2- 172.21.15.12:0/792666990 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559dd4b700c0 0x559dd4b724b0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7ffae00114f0 tx=0x7ffae00119b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7854263c57e5c6a9 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:23:50.366 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:50.361+0000 7ffadea00640 1 -- 172.21.15.12:0/792666990 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffae001e070 con 0x559dd4b700c0 2024-09-11T00:23:50.366 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:50.361+0000 7ffadea00640 1 -- 172.21.15.12:0/792666990 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffae0011310 con 0x559dd4b700c0 2024-09-11T00:23:50.366 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:50.361+0000 7ffadea00640 1 -- 172.21.15.12:0/792666990 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffae00198d0 con 0x559dd4b700c0 2024-09-11T00:23:50.367 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:50.361+0000 7ffaea000640 1 --2- 172.21.15.12:0/1817576800 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7ffad40b52e0 0x7ffad40b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:23:50.367 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:50.362+0000 7ffaea000640 1 -- 172.21.15.12:0/1817576800 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9208+0+0 (secure 0 0 0) 0x7ffae4106750 con 0x7ffaec1066a0 2024-09-11T00:23:50.367 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:50.362+0000 7ffaf1200640 1 --2- 172.21.15.12:0/1817576800 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7ffad40b52e0 0x7ffad40b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:23:50.367 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:50.362+0000 7ffaf745b740 1 -- 172.21.15.12:0/792666990 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559dd4b700c0 msgr2=0x559dd4b724b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:23:50.367 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:50.362+0000 7ffaf745b740 1 --2- 172.21.15.12:0/792666990 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559dd4b700c0 0x559dd4b724b0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7ffae00114f0 tx=0x7ffae00119b0 comp rx=0 tx=0).stop 2024-09-11T00:23:50.367 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:50.362+0000 7ffaf745b740 1 -- 172.21.15.12:0/792666990 shutdown_connections 2024-09-11T00:23:50.367 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:50.362+0000 7ffaf745b740 1 --2- 172.21.15.12:0/792666990 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559dd4b72b00 0x559dd4b74fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:50.367 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:50.362+0000 7ffaf745b740 1 --2- 172.21.15.12:0/792666990 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559dd4b700c0 0x559dd4b724b0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:50.367 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:50.362+0000 7ffaf745b740 1 --2- 172.21.15.12:0/792666990 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559dd4b6d790 0x559dd4b6fb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:50.367 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:50.362+0000 7ffaf745b740 1 -- 172.21.15.12:0/792666990 >> 172.21.15.12:0/792666990 conn(0x559dd4b688e0 msgr2=0x559dd4b69670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:23:50.367 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:50.362+0000 7ffaf745b740 1 -- 172.21.15.12:0/792666990 shutdown_connections 2024-09-11T00:23:50.368 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:50.362+0000 7ffaf745b740 1 -- 172.21.15.12:0/792666990 wait complete. 2024-09-11T00:23:50.368 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:50.362+0000 7ffaf1200640 1 --2- 172.21.15.12:0/1817576800 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7ffad40b52e0 0x7ffad40b77a0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7ffad8009fd0 tx=0x7ffad8004730 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:23:50.368 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:50.363+0000 7ffaf745b740 1 -- start start 2024-09-11T00:23:50.368 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:50.363+0000 7ffaf745b740 1 Processor -- start 2024-09-11T00:23:50.368 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:50.363+0000 7ffaf745b740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559dd4b6d790 0x559dd4b76ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:23:50.368 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:50.363+0000 7ffaf745b740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559dd4b700c0 0x559dd4b77570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:23:50.368 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:50.363+0000 7ffaf745b740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559dd4b72b00 0x559dd4b78140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:23:50.368 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:50.363+0000 7ffaf745b740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x559dd4a07ab0 con 0x559dd4b72b00 2024-09-11T00:23:50.368 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:50.363+0000 7ffaf745b740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x559dd4a76570 con 0x559dd4b700c0 2024-09-11T00:23:50.368 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:50.363+0000 7ffaf745b740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x559dd4b75ae0 con 0x559dd4b6d790 2024-09-11T00:23:50.368 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:50.363+0000 7ffaf745b740 10 client.0 _ll_register_callbacks cb 0x559dd4a85100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:23:50.368 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:50.364+0000 7ffaf2600640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559dd4b72b00 0x559dd4b78140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:23:50.369 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:50.364+0000 7ffaf1c00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559dd4b6d790 0x559dd4b76ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:23:50.369 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:50.364+0000 7ffaf2600640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559dd4b72b00 0x559dd4b78140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:50334/0 (socket says 172.21.15.12:50334) 2024-09-11T00:23:50.369 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:50.364+0000 7ffaf2600640 1 -- 172.21.15.12:0/1704764167 learned_addr learned my addr 172.21.15.12:0/1704764167 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:23:50.369 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:50.364+0000 7ffaf1200640 1 --2- 172.21.15.12:0/1704764167 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559dd4b700c0 0x559dd4b77570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:23:50.369 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:50.364+0000 7ffaf1c00640 1 -- 172.21.15.12:0/1704764167 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559dd4b700c0 msgr2=0x559dd4b77570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:23:50.369 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:50.364+0000 7ffaf1c00640 1 --2- 172.21.15.12:0/1704764167 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559dd4b700c0 0x559dd4b77570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:50.369 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:50.364+0000 7ffaf1c00640 1 -- 172.21.15.12:0/1704764167 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559dd4b72b00 msgr2=0x559dd4b78140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:23:50.369 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:50.364+0000 7ffaf1c00640 1 --2- 172.21.15.12:0/1704764167 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559dd4b72b00 0x559dd4b78140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:50.369 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:50.364+0000 7ffaf1c00640 1 -- 172.21.15.12:0/1704764167 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559dd4831790 con 0x559dd4b6d790 2024-09-11T00:23:50.369 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:50.364+0000 7ffaf1200640 1 --2- 172.21.15.12:0/1704764167 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559dd4b700c0 0x559dd4b77570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T00:23:50.369 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:50.364+0000 7ffaf1c00640 1 --2- 172.21.15.12:0/1704764167 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559dd4b6d790 0x559dd4b76ed0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7ffae00114c0 tx=0x7ffae0009c80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4e226130adad4043 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:23:50.370 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:50.364+0000 7ffad3400640 10 client.0 ms_handle_connect on v2:172.21.15.178:3300/0 2024-09-11T00:23:50.370 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:50.365+0000 7ffad3400640 1 -- 172.21.15.12:0/1704764167 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffae001e050 con 0x559dd4b6d790 2024-09-11T00:23:50.370 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:50.365+0000 7ffaf745b740 1 -- 172.21.15.12:0/1704764167 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffae0011310 con 0x559dd4b6d790 2024-09-11T00:23:50.370 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:50.365+0000 7ffad3400640 1 -- 172.21.15.12:0/1704764167 <== mon.1 v2:172.21.15.178:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9208+0+0 (secure 0 0 0) 0x7ffae000f770 con 0x559dd4b6d790 2024-09-11T00:23:50.370 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:50.365+0000 7ffad3400640 1 -- 172.21.15.12:0/1704764167 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=65) ==== 24+0+0 (secure 0 0 0) 0x7ffae0032050 con 0x559dd4b6d790 2024-09-11T00:23:50.370 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:50.365+0000 7ffaf745b740 10 client.27313 fetch_fsmap learned FSMap version 65 2024-09-11T00:23:50.370 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:50.365+0000 7ffaf745b740 1 -- 172.21.15.12:0/1704764167 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,fsmap=65,monmap=4+}) -- 0x559dd48aa620 con 0x559dd4b6d790 2024-09-11T00:23:50.372 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:50.365+0000 7ffad3400640 1 -- 172.21.15.12:0/1704764167 <== mon.1 v2:172.21.15.178:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffae0011310 con 0x559dd4b6d790 2024-09-11T00:23:50.372 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:50.366+0000 7ffad3400640 1 -- 172.21.15.12:0/1704764167 <== mon.1 v2:172.21.15.178:3300/0 5 ==== fsmap(e 65) ==== 8386+0+0 (secure 0 0 0) 0x7ffae002d030 con 0x559dd4b6d790 2024-09-11T00:23:50.372 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:50.366+0000 7ffaf745b740 10 client.27313 fetch_fsmap finished waiting for FSMap version 65 2024-09-11T00:23:50.372 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:50.366+0000 7ffaf745b740 10 client.27313 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T00:23:50.372 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:50.366+0000 7ffaf745b740 20 client.27313 populate_metadata read hostname 'smithi012' 2024-09-11T00:23:50.372 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:50.366+0000 7ffaf745b740 1 --2- 172.21.15.12:0/1704764167 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x559dd4b82520 0x559dd4b84950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:23:50.372 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:50.366+0000 7ffaf745b740 4 client.27313 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T00:23:50.373 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:50.366+0000 7ffaf745b740 1 -- 172.21.15.12:0/1704764167 --> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559dd4ae2600 con 0x559dd4b82520 2024-09-11T00:23:50.373 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:50.366+0000 7ffaf1200640 1 --2- 172.21.15.12:0/1704764167 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x559dd4b82520 0x559dd4b84950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:23:50.373 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:50.367+0000 7ffaf1200640 1 --2- 172.21.15.12:0/1704764167 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x559dd4b82520 0x559dd4b84950 crc :-1 s=READY pgs=239 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2e81d5c3e1ebc9a9 server_cookie=92dad6ec6f3f880b in_seq=0 out_seq=0 2024-09-11T00:23:50.373 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:50.367+0000 7ffad3400640 10 client.27313 ms_handle_connect on v2:172.21.15.165:6838/501312984 2024-09-11T00:23:50.373 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:50.368+0000 7ffad3400640 1 -- 172.21.15.12:0/1704764167 <== mds.0 v2:172.21.15.165:6838/501312984 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x559dd4ae2600 con 0x559dd4b82520 2024-09-11T00:23:50.373 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:50.368+0000 7ffad3400640 10 client.27313 handle_command_reply: tid=0 2024-09-11T00:23:50.373 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:50.368+0000 7ffaf745b740 1 client.27313 shutdown 2024-09-11T00:23:50.373 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:50.368+0000 7ffaf745b740 1 -- 172.21.15.12:0/1704764167 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559dd4b6d790 msgr2=0x559dd4b76ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:23:50.373 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:50.368+0000 7ffaf745b740 1 --2- 172.21.15.12:0/1704764167 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559dd4b6d790 0x559dd4b76ed0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7ffae00114c0 tx=0x7ffae0009c80 comp rx=0 tx=0).stop 2024-09-11T00:23:50.373 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:50.368+0000 7ffaf745b740 1 -- 172.21.15.12:0/1704764167 shutdown_connections 2024-09-11T00:23:50.373 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:50.368+0000 7ffaf745b740 1 --2- 172.21.15.12:0/1704764167 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x559dd4b82520 0x559dd4b84950 crc :-1 s=READY pgs=239 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:50.373 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:50.368+0000 7ffaf745b740 1 --2- 172.21.15.12:0/1704764167 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559dd4b72b00 0x559dd4b78140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:50.374 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:50.368+0000 7ffaf745b740 1 --2- 172.21.15.12:0/1704764167 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559dd4b700c0 0x559dd4b77570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:50.374 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:50.368+0000 7ffad3400640 0 client.27313 ms_handle_reset on v2:172.21.15.165:6838/501312984 2024-09-11T00:23:50.374 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:50.368+0000 7ffaf745b740 1 --2- 172.21.15.12:0/1704764167 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559dd4b6d790 0x559dd4b76ed0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:50.374 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:50.368+0000 7ffaf745b740 1 -- 172.21.15.12:0/1704764167 >> 172.21.15.12:0/1704764167 conn(0x559dd4b688e0 msgr2=0x559dd4b67a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:23:50.374 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:50.368+0000 7ffaf745b740 1 -- 172.21.15.12:0/1704764167 shutdown_connections 2024-09-11T00:23:50.374 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:50.368+0000 7ffaf745b740 1 -- 172.21.15.12:0/1704764167 wait complete. 2024-09-11T00:23:50.374 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:50.369+0000 7ffaf745b740 20 client.27313 trim_cache size 0 max 16384 2024-09-11T00:23:50.386 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:50.384+0000 7ffaf745b740 1 Processor -- start 2024-09-11T00:23:50.386 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:50.384+0000 7ffaf745b740 1 -- start start 2024-09-11T00:23:50.386 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:50.384+0000 7ffaf745b740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559dd4b6d790 0x559dd4b89250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:23:50.387 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:50.384+0000 7ffaf745b740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559dd4b700c0 0x559dd4b89790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:23:50.387 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:50.384+0000 7ffaf745b740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559dd4b72b00 0x559dd4b89cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:23:50.387 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:50.384+0000 7ffaf745b740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7ffaec1ad8f0 con 0x559dd4b700c0 2024-09-11T00:23:50.387 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:50.384+0000 7ffaf745b740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7ffaec1af490 con 0x559dd4b6d790 2024-09-11T00:23:50.387 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:50.384+0000 7ffaf745b740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7ffaec1adff0 con 0x559dd4b72b00 2024-09-11T00:23:50.387 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:50.384+0000 7ffaf1200640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559dd4b700c0 0x559dd4b89790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:23:50.387 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:50.384+0000 7ffaf1200640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559dd4b700c0 0x559dd4b89790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:50344/0 (socket says 172.21.15.12:50344) 2024-09-11T00:23:50.387 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:50.384+0000 7ffaf1200640 1 -- 172.21.15.12:0/500592564 learned_addr learned my addr 172.21.15.12:0/500592564 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:23:50.387 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:50.384+0000 7ffaf1c00640 1 --2- 172.21.15.12:0/500592564 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559dd4b6d790 0x559dd4b89250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:23:50.387 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:50.384+0000 7ffaf2600640 1 --2- 172.21.15.12:0/500592564 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559dd4b72b00 0x559dd4b89cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:23:50.387 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:50.385+0000 7ffaf1200640 1 -- 172.21.15.12:0/500592564 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559dd4b72b00 msgr2=0x559dd4b89cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:23:50.388 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:50.385+0000 7ffaf1200640 1 --2- 172.21.15.12:0/500592564 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559dd4b72b00 0x559dd4b89cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:50.388 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:50.385+0000 7ffaf1200640 1 -- 172.21.15.12:0/500592564 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559dd4b6d790 msgr2=0x559dd4b89250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:23:50.388 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:50.385+0000 7ffaf1200640 1 --2- 172.21.15.12:0/500592564 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559dd4b6d790 0x559dd4b89250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:50.388 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:50.385+0000 7ffaf1200640 1 -- 172.21.15.12:0/500592564 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559dd4b84fd0 con 0x559dd4b700c0 2024-09-11T00:23:50.388 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:50.385+0000 7ffaf1c00640 1 --2- 172.21.15.12:0/500592564 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559dd4b6d790 0x559dd4b89250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T00:23:50.388 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:50.385+0000 7ffaf1200640 1 --2- 172.21.15.12:0/500592564 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559dd4b700c0 0x559dd4b89790 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7ffad8009d60 tx=0x7ffad8013040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c2179fd031b81b4 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:23:50.388 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:50.385+0000 7ffade000640 1 -- 172.21.15.12:0/500592564 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffad8012c50 con 0x559dd4b700c0 2024-09-11T00:23:50.388 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:50.385+0000 7ffade000640 1 -- 172.21.15.12:0/500592564 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffad8021e10 con 0x559dd4b700c0 2024-09-11T00:23:50.388 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:50.385+0000 7ffade000640 1 -- 172.21.15.12:0/500592564 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffad802b9c0 con 0x559dd4b700c0 2024-09-11T00:23:50.391 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:50.386+0000 7ffaf745b740 1 -- 172.21.15.12:0/500592564 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559dd4b700c0 msgr2=0x559dd4b89790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:23:50.391 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:50.386+0000 7ffaf745b740 1 --2- 172.21.15.12:0/500592564 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559dd4b700c0 0x559dd4b89790 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7ffad8009d60 tx=0x7ffad8013040 comp rx=0 tx=0).stop 2024-09-11T00:23:50.391 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:50.386+0000 7ffaf745b740 1 -- 172.21.15.12:0/500592564 shutdown_connections 2024-09-11T00:23:50.392 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:50.386+0000 7ffaf745b740 1 --2- 172.21.15.12:0/500592564 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559dd4b72b00 0x559dd4b89cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:50.392 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:50.386+0000 7ffaf745b740 1 --2- 172.21.15.12:0/500592564 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559dd4b700c0 0x559dd4b89790 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:50.392 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:50.386+0000 7ffaf745b740 1 --2- 172.21.15.12:0/500592564 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559dd4b6d790 0x559dd4b89250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:50.392 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:50.386+0000 7ffaf745b740 1 -- 172.21.15.12:0/500592564 >> 172.21.15.12:0/500592564 conn(0x559dd4b688e0 msgr2=0x559dd4b6b930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:23:50.392 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:50.386+0000 7ffaf745b740 1 -- 172.21.15.12:0/500592564 shutdown_connections 2024-09-11T00:23:50.392 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:50.386+0000 7ffaf745b740 1 -- 172.21.15.12:0/500592564 wait complete. 2024-09-11T00:23:50.392 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:50.386+0000 7ffaf745b740 1 -- start start 2024-09-11T00:23:50.392 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:50.386+0000 7ffaf745b740 1 Processor -- start 2024-09-11T00:23:50.392 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:50.387+0000 7ffaf745b740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559dd4b6d790 0x559dd4b7da40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:23:50.392 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:50.387+0000 7ffaf745b740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559dd4b700c0 0x559dd4b7df80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:23:50.392 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:50.387+0000 7ffaf745b740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559dd4b72b00 0x559dd4b7e4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:23:50.392 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:50.387+0000 7ffaf745b740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7ffaec1aedc0 con 0x559dd4b6d790 2024-09-11T00:23:50.392 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:50.387+0000 7ffaf745b740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7ffaec1ae6c0 con 0x559dd4b700c0 2024-09-11T00:23:50.393 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:50.387+0000 7ffaf745b740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7ffaec1ab8d0 con 0x559dd4b72b00 2024-09-11T00:23:50.393 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:50.387+0000 7ffaf745b740 10 client.0 _ll_register_callbacks cb 0x7ffae001c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:23:50.393 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:50.387+0000 7ffaf1c00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559dd4b6d790 0x559dd4b7da40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:23:50.393 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:50.387+0000 7ffaf1c00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559dd4b6d790 0x559dd4b7da40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:50348/0 (socket says 172.21.15.12:50348) 2024-09-11T00:23:50.393 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:50.387+0000 7ffaf1c00640 1 -- 172.21.15.12:0/115114950 learned_addr learned my addr 172.21.15.12:0/115114950 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:23:50.393 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:50.387+0000 7ffaf2600640 1 --2- 172.21.15.12:0/115114950 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559dd4b72b00 0x559dd4b7e4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:23:50.393 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:50.387+0000 7ffaf1200640 1 --2- 172.21.15.12:0/115114950 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559dd4b700c0 0x559dd4b7df80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:23:50.393 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:50.387+0000 7ffaf1c00640 1 -- 172.21.15.12:0/115114950 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559dd4b72b00 msgr2=0x559dd4b7e4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:23:50.393 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:50.387+0000 7ffaf1c00640 1 --2- 172.21.15.12:0/115114950 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559dd4b72b00 0x559dd4b7e4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:50.393 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:50.387+0000 7ffaf1c00640 1 -- 172.21.15.12:0/115114950 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559dd4b700c0 msgr2=0x559dd4b7df80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:23:50.393 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:50.387+0000 7ffaf1c00640 1 --2- 172.21.15.12:0/115114950 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559dd4b700c0 0x559dd4b7df80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:50.393 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:50.387+0000 7ffaf1c00640 1 -- 172.21.15.12:0/115114950 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559dd4b7c680 con 0x559dd4b6d790 2024-09-11T00:23:50.393 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:50.387+0000 7ffaf2600640 1 --2- 172.21.15.12:0/115114950 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559dd4b72b00 0x559dd4b7e4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T00:23:50.394 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:50.387+0000 7ffaf1200640 1 --2- 172.21.15.12:0/115114950 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559dd4b700c0 0x559dd4b7df80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T00:23:50.394 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:50.388+0000 7ffaf1c00640 1 --2- 172.21.15.12:0/115114950 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559dd4b6d790 0x559dd4b7da40 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7ffae00119f0 tx=0x7ffae0009cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=efeba4d8ef7c79f6 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:23:50.394 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:50.388+0000 7ffad3400640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-09-11T00:23:50.394 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:50.388+0000 7ffad3400640 1 -- 172.21.15.12:0/115114950 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffae003b060 con 0x559dd4b6d790 2024-09-11T00:23:50.394 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:50.388+0000 7ffaf745b740 1 -- 172.21.15.12:0/115114950 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffad8021e10 con 0x559dd4b6d790 2024-09-11T00:23:50.394 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:50.388+0000 7ffad3400640 1 -- 172.21.15.12:0/115114950 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9208+0+0 (secure 0 0 0) 0x7ffae0005750 con 0x559dd4b6d790 2024-09-11T00:23:50.394 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:50.388+0000 7ffad3400640 1 -- 172.21.15.12:0/115114950 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=65) ==== 24+0+0 (secure 0 0 0) 0x7ffae001e070 con 0x559dd4b6d790 2024-09-11T00:23:50.394 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:50.388+0000 7ffaf745b740 10 client.17346 fetch_fsmap learned FSMap version 65 2024-09-11T00:23:50.394 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:50.388+0000 7ffaf745b740 1 -- 172.21.15.12:0/115114950 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=65,monmap=4+}) -- 0x559dd4bef830 con 0x559dd4b6d790 2024-09-11T00:23:50.394 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:50.388+0000 7ffad3400640 1 -- 172.21.15.12:0/115114950 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffad8021e10 con 0x559dd4b6d790 2024-09-11T00:23:50.394 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:50.389+0000 7ffad3400640 1 -- 172.21.15.12:0/115114950 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 65) ==== 8386+0+0 (secure 0 0 0) 0x7ffae002d070 con 0x559dd4b6d790 2024-09-11T00:23:50.394 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:50.389+0000 7ffaf745b740 10 client.17346 fetch_fsmap finished waiting for FSMap version 65 2024-09-11T00:23:50.394 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:50.389+0000 7ffaf745b740 10 client.17346 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T00:23:50.395 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:50.389+0000 7ffaf745b740 20 client.17346 populate_metadata read hostname 'smithi012' 2024-09-11T00:23:50.395 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:50.389+0000 7ffaf745b740 1 --2- 172.21.15.12:0/115114950 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x559dd4bf01b0 0x559dd4bf25d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:23:50.395 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:50.389+0000 7ffaf745b740 4 client.17346 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T00:23:50.395 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:50.389+0000 7ffaf745b740 1 -- 172.21.15.12:0/115114950 --> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] -- command(tid 0: {"prefix": "flush journal"}) -- 0x559dd4ade0b0 con 0x559dd4bf01b0 2024-09-11T00:23:50.395 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:50.389+0000 7ffaf1200640 1 --2- 172.21.15.12:0/115114950 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x559dd4bf01b0 0x559dd4bf25d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:23:50.395 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:50.390+0000 7ffaf1200640 1 --2- 172.21.15.12:0/115114950 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x559dd4bf01b0 0x559dd4bf25d0 crc :-1 s=READY pgs=240 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=313f17d4856f56bd server_cookie=58704dd088499115 in_seq=0 out_seq=0 2024-09-11T00:23:50.395 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:50.390+0000 7ffad3400640 10 client.17346 ms_handle_connect on v2:172.21.15.165:6838/501312984 2024-09-11T00:23:50.433 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:50.431+0000 7ffad3400640 1 -- 172.21.15.12:0/115114950 <== mds.0 v2:172.21.15.165:6838/501312984 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x559dd4ade0b0 con 0x559dd4bf01b0 2024-09-11T00:23:50.433 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:50.431+0000 7ffad3400640 10 client.17346 handle_command_reply: tid=0 2024-09-11T00:23:50.433 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:50.431+0000 7ffaf745b740 1 client.17346 shutdown 2024-09-11T00:23:50.433 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:50.431+0000 7ffaf745b740 1 -- 172.21.15.12:0/115114950 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559dd4b6d790 msgr2=0x559dd4b7da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:23:50.434 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:50.431+0000 7ffaf745b740 1 --2- 172.21.15.12:0/115114950 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559dd4b6d790 0x559dd4b7da40 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7ffae00119f0 tx=0x7ffae0009cf0 comp rx=0 tx=0).stop 2024-09-11T00:23:50.434 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:50.432+0000 7ffaf745b740 1 -- 172.21.15.12:0/115114950 shutdown_connections 2024-09-11T00:23:50.434 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:50.432+0000 7ffaf745b740 1 --2- 172.21.15.12:0/115114950 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x559dd4bf01b0 0x559dd4bf25d0 crc :-1 s=READY pgs=240 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:50.434 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:50.432+0000 7ffaf745b740 1 --2- 172.21.15.12:0/115114950 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559dd4b72b00 0x559dd4b7e4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:50.434 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:50.432+0000 7ffad3400640 0 client.17346 ms_handle_reset on v2:172.21.15.165:6838/501312984 2024-09-11T00:23:50.434 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:50.432+0000 7ffaf745b740 1 --2- 172.21.15.12:0/115114950 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559dd4b700c0 0x559dd4b7df80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:50.434 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:50.432+0000 7ffaf745b740 1 --2- 172.21.15.12:0/115114950 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559dd4b6d790 0x559dd4b7da40 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:50.434 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:50.432+0000 7ffaf745b740 1 -- 172.21.15.12:0/115114950 >> 172.21.15.12:0/115114950 conn(0x559dd4b688e0 msgr2=0x559dd4b67240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:23:50.434 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:50.432+0000 7ffaf745b740 1 -- 172.21.15.12:0/115114950 shutdown_connections 2024-09-11T00:23:50.434 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:50.432+0000 7ffaf745b740 1 -- 172.21.15.12:0/115114950 wait complete. 2024-09-11T00:23:50.434 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:50.432+0000 7ffaf745b740 20 client.17346 trim_cache size 0 max 16384 2024-09-11T00:23:50.435 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:{ 2024-09-11T00:23:50.435 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "message": "", 2024-09-11T00:23:50.435 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "return_code": 0 2024-09-11T00:23:50.435 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:} 2024-09-11T00:23:50.436 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:50.433+0000 7ffaf4600640 1 -- 172.21.15.12:0/1817576800 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7ffad40b52e0 msgr2=0x7ffad40b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:23:50.436 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:50.433+0000 7ffaf4600640 1 --2- 172.21.15.12:0/1817576800 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7ffad40b52e0 0x7ffad40b77a0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7ffad8009fd0 tx=0x7ffad8004730 comp rx=0 tx=0).stop 2024-09-11T00:23:50.436 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:50.433+0000 7ffaf4600640 1 -- 172.21.15.12:0/1817576800 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ffaec1066a0 msgr2=0x7ffaec19c1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:23:50.436 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:50.433+0000 7ffaf4600640 1 --2- 172.21.15.12:0/1817576800 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ffaec1066a0 0x7ffaec19c1b0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7ffae4031500 tx=0x7ffae40042f0 comp rx=0 tx=0).stop 2024-09-11T00:23:50.436 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:50.433+0000 7ffaf4600640 1 -- 172.21.15.12:0/1817576800 shutdown_connections 2024-09-11T00:23:50.436 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:50.433+0000 7ffaf4600640 1 --2- 172.21.15.12:0/1817576800 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7ffad40b52e0 0x7ffad40b77a0 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:50.436 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:50.433+0000 7ffaf4600640 1 --2- 172.21.15.12:0/1817576800 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ffaec1066a0 0x7ffaec19c1b0 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:50.436 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:50.433+0000 7ffaf4600640 1 --2- 172.21.15.12:0/1817576800 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ffaec105ce0 0x7ffaec19b580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:50.436 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:50.433+0000 7ffaf4600640 1 --2- 172.21.15.12:0/1817576800 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ffaec104b90 0x7ffaec19af30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:50.436 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:50.433+0000 7ffaf4600640 1 -- 172.21.15.12:0/1817576800 >> 172.21.15.12:0/1817576800 conn(0x7ffaec100470 msgr2=0x7ffaec1090d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:23:50.436 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:50.433+0000 7ffaf4600640 1 -- 172.21.15.12:0/1817576800 shutdown_connections 2024-09-11T00:23:50.436 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:23:50.433+0000 7ffaf4600640 1 -- 172.21.15.12:0/1817576800 wait complete. 2024-09-11T00:23:51.177 INFO:tasks.workunit.client.0.smithi012.stdout: 10 69399 18.32 MB/sec execute 107 sec latency 10500.096 ms 2024-09-11T00:23:52.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:23:51 smithi165 ceph-mon[35356]: pgmap v902: 129 pgs: 129 active+clean; 496 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 557 KiB/s rd, 445 KiB/s wr, 122 op/s 2024-09-11T00:23:52.063 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:23:51 smithi012 ceph-mon[30333]: pgmap v902: 129 pgs: 129 active+clean; 496 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 557 KiB/s rd, 445 KiB/s wr, 122 op/s 2024-09-11T00:23:52.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:23:51 smithi178 ceph-mon[35919]: pgmap v902: 129 pgs: 129 active+clean; 496 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 557 KiB/s rd, 445 KiB/s wr, 122 op/s 2024-09-11T00:23:52.177 INFO:tasks.workunit.client.0.smithi012.stdout: 10 69399 18.15 MB/sec execute 108 sec latency 11500.194 ms 2024-09-11T00:23:53.177 INFO:tasks.workunit.client.0.smithi012.stdout: 10 69399 17.99 MB/sec execute 109 sec latency 12500.322 ms 2024-09-11T00:23:54.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:23:53 smithi165 ceph-mon[35356]: pgmap v903: 129 pgs: 129 active+clean; 492 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 562 KiB/s rd, 442 KiB/s wr, 121 op/s 2024-09-11T00:23:54.063 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:23:53 smithi012 ceph-mon[30333]: pgmap v903: 129 pgs: 129 active+clean; 492 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 562 KiB/s rd, 442 KiB/s wr, 121 op/s 2024-09-11T00:23:54.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:23:53 smithi178 ceph-mon[35919]: pgmap v903: 129 pgs: 129 active+clean; 492 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 562 KiB/s rd, 442 KiB/s wr, 121 op/s 2024-09-11T00:23:54.177 INFO:tasks.workunit.client.0.smithi012.stdout: 10 69399 17.82 MB/sec execute 110 sec latency 13500.414 ms 2024-09-11T00:23:55.177 INFO:tasks.workunit.client.0.smithi012.stdout: 10 69399 17.66 MB/sec execute 111 sec latency 14500.549 ms 2024-09-11T00:23:55.955 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:23:55 smithi012 ceph-mon[30333]: pgmap v904: 129 pgs: 129 active+clean; 492 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 392 KiB/s rd, 224 KiB/s wr, 77 op/s 2024-09-11T00:23:56.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:23:55 smithi165 ceph-mon[35356]: pgmap v904: 129 pgs: 129 active+clean; 492 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 392 KiB/s rd, 224 KiB/s wr, 77 op/s 2024-09-11T00:23:56.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:23:55 smithi178 ceph-mon[35919]: pgmap v904: 129 pgs: 129 active+clean; 492 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 392 KiB/s rd, 224 KiB/s wr, 77 op/s 2024-09-11T00:23:56.177 INFO:tasks.workunit.client.0.smithi012.stdout: 10 69399 17.50 MB/sec execute 112 sec latency 15500.689 ms 2024-09-11T00:23:57.177 INFO:tasks.workunit.client.0.smithi012.stdout: 10 69399 17.35 MB/sec execute 113 sec latency 16500.761 ms 2024-09-11T00:23:58.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:23:57 smithi165 ceph-mon[35356]: pgmap v905: 129 pgs: 129 active+clean; 492 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 399 KiB/s rd, 215 KiB/s wr, 85 op/s 2024-09-11T00:23:58.063 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:23:57 smithi012 ceph-mon[30333]: pgmap v905: 129 pgs: 129 active+clean; 492 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 399 KiB/s rd, 215 KiB/s wr, 85 op/s 2024-09-11T00:23:58.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:23:57 smithi178 ceph-mon[35919]: pgmap v905: 129 pgs: 129 active+clean; 492 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 399 KiB/s rd, 215 KiB/s wr, 85 op/s 2024-09-11T00:23:58.178 INFO:tasks.workunit.client.0.smithi012.stdout: 10 69399 17.20 MB/sec execute 114 sec latency 17500.884 ms 2024-09-11T00:23:59.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:23:58 smithi165 ceph-mon[35356]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T00:23:59.063 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:23:58 smithi012 ceph-mon[30333]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T00:23:59.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:23:58 smithi178 ceph-mon[35919]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T00:23:59.178 INFO:tasks.workunit.client.0.smithi012.stdout: 10 69399 17.05 MB/sec execute 115 sec latency 18501.009 ms 2024-09-11T00:24:00.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:23:59 smithi165 ceph-mon[35356]: pgmap v906: 129 pgs: 129 active+clean; 492 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 55 KiB/s wr, 27 op/s 2024-09-11T00:24:00.063 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:23:59 smithi012 ceph-mon[30333]: pgmap v906: 129 pgs: 129 active+clean; 492 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 55 KiB/s wr, 27 op/s 2024-09-11T00:24:00.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:23:59 smithi178 ceph-mon[35919]: pgmap v906: 129 pgs: 129 active+clean; 492 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 55 KiB/s wr, 27 op/s 2024-09-11T00:24:00.178 INFO:tasks.workunit.client.0.smithi012.stdout: 10 69399 16.90 MB/sec execute 116 sec latency 19501.170 ms 2024-09-11T00:24:01.178 INFO:tasks.workunit.client.0.smithi012.stdout: 10 69399 16.76 MB/sec execute 117 sec latency 20501.246 ms 2024-09-11T00:24:02.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:24:01 smithi165 ceph-mon[35356]: pgmap v907: 129 pgs: 129 active+clean; 493 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 267 KiB/s wr, 34 op/s 2024-09-11T00:24:02.063 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:24:01 smithi012 ceph-mon[30333]: pgmap v907: 129 pgs: 129 active+clean; 493 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 267 KiB/s wr, 34 op/s 2024-09-11T00:24:02.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:24:01 smithi178 ceph-mon[35919]: pgmap v907: 129 pgs: 129 active+clean; 493 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 267 KiB/s wr, 34 op/s 2024-09-11T00:24:02.178 INFO:tasks.workunit.client.0.smithi012.stdout: 10 69399 16.61 MB/sec execute 118 sec latency 21501.297 ms 2024-09-11T00:24:03.178 INFO:tasks.workunit.client.0.smithi012.stdout: 10 69399 16.48 MB/sec execute 119 sec latency 22501.390 ms 2024-09-11T00:24:03.562 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:03.559+0000 7fae5a400640 1 -- 172.21.15.12:0/2269732846 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fae541084a0 msgr2=0x7fae5410a960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:24:03.562 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:03.559+0000 7fae5a400640 1 --2- 172.21.15.12:0/2269732846 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fae541084a0 0x7fae5410a960 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7fae4c009ca0 tx=0x7fae4c02a8b0 comp rx=0 tx=0).stop 2024-09-11T00:24:03.562 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:03.559+0000 7fae5a400640 1 -- 172.21.15.12:0/2269732846 shutdown_connections 2024-09-11T00:24:03.562 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:03.559+0000 7fae5a400640 1 --2- 172.21.15.12:0/2269732846 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fae541084a0 0x7fae5410a960 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:24:03.562 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:03.559+0000 7fae5a400640 1 --2- 172.21.15.12:0/2269732846 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fae541058f0 0x7fae54107d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:24:03.562 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:03.559+0000 7fae5a400640 1 --2- 172.21.15.12:0/2269732846 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fae5406b470 0x7fae54105360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:24:03.562 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:03.559+0000 7fae5a400640 1 -- 172.21.15.12:0/2269732846 >> 172.21.15.12:0/2269732846 conn(0x7fae540fcf30 msgr2=0x7fae540ff370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:24:03.562 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:03.559+0000 7fae5a400640 1 -- 172.21.15.12:0/2269732846 shutdown_connections 2024-09-11T00:24:03.563 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:03.559+0000 7fae5a400640 1 -- 172.21.15.12:0/2269732846 wait complete. 2024-09-11T00:24:03.564 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:03.560+0000 7fae5a400640 1 Processor -- start 2024-09-11T00:24:03.564 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:03.560+0000 7fae5a400640 1 -- start start 2024-09-11T00:24:03.564 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:03.561+0000 7fae5a400640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fae5406b470 0x7fae54102aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:24:03.564 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:03.561+0000 7fae5a400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fae541058f0 0x7fae54101300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:24:03.564 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:03.561+0000 7fae5a400640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fae541084a0 0x7fae54101840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:24:03.564 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:03.561+0000 7fae5a400640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fae5410e440 con 0x7fae541084a0 2024-09-11T00:24:03.564 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:03.561+0000 7fae5a400640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fae5410e2c0 con 0x7fae541058f0 2024-09-11T00:24:03.564 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:03.561+0000 7fae5a400640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fae5410e5c0 con 0x7fae5406b470 2024-09-11T00:24:03.565 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:03.561+0000 7fae53e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fae541058f0 0x7fae54101300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:24:03.565 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:03.561+0000 7fae59a00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fae541084a0 0x7fae54101840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:24:03.565 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:03.561+0000 7fae59000640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fae5406b470 0x7fae54102aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:24:03.565 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:03.561+0000 7fae53e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fae541058f0 0x7fae54101300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.12:60682/0 (socket says 172.21.15.12:60682) 2024-09-11T00:24:03.565 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:03.561+0000 7fae53e00640 1 -- 172.21.15.12:0/4052879519 learned_addr learned my addr 172.21.15.12:0/4052879519 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:24:03.565 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:03.562+0000 7fae53e00640 1 -- 172.21.15.12:0/4052879519 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fae5406b470 msgr2=0x7fae54102aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:24:03.565 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:03.562+0000 7fae53e00640 1 --2- 172.21.15.12:0/4052879519 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fae5406b470 0x7fae54102aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:24:03.565 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:03.562+0000 7fae53e00640 1 -- 172.21.15.12:0/4052879519 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fae541084a0 msgr2=0x7fae54101840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:24:03.565 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:03.562+0000 7fae53e00640 1 --2- 172.21.15.12:0/4052879519 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fae541084a0 0x7fae54101840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:24:03.565 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:03.562+0000 7fae53e00640 1 -- 172.21.15.12:0/4052879519 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae541020b0 con 0x7fae541058f0 2024-09-11T00:24:03.565 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:03.562+0000 7fae59000640 1 --2- 172.21.15.12:0/4052879519 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fae5406b470 0x7fae54102aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:24:03.565 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:03.562+0000 7fae53e00640 1 --2- 172.21.15.12:0/4052879519 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fae541058f0 0x7fae54101300 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fae4000e9f0 tx=0x7fae4000edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:24:03.566 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:03.562+0000 7fae51600640 1 -- 172.21.15.12:0/4052879519 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae4000cd80 con 0x7fae541058f0 2024-09-11T00:24:03.566 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:03.562+0000 7fae5a400640 1 -- 172.21.15.12:0/4052879519 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fae541a2be0 con 0x7fae541058f0 2024-09-11T00:24:03.566 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:03.562+0000 7fae51600640 1 -- 172.21.15.12:0/4052879519 <== mon.2 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fae400054d0 con 0x7fae541058f0 2024-09-11T00:24:03.566 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:03.562+0000 7fae51600640 1 -- 172.21.15.12:0/4052879519 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae40010690 con 0x7fae541058f0 2024-09-11T00:24:03.566 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:03.562+0000 7fae5a400640 1 -- 172.21.15.12:0/4052879519 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fae541a30b0 con 0x7fae541058f0 2024-09-11T00:24:03.567 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:03.565+0000 7fae5e8d0740 1 Processor -- start 2024-09-11T00:24:03.567 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:03.565+0000 7fae5e8d0740 1 -- start start 2024-09-11T00:24:03.568 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:03.566+0000 7fae5e8d0740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558497f62970 0x558497f64d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:24:03.568 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:03.566+0000 7fae5e8d0740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558497f652a0 0x558497f67690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:24:03.568 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:03.566+0000 7fae5e8d0740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558497f67ce0 0x558497f6a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:24:03.568 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:03.566+0000 7fae5e8d0740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x558497ed56b0 con 0x558497f67ce0 2024-09-11T00:24:03.569 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:03.566+0000 7fae5e8d0740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x558497ec4ae0 con 0x558497f652a0 2024-09-11T00:24:03.569 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:03.566+0000 7fae5e8d0740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x558497ec88f0 con 0x558497f62970 2024-09-11T00:24:03.569 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:03.566+0000 7fae59000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558497f652a0 0x558497f67690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:24:03.569 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:03.566+0000 7fae59000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558497f652a0 0x558497f67690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.12:60692/0 (socket says 172.21.15.12:60692) 2024-09-11T00:24:03.569 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:03.566+0000 7fae59000640 1 -- 172.21.15.12:0/3116390012 learned_addr learned my addr 172.21.15.12:0/3116390012 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:24:03.569 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:03.566+0000 7fae59a00640 1 --2- 172.21.15.12:0/3116390012 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558497f62970 0x558497f64d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:24:03.569 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:03.566+0000 7fae51600640 1 -- 172.21.15.12:0/4052879519 <== mon.2 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7fae40004e90 con 0x7fae541058f0 2024-09-11T00:24:03.569 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:03.566+0000 7fae59000640 1 -- 172.21.15.12:0/3116390012 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558497f62970 msgr2=0x558497f64d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:24:03.569 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:03.566+0000 7fae59000640 1 --2- 172.21.15.12:0/3116390012 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558497f62970 0x558497f64d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:24:03.569 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:03.566+0000 7fae59000640 1 -- 172.21.15.12:0/3116390012 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558497f67ce0 msgr2=0x558497f6a1a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-11T00:24:03.571 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:03.567+0000 7fae59000640 1 --2- 172.21.15.12:0/3116390012 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558497f67ce0 0x558497f6a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:24:03.571 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:03.567+0000 7fae59000640 1 -- 172.21.15.12:0/3116390012 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558497e91780 con 0x558497f652a0 2024-09-11T00:24:03.571 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:03.567+0000 7fae59000640 1 --2- 172.21.15.12:0/3116390012 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558497f652a0 0x558497f67690 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fae480115f0 tx=0x7fae48011ab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e87698cb627c0de9 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:24:03.571 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:03.567+0000 7fae46000640 1 -- 172.21.15.12:0/3116390012 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae4801e070 con 0x558497f652a0 2024-09-11T00:24:03.571 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:03.567+0000 7fae46000640 1 -- 172.21.15.12:0/3116390012 <== mon.2 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fae4800faf0 con 0x558497f652a0 2024-09-11T00:24:03.571 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:03.567+0000 7fae46000640 1 -- 172.21.15.12:0/3116390012 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae48019b10 con 0x558497f652a0 2024-09-11T00:24:03.571 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:03.568+0000 7fae5e8d0740 1 -- 172.21.15.12:0/3116390012 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558497f652a0 msgr2=0x558497f67690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:24:03.571 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:03.568+0000 7fae5e8d0740 1 --2- 172.21.15.12:0/3116390012 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558497f652a0 0x558497f67690 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fae480115f0 tx=0x7fae48011ab0 comp rx=0 tx=0).stop 2024-09-11T00:24:03.571 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:03.568+0000 7fae5e8d0740 1 -- 172.21.15.12:0/3116390012 shutdown_connections 2024-09-11T00:24:03.571 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:03.568+0000 7fae5e8d0740 1 --2- 172.21.15.12:0/3116390012 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558497f67ce0 0x558497f6a1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:24:03.571 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:03.568+0000 7fae5e8d0740 1 --2- 172.21.15.12:0/3116390012 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558497f652a0 0x558497f67690 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:24:03.571 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:03.568+0000 7fae5e8d0740 1 --2- 172.21.15.12:0/3116390012 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558497f62970 0x558497f64d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:24:03.572 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:03.568+0000 7fae5e8d0740 1 -- 172.21.15.12:0/3116390012 >> 172.21.15.12:0/3116390012 conn(0x558497f5dac0 msgr2=0x558497f5e850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:24:03.572 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:03.568+0000 7fae5e8d0740 1 -- 172.21.15.12:0/3116390012 shutdown_connections 2024-09-11T00:24:03.572 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:03.568+0000 7fae5e8d0740 1 -- 172.21.15.12:0/3116390012 wait complete. 2024-09-11T00:24:03.572 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:03.568+0000 7fae51600640 1 --2- 172.21.15.12:0/4052879519 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fae3c0b51f0 0x7fae3c0b76b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:24:03.572 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:03.569+0000 7fae51600640 1 -- 172.21.15.12:0/4052879519 <== mon.2 v2:172.21.15.165:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9208+0+0 (secure 0 0 0) 0x7fae400e6380 con 0x7fae541058f0 2024-09-11T00:24:03.572 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:03.569+0000 7fae59000640 1 --2- 172.21.15.12:0/4052879519 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fae3c0b51f0 0x7fae3c0b76b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:24:03.574 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:03.569+0000 7fae5e8d0740 1 -- start start 2024-09-11T00:24:03.574 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:03.569+0000 7fae5e8d0740 1 Processor -- start 2024-09-11T00:24:03.574 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:03.569+0000 7fae59000640 1 --2- 172.21.15.12:0/4052879519 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fae3c0b51f0 0x7fae3c0b76b0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fae48014040 tx=0x7fae48000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:24:03.575 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:03.570+0000 7fae5e8d0740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558497f62970 0x558497f6c0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:24:03.575 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:03.570+0000 7fae5e8d0740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558497f652a0 0x558497f6c760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:24:03.575 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:03.570+0000 7fae5e8d0740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558497f67ce0 0x558497f6d330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:24:03.575 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:03.570+0000 7fae5e8d0740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x558497dfcab0 con 0x558497f652a0 2024-09-11T00:24:03.575 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:03.570+0000 7fae5e8d0740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x558497e6b570 con 0x558497f62970 2024-09-11T00:24:03.575 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:03.570+0000 7fae5e8d0740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x558497f6acc0 con 0x558497f67ce0 2024-09-11T00:24:03.575 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:03.570+0000 7fae5e8d0740 10 client.0 _ll_register_callbacks cb 0x558497e7a100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:24:03.575 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:03.570+0000 7fae53e00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558497f652a0 0x558497f6c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:24:03.575 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:03.570+0000 7fae59000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558497f62970 0x558497f6c0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:24:03.575 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:03.570+0000 7fae59a00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558497f67ce0 0x558497f6d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:24:03.575 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:03.570+0000 7fae53e00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558497f652a0 0x558497f6c760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:53018/0 (socket says 172.21.15.12:53018) 2024-09-11T00:24:03.575 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:03.570+0000 7fae53e00640 1 -- 172.21.15.12:0/3942409534 learned_addr learned my addr 172.21.15.12:0/3942409534 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:24:03.576 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:03.570+0000 7fae53e00640 1 -- 172.21.15.12:0/3942409534 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558497f67ce0 msgr2=0x558497f6d330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:24:03.576 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:03.570+0000 7fae53e00640 1 --2- 172.21.15.12:0/3942409534 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558497f67ce0 0x558497f6d330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:24:03.576 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:03.570+0000 7fae53e00640 1 -- 172.21.15.12:0/3942409534 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558497f62970 msgr2=0x558497f6c0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:24:03.576 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:03.570+0000 7fae53e00640 1 --2- 172.21.15.12:0/3942409534 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558497f62970 0x558497f6c0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:24:03.576 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:03.570+0000 7fae53e00640 1 -- 172.21.15.12:0/3942409534 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558497c26790 con 0x558497f652a0 2024-09-11T00:24:03.576 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:03.570+0000 7fae59000640 1 --2- 172.21.15.12:0/3942409534 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558497f62970 0x558497f6c0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:24:03.576 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:03.571+0000 7fae53e00640 1 --2- 172.21.15.12:0/3942409534 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558497f652a0 0x558497f6c760 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7fae400a99e0 tx=0x7fae400048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64debe8723e3d7bc server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:24:03.576 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:03.571+0000 7fae3aa00640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-09-11T00:24:03.576 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:03.571+0000 7fae3aa00640 1 -- 172.21.15.12:0/3942409534 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae400ace70 con 0x558497f652a0 2024-09-11T00:24:03.576 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:03.571+0000 7fae3aa00640 1 -- 172.21.15.12:0/3942409534 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9208+0+0 (secure 0 0 0) 0x7fae400b5870 con 0x558497f652a0 2024-09-11T00:24:03.576 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:03.571+0000 7fae5e8d0740 1 -- 172.21.15.12:0/3942409534 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fae4800faf0 con 0x558497f652a0 2024-09-11T00:24:03.576 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:03.571+0000 7fae3aa00640 1 -- 172.21.15.12:0/3942409534 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=66) ==== 24+0+0 (secure 0 0 0) 0x7fae400ac770 con 0x558497f652a0 2024-09-11T00:24:03.576 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:03.571+0000 7fae5e8d0740 10 client.17355 fetch_fsmap learned FSMap version 66 2024-09-11T00:24:03.577 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:03.571+0000 7fae5e8d0740 1 -- 172.21.15.12:0/3942409534 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=66,monmap=4+}) -- 0x558497c9f620 con 0x558497f652a0 2024-09-11T00:24:03.577 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:03.571+0000 7fae3aa00640 1 -- 172.21.15.12:0/3942409534 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fae4800faf0 con 0x558497f652a0 2024-09-11T00:24:03.577 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:03.572+0000 7fae3aa00640 1 -- 172.21.15.12:0/3942409534 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 66) ==== 8382+0+0 (secure 0 0 0) 0x7fae400c3080 con 0x558497f652a0 2024-09-11T00:24:03.577 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:03.572+0000 7fae5e8d0740 10 client.17355 fetch_fsmap finished waiting for FSMap version 66 2024-09-11T00:24:03.577 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:03.572+0000 7fae5e8d0740 10 client.17355 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T00:24:03.577 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:03.572+0000 7fae5e8d0740 20 client.17355 populate_metadata read hostname 'smithi012' 2024-09-11T00:24:03.577 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:03.572+0000 7fae5e8d0740 1 --2- 172.21.15.12:0/3942409534 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x558497f77a00 0x558497f79e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:24:03.577 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:03.572+0000 7fae5e8d0740 4 client.17355 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T00:24:03.577 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:03.572+0000 7fae5e8d0740 1 -- 172.21.15.12:0/3942409534 --> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558497ed7600 con 0x558497f77a00 2024-09-11T00:24:03.577 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:03.572+0000 7fae59000640 1 --2- 172.21.15.12:0/3942409534 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x558497f77a00 0x558497f79e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:24:03.579 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:03.573+0000 7fae59000640 1 --2- 172.21.15.12:0/3942409534 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x558497f77a00 0x558497f79e20 crc :-1 s=READY pgs=241 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7e95c196b95c04bd server_cookie=af1181801056afc4 in_seq=0 out_seq=0 2024-09-11T00:24:03.579 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:03.573+0000 7fae3aa00640 10 client.17355 ms_handle_connect on v2:172.21.15.165:6838/501312984 2024-09-11T00:24:03.579 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:03.573+0000 7fae3aa00640 1 -- 172.21.15.12:0/3942409534 <== mds.0 v2:172.21.15.165:6838/501312984 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x558497ed7600 con 0x558497f77a00 2024-09-11T00:24:03.579 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:03.573+0000 7fae3aa00640 10 client.17355 handle_command_reply: tid=0 2024-09-11T00:24:03.579 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:03.574+0000 7fae5e8d0740 1 client.17355 shutdown 2024-09-11T00:24:03.579 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:03.574+0000 7fae5e8d0740 1 -- 172.21.15.12:0/3942409534 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558497f652a0 msgr2=0x558497f6c760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:24:03.579 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:03.574+0000 7fae5e8d0740 1 --2- 172.21.15.12:0/3942409534 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558497f652a0 0x558497f6c760 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7fae400a99e0 tx=0x7fae400048f0 comp rx=0 tx=0).stop 2024-09-11T00:24:03.579 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:03.574+0000 7fae5e8d0740 1 -- 172.21.15.12:0/3942409534 shutdown_connections 2024-09-11T00:24:03.579 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:03.574+0000 7fae5e8d0740 1 --2- 172.21.15.12:0/3942409534 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x558497f77a00 0x558497f79e20 crc :-1 s=READY pgs=241 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:24:03.579 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:03.574+0000 7fae5e8d0740 1 --2- 172.21.15.12:0/3942409534 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558497f67ce0 0x558497f6d330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:24:03.579 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:03.574+0000 7fae5e8d0740 1 --2- 172.21.15.12:0/3942409534 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558497f652a0 0x558497f6c760 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:24:03.580 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:03.574+0000 7fae3aa00640 0 client.17355 ms_handle_reset on v2:172.21.15.165:6838/501312984 2024-09-11T00:24:03.580 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:03.574+0000 7fae5e8d0740 1 --2- 172.21.15.12:0/3942409534 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558497f62970 0x558497f6c0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:24:03.580 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:03.574+0000 7fae5e8d0740 1 -- 172.21.15.12:0/3942409534 >> 172.21.15.12:0/3942409534 conn(0x558497f5dac0 msgr2=0x558497f5cbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:24:03.580 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:03.574+0000 7fae5e8d0740 1 -- 172.21.15.12:0/3942409534 shutdown_connections 2024-09-11T00:24:03.580 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:03.574+0000 7fae5e8d0740 1 -- 172.21.15.12:0/3942409534 wait complete. 2024-09-11T00:24:03.580 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:03.574+0000 7fae5e8d0740 20 client.17355 trim_cache size 0 max 16384 2024-09-11T00:24:03.593 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:03.590+0000 7fae5e8d0740 1 Processor -- start 2024-09-11T00:24:03.593 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:03.590+0000 7fae5e8d0740 1 -- start start 2024-09-11T00:24:03.593 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:03.590+0000 7fae5e8d0740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558497f62970 0x558497f6b090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:24:03.593 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:03.590+0000 7fae5e8d0740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558497f652a0 0x558497f6b5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:24:03.593 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:03.590+0000 7fae5e8d0740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558497f67ce0 0x558497f5f8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:24:03.593 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:03.590+0000 7fae5e8d0740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fae541ad670 con 0x558497f652a0 2024-09-11T00:24:03.593 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:03.590+0000 7fae5e8d0740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fae541af1e0 con 0x558497f67ce0 2024-09-11T00:24:03.593 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:03.590+0000 7fae5e8d0740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fae541add70 con 0x558497f62970 2024-09-11T00:24:03.593 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:03.590+0000 7fae53e00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558497f652a0 0x558497f6b5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:24:03.593 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:03.590+0000 7fae53e00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558497f652a0 0x558497f6b5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:53020/0 (socket says 172.21.15.12:53020) 2024-09-11T00:24:03.594 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:03.590+0000 7fae53e00640 1 -- 172.21.15.12:0/3546848767 learned_addr learned my addr 172.21.15.12:0/3546848767 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:24:03.594 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:03.590+0000 7fae59a00640 1 --2- 172.21.15.12:0/3546848767 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558497f67ce0 0x558497f5f8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:24:03.594 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:03.590+0000 7fae59000640 1 --2- 172.21.15.12:0/3546848767 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558497f62970 0x558497f6b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:24:03.594 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:03.591+0000 7fae59a00640 1 -- 172.21.15.12:0/3546848767 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558497f62970 msgr2=0x558497f6b090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:24:03.594 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:03.591+0000 7fae59a00640 1 --2- 172.21.15.12:0/3546848767 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558497f62970 0x558497f6b090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:24:03.594 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:03.591+0000 7fae59a00640 1 -- 172.21.15.12:0/3546848767 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558497f652a0 msgr2=0x558497f6b5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:24:03.594 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:03.591+0000 7fae59a00640 1 --2- 172.21.15.12:0/3546848767 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558497f652a0 0x558497f6b5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:24:03.594 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:03.591+0000 7fae59a00640 1 -- 172.21.15.12:0/3546848767 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558497f76320 con 0x558497f67ce0 2024-09-11T00:24:03.594 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:03.591+0000 7fae53e00640 1 --2- 172.21.15.12:0/3546848767 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558497f652a0 0x558497f6b5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:24:03.598 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:03.591+0000 7fae59a00640 1 --2- 172.21.15.12:0/3546848767 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558497f67ce0 0x558497f5f8f0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fae4c006010 tx=0x7fae4c030800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cf4b66f2017cfd13 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:24:03.598 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:03.591+0000 7fae45600640 1 -- 172.21.15.12:0/3546848767 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae4c030900 con 0x558497f67ce0 2024-09-11T00:24:03.598 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:03.591+0000 7fae45600640 1 -- 172.21.15.12:0/3546848767 <== mon.2 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fae4c04b070 con 0x558497f67ce0 2024-09-11T00:24:03.598 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:03.591+0000 7fae45600640 1 -- 172.21.15.12:0/3546848767 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae4c004410 con 0x558497f67ce0 2024-09-11T00:24:03.598 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:03.592+0000 7fae5e8d0740 1 -- 172.21.15.12:0/3546848767 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558497f67ce0 msgr2=0x558497f5f8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:24:03.598 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:03.592+0000 7fae5e8d0740 1 --2- 172.21.15.12:0/3546848767 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558497f67ce0 0x558497f5f8f0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fae4c006010 tx=0x7fae4c030800 comp rx=0 tx=0).stop 2024-09-11T00:24:03.598 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:03.592+0000 7fae5e8d0740 1 -- 172.21.15.12:0/3546848767 shutdown_connections 2024-09-11T00:24:03.598 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:03.592+0000 7fae5e8d0740 1 --2- 172.21.15.12:0/3546848767 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558497f67ce0 0x558497f5f8f0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:24:03.598 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:03.592+0000 7fae5e8d0740 1 --2- 172.21.15.12:0/3546848767 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558497f652a0 0x558497f6b5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:24:03.599 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:03.592+0000 7fae5e8d0740 1 --2- 172.21.15.12:0/3546848767 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558497f62970 0x558497f6b090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:24:03.599 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:03.592+0000 7fae5e8d0740 1 -- 172.21.15.12:0/3546848767 >> 172.21.15.12:0/3546848767 conn(0x558497f5dac0 msgr2=0x558497f68170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:24:03.599 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:03.592+0000 7fae5e8d0740 1 -- 172.21.15.12:0/3546848767 shutdown_connections 2024-09-11T00:24:03.599 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:03.592+0000 7fae5e8d0740 1 -- 172.21.15.12:0/3546848767 wait complete. 2024-09-11T00:24:03.599 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:03.592+0000 7fae5e8d0740 1 -- start start 2024-09-11T00:24:03.599 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:03.593+0000 7fae5e8d0740 1 Processor -- start 2024-09-11T00:24:03.599 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:03.593+0000 7fae5e8d0740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558497f62970 0x558497f73850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:24:03.599 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:03.593+0000 7fae5e8d0740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558497f652a0 0x558497f72080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:24:03.599 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:03.593+0000 7fae5e8d0740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558497f67ce0 0x558497f725c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:24:03.599 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:03.593+0000 7fae5e8d0740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fae541aeb10 con 0x558497f652a0 2024-09-11T00:24:03.599 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:03.593+0000 7fae5e8d0740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fae541ae440 con 0x558497f67ce0 2024-09-11T00:24:03.599 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:03.593+0000 7fae5e8d0740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fae541ab650 con 0x558497f62970 2024-09-11T00:24:03.599 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:03.593+0000 7fae5e8d0740 10 client.0 _ll_register_callbacks cb 0x7fae40009720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:24:03.600 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:03.593+0000 7fae53e00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558497f652a0 0x558497f72080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:24:03.600 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:03.593+0000 7fae53e00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558497f652a0 0x558497f72080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:53022/0 (socket says 172.21.15.12:53022) 2024-09-11T00:24:03.600 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:03.593+0000 7fae53e00640 1 -- 172.21.15.12:0/2789210757 learned_addr learned my addr 172.21.15.12:0/2789210757 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:24:03.600 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:03.593+0000 7fae59000640 1 --2- 172.21.15.12:0/2789210757 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558497f62970 0x558497f73850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:24:03.600 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:03.593+0000 7fae59a00640 1 --2- 172.21.15.12:0/2789210757 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558497f67ce0 0x558497f725c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:24:03.600 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:03.594+0000 7fae53e00640 1 -- 172.21.15.12:0/2789210757 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558497f62970 msgr2=0x558497f73850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:24:03.600 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:03.594+0000 7fae53e00640 1 --2- 172.21.15.12:0/2789210757 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558497f62970 0x558497f73850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:24:03.600 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:03.594+0000 7fae53e00640 1 -- 172.21.15.12:0/2789210757 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558497f67ce0 msgr2=0x558497f725c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:24:03.600 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:03.594+0000 7fae53e00640 1 --2- 172.21.15.12:0/2789210757 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558497f67ce0 0x558497f725c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:24:03.600 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:03.594+0000 7fae53e00640 1 -- 172.21.15.12:0/2789210757 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558497f72b00 con 0x558497f652a0 2024-09-11T00:24:03.600 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:03.594+0000 7fae59000640 1 --2- 172.21.15.12:0/2789210757 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558497f62970 0x558497f73850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:24:03.600 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:03.594+0000 7fae53e00640 1 --2- 172.21.15.12:0/2789210757 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558497f652a0 0x558497f72080 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7fae4000c6a0 tx=0x7fae400cfc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d50e89ae06bf41f server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:24:03.600 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:03.594+0000 7fae3aa00640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-09-11T00:24:03.601 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:03.594+0000 7fae3aa00640 1 -- 172.21.15.12:0/2789210757 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae400abee0 con 0x558497f652a0 2024-09-11T00:24:03.601 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:03.594+0000 7fae3aa00640 1 -- 172.21.15.12:0/2789210757 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9208+0+0 (secure 0 0 0) 0x7fae400c72a0 con 0x558497f652a0 2024-09-11T00:24:03.601 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:03.594+0000 7fae5e8d0740 1 -- 172.21.15.12:0/2789210757 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fae4c04b070 con 0x558497f652a0 2024-09-11T00:24:03.601 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:03.594+0000 7fae3aa00640 1 -- 172.21.15.12:0/2789210757 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=66) ==== 24+0+0 (secure 0 0 0) 0x7fae400ace70 con 0x558497f652a0 2024-09-11T00:24:03.601 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:03.595+0000 7fae5e8d0740 10 client.17361 fetch_fsmap learned FSMap version 66 2024-09-11T00:24:03.601 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:03.595+0000 7fae5e8d0740 1 -- 172.21.15.12:0/2789210757 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=66,monmap=4+}) -- 0x558497fe55d0 con 0x558497f652a0 2024-09-11T00:24:03.601 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:03.595+0000 7fae3aa00640 1 -- 172.21.15.12:0/2789210757 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fae4c04b070 con 0x558497f652a0 2024-09-11T00:24:03.601 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:03.595+0000 7fae3aa00640 1 -- 172.21.15.12:0/2789210757 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 66) ==== 8382+0+0 (secure 0 0 0) 0x7fae400adaa0 con 0x558497f652a0 2024-09-11T00:24:03.601 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:03.595+0000 7fae5e8d0740 10 client.17361 fetch_fsmap finished waiting for FSMap version 66 2024-09-11T00:24:03.601 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:03.595+0000 7fae5e8d0740 10 client.17361 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T00:24:03.601 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:03.595+0000 7fae5e8d0740 20 client.17361 populate_metadata read hostname 'smithi012' 2024-09-11T00:24:03.601 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:03.595+0000 7fae5e8d0740 1 --2- 172.21.15.12:0/2789210757 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x558497fe5f80 0x558497fe83a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:24:03.601 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:03.595+0000 7fae5e8d0740 4 client.17361 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T00:24:03.602 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:03.595+0000 7fae5e8d0740 1 -- 172.21.15.12:0/2789210757 --> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] -- command(tid 0: {"prefix": "flush journal"}) -- 0x558497ed30b0 con 0x558497fe5f80 2024-09-11T00:24:03.602 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:03.596+0000 7fae59000640 1 --2- 172.21.15.12:0/2789210757 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x558497fe5f80 0x558497fe83a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:24:03.602 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:03.596+0000 7fae59000640 1 --2- 172.21.15.12:0/2789210757 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x558497fe5f80 0x558497fe83a0 crc :-1 s=READY pgs=242 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cb6e76c454b84376 server_cookie=827fd1732122b5a5 in_seq=0 out_seq=0 2024-09-11T00:24:03.602 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:03.596+0000 7fae3aa00640 10 client.17361 ms_handle_connect on v2:172.21.15.165:6838/501312984 2024-09-11T00:24:03.631 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:03.628+0000 7fae3aa00640 1 -- 172.21.15.12:0/2789210757 <== mds.0 v2:172.21.15.165:6838/501312984 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x558497ed30b0 con 0x558497fe5f80 2024-09-11T00:24:03.631 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:03.628+0000 7fae3aa00640 10 client.17361 handle_command_reply: tid=0 2024-09-11T00:24:03.631 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:03.628+0000 7fae5e8d0740 1 client.17361 shutdown 2024-09-11T00:24:03.631 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:03.628+0000 7fae5e8d0740 1 -- 172.21.15.12:0/2789210757 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558497f652a0 msgr2=0x558497f72080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:24:03.631 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:03.628+0000 7fae5e8d0740 1 --2- 172.21.15.12:0/2789210757 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558497f652a0 0x558497f72080 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7fae4000c6a0 tx=0x7fae400cfc20 comp rx=0 tx=0).stop 2024-09-11T00:24:03.631 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:03.628+0000 7fae5e8d0740 1 -- 172.21.15.12:0/2789210757 shutdown_connections 2024-09-11T00:24:03.632 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:03.628+0000 7fae5e8d0740 1 --2- 172.21.15.12:0/2789210757 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x558497fe5f80 0x558497fe83a0 crc :-1 s=READY pgs=242 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:24:03.632 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:03.628+0000 7fae5e8d0740 1 --2- 172.21.15.12:0/2789210757 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558497f67ce0 0x558497f725c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:24:03.632 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:03.628+0000 7fae5e8d0740 1 --2- 172.21.15.12:0/2789210757 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558497f652a0 0x558497f72080 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:24:03.632 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:03.628+0000 7fae3aa00640 0 client.17361 ms_handle_reset on v2:172.21.15.165:6838/501312984 2024-09-11T00:24:03.632 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:03.628+0000 7fae5e8d0740 1 --2- 172.21.15.12:0/2789210757 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558497f62970 0x558497f73850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:24:03.632 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:03.628+0000 7fae5e8d0740 1 -- 172.21.15.12:0/2789210757 >> 172.21.15.12:0/2789210757 conn(0x558497f5dac0 msgr2=0x558497f6b090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:24:03.632 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:03.628+0000 7fae5e8d0740 1 -- 172.21.15.12:0/2789210757 shutdown_connections 2024-09-11T00:24:03.632 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:03.628+0000 7fae5e8d0740 1 -- 172.21.15.12:0/2789210757 wait complete. 2024-09-11T00:24:03.632 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:03.629+0000 7fae5e8d0740 20 client.17361 trim_cache size 0 max 16384 2024-09-11T00:24:03.633 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:{ 2024-09-11T00:24:03.633 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "message": "", 2024-09-11T00:24:03.633 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "return_code": 0 2024-09-11T00:24:03.633 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:} 2024-09-11T00:24:03.634 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:03.629+0000 7fae5a400640 1 -- 172.21.15.12:0/4052879519 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fae3c0b51f0 msgr2=0x7fae3c0b76b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:24:03.634 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:03.629+0000 7fae5a400640 1 --2- 172.21.15.12:0/4052879519 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fae3c0b51f0 0x7fae3c0b76b0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fae48014040 tx=0x7fae48000bc0 comp rx=0 tx=0).stop 2024-09-11T00:24:03.634 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:03.629+0000 7fae5a400640 1 -- 172.21.15.12:0/4052879519 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fae541058f0 msgr2=0x7fae54101300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:24:03.634 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:03.629+0000 7fae5a400640 1 --2- 172.21.15.12:0/4052879519 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fae541058f0 0x7fae54101300 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fae4000e9f0 tx=0x7fae4000edf0 comp rx=0 tx=0).stop 2024-09-11T00:24:03.634 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:03.629+0000 7fae5a400640 1 -- 172.21.15.12:0/4052879519 shutdown_connections 2024-09-11T00:24:03.634 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:03.629+0000 7fae5a400640 1 --2- 172.21.15.12:0/4052879519 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fae3c0b51f0 0x7fae3c0b76b0 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:24:03.634 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:03.629+0000 7fae5a400640 1 --2- 172.21.15.12:0/4052879519 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fae541084a0 0x7fae54101840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:24:03.634 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:03.629+0000 7fae5a400640 1 --2- 172.21.15.12:0/4052879519 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fae541058f0 0x7fae54101300 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:24:03.635 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:03.629+0000 7fae5a400640 1 --2- 172.21.15.12:0/4052879519 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fae5406b470 0x7fae54102aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:24:03.635 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:03.629+0000 7fae5a400640 1 -- 172.21.15.12:0/4052879519 >> 172.21.15.12:0/4052879519 conn(0x7fae540fcf30 msgr2=0x7fae54103b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:24:03.635 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:03.629+0000 7fae5a400640 1 -- 172.21.15.12:0/4052879519 shutdown_connections 2024-09-11T00:24:03.635 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:03.629+0000 7fae5a400640 1 -- 172.21.15.12:0/4052879519 wait complete. 2024-09-11T00:24:04.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:24:03 smithi165 ceph-mon[35356]: pgmap v908: 129 pgs: 129 active+clean; 493 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 313 KiB/s wr, 32 op/s 2024-09-11T00:24:04.063 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:24:03 smithi012 ceph-mon[30333]: pgmap v908: 129 pgs: 129 active+clean; 493 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 313 KiB/s wr, 32 op/s 2024-09-11T00:24:04.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:24:03 smithi178 ceph-mon[35919]: pgmap v908: 129 pgs: 129 active+clean; 493 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 313 KiB/s wr, 32 op/s 2024-09-11T00:24:04.178 INFO:tasks.workunit.client.0.smithi012.stdout: 10 69399 16.34 MB/sec execute 120 sec latency 23501.459 ms 2024-09-11T00:24:05.179 INFO:tasks.workunit.client.0.smithi012.stdout: 10 69399 16.20 MB/sec execute 121 sec latency 24501.591 ms 2024-09-11T00:24:06.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:24:05 smithi165 ceph-mon[35356]: pgmap v909: 129 pgs: 129 active+clean; 493 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 303 KiB/s wr, 23 op/s 2024-09-11T00:24:06.063 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:24:05 smithi012 ceph-mon[30333]: pgmap v909: 129 pgs: 129 active+clean; 493 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 303 KiB/s wr, 23 op/s 2024-09-11T00:24:06.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:24:05 smithi178 ceph-mon[35919]: pgmap v909: 129 pgs: 129 active+clean; 493 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 303 KiB/s wr, 23 op/s 2024-09-11T00:24:06.178 INFO:tasks.workunit.client.0.smithi012.stdout: 10 69399 16.07 MB/sec execute 122 sec latency 25501.748 ms 2024-09-11T00:24:06.857 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid ff3ad74a-6fcf-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T00:24:07.179 INFO:tasks.workunit.client.0.smithi012.stdout: 10 69399 15.94 MB/sec execute 123 sec latency 26501.811 ms 2024-09-11T00:24:07.251 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:24:07.675 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:24:07 smithi012 ceph-mon[30333]: pgmap v910: 129 pgs: 129 active+clean; 492 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 314 KiB/s wr, 36 op/s 2024-09-11T00:24:07.807 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:24:07.787+0000 7fe054a00640 1 -- 172.21.15.12:0/3229125398 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe050075540 msgr2=0x7fe050075920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:24:07.807 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:24:07.787+0000 7fe054a00640 1 --2- 172.21.15.12:0/3229125398 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe050075540 0x7fe050075920 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fe040009900 tx=0x7fe04002a400 comp rx=0 tx=0).stop 2024-09-11T00:24:07.807 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:24:07.787+0000 7fe054a00640 1 -- 172.21.15.12:0/3229125398 shutdown_connections 2024-09-11T00:24:07.807 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:24:07.787+0000 7fe054a00640 1 --2- 172.21.15.12:0/3229125398 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe05010f3e0 0x7fe0501117d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:24:07.808 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:24:07.787+0000 7fe054a00640 1 --2- 172.21.15.12:0/3229125398 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe050075f40 0x7fe05010ee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:24:07.808 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:24:07.787+0000 7fe054a00640 1 --2- 172.21.15.12:0/3229125398 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe050075540 0x7fe050075920 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:24:07.808 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:24:07.787+0000 7fe054a00640 1 -- 172.21.15.12:0/3229125398 >> 172.21.15.12:0/3229125398 conn(0x7fe0500fe830 msgr2=0x7fe050100c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:24:07.808 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:24:07.787+0000 7fe054a00640 1 -- 172.21.15.12:0/3229125398 shutdown_connections 2024-09-11T00:24:07.808 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:24:07.787+0000 7fe054a00640 1 -- 172.21.15.12:0/3229125398 wait complete. 2024-09-11T00:24:07.808 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:24:07.788+0000 7fe054a00640 1 Processor -- start 2024-09-11T00:24:07.808 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:24:07.788+0000 7fe054a00640 1 -- start start 2024-09-11T00:24:07.808 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:24:07.789+0000 7fe054a00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe050075540 0x7fe05019b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:24:07.808 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:24:07.789+0000 7fe054a00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe050075f40 0x7fe05019ba00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:24:07.809 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:24:07.789+0000 7fe054a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe05010f3e0 0x7fe0501a0da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:24:07.809 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:24:07.789+0000 7fe054a00640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fe050114fd0 con 0x7fe050075f40 2024-09-11T00:24:07.809 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:24:07.789+0000 7fe054a00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fe050114e50 con 0x7fe05010f3e0 2024-09-11T00:24:07.809 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:24:07.789+0000 7fe054a00640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fe050115150 con 0x7fe050075540 2024-09-11T00:24:07.809 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:24:07.789+0000 7fe045400640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe050075f40 0x7fe05019ba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:24:07.809 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:24:07.789+0000 7fe04de00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe050075540 0x7fe05019b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:24:07.809 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:24:07.789+0000 7fe045400640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe050075f40 0x7fe05019ba00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:53054/0 (socket says 172.21.15.12:53054) 2024-09-11T00:24:07.809 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:24:07.789+0000 7fe04de00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe050075540 0x7fe05019b4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.12:44090/0 (socket says 172.21.15.12:44090) 2024-09-11T00:24:07.809 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:24:07.789+0000 7fe045400640 1 -- 172.21.15.12:0/3070633337 learned_addr learned my addr 172.21.15.12:0/3070633337 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:24:07.809 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:24:07.789+0000 7fe04e800640 1 --2- 172.21.15.12:0/3070633337 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe05010f3e0 0x7fe0501a0da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:24:07.809 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:24:07.790+0000 7fe04de00640 1 -- 172.21.15.12:0/3070633337 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe05010f3e0 msgr2=0x7fe0501a0da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:24:07.810 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:24:07.790+0000 7fe04de00640 1 --2- 172.21.15.12:0/3070633337 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe05010f3e0 0x7fe0501a0da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:24:07.810 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:24:07.790+0000 7fe04de00640 1 -- 172.21.15.12:0/3070633337 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe050075f40 msgr2=0x7fe05019ba00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:24:07.810 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:24:07.790+0000 7fe04de00640 1 --2- 172.21.15.12:0/3070633337 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe050075f40 0x7fe05019ba00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:24:07.810 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:24:07.790+0000 7fe04de00640 1 -- 172.21.15.12:0/3070633337 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0501a1570 con 0x7fe050075540 2024-09-11T00:24:07.810 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:24:07.790+0000 7fe04de00640 1 --2- 172.21.15.12:0/3070633337 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe050075540 0x7fe05019b4c0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fe04002a9f0 tx=0x7fe040003b90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:24:07.810 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:24:07.790+0000 7fe047400640 1 -- 172.21.15.12:0/3070633337 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe04003f070 con 0x7fe050075540 2024-09-11T00:24:07.810 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:24:07.790+0000 7fe047400640 1 -- 172.21.15.12:0/3070633337 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe040031b50 con 0x7fe050075540 2024-09-11T00:24:07.810 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:24:07.790+0000 7fe047400640 1 -- 172.21.15.12:0/3070633337 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe040035030 con 0x7fe050075540 2024-09-11T00:24:07.810 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:24:07.790+0000 7fe054a00640 1 -- 172.21.15.12:0/3070633337 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe0501a1800 con 0x7fe050075540 2024-09-11T00:24:07.810 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:24:07.790+0000 7fe054a00640 1 -- 172.21.15.12:0/3070633337 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe0501a7f20 con 0x7fe050075540 2024-09-11T00:24:07.810 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:24:07.791+0000 7fe054a00640 1 -- 172.21.15.12:0/3070633337 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe050075920 con 0x7fe050075540 2024-09-11T00:24:07.811 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:24:07.793+0000 7fe047400640 1 -- 172.21.15.12:0/3070633337 <== mon.1 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7fe0400316a0 con 0x7fe050075540 2024-09-11T00:24:07.811 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:24:07.795+0000 7fe047400640 1 --2- 172.21.15.12:0/3070633337 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fe0340b52e0 0x7fe0340b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:24:07.811 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:24:07.795+0000 7fe047400640 1 -- 172.21.15.12:0/3070633337 <== mon.1 v2:172.21.15.178:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9208+0+0 (secure 0 0 0) 0x7fe040106a00 con 0x7fe050075540 2024-09-11T00:24:07.811 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:24:07.795+0000 7fe045400640 1 --2- 172.21.15.12:0/3070633337 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fe0340b52e0 0x7fe0340b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:24:07.811 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:24:07.796+0000 7fe045400640 1 --2- 172.21.15.12:0/3070633337 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fe0340b52e0 0x7fe0340b77a0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fe050101080 tx=0x7fe038005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:24:07.811 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:24:07.797+0000 7fe047400640 1 -- 172.21.15.12:0/3070633337 <== mon.1 v2:172.21.15.178:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7fe040031d00 con 0x7fe050075540 2024-09-11T00:24:08.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:24:07 smithi165 ceph-mon[35356]: pgmap v910: 129 pgs: 129 active+clean; 492 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 314 KiB/s wr, 36 op/s 2024-09-11T00:24:08.038 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:24:08.036+0000 7fe054a00640 1 -- 172.21.15.12:0/3070633337 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe0501a8210 con 0x7fe050075540 2024-09-11T00:24:08.039 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:24:08.037+0000 7fe047400640 1 -- 172.21.15.12:0/3070633337 <== mon.1 v2:172.21.15.178:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 66 v66) ==== 94+0+11969 (secure 0 0 0) 0x7fe0400cce40 con 0x7fe050075540 2024-09-11T00:24:08.039 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-11T00:24:08.040 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":66,"btime":"2024-09-11T00:23:57:596568+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14577,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6835/447171591","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":447171591},{"type":"v1","addr":"172.21.15.12:6835","nonce":447171591}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14589,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/901486040","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":901486040},{"type":"v1","addr":"172.21.15.12:6837","nonce":901486040}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":66,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T00:04:04.739881+0000","modified":"2024-09-11T00:23:56.974153+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24674,"mds_1":24643,"mds_2":24638,"mds_3":24637,"mds_4":24614},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14631":{"gid":14631,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.12:6841/3069919437","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6840","nonce":3069919437},{"type":"v1","addr":"172.21.15.12:6841","nonce":3069919437}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24562":{"gid":24562,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6833/1141636112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":1141636112},{"type":"v1","addr":"172.21.15.178:6833","nonce":1141636112}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24581":{"gid":24581,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6833/3111980970","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6832","nonce":3111980970},{"type":"v1","addr":"172.21.15.165:6833","nonce":3111980970}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24607":{"gid":24607,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6835/4084868287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":4084868287},{"type":"v1","addr":"172.21.15.178:6835","nonce":4084868287}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24614":{"gid":24614,"name":"e","rank":4,"incarnation":26,"state":"up:active","state_seq":21,"addr":"172.21.15.165:6835/2425395274","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":2425395274},{"type":"v1","addr":"172.21.15.165:6835","nonce":2425395274}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24637":{"gid":24637,"name":"i","rank":3,"incarnation":24,"state":"up:active","state_seq":18,"addr":"172.21.15.178:6839/70628097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6838","nonce":70628097},{"type":"v1","addr":"172.21.15.178:6839","nonce":70628097}]},"join_fscid":-1,"export_targets":[1],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24638":{"gid":24638,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":18,"addr":"172.21.15.165:6837/6215088","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6836","nonce":6215088},{"type":"v1","addr":"172.21.15.165:6837","nonce":6215088}]},"join_fscid":-1,"export_targets":[0,3],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24643":{"gid":24643,"name":"g","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.12:6839/3124093009","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6838","nonce":3124093009},{"type":"v1","addr":"172.21.15.12:6839","nonce":3124093009}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24674":{"gid":24674,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.165:6839/501312984","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6838","nonce":501312984},{"type":"v1","addr":"172.21.15.165:6839","nonce":501312984}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25672":{"gid":25672,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6837/308601741","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":308601741},{"type":"v1","addr":"172.21.15.178:6837","nonce":308601741}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24674,"qdb_cluster":[24674,24643,24638,24637,24614]},"id":1}]} 2024-09-11T00:24:08.040 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 66 2024-09-11T00:24:08.042 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:24:08.039+0000 7fe054a00640 1 -- 172.21.15.12:0/3070633337 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fe0340b52e0 msgr2=0x7fe0340b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:24:08.042 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:24:08.039+0000 7fe054a00640 1 --2- 172.21.15.12:0/3070633337 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fe0340b52e0 0x7fe0340b77a0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fe050101080 tx=0x7fe038005f90 comp rx=0 tx=0).stop 2024-09-11T00:24:08.042 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:24:08.039+0000 7fe054a00640 1 -- 172.21.15.12:0/3070633337 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe050075540 msgr2=0x7fe05019b4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:24:08.042 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:24:08.039+0000 7fe054a00640 1 --2- 172.21.15.12:0/3070633337 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe050075540 0x7fe05019b4c0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fe04002a9f0 tx=0x7fe040003b90 comp rx=0 tx=0).stop 2024-09-11T00:24:08.042 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:24:08.039+0000 7fe054a00640 1 -- 172.21.15.12:0/3070633337 shutdown_connections 2024-09-11T00:24:08.042 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:24:08.039+0000 7fe054a00640 1 --2- 172.21.15.12:0/3070633337 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fe0340b52e0 0x7fe0340b77a0 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:24:08.043 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:24:08.039+0000 7fe054a00640 1 --2- 172.21.15.12:0/3070633337 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe05010f3e0 0x7fe0501a0da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:24:08.043 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:24:08.039+0000 7fe054a00640 1 --2- 172.21.15.12:0/3070633337 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe050075f40 0x7fe05019ba00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:24:08.043 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:24:08.039+0000 7fe054a00640 1 --2- 172.21.15.12:0/3070633337 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe050075540 0x7fe05019b4c0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:24:08.043 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:24:08.039+0000 7fe054a00640 1 -- 172.21.15.12:0/3070633337 >> 172.21.15.12:0/3070633337 conn(0x7fe0500fe830 msgr2=0x7fe0501115a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:24:08.043 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:24:08.039+0000 7fe054a00640 1 -- 172.21.15.12:0/3070633337 shutdown_connections 2024-09-11T00:24:08.043 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:24:08.040+0000 7fe054a00640 1 -- 172.21.15.12:0/3070633337 wait complete. 2024-09-11T00:24:08.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:24:07 smithi178 ceph-mon[35919]: pgmap v910: 129 pgs: 129 active+clean; 492 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 314 KiB/s wr, 36 op/s 2024-09-11T00:24:08.179 INFO:tasks.workunit.client.0.smithi012.stdout: 10 69399 15.81 MB/sec execute 124 sec latency 27501.920 ms 2024-09-11T00:24:08.753 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:24:08 smithi012 ceph-mon[30333]: from='client.? 172.21.15.12:0/3070633337' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T00:24:08.792 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24674 quiesce db --set-id 49e32b0e --release --await' 2024-09-11T00:24:08.792 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid ff3ad74a-6fcf-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24674 quiesce db --set-id 49e32b0e --release --await 2024-09-11T00:24:09.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:24:08 smithi165 ceph-mon[35356]: from='client.? 172.21.15.12:0/3070633337' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T00:24:09.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:24:08 smithi178 ceph-mon[35919]: from='client.? 172.21.15.12:0/3070633337' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T00:24:09.177 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:24:09.178 INFO:tasks.workunit.client.0.smithi012.stdout: 10 69399 15.68 MB/sec execute 125 sec latency 28501.979 ms 2024-09-11T00:24:09.817 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:24:09.784+0000 7fb88c600640 1 -- 172.21.15.12:0/1522182158 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb884104fa0 msgr2=0x7fb884105380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:24:09.817 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:24:09.784+0000 7fb88c600640 1 --2- 172.21.15.12:0/1522182158 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb884104fa0 0x7fb884105380 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7fb878009900 tx=0x7fb87802a3f0 comp rx=0 tx=0).stop 2024-09-11T00:24:09.817 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:24:09.784+0000 7fb88c600640 1 -- 172.21.15.12:0/1522182158 shutdown_connections 2024-09-11T00:24:09.817 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:24:09.784+0000 7fb88c600640 1 --2- 172.21.15.12:0/1522182158 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb88410b330 0x7fb884111b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:24:09.818 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:24:09.784+0000 7fb88c600640 1 --2- 172.21.15.12:0/1522182158 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb88410a9b0 0x7fb88410adf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:24:09.818 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:24:09.784+0000 7fb88c600640 1 --2- 172.21.15.12:0/1522182158 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb884104fa0 0x7fb884105380 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:24:09.818 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:24:09.784+0000 7fb88c600640 1 -- 172.21.15.12:0/1522182158 >> 172.21.15.12:0/1522182158 conn(0x7fb884100a50 msgr2=0x7fb884102e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:24:09.818 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:24:09.784+0000 7fb88c600640 1 -- 172.21.15.12:0/1522182158 shutdown_connections 2024-09-11T00:24:09.818 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:24:09.784+0000 7fb88c600640 1 -- 172.21.15.12:0/1522182158 wait complete. 2024-09-11T00:24:09.818 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:24:09.785+0000 7fb88c600640 1 Processor -- start 2024-09-11T00:24:09.818 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:24:09.785+0000 7fb88c600640 1 -- start start 2024-09-11T00:24:09.818 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:24:09.786+0000 7fb88c600640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb884104fa0 0x7fb88419b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:24:09.818 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:24:09.786+0000 7fb88c600640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb88410a9b0 0x7fb88419ba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:24:09.819 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:24:09.786+0000 7fb88c600640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb88410b330 0x7fb8841a0dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:24:09.819 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:24:09.786+0000 7fb88c600640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fb884114fb0 con 0x7fb884104fa0 2024-09-11T00:24:09.819 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:24:09.786+0000 7fb88c600640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fb884114e30 con 0x7fb88410b330 2024-09-11T00:24:09.819 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:24:09.786+0000 7fb88c600640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fb884115130 con 0x7fb88410a9b0 2024-09-11T00:24:09.819 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:24:09.786+0000 7fb889c00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb884104fa0 0x7fb88419b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:24:09.819 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:24:09.786+0000 7fb889c00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb884104fa0 0x7fb88419b4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:55876/0 (socket says 172.21.15.12:55876) 2024-09-11T00:24:09.819 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:24:09.786+0000 7fb889c00640 1 -- 172.21.15.12:0/352714567 learned_addr learned my addr 172.21.15.12:0/352714567 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:24:09.819 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:24:09.786+0000 7fb88a600640 1 --2- 172.21.15.12:0/352714567 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb88410b330 0x7fb8841a0dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:24:09.819 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:24:09.786+0000 7fb889200640 1 --2- 172.21.15.12:0/352714567 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb88410a9b0 0x7fb88419ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:24:09.820 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:24:09.786+0000 7fb889c00640 1 -- 172.21.15.12:0/352714567 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb88410a9b0 msgr2=0x7fb88419ba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:24:09.820 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:24:09.786+0000 7fb889c00640 1 --2- 172.21.15.12:0/352714567 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb88410a9b0 0x7fb88419ba20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:24:09.820 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:24:09.786+0000 7fb889c00640 1 -- 172.21.15.12:0/352714567 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb88410b330 msgr2=0x7fb8841a0dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:24:09.820 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:24:09.786+0000 7fb889c00640 1 --2- 172.21.15.12:0/352714567 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb88410b330 0x7fb8841a0dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:24:09.820 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:24:09.786+0000 7fb889c00640 1 -- 172.21.15.12:0/352714567 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8841a1590 con 0x7fb884104fa0 2024-09-11T00:24:09.820 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:24:09.787+0000 7fb889c00640 1 --2- 172.21.15.12:0/352714567 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb884104fa0 0x7fb88419b4e0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7fb878005ab0 tx=0x7fb878003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:24:09.820 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:24:09.787+0000 7fb882000640 1 -- 172.21.15.12:0/352714567 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb87803f070 con 0x7fb884104fa0 2024-09-11T00:24:09.820 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:24:09.787+0000 7fb882000640 1 -- 172.21.15.12:0/352714567 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb878031b60 con 0x7fb884104fa0 2024-09-11T00:24:09.821 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:24:09.787+0000 7fb882000640 1 -- 172.21.15.12:0/352714567 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb878035030 con 0x7fb884104fa0 2024-09-11T00:24:09.821 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:24:09.787+0000 7fb88c600640 1 -- 172.21.15.12:0/352714567 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb8841a1820 con 0x7fb884104fa0 2024-09-11T00:24:09.821 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:24:09.787+0000 7fb88c600640 1 -- 172.21.15.12:0/352714567 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8841a7f40 con 0x7fb884104fa0 2024-09-11T00:24:09.821 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:24:09.790+0000 7fb88f1ca740 1 Processor -- start 2024-09-11T00:24:09.821 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:24:09.791+0000 7fb88f1ca740 1 -- start start 2024-09-11T00:24:09.821 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:24:09.791+0000 7fb882000640 1 -- 172.21.15.12:0/352714567 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7fb8780316b0 con 0x7fb884104fa0 2024-09-11T00:24:09.821 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:24:09.791+0000 7fb88f1ca740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5581d19bbff0 0x5581d19bc3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:24:09.821 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:24:09.791+0000 7fb88f1ca740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5581d19bc910 0x5581d19c8360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:24:09.821 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:24:09.791+0000 7fb88f1ca740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5581d19c88a0 0x5581d19cac90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:24:09.821 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:24:09.791+0000 7fb88f1ca740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x5581d1933290 con 0x5581d19c88a0 2024-09-11T00:24:09.821 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:24:09.791+0000 7fb88f1ca740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x5581d19226c0 con 0x5581d19bbff0 2024-09-11T00:24:09.821 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:24:09.791+0000 7fb88f1ca740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5581d19264d0 con 0x5581d19bc910 2024-09-11T00:24:09.822 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:24:09.791+0000 7fb889200640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5581d19c88a0 0x5581d19cac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:24:09.822 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:24:09.791+0000 7fb88a600640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5581d19bbff0 0x5581d19bc3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:24:09.822 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:24:09.791+0000 7fb88a600640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5581d19bbff0 0x5581d19bc3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.12:57986/0 (socket says 172.21.15.12:57986) 2024-09-11T00:24:09.822 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:24:09.791+0000 7fb88a600640 1 -- 172.21.15.12:0/4067502055 learned_addr learned my addr 172.21.15.12:0/4067502055 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:24:09.822 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:24:09.791+0000 7fb889c00640 1 --2- 172.21.15.12:0/4067502055 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5581d19bc910 0x5581d19c8360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:24:09.822 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:24:09.792+0000 7fb88a600640 1 -- 172.21.15.12:0/4067502055 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5581d19bc910 msgr2=0x5581d19c8360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:24:09.822 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:24:09.792+0000 7fb88a600640 1 --2- 172.21.15.12:0/4067502055 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5581d19bc910 0x5581d19c8360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:24:09.822 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:24:09.792+0000 7fb88a600640 1 -- 172.21.15.12:0/4067502055 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5581d19c88a0 msgr2=0x5581d19cac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:24:09.822 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:24:09.792+0000 7fb88a600640 1 --2- 172.21.15.12:0/4067502055 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5581d19c88a0 0x5581d19cac90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:24:09.822 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:24:09.792+0000 7fb88a600640 1 -- 172.21.15.12:0/4067502055 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5581d18af460 con 0x5581d19bbff0 2024-09-11T00:24:09.823 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:24:09.792+0000 7fb889200640 1 --2- 172.21.15.12:0/4067502055 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5581d19c88a0 0x5581d19cac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:24:09.823 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:24:09.792+0000 7fb88a600640 1 --2- 172.21.15.12:0/4067502055 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5581d19bbff0 0x5581d19bc3d0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fb87c011440 tx=0x7fb87c011900 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6e92db609cb7850f server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:24:09.823 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:24:09.792+0000 7fb876a00640 1 -- 172.21.15.12:0/4067502055 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb87c003c30 con 0x5581d19bbff0 2024-09-11T00:24:09.823 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:24:09.792+0000 7fb876a00640 1 -- 172.21.15.12:0/4067502055 <== mon.2 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb87c003dd0 con 0x5581d19bbff0 2024-09-11T00:24:09.823 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:24:09.793+0000 7fb876a00640 1 -- 172.21.15.12:0/4067502055 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb87c00fa40 con 0x5581d19bbff0 2024-09-11T00:24:09.823 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:24:09.793+0000 7fb882000640 1 --2- 172.21.15.12:0/352714567 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fb86c0b5360 0x7fb86c0b7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:24:09.823 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:24:09.793+0000 7fb882000640 1 -- 172.21.15.12:0/352714567 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9208+0+0 (secure 0 0 0) 0x7fb878106560 con 0x7fb884104fa0 2024-09-11T00:24:09.823 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:24:09.793+0000 7fb889c00640 1 --2- 172.21.15.12:0/352714567 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fb86c0b5360 0x7fb86c0b7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:24:09.823 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:24:09.793+0000 7fb88f1ca740 1 -- 172.21.15.12:0/4067502055 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5581d19bbff0 msgr2=0x5581d19bc3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:24:09.823 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:24:09.793+0000 7fb88f1ca740 1 --2- 172.21.15.12:0/4067502055 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5581d19bbff0 0x5581d19bc3d0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fb87c011440 tx=0x7fb87c011900 comp rx=0 tx=0).stop 2024-09-11T00:24:09.824 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:24:09.793+0000 7fb88f1ca740 1 -- 172.21.15.12:0/4067502055 shutdown_connections 2024-09-11T00:24:09.824 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:24:09.793+0000 7fb88f1ca740 1 --2- 172.21.15.12:0/4067502055 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5581d19c88a0 0x5581d19cac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:24:09.824 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:24:09.793+0000 7fb88f1ca740 1 --2- 172.21.15.12:0/4067502055 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5581d19bc910 0x5581d19c8360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:24:09.824 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:24:09.793+0000 7fb88f1ca740 1 --2- 172.21.15.12:0/4067502055 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5581d19bbff0 0x5581d19bc3d0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:24:09.824 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:24:09.793+0000 7fb88f1ca740 1 -- 172.21.15.12:0/4067502055 >> 172.21.15.12:0/4067502055 conn(0x5581d19b9e20 msgr2=0x5581d19bafc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:24:09.824 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:24:09.793+0000 7fb889c00640 1 --2- 172.21.15.12:0/352714567 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fb86c0b5360 0x7fb86c0b7820 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fb878106800 tx=0x7fb87810c060 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:24:09.824 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:24:09.794+0000 7fb88f1ca740 1 -- 172.21.15.12:0/4067502055 shutdown_connections 2024-09-11T00:24:09.824 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:24:09.794+0000 7fb88f1ca740 1 -- 172.21.15.12:0/4067502055 wait complete. 2024-09-11T00:24:09.824 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:24:09.794+0000 7fb88f1ca740 1 -- start start 2024-09-11T00:24:09.824 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:24:09.794+0000 7fb88f1ca740 1 Processor -- start 2024-09-11T00:24:09.825 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:24:09.795+0000 7fb88f1ca740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5581d19bbff0 0x5581d19c5890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:24:09.825 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:24:09.795+0000 7fb88f1ca740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5581d19bc910 0x5581d19c5dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:24:09.825 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:24:09.795+0000 7fb88f1ca740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5581d19c88a0 0x5581d19c6310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:24:09.825 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:24:09.795+0000 7fb88f1ca740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x5581d1783bb0 con 0x5581d19c88a0 2024-09-11T00:24:09.825 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:24:09.795+0000 7fb88f1ca740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x5581d18bd6c0 con 0x5581d19bc910 2024-09-11T00:24:09.825 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:24:09.795+0000 7fb88f1ca740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5581d19cb550 con 0x5581d19bbff0 2024-09-11T00:24:09.825 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:24:09.795+0000 7fb88f1ca740 10 client.0 _ll_register_callbacks cb 0x5581d18d8490 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:24:09.825 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:24:09.795+0000 7fb88a600640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5581d19c88a0 0x5581d19c6310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:24:09.825 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:24:09.795+0000 7fb88a600640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5581d19bbff0 0x5581d19c5890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:24:09.825 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:24:09.795+0000 7fb889200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5581d19bc910 0x5581d19c5dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:24:09.825 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:24:09.795+0000 7fb889200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5581d19bc910 0x5581d19c5dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.12:58002/0 (socket says 172.21.15.12:58002) 2024-09-11T00:24:09.825 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:24:09.795+0000 7fb889200640 1 -- 172.21.15.12:0/3666934869 learned_addr learned my addr 172.21.15.12:0/3666934869 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:24:09.826 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:24:09.795+0000 7fb889200640 1 -- 172.21.15.12:0/3666934869 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5581d19bbff0 msgr2=0x5581d19c5890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:24:09.826 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:24:09.795+0000 7fb889200640 1 --2- 172.21.15.12:0/3666934869 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5581d19bbff0 0x5581d19c5890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:24:09.826 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:24:09.796+0000 7fb889200640 1 -- 172.21.15.12:0/3666934869 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5581d19c88a0 msgr2=0x5581d19c6310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:24:09.826 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:24:09.796+0000 7fb889200640 1 --2- 172.21.15.12:0/3666934869 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5581d19c88a0 0x5581d19c6310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:24:09.826 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:24:09.796+0000 7fb889200640 1 -- 172.21.15.12:0/3666934869 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5581d1680b00 con 0x5581d19bc910 2024-09-11T00:24:09.826 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:24:09.796+0000 7fb889200640 1 --2- 172.21.15.12:0/3666934869 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5581d19bc910 0x5581d19c5dd0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fb8700126e0 tx=0x7fb870012ba0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=17cf9f19206e224d server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:24:09.826 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:24:09.796+0000 7fb86b400640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-09-11T00:24:09.826 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:24:09.796+0000 7fb86b400640 1 -- 172.21.15.12:0/3666934869 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb870004050 con 0x5581d19bc910 2024-09-11T00:24:09.826 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:24:09.796+0000 7fb88f1ca740 1 -- 172.21.15.12:0/3666934869 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb87c003dd0 con 0x5581d19bc910 2024-09-11T00:24:09.827 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:24:09.796+0000 7fb86b400640 1 -- 172.21.15.12:0/3666934869 <== mon.2 v2:172.21.15.165:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9208+0+0 (secure 0 0 0) 0x7fb8700041f0 con 0x5581d19bc910 2024-09-11T00:24:09.827 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:24:09.796+0000 7fb86b400640 1 -- 172.21.15.12:0/3666934869 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=66) ==== 24+0+0 (secure 0 0 0) 0x7fb870025050 con 0x5581d19bc910 2024-09-11T00:24:09.827 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:24:09.797+0000 7fb88f1ca740 10 client.27383 fetch_fsmap learned FSMap version 66 2024-09-11T00:24:09.827 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:24:09.797+0000 7fb88f1ca740 1 -- 172.21.15.12:0/3666934869 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=66,monmap=4+}) -- 0x5581d16f7960 con 0x5581d19bc910 2024-09-11T00:24:09.827 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:24:09.797+0000 7fb86b400640 1 -- 172.21.15.12:0/3666934869 <== mon.2 v2:172.21.15.165:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb87c003dd0 con 0x5581d19bc910 2024-09-11T00:24:09.827 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:24:09.797+0000 7fb86b400640 1 -- 172.21.15.12:0/3666934869 <== mon.2 v2:172.21.15.165:3300/0 5 ==== fsmap(e 66) ==== 8382+0+0 (secure 0 0 0) 0x7fb870003a10 con 0x5581d19bc910 2024-09-11T00:24:09.827 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:24:09.797+0000 7fb88f1ca740 10 client.27383 fetch_fsmap finished waiting for FSMap version 66 2024-09-11T00:24:09.828 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:24:09.797+0000 7fb88f1ca740 10 client.27383 resolve_mds: validated gid 24674 aka daemon mds.b 2024-09-11T00:24:09.828 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:24:09.797+0000 7fb88f1ca740 20 client.27383 populate_metadata read hostname 'smithi012' 2024-09-11T00:24:09.828 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:24:09.797+0000 7fb88f1ca740 1 --2- 172.21.15.12:0/3666934869 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x5581d19d84f0 0x5581d19da8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:24:09.828 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:24:09.797+0000 7fb88f1ca740 4 client.27383 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T00:24:09.828 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:24:09.797+0000 7fb88f1ca740 1 -- 172.21.15.12:0/3666934869 --> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5581d19351e0 con 0x5581d19d84f0 2024-09-11T00:24:09.828 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:24:09.798+0000 7fb88a600640 1 --2- 172.21.15.12:0/3666934869 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x5581d19d84f0 0x5581d19da8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:24:09.828 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:24:09.798+0000 7fb88a600640 1 --2- 172.21.15.12:0/3666934869 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x5581d19d84f0 0x5581d19da8e0 crc :-1 s=READY pgs=243 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f2cc56e8bf8e258a server_cookie=8d64cb7df346a900 in_seq=0 out_seq=0 2024-09-11T00:24:09.829 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:24:09.798+0000 7fb86b400640 10 client.27383 ms_handle_connect on v2:172.21.15.165:6838/501312984 2024-09-11T00:24:09.829 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:24:09.799+0000 7fb86b400640 1 -- 172.21.15.12:0/3666934869 <== mds.0 v2:172.21.15.165:6838/501312984 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5581d19351e0 con 0x5581d19d84f0 2024-09-11T00:24:09.829 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:24:09.799+0000 7fb86b400640 10 client.27383 handle_command_reply: tid=0 2024-09-11T00:24:09.829 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:24:09.799+0000 7fb88f1ca740 1 client.27383 shutdown 2024-09-11T00:24:09.829 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:24:09.799+0000 7fb88f1ca740 1 -- 172.21.15.12:0/3666934869 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5581d19bc910 msgr2=0x5581d19c5dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:24:09.829 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:24:09.799+0000 7fb88f1ca740 1 --2- 172.21.15.12:0/3666934869 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5581d19bc910 0x5581d19c5dd0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fb8700126e0 tx=0x7fb870012ba0 comp rx=0 tx=0).stop 2024-09-11T00:24:09.829 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:24:09.799+0000 7fb88f1ca740 1 -- 172.21.15.12:0/3666934869 shutdown_connections 2024-09-11T00:24:09.829 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:24:09.799+0000 7fb88f1ca740 1 --2- 172.21.15.12:0/3666934869 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x5581d19d84f0 0x5581d19da8e0 crc :-1 s=READY pgs=243 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:24:09.829 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:24:09.799+0000 7fb88f1ca740 1 --2- 172.21.15.12:0/3666934869 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5581d19c88a0 0x5581d19c6310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:24:09.829 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:24:09.799+0000 7fb88f1ca740 1 --2- 172.21.15.12:0/3666934869 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5581d19bc910 0x5581d19c5dd0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:24:09.830 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:24:09.799+0000 7fb88f1ca740 1 --2- 172.21.15.12:0/3666934869 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5581d19bbff0 0x5581d19c5890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:24:09.830 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:24:09.799+0000 7fb86b400640 0 client.27383 ms_handle_reset on v2:172.21.15.165:6838/501312984 2024-09-11T00:24:09.830 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:24:09.799+0000 7fb88f1ca740 1 -- 172.21.15.12:0/3666934869 >> 172.21.15.12:0/3666934869 conn(0x5581d19b9e20 msgr2=0x5581d19bf9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:24:09.830 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:24:09.799+0000 7fb88f1ca740 1 -- 172.21.15.12:0/3666934869 shutdown_connections 2024-09-11T00:24:09.830 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:24:09.799+0000 7fb88f1ca740 1 -- 172.21.15.12:0/3666934869 wait complete. 2024-09-11T00:24:09.830 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:24:09.799+0000 7fb88f1ca740 20 client.27383 trim_cache size 0 max 16384 2024-09-11T00:24:09.835 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:24:09.818+0000 7fb88f1ca740 1 Processor -- start 2024-09-11T00:24:09.836 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:24:09.818+0000 7fb88f1ca740 1 -- start start 2024-09-11T00:24:09.836 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:24:09.818+0000 7fb88f1ca740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5581d19bbff0 0x5581d19c0cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:24:09.836 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:24:09.818+0000 7fb88f1ca740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5581d19bc910 0x5581d19b8c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:24:09.836 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:24:09.818+0000 7fb88f1ca740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5581d19c88a0 0x5581d19c2aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:24:09.836 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:24:09.818+0000 7fb88f1ca740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fb8841b1e50 con 0x5581d19c88a0 2024-09-11T00:24:09.836 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:24:09.818+0000 7fb88f1ca740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fb8841b39c0 con 0x5581d19bbff0 2024-09-11T00:24:09.836 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:24:09.818+0000 7fb88f1ca740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fb8841b2550 con 0x5581d19bc910 2024-09-11T00:24:09.836 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:24:09.819+0000 7fb889200640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5581d19bc910 0x5581d19b8c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:24:09.836 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:24:09.819+0000 7fb889200640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5581d19bc910 0x5581d19b8c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.12:47670/0 (socket says 172.21.15.12:47670) 2024-09-11T00:24:09.836 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:24:09.819+0000 7fb889200640 1 -- 172.21.15.12:0/3967021674 learned_addr learned my addr 172.21.15.12:0/3967021674 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:24:09.836 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:24:09.819+0000 7fb88a600640 1 --2- 172.21.15.12:0/3967021674 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5581d19c88a0 0x5581d19c2aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:24:09.837 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:24:09.819+0000 7fb88a600640 1 --2- 172.21.15.12:0/3967021674 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5581d19bbff0 0x5581d19c0cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:24:09.837 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:24:09.819+0000 7fb889200640 1 -- 172.21.15.12:0/3967021674 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5581d19bbff0 msgr2=0x5581d19c0cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:24:09.837 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:24:09.819+0000 7fb889200640 1 --2- 172.21.15.12:0/3967021674 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5581d19bbff0 0x5581d19c0cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:24:09.837 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:24:09.819+0000 7fb889200640 1 -- 172.21.15.12:0/3967021674 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5581d19c88a0 msgr2=0x5581d19c2aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:24:09.837 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:24:09.819+0000 7fb889200640 1 --2- 172.21.15.12:0/3967021674 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5581d19c88a0 0x5581d19c2aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:24:09.837 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:24:09.819+0000 7fb889200640 1 -- 172.21.15.12:0/3967021674 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5581d19d70a0 con 0x5581d19bc910 2024-09-11T00:24:09.837 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:24:09.819+0000 7fb88a600640 1 --2- 172.21.15.12:0/3967021674 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5581d19c88a0 0x5581d19c2aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:24:09.837 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:24:09.819+0000 7fb889200640 1 --2- 172.21.15.12:0/3967021674 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5581d19bc910 0x5581d19b8c00 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fb870012420 tx=0x7fb87000f810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=317fb426a81a7cb3 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:24:09.837 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:24:09.820+0000 7fb876000640 1 -- 172.21.15.12:0/3967021674 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb87002e070 con 0x5581d19bc910 2024-09-11T00:24:09.837 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:24:09.820+0000 7fb876000640 1 -- 172.21.15.12:0/3967021674 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb870005590 con 0x5581d19bc910 2024-09-11T00:24:09.837 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:24:09.820+0000 7fb876000640 1 -- 172.21.15.12:0/3967021674 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8700035f0 con 0x5581d19bc910 2024-09-11T00:24:09.837 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:24:09.820+0000 7fb88f1ca740 1 -- 172.21.15.12:0/3967021674 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5581d19bc910 msgr2=0x5581d19b8c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:24:09.838 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:24:09.820+0000 7fb88f1ca740 1 --2- 172.21.15.12:0/3967021674 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5581d19bc910 0x5581d19b8c00 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fb870012420 tx=0x7fb87000f810 comp rx=0 tx=0).stop 2024-09-11T00:24:09.838 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:24:09.820+0000 7fb88f1ca740 1 -- 172.21.15.12:0/3967021674 shutdown_connections 2024-09-11T00:24:09.838 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:24:09.820+0000 7fb88f1ca740 1 --2- 172.21.15.12:0/3967021674 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5581d19c88a0 0x5581d19c2aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:24:09.838 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:24:09.820+0000 7fb88f1ca740 1 --2- 172.21.15.12:0/3967021674 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5581d19bc910 0x5581d19b8c00 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:24:09.838 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:24:09.820+0000 7fb88f1ca740 1 --2- 172.21.15.12:0/3967021674 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5581d19bbff0 0x5581d19c0cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:24:09.838 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:24:09.820+0000 7fb88f1ca740 1 -- 172.21.15.12:0/3967021674 >> 172.21.15.12:0/3967021674 conn(0x5581d19b9e20 msgr2=0x5581d19c76e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:24:09.838 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:24:09.821+0000 7fb88f1ca740 1 -- 172.21.15.12:0/3967021674 shutdown_connections 2024-09-11T00:24:09.838 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:24:09.821+0000 7fb88f1ca740 1 -- 172.21.15.12:0/3967021674 wait complete. 2024-09-11T00:24:09.838 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:24:09.821+0000 7fb88f1ca740 1 -- start start 2024-09-11T00:24:09.838 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:24:09.821+0000 7fb88f1ca740 1 Processor -- start 2024-09-11T00:24:09.838 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:24:09.822+0000 7fb88f1ca740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5581d19bbff0 0x5581d19cd8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:24:09.838 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:24:09.822+0000 7fb88f1ca740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5581d19bc910 0x5581d19cdde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:24:09.838 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:24:09.822+0000 7fb88f1ca740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5581d19c88a0 0x5581d19ce320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:24:09.839 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:24:09.822+0000 7fb88f1ca740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fb8841b32f0 con 0x5581d19bbff0 2024-09-11T00:24:09.839 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:24:09.822+0000 7fb88f1ca740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fb8841b2c20 con 0x5581d19c88a0 2024-09-11T00:24:09.839 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:24:09.822+0000 7fb88f1ca740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fb8841afe30 con 0x5581d19bc910 2024-09-11T00:24:09.839 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:24:09.822+0000 7fb88f1ca740 10 client.0 _ll_register_callbacks cb 0x7fb870009910 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:24:09.839 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:24:09.822+0000 7fb889200640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5581d19bc910 0x5581d19cdde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:24:09.839 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:24:09.822+0000 7fb889200640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5581d19bc910 0x5581d19cdde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.12:47674/0 (socket says 172.21.15.12:47674) 2024-09-11T00:24:09.839 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:24:09.822+0000 7fb88a600640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5581d19bbff0 0x5581d19cd8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:24:09.839 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:24:09.822+0000 7fb889200640 1 -- 172.21.15.12:0/2471193960 learned_addr learned my addr 172.21.15.12:0/2471193960 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:24:09.839 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:24:09.822+0000 7fb88a600640 1 --2- 172.21.15.12:0/2471193960 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5581d19c88a0 0x5581d19ce320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:24:09.839 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:24:09.822+0000 7fb889200640 1 -- 172.21.15.12:0/2471193960 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5581d19c88a0 msgr2=0x5581d19ce320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:24:09.839 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:24:09.822+0000 7fb889200640 1 --2- 172.21.15.12:0/2471193960 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5581d19c88a0 0x5581d19ce320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:24:09.839 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:24:09.822+0000 7fb889200640 1 -- 172.21.15.12:0/2471193960 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5581d19bbff0 msgr2=0x5581d19cd8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:24:09.840 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:24:09.822+0000 7fb889200640 1 --2- 172.21.15.12:0/2471193960 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5581d19bbff0 0x5581d19cd8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:24:09.840 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:24:09.822+0000 7fb889200640 1 -- 172.21.15.12:0/2471193960 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5581d19ba8a0 con 0x5581d19bc910 2024-09-11T00:24:09.840 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:24:09.822+0000 7fb88a600640 1 --2- 172.21.15.12:0/2471193960 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5581d19c88a0 0x5581d19ce320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T00:24:09.840 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:24:09.823+0000 7fb889200640 1 --2- 172.21.15.12:0/2471193960 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5581d19bc910 0x5581d19cdde0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fb870012be0 tx=0x7fb87000fb50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f1f7db97c73a92 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:24:09.840 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:24:09.823+0000 7fb86b400640 10 client.0 ms_handle_connect on v2:172.21.15.178:3300/0 2024-09-11T00:24:09.840 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:24:09.823+0000 7fb86b400640 1 -- 172.21.15.12:0/2471193960 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb87002e070 con 0x5581d19bc910 2024-09-11T00:24:09.840 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:24:09.823+0000 7fb88f1ca740 1 -- 172.21.15.12:0/2471193960 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb870005590 con 0x5581d19bc910 2024-09-11T00:24:09.840 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:24:09.823+0000 7fb86b400640 1 -- 172.21.15.12:0/2471193960 <== mon.1 v2:172.21.15.178:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9208+0+0 (secure 0 0 0) 0x7fb870022040 con 0x5581d19bc910 2024-09-11T00:24:09.840 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:24:09.823+0000 7fb86b400640 1 -- 172.21.15.12:0/2471193960 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=66) ==== 24+0+0 (secure 0 0 0) 0x7fb87003c050 con 0x5581d19bc910 2024-09-11T00:24:09.840 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:24:09.823+0000 7fb88f1ca740 10 client.27361 fetch_fsmap learned FSMap version 66 2024-09-11T00:24:09.840 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:24:09.823+0000 7fb88f1ca740 1 -- 172.21.15.12:0/2471193960 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,fsmap=66,monmap=4+}) -- 0x5581d1a46640 con 0x5581d19bc910 2024-09-11T00:24:09.841 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:24:09.824+0000 7fb86b400640 1 -- 172.21.15.12:0/2471193960 <== mon.1 v2:172.21.15.178:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb870005590 con 0x5581d19bc910 2024-09-11T00:24:09.841 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:24:09.824+0000 7fb86b400640 1 -- 172.21.15.12:0/2471193960 <== mon.1 v2:172.21.15.178:3300/0 5 ==== fsmap(e 66) ==== 8382+0+0 (secure 0 0 0) 0x7fb870010600 con 0x5581d19bc910 2024-09-11T00:24:09.841 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:24:09.824+0000 7fb88f1ca740 10 client.27361 fetch_fsmap finished waiting for FSMap version 66 2024-09-11T00:24:09.841 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:24:09.824+0000 7fb88f1ca740 10 client.27361 resolve_mds: validated gid 24674 aka daemon mds.b 2024-09-11T00:24:09.841 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:24:09.824+0000 7fb88f1ca740 20 client.27361 populate_metadata read hostname 'smithi012' 2024-09-11T00:24:09.841 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:24:09.824+0000 7fb88f1ca740 1 --2- 172.21.15.12:0/2471193960 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x5581d1a46d30 0x5581d1a49150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:24:09.841 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:24:09.824+0000 7fb88f1ca740 4 client.27361 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "49e32b0e", "release": true, "await": true}] 2024-09-11T00:24:09.841 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:24:09.824+0000 7fb88f1ca740 1 -- 172.21.15.12:0/2471193960 --> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] -- command(tid 0: {"prefix": "quiesce db", "set_id": "49e32b0e", "release": true, "await": true}) -- 0x5581d1930c90 con 0x5581d1a46d30 2024-09-11T00:24:09.841 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:24:09.824+0000 7fb88a600640 1 --2- 172.21.15.12:0/2471193960 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x5581d1a46d30 0x5581d1a49150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:24:09.841 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:24:09.825+0000 7fb88a600640 1 --2- 172.21.15.12:0/2471193960 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x5581d1a46d30 0x5581d1a49150 crc :-1 s=READY pgs=244 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=15c5a6436f36f1ff server_cookie=ffc6e1a434f83f8d in_seq=0 out_seq=0 2024-09-11T00:24:09.841 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:24:09.825+0000 7fb86b400640 10 client.27361 ms_handle_connect on v2:172.21.15.165:6838/501312984 2024-09-11T00:24:09.858 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:24:09.855+0000 7fb86b400640 1 -- 172.21.15.12:0/2471193960 <== mds.0 v2:172.21.15.165:6838/501312984 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x5581d1930c90 con 0x5581d1a46d30 2024-09-11T00:24:09.858 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:24:09.855+0000 7fb86b400640 10 client.27361 handle_command_reply: tid=0 2024-09-11T00:24:09.858 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:24:09.855+0000 7fb88f1ca740 1 client.27361 shutdown 2024-09-11T00:24:09.858 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:24:09.855+0000 7fb88f1ca740 1 -- 172.21.15.12:0/2471193960 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5581d19bc910 msgr2=0x5581d19cdde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:24:09.858 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:24:09.855+0000 7fb88f1ca740 1 --2- 172.21.15.12:0/2471193960 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5581d19bc910 0x5581d19cdde0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fb870012be0 tx=0x7fb87000fb50 comp rx=0 tx=0).stop 2024-09-11T00:24:09.858 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:24:09.855+0000 7fb88f1ca740 1 -- 172.21.15.12:0/2471193960 shutdown_connections 2024-09-11T00:24:09.858 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:24:09.855+0000 7fb88f1ca740 1 --2- 172.21.15.12:0/2471193960 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x5581d1a46d30 0x5581d1a49150 crc :-1 s=READY pgs=244 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:24:09.858 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:24:09.855+0000 7fb88f1ca740 1 --2- 172.21.15.12:0/2471193960 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5581d19c88a0 0x5581d19ce320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:24:09.859 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:24:09.855+0000 7fb88f1ca740 1 --2- 172.21.15.12:0/2471193960 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5581d19bc910 0x5581d19cdde0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:24:09.859 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:24:09.855+0000 7fb86b400640 0 client.27361 ms_handle_reset on v2:172.21.15.165:6838/501312984 2024-09-11T00:24:09.859 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:24:09.855+0000 7fb88f1ca740 1 --2- 172.21.15.12:0/2471193960 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5581d19bbff0 0x5581d19cd8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:24:09.859 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:24:09.855+0000 7fb88f1ca740 1 -- 172.21.15.12:0/2471193960 >> 172.21.15.12:0/2471193960 conn(0x5581d19b9e20 msgr2=0x5581d19c4110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:24:09.859 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:24:09.856+0000 7fb88f1ca740 1 -- 172.21.15.12:0/2471193960 shutdown_connections 2024-09-11T00:24:09.859 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:24:09.856+0000 7fb88f1ca740 1 -- 172.21.15.12:0/2471193960 wait complete. 2024-09-11T00:24:09.859 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:24:09.856+0000 7fb88f1ca740 20 client.27361 trim_cache size 0 max 16384 2024-09-11T00:24:09.860 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-09-11T00:24:09.860 INFO:teuthology.orchestra.run.smithi012.stdout: "epoch": 66, 2024-09-11T00:24:09.860 INFO:teuthology.orchestra.run.smithi012.stdout: "leader": 24674, 2024-09-11T00:24:09.860 INFO:teuthology.orchestra.run.smithi012.stdout: "set_version": 52, 2024-09-11T00:24:09.860 INFO:teuthology.orchestra.run.smithi012.stdout: "sets": { 2024-09-11T00:24:09.860 INFO:teuthology.orchestra.run.smithi012.stdout: "49e32b0e": { 2024-09-11T00:24:09.860 INFO:teuthology.orchestra.run.smithi012.stdout: "version": 52, 2024-09-11T00:24:09.860 INFO:teuthology.orchestra.run.smithi012.stdout: "age_ref": 0.0, 2024-09-11T00:24:09.860 INFO:teuthology.orchestra.run.smithi012.stdout: "state": { 2024-09-11T00:24:09.861 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "RELEASED", 2024-09-11T00:24:09.861 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 0.0 2024-09-11T00:24:09.861 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:24:09.861 INFO:teuthology.orchestra.run.smithi012.stdout: "timeout": 90.0, 2024-09-11T00:24:09.861 INFO:teuthology.orchestra.run.smithi012.stdout: "expiration": 154.8, 2024-09-11T00:24:09.861 INFO:teuthology.orchestra.run.smithi012.stdout: "members": { 2024-09-11T00:24:09.861 INFO:teuthology.orchestra.run.smithi012.stdout: "file:/": { 2024-09-11T00:24:09.861 INFO:teuthology.orchestra.run.smithi012.stdout: "excluded": false, 2024-09-11T00:24:09.861 INFO:teuthology.orchestra.run.smithi012.stdout: "state": { 2024-09-11T00:24:09.861 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "RELEASED", 2024-09-11T00:24:09.861 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 0.0 2024-09-11T00:24:09.861 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:24:09.861 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:24:09.861 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:24:09.862 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:24:09.862 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:24:09.862 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-09-11T00:24:09.862 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:24:09.857+0000 7fb88c600640 1 -- 172.21.15.12:0/352714567 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fb86c0b5360 msgr2=0x7fb86c0b7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:24:09.862 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:24:09.857+0000 7fb88c600640 1 --2- 172.21.15.12:0/352714567 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fb86c0b5360 0x7fb86c0b7820 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fb878106800 tx=0x7fb87810c060 comp rx=0 tx=0).stop 2024-09-11T00:24:09.862 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:24:09.857+0000 7fb88c600640 1 -- 172.21.15.12:0/352714567 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb884104fa0 msgr2=0x7fb88419b4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:24:09.863 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:24:09.857+0000 7fb88c600640 1 --2- 172.21.15.12:0/352714567 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb884104fa0 0x7fb88419b4e0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7fb878005ab0 tx=0x7fb878003c10 comp rx=0 tx=0).stop 2024-09-11T00:24:09.863 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:24:09.858+0000 7fb88c600640 1 -- 172.21.15.12:0/352714567 shutdown_connections 2024-09-11T00:24:09.863 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:24:09.858+0000 7fb88c600640 1 --2- 172.21.15.12:0/352714567 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fb86c0b5360 0x7fb86c0b7820 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:24:09.863 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:24:09.858+0000 7fb88c600640 1 --2- 172.21.15.12:0/352714567 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb88410b330 0x7fb8841a0dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:24:09.863 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:24:09.858+0000 7fb88c600640 1 --2- 172.21.15.12:0/352714567 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb88410a9b0 0x7fb88419ba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:24:09.863 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:24:09.858+0000 7fb88c600640 1 --2- 172.21.15.12:0/352714567 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb884104fa0 0x7fb88419b4e0 secure :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7fb878005ab0 tx=0x7fb878003c10 comp rx=0 tx=0).stop 2024-09-11T00:24:09.863 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:24:09.858+0000 7fb88c600640 1 -- 172.21.15.12:0/352714567 >> 172.21.15.12:0/352714567 conn(0x7fb884100a50 msgr2=0x7fb884102e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:24:09.863 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:24:09.858+0000 7fb88c600640 1 -- 172.21.15.12:0/352714567 shutdown_connections 2024-09-11T00:24:09.863 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:24:09.858+0000 7fb88c600640 1 -- 172.21.15.12:0/352714567 wait complete. 2024-09-11T00:24:10.033 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:24:09 smithi165 ceph-mon[35356]: pgmap v911: 129 pgs: 129 active+clean; 492 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 307 KiB/s wr, 25 op/s 2024-09-11T00:24:10.040 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:24:09 smithi012 ceph-mon[30333]: pgmap v911: 129 pgs: 129 active+clean; 492 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 307 KiB/s wr, 25 op/s 2024-09-11T00:24:10.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:24:09 smithi178 ceph-mon[35919]: pgmap v911: 129 pgs: 129 active+clean; 492 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 307 KiB/s wr, 25 op/s 2024-09-11T00:24:10.179 INFO:tasks.workunit.client.0.smithi012.stdout: 10 69590 15.61 MB/sec execute 126 sec latency 29155.619 ms 2024-09-11T00:24:10.769 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '49e32b0e', total seconds elapsed: 38.7 2024-09-11T00:24:11.179 INFO:tasks.workunit.client.0.smithi012.stdout: 10 70433 15.83 MB/sec execute 127 sec latency 124.609 ms 2024-09-11T00:24:12.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:24:11 smithi165 ceph-mon[35356]: pgmap v912: 129 pgs: 129 active+clean; 493 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 420 KiB/s rd, 353 KiB/s wr, 31 op/s 2024-09-11T00:24:12.063 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:24:11 smithi012 ceph-mon[30333]: pgmap v912: 129 pgs: 129 active+clean; 493 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 420 KiB/s rd, 353 KiB/s wr, 31 op/s 2024-09-11T00:24:12.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:24:11 smithi178 ceph-mon[35919]: pgmap v912: 129 pgs: 129 active+clean; 493 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 420 KiB/s rd, 353 KiB/s wr, 31 op/s 2024-09-11T00:24:12.179 INFO:tasks.workunit.client.0.smithi012.stdout: 10 71409 16.06 MB/sec execute 128 sec latency 185.656 ms 2024-09-11T00:24:13.179 INFO:tasks.workunit.client.0.smithi012.stdout: 10 72413 16.46 MB/sec execute 129 sec latency 199.066 ms 2024-09-11T00:24:14.004 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:24:13 smithi012 ceph-mon[30333]: pgmap v913: 129 pgs: 129 active+clean; 493 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 373 KiB/s wr, 63 op/s 2024-09-11T00:24:14.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:24:13 smithi165 ceph-mon[35356]: pgmap v913: 129 pgs: 129 active+clean; 493 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 373 KiB/s wr, 63 op/s 2024-09-11T00:24:14.082 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:24:13 smithi178 ceph-mon[35919]: pgmap v913: 129 pgs: 129 active+clean; 493 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 373 KiB/s wr, 63 op/s 2024-09-11T00:24:14.179 INFO:tasks.workunit.client.0.smithi012.stdout: 10 73052 16.60 MB/sec execute 130 sec latency 366.351 ms 2024-09-11T00:24:15.179 INFO:tasks.workunit.client.0.smithi012.stdout: 10 73667 16.67 MB/sec execute 131 sec latency 279.565 ms 2024-09-11T00:24:15.813 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:24:15 smithi012 ceph-mon[30333]: pgmap v914: 129 pgs: 129 active+clean; 493 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 357 KiB/s wr, 62 op/s 2024-09-11T00:24:16.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:24:15 smithi165 ceph-mon[35356]: pgmap v914: 129 pgs: 129 active+clean; 493 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 357 KiB/s wr, 62 op/s 2024-09-11T00:24:16.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:24:15 smithi178 ceph-mon[35919]: pgmap v914: 129 pgs: 129 active+clean; 493 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 357 KiB/s wr, 62 op/s 2024-09-11T00:24:16.179 INFO:tasks.workunit.client.0.smithi012.stdout: 10 74485 16.86 MB/sec execute 132 sec latency 403.367 ms 2024-09-11T00:24:16.827 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:16.824+0000 7f5206200640 1 -- 172.21.15.12:0/3890068329 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f52001066b0 msgr2=0x7f520010d030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:24:16.827 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:16.824+0000 7f5206200640 1 --2- 172.21.15.12:0/3890068329 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f52001066b0 0x7f520010d030 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f51f8009ca0 tx=0x7f51f802a8c0 comp rx=0 tx=0).stop 2024-09-11T00:24:16.828 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:16.825+0000 7f5206200640 1 -- 172.21.15.12:0/3890068329 shutdown_connections 2024-09-11T00:24:16.828 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:16.825+0000 7f5206200640 1 --2- 172.21.15.12:0/3890068329 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f52001066b0 0x7f520010d030 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:24:16.828 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:16.825+0000 7f5206200640 1 --2- 172.21.15.12:0/3890068329 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5200105cf0 0x7f5200106170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:24:16.828 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:16.825+0000 7f5206200640 1 --2- 172.21.15.12:0/3890068329 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5200104ba0 0x7f5200104fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:24:16.828 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:16.825+0000 7f5206200640 1 -- 172.21.15.12:0/3890068329 >> 172.21.15.12:0/3890068329 conn(0x7f5200100460 msgr2=0x7f5200102880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:24:16.828 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:16.825+0000 7f5206200640 1 -- 172.21.15.12:0/3890068329 shutdown_connections 2024-09-11T00:24:16.829 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:16.826+0000 7f5206200640 1 -- 172.21.15.12:0/3890068329 wait complete. 2024-09-11T00:24:16.829 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:16.827+0000 7f5206200640 1 Processor -- start 2024-09-11T00:24:16.831 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:16.828+0000 7f5206200640 1 -- start start 2024-09-11T00:24:16.831 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:16.828+0000 7f5206200640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5200104ba0 0x7f520019af50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:24:16.831 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:16.828+0000 7f5206200640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5200105cf0 0x7f520019b5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:24:16.831 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:16.828+0000 7f5206200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f52001066b0 0x7f520019c1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:24:16.831 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:16.828+0000 7f5206200640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f52001108c0 con 0x7f5200104ba0 2024-09-11T00:24:16.832 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:16.828+0000 7f5206200640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f5200110740 con 0x7f52001066b0 2024-09-11T00:24:16.832 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:16.828+0000 7f5206200640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f5200110a40 con 0x7f5200105cf0 2024-09-11T00:24:16.832 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:16.829+0000 7f51fea00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5200105cf0 0x7f520019b5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:24:16.832 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:16.829+0000 7f51fea00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5200105cf0 0x7f520019b5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.12:47692/0 (socket says 172.21.15.12:47692) 2024-09-11T00:24:16.832 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:16.829+0000 7f51ffe00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f52001066b0 0x7f520019c1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:24:16.832 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:16.829+0000 7f51ffe00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f52001066b0 0x7f520019c1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.12:58032/0 (socket says 172.21.15.12:58032) 2024-09-11T00:24:16.832 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:16.829+0000 7f51ffe00640 1 -- 172.21.15.12:0/2698383519 learned_addr learned my addr 172.21.15.12:0/2698383519 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:24:16.832 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:16.829+0000 7f51ff400640 1 --2- 172.21.15.12:0/2698383519 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5200104ba0 0x7f520019af50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:24:16.833 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:16.830+0000 7f51ffe00640 1 -- 172.21.15.12:0/2698383519 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5200105cf0 msgr2=0x7f520019b5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:24:16.833 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:16.830+0000 7f51ffe00640 1 --2- 172.21.15.12:0/2698383519 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5200105cf0 0x7f520019b5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:24:16.833 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:16.830+0000 7f51ffe00640 1 -- 172.21.15.12:0/2698383519 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5200104ba0 msgr2=0x7f520019af50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:24:16.833 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:16.831+0000 7f51ffe00640 1 --2- 172.21.15.12:0/2698383519 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5200104ba0 0x7f520019af50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:24:16.834 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:16.831+0000 7f51ffe00640 1 -- 172.21.15.12:0/2698383519 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f520019ca90 con 0x7f52001066b0 2024-09-11T00:24:16.834 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:16.831+0000 7f51ff400640 1 --2- 172.21.15.12:0/2698383519 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5200104ba0 0x7f520019af50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T00:24:16.834 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:16.831+0000 7f51ffe00640 1 --2- 172.21.15.12:0/2698383519 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f52001066b0 0x7f520019c1d0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f51f8003ba0 tx=0x7f51f8003bd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:24:16.834 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:16.831+0000 7f51f7e00640 1 -- 172.21.15.12:0/2698383519 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51f80317c0 con 0x7f52001066b0 2024-09-11T00:24:16.834 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:16.831+0000 7f51f7e00640 1 -- 172.21.15.12:0/2698383519 <== mon.2 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f51f80044f0 con 0x7f52001066b0 2024-09-11T00:24:16.835 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:16.831+0000 7f51f7e00640 1 -- 172.21.15.12:0/2698383519 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51f802fc90 con 0x7f52001066b0 2024-09-11T00:24:16.835 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:16.832+0000 7f5206200640 1 -- 172.21.15.12:0/2698383519 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f520019cd20 con 0x7f52001066b0 2024-09-11T00:24:16.835 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:16.832+0000 7f5206200640 1 -- 172.21.15.12:0/2698383519 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f520006cdd0 con 0x7f52001066b0 2024-09-11T00:24:16.839 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:16.837+0000 7f51f7e00640 1 -- 172.21.15.12:0/2698383519 <== mon.2 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7f51f8004040 con 0x7f52001066b0 2024-09-11T00:24:16.842 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:16.840+0000 7f52091ca740 1 Processor -- start 2024-09-11T00:24:16.842 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:16.840+0000 7f52091ca740 1 -- start start 2024-09-11T00:24:16.843 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:16.840+0000 7f52091ca740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563bfbef5fa0 0x563bfbef8390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:24:16.843 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:16.840+0000 7f52091ca740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563bfbef88d0 0x563bfbefacc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:24:16.843 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:16.840+0000 7f52091ca740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563bfbefb310 0x563bfbefd7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:24:16.843 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:16.840+0000 7f52091ca740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x563bfbe606b0 con 0x563bfbef88d0 2024-09-11T00:24:16.843 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:16.840+0000 7f52091ca740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x563bfbe4fae0 con 0x563bfbefb310 2024-09-11T00:24:16.843 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:16.840+0000 7f52091ca740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x563bfbe538f0 con 0x563bfbef5fa0 2024-09-11T00:24:16.845 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:16.842+0000 7f51f7e00640 1 --2- 172.21.15.12:0/2698383519 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f51e40b5360 0x7f51e40b7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:24:16.846 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:16.842+0000 7f51fea00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563bfbef5fa0 0x563bfbef8390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:24:16.846 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:16.842+0000 7f51fea00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563bfbef5fa0 0x563bfbef8390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.12:47704/0 (socket says 172.21.15.12:47704) 2024-09-11T00:24:16.846 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:16.842+0000 7f51fea00640 1 -- 172.21.15.12:0/573558526 learned_addr learned my addr 172.21.15.12:0/573558526 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:24:16.846 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:16.842+0000 7f51fea00640 1 --2- 172.21.15.12:0/573558526 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563bfbefb310 0x563bfbefd7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:24:16.846 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:16.843+0000 7f51fea00640 1 -- 172.21.15.12:0/573558526 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563bfbefb310 msgr2=0x563bfbefd7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:24:16.846 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:16.843+0000 7f51fea00640 1 --2- 172.21.15.12:0/573558526 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563bfbefb310 0x563bfbefd7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:24:16.846 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:16.843+0000 7f51fea00640 1 -- 172.21.15.12:0/573558526 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563bfbef88d0 msgr2=0x563bfbefacc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:24:16.846 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:16.843+0000 7f51fea00640 1 --2- 172.21.15.12:0/573558526 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563bfbef88d0 0x563bfbefacc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:24:16.846 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:16.843+0000 7f51fea00640 1 -- 172.21.15.12:0/573558526 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563bfbe1c780 con 0x563bfbef5fa0 2024-09-11T00:24:16.846 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:16.843+0000 7f51fea00640 1 --2- 172.21.15.12:0/573558526 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563bfbef5fa0 0x563bfbef8390 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f51ec012820 tx=0x7f51ec00b7e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6b8045d636d2685f server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:24:16.847 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:16.844+0000 7f51ebe00640 1 -- 172.21.15.12:0/573558526 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51ec012da0 con 0x563bfbef5fa0 2024-09-11T00:24:16.847 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:16.844+0000 7f51f7e00640 1 -- 172.21.15.12:0/2698383519 <== mon.2 v2:172.21.15.165:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9208+0+0 (secure 0 0 0) 0x7f51f81073c0 con 0x7f52001066b0 2024-09-11T00:24:16.847 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:16.844+0000 7f51ebe00640 1 -- 172.21.15.12:0/573558526 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f51ec010c20 con 0x563bfbef5fa0 2024-09-11T00:24:16.847 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:16.844+0000 7f51ebe00640 1 -- 172.21.15.12:0/573558526 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51ec01eb80 con 0x563bfbef5fa0 2024-09-11T00:24:16.847 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:16.844+0000 7f51ffe00640 1 --2- 172.21.15.12:0/2698383519 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f51e40b5360 0x7f51e40b7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:24:16.848 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:16.845+0000 7f52091ca740 1 -- 172.21.15.12:0/573558526 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563bfbef5fa0 msgr2=0x563bfbef8390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:24:16.848 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:16.845+0000 7f52091ca740 1 --2- 172.21.15.12:0/573558526 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563bfbef5fa0 0x563bfbef8390 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f51ec012820 tx=0x7f51ec00b7e0 comp rx=0 tx=0).stop 2024-09-11T00:24:16.848 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:16.845+0000 7f51ffe00640 1 --2- 172.21.15.12:0/2698383519 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f51e40b5360 0x7f51e40b7820 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f51f8005e30 tx=0x7f51f80093d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:24:16.848 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:16.845+0000 7f52091ca740 1 -- 172.21.15.12:0/573558526 shutdown_connections 2024-09-11T00:24:16.848 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:16.845+0000 7f52091ca740 1 --2- 172.21.15.12:0/573558526 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563bfbefb310 0x563bfbefd7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:24:16.848 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:16.845+0000 7f52091ca740 1 --2- 172.21.15.12:0/573558526 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563bfbef88d0 0x563bfbefacc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:24:16.848 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:16.845+0000 7f52091ca740 1 --2- 172.21.15.12:0/573558526 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563bfbef5fa0 0x563bfbef8390 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:24:16.848 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:16.845+0000 7f52091ca740 1 -- 172.21.15.12:0/573558526 >> 172.21.15.12:0/573558526 conn(0x563bfbee88e0 msgr2=0x563bfbee9670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:24:16.848 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:16.845+0000 7f52091ca740 1 -- 172.21.15.12:0/573558526 shutdown_connections 2024-09-11T00:24:16.848 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:16.845+0000 7f52091ca740 1 -- 172.21.15.12:0/573558526 wait complete. 2024-09-11T00:24:16.848 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:16.846+0000 7f52091ca740 1 -- start start 2024-09-11T00:24:16.849 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:16.846+0000 7f52091ca740 1 Processor -- start 2024-09-11T00:24:16.849 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:16.847+0000 7f52091ca740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563bfbef5fa0 0x563bfbeff730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:24:16.849 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:16.847+0000 7f52091ca740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563bfbef88d0 0x563bfbeffdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:24:16.850 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:16.847+0000 7f52091ca740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563bfbefb310 0x563bfbf009a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:24:16.850 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:16.847+0000 7f52091ca740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x563bfbd87ab0 con 0x563bfbef5fa0 2024-09-11T00:24:16.850 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:16.847+0000 7f52091ca740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x563bfbdf6570 con 0x563bfbefb310 2024-09-11T00:24:16.851 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:16.847+0000 7f52091ca740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x563bfbefe2f0 con 0x563bfbef88d0 2024-09-11T00:24:16.851 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:16.847+0000 7f51fea00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563bfbef5fa0 0x563bfbeff730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:24:16.851 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:16.847+0000 7f52091ca740 10 client.0 _ll_register_callbacks cb 0x563bfbe05100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:24:16.851 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:16.847+0000 7f51ff400640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563bfbef88d0 0x563bfbeffdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:24:16.851 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:16.847+0000 7f51ff400640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563bfbef88d0 0x563bfbeffdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.12:47710/0 (socket says 172.21.15.12:47710) 2024-09-11T00:24:16.851 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:16.847+0000 7f51ff400640 1 -- 172.21.15.12:0/2748532370 learned_addr learned my addr 172.21.15.12:0/2748532370 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:24:16.851 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:16.848+0000 7f51fea00640 1 --2- 172.21.15.12:0/2748532370 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563bfbefb310 0x563bfbf009a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:24:16.852 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:16.849+0000 7f51ff400640 1 -- 172.21.15.12:0/2748532370 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563bfbefb310 msgr2=0x563bfbf009a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:24:16.852 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:16.849+0000 7f51ff400640 1 --2- 172.21.15.12:0/2748532370 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563bfbefb310 0x563bfbf009a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:24:16.852 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:16.849+0000 7f51ff400640 1 -- 172.21.15.12:0/2748532370 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563bfbef5fa0 msgr2=0x563bfbeff730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:24:16.852 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:16.850+0000 7f51ff400640 1 --2- 172.21.15.12:0/2748532370 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563bfbef5fa0 0x563bfbeff730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:24:16.852 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:16.850+0000 7f51ff400640 1 -- 172.21.15.12:0/2748532370 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563bfbbb1790 con 0x563bfbef88d0 2024-09-11T00:24:16.852 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:16.850+0000 7f51fea00640 1 --2- 172.21.15.12:0/2748532370 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563bfbefb310 0x563bfbf009a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T00:24:16.853 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:16.850+0000 7f51ff400640 1 --2- 172.21.15.12:0/2748532370 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563bfbef88d0 0x563bfbeffdd0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f51f000c200 tx=0x7f51f000c6c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fd11df834c8ebc3c server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:24:16.853 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:16.851+0000 7f51e8c00640 10 client.0 ms_handle_connect on v2:172.21.15.178:3300/0 2024-09-11T00:24:16.853 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:16.851+0000 7f51e8c00640 1 -- 172.21.15.12:0/2748532370 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51f0015070 con 0x563bfbef88d0 2024-09-11T00:24:16.853 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:16.851+0000 7f52091ca740 1 -- 172.21.15.12:0/2748532370 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f51ec010c20 con 0x563bfbef88d0 2024-09-11T00:24:16.854 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:16.851+0000 7f51e8c00640 1 -- 172.21.15.12:0/2748532370 <== mon.1 v2:172.21.15.178:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9208+0+0 (secure 0 0 0) 0x7f51f0011600 con 0x563bfbef88d0 2024-09-11T00:24:16.854 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:16.852+0000 7f51e8c00640 1 -- 172.21.15.12:0/2748532370 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=67) ==== 24+0+0 (secure 0 0 0) 0x7f51f0021050 con 0x563bfbef88d0 2024-09-11T00:24:16.854 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:16.852+0000 7f52091ca740 10 client.27373 fetch_fsmap learned FSMap version 67 2024-09-11T00:24:16.854 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:16.852+0000 7f52091ca740 1 -- 172.21.15.12:0/2748532370 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,fsmap=67,monmap=4+}) -- 0x563bfbc2a620 con 0x563bfbef88d0 2024-09-11T00:24:16.856 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:16.852+0000 7f51e8c00640 1 -- 172.21.15.12:0/2748532370 <== mon.1 v2:172.21.15.178:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f51ec010c20 con 0x563bfbef88d0 2024-09-11T00:24:16.856 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:16.854+0000 7f51e8c00640 1 -- 172.21.15.12:0/2748532370 <== mon.1 v2:172.21.15.178:3300/0 5 ==== fsmap(e 67) ==== 8390+0+0 (secure 0 0 0) 0x7f51f0019820 con 0x563bfbef88d0 2024-09-11T00:24:16.856 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:16.854+0000 7f52091ca740 10 client.27373 fetch_fsmap finished waiting for FSMap version 67 2024-09-11T00:24:16.856 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:16.854+0000 7f52091ca740 10 client.27373 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T00:24:16.856 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:16.854+0000 7f52091ca740 20 client.27373 populate_metadata read hostname 'smithi012' 2024-09-11T00:24:16.857 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:16.854+0000 7f52091ca740 1 --2- 172.21.15.12:0/2748532370 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x563bfbf0b110 0x563bfbf0d550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:24:16.857 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:16.854+0000 7f52091ca740 4 client.27373 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T00:24:16.857 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:16.854+0000 7f52091ca740 1 -- 172.21.15.12:0/2748532370 --> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563bfbe62600 con 0x563bfbf0b110 2024-09-11T00:24:16.858 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:16.856+0000 7f51fea00640 1 --2- 172.21.15.12:0/2748532370 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x563bfbf0b110 0x563bfbf0d550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:24:16.859 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:16.856+0000 7f51fea00640 1 --2- 172.21.15.12:0/2748532370 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x563bfbf0b110 0x563bfbf0d550 crc :-1 s=READY pgs=245 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6a5ef41b565e1628 server_cookie=701e1f0852980d35 in_seq=0 out_seq=0 2024-09-11T00:24:16.859 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:16.856+0000 7f51e8c00640 10 client.27373 ms_handle_connect on v2:172.21.15.165:6838/501312984 2024-09-11T00:24:16.860 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:16.857+0000 7f51e8c00640 1 -- 172.21.15.12:0/2748532370 <== mds.0 v2:172.21.15.165:6838/501312984 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x563bfbe62600 con 0x563bfbf0b110 2024-09-11T00:24:16.860 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:16.857+0000 7f51e8c00640 10 client.27373 handle_command_reply: tid=0 2024-09-11T00:24:16.860 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:16.857+0000 7f52091ca740 1 client.27373 shutdown 2024-09-11T00:24:16.861 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:16.859+0000 7f52091ca740 1 -- 172.21.15.12:0/2748532370 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563bfbef88d0 msgr2=0x563bfbeffdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:24:16.861 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:16.859+0000 7f52091ca740 1 --2- 172.21.15.12:0/2748532370 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563bfbef88d0 0x563bfbeffdd0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f51f000c200 tx=0x7f51f000c6c0 comp rx=0 tx=0).stop 2024-09-11T00:24:16.861 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:16.859+0000 7f52091ca740 1 -- 172.21.15.12:0/2748532370 shutdown_connections 2024-09-11T00:24:16.861 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:16.859+0000 7f52091ca740 1 --2- 172.21.15.12:0/2748532370 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x563bfbf0b110 0x563bfbf0d550 crc :-1 s=READY pgs=245 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:24:16.861 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:16.859+0000 7f52091ca740 1 --2- 172.21.15.12:0/2748532370 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563bfbefb310 0x563bfbf009a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:24:16.862 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:16.859+0000 7f52091ca740 1 --2- 172.21.15.12:0/2748532370 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563bfbef88d0 0x563bfbeffdd0 secure :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f51f000c200 tx=0x7f51f000c6c0 comp rx=0 tx=0).stop 2024-09-11T00:24:16.862 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:16.859+0000 7f52091ca740 1 --2- 172.21.15.12:0/2748532370 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563bfbef5fa0 0x563bfbeff730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:24:16.862 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:16.859+0000 7f52091ca740 1 -- 172.21.15.12:0/2748532370 >> 172.21.15.12:0/2748532370 conn(0x563bfbee88e0 msgr2=0x563bfbee7a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:24:16.862 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:16.859+0000 7f51e8c00640 0 client.27373 ms_handle_reset on v2:172.21.15.165:6838/501312984 2024-09-11T00:24:16.862 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:16.859+0000 7f52091ca740 1 -- 172.21.15.12:0/2748532370 shutdown_connections 2024-09-11T00:24:16.863 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:16.860+0000 7f52091ca740 1 -- 172.21.15.12:0/2748532370 wait complete. 2024-09-11T00:24:16.863 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:16.860+0000 7f52091ca740 20 client.27373 trim_cache size 0 max 16384 2024-09-11T00:24:16.899 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:16.897+0000 7f52091ca740 1 Processor -- start 2024-09-11T00:24:16.901 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:16.897+0000 7f52091ca740 1 -- start start 2024-09-11T00:24:16.901 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:16.897+0000 7f52091ca740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563bfbef5fa0 0x563bfbee9300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:24:16.901 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:16.897+0000 7f52091ca740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563bfbefb310 0x563bfbf07a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:24:16.901 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:16.897+0000 7f52091ca740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563bfbeed390 0x563bfbee9840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:24:16.901 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:16.897+0000 7f52091ca740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f5200072e90 con 0x563bfbefb310 2024-09-11T00:24:16.901 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:16.897+0000 7f52091ca740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f5200074a60 con 0x563bfbef5fa0 2024-09-11T00:24:16.901 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:16.897+0000 7f52091ca740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f5200073590 con 0x563bfbeed390 2024-09-11T00:24:16.901 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:16.898+0000 7f51ff400640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563bfbefb310 0x563bfbf07a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:24:16.901 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:16.898+0000 7f51fea00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563bfbef5fa0 0x563bfbee9300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:24:16.902 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:16.898+0000 7f51fea00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563bfbef5fa0 0x563bfbee9300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.12:58058/0 (socket says 172.21.15.12:58058) 2024-09-11T00:24:16.902 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:16.898+0000 7f51fea00640 1 -- 172.21.15.12:0/2093596819 learned_addr learned my addr 172.21.15.12:0/2093596819 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:24:16.902 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:16.898+0000 7f51fea00640 1 --2- 172.21.15.12:0/2093596819 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563bfbeed390 0x563bfbee9840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:24:16.903 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:16.899+0000 7f51fea00640 1 -- 172.21.15.12:0/2093596819 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563bfbef5fa0 msgr2=0x563bfbee9300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:24:16.903 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:16.899+0000 7f51fea00640 1 --2- 172.21.15.12:0/2093596819 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563bfbef5fa0 0x563bfbee9300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:24:16.904 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:16.899+0000 7f51fea00640 1 -- 172.21.15.12:0/2093596819 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563bfbefb310 msgr2=0x563bfbf07a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:24:16.904 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:16.900+0000 7f51fea00640 1 --2- 172.21.15.12:0/2093596819 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563bfbefb310 0x563bfbf07a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:24:16.904 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:16.900+0000 7f51fea00640 1 -- 172.21.15.12:0/2093596819 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563bfbef8700 con 0x563bfbeed390 2024-09-11T00:24:16.904 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:16.901+0000 7f51fea00640 1 --2- 172.21.15.12:0/2093596819 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563bfbeed390 0x563bfbee9840 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f51ec0127c0 tx=0x7f51ec00b570 comp rx=0 tx=0).ready entity=mon.1 client_cookie=eeedb1bfb2ae2712 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:24:16.904 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:16.902+0000 7f51eb400640 1 -- 172.21.15.12:0/2093596819 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51ec010740 con 0x563bfbeed390 2024-09-11T00:24:16.904 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:16.902+0000 7f51eb400640 1 -- 172.21.15.12:0/2093596819 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f51ec016040 con 0x563bfbeed390 2024-09-11T00:24:16.904 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:16.902+0000 7f51eb400640 1 -- 172.21.15.12:0/2093596819 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51ec01ea90 con 0x563bfbeed390 2024-09-11T00:24:16.906 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:16.904+0000 7f52091ca740 1 -- 172.21.15.12:0/2093596819 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563bfbeed390 msgr2=0x563bfbee9840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:24:16.906 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:16.904+0000 7f52091ca740 1 --2- 172.21.15.12:0/2093596819 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563bfbeed390 0x563bfbee9840 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f51ec0127c0 tx=0x7f51ec00b570 comp rx=0 tx=0).stop 2024-09-11T00:24:16.906 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:16.904+0000 7f52091ca740 1 -- 172.21.15.12:0/2093596819 shutdown_connections 2024-09-11T00:24:16.906 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:16.904+0000 7f52091ca740 1 --2- 172.21.15.12:0/2093596819 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563bfbeed390 0x563bfbee9840 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:24:16.906 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:16.904+0000 7f52091ca740 1 --2- 172.21.15.12:0/2093596819 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563bfbefb310 0x563bfbf07a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:24:16.907 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:16.904+0000 7f52091ca740 1 --2- 172.21.15.12:0/2093596819 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563bfbef5fa0 0x563bfbee9300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:24:16.907 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:16.904+0000 7f52091ca740 1 -- 172.21.15.12:0/2093596819 >> 172.21.15.12:0/2093596819 conn(0x563bfbee88e0 msgr2=0x563bfbeecc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:24:16.907 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:16.904+0000 7f52091ca740 1 -- 172.21.15.12:0/2093596819 shutdown_connections 2024-09-11T00:24:16.907 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:16.905+0000 7f52091ca740 1 -- 172.21.15.12:0/2093596819 wait complete. 2024-09-11T00:24:16.907 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:16.905+0000 7f52091ca740 1 -- start start 2024-09-11T00:24:16.908 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:16.905+0000 7f52091ca740 1 Processor -- start 2024-09-11T00:24:16.909 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:16.907+0000 7f52091ca740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563bfbeed390 0x563bfbf06880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:24:16.909 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:16.907+0000 7f52091ca740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563bfbef5fa0 0x563bfbf06dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:24:16.909 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:16.907+0000 7f52091ca740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563bfbefb310 0x563bfbf07300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:24:16.910 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:16.907+0000 7f51fea00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563bfbeed390 0x563bfbf06880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:24:16.910 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:16.907+0000 7f51fea00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563bfbeed390 0x563bfbf06880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.12:58064/0 (socket says 172.21.15.12:58064) 2024-09-11T00:24:16.910 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:16.907+0000 7f51fea00640 1 -- 172.21.15.12:0/3126265945 learned_addr learned my addr 172.21.15.12:0/3126265945 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:24:16.910 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:16.907+0000 7f52091ca740 1 -- 172.21.15.12:0/3126265945 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f5200074390 con 0x563bfbef5fa0 2024-09-11T00:24:16.910 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:16.907+0000 7f52091ca740 1 -- 172.21.15.12:0/3126265945 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f5200073cc0 con 0x563bfbeed390 2024-09-11T00:24:16.910 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:16.907+0000 7f52091ca740 1 -- 172.21.15.12:0/3126265945 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f5200070e70 con 0x563bfbefb310 2024-09-11T00:24:16.910 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:16.907+0000 7f52091ca740 10 client.0 _ll_register_callbacks cb 0x7f51f000a0d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:24:16.911 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:16.908+0000 7f51ff400640 1 --2- 172.21.15.12:0/3126265945 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563bfbef5fa0 0x563bfbf06dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:24:16.911 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:16.909+0000 7f51fea00640 1 --2- 172.21.15.12:0/3126265945 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563bfbefb310 0x563bfbf07300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:24:16.911 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:16.909+0000 7f51fea00640 1 -- 172.21.15.12:0/3126265945 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563bfbefb310 msgr2=0x563bfbf07300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:24:16.911 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:16.909+0000 7f51fea00640 1 --2- 172.21.15.12:0/3126265945 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563bfbefb310 0x563bfbf07300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:24:16.911 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:16.909+0000 7f51fea00640 1 -- 172.21.15.12:0/3126265945 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563bfbef5fa0 msgr2=0x563bfbf06dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:24:16.912 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:16.909+0000 7f51fea00640 1 --2- 172.21.15.12:0/3126265945 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563bfbef5fa0 0x563bfbf06dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:24:16.912 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:16.909+0000 7f51fea00640 1 -- 172.21.15.12:0/3126265945 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563bfbefb030 con 0x563bfbeed390 2024-09-11T00:24:16.912 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:16.910+0000 7f51fea00640 1 --2- 172.21.15.12:0/3126265945 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563bfbeed390 0x563bfbf06880 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f51ec010f00 tx=0x7f51ec010e50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7e036e4e2e71d7b2 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:24:16.912 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:16.910+0000 7f51e8c00640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-09-11T00:24:16.913 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:16.911+0000 7f51e8c00640 1 -- 172.21.15.12:0/3126265945 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51ec03a040 con 0x563bfbeed390 2024-09-11T00:24:16.914 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:16.911+0000 7f51e8c00640 1 -- 172.21.15.12:0/3126265945 <== mon.2 v2:172.21.15.165:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9208+0+0 (secure 0 0 0) 0x7f51ec010740 con 0x563bfbeed390 2024-09-11T00:24:16.914 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:16.911+0000 7f52091ca740 1 -- 172.21.15.12:0/3126265945 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f51ec016040 con 0x563bfbeed390 2024-09-11T00:24:16.915 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:16.913+0000 7f51e8c00640 1 -- 172.21.15.12:0/3126265945 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=67) ==== 24+0+0 (secure 0 0 0) 0x7f51ec045050 con 0x563bfbeed390 2024-09-11T00:24:16.915 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:16.913+0000 7f52091ca740 10 client.27407 fetch_fsmap learned FSMap version 67 2024-09-11T00:24:16.916 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:16.913+0000 7f52091ca740 1 -- 172.21.15.12:0/3126265945 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=67,monmap=4+}) -- 0x563bfbf78350 con 0x563bfbeed390 2024-09-11T00:24:16.916 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:16.914+0000 7f51e8c00640 1 -- 172.21.15.12:0/3126265945 <== mon.2 v2:172.21.15.165:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f51ec016040 con 0x563bfbeed390 2024-09-11T00:24:16.917 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:16.915+0000 7f51e8c00640 1 -- 172.21.15.12:0/3126265945 <== mon.2 v2:172.21.15.165:3300/0 5 ==== fsmap(e 67) ==== 8390+0+0 (secure 0 0 0) 0x7f51ec031030 con 0x563bfbeed390 2024-09-11T00:24:16.917 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:16.915+0000 7f52091ca740 10 client.27407 fetch_fsmap finished waiting for FSMap version 67 2024-09-11T00:24:16.917 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:16.915+0000 7f52091ca740 10 client.27407 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T00:24:16.917 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:16.915+0000 7f52091ca740 20 client.27407 populate_metadata read hostname 'smithi012' 2024-09-11T00:24:16.917 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:16.915+0000 7f52091ca740 1 --2- 172.21.15.12:0/3126265945 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x563bfbf78ad0 0x563bfbf7aef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:24:16.918 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:16.915+0000 7f52091ca740 4 client.27407 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T00:24:16.918 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:16.915+0000 7f52091ca740 1 -- 172.21.15.12:0/3126265945 --> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] -- command(tid 0: {"prefix": "flush journal"}) -- 0x563bfbe5e0b0 con 0x563bfbf78ad0 2024-09-11T00:24:16.918 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:16.916+0000 7f51ff400640 1 --2- 172.21.15.12:0/3126265945 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x563bfbf78ad0 0x563bfbf7aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:24:16.920 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:16.917+0000 7f51ff400640 1 --2- 172.21.15.12:0/3126265945 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x563bfbf78ad0 0x563bfbf7aef0 crc :-1 s=READY pgs=246 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5ff172d7c9912373 server_cookie=6bf82e0fb550f5f9 in_seq=0 out_seq=0 2024-09-11T00:24:16.920 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:16.918+0000 7f51e8c00640 10 client.27407 ms_handle_connect on v2:172.21.15.165:6838/501312984 2024-09-11T00:24:17.177 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:17.175+0000 7f51e8c00640 1 -- 172.21.15.12:0/3126265945 <== mds.0 v2:172.21.15.165:6838/501312984 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x563bfbe5e0b0 con 0x563bfbf78ad0 2024-09-11T00:24:17.178 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:17.175+0000 7f51e8c00640 10 client.27407 handle_command_reply: tid=0 2024-09-11T00:24:17.178 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:17.175+0000 7f52091ca740 1 client.27407 shutdown 2024-09-11T00:24:17.178 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:17.176+0000 7f52091ca740 1 -- 172.21.15.12:0/3126265945 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563bfbeed390 msgr2=0x563bfbf06880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:24:17.178 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:17.176+0000 7f52091ca740 1 --2- 172.21.15.12:0/3126265945 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563bfbeed390 0x563bfbf06880 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f51ec010f00 tx=0x7f51ec010e50 comp rx=0 tx=0).stop 2024-09-11T00:24:17.178 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:17.176+0000 7f52091ca740 1 -- 172.21.15.12:0/3126265945 shutdown_connections 2024-09-11T00:24:17.179 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:17.176+0000 7f52091ca740 1 --2- 172.21.15.12:0/3126265945 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x563bfbf78ad0 0x563bfbf7aef0 crc :-1 s=READY pgs=246 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:24:17.179 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:17.176+0000 7f52091ca740 1 --2- 172.21.15.12:0/3126265945 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563bfbefb310 0x563bfbf07300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:24:17.179 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:17.176+0000 7f52091ca740 1 --2- 172.21.15.12:0/3126265945 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563bfbef5fa0 0x563bfbf06dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:24:17.179 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:17.176+0000 7f52091ca740 1 --2- 172.21.15.12:0/3126265945 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563bfbeed390 0x563bfbf06880 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:24:17.179 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:17.176+0000 7f52091ca740 1 -- 172.21.15.12:0/3126265945 >> 172.21.15.12:0/3126265945 conn(0x563bfbee88e0 msgr2=0x563bfbeec850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:24:17.179 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:17.176+0000 7f51e8c00640 0 client.27407 ms_handle_reset on v2:172.21.15.165:6838/501312984 2024-09-11T00:24:17.179 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:17.177+0000 7f52091ca740 1 -- 172.21.15.12:0/3126265945 shutdown_connections 2024-09-11T00:24:17.180 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:17.177+0000 7f52091ca740 1 -- 172.21.15.12:0/3126265945 wait complete. 2024-09-11T00:24:17.180 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:17.177+0000 7f52091ca740 20 client.27407 trim_cache size 0 max 16384 2024-09-11T00:24:17.180 INFO:tasks.workunit.client.0.smithi012.stdout: 10 75341 17.11 MB/sec execute 133 sec latency 224.079 ms 2024-09-11T00:24:17.181 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:{ 2024-09-11T00:24:17.181 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "message": "", 2024-09-11T00:24:17.181 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "return_code": 0 2024-09-11T00:24:17.181 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:} 2024-09-11T00:24:17.182 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:17.180+0000 7f5206200640 1 -- 172.21.15.12:0/2698383519 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f51e40b5360 msgr2=0x7f51e40b7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:24:17.182 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:17.180+0000 7f5206200640 1 --2- 172.21.15.12:0/2698383519 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f51e40b5360 0x7f51e40b7820 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f51f8005e30 tx=0x7f51f80093d0 comp rx=0 tx=0).stop 2024-09-11T00:24:17.182 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:17.180+0000 7f5206200640 1 -- 172.21.15.12:0/2698383519 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f52001066b0 msgr2=0x7f520019c1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:24:17.182 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:17.180+0000 7f5206200640 1 --2- 172.21.15.12:0/2698383519 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f52001066b0 0x7f520019c1d0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f51f8003ba0 tx=0x7f51f8003bd0 comp rx=0 tx=0).stop 2024-09-11T00:24:17.183 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:17.180+0000 7f5206200640 1 -- 172.21.15.12:0/2698383519 shutdown_connections 2024-09-11T00:24:17.183 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:17.180+0000 7f5206200640 1 --2- 172.21.15.12:0/2698383519 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f51e40b5360 0x7f51e40b7820 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:24:17.183 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:17.180+0000 7f5206200640 1 --2- 172.21.15.12:0/2698383519 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f52001066b0 0x7f520019c1d0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:24:17.183 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:17.180+0000 7f5206200640 1 --2- 172.21.15.12:0/2698383519 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5200105cf0 0x7f520019b5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:24:17.183 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:17.180+0000 7f5206200640 1 --2- 172.21.15.12:0/2698383519 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5200104ba0 0x7f520019af50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:24:17.183 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:17.180+0000 7f5206200640 1 -- 172.21.15.12:0/2698383519 >> 172.21.15.12:0/2698383519 conn(0x7f5200100460 msgr2=0x7f5200109010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:24:17.183 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:17.181+0000 7f5206200640 1 -- 172.21.15.12:0/2698383519 shutdown_connections 2024-09-11T00:24:17.184 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:17.181+0000 7f5206200640 1 -- 172.21.15.12:0/2698383519 wait complete. 2024-09-11T00:24:18.033 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:24:17 smithi165 ceph-mon[35356]: pgmap v915: 129 pgs: 129 active+clean; 504 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 2.3 MiB/s wr, 345 op/s 2024-09-11T00:24:18.033 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:24:17 smithi165 ceph-mon[35356]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T00:24:18.063 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:24:17 smithi012 ceph-mon[30333]: pgmap v915: 129 pgs: 129 active+clean; 504 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 2.3 MiB/s wr, 345 op/s 2024-09-11T00:24:18.063 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:24:17 smithi012 ceph-mon[30333]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T00:24:18.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:24:17 smithi178 ceph-mon[35919]: pgmap v915: 129 pgs: 129 active+clean; 504 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 2.3 MiB/s wr, 345 op/s 2024-09-11T00:24:18.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:24:17 smithi178 ceph-mon[35919]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T00:24:18.179 INFO:tasks.workunit.client.0.smithi012.stdout: 10 76201 17.38 MB/sec execute 134 sec latency 243.595 ms 2024-09-11T00:24:19.180 INFO:tasks.workunit.client.0.smithi012.stdout: 10 77028 17.51 MB/sec execute 135 sec latency 74.546 ms 2024-09-11T00:24:20.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:24:19 smithi165 ceph-mon[35356]: pgmap v916: 129 pgs: 129 active+clean; 504 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 2.3 MiB/s wr, 342 op/s 2024-09-11T00:24:20.064 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:24:19 smithi012 ceph-mon[30333]: pgmap v916: 129 pgs: 129 active+clean; 504 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 2.3 MiB/s wr, 342 op/s 2024-09-11T00:24:20.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:24:19 smithi178 ceph-mon[35919]: pgmap v916: 129 pgs: 129 active+clean; 504 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 2.3 MiB/s wr, 342 op/s 2024-09-11T00:24:20.179 INFO:tasks.workunit.client.0.smithi012.stdout: 10 77753 17.68 MB/sec execute 136 sec latency 143.132 ms 2024-09-11T00:24:21.180 INFO:tasks.workunit.client.0.smithi012.stdout: 10 78622 17.94 MB/sec execute 137 sec latency 177.570 ms 2024-09-11T00:24:21.813 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:24:21 smithi012 ceph-mon[30333]: pgmap v917: 129 pgs: 129 active+clean; 518 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 3.8 MiB/s wr, 582 op/s 2024-09-11T00:24:22.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:24:21 smithi165 ceph-mon[35356]: pgmap v917: 129 pgs: 129 active+clean; 518 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 3.8 MiB/s wr, 582 op/s 2024-09-11T00:24:22.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:24:21 smithi178 ceph-mon[35919]: pgmap v917: 129 pgs: 129 active+clean; 518 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 3.8 MiB/s wr, 582 op/s 2024-09-11T00:24:22.180 INFO:tasks.workunit.client.0.smithi012.stdout: 10 79388 18.11 MB/sec execute 138 sec latency 227.364 ms 2024-09-11T00:24:23.180 INFO:tasks.workunit.client.0.smithi012.stdout: 10 80233 18.28 MB/sec execute 139 sec latency 83.223 ms 2024-09-11T00:24:24.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:24:23 smithi165 ceph-mon[35356]: pgmap v918: 129 pgs: 129 active+clean; 523 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 4.3 MiB/s wr, 662 op/s 2024-09-11T00:24:24.063 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:24:23 smithi012 ceph-mon[30333]: pgmap v918: 129 pgs: 129 active+clean; 523 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 4.3 MiB/s wr, 662 op/s 2024-09-11T00:24:24.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:24:23 smithi178 ceph-mon[35919]: pgmap v918: 129 pgs: 129 active+clean; 523 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 4.3 MiB/s wr, 662 op/s 2024-09-11T00:24:24.180 INFO:tasks.workunit.client.0.smithi012.stdout: 10 80996 18.45 MB/sec execute 140 sec latency 46.119 ms 2024-09-11T00:24:25.180 INFO:tasks.workunit.client.0.smithi012.stdout: 10 81885 18.64 MB/sec execute 141 sec latency 68.631 ms 2024-09-11T00:24:26.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:24:25 smithi165 ceph-mon[35356]: pgmap v919: 129 pgs: 129 active+clean; 523 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 4.1 MiB/s wr, 626 op/s 2024-09-11T00:24:26.063 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:24:25 smithi012 ceph-mon[30333]: pgmap v919: 129 pgs: 129 active+clean; 523 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 4.1 MiB/s wr, 626 op/s 2024-09-11T00:24:26.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:24:25 smithi178 ceph-mon[35919]: pgmap v919: 129 pgs: 129 active+clean; 523 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 4.1 MiB/s wr, 626 op/s 2024-09-11T00:24:26.180 INFO:tasks.workunit.client.0.smithi012.stdout: 10 82825 18.89 MB/sec execute 142 sec latency 219.819 ms 2024-09-11T00:24:27.180 INFO:tasks.workunit.client.0.smithi012.stdout: 10 83637 19.08 MB/sec execute 143 sec latency 113.014 ms 2024-09-11T00:24:27.994 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:24:27 smithi012 ceph-mon[30333]: pgmap v920: 129 pgs: 129 active+clean; 540 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 19 MiB/s rd, 5.8 MiB/s wr, 857 op/s 2024-09-11T00:24:28.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:24:27 smithi165 ceph-mon[35356]: pgmap v920: 129 pgs: 129 active+clean; 540 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 19 MiB/s rd, 5.8 MiB/s wr, 857 op/s 2024-09-11T00:24:28.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:24:27 smithi178 ceph-mon[35919]: pgmap v920: 129 pgs: 129 active+clean; 540 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 19 MiB/s rd, 5.8 MiB/s wr, 857 op/s 2024-09-11T00:24:28.180 INFO:tasks.workunit.client.0.smithi012.stdout: 10 84331 19.22 MB/sec execute 144 sec latency 153.008 ms 2024-09-11T00:24:29.180 INFO:tasks.workunit.client.0.smithi012.stdout: 10 84781 19.20 MB/sec execute 145 sec latency 849.084 ms 2024-09-11T00:24:30.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:24:29 smithi165 ceph-mon[35356]: pgmap v921: 129 pgs: 129 active+clean; 543 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 4.2 MiB/s wr, 611 op/s 2024-09-11T00:24:30.063 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:24:29 smithi012 ceph-mon[30333]: pgmap v921: 129 pgs: 129 active+clean; 543 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 4.2 MiB/s wr, 611 op/s 2024-09-11T00:24:30.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:24:29 smithi178 ceph-mon[35919]: pgmap v921: 129 pgs: 129 active+clean; 543 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 4.2 MiB/s wr, 611 op/s 2024-09-11T00:24:30.180 INFO:tasks.workunit.client.0.smithi012.stdout: 10 85586 19.35 MB/sec execute 146 sec latency 113.058 ms 2024-09-11T00:24:30.366 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:30.364+0000 7fbca3000640 1 -- 172.21.15.12:0/1731032765 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbc9c10a6e0 msgr2=0x7fbc9c10cb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:24:30.366 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:30.364+0000 7fbca3000640 1 --2- 172.21.15.12:0/1731032765 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbc9c10a6e0 0x7fbc9c10cb80 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7fbc94009ca0 tx=0x7fbc9402a8f0 comp rx=0 tx=0).stop 2024-09-11T00:24:30.367 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:30.365+0000 7fbca3000640 1 -- 172.21.15.12:0/1731032765 shutdown_connections 2024-09-11T00:24:30.367 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:30.365+0000 7fbca3000640 1 --2- 172.21.15.12:0/1731032765 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbc9c10a6e0 0x7fbc9c10cb80 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:24:30.367 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:30.365+0000 7fbca3000640 1 --2- 172.21.15.12:0/1731032765 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbc9c107ca0 0x7fbc9c10a090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:24:30.367 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:30.365+0000 7fbca3000640 1 --2- 172.21.15.12:0/1731032765 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbc9c06b360 0x7fbc9c107760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:24:30.367 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:30.365+0000 7fbca3000640 1 -- 172.21.15.12:0/1731032765 >> 172.21.15.12:0/1731032765 conn(0x7fbc9c0fcf30 msgr2=0x7fbc9c0ff370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:24:30.368 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:30.366+0000 7fbca3000640 1 -- 172.21.15.12:0/1731032765 shutdown_connections 2024-09-11T00:24:30.368 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:30.366+0000 7fbca3000640 1 -- 172.21.15.12:0/1731032765 wait complete. 2024-09-11T00:24:30.370 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:30.368+0000 7fbca3000640 1 Processor -- start 2024-09-11T00:24:30.370 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:30.368+0000 7fbca3000640 1 -- start start 2024-09-11T00:24:30.371 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:30.369+0000 7fbca3000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbc9c06b360 0x7fbc9c19f2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:24:30.371 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:30.369+0000 7fbca3000640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbc9c107ca0 0x7fbc9c19f930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:24:30.371 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:30.369+0000 7fbca3000640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbc9c10a6e0 0x7fbc9c1a0560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:24:30.371 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:30.369+0000 7fbca3000640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fbc9c110780 con 0x7fbc9c10a6e0 2024-09-11T00:24:30.371 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:30.369+0000 7fbca3000640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fbc9c110600 con 0x7fbc9c06b360 2024-09-11T00:24:30.371 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:30.369+0000 7fbca3000640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fbc9c110900 con 0x7fbc9c107ca0 2024-09-11T00:24:30.372 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:30.369+0000 7fbca1200640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbc9c107ca0 0x7fbc9c19f930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:24:30.372 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:30.369+0000 7fbca1200640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbc9c107ca0 0x7fbc9c19f930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.12:54624/0 (socket says 172.21.15.12:54624) 2024-09-11T00:24:30.372 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:30.369+0000 7fbca1200640 1 -- 172.21.15.12:0/1753046081 learned_addr learned my addr 172.21.15.12:0/1753046081 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:24:30.374 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:30.370+0000 7fbca1200640 1 -- 172.21.15.12:0/1753046081 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbc9c06b360 msgr2=0x7fbc9c19f2e0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-11T00:24:30.374 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:30.370+0000 7fbca1200640 1 --2- 172.21.15.12:0/1753046081 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbc9c06b360 0x7fbc9c19f2e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:24:30.374 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:30.370+0000 7fbca1200640 1 -- 172.21.15.12:0/1753046081 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbc9c10a6e0 msgr2=0x7fbc9c1a0560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:24:30.374 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:30.370+0000 7fbca1200640 1 --2- 172.21.15.12:0/1753046081 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbc9c10a6e0 0x7fbc9c1a0560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:24:30.375 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:30.371+0000 7fbca1200640 1 -- 172.21.15.12:0/1753046081 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc9c1a0e20 con 0x7fbc9c107ca0 2024-09-11T00:24:30.375 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:30.373+0000 7fbca1200640 1 --2- 172.21.15.12:0/1753046081 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbc9c107ca0 0x7fbc9c19f930 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fbc8800b470 tx=0x7fbc8800b930 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:24:30.376 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:30.374+0000 7fbc9a000640 1 -- 172.21.15.12:0/1753046081 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc88015020 con 0x7fbc9c107ca0 2024-09-11T00:24:30.376 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:30.374+0000 7fbc9a000640 1 -- 172.21.15.12:0/1753046081 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbc88011d40 con 0x7fbc9c107ca0 2024-09-11T00:24:30.376 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:30.374+0000 7fbc9a000640 1 -- 172.21.15.12:0/1753046081 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc8800f340 con 0x7fbc9c107ca0 2024-09-11T00:24:30.377 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:30.374+0000 7fbca3000640 1 -- 172.21.15.12:0/1753046081 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbc9c1a1110 con 0x7fbc9c107ca0 2024-09-11T00:24:30.377 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:30.374+0000 7fbca3000640 1 -- 172.21.15.12:0/1753046081 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbc9c1a77e0 con 0x7fbc9c107ca0 2024-09-11T00:24:30.383 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:30.381+0000 7fbc9a000640 1 -- 172.21.15.12:0/1753046081 <== mon.1 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7fbc88011890 con 0x7fbc9c107ca0 2024-09-11T00:24:30.386 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:30.384+0000 7fbca7430740 1 Processor -- start 2024-09-11T00:24:30.387 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:30.384+0000 7fbca7430740 1 -- start start 2024-09-11T00:24:30.388 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:30.386+0000 7fbca7430740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561936eb3970 0x561936eb5d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:24:30.388 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:30.386+0000 7fbc9a000640 1 --2- 172.21.15.12:0/1753046081 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fbc840b5360 0x7fbc840b7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:24:30.389 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:30.386+0000 7fbca7430740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561936eb62a0 0x561936eb8690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:24:30.389 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:30.386+0000 7fbca7430740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561936eb8ce0 0x561936ebb1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:24:30.389 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:30.386+0000 7fbca7430740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x561936e266b0 con 0x561936eb3970 2024-09-11T00:24:30.389 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:30.386+0000 7fbca7430740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x561936e15ae0 con 0x561936eb8ce0 2024-09-11T00:24:30.389 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:30.386+0000 7fbca7430740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x561936e198f0 con 0x561936eb62a0 2024-09-11T00:24:30.389 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:30.386+0000 7fbca2600640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561936eb3970 0x561936eb5d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:24:30.389 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:30.386+0000 7fbca2600640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561936eb3970 0x561936eb5d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:58548/0 (socket says 172.21.15.12:58548) 2024-09-11T00:24:30.389 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:30.386+0000 7fbca2600640 1 -- 172.21.15.12:0/3608112920 learned_addr learned my addr 172.21.15.12:0/3608112920 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:24:30.390 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:30.387+0000 7fbca2600640 1 --2- 172.21.15.12:0/3608112920 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561936eb8ce0 0x561936ebb1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:24:30.390 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:30.387+0000 7fbca1200640 1 --2- 172.21.15.12:0/3608112920 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561936eb62a0 0x561936eb8690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:24:30.391 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:30.388+0000 7fbc9a000640 1 -- 172.21.15.12:0/1753046081 <== mon.1 v2:172.21.15.178:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9208+0+0 (secure 0 0 0) 0x7fbc880e5c50 con 0x7fbc9c107ca0 2024-09-11T00:24:30.391 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:30.389+0000 7fbca1c00640 1 --2- 172.21.15.12:0/1753046081 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fbc840b5360 0x7fbc840b7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:24:30.392 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:30.389+0000 7fbca1200640 1 -- 172.21.15.12:0/3608112920 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561936eb8ce0 msgr2=0x561936ebb1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:24:30.392 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:30.389+0000 7fbca1200640 1 --2- 172.21.15.12:0/3608112920 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561936eb8ce0 0x561936ebb1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:24:30.392 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:30.389+0000 7fbca1200640 1 -- 172.21.15.12:0/3608112920 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561936eb3970 msgr2=0x561936eb5d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:24:30.392 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:30.389+0000 7fbca1200640 1 --2- 172.21.15.12:0/3608112920 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561936eb3970 0x561936eb5d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:24:30.392 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:30.389+0000 7fbca1200640 1 -- 172.21.15.12:0/3608112920 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561936de2780 con 0x561936eb62a0 2024-09-11T00:24:30.393 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:30.390+0000 7fbca1200640 1 --2- 172.21.15.12:0/3608112920 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561936eb62a0 0x561936eb8690 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fbc880e33d0 tx=0x7fbc880a8750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=df0278063a257a9f server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:24:30.393 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:30.390+0000 7fbc8ea00640 1 -- 172.21.15.12:0/3608112920 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc88091e60 con 0x561936eb62a0 2024-09-11T00:24:30.393 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:30.391+0000 7fbc8ea00640 1 -- 172.21.15.12:0/3608112920 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbc88010eb0 con 0x561936eb62a0 2024-09-11T00:24:30.393 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:30.391+0000 7fbc8ea00640 1 -- 172.21.15.12:0/3608112920 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc880ac040 con 0x561936eb62a0 2024-09-11T00:24:30.394 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:30.392+0000 7fbca7430740 1 -- 172.21.15.12:0/3608112920 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561936eb62a0 msgr2=0x561936eb8690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:24:30.394 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:30.392+0000 7fbca7430740 1 --2- 172.21.15.12:0/3608112920 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561936eb62a0 0x561936eb8690 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fbc880e33d0 tx=0x7fbc880a8750 comp rx=0 tx=0).stop 2024-09-11T00:24:30.394 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:30.392+0000 7fbca7430740 1 -- 172.21.15.12:0/3608112920 shutdown_connections 2024-09-11T00:24:30.395 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:30.392+0000 7fbca7430740 1 --2- 172.21.15.12:0/3608112920 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561936eb8ce0 0x561936ebb1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:24:30.395 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:30.392+0000 7fbca7430740 1 --2- 172.21.15.12:0/3608112920 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561936eb62a0 0x561936eb8690 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:24:30.395 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:30.392+0000 7fbca7430740 1 --2- 172.21.15.12:0/3608112920 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561936eb3970 0x561936eb5d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:24:30.395 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:30.392+0000 7fbca7430740 1 -- 172.21.15.12:0/3608112920 >> 172.21.15.12:0/3608112920 conn(0x561936eaeac0 msgr2=0x561936eaf850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:24:30.395 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:30.393+0000 7fbca1c00640 1 --2- 172.21.15.12:0/1753046081 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fbc840b5360 0x7fbc840b7820 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fbc90009670 tx=0x7fbc90006bd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:24:30.396 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:30.393+0000 7fbca7430740 1 -- 172.21.15.12:0/3608112920 shutdown_connections 2024-09-11T00:24:30.396 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:30.393+0000 7fbca7430740 1 -- 172.21.15.12:0/3608112920 wait complete. 2024-09-11T00:24:30.397 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:30.394+0000 7fbca7430740 1 -- start start 2024-09-11T00:24:30.397 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:30.395+0000 7fbca7430740 1 Processor -- start 2024-09-11T00:24:30.398 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:30.396+0000 7fbca7430740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561936eb3970 0x561936ebd0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:24:30.398 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:30.396+0000 7fbca7430740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561936eb62a0 0x561936ebd790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:24:30.399 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:30.396+0000 7fbca7430740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561936eb8ce0 0x561936ebe360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:24:30.399 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:30.396+0000 7fbca7430740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x561936d4dab0 con 0x561936eb3970 2024-09-11T00:24:30.399 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:30.396+0000 7fbca7430740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x561936dbc570 con 0x561936eb8ce0 2024-09-11T00:24:30.400 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:30.396+0000 7fbca7430740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x561936ebbca0 con 0x561936eb62a0 2024-09-11T00:24:30.400 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:30.396+0000 7fbca7430740 10 client.0 _ll_register_callbacks cb 0x561936dcb100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:24:30.400 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:30.396+0000 7fbca1c00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561936eb3970 0x561936ebd0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:24:30.400 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:30.396+0000 7fbca1c00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561936eb3970 0x561936ebd0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:58556/0 (socket says 172.21.15.12:58556) 2024-09-11T00:24:30.400 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:30.396+0000 7fbca1c00640 1 -- 172.21.15.12:0/259853260 learned_addr learned my addr 172.21.15.12:0/259853260 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:24:30.400 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:30.397+0000 7fbca1200640 1 --2- 172.21.15.12:0/259853260 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561936eb62a0 0x561936ebd790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:24:30.401 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:30.397+0000 7fbca1c00640 1 -- 172.21.15.12:0/259853260 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561936eb62a0 msgr2=0x561936ebd790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:24:30.401 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:30.397+0000 7fbca1c00640 1 --2- 172.21.15.12:0/259853260 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561936eb62a0 0x561936ebd790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:24:30.401 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:30.397+0000 7fbca1c00640 1 -- 172.21.15.12:0/259853260 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561936eb8ce0 msgr2=0x561936ebe360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:24:30.401 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:30.397+0000 7fbca1c00640 1 --2- 172.21.15.12:0/259853260 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561936eb8ce0 0x561936ebe360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:24:30.401 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:30.397+0000 7fbca1c00640 1 -- 172.21.15.12:0/259853260 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561936b77790 con 0x561936eb3970 2024-09-11T00:24:30.401 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:30.397+0000 7fbca1c00640 1 --2- 172.21.15.12:0/259853260 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561936eb3970 0x561936ebd0f0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7fbc90013fc0 tx=0x7fbc900046e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51ce50fe3916ff94 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:24:30.401 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:30.398+0000 7fbc83400640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-09-11T00:24:30.401 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:30.398+0000 7fbc83400640 1 -- 172.21.15.12:0/259853260 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc9001d5f0 con 0x561936eb3970 2024-09-11T00:24:30.401 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:30.398+0000 7fbc83400640 1 -- 172.21.15.12:0/259853260 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9208+0+0 (secure 0 0 0) 0x7fbc9001d790 con 0x561936eb3970 2024-09-11T00:24:30.401 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:30.398+0000 7fbca7430740 1 -- 172.21.15.12:0/259853260 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x561936e28600 con 0x561936eb3970 2024-09-11T00:24:30.402 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:30.398+0000 7fbc83400640 1 -- 172.21.15.12:0/259853260 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=67) ==== 24+0+0 (secure 0 0 0) 0x7fbc9002b050 con 0x561936eb3970 2024-09-11T00:24:30.402 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:30.399+0000 7fbca7430740 10 client.17403 fetch_fsmap learned FSMap version 67 2024-09-11T00:24:30.402 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:30.399+0000 7fbca7430740 1 -- 172.21.15.12:0/259853260 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=67,monmap=4+}) -- 0x561936bf0620 con 0x561936eb3970 2024-09-11T00:24:30.402 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:30.399+0000 7fbc83400640 1 -- 172.21.15.12:0/259853260 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x561936e28600 con 0x561936eb3970 2024-09-11T00:24:30.403 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:30.400+0000 7fbc83400640 1 -- 172.21.15.12:0/259853260 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 67) ==== 8390+0+0 (secure 0 0 0) 0x7fbc90012070 con 0x561936eb3970 2024-09-11T00:24:30.403 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:30.400+0000 7fbca7430740 10 client.17403 fetch_fsmap finished waiting for FSMap version 67 2024-09-11T00:24:30.403 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:30.400+0000 7fbca7430740 10 client.17403 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T00:24:30.403 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:30.400+0000 7fbca7430740 20 client.17403 populate_metadata read hostname 'smithi012' 2024-09-11T00:24:30.403 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:30.400+0000 7fbca7430740 1 --2- 172.21.15.12:0/259853260 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x561936ec8a50 0x561936ecaec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:24:30.403 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:30.400+0000 7fbca7430740 4 client.17403 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T00:24:30.403 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:30.400+0000 7fbca7430740 1 -- 172.21.15.12:0/259853260 --> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561936e240b0 con 0x561936ec8a50 2024-09-11T00:24:30.403 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:30.401+0000 7fbca1200640 1 --2- 172.21.15.12:0/259853260 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x561936ec8a50 0x561936ecaec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:24:30.405 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:30.402+0000 7fbca1200640 1 --2- 172.21.15.12:0/259853260 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x561936ec8a50 0x561936ecaec0 crc :-1 s=READY pgs=247 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fa79d86ec0fd3026 server_cookie=44c6366c4f264c5c in_seq=0 out_seq=0 2024-09-11T00:24:30.405 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:30.403+0000 7fbc83400640 10 client.17403 ms_handle_connect on v2:172.21.15.165:6838/501312984 2024-09-11T00:24:30.407 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:30.405+0000 7fbc83400640 1 -- 172.21.15.12:0/259853260 <== mds.0 v2:172.21.15.165:6838/501312984 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x561936e240b0 con 0x561936ec8a50 2024-09-11T00:24:30.407 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:30.405+0000 7fbc83400640 10 client.17403 handle_command_reply: tid=0 2024-09-11T00:24:30.407 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:30.405+0000 7fbca7430740 1 client.17403 shutdown 2024-09-11T00:24:30.408 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:30.406+0000 7fbca7430740 1 -- 172.21.15.12:0/259853260 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561936eb3970 msgr2=0x561936ebd0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:24:30.408 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:30.406+0000 7fbca7430740 1 --2- 172.21.15.12:0/259853260 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561936eb3970 0x561936ebd0f0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7fbc90013fc0 tx=0x7fbc900046e0 comp rx=0 tx=0).stop 2024-09-11T00:24:30.408 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:30.406+0000 7fbca7430740 1 -- 172.21.15.12:0/259853260 shutdown_connections 2024-09-11T00:24:30.409 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:30.406+0000 7fbca7430740 1 --2- 172.21.15.12:0/259853260 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x561936ec8a50 0x561936ecaec0 crc :-1 s=READY pgs=247 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:24:30.409 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:30.406+0000 7fbc83400640 0 client.17403 ms_handle_reset on v2:172.21.15.165:6838/501312984 2024-09-11T00:24:30.409 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:30.406+0000 7fbca7430740 1 --2- 172.21.15.12:0/259853260 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561936eb8ce0 0x561936ebe360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:24:30.409 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:30.406+0000 7fbca7430740 1 --2- 172.21.15.12:0/259853260 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561936eb62a0 0x561936ebd790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:24:30.409 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:30.406+0000 7fbca7430740 1 --2- 172.21.15.12:0/259853260 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561936eb3970 0x561936ebd0f0 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:24:30.409 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:30.406+0000 7fbca7430740 1 -- 172.21.15.12:0/259853260 >> 172.21.15.12:0/259853260 conn(0x561936eaeac0 msgr2=0x561936eadbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:24:30.410 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:30.407+0000 7fbca7430740 1 -- 172.21.15.12:0/259853260 shutdown_connections 2024-09-11T00:24:30.410 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:30.407+0000 7fbca7430740 1 -- 172.21.15.12:0/259853260 wait complete. 2024-09-11T00:24:30.410 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:30.407+0000 7fbca7430740 20 client.17403 trim_cache size 0 max 16384 2024-09-11T00:24:30.440 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:30.437+0000 7fbca7430740 1 Processor -- start 2024-09-11T00:24:30.440 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:30.438+0000 7fbca7430740 1 -- start start 2024-09-11T00:24:30.441 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:30.438+0000 7fbca7430740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561936eb3970 0x561936eaf320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:24:30.442 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:30.438+0000 7fbca7430740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561936eb62a0 0x561936eaf860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:24:30.442 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:30.438+0000 7fbca7430740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561936eb8ce0 0x561936eb1900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:24:30.442 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:30.438+0000 7fbca7430740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fbc9c1b1d60 con 0x561936eb62a0 2024-09-11T00:24:30.442 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:30.438+0000 7fbca7430740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fbc9c1b3930 con 0x561936eb8ce0 2024-09-11T00:24:30.442 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:30.438+0000 7fbca7430740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fbc9c1b2460 con 0x561936eb3970 2024-09-11T00:24:30.442 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:30.439+0000 7fbca1c00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561936eb3970 0x561936eaf320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:24:30.442 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:30.439+0000 7fbca1c00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561936eb3970 0x561936eaf320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.12:54646/0 (socket says 172.21.15.12:54646) 2024-09-11T00:24:30.442 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:30.439+0000 7fbca1200640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561936eb62a0 0x561936eaf860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:24:30.442 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:30.439+0000 7fbca1c00640 1 -- 172.21.15.12:0/1577367760 learned_addr learned my addr 172.21.15.12:0/1577367760 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:24:30.443 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:30.440+0000 7fbca2600640 1 --2- 172.21.15.12:0/1577367760 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561936eb8ce0 0x561936eb1900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:24:30.443 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:30.440+0000 7fbca1200640 1 -- 172.21.15.12:0/1577367760 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561936eb3970 msgr2=0x561936eaf320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:24:30.443 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:30.440+0000 7fbca1200640 1 --2- 172.21.15.12:0/1577367760 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561936eb3970 0x561936eaf320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:24:30.443 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:30.440+0000 7fbca1200640 1 -- 172.21.15.12:0/1577367760 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561936eb8ce0 msgr2=0x561936eb1900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:24:30.443 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:30.440+0000 7fbca1200640 1 --2- 172.21.15.12:0/1577367760 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561936eb8ce0 0x561936eb1900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:24:30.443 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:30.440+0000 7fbca1200640 1 -- 172.21.15.12:0/1577367760 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561936ebe6d0 con 0x561936eb62a0 2024-09-11T00:24:30.443 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:30.440+0000 7fbca1c00640 1 --2- 172.21.15.12:0/1577367760 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561936eb3970 0x561936eaf320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T00:24:30.444 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:30.441+0000 7fbca1200640 1 --2- 172.21.15.12:0/1577367760 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561936eb62a0 0x561936eaf860 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7fbc880abf80 tx=0x7fbc880a87c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f917612b777f2ab7 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:24:30.444 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:30.442+0000 7fbc8e000640 1 -- 172.21.15.12:0/1577367760 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc88091e60 con 0x561936eb62a0 2024-09-11T00:24:30.445 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:30.442+0000 7fbc8e000640 1 -- 172.21.15.12:0/1577367760 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbc880ac740 con 0x561936eb62a0 2024-09-11T00:24:30.445 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:30.443+0000 7fbc8e000640 1 -- 172.21.15.12:0/1577367760 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc880b55d0 con 0x561936eb62a0 2024-09-11T00:24:30.447 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:30.444+0000 7fbca7430740 1 -- 172.21.15.12:0/1577367760 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561936eb62a0 msgr2=0x561936eaf860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:24:30.447 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:30.445+0000 7fbca7430740 1 --2- 172.21.15.12:0/1577367760 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561936eb62a0 0x561936eaf860 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7fbc880abf80 tx=0x7fbc880a87c0 comp rx=0 tx=0).stop 2024-09-11T00:24:30.447 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:30.445+0000 7fbca7430740 1 -- 172.21.15.12:0/1577367760 shutdown_connections 2024-09-11T00:24:30.447 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:30.445+0000 7fbca7430740 1 --2- 172.21.15.12:0/1577367760 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561936eb8ce0 0x561936eb1900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:24:30.447 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:30.445+0000 7fbca7430740 1 --2- 172.21.15.12:0/1577367760 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561936eb62a0 0x561936eaf860 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:24:30.447 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:30.445+0000 7fbca7430740 1 --2- 172.21.15.12:0/1577367760 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561936eb3970 0x561936eaf320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:24:30.447 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:30.445+0000 7fbca7430740 1 -- 172.21.15.12:0/1577367760 >> 172.21.15.12:0/1577367760 conn(0x561936eaeac0 msgr2=0x561936eb9140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:24:30.447 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:30.445+0000 7fbca7430740 1 -- 172.21.15.12:0/1577367760 shutdown_connections 2024-09-11T00:24:30.448 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:30.445+0000 7fbca7430740 1 -- 172.21.15.12:0/1577367760 wait complete. 2024-09-11T00:24:30.448 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:30.445+0000 7fbca7430740 1 -- start start 2024-09-11T00:24:30.448 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:30.446+0000 7fbca7430740 1 Processor -- start 2024-09-11T00:24:30.449 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:30.446+0000 7fbca7430740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561936eb3970 0x561936ec4e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:24:30.449 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:30.446+0000 7fbca7430740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561936eb62a0 0x561936ec3ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:24:30.449 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:30.446+0000 7fbca7430740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561936eb8ce0 0x561936ec4290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:24:30.449 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:30.447+0000 7fbca7430740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fbc9c1b3260 con 0x561936eb3970 2024-09-11T00:24:30.449 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:30.447+0000 7fbca7430740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fbc9c1b2b90 con 0x561936eb8ce0 2024-09-11T00:24:30.449 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:30.447+0000 7fbca7430740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fbc9c1afd40 con 0x561936eb62a0 2024-09-11T00:24:30.449 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:30.447+0000 7fbca7430740 10 client.0 _ll_register_callbacks cb 0x7fbc90013430 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:24:30.449 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:30.447+0000 7fbca1200640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561936eb62a0 0x561936ec3ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:24:30.450 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:30.447+0000 7fbca1200640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561936eb62a0 0x561936ec3ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.12:54662/0 (socket says 172.21.15.12:54662) 2024-09-11T00:24:30.450 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:30.447+0000 7fbca1200640 1 -- 172.21.15.12:0/4158351548 learned_addr learned my addr 172.21.15.12:0/4158351548 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:24:30.451 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:30.447+0000 7fbca1c00640 1 --2- 172.21.15.12:0/4158351548 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561936eb3970 0x561936ec4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:24:30.451 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:30.447+0000 7fbca2600640 1 --2- 172.21.15.12:0/4158351548 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561936eb8ce0 0x561936ec4290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:24:30.451 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:30.447+0000 7fbca1200640 1 -- 172.21.15.12:0/4158351548 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561936eb8ce0 msgr2=0x561936ec4290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:24:30.451 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:30.447+0000 7fbca1200640 1 --2- 172.21.15.12:0/4158351548 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561936eb8ce0 0x561936ec4290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:24:30.451 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:30.447+0000 7fbca1200640 1 -- 172.21.15.12:0/4158351548 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561936eb3970 msgr2=0x561936ec4e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:24:30.452 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:30.447+0000 7fbca1200640 1 --2- 172.21.15.12:0/4158351548 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561936eb3970 0x561936ec4e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:24:30.452 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:30.447+0000 7fbca1200640 1 -- 172.21.15.12:0/4158351548 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561936ec2640 con 0x561936eb62a0 2024-09-11T00:24:30.452 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:30.447+0000 7fbca2600640 1 --2- 172.21.15.12:0/4158351548 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561936eb8ce0 0x561936ec4290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T00:24:30.452 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:30.447+0000 7fbca1c00640 1 --2- 172.21.15.12:0/4158351548 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561936eb3970 0x561936ec4e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T00:24:30.452 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:30.448+0000 7fbca1200640 1 --2- 172.21.15.12:0/4158351548 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561936eb62a0 0x561936ec3ba0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fbc88000c00 tx=0x7fbc880abe90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bcc3ea67d197c8e1 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:24:30.452 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:30.448+0000 7fbc83400640 10 client.0 ms_handle_connect on v2:172.21.15.178:3300/0 2024-09-11T00:24:30.452 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:30.448+0000 7fbc83400640 1 -- 172.21.15.12:0/4158351548 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc88091e60 con 0x561936eb62a0 2024-09-11T00:24:30.452 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:30.448+0000 7fbc83400640 1 -- 172.21.15.12:0/4158351548 <== mon.1 v2:172.21.15.178:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9208+0+0 (secure 0 0 0) 0x7fbc880c1bc0 con 0x561936eb62a0 2024-09-11T00:24:30.452 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:30.448+0000 7fbca7430740 1 -- 172.21.15.12:0/4158351548 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbc880ac740 con 0x561936eb62a0 2024-09-11T00:24:30.452 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:30.448+0000 7fbc83400640 1 -- 172.21.15.12:0/4158351548 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=67) ==== 24+0+0 (secure 0 0 0) 0x7fbc880ab7b0 con 0x561936eb62a0 2024-09-11T00:24:30.452 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:30.448+0000 7fbca7430740 10 client.27394 fetch_fsmap learned FSMap version 67 2024-09-11T00:24:30.452 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:30.448+0000 7fbca7430740 1 -- 172.21.15.12:0/4158351548 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,fsmap=67,monmap=4+}) -- 0x561936f35c60 con 0x561936eb62a0 2024-09-11T00:24:30.452 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:30.448+0000 7fbc83400640 1 -- 172.21.15.12:0/4158351548 <== mon.1 v2:172.21.15.178:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbc880ac040 con 0x561936eb62a0 2024-09-11T00:24:30.453 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:30.449+0000 7fbc83400640 1 -- 172.21.15.12:0/4158351548 <== mon.1 v2:172.21.15.178:3300/0 5 ==== fsmap(e 67) ==== 8390+0+0 (secure 0 0 0) 0x7fbc880a7990 con 0x561936eb62a0 2024-09-11T00:24:30.453 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:30.449+0000 7fbca7430740 10 client.27394 fetch_fsmap finished waiting for FSMap version 67 2024-09-11T00:24:30.453 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:30.449+0000 7fbca7430740 10 client.27394 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T00:24:30.453 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:30.449+0000 7fbca7430740 20 client.27394 populate_metadata read hostname 'smithi012' 2024-09-11T00:24:30.453 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:30.449+0000 7fbca7430740 1 --2- 172.21.15.12:0/4158351548 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x561936ec8a50 0x561936f380b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:24:30.453 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:30.449+0000 7fbca7430740 4 client.27394 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T00:24:30.453 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:30.449+0000 7fbca7430740 1 -- 172.21.15.12:0/4158351548 --> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7fbc880ac740 con 0x561936ec8a50 2024-09-11T00:24:30.453 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:30.450+0000 7fbca1c00640 1 --2- 172.21.15.12:0/4158351548 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x561936ec8a50 0x561936f380b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:24:30.454 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:30.452+0000 7fbca1c00640 1 --2- 172.21.15.12:0/4158351548 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x561936ec8a50 0x561936f380b0 crc :-1 s=READY pgs=248 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=601638982563cebe server_cookie=552f644ee008c9d0 in_seq=0 out_seq=0 2024-09-11T00:24:30.454 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:30.452+0000 7fbc83400640 10 client.27394 ms_handle_connect on v2:172.21.15.165:6838/501312984 2024-09-11T00:24:30.766 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:30.764+0000 7fbc83400640 1 -- 172.21.15.12:0/4158351548 <== mds.0 v2:172.21.15.165:6838/501312984 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x7fbc880ac740 con 0x561936ec8a50 2024-09-11T00:24:30.767 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:30.764+0000 7fbc83400640 10 client.27394 handle_command_reply: tid=0 2024-09-11T00:24:30.767 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:30.764+0000 7fbca7430740 1 client.27394 shutdown 2024-09-11T00:24:30.767 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:30.764+0000 7fbca7430740 1 -- 172.21.15.12:0/4158351548 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561936eb62a0 msgr2=0x561936ec3ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:24:30.767 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:30.764+0000 7fbca7430740 1 --2- 172.21.15.12:0/4158351548 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561936eb62a0 0x561936ec3ba0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fbc88000c00 tx=0x7fbc880abe90 comp rx=0 tx=0).stop 2024-09-11T00:24:30.767 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:30.765+0000 7fbca7430740 1 -- 172.21.15.12:0/4158351548 shutdown_connections 2024-09-11T00:24:30.768 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:30.765+0000 7fbca7430740 1 --2- 172.21.15.12:0/4158351548 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x561936ec8a50 0x561936f380b0 crc :-1 s=READY pgs=248 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:24:30.768 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:30.765+0000 7fbca7430740 1 --2- 172.21.15.12:0/4158351548 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561936eb8ce0 0x561936ec4290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:24:30.768 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:30.765+0000 7fbca7430740 1 --2- 172.21.15.12:0/4158351548 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561936eb62a0 0x561936ec3ba0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:24:30.768 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:30.765+0000 7fbca7430740 1 --2- 172.21.15.12:0/4158351548 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561936eb3970 0x561936ec4e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:24:30.768 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:30.765+0000 7fbca7430740 1 -- 172.21.15.12:0/4158351548 >> 172.21.15.12:0/4158351548 conn(0x561936eaeac0 msgr2=0x561936eb2ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:24:30.768 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:30.765+0000 7fbc83400640 0 client.27394 ms_handle_reset on v2:172.21.15.165:6838/501312984 2024-09-11T00:24:30.768 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:30.766+0000 7fbca7430740 1 -- 172.21.15.12:0/4158351548 shutdown_connections 2024-09-11T00:24:30.769 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:30.766+0000 7fbca7430740 1 -- 172.21.15.12:0/4158351548 wait complete. 2024-09-11T00:24:30.769 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:30.767+0000 7fbca7430740 20 client.27394 trim_cache size 0 max 16384 2024-09-11T00:24:30.771 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:{ 2024-09-11T00:24:30.771 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "message": "", 2024-09-11T00:24:30.771 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "return_code": 0 2024-09-11T00:24:30.771 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:} 2024-09-11T00:24:30.773 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:30.771+0000 7fbca3000640 1 -- 172.21.15.12:0/1753046081 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fbc840b5360 msgr2=0x7fbc840b7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:24:30.773 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:30.771+0000 7fbca3000640 1 --2- 172.21.15.12:0/1753046081 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fbc840b5360 0x7fbc840b7820 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fbc90009670 tx=0x7fbc90006bd0 comp rx=0 tx=0).stop 2024-09-11T00:24:30.773 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:30.771+0000 7fbca3000640 1 -- 172.21.15.12:0/1753046081 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbc9c107ca0 msgr2=0x7fbc9c19f930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:24:30.773 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:30.771+0000 7fbca3000640 1 --2- 172.21.15.12:0/1753046081 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbc9c107ca0 0x7fbc9c19f930 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fbc8800b470 tx=0x7fbc8800b930 comp rx=0 tx=0).stop 2024-09-11T00:24:30.774 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:30.771+0000 7fbca3000640 1 -- 172.21.15.12:0/1753046081 shutdown_connections 2024-09-11T00:24:30.774 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:30.771+0000 7fbca3000640 1 --2- 172.21.15.12:0/1753046081 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fbc840b5360 0x7fbc840b7820 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:24:30.774 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:30.771+0000 7fbca3000640 1 --2- 172.21.15.12:0/1753046081 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbc9c10a6e0 0x7fbc9c1a0560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:24:30.774 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:30.771+0000 7fbca3000640 1 --2- 172.21.15.12:0/1753046081 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbc9c107ca0 0x7fbc9c19f930 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:24:30.774 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:30.771+0000 7fbca3000640 1 --2- 172.21.15.12:0/1753046081 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbc9c06b360 0x7fbc9c19f2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:24:30.775 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:30.771+0000 7fbca3000640 1 -- 172.21.15.12:0/1753046081 >> 172.21.15.12:0/1753046081 conn(0x7fbc9c0fcf30 msgr2=0x7fbc9c1088a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:24:30.775 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:30.772+0000 7fbca3000640 1 -- 172.21.15.12:0/1753046081 shutdown_connections 2024-09-11T00:24:30.775 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:30.772+0000 7fbca3000640 1 -- 172.21.15.12:0/1753046081 wait complete. 2024-09-11T00:24:31.181 INFO:tasks.workunit.client.0.smithi012.stdout: 10 86469 19.54 MB/sec execute 147 sec latency 210.385 ms 2024-09-11T00:24:31.884 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:24:31 smithi012 ceph-mon[30333]: pgmap v922: 129 pgs: 129 active+clean; 551 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 5.1 MiB/s wr, 711 op/s 2024-09-11T00:24:32.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:24:31 smithi165 ceph-mon[35356]: pgmap v922: 129 pgs: 129 active+clean; 551 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 5.1 MiB/s wr, 711 op/s 2024-09-11T00:24:32.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:24:31 smithi178 ceph-mon[35919]: pgmap v922: 129 pgs: 129 active+clean; 551 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 5.1 MiB/s wr, 711 op/s 2024-09-11T00:24:32.181 INFO:tasks.workunit.client.0.smithi012.stdout: 10 87408 19.83 MB/sec execute 148 sec latency 61.532 ms 2024-09-11T00:24:33.181 INFO:tasks.workunit.client.0.smithi012.stdout: 10 88197 19.94 MB/sec execute 149 sec latency 165.285 ms 2024-09-11T00:24:34.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:24:33 smithi165 ceph-mon[35356]: pgmap v923: 129 pgs: 129 active+clean; 561 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 4.6 MiB/s wr, 658 op/s 2024-09-11T00:24:34.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:24:33 smithi165 ceph-mon[35356]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T00:24:34.063 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:24:33 smithi012 ceph-mon[30333]: pgmap v923: 129 pgs: 129 active+clean; 561 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 4.6 MiB/s wr, 658 op/s 2024-09-11T00:24:34.063 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:24:33 smithi012 ceph-mon[30333]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T00:24:34.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:24:33 smithi178 ceph-mon[35919]: pgmap v923: 129 pgs: 129 active+clean; 561 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 4.6 MiB/s wr, 658 op/s 2024-09-11T00:24:34.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:24:33 smithi178 ceph-mon[35919]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T00:24:34.181 INFO:tasks.workunit.client.0.smithi012.stdout: 10 89036 20.10 MB/sec execute 150 sec latency 118.043 ms 2024-09-11T00:24:35.181 INFO:tasks.workunit.client.0.smithi012.stdout: 10 89898 20.26 MB/sec execute 151 sec latency 68.146 ms 2024-09-11T00:24:35.981 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:24:35 smithi012 ceph-mon[30333]: pgmap v924: 129 pgs: 129 active+clean; 561 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 MiB/s rd, 4.1 MiB/s wr, 582 op/s 2024-09-11T00:24:36.033 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:24:35 smithi165 ceph-mon[35356]: pgmap v924: 129 pgs: 129 active+clean; 561 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 MiB/s rd, 4.1 MiB/s wr, 582 op/s 2024-09-11T00:24:36.090 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:24:35 smithi178 ceph-mon[35919]: pgmap v924: 129 pgs: 129 active+clean; 561 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 MiB/s rd, 4.1 MiB/s wr, 582 op/s 2024-09-11T00:24:36.189 INFO:tasks.workunit.client.0.smithi012.stdout: 10 90126 20.20 MB/sec execute 152 sec latency 799.767 ms 2024-09-11T00:24:37.181 INFO:tasks.workunit.client.0.smithi012.stdout: 10 90127 20.06 MB/sec execute 153 sec latency 1790.580 ms 2024-09-11T00:24:37.978 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:24:37 smithi012 ceph-mon[30333]: pgmap v925: 129 pgs: 129 active+clean; 579 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 6.7 MiB/s wr, 872 op/s 2024-09-11T00:24:38.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:24:37 smithi165 ceph-mon[35356]: pgmap v925: 129 pgs: 129 active+clean; 579 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 6.7 MiB/s wr, 872 op/s 2024-09-11T00:24:38.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:24:37 smithi178 ceph-mon[35919]: pgmap v925: 129 pgs: 129 active+clean; 579 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 6.7 MiB/s wr, 872 op/s 2024-09-11T00:24:38.181 INFO:tasks.workunit.client.0.smithi012.stdout: 10 90127 19.93 MB/sec execute 154 sec latency 2790.656 ms 2024-09-11T00:24:38.813 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:24:38 smithi012 ceph-mon[30333]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T00:24:39.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:24:38 smithi165 ceph-mon[35356]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T00:24:39.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:24:38 smithi178 ceph-mon[35919]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T00:24:39.181 INFO:tasks.workunit.client.0.smithi012.stdout: 10 90129 19.81 MB/sec execute 155 sec latency 3765.787 ms 2024-09-11T00:24:39.910 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:24:39 smithi012 ceph-mon[30333]: pgmap v926: 129 pgs: 129 active+clean; 585 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 MiB/s rd, 5.5 MiB/s wr, 649 op/s 2024-09-11T00:24:40.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:24:39 smithi165 ceph-mon[35356]: pgmap v926: 129 pgs: 129 active+clean; 585 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 MiB/s rd, 5.5 MiB/s wr, 649 op/s 2024-09-11T00:24:40.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:24:39 smithi178 ceph-mon[35919]: pgmap v926: 129 pgs: 129 active+clean; 585 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 MiB/s rd, 5.5 MiB/s wr, 649 op/s 2024-09-11T00:24:40.181 INFO:tasks.workunit.client.0.smithi012.stdout: 10 90129 19.68 MB/sec execute 156 sec latency 4376.722 ms 2024-09-11T00:24:41.182 INFO:tasks.workunit.client.0.smithi012.stdout: 10 90129 19.55 MB/sec execute 157 sec latency 5377.539 ms 2024-09-11T00:24:41.819 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:24:41 smithi012 ceph-mon[30333]: pgmap v927: 129 pgs: 129 active+clean; 630 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 MiB/s rd, 12 MiB/s wr, 780 op/s 2024-09-11T00:24:42.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:24:41 smithi165 ceph-mon[35356]: pgmap v927: 129 pgs: 129 active+clean; 630 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 MiB/s rd, 12 MiB/s wr, 780 op/s 2024-09-11T00:24:42.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:24:41 smithi178 ceph-mon[35919]: pgmap v927: 129 pgs: 129 active+clean; 630 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 MiB/s rd, 12 MiB/s wr, 780 op/s 2024-09-11T00:24:42.183 INFO:tasks.workunit.client.0.smithi012.stdout: 10 90130 19.43 MB/sec execute 158 sec latency 6084.436 ms 2024-09-11T00:24:43.182 INFO:tasks.workunit.client.0.smithi012.stdout: 10 90130 19.31 MB/sec execute 159 sec latency 6900.871 ms 2024-09-11T00:24:43.901 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:43.898+0000 7f49bd800640 1 -- 172.21.15.12:0/3159913113 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f49b8105cf0 msgr2=0x7f49b8106170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:24:43.901 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:43.898+0000 7f49bd800640 1 --2- 172.21.15.12:0/3159913113 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f49b8105cf0 0x7f49b8106170 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f49a4009930 tx=0x7f49a402a420 comp rx=0 tx=0).stop 2024-09-11T00:24:43.901 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:43.898+0000 7f49bd800640 1 -- 172.21.15.12:0/3159913113 shutdown_connections 2024-09-11T00:24:43.901 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:43.898+0000 7f49bd800640 1 --2- 172.21.15.12:0/3159913113 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f49b81066b0 0x7f49b810d030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:24:43.901 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:43.898+0000 7f49bd800640 1 --2- 172.21.15.12:0/3159913113 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f49b8105cf0 0x7f49b8106170 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:24:43.901 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:43.898+0000 7f49bd800640 1 --2- 172.21.15.12:0/3159913113 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f49b8104ba0 0x7f49b8104fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:24:43.902 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:43.898+0000 7f49bd800640 1 -- 172.21.15.12:0/3159913113 >> 172.21.15.12:0/3159913113 conn(0x7f49b8100460 msgr2=0x7f49b8102880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:24:43.902 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:43.898+0000 7f49bd800640 1 -- 172.21.15.12:0/3159913113 shutdown_connections 2024-09-11T00:24:43.902 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:43.898+0000 7f49bd800640 1 -- 172.21.15.12:0/3159913113 wait complete. 2024-09-11T00:24:43.905 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:43.899+0000 7f49bd800640 1 Processor -- start 2024-09-11T00:24:43.905 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:43.899+0000 7f49bd800640 1 -- start start 2024-09-11T00:24:43.905 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:43.899+0000 7f49bd800640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f49b8104ba0 0x7f49b819aff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:24:43.905 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:43.899+0000 7f49bd800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f49b8105cf0 0x7f49b819b640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:24:43.906 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:43.899+0000 7f49bd800640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f49b81066b0 0x7f49b819c270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:24:43.906 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:43.899+0000 7f49bd800640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f49b8110840 con 0x7f49b81066b0 2024-09-11T00:24:43.906 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:43.899+0000 7f49bd800640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f49b81106c0 con 0x7f49b8105cf0 2024-09-11T00:24:43.906 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:43.899+0000 7f49bd800640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f49b81109c0 con 0x7f49b8104ba0 2024-09-11T00:24:43.906 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:43.899+0000 7f49b6a00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f49b8104ba0 0x7f49b819aff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:24:43.906 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:43.899+0000 7f49b7400640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f49b81066b0 0x7f49b819c270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:24:43.907 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:43.899+0000 7f49b6a00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f49b8104ba0 0x7f49b819aff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.12:44302/0 (socket says 172.21.15.12:44302) 2024-09-11T00:24:43.907 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:43.899+0000 7f49b6a00640 1 -- 172.21.15.12:0/1647099105 learned_addr learned my addr 172.21.15.12:0/1647099105 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:24:43.907 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:43.899+0000 7f49b6000640 1 --2- 172.21.15.12:0/1647099105 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f49b8105cf0 0x7f49b819b640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:24:43.907 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:43.899+0000 7f49b7400640 1 -- 172.21.15.12:0/1647099105 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f49b8104ba0 msgr2=0x7f49b819aff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:24:43.907 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:43.899+0000 7f49b7400640 1 --2- 172.21.15.12:0/1647099105 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f49b8104ba0 0x7f49b819aff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:24:43.907 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:43.899+0000 7f49b7400640 1 -- 172.21.15.12:0/1647099105 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f49b8105cf0 msgr2=0x7f49b819b640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:24:43.908 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:43.899+0000 7f49b7400640 1 --2- 172.21.15.12:0/1647099105 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f49b8105cf0 0x7f49b819b640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:24:43.908 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:43.899+0000 7f49b7400640 1 -- 172.21.15.12:0/1647099105 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49b819cb30 con 0x7f49b81066b0 2024-09-11T00:24:43.908 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:43.899+0000 7f49b6a00640 1 --2- 172.21.15.12:0/1647099105 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f49b8104ba0 0x7f49b819aff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:24:43.908 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:43.901+0000 7f49b7400640 1 --2- 172.21.15.12:0/1647099105 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f49b81066b0 0x7f49b819c270 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f49b000e940 tx=0x7f49b000ee00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:24:43.908 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:43.901+0000 7f49af400640 1 -- 172.21.15.12:0/1647099105 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49b0004d20 con 0x7f49b81066b0 2024-09-11T00:24:43.908 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:43.901+0000 7f49af400640 1 -- 172.21.15.12:0/1647099105 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f49b0005020 con 0x7f49b81066b0 2024-09-11T00:24:43.909 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:43.901+0000 7f49bd800640 1 -- 172.21.15.12:0/1647099105 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49b819ce20 con 0x7f49b81066b0 2024-09-11T00:24:43.909 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:43.901+0000 7f49af400640 1 -- 172.21.15.12:0/1647099105 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49b00116d0 con 0x7f49b81066b0 2024-09-11T00:24:43.909 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:43.901+0000 7f49bd800640 1 -- 172.21.15.12:0/1647099105 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49b81a34f0 con 0x7f49b81066b0 2024-09-11T00:24:43.915 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:43.901+0000 7f49c07fd740 1 Processor -- start 2024-09-11T00:24:43.915 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:43.901+0000 7f49af400640 1 -- 172.21.15.12:0/1647099105 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7f49b000f040 con 0x7f49b81066b0 2024-09-11T00:24:43.915 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:43.901+0000 7f49c07fd740 1 -- start start 2024-09-11T00:24:43.915 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:43.901+0000 7f49c07fd740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5648065e6790 0x5648065e8b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:24:43.915 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:43.901+0000 7f49c07fd740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5648065e90c0 0x5648065eb4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:24:43.915 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:43.901+0000 7f49c07fd740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5648065ebb00 0x5648065edfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:24:43.915 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:43.901+0000 7f49c07fd740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x5648065596b0 con 0x5648065e6790 2024-09-11T00:24:43.915 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:43.901+0000 7f49c07fd740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x564806548ae0 con 0x5648065ebb00 2024-09-11T00:24:43.916 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:43.901+0000 7f49c07fd740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x56480654c8f0 con 0x5648065e90c0 2024-09-11T00:24:43.916 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:43.901+0000 7f49b6000640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5648065e6790 0x5648065e8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:24:43.916 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:43.901+0000 7f49b6000640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5648065e6790 0x5648065e8b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:57812/0 (socket says 172.21.15.12:57812) 2024-09-11T00:24:43.916 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:43.901+0000 7f49b6000640 1 -- 172.21.15.12:0/1066156059 learned_addr learned my addr 172.21.15.12:0/1066156059 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:24:43.916 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:43.901+0000 7f49b6000640 1 --2- 172.21.15.12:0/1066156059 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5648065ebb00 0x5648065edfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:24:43.916 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:43.901+0000 7f49b6000640 1 -- 172.21.15.12:0/1066156059 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5648065e90c0 msgr2=0x5648065eb4b0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-11T00:24:43.916 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:43.901+0000 7f49b6000640 1 --2- 172.21.15.12:0/1066156059 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5648065e90c0 0x5648065eb4b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:24:43.916 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:43.906+0000 7f49b6000640 1 -- 172.21.15.12:0/1066156059 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5648065ebb00 msgr2=0x5648065edfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:24:43.916 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:43.906+0000 7f49b6000640 1 --2- 172.21.15.12:0/1066156059 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5648065ebb00 0x5648065edfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:24:43.916 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:43.906+0000 7f49b6000640 1 -- 172.21.15.12:0/1066156059 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564806515780 con 0x5648065e6790 2024-09-11T00:24:43.916 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:43.906+0000 7f49b6000640 1 --2- 172.21.15.12:0/1066156059 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5648065e6790 0x5648065e8b80 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f49a4031500 tx=0x7f49a4000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9deea971c7b4be7f server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:24:43.916 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:43.906+0000 7f49a3e00640 1 -- 172.21.15.12:0/1066156059 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49a4035070 con 0x5648065e6790 2024-09-11T00:24:43.917 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:43.906+0000 7f49af400640 1 --2- 172.21.15.12:0/1647099105 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f499c0b52e0 0x7f499c0b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:24:43.917 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:43.906+0000 7f49a3e00640 1 -- 172.21.15.12:0/1066156059 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f49a4003aa0 con 0x5648065e6790 2024-09-11T00:24:43.917 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:43.906+0000 7f49a3e00640 1 -- 172.21.15.12:0/1066156059 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49a403d940 con 0x5648065e6790 2024-09-11T00:24:43.917 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:43.906+0000 7f49af400640 1 -- 172.21.15.12:0/1647099105 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9208+0+0 (secure 0 0 0) 0x7f49b00e5cd0 con 0x7f49b81066b0 2024-09-11T00:24:43.917 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:43.906+0000 7f49b6a00640 1 --2- 172.21.15.12:0/1647099105 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f499c0b52e0 0x7f499c0b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:24:43.917 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:43.906+0000 7f49c07fd740 1 -- 172.21.15.12:0/1066156059 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5648065e6790 msgr2=0x5648065e8b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:24:43.917 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:43.906+0000 7f49c07fd740 1 --2- 172.21.15.12:0/1066156059 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5648065e6790 0x5648065e8b80 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f49a4031500 tx=0x7f49a4000bc0 comp rx=0 tx=0).stop 2024-09-11T00:24:43.917 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:43.906+0000 7f49c07fd740 1 -- 172.21.15.12:0/1066156059 shutdown_connections 2024-09-11T00:24:43.917 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:43.906+0000 7f49c07fd740 1 --2- 172.21.15.12:0/1066156059 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5648065ebb00 0x5648065edfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:24:43.917 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:43.906+0000 7f49c07fd740 1 --2- 172.21.15.12:0/1066156059 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5648065e90c0 0x5648065eb4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:24:43.917 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:43.906+0000 7f49c07fd740 1 --2- 172.21.15.12:0/1066156059 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5648065e6790 0x5648065e8b80 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:24:43.917 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:43.906+0000 7f49c07fd740 1 -- 172.21.15.12:0/1066156059 >> 172.21.15.12:0/1066156059 conn(0x5648065e18e0 msgr2=0x5648065e2670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:24:43.917 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:43.906+0000 7f49b6a00640 1 --2- 172.21.15.12:0/1647099105 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f499c0b52e0 0x7f499c0b77a0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f49a8011550 tx=0x7f49a8005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:24:43.918 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:43.906+0000 7f49c07fd740 1 -- 172.21.15.12:0/1066156059 shutdown_connections 2024-09-11T00:24:43.918 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:43.906+0000 7f49c07fd740 1 -- 172.21.15.12:0/1066156059 wait complete. 2024-09-11T00:24:43.918 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:43.906+0000 7f49c07fd740 1 -- start start 2024-09-11T00:24:43.918 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:43.906+0000 7f49c07fd740 1 Processor -- start 2024-09-11T00:24:43.918 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:43.906+0000 7f49c07fd740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5648065e6790 0x5648065efef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:24:43.918 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:43.906+0000 7f49c07fd740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5648065e90c0 0x5648065f0590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:24:43.918 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:43.906+0000 7f49c07fd740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5648065ebb00 0x5648065f10f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:24:43.918 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:43.906+0000 7f49c07fd740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x564806480ab0 con 0x5648065e6790 2024-09-11T00:24:43.918 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:43.906+0000 7f49c07fd740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x5648064ef570 con 0x5648065e90c0 2024-09-11T00:24:43.918 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:43.906+0000 7f49c07fd740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5648065eea80 con 0x5648065ebb00 2024-09-11T00:24:43.918 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:43.906+0000 7f49c07fd740 10 client.0 _ll_register_callbacks cb 0x5648064fe100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:24:43.918 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:43.906+0000 7f49b6a00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5648065e6790 0x5648065efef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:24:43.919 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:43.906+0000 7f49b6000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5648065e90c0 0x5648065f0590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:24:43.919 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:43.906+0000 7f49b6a00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5648065e6790 0x5648065efef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:57820/0 (socket says 172.21.15.12:57820) 2024-09-11T00:24:43.919 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:43.906+0000 7f49b6000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5648065e90c0 0x5648065f0590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.12:52758/0 (socket says 172.21.15.12:52758) 2024-09-11T00:24:43.919 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:43.906+0000 7f49b7400640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5648065ebb00 0x5648065f10f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:24:43.919 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:43.906+0000 7f49b6a00640 1 -- 172.21.15.12:0/293780111 learned_addr learned my addr 172.21.15.12:0/293780111 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:24:43.919 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:43.906+0000 7f49b6000640 1 -- 172.21.15.12:0/293780111 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5648065ebb00 msgr2=0x5648065f10f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:24:43.919 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:43.906+0000 7f49b6000640 1 --2- 172.21.15.12:0/293780111 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5648065ebb00 0x5648065f10f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:24:43.919 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:43.906+0000 7f49b6000640 1 -- 172.21.15.12:0/293780111 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5648065e6790 msgr2=0x5648065efef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:24:43.919 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:43.906+0000 7f49b6000640 1 --2- 172.21.15.12:0/293780111 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5648065e6790 0x5648065efef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:24:43.919 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:43.906+0000 7f49b6000640 1 -- 172.21.15.12:0/293780111 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5648062aa790 con 0x5648065e90c0 2024-09-11T00:24:43.919 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:43.906+0000 7f49b6000640 1 --2- 172.21.15.12:0/293780111 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5648065e90c0 0x5648065f0590 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f49a402a940 tx=0x7f49a4003e80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=14181cfea5cef354 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:24:43.919 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:43.906+0000 7f49a0c00640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-09-11T00:24:43.919 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:43.906+0000 7f49a0c00640 1 -- 172.21.15.12:0/293780111 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49a404e020 con 0x5648065e90c0 2024-09-11T00:24:43.920 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:43.906+0000 7f49c07fd740 1 -- 172.21.15.12:0/293780111 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f49a4003aa0 con 0x5648065e90c0 2024-09-11T00:24:43.920 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:43.906+0000 7f49a0c00640 1 -- 172.21.15.12:0/293780111 <== mon.2 v2:172.21.15.165:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9208+0+0 (secure 0 0 0) 0x7f49a403d6c0 con 0x5648065e90c0 2024-09-11T00:24:43.920 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:43.906+0000 7f49a0c00640 1 -- 172.21.15.12:0/293780111 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=69) ==== 24+0+0 (secure 0 0 0) 0x7f49a4059050 con 0x5648065e90c0 2024-09-11T00:24:43.920 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:43.906+0000 7f49c07fd740 10 client.27428 fetch_fsmap learned FSMap version 69 2024-09-11T00:24:43.920 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:43.906+0000 7f49c07fd740 1 -- 172.21.15.12:0/293780111 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=69,monmap=4+}) -- 0x564806323620 con 0x5648065e90c0 2024-09-11T00:24:43.920 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:43.911+0000 7f49a0c00640 1 -- 172.21.15.12:0/293780111 <== mon.2 v2:172.21.15.165:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f49a4003aa0 con 0x5648065e90c0 2024-09-11T00:24:43.920 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:43.911+0000 7f49a0c00640 1 -- 172.21.15.12:0/293780111 <== mon.2 v2:172.21.15.165:3300/0 5 ==== fsmap(e 69) ==== 8390+0+0 (secure 0 0 0) 0x7f49a4031ad0 con 0x5648065e90c0 2024-09-11T00:24:43.920 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:43.911+0000 7f49c07fd740 10 client.27428 fetch_fsmap finished waiting for FSMap version 69 2024-09-11T00:24:43.920 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:43.911+0000 7f49c07fd740 10 client.27428 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T00:24:43.920 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:43.911+0000 7f49c07fd740 20 client.27428 populate_metadata read hostname 'smithi012' 2024-09-11T00:24:43.920 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:43.911+0000 7f49c07fd740 1 --2- 172.21.15.12:0/293780111 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x5648065fb320 0x5648065fd710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:24:43.920 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:43.911+0000 7f49c07fd740 4 client.27428 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T00:24:43.920 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:43.911+0000 7f49c07fd740 1 -- 172.21.15.12:0/293780111 --> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56480655b600 con 0x5648065fb320 2024-09-11T00:24:43.921 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:43.911+0000 7f49b6a00640 1 --2- 172.21.15.12:0/293780111 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x5648065fb320 0x5648065fd710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:24:43.921 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:43.911+0000 7f49a0c00640 10 client.27428 ms_handle_connect on v2:172.21.15.165:6838/501312984 2024-09-11T00:24:43.921 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:43.911+0000 7f49b6a00640 1 --2- 172.21.15.12:0/293780111 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x5648065fb320 0x5648065fd710 crc :-1 s=READY pgs=249 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=59103ee28ee176ed server_cookie=a2d73caab6457d02 in_seq=0 out_seq=0 2024-09-11T00:24:43.922 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:43.913+0000 7f49a0c00640 1 -- 172.21.15.12:0/293780111 <== mds.0 v2:172.21.15.165:6838/501312984 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x56480655b600 con 0x5648065fb320 2024-09-11T00:24:43.922 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:43.913+0000 7f49a0c00640 10 client.27428 handle_command_reply: tid=0 2024-09-11T00:24:43.922 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:43.914+0000 7f49c07fd740 1 client.27428 shutdown 2024-09-11T00:24:43.922 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:43.914+0000 7f49c07fd740 1 -- 172.21.15.12:0/293780111 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5648065e90c0 msgr2=0x5648065f0590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:24:43.922 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:43.914+0000 7f49c07fd740 1 --2- 172.21.15.12:0/293780111 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5648065e90c0 0x5648065f0590 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f49a402a940 tx=0x7f49a4003e80 comp rx=0 tx=0).stop 2024-09-11T00:24:43.923 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:43.914+0000 7f49c07fd740 1 -- 172.21.15.12:0/293780111 shutdown_connections 2024-09-11T00:24:43.923 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:43.914+0000 7f49c07fd740 1 --2- 172.21.15.12:0/293780111 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x5648065fb320 0x5648065fd710 crc :-1 s=READY pgs=249 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:24:43.923 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:43.914+0000 7f49c07fd740 1 --2- 172.21.15.12:0/293780111 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5648065ebb00 0x5648065f10f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:24:43.923 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:43.914+0000 7f49c07fd740 1 --2- 172.21.15.12:0/293780111 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5648065e90c0 0x5648065f0590 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:24:43.923 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:43.914+0000 7f49c07fd740 1 --2- 172.21.15.12:0/293780111 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5648065e6790 0x5648065efef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:24:43.923 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:43.914+0000 7f49c07fd740 1 -- 172.21.15.12:0/293780111 >> 172.21.15.12:0/293780111 conn(0x5648065e18e0 msgr2=0x5648065e0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:24:43.923 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:43.914+0000 7f49a0c00640 0 client.27428 ms_handle_reset on v2:172.21.15.165:6838/501312984 2024-09-11T00:24:43.923 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:43.914+0000 7f49c07fd740 1 -- 172.21.15.12:0/293780111 shutdown_connections 2024-09-11T00:24:43.923 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:43.914+0000 7f49c07fd740 1 -- 172.21.15.12:0/293780111 wait complete. 2024-09-11T00:24:43.923 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:43.914+0000 7f49c07fd740 20 client.27428 trim_cache size 0 max 16384 2024-09-11T00:24:43.933 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:43.929+0000 7f49c07fd740 1 Processor -- start 2024-09-11T00:24:43.933 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:43.929+0000 7f49c07fd740 1 -- start start 2024-09-11T00:24:43.933 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:43.929+0000 7f49c07fd740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5648065e6790 0x5648065f0600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:24:43.933 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:43.930+0000 7f49c07fd740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5648065e90c0 0x5648065e3710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:24:43.934 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:43.930+0000 7f49c07fd740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5648065ebb00 0x5648065e3c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:24:43.934 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:43.930+0000 7f49c07fd740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f49b81ada40 con 0x5648065e6790 2024-09-11T00:24:43.934 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:43.930+0000 7f49c07fd740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f49b81af5e0 con 0x5648065ebb00 2024-09-11T00:24:43.934 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:43.930+0000 7f49c07fd740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f49b81ae110 con 0x5648065e90c0 2024-09-11T00:24:43.934 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:43.930+0000 7f49b6a00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5648065e6790 0x5648065f0600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:24:43.934 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:43.930+0000 7f49b6000640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5648065e90c0 0x5648065e3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:24:43.934 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:43.930+0000 7f49b7400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5648065ebb00 0x5648065e3c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:24:43.934 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:43.930+0000 7f49b6000640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5648065e90c0 0x5648065e3710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.12:44332/0 (socket says 172.21.15.12:44332) 2024-09-11T00:24:43.935 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:43.930+0000 7f49b6a00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5648065e6790 0x5648065f0600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:57822/0 (socket says 172.21.15.12:57822) 2024-09-11T00:24:43.935 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:43.930+0000 7f49b6000640 1 -- 172.21.15.12:0/4065220250 learned_addr learned my addr 172.21.15.12:0/4065220250 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:24:43.935 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:43.930+0000 7f49b6000640 1 -- 172.21.15.12:0/4065220250 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5648065ebb00 msgr2=0x5648065e3c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:24:43.935 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:43.930+0000 7f49b6000640 1 --2- 172.21.15.12:0/4065220250 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5648065ebb00 0x5648065e3c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:24:43.935 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:43.930+0000 7f49b6000640 1 -- 172.21.15.12:0/4065220250 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5648065e6790 msgr2=0x5648065f0600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:24:43.935 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:43.930+0000 7f49b6000640 1 --2- 172.21.15.12:0/4065220250 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5648065e6790 0x5648065f0600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:24:43.935 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:43.930+0000 7f49b6000640 1 -- 172.21.15.12:0/4065220250 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5648065eb820 con 0x5648065e90c0 2024-09-11T00:24:43.935 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:43.930+0000 7f49b7400640 1 --2- 172.21.15.12:0/4065220250 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5648065ebb00 0x5648065e3c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:24:43.935 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:43.930+0000 7f49b6000640 1 --2- 172.21.15.12:0/4065220250 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5648065e90c0 0x5648065e3710 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f49a4000c00 tx=0x7f49a404a5f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=11e82708f915379f server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:24:43.935 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:43.931+0000 7f49a3400640 1 -- 172.21.15.12:0/4065220250 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49a404e040 con 0x5648065e90c0 2024-09-11T00:24:43.935 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:43.931+0000 7f49a3400640 1 -- 172.21.15.12:0/4065220250 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f49a4041040 con 0x5648065e90c0 2024-09-11T00:24:43.936 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:43.931+0000 7f49a3400640 1 -- 172.21.15.12:0/4065220250 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49a4035040 con 0x5648065e90c0 2024-09-11T00:24:43.937 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:43.931+0000 7f49c07fd740 1 -- 172.21.15.12:0/4065220250 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5648065e90c0 msgr2=0x5648065e3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:24:43.938 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:43.931+0000 7f49c07fd740 1 --2- 172.21.15.12:0/4065220250 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5648065e90c0 0x5648065e3710 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f49a4000c00 tx=0x7f49a404a5f0 comp rx=0 tx=0).stop 2024-09-11T00:24:43.938 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:43.931+0000 7f49c07fd740 1 -- 172.21.15.12:0/4065220250 shutdown_connections 2024-09-11T00:24:43.938 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:43.931+0000 7f49c07fd740 1 --2- 172.21.15.12:0/4065220250 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5648065ebb00 0x5648065e3c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:24:43.938 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:43.931+0000 7f49c07fd740 1 --2- 172.21.15.12:0/4065220250 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5648065e90c0 0x5648065e3710 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:24:43.938 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:43.931+0000 7f49c07fd740 1 --2- 172.21.15.12:0/4065220250 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5648065e6790 0x5648065f0600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:24:43.938 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:43.931+0000 7f49c07fd740 1 -- 172.21.15.12:0/4065220250 >> 172.21.15.12:0/4065220250 conn(0x5648065e18e0 msgr2=0x5648065efef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:24:43.939 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:43.931+0000 7f49c07fd740 1 -- 172.21.15.12:0/4065220250 shutdown_connections 2024-09-11T00:24:43.939 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:43.931+0000 7f49c07fd740 1 -- 172.21.15.12:0/4065220250 wait complete. 2024-09-11T00:24:43.939 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:43.931+0000 7f49c07fd740 1 -- start start 2024-09-11T00:24:43.939 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:43.931+0000 7f49c07fd740 1 Processor -- start 2024-09-11T00:24:43.939 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:43.931+0000 7f49c07fd740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5648065e6790 0x5648065f6a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:24:43.939 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:43.931+0000 7f49c07fd740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5648065e90c0 0x5648065f6f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:24:43.939 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:43.931+0000 7f49c07fd740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5648065ebb00 0x5648065f74a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:24:43.939 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:43.931+0000 7f49c07fd740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f49b81aef10 con 0x5648065e90c0 2024-09-11T00:24:43.940 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:43.931+0000 7f49c07fd740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f49b81ae840 con 0x5648065e6790 2024-09-11T00:24:43.940 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:43.931+0000 7f49c07fd740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f49b81aba20 con 0x5648065ebb00 2024-09-11T00:24:43.940 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:43.931+0000 7f49c07fd740 10 client.0 _ll_register_callbacks cb 0x7f49a40041e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:24:43.940 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:43.931+0000 7f49b6000640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5648065e90c0 0x5648065f6f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:24:43.940 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:43.931+0000 7f49b6000640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5648065e90c0 0x5648065f6f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:57828/0 (socket says 172.21.15.12:57828) 2024-09-11T00:24:43.940 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:43.931+0000 7f49b6000640 1 -- 172.21.15.12:0/1611001330 learned_addr learned my addr 172.21.15.12:0/1611001330 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:24:43.940 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:43.931+0000 7f49b7400640 1 --2- 172.21.15.12:0/1611001330 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5648065ebb00 0x5648065f74a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:24:43.940 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:43.931+0000 7f49b6000640 1 -- 172.21.15.12:0/1611001330 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5648065ebb00 msgr2=0x5648065f74a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:24:43.941 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:43.931+0000 7f49b6000640 1 --2- 172.21.15.12:0/1611001330 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5648065ebb00 0x5648065f74a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:24:43.941 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:43.931+0000 7f49b6000640 1 -- 172.21.15.12:0/1611001330 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5648065e6790 msgr2=0x5648065f6a20 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-11T00:24:43.942 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:43.931+0000 7f49b6000640 1 --2- 172.21.15.12:0/1611001330 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5648065e6790 0x5648065f6a20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:24:43.942 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:43.931+0000 7f49b6000640 1 -- 172.21.15.12:0/1611001330 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5648065f5700 con 0x5648065e90c0 2024-09-11T00:24:43.942 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:43.931+0000 7f49b7400640 1 --2- 172.21.15.12:0/1611001330 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5648065ebb00 0x5648065f74a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:24:43.942 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:43.931+0000 7f49b6000640 1 --2- 172.21.15.12:0/1611001330 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5648065e90c0 0x5648065f6f60 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f49a402a940 tx=0x7f49a40095e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66c1547022196a05 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:24:43.943 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:43.931+0000 7f49a0c00640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-09-11T00:24:43.943 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:43.931+0000 7f49a0c00640 1 -- 172.21.15.12:0/1611001330 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49a404e070 con 0x5648065e90c0 2024-09-11T00:24:43.943 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:43.931+0000 7f49c07fd740 1 -- 172.21.15.12:0/1611001330 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f49a4041040 con 0x5648065e90c0 2024-09-11T00:24:43.943 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:43.931+0000 7f49a0c00640 1 -- 172.21.15.12:0/1611001330 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9208+0+0 (secure 0 0 0) 0x7f49a404a9b0 con 0x5648065e90c0 2024-09-11T00:24:43.943 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:43.931+0000 7f49a0c00640 1 -- 172.21.15.12:0/1611001330 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=69) ==== 24+0+0 (secure 0 0 0) 0x7f49a406d050 con 0x5648065e90c0 2024-09-11T00:24:43.943 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:43.931+0000 7f49c07fd740 10 client.17427 fetch_fsmap learned FSMap version 69 2024-09-11T00:24:43.943 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:43.931+0000 7f49c07fd740 1 -- 172.21.15.12:0/1611001330 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=69,monmap=4+}) -- 0x564806668590 con 0x5648065e90c0 2024-09-11T00:24:43.943 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:43.931+0000 7f49a0c00640 1 -- 172.21.15.12:0/1611001330 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f49a4041040 con 0x5648065e90c0 2024-09-11T00:24:43.943 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:43.931+0000 7f49a0c00640 1 -- 172.21.15.12:0/1611001330 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 69) ==== 8390+0+0 (secure 0 0 0) 0x7f49a403d470 con 0x5648065e90c0 2024-09-11T00:24:43.944 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:43.931+0000 7f49c07fd740 10 client.17427 fetch_fsmap finished waiting for FSMap version 69 2024-09-11T00:24:43.944 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:43.931+0000 7f49c07fd740 10 client.17427 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T00:24:43.944 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:43.931+0000 7f49c07fd740 20 client.17427 populate_metadata read hostname 'smithi012' 2024-09-11T00:24:43.944 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:43.931+0000 7f49c07fd740 1 --2- 172.21.15.12:0/1611001330 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x564806668e80 0x56480666b2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:24:43.944 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:43.931+0000 7f49c07fd740 4 client.17427 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T00:24:43.944 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:43.931+0000 7f49c07fd740 1 -- 172.21.15.12:0/1611001330 --> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5648065570b0 con 0x564806668e80 2024-09-11T00:24:43.944 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:43.937+0000 7f49b6a00640 1 --2- 172.21.15.12:0/1611001330 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x564806668e80 0x56480666b2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:24:43.974 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:43.973+0000 7f49b6a00640 1 --2- 172.21.15.12:0/1611001330 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x564806668e80 0x56480666b2a0 crc :-1 s=READY pgs=250 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=351b0a423011b87f server_cookie=3100470ace6fdfbe in_seq=0 out_seq=0 2024-09-11T00:24:43.974 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:43.973+0000 7f49a0c00640 10 client.17427 ms_handle_connect on v2:172.21.15.165:6838/501312984 2024-09-11T00:24:43.998 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:24:43 smithi012 ceph-mon[30333]: pgmap v928: 129 pgs: 129 active+clean; 631 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 12 MiB/s wr, 724 op/s 2024-09-11T00:24:44.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:24:43 smithi165 ceph-mon[35356]: pgmap v928: 129 pgs: 129 active+clean; 631 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 12 MiB/s wr, 724 op/s 2024-09-11T00:24:44.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:24:43 smithi178 ceph-mon[35919]: pgmap v928: 129 pgs: 129 active+clean; 631 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 12 MiB/s wr, 724 op/s 2024-09-11T00:24:44.143 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:44.140+0000 7f49a0c00640 1 -- 172.21.15.12:0/1611001330 <== mds.0 v2:172.21.15.165:6838/501312984 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5648065570b0 con 0x564806668e80 2024-09-11T00:24:44.143 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:44.140+0000 7f49a0c00640 10 client.17427 handle_command_reply: tid=0 2024-09-11T00:24:44.144 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:44.140+0000 7f49c07fd740 1 client.17427 shutdown 2024-09-11T00:24:44.144 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:44.140+0000 7f49c07fd740 1 -- 172.21.15.12:0/1611001330 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5648065e90c0 msgr2=0x5648065f6f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:24:44.144 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:44.140+0000 7f49c07fd740 1 --2- 172.21.15.12:0/1611001330 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5648065e90c0 0x5648065f6f60 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f49a402a940 tx=0x7f49a40095e0 comp rx=0 tx=0).stop 2024-09-11T00:24:44.144 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:44.140+0000 7f49c07fd740 1 -- 172.21.15.12:0/1611001330 shutdown_connections 2024-09-11T00:24:44.144 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:44.140+0000 7f49c07fd740 1 --2- 172.21.15.12:0/1611001330 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x564806668e80 0x56480666b2a0 crc :-1 s=READY pgs=250 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:24:44.144 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:44.140+0000 7f49c07fd740 1 --2- 172.21.15.12:0/1611001330 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5648065ebb00 0x5648065f74a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:24:44.144 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:44.140+0000 7f49c07fd740 1 --2- 172.21.15.12:0/1611001330 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5648065e90c0 0x5648065f6f60 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:24:44.144 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:44.140+0000 7f49a0c00640 0 client.17427 ms_handle_reset on v2:172.21.15.165:6838/501312984 2024-09-11T00:24:44.144 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:44.140+0000 7f49c07fd740 1 --2- 172.21.15.12:0/1611001330 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5648065e6790 0x5648065f6a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:24:44.145 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:44.140+0000 7f49c07fd740 1 -- 172.21.15.12:0/1611001330 >> 172.21.15.12:0/1611001330 conn(0x5648065e18e0 msgr2=0x5648065e0240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:24:44.145 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:44.140+0000 7f49c07fd740 1 -- 172.21.15.12:0/1611001330 shutdown_connections 2024-09-11T00:24:44.145 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:44.140+0000 7f49c07fd740 1 -- 172.21.15.12:0/1611001330 wait complete. 2024-09-11T00:24:44.145 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:44.140+0000 7f49c07fd740 20 client.17427 trim_cache size 0 max 16384 2024-09-11T00:24:44.145 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:{ 2024-09-11T00:24:44.145 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "message": "", 2024-09-11T00:24:44.145 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "return_code": 0 2024-09-11T00:24:44.145 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:} 2024-09-11T00:24:44.146 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:44.140+0000 7f49bd800640 1 -- 172.21.15.12:0/1647099105 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f499c0b52e0 msgr2=0x7f499c0b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:24:44.146 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:44.140+0000 7f49bd800640 1 --2- 172.21.15.12:0/1647099105 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f499c0b52e0 0x7f499c0b77a0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f49a8011550 tx=0x7f49a8005f90 comp rx=0 tx=0).stop 2024-09-11T00:24:44.146 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:44.140+0000 7f49bd800640 1 -- 172.21.15.12:0/1647099105 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f49b81066b0 msgr2=0x7f49b819c270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:24:44.146 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:44.140+0000 7f49bd800640 1 --2- 172.21.15.12:0/1647099105 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f49b81066b0 0x7f49b819c270 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f49b000e940 tx=0x7f49b000ee00 comp rx=0 tx=0).stop 2024-09-11T00:24:44.146 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:44.140+0000 7f49bd800640 1 -- 172.21.15.12:0/1647099105 shutdown_connections 2024-09-11T00:24:44.146 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:44.140+0000 7f49bd800640 1 --2- 172.21.15.12:0/1647099105 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f499c0b52e0 0x7f499c0b77a0 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:24:44.147 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:44.140+0000 7f49bd800640 1 --2- 172.21.15.12:0/1647099105 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f49b81066b0 0x7f49b819c270 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:24:44.147 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:44.140+0000 7f49bd800640 1 --2- 172.21.15.12:0/1647099105 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f49b8105cf0 0x7f49b819b640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:24:44.147 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:44.140+0000 7f49bd800640 1 --2- 172.21.15.12:0/1647099105 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f49b8104ba0 0x7f49b819aff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:24:44.147 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:44.140+0000 7f49bd800640 1 -- 172.21.15.12:0/1647099105 >> 172.21.15.12:0/1647099105 conn(0x7f49b8100460 msgr2=0x7f49b8107000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:24:44.147 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:44.140+0000 7f49bd800640 1 -- 172.21.15.12:0/1647099105 shutdown_connections 2024-09-11T00:24:44.147 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:44.140+0000 7f49bd800640 1 -- 172.21.15.12:0/1647099105 wait complete. 2024-09-11T00:24:44.182 INFO:tasks.workunit.client.0.smithi012.stdout: 10 90130 19.19 MB/sec execute 160 sec latency 7900.927 ms 2024-09-11T00:24:45.183 INFO:tasks.workunit.client.0.smithi012.stdout: 10 90130 19.07 MB/sec execute 161 sec latency 8900.975 ms 2024-09-11T00:24:45.975 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:24:45 smithi012 ceph-mon[30333]: pgmap v929: 129 pgs: 129 active+clean; 628 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 11 MiB/s wr, 554 op/s 2024-09-11T00:24:46.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:24:45 smithi165 ceph-mon[35356]: pgmap v929: 129 pgs: 129 active+clean; 628 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 11 MiB/s wr, 554 op/s 2024-09-11T00:24:46.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:24:45 smithi178 ceph-mon[35919]: pgmap v929: 129 pgs: 129 active+clean; 628 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 11 MiB/s wr, 554 op/s 2024-09-11T00:24:46.182 INFO:tasks.workunit.client.0.smithi012.stdout: 10 90130 18.95 MB/sec execute 162 sec latency 9901.044 ms 2024-09-11T00:24:47.183 INFO:tasks.workunit.client.0.smithi012.stdout: 10 90130 18.83 MB/sec execute 163 sec latency 10901.094 ms 2024-09-11T00:24:47.932 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:24:47 smithi012 ceph-mon[30333]: pgmap v930: 129 pgs: 129 active+clean; 644 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 16 MiB/s wr, 746 op/s 2024-09-11T00:24:48.036 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:24:47 smithi165 ceph-mon[35356]: pgmap v930: 129 pgs: 129 active+clean; 644 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 16 MiB/s wr, 746 op/s 2024-09-11T00:24:48.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:24:47 smithi178 ceph-mon[35919]: pgmap v930: 129 pgs: 129 active+clean; 644 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 16 MiB/s wr, 746 op/s 2024-09-11T00:24:48.182 INFO:tasks.workunit.client.0.smithi012.stdout: 10 90130 18.72 MB/sec execute 164 sec latency 11901.157 ms 2024-09-11T00:24:49.063 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:24:48 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T00:24:49.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:24:48 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T00:24:49.182 INFO:tasks.workunit.client.0.smithi012.stdout: 10 90130 18.60 MB/sec execute 165 sec latency 12901.215 ms 2024-09-11T00:24:49.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:24:48 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T00:24:49.893 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:24:49 smithi012 ceph-mon[30333]: pgmap v931: 129 pgs: 129 active+clean; 640 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 976 KiB/s rd, 14 MiB/s wr, 464 op/s 2024-09-11T00:24:49.893 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:24:49 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:24:49.893 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:24:49 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T00:24:49.894 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:24:49 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:24:50.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:24:49 smithi178 ceph-mon[35919]: pgmap v931: 129 pgs: 129 active+clean; 640 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 976 KiB/s rd, 14 MiB/s wr, 464 op/s 2024-09-11T00:24:50.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:24:49 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:24:50.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:24:49 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T00:24:50.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:24:49 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:24:50.183 INFO:tasks.workunit.client.0.smithi012.stdout: 10 90130 18.49 MB/sec execute 166 sec latency 13901.277 ms 2024-09-11T00:24:50.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:24:49 smithi165 ceph-mon[35356]: pgmap v931: 129 pgs: 129 active+clean; 640 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 976 KiB/s rd, 14 MiB/s wr, 464 op/s 2024-09-11T00:24:50.283 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:24:49 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:24:50.283 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:24:49 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T00:24:50.283 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:24:49 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:24:51.183 INFO:tasks.workunit.client.0.smithi012.stdout: 10 90130 18.38 MB/sec execute 167 sec latency 14901.339 ms 2024-09-11T00:24:52.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:24:51 smithi178 ceph-mon[35919]: pgmap v932: 129 pgs: 129 active+clean; 622 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 837 KiB/s rd, 13 MiB/s wr, 483 op/s 2024-09-11T00:24:52.091 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:24:51 smithi012 ceph-mon[30333]: pgmap v932: 129 pgs: 129 active+clean; 622 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 837 KiB/s rd, 13 MiB/s wr, 483 op/s 2024-09-11T00:24:52.183 INFO:tasks.workunit.client.0.smithi012.stdout: 10 90130 18.27 MB/sec execute 168 sec latency 15901.402 ms 2024-09-11T00:24:52.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:24:51 smithi165 ceph-mon[35356]: pgmap v932: 129 pgs: 129 active+clean; 622 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 837 KiB/s rd, 13 MiB/s wr, 483 op/s 2024-09-11T00:24:53.183 INFO:tasks.workunit.client.0.smithi012.stdout: 10 90130 18.16 MB/sec execute 169 sec latency 16901.443 ms 2024-09-11T00:24:54.063 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:24:53 smithi012 ceph-mon[30333]: pgmap v933: 129 pgs: 129 active+clean; 616 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 475 KiB/s rd, 6.6 MiB/s wr, 331 op/s 2024-09-11T00:24:54.183 INFO:tasks.workunit.client.0.smithi012.stdout: 10 90130 18.06 MB/sec execute 170 sec latency 17901.487 ms 2024-09-11T00:24:54.283 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:24:53 smithi165 ceph-mon[35356]: pgmap v933: 129 pgs: 129 active+clean; 616 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 475 KiB/s rd, 6.6 MiB/s wr, 331 op/s 2024-09-11T00:24:54.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:24:53 smithi178 ceph-mon[35919]: pgmap v933: 129 pgs: 129 active+clean; 616 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 475 KiB/s rd, 6.6 MiB/s wr, 331 op/s 2024-09-11T00:24:55.184 INFO:tasks.workunit.client.0.smithi012.stdout: 10 90130 17.95 MB/sec execute 171 sec latency 18901.528 ms 2024-09-11T00:24:56.014 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:24:55 smithi012 ceph-mon[30333]: pgmap v934: 129 pgs: 129 active+clean; 615 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 280 KiB/s rd, 5.4 MiB/s wr, 280 op/s 2024-09-11T00:24:56.183 INFO:tasks.workunit.client.0.smithi012.stdout: 10 90130 17.85 MB/sec execute 172 sec latency 19901.590 ms 2024-09-11T00:24:56.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:24:55 smithi165 ceph-mon[35356]: pgmap v934: 129 pgs: 129 active+clean; 615 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 280 KiB/s rd, 5.4 MiB/s wr, 280 op/s 2024-09-11T00:24:56.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:24:55 smithi178 ceph-mon[35919]: pgmap v934: 129 pgs: 129 active+clean; 615 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 280 KiB/s rd, 5.4 MiB/s wr, 280 op/s 2024-09-11T00:24:57.183 INFO:tasks.workunit.client.0.smithi012.stdout: 10 90130 17.74 MB/sec execute 173 sec latency 20901.630 ms 2024-09-11T00:24:57.262 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:57.256+0000 7f720bc00640 1 -- 172.21.15.12:0/3240474179 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7204104ba0 msgr2=0x7f7204104fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:24:57.262 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:57.256+0000 7f720bc00640 1 --2- 172.21.15.12:0/3240474179 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7204104ba0 0x7f7204104fa0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f71f8009930 tx=0x7f71f802a420 comp rx=0 tx=0).stop 2024-09-11T00:24:57.262 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:57.256+0000 7f720bc00640 1 -- 172.21.15.12:0/3240474179 shutdown_connections 2024-09-11T00:24:57.262 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:57.256+0000 7f720bc00640 1 --2- 172.21.15.12:0/3240474179 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f72041066b0 0x7f720410d030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:24:57.262 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:57.256+0000 7f720bc00640 1 --2- 172.21.15.12:0/3240474179 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7204105cf0 0x7f7204106170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:24:57.262 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:57.256+0000 7f720bc00640 1 --2- 172.21.15.12:0/3240474179 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7204104ba0 0x7f7204104fa0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:24:57.262 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:57.256+0000 7f720bc00640 1 -- 172.21.15.12:0/3240474179 >> 172.21.15.12:0/3240474179 conn(0x7f7204100460 msgr2=0x7f7204102880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:24:57.262 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:57.260+0000 7f720bc00640 1 -- 172.21.15.12:0/3240474179 shutdown_connections 2024-09-11T00:24:57.263 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:57.260+0000 7f720bc00640 1 -- 172.21.15.12:0/3240474179 wait complete. 2024-09-11T00:24:57.263 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:57.260+0000 7f720bc00640 1 Processor -- start 2024-09-11T00:24:57.263 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:57.260+0000 7f720bc00640 1 -- start start 2024-09-11T00:24:57.265 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:57.260+0000 7f720bc00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7204104ba0 0x7f720419af90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:24:57.265 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:57.260+0000 7f720bc00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7204105cf0 0x7f720419b5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:24:57.265 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:57.260+0000 7f720bc00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f72041066b0 0x7f720419c210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:24:57.265 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:57.260+0000 7f720bc00640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f7204110810 con 0x7f7204105cf0 2024-09-11T00:24:57.265 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:57.260+0000 7f720bc00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f7204110690 con 0x7f72041066b0 2024-09-11T00:24:57.265 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:57.260+0000 7f720bc00640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f7204110990 con 0x7f7204104ba0 2024-09-11T00:24:57.265 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:57.260+0000 7f7203e00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7204105cf0 0x7f720419b5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:24:57.265 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:57.260+0000 7f7203e00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7204105cf0 0x7f720419b5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:38314/0 (socket says 172.21.15.12:38314) 2024-09-11T00:24:57.265 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:57.260+0000 7f7203e00640 1 -- 172.21.15.12:0/1715288657 learned_addr learned my addr 172.21.15.12:0/1715288657 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:24:57.265 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:57.260+0000 7f7209200640 1 --2- 172.21.15.12:0/1715288657 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7204104ba0 0x7f720419af90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:24:57.265 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:57.260+0000 7f7209c00640 1 --2- 172.21.15.12:0/1715288657 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f72041066b0 0x7f720419c210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:24:57.266 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:57.260+0000 7f7203e00640 1 -- 172.21.15.12:0/1715288657 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7204104ba0 msgr2=0x7f720419af90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:24:57.266 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:57.260+0000 7f7203e00640 1 --2- 172.21.15.12:0/1715288657 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7204104ba0 0x7f720419af90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:24:57.266 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:57.260+0000 7f7203e00640 1 -- 172.21.15.12:0/1715288657 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f72041066b0 msgr2=0x7f720419c210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:24:57.266 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:57.260+0000 7f7203e00640 1 --2- 172.21.15.12:0/1715288657 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f72041066b0 0x7f720419c210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:24:57.266 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:57.260+0000 7f7203e00640 1 -- 172.21.15.12:0/1715288657 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f720419cad0 con 0x7f7204105cf0 2024-09-11T00:24:57.266 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:57.260+0000 7f7203e00640 1 --2- 172.21.15.12:0/1715288657 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7204105cf0 0x7f720419b5e0 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f71f000e9f0 tx=0x7f71f000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:24:57.266 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:57.260+0000 7f7201600640 1 -- 172.21.15.12:0/1715288657 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71f000cd80 con 0x7f7204105cf0 2024-09-11T00:24:57.266 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:57.260+0000 7f720bc00640 1 -- 172.21.15.12:0/1715288657 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f720419cdc0 con 0x7f7204105cf0 2024-09-11T00:24:57.266 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:57.260+0000 7f7201600640 1 -- 172.21.15.12:0/1715288657 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f71f00054d0 con 0x7f7204105cf0 2024-09-11T00:24:57.266 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:57.260+0000 7f7201600640 1 -- 172.21.15.12:0/1715288657 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71f0010690 con 0x7f7204105cf0 2024-09-11T00:24:57.266 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:57.260+0000 7f720bc00640 1 -- 172.21.15.12:0/1715288657 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f72041a3490 con 0x7f7204105cf0 2024-09-11T00:24:57.267 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:57.263+0000 7f720ebf4740 1 Processor -- start 2024-09-11T00:24:57.267 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:57.263+0000 7f720ebf4740 1 -- start start 2024-09-11T00:24:57.267 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:57.263+0000 7f720ebf4740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f126948790 0x55f12694ab80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:24:57.267 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:57.263+0000 7f720ebf4740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f12694b0c0 0x55f12694d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:24:57.267 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:57.263+0000 7f720ebf4740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f12694db00 0x55f12694ffc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:24:57.268 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:57.263+0000 7f720ebf4740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55f1268bb6b0 con 0x55f12694b0c0 2024-09-11T00:24:57.268 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:57.263+0000 7f720ebf4740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55f1268aaae0 con 0x55f126948790 2024-09-11T00:24:57.268 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:57.263+0000 7f720ebf4740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55f1268ae8f0 con 0x55f12694db00 2024-09-11T00:24:57.269 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:57.263+0000 7f7209c00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f126948790 0x55f12694ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:24:57.269 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:57.263+0000 7f7209200640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f12694b0c0 0x55f12694d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:24:57.269 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:57.263+0000 7f7209c00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f126948790 0x55f12694ab80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.12:50664/0 (socket says 172.21.15.12:50664) 2024-09-11T00:24:57.269 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:57.263+0000 7f7209c00640 1 -- 172.21.15.12:0/3551497037 learned_addr learned my addr 172.21.15.12:0/3551497037 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:24:57.269 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:57.263+0000 7f7209c00640 1 -- 172.21.15.12:0/3551497037 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f12694db00 msgr2=0x55f12694ffc0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-11T00:24:57.269 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:57.263+0000 7f7209c00640 1 --2- 172.21.15.12:0/3551497037 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f12694db00 0x55f12694ffc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:24:57.269 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:57.263+0000 7f7209c00640 1 -- 172.21.15.12:0/3551497037 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f12694b0c0 msgr2=0x55f12694d4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:24:57.269 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:57.263+0000 7f7209c00640 1 --2- 172.21.15.12:0/3551497037 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f12694b0c0 0x55f12694d4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:24:57.269 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:57.263+0000 7f7209c00640 1 -- 172.21.15.12:0/3551497037 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f126877780 con 0x55f126948790 2024-09-11T00:24:57.270 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:57.263+0000 7f7201600640 1 -- 172.21.15.12:0/1715288657 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7f71f0004e90 con 0x7f7204105cf0 2024-09-11T00:24:57.270 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:57.263+0000 7f7209c00640 1 --2- 172.21.15.12:0/3551497037 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f126948790 0x55f12694ab80 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f71fc009f90 tx=0x7f71fc0118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2f7283ac507c0955 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:24:57.270 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:57.263+0000 7f71f6000640 1 -- 172.21.15.12:0/3551497037 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71fc004b20 con 0x55f126948790 2024-09-11T00:24:57.270 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:57.263+0000 7f71f6000640 1 -- 172.21.15.12:0/3551497037 <== mon.2 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f71fc004cc0 con 0x55f126948790 2024-09-11T00:24:57.270 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:57.263+0000 7f71f6000640 1 -- 172.21.15.12:0/3551497037 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71fc004860 con 0x55f126948790 2024-09-11T00:24:57.273 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:57.263+0000 7f720ebf4740 1 -- 172.21.15.12:0/3551497037 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f126948790 msgr2=0x55f12694ab80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:24:57.273 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:57.263+0000 7f720ebf4740 1 --2- 172.21.15.12:0/3551497037 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f126948790 0x55f12694ab80 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f71fc009f90 tx=0x7f71fc0118d0 comp rx=0 tx=0).stop 2024-09-11T00:24:57.273 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:57.263+0000 7f720ebf4740 1 -- 172.21.15.12:0/3551497037 shutdown_connections 2024-09-11T00:24:57.273 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:57.263+0000 7f720ebf4740 1 --2- 172.21.15.12:0/3551497037 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f12694db00 0x55f12694ffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:24:57.273 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:57.263+0000 7f720ebf4740 1 --2- 172.21.15.12:0/3551497037 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f12694b0c0 0x55f12694d4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:24:57.273 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:57.263+0000 7f720ebf4740 1 --2- 172.21.15.12:0/3551497037 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f126948790 0x55f12694ab80 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:24:57.273 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:57.263+0000 7f720ebf4740 1 -- 172.21.15.12:0/3551497037 >> 172.21.15.12:0/3551497037 conn(0x55f1269438e0 msgr2=0x55f126944670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:24:57.273 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:57.263+0000 7f720ebf4740 1 -- 172.21.15.12:0/3551497037 shutdown_connections 2024-09-11T00:24:57.273 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:57.263+0000 7f720ebf4740 1 -- 172.21.15.12:0/3551497037 wait complete. 2024-09-11T00:24:57.273 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:57.268+0000 7f7201600640 1 --2- 172.21.15.12:0/1715288657 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f71ec0b5270 0x7f71ec0b7730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:24:57.274 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:57.268+0000 7f720ebf4740 1 -- start start 2024-09-11T00:24:57.274 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:57.268+0000 7f7201600640 1 -- 172.21.15.12:0/1715288657 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9208+0+0 (secure 0 0 0) 0x7f71f00e63e0 con 0x7f7204105cf0 2024-09-11T00:24:57.274 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:57.268+0000 7f7209c00640 1 --2- 172.21.15.12:0/1715288657 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f71ec0b5270 0x7f71ec0b7730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:24:57.274 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:57.268+0000 7f720ebf4740 1 Processor -- start 2024-09-11T00:24:57.274 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:57.268+0000 7f720ebf4740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f126948790 0x55f126951e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:24:57.274 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:57.268+0000 7f720ebf4740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f12694b0c0 0x55f126952530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:24:57.274 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:57.268+0000 7f720ebf4740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f12694db00 0x55f126953100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:24:57.274 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:57.268+0000 7f720ebf4740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55f1267e2ab0 con 0x55f126948790 2024-09-11T00:24:57.274 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:57.268+0000 7f720ebf4740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55f126851570 con 0x55f12694db00 2024-09-11T00:24:57.274 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:57.268+0000 7f720ebf4740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55f126950ae0 con 0x55f12694b0c0 2024-09-11T00:24:57.274 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:57.268+0000 7f720ebf4740 10 client.0 _ll_register_callbacks cb 0x55f126860100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:24:57.274 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:57.268+0000 7f7209c00640 1 --2- 172.21.15.12:0/1715288657 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f71ec0b5270 0x7f71ec0b7730 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f71fc01b040 tx=0x7f71fc005bd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:24:57.274 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:57.268+0000 7f7209200640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f126948790 0x55f126951e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:24:57.275 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:57.268+0000 7f7209200640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f126948790 0x55f126951e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:38346/0 (socket says 172.21.15.12:38346) 2024-09-11T00:24:57.275 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:57.268+0000 7f7209200640 1 -- 172.21.15.12:0/1300155970 learned_addr learned my addr 172.21.15.12:0/1300155970 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:24:57.275 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:57.268+0000 7f7203e00640 1 --2- 172.21.15.12:0/1300155970 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f12694b0c0 0x55f126952530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:24:57.275 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:57.268+0000 7f7209c00640 1 --2- 172.21.15.12:0/1300155970 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f12694db00 0x55f126953100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:24:57.275 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:57.268+0000 7f7209200640 1 -- 172.21.15.12:0/1300155970 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f12694b0c0 msgr2=0x55f126952530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:24:57.275 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:57.268+0000 7f7209200640 1 --2- 172.21.15.12:0/1300155970 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f12694b0c0 0x55f126952530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:24:57.275 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:57.268+0000 7f7209200640 1 -- 172.21.15.12:0/1300155970 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f12694db00 msgr2=0x55f126953100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:24:57.275 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:57.268+0000 7f7209200640 1 --2- 172.21.15.12:0/1300155970 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f12694db00 0x55f126953100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:24:57.275 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:57.268+0000 7f7209200640 1 -- 172.21.15.12:0/1300155970 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f12660c790 con 0x55f126948790 2024-09-11T00:24:57.275 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:57.268+0000 7f7209c00640 1 --2- 172.21.15.12:0/1300155970 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f12694db00 0x55f126953100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T00:24:57.275 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:57.268+0000 7f7203e00640 1 --2- 172.21.15.12:0/1300155970 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f12694b0c0 0x55f126952530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:24:57.275 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:57.268+0000 7f7209200640 1 --2- 172.21.15.12:0/1300155970 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f126948790 0x55f126951e90 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f71f802a940 tx=0x7f71f8031db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1b433f1f4ae4b5d server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:24:57.275 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:57.268+0000 7f71eaa00640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-09-11T00:24:57.276 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:57.268+0000 7f71eaa00640 1 -- 172.21.15.12:0/1300155970 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71f8035040 con 0x55f126948790 2024-09-11T00:24:57.276 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:57.268+0000 7f71eaa00640 1 -- 172.21.15.12:0/1300155970 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9208+0+0 (secure 0 0 0) 0x7f71f8004040 con 0x55f126948790 2024-09-11T00:24:57.276 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:57.268+0000 7f720ebf4740 1 -- 172.21.15.12:0/1300155970 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f71fc004cc0 con 0x55f126948790 2024-09-11T00:24:57.276 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:57.268+0000 7f71eaa00640 1 -- 172.21.15.12:0/1300155970 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=69) ==== 24+0+0 (secure 0 0 0) 0x7f71f804e050 con 0x55f126948790 2024-09-11T00:24:57.276 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:57.268+0000 7f720ebf4740 10 client.17439 fetch_fsmap learned FSMap version 69 2024-09-11T00:24:57.276 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:57.268+0000 7f720ebf4740 1 -- 172.21.15.12:0/1300155970 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=69,monmap=4+}) -- 0x55f126685620 con 0x55f126948790 2024-09-11T00:24:57.278 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:57.268+0000 7f71eaa00640 1 -- 172.21.15.12:0/1300155970 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f71fc004cc0 con 0x55f126948790 2024-09-11T00:24:57.278 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:57.268+0000 7f71eaa00640 1 -- 172.21.15.12:0/1300155970 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 69) ==== 8390+0+0 (secure 0 0 0) 0x7f71f803c440 con 0x55f126948790 2024-09-11T00:24:57.278 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:57.268+0000 7f720ebf4740 10 client.17439 fetch_fsmap finished waiting for FSMap version 69 2024-09-11T00:24:57.278 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:57.268+0000 7f720ebf4740 10 client.17439 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T00:24:57.278 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:57.268+0000 7f720ebf4740 20 client.17439 populate_metadata read hostname 'smithi012' 2024-09-11T00:24:57.278 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:57.268+0000 7f720ebf4740 1 --2- 172.21.15.12:0/1300155970 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55f12695d750 0x55f12695fba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:24:57.278 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:57.268+0000 7f720ebf4740 4 client.17439 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T00:24:57.278 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:57.268+0000 7f720ebf4740 1 -- 172.21.15.12:0/1300155970 --> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f1268bd600 con 0x55f12695d750 2024-09-11T00:24:57.278 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:57.268+0000 7f7203e00640 1 --2- 172.21.15.12:0/1300155970 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55f12695d750 0x55f12695fba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:24:57.278 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:57.268+0000 7f7203e00640 1 --2- 172.21.15.12:0/1300155970 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55f12695d750 0x55f12695fba0 crc :-1 s=READY pgs=251 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d253fda8c7183616 server_cookie=a792cf011060125e in_seq=0 out_seq=0 2024-09-11T00:24:57.279 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:57.268+0000 7f71eaa00640 10 client.17439 ms_handle_connect on v2:172.21.15.165:6838/501312984 2024-09-11T00:24:57.279 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:57.273+0000 7f71eaa00640 1 -- 172.21.15.12:0/1300155970 <== mds.0 v2:172.21.15.165:6838/501312984 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55f1268bd600 con 0x55f12695d750 2024-09-11T00:24:57.279 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:57.273+0000 7f71eaa00640 10 client.17439 handle_command_reply: tid=0 2024-09-11T00:24:57.279 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:57.273+0000 7f720ebf4740 1 client.17439 shutdown 2024-09-11T00:24:57.279 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:57.273+0000 7f720ebf4740 1 -- 172.21.15.12:0/1300155970 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f126948790 msgr2=0x55f126951e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:24:57.279 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:57.273+0000 7f720ebf4740 1 --2- 172.21.15.12:0/1300155970 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f126948790 0x55f126951e90 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f71f802a940 tx=0x7f71f8031db0 comp rx=0 tx=0).stop 2024-09-11T00:24:57.279 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:57.273+0000 7f720ebf4740 1 -- 172.21.15.12:0/1300155970 shutdown_connections 2024-09-11T00:24:57.279 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:57.273+0000 7f720ebf4740 1 --2- 172.21.15.12:0/1300155970 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55f12695d750 0x55f12695fba0 crc :-1 s=READY pgs=251 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:24:57.279 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:57.273+0000 7f720ebf4740 1 --2- 172.21.15.12:0/1300155970 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f12694db00 0x55f126953100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:24:57.279 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:57.273+0000 7f720ebf4740 1 --2- 172.21.15.12:0/1300155970 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f12694b0c0 0x55f126952530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:24:57.279 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:57.273+0000 7f720ebf4740 1 --2- 172.21.15.12:0/1300155970 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f126948790 0x55f126951e90 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:24:57.279 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:57.273+0000 7f720ebf4740 1 -- 172.21.15.12:0/1300155970 >> 172.21.15.12:0/1300155970 conn(0x55f1269438e0 msgr2=0x55f126942a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:24:57.279 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:57.273+0000 7f71eaa00640 0 client.17439 ms_handle_reset on v2:172.21.15.165:6838/501312984 2024-09-11T00:24:57.280 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:57.273+0000 7f720ebf4740 1 -- 172.21.15.12:0/1300155970 shutdown_connections 2024-09-11T00:24:57.280 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:57.273+0000 7f720ebf4740 1 -- 172.21.15.12:0/1300155970 wait complete. 2024-09-11T00:24:57.280 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:57.273+0000 7f720ebf4740 20 client.17439 trim_cache size 0 max 16384 2024-09-11T00:24:57.306 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:57.299+0000 7f720ebf4740 1 Processor -- start 2024-09-11T00:24:57.306 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:57.299+0000 7f720ebf4740 1 -- start start 2024-09-11T00:24:57.306 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:57.299+0000 7f720ebf4740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f126948790 0x55f126946930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:24:57.306 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:57.299+0000 7f720ebf4740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f12694b0c0 0x55f126946e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:24:57.306 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:57.299+0000 7f720ebf4740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f12694db00 0x55f126942240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:24:57.306 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:57.299+0000 7f720ebf4740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f72041ad930 con 0x55f12694b0c0 2024-09-11T00:24:57.306 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:57.299+0000 7f720ebf4740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f72041af4d0 con 0x55f126948790 2024-09-11T00:24:57.306 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:57.299+0000 7f720ebf4740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f72041ae000 con 0x55f12694db00 2024-09-11T00:24:57.306 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:57.299+0000 7f7203e00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f12694b0c0 0x55f126946e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:24:57.307 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:57.299+0000 7f7209200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f126948790 0x55f126946930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:24:57.307 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:57.299+0000 7f7203e00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f12694b0c0 0x55f126946e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:38348/0 (socket says 172.21.15.12:38348) 2024-09-11T00:24:57.307 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:57.299+0000 7f7203e00640 1 -- 172.21.15.12:0/2275458073 learned_addr learned my addr 172.21.15.12:0/2275458073 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:24:57.307 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:57.299+0000 7f7209c00640 1 --2- 172.21.15.12:0/2275458073 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f12694db00 0x55f126942240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:24:57.307 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:57.299+0000 7f7209200640 1 -- 172.21.15.12:0/2275458073 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f12694db00 msgr2=0x55f126942240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:24:57.307 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:57.299+0000 7f7209200640 1 --2- 172.21.15.12:0/2275458073 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f12694db00 0x55f126942240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:24:57.307 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:57.299+0000 7f7209200640 1 -- 172.21.15.12:0/2275458073 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f12694b0c0 msgr2=0x55f126946e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:24:57.307 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:57.299+0000 7f7209200640 1 --2- 172.21.15.12:0/2275458073 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f12694b0c0 0x55f126946e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:24:57.307 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:57.299+0000 7f7209200640 1 -- 172.21.15.12:0/2275458073 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f12694aef0 con 0x55f126948790 2024-09-11T00:24:57.307 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:57.299+0000 7f7203e00640 1 --2- 172.21.15.12:0/2275458073 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f12694b0c0 0x55f126946e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:24:57.307 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:57.299+0000 7f7209200640 1 --2- 172.21.15.12:0/2275458073 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f126948790 0x55f126946930 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f71f8005ab0 tx=0x7f71f802a420 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fe30baf869c70c34 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:24:57.307 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:57.299+0000 7f71f5600640 1 -- 172.21.15.12:0/2275458073 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71f8057070 con 0x55f126948790 2024-09-11T00:24:57.307 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:57.299+0000 7f71f5600640 1 -- 172.21.15.12:0/2275458073 <== mon.2 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f71f803c640 con 0x55f126948790 2024-09-11T00:24:57.308 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:57.299+0000 7f71f5600640 1 -- 172.21.15.12:0/2275458073 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71f8035040 con 0x55f126948790 2024-09-11T00:24:57.310 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:57.304+0000 7f720ebf4740 1 -- 172.21.15.12:0/2275458073 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f126948790 msgr2=0x55f126946930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:24:57.310 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:57.304+0000 7f720ebf4740 1 --2- 172.21.15.12:0/2275458073 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f126948790 0x55f126946930 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f71f8005ab0 tx=0x7f71f802a420 comp rx=0 tx=0).stop 2024-09-11T00:24:57.310 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:57.304+0000 7f720ebf4740 1 -- 172.21.15.12:0/2275458073 shutdown_connections 2024-09-11T00:24:57.311 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:57.304+0000 7f720ebf4740 1 --2- 172.21.15.12:0/2275458073 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f12694db00 0x55f126942240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:24:57.311 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:57.304+0000 7f720ebf4740 1 --2- 172.21.15.12:0/2275458073 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f12694b0c0 0x55f126946e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:24:57.311 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:57.304+0000 7f720ebf4740 1 --2- 172.21.15.12:0/2275458073 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f126948790 0x55f126946930 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:24:57.311 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:57.304+0000 7f720ebf4740 1 -- 172.21.15.12:0/2275458073 >> 172.21.15.12:0/2275458073 conn(0x55f1269438e0 msgr2=0x55f126947c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:24:57.311 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:57.304+0000 7f720ebf4740 1 -- 172.21.15.12:0/2275458073 shutdown_connections 2024-09-11T00:24:57.311 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:57.304+0000 7f720ebf4740 1 -- 172.21.15.12:0/2275458073 wait complete. 2024-09-11T00:24:57.311 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:57.304+0000 7f720ebf4740 1 -- start start 2024-09-11T00:24:57.311 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:57.304+0000 7f720ebf4740 1 Processor -- start 2024-09-11T00:24:57.311 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:57.304+0000 7f720ebf4740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f126948790 0x55f126959f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:24:57.311 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:57.304+0000 7f720ebf4740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f12694b0c0 0x55f126958c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:24:57.311 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:57.304+0000 7f720ebf4740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f12694db00 0x55f1269591d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:24:57.311 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:57.304+0000 7f720ebf4740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f72041aee00 con 0x55f12694db00 2024-09-11T00:24:57.311 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:57.304+0000 7f720ebf4740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f72041ae730 con 0x55f126948790 2024-09-11T00:24:57.312 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:57.304+0000 7f720ebf4740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f72041ab910 con 0x55f12694b0c0 2024-09-11T00:24:57.312 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:57.304+0000 7f720ebf4740 10 client.0 _ll_register_callbacks cb 0x7f71f8009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:24:57.312 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:57.304+0000 7f7209c00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f12694db00 0x55f1269591d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:24:57.312 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:57.304+0000 7f7209c00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f12694db00 0x55f1269591d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:38350/0 (socket says 172.21.15.12:38350) 2024-09-11T00:24:57.312 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:57.304+0000 7f7209c00640 1 -- 172.21.15.12:0/217362244 learned_addr learned my addr 172.21.15.12:0/217362244 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:24:57.312 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:57.304+0000 7f7209200640 1 --2- 172.21.15.12:0/217362244 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f126948790 0x55f126959f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:24:57.312 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:57.304+0000 7f7203e00640 1 --2- 172.21.15.12:0/217362244 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f12694b0c0 0x55f126958c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:24:57.312 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:57.304+0000 7f7209c00640 1 -- 172.21.15.12:0/217362244 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f12694b0c0 msgr2=0x55f126958c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:24:57.312 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:57.304+0000 7f7209c00640 1 --2- 172.21.15.12:0/217362244 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f12694b0c0 0x55f126958c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:24:57.312 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:57.304+0000 7f7209c00640 1 -- 172.21.15.12:0/217362244 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f126948790 msgr2=0x55f126959f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:24:57.312 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:57.304+0000 7f7209c00640 1 --2- 172.21.15.12:0/217362244 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f126948790 0x55f126959f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:24:57.312 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:57.304+0000 7f7209c00640 1 -- 172.21.15.12:0/217362244 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f126959a00 con 0x55f12694db00 2024-09-11T00:24:57.313 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:57.304+0000 7f7209c00640 1 --2- 172.21.15.12:0/217362244 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f12694db00 0x55f1269591d0 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f71fc000f80 tx=0x7f71fc0118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=700d35ce265bb2ba server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:24:57.313 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:57.304+0000 7f71eaa00640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-09-11T00:24:57.313 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:57.304+0000 7f71eaa00640 1 -- 172.21.15.12:0/217362244 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71fc02f7d0 con 0x55f12694db00 2024-09-11T00:24:57.313 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:57.304+0000 7f71eaa00640 1 -- 172.21.15.12:0/217362244 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9208+0+0 (secure 0 0 0) 0x7f71fc02f970 con 0x55f12694db00 2024-09-11T00:24:57.313 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:57.304+0000 7f720ebf4740 1 -- 172.21.15.12:0/217362244 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f71f803c640 con 0x55f12694db00 2024-09-11T00:24:57.313 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:57.304+0000 7f71eaa00640 1 -- 172.21.15.12:0/217362244 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=69) ==== 24+0+0 (secure 0 0 0) 0x7f71fc042050 con 0x55f12694db00 2024-09-11T00:24:57.313 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:57.304+0000 7f720ebf4740 10 client.17445 fetch_fsmap learned FSMap version 69 2024-09-11T00:24:57.313 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:57.304+0000 7f720ebf4740 1 -- 172.21.15.12:0/217362244 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=69,monmap=4+}) -- 0x55f1269ca840 con 0x55f12694db00 2024-09-11T00:24:57.313 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:57.304+0000 7f71eaa00640 1 -- 172.21.15.12:0/217362244 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f71f803c640 con 0x55f12694db00 2024-09-11T00:24:57.313 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:57.304+0000 7f71eaa00640 1 -- 172.21.15.12:0/217362244 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 69) ==== 8390+0+0 (secure 0 0 0) 0x7f71fc024c90 con 0x55f12694db00 2024-09-11T00:24:57.313 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:57.304+0000 7f720ebf4740 10 client.17445 fetch_fsmap finished waiting for FSMap version 69 2024-09-11T00:24:57.313 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:57.304+0000 7f720ebf4740 10 client.17445 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T00:24:57.313 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:57.304+0000 7f720ebf4740 20 client.17445 populate_metadata read hostname 'smithi012' 2024-09-11T00:24:57.314 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:57.304+0000 7f720ebf4740 1 --2- 172.21.15.12:0/217362244 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55f1269cb080 0x55f1269cd4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:24:57.314 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:57.304+0000 7f720ebf4740 4 client.17445 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T00:24:57.314 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:57.304+0000 7f720ebf4740 1 -- 172.21.15.12:0/217362244 --> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55f12695ff30 con 0x55f1269cb080 2024-09-11T00:24:57.314 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:57.304+0000 7f7209200640 1 --2- 172.21.15.12:0/217362244 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55f1269cb080 0x55f1269cd4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:24:57.314 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:57.304+0000 7f7209200640 1 --2- 172.21.15.12:0/217362244 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55f1269cb080 0x55f1269cd4a0 crc :-1 s=READY pgs=252 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f0b20aa72eee524d server_cookie=d7c5c7f0147f0931 in_seq=0 out_seq=0 2024-09-11T00:24:57.314 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:57.304+0000 7f71eaa00640 10 client.17445 ms_handle_connect on v2:172.21.15.165:6838/501312984 2024-09-11T00:24:57.366 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:57.364+0000 7f71eaa00640 1 -- 172.21.15.12:0/217362244 <== mds.0 v2:172.21.15.165:6838/501312984 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55f12695ff30 con 0x55f1269cb080 2024-09-11T00:24:57.366 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:57.364+0000 7f71eaa00640 10 client.17445 handle_command_reply: tid=0 2024-09-11T00:24:57.366 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:57.364+0000 7f720ebf4740 1 client.17445 shutdown 2024-09-11T00:24:57.366 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:57.364+0000 7f720ebf4740 1 -- 172.21.15.12:0/217362244 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f12694db00 msgr2=0x55f1269591d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:24:57.366 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:57.364+0000 7f720ebf4740 1 --2- 172.21.15.12:0/217362244 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f12694db00 0x55f1269591d0 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f71fc000f80 tx=0x7f71fc0118d0 comp rx=0 tx=0).stop 2024-09-11T00:24:57.366 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:57.364+0000 7f720ebf4740 1 -- 172.21.15.12:0/217362244 shutdown_connections 2024-09-11T00:24:57.366 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:57.364+0000 7f720ebf4740 1 --2- 172.21.15.12:0/217362244 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55f1269cb080 0x55f1269cd4a0 crc :-1 s=READY pgs=252 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:24:57.367 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:57.364+0000 7f720ebf4740 1 --2- 172.21.15.12:0/217362244 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f12694db00 0x55f1269591d0 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:24:57.367 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:57.364+0000 7f720ebf4740 1 --2- 172.21.15.12:0/217362244 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f12694b0c0 0x55f126958c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:24:57.367 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:57.364+0000 7f720ebf4740 1 --2- 172.21.15.12:0/217362244 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f126948790 0x55f126959f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:24:57.367 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:57.364+0000 7f71eaa00640 0 client.17445 ms_handle_reset on v2:172.21.15.165:6838/501312984 2024-09-11T00:24:57.367 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:57.364+0000 7f720ebf4740 1 -- 172.21.15.12:0/217362244 >> 172.21.15.12:0/217362244 conn(0x55f1269438e0 msgr2=0x55f126947850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:24:57.367 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:57.364+0000 7f720ebf4740 1 -- 172.21.15.12:0/217362244 shutdown_connections 2024-09-11T00:24:57.367 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:57.364+0000 7f720ebf4740 1 -- 172.21.15.12:0/217362244 wait complete. 2024-09-11T00:24:57.367 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:57.364+0000 7f720ebf4740 20 client.17445 trim_cache size 0 max 16384 2024-09-11T00:24:57.368 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:{ 2024-09-11T00:24:57.368 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "message": "", 2024-09-11T00:24:57.368 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "return_code": 0 2024-09-11T00:24:57.368 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:} 2024-09-11T00:24:57.369 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:57.364+0000 7f720bc00640 1 -- 172.21.15.12:0/1715288657 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f71ec0b5270 msgr2=0x7f71ec0b7730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:24:57.369 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:57.364+0000 7f720bc00640 1 --2- 172.21.15.12:0/1715288657 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f71ec0b5270 0x7f71ec0b7730 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f71fc01b040 tx=0x7f71fc005bd0 comp rx=0 tx=0).stop 2024-09-11T00:24:57.369 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:57.364+0000 7f720bc00640 1 -- 172.21.15.12:0/1715288657 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7204105cf0 msgr2=0x7f720419b5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:24:57.369 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:57.364+0000 7f720bc00640 1 --2- 172.21.15.12:0/1715288657 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7204105cf0 0x7f720419b5e0 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f71f000e9f0 tx=0x7f71f000edf0 comp rx=0 tx=0).stop 2024-09-11T00:24:57.369 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:57.364+0000 7f720bc00640 1 -- 172.21.15.12:0/1715288657 shutdown_connections 2024-09-11T00:24:57.369 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:57.364+0000 7f720bc00640 1 --2- 172.21.15.12:0/1715288657 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f71ec0b5270 0x7f71ec0b7730 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:24:57.369 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:57.364+0000 7f720bc00640 1 --2- 172.21.15.12:0/1715288657 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f72041066b0 0x7f720419c210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:24:57.369 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:57.364+0000 7f720bc00640 1 --2- 172.21.15.12:0/1715288657 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7204105cf0 0x7f720419b5e0 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:24:57.369 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:57.364+0000 7f720bc00640 1 --2- 172.21.15.12:0/1715288657 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7204104ba0 0x7f720419af90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:24:57.369 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:57.364+0000 7f720bc00640 1 -- 172.21.15.12:0/1715288657 >> 172.21.15.12:0/1715288657 conn(0x7f7204100460 msgr2=0x7f72041090e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:24:57.369 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:57.364+0000 7f720bc00640 1 -- 172.21.15.12:0/1715288657 shutdown_connections 2024-09-11T00:24:57.369 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:24:57.364+0000 7f720bc00640 1 -- 172.21.15.12:0/1715288657 wait complete. 2024-09-11T00:24:58.020 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:24:57 smithi012 ceph-mon[30333]: pgmap v935: 129 pgs: 129 active+clean; 597 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 414 KiB/s rd, 5.3 MiB/s wr, 307 op/s 2024-09-11T00:24:58.183 INFO:tasks.workunit.client.0.smithi012.stdout: 10 90130 17.64 MB/sec execute 174 sec latency 21901.685 ms 2024-09-11T00:24:58.283 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:24:57 smithi165 ceph-mon[35356]: pgmap v935: 129 pgs: 129 active+clean; 597 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 414 KiB/s rd, 5.3 MiB/s wr, 307 op/s 2024-09-11T00:24:58.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:24:57 smithi178 ceph-mon[35919]: pgmap v935: 129 pgs: 129 active+clean; 597 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 414 KiB/s rd, 5.3 MiB/s wr, 307 op/s 2024-09-11T00:24:59.183 INFO:tasks.workunit.client.0.smithi012.stdout: 10 90130 17.54 MB/sec execute 175 sec latency 22901.748 ms 2024-09-11T00:25:00.001 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:24:59 smithi012 ceph-mon[30333]: pgmap v936: 129 pgs: 129 active+clean; 596 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 167 KiB/s rd, 184 KiB/s wr, 108 op/s 2024-09-11T00:25:00.183 INFO:tasks.workunit.client.0.smithi012.stdout: 10 90130 17.44 MB/sec execute 176 sec latency 23901.804 ms 2024-09-11T00:25:00.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:24:59 smithi165 ceph-mon[35356]: pgmap v936: 129 pgs: 129 active+clean; 596 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 167 KiB/s rd, 184 KiB/s wr, 108 op/s 2024-09-11T00:25:00.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:24:59 smithi178 ceph-mon[35919]: pgmap v936: 129 pgs: 129 active+clean; 596 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 167 KiB/s rd, 184 KiB/s wr, 108 op/s 2024-09-11T00:25:01.183 INFO:tasks.workunit.client.0.smithi012.stdout: 10 90131 17.34 MB/sec execute 177 sec latency 24901.861 ms 2024-09-11T00:25:01.985 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:25:01 smithi012 ceph-mon[30333]: pgmap v937: 129 pgs: 129 active+clean; 589 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 186 KiB/s rd, 184 KiB/s wr, 121 op/s 2024-09-11T00:25:02.183 INFO:tasks.workunit.client.0.smithi012.stdout: 10 90131 17.25 MB/sec execute 178 sec latency 25901.908 ms 2024-09-11T00:25:02.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:25:01 smithi165 ceph-mon[35356]: pgmap v937: 129 pgs: 129 active+clean; 589 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 186 KiB/s rd, 184 KiB/s wr, 121 op/s 2024-09-11T00:25:02.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:25:01 smithi178 ceph-mon[35919]: pgmap v937: 129 pgs: 129 active+clean; 589 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 186 KiB/s rd, 184 KiB/s wr, 121 op/s 2024-09-11T00:25:03.184 INFO:tasks.workunit.client.0.smithi012.stdout: 10 90131 17.15 MB/sec execute 179 sec latency 26901.971 ms 2024-09-11T00:25:03.909 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:25:03 smithi012 ceph-mon[30333]: pgmap v938: 129 pgs: 129 active+clean; 590 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 187 KiB/s rd, 165 KiB/s wr, 98 op/s 2024-09-11T00:25:04.184 INFO:tasks.workunit.client.0.smithi012.stdout: 10 90131 17.05 MB/sec execute 180 sec latency 27902.049 ms 2024-09-11T00:25:04.283 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:25:03 smithi165 ceph-mon[35356]: pgmap v938: 129 pgs: 129 active+clean; 590 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 187 KiB/s rd, 165 KiB/s wr, 98 op/s 2024-09-11T00:25:04.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:25:03 smithi178 ceph-mon[35919]: pgmap v938: 129 pgs: 129 active+clean; 590 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 187 KiB/s rd, 165 KiB/s wr, 98 op/s 2024-09-11T00:25:05.184 INFO:tasks.workunit.client.0.smithi012.stdout: 10 90131 16.96 MB/sec execute 181 sec latency 28902.110 ms 2024-09-11T00:25:05.918 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:25:05 smithi012 ceph-mon[30333]: pgmap v939: 129 pgs: 129 active+clean; 588 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 159 KiB/s rd, 138 KiB/s wr, 84 op/s 2024-09-11T00:25:06.184 INFO:tasks.workunit.client.0.smithi012.stdout: 10 90131 16.87 MB/sec execute 182 sec latency 29902.165 ms 2024-09-11T00:25:06.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:25:05 smithi165 ceph-mon[35356]: pgmap v939: 129 pgs: 129 active+clean; 588 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 159 KiB/s rd, 138 KiB/s wr, 84 op/s 2024-09-11T00:25:06.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:25:05 smithi178 ceph-mon[35919]: pgmap v939: 129 pgs: 129 active+clean; 588 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 159 KiB/s rd, 138 KiB/s wr, 84 op/s 2024-09-11T00:25:07.184 INFO:tasks.workunit.client.0.smithi012.stdout: 10 90158 16.81 MB/sec execute 183 sec latency 30902.225 ms 2024-09-11T00:25:07.879 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:25:07 smithi012 ceph-mon[30333]: pgmap v940: 129 pgs: 129 active+clean; 556 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 215 KiB/s rd, 206 KiB/s wr, 127 op/s 2024-09-11T00:25:08.184 INFO:tasks.workunit.client.0.smithi012.stdout: 10 90194 16.72 MB/sec execute 184 sec latency 31893.834 ms 2024-09-11T00:25:08.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:25:07 smithi165 ceph-mon[35356]: pgmap v940: 129 pgs: 129 active+clean; 556 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 215 KiB/s rd, 206 KiB/s wr, 127 op/s 2024-09-11T00:25:08.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:25:07 smithi178 ceph-mon[35919]: pgmap v940: 129 pgs: 129 active+clean; 556 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 215 KiB/s rd, 206 KiB/s wr, 127 op/s 2024-09-11T00:25:09.184 INFO:tasks.workunit.client.0.smithi012.stdout: 10 90291 16.65 MB/sec execute 185 sec latency 13312.107 ms 2024-09-11T00:25:09.879 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:25:09 smithi012 ceph-mon[30333]: pgmap v941: 129 pgs: 129 active+clean; 552 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 131 KiB/s rd, 142 KiB/s wr, 86 op/s 2024-09-11T00:25:10.184 INFO:tasks.workunit.client.0.smithi012.stdout: 10 90581 16.64 MB/sec execute 186 sec latency 13811.089 ms 2024-09-11T00:25:10.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:25:09 smithi165 ceph-mon[35356]: pgmap v941: 129 pgs: 129 active+clean; 552 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 131 KiB/s rd, 142 KiB/s wr, 86 op/s 2024-09-11T00:25:10.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:25:09 smithi178 ceph-mon[35919]: pgmap v941: 129 pgs: 129 active+clean; 552 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 131 KiB/s rd, 142 KiB/s wr, 86 op/s 2024-09-11T00:25:10.551 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:10.549+0000 7f1b5a400640 1 -- 172.21.15.12:0/3405616756 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1b541066b0 msgr2=0x7f1b5410d030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:25:10.551 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:10.549+0000 7f1b5a400640 1 --2- 172.21.15.12:0/3405616756 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1b541066b0 0x7f1b5410d030 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f1b4c009ca0 tx=0x7f1b4c02a8c0 comp rx=0 tx=0).stop 2024-09-11T00:25:10.551 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:10.550+0000 7f1b5a400640 1 -- 172.21.15.12:0/3405616756 shutdown_connections 2024-09-11T00:25:10.552 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:10.550+0000 7f1b5a400640 1 --2- 172.21.15.12:0/3405616756 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1b541066b0 0x7f1b5410d030 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:10.552 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:10.550+0000 7f1b5a400640 1 --2- 172.21.15.12:0/3405616756 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1b54105cf0 0x7f1b54106170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:10.552 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:10.550+0000 7f1b5a400640 1 --2- 172.21.15.12:0/3405616756 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1b54104ba0 0x7f1b54104fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:10.552 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:10.550+0000 7f1b5a400640 1 -- 172.21.15.12:0/3405616756 >> 172.21.15.12:0/3405616756 conn(0x7f1b54100440 msgr2=0x7f1b54102880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:25:10.552 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:10.551+0000 7f1b5a400640 1 -- 172.21.15.12:0/3405616756 shutdown_connections 2024-09-11T00:25:10.553 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:10.551+0000 7f1b5a400640 1 -- 172.21.15.12:0/3405616756 wait complete. 2024-09-11T00:25:10.553 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:10.552+0000 7f1b5a400640 1 Processor -- start 2024-09-11T00:25:10.555 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:10.553+0000 7f1b5a400640 1 -- start start 2024-09-11T00:25:10.555 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:10.553+0000 7f1b5a400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1b54104ba0 0x7f1b54196b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:25:10.555 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:10.553+0000 7f1b5a400640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1b54105cf0 0x7f1b541971b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:25:10.555 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:10.553+0000 7f1b5a400640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1b541066b0 0x7f1b54197de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:25:10.556 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:10.553+0000 7f1b52a00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1b54105cf0 0x7f1b541971b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:25:10.556 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:10.553+0000 7f1b52a00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1b54105cf0 0x7f1b541971b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.12:40668/0 (socket says 172.21.15.12:40668) 2024-09-11T00:25:10.556 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:10.553+0000 7f1b52a00640 1 -- 172.21.15.12:0/1290677658 learned_addr learned my addr 172.21.15.12:0/1290677658 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:25:10.556 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:10.553+0000 7f1b5a400640 1 -- 172.21.15.12:0/1290677658 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f1b54110870 con 0x7f1b541066b0 2024-09-11T00:25:10.556 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:10.553+0000 7f1b5a400640 1 -- 172.21.15.12:0/1290677658 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f1b541106f0 con 0x7f1b54104ba0 2024-09-11T00:25:10.556 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:10.553+0000 7f1b5a400640 1 -- 172.21.15.12:0/1290677658 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f1b541109f0 con 0x7f1b54105cf0 2024-09-11T00:25:10.557 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:10.554+0000 7f1b52a00640 1 -- 172.21.15.12:0/1290677658 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1b54104ba0 msgr2=0x7f1b54196b60 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-11T00:25:10.557 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:10.554+0000 7f1b52a00640 1 --2- 172.21.15.12:0/1290677658 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1b54104ba0 0x7f1b54196b60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:10.557 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:10.554+0000 7f1b52a00640 1 -- 172.21.15.12:0/1290677658 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1b541066b0 msgr2=0x7f1b54197de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:25:10.557 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:10.554+0000 7f1b53e00640 1 --2- 172.21.15.12:0/1290677658 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1b541066b0 0x7f1b54197de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:25:10.557 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:10.554+0000 7f1b52a00640 1 --2- 172.21.15.12:0/1290677658 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1b541066b0 0x7f1b54197de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:10.557 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:10.554+0000 7f1b52a00640 1 -- 172.21.15.12:0/1290677658 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b541986a0 con 0x7f1b54105cf0 2024-09-11T00:25:10.557 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:10.555+0000 7f1b53e00640 1 --2- 172.21.15.12:0/1290677658 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1b541066b0 0x7f1b54197de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T00:25:10.557 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:10.555+0000 7f1b52a00640 1 --2- 172.21.15.12:0/1290677658 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1b54105cf0 0x7f1b541971b0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f1b4400ea60 tx=0x7f1b4400ef20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:25:10.558 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:10.556+0000 7f1b4be00640 1 -- 172.21.15.12:0/1290677658 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b4400cde0 con 0x7f1b54105cf0 2024-09-11T00:25:10.558 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:10.556+0000 7f1b4be00640 1 -- 172.21.15.12:0/1290677658 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1b440035f0 con 0x7f1b54105cf0 2024-09-11T00:25:10.558 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:10.557+0000 7f1b4be00640 1 -- 172.21.15.12:0/1290677658 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b44010690 con 0x7f1b54105cf0 2024-09-11T00:25:10.559 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:10.557+0000 7f1b5a400640 1 -- 172.21.15.12:0/1290677658 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1b54198990 con 0x7f1b54105cf0 2024-09-11T00:25:10.559 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:10.557+0000 7f1b5a400640 1 -- 172.21.15.12:0/1290677658 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1b5419d0d0 con 0x7f1b54105cf0 2024-09-11T00:25:10.563 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:10.562+0000 7f1b5d420740 1 Processor -- start 2024-09-11T00:25:10.564 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:10.563+0000 7f1b5d420740 1 -- start start 2024-09-11T00:25:10.565 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:10.563+0000 7f1b5d420740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557188146790 0x557188148b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:25:10.565 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:10.563+0000 7f1b5d420740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5571881490c0 0x55718814b4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:25:10.565 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:10.563+0000 7f1b5d420740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55718814bb00 0x55718814dfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:25:10.565 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:10.563+0000 7f1b5d420740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x5571880b96b0 con 0x557188146790 2024-09-11T00:25:10.565 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:10.563+0000 7f1b5d420740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x5571880a8ae0 con 0x55718814bb00 2024-09-11T00:25:10.565 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:10.563+0000 7f1b5d420740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5571880ac8f0 con 0x5571881490c0 2024-09-11T00:25:10.568 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:10.564+0000 7f1b53e00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557188146790 0x557188148b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:25:10.568 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:10.564+0000 7f1b53e00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557188146790 0x557188148b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:34232/0 (socket says 172.21.15.12:34232) 2024-09-11T00:25:10.568 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:10.564+0000 7f1b53e00640 1 -- 172.21.15.12:0/600176914 learned_addr learned my addr 172.21.15.12:0/600176914 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:25:10.568 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:10.564+0000 7f1b53400640 1 --2- 172.21.15.12:0/600176914 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5571881490c0 0x55718814b4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:25:10.568 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:10.565+0000 7f1b53400640 1 -- 172.21.15.12:0/600176914 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55718814bb00 msgr2=0x55718814dfc0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-11T00:25:10.568 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:10.565+0000 7f1b53400640 1 --2- 172.21.15.12:0/600176914 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55718814bb00 0x55718814dfc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:10.568 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:10.565+0000 7f1b53400640 1 -- 172.21.15.12:0/600176914 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557188146790 msgr2=0x557188148b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:25:10.568 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:10.565+0000 7f1b53400640 1 --2- 172.21.15.12:0/600176914 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557188146790 0x557188148b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:10.568 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:10.565+0000 7f1b53400640 1 -- 172.21.15.12:0/600176914 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557188075780 con 0x5571881490c0 2024-09-11T00:25:10.568 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:10.565+0000 7f1b53400640 1 --2- 172.21.15.12:0/600176914 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5571881490c0 0x55718814b4b0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f1b400096e0 tx=0x7f1b40009ba0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8189d6b57ab94162 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:25:10.568 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:10.566+0000 7f1b48c00640 1 -- 172.21.15.12:0/600176914 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b40015020 con 0x5571881490c0 2024-09-11T00:25:10.569 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:10.566+0000 7f1b48c00640 1 -- 172.21.15.12:0/600176914 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1b4000bd70 con 0x5571881490c0 2024-09-11T00:25:10.569 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:10.566+0000 7f1b48c00640 1 -- 172.21.15.12:0/600176914 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b40010380 con 0x5571881490c0 2024-09-11T00:25:10.569 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:10.567+0000 7f1b5d420740 1 -- 172.21.15.12:0/600176914 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5571881490c0 msgr2=0x55718814b4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:25:10.569 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:10.567+0000 7f1b5d420740 1 --2- 172.21.15.12:0/600176914 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5571881490c0 0x55718814b4b0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f1b400096e0 tx=0x7f1b40009ba0 comp rx=0 tx=0).stop 2024-09-11T00:25:10.570 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:10.568+0000 7f1b5d420740 1 -- 172.21.15.12:0/600176914 shutdown_connections 2024-09-11T00:25:10.570 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:10.568+0000 7f1b5d420740 1 --2- 172.21.15.12:0/600176914 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55718814bb00 0x55718814dfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:10.570 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:10.568+0000 7f1b5d420740 1 --2- 172.21.15.12:0/600176914 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5571881490c0 0x55718814b4b0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:10.570 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:10.568+0000 7f1b5d420740 1 --2- 172.21.15.12:0/600176914 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557188146790 0x557188148b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:10.570 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:10.568+0000 7f1b5d420740 1 -- 172.21.15.12:0/600176914 >> 172.21.15.12:0/600176914 conn(0x5571881418e0 msgr2=0x557188142670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:25:10.570 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:10.568+0000 7f1b4be00640 1 -- 172.21.15.12:0/1290677658 <== mon.1 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7f1b44004020 con 0x7f1b54105cf0 2024-09-11T00:25:10.570 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:10.568+0000 7f1b5d420740 1 -- 172.21.15.12:0/600176914 shutdown_connections 2024-09-11T00:25:10.570 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:10.568+0000 7f1b5d420740 1 -- 172.21.15.12:0/600176914 wait complete. 2024-09-11T00:25:10.571 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:10.570+0000 7f1b5d420740 1 -- start start 2024-09-11T00:25:10.571 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:10.570+0000 7f1b5d420740 1 Processor -- start 2024-09-11T00:25:10.572 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:10.571+0000 7f1b4be00640 1 --2- 172.21.15.12:0/1290677658 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f1b380b5270 0x7f1b380b7730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:25:10.572 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:10.571+0000 7f1b5d420740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557188146790 0x557188150020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:25:10.572 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:10.571+0000 7f1b5d420740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5571881490c0 0x5571881506c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:25:10.573 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:10.571+0000 7f1b5d420740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557188151270 0x5571881516f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:25:10.573 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:10.571+0000 7f1b5d420740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x557187fe0ab0 con 0x557188146790 2024-09-11T00:25:10.573 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:10.571+0000 7f1b5d420740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55718804f570 con 0x557188151270 2024-09-11T00:25:10.573 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:10.571+0000 7f1b5d420740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55718814eae0 con 0x5571881490c0 2024-09-11T00:25:10.573 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:10.571+0000 7f1b4be00640 1 -- 172.21.15.12:0/1290677658 <== mon.1 v2:172.21.15.178:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9208+0+0 (secure 0 0 0) 0x7f1b440e6330 con 0x7f1b54105cf0 2024-09-11T00:25:10.573 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:10.571+0000 7f1b5d420740 10 client.0 _ll_register_callbacks cb 0x55718805e100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:25:10.573 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:10.572+0000 7f1b53400640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557188146790 0x557188150020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:25:10.574 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:10.572+0000 7f1b53e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557188151270 0x5571881516f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:25:10.574 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:10.572+0000 7f1b53e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557188151270 0x5571881516f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.12:60310/0 (socket says 172.21.15.12:60310) 2024-09-11T00:25:10.574 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:10.572+0000 7f1b53e00640 1 -- 172.21.15.12:0/2598830569 learned_addr learned my addr 172.21.15.12:0/2598830569 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:25:10.574 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:10.572+0000 7f1b53e00640 1 --2- 172.21.15.12:0/1290677658 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f1b380b5270 0x7f1b380b7730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:25:10.574 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:10.573+0000 7f1b53400640 1 -- 172.21.15.12:0/2598830569 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5571881490c0 msgr2=0x5571881506c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:25:10.574 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:10.573+0000 7f1b53400640 1 --2- 172.21.15.12:0/2598830569 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5571881490c0 0x5571881506c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:10.574 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:10.573+0000 7f1b53400640 1 -- 172.21.15.12:0/2598830569 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557188151270 msgr2=0x5571881516f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:25:10.575 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:10.573+0000 7f1b53400640 1 --2- 172.21.15.12:0/2598830569 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557188151270 0x5571881516f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:10.575 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:10.573+0000 7f1b53400640 1 -- 172.21.15.12:0/2598830569 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557187e0a790 con 0x557188146790 2024-09-11T00:25:10.575 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:10.574+0000 7f1b53400640 1 --2- 172.21.15.12:0/2598830569 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557188146790 0x557188150020 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f1b400096b0 tx=0x7f1b40011650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=753c615c49483abd server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:25:10.575 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:10.574+0000 7f1b3d600640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-09-11T00:25:10.575 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:10.574+0000 7f1b53e00640 1 --2- 172.21.15.12:0/1290677658 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f1b380b5270 0x7f1b380b7730 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f1b4c009d90 tx=0x7f1b4c0095c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:25:10.576 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:10.574+0000 7f1b3d600640 1 -- 172.21.15.12:0/2598830569 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b40006b60 con 0x557188146790 2024-09-11T00:25:10.576 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:10.575+0000 7f1b5d420740 1 -- 172.21.15.12:0/2598830569 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1b4000bd70 con 0x557188146790 2024-09-11T00:25:10.577 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:10.575+0000 7f1b3d600640 1 -- 172.21.15.12:0/2598830569 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9208+0+0 (secure 0 0 0) 0x7f1b4000b8d0 con 0x557188146790 2024-09-11T00:25:10.577 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:10.576+0000 7f1b3d600640 1 -- 172.21.15.12:0/2598830569 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=69) ==== 24+0+0 (secure 0 0 0) 0x7f1b40006b60 con 0x557188146790 2024-09-11T00:25:10.578 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:10.576+0000 7f1b5d420740 10 client.17454 fetch_fsmap learned FSMap version 69 2024-09-11T00:25:10.578 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:10.576+0000 7f1b5d420740 1 -- 172.21.15.12:0/2598830569 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=69,monmap=4+}) -- 0x557187e83620 con 0x557188146790 2024-09-11T00:25:10.578 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:10.577+0000 7f1b3d600640 1 -- 172.21.15.12:0/2598830569 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1b4000bd70 con 0x557188146790 2024-09-11T00:25:10.579 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:10.578+0000 7f1b3d600640 1 -- 172.21.15.12:0/2598830569 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 69) ==== 8390+0+0 (secure 0 0 0) 0x7f1b4001a430 con 0x557188146790 2024-09-11T00:25:10.580 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:10.578+0000 7f1b5d420740 10 client.17454 fetch_fsmap finished waiting for FSMap version 69 2024-09-11T00:25:10.580 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:10.579+0000 7f1b5d420740 10 client.17454 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T00:25:10.580 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:10.579+0000 7f1b5d420740 20 client.17454 populate_metadata read hostname 'smithi012' 2024-09-11T00:25:10.580 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:10.579+0000 7f1b5d420740 1 --2- 172.21.15.12:0/2598830569 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55718815b990 0x55718815dd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:25:10.581 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:10.579+0000 7f1b5d420740 4 client.17454 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T00:25:10.581 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:10.579+0000 7f1b5d420740 1 -- 172.21.15.12:0/2598830569 --> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5571880bb600 con 0x55718815b990 2024-09-11T00:25:10.581 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:10.579+0000 7f1b52a00640 1 --2- 172.21.15.12:0/2598830569 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55718815b990 0x55718815dd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:25:10.582 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:10.581+0000 7f1b52a00640 1 --2- 172.21.15.12:0/2598830569 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55718815b990 0x55718815dd80 crc :-1 s=READY pgs=253 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5b5a3c6a15015a97 server_cookie=3c2c10c7f153f2d5 in_seq=0 out_seq=0 2024-09-11T00:25:10.582 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:10.581+0000 7f1b3d600640 10 client.17454 ms_handle_connect on v2:172.21.15.165:6838/501312984 2024-09-11T00:25:10.583 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:10.582+0000 7f1b3d600640 1 -- 172.21.15.12:0/2598830569 <== mds.0 v2:172.21.15.165:6838/501312984 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5571880bb600 con 0x55718815b990 2024-09-11T00:25:10.583 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:10.582+0000 7f1b3d600640 10 client.17454 handle_command_reply: tid=0 2024-09-11T00:25:10.583 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:10.582+0000 7f1b5d420740 1 client.17454 shutdown 2024-09-11T00:25:10.585 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:10.583+0000 7f1b5d420740 1 -- 172.21.15.12:0/2598830569 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557188146790 msgr2=0x557188150020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:25:10.585 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:10.584+0000 7f1b5d420740 1 --2- 172.21.15.12:0/2598830569 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557188146790 0x557188150020 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f1b400096b0 tx=0x7f1b40011650 comp rx=0 tx=0).stop 2024-09-11T00:25:10.585 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:10.584+0000 7f1b5d420740 1 -- 172.21.15.12:0/2598830569 shutdown_connections 2024-09-11T00:25:10.585 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:10.584+0000 7f1b5d420740 1 --2- 172.21.15.12:0/2598830569 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55718815b990 0x55718815dd80 crc :-1 s=READY pgs=253 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:10.585 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:10.584+0000 7f1b5d420740 1 --2- 172.21.15.12:0/2598830569 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557188151270 0x5571881516f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:10.585 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:10.584+0000 7f1b5d420740 1 --2- 172.21.15.12:0/2598830569 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5571881490c0 0x5571881506c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:10.586 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:10.584+0000 7f1b5d420740 1 --2- 172.21.15.12:0/2598830569 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557188146790 0x557188150020 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:10.586 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:10.584+0000 7f1b3d600640 0 client.17454 ms_handle_reset on v2:172.21.15.165:6838/501312984 2024-09-11T00:25:10.586 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:10.584+0000 7f1b5d420740 1 -- 172.21.15.12:0/2598830569 >> 172.21.15.12:0/2598830569 conn(0x5571881418e0 msgr2=0x557188140a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:25:10.586 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:10.584+0000 7f1b5d420740 1 -- 172.21.15.12:0/2598830569 shutdown_connections 2024-09-11T00:25:10.586 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:10.585+0000 7f1b5d420740 1 -- 172.21.15.12:0/2598830569 wait complete. 2024-09-11T00:25:10.587 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:10.585+0000 7f1b5d420740 20 client.17454 trim_cache size 0 max 16384 2024-09-11T00:25:10.624 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:10.623+0000 7f1b5d420740 1 Processor -- start 2024-09-11T00:25:10.624 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:10.623+0000 7f1b5d420740 1 -- start start 2024-09-11T00:25:10.625 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:10.623+0000 7f1b5d420740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557188146790 0x557188142140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:25:10.625 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:10.623+0000 7f1b5d420740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5571881490c0 0x557188142680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:25:10.625 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:10.624+0000 7f1b5d420740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557188151270 0x5571881437b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:25:10.625 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:10.624+0000 7f1b5d420740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f1b541c2ec0 con 0x5571881490c0 2024-09-11T00:25:10.625 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:10.624+0000 7f1b5d420740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f1b541c4a60 con 0x557188151270 2024-09-11T00:25:10.625 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:10.624+0000 7f1b5d420740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f1b541c3590 con 0x557188146790 2024-09-11T00:25:10.626 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:10.624+0000 7f1b53e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557188151270 0x5571881437b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:25:10.626 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:10.624+0000 7f1b52a00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5571881490c0 0x557188142680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:25:10.626 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:10.624+0000 7f1b53400640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557188146790 0x557188142140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:25:10.626 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:10.624+0000 7f1b53400640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557188146790 0x557188142140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.12:40700/0 (socket says 172.21.15.12:40700) 2024-09-11T00:25:10.626 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:10.624+0000 7f1b53400640 1 -- 172.21.15.12:0/183538338 learned_addr learned my addr 172.21.15.12:0/183538338 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:25:10.627 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:10.625+0000 7f1b53400640 1 -- 172.21.15.12:0/183538338 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557188151270 msgr2=0x5571881437b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:25:10.627 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:10.625+0000 7f1b53400640 1 --2- 172.21.15.12:0/183538338 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557188151270 0x5571881437b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:10.627 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:10.625+0000 7f1b53400640 1 -- 172.21.15.12:0/183538338 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5571881490c0 msgr2=0x557188142680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:25:10.627 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:10.625+0000 7f1b53400640 1 --2- 172.21.15.12:0/183538338 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5571881490c0 0x557188142680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:10.627 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:10.625+0000 7f1b53400640 1 -- 172.21.15.12:0/183538338 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557188148ef0 con 0x557188146790 2024-09-11T00:25:10.627 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:10.625+0000 7f1b52a00640 1 --2- 172.21.15.12:0/183538338 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5571881490c0 0x557188142680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T00:25:10.627 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:10.625+0000 7f1b53400640 1 --2- 172.21.15.12:0/183538338 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557188146790 0x557188142140 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f1b40009be0 tx=0x7f1b4001afa0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=47fc74b8feb5b8ea server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:25:10.628 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:10.626+0000 7f1b3fe00640 1 -- 172.21.15.12:0/183538338 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b40015040 con 0x557188146790 2024-09-11T00:25:10.628 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:10.626+0000 7f1b3fe00640 1 -- 172.21.15.12:0/183538338 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1b4001f070 con 0x557188146790 2024-09-11T00:25:10.628 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:10.627+0000 7f1b3fe00640 1 -- 172.21.15.12:0/183538338 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b40024470 con 0x557188146790 2024-09-11T00:25:10.628 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:10.627+0000 7f1b5d420740 1 -- 172.21.15.12:0/183538338 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557188146790 msgr2=0x557188142140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:25:10.628 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:10.627+0000 7f1b5d420740 1 --2- 172.21.15.12:0/183538338 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557188146790 0x557188142140 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f1b40009be0 tx=0x7f1b4001afa0 comp rx=0 tx=0).stop 2024-09-11T00:25:10.629 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:10.627+0000 7f1b5d420740 1 -- 172.21.15.12:0/183538338 shutdown_connections 2024-09-11T00:25:10.629 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:10.627+0000 7f1b5d420740 1 --2- 172.21.15.12:0/183538338 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557188151270 0x5571881437b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:10.629 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:10.627+0000 7f1b5d420740 1 --2- 172.21.15.12:0/183538338 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5571881490c0 0x557188142680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:10.629 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:10.627+0000 7f1b5d420740 1 --2- 172.21.15.12:0/183538338 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557188146790 0x557188142140 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:10.629 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:10.627+0000 7f1b5d420740 1 -- 172.21.15.12:0/183538338 >> 172.21.15.12:0/183538338 conn(0x5571881418e0 msgr2=0x55718814bf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:25:10.629 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:10.627+0000 7f1b5d420740 1 -- 172.21.15.12:0/183538338 shutdown_connections 2024-09-11T00:25:10.629 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:10.628+0000 7f1b5d420740 1 -- 172.21.15.12:0/183538338 wait complete. 2024-09-11T00:25:10.630 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:10.628+0000 7f1b5d420740 1 -- start start 2024-09-11T00:25:10.630 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:10.629+0000 7f1b5d420740 1 Processor -- start 2024-09-11T00:25:10.632 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:10.630+0000 7f1b5d420740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557188146790 0x557188157da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:25:10.632 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:10.630+0000 7f1b5d420740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5571881490c0 0x557188156aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:25:10.632 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:10.630+0000 7f1b5d420740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557188151270 0x557188156fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:25:10.632 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:10.630+0000 7f1b5d420740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f1b541c4390 con 0x557188151270 2024-09-11T00:25:10.632 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:10.630+0000 7f1b5d420740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f1b541c3cc0 con 0x5571881490c0 2024-09-11T00:25:10.632 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:10.630+0000 7f1b5d420740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f1b54073a90 con 0x557188146790 2024-09-11T00:25:10.632 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:10.630+0000 7f1b5d420740 10 client.0 _ll_register_callbacks cb 0x7f1b40009530 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:25:10.632 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:10.631+0000 7f1b53e00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557188151270 0x557188156fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:25:10.632 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:10.631+0000 7f1b53400640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557188146790 0x557188157da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:25:10.633 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:10.631+0000 7f1b53400640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557188146790 0x557188157da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.12:40716/0 (socket says 172.21.15.12:40716) 2024-09-11T00:25:10.634 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:10.632+0000 7f1b53400640 1 -- 172.21.15.12:0/1091824208 learned_addr learned my addr 172.21.15.12:0/1091824208 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:25:10.634 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:10.631+0000 7f1b53e00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557188151270 0x557188156fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:34266/0 (socket says 172.21.15.12:34266) 2024-09-11T00:25:10.634 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:10.631+0000 7f1b52a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5571881490c0 0x557188156aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:25:10.634 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:10.633+0000 7f1b53400640 1 -- 172.21.15.12:0/1091824208 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5571881490c0 msgr2=0x557188156aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:25:10.634 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:10.633+0000 7f1b53400640 1 --2- 172.21.15.12:0/1091824208 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5571881490c0 0x557188156aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:10.634 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:10.633+0000 7f1b53400640 1 -- 172.21.15.12:0/1091824208 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557188151270 msgr2=0x557188156fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:25:10.634 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:10.633+0000 7f1b53400640 1 --2- 172.21.15.12:0/1091824208 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557188151270 0x557188156fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:10.634 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:10.633+0000 7f1b53400640 1 -- 172.21.15.12:0/1091824208 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557188157720 con 0x557188146790 2024-09-11T00:25:10.634 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:10.633+0000 7f1b53e00640 1 --2- 172.21.15.12:0/1091824208 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557188151270 0x557188156fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T00:25:10.635 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:10.633+0000 7f1b53400640 1 --2- 172.21.15.12:0/1091824208 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557188146790 0x557188157da0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f1b40004700 tx=0x7f1b4001a660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f8c78479827a48f1 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:25:10.635 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:10.633+0000 7f1b3d600640 10 client.0 ms_handle_connect on v2:172.21.15.178:3300/0 2024-09-11T00:25:10.636 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:10.635+0000 7f1b3d600640 1 -- 172.21.15.12:0/1091824208 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b4001aba0 con 0x557188146790 2024-09-11T00:25:10.636 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:10.635+0000 7f1b3d600640 1 -- 172.21.15.12:0/1091824208 <== mon.1 v2:172.21.15.178:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9208+0+0 (secure 0 0 0) 0x7f1b40024470 con 0x557188146790 2024-09-11T00:25:10.636 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:10.635+0000 7f1b5d420740 1 -- 172.21.15.12:0/1091824208 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1b4001f070 con 0x557188146790 2024-09-11T00:25:10.636 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:10.635+0000 7f1b3d600640 1 -- 172.21.15.12:0/1091824208 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=69) ==== 24+0+0 (secure 0 0 0) 0x7f1b4003f050 con 0x557188146790 2024-09-11T00:25:10.637 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:10.635+0000 7f1b5d420740 10 client.27436 fetch_fsmap learned FSMap version 69 2024-09-11T00:25:10.637 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:10.635+0000 7f1b5d420740 1 -- 172.21.15.12:0/1091824208 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,fsmap=69,monmap=4+}) -- 0x5571881c8890 con 0x557188146790 2024-09-11T00:25:10.637 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:10.636+0000 7f1b3d600640 1 -- 172.21.15.12:0/1091824208 <== mon.1 v2:172.21.15.178:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1b4001f070 con 0x557188146790 2024-09-11T00:25:10.640 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:10.638+0000 7f1b3d600640 1 -- 172.21.15.12:0/1091824208 <== mon.1 v2:172.21.15.178:3300/0 5 ==== fsmap(e 69) ==== 8390+0+0 (secure 0 0 0) 0x7f1b40024670 con 0x557188146790 2024-09-11T00:25:10.640 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:10.638+0000 7f1b5d420740 10 client.27436 fetch_fsmap finished waiting for FSMap version 69 2024-09-11T00:25:10.640 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:10.638+0000 7f1b5d420740 10 client.27436 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T00:25:10.640 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:10.638+0000 7f1b5d420740 20 client.27436 populate_metadata read hostname 'smithi012' 2024-09-11T00:25:10.640 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:10.638+0000 7f1b5d420740 1 --2- 172.21.15.12:0/1091824208 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x5571881c9060 0x5571881cb480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:25:10.640 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:10.638+0000 7f1b5d420740 4 client.27436 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T00:25:10.640 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:10.638+0000 7f1b5d420740 1 -- 172.21.15.12:0/1091824208 --> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5571880b70b0 con 0x5571881c9060 2024-09-11T00:25:10.640 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:10.638+0000 7f1b52a00640 1 --2- 172.21.15.12:0/1091824208 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x5571881c9060 0x5571881cb480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:25:10.641 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:10.639+0000 7f1b52a00640 1 --2- 172.21.15.12:0/1091824208 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x5571881c9060 0x5571881cb480 crc :-1 s=READY pgs=254 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8dba5b4d549969d3 server_cookie=29c7ab384055d457 in_seq=0 out_seq=0 2024-09-11T00:25:10.641 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:10.639+0000 7f1b3d600640 10 client.27436 ms_handle_connect on v2:172.21.15.165:6838/501312984 2024-09-11T00:25:10.780 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:10.779+0000 7f1b3d600640 1 -- 172.21.15.12:0/1091824208 <== mds.0 v2:172.21.15.165:6838/501312984 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5571880b70b0 con 0x5571881c9060 2024-09-11T00:25:10.781 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:10.779+0000 7f1b3d600640 10 client.27436 handle_command_reply: tid=0 2024-09-11T00:25:10.782 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:10.779+0000 7f1b5d420740 1 client.27436 shutdown 2024-09-11T00:25:10.782 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:10.780+0000 7f1b5d420740 1 -- 172.21.15.12:0/1091824208 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557188146790 msgr2=0x557188157da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:25:10.782 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:10.780+0000 7f1b5d420740 1 --2- 172.21.15.12:0/1091824208 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557188146790 0x557188157da0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f1b40004700 tx=0x7f1b4001a660 comp rx=0 tx=0).stop 2024-09-11T00:25:10.782 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:10.780+0000 7f1b5d420740 1 -- 172.21.15.12:0/1091824208 shutdown_connections 2024-09-11T00:25:10.782 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:10.780+0000 7f1b5d420740 1 --2- 172.21.15.12:0/1091824208 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x5571881c9060 0x5571881cb480 crc :-1 s=READY pgs=254 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:10.782 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:10.780+0000 7f1b5d420740 1 --2- 172.21.15.12:0/1091824208 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557188151270 0x557188156fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:10.782 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:10.780+0000 7f1b5d420740 1 --2- 172.21.15.12:0/1091824208 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5571881490c0 0x557188156aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:10.782 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:10.780+0000 7f1b3d600640 0 client.27436 ms_handle_reset on v2:172.21.15.165:6838/501312984 2024-09-11T00:25:10.782 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:10.780+0000 7f1b5d420740 1 --2- 172.21.15.12:0/1091824208 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557188146790 0x557188157da0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:10.782 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:10.780+0000 7f1b5d420740 1 -- 172.21.15.12:0/1091824208 >> 172.21.15.12:0/1091824208 conn(0x5571881418e0 msgr2=0x557188142140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:25:10.783 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:10.780+0000 7f1b5d420740 1 -- 172.21.15.12:0/1091824208 shutdown_connections 2024-09-11T00:25:10.783 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:10.780+0000 7f1b5d420740 1 -- 172.21.15.12:0/1091824208 wait complete. 2024-09-11T00:25:10.783 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:10.781+0000 7f1b5d420740 20 client.27436 trim_cache size 0 max 16384 2024-09-11T00:25:10.783 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:{ 2024-09-11T00:25:10.784 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "message": "", 2024-09-11T00:25:10.784 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "return_code": 0 2024-09-11T00:25:10.784 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:} 2024-09-11T00:25:10.785 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:10.783+0000 7f1b5a400640 1 -- 172.21.15.12:0/1290677658 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f1b380b5270 msgr2=0x7f1b380b7730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:25:10.785 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:10.783+0000 7f1b5a400640 1 --2- 172.21.15.12:0/1290677658 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f1b380b5270 0x7f1b380b7730 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f1b4c009d90 tx=0x7f1b4c0095c0 comp rx=0 tx=0).stop 2024-09-11T00:25:10.785 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:10.783+0000 7f1b5a400640 1 -- 172.21.15.12:0/1290677658 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1b54105cf0 msgr2=0x7f1b541971b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:25:10.785 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:10.783+0000 7f1b5a400640 1 --2- 172.21.15.12:0/1290677658 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1b54105cf0 0x7f1b541971b0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f1b4400ea60 tx=0x7f1b4400ef20 comp rx=0 tx=0).stop 2024-09-11T00:25:10.785 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:10.783+0000 7f1b5a400640 1 -- 172.21.15.12:0/1290677658 shutdown_connections 2024-09-11T00:25:10.785 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:10.783+0000 7f1b5a400640 1 --2- 172.21.15.12:0/1290677658 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f1b380b5270 0x7f1b380b7730 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:10.785 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:10.783+0000 7f1b5a400640 1 --2- 172.21.15.12:0/1290677658 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1b541066b0 0x7f1b54197de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:10.785 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:10.783+0000 7f1b5a400640 1 --2- 172.21.15.12:0/1290677658 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1b54105cf0 0x7f1b541971b0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:10.785 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:10.783+0000 7f1b5a400640 1 --2- 172.21.15.12:0/1290677658 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1b54104ba0 0x7f1b54196b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:10.785 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:10.783+0000 7f1b5a400640 1 -- 172.21.15.12:0/1290677658 >> 172.21.15.12:0/1290677658 conn(0x7f1b54100440 msgr2=0x7f1b540795a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:25:10.785 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:10.784+0000 7f1b5a400640 1 -- 172.21.15.12:0/1290677658 shutdown_connections 2024-09-11T00:25:10.786 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:10.784+0000 7f1b5a400640 1 -- 172.21.15.12:0/1290677658 wait complete. 2024-09-11T00:25:11.184 INFO:tasks.workunit.client.0.smithi012.stdout: 10 91466 16.80 MB/sec execute 187 sec latency 9976.661 ms 2024-09-11T00:25:12.184 INFO:tasks.workunit.client.0.smithi012.stdout: 10 92239 17.00 MB/sec execute 188 sec latency 66.198 ms 2024-09-11T00:25:12.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:25:11 smithi165 ceph-mon[35356]: pgmap v942: 129 pgs: 129 active+clean; 554 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 643 KiB/s rd, 369 KiB/s wr, 103 op/s 2024-09-11T00:25:12.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:25:11 smithi012 ceph-mon[30333]: pgmap v942: 129 pgs: 129 active+clean; 554 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 643 KiB/s rd, 369 KiB/s wr, 103 op/s 2024-09-11T00:25:12.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:25:11 smithi178 ceph-mon[35919]: pgmap v942: 129 pgs: 129 active+clean; 554 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 643 KiB/s rd, 369 KiB/s wr, 103 op/s 2024-09-11T00:25:13.184 INFO:tasks.workunit.client.0.smithi012.stdout: 10 92900 17.07 MB/sec execute 189 sec latency 59.115 ms 2024-09-11T00:25:14.149 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:25:13 smithi012 ceph-mon[30333]: pgmap v943: 129 pgs: 129 active+clean; 558 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 772 KiB/s wr, 151 op/s 2024-09-11T00:25:14.184 INFO:tasks.workunit.client.0.smithi012.stdout: 10 93620 17.16 MB/sec execute 190 sec latency 70.547 ms 2024-09-11T00:25:14.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:25:13 smithi165 ceph-mon[35356]: pgmap v943: 129 pgs: 129 active+clean; 558 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 772 KiB/s wr, 151 op/s 2024-09-11T00:25:14.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:25:13 smithi178 ceph-mon[35919]: pgmap v943: 129 pgs: 129 active+clean; 558 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 772 KiB/s wr, 151 op/s 2024-09-11T00:25:15.185 INFO:tasks.workunit.client.0.smithi012.stdout: 10 94583 17.34 MB/sec execute 191 sec latency 48.446 ms 2024-09-11T00:25:15.398 INFO:tasks.quiescer.fs.[cephfs]:Total duration (42.0) is longer than `split_if_longer` (35.0), will split into two consecutive quiesces 2024-09-11T00:25:15.398 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 21.0 2024-09-11T00:25:15.399 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid ff3ad74a-6fcf-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T00:25:16.055 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:25:16.188 INFO:tasks.workunit.client.0.smithi012.stdout: 10 95461 17.50 MB/sec execute 192 sec latency 69.069 ms 2024-09-11T00:25:16.253 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:25:15 smithi012 ceph-mon[30333]: pgmap v944: 129 pgs: 129 active+clean; 558 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 742 KiB/s wr, 154 op/s 2024-09-11T00:25:16.286 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:25:15 smithi165 ceph-mon[35356]: pgmap v944: 129 pgs: 129 active+clean; 558 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 742 KiB/s wr, 154 op/s 2024-09-11T00:25:16.337 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:25:15 smithi178 ceph-mon[35919]: pgmap v944: 129 pgs: 129 active+clean; 558 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 742 KiB/s wr, 154 op/s 2024-09-11T00:25:17.187 INFO:tasks.workunit.client.0.smithi012.stdout: 10 96397 17.70 MB/sec execute 193 sec latency 63.807 ms 2024-09-11T00:25:17.306 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:17.304+0000 7f0f6c600640 1 -- 172.21.15.12:0/2325384489 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0f64117e00 msgr2=0x7f0f6411a1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:25:17.306 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:17.304+0000 7f0f6c600640 1 --2- 172.21.15.12:0/2325384489 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0f64117e00 0x7f0f6411a1f0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f0f54009930 tx=0x7f0f5402a4d0 comp rx=0 tx=0).stop 2024-09-11T00:25:17.306 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:17.304+0000 7f0f6c600640 1 -- 172.21.15.12:0/2325384489 shutdown_connections 2024-09-11T00:25:17.307 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:17.304+0000 7f0f6c600640 1 --2- 172.21.15.12:0/2325384489 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0f64117e00 0x7f0f6411a1f0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:17.307 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:17.304+0000 7f0f6c600640 1 --2- 172.21.15.12:0/2325384489 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0f6410aa70 0x7f0f641178c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:17.307 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:17.304+0000 7f0f6c600640 1 --2- 172.21.15.12:0/2325384489 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0f6410a150 0x7f0f6410a530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:17.307 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:17.304+0000 7f0f6c600640 1 -- 172.21.15.12:0/2325384489 >> 172.21.15.12:0/2325384489 conn(0x7f0f6406e2e0 msgr2=0x7f0f6406e6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:25:17.307 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:17.304+0000 7f0f6c600640 1 -- 172.21.15.12:0/2325384489 shutdown_connections 2024-09-11T00:25:17.307 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:17.304+0000 7f0f6c600640 1 -- 172.21.15.12:0/2325384489 wait complete. 2024-09-11T00:25:17.309 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:17.306+0000 7f0f6c600640 1 Processor -- start 2024-09-11T00:25:17.309 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:17.306+0000 7f0f6c600640 1 -- start start 2024-09-11T00:25:17.309 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:17.306+0000 7f0f6c600640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0f6410a150 0x7f0f64115620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:25:17.309 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:17.306+0000 7f0f6c600640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0f6410aa70 0x7f0f64115b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:25:17.309 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:17.306+0000 7f0f6c600640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0f64117e00 0x7f0f641ac880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:25:17.309 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:17.306+0000 7f0f6c600640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f0f6410d940 con 0x7f0f6410a150 2024-09-11T00:25:17.309 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:17.306+0000 7f0f6c600640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f0f6410d7c0 con 0x7f0f64117e00 2024-09-11T00:25:17.309 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:17.306+0000 7f0f6c600640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f0f6410dac0 con 0x7f0f6410aa70 2024-09-11T00:25:17.309 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:17.307+0000 7f0f69200640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0f6410aa70 0x7f0f64115b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:25:17.309 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:17.307+0000 7f0f69200640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0f6410aa70 0x7f0f64115b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.12:40742/0 (socket says 172.21.15.12:40742) 2024-09-11T00:25:17.310 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:17.307+0000 7f0f69200640 1 -- 172.21.15.12:0/1560188019 learned_addr learned my addr 172.21.15.12:0/1560188019 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:25:17.310 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:17.307+0000 7f0f69c00640 1 --2- 172.21.15.12:0/1560188019 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0f6410a150 0x7f0f64115620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:25:17.310 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:17.307+0000 7f0f6a600640 1 --2- 172.21.15.12:0/1560188019 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0f64117e00 0x7f0f641ac880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:25:17.310 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:17.307+0000 7f0f69200640 1 -- 172.21.15.12:0/1560188019 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0f64117e00 msgr2=0x7f0f641ac880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:25:17.310 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:17.308+0000 7f0f69200640 1 --2- 172.21.15.12:0/1560188019 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0f64117e00 0x7f0f641ac880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:17.310 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:17.308+0000 7f0f69200640 1 -- 172.21.15.12:0/1560188019 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0f6410a150 msgr2=0x7f0f64115620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:25:17.310 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:17.308+0000 7f0f69200640 1 --2- 172.21.15.12:0/1560188019 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0f6410a150 0x7f0f64115620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:17.311 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:17.308+0000 7f0f69200640 1 -- 172.21.15.12:0/1560188019 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f641ad000 con 0x7f0f6410aa70 2024-09-11T00:25:17.311 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:17.308+0000 7f0f6a600640 1 --2- 172.21.15.12:0/1560188019 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0f64117e00 0x7f0f641ac880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T00:25:17.311 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:17.310+0000 7f0f69200640 1 --2- 172.21.15.12:0/1560188019 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0f6410aa70 0x7f0f64115b60 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f0f5000ea60 tx=0x7f0f5000ef20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:25:17.312 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:17.310+0000 7f0f62000640 1 -- 172.21.15.12:0/1560188019 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f5000cde0 con 0x7f0f6410aa70 2024-09-11T00:25:17.312 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:17.310+0000 7f0f62000640 1 -- 172.21.15.12:0/1560188019 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0f500035f0 con 0x7f0f6410aa70 2024-09-11T00:25:17.312 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:17.310+0000 7f0f62000640 1 -- 172.21.15.12:0/1560188019 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f50010690 con 0x7f0f6410aa70 2024-09-11T00:25:17.312 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:17.310+0000 7f0f6c600640 1 -- 172.21.15.12:0/1560188019 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0f641ad2f0 con 0x7f0f6410aa70 2024-09-11T00:25:17.312 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:17.310+0000 7f0f6c600640 1 -- 172.21.15.12:0/1560188019 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0f641b4b90 con 0x7f0f6410aa70 2024-09-11T00:25:17.314 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:17.312+0000 7f0f6c600640 1 -- 172.21.15.12:0/1560188019 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0f6410a530 con 0x7f0f6410aa70 2024-09-11T00:25:17.321 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:17.317+0000 7f0f62000640 1 -- 172.21.15.12:0/1560188019 <== mon.1 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7f0f50004000 con 0x7f0f6410aa70 2024-09-11T00:25:17.321 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:17.319+0000 7f0f62000640 1 --2- 172.21.15.12:0/1560188019 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f0f480b5430 0x7f0f480b78f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:25:17.322 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:17.320+0000 7f0f69c00640 1 --2- 172.21.15.12:0/1560188019 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f0f480b5430 0x7f0f480b78f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:25:17.322 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:17.321+0000 7f0f62000640 1 -- 172.21.15.12:0/1560188019 <== mon.1 v2:172.21.15.178:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9208+0+0 (secure 0 0 0) 0x7f0f500e6390 con 0x7f0f6410aa70 2024-09-11T00:25:17.323 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:17.321+0000 7f0f62000640 1 -- 172.21.15.12:0/1560188019 <== mon.1 v2:172.21.15.178:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f0f500acfa0 con 0x7f0f6410aa70 2024-09-11T00:25:17.326 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:17.325+0000 7f0f69c00640 1 --2- 172.21.15.12:0/1560188019 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f0f480b5430 0x7f0f480b78f0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f0f5c009f20 tx=0x7f0f5c0094f0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:25:17.784 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:17.783+0000 7f0f6c600640 1 -- 172.21.15.12:0/1560188019 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0f64116750 con 0x7f0f6410aa70 2024-09-11T00:25:17.785 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:17.784+0000 7f0f62000640 1 -- 172.21.15.12:0/1560188019 <== mon.1 v2:172.21.15.178:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 69 v69) ==== 94+0+11972 (secure 0 0 0) 0x7f0f64116750 con 0x7f0f6410aa70 2024-09-11T00:25:17.786 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-11T00:25:17.786 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":69,"btime":"2024-09-11T00:24:37:715541+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14577,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6835/447171591","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":447171591},{"type":"v1","addr":"172.21.15.12:6835","nonce":447171591}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14589,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/901486040","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":901486040},{"type":"v1","addr":"172.21.15.12:6837","nonce":901486040}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":69,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T00:04:04.739881+0000","modified":"2024-09-11T00:24:36.738582+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24674,"mds_1":24643,"mds_2":24638,"mds_3":24637,"mds_4":24614},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14631":{"gid":14631,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.12:6841/3069919437","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6840","nonce":3069919437},{"type":"v1","addr":"172.21.15.12:6841","nonce":3069919437}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24562":{"gid":24562,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6833/1141636112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":1141636112},{"type":"v1","addr":"172.21.15.178:6833","nonce":1141636112}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24581":{"gid":24581,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6833/3111980970","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6832","nonce":3111980970},{"type":"v1","addr":"172.21.15.165:6833","nonce":3111980970}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24607":{"gid":24607,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6835/4084868287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":4084868287},{"type":"v1","addr":"172.21.15.178:6835","nonce":4084868287}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24614":{"gid":24614,"name":"e","rank":4,"incarnation":26,"state":"up:active","state_seq":21,"addr":"172.21.15.165:6835/2425395274","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":2425395274},{"type":"v1","addr":"172.21.15.165:6835","nonce":2425395274}]},"join_fscid":-1,"export_targets":[1],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24637":{"gid":24637,"name":"i","rank":3,"incarnation":24,"state":"up:active","state_seq":18,"addr":"172.21.15.178:6839/70628097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6838","nonce":70628097},{"type":"v1","addr":"172.21.15.178:6839","nonce":70628097}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24638":{"gid":24638,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":18,"addr":"172.21.15.165:6837/6215088","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6836","nonce":6215088},{"type":"v1","addr":"172.21.15.165:6837","nonce":6215088}]},"join_fscid":-1,"export_targets":[0,3],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24643":{"gid":24643,"name":"g","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.12:6839/3124093009","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6838","nonce":3124093009},{"type":"v1","addr":"172.21.15.12:6839","nonce":3124093009}]},"join_fscid":-1,"export_targets":[0,3],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24674":{"gid":24674,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.165:6839/501312984","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6838","nonce":501312984},{"type":"v1","addr":"172.21.15.165:6839","nonce":501312984}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25672":{"gid":25672,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6837/308601741","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":308601741},{"type":"v1","addr":"172.21.15.178:6837","nonce":308601741}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24674,"qdb_cluster":[24614,24637,24638,24643,24674]},"id":1}]} 2024-09-11T00:25:17.786 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 69 2024-09-11T00:25:17.789 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:17.787+0000 7f0f6c600640 1 -- 172.21.15.12:0/1560188019 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f0f480b5430 msgr2=0x7f0f480b78f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:25:17.789 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:17.787+0000 7f0f6c600640 1 --2- 172.21.15.12:0/1560188019 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f0f480b5430 0x7f0f480b78f0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f0f5c009f20 tx=0x7f0f5c0094f0 comp rx=0 tx=0).stop 2024-09-11T00:25:17.789 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:17.787+0000 7f0f6c600640 1 -- 172.21.15.12:0/1560188019 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0f6410aa70 msgr2=0x7f0f64115b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:25:17.789 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:17.787+0000 7f0f6c600640 1 --2- 172.21.15.12:0/1560188019 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0f6410aa70 0x7f0f64115b60 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f0f5000ea60 tx=0x7f0f5000ef20 comp rx=0 tx=0).stop 2024-09-11T00:25:17.791 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:17.789+0000 7f0f6c600640 1 -- 172.21.15.12:0/1560188019 shutdown_connections 2024-09-11T00:25:17.791 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:17.790+0000 7f0f6c600640 1 --2- 172.21.15.12:0/1560188019 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f0f480b5430 0x7f0f480b78f0 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:17.791 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:17.790+0000 7f0f6c600640 1 --2- 172.21.15.12:0/1560188019 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0f64117e00 0x7f0f641ac880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:17.791 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:17.790+0000 7f0f6c600640 1 --2- 172.21.15.12:0/1560188019 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0f6410aa70 0x7f0f64115b60 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:17.791 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:17.790+0000 7f0f6c600640 1 --2- 172.21.15.12:0/1560188019 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0f6410a150 0x7f0f64115620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:17.791 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:17.790+0000 7f0f6c600640 1 -- 172.21.15.12:0/1560188019 >> 172.21.15.12:0/1560188019 conn(0x7f0f6406e2e0 msgr2=0x7f0f641181e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:25:17.792 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:17.790+0000 7f0f6c600640 1 -- 172.21.15.12:0/1560188019 shutdown_connections 2024-09-11T00:25:17.792 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:17.790+0000 7f0f6c600640 1 -- 172.21.15.12:0/1560188019 wait complete. 2024-09-11T00:25:18.064 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:25:17 smithi012 ceph-mon[30333]: pgmap v945: 129 pgs: 129 active+clean; 562 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 2.2 MiB/s wr, 392 op/s 2024-09-11T00:25:18.064 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:25:17 smithi012 ceph-mon[30333]: from='client.? 172.21.15.12:0/1560188019' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T00:25:18.188 INFO:tasks.workunit.client.0.smithi012.stdout: 10 97098 17.80 MB/sec execute 194 sec latency 96.128 ms 2024-09-11T00:25:18.285 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:25:17 smithi165 ceph-mon[35356]: pgmap v945: 129 pgs: 129 active+clean; 562 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 2.2 MiB/s wr, 392 op/s 2024-09-11T00:25:18.286 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:25:17 smithi165 ceph-mon[35356]: from='client.? 172.21.15.12:0/1560188019' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T00:25:18.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:25:17 smithi178 ceph-mon[35919]: pgmap v945: 129 pgs: 129 active+clean; 562 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 2.2 MiB/s wr, 392 op/s 2024-09-11T00:25:18.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:25:17 smithi178 ceph-mon[35919]: from='client.? 172.21.15.12:0/1560188019' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T00:25:18.732 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24674 quiesce db / --timeout 90 --expiration 141.0 --await-for 45.0' 2024-09-11T00:25:18.733 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid ff3ad74a-6fcf-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24674 quiesce db / --timeout 90 --expiration 141.0 --await-for 45.0 2024-09-11T00:25:19.187 INFO:tasks.workunit.client.0.smithi012.stdout: 10 97860 17.94 MB/sec execute 195 sec latency 65.966 ms 2024-09-11T00:25:19.415 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:25:20.063 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:25:19 smithi012 ceph-mon[30333]: pgmap v946: 129 pgs: 129 active+clean; 565 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 2.4 MiB/s wr, 381 op/s 2024-09-11T00:25:20.169 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:20.166+0000 7f4cdc200640 1 -- 172.21.15.12:0/995183293 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4cd4106ef0 msgr2=0x7f4cd4107340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:25:20.169 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:20.166+0000 7f4cdc200640 1 --2- 172.21.15.12:0/995183293 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4cd4106ef0 0x7f4cd4107340 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f4cc8009900 tx=0x7f4cc802a3f0 comp rx=0 tx=0).stop 2024-09-11T00:25:20.169 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:20.167+0000 7f4cdc200640 1 -- 172.21.15.12:0/995183293 shutdown_connections 2024-09-11T00:25:20.170 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:20.167+0000 7f4cdc200640 1 --2- 172.21.15.12:0/995183293 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4cd4106ef0 0x7f4cd4107340 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:20.170 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:20.167+0000 7f4cdc200640 1 --2- 172.21.15.12:0/995183293 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4cd41023d0 0x7f4cd41068a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:20.170 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:20.167+0000 7f4cdc200640 1 --2- 172.21.15.12:0/995183293 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4cd4101ab0 0x7f4cd4101e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:20.170 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:20.167+0000 7f4cdc200640 1 -- 172.21.15.12:0/995183293 >> 172.21.15.12:0/995183293 conn(0x7f4cd40fd700 msgr2=0x7f4cd40ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:25:20.170 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:20.168+0000 7f4cdc200640 1 -- 172.21.15.12:0/995183293 shutdown_connections 2024-09-11T00:25:20.170 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:20.168+0000 7f4cdc200640 1 -- 172.21.15.12:0/995183293 wait complete. 2024-09-11T00:25:20.171 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:20.169+0000 7f4cdc200640 1 Processor -- start 2024-09-11T00:25:20.171 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:20.169+0000 7f4cdc200640 1 -- start start 2024-09-11T00:25:20.172 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:20.170+0000 7f4cdc200640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4cd4101ab0 0x7f4cd419f850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:25:20.172 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:20.170+0000 7f4cdc200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4cd41023d0 0x7f4cd419fd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:25:20.172 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:20.170+0000 7f4cdc200640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4cd4106ef0 0x7f4cd41a5130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:25:20.172 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:20.170+0000 7f4cdc200640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f4cd4119380 con 0x7f4cd4106ef0 2024-09-11T00:25:20.172 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:20.170+0000 7f4cdc200640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f4cd4119200 con 0x7f4cd41023d0 2024-09-11T00:25:20.172 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:20.170+0000 7f4cdc200640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f4cd4119500 con 0x7f4cd4101ab0 2024-09-11T00:25:20.173 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:20.171+0000 7f4cd9800640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4cd4101ab0 0x7f4cd419f850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:25:20.173 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:20.171+0000 7f4cd9800640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4cd4101ab0 0x7f4cd419f850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.12:54142/0 (socket says 172.21.15.12:54142) 2024-09-11T00:25:20.174 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:20.171+0000 7f4cd9800640 1 -- 172.21.15.12:0/513610476 learned_addr learned my addr 172.21.15.12:0/513610476 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:25:20.174 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:20.171+0000 7f4cd8e00640 1 --2- 172.21.15.12:0/513610476 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4cd41023d0 0x7f4cd419fd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:25:20.174 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:20.171+0000 7f4cda200640 1 --2- 172.21.15.12:0/513610476 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4cd4106ef0 0x7f4cd41a5130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:25:20.174 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:20.171+0000 7f4cd8e00640 1 -- 172.21.15.12:0/513610476 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4cd4101ab0 msgr2=0x7f4cd419f850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:25:20.174 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:20.171+0000 7f4cd8e00640 1 --2- 172.21.15.12:0/513610476 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4cd4101ab0 0x7f4cd419f850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:20.174 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:20.171+0000 7f4cd8e00640 1 -- 172.21.15.12:0/513610476 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4cd4106ef0 msgr2=0x7f4cd41a5130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:25:20.174 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:20.171+0000 7f4cd8e00640 1 --2- 172.21.15.12:0/513610476 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4cd4106ef0 0x7f4cd41a5130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:20.175 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:20.171+0000 7f4cd8e00640 1 -- 172.21.15.12:0/513610476 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4cd41a5900 con 0x7f4cd41023d0 2024-09-11T00:25:20.175 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:20.172+0000 7f4cd8e00640 1 --2- 172.21.15.12:0/513610476 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4cd41023d0 0x7f4cd419fd90 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f4ccc005fd0 tx=0x7f4ccc00eda0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:25:20.176 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:20.173+0000 7f4cd2000640 1 -- 172.21.15.12:0/513610476 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ccc003e40 con 0x7f4cd41023d0 2024-09-11T00:25:20.176 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:20.173+0000 7f4cd2000640 1 -- 172.21.15.12:0/513610476 <== mon.2 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4ccc00cd00 con 0x7f4cd41023d0 2024-09-11T00:25:20.176 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:20.173+0000 7f4cd2000640 1 -- 172.21.15.12:0/513610476 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ccc0116d0 con 0x7f4cd41023d0 2024-09-11T00:25:20.176 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:20.173+0000 7f4cdc200640 1 -- 172.21.15.12:0/513610476 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4cd41a5bf0 con 0x7f4cd41023d0 2024-09-11T00:25:20.176 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:20.173+0000 7f4cdc200640 1 -- 172.21.15.12:0/513610476 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4cd41ac2b0 con 0x7f4cd41023d0 2024-09-11T00:25:20.180 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:20.178+0000 7f4cdecd3740 1 Processor -- start 2024-09-11T00:25:20.182 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:20.179+0000 7f4cdecd3740 1 -- start start 2024-09-11T00:25:20.182 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:20.179+0000 7f4cdecd3740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ed76c9c180 0x55ed76c9c560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:25:20.182 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:20.181+0000 7f4cdecd3740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ed76c9caa0 0x55ed76ca84f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:25:20.183 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:20.181+0000 7f4cda200640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ed76c9c180 0x55ed76c9c560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:25:20.183 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:20.181+0000 7f4cda200640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ed76c9c180 0x55ed76c9c560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.12:54144/0 (socket says 172.21.15.12:54144) 2024-09-11T00:25:20.183 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:20.182+0000 7f4cdecd3740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ed76ca8a30 0x55ed76caae20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:25:20.183 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:20.182+0000 7f4cda200640 1 -- 172.21.15.12:0/4170824353 learned_addr learned my addr 172.21.15.12:0/4170824353 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:25:20.184 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:20.182+0000 7f4cd9800640 1 --2- 172.21.15.12:0/4170824353 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ed76c9caa0 0x55ed76ca84f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:25:20.184 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:20.182+0000 7f4cd2000640 1 -- 172.21.15.12:0/513610476 <== mon.2 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7f4ccc00f040 con 0x7f4cd41023d0 2024-09-11T00:25:20.184 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:20.182+0000 7f4cdecd3740 1 -- 172.21.15.12:0/4170824353 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55ed76c11ca0 con 0x55ed76ca8a30 2024-09-11T00:25:20.184 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:20.182+0000 7f4cdecd3740 1 -- 172.21.15.12:0/4170824353 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55ed76c010d0 con 0x55ed76c9caa0 2024-09-11T00:25:20.184 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:20.182+0000 7f4cdecd3740 1 -- 172.21.15.12:0/4170824353 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55ed76c04ee0 con 0x55ed76c9c180 2024-09-11T00:25:20.185 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:20.183+0000 7f4cda200640 1 -- 172.21.15.12:0/4170824353 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ed76c9caa0 msgr2=0x55ed76ca84f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:25:20.185 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:20.183+0000 7f4cda200640 1 --2- 172.21.15.12:0/4170824353 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ed76c9caa0 0x55ed76ca84f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:20.185 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:20.183+0000 7f4cda200640 1 -- 172.21.15.12:0/4170824353 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ed76ca8a30 msgr2=0x55ed76caae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:25:20.185 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:20.183+0000 7f4cda200640 1 --2- 172.21.15.12:0/4170824353 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ed76ca8a30 0x55ed76caae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:20.185 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:20.183+0000 7f4cda200640 1 -- 172.21.15.12:0/4170824353 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ed76bb3400 con 0x55ed76c9c180 2024-09-11T00:25:20.186 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:20.184+0000 7f4cda200640 1 --2- 172.21.15.12:0/4170824353 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ed76c9c180 0x55ed76c9c560 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f4cc802a9e0 tx=0x7f4cc8004140 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e6688fa61bbf3676 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:25:20.186 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:20.185+0000 7f4cd2000640 1 --2- 172.21.15.12:0/513610476 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f4cbc0b5420 0x7f4cbc0b78e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:25:20.187 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:20.185+0000 7f4cc6a00640 1 -- 172.21.15.12:0/4170824353 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cc8035030 con 0x55ed76c9c180 2024-09-11T00:25:20.187 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:20.185+0000 7f4cd2000640 1 -- 172.21.15.12:0/513610476 <== mon.2 v2:172.21.15.165:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9208+0+0 (secure 0 0 0) 0x7f4ccc0e61e0 con 0x7f4cd41023d0 2024-09-11T00:25:20.187 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:20.185+0000 7f4cc6a00640 1 -- 172.21.15.12:0/4170824353 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4cc8004470 con 0x55ed76c9c180 2024-09-11T00:25:20.187 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:20.185+0000 7f4cc6a00640 1 -- 172.21.15.12:0/4170824353 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cc8004e10 con 0x55ed76c9c180 2024-09-11T00:25:20.187 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:20.186+0000 7f4cd9800640 1 --2- 172.21.15.12:0/513610476 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f4cbc0b5420 0x7f4cbc0b78e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:25:20.188 INFO:tasks.workunit.client.0.smithi012.stdout: 10 98666 17.99 MB/sec execute 196 sec latency 54.001 ms 2024-09-11T00:25:20.189 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:20.186+0000 7f4cdecd3740 1 -- 172.21.15.12:0/4170824353 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ed76c9c180 msgr2=0x55ed76c9c560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:25:20.189 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:20.186+0000 7f4cdecd3740 1 --2- 172.21.15.12:0/4170824353 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ed76c9c180 0x55ed76c9c560 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f4cc802a9e0 tx=0x7f4cc8004140 comp rx=0 tx=0).stop 2024-09-11T00:25:20.189 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:20.187+0000 7f4cdecd3740 1 -- 172.21.15.12:0/4170824353 shutdown_connections 2024-09-11T00:25:20.189 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:20.187+0000 7f4cdecd3740 1 --2- 172.21.15.12:0/4170824353 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ed76ca8a30 0x55ed76caae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:20.189 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:20.187+0000 7f4cdecd3740 1 --2- 172.21.15.12:0/4170824353 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ed76c9caa0 0x55ed76ca84f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:20.189 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:20.187+0000 7f4cdecd3740 1 --2- 172.21.15.12:0/4170824353 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ed76c9c180 0x55ed76c9c560 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:20.189 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:20.187+0000 7f4cdecd3740 1 -- 172.21.15.12:0/4170824353 >> 172.21.15.12:0/4170824353 conn(0x55ed76c99fb0 msgr2=0x55ed76c9b150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:25:20.189 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:20.187+0000 7f4cdecd3740 1 -- 172.21.15.12:0/4170824353 shutdown_connections 2024-09-11T00:25:20.190 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:20.188+0000 7f4cdecd3740 1 -- 172.21.15.12:0/4170824353 wait complete. 2024-09-11T00:25:20.191 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:20.188+0000 7f4cd9800640 1 --2- 172.21.15.12:0/513610476 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f4cbc0b5420 0x7f4cbc0b78e0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x55ed76c9ae90 tx=0x7f4cc0005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:25:20.191 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:20.188+0000 7f4cdecd3740 1 -- start start 2024-09-11T00:25:20.191 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:20.189+0000 7f4cdecd3740 1 Processor -- start 2024-09-11T00:25:20.192 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:20.190+0000 7f4cdecd3740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ed76c9c180 0x55ed76ca83c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:25:20.192 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:20.190+0000 7f4cdecd3740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ed76c9caa0 0x55ed76ca5810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:25:20.192 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:20.190+0000 7f4cdecd3740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ed76ca8a30 0x55ed76ca5ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:25:20.192 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:20.191+0000 7f4cd9800640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ed76c9c180 0x55ed76ca83c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:25:20.192 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:20.191+0000 7f4cd9800640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ed76c9c180 0x55ed76ca83c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.12:54152/0 (socket says 172.21.15.12:54152) 2024-09-11T00:25:20.193 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:20.191+0000 7f4cd8e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ed76c9caa0 0x55ed76ca5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:25:20.193 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:20.191+0000 7f4cd8e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ed76c9caa0 0x55ed76ca5810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.12:43420/0 (socket says 172.21.15.12:43420) 2024-09-11T00:25:20.193 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:20.191+0000 7f4cdecd3740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55ed76a627a0 con 0x55ed76ca8a30 2024-09-11T00:25:20.193 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:20.191+0000 7f4cdecd3740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55ed76ba6a10 con 0x55ed76c9caa0 2024-09-11T00:25:20.194 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:20.191+0000 7f4cdecd3740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55ed76cab650 con 0x55ed76c9c180 2024-09-11T00:25:20.194 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:20.191+0000 7f4cdecd3740 10 client.0 _ll_register_callbacks cb 0x55ed76bb6be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:25:20.194 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:20.191+0000 7f4cd9800640 1 -- 172.21.15.12:0/1146378462 learned_addr learned my addr 172.21.15.12:0/1146378462 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:25:20.194 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:20.192+0000 7f4cd8e00640 1 -- 172.21.15.12:0/1146378462 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ed76c9c180 msgr2=0x55ed76ca83c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:25:20.195 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:20.192+0000 7f4cd8e00640 1 --2- 172.21.15.12:0/1146378462 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ed76c9c180 0x55ed76ca83c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:20.195 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:20.192+0000 7f4cd8e00640 1 -- 172.21.15.12:0/1146378462 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ed76ca8a30 msgr2=0x55ed76ca5ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:25:20.195 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:20.192+0000 7f4cd8e00640 1 --2- 172.21.15.12:0/1146378462 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ed76ca8a30 0x55ed76ca5ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:20.195 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:20.192+0000 7f4cd8e00640 1 -- 172.21.15.12:0/1146378462 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ed7696eb90 con 0x55ed76c9caa0 2024-09-11T00:25:20.195 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:20.193+0000 7f4cd8e00640 1 --2- 172.21.15.12:0/1146378462 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ed76c9caa0 0x55ed76ca5810 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f4ccc00c640 tx=0x7f4ccc0a8410 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f9a0d886f74f7e37 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:25:20.195 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:20.193+0000 7f4cbb400640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-09-11T00:25:20.196 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:20.193+0000 7f4cbb400640 1 -- 172.21.15.12:0/1146378462 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ccc0abce0 con 0x55ed76c9caa0 2024-09-11T00:25:20.196 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:20.193+0000 7f4cbb400640 1 -- 172.21.15.12:0/1146378462 <== mon.2 v2:172.21.15.165:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9208+0+0 (secure 0 0 0) 0x7f4ccc0b5870 con 0x55ed76c9caa0 2024-09-11T00:25:20.196 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:20.194+0000 7f4cdecd3740 1 -- 172.21.15.12:0/1146378462 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4cc8004470 con 0x55ed76c9caa0 2024-09-11T00:25:20.197 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:20.196+0000 7f4cbb400640 1 -- 172.21.15.12:0/1146378462 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=69) ==== 24+0+0 (secure 0 0 0) 0x7f4ccc0ac080 con 0x55ed76c9caa0 2024-09-11T00:25:20.198 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:20.196+0000 7f4cdecd3740 10 client.27479 fetch_fsmap learned FSMap version 69 2024-09-11T00:25:20.198 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:20.196+0000 7f4cdecd3740 1 -- 172.21.15.12:0/1146378462 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=69,monmap=4+}) -- 0x55ed76cb7080 con 0x55ed76c9caa0 2024-09-11T00:25:20.200 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:20.197+0000 7f4cbb400640 1 -- 172.21.15.12:0/1146378462 <== mon.2 v2:172.21.15.165:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4cc8004470 con 0x55ed76c9caa0 2024-09-11T00:25:20.201 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:20.197+0000 7f4cbb400640 1 -- 172.21.15.12:0/1146378462 <== mon.2 v2:172.21.15.165:3300/0 5 ==== fsmap(e 69) ==== 8390+0+0 (secure 0 0 0) 0x7f4ccc00c880 con 0x55ed76c9caa0 2024-09-11T00:25:20.201 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:20.197+0000 7f4cdecd3740 10 client.27479 fetch_fsmap finished waiting for FSMap version 69 2024-09-11T00:25:20.201 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:20.197+0000 7f4cdecd3740 10 client.27479 resolve_mds: validated gid 24674 aka daemon mds.b 2024-09-11T00:25:20.201 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:20.197+0000 7f4cdecd3740 20 client.27479 populate_metadata read hostname 'smithi012' 2024-09-11T00:25:20.201 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:20.197+0000 7f4cdecd3740 1 --2- 172.21.15.12:0/1146378462 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55ed76cb8540 0x55ed76cba930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:25:20.201 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:20.197+0000 7f4cdecd3740 4 client.27479 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T00:25:20.201 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:20.197+0000 7f4cdecd3740 1 -- 172.21.15.12:0/1146378462 --> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ed76c13bf0 con 0x55ed76cb8540 2024-09-11T00:25:20.202 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:20.197+0000 7f4cd9800640 1 --2- 172.21.15.12:0/1146378462 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55ed76cb8540 0x55ed76cba930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:25:20.202 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:20.198+0000 7f4cd9800640 1 --2- 172.21.15.12:0/1146378462 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55ed76cb8540 0x55ed76cba930 crc :-1 s=READY pgs=255 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=de093064e26f2070 server_cookie=369b239c2c121c0d in_seq=0 out_seq=0 2024-09-11T00:25:20.202 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:20.199+0000 7f4cbb400640 10 client.27479 ms_handle_connect on v2:172.21.15.165:6838/501312984 2024-09-11T00:25:20.203 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:20.201+0000 7f4cbb400640 1 -- 172.21.15.12:0/1146378462 <== mds.0 v2:172.21.15.165:6838/501312984 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55ed76c13bf0 con 0x55ed76cb8540 2024-09-11T00:25:20.203 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:20.201+0000 7f4cbb400640 10 client.27479 handle_command_reply: tid=0 2024-09-11T00:25:20.203 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:20.201+0000 7f4cdecd3740 1 client.27479 shutdown 2024-09-11T00:25:20.204 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:20.202+0000 7f4cdecd3740 1 -- 172.21.15.12:0/1146378462 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ed76c9caa0 msgr2=0x55ed76ca5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:25:20.204 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:20.202+0000 7f4cdecd3740 1 --2- 172.21.15.12:0/1146378462 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ed76c9caa0 0x55ed76ca5810 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f4ccc00c640 tx=0x7f4ccc0a8410 comp rx=0 tx=0).stop 2024-09-11T00:25:20.204 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:20.203+0000 7f4cdecd3740 1 -- 172.21.15.12:0/1146378462 shutdown_connections 2024-09-11T00:25:20.206 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:20.203+0000 7f4cdecd3740 1 --2- 172.21.15.12:0/1146378462 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55ed76cb8540 0x55ed76cba930 crc :-1 s=READY pgs=255 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:20.206 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:20.203+0000 7f4cdecd3740 1 --2- 172.21.15.12:0/1146378462 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ed76ca8a30 0x55ed76ca5ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:20.206 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:20.203+0000 7f4cdecd3740 1 --2- 172.21.15.12:0/1146378462 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ed76c9caa0 0x55ed76ca5810 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:20.206 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:20.203+0000 7f4cdecd3740 1 --2- 172.21.15.12:0/1146378462 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ed76c9c180 0x55ed76ca83c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:20.206 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:20.203+0000 7f4cdecd3740 1 -- 172.21.15.12:0/1146378462 >> 172.21.15.12:0/1146378462 conn(0x55ed76c99fb0 msgr2=0x55ed76c9fb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:25:20.206 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:20.203+0000 7f4cbb400640 0 client.27479 ms_handle_reset on v2:172.21.15.165:6838/501312984 2024-09-11T00:25:20.207 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:20.204+0000 7f4cdecd3740 1 -- 172.21.15.12:0/1146378462 shutdown_connections 2024-09-11T00:25:20.207 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:20.204+0000 7f4cdecd3740 1 -- 172.21.15.12:0/1146378462 wait complete. 2024-09-11T00:25:20.207 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:20.204+0000 7f4cdecd3740 20 client.27479 trim_cache size 0 max 16384 2024-09-11T00:25:20.243 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:20.240+0000 7f4cdecd3740 1 Processor -- start 2024-09-11T00:25:20.243 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:20.241+0000 7f4cdecd3740 1 -- start start 2024-09-11T00:25:20.247 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:20.242+0000 7f4cdecd3740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ed76c9c180 0x55ed76cb4880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:25:20.247 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:20.242+0000 7f4cdecd3740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ed76c9caa0 0x55ed76ca2a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:25:20.247 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:20.242+0000 7f4cdecd3740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ed76ca8a30 0x55ed76ca2f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:25:20.247 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:20.242+0000 7f4cdecd3740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f4cd41b6270 con 0x55ed76ca8a30 2024-09-11T00:25:20.248 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:20.242+0000 7f4cdecd3740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f4cd41b7e10 con 0x55ed76c9caa0 2024-09-11T00:25:20.248 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:20.242+0000 7f4cdecd3740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f4cd41b6940 con 0x55ed76c9c180 2024-09-11T00:25:20.248 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:20.242+0000 7f4cda200640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ed76ca8a30 0x55ed76ca2f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:25:20.248 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:20.242+0000 7f4cda200640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ed76ca8a30 0x55ed76ca2f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:57358/0 (socket says 172.21.15.12:57358) 2024-09-11T00:25:20.248 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:20.242+0000 7f4cda200640 1 -- 172.21.15.12:0/722647063 learned_addr learned my addr 172.21.15.12:0/722647063 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:25:20.248 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:20.242+0000 7f4cd8e00640 1 --2- 172.21.15.12:0/722647063 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ed76c9caa0 0x55ed76ca2a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:25:20.248 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:20.243+0000 7f4cd8e00640 1 -- 172.21.15.12:0/722647063 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ed76c9c180 msgr2=0x55ed76cb4880 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-11T00:25:20.248 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:20.244+0000 7f4cd8e00640 1 --2- 172.21.15.12:0/722647063 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ed76c9c180 0x55ed76cb4880 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:20.248 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:20.244+0000 7f4cd8e00640 1 -- 172.21.15.12:0/722647063 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ed76ca8a30 msgr2=0x55ed76ca2f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:25:20.248 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:20.244+0000 7f4cd8e00640 1 --2- 172.21.15.12:0/722647063 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ed76ca8a30 0x55ed76ca2f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:20.249 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:20.244+0000 7f4cd8e00640 1 -- 172.21.15.12:0/722647063 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ed76c9aa30 con 0x55ed76c9caa0 2024-09-11T00:25:20.249 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:20.244+0000 7f4cd8e00640 1 --2- 172.21.15.12:0/722647063 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ed76c9caa0 0x55ed76ca2a40 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f4ccc0a8b40 tx=0x7f4ccc004550 comp rx=0 tx=0).ready entity=mon.2 client_cookie=260460583a918bbb server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:25:20.249 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:20.245+0000 7f4cda200640 1 --2- 172.21.15.12:0/722647063 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ed76ca8a30 0x55ed76ca2f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T00:25:20.249 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:20.245+0000 7f4cc6000640 1 -- 172.21.15.12:0/722647063 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ccc00c970 con 0x55ed76c9caa0 2024-09-11T00:25:20.249 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:20.245+0000 7f4cc6000640 1 -- 172.21.15.12:0/722647063 <== mon.2 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4ccc05c040 con 0x55ed76c9caa0 2024-09-11T00:25:20.249 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:20.245+0000 7f4cc6000640 1 -- 172.21.15.12:0/722647063 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ccc0b58d0 con 0x55ed76c9caa0 2024-09-11T00:25:20.250 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:20.247+0000 7f4cdecd3740 1 -- 172.21.15.12:0/722647063 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ed76c9caa0 msgr2=0x55ed76ca2a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:25:20.250 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:20.247+0000 7f4cdecd3740 1 --2- 172.21.15.12:0/722647063 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ed76c9caa0 0x55ed76ca2a40 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f4ccc0a8b40 tx=0x7f4ccc004550 comp rx=0 tx=0).stop 2024-09-11T00:25:20.251 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:20.247+0000 7f4cdecd3740 1 -- 172.21.15.12:0/722647063 shutdown_connections 2024-09-11T00:25:20.251 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:20.247+0000 7f4cdecd3740 1 --2- 172.21.15.12:0/722647063 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ed76ca8a30 0x55ed76ca2f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:20.251 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:20.247+0000 7f4cdecd3740 1 --2- 172.21.15.12:0/722647063 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ed76c9caa0 0x55ed76ca2a40 secure :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f4ccc0a8b40 tx=0x7f4ccc004550 comp rx=0 tx=0).stop 2024-09-11T00:25:20.251 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:20.247+0000 7f4cdecd3740 1 --2- 172.21.15.12:0/722647063 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ed76c9c180 0x55ed76cb4880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:20.251 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:20.247+0000 7f4cdecd3740 1 -- 172.21.15.12:0/722647063 >> 172.21.15.12:0/722647063 conn(0x55ed76c99fb0 msgr2=0x55ed76c990b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:25:20.251 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:20.248+0000 7f4cdecd3740 1 -- 172.21.15.12:0/722647063 shutdown_connections 2024-09-11T00:25:20.252 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:20.248+0000 7f4cdecd3740 1 -- 172.21.15.12:0/722647063 wait complete. 2024-09-11T00:25:20.253 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:20.249+0000 7f4cdecd3740 1 -- start start 2024-09-11T00:25:20.253 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:20.252+0000 7f4cdecd3740 1 Processor -- start 2024-09-11T00:25:20.255 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:20.253+0000 7f4cdecd3740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ed76c9c180 0x55ed76cad900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:25:20.255 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:20.253+0000 7f4cdecd3740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ed76ca8a30 0x55ed76cade40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:25:20.256 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:20.253+0000 7f4cdecd3740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ed76cae380 0x55ed76d253f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:25:20.256 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:20.253+0000 7f4cdecd3740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f4cd41b7740 con 0x55ed76c9c180 2024-09-11T00:25:20.256 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:20.253+0000 7f4cdecd3740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f4cd41b7070 con 0x55ed76cae380 2024-09-11T00:25:20.256 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:20.253+0000 7f4cdecd3740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f4cd41b4250 con 0x55ed76ca8a30 2024-09-11T00:25:20.256 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:20.253+0000 7f4cdecd3740 10 client.0 _ll_register_callbacks cb 0x7f4ccc005b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:25:20.257 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:20.254+0000 7f4cd8e00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ed76ca8a30 0x55ed76cade40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:25:20.257 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:20.254+0000 7f4cd8e00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ed76ca8a30 0x55ed76cade40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.12:54160/0 (socket says 172.21.15.12:54160) 2024-09-11T00:25:20.257 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:20.254+0000 7f4cd8e00640 1 -- 172.21.15.12:0/2776556708 learned_addr learned my addr 172.21.15.12:0/2776556708 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:25:20.258 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:20.255+0000 7f4cda200640 1 --2- 172.21.15.12:0/2776556708 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ed76cae380 0x55ed76d253f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:25:20.258 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:20.255+0000 7f4cd8e00640 1 -- 172.21.15.12:0/2776556708 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ed76cae380 msgr2=0x55ed76d253f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:25:20.258 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:20.255+0000 7f4cd8e00640 1 --2- 172.21.15.12:0/2776556708 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ed76cae380 0x55ed76d253f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:20.259 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:20.255+0000 7f4cd9800640 1 --2- 172.21.15.12:0/2776556708 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ed76c9c180 0x55ed76cad900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:25:20.259 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:20.256+0000 7f4cd8e00640 1 -- 172.21.15.12:0/2776556708 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ed76c9c180 msgr2=0x55ed76cad900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:25:20.259 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:20.257+0000 7f4cd8e00640 1 --2- 172.21.15.12:0/2776556708 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ed76c9c180 0x55ed76cad900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:20.259 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:20.257+0000 7f4cd8e00640 1 -- 172.21.15.12:0/2776556708 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ed76d25980 con 0x55ed76ca8a30 2024-09-11T00:25:20.261 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:20.258+0000 7f4cda200640 1 --2- 172.21.15.12:0/2776556708 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ed76cae380 0x55ed76d253f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T00:25:20.261 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:20.258+0000 7f4cd9800640 1 --2- 172.21.15.12:0/2776556708 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ed76c9c180 0x55ed76cad900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T00:25:20.261 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:20.258+0000 7f4cd8e00640 1 --2- 172.21.15.12:0/2776556708 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ed76ca8a30 0x55ed76cade40 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f4ccc004420 tx=0x7f4ccc0b58a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=73e00dc7a5d2d145 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:25:20.261 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:20.258+0000 7f4cbb400640 10 client.0 ms_handle_connect on v2:172.21.15.178:3300/0 2024-09-11T00:25:20.261 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:20.258+0000 7f4cbb400640 1 -- 172.21.15.12:0/2776556708 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ccc0ac340 con 0x55ed76ca8a30 2024-09-11T00:25:20.261 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:20.258+0000 7f4cbb400640 1 -- 172.21.15.12:0/2776556708 <== mon.1 v2:172.21.15.178:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9208+0+0 (secure 0 0 0) 0x7f4ccc0ab9e0 con 0x55ed76ca8a30 2024-09-11T00:25:20.261 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:20.259+0000 7f4cdecd3740 1 -- 172.21.15.12:0/2776556708 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4ccc05c040 con 0x55ed76ca8a30 2024-09-11T00:25:20.262 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:20.260+0000 7f4cbb400640 1 -- 172.21.15.12:0/2776556708 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=69) ==== 24+0+0 (secure 0 0 0) 0x7f4ccc0cd050 con 0x55ed76ca8a30 2024-09-11T00:25:20.262 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:20.260+0000 7f4cdecd3740 10 client.27457 fetch_fsmap learned FSMap version 69 2024-09-11T00:25:20.262 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:20.260+0000 7f4cdecd3740 1 -- 172.21.15.12:0/2776556708 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,fsmap=69,monmap=4+}) -- 0x55ed76d37c50 con 0x55ed76ca8a30 2024-09-11T00:25:20.262 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:20.260+0000 7f4cbb400640 1 -- 172.21.15.12:0/2776556708 <== mon.1 v2:172.21.15.178:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4ccc05c040 con 0x55ed76ca8a30 2024-09-11T00:25:20.262 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:20.261+0000 7f4cbb400640 1 -- 172.21.15.12:0/2776556708 <== mon.1 v2:172.21.15.178:3300/0 5 ==== fsmap(e 69) ==== 8390+0+0 (secure 0 0 0) 0x7f4ccc0ab240 con 0x55ed76ca8a30 2024-09-11T00:25:20.265 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:20.263+0000 7f4cdecd3740 10 client.27457 fetch_fsmap finished waiting for FSMap version 69 2024-09-11T00:25:20.265 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:20.263+0000 7f4cdecd3740 10 client.27457 resolve_mds: validated gid 24674 aka daemon mds.b 2024-09-11T00:25:20.265 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:20.263+0000 7f4cdecd3740 20 client.27457 populate_metadata read hostname 'smithi012' 2024-09-11T00:25:20.266 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:20.263+0000 7f4cdecd3740 1 --2- 172.21.15.12:0/2776556708 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55ed76d381c0 0x55ed76d3a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:25:20.266 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:20.263+0000 7f4cdecd3740 4 client.27457 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 141.0, "await_for": 45.0}] 2024-09-11T00:25:20.266 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:20.264+0000 7f4cdecd3740 1 -- 172.21.15.12:0/2776556708 --> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 141.0, "await_for": 45.0}) -- 0x55ed76c0f6a0 con 0x55ed76d381c0 2024-09-11T00:25:20.266 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:20.264+0000 7f4cd9800640 1 --2- 172.21.15.12:0/2776556708 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55ed76d381c0 0x55ed76d3a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:25:20.270 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:20.269+0000 7f4cbb400640 10 client.27457 ms_handle_connect on v2:172.21.15.165:6838/501312984 2024-09-11T00:25:20.270 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:20.269+0000 7f4cd9800640 1 --2- 172.21.15.12:0/2776556708 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55ed76d381c0 0x55ed76d3a5e0 crc :-1 s=READY pgs=256 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=65dcc31f69ab3656 server_cookie=f44838ba91eba21e in_seq=0 out_seq=0 2024-09-11T00:25:20.283 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:25:19 smithi165 ceph-mon[35356]: pgmap v946: 129 pgs: 129 active+clean; 565 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 2.4 MiB/s wr, 381 op/s 2024-09-11T00:25:20.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:25:19 smithi178 ceph-mon[35919]: pgmap v946: 129 pgs: 129 active+clean; 565 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 2.4 MiB/s wr, 381 op/s 2024-09-11T00:25:20.936 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:20.934+0000 7f4cbb400640 1 -- 172.21.15.12:0/2776556708 <== mds.0 v2:172.21.15.165:6838/501312984 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55ed76c0f6a0 con 0x55ed76d381c0 2024-09-11T00:25:20.937 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:20.934+0000 7f4cbb400640 10 client.27457 handle_command_reply: tid=0 2024-09-11T00:25:20.937 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:20.934+0000 7f4cdecd3740 1 client.27457 shutdown 2024-09-11T00:25:20.937 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:20.934+0000 7f4cdecd3740 1 -- 172.21.15.12:0/2776556708 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ed76ca8a30 msgr2=0x55ed76cade40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:25:20.937 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:20.934+0000 7f4cdecd3740 1 --2- 172.21.15.12:0/2776556708 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ed76ca8a30 0x55ed76cade40 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f4ccc004420 tx=0x7f4ccc0b58a0 comp rx=0 tx=0).stop 2024-09-11T00:25:20.937 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:20.935+0000 7f4cdecd3740 1 -- 172.21.15.12:0/2776556708 shutdown_connections 2024-09-11T00:25:20.937 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:20.935+0000 7f4cdecd3740 1 --2- 172.21.15.12:0/2776556708 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55ed76d381c0 0x55ed76d3a5e0 crc :-1 s=READY pgs=256 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:20.937 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:20.935+0000 7f4cdecd3740 1 --2- 172.21.15.12:0/2776556708 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ed76cae380 0x55ed76d253f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:20.937 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:20.935+0000 7f4cdecd3740 1 --2- 172.21.15.12:0/2776556708 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ed76ca8a30 0x55ed76cade40 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:20.937 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:20.935+0000 7f4cdecd3740 1 --2- 172.21.15.12:0/2776556708 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ed76c9c180 0x55ed76cad900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:20.937 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:20.935+0000 7f4cbb400640 0 client.27457 ms_handle_reset on v2:172.21.15.165:6838/501312984 2024-09-11T00:25:20.937 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:20.935+0000 7f4cdecd3740 1 -- 172.21.15.12:0/2776556708 >> 172.21.15.12:0/2776556708 conn(0x55ed76c99fb0 msgr2=0x55ed76ca6e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:25:20.938 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:20.935+0000 7f4cdecd3740 1 -- 172.21.15.12:0/2776556708 shutdown_connections 2024-09-11T00:25:20.938 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:20.935+0000 7f4cdecd3740 1 -- 172.21.15.12:0/2776556708 wait complete. 2024-09-11T00:25:20.938 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:20.936+0000 7f4cdecd3740 20 client.27457 trim_cache size 0 max 16384 2024-09-11T00:25:20.938 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-09-11T00:25:20.938 INFO:teuthology.orchestra.run.smithi012.stdout: "epoch": 69, 2024-09-11T00:25:20.939 INFO:teuthology.orchestra.run.smithi012.stdout: "leader": 24674, 2024-09-11T00:25:20.939 INFO:teuthology.orchestra.run.smithi012.stdout: "set_version": 54, 2024-09-11T00:25:20.939 INFO:teuthology.orchestra.run.smithi012.stdout: "sets": { 2024-09-11T00:25:20.939 INFO:teuthology.orchestra.run.smithi012.stdout: "18bbb50e": { 2024-09-11T00:25:20.939 INFO:teuthology.orchestra.run.smithi012.stdout: "version": 54, 2024-09-11T00:25:20.939 INFO:teuthology.orchestra.run.smithi012.stdout: "age_ref": 0.0, 2024-09-11T00:25:20.939 INFO:teuthology.orchestra.run.smithi012.stdout: "state": { 2024-09-11T00:25:20.939 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "QUIESCED", 2024-09-11T00:25:20.939 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 0.0 2024-09-11T00:25:20.939 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:25:20.939 INFO:teuthology.orchestra.run.smithi012.stdout: "timeout": 90.0, 2024-09-11T00:25:20.939 INFO:teuthology.orchestra.run.smithi012.stdout: "expiration": 141.0, 2024-09-11T00:25:20.939 INFO:teuthology.orchestra.run.smithi012.stdout: "members": { 2024-09-11T00:25:20.940 INFO:teuthology.orchestra.run.smithi012.stdout: "file:/": { 2024-09-11T00:25:20.940 INFO:teuthology.orchestra.run.smithi012.stdout: "excluded": false, 2024-09-11T00:25:20.940 INFO:teuthology.orchestra.run.smithi012.stdout: "state": { 2024-09-11T00:25:20.940 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "QUIESCED", 2024-09-11T00:25:20.940 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 0.0 2024-09-11T00:25:20.940 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:25:20.940 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:25:20.940 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:25:20.940 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:25:20.940 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:25:20.940 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-09-11T00:25:20.941 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:20.937+0000 7f4cdc200640 1 -- 172.21.15.12:0/513610476 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f4cbc0b5420 msgr2=0x7f4cbc0b78e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:25:20.941 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:20.937+0000 7f4cdc200640 1 --2- 172.21.15.12:0/513610476 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f4cbc0b5420 0x7f4cbc0b78e0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x55ed76c9ae90 tx=0x7f4cc0005f90 comp rx=0 tx=0).stop 2024-09-11T00:25:20.941 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:20.937+0000 7f4cdc200640 1 -- 172.21.15.12:0/513610476 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4cd41023d0 msgr2=0x7f4cd419fd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:25:20.941 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:20.937+0000 7f4cdc200640 1 --2- 172.21.15.12:0/513610476 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4cd41023d0 0x7f4cd419fd90 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f4ccc005fd0 tx=0x7f4ccc00eda0 comp rx=0 tx=0).stop 2024-09-11T00:25:20.941 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:20.937+0000 7f4cdc200640 1 -- 172.21.15.12:0/513610476 shutdown_connections 2024-09-11T00:25:20.941 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:20.937+0000 7f4cdc200640 1 --2- 172.21.15.12:0/513610476 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f4cbc0b5420 0x7f4cbc0b78e0 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:20.941 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:20.937+0000 7f4cdc200640 1 --2- 172.21.15.12:0/513610476 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4cd4106ef0 0x7f4cd41a5130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:20.941 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:20.937+0000 7f4cdc200640 1 --2- 172.21.15.12:0/513610476 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4cd41023d0 0x7f4cd419fd90 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:20.941 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:20.937+0000 7f4cdc200640 1 --2- 172.21.15.12:0/513610476 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4cd4101ab0 0x7f4cd419f850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:20.942 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:20.937+0000 7f4cdc200640 1 -- 172.21.15.12:0/513610476 >> 172.21.15.12:0/513610476 conn(0x7f4cd40fd700 msgr2=0x7f4cd4104820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:25:20.942 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:20.939+0000 7f4cdc200640 1 -- 172.21.15.12:0/513610476 shutdown_connections 2024-09-11T00:25:20.942 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:20.939+0000 7f4cdc200640 1 -- 172.21.15.12:0/513610476 wait complete. 2024-09-11T00:25:21.188 INFO:tasks.workunit.client.0.smithi012.stdout: 10 98759 17.93 MB/sec execute 197 sec latency 943.172 ms 2024-09-11T00:25:21.956 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '18bbb50e', quiesce took 6.6 seconds. Will release after: 14.4 2024-09-11T00:25:22.188 INFO:tasks.workunit.client.0.smithi012.stdout: 10 98759 17.84 MB/sec execute 198 sec latency 1943.620 ms 2024-09-11T00:25:22.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:25:21 smithi165 ceph-mon[35356]: pgmap v947: 129 pgs: 129 active+clean; 577 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 3.8 MiB/s wr, 560 op/s 2024-09-11T00:25:22.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:25:21 smithi012 ceph-mon[30333]: pgmap v947: 129 pgs: 129 active+clean; 577 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 3.8 MiB/s wr, 560 op/s 2024-09-11T00:25:22.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:25:21 smithi178 ceph-mon[35919]: pgmap v947: 129 pgs: 129 active+clean; 577 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 3.8 MiB/s wr, 560 op/s 2024-09-11T00:25:23.188 INFO:tasks.workunit.client.0.smithi012.stdout: 10 98759 17.75 MB/sec execute 199 sec latency 2943.703 ms 2024-09-11T00:25:23.932 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:23.930+0000 7f0283e00640 1 -- 172.21.15.12:0/3745897843 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f02841049b0 msgr2=0x7f0284104db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:25:23.932 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:23.930+0000 7f0283e00640 1 --2- 172.21.15.12:0/3745897843 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f02841049b0 0x7f0284104db0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f0274009930 tx=0x7f027402a420 comp rx=0 tx=0).stop 2024-09-11T00:25:23.932 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:23.931+0000 7f0283e00640 1 -- 172.21.15.12:0/3745897843 shutdown_connections 2024-09-11T00:25:23.933 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:23.931+0000 7f0283e00640 1 --2- 172.21.15.12:0/3745897843 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f02841064c0 0x7f028410ce40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:23.933 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:23.931+0000 7f0283e00640 1 --2- 172.21.15.12:0/3745897843 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0284105b00 0x7f0284105f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:23.933 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:23.931+0000 7f0283e00640 1 --2- 172.21.15.12:0/3745897843 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f02841049b0 0x7f0284104db0 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:23.933 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:23.931+0000 7f0283e00640 1 -- 172.21.15.12:0/3745897843 >> 172.21.15.12:0/3745897843 conn(0x7f0284100250 msgr2=0x7f0284102690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:25:23.933 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:23.931+0000 7f0283e00640 1 -- 172.21.15.12:0/3745897843 shutdown_connections 2024-09-11T00:25:23.933 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:23.931+0000 7f0283e00640 1 -- 172.21.15.12:0/3745897843 wait complete. 2024-09-11T00:25:23.935 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:23.932+0000 7f0283e00640 1 Processor -- start 2024-09-11T00:25:23.935 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:23.932+0000 7f0283e00640 1 -- start start 2024-09-11T00:25:23.935 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:23.932+0000 7f0283e00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f02841049b0 0x7f028419adc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:25:23.935 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:23.932+0000 7f0283e00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0284105b00 0x7f028419b410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:25:23.935 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:23.932+0000 7f0283e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f02841064c0 0x7f028419c040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:25:23.936 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:23.932+0000 7f0283e00640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f0284110670 con 0x7f02841049b0 2024-09-11T00:25:23.936 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:23.932+0000 7f0283e00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f02841104f0 con 0x7f02841064c0 2024-09-11T00:25:23.936 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:23.932+0000 7f0283e00640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f02841107f0 con 0x7f0284105b00 2024-09-11T00:25:23.936 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:23.932+0000 7f0282a00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f02841049b0 0x7f028419adc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:25:23.936 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:23.932+0000 7f0282a00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f02841049b0 0x7f028419adc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:57384/0 (socket says 172.21.15.12:57384) 2024-09-11T00:25:23.936 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:23.932+0000 7f0282a00640 1 -- 172.21.15.12:0/2536555485 learned_addr learned my addr 172.21.15.12:0/2536555485 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:25:23.936 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:23.932+0000 7f0282000640 1 --2- 172.21.15.12:0/2536555485 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0284105b00 0x7f028419b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:25:23.936 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:23.933+0000 7f0283400640 1 --2- 172.21.15.12:0/2536555485 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f02841064c0 0x7f028419c040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:25:23.936 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:23.933+0000 7f0282a00640 1 -- 172.21.15.12:0/2536555485 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0284105b00 msgr2=0x7f028419b410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:25:23.936 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:23.933+0000 7f0282a00640 1 --2- 172.21.15.12:0/2536555485 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0284105b00 0x7f028419b410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:23.936 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:23.933+0000 7f0282a00640 1 -- 172.21.15.12:0/2536555485 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f02841064c0 msgr2=0x7f028419c040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:25:23.937 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:23.933+0000 7f0282a00640 1 --2- 172.21.15.12:0/2536555485 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f02841064c0 0x7f028419c040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:23.937 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:23.933+0000 7f0282a00640 1 -- 172.21.15.12:0/2536555485 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f028419c900 con 0x7f02841049b0 2024-09-11T00:25:23.937 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:23.933+0000 7f0282a00640 1 --2- 172.21.15.12:0/2536555485 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f02841049b0 0x7f028419adc0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f0274031500 tx=0x7f0274003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:25:23.937 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:23.933+0000 7f027b400640 1 -- 172.21.15.12:0/2536555485 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f027403f070 con 0x7f02841049b0 2024-09-11T00:25:23.937 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:23.933+0000 7f027b400640 1 -- 172.21.15.12:0/2536555485 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0274031c50 con 0x7f02841049b0 2024-09-11T00:25:23.937 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:23.933+0000 7f027b400640 1 -- 172.21.15.12:0/2536555485 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0274035030 con 0x7f02841049b0 2024-09-11T00:25:23.937 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:23.933+0000 7f0283e00640 1 -- 172.21.15.12:0/2536555485 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f028419cb90 con 0x7f02841049b0 2024-09-11T00:25:23.937 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:23.934+0000 7f0283e00640 1 -- 172.21.15.12:0/2536555485 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02841a30e0 con 0x7f02841049b0 2024-09-11T00:25:23.938 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:23.937+0000 7f028cc54740 1 Processor -- start 2024-09-11T00:25:23.938 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:23.937+0000 7f028cc54740 1 -- start start 2024-09-11T00:25:23.939 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:23.937+0000 7f028cc54740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56081b2a5970 0x56081b2a7d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:25:23.939 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:23.937+0000 7f028cc54740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56081b2a82a0 0x56081b2aa690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:25:23.939 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:23.937+0000 7f028cc54740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56081b2aace0 0x56081b2ad1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:25:23.939 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:23.937+0000 7f028cc54740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x56081b2186b0 con 0x56081b2aace0 2024-09-11T00:25:23.939 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:23.937+0000 7f028cc54740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x56081b207ae0 con 0x56081b2a82a0 2024-09-11T00:25:23.940 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:23.937+0000 7f028cc54740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x56081b20b8f0 con 0x56081b2a5970 2024-09-11T00:25:23.940 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:23.937+0000 7f027b400640 1 -- 172.21.15.12:0/2536555485 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7f027403b460 con 0x7f02841049b0 2024-09-11T00:25:23.940 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:23.938+0000 7f0282000640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56081b2aace0 0x56081b2ad1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:25:23.940 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:23.938+0000 7f0283400640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56081b2a5970 0x56081b2a7d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:25:23.940 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:23.938+0000 7f0283400640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56081b2a5970 0x56081b2a7d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.12:54188/0 (socket says 172.21.15.12:54188) 2024-09-11T00:25:23.940 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:23.938+0000 7f0283400640 1 -- 172.21.15.12:0/162314616 learned_addr learned my addr 172.21.15.12:0/162314616 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:25:23.940 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:23.938+0000 7f0282a00640 1 --2- 172.21.15.12:0/162314616 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56081b2a82a0 0x56081b2aa690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:25:23.940 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:23.938+0000 7f0282000640 1 -- 172.21.15.12:0/162314616 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56081b2a5970 msgr2=0x56081b2a7d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:25:23.940 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:23.938+0000 7f0282000640 1 --2- 172.21.15.12:0/162314616 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56081b2a5970 0x56081b2a7d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:23.940 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:23.938+0000 7f0282000640 1 -- 172.21.15.12:0/162314616 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56081b2a82a0 msgr2=0x56081b2aa690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:25:23.940 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:23.938+0000 7f0282000640 1 --2- 172.21.15.12:0/162314616 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56081b2a82a0 0x56081b2aa690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:23.940 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:23.938+0000 7f0282000640 1 -- 172.21.15.12:0/162314616 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56081b1d4780 con 0x56081b2aace0 2024-09-11T00:25:23.941 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:23.938+0000 7f0283400640 1 --2- 172.21.15.12:0/162314616 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56081b2a5970 0x56081b2a7d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:25:23.944 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:23.938+0000 7f0282000640 1 --2- 172.21.15.12:0/162314616 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56081b2aace0 0x56081b2ad1a0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f0270009c20 tx=0x7f0270011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb0dbaec98a3232a server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:25:23.944 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:23.939+0000 7f026fe00640 1 -- 172.21.15.12:0/162314616 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f027001e070 con 0x56081b2aace0 2024-09-11T00:25:23.944 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:23.939+0000 7f026fe00640 1 -- 172.21.15.12:0/162314616 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0270009dd0 con 0x56081b2aace0 2024-09-11T00:25:23.944 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:23.939+0000 7f026fe00640 1 -- 172.21.15.12:0/162314616 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0270018710 con 0x56081b2aace0 2024-09-11T00:25:23.944 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:23.939+0000 7f027b400640 1 --2- 172.21.15.12:0/2536555485 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f02680b52e0 0x7f02680b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:25:23.945 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:23.939+0000 7f028cc54740 1 -- 172.21.15.12:0/162314616 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56081b2aace0 msgr2=0x56081b2ad1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:25:23.945 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:23.939+0000 7f028cc54740 1 --2- 172.21.15.12:0/162314616 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56081b2aace0 0x56081b2ad1a0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f0270009c20 tx=0x7f0270011810 comp rx=0 tx=0).stop 2024-09-11T00:25:23.945 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:23.939+0000 7f027b400640 1 -- 172.21.15.12:0/2536555485 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9208+0+0 (secure 0 0 0) 0x7f02741069e0 con 0x7f02841049b0 2024-09-11T00:25:23.945 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:23.939+0000 7f0283400640 1 --2- 172.21.15.12:0/2536555485 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f02680b52e0 0x7f02680b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:25:23.945 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:23.939+0000 7f028cc54740 1 -- 172.21.15.12:0/162314616 shutdown_connections 2024-09-11T00:25:23.945 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:23.939+0000 7f028cc54740 1 --2- 172.21.15.12:0/162314616 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56081b2aace0 0x56081b2ad1a0 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:23.945 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:23.939+0000 7f028cc54740 1 --2- 172.21.15.12:0/162314616 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56081b2a82a0 0x56081b2aa690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:23.945 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:23.939+0000 7f028cc54740 1 --2- 172.21.15.12:0/162314616 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56081b2a5970 0x56081b2a7d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:23.945 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:23.940+0000 7f028cc54740 1 -- 172.21.15.12:0/162314616 >> 172.21.15.12:0/162314616 conn(0x56081b2a0ac0 msgr2=0x56081b2a1850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:25:23.945 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:23.940+0000 7f028cc54740 1 -- 172.21.15.12:0/162314616 shutdown_connections 2024-09-11T00:25:23.946 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:23.940+0000 7f028cc54740 1 -- 172.21.15.12:0/162314616 wait complete. 2024-09-11T00:25:23.946 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:23.940+0000 7f0283400640 1 --2- 172.21.15.12:0/2536555485 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f02680b52e0 0x7f02680b77a0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f027c009fd0 tx=0x7f027c009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:25:23.946 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:23.940+0000 7f028cc54740 1 -- start start 2024-09-11T00:25:23.946 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:23.940+0000 7f028cc54740 1 Processor -- start 2024-09-11T00:25:23.946 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:23.941+0000 7f028cc54740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56081b2a5970 0x56081b2af070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:25:23.946 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:23.941+0000 7f028cc54740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56081b2a82a0 0x56081b2af710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:25:23.946 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:23.941+0000 7f028cc54740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56081b2aace0 0x56081b2b02e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:25:23.946 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:23.941+0000 7f028cc54740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x56081b13fab0 con 0x56081b2a5970 2024-09-11T00:25:23.946 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:23.941+0000 7f028cc54740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x56081b1ae570 con 0x56081b2aace0 2024-09-11T00:25:23.946 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:23.941+0000 7f028cc54740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x56081b2adcc0 con 0x56081b2a82a0 2024-09-11T00:25:23.947 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:23.941+0000 7f028cc54740 10 client.0 _ll_register_callbacks cb 0x56081b1bd100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:25:23.947 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:23.941+0000 7f0282a00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56081b2a5970 0x56081b2af070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:25:23.947 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:23.941+0000 7f0282a00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56081b2a5970 0x56081b2af070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:57402/0 (socket says 172.21.15.12:57402) 2024-09-11T00:25:23.947 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:23.941+0000 7f0282a00640 1 -- 172.21.15.12:0/4169589661 learned_addr learned my addr 172.21.15.12:0/4169589661 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:25:23.947 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:23.941+0000 7f0283400640 1 --2- 172.21.15.12:0/4169589661 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56081b2aace0 0x56081b2b02e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:25:23.947 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:23.941+0000 7f0282000640 1 --2- 172.21.15.12:0/4169589661 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56081b2a82a0 0x56081b2af710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:25:23.947 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:23.942+0000 7f0282a00640 1 -- 172.21.15.12:0/4169589661 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56081b2a82a0 msgr2=0x56081b2af710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:25:23.947 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:23.942+0000 7f0282a00640 1 --2- 172.21.15.12:0/4169589661 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56081b2a82a0 0x56081b2af710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:23.947 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:23.942+0000 7f0282a00640 1 -- 172.21.15.12:0/4169589661 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56081b2aace0 msgr2=0x56081b2b02e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:25:23.947 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:23.942+0000 7f0282a00640 1 --2- 172.21.15.12:0/4169589661 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56081b2aace0 0x56081b2b02e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:23.947 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:23.942+0000 7f0282a00640 1 -- 172.21.15.12:0/4169589661 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56081af69790 con 0x56081b2a5970 2024-09-11T00:25:23.947 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:23.942+0000 7f0282a00640 1 --2- 172.21.15.12:0/4169589661 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56081b2a5970 0x56081b2af070 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f02740ccb20 tx=0x7f0274005e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ab256efba0bb77a server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:25:23.948 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:23.942+0000 7f026cc00640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-09-11T00:25:23.948 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:23.942+0000 7f026cc00640 1 -- 172.21.15.12:0/4169589661 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0274049030 con 0x56081b2a5970 2024-09-11T00:25:23.948 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:23.942+0000 7f026cc00640 1 -- 172.21.15.12:0/4169589661 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9208+0+0 (secure 0 0 0) 0x7f02740d5e70 con 0x56081b2a5970 2024-09-11T00:25:23.948 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:23.942+0000 7f028cc54740 1 -- 172.21.15.12:0/4169589661 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0270009dd0 con 0x56081b2a5970 2024-09-11T00:25:23.948 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:23.942+0000 7f026cc00640 1 -- 172.21.15.12:0/4169589661 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=70) ==== 24+0+0 (secure 0 0 0) 0x7f02740ccbf0 con 0x56081b2a5970 2024-09-11T00:25:23.948 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:23.942+0000 7f028cc54740 10 client.17484 fetch_fsmap learned FSMap version 70 2024-09-11T00:25:23.948 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:23.942+0000 7f028cc54740 1 -- 172.21.15.12:0/4169589661 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=70,monmap=4+}) -- 0x56081afe2620 con 0x56081b2a5970 2024-09-11T00:25:23.948 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:23.943+0000 7f026cc00640 1 -- 172.21.15.12:0/4169589661 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0270009dd0 con 0x56081b2a5970 2024-09-11T00:25:23.949 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:23.943+0000 7f026cc00640 1 -- 172.21.15.12:0/4169589661 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 70) ==== 8394+0+0 (secure 0 0 0) 0x7f02740cc080 con 0x56081b2a5970 2024-09-11T00:25:23.951 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:23.943+0000 7f028cc54740 10 client.17484 fetch_fsmap finished waiting for FSMap version 70 2024-09-11T00:25:23.951 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:23.943+0000 7f028cc54740 10 client.17484 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T00:25:23.951 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:23.943+0000 7f028cc54740 20 client.17484 populate_metadata read hostname 'smithi012' 2024-09-11T00:25:23.951 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:23.943+0000 7f028cc54740 1 --2- 172.21.15.12:0/4169589661 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x56081b2ba8a0 0x56081b2bcd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:25:23.951 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:23.943+0000 7f028cc54740 4 client.17484 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T00:25:23.951 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:23.943+0000 7f028cc54740 1 -- 172.21.15.12:0/4169589661 --> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56081b21a600 con 0x56081b2ba8a0 2024-09-11T00:25:23.952 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:23.944+0000 7f0282000640 1 --2- 172.21.15.12:0/4169589661 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x56081b2ba8a0 0x56081b2bcd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:25:23.952 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:23.944+0000 7f0282000640 1 --2- 172.21.15.12:0/4169589661 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x56081b2ba8a0 0x56081b2bcd10 crc :-1 s=READY pgs=257 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=329401185b5891c3 server_cookie=a12dce6c87e1e362 in_seq=0 out_seq=0 2024-09-11T00:25:23.952 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:23.944+0000 7f026cc00640 10 client.17484 ms_handle_connect on v2:172.21.15.165:6838/501312984 2024-09-11T00:25:23.952 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:23.945+0000 7f026cc00640 1 -- 172.21.15.12:0/4169589661 <== mds.0 v2:172.21.15.165:6838/501312984 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x56081b21a600 con 0x56081b2ba8a0 2024-09-11T00:25:23.952 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:23.945+0000 7f026cc00640 10 client.17484 handle_command_reply: tid=0 2024-09-11T00:25:23.952 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:23.945+0000 7f028cc54740 1 client.17484 shutdown 2024-09-11T00:25:23.952 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:23.945+0000 7f028cc54740 1 -- 172.21.15.12:0/4169589661 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56081b2a5970 msgr2=0x56081b2af070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:25:23.952 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:23.945+0000 7f028cc54740 1 --2- 172.21.15.12:0/4169589661 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56081b2a5970 0x56081b2af070 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f02740ccb20 tx=0x7f0274005e10 comp rx=0 tx=0).stop 2024-09-11T00:25:23.952 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:23.945+0000 7f028cc54740 1 -- 172.21.15.12:0/4169589661 shutdown_connections 2024-09-11T00:25:23.952 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:23.945+0000 7f028cc54740 1 --2- 172.21.15.12:0/4169589661 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x56081b2ba8a0 0x56081b2bcd10 crc :-1 s=READY pgs=257 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:23.953 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:23.945+0000 7f028cc54740 1 --2- 172.21.15.12:0/4169589661 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56081b2aace0 0x56081b2b02e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:23.953 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:23.945+0000 7f028cc54740 1 --2- 172.21.15.12:0/4169589661 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56081b2a82a0 0x56081b2af710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:23.953 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:23.945+0000 7f028cc54740 1 --2- 172.21.15.12:0/4169589661 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56081b2a5970 0x56081b2af070 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:23.953 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:23.945+0000 7f028cc54740 1 -- 172.21.15.12:0/4169589661 >> 172.21.15.12:0/4169589661 conn(0x56081b2a0ac0 msgr2=0x56081b29fbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:25:23.953 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:23.946+0000 7f026cc00640 0 client.17484 ms_handle_reset on v2:172.21.15.165:6838/501312984 2024-09-11T00:25:23.953 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:23.946+0000 7f028cc54740 1 -- 172.21.15.12:0/4169589661 shutdown_connections 2024-09-11T00:25:23.953 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:23.946+0000 7f028cc54740 1 -- 172.21.15.12:0/4169589661 wait complete. 2024-09-11T00:25:23.954 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:23.946+0000 7f028cc54740 20 client.17484 trim_cache size 0 max 16384 2024-09-11T00:25:23.963 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:23.961+0000 7f028cc54740 1 Processor -- start 2024-09-11T00:25:23.963 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:23.961+0000 7f028cc54740 1 -- start start 2024-09-11T00:25:23.963 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:23.961+0000 7f028cc54740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56081b2a5970 0x56081b2aede0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:25:23.963 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:23.962+0000 7f028cc54740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56081b2a82a0 0x56081b2a4e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:25:23.964 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:23.962+0000 7f028cc54740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56081b2aace0 0x56081b2a5340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:25:23.964 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:23.962+0000 7f028cc54740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f02841ad750 con 0x56081b2a5970 2024-09-11T00:25:23.964 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:23.962+0000 7f028cc54740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f02841af320 con 0x56081b2a82a0 2024-09-11T00:25:23.964 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:23.962+0000 7f028cc54740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f02841ade20 con 0x56081b2aace0 2024-09-11T00:25:23.964 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:23.962+0000 7f0282a00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56081b2a5970 0x56081b2aede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:25:23.964 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:23.962+0000 7f0282a00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56081b2a5970 0x56081b2aede0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:57416/0 (socket says 172.21.15.12:57416) 2024-09-11T00:25:23.964 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:23.962+0000 7f0282a00640 1 -- 172.21.15.12:0/330112272 learned_addr learned my addr 172.21.15.12:0/330112272 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:25:23.964 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:23.962+0000 7f0282000640 1 --2- 172.21.15.12:0/330112272 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56081b2a82a0 0x56081b2a4e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:25:23.964 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:23.962+0000 7f0283400640 1 --2- 172.21.15.12:0/330112272 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56081b2aace0 0x56081b2a5340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:25:23.964 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:23.962+0000 7f0282a00640 1 -- 172.21.15.12:0/330112272 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56081b2aace0 msgr2=0x56081b2a5340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:25:23.964 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:23.962+0000 7f0282a00640 1 --2- 172.21.15.12:0/330112272 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56081b2aace0 0x56081b2a5340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:23.964 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:23.962+0000 7f0282a00640 1 -- 172.21.15.12:0/330112272 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56081b2a82a0 msgr2=0x56081b2a4e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:25:23.965 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:23.962+0000 7f0282a00640 1 --2- 172.21.15.12:0/330112272 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56081b2a82a0 0x56081b2a4e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:23.965 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:23.962+0000 7f0282a00640 1 -- 172.21.15.12:0/330112272 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56081b2aeb70 con 0x56081b2a5970 2024-09-11T00:25:23.967 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:23.962+0000 7f0282a00640 1 --2- 172.21.15.12:0/330112272 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56081b2a5970 0x56081b2aede0 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f0274000c00 tx=0x7f027402a420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3eb0387a4c010ff server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:25:23.967 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:23.963+0000 7f026f400640 1 -- 172.21.15.12:0/330112272 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0274049030 con 0x56081b2a5970 2024-09-11T00:25:23.967 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:23.963+0000 7f026f400640 1 -- 172.21.15.12:0/330112272 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f02740b81f0 con 0x56081b2a5970 2024-09-11T00:25:23.967 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:23.963+0000 7f026f400640 1 -- 172.21.15.12:0/330112272 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02740cc590 con 0x56081b2a5970 2024-09-11T00:25:23.968 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:23.963+0000 7f028cc54740 1 -- 172.21.15.12:0/330112272 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56081b2a5970 msgr2=0x56081b2aede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:25:23.968 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:23.963+0000 7f028cc54740 1 --2- 172.21.15.12:0/330112272 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56081b2a5970 0x56081b2aede0 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f0274000c00 tx=0x7f027402a420 comp rx=0 tx=0).stop 2024-09-11T00:25:23.968 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:23.964+0000 7f028cc54740 1 -- 172.21.15.12:0/330112272 shutdown_connections 2024-09-11T00:25:23.968 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:23.964+0000 7f028cc54740 1 --2- 172.21.15.12:0/330112272 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56081b2aace0 0x56081b2a5340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:23.968 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:23.964+0000 7f028cc54740 1 --2- 172.21.15.12:0/330112272 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56081b2a82a0 0x56081b2a4e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:23.968 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:23.964+0000 7f028cc54740 1 --2- 172.21.15.12:0/330112272 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56081b2a5970 0x56081b2aede0 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:23.968 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:23.964+0000 7f028cc54740 1 -- 172.21.15.12:0/330112272 >> 172.21.15.12:0/330112272 conn(0x56081b2a0ac0 msgr2=0x56081b2bc8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:25:23.968 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:23.964+0000 7f028cc54740 1 -- 172.21.15.12:0/330112272 shutdown_connections 2024-09-11T00:25:23.968 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:23.964+0000 7f028cc54740 1 -- 172.21.15.12:0/330112272 wait complete. 2024-09-11T00:25:23.968 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:23.964+0000 7f028cc54740 1 -- start start 2024-09-11T00:25:23.968 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:23.964+0000 7f028cc54740 1 Processor -- start 2024-09-11T00:25:23.968 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:23.965+0000 7f028cc54740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56081b2a5970 0x56081b2b6820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:25:23.969 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:23.965+0000 7f028cc54740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56081b2a82a0 0x56081b2b5520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:25:23.969 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:23.965+0000 7f028cc54740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56081b2aace0 0x56081b2b5a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:25:23.969 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:23.965+0000 7f028cc54740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f02841aec50 con 0x56081b2aace0 2024-09-11T00:25:23.969 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:23.965+0000 7f028cc54740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f02841ae550 con 0x56081b2a82a0 2024-09-11T00:25:23.969 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:23.965+0000 7f028cc54740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f02841ab730 con 0x56081b2a5970 2024-09-11T00:25:23.969 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:23.965+0000 7f028cc54740 10 client.0 _ll_register_callbacks cb 0x7f027410c220 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:25:23.969 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:23.965+0000 7f0282a00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56081b2a5970 0x56081b2b6820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:25:23.969 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:23.965+0000 7f0282000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56081b2a82a0 0x56081b2b5520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:25:23.969 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:23.965+0000 7f0282a00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56081b2a5970 0x56081b2b6820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.12:54220/0 (socket says 172.21.15.12:54220) 2024-09-11T00:25:23.969 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:23.965+0000 7f0282a00640 1 -- 172.21.15.12:0/1862014001 learned_addr learned my addr 172.21.15.12:0/1862014001 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:25:23.969 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:23.965+0000 7f0283400640 1 --2- 172.21.15.12:0/1862014001 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56081b2aace0 0x56081b2b5a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:25:23.969 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:23.965+0000 7f0282a00640 1 -- 172.21.15.12:0/1862014001 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56081b2a82a0 msgr2=0x56081b2b5520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:25:23.969 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:23.965+0000 7f0282a00640 1 --2- 172.21.15.12:0/1862014001 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56081b2a82a0 0x56081b2b5520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:23.970 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:23.965+0000 7f0282a00640 1 -- 172.21.15.12:0/1862014001 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56081b2aace0 msgr2=0x56081b2b5a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:25:23.970 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:23.965+0000 7f0282a00640 1 --2- 172.21.15.12:0/1862014001 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56081b2aace0 0x56081b2b5a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:23.970 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:23.965+0000 7f0282a00640 1 -- 172.21.15.12:0/1862014001 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56081b2b5fa0 con 0x56081b2a5970 2024-09-11T00:25:23.970 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:23.965+0000 7f0282000640 1 --2- 172.21.15.12:0/1862014001 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56081b2a82a0 0x56081b2b5520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:25:23.970 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:23.966+0000 7f0282a00640 1 --2- 172.21.15.12:0/1862014001 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56081b2a5970 0x56081b2b6820 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f0274005ab0 tx=0x7f02740ccf80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6bc8bfefe241022e server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:25:23.970 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:23.966+0000 7f026cc00640 10 client.0 ms_handle_connect on v2:172.21.15.178:3300/0 2024-09-11T00:25:23.970 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:23.966+0000 7f026cc00640 1 -- 172.21.15.12:0/1862014001 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02740f3070 con 0x56081b2a5970 2024-09-11T00:25:23.970 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:23.966+0000 7f028cc54740 1 -- 172.21.15.12:0/1862014001 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f02740b81f0 con 0x56081b2a5970 2024-09-11T00:25:23.970 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:23.966+0000 7f026cc00640 1 -- 172.21.15.12:0/1862014001 <== mon.1 v2:172.21.15.178:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9208+0+0 (secure 0 0 0) 0x7f02740d5e70 con 0x56081b2a5970 2024-09-11T00:25:23.970 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:23.966+0000 7f026cc00640 1 -- 172.21.15.12:0/1862014001 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=70) ==== 24+0+0 (secure 0 0 0) 0x7f02740e7050 con 0x56081b2a5970 2024-09-11T00:25:23.970 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:23.966+0000 7f028cc54740 10 client.27475 fetch_fsmap learned FSMap version 70 2024-09-11T00:25:23.970 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:23.966+0000 7f028cc54740 1 -- 172.21.15.12:0/1862014001 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,fsmap=70,monmap=4+}) -- 0x56081b2b5fa0 con 0x56081b2a5970 2024-09-11T00:25:23.971 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:23.966+0000 7f026cc00640 1 -- 172.21.15.12:0/1862014001 <== mon.1 v2:172.21.15.178:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f02740b81f0 con 0x56081b2a5970 2024-09-11T00:25:23.971 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:23.967+0000 7f026cc00640 1 -- 172.21.15.12:0/1862014001 <== mon.1 v2:172.21.15.178:3300/0 5 ==== fsmap(e 70) ==== 8394+0+0 (secure 0 0 0) 0x7f0274078ce0 con 0x56081b2a5970 2024-09-11T00:25:23.971 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:23.967+0000 7f028cc54740 10 client.27475 fetch_fsmap finished waiting for FSMap version 70 2024-09-11T00:25:23.972 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:23.967+0000 7f028cc54740 10 client.27475 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T00:25:23.972 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:23.967+0000 7f028cc54740 20 client.27475 populate_metadata read hostname 'smithi012' 2024-09-11T00:25:23.972 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:23.967+0000 7f028cc54740 1 --2- 172.21.15.12:0/1862014001 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x56081b327be0 0x56081b32a000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:25:23.972 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:23.967+0000 7f028cc54740 4 client.27475 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T00:25:23.972 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:23.967+0000 7f028cc54740 1 -- 172.21.15.12:0/1862014001 --> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56081b2160b0 con 0x56081b327be0 2024-09-11T00:25:23.972 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:23.967+0000 7f0282000640 1 --2- 172.21.15.12:0/1862014001 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x56081b327be0 0x56081b32a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:25:23.972 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:23.967+0000 7f0282000640 1 --2- 172.21.15.12:0/1862014001 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x56081b327be0 0x56081b32a000 crc :-1 s=READY pgs=258 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6780008d6b4a43d0 server_cookie=22bcb7549c5de2db in_seq=0 out_seq=0 2024-09-11T00:25:23.972 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:23.967+0000 7f026cc00640 10 client.27475 ms_handle_connect on v2:172.21.15.165:6838/501312984 2024-09-11T00:25:24.188 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:24.186+0000 7f026cc00640 1 -- 172.21.15.12:0/1862014001 <== mds.0 v2:172.21.15.165:6838/501312984 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56081b2160b0 con 0x56081b327be0 2024-09-11T00:25:24.188 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:24.186+0000 7f026cc00640 10 client.27475 handle_command_reply: tid=0 2024-09-11T00:25:24.188 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:24.186+0000 7f028cc54740 1 client.27475 shutdown 2024-09-11T00:25:24.189 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:24.187+0000 7f028cc54740 1 -- 172.21.15.12:0/1862014001 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56081b2a5970 msgr2=0x56081b2b6820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:25:24.189 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:24.187+0000 7f028cc54740 1 --2- 172.21.15.12:0/1862014001 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56081b2a5970 0x56081b2b6820 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f0274005ab0 tx=0x7f02740ccf80 comp rx=0 tx=0).stop 2024-09-11T00:25:24.189 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:24.187+0000 7f028cc54740 1 -- 172.21.15.12:0/1862014001 shutdown_connections 2024-09-11T00:25:24.189 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:24.187+0000 7f028cc54740 1 --2- 172.21.15.12:0/1862014001 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x56081b327be0 0x56081b32a000 crc :-1 s=READY pgs=258 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:24.189 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:24.187+0000 7f028cc54740 1 --2- 172.21.15.12:0/1862014001 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56081b2aace0 0x56081b2b5a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:24.189 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:24.187+0000 7f028cc54740 1 --2- 172.21.15.12:0/1862014001 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56081b2a82a0 0x56081b2b5520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:24.189 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:24.187+0000 7f028cc54740 1 --2- 172.21.15.12:0/1862014001 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56081b2a5970 0x56081b2b6820 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:24.189 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:24.187+0000 7f028cc54740 1 -- 172.21.15.12:0/1862014001 >> 172.21.15.12:0/1862014001 conn(0x56081b2a0ac0 msgr2=0x56081b2ab0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:25:24.189 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:24.187+0000 7f026cc00640 0 client.27475 ms_handle_reset on v2:172.21.15.165:6838/501312984 2024-09-11T00:25:24.189 INFO:tasks.workunit.client.0.smithi012.stdout: 10 98759 17.66 MB/sec execute 200 sec latency 3943.804 ms 2024-09-11T00:25:24.190 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:24.187+0000 7f028cc54740 1 -- 172.21.15.12:0/1862014001 shutdown_connections 2024-09-11T00:25:24.190 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:24.187+0000 7f028cc54740 1 -- 172.21.15.12:0/1862014001 wait complete. 2024-09-11T00:25:24.190 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:24.187+0000 7f028cc54740 20 client.27475 trim_cache size 0 max 16384 2024-09-11T00:25:24.190 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:{ 2024-09-11T00:25:24.191 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "message": "", 2024-09-11T00:25:24.191 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "return_code": 0 2024-09-11T00:25:24.191 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:} 2024-09-11T00:25:24.191 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:24.188+0000 7f0283e00640 1 -- 172.21.15.12:0/2536555485 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f02680b52e0 msgr2=0x7f02680b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:25:24.191 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:24.188+0000 7f0283e00640 1 --2- 172.21.15.12:0/2536555485 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f02680b52e0 0x7f02680b77a0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f027c009fd0 tx=0x7f027c009500 comp rx=0 tx=0).stop 2024-09-11T00:25:24.192 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:24.188+0000 7f0283e00640 1 -- 172.21.15.12:0/2536555485 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f02841049b0 msgr2=0x7f028419adc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:25:24.192 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:24.188+0000 7f0283e00640 1 --2- 172.21.15.12:0/2536555485 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f02841049b0 0x7f028419adc0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f0274031500 tx=0x7f0274003c10 comp rx=0 tx=0).stop 2024-09-11T00:25:24.192 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:24.188+0000 7f0283e00640 1 -- 172.21.15.12:0/2536555485 shutdown_connections 2024-09-11T00:25:24.192 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:24.188+0000 7f0283e00640 1 --2- 172.21.15.12:0/2536555485 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f02680b52e0 0x7f02680b77a0 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:24.192 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:24.188+0000 7f0283e00640 1 --2- 172.21.15.12:0/2536555485 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f02841064c0 0x7f028419c040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:24.192 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:24.188+0000 7f0283e00640 1 --2- 172.21.15.12:0/2536555485 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0284105b00 0x7f028419b410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:24.192 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:24.188+0000 7f0283e00640 1 --2- 172.21.15.12:0/2536555485 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f02841049b0 0x7f028419adc0 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:24.192 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:24.188+0000 7f0283e00640 1 -- 172.21.15.12:0/2536555485 >> 172.21.15.12:0/2536555485 conn(0x7f0284100250 msgr2=0x7f0284108ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:25:24.192 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:24.188+0000 7f0283e00640 1 -- 172.21.15.12:0/2536555485 shutdown_connections 2024-09-11T00:25:24.192 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:24.188+0000 7f0283e00640 1 -- 172.21.15.12:0/2536555485 wait complete. 2024-09-11T00:25:24.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:25:23 smithi165 ceph-mon[35356]: pgmap v948: 129 pgs: 129 active+clean; 582 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 4.1 MiB/s wr, 643 op/s 2024-09-11T00:25:24.283 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:25:23 smithi165 ceph-mon[35356]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T00:25:24.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:25:23 smithi012 ceph-mon[30333]: pgmap v948: 129 pgs: 129 active+clean; 582 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 4.1 MiB/s wr, 643 op/s 2024-09-11T00:25:24.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:25:23 smithi012 ceph-mon[30333]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T00:25:24.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:25:23 smithi178 ceph-mon[35919]: pgmap v948: 129 pgs: 129 active+clean; 582 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 4.1 MiB/s wr, 643 op/s 2024-09-11T00:25:24.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:25:23 smithi178 ceph-mon[35919]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T00:25:25.189 INFO:tasks.workunit.client.0.smithi012.stdout: 10 98759 17.57 MB/sec execute 201 sec latency 4943.898 ms 2024-09-11T00:25:26.189 INFO:tasks.workunit.client.0.smithi012.stdout: 10 98759 17.49 MB/sec execute 202 sec latency 5944.066 ms 2024-09-11T00:25:26.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:25:25 smithi165 ceph-mon[35356]: pgmap v949: 129 pgs: 129 active+clean; 582 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 3.8 MiB/s wr, 577 op/s 2024-09-11T00:25:26.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:25:25 smithi012 ceph-mon[30333]: pgmap v949: 129 pgs: 129 active+clean; 582 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 3.8 MiB/s wr, 577 op/s 2024-09-11T00:25:26.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:25:25 smithi178 ceph-mon[35919]: pgmap v949: 129 pgs: 129 active+clean; 582 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 3.8 MiB/s wr, 577 op/s 2024-09-11T00:25:27.189 INFO:tasks.workunit.client.0.smithi012.stdout: 10 98759 17.40 MB/sec execute 203 sec latency 6944.241 ms 2024-09-11T00:25:28.183 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:25:27 smithi012 ceph-mon[30333]: pgmap v950: 129 pgs: 129 active+clean; 579 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 3.9 MiB/s wr, 663 op/s 2024-09-11T00:25:28.193 INFO:tasks.workunit.client.0.smithi012.stdout: 10 98759 17.31 MB/sec execute 204 sec latency 7949.025 ms 2024-09-11T00:25:28.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:25:27 smithi165 ceph-mon[35356]: pgmap v950: 129 pgs: 129 active+clean; 579 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 3.9 MiB/s wr, 663 op/s 2024-09-11T00:25:28.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:25:27 smithi178 ceph-mon[35919]: pgmap v950: 129 pgs: 129 active+clean; 579 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 3.9 MiB/s wr, 663 op/s 2024-09-11T00:25:29.197 INFO:tasks.workunit.client.0.smithi012.stdout: 10 98759 17.23 MB/sec execute 205 sec latency 8952.045 ms 2024-09-11T00:25:30.197 INFO:tasks.workunit.client.0.smithi012.stdout: 10 98759 17.15 MB/sec execute 206 sec latency 9952.108 ms 2024-09-11T00:25:30.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:25:29 smithi165 ceph-mon[35356]: pgmap v951: 129 pgs: 129 active+clean; 579 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 2.4 MiB/s wr, 429 op/s 2024-09-11T00:25:30.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:25:29 smithi012 ceph-mon[30333]: pgmap v951: 129 pgs: 129 active+clean; 579 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 2.4 MiB/s wr, 429 op/s 2024-09-11T00:25:30.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:25:29 smithi178 ceph-mon[35919]: pgmap v951: 129 pgs: 129 active+clean; 579 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 2.4 MiB/s wr, 429 op/s 2024-09-11T00:25:31.096 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:25:30 smithi012 ceph-mon[30333]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T00:25:31.197 INFO:tasks.workunit.client.0.smithi012.stdout: 10 98759 17.06 MB/sec execute 207 sec latency 10952.162 ms 2024-09-11T00:25:31.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:25:30 smithi165 ceph-mon[35356]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T00:25:31.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:25:30 smithi178 ceph-mon[35919]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T00:25:32.042 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:25:31 smithi012 ceph-mon[30333]: pgmap v952: 129 pgs: 129 active+clean; 594 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 4.1 MiB/s wr, 424 op/s 2024-09-11T00:25:32.197 INFO:tasks.workunit.client.0.smithi012.stdout: 10 98759 16.98 MB/sec execute 208 sec latency 11952.263 ms 2024-09-11T00:25:32.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:25:31 smithi165 ceph-mon[35356]: pgmap v952: 129 pgs: 129 active+clean; 594 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 4.1 MiB/s wr, 424 op/s 2024-09-11T00:25:32.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:25:31 smithi178 ceph-mon[35919]: pgmap v952: 129 pgs: 129 active+clean; 594 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 4.1 MiB/s wr, 424 op/s 2024-09-11T00:25:33.201 INFO:tasks.workunit.client.0.smithi012.stdout: 10 98759 16.90 MB/sec execute 209 sec latency 12952.320 ms 2024-09-11T00:25:34.063 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:25:33 smithi012 ceph-mon[30333]: pgmap v953: 129 pgs: 129 active+clean; 598 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.3 MiB/s wr, 259 op/s 2024-09-11T00:25:34.197 INFO:tasks.workunit.client.0.smithi012.stdout: 10 98759 16.82 MB/sec execute 210 sec latency 13952.372 ms 2024-09-11T00:25:34.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:25:33 smithi165 ceph-mon[35356]: pgmap v953: 129 pgs: 129 active+clean; 598 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.3 MiB/s wr, 259 op/s 2024-09-11T00:25:34.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:25:33 smithi178 ceph-mon[35919]: pgmap v953: 129 pgs: 129 active+clean; 598 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.3 MiB/s wr, 259 op/s 2024-09-11T00:25:35.197 INFO:tasks.workunit.client.0.smithi012.stdout: 10 98759 16.74 MB/sec execute 211 sec latency 14952.429 ms 2024-09-11T00:25:36.045 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:25:35 smithi012 ceph-mon[30333]: pgmap v954: 129 pgs: 129 active+clean; 598 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 517 KiB/s rd, 2.9 MiB/s wr, 162 op/s 2024-09-11T00:25:36.197 INFO:tasks.workunit.client.0.smithi012.stdout: 10 98759 16.66 MB/sec execute 212 sec latency 15952.480 ms 2024-09-11T00:25:36.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:25:35 smithi165 ceph-mon[35356]: pgmap v954: 129 pgs: 129 active+clean; 598 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 517 KiB/s rd, 2.9 MiB/s wr, 162 op/s 2024-09-11T00:25:36.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:25:35 smithi178 ceph-mon[35919]: pgmap v954: 129 pgs: 129 active+clean; 598 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 517 KiB/s rd, 2.9 MiB/s wr, 162 op/s 2024-09-11T00:25:36.357 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid ff3ad74a-6fcf-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T00:25:36.852 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:25:37.197 INFO:tasks.workunit.client.0.smithi012.stdout: 10 98759 16.58 MB/sec execute 213 sec latency 16952.528 ms 2024-09-11T00:25:37.318 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:37.312+0000 7fb59d200640 1 -- 172.21.15.12:0/4245584815 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb598102210 msgr2=0x7fb598102b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:25:37.318 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:37.312+0000 7fb59d200640 1 --2- 172.21.15.12:0/4245584815 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb598102210 0x7fb598102b60 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fb590009be0 tx=0x7fb59002a6c0 comp rx=0 tx=0).stop 2024-09-11T00:25:37.318 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:37.312+0000 7fb59d200640 1 -- 172.21.15.12:0/4245584815 shutdown_connections 2024-09-11T00:25:37.318 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:37.312+0000 7fb59d200640 1 --2- 172.21.15.12:0/4245584815 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb598102210 0x7fb598102b60 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:37.318 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:37.312+0000 7fb59d200640 1 --2- 172.21.15.12:0/4245584815 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb598101850 0x7fb598101cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:37.318 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:37.312+0000 7fb59d200640 1 --2- 172.21.15.12:0/4245584815 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb598100990 0x7fb598100d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:37.319 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:37.312+0000 7fb59d200640 1 -- 172.21.15.12:0/4245584815 >> 172.21.15.12:0/4245584815 conn(0x7fb5980fc440 msgr2=0x7fb5980fe880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:25:37.319 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:37.312+0000 7fb59d200640 1 -- 172.21.15.12:0/4245584815 shutdown_connections 2024-09-11T00:25:37.319 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:37.312+0000 7fb59d200640 1 -- 172.21.15.12:0/4245584815 wait complete. 2024-09-11T00:25:37.320 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:37.317+0000 7fb59d200640 1 Processor -- start 2024-09-11T00:25:37.320 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:37.317+0000 7fb59d200640 1 -- start start 2024-09-11T00:25:37.320 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:37.317+0000 7fb59d200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb598100990 0x7fb59819af10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:25:37.320 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:37.317+0000 7fb59d200640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb598101850 0x7fb59819b560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:25:37.320 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:37.317+0000 7fb59d200640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb598102210 0x7fb59819c190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:25:37.320 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:37.317+0000 7fb59d200640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fb598110800 con 0x7fb598102210 2024-09-11T00:25:37.321 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:37.317+0000 7fb59d200640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fb598110680 con 0x7fb598100990 2024-09-11T00:25:37.321 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:37.317+0000 7fb59d200640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fb598110980 con 0x7fb598101850 2024-09-11T00:25:37.321 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:37.317+0000 7fb595e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb598100990 0x7fb59819af10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:25:37.321 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:37.317+0000 7fb595e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb598100990 0x7fb59819af10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.12:55642/0 (socket says 172.21.15.12:55642) 2024-09-11T00:25:37.321 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:37.317+0000 7fb595e00640 1 -- 172.21.15.12:0/3769216662 learned_addr learned my addr 172.21.15.12:0/3769216662 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:25:37.321 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:37.317+0000 7fb596800640 1 --2- 172.21.15.12:0/3769216662 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb598102210 0x7fb59819c190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:25:37.321 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:37.317+0000 7fb595e00640 1 -- 172.21.15.12:0/3769216662 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb598101850 msgr2=0x7fb59819b560 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-11T00:25:37.321 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:37.317+0000 7fb595e00640 1 --2- 172.21.15.12:0/3769216662 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb598101850 0x7fb59819b560 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:37.321 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:37.317+0000 7fb595e00640 1 -- 172.21.15.12:0/3769216662 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb598102210 msgr2=0x7fb59819c190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:25:37.321 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:37.317+0000 7fb595e00640 1 --2- 172.21.15.12:0/3769216662 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb598102210 0x7fb59819c190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:37.321 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:37.317+0000 7fb595e00640 1 -- 172.21.15.12:0/3769216662 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb59819ca50 con 0x7fb598100990 2024-09-11T00:25:37.322 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:37.317+0000 7fb596800640 1 --2- 172.21.15.12:0/3769216662 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb598102210 0x7fb59819c190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:25:37.322 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:37.319+0000 7fb595e00640 1 --2- 172.21.15.12:0/3769216662 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb598100990 0x7fb59819af10 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fb58800d770 tx=0x7fb58800dc30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:25:37.323 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:37.319+0000 7fb58ea00640 1 -- 172.21.15.12:0/3769216662 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb588017070 con 0x7fb598100990 2024-09-11T00:25:37.323 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:37.319+0000 7fb59d200640 1 -- 172.21.15.12:0/3769216662 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb59819cd40 con 0x7fb598100990 2024-09-11T00:25:37.323 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:37.319+0000 7fb58ea00640 1 -- 172.21.15.12:0/3769216662 <== mon.2 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb5880054d0 con 0x7fb598100990 2024-09-11T00:25:37.323 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:37.319+0000 7fb58ea00640 1 -- 172.21.15.12:0/3769216662 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb588004d50 con 0x7fb598100990 2024-09-11T00:25:37.323 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:37.319+0000 7fb59d200640 1 -- 172.21.15.12:0/3769216662 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5981a3410 con 0x7fb598100990 2024-09-11T00:25:37.331 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:37.330+0000 7fb59ffd4740 1 Processor -- start 2024-09-11T00:25:37.333 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:37.330+0000 7fb59ffd4740 1 -- start start 2024-09-11T00:25:37.333 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:37.330+0000 7fb58ea00640 1 -- 172.21.15.12:0/3769216662 <== mon.2 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7fb588009c30 con 0x7fb598100990 2024-09-11T00:25:37.333 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:37.330+0000 7fb59ffd4740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b828256790 0x55b828258b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:25:37.333 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:37.330+0000 7fb59ffd4740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b8282590c0 0x55b82825b4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:25:37.333 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:37.330+0000 7fb59ffd4740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b82825bb00 0x55b82825dfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:25:37.334 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:37.330+0000 7fb59ffd4740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55b8281c96b0 con 0x55b82825bb00 2024-09-11T00:25:37.334 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:37.330+0000 7fb59ffd4740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55b8281b8ae0 con 0x55b8282590c0 2024-09-11T00:25:37.334 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:37.330+0000 7fb59ffd4740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55b8281bc8f0 con 0x55b828256790 2024-09-11T00:25:37.334 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:37.331+0000 7fb596800640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b828256790 0x55b828258b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:25:37.334 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:37.331+0000 7fb596800640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b828256790 0x55b828258b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.12:32830/0 (socket says 172.21.15.12:32830) 2024-09-11T00:25:37.334 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:37.331+0000 7fb596800640 1 -- 172.21.15.12:0/587320507 learned_addr learned my addr 172.21.15.12:0/587320507 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:25:37.334 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:37.331+0000 7fb595e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b8282590c0 0x55b82825b4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:25:37.334 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:37.331+0000 7fb595400640 1 --2- 172.21.15.12:0/587320507 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b82825bb00 0x55b82825dfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:25:37.334 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:37.331+0000 7fb596800640 1 -- 172.21.15.12:0/587320507 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b8282590c0 msgr2=0x55b82825b4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:25:37.334 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:37.331+0000 7fb596800640 1 --2- 172.21.15.12:0/587320507 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b8282590c0 0x55b82825b4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:37.335 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:37.331+0000 7fb596800640 1 -- 172.21.15.12:0/587320507 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b82825bb00 msgr2=0x55b82825dfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:25:37.335 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:37.331+0000 7fb596800640 1 --2- 172.21.15.12:0/587320507 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b82825bb00 0x55b82825dfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:37.335 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:37.331+0000 7fb596800640 1 -- 172.21.15.12:0/587320507 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b828185780 con 0x55b828256790 2024-09-11T00:25:37.336 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:37.331+0000 7fb596800640 1 --2- 172.21.15.12:0/587320507 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b828256790 0x55b828258b80 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fb590004af0 tx=0x7fb590005e30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9153aadd52a11d24 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:25:37.336 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:37.331+0000 7fb58ea00640 1 --2- 172.21.15.12:0/3769216662 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fb57c0b5360 0x7fb57c0b7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:25:37.336 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:37.331+0000 7fb58ea00640 1 -- 172.21.15.12:0/3769216662 <== mon.2 v2:172.21.15.165:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9208+0+0 (secure 0 0 0) 0x7fb5880e5a30 con 0x7fb598100990 2024-09-11T00:25:37.336 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:37.331+0000 7fb595e00640 1 --2- 172.21.15.12:0/3769216662 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fb57c0b5360 0x7fb57c0b7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:25:37.336 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:37.331+0000 7fb595e00640 1 --2- 172.21.15.12:0/3769216662 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fb57c0b5360 0x7fb57c0b7820 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fb588009610 tx=0x7fb5880eb530 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:25:37.337 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:37.331+0000 7fb583400640 1 -- 172.21.15.12:0/587320507 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb590041030 con 0x55b828256790 2024-09-11T00:25:37.337 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:37.331+0000 7fb583400640 1 -- 172.21.15.12:0/587320507 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb590003970 con 0x55b828256790 2024-09-11T00:25:37.337 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:37.331+0000 7fb583400640 1 -- 172.21.15.12:0/587320507 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb590036040 con 0x55b828256790 2024-09-11T00:25:37.339 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:37.331+0000 7fb59ffd4740 1 -- 172.21.15.12:0/587320507 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b828256790 msgr2=0x55b828258b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:25:37.339 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:37.331+0000 7fb59ffd4740 1 --2- 172.21.15.12:0/587320507 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b828256790 0x55b828258b80 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fb590004af0 tx=0x7fb590005e30 comp rx=0 tx=0).stop 2024-09-11T00:25:37.339 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:37.331+0000 7fb59ffd4740 1 -- 172.21.15.12:0/587320507 shutdown_connections 2024-09-11T00:25:37.339 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:37.331+0000 7fb59ffd4740 1 --2- 172.21.15.12:0/587320507 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b82825bb00 0x55b82825dfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:37.339 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:37.331+0000 7fb59ffd4740 1 --2- 172.21.15.12:0/587320507 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b8282590c0 0x55b82825b4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:37.339 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:37.331+0000 7fb59ffd4740 1 --2- 172.21.15.12:0/587320507 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b828256790 0x55b828258b80 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:37.339 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:37.331+0000 7fb59ffd4740 1 -- 172.21.15.12:0/587320507 >> 172.21.15.12:0/587320507 conn(0x55b8282518e0 msgr2=0x55b828252670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:25:37.339 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:37.331+0000 7fb59ffd4740 1 -- 172.21.15.12:0/587320507 shutdown_connections 2024-09-11T00:25:37.340 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:37.331+0000 7fb59ffd4740 1 -- 172.21.15.12:0/587320507 wait complete. 2024-09-11T00:25:37.340 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:37.331+0000 7fb59ffd4740 1 -- start start 2024-09-11T00:25:37.340 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:37.331+0000 7fb59ffd4740 1 Processor -- start 2024-09-11T00:25:37.340 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:37.337+0000 7fb59ffd4740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b828256790 0x55b82825fdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:25:37.340 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:37.337+0000 7fb59ffd4740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b8282590c0 0x55b828260470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:25:37.340 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:37.337+0000 7fb59ffd4740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b82825bb00 0x55b828261040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:25:37.340 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:37.337+0000 7fb59ffd4740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55b8280f0ab0 con 0x55b828256790 2024-09-11T00:25:37.340 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:37.337+0000 7fb59ffd4740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55b82815f570 con 0x55b8282590c0 2024-09-11T00:25:37.340 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:37.337+0000 7fb59ffd4740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55b82825ea80 con 0x55b82825bb00 2024-09-11T00:25:37.340 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:37.337+0000 7fb59ffd4740 10 client.0 _ll_register_callbacks cb 0x55b82816e100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:25:37.340 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:37.337+0000 7fb596800640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b828256790 0x55b82825fdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:25:37.340 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:37.337+0000 7fb596800640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b828256790 0x55b82825fdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:58224/0 (socket says 172.21.15.12:58224) 2024-09-11T00:25:37.340 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:37.337+0000 7fb596800640 1 -- 172.21.15.12:0/789054319 learned_addr learned my addr 172.21.15.12:0/789054319 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:25:37.341 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:37.337+0000 7fb596800640 1 --2- 172.21.15.12:0/789054319 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b82825bb00 0x55b828261040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:25:37.342 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:37.337+0000 7fb596800640 1 -- 172.21.15.12:0/789054319 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b82825bb00 msgr2=0x55b828261040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:25:37.342 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:37.337+0000 7fb596800640 1 --2- 172.21.15.12:0/789054319 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b82825bb00 0x55b828261040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:37.343 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:37.337+0000 7fb596800640 1 -- 172.21.15.12:0/789054319 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b8282590c0 msgr2=0x55b828260470 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-11T00:25:37.343 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:37.337+0000 7fb596800640 1 --2- 172.21.15.12:0/789054319 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b8282590c0 0x55b828260470 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:37.343 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:37.337+0000 7fb596800640 1 -- 172.21.15.12:0/789054319 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b827f1a790 con 0x55b828256790 2024-09-11T00:25:37.343 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:37.337+0000 7fb596800640 1 --2- 172.21.15.12:0/789054319 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b828256790 0x55b82825fdd0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7fb59002a700 tx=0x7fb590004000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5f2b60e4b55da1c server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:25:37.343 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:37.337+0000 7fb577e00640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-09-11T00:25:37.343 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:37.337+0000 7fb577e00640 1 -- 172.21.15.12:0/789054319 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb59004d030 con 0x55b828256790 2024-09-11T00:25:37.343 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:37.337+0000 7fb577e00640 1 -- 172.21.15.12:0/789054319 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9208+0+0 (secure 0 0 0) 0x7fb590030d00 con 0x55b828256790 2024-09-11T00:25:37.343 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:37.337+0000 7fb59ffd4740 1 -- 172.21.15.12:0/789054319 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb590003970 con 0x55b828256790 2024-09-11T00:25:37.343 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:37.337+0000 7fb577e00640 1 -- 172.21.15.12:0/789054319 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=71) ==== 24+0+0 (secure 0 0 0) 0x7fb59005a050 con 0x55b828256790 2024-09-11T00:25:37.343 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:37.337+0000 7fb59ffd4740 10 client.17502 fetch_fsmap learned FSMap version 71 2024-09-11T00:25:37.343 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:37.337+0000 7fb59ffd4740 1 -- 172.21.15.12:0/789054319 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=71,monmap=4+}) -- 0x55b827f93620 con 0x55b828256790 2024-09-11T00:25:37.343 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:37.337+0000 7fb577e00640 1 -- 172.21.15.12:0/789054319 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb590003970 con 0x55b828256790 2024-09-11T00:25:37.343 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:37.337+0000 7fb577e00640 1 -- 172.21.15.12:0/789054319 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 71) ==== 8390+0+0 (secure 0 0 0) 0x7fb59003d840 con 0x55b828256790 2024-09-11T00:25:37.344 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:37.337+0000 7fb59ffd4740 10 client.17502 fetch_fsmap finished waiting for FSMap version 71 2024-09-11T00:25:37.344 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:37.337+0000 7fb59ffd4740 10 client.17502 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T00:25:37.344 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:37.337+0000 7fb59ffd4740 20 client.17502 populate_metadata read hostname 'smithi012' 2024-09-11T00:25:37.344 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:37.337+0000 7fb59ffd4740 1 --2- 172.21.15.12:0/789054319 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55b82826b270 0x55b82826d660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:25:37.344 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:37.337+0000 7fb59ffd4740 4 client.17502 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T00:25:37.344 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:37.337+0000 7fb59ffd4740 1 -- 172.21.15.12:0/789054319 --> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b8281cb600 con 0x55b82826b270 2024-09-11T00:25:37.344 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:37.337+0000 7fb595400640 1 --2- 172.21.15.12:0/789054319 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55b82826b270 0x55b82826d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:25:37.344 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:37.337+0000 7fb595400640 1 --2- 172.21.15.12:0/789054319 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55b82826b270 0x55b82826d660 crc :-1 s=READY pgs=259 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=493275eefaf0b5c4 server_cookie=7582bbacddd587ca in_seq=0 out_seq=0 2024-09-11T00:25:37.344 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:37.337+0000 7fb577e00640 10 client.17502 ms_handle_connect on v2:172.21.15.165:6838/501312984 2024-09-11T00:25:37.344 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:37.337+0000 7fb577e00640 1 -- 172.21.15.12:0/789054319 <== mds.0 v2:172.21.15.165:6838/501312984 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55b8281cb600 con 0x55b82826b270 2024-09-11T00:25:37.344 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:37.337+0000 7fb577e00640 10 client.17502 handle_command_reply: tid=0 2024-09-11T00:25:37.346 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:37.337+0000 7fb59ffd4740 1 client.17502 shutdown 2024-09-11T00:25:37.346 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:37.337+0000 7fb59ffd4740 1 -- 172.21.15.12:0/789054319 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b828256790 msgr2=0x55b82825fdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:25:37.346 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:37.337+0000 7fb59ffd4740 1 --2- 172.21.15.12:0/789054319 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b828256790 0x55b82825fdd0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7fb59002a700 tx=0x7fb590004000 comp rx=0 tx=0).stop 2024-09-11T00:25:37.346 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:37.337+0000 7fb59ffd4740 1 -- 172.21.15.12:0/789054319 shutdown_connections 2024-09-11T00:25:37.346 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:37.337+0000 7fb59ffd4740 1 --2- 172.21.15.12:0/789054319 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55b82826b270 0x55b82826d660 crc :-1 s=READY pgs=259 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:37.346 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:37.337+0000 7fb59ffd4740 1 --2- 172.21.15.12:0/789054319 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b82825bb00 0x55b828261040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:37.346 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:37.337+0000 7fb59ffd4740 1 --2- 172.21.15.12:0/789054319 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b8282590c0 0x55b828260470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:37.346 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:37.337+0000 7fb59ffd4740 1 --2- 172.21.15.12:0/789054319 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b828256790 0x55b82825fdd0 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:37.347 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:37.337+0000 7fb59ffd4740 1 -- 172.21.15.12:0/789054319 >> 172.21.15.12:0/789054319 conn(0x55b8282518e0 msgr2=0x55b828250a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:25:37.347 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:37.337+0000 7fb577e00640 0 client.17502 ms_handle_reset on v2:172.21.15.165:6838/501312984 2024-09-11T00:25:37.347 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:37.337+0000 7fb59ffd4740 1 -- 172.21.15.12:0/789054319 shutdown_connections 2024-09-11T00:25:37.347 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:37.337+0000 7fb59ffd4740 1 -- 172.21.15.12:0/789054319 wait complete. 2024-09-11T00:25:37.347 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:37.337+0000 7fb59ffd4740 20 client.17502 trim_cache size 0 max 16384 2024-09-11T00:25:37.367 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:37.361+0000 7fb59ffd4740 1 Processor -- start 2024-09-11T00:25:37.368 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:37.366+0000 7fb59ffd4740 1 -- start start 2024-09-11T00:25:37.368 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:37.366+0000 7fb59ffd4740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b828256790 0x55b828253710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:25:37.368 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:37.366+0000 7fb59ffd4740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b8282590c0 0x55b828253c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:25:37.368 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:37.366+0000 7fb59ffd4740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b82825bb00 0x55b828254190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:25:37.368 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:37.366+0000 7fb59ffd4740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fb5981ad7b0 con 0x55b8282590c0 2024-09-11T00:25:37.368 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:37.366+0000 7fb59ffd4740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fb5981af380 con 0x55b82825bb00 2024-09-11T00:25:37.368 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:37.366+0000 7fb59ffd4740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fb5981ade80 con 0x55b828256790 2024-09-11T00:25:37.369 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:37.366+0000 7fb596800640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b828256790 0x55b828253710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:25:37.369 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:37.366+0000 7fb596800640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b828256790 0x55b828253710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.12:32854/0 (socket says 172.21.15.12:32854) 2024-09-11T00:25:37.369 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:37.366+0000 7fb596800640 1 -- 172.21.15.12:0/294054143 learned_addr learned my addr 172.21.15.12:0/294054143 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:25:37.369 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:37.366+0000 7fb596800640 1 --2- 172.21.15.12:0/294054143 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b82825bb00 0x55b828254190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:25:37.370 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:37.366+0000 7fb595400640 1 --2- 172.21.15.12:0/294054143 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b8282590c0 0x55b828253c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:25:37.370 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:37.367+0000 7fb596800640 1 -- 172.21.15.12:0/294054143 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b82825bb00 msgr2=0x55b828254190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:25:37.370 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:37.367+0000 7fb596800640 1 --2- 172.21.15.12:0/294054143 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b82825bb00 0x55b828254190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:37.370 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:37.367+0000 7fb596800640 1 -- 172.21.15.12:0/294054143 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b8282590c0 msgr2=0x55b828253c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:25:37.370 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:37.367+0000 7fb596800640 1 --2- 172.21.15.12:0/294054143 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b8282590c0 0x55b828253c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:37.370 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:37.367+0000 7fb596800640 1 -- 172.21.15.12:0/294054143 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b82826d9d0 con 0x55b828256790 2024-09-11T00:25:37.370 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:37.367+0000 7fb595400640 1 --2- 172.21.15.12:0/294054143 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b8282590c0 0x55b828253c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T00:25:37.370 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:37.367+0000 7fb596800640 1 --2- 172.21.15.12:0/294054143 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b828256790 0x55b828253710 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fb590006010 tx=0x7fb59003f040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6201ab252f111478 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:25:37.370 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:37.367+0000 7fb582a00640 1 -- 172.21.15.12:0/294054143 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb59004d070 con 0x55b828256790 2024-09-11T00:25:37.371 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:37.367+0000 7fb582a00640 1 -- 172.21.15.12:0/294054143 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb590042040 con 0x55b828256790 2024-09-11T00:25:37.371 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:37.367+0000 7fb582a00640 1 -- 172.21.15.12:0/294054143 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb590036070 con 0x55b828256790 2024-09-11T00:25:37.371 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:37.367+0000 7fb59ffd4740 1 -- 172.21.15.12:0/294054143 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b828256790 msgr2=0x55b828253710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:25:37.371 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:37.367+0000 7fb59ffd4740 1 --2- 172.21.15.12:0/294054143 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b828256790 0x55b828253710 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fb590006010 tx=0x7fb59003f040 comp rx=0 tx=0).stop 2024-09-11T00:25:37.373 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:37.367+0000 7fb59ffd4740 1 -- 172.21.15.12:0/294054143 shutdown_connections 2024-09-11T00:25:37.373 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:37.367+0000 7fb59ffd4740 1 --2- 172.21.15.12:0/294054143 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b82825bb00 0x55b828254190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:37.373 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:37.367+0000 7fb59ffd4740 1 --2- 172.21.15.12:0/294054143 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b8282590c0 0x55b828253c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:37.374 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:37.367+0000 7fb59ffd4740 1 --2- 172.21.15.12:0/294054143 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b828256790 0x55b828253710 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:37.374 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:37.367+0000 7fb59ffd4740 1 -- 172.21.15.12:0/294054143 >> 172.21.15.12:0/294054143 conn(0x55b8282518e0 msgr2=0x55b82825bee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:25:37.374 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:37.373+0000 7fb59ffd4740 1 -- 172.21.15.12:0/294054143 shutdown_connections 2024-09-11T00:25:37.375 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:37.373+0000 7fb59ffd4740 1 -- 172.21.15.12:0/294054143 wait complete. 2024-09-11T00:25:37.375 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:37.373+0000 7fb59ffd4740 1 -- start start 2024-09-11T00:25:37.375 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:37.373+0000 7fb59ffd4740 1 Processor -- start 2024-09-11T00:25:37.376 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:37.373+0000 7fb59ffd4740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b828256790 0x55b828267100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:25:37.376 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:37.373+0000 7fb59ffd4740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b8282590c0 0x55b828267640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:25:37.376 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:37.373+0000 7fb59ffd4740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b82825bb00 0x55b8282659d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:25:37.376 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:37.373+0000 7fb59ffd4740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fb5981aecb0 con 0x55b828256790 2024-09-11T00:25:37.376 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:37.373+0000 7fb59ffd4740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fb5981ae5b0 con 0x55b8282590c0 2024-09-11T00:25:37.376 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:37.373+0000 7fb59ffd4740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fb5981ab790 con 0x55b82825bb00 2024-09-11T00:25:37.377 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:37.373+0000 7fb59ffd4740 10 client.0 _ll_register_callbacks cb 0x7fb5840093f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:25:37.377 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:37.373+0000 7fb595400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b8282590c0 0x55b828267640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:25:37.377 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:37.373+0000 7fb595400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b8282590c0 0x55b828267640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.12:55674/0 (socket says 172.21.15.12:55674) 2024-09-11T00:25:37.378 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:37.373+0000 7fb595400640 1 -- 172.21.15.12:0/3621243187 learned_addr learned my addr 172.21.15.12:0/3621243187 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:25:37.378 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:37.373+0000 7fb596800640 1 --2- 172.21.15.12:0/3621243187 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b828256790 0x55b828267100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:25:37.378 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:37.373+0000 7fb596800640 1 --2- 172.21.15.12:0/3621243187 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b82825bb00 0x55b8282659d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:25:37.378 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:37.373+0000 7fb595400640 1 -- 172.21.15.12:0/3621243187 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b82825bb00 msgr2=0x55b8282659d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:25:37.378 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:37.373+0000 7fb595400640 1 --2- 172.21.15.12:0/3621243187 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b82825bb00 0x55b8282659d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:37.378 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:37.373+0000 7fb595400640 1 -- 172.21.15.12:0/3621243187 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b828256790 msgr2=0x55b828267100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:25:37.378 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:37.373+0000 7fb595400640 1 --2- 172.21.15.12:0/3621243187 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b828256790 0x55b828267100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:37.378 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:37.373+0000 7fb595400640 1 -- 172.21.15.12:0/3621243187 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b8282613b0 con 0x55b8282590c0 2024-09-11T00:25:37.378 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:37.373+0000 7fb596800640 1 --2- 172.21.15.12:0/3621243187 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b828256790 0x55b828267100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T00:25:37.404 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:37.402+0000 7fb595400640 1 --2- 172.21.15.12:0/3621243187 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b8282590c0 0x55b828267640 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x55b82826de60 tx=0x7fb584010800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=27d3f23ef83aef5d server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:25:37.404 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:37.402+0000 7fb577e00640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-09-11T00:25:37.405 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:37.402+0000 7fb577e00640 1 -- 172.21.15.12:0/3621243187 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb584004250 con 0x55b8282590c0 2024-09-11T00:25:37.405 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:37.402+0000 7fb577e00640 1 -- 172.21.15.12:0/3621243187 <== mon.2 v2:172.21.15.165:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9208+0+0 (secure 0 0 0) 0x7fb5840043f0 con 0x55b8282590c0 2024-09-11T00:25:37.405 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:37.402+0000 7fb59ffd4740 1 -- 172.21.15.12:0/3621243187 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb590042040 con 0x55b8282590c0 2024-09-11T00:25:37.406 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:37.402+0000 7fb577e00640 1 -- 172.21.15.12:0/3621243187 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=71) ==== 24+0+0 (secure 0 0 0) 0x7fb5840035f0 con 0x55b8282590c0 2024-09-11T00:25:37.406 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:37.402+0000 7fb59ffd4740 10 client.27515 fetch_fsmap learned FSMap version 71 2024-09-11T00:25:37.406 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:37.402+0000 7fb59ffd4740 1 -- 172.21.15.12:0/3621243187 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=71,monmap=4+}) -- 0x55b8282e11c0 con 0x55b8282590c0 2024-09-11T00:25:37.406 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:37.402+0000 7fb577e00640 1 -- 172.21.15.12:0/3621243187 <== mon.2 v2:172.21.15.165:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb590042040 con 0x55b8282590c0 2024-09-11T00:25:37.406 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:37.402+0000 7fb577e00640 1 -- 172.21.15.12:0/3621243187 <== mon.2 v2:172.21.15.165:3300/0 5 ==== fsmap(e 71) ==== 8390+0+0 (secure 0 0 0) 0x7fb584020070 con 0x55b8282590c0 2024-09-11T00:25:37.406 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:37.402+0000 7fb59ffd4740 10 client.27515 fetch_fsmap finished waiting for FSMap version 71 2024-09-11T00:25:37.407 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:37.402+0000 7fb59ffd4740 10 client.27515 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T00:25:37.407 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:37.402+0000 7fb59ffd4740 20 client.27515 populate_metadata read hostname 'smithi012' 2024-09-11T00:25:37.407 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:37.402+0000 7fb59ffd4740 1 --2- 172.21.15.12:0/3621243187 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55b8282e1bd0 0x55b8282e3ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:25:37.407 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:37.402+0000 7fb59ffd4740 4 client.27515 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T00:25:37.407 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:37.402+0000 7fb59ffd4740 1 -- 172.21.15.12:0/3621243187 --> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55b8281c70b0 con 0x55b8282e1bd0 2024-09-11T00:25:37.407 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:37.402+0000 7fb596800640 1 --2- 172.21.15.12:0/3621243187 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55b8282e1bd0 0x55b8282e3ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:25:37.407 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:37.402+0000 7fb596800640 1 --2- 172.21.15.12:0/3621243187 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55b8282e1bd0 0x55b8282e3ff0 crc :-1 s=READY pgs=260 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2af89aa0a2fd8345 server_cookie=c52a2e1893cb3750 in_seq=0 out_seq=0 2024-09-11T00:25:37.407 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:37.402+0000 7fb577e00640 10 client.27515 ms_handle_connect on v2:172.21.15.165:6838/501312984 2024-09-11T00:25:37.455 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:37.450+0000 7fb577e00640 1 -- 172.21.15.12:0/3621243187 <== mds.0 v2:172.21.15.165:6838/501312984 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55b8281c70b0 con 0x55b8282e1bd0 2024-09-11T00:25:37.455 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:37.450+0000 7fb577e00640 10 client.27515 handle_command_reply: tid=0 2024-09-11T00:25:37.456 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:37.450+0000 7fb59ffd4740 1 client.27515 shutdown 2024-09-11T00:25:37.456 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:37.450+0000 7fb59ffd4740 1 -- 172.21.15.12:0/3621243187 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b8282590c0 msgr2=0x55b828267640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:25:37.456 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:37.450+0000 7fb59ffd4740 1 --2- 172.21.15.12:0/3621243187 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b8282590c0 0x55b828267640 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x55b82826de60 tx=0x7fb584010800 comp rx=0 tx=0).stop 2024-09-11T00:25:37.456 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:37.450+0000 7fb59ffd4740 1 -- 172.21.15.12:0/3621243187 shutdown_connections 2024-09-11T00:25:37.456 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:37.450+0000 7fb59ffd4740 1 --2- 172.21.15.12:0/3621243187 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55b8282e1bd0 0x55b8282e3ff0 crc :-1 s=READY pgs=260 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:37.456 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:37.450+0000 7fb59ffd4740 1 --2- 172.21.15.12:0/3621243187 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b82825bb00 0x55b8282659d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:37.456 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:37.450+0000 7fb59ffd4740 1 --2- 172.21.15.12:0/3621243187 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b8282590c0 0x55b828267640 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:37.456 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:37.450+0000 7fb59ffd4740 1 --2- 172.21.15.12:0/3621243187 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b828256790 0x55b828267100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:37.456 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:37.450+0000 7fb577e00640 0 client.27515 ms_handle_reset on v2:172.21.15.165:6838/501312984 2024-09-11T00:25:37.456 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:37.450+0000 7fb59ffd4740 1 -- 172.21.15.12:0/3621243187 >> 172.21.15.12:0/3621243187 conn(0x55b8282518e0 msgr2=0x55b828255fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:25:37.457 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:37.450+0000 7fb59ffd4740 1 -- 172.21.15.12:0/3621243187 shutdown_connections 2024-09-11T00:25:37.457 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:37.450+0000 7fb59ffd4740 1 -- 172.21.15.12:0/3621243187 wait complete. 2024-09-11T00:25:37.457 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:37.450+0000 7fb59ffd4740 20 client.27515 trim_cache size 0 max 16384 2024-09-11T00:25:37.457 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:{ 2024-09-11T00:25:37.457 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "message": "", 2024-09-11T00:25:37.458 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "return_code": 0 2024-09-11T00:25:37.458 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:} 2024-09-11T00:25:37.458 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:37.450+0000 7fb59d200640 1 -- 172.21.15.12:0/3769216662 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fb57c0b5360 msgr2=0x7fb57c0b7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:25:37.458 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:37.450+0000 7fb59d200640 1 --2- 172.21.15.12:0/3769216662 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fb57c0b5360 0x7fb57c0b7820 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fb588009610 tx=0x7fb5880eb530 comp rx=0 tx=0).stop 2024-09-11T00:25:37.459 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:37.450+0000 7fb59d200640 1 -- 172.21.15.12:0/3769216662 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb598100990 msgr2=0x7fb59819af10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:25:37.459 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:37.450+0000 7fb59d200640 1 --2- 172.21.15.12:0/3769216662 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb598100990 0x7fb59819af10 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fb58800d770 tx=0x7fb58800dc30 comp rx=0 tx=0).stop 2024-09-11T00:25:37.459 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:37.450+0000 7fb59d200640 1 -- 172.21.15.12:0/3769216662 shutdown_connections 2024-09-11T00:25:37.459 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:37.450+0000 7fb59d200640 1 --2- 172.21.15.12:0/3769216662 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fb57c0b5360 0x7fb57c0b7820 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:37.459 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:37.450+0000 7fb59d200640 1 --2- 172.21.15.12:0/3769216662 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb598102210 0x7fb59819c190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:37.459 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:37.450+0000 7fb59d200640 1 --2- 172.21.15.12:0/3769216662 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb598101850 0x7fb59819b560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:37.459 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:37.450+0000 7fb59d200640 1 --2- 172.21.15.12:0/3769216662 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb598100990 0x7fb59819af10 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:37.459 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:37.450+0000 7fb59d200640 1 -- 172.21.15.12:0/3769216662 >> 172.21.15.12:0/3769216662 conn(0x7fb5980fc440 msgr2=0x7fb5980fdf00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:25:37.459 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:37.450+0000 7fb59d200640 1 -- 172.21.15.12:0/3769216662 shutdown_connections 2024-09-11T00:25:37.459 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:37.450+0000 7fb59d200640 1 -- 172.21.15.12:0/3769216662 wait complete. 2024-09-11T00:25:38.197 INFO:tasks.workunit.client.0.smithi012.stdout: 10 98759 16.50 MB/sec execute 214 sec latency 17952.685 ms 2024-09-11T00:25:38.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:25:38 smithi165 ceph-mon[35356]: pgmap v955: 129 pgs: 129 active+clean; 613 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 639 KiB/s rd, 6.3 MiB/s wr, 273 op/s 2024-09-11T00:25:38.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:25:38 smithi012 ceph-mon[30333]: pgmap v955: 129 pgs: 129 active+clean; 613 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 639 KiB/s rd, 6.3 MiB/s wr, 273 op/s 2024-09-11T00:25:38.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:25:38 smithi178 ceph-mon[35919]: pgmap v955: 129 pgs: 129 active+clean; 613 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 639 KiB/s rd, 6.3 MiB/s wr, 273 op/s 2024-09-11T00:25:39.197 INFO:tasks.workunit.client.0.smithi012.stdout: 10 98759 16.43 MB/sec execute 215 sec latency 18952.747 ms 2024-09-11T00:25:40.197 INFO:tasks.workunit.client.0.smithi012.stdout: 10 98759 16.35 MB/sec execute 216 sec latency 19952.805 ms 2024-09-11T00:25:40.248 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:25:40 smithi012 ceph-mon[30333]: pgmap v956: 129 pgs: 129 active+clean; 614 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 181 KiB/s rd, 6.4 MiB/s wr, 188 op/s 2024-09-11T00:25:40.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:25:40 smithi165 ceph-mon[35356]: pgmap v956: 129 pgs: 129 active+clean; 614 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 181 KiB/s rd, 6.4 MiB/s wr, 188 op/s 2024-09-11T00:25:40.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:25:40 smithi178 ceph-mon[35919]: pgmap v956: 129 pgs: 129 active+clean; 614 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 181 KiB/s rd, 6.4 MiB/s wr, 188 op/s 2024-09-11T00:25:41.197 INFO:tasks.workunit.client.0.smithi012.stdout: 10 98759 16.28 MB/sec execute 217 sec latency 20952.859 ms 2024-09-11T00:25:42.201 INFO:tasks.workunit.client.0.smithi012.stdout: 10 98759 16.20 MB/sec execute 218 sec latency 21952.926 ms 2024-09-11T00:25:42.220 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:25:42 smithi012 ceph-mon[30333]: pgmap v957: 129 pgs: 129 active+clean; 585 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 245 KiB/s rd, 6.5 MiB/s wr, 239 op/s 2024-09-11T00:25:42.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:25:42 smithi178 ceph-mon[35919]: pgmap v957: 129 pgs: 129 active+clean; 585 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 245 KiB/s rd, 6.5 MiB/s wr, 239 op/s 2024-09-11T00:25:42.391 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:42.215+0000 7fa94d600640 1 -- 172.21.15.12:0/3020870369 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa948104fa0 msgr2=0x7fa948105380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:25:42.391 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:42.215+0000 7fa94d600640 1 --2- 172.21.15.12:0/3020870369 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa948104fa0 0x7fa948105380 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7fa938009900 tx=0x7fa93802a3f0 comp rx=0 tx=0).stop 2024-09-11T00:25:42.391 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:42.215+0000 7fa94d600640 1 -- 172.21.15.12:0/3020870369 shutdown_connections 2024-09-11T00:25:42.391 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:42.215+0000 7fa94d600640 1 --2- 172.21.15.12:0/3020870369 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa94810b330 0x7fa948111b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:42.391 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:42.215+0000 7fa94d600640 1 --2- 172.21.15.12:0/3020870369 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa94810a9b0 0x7fa94810adf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:42.391 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:42.215+0000 7fa94d600640 1 --2- 172.21.15.12:0/3020870369 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa948104fa0 0x7fa948105380 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:42.392 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:42.215+0000 7fa94d600640 1 -- 172.21.15.12:0/3020870369 >> 172.21.15.12:0/3020870369 conn(0x7fa948100a50 msgr2=0x7fa948102e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:25:42.392 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:42.215+0000 7fa94d600640 1 -- 172.21.15.12:0/3020870369 shutdown_connections 2024-09-11T00:25:42.392 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:42.215+0000 7fa94d600640 1 -- 172.21.15.12:0/3020870369 wait complete. 2024-09-11T00:25:42.392 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:42.215+0000 7fa94d600640 1 Processor -- start 2024-09-11T00:25:42.392 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:42.215+0000 7fa94d600640 1 -- start start 2024-09-11T00:25:42.392 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:42.215+0000 7fa94d600640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa948104fa0 0x7fa94819b4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:25:42.392 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:42.215+0000 7fa94d600640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa94810a9b0 0x7fa94819ba30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:25:42.392 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:42.215+0000 7fa94d600640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa94810b330 0x7fa9481a0dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:25:42.392 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:42.215+0000 7fa94d600640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fa948114f60 con 0x7fa94810b330 2024-09-11T00:25:42.392 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:42.215+0000 7fa94d600640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fa948114de0 con 0x7fa948104fa0 2024-09-11T00:25:42.393 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:42.215+0000 7fa94d600640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fa9481150e0 con 0x7fa94810a9b0 2024-09-11T00:25:42.393 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:42.215+0000 7fa947400640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa94810b330 0x7fa9481a0dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:25:42.393 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:42.215+0000 7fa946a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa948104fa0 0x7fa94819b4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:25:42.393 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:42.215+0000 7fa947400640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa94810b330 0x7fa9481a0dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:40972/0 (socket says 172.21.15.12:40972) 2024-09-11T00:25:42.393 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:42.215+0000 7fa947400640 1 -- 172.21.15.12:0/1931016018 learned_addr learned my addr 172.21.15.12:0/1931016018 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:25:42.393 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:42.215+0000 7fa946000640 1 --2- 172.21.15.12:0/1931016018 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa94810a9b0 0x7fa94819ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:25:42.393 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:42.215+0000 7fa947400640 1 -- 172.21.15.12:0/1931016018 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa94810a9b0 msgr2=0x7fa94819ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:25:42.393 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:42.215+0000 7fa947400640 1 --2- 172.21.15.12:0/1931016018 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa94810a9b0 0x7fa94819ba30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:42.393 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:42.215+0000 7fa947400640 1 -- 172.21.15.12:0/1931016018 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa948104fa0 msgr2=0x7fa94819b4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:25:42.393 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:42.215+0000 7fa947400640 1 --2- 172.21.15.12:0/1931016018 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa948104fa0 0x7fa94819b4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:42.393 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:42.215+0000 7fa947400640 1 -- 172.21.15.12:0/1931016018 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9481a15a0 con 0x7fa94810b330 2024-09-11T00:25:42.393 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:42.215+0000 7fa946a00640 1 --2- 172.21.15.12:0/1931016018 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa948104fa0 0x7fa94819b4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:25:42.394 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:42.215+0000 7fa947400640 1 --2- 172.21.15.12:0/1931016018 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa94810b330 0x7fa9481a0dd0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7fa94000eb10 tx=0x7fa94000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:25:42.394 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:42.215+0000 7fa93f400640 1 -- 172.21.15.12:0/1931016018 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa940004b20 con 0x7fa94810b330 2024-09-11T00:25:42.394 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:42.215+0000 7fa93f400640 1 -- 172.21.15.12:0/1931016018 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa940004cc0 con 0x7fa94810b330 2024-09-11T00:25:42.394 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:42.215+0000 7fa93f400640 1 -- 172.21.15.12:0/1931016018 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa940010670 con 0x7fa94810b330 2024-09-11T00:25:42.394 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:42.215+0000 7fa94d600640 1 -- 172.21.15.12:0/1931016018 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa9481a1890 con 0x7fa94810b330 2024-09-11T00:25:42.394 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:42.215+0000 7fa94d600640 1 -- 172.21.15.12:0/1931016018 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9481a7f50 con 0x7fa94810b330 2024-09-11T00:25:42.394 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:42.220+0000 7fa94d600640 1 -- 172.21.15.12:0/1931016018 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa948105380 con 0x7fa94810b330 2024-09-11T00:25:42.394 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:42.220+0000 7fa93f400640 1 -- 172.21.15.12:0/1931016018 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7fa940005020 con 0x7fa94810b330 2024-09-11T00:25:42.394 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:42.223+0000 7fa93f400640 1 --2- 172.21.15.12:0/1931016018 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fa92c0b52e0 0x7fa92c0b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:25:42.394 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:42.223+0000 7fa93f400640 1 -- 172.21.15.12:0/1931016018 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9208+0+0 (secure 0 0 0) 0x7fa9400e6450 con 0x7fa94810b330 2024-09-11T00:25:42.394 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:42.223+0000 7fa946a00640 1 --2- 172.21.15.12:0/1931016018 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fa92c0b52e0 0x7fa92c0b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:25:42.394 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:42.223+0000 7fa946a00640 1 --2- 172.21.15.12:0/1931016018 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fa92c0b52e0 0x7fa92c0b77a0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fa938031440 tx=0x7fa938005b50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:25:42.395 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:42.223+0000 7fa93f400640 1 -- 172.21.15.12:0/1931016018 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7fa9400acf40 con 0x7fa94810b330 2024-09-11T00:25:42.466 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:42.465+0000 7fa94d600640 1 -- 172.21.15.12:0/1931016018 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa94802e5c0 con 0x7fa94810b330 2024-09-11T00:25:42.467 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:42.465+0000 7fa93f400640 1 -- 172.21.15.12:0/1931016018 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 71 v71) ==== 94+0+11971 (secure 0 0 0) 0x7fa9400ac840 con 0x7fa94810b330 2024-09-11T00:25:42.468 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-11T00:25:42.468 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":71,"btime":"2024-09-11T00:25:29:916977+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14577,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6835/447171591","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":447171591},{"type":"v1","addr":"172.21.15.12:6835","nonce":447171591}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14589,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/901486040","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":901486040},{"type":"v1","addr":"172.21.15.12:6837","nonce":901486040}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":71,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T00:04:04.739881+0000","modified":"2024-09-11T00:25:29.554381+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24674,"mds_1":24643,"mds_2":24638,"mds_3":24637,"mds_4":24614},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14631":{"gid":14631,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.12:6841/3069919437","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6840","nonce":3069919437},{"type":"v1","addr":"172.21.15.12:6841","nonce":3069919437}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24562":{"gid":24562,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6833/1141636112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":1141636112},{"type":"v1","addr":"172.21.15.178:6833","nonce":1141636112}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24581":{"gid":24581,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6833/3111980970","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6832","nonce":3111980970},{"type":"v1","addr":"172.21.15.165:6833","nonce":3111980970}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24607":{"gid":24607,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6835/4084868287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":4084868287},{"type":"v1","addr":"172.21.15.178:6835","nonce":4084868287}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24614":{"gid":24614,"name":"e","rank":4,"incarnation":26,"state":"up:active","state_seq":21,"addr":"172.21.15.165:6835/2425395274","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":2425395274},{"type":"v1","addr":"172.21.15.165:6835","nonce":2425395274}]},"join_fscid":-1,"export_targets":[1],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24637":{"gid":24637,"name":"i","rank":3,"incarnation":24,"state":"up:active","state_seq":18,"addr":"172.21.15.178:6839/70628097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6838","nonce":70628097},{"type":"v1","addr":"172.21.15.178:6839","nonce":70628097}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24638":{"gid":24638,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":18,"addr":"172.21.15.165:6837/6215088","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6836","nonce":6215088},{"type":"v1","addr":"172.21.15.165:6837","nonce":6215088}]},"join_fscid":-1,"export_targets":[3],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24643":{"gid":24643,"name":"g","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.12:6839/3124093009","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6838","nonce":3124093009},{"type":"v1","addr":"172.21.15.12:6839","nonce":3124093009}]},"join_fscid":-1,"export_targets":[0,3],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24674":{"gid":24674,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.165:6839/501312984","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6838","nonce":501312984},{"type":"v1","addr":"172.21.15.165:6839","nonce":501312984}]},"join_fscid":-1,"export_targets":[1],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25672":{"gid":25672,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6837/308601741","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":308601741},{"type":"v1","addr":"172.21.15.178:6837","nonce":308601741}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24674,"qdb_cluster":[24614,24637,24638,24643,24674]},"id":1}]} 2024-09-11T00:25:42.468 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 71 2024-09-11T00:25:42.469 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:42.467+0000 7fa94d600640 1 -- 172.21.15.12:0/1931016018 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fa92c0b52e0 msgr2=0x7fa92c0b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:25:42.469 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:42.467+0000 7fa94d600640 1 --2- 172.21.15.12:0/1931016018 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fa92c0b52e0 0x7fa92c0b77a0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fa938031440 tx=0x7fa938005b50 comp rx=0 tx=0).stop 2024-09-11T00:25:42.469 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:42.468+0000 7fa94d600640 1 -- 172.21.15.12:0/1931016018 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa94810b330 msgr2=0x7fa9481a0dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:25:42.469 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:42.468+0000 7fa94d600640 1 --2- 172.21.15.12:0/1931016018 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa94810b330 0x7fa9481a0dd0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7fa94000eb10 tx=0x7fa94000c2d0 comp rx=0 tx=0).stop 2024-09-11T00:25:42.470 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:42.468+0000 7fa94d600640 1 -- 172.21.15.12:0/1931016018 shutdown_connections 2024-09-11T00:25:42.471 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:42.468+0000 7fa94d600640 1 --2- 172.21.15.12:0/1931016018 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fa92c0b52e0 0x7fa92c0b77a0 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:42.471 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:42.468+0000 7fa94d600640 1 --2- 172.21.15.12:0/1931016018 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa94810b330 0x7fa9481a0dd0 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:42.471 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:42.468+0000 7fa94d600640 1 --2- 172.21.15.12:0/1931016018 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa94810a9b0 0x7fa94819ba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:42.471 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:42.468+0000 7fa94d600640 1 --2- 172.21.15.12:0/1931016018 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa948104fa0 0x7fa94819b4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:42.471 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:42.468+0000 7fa94d600640 1 -- 172.21.15.12:0/1931016018 >> 172.21.15.12:0/1931016018 conn(0x7fa948100a50 msgr2=0x7fa948101390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:25:42.471 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:42.468+0000 7fa94d600640 1 -- 172.21.15.12:0/1931016018 shutdown_connections 2024-09-11T00:25:42.471 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:42.468+0000 7fa94d600640 1 -- 172.21.15.12:0/1931016018 wait complete. 2024-09-11T00:25:42.532 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:25:42 smithi165 ceph-mon[35356]: pgmap v957: 129 pgs: 129 active+clean; 585 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 245 KiB/s rd, 6.5 MiB/s wr, 239 op/s 2024-09-11T00:25:43.197 INFO:tasks.workunit.client.0.smithi012.stdout: 10 98759 16.13 MB/sec execute 219 sec latency 22952.984 ms 2024-09-11T00:25:43.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:25:43 smithi012 ceph-mon[30333]: from='client.? 172.21.15.12:0/1931016018' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T00:25:43.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:25:43 smithi178 ceph-mon[35919]: from='client.? 172.21.15.12:0/1931016018' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T00:25:43.532 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:25:43 smithi165 ceph-mon[35356]: from='client.? 172.21.15.12:0/1931016018' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T00:25:43.817 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24674 quiesce db --set-id 18bbb50e --release --await' 2024-09-11T00:25:43.817 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid ff3ad74a-6fcf-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24674 quiesce db --set-id 18bbb50e --release --await 2024-09-11T00:25:44.197 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:25:44.198 INFO:tasks.workunit.client.0.smithi012.stdout: 10 98759 16.05 MB/sec execute 220 sec latency 23953.046 ms 2024-09-11T00:25:44.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:25:44 smithi012 ceph-mon[30333]: pgmap v958: 129 pgs: 129 active+clean; 578 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 302 KiB/s rd, 4.6 MiB/s wr, 229 op/s 2024-09-11T00:25:44.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:25:44 smithi178 ceph-mon[35919]: pgmap v958: 129 pgs: 129 active+clean; 578 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 302 KiB/s rd, 4.6 MiB/s wr, 229 op/s 2024-09-11T00:25:44.533 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:25:44 smithi165 ceph-mon[35356]: pgmap v958: 129 pgs: 129 active+clean; 578 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 302 KiB/s rd, 4.6 MiB/s wr, 229 op/s 2024-09-11T00:25:44.755 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:44.753+0000 7f496fc00640 1 -- 172.21.15.12:0/455662282 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4968104fa0 msgr2=0x7f4968105380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:25:44.755 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:44.753+0000 7f496fc00640 1 --2- 172.21.15.12:0/455662282 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4968104fa0 0x7f4968105380 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f4958009ca0 tx=0x7f495802a8c0 comp rx=0 tx=0).stop 2024-09-11T00:25:44.755 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:44.753+0000 7f496fc00640 1 -- 172.21.15.12:0/455662282 shutdown_connections 2024-09-11T00:25:44.756 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:44.753+0000 7f496fc00640 1 --2- 172.21.15.12:0/455662282 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f496810b330 0x7f4968111b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:44.756 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:44.753+0000 7f496fc00640 1 --2- 172.21.15.12:0/455662282 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f496810a9b0 0x7f496810adf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:44.756 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:44.753+0000 7f496fc00640 1 --2- 172.21.15.12:0/455662282 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4968104fa0 0x7f4968105380 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:44.756 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:44.753+0000 7f496fc00640 1 -- 172.21.15.12:0/455662282 >> 172.21.15.12:0/455662282 conn(0x7f4968100a50 msgr2=0x7f4968102e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:25:44.756 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:44.753+0000 7f496fc00640 1 -- 172.21.15.12:0/455662282 shutdown_connections 2024-09-11T00:25:44.756 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:44.753+0000 7f496fc00640 1 -- 172.21.15.12:0/455662282 wait complete. 2024-09-11T00:25:44.758 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:44.754+0000 7f496fc00640 1 Processor -- start 2024-09-11T00:25:44.758 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:44.754+0000 7f496fc00640 1 -- start start 2024-09-11T00:25:44.758 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:44.754+0000 7f496fc00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4968104fa0 0x7f496819b4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:25:44.758 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:44.754+0000 7f496fc00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f496810a9b0 0x7f496819b9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:25:44.758 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:44.754+0000 7f496fc00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f496810b330 0x7f49681a0d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:25:44.759 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:44.754+0000 7f496fc00640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f4968114fb0 con 0x7f4968104fa0 2024-09-11T00:25:44.759 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:44.754+0000 7f496fc00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f4968114e30 con 0x7f496810a9b0 2024-09-11T00:25:44.759 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:44.754+0000 7f496fc00640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f4968115130 con 0x7f496810b330 2024-09-11T00:25:44.759 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:44.755+0000 7f496d200640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4968104fa0 0x7f496819b4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:25:44.759 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:44.755+0000 7f496d200640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4968104fa0 0x7f496819b4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:40980/0 (socket says 172.21.15.12:40980) 2024-09-11T00:25:44.759 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:44.755+0000 7f496d200640 1 -- 172.21.15.12:0/3537182108 learned_addr learned my addr 172.21.15.12:0/3537182108 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:25:44.759 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:44.755+0000 7f4967e00640 1 --2- 172.21.15.12:0/3537182108 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f496810a9b0 0x7f496819b9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:25:44.759 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:44.755+0000 7f496dc00640 1 --2- 172.21.15.12:0/3537182108 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f496810b330 0x7f49681a0d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:25:44.759 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:44.755+0000 7f496d200640 1 -- 172.21.15.12:0/3537182108 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f496810b330 msgr2=0x7f49681a0d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:25:44.759 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:44.755+0000 7f496d200640 1 --2- 172.21.15.12:0/3537182108 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f496810b330 0x7f49681a0d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:44.759 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:44.755+0000 7f496d200640 1 -- 172.21.15.12:0/3537182108 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f496810a9b0 msgr2=0x7f496819b9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:25:44.759 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:44.755+0000 7f496d200640 1 --2- 172.21.15.12:0/3537182108 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f496810a9b0 0x7f496819b9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:44.759 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:44.755+0000 7f496d200640 1 -- 172.21.15.12:0/3537182108 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49681a1550 con 0x7f4968104fa0 2024-09-11T00:25:44.760 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:44.756+0000 7f496d200640 1 --2- 172.21.15.12:0/3537182108 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4968104fa0 0x7f496819b4a0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f4958005e30 tx=0x7f4958031840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:25:44.760 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:44.756+0000 7f4965600640 1 -- 172.21.15.12:0/3537182108 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4958003b70 con 0x7f4968104fa0 2024-09-11T00:25:44.760 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:44.756+0000 7f4965600640 1 -- 172.21.15.12:0/3537182108 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4958003d10 con 0x7f4968104fa0 2024-09-11T00:25:44.760 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:44.756+0000 7f496fc00640 1 -- 172.21.15.12:0/3537182108 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49681a17e0 con 0x7f4968104fa0 2024-09-11T00:25:44.760 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:44.756+0000 7f4965600640 1 -- 172.21.15.12:0/3537182108 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f495802fa50 con 0x7f4968104fa0 2024-09-11T00:25:44.760 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:44.756+0000 7f496fc00640 1 -- 172.21.15.12:0/3537182108 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49681a7f00 con 0x7f4968104fa0 2024-09-11T00:25:44.761 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:44.759+0000 7f497261d740 1 Processor -- start 2024-09-11T00:25:44.761 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:44.759+0000 7f497261d740 1 -- start start 2024-09-11T00:25:44.761 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:44.759+0000 7f4965600640 1 -- 172.21.15.12:0/3537182108 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7f495802fbf0 con 0x7f4968104fa0 2024-09-11T00:25:44.762 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:44.760+0000 7f497261d740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55af87d3dff0 0x55af87d3e3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:25:44.762 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:44.760+0000 7f497261d740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55af87d3e910 0x55af87d4a360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:25:44.762 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:44.760+0000 7f497261d740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55af87d4a8a0 0x55af87d4cc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:25:44.762 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:44.760+0000 7f497261d740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55af87cb5290 con 0x55af87d3e910 2024-09-11T00:25:44.762 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:44.760+0000 7f497261d740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55af87ca46c0 con 0x55af87d3dff0 2024-09-11T00:25:44.762 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:44.760+0000 7f497261d740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55af87ca84d0 con 0x55af87d4a8a0 2024-09-11T00:25:44.763 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:44.760+0000 7f496d200640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55af87d3e910 0x55af87d4a360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:25:44.763 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:44.760+0000 7f496dc00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55af87d3dff0 0x55af87d3e3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:25:44.763 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:44.760+0000 7f496dc00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55af87d3dff0 0x55af87d3e3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.12:35378/0 (socket says 172.21.15.12:35378) 2024-09-11T00:25:44.764 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:44.760+0000 7f496dc00640 1 -- 172.21.15.12:0/1039287422 learned_addr learned my addr 172.21.15.12:0/1039287422 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:25:44.764 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:44.760+0000 7f4967e00640 1 --2- 172.21.15.12:0/1039287422 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55af87d4a8a0 0x55af87d4cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:25:44.764 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:44.761+0000 7f496dc00640 1 -- 172.21.15.12:0/1039287422 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55af87d4a8a0 msgr2=0x55af87d4cc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:25:44.764 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:44.761+0000 7f496dc00640 1 --2- 172.21.15.12:0/1039287422 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55af87d4a8a0 0x55af87d4cc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:44.764 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:44.761+0000 7f496dc00640 1 -- 172.21.15.12:0/1039287422 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55af87d3e910 msgr2=0x55af87d4a360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:25:44.764 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:44.761+0000 7f496dc00640 1 --2- 172.21.15.12:0/1039287422 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55af87d3e910 0x55af87d4a360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:44.764 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:44.761+0000 7f496dc00640 1 -- 172.21.15.12:0/1039287422 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55af87c31460 con 0x55af87d3dff0 2024-09-11T00:25:44.764 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:44.761+0000 7f4967e00640 1 --2- 172.21.15.12:0/1039287422 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55af87d4a8a0 0x55af87d4cc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T00:25:44.765 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:44.761+0000 7f496dc00640 1 --2- 172.21.15.12:0/1039287422 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55af87d3dff0 0x55af87d3e3d0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f496000b700 tx=0x7f496000bbc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=427efba5b37bb925 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:25:44.765 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:44.761+0000 7f4965600640 1 --2- 172.21.15.12:0/3537182108 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f494c0b5360 0x7f494c0b7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:25:44.765 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:44.761+0000 7f495e000640 1 -- 172.21.15.12:0/1039287422 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4960003dd0 con 0x55af87d3dff0 2024-09-11T00:25:44.765 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:44.761+0000 7f495e000640 1 -- 172.21.15.12:0/1039287422 <== mon.2 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4960015070 con 0x55af87d3dff0 2024-09-11T00:25:44.765 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:44.761+0000 7f495e000640 1 -- 172.21.15.12:0/1039287422 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49600106a0 con 0x55af87d3dff0 2024-09-11T00:25:44.765 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:44.761+0000 7f4965600640 1 -- 172.21.15.12:0/3537182108 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9208+0+0 (secure 0 0 0) 0x7f4958107110 con 0x7f4968104fa0 2024-09-11T00:25:44.767 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:44.762+0000 7f496d200640 1 --2- 172.21.15.12:0/3537182108 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f494c0b5360 0x7f494c0b7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:25:44.767 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:44.762+0000 7f497261d740 1 -- 172.21.15.12:0/1039287422 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55af87d3dff0 msgr2=0x55af87d3e3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:25:44.767 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:44.762+0000 7f497261d740 1 --2- 172.21.15.12:0/1039287422 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55af87d3dff0 0x55af87d3e3d0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f496000b700 tx=0x7f496000bbc0 comp rx=0 tx=0).stop 2024-09-11T00:25:44.767 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:44.762+0000 7f497261d740 1 -- 172.21.15.12:0/1039287422 shutdown_connections 2024-09-11T00:25:44.767 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:44.762+0000 7f497261d740 1 --2- 172.21.15.12:0/1039287422 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55af87d4a8a0 0x55af87d4cc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:44.767 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:44.762+0000 7f497261d740 1 --2- 172.21.15.12:0/1039287422 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55af87d3e910 0x55af87d4a360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:44.767 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:44.762+0000 7f497261d740 1 --2- 172.21.15.12:0/1039287422 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55af87d3dff0 0x55af87d3e3d0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:44.767 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:44.762+0000 7f497261d740 1 -- 172.21.15.12:0/1039287422 >> 172.21.15.12:0/1039287422 conn(0x55af87d3be20 msgr2=0x55af87d3cfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:25:44.767 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:44.762+0000 7f496d200640 1 --2- 172.21.15.12:0/3537182108 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f494c0b5360 0x7f494c0b7820 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f49580cd430 tx=0x7f495810ab50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:25:44.768 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:44.762+0000 7f497261d740 1 -- 172.21.15.12:0/1039287422 shutdown_connections 2024-09-11T00:25:44.768 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:44.762+0000 7f497261d740 1 -- 172.21.15.12:0/1039287422 wait complete. 2024-09-11T00:25:44.768 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:44.763+0000 7f497261d740 1 -- start start 2024-09-11T00:25:44.768 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:44.763+0000 7f497261d740 1 Processor -- start 2024-09-11T00:25:44.768 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:44.764+0000 7f497261d740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55af87d3dff0 0x55af87d47870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:25:44.768 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:44.764+0000 7f497261d740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55af87d3e910 0x55af87d47db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:25:44.768 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:44.764+0000 7f497261d740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55af87d4a8a0 0x55af87d482f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:25:44.768 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:44.764+0000 7f497261d740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55af87b05bb0 con 0x55af87d3e910 2024-09-11T00:25:44.768 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:44.764+0000 7f497261d740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55af87c3f6c0 con 0x55af87d3dff0 2024-09-11T00:25:44.769 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:44.764+0000 7f497261d740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55af87d4d550 con 0x55af87d4a8a0 2024-09-11T00:25:44.769 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:44.764+0000 7f497261d740 10 client.0 _ll_register_callbacks cb 0x55af87c5a490 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:25:44.769 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:44.764+0000 7f4967e00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55af87d3e910 0x55af87d47db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:25:44.769 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:44.764+0000 7f496dc00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55af87d3dff0 0x55af87d47870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:25:44.769 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:44.764+0000 7f496dc00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55af87d4a8a0 0x55af87d482f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:25:44.769 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:44.764+0000 7f496dc00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55af87d4a8a0 0x55af87d482f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.12:59254/0 (socket says 172.21.15.12:59254) 2024-09-11T00:25:44.769 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:44.764+0000 7f496dc00640 1 -- 172.21.15.12:0/3811957323 learned_addr learned my addr 172.21.15.12:0/3811957323 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:25:44.769 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:44.764+0000 7f4967e00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55af87d3e910 0x55af87d47db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:40998/0 (socket says 172.21.15.12:40998) 2024-09-11T00:25:44.769 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:44.765+0000 7f496dc00640 1 -- 172.21.15.12:0/3811957323 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55af87d3dff0 msgr2=0x55af87d47870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:25:44.769 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:44.765+0000 7f496dc00640 1 --2- 172.21.15.12:0/3811957323 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55af87d3dff0 0x55af87d47870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:44.769 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:44.765+0000 7f496dc00640 1 -- 172.21.15.12:0/3811957323 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55af87d3e910 msgr2=0x55af87d47db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:25:44.769 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:44.765+0000 7f496dc00640 1 --2- 172.21.15.12:0/3811957323 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55af87d3e910 0x55af87d47db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:44.769 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:44.765+0000 7f496dc00640 1 -- 172.21.15.12:0/3811957323 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55af87a02b00 con 0x55af87d4a8a0 2024-09-11T00:25:44.770 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:44.765+0000 7f496dc00640 1 --2- 172.21.15.12:0/3811957323 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55af87d4a8a0 0x55af87d482f0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f496000b6d0 tx=0x7f4960000bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6a3638dff2c8c5f1 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:25:44.770 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:44.765+0000 7f4952a00640 10 client.0 ms_handle_connect on v2:172.21.15.178:3300/0 2024-09-11T00:25:44.772 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:44.765+0000 7f4952a00640 1 -- 172.21.15.12:0/3811957323 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4960023040 con 0x55af87d4a8a0 2024-09-11T00:25:44.772 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:44.765+0000 7f497261d740 1 -- 172.21.15.12:0/3811957323 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4960015070 con 0x55af87d4a8a0 2024-09-11T00:25:44.772 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:44.765+0000 7f4952a00640 1 -- 172.21.15.12:0/3811957323 <== mon.1 v2:172.21.15.178:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9208+0+0 (secure 0 0 0) 0x7f49600118f0 con 0x55af87d4a8a0 2024-09-11T00:25:44.772 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:44.766+0000 7f4952a00640 1 -- 172.21.15.12:0/3811957323 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=71) ==== 24+0+0 (secure 0 0 0) 0x7f4960033050 con 0x55af87d4a8a0 2024-09-11T00:25:44.772 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:44.766+0000 7f497261d740 10 client.27508 fetch_fsmap learned FSMap version 71 2024-09-11T00:25:44.772 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:44.766+0000 7f497261d740 1 -- 172.21.15.12:0/3811957323 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,fsmap=71,monmap=4+}) -- 0x55af87a79960 con 0x55af87d4a8a0 2024-09-11T00:25:44.772 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:44.766+0000 7f4952a00640 1 -- 172.21.15.12:0/3811957323 <== mon.1 v2:172.21.15.178:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4960015070 con 0x55af87d4a8a0 2024-09-11T00:25:44.772 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:44.766+0000 7f4952a00640 1 -- 172.21.15.12:0/3811957323 <== mon.1 v2:172.21.15.178:3300/0 5 ==== fsmap(e 71) ==== 8390+0+0 (secure 0 0 0) 0x7f4960004c10 con 0x55af87d4a8a0 2024-09-11T00:25:44.772 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:44.767+0000 7f497261d740 10 client.27508 fetch_fsmap finished waiting for FSMap version 71 2024-09-11T00:25:44.773 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:44.767+0000 7f497261d740 10 client.27508 resolve_mds: validated gid 24674 aka daemon mds.b 2024-09-11T00:25:44.773 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:44.767+0000 7f497261d740 20 client.27508 populate_metadata read hostname 'smithi012' 2024-09-11T00:25:44.773 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:44.767+0000 7f497261d740 1 --2- 172.21.15.12:0/3811957323 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55af87d5a380 0x55af87d5c770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:25:44.773 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:44.767+0000 7f497261d740 4 client.27508 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T00:25:44.773 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:44.767+0000 7f497261d740 1 -- 172.21.15.12:0/3811957323 --> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55af87cb71e0 con 0x55af87d5a380 2024-09-11T00:25:44.773 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:44.767+0000 7f4967e00640 1 --2- 172.21.15.12:0/3811957323 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55af87d5a380 0x55af87d5c770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:25:44.773 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:44.768+0000 7f4967e00640 1 --2- 172.21.15.12:0/3811957323 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55af87d5a380 0x55af87d5c770 crc :-1 s=READY pgs=261 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dd2066f2000f0b43 server_cookie=d3cdf81f1f84ffb5 in_seq=0 out_seq=0 2024-09-11T00:25:44.773 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:44.768+0000 7f4952a00640 10 client.27508 ms_handle_connect on v2:172.21.15.165:6838/501312984 2024-09-11T00:25:44.773 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:44.768+0000 7f4952a00640 1 -- 172.21.15.12:0/3811957323 <== mds.0 v2:172.21.15.165:6838/501312984 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55af87cb71e0 con 0x55af87d5a380 2024-09-11T00:25:44.773 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:44.768+0000 7f4952a00640 10 client.27508 handle_command_reply: tid=0 2024-09-11T00:25:44.773 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:44.768+0000 7f497261d740 1 client.27508 shutdown 2024-09-11T00:25:44.773 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:44.769+0000 7f497261d740 1 -- 172.21.15.12:0/3811957323 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55af87d4a8a0 msgr2=0x55af87d482f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:25:44.773 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:44.769+0000 7f497261d740 1 --2- 172.21.15.12:0/3811957323 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55af87d4a8a0 0x55af87d482f0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f496000b6d0 tx=0x7f4960000bc0 comp rx=0 tx=0).stop 2024-09-11T00:25:44.773 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:44.769+0000 7f497261d740 1 -- 172.21.15.12:0/3811957323 shutdown_connections 2024-09-11T00:25:44.774 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:44.769+0000 7f497261d740 1 --2- 172.21.15.12:0/3811957323 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55af87d5a380 0x55af87d5c770 crc :-1 s=READY pgs=261 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:44.774 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:44.769+0000 7f497261d740 1 --2- 172.21.15.12:0/3811957323 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55af87d4a8a0 0x55af87d482f0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:44.774 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:44.769+0000 7f497261d740 1 --2- 172.21.15.12:0/3811957323 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55af87d3e910 0x55af87d47db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:44.774 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:44.769+0000 7f4952a00640 0 client.27508 ms_handle_reset on v2:172.21.15.165:6838/501312984 2024-09-11T00:25:44.774 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:44.769+0000 7f497261d740 1 --2- 172.21.15.12:0/3811957323 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55af87d3dff0 0x55af87d47870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:44.774 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:44.769+0000 7f497261d740 1 -- 172.21.15.12:0/3811957323 >> 172.21.15.12:0/3811957323 conn(0x55af87d3be20 msgr2=0x55af87d419c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:25:44.774 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:44.769+0000 7f497261d740 1 -- 172.21.15.12:0/3811957323 shutdown_connections 2024-09-11T00:25:44.774 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:44.769+0000 7f497261d740 1 -- 172.21.15.12:0/3811957323 wait complete. 2024-09-11T00:25:44.774 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:44.769+0000 7f497261d740 20 client.27508 trim_cache size 0 max 16384 2024-09-11T00:25:44.789 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:44.787+0000 7f497261d740 1 Processor -- start 2024-09-11T00:25:44.790 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:44.787+0000 7f497261d740 1 -- start start 2024-09-11T00:25:44.790 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:44.788+0000 7f497261d740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55af87d3dff0 0x55af87d45300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:25:44.790 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:44.788+0000 7f497261d740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55af87d3e910 0x55af87d61f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:25:44.790 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:44.788+0000 7f497261d740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55af87d4a8a0 0x55af87d62440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:25:44.790 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:44.788+0000 7f497261d740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f49681b1dc0 con 0x55af87d3dff0 2024-09-11T00:25:44.790 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:44.788+0000 7f497261d740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f49681b3990 con 0x55af87d3e910 2024-09-11T00:25:44.790 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:44.788+0000 7f497261d740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f49681b2490 con 0x55af87d4a8a0 2024-09-11T00:25:44.790 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:44.788+0000 7f496dc00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55af87d3dff0 0x55af87d45300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:25:44.790 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:44.788+0000 7f4967e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55af87d3e910 0x55af87d61f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:25:44.790 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:44.788+0000 7f496dc00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55af87d3dff0 0x55af87d45300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:41012/0 (socket says 172.21.15.12:41012) 2024-09-11T00:25:44.790 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:44.788+0000 7f496dc00640 1 -- 172.21.15.12:0/2612864951 learned_addr learned my addr 172.21.15.12:0/2612864951 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:25:44.791 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:44.788+0000 7f496dc00640 1 -- 172.21.15.12:0/2612864951 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55af87d4a8a0 msgr2=0x55af87d62440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:25:44.791 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:44.788+0000 7f496dc00640 1 --2- 172.21.15.12:0/2612864951 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55af87d4a8a0 0x55af87d62440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:44.791 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:44.788+0000 7f496dc00640 1 -- 172.21.15.12:0/2612864951 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55af87d3e910 msgr2=0x55af87d61f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:25:44.791 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:44.788+0000 7f496dc00640 1 --2- 172.21.15.12:0/2612864951 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55af87d3e910 0x55af87d61f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:44.792 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:44.788+0000 7f496dc00640 1 -- 172.21.15.12:0/2612864951 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55af87d3c8a0 con 0x55af87d3dff0 2024-09-11T00:25:44.792 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:44.788+0000 7f4967e00640 1 --2- 172.21.15.12:0/2612864951 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55af87d3e910 0x55af87d61f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:25:44.792 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:44.789+0000 7f496dc00640 1 --2- 172.21.15.12:0/2612864951 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55af87d3dff0 0x55af87d45300 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f496000bc00 tx=0x7f4960011d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b8362bb8f82d759 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:25:44.816 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:44.789+0000 7f495d600640 1 -- 172.21.15.12:0/2612864951 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4960003c50 con 0x55af87d3dff0 2024-09-11T00:25:44.816 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:44.789+0000 7f495d600640 1 -- 172.21.15.12:0/2612864951 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4960015070 con 0x55af87d3dff0 2024-09-11T00:25:44.816 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:44.789+0000 7f495d600640 1 -- 172.21.15.12:0/2612864951 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49600054b0 con 0x55af87d3dff0 2024-09-11T00:25:44.817 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:44.789+0000 7f497261d740 1 -- 172.21.15.12:0/2612864951 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55af87d3dff0 msgr2=0x55af87d45300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:25:44.817 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:44.789+0000 7f497261d740 1 --2- 172.21.15.12:0/2612864951 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55af87d3dff0 0x55af87d45300 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f496000bc00 tx=0x7f4960011d60 comp rx=0 tx=0).stop 2024-09-11T00:25:44.817 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:44.790+0000 7f497261d740 1 -- 172.21.15.12:0/2612864951 shutdown_connections 2024-09-11T00:25:44.817 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:44.790+0000 7f497261d740 1 --2- 172.21.15.12:0/2612864951 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55af87d4a8a0 0x55af87d62440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:44.817 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:44.790+0000 7f497261d740 1 --2- 172.21.15.12:0/2612864951 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55af87d3e910 0x55af87d61f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:44.817 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:44.790+0000 7f497261d740 1 --2- 172.21.15.12:0/2612864951 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55af87d3dff0 0x55af87d45300 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:44.817 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:44.790+0000 7f497261d740 1 -- 172.21.15.12:0/2612864951 >> 172.21.15.12:0/2612864951 conn(0x55af87d3be20 msgr2=0x55af87d496c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:25:44.817 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:44.790+0000 7f497261d740 1 -- 172.21.15.12:0/2612864951 shutdown_connections 2024-09-11T00:25:44.818 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:44.790+0000 7f497261d740 1 -- 172.21.15.12:0/2612864951 wait complete. 2024-09-11T00:25:44.818 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:44.790+0000 7f497261d740 1 -- start start 2024-09-11T00:25:44.818 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:44.790+0000 7f497261d740 1 Processor -- start 2024-09-11T00:25:44.818 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:44.791+0000 7f497261d740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55af87d3dff0 0x55af87d52b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:25:44.818 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:44.791+0000 7f497261d740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55af87d3e910 0x55af87d53080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:25:44.818 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:44.791+0000 7f497261d740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55af87d4a8a0 0x55af87d4d850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:25:44.818 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:44.791+0000 7f497261d740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f49681b32c0 con 0x55af87d4a8a0 2024-09-11T00:25:44.818 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:44.791+0000 7f497261d740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f49681b2bc0 con 0x55af87d3e910 2024-09-11T00:25:44.818 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:44.791+0000 7f497261d740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f49681afda0 con 0x55af87d3dff0 2024-09-11T00:25:44.818 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:44.791+0000 7f497261d740 10 client.0 _ll_register_callbacks cb 0x7f495810aa20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:25:44.818 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:44.791+0000 7f4967e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55af87d3e910 0x55af87d53080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:25:44.819 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:44.791+0000 7f496dc00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55af87d3dff0 0x55af87d52b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:25:44.819 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:44.791+0000 7f4967e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55af87d3e910 0x55af87d53080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.12:35404/0 (socket says 172.21.15.12:35404) 2024-09-11T00:25:44.819 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:44.791+0000 7f4967e00640 1 -- 172.21.15.12:0/1901916187 learned_addr learned my addr 172.21.15.12:0/1901916187 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:25:44.819 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:44.791+0000 7f496dc00640 1 --2- 172.21.15.12:0/1901916187 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55af87d4a8a0 0x55af87d4d850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:25:44.819 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:44.791+0000 7f4967e00640 1 -- 172.21.15.12:0/1901916187 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55af87d3dff0 msgr2=0x55af87d52b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:25:44.819 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:44.791+0000 7f4967e00640 1 --2- 172.21.15.12:0/1901916187 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55af87d3dff0 0x55af87d52b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:44.819 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:44.792+0000 7f4967e00640 1 -- 172.21.15.12:0/1901916187 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55af87d4a8a0 msgr2=0x55af87d4d850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:25:44.820 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:44.792+0000 7f4967e00640 1 --2- 172.21.15.12:0/1901916187 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55af87d4a8a0 0x55af87d4d850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:44.820 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:44.792+0000 7f4967e00640 1 -- 172.21.15.12:0/1901916187 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55af87d4e080 con 0x55af87d3e910 2024-09-11T00:25:44.820 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:44.792+0000 7f496dc00640 1 --2- 172.21.15.12:0/1901916187 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55af87d3dff0 0x55af87d52b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T00:25:44.820 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:44.792+0000 7f4967e00640 1 --2- 172.21.15.12:0/1901916187 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55af87d3e910 0x55af87d53080 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f4954004700 tx=0x7f49540108c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bd673b3c1329e476 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:25:44.820 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:44.792+0000 7f4952a00640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-09-11T00:25:44.820 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:44.792+0000 7f4952a00640 1 -- 172.21.15.12:0/1901916187 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4954004310 con 0x55af87d3e910 2024-09-11T00:25:44.820 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:44.792+0000 7f497261d740 1 -- 172.21.15.12:0/1901916187 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4960015070 con 0x55af87d3e910 2024-09-11T00:25:44.820 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:44.792+0000 7f4952a00640 1 -- 172.21.15.12:0/1901916187 <== mon.2 v2:172.21.15.165:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9208+0+0 (secure 0 0 0) 0x7f49540035f0 con 0x55af87d3e910 2024-09-11T00:25:44.820 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:44.793+0000 7f4952a00640 1 -- 172.21.15.12:0/1901916187 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=71) ==== 24+0+0 (secure 0 0 0) 0x7f4954031050 con 0x55af87d3e910 2024-09-11T00:25:44.820 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:44.793+0000 7f497261d740 10 client.27539 fetch_fsmap learned FSMap version 71 2024-09-11T00:25:44.820 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:44.793+0000 7f497261d740 1 -- 172.21.15.12:0/1901916187 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=71,monmap=4+}) -- 0x55af87dd0580 con 0x55af87d3e910 2024-09-11T00:25:44.821 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:44.793+0000 7f4952a00640 1 -- 172.21.15.12:0/1901916187 <== mon.2 v2:172.21.15.165:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4960015070 con 0x55af87d3e910 2024-09-11T00:25:44.821 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:44.793+0000 7f4952a00640 1 -- 172.21.15.12:0/1901916187 <== mon.2 v2:172.21.15.165:3300/0 5 ==== fsmap(e 71) ==== 8390+0+0 (secure 0 0 0) 0x7f495402c030 con 0x55af87d3e910 2024-09-11T00:25:44.821 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:44.793+0000 7f497261d740 10 client.27539 fetch_fsmap finished waiting for FSMap version 71 2024-09-11T00:25:44.821 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:44.793+0000 7f497261d740 10 client.27539 resolve_mds: validated gid 24674 aka daemon mds.b 2024-09-11T00:25:44.821 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:44.793+0000 7f497261d740 20 client.27539 populate_metadata read hostname 'smithi012' 2024-09-11T00:25:44.821 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:44.793+0000 7f497261d740 1 --2- 172.21.15.12:0/1901916187 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55af87dd0d60 0x55af87dd3180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:25:44.821 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:44.793+0000 7f497261d740 4 client.27539 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "18bbb50e", "release": true, "await": true}] 2024-09-11T00:25:44.821 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:44.793+0000 7f497261d740 1 -- 172.21.15.12:0/1901916187 --> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] -- command(tid 0: {"prefix": "quiesce db", "set_id": "18bbb50e", "release": true, "await": true}) -- 0x55af87cb2c90 con 0x55af87dd0d60 2024-09-11T00:25:44.821 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:44.794+0000 7f496dc00640 1 --2- 172.21.15.12:0/1901916187 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55af87dd0d60 0x55af87dd3180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:25:44.821 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:44.794+0000 7f496dc00640 1 --2- 172.21.15.12:0/1901916187 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55af87dd0d60 0x55af87dd3180 crc :-1 s=READY pgs=262 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f3c20168bcd0ef86 server_cookie=6e15ef198bb2e309 in_seq=0 out_seq=0 2024-09-11T00:25:44.821 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:44.794+0000 7f4952a00640 10 client.27539 ms_handle_connect on v2:172.21.15.165:6838/501312984 2024-09-11T00:25:44.829 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:44.825+0000 7f4952a00640 1 -- 172.21.15.12:0/1901916187 <== mds.0 v2:172.21.15.165:6838/501312984 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55af87cb2c90 con 0x55af87dd0d60 2024-09-11T00:25:44.829 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:44.825+0000 7f4952a00640 10 client.27539 handle_command_reply: tid=0 2024-09-11T00:25:44.829 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:44.825+0000 7f497261d740 1 client.27539 shutdown 2024-09-11T00:25:44.829 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:44.825+0000 7f497261d740 1 -- 172.21.15.12:0/1901916187 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55af87d3e910 msgr2=0x55af87d53080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:25:44.829 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:44.825+0000 7f497261d740 1 --2- 172.21.15.12:0/1901916187 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55af87d3e910 0x55af87d53080 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f4954004700 tx=0x7f49540108c0 comp rx=0 tx=0).stop 2024-09-11T00:25:44.829 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:44.826+0000 7f497261d740 1 -- 172.21.15.12:0/1901916187 shutdown_connections 2024-09-11T00:25:44.829 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:44.826+0000 7f497261d740 1 --2- 172.21.15.12:0/1901916187 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55af87dd0d60 0x55af87dd3180 crc :-1 s=READY pgs=262 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:44.829 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:44.826+0000 7f497261d740 1 --2- 172.21.15.12:0/1901916187 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55af87d4a8a0 0x55af87d4d850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:44.830 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:44.826+0000 7f497261d740 1 --2- 172.21.15.12:0/1901916187 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55af87d3e910 0x55af87d53080 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:44.830 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:44.826+0000 7f497261d740 1 --2- 172.21.15.12:0/1901916187 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55af87d3dff0 0x55af87d52b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:44.830 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:44.826+0000 7f4952a00640 0 client.27539 ms_handle_reset on v2:172.21.15.165:6838/501312984 2024-09-11T00:25:44.830 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:44.826+0000 7f497261d740 1 -- 172.21.15.12:0/1901916187 >> 172.21.15.12:0/1901916187 conn(0x55af87d3be20 msgr2=0x55af87d477e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:25:44.830 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:44.826+0000 7f497261d740 1 -- 172.21.15.12:0/1901916187 shutdown_connections 2024-09-11T00:25:44.830 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:44.826+0000 7f497261d740 1 -- 172.21.15.12:0/1901916187 wait complete. 2024-09-11T00:25:44.830 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:44.826+0000 7f497261d740 20 client.27539 trim_cache size 0 max 16384 2024-09-11T00:25:44.838 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-09-11T00:25:44.838 INFO:teuthology.orchestra.run.smithi012.stdout: "epoch": 71, 2024-09-11T00:25:44.838 INFO:teuthology.orchestra.run.smithi012.stdout: "leader": 24674, 2024-09-11T00:25:44.838 INFO:teuthology.orchestra.run.smithi012.stdout: "set_version": 56, 2024-09-11T00:25:44.838 INFO:teuthology.orchestra.run.smithi012.stdout: "sets": { 2024-09-11T00:25:44.839 INFO:teuthology.orchestra.run.smithi012.stdout: "18bbb50e": { 2024-09-11T00:25:44.839 INFO:teuthology.orchestra.run.smithi012.stdout: "version": 56, 2024-09-11T00:25:44.839 INFO:teuthology.orchestra.run.smithi012.stdout: "age_ref": 0.0, 2024-09-11T00:25:44.839 INFO:teuthology.orchestra.run.smithi012.stdout: "state": { 2024-09-11T00:25:44.839 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "RELEASED", 2024-09-11T00:25:44.839 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 0.0 2024-09-11T00:25:44.839 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:25:44.839 INFO:teuthology.orchestra.run.smithi012.stdout: "timeout": 90.0, 2024-09-11T00:25:44.840 INFO:teuthology.orchestra.run.smithi012.stdout: "expiration": 141.0, 2024-09-11T00:25:44.840 INFO:teuthology.orchestra.run.smithi012.stdout: "members": { 2024-09-11T00:25:44.840 INFO:teuthology.orchestra.run.smithi012.stdout: "file:/": { 2024-09-11T00:25:44.840 INFO:teuthology.orchestra.run.smithi012.stdout: "excluded": false, 2024-09-11T00:25:44.840 INFO:teuthology.orchestra.run.smithi012.stdout: "state": { 2024-09-11T00:25:44.840 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "RELEASED", 2024-09-11T00:25:44.840 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 0.0 2024-09-11T00:25:44.840 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:25:44.840 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:25:44.840 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:25:44.840 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:25:44.840 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:25:44.840 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-09-11T00:25:44.841 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:44.827+0000 7f496fc00640 1 -- 172.21.15.12:0/3537182108 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f494c0b5360 msgr2=0x7f494c0b7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:25:44.841 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:44.827+0000 7f496fc00640 1 --2- 172.21.15.12:0/3537182108 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f494c0b5360 0x7f494c0b7820 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f49580cd430 tx=0x7f495810ab50 comp rx=0 tx=0).stop 2024-09-11T00:25:44.841 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:44.827+0000 7f496fc00640 1 -- 172.21.15.12:0/3537182108 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4968104fa0 msgr2=0x7f496819b4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:25:44.841 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:44.827+0000 7f496fc00640 1 --2- 172.21.15.12:0/3537182108 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4968104fa0 0x7f496819b4a0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f4958005e30 tx=0x7f4958031840 comp rx=0 tx=0).stop 2024-09-11T00:25:44.842 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:44.827+0000 7f496fc00640 1 -- 172.21.15.12:0/3537182108 shutdown_connections 2024-09-11T00:25:44.842 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:44.827+0000 7f496fc00640 1 --2- 172.21.15.12:0/3537182108 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f494c0b5360 0x7f494c0b7820 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:44.842 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:44.827+0000 7f496fc00640 1 --2- 172.21.15.12:0/3537182108 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f496810b330 0x7f49681a0d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:44.842 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:44.827+0000 7f496fc00640 1 --2- 172.21.15.12:0/3537182108 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f496810a9b0 0x7f496819b9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:44.842 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:44.827+0000 7f496fc00640 1 --2- 172.21.15.12:0/3537182108 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4968104fa0 0x7f496819b4a0 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:44.842 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:44.827+0000 7f496fc00640 1 -- 172.21.15.12:0/3537182108 >> 172.21.15.12:0/3537182108 conn(0x7f4968100a50 msgr2=0x7f4968102e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:25:44.842 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:44.827+0000 7f496fc00640 1 -- 172.21.15.12:0/3537182108 shutdown_connections 2024-09-11T00:25:44.842 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:44.827+0000 7f496fc00640 1 -- 172.21.15.12:0/3537182108 wait complete. 2024-09-11T00:25:45.204 INFO:tasks.workunit.client.0.smithi012.stdout: 10 98764 15.98 MB/sec execute 221 sec latency 24891.253 ms 2024-09-11T00:25:45.823 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '18bbb50e', total seconds elapsed: 30.4 2024-09-11T00:25:45.823 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 21.0 2024-09-11T00:25:45.823 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid ff3ad74a-6fcf-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T00:25:46.204 INFO:tasks.workunit.client.0.smithi012.stdout: 10 99587 16.17 MB/sec execute 222 sec latency 25254.254 ms 2024-09-11T00:25:46.314 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:25:46 smithi012 ceph-mon[30333]: pgmap v959: 129 pgs: 129 active+clean; 577 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 297 KiB/s rd, 4.0 MiB/s wr, 213 op/s 2024-09-11T00:25:46.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:25:46 smithi178 ceph-mon[35919]: pgmap v959: 129 pgs: 129 active+clean; 577 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 297 KiB/s rd, 4.0 MiB/s wr, 213 op/s 2024-09-11T00:25:46.508 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:25:46.532 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:25:46 smithi165 ceph-mon[35356]: pgmap v959: 129 pgs: 129 active+clean; 577 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 297 KiB/s rd, 4.0 MiB/s wr, 213 op/s 2024-09-11T00:25:47.204 INFO:tasks.workunit.client.0.smithi012.stdout: 10 100458 16.28 MB/sec execute 223 sec latency 64.149 ms 2024-09-11T00:25:48.105 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:25:48 smithi012 ceph-mon[30333]: pgmap v960: 129 pgs: 129 active+clean; 575 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 686 KiB/s rd, 4.0 MiB/s wr, 242 op/s 2024-09-11T00:25:48.205 INFO:tasks.workunit.client.0.smithi012.stdout: 10 101200 16.39 MB/sec execute 224 sec latency 100.210 ms 2024-09-11T00:25:48.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:25:48 smithi178 ceph-mon[35919]: pgmap v960: 129 pgs: 129 active+clean; 575 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 686 KiB/s rd, 4.0 MiB/s wr, 242 op/s 2024-09-11T00:25:48.334 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:25:48 smithi165 ceph-mon[35356]: pgmap v960: 129 pgs: 129 active+clean; 575 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 686 KiB/s rd, 4.0 MiB/s wr, 242 op/s 2024-09-11T00:25:48.441 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:48.439+0000 7fc16ea00640 1 -- 172.21.15.12:0/3621486882 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc1681093d0 msgr2=0x7fc1681097b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:25:48.442 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:48.439+0000 7fc16ea00640 1 --2- 172.21.15.12:0/3621486882 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc1681093d0 0x7fc1681097b0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fc158009900 tx=0x7fc15802a3f0 comp rx=0 tx=0).stop 2024-09-11T00:25:48.442 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:48.440+0000 7fc16ea00640 1 -- 172.21.15.12:0/3621486882 shutdown_connections 2024-09-11T00:25:48.442 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:48.440+0000 7fc16ea00640 1 --2- 172.21.15.12:0/3621486882 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc16810f760 0x7fc168115f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:48.442 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:48.440+0000 7fc16ea00640 1 --2- 172.21.15.12:0/3621486882 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc16810ede0 0x7fc16810f220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:48.442 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:48.440+0000 7fc16ea00640 1 --2- 172.21.15.12:0/3621486882 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc1681093d0 0x7fc1681097b0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:48.442 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:48.440+0000 7fc16ea00640 1 -- 172.21.15.12:0/3621486882 >> 172.21.15.12:0/3621486882 conn(0x7fc168078a40 msgr2=0x7fc16807ae60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:25:48.442 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:48.441+0000 7fc16ea00640 1 -- 172.21.15.12:0/3621486882 shutdown_connections 2024-09-11T00:25:48.443 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:48.441+0000 7fc16ea00640 1 -- 172.21.15.12:0/3621486882 wait complete. 2024-09-11T00:25:48.444 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:48.443+0000 7fc16ea00640 1 Processor -- start 2024-09-11T00:25:48.445 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:48.443+0000 7fc16ea00640 1 -- start start 2024-09-11T00:25:48.445 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:48.444+0000 7fc16ea00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc1681093d0 0x7fc16819f8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:25:48.446 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:48.444+0000 7fc16ea00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc16810ede0 0x7fc16819fdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:25:48.446 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:48.444+0000 7fc16ea00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc16810f760 0x7fc1681a5190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:25:48.446 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:48.445+0000 7fc167e00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc1681093d0 0x7fc16819f8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:25:48.446 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:48.445+0000 7fc167e00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc1681093d0 0x7fc16819f8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.12:59286/0 (socket says 172.21.15.12:59286) 2024-09-11T00:25:48.446 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:48.445+0000 7fc15fe00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc16810ede0 0x7fc16819fdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:25:48.447 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:48.445+0000 7fc167e00640 1 -- 172.21.15.12:0/593296027 learned_addr learned my addr 172.21.15.12:0/593296027 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:25:48.447 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:48.445+0000 7fc16ea00640 1 -- 172.21.15.12:0/593296027 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fc1681193c0 con 0x7fc16810f760 2024-09-11T00:25:48.447 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:48.445+0000 7fc16ea00640 1 -- 172.21.15.12:0/593296027 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fc168119240 con 0x7fc16810ede0 2024-09-11T00:25:48.447 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:48.445+0000 7fc16ea00640 1 -- 172.21.15.12:0/593296027 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fc168119540 con 0x7fc1681093d0 2024-09-11T00:25:48.447 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:48.445+0000 7fc16ca00640 1 --2- 172.21.15.12:0/593296027 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc16810f760 0x7fc1681a5190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:25:48.447 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:48.445+0000 7fc167e00640 1 -- 172.21.15.12:0/593296027 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc16810ede0 msgr2=0x7fc16819fdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:25:48.448 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:48.446+0000 7fc167e00640 1 --2- 172.21.15.12:0/593296027 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc16810ede0 0x7fc16819fdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:48.448 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:48.446+0000 7fc167e00640 1 -- 172.21.15.12:0/593296027 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc16810f760 msgr2=0x7fc1681a5190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:25:48.448 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:48.446+0000 7fc167e00640 1 --2- 172.21.15.12:0/593296027 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc16810f760 0x7fc1681a5190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:48.448 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:48.446+0000 7fc167e00640 1 -- 172.21.15.12:0/593296027 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1681a5960 con 0x7fc1681093d0 2024-09-11T00:25:48.448 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:48.446+0000 7fc16ca00640 1 --2- 172.21.15.12:0/593296027 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc16810f760 0x7fc1681a5190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T00:25:48.448 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:48.446+0000 7fc15fe00640 1 --2- 172.21.15.12:0/593296027 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc16810ede0 0x7fc16819fdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T00:25:48.448 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:48.447+0000 7fc167e00640 1 --2- 172.21.15.12:0/593296027 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc1681093d0 0x7fc16819f8b0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fc158005ab0 tx=0x7fc158003cd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:25:48.449 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:48.447+0000 7fc165600640 1 -- 172.21.15.12:0/593296027 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc15803f070 con 0x7fc1681093d0 2024-09-11T00:25:48.449 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:48.447+0000 7fc165600640 1 -- 172.21.15.12:0/593296027 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc158031d60 con 0x7fc1681093d0 2024-09-11T00:25:48.449 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:48.447+0000 7fc165600640 1 -- 172.21.15.12:0/593296027 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc158035030 con 0x7fc1681093d0 2024-09-11T00:25:48.449 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:48.447+0000 7fc16ea00640 1 -- 172.21.15.12:0/593296027 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc1681a5bf0 con 0x7fc1681093d0 2024-09-11T00:25:48.449 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:48.447+0000 7fc16ea00640 1 -- 172.21.15.12:0/593296027 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1681ac310 con 0x7fc1681093d0 2024-09-11T00:25:48.450 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:48.449+0000 7fc16ea00640 1 -- 172.21.15.12:0/593296027 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc1681097b0 con 0x7fc1681093d0 2024-09-11T00:25:48.453 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:48.451+0000 7fc165600640 1 -- 172.21.15.12:0/593296027 <== mon.1 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7fc15803b580 con 0x7fc1681093d0 2024-09-11T00:25:48.454 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:48.453+0000 7fc165600640 1 --2- 172.21.15.12:0/593296027 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fc14c0b52e0 0x7fc14c0b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:25:48.454 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:48.453+0000 7fc165600640 1 -- 172.21.15.12:0/593296027 <== mon.1 v2:172.21.15.178:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9208+0+0 (secure 0 0 0) 0x7fc158106f00 con 0x7fc1681093d0 2024-09-11T00:25:48.455 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:48.454+0000 7fc15fe00640 1 --2- 172.21.15.12:0/593296027 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fc14c0b52e0 0x7fc14c0b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:25:48.456 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:48.454+0000 7fc15fe00640 1 --2- 172.21.15.12:0/593296027 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fc14c0b52e0 0x7fc14c0b77a0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fc16810e440 tx=0x7fc150005d90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:25:48.457 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:48.455+0000 7fc165600640 1 -- 172.21.15.12:0/593296027 <== mon.1 v2:172.21.15.178:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7fc158049030 con 0x7fc1681093d0 2024-09-11T00:25:48.859 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:48.858+0000 7fc16ea00640 1 -- 172.21.15.12:0/593296027 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc1681ac5a0 con 0x7fc1681093d0 2024-09-11T00:25:48.860 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:48.859+0000 7fc165600640 1 -- 172.21.15.12:0/593296027 <== mon.1 v2:172.21.15.178:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 71 v71) ==== 94+0+11971 (secure 0 0 0) 0x7fc1580cd990 con 0x7fc1681093d0 2024-09-11T00:25:48.864 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-11T00:25:48.864 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":71,"btime":"2024-09-11T00:25:29:916977+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14577,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6835/447171591","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":447171591},{"type":"v1","addr":"172.21.15.12:6835","nonce":447171591}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14589,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/901486040","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":901486040},{"type":"v1","addr":"172.21.15.12:6837","nonce":901486040}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":71,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T00:04:04.739881+0000","modified":"2024-09-11T00:25:29.554381+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24674,"mds_1":24643,"mds_2":24638,"mds_3":24637,"mds_4":24614},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14631":{"gid":14631,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.12:6841/3069919437","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6840","nonce":3069919437},{"type":"v1","addr":"172.21.15.12:6841","nonce":3069919437}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24562":{"gid":24562,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6833/1141636112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":1141636112},{"type":"v1","addr":"172.21.15.178:6833","nonce":1141636112}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24581":{"gid":24581,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6833/3111980970","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6832","nonce":3111980970},{"type":"v1","addr":"172.21.15.165:6833","nonce":3111980970}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24607":{"gid":24607,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6835/4084868287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":4084868287},{"type":"v1","addr":"172.21.15.178:6835","nonce":4084868287}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24614":{"gid":24614,"name":"e","rank":4,"incarnation":26,"state":"up:active","state_seq":21,"addr":"172.21.15.165:6835/2425395274","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":2425395274},{"type":"v1","addr":"172.21.15.165:6835","nonce":2425395274}]},"join_fscid":-1,"export_targets":[1],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24637":{"gid":24637,"name":"i","rank":3,"incarnation":24,"state":"up:active","state_seq":18,"addr":"172.21.15.178:6839/70628097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6838","nonce":70628097},{"type":"v1","addr":"172.21.15.178:6839","nonce":70628097}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24638":{"gid":24638,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":18,"addr":"172.21.15.165:6837/6215088","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6836","nonce":6215088},{"type":"v1","addr":"172.21.15.165:6837","nonce":6215088}]},"join_fscid":-1,"export_targets":[3],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24643":{"gid":24643,"name":"g","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.12:6839/3124093009","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6838","nonce":3124093009},{"type":"v1","addr":"172.21.15.12:6839","nonce":3124093009}]},"join_fscid":-1,"export_targets":[0,3],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24674":{"gid":24674,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.165:6839/501312984","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6838","nonce":501312984},{"type":"v1","addr":"172.21.15.165:6839","nonce":501312984}]},"join_fscid":-1,"export_targets":[1],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25672":{"gid":25672,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6837/308601741","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":308601741},{"type":"v1","addr":"172.21.15.178:6837","nonce":308601741}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24674,"qdb_cluster":[24614,24637,24638,24643,24674]},"id":1}]} 2024-09-11T00:25:48.865 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 71 2024-09-11T00:25:48.867 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:48.865+0000 7fc16ea00640 1 -- 172.21.15.12:0/593296027 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fc14c0b52e0 msgr2=0x7fc14c0b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:25:48.867 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:48.865+0000 7fc16ea00640 1 --2- 172.21.15.12:0/593296027 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fc14c0b52e0 0x7fc14c0b77a0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fc16810e440 tx=0x7fc150005d90 comp rx=0 tx=0).stop 2024-09-11T00:25:48.867 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:48.865+0000 7fc16ea00640 1 -- 172.21.15.12:0/593296027 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc1681093d0 msgr2=0x7fc16819f8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:25:48.867 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:48.865+0000 7fc16ea00640 1 --2- 172.21.15.12:0/593296027 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc1681093d0 0x7fc16819f8b0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fc158005ab0 tx=0x7fc158003cd0 comp rx=0 tx=0).stop 2024-09-11T00:25:48.868 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:48.865+0000 7fc16ea00640 1 -- 172.21.15.12:0/593296027 shutdown_connections 2024-09-11T00:25:48.868 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:48.865+0000 7fc16ea00640 1 --2- 172.21.15.12:0/593296027 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fc14c0b52e0 0x7fc14c0b77a0 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:48.868 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:48.865+0000 7fc16ea00640 1 --2- 172.21.15.12:0/593296027 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc16810f760 0x7fc1681a5190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:48.868 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:48.865+0000 7fc16ea00640 1 --2- 172.21.15.12:0/593296027 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc16810ede0 0x7fc16819fdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:48.868 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:48.865+0000 7fc16ea00640 1 --2- 172.21.15.12:0/593296027 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc1681093d0 0x7fc16819f8b0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:48.868 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:48.865+0000 7fc16ea00640 1 -- 172.21.15.12:0/593296027 >> 172.21.15.12:0/593296027 conn(0x7fc168078a40 msgr2=0x7fc1680791f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:25:48.868 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:48.867+0000 7fc16ea00640 1 -- 172.21.15.12:0/593296027 shutdown_connections 2024-09-11T00:25:48.868 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:48.867+0000 7fc16ea00640 1 -- 172.21.15.12:0/593296027 wait complete. 2024-09-11T00:25:49.205 INFO:tasks.workunit.client.0.smithi012.stdout: 10 102062 16.51 MB/sec execute 225 sec latency 56.243 ms 2024-09-11T00:25:49.232 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:25:49 smithi012 ceph-mon[30333]: from='client.? 172.21.15.12:0/593296027' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T00:25:49.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:25:49 smithi178 ceph-mon[35919]: from='client.? 172.21.15.12:0/593296027' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T00:25:49.532 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:25:49 smithi165 ceph-mon[35356]: from='client.? 172.21.15.12:0/593296027' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T00:25:49.773 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24674 quiesce db / --timeout 90 --expiration 141.0 --await-for 45.0' 2024-09-11T00:25:49.773 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid ff3ad74a-6fcf-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24674 quiesce db / --timeout 90 --expiration 141.0 --await-for 45.0 2024-09-11T00:25:50.205 INFO:tasks.workunit.client.0.smithi012.stdout: 10 103046 16.68 MB/sec execute 226 sec latency 48.725 ms 2024-09-11T00:25:50.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:25:50 smithi178 ceph-mon[35919]: pgmap v961: 129 pgs: 129 active+clean; 575 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 690 KiB/s rd, 597 KiB/s wr, 126 op/s 2024-09-11T00:25:50.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:25:50 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T00:25:50.515 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:25:50.532 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:25:50 smithi165 ceph-mon[35356]: pgmap v961: 129 pgs: 129 active+clean; 575 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 690 KiB/s rd, 597 KiB/s wr, 126 op/s 2024-09-11T00:25:50.532 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:25:50 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T00:25:50.557 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:25:50 smithi012 ceph-mon[30333]: pgmap v961: 129 pgs: 129 active+clean; 575 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 690 KiB/s rd, 597 KiB/s wr, 126 op/s 2024-09-11T00:25:50.557 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:25:50 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T00:25:50.718 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:50.717+0000 7fa800200640 1 -- 172.21.15.12:0/3771877728 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa7f810ab10 msgr2=0x7fa7f81136b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:25:50.718 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:50.717+0000 7fa800200640 1 --2- 172.21.15.12:0/3771877728 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa7f810ab10 0x7fa7f81136b0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fa7f0009ca0 tx=0x7fa7f002a8b0 comp rx=0 tx=0).stop 2024-09-11T00:25:50.719 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:50.717+0000 7fa800200640 1 -- 172.21.15.12:0/3771877728 shutdown_connections 2024-09-11T00:25:50.719 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:50.717+0000 7fa800200640 1 --2- 172.21.15.12:0/3771877728 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa7f810ab10 0x7fa7f81136b0 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:50.719 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:50.717+0000 7fa800200640 1 --2- 172.21.15.12:0/3771877728 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa7f810a150 0x7fa7f810a5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:50.719 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:50.717+0000 7fa800200640 1 --2- 172.21.15.12:0/3771877728 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa7f8074470 0x7fa7f8074870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:50.719 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:50.717+0000 7fa800200640 1 -- 172.21.15.12:0/3771877728 >> 172.21.15.12:0/3771877728 conn(0x7fa7f806fa40 msgr2=0x7fa7f8071e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:25:50.719 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:50.717+0000 7fa800200640 1 -- 172.21.15.12:0/3771877728 shutdown_connections 2024-09-11T00:25:50.719 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:50.717+0000 7fa800200640 1 -- 172.21.15.12:0/3771877728 wait complete. 2024-09-11T00:25:50.722 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:50.719+0000 7fa800200640 1 Processor -- start 2024-09-11T00:25:50.722 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:50.719+0000 7fa800200640 1 -- start start 2024-09-11T00:25:50.723 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:50.721+0000 7fa800200640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa7f8074470 0x7fa7f81a7b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:25:50.723 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:50.721+0000 7fa800200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa7f810a150 0x7fa7f81a81d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:25:50.723 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:50.721+0000 7fa800200640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa7f810ab10 0x7fa7f81a8e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:25:50.723 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:50.721+0000 7fa800200640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fa7f8116ea0 con 0x7fa7f8074470 2024-09-11T00:25:50.723 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:50.721+0000 7fa800200640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fa7f8116d20 con 0x7fa7f810a150 2024-09-11T00:25:50.724 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:50.721+0000 7fa800200640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fa7f8117020 con 0x7fa7f810ab10 2024-09-11T00:25:50.725 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:50.722+0000 7fa7fce00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa7f810a150 0x7fa7f81a81d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:25:50.725 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:50.722+0000 7fa7fce00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa7f810a150 0x7fa7f81a81d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.12:42518/0 (socket says 172.21.15.12:42518) 2024-09-11T00:25:50.725 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:50.722+0000 7fa7fce00640 1 -- 172.21.15.12:0/1409764276 learned_addr learned my addr 172.21.15.12:0/1409764276 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:25:50.725 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:50.722+0000 7fa7fe200640 1 --2- 172.21.15.12:0/1409764276 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa7f810ab10 0x7fa7f81a8e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:25:50.726 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:50.722+0000 7fa7fd800640 1 --2- 172.21.15.12:0/1409764276 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa7f8074470 0x7fa7f81a7b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:25:50.726 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:50.723+0000 7fa7fd800640 1 -- 172.21.15.12:0/1409764276 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa7f810ab10 msgr2=0x7fa7f81a8e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:25:50.726 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:50.723+0000 7fa7fd800640 1 --2- 172.21.15.12:0/1409764276 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa7f810ab10 0x7fa7f81a8e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:50.726 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:50.723+0000 7fa7fd800640 1 -- 172.21.15.12:0/1409764276 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa7f810a150 msgr2=0x7fa7f81a81d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:25:50.726 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:50.723+0000 7fa7fd800640 1 --2- 172.21.15.12:0/1409764276 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa7f810a150 0x7fa7f81a81d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:50.726 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:50.723+0000 7fa7fd800640 1 -- 172.21.15.12:0/1409764276 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7f81a96c0 con 0x7fa7f8074470 2024-09-11T00:25:50.726 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:50.723+0000 7fa7fce00640 1 --2- 172.21.15.12:0/1409764276 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa7f810a150 0x7fa7f81a81d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:25:50.728 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:50.726+0000 7fa7fd800640 1 --2- 172.21.15.12:0/1409764276 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa7f8074470 0x7fa7f81a7b80 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7fa7ec009e90 tx=0x7fa7ec00ed90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:25:50.729 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:50.727+0000 7fa7f6000640 1 -- 172.21.15.12:0/1409764276 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7ec00cc40 con 0x7fa7f8074470 2024-09-11T00:25:50.729 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:50.727+0000 7fa800200640 1 -- 172.21.15.12:0/1409764276 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa7f81a99b0 con 0x7fa7f8074470 2024-09-11T00:25:50.729 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:50.727+0000 7fa7f6000640 1 -- 172.21.15.12:0/1409764276 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa7ec00cde0 con 0x7fa7f8074470 2024-09-11T00:25:50.730 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:50.727+0000 7fa7f6000640 1 -- 172.21.15.12:0/1409764276 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7ec029880 con 0x7fa7f8074470 2024-09-11T00:25:50.730 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:50.727+0000 7fa800200640 1 -- 172.21.15.12:0/1409764276 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7f819c7e0 con 0x7fa7f8074470 2024-09-11T00:25:50.736 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:50.735+0000 7fa803069740 1 Processor -- start 2024-09-11T00:25:50.737 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:50.736+0000 7fa803069740 1 -- start start 2024-09-11T00:25:50.737 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:50.736+0000 7fa7f6000640 1 -- 172.21.15.12:0/1409764276 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7fa7ec021b10 con 0x7fa7f8074470 2024-09-11T00:25:50.738 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:50.736+0000 7fa803069740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561494c1d790 0x561494c1fb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:25:50.738 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:50.737+0000 7fa803069740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561494c200c0 0x561494c224b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:25:50.739 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:50.737+0000 7fa803069740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561494c22b00 0x561494c24fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:25:50.739 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:50.737+0000 7fa803069740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x561494b906b0 con 0x561494c22b00 2024-09-11T00:25:50.739 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:50.737+0000 7fa7fe200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561494c1d790 0x561494c1fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:25:50.739 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:50.737+0000 7fa7fe200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561494c1d790 0x561494c1fb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.12:42524/0 (socket says 172.21.15.12:42524) 2024-09-11T00:25:50.739 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:50.737+0000 7fa7fe200640 1 -- 172.21.15.12:0/3984324924 learned_addr learned my addr 172.21.15.12:0/3984324924 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:25:50.739 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:50.737+0000 7fa803069740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x561494b7fae0 con 0x561494c1d790 2024-09-11T00:25:50.740 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:50.737+0000 7fa803069740 1 -- 172.21.15.12:0/3984324924 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x561494b838f0 con 0x561494c200c0 2024-09-11T00:25:50.741 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:50.737+0000 7fa7fd800640 1 --2- 172.21.15.12:0/3984324924 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561494c200c0 0x561494c224b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:25:50.741 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:50.738+0000 7fa7fe200640 1 -- 172.21.15.12:0/3984324924 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561494c200c0 msgr2=0x561494c224b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:25:50.741 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:50.738+0000 7fa7fe200640 1 --2- 172.21.15.12:0/3984324924 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561494c200c0 0x561494c224b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:50.741 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:50.738+0000 7fa7fe200640 1 -- 172.21.15.12:0/3984324924 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561494c22b00 msgr2=0x561494c24fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:25:50.741 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:50.738+0000 7fa7fe200640 1 --2- 172.21.15.12:0/3984324924 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561494c22b00 0x561494c24fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:50.742 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:50.738+0000 7fa7fe200640 1 -- 172.21.15.12:0/3984324924 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561494b4c780 con 0x561494c1d790 2024-09-11T00:25:50.742 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:50.739+0000 7fa7f6000640 1 --2- 172.21.15.12:0/1409764276 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fa7e00b5360 0x7fa7e00b7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:25:50.742 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:50.739+0000 7fa7fe200640 1 --2- 172.21.15.12:0/3984324924 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561494c1d790 0x561494c1fb80 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fa7f002aea0 tx=0x7fa7f0009a20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7bd66d788602159a server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:25:50.742 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:50.739+0000 7fa7f6000640 1 -- 172.21.15.12:0/1409764276 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9208+0+0 (secure 0 0 0) 0x7fa7ec0f71e0 con 0x7fa7f8074470 2024-09-11T00:25:50.742 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:50.739+0000 7fa7fd800640 1 --2- 172.21.15.12:0/1409764276 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fa7e00b5360 0x7fa7e00b7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:25:50.743 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:50.740+0000 7fa7eaa00640 1 -- 172.21.15.12:0/3984324924 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7f0030900 con 0x561494c1d790 2024-09-11T00:25:50.744 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:50.740+0000 7fa7eaa00640 1 -- 172.21.15.12:0/3984324924 <== mon.2 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa7f0036030 con 0x561494c1d790 2024-09-11T00:25:50.744 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:50.740+0000 7fa7eaa00640 1 -- 172.21.15.12:0/3984324924 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7f0004600 con 0x561494c1d790 2024-09-11T00:25:50.744 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:50.742+0000 7fa7fd800640 1 --2- 172.21.15.12:0/1409764276 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fa7e00b5360 0x7fa7e00b7820 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fa7ec0f4910 tx=0x7fa7ec021eb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:25:50.745 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:50.743+0000 7fa803069740 1 -- 172.21.15.12:0/3984324924 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561494c1d790 msgr2=0x561494c1fb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:25:50.745 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:50.743+0000 7fa803069740 1 --2- 172.21.15.12:0/3984324924 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561494c1d790 0x561494c1fb80 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fa7f002aea0 tx=0x7fa7f0009a20 comp rx=0 tx=0).stop 2024-09-11T00:25:50.745 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:50.743+0000 7fa803069740 1 -- 172.21.15.12:0/3984324924 shutdown_connections 2024-09-11T00:25:50.745 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:50.743+0000 7fa803069740 1 --2- 172.21.15.12:0/3984324924 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561494c22b00 0x561494c24fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:50.745 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:50.743+0000 7fa803069740 1 --2- 172.21.15.12:0/3984324924 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561494c200c0 0x561494c224b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:50.746 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:50.743+0000 7fa803069740 1 --2- 172.21.15.12:0/3984324924 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561494c1d790 0x561494c1fb80 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:50.746 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:50.743+0000 7fa803069740 1 -- 172.21.15.12:0/3984324924 >> 172.21.15.12:0/3984324924 conn(0x561494c188e0 msgr2=0x561494c19670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:25:50.746 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:50.743+0000 7fa803069740 1 -- 172.21.15.12:0/3984324924 shutdown_connections 2024-09-11T00:25:50.747 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:50.744+0000 7fa803069740 1 -- 172.21.15.12:0/3984324924 wait complete. 2024-09-11T00:25:50.747 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:50.744+0000 7fa803069740 1 -- start start 2024-09-11T00:25:50.747 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:50.745+0000 7fa803069740 1 Processor -- start 2024-09-11T00:25:50.749 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:50.746+0000 7fa803069740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561494c1d790 0x561494c26e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:25:50.749 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:50.746+0000 7fa803069740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561494c200c0 0x561494c27500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:25:50.750 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:50.746+0000 7fa803069740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561494c22b00 0x561494c280d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:25:50.750 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:50.746+0000 7fa803069740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x561494ab7ab0 con 0x561494c1d790 2024-09-11T00:25:50.750 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:50.746+0000 7fa803069740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x561494b26570 con 0x561494c200c0 2024-09-11T00:25:50.750 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:50.746+0000 7fa803069740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x561494c25a80 con 0x561494c22b00 2024-09-11T00:25:50.750 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:50.746+0000 7fa803069740 10 client.0 _ll_register_callbacks cb 0x561494b35100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:25:50.750 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:50.747+0000 7fa7fce00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561494c200c0 0x561494c27500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:25:50.751 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:50.747+0000 7fa7fce00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561494c200c0 0x561494c27500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.12:42528/0 (socket says 172.21.15.12:42528) 2024-09-11T00:25:50.751 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:50.747+0000 7fa7fce00640 1 -- 172.21.15.12:0/1801816355 learned_addr learned my addr 172.21.15.12:0/1801816355 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:25:50.751 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:50.747+0000 7fa7fce00640 1 -- 172.21.15.12:0/1801816355 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561494c22b00 msgr2=0x561494c280d0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-11T00:25:50.751 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:50.747+0000 7fa7fce00640 1 --2- 172.21.15.12:0/1801816355 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561494c22b00 0x561494c280d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:50.751 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:50.747+0000 7fa7fce00640 1 -- 172.21.15.12:0/1801816355 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561494c1d790 msgr2=0x561494c26e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:25:50.751 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:50.747+0000 7fa7fce00640 1 --2- 172.21.15.12:0/1801816355 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561494c1d790 0x561494c26e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:50.752 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:50.747+0000 7fa7fce00640 1 -- 172.21.15.12:0/1801816355 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5614948e1790 con 0x561494c200c0 2024-09-11T00:25:50.752 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:50.748+0000 7fa7fce00640 1 --2- 172.21.15.12:0/1801816355 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561494c200c0 0x561494c27500 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fa7e400e7f0 tx=0x7fa7e400ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=338c1fc79ac8d72 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:25:50.752 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:50.748+0000 7fa7df400640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-09-11T00:25:50.753 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:50.748+0000 7fa7df400640 1 -- 172.21.15.12:0/1801816355 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7e4017070 con 0x561494c200c0 2024-09-11T00:25:50.753 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:50.748+0000 7fa7df400640 1 -- 172.21.15.12:0/1801816355 <== mon.2 v2:172.21.15.165:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9208+0+0 (secure 0 0 0) 0x7fa7e400c860 con 0x561494c200c0 2024-09-11T00:25:50.753 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:50.748+0000 7fa803069740 1 -- 172.21.15.12:0/1801816355 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa7f0036030 con 0x561494c200c0 2024-09-11T00:25:50.753 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:50.749+0000 7fa7df400640 1 -- 172.21.15.12:0/1801816355 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=72) ==== 24+0+0 (secure 0 0 0) 0x7fa7e4023050 con 0x561494c200c0 2024-09-11T00:25:50.753 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:50.749+0000 7fa803069740 10 client.27554 fetch_fsmap learned FSMap version 72 2024-09-11T00:25:50.753 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:50.749+0000 7fa803069740 1 -- 172.21.15.12:0/1801816355 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=72,monmap=4+}) -- 0x56149495a620 con 0x561494c200c0 2024-09-11T00:25:50.753 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:50.750+0000 7fa7df400640 1 -- 172.21.15.12:0/1801816355 <== mon.2 v2:172.21.15.165:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa7f0036030 con 0x561494c200c0 2024-09-11T00:25:50.753 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:50.750+0000 7fa7df400640 1 -- 172.21.15.12:0/1801816355 <== mon.2 v2:172.21.15.165:3300/0 5 ==== fsmap(e 72) ==== 8386+0+0 (secure 0 0 0) 0x7fa7e4013c90 con 0x561494c200c0 2024-09-11T00:25:50.753 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:50.750+0000 7fa803069740 10 client.27554 fetch_fsmap finished waiting for FSMap version 72 2024-09-11T00:25:50.753 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:50.750+0000 7fa803069740 10 client.27554 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T00:25:50.754 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:50.750+0000 7fa803069740 20 client.27554 populate_metadata read hostname 'smithi012' 2024-09-11T00:25:50.754 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:50.750+0000 7fa803069740 1 --2- 172.21.15.12:0/1801816355 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x561494c32c70 0x561494c35130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:25:50.754 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:50.751+0000 7fa803069740 4 client.27554 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T00:25:50.754 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:50.751+0000 7fa803069740 1 -- 172.21.15.12:0/1801816355 --> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561494b92600 con 0x561494c32c70 2024-09-11T00:25:50.754 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:50.751+0000 7fa7fe200640 1 --2- 172.21.15.12:0/1801816355 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x561494c32c70 0x561494c35130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:25:50.755 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:50.753+0000 7fa7df400640 10 client.27554 ms_handle_connect on v2:172.21.15.165:6838/501312984 2024-09-11T00:25:50.755 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:50.753+0000 7fa7fe200640 1 --2- 172.21.15.12:0/1801816355 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x561494c32c70 0x561494c35130 crc :-1 s=READY pgs=263 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cad0f7240ee23cef server_cookie=d70ea152a6cb882 in_seq=0 out_seq=0 2024-09-11T00:25:50.757 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:50.755+0000 7fa7df400640 1 -- 172.21.15.12:0/1801816355 <== mds.0 v2:172.21.15.165:6838/501312984 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x561494b92600 con 0x561494c32c70 2024-09-11T00:25:50.757 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:50.755+0000 7fa7df400640 10 client.27554 handle_command_reply: tid=0 2024-09-11T00:25:50.757 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:50.755+0000 7fa803069740 1 client.27554 shutdown 2024-09-11T00:25:50.757 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:50.755+0000 7fa803069740 1 -- 172.21.15.12:0/1801816355 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561494c200c0 msgr2=0x561494c27500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:25:50.757 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:50.755+0000 7fa803069740 1 --2- 172.21.15.12:0/1801816355 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561494c200c0 0x561494c27500 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fa7e400e7f0 tx=0x7fa7e400ecb0 comp rx=0 tx=0).stop 2024-09-11T00:25:50.757 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:50.755+0000 7fa803069740 1 -- 172.21.15.12:0/1801816355 shutdown_connections 2024-09-11T00:25:50.757 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:50.755+0000 7fa803069740 1 --2- 172.21.15.12:0/1801816355 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x561494c32c70 0x561494c35130 crc :-1 s=READY pgs=263 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:50.758 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:50.755+0000 7fa803069740 1 --2- 172.21.15.12:0/1801816355 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561494c22b00 0x561494c280d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:50.758 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:50.755+0000 7fa803069740 1 --2- 172.21.15.12:0/1801816355 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561494c200c0 0x561494c27500 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:50.758 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:50.755+0000 7fa7df400640 0 client.27554 ms_handle_reset on v2:172.21.15.165:6838/501312984 2024-09-11T00:25:50.758 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:50.755+0000 7fa803069740 1 --2- 172.21.15.12:0/1801816355 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561494c1d790 0x561494c26e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:50.758 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:50.755+0000 7fa803069740 1 -- 172.21.15.12:0/1801816355 >> 172.21.15.12:0/1801816355 conn(0x561494c188e0 msgr2=0x561494c17a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:25:50.759 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:50.756+0000 7fa803069740 1 -- 172.21.15.12:0/1801816355 shutdown_connections 2024-09-11T00:25:50.759 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:50.756+0000 7fa803069740 1 -- 172.21.15.12:0/1801816355 wait complete. 2024-09-11T00:25:50.759 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:50.756+0000 7fa803069740 20 client.27554 trim_cache size 0 max 16384 2024-09-11T00:25:50.788 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:50.785+0000 7fa803069740 1 Processor -- start 2024-09-11T00:25:50.788 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:50.785+0000 7fa803069740 1 -- start start 2024-09-11T00:25:50.789 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:50.786+0000 7fa803069740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561494c1d790 0x561494c26e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:25:50.789 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:50.786+0000 7fa803069740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561494c200c0 0x561494c273a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:25:50.789 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:50.786+0000 7fa803069740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561494c22b00 0x561494c1a710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:25:50.789 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:50.787+0000 7fa803069740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fa7f81b1dc0 con 0x561494c1d790 2024-09-11T00:25:50.789 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:50.787+0000 7fa7fce00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561494c200c0 0x561494c273a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:25:50.789 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:50.787+0000 7fa7fce00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561494c200c0 0x561494c273a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.12:36872/0 (socket says 172.21.15.12:36872) 2024-09-11T00:25:50.790 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:50.787+0000 7fa7fe200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561494c22b00 0x561494c1a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:25:50.790 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:50.787+0000 7fa803069740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fa7f81b3990 con 0x561494c22b00 2024-09-11T00:25:50.790 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:50.787+0000 7fa7fe200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561494c22b00 0x561494c1a710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.12:42538/0 (socket says 172.21.15.12:42538) 2024-09-11T00:25:50.790 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:50.787+0000 7fa803069740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fa7f81b2490 con 0x561494c200c0 2024-09-11T00:25:50.790 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:50.787+0000 7fa7fe200640 1 -- 172.21.15.12:0/4128372898 learned_addr learned my addr 172.21.15.12:0/4128372898 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:25:50.790 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:50.787+0000 7fa7fe200640 1 --2- 172.21.15.12:0/4128372898 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561494c1d790 0x561494c26e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:25:50.790 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:50.787+0000 7fa7fe200640 1 -- 172.21.15.12:0/4128372898 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561494c200c0 msgr2=0x561494c273a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:25:50.790 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:50.787+0000 7fa7fe200640 1 --2- 172.21.15.12:0/4128372898 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561494c200c0 0x561494c273a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:50.790 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:50.787+0000 7fa7fe200640 1 -- 172.21.15.12:0/4128372898 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561494c1d790 msgr2=0x561494c26e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:25:50.790 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:50.787+0000 7fa7fe200640 1 --2- 172.21.15.12:0/4128372898 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561494c1d790 0x561494c26e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:50.790 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:50.787+0000 7fa7fe200640 1 -- 172.21.15.12:0/4128372898 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561494c1fef0 con 0x561494c22b00 2024-09-11T00:25:50.791 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:50.790+0000 7fa7fe200640 1 --2- 172.21.15.12:0/4128372898 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561494c22b00 0x561494c1a710 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x561494c27bb0 tx=0x7fa7f0047040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bb61fb684dfbf31b server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:25:50.792 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:50.791+0000 7fa7ea000640 1 -- 172.21.15.12:0/4128372898 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7f00304e0 con 0x561494c22b00 2024-09-11T00:25:50.792 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:50.791+0000 7fa7ea000640 1 -- 172.21.15.12:0/4128372898 <== mon.2 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa7f0036070 con 0x561494c22b00 2024-09-11T00:25:50.792 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:50.791+0000 7fa7ea000640 1 -- 172.21.15.12:0/4128372898 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7f00043b0 con 0x561494c22b00 2024-09-11T00:25:50.793 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:50.792+0000 7fa803069740 1 -- 172.21.15.12:0/4128372898 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561494c22b00 msgr2=0x561494c1a710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:25:50.794 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:50.792+0000 7fa803069740 1 --2- 172.21.15.12:0/4128372898 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561494c22b00 0x561494c1a710 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x561494c27bb0 tx=0x7fa7f0047040 comp rx=0 tx=0).stop 2024-09-11T00:25:50.794 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:50.792+0000 7fa803069740 1 -- 172.21.15.12:0/4128372898 shutdown_connections 2024-09-11T00:25:50.794 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:50.792+0000 7fa803069740 1 --2- 172.21.15.12:0/4128372898 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561494c22b00 0x561494c1a710 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:50.794 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:50.793+0000 7fa803069740 1 --2- 172.21.15.12:0/4128372898 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561494c200c0 0x561494c273a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:50.794 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:50.793+0000 7fa803069740 1 --2- 172.21.15.12:0/4128372898 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561494c1d790 0x561494c26e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:50.794 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:50.793+0000 7fa803069740 1 -- 172.21.15.12:0/4128372898 >> 172.21.15.12:0/4128372898 conn(0x561494c188e0 msgr2=0x561494c192a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:25:50.795 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:50.793+0000 7fa803069740 1 -- 172.21.15.12:0/4128372898 shutdown_connections 2024-09-11T00:25:50.795 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:50.794+0000 7fa803069740 1 -- 172.21.15.12:0/4128372898 wait complete. 2024-09-11T00:25:50.797 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:50.794+0000 7fa803069740 1 -- start start 2024-09-11T00:25:50.797 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:50.794+0000 7fa803069740 1 Processor -- start 2024-09-11T00:25:50.797 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:50.795+0000 7fa803069740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561494c1d790 0x561494c99a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:25:50.797 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:50.795+0000 7fa803069740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561494c200c0 0x561494c9bf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:25:50.797 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:50.795+0000 7fa803069740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561494c22b00 0x561494c9e540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:25:50.797 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:50.795+0000 7fa803069740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fa7f81b32c0 con 0x561494c22b00 2024-09-11T00:25:50.797 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:50.795+0000 7fa803069740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fa7f81b2bc0 con 0x561494c200c0 2024-09-11T00:25:50.797 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:50.795+0000 7fa803069740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fa7f81afea0 con 0x561494c1d790 2024-09-11T00:25:50.798 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:50.795+0000 7fa803069740 10 client.0 _ll_register_callbacks cb 0x7fa7e4009840 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:25:50.798 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:50.796+0000 7fa7fce00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561494c200c0 0x561494c9bf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:25:50.798 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:50.796+0000 7fa7fce00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561494c200c0 0x561494c9bf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.12:42546/0 (socket says 172.21.15.12:42546) 2024-09-11T00:25:50.798 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:50.796+0000 7fa7fce00640 1 -- 172.21.15.12:0/3463017458 learned_addr learned my addr 172.21.15.12:0/3463017458 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:25:50.798 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:50.796+0000 7fa7fe200640 1 --2- 172.21.15.12:0/3463017458 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561494c1d790 0x561494c99a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:25:50.799 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:50.796+0000 7fa7fce00640 1 -- 172.21.15.12:0/3463017458 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561494c1d790 msgr2=0x561494c99a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:25:50.799 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:50.796+0000 7fa7fce00640 1 --2- 172.21.15.12:0/3463017458 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561494c1d790 0x561494c99a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:50.799 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:50.796+0000 7fa7fce00640 1 -- 172.21.15.12:0/3463017458 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561494c22b00 msgr2=0x561494c9e540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:25:50.799 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:50.796+0000 7fa7fce00640 1 --2- 172.21.15.12:0/3463017458 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561494c22b00 0x561494c9e540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:50.799 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:50.796+0000 7fa7fce00640 1 -- 172.21.15.12:0/3463017458 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561494c9ed70 con 0x561494c200c0 2024-09-11T00:25:50.800 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:50.797+0000 7fa7fce00640 1 --2- 172.21.15.12:0/3463017458 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561494c200c0 0x561494c9bf90 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fa7e40125f0 tx=0x7fa7e4012880 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7e4438b268bd84dc server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:25:50.800 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:50.797+0000 7fa7df400640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-09-11T00:25:50.801 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:50.798+0000 7fa7df400640 1 -- 172.21.15.12:0/3463017458 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7e4017040 con 0x561494c200c0 2024-09-11T00:25:50.801 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:50.798+0000 7fa7df400640 1 -- 172.21.15.12:0/3463017458 <== mon.2 v2:172.21.15.165:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9208+0+0 (secure 0 0 0) 0x7fa7e40059c0 con 0x561494c200c0 2024-09-11T00:25:50.801 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:50.798+0000 7fa803069740 1 -- 172.21.15.12:0/3463017458 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa7f0036070 con 0x561494c200c0 2024-09-11T00:25:50.801 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:50.799+0000 7fa7df400640 1 -- 172.21.15.12:0/3463017458 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=72) ==== 24+0+0 (secure 0 0 0) 0x7fa7e4030050 con 0x561494c200c0 2024-09-11T00:25:50.801 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:50.799+0000 7fa803069740 10 client.27560 fetch_fsmap learned FSMap version 72 2024-09-11T00:25:50.801 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:50.799+0000 7fa803069740 1 -- 172.21.15.12:0/3463017458 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=72,monmap=4+}) -- 0x561494ca0120 con 0x561494c200c0 2024-09-11T00:25:50.801 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:50.799+0000 7fa7df400640 1 -- 172.21.15.12:0/3463017458 <== mon.2 v2:172.21.15.165:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa7f0036070 con 0x561494c200c0 2024-09-11T00:25:50.802 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:50.799+0000 7fa7df400640 1 -- 172.21.15.12:0/3463017458 <== mon.2 v2:172.21.15.165:3300/0 5 ==== fsmap(e 72) ==== 8386+0+0 (secure 0 0 0) 0x7fa7e4006030 con 0x561494c200c0 2024-09-11T00:25:50.802 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:50.800+0000 7fa803069740 10 client.27560 fetch_fsmap finished waiting for FSMap version 72 2024-09-11T00:25:50.802 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:50.800+0000 7fa803069740 10 client.27560 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T00:25:50.802 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:50.800+0000 7fa803069740 20 client.27560 populate_metadata read hostname 'smithi012' 2024-09-11T00:25:50.802 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:50.800+0000 7fa803069740 1 --2- 172.21.15.12:0/3463017458 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x561494ca0b30 0x561494ca2f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:25:50.803 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:50.800+0000 7fa803069740 4 client.27560 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T00:25:50.803 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:50.800+0000 7fa803069740 1 -- 172.21.15.12:0/3463017458 --> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] -- command(tid 0: {"prefix": "flush journal"}) -- 0x561494b8e0b0 con 0x561494ca0b30 2024-09-11T00:25:50.803 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:50.800+0000 7fa7fe200640 1 --2- 172.21.15.12:0/3463017458 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x561494ca0b30 0x561494ca2f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:25:50.803 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:50.802+0000 7fa7fe200640 1 --2- 172.21.15.12:0/3463017458 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x561494ca0b30 0x561494ca2f50 crc :-1 s=READY pgs=264 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e320e4a27c84f996 server_cookie=3183575d85accf9d in_seq=0 out_seq=0 2024-09-11T00:25:50.804 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:50.802+0000 7fa7df400640 10 client.27560 ms_handle_connect on v2:172.21.15.165:6838/501312984 2024-09-11T00:25:51.110 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:51.103+0000 7fa7df400640 1 -- 172.21.15.12:0/3463017458 <== mds.0 v2:172.21.15.165:6838/501312984 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x561494b8e0b0 con 0x561494ca0b30 2024-09-11T00:25:51.110 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:51.103+0000 7fa7df400640 10 client.27560 handle_command_reply: tid=0 2024-09-11T00:25:51.110 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:51.103+0000 7fa803069740 1 client.27560 shutdown 2024-09-11T00:25:51.110 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:51.103+0000 7fa803069740 1 -- 172.21.15.12:0/3463017458 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561494c200c0 msgr2=0x561494c9bf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:25:51.110 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:51.103+0000 7fa803069740 1 --2- 172.21.15.12:0/3463017458 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561494c200c0 0x561494c9bf90 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fa7e40125f0 tx=0x7fa7e4012880 comp rx=0 tx=0).stop 2024-09-11T00:25:51.110 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:51.103+0000 7fa803069740 1 -- 172.21.15.12:0/3463017458 shutdown_connections 2024-09-11T00:25:51.110 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:51.103+0000 7fa803069740 1 --2- 172.21.15.12:0/3463017458 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x561494ca0b30 0x561494ca2f50 crc :-1 s=READY pgs=264 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:51.110 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:51.103+0000 7fa803069740 1 --2- 172.21.15.12:0/3463017458 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561494c22b00 0x561494c9e540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:51.111 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:51.103+0000 7fa803069740 1 --2- 172.21.15.12:0/3463017458 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561494c200c0 0x561494c9bf90 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:51.111 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:51.103+0000 7fa803069740 1 --2- 172.21.15.12:0/3463017458 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561494c1d790 0x561494c99a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:51.111 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:51.103+0000 7fa803069740 1 -- 172.21.15.12:0/3463017458 >> 172.21.15.12:0/3463017458 conn(0x561494c188e0 msgr2=0x561494c26e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:25:51.111 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:51.103+0000 7fa7df400640 0 client.27560 ms_handle_reset on v2:172.21.15.165:6838/501312984 2024-09-11T00:25:51.111 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:51.108+0000 7fa803069740 1 -- 172.21.15.12:0/3463017458 shutdown_connections 2024-09-11T00:25:51.111 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:51.108+0000 7fa803069740 1 -- 172.21.15.12:0/3463017458 wait complete. 2024-09-11T00:25:51.111 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:51.108+0000 7fa803069740 20 client.27560 trim_cache size 0 max 16384 2024-09-11T00:25:51.111 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:{ 2024-09-11T00:25:51.111 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "message": "", 2024-09-11T00:25:51.111 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "return_code": 0 2024-09-11T00:25:51.112 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:} 2024-09-11T00:25:51.112 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:51.111+0000 7fa800200640 1 -- 172.21.15.12:0/1409764276 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fa7e00b5360 msgr2=0x7fa7e00b7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:25:51.112 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:51.111+0000 7fa800200640 1 --2- 172.21.15.12:0/1409764276 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fa7e00b5360 0x7fa7e00b7820 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fa7ec0f4910 tx=0x7fa7ec021eb0 comp rx=0 tx=0).stop 2024-09-11T00:25:51.112 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:51.111+0000 7fa800200640 1 -- 172.21.15.12:0/1409764276 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa7f8074470 msgr2=0x7fa7f81a7b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:25:51.112 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:51.111+0000 7fa800200640 1 --2- 172.21.15.12:0/1409764276 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa7f8074470 0x7fa7f81a7b80 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7fa7ec009e90 tx=0x7fa7ec00ed90 comp rx=0 tx=0).stop 2024-09-11T00:25:51.113 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:51.111+0000 7fa800200640 1 -- 172.21.15.12:0/1409764276 shutdown_connections 2024-09-11T00:25:51.113 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:51.111+0000 7fa800200640 1 --2- 172.21.15.12:0/1409764276 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fa7e00b5360 0x7fa7e00b7820 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:51.113 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:51.111+0000 7fa800200640 1 --2- 172.21.15.12:0/1409764276 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa7f810ab10 0x7fa7f81a8e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:51.113 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:51.111+0000 7fa800200640 1 --2- 172.21.15.12:0/1409764276 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa7f810a150 0x7fa7f81a81d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:51.113 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:51.111+0000 7fa800200640 1 --2- 172.21.15.12:0/1409764276 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa7f8074470 0x7fa7f81a7b80 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:51.113 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:51.111+0000 7fa800200640 1 -- 172.21.15.12:0/1409764276 >> 172.21.15.12:0/1409764276 conn(0x7fa7f806fa40 msgr2=0x7fa7f8071a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:25:51.114 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:51.112+0000 7fa800200640 1 -- 172.21.15.12:0/1409764276 shutdown_connections 2024-09-11T00:25:51.115 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:25:51.113+0000 7fa800200640 1 -- 172.21.15.12:0/1409764276 wait complete. 2024-09-11T00:25:51.205 INFO:tasks.workunit.client.0.smithi012.stdout: 10 103635 16.78 MB/sec execute 227 sec latency 265.789 ms 2024-09-11T00:25:51.227 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:25:51 smithi012 ceph-mon[30333]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T00:25:51.227 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:25:51 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:25:51.227 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:25:51 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T00:25:51.227 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:25:51 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:25:51.364 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:51.361+0000 7f8251600640 1 -- 172.21.15.12:0/260681427 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f824c113990 msgr2=0x7f824c115d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:25:51.364 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:51.361+0000 7f824a000640 1 -- 172.21.15.12:0/260681427 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f824402f310 con 0x7f824c113990 2024-09-11T00:25:51.364 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:51.361+0000 7f8251600640 1 --2- 172.21.15.12:0/260681427 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f824c113990 0x7f824c115d80 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f8244009ca0 tx=0x7f824402a8c0 comp rx=0 tx=0).stop 2024-09-11T00:25:51.367 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:51.366+0000 7f8251600640 1 -- 172.21.15.12:0/260681427 shutdown_connections 2024-09-11T00:25:51.367 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:51.366+0000 7f8251600640 1 --2- 172.21.15.12:0/260681427 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f824c113990 0x7f824c115d80 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:51.367 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:51.366+0000 7f8251600640 1 --2- 172.21.15.12:0/260681427 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f824c077ff0 0x7f824c113450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:51.367 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:51.366+0000 7f8251600640 1 --2- 172.21.15.12:0/260681427 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f824c0776d0 0x7f824c077ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:51.367 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:51.366+0000 7f8251600640 1 -- 172.21.15.12:0/260681427 >> 172.21.15.12:0/260681427 conn(0x7f824c100a50 msgr2=0x7f824c102e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:25:51.368 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:51.366+0000 7f8251600640 1 -- 172.21.15.12:0/260681427 shutdown_connections 2024-09-11T00:25:51.369 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:51.368+0000 7f8251600640 1 -- 172.21.15.12:0/260681427 wait complete. 2024-09-11T00:25:51.372 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:51.370+0000 7f8251600640 1 Processor -- start 2024-09-11T00:25:51.372 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:51.371+0000 7f8251600640 1 -- start start 2024-09-11T00:25:51.374 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:51.372+0000 7f8251600640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f824c0776d0 0x7f824c19f680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:25:51.374 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:51.373+0000 7f8251600640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f824c077ff0 0x7f824c19fbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:25:51.375 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:51.373+0000 7f8251600640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f824c113990 0x7f824c1a4f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:25:51.375 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:51.373+0000 7f8251600640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f824c1193b0 con 0x7f824c077ff0 2024-09-11T00:25:51.375 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:51.373+0000 7f8251600640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f824c119230 con 0x7f824c113990 2024-09-11T00:25:51.376 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:51.373+0000 7f8251600640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f824c119530 con 0x7f824c0776d0 2024-09-11T00:25:51.376 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:51.373+0000 7f824b400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f824c113990 0x7f824c1a4f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:25:51.376 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:51.373+0000 7f824aa00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f824c0776d0 0x7f824c19f680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:25:51.377 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:51.373+0000 7f824b400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f824c113990 0x7f824c1a4f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.12:42566/0 (socket says 172.21.15.12:42566) 2024-09-11T00:25:51.377 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:51.373+0000 7f824b400640 1 -- 172.21.15.12:0/273558055 learned_addr learned my addr 172.21.15.12:0/273558055 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:25:51.377 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:51.374+0000 7f824b400640 1 -- 172.21.15.12:0/273558055 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f824c0776d0 msgr2=0x7f824c19f680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:25:51.377 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:51.374+0000 7f824b400640 1 --2- 172.21.15.12:0/273558055 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f824c0776d0 0x7f824c19f680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:51.378 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:51.374+0000 7f824b400640 1 -- 172.21.15.12:0/273558055 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f824c077ff0 msgr2=0x7f824c19fbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:25:51.378 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:51.374+0000 7f824b400640 1 --2- 172.21.15.12:0/273558055 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f824c077ff0 0x7f824c19fbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:51.378 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:51.374+0000 7f824b400640 1 -- 172.21.15.12:0/273558055 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f824c1a5730 con 0x7f824c113990 2024-09-11T00:25:51.378 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:51.374+0000 7f824aa00640 1 --2- 172.21.15.12:0/273558055 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f824c0776d0 0x7f824c19f680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:25:51.378 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:51.374+0000 7f824b400640 1 --2- 172.21.15.12:0/273558055 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f824c113990 0x7f824c1a4f60 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f8244005e30 tx=0x7f82440093d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:25:51.378 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:51.375+0000 7f8243400640 1 -- 172.21.15.12:0/273558055 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8244003bf0 con 0x7f824c113990 2024-09-11T00:25:51.378 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:51.375+0000 7f8243400640 1 -- 172.21.15.12:0/273558055 <== mon.2 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8244003d90 con 0x7f824c113990 2024-09-11T00:25:51.378 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:51.375+0000 7f8243400640 1 -- 172.21.15.12:0/273558055 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f824402fc10 con 0x7f824c113990 2024-09-11T00:25:51.379 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:51.376+0000 7f8251600640 1 -- 172.21.15.12:0/273558055 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f824c1a59c0 con 0x7f824c113990 2024-09-11T00:25:51.379 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:51.377+0000 7f8251600640 1 -- 172.21.15.12:0/273558055 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f824c1ac0e0 con 0x7f824c113990 2024-09-11T00:25:51.383 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:51.382+0000 7f8253fc6740 1 Processor -- start 2024-09-11T00:25:51.392 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:51.383+0000 7f8243400640 1 -- 172.21.15.12:0/273558055 <== mon.2 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7f8244004040 con 0x7f824c113990 2024-09-11T00:25:51.392 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:51.387+0000 7f8243400640 1 --2- 172.21.15.12:0/273558055 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f82300b5460 0x7f82300b7920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:25:51.392 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:51.387+0000 7f8253fc6740 1 -- start start 2024-09-11T00:25:51.392 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:51.387+0000 7f8243400640 1 -- 172.21.15.12:0/273558055 <== mon.2 v2:172.21.15.165:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9208+0+0 (secure 0 0 0) 0x7f8244107330 con 0x7f824c113990 2024-09-11T00:25:51.392 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:51.389+0000 7f8253fc6740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a02f957180 0x55a02f957560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:25:51.392 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:51.389+0000 7f8253fc6740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a02f957aa0 0x55a02f9634f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:25:51.393 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:51.389+0000 7f8253fc6740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a02f963a30 0x55a02f965e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:25:51.393 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:51.390+0000 7f824aa00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a02f957180 0x55a02f957560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:25:51.393 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:51.390+0000 7f824aa00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a02f957180 0x55a02f957560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:39454/0 (socket says 172.21.15.12:39454) 2024-09-11T00:25:51.393 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:51.390+0000 7f824aa00640 1 -- 172.21.15.12:0/1951213173 learned_addr learned my addr 172.21.15.12:0/1951213173 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:25:51.393 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:51.390+0000 7f8253fc6740 1 -- 172.21.15.12:0/1951213173 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55a02f8ccca0 con 0x55a02f957180 2024-09-11T00:25:51.393 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:51.390+0000 7f8253fc6740 1 -- 172.21.15.12:0/1951213173 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55a02f8bc0d0 con 0x55a02f957aa0 2024-09-11T00:25:51.394 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:51.390+0000 7f8253fc6740 1 -- 172.21.15.12:0/1951213173 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55a02f8bfee0 con 0x55a02f963a30 2024-09-11T00:25:51.394 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:51.390+0000 7f824aa00640 1 -- 172.21.15.12:0/1951213173 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a02f963a30 msgr2=0x55a02f965e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:25:51.394 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:51.390+0000 7f824aa00640 1 --2- 172.21.15.12:0/1951213173 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a02f963a30 0x55a02f965e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:51.394 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:51.390+0000 7f824aa00640 1 -- 172.21.15.12:0/1951213173 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a02f957aa0 msgr2=0x55a02f9634f0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-11T00:25:51.394 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:51.390+0000 7f824aa00640 1 --2- 172.21.15.12:0/1951213173 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a02f957aa0 0x55a02f9634f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:51.394 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:51.390+0000 7f824aa00640 1 -- 172.21.15.12:0/1951213173 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a02f86e400 con 0x55a02f957180 2024-09-11T00:25:51.394 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:51.391+0000 7f8243e00640 1 --2- 172.21.15.12:0/273558055 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f82300b5460 0x7f82300b7920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:25:51.395 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:51.391+0000 7f824aa00640 1 --2- 172.21.15.12:0/1951213173 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a02f957180 0x55a02f957560 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f82340104d0 tx=0x7f8234010990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=afde7fc2b1b4374f server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:25:51.395 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:51.394+0000 7f8243e00640 1 --2- 172.21.15.12:0/273558055 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f82300b5460 0x7f82300b7920 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f823c0058c0 tx=0x7f823c00bda0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:25:51.397 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:51.394+0000 7f823b400640 1 -- 172.21.15.12:0/1951213173 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8234004220 con 0x55a02f957180 2024-09-11T00:25:51.397 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:51.394+0000 7f823b400640 1 -- 172.21.15.12:0/1951213173 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f82340043c0 con 0x55a02f957180 2024-09-11T00:25:51.398 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:51.394+0000 7f823b400640 1 -- 172.21.15.12:0/1951213173 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8234011aa0 con 0x55a02f957180 2024-09-11T00:25:51.398 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:51.395+0000 7f8253fc6740 1 -- 172.21.15.12:0/1951213173 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a02f957180 msgr2=0x55a02f957560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:25:51.398 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:51.395+0000 7f8253fc6740 1 --2- 172.21.15.12:0/1951213173 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a02f957180 0x55a02f957560 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f82340104d0 tx=0x7f8234010990 comp rx=0 tx=0).stop 2024-09-11T00:25:51.398 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:51.396+0000 7f8253fc6740 1 -- 172.21.15.12:0/1951213173 shutdown_connections 2024-09-11T00:25:51.398 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:51.396+0000 7f8253fc6740 1 --2- 172.21.15.12:0/1951213173 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a02f963a30 0x55a02f965e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:51.398 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:51.396+0000 7f8253fc6740 1 --2- 172.21.15.12:0/1951213173 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a02f957aa0 0x55a02f9634f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:51.398 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:51.396+0000 7f8253fc6740 1 --2- 172.21.15.12:0/1951213173 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a02f957180 0x55a02f957560 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:51.398 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:51.396+0000 7f8253fc6740 1 -- 172.21.15.12:0/1951213173 >> 172.21.15.12:0/1951213173 conn(0x55a02f954fb0 msgr2=0x55a02f956150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:25:51.399 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:51.396+0000 7f8253fc6740 1 -- 172.21.15.12:0/1951213173 shutdown_connections 2024-09-11T00:25:51.399 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:51.396+0000 7f8253fc6740 1 -- 172.21.15.12:0/1951213173 wait complete. 2024-09-11T00:25:51.399 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:51.397+0000 7f8253fc6740 1 -- start start 2024-09-11T00:25:51.399 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:51.397+0000 7f8253fc6740 1 Processor -- start 2024-09-11T00:25:51.400 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:51.398+0000 7f8253fc6740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a02f957180 0x55a02f963390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:25:51.400 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:51.398+0000 7f8253fc6740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a02f957aa0 0x55a02f9607e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:25:51.401 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:51.399+0000 7f824aa00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a02f957180 0x55a02f963390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:25:51.401 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:51.399+0000 7f824aa00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a02f957180 0x55a02f963390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.12:42582/0 (socket says 172.21.15.12:42582) 2024-09-11T00:25:51.401 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:51.399+0000 7f824aa00640 1 -- 172.21.15.12:0/2027653129 learned_addr learned my addr 172.21.15.12:0/2027653129 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:25:51.401 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:51.399+0000 7f8243e00640 1 --2- 172.21.15.12:0/2027653129 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a02f957aa0 0x55a02f9607e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:25:51.401 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:51.399+0000 7f8253fc6740 1 --2- 172.21.15.12:0/2027653129 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a02f963a30 0x55a02f960e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:25:51.401 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:51.399+0000 7f824b400640 1 --2- 172.21.15.12:0/2027653129 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a02f963a30 0x55a02f960e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:25:51.402 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:51.400+0000 7f8253fc6740 1 -- 172.21.15.12:0/2027653129 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55a02f71d7a0 con 0x55a02f957aa0 2024-09-11T00:25:51.402 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:51.400+0000 7f8253fc6740 1 -- 172.21.15.12:0/2027653129 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55a02f861a10 con 0x55a02f957180 2024-09-11T00:25:51.402 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:51.400+0000 7f8253fc6740 1 -- 172.21.15.12:0/2027653129 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55a02f966700 con 0x55a02f963a30 2024-09-11T00:25:51.403 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:51.401+0000 7f8253fc6740 10 client.0 _ll_register_callbacks cb 0x55a02f871be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:25:51.403 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:51.401+0000 7f824b400640 1 -- 172.21.15.12:0/2027653129 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a02f957180 msgr2=0x55a02f963390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:25:51.403 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:51.401+0000 7f824b400640 1 --2- 172.21.15.12:0/2027653129 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a02f957180 0x55a02f963390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:51.403 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:51.401+0000 7f824b400640 1 -- 172.21.15.12:0/2027653129 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a02f957aa0 msgr2=0x55a02f9607e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:25:51.403 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:51.401+0000 7f824b400640 1 --2- 172.21.15.12:0/2027653129 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a02f957aa0 0x55a02f9607e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:51.403 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:51.401+0000 7f824b400640 1 -- 172.21.15.12:0/2027653129 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a02f629b90 con 0x55a02f963a30 2024-09-11T00:25:51.404 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:51.403+0000 7f824b400640 1 --2- 172.21.15.12:0/2027653129 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a02f963a30 0x55a02f960e60 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f8244104a60 tx=0x7f82440cd130 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f2053bbda31907aa server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:25:51.404 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:51.403+0000 7f822fe00640 10 client.0 ms_handle_connect on v2:172.21.15.178:3300/0 2024-09-11T00:25:51.405 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:51.403+0000 7f822fe00640 1 -- 172.21.15.12:0/2027653129 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82440cde20 con 0x55a02f963a30 2024-09-11T00:25:51.406 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:51.404+0000 7f8253fc6740 1 -- 172.21.15.12:0/2027653129 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f82340043c0 con 0x55a02f963a30 2024-09-11T00:25:51.406 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:51.404+0000 7f822fe00640 1 -- 172.21.15.12:0/2027653129 <== mon.1 v2:172.21.15.178:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9208+0+0 (secure 0 0 0) 0x7f82440d6820 con 0x55a02f963a30 2024-09-11T00:25:51.406 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:51.404+0000 7f822fe00640 1 -- 172.21.15.12:0/2027653129 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=72) ==== 24+0+0 (secure 0 0 0) 0x7f82440cd720 con 0x55a02f963a30 2024-09-11T00:25:51.406 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:51.404+0000 7f8253fc6740 10 client.27538 fetch_fsmap learned FSMap version 72 2024-09-11T00:25:51.406 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:51.404+0000 7f8253fc6740 1 -- 172.21.15.12:0/2027653129 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,fsmap=72,monmap=4+}) -- 0x55a02f9720e0 con 0x55a02f963a30 2024-09-11T00:25:51.406 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:51.405+0000 7f822fe00640 1 -- 172.21.15.12:0/2027653129 <== mon.1 v2:172.21.15.178:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f82340043c0 con 0x55a02f963a30 2024-09-11T00:25:51.409 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:51.408+0000 7f822fe00640 1 -- 172.21.15.12:0/2027653129 <== mon.1 v2:172.21.15.178:3300/0 5 ==== fsmap(e 72) ==== 8386+0+0 (secure 0 0 0) 0x7f82440cc980 con 0x55a02f963a30 2024-09-11T00:25:51.409 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:51.408+0000 7f8253fc6740 10 client.27538 fetch_fsmap finished waiting for FSMap version 72 2024-09-11T00:25:51.410 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:51.408+0000 7f8253fc6740 10 client.27538 resolve_mds: validated gid 24674 aka daemon mds.b 2024-09-11T00:25:51.410 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:51.409+0000 7f8253fc6740 20 client.27538 populate_metadata read hostname 'smithi012' 2024-09-11T00:25:51.411 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:51.409+0000 7f8253fc6740 1 --2- 172.21.15.12:0/2027653129 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55a02f973540 0x55a02f975960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:25:51.411 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:51.409+0000 7f8253fc6740 4 client.27538 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T00:25:51.411 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:51.409+0000 7f8253fc6740 1 -- 172.21.15.12:0/2027653129 --> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a02f8cebf0 con 0x55a02f973540 2024-09-11T00:25:51.411 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:51.410+0000 7f824aa00640 1 --2- 172.21.15.12:0/2027653129 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55a02f973540 0x55a02f975960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:25:51.413 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:51.411+0000 7f824aa00640 1 --2- 172.21.15.12:0/2027653129 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55a02f973540 0x55a02f975960 crc :-1 s=READY pgs=265 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f9cbf30812552aad server_cookie=40ff9b42965731fe in_seq=0 out_seq=0 2024-09-11T00:25:51.413 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:51.411+0000 7f822fe00640 10 client.27538 ms_handle_connect on v2:172.21.15.165:6838/501312984 2024-09-11T00:25:51.414 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:51.412+0000 7f822fe00640 1 -- 172.21.15.12:0/2027653129 <== mds.0 v2:172.21.15.165:6838/501312984 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55a02f8cebf0 con 0x55a02f973540 2024-09-11T00:25:51.414 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:51.412+0000 7f822fe00640 10 client.27538 handle_command_reply: tid=0 2024-09-11T00:25:51.414 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:51.413+0000 7f8253fc6740 1 client.27538 shutdown 2024-09-11T00:25:51.415 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:51.413+0000 7f8253fc6740 1 -- 172.21.15.12:0/2027653129 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a02f963a30 msgr2=0x55a02f960e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:25:51.415 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:51.413+0000 7f8253fc6740 1 --2- 172.21.15.12:0/2027653129 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a02f963a30 0x55a02f960e60 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f8244104a60 tx=0x7f82440cd130 comp rx=0 tx=0).stop 2024-09-11T00:25:51.415 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:51.413+0000 7f8253fc6740 1 -- 172.21.15.12:0/2027653129 shutdown_connections 2024-09-11T00:25:51.415 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:51.413+0000 7f8253fc6740 1 --2- 172.21.15.12:0/2027653129 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55a02f973540 0x55a02f975960 crc :-1 s=READY pgs=265 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:51.415 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:51.413+0000 7f8253fc6740 1 --2- 172.21.15.12:0/2027653129 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a02f963a30 0x55a02f960e60 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:51.415 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:51.413+0000 7f8253fc6740 1 --2- 172.21.15.12:0/2027653129 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a02f957aa0 0x55a02f9607e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:51.415 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:51.413+0000 7f8253fc6740 1 --2- 172.21.15.12:0/2027653129 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a02f957180 0x55a02f963390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:51.415 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:51.413+0000 7f8253fc6740 1 -- 172.21.15.12:0/2027653129 >> 172.21.15.12:0/2027653129 conn(0x55a02f954fb0 msgr2=0x55a02f95ab50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:25:51.416 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:51.413+0000 7f822fe00640 0 client.27538 ms_handle_reset on v2:172.21.15.165:6838/501312984 2024-09-11T00:25:51.416 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:51.414+0000 7f8253fc6740 1 -- 172.21.15.12:0/2027653129 shutdown_connections 2024-09-11T00:25:51.416 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:51.414+0000 7f8253fc6740 1 -- 172.21.15.12:0/2027653129 wait complete. 2024-09-11T00:25:51.416 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:51.414+0000 7f8253fc6740 20 client.27538 trim_cache size 0 max 16384 2024-09-11T00:25:51.448 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:51.445+0000 7f8253fc6740 1 Processor -- start 2024-09-11T00:25:51.448 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:51.446+0000 7f8253fc6740 1 -- start start 2024-09-11T00:25:51.449 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:51.447+0000 7f8253fc6740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a02f957180 0x55a02f960070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:25:51.449 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:51.447+0000 7f8253fc6740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a02f957aa0 0x55a02f964e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:25:51.449 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:51.447+0000 7f8253fc6740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a02f963a30 0x55a02f965360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:25:51.449 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:51.447+0000 7f8253fc6740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f824c1b60e0 con 0x55a02f963a30 2024-09-11T00:25:51.449 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:51.447+0000 7f8253fc6740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f824c1b7cb0 con 0x55a02f957180 2024-09-11T00:25:51.449 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:51.447+0000 7f8253fc6740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f824c1b67b0 con 0x55a02f957aa0 2024-09-11T00:25:51.452 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:51.448+0000 7f824aa00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a02f957180 0x55a02f960070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:25:51.452 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:51.448+0000 7f824b400640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a02f963a30 0x55a02f965360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:25:51.452 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:51.449+0000 7f824aa00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a02f957180 0x55a02f960070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.12:42588/0 (socket says 172.21.15.12:42588) 2024-09-11T00:25:51.452 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:51.449+0000 7f824aa00640 1 -- 172.21.15.12:0/653222308 learned_addr learned my addr 172.21.15.12:0/653222308 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:25:51.452 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:51.451+0000 7f8243e00640 1 --2- 172.21.15.12:0/653222308 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a02f957aa0 0x55a02f964e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:25:51.453 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:51.451+0000 7f824aa00640 1 -- 172.21.15.12:0/653222308 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a02f957aa0 msgr2=0x55a02f964e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:25:51.453 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:51.451+0000 7f824aa00640 1 --2- 172.21.15.12:0/653222308 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a02f957aa0 0x55a02f964e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:51.453 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:51.451+0000 7f824aa00640 1 -- 172.21.15.12:0/653222308 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a02f963a30 msgr2=0x55a02f965360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:25:51.453 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:51.451+0000 7f824aa00640 1 --2- 172.21.15.12:0/653222308 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a02f963a30 0x55a02f965360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:51.453 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:51.451+0000 7f824aa00640 1 -- 172.21.15.12:0/653222308 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a02f956b60 con 0x55a02f957180 2024-09-11T00:25:51.453 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:51.452+0000 7f824aa00640 1 --2- 172.21.15.12:0/653222308 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a02f957180 0x55a02f960070 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f82340115f0 tx=0x7f823400b390 comp rx=0 tx=0).ready entity=mon.2 client_cookie=37525f3aa48b741a server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:25:51.453 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:51.452+0000 7f823aa00640 1 -- 172.21.15.12:0/653222308 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8234004440 con 0x55a02f957180 2024-09-11T00:25:51.454 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:51.452+0000 7f823aa00640 1 -- 172.21.15.12:0/653222308 <== mon.2 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f82340054f0 con 0x55a02f957180 2024-09-11T00:25:51.454 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:51.452+0000 7f823aa00640 1 -- 172.21.15.12:0/653222308 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8234004660 con 0x55a02f957180 2024-09-11T00:25:51.454 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:51.453+0000 7f8253fc6740 1 -- 172.21.15.12:0/653222308 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a02f957180 msgr2=0x55a02f960070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:25:51.454 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:51.453+0000 7f8253fc6740 1 --2- 172.21.15.12:0/653222308 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a02f957180 0x55a02f960070 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f82340115f0 tx=0x7f823400b390 comp rx=0 tx=0).stop 2024-09-11T00:25:51.455 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:51.453+0000 7f8253fc6740 1 -- 172.21.15.12:0/653222308 shutdown_connections 2024-09-11T00:25:51.455 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:51.453+0000 7f8253fc6740 1 --2- 172.21.15.12:0/653222308 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a02f963a30 0x55a02f965360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:51.455 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:51.453+0000 7f8253fc6740 1 --2- 172.21.15.12:0/653222308 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a02f957aa0 0x55a02f964e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:51.455 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:51.453+0000 7f8253fc6740 1 --2- 172.21.15.12:0/653222308 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a02f957180 0x55a02f960070 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:51.455 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:51.453+0000 7f8253fc6740 1 -- 172.21.15.12:0/653222308 >> 172.21.15.12:0/653222308 conn(0x55a02f954fb0 msgr2=0x55a02f962630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:25:51.455 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:51.454+0000 7f8253fc6740 1 -- 172.21.15.12:0/653222308 shutdown_connections 2024-09-11T00:25:51.456 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:51.454+0000 7f8253fc6740 1 -- 172.21.15.12:0/653222308 wait complete. 2024-09-11T00:25:51.456 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:51.454+0000 7f8253fc6740 1 -- start start 2024-09-11T00:25:51.456 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:51.455+0000 7f8253fc6740 1 Processor -- start 2024-09-11T00:25:51.457 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:51.455+0000 7f8253fc6740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a02f957180 0x55a02f96bbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:25:51.457 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:51.455+0000 7f8253fc6740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a02f957aa0 0x55a02f96c100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:25:51.457 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:51.455+0000 7f8253fc6740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a02f963a30 0x55a02f95ca00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:25:51.457 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:51.455+0000 7f8253fc6740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f824c1b75e0 con 0x55a02f957180 2024-09-11T00:25:51.457 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:51.455+0000 7f8253fc6740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f824c1b6ee0 con 0x55a02f963a30 2024-09-11T00:25:51.457 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:51.456+0000 7f8253fc6740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f824c1b4170 con 0x55a02f957aa0 2024-09-11T00:25:51.458 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:51.456+0000 7f8253fc6740 10 client.0 _ll_register_callbacks cb 0x7f82440c95c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:25:51.458 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:51.457+0000 7f8243e00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a02f957aa0 0x55a02f96c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:25:51.458 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:51.457+0000 7f8243e00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a02f957aa0 0x55a02f96c100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.12:36940/0 (socket says 172.21.15.12:36940) 2024-09-11T00:25:51.458 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:51.457+0000 7f824b400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a02f963a30 0x55a02f95ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:25:51.459 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:51.457+0000 7f824b400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a02f963a30 0x55a02f95ca00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.12:42604/0 (socket says 172.21.15.12:42604) 2024-09-11T00:25:51.459 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:51.457+0000 7f824b400640 1 -- 172.21.15.12:0/2078101248 learned_addr learned my addr 172.21.15.12:0/2078101248 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:25:51.459 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:51.457+0000 7f824aa00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a02f957180 0x55a02f96bbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:25:51.460 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:51.458+0000 7f824b400640 1 -- 172.21.15.12:0/2078101248 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a02f957aa0 msgr2=0x55a02f96c100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:25:51.460 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:51.458+0000 7f824b400640 1 --2- 172.21.15.12:0/2078101248 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a02f957aa0 0x55a02f96c100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:51.461 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:51.458+0000 7f824b400640 1 -- 172.21.15.12:0/2078101248 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a02f957180 msgr2=0x55a02f96bbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:25:51.461 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:51.458+0000 7f824b400640 1 --2- 172.21.15.12:0/2078101248 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a02f957180 0x55a02f96bbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:51.461 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:51.458+0000 7f824b400640 1 -- 172.21.15.12:0/2078101248 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a02f95d070 con 0x55a02f963a30 2024-09-11T00:25:51.461 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:51.458+0000 7f824b400640 1 --2- 172.21.15.12:0/2078101248 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a02f963a30 0x55a02f95ca00 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f824402f6f0 tx=0x7f82440d6a50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b88ad94f655949f7 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:25:51.461 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:51.458+0000 7f824aa00640 1 --2- 172.21.15.12:0/2078101248 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a02f957180 0x55a02f96bbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T00:25:51.461 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:51.458+0000 7f822fe00640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-09-11T00:25:51.461 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:51.459+0000 7f822fe00640 1 -- 172.21.15.12:0/2078101248 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8244004d40 con 0x55a02f963a30 2024-09-11T00:25:51.461 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:51.459+0000 7f8253fc6740 1 -- 172.21.15.12:0/2078101248 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f82340054f0 con 0x55a02f963a30 2024-09-11T00:25:51.461 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:51.459+0000 7f822fe00640 1 -- 172.21.15.12:0/2078101248 <== mon.2 v2:172.21.15.165:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9208+0+0 (secure 0 0 0) 0x7f8244004b40 con 0x55a02f963a30 2024-09-11T00:25:51.462 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:51.461+0000 7f822fe00640 1 -- 172.21.15.12:0/2078101248 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=72) ==== 24+0+0 (secure 0 0 0) 0x7f8244004d40 con 0x55a02f963a30 2024-09-11T00:25:51.462 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:51.461+0000 7f8253fc6740 10 client.27575 fetch_fsmap learned FSMap version 72 2024-09-11T00:25:51.463 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:51.461+0000 7f8253fc6740 1 -- 172.21.15.12:0/2078101248 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=72,monmap=4+}) -- 0x55a02f95d070 con 0x55a02f963a30 2024-09-11T00:25:51.463 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:51.462+0000 7f822fe00640 1 -- 172.21.15.12:0/2078101248 <== mon.2 v2:172.21.15.165:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f82340054f0 con 0x55a02f963a30 2024-09-11T00:25:51.463 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:51.462+0000 7f822fe00640 1 -- 172.21.15.12:0/2078101248 <== mon.2 v2:172.21.15.165:3300/0 5 ==== fsmap(e 72) ==== 8386+0+0 (secure 0 0 0) 0x7f82440cc3f0 con 0x55a02f963a30 2024-09-11T00:25:51.464 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:51.462+0000 7f8253fc6740 10 client.27575 fetch_fsmap finished waiting for FSMap version 72 2024-09-11T00:25:51.464 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:51.462+0000 7f8253fc6740 10 client.27575 resolve_mds: validated gid 24674 aka daemon mds.b 2024-09-11T00:25:51.464 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:51.462+0000 7f8253fc6740 20 client.27575 populate_metadata read hostname 'smithi012' 2024-09-11T00:25:51.464 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:51.462+0000 7f8253fc6740 1 --2- 172.21.15.12:0/2078101248 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55a02f9f2410 0x55a02f9f4800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:25:51.465 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:51.462+0000 7f8253fc6740 4 client.27575 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 141.0, "await_for": 45.0}] 2024-09-11T00:25:51.465 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:51.462+0000 7f8253fc6740 1 -- 172.21.15.12:0/2078101248 --> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 141.0, "await_for": 45.0}) -- 0x55a02f8ca6a0 con 0x55a02f9f2410 2024-09-11T00:25:51.465 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:51.464+0000 7f824aa00640 1 --2- 172.21.15.12:0/2078101248 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55a02f9f2410 0x55a02f9f4800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:25:51.466 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:51.464+0000 7f824aa00640 1 --2- 172.21.15.12:0/2078101248 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55a02f9f2410 0x55a02f9f4800 crc :-1 s=READY pgs=266 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ba8033bc838b367b server_cookie=468859cc295ea7ea in_seq=0 out_seq=0 2024-09-11T00:25:51.466 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:51.464+0000 7f822fe00640 10 client.27575 ms_handle_connect on v2:172.21.15.165:6838/501312984 2024-09-11T00:25:51.533 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:25:51 smithi165 ceph-mon[35356]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T00:25:51.533 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:25:51 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:25:51.533 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:25:51 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T00:25:51.533 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:25:51 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:25:51.581 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:25:51 smithi178 ceph-mon[35919]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T00:25:51.581 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:25:51 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:25:51.581 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:25:51 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T00:25:51.581 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:25:51 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:25:51.840 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:51.838+0000 7f822fe00640 1 -- 172.21.15.12:0/2078101248 <== mds.0 v2:172.21.15.165:6838/501312984 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55a02f8ca6a0 con 0x55a02f9f2410 2024-09-11T00:25:51.840 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:51.838+0000 7f822fe00640 10 client.27575 handle_command_reply: tid=0 2024-09-11T00:25:51.840 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:51.838+0000 7f8253fc6740 1 client.27575 shutdown 2024-09-11T00:25:51.840 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:51.838+0000 7f8253fc6740 1 -- 172.21.15.12:0/2078101248 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a02f963a30 msgr2=0x55a02f95ca00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:25:51.840 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:51.838+0000 7f8253fc6740 1 --2- 172.21.15.12:0/2078101248 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a02f963a30 0x55a02f95ca00 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f824402f6f0 tx=0x7f82440d6a50 comp rx=0 tx=0).stop 2024-09-11T00:25:51.840 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:51.838+0000 7f8253fc6740 1 -- 172.21.15.12:0/2078101248 shutdown_connections 2024-09-11T00:25:51.840 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:51.838+0000 7f8253fc6740 1 --2- 172.21.15.12:0/2078101248 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55a02f9f2410 0x55a02f9f4800 crc :-1 s=READY pgs=266 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:51.840 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:51.838+0000 7f8253fc6740 1 --2- 172.21.15.12:0/2078101248 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a02f963a30 0x55a02f95ca00 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:51.840 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:51.838+0000 7f8253fc6740 1 --2- 172.21.15.12:0/2078101248 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a02f957aa0 0x55a02f96c100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:51.840 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:51.838+0000 7f8253fc6740 1 --2- 172.21.15.12:0/2078101248 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a02f957180 0x55a02f96bbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:51.841 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:51.838+0000 7f8253fc6740 1 -- 172.21.15.12:0/2078101248 >> 172.21.15.12:0/2078101248 conn(0x55a02f954fb0 msgr2=0x55a02f953890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:25:51.841 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:51.838+0000 7f822fe00640 0 client.27575 ms_handle_reset on v2:172.21.15.165:6838/501312984 2024-09-11T00:25:51.841 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:51.839+0000 7f8253fc6740 1 -- 172.21.15.12:0/2078101248 shutdown_connections 2024-09-11T00:25:51.841 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:51.839+0000 7f8253fc6740 1 -- 172.21.15.12:0/2078101248 wait complete. 2024-09-11T00:25:51.841 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:51.839+0000 7f8253fc6740 20 client.27575 trim_cache size 0 max 16384 2024-09-11T00:25:51.841 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-09-11T00:25:51.842 INFO:teuthology.orchestra.run.smithi012.stdout: "epoch": 72, 2024-09-11T00:25:51.842 INFO:teuthology.orchestra.run.smithi012.stdout: "leader": 24674, 2024-09-11T00:25:51.842 INFO:teuthology.orchestra.run.smithi012.stdout: "set_version": 58, 2024-09-11T00:25:51.842 INFO:teuthology.orchestra.run.smithi012.stdout: "sets": { 2024-09-11T00:25:51.842 INFO:teuthology.orchestra.run.smithi012.stdout: "4921b205": { 2024-09-11T00:25:51.842 INFO:teuthology.orchestra.run.smithi012.stdout: "version": 58, 2024-09-11T00:25:51.842 INFO:teuthology.orchestra.run.smithi012.stdout: "age_ref": 0.0, 2024-09-11T00:25:51.842 INFO:teuthology.orchestra.run.smithi012.stdout: "state": { 2024-09-11T00:25:51.842 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "QUIESCED", 2024-09-11T00:25:51.842 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 0.0 2024-09-11T00:25:51.842 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:25:51.842 INFO:teuthology.orchestra.run.smithi012.stdout: "timeout": 90.0, 2024-09-11T00:25:51.842 INFO:teuthology.orchestra.run.smithi012.stdout: "expiration": 141.0, 2024-09-11T00:25:51.843 INFO:teuthology.orchestra.run.smithi012.stdout: "members": { 2024-09-11T00:25:51.843 INFO:teuthology.orchestra.run.smithi012.stdout: "file:/": { 2024-09-11T00:25:51.843 INFO:teuthology.orchestra.run.smithi012.stdout: "excluded": false, 2024-09-11T00:25:51.843 INFO:teuthology.orchestra.run.smithi012.stdout: "state": { 2024-09-11T00:25:51.843 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "QUIESCED", 2024-09-11T00:25:51.843 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 0.0 2024-09-11T00:25:51.843 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:25:51.843 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:25:51.843 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:25:51.843 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:25:51.843 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:25:51.843 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-09-11T00:25:51.844 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:51.841+0000 7f8251600640 1 -- 172.21.15.12:0/273558055 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f82300b5460 msgr2=0x7f82300b7920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:25:51.844 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:51.841+0000 7f8251600640 1 --2- 172.21.15.12:0/273558055 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f82300b5460 0x7f82300b7920 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f823c0058c0 tx=0x7f823c00bda0 comp rx=0 tx=0).stop 2024-09-11T00:25:51.844 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:51.841+0000 7f8251600640 1 -- 172.21.15.12:0/273558055 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f824c113990 msgr2=0x7f824c1a4f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:25:51.844 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:51.841+0000 7f8251600640 1 --2- 172.21.15.12:0/273558055 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f824c113990 0x7f824c1a4f60 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f8244005e30 tx=0x7f82440093d0 comp rx=0 tx=0).stop 2024-09-11T00:25:51.844 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:51.841+0000 7f8251600640 1 -- 172.21.15.12:0/273558055 shutdown_connections 2024-09-11T00:25:51.844 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:51.841+0000 7f8251600640 1 --2- 172.21.15.12:0/273558055 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f82300b5460 0x7f82300b7920 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:51.844 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:51.841+0000 7f8251600640 1 --2- 172.21.15.12:0/273558055 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f824c113990 0x7f824c1a4f60 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:51.845 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:51.841+0000 7f8251600640 1 --2- 172.21.15.12:0/273558055 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f824c077ff0 0x7f824c19fbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:51.845 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:51.841+0000 7f8251600640 1 --2- 172.21.15.12:0/273558055 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f824c0776d0 0x7f824c19f680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:51.845 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:51.841+0000 7f8251600640 1 -- 172.21.15.12:0/273558055 >> 172.21.15.12:0/273558055 conn(0x7f824c100a50 msgr2=0x7f824c102e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:25:51.845 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:51.842+0000 7f8251600640 1 -- 172.21.15.12:0/273558055 shutdown_connections 2024-09-11T00:25:51.845 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:25:51.842+0000 7f8251600640 1 -- 172.21.15.12:0/273558055 wait complete. 2024-09-11T00:25:52.206 INFO:tasks.workunit.client.0.smithi012.stdout: 10 103757 16.72 MB/sec execute 228 sec latency 755.193 ms 2024-09-11T00:25:52.532 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:25:52 smithi165 ceph-mon[35356]: pgmap v962: 129 pgs: 129 active+clean; 582 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 1.3 MiB/s wr, 248 op/s 2024-09-11T00:25:52.563 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:25:52 smithi012 ceph-mon[30333]: pgmap v962: 129 pgs: 129 active+clean; 582 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 1.3 MiB/s wr, 248 op/s 2024-09-11T00:25:52.581 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:25:52 smithi178 ceph-mon[35919]: pgmap v962: 129 pgs: 129 active+clean; 582 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 1.3 MiB/s wr, 248 op/s 2024-09-11T00:25:53.206 INFO:tasks.workunit.client.0.smithi012.stdout: 10 103757 16.65 MB/sec execute 229 sec latency 1755.291 ms 2024-09-11T00:25:54.183 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '4921b205', quiesce took 8.4 seconds. Will release after: 12.6 2024-09-11T00:25:54.206 INFO:tasks.workunit.client.0.smithi012.stdout: 10 103757 16.58 MB/sec execute 230 sec latency 2755.409 ms 2024-09-11T00:25:54.532 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:25:54 smithi165 ceph-mon[35356]: pgmap v963: 129 pgs: 129 active+clean; 592 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 2.3 MiB/s wr, 362 op/s 2024-09-11T00:25:54.563 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:25:54 smithi012 ceph-mon[30333]: pgmap v963: 129 pgs: 129 active+clean; 592 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 2.3 MiB/s wr, 362 op/s 2024-09-11T00:25:54.581 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:25:54 smithi178 ceph-mon[35919]: pgmap v963: 129 pgs: 129 active+clean; 592 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 2.3 MiB/s wr, 362 op/s 2024-09-11T00:25:55.206 INFO:tasks.workunit.client.0.smithi012.stdout: 10 103757 16.51 MB/sec execute 231 sec latency 3755.541 ms 2024-09-11T00:25:56.206 INFO:tasks.workunit.client.0.smithi012.stdout: 10 103757 16.43 MB/sec execute 232 sec latency 4755.674 ms 2024-09-11T00:25:56.532 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:25:56 smithi165 ceph-mon[35356]: pgmap v964: 129 pgs: 129 active+clean; 592 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 2.2 MiB/s wr, 348 op/s 2024-09-11T00:25:56.563 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:25:56 smithi012 ceph-mon[30333]: pgmap v964: 129 pgs: 129 active+clean; 592 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 2.2 MiB/s wr, 348 op/s 2024-09-11T00:25:56.581 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:25:56 smithi178 ceph-mon[35919]: pgmap v964: 129 pgs: 129 active+clean; 592 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 2.2 MiB/s wr, 348 op/s 2024-09-11T00:25:57.206 INFO:tasks.workunit.client.0.smithi012.stdout: 10 103757 16.36 MB/sec execute 233 sec latency 5755.825 ms 2024-09-11T00:25:58.206 INFO:tasks.workunit.client.0.smithi012.stdout: 10 103757 16.29 MB/sec execute 234 sec latency 6755.986 ms 2024-09-11T00:25:58.532 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:25:58 smithi165 ceph-mon[35356]: pgmap v965: 129 pgs: 129 active+clean; 589 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 2.6 MiB/s wr, 457 op/s 2024-09-11T00:25:58.563 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:25:58 smithi012 ceph-mon[30333]: pgmap v965: 129 pgs: 129 active+clean; 589 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 2.6 MiB/s wr, 457 op/s 2024-09-11T00:25:58.581 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:25:58 smithi178 ceph-mon[35919]: pgmap v965: 129 pgs: 129 active+clean; 589 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 2.6 MiB/s wr, 457 op/s 2024-09-11T00:25:59.207 INFO:tasks.workunit.client.0.smithi012.stdout: 10 103757 16.22 MB/sec execute 235 sec latency 7756.129 ms 2024-09-11T00:26:00.207 INFO:tasks.workunit.client.0.smithi012.stdout: 10 103757 16.16 MB/sec execute 236 sec latency 8756.264 ms 2024-09-11T00:26:00.532 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:26:00 smithi165 ceph-mon[35356]: pgmap v966: 129 pgs: 129 active+clean; 589 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 2.5 MiB/s wr, 422 op/s 2024-09-11T00:26:00.563 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:26:00 smithi012 ceph-mon[30333]: pgmap v966: 129 pgs: 129 active+clean; 589 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 2.5 MiB/s wr, 422 op/s 2024-09-11T00:26:00.581 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:26:00 smithi178 ceph-mon[35919]: pgmap v966: 129 pgs: 129 active+clean; 589 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 2.5 MiB/s wr, 422 op/s 2024-09-11T00:26:01.207 INFO:tasks.workunit.client.0.smithi012.stdout: 10 103757 16.09 MB/sec execute 237 sec latency 9756.407 ms 2024-09-11T00:26:02.207 INFO:tasks.workunit.client.0.smithi012.stdout: 10 103757 16.02 MB/sec execute 238 sec latency 10756.598 ms 2024-09-11T00:26:02.533 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:26:02 smithi165 ceph-mon[35356]: pgmap v967: 129 pgs: 129 active+clean; 589 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 2.4 MiB/s wr, 423 op/s 2024-09-11T00:26:02.563 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:26:02 smithi012 ceph-mon[30333]: pgmap v967: 129 pgs: 129 active+clean; 589 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 2.4 MiB/s wr, 423 op/s 2024-09-11T00:26:02.581 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:26:02 smithi178 ceph-mon[35919]: pgmap v967: 129 pgs: 129 active+clean; 589 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 2.4 MiB/s wr, 423 op/s 2024-09-11T00:26:03.207 INFO:tasks.workunit.client.0.smithi012.stdout: 10 103757 15.95 MB/sec execute 239 sec latency 11756.743 ms 2024-09-11T00:26:04.207 INFO:tasks.workunit.client.0.smithi012.stdout: 10 103757 15.89 MB/sec execute 240 sec latency 12756.835 ms 2024-09-11T00:26:04.272 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:04.270+0000 7f249a200640 1 -- 172.21.15.12:0/1798127990 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2494105b00 msgr2=0x7f2494105f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:26:04.273 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:04.270+0000 7f249a200640 1 --2- 172.21.15.12:0/1798127990 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2494105b00 0x7f2494105f80 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f247c009930 tx=0x7f247c02a420 comp rx=0 tx=0).stop 2024-09-11T00:26:04.273 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:04.270+0000 7f249a200640 1 -- 172.21.15.12:0/1798127990 shutdown_connections 2024-09-11T00:26:04.273 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:04.270+0000 7f249a200640 1 --2- 172.21.15.12:0/1798127990 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f24941064c0 0x7f249410ce40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:26:04.273 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:04.270+0000 7f249a200640 1 --2- 172.21.15.12:0/1798127990 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2494105b00 0x7f2494105f80 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:26:04.273 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:04.270+0000 7f249a200640 1 --2- 172.21.15.12:0/1798127990 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f24941049b0 0x7f2494104db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:26:04.273 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:04.270+0000 7f249a200640 1 -- 172.21.15.12:0/1798127990 >> 172.21.15.12:0/1798127990 conn(0x7f2494100250 msgr2=0x7f2494102690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:26:04.273 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:04.270+0000 7f249a200640 1 -- 172.21.15.12:0/1798127990 shutdown_connections 2024-09-11T00:26:04.273 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:04.270+0000 7f249a200640 1 -- 172.21.15.12:0/1798127990 wait complete. 2024-09-11T00:26:04.275 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:04.271+0000 7f249a200640 1 Processor -- start 2024-09-11T00:26:04.275 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:04.271+0000 7f249a200640 1 -- start start 2024-09-11T00:26:04.275 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:04.271+0000 7f249a200640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f24941049b0 0x7f249419adc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:26:04.275 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:04.271+0000 7f249a200640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2494105b00 0x7f249419b410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:26:04.276 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:04.271+0000 7f249a200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f24941064c0 0x7f249419c040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:26:04.276 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:04.271+0000 7f249a200640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f2494110600 con 0x7f2494105b00 2024-09-11T00:26:04.276 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:04.271+0000 7f249a200640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f2494110480 con 0x7f24941064c0 2024-09-11T00:26:04.276 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:04.271+0000 7f249a200640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f2494110780 con 0x7f24941049b0 2024-09-11T00:26:04.276 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:04.272+0000 7f2498e00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f24941049b0 0x7f249419adc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:26:04.276 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:04.272+0000 7f248be00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2494105b00 0x7f249419b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:26:04.276 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:04.272+0000 7f2498e00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f24941049b0 0x7f249419adc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.12:60580/0 (socket says 172.21.15.12:60580) 2024-09-11T00:26:04.276 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:04.272+0000 7f2498e00640 1 -- 172.21.15.12:0/3695060963 learned_addr learned my addr 172.21.15.12:0/3695060963 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:26:04.276 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:04.272+0000 7f2499800640 1 --2- 172.21.15.12:0/3695060963 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f24941064c0 0x7f249419c040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:26:04.276 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:04.272+0000 7f248be00640 1 -- 172.21.15.12:0/3695060963 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f24941049b0 msgr2=0x7f249419adc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:26:04.276 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:04.272+0000 7f248be00640 1 --2- 172.21.15.12:0/3695060963 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f24941049b0 0x7f249419adc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:26:04.277 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:04.272+0000 7f248be00640 1 -- 172.21.15.12:0/3695060963 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f24941064c0 msgr2=0x7f249419c040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:26:04.277 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:04.272+0000 7f248be00640 1 --2- 172.21.15.12:0/3695060963 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f24941064c0 0x7f249419c040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:26:04.277 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:04.272+0000 7f248be00640 1 -- 172.21.15.12:0/3695060963 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f249419c900 con 0x7f2494105b00 2024-09-11T00:26:04.277 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:04.272+0000 7f2498e00640 1 --2- 172.21.15.12:0/3695060963 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f24941049b0 0x7f249419adc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:26:04.277 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:04.273+0000 7f248be00640 1 --2- 172.21.15.12:0/3695060963 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2494105b00 0x7f249419b410 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f247c009230 tx=0x7f247c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:26:04.277 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:04.273+0000 7f2492000640 1 -- 172.21.15.12:0/3695060963 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f247c03f030 con 0x7f2494105b00 2024-09-11T00:26:04.277 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:04.273+0000 7f249a200640 1 -- 172.21.15.12:0/3695060963 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f249419cb90 con 0x7f2494105b00 2024-09-11T00:26:04.277 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:04.273+0000 7f2492000640 1 -- 172.21.15.12:0/3695060963 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f247c031910 con 0x7f2494105b00 2024-09-11T00:26:04.277 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:04.273+0000 7f249a200640 1 -- 172.21.15.12:0/3695060963 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24941a32c0 con 0x7f2494105b00 2024-09-11T00:26:04.277 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:04.273+0000 7f2492000640 1 -- 172.21.15.12:0/3695060963 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f247c049060 con 0x7f2494105b00 2024-09-11T00:26:04.279 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:04.276+0000 7f249e62c740 1 Processor -- start 2024-09-11T00:26:04.280 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:04.276+0000 7f249e62c740 1 -- start start 2024-09-11T00:26:04.280 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:04.277+0000 7f2492000640 1 -- 172.21.15.12:0/3695060963 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7f247c03b980 con 0x7f2494105b00 2024-09-11T00:26:04.280 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:04.277+0000 7f249e62c740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564104230970 0x564104232d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:26:04.280 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:04.277+0000 7f249e62c740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5641042332a0 0x564104235690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:26:04.280 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:04.277+0000 7f249e62c740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564104235ce0 0x5641042381a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:26:04.280 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:04.277+0000 7f249e62c740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x5641041a36b0 con 0x564104230970 2024-09-11T00:26:04.280 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:04.277+0000 7f249e62c740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x564104192ae0 con 0x5641042332a0 2024-09-11T00:26:04.280 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:04.277+0000 7f249e62c740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5641041968f0 con 0x564104235ce0 2024-09-11T00:26:04.280 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:04.277+0000 7f2498e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5641042332a0 0x564104235690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:26:04.280 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:04.277+0000 7f2498e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5641042332a0 0x564104235690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.12:42786/0 (socket says 172.21.15.12:42786) 2024-09-11T00:26:04.280 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:04.277+0000 7f2499800640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564104230970 0x564104232d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:26:04.281 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:04.277+0000 7f2498e00640 1 -- 172.21.15.12:0/156624910 learned_addr learned my addr 172.21.15.12:0/156624910 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:26:04.281 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:04.277+0000 7f248be00640 1 --2- 172.21.15.12:0/156624910 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564104235ce0 0x5641042381a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:26:04.282 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:04.278+0000 7f2498e00640 1 -- 172.21.15.12:0/156624910 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564104235ce0 msgr2=0x5641042381a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:26:04.282 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:04.278+0000 7f2498e00640 1 --2- 172.21.15.12:0/156624910 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564104235ce0 0x5641042381a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:26:04.283 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:04.278+0000 7f2498e00640 1 -- 172.21.15.12:0/156624910 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564104230970 msgr2=0x564104232d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:26:04.283 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:04.278+0000 7f2498e00640 1 --2- 172.21.15.12:0/156624910 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564104230970 0x564104232d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:26:04.283 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:04.278+0000 7f2498e00640 1 -- 172.21.15.12:0/156624910 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56410415f780 con 0x5641042332a0 2024-09-11T00:26:04.283 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:04.278+0000 7f2498e00640 1 --2- 172.21.15.12:0/156624910 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5641042332a0 0x564104235690 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f2484011650 tx=0x7f2484009f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e3ab85ec152035e3 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:26:04.283 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:04.278+0000 7f248a000640 1 -- 172.21.15.12:0/156624910 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f248401e070 con 0x5641042332a0 2024-09-11T00:26:04.283 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:04.279+0000 7f248a000640 1 -- 172.21.15.12:0/156624910 <== mon.2 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f248400fb70 con 0x5641042332a0 2024-09-11T00:26:04.283 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:04.279+0000 7f248a000640 1 -- 172.21.15.12:0/156624910 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2484019c30 con 0x5641042332a0 2024-09-11T00:26:04.283 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:04.279+0000 7f2492000640 1 --2- 172.21.15.12:0/3695060963 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f24780b52e0 0x7f24780b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:26:04.283 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:04.279+0000 7f2492000640 1 -- 172.21.15.12:0/3695060963 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9208+0+0 (secure 0 0 0) 0x7f247c106a80 con 0x7f2494105b00 2024-09-11T00:26:04.283 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:04.279+0000 7f2499800640 1 --2- 172.21.15.12:0/3695060963 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f24780b52e0 0x7f24780b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:26:04.283 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:04.279+0000 7f249e62c740 1 -- 172.21.15.12:0/156624910 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5641042332a0 msgr2=0x564104235690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:26:04.283 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:04.279+0000 7f249e62c740 1 --2- 172.21.15.12:0/156624910 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5641042332a0 0x564104235690 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f2484011650 tx=0x7f2484009f90 comp rx=0 tx=0).stop 2024-09-11T00:26:04.284 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:04.279+0000 7f249e62c740 1 -- 172.21.15.12:0/156624910 shutdown_connections 2024-09-11T00:26:04.284 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:04.279+0000 7f249e62c740 1 --2- 172.21.15.12:0/156624910 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564104235ce0 0x5641042381a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:26:04.284 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:04.279+0000 7f249e62c740 1 --2- 172.21.15.12:0/156624910 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5641042332a0 0x564104235690 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:26:04.284 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:04.279+0000 7f249e62c740 1 --2- 172.21.15.12:0/156624910 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564104230970 0x564104232d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:26:04.284 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:04.279+0000 7f249e62c740 1 -- 172.21.15.12:0/156624910 >> 172.21.15.12:0/156624910 conn(0x56410422bac0 msgr2=0x56410422c850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:26:04.284 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:04.280+0000 7f249e62c740 1 -- 172.21.15.12:0/156624910 shutdown_connections 2024-09-11T00:26:04.284 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:04.280+0000 7f249e62c740 1 -- 172.21.15.12:0/156624910 wait complete. 2024-09-11T00:26:04.284 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:04.280+0000 7f2499800640 1 --2- 172.21.15.12:0/3695060963 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f24780b52e0 0x7f24780b77a0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f249419bb40 tx=0x7f248c009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:26:04.284 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:04.280+0000 7f249e62c740 1 -- start start 2024-09-11T00:26:04.284 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:04.280+0000 7f249e62c740 1 Processor -- start 2024-09-11T00:26:04.288 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:04.281+0000 7f249e62c740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564104230970 0x56410423a070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:26:04.288 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:04.281+0000 7f249e62c740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5641042332a0 0x56410423a710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:26:04.288 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:04.281+0000 7f249e62c740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564104235ce0 0x56410423b2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:26:04.288 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:04.281+0000 7f249e62c740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x5641040caab0 con 0x564104235ce0 2024-09-11T00:26:04.288 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:04.281+0000 7f249e62c740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x564104139570 con 0x564104230970 2024-09-11T00:26:04.288 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:04.281+0000 7f249e62c740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x564104238cc0 con 0x5641042332a0 2024-09-11T00:26:04.288 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:04.281+0000 7f249e62c740 10 client.0 _ll_register_callbacks cb 0x564104148100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:26:04.288 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:04.281+0000 7f2499800640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564104235ce0 0x56410423b2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:26:04.288 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:04.281+0000 7f248be00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5641042332a0 0x56410423a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:26:04.288 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:04.281+0000 7f2498e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564104230970 0x56410423a070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:26:04.289 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:04.281+0000 7f248be00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5641042332a0 0x56410423a710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.12:60596/0 (socket says 172.21.15.12:60596) 2024-09-11T00:26:04.289 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:04.281+0000 7f2499800640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564104235ce0 0x56410423b2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:34696/0 (socket says 172.21.15.12:34696) 2024-09-11T00:26:04.289 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:04.281+0000 7f2499800640 1 -- 172.21.15.12:0/3878981399 learned_addr learned my addr 172.21.15.12:0/3878981399 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:26:04.289 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:04.281+0000 7f2498e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564104230970 0x56410423a070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.12:42802/0 (socket says 172.21.15.12:42802) 2024-09-11T00:26:04.289 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:04.281+0000 7f2499800640 1 -- 172.21.15.12:0/3878981399 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5641042332a0 msgr2=0x56410423a710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:26:04.289 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:04.281+0000 7f2499800640 1 --2- 172.21.15.12:0/3878981399 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5641042332a0 0x56410423a710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:26:04.289 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:04.281+0000 7f2499800640 1 -- 172.21.15.12:0/3878981399 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564104230970 msgr2=0x56410423a070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:26:04.289 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:04.281+0000 7f2499800640 1 --2- 172.21.15.12:0/3878981399 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564104230970 0x56410423a070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:26:04.289 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:04.281+0000 7f2499800640 1 -- 172.21.15.12:0/3878981399 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564103ef4790 con 0x564104235ce0 2024-09-11T00:26:04.289 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:04.281+0000 7f2498e00640 1 --2- 172.21.15.12:0/3878981399 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564104230970 0x56410423a070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:26:04.289 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:04.282+0000 7f2499800640 1 --2- 172.21.15.12:0/3878981399 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564104235ce0 0x56410423b2e0 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f248c004060 tx=0x7f248c004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16c2547c85687a7c server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:26:04.289 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:04.282+0000 7f2482a00640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-09-11T00:26:04.289 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:04.282+0000 7f2482a00640 1 -- 172.21.15.12:0/3878981399 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f248c00f8b0 con 0x564104235ce0 2024-09-11T00:26:04.290 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:04.282+0000 7f2482a00640 1 -- 172.21.15.12:0/3878981399 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9208+0+0 (secure 0 0 0) 0x7f248c00fa50 con 0x564104235ce0 2024-09-11T00:26:04.290 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:04.282+0000 7f249e62c740 1 -- 172.21.15.12:0/3878981399 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f248400fb70 con 0x564104235ce0 2024-09-11T00:26:04.290 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:04.282+0000 7f2482a00640 1 -- 172.21.15.12:0/3878981399 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=72) ==== 24+0+0 (secure 0 0 0) 0x7f248c00fc50 con 0x564104235ce0 2024-09-11T00:26:04.290 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:04.282+0000 7f249e62c740 10 client.17562 fetch_fsmap learned FSMap version 72 2024-09-11T00:26:04.290 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:04.282+0000 7f249e62c740 1 -- 172.21.15.12:0/3878981399 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=72,monmap=4+}) -- 0x564103f6d620 con 0x564104235ce0 2024-09-11T00:26:04.290 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:04.283+0000 7f2482a00640 1 -- 172.21.15.12:0/3878981399 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f248400fb70 con 0x564104235ce0 2024-09-11T00:26:04.290 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:04.283+0000 7f2482a00640 1 -- 172.21.15.12:0/3878981399 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 72) ==== 8386+0+0 (secure 0 0 0) 0x7f248c026070 con 0x564104235ce0 2024-09-11T00:26:04.290 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:04.283+0000 7f249e62c740 10 client.17562 fetch_fsmap finished waiting for FSMap version 72 2024-09-11T00:26:04.290 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:04.283+0000 7f249e62c740 10 client.17562 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T00:26:04.290 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:04.283+0000 7f249e62c740 20 client.17562 populate_metadata read hostname 'smithi012' 2024-09-11T00:26:04.290 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:04.283+0000 7f249e62c740 1 --2- 172.21.15.12:0/3878981399 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x564104245a40 0x564104247ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:26:04.290 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:04.283+0000 7f249e62c740 4 client.17562 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T00:26:04.290 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:04.283+0000 7f249e62c740 1 -- 172.21.15.12:0/3878981399 --> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5641041a5600 con 0x564104245a40 2024-09-11T00:26:04.290 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:04.283+0000 7f2498e00640 1 --2- 172.21.15.12:0/3878981399 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x564104245a40 0x564104247ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:26:04.291 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:04.284+0000 7f2498e00640 1 --2- 172.21.15.12:0/3878981399 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x564104245a40 0x564104247ec0 crc :-1 s=READY pgs=267 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=787a83bc84d54dd0 server_cookie=d7801274d47db63 in_seq=0 out_seq=0 2024-09-11T00:26:04.291 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:04.284+0000 7f2482a00640 10 client.17562 ms_handle_connect on v2:172.21.15.165:6838/501312984 2024-09-11T00:26:04.291 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:04.285+0000 7f2482a00640 1 -- 172.21.15.12:0/3878981399 <== mds.0 v2:172.21.15.165:6838/501312984 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5641041a5600 con 0x564104245a40 2024-09-11T00:26:04.291 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:04.285+0000 7f2482a00640 10 client.17562 handle_command_reply: tid=0 2024-09-11T00:26:04.291 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:04.285+0000 7f249e62c740 1 client.17562 shutdown 2024-09-11T00:26:04.291 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:04.285+0000 7f249e62c740 1 -- 172.21.15.12:0/3878981399 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564104235ce0 msgr2=0x56410423b2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:26:04.291 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:04.285+0000 7f249e62c740 1 --2- 172.21.15.12:0/3878981399 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564104235ce0 0x56410423b2e0 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f248c004060 tx=0x7f248c004ab0 comp rx=0 tx=0).stop 2024-09-11T00:26:04.291 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:04.285+0000 7f249e62c740 1 -- 172.21.15.12:0/3878981399 shutdown_connections 2024-09-11T00:26:04.291 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:04.285+0000 7f249e62c740 1 --2- 172.21.15.12:0/3878981399 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x564104245a40 0x564104247ec0 crc :-1 s=READY pgs=267 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:26:04.291 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:04.285+0000 7f249e62c740 1 --2- 172.21.15.12:0/3878981399 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564104235ce0 0x56410423b2e0 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:26:04.291 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:04.285+0000 7f249e62c740 1 --2- 172.21.15.12:0/3878981399 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5641042332a0 0x56410423a710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:26:04.291 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:04.285+0000 7f2482a00640 0 client.17562 ms_handle_reset on v2:172.21.15.165:6838/501312984 2024-09-11T00:26:04.291 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:04.285+0000 7f249e62c740 1 --2- 172.21.15.12:0/3878981399 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564104230970 0x56410423a070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:26:04.292 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:04.285+0000 7f249e62c740 1 -- 172.21.15.12:0/3878981399 >> 172.21.15.12:0/3878981399 conn(0x56410422bac0 msgr2=0x56410422abc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:26:04.292 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:04.285+0000 7f249e62c740 1 -- 172.21.15.12:0/3878981399 shutdown_connections 2024-09-11T00:26:04.292 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:04.285+0000 7f249e62c740 1 -- 172.21.15.12:0/3878981399 wait complete. 2024-09-11T00:26:04.292 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:04.286+0000 7f249e62c740 20 client.17562 trim_cache size 0 max 16384 2024-09-11T00:26:04.307 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:04.301+0000 7f249e62c740 1 Processor -- start 2024-09-11T00:26:04.307 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:04.301+0000 7f249e62c740 1 -- start start 2024-09-11T00:26:04.307 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:04.301+0000 7f249e62c740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564104230970 0x5641042397d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:26:04.307 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:04.301+0000 7f249e62c740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5641042332a0 0x56410422fca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:26:04.307 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:04.301+0000 7f249e62c740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564104235ce0 0x5641042301e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:26:04.308 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:04.301+0000 7f249e62c740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f24941ad760 con 0x564104235ce0 2024-09-11T00:26:04.308 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:04.301+0000 7f249e62c740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f24941af330 con 0x5641042332a0 2024-09-11T00:26:04.308 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:04.301+0000 7f249e62c740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f24941ade30 con 0x564104230970 2024-09-11T00:26:04.308 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:04.302+0000 7f2498e00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564104230970 0x5641042397d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:26:04.308 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:04.302+0000 7f2499800640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564104235ce0 0x5641042301e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:26:04.308 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:04.302+0000 7f2498e00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564104230970 0x5641042397d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.12:60612/0 (socket says 172.21.15.12:60612) 2024-09-11T00:26:04.308 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:04.302+0000 7f2498e00640 1 -- 172.21.15.12:0/1626749902 learned_addr learned my addr 172.21.15.12:0/1626749902 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:26:04.308 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:04.302+0000 7f248be00640 1 --2- 172.21.15.12:0/1626749902 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5641042332a0 0x56410422fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:26:04.308 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:04.302+0000 7f2498e00640 1 -- 172.21.15.12:0/1626749902 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5641042332a0 msgr2=0x56410422fca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:26:04.308 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:04.302+0000 7f2498e00640 1 --2- 172.21.15.12:0/1626749902 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5641042332a0 0x56410422fca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:26:04.308 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:04.302+0000 7f2498e00640 1 -- 172.21.15.12:0/1626749902 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564104235ce0 msgr2=0x5641042301e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:26:04.308 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:04.302+0000 7f2498e00640 1 --2- 172.21.15.12:0/1626749902 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564104235ce0 0x5641042301e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:26:04.308 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:04.302+0000 7f2498e00640 1 -- 172.21.15.12:0/1626749902 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5641042330d0 con 0x564104230970 2024-09-11T00:26:04.309 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:04.302+0000 7f2499800640 1 --2- 172.21.15.12:0/1626749902 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564104235ce0 0x5641042301e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:26:04.309 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:04.302+0000 7f2498e00640 1 --2- 172.21.15.12:0/1626749902 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564104230970 0x5641042397d0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f2484011fd0 tx=0x7f2484011580 comp rx=0 tx=0).ready entity=mon.1 client_cookie=edd55396e97434be server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:26:04.309 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:04.303+0000 7f2489600640 1 -- 172.21.15.12:0/1626749902 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f248401e040 con 0x564104230970 2024-09-11T00:26:04.309 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:04.303+0000 7f2489600640 1 -- 172.21.15.12:0/1626749902 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f248400faa0 con 0x564104230970 2024-09-11T00:26:04.309 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:04.303+0000 7f2489600640 1 -- 172.21.15.12:0/1626749902 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2484019940 con 0x564104230970 2024-09-11T00:26:04.309 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:04.303+0000 7f249e62c740 1 -- 172.21.15.12:0/1626749902 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564104230970 msgr2=0x5641042397d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:26:04.309 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:04.303+0000 7f249e62c740 1 --2- 172.21.15.12:0/1626749902 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564104230970 0x5641042397d0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f2484011fd0 tx=0x7f2484011580 comp rx=0 tx=0).stop 2024-09-11T00:26:04.309 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:04.303+0000 7f249e62c740 1 -- 172.21.15.12:0/1626749902 shutdown_connections 2024-09-11T00:26:04.309 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:04.303+0000 7f249e62c740 1 --2- 172.21.15.12:0/1626749902 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564104235ce0 0x5641042301e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:26:04.310 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:04.303+0000 7f249e62c740 1 --2- 172.21.15.12:0/1626749902 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5641042332a0 0x56410422fca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:26:04.310 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:04.303+0000 7f249e62c740 1 --2- 172.21.15.12:0/1626749902 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564104230970 0x5641042397d0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:26:04.310 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:04.303+0000 7f249e62c740 1 -- 172.21.15.12:0/1626749902 >> 172.21.15.12:0/1626749902 conn(0x56410422bac0 msgr2=0x5641042390c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:26:04.310 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:04.303+0000 7f249e62c740 1 -- 172.21.15.12:0/1626749902 shutdown_connections 2024-09-11T00:26:04.310 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:04.303+0000 7f249e62c740 1 -- 172.21.15.12:0/1626749902 wait complete. 2024-09-11T00:26:04.310 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:04.304+0000 7f249e62c740 1 -- start start 2024-09-11T00:26:04.310 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:04.304+0000 7f249e62c740 1 Processor -- start 2024-09-11T00:26:04.310 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:04.304+0000 7f249e62c740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564104230970 0x564104241050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:26:04.310 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:04.304+0000 7f249e62c740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5641042332a0 0x564104241680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:26:04.310 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:04.304+0000 7f249e62c740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564104235ce0 0x56410423fb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:26:04.310 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:04.304+0000 7f249e62c740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f24941aec60 con 0x5641042332a0 2024-09-11T00:26:04.310 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:04.304+0000 7f249e62c740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f24941ae560 con 0x564104235ce0 2024-09-11T00:26:04.310 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:04.304+0000 7f249e62c740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f24941ab740 con 0x564104230970 2024-09-11T00:26:04.311 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:04.304+0000 7f249e62c740 10 client.0 _ll_register_callbacks cb 0x7f248c009e40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:26:04.311 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:04.305+0000 7f248be00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5641042332a0 0x564104241680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:26:04.311 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:04.305+0000 7f2498e00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564104230970 0x564104241050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:26:04.311 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:04.305+0000 7f248be00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5641042332a0 0x564104241680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:34710/0 (socket says 172.21.15.12:34710) 2024-09-11T00:26:04.311 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:04.305+0000 7f248be00640 1 -- 172.21.15.12:0/2214057423 learned_addr learned my addr 172.21.15.12:0/2214057423 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:26:04.311 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:04.305+0000 7f2499800640 1 --2- 172.21.15.12:0/2214057423 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564104235ce0 0x56410423fb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:26:04.311 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:04.305+0000 7f248be00640 1 -- 172.21.15.12:0/2214057423 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564104230970 msgr2=0x564104241050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:26:04.311 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:04.305+0000 7f248be00640 1 --2- 172.21.15.12:0/2214057423 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564104230970 0x564104241050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:26:04.311 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:04.305+0000 7f248be00640 1 -- 172.21.15.12:0/2214057423 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564104235ce0 msgr2=0x56410423fb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:26:04.311 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:04.305+0000 7f248be00640 1 --2- 172.21.15.12:0/2214057423 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564104235ce0 0x56410423fb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:26:04.311 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:04.305+0000 7f248be00640 1 -- 172.21.15.12:0/2214057423 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5641042400b0 con 0x5641042332a0 2024-09-11T00:26:04.311 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:04.305+0000 7f2498e00640 1 --2- 172.21.15.12:0/2214057423 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564104230970 0x564104241050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:26:04.313 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:04.305+0000 7f248be00640 1 --2- 172.21.15.12:0/2214057423 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5641042332a0 0x564104241680 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f247c10a260 tx=0x7f247c03b460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a47bcbc2650fe1fc server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:26:04.313 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:04.305+0000 7f2482a00640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-09-11T00:26:04.313 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:04.306+0000 7f2482a00640 1 -- 172.21.15.12:0/2214057423 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f247c035040 con 0x5641042332a0 2024-09-11T00:26:04.313 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:04.306+0000 7f2482a00640 1 -- 172.21.15.12:0/2214057423 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9208+0+0 (secure 0 0 0) 0x7f247c0cd320 con 0x5641042332a0 2024-09-11T00:26:04.313 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:04.306+0000 7f249e62c740 1 -- 172.21.15.12:0/2214057423 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f248400faa0 con 0x5641042332a0 2024-09-11T00:26:04.313 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:04.306+0000 7f2482a00640 1 -- 172.21.15.12:0/2214057423 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=72) ==== 24+0+0 (secure 0 0 0) 0x7f247c0c8d60 con 0x5641042332a0 2024-09-11T00:26:04.313 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:04.306+0000 7f249e62c740 10 client.17568 fetch_fsmap learned FSMap version 72 2024-09-11T00:26:04.313 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:04.306+0000 7f249e62c740 1 -- 172.21.15.12:0/2214057423 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=72,monmap=4+}) -- 0x5641042b3300 con 0x5641042332a0 2024-09-11T00:26:04.314 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:04.306+0000 7f2482a00640 1 -- 172.21.15.12:0/2214057423 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f248400faa0 con 0x5641042332a0 2024-09-11T00:26:04.314 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:04.306+0000 7f2482a00640 1 -- 172.21.15.12:0/2214057423 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 72) ==== 8386+0+0 (secure 0 0 0) 0x7f247c0b2ce0 con 0x5641042332a0 2024-09-11T00:26:04.314 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:04.306+0000 7f249e62c740 10 client.17568 fetch_fsmap finished waiting for FSMap version 72 2024-09-11T00:26:04.314 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:04.306+0000 7f249e62c740 10 client.17568 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T00:26:04.314 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:04.306+0000 7f249e62c740 20 client.17568 populate_metadata read hostname 'smithi012' 2024-09-11T00:26:04.314 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:04.306+0000 7f249e62c740 1 --2- 172.21.15.12:0/2214057423 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x5641042b3d20 0x5641042b6140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:26:04.314 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:04.306+0000 7f249e62c740 4 client.17568 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T00:26:04.314 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:04.306+0000 7f249e62c740 1 -- 172.21.15.12:0/2214057423 --> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5641041a10b0 con 0x5641042b3d20 2024-09-11T00:26:04.314 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:04.307+0000 7f2498e00640 1 --2- 172.21.15.12:0/2214057423 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x5641042b3d20 0x5641042b6140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:26:04.314 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:04.307+0000 7f2498e00640 1 --2- 172.21.15.12:0/2214057423 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x5641042b3d20 0x5641042b6140 crc :-1 s=READY pgs=268 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7f9afe0db2d0652f server_cookie=69452f84d5523e06 in_seq=0 out_seq=0 2024-09-11T00:26:04.314 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:04.307+0000 7f2482a00640 10 client.17568 ms_handle_connect on v2:172.21.15.165:6838/501312984 2024-09-11T00:26:04.356 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:04.353+0000 7f2482a00640 1 -- 172.21.15.12:0/2214057423 <== mds.0 v2:172.21.15.165:6838/501312984 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5641041a10b0 con 0x5641042b3d20 2024-09-11T00:26:04.356 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:04.353+0000 7f2482a00640 10 client.17568 handle_command_reply: tid=0 2024-09-11T00:26:04.356 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:04.353+0000 7f249e62c740 1 client.17568 shutdown 2024-09-11T00:26:04.356 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:04.353+0000 7f249e62c740 1 -- 172.21.15.12:0/2214057423 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5641042332a0 msgr2=0x564104241680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:26:04.356 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:04.353+0000 7f249e62c740 1 --2- 172.21.15.12:0/2214057423 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5641042332a0 0x564104241680 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f247c10a260 tx=0x7f247c03b460 comp rx=0 tx=0).stop 2024-09-11T00:26:04.356 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:04.353+0000 7f249e62c740 1 -- 172.21.15.12:0/2214057423 shutdown_connections 2024-09-11T00:26:04.356 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:04.353+0000 7f249e62c740 1 --2- 172.21.15.12:0/2214057423 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x5641042b3d20 0x5641042b6140 crc :-1 s=READY pgs=268 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:26:04.356 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:04.353+0000 7f249e62c740 1 --2- 172.21.15.12:0/2214057423 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564104235ce0 0x56410423fb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:26:04.356 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:04.353+0000 7f249e62c740 1 --2- 172.21.15.12:0/2214057423 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5641042332a0 0x564104241680 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:26:04.356 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:04.353+0000 7f249e62c740 1 --2- 172.21.15.12:0/2214057423 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564104230970 0x564104241050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:26:04.357 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:04.353+0000 7f2482a00640 0 client.17568 ms_handle_reset on v2:172.21.15.165:6838/501312984 2024-09-11T00:26:04.357 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:04.353+0000 7f249e62c740 1 -- 172.21.15.12:0/2214057423 >> 172.21.15.12:0/2214057423 conn(0x56410422bac0 msgr2=0x56410422fca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:26:04.357 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:04.353+0000 7f249e62c740 1 -- 172.21.15.12:0/2214057423 shutdown_connections 2024-09-11T00:26:04.357 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:04.353+0000 7f249e62c740 1 -- 172.21.15.12:0/2214057423 wait complete. 2024-09-11T00:26:04.357 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:04.353+0000 7f249e62c740 20 client.17568 trim_cache size 0 max 16384 2024-09-11T00:26:04.357 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:{ 2024-09-11T00:26:04.357 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "message": "", 2024-09-11T00:26:04.357 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "return_code": 0 2024-09-11T00:26:04.357 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:} 2024-09-11T00:26:04.358 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:04.354+0000 7f249a200640 1 -- 172.21.15.12:0/3695060963 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f24780b52e0 msgr2=0x7f24780b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:26:04.358 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:04.354+0000 7f249a200640 1 --2- 172.21.15.12:0/3695060963 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f24780b52e0 0x7f24780b77a0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f249419bb40 tx=0x7f248c009500 comp rx=0 tx=0).stop 2024-09-11T00:26:04.358 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:04.354+0000 7f249a200640 1 -- 172.21.15.12:0/3695060963 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2494105b00 msgr2=0x7f249419b410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:26:04.358 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:04.354+0000 7f249a200640 1 --2- 172.21.15.12:0/3695060963 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2494105b00 0x7f249419b410 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f247c009230 tx=0x7f247c0042f0 comp rx=0 tx=0).stop 2024-09-11T00:26:04.358 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:04.354+0000 7f249a200640 1 -- 172.21.15.12:0/3695060963 shutdown_connections 2024-09-11T00:26:04.358 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:04.354+0000 7f249a200640 1 --2- 172.21.15.12:0/3695060963 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f24780b52e0 0x7f24780b77a0 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:26:04.359 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:04.354+0000 7f249a200640 1 --2- 172.21.15.12:0/3695060963 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f24941064c0 0x7f249419c040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:26:04.359 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:04.354+0000 7f249a200640 1 --2- 172.21.15.12:0/3695060963 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2494105b00 0x7f249419b410 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:26:04.359 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:04.354+0000 7f249a200640 1 --2- 172.21.15.12:0/3695060963 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f24941049b0 0x7f249419adc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:26:04.359 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:04.354+0000 7f249a200640 1 -- 172.21.15.12:0/3695060963 >> 172.21.15.12:0/3695060963 conn(0x7f2494100250 msgr2=0x7f2494108ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:26:04.359 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:04.354+0000 7f249a200640 1 -- 172.21.15.12:0/3695060963 shutdown_connections 2024-09-11T00:26:04.359 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:04.354+0000 7f249a200640 1 -- 172.21.15.12:0/3695060963 wait complete. 2024-09-11T00:26:04.532 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:26:04 smithi165 ceph-mon[35356]: pgmap v968: 129 pgs: 129 active+clean; 589 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 1.4 MiB/s wr, 293 op/s 2024-09-11T00:26:04.563 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:26:04 smithi012 ceph-mon[30333]: pgmap v968: 129 pgs: 129 active+clean; 589 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 1.4 MiB/s wr, 293 op/s 2024-09-11T00:26:04.581 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:26:04 smithi178 ceph-mon[35919]: pgmap v968: 129 pgs: 129 active+clean; 589 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 1.4 MiB/s wr, 293 op/s 2024-09-11T00:26:05.207 INFO:tasks.workunit.client.0.smithi012.stdout: 10 103757 15.82 MB/sec execute 241 sec latency 13756.967 ms 2024-09-11T00:26:06.207 INFO:tasks.workunit.client.0.smithi012.stdout: 10 103757 15.75 MB/sec execute 242 sec latency 14757.030 ms 2024-09-11T00:26:06.337 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:26:06 smithi012 ceph-mon[30333]: pgmap v969: 129 pgs: 129 active+clean; 589 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 820 KiB/s rd, 404 KiB/s wr, 123 op/s 2024-09-11T00:26:06.532 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:26:06 smithi165 ceph-mon[35356]: pgmap v969: 129 pgs: 129 active+clean; 589 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 820 KiB/s rd, 404 KiB/s wr, 123 op/s 2024-09-11T00:26:06.581 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:26:06 smithi178 ceph-mon[35919]: pgmap v969: 129 pgs: 129 active+clean; 589 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 820 KiB/s rd, 404 KiB/s wr, 123 op/s 2024-09-11T00:26:06.783 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid ff3ad74a-6fcf-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T00:26:07.208 INFO:tasks.workunit.client.0.smithi012.stdout: 10 103757 15.69 MB/sec execute 243 sec latency 15757.091 ms 2024-09-11T00:26:07.244 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:26:08.207 INFO:tasks.workunit.client.0.smithi012.stdout: 10 103757 15.63 MB/sec execute 244 sec latency 16757.160 ms 2024-09-11T00:26:08.509 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:26:08 smithi012 ceph-mon[30333]: pgmap v970: 129 pgs: 129 active+clean; 592 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 827 KiB/s rd, 814 KiB/s wr, 138 op/s 2024-09-11T00:26:08.532 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:26:08 smithi165 ceph-mon[35356]: pgmap v970: 129 pgs: 129 active+clean; 592 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 827 KiB/s rd, 814 KiB/s wr, 138 op/s 2024-09-11T00:26:08.581 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:26:08 smithi178 ceph-mon[35919]: pgmap v970: 129 pgs: 129 active+clean; 592 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 827 KiB/s rd, 814 KiB/s wr, 138 op/s 2024-09-11T00:26:08.997 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:26:08.994+0000 7fb88f200640 1 -- 172.21.15.12:0/937534310 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb890108fa0 msgr2=0x7fb89010f7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:26:08.997 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:26:08.994+0000 7fb88f200640 1 --2- 172.21.15.12:0/937534310 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb890108fa0 0x7fb89010f7a0 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7fb888009ca0 tx=0x7fb88802a8c0 comp rx=0 tx=0).stop 2024-09-11T00:26:08.997 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:26:08.994+0000 7fb88f200640 1 -- 172.21.15.12:0/937534310 shutdown_connections 2024-09-11T00:26:08.997 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:26:08.994+0000 7fb88f200640 1 --2- 172.21.15.12:0/937534310 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb890108fa0 0x7fb89010f7a0 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:26:08.997 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:26:08.994+0000 7fb88f200640 1 --2- 172.21.15.12:0/937534310 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb890108620 0x7fb890108a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:26:08.997 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:26:08.994+0000 7fb88f200640 1 --2- 172.21.15.12:0/937534310 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb890102c10 0x7fb890102ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:26:08.997 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:26:08.994+0000 7fb88f200640 1 -- 172.21.15.12:0/937534310 >> 172.21.15.12:0/937534310 conn(0x7fb8900fe6c0 msgr2=0x7fb890100ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:26:08.997 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:26:08.994+0000 7fb88f200640 1 -- 172.21.15.12:0/937534310 shutdown_connections 2024-09-11T00:26:08.997 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:26:08.994+0000 7fb88f200640 1 -- 172.21.15.12:0/937534310 wait complete. 2024-09-11T00:26:08.998 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:26:08.995+0000 7fb88f200640 1 Processor -- start 2024-09-11T00:26:08.998 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:26:08.995+0000 7fb88f200640 1 -- start start 2024-09-11T00:26:08.998 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:26:08.996+0000 7fb88f200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb890102c10 0x7fb8900754a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:26:08.999 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:26:08.996+0000 7fb88f200640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb890108620 0x7fb8900759e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:26:08.999 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:26:08.996+0000 7fb88f200640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb890108fa0 0x7fb890075f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:26:08.999 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:26:08.996+0000 7fb88f200640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fb890112c30 con 0x7fb890108fa0 2024-09-11T00:26:08.999 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:26:08.996+0000 7fb88f200640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fb890112ab0 con 0x7fb890102c10 2024-09-11T00:26:08.999 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:26:08.996+0000 7fb88f200640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fb890112db0 con 0x7fb890108620 2024-09-11T00:26:08.999 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:26:08.996+0000 7fb88de00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb890102c10 0x7fb8900754a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:26:08.999 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:26:08.996+0000 7fb887e00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb890108620 0x7fb8900759e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:26:08.999 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:26:08.996+0000 7fb88e800640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb890108fa0 0x7fb890075f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:26:08.999 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:26:08.996+0000 7fb88de00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb890102c10 0x7fb8900754a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.12:37204/0 (socket says 172.21.15.12:37204) 2024-09-11T00:26:08.999 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:26:08.996+0000 7fb88de00640 1 -- 172.21.15.12:0/4005567070 learned_addr learned my addr 172.21.15.12:0/4005567070 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:26:08.999 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:26:08.996+0000 7fb887e00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb890108620 0x7fb8900759e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.12:49120/0 (socket says 172.21.15.12:49120) 2024-09-11T00:26:09.000 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:26:08.997+0000 7fb88de00640 1 -- 172.21.15.12:0/4005567070 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb890108620 msgr2=0x7fb8900759e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:26:09.000 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:26:08.997+0000 7fb88de00640 1 --2- 172.21.15.12:0/4005567070 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb890108620 0x7fb8900759e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:26:09.001 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:26:08.997+0000 7fb88de00640 1 -- 172.21.15.12:0/4005567070 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb890108fa0 msgr2=0x7fb890075f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:26:09.001 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:26:08.997+0000 7fb88de00640 1 --2- 172.21.15.12:0/4005567070 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb890108fa0 0x7fb890075f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:26:09.001 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:26:08.997+0000 7fb88de00640 1 -- 172.21.15.12:0/4005567070 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8901a7420 con 0x7fb890102c10 2024-09-11T00:26:09.001 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:26:08.997+0000 7fb88de00640 1 --2- 172.21.15.12:0/4005567070 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb890102c10 0x7fb8900754a0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fb88000e9f0 tx=0x7fb88000eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:26:09.001 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:26:08.997+0000 7fb886a00640 1 -- 172.21.15.12:0/4005567070 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb88000cd80 con 0x7fb890102c10 2024-09-11T00:26:09.001 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:26:08.997+0000 7fb886a00640 1 -- 172.21.15.12:0/4005567070 <== mon.2 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb8800108e0 con 0x7fb890102c10 2024-09-11T00:26:09.001 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:26:08.997+0000 7fb88f200640 1 -- 172.21.15.12:0/4005567070 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb8901a7710 con 0x7fb890102c10 2024-09-11T00:26:09.001 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:26:08.997+0000 7fb886a00640 1 -- 172.21.15.12:0/4005567070 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb880018980 con 0x7fb890102c10 2024-09-11T00:26:09.001 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:26:08.997+0000 7fb88f200640 1 -- 172.21.15.12:0/4005567070 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8901a7c50 con 0x7fb890102c10 2024-09-11T00:26:09.001 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:26:08.999+0000 7fb88f200640 1 -- 172.21.15.12:0/4005567070 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb890102ff0 con 0x7fb890102c10 2024-09-11T00:26:09.003 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:26:09.001+0000 7fb886a00640 1 -- 172.21.15.12:0/4005567070 <== mon.2 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7fb880010430 con 0x7fb890102c10 2024-09-11T00:26:09.005 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:26:09.002+0000 7fb886a00640 1 --2- 172.21.15.12:0/4005567070 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fb8740b5390 0x7fb8740b7850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:26:09.005 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:26:09.002+0000 7fb886a00640 1 -- 172.21.15.12:0/4005567070 <== mon.2 v2:172.21.15.165:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9208+0+0 (secure 0 0 0) 0x7fb8800e62f0 con 0x7fb890102c10 2024-09-11T00:26:09.005 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:26:09.002+0000 7fb887e00640 1 --2- 172.21.15.12:0/4005567070 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fb8740b5390 0x7fb8740b7850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:26:09.005 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:26:09.003+0000 7fb887e00640 1 --2- 172.21.15.12:0/4005567070 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fb8740b5390 0x7fb8740b7850 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fb890076f90 tx=0x7fb878005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:26:09.008 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:26:09.006+0000 7fb886a00640 1 -- 172.21.15.12:0/4005567070 <== mon.2 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7fb8800ac730 con 0x7fb890102c10 2024-09-11T00:26:09.209 INFO:tasks.workunit.client.0.smithi012.stdout: 10 103757 15.56 MB/sec execute 245 sec latency 17758.412 ms 2024-09-11T00:26:09.326 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:26:09.323+0000 7fb88f200640 1 -- 172.21.15.12:0/4005567070 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb8901a7f10 con 0x7fb890102c10 2024-09-11T00:26:09.327 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:26:09.324+0000 7fb886a00640 1 -- 172.21.15.12:0/4005567070 <== mon.2 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 72 v72) ==== 94+0+11970 (secure 0 0 0) 0x7fb8800b1160 con 0x7fb890102c10 2024-09-11T00:26:09.327 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-11T00:26:09.327 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":72,"btime":"2024-09-11T00:25:50:075664+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14577,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6835/447171591","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":447171591},{"type":"v1","addr":"172.21.15.12:6835","nonce":447171591}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14589,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/901486040","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":901486040},{"type":"v1","addr":"172.21.15.12:6837","nonce":901486040}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":72,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T00:04:04.739881+0000","modified":"2024-09-11T00:25:49.554724+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24674,"mds_1":24643,"mds_2":24638,"mds_3":24637,"mds_4":24614},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14631":{"gid":14631,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.12:6841/3069919437","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6840","nonce":3069919437},{"type":"v1","addr":"172.21.15.12:6841","nonce":3069919437}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24562":{"gid":24562,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6833/1141636112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":1141636112},{"type":"v1","addr":"172.21.15.178:6833","nonce":1141636112}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24581":{"gid":24581,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6833/3111980970","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6832","nonce":3111980970},{"type":"v1","addr":"172.21.15.165:6833","nonce":3111980970}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24607":{"gid":24607,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6835/4084868287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":4084868287},{"type":"v1","addr":"172.21.15.178:6835","nonce":4084868287}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24614":{"gid":24614,"name":"e","rank":4,"incarnation":26,"state":"up:active","state_seq":21,"addr":"172.21.15.165:6835/2425395274","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":2425395274},{"type":"v1","addr":"172.21.15.165:6835","nonce":2425395274}]},"join_fscid":-1,"export_targets":[1],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24637":{"gid":24637,"name":"i","rank":3,"incarnation":24,"state":"up:active","state_seq":18,"addr":"172.21.15.178:6839/70628097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6838","nonce":70628097},{"type":"v1","addr":"172.21.15.178:6839","nonce":70628097}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24638":{"gid":24638,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":18,"addr":"172.21.15.165:6837/6215088","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6836","nonce":6215088},{"type":"v1","addr":"172.21.15.165:6837","nonce":6215088}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24643":{"gid":24643,"name":"g","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.12:6839/3124093009","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6838","nonce":3124093009},{"type":"v1","addr":"172.21.15.12:6839","nonce":3124093009}]},"join_fscid":-1,"export_targets":[0,3],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24674":{"gid":24674,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.165:6839/501312984","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6838","nonce":501312984},{"type":"v1","addr":"172.21.15.165:6839","nonce":501312984}]},"join_fscid":-1,"export_targets":[1],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25672":{"gid":25672,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6837/308601741","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":308601741},{"type":"v1","addr":"172.21.15.178:6837","nonce":308601741}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24674,"qdb_cluster":[24674,24643,24638,24637,24614]},"id":1}]} 2024-09-11T00:26:09.328 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 72 2024-09-11T00:26:09.329 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:26:09.327+0000 7fb88f200640 1 -- 172.21.15.12:0/4005567070 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fb8740b5390 msgr2=0x7fb8740b7850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:26:09.329 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:26:09.327+0000 7fb88f200640 1 --2- 172.21.15.12:0/4005567070 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fb8740b5390 0x7fb8740b7850 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fb890076f90 tx=0x7fb878005f90 comp rx=0 tx=0).stop 2024-09-11T00:26:09.330 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:26:09.327+0000 7fb88f200640 1 -- 172.21.15.12:0/4005567070 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb890102c10 msgr2=0x7fb8900754a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:26:09.330 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:26:09.327+0000 7fb88f200640 1 --2- 172.21.15.12:0/4005567070 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb890102c10 0x7fb8900754a0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fb88000e9f0 tx=0x7fb88000eeb0 comp rx=0 tx=0).stop 2024-09-11T00:26:09.330 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:26:09.327+0000 7fb88f200640 1 -- 172.21.15.12:0/4005567070 shutdown_connections 2024-09-11T00:26:09.330 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:26:09.327+0000 7fb88f200640 1 --2- 172.21.15.12:0/4005567070 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fb8740b5390 0x7fb8740b7850 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:26:09.330 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:26:09.327+0000 7fb88f200640 1 --2- 172.21.15.12:0/4005567070 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb890108fa0 0x7fb890075f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:26:09.330 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:26:09.327+0000 7fb88f200640 1 --2- 172.21.15.12:0/4005567070 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb890108620 0x7fb8900759e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:26:09.330 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:26:09.327+0000 7fb88f200640 1 --2- 172.21.15.12:0/4005567070 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb890102c10 0x7fb8900754a0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:26:09.330 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:26:09.327+0000 7fb88f200640 1 -- 172.21.15.12:0/4005567070 >> 172.21.15.12:0/4005567070 conn(0x7fb8900fe6c0 msgr2=0x7fb890100ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:26:09.330 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:26:09.327+0000 7fb88f200640 1 -- 172.21.15.12:0/4005567070 shutdown_connections 2024-09-11T00:26:09.330 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:26:09.327+0000 7fb88f200640 1 -- 172.21.15.12:0/4005567070 wait complete. 2024-09-11T00:26:10.209 INFO:tasks.workunit.client.0.smithi012.stdout: 10 103757 15.50 MB/sec execute 246 sec latency 18758.495 ms 2024-09-11T00:26:10.226 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24674 quiesce db --set-id 4921b205 --release --await' 2024-09-11T00:26:10.226 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid ff3ad74a-6fcf-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24674 quiesce db --set-id 4921b205 --release --await 2024-09-11T00:26:10.532 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:26:10 smithi165 ceph-mon[35356]: pgmap v971: 129 pgs: 129 active+clean; 592 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 597 KiB/s wr, 32 op/s 2024-09-11T00:26:10.533 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:26:10 smithi165 ceph-mon[35356]: from='client.? 172.21.15.12:0/4005567070' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T00:26:10.563 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:26:10 smithi012 ceph-mon[30333]: pgmap v971: 129 pgs: 129 active+clean; 592 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 597 KiB/s wr, 32 op/s 2024-09-11T00:26:10.563 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:26:10 smithi012 ceph-mon[30333]: from='client.? 172.21.15.12:0/4005567070' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T00:26:10.581 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:26:10 smithi178 ceph-mon[35919]: pgmap v971: 129 pgs: 129 active+clean; 592 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 597 KiB/s wr, 32 op/s 2024-09-11T00:26:10.581 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:26:10 smithi178 ceph-mon[35919]: from='client.? 172.21.15.12:0/4005567070' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T00:26:10.704 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:26:11.217 INFO:tasks.workunit.client.0.smithi012.stdout: 10 103757 15.44 MB/sec execute 247 sec latency 19766.674 ms 2024-09-11T00:26:11.675 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:26:11.504+0000 7f80d8a00640 1 -- 172.21.15.12:0/1161042419 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f80d4108620 msgr2=0x7f80d4108a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:26:11.675 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:26:11.504+0000 7f80d8a00640 1 --2- 172.21.15.12:0/1161042419 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f80d4108620 0x7f80d4108a60 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f80c0009900 tx=0x7f80c002a410 comp rx=0 tx=0).stop 2024-09-11T00:26:11.675 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:26:11.504+0000 7f80d8a00640 1 -- 172.21.15.12:0/1161042419 shutdown_connections 2024-09-11T00:26:11.675 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:26:11.504+0000 7f80d8a00640 1 --2- 172.21.15.12:0/1161042419 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f80d4108fa0 0x7f80d410f7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:26:11.676 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:26:11.504+0000 7f80d8a00640 1 --2- 172.21.15.12:0/1161042419 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f80d4108620 0x7f80d4108a60 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:26:11.676 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:26:11.504+0000 7f80d8a00640 1 --2- 172.21.15.12:0/1161042419 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f80d4102c10 0x7f80d4102ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:26:11.676 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:26:11.504+0000 7f80d8a00640 1 -- 172.21.15.12:0/1161042419 >> 172.21.15.12:0/1161042419 conn(0x7f80d40fe660 msgr2=0x7f80d4100a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:26:11.676 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:26:11.504+0000 7f80d8a00640 1 -- 172.21.15.12:0/1161042419 shutdown_connections 2024-09-11T00:26:11.676 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:26:11.504+0000 7f80d8a00640 1 -- 172.21.15.12:0/1161042419 wait complete. 2024-09-11T00:26:11.676 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:26:11.505+0000 7f80d8a00640 1 Processor -- start 2024-09-11T00:26:11.676 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:26:11.505+0000 7f80d8a00640 1 -- start start 2024-09-11T00:26:11.676 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:26:11.506+0000 7f80d8a00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f80d4102c10 0x7f80d4109790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:26:11.676 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:26:11.506+0000 7f80d8a00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f80d4108620 0x7f80d4109cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:26:11.676 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:26:11.506+0000 7f80d8a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f80d4108fa0 0x7f80d410a210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:26:11.676 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:26:11.506+0000 7f80d8a00640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f80d4112cb0 con 0x7f80d4108620 2024-09-11T00:26:11.676 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:26:11.506+0000 7f80d8a00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f80d4112b30 con 0x7f80d4108fa0 2024-09-11T00:26:11.676 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:26:11.506+0000 7f80d8a00640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f80d4112e30 con 0x7f80d4102c10 2024-09-11T00:26:11.677 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:26:11.506+0000 7f80d1e00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f80d4108620 0x7f80d4109cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:26:11.677 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:26:11.506+0000 7f80d1e00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f80d4108620 0x7f80d4109cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:53658/0 (socket says 172.21.15.12:53658) 2024-09-11T00:26:11.677 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:26:11.506+0000 7f80d1e00640 1 -- 172.21.15.12:0/3247255676 learned_addr learned my addr 172.21.15.12:0/3247255676 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:26:11.677 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:26:11.506+0000 7f80d1e00640 1 -- 172.21.15.12:0/3247255676 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f80d4102c10 msgr2=0x7f80d4109790 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-11T00:26:11.677 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:26:11.506+0000 7f80d1e00640 1 --2- 172.21.15.12:0/3247255676 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f80d4102c10 0x7f80d4109790 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:26:11.677 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:26:11.506+0000 7f80d1e00640 1 -- 172.21.15.12:0/3247255676 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f80d4108fa0 msgr2=0x7f80d410a210 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-11T00:26:11.677 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:26:11.506+0000 7f80d1e00640 1 --2- 172.21.15.12:0/3247255676 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f80d4108fa0 0x7f80d410a210 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:26:11.677 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:26:11.506+0000 7f80d1e00640 1 -- 172.21.15.12:0/3247255676 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80d41ab960 con 0x7f80d4108620 2024-09-11T00:26:11.677 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:26:11.507+0000 7f80d1e00640 1 --2- 172.21.15.12:0/3247255676 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f80d4108620 0x7f80d4109cd0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f80c0009a20 tx=0x7f80c0003b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:26:11.677 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:26:11.507+0000 7f80cb400640 1 -- 172.21.15.12:0/3247255676 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80c003f070 con 0x7f80d4108620 2024-09-11T00:26:11.677 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:26:11.507+0000 7f80cb400640 1 -- 172.21.15.12:0/3247255676 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f80c0031b50 con 0x7f80d4108620 2024-09-11T00:26:11.677 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:26:11.507+0000 7f80d8a00640 1 -- 172.21.15.12:0/3247255676 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f80d41abbf0 con 0x7f80d4108620 2024-09-11T00:26:11.678 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:26:11.507+0000 7f80cb400640 1 -- 172.21.15.12:0/3247255676 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80c0035030 con 0x7f80d4108620 2024-09-11T00:26:11.678 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:26:11.508+0000 7f80d8a00640 1 -- 172.21.15.12:0/3247255676 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80d41ac120 con 0x7f80d4108620 2024-09-11T00:26:11.678 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:26:11.511+0000 7f80dbe4a740 1 Processor -- start 2024-09-11T00:26:11.678 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:26:11.511+0000 7f80dbe4a740 1 -- start start 2024-09-11T00:26:11.678 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:26:11.512+0000 7f80dbe4a740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563d8cac11d0 0x563d8cac15b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:26:11.678 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:26:11.512+0000 7f80dbe4a740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563d8cac1af0 0x563d8cacd540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:26:11.678 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:26:11.512+0000 7f80dbe4a740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563d8cacda80 0x563d8cacfe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:26:11.678 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:26:11.512+0000 7f80dbe4a740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x563d8ca38290 con 0x563d8cacda80 2024-09-11T00:26:11.678 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:26:11.512+0000 7f80dbe4a740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x563d8ca276c0 con 0x563d8cac11d0 2024-09-11T00:26:11.678 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:26:11.512+0000 7f80dbe4a740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x563d8ca2b4d0 con 0x563d8cac1af0 2024-09-11T00:26:11.678 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:26:11.512+0000 7f80d2800640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563d8cac1af0 0x563d8cacd540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:26:11.678 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:26:11.512+0000 7f80d2800640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563d8cac1af0 0x563d8cacd540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.12:49146/0 (socket says 172.21.15.12:49146) 2024-09-11T00:26:11.678 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:26:11.512+0000 7f80d2800640 1 -- 172.21.15.12:0/1885375621 learned_addr learned my addr 172.21.15.12:0/1885375621 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:26:11.679 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:26:11.512+0000 7f80cb400640 1 -- 172.21.15.12:0/3247255676 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7f80c004c060 con 0x7f80d4108620 2024-09-11T00:26:11.679 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:26:11.513+0000 7f80d1e00640 1 --2- 172.21.15.12:0/1885375621 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563d8cacda80 0x563d8cacfe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:26:11.679 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:26:11.513+0000 7f80d1e00640 1 -- 172.21.15.12:0/1885375621 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563d8cac1af0 msgr2=0x563d8cacd540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:26:11.679 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:26:11.513+0000 7f80d1e00640 1 --2- 172.21.15.12:0/1885375621 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563d8cac1af0 0x563d8cacd540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:26:11.679 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:26:11.513+0000 7f80d1e00640 1 -- 172.21.15.12:0/1885375621 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563d8cac11d0 msgr2=0x563d8cac15b0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-11T00:26:11.679 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:26:11.513+0000 7f80d1e00640 1 --2- 172.21.15.12:0/1885375621 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563d8cac11d0 0x563d8cac15b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:26:11.679 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:26:11.513+0000 7f80d1e00640 1 -- 172.21.15.12:0/1885375621 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563d8c9b4460 con 0x563d8cacda80 2024-09-11T00:26:11.679 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:26:11.514+0000 7f80d1e00640 1 --2- 172.21.15.12:0/1885375621 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563d8cacda80 0x563d8cacfe70 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f80c01039e0 tx=0x7f80c010e640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59e78dc7fa7db6a1 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:26:11.679 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:26:11.514+0000 7f80bf400640 1 -- 172.21.15.12:0/1885375621 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80c01080d0 con 0x563d8cacda80 2024-09-11T00:26:11.679 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:26:11.514+0000 7f80bf400640 1 -- 172.21.15.12:0/1885375621 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f80c0113030 con 0x563d8cacda80 2024-09-11T00:26:11.679 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:26:11.514+0000 7f80bf400640 1 -- 172.21.15.12:0/1885375621 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80c0118640 con 0x563d8cacda80 2024-09-11T00:26:11.679 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:26:11.514+0000 7f80cb400640 1 --2- 172.21.15.12:0/3247255676 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f80b80b5410 0x7f80b80b78d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:26:11.680 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:26:11.515+0000 7f80cb400640 1 -- 172.21.15.12:0/3247255676 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9208+0+0 (secure 0 0 0) 0x7f80c0106310 con 0x7f80d4108620 2024-09-11T00:26:11.680 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:26:11.515+0000 7f80d3200640 1 --2- 172.21.15.12:0/3247255676 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f80b80b5410 0x7f80b80b78d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:26:11.680 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:26:11.515+0000 7f80dbe4a740 1 -- 172.21.15.12:0/1885375621 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563d8cacda80 msgr2=0x563d8cacfe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:26:11.680 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:26:11.515+0000 7f80dbe4a740 1 --2- 172.21.15.12:0/1885375621 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563d8cacda80 0x563d8cacfe70 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f80c01039e0 tx=0x7f80c010e640 comp rx=0 tx=0).stop 2024-09-11T00:26:11.680 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:26:11.515+0000 7f80dbe4a740 1 -- 172.21.15.12:0/1885375621 shutdown_connections 2024-09-11T00:26:11.680 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:26:11.515+0000 7f80dbe4a740 1 --2- 172.21.15.12:0/1885375621 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563d8cacda80 0x563d8cacfe70 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:26:11.680 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:26:11.515+0000 7f80dbe4a740 1 --2- 172.21.15.12:0/1885375621 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563d8cac1af0 0x563d8cacd540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:26:11.680 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:26:11.515+0000 7f80dbe4a740 1 --2- 172.21.15.12:0/1885375621 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563d8cac11d0 0x563d8cac15b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:26:11.680 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:26:11.515+0000 7f80dbe4a740 1 -- 172.21.15.12:0/1885375621 >> 172.21.15.12:0/1885375621 conn(0x563d8cabf000 msgr2=0x563d8cac01a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:26:11.680 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:26:11.515+0000 7f80dbe4a740 1 -- 172.21.15.12:0/1885375621 shutdown_connections 2024-09-11T00:26:11.680 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:26:11.515+0000 7f80dbe4a740 1 -- 172.21.15.12:0/1885375621 wait complete. 2024-09-11T00:26:11.680 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:26:11.516+0000 7f80d3200640 1 --2- 172.21.15.12:0/3247255676 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f80b80b5410 0x7f80b80b78d0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f80cc00aab0 tx=0x7f80cc009250 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:26:11.681 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:26:11.516+0000 7f80dbe4a740 1 -- start start 2024-09-11T00:26:11.681 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:26:11.516+0000 7f80dbe4a740 1 Processor -- start 2024-09-11T00:26:11.681 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:26:11.517+0000 7f80dbe4a740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563d8cac11d0 0x563d8cacaa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:26:11.681 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:26:11.517+0000 7f80dbe4a740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563d8cac1af0 0x563d8cacafd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:26:11.681 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:26:11.517+0000 7f80dbe4a740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563d8cacda80 0x563d8cacb510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:26:11.681 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:26:11.517+0000 7f80dbe4a740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x563d8c888bb0 con 0x563d8cac1af0 2024-09-11T00:26:11.681 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:26:11.517+0000 7f80dbe4a740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x563d8c9c26c0 con 0x563d8cacda80 2024-09-11T00:26:11.681 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:26:11.517+0000 7f80dbe4a740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x563d8cad04f0 con 0x563d8cac11d0 2024-09-11T00:26:11.681 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:26:11.517+0000 7f80dbe4a740 10 client.0 _ll_register_callbacks cb 0x563d8c9dd490 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:26:11.681 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:26:11.517+0000 7f80d2800640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563d8cac11d0 0x563d8cacaa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:26:11.681 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:26:11.517+0000 7f80d1e00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563d8cac1af0 0x563d8cacafd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:26:11.681 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:26:11.517+0000 7f80d2800640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563d8cac11d0 0x563d8cacaa90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.12:49152/0 (socket says 172.21.15.12:49152) 2024-09-11T00:26:11.681 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:26:11.517+0000 7f80d2800640 1 -- 172.21.15.12:0/2213484245 learned_addr learned my addr 172.21.15.12:0/2213484245 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:26:11.682 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:26:11.518+0000 7f80d3200640 1 --2- 172.21.15.12:0/2213484245 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563d8cacda80 0x563d8cacb510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:26:11.682 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:26:11.518+0000 7f80d1e00640 1 -- 172.21.15.12:0/2213484245 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563d8cac11d0 msgr2=0x563d8cacaa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:26:11.682 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:26:11.518+0000 7f80d1e00640 1 --2- 172.21.15.12:0/2213484245 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563d8cac11d0 0x563d8cacaa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:26:11.682 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:26:11.518+0000 7f80d1e00640 1 -- 172.21.15.12:0/2213484245 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563d8cacda80 msgr2=0x563d8cacb510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:26:11.682 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:26:11.518+0000 7f80d1e00640 1 --2- 172.21.15.12:0/2213484245 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563d8cacda80 0x563d8cacb510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:26:11.682 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:26:11.518+0000 7f80d1e00640 1 -- 172.21.15.12:0/2213484245 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563d8c785b00 con 0x563d8cac1af0 2024-09-11T00:26:11.682 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:26:11.518+0000 7f80d2800640 1 --2- 172.21.15.12:0/2213484245 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563d8cac11d0 0x563d8cacaa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:26:11.682 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:26:11.518+0000 7f80d3200640 1 --2- 172.21.15.12:0/2213484245 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563d8cacda80 0x563d8cacb510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T00:26:11.682 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:26:11.518+0000 7f80d1e00640 1 --2- 172.21.15.12:0/2213484245 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563d8cac1af0 0x563d8cacafd0 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f80c0000c00 tx=0x7f80c0003f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea1041527a23db93 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:26:11.682 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:26:11.518+0000 7f80b3e00640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-09-11T00:26:11.682 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:26:11.518+0000 7f80b3e00640 1 -- 172.21.15.12:0/2213484245 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80c0049070 con 0x563d8cac1af0 2024-09-11T00:26:11.682 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:26:11.518+0000 7f80b3e00640 1 -- 172.21.15.12:0/2213484245 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9208+0+0 (secure 0 0 0) 0x7f80c010e150 con 0x563d8cac1af0 2024-09-11T00:26:11.683 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:26:11.519+0000 7f80dbe4a740 1 -- 172.21.15.12:0/2213484245 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f80c0113030 con 0x563d8cac1af0 2024-09-11T00:26:11.683 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:26:11.519+0000 7f80b3e00640 1 -- 172.21.15.12:0/2213484245 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=72) ==== 24+0+0 (secure 0 0 0) 0x7f80c00c8d50 con 0x563d8cac1af0 2024-09-11T00:26:11.683 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:26:11.519+0000 7f80dbe4a740 10 client.17586 fetch_fsmap learned FSMap version 72 2024-09-11T00:26:11.683 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:26:11.519+0000 7f80dbe4a740 1 -- 172.21.15.12:0/2213484245 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=72,monmap=4+}) -- 0x563d8c7fc960 con 0x563d8cac1af0 2024-09-11T00:26:11.683 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:26:11.519+0000 7f80b3e00640 1 -- 172.21.15.12:0/2213484245 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f80c0113030 con 0x563d8cac1af0 2024-09-11T00:26:11.683 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:26:11.519+0000 7f80b3e00640 1 -- 172.21.15.12:0/2213484245 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 72) ==== 8386+0+0 (secure 0 0 0) 0x7f80c00c8970 con 0x563d8cac1af0 2024-09-11T00:26:11.683 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:26:11.519+0000 7f80dbe4a740 10 client.17586 fetch_fsmap finished waiting for FSMap version 72 2024-09-11T00:26:11.683 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:26:11.519+0000 7f80dbe4a740 10 client.17586 resolve_mds: validated gid 24674 aka daemon mds.b 2024-09-11T00:26:11.683 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:26:11.519+0000 7f80dbe4a740 20 client.17586 populate_metadata read hostname 'smithi012' 2024-09-11T00:26:11.683 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:26:11.520+0000 7f80dbe4a740 1 --2- 172.21.15.12:0/2213484245 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x563d8cadd240 0x563d8cadf630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:26:11.683 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:26:11.520+0000 7f80dbe4a740 4 client.17586 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T00:26:11.683 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:26:11.520+0000 7f80dbe4a740 1 -- 172.21.15.12:0/2213484245 --> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563d8ca3a1e0 con 0x563d8cadd240 2024-09-11T00:26:11.684 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:26:11.520+0000 7f80d2800640 1 --2- 172.21.15.12:0/2213484245 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x563d8cadd240 0x563d8cadf630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:26:11.684 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:26:11.521+0000 7f80d2800640 1 --2- 172.21.15.12:0/2213484245 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x563d8cadd240 0x563d8cadf630 crc :-1 s=READY pgs=269 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8be45b78cad3755e server_cookie=9708483165589946 in_seq=0 out_seq=0 2024-09-11T00:26:11.684 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:26:11.521+0000 7f80b3e00640 10 client.17586 ms_handle_connect on v2:172.21.15.165:6838/501312984 2024-09-11T00:26:11.684 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:26:11.522+0000 7f80b3e00640 1 -- 172.21.15.12:0/2213484245 <== mds.0 v2:172.21.15.165:6838/501312984 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x563d8ca3a1e0 con 0x563d8cadd240 2024-09-11T00:26:11.684 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:26:11.522+0000 7f80b3e00640 10 client.17586 handle_command_reply: tid=0 2024-09-11T00:26:11.684 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:26:11.522+0000 7f80dbe4a740 1 client.17586 shutdown 2024-09-11T00:26:11.684 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:26:11.522+0000 7f80dbe4a740 1 -- 172.21.15.12:0/2213484245 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563d8cac1af0 msgr2=0x563d8cacafd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:26:11.684 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:26:11.522+0000 7f80dbe4a740 1 --2- 172.21.15.12:0/2213484245 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563d8cac1af0 0x563d8cacafd0 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f80c0000c00 tx=0x7f80c0003f90 comp rx=0 tx=0).stop 2024-09-11T00:26:11.684 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:26:11.522+0000 7f80dbe4a740 1 -- 172.21.15.12:0/2213484245 shutdown_connections 2024-09-11T00:26:11.684 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:26:11.522+0000 7f80dbe4a740 1 --2- 172.21.15.12:0/2213484245 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x563d8cadd240 0x563d8cadf630 crc :-1 s=READY pgs=269 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:26:11.684 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:26:11.522+0000 7f80dbe4a740 1 --2- 172.21.15.12:0/2213484245 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563d8cacda80 0x563d8cacb510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:26:11.684 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:26:11.522+0000 7f80dbe4a740 1 --2- 172.21.15.12:0/2213484245 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563d8cac1af0 0x563d8cacafd0 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:26:11.684 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:26:11.522+0000 7f80dbe4a740 1 --2- 172.21.15.12:0/2213484245 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563d8cac11d0 0x563d8cacaa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:26:11.685 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:26:11.522+0000 7f80dbe4a740 1 -- 172.21.15.12:0/2213484245 >> 172.21.15.12:0/2213484245 conn(0x563d8cabf000 msgr2=0x563d8cac4ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:26:11.685 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:26:11.522+0000 7f80b3e00640 0 client.17586 ms_handle_reset on v2:172.21.15.165:6838/501312984 2024-09-11T00:26:11.685 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:26:11.522+0000 7f80dbe4a740 1 -- 172.21.15.12:0/2213484245 shutdown_connections 2024-09-11T00:26:11.685 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:26:11.522+0000 7f80dbe4a740 1 -- 172.21.15.12:0/2213484245 wait complete. 2024-09-11T00:26:11.685 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:26:11.522+0000 7f80dbe4a740 20 client.17586 trim_cache size 0 max 16384 2024-09-11T00:26:11.685 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:26:11.555+0000 7f80dbe4a740 1 Processor -- start 2024-09-11T00:26:11.685 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:26:11.555+0000 7f80dbe4a740 1 -- start start 2024-09-11T00:26:11.685 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:26:11.555+0000 7f80dbe4a740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563d8cac11d0 0x563d8cad8de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:26:11.685 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:26:11.555+0000 7f80dbe4a740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563d8cac1af0 0x563d8cad9320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:26:11.685 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:26:11.555+0000 7f80dbe4a740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563d8cacda80 0x563d8cac7ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:26:11.685 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:26:11.555+0000 7f80dbe4a740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f80d41b60e0 con 0x563d8cac11d0 2024-09-11T00:26:11.685 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:26:11.555+0000 7f80dbe4a740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f80d41b7cb0 con 0x563d8cac1af0 2024-09-11T00:26:11.686 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:26:11.555+0000 7f80dbe4a740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f80d41b67b0 con 0x563d8cacda80 2024-09-11T00:26:11.686 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:26:11.555+0000 7f80d2800640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563d8cac11d0 0x563d8cad8de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:26:11.686 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:26:11.555+0000 7f80d2800640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563d8cac11d0 0x563d8cad8de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:53682/0 (socket says 172.21.15.12:53682) 2024-09-11T00:26:11.686 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:26:11.555+0000 7f80d2800640 1 -- 172.21.15.12:0/1252461723 learned_addr learned my addr 172.21.15.12:0/1252461723 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:26:11.686 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:26:11.556+0000 7f80d2800640 1 -- 172.21.15.12:0/1252461723 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563d8cacda80 msgr2=0x563d8cac7ca0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-11T00:26:11.686 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:26:11.556+0000 7f80d2800640 1 --2- 172.21.15.12:0/1252461723 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563d8cacda80 0x563d8cac7ca0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:26:11.686 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:26:11.556+0000 7f80d2800640 1 -- 172.21.15.12:0/1252461723 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563d8cac1af0 msgr2=0x563d8cad9320 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-11T00:26:11.686 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:26:11.556+0000 7f80d2800640 1 --2- 172.21.15.12:0/1252461723 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563d8cac1af0 0x563d8cad9320 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:26:11.686 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:26:11.556+0000 7f80d2800640 1 -- 172.21.15.12:0/1252461723 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563d8cabee30 con 0x563d8cac11d0 2024-09-11T00:26:11.686 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:26:11.556+0000 7f80d2800640 1 --2- 172.21.15.12:0/1252461723 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563d8cac11d0 0x563d8cad8de0 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x563d8cadfe50 tx=0x7f80c4014a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0addff00bde5463 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:26:11.686 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:26:11.556+0000 7f80bea00640 1 -- 172.21.15.12:0/1252461723 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80c40141d0 con 0x563d8cac11d0 2024-09-11T00:26:11.686 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:26:11.556+0000 7f80bea00640 1 -- 172.21.15.12:0/1252461723 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f80c4014370 con 0x563d8cac11d0 2024-09-11T00:26:11.686 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:26:11.556+0000 7f80bea00640 1 -- 172.21.15.12:0/1252461723 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80c4012550 con 0x563d8cac11d0 2024-09-11T00:26:11.687 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:26:11.557+0000 7f80dbe4a740 1 -- 172.21.15.12:0/1252461723 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563d8cac11d0 msgr2=0x563d8cad8de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:26:11.687 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:26:11.557+0000 7f80dbe4a740 1 --2- 172.21.15.12:0/1252461723 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563d8cac11d0 0x563d8cad8de0 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x563d8cadfe50 tx=0x7f80c4014a00 comp rx=0 tx=0).stop 2024-09-11T00:26:11.687 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:26:11.557+0000 7f80dbe4a740 1 -- 172.21.15.12:0/1252461723 shutdown_connections 2024-09-11T00:26:11.687 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:26:11.557+0000 7f80dbe4a740 1 --2- 172.21.15.12:0/1252461723 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563d8cacda80 0x563d8cac7ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:26:11.687 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:26:11.557+0000 7f80dbe4a740 1 --2- 172.21.15.12:0/1252461723 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563d8cac1af0 0x563d8cad9320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:26:11.687 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:26:11.557+0000 7f80dbe4a740 1 --2- 172.21.15.12:0/1252461723 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563d8cac11d0 0x563d8cad8de0 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:26:11.687 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:26:11.557+0000 7f80dbe4a740 1 -- 172.21.15.12:0/1252461723 >> 172.21.15.12:0/1252461723 conn(0x563d8cabf000 msgr2=0x563d8cad86d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:26:11.687 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:26:11.558+0000 7f80dbe4a740 1 -- 172.21.15.12:0/1252461723 shutdown_connections 2024-09-11T00:26:11.687 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:26:11.558+0000 7f80dbe4a740 1 -- 172.21.15.12:0/1252461723 wait complete. 2024-09-11T00:26:11.687 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:26:11.558+0000 7f80dbe4a740 1 -- start start 2024-09-11T00:26:11.687 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:26:11.559+0000 7f80dbe4a740 1 Processor -- start 2024-09-11T00:26:11.687 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:26:11.559+0000 7f80dbe4a740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563d8cac1af0 0x563d8cad5e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:26:11.687 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:26:11.559+0000 7f80dbe4a740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563d8cacda80 0x563d8cad2290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:26:11.688 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:26:11.559+0000 7f80dbe4a740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563d8cad27d0 0x563d8cad2c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:26:11.688 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:26:11.559+0000 7f80dbe4a740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f80d41b75e0 con 0x563d8cad27d0 2024-09-11T00:26:11.688 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:26:11.559+0000 7f80dbe4a740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f80d41b6ee0 con 0x563d8cac1af0 2024-09-11T00:26:11.688 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:26:11.559+0000 7f80dbe4a740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f80d41b40c0 con 0x563d8cacda80 2024-09-11T00:26:11.688 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:26:11.559+0000 7f80dbe4a740 10 client.0 _ll_register_callbacks cb 0x7f80c010e400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:26:11.688 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:26:11.559+0000 7f80d1e00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563d8cacda80 0x563d8cad2290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:26:11.688 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:26:11.559+0000 7f80d1e00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563d8cacda80 0x563d8cad2290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.12:49172/0 (socket says 172.21.15.12:49172) 2024-09-11T00:26:11.688 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:26:11.559+0000 7f80d1e00640 1 -- 172.21.15.12:0/4126373229 learned_addr learned my addr 172.21.15.12:0/4126373229 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:26:11.688 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:26:11.560+0000 7f80d3200640 1 --2- 172.21.15.12:0/4126373229 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563d8cad27d0 0x563d8cad2c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:26:11.688 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:26:11.560+0000 7f80d1e00640 1 -- 172.21.15.12:0/4126373229 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563d8cac1af0 msgr2=0x563d8cad5e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:26:11.688 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:26:11.560+0000 7f80d1e00640 1 --2- 172.21.15.12:0/4126373229 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563d8cac1af0 0x563d8cad5e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:26:11.688 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:26:11.560+0000 7f80d1e00640 1 -- 172.21.15.12:0/4126373229 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563d8cad27d0 msgr2=0x563d8cad2c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:26:11.689 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:26:11.560+0000 7f80d1e00640 1 --2- 172.21.15.12:0/4126373229 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563d8cad27d0 0x563d8cad2c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:26:11.689 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:26:11.560+0000 7f80d1e00640 1 -- 172.21.15.12:0/4126373229 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563d8cb4a190 con 0x563d8cacda80 2024-09-11T00:26:11.689 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:26:11.560+0000 7f80d1e00640 1 --2- 172.21.15.12:0/4126373229 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563d8cacda80 0x563d8cad2290 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f80c00d5de0 tx=0x7f80c002a410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4fa813dd88c16ed5 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:26:11.689 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:26:11.560+0000 7f80b3e00640 10 client.0 ms_handle_connect on v2:172.21.15.178:3300/0 2024-09-11T00:26:11.689 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:26:11.560+0000 7f80b3e00640 1 -- 172.21.15.12:0/4126373229 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80c0049070 con 0x563d8cacda80 2024-09-11T00:26:11.689 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:26:11.560+0000 7f80dbe4a740 1 -- 172.21.15.12:0/4126373229 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f80c4014370 con 0x563d8cacda80 2024-09-11T00:26:11.689 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:26:11.560+0000 7f80b3e00640 1 -- 172.21.15.12:0/4126373229 <== mon.1 v2:172.21.15.178:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9208+0+0 (secure 0 0 0) 0x7f80c003b460 con 0x563d8cacda80 2024-09-11T00:26:11.689 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:26:11.561+0000 7f80b3e00640 1 -- 172.21.15.12:0/4126373229 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=72) ==== 24+0+0 (secure 0 0 0) 0x7f80c00cbe70 con 0x563d8cacda80 2024-09-11T00:26:11.689 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:26:11.561+0000 7f80dbe4a740 10 client.27580 fetch_fsmap learned FSMap version 72 2024-09-11T00:26:11.689 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:26:11.561+0000 7f80dbe4a740 1 -- 172.21.15.12:0/4126373229 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,fsmap=72,monmap=4+}) -- 0x563d8cb4a190 con 0x563d8cacda80 2024-09-11T00:26:11.689 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:26:11.561+0000 7f80b3e00640 1 -- 172.21.15.12:0/4126373229 <== mon.1 v2:172.21.15.178:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f80c4014370 con 0x563d8cacda80 2024-09-11T00:26:11.689 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:26:11.561+0000 7f80b3e00640 1 -- 172.21.15.12:0/4126373229 <== mon.1 v2:172.21.15.178:3300/0 5 ==== fsmap(e 72) ==== 8386+0+0 (secure 0 0 0) 0x7f80c00c8970 con 0x563d8cacda80 2024-09-11T00:26:11.689 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:26:11.561+0000 7f80dbe4a740 10 client.27580 fetch_fsmap finished waiting for FSMap version 72 2024-09-11T00:26:11.690 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:26:11.561+0000 7f80dbe4a740 10 client.27580 resolve_mds: validated gid 24674 aka daemon mds.b 2024-09-11T00:26:11.690 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:26:11.561+0000 7f80dbe4a740 20 client.27580 populate_metadata read hostname 'smithi012' 2024-09-11T00:26:11.690 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:26:11.561+0000 7f80dbe4a740 1 --2- 172.21.15.12:0/4126373229 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x563d8cb4b8a0 0x563d8cb4dcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:26:11.690 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:26:11.561+0000 7f80dbe4a740 4 client.27580 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "4921b205", "release": true, "await": true}] 2024-09-11T00:26:11.690 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:26:11.561+0000 7f80dbe4a740 1 -- 172.21.15.12:0/4126373229 --> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] -- command(tid 0: {"prefix": "quiesce db", "set_id": "4921b205", "release": true, "await": true}) -- 0x563d8ca35c90 con 0x563d8cb4b8a0 2024-09-11T00:26:11.690 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:26:11.562+0000 7f80d2800640 1 --2- 172.21.15.12:0/4126373229 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x563d8cb4b8a0 0x563d8cb4dcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:26:11.690 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:26:11.562+0000 7f80d2800640 1 --2- 172.21.15.12:0/4126373229 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x563d8cb4b8a0 0x563d8cb4dcc0 crc :-1 s=READY pgs=270 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f4360ea34121b266 server_cookie=468ac0dde4ce677b in_seq=0 out_seq=0 2024-09-11T00:26:11.691 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:26:11.562+0000 7f80b3e00640 10 client.27580 ms_handle_connect on v2:172.21.15.165:6838/501312984 2024-09-11T00:26:11.691 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:26:11.593+0000 7f80b3e00640 1 -- 172.21.15.12:0/4126373229 <== mds.0 v2:172.21.15.165:6838/501312984 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x563d8ca35c90 con 0x563d8cb4b8a0 2024-09-11T00:26:11.691 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:26:11.593+0000 7f80b3e00640 10 client.27580 handle_command_reply: tid=0 2024-09-11T00:26:11.691 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:26:11.593+0000 7f80dbe4a740 1 client.27580 shutdown 2024-09-11T00:26:11.691 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:26:11.593+0000 7f80dbe4a740 1 -- 172.21.15.12:0/4126373229 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563d8cacda80 msgr2=0x563d8cad2290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:26:11.691 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:26:11.593+0000 7f80dbe4a740 1 --2- 172.21.15.12:0/4126373229 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563d8cacda80 0x563d8cad2290 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f80c00d5de0 tx=0x7f80c002a410 comp rx=0 tx=0).stop 2024-09-11T00:26:11.691 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:26:11.593+0000 7f80dbe4a740 1 -- 172.21.15.12:0/4126373229 shutdown_connections 2024-09-11T00:26:11.691 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:26:11.593+0000 7f80dbe4a740 1 --2- 172.21.15.12:0/4126373229 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x563d8cb4b8a0 0x563d8cb4dcc0 crc :-1 s=READY pgs=270 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:26:11.691 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:26:11.593+0000 7f80dbe4a740 1 --2- 172.21.15.12:0/4126373229 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563d8cad27d0 0x563d8cad2c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:26:11.691 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:26:11.593+0000 7f80dbe4a740 1 --2- 172.21.15.12:0/4126373229 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563d8cacda80 0x563d8cad2290 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:26:11.691 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:26:11.593+0000 7f80dbe4a740 1 --2- 172.21.15.12:0/4126373229 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563d8cac1af0 0x563d8cad5e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:26:11.691 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:26:11.593+0000 7f80dbe4a740 1 -- 172.21.15.12:0/4126373229 >> 172.21.15.12:0/4126373229 conn(0x563d8cabf000 msgr2=0x563d8cad8ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:26:11.691 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:26:11.593+0000 7f80b3e00640 0 client.27580 ms_handle_reset on v2:172.21.15.165:6838/501312984 2024-09-11T00:26:11.692 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:26:11.593+0000 7f80dbe4a740 1 -- 172.21.15.12:0/4126373229 shutdown_connections 2024-09-11T00:26:11.692 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:26:11.593+0000 7f80dbe4a740 1 -- 172.21.15.12:0/4126373229 wait complete. 2024-09-11T00:26:11.692 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:26:11.594+0000 7f80dbe4a740 20 client.27580 trim_cache size 0 max 16384 2024-09-11T00:26:11.692 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-09-11T00:26:11.692 INFO:teuthology.orchestra.run.smithi012.stdout: "epoch": 72, 2024-09-11T00:26:11.692 INFO:teuthology.orchestra.run.smithi012.stdout: "leader": 24674, 2024-09-11T00:26:11.692 INFO:teuthology.orchestra.run.smithi012.stdout: "set_version": 60, 2024-09-11T00:26:11.692 INFO:teuthology.orchestra.run.smithi012.stdout: "sets": { 2024-09-11T00:26:11.692 INFO:teuthology.orchestra.run.smithi012.stdout: "4921b205": { 2024-09-11T00:26:11.693 INFO:teuthology.orchestra.run.smithi012.stdout: "version": 60, 2024-09-11T00:26:11.693 INFO:teuthology.orchestra.run.smithi012.stdout: "age_ref": 0.0, 2024-09-11T00:26:11.693 INFO:teuthology.orchestra.run.smithi012.stdout: "state": { 2024-09-11T00:26:11.693 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "RELEASED", 2024-09-11T00:26:11.693 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 0.0 2024-09-11T00:26:11.693 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:26:11.693 INFO:teuthology.orchestra.run.smithi012.stdout: "timeout": 90.0, 2024-09-11T00:26:11.693 INFO:teuthology.orchestra.run.smithi012.stdout: "expiration": 141.0, 2024-09-11T00:26:11.693 INFO:teuthology.orchestra.run.smithi012.stdout: "members": { 2024-09-11T00:26:11.693 INFO:teuthology.orchestra.run.smithi012.stdout: "file:/": { 2024-09-11T00:26:11.693 INFO:teuthology.orchestra.run.smithi012.stdout: "excluded": false, 2024-09-11T00:26:11.693 INFO:teuthology.orchestra.run.smithi012.stdout: "state": { 2024-09-11T00:26:11.693 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "RELEASED", 2024-09-11T00:26:11.693 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 0.0 2024-09-11T00:26:11.694 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:26:11.694 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:26:11.694 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:26:11.694 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:26:11.694 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:26:11.694 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-09-11T00:26:11.694 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:26:11.594+0000 7f80d8a00640 1 -- 172.21.15.12:0/3247255676 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f80b80b5410 msgr2=0x7f80b80b78d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:26:11.694 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:26:11.594+0000 7f80d8a00640 1 --2- 172.21.15.12:0/3247255676 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f80b80b5410 0x7f80b80b78d0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f80cc00aab0 tx=0x7f80cc009250 comp rx=0 tx=0).stop 2024-09-11T00:26:11.694 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:26:11.594+0000 7f80d8a00640 1 -- 172.21.15.12:0/3247255676 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f80d4108620 msgr2=0x7f80d4109cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:26:11.694 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:26:11.594+0000 7f80d8a00640 1 --2- 172.21.15.12:0/3247255676 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f80d4108620 0x7f80d4109cd0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f80c0009a20 tx=0x7f80c0003b90 comp rx=0 tx=0).stop 2024-09-11T00:26:11.694 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:26:11.594+0000 7f80d8a00640 1 -- 172.21.15.12:0/3247255676 shutdown_connections 2024-09-11T00:26:11.695 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:26:11.594+0000 7f80d8a00640 1 --2- 172.21.15.12:0/3247255676 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f80b80b5410 0x7f80b80b78d0 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:26:11.695 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:26:11.594+0000 7f80d8a00640 1 --2- 172.21.15.12:0/3247255676 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f80d4108fa0 0x7f80d410a210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:26:11.695 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:26:11.594+0000 7f80d8a00640 1 --2- 172.21.15.12:0/3247255676 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f80d4108620 0x7f80d4109cd0 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:26:11.695 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:26:11.594+0000 7f80d8a00640 1 --2- 172.21.15.12:0/3247255676 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f80d4102c10 0x7f80d4109790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:26:11.695 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:26:11.595+0000 7f80d8a00640 1 -- 172.21.15.12:0/3247255676 >> 172.21.15.12:0/3247255676 conn(0x7f80d40fe660 msgr2=0x7f80d4105c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:26:11.695 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:26:11.595+0000 7f80d8a00640 1 -- 172.21.15.12:0/3247255676 shutdown_connections 2024-09-11T00:26:11.695 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:26:11.595+0000 7f80d8a00640 1 -- 172.21.15.12:0/3247255676 wait complete. 2024-09-11T00:26:12.217 INFO:tasks.workunit.client.0.smithi012.stdout: 10 103765 15.38 MB/sec execute 248 sec latency 20766.719 ms 2024-09-11T00:26:12.532 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:26:12 smithi165 ceph-mon[35356]: pgmap v972: 129 pgs: 129 active+clean; 595 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 2.5 MiB/s wr, 80 op/s 2024-09-11T00:26:12.552 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:26:12 smithi012 ceph-mon[30333]: pgmap v972: 129 pgs: 129 active+clean; 595 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 2.5 MiB/s wr, 80 op/s 2024-09-11T00:26:12.581 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:26:12 smithi178 ceph-mon[35919]: pgmap v972: 129 pgs: 129 active+clean; 595 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 2.5 MiB/s wr, 80 op/s 2024-09-11T00:26:13.217 INFO:tasks.workunit.client.0.smithi012.stdout: 10 103769 15.32 MB/sec execute 249 sec latency 21706.126 ms 2024-09-11T00:26:13.292 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '4921b205', total seconds elapsed: 27.5 2024-09-11T00:26:14.212 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:26:14 smithi012 ceph-mon[30333]: pgmap v973: 129 pgs: 129 active+clean; 596 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 3.0 MiB/s wr, 99 op/s 2024-09-11T00:26:14.212 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:26:14 smithi012 ceph-mon[30333]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T00:26:14.217 INFO:tasks.workunit.client.0.smithi012.stdout: 10 104056 15.33 MB/sec execute 250 sec latency 22706.238 ms 2024-09-11T00:26:14.532 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:26:14 smithi165 ceph-mon[35356]: pgmap v973: 129 pgs: 129 active+clean; 596 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 3.0 MiB/s wr, 99 op/s 2024-09-11T00:26:14.533 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:26:14 smithi165 ceph-mon[35356]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T00:26:14.581 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:26:14 smithi178 ceph-mon[35919]: pgmap v973: 129 pgs: 129 active+clean; 596 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 3.0 MiB/s wr, 99 op/s 2024-09-11T00:26:14.581 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:26:14 smithi178 ceph-mon[35919]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T00:26:15.218 INFO:tasks.workunit.client.0.smithi012.stdout: 10 104809 15.37 MB/sec execute 251 sec latency 22900.290 ms 2024-09-11T00:26:16.218 INFO:tasks.workunit.client.0.smithi012.stdout: 10 105800 15.53 MB/sec execute 252 sec latency 85.581 ms 2024-09-11T00:26:16.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:26:16 smithi012 ceph-mon[30333]: pgmap v974: 129 pgs: 129 active+clean; 596 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 3.2 MiB/s wr, 100 op/s 2024-09-11T00:26:16.532 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:26:16 smithi165 ceph-mon[35356]: pgmap v974: 129 pgs: 129 active+clean; 596 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 3.2 MiB/s wr, 100 op/s 2024-09-11T00:26:16.581 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:26:16 smithi178 ceph-mon[35919]: pgmap v974: 129 pgs: 129 active+clean; 596 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 3.2 MiB/s wr, 100 op/s 2024-09-11T00:26:17.219 INFO:tasks.workunit.client.0.smithi012.stdout: 10 106610 15.66 MB/sec execute 253 sec latency 61.105 ms 2024-09-11T00:26:17.552 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:17.550+0000 7f35a0e00640 1 -- 172.21.15.12:0/4178184101 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f359c105b20 msgr2=0x7f359c105fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:26:17.553 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:17.550+0000 7f35a0e00640 1 --2- 172.21.15.12:0/4178184101 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f359c105b20 0x7f359c105fa0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f3588009890 tx=0x7f358802a360 comp rx=0 tx=0).stop 2024-09-11T00:26:17.553 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:17.550+0000 7f35a0e00640 1 -- 172.21.15.12:0/4178184101 shutdown_connections 2024-09-11T00:26:17.553 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:17.550+0000 7f35a0e00640 1 --2- 172.21.15.12:0/4178184101 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f359c1064e0 0x7f359c10ce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:26:17.553 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:17.550+0000 7f35a0e00640 1 --2- 172.21.15.12:0/4178184101 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f359c105b20 0x7f359c105fa0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:26:17.554 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:17.550+0000 7f35a0e00640 1 --2- 172.21.15.12:0/4178184101 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f359c1049d0 0x7f359c104dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:26:17.554 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:17.550+0000 7f35a0e00640 1 -- 172.21.15.12:0/4178184101 >> 172.21.15.12:0/4178184101 conn(0x7f359c100270 msgr2=0x7f359c1026b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:26:17.554 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:17.550+0000 7f35a0e00640 1 -- 172.21.15.12:0/4178184101 shutdown_connections 2024-09-11T00:26:17.554 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:17.550+0000 7f35a0e00640 1 -- 172.21.15.12:0/4178184101 wait complete. 2024-09-11T00:26:17.556 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:17.551+0000 7f35a0e00640 1 Processor -- start 2024-09-11T00:26:17.556 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:17.551+0000 7f35a0e00640 1 -- start start 2024-09-11T00:26:17.556 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:17.552+0000 7f35a0e00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f359c1049d0 0x7f359c19ae90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:26:17.556 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:17.552+0000 7f35a0e00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f359c105b20 0x7f359c19b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:26:17.556 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:17.552+0000 7f35a0e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f359c1064e0 0x7f359c19c0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:26:17.557 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:17.552+0000 7f35a0e00640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f359c110690 con 0x7f359c105b20 2024-09-11T00:26:17.557 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:17.552+0000 7f35a0e00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f359c110510 con 0x7f359c1064e0 2024-09-11T00:26:17.557 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:17.552+0000 7f35a0e00640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f359c110810 con 0x7f359c1049d0 2024-09-11T00:26:17.557 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:17.552+0000 7f359be00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f359c1064e0 0x7f359c19c0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:26:17.557 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:17.552+0000 7f359be00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f359c1064e0 0x7f359c19c0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.12:37266/0 (socket says 172.21.15.12:37266) 2024-09-11T00:26:17.557 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:17.552+0000 7f359be00640 1 -- 172.21.15.12:0/3348902585 learned_addr learned my addr 172.21.15.12:0/3348902585 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:26:17.557 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:17.552+0000 7f359b400640 1 --2- 172.21.15.12:0/3348902585 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f359c1049d0 0x7f359c19ae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:26:17.557 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:17.553+0000 7f359aa00640 1 --2- 172.21.15.12:0/3348902585 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f359c105b20 0x7f359c19b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:26:17.558 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:17.553+0000 7f359be00640 1 -- 172.21.15.12:0/3348902585 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f359c1049d0 msgr2=0x7f359c19ae90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:26:17.558 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:17.553+0000 7f359be00640 1 --2- 172.21.15.12:0/3348902585 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f359c1049d0 0x7f359c19ae90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:26:17.558 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:17.553+0000 7f359be00640 1 -- 172.21.15.12:0/3348902585 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f359c105b20 msgr2=0x7f359c19b4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:26:17.558 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:17.553+0000 7f359be00640 1 --2- 172.21.15.12:0/3348902585 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f359c105b20 0x7f359c19b4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:26:17.558 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:17.553+0000 7f359be00640 1 -- 172.21.15.12:0/3348902585 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f359c19c960 con 0x7f359c1064e0 2024-09-11T00:26:17.558 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:17.553+0000 7f359aa00640 1 --2- 172.21.15.12:0/3348902585 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f359c105b20 0x7f359c19b4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T00:26:17.558 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:17.553+0000 7f359be00640 1 --2- 172.21.15.12:0/3348902585 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f359c1064e0 0x7f359c19c0a0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f359400eb10 tx=0x7f359400ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:26:17.559 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:17.557+0000 7f3593e00640 1 -- 172.21.15.12:0/3348902585 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f359400ce20 con 0x7f359c1064e0 2024-09-11T00:26:17.560 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:17.557+0000 7f3593e00640 1 -- 172.21.15.12:0/3348902585 <== mon.2 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3594005020 con 0x7f359c1064e0 2024-09-11T00:26:17.560 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:17.557+0000 7f3593e00640 1 -- 172.21.15.12:0/3348902585 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3594010690 con 0x7f359c1064e0 2024-09-11T00:26:17.560 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:17.557+0000 7f35a0e00640 1 -- 172.21.15.12:0/3348902585 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f359c19cc50 con 0x7f359c1064e0 2024-09-11T00:26:17.560 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:17.557+0000 7f35a0e00640 1 -- 172.21.15.12:0/3348902585 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f359c06cd50 con 0x7f359c1064e0 2024-09-11T00:26:17.565 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:17.561+0000 7f35a52aa740 1 Processor -- start 2024-09-11T00:26:17.568 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:17.564+0000 7f3593e00640 1 -- 172.21.15.12:0/3348902585 <== mon.2 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7f3594005510 con 0x7f359c1064e0 2024-09-11T00:26:17.569 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:17.565+0000 7f35a52aa740 1 -- start start 2024-09-11T00:26:17.569 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:17.565+0000 7f35a52aa740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f8cae1e180 0x55f8cae20570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:26:17.569 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:17.565+0000 7f35a52aa740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f8cae20ab0 0x55f8cae22ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:26:17.569 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:17.565+0000 7f35a52aa740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f8cae234f0 0x55f8cae259b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:26:17.569 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:17.565+0000 7f35a52aa740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55f8cad886b0 con 0x55f8cae234f0 2024-09-11T00:26:17.569 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:17.565+0000 7f35a52aa740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55f8cad77ae0 con 0x55f8cae1e180 2024-09-11T00:26:17.570 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:17.565+0000 7f35a52aa740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55f8cad7b8f0 con 0x55f8cae20ab0 2024-09-11T00:26:17.570 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:17.565+0000 7f359b400640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f8cae20ab0 0x55f8cae22ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:26:17.570 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:17.565+0000 7f359b400640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f8cae20ab0 0x55f8cae22ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.12:49204/0 (socket says 172.21.15.12:49204) 2024-09-11T00:26:17.570 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:17.565+0000 7f359b400640 1 -- 172.21.15.12:0/3926608834 learned_addr learned my addr 172.21.15.12:0/3926608834 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:26:17.572 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:17.566+0000 7f359b400640 1 -- 172.21.15.12:0/3926608834 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f8cae1e180 msgr2=0x55f8cae20570 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-11T00:26:17.572 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:17.566+0000 7f359b400640 1 --2- 172.21.15.12:0/3926608834 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f8cae1e180 0x55f8cae20570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:26:17.572 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:17.566+0000 7f359b400640 1 -- 172.21.15.12:0/3926608834 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f8cae234f0 msgr2=0x55f8cae259b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-11T00:26:17.572 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:17.567+0000 7f359aa00640 1 --2- 172.21.15.12:0/3926608834 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f8cae234f0 0x55f8cae259b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:26:17.572 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:17.567+0000 7f359b400640 1 --2- 172.21.15.12:0/3926608834 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f8cae234f0 0x55f8cae259b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:26:17.573 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:17.567+0000 7f359b400640 1 -- 172.21.15.12:0/3926608834 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f8cad44780 con 0x55f8cae20ab0 2024-09-11T00:26:17.573 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:17.567+0000 7f359aa00640 1 --2- 172.21.15.12:0/3926608834 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f8cae234f0 0x55f8cae259b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T00:26:17.573 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:17.567+0000 7f359b400640 1 --2- 172.21.15.12:0/3926608834 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f8cae20ab0 0x55f8cae22ea0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f358c009ce0 tx=0x7f358c011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fb213b5218fa3115 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:26:17.573 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:17.568+0000 7f3590c00640 1 -- 172.21.15.12:0/3926608834 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f358c01e070 con 0x55f8cae20ab0 2024-09-11T00:26:17.573 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:17.568+0000 7f3590c00640 1 -- 172.21.15.12:0/3926608834 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f358c019ae0 con 0x55f8cae20ab0 2024-09-11T00:26:17.574 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:17.568+0000 7f3590c00640 1 -- 172.21.15.12:0/3926608834 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f358c01e070 con 0x55f8cae20ab0 2024-09-11T00:26:17.574 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:17.568+0000 7f3593e00640 1 --2- 172.21.15.12:0/3348902585 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f35800b5420 0x7f35800b78e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:26:17.574 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:17.569+0000 7f3593e00640 1 -- 172.21.15.12:0/3348902585 <== mon.2 v2:172.21.15.165:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9208+0+0 (secure 0 0 0) 0x7f35940e6620 con 0x7f359c1064e0 2024-09-11T00:26:17.577 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:17.570+0000 7f35a52aa740 1 -- 172.21.15.12:0/3926608834 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f8cae20ab0 msgr2=0x55f8cae22ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:26:17.577 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:17.570+0000 7f35a52aa740 1 --2- 172.21.15.12:0/3926608834 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f8cae20ab0 0x55f8cae22ea0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f358c009ce0 tx=0x7f358c011810 comp rx=0 tx=0).stop 2024-09-11T00:26:17.577 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:17.570+0000 7f35a52aa740 1 -- 172.21.15.12:0/3926608834 shutdown_connections 2024-09-11T00:26:17.578 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:17.570+0000 7f35a52aa740 1 --2- 172.21.15.12:0/3926608834 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f8cae234f0 0x55f8cae259b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:26:17.578 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:17.570+0000 7f35a52aa740 1 --2- 172.21.15.12:0/3926608834 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f8cae20ab0 0x55f8cae22ea0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:26:17.578 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:17.570+0000 7f35a52aa740 1 --2- 172.21.15.12:0/3926608834 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f8cae1e180 0x55f8cae20570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:26:17.578 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:17.570+0000 7f35a52aa740 1 -- 172.21.15.12:0/3926608834 >> 172.21.15.12:0/3926608834 conn(0x55f8cae192d0 msgr2=0x55f8cae1a060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:26:17.578 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:17.570+0000 7f35a52aa740 1 -- 172.21.15.12:0/3926608834 shutdown_connections 2024-09-11T00:26:17.578 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:17.570+0000 7f35a52aa740 1 -- 172.21.15.12:0/3926608834 wait complete. 2024-09-11T00:26:17.578 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:17.570+0000 7f359aa00640 1 --2- 172.21.15.12:0/3348902585 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f35800b5420 0x7f35800b78e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:26:17.579 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:17.571+0000 7f35a52aa740 1 -- start start 2024-09-11T00:26:17.579 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:17.571+0000 7f35a52aa740 1 Processor -- start 2024-09-11T00:26:17.579 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:17.571+0000 7f359aa00640 1 --2- 172.21.15.12:0/3348902585 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f35800b5420 0x7f35800b78e0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f35880095f0 tx=0x7f3588000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:26:17.579 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:17.572+0000 7f35a52aa740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f8cae1e180 0x55f8cae27860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:26:17.579 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:17.572+0000 7f35a52aa740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f8cae20ab0 0x55f8cae27f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:26:17.579 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:17.572+0000 7f35a52aa740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f8cae234f0 0x55f8cae28ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:26:17.579 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:17.572+0000 7f35a52aa740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55f8cacafab0 con 0x55f8cae1e180 2024-09-11T00:26:17.579 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:17.572+0000 7f35a52aa740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55f8cad1e570 con 0x55f8cae20ab0 2024-09-11T00:26:17.579 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:17.572+0000 7f35a52aa740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55f8cae264d0 con 0x55f8cae234f0 2024-09-11T00:26:17.579 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:17.572+0000 7f35a52aa740 10 client.0 _ll_register_callbacks cb 0x55f8cad2d100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:26:17.579 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:17.572+0000 7f359aa00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f8cae20ab0 0x55f8cae27f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:26:17.579 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:17.572+0000 7f359aa00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f8cae20ab0 0x55f8cae27f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.12:37272/0 (socket says 172.21.15.12:37272) 2024-09-11T00:26:17.580 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:17.572+0000 7f359aa00640 1 -- 172.21.15.12:0/3492369046 learned_addr learned my addr 172.21.15.12:0/3492369046 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:26:17.580 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:17.572+0000 7f359be00640 1 --2- 172.21.15.12:0/3492369046 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f8cae234f0 0x55f8cae28ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:26:17.580 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:17.572+0000 7f359b400640 1 --2- 172.21.15.12:0/3492369046 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f8cae1e180 0x55f8cae27860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:26:17.580 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:17.573+0000 7f359aa00640 1 -- 172.21.15.12:0/3492369046 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f8cae234f0 msgr2=0x55f8cae28ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:26:17.580 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:17.573+0000 7f359aa00640 1 --2- 172.21.15.12:0/3492369046 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f8cae234f0 0x55f8cae28ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:26:17.580 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:17.573+0000 7f359aa00640 1 -- 172.21.15.12:0/3492369046 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f8cae1e180 msgr2=0x55f8cae27860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:26:17.580 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:17.573+0000 7f359aa00640 1 --2- 172.21.15.12:0/3492369046 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f8cae1e180 0x55f8cae27860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:26:17.580 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:17.573+0000 7f359aa00640 1 -- 172.21.15.12:0/3492369046 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f8caad9790 con 0x55f8cae20ab0 2024-09-11T00:26:17.581 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:17.573+0000 7f359be00640 1 --2- 172.21.15.12:0/3492369046 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f8cae234f0 0x55f8cae28ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:26:17.581 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:17.573+0000 7f359b400640 1 --2- 172.21.15.12:0/3492369046 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f8cae1e180 0x55f8cae27860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T00:26:17.581 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:17.573+0000 7f359aa00640 1 --2- 172.21.15.12:0/3492369046 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f8cae20ab0 0x55f8cae27f00 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f3588034000 tx=0x7f358802a360 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9a702941898adb64 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:26:17.581 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:17.574+0000 7f3585600640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-09-11T00:26:17.581 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:17.574+0000 7f3585600640 1 -- 172.21.15.12:0/3492369046 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3588043040 con 0x55f8cae20ab0 2024-09-11T00:26:17.581 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:17.574+0000 7f3585600640 1 -- 172.21.15.12:0/3492369046 <== mon.2 v2:172.21.15.165:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9208+0+0 (secure 0 0 0) 0x7f3588005040 con 0x55f8cae20ab0 2024-09-11T00:26:17.581 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:17.575+0000 7f35a52aa740 1 -- 172.21.15.12:0/3492369046 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f358c019ae0 con 0x55f8cae20ab0 2024-09-11T00:26:17.582 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:17.576+0000 7f3585600640 1 -- 172.21.15.12:0/3492369046 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=74) ==== 24+0+0 (secure 0 0 0) 0x7f3588037040 con 0x55f8cae20ab0 2024-09-11T00:26:17.582 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:17.576+0000 7f35a52aa740 10 client.27611 fetch_fsmap learned FSMap version 74 2024-09-11T00:26:17.582 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:17.576+0000 7f35a52aa740 1 -- 172.21.15.12:0/3492369046 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=74,monmap=4+}) -- 0x55f8cab52620 con 0x55f8cae20ab0 2024-09-11T00:26:17.582 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:17.577+0000 7f3585600640 1 -- 172.21.15.12:0/3492369046 <== mon.2 v2:172.21.15.165:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f358c019ae0 con 0x55f8cae20ab0 2024-09-11T00:26:17.582 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:17.578+0000 7f3585600640 1 -- 172.21.15.12:0/3492369046 <== mon.2 v2:172.21.15.165:3300/0 5 ==== fsmap(e 74) ==== 8398+0+0 (secure 0 0 0) 0x7f358804f030 con 0x55f8cae20ab0 2024-09-11T00:26:17.582 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:17.579+0000 7f35a52aa740 10 client.27611 fetch_fsmap finished waiting for FSMap version 74 2024-09-11T00:26:17.582 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:17.579+0000 7f35a52aa740 10 client.27611 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T00:26:17.582 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:17.579+0000 7f35a52aa740 20 client.27611 populate_metadata read hostname 'smithi012' 2024-09-11T00:26:17.583 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:17.579+0000 7f35a52aa740 1 --2- 172.21.15.12:0/3492369046 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55f8cae33570 0x55f8cae35a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:26:17.583 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:17.579+0000 7f35a52aa740 4 client.27611 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T00:26:17.583 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:17.579+0000 7f35a52aa740 1 -- 172.21.15.12:0/3492369046 --> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f8cad8a600 con 0x55f8cae33570 2024-09-11T00:26:17.583 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:17.580+0000 7f359b400640 1 --2- 172.21.15.12:0/3492369046 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55f8cae33570 0x55f8cae35a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:26:17.583 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:17.580+0000 7f359b400640 1 --2- 172.21.15.12:0/3492369046 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55f8cae33570 0x55f8cae35a30 crc :-1 s=READY pgs=271 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dcb3b6b5c275657c server_cookie=6a4571ba4146c304 in_seq=0 out_seq=0 2024-09-11T00:26:17.583 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:17.580+0000 7f3585600640 10 client.27611 ms_handle_connect on v2:172.21.15.165:6838/501312984 2024-09-11T00:26:17.584 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:17.582+0000 7f3585600640 1 -- 172.21.15.12:0/3492369046 <== mds.0 v2:172.21.15.165:6838/501312984 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55f8cad8a600 con 0x55f8cae33570 2024-09-11T00:26:17.584 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:17.582+0000 7f3585600640 10 client.27611 handle_command_reply: tid=0 2024-09-11T00:26:17.584 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:17.582+0000 7f35a52aa740 1 client.27611 shutdown 2024-09-11T00:26:17.585 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:17.582+0000 7f35a52aa740 1 -- 172.21.15.12:0/3492369046 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f8cae20ab0 msgr2=0x55f8cae27f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:26:17.585 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:17.582+0000 7f35a52aa740 1 --2- 172.21.15.12:0/3492369046 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f8cae20ab0 0x55f8cae27f00 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f3588034000 tx=0x7f358802a360 comp rx=0 tx=0).stop 2024-09-11T00:26:17.585 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:17.582+0000 7f35a52aa740 1 -- 172.21.15.12:0/3492369046 shutdown_connections 2024-09-11T00:26:17.585 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:17.582+0000 7f35a52aa740 1 --2- 172.21.15.12:0/3492369046 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55f8cae33570 0x55f8cae35a30 crc :-1 s=READY pgs=271 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:26:17.585 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:17.582+0000 7f35a52aa740 1 --2- 172.21.15.12:0/3492369046 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f8cae234f0 0x55f8cae28ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:26:17.585 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:17.582+0000 7f35a52aa740 1 --2- 172.21.15.12:0/3492369046 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f8cae20ab0 0x55f8cae27f00 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:26:17.585 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:17.582+0000 7f3585600640 0 client.27611 ms_handle_reset on v2:172.21.15.165:6838/501312984 2024-09-11T00:26:17.585 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:17.582+0000 7f35a52aa740 1 --2- 172.21.15.12:0/3492369046 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f8cae1e180 0x55f8cae27860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:26:17.585 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:17.582+0000 7f35a52aa740 1 -- 172.21.15.12:0/3492369046 >> 172.21.15.12:0/3492369046 conn(0x55f8cae192d0 msgr2=0x55f8cae0fbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:26:17.586 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:17.583+0000 7f35a52aa740 1 -- 172.21.15.12:0/3492369046 shutdown_connections 2024-09-11T00:26:17.586 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:17.583+0000 7f35a52aa740 1 -- 172.21.15.12:0/3492369046 wait complete. 2024-09-11T00:26:17.586 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:17.583+0000 7f35a52aa740 20 client.27611 trim_cache size 0 max 16384 2024-09-11T00:26:17.610 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:17.605+0000 7f35a52aa740 1 Processor -- start 2024-09-11T00:26:17.611 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:17.605+0000 7f35a52aa740 1 -- start start 2024-09-11T00:26:17.611 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:17.605+0000 7f35a52aa740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f8cae1e180 0x55f8cae27860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:26:17.611 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:17.605+0000 7f35a52aa740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f8cae20ab0 0x55f8cae1d670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:26:17.611 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:17.605+0000 7f35a52aa740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f8cae234f0 0x55f8cae1dbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:26:17.611 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:17.605+0000 7f35a52aa740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f359c072e10 con 0x55f8cae1e180 2024-09-11T00:26:17.611 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:17.605+0000 7f35a52aa740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f359c074a70 con 0x55f8cae234f0 2024-09-11T00:26:17.611 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:17.605+0000 7f35a52aa740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f359c073510 con 0x55f8cae20ab0 2024-09-11T00:26:17.611 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:17.606+0000 7f359be00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f8cae234f0 0x55f8cae1dbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:26:17.611 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:17.606+0000 7f359be00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f8cae234f0 0x55f8cae1dbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.12:37278/0 (socket says 172.21.15.12:37278) 2024-09-11T00:26:17.611 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:17.606+0000 7f359be00640 1 -- 172.21.15.12:0/1965231602 learned_addr learned my addr 172.21.15.12:0/1965231602 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:26:17.611 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:17.606+0000 7f359aa00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f8cae20ab0 0x55f8cae1d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:26:17.611 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:17.606+0000 7f359b400640 1 --2- 172.21.15.12:0/1965231602 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f8cae1e180 0x55f8cae27860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:26:17.612 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:17.606+0000 7f359aa00640 1 -- 172.21.15.12:0/1965231602 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f8cae234f0 msgr2=0x55f8cae1dbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:26:17.612 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:17.606+0000 7f359aa00640 1 --2- 172.21.15.12:0/1965231602 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f8cae234f0 0x55f8cae1dbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:26:17.612 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:17.606+0000 7f359aa00640 1 -- 172.21.15.12:0/1965231602 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f8cae1e180 msgr2=0x55f8cae27860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:26:17.612 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:17.606+0000 7f359aa00640 1 --2- 172.21.15.12:0/1965231602 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f8cae1e180 0x55f8cae27860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:26:17.612 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:17.606+0000 7f359aa00640 1 -- 172.21.15.12:0/1965231602 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f8cae208e0 con 0x55f8cae20ab0 2024-09-11T00:26:17.612 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:17.606+0000 7f359aa00640 1 --2- 172.21.15.12:0/1965231602 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f8cae20ab0 0x55f8cae1d670 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f3588009250 tx=0x7f3588030fa0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d033ce233178e52b server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:26:17.612 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:17.607+0000 7f3587e00640 1 -- 172.21.15.12:0/1965231602 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3588043040 con 0x55f8cae20ab0 2024-09-11T00:26:17.612 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:17.607+0000 7f3587e00640 1 -- 172.21.15.12:0/1965231602 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3588003960 con 0x55f8cae20ab0 2024-09-11T00:26:17.612 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:17.607+0000 7f3587e00640 1 -- 172.21.15.12:0/1965231602 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3588005240 con 0x55f8cae20ab0 2024-09-11T00:26:17.612 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:17.607+0000 7f35a52aa740 1 -- 172.21.15.12:0/1965231602 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f8cae20ab0 msgr2=0x55f8cae1d670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:26:17.612 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:17.607+0000 7f35a52aa740 1 --2- 172.21.15.12:0/1965231602 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f8cae20ab0 0x55f8cae1d670 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f3588009250 tx=0x7f3588030fa0 comp rx=0 tx=0).stop 2024-09-11T00:26:17.612 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:17.608+0000 7f35a52aa740 1 -- 172.21.15.12:0/1965231602 shutdown_connections 2024-09-11T00:26:17.612 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:17.608+0000 7f35a52aa740 1 --2- 172.21.15.12:0/1965231602 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f8cae234f0 0x55f8cae1dbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:26:17.613 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:17.608+0000 7f35a52aa740 1 --2- 172.21.15.12:0/1965231602 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f8cae20ab0 0x55f8cae1d670 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:26:17.613 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:17.608+0000 7f35a52aa740 1 --2- 172.21.15.12:0/1965231602 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f8cae1e180 0x55f8cae27860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:26:17.613 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:17.608+0000 7f35a52aa740 1 -- 172.21.15.12:0/1965231602 >> 172.21.15.12:0/1965231602 conn(0x55f8cae192d0 msgr2=0x55f8cae19bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:26:17.613 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:17.608+0000 7f35a52aa740 1 -- 172.21.15.12:0/1965231602 shutdown_connections 2024-09-11T00:26:17.613 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:17.608+0000 7f35a52aa740 1 -- 172.21.15.12:0/1965231602 wait complete. 2024-09-11T00:26:17.613 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:17.608+0000 7f35a52aa740 1 -- start start 2024-09-11T00:26:17.613 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:17.608+0000 7f35a52aa740 1 Processor -- start 2024-09-11T00:26:17.614 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:17.609+0000 7f35a52aa740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f8cae1e180 0x55f8cae2c830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:26:17.615 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:17.609+0000 7f35a52aa740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f8cae20ab0 0x55f8cae2ced0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:26:17.615 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:17.609+0000 7f35a52aa740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f8cae234f0 0x55f8cae2daa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:26:17.615 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:17.609+0000 7f35a52aa740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f359c0743a0 con 0x55f8cae234f0 2024-09-11T00:26:17.615 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:17.609+0000 7f35a52aa740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f359c073c40 con 0x55f8cae1e180 2024-09-11T00:26:17.615 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:17.609+0000 7f35a52aa740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f359c070e10 con 0x55f8cae20ab0 2024-09-11T00:26:17.615 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:17.609+0000 7f35a52aa740 10 client.0 _ll_register_callbacks cb 0x7f3588040040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:26:17.615 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:17.609+0000 7f359b400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f8cae1e180 0x55f8cae2c830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:26:17.615 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:17.609+0000 7f359b400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f8cae1e180 0x55f8cae2c830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.12:37282/0 (socket says 172.21.15.12:37282) 2024-09-11T00:26:17.615 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:17.609+0000 7f359b400640 1 -- 172.21.15.12:0/560532071 learned_addr learned my addr 172.21.15.12:0/560532071 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:26:17.615 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:17.609+0000 7f359aa00640 1 --2- 172.21.15.12:0/560532071 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f8cae20ab0 0x55f8cae2ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:26:17.615 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:17.609+0000 7f359be00640 1 --2- 172.21.15.12:0/560532071 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f8cae234f0 0x55f8cae2daa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:26:17.615 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:17.610+0000 7f359b400640 1 -- 172.21.15.12:0/560532071 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f8cae20ab0 msgr2=0x55f8cae2ced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:26:17.616 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:17.610+0000 7f359b400640 1 --2- 172.21.15.12:0/560532071 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f8cae20ab0 0x55f8cae2ced0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:26:17.616 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:17.610+0000 7f359b400640 1 -- 172.21.15.12:0/560532071 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f8cae234f0 msgr2=0x55f8cae2daa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:26:17.616 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:17.610+0000 7f359b400640 1 --2- 172.21.15.12:0/560532071 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f8cae234f0 0x55f8cae2daa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:26:17.616 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:17.610+0000 7f359b400640 1 -- 172.21.15.12:0/560532071 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f8cae2e2d0 con 0x55f8cae1e180 2024-09-11T00:26:17.616 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:17.610+0000 7f359b400640 1 --2- 172.21.15.12:0/560532071 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f8cae1e180 0x55f8cae2c830 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f358c006010 tx=0x7f358c01af50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a2b721b3bf314c7b server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:26:17.616 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:17.610+0000 7f3585600640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-09-11T00:26:17.616 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:17.611+0000 7f3585600640 1 -- 172.21.15.12:0/560532071 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f358c031060 con 0x55f8cae1e180 2024-09-11T00:26:17.616 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:17.611+0000 7f3585600640 1 -- 172.21.15.12:0/560532071 <== mon.2 v2:172.21.15.165:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9208+0+0 (secure 0 0 0) 0x7f358c0054d0 con 0x55f8cae1e180 2024-09-11T00:26:17.616 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:17.611+0000 7f35a52aa740 1 -- 172.21.15.12:0/560532071 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3588003960 con 0x55f8cae1e180 2024-09-11T00:26:17.616 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:17.612+0000 7f3585600640 1 -- 172.21.15.12:0/560532071 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=74) ==== 24+0+0 (secure 0 0 0) 0x7f358c01e040 con 0x55f8cae1e180 2024-09-11T00:26:17.616 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:17.612+0000 7f35a52aa740 10 client.27617 fetch_fsmap learned FSMap version 74 2024-09-11T00:26:17.616 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:17.612+0000 7f35a52aa740 1 -- 172.21.15.12:0/560532071 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=74,monmap=4+}) -- 0x55f8cae2f6a0 con 0x55f8cae1e180 2024-09-11T00:26:17.616 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:17.612+0000 7f3585600640 1 -- 172.21.15.12:0/560532071 <== mon.2 v2:172.21.15.165:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3588003960 con 0x55f8cae1e180 2024-09-11T00:26:17.617 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:17.612+0000 7f3585600640 1 -- 172.21.15.12:0/560532071 <== mon.2 v2:172.21.15.165:3300/0 5 ==== fsmap(e 74) ==== 8398+0+0 (secure 0 0 0) 0x7f358c00e6f0 con 0x55f8cae1e180 2024-09-11T00:26:17.617 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:17.613+0000 7f35a52aa740 10 client.27617 fetch_fsmap finished waiting for FSMap version 74 2024-09-11T00:26:17.617 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:17.613+0000 7f35a52aa740 10 client.27617 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T00:26:17.617 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:17.613+0000 7f35a52aa740 20 client.27617 populate_metadata read hostname 'smithi012' 2024-09-11T00:26:17.617 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:17.613+0000 7f35a52aa740 1 --2- 172.21.15.12:0/560532071 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55f8caea0950 0x55f8caea2d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:26:17.617 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:17.613+0000 7f35a52aa740 4 client.27617 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T00:26:17.618 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:17.613+0000 7f35a52aa740 1 -- 172.21.15.12:0/560532071 --> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55f8cad860b0 con 0x55f8caea0950 2024-09-11T00:26:17.618 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:17.614+0000 7f359aa00640 1 --2- 172.21.15.12:0/560532071 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55f8caea0950 0x55f8caea2d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:26:17.618 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:17.614+0000 7f359aa00640 1 --2- 172.21.15.12:0/560532071 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55f8caea0950 0x55f8caea2d70 crc :-1 s=READY pgs=272 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=288d9aa32f10c903 server_cookie=cb5422d9d42b0212 in_seq=0 out_seq=0 2024-09-11T00:26:17.618 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:17.614+0000 7f3585600640 10 client.27617 ms_handle_connect on v2:172.21.15.165:6838/501312984 2024-09-11T00:26:17.818 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:17.815+0000 7f3585600640 1 -- 172.21.15.12:0/560532071 <== mds.0 v2:172.21.15.165:6838/501312984 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55f8cad860b0 con 0x55f8caea0950 2024-09-11T00:26:17.818 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:17.815+0000 7f3585600640 10 client.27617 handle_command_reply: tid=0 2024-09-11T00:26:17.818 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:17.815+0000 7f35a52aa740 1 client.27617 shutdown 2024-09-11T00:26:17.818 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:17.815+0000 7f35a52aa740 1 -- 172.21.15.12:0/560532071 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f8cae1e180 msgr2=0x55f8cae2c830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:26:17.818 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:17.816+0000 7f35a52aa740 1 --2- 172.21.15.12:0/560532071 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f8cae1e180 0x55f8cae2c830 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f358c006010 tx=0x7f358c01af50 comp rx=0 tx=0).stop 2024-09-11T00:26:17.819 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:17.816+0000 7f35a52aa740 1 -- 172.21.15.12:0/560532071 shutdown_connections 2024-09-11T00:26:17.819 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:17.816+0000 7f35a52aa740 1 --2- 172.21.15.12:0/560532071 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55f8caea0950 0x55f8caea2d70 crc :-1 s=READY pgs=272 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:26:17.819 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:17.816+0000 7f35a52aa740 1 --2- 172.21.15.12:0/560532071 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f8cae234f0 0x55f8cae2daa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:26:17.819 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:17.816+0000 7f35a52aa740 1 --2- 172.21.15.12:0/560532071 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f8cae20ab0 0x55f8cae2ced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:26:17.819 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:17.816+0000 7f35a52aa740 1 --2- 172.21.15.12:0/560532071 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f8cae1e180 0x55f8cae2c830 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:26:17.819 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:17.816+0000 7f35a52aa740 1 -- 172.21.15.12:0/560532071 >> 172.21.15.12:0/560532071 conn(0x55f8cae192d0 msgr2=0x55f8cae1c320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:26:17.819 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:17.816+0000 7f3585600640 0 client.27617 ms_handle_reset on v2:172.21.15.165:6838/501312984 2024-09-11T00:26:17.819 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:17.817+0000 7f35a52aa740 1 -- 172.21.15.12:0/560532071 shutdown_connections 2024-09-11T00:26:17.819 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:17.817+0000 7f35a52aa740 1 -- 172.21.15.12:0/560532071 wait complete. 2024-09-11T00:26:17.820 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:17.818+0000 7f35a52aa740 20 client.27617 trim_cache size 0 max 16384 2024-09-11T00:26:17.820 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:{ 2024-09-11T00:26:17.820 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "message": "", 2024-09-11T00:26:17.821 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "return_code": 0 2024-09-11T00:26:17.821 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:} 2024-09-11T00:26:17.821 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:17.818+0000 7f35a0e00640 1 -- 172.21.15.12:0/3348902585 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f35800b5420 msgr2=0x7f35800b78e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:26:17.821 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:17.818+0000 7f35a0e00640 1 --2- 172.21.15.12:0/3348902585 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f35800b5420 0x7f35800b78e0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f35880095f0 tx=0x7f3588000bc0 comp rx=0 tx=0).stop 2024-09-11T00:26:17.821 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:17.818+0000 7f35a0e00640 1 -- 172.21.15.12:0/3348902585 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f359c1064e0 msgr2=0x7f359c19c0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:26:17.821 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:17.818+0000 7f35a0e00640 1 --2- 172.21.15.12:0/3348902585 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f359c1064e0 0x7f359c19c0a0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f359400eb10 tx=0x7f359400ef10 comp rx=0 tx=0).stop 2024-09-11T00:26:17.822 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:17.819+0000 7f35a0e00640 1 -- 172.21.15.12:0/3348902585 shutdown_connections 2024-09-11T00:26:17.822 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:17.819+0000 7f35a0e00640 1 --2- 172.21.15.12:0/3348902585 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f35800b5420 0x7f35800b78e0 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:26:17.822 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:17.819+0000 7f35a0e00640 1 --2- 172.21.15.12:0/3348902585 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f359c1064e0 0x7f359c19c0a0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:26:17.822 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:17.819+0000 7f35a0e00640 1 --2- 172.21.15.12:0/3348902585 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f359c105b20 0x7f359c19b4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:26:17.822 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:17.819+0000 7f35a0e00640 1 --2- 172.21.15.12:0/3348902585 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f359c1049d0 0x7f359c19ae90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:26:17.822 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:17.819+0000 7f35a0e00640 1 -- 172.21.15.12:0/3348902585 >> 172.21.15.12:0/3348902585 conn(0x7f359c100270 msgr2=0x7f359c108ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:26:17.822 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:17.819+0000 7f35a0e00640 1 -- 172.21.15.12:0/3348902585 shutdown_connections 2024-09-11T00:26:17.822 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:17.820+0000 7f35a0e00640 1 -- 172.21.15.12:0/3348902585 wait complete. 2024-09-11T00:26:18.219 INFO:tasks.workunit.client.0.smithi012.stdout: 10 107283 15.77 MB/sec execute 254 sec latency 123.346 ms 2024-09-11T00:26:18.533 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:26:18 smithi165 ceph-mon[35356]: pgmap v975: 129 pgs: 129 active+clean; 605 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 5.3 MiB/s wr, 294 op/s 2024-09-11T00:26:18.533 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:26:18 smithi165 ceph-mon[35356]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T00:26:18.576 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:26:18 smithi012 ceph-mon[30333]: pgmap v975: 129 pgs: 129 active+clean; 605 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 5.3 MiB/s wr, 294 op/s 2024-09-11T00:26:18.576 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:26:18 smithi012 ceph-mon[30333]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T00:26:18.583 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:26:18 smithi178 ceph-mon[35919]: pgmap v975: 129 pgs: 129 active+clean; 605 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 5.3 MiB/s wr, 294 op/s 2024-09-11T00:26:18.583 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:26:18 smithi178 ceph-mon[35919]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T00:26:19.219 INFO:tasks.workunit.client.0.smithi012.stdout: 10 108041 15.87 MB/sec execute 255 sec latency 137.011 ms 2024-09-11T00:26:19.532 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:26:19 smithi165 ceph-mon[35356]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T00:26:19.563 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:26:19 smithi012 ceph-mon[30333]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T00:26:19.581 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:26:19 smithi178 ceph-mon[35919]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T00:26:20.220 INFO:tasks.workunit.client.0.smithi012.stdout: 10 108865 15.94 MB/sec execute 256 sec latency 103.350 ms 2024-09-11T00:26:20.522 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:26:20 smithi012 ceph-mon[30333]: pgmap v976: 129 pgs: 129 active+clean; 604 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 5.0 MiB/s wr, 278 op/s 2024-09-11T00:26:20.532 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:26:20 smithi165 ceph-mon[35356]: pgmap v976: 129 pgs: 129 active+clean; 604 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 5.0 MiB/s wr, 278 op/s 2024-09-11T00:26:20.581 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:26:20 smithi178 ceph-mon[35919]: pgmap v976: 129 pgs: 129 active+clean; 604 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 5.0 MiB/s wr, 278 op/s 2024-09-11T00:26:21.220 INFO:tasks.workunit.client.0.smithi012.stdout: 10 109731 16.07 MB/sec execute 257 sec latency 56.896 ms 2024-09-11T00:26:22.220 INFO:tasks.workunit.client.0.smithi012.stdout: 10 110561 16.19 MB/sec execute 258 sec latency 87.097 ms 2024-09-11T00:26:22.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:26:22 smithi012 ceph-mon[30333]: pgmap v977: 129 pgs: 129 active+clean; 597 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 5.6 MiB/s wr, 388 op/s 2024-09-11T00:26:22.532 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:26:22 smithi165 ceph-mon[35356]: pgmap v977: 129 pgs: 129 active+clean; 597 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 5.6 MiB/s wr, 388 op/s 2024-09-11T00:26:22.581 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:26:22 smithi178 ceph-mon[35919]: pgmap v977: 129 pgs: 129 active+clean; 597 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 5.6 MiB/s wr, 388 op/s 2024-09-11T00:26:23.220 INFO:tasks.workunit.client.0.smithi012.stdout: 10 111361 16.30 MB/sec execute 259 sec latency 63.897 ms 2024-09-11T00:26:23.532 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:26:23 smithi165 ceph-mon[35356]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T00:26:23.533 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:26:23 smithi165 ceph-mon[35356]: pgmap v978: 129 pgs: 129 active+clean; 607 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 MiB/s rd, 4.9 MiB/s wr, 539 op/s 2024-09-11T00:26:23.563 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:26:23 smithi012 ceph-mon[30333]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T00:26:23.563 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:26:23 smithi012 ceph-mon[30333]: pgmap v978: 129 pgs: 129 active+clean; 607 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 MiB/s rd, 4.9 MiB/s wr, 539 op/s 2024-09-11T00:26:23.581 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:26:23 smithi178 ceph-mon[35919]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T00:26:23.581 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:26:23 smithi178 ceph-mon[35919]: pgmap v978: 129 pgs: 129 active+clean; 607 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 MiB/s rd, 4.9 MiB/s wr, 539 op/s 2024-09-11T00:26:24.220 INFO:tasks.workunit.client.0.smithi012.stdout: 10 112152 16.38 MB/sec execute 260 sec latency 66.707 ms 2024-09-11T00:26:25.221 INFO:tasks.workunit.client.0.smithi012.stdout: 10 112781 16.45 MB/sec execute 261 sec latency 170.214 ms 2024-09-11T00:26:25.813 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:26:25 smithi012 ceph-mon[30333]: pgmap v979: 129 pgs: 129 active+clean; 607 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 MiB/s rd, 4.4 MiB/s wr, 519 op/s 2024-09-11T00:26:26.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:26:25 smithi165 ceph-mon[35356]: pgmap v979: 129 pgs: 129 active+clean; 607 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 MiB/s rd, 4.4 MiB/s wr, 519 op/s 2024-09-11T00:26:26.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:26:25 smithi178 ceph-mon[35919]: pgmap v979: 129 pgs: 129 active+clean; 607 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 MiB/s rd, 4.4 MiB/s wr, 519 op/s 2024-09-11T00:26:26.221 INFO:tasks.workunit.client.0.smithi012.stdout: 10 113636 16.58 MB/sec execute 262 sec latency 133.669 ms 2024-09-11T00:26:27.221 INFO:tasks.workunit.client.0.smithi012.stdout: 10 114290 16.64 MB/sec execute 263 sec latency 158.308 ms 2024-09-11T00:26:27.792 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:26:27 smithi012 ceph-mon[30333]: pgmap v980: 129 pgs: 129 active+clean; 623 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 5.9 MiB/s wr, 748 op/s 2024-09-11T00:26:27.792 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:26:27 smithi012 ceph-mon[30333]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T00:26:28.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:26:27 smithi165 ceph-mon[35356]: pgmap v980: 129 pgs: 129 active+clean; 623 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 5.9 MiB/s wr, 748 op/s 2024-09-11T00:26:28.033 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:26:27 smithi165 ceph-mon[35356]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T00:26:28.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:26:27 smithi178 ceph-mon[35919]: pgmap v980: 129 pgs: 129 active+clean; 623 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 5.9 MiB/s wr, 748 op/s 2024-09-11T00:26:28.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:26:27 smithi178 ceph-mon[35919]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T00:26:28.221 INFO:tasks.workunit.client.0.smithi012.stdout: 10 114518 16.63 MB/sec execute 264 sec latency 692.752 ms 2024-09-11T00:26:29.221 INFO:tasks.workunit.client.0.smithi012.stdout: 10 114520 16.57 MB/sec execute 265 sec latency 1692.809 ms 2024-09-11T00:26:29.766 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:26:29 smithi012 ceph-mon[30333]: pgmap v981: 129 pgs: 129 active+clean; 624 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 3.9 MiB/s wr, 576 op/s 2024-09-11T00:26:30.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:26:29 smithi165 ceph-mon[35356]: pgmap v981: 129 pgs: 129 active+clean; 624 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 3.9 MiB/s wr, 576 op/s 2024-09-11T00:26:30.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:26:29 smithi178 ceph-mon[35919]: pgmap v981: 129 pgs: 129 active+clean; 624 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 3.9 MiB/s wr, 576 op/s 2024-09-11T00:26:30.221 INFO:tasks.workunit.client.0.smithi012.stdout: 10 114580 16.53 MB/sec execute 266 sec latency 2521.914 ms 2024-09-11T00:26:30.984 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:30.980+0000 7f3c0a400640 1 -- 172.21.15.12:0/64155519 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3c04104ba0 msgr2=0x7f3c04104fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:26:30.984 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:30.980+0000 7f3c0a400640 1 --2- 172.21.15.12:0/64155519 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3c04104ba0 0x7f3c04104fa0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f3bf4009930 tx=0x7f3bf402a440 comp rx=0 tx=0).stop 2024-09-11T00:26:30.984 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:30.980+0000 7f3c0a400640 1 -- 172.21.15.12:0/64155519 shutdown_connections 2024-09-11T00:26:30.984 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:30.980+0000 7f3c0a400640 1 --2- 172.21.15.12:0/64155519 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3c041066b0 0x7f3c0410d030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:26:30.984 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:30.980+0000 7f3c0a400640 1 --2- 172.21.15.12:0/64155519 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3c04105cf0 0x7f3c04106170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:26:30.984 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:30.980+0000 7f3c0a400640 1 --2- 172.21.15.12:0/64155519 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3c04104ba0 0x7f3c04104fa0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:26:30.984 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:30.980+0000 7f3c0a400640 1 -- 172.21.15.12:0/64155519 >> 172.21.15.12:0/64155519 conn(0x7f3c04100460 msgr2=0x7f3c04102880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:26:30.985 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:30.980+0000 7f3c0a400640 1 -- 172.21.15.12:0/64155519 shutdown_connections 2024-09-11T00:26:30.985 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:30.980+0000 7f3c0a400640 1 -- 172.21.15.12:0/64155519 wait complete. 2024-09-11T00:26:30.987 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:30.985+0000 7f3c0a400640 1 Processor -- start 2024-09-11T00:26:30.988 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:30.985+0000 7f3c0a400640 1 -- start start 2024-09-11T00:26:30.991 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:30.985+0000 7f3c0a400640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3c04104ba0 0x7f3c0419f380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:26:30.991 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:30.985+0000 7f3c0a400640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3c04105cf0 0x7f3c0419f9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:26:30.991 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:30.985+0000 7f3c0a400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3c041066b0 0x7f3c041a0600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:26:30.991 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:30.985+0000 7f3c0a400640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f3c04078240 con 0x7f3c04105cf0 2024-09-11T00:26:30.991 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:30.985+0000 7f3c0a400640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f3c040780c0 con 0x7f3c041066b0 2024-09-11T00:26:30.991 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:30.985+0000 7f3c0a400640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f3c040783c0 con 0x7f3c04104ba0 2024-09-11T00:26:30.991 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:30.985+0000 7f3c03400640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3c04104ba0 0x7f3c0419f380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:26:30.991 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:30.985+0000 7f3c03400640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3c04104ba0 0x7f3c0419f380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.12:50078/0 (socket says 172.21.15.12:50078) 2024-09-11T00:26:30.991 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:30.985+0000 7f3c03400640 1 -- 172.21.15.12:0/2203554600 learned_addr learned my addr 172.21.15.12:0/2203554600 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:26:30.991 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:30.985+0000 7f3c02a00640 1 --2- 172.21.15.12:0/2203554600 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3c04105cf0 0x7f3c0419f9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:26:30.991 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:30.985+0000 7f3c03e00640 1 --2- 172.21.15.12:0/2203554600 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3c041066b0 0x7f3c041a0600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:26:30.992 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:30.985+0000 7f3c03400640 1 -- 172.21.15.12:0/2203554600 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3c041066b0 msgr2=0x7f3c041a0600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:26:30.992 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:30.985+0000 7f3c03400640 1 --2- 172.21.15.12:0/2203554600 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3c041066b0 0x7f3c041a0600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:26:30.992 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:30.985+0000 7f3c03400640 1 -- 172.21.15.12:0/2203554600 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3c04105cf0 msgr2=0x7f3c0419f9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:26:30.992 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:30.985+0000 7f3c03400640 1 --2- 172.21.15.12:0/2203554600 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3c04105cf0 0x7f3c0419f9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:26:30.992 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:30.985+0000 7f3c03400640 1 -- 172.21.15.12:0/2203554600 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c041a0ec0 con 0x7f3c04104ba0 2024-09-11T00:26:30.992 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:30.985+0000 7f3c03e00640 1 --2- 172.21.15.12:0/2203554600 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3c041066b0 0x7f3c041a0600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T00:26:30.992 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:30.990+0000 7f3c03400640 1 --2- 172.21.15.12:0/2203554600 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3c04104ba0 0x7f3c0419f380 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f3bf4009850 tx=0x7f3bf4003c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:26:30.993 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:30.990+0000 7f3bfbe00640 1 -- 172.21.15.12:0/2203554600 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bf403f070 con 0x7f3c04104ba0 2024-09-11T00:26:30.993 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:30.990+0000 7f3bfbe00640 1 -- 172.21.15.12:0/2203554600 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3bf4031c50 con 0x7f3c04104ba0 2024-09-11T00:26:30.993 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:30.990+0000 7f3bfbe00640 1 -- 172.21.15.12:0/2203554600 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bf4035030 con 0x7f3c04104ba0 2024-09-11T00:26:30.993 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:30.990+0000 7f3c0a400640 1 -- 172.21.15.12:0/2203554600 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3c041a1150 con 0x7f3c04104ba0 2024-09-11T00:26:30.993 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:30.990+0000 7f3c0a400640 1 -- 172.21.15.12:0/2203554600 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3c0410fd70 con 0x7f3c04104ba0 2024-09-11T00:26:30.999 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:30.992+0000 7f3c0d22c740 1 Processor -- start 2024-09-11T00:26:31.003 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:30.997+0000 7f3c0d22c740 1 -- start start 2024-09-11T00:26:31.004 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:30.997+0000 7f3c0d22c740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55da02b01790 0x55da02b03b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:26:31.004 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:30.997+0000 7f3c0d22c740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55da02b040c0 0x55da02b064b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:26:31.005 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:30.997+0000 7f3c0d22c740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55da02b06b00 0x55da02b08fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:26:31.005 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:30.997+0000 7f3c0d22c740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55da02a746b0 con 0x55da02b040c0 2024-09-11T00:26:31.005 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:30.997+0000 7f3c0d22c740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55da02a63ae0 con 0x55da02b01790 2024-09-11T00:26:31.005 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:30.997+0000 7f3c0d22c740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55da02a678f0 con 0x55da02b06b00 2024-09-11T00:26:31.005 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:30.997+0000 7f3bfbe00640 1 -- 172.21.15.12:0/2203554600 <== mon.1 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7f3bf403b460 con 0x7f3c04104ba0 2024-09-11T00:26:31.006 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:31.003+0000 7f3bfbe00640 1 --2- 172.21.15.12:0/2203554600 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f3be80b53a0 0x7f3be80b7860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:26:31.006 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:31.003+0000 7f3c03400640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55da02b040c0 0x55da02b064b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:26:31.006 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:31.003+0000 7f3c03e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55da02b01790 0x55da02b03b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:26:31.006 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:31.003+0000 7f3c03400640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55da02b040c0 0x55da02b064b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:55846/0 (socket says 172.21.15.12:55846) 2024-09-11T00:26:31.006 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:31.003+0000 7f3c03400640 1 -- 172.21.15.12:0/1744286225 learned_addr learned my addr 172.21.15.12:0/1744286225 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:26:31.007 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:31.003+0000 7f3c02a00640 1 --2- 172.21.15.12:0/1744286225 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55da02b06b00 0x55da02b08fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:26:31.007 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:31.003+0000 7f3c03e00640 1 --2- 172.21.15.12:0/2203554600 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f3be80b53a0 0x7f3be80b7860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:26:31.007 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:31.003+0000 7f3c03400640 1 -- 172.21.15.12:0/1744286225 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55da02b06b00 msgr2=0x55da02b08fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:26:31.007 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:31.003+0000 7f3c03400640 1 --2- 172.21.15.12:0/1744286225 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55da02b06b00 0x55da02b08fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:26:31.007 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:31.003+0000 7f3c03400640 1 -- 172.21.15.12:0/1744286225 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55da02b01790 msgr2=0x55da02b03b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:26:31.007 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:31.003+0000 7f3c03400640 1 --2- 172.21.15.12:0/1744286225 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55da02b01790 0x55da02b03b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:26:31.007 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:31.003+0000 7f3c03400640 1 -- 172.21.15.12:0/1744286225 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55da02a30780 con 0x55da02b040c0 2024-09-11T00:26:31.007 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:31.003+0000 7f3bfbe00640 1 -- 172.21.15.12:0/2203554600 <== mon.1 v2:172.21.15.178:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9208+0+0 (secure 0 0 0) 0x7f3bf4106f60 con 0x7f3c04104ba0 2024-09-11T00:26:31.007 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:31.003+0000 7f3c03e00640 1 --2- 172.21.15.12:0/2203554600 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f3be80b53a0 0x7f3be80b7860 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f3bfc009fd0 tx=0x7f3bfc004ab0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:26:31.008 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:31.003+0000 7f3c03400640 1 --2- 172.21.15.12:0/1744286225 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55da02b040c0 0x55da02b064b0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f3bf40cd050 tx=0x7f3bf409a790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7cf4c0615d6cba6 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:26:31.009 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:31.003+0000 7f3bf8c00640 1 -- 172.21.15.12:0/1744286225 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bf4049070 con 0x55da02b040c0 2024-09-11T00:26:31.009 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:31.003+0000 7f3bf8c00640 1 -- 172.21.15.12:0/1744286225 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3bf4091e90 con 0x55da02b040c0 2024-09-11T00:26:31.009 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:31.003+0000 7f3bf8c00640 1 -- 172.21.15.12:0/1744286225 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bf4091c50 con 0x55da02b040c0 2024-09-11T00:26:31.010 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:31.003+0000 7f3c0d22c740 1 -- 172.21.15.12:0/1744286225 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55da02b040c0 msgr2=0x55da02b064b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:26:31.010 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:31.003+0000 7f3c0d22c740 1 --2- 172.21.15.12:0/1744286225 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55da02b040c0 0x55da02b064b0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f3bf40cd050 tx=0x7f3bf409a790 comp rx=0 tx=0).stop 2024-09-11T00:26:31.010 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:31.003+0000 7f3c0d22c740 1 -- 172.21.15.12:0/1744286225 shutdown_connections 2024-09-11T00:26:31.010 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:31.003+0000 7f3c0d22c740 1 --2- 172.21.15.12:0/1744286225 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55da02b06b00 0x55da02b08fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:26:31.011 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:31.003+0000 7f3c0d22c740 1 --2- 172.21.15.12:0/1744286225 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55da02b040c0 0x55da02b064b0 secure :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f3bf40cd050 tx=0x7f3bf409a790 comp rx=0 tx=0).stop 2024-09-11T00:26:31.011 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:31.003+0000 7f3c0d22c740 1 --2- 172.21.15.12:0/1744286225 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55da02b01790 0x55da02b03b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:26:31.011 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:31.003+0000 7f3c0d22c740 1 -- 172.21.15.12:0/1744286225 >> 172.21.15.12:0/1744286225 conn(0x55da02afc8e0 msgr2=0x55da02afd670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:26:31.011 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:31.003+0000 7f3c0d22c740 1 -- 172.21.15.12:0/1744286225 shutdown_connections 2024-09-11T00:26:31.011 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:31.003+0000 7f3c0d22c740 1 -- 172.21.15.12:0/1744286225 wait complete. 2024-09-11T00:26:31.011 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:31.009+0000 7f3c0d22c740 1 -- start start 2024-09-11T00:26:31.012 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:31.009+0000 7f3c0d22c740 1 Processor -- start 2024-09-11T00:26:31.014 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:31.009+0000 7f3c0d22c740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55da02b01790 0x55da02b0af70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:26:31.014 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:31.009+0000 7f3c0d22c740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55da02b06b00 0x55da02b0b610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:26:31.014 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:31.009+0000 7f3c0d22c740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55da02b0c150 0x55da02b0c5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:26:31.014 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:31.009+0000 7f3c0d22c740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55da0299bab0 con 0x55da02b0c150 2024-09-11T00:26:31.015 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:31.009+0000 7f3c0d22c740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55da02a0a570 con 0x55da02b06b00 2024-09-11T00:26:31.015 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:31.009+0000 7f3c0d22c740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55da02b098a0 con 0x55da02b01790 2024-09-11T00:26:31.015 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:31.009+0000 7f3c02a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55da02b06b00 0x55da02b0b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:26:31.015 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:31.009+0000 7f3c03400640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55da02b01790 0x55da02b0af70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:26:31.015 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:31.009+0000 7f3c0d22c740 10 client.0 _ll_register_callbacks cb 0x55da02a19100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:26:31.015 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:31.009+0000 7f3c03400640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55da02b01790 0x55da02b0af70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.12:50098/0 (socket says 172.21.15.12:50098) 2024-09-11T00:26:31.015 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:31.009+0000 7f3c03400640 1 -- 172.21.15.12:0/2475507482 learned_addr learned my addr 172.21.15.12:0/2475507482 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:26:31.015 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:31.009+0000 7f3c03e00640 1 --2- 172.21.15.12:0/2475507482 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55da02b0c150 0x55da02b0c5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:26:31.015 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:31.009+0000 7f3c03400640 1 -- 172.21.15.12:0/2475507482 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55da02b06b00 msgr2=0x55da02b0b610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:26:31.015 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:31.009+0000 7f3c03400640 1 --2- 172.21.15.12:0/2475507482 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55da02b06b00 0x55da02b0b610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:26:31.015 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:31.009+0000 7f3c03400640 1 -- 172.21.15.12:0/2475507482 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55da02b0c150 msgr2=0x55da02b0c5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:26:31.015 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:31.009+0000 7f3c03400640 1 --2- 172.21.15.12:0/2475507482 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55da02b0c150 0x55da02b0c5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:26:31.015 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:31.009+0000 7f3c03400640 1 -- 172.21.15.12:0/2475507482 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55da027c5790 con 0x55da02b01790 2024-09-11T00:26:31.016 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:31.009+0000 7f3c03400640 1 --2- 172.21.15.12:0/2475507482 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55da02b01790 0x55da02b0af70 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f3bf40c9890 tx=0x7f3bf402a440 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8a016cf41f2cb3a2 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:26:31.016 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:31.009+0000 7f3bed600640 10 client.0 ms_handle_connect on v2:172.21.15.178:3300/0 2024-09-11T00:26:31.016 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:31.009+0000 7f3bed600640 1 -- 172.21.15.12:0/2475507482 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bf4049050 con 0x55da02b01790 2024-09-11T00:26:31.016 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:31.009+0000 7f3bed600640 1 -- 172.21.15.12:0/2475507482 <== mon.1 v2:172.21.15.178:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9208+0+0 (secure 0 0 0) 0x7f3bf40d6450 con 0x55da02b01790 2024-09-11T00:26:31.016 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:31.009+0000 7f3c0d22c740 1 -- 172.21.15.12:0/2475507482 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3bf4091e90 con 0x55da02b01790 2024-09-11T00:26:31.017 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:31.014+0000 7f3bed600640 1 -- 172.21.15.12:0/2475507482 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=77) ==== 24+0+0 (secure 0 0 0) 0x7f3bf40cda50 con 0x55da02b01790 2024-09-11T00:26:31.017 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:31.014+0000 7f3c0d22c740 10 client.27607 fetch_fsmap learned FSMap version 77 2024-09-11T00:26:31.017 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:31.014+0000 7f3c0d22c740 1 -- 172.21.15.12:0/2475507482 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,fsmap=77,monmap=4+}) -- 0x55da0283e620 con 0x55da02b01790 2024-09-11T00:26:31.017 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:31.014+0000 7f3bed600640 1 -- 172.21.15.12:0/2475507482 <== mon.1 v2:172.21.15.178:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3bf4091e90 con 0x55da02b01790 2024-09-11T00:26:31.017 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:31.014+0000 7f3bed600640 1 -- 172.21.15.12:0/2475507482 <== mon.1 v2:172.21.15.178:3300/0 5 ==== fsmap(e 77) ==== 8410+0+0 (secure 0 0 0) 0x7f3bf40f4100 con 0x55da02b01790 2024-09-11T00:26:31.017 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:31.014+0000 7f3c0d22c740 10 client.27607 fetch_fsmap finished waiting for FSMap version 77 2024-09-11T00:26:31.017 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:31.014+0000 7f3c0d22c740 10 client.27607 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T00:26:31.017 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:31.014+0000 7f3c0d22c740 20 client.27607 populate_metadata read hostname 'smithi012' 2024-09-11T00:26:31.018 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:31.015+0000 7f3c0d22c740 1 --2- 172.21.15.12:0/2475507482 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55da02b167b0 0x55da02b18ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:26:31.018 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:31.015+0000 7f3c0d22c740 4 client.27607 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T00:26:31.018 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:31.015+0000 7f3c0d22c740 1 -- 172.21.15.12:0/2475507482 --> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55da02a76600 con 0x55da02b167b0 2024-09-11T00:26:31.018 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:31.015+0000 7f3c02a00640 1 --2- 172.21.15.12:0/2475507482 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55da02b167b0 0x55da02b18ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:26:31.018 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:31.015+0000 7f3c02a00640 1 --2- 172.21.15.12:0/2475507482 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55da02b167b0 0x55da02b18ba0 crc :-1 s=READY pgs=273 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f77d96bd4e745b47 server_cookie=c8cb42048d8cc141 in_seq=0 out_seq=0 2024-09-11T00:26:31.018 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:31.015+0000 7f3bed600640 10 client.27607 ms_handle_connect on v2:172.21.15.165:6838/501312984 2024-09-11T00:26:31.019 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:31.015+0000 7f3bed600640 1 -- 172.21.15.12:0/2475507482 <== mds.0 v2:172.21.15.165:6838/501312984 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55da02a76600 con 0x55da02b167b0 2024-09-11T00:26:31.019 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:31.015+0000 7f3bed600640 10 client.27607 handle_command_reply: tid=0 2024-09-11T00:26:31.019 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:31.015+0000 7f3c0d22c740 1 client.27607 shutdown 2024-09-11T00:26:31.020 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:31.015+0000 7f3c0d22c740 1 -- 172.21.15.12:0/2475507482 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55da02b01790 msgr2=0x55da02b0af70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:26:31.020 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:31.015+0000 7f3c0d22c740 1 --2- 172.21.15.12:0/2475507482 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55da02b01790 0x55da02b0af70 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f3bf40c9890 tx=0x7f3bf402a440 comp rx=0 tx=0).stop 2024-09-11T00:26:31.020 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:31.015+0000 7f3c0d22c740 1 -- 172.21.15.12:0/2475507482 shutdown_connections 2024-09-11T00:26:31.020 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:31.015+0000 7f3c0d22c740 1 --2- 172.21.15.12:0/2475507482 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55da02b167b0 0x55da02b18ba0 crc :-1 s=READY pgs=273 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:26:31.020 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:31.015+0000 7f3c0d22c740 1 --2- 172.21.15.12:0/2475507482 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55da02b0c150 0x55da02b0c5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:26:31.020 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:31.015+0000 7f3c0d22c740 1 --2- 172.21.15.12:0/2475507482 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55da02b06b00 0x55da02b0b610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:26:31.020 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:31.015+0000 7f3c0d22c740 1 --2- 172.21.15.12:0/2475507482 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55da02b01790 0x55da02b0af70 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:26:31.020 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:31.015+0000 7f3bed600640 0 client.27607 ms_handle_reset on v2:172.21.15.165:6838/501312984 2024-09-11T00:26:31.020 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:31.015+0000 7f3c0d22c740 1 -- 172.21.15.12:0/2475507482 >> 172.21.15.12:0/2475507482 conn(0x55da02afc8e0 msgr2=0x55da02afba60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:26:31.022 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:31.015+0000 7f3c0d22c740 1 -- 172.21.15.12:0/2475507482 shutdown_connections 2024-09-11T00:26:31.022 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:31.015+0000 7f3c0d22c740 1 -- 172.21.15.12:0/2475507482 wait complete. 2024-09-11T00:26:31.022 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:31.015+0000 7f3c0d22c740 20 client.27607 trim_cache size 0 max 16384 2024-09-11T00:26:31.057 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:31.051+0000 7f3c0d22c740 1 Processor -- start 2024-09-11T00:26:31.057 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:31.051+0000 7f3c0d22c740 1 -- start start 2024-09-11T00:26:31.057 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:31.051+0000 7f3c0d22c740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55da02b01790 0x55da02afd380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:26:31.057 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:31.051+0000 7f3c0d22c740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55da02b06b00 0x55da02b12ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:26:31.058 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:31.051+0000 7f3c0d22c740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55da02b0c150 0x55da02afd8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:26:31.058 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:31.051+0000 7f3c0d22c740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f3c0406eb90 con 0x55da02b01790 2024-09-11T00:26:31.058 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:31.051+0000 7f3c0d22c740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f3c04070880 con 0x55da02b0c150 2024-09-11T00:26:31.058 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:31.051+0000 7f3c0d22c740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f3c0406f320 con 0x55da02b06b00 2024-09-11T00:26:31.058 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:31.051+0000 7f3c03400640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55da02b01790 0x55da02afd380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:26:31.058 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:31.051+0000 7f3c03400640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55da02b01790 0x55da02afd380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:55864/0 (socket says 172.21.15.12:55864) 2024-09-11T00:26:31.058 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:31.051+0000 7f3c02a00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55da02b06b00 0x55da02b12ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:26:31.058 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:31.051+0000 7f3c03400640 1 -- 172.21.15.12:0/2574286555 learned_addr learned my addr 172.21.15.12:0/2574286555 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:26:31.059 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:31.056+0000 7f3c03e00640 1 --2- 172.21.15.12:0/2574286555 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55da02b0c150 0x55da02afd8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:26:31.059 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:31.056+0000 7f3c03400640 1 -- 172.21.15.12:0/2574286555 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55da02b06b00 msgr2=0x55da02b12ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:26:31.059 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:31.056+0000 7f3c03400640 1 --2- 172.21.15.12:0/2574286555 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55da02b06b00 0x55da02b12ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:26:31.059 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:31.056+0000 7f3c03400640 1 -- 172.21.15.12:0/2574286555 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55da02b0c150 msgr2=0x55da02afd8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:26:31.059 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:31.056+0000 7f3c03400640 1 --2- 172.21.15.12:0/2574286555 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55da02b0c150 0x55da02afd8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:26:31.059 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:31.056+0000 7f3c03400640 1 -- 172.21.15.12:0/2574286555 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55da02b0aa70 con 0x55da02b01790 2024-09-11T00:26:31.059 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:31.056+0000 7f3c03400640 1 --2- 172.21.15.12:0/2574286555 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55da02b01790 0x55da02afd380 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f3bf40cb2c0 tx=0x7f3bf40cd120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f65437934ccc59f2 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:26:31.060 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:31.056+0000 7f3befe00640 1 -- 172.21.15.12:0/2574286555 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bf4049040 con 0x55da02b01790 2024-09-11T00:26:31.060 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:31.056+0000 7f3befe00640 1 -- 172.21.15.12:0/2574286555 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3bf40cb500 con 0x55da02b01790 2024-09-11T00:26:31.060 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:31.056+0000 7f3befe00640 1 -- 172.21.15.12:0/2574286555 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bf40f4100 con 0x55da02b01790 2024-09-11T00:26:31.061 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:31.056+0000 7f3c0d22c740 1 -- 172.21.15.12:0/2574286555 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55da02b01790 msgr2=0x55da02afd380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:26:31.061 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:31.056+0000 7f3c0d22c740 1 --2- 172.21.15.12:0/2574286555 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55da02b01790 0x55da02afd380 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f3bf40cb2c0 tx=0x7f3bf40cd120 comp rx=0 tx=0).stop 2024-09-11T00:26:31.061 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:31.056+0000 7f3c0d22c740 1 -- 172.21.15.12:0/2574286555 shutdown_connections 2024-09-11T00:26:31.061 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:31.056+0000 7f3c0d22c740 1 --2- 172.21.15.12:0/2574286555 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55da02b0c150 0x55da02afd8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:26:31.061 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:31.056+0000 7f3c0d22c740 1 --2- 172.21.15.12:0/2574286555 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55da02b06b00 0x55da02b12ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:26:31.061 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:31.056+0000 7f3c0d22c740 1 --2- 172.21.15.12:0/2574286555 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55da02b01790 0x55da02afd380 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:26:31.061 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:31.056+0000 7f3c0d22c740 1 -- 172.21.15.12:0/2574286555 >> 172.21.15.12:0/2574286555 conn(0x55da02afc8e0 msgr2=0x55da02b0a110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:26:31.062 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:31.056+0000 7f3c0d22c740 1 -- 172.21.15.12:0/2574286555 shutdown_connections 2024-09-11T00:26:31.062 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:31.056+0000 7f3c0d22c740 1 -- 172.21.15.12:0/2574286555 wait complete. 2024-09-11T00:26:31.062 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:31.056+0000 7f3c0d22c740 1 -- start start 2024-09-11T00:26:31.062 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:31.056+0000 7f3c0d22c740 1 Processor -- start 2024-09-11T00:26:31.063 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:31.061+0000 7f3c0d22c740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55da02b01790 0x55da02b128f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:26:31.063 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:31.061+0000 7f3c0d22c740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55da02b06b00 0x55da02b12f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:26:31.063 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:31.061+0000 7f3c0d22c740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55da02b0c150 0x55da02b116e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:26:31.063 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:31.061+0000 7f3c0d22c740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f3c040701b0 con 0x55da02b01790 2024-09-11T00:26:31.063 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:31.061+0000 7f3c0d22c740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f3c0406fa20 con 0x55da02b0c150 2024-09-11T00:26:31.063 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:31.061+0000 7f3c0d22c740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f3c0406cb70 con 0x55da02b06b00 2024-09-11T00:26:31.063 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:31.061+0000 7f3c0d22c740 10 client.0 _ll_register_callbacks cb 0x7f3bfc0097d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:26:31.064 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:31.061+0000 7f3c03400640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55da02b01790 0x55da02b128f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:26:31.064 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:31.061+0000 7f3c02a00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55da02b06b00 0x55da02b12f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:26:31.064 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:31.061+0000 7f3c02a00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55da02b06b00 0x55da02b12f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.12:50102/0 (socket says 172.21.15.12:50102) 2024-09-11T00:26:31.064 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:31.061+0000 7f3c02a00640 1 -- 172.21.15.12:0/1553558339 learned_addr learned my addr 172.21.15.12:0/1553558339 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:26:31.064 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:31.062+0000 7f3c03e00640 1 --2- 172.21.15.12:0/1553558339 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55da02b0c150 0x55da02b116e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:26:31.065 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:31.062+0000 7f3c03e00640 1 -- 172.21.15.12:0/1553558339 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55da02b06b00 msgr2=0x55da02b12f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:26:31.065 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:31.062+0000 7f3c03e00640 1 --2- 172.21.15.12:0/1553558339 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55da02b06b00 0x55da02b12f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:26:31.065 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:31.062+0000 7f3c03e00640 1 -- 172.21.15.12:0/1553558339 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55da02b01790 msgr2=0x55da02b128f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:26:31.065 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:31.062+0000 7f3c03e00640 1 --2- 172.21.15.12:0/1553558339 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55da02b01790 0x55da02b128f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:26:31.065 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:31.062+0000 7f3c03e00640 1 -- 172.21.15.12:0/1553558339 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55da02b19220 con 0x55da02b0c150 2024-09-11T00:26:31.065 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:31.062+0000 7f3c02a00640 1 --2- 172.21.15.12:0/1553558339 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55da02b06b00 0x55da02b12f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T00:26:31.065 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:31.062+0000 7f3c03e00640 1 --2- 172.21.15.12:0/1553558339 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55da02b0c150 0x55da02b116e0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f3bfc005c40 tx=0x7f3bfc0094a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=73dc45bdb55e2203 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:26:31.066 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:31.062+0000 7f3bed600640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-09-11T00:26:31.066 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:31.062+0000 7f3bed600640 1 -- 172.21.15.12:0/1553558339 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bfc031ba0 con 0x55da02b0c150 2024-09-11T00:26:31.066 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:31.062+0000 7f3bed600640 1 -- 172.21.15.12:0/1553558339 <== mon.2 v2:172.21.15.165:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9208+0+0 (secure 0 0 0) 0x7f3bfc031d40 con 0x55da02b0c150 2024-09-11T00:26:31.066 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:31.062+0000 7f3c0d22c740 1 -- 172.21.15.12:0/1553558339 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3bf40cb500 con 0x55da02b0c150 2024-09-11T00:26:31.067 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:31.062+0000 7f3bed600640 1 -- 172.21.15.12:0/1553558339 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=77) ==== 24+0+0 (secure 0 0 0) 0x7f3bfc049050 con 0x55da02b0c150 2024-09-11T00:26:31.067 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:31.062+0000 7f3c0d22c740 10 client.27635 fetch_fsmap learned FSMap version 77 2024-09-11T00:26:31.067 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:31.062+0000 7f3c0d22c740 1 -- 172.21.15.12:0/1553558339 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=77,monmap=4+}) -- 0x55da02b10c30 con 0x55da02b0c150 2024-09-11T00:26:31.067 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:31.062+0000 7f3bed600640 1 -- 172.21.15.12:0/1553558339 <== mon.2 v2:172.21.15.165:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3bf40cb500 con 0x55da02b0c150 2024-09-11T00:26:31.067 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:31.062+0000 7f3bed600640 1 -- 172.21.15.12:0/1553558339 <== mon.2 v2:172.21.15.165:3300/0 5 ==== fsmap(e 77) ==== 8410+0+0 (secure 0 0 0) 0x7f3bfc01f070 con 0x55da02b0c150 2024-09-11T00:26:31.068 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:31.062+0000 7f3c0d22c740 10 client.27635 fetch_fsmap finished waiting for FSMap version 77 2024-09-11T00:26:31.068 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:31.062+0000 7f3c0d22c740 10 client.27635 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T00:26:31.068 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:31.062+0000 7f3c0d22c740 20 client.27635 populate_metadata read hostname 'smithi012' 2024-09-11T00:26:31.068 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:31.062+0000 7f3c0d22c740 1 --2- 172.21.15.12:0/1553558339 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55da02b83c60 0x55da02b86080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:26:31.068 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:31.062+0000 7f3c0d22c740 4 client.27635 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T00:26:31.068 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:31.062+0000 7f3c0d22c740 1 -- 172.21.15.12:0/1553558339 --> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55da02a720b0 con 0x55da02b83c60 2024-09-11T00:26:31.068 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:31.062+0000 7f3c03400640 1 --2- 172.21.15.12:0/1553558339 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55da02b83c60 0x55da02b86080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:26:31.068 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:31.062+0000 7f3c03400640 1 --2- 172.21.15.12:0/1553558339 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55da02b83c60 0x55da02b86080 crc :-1 s=READY pgs=274 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=68a956a9684d7d19 server_cookie=d837cc100d909ae6 in_seq=0 out_seq=0 2024-09-11T00:26:31.068 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:31.062+0000 7f3bed600640 10 client.27635 ms_handle_connect on v2:172.21.15.165:6838/501312984 2024-09-11T00:26:31.221 INFO:tasks.workunit.client.0.smithi012.stdout: 10 114723 16.50 MB/sec execute 267 sec latency 2731.556 ms 2024-09-11T00:26:31.254 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:31.251+0000 7f3bed600640 1 -- 172.21.15.12:0/1553558339 <== mds.0 v2:172.21.15.165:6838/501312984 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55da02a720b0 con 0x55da02b83c60 2024-09-11T00:26:31.254 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:31.251+0000 7f3bed600640 10 client.27635 handle_command_reply: tid=0 2024-09-11T00:26:31.254 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:31.251+0000 7f3c0d22c740 1 client.27635 shutdown 2024-09-11T00:26:31.254 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:31.251+0000 7f3c0d22c740 1 -- 172.21.15.12:0/1553558339 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55da02b0c150 msgr2=0x55da02b116e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:26:31.254 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:31.251+0000 7f3c0d22c740 1 --2- 172.21.15.12:0/1553558339 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55da02b0c150 0x55da02b116e0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f3bfc005c40 tx=0x7f3bfc0094a0 comp rx=0 tx=0).stop 2024-09-11T00:26:31.254 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:31.251+0000 7f3c0d22c740 1 -- 172.21.15.12:0/1553558339 shutdown_connections 2024-09-11T00:26:31.254 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:31.251+0000 7f3c0d22c740 1 --2- 172.21.15.12:0/1553558339 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55da02b83c60 0x55da02b86080 crc :-1 s=READY pgs=274 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:26:31.254 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:31.251+0000 7f3c0d22c740 1 --2- 172.21.15.12:0/1553558339 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55da02b0c150 0x55da02b116e0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:26:31.254 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:31.251+0000 7f3c0d22c740 1 --2- 172.21.15.12:0/1553558339 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55da02b06b00 0x55da02b12f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:26:31.254 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:31.251+0000 7f3c0d22c740 1 --2- 172.21.15.12:0/1553558339 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55da02b01790 0x55da02b128f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:26:31.255 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:31.251+0000 7f3bed600640 0 client.27635 ms_handle_reset on v2:172.21.15.165:6838/501312984 2024-09-11T00:26:31.255 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:31.251+0000 7f3c0d22c740 1 -- 172.21.15.12:0/1553558339 >> 172.21.15.12:0/1553558339 conn(0x55da02afc8e0 msgr2=0x55da02afd380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:26:31.255 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:31.251+0000 7f3c0d22c740 1 -- 172.21.15.12:0/1553558339 shutdown_connections 2024-09-11T00:26:31.255 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:31.251+0000 7f3c0d22c740 1 -- 172.21.15.12:0/1553558339 wait complete. 2024-09-11T00:26:31.255 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:31.251+0000 7f3c0d22c740 20 client.27635 trim_cache size 0 max 16384 2024-09-11T00:26:31.255 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:{ 2024-09-11T00:26:31.255 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "message": "", 2024-09-11T00:26:31.255 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "return_code": 0 2024-09-11T00:26:31.255 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:} 2024-09-11T00:26:31.256 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:31.251+0000 7f3c0a400640 1 -- 172.21.15.12:0/2203554600 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f3be80b53a0 msgr2=0x7f3be80b7860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:26:31.256 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:31.251+0000 7f3c0a400640 1 --2- 172.21.15.12:0/2203554600 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f3be80b53a0 0x7f3be80b7860 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f3bfc009fd0 tx=0x7f3bfc004ab0 comp rx=0 tx=0).stop 2024-09-11T00:26:31.256 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:31.251+0000 7f3c0a400640 1 -- 172.21.15.12:0/2203554600 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3c04104ba0 msgr2=0x7f3c0419f380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:26:31.256 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:31.251+0000 7f3c0a400640 1 --2- 172.21.15.12:0/2203554600 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3c04104ba0 0x7f3c0419f380 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f3bf4009850 tx=0x7f3bf4003c00 comp rx=0 tx=0).stop 2024-09-11T00:26:31.257 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:31.251+0000 7f3c0a400640 1 -- 172.21.15.12:0/2203554600 shutdown_connections 2024-09-11T00:26:31.257 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:31.251+0000 7f3c0a400640 1 --2- 172.21.15.12:0/2203554600 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f3be80b53a0 0x7f3be80b7860 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:26:31.257 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:31.251+0000 7f3c0a400640 1 --2- 172.21.15.12:0/2203554600 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3c041066b0 0x7f3c041a0600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:26:31.257 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:31.251+0000 7f3c0a400640 1 --2- 172.21.15.12:0/2203554600 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3c04105cf0 0x7f3c0419f9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:26:31.257 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:31.251+0000 7f3c0a400640 1 --2- 172.21.15.12:0/2203554600 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3c04104ba0 0x7f3c0419f380 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:26:31.257 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:31.251+0000 7f3c0a400640 1 -- 172.21.15.12:0/2203554600 >> 172.21.15.12:0/2203554600 conn(0x7f3c04100460 msgr2=0x7f3c0407ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:26:31.257 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:31.251+0000 7f3c0a400640 1 -- 172.21.15.12:0/2203554600 shutdown_connections 2024-09-11T00:26:31.257 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:31.251+0000 7f3c0a400640 1 -- 172.21.15.12:0/2203554600 wait complete. 2024-09-11T00:26:32.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:26:31 smithi165 ceph-mon[35356]: pgmap v982: 129 pgs: 129 active+clean; 633 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 6.0 MiB/s wr, 694 op/s 2024-09-11T00:26:32.063 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:26:31 smithi012 ceph-mon[30333]: pgmap v982: 129 pgs: 129 active+clean; 633 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 6.0 MiB/s wr, 694 op/s 2024-09-11T00:26:32.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:26:31 smithi178 ceph-mon[35919]: pgmap v982: 129 pgs: 129 active+clean; 633 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 6.0 MiB/s wr, 694 op/s 2024-09-11T00:26:32.221 INFO:tasks.workunit.client.0.smithi012.stdout: 10 114723 16.44 MB/sec execute 268 sec latency 3731.617 ms 2024-09-11T00:26:32.815 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:26:32 smithi012 ceph-mon[30333]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T00:26:33.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:26:32 smithi165 ceph-mon[35356]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T00:26:33.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:26:32 smithi178 ceph-mon[35919]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T00:26:33.233 INFO:tasks.workunit.client.0.smithi012.stdout: 10 114742 16.38 MB/sec execute 269 sec latency 4204.571 ms 2024-09-11T00:26:33.913 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:26:33 smithi012 ceph-mon[30333]: pgmap v983: 129 pgs: 129 active+clean; 630 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 6.2 MiB/s wr, 639 op/s 2024-09-11T00:26:34.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:26:33 smithi178 ceph-mon[35919]: pgmap v983: 129 pgs: 129 active+clean; 630 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 6.2 MiB/s wr, 639 op/s 2024-09-11T00:26:34.233 INFO:tasks.workunit.client.0.smithi012.stdout: 10 114745 16.32 MB/sec execute 270 sec latency 4277.943 ms 2024-09-11T00:26:34.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:26:33 smithi165 ceph-mon[35356]: pgmap v983: 129 pgs: 129 active+clean; 630 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 6.2 MiB/s wr, 639 op/s 2024-09-11T00:26:35.233 INFO:tasks.workunit.client.0.smithi012.stdout: 10 114745 16.26 MB/sec execute 271 sec latency 5278.024 ms 2024-09-11T00:26:35.844 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:26:35 smithi012 ceph-mon[30333]: pgmap v984: 129 pgs: 129 active+clean; 634 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 5.4 MiB/s wr, 454 op/s 2024-09-11T00:26:36.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:26:35 smithi178 ceph-mon[35919]: pgmap v984: 129 pgs: 129 active+clean; 634 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 5.4 MiB/s wr, 454 op/s 2024-09-11T00:26:36.233 INFO:tasks.workunit.client.0.smithi012.stdout: 10 114745 16.20 MB/sec execute 272 sec latency 6278.076 ms 2024-09-11T00:26:36.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:26:35 smithi165 ceph-mon[35356]: pgmap v984: 129 pgs: 129 active+clean; 634 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 5.4 MiB/s wr, 454 op/s 2024-09-11T00:26:37.233 INFO:tasks.workunit.client.0.smithi012.stdout: 10 114754 16.14 MB/sec execute 273 sec latency 7278.125 ms 2024-09-11T00:26:38.063 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:26:37 smithi012 ceph-mon[30333]: pgmap v985: 129 pgs: 129 active+clean; 658 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 10 MiB/s wr, 646 op/s 2024-09-11T00:26:38.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:26:37 smithi178 ceph-mon[35919]: pgmap v985: 129 pgs: 129 active+clean; 658 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 10 MiB/s wr, 646 op/s 2024-09-11T00:26:38.233 INFO:tasks.workunit.client.0.smithi012.stdout: 10 114754 16.08 MB/sec execute 274 sec latency 8278.172 ms 2024-09-11T00:26:38.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:26:37 smithi165 ceph-mon[35356]: pgmap v985: 129 pgs: 129 active+clean; 658 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 10 MiB/s wr, 646 op/s 2024-09-11T00:26:39.233 INFO:tasks.workunit.client.0.smithi012.stdout: 10 114754 16.02 MB/sec execute 275 sec latency 9278.219 ms 2024-09-11T00:26:40.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:26:39 smithi178 ceph-mon[35919]: pgmap v986: 129 pgs: 129 active+clean; 659 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 8.9 MiB/s wr, 427 op/s 2024-09-11T00:26:40.233 INFO:tasks.workunit.client.0.smithi012.stdout: 10 114754 15.96 MB/sec execute 276 sec latency 10278.271 ms 2024-09-11T00:26:40.283 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:26:39 smithi165 ceph-mon[35356]: pgmap v986: 129 pgs: 129 active+clean; 659 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 8.9 MiB/s wr, 427 op/s 2024-09-11T00:26:40.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:26:39 smithi012 ceph-mon[30333]: pgmap v986: 129 pgs: 129 active+clean; 659 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 8.9 MiB/s wr, 427 op/s 2024-09-11T00:26:41.233 INFO:tasks.workunit.client.0.smithi012.stdout: 10 114754 15.91 MB/sec execute 277 sec latency 11278.321 ms 2024-09-11T00:26:41.239 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:26:40 smithi012 ceph-mon[30333]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T00:26:41.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:26:40 smithi165 ceph-mon[35356]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T00:26:41.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:26:40 smithi178 ceph-mon[35919]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T00:26:42.106 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:26:41 smithi012 ceph-mon[30333]: pgmap v987: 129 pgs: 129 active+clean; 636 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 8.8 MiB/s wr, 449 op/s 2024-09-11T00:26:42.233 INFO:tasks.workunit.client.0.smithi012.stdout: 10 114755 15.85 MB/sec execute 278 sec latency 12278.372 ms 2024-09-11T00:26:42.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:26:41 smithi165 ceph-mon[35356]: pgmap v987: 129 pgs: 129 active+clean; 636 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 8.8 MiB/s wr, 449 op/s 2024-09-11T00:26:42.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:26:41 smithi178 ceph-mon[35919]: pgmap v987: 129 pgs: 129 active+clean; 636 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 8.8 MiB/s wr, 449 op/s 2024-09-11T00:26:43.005 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:26:42 smithi012 ceph-mon[30333]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T00:26:43.233 INFO:tasks.workunit.client.0.smithi012.stdout: 10 114756 15.79 MB/sec execute 279 sec latency 13278.423 ms 2024-09-11T00:26:43.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:26:42 smithi165 ceph-mon[35356]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T00:26:43.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:26:42 smithi178 ceph-mon[35919]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T00:26:44.116 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:26:43 smithi012 ceph-mon[30333]: pgmap v988: 129 pgs: 129 active+clean; 636 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 882 KiB/s rd, 6.6 MiB/s wr, 327 op/s 2024-09-11T00:26:44.233 INFO:tasks.workunit.client.0.smithi012.stdout: 10 114759 15.74 MB/sec execute 280 sec latency 14172.870 ms 2024-09-11T00:26:44.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:26:43 smithi165 ceph-mon[35356]: pgmap v988: 129 pgs: 129 active+clean; 636 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 882 KiB/s rd, 6.6 MiB/s wr, 327 op/s 2024-09-11T00:26:44.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:26:43 smithi178 ceph-mon[35919]: pgmap v988: 129 pgs: 129 active+clean; 636 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 882 KiB/s rd, 6.6 MiB/s wr, 327 op/s 2024-09-11T00:26:44.370 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:44.367+0000 7fbc75800640 1 -- 172.21.15.12:0/3252497618 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbc701066b0 msgr2=0x7fbc7010d030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:26:44.370 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:44.367+0000 7fbc75800640 1 --2- 172.21.15.12:0/3252497618 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbc701066b0 0x7fbc7010d030 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7fbc68009cd0 tx=0x7fbc6802a910 comp rx=0 tx=0).stop 2024-09-11T00:26:44.370 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:44.368+0000 7fbc75800640 1 -- 172.21.15.12:0/3252497618 shutdown_connections 2024-09-11T00:26:44.370 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:44.368+0000 7fbc75800640 1 --2- 172.21.15.12:0/3252497618 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbc701066b0 0x7fbc7010d030 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:26:44.370 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:44.368+0000 7fbc75800640 1 --2- 172.21.15.12:0/3252497618 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbc70105cf0 0x7fbc70106170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:26:44.370 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:44.368+0000 7fbc75800640 1 --2- 172.21.15.12:0/3252497618 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbc70104ba0 0x7fbc70104fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:26:44.370 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:44.368+0000 7fbc75800640 1 -- 172.21.15.12:0/3252497618 >> 172.21.15.12:0/3252497618 conn(0x7fbc70100460 msgr2=0x7fbc70102880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:26:44.370 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:44.368+0000 7fbc75800640 1 -- 172.21.15.12:0/3252497618 shutdown_connections 2024-09-11T00:26:44.370 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:44.368+0000 7fbc75800640 1 -- 172.21.15.12:0/3252497618 wait complete. 2024-09-11T00:26:44.373 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:44.368+0000 7fbc75800640 1 Processor -- start 2024-09-11T00:26:44.373 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:44.369+0000 7fbc75800640 1 -- start start 2024-09-11T00:26:44.373 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:44.369+0000 7fbc75800640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbc70104ba0 0x7fbc7019af90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:26:44.373 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:44.369+0000 7fbc75800640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbc70105cf0 0x7fbc7019b5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:26:44.373 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:44.369+0000 7fbc75800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbc701066b0 0x7fbc7019c210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:26:44.373 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:44.369+0000 7fbc75800640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fbc70110890 con 0x7fbc70104ba0 2024-09-11T00:26:44.373 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:44.369+0000 7fbc75800640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fbc70110710 con 0x7fbc701066b0 2024-09-11T00:26:44.373 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:44.369+0000 7fbc75800640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fbc70110a10 con 0x7fbc70105cf0 2024-09-11T00:26:44.373 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:44.369+0000 7fbc6ea00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbc70104ba0 0x7fbc7019af90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:26:44.373 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:44.370+0000 7fbc6ea00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbc70104ba0 0x7fbc7019af90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:48142/0 (socket says 172.21.15.12:48142) 2024-09-11T00:26:44.373 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:44.370+0000 7fbc6ea00640 1 -- 172.21.15.12:0/592296523 learned_addr learned my addr 172.21.15.12:0/592296523 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:26:44.374 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:44.370+0000 7fbc6e000640 1 --2- 172.21.15.12:0/592296523 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbc70105cf0 0x7fbc7019b5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:26:44.374 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:44.370+0000 7fbc6f400640 1 --2- 172.21.15.12:0/592296523 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbc701066b0 0x7fbc7019c210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:26:44.374 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:44.370+0000 7fbc6ea00640 1 -- 172.21.15.12:0/592296523 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbc70105cf0 msgr2=0x7fbc7019b5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:26:44.374 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:44.370+0000 7fbc6ea00640 1 --2- 172.21.15.12:0/592296523 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbc70105cf0 0x7fbc7019b5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:26:44.374 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:44.370+0000 7fbc6ea00640 1 -- 172.21.15.12:0/592296523 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbc701066b0 msgr2=0x7fbc7019c210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:26:44.374 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:44.370+0000 7fbc6ea00640 1 --2- 172.21.15.12:0/592296523 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbc701066b0 0x7fbc7019c210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:26:44.374 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:44.370+0000 7fbc6ea00640 1 -- 172.21.15.12:0/592296523 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc7019cad0 con 0x7fbc70104ba0 2024-09-11T00:26:44.374 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:44.370+0000 7fbc6ea00640 1 --2- 172.21.15.12:0/592296523 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbc70104ba0 0x7fbc7019af90 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7fbc6000b450 tx=0x7fbc6000b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:26:44.374 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:44.370+0000 7fbc67400640 1 -- 172.21.15.12:0/592296523 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc60015020 con 0x7fbc70104ba0 2024-09-11T00:26:44.374 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:44.370+0000 7fbc67400640 1 -- 172.21.15.12:0/592296523 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbc60011d40 con 0x7fbc70104ba0 2024-09-11T00:26:44.374 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:44.370+0000 7fbc75800640 1 -- 172.21.15.12:0/592296523 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbc7019cdc0 con 0x7fbc70104ba0 2024-09-11T00:26:44.374 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:44.370+0000 7fbc67400640 1 -- 172.21.15.12:0/592296523 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc6000f340 con 0x7fbc70104ba0 2024-09-11T00:26:44.374 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:44.370+0000 7fbc75800640 1 -- 172.21.15.12:0/592296523 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbc701a3490 con 0x7fbc70104ba0 2024-09-11T00:26:44.376 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:44.374+0000 7fbc78322740 1 Processor -- start 2024-09-11T00:26:44.376 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:44.374+0000 7fbc78322740 1 -- start start 2024-09-11T00:26:44.378 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:44.374+0000 7fbc78322740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557b4462a790 0x557b4462cb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:26:44.378 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:44.374+0000 7fbc78322740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557b4462d0c0 0x557b4462f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:26:44.378 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:44.374+0000 7fbc78322740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557b4462fb00 0x557b44631fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:26:44.378 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:44.374+0000 7fbc78322740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x557b4459d6b0 con 0x557b4462a790 2024-09-11T00:26:44.378 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:44.374+0000 7fbc78322740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x557b4458cae0 con 0x557b4462fb00 2024-09-11T00:26:44.378 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:44.374+0000 7fbc78322740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x557b445908f0 con 0x557b4462d0c0 2024-09-11T00:26:44.378 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:44.374+0000 7fbc6f400640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557b4462a790 0x557b4462cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:26:44.378 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:44.374+0000 7fbc6f400640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557b4462a790 0x557b4462cb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:48146/0 (socket says 172.21.15.12:48146) 2024-09-11T00:26:44.378 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:44.374+0000 7fbc6f400640 1 -- 172.21.15.12:0/1728870219 learned_addr learned my addr 172.21.15.12:0/1728870219 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:26:44.378 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:44.374+0000 7fbc67400640 1 -- 172.21.15.12:0/592296523 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7fbc60011890 con 0x7fbc70104ba0 2024-09-11T00:26:44.379 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:44.376+0000 7fbc6e000640 1 --2- 172.21.15.12:0/1728870219 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557b4462fb00 0x557b44631fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:26:44.379 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:44.376+0000 7fbc6f400640 1 -- 172.21.15.12:0/1728870219 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557b4462d0c0 msgr2=0x557b4462f4b0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-11T00:26:44.379 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:44.376+0000 7fbc6f400640 1 --2- 172.21.15.12:0/1728870219 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557b4462d0c0 0x557b4462f4b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:26:44.379 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:44.376+0000 7fbc6f400640 1 -- 172.21.15.12:0/1728870219 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557b4462fb00 msgr2=0x557b44631fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:26:44.379 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:44.376+0000 7fbc6f400640 1 --2- 172.21.15.12:0/1728870219 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557b4462fb00 0x557b44631fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:26:44.379 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:44.376+0000 7fbc6f400640 1 -- 172.21.15.12:0/1728870219 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557b44559780 con 0x557b4462a790 2024-09-11T00:26:44.379 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:44.376+0000 7fbc6f400640 1 --2- 172.21.15.12:0/1728870219 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557b4462a790 0x557b4462cb80 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7fbc68031a30 tx=0x7fbc6802f440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=503ca0acb7c01043 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:26:44.381 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:44.376+0000 7fbc5be00640 1 -- 172.21.15.12:0/1728870219 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc68031660 con 0x557b4462a790 2024-09-11T00:26:44.381 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:44.376+0000 7fbc5be00640 1 -- 172.21.15.12:0/1728870219 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbc68036030 con 0x557b4462a790 2024-09-11T00:26:44.382 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:44.376+0000 7fbc5be00640 1 -- 172.21.15.12:0/1728870219 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc680045a0 con 0x557b4462a790 2024-09-11T00:26:44.382 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:44.376+0000 7fbc78322740 1 -- 172.21.15.12:0/1728870219 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557b4462a790 msgr2=0x557b4462cb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:26:44.382 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:44.376+0000 7fbc78322740 1 --2- 172.21.15.12:0/1728870219 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557b4462a790 0x557b4462cb80 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7fbc68031a30 tx=0x7fbc6802f440 comp rx=0 tx=0).stop 2024-09-11T00:26:44.382 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:44.376+0000 7fbc78322740 1 -- 172.21.15.12:0/1728870219 shutdown_connections 2024-09-11T00:26:44.382 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:44.376+0000 7fbc78322740 1 --2- 172.21.15.12:0/1728870219 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557b4462fb00 0x557b44631fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:26:44.382 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:44.376+0000 7fbc78322740 1 --2- 172.21.15.12:0/1728870219 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557b4462d0c0 0x557b4462f4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:26:44.382 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:44.376+0000 7fbc78322740 1 --2- 172.21.15.12:0/1728870219 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557b4462a790 0x557b4462cb80 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:26:44.382 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:44.376+0000 7fbc78322740 1 -- 172.21.15.12:0/1728870219 >> 172.21.15.12:0/1728870219 conn(0x557b446258e0 msgr2=0x557b44626670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:26:44.382 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:44.376+0000 7fbc78322740 1 -- 172.21.15.12:0/1728870219 shutdown_connections 2024-09-11T00:26:44.382 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:44.376+0000 7fbc78322740 1 -- 172.21.15.12:0/1728870219 wait complete. 2024-09-11T00:26:44.382 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:44.376+0000 7fbc67400640 1 --2- 172.21.15.12:0/592296523 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fbc540b5270 0x7fbc540b7730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:26:44.382 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:44.376+0000 7fbc67400640 1 -- 172.21.15.12:0/592296523 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9208+0+0 (secure 0 0 0) 0x7fbc600e5bc0 con 0x7fbc70104ba0 2024-09-11T00:26:44.383 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:44.376+0000 7fbc6f400640 1 --2- 172.21.15.12:0/592296523 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fbc540b5270 0x7fbc540b7730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:26:44.383 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:44.376+0000 7fbc78322740 1 -- start start 2024-09-11T00:26:44.383 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:44.376+0000 7fbc78322740 1 Processor -- start 2024-09-11T00:26:44.383 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:44.376+0000 7fbc78322740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557b4462a790 0x557b44633e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:26:44.383 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:44.376+0000 7fbc6f400640 1 --2- 172.21.15.12:0/592296523 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fbc540b5270 0x7fbc540b7730 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fbc68009430 tx=0x7fbc68031000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:26:44.383 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:44.376+0000 7fbc78322740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557b4462d0c0 0x557b44634530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:26:44.383 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:44.376+0000 7fbc78322740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557b4462fb00 0x557b44635100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:26:44.383 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:44.376+0000 7fbc78322740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x557b444c4ab0 con 0x557b4462d0c0 2024-09-11T00:26:44.383 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:44.376+0000 7fbc78322740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x557b44533570 con 0x557b4462a790 2024-09-11T00:26:44.383 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:44.376+0000 7fbc78322740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x557b44632a80 con 0x557b4462fb00 2024-09-11T00:26:44.383 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:44.376+0000 7fbc78322740 10 client.0 _ll_register_callbacks cb 0x557b44542100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:26:44.383 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:44.376+0000 7fbc6e000640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557b4462d0c0 0x557b44634530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:26:44.383 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:44.376+0000 7fbc6e000640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557b4462d0c0 0x557b44634530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:48158/0 (socket says 172.21.15.12:48158) 2024-09-11T00:26:44.384 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:44.376+0000 7fbc6e000640 1 -- 172.21.15.12:0/3423253634 learned_addr learned my addr 172.21.15.12:0/3423253634 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:26:44.384 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:44.376+0000 7fbc6ea00640 1 --2- 172.21.15.12:0/3423253634 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557b4462a790 0x557b44633e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:26:44.384 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:44.376+0000 7fbc6e000640 1 -- 172.21.15.12:0/3423253634 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557b4462fb00 msgr2=0x557b44635100 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-11T00:26:44.384 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:44.376+0000 7fbc6e000640 1 --2- 172.21.15.12:0/3423253634 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557b4462fb00 0x557b44635100 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:26:44.384 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:44.376+0000 7fbc6e000640 1 -- 172.21.15.12:0/3423253634 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557b4462a790 msgr2=0x557b44633e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:26:44.384 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:44.376+0000 7fbc6e000640 1 --2- 172.21.15.12:0/3423253634 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557b4462a790 0x557b44633e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:26:44.384 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:44.376+0000 7fbc6e000640 1 -- 172.21.15.12:0/3423253634 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557b442ee790 con 0x557b4462d0c0 2024-09-11T00:26:44.384 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:44.376+0000 7fbc6ea00640 1 --2- 172.21.15.12:0/3423253634 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557b4462a790 0x557b44633e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T00:26:44.384 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:44.376+0000 7fbc6e000640 1 --2- 172.21.15.12:0/3423253634 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557b4462d0c0 0x557b44634530 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7fbc5c011440 tx=0x7fbc5c011900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75772a52ef1c75ba server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:26:44.384 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:44.376+0000 7fbc58c00640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-09-11T00:26:44.385 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:44.376+0000 7fbc58c00640 1 -- 172.21.15.12:0/3423253634 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc5c005020 con 0x557b4462d0c0 2024-09-11T00:26:44.386 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:44.376+0000 7fbc58c00640 1 -- 172.21.15.12:0/3423253634 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9208+0+0 (secure 0 0 0) 0x7fbc5c0051c0 con 0x557b4462d0c0 2024-09-11T00:26:44.386 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:44.376+0000 7fbc78322740 1 -- 172.21.15.12:0/3423253634 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbc68036030 con 0x557b4462d0c0 2024-09-11T00:26:44.386 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:44.376+0000 7fbc58c00640 1 -- 172.21.15.12:0/3423253634 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=80) ==== 24+0+0 (secure 0 0 0) 0x7fbc5c025050 con 0x557b4462d0c0 2024-09-11T00:26:44.386 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:44.376+0000 7fbc78322740 10 client.17631 fetch_fsmap learned FSMap version 80 2024-09-11T00:26:44.386 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:44.376+0000 7fbc78322740 1 -- 172.21.15.12:0/3423253634 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=80,monmap=4+}) -- 0x557b44367620 con 0x557b4462d0c0 2024-09-11T00:26:44.386 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:44.376+0000 7fbc58c00640 1 -- 172.21.15.12:0/3423253634 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbc68036030 con 0x557b4462d0c0 2024-09-11T00:26:44.386 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:44.376+0000 7fbc58c00640 1 -- 172.21.15.12:0/3423253634 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 80) ==== 8414+0+0 (secure 0 0 0) 0x7fbc5c0107e0 con 0x557b4462d0c0 2024-09-11T00:26:44.386 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:44.376+0000 7fbc78322740 10 client.17631 fetch_fsmap finished waiting for FSMap version 80 2024-09-11T00:26:44.386 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:44.376+0000 7fbc78322740 10 client.17631 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T00:26:44.386 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:44.381+0000 7fbc78322740 20 client.17631 populate_metadata read hostname 'smithi012' 2024-09-11T00:26:44.386 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:44.381+0000 7fbc78322740 1 --2- 172.21.15.12:0/3423253634 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x557b4463f890 0x557b44641cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:26:44.387 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:44.381+0000 7fbc78322740 4 client.17631 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T00:26:44.387 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:44.381+0000 7fbc78322740 1 -- 172.21.15.12:0/3423253634 --> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557b4459f600 con 0x557b4463f890 2024-09-11T00:26:44.401 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:44.400+0000 7fbc6ea00640 1 --2- 172.21.15.12:0/3423253634 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x557b4463f890 0x557b44641cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:26:44.460 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:44.459+0000 7fbc6ea00640 1 --2- 172.21.15.12:0/3423253634 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x557b4463f890 0x557b44641cf0 crc :-1 s=READY pgs=275 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=10c59b7b77af770e server_cookie=66f456d4cb120c2 in_seq=0 out_seq=0 2024-09-11T00:26:44.476 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:44.459+0000 7fbc58c00640 10 client.17631 ms_handle_connect on v2:172.21.15.165:6838/501312984 2024-09-11T00:26:44.477 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:44.471+0000 7fbc58c00640 1 -- 172.21.15.12:0/3423253634 <== mds.0 v2:172.21.15.165:6838/501312984 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x557b4459f600 con 0x557b4463f890 2024-09-11T00:26:44.477 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:44.471+0000 7fbc58c00640 10 client.17631 handle_command_reply: tid=0 2024-09-11T00:26:44.478 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:44.471+0000 7fbc78322740 1 client.17631 shutdown 2024-09-11T00:26:44.478 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:44.471+0000 7fbc78322740 1 -- 172.21.15.12:0/3423253634 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557b4462d0c0 msgr2=0x557b44634530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:26:44.478 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:44.471+0000 7fbc78322740 1 --2- 172.21.15.12:0/3423253634 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557b4462d0c0 0x557b44634530 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7fbc5c011440 tx=0x7fbc5c011900 comp rx=0 tx=0).stop 2024-09-11T00:26:44.478 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:44.471+0000 7fbc78322740 1 -- 172.21.15.12:0/3423253634 shutdown_connections 2024-09-11T00:26:44.478 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:44.471+0000 7fbc78322740 1 --2- 172.21.15.12:0/3423253634 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x557b4463f890 0x557b44641cf0 crc :-1 s=READY pgs=275 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:26:44.478 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:44.471+0000 7fbc78322740 1 --2- 172.21.15.12:0/3423253634 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557b4462fb00 0x557b44635100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:26:44.478 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:44.471+0000 7fbc78322740 1 --2- 172.21.15.12:0/3423253634 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557b4462d0c0 0x557b44634530 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:26:44.478 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:44.471+0000 7fbc78322740 1 --2- 172.21.15.12:0/3423253634 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557b4462a790 0x557b44633e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:26:44.478 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:44.471+0000 7fbc58c00640 0 client.17631 ms_handle_reset on v2:172.21.15.165:6838/501312984 2024-09-11T00:26:44.478 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:44.471+0000 7fbc78322740 1 -- 172.21.15.12:0/3423253634 >> 172.21.15.12:0/3423253634 conn(0x557b446258e0 msgr2=0x557b44624a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:26:44.478 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:44.471+0000 7fbc78322740 1 -- 172.21.15.12:0/3423253634 shutdown_connections 2024-09-11T00:26:44.478 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:44.471+0000 7fbc78322740 1 -- 172.21.15.12:0/3423253634 wait complete. 2024-09-11T00:26:44.479 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:44.471+0000 7fbc78322740 20 client.17631 trim_cache size 0 max 16384 2024-09-11T00:26:44.489 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:44.486+0000 7fbc78322740 1 Processor -- start 2024-09-11T00:26:44.490 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:44.486+0000 7fbc78322740 1 -- start start 2024-09-11T00:26:44.490 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:44.486+0000 7fbc78322740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557b4462a790 0x557b44633e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:26:44.490 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:44.486+0000 7fbc78322740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557b4462d0c0 0x557b446343d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:26:44.490 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:44.486+0000 7fbc78322740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557b4462fb00 0x557b4462fee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:26:44.490 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:44.486+0000 7fbc78322740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fbc701ad870 con 0x557b4462a790 2024-09-11T00:26:44.490 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:44.486+0000 7fbc78322740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fbc701af530 con 0x557b4462fb00 2024-09-11T00:26:44.490 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:44.486+0000 7fbc78322740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fbc701ae000 con 0x557b4462d0c0 2024-09-11T00:26:44.490 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:44.486+0000 7fbc6ea00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557b4462a790 0x557b44633e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:26:44.490 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:44.486+0000 7fbc6ea00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557b4462a790 0x557b44633e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:48172/0 (socket says 172.21.15.12:48172) 2024-09-11T00:26:44.490 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:44.486+0000 7fbc6ea00640 1 -- 172.21.15.12:0/3784359698 learned_addr learned my addr 172.21.15.12:0/3784359698 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:26:44.490 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:44.486+0000 7fbc6e000640 1 --2- 172.21.15.12:0/3784359698 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557b4462d0c0 0x557b446343d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:26:44.491 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:44.486+0000 7fbc6f400640 1 --2- 172.21.15.12:0/3784359698 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557b4462fb00 0x557b4462fee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:26:44.491 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:44.486+0000 7fbc6ea00640 1 -- 172.21.15.12:0/3784359698 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557b4462d0c0 msgr2=0x557b446343d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:26:44.491 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:44.486+0000 7fbc6ea00640 1 --2- 172.21.15.12:0/3784359698 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557b4462d0c0 0x557b446343d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:26:44.491 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:44.486+0000 7fbc6ea00640 1 -- 172.21.15.12:0/3784359698 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557b4462fb00 msgr2=0x557b4462fee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:26:44.491 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:44.486+0000 7fbc6ea00640 1 --2- 172.21.15.12:0/3784359698 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557b4462fb00 0x557b4462fee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:26:44.491 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:44.486+0000 7fbc6ea00640 1 -- 172.21.15.12:0/3784359698 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557b4462f820 con 0x557b4462a790 2024-09-11T00:26:44.491 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:44.486+0000 7fbc6ea00640 1 --2- 172.21.15.12:0/3784359698 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557b4462a790 0x557b44633e90 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7fbc600cf150 tx=0x7fbc600108d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=195d30d084eb2970 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:26:44.491 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:44.486+0000 7fbc5b400640 1 -- 172.21.15.12:0/3784359698 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc600ac650 con 0x557b4462a790 2024-09-11T00:26:44.491 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:44.486+0000 7fbc5b400640 1 -- 172.21.15.12:0/3784359698 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbc600abf50 con 0x557b4462a790 2024-09-11T00:26:44.491 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:44.486+0000 7fbc5b400640 1 -- 172.21.15.12:0/3784359698 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc600930b0 con 0x557b4462a790 2024-09-11T00:26:44.495 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:44.486+0000 7fbc78322740 1 -- 172.21.15.12:0/3784359698 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557b4462a790 msgr2=0x557b44633e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:26:44.495 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:44.486+0000 7fbc78322740 1 --2- 172.21.15.12:0/3784359698 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557b4462a790 0x557b44633e90 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7fbc600cf150 tx=0x7fbc600108d0 comp rx=0 tx=0).stop 2024-09-11T00:26:44.495 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:44.486+0000 7fbc78322740 1 -- 172.21.15.12:0/3784359698 shutdown_connections 2024-09-11T00:26:44.495 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:44.486+0000 7fbc78322740 1 --2- 172.21.15.12:0/3784359698 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557b4462fb00 0x557b4462fee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:26:44.495 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:44.486+0000 7fbc78322740 1 --2- 172.21.15.12:0/3784359698 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557b4462d0c0 0x557b446343d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:26:44.495 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:44.486+0000 7fbc78322740 1 --2- 172.21.15.12:0/3784359698 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557b4462a790 0x557b44633e90 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:26:44.495 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:44.486+0000 7fbc78322740 1 -- 172.21.15.12:0/3784359698 >> 172.21.15.12:0/3784359698 conn(0x557b446258e0 msgr2=0x557b44626140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:26:44.495 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:44.486+0000 7fbc78322740 1 -- 172.21.15.12:0/3784359698 shutdown_connections 2024-09-11T00:26:44.495 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:44.486+0000 7fbc78322740 1 -- 172.21.15.12:0/3784359698 wait complete. 2024-09-11T00:26:44.495 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:44.486+0000 7fbc78322740 1 -- start start 2024-09-11T00:26:44.495 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:44.486+0000 7fbc78322740 1 Processor -- start 2024-09-11T00:26:44.496 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:44.490+0000 7fbc78322740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557b4462a790 0x557b4463bb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:26:44.496 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:44.490+0000 7fbc78322740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557b4462d0c0 0x557b4463a910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:26:44.496 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:44.490+0000 7fbc78322740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557b4462fb00 0x557b4463b000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:26:44.496 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:44.490+0000 7fbc78322740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fbc701aee60 con 0x557b4462d0c0 2024-09-11T00:26:44.496 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:44.490+0000 7fbc78322740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fbc701ae6d0 con 0x557b4462a790 2024-09-11T00:26:44.496 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:44.490+0000 7fbc78322740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fbc701ab850 con 0x557b4462fb00 2024-09-11T00:26:44.496 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:44.490+0000 7fbc78322740 10 client.0 _ll_register_callbacks cb 0x7fbc5c00bc10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:26:44.496 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:44.490+0000 7fbc6e000640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557b4462d0c0 0x557b4463a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:26:44.496 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:44.490+0000 7fbc6ea00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557b4462a790 0x557b4463bb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:26:44.496 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:44.490+0000 7fbc6ea00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557b4462a790 0x557b4463bb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.12:45722/0 (socket says 172.21.15.12:45722) 2024-09-11T00:26:44.496 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:44.490+0000 7fbc6ea00640 1 -- 172.21.15.12:0/3320744308 learned_addr learned my addr 172.21.15.12:0/3320744308 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:26:44.496 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:44.490+0000 7fbc6f400640 1 --2- 172.21.15.12:0/3320744308 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557b4462fb00 0x557b4463b000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:26:44.497 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:44.490+0000 7fbc6ea00640 1 -- 172.21.15.12:0/3320744308 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557b4462fb00 msgr2=0x557b4463b000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:26:44.497 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:44.490+0000 7fbc6ea00640 1 --2- 172.21.15.12:0/3320744308 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557b4462fb00 0x557b4463b000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:26:44.497 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:44.490+0000 7fbc6ea00640 1 -- 172.21.15.12:0/3320744308 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557b4462d0c0 msgr2=0x557b4463a910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:26:44.497 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:44.490+0000 7fbc6ea00640 1 --2- 172.21.15.12:0/3320744308 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557b4462d0c0 0x557b4463a910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:26:44.497 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:44.490+0000 7fbc6ea00640 1 -- 172.21.15.12:0/3320744308 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557b446393b0 con 0x557b4462a790 2024-09-11T00:26:44.497 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:44.490+0000 7fbc6e000640 1 --2- 172.21.15.12:0/3320744308 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557b4462d0c0 0x557b4463a910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:26:44.497 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:44.490+0000 7fbc6ea00640 1 --2- 172.21.15.12:0/3320744308 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557b4462a790 0x557b4463bb90 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fbc600abc00 tx=0x7fbc600106e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f2d3dba71aa7be7d server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:26:44.497 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:44.490+0000 7fbc58c00640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-09-11T00:26:44.497 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:44.490+0000 7fbc58c00640 1 -- 172.21.15.12:0/3320744308 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc600b0a30 con 0x557b4462a790 2024-09-11T00:26:44.497 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:44.490+0000 7fbc58c00640 1 -- 172.21.15.12:0/3320744308 <== mon.2 v2:172.21.15.165:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9208+0+0 (secure 0 0 0) 0x7fbc600ab1b0 con 0x557b4462a790 2024-09-11T00:26:44.497 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:44.490+0000 7fbc78322740 1 -- 172.21.15.12:0/3320744308 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbc600abf50 con 0x557b4462a790 2024-09-11T00:26:44.497 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:44.490+0000 7fbc58c00640 1 -- 172.21.15.12:0/3320744308 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=80) ==== 24+0+0 (secure 0 0 0) 0x7fbc600ac650 con 0x557b4462a790 2024-09-11T00:26:44.497 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:44.490+0000 7fbc78322740 10 client.27653 fetch_fsmap learned FSMap version 80 2024-09-11T00:26:44.498 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:44.490+0000 7fbc78322740 1 -- 172.21.15.12:0/3320744308 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=80,monmap=4+}) -- 0x557b446aca50 con 0x557b4462a790 2024-09-11T00:26:44.498 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:44.490+0000 7fbc58c00640 1 -- 172.21.15.12:0/3320744308 <== mon.2 v2:172.21.15.165:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbc600abf50 con 0x557b4462a790 2024-09-11T00:26:44.498 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:44.490+0000 7fbc58c00640 1 -- 172.21.15.12:0/3320744308 <== mon.2 v2:172.21.15.165:3300/0 5 ==== fsmap(e 80) ==== 8414+0+0 (secure 0 0 0) 0x7fbc600c1ad0 con 0x557b4462a790 2024-09-11T00:26:44.498 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:44.490+0000 7fbc78322740 10 client.27653 fetch_fsmap finished waiting for FSMap version 80 2024-09-11T00:26:44.498 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:44.490+0000 7fbc78322740 10 client.27653 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T00:26:44.498 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:44.490+0000 7fbc78322740 20 client.27653 populate_metadata read hostname 'smithi012' 2024-09-11T00:26:44.498 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:44.490+0000 7fbc78322740 1 --2- 172.21.15.12:0/3320744308 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x557b446ad540 0x557b446af960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:26:44.498 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:44.490+0000 7fbc78322740 4 client.27653 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T00:26:44.498 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:44.490+0000 7fbc78322740 1 -- 172.21.15.12:0/3320744308 --> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] -- command(tid 0: {"prefix": "flush journal"}) -- 0x557b4459b0b0 con 0x557b446ad540 2024-09-11T00:26:44.498 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:44.490+0000 7fbc6e000640 1 --2- 172.21.15.12:0/3320744308 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x557b446ad540 0x557b446af960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:26:44.498 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:44.490+0000 7fbc6e000640 1 --2- 172.21.15.12:0/3320744308 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x557b446ad540 0x557b446af960 crc :-1 s=READY pgs=276 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b0df3d65dcf5b41a server_cookie=ed1a9ba22758a82c in_seq=0 out_seq=0 2024-09-11T00:26:44.498 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:44.490+0000 7fbc58c00640 10 client.27653 ms_handle_connect on v2:172.21.15.165:6838/501312984 2024-09-11T00:26:44.563 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:44.560+0000 7fbc58c00640 1 -- 172.21.15.12:0/3320744308 <== mds.0 v2:172.21.15.165:6838/501312984 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x557b4459b0b0 con 0x557b446ad540 2024-09-11T00:26:44.564 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:44.560+0000 7fbc58c00640 10 client.27653 handle_command_reply: tid=0 2024-09-11T00:26:44.564 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:44.560+0000 7fbc78322740 1 client.27653 shutdown 2024-09-11T00:26:44.564 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:44.560+0000 7fbc78322740 1 -- 172.21.15.12:0/3320744308 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557b4462a790 msgr2=0x557b4463bb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:26:44.564 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:44.560+0000 7fbc78322740 1 --2- 172.21.15.12:0/3320744308 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557b4462a790 0x557b4463bb90 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fbc600abc00 tx=0x7fbc600106e0 comp rx=0 tx=0).stop 2024-09-11T00:26:44.564 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:44.560+0000 7fbc78322740 1 -- 172.21.15.12:0/3320744308 shutdown_connections 2024-09-11T00:26:44.564 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:44.560+0000 7fbc78322740 1 --2- 172.21.15.12:0/3320744308 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x557b446ad540 0x557b446af960 crc :-1 s=READY pgs=276 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:26:44.564 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:44.560+0000 7fbc78322740 1 --2- 172.21.15.12:0/3320744308 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557b4462fb00 0x557b4463b000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:26:44.564 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:44.560+0000 7fbc78322740 1 --2- 172.21.15.12:0/3320744308 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557b4462d0c0 0x557b4463a910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:26:44.565 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:44.560+0000 7fbc58c00640 0 client.27653 ms_handle_reset on v2:172.21.15.165:6838/501312984 2024-09-11T00:26:44.565 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:44.560+0000 7fbc78322740 1 --2- 172.21.15.12:0/3320744308 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557b4462a790 0x557b4463bb90 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:26:44.565 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:44.560+0000 7fbc78322740 1 -- 172.21.15.12:0/3320744308 >> 172.21.15.12:0/3320744308 conn(0x557b446258e0 msgr2=0x557b4462fee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:26:44.565 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:44.561+0000 7fbc78322740 1 -- 172.21.15.12:0/3320744308 shutdown_connections 2024-09-11T00:26:44.565 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:44.561+0000 7fbc78322740 1 -- 172.21.15.12:0/3320744308 wait complete. 2024-09-11T00:26:44.565 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:44.561+0000 7fbc78322740 20 client.27653 trim_cache size 0 max 16384 2024-09-11T00:26:44.565 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:{ 2024-09-11T00:26:44.565 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "message": "", 2024-09-11T00:26:44.566 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "return_code": 0 2024-09-11T00:26:44.566 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:} 2024-09-11T00:26:44.566 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:44.561+0000 7fbc75800640 1 -- 172.21.15.12:0/592296523 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fbc540b5270 msgr2=0x7fbc540b7730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:26:44.566 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:44.561+0000 7fbc75800640 1 --2- 172.21.15.12:0/592296523 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fbc540b5270 0x7fbc540b7730 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fbc68009430 tx=0x7fbc68031000 comp rx=0 tx=0).stop 2024-09-11T00:26:44.566 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:44.561+0000 7fbc75800640 1 -- 172.21.15.12:0/592296523 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbc70104ba0 msgr2=0x7fbc7019af90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:26:44.566 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:44.561+0000 7fbc75800640 1 --2- 172.21.15.12:0/592296523 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbc70104ba0 0x7fbc7019af90 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7fbc6000b450 tx=0x7fbc6000b910 comp rx=0 tx=0).stop 2024-09-11T00:26:44.567 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:44.561+0000 7fbc75800640 1 -- 172.21.15.12:0/592296523 shutdown_connections 2024-09-11T00:26:44.567 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:44.561+0000 7fbc75800640 1 --2- 172.21.15.12:0/592296523 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fbc540b5270 0x7fbc540b7730 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:26:44.567 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:44.561+0000 7fbc75800640 1 --2- 172.21.15.12:0/592296523 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbc701066b0 0x7fbc7019c210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:26:44.567 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:44.561+0000 7fbc75800640 1 --2- 172.21.15.12:0/592296523 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbc70105cf0 0x7fbc7019b5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:26:44.567 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:44.561+0000 7fbc75800640 1 --2- 172.21.15.12:0/592296523 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbc70104ba0 0x7fbc7019af90 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:26:44.567 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:44.561+0000 7fbc75800640 1 -- 172.21.15.12:0/592296523 >> 172.21.15.12:0/592296523 conn(0x7fbc70100460 msgr2=0x7fbc70109010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:26:44.567 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:44.562+0000 7fbc75800640 1 -- 172.21.15.12:0/592296523 shutdown_connections 2024-09-11T00:26:44.567 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:44.562+0000 7fbc75800640 1 -- 172.21.15.12:0/592296523 wait complete. 2024-09-11T00:26:45.233 INFO:tasks.workunit.client.0.smithi012.stdout: 10 114924 15.70 MB/sec execute 281 sec latency 11728.863 ms 2024-09-11T00:26:46.063 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:26:45 smithi012 ceph-mon[30333]: pgmap v989: 129 pgs: 129 active+clean; 636 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 611 KiB/s rd, 5.7 MiB/s wr, 268 op/s 2024-09-11T00:26:46.234 INFO:tasks.workunit.client.0.smithi012.stdout: 10 115339 15.70 MB/sec execute 282 sec latency 11860.205 ms 2024-09-11T00:26:46.283 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:26:45 smithi165 ceph-mon[35356]: pgmap v989: 129 pgs: 129 active+clean; 636 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 611 KiB/s rd, 5.7 MiB/s wr, 268 op/s 2024-09-11T00:26:46.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:26:45 smithi178 ceph-mon[35919]: pgmap v989: 129 pgs: 129 active+clean; 636 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 611 KiB/s rd, 5.7 MiB/s wr, 268 op/s 2024-09-11T00:26:47.234 INFO:tasks.workunit.client.0.smithi012.stdout: 10 116201 15.85 MB/sec execute 283 sec latency 61.803 ms 2024-09-11T00:26:47.864 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:26:47 smithi012 ceph-mon[30333]: pgmap v990: 129 pgs: 129 active+clean; 634 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 5.3 MiB/s wr, 294 op/s 2024-09-11T00:26:48.132 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:26:47 smithi178 ceph-mon[35919]: pgmap v990: 129 pgs: 129 active+clean; 634 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 5.3 MiB/s wr, 294 op/s 2024-09-11T00:26:48.234 INFO:tasks.workunit.client.0.smithi012.stdout: 10 117082 15.96 MB/sec execute 284 sec latency 61.641 ms 2024-09-11T00:26:48.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:26:47 smithi165 ceph-mon[35356]: pgmap v990: 129 pgs: 129 active+clean; 634 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 5.3 MiB/s wr, 294 op/s 2024-09-11T00:26:49.234 INFO:tasks.workunit.client.0.smithi012.stdout: 10 117766 16.01 MB/sec execute 285 sec latency 71.992 ms 2024-09-11T00:26:50.065 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:26:49 smithi012 ceph-mon[30333]: pgmap v991: 129 pgs: 129 active+clean; 635 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 847 KiB/s rd, 660 KiB/s wr, 116 op/s 2024-09-11T00:26:50.234 INFO:tasks.workunit.client.0.smithi012.stdout: 10 118565 16.09 MB/sec execute 286 sec latency 80.299 ms 2024-09-11T00:26:50.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:26:49 smithi165 ceph-mon[35356]: pgmap v991: 129 pgs: 129 active+clean; 635 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 847 KiB/s rd, 660 KiB/s wr, 116 op/s 2024-09-11T00:26:50.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:26:49 smithi178 ceph-mon[35919]: pgmap v991: 129 pgs: 129 active+clean; 635 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 847 KiB/s rd, 660 KiB/s wr, 116 op/s 2024-09-11T00:26:51.234 INFO:tasks.workunit.client.0.smithi012.stdout: 10 119555 16.23 MB/sec execute 287 sec latency 95.699 ms 2024-09-11T00:26:51.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:26:50 smithi165 ceph-mon[35356]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T00:26:51.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:26:50 smithi012 ceph-mon[30333]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T00:26:51.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:26:50 smithi178 ceph-mon[35919]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T00:26:52.234 INFO:tasks.workunit.client.0.smithi012.stdout: 10 120439 16.35 MB/sec execute 288 sec latency 70.929 ms 2024-09-11T00:26:52.264 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:26:51 smithi178 ceph-mon[35919]: pgmap v992: 129 pgs: 129 active+clean; 627 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 1.3 MiB/s wr, 251 op/s 2024-09-11T00:26:52.264 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:26:51 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T00:26:52.280 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:26:51 smithi165 ceph-mon[35356]: pgmap v992: 129 pgs: 129 active+clean; 627 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 1.3 MiB/s wr, 251 op/s 2024-09-11T00:26:52.280 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:26:51 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T00:26:52.314 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:26:51 smithi012 ceph-mon[30333]: pgmap v992: 129 pgs: 129 active+clean; 627 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 1.3 MiB/s wr, 251 op/s 2024-09-11T00:26:52.314 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:26:51 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T00:26:53.234 INFO:tasks.workunit.client.0.smithi012.stdout: 10 121078 16.42 MB/sec execute 289 sec latency 105.523 ms 2024-09-11T00:26:54.195 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:26:54 smithi012 ceph-mon[30333]: pgmap v993: 129 pgs: 129 active+clean; 635 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 2.1 MiB/s wr, 342 op/s 2024-09-11T00:26:54.195 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:26:54 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:26:54.196 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:26:54 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:26:54.196 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:26:54 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:26:54.196 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:26:54 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:26:54.196 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:26:54 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:26:54.196 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:26:54 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:26:54.234 INFO:tasks.workunit.client.0.smithi012.stdout: 10 121882 16.49 MB/sec execute 290 sec latency 78.798 ms 2024-09-11T00:26:54.533 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:26:54 smithi165 ceph-mon[35356]: pgmap v993: 129 pgs: 129 active+clean; 635 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 2.1 MiB/s wr, 342 op/s 2024-09-11T00:26:54.533 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:26:54 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:26:54.533 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:26:54 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:26:54.533 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:26:54 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:26:54.533 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:26:54 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:26:54.533 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:26:54 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:26:54.533 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:26:54 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:26:54.581 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:26:54 smithi178 ceph-mon[35919]: pgmap v993: 129 pgs: 129 active+clean; 635 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 2.1 MiB/s wr, 342 op/s 2024-09-11T00:26:54.581 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:26:54 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:26:54.581 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:26:54 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:26:54.581 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:26:54 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:26:54.581 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:26:54 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:26:54.581 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:26:54 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:26:54.581 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:26:54 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:26:55.236 INFO:tasks.workunit.client.0.smithi012.stdout: 10 122781 16.61 MB/sec execute 291 sec latency 55.298 ms 2024-09-11T00:26:55.782 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:26:55 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:26:55.782 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:26:55 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:26:55.783 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:26:55 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:26:55.783 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:26:55 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:26:55.783 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:26:55 smithi165 ceph-mon[35356]: pgmap v994: 129 pgs: 129 active+clean; 635 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 2.2 MiB/s wr, 342 op/s 2024-09-11T00:26:55.785 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:26:55 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:26:55.786 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:26:55 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:26:55.786 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:26:55 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:26:55.786 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:26:55 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:26:55.786 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:26:55 smithi012 ceph-mon[30333]: pgmap v994: 129 pgs: 129 active+clean; 635 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 2.2 MiB/s wr, 342 op/s 2024-09-11T00:26:55.831 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:26:55 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:26:55.831 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:26:55 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:26:55.831 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:26:55 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:26:55.831 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:26:55 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:26:55.831 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:26:55 smithi178 ceph-mon[35919]: pgmap v994: 129 pgs: 129 active+clean; 635 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 2.2 MiB/s wr, 342 op/s 2024-09-11T00:26:56.237 INFO:tasks.workunit.client.0.smithi012.stdout: 10 123346 16.67 MB/sec execute 292 sec latency 70.542 ms 2024-09-11T00:26:57.236 INFO:tasks.workunit.client.0.smithi012.stdout: 10 124179 16.76 MB/sec execute 293 sec latency 65.987 ms 2024-09-11T00:26:57.761 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:57.758+0000 7f9f49a00640 1 -- 172.21.15.12:0/2426747892 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9f44104ba0 msgr2=0x7f9f44104fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:26:57.761 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:57.758+0000 7f9f49a00640 1 --2- 172.21.15.12:0/2426747892 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9f44104ba0 0x7f9f44104fa0 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f9f34009890 tx=0x7f9f3402a380 comp rx=0 tx=0).stop 2024-09-11T00:26:57.761 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:57.758+0000 7f9f49a00640 1 -- 172.21.15.12:0/2426747892 shutdown_connections 2024-09-11T00:26:57.761 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:57.758+0000 7f9f49a00640 1 --2- 172.21.15.12:0/2426747892 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9f441066b0 0x7f9f4410d030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:26:57.761 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:57.758+0000 7f9f49a00640 1 --2- 172.21.15.12:0/2426747892 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9f44105cf0 0x7f9f44106170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:26:57.761 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:57.758+0000 7f9f49a00640 1 --2- 172.21.15.12:0/2426747892 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9f44104ba0 0x7f9f44104fa0 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:26:57.761 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:57.758+0000 7f9f49a00640 1 -- 172.21.15.12:0/2426747892 >> 172.21.15.12:0/2426747892 conn(0x7f9f44100440 msgr2=0x7f9f44102880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:26:57.761 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:57.758+0000 7f9f49a00640 1 -- 172.21.15.12:0/2426747892 shutdown_connections 2024-09-11T00:26:57.762 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:57.758+0000 7f9f49a00640 1 -- 172.21.15.12:0/2426747892 wait complete. 2024-09-11T00:26:57.762 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:57.759+0000 7f9f49a00640 1 Processor -- start 2024-09-11T00:26:57.762 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:57.760+0000 7f9f49a00640 1 -- start start 2024-09-11T00:26:57.763 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:57.760+0000 7f9f49a00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9f44104ba0 0x7f9f4419f3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:26:57.763 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:57.760+0000 7f9f49a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9f44105cf0 0x7f9f4419fa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:26:57.763 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:57.760+0000 7f9f49a00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9f441066b0 0x7f9f441a0650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:26:57.763 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:57.760+0000 7f9f49a00640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f9f441107f0 con 0x7f9f44104ba0 2024-09-11T00:26:57.763 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:57.760+0000 7f9f49a00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f9f44110670 con 0x7f9f44105cf0 2024-09-11T00:26:57.763 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:57.760+0000 7f9f49a00640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f9f44110970 con 0x7f9f441066b0 2024-09-11T00:26:57.763 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:57.760+0000 7f9f42000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9f44105cf0 0x7f9f4419fa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:26:57.763 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:57.760+0000 7f9f42a00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9f44104ba0 0x7f9f4419f3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:26:57.763 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:57.760+0000 7f9f42000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9f44105cf0 0x7f9f4419fa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.12:54508/0 (socket says 172.21.15.12:54508) 2024-09-11T00:26:57.763 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:57.760+0000 7f9f42000640 1 -- 172.21.15.12:0/3288958064 learned_addr learned my addr 172.21.15.12:0/3288958064 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:26:57.764 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:57.761+0000 7f9f43400640 1 --2- 172.21.15.12:0/3288958064 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9f441066b0 0x7f9f441a0650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:26:57.764 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:57.762+0000 7f9f42000640 1 -- 172.21.15.12:0/3288958064 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9f441066b0 msgr2=0x7f9f441a0650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:26:57.764 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:57.762+0000 7f9f42000640 1 --2- 172.21.15.12:0/3288958064 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9f441066b0 0x7f9f441a0650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:26:57.764 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:57.762+0000 7f9f42000640 1 -- 172.21.15.12:0/3288958064 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9f44104ba0 msgr2=0x7f9f4419f3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:26:57.765 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:57.762+0000 7f9f42000640 1 --2- 172.21.15.12:0/3288958064 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9f44104ba0 0x7f9f4419f3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:26:57.765 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:57.762+0000 7f9f42000640 1 -- 172.21.15.12:0/3288958064 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f441a0f10 con 0x7f9f44105cf0 2024-09-11T00:26:57.765 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:57.762+0000 7f9f42a00640 1 --2- 172.21.15.12:0/3288958064 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9f44104ba0 0x7f9f4419f3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:26:57.766 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:57.764+0000 7f9f42000640 1 --2- 172.21.15.12:0/3288958064 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9f44105cf0 0x7f9f4419fa20 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f9f3000d950 tx=0x7f9f3000de10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:26:57.767 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:57.765+0000 7f9f3b400640 1 -- 172.21.15.12:0/3288958064 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f30017070 con 0x7f9f44105cf0 2024-09-11T00:26:57.767 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:57.765+0000 7f9f3b400640 1 -- 172.21.15.12:0/3288958064 <== mon.2 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9f30009c30 con 0x7f9f44105cf0 2024-09-11T00:26:57.767 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:57.765+0000 7f9f3b400640 1 -- 172.21.15.12:0/3288958064 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f3000f620 con 0x7f9f44105cf0 2024-09-11T00:26:57.767 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:57.765+0000 7f9f49a00640 1 -- 172.21.15.12:0/3288958064 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9f441a1200 con 0x7f9f44105cf0 2024-09-11T00:26:57.768 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:57.766+0000 7f9f49a00640 1 -- 172.21.15.12:0/3288958064 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9f441960b0 con 0x7f9f44105cf0 2024-09-11T00:26:57.773 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:57.771+0000 7f9f4ca35740 1 Processor -- start 2024-09-11T00:26:57.775 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:57.772+0000 7f9f4ca35740 1 -- start start 2024-09-11T00:26:57.776 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:57.773+0000 7f9f4ca35740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562456a96790 0x562456a98b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:26:57.776 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:57.773+0000 7f9f4ca35740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562456a990c0 0x562456a9b4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:26:57.776 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:57.773+0000 7f9f4ca35740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562456a9bb00 0x562456a9dfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:26:57.776 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:57.773+0000 7f9f4ca35740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x562456a096b0 con 0x562456a96790 2024-09-11T00:26:57.776 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:57.773+0000 7f9f4ca35740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x5624569f8ae0 con 0x562456a9bb00 2024-09-11T00:26:57.776 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:57.773+0000 7f9f4ca35740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5624569fc8f0 con 0x562456a990c0 2024-09-11T00:26:57.776 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:57.773+0000 7f9f43400640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562456a96790 0x562456a98b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:26:57.776 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:57.773+0000 7f9f43400640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562456a96790 0x562456a98b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:49224/0 (socket says 172.21.15.12:49224) 2024-09-11T00:26:57.776 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:57.773+0000 7f9f43400640 1 -- 172.21.15.12:0/3944661444 learned_addr learned my addr 172.21.15.12:0/3944661444 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:26:57.776 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:57.773+0000 7f9f3b400640 1 -- 172.21.15.12:0/3288958064 <== mon.2 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7f9f30005060 con 0x7f9f44105cf0 2024-09-11T00:26:57.777 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:57.774+0000 7f9f43400640 1 -- 172.21.15.12:0/3944661444 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562456a990c0 msgr2=0x562456a9b4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:26:57.777 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:57.774+0000 7f9f43400640 1 --2- 172.21.15.12:0/3944661444 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562456a990c0 0x562456a9b4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:26:57.777 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:57.774+0000 7f9f43400640 1 -- 172.21.15.12:0/3944661444 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562456a9bb00 msgr2=0x562456a9dfc0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-11T00:26:57.777 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:57.774+0000 7f9f43400640 1 --2- 172.21.15.12:0/3944661444 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562456a9bb00 0x562456a9dfc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:26:57.777 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:57.774+0000 7f9f43400640 1 -- 172.21.15.12:0/3944661444 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5624569c5780 con 0x562456a96790 2024-09-11T00:26:57.777 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:57.774+0000 7f9f43400640 1 --2- 172.21.15.12:0/3944661444 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562456a96790 0x562456a98b80 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f9f3c00f430 tx=0x7f9f3c00f8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b83a9db23fa6b743 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:26:57.778 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:57.775+0000 7f9f2fe00640 1 -- 172.21.15.12:0/3944661444 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f3c01d070 con 0x562456a96790 2024-09-11T00:26:57.778 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:57.775+0000 7f9f2fe00640 1 -- 172.21.15.12:0/3944661444 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9f3c00caf0 con 0x562456a96790 2024-09-11T00:26:57.778 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:57.775+0000 7f9f2fe00640 1 -- 172.21.15.12:0/3944661444 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f3c018bc0 con 0x562456a96790 2024-09-11T00:26:57.778 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:57.776+0000 7f9f4ca35740 1 -- 172.21.15.12:0/3944661444 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562456a96790 msgr2=0x562456a98b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:26:57.778 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:57.776+0000 7f9f4ca35740 1 --2- 172.21.15.12:0/3944661444 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562456a96790 0x562456a98b80 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f9f3c00f430 tx=0x7f9f3c00f8f0 comp rx=0 tx=0).stop 2024-09-11T00:26:57.778 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:57.776+0000 7f9f3b400640 1 --2- 172.21.15.12:0/3288958064 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f9f280b54b0 0x7f9f280b7970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:26:57.779 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:57.776+0000 7f9f3b400640 1 -- 172.21.15.12:0/3288958064 <== mon.2 v2:172.21.15.165:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9208+0+0 (secure 0 0 0) 0x7f9f300e57f0 con 0x7f9f44105cf0 2024-09-11T00:26:57.779 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:57.777+0000 7f9f43400640 1 --2- 172.21.15.12:0/3288958064 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f9f280b54b0 0x7f9f280b7970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:26:57.779 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:57.777+0000 7f9f4ca35740 1 -- 172.21.15.12:0/3944661444 shutdown_connections 2024-09-11T00:26:57.779 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:57.777+0000 7f9f4ca35740 1 --2- 172.21.15.12:0/3944661444 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562456a9bb00 0x562456a9dfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:26:57.779 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:57.777+0000 7f9f4ca35740 1 --2- 172.21.15.12:0/3944661444 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562456a990c0 0x562456a9b4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:26:57.779 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:57.777+0000 7f9f4ca35740 1 --2- 172.21.15.12:0/3944661444 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562456a96790 0x562456a98b80 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:26:57.779 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:57.777+0000 7f9f4ca35740 1 -- 172.21.15.12:0/3944661444 >> 172.21.15.12:0/3944661444 conn(0x562456a918e0 msgr2=0x562456a92670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:26:57.780 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:57.778+0000 7f9f4ca35740 1 -- 172.21.15.12:0/3944661444 shutdown_connections 2024-09-11T00:26:57.780 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:57.778+0000 7f9f4ca35740 1 -- 172.21.15.12:0/3944661444 wait complete. 2024-09-11T00:26:57.780 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:57.778+0000 7f9f43400640 1 --2- 172.21.15.12:0/3288958064 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f9f280b54b0 0x7f9f280b7970 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f9f3c005c10 tx=0x7f9f3c018410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:26:57.781 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:57.779+0000 7f9f4ca35740 1 -- start start 2024-09-11T00:26:57.781 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:57.779+0000 7f9f4ca35740 1 Processor -- start 2024-09-11T00:26:57.783 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:57.780+0000 7f9f4ca35740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562456a96790 0x562456ab0f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:26:57.783 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:57.780+0000 7f9f4ca35740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562456a990c0 0x562456ab15b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:26:57.783 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:57.780+0000 7f9f4ca35740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562456a9bb00 0x562456ab2180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:26:57.783 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:57.780+0000 7f9f4ca35740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x562456930ab0 con 0x562456a9bb00 2024-09-11T00:26:57.783 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:57.780+0000 7f9f4ca35740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x56245699f570 con 0x562456a990c0 2024-09-11T00:26:57.783 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:57.780+0000 7f9f4ca35740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x562456aafaf0 con 0x562456a96790 2024-09-11T00:26:57.784 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:57.780+0000 7f9f4ca35740 10 client.0 _ll_register_callbacks cb 0x5624569ae100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:26:57.785 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:57.780+0000 7f9f43400640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562456a9bb00 0x562456ab2180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:26:57.785 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:57.780+0000 7f9f43400640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562456a9bb00 0x562456ab2180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:49234/0 (socket says 172.21.15.12:49234) 2024-09-11T00:26:57.785 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:57.780+0000 7f9f43400640 1 -- 172.21.15.12:0/2216818339 learned_addr learned my addr 172.21.15.12:0/2216818339 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:26:57.785 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:57.781+0000 7f9f42a00640 1 --2- 172.21.15.12:0/2216818339 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562456a96790 0x562456ab0f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:26:57.785 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:57.781+0000 7f9f43400640 1 -- 172.21.15.12:0/2216818339 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562456a96790 msgr2=0x562456ab0f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:26:57.785 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:57.781+0000 7f9f43400640 1 --2- 172.21.15.12:0/2216818339 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562456a96790 0x562456ab0f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:26:57.785 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:57.781+0000 7f9f43400640 1 -- 172.21.15.12:0/2216818339 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562456a990c0 msgr2=0x562456ab15b0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-11T00:26:57.785 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:57.781+0000 7f9f43400640 1 --2- 172.21.15.12:0/2216818339 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562456a990c0 0x562456ab15b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:26:57.785 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:57.781+0000 7f9f43400640 1 -- 172.21.15.12:0/2216818339 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56245675a790 con 0x562456a9bb00 2024-09-11T00:26:57.785 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:57.782+0000 7f9f43400640 1 --2- 172.21.15.12:0/2216818339 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562456a9bb00 0x562456ab2180 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f9f3c019fd0 tx=0x7f9f3c00f8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28b64fb60f7dd9ab server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:26:57.785 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:57.782+0000 7f9f2cc00640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-09-11T00:26:57.786 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:57.783+0000 7f9f2cc00640 1 -- 172.21.15.12:0/2216818339 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f3c034060 con 0x562456a9bb00 2024-09-11T00:26:57.786 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:57.783+0000 7f9f2cc00640 1 -- 172.21.15.12:0/2216818339 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9208+0+0 (secure 0 0 0) 0x7f9f3c018980 con 0x562456a9bb00 2024-09-11T00:26:57.786 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:57.783+0000 7f9f4ca35740 1 -- 172.21.15.12:0/2216818339 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9f3c00caf0 con 0x562456a9bb00 2024-09-11T00:26:57.786 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:57.783+0000 7f9f2cc00640 1 -- 172.21.15.12:0/2216818339 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=82) ==== 24+0+0 (secure 0 0 0) 0x7f9f3c01d050 con 0x562456a9bb00 2024-09-11T00:26:57.787 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:57.784+0000 7f9f4ca35740 10 client.17649 fetch_fsmap learned FSMap version 82 2024-09-11T00:26:57.787 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:57.784+0000 7f9f4ca35740 1 -- 172.21.15.12:0/2216818339 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=82,monmap=4+}) -- 0x5624567d3620 con 0x562456a9bb00 2024-09-11T00:26:57.787 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:57.785+0000 7f9f2cc00640 1 -- 172.21.15.12:0/2216818339 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9f3c00caf0 con 0x562456a9bb00 2024-09-11T00:26:57.788 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:57.785+0000 7f9f2cc00640 1 -- 172.21.15.12:0/2216818339 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 82) ==== 8418+0+0 (secure 0 0 0) 0x7f9f3c00b040 con 0x562456a9bb00 2024-09-11T00:26:57.788 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:57.786+0000 7f9f4ca35740 10 client.17649 fetch_fsmap finished waiting for FSMap version 82 2024-09-11T00:26:57.788 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:57.786+0000 7f9f4ca35740 10 client.17649 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T00:26:57.788 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:57.786+0000 7f9f4ca35740 20 client.17649 populate_metadata read hostname 'smithi012' 2024-09-11T00:26:57.788 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:57.786+0000 7f9f4ca35740 1 --2- 172.21.15.12:0/2216818339 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x562456abc580 0x562456abe9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:26:57.788 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:57.786+0000 7f9f4ca35740 4 client.17649 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T00:26:57.788 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:57.786+0000 7f9f4ca35740 1 -- 172.21.15.12:0/2216818339 --> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562456a0b600 con 0x562456abc580 2024-09-11T00:26:57.789 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:57.786+0000 7f9f42a00640 1 --2- 172.21.15.12:0/2216818339 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x562456abc580 0x562456abe9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:26:57.789 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:57.787+0000 7f9f42a00640 1 --2- 172.21.15.12:0/2216818339 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x562456abc580 0x562456abe9c0 crc :-1 s=READY pgs=277 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=28f511613ebf6a3 server_cookie=e4072beb958ff09c in_seq=0 out_seq=0 2024-09-11T00:26:57.789 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:57.787+0000 7f9f2cc00640 10 client.17649 ms_handle_connect on v2:172.21.15.165:6838/501312984 2024-09-11T00:26:57.790 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:57.788+0000 7f9f2cc00640 1 -- 172.21.15.12:0/2216818339 <== mds.0 v2:172.21.15.165:6838/501312984 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x562456a0b600 con 0x562456abc580 2024-09-11T00:26:57.791 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:57.788+0000 7f9f2cc00640 10 client.17649 handle_command_reply: tid=0 2024-09-11T00:26:57.791 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:57.789+0000 7f9f4ca35740 1 client.17649 shutdown 2024-09-11T00:26:57.792 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:57.789+0000 7f9f4ca35740 1 -- 172.21.15.12:0/2216818339 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562456a9bb00 msgr2=0x562456ab2180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:26:57.792 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:57.789+0000 7f9f4ca35740 1 --2- 172.21.15.12:0/2216818339 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562456a9bb00 0x562456ab2180 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f9f3c019fd0 tx=0x7f9f3c00f8f0 comp rx=0 tx=0).stop 2024-09-11T00:26:57.792 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:57.790+0000 7f9f4ca35740 1 -- 172.21.15.12:0/2216818339 shutdown_connections 2024-09-11T00:26:57.792 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:57.790+0000 7f9f4ca35740 1 --2- 172.21.15.12:0/2216818339 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x562456abc580 0x562456abe9c0 crc :-1 s=READY pgs=277 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:26:57.793 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:57.790+0000 7f9f4ca35740 1 --2- 172.21.15.12:0/2216818339 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562456a9bb00 0x562456ab2180 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:26:57.793 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:57.790+0000 7f9f2cc00640 0 client.17649 ms_handle_reset on v2:172.21.15.165:6838/501312984 2024-09-11T00:26:57.793 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:57.790+0000 7f9f4ca35740 1 --2- 172.21.15.12:0/2216818339 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562456a990c0 0x562456ab15b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:26:57.793 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:57.790+0000 7f9f4ca35740 1 --2- 172.21.15.12:0/2216818339 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562456a96790 0x562456ab0f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:26:57.793 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:57.790+0000 7f9f4ca35740 1 -- 172.21.15.12:0/2216818339 >> 172.21.15.12:0/2216818339 conn(0x562456a918e0 msgr2=0x562456a90a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:26:57.793 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:57.790+0000 7f9f4ca35740 1 -- 172.21.15.12:0/2216818339 shutdown_connections 2024-09-11T00:26:57.793 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:57.790+0000 7f9f4ca35740 1 -- 172.21.15.12:0/2216818339 wait complete. 2024-09-11T00:26:57.793 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:57.791+0000 7f9f4ca35740 20 client.17649 trim_cache size 0 max 16384 2024-09-11T00:26:57.817 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:57.814+0000 7f9f4ca35740 1 Processor -- start 2024-09-11T00:26:57.817 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:57.814+0000 7f9f4ca35740 1 -- start start 2024-09-11T00:26:57.817 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:57.815+0000 7f9f4ca35740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562456a96790 0x562456ab0f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:26:57.817 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:57.815+0000 7f9f4ca35740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562456a990c0 0x562456a92140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:26:57.817 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:57.815+0000 7f9f4ca35740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562456a9bb00 0x562456a92680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:26:57.817 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:57.815+0000 7f9f4ca35740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f9f4406ebd0 con 0x562456a96790 2024-09-11T00:26:57.818 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:57.815+0000 7f9f4ca35740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f9f44070890 con 0x562456a9bb00 2024-09-11T00:26:57.818 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:57.815+0000 7f9f4ca35740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f9f4406f360 con 0x562456a990c0 2024-09-11T00:26:57.818 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:57.815+0000 7f9f42000640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562456a990c0 0x562456a92140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:26:57.818 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:57.815+0000 7f9f42a00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562456a96790 0x562456ab0f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:26:57.818 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:57.815+0000 7f9f42000640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562456a990c0 0x562456a92140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.12:43310/0 (socket says 172.21.15.12:43310) 2024-09-11T00:26:57.818 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:57.815+0000 7f9f42000640 1 -- 172.21.15.12:0/3671077959 learned_addr learned my addr 172.21.15.12:0/3671077959 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:26:57.819 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:57.815+0000 7f9f43400640 1 --2- 172.21.15.12:0/3671077959 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562456a9bb00 0x562456a92680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:26:57.819 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:57.815+0000 7f9f42000640 1 -- 172.21.15.12:0/3671077959 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562456a9bb00 msgr2=0x562456a92680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:26:57.819 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:57.815+0000 7f9f42000640 1 --2- 172.21.15.12:0/3671077959 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562456a9bb00 0x562456a92680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:26:57.819 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:57.815+0000 7f9f42000640 1 -- 172.21.15.12:0/3671077959 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562456a96790 msgr2=0x562456ab0f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:26:57.820 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:57.816+0000 7f9f42000640 1 --2- 172.21.15.12:0/3671077959 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562456a96790 0x562456ab0f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:26:57.820 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:57.816+0000 7f9f42000640 1 -- 172.21.15.12:0/3671077959 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562456a95900 con 0x562456a990c0 2024-09-11T00:26:57.820 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:57.816+0000 7f9f43400640 1 --2- 172.21.15.12:0/3671077959 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562456a9bb00 0x562456a92680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T00:26:57.820 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:57.816+0000 7f9f42000640 1 --2- 172.21.15.12:0/3671077959 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562456a990c0 0x562456a92140 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f9f300a8df0 tx=0x7f9f30005880 comp rx=0 tx=0).ready entity=mon.1 client_cookie=abd4a431fb7fc9e6 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:26:57.820 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:57.816+0000 7f9f2f400640 1 -- 172.21.15.12:0/3671077959 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f30020070 con 0x562456a990c0 2024-09-11T00:26:57.820 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:57.816+0000 7f9f2f400640 1 -- 172.21.15.12:0/3671077959 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9f300ab690 con 0x562456a990c0 2024-09-11T00:26:57.820 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:57.816+0000 7f9f2f400640 1 -- 172.21.15.12:0/3671077959 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f300b4ee0 con 0x562456a990c0 2024-09-11T00:26:57.821 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:57.817+0000 7f9f4ca35740 1 -- 172.21.15.12:0/3671077959 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562456a990c0 msgr2=0x562456a92140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:26:57.821 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:57.817+0000 7f9f4ca35740 1 --2- 172.21.15.12:0/3671077959 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562456a990c0 0x562456a92140 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f9f300a8df0 tx=0x7f9f30005880 comp rx=0 tx=0).stop 2024-09-11T00:26:57.821 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:57.817+0000 7f9f4ca35740 1 -- 172.21.15.12:0/3671077959 shutdown_connections 2024-09-11T00:26:57.821 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:57.817+0000 7f9f4ca35740 1 --2- 172.21.15.12:0/3671077959 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562456a9bb00 0x562456a92680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:26:57.821 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:57.817+0000 7f9f4ca35740 1 --2- 172.21.15.12:0/3671077959 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562456a990c0 0x562456a92140 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:26:57.821 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:57.817+0000 7f9f4ca35740 1 --2- 172.21.15.12:0/3671077959 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562456a96790 0x562456ab0f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:26:57.822 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:57.817+0000 7f9f4ca35740 1 -- 172.21.15.12:0/3671077959 >> 172.21.15.12:0/3671077959 conn(0x562456a918e0 msgr2=0x562456a95c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:26:57.822 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:57.818+0000 7f9f4ca35740 1 -- 172.21.15.12:0/3671077959 shutdown_connections 2024-09-11T00:26:57.822 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:57.818+0000 7f9f4ca35740 1 -- 172.21.15.12:0/3671077959 wait complete. 2024-09-11T00:26:57.822 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:57.819+0000 7f9f4ca35740 1 -- start start 2024-09-11T00:26:57.822 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:57.819+0000 7f9f4ca35740 1 Processor -- start 2024-09-11T00:26:57.823 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:57.820+0000 7f9f4ca35740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562456a96790 0x562456ab8520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:26:57.823 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:57.820+0000 7f9f4ca35740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562456a990c0 0x562456ab8bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:26:57.823 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:57.820+0000 7f9f4ca35740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562456a9bb00 0x562456ab7280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:26:57.823 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:57.820+0000 7f9f4ca35740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f9f440701c0 con 0x562456a990c0 2024-09-11T00:26:57.823 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:57.820+0000 7f9f4ca35740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f9f4406fa30 con 0x562456a96790 2024-09-11T00:26:57.823 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:57.820+0000 7f9f4ca35740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f9f4406cbb0 con 0x562456a9bb00 2024-09-11T00:26:57.823 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:57.820+0000 7f9f4ca35740 10 client.0 _ll_register_callbacks cb 0x7f9f3c01b070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:26:57.823 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:57.820+0000 7f9f42a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562456a96790 0x562456ab8520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:26:57.823 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:57.820+0000 7f9f42a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562456a96790 0x562456ab8520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.12:54544/0 (socket says 172.21.15.12:54544) 2024-09-11T00:26:57.823 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:57.820+0000 7f9f42a00640 1 -- 172.21.15.12:0/3606054862 learned_addr learned my addr 172.21.15.12:0/3606054862 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:26:57.823 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:57.820+0000 7f9f43400640 1 --2- 172.21.15.12:0/3606054862 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562456a9bb00 0x562456ab7280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:26:57.824 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:57.821+0000 7f9f42a00640 1 -- 172.21.15.12:0/3606054862 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562456a9bb00 msgr2=0x562456ab7280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:26:57.825 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:57.821+0000 7f9f42a00640 1 --2- 172.21.15.12:0/3606054862 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562456a9bb00 0x562456ab7280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:26:57.825 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:57.821+0000 7f9f42a00640 1 -- 172.21.15.12:0/3606054862 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562456a990c0 msgr2=0x562456ab8bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:26:57.825 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:57.821+0000 7f9f42000640 1 --2- 172.21.15.12:0/3606054862 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562456a990c0 0x562456ab8bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:26:57.825 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:57.821+0000 7f9f42a00640 1 --2- 172.21.15.12:0/3606054862 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562456a990c0 0x562456ab8bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:26:57.825 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:57.821+0000 7f9f42a00640 1 -- 172.21.15.12:0/3606054862 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562456ab7ab0 con 0x562456a96790 2024-09-11T00:26:57.825 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:57.822+0000 7f9f42000640 1 --2- 172.21.15.12:0/3606054862 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562456a990c0 0x562456ab8bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T00:26:57.825 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:57.822+0000 7f9f43400640 1 --2- 172.21.15.12:0/3606054862 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562456a9bb00 0x562456ab7280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:26:57.825 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:57.822+0000 7f9f42a00640 1 --2- 172.21.15.12:0/3606054862 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562456a96790 0x562456ab8520 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f9f34030cd0 tx=0x7f9f34030fb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c47e01de47647e36 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:26:57.825 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:57.822+0000 7f9f2cc00640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-09-11T00:26:57.825 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:57.822+0000 7f9f2cc00640 1 -- 172.21.15.12:0/3606054862 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f3404b070 con 0x562456a96790 2024-09-11T00:26:57.826 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:57.823+0000 7f9f2cc00640 1 -- 172.21.15.12:0/3606054862 <== mon.2 v2:172.21.15.165:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9208+0+0 (secure 0 0 0) 0x7f9f34030810 con 0x562456a96790 2024-09-11T00:26:57.826 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:57.823+0000 7f9f4ca35740 1 -- 172.21.15.12:0/3606054862 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9f300ab690 con 0x562456a96790 2024-09-11T00:26:57.826 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:57.823+0000 7f9f2cc00640 1 -- 172.21.15.12:0/3606054862 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=82) ==== 24+0+0 (secure 0 0 0) 0x7f9f34055050 con 0x562456a96790 2024-09-11T00:26:57.826 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:57.823+0000 7f9f4ca35740 10 client.27662 fetch_fsmap learned FSMap version 82 2024-09-11T00:26:57.826 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:57.823+0000 7f9f4ca35740 1 -- 172.21.15.12:0/3606054862 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=82,monmap=4+}) -- 0x562456ab6900 con 0x562456a96790 2024-09-11T00:26:57.827 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:57.824+0000 7f9f2cc00640 1 -- 172.21.15.12:0/3606054862 <== mon.2 v2:172.21.15.165:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9f340327f0 con 0x562456a96790 2024-09-11T00:26:57.827 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:57.824+0000 7f9f2cc00640 1 -- 172.21.15.12:0/3606054862 <== mon.2 v2:172.21.15.165:3300/0 5 ==== fsmap(e 82) ==== 8418+0+0 (secure 0 0 0) 0x7f9f34003710 con 0x562456a96790 2024-09-11T00:26:57.827 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:57.825+0000 7f9f4ca35740 10 client.27662 fetch_fsmap finished waiting for FSMap version 82 2024-09-11T00:26:57.827 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:57.825+0000 7f9f4ca35740 10 client.27662 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T00:26:57.827 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:57.825+0000 7f9f4ca35740 20 client.27662 populate_metadata read hostname 'smithi012' 2024-09-11T00:26:57.827 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:57.825+0000 7f9f4ca35740 1 --2- 172.21.15.12:0/3606054862 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x562456abc580 0x562456b2bac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:26:57.827 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:57.825+0000 7f9f4ca35740 4 client.27662 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T00:26:57.828 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:57.825+0000 7f9f4ca35740 1 -- 172.21.15.12:0/3606054862 --> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7f9f300ab690 con 0x562456abc580 2024-09-11T00:26:57.828 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:57.825+0000 7f9f42000640 1 --2- 172.21.15.12:0/3606054862 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x562456abc580 0x562456b2bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:26:57.829 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:57.827+0000 7f9f42000640 1 --2- 172.21.15.12:0/3606054862 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x562456abc580 0x562456b2bac0 crc :-1 s=READY pgs=278 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ea0386f3f317a0b5 server_cookie=52aaf860e286ca6f in_seq=0 out_seq=0 2024-09-11T00:26:57.830 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:57.827+0000 7f9f2cc00640 10 client.27662 ms_handle_connect on v2:172.21.15.165:6838/501312984 2024-09-11T00:26:58.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:26:57 smithi165 ceph-mon[35356]: pgmap v995: 129 pgs: 129 active+clean; 651 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 MiB/s rd, 3.8 MiB/s wr, 578 op/s 2024-09-11T00:26:58.033 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:26:57 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:26:58.033 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:26:57 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:26:58.033 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:26:57 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:26:58.033 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:26:57 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T00:26:58.033 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:26:57 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:26:58.063 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:26:57 smithi012 ceph-mon[30333]: pgmap v995: 129 pgs: 129 active+clean; 651 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 MiB/s rd, 3.8 MiB/s wr, 578 op/s 2024-09-11T00:26:58.063 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:26:57 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:26:58.064 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:26:57 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:26:58.064 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:26:57 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:26:58.064 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:26:57 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T00:26:58.064 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:26:57 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:26:58.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:26:57 smithi178 ceph-mon[35919]: pgmap v995: 129 pgs: 129 active+clean; 651 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 MiB/s rd, 3.8 MiB/s wr, 578 op/s 2024-09-11T00:26:58.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:26:57 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:26:58.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:26:57 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:26:58.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:26:57 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:26:58.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:26:57 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T00:26:58.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:26:57 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:26:58.169 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:58.167+0000 7f9f2cc00640 1 -- 172.21.15.12:0/3606054862 <== mds.0 v2:172.21.15.165:6838/501312984 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x7f9f300ab690 con 0x562456abc580 2024-09-11T00:26:58.169 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:58.167+0000 7f9f2cc00640 10 client.27662 handle_command_reply: tid=0 2024-09-11T00:26:58.170 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:58.167+0000 7f9f4ca35740 1 client.27662 shutdown 2024-09-11T00:26:58.170 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:58.167+0000 7f9f4ca35740 1 -- 172.21.15.12:0/3606054862 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562456a96790 msgr2=0x562456ab8520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:26:58.170 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:58.167+0000 7f9f4ca35740 1 --2- 172.21.15.12:0/3606054862 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562456a96790 0x562456ab8520 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f9f34030cd0 tx=0x7f9f34030fb0 comp rx=0 tx=0).stop 2024-09-11T00:26:58.170 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:58.168+0000 7f9f4ca35740 1 -- 172.21.15.12:0/3606054862 shutdown_connections 2024-09-11T00:26:58.170 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:58.168+0000 7f9f4ca35740 1 --2- 172.21.15.12:0/3606054862 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x562456abc580 0x562456b2bac0 crc :-1 s=READY pgs=278 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:26:58.170 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:58.168+0000 7f9f4ca35740 1 --2- 172.21.15.12:0/3606054862 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562456a9bb00 0x562456ab7280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:26:58.170 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:58.168+0000 7f9f4ca35740 1 --2- 172.21.15.12:0/3606054862 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562456a990c0 0x562456ab8bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:26:58.171 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:58.168+0000 7f9f4ca35740 1 --2- 172.21.15.12:0/3606054862 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562456a96790 0x562456ab8520 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:26:58.171 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:58.168+0000 7f9f4ca35740 1 -- 172.21.15.12:0/3606054862 >> 172.21.15.12:0/3606054862 conn(0x562456a918e0 msgr2=0x562456a95b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:26:58.171 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:58.168+0000 7f9f2cc00640 0 client.27662 ms_handle_reset on v2:172.21.15.165:6838/501312984 2024-09-11T00:26:58.171 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:58.168+0000 7f9f4ca35740 1 -- 172.21.15.12:0/3606054862 shutdown_connections 2024-09-11T00:26:58.171 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:58.168+0000 7f9f4ca35740 1 -- 172.21.15.12:0/3606054862 wait complete. 2024-09-11T00:26:58.172 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:58.169+0000 7f9f4ca35740 20 client.27662 trim_cache size 0 max 16384 2024-09-11T00:26:58.172 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:{ 2024-09-11T00:26:58.172 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "message": "", 2024-09-11T00:26:58.172 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "return_code": 0 2024-09-11T00:26:58.172 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:} 2024-09-11T00:26:58.172 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:58.170+0000 7f9f49a00640 1 -- 172.21.15.12:0/3288958064 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f9f280b54b0 msgr2=0x7f9f280b7970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:26:58.173 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:58.170+0000 7f9f49a00640 1 --2- 172.21.15.12:0/3288958064 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f9f280b54b0 0x7f9f280b7970 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f9f3c005c10 tx=0x7f9f3c018410 comp rx=0 tx=0).stop 2024-09-11T00:26:58.173 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:58.170+0000 7f9f49a00640 1 -- 172.21.15.12:0/3288958064 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9f44105cf0 msgr2=0x7f9f4419fa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:26:58.173 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:58.170+0000 7f9f49a00640 1 --2- 172.21.15.12:0/3288958064 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9f44105cf0 0x7f9f4419fa20 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f9f3000d950 tx=0x7f9f3000de10 comp rx=0 tx=0).stop 2024-09-11T00:26:58.173 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:58.171+0000 7f9f49a00640 1 -- 172.21.15.12:0/3288958064 shutdown_connections 2024-09-11T00:26:58.173 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:58.171+0000 7f9f49a00640 1 --2- 172.21.15.12:0/3288958064 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f9f280b54b0 0x7f9f280b7970 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:26:58.173 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:58.171+0000 7f9f49a00640 1 --2- 172.21.15.12:0/3288958064 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9f441066b0 0x7f9f441a0650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:26:58.174 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:58.171+0000 7f9f49a00640 1 --2- 172.21.15.12:0/3288958064 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9f44105cf0 0x7f9f4419fa20 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:26:58.174 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:58.171+0000 7f9f49a00640 1 --2- 172.21.15.12:0/3288958064 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9f44104ba0 0x7f9f4419f3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:26:58.174 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:58.171+0000 7f9f49a00640 1 -- 172.21.15.12:0/3288958064 >> 172.21.15.12:0/3288958064 conn(0x7f9f44100440 msgr2=0x7f9f440795a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:26:58.174 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:58.171+0000 7f9f49a00640 1 -- 172.21.15.12:0/3288958064 shutdown_connections 2024-09-11T00:26:58.174 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:26:58.171+0000 7f9f49a00640 1 -- 172.21.15.12:0/3288958064 wait complete. 2024-09-11T00:26:58.236 INFO:tasks.workunit.client.0.smithi012.stdout: 10 124870 16.81 MB/sec execute 294 sec latency 240.515 ms 2024-09-11T00:26:59.033 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:26:58 smithi165 ceph-mon[35356]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T00:26:59.064 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:26:58 smithi012 ceph-mon[30333]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T00:26:59.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:26:58 smithi178 ceph-mon[35919]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T00:26:59.236 INFO:tasks.workunit.client.0.smithi012.stdout: 10 125637 16.90 MB/sec execute 295 sec latency 91.078 ms 2024-09-11T00:27:00.033 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:26:59 smithi165 ceph-mon[35356]: pgmap v996: 129 pgs: 129 active+clean; 654 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 4.0 MiB/s wr, 574 op/s 2024-09-11T00:27:00.055 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:26:59 smithi012 ceph-mon[30333]: pgmap v996: 129 pgs: 129 active+clean; 654 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 4.0 MiB/s wr, 574 op/s 2024-09-11T00:27:00.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:26:59 smithi178 ceph-mon[35919]: pgmap v996: 129 pgs: 129 active+clean; 654 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 4.0 MiB/s wr, 574 op/s 2024-09-11T00:27:00.242 INFO:tasks.workunit.client.0.smithi012.stdout: 10 126435 16.99 MB/sec execute 296 sec latency 54.387 ms 2024-09-11T00:27:01.033 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:27:00 smithi165 ceph-mon[35356]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T00:27:01.063 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:27:00 smithi012 ceph-mon[30333]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T00:27:01.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:27:00 smithi178 ceph-mon[35919]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T00:27:01.236 INFO:tasks.workunit.client.0.smithi012.stdout: 10 127268 17.09 MB/sec execute 297 sec latency 48.897 ms 2024-09-11T00:27:02.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:27:01 smithi165 ceph-mon[35356]: pgmap v997: 129 pgs: 129 active+clean; 656 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 4.7 MiB/s wr, 698 op/s 2024-09-11T00:27:02.064 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:27:01 smithi012 ceph-mon[30333]: pgmap v997: 129 pgs: 129 active+clean; 656 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 4.7 MiB/s wr, 698 op/s 2024-09-11T00:27:02.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:27:01 smithi178 ceph-mon[35919]: pgmap v997: 129 pgs: 129 active+clean; 656 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 4.7 MiB/s wr, 698 op/s 2024-09-11T00:27:02.236 INFO:tasks.workunit.client.0.smithi012.stdout: 10 128259 17.23 MB/sec execute 298 sec latency 47.402 ms 2024-09-11T00:27:03.237 INFO:tasks.workunit.client.0.smithi012.stdout: 10 128883 17.26 MB/sec execute 299 sec latency 119.350 ms 2024-09-11T00:27:03.997 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:27:03 smithi012 ceph-mon[30333]: pgmap v998: 129 pgs: 129 active+clean; 667 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 4.7 MiB/s wr, 750 op/s 2024-09-11T00:27:04.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:27:03 smithi165 ceph-mon[35356]: pgmap v998: 129 pgs: 129 active+clean; 667 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 4.7 MiB/s wr, 750 op/s 2024-09-11T00:27:04.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:27:03 smithi178 ceph-mon[35919]: pgmap v998: 129 pgs: 129 active+clean; 667 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 4.7 MiB/s wr, 750 op/s 2024-09-11T00:27:04.236 INFO:tasks.workunit.client.0.smithi012.stdout: 10 129165 17.26 MB/sec execute 300 sec latency 403.992 ms 2024-09-11T00:27:05.237 INFO:tasks.workunit.client.0.smithi012.stdout: 10 129167 17.20 MB/sec execute 301 sec latency 1404.079 ms 2024-09-11T00:27:05.973 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:27:05 smithi012 ceph-mon[30333]: pgmap v999: 129 pgs: 129 active+clean; 671 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 MiB/s rd, 4.3 MiB/s wr, 626 op/s 2024-09-11T00:27:06.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:27:05 smithi165 ceph-mon[35356]: pgmap v999: 129 pgs: 129 active+clean; 671 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 MiB/s rd, 4.3 MiB/s wr, 626 op/s 2024-09-11T00:27:06.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:27:05 smithi178 ceph-mon[35919]: pgmap v999: 129 pgs: 129 active+clean; 671 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 MiB/s rd, 4.3 MiB/s wr, 626 op/s 2024-09-11T00:27:06.237 INFO:tasks.workunit.client.0.smithi012.stdout: 10 129167 17.14 MB/sec execute 302 sec latency 2404.143 ms 2024-09-11T00:27:07.237 INFO:tasks.workunit.client.0.smithi012.stdout: 10 129170 17.09 MB/sec execute 303 sec latency 3404.228 ms 2024-09-11T00:27:07.941 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:27:07 smithi012 ceph-mon[30333]: pgmap v1000: 129 pgs: 129 active+clean; 724 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 10 MiB/s wr, 870 op/s 2024-09-11T00:27:08.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:27:07 smithi165 ceph-mon[35356]: pgmap v1000: 129 pgs: 129 active+clean; 724 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 10 MiB/s wr, 870 op/s 2024-09-11T00:27:08.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:27:07 smithi178 ceph-mon[35919]: pgmap v1000: 129 pgs: 129 active+clean; 724 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 10 MiB/s wr, 870 op/s 2024-09-11T00:27:08.237 INFO:tasks.workunit.client.0.smithi012.stdout: 10 129170 17.03 MB/sec execute 304 sec latency 4404.277 ms 2024-09-11T00:27:09.237 INFO:tasks.workunit.client.0.smithi012.stdout: 10 129170 16.98 MB/sec execute 305 sec latency 5404.322 ms 2024-09-11T00:27:10.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:27:09 smithi165 ceph-mon[35356]: pgmap v1001: 129 pgs: 129 active+clean; 729 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 9.0 MiB/s wr, 654 op/s 2024-09-11T00:27:10.063 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:27:09 smithi012 ceph-mon[30333]: pgmap v1001: 129 pgs: 129 active+clean; 729 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 9.0 MiB/s wr, 654 op/s 2024-09-11T00:27:10.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:27:09 smithi178 ceph-mon[35919]: pgmap v1001: 129 pgs: 129 active+clean; 729 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 9.0 MiB/s wr, 654 op/s 2024-09-11T00:27:10.237 INFO:tasks.workunit.client.0.smithi012.stdout: 10 129170 16.92 MB/sec execute 306 sec latency 6405.048 ms 2024-09-11T00:27:11.238 INFO:tasks.workunit.client.0.smithi012.stdout: 10 129170 16.86 MB/sec execute 307 sec latency 7405.116 ms 2024-09-11T00:27:11.308 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:11.306+0000 7f24a8e00640 1 -- 172.21.15.12:0/1255653649 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f24a4102790 msgr2=0x7f24a4102b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:27:11.308 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:11.306+0000 7f24a8e00640 1 --2- 172.21.15.12:0/1255653649 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f24a4102790 0x7f24a4102b90 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f2494009930 tx=0x7f249402a420 comp rx=0 tx=0).stop 2024-09-11T00:27:11.308 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:11.306+0000 7f24a8e00640 1 -- 172.21.15.12:0/1255653649 shutdown_connections 2024-09-11T00:27:11.309 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:11.306+0000 7f24a8e00640 1 --2- 172.21.15.12:0/1255653649 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f24a41042a0 0x7f24a410ac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:27:11.309 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:11.306+0000 7f24a8e00640 1 --2- 172.21.15.12:0/1255653649 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f24a41038e0 0x7f24a4103d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:27:11.309 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:11.306+0000 7f24a8e00640 1 --2- 172.21.15.12:0/1255653649 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f24a4102790 0x7f24a4102b90 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:27:11.309 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:11.306+0000 7f24a8e00640 1 -- 172.21.15.12:0/1255653649 >> 172.21.15.12:0/1255653649 conn(0x7f24a40fe030 msgr2=0x7f24a4100470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:27:11.309 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:11.307+0000 7f24a8e00640 1 -- 172.21.15.12:0/1255653649 shutdown_connections 2024-09-11T00:27:11.309 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:11.307+0000 7f24a8e00640 1 -- 172.21.15.12:0/1255653649 wait complete. 2024-09-11T00:27:11.311 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:11.307+0000 7f24a8e00640 1 Processor -- start 2024-09-11T00:27:11.311 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:11.307+0000 7f24a8e00640 1 -- start start 2024-09-11T00:27:11.311 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:11.307+0000 7f24a8e00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f24a4102790 0x7f24a4198bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:27:11.311 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:11.307+0000 7f24a8e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f24a41038e0 0x7f24a4199200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:27:11.311 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:11.307+0000 7f24a8e00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f24a41042a0 0x7f24a4199e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:27:11.311 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:11.307+0000 7f24a8e00640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f24a410e450 con 0x7f24a4102790 2024-09-11T00:27:11.311 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:11.307+0000 7f24a8e00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f24a410e2d0 con 0x7f24a41038e0 2024-09-11T00:27:11.311 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:11.307+0000 7f24a8e00640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f24a410e5d0 con 0x7f24a41042a0 2024-09-11T00:27:11.312 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:11.307+0000 7f24a3400640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f24a4102790 0x7f24a4198bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:27:11.312 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:11.307+0000 7f24a3400640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f24a4102790 0x7f24a4198bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:34572/0 (socket says 172.21.15.12:34572) 2024-09-11T00:27:11.312 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:11.307+0000 7f24a3400640 1 -- 172.21.15.12:0/3139537213 learned_addr learned my addr 172.21.15.12:0/3139537213 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:27:11.312 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:11.307+0000 7f24a2a00640 1 --2- 172.21.15.12:0/3139537213 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f24a41038e0 0x7f24a4199200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:27:11.312 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:11.307+0000 7f24a3400640 1 -- 172.21.15.12:0/3139537213 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f24a41042a0 msgr2=0x7f24a4199e30 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-11T00:27:11.312 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:11.307+0000 7f24a3400640 1 --2- 172.21.15.12:0/3139537213 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f24a41042a0 0x7f24a4199e30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:27:11.312 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:11.307+0000 7f24a3400640 1 -- 172.21.15.12:0/3139537213 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f24a41038e0 msgr2=0x7f24a4199200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:27:11.312 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:11.307+0000 7f24a3400640 1 --2- 172.21.15.12:0/3139537213 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f24a41038e0 0x7f24a4199200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:27:11.312 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:11.307+0000 7f24a3400640 1 -- 172.21.15.12:0/3139537213 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24a419a6f0 con 0x7f24a4102790 2024-09-11T00:27:11.312 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:11.307+0000 7f24a3400640 1 --2- 172.21.15.12:0/3139537213 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f24a4102790 0x7f24a4198bb0 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f2494031500 tx=0x7f2494003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:27:11.312 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:11.307+0000 7f249be00640 1 -- 172.21.15.12:0/3139537213 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f249403f070 con 0x7f24a4102790 2024-09-11T00:27:11.312 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:11.307+0000 7f249be00640 1 -- 172.21.15.12:0/3139537213 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2494031c50 con 0x7f24a4102790 2024-09-11T00:27:11.313 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:11.307+0000 7f249be00640 1 -- 172.21.15.12:0/3139537213 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2494035030 con 0x7f24a4102790 2024-09-11T00:27:11.313 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:11.309+0000 7f24a8e00640 1 -- 172.21.15.12:0/3139537213 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24a419a980 con 0x7f24a4102790 2024-09-11T00:27:11.313 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:11.309+0000 7f24a8e00640 1 -- 172.21.15.12:0/3139537213 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24a41a0ed0 con 0x7f24a4102790 2024-09-11T00:27:11.314 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:11.309+0000 7f24ad25c740 1 Processor -- start 2024-09-11T00:27:11.315 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:11.309+0000 7f24ad25c740 1 -- start start 2024-09-11T00:27:11.315 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:11.309+0000 7f24ad25c740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561afa6a3970 0x561afa6a5d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:27:11.315 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:11.309+0000 7f24ad25c740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561afa6a62a0 0x561afa6a8690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:27:11.315 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:11.309+0000 7f24ad25c740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561afa6a8ce0 0x561afa6ab1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:27:11.315 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:11.309+0000 7f24ad25c740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x561afa6166b0 con 0x561afa6a62a0 2024-09-11T00:27:11.315 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:11.309+0000 7f24ad25c740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x561afa605ae0 con 0x561afa6a3970 2024-09-11T00:27:11.315 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:11.309+0000 7f24ad25c740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x561afa6098f0 con 0x561afa6a8ce0 2024-09-11T00:27:11.315 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:11.309+0000 7f249be00640 1 -- 172.21.15.12:0/3139537213 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7f249403b460 con 0x7f24a4102790 2024-09-11T00:27:11.315 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:11.309+0000 7f24a2a00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561afa6a8ce0 0x561afa6ab1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:27:11.315 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:11.309+0000 7f24a2a00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561afa6a8ce0 0x561afa6ab1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.12:58902/0 (socket says 172.21.15.12:58902) 2024-09-11T00:27:11.315 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:11.309+0000 7f24a2a00640 1 -- 172.21.15.12:0/3162518367 learned_addr learned my addr 172.21.15.12:0/3162518367 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:27:11.316 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:11.309+0000 7f24a3400640 1 --2- 172.21.15.12:0/3162518367 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561afa6a62a0 0x561afa6a8690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:27:11.317 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:11.309+0000 7f24a3400640 1 -- 172.21.15.12:0/3162518367 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561afa6a8ce0 msgr2=0x561afa6ab1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:27:11.318 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:11.309+0000 7f24a3400640 1 --2- 172.21.15.12:0/3162518367 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561afa6a8ce0 0x561afa6ab1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:27:11.318 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:11.309+0000 7f24a3400640 1 -- 172.21.15.12:0/3162518367 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561afa6a3970 msgr2=0x561afa6a5d60 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-11T00:27:11.318 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:11.309+0000 7f24a3400640 1 --2- 172.21.15.12:0/3162518367 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561afa6a3970 0x561afa6a5d60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:27:11.318 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:11.309+0000 7f24a3400640 1 -- 172.21.15.12:0/3162518367 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561afa5d2780 con 0x561afa6a62a0 2024-09-11T00:27:11.318 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:11.309+0000 7f24a3400640 1 --2- 172.21.15.12:0/3162518367 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561afa6a62a0 0x561afa6a8690 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f24941040a0 tx=0x7f249410cc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=115f158dad83f6df server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:27:11.318 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:11.309+0000 7f2498c00640 1 -- 172.21.15.12:0/3162518367 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2494116020 con 0x561afa6a62a0 2024-09-11T00:27:11.318 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:11.309+0000 7f2498c00640 1 -- 172.21.15.12:0/3162518367 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f249410eb80 con 0x561afa6a62a0 2024-09-11T00:27:11.318 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:11.309+0000 7f2498c00640 1 -- 172.21.15.12:0/3162518367 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2494111620 con 0x561afa6a62a0 2024-09-11T00:27:11.318 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:11.315+0000 7f249be00640 1 --2- 172.21.15.12:0/3139537213 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f24880b5360 0x7f24880b7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:27:11.318 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:11.315+0000 7f24ad25c740 1 -- 172.21.15.12:0/3162518367 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561afa6a62a0 msgr2=0x561afa6a8690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:27:11.318 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:11.315+0000 7f24ad25c740 1 --2- 172.21.15.12:0/3162518367 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561afa6a62a0 0x561afa6a8690 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f24941040a0 tx=0x7f249410cc50 comp rx=0 tx=0).stop 2024-09-11T00:27:11.318 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:11.315+0000 7f249be00640 1 -- 172.21.15.12:0/3139537213 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9208+0+0 (secure 0 0 0) 0x7f2494106a30 con 0x7f24a4102790 2024-09-11T00:27:11.319 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:11.315+0000 7f24ad25c740 1 -- 172.21.15.12:0/3162518367 shutdown_connections 2024-09-11T00:27:11.319 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:11.315+0000 7f24ad25c740 1 --2- 172.21.15.12:0/3162518367 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561afa6a8ce0 0x561afa6ab1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:27:11.319 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:11.315+0000 7f24ad25c740 1 --2- 172.21.15.12:0/3162518367 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561afa6a62a0 0x561afa6a8690 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:27:11.319 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:11.315+0000 7f24ad25c740 1 --2- 172.21.15.12:0/3162518367 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561afa6a3970 0x561afa6a5d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:27:11.319 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:11.315+0000 7f24ad25c740 1 -- 172.21.15.12:0/3162518367 >> 172.21.15.12:0/3162518367 conn(0x561afa69eac0 msgr2=0x561afa69f850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:27:11.319 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:11.315+0000 7f24a3e00640 1 --2- 172.21.15.12:0/3139537213 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f24880b5360 0x7f24880b7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:27:11.319 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:11.315+0000 7f24ad25c740 1 -- 172.21.15.12:0/3162518367 shutdown_connections 2024-09-11T00:27:11.319 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:11.315+0000 7f24ad25c740 1 -- 172.21.15.12:0/3162518367 wait complete. 2024-09-11T00:27:11.319 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:11.315+0000 7f24a3e00640 1 --2- 172.21.15.12:0/3139537213 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f24880b5360 0x7f24880b7820 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f24a4199930 tx=0x7f249c009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:27:11.320 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:11.315+0000 7f24ad25c740 1 -- start start 2024-09-11T00:27:11.320 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:11.315+0000 7f24ad25c740 1 Processor -- start 2024-09-11T00:27:11.324 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:11.315+0000 7f24ad25c740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561afa6a3970 0x561afa6b1200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:27:11.324 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:11.315+0000 7f24ad25c740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561afa6a62a0 0x561afa6b18a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:27:11.324 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:11.315+0000 7f24ad25c740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561afa6a8ce0 0x561afa6b2470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:27:11.324 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:11.315+0000 7f24ad25c740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x561afa53dab0 con 0x561afa6a62a0 2024-09-11T00:27:11.324 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:11.315+0000 7f24ad25c740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x561afa5ac570 con 0x561afa6a3970 2024-09-11T00:27:11.324 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:11.315+0000 7f24ad25c740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x561afa6aba80 con 0x561afa6a8ce0 2024-09-11T00:27:11.324 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:11.315+0000 7f24ad25c740 10 client.0 _ll_register_callbacks cb 0x561afa5bb100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:27:11.324 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:11.315+0000 7f24a3400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561afa6a3970 0x561afa6b1200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:27:11.325 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:11.315+0000 7f24a2a00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561afa6a62a0 0x561afa6b18a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:27:11.325 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:11.315+0000 7f24a3400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561afa6a3970 0x561afa6b1200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.12:54512/0 (socket says 172.21.15.12:54512) 2024-09-11T00:27:11.325 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:11.315+0000 7f24a3400640 1 -- 172.21.15.12:0/1010222387 learned_addr learned my addr 172.21.15.12:0/1010222387 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:27:11.325 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:11.315+0000 7f24a3e00640 1 --2- 172.21.15.12:0/1010222387 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561afa6a8ce0 0x561afa6b2470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:27:11.325 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:11.315+0000 7f24a3400640 1 -- 172.21.15.12:0/1010222387 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561afa6a8ce0 msgr2=0x561afa6b2470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:27:11.325 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:11.315+0000 7f24a3400640 1 --2- 172.21.15.12:0/1010222387 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561afa6a8ce0 0x561afa6b2470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:27:11.325 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:11.315+0000 7f24a3400640 1 -- 172.21.15.12:0/1010222387 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561afa6a62a0 msgr2=0x561afa6b18a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:27:11.325 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:11.315+0000 7f24a3400640 1 --2- 172.21.15.12:0/1010222387 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561afa6a62a0 0x561afa6b18a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:27:11.325 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:11.315+0000 7f24a3400640 1 -- 172.21.15.12:0/1010222387 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561afa367790 con 0x561afa6a3970 2024-09-11T00:27:11.325 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:11.315+0000 7f24a3e00640 1 --2- 172.21.15.12:0/1010222387 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561afa6a8ce0 0x561afa6b2470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T00:27:11.325 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:11.315+0000 7f24a3400640 1 --2- 172.21.15.12:0/1010222387 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561afa6a3970 0x561afa6b1200 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f2494005ec0 tx=0x7f2494005e10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d7c57dc66e765cb2 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:27:11.325 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:11.315+0000 7f248d600640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-09-11T00:27:11.326 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:11.315+0000 7f248d600640 1 -- 172.21.15.12:0/1010222387 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2494049070 con 0x561afa6a3970 2024-09-11T00:27:11.326 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:11.315+0000 7f248d600640 1 -- 172.21.15.12:0/1010222387 <== mon.2 v2:172.21.15.165:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9208+0+0 (secure 0 0 0) 0x7f249410ed30 con 0x561afa6a3970 2024-09-11T00:27:11.326 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:11.315+0000 7f24ad25c740 1 -- 172.21.15.12:0/1010222387 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f249410eb80 con 0x561afa6a3970 2024-09-11T00:27:11.326 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:11.315+0000 7f248d600640 1 -- 172.21.15.12:0/1010222387 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=83) ==== 24+0+0 (secure 0 0 0) 0x7f2494116070 con 0x561afa6a3970 2024-09-11T00:27:11.326 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:11.315+0000 7f24ad25c740 10 client.27671 fetch_fsmap learned FSMap version 83 2024-09-11T00:27:11.326 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:11.315+0000 7f24ad25c740 1 -- 172.21.15.12:0/1010222387 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=83,monmap=4+}) -- 0x561afa3e0620 con 0x561afa6a3970 2024-09-11T00:27:11.326 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:11.315+0000 7f248d600640 1 -- 172.21.15.12:0/1010222387 <== mon.2 v2:172.21.15.165:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f249410eb80 con 0x561afa6a3970 2024-09-11T00:27:11.326 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:11.315+0000 7f248d600640 1 -- 172.21.15.12:0/1010222387 <== mon.2 v2:172.21.15.165:3300/0 5 ==== fsmap(e 83) ==== 8422+0+0 (secure 0 0 0) 0x7f24940c9030 con 0x561afa6a3970 2024-09-11T00:27:11.326 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:11.315+0000 7f24ad25c740 10 client.27671 fetch_fsmap finished waiting for FSMap version 83 2024-09-11T00:27:11.326 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:11.315+0000 7f24ad25c740 10 client.27671 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T00:27:11.326 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:11.315+0000 7f24ad25c740 20 client.27671 populate_metadata read hostname 'smithi012' 2024-09-11T00:27:11.326 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:11.315+0000 7f24ad25c740 1 --2- 172.21.15.12:0/1010222387 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x561afa6bc990 0x561afa6bedd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:27:11.327 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:11.315+0000 7f24ad25c740 4 client.27671 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T00:27:11.327 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:11.315+0000 7f24ad25c740 1 -- 172.21.15.12:0/1010222387 --> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561afa618600 con 0x561afa6bc990 2024-09-11T00:27:11.327 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:11.315+0000 7f24a2a00640 1 --2- 172.21.15.12:0/1010222387 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x561afa6bc990 0x561afa6bedd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:27:11.327 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:11.315+0000 7f24a2a00640 1 --2- 172.21.15.12:0/1010222387 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x561afa6bc990 0x561afa6bedd0 crc :-1 s=READY pgs=279 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=24d7590a58d083fa server_cookie=df0aea3f8f09ebc7 in_seq=0 out_seq=0 2024-09-11T00:27:11.327 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:11.315+0000 7f248d600640 10 client.27671 ms_handle_connect on v2:172.21.15.165:6838/501312984 2024-09-11T00:27:11.327 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:11.320+0000 7f248d600640 1 -- 172.21.15.12:0/1010222387 <== mds.0 v2:172.21.15.165:6838/501312984 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x561afa618600 con 0x561afa6bc990 2024-09-11T00:27:11.327 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:11.320+0000 7f248d600640 10 client.27671 handle_command_reply: tid=0 2024-09-11T00:27:11.327 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:11.320+0000 7f24ad25c740 1 client.27671 shutdown 2024-09-11T00:27:11.327 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:11.320+0000 7f24ad25c740 1 -- 172.21.15.12:0/1010222387 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561afa6a3970 msgr2=0x561afa6b1200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:27:11.328 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:11.320+0000 7f24ad25c740 1 --2- 172.21.15.12:0/1010222387 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561afa6a3970 0x561afa6b1200 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f2494005ec0 tx=0x7f2494005e10 comp rx=0 tx=0).stop 2024-09-11T00:27:11.328 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:11.320+0000 7f24ad25c740 1 -- 172.21.15.12:0/1010222387 shutdown_connections 2024-09-11T00:27:11.328 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:11.320+0000 7f24ad25c740 1 --2- 172.21.15.12:0/1010222387 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x561afa6bc990 0x561afa6bedd0 crc :-1 s=READY pgs=279 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:27:11.328 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:11.320+0000 7f24ad25c740 1 --2- 172.21.15.12:0/1010222387 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561afa6a8ce0 0x561afa6b2470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:27:11.328 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:11.320+0000 7f24ad25c740 1 --2- 172.21.15.12:0/1010222387 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561afa6a62a0 0x561afa6b18a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:27:11.328 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:11.320+0000 7f24ad25c740 1 --2- 172.21.15.12:0/1010222387 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561afa6a3970 0x561afa6b1200 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:27:11.328 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:11.320+0000 7f24ad25c740 1 -- 172.21.15.12:0/1010222387 >> 172.21.15.12:0/1010222387 conn(0x561afa69eac0 msgr2=0x561afa69dbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:27:11.328 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:11.320+0000 7f248d600640 0 client.27671 ms_handle_reset on v2:172.21.15.165:6838/501312984 2024-09-11T00:27:11.328 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:11.320+0000 7f24ad25c740 1 -- 172.21.15.12:0/1010222387 shutdown_connections 2024-09-11T00:27:11.328 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:11.320+0000 7f24ad25c740 1 -- 172.21.15.12:0/1010222387 wait complete. 2024-09-11T00:27:11.328 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:11.320+0000 7f24ad25c740 20 client.27671 trim_cache size 0 max 16384 2024-09-11T00:27:11.338 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:11.333+0000 7f24ad25c740 1 Processor -- start 2024-09-11T00:27:11.339 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:11.333+0000 7f24ad25c740 1 -- start start 2024-09-11T00:27:11.339 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:11.333+0000 7f24ad25c740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561afa6a3970 0x561afa6abdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:27:11.339 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:11.333+0000 7f24ad25c740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561afa6a62a0 0x561afa6ac2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:27:11.339 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:11.333+0000 7f24ad25c740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561afa6a8ce0 0x561afa6a9140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:27:11.339 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:11.333+0000 7f24ad25c740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f24a41a9350 con 0x561afa6a3970 2024-09-11T00:27:11.339 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:11.333+0000 7f24ad25c740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f24a41ab010 con 0x561afa6a62a0 2024-09-11T00:27:11.339 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:11.333+0000 7f24ad25c740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f24a41a9ae0 con 0x561afa6a8ce0 2024-09-11T00:27:11.339 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:11.333+0000 7f24a3400640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561afa6a3970 0x561afa6abdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:27:11.339 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:11.333+0000 7f24a3400640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561afa6a3970 0x561afa6abdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:34580/0 (socket says 172.21.15.12:34580) 2024-09-11T00:27:11.339 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:11.333+0000 7f24a3400640 1 -- 172.21.15.12:0/944342923 learned_addr learned my addr 172.21.15.12:0/944342923 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:27:11.340 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:11.333+0000 7f24a3e00640 1 --2- 172.21.15.12:0/944342923 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561afa6a8ce0 0x561afa6a9140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:27:11.340 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:11.333+0000 7f24a2a00640 1 --2- 172.21.15.12:0/944342923 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561afa6a62a0 0x561afa6ac2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:27:11.340 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:11.333+0000 7f24a3400640 1 -- 172.21.15.12:0/944342923 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561afa6a8ce0 msgr2=0x561afa6a9140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:27:11.340 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:11.333+0000 7f24a3400640 1 --2- 172.21.15.12:0/944342923 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561afa6a8ce0 0x561afa6a9140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:27:11.340 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:11.333+0000 7f24a3400640 1 -- 172.21.15.12:0/944342923 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561afa6a62a0 msgr2=0x561afa6ac2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:27:11.340 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:11.333+0000 7f24a3400640 1 --2- 172.21.15.12:0/944342923 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561afa6a62a0 0x561afa6ac2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:27:11.340 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:11.333+0000 7f24a3400640 1 -- 172.21.15.12:0/944342923 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561afa6a60d0 con 0x561afa6a3970 2024-09-11T00:27:11.340 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:11.333+0000 7f24a3400640 1 --2- 172.21.15.12:0/944342923 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561afa6a3970 0x561afa6abdb0 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f24940c9390 tx=0x7f249402a420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=636d538d2f5cfbfd server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:27:11.340 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:11.333+0000 7f248fe00640 1 -- 172.21.15.12:0/944342923 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2494049070 con 0x561afa6a3970 2024-09-11T00:27:11.340 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:11.333+0000 7f248fe00640 1 -- 172.21.15.12:0/944342923 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2494031e00 con 0x561afa6a3970 2024-09-11T00:27:11.340 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:11.333+0000 7f248fe00640 1 -- 172.21.15.12:0/944342923 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f249407c070 con 0x561afa6a3970 2024-09-11T00:27:11.344 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:11.333+0000 7f24ad25c740 1 -- 172.21.15.12:0/944342923 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561afa6a3970 msgr2=0x561afa6abdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:27:11.344 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:11.333+0000 7f24ad25c740 1 --2- 172.21.15.12:0/944342923 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561afa6a3970 0x561afa6abdb0 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f24940c9390 tx=0x7f249402a420 comp rx=0 tx=0).stop 2024-09-11T00:27:11.344 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:11.333+0000 7f24ad25c740 1 -- 172.21.15.12:0/944342923 shutdown_connections 2024-09-11T00:27:11.344 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:11.333+0000 7f24ad25c740 1 --2- 172.21.15.12:0/944342923 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561afa6a8ce0 0x561afa6a9140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:27:11.344 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:11.333+0000 7f24ad25c740 1 --2- 172.21.15.12:0/944342923 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561afa6a62a0 0x561afa6ac2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:27:11.344 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:11.333+0000 7f24ad25c740 1 --2- 172.21.15.12:0/944342923 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561afa6a3970 0x561afa6abdb0 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:27:11.345 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:11.333+0000 7f24ad25c740 1 -- 172.21.15.12:0/944342923 >> 172.21.15.12:0/944342923 conn(0x561afa69eac0 msgr2=0x561afa6a2e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:27:11.345 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:11.333+0000 7f24ad25c740 1 -- 172.21.15.12:0/944342923 shutdown_connections 2024-09-11T00:27:11.345 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:11.333+0000 7f24ad25c740 1 -- 172.21.15.12:0/944342923 wait complete. 2024-09-11T00:27:11.345 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:11.339+0000 7f24ad25c740 1 -- start start 2024-09-11T00:27:11.345 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:11.339+0000 7f24ad25c740 1 Processor -- start 2024-09-11T00:27:11.345 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:11.339+0000 7f24ad25c740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561afa6a3970 0x561afa6b8880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:27:11.345 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:11.339+0000 7f24ad25c740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561afa6a62a0 0x561afa6b8f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:27:11.345 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:11.339+0000 7f24ad25c740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561afa6a8ce0 0x561afa6b7670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:27:11.345 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:11.339+0000 7f24ad25c740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f24a41aa940 con 0x561afa6a62a0 2024-09-11T00:27:11.345 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:11.339+0000 7f24ad25c740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f24a41aa1b0 con 0x561afa6a8ce0 2024-09-11T00:27:11.345 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:11.339+0000 7f24ad25c740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f24a41a7330 con 0x561afa6a3970 2024-09-11T00:27:11.346 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:11.339+0000 7f24ad25c740 10 client.0 _ll_register_callbacks cb 0x7f249410ca10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:27:11.346 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:11.339+0000 7f24a2a00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561afa6a62a0 0x561afa6b8f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:27:11.346 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:11.339+0000 7f24a3e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561afa6a8ce0 0x561afa6b7670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:27:11.346 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:11.339+0000 7f24a3e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561afa6a8ce0 0x561afa6b7670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.12:54534/0 (socket says 172.21.15.12:54534) 2024-09-11T00:27:11.346 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:11.339+0000 7f24a3e00640 1 -- 172.21.15.12:0/2857159894 learned_addr learned my addr 172.21.15.12:0/2857159894 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:27:11.346 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:11.339+0000 7f24a2a00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561afa6a62a0 0x561afa6b8f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:34596/0 (socket says 172.21.15.12:34596) 2024-09-11T00:27:11.346 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:11.339+0000 7f24a3400640 1 --2- 172.21.15.12:0/2857159894 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561afa6a3970 0x561afa6b8880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:27:11.346 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:11.339+0000 7f24a3e00640 1 -- 172.21.15.12:0/2857159894 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561afa6a3970 msgr2=0x561afa6b8880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:27:11.346 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:11.339+0000 7f24a3e00640 1 --2- 172.21.15.12:0/2857159894 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561afa6a3970 0x561afa6b8880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:27:11.346 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:11.339+0000 7f24a3e00640 1 -- 172.21.15.12:0/2857159894 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561afa6a62a0 msgr2=0x561afa6b8f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:27:11.346 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:11.339+0000 7f24a3e00640 1 --2- 172.21.15.12:0/2857159894 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561afa6a62a0 0x561afa6b8f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:27:11.346 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:11.339+0000 7f24a3e00640 1 -- 172.21.15.12:0/2857159894 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561afa6b7e70 con 0x561afa6a8ce0 2024-09-11T00:27:11.346 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:11.339+0000 7f24a2a00640 1 --2- 172.21.15.12:0/2857159894 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561afa6a62a0 0x561afa6b8f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:27:11.347 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:11.339+0000 7f24a3e00640 1 --2- 172.21.15.12:0/2857159894 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561afa6a8ce0 0x561afa6b7670 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f249c01f850 tx=0x7f249c004ab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1ab343fc739c3987 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:27:11.347 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:11.339+0000 7f248d600640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-09-11T00:27:11.347 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:11.339+0000 7f248d600640 1 -- 172.21.15.12:0/2857159894 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f249c010c80 con 0x561afa6a8ce0 2024-09-11T00:27:11.347 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:11.339+0000 7f248d600640 1 -- 172.21.15.12:0/2857159894 <== mon.2 v2:172.21.15.165:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9208+0+0 (secure 0 0 0) 0x7f249c0273e0 con 0x561afa6a8ce0 2024-09-11T00:27:11.347 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:11.339+0000 7f24ad25c740 1 -- 172.21.15.12:0/2857159894 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x561afa6140b0 con 0x561afa6a8ce0 2024-09-11T00:27:11.347 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:11.339+0000 7f248d600640 1 -- 172.21.15.12:0/2857159894 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=83) ==== 24+0+0 (secure 0 0 0) 0x7f249c010e20 con 0x561afa6a8ce0 2024-09-11T00:27:11.347 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:11.339+0000 7f24ad25c740 10 client.27677 fetch_fsmap learned FSMap version 83 2024-09-11T00:27:11.347 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:11.339+0000 7f24ad25c740 1 -- 172.21.15.12:0/2857159894 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=83,monmap=4+}) -- 0x561afa6b6e00 con 0x561afa6a8ce0 2024-09-11T00:27:11.347 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:11.339+0000 7f248d600640 1 -- 172.21.15.12:0/2857159894 <== mon.2 v2:172.21.15.165:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x561afa6140b0 con 0x561afa6a8ce0 2024-09-11T00:27:11.347 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:11.339+0000 7f248d600640 1 -- 172.21.15.12:0/2857159894 <== mon.2 v2:172.21.15.165:3300/0 5 ==== fsmap(e 83) ==== 8422+0+0 (secure 0 0 0) 0x7f249c026070 con 0x561afa6a8ce0 2024-09-11T00:27:11.347 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:11.339+0000 7f24ad25c740 10 client.27677 fetch_fsmap finished waiting for FSMap version 83 2024-09-11T00:27:11.347 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:11.339+0000 7f24ad25c740 10 client.27677 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T00:27:11.348 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:11.339+0000 7f24ad25c740 20 client.27677 populate_metadata read hostname 'smithi012' 2024-09-11T00:27:11.348 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:11.339+0000 7f24ad25c740 1 --2- 172.21.15.12:0/2857159894 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x561afa72a050 0x561afa72c470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:27:11.348 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:11.339+0000 7f24ad25c740 4 client.27677 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T00:27:11.348 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:11.339+0000 7f24ad25c740 1 -- 172.21.15.12:0/2857159894 --> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] -- command(tid 0: {"prefix": "flush journal"}) -- 0x561afa6114e0 con 0x561afa72a050 2024-09-11T00:27:11.348 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:11.339+0000 7f24a3400640 1 --2- 172.21.15.12:0/2857159894 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x561afa72a050 0x561afa72c470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:27:11.348 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:11.339+0000 7f24a3400640 1 --2- 172.21.15.12:0/2857159894 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x561afa72a050 0x561afa72c470 crc :-1 s=READY pgs=280 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b68d6b3ea2c4c402 server_cookie=330bb86fc0c55809 in_seq=0 out_seq=0 2024-09-11T00:27:11.348 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:11.339+0000 7f248d600640 10 client.27677 ms_handle_connect on v2:172.21.15.165:6838/501312984 2024-09-11T00:27:11.628 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:11.626+0000 7f248d600640 1 -- 172.21.15.12:0/2857159894 <== mds.0 v2:172.21.15.165:6838/501312984 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x561afa6114e0 con 0x561afa72a050 2024-09-11T00:27:11.629 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:11.626+0000 7f248d600640 10 client.27677 handle_command_reply: tid=0 2024-09-11T00:27:11.629 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:11.626+0000 7f24ad25c740 1 client.27677 shutdown 2024-09-11T00:27:11.629 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:11.626+0000 7f24ad25c740 1 -- 172.21.15.12:0/2857159894 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561afa6a8ce0 msgr2=0x561afa6b7670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:27:11.629 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:11.626+0000 7f24ad25c740 1 --2- 172.21.15.12:0/2857159894 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561afa6a8ce0 0x561afa6b7670 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f249c01f850 tx=0x7f249c004ab0 comp rx=0 tx=0).stop 2024-09-11T00:27:11.629 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:11.626+0000 7f24ad25c740 1 -- 172.21.15.12:0/2857159894 shutdown_connections 2024-09-11T00:27:11.629 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:11.626+0000 7f24ad25c740 1 --2- 172.21.15.12:0/2857159894 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x561afa72a050 0x561afa72c470 crc :-1 s=READY pgs=280 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:27:11.629 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:11.626+0000 7f24ad25c740 1 --2- 172.21.15.12:0/2857159894 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561afa6a8ce0 0x561afa6b7670 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:27:11.629 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:11.626+0000 7f24ad25c740 1 --2- 172.21.15.12:0/2857159894 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561afa6a62a0 0x561afa6b8f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:27:11.629 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:11.626+0000 7f24ad25c740 1 --2- 172.21.15.12:0/2857159894 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561afa6a3970 0x561afa6b8880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:27:11.629 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:11.626+0000 7f24ad25c740 1 -- 172.21.15.12:0/2857159894 >> 172.21.15.12:0/2857159894 conn(0x561afa69eac0 msgr2=0x561afa69f560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:27:11.629 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:11.626+0000 7f248d600640 0 client.27677 ms_handle_reset on v2:172.21.15.165:6838/501312984 2024-09-11T00:27:11.629 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:11.626+0000 7f24ad25c740 1 -- 172.21.15.12:0/2857159894 shutdown_connections 2024-09-11T00:27:11.630 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:11.626+0000 7f24ad25c740 1 -- 172.21.15.12:0/2857159894 wait complete. 2024-09-11T00:27:11.630 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:11.626+0000 7f24ad25c740 20 client.27677 trim_cache size 0 max 16384 2024-09-11T00:27:11.630 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:{ 2024-09-11T00:27:11.630 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "message": "", 2024-09-11T00:27:11.630 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "return_code": 0 2024-09-11T00:27:11.630 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:} 2024-09-11T00:27:11.631 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:11.626+0000 7f24a8e00640 1 -- 172.21.15.12:0/3139537213 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f24880b5360 msgr2=0x7f24880b7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:27:11.631 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:11.626+0000 7f24a8e00640 1 --2- 172.21.15.12:0/3139537213 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f24880b5360 0x7f24880b7820 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f24a4199930 tx=0x7f249c009500 comp rx=0 tx=0).stop 2024-09-11T00:27:11.631 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:11.626+0000 7f24a8e00640 1 -- 172.21.15.12:0/3139537213 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f24a4102790 msgr2=0x7f24a4198bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:27:11.631 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:11.626+0000 7f24a8e00640 1 --2- 172.21.15.12:0/3139537213 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f24a4102790 0x7f24a4198bb0 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f2494031500 tx=0x7f2494003c10 comp rx=0 tx=0).stop 2024-09-11T00:27:11.631 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:11.626+0000 7f24a8e00640 1 -- 172.21.15.12:0/3139537213 shutdown_connections 2024-09-11T00:27:11.631 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:11.626+0000 7f24a8e00640 1 --2- 172.21.15.12:0/3139537213 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f24880b5360 0x7f24880b7820 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:27:11.631 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:11.626+0000 7f24a8e00640 1 --2- 172.21.15.12:0/3139537213 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f24a41042a0 0x7f24a4199e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:27:11.631 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:11.626+0000 7f24a8e00640 1 --2- 172.21.15.12:0/3139537213 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f24a41038e0 0x7f24a4199200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:27:11.632 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:11.626+0000 7f24a8e00640 1 --2- 172.21.15.12:0/3139537213 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f24a4102790 0x7f24a4198bb0 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:27:11.632 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:11.626+0000 7f24a8e00640 1 -- 172.21.15.12:0/3139537213 >> 172.21.15.12:0/3139537213 conn(0x7f24a40fe030 msgr2=0x7f24a4106cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:27:11.632 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:11.626+0000 7f24a8e00640 1 -- 172.21.15.12:0/3139537213 shutdown_connections 2024-09-11T00:27:11.632 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:11.626+0000 7f24a8e00640 1 -- 172.21.15.12:0/3139537213 wait complete. 2024-09-11T00:27:12.022 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:27:11 smithi012 ceph-mon[30333]: pgmap v1002: 129 pgs: 129 active+clean; 742 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 11 MiB/s wr, 706 op/s 2024-09-11T00:27:12.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:27:11 smithi165 ceph-mon[35356]: pgmap v1002: 129 pgs: 129 active+clean; 742 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 11 MiB/s wr, 706 op/s 2024-09-11T00:27:12.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:27:11 smithi178 ceph-mon[35919]: pgmap v1002: 129 pgs: 129 active+clean; 742 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 11 MiB/s wr, 706 op/s 2024-09-11T00:27:12.237 INFO:tasks.workunit.client.0.smithi012.stdout: 10 129174 16.81 MB/sec execute 308 sec latency 8405.165 ms 2024-09-11T00:27:13.238 INFO:tasks.workunit.client.0.smithi012.stdout: 10 129174 16.76 MB/sec execute 309 sec latency 9405.220 ms 2024-09-11T00:27:13.979 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:27:13 smithi012 ceph-mon[30333]: pgmap v1003: 129 pgs: 129 active+clean; 747 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 12 MiB/s wr, 615 op/s 2024-09-11T00:27:14.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:27:13 smithi165 ceph-mon[35356]: pgmap v1003: 129 pgs: 129 active+clean; 747 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 12 MiB/s wr, 615 op/s 2024-09-11T00:27:14.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:27:13 smithi178 ceph-mon[35919]: pgmap v1003: 129 pgs: 129 active+clean; 747 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 12 MiB/s wr, 615 op/s 2024-09-11T00:27:14.238 INFO:tasks.workunit.client.0.smithi012.stdout: 10 129175 16.70 MB/sec execute 310 sec latency 10405.274 ms 2024-09-11T00:27:15.238 INFO:tasks.workunit.client.0.smithi012.stdout: 10 129175 16.65 MB/sec execute 311 sec latency 11405.339 ms 2024-09-11T00:27:15.960 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:27:15 smithi012 ceph-mon[30333]: pgmap v1004: 129 pgs: 129 active+clean; 749 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 11 MiB/s wr, 425 op/s 2024-09-11T00:27:16.033 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:27:15 smithi165 ceph-mon[35356]: pgmap v1004: 129 pgs: 129 active+clean; 749 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 11 MiB/s wr, 425 op/s 2024-09-11T00:27:16.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:27:15 smithi178 ceph-mon[35919]: pgmap v1004: 129 pgs: 129 active+clean; 749 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 11 MiB/s wr, 425 op/s 2024-09-11T00:27:16.238 INFO:tasks.workunit.client.0.smithi012.stdout: 10 129175 16.59 MB/sec execute 312 sec latency 12405.409 ms 2024-09-11T00:27:17.238 INFO:tasks.workunit.client.0.smithi012.stdout: 10 129177 16.54 MB/sec execute 313 sec latency 13405.460 ms 2024-09-11T00:27:17.944 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:27:17 smithi012 ceph-mon[30333]: pgmap v1005: 129 pgs: 129 active+clean; 731 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 12 MiB/s wr, 522 op/s 2024-09-11T00:27:18.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:27:17 smithi165 ceph-mon[35356]: pgmap v1005: 129 pgs: 129 active+clean; 731 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 12 MiB/s wr, 522 op/s 2024-09-11T00:27:18.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:27:17 smithi178 ceph-mon[35919]: pgmap v1005: 129 pgs: 129 active+clean; 731 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 12 MiB/s wr, 522 op/s 2024-09-11T00:27:18.238 INFO:tasks.workunit.client.0.smithi012.stdout: 10 129177 16.49 MB/sec execute 314 sec latency 14405.514 ms 2024-09-11T00:27:19.238 INFO:tasks.workunit.client.0.smithi012.stdout: 10 129177 16.44 MB/sec execute 315 sec latency 15190.912 ms 2024-09-11T00:27:19.929 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:27:19 smithi012 ceph-mon[30333]: pgmap v1006: 129 pgs: 129 active+clean; 727 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 364 KiB/s rd, 5.6 MiB/s wr, 281 op/s 2024-09-11T00:27:20.033 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:27:19 smithi165 ceph-mon[35356]: pgmap v1006: 129 pgs: 129 active+clean; 727 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 364 KiB/s rd, 5.6 MiB/s wr, 281 op/s 2024-09-11T00:27:20.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:27:19 smithi178 ceph-mon[35919]: pgmap v1006: 129 pgs: 129 active+clean; 727 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 364 KiB/s rd, 5.6 MiB/s wr, 281 op/s 2024-09-11T00:27:20.238 INFO:tasks.workunit.client.0.smithi012.stdout: 10 129177 16.38 MB/sec execute 316 sec latency 16190.955 ms 2024-09-11T00:27:21.245 INFO:tasks.workunit.client.0.smithi012.stdout: 10 129180 16.33 MB/sec execute 317 sec latency 17191.062 ms 2024-09-11T00:27:21.944 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:27:21 smithi012 ceph-mon[30333]: pgmap v1007: 129 pgs: 129 active+clean; 708 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 371 KiB/s rd, 5.2 MiB/s wr, 291 op/s 2024-09-11T00:27:22.033 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:27:21 smithi165 ceph-mon[35356]: pgmap v1007: 129 pgs: 129 active+clean; 708 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 371 KiB/s rd, 5.2 MiB/s wr, 291 op/s 2024-09-11T00:27:22.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:27:21 smithi178 ceph-mon[35919]: pgmap v1007: 129 pgs: 129 active+clean; 708 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 371 KiB/s rd, 5.2 MiB/s wr, 291 op/s 2024-09-11T00:27:22.238 INFO:tasks.workunit.client.0.smithi012.stdout: 10 129180 16.28 MB/sec execute 318 sec latency 18191.178 ms 2024-09-11T00:27:23.238 INFO:tasks.workunit.client.0.smithi012.stdout: 10 129180 16.23 MB/sec execute 319 sec latency 19191.241 ms 2024-09-11T00:27:23.950 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:27:23 smithi012 ceph-mon[30333]: pgmap v1008: 129 pgs: 129 active+clean; 706 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 398 KiB/s rd, 2.6 MiB/s wr, 214 op/s 2024-09-11T00:27:24.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:27:23 smithi178 ceph-mon[35919]: pgmap v1008: 129 pgs: 129 active+clean; 706 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 398 KiB/s rd, 2.6 MiB/s wr, 214 op/s 2024-09-11T00:27:24.238 INFO:tasks.workunit.client.0.smithi012.stdout: 10 129180 16.18 MB/sec execute 320 sec latency 19885.387 ms 2024-09-11T00:27:24.283 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:27:23 smithi165 ceph-mon[35356]: pgmap v1008: 129 pgs: 129 active+clean; 706 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 398 KiB/s rd, 2.6 MiB/s wr, 214 op/s 2024-09-11T00:27:24.753 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:24.751+0000 7f028b800640 1 -- 172.21.15.12:0/2863798203 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f02841066b0 msgr2=0x7f028410d030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:27:24.754 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:24.751+0000 7f028b800640 1 --2- 172.21.15.12:0/2863798203 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f02841066b0 0x7f028410d030 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f027c009cd0 tx=0x7f027c02a8e0 comp rx=0 tx=0).stop 2024-09-11T00:27:24.754 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:24.751+0000 7f028b800640 1 -- 172.21.15.12:0/2863798203 shutdown_connections 2024-09-11T00:27:24.754 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:24.751+0000 7f028b800640 1 --2- 172.21.15.12:0/2863798203 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f02841066b0 0x7f028410d030 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:27:24.754 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:24.751+0000 7f028b800640 1 --2- 172.21.15.12:0/2863798203 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0284105cf0 0x7f0284106170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:27:24.754 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:24.751+0000 7f028b800640 1 --2- 172.21.15.12:0/2863798203 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0284104ba0 0x7f0284104fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:27:24.754 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:24.751+0000 7f028b800640 1 -- 172.21.15.12:0/2863798203 >> 172.21.15.12:0/2863798203 conn(0x7f0284100460 msgr2=0x7f0284102880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:27:24.754 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:24.752+0000 7f028b800640 1 -- 172.21.15.12:0/2863798203 shutdown_connections 2024-09-11T00:27:24.754 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:24.752+0000 7f028b800640 1 -- 172.21.15.12:0/2863798203 wait complete. 2024-09-11T00:27:24.756 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:24.752+0000 7f028b800640 1 Processor -- start 2024-09-11T00:27:24.756 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:24.752+0000 7f028b800640 1 -- start start 2024-09-11T00:27:24.756 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:24.752+0000 7f028b800640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0284104ba0 0x7f028419aef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:27:24.756 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:24.752+0000 7f028b800640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0284105cf0 0x7f028419b540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:27:24.756 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:24.752+0000 7f028b800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f02841066b0 0x7f028419c170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:27:24.756 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:24.752+0000 7f028b800640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f02841107f0 con 0x7f0284105cf0 2024-09-11T00:27:24.757 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:24.752+0000 7f028b800640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f0284110670 con 0x7f02841066b0 2024-09-11T00:27:24.757 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:24.752+0000 7f028b800640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f0284110970 con 0x7f0284104ba0 2024-09-11T00:27:24.757 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:24.752+0000 7f0283e00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0284105cf0 0x7f028419b540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:27:24.757 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:24.752+0000 7f0283e00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0284105cf0 0x7f028419b540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:34240/0 (socket says 172.21.15.12:34240) 2024-09-11T00:27:24.757 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:24.752+0000 7f0283e00640 1 -- 172.21.15.12:0/3366639380 learned_addr learned my addr 172.21.15.12:0/3366639380 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:27:24.757 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:24.752+0000 7f0288e00640 1 --2- 172.21.15.12:0/3366639380 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0284104ba0 0x7f028419aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:27:24.757 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:24.752+0000 7f0283e00640 1 -- 172.21.15.12:0/3366639380 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0284104ba0 msgr2=0x7f028419aef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:27:24.757 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:24.752+0000 7f0283e00640 1 --2- 172.21.15.12:0/3366639380 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0284104ba0 0x7f028419aef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:27:24.757 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:24.752+0000 7f0283e00640 1 -- 172.21.15.12:0/3366639380 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f02841066b0 msgr2=0x7f028419c170 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-11T00:27:24.757 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:24.752+0000 7f0283e00640 1 --2- 172.21.15.12:0/3366639380 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f02841066b0 0x7f028419c170 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:27:24.757 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:24.752+0000 7f0283e00640 1 -- 172.21.15.12:0/3366639380 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f028419ca30 con 0x7f0284105cf0 2024-09-11T00:27:24.757 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:24.752+0000 7f0283e00640 1 --2- 172.21.15.12:0/3366639380 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0284105cf0 0x7f028419b540 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f027000e9f0 tx=0x7f027000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:27:24.757 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:24.752+0000 7f0281600640 1 -- 172.21.15.12:0/3366639380 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f027000cd80 con 0x7f0284105cf0 2024-09-11T00:27:24.758 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:24.752+0000 7f028b800640 1 -- 172.21.15.12:0/3366639380 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f028419cd20 con 0x7f0284105cf0 2024-09-11T00:27:24.758 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:24.752+0000 7f0281600640 1 -- 172.21.15.12:0/3366639380 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f02700108e0 con 0x7f0284105cf0 2024-09-11T00:27:24.758 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:24.752+0000 7f0281600640 1 -- 172.21.15.12:0/3366639380 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0270018980 con 0x7f0284105cf0 2024-09-11T00:27:24.758 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:24.752+0000 7f028b800640 1 -- 172.21.15.12:0/3366639380 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02841a33f0 con 0x7f0284105cf0 2024-09-11T00:27:24.760 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:24.757+0000 7f028e7bf740 1 Processor -- start 2024-09-11T00:27:24.760 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:24.758+0000 7f028e7bf740 1 -- start start 2024-09-11T00:27:24.760 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:24.758+0000 7f028e7bf740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d72e6af790 0x55d72e6b1b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:27:24.760 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:24.758+0000 7f028e7bf740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d72e6b20c0 0x55d72e6b44b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:27:24.760 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:24.758+0000 7f028e7bf740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d72e6b4b00 0x55d72e6b6fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:27:24.760 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:24.758+0000 7f028e7bf740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55d72e6226b0 con 0x55d72e6b20c0 2024-09-11T00:27:24.760 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:24.758+0000 7f028e7bf740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55d72e611ae0 con 0x55d72e6b4b00 2024-09-11T00:27:24.760 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:24.758+0000 7f028e7bf740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55d72e6158f0 con 0x55d72e6af790 2024-09-11T00:27:24.760 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:24.758+0000 7f0281600640 1 -- 172.21.15.12:0/3366639380 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7f0270010430 con 0x7f0284105cf0 2024-09-11T00:27:24.760 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:24.758+0000 7f0288e00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d72e6b20c0 0x55d72e6b44b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:27:24.760 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:24.758+0000 7f0288e00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d72e6b20c0 0x55d72e6b44b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:34252/0 (socket says 172.21.15.12:34252) 2024-09-11T00:27:24.760 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:24.758+0000 7f0288e00640 1 -- 172.21.15.12:0/2661111660 learned_addr learned my addr 172.21.15.12:0/2661111660 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:27:24.762 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:24.759+0000 7f0283e00640 1 --2- 172.21.15.12:0/2661111660 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d72e6b4b00 0x55d72e6b6fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:27:24.762 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:24.759+0000 7f0288e00640 1 -- 172.21.15.12:0/2661111660 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d72e6af790 msgr2=0x55d72e6b1b80 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-11T00:27:24.762 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:24.759+0000 7f0288e00640 1 --2- 172.21.15.12:0/2661111660 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d72e6af790 0x55d72e6b1b80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:27:24.762 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:24.759+0000 7f0288e00640 1 -- 172.21.15.12:0/2661111660 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d72e6b4b00 msgr2=0x55d72e6b6fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:27:24.762 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:24.759+0000 7f0288e00640 1 --2- 172.21.15.12:0/2661111660 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d72e6b4b00 0x55d72e6b6fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:27:24.762 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:24.759+0000 7f0288e00640 1 -- 172.21.15.12:0/2661111660 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d72e5de780 con 0x55d72e6b20c0 2024-09-11T00:27:24.762 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:24.759+0000 7f0288e00640 1 --2- 172.21.15.12:0/2661111660 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d72e6b20c0 0x55d72e6b44b0 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f0278009cf0 tx=0x7f0278011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4483ccca3fe5dd44 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:27:24.763 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:24.759+0000 7f0276000640 1 -- 172.21.15.12:0/2661111660 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0278004230 con 0x55d72e6b20c0 2024-09-11T00:27:24.763 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:24.759+0000 7f0276000640 1 -- 172.21.15.12:0/2661111660 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f02780043d0 con 0x55d72e6b20c0 2024-09-11T00:27:24.763 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:24.759+0000 7f0276000640 1 -- 172.21.15.12:0/2661111660 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0278019860 con 0x55d72e6b20c0 2024-09-11T00:27:24.763 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:24.759+0000 7f0281600640 1 --2- 172.21.15.12:0/3366639380 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f026c0b52e0 0x7f026c0b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:27:24.763 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:24.759+0000 7f028e7bf740 1 -- 172.21.15.12:0/2661111660 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d72e6b20c0 msgr2=0x55d72e6b44b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:27:24.763 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:24.759+0000 7f028e7bf740 1 --2- 172.21.15.12:0/2661111660 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d72e6b20c0 0x55d72e6b44b0 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f0278009cf0 tx=0x7f0278011810 comp rx=0 tx=0).stop 2024-09-11T00:27:24.763 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:24.759+0000 7f0281600640 1 -- 172.21.15.12:0/3366639380 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9208+0+0 (secure 0 0 0) 0x7f02700e63a0 con 0x7f0284105cf0 2024-09-11T00:27:24.763 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:24.759+0000 7f028e7bf740 1 -- 172.21.15.12:0/2661111660 shutdown_connections 2024-09-11T00:27:24.763 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:24.759+0000 7f028e7bf740 1 --2- 172.21.15.12:0/2661111660 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d72e6b4b00 0x55d72e6b6fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:27:24.763 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:24.759+0000 7f028e7bf740 1 --2- 172.21.15.12:0/2661111660 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d72e6b20c0 0x55d72e6b44b0 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:27:24.763 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:24.759+0000 7f028e7bf740 1 --2- 172.21.15.12:0/2661111660 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d72e6af790 0x55d72e6b1b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:27:24.763 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:24.759+0000 7f028e7bf740 1 -- 172.21.15.12:0/2661111660 >> 172.21.15.12:0/2661111660 conn(0x55d72e6aa8e0 msgr2=0x55d72e6ab670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:27:24.763 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:24.759+0000 7f0289800640 1 --2- 172.21.15.12:0/3366639380 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f026c0b52e0 0x7f026c0b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:27:24.764 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:24.759+0000 7f028e7bf740 1 -- 172.21.15.12:0/2661111660 shutdown_connections 2024-09-11T00:27:24.764 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:24.759+0000 7f028e7bf740 1 -- 172.21.15.12:0/2661111660 wait complete. 2024-09-11T00:27:24.766 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:24.759+0000 7f028e7bf740 1 -- start start 2024-09-11T00:27:24.767 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:24.759+0000 7f0289800640 1 --2- 172.21.15.12:0/3366639380 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f026c0b52e0 0x7f026c0b77a0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f027c006010 tx=0x7f027c005d80 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:27:24.767 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:24.759+0000 7f028e7bf740 1 Processor -- start 2024-09-11T00:27:24.767 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:24.759+0000 7f028e7bf740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d72e6af790 0x55d72e6b8ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:27:24.767 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:24.759+0000 7f028e7bf740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d72e6b20c0 0x55d72e6b9590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:27:24.767 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:24.759+0000 7f028e7bf740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d72e6b4b00 0x55d72e6ba0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:27:24.767 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:24.759+0000 7f028e7bf740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55d72e549ab0 con 0x55d72e6b20c0 2024-09-11T00:27:24.767 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:24.759+0000 7f028e7bf740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55d72e5b8570 con 0x55d72e6b4b00 2024-09-11T00:27:24.767 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:24.759+0000 7f028e7bf740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55d72e6b7ae0 con 0x55d72e6af790 2024-09-11T00:27:24.767 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:24.759+0000 7f028e7bf740 10 client.0 _ll_register_callbacks cb 0x55d72e5c7100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:27:24.767 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:24.759+0000 7f0283e00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d72e6b20c0 0x55d72e6b9590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:27:24.767 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:24.759+0000 7f0288e00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d72e6af790 0x55d72e6b8ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:27:24.768 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:24.759+0000 7f0288e00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d72e6af790 0x55d72e6b8ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.12:33930/0 (socket says 172.21.15.12:33930) 2024-09-11T00:27:24.768 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:24.759+0000 7f0283e00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d72e6b20c0 0x55d72e6b9590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:34264/0 (socket says 172.21.15.12:34264) 2024-09-11T00:27:24.768 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:24.759+0000 7f0288e00640 1 -- 172.21.15.12:0/1444795226 learned_addr learned my addr 172.21.15.12:0/1444795226 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:27:24.768 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:24.759+0000 7f0283e00640 1 -- 172.21.15.12:0/1444795226 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d72e6af790 msgr2=0x55d72e6b8ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:27:24.768 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:24.759+0000 7f0283e00640 1 --2- 172.21.15.12:0/1444795226 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d72e6af790 0x55d72e6b8ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:27:24.768 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:24.759+0000 7f0283e00640 1 -- 172.21.15.12:0/1444795226 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d72e6b4b00 msgr2=0x55d72e6ba0f0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-11T00:27:24.768 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:24.759+0000 7f0283e00640 1 --2- 172.21.15.12:0/1444795226 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d72e6b4b00 0x55d72e6ba0f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:27:24.768 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:24.759+0000 7f0283e00640 1 -- 172.21.15.12:0/1444795226 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d72e373790 con 0x55d72e6b20c0 2024-09-11T00:27:24.768 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:24.759+0000 7f0288e00640 1 --2- 172.21.15.12:0/1444795226 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d72e6af790 0x55d72e6b8ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:27:24.768 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:24.759+0000 7f0283e00640 1 --2- 172.21.15.12:0/1444795226 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d72e6b20c0 0x55d72e6b9590 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f02700e3a70 tx=0x7f02700ac500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72701ffa74c82b8a server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:27:24.768 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:24.759+0000 7f026aa00640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-09-11T00:27:24.768 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:24.759+0000 7f026aa00640 1 -- 172.21.15.12:0/1444795226 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02700ac240 con 0x55d72e6b20c0 2024-09-11T00:27:24.769 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:24.759+0000 7f026aa00640 1 -- 172.21.15.12:0/1444795226 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9208+0+0 (secure 0 0 0) 0x7f02700b5830 con 0x55d72e6b20c0 2024-09-11T00:27:24.769 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:24.759+0000 7f028e7bf740 1 -- 172.21.15.12:0/1444795226 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f02780043d0 con 0x55d72e6b20c0 2024-09-11T00:27:24.769 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:24.759+0000 7f026aa00640 1 -- 172.21.15.12:0/1444795226 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=83) ==== 24+0+0 (secure 0 0 0) 0x7f02700ace30 con 0x55d72e6b20c0 2024-09-11T00:27:24.769 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:24.759+0000 7f028e7bf740 10 client.17682 fetch_fsmap learned FSMap version 83 2024-09-11T00:27:24.769 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:24.759+0000 7f028e7bf740 1 -- 172.21.15.12:0/1444795226 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=83,monmap=4+}) -- 0x55d72e3ec620 con 0x55d72e6b20c0 2024-09-11T00:27:24.769 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:24.764+0000 7f026aa00640 1 -- 172.21.15.12:0/1444795226 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f02780043d0 con 0x55d72e6b20c0 2024-09-11T00:27:24.769 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:24.764+0000 7f026aa00640 1 -- 172.21.15.12:0/1444795226 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 83) ==== 8422+0+0 (secure 0 0 0) 0x7f02700a8080 con 0x55d72e6b20c0 2024-09-11T00:27:24.769 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:24.764+0000 7f028e7bf740 10 client.17682 fetch_fsmap finished waiting for FSMap version 83 2024-09-11T00:27:24.769 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:24.764+0000 7f028e7bf740 10 client.17682 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T00:27:24.769 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:24.764+0000 7f028e7bf740 20 client.17682 populate_metadata read hostname 'smithi012' 2024-09-11T00:27:24.769 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:24.764+0000 7f028e7bf740 1 --2- 172.21.15.12:0/1444795226 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55d72e6c4760 0x55d72e6c6bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:27:24.769 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:24.764+0000 7f028e7bf740 4 client.17682 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T00:27:24.769 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:24.764+0000 7f028e7bf740 1 -- 172.21.15.12:0/1444795226 --> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d72e624600 con 0x55d72e6c4760 2024-09-11T00:27:24.770 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:24.764+0000 7f0288e00640 1 --2- 172.21.15.12:0/1444795226 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55d72e6c4760 0x55d72e6c6bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:27:24.770 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:24.764+0000 7f0288e00640 1 --2- 172.21.15.12:0/1444795226 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55d72e6c4760 0x55d72e6c6bc0 crc :-1 s=READY pgs=281 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2f263774af6d58d6 server_cookie=399f7e07c59944b8 in_seq=0 out_seq=0 2024-09-11T00:27:24.770 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:24.764+0000 7f026aa00640 10 client.17682 ms_handle_connect on v2:172.21.15.165:6838/501312984 2024-09-11T00:27:24.771 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:24.764+0000 7f026aa00640 1 -- 172.21.15.12:0/1444795226 <== mds.0 v2:172.21.15.165:6838/501312984 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55d72e624600 con 0x55d72e6c4760 2024-09-11T00:27:24.771 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:24.764+0000 7f026aa00640 10 client.17682 handle_command_reply: tid=0 2024-09-11T00:27:24.771 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:24.764+0000 7f028e7bf740 1 client.17682 shutdown 2024-09-11T00:27:24.771 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:24.764+0000 7f028e7bf740 1 -- 172.21.15.12:0/1444795226 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d72e6b20c0 msgr2=0x55d72e6b9590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:27:24.771 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:24.764+0000 7f028e7bf740 1 --2- 172.21.15.12:0/1444795226 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d72e6b20c0 0x55d72e6b9590 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f02700e3a70 tx=0x7f02700ac500 comp rx=0 tx=0).stop 2024-09-11T00:27:24.771 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:24.764+0000 7f028e7bf740 1 -- 172.21.15.12:0/1444795226 shutdown_connections 2024-09-11T00:27:24.771 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:24.764+0000 7f028e7bf740 1 --2- 172.21.15.12:0/1444795226 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55d72e6c4760 0x55d72e6c6bc0 crc :-1 s=READY pgs=281 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:27:24.771 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:24.764+0000 7f028e7bf740 1 --2- 172.21.15.12:0/1444795226 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d72e6b4b00 0x55d72e6ba0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:27:24.771 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:24.764+0000 7f028e7bf740 1 --2- 172.21.15.12:0/1444795226 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d72e6b20c0 0x55d72e6b9590 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:27:24.771 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:24.764+0000 7f028e7bf740 1 --2- 172.21.15.12:0/1444795226 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d72e6af790 0x55d72e6b8ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:27:24.772 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:24.764+0000 7f028e7bf740 1 -- 172.21.15.12:0/1444795226 >> 172.21.15.12:0/1444795226 conn(0x55d72e6aa8e0 msgr2=0x55d72e6a9a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:27:24.772 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:24.764+0000 7f026aa00640 0 client.17682 ms_handle_reset on v2:172.21.15.165:6838/501312984 2024-09-11T00:27:24.772 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:24.764+0000 7f028e7bf740 1 -- 172.21.15.12:0/1444795226 shutdown_connections 2024-09-11T00:27:24.772 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:24.764+0000 7f028e7bf740 1 -- 172.21.15.12:0/1444795226 wait complete. 2024-09-11T00:27:24.772 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:24.764+0000 7f028e7bf740 20 client.17682 trim_cache size 0 max 16384 2024-09-11T00:27:24.784 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:24.781+0000 7f028e7bf740 1 Processor -- start 2024-09-11T00:27:24.784 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:24.781+0000 7f028e7bf740 1 -- start start 2024-09-11T00:27:24.784 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:24.781+0000 7f028e7bf740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d72e6af790 0x55d72e6ac710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:27:24.784 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:24.781+0000 7f028e7bf740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d72e6b20c0 0x55d72e6acc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:27:24.784 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:24.781+0000 7f028e7bf740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d72e6b4b00 0x55d72e6ad190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:27:24.785 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:24.781+0000 7f028e7bf740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f02841ad9d0 con 0x55d72e6b20c0 2024-09-11T00:27:24.785 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:24.781+0000 7f028e7bf740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f02841af690 con 0x55d72e6b4b00 2024-09-11T00:27:24.785 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:24.781+0000 7f028e7bf740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f02841ae160 con 0x55d72e6af790 2024-09-11T00:27:24.785 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:24.781+0000 7f0283e00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d72e6b20c0 0x55d72e6acc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:27:24.785 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:24.781+0000 7f0283e00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d72e6b20c0 0x55d72e6acc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:34274/0 (socket says 172.21.15.12:34274) 2024-09-11T00:27:24.785 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:24.781+0000 7f0283e00640 1 -- 172.21.15.12:0/1581332234 learned_addr learned my addr 172.21.15.12:0/1581332234 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:27:24.785 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:24.781+0000 7f0288e00640 1 --2- 172.21.15.12:0/1581332234 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d72e6af790 0x55d72e6ac710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:27:24.785 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:24.781+0000 7f0283e00640 1 -- 172.21.15.12:0/1581332234 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d72e6af790 msgr2=0x55d72e6ac710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:27:24.785 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:24.781+0000 7f0283e00640 1 --2- 172.21.15.12:0/1581332234 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d72e6af790 0x55d72e6ac710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:27:24.785 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:24.781+0000 7f0283e00640 1 -- 172.21.15.12:0/1581332234 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d72e6b4b00 msgr2=0x55d72e6ad190 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-11T00:27:24.785 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:24.781+0000 7f0283e00640 1 --2- 172.21.15.12:0/1581332234 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d72e6b4b00 0x55d72e6ad190 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:27:24.785 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:24.781+0000 7f0283e00640 1 -- 172.21.15.12:0/1581332234 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d72e6b1ef0 con 0x55d72e6b20c0 2024-09-11T00:27:24.785 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:24.781+0000 7f0288e00640 1 --2- 172.21.15.12:0/1581332234 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d72e6af790 0x55d72e6ac710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T00:27:24.786 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:24.781+0000 7f0283e00640 1 --2- 172.21.15.12:0/1581332234 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d72e6b20c0 0x55d72e6acc50 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f02700ac6c0 tx=0x7f02700ac630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6560bd09ceb4317a server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:27:24.788 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:24.781+0000 7f0275600640 1 -- 172.21.15.12:0/1581332234 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02700ac080 con 0x55d72e6b20c0 2024-09-11T00:27:24.788 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:24.781+0000 7f0275600640 1 -- 172.21.15.12:0/1581332234 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f02700ac240 con 0x55d72e6b20c0 2024-09-11T00:27:24.788 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:24.781+0000 7f0275600640 1 -- 172.21.15.12:0/1581332234 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02700ab990 con 0x55d72e6b20c0 2024-09-11T00:27:24.788 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:24.781+0000 7f028e7bf740 1 -- 172.21.15.12:0/1581332234 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d72e6b20c0 msgr2=0x55d72e6acc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:27:24.788 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:24.781+0000 7f028e7bf740 1 --2- 172.21.15.12:0/1581332234 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d72e6b20c0 0x55d72e6acc50 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f02700ac6c0 tx=0x7f02700ac630 comp rx=0 tx=0).stop 2024-09-11T00:27:24.789 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:24.781+0000 7f028e7bf740 1 -- 172.21.15.12:0/1581332234 shutdown_connections 2024-09-11T00:27:24.789 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:24.781+0000 7f028e7bf740 1 --2- 172.21.15.12:0/1581332234 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d72e6b4b00 0x55d72e6ad190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:27:24.789 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:24.781+0000 7f028e7bf740 1 --2- 172.21.15.12:0/1581332234 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d72e6b20c0 0x55d72e6acc50 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:27:24.789 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:24.781+0000 7f028e7bf740 1 --2- 172.21.15.12:0/1581332234 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d72e6af790 0x55d72e6ac710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:27:24.789 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:24.781+0000 7f028e7bf740 1 -- 172.21.15.12:0/1581332234 >> 172.21.15.12:0/1581332234 conn(0x55d72e6aa8e0 msgr2=0x55d72e6b4ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:27:24.789 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:24.781+0000 7f028e7bf740 1 -- 172.21.15.12:0/1581332234 shutdown_connections 2024-09-11T00:27:24.789 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:24.781+0000 7f028e7bf740 1 -- 172.21.15.12:0/1581332234 wait complete. 2024-09-11T00:27:24.789 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:24.781+0000 7f028e7bf740 1 -- start start 2024-09-11T00:27:24.789 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:24.781+0000 7f028e7bf740 1 Processor -- start 2024-09-11T00:27:24.789 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:24.781+0000 7f028e7bf740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d72e6af790 0x55d72e6bfe30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:27:24.789 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:24.781+0000 7f028e7bf740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d72e6b20c0 0x55d72e6c0460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:27:24.789 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:24.781+0000 7f028e7bf740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d72e6b4b00 0x55d72e6be950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:27:24.790 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:24.781+0000 7f028e7bf740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f02841aefc0 con 0x55d72e6b4b00 2024-09-11T00:27:24.790 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:24.781+0000 7f028e7bf740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f02841ae830 con 0x55d72e6b20c0 2024-09-11T00:27:24.790 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:24.781+0000 7f028e7bf740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f02841ab9b0 con 0x55d72e6af790 2024-09-11T00:27:24.790 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:24.781+0000 7f028e7bf740 10 client.0 _ll_register_callbacks cb 0x7f0270009700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:27:24.790 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:24.781+0000 7f0288e00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d72e6af790 0x55d72e6bfe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:27:24.790 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:24.781+0000 7f0288e00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d72e6af790 0x55d72e6bfe30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.12:33950/0 (socket says 172.21.15.12:33950) 2024-09-11T00:27:24.790 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:24.781+0000 7f0288e00640 1 -- 172.21.15.12:0/1446243173 learned_addr learned my addr 172.21.15.12:0/1446243173 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:27:24.790 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:24.781+0000 7f0289800640 1 --2- 172.21.15.12:0/1446243173 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d72e6b4b00 0x55d72e6be950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:27:24.790 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:24.781+0000 7f0283e00640 1 --2- 172.21.15.12:0/1446243173 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d72e6b20c0 0x55d72e6c0460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:27:24.790 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:24.786+0000 7f0288e00640 1 -- 172.21.15.12:0/1446243173 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d72e6b20c0 msgr2=0x55d72e6c0460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:27:24.790 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:24.786+0000 7f0288e00640 1 --2- 172.21.15.12:0/1446243173 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d72e6b20c0 0x55d72e6c0460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:27:24.790 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:24.786+0000 7f0288e00640 1 -- 172.21.15.12:0/1446243173 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d72e6b4b00 msgr2=0x55d72e6be950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:27:24.790 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:24.786+0000 7f0288e00640 1 --2- 172.21.15.12:0/1446243173 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d72e6b4b00 0x55d72e6be950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:27:24.791 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:24.786+0000 7f0288e00640 1 -- 172.21.15.12:0/1446243173 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d72e6bee90 con 0x55d72e6af790 2024-09-11T00:27:24.791 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:24.786+0000 7f0288e00640 1 --2- 172.21.15.12:0/1446243173 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d72e6af790 0x55d72e6bfe30 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f0278009ab0 tx=0x7f02780052c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d22945485e658885 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:27:24.791 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:24.786+0000 7f026aa00640 10 client.0 ms_handle_connect on v2:172.21.15.178:3300/0 2024-09-11T00:27:24.793 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:24.787+0000 7f026aa00640 1 -- 172.21.15.12:0/1446243173 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02780043d0 con 0x55d72e6af790 2024-09-11T00:27:24.793 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:24.787+0000 7f028e7bf740 1 -- 172.21.15.12:0/1446243173 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f02700ac240 con 0x55d72e6af790 2024-09-11T00:27:24.793 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:24.787+0000 7f026aa00640 1 -- 172.21.15.12:0/1446243173 <== mon.1 v2:172.21.15.178:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9208+0+0 (secure 0 0 0) 0x7f0278004660 con 0x55d72e6af790 2024-09-11T00:27:24.793 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:24.787+0000 7f026aa00640 1 -- 172.21.15.12:0/1446243173 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=83) ==== 24+0+0 (secure 0 0 0) 0x7f027803a050 con 0x55d72e6af790 2024-09-11T00:27:24.793 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:24.787+0000 7f028e7bf740 10 client.27664 fetch_fsmap learned FSMap version 83 2024-09-11T00:27:24.793 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:24.787+0000 7f028e7bf740 1 -- 172.21.15.12:0/1446243173 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,fsmap=83,monmap=4+}) -- 0x55d72e731ec0 con 0x55d72e6af790 2024-09-11T00:27:24.793 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:24.787+0000 7f026aa00640 1 -- 172.21.15.12:0/1446243173 <== mon.1 v2:172.21.15.178:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f02700ac240 con 0x55d72e6af790 2024-09-11T00:27:24.793 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:24.787+0000 7f026aa00640 1 -- 172.21.15.12:0/1446243173 <== mon.1 v2:172.21.15.178:3300/0 5 ==== fsmap(e 83) ==== 8422+0+0 (secure 0 0 0) 0x7f02780037f0 con 0x55d72e6af790 2024-09-11T00:27:24.793 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:24.787+0000 7f028e7bf740 10 client.27664 fetch_fsmap finished waiting for FSMap version 83 2024-09-11T00:27:24.793 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:24.787+0000 7f028e7bf740 10 client.27664 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T00:27:24.793 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:24.787+0000 7f028e7bf740 20 client.27664 populate_metadata read hostname 'smithi012' 2024-09-11T00:27:24.794 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:24.787+0000 7f028e7bf740 1 --2- 172.21.15.12:0/1446243173 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55d72e732800 0x55d72e734c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:27:24.794 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:24.787+0000 7f028e7bf740 4 client.27664 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T00:27:24.794 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:24.787+0000 7f028e7bf740 1 -- 172.21.15.12:0/1446243173 --> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55d72e6200b0 con 0x55d72e732800 2024-09-11T00:27:24.794 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:24.787+0000 7f0283e00640 1 --2- 172.21.15.12:0/1446243173 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55d72e732800 0x55d72e734c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:27:24.794 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:24.787+0000 7f0283e00640 1 --2- 172.21.15.12:0/1446243173 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55d72e732800 0x55d72e734c20 crc :-1 s=READY pgs=282 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e52f63e17fd300e8 server_cookie=64d92ffb4a56ff3e in_seq=0 out_seq=0 2024-09-11T00:27:24.794 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:24.787+0000 7f026aa00640 10 client.27664 ms_handle_connect on v2:172.21.15.165:6838/501312984 2024-09-11T00:27:24.861 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:24.859+0000 7f026aa00640 1 -- 172.21.15.12:0/1446243173 <== mds.0 v2:172.21.15.165:6838/501312984 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55d72e6200b0 con 0x55d72e732800 2024-09-11T00:27:24.861 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:24.859+0000 7f026aa00640 10 client.27664 handle_command_reply: tid=0 2024-09-11T00:27:24.861 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:24.859+0000 7f028e7bf740 1 client.27664 shutdown 2024-09-11T00:27:24.862 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:24.859+0000 7f028e7bf740 1 -- 172.21.15.12:0/1446243173 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d72e6af790 msgr2=0x55d72e6bfe30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:27:24.862 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:24.859+0000 7f028e7bf740 1 --2- 172.21.15.12:0/1446243173 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d72e6af790 0x55d72e6bfe30 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f0278009ab0 tx=0x7f02780052c0 comp rx=0 tx=0).stop 2024-09-11T00:27:24.862 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:24.859+0000 7f028e7bf740 1 -- 172.21.15.12:0/1446243173 shutdown_connections 2024-09-11T00:27:24.862 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:24.859+0000 7f028e7bf740 1 --2- 172.21.15.12:0/1446243173 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55d72e732800 0x55d72e734c20 crc :-1 s=READY pgs=282 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:27:24.862 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:24.859+0000 7f028e7bf740 1 --2- 172.21.15.12:0/1446243173 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d72e6b4b00 0x55d72e6be950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:27:24.862 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:24.859+0000 7f026aa00640 0 client.27664 ms_handle_reset on v2:172.21.15.165:6838/501312984 2024-09-11T00:27:24.862 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:24.859+0000 7f028e7bf740 1 --2- 172.21.15.12:0/1446243173 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d72e6b20c0 0x55d72e6c0460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:27:24.862 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:24.859+0000 7f028e7bf740 1 --2- 172.21.15.12:0/1446243173 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d72e6af790 0x55d72e6bfe30 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:27:24.862 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:24.859+0000 7f028e7bf740 1 -- 172.21.15.12:0/1446243173 >> 172.21.15.12:0/1446243173 conn(0x55d72e6aa8e0 msgr2=0x55d72e6af070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:27:24.862 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:24.860+0000 7f028e7bf740 1 -- 172.21.15.12:0/1446243173 shutdown_connections 2024-09-11T00:27:24.862 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:24.860+0000 7f028e7bf740 1 -- 172.21.15.12:0/1446243173 wait complete. 2024-09-11T00:27:24.862 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:24.860+0000 7f028e7bf740 20 client.27664 trim_cache size 0 max 16384 2024-09-11T00:27:24.863 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:{ 2024-09-11T00:27:24.863 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "message": "", 2024-09-11T00:27:24.863 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "return_code": 0 2024-09-11T00:27:24.863 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:} 2024-09-11T00:27:24.864 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:24.860+0000 7f028b800640 1 -- 172.21.15.12:0/3366639380 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f026c0b52e0 msgr2=0x7f026c0b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:27:24.864 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:24.860+0000 7f028b800640 1 --2- 172.21.15.12:0/3366639380 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f026c0b52e0 0x7f026c0b77a0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f027c006010 tx=0x7f027c005d80 comp rx=0 tx=0).stop 2024-09-11T00:27:24.864 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:24.860+0000 7f028b800640 1 -- 172.21.15.12:0/3366639380 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0284105cf0 msgr2=0x7f028419b540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:27:24.864 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:24.860+0000 7f028b800640 1 --2- 172.21.15.12:0/3366639380 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0284105cf0 0x7f028419b540 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f027000e9f0 tx=0x7f027000eeb0 comp rx=0 tx=0).stop 2024-09-11T00:27:24.864 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:24.860+0000 7f028b800640 1 -- 172.21.15.12:0/3366639380 shutdown_connections 2024-09-11T00:27:24.864 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:24.860+0000 7f028b800640 1 --2- 172.21.15.12:0/3366639380 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f026c0b52e0 0x7f026c0b77a0 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:27:24.864 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:24.860+0000 7f028b800640 1 --2- 172.21.15.12:0/3366639380 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f02841066b0 0x7f028419c170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:27:24.864 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:24.860+0000 7f028b800640 1 --2- 172.21.15.12:0/3366639380 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0284105cf0 0x7f028419b540 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:27:24.864 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:24.860+0000 7f028b800640 1 --2- 172.21.15.12:0/3366639380 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0284104ba0 0x7f028419aef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:27:24.865 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:24.860+0000 7f028b800640 1 -- 172.21.15.12:0/3366639380 >> 172.21.15.12:0/3366639380 conn(0x7f0284100460 msgr2=0x7f0284109010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:27:24.865 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:24.860+0000 7f028b800640 1 -- 172.21.15.12:0/3366639380 shutdown_connections 2024-09-11T00:27:24.865 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:24.861+0000 7f028b800640 1 -- 172.21.15.12:0/3366639380 wait complete. 2024-09-11T00:27:25.238 INFO:tasks.workunit.client.0.smithi012.stdout: 10 129180 16.13 MB/sec execute 321 sec latency 20885.453 ms 2024-09-11T00:27:26.005 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:27:25 smithi012 ceph-mon[30333]: pgmap v1009: 129 pgs: 129 active+clean; 705 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 345 KiB/s rd, 1.3 MiB/s wr, 164 op/s 2024-09-11T00:27:26.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:27:25 smithi178 ceph-mon[35919]: pgmap v1009: 129 pgs: 129 active+clean; 705 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 345 KiB/s rd, 1.3 MiB/s wr, 164 op/s 2024-09-11T00:27:26.238 INFO:tasks.workunit.client.0.smithi012.stdout: 10 129180 16.08 MB/sec execute 322 sec latency 21885.512 ms 2024-09-11T00:27:26.283 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:27:25 smithi165 ceph-mon[35356]: pgmap v1009: 129 pgs: 129 active+clean; 705 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 345 KiB/s rd, 1.3 MiB/s wr, 164 op/s 2024-09-11T00:27:26.843 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:27:26 smithi012 ceph-mon[30333]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T00:27:27.239 INFO:tasks.workunit.client.0.smithi012.stdout: 10 129180 16.03 MB/sec execute 323 sec latency 22885.573 ms 2024-09-11T00:27:27.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:27:26 smithi165 ceph-mon[35356]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T00:27:27.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:27:26 smithi178 ceph-mon[35919]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T00:27:27.965 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:27:27 smithi012 ceph-mon[30333]: pgmap v1010: 129 pgs: 129 active+clean; 693 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 384 KiB/s rd, 1.0 MiB/s wr, 185 op/s 2024-09-11T00:27:28.239 INFO:tasks.workunit.client.0.smithi012.stdout: 10 129180 15.98 MB/sec execute 324 sec latency 23885.637 ms 2024-09-11T00:27:28.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:27:27 smithi165 ceph-mon[35356]: pgmap v1010: 129 pgs: 129 active+clean; 693 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 384 KiB/s rd, 1.0 MiB/s wr, 185 op/s 2024-09-11T00:27:28.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:27:27 smithi178 ceph-mon[35919]: pgmap v1010: 129 pgs: 129 active+clean; 693 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 384 KiB/s rd, 1.0 MiB/s wr, 185 op/s 2024-09-11T00:27:29.239 INFO:tasks.workunit.client.0.smithi012.stdout: 10 129180 15.93 MB/sec execute 325 sec latency 24885.700 ms 2024-09-11T00:27:29.929 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:27:29 smithi012 ceph-mon[30333]: pgmap v1011: 129 pgs: 129 active+clean; 693 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 229 KiB/s rd, 189 KiB/s wr, 79 op/s 2024-09-11T00:27:30.239 INFO:tasks.workunit.client.0.smithi012.stdout: 10 129289 15.90 MB/sec execute 326 sec latency 25885.750 ms 2024-09-11T00:27:30.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:27:29 smithi165 ceph-mon[35356]: pgmap v1011: 129 pgs: 129 active+clean; 693 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 229 KiB/s rd, 189 KiB/s wr, 79 op/s 2024-09-11T00:27:30.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:27:29 smithi178 ceph-mon[35919]: pgmap v1011: 129 pgs: 129 active+clean; 693 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 229 KiB/s rd, 189 KiB/s wr, 79 op/s 2024-09-11T00:27:31.239 INFO:tasks.workunit.client.0.smithi012.stdout: 10 129452 15.88 MB/sec execute 327 sec latency 26132.117 ms 2024-09-11T00:27:31.293 INFO:tasks.quiescer.fs.[cephfs]:Total duration (60) is longer than `split_if_longer` (35.0), will split into two consecutive quiesces 2024-09-11T00:27:31.293 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 30.0 2024-09-11T00:27:31.294 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid ff3ad74a-6fcf-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T00:27:31.715 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:27:32.111 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:27:31 smithi012 ceph-mon[30333]: pgmap v1012: 129 pgs: 129 active+clean; 693 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 230 KiB/s rd, 196 KiB/s wr, 77 op/s 2024-09-11T00:27:32.111 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:27:31 smithi012 ceph-mon[30333]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T00:27:32.241 INFO:tasks.workunit.client.0.smithi012.stdout: 10 129506 15.84 MB/sec execute 328 sec latency 20846.105 ms 2024-09-11T00:27:32.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:27:31 smithi165 ceph-mon[35356]: pgmap v1012: 129 pgs: 129 active+clean; 693 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 230 KiB/s rd, 196 KiB/s wr, 77 op/s 2024-09-11T00:27:32.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:27:31 smithi165 ceph-mon[35356]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T00:27:32.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:27:31 smithi178 ceph-mon[35919]: pgmap v1012: 129 pgs: 129 active+clean; 693 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 230 KiB/s rd, 196 KiB/s wr, 77 op/s 2024-09-11T00:27:32.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:27:31 smithi178 ceph-mon[35919]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T00:27:32.380 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:27:32.376+0000 7f7e8b400640 1 -- 172.21.15.12:0/1607909050 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7e841139b0 msgr2=0x7f7e84115da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:27:32.380 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:27:32.376+0000 7f7e8b400640 1 --2- 172.21.15.12:0/1607909050 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7e841139b0 0x7f7e84115da0 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f7e7c009ca0 tx=0x7f7e7c02a8c0 comp rx=0 tx=0).stop 2024-09-11T00:27:32.380 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:27:32.376+0000 7f7e8b400640 1 -- 172.21.15.12:0/1607909050 shutdown_connections 2024-09-11T00:27:32.380 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:27:32.376+0000 7f7e8b400640 1 --2- 172.21.15.12:0/1607909050 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7e841139b0 0x7f7e84115da0 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:27:32.380 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:27:32.376+0000 7f7e8b400640 1 --2- 172.21.15.12:0/1607909050 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7e84077ff0 0x7f7e84113470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:27:32.380 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:27:32.376+0000 7f7e8b400640 1 --2- 172.21.15.12:0/1607909050 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7e840776d0 0x7f7e84077ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:27:32.381 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:27:32.376+0000 7f7e8b400640 1 -- 172.21.15.12:0/1607909050 >> 172.21.15.12:0/1607909050 conn(0x7f7e841009f0 msgr2=0x7f7e84102e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:27:32.381 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:27:32.376+0000 7f7e8b400640 1 -- 172.21.15.12:0/1607909050 shutdown_connections 2024-09-11T00:27:32.381 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:27:32.376+0000 7f7e8b400640 1 -- 172.21.15.12:0/1607909050 wait complete. 2024-09-11T00:27:32.383 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:27:32.376+0000 7f7e8b400640 1 Processor -- start 2024-09-11T00:27:32.383 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:27:32.376+0000 7f7e8b400640 1 -- start start 2024-09-11T00:27:32.383 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:27:32.376+0000 7f7e8b400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7e840776d0 0x7f7e8419f8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:27:32.383 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:27:32.376+0000 7f7e8b400640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7e84077ff0 0x7f7e8419fe30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:27:32.383 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:27:32.376+0000 7f7e8b400640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7e841139b0 0x7f7e841a51d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:27:32.384 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:27:32.376+0000 7f7e8b400640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f7e84119390 con 0x7f7e841139b0 2024-09-11T00:27:32.384 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:27:32.376+0000 7f7e8b400640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f7e84119210 con 0x7f7e840776d0 2024-09-11T00:27:32.384 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:27:32.376+0000 7f7e8b400640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f7e84119510 con 0x7f7e84077ff0 2024-09-11T00:27:32.384 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:27:32.381+0000 7f7e88a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7e840776d0 0x7f7e8419f8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:27:32.384 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:27:32.381+0000 7f7e88a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7e840776d0 0x7f7e8419f8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.12:59520/0 (socket says 172.21.15.12:59520) 2024-09-11T00:27:32.384 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:27:32.381+0000 7f7e88a00640 1 -- 172.21.15.12:0/3117029484 learned_addr learned my addr 172.21.15.12:0/3117029484 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:27:32.385 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:27:32.381+0000 7f7e83e00640 1 --2- 172.21.15.12:0/3117029484 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7e84077ff0 0x7f7e8419fe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:27:32.385 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:27:32.381+0000 7f7e89400640 1 --2- 172.21.15.12:0/3117029484 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7e841139b0 0x7f7e841a51d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:27:32.386 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:27:32.381+0000 7f7e88a00640 1 -- 172.21.15.12:0/3117029484 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7e84077ff0 msgr2=0x7f7e8419fe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:27:32.386 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:27:32.381+0000 7f7e88a00640 1 --2- 172.21.15.12:0/3117029484 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7e84077ff0 0x7f7e8419fe30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:27:32.386 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:27:32.381+0000 7f7e88a00640 1 -- 172.21.15.12:0/3117029484 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7e841139b0 msgr2=0x7f7e841a51d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:27:32.387 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:27:32.381+0000 7f7e88a00640 1 --2- 172.21.15.12:0/3117029484 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7e841139b0 0x7f7e841a51d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:27:32.387 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:27:32.381+0000 7f7e88a00640 1 -- 172.21.15.12:0/3117029484 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e841a59a0 con 0x7f7e840776d0 2024-09-11T00:27:32.388 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:27:32.385+0000 7f7e88a00640 1 --2- 172.21.15.12:0/3117029484 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7e840776d0 0x7f7e8419f8f0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f7e7800ea60 tx=0x7f7e7800ef20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:27:32.388 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:27:32.385+0000 7f7e81600640 1 -- 172.21.15.12:0/3117029484 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e7800cde0 con 0x7f7e840776d0 2024-09-11T00:27:32.388 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:27:32.385+0000 7f7e81600640 1 -- 172.21.15.12:0/3117029484 <== mon.2 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7e780035f0 con 0x7f7e840776d0 2024-09-11T00:27:32.388 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:27:32.385+0000 7f7e8b400640 1 -- 172.21.15.12:0/3117029484 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7e841a5c90 con 0x7f7e840776d0 2024-09-11T00:27:32.389 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:27:32.385+0000 7f7e81600640 1 -- 172.21.15.12:0/3117029484 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e78010690 con 0x7f7e840776d0 2024-09-11T00:27:32.389 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:27:32.385+0000 7f7e8b400640 1 -- 172.21.15.12:0/3117029484 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7e841ac520 con 0x7f7e840776d0 2024-09-11T00:27:32.389 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:27:32.385+0000 7f7e8b400640 1 -- 172.21.15.12:0/3117029484 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7e84077ab0 con 0x7f7e840776d0 2024-09-11T00:27:32.395 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:27:32.391+0000 7f7e81600640 1 -- 172.21.15.12:0/3117029484 <== mon.2 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7f7e78004000 con 0x7f7e840776d0 2024-09-11T00:27:32.395 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:27:32.391+0000 7f7e81600640 1 --2- 172.21.15.12:0/3117029484 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f7e6c0b5360 0x7f7e6c0b7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:27:32.396 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:27:32.391+0000 7f7e81600640 1 -- 172.21.15.12:0/3117029484 <== mon.2 v2:172.21.15.165:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9208+0+0 (secure 0 0 0) 0x7f7e780e66d0 con 0x7f7e840776d0 2024-09-11T00:27:32.396 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:27:32.391+0000 7f7e83e00640 1 --2- 172.21.15.12:0/3117029484 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f7e6c0b5360 0x7f7e6c0b7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:27:32.396 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:27:32.391+0000 7f7e81600640 1 -- 172.21.15.12:0/3117029484 <== mon.2 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f7e780e6bf0 con 0x7f7e840776d0 2024-09-11T00:27:32.398 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:27:32.391+0000 7f7e83e00640 1 --2- 172.21.15.12:0/3117029484 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f7e6c0b5360 0x7f7e6c0b7820 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f7e84115fc0 tx=0x7f7e70005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:27:32.661 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:27:32.660+0000 7f7e8b400640 1 -- 172.21.15.12:0/3117029484 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7e8410eba0 con 0x7f7e840776d0 2024-09-11T00:27:32.662 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:27:32.661+0000 7f7e81600640 1 -- 172.21.15.12:0/3117029484 <== mon.2 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 85 v85) ==== 94+0+11992 (secure 0 0 0) 0x7f7e780ad1c0 con 0x7f7e840776d0 2024-09-11T00:27:32.662 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-11T00:27:32.663 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 85 2024-09-11T00:27:32.663 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":85,"btime":"2024-09-11T00:27:30:840379+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14577,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6835/447171591","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":447171591},{"type":"v1","addr":"172.21.15.12:6835","nonce":447171591}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14589,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/901486040","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":901486040},{"type":"v1","addr":"172.21.15.12:6837","nonce":901486040}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":85,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T00:04:04.739881+0000","modified":"2024-09-11T00:27:30.670104+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24674,"mds_1":24643,"mds_2":24638,"mds_3":24637,"mds_4":24614},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14631":{"gid":14631,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.12:6841/3069919437","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6840","nonce":3069919437},{"type":"v1","addr":"172.21.15.12:6841","nonce":3069919437}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24562":{"gid":24562,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6833/1141636112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":1141636112},{"type":"v1","addr":"172.21.15.178:6833","nonce":1141636112}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24581":{"gid":24581,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6833/3111980970","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6832","nonce":3111980970},{"type":"v1","addr":"172.21.15.165:6833","nonce":3111980970}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24607":{"gid":24607,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6835/4084868287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":4084868287},{"type":"v1","addr":"172.21.15.178:6835","nonce":4084868287}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24614":{"gid":24614,"name":"e","rank":4,"incarnation":26,"state":"up:active","state_seq":21,"addr":"172.21.15.165:6835/2425395274","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":2425395274},{"type":"v1","addr":"172.21.15.165:6835","nonce":2425395274}]},"join_fscid":-1,"export_targets":[0],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24637":{"gid":24637,"name":"i","rank":3,"incarnation":24,"state":"up:active","state_seq":18,"addr":"172.21.15.178:6839/70628097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6838","nonce":70628097},{"type":"v1","addr":"172.21.15.178:6839","nonce":70628097}]},"join_fscid":-1,"export_targets":[0,1,2,4],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24638":{"gid":24638,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":18,"addr":"172.21.15.165:6837/6215088","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6836","nonce":6215088},{"type":"v1","addr":"172.21.15.165:6837","nonce":6215088}]},"join_fscid":-1,"export_targets":[0,1,3,4],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24643":{"gid":24643,"name":"g","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.12:6839/3124093009","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6838","nonce":3124093009},{"type":"v1","addr":"172.21.15.12:6839","nonce":3124093009}]},"join_fscid":-1,"export_targets":[0,2,3],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24674":{"gid":24674,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.165:6839/501312984","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6838","nonce":501312984},{"type":"v1","addr":"172.21.15.165:6839","nonce":501312984}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25672":{"gid":25672,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6837/308601741","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":308601741},{"type":"v1","addr":"172.21.15.178:6837","nonce":308601741}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24674,"qdb_cluster":[24614,24637,24638,24643,24674]},"id":1}]} 2024-09-11T00:27:32.669 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:27:32.667+0000 7f7e8b400640 1 -- 172.21.15.12:0/3117029484 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f7e6c0b5360 msgr2=0x7f7e6c0b7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:27:32.670 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:27:32.667+0000 7f7e8b400640 1 --2- 172.21.15.12:0/3117029484 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f7e6c0b5360 0x7f7e6c0b7820 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f7e84115fc0 tx=0x7f7e70005f90 comp rx=0 tx=0).stop 2024-09-11T00:27:32.670 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:27:32.667+0000 7f7e8b400640 1 -- 172.21.15.12:0/3117029484 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7e840776d0 msgr2=0x7f7e8419f8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:27:32.670 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:27:32.667+0000 7f7e8b400640 1 --2- 172.21.15.12:0/3117029484 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7e840776d0 0x7f7e8419f8f0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f7e7800ea60 tx=0x7f7e7800ef20 comp rx=0 tx=0).stop 2024-09-11T00:27:32.670 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:27:32.667+0000 7f7e8b400640 1 -- 172.21.15.12:0/3117029484 shutdown_connections 2024-09-11T00:27:32.670 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:27:32.667+0000 7f7e8b400640 1 --2- 172.21.15.12:0/3117029484 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f7e6c0b5360 0x7f7e6c0b7820 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:27:32.670 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:27:32.667+0000 7f7e8b400640 1 --2- 172.21.15.12:0/3117029484 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7e841139b0 0x7f7e841a51d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:27:32.670 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:27:32.667+0000 7f7e8b400640 1 --2- 172.21.15.12:0/3117029484 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7e84077ff0 0x7f7e8419fe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:27:32.680 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:27:32.667+0000 7f7e8b400640 1 --2- 172.21.15.12:0/3117029484 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7e840776d0 0x7f7e8419f8f0 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:27:32.680 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:27:32.667+0000 7f7e8b400640 1 -- 172.21.15.12:0/3117029484 >> 172.21.15.12:0/3117029484 conn(0x7f7e841009f0 msgr2=0x7f7e84102de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:27:32.680 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:27:32.667+0000 7f7e8b400640 1 -- 172.21.15.12:0/3117029484 shutdown_connections 2024-09-11T00:27:32.680 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:27:32.667+0000 7f7e8b400640 1 -- 172.21.15.12:0/3117029484 wait complete. 2024-09-11T00:27:33.177 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:27:32 smithi012 ceph-mon[30333]: from='client.? 172.21.15.12:0/3117029484' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T00:27:33.241 INFO:tasks.workunit.client.0.smithi012.stdout: 10 129724 15.83 MB/sec execute 329 sec latency 21846.160 ms 2024-09-11T00:27:33.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:27:32 smithi165 ceph-mon[35356]: from='client.? 172.21.15.12:0/3117029484' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T00:27:33.332 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:27:32 smithi178 ceph-mon[35919]: from='client.? 172.21.15.12:0/3117029484' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T00:27:33.655 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24674 quiesce db / --timeout 90 --expiration 150.0 --await-for 45.0' 2024-09-11T00:27:33.656 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid ff3ad74a-6fcf-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24674 quiesce db / --timeout 90 --expiration 150.0 --await-for 45.0 2024-09-11T00:27:34.013 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:27:33 smithi012 ceph-mon[30333]: pgmap v1013: 129 pgs: 129 active+clean; 694 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 193 KiB/s rd, 197 KiB/s wr, 64 op/s 2024-09-11T00:27:34.241 INFO:tasks.workunit.client.0.smithi012.stdout: 10 130341 15.86 MB/sec execute 330 sec latency 22227.322 ms 2024-09-11T00:27:34.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:27:33 smithi165 ceph-mon[35356]: pgmap v1013: 129 pgs: 129 active+clean; 694 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 193 KiB/s rd, 197 KiB/s wr, 64 op/s 2024-09-11T00:27:34.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:27:33 smithi178 ceph-mon[35919]: pgmap v1013: 129 pgs: 129 active+clean; 694 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 193 KiB/s rd, 197 KiB/s wr, 64 op/s 2024-09-11T00:27:34.335 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:27:35.104 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:27:35.101+0000 7f8578c00640 1 -- 172.21.15.12:0/2523552683 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f857410b340 msgr2=0x7f8574111b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:27:35.104 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:27:35.101+0000 7f8578c00640 1 --2- 172.21.15.12:0/2523552683 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f857410b340 0x7f8574111b40 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f856c009900 tx=0x7f856c02a3f0 comp rx=0 tx=0).stop 2024-09-11T00:27:35.104 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:27:35.101+0000 7f8578c00640 1 -- 172.21.15.12:0/2523552683 shutdown_connections 2024-09-11T00:27:35.105 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:27:35.101+0000 7f8578c00640 1 --2- 172.21.15.12:0/2523552683 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f857410b340 0x7f8574111b40 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:27:35.105 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:27:35.101+0000 7f8578c00640 1 --2- 172.21.15.12:0/2523552683 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f857410a9c0 0x7f857410ae00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:27:35.105 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:27:35.101+0000 7f8578c00640 1 --2- 172.21.15.12:0/2523552683 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8574104fb0 0x7f8574105390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:27:35.105 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:27:35.101+0000 7f8578c00640 1 -- 172.21.15.12:0/2523552683 >> 172.21.15.12:0/2523552683 conn(0x7f8574100a20 msgr2=0x7f8574102e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:27:35.105 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:27:35.102+0000 7f8578c00640 1 -- 172.21.15.12:0/2523552683 shutdown_connections 2024-09-11T00:27:35.105 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:27:35.102+0000 7f8578c00640 1 -- 172.21.15.12:0/2523552683 wait complete. 2024-09-11T00:27:35.107 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:27:35.103+0000 7f8578c00640 1 Processor -- start 2024-09-11T00:27:35.107 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:27:35.103+0000 7f8578c00640 1 -- start start 2024-09-11T00:27:35.107 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:27:35.104+0000 7f8578c00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8574104fb0 0x7f857419b400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:27:35.108 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:27:35.104+0000 7f8578c00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f857410a9c0 0x7f857419b940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:27:35.108 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:27:35.104+0000 7f8578c00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f857410b340 0x7f85741a0ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:27:35.108 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:27:35.104+0000 7f8578c00640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f8574114f70 con 0x7f8574104fb0 2024-09-11T00:27:35.108 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:27:35.104+0000 7f8578c00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f8574114df0 con 0x7f857410b340 2024-09-11T00:27:35.108 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:27:35.104+0000 7f8578c00640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f85741150f0 con 0x7f857410a9c0 2024-09-11T00:27:35.109 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:27:35.104+0000 7f8572800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f857410b340 0x7f85741a0ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:27:35.109 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:27:35.104+0000 7f8572800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f857410b340 0x7f85741a0ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.12:59546/0 (socket says 172.21.15.12:59546) 2024-09-11T00:27:35.109 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:27:35.104+0000 7f8572800640 1 -- 172.21.15.12:0/1675629344 learned_addr learned my addr 172.21.15.12:0/1675629344 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:27:35.109 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:27:35.104+0000 7f8571400640 1 --2- 172.21.15.12:0/1675629344 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f857410a9c0 0x7f857419b940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:27:35.109 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:27:35.105+0000 7f8571e00640 1 --2- 172.21.15.12:0/1675629344 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8574104fb0 0x7f857419b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:27:35.109 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:27:35.105+0000 7f8572800640 1 -- 172.21.15.12:0/1675629344 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f857410a9c0 msgr2=0x7f857419b940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:27:35.109 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:27:35.105+0000 7f8572800640 1 --2- 172.21.15.12:0/1675629344 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f857410a9c0 0x7f857419b940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:27:35.109 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:27:35.105+0000 7f8572800640 1 -- 172.21.15.12:0/1675629344 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8574104fb0 msgr2=0x7f857419b400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:27:35.109 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:27:35.105+0000 7f8572800640 1 --2- 172.21.15.12:0/1675629344 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8574104fb0 0x7f857419b400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:27:35.110 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:27:35.105+0000 7f8572800640 1 -- 172.21.15.12:0/1675629344 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85741a14b0 con 0x7f857410b340 2024-09-11T00:27:35.110 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:27:35.105+0000 7f8571e00640 1 --2- 172.21.15.12:0/1675629344 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8574104fb0 0x7f857419b400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T00:27:35.110 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:27:35.105+0000 7f8572800640 1 --2- 172.21.15.12:0/1675629344 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f857410b340 0x7f85741a0ce0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f856c02a960 tx=0x7f856c003cd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:27:35.110 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:27:35.105+0000 7f8571400640 1 --2- 172.21.15.12:0/1675629344 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f857410a9c0 0x7f857419b940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T00:27:35.110 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:27:35.106+0000 7f856aa00640 1 -- 172.21.15.12:0/1675629344 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f856c03f070 con 0x7f857410b340 2024-09-11T00:27:35.110 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:27:35.107+0000 7f856aa00640 1 -- 172.21.15.12:0/1675629344 <== mon.2 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f856c031b50 con 0x7f857410b340 2024-09-11T00:27:35.110 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:27:35.107+0000 7f856aa00640 1 -- 172.21.15.12:0/1675629344 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f856c035030 con 0x7f857410b340 2024-09-11T00:27:35.110 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:27:35.107+0000 7f8578c00640 1 -- 172.21.15.12:0/1675629344 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85741a1740 con 0x7f857410b340 2024-09-11T00:27:35.111 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:27:35.107+0000 7f8578c00640 1 -- 172.21.15.12:0/1675629344 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f857406cb70 con 0x7f857410b340 2024-09-11T00:27:35.115 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:27:35.112+0000 7f857b5fc740 1 Processor -- start 2024-09-11T00:27:35.115 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:27:35.112+0000 7f856aa00640 1 -- 172.21.15.12:0/1675629344 <== mon.2 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7f856c03b460 con 0x7f857410b340 2024-09-11T00:27:35.115 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:27:35.114+0000 7f857b5fc740 1 -- start start 2024-09-11T00:27:35.116 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:27:35.114+0000 7f857b5fc740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5579425b3180 0x5579425b3560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:27:35.116 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:27:35.114+0000 7f857b5fc740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5579425b3aa0 0x5579425bf4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:27:35.116 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:27:35.114+0000 7f857b5fc740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5579425bfa30 0x5579425c1e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:27:35.116 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:27:35.114+0000 7f857b5fc740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x557942528ca0 con 0x5579425bfa30 2024-09-11T00:27:35.116 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:27:35.114+0000 7f857b5fc740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x5579425180d0 con 0x5579425b3180 2024-09-11T00:27:35.116 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:27:35.114+0000 7f857b5fc740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55794251bee0 con 0x5579425b3aa0 2024-09-11T00:27:35.116 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:27:35.114+0000 7f8571e00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5579425b3aa0 0x5579425bf4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:27:35.117 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:27:35.114+0000 7f8571400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5579425b3180 0x5579425b3560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:27:35.117 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:27:35.115+0000 7f8571400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5579425b3180 0x5579425b3560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.12:59548/0 (socket says 172.21.15.12:59548) 2024-09-11T00:27:35.117 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:27:35.115+0000 7f8571400640 1 -- 172.21.15.12:0/2106780507 learned_addr learned my addr 172.21.15.12:0/2106780507 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:27:35.117 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:27:35.115+0000 7f8571e00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5579425b3aa0 0x5579425bf4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.12:37870/0 (socket says 172.21.15.12:37870) 2024-09-11T00:27:35.117 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:27:35.115+0000 7f856aa00640 1 --2- 172.21.15.12:0/1675629344 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f85580b53a0 0x7f85580b7860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:27:35.117 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:27:35.115+0000 7f8571400640 1 -- 172.21.15.12:0/2106780507 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5579425b3aa0 msgr2=0x5579425bf4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:27:35.117 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:27:35.115+0000 7f8571400640 1 --2- 172.21.15.12:0/2106780507 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5579425b3aa0 0x5579425bf4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:27:35.118 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:27:35.115+0000 7f856aa00640 1 -- 172.21.15.12:0/1675629344 <== mon.2 v2:172.21.15.165:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9208+0+0 (secure 0 0 0) 0x7f856c106b00 con 0x7f857410b340 2024-09-11T00:27:35.118 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:27:35.115+0000 7f8571400640 1 -- 172.21.15.12:0/2106780507 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5579425bfa30 msgr2=0x5579425c1e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:27:35.118 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:27:35.115+0000 7f8571400640 1 --2- 172.21.15.12:0/2106780507 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5579425bfa30 0x5579425c1e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:27:35.118 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:27:35.115+0000 7f8571400640 1 -- 172.21.15.12:0/2106780507 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5579424ca400 con 0x5579425b3180 2024-09-11T00:27:35.118 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:27:35.115+0000 7f8571400640 1 --2- 172.21.15.12:0/2106780507 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5579425b3180 0x5579425b3560 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f855c00bc70 tx=0x7f855c010740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=31224fb142b39382 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:27:35.118 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:27:35.115+0000 7f8563000640 1 -- 172.21.15.12:0/2106780507 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f855c004020 con 0x5579425b3180 2024-09-11T00:27:35.119 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:27:35.116+0000 7f8563000640 1 -- 172.21.15.12:0/2106780507 <== mon.2 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f855c0041c0 con 0x5579425b3180 2024-09-11T00:27:35.119 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:27:35.116+0000 7f8572800640 1 --2- 172.21.15.12:0/1675629344 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f85580b53a0 0x7f85580b7860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:27:35.119 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:27:35.116+0000 7f8563000640 1 -- 172.21.15.12:0/2106780507 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f855c012650 con 0x5579425b3180 2024-09-11T00:27:35.119 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:27:35.117+0000 7f857b5fc740 1 -- 172.21.15.12:0/2106780507 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5579425b3180 msgr2=0x5579425b3560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:27:35.119 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:27:35.117+0000 7f857b5fc740 1 --2- 172.21.15.12:0/2106780507 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5579425b3180 0x5579425b3560 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f855c00bc70 tx=0x7f855c010740 comp rx=0 tx=0).stop 2024-09-11T00:27:35.119 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:27:35.117+0000 7f8572800640 1 --2- 172.21.15.12:0/1675629344 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f85580b53a0 0x7f85580b7860 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f856c0ca100 tx=0x7f856c009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:27:35.119 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:27:35.117+0000 7f857b5fc740 1 -- 172.21.15.12:0/2106780507 shutdown_connections 2024-09-11T00:27:35.119 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:27:35.117+0000 7f857b5fc740 1 --2- 172.21.15.12:0/2106780507 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5579425bfa30 0x5579425c1e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:27:35.119 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:27:35.117+0000 7f857b5fc740 1 --2- 172.21.15.12:0/2106780507 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5579425b3aa0 0x5579425bf4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:27:35.119 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:27:35.117+0000 7f857b5fc740 1 --2- 172.21.15.12:0/2106780507 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5579425b3180 0x5579425b3560 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:27:35.120 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:27:35.117+0000 7f857b5fc740 1 -- 172.21.15.12:0/2106780507 >> 172.21.15.12:0/2106780507 conn(0x5579425b0fb0 msgr2=0x5579425b2150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:27:35.121 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:27:35.118+0000 7f857b5fc740 1 -- 172.21.15.12:0/2106780507 shutdown_connections 2024-09-11T00:27:35.121 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:27:35.118+0000 7f857b5fc740 1 -- 172.21.15.12:0/2106780507 wait complete. 2024-09-11T00:27:35.121 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:27:35.120+0000 7f857b5fc740 1 -- start start 2024-09-11T00:27:35.121 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:27:35.120+0000 7f857b5fc740 1 Processor -- start 2024-09-11T00:27:35.122 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:27:35.121+0000 7f857b5fc740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5579425b3180 0x5579425bf340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:27:35.122 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:27:35.121+0000 7f857b5fc740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5579425b3aa0 0x5579425bc790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:27:35.122 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:27:35.121+0000 7f857b5fc740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5579425bfa30 0x5579425bcdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:27:35.122 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:27:35.121+0000 7f857b5fc740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x5579423797a0 con 0x5579425bfa30 2024-09-11T00:27:35.122 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:27:35.121+0000 7f857b5fc740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x5579424bda10 con 0x5579425b3aa0 2024-09-11T00:27:35.123 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:27:35.121+0000 7f857b5fc740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5579425c26b0 con 0x5579425b3180 2024-09-11T00:27:35.123 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:27:35.121+0000 7f857b5fc740 10 client.0 _ll_register_callbacks cb 0x5579424cdbe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:27:35.123 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:27:35.121+0000 7f8571e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5579425b3aa0 0x5579425bc790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:27:35.124 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:27:35.121+0000 7f8571e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5579425b3aa0 0x5579425bc790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.12:59554/0 (socket says 172.21.15.12:59554) 2024-09-11T00:27:35.124 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:27:35.121+0000 7f8571e00640 1 -- 172.21.15.12:0/541849906 learned_addr learned my addr 172.21.15.12:0/541849906 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:27:35.124 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:27:35.121+0000 7f8571400640 1 --2- 172.21.15.12:0/541849906 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5579425b3180 0x5579425bf340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:27:35.124 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:27:35.121+0000 7f8571400640 1 --2- 172.21.15.12:0/541849906 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5579425bfa30 0x5579425bcdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:27:35.124 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:27:35.122+0000 7f8571e00640 1 -- 172.21.15.12:0/541849906 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5579425b3180 msgr2=0x5579425bf340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:27:35.124 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:27:35.122+0000 7f8571e00640 1 --2- 172.21.15.12:0/541849906 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5579425b3180 0x5579425bf340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:27:35.124 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:27:35.122+0000 7f8571e00640 1 -- 172.21.15.12:0/541849906 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5579425bfa30 msgr2=0x5579425bcdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:27:35.124 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:27:35.122+0000 7f8571e00640 1 --2- 172.21.15.12:0/541849906 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5579425bfa30 0x5579425bcdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:27:35.124 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:27:35.122+0000 7f8571e00640 1 -- 172.21.15.12:0/541849906 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557942285b90 con 0x5579425b3aa0 2024-09-11T00:27:35.125 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:27:35.122+0000 7f8571e00640 1 --2- 172.21.15.12:0/541849906 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5579425b3aa0 0x5579425bc790 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f8564013d30 tx=0x7f85640117b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=13e7017e768a8022 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:27:35.125 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:27:35.122+0000 7f8557400640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-09-11T00:27:35.125 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:27:35.123+0000 7f8557400640 1 -- 172.21.15.12:0/541849906 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f856401c640 con 0x5579425b3aa0 2024-09-11T00:27:35.125 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:27:35.123+0000 7f857b5fc740 1 -- 172.21.15.12:0/541849906 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f855c0041c0 con 0x5579425b3aa0 2024-09-11T00:27:35.125 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:27:35.123+0000 7f8557400640 1 -- 172.21.15.12:0/541849906 <== mon.2 v2:172.21.15.165:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9208+0+0 (secure 0 0 0) 0x7f856401b3e0 con 0x5579425b3aa0 2024-09-11T00:27:35.126 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:27:35.124+0000 7f8557400640 1 -- 172.21.15.12:0/541849906 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=85) ==== 24+0+0 (secure 0 0 0) 0x7f856401c7e0 con 0x5579425b3aa0 2024-09-11T00:27:35.126 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:27:35.124+0000 7f857b5fc740 10 client.27704 fetch_fsmap learned FSMap version 85 2024-09-11T00:27:35.126 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:27:35.124+0000 7f857b5fc740 1 -- 172.21.15.12:0/541849906 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=85,monmap=4+}) -- 0x5579425ce0c0 con 0x5579425b3aa0 2024-09-11T00:27:35.126 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:27:35.124+0000 7f8557400640 1 -- 172.21.15.12:0/541849906 <== mon.2 v2:172.21.15.165:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f855c0041c0 con 0x5579425b3aa0 2024-09-11T00:27:35.127 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:27:35.125+0000 7f8557400640 1 -- 172.21.15.12:0/541849906 <== mon.2 v2:172.21.15.165:3300/0 5 ==== fsmap(e 85) ==== 8434+0+0 (secure 0 0 0) 0x7f856401a070 con 0x5579425b3aa0 2024-09-11T00:27:35.127 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:27:35.125+0000 7f857b5fc740 10 client.27704 fetch_fsmap finished waiting for FSMap version 85 2024-09-11T00:27:35.127 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:27:35.125+0000 7f857b5fc740 10 client.27704 resolve_mds: validated gid 24674 aka daemon mds.b 2024-09-11T00:27:35.127 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:27:35.126+0000 7f857b5fc740 20 client.27704 populate_metadata read hostname 'smithi012' 2024-09-11T00:27:35.127 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:27:35.126+0000 7f857b5fc740 1 --2- 172.21.15.12:0/541849906 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x5579425cf640 0x5579425d1a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:27:35.127 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:27:35.126+0000 7f857b5fc740 4 client.27704 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T00:27:35.127 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:27:35.126+0000 7f857b5fc740 1 -- 172.21.15.12:0/541849906 --> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55794252abf0 con 0x5579425cf640 2024-09-11T00:27:35.128 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:27:35.126+0000 7f8571400640 1 --2- 172.21.15.12:0/541849906 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x5579425cf640 0x5579425d1a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:27:35.128 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:27:35.127+0000 7f8571400640 1 --2- 172.21.15.12:0/541849906 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x5579425cf640 0x5579425d1a60 crc :-1 s=READY pgs=283 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c6579e5f2b61c066 server_cookie=d2c21a8abfb435db in_seq=0 out_seq=0 2024-09-11T00:27:35.129 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:27:35.127+0000 7f8557400640 10 client.27704 ms_handle_connect on v2:172.21.15.165:6838/501312984 2024-09-11T00:27:35.130 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:27:35.128+0000 7f8557400640 1 -- 172.21.15.12:0/541849906 <== mds.0 v2:172.21.15.165:6838/501312984 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55794252abf0 con 0x5579425cf640 2024-09-11T00:27:35.130 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:27:35.128+0000 7f8557400640 10 client.27704 handle_command_reply: tid=0 2024-09-11T00:27:35.130 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:27:35.128+0000 7f857b5fc740 1 client.27704 shutdown 2024-09-11T00:27:35.131 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:27:35.130+0000 7f857b5fc740 1 -- 172.21.15.12:0/541849906 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5579425b3aa0 msgr2=0x5579425bc790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:27:35.131 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:27:35.130+0000 7f857b5fc740 1 --2- 172.21.15.12:0/541849906 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5579425b3aa0 0x5579425bc790 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f8564013d30 tx=0x7f85640117b0 comp rx=0 tx=0).stop 2024-09-11T00:27:35.132 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:27:35.130+0000 7f857b5fc740 1 -- 172.21.15.12:0/541849906 shutdown_connections 2024-09-11T00:27:35.132 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:27:35.130+0000 7f857b5fc740 1 --2- 172.21.15.12:0/541849906 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x5579425cf640 0x5579425d1a60 crc :-1 s=READY pgs=283 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:27:35.132 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:27:35.130+0000 7f857b5fc740 1 --2- 172.21.15.12:0/541849906 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5579425bfa30 0x5579425bcdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:27:35.132 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:27:35.130+0000 7f857b5fc740 1 --2- 172.21.15.12:0/541849906 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5579425b3aa0 0x5579425bc790 secure :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f8564013d30 tx=0x7f85640117b0 comp rx=0 tx=0).stop 2024-09-11T00:27:35.132 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:27:35.130+0000 7f857b5fc740 1 --2- 172.21.15.12:0/541849906 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5579425b3180 0x5579425bf340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:27:35.132 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:27:35.130+0000 7f857b5fc740 1 -- 172.21.15.12:0/541849906 >> 172.21.15.12:0/541849906 conn(0x5579425b0fb0 msgr2=0x5579425b6b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:27:35.132 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:27:35.130+0000 7f8557400640 0 client.27704 ms_handle_reset on v2:172.21.15.165:6838/501312984 2024-09-11T00:27:35.132 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:27:35.130+0000 7f857b5fc740 1 -- 172.21.15.12:0/541849906 shutdown_connections 2024-09-11T00:27:35.133 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:27:35.131+0000 7f857b5fc740 1 -- 172.21.15.12:0/541849906 wait complete. 2024-09-11T00:27:35.133 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:27:35.131+0000 7f857b5fc740 20 client.27704 trim_cache size 0 max 16384 2024-09-11T00:27:35.170 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:27:35.167+0000 7f857b5fc740 1 Processor -- start 2024-09-11T00:27:35.170 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:27:35.168+0000 7f857b5fc740 1 -- start start 2024-09-11T00:27:35.170 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:27:35.168+0000 7f857b5fc740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5579425b3180 0x5579425b08a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:27:35.171 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:27:35.168+0000 7f857b5fc740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5579425bfa30 0x5579425c0e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:27:35.171 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:27:35.168+0000 7f857b5fc740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5579425b80b0 0x5579425c1360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:27:35.171 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:27:35.168+0000 7f857b5fc740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f85741ba5b0 con 0x5579425b80b0 2024-09-11T00:27:35.171 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:27:35.168+0000 7f857b5fc740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f85741bc300 con 0x5579425bfa30 2024-09-11T00:27:35.171 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:27:35.168+0000 7f857b5fc740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f85741bad40 con 0x5579425b3180 2024-09-11T00:27:35.174 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:27:35.169+0000 7f8571400640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5579425b3180 0x5579425b08a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:27:35.174 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:27:35.169+0000 7f8571400640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5579425b3180 0x5579425b08a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.12:37890/0 (socket says 172.21.15.12:37890) 2024-09-11T00:27:35.174 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:27:35.169+0000 7f8571400640 1 -- 172.21.15.12:0/2824070181 learned_addr learned my addr 172.21.15.12:0/2824070181 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:27:35.175 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:27:35.169+0000 7f8571400640 1 --2- 172.21.15.12:0/2824070181 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5579425b80b0 0x5579425c1360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:27:35.175 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:27:35.169+0000 7f8571e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5579425bfa30 0x5579425c0e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:27:35.175 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:27:35.170+0000 7f8571e00640 1 -- 172.21.15.12:0/2824070181 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5579425b3180 msgr2=0x5579425b08a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:27:35.175 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:27:35.170+0000 7f8571e00640 1 --2- 172.21.15.12:0/2824070181 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5579425b3180 0x5579425b08a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:27:35.175 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:27:35.170+0000 7f8571e00640 1 -- 172.21.15.12:0/2824070181 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5579425b80b0 msgr2=0x5579425c1360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:27:35.175 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:27:35.170+0000 7f8571e00640 1 --2- 172.21.15.12:0/2824070181 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5579425b80b0 0x5579425c1360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:27:35.176 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:27:35.170+0000 7f8571e00640 1 -- 172.21.15.12:0/2824070181 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5579425b0de0 con 0x5579425bfa30 2024-09-11T00:27:35.176 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:27:35.170+0000 7f8571e00640 1 --2- 172.21.15.12:0/2824070181 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5579425bfa30 0x5579425c0e20 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f856400c000 tx=0x7f8564009b40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7514ea610bfd2361 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:27:35.176 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:27:35.170+0000 7f8562600640 1 -- 172.21.15.12:0/2824070181 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f856402e020 con 0x5579425bfa30 2024-09-11T00:27:35.176 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:27:35.171+0000 7f8562600640 1 -- 172.21.15.12:0/2824070181 <== mon.2 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f856401c980 con 0x5579425bfa30 2024-09-11T00:27:35.176 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:27:35.171+0000 7f8562600640 1 -- 172.21.15.12:0/2824070181 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f856401b640 con 0x5579425bfa30 2024-09-11T00:27:35.176 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:27:35.172+0000 7f857b5fc740 1 -- 172.21.15.12:0/2824070181 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5579425bfa30 msgr2=0x5579425c0e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:27:35.176 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:27:35.172+0000 7f857b5fc740 1 --2- 172.21.15.12:0/2824070181 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5579425bfa30 0x5579425c0e20 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f856400c000 tx=0x7f8564009b40 comp rx=0 tx=0).stop 2024-09-11T00:27:35.176 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:27:35.172+0000 7f857b5fc740 1 -- 172.21.15.12:0/2824070181 shutdown_connections 2024-09-11T00:27:35.176 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:27:35.172+0000 7f857b5fc740 1 --2- 172.21.15.12:0/2824070181 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5579425b80b0 0x5579425c1360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:27:35.177 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:27:35.172+0000 7f857b5fc740 1 --2- 172.21.15.12:0/2824070181 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5579425bfa30 0x5579425c0e20 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:27:35.177 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:27:35.172+0000 7f857b5fc740 1 --2- 172.21.15.12:0/2824070181 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5579425b3180 0x5579425b08a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:27:35.177 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:27:35.172+0000 7f857b5fc740 1 -- 172.21.15.12:0/2824070181 >> 172.21.15.12:0/2824070181 conn(0x5579425b0fb0 msgr2=0x5579425b79a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:27:35.177 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:27:35.172+0000 7f857b5fc740 1 -- 172.21.15.12:0/2824070181 shutdown_connections 2024-09-11T00:27:35.178 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:27:35.173+0000 7f857b5fc740 1 -- 172.21.15.12:0/2824070181 wait complete. 2024-09-11T00:27:35.178 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:27:35.173+0000 7f857b5fc740 1 -- start start 2024-09-11T00:27:35.178 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:27:35.174+0000 7f857b5fc740 1 Processor -- start 2024-09-11T00:27:35.178 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:27:35.175+0000 7f857b5fc740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5579425b3180 0x5579425c8010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:27:35.178 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:27:35.175+0000 7f857b5fc740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5579425b80b0 0x5579425c2900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:27:35.178 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:27:35.175+0000 7f857b5fc740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5579425bfa30 0x5579425c2e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:27:35.178 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:27:35.175+0000 7f857b5fc740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f85741bbc30 con 0x5579425bfa30 2024-09-11T00:27:35.178 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:27:35.175+0000 7f857b5fc740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f85741bb4a0 con 0x5579425b3180 2024-09-11T00:27:35.178 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:27:35.175+0000 7f857b5fc740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f8574074a90 con 0x5579425b80b0 2024-09-11T00:27:35.178 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:27:35.175+0000 7f857b5fc740 10 client.0 _ll_register_callbacks cb 0x7f8564009bc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:27:35.178 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:27:35.176+0000 7f8571400640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5579425bfa30 0x5579425c2e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:27:35.179 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:27:35.176+0000 7f8571400640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5579425bfa30 0x5579425c2e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:33660/0 (socket says 172.21.15.12:33660) 2024-09-11T00:27:35.179 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:27:35.177+0000 7f8571400640 1 -- 172.21.15.12:0/2037160845 learned_addr learned my addr 172.21.15.12:0/2037160845 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:27:35.179 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:27:35.177+0000 7f8571400640 1 --2- 172.21.15.12:0/2037160845 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5579425b3180 0x5579425c8010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:27:35.179 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:27:35.177+0000 7f8571e00640 1 --2- 172.21.15.12:0/2037160845 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5579425b80b0 0x5579425c2900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:27:35.179 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:27:35.178+0000 7f8571e00640 1 -- 172.21.15.12:0/2037160845 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5579425b3180 msgr2=0x5579425c8010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:27:35.180 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:27:35.178+0000 7f8571e00640 1 --2- 172.21.15.12:0/2037160845 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5579425b3180 0x5579425c8010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:27:35.180 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:27:35.178+0000 7f8571e00640 1 -- 172.21.15.12:0/2037160845 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5579425bfa30 msgr2=0x5579425c2e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:27:35.180 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:27:35.178+0000 7f8571e00640 1 --2- 172.21.15.12:0/2037160845 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5579425bfa30 0x5579425c2e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:27:35.180 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:27:35.178+0000 7f8571e00640 1 -- 172.21.15.12:0/2037160845 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5579425c35b0 con 0x5579425b80b0 2024-09-11T00:27:35.180 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:27:35.179+0000 7f8571e00640 1 --2- 172.21.15.12:0/2037160845 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5579425b80b0 0x5579425c2900 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f856401bf10 tx=0x7f8564011df0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c1e7b1151c9a4675 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:27:35.181 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:27:35.179+0000 7f8557400640 10 client.0 ms_handle_connect on v2:172.21.15.178:3300/0 2024-09-11T00:27:35.181 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:27:35.179+0000 7f8557400640 1 -- 172.21.15.12:0/2037160845 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f856402e070 con 0x5579425b80b0 2024-09-11T00:27:35.181 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:27:35.179+0000 7f8557400640 1 -- 172.21.15.12:0/2037160845 <== mon.1 v2:172.21.15.178:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9208+0+0 (secure 0 0 0) 0x7f8564003970 con 0x5579425b80b0 2024-09-11T00:27:35.181 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:27:35.180+0000 7f857b5fc740 1 -- 172.21.15.12:0/2037160845 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f856401c980 con 0x5579425b80b0 2024-09-11T00:27:35.181 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:27:35.180+0000 7f8557400640 1 -- 172.21.15.12:0/2037160845 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=85) ==== 24+0+0 (secure 0 0 0) 0x7f8564003bd0 con 0x5579425b80b0 2024-09-11T00:27:35.181 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:27:35.180+0000 7f857b5fc740 10 client.27685 fetch_fsmap learned FSMap version 85 2024-09-11T00:27:35.181 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:27:35.180+0000 7f857b5fc740 1 -- 172.21.15.12:0/2037160845 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,fsmap=85,monmap=4+}) -- 0x557942645ca0 con 0x5579425b80b0 2024-09-11T00:27:35.182 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:27:35.180+0000 7f8557400640 1 -- 172.21.15.12:0/2037160845 <== mon.1 v2:172.21.15.178:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f856401c980 con 0x5579425b80b0 2024-09-11T00:27:35.182 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:27:35.181+0000 7f8557400640 1 -- 172.21.15.12:0/2037160845 <== mon.1 v2:172.21.15.178:3300/0 5 ==== fsmap(e 85) ==== 8434+0+0 (secure 0 0 0) 0x7f856401a070 con 0x5579425b80b0 2024-09-11T00:27:35.182 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:27:35.181+0000 7f857b5fc740 10 client.27685 fetch_fsmap finished waiting for FSMap version 85 2024-09-11T00:27:35.182 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:27:35.181+0000 7f857b5fc740 10 client.27685 resolve_mds: validated gid 24674 aka daemon mds.b 2024-09-11T00:27:35.183 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:27:35.181+0000 7f857b5fc740 20 client.27685 populate_metadata read hostname 'smithi012' 2024-09-11T00:27:35.183 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:27:35.181+0000 7f857b5fc740 1 --2- 172.21.15.12:0/2037160845 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x557942646460 0x557942648880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:27:35.184 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:27:35.181+0000 7f857b5fc740 4 client.27685 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 150.0, "await_for": 45.0}] 2024-09-11T00:27:35.184 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:27:35.181+0000 7f857b5fc740 1 -- 172.21.15.12:0/2037160845 --> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 150.0, "await_for": 45.0}) -- 0x5579425b2ba0 con 0x557942646460 2024-09-11T00:27:35.184 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:27:35.182+0000 7f8571400640 1 --2- 172.21.15.12:0/2037160845 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x557942646460 0x557942648880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:27:35.185 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:27:35.184+0000 7f8571400640 1 --2- 172.21.15.12:0/2037160845 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x557942646460 0x557942648880 crc :-1 s=READY pgs=284 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4bacb73c41b1ff90 server_cookie=ec91461238dd3bff in_seq=0 out_seq=0 2024-09-11T00:27:35.185 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:27:35.184+0000 7f8557400640 10 client.27685 ms_handle_connect on v2:172.21.15.165:6838/501312984 2024-09-11T00:27:35.241 INFO:tasks.workunit.client.0.smithi012.stdout: 10 131011 15.93 MB/sec execute 331 sec latency 71.129 ms 2024-09-11T00:27:35.819 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:27:35.817+0000 7f8557400640 1 -- 172.21.15.12:0/2037160845 <== mds.0 v2:172.21.15.165:6838/501312984 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x5579425b2ba0 con 0x557942646460 2024-09-11T00:27:35.819 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:27:35.817+0000 7f8557400640 10 client.27685 handle_command_reply: tid=0 2024-09-11T00:27:35.819 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:27:35.817+0000 7f857b5fc740 1 client.27685 shutdown 2024-09-11T00:27:35.819 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:27:35.817+0000 7f857b5fc740 1 -- 172.21.15.12:0/2037160845 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5579425b80b0 msgr2=0x5579425c2900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:27:35.819 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:27:35.817+0000 7f857b5fc740 1 --2- 172.21.15.12:0/2037160845 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5579425b80b0 0x5579425c2900 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f856401bf10 tx=0x7f8564011df0 comp rx=0 tx=0).stop 2024-09-11T00:27:35.819 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:27:35.817+0000 7f857b5fc740 1 -- 172.21.15.12:0/2037160845 shutdown_connections 2024-09-11T00:27:35.819 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:27:35.817+0000 7f857b5fc740 1 --2- 172.21.15.12:0/2037160845 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x557942646460 0x557942648880 crc :-1 s=READY pgs=284 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:27:35.819 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:27:35.817+0000 7f857b5fc740 1 --2- 172.21.15.12:0/2037160845 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5579425bfa30 0x5579425c2e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:27:35.819 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:27:35.817+0000 7f8557400640 0 client.27685 ms_handle_reset on v2:172.21.15.165:6838/501312984 2024-09-11T00:27:35.819 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:27:35.817+0000 7f857b5fc740 1 --2- 172.21.15.12:0/2037160845 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5579425b80b0 0x5579425c2900 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:27:35.820 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:27:35.817+0000 7f857b5fc740 1 --2- 172.21.15.12:0/2037160845 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5579425b3180 0x5579425c8010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:27:35.820 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:27:35.817+0000 7f857b5fc740 1 -- 172.21.15.12:0/2037160845 >> 172.21.15.12:0/2037160845 conn(0x5579425b0fb0 msgr2=0x5579425bb010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:27:35.820 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:27:35.817+0000 7f857b5fc740 1 -- 172.21.15.12:0/2037160845 shutdown_connections 2024-09-11T00:27:35.820 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:27:35.817+0000 7f857b5fc740 1 -- 172.21.15.12:0/2037160845 wait complete. 2024-09-11T00:27:35.820 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:27:35.817+0000 7f857b5fc740 20 client.27685 trim_cache size 0 max 16384 2024-09-11T00:27:35.821 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-09-11T00:27:35.821 INFO:teuthology.orchestra.run.smithi012.stdout: "epoch": 85, 2024-09-11T00:27:35.821 INFO:teuthology.orchestra.run.smithi012.stdout: "leader": 24674, 2024-09-11T00:27:35.821 INFO:teuthology.orchestra.run.smithi012.stdout: "set_version": 62, 2024-09-11T00:27:35.821 INFO:teuthology.orchestra.run.smithi012.stdout: "sets": { 2024-09-11T00:27:35.822 INFO:teuthology.orchestra.run.smithi012.stdout: "fae7af54": { 2024-09-11T00:27:35.822 INFO:teuthology.orchestra.run.smithi012.stdout: "version": 62, 2024-09-11T00:27:35.822 INFO:teuthology.orchestra.run.smithi012.stdout: "age_ref": 0.0, 2024-09-11T00:27:35.822 INFO:teuthology.orchestra.run.smithi012.stdout: "state": { 2024-09-11T00:27:35.822 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "QUIESCED", 2024-09-11T00:27:35.822 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 0.0 2024-09-11T00:27:35.822 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:27:35.822 INFO:teuthology.orchestra.run.smithi012.stdout: "timeout": 90.0, 2024-09-11T00:27:35.822 INFO:teuthology.orchestra.run.smithi012.stdout: "expiration": 150.0, 2024-09-11T00:27:35.822 INFO:teuthology.orchestra.run.smithi012.stdout: "members": { 2024-09-11T00:27:35.822 INFO:teuthology.orchestra.run.smithi012.stdout: "file:/": { 2024-09-11T00:27:35.822 INFO:teuthology.orchestra.run.smithi012.stdout: "excluded": false, 2024-09-11T00:27:35.822 INFO:teuthology.orchestra.run.smithi012.stdout: "state": { 2024-09-11T00:27:35.823 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "QUIESCED", 2024-09-11T00:27:35.823 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 0.0 2024-09-11T00:27:35.823 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:27:35.823 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:27:35.823 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:27:35.823 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:27:35.823 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:27:35.823 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-09-11T00:27:35.824 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:27:35.818+0000 7f8578c00640 1 -- 172.21.15.12:0/1675629344 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f85580b53a0 msgr2=0x7f85580b7860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:27:35.824 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:27:35.818+0000 7f8578c00640 1 --2- 172.21.15.12:0/1675629344 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f85580b53a0 0x7f85580b7860 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f856c0ca100 tx=0x7f856c009260 comp rx=0 tx=0).stop 2024-09-11T00:27:35.824 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:27:35.818+0000 7f8578c00640 1 -- 172.21.15.12:0/1675629344 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f857410b340 msgr2=0x7f85741a0ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:27:35.824 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:27:35.818+0000 7f8578c00640 1 --2- 172.21.15.12:0/1675629344 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f857410b340 0x7f85741a0ce0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f856c02a960 tx=0x7f856c003cd0 comp rx=0 tx=0).stop 2024-09-11T00:27:35.824 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:27:35.818+0000 7f8578c00640 1 -- 172.21.15.12:0/1675629344 shutdown_connections 2024-09-11T00:27:35.824 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:27:35.818+0000 7f8578c00640 1 --2- 172.21.15.12:0/1675629344 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f85580b53a0 0x7f85580b7860 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:27:35.824 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:27:35.818+0000 7f8578c00640 1 --2- 172.21.15.12:0/1675629344 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f857410b340 0x7f85741a0ce0 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:27:35.824 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:27:35.818+0000 7f8578c00640 1 --2- 172.21.15.12:0/1675629344 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f857410a9c0 0x7f857419b940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:27:35.824 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:27:35.818+0000 7f8578c00640 1 --2- 172.21.15.12:0/1675629344 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8574104fb0 0x7f857419b400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:27:35.824 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:27:35.818+0000 7f8578c00640 1 -- 172.21.15.12:0/1675629344 >> 172.21.15.12:0/1675629344 conn(0x7f8574100a20 msgr2=0x7f8574102e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:27:35.824 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:27:35.819+0000 7f8578c00640 1 -- 172.21.15.12:0/1675629344 shutdown_connections 2024-09-11T00:27:35.825 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:27:35.819+0000 7f8578c00640 1 -- 172.21.15.12:0/1675629344 wait complete. 2024-09-11T00:27:36.242 INFO:tasks.workunit.client.0.smithi012.stdout: 10 131026 15.89 MB/sec execute 332 sec latency 1022.273 ms 2024-09-11T00:27:36.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:27:35 smithi165 ceph-mon[35356]: pgmap v1014: 129 pgs: 129 active+clean; 692 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 230 KiB/s rd, 147 KiB/s wr, 58 op/s 2024-09-11T00:27:36.315 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:27:35 smithi012 ceph-mon[30333]: pgmap v1014: 129 pgs: 129 active+clean; 692 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 230 KiB/s rd, 147 KiB/s wr, 58 op/s 2024-09-11T00:27:36.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:27:35 smithi178 ceph-mon[35919]: pgmap v1014: 129 pgs: 129 active+clean; 692 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 230 KiB/s rd, 147 KiB/s wr, 58 op/s 2024-09-11T00:27:37.242 INFO:tasks.workunit.client.0.smithi012.stdout: 10 131026 15.84 MB/sec execute 333 sec latency 2022.393 ms 2024-09-11T00:27:37.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:27:36 smithi165 ceph-mon[35356]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T00:27:37.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:27:36 smithi012 ceph-mon[30333]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T00:27:37.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:27:36 smithi178 ceph-mon[35919]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T00:27:37.979 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'fae7af54', quiesce took 6.7 seconds. Will release after: 23.3 2024-09-11T00:27:37.997 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:37.995+0000 7fedafe00640 1 -- 172.21.15.12:0/1105117992 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7feda8078b70 msgr2=0x7feda8078ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:27:37.997 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:37.995+0000 7fedafe00640 1 --2- 172.21.15.12:0/1105117992 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7feda8078b70 0x7feda8078ff0 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7fed94009900 tx=0x7fed9402a3f0 comp rx=0 tx=0).stop 2024-09-11T00:27:37.997 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:37.995+0000 7fedafe00640 1 -- 172.21.15.12:0/1105117992 shutdown_connections 2024-09-11T00:27:37.997 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:37.995+0000 7fedafe00640 1 --2- 172.21.15.12:0/1105117992 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7feda8079530 0x7feda8079e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:27:37.997 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:37.995+0000 7fedafe00640 1 --2- 172.21.15.12:0/1105117992 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7feda8078b70 0x7feda8078ff0 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:27:37.997 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:37.995+0000 7fedafe00640 1 --2- 172.21.15.12:0/1105117992 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7feda8077a20 0x7feda8077e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:27:37.997 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:37.995+0000 7fedafe00640 1 -- 172.21.15.12:0/1105117992 >> 172.21.15.12:0/1105117992 conn(0x7feda8100470 msgr2=0x7feda8102890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:27:37.998 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:37.995+0000 7fedafe00640 1 -- 172.21.15.12:0/1105117992 shutdown_connections 2024-09-11T00:27:37.998 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:37.995+0000 7fedafe00640 1 -- 172.21.15.12:0/1105117992 wait complete. 2024-09-11T00:27:37.999 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:37.996+0000 7fedafe00640 1 Processor -- start 2024-09-11T00:27:37.999 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:37.997+0000 7fedafe00640 1 -- start start 2024-09-11T00:27:37.999 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:37.997+0000 7fedafe00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7feda8077a20 0x7feda819f370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:27:37.999 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:37.997+0000 7fedafe00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7feda8078b70 0x7feda819f9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:27:37.999 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:37.997+0000 7fedafe00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7feda8079530 0x7feda81a05f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:27:37.999 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:37.997+0000 7fedafe00640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7feda8114bd0 con 0x7feda8079530 2024-09-11T00:27:37.999 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:37.997+0000 7fedafe00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7feda8114a50 con 0x7feda8077a20 2024-09-11T00:27:37.999 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:37.997+0000 7fedafe00640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7feda8114d50 con 0x7feda8078b70 2024-09-11T00:27:38.000 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:37.997+0000 7fedad400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7feda8077a20 0x7feda819f370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:27:38.000 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:37.997+0000 7fedade00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7feda8079530 0x7feda81a05f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:27:38.000 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:37.997+0000 7fedad400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7feda8077a20 0x7feda819f370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.12:59602/0 (socket says 172.21.15.12:59602) 2024-09-11T00:27:38.000 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:37.997+0000 7fedad400640 1 -- 172.21.15.12:0/2355853150 learned_addr learned my addr 172.21.15.12:0/2355853150 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:27:38.000 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:37.997+0000 7fedaca00640 1 --2- 172.21.15.12:0/2355853150 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7feda8078b70 0x7feda819f9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:27:38.000 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:37.998+0000 7fedad400640 1 -- 172.21.15.12:0/2355853150 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7feda8078b70 msgr2=0x7feda819f9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:27:38.000 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:37.998+0000 7fedad400640 1 --2- 172.21.15.12:0/2355853150 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7feda8078b70 0x7feda819f9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:27:38.000 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:37.998+0000 7fedad400640 1 -- 172.21.15.12:0/2355853150 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7feda8079530 msgr2=0x7feda81a05f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:27:38.000 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:37.998+0000 7fedad400640 1 --2- 172.21.15.12:0/2355853150 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7feda8079530 0x7feda81a05f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:27:38.000 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:37.998+0000 7fedad400640 1 -- 172.21.15.12:0/2355853150 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feda81a0eb0 con 0x7feda8077a20 2024-09-11T00:27:38.001 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:37.998+0000 7fedad400640 1 --2- 172.21.15.12:0/2355853150 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7feda8077a20 0x7feda819f370 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fed9c00ea60 tx=0x7fed9c00ef20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:27:38.001 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:37.998+0000 7feda6000640 1 -- 172.21.15.12:0/2355853150 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed9c00cde0 con 0x7feda8077a20 2024-09-11T00:27:38.001 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:37.998+0000 7fedafe00640 1 -- 172.21.15.12:0/2355853150 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feda81a11a0 con 0x7feda8077a20 2024-09-11T00:27:38.001 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:37.998+0000 7feda6000640 1 -- 172.21.15.12:0/2355853150 <== mon.2 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fed9c0035f0 con 0x7feda8077a20 2024-09-11T00:27:38.001 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:37.998+0000 7feda6000640 1 -- 172.21.15.12:0/2355853150 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed9c010690 con 0x7feda8077a20 2024-09-11T00:27:38.001 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:37.999+0000 7fedafe00640 1 -- 172.21.15.12:0/2355853150 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feda81a7870 con 0x7feda8077a20 2024-09-11T00:27:38.003 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:38.002+0000 7fedb2e3d740 1 Processor -- start 2024-09-11T00:27:38.003 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:38.002+0000 7fedb2e3d740 1 -- start start 2024-09-11T00:27:38.006 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:38.002+0000 7feda6000640 1 -- 172.21.15.12:0/2355853150 <== mon.2 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7fed9c004000 con 0x7feda8077a20 2024-09-11T00:27:38.006 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:38.002+0000 7fedb2e3d740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562219bb0790 0x562219bb2b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:27:38.006 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:38.002+0000 7fedb2e3d740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562219bb30c0 0x562219bb54b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:27:38.006 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:38.002+0000 7fedb2e3d740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562219bb5b00 0x562219bb7fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:27:38.006 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:38.002+0000 7fedb2e3d740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x562219b236b0 con 0x562219bb0790 2024-09-11T00:27:38.006 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:38.002+0000 7fedb2e3d740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x562219b12ae0 con 0x562219bb5b00 2024-09-11T00:27:38.006 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:38.002+0000 7fedb2e3d740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x562219b168f0 con 0x562219bb30c0 2024-09-11T00:27:38.006 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:38.002+0000 7fedade00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562219bb0790 0x562219bb2b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:27:38.006 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:38.002+0000 7fedade00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562219bb0790 0x562219bb2b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:33678/0 (socket says 172.21.15.12:33678) 2024-09-11T00:27:38.007 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:38.002+0000 7fedade00640 1 -- 172.21.15.12:0/1064076589 learned_addr learned my addr 172.21.15.12:0/1064076589 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:27:38.007 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:38.003+0000 7fedad400640 1 --2- 172.21.15.12:0/1064076589 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562219bb30c0 0x562219bb54b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:27:38.007 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:38.003+0000 7fedaca00640 1 --2- 172.21.15.12:0/1064076589 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562219bb5b00 0x562219bb7fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:27:38.007 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:38.003+0000 7fedade00640 1 -- 172.21.15.12:0/1064076589 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562219bb30c0 msgr2=0x562219bb54b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:27:38.007 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:38.003+0000 7fedade00640 1 --2- 172.21.15.12:0/1064076589 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562219bb30c0 0x562219bb54b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:27:38.007 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:38.003+0000 7fedade00640 1 -- 172.21.15.12:0/1064076589 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562219bb5b00 msgr2=0x562219bb7fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:27:38.007 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:38.003+0000 7fedade00640 1 --2- 172.21.15.12:0/1064076589 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562219bb5b00 0x562219bb7fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:27:38.007 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:38.003+0000 7fedade00640 1 -- 172.21.15.12:0/1064076589 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562219adf780 con 0x562219bb0790 2024-09-11T00:27:38.007 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:38.003+0000 7fedad400640 1 --2- 172.21.15.12:0/1064076589 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562219bb30c0 0x562219bb54b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T00:27:38.007 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:38.003+0000 7fedaca00640 1 --2- 172.21.15.12:0/1064076589 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562219bb5b00 0x562219bb7fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T00:27:38.008 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:38.003+0000 7fedade00640 1 --2- 172.21.15.12:0/1064076589 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562219bb0790 0x562219bb2b80 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7feda0009fc0 tx=0x7feda0010800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e055ee12d1ea87 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:27:38.008 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:38.003+0000 7fed9a600640 1 -- 172.21.15.12:0/1064076589 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feda0016070 con 0x562219bb0790 2024-09-11T00:27:38.008 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:38.004+0000 7fed9a600640 1 -- 172.21.15.12:0/1064076589 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feda0004d20 con 0x562219bb0790 2024-09-11T00:27:38.008 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:38.004+0000 7fed9a600640 1 -- 172.21.15.12:0/1064076589 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feda0005350 con 0x562219bb0790 2024-09-11T00:27:38.008 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:38.004+0000 7feda6000640 1 --2- 172.21.15.12:0/2355853150 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fed900b5360 0x7fed900b7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:27:38.008 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:38.004+0000 7feda6000640 1 -- 172.21.15.12:0/2355853150 <== mon.2 v2:172.21.15.165:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9208+0+0 (secure 0 0 0) 0x7fed9c0e6240 con 0x7feda8077a20 2024-09-11T00:27:38.008 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:38.004+0000 7fedb2e3d740 1 -- 172.21.15.12:0/1064076589 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562219bb0790 msgr2=0x562219bb2b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:27:38.008 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:38.004+0000 7fedb2e3d740 1 --2- 172.21.15.12:0/1064076589 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562219bb0790 0x562219bb2b80 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7feda0009fc0 tx=0x7feda0010800 comp rx=0 tx=0).stop 2024-09-11T00:27:38.008 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:38.004+0000 7fedad400640 1 --2- 172.21.15.12:0/2355853150 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fed900b5360 0x7fed900b7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:27:38.008 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:38.004+0000 7fedb2e3d740 1 -- 172.21.15.12:0/1064076589 shutdown_connections 2024-09-11T00:27:38.008 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:38.004+0000 7fedb2e3d740 1 --2- 172.21.15.12:0/1064076589 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562219bb5b00 0x562219bb7fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:27:38.009 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:38.004+0000 7fedb2e3d740 1 --2- 172.21.15.12:0/1064076589 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562219bb30c0 0x562219bb54b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:27:38.009 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:38.004+0000 7fedb2e3d740 1 --2- 172.21.15.12:0/1064076589 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562219bb0790 0x562219bb2b80 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:27:38.009 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:38.004+0000 7fedb2e3d740 1 -- 172.21.15.12:0/1064076589 >> 172.21.15.12:0/1064076589 conn(0x562219bab8e0 msgr2=0x562219bac670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:27:38.009 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:38.004+0000 7fedb2e3d740 1 -- 172.21.15.12:0/1064076589 shutdown_connections 2024-09-11T00:27:38.009 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:38.005+0000 7fedb2e3d740 1 -- 172.21.15.12:0/1064076589 wait complete. 2024-09-11T00:27:38.013 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:38.005+0000 7fedad400640 1 --2- 172.21.15.12:0/2355853150 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fed900b5360 0x7fed900b7820 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fed9c0ac560 tx=0x7fed9c00ef90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:27:38.013 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:38.005+0000 7fedb2e3d740 1 -- start start 2024-09-11T00:27:38.013 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:38.005+0000 7fedb2e3d740 1 Processor -- start 2024-09-11T00:27:38.013 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:38.005+0000 7fedb2e3d740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562219bb0790 0x562219bb9f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:27:38.013 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:38.005+0000 7fedb2e3d740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562219bb30c0 0x562219bba5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:27:38.013 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:38.006+0000 7fedb2e3d740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562219bb5b00 0x562219bbb140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:27:38.013 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:38.006+0000 7fedb2e3d740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x562219a4aab0 con 0x562219bb30c0 2024-09-11T00:27:38.013 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:38.006+0000 7fedb2e3d740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x562219ab9570 con 0x562219bb0790 2024-09-11T00:27:38.013 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:38.006+0000 7fedb2e3d740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x562219bb8b30 con 0x562219bb5b00 2024-09-11T00:27:38.013 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:38.006+0000 7fedb2e3d740 10 client.0 _ll_register_callbacks cb 0x562219ac8100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:27:38.014 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:38.006+0000 7fedaca00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562219bb30c0 0x562219bba5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:27:38.014 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:38.006+0000 7fedaca00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562219bb30c0 0x562219bba5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:33692/0 (socket says 172.21.15.12:33692) 2024-09-11T00:27:38.014 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:38.006+0000 7fedaca00640 1 -- 172.21.15.12:0/2776830443 learned_addr learned my addr 172.21.15.12:0/2776830443 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:27:38.014 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:38.006+0000 7fedade00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562219bb0790 0x562219bb9f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:27:38.014 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:38.006+0000 7fedade00640 1 --2- 172.21.15.12:0/2776830443 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562219bb5b00 0x562219bbb140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:27:38.014 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:38.006+0000 7fedaca00640 1 -- 172.21.15.12:0/2776830443 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562219bb5b00 msgr2=0x562219bbb140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:27:38.014 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:38.006+0000 7fedaca00640 1 --2- 172.21.15.12:0/2776830443 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562219bb5b00 0x562219bbb140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:27:38.014 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:38.006+0000 7fedaca00640 1 -- 172.21.15.12:0/2776830443 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562219bb0790 msgr2=0x562219bb9f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:27:38.014 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:38.006+0000 7fedaca00640 1 --2- 172.21.15.12:0/2776830443 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562219bb0790 0x562219bb9f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:27:38.014 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:38.006+0000 7fedaca00640 1 -- 172.21.15.12:0/2776830443 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562219874790 con 0x562219bb30c0 2024-09-11T00:27:38.014 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:38.006+0000 7fedade00640 1 --2- 172.21.15.12:0/2776830443 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562219bb0790 0x562219bb9f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:27:38.015 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:38.007+0000 7fedaca00640 1 --2- 172.21.15.12:0/2776830443 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562219bb30c0 0x562219bba5e0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7fed94009900 tx=0x7fed94009680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af6f6572b71fabbd server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:27:38.015 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:38.007+0000 7fed8ea00640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-09-11T00:27:38.015 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:38.007+0000 7fed8ea00640 1 -- 172.21.15.12:0/2776830443 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed94035070 con 0x562219bb30c0 2024-09-11T00:27:38.015 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:38.007+0000 7fedb2e3d740 1 -- 172.21.15.12:0/2776830443 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feda0004d20 con 0x562219bb30c0 2024-09-11T00:27:38.015 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:38.007+0000 7fed8ea00640 1 -- 172.21.15.12:0/2776830443 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9208+0+0 (secure 0 0 0) 0x7fed94004040 con 0x562219bb30c0 2024-09-11T00:27:38.015 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:38.007+0000 7fed8ea00640 1 -- 172.21.15.12:0/2776830443 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=86) ==== 24+0+0 (secure 0 0 0) 0x7fed9404e050 con 0x562219bb30c0 2024-09-11T00:27:38.015 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:38.007+0000 7fedb2e3d740 10 client.17721 fetch_fsmap learned FSMap version 86 2024-09-11T00:27:38.015 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:38.007+0000 7fedb2e3d740 1 -- 172.21.15.12:0/2776830443 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=86,monmap=4+}) -- 0x5622198ed620 con 0x562219bb30c0 2024-09-11T00:27:38.015 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:38.007+0000 7fed8ea00640 1 -- 172.21.15.12:0/2776830443 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feda0004d20 con 0x562219bb30c0 2024-09-11T00:27:38.015 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:38.008+0000 7fed8ea00640 1 -- 172.21.15.12:0/2776830443 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 86) ==== 8438+0+0 (secure 0 0 0) 0x7fed94043020 con 0x562219bb30c0 2024-09-11T00:27:38.016 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:38.008+0000 7fedb2e3d740 10 client.17721 fetch_fsmap finished waiting for FSMap version 86 2024-09-11T00:27:38.016 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:38.008+0000 7fedb2e3d740 10 client.17721 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T00:27:38.016 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:38.008+0000 7fedb2e3d740 20 client.17721 populate_metadata read hostname 'smithi012' 2024-09-11T00:27:38.016 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:38.008+0000 7fedb2e3d740 1 --2- 172.21.15.12:0/2776830443 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x562219bc5890 0x562219bc7cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:27:38.016 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:38.008+0000 7fedb2e3d740 4 client.17721 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T00:27:38.016 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:38.008+0000 7fedb2e3d740 1 -- 172.21.15.12:0/2776830443 --> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562219b25600 con 0x562219bc5890 2024-09-11T00:27:38.016 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:38.008+0000 7fedade00640 1 --2- 172.21.15.12:0/2776830443 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x562219bc5890 0x562219bc7cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:27:38.016 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:38.009+0000 7fedade00640 1 --2- 172.21.15.12:0/2776830443 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x562219bc5890 0x562219bc7cb0 crc :-1 s=READY pgs=285 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6c3ce6707387e54a server_cookie=7fc91fa15b549481 in_seq=0 out_seq=0 2024-09-11T00:27:38.016 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:38.009+0000 7fed8ea00640 10 client.17721 ms_handle_connect on v2:172.21.15.165:6838/501312984 2024-09-11T00:27:38.016 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:38.010+0000 7fed8ea00640 1 -- 172.21.15.12:0/2776830443 <== mds.0 v2:172.21.15.165:6838/501312984 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x562219b25600 con 0x562219bc5890 2024-09-11T00:27:38.016 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:38.010+0000 7fed8ea00640 10 client.17721 handle_command_reply: tid=0 2024-09-11T00:27:38.016 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:38.010+0000 7fedb2e3d740 1 client.17721 shutdown 2024-09-11T00:27:38.017 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:38.010+0000 7fedb2e3d740 1 -- 172.21.15.12:0/2776830443 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562219bb30c0 msgr2=0x562219bba5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:27:38.017 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:38.010+0000 7fedb2e3d740 1 --2- 172.21.15.12:0/2776830443 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562219bb30c0 0x562219bba5e0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7fed94009900 tx=0x7fed94009680 comp rx=0 tx=0).stop 2024-09-11T00:27:38.017 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:38.010+0000 7fedb2e3d740 1 -- 172.21.15.12:0/2776830443 shutdown_connections 2024-09-11T00:27:38.017 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:38.010+0000 7fedb2e3d740 1 --2- 172.21.15.12:0/2776830443 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x562219bc5890 0x562219bc7cb0 crc :-1 s=READY pgs=285 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:27:38.017 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:38.010+0000 7fedb2e3d740 1 --2- 172.21.15.12:0/2776830443 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562219bb5b00 0x562219bbb140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:27:38.017 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:38.010+0000 7fedb2e3d740 1 --2- 172.21.15.12:0/2776830443 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562219bb30c0 0x562219bba5e0 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:27:38.017 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:38.010+0000 7fedb2e3d740 1 --2- 172.21.15.12:0/2776830443 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562219bb0790 0x562219bb9f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:27:38.017 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:38.010+0000 7fed8ea00640 0 client.17721 ms_handle_reset on v2:172.21.15.165:6838/501312984 2024-09-11T00:27:38.017 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:38.010+0000 7fedb2e3d740 1 -- 172.21.15.12:0/2776830443 >> 172.21.15.12:0/2776830443 conn(0x562219bab8e0 msgr2=0x562219baaa60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:27:38.017 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:38.010+0000 7fedb2e3d740 1 -- 172.21.15.12:0/2776830443 shutdown_connections 2024-09-11T00:27:38.018 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:38.010+0000 7fedb2e3d740 1 -- 172.21.15.12:0/2776830443 wait complete. 2024-09-11T00:27:38.018 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:38.010+0000 7fedb2e3d740 20 client.17721 trim_cache size 0 max 16384 2024-09-11T00:27:38.028 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:38.026+0000 7fedb2e3d740 1 Processor -- start 2024-09-11T00:27:38.029 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:38.026+0000 7fedb2e3d740 1 -- start start 2024-09-11T00:27:38.029 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:38.026+0000 7fedb2e3d740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562219bb0790 0x562219ba6960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:27:38.029 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:38.026+0000 7fedb2e3d740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562219bb30c0 0x562219bb9f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:27:38.029 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:38.026+0000 7fedb2e3d740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562219bb5b00 0x562219bba480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:27:38.029 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:38.026+0000 7fedb2e3d740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7feda81b1d70 con 0x562219bb0790 2024-09-11T00:27:38.029 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:38.026+0000 7fedb2e3d740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7feda81b3af0 con 0x562219bb5b00 2024-09-11T00:27:38.029 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:38.026+0000 7fedb2e3d740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7feda81b2500 con 0x562219bb30c0 2024-09-11T00:27:38.029 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:38.026+0000 7fedade00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562219bb0790 0x562219ba6960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:27:38.029 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:38.026+0000 7fedade00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562219bb0790 0x562219ba6960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:33694/0 (socket says 172.21.15.12:33694) 2024-09-11T00:27:38.029 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:38.026+0000 7fedade00640 1 -- 172.21.15.12:0/1747161678 learned_addr learned my addr 172.21.15.12:0/1747161678 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:27:38.029 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:38.026+0000 7fedaca00640 1 --2- 172.21.15.12:0/1747161678 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562219bb30c0 0x562219bb9f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:27:38.030 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:38.027+0000 7fedade00640 1 -- 172.21.15.12:0/1747161678 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562219bb30c0 msgr2=0x562219bb9f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:27:38.030 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:38.027+0000 7fedade00640 1 --2- 172.21.15.12:0/1747161678 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562219bb30c0 0x562219bb9f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:27:38.030 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:38.027+0000 7fedade00640 1 -- 172.21.15.12:0/1747161678 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562219bb5b00 msgr2=0x562219bba480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:27:38.030 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:38.027+0000 7fedade00640 1 --2- 172.21.15.12:0/1747161678 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562219bb5b00 0x562219bba480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:27:38.030 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:38.027+0000 7fedade00640 1 -- 172.21.15.12:0/1747161678 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562219bacb40 con 0x562219bb0790 2024-09-11T00:27:38.030 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:38.027+0000 7fedaca00640 1 --2- 172.21.15.12:0/1747161678 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562219bb30c0 0x562219bb9f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:27:38.030 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:38.027+0000 7fedade00640 1 --2- 172.21.15.12:0/1747161678 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562219bb0790 0x562219ba6960 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7feda0010d00 tx=0x7feda0004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be3716172c3d3999 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:27:38.030 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:38.027+0000 7fed99c00640 1 -- 172.21.15.12:0/1747161678 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feda0016040 con 0x562219bb0790 2024-09-11T00:27:38.032 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:38.027+0000 7fed99c00640 1 -- 172.21.15.12:0/1747161678 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feda0003e20 con 0x562219bb0790 2024-09-11T00:27:38.032 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:38.027+0000 7fed99c00640 1 -- 172.21.15.12:0/1747161678 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feda0005940 con 0x562219bb0790 2024-09-11T00:27:38.035 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:38.028+0000 7fedb2e3d740 1 -- 172.21.15.12:0/1747161678 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562219bb0790 msgr2=0x562219ba6960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:27:38.035 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:38.028+0000 7fedb2e3d740 1 --2- 172.21.15.12:0/1747161678 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562219bb0790 0x562219ba6960 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7feda0010d00 tx=0x7feda0004af0 comp rx=0 tx=0).stop 2024-09-11T00:27:38.035 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:38.028+0000 7fedb2e3d740 1 -- 172.21.15.12:0/1747161678 shutdown_connections 2024-09-11T00:27:38.035 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:38.028+0000 7fedb2e3d740 1 --2- 172.21.15.12:0/1747161678 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562219bb5b00 0x562219bba480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:27:38.035 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:38.028+0000 7fedb2e3d740 1 --2- 172.21.15.12:0/1747161678 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562219bb30c0 0x562219bb9f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:27:38.036 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:38.028+0000 7fedb2e3d740 1 --2- 172.21.15.12:0/1747161678 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562219bb0790 0x562219ba6960 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:27:38.036 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:38.028+0000 7fedb2e3d740 1 -- 172.21.15.12:0/1747161678 >> 172.21.15.12:0/1747161678 conn(0x562219bab8e0 msgr2=0x562219bb5ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:27:38.036 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:38.028+0000 7fedb2e3d740 1 -- 172.21.15.12:0/1747161678 shutdown_connections 2024-09-11T00:27:38.036 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:38.028+0000 7fedb2e3d740 1 -- 172.21.15.12:0/1747161678 wait complete. 2024-09-11T00:27:38.036 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:38.028+0000 7fedb2e3d740 1 -- start start 2024-09-11T00:27:38.036 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:38.028+0000 7fedb2e3d740 1 Processor -- start 2024-09-11T00:27:38.036 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:38.029+0000 7fedb2e3d740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562219bb0790 0x562219bc0bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:27:38.036 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:38.029+0000 7fedb2e3d740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562219bb30c0 0x562219bc1110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:27:38.036 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:38.029+0000 7fedb2e3d740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562219bb5b00 0x562219bbf740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:27:38.036 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:38.029+0000 7fedb2e3d740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7feda81b3420 con 0x562219bb0790 2024-09-11T00:27:38.036 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:38.029+0000 7fedb2e3d740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7feda81b2c90 con 0x562219bb30c0 2024-09-11T00:27:38.036 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:38.029+0000 7fedb2e3d740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7feda81afd50 con 0x562219bb5b00 2024-09-11T00:27:38.037 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:38.029+0000 7fedb2e3d740 10 client.0 _ll_register_callbacks cb 0x7fed94009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:27:38.037 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:38.029+0000 7fedaca00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562219bb30c0 0x562219bc1110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:27:38.037 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:38.029+0000 7fedade00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562219bb0790 0x562219bc0bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:27:38.037 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:38.029+0000 7fedaca00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562219bb30c0 0x562219bc1110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.12:59644/0 (socket says 172.21.15.12:59644) 2024-09-11T00:27:38.037 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:38.029+0000 7fedaca00640 1 -- 172.21.15.12:0/3081500484 learned_addr learned my addr 172.21.15.12:0/3081500484 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:27:38.037 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:38.029+0000 7fedade00640 1 --2- 172.21.15.12:0/3081500484 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562219bb5b00 0x562219bbf740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:27:38.037 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:38.029+0000 7fedaca00640 1 -- 172.21.15.12:0/3081500484 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562219bb5b00 msgr2=0x562219bbf740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:27:38.037 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:38.029+0000 7fedaca00640 1 --2- 172.21.15.12:0/3081500484 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562219bb5b00 0x562219bbf740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:27:38.037 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:38.029+0000 7fedaca00640 1 -- 172.21.15.12:0/3081500484 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562219bb0790 msgr2=0x562219bc0bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:27:38.037 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:38.029+0000 7fedaca00640 1 --2- 172.21.15.12:0/3081500484 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562219bb0790 0x562219bc0bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:27:38.037 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:38.029+0000 7fedaca00640 1 -- 172.21.15.12:0/3081500484 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562219bb5820 con 0x562219bb30c0 2024-09-11T00:27:38.037 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:38.030+0000 7fedade00640 1 --2- 172.21.15.12:0/3081500484 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562219bb5b00 0x562219bbf740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T00:27:38.038 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:38.030+0000 7fedaca00640 1 --2- 172.21.15.12:0/3081500484 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562219bb30c0 0x562219bc1110 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fed94003e90 tx=0x7fed940092a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c4e2222a20dcb895 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:27:38.038 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:38.030+0000 7fed8ea00640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-09-11T00:27:38.038 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:38.030+0000 7fed8ea00640 1 -- 172.21.15.12:0/3081500484 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed94057060 con 0x562219bb30c0 2024-09-11T00:27:38.038 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:38.030+0000 7fed8ea00640 1 -- 172.21.15.12:0/3081500484 <== mon.2 v2:172.21.15.165:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9208+0+0 (secure 0 0 0) 0x7fed940456c0 con 0x562219bb30c0 2024-09-11T00:27:38.038 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:38.030+0000 7fedb2e3d740 1 -- 172.21.15.12:0/3081500484 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feda0003e20 con 0x562219bb30c0 2024-09-11T00:27:38.038 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:38.030+0000 7fed8ea00640 1 -- 172.21.15.12:0/3081500484 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=86) ==== 24+0+0 (secure 0 0 0) 0x7fed9405e050 con 0x562219bb30c0 2024-09-11T00:27:38.038 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:38.030+0000 7fedb2e3d740 10 client.27725 fetch_fsmap learned FSMap version 86 2024-09-11T00:27:38.038 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:38.030+0000 7fedb2e3d740 1 -- 172.21.15.12:0/3081500484 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=86,monmap=4+}) -- 0x562219c32c90 con 0x562219bb30c0 2024-09-11T00:27:38.038 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:38.031+0000 7fed8ea00640 1 -- 172.21.15.12:0/3081500484 <== mon.2 v2:172.21.15.165:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feda0003e20 con 0x562219bb30c0 2024-09-11T00:27:38.038 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:38.031+0000 7fed8ea00640 1 -- 172.21.15.12:0/3081500484 <== mon.2 v2:172.21.15.165:3300/0 5 ==== fsmap(e 86) ==== 8438+0+0 (secure 0 0 0) 0x7fed94043040 con 0x562219bb30c0 2024-09-11T00:27:38.038 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:38.031+0000 7fedb2e3d740 10 client.27725 fetch_fsmap finished waiting for FSMap version 86 2024-09-11T00:27:38.038 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:38.031+0000 7fedb2e3d740 10 client.27725 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T00:27:38.038 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:38.031+0000 7fedb2e3d740 20 client.27725 populate_metadata read hostname 'smithi012' 2024-09-11T00:27:38.039 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:38.031+0000 7fedb2e3d740 1 --2- 172.21.15.12:0/3081500484 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x562219c33730 0x562219c35b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:27:38.039 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:38.031+0000 7fedb2e3d740 4 client.27725 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T00:27:38.039 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:38.031+0000 7fedb2e3d740 1 -- 172.21.15.12:0/3081500484 --> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] -- command(tid 0: {"prefix": "flush journal"}) -- 0x562219b210b0 con 0x562219c33730 2024-09-11T00:27:38.039 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:38.031+0000 7fedade00640 1 --2- 172.21.15.12:0/3081500484 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x562219c33730 0x562219c35b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:27:38.039 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:38.032+0000 7fedade00640 1 --2- 172.21.15.12:0/3081500484 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x562219c33730 0x562219c35b50 crc :-1 s=READY pgs=286 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f427ecff01d4cb4a server_cookie=b18837d485f4a438 in_seq=0 out_seq=0 2024-09-11T00:27:38.039 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:38.032+0000 7fed8ea00640 10 client.27725 ms_handle_connect on v2:172.21.15.165:6838/501312984 2024-09-11T00:27:38.143 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:38.141+0000 7fed8ea00640 1 -- 172.21.15.12:0/3081500484 <== mds.0 v2:172.21.15.165:6838/501312984 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x562219b210b0 con 0x562219c33730 2024-09-11T00:27:38.143 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:38.141+0000 7fed8ea00640 10 client.27725 handle_command_reply: tid=0 2024-09-11T00:27:38.143 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:38.141+0000 7fedb2e3d740 1 client.27725 shutdown 2024-09-11T00:27:38.143 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:38.141+0000 7fedb2e3d740 1 -- 172.21.15.12:0/3081500484 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562219bb30c0 msgr2=0x562219bc1110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:27:38.143 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:38.141+0000 7fedb2e3d740 1 --2- 172.21.15.12:0/3081500484 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562219bb30c0 0x562219bc1110 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fed94003e90 tx=0x7fed940092a0 comp rx=0 tx=0).stop 2024-09-11T00:27:38.143 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:38.141+0000 7fedb2e3d740 1 -- 172.21.15.12:0/3081500484 shutdown_connections 2024-09-11T00:27:38.143 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:38.141+0000 7fedb2e3d740 1 --2- 172.21.15.12:0/3081500484 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x562219c33730 0x562219c35b50 crc :-1 s=READY pgs=286 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:27:38.144 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:38.141+0000 7fedb2e3d740 1 --2- 172.21.15.12:0/3081500484 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562219bb5b00 0x562219bbf740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:27:38.144 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:38.141+0000 7fedb2e3d740 1 --2- 172.21.15.12:0/3081500484 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562219bb30c0 0x562219bc1110 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:27:38.144 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:38.141+0000 7fedb2e3d740 1 --2- 172.21.15.12:0/3081500484 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562219bb0790 0x562219bc0bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:27:38.144 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:38.141+0000 7fed8ea00640 0 client.27725 ms_handle_reset on v2:172.21.15.165:6838/501312984 2024-09-11T00:27:38.144 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:38.141+0000 7fedb2e3d740 1 -- 172.21.15.12:0/3081500484 >> 172.21.15.12:0/3081500484 conn(0x562219bab8e0 msgr2=0x562219bb9f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:27:38.144 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:38.141+0000 7fedb2e3d740 1 -- 172.21.15.12:0/3081500484 shutdown_connections 2024-09-11T00:27:38.144 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:38.141+0000 7fedb2e3d740 1 -- 172.21.15.12:0/3081500484 wait complete. 2024-09-11T00:27:38.144 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:38.141+0000 7fedb2e3d740 20 client.27725 trim_cache size 0 max 16384 2024-09-11T00:27:38.144 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:{ 2024-09-11T00:27:38.145 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "message": "", 2024-09-11T00:27:38.145 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "return_code": 0 2024-09-11T00:27:38.145 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:} 2024-09-11T00:27:38.145 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:38.142+0000 7fedafe00640 1 -- 172.21.15.12:0/2355853150 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fed900b5360 msgr2=0x7fed900b7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:27:38.146 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:38.142+0000 7fedafe00640 1 --2- 172.21.15.12:0/2355853150 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fed900b5360 0x7fed900b7820 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fed9c0ac560 tx=0x7fed9c00ef90 comp rx=0 tx=0).stop 2024-09-11T00:27:38.146 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:38.142+0000 7fedafe00640 1 -- 172.21.15.12:0/2355853150 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7feda8077a20 msgr2=0x7feda819f370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:27:38.146 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:38.142+0000 7fedafe00640 1 --2- 172.21.15.12:0/2355853150 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7feda8077a20 0x7feda819f370 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fed9c00ea60 tx=0x7fed9c00ef20 comp rx=0 tx=0).stop 2024-09-11T00:27:38.146 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:38.142+0000 7fedafe00640 1 -- 172.21.15.12:0/2355853150 shutdown_connections 2024-09-11T00:27:38.146 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:38.142+0000 7fedafe00640 1 --2- 172.21.15.12:0/2355853150 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fed900b5360 0x7fed900b7820 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:27:38.146 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:38.142+0000 7fedafe00640 1 --2- 172.21.15.12:0/2355853150 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7feda8079530 0x7feda81a05f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:27:38.146 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:38.142+0000 7fedafe00640 1 --2- 172.21.15.12:0/2355853150 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7feda8078b70 0x7feda819f9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:27:38.146 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:38.142+0000 7fedafe00640 1 --2- 172.21.15.12:0/2355853150 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7feda8077a20 0x7feda819f370 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:27:38.146 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:38.142+0000 7fedafe00640 1 -- 172.21.15.12:0/2355853150 >> 172.21.15.12:0/2355853150 conn(0x7feda8100470 msgr2=0x7feda8101f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:27:38.146 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:38.142+0000 7fedafe00640 1 -- 172.21.15.12:0/2355853150 shutdown_connections 2024-09-11T00:27:38.146 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:38.142+0000 7fedafe00640 1 -- 172.21.15.12:0/2355853150 wait complete. 2024-09-11T00:27:38.242 INFO:tasks.workunit.client.0.smithi012.stdout: 10 131026 15.79 MB/sec execute 334 sec latency 3022.525 ms 2024-09-11T00:27:38.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:27:37 smithi165 ceph-mon[35356]: pgmap v1015: 129 pgs: 129 active+clean; 670 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 919 KiB/s wr, 221 op/s 2024-09-11T00:27:38.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:27:37 smithi012 ceph-mon[30333]: pgmap v1015: 129 pgs: 129 active+clean; 670 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 919 KiB/s wr, 221 op/s 2024-09-11T00:27:38.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:27:37 smithi178 ceph-mon[35919]: pgmap v1015: 129 pgs: 129 active+clean; 670 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 919 KiB/s wr, 221 op/s 2024-09-11T00:27:39.242 INFO:tasks.workunit.client.0.smithi012.stdout: 10 131026 15.75 MB/sec execute 335 sec latency 4022.694 ms 2024-09-11T00:27:40.242 INFO:tasks.workunit.client.0.smithi012.stdout: 10 131026 15.70 MB/sec execute 336 sec latency 5022.787 ms 2024-09-11T00:27:40.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:27:39 smithi165 ceph-mon[35356]: pgmap v1016: 129 pgs: 129 active+clean; 670 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 980 KiB/s wr, 209 op/s 2024-09-11T00:27:40.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:27:39 smithi012 ceph-mon[30333]: pgmap v1016: 129 pgs: 129 active+clean; 670 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 980 KiB/s wr, 209 op/s 2024-09-11T00:27:40.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:27:39 smithi178 ceph-mon[35919]: pgmap v1016: 129 pgs: 129 active+clean; 670 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 980 KiB/s wr, 209 op/s 2024-09-11T00:27:41.242 INFO:tasks.workunit.client.0.smithi012.stdout: 10 131026 15.65 MB/sec execute 337 sec latency 6022.940 ms 2024-09-11T00:27:42.242 INFO:tasks.workunit.client.0.smithi012.stdout: 10 131026 15.61 MB/sec execute 338 sec latency 7023.025 ms 2024-09-11T00:27:42.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:27:41 smithi165 ceph-mon[35356]: pgmap v1017: 129 pgs: 129 active+clean; 670 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 1.1 MiB/s wr, 243 op/s 2024-09-11T00:27:42.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:27:41 smithi012 ceph-mon[30333]: pgmap v1017: 129 pgs: 129 active+clean; 670 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 1.1 MiB/s wr, 243 op/s 2024-09-11T00:27:42.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:27:41 smithi178 ceph-mon[35919]: pgmap v1017: 129 pgs: 129 active+clean; 670 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 1.1 MiB/s wr, 243 op/s 2024-09-11T00:27:43.242 INFO:tasks.workunit.client.0.smithi012.stdout: 10 131026 15.56 MB/sec execute 339 sec latency 8023.091 ms 2024-09-11T00:27:44.243 INFO:tasks.workunit.client.0.smithi012.stdout: 10 131026 15.52 MB/sec execute 340 sec latency 9023.216 ms 2024-09-11T00:27:44.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:27:43 smithi165 ceph-mon[35356]: pgmap v1018: 129 pgs: 129 active+clean; 670 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 1.1 MiB/s wr, 257 op/s 2024-09-11T00:27:44.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:27:43 smithi012 ceph-mon[30333]: pgmap v1018: 129 pgs: 129 active+clean; 670 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 1.1 MiB/s wr, 257 op/s 2024-09-11T00:27:44.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:27:43 smithi178 ceph-mon[35919]: pgmap v1018: 129 pgs: 129 active+clean; 670 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 1.1 MiB/s wr, 257 op/s 2024-09-11T00:27:45.243 INFO:tasks.workunit.client.0.smithi012.stdout: 10 131026 15.47 MB/sec execute 341 sec latency 10023.367 ms 2024-09-11T00:27:46.243 INFO:tasks.workunit.client.0.smithi012.stdout: 10 131026 15.42 MB/sec execute 342 sec latency 11023.536 ms 2024-09-11T00:27:46.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:27:45 smithi165 ceph-mon[35356]: pgmap v1019: 129 pgs: 129 active+clean; 670 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 1.0 MiB/s wr, 241 op/s 2024-09-11T00:27:46.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:27:45 smithi012 ceph-mon[30333]: pgmap v1019: 129 pgs: 129 active+clean; 670 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 1.0 MiB/s wr, 241 op/s 2024-09-11T00:27:46.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:27:45 smithi178 ceph-mon[35919]: pgmap v1019: 129 pgs: 129 active+clean; 670 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 1.0 MiB/s wr, 241 op/s 2024-09-11T00:27:47.243 INFO:tasks.workunit.client.0.smithi012.stdout: 10 131026 15.38 MB/sec execute 343 sec latency 12023.649 ms 2024-09-11T00:27:48.243 INFO:tasks.workunit.client.0.smithi012.stdout: 10 131026 15.33 MB/sec execute 344 sec latency 13023.714 ms 2024-09-11T00:27:48.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:27:47 smithi165 ceph-mon[35356]: pgmap v1020: 129 pgs: 129 active+clean; 670 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 1.0 MiB/s wr, 244 op/s 2024-09-11T00:27:48.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:27:47 smithi012 ceph-mon[30333]: pgmap v1020: 129 pgs: 129 active+clean; 670 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 1.0 MiB/s wr, 244 op/s 2024-09-11T00:27:48.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:27:47 smithi178 ceph-mon[35919]: pgmap v1020: 129 pgs: 129 active+clean; 670 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 1.0 MiB/s wr, 244 op/s 2024-09-11T00:27:49.243 INFO:tasks.workunit.client.0.smithi012.stdout: 10 131026 15.29 MB/sec execute 345 sec latency 14023.850 ms 2024-09-11T00:27:50.243 INFO:tasks.workunit.client.0.smithi012.stdout: 10 131026 15.25 MB/sec execute 346 sec latency 15023.952 ms 2024-09-11T00:27:50.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:27:49 smithi165 ceph-mon[35356]: pgmap v1021: 129 pgs: 129 active+clean; 670 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 568 KiB/s rd, 278 KiB/s wr, 80 op/s 2024-09-11T00:27:50.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:27:49 smithi012 ceph-mon[30333]: pgmap v1021: 129 pgs: 129 active+clean; 670 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 568 KiB/s rd, 278 KiB/s wr, 80 op/s 2024-09-11T00:27:50.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:27:49 smithi178 ceph-mon[35919]: pgmap v1021: 129 pgs: 129 active+clean; 670 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 568 KiB/s rd, 278 KiB/s wr, 80 op/s 2024-09-11T00:27:51.247 INFO:tasks.workunit.client.0.smithi012.stdout: 10 131026 15.20 MB/sec execute 347 sec latency 16027.568 ms 2024-09-11T00:27:51.356 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:51.354+0000 7f0ad2c00640 1 -- 172.21.15.12:0/781904862 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0acc074290 msgr2=0x7f0acc074690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:27:51.356 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:51.354+0000 7f0ad2c00640 1 --2- 172.21.15.12:0/781904862 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0acc074290 0x7f0acc074690 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f0ab4009900 tx=0x7f0ab402a440 comp rx=0 tx=0).stop 2024-09-11T00:27:51.356 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:51.354+0000 7f0ad2c00640 1 -- 172.21.15.12:0/781904862 shutdown_connections 2024-09-11T00:27:51.356 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:51.354+0000 7f0ad2c00640 1 --2- 172.21.15.12:0/781904862 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0acc10ab10 0x7f0acc11a090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:27:51.356 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:51.354+0000 7f0ad2c00640 1 --2- 172.21.15.12:0/781904862 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0acc10a150 0x7f0acc10a5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:27:51.356 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:51.354+0000 7f0ad2c00640 1 --2- 172.21.15.12:0/781904862 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0acc074290 0x7f0acc074690 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:27:51.356 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:51.354+0000 7f0ad2c00640 1 -- 172.21.15.12:0/781904862 >> 172.21.15.12:0/781904862 conn(0x7f0acc06fa40 msgr2=0x7f0acc071e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:27:51.357 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:51.355+0000 7f0ad2c00640 1 -- 172.21.15.12:0/781904862 shutdown_connections 2024-09-11T00:27:51.357 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:51.355+0000 7f0ad2c00640 1 -- 172.21.15.12:0/781904862 wait complete. 2024-09-11T00:27:51.359 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:51.358+0000 7f0ad2c00640 1 Processor -- start 2024-09-11T00:27:51.359 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:51.358+0000 7f0ad2c00640 1 -- start start 2024-09-11T00:27:51.360 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:51.359+0000 7f0ad2c00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0acc10a150 0x7f0acc1a7f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:27:51.360 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:51.359+0000 7f0ad2c00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0acc10ab10 0x7f0acc1a8590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:27:51.360 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:51.359+0000 7f0ad2c00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0acc1a91c0 0x7f0acc1a9670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:27:51.361 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:51.359+0000 7f0ac3e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0acc10ab10 0x7f0acc1a8590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:27:51.361 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:51.359+0000 7f0acbe00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0acc10a150 0x7f0acc1a7f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:27:51.361 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:51.359+0000 7f0ac3e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0acc10ab10 0x7f0acc1a8590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.12:59446/0 (socket says 172.21.15.12:59446) 2024-09-11T00:27:51.362 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:51.360+0000 7f0acbe00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0acc10a150 0x7f0acc1a7f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.12:36704/0 (socket says 172.21.15.12:36704) 2024-09-11T00:27:51.362 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:51.360+0000 7f0acbe00640 1 -- 172.21.15.12:0/857589829 learned_addr learned my addr 172.21.15.12:0/857589829 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:27:51.362 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:51.359+0000 7f0ad2c00640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f0acc11d4e0 con 0x7f0acc1a91c0 2024-09-11T00:27:51.363 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:51.360+0000 7f0ad2c00640 1 -- 172.21.15.12:0/857589829 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f0acc11d360 con 0x7f0acc10ab10 2024-09-11T00:27:51.363 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:51.360+0000 7f0ad2c00640 1 -- 172.21.15.12:0/857589829 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f0acc11d660 con 0x7f0acc10a150 2024-09-11T00:27:51.363 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:51.360+0000 7f0ad0c00640 1 --2- 172.21.15.12:0/857589829 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0acc1a91c0 0x7f0acc1a9670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:27:51.363 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:51.360+0000 7f0acbe00640 1 -- 172.21.15.12:0/857589829 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0acc10ab10 msgr2=0x7f0acc1a8590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:27:51.363 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:51.360+0000 7f0acbe00640 1 --2- 172.21.15.12:0/857589829 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0acc10ab10 0x7f0acc1a8590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:27:51.363 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:51.360+0000 7f0acbe00640 1 -- 172.21.15.12:0/857589829 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0acc1a91c0 msgr2=0x7f0acc1a9670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:27:51.363 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:51.360+0000 7f0acbe00640 1 --2- 172.21.15.12:0/857589829 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0acc1a91c0 0x7f0acc1a9670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:27:51.363 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:51.360+0000 7f0acbe00640 1 -- 172.21.15.12:0/857589829 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0acc1a9e70 con 0x7f0acc10a150 2024-09-11T00:27:51.363 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:51.360+0000 7f0ac3e00640 1 --2- 172.21.15.12:0/857589829 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0acc10ab10 0x7f0acc1a8590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello state changed while learned_addr, mark_down or replacing must be happened just now 2024-09-11T00:27:51.363 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:51.361+0000 7f0acbe00640 1 --2- 172.21.15.12:0/857589829 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0acc10a150 0x7f0acc1a7f40 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f0ab4005940 tx=0x7f0ab40043c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:27:51.364 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:51.362+0000 7f0ac9600640 1 -- 172.21.15.12:0/857589829 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ab403f030 con 0x7f0acc10a150 2024-09-11T00:27:51.364 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:51.362+0000 7f0ac9600640 1 -- 172.21.15.12:0/857589829 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0ab4003cb0 con 0x7f0acc10a150 2024-09-11T00:27:51.364 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:51.362+0000 7f0ac9600640 1 -- 172.21.15.12:0/857589829 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ab4035070 con 0x7f0acc10a150 2024-09-11T00:27:51.365 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:51.362+0000 7f0ad2c00640 1 -- 172.21.15.12:0/857589829 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0acc1b0260 con 0x7f0acc10a150 2024-09-11T00:27:51.365 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:51.362+0000 7f0ad2c00640 1 -- 172.21.15.12:0/857589829 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0acc1b06f0 con 0x7f0acc10a150 2024-09-11T00:27:51.368 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:51.366+0000 7f0ad5a0c740 1 Processor -- start 2024-09-11T00:27:51.368 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:51.367+0000 7f0ad5a0c740 1 -- start start 2024-09-11T00:27:51.368 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:51.367+0000 7f0ad5a0c740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56140caf1790 0x56140caf3b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:27:51.369 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:51.367+0000 7f0ad0c00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56140caf1790 0x56140caf3b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:27:51.369 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:51.367+0000 7f0ad5a0c740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56140caf40c0 0x56140caf64b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:27:51.369 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:51.367+0000 7f0ad5a0c740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56140caf6b00 0x56140caf8fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:27:51.369 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:51.367+0000 7f0ad5a0c740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x56140ca646b0 con 0x56140caf1790 2024-09-11T00:27:51.369 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:51.367+0000 7f0ad5a0c740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x56140ca53ae0 con 0x56140caf40c0 2024-09-11T00:27:51.369 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:51.367+0000 7f0ad5a0c740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x56140ca578f0 con 0x56140caf6b00 2024-09-11T00:27:51.369 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:51.368+0000 7f0ad0c00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56140caf1790 0x56140caf3b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:33628/0 (socket says 172.21.15.12:33628) 2024-09-11T00:27:51.369 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:51.368+0000 7f0ad0c00640 1 -- 172.21.15.12:0/2072391613 learned_addr learned my addr 172.21.15.12:0/2072391613 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:27:51.370 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:51.368+0000 7f0ac9600640 1 -- 172.21.15.12:0/857589829 <== mon.1 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7f0ab403bbc0 con 0x7f0acc10a150 2024-09-11T00:27:51.370 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:51.368+0000 7f0acbe00640 1 --2- 172.21.15.12:0/2072391613 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56140caf40c0 0x56140caf64b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:27:51.370 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:51.368+0000 7f0ad0c00640 1 -- 172.21.15.12:0/2072391613 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56140caf6b00 msgr2=0x56140caf8fc0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-11T00:27:51.370 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:51.368+0000 7f0ad0c00640 1 --2- 172.21.15.12:0/2072391613 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56140caf6b00 0x56140caf8fc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:27:51.370 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:51.368+0000 7f0ad0c00640 1 -- 172.21.15.12:0/2072391613 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56140caf40c0 msgr2=0x56140caf64b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:27:51.370 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:51.368+0000 7f0ad0c00640 1 --2- 172.21.15.12:0/2072391613 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56140caf40c0 0x56140caf64b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:27:51.370 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:51.368+0000 7f0ad0c00640 1 -- 172.21.15.12:0/2072391613 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56140ca20780 con 0x56140caf1790 2024-09-11T00:27:51.371 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:51.368+0000 7f0acbe00640 1 --2- 172.21.15.12:0/2072391613 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56140caf40c0 0x56140caf64b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T00:27:51.371 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:51.369+0000 7f0ad0c00640 1 --2- 172.21.15.12:0/2072391613 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56140caf1790 0x56140caf3b80 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f0ac4007d90 tx=0x7f0ac400c540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2d52a54a64b34bc server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:27:51.372 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:51.371+0000 7f0ac1200640 1 -- 172.21.15.12:0/2072391613 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ac4015020 con 0x56140caf1790 2024-09-11T00:27:51.373 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:51.371+0000 7f0ac9600640 1 --2- 172.21.15.12:0/857589829 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f0ab00b5360 0x7f0ab00b7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:27:51.373 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:51.372+0000 7f0ac1200640 1 -- 172.21.15.12:0/2072391613 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0ac400ce50 con 0x56140caf1790 2024-09-11T00:27:51.373 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:51.372+0000 7f0ac1200640 1 -- 172.21.15.12:0/2072391613 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ac400d040 con 0x56140caf1790 2024-09-11T00:27:51.374 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:51.373+0000 7f0ac9600640 1 -- 172.21.15.12:0/857589829 <== mon.1 v2:172.21.15.178:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9208+0+0 (secure 0 0 0) 0x7f0ab4106960 con 0x7f0acc10a150 2024-09-11T00:27:51.374 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:51.373+0000 7f0ad5a0c740 1 -- 172.21.15.12:0/2072391613 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56140caf1790 msgr2=0x56140caf3b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:27:51.374 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:51.373+0000 7f0ad5a0c740 1 --2- 172.21.15.12:0/2072391613 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56140caf1790 0x56140caf3b80 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f0ac4007d90 tx=0x7f0ac400c540 comp rx=0 tx=0).stop 2024-09-11T00:27:51.375 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:51.373+0000 7f0ad5a0c740 1 -- 172.21.15.12:0/2072391613 shutdown_connections 2024-09-11T00:27:51.375 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:51.374+0000 7f0acbe00640 1 --2- 172.21.15.12:0/857589829 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f0ab00b5360 0x7f0ab00b7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:27:51.375 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:51.374+0000 7f0ad5a0c740 1 --2- 172.21.15.12:0/2072391613 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56140caf6b00 0x56140caf8fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:27:51.375 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:51.374+0000 7f0ad5a0c740 1 --2- 172.21.15.12:0/2072391613 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56140caf40c0 0x56140caf64b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:27:51.375 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:51.374+0000 7f0ad5a0c740 1 --2- 172.21.15.12:0/2072391613 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56140caf1790 0x56140caf3b80 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:27:51.376 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:51.374+0000 7f0ad5a0c740 1 -- 172.21.15.12:0/2072391613 >> 172.21.15.12:0/2072391613 conn(0x56140caec8e0 msgr2=0x56140caed670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:27:51.376 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:51.375+0000 7f0ad5a0c740 1 -- 172.21.15.12:0/2072391613 shutdown_connections 2024-09-11T00:27:51.377 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:51.375+0000 7f0acbe00640 1 --2- 172.21.15.12:0/857589829 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f0ab00b5360 0x7f0ab00b7820 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f0ab4106c00 tx=0x7f0ab403b640 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:27:51.377 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:51.375+0000 7f0ad5a0c740 1 -- 172.21.15.12:0/2072391613 wait complete. 2024-09-11T00:27:51.377 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:51.376+0000 7f0ad5a0c740 1 -- start start 2024-09-11T00:27:51.377 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:51.376+0000 7f0ad5a0c740 1 Processor -- start 2024-09-11T00:27:51.379 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:51.377+0000 7f0ad5a0c740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56140caf1790 0x56140cafaf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:27:51.379 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:51.377+0000 7f0ad5a0c740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56140caf40c0 0x56140cafb620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:27:51.379 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:51.377+0000 7f0ad5a0c740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56140caf6b00 0x56140cafc180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:27:51.379 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:51.377+0000 7f0ad5a0c740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x56140c98bab0 con 0x56140caf40c0 2024-09-11T00:27:51.379 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:51.377+0000 7f0ad5a0c740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x56140c9fa570 con 0x56140caf1790 2024-09-11T00:27:51.379 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:51.377+0000 7f0ad5a0c740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x56140caf9ae0 con 0x56140caf6b00 2024-09-11T00:27:51.379 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:51.377+0000 7f0ad5a0c740 10 client.0 _ll_register_callbacks cb 0x56140ca09100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:27:51.380 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:51.378+0000 7f0ad0c00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56140caf6b00 0x56140cafc180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:27:51.380 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:51.378+0000 7f0ad0c00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56140caf6b00 0x56140cafc180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.12:36712/0 (socket says 172.21.15.12:36712) 2024-09-11T00:27:51.380 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:51.378+0000 7f0ad0c00640 1 -- 172.21.15.12:0/1079434291 learned_addr learned my addr 172.21.15.12:0/1079434291 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:27:51.380 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:51.378+0000 7f0ad0c00640 1 --2- 172.21.15.12:0/1079434291 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56140caf1790 0x56140cafaf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:27:51.380 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:51.379+0000 7f0ac3e00640 1 --2- 172.21.15.12:0/1079434291 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56140caf40c0 0x56140cafb620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:27:51.381 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:51.379+0000 7f0ad0c00640 1 -- 172.21.15.12:0/1079434291 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56140caf1790 msgr2=0x56140cafaf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:27:51.381 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:51.379+0000 7f0ad0c00640 1 --2- 172.21.15.12:0/1079434291 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56140caf1790 0x56140cafaf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:27:51.381 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:51.379+0000 7f0ad0c00640 1 -- 172.21.15.12:0/1079434291 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56140caf40c0 msgr2=0x56140cafb620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:27:51.381 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:51.379+0000 7f0ad0c00640 1 --2- 172.21.15.12:0/1079434291 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56140caf40c0 0x56140cafb620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:27:51.381 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:51.379+0000 7f0ad0c00640 1 -- 172.21.15.12:0/1079434291 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56140c7b5790 con 0x56140caf6b00 2024-09-11T00:27:51.381 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:51.379+0000 7f0ac3e00640 1 --2- 172.21.15.12:0/1079434291 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56140caf40c0 0x56140cafb620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T00:27:51.383 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:51.379+0000 7f0ad0c00640 1 --2- 172.21.15.12:0/1079434291 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56140caf6b00 0x56140cafc180 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f0ac4011f10 tx=0x7f0ac40072c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b4703aaf967b8b81 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:27:51.383 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:51.380+0000 7f0ab9600640 10 client.0 ms_handle_connect on v2:172.21.15.178:3300/0 2024-09-11T00:27:51.383 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:51.380+0000 7f0ab9600640 1 -- 172.21.15.12:0/1079434291 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ac4015030 con 0x56140caf6b00 2024-09-11T00:27:51.383 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:51.380+0000 7f0ab9600640 1 -- 172.21.15.12:0/1079434291 <== mon.1 v2:172.21.15.178:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9208+0+0 (secure 0 0 0) 0x7f0ac4010600 con 0x56140caf6b00 2024-09-11T00:27:51.383 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:51.380+0000 7f0ad5a0c740 1 -- 172.21.15.12:0/1079434291 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0ac400ce50 con 0x56140caf6b00 2024-09-11T00:27:51.383 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:51.380+0000 7f0ab9600640 1 -- 172.21.15.12:0/1079434291 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=86) ==== 24+0+0 (secure 0 0 0) 0x7f0ac4032050 con 0x56140caf6b00 2024-09-11T00:27:51.383 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:51.380+0000 7f0ad5a0c740 10 client.27706 fetch_fsmap learned FSMap version 86 2024-09-11T00:27:51.384 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:51.380+0000 7f0ad5a0c740 1 -- 172.21.15.12:0/1079434291 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,fsmap=86,monmap=4+}) -- 0x56140c82e620 con 0x56140caf6b00 2024-09-11T00:27:51.384 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:51.381+0000 7f0ab9600640 1 -- 172.21.15.12:0/1079434291 <== mon.1 v2:172.21.15.178:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0ac400ce50 con 0x56140caf6b00 2024-09-11T00:27:51.384 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:51.381+0000 7f0ab9600640 1 -- 172.21.15.12:0/1079434291 <== mon.1 v2:172.21.15.178:3300/0 5 ==== fsmap(e 86) ==== 8438+0+0 (secure 0 0 0) 0x7f0ac4010050 con 0x56140caf6b00 2024-09-11T00:27:51.384 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:51.381+0000 7f0ad5a0c740 10 client.27706 fetch_fsmap finished waiting for FSMap version 86 2024-09-11T00:27:51.384 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:51.381+0000 7f0ad5a0c740 10 client.27706 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T00:27:51.384 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:51.381+0000 7f0ad5a0c740 20 client.27706 populate_metadata read hostname 'smithi012' 2024-09-11T00:27:51.384 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:51.381+0000 7f0ad5a0c740 1 --2- 172.21.15.12:0/1079434291 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x56140cb06610 0x56140cb08a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:27:51.384 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:51.381+0000 7f0ad5a0c740 4 client.27706 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T00:27:51.384 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:51.381+0000 7f0ad5a0c740 1 -- 172.21.15.12:0/1079434291 --> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56140ca66600 con 0x56140cb06610 2024-09-11T00:27:51.384 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:51.382+0000 7f0ac3e00640 1 --2- 172.21.15.12:0/1079434291 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x56140cb06610 0x56140cb08a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:27:51.407 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:51.406+0000 7f0ac3e00640 1 --2- 172.21.15.12:0/1079434291 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x56140cb06610 0x56140cb08a30 crc :-1 s=READY pgs=287 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=44cf2fb5e94ef824 server_cookie=23e536e2ab586d17 in_seq=0 out_seq=0 2024-09-11T00:27:51.407 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:51.406+0000 7f0ab9600640 10 client.27706 ms_handle_connect on v2:172.21.15.165:6838/501312984 2024-09-11T00:27:51.408 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:51.406+0000 7f0ab9600640 1 -- 172.21.15.12:0/1079434291 <== mds.0 v2:172.21.15.165:6838/501312984 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x56140ca66600 con 0x56140cb06610 2024-09-11T00:27:51.408 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:51.406+0000 7f0ab9600640 10 client.27706 handle_command_reply: tid=0 2024-09-11T00:27:51.408 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:51.407+0000 7f0ad5a0c740 1 client.27706 shutdown 2024-09-11T00:27:51.409 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:51.407+0000 7f0ad5a0c740 1 -- 172.21.15.12:0/1079434291 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56140caf6b00 msgr2=0x56140cafc180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:27:51.409 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:51.407+0000 7f0ad5a0c740 1 --2- 172.21.15.12:0/1079434291 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56140caf6b00 0x56140cafc180 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f0ac4011f10 tx=0x7f0ac40072c0 comp rx=0 tx=0).stop 2024-09-11T00:27:51.409 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:51.407+0000 7f0ad5a0c740 1 -- 172.21.15.12:0/1079434291 shutdown_connections 2024-09-11T00:27:51.409 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:51.407+0000 7f0ad5a0c740 1 --2- 172.21.15.12:0/1079434291 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x56140cb06610 0x56140cb08a30 crc :-1 s=READY pgs=287 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:27:51.409 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:51.407+0000 7f0ad5a0c740 1 --2- 172.21.15.12:0/1079434291 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56140caf6b00 0x56140cafc180 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:27:51.409 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:51.407+0000 7f0ad5a0c740 1 --2- 172.21.15.12:0/1079434291 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56140caf40c0 0x56140cafb620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:27:51.409 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:51.407+0000 7f0ab9600640 0 client.27706 ms_handle_reset on v2:172.21.15.165:6838/501312984 2024-09-11T00:27:51.409 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:51.407+0000 7f0ad5a0c740 1 --2- 172.21.15.12:0/1079434291 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56140caf1790 0x56140cafaf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:27:51.409 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:51.407+0000 7f0ad5a0c740 1 -- 172.21.15.12:0/1079434291 >> 172.21.15.12:0/1079434291 conn(0x56140caec8e0 msgr2=0x56140caeba60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:27:51.410 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:51.407+0000 7f0ad5a0c740 1 -- 172.21.15.12:0/1079434291 shutdown_connections 2024-09-11T00:27:51.410 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:51.407+0000 7f0ad5a0c740 1 -- 172.21.15.12:0/1079434291 wait complete. 2024-09-11T00:27:51.410 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:51.407+0000 7f0ad5a0c740 20 client.27706 trim_cache size 0 max 16384 2024-09-11T00:27:51.426 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:51.423+0000 7f0ad5a0c740 1 Processor -- start 2024-09-11T00:27:51.426 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:51.423+0000 7f0ad5a0c740 1 -- start start 2024-09-11T00:27:51.426 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:51.423+0000 7f0ad5a0c740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56140caec8e0 0x56140caf75f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:27:51.426 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:51.423+0000 7f0ad5a0c740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56140caf1790 0x56140cb02a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:27:51.426 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:51.423+0000 7f0ad5a0c740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56140caf40c0 0x56140caef720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:27:51.427 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:51.423+0000 7f0ad5a0c740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f0acc1bacd0 con 0x56140caec8e0 2024-09-11T00:27:51.427 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:51.423+0000 7f0ad5a0c740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f0acc1bca50 con 0x56140caf40c0 2024-09-11T00:27:51.427 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:51.423+0000 7f0ad5a0c740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f0acc1bb460 con 0x56140caf1790 2024-09-11T00:27:51.427 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:51.424+0000 7f0ad0c00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56140caec8e0 0x56140caf75f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:27:51.427 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:51.424+0000 7f0ad0c00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56140caec8e0 0x56140caf75f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:33640/0 (socket says 172.21.15.12:33640) 2024-09-11T00:27:51.427 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:51.424+0000 7f0ad0c00640 1 -- 172.21.15.12:0/894540906 learned_addr learned my addr 172.21.15.12:0/894540906 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:27:51.427 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:51.424+0000 7f0ac3e00640 1 --2- 172.21.15.12:0/894540906 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56140caf1790 0x56140cb02a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:27:51.427 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:51.424+0000 7f0ad0c00640 1 --2- 172.21.15.12:0/894540906 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56140caf40c0 0x56140caef720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:27:51.427 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:51.424+0000 7f0ad0c00640 1 -- 172.21.15.12:0/894540906 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56140caf1790 msgr2=0x56140cb02a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:27:51.427 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:51.424+0000 7f0ad0c00640 1 --2- 172.21.15.12:0/894540906 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56140caf1790 0x56140cb02a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:27:51.428 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:51.424+0000 7f0ad0c00640 1 -- 172.21.15.12:0/894540906 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56140caf40c0 msgr2=0x56140caef720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:27:51.428 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:51.424+0000 7f0ad0c00640 1 --2- 172.21.15.12:0/894540906 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56140caf40c0 0x56140caef720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:27:51.428 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:51.424+0000 7f0ad0c00640 1 -- 172.21.15.12:0/894540906 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56140cafaa80 con 0x56140caec8e0 2024-09-11T00:27:51.428 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:51.424+0000 7f0ad0c00640 1 --2- 172.21.15.12:0/894540906 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56140caec8e0 0x56140caf75f0 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f0ac4010b90 tx=0x7f0ac4007330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b361edb9f6f7aaf5 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:27:51.428 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:51.425+0000 7f0abbe00640 1 -- 172.21.15.12:0/894540906 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ac402f020 con 0x56140caec8e0 2024-09-11T00:27:51.428 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:51.425+0000 7f0abbe00640 1 -- 172.21.15.12:0/894540906 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0ac400ca40 con 0x56140caec8e0 2024-09-11T00:27:51.428 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:51.425+0000 7f0abbe00640 1 -- 172.21.15.12:0/894540906 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ac4015070 con 0x56140caec8e0 2024-09-11T00:27:51.431 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:51.425+0000 7f0ad5a0c740 1 -- 172.21.15.12:0/894540906 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56140caec8e0 msgr2=0x56140caf75f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:27:51.431 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:51.425+0000 7f0ad5a0c740 1 --2- 172.21.15.12:0/894540906 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56140caec8e0 0x56140caf75f0 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f0ac4010b90 tx=0x7f0ac4007330 comp rx=0 tx=0).stop 2024-09-11T00:27:51.431 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:51.425+0000 7f0ad5a0c740 1 -- 172.21.15.12:0/894540906 shutdown_connections 2024-09-11T00:27:51.431 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:51.425+0000 7f0ad5a0c740 1 --2- 172.21.15.12:0/894540906 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56140caf40c0 0x56140caef720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:27:51.432 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:51.425+0000 7f0ad5a0c740 1 --2- 172.21.15.12:0/894540906 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56140caf1790 0x56140cb02a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:27:51.432 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:51.426+0000 7f0ad5a0c740 1 --2- 172.21.15.12:0/894540906 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56140caec8e0 0x56140caf75f0 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:27:51.432 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:51.426+0000 7f0ad5a0c740 1 -- 172.21.15.12:0/894540906 >> 172.21.15.12:0/894540906 conn(0x56140caec500 msgr2=0x56140caf6ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:27:51.432 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:51.426+0000 7f0ad5a0c740 1 -- 172.21.15.12:0/894540906 shutdown_connections 2024-09-11T00:27:51.432 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:51.426+0000 7f0ad5a0c740 1 -- 172.21.15.12:0/894540906 wait complete. 2024-09-11T00:27:51.432 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:51.426+0000 7f0ad5a0c740 1 -- start start 2024-09-11T00:27:51.432 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:51.426+0000 7f0ad5a0c740 1 Processor -- start 2024-09-11T00:27:51.432 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:51.426+0000 7f0ad5a0c740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56140caec8e0 0x56140cb02c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:27:51.433 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:51.427+0000 7f0ad5a0c740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56140caf1790 0x56140cb019b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:27:51.433 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:51.427+0000 7f0ad5a0c740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56140caf40c0 0x56140cb020a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:27:51.433 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:51.427+0000 7f0ad5a0c740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f0acc1bc380 con 0x56140caf40c0 2024-09-11T00:27:51.433 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:51.427+0000 7f0ad5a0c740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f0acc1bbbf0 con 0x56140caec8e0 2024-09-11T00:27:51.433 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:51.427+0000 7f0ad5a0c740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f0acc1b8cb0 con 0x56140caf1790 2024-09-11T00:27:51.433 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:51.427+0000 7f0ad5a0c740 10 client.0 _ll_register_callbacks cb 0x7f0ac4007be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:27:51.433 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:51.427+0000 7f0ad0c00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56140caf40c0 0x56140cb020a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:27:51.433 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:51.427+0000 7f0ad0c00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56140caec8e0 0x56140cb02c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:27:51.433 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:51.427+0000 7f0ac3e00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56140caf1790 0x56140cb019b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:27:51.433 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:51.427+0000 7f0ad0c00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56140caec8e0 0x56140cb02c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.12:59490/0 (socket says 172.21.15.12:59490) 2024-09-11T00:27:51.433 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:51.427+0000 7f0ad0c00640 1 -- 172.21.15.12:0/2588855268 learned_addr learned my addr 172.21.15.12:0/2588855268 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:27:51.433 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:51.427+0000 7f0ac3e00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56140caf1790 0x56140cb019b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.12:36718/0 (socket says 172.21.15.12:36718) 2024-09-11T00:27:51.434 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:51.427+0000 7f0ac3e00640 1 -- 172.21.15.12:0/2588855268 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56140caec8e0 msgr2=0x56140cb02c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:27:51.434 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:51.427+0000 7f0ac3e00640 1 --2- 172.21.15.12:0/2588855268 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56140caec8e0 0x56140cb02c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:27:51.434 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:51.427+0000 7f0ac3e00640 1 -- 172.21.15.12:0/2588855268 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56140caf40c0 msgr2=0x56140cb020a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:27:51.434 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:51.427+0000 7f0ac3e00640 1 --2- 172.21.15.12:0/2588855268 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56140caf40c0 0x56140cb020a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:27:51.434 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:51.427+0000 7f0ac3e00640 1 -- 172.21.15.12:0/2588855268 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56140cb00450 con 0x56140caf1790 2024-09-11T00:27:51.434 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:51.428+0000 7f0ac3e00640 1 --2- 172.21.15.12:0/2588855268 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56140caf1790 0x56140cb019b0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x56140cb09030 tx=0x7f0abc00bef0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6fdd8c1bb4be4d70 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:27:51.434 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:51.428+0000 7f0ab9600640 10 client.0 ms_handle_connect on v2:172.21.15.178:3300/0 2024-09-11T00:27:51.434 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:51.428+0000 7f0ab9600640 1 -- 172.21.15.12:0/2588855268 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0abc01f030 con 0x56140caf1790 2024-09-11T00:27:51.434 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:51.428+0000 7f0ad5a0c740 1 -- 172.21.15.12:0/2588855268 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0ac400ca40 con 0x56140caf1790 2024-09-11T00:27:51.434 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:51.428+0000 7f0ab9600640 1 -- 172.21.15.12:0/2588855268 <== mon.1 v2:172.21.15.178:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9208+0+0 (secure 0 0 0) 0x7f0abc016ce0 con 0x56140caf1790 2024-09-11T00:27:51.434 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:51.428+0000 7f0ab9600640 1 -- 172.21.15.12:0/2588855268 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=86) ==== 24+0+0 (secure 0 0 0) 0x7f0abc02c050 con 0x56140caf1790 2024-09-11T00:27:51.434 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:51.428+0000 7f0ad5a0c740 10 client.27712 fetch_fsmap learned FSMap version 86 2024-09-11T00:27:51.434 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:51.428+0000 7f0ad5a0c740 1 -- 172.21.15.12:0/2588855268 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,fsmap=86,monmap=4+}) -- 0x56140cb73860 con 0x56140caf1790 2024-09-11T00:27:51.435 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:51.429+0000 7f0ab9600640 1 -- 172.21.15.12:0/2588855268 <== mon.1 v2:172.21.15.178:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0ac400ca40 con 0x56140caf1790 2024-09-11T00:27:51.435 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:51.429+0000 7f0ab9600640 1 -- 172.21.15.12:0/2588855268 <== mon.1 v2:172.21.15.178:3300/0 5 ==== fsmap(e 86) ==== 8438+0+0 (secure 0 0 0) 0x7f0abc01b480 con 0x56140caf1790 2024-09-11T00:27:51.435 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:51.429+0000 7f0ad5a0c740 10 client.27712 fetch_fsmap finished waiting for FSMap version 86 2024-09-11T00:27:51.435 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:51.429+0000 7f0ad5a0c740 10 client.27712 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T00:27:51.435 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:51.429+0000 7f0ad5a0c740 20 client.27712 populate_metadata read hostname 'smithi012' 2024-09-11T00:27:51.435 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:51.429+0000 7f0ad5a0c740 1 --2- 172.21.15.12:0/2588855268 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x56140caf6b00 0x56140cb76340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:27:51.435 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:51.429+0000 7f0ad5a0c740 4 client.27712 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T00:27:51.435 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:51.429+0000 7f0ad5a0c740 1 -- 172.21.15.12:0/2588855268 --> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56140ca620b0 con 0x56140caf6b00 2024-09-11T00:27:51.435 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:51.429+0000 7f0ad0c00640 1 --2- 172.21.15.12:0/2588855268 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x56140caf6b00 0x56140cb76340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:27:51.435 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:51.430+0000 7f0ad0c00640 1 --2- 172.21.15.12:0/2588855268 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x56140caf6b00 0x56140cb76340 crc :-1 s=READY pgs=288 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a67bb4966d2deef7 server_cookie=55ec4c2cc14a6abe in_seq=0 out_seq=0 2024-09-11T00:27:51.435 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:51.430+0000 7f0ab9600640 10 client.27712 ms_handle_connect on v2:172.21.15.165:6838/501312984 2024-09-11T00:27:51.553 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:51.548+0000 7f0ab9600640 1 -- 172.21.15.12:0/2588855268 <== mds.0 v2:172.21.15.165:6838/501312984 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56140ca620b0 con 0x56140caf6b00 2024-09-11T00:27:51.553 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:51.548+0000 7f0ab9600640 10 client.27712 handle_command_reply: tid=0 2024-09-11T00:27:51.553 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:51.548+0000 7f0ad5a0c740 1 client.27712 shutdown 2024-09-11T00:27:51.553 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:51.548+0000 7f0ad5a0c740 1 -- 172.21.15.12:0/2588855268 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56140caf1790 msgr2=0x56140cb019b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:27:51.553 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:51.548+0000 7f0ad5a0c740 1 --2- 172.21.15.12:0/2588855268 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56140caf1790 0x56140cb019b0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x56140cb09030 tx=0x7f0abc00bef0 comp rx=0 tx=0).stop 2024-09-11T00:27:51.554 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:51.548+0000 7f0ad5a0c740 1 -- 172.21.15.12:0/2588855268 shutdown_connections 2024-09-11T00:27:51.554 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:51.548+0000 7f0ad5a0c740 1 --2- 172.21.15.12:0/2588855268 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x56140caf6b00 0x56140cb76340 crc :-1 s=READY pgs=288 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:27:51.554 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:51.548+0000 7f0ad5a0c740 1 --2- 172.21.15.12:0/2588855268 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56140caf40c0 0x56140cb020a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:27:51.554 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:51.548+0000 7f0ad5a0c740 1 --2- 172.21.15.12:0/2588855268 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56140caf1790 0x56140cb019b0 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:27:51.554 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:51.548+0000 7f0ab9600640 0 client.27712 ms_handle_reset on v2:172.21.15.165:6838/501312984 2024-09-11T00:27:51.554 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:51.549+0000 7f0ad5a0c740 1 --2- 172.21.15.12:0/2588855268 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56140caec8e0 0x56140cb02c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:27:51.554 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:51.549+0000 7f0ad5a0c740 1 -- 172.21.15.12:0/2588855268 >> 172.21.15.12:0/2588855268 conn(0x56140caec500 msgr2=0x56140caeb240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:27:51.554 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:51.549+0000 7f0ad5a0c740 1 -- 172.21.15.12:0/2588855268 shutdown_connections 2024-09-11T00:27:51.554 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:51.549+0000 7f0ad5a0c740 1 -- 172.21.15.12:0/2588855268 wait complete. 2024-09-11T00:27:51.554 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:51.549+0000 7f0ad5a0c740 20 client.27712 trim_cache size 0 max 16384 2024-09-11T00:27:51.555 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:{ 2024-09-11T00:27:51.555 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "message": "", 2024-09-11T00:27:51.555 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "return_code": 0 2024-09-11T00:27:51.555 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:} 2024-09-11T00:27:51.555 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:51.550+0000 7f0ad2c00640 1 -- 172.21.15.12:0/857589829 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f0ab00b5360 msgr2=0x7f0ab00b7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:27:51.556 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:51.550+0000 7f0ad2c00640 1 --2- 172.21.15.12:0/857589829 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f0ab00b5360 0x7f0ab00b7820 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f0ab4106c00 tx=0x7f0ab403b640 comp rx=0 tx=0).stop 2024-09-11T00:27:51.556 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:51.550+0000 7f0ad2c00640 1 -- 172.21.15.12:0/857589829 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0acc10a150 msgr2=0x7f0acc1a7f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:27:51.556 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:51.550+0000 7f0ad2c00640 1 --2- 172.21.15.12:0/857589829 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0acc10a150 0x7f0acc1a7f40 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f0ab4005940 tx=0x7f0ab40043c0 comp rx=0 tx=0).stop 2024-09-11T00:27:51.556 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:51.550+0000 7f0ad2c00640 1 -- 172.21.15.12:0/857589829 shutdown_connections 2024-09-11T00:27:51.556 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:51.550+0000 7f0ad2c00640 1 --2- 172.21.15.12:0/857589829 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f0ab00b5360 0x7f0ab00b7820 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:27:51.556 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:51.550+0000 7f0ad2c00640 1 --2- 172.21.15.12:0/857589829 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0acc1a91c0 0x7f0acc1a9670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:27:51.556 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:51.550+0000 7f0ad2c00640 1 --2- 172.21.15.12:0/857589829 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0acc10ab10 0x7f0acc1a8590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:27:51.556 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:51.550+0000 7f0ad2c00640 1 --2- 172.21.15.12:0/857589829 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0acc10a150 0x7f0acc1a7f40 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:27:51.556 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:51.550+0000 7f0ad2c00640 1 -- 172.21.15.12:0/857589829 >> 172.21.15.12:0/857589829 conn(0x7f0acc06fa40 msgr2=0x7f0acc071710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:27:51.556 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:51.550+0000 7f0ad2c00640 1 -- 172.21.15.12:0/857589829 shutdown_connections 2024-09-11T00:27:51.556 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:27:51.550+0000 7f0ad2c00640 1 -- 172.21.15.12:0/857589829 wait complete. 2024-09-11T00:27:52.247 INFO:tasks.workunit.client.0.smithi012.stdout: 10 131026 15.16 MB/sec execute 348 sec latency 17027.630 ms 2024-09-11T00:27:52.255 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:27:51 smithi012 ceph-mon[30333]: pgmap v1022: 129 pgs: 129 active+clean; 670 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 353 KiB/s rd, 197 KiB/s wr, 61 op/s 2024-09-11T00:27:52.283 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:27:51 smithi165 ceph-mon[35356]: pgmap v1022: 129 pgs: 129 active+clean; 670 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 353 KiB/s rd, 197 KiB/s wr, 61 op/s 2024-09-11T00:27:52.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:27:51 smithi178 ceph-mon[35919]: pgmap v1022: 129 pgs: 129 active+clean; 670 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 353 KiB/s rd, 197 KiB/s wr, 61 op/s 2024-09-11T00:27:53.247 INFO:tasks.workunit.client.0.smithi012.stdout: 10 131026 15.11 MB/sec execute 349 sec latency 18027.771 ms 2024-09-11T00:27:54.247 INFO:tasks.workunit.client.0.smithi012.stdout: 10 131026 15.07 MB/sec execute 350 sec latency 19027.922 ms 2024-09-11T00:27:54.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:27:53 smithi165 ceph-mon[35356]: pgmap v1023: 129 pgs: 129 active+clean; 671 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 181 KiB/s wr, 37 op/s 2024-09-11T00:27:54.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:27:53 smithi012 ceph-mon[30333]: pgmap v1023: 129 pgs: 129 active+clean; 671 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 181 KiB/s wr, 37 op/s 2024-09-11T00:27:54.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:27:53 smithi178 ceph-mon[35919]: pgmap v1023: 129 pgs: 129 active+clean; 671 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 181 KiB/s wr, 37 op/s 2024-09-11T00:27:55.247 INFO:tasks.workunit.client.0.smithi012.stdout: 10 131026 15.03 MB/sec execute 351 sec latency 20028.058 ms 2024-09-11T00:27:56.248 INFO:tasks.workunit.client.0.smithi012.stdout: 10 131026 14.99 MB/sec execute 352 sec latency 21028.136 ms 2024-09-11T00:27:56.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:27:56 smithi165 ceph-mon[35356]: pgmap v1024: 129 pgs: 129 active+clean; 672 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 259 KiB/s wr, 22 op/s 2024-09-11T00:27:56.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:27:56 smithi012 ceph-mon[30333]: pgmap v1024: 129 pgs: 129 active+clean; 672 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 259 KiB/s wr, 22 op/s 2024-09-11T00:27:56.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:27:56 smithi178 ceph-mon[35919]: pgmap v1024: 129 pgs: 129 active+clean; 672 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 259 KiB/s wr, 22 op/s 2024-09-11T00:27:57.248 INFO:tasks.workunit.client.0.smithi012.stdout: 10 131026 14.94 MB/sec execute 353 sec latency 22028.194 ms 2024-09-11T00:27:57.283 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:27:57 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T00:27:57.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:27:57 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T00:27:57.344 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:27:57 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T00:27:58.248 INFO:tasks.workunit.client.0.smithi012.stdout: 10 131026 14.90 MB/sec execute 354 sec latency 23028.259 ms 2024-09-11T00:27:58.533 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:27:58 smithi165 ceph-mon[35356]: pgmap v1025: 129 pgs: 129 active+clean; 677 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 1.7 MiB/s wr, 77 op/s 2024-09-11T00:27:58.533 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:27:58 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:27:58.533 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:27:58 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:27:58.533 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:27:58 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:27:58.533 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:27:58 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:27:58.533 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:27:58 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:27:58.533 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:27:58 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:27:58.533 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:27:58 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:27:58.533 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:27:58 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T00:27:58.534 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:27:58 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:27:58.563 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:27:58 smithi012 ceph-mon[30333]: pgmap v1025: 129 pgs: 129 active+clean; 677 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 1.7 MiB/s wr, 77 op/s 2024-09-11T00:27:58.563 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:27:58 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:27:58.563 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:27:58 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:27:58.563 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:27:58 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:27:58.563 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:27:58 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:27:58.563 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:27:58 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:27:58.563 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:27:58 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:27:58.563 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:27:58 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:27:58.564 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:27:58 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T00:27:58.564 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:27:58 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:27:58.581 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:27:58 smithi178 ceph-mon[35919]: pgmap v1025: 129 pgs: 129 active+clean; 677 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 1.7 MiB/s wr, 77 op/s 2024-09-11T00:27:58.581 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:27:58 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:27:58.581 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:27:58 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:27:58.581 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:27:58 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:27:58.581 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:27:58 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:27:58.581 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:27:58 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:27:58.581 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:27:58 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:27:58.581 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:27:58 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:27:58.581 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:27:58 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T00:27:58.582 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:27:58 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:27:59.248 INFO:tasks.workunit.client.0.smithi012.stdout: 10 131026 14.86 MB/sec execute 355 sec latency 24028.334 ms 2024-09-11T00:28:00.248 INFO:tasks.workunit.client.0.smithi012.stdout: 10 131026 14.82 MB/sec execute 356 sec latency 25028.400 ms 2024-09-11T00:28:00.532 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:28:00 smithi165 ceph-mon[35356]: pgmap v1026: 129 pgs: 129 active+clean; 677 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 1.6 MiB/s wr, 72 op/s 2024-09-11T00:28:00.563 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:28:00 smithi012 ceph-mon[30333]: pgmap v1026: 129 pgs: 129 active+clean; 677 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 1.6 MiB/s wr, 72 op/s 2024-09-11T00:28:00.581 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:28:00 smithi178 ceph-mon[35919]: pgmap v1026: 129 pgs: 129 active+clean; 677 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 1.6 MiB/s wr, 72 op/s 2024-09-11T00:28:01.248 INFO:tasks.workunit.client.0.smithi012.stdout: 10 131026 14.78 MB/sec execute 357 sec latency 26028.465 ms 2024-09-11T00:28:01.280 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid ff3ad74a-6fcf-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T00:28:01.673 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:28:02.248 INFO:tasks.workunit.client.0.smithi012.stdout: 10 131026 14.73 MB/sec execute 358 sec latency 27028.523 ms 2024-09-11T00:28:02.334 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:02.244+0000 7f11fec00640 1 -- 172.21.15.12:0/2038318662 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f11f81023d0 msgr2=0x7f11f81068a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:28:02.334 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:02.244+0000 7f11fec00640 1 --2- 172.21.15.12:0/2038318662 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f11f81023d0 0x7f11f81068a0 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f11e0009900 tx=0x7f11e002a410 comp rx=0 tx=0).stop 2024-09-11T00:28:02.335 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:02.244+0000 7f11fec00640 1 -- 172.21.15.12:0/2038318662 shutdown_connections 2024-09-11T00:28:02.335 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:02.244+0000 7f11fec00640 1 --2- 172.21.15.12:0/2038318662 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f11f8106ef0 0x7f11f8107340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:02.335 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:02.244+0000 7f11fec00640 1 --2- 172.21.15.12:0/2038318662 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f11f81023d0 0x7f11f81068a0 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:02.335 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:02.244+0000 7f11fec00640 1 --2- 172.21.15.12:0/2038318662 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f11f8101ab0 0x7f11f8101e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:02.335 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:02.244+0000 7f11fec00640 1 -- 172.21.15.12:0/2038318662 >> 172.21.15.12:0/2038318662 conn(0x7f11f80fd700 msgr2=0x7f11f80ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:28:02.335 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:02.245+0000 7f11fec00640 1 -- 172.21.15.12:0/2038318662 shutdown_connections 2024-09-11T00:28:02.335 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:02.245+0000 7f11fec00640 1 -- 172.21.15.12:0/2038318662 wait complete. 2024-09-11T00:28:02.335 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:02.245+0000 7f11fec00640 1 Processor -- start 2024-09-11T00:28:02.335 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:02.246+0000 7f11fec00640 1 -- start start 2024-09-11T00:28:02.335 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:02.246+0000 7f11fec00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f11f8101ab0 0x7f11f819f890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:28:02.335 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:02.246+0000 7f11fec00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f11f81023d0 0x7f11f819fdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:28:02.336 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:02.246+0000 7f11fec00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f11f8106ef0 0x7f11f81a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:28:02.336 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:02.246+0000 7f11fec00640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f11f8119360 con 0x7f11f8101ab0 2024-09-11T00:28:02.336 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:02.246+0000 7f11fec00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f11f81191e0 con 0x7f11f81023d0 2024-09-11T00:28:02.336 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:02.246+0000 7f11fec00640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f11f81194e0 con 0x7f11f8106ef0 2024-09-11T00:28:02.336 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:02.246+0000 7f11ef400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f11f81023d0 0x7f11f819fdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:28:02.336 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:02.246+0000 7f11fcc00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f11f8106ef0 0x7f11f81a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:28:02.336 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:02.246+0000 7f11f7e00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f11f8101ab0 0x7f11f819f890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:28:02.336 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:02.246+0000 7f11ef400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f11f81023d0 0x7f11f819fdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.12:34778/0 (socket says 172.21.15.12:34778) 2024-09-11T00:28:02.336 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:02.246+0000 7f11ef400640 1 -- 172.21.15.12:0/2249321149 learned_addr learned my addr 172.21.15.12:0/2249321149 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:28:02.336 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:02.247+0000 7f11fcc00640 1 -- 172.21.15.12:0/2249321149 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f11f81023d0 msgr2=0x7f11f819fdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:28:02.336 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:02.247+0000 7f11fcc00640 1 --2- 172.21.15.12:0/2249321149 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f11f81023d0 0x7f11f819fdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:02.337 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:02.247+0000 7f11fcc00640 1 -- 172.21.15.12:0/2249321149 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f11f8101ab0 msgr2=0x7f11f819f890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:28:02.337 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:02.247+0000 7f11fcc00640 1 --2- 172.21.15.12:0/2249321149 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f11f8101ab0 0x7f11f819f890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:02.337 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:02.247+0000 7f11fcc00640 1 -- 172.21.15.12:0/2249321149 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11f81a5940 con 0x7f11f8106ef0 2024-09-11T00:28:02.337 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:02.247+0000 7f11fcc00640 1 --2- 172.21.15.12:0/2249321149 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f11f8106ef0 0x7f11f81a5170 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f11f0009fd0 tx=0x7f11f000ef10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:28:02.337 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:02.248+0000 7f11f5600640 1 -- 172.21.15.12:0/2249321149 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11f000ce20 con 0x7f11f8106ef0 2024-09-11T00:28:02.337 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:02.248+0000 7f11f5600640 1 -- 172.21.15.12:0/2249321149 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f11f0005020 con 0x7f11f8106ef0 2024-09-11T00:28:02.337 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:02.248+0000 7f11fec00640 1 -- 172.21.15.12:0/2249321149 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f11f81a5c30 con 0x7f11f8106ef0 2024-09-11T00:28:02.337 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:02.248+0000 7f11f5600640 1 -- 172.21.15.12:0/2249321149 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11f0010690 con 0x7f11f8106ef0 2024-09-11T00:28:02.337 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:02.248+0000 7f11fec00640 1 -- 172.21.15.12:0/2249321149 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11f81ac2f0 con 0x7f11f8106ef0 2024-09-11T00:28:02.337 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:02.249+0000 7f11fec00640 1 -- 172.21.15.12:0/2249321149 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f11f81068a0 con 0x7f11f8106ef0 2024-09-11T00:28:02.337 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:02.253+0000 7f11f5600640 1 -- 172.21.15.12:0/2249321149 <== mon.1 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7f11f00054d0 con 0x7f11f8106ef0 2024-09-11T00:28:02.337 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:02.254+0000 7f11f5600640 1 --2- 172.21.15.12:0/2249321149 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f11dc0b52e0 0x7f11dc0b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:28:02.338 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:02.255+0000 7f11f5600640 1 -- 172.21.15.12:0/2249321149 <== mon.1 v2:172.21.15.178:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9208+0+0 (secure 0 0 0) 0x7f11f00e6610 con 0x7f11f8106ef0 2024-09-11T00:28:02.338 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:02.255+0000 7f11f7e00640 1 --2- 172.21.15.12:0/2249321149 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f11dc0b52e0 0x7f11dc0b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:28:02.338 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:02.255+0000 7f11f7e00640 1 --2- 172.21.15.12:0/2249321149 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f11dc0b52e0 0x7f11dc0b77a0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f11e8005d00 tx=0x7f11e800b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:28:02.338 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:02.258+0000 7f11f5600640 1 -- 172.21.15.12:0/2249321149 <== mon.1 v2:172.21.15.178:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f11f00ad100 con 0x7f11f8106ef0 2024-09-11T00:28:02.400 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:28:02 smithi012 ceph-mon[30333]: pgmap v1027: 129 pgs: 129 active+clean; 677 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 1.7 MiB/s wr, 75 op/s 2024-09-11T00:28:02.499 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:02.497+0000 7f11fec00640 1 -- 172.21.15.12:0/2249321149 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f11f81ac610 con 0x7f11f8106ef0 2024-09-11T00:28:02.500 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:02.498+0000 7f11f5600640 1 -- 172.21.15.12:0/2249321149 <== mon.1 v2:172.21.15.178:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 86 v86) ==== 94+0+11994 (secure 0 0 0) 0x7f11f00aca00 con 0x7f11f8106ef0 2024-09-11T00:28:02.501 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-11T00:28:02.501 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":86,"btime":"2024-09-11T00:27:35:938038+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14577,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6835/447171591","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":447171591},{"type":"v1","addr":"172.21.15.12:6835","nonce":447171591}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14589,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/901486040","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":901486040},{"type":"v1","addr":"172.21.15.12:6837","nonce":901486040}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":86,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T00:04:04.739881+0000","modified":"2024-09-11T00:27:35.671027+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24674,"mds_1":24643,"mds_2":24638,"mds_3":24637,"mds_4":24614},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14631":{"gid":14631,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.12:6841/3069919437","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6840","nonce":3069919437},{"type":"v1","addr":"172.21.15.12:6841","nonce":3069919437}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24562":{"gid":24562,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6833/1141636112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":1141636112},{"type":"v1","addr":"172.21.15.178:6833","nonce":1141636112}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24581":{"gid":24581,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6833/3111980970","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6832","nonce":3111980970},{"type":"v1","addr":"172.21.15.165:6833","nonce":3111980970}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24607":{"gid":24607,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6835/4084868287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":4084868287},{"type":"v1","addr":"172.21.15.178:6835","nonce":4084868287}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24614":{"gid":24614,"name":"e","rank":4,"incarnation":26,"state":"up:active","state_seq":21,"addr":"172.21.15.165:6835/2425395274","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":2425395274},{"type":"v1","addr":"172.21.15.165:6835","nonce":2425395274}]},"join_fscid":-1,"export_targets":[0],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24637":{"gid":24637,"name":"i","rank":3,"incarnation":24,"state":"up:active","state_seq":18,"addr":"172.21.15.178:6839/70628097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6838","nonce":70628097},{"type":"v1","addr":"172.21.15.178:6839","nonce":70628097}]},"join_fscid":-1,"export_targets":[0,1,2,4],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24638":{"gid":24638,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":18,"addr":"172.21.15.165:6837/6215088","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6836","nonce":6215088},{"type":"v1","addr":"172.21.15.165:6837","nonce":6215088}]},"join_fscid":-1,"export_targets":[0,1,3,4],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24643":{"gid":24643,"name":"g","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.12:6839/3124093009","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6838","nonce":3124093009},{"type":"v1","addr":"172.21.15.12:6839","nonce":3124093009}]},"join_fscid":-1,"export_targets":[0,2,3,4],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24674":{"gid":24674,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.165:6839/501312984","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6838","nonce":501312984},{"type":"v1","addr":"172.21.15.165:6839","nonce":501312984}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25672":{"gid":25672,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6837/308601741","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":308601741},{"type":"v1","addr":"172.21.15.178:6837","nonce":308601741}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24674,"qdb_cluster":[24674,24643,24638,24637,24614]},"id":1}]} 2024-09-11T00:28:02.501 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 86 2024-09-11T00:28:02.503 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:02.501+0000 7f11fec00640 1 -- 172.21.15.12:0/2249321149 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f11dc0b52e0 msgr2=0x7f11dc0b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:28:02.503 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:02.501+0000 7f11fec00640 1 --2- 172.21.15.12:0/2249321149 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f11dc0b52e0 0x7f11dc0b77a0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f11e8005d00 tx=0x7f11e800b410 comp rx=0 tx=0).stop 2024-09-11T00:28:02.503 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:02.501+0000 7f11fec00640 1 -- 172.21.15.12:0/2249321149 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f11f8106ef0 msgr2=0x7f11f81a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:28:02.504 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:02.501+0000 7f11fec00640 1 --2- 172.21.15.12:0/2249321149 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f11f8106ef0 0x7f11f81a5170 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f11f0009fd0 tx=0x7f11f000ef10 comp rx=0 tx=0).stop 2024-09-11T00:28:02.504 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:02.501+0000 7f11fec00640 1 -- 172.21.15.12:0/2249321149 shutdown_connections 2024-09-11T00:28:02.504 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:02.501+0000 7f11fec00640 1 --2- 172.21.15.12:0/2249321149 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f11dc0b52e0 0x7f11dc0b77a0 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:02.504 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:02.501+0000 7f11fec00640 1 --2- 172.21.15.12:0/2249321149 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f11f8106ef0 0x7f11f81a5170 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:02.504 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:02.501+0000 7f11fec00640 1 --2- 172.21.15.12:0/2249321149 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f11f81023d0 0x7f11f819fdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:02.504 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:02.501+0000 7f11fec00640 1 --2- 172.21.15.12:0/2249321149 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f11f8101ab0 0x7f11f819f890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:02.504 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:02.501+0000 7f11fec00640 1 -- 172.21.15.12:0/2249321149 >> 172.21.15.12:0/2249321149 conn(0x7f11f80fd700 msgr2=0x7f11f81047e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:28:02.504 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:02.501+0000 7f11fec00640 1 -- 172.21.15.12:0/2249321149 shutdown_connections 2024-09-11T00:28:02.504 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:02.501+0000 7f11fec00640 1 -- 172.21.15.12:0/2249321149 wait complete. 2024-09-11T00:28:02.532 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:28:02 smithi165 ceph-mon[35356]: pgmap v1027: 129 pgs: 129 active+clean; 677 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 1.7 MiB/s wr, 75 op/s 2024-09-11T00:28:02.581 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:28:02 smithi178 ceph-mon[35919]: pgmap v1027: 129 pgs: 129 active+clean; 677 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 1.7 MiB/s wr, 75 op/s 2024-09-11T00:28:03.239 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24674 quiesce db --set-id fae7af54 --release --await' 2024-09-11T00:28:03.239 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid ff3ad74a-6fcf-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24674 quiesce db --set-id fae7af54 --release --await 2024-09-11T00:28:03.248 INFO:tasks.workunit.client.0.smithi012.stdout: 10 131026 14.69 MB/sec execute 359 sec latency 28028.590 ms 2024-09-11T00:28:03.532 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:28:03 smithi165 ceph-mon[35356]: from='client.? 172.21.15.12:0/2249321149' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T00:28:03.563 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:28:03 smithi012 ceph-mon[30333]: from='client.? 172.21.15.12:0/2249321149' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T00:28:03.581 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:28:03 smithi178 ceph-mon[35919]: from='client.? 172.21.15.12:0/2249321149' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T00:28:03.622 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:28:04.194 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:04.173+0000 7f7c0ae00640 1 -- 172.21.15.12:0/3851003644 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7c04075540 msgr2=0x7f7c04075920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:28:04.194 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:04.173+0000 7f7c0ae00640 1 --2- 172.21.15.12:0/3851003644 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7c04075540 0x7f7c04075920 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f7bf4009900 tx=0x7f7bf402a3f0 comp rx=0 tx=0).stop 2024-09-11T00:28:04.194 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:04.173+0000 7f7c0ae00640 1 -- 172.21.15.12:0/3851003644 shutdown_connections 2024-09-11T00:28:04.194 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:04.173+0000 7f7c0ae00640 1 --2- 172.21.15.12:0/3851003644 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7c0410f3e0 0x7f7c041117d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:04.194 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:04.173+0000 7f7c0ae00640 1 --2- 172.21.15.12:0/3851003644 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7c04075f40 0x7f7c0410ee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:04.195 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:04.173+0000 7f7c0ae00640 1 --2- 172.21.15.12:0/3851003644 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7c04075540 0x7f7c04075920 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:04.195 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:04.173+0000 7f7c0ae00640 1 -- 172.21.15.12:0/3851003644 >> 172.21.15.12:0/3851003644 conn(0x7f7c040fe830 msgr2=0x7f7c04100c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:28:04.195 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:04.173+0000 7f7c0ae00640 1 -- 172.21.15.12:0/3851003644 shutdown_connections 2024-09-11T00:28:04.195 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:04.173+0000 7f7c0ae00640 1 -- 172.21.15.12:0/3851003644 wait complete. 2024-09-11T00:28:04.195 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:04.174+0000 7f7c0ae00640 1 Processor -- start 2024-09-11T00:28:04.195 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:04.174+0000 7f7c0ae00640 1 -- start start 2024-09-11T00:28:04.195 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:04.175+0000 7f7c0ae00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7c04075540 0x7f7c0419b4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:28:04.195 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:04.175+0000 7f7c0ae00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7c04075f40 0x7f7c0419b9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:28:04.195 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:04.175+0000 7f7c0ae00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7c0410f3e0 0x7f7c041a0d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:28:04.195 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:04.175+0000 7f7c0ae00640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f7c04114fd0 con 0x7f7c0410f3e0 2024-09-11T00:28:04.195 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:04.175+0000 7f7c0ae00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f7c04114e50 con 0x7f7c04075f40 2024-09-11T00:28:04.195 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:04.175+0000 7f7c0ae00640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f7c04115150 con 0x7f7c04075540 2024-09-11T00:28:04.195 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:04.175+0000 7f7c03e00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7c04075540 0x7f7c0419b4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:28:04.196 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:04.175+0000 7f7c08e00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7c0410f3e0 0x7f7c041a0d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:28:04.196 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:04.175+0000 7f7bfb400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7c04075f40 0x7f7c0419b9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:28:04.196 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:04.175+0000 7f7c03e00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7c04075540 0x7f7c0419b4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.12:38322/0 (socket says 172.21.15.12:38322) 2024-09-11T00:28:04.196 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:04.175+0000 7f7c03e00640 1 -- 172.21.15.12:0/282615662 learned_addr learned my addr 172.21.15.12:0/282615662 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:28:04.196 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:04.176+0000 7f7c03e00640 1 -- 172.21.15.12:0/282615662 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7c04075f40 msgr2=0x7f7c0419b9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:28:04.196 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:04.176+0000 7f7c03e00640 1 --2- 172.21.15.12:0/282615662 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7c04075f40 0x7f7c0419b9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:04.196 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:04.176+0000 7f7c03e00640 1 -- 172.21.15.12:0/282615662 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7c0410f3e0 msgr2=0x7f7c041a0d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:28:04.196 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:04.176+0000 7f7c03e00640 1 --2- 172.21.15.12:0/282615662 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7c0410f3e0 0x7f7c041a0d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:04.196 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:04.176+0000 7f7c03e00640 1 -- 172.21.15.12:0/282615662 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c041a1550 con 0x7f7c04075540 2024-09-11T00:28:04.196 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:04.176+0000 7f7c03e00640 1 --2- 172.21.15.12:0/282615662 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7c04075540 0x7f7c0419b4a0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f7bf4005ab0 tx=0x7f7bf4003c10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:28:04.196 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:04.176+0000 7f7c01600640 1 -- 172.21.15.12:0/282615662 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bf403f070 con 0x7f7c04075540 2024-09-11T00:28:04.196 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:04.176+0000 7f7c01600640 1 -- 172.21.15.12:0/282615662 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7bf4031b60 con 0x7f7c04075540 2024-09-11T00:28:04.197 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:04.176+0000 7f7c0ae00640 1 -- 172.21.15.12:0/282615662 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7c041a17e0 con 0x7f7c04075540 2024-09-11T00:28:04.197 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:04.176+0000 7f7c01600640 1 -- 172.21.15.12:0/282615662 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bf4035030 con 0x7f7c04075540 2024-09-11T00:28:04.197 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:04.177+0000 7f7c0ae00640 1 -- 172.21.15.12:0/282615662 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7c041a7f00 con 0x7f7c04075540 2024-09-11T00:28:04.197 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:04.180+0000 7f7c0d88f740 1 Processor -- start 2024-09-11T00:28:04.197 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:04.181+0000 7f7c0d88f740 1 -- start start 2024-09-11T00:28:04.197 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:04.181+0000 7f7c0d88f740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5649f0570ff0 0x5649f05713d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:28:04.197 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:04.181+0000 7f7c0d88f740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5649f0571910 0x5649f057d360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:28:04.197 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:04.181+0000 7f7c0d88f740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5649f057d8a0 0x5649f057fc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:28:04.197 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:04.181+0000 7f7c0d88f740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x5649f04e8290 con 0x5649f0571910 2024-09-11T00:28:04.197 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:04.181+0000 7f7c0d88f740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x5649f04d76c0 con 0x5649f057d8a0 2024-09-11T00:28:04.197 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:04.181+0000 7f7c0d88f740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5649f04db4d0 con 0x5649f0570ff0 2024-09-11T00:28:04.197 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:04.181+0000 7f7c08e00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5649f0570ff0 0x5649f05713d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:28:04.197 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:04.181+0000 7f7c08e00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5649f0570ff0 0x5649f05713d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.12:38336/0 (socket says 172.21.15.12:38336) 2024-09-11T00:28:04.198 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:04.181+0000 7f7c08e00640 1 -- 172.21.15.12:0/2279442050 learned_addr learned my addr 172.21.15.12:0/2279442050 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:28:04.198 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:04.182+0000 7f7c01600640 1 -- 172.21.15.12:0/282615662 <== mon.1 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7f7bf403b460 con 0x7f7c04075540 2024-09-11T00:28:04.198 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:04.182+0000 7f7bfb400640 1 --2- 172.21.15.12:0/2279442050 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5649f057d8a0 0x5649f057fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:28:04.198 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:04.182+0000 7f7c03e00640 1 --2- 172.21.15.12:0/2279442050 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5649f0571910 0x5649f057d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:28:04.198 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:04.182+0000 7f7c08e00640 1 -- 172.21.15.12:0/2279442050 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5649f057d8a0 msgr2=0x5649f057fc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:28:04.198 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:04.182+0000 7f7c08e00640 1 --2- 172.21.15.12:0/2279442050 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5649f057d8a0 0x5649f057fc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:04.198 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:04.182+0000 7f7c08e00640 1 -- 172.21.15.12:0/2279442050 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5649f0571910 msgr2=0x5649f057d360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:28:04.198 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:04.182+0000 7f7c08e00640 1 --2- 172.21.15.12:0/2279442050 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5649f0571910 0x5649f057d360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:04.198 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:04.182+0000 7f7c08e00640 1 -- 172.21.15.12:0/2279442050 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5649f0464460 con 0x5649f0570ff0 2024-09-11T00:28:04.198 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:04.182+0000 7f7c03e00640 1 --2- 172.21.15.12:0/2279442050 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5649f0571910 0x5649f057d360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T00:28:04.199 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:04.182+0000 7f7c08e00640 1 --2- 172.21.15.12:0/2279442050 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5649f0570ff0 0x5649f05713d0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f7bfc009fc0 tx=0x7f7bfc0118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dfeae171eb45f720 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:28:04.199 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:04.183+0000 7f7bf9600640 1 -- 172.21.15.12:0/2279442050 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bfc004d20 con 0x5649f0570ff0 2024-09-11T00:28:04.199 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:04.183+0000 7f7bf9600640 1 -- 172.21.15.12:0/2279442050 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7bfc004050 con 0x5649f0570ff0 2024-09-11T00:28:04.199 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:04.183+0000 7f7bf9600640 1 -- 172.21.15.12:0/2279442050 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bfc005280 con 0x5649f0570ff0 2024-09-11T00:28:04.199 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:04.183+0000 7f7c0d88f740 1 -- 172.21.15.12:0/2279442050 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5649f0570ff0 msgr2=0x5649f05713d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:28:04.199 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:04.183+0000 7f7c0d88f740 1 --2- 172.21.15.12:0/2279442050 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5649f0570ff0 0x5649f05713d0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f7bfc009fc0 tx=0x7f7bfc0118d0 comp rx=0 tx=0).stop 2024-09-11T00:28:04.199 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:04.183+0000 7f7c0d88f740 1 -- 172.21.15.12:0/2279442050 shutdown_connections 2024-09-11T00:28:04.199 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:04.183+0000 7f7c0d88f740 1 --2- 172.21.15.12:0/2279442050 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5649f057d8a0 0x5649f057fc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:04.199 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:04.183+0000 7f7c0d88f740 1 --2- 172.21.15.12:0/2279442050 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5649f0571910 0x5649f057d360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:04.199 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:04.183+0000 7f7c0d88f740 1 --2- 172.21.15.12:0/2279442050 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5649f0570ff0 0x5649f05713d0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:04.199 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:04.183+0000 7f7c0d88f740 1 -- 172.21.15.12:0/2279442050 >> 172.21.15.12:0/2279442050 conn(0x5649f056ee20 msgr2=0x5649f056ffc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:28:04.199 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:04.183+0000 7f7c0d88f740 1 -- 172.21.15.12:0/2279442050 shutdown_connections 2024-09-11T00:28:04.199 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:04.184+0000 7f7c0d88f740 1 -- 172.21.15.12:0/2279442050 wait complete. 2024-09-11T00:28:04.200 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:04.184+0000 7f7c01600640 1 --2- 172.21.15.12:0/282615662 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f7be80b51f0 0x7f7be80b76b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:28:04.200 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:04.184+0000 7f7c01600640 1 -- 172.21.15.12:0/282615662 <== mon.1 v2:172.21.15.178:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9208+0+0 (secure 0 0 0) 0x7f7bf4106950 con 0x7f7c04075540 2024-09-11T00:28:04.200 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:04.184+0000 7f7c0d88f740 1 -- start start 2024-09-11T00:28:04.200 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:04.184+0000 7f7bfb400640 1 --2- 172.21.15.12:0/282615662 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f7be80b51f0 0x7f7be80b76b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:28:04.200 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:04.184+0000 7f7c0d88f740 1 Processor -- start 2024-09-11T00:28:04.200 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:04.185+0000 7f7bfb400640 1 --2- 172.21.15.12:0/282615662 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f7be80b51f0 0x7f7be80b76b0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x5649f0570fa0 tx=0x7f7bec005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:28:04.200 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:04.185+0000 7f7c0d88f740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5649f0570ff0 0x5649f057a860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:28:04.200 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:04.185+0000 7f7c0d88f740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5649f0571910 0x5649f057ada0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:28:04.200 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:04.185+0000 7f7c0d88f740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5649f057d8a0 0x5649f057b2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:28:04.200 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:04.185+0000 7f7c0d88f740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x5649f0338bb0 con 0x5649f057d8a0 2024-09-11T00:28:04.200 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:04.185+0000 7f7c0d88f740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x5649f04726c0 con 0x5649f0570ff0 2024-09-11T00:28:04.200 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:04.185+0000 7f7c0d88f740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5649f0580550 con 0x5649f0571910 2024-09-11T00:28:04.200 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:04.185+0000 7f7c0d88f740 10 client.0 _ll_register_callbacks cb 0x5649f048d490 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:28:04.201 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:04.185+0000 7f7c08e00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5649f057d8a0 0x5649f057b2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:28:04.201 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:04.185+0000 7f7bfb400640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5649f0571910 0x5649f057ada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:28:04.201 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:04.185+0000 7f7c08e00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5649f057d8a0 0x5649f057b2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:54748/0 (socket says 172.21.15.12:54748) 2024-09-11T00:28:04.201 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:04.185+0000 7f7c08e00640 1 -- 172.21.15.12:0/175073834 learned_addr learned my addr 172.21.15.12:0/175073834 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:28:04.201 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:04.185+0000 7f7c03e00640 1 --2- 172.21.15.12:0/175073834 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5649f0570ff0 0x5649f057a860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:28:04.201 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:04.186+0000 7f7c08e00640 1 -- 172.21.15.12:0/175073834 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5649f0571910 msgr2=0x5649f057ada0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:28:04.201 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:04.186+0000 7f7c08e00640 1 --2- 172.21.15.12:0/175073834 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5649f0571910 0x5649f057ada0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:04.201 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:04.186+0000 7f7c08e00640 1 -- 172.21.15.12:0/175073834 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5649f0570ff0 msgr2=0x5649f057a860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:28:04.201 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:04.186+0000 7f7c08e00640 1 --2- 172.21.15.12:0/175073834 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5649f0570ff0 0x5649f057a860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:04.201 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:04.186+0000 7f7c08e00640 1 -- 172.21.15.12:0/175073834 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5649f0235b00 con 0x5649f057d8a0 2024-09-11T00:28:04.201 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:04.186+0000 7f7c03e00640 1 --2- 172.21.15.12:0/175073834 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5649f0570ff0 0x5649f057a860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:28:04.201 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:04.186+0000 7f7c08e00640 1 --2- 172.21.15.12:0/175073834 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5649f057d8a0 0x5649f057b2e0 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f7bfc009250 tx=0x7f7bfc004430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7150da03a0f08bfd server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:28:04.202 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:04.186+0000 7f7bf2000640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-09-11T00:28:04.202 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:04.186+0000 7f7bf2000640 1 -- 172.21.15.12:0/175073834 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bfc026020 con 0x5649f057d8a0 2024-09-11T00:28:04.202 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:04.186+0000 7f7c0d88f740 1 -- 172.21.15.12:0/175073834 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7bfc004050 con 0x5649f057d8a0 2024-09-11T00:28:04.202 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:04.186+0000 7f7bf2000640 1 -- 172.21.15.12:0/175073834 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9208+0+0 (secure 0 0 0) 0x7f7bfc005040 con 0x5649f057d8a0 2024-09-11T00:28:04.202 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:04.187+0000 7f7bf2000640 1 -- 172.21.15.12:0/175073834 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=86) ==== 24+0+0 (secure 0 0 0) 0x7f7bfc031050 con 0x5649f057d8a0 2024-09-11T00:28:04.202 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:04.187+0000 7f7c0d88f740 10 client.17754 fetch_fsmap learned FSMap version 86 2024-09-11T00:28:04.202 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:04.187+0000 7f7c0d88f740 1 -- 172.21.15.12:0/175073834 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=86,monmap=4+}) -- 0x5649f02ac960 con 0x5649f057d8a0 2024-09-11T00:28:04.202 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:04.187+0000 7f7bf2000640 1 -- 172.21.15.12:0/175073834 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7bfc004050 con 0x5649f057d8a0 2024-09-11T00:28:04.202 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:04.187+0000 7f7bf2000640 1 -- 172.21.15.12:0/175073834 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 86) ==== 8438+0+0 (secure 0 0 0) 0x7f7bfc00f4e0 con 0x5649f057d8a0 2024-09-11T00:28:04.202 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:04.187+0000 7f7c0d88f740 10 client.17754 fetch_fsmap finished waiting for FSMap version 86 2024-09-11T00:28:04.202 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:04.187+0000 7f7c0d88f740 10 client.17754 resolve_mds: validated gid 24674 aka daemon mds.b 2024-09-11T00:28:04.202 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:04.187+0000 7f7c0d88f740 20 client.17754 populate_metadata read hostname 'smithi012' 2024-09-11T00:28:04.202 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:04.187+0000 7f7c0d88f740 1 --2- 172.21.15.12:0/175073834 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x5649f058d2e0 0x5649f058f740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:28:04.203 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:04.187+0000 7f7c0d88f740 4 client.17754 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T00:28:04.203 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:04.187+0000 7f7c0d88f740 1 -- 172.21.15.12:0/175073834 --> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5649f04ea1e0 con 0x5649f058d2e0 2024-09-11T00:28:04.203 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:04.188+0000 7f7c03e00640 1 --2- 172.21.15.12:0/175073834 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x5649f058d2e0 0x5649f058f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:28:04.203 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:04.188+0000 7f7c03e00640 1 --2- 172.21.15.12:0/175073834 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x5649f058d2e0 0x5649f058f740 crc :-1 s=READY pgs=289 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bea5b932e9af727d server_cookie=6f01acd371ede9bf in_seq=0 out_seq=0 2024-09-11T00:28:04.203 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:04.188+0000 7f7bf2000640 10 client.17754 ms_handle_connect on v2:172.21.15.165:6838/501312984 2024-09-11T00:28:04.203 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:04.189+0000 7f7bf2000640 1 -- 172.21.15.12:0/175073834 <== mds.0 v2:172.21.15.165:6838/501312984 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5649f04ea1e0 con 0x5649f058d2e0 2024-09-11T00:28:04.203 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:04.189+0000 7f7bf2000640 10 client.17754 handle_command_reply: tid=0 2024-09-11T00:28:04.203 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:04.189+0000 7f7c0d88f740 1 client.17754 shutdown 2024-09-11T00:28:04.203 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:04.189+0000 7f7c0d88f740 1 -- 172.21.15.12:0/175073834 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5649f057d8a0 msgr2=0x5649f057b2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:28:04.203 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:04.189+0000 7f7c0d88f740 1 --2- 172.21.15.12:0/175073834 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5649f057d8a0 0x5649f057b2e0 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f7bfc009250 tx=0x7f7bfc004430 comp rx=0 tx=0).stop 2024-09-11T00:28:04.203 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:04.189+0000 7f7c0d88f740 1 -- 172.21.15.12:0/175073834 shutdown_connections 2024-09-11T00:28:04.203 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:04.189+0000 7f7c0d88f740 1 --2- 172.21.15.12:0/175073834 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x5649f058d2e0 0x5649f058f740 crc :-1 s=READY pgs=289 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:04.203 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:04.189+0000 7f7c0d88f740 1 --2- 172.21.15.12:0/175073834 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5649f057d8a0 0x5649f057b2e0 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:04.204 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:04.189+0000 7f7c0d88f740 1 --2- 172.21.15.12:0/175073834 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5649f0571910 0x5649f057ada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:04.204 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:04.189+0000 7f7c0d88f740 1 --2- 172.21.15.12:0/175073834 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5649f0570ff0 0x5649f057a860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:04.204 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:04.189+0000 7f7bf2000640 0 client.17754 ms_handle_reset on v2:172.21.15.165:6838/501312984 2024-09-11T00:28:04.204 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:04.189+0000 7f7c0d88f740 1 -- 172.21.15.12:0/175073834 >> 172.21.15.12:0/175073834 conn(0x5649f056ee20 msgr2=0x5649f05749c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:28:04.204 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:04.190+0000 7f7c0d88f740 1 -- 172.21.15.12:0/175073834 shutdown_connections 2024-09-11T00:28:04.204 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:04.190+0000 7f7c0d88f740 1 -- 172.21.15.12:0/175073834 wait complete. 2024-09-11T00:28:04.204 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:04.190+0000 7f7c0d88f740 20 client.17754 trim_cache size 0 max 16384 2024-09-11T00:28:04.210 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:04.208+0000 7f7c0d88f740 1 Processor -- start 2024-09-11T00:28:04.211 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:04.208+0000 7f7c0d88f740 1 -- start start 2024-09-11T00:28:04.212 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:04.209+0000 7f7c0d88f740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5649f0570ff0 0x5649f0595060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:28:04.212 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:04.209+0000 7f7c0d88f740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5649f0571910 0x5649f05955a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:28:04.212 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:04.209+0000 7f7c0d88f740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5649f057d8a0 0x5649f05741a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:28:04.212 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:04.209+0000 7f7c0d88f740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f7c041b1dc0 con 0x5649f0570ff0 2024-09-11T00:28:04.213 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:04.209+0000 7f7c0d88f740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f7c041b3b40 con 0x5649f0571910 2024-09-11T00:28:04.213 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:04.209+0000 7f7c0d88f740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f7c041b2550 con 0x5649f057d8a0 2024-09-11T00:28:04.213 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:04.209+0000 7f7bfb400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5649f0571910 0x5649f05955a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:28:04.213 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:04.209+0000 7f7c03e00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5649f0570ff0 0x5649f0595060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:28:04.213 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:04.209+0000 7f7bfb400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5649f0571910 0x5649f05955a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.12:34828/0 (socket says 172.21.15.12:34828) 2024-09-11T00:28:04.213 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:04.209+0000 7f7bfb400640 1 -- 172.21.15.12:0/1756216915 learned_addr learned my addr 172.21.15.12:0/1756216915 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:28:04.213 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:04.209+0000 7f7c08e00640 1 --2- 172.21.15.12:0/1756216915 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5649f057d8a0 0x5649f05741a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:28:04.213 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:04.209+0000 7f7bfb400640 1 -- 172.21.15.12:0/1756216915 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5649f057d8a0 msgr2=0x5649f05741a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:28:04.213 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:04.209+0000 7f7bfb400640 1 --2- 172.21.15.12:0/1756216915 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5649f057d8a0 0x5649f05741a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:04.213 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:04.209+0000 7f7bfb400640 1 -- 172.21.15.12:0/1756216915 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5649f0570ff0 msgr2=0x5649f0595060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:28:04.213 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:04.209+0000 7f7bfb400640 1 --2- 172.21.15.12:0/1756216915 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5649f0570ff0 0x5649f0595060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:04.214 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:04.209+0000 7f7bfb400640 1 -- 172.21.15.12:0/1756216915 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5649f0570500 con 0x5649f0571910 2024-09-11T00:28:04.214 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:04.209+0000 7f7c03e00640 1 --2- 172.21.15.12:0/1756216915 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5649f0570ff0 0x5649f0595060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:28:04.214 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:04.210+0000 7f7bfb400640 1 --2- 172.21.15.12:0/1756216915 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5649f0571910 0x5649f05955a0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f7bec004390 tx=0x7f7bec01b000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f2daf8fe40ba0050 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:28:04.214 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:04.210+0000 7f7bf8c00640 1 -- 172.21.15.12:0/1756216915 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bec0043c0 con 0x5649f0571910 2024-09-11T00:28:04.214 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:04.210+0000 7f7bf8c00640 1 -- 172.21.15.12:0/1756216915 <== mon.2 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7bec0054d0 con 0x5649f0571910 2024-09-11T00:28:04.214 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:04.210+0000 7f7bf8c00640 1 -- 172.21.15.12:0/1756216915 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bec01f430 con 0x5649f0571910 2024-09-11T00:28:04.217 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:04.211+0000 7f7c0d88f740 1 -- 172.21.15.12:0/1756216915 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5649f0571910 msgr2=0x5649f05955a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:28:04.218 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:04.211+0000 7f7c0d88f740 1 --2- 172.21.15.12:0/1756216915 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5649f0571910 0x5649f05955a0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f7bec004390 tx=0x7f7bec01b000 comp rx=0 tx=0).stop 2024-09-11T00:28:04.218 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:04.211+0000 7f7c0d88f740 1 -- 172.21.15.12:0/1756216915 shutdown_connections 2024-09-11T00:28:04.218 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:04.211+0000 7f7c0d88f740 1 --2- 172.21.15.12:0/1756216915 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5649f057d8a0 0x5649f05741a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:04.218 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:04.211+0000 7f7c0d88f740 1 --2- 172.21.15.12:0/1756216915 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5649f0571910 0x5649f05955a0 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:04.218 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:04.211+0000 7f7c0d88f740 1 --2- 172.21.15.12:0/1756216915 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5649f0570ff0 0x5649f0595060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:04.218 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:04.211+0000 7f7c0d88f740 1 -- 172.21.15.12:0/1756216915 >> 172.21.15.12:0/1756216915 conn(0x5649f056ee20 msgr2=0x5649f057c6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:28:04.218 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:04.211+0000 7f7c0d88f740 1 -- 172.21.15.12:0/1756216915 shutdown_connections 2024-09-11T00:28:04.218 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:04.211+0000 7f7c0d88f740 1 -- 172.21.15.12:0/1756216915 wait complete. 2024-09-11T00:28:04.218 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:04.211+0000 7f7c0d88f740 1 -- start start 2024-09-11T00:28:04.218 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:04.211+0000 7f7c0d88f740 1 Processor -- start 2024-09-11T00:28:04.218 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:04.212+0000 7f7c0d88f740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5649f0570ff0 0x5649f0585a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:28:04.218 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:04.212+0000 7f7c0d88f740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5649f0571910 0x5649f0585f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:28:04.219 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:04.212+0000 7f7c0d88f740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5649f057d8a0 0x5649f05807d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:28:04.219 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:04.212+0000 7f7c0d88f740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f7c041b3470 con 0x5649f0571910 2024-09-11T00:28:04.219 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:04.212+0000 7f7c0d88f740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f7c041b2ce0 con 0x5649f057d8a0 2024-09-11T00:28:04.219 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:04.212+0000 7f7c0d88f740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f7c041afda0 con 0x5649f0570ff0 2024-09-11T00:28:04.219 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:04.212+0000 7f7c0d88f740 10 client.0 _ll_register_callbacks cb 0x7f7bec0097a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:28:04.219 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:04.212+0000 7f7c03e00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5649f0570ff0 0x5649f0585a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:28:04.219 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:04.212+0000 7f7bfb400640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5649f0571910 0x5649f0585f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:28:04.219 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:04.212+0000 7f7c03e00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5649f0570ff0 0x5649f0585a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.12:38372/0 (socket says 172.21.15.12:38372) 2024-09-11T00:28:04.219 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:04.212+0000 7f7c03e00640 1 -- 172.21.15.12:0/841237301 learned_addr learned my addr 172.21.15.12:0/841237301 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:28:04.219 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:04.212+0000 7f7c08e00640 1 --2- 172.21.15.12:0/841237301 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5649f057d8a0 0x5649f05807d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:28:04.219 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:04.213+0000 7f7c03e00640 1 -- 172.21.15.12:0/841237301 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5649f057d8a0 msgr2=0x5649f05807d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:28:04.219 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:04.213+0000 7f7c03e00640 1 --2- 172.21.15.12:0/841237301 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5649f057d8a0 0x5649f05807d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:04.219 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:04.213+0000 7f7c03e00640 1 -- 172.21.15.12:0/841237301 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5649f0571910 msgr2=0x5649f0585f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:28:04.220 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:04.213+0000 7f7c03e00640 1 --2- 172.21.15.12:0/841237301 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5649f0571910 0x5649f0585f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:04.220 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:04.213+0000 7f7c03e00640 1 -- 172.21.15.12:0/841237301 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5649f0580dc0 con 0x5649f0570ff0 2024-09-11T00:28:04.220 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:04.213+0000 7f7bfb400640 1 --2- 172.21.15.12:0/841237301 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5649f0571910 0x5649f0585f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:28:04.220 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:04.213+0000 7f7c03e00640 1 --2- 172.21.15.12:0/841237301 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5649f0570ff0 0x5649f0585a00 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f7bf4031440 tx=0x7f7bf4009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d70e057856ddab6a server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:28:04.220 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:04.213+0000 7f7bf2000640 10 client.0 ms_handle_connect on v2:172.21.15.178:3300/0 2024-09-11T00:28:04.220 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:04.213+0000 7f7bf2000640 1 -- 172.21.15.12:0/841237301 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bf4049040 con 0x5649f0570ff0 2024-09-11T00:28:04.220 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:04.213+0000 7f7bf2000640 1 -- 172.21.15.12:0/841237301 <== mon.1 v2:172.21.15.178:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9208+0+0 (secure 0 0 0) 0x7f7bf40c9000 con 0x5649f0570ff0 2024-09-11T00:28:04.220 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:04.213+0000 7f7c0d88f740 1 -- 172.21.15.12:0/841237301 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7bec0054d0 con 0x5649f0570ff0 2024-09-11T00:28:04.220 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:04.214+0000 7f7bf2000640 1 -- 172.21.15.12:0/841237301 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=86) ==== 24+0+0 (secure 0 0 0) 0x7f7bf40d6270 con 0x5649f0570ff0 2024-09-11T00:28:04.220 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:04.214+0000 7f7c0d88f740 10 client.27736 fetch_fsmap learned FSMap version 86 2024-09-11T00:28:04.220 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:04.214+0000 7f7c0d88f740 1 -- 172.21.15.12:0/841237301 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,fsmap=86,monmap=4+}) -- 0x5649f05fab80 con 0x5649f0570ff0 2024-09-11T00:28:04.220 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:04.214+0000 7f7bf2000640 1 -- 172.21.15.12:0/841237301 <== mon.1 v2:172.21.15.178:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7bec0054d0 con 0x5649f0570ff0 2024-09-11T00:28:04.220 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:04.214+0000 7f7bf2000640 1 -- 172.21.15.12:0/841237301 <== mon.1 v2:172.21.15.178:3300/0 5 ==== fsmap(e 86) ==== 8438+0+0 (secure 0 0 0) 0x7f7bf40b2bb0 con 0x5649f0570ff0 2024-09-11T00:28:04.221 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:04.214+0000 7f7c0d88f740 10 client.27736 fetch_fsmap finished waiting for FSMap version 86 2024-09-11T00:28:04.221 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:04.214+0000 7f7c0d88f740 10 client.27736 resolve_mds: validated gid 24674 aka daemon mds.b 2024-09-11T00:28:04.221 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:04.214+0000 7f7c0d88f740 20 client.27736 populate_metadata read hostname 'smithi012' 2024-09-11T00:28:04.221 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:04.214+0000 7f7c0d88f740 1 --2- 172.21.15.12:0/841237301 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x5649f05fb300 0x5649f05fd720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:28:04.221 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:04.214+0000 7f7c0d88f740 4 client.27736 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "fae7af54", "release": true, "await": true}] 2024-09-11T00:28:04.221 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:04.214+0000 7f7c0d88f740 1 -- 172.21.15.12:0/841237301 --> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] -- command(tid 0: {"prefix": "quiesce db", "set_id": "fae7af54", "release": true, "await": true}) -- 0x5649f04e5c90 con 0x5649f05fb300 2024-09-11T00:28:04.221 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:04.215+0000 7f7bfb400640 1 --2- 172.21.15.12:0/841237301 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x5649f05fb300 0x5649f05fd720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:28:04.221 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:04.215+0000 7f7bfb400640 1 --2- 172.21.15.12:0/841237301 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x5649f05fb300 0x5649f05fd720 crc :-1 s=READY pgs=290 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6e64daf20532d96f server_cookie=1965d551f398fb68 in_seq=0 out_seq=0 2024-09-11T00:28:04.221 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:04.215+0000 7f7bf2000640 10 client.27736 ms_handle_connect on v2:172.21.15.165:6838/501312984 2024-09-11T00:28:04.248 INFO:tasks.workunit.client.0.smithi012.stdout: 10 131027 14.65 MB/sec execute 360 sec latency 29028.639 ms 2024-09-11T00:28:04.251 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:04.249+0000 7f7bf2000640 1 -- 172.21.15.12:0/841237301 <== mds.0 v2:172.21.15.165:6838/501312984 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x5649f04e5c90 con 0x5649f05fb300 2024-09-11T00:28:04.251 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:04.249+0000 7f7bf2000640 10 client.27736 handle_command_reply: tid=0 2024-09-11T00:28:04.251 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:04.249+0000 7f7c0d88f740 1 client.27736 shutdown 2024-09-11T00:28:04.252 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:04.249+0000 7f7c0d88f740 1 -- 172.21.15.12:0/841237301 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5649f0570ff0 msgr2=0x5649f0585a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:28:04.252 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:04.249+0000 7f7c0d88f740 1 --2- 172.21.15.12:0/841237301 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5649f0570ff0 0x5649f0585a00 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f7bf4031440 tx=0x7f7bf4009260 comp rx=0 tx=0).stop 2024-09-11T00:28:04.252 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:04.249+0000 7f7c0d88f740 1 -- 172.21.15.12:0/841237301 shutdown_connections 2024-09-11T00:28:04.252 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:04.249+0000 7f7c0d88f740 1 --2- 172.21.15.12:0/841237301 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x5649f05fb300 0x5649f05fd720 crc :-1 s=READY pgs=290 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:04.252 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:04.249+0000 7f7c0d88f740 1 --2- 172.21.15.12:0/841237301 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5649f057d8a0 0x5649f05807d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:04.252 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:04.249+0000 7f7c0d88f740 1 --2- 172.21.15.12:0/841237301 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5649f0571910 0x5649f0585f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:04.252 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:04.249+0000 7f7c0d88f740 1 --2- 172.21.15.12:0/841237301 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5649f0570ff0 0x5649f0585a00 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:04.253 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:04.249+0000 7f7c0d88f740 1 -- 172.21.15.12:0/841237301 >> 172.21.15.12:0/841237301 conn(0x5649f056ee20 msgr2=0x5649f05741a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:28:04.253 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:04.249+0000 7f7bf2000640 0 client.27736 ms_handle_reset on v2:172.21.15.165:6838/501312984 2024-09-11T00:28:04.254 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:04.251+0000 7f7c0d88f740 1 -- 172.21.15.12:0/841237301 shutdown_connections 2024-09-11T00:28:04.254 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:04.252+0000 7f7c0d88f740 1 -- 172.21.15.12:0/841237301 wait complete. 2024-09-11T00:28:04.254 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:04.252+0000 7f7c0d88f740 20 client.27736 trim_cache size 0 max 16384 2024-09-11T00:28:04.255 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-09-11T00:28:04.255 INFO:teuthology.orchestra.run.smithi012.stdout: "epoch": 86, 2024-09-11T00:28:04.255 INFO:teuthology.orchestra.run.smithi012.stdout: "leader": 24674, 2024-09-11T00:28:04.255 INFO:teuthology.orchestra.run.smithi012.stdout: "set_version": 64, 2024-09-11T00:28:04.255 INFO:teuthology.orchestra.run.smithi012.stdout: "sets": { 2024-09-11T00:28:04.256 INFO:teuthology.orchestra.run.smithi012.stdout: "fae7af54": { 2024-09-11T00:28:04.256 INFO:teuthology.orchestra.run.smithi012.stdout: "version": 64, 2024-09-11T00:28:04.256 INFO:teuthology.orchestra.run.smithi012.stdout: "age_ref": 0.0, 2024-09-11T00:28:04.256 INFO:teuthology.orchestra.run.smithi012.stdout: "state": { 2024-09-11T00:28:04.256 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "RELEASED", 2024-09-11T00:28:04.256 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 0.0 2024-09-11T00:28:04.256 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:28:04.256 INFO:teuthology.orchestra.run.smithi012.stdout: "timeout": 90.0, 2024-09-11T00:28:04.256 INFO:teuthology.orchestra.run.smithi012.stdout: "expiration": 150.0, 2024-09-11T00:28:04.256 INFO:teuthology.orchestra.run.smithi012.stdout: "members": { 2024-09-11T00:28:04.256 INFO:teuthology.orchestra.run.smithi012.stdout: "file:/": { 2024-09-11T00:28:04.256 INFO:teuthology.orchestra.run.smithi012.stdout: "excluded": false, 2024-09-11T00:28:04.257 INFO:teuthology.orchestra.run.smithi012.stdout: "state": { 2024-09-11T00:28:04.257 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "RELEASED", 2024-09-11T00:28:04.257 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 0.0 2024-09-11T00:28:04.257 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:28:04.257 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:28:04.257 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:28:04.257 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:28:04.257 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:28:04.257 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-09-11T00:28:04.258 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:04.253+0000 7f7c0ae00640 1 -- 172.21.15.12:0/282615662 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f7be80b51f0 msgr2=0x7f7be80b76b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:28:04.258 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:04.253+0000 7f7c0ae00640 1 --2- 172.21.15.12:0/282615662 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f7be80b51f0 0x7f7be80b76b0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x5649f0570fa0 tx=0x7f7bec005f90 comp rx=0 tx=0).stop 2024-09-11T00:28:04.258 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:04.253+0000 7f7c0ae00640 1 -- 172.21.15.12:0/282615662 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7c04075540 msgr2=0x7f7c0419b4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:28:04.258 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:04.253+0000 7f7c0ae00640 1 --2- 172.21.15.12:0/282615662 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7c04075540 0x7f7c0419b4a0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f7bf4005ab0 tx=0x7f7bf4003c10 comp rx=0 tx=0).stop 2024-09-11T00:28:04.258 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:04.253+0000 7f7c0ae00640 1 -- 172.21.15.12:0/282615662 shutdown_connections 2024-09-11T00:28:04.258 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:04.253+0000 7f7c0ae00640 1 --2- 172.21.15.12:0/282615662 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f7be80b51f0 0x7f7be80b76b0 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:04.258 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:04.253+0000 7f7c0ae00640 1 --2- 172.21.15.12:0/282615662 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7c0410f3e0 0x7f7c041a0d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:04.258 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:04.253+0000 7f7c0ae00640 1 --2- 172.21.15.12:0/282615662 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7c04075f40 0x7f7c0419b9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:04.259 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:04.253+0000 7f7c0ae00640 1 --2- 172.21.15.12:0/282615662 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7c04075540 0x7f7c0419b4a0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:04.259 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:04.254+0000 7f7c0ae00640 1 -- 172.21.15.12:0/282615662 >> 172.21.15.12:0/282615662 conn(0x7f7c040fe830 msgr2=0x7f7c041115c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:28:04.259 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:04.254+0000 7f7c0ae00640 1 -- 172.21.15.12:0/282615662 shutdown_connections 2024-09-11T00:28:04.259 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:04.254+0000 7f7c0ae00640 1 -- 172.21.15.12:0/282615662 wait complete. 2024-09-11T00:28:04.314 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:28:04 smithi012 ceph-mon[30333]: pgmap v1028: 129 pgs: 129 active+clean; 677 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 1.7 MiB/s wr, 78 op/s 2024-09-11T00:28:04.532 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:28:04 smithi165 ceph-mon[35356]: pgmap v1028: 129 pgs: 129 active+clean; 677 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 1.7 MiB/s wr, 78 op/s 2024-09-11T00:28:04.581 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:28:04 smithi178 ceph-mon[35919]: pgmap v1028: 129 pgs: 129 active+clean; 677 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 1.7 MiB/s wr, 78 op/s 2024-09-11T00:28:04.753 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:04.750+0000 7f8e26a00640 1 -- 172.21.15.12:0/2271242543 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8e20109230 msgr2=0x7f8e20109630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:28:04.753 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:04.750+0000 7f8e26a00640 1 --2- 172.21.15.12:0/2271242543 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8e20109230 0x7f8e20109630 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f8e10009900 tx=0x7f8e1002a3f0 comp rx=0 tx=0).stop 2024-09-11T00:28:04.753 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:04.751+0000 7f8e26a00640 1 -- 172.21.15.12:0/2271242543 shutdown_connections 2024-09-11T00:28:04.754 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:04.751+0000 7f8e26a00640 1 --2- 172.21.15.12:0/2271242543 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8e2010aac0 0x7f8e20111420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:04.754 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:04.751+0000 7f8e26a00640 1 --2- 172.21.15.12:0/2271242543 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8e2010a100 0x7f8e2010a580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:04.754 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:04.751+0000 7f8e26a00640 1 --2- 172.21.15.12:0/2271242543 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8e20109230 0x7f8e20109630 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:04.754 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:04.751+0000 7f8e26a00640 1 -- 172.21.15.12:0/2271242543 >> 172.21.15.12:0/2271242543 conn(0x7f8e200783d0 msgr2=0x7f8e2007a810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:28:04.754 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:04.751+0000 7f8e26a00640 1 -- 172.21.15.12:0/2271242543 shutdown_connections 2024-09-11T00:28:04.754 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:04.751+0000 7f8e26a00640 1 -- 172.21.15.12:0/2271242543 wait complete. 2024-09-11T00:28:04.755 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:04.753+0000 7f8e26a00640 1 Processor -- start 2024-09-11T00:28:04.755 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:04.753+0000 7f8e26a00640 1 -- start start 2024-09-11T00:28:04.757 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:04.754+0000 7f8e26a00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8e20109230 0x7f8e2019f340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:28:04.757 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:04.754+0000 7f8e26a00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8e2010a100 0x7f8e2019f990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:28:04.757 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:04.754+0000 7f8e26a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8e2010aac0 0x7f8e201a05c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:28:04.757 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:04.755+0000 7f8e1fe00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8e20109230 0x7f8e2019f340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:28:04.757 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:04.755+0000 7f8e1f400640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8e2010a100 0x7f8e2019f990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:28:04.758 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:04.755+0000 7f8e1f400640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8e2010a100 0x7f8e2019f990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.12:38380/0 (socket says 172.21.15.12:38380) 2024-09-11T00:28:04.758 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:04.755+0000 7f8e1f400640 1 -- 172.21.15.12:0/132210873 learned_addr learned my addr 172.21.15.12:0/132210873 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:28:04.758 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:04.755+0000 7f8e26a00640 1 -- 172.21.15.12:0/132210873 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f8e20114cb0 con 0x7f8e20109230 2024-09-11T00:28:04.758 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:04.756+0000 7f8e26a00640 1 -- 172.21.15.12:0/132210873 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f8e20114b30 con 0x7f8e2010aac0 2024-09-11T00:28:04.758 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:04.756+0000 7f8e26a00640 1 -- 172.21.15.12:0/132210873 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f8e20114e30 con 0x7f8e2010a100 2024-09-11T00:28:04.759 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:04.757+0000 7f8e24a00640 1 --2- 172.21.15.12:0/132210873 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8e2010aac0 0x7f8e201a05c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:28:04.759 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:04.757+0000 7f8e1fe00640 1 -- 172.21.15.12:0/132210873 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8e2010a100 msgr2=0x7f8e2019f990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:28:04.760 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:04.757+0000 7f8e1fe00640 1 --2- 172.21.15.12:0/132210873 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8e2010a100 0x7f8e2019f990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:04.760 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:04.757+0000 7f8e1fe00640 1 -- 172.21.15.12:0/132210873 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8e2010aac0 msgr2=0x7f8e201a05c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:28:04.760 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:04.757+0000 7f8e1fe00640 1 --2- 172.21.15.12:0/132210873 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8e2010aac0 0x7f8e201a05c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:04.760 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:04.757+0000 7f8e1fe00640 1 -- 172.21.15.12:0/132210873 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e201a0e80 con 0x7f8e20109230 2024-09-11T00:28:04.760 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:04.757+0000 7f8e24a00640 1 --2- 172.21.15.12:0/132210873 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8e2010aac0 0x7f8e201a05c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T00:28:04.760 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:04.757+0000 7f8e1f400640 1 --2- 172.21.15.12:0/132210873 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8e2010a100 0x7f8e2019f990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T00:28:04.760 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:04.758+0000 7f8e1fe00640 1 --2- 172.21.15.12:0/132210873 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8e20109230 0x7f8e2019f340 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f8e10009850 tx=0x7f8e10003d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:28:04.762 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:04.759+0000 7f8e1cc00640 1 -- 172.21.15.12:0/132210873 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e1003f070 con 0x7f8e20109230 2024-09-11T00:28:04.762 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:04.759+0000 7f8e1cc00640 1 -- 172.21.15.12:0/132210873 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8e10031df0 con 0x7f8e20109230 2024-09-11T00:28:04.762 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:04.759+0000 7f8e1cc00640 1 -- 172.21.15.12:0/132210873 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e10035030 con 0x7f8e20109230 2024-09-11T00:28:04.762 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:04.760+0000 7f8e26a00640 1 -- 172.21.15.12:0/132210873 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8e201a1110 con 0x7f8e20109230 2024-09-11T00:28:04.762 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:04.760+0000 7f8e26a00640 1 -- 172.21.15.12:0/132210873 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8e201a7660 con 0x7f8e20109230 2024-09-11T00:28:04.767 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:04.764+0000 7f8e29a2a740 1 Processor -- start 2024-09-11T00:28:04.767 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:04.765+0000 7f8e29a2a740 1 -- start start 2024-09-11T00:28:04.768 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:04.766+0000 7f8e29a2a740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d8c8b17790 0x55d8c8b19b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:28:04.768 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:04.766+0000 7f8e29a2a740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d8c8b1a0c0 0x55d8c8b1c4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:28:04.768 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:04.766+0000 7f8e29a2a740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d8c8b1cb00 0x55d8c8b1efc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:28:04.768 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:04.766+0000 7f8e29a2a740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55d8c8a8a6b0 con 0x55d8c8b1cb00 2024-09-11T00:28:04.768 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:04.766+0000 7f8e29a2a740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55d8c8a79ae0 con 0x55d8c8b17790 2024-09-11T00:28:04.768 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:04.766+0000 7f8e29a2a740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55d8c8a7d8f0 con 0x55d8c8b1a0c0 2024-09-11T00:28:04.771 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:04.766+0000 7f8e1cc00640 1 -- 172.21.15.12:0/132210873 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7f8e1003b640 con 0x7f8e20109230 2024-09-11T00:28:04.771 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:04.766+0000 7f8e24a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d8c8b17790 0x55d8c8b19b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:28:04.771 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:04.766+0000 7f8e1f400640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d8c8b1cb00 0x55d8c8b1efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:28:04.771 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:04.766+0000 7f8e24a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d8c8b17790 0x55d8c8b19b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.12:34866/0 (socket says 172.21.15.12:34866) 2024-09-11T00:28:04.771 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:04.766+0000 7f8e24a00640 1 -- 172.21.15.12:0/3055552758 learned_addr learned my addr 172.21.15.12:0/3055552758 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:28:04.771 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:04.767+0000 7f8e24a00640 1 -- 172.21.15.12:0/3055552758 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d8c8b1a0c0 msgr2=0x55d8c8b1c4b0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-11T00:28:04.771 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:04.767+0000 7f8e24a00640 1 --2- 172.21.15.12:0/3055552758 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d8c8b1a0c0 0x55d8c8b1c4b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:04.771 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:04.767+0000 7f8e24a00640 1 -- 172.21.15.12:0/3055552758 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d8c8b1cb00 msgr2=0x55d8c8b1efc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:28:04.771 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:04.767+0000 7f8e24a00640 1 --2- 172.21.15.12:0/3055552758 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d8c8b1cb00 0x55d8c8b1efc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:04.771 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:04.767+0000 7f8e24a00640 1 -- 172.21.15.12:0/3055552758 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d8c8a46780 con 0x55d8c8b17790 2024-09-11T00:28:04.771 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:04.767+0000 7f8e1f400640 1 --2- 172.21.15.12:0/3055552758 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d8c8b1cb00 0x55d8c8b1efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:28:04.772 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:04.767+0000 7f8e24a00640 1 --2- 172.21.15.12:0/3055552758 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d8c8b17790 0x55d8c8b19b80 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f8e18009f80 tx=0x7f8e1800c8b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cd4d824302135372 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:28:04.772 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:04.768+0000 7f8e15200640 1 -- 172.21.15.12:0/3055552758 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e1801a070 con 0x55d8c8b17790 2024-09-11T00:28:04.772 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:04.768+0000 7f8e15200640 1 -- 172.21.15.12:0/3055552758 <== mon.2 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8e180158c0 con 0x55d8c8b17790 2024-09-11T00:28:04.772 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:04.768+0000 7f8e15200640 1 -- 172.21.15.12:0/3055552758 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e1800a960 con 0x55d8c8b17790 2024-09-11T00:28:04.772 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:04.768+0000 7f8e1cc00640 1 --2- 172.21.15.12:0/132210873 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f8e040b54f0 0x7f8e040b79b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:28:04.773 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:04.769+0000 7f8e1cc00640 1 -- 172.21.15.12:0/132210873 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9208+0+0 (secure 0 0 0) 0x7f8e10106c80 con 0x7f8e20109230 2024-09-11T00:28:04.773 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:04.770+0000 7f8e29a2a740 1 -- 172.21.15.12:0/3055552758 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d8c8b17790 msgr2=0x55d8c8b19b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:28:04.773 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:04.770+0000 7f8e29a2a740 1 --2- 172.21.15.12:0/3055552758 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d8c8b17790 0x55d8c8b19b80 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f8e18009f80 tx=0x7f8e1800c8b0 comp rx=0 tx=0).stop 2024-09-11T00:28:04.773 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:04.770+0000 7f8e1fe00640 1 --2- 172.21.15.12:0/132210873 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f8e040b54f0 0x7f8e040b79b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:28:04.773 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:04.770+0000 7f8e29a2a740 1 -- 172.21.15.12:0/3055552758 shutdown_connections 2024-09-11T00:28:04.773 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:04.770+0000 7f8e29a2a740 1 --2- 172.21.15.12:0/3055552758 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d8c8b1cb00 0x55d8c8b1efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:04.773 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:04.770+0000 7f8e29a2a740 1 --2- 172.21.15.12:0/3055552758 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d8c8b1a0c0 0x55d8c8b1c4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:04.773 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:04.770+0000 7f8e29a2a740 1 --2- 172.21.15.12:0/3055552758 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d8c8b17790 0x55d8c8b19b80 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:04.773 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:04.770+0000 7f8e29a2a740 1 -- 172.21.15.12:0/3055552758 >> 172.21.15.12:0/3055552758 conn(0x55d8c8b128e0 msgr2=0x55d8c8b13670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:28:04.773 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:04.770+0000 7f8e29a2a740 1 -- 172.21.15.12:0/3055552758 shutdown_connections 2024-09-11T00:28:04.774 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:04.771+0000 7f8e29a2a740 1 -- 172.21.15.12:0/3055552758 wait complete. 2024-09-11T00:28:04.774 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:04.771+0000 7f8e1fe00640 1 --2- 172.21.15.12:0/132210873 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f8e040b54f0 0x7f8e040b79b0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f8e10032fd0 tx=0x7f8e1003b9e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:28:04.774 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:04.771+0000 7f8e29a2a740 1 -- start start 2024-09-11T00:28:04.774 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:04.772+0000 7f8e29a2a740 1 Processor -- start 2024-09-11T00:28:04.775 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:04.772+0000 7f8e29a2a740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d8c8b17790 0x55d8c8b20f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:28:04.775 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:04.772+0000 7f8e29a2a740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d8c8b1a0c0 0x55d8c8b215f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:28:04.775 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:04.772+0000 7f8e29a2a740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d8c8b1cb00 0x55d8c8b221c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:28:04.775 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:04.772+0000 7f8e29a2a740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55d8c89b1ab0 con 0x55d8c8b1a0c0 2024-09-11T00:28:04.775 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:04.772+0000 7f8e29a2a740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55d8c8a20570 con 0x55d8c8b1cb00 2024-09-11T00:28:04.775 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:04.772+0000 7f8e29a2a740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55d8c8b1fae0 con 0x55d8c8b17790 2024-09-11T00:28:04.775 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:04.772+0000 7f8e29a2a740 10 client.0 _ll_register_callbacks cb 0x55d8c8a2f100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:28:04.776 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:04.773+0000 7f8e1f400640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d8c8b1a0c0 0x55d8c8b215f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:28:04.776 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:04.773+0000 7f8e24a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d8c8b1cb00 0x55d8c8b221c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:28:04.776 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:04.773+0000 7f8e24a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d8c8b1cb00 0x55d8c8b221c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.12:34882/0 (socket says 172.21.15.12:34882) 2024-09-11T00:28:04.776 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:04.773+0000 7f8e24a00640 1 -- 172.21.15.12:0/1366785660 learned_addr learned my addr 172.21.15.12:0/1366785660 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:28:04.776 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:04.773+0000 7f8e24a00640 1 --2- 172.21.15.12:0/1366785660 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d8c8b17790 0x55d8c8b20f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:28:04.777 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:04.774+0000 7f8e1f400640 1 -- 172.21.15.12:0/1366785660 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d8c8b17790 msgr2=0x55d8c8b20f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:28:04.777 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:04.774+0000 7f8e1f400640 1 --2- 172.21.15.12:0/1366785660 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d8c8b17790 0x55d8c8b20f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:04.777 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:04.774+0000 7f8e1f400640 1 -- 172.21.15.12:0/1366785660 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d8c8b1cb00 msgr2=0x55d8c8b221c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:28:04.777 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:04.774+0000 7f8e1f400640 1 --2- 172.21.15.12:0/1366785660 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d8c8b1cb00 0x55d8c8b221c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:04.777 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:04.774+0000 7f8e1f400640 1 -- 172.21.15.12:0/1366785660 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d8c87db790 con 0x55d8c8b1a0c0 2024-09-11T00:28:04.778 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:04.775+0000 7f8e1f400640 1 --2- 172.21.15.12:0/1366785660 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d8c8b1a0c0 0x55d8c8b215f0 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f8e0c012700 tx=0x7f8e0c012bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a7f4aa74c01b4f0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:28:04.778 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:04.775+0000 7f8e09600640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-09-11T00:28:04.778 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:04.775+0000 7f8e09600640 1 -- 172.21.15.12:0/1366785660 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e0c010da0 con 0x55d8c8b1a0c0 2024-09-11T00:28:04.778 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:04.775+0000 7f8e09600640 1 -- 172.21.15.12:0/1366785660 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9208+0+0 (secure 0 0 0) 0x7f8e0c005020 con 0x55d8c8b1a0c0 2024-09-11T00:28:04.778 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:04.776+0000 7f8e29a2a740 1 -- 172.21.15.12:0/1366785660 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8e180158c0 con 0x55d8c8b1a0c0 2024-09-11T00:28:04.778 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:04.776+0000 7f8e09600640 1 -- 172.21.15.12:0/1366785660 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=86) ==== 24+0+0 (secure 0 0 0) 0x7f8e0c005220 con 0x55d8c8b1a0c0 2024-09-11T00:28:04.779 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:04.776+0000 7f8e29a2a740 10 client.17769 fetch_fsmap learned FSMap version 86 2024-09-11T00:28:04.779 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:04.776+0000 7f8e29a2a740 1 -- 172.21.15.12:0/1366785660 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=86,monmap=4+}) -- 0x55d8c8854620 con 0x55d8c8b1a0c0 2024-09-11T00:28:04.779 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:04.777+0000 7f8e09600640 1 -- 172.21.15.12:0/1366785660 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8e180158c0 con 0x55d8c8b1a0c0 2024-09-11T00:28:04.779 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:04.777+0000 7f8e09600640 1 -- 172.21.15.12:0/1366785660 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 86) ==== 8438+0+0 (secure 0 0 0) 0x7f8e0c019070 con 0x55d8c8b1a0c0 2024-09-11T00:28:04.779 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:04.777+0000 7f8e29a2a740 10 client.17769 fetch_fsmap finished waiting for FSMap version 86 2024-09-11T00:28:04.780 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:04.777+0000 7f8e29a2a740 10 client.17769 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T00:28:04.780 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:04.778+0000 7f8e29a2a740 20 client.17769 populate_metadata read hostname 'smithi012' 2024-09-11T00:28:04.780 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:04.778+0000 7f8e29a2a740 1 --2- 172.21.15.12:0/1366785660 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55d8c8b2c920 0x55d8c8b2ed40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:28:04.781 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:04.778+0000 7f8e29a2a740 4 client.17769 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T00:28:04.781 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:04.778+0000 7f8e29a2a740 1 -- 172.21.15.12:0/1366785660 --> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d8c8a8c600 con 0x55d8c8b2c920 2024-09-11T00:28:04.781 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:04.779+0000 7f8e24a00640 1 --2- 172.21.15.12:0/1366785660 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55d8c8b2c920 0x55d8c8b2ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:28:04.782 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:04.780+0000 7f8e24a00640 1 --2- 172.21.15.12:0/1366785660 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55d8c8b2c920 0x55d8c8b2ed40 crc :-1 s=READY pgs=291 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dc02714dd8b31cb6 server_cookie=3d86c41c3f500826 in_seq=0 out_seq=0 2024-09-11T00:28:04.782 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:04.780+0000 7f8e09600640 10 client.17769 ms_handle_connect on v2:172.21.15.165:6838/501312984 2024-09-11T00:28:04.783 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:04.781+0000 7f8e09600640 1 -- 172.21.15.12:0/1366785660 <== mds.0 v2:172.21.15.165:6838/501312984 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55d8c8a8c600 con 0x55d8c8b2c920 2024-09-11T00:28:04.783 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:04.781+0000 7f8e09600640 10 client.17769 handle_command_reply: tid=0 2024-09-11T00:28:04.783 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:04.781+0000 7f8e29a2a740 1 client.17769 shutdown 2024-09-11T00:28:04.784 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:04.782+0000 7f8e29a2a740 1 -- 172.21.15.12:0/1366785660 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d8c8b1a0c0 msgr2=0x55d8c8b215f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:28:04.784 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:04.782+0000 7f8e29a2a740 1 --2- 172.21.15.12:0/1366785660 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d8c8b1a0c0 0x55d8c8b215f0 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f8e0c012700 tx=0x7f8e0c012bc0 comp rx=0 tx=0).stop 2024-09-11T00:28:04.784 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:04.782+0000 7f8e29a2a740 1 -- 172.21.15.12:0/1366785660 shutdown_connections 2024-09-11T00:28:04.784 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:04.782+0000 7f8e29a2a740 1 --2- 172.21.15.12:0/1366785660 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55d8c8b2c920 0x55d8c8b2ed40 crc :-1 s=READY pgs=291 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:04.785 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:04.782+0000 7f8e29a2a740 1 --2- 172.21.15.12:0/1366785660 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d8c8b1cb00 0x55d8c8b221c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:04.785 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:04.782+0000 7f8e29a2a740 1 --2- 172.21.15.12:0/1366785660 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d8c8b1a0c0 0x55d8c8b215f0 secure :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f8e0c012700 tx=0x7f8e0c012bc0 comp rx=0 tx=0).stop 2024-09-11T00:28:04.785 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:04.782+0000 7f8e29a2a740 1 --2- 172.21.15.12:0/1366785660 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d8c8b17790 0x55d8c8b20f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:04.785 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:04.782+0000 7f8e29a2a740 1 -- 172.21.15.12:0/1366785660 >> 172.21.15.12:0/1366785660 conn(0x55d8c8b128e0 msgr2=0x55d8c8b11a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:28:04.785 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:04.782+0000 7f8e09600640 0 client.17769 ms_handle_reset on v2:172.21.15.165:6838/501312984 2024-09-11T00:28:04.785 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:04.782+0000 7f8e29a2a740 1 -- 172.21.15.12:0/1366785660 shutdown_connections 2024-09-11T00:28:04.785 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:04.783+0000 7f8e29a2a740 1 -- 172.21.15.12:0/1366785660 wait complete. 2024-09-11T00:28:04.785 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:04.783+0000 7f8e29a2a740 20 client.17769 trim_cache size 0 max 16384 2024-09-11T00:28:04.817 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:04.815+0000 7f8e29a2a740 1 Processor -- start 2024-09-11T00:28:04.817 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:04.815+0000 7f8e29a2a740 1 -- start start 2024-09-11T00:28:04.818 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:04.815+0000 7f8e29a2a740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d8c8b17790 0x55d8c8b0d960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:28:04.818 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:04.815+0000 7f8e29a2a740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d8c8b1cb00 0x55d8c8b1cee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:28:04.818 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:04.816+0000 7f8e29a2a740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d8c8b1d420 0x55d8c8b20f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:28:04.818 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:04.816+0000 7f8e29a2a740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f8e201b1cc0 con 0x55d8c8b1d420 2024-09-11T00:28:04.818 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:04.816+0000 7f8e29a2a740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f8e201b3a40 con 0x55d8c8b17790 2024-09-11T00:28:04.819 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:04.816+0000 7f8e29a2a740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f8e201b2450 con 0x55d8c8b1cb00 2024-09-11T00:28:04.819 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:04.816+0000 7f8e24a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d8c8b17790 0x55d8c8b0d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:28:04.819 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:04.816+0000 7f8e24a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d8c8b17790 0x55d8c8b0d960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.12:34886/0 (socket says 172.21.15.12:34886) 2024-09-11T00:28:04.819 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:04.816+0000 7f8e24a00640 1 -- 172.21.15.12:0/1040757537 learned_addr learned my addr 172.21.15.12:0/1040757537 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:28:04.820 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:04.817+0000 7f8e24a00640 1 -- 172.21.15.12:0/1040757537 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d8c8b1cb00 msgr2=0x55d8c8b1cee0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-11T00:28:04.820 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:04.817+0000 7f8e24a00640 1 --2- 172.21.15.12:0/1040757537 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d8c8b1cb00 0x55d8c8b1cee0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:04.820 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:04.817+0000 7f8e24a00640 1 -- 172.21.15.12:0/1040757537 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d8c8b1d420 msgr2=0x55d8c8b20f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:28:04.820 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:04.817+0000 7f8e24a00640 1 --2- 172.21.15.12:0/1040757537 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d8c8b1d420 0x55d8c8b20f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:04.820 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:04.817+0000 7f8e24a00640 1 -- 172.21.15.12:0/1040757537 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d8c8b295b0 con 0x55d8c8b17790 2024-09-11T00:28:04.820 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:04.817+0000 7f8e24a00640 1 --2- 172.21.15.12:0/1040757537 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d8c8b17790 0x55d8c8b0d960 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f8e180093f0 tx=0x7f8e18004b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f702ad02c5525b60 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:28:04.820 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:04.818+0000 7f8e0be00640 1 -- 172.21.15.12:0/1040757537 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e1801a030 con 0x55d8c8b17790 2024-09-11T00:28:04.821 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:04.818+0000 7f8e0be00640 1 -- 172.21.15.12:0/1040757537 <== mon.2 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8e1801b040 con 0x55d8c8b17790 2024-09-11T00:28:04.821 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:04.818+0000 7f8e0be00640 1 -- 172.21.15.12:0/1040757537 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e1800a710 con 0x55d8c8b17790 2024-09-11T00:28:04.821 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:04.819+0000 7f8e29a2a740 1 -- 172.21.15.12:0/1040757537 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d8c8b17790 msgr2=0x55d8c8b0d960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:28:04.821 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:04.819+0000 7f8e29a2a740 1 --2- 172.21.15.12:0/1040757537 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d8c8b17790 0x55d8c8b0d960 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f8e180093f0 tx=0x7f8e18004b50 comp rx=0 tx=0).stop 2024-09-11T00:28:04.821 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:04.819+0000 7f8e29a2a740 1 -- 172.21.15.12:0/1040757537 shutdown_connections 2024-09-11T00:28:04.821 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:04.819+0000 7f8e29a2a740 1 --2- 172.21.15.12:0/1040757537 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d8c8b1d420 0x55d8c8b20f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:04.821 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:04.819+0000 7f8e29a2a740 1 --2- 172.21.15.12:0/1040757537 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d8c8b1cb00 0x55d8c8b1cee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:04.821 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:04.819+0000 7f8e29a2a740 1 --2- 172.21.15.12:0/1040757537 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d8c8b17790 0x55d8c8b0d960 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:04.822 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:04.819+0000 7f8e29a2a740 1 -- 172.21.15.12:0/1040757537 >> 172.21.15.12:0/1040757537 conn(0x55d8c8b128e0 msgr2=0x55d8c8b16cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:28:04.822 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:04.820+0000 7f8e29a2a740 1 -- 172.21.15.12:0/1040757537 shutdown_connections 2024-09-11T00:28:04.823 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:04.820+0000 7f8e29a2a740 1 -- 172.21.15.12:0/1040757537 wait complete. 2024-09-11T00:28:04.823 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:04.821+0000 7f8e29a2a740 1 -- start start 2024-09-11T00:28:04.823 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:04.821+0000 7f8e29a2a740 1 Processor -- start 2024-09-11T00:28:04.824 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:04.822+0000 7f8e29a2a740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d8c8b17790 0x55d8c8b27c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:28:04.825 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:04.822+0000 7f8e29a2a740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d8c8b1cb00 0x55d8c8b28160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:28:04.825 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:04.822+0000 7f8e29a2a740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d8c8b1d420 0x55d8c8b286e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:28:04.825 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:04.822+0000 7f8e29a2a740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f8e201b3370 con 0x55d8c8b17790 2024-09-11T00:28:04.825 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:04.822+0000 7f8e29a2a740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f8e201b2be0 con 0x55d8c8b1cb00 2024-09-11T00:28:04.825 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:04.822+0000 7f8e29a2a740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f8e201afca0 con 0x55d8c8b1d420 2024-09-11T00:28:04.826 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:04.822+0000 7f8e29a2a740 10 client.0 _ll_register_callbacks cb 0x7f8e0c00a970 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:28:04.826 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:04.823+0000 7f8e1f400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d8c8b1cb00 0x55d8c8b28160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:28:04.826 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:04.823+0000 7f8e1f400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d8c8b1cb00 0x55d8c8b28160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.12:34888/0 (socket says 172.21.15.12:34888) 2024-09-11T00:28:04.826 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:04.823+0000 7f8e1f400640 1 -- 172.21.15.12:0/1584518699 learned_addr learned my addr 172.21.15.12:0/1584518699 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:28:04.826 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:04.823+0000 7f8e24a00640 1 --2- 172.21.15.12:0/1584518699 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d8c8b1d420 0x55d8c8b286e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:28:04.826 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:04.823+0000 7f8e24a00640 1 --2- 172.21.15.12:0/1584518699 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d8c8b17790 0x55d8c8b27c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:28:04.827 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:04.823+0000 7f8e1f400640 1 -- 172.21.15.12:0/1584518699 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d8c8b1d420 msgr2=0x55d8c8b286e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:28:04.827 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:04.823+0000 7f8e1f400640 1 --2- 172.21.15.12:0/1584518699 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d8c8b1d420 0x55d8c8b286e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:04.827 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:04.824+0000 7f8e1f400640 1 -- 172.21.15.12:0/1584518699 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d8c8b17790 msgr2=0x55d8c8b27c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:28:04.827 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:04.824+0000 7f8e1f400640 1 --2- 172.21.15.12:0/1584518699 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d8c8b17790 0x55d8c8b27c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:04.827 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:04.824+0000 7f8e1f400640 1 -- 172.21.15.12:0/1584518699 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d8c8b26a30 con 0x55d8c8b1cb00 2024-09-11T00:28:04.828 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:04.824+0000 7f8e24a00640 1 --2- 172.21.15.12:0/1584518699 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d8c8b1d420 0x55d8c8b286e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T00:28:04.828 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:04.824+0000 7f8e1f400640 1 --2- 172.21.15.12:0/1584518699 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d8c8b1cb00 0x55d8c8b28160 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f8e0c024700 tx=0x7f8e0c024eb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=186871230015f7ce server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:28:04.828 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:04.824+0000 7f8e09600640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-09-11T00:28:04.828 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:04.825+0000 7f8e09600640 1 -- 172.21.15.12:0/1584518699 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e0c010da0 con 0x55d8c8b1cb00 2024-09-11T00:28:04.828 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:04.825+0000 7f8e09600640 1 -- 172.21.15.12:0/1584518699 <== mon.2 v2:172.21.15.165:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9208+0+0 (secure 0 0 0) 0x7f8e0c004660 con 0x55d8c8b1cb00 2024-09-11T00:28:04.828 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:04.825+0000 7f8e29a2a740 1 -- 172.21.15.12:0/1584518699 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8e1801b040 con 0x55d8c8b1cb00 2024-09-11T00:28:04.829 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:04.825+0000 7f8e09600640 1 -- 172.21.15.12:0/1584518699 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=86) ==== 24+0+0 (secure 0 0 0) 0x7f8e0c036050 con 0x55d8c8b1cb00 2024-09-11T00:28:04.829 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:04.826+0000 7f8e29a2a740 10 client.27776 fetch_fsmap learned FSMap version 86 2024-09-11T00:28:04.829 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:04.826+0000 7f8e29a2a740 1 -- 172.21.15.12:0/1584518699 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=86,monmap=4+}) -- 0x55d8c8b99de0 con 0x55d8c8b1cb00 2024-09-11T00:28:04.829 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:04.826+0000 7f8e09600640 1 -- 172.21.15.12:0/1584518699 <== mon.2 v2:172.21.15.165:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8e1801b040 con 0x55d8c8b1cb00 2024-09-11T00:28:04.829 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:04.827+0000 7f8e09600640 1 -- 172.21.15.12:0/1584518699 <== mon.2 v2:172.21.15.165:3300/0 5 ==== fsmap(e 86) ==== 8438+0+0 (secure 0 0 0) 0x7f8e0c019040 con 0x55d8c8b1cb00 2024-09-11T00:28:04.830 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:04.827+0000 7f8e29a2a740 10 client.27776 fetch_fsmap finished waiting for FSMap version 86 2024-09-11T00:28:04.830 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:04.827+0000 7f8e29a2a740 10 client.27776 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T00:28:04.830 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:04.827+0000 7f8e29a2a740 20 client.27776 populate_metadata read hostname 'smithi012' 2024-09-11T00:28:04.830 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:04.827+0000 7f8e29a2a740 1 --2- 172.21.15.12:0/1584518699 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55d8c8b9a7d0 0x55d8c8b9cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:28:04.830 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:04.827+0000 7f8e29a2a740 4 client.27776 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T00:28:04.830 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:04.827+0000 7f8e29a2a740 1 -- 172.21.15.12:0/1584518699 --> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55d8c8a880b0 con 0x55d8c8b9a7d0 2024-09-11T00:28:04.830 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:04.827+0000 7f8e24a00640 1 --2- 172.21.15.12:0/1584518699 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55d8c8b9a7d0 0x55d8c8b9cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:28:04.831 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:04.828+0000 7f8e24a00640 1 --2- 172.21.15.12:0/1584518699 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55d8c8b9a7d0 0x55d8c8b9cbf0 crc :-1 s=READY pgs=292 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c93e575792ba64b5 server_cookie=2293ebd8b0729cd9 in_seq=0 out_seq=0 2024-09-11T00:28:04.831 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:04.829+0000 7f8e09600640 10 client.27776 ms_handle_connect on v2:172.21.15.165:6838/501312984 2024-09-11T00:28:04.948 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:04.945+0000 7f8e09600640 1 -- 172.21.15.12:0/1584518699 <== mds.0 v2:172.21.15.165:6838/501312984 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55d8c8a880b0 con 0x55d8c8b9a7d0 2024-09-11T00:28:04.948 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:04.945+0000 7f8e09600640 10 client.27776 handle_command_reply: tid=0 2024-09-11T00:28:04.948 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:04.945+0000 7f8e29a2a740 1 client.27776 shutdown 2024-09-11T00:28:04.949 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:04.946+0000 7f8e29a2a740 1 -- 172.21.15.12:0/1584518699 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d8c8b1cb00 msgr2=0x55d8c8b28160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:28:04.949 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:04.946+0000 7f8e29a2a740 1 --2- 172.21.15.12:0/1584518699 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d8c8b1cb00 0x55d8c8b28160 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f8e0c024700 tx=0x7f8e0c024eb0 comp rx=0 tx=0).stop 2024-09-11T00:28:04.949 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:04.946+0000 7f8e29a2a740 1 -- 172.21.15.12:0/1584518699 shutdown_connections 2024-09-11T00:28:04.949 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:04.946+0000 7f8e29a2a740 1 --2- 172.21.15.12:0/1584518699 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55d8c8b9a7d0 0x55d8c8b9cbf0 crc :-1 s=READY pgs=292 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:04.949 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:04.946+0000 7f8e09600640 0 client.27776 ms_handle_reset on v2:172.21.15.165:6838/501312984 2024-09-11T00:28:04.949 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:04.946+0000 7f8e29a2a740 1 --2- 172.21.15.12:0/1584518699 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d8c8b1d420 0x55d8c8b286e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:04.949 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:04.946+0000 7f8e29a2a740 1 --2- 172.21.15.12:0/1584518699 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d8c8b1cb00 0x55d8c8b28160 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:04.950 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:04.946+0000 7f8e29a2a740 1 --2- 172.21.15.12:0/1584518699 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d8c8b17790 0x55d8c8b27c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:04.950 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:04.946+0000 7f8e29a2a740 1 -- 172.21.15.12:0/1584518699 >> 172.21.15.12:0/1584518699 conn(0x55d8c8b128e0 msgr2=0x55d8c8b20f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:28:04.950 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:04.947+0000 7f8e29a2a740 1 -- 172.21.15.12:0/1584518699 shutdown_connections 2024-09-11T00:28:04.950 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:04.947+0000 7f8e29a2a740 1 -- 172.21.15.12:0/1584518699 wait complete. 2024-09-11T00:28:04.950 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:04.947+0000 7f8e29a2a740 20 client.27776 trim_cache size 0 max 16384 2024-09-11T00:28:04.950 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:{ 2024-09-11T00:28:04.950 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "message": "", 2024-09-11T00:28:04.950 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "return_code": 0 2024-09-11T00:28:04.950 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:} 2024-09-11T00:28:04.951 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:04.949+0000 7f8e26a00640 1 -- 172.21.15.12:0/132210873 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f8e040b54f0 msgr2=0x7f8e040b79b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:28:04.951 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:04.949+0000 7f8e26a00640 1 --2- 172.21.15.12:0/132210873 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f8e040b54f0 0x7f8e040b79b0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f8e10032fd0 tx=0x7f8e1003b9e0 comp rx=0 tx=0).stop 2024-09-11T00:28:04.951 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:04.949+0000 7f8e26a00640 1 -- 172.21.15.12:0/132210873 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8e20109230 msgr2=0x7f8e2019f340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:28:04.951 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:04.949+0000 7f8e26a00640 1 --2- 172.21.15.12:0/132210873 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8e20109230 0x7f8e2019f340 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f8e10009850 tx=0x7f8e10003d60 comp rx=0 tx=0).stop 2024-09-11T00:28:04.951 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:04.949+0000 7f8e26a00640 1 -- 172.21.15.12:0/132210873 shutdown_connections 2024-09-11T00:28:04.951 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:04.949+0000 7f8e26a00640 1 --2- 172.21.15.12:0/132210873 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f8e040b54f0 0x7f8e040b79b0 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:04.952 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:04.949+0000 7f8e26a00640 1 --2- 172.21.15.12:0/132210873 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8e2010aac0 0x7f8e201a05c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:04.952 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:04.949+0000 7f8e26a00640 1 --2- 172.21.15.12:0/132210873 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8e2010a100 0x7f8e2019f990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:04.952 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:04.949+0000 7f8e26a00640 1 --2- 172.21.15.12:0/132210873 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8e20109230 0x7f8e2019f340 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:04.952 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:04.949+0000 7f8e26a00640 1 -- 172.21.15.12:0/132210873 >> 172.21.15.12:0/132210873 conn(0x7f8e200783d0 msgr2=0x7f8e2010d400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:28:04.953 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:04.950+0000 7f8e26a00640 1 -- 172.21.15.12:0/132210873 shutdown_connections 2024-09-11T00:28:04.953 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:04.951+0000 7f8e26a00640 1 -- 172.21.15.12:0/132210873 wait complete. 2024-09-11T00:28:05.171 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'fae7af54', total seconds elapsed: 33.9 2024-09-11T00:28:05.172 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 30.0 2024-09-11T00:28:05.172 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid ff3ad74a-6fcf-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T00:28:05.248 INFO:tasks.workunit.client.0.smithi012.stdout: 10 131811 14.76 MB/sec execute 361 sec latency 29036.979 ms 2024-09-11T00:28:05.581 INFO:journalctl@ceph.mds.c.smithi178.stdout:Sep 11 00:28:05 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-mds-c[66074]: ignoring --setuser ceph since I am not root 2024-09-11T00:28:05.582 INFO:journalctl@ceph.mds.c.smithi178.stdout:Sep 11 00:28:05 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-mds-c[66074]: ignoring --setgroup ceph since I am not root 2024-09-11T00:28:05.582 INFO:journalctl@ceph.mds.c.smithi178.stdout:Sep 11 00:28:05 smithi178 ceph-mds[66078]: -- 172.21.15.178:0/1108688755 <== mon.2 v2:172.21.15.165:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55e8371d5c20 con 0x55e8371b2800 2024-09-11T00:28:05.582 INFO:journalctl@ceph.mds.c.smithi178.stdout:Sep 11 00:28:05 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-mds-c[66074]: starting mds.c at 2024-09-11T00:28:05.874 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:28:06.248 INFO:tasks.workunit.client.0.smithi012.stdout: 10 132436 14.81 MB/sec execute 362 sec latency 73.421 ms 2024-09-11T00:28:06.314 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:28:06 smithi012 ceph-mon[30333]: pgmap v1029: 129 pgs: 129 active+clean; 677 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 1.5 MiB/s wr, 67 op/s 2024-09-11T00:28:06.314 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:28:06 smithi012 ceph-mon[30333]: daemon mds.c restarted 2024-09-11T00:28:06.532 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:28:06 smithi165 ceph-mon[35356]: pgmap v1029: 129 pgs: 129 active+clean; 677 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 1.5 MiB/s wr, 67 op/s 2024-09-11T00:28:06.532 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:28:06 smithi165 ceph-mon[35356]: daemon mds.c restarted 2024-09-11T00:28:06.581 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:28:06 smithi178 ceph-mon[35919]: pgmap v1029: 129 pgs: 129 active+clean; 677 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 1.5 MiB/s wr, 67 op/s 2024-09-11T00:28:06.581 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:28:06 smithi178 ceph-mon[35919]: daemon mds.c restarted 2024-09-11T00:28:06.597 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:06.594+0000 7f47de200640 1 -- 172.21.15.12:0/4041295994 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f47d8107360 msgr2=0x7f47d810db60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:28:06.597 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:06.594+0000 7f47de200640 1 --2- 172.21.15.12:0/4041295994 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f47d8107360 0x7f47d810db60 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f47c8009900 tx=0x7f47c802a3f0 comp rx=0 tx=0).stop 2024-09-11T00:28:06.598 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:06.596+0000 7f47de200640 1 -- 172.21.15.12:0/4041295994 shutdown_connections 2024-09-11T00:28:06.598 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:06.596+0000 7f47de200640 1 --2- 172.21.15.12:0/4041295994 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f47d8107360 0x7f47d810db60 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:06.598 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:06.596+0000 7f47de200640 1 --2- 172.21.15.12:0/4041295994 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f47d81069e0 0x7f47d8106e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:06.598 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:06.596+0000 7f47de200640 1 --2- 172.21.15.12:0/4041295994 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f47d8100fd0 0x7f47d81013b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:06.598 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:06.596+0000 7f47de200640 1 -- 172.21.15.12:0/4041295994 >> 172.21.15.12:0/4041295994 conn(0x7f47d80fca20 msgr2=0x7f47d80fee40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:28:06.599 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:06.596+0000 7f47de200640 1 -- 172.21.15.12:0/4041295994 shutdown_connections 2024-09-11T00:28:06.599 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:06.596+0000 7f47de200640 1 -- 172.21.15.12:0/4041295994 wait complete. 2024-09-11T00:28:06.599 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:06.597+0000 7f47de200640 1 Processor -- start 2024-09-11T00:28:06.600 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:06.598+0000 7f47de200640 1 -- start start 2024-09-11T00:28:06.601 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:06.598+0000 7f47de200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f47d8100fd0 0x7f47d81992c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:28:06.601 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:06.598+0000 7f47de200640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f47d81069e0 0x7f47d8199800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:28:06.601 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:06.598+0000 7f47de200640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f47d8107360 0x7f47d819eba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:28:06.601 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:06.598+0000 7f47de200640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f47d8112e30 con 0x7f47d81069e0 2024-09-11T00:28:06.601 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:06.598+0000 7f47de200640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f47d8112cb0 con 0x7f47d8100fd0 2024-09-11T00:28:06.601 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:06.598+0000 7f47de200640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f47d8112fb0 con 0x7f47d8107360 2024-09-11T00:28:06.601 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:06.599+0000 7f47d7e00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f47d8107360 0x7f47d819eba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:28:06.601 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:06.599+0000 7f47d7400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f47d8100fd0 0x7f47d81992c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:28:06.602 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:06.599+0000 7f47d7e00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f47d8107360 0x7f47d819eba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.12:38422/0 (socket says 172.21.15.12:38422) 2024-09-11T00:28:06.602 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:06.599+0000 7f47d7e00640 1 -- 172.21.15.12:0/2931424924 learned_addr learned my addr 172.21.15.12:0/2931424924 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:28:06.602 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:06.599+0000 7f47d6a00640 1 --2- 172.21.15.12:0/2931424924 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f47d81069e0 0x7f47d8199800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:28:06.602 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:06.600+0000 7f47d7400640 1 -- 172.21.15.12:0/2931424924 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f47d8107360 msgr2=0x7f47d819eba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:28:06.603 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:06.600+0000 7f47d7400640 1 --2- 172.21.15.12:0/2931424924 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f47d8107360 0x7f47d819eba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:06.603 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:06.600+0000 7f47d7400640 1 -- 172.21.15.12:0/2931424924 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f47d81069e0 msgr2=0x7f47d8199800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:28:06.603 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:06.600+0000 7f47d7400640 1 --2- 172.21.15.12:0/2931424924 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f47d81069e0 0x7f47d8199800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:06.604 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:06.600+0000 7f47d7400640 1 -- 172.21.15.12:0/2931424924 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47d819f370 con 0x7f47d8100fd0 2024-09-11T00:28:06.604 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:06.600+0000 7f47d6a00640 1 --2- 172.21.15.12:0/2931424924 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f47d81069e0 0x7f47d8199800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:28:06.605 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:06.603+0000 7f47d7400640 1 --2- 172.21.15.12:0/2931424924 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f47d8100fd0 0x7f47d81992c0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f47c400e9f0 tx=0x7f47c400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:28:06.606 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:06.603+0000 7f47cfe00640 1 -- 172.21.15.12:0/2931424924 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47c400ccc0 con 0x7f47d8100fd0 2024-09-11T00:28:06.606 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:06.603+0000 7f47cfe00640 1 -- 172.21.15.12:0/2931424924 <== mon.2 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f47c40108e0 con 0x7f47d8100fd0 2024-09-11T00:28:06.606 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:06.603+0000 7f47cfe00640 1 -- 172.21.15.12:0/2931424924 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47c4018880 con 0x7f47d8100fd0 2024-09-11T00:28:06.606 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:06.603+0000 7f47de200640 1 -- 172.21.15.12:0/2931424924 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f47d819f660 con 0x7f47d8100fd0 2024-09-11T00:28:06.606 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:06.604+0000 7f47de200640 1 -- 172.21.15.12:0/2931424924 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47d806cb70 con 0x7f47d8100fd0 2024-09-11T00:28:06.609 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:06.606+0000 7f47de200640 1 -- 172.21.15.12:0/2931424924 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f47d81013b0 con 0x7f47d8100fd0 2024-09-11T00:28:06.613 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:06.611+0000 7f47cfe00640 1 -- 172.21.15.12:0/2931424924 <== mon.2 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7f47c4010430 con 0x7f47d8100fd0 2024-09-11T00:28:06.616 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:06.613+0000 7f47cfe00640 1 --2- 172.21.15.12:0/2931424924 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f47bc0b5430 0x7f47bc0b78f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:28:06.616 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:06.613+0000 7f47cfe00640 1 -- 172.21.15.12:0/2931424924 <== mon.2 v2:172.21.15.165:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9208+0+0 (secure 0 0 0) 0x7f47c40e6250 con 0x7f47d8100fd0 2024-09-11T00:28:06.617 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:06.615+0000 7f47d6a00640 1 --2- 172.21.15.12:0/2931424924 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f47bc0b5430 0x7f47bc0b78f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:28:06.618 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:06.616+0000 7f47d6a00640 1 --2- 172.21.15.12:0/2931424924 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f47bc0b5430 0x7f47bc0b78f0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f47d0009fd0 tx=0x7f47d0009680 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:28:06.623 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:06.620+0000 7f47cfe00640 1 -- 172.21.15.12:0/2931424924 <== mon.2 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f47c40acd40 con 0x7f47d8100fd0 2024-09-11T00:28:07.063 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:07.061+0000 7f47de200640 1 -- 172.21.15.12:0/2931424924 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f47d802e5c0 con 0x7f47d8100fd0 2024-09-11T00:28:07.064 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:07.061+0000 7f47cfe00640 1 -- 172.21.15.12:0/2931424924 <== mon.2 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 88 v88) ==== 94+0+11997 (secure 0 0 0) 0x7f47c40ac640 con 0x7f47d8100fd0 2024-09-11T00:28:07.066 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-11T00:28:07.067 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":88,"btime":"2024-09-11T00:28:06:291486+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14577,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6835/447171591","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":447171591},{"type":"v1","addr":"172.21.15.12:6835","nonce":447171591}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14589,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/901486040","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":901486040},{"type":"v1","addr":"172.21.15.12:6837","nonce":901486040}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":88,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T00:04:04.739881+0000","modified":"2024-09-11T00:28:06.291465+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24674,"mds_1":24643,"mds_2":24638,"mds_3":24637,"mds_4":24614},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14631":{"gid":14631,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.12:6841/3069919437","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6840","nonce":3069919437},{"type":"v1","addr":"172.21.15.12:6841","nonce":3069919437}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24562":{"gid":24562,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6833/1141636112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":1141636112},{"type":"v1","addr":"172.21.15.178:6833","nonce":1141636112}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24581":{"gid":24581,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6833/3111980970","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6832","nonce":3111980970},{"type":"v1","addr":"172.21.15.165:6833","nonce":3111980970}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24607":{"gid":24607,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6835/4084868287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":4084868287},{"type":"v1","addr":"172.21.15.178:6835","nonce":4084868287}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24614":{"gid":24614,"name":"e","rank":4,"incarnation":26,"state":"up:active","state_seq":21,"addr":"172.21.15.165:6835/2425395274","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":2425395274},{"type":"v1","addr":"172.21.15.165:6835","nonce":2425395274}]},"join_fscid":-1,"export_targets":[0],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24637":{"gid":24637,"name":"i","rank":3,"incarnation":24,"state":"up:active","state_seq":18,"addr":"172.21.15.178:6839/70628097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6838","nonce":70628097},{"type":"v1","addr":"172.21.15.178:6839","nonce":70628097}]},"join_fscid":-1,"export_targets":[0,1,2,4],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24638":{"gid":24638,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":18,"addr":"172.21.15.165:6837/6215088","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6836","nonce":6215088},{"type":"v1","addr":"172.21.15.165:6837","nonce":6215088}]},"join_fscid":-1,"export_targets":[0,1,3,4],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24643":{"gid":24643,"name":"g","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.12:6839/3124093009","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6838","nonce":3124093009},{"type":"v1","addr":"172.21.15.12:6839","nonce":3124093009}]},"join_fscid":-1,"export_targets":[0,2,3,4],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24674":{"gid":24674,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.165:6839/501312984","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6838","nonce":501312984},{"type":"v1","addr":"172.21.15.165:6839","nonce":501312984}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_27751":{"gid":27751,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6837/1436600611","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":1436600611},{"type":"v1","addr":"172.21.15.178:6837","nonce":1436600611}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24674,"qdb_cluster":[24674,24643,24638,24637,24614]},"id":1}]} 2024-09-11T00:28:07.067 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 88 2024-09-11T00:28:07.067 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:07.065+0000 7f47de200640 1 -- 172.21.15.12:0/2931424924 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f47bc0b5430 msgr2=0x7f47bc0b78f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:28:07.067 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:07.065+0000 7f47de200640 1 --2- 172.21.15.12:0/2931424924 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f47bc0b5430 0x7f47bc0b78f0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f47d0009fd0 tx=0x7f47d0009680 comp rx=0 tx=0).stop 2024-09-11T00:28:07.068 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:07.065+0000 7f47de200640 1 -- 172.21.15.12:0/2931424924 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f47d8100fd0 msgr2=0x7f47d81992c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:28:07.068 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:07.065+0000 7f47de200640 1 --2- 172.21.15.12:0/2931424924 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f47d8100fd0 0x7f47d81992c0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f47c400e9f0 tx=0x7f47c400eeb0 comp rx=0 tx=0).stop 2024-09-11T00:28:07.068 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:07.065+0000 7f47de200640 1 -- 172.21.15.12:0/2931424924 shutdown_connections 2024-09-11T00:28:07.068 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:07.065+0000 7f47de200640 1 --2- 172.21.15.12:0/2931424924 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f47bc0b5430 0x7f47bc0b78f0 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:07.068 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:07.065+0000 7f47de200640 1 --2- 172.21.15.12:0/2931424924 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f47d8107360 0x7f47d819eba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:07.068 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:07.065+0000 7f47de200640 1 --2- 172.21.15.12:0/2931424924 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f47d81069e0 0x7f47d8199800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:07.069 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:07.065+0000 7f47de200640 1 --2- 172.21.15.12:0/2931424924 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f47d8100fd0 0x7f47d81992c0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:07.069 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:07.065+0000 7f47de200640 1 -- 172.21.15.12:0/2931424924 >> 172.21.15.12:0/2931424924 conn(0x7f47d80fca20 msgr2=0x7f47d80fd2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:28:07.069 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:07.066+0000 7f47de200640 1 -- 172.21.15.12:0/2931424924 shutdown_connections 2024-09-11T00:28:07.069 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:07.066+0000 7f47de200640 1 -- 172.21.15.12:0/2931424924 wait complete. 2024-09-11T00:28:07.248 INFO:tasks.workunit.client.0.smithi012.stdout: 10 132947 14.83 MB/sec execute 363 sec latency 127.405 ms 2024-09-11T00:28:07.532 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:28:07 smithi165 ceph-mon[35356]: osdmap e87: 12 total, 12 up, 12 in 2024-09-11T00:28:07.533 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:28:07 smithi165 ceph-mon[35356]: mds.? [v2:172.21.15.178:6836/1436600611,v1:172.21.15.178:6837/1436600611] up:boot 2024-09-11T00:28:07.533 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:28:07 smithi165 ceph-mon[35356]: fsmap cephfs:5 4 up:standby-replay 3 up:standby 5 up:active 2024-09-11T00:28:07.533 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:28:07 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "c"} : dispatch 2024-09-11T00:28:07.533 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:28:07 smithi165 ceph-mon[35356]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T00:28:07.533 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:28:07 smithi165 ceph-mon[35356]: from='client.? 172.21.15.12:0/2931424924' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T00:28:07.564 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:28:07 smithi012 ceph-mon[30333]: osdmap e87: 12 total, 12 up, 12 in 2024-09-11T00:28:07.564 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:28:07 smithi012 ceph-mon[30333]: mds.? [v2:172.21.15.178:6836/1436600611,v1:172.21.15.178:6837/1436600611] up:boot 2024-09-11T00:28:07.564 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:28:07 smithi012 ceph-mon[30333]: fsmap cephfs:5 4 up:standby-replay 3 up:standby 5 up:active 2024-09-11T00:28:07.564 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:28:07 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "c"} : dispatch 2024-09-11T00:28:07.564 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:28:07 smithi012 ceph-mon[30333]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T00:28:07.565 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:28:07 smithi012 ceph-mon[30333]: from='client.? 172.21.15.12:0/2931424924' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T00:28:07.581 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:28:07 smithi178 ceph-mon[35919]: osdmap e87: 12 total, 12 up, 12 in 2024-09-11T00:28:07.581 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:28:07 smithi178 ceph-mon[35919]: mds.? [v2:172.21.15.178:6836/1436600611,v1:172.21.15.178:6837/1436600611] up:boot 2024-09-11T00:28:07.581 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:28:07 smithi178 ceph-mon[35919]: fsmap cephfs:5 4 up:standby-replay 3 up:standby 5 up:active 2024-09-11T00:28:07.581 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:28:07 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "c"} : dispatch 2024-09-11T00:28:07.581 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:28:07 smithi178 ceph-mon[35919]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T00:28:07.581 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:28:07 smithi178 ceph-mon[35919]: from='client.? 172.21.15.12:0/2931424924' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T00:28:08.248 INFO:tasks.workunit.client.0.smithi012.stdout: 10 133609 14.88 MB/sec execute 364 sec latency 105.251 ms 2024-09-11T00:28:08.532 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:28:08 smithi165 ceph-mon[35356]: pgmap v1031: 129 pgs: 129 active+clean; 676 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 545 KiB/s wr, 108 op/s 2024-09-11T00:28:08.532 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:28:08 smithi165 ceph-mon[35356]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T00:28:08.563 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:28:08 smithi012 ceph-mon[30333]: pgmap v1031: 129 pgs: 129 active+clean; 676 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 545 KiB/s wr, 108 op/s 2024-09-11T00:28:08.563 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:28:08 smithi012 ceph-mon[30333]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T00:28:08.581 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:28:08 smithi178 ceph-mon[35919]: pgmap v1031: 129 pgs: 129 active+clean; 676 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 545 KiB/s wr, 108 op/s 2024-09-11T00:28:08.581 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:28:08 smithi178 ceph-mon[35919]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T00:28:09.198 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24674 quiesce db / --timeout 90 --expiration 150.0 --await-for 45.0' 2024-09-11T00:28:09.199 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid ff3ad74a-6fcf-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24674 quiesce db / --timeout 90 --expiration 150.0 --await-for 45.0 2024-09-11T00:28:09.248 INFO:tasks.workunit.client.0.smithi012.stdout: 10 134361 14.95 MB/sec execute 365 sec latency 117.857 ms 2024-09-11T00:28:09.563 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:28:09 smithi012 ceph-mon[30333]: pgmap v1032: 129 pgs: 129 active+clean; 677 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 690 KiB/s wr, 135 op/s 2024-09-11T00:28:09.581 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:28:09 smithi178 ceph-mon[35919]: pgmap v1032: 129 pgs: 129 active+clean; 677 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 690 KiB/s wr, 135 op/s 2024-09-11T00:28:09.782 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:28:09 smithi165 ceph-mon[35356]: pgmap v1032: 129 pgs: 129 active+clean; 677 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 690 KiB/s wr, 135 op/s 2024-09-11T00:28:10.250 INFO:tasks.workunit.client.0.smithi012.stdout: 10 134728 14.97 MB/sec execute 366 sec latency 451.009 ms 2024-09-11T00:28:10.945 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:28:11.250 INFO:tasks.workunit.client.0.smithi012.stdout: 10 135003 14.96 MB/sec execute 367 sec latency 978.533 ms 2024-09-11T00:28:11.804 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:28:11 smithi012 ceph-mon[30333]: pgmap v1033: 129 pgs: 129 active+clean; 683 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 2.9 MiB/s wr, 325 op/s 2024-09-11T00:28:11.992 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:11.989+0000 7f0c5ec00640 1 -- 172.21.15.12:0/3439804333 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0c580776d0 msgr2=0x7f0c58077ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:28:11.992 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:11.989+0000 7f0c5ec00640 1 --2- 172.21.15.12:0/3439804333 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0c580776d0 0x7f0c58077ab0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f0c4c009900 tx=0x7f0c4c02a410 comp rx=0 tx=0).stop 2024-09-11T00:28:11.993 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:11.990+0000 7f0c5ec00640 1 -- 172.21.15.12:0/3439804333 shutdown_connections 2024-09-11T00:28:11.993 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:11.990+0000 7f0c5ec00640 1 --2- 172.21.15.12:0/3439804333 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0c58113800 0x7f0c58115bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:11.993 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:11.990+0000 7f0c5ec00640 1 --2- 172.21.15.12:0/3439804333 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0c58077ff0 0x7f0c581132c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:11.993 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:11.990+0000 7f0c5ec00640 1 --2- 172.21.15.12:0/3439804333 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0c580776d0 0x7f0c58077ab0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:11.993 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:11.990+0000 7f0c5ec00640 1 -- 172.21.15.12:0/3439804333 >> 172.21.15.12:0/3439804333 conn(0x7f0c58100840 msgr2=0x7f0c58102c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:28:11.993 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:11.990+0000 7f0c5ec00640 1 -- 172.21.15.12:0/3439804333 shutdown_connections 2024-09-11T00:28:11.993 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:11.990+0000 7f0c5ec00640 1 -- 172.21.15.12:0/3439804333 wait complete. 2024-09-11T00:28:11.994 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:11.992+0000 7f0c5ec00640 1 Processor -- start 2024-09-11T00:28:11.995 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:11.993+0000 7f0c5ec00640 1 -- start start 2024-09-11T00:28:11.995 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:11.993+0000 7f0c5ec00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0c580776d0 0x7f0c5819f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:28:11.996 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:11.993+0000 7f0c5ec00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0c58077ff0 0x7f0c5819fbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:28:11.996 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:11.993+0000 7f0c5ec00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0c58113800 0x7f0c581a4f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:28:11.996 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:11.993+0000 7f0c5ec00640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f0c58119240 con 0x7f0c58113800 2024-09-11T00:28:11.997 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:11.993+0000 7f0c5ec00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f0c581190c0 con 0x7f0c580776d0 2024-09-11T00:28:11.997 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:11.993+0000 7f0c5ec00640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f0c581193c0 con 0x7f0c58077ff0 2024-09-11T00:28:11.997 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:11.994+0000 7f0c5ce00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0c58077ff0 0x7f0c5819fbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:28:11.997 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:11.994+0000 7f0c5ce00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0c58077ff0 0x7f0c5819fbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.12:52338/0 (socket says 172.21.15.12:52338) 2024-09-11T00:28:11.998 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:11.994+0000 7f0c5ce00640 1 -- 172.21.15.12:0/2174709077 learned_addr learned my addr 172.21.15.12:0/2174709077 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:28:11.998 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:11.994+0000 7f0c5d800640 1 --2- 172.21.15.12:0/2174709077 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0c580776d0 0x7f0c5819f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:28:11.998 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:11.994+0000 7f0c5e200640 1 --2- 172.21.15.12:0/2174709077 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0c58113800 0x7f0c581a4f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:28:11.998 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:11.994+0000 7f0c5d800640 1 -- 172.21.15.12:0/2174709077 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0c58077ff0 msgr2=0x7f0c5819fbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:28:11.998 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:11.994+0000 7f0c5d800640 1 --2- 172.21.15.12:0/2174709077 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0c58077ff0 0x7f0c5819fbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:11.998 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:11.994+0000 7f0c5d800640 1 -- 172.21.15.12:0/2174709077 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0c58113800 msgr2=0x7f0c581a4f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:28:11.998 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:11.994+0000 7f0c5d800640 1 --2- 172.21.15.12:0/2174709077 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0c58113800 0x7f0c581a4f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:11.998 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:11.994+0000 7f0c5d800640 1 -- 172.21.15.12:0/2174709077 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c581a5760 con 0x7f0c580776d0 2024-09-11T00:28:11.998 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:11.995+0000 7f0c5d800640 1 --2- 172.21.15.12:0/2174709077 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0c580776d0 0x7f0c5819f6b0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f0c4c02aa00 tx=0x7f0c4c003cd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:28:11.999 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:11.995+0000 7f0c56000640 1 -- 172.21.15.12:0/2174709077 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c4c03f070 con 0x7f0c580776d0 2024-09-11T00:28:11.999 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:11.995+0000 7f0c56000640 1 -- 172.21.15.12:0/2174709077 <== mon.2 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0c4c003e40 con 0x7f0c580776d0 2024-09-11T00:28:11.999 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:11.995+0000 7f0c56000640 1 -- 172.21.15.12:0/2174709077 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c4c035030 con 0x7f0c580776d0 2024-09-11T00:28:11.999 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:11.995+0000 7f0c5ec00640 1 -- 172.21.15.12:0/2174709077 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0c581a59f0 con 0x7f0c580776d0 2024-09-11T00:28:11.999 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:11.996+0000 7f0c5ec00640 1 -- 172.21.15.12:0/2174709077 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0c581ac110 con 0x7f0c580776d0 2024-09-11T00:28:12.004 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:12.000+0000 7f0c62cd6740 1 Processor -- start 2024-09-11T00:28:12.005 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:12.002+0000 7f0c62cd6740 1 -- start start 2024-09-11T00:28:12.005 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:12.002+0000 7f0c62cd6740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558ceb841310 0x558ceb8416f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:28:12.005 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:12.003+0000 7f0c62cd6740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558ceb841c30 0x558ceb84d680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:28:12.005 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:12.003+0000 7f0c62cd6740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558ceb84dbc0 0x558ceb84ffb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:28:12.005 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:12.003+0000 7f0c62cd6740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x558ceb7b6ca0 con 0x558ceb841c30 2024-09-11T00:28:12.005 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:12.003+0000 7f0c62cd6740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x558ceb7a60d0 con 0x558ceb841310 2024-09-11T00:28:12.005 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:12.003+0000 7f0c62cd6740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x558ceb7a9ee0 con 0x558ceb84dbc0 2024-09-11T00:28:12.007 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:12.003+0000 7f0c5ce00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558ceb84dbc0 0x558ceb84ffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:28:12.007 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:12.003+0000 7f0c5ce00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558ceb84dbc0 0x558ceb84ffb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.12:52348/0 (socket says 172.21.15.12:52348) 2024-09-11T00:28:12.008 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:12.003+0000 7f0c5ce00640 1 -- 172.21.15.12:0/2889508416 learned_addr learned my addr 172.21.15.12:0/2889508416 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:28:12.008 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:12.004+0000 7f0c5ce00640 1 -- 172.21.15.12:0/2889508416 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558ceb841310 msgr2=0x558ceb8416f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-11T00:28:12.008 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:12.004+0000 7f0c5ce00640 1 --2- 172.21.15.12:0/2889508416 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558ceb841310 0x558ceb8416f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:12.008 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:12.004+0000 7f0c5ce00640 1 -- 172.21.15.12:0/2889508416 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558ceb841c30 msgr2=0x558ceb84d680 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-11T00:28:12.008 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:12.004+0000 7f0c5ce00640 1 --2- 172.21.15.12:0/2889508416 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558ceb841c30 0x558ceb84d680 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:12.008 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:12.004+0000 7f0c5ce00640 1 -- 172.21.15.12:0/2889508416 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558ceb758400 con 0x558ceb84dbc0 2024-09-11T00:28:12.008 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:12.004+0000 7f0c5ce00640 1 --2- 172.21.15.12:0/2889508416 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558ceb84dbc0 0x558ceb84ffb0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f0c4400bd60 tx=0x7f0c440107a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dafcbc42a168257c server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:28:12.008 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:12.004+0000 7f0c4aa00640 1 -- 172.21.15.12:0/2889508416 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c44011640 con 0x558ceb84dbc0 2024-09-11T00:28:12.008 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:12.004+0000 7f0c4aa00640 1 -- 172.21.15.12:0/2889508416 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0c44011c90 con 0x558ceb84dbc0 2024-09-11T00:28:12.008 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:12.004+0000 7f0c4aa00640 1 -- 172.21.15.12:0/2889508416 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c4401bc90 con 0x558ceb84dbc0 2024-09-11T00:28:12.009 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:12.005+0000 7f0c56000640 1 -- 172.21.15.12:0/2174709077 <== mon.2 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7f0c4c0318b0 con 0x7f0c580776d0 2024-09-11T00:28:12.009 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:12.006+0000 7f0c62cd6740 1 -- 172.21.15.12:0/2889508416 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558ceb84dbc0 msgr2=0x558ceb84ffb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:28:12.009 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:12.006+0000 7f0c62cd6740 1 --2- 172.21.15.12:0/2889508416 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558ceb84dbc0 0x558ceb84ffb0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f0c4400bd60 tx=0x7f0c440107a0 comp rx=0 tx=0).stop 2024-09-11T00:28:12.009 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:12.006+0000 7f0c62cd6740 1 -- 172.21.15.12:0/2889508416 shutdown_connections 2024-09-11T00:28:12.009 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:12.006+0000 7f0c62cd6740 1 --2- 172.21.15.12:0/2889508416 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558ceb84dbc0 0x558ceb84ffb0 secure :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f0c4400bd60 tx=0x7f0c440107a0 comp rx=0 tx=0).stop 2024-09-11T00:28:12.010 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:12.006+0000 7f0c62cd6740 1 --2- 172.21.15.12:0/2889508416 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558ceb841c30 0x558ceb84d680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:12.010 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:12.006+0000 7f0c62cd6740 1 --2- 172.21.15.12:0/2889508416 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558ceb841310 0x558ceb8416f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:12.010 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:12.006+0000 7f0c62cd6740 1 -- 172.21.15.12:0/2889508416 >> 172.21.15.12:0/2889508416 conn(0x558ceb83f140 msgr2=0x558ceb8402e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:28:12.010 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:12.006+0000 7f0c62cd6740 1 -- 172.21.15.12:0/2889508416 shutdown_connections 2024-09-11T00:28:12.010 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:12.006+0000 7f0c62cd6740 1 -- 172.21.15.12:0/2889508416 wait complete. 2024-09-11T00:28:12.010 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:12.007+0000 7f0c62cd6740 1 -- start start 2024-09-11T00:28:12.010 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:12.008+0000 7f0c56000640 1 --2- 172.21.15.12:0/2174709077 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f0c400b5440 0x7f0c400b7900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:28:12.011 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:12.008+0000 7f0c56000640 1 -- 172.21.15.12:0/2174709077 <== mon.2 v2:172.21.15.165:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9208+0+0 (secure 0 0 0) 0x7f0c4c106a40 con 0x7f0c580776d0 2024-09-11T00:28:12.011 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:12.008+0000 7f0c62cd6740 1 Processor -- start 2024-09-11T00:28:12.011 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:12.009+0000 7f0c5e200640 1 --2- 172.21.15.12:0/2174709077 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f0c400b5440 0x7f0c400b7900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:28:12.012 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:12.010+0000 7f0c62cd6740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558ceb841310 0x558ceb84aba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:28:12.012 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:12.010+0000 7f0c62cd6740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558ceb841c30 0x558ceb84b0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:28:12.013 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:12.010+0000 7f0c62cd6740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558ceb84b620 0x558ceb859bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:28:12.013 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:12.010+0000 7f0c62cd6740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x558ceb6077a0 con 0x558ceb841310 2024-09-11T00:28:12.013 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:12.010+0000 7f0c62cd6740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x558ceb74ba10 con 0x558ceb841c30 2024-09-11T00:28:12.013 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:12.010+0000 7f0c62cd6740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x558ceb850840 con 0x558ceb84b620 2024-09-11T00:28:12.013 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:12.010+0000 7f0c62cd6740 10 client.0 _ll_register_callbacks cb 0x558ceb75bbe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:28:12.014 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:12.010+0000 7f0c5d800640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558ceb841310 0x558ceb84aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:28:12.014 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:12.010+0000 7f0c5d800640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558ceb841310 0x558ceb84aba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:33696/0 (socket says 172.21.15.12:33696) 2024-09-11T00:28:12.014 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:12.011+0000 7f0c5d800640 1 -- 172.21.15.12:0/1164284820 learned_addr learned my addr 172.21.15.12:0/1164284820 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:28:12.014 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:12.011+0000 7f0c5ce00640 1 --2- 172.21.15.12:0/1164284820 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558ceb841c30 0x558ceb84b0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:28:12.015 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:12.011+0000 7f0c5ce00640 1 -- 172.21.15.12:0/1164284820 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558ceb84b620 msgr2=0x558ceb859bc0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-11T00:28:12.015 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:12.011+0000 7f0c5ce00640 1 --2- 172.21.15.12:0/1164284820 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558ceb84b620 0x558ceb859bc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:12.015 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:12.011+0000 7f0c5ce00640 1 -- 172.21.15.12:0/1164284820 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558ceb841310 msgr2=0x558ceb84aba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:28:12.015 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:12.011+0000 7f0c5ce00640 1 --2- 172.21.15.12:0/1164284820 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558ceb841310 0x558ceb84aba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:12.015 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:12.011+0000 7f0c5ce00640 1 -- 172.21.15.12:0/1164284820 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558ceb513b90 con 0x558ceb841c30 2024-09-11T00:28:12.015 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:12.012+0000 7f0c5ce00640 1 --2- 172.21.15.12:0/1164284820 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558ceb841c30 0x558ceb84b0e0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f0c44014000 tx=0x7f0c4401b790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9a72ff2dfdd4b744 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:28:12.015 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:12.012+0000 7f0c5e200640 1 --2- 172.21.15.12:0/2174709077 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f0c400b5440 0x7f0c400b7900 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f0c5806bf90 tx=0x7f0c50009870 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:28:12.015 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:12.012+0000 7f0c3f400640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-09-11T00:28:12.015 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:12.012+0000 7f0c3f400640 1 -- 172.21.15.12:0/1164284820 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c4402a060 con 0x558ceb841c30 2024-09-11T00:28:12.016 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:12.012+0000 7f0c3f400640 1 -- 172.21.15.12:0/1164284820 <== mon.2 v2:172.21.15.165:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 9208+0+0 (secure 0 0 0) 0x7f0c4401be00 con 0x558ceb841c30 2024-09-11T00:28:12.017 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:12.013+0000 7f0c62cd6740 1 -- 172.21.15.12:0/1164284820 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0c44011c90 con 0x558ceb841c30 2024-09-11T00:28:12.017 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:12.013+0000 7f0c3f400640 1 -- 172.21.15.12:0/1164284820 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=89) ==== 24+0+0 (secure 0 0 0) 0x7f0c44031050 con 0x558ceb841c30 2024-09-11T00:28:12.017 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:12.013+0000 7f0c62cd6740 10 client.27797 fetch_fsmap learned FSMap version 89 2024-09-11T00:28:12.017 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:12.013+0000 7f0c62cd6740 1 -- 172.21.15.12:0/1164284820 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=89,monmap=4+}) -- 0x558ceb85c840 con 0x558ceb841c30 2024-09-11T00:28:12.017 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:12.014+0000 7f0c3f400640 1 -- 172.21.15.12:0/1164284820 <== mon.2 v2:172.21.15.165:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0c44011c90 con 0x558ceb841c30 2024-09-11T00:28:12.017 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:12.014+0000 7f0c3f400640 1 -- 172.21.15.12:0/1164284820 <== mon.2 v2:172.21.15.165:3300/0 5 ==== fsmap(e 89) ==== 8426+0+0 (secure 0 0 0) 0x7f0c44011750 con 0x558ceb841c30 2024-09-11T00:28:12.017 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:12.014+0000 7f0c62cd6740 10 client.27797 fetch_fsmap finished waiting for FSMap version 89 2024-09-11T00:28:12.017 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:12.014+0000 7f0c62cd6740 10 client.27797 resolve_mds: validated gid 24674 aka daemon mds.b 2024-09-11T00:28:12.018 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:12.014+0000 7f0c62cd6740 20 client.27797 populate_metadata read hostname 'smithi012' 2024-09-11T00:28:12.018 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:12.014+0000 7f0c62cd6740 1 --2- 172.21.15.12:0/1164284820 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x558ceb85db90 0x558ceb85ff80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:28:12.018 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:12.014+0000 7f0c62cd6740 4 client.27797 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T00:28:12.018 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:12.014+0000 7f0c62cd6740 1 -- 172.21.15.12:0/1164284820 --> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558ceb7b8bf0 con 0x558ceb85db90 2024-09-11T00:28:12.018 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:12.015+0000 7f0c5d800640 1 --2- 172.21.15.12:0/1164284820 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x558ceb85db90 0x558ceb85ff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:28:12.019 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:12.016+0000 7f0c5d800640 1 --2- 172.21.15.12:0/1164284820 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x558ceb85db90 0x558ceb85ff80 crc :-1 s=READY pgs=293 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e87cf8d72779b3c2 server_cookie=74741c6a141c3456 in_seq=0 out_seq=0 2024-09-11T00:28:12.019 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:12.016+0000 7f0c3f400640 10 client.27797 ms_handle_connect on v2:172.21.15.165:6838/501312984 2024-09-11T00:28:12.020 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:12.018+0000 7f0c3f400640 1 -- 172.21.15.12:0/1164284820 <== mds.0 v2:172.21.15.165:6838/501312984 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x558ceb7b8bf0 con 0x558ceb85db90 2024-09-11T00:28:12.021 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:12.018+0000 7f0c3f400640 10 client.27797 handle_command_reply: tid=0 2024-09-11T00:28:12.021 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:12.018+0000 7f0c62cd6740 1 client.27797 shutdown 2024-09-11T00:28:12.021 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:12.018+0000 7f0c62cd6740 1 -- 172.21.15.12:0/1164284820 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558ceb841c30 msgr2=0x558ceb84b0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:28:12.021 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:12.018+0000 7f0c62cd6740 1 --2- 172.21.15.12:0/1164284820 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558ceb841c30 0x558ceb84b0e0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f0c44014000 tx=0x7f0c4401b790 comp rx=0 tx=0).stop 2024-09-11T00:28:12.021 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:12.018+0000 7f0c62cd6740 1 -- 172.21.15.12:0/1164284820 shutdown_connections 2024-09-11T00:28:12.022 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:12.018+0000 7f0c62cd6740 1 --2- 172.21.15.12:0/1164284820 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x558ceb85db90 0x558ceb85ff80 crc :-1 s=READY pgs=293 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:12.022 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:12.018+0000 7f0c62cd6740 1 --2- 172.21.15.12:0/1164284820 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558ceb84b620 0x558ceb859bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:12.022 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:12.018+0000 7f0c62cd6740 1 --2- 172.21.15.12:0/1164284820 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558ceb841c30 0x558ceb84b0e0 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:12.022 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:12.018+0000 7f0c3f400640 0 client.27797 ms_handle_reset on v2:172.21.15.165:6838/501312984 2024-09-11T00:28:12.022 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:12.018+0000 7f0c62cd6740 1 --2- 172.21.15.12:0/1164284820 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558ceb841310 0x558ceb84aba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:12.022 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:12.018+0000 7f0c62cd6740 1 -- 172.21.15.12:0/1164284820 >> 172.21.15.12:0/1164284820 conn(0x558ceb83f140 msgr2=0x558ceb844ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:28:12.022 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:12.019+0000 7f0c62cd6740 1 -- 172.21.15.12:0/1164284820 shutdown_connections 2024-09-11T00:28:12.022 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:12.019+0000 7f0c62cd6740 1 -- 172.21.15.12:0/1164284820 wait complete. 2024-09-11T00:28:12.022 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:12.019+0000 7f0c62cd6740 20 client.27797 trim_cache size 0 max 16384 2024-09-11T00:28:12.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:28:11 smithi165 ceph-mon[35356]: pgmap v1033: 129 pgs: 129 active+clean; 683 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 2.9 MiB/s wr, 325 op/s 2024-09-11T00:28:12.049 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:12.046+0000 7f0c62cd6740 1 Processor -- start 2024-09-11T00:28:12.049 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:12.046+0000 7f0c62cd6740 1 -- start start 2024-09-11T00:28:12.049 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:12.047+0000 7f0c62cd6740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558ceb841310 0x558ceb83eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:28:12.049 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:12.047+0000 7f0c62cd6740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558ceb841c30 0x558ceb859a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:28:12.050 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:12.047+0000 7f0c62cd6740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558ceb84b620 0x558ceb847dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:28:12.050 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:12.047+0000 7f0c62cd6740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f0c581b5f90 con 0x558ceb841310 2024-09-11T00:28:12.050 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:12.047+0000 7f0c62cd6740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f0c581b7c80 con 0x558ceb84b620 2024-09-11T00:28:12.050 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:12.047+0000 7f0c62cd6740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f0c581b66f0 con 0x558ceb841c30 2024-09-11T00:28:12.050 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:12.047+0000 7f0c5ce00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558ceb841c30 0x558ceb859a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:28:12.051 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:12.047+0000 7f0c5d800640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558ceb841310 0x558ceb83eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:28:12.051 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:12.047+0000 7f0c5ce00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558ceb841c30 0x558ceb859a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.12:52364/0 (socket says 172.21.15.12:52364) 2024-09-11T00:28:12.051 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:12.047+0000 7f0c5ce00640 1 -- 172.21.15.12:0/4218542128 learned_addr learned my addr 172.21.15.12:0/4218542128 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:28:12.051 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:12.048+0000 7f0c5e200640 1 --2- 172.21.15.12:0/4218542128 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558ceb84b620 0x558ceb847dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:28:12.052 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:12.048+0000 7f0c5ce00640 1 -- 172.21.15.12:0/4218542128 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558ceb84b620 msgr2=0x558ceb847dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:28:12.052 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:12.048+0000 7f0c5ce00640 1 --2- 172.21.15.12:0/4218542128 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558ceb84b620 0x558ceb847dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:12.052 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:12.048+0000 7f0c5ce00640 1 -- 172.21.15.12:0/4218542128 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558ceb841310 msgr2=0x558ceb83eaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:28:12.052 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:12.048+0000 7f0c5ce00640 1 --2- 172.21.15.12:0/4218542128 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558ceb841310 0x558ceb83eaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:12.052 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:12.048+0000 7f0c5ce00640 1 -- 172.21.15.12:0/4218542128 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558ceb841a60 con 0x558ceb841c30 2024-09-11T00:28:12.052 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:12.048+0000 7f0c5e200640 1 --2- 172.21.15.12:0/4218542128 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558ceb84b620 0x558ceb847dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T00:28:12.052 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:12.048+0000 7f0c5ce00640 1 --2- 172.21.15.12:0/4218542128 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558ceb841c30 0x558ceb859a50 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f0c44005bc0 tx=0x7f0c4400b5b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1a3311dd34e7df0e server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:28:12.052 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:12.048+0000 7f0c4a000640 1 -- 172.21.15.12:0/4218542128 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c4402a050 con 0x558ceb841c30 2024-09-11T00:28:12.052 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:12.049+0000 7f0c4a000640 1 -- 172.21.15.12:0/4218542128 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0c44004940 con 0x558ceb841c30 2024-09-11T00:28:12.052 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:12.049+0000 7f0c4a000640 1 -- 172.21.15.12:0/4218542128 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c44025990 con 0x558ceb841c30 2024-09-11T00:28:12.053 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:12.049+0000 7f0c62cd6740 1 -- 172.21.15.12:0/4218542128 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558ceb841c30 msgr2=0x558ceb859a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:28:12.053 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:12.049+0000 7f0c62cd6740 1 --2- 172.21.15.12:0/4218542128 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558ceb841c30 0x558ceb859a50 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f0c44005bc0 tx=0x7f0c4400b5b0 comp rx=0 tx=0).stop 2024-09-11T00:28:12.053 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:12.050+0000 7f0c62cd6740 1 -- 172.21.15.12:0/4218542128 shutdown_connections 2024-09-11T00:28:12.053 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:12.050+0000 7f0c62cd6740 1 --2- 172.21.15.12:0/4218542128 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558ceb84b620 0x558ceb847dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:12.053 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:12.050+0000 7f0c62cd6740 1 --2- 172.21.15.12:0/4218542128 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558ceb841c30 0x558ceb859a50 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:12.054 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:12.050+0000 7f0c62cd6740 1 --2- 172.21.15.12:0/4218542128 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558ceb841310 0x558ceb83eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:12.054 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:12.050+0000 7f0c62cd6740 1 -- 172.21.15.12:0/4218542128 >> 172.21.15.12:0/4218542128 conn(0x558ceb83f140 msgr2=0x558ceb859340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:28:12.054 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:12.050+0000 7f0c62cd6740 1 -- 172.21.15.12:0/4218542128 shutdown_connections 2024-09-11T00:28:12.054 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:12.050+0000 7f0c62cd6740 1 -- 172.21.15.12:0/4218542128 wait complete. 2024-09-11T00:28:12.054 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:12.050+0000 7f0c62cd6740 1 -- start start 2024-09-11T00:28:12.054 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:12.051+0000 7f0c62cd6740 1 Processor -- start 2024-09-11T00:28:12.055 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:12.051+0000 7f0c62cd6740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558ceb841310 0x558ceb8639d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:28:12.055 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:12.051+0000 7f0c62cd6740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558ceb841c30 0x558ceb861f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:28:12.055 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:12.051+0000 7f0c62cd6740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558ceb84b620 0x558ceb8626d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:28:12.055 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:12.051+0000 7f0c62cd6740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f0c581b75b0 con 0x558ceb841c30 2024-09-11T00:28:12.055 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:12.051+0000 7f0c62cd6740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f0c581b6e80 con 0x558ceb84b620 2024-09-11T00:28:12.055 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:12.051+0000 7f0c62cd6740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f0c581b3f70 con 0x558ceb841310 2024-09-11T00:28:12.055 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:12.051+0000 7f0c62cd6740 10 client.0 _ll_register_callbacks cb 0x7f0c4400bbb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:28:12.055 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:12.052+0000 7f0c5ce00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558ceb841c30 0x558ceb861f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:28:12.055 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:12.052+0000 7f0c5ce00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558ceb841c30 0x558ceb861f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:33724/0 (socket says 172.21.15.12:33724) 2024-09-11T00:28:12.056 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:12.052+0000 7f0c5ce00640 1 -- 172.21.15.12:0/2190529264 learned_addr learned my addr 172.21.15.12:0/2190529264 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:28:12.056 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:12.052+0000 7f0c5d800640 1 --2- 172.21.15.12:0/2190529264 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558ceb841310 0x558ceb8639d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:28:12.056 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:12.052+0000 7f0c5e200640 1 --2- 172.21.15.12:0/2190529264 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558ceb84b620 0x558ceb8626d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:28:12.056 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:12.052+0000 7f0c5ce00640 1 -- 172.21.15.12:0/2190529264 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558ceb841310 msgr2=0x558ceb8639d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:28:12.056 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:12.052+0000 7f0c5ce00640 1 --2- 172.21.15.12:0/2190529264 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558ceb841310 0x558ceb8639d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:12.056 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:12.052+0000 7f0c5ce00640 1 -- 172.21.15.12:0/2190529264 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558ceb84b620 msgr2=0x558ceb8626d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:28:12.057 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:12.052+0000 7f0c5ce00640 1 --2- 172.21.15.12:0/2190529264 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558ceb84b620 0x558ceb8626d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:12.057 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:12.052+0000 7f0c5ce00640 1 -- 172.21.15.12:0/2190529264 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558ceb8ca710 con 0x558ceb841c30 2024-09-11T00:28:12.057 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:12.053+0000 7f0c5ce00640 1 --2- 172.21.15.12:0/2190529264 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558ceb841c30 0x558ceb861f40 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f0c44000c00 tx=0x7f0c44011670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e0d948442ed2cb1 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:28:12.057 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:12.053+0000 7f0c3f400640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-09-11T00:28:12.057 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:12.054+0000 7f0c3f400640 1 -- 172.21.15.12:0/2190529264 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c44044020 con 0x558ceb841c30 2024-09-11T00:28:12.057 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:12.054+0000 7f0c3f400640 1 -- 172.21.15.12:0/2190529264 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 9208+0+0 (secure 0 0 0) 0x7f0c44025710 con 0x558ceb841c30 2024-09-11T00:28:12.057 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:12.054+0000 7f0c62cd6740 1 -- 172.21.15.12:0/2190529264 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0c44004940 con 0x558ceb841c30 2024-09-11T00:28:12.058 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:12.054+0000 7f0c3f400640 1 -- 172.21.15.12:0/2190529264 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=89) ==== 24+0+0 (secure 0 0 0) 0x7f0c4404f050 con 0x558ceb841c30 2024-09-11T00:28:12.058 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:12.054+0000 7f0c62cd6740 10 client.17793 fetch_fsmap learned FSMap version 89 2024-09-11T00:28:12.058 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:12.054+0000 7f0c62cd6740 1 -- 172.21.15.12:0/2190529264 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=89,monmap=4+}) -- 0x558ceb8cb930 con 0x558ceb841c30 2024-09-11T00:28:12.058 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:12.055+0000 7f0c3f400640 1 -- 172.21.15.12:0/2190529264 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0c44004940 con 0x558ceb841c30 2024-09-11T00:28:12.058 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:12.055+0000 7f0c3f400640 1 -- 172.21.15.12:0/2190529264 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 89) ==== 8426+0+0 (secure 0 0 0) 0x7f0c4403e460 con 0x558ceb841c30 2024-09-11T00:28:12.058 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:12.056+0000 7f0c62cd6740 10 client.17793 fetch_fsmap finished waiting for FSMap version 89 2024-09-11T00:28:12.059 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:12.056+0000 7f0c62cd6740 10 client.17793 resolve_mds: validated gid 24674 aka daemon mds.b 2024-09-11T00:28:12.059 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:12.056+0000 7f0c62cd6740 20 client.17793 populate_metadata read hostname 'smithi012' 2024-09-11T00:28:12.059 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:12.056+0000 7f0c62cd6740 1 --2- 172.21.15.12:0/2190529264 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x558ceb8cc090 0x558ceb8ce4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:28:12.059 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:12.056+0000 7f0c62cd6740 4 client.17793 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 150.0, "await_for": 45.0}] 2024-09-11T00:28:12.059 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:12.056+0000 7f0c62cd6740 1 -- 172.21.15.12:0/2190529264 --> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 150.0, "await_for": 45.0}) -- 0x558ceb7b46a0 con 0x558ceb8cc090 2024-09-11T00:28:12.059 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:12.056+0000 7f0c5d800640 1 --2- 172.21.15.12:0/2190529264 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x558ceb8cc090 0x558ceb8ce4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:28:12.060 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:12.057+0000 7f0c5d800640 1 --2- 172.21.15.12:0/2190529264 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x558ceb8cc090 0x558ceb8ce4b0 crc :-1 s=READY pgs=294 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=73a0faecccf18e5c server_cookie=1312ae8c12b86daf in_seq=0 out_seq=0 2024-09-11T00:28:12.060 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:12.057+0000 7f0c3f400640 10 client.17793 ms_handle_connect on v2:172.21.15.165:6838/501312984 2024-09-11T00:28:12.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:28:11 smithi178 ceph-mon[35919]: pgmap v1033: 129 pgs: 129 active+clean; 683 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 2.9 MiB/s wr, 325 op/s 2024-09-11T00:28:12.250 INFO:tasks.workunit.client.0.smithi012.stdout: 10 135542 15.00 MB/sec execute 368 sec latency 199.129 ms 2024-09-11T00:28:13.250 INFO:tasks.workunit.client.0.smithi012.stdout: 10 135552 14.96 MB/sec execute 369 sec latency 1187.239 ms 2024-09-11T00:28:14.033 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:28:13 smithi165 ceph-mon[35356]: pgmap v1034: 129 pgs: 129 active+clean; 690 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 MiB/s rd, 4.4 MiB/s wr, 443 op/s 2024-09-11T00:28:14.063 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:28:13 smithi012 ceph-mon[30333]: pgmap v1034: 129 pgs: 129 active+clean; 690 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 MiB/s rd, 4.4 MiB/s wr, 443 op/s 2024-09-11T00:28:14.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:28:13 smithi178 ceph-mon[35919]: pgmap v1034: 129 pgs: 129 active+clean; 690 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 MiB/s rd, 4.4 MiB/s wr, 443 op/s 2024-09-11T00:28:14.250 INFO:tasks.workunit.client.0.smithi012.stdout: 10 135552 14.92 MB/sec execute 370 sec latency 2187.284 ms 2024-09-11T00:28:15.250 INFO:tasks.workunit.client.0.smithi012.stdout: 10 135552 14.88 MB/sec execute 371 sec latency 3187.375 ms 2024-09-11T00:28:16.033 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:28:15 smithi165 ceph-mon[35356]: pgmap v1035: 129 pgs: 129 active+clean; 691 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 MiB/s rd, 4.8 MiB/s wr, 452 op/s 2024-09-11T00:28:16.063 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:28:15 smithi012 ceph-mon[30333]: pgmap v1035: 129 pgs: 129 active+clean; 691 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 MiB/s rd, 4.8 MiB/s wr, 452 op/s 2024-09-11T00:28:16.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:28:15 smithi178 ceph-mon[35919]: pgmap v1035: 129 pgs: 129 active+clean; 691 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 MiB/s rd, 4.8 MiB/s wr, 452 op/s 2024-09-11T00:28:16.250 INFO:tasks.workunit.client.0.smithi012.stdout: 10 135552 14.84 MB/sec execute 372 sec latency 4187.505 ms 2024-09-11T00:28:16.773 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:16.770+0000 7f0c3f400640 1 -- 172.21.15.12:0/2190529264 <== mds.0 v2:172.21.15.165:6838/501312984 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x558ceb7b46a0 con 0x558ceb8cc090 2024-09-11T00:28:16.773 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:16.770+0000 7f0c3f400640 10 client.17793 handle_command_reply: tid=0 2024-09-11T00:28:16.773 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:16.770+0000 7f0c62cd6740 1 client.17793 shutdown 2024-09-11T00:28:16.773 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:16.770+0000 7f0c62cd6740 1 -- 172.21.15.12:0/2190529264 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558ceb841c30 msgr2=0x558ceb861f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:28:16.773 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:16.770+0000 7f0c62cd6740 1 --2- 172.21.15.12:0/2190529264 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558ceb841c30 0x558ceb861f40 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f0c44000c00 tx=0x7f0c44011670 comp rx=0 tx=0).stop 2024-09-11T00:28:16.774 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:16.770+0000 7f0c62cd6740 1 -- 172.21.15.12:0/2190529264 shutdown_connections 2024-09-11T00:28:16.774 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:16.770+0000 7f0c62cd6740 1 --2- 172.21.15.12:0/2190529264 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x558ceb8cc090 0x558ceb8ce4b0 crc :-1 s=READY pgs=294 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:16.774 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:16.770+0000 7f0c62cd6740 1 --2- 172.21.15.12:0/2190529264 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558ceb84b620 0x558ceb8626d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:16.774 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:16.770+0000 7f0c62cd6740 1 --2- 172.21.15.12:0/2190529264 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558ceb841c30 0x558ceb861f40 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:16.774 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:16.770+0000 7f0c62cd6740 1 --2- 172.21.15.12:0/2190529264 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558ceb841310 0x558ceb8639d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:16.774 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:16.770+0000 7f0c62cd6740 1 -- 172.21.15.12:0/2190529264 >> 172.21.15.12:0/2190529264 conn(0x558ceb83f140 msgr2=0x558ceb83dae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:28:16.774 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:16.770+0000 7f0c3f400640 0 client.17793 ms_handle_reset on v2:172.21.15.165:6838/501312984 2024-09-11T00:28:16.774 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:16.770+0000 7f0c62cd6740 1 -- 172.21.15.12:0/2190529264 shutdown_connections 2024-09-11T00:28:16.774 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:16.771+0000 7f0c62cd6740 1 -- 172.21.15.12:0/2190529264 wait complete. 2024-09-11T00:28:16.774 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:16.771+0000 7f0c62cd6740 20 client.17793 trim_cache size 0 max 16384 2024-09-11T00:28:16.775 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-09-11T00:28:16.775 INFO:teuthology.orchestra.run.smithi012.stdout: "epoch": 89, 2024-09-11T00:28:16.775 INFO:teuthology.orchestra.run.smithi012.stdout: "leader": 24674, 2024-09-11T00:28:16.775 INFO:teuthology.orchestra.run.smithi012.stdout: "set_version": 66, 2024-09-11T00:28:16.775 INFO:teuthology.orchestra.run.smithi012.stdout: "sets": { 2024-09-11T00:28:16.775 INFO:teuthology.orchestra.run.smithi012.stdout: "248955d7": { 2024-09-11T00:28:16.775 INFO:teuthology.orchestra.run.smithi012.stdout: "version": 66, 2024-09-11T00:28:16.775 INFO:teuthology.orchestra.run.smithi012.stdout: "age_ref": 0.0, 2024-09-11T00:28:16.775 INFO:teuthology.orchestra.run.smithi012.stdout: "state": { 2024-09-11T00:28:16.775 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "QUIESCED", 2024-09-11T00:28:16.775 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 0.0 2024-09-11T00:28:16.776 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:28:16.776 INFO:teuthology.orchestra.run.smithi012.stdout: "timeout": 90.0, 2024-09-11T00:28:16.776 INFO:teuthology.orchestra.run.smithi012.stdout: "expiration": 150.0, 2024-09-11T00:28:16.776 INFO:teuthology.orchestra.run.smithi012.stdout: "members": { 2024-09-11T00:28:16.776 INFO:teuthology.orchestra.run.smithi012.stdout: "file:/": { 2024-09-11T00:28:16.776 INFO:teuthology.orchestra.run.smithi012.stdout: "excluded": false, 2024-09-11T00:28:16.776 INFO:teuthology.orchestra.run.smithi012.stdout: "state": { 2024-09-11T00:28:16.776 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "QUIESCED", 2024-09-11T00:28:16.776 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 0.0 2024-09-11T00:28:16.776 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:28:16.776 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:28:16.776 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:28:16.776 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:28:16.777 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:28:16.777 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-09-11T00:28:16.777 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:16.772+0000 7f0c5ec00640 1 -- 172.21.15.12:0/2174709077 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f0c400b5440 msgr2=0x7f0c400b7900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:28:16.777 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:16.772+0000 7f0c5ec00640 1 --2- 172.21.15.12:0/2174709077 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f0c400b5440 0x7f0c400b7900 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f0c5806bf90 tx=0x7f0c50009870 comp rx=0 tx=0).stop 2024-09-11T00:28:16.777 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:16.772+0000 7f0c5ec00640 1 -- 172.21.15.12:0/2174709077 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0c580776d0 msgr2=0x7f0c5819f6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:28:16.777 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:16.772+0000 7f0c5ec00640 1 --2- 172.21.15.12:0/2174709077 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0c580776d0 0x7f0c5819f6b0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f0c4c02aa00 tx=0x7f0c4c003cd0 comp rx=0 tx=0).stop 2024-09-11T00:28:16.778 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:16.772+0000 7f0c5ec00640 1 -- 172.21.15.12:0/2174709077 shutdown_connections 2024-09-11T00:28:16.778 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:16.772+0000 7f0c5ec00640 1 --2- 172.21.15.12:0/2174709077 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f0c400b5440 0x7f0c400b7900 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:16.778 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:16.772+0000 7f0c5ec00640 1 --2- 172.21.15.12:0/2174709077 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0c58113800 0x7f0c581a4f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:16.778 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:16.772+0000 7f0c5ec00640 1 --2- 172.21.15.12:0/2174709077 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0c58077ff0 0x7f0c5819fbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:16.778 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:16.772+0000 7f0c5ec00640 1 --2- 172.21.15.12:0/2174709077 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0c580776d0 0x7f0c5819f6b0 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:16.778 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:16.772+0000 7f0c5ec00640 1 -- 172.21.15.12:0/2174709077 >> 172.21.15.12:0/2174709077 conn(0x7f0c58100840 msgr2=0x7f0c58102c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:28:16.778 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:16.772+0000 7f0c5ec00640 1 -- 172.21.15.12:0/2174709077 shutdown_connections 2024-09-11T00:28:16.778 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:16.772+0000 7f0c5ec00640 1 -- 172.21.15.12:0/2174709077 wait complete. 2024-09-11T00:28:17.250 INFO:tasks.workunit.client.0.smithi012.stdout: 10 135552 14.80 MB/sec execute 373 sec latency 5187.599 ms 2024-09-11T00:28:17.594 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '248955d7', quiesce took 12.4 seconds. Will release after: 17.6 2024-09-11T00:28:17.813 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:28:17 smithi012 ceph-mon[30333]: pgmap v1036: 129 pgs: 129 active+clean; 686 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 7.3 MiB/s wr, 540 op/s 2024-09-11T00:28:18.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:28:17 smithi165 ceph-mon[35356]: pgmap v1036: 129 pgs: 129 active+clean; 686 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 7.3 MiB/s wr, 540 op/s 2024-09-11T00:28:18.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:28:17 smithi178 ceph-mon[35919]: pgmap v1036: 129 pgs: 129 active+clean; 686 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 7.3 MiB/s wr, 540 op/s 2024-09-11T00:28:18.154 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:18.151+0000 7ff9e7000640 1 -- 172.21.15.12:0/3310415177 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff9e01049b0 msgr2=0x7ff9e0104db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:28:18.154 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:18.151+0000 7ff9e7000640 1 --2- 172.21.15.12:0/3310415177 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff9e01049b0 0x7ff9e0104db0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7ff9d4009930 tx=0x7ff9d402a420 comp rx=0 tx=0).stop 2024-09-11T00:28:18.155 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:18.152+0000 7ff9e7000640 1 -- 172.21.15.12:0/3310415177 shutdown_connections 2024-09-11T00:28:18.155 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:18.152+0000 7ff9e7000640 1 --2- 172.21.15.12:0/3310415177 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff9e01064c0 0x7ff9e010ce40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:18.155 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:18.152+0000 7ff9e7000640 1 --2- 172.21.15.12:0/3310415177 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff9e0105b00 0x7ff9e0105f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:18.155 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:18.152+0000 7ff9e7000640 1 --2- 172.21.15.12:0/3310415177 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff9e01049b0 0x7ff9e0104db0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:18.155 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:18.152+0000 7ff9e7000640 1 -- 172.21.15.12:0/3310415177 >> 172.21.15.12:0/3310415177 conn(0x7ff9e0100250 msgr2=0x7ff9e0102690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:28:18.155 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:18.152+0000 7ff9e7000640 1 -- 172.21.15.12:0/3310415177 shutdown_connections 2024-09-11T00:28:18.155 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:18.152+0000 7ff9e7000640 1 -- 172.21.15.12:0/3310415177 wait complete. 2024-09-11T00:28:18.157 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:18.153+0000 7ff9e7000640 1 Processor -- start 2024-09-11T00:28:18.157 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:18.153+0000 7ff9e7000640 1 -- start start 2024-09-11T00:28:18.157 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:18.153+0000 7ff9e7000640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff9e01049b0 0x7ff9e019ada0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:28:18.157 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:18.153+0000 7ff9e7000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff9e0105b00 0x7ff9e019b3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:28:18.157 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:18.153+0000 7ff9e7000640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff9e01064c0 0x7ff9e019c020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:28:18.157 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:18.153+0000 7ff9e7000640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7ff9e0110670 con 0x7ff9e01064c0 2024-09-11T00:28:18.157 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:18.153+0000 7ff9e7000640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7ff9e01104f0 con 0x7ff9e0105b00 2024-09-11T00:28:18.158 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:18.153+0000 7ff9e7000640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7ff9e01107f0 con 0x7ff9e01049b0 2024-09-11T00:28:18.158 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:18.153+0000 7ff9e6600640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff9e01064c0 0x7ff9e019c020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:28:18.158 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:18.153+0000 7ff9e6600640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff9e01064c0 0x7ff9e019c020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:33750/0 (socket says 172.21.15.12:33750) 2024-09-11T00:28:18.158 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:18.153+0000 7ff9e6600640 1 -- 172.21.15.12:0/2420591380 learned_addr learned my addr 172.21.15.12:0/2420591380 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:28:18.158 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:18.154+0000 7ff9e5200640 1 --2- 172.21.15.12:0/2420591380 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff9e0105b00 0x7ff9e019b3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:28:18.158 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:18.154+0000 7ff9e5c00640 1 --2- 172.21.15.12:0/2420591380 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff9e01049b0 0x7ff9e019ada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:28:18.158 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:18.154+0000 7ff9e6600640 1 -- 172.21.15.12:0/2420591380 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff9e01049b0 msgr2=0x7ff9e019ada0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:28:18.158 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:18.154+0000 7ff9e6600640 1 --2- 172.21.15.12:0/2420591380 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff9e01049b0 0x7ff9e019ada0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:18.158 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:18.154+0000 7ff9e6600640 1 -- 172.21.15.12:0/2420591380 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff9e0105b00 msgr2=0x7ff9e019b3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:28:18.158 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:18.154+0000 7ff9e6600640 1 --2- 172.21.15.12:0/2420591380 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff9e0105b00 0x7ff9e019b3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:18.158 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:18.154+0000 7ff9e6600640 1 -- 172.21.15.12:0/2420591380 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9e019c8e0 con 0x7ff9e01064c0 2024-09-11T00:28:18.158 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:18.154+0000 7ff9e6600640 1 --2- 172.21.15.12:0/2420591380 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff9e01064c0 0x7ff9e019c020 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7ff9d800eb10 tx=0x7ff9d800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:28:18.158 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:18.155+0000 7ff9de000640 1 -- 172.21.15.12:0/2420591380 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9d8004b20 con 0x7ff9e01064c0 2024-09-11T00:28:18.159 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:18.155+0000 7ff9e7000640 1 -- 172.21.15.12:0/2420591380 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff9e019cbd0 con 0x7ff9e01064c0 2024-09-11T00:28:18.159 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:18.155+0000 7ff9de000640 1 -- 172.21.15.12:0/2420591380 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff9d8004cc0 con 0x7ff9e01064c0 2024-09-11T00:28:18.159 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:18.155+0000 7ff9de000640 1 -- 172.21.15.12:0/2420591380 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9d8010690 con 0x7ff9e01064c0 2024-09-11T00:28:18.159 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:18.155+0000 7ff9e7000640 1 -- 172.21.15.12:0/2420591380 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9e01a32a0 con 0x7ff9e01064c0 2024-09-11T00:28:18.160 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:18.158+0000 7ff9eb67a740 1 Processor -- start 2024-09-11T00:28:18.160 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:18.158+0000 7ff9de000640 1 -- 172.21.15.12:0/2420591380 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7ff9d8005020 con 0x7ff9e01064c0 2024-09-11T00:28:18.161 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:18.158+0000 7ff9eb67a740 1 -- start start 2024-09-11T00:28:18.163 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:18.159+0000 7ff9eb67a740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564656aee970 0x564656af0d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:28:18.163 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:18.159+0000 7ff9eb67a740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564656af12a0 0x564656af3690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:28:18.163 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:18.159+0000 7ff9eb67a740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564656af3ce0 0x564656af61a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:28:18.163 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:18.159+0000 7ff9eb67a740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x564656a616b0 con 0x564656af3ce0 2024-09-11T00:28:18.163 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:18.159+0000 7ff9eb67a740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x564656a50ae0 con 0x564656aee970 2024-09-11T00:28:18.163 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:18.159+0000 7ff9eb67a740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x564656a548f0 con 0x564656af12a0 2024-09-11T00:28:18.163 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:18.159+0000 7ff9e5c00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564656af12a0 0x564656af3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:28:18.164 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:18.159+0000 7ff9e5200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564656aee970 0x564656af0d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:28:18.164 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:18.159+0000 7ff9e5c00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564656af12a0 0x564656af3690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.12:52404/0 (socket says 172.21.15.12:52404) 2024-09-11T00:28:18.164 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:18.159+0000 7ff9e5c00640 1 -- 172.21.15.12:0/2429634708 learned_addr learned my addr 172.21.15.12:0/2429634708 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:28:18.164 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:18.159+0000 7ff9e5200640 1 --2- 172.21.15.12:0/2429634708 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564656af3ce0 0x564656af61a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:28:18.164 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:18.159+0000 7ff9e5c00640 1 -- 172.21.15.12:0/2429634708 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564656aee970 msgr2=0x564656af0d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:28:18.164 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:18.159+0000 7ff9e5c00640 1 --2- 172.21.15.12:0/2429634708 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564656aee970 0x564656af0d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:18.164 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:18.159+0000 7ff9e5c00640 1 -- 172.21.15.12:0/2429634708 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564656af3ce0 msgr2=0x564656af61a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:28:18.164 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:18.159+0000 7ff9e5c00640 1 --2- 172.21.15.12:0/2429634708 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564656af3ce0 0x564656af61a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:18.164 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:18.159+0000 7ff9e5c00640 1 -- 172.21.15.12:0/2429634708 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564656a1d780 con 0x564656af12a0 2024-09-11T00:28:18.164 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:18.160+0000 7ff9e5200640 1 --2- 172.21.15.12:0/2429634708 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564656af3ce0 0x564656af61a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T00:28:18.164 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:18.160+0000 7ff9de000640 1 --2- 172.21.15.12:0/2420591380 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7ff9c80b52e0 0x7ff9c80b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:28:18.164 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:18.160+0000 7ff9de000640 1 -- 172.21.15.12:0/2420591380 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9208+0+0 (secure 0 0 0) 0x7ff9d80e6580 con 0x7ff9e01064c0 2024-09-11T00:28:18.165 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:18.160+0000 7ff9e5c00640 1 --2- 172.21.15.12:0/2429634708 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564656af12a0 0x564656af3690 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7ff9d4006010 tx=0x7ff9d4004040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8a3d26e14094b961 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:28:18.165 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:18.160+0000 7ff9e5200640 1 --2- 172.21.15.12:0/2420591380 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7ff9c80b52e0 0x7ff9c80b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:28:18.165 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:18.160+0000 7ff9d2a00640 1 -- 172.21.15.12:0/2429634708 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9d4035030 con 0x564656af12a0 2024-09-11T00:28:18.165 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:18.161+0000 7ff9d2a00640 1 -- 172.21.15.12:0/2429634708 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff9d4040070 con 0x564656af12a0 2024-09-11T00:28:18.165 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:18.161+0000 7ff9d2a00640 1 -- 172.21.15.12:0/2429634708 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9d4031890 con 0x564656af12a0 2024-09-11T00:28:18.168 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:18.161+0000 7ff9eb67a740 1 -- 172.21.15.12:0/2429634708 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564656af12a0 msgr2=0x564656af3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:28:18.168 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:18.161+0000 7ff9eb67a740 1 --2- 172.21.15.12:0/2429634708 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564656af12a0 0x564656af3690 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7ff9d4006010 tx=0x7ff9d4004040 comp rx=0 tx=0).stop 2024-09-11T00:28:18.168 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:18.161+0000 7ff9e5200640 1 --2- 172.21.15.12:0/2420591380 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7ff9c80b52e0 0x7ff9c80b77a0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7ff9cc009730 tx=0x7ff9cc013040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:28:18.168 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:18.161+0000 7ff9eb67a740 1 -- 172.21.15.12:0/2429634708 shutdown_connections 2024-09-11T00:28:18.168 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:18.161+0000 7ff9eb67a740 1 --2- 172.21.15.12:0/2429634708 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564656af3ce0 0x564656af61a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:18.168 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:18.161+0000 7ff9eb67a740 1 --2- 172.21.15.12:0/2429634708 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564656af12a0 0x564656af3690 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:18.169 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:18.161+0000 7ff9eb67a740 1 --2- 172.21.15.12:0/2429634708 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564656aee970 0x564656af0d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:18.169 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:18.161+0000 7ff9eb67a740 1 -- 172.21.15.12:0/2429634708 >> 172.21.15.12:0/2429634708 conn(0x564656ae9ac0 msgr2=0x564656aea850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:28:18.169 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:18.161+0000 7ff9eb67a740 1 -- 172.21.15.12:0/2429634708 shutdown_connections 2024-09-11T00:28:18.169 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:18.162+0000 7ff9eb67a740 1 -- 172.21.15.12:0/2429634708 wait complete. 2024-09-11T00:28:18.169 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:18.162+0000 7ff9eb67a740 1 -- start start 2024-09-11T00:28:18.169 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:18.162+0000 7ff9eb67a740 1 Processor -- start 2024-09-11T00:28:18.169 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:18.163+0000 7ff9eb67a740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564656aee970 0x564656af8020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:28:18.169 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:18.163+0000 7ff9eb67a740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564656af12a0 0x564656af86c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:28:18.169 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:18.163+0000 7ff9eb67a740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564656af3ce0 0x564656af9290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:28:18.169 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:18.163+0000 7ff9eb67a740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x564656988ab0 con 0x564656aee970 2024-09-11T00:28:18.169 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:18.163+0000 7ff9eb67a740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x5646569f7570 con 0x564656af3ce0 2024-09-11T00:28:18.170 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:18.163+0000 7ff9eb67a740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x564656af6c60 con 0x564656af12a0 2024-09-11T00:28:18.170 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:18.163+0000 7ff9eb67a740 10 client.0 _ll_register_callbacks cb 0x564656a06100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:28:18.170 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:18.163+0000 7ff9e5c00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564656aee970 0x564656af8020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:28:18.170 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:18.163+0000 7ff9e5c00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564656aee970 0x564656af8020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:33762/0 (socket says 172.21.15.12:33762) 2024-09-11T00:28:18.170 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:18.163+0000 7ff9e5c00640 1 -- 172.21.15.12:0/2373568622 learned_addr learned my addr 172.21.15.12:0/2373568622 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:28:18.170 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:18.163+0000 7ff9e5200640 1 --2- 172.21.15.12:0/2373568622 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564656af12a0 0x564656af86c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:28:18.170 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:18.163+0000 7ff9e6600640 1 --2- 172.21.15.12:0/2373568622 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564656af3ce0 0x564656af9290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:28:18.170 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:18.163+0000 7ff9e5c00640 1 -- 172.21.15.12:0/2373568622 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564656af12a0 msgr2=0x564656af86c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:28:18.170 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:18.163+0000 7ff9e5c00640 1 --2- 172.21.15.12:0/2373568622 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564656af12a0 0x564656af86c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:18.170 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:18.163+0000 7ff9e5c00640 1 -- 172.21.15.12:0/2373568622 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564656af3ce0 msgr2=0x564656af9290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:28:18.170 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:18.163+0000 7ff9e5c00640 1 --2- 172.21.15.12:0/2373568622 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564656af3ce0 0x564656af9290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:18.170 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:18.163+0000 7ff9e5c00640 1 -- 172.21.15.12:0/2373568622 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5646567b2790 con 0x564656aee970 2024-09-11T00:28:18.170 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:18.163+0000 7ff9e6600640 1 --2- 172.21.15.12:0/2373568622 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564656af3ce0 0x564656af9290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:28:18.171 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:18.164+0000 7ff9e5c00640 1 --2- 172.21.15.12:0/2373568622 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564656aee970 0x564656af8020 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7ff9d4009900 tx=0x7ff9d40044e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5884bf298432dfe4 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:28:18.171 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:18.164+0000 7ff9c7400640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-09-11T00:28:18.171 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:18.164+0000 7ff9c7400640 1 -- 172.21.15.12:0/2373568622 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9d4035070 con 0x564656aee970 2024-09-11T00:28:18.171 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:18.164+0000 7ff9c7400640 1 -- 172.21.15.12:0/2373568622 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 9208+0+0 (secure 0 0 0) 0x7ff9d4003a80 con 0x564656aee970 2024-09-11T00:28:18.171 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:18.164+0000 7ff9eb67a740 1 -- 172.21.15.12:0/2373568622 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff9d4040070 con 0x564656aee970 2024-09-11T00:28:18.171 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:18.164+0000 7ff9c7400640 1 -- 172.21.15.12:0/2373568622 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=90) ==== 24+0+0 (secure 0 0 0) 0x7ff9d4003c80 con 0x564656aee970 2024-09-11T00:28:18.171 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:18.165+0000 7ff9eb67a740 10 client.17805 fetch_fsmap learned FSMap version 90 2024-09-11T00:28:18.171 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:18.165+0000 7ff9eb67a740 1 -- 172.21.15.12:0/2373568622 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=90,monmap=4+}) -- 0x56465682b620 con 0x564656aee970 2024-09-11T00:28:18.171 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:18.165+0000 7ff9c7400640 1 -- 172.21.15.12:0/2373568622 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff9d4040070 con 0x564656aee970 2024-09-11T00:28:18.171 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:18.165+0000 7ff9c7400640 1 -- 172.21.15.12:0/2373568622 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 90) ==== 8422+0+0 (secure 0 0 0) 0x7ff9d40457a0 con 0x564656aee970 2024-09-11T00:28:18.171 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:18.165+0000 7ff9eb67a740 10 client.17805 fetch_fsmap finished waiting for FSMap version 90 2024-09-11T00:28:18.171 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:18.165+0000 7ff9eb67a740 10 client.17805 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T00:28:18.172 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:18.165+0000 7ff9eb67a740 20 client.17805 populate_metadata read hostname 'smithi012' 2024-09-11T00:28:18.172 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:18.165+0000 7ff9eb67a740 1 --2- 172.21.15.12:0/2373568622 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x564656b034c0 0x564656b058b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:28:18.172 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:18.165+0000 7ff9eb67a740 4 client.17805 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T00:28:18.172 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:18.165+0000 7ff9eb67a740 1 -- 172.21.15.12:0/2373568622 --> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564656a63600 con 0x564656b034c0 2024-09-11T00:28:18.172 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:18.166+0000 7ff9e5200640 1 --2- 172.21.15.12:0/2373568622 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x564656b034c0 0x564656b058b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:28:18.173 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:18.166+0000 7ff9e5200640 1 --2- 172.21.15.12:0/2373568622 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x564656b034c0 0x564656b058b0 crc :-1 s=READY pgs=295 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=90ce5634f1c46a66 server_cookie=bcf53c731cd562e7 in_seq=0 out_seq=0 2024-09-11T00:28:18.173 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:18.166+0000 7ff9c7400640 10 client.17805 ms_handle_connect on v2:172.21.15.165:6838/501312984 2024-09-11T00:28:18.173 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:18.167+0000 7ff9c7400640 1 -- 172.21.15.12:0/2373568622 <== mds.0 v2:172.21.15.165:6838/501312984 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x564656a63600 con 0x564656b034c0 2024-09-11T00:28:18.173 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:18.167+0000 7ff9c7400640 10 client.17805 handle_command_reply: tid=0 2024-09-11T00:28:18.173 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:18.167+0000 7ff9eb67a740 1 client.17805 shutdown 2024-09-11T00:28:18.173 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:18.167+0000 7ff9eb67a740 1 -- 172.21.15.12:0/2373568622 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564656aee970 msgr2=0x564656af8020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:28:18.174 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:18.167+0000 7ff9eb67a740 1 --2- 172.21.15.12:0/2373568622 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564656aee970 0x564656af8020 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7ff9d4009900 tx=0x7ff9d40044e0 comp rx=0 tx=0).stop 2024-09-11T00:28:18.174 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:18.167+0000 7ff9eb67a740 1 -- 172.21.15.12:0/2373568622 shutdown_connections 2024-09-11T00:28:18.174 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:18.167+0000 7ff9eb67a740 1 --2- 172.21.15.12:0/2373568622 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x564656b034c0 0x564656b058b0 crc :-1 s=READY pgs=295 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:18.174 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:18.167+0000 7ff9eb67a740 1 --2- 172.21.15.12:0/2373568622 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564656af3ce0 0x564656af9290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:18.174 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:18.167+0000 7ff9eb67a740 1 --2- 172.21.15.12:0/2373568622 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564656af12a0 0x564656af86c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:18.174 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:18.167+0000 7ff9eb67a740 1 --2- 172.21.15.12:0/2373568622 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564656aee970 0x564656af8020 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:18.174 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:18.167+0000 7ff9eb67a740 1 -- 172.21.15.12:0/2373568622 >> 172.21.15.12:0/2373568622 conn(0x564656ae9ac0 msgr2=0x564656ae8bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:28:18.174 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:18.167+0000 7ff9c7400640 0 client.17805 ms_handle_reset on v2:172.21.15.165:6838/501312984 2024-09-11T00:28:18.174 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:18.168+0000 7ff9eb67a740 1 -- 172.21.15.12:0/2373568622 shutdown_connections 2024-09-11T00:28:18.174 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:18.168+0000 7ff9eb67a740 1 -- 172.21.15.12:0/2373568622 wait complete. 2024-09-11T00:28:18.174 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:18.168+0000 7ff9eb67a740 20 client.17805 trim_cache size 0 max 16384 2024-09-11T00:28:18.186 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:18.183+0000 7ff9eb67a740 1 Processor -- start 2024-09-11T00:28:18.186 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:18.183+0000 7ff9eb67a740 1 -- start start 2024-09-11T00:28:18.186 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:18.184+0000 7ff9eb67a740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564656aee970 0x564656aecb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:28:18.186 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:18.184+0000 7ff9eb67a740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564656af12a0 0x564656aed050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:28:18.186 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:18.184+0000 7ff9eb67a740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564656af3ce0 0x564656b0a100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:28:18.186 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:18.184+0000 7ff9eb67a740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7ff9e01ad7b0 con 0x564656aee970 2024-09-11T00:28:18.186 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:18.184+0000 7ff9eb67a740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7ff9e01af470 con 0x564656af12a0 2024-09-11T00:28:18.186 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:18.184+0000 7ff9eb67a740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7ff9e01adee0 con 0x564656af3ce0 2024-09-11T00:28:18.188 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:18.184+0000 7ff9e5c00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564656aee970 0x564656aecb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:28:18.188 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:18.184+0000 7ff9e5c00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564656aee970 0x564656aecb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:33774/0 (socket says 172.21.15.12:33774) 2024-09-11T00:28:18.189 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:18.184+0000 7ff9e5c00640 1 -- 172.21.15.12:0/143178087 learned_addr learned my addr 172.21.15.12:0/143178087 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:28:18.189 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:18.184+0000 7ff9e6600640 1 --2- 172.21.15.12:0/143178087 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564656af3ce0 0x564656b0a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:28:18.189 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:18.184+0000 7ff9e5200640 1 --2- 172.21.15.12:0/143178087 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564656af12a0 0x564656aed050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:28:18.189 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:18.184+0000 7ff9e5c00640 1 -- 172.21.15.12:0/143178087 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564656af3ce0 msgr2=0x564656b0a100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:28:18.189 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:18.184+0000 7ff9e5c00640 1 --2- 172.21.15.12:0/143178087 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564656af3ce0 0x564656b0a100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:18.189 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:18.184+0000 7ff9e5c00640 1 -- 172.21.15.12:0/143178087 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564656af12a0 msgr2=0x564656aed050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:28:18.189 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:18.184+0000 7ff9e5c00640 1 --2- 172.21.15.12:0/143178087 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564656af12a0 0x564656aed050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:18.189 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:18.184+0000 7ff9e5c00640 1 -- 172.21.15.12:0/143178087 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564656b00680 con 0x564656aee970 2024-09-11T00:28:18.189 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:18.185+0000 7ff9e5c00640 1 --2- 172.21.15.12:0/143178087 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564656aee970 0x564656aecb10 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7ff9d402a940 tx=0x7ff9d4003e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=596f497eccc89e0c server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:28:18.189 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:18.185+0000 7ff9d2000640 1 -- 172.21.15.12:0/143178087 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9d4035070 con 0x564656aee970 2024-09-11T00:28:18.189 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:18.185+0000 7ff9d2000640 1 -- 172.21.15.12:0/143178087 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff9d4040040 con 0x564656aee970 2024-09-11T00:28:18.190 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:18.185+0000 7ff9d2000640 1 -- 172.21.15.12:0/143178087 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9d403c890 con 0x564656aee970 2024-09-11T00:28:18.190 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:18.186+0000 7ff9eb67a740 1 -- 172.21.15.12:0/143178087 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564656aee970 msgr2=0x564656aecb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:28:18.190 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:18.186+0000 7ff9eb67a740 1 --2- 172.21.15.12:0/143178087 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564656aee970 0x564656aecb10 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7ff9d402a940 tx=0x7ff9d4003e20 comp rx=0 tx=0).stop 2024-09-11T00:28:18.190 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:18.186+0000 7ff9eb67a740 1 -- 172.21.15.12:0/143178087 shutdown_connections 2024-09-11T00:28:18.190 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:18.186+0000 7ff9eb67a740 1 --2- 172.21.15.12:0/143178087 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564656af3ce0 0x564656b0a100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:18.190 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:18.186+0000 7ff9eb67a740 1 --2- 172.21.15.12:0/143178087 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564656af12a0 0x564656aed050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:18.190 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:18.186+0000 7ff9eb67a740 1 --2- 172.21.15.12:0/143178087 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564656aee970 0x564656aecb10 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:18.190 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:18.186+0000 7ff9eb67a740 1 -- 172.21.15.12:0/143178087 >> 172.21.15.12:0/143178087 conn(0x564656ae9ac0 msgr2=0x564656aea560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:28:18.190 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:18.186+0000 7ff9eb67a740 1 -- 172.21.15.12:0/143178087 shutdown_connections 2024-09-11T00:28:18.190 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:18.186+0000 7ff9eb67a740 1 -- 172.21.15.12:0/143178087 wait complete. 2024-09-11T00:28:18.190 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:18.186+0000 7ff9eb67a740 1 -- start start 2024-09-11T00:28:18.193 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:18.186+0000 7ff9eb67a740 1 Processor -- start 2024-09-11T00:28:18.193 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:18.187+0000 7ff9eb67a740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564656aee970 0x564656aff710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:28:18.193 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:18.187+0000 7ff9eb67a740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564656af12a0 0x564656affdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:28:18.193 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:18.187+0000 7ff9eb67a740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564656af3ce0 0x564656afe550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:28:18.193 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:18.187+0000 7ff9eb67a740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7ff9e01aeda0 con 0x564656aee970 2024-09-11T00:28:18.193 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:18.187+0000 7ff9eb67a740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7ff9e01ae670 con 0x564656af3ce0 2024-09-11T00:28:18.193 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:18.187+0000 7ff9eb67a740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7ff9e01ab790 con 0x564656af12a0 2024-09-11T00:28:18.193 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:18.187+0000 7ff9eb67a740 10 client.0 _ll_register_callbacks cb 0x7ff9cc009a10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:28:18.193 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:18.187+0000 7ff9e5c00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564656aee970 0x564656aff710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:28:18.193 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:18.187+0000 7ff9e5c00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564656aee970 0x564656aff710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:33788/0 (socket says 172.21.15.12:33788) 2024-09-11T00:28:18.193 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:18.187+0000 7ff9e5c00640 1 -- 172.21.15.12:0/1628771005 learned_addr learned my addr 172.21.15.12:0/1628771005 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:28:18.194 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:18.187+0000 7ff9e6600640 1 --2- 172.21.15.12:0/1628771005 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564656af3ce0 0x564656afe550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:28:18.194 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:18.187+0000 7ff9e5200640 1 --2- 172.21.15.12:0/1628771005 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564656af12a0 0x564656affdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:28:18.194 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:18.187+0000 7ff9e5c00640 1 -- 172.21.15.12:0/1628771005 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564656af12a0 msgr2=0x564656affdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:28:18.194 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:18.187+0000 7ff9e5c00640 1 --2- 172.21.15.12:0/1628771005 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564656af12a0 0x564656affdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:18.194 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:18.187+0000 7ff9e5c00640 1 -- 172.21.15.12:0/1628771005 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564656af3ce0 msgr2=0x564656afe550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:28:18.194 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:18.187+0000 7ff9e5c00640 1 --2- 172.21.15.12:0/1628771005 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564656af3ce0 0x564656afe550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:18.194 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:18.187+0000 7ff9e5c00640 1 -- 172.21.15.12:0/1628771005 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564656afed80 con 0x564656aee970 2024-09-11T00:28:18.194 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:18.188+0000 7ff9e5c00640 1 --2- 172.21.15.12:0/1628771005 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564656aee970 0x564656aff710 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7ff9d4045d60 tx=0x7ff9d4009230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4db2fb490c88217 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:28:18.194 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:18.188+0000 7ff9c7400640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-09-11T00:28:18.194 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:18.188+0000 7ff9c7400640 1 -- 172.21.15.12:0/1628771005 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9d4035070 con 0x564656aee970 2024-09-11T00:28:18.194 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:18.188+0000 7ff9c7400640 1 -- 172.21.15.12:0/1628771005 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 9208+0+0 (secure 0 0 0) 0x7ff9d40315a0 con 0x564656aee970 2024-09-11T00:28:18.194 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:18.188+0000 7ff9eb67a740 1 -- 172.21.15.12:0/1628771005 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff9d4040040 con 0x564656aee970 2024-09-11T00:28:18.195 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:18.188+0000 7ff9c7400640 1 -- 172.21.15.12:0/1628771005 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=90) ==== 24+0+0 (secure 0 0 0) 0x7ff9d40044e0 con 0x564656aee970 2024-09-11T00:28:18.195 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:18.188+0000 7ff9eb67a740 10 client.17811 fetch_fsmap learned FSMap version 90 2024-09-11T00:28:18.195 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:18.188+0000 7ff9eb67a740 1 -- 172.21.15.12:0/1628771005 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=90,monmap=4+}) -- 0x564656afdc40 con 0x564656aee970 2024-09-11T00:28:18.195 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:18.189+0000 7ff9c7400640 1 -- 172.21.15.12:0/1628771005 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff9d4040040 con 0x564656aee970 2024-09-11T00:28:18.195 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:18.189+0000 7ff9c7400640 1 -- 172.21.15.12:0/1628771005 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 90) ==== 8422+0+0 (secure 0 0 0) 0x7ff9d405c6a0 con 0x564656aee970 2024-09-11T00:28:18.195 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:18.189+0000 7ff9eb67a740 10 client.17811 fetch_fsmap finished waiting for FSMap version 90 2024-09-11T00:28:18.195 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:18.189+0000 7ff9eb67a740 10 client.17811 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T00:28:18.195 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:18.189+0000 7ff9eb67a740 20 client.17811 populate_metadata read hostname 'smithi012' 2024-09-11T00:28:18.195 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:18.189+0000 7ff9eb67a740 1 --2- 172.21.15.12:0/1628771005 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x564656b707f0 0x564656b72c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:28:18.195 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:18.189+0000 7ff9eb67a740 4 client.17811 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T00:28:18.195 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:18.189+0000 7ff9eb67a740 1 -- 172.21.15.12:0/1628771005 --> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] -- command(tid 0: {"prefix": "flush journal"}) -- 0x564656a5f0b0 con 0x564656b707f0 2024-09-11T00:28:18.195 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:18.189+0000 7ff9e5200640 1 --2- 172.21.15.12:0/1628771005 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x564656b707f0 0x564656b72c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:28:18.195 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:18.190+0000 7ff9e5200640 1 --2- 172.21.15.12:0/1628771005 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x564656b707f0 0x564656b72c10 crc :-1 s=READY pgs=296 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fcd42f5fa997a73a server_cookie=48b41e6a2fa259d0 in_seq=0 out_seq=0 2024-09-11T00:28:18.196 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:18.190+0000 7ff9c7400640 10 client.17811 ms_handle_connect on v2:172.21.15.165:6838/501312984 2024-09-11T00:28:18.250 INFO:tasks.workunit.client.0.smithi012.stdout: 10 135552 14.76 MB/sec execute 374 sec latency 6187.655 ms 2024-09-11T00:28:18.280 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:18.277+0000 7ff9c7400640 1 -- 172.21.15.12:0/1628771005 <== mds.0 v2:172.21.15.165:6838/501312984 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x564656a5f0b0 con 0x564656b707f0 2024-09-11T00:28:18.280 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:18.277+0000 7ff9c7400640 10 client.17811 handle_command_reply: tid=0 2024-09-11T00:28:18.280 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:18.277+0000 7ff9eb67a740 1 client.17811 shutdown 2024-09-11T00:28:18.280 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:18.277+0000 7ff9eb67a740 1 -- 172.21.15.12:0/1628771005 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564656aee970 msgr2=0x564656aff710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:28:18.280 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:18.277+0000 7ff9eb67a740 1 --2- 172.21.15.12:0/1628771005 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564656aee970 0x564656aff710 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7ff9d4045d60 tx=0x7ff9d4009230 comp rx=0 tx=0).stop 2024-09-11T00:28:18.280 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:18.277+0000 7ff9eb67a740 1 -- 172.21.15.12:0/1628771005 shutdown_connections 2024-09-11T00:28:18.281 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:18.277+0000 7ff9eb67a740 1 --2- 172.21.15.12:0/1628771005 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x564656b707f0 0x564656b72c10 crc :-1 s=READY pgs=296 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:18.281 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:18.277+0000 7ff9eb67a740 1 --2- 172.21.15.12:0/1628771005 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564656af3ce0 0x564656afe550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:18.281 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:18.277+0000 7ff9c7400640 0 client.17811 ms_handle_reset on v2:172.21.15.165:6838/501312984 2024-09-11T00:28:18.281 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:18.277+0000 7ff9eb67a740 1 --2- 172.21.15.12:0/1628771005 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564656af12a0 0x564656affdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:18.281 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:18.277+0000 7ff9eb67a740 1 --2- 172.21.15.12:0/1628771005 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564656aee970 0x564656aff710 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:18.281 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:18.277+0000 7ff9eb67a740 1 -- 172.21.15.12:0/1628771005 >> 172.21.15.12:0/1628771005 conn(0x564656ae9ac0 msgr2=0x564656aeda20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:28:18.281 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:18.277+0000 7ff9eb67a740 1 -- 172.21.15.12:0/1628771005 shutdown_connections 2024-09-11T00:28:18.281 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:18.277+0000 7ff9eb67a740 1 -- 172.21.15.12:0/1628771005 wait complete. 2024-09-11T00:28:18.281 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:18.277+0000 7ff9eb67a740 20 client.17811 trim_cache size 0 max 16384 2024-09-11T00:28:18.282 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:{ 2024-09-11T00:28:18.282 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "message": "", 2024-09-11T00:28:18.282 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "return_code": 0 2024-09-11T00:28:18.282 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:} 2024-09-11T00:28:18.283 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:18.278+0000 7ff9e7000640 1 -- 172.21.15.12:0/2420591380 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7ff9c80b52e0 msgr2=0x7ff9c80b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:28:18.283 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:18.278+0000 7ff9e7000640 1 --2- 172.21.15.12:0/2420591380 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7ff9c80b52e0 0x7ff9c80b77a0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7ff9cc009730 tx=0x7ff9cc013040 comp rx=0 tx=0).stop 2024-09-11T00:28:18.283 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:18.278+0000 7ff9e7000640 1 -- 172.21.15.12:0/2420591380 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff9e01064c0 msgr2=0x7ff9e019c020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:28:18.283 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:18.278+0000 7ff9e7000640 1 --2- 172.21.15.12:0/2420591380 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff9e01064c0 0x7ff9e019c020 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7ff9d800eb10 tx=0x7ff9d800c2d0 comp rx=0 tx=0).stop 2024-09-11T00:28:18.283 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:18.278+0000 7ff9e7000640 1 -- 172.21.15.12:0/2420591380 shutdown_connections 2024-09-11T00:28:18.283 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:18.278+0000 7ff9e7000640 1 --2- 172.21.15.12:0/2420591380 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7ff9c80b52e0 0x7ff9c80b77a0 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:18.283 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:18.278+0000 7ff9e7000640 1 --2- 172.21.15.12:0/2420591380 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff9e01064c0 0x7ff9e019c020 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:18.284 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:18.278+0000 7ff9e7000640 1 --2- 172.21.15.12:0/2420591380 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff9e0105b00 0x7ff9e019b3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:18.284 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:18.278+0000 7ff9e7000640 1 --2- 172.21.15.12:0/2420591380 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff9e01049b0 0x7ff9e019ada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:18.284 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:18.278+0000 7ff9e7000640 1 -- 172.21.15.12:0/2420591380 >> 172.21.15.12:0/2420591380 conn(0x7ff9e0100250 msgr2=0x7ff9e0108ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:28:18.284 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:18.278+0000 7ff9e7000640 1 -- 172.21.15.12:0/2420591380 shutdown_connections 2024-09-11T00:28:18.284 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:18.278+0000 7ff9e7000640 1 -- 172.21.15.12:0/2420591380 wait complete. 2024-09-11T00:28:19.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:28:18 smithi165 ceph-mon[35356]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T00:28:19.063 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:28:18 smithi012 ceph-mon[30333]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T00:28:19.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:28:18 smithi178 ceph-mon[35919]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T00:28:19.250 INFO:tasks.workunit.client.0.smithi012.stdout: 10 135552 14.72 MB/sec execute 375 sec latency 7187.712 ms 2024-09-11T00:28:20.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:28:19 smithi165 ceph-mon[35356]: pgmap v1037: 129 pgs: 129 active+clean; 686 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 6.3 MiB/s wr, 460 op/s 2024-09-11T00:28:20.063 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:28:19 smithi012 ceph-mon[30333]: pgmap v1037: 129 pgs: 129 active+clean; 686 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 6.3 MiB/s wr, 460 op/s 2024-09-11T00:28:20.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:28:19 smithi178 ceph-mon[35919]: pgmap v1037: 129 pgs: 129 active+clean; 686 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 6.3 MiB/s wr, 460 op/s 2024-09-11T00:28:20.251 INFO:tasks.workunit.client.0.smithi012.stdout: 10 135552 14.68 MB/sec execute 376 sec latency 8187.786 ms 2024-09-11T00:28:21.251 INFO:tasks.workunit.client.0.smithi012.stdout: 10 135552 14.64 MB/sec execute 377 sec latency 9187.903 ms 2024-09-11T00:28:22.033 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:28:21 smithi165 ceph-mon[35356]: pgmap v1038: 129 pgs: 129 active+clean; 685 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 6.2 MiB/s wr, 454 op/s 2024-09-11T00:28:22.063 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:28:21 smithi012 ceph-mon[30333]: pgmap v1038: 129 pgs: 129 active+clean; 685 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 6.2 MiB/s wr, 454 op/s 2024-09-11T00:28:22.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:28:21 smithi178 ceph-mon[35919]: pgmap v1038: 129 pgs: 129 active+clean; 685 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 6.2 MiB/s wr, 454 op/s 2024-09-11T00:28:22.251 INFO:tasks.workunit.client.0.smithi012.stdout: 10 135552 14.60 MB/sec execute 378 sec latency 10187.983 ms 2024-09-11T00:28:23.251 INFO:tasks.workunit.client.0.smithi012.stdout: 10 135552 14.56 MB/sec execute 379 sec latency 11188.105 ms 2024-09-11T00:28:24.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:28:23 smithi165 ceph-mon[35356]: pgmap v1039: 129 pgs: 129 active+clean; 685 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 4.3 MiB/s wr, 308 op/s 2024-09-11T00:28:24.063 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:28:23 smithi012 ceph-mon[30333]: pgmap v1039: 129 pgs: 129 active+clean; 685 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 4.3 MiB/s wr, 308 op/s 2024-09-11T00:28:24.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:28:23 smithi178 ceph-mon[35919]: pgmap v1039: 129 pgs: 129 active+clean; 685 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 4.3 MiB/s wr, 308 op/s 2024-09-11T00:28:24.251 INFO:tasks.workunit.client.0.smithi012.stdout: 10 135552 14.53 MB/sec execute 380 sec latency 12188.203 ms 2024-09-11T00:28:25.251 INFO:tasks.workunit.client.0.smithi012.stdout: 10 135552 14.49 MB/sec execute 381 sec latency 13188.308 ms 2024-09-11T00:28:26.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:28:25 smithi165 ceph-mon[35356]: pgmap v1040: 129 pgs: 129 active+clean; 685 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.0 MiB/s wr, 204 op/s 2024-09-11T00:28:26.063 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:28:25 smithi012 ceph-mon[30333]: pgmap v1040: 129 pgs: 129 active+clean; 685 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.0 MiB/s wr, 204 op/s 2024-09-11T00:28:26.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:28:25 smithi178 ceph-mon[35919]: pgmap v1040: 129 pgs: 129 active+clean; 685 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.0 MiB/s wr, 204 op/s 2024-09-11T00:28:26.251 INFO:tasks.workunit.client.0.smithi012.stdout: 10 135552 14.45 MB/sec execute 382 sec latency 14188.464 ms 2024-09-11T00:28:27.251 INFO:tasks.workunit.client.0.smithi012.stdout: 10 135552 14.41 MB/sec execute 383 sec latency 15188.554 ms 2024-09-11T00:28:28.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:28:27 smithi165 ceph-mon[35356]: pgmap v1041: 129 pgs: 129 active+clean; 685 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.7 MiB/s wr, 204 op/s 2024-09-11T00:28:28.063 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:28:27 smithi012 ceph-mon[30333]: pgmap v1041: 129 pgs: 129 active+clean; 685 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.7 MiB/s wr, 204 op/s 2024-09-11T00:28:28.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:28:27 smithi178 ceph-mon[35919]: pgmap v1041: 129 pgs: 129 active+clean; 685 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.7 MiB/s wr, 204 op/s 2024-09-11T00:28:28.252 INFO:tasks.workunit.client.0.smithi012.stdout: 10 135552 14.37 MB/sec execute 384 sec latency 16188.682 ms 2024-09-11T00:28:29.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:28:28 smithi165 ceph-mon[35356]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T00:28:29.063 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:28:28 smithi012 ceph-mon[30333]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T00:28:29.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:28:28 smithi178 ceph-mon[35919]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T00:28:29.252 INFO:tasks.workunit.client.0.smithi012.stdout: 10 135552 14.34 MB/sec execute 385 sec latency 17188.840 ms 2024-09-11T00:28:30.033 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:28:29 smithi165 ceph-mon[35356]: pgmap v1042: 129 pgs: 129 active+clean; 685 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 29 KiB/s wr, 41 op/s 2024-09-11T00:28:30.063 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:28:29 smithi012 ceph-mon[30333]: pgmap v1042: 129 pgs: 129 active+clean; 685 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 29 KiB/s wr, 41 op/s 2024-09-11T00:28:30.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:28:29 smithi178 ceph-mon[35919]: pgmap v1042: 129 pgs: 129 active+clean; 685 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 29 KiB/s wr, 41 op/s 2024-09-11T00:28:30.252 INFO:tasks.workunit.client.0.smithi012.stdout: 10 135552 14.30 MB/sec execute 386 sec latency 18189.003 ms 2024-09-11T00:28:31.252 INFO:tasks.workunit.client.0.smithi012.stdout: 10 135552 14.26 MB/sec execute 387 sec latency 19189.163 ms 2024-09-11T00:28:31.426 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:31.423+0000 7f5f8be00640 1 -- 172.21.15.12:0/163748559 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5f8c1049b0 msgr2=0x7f5f8c104db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:28:31.426 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:31.423+0000 7f5f8be00640 1 --2- 172.21.15.12:0/163748559 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5f8c1049b0 0x7f5f8c104db0 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f5f7c009930 tx=0x7f5f7c02a420 comp rx=0 tx=0).stop 2024-09-11T00:28:31.426 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:31.423+0000 7f5f8be00640 1 -- 172.21.15.12:0/163748559 shutdown_connections 2024-09-11T00:28:31.426 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:31.423+0000 7f5f8be00640 1 --2- 172.21.15.12:0/163748559 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5f8c1064c0 0x7f5f8c10ce40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:31.426 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:31.423+0000 7f5f8be00640 1 --2- 172.21.15.12:0/163748559 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5f8c105b00 0x7f5f8c105f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:31.426 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:31.423+0000 7f5f8be00640 1 --2- 172.21.15.12:0/163748559 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5f8c1049b0 0x7f5f8c104db0 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:31.426 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:31.423+0000 7f5f8be00640 1 -- 172.21.15.12:0/163748559 >> 172.21.15.12:0/163748559 conn(0x7f5f8c100270 msgr2=0x7f5f8c102690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:28:31.426 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:31.423+0000 7f5f8be00640 1 -- 172.21.15.12:0/163748559 shutdown_connections 2024-09-11T00:28:31.426 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:31.423+0000 7f5f8be00640 1 -- 172.21.15.12:0/163748559 wait complete. 2024-09-11T00:28:31.429 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:31.424+0000 7f5f8be00640 1 Processor -- start 2024-09-11T00:28:31.429 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:31.424+0000 7f5f8be00640 1 -- start start 2024-09-11T00:28:31.429 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:31.425+0000 7f5f8be00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5f8c1049b0 0x7f5f8c19add0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:28:31.429 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:31.425+0000 7f5f8be00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5f8c105b00 0x7f5f8c19b420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:28:31.429 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:31.425+0000 7f5f8be00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5f8c1064c0 0x7f5f8c19c050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:28:31.430 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:31.425+0000 7f5f8be00640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f5f8c110620 con 0x7f5f8c1064c0 2024-09-11T00:28:31.430 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:31.425+0000 7f5f8be00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f5f8c1104a0 con 0x7f5f8c1049b0 2024-09-11T00:28:31.430 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:31.425+0000 7f5f8be00640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f5f8c1107a0 con 0x7f5f8c105b00 2024-09-11T00:28:31.430 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:31.425+0000 7f5f8b400640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5f8c1064c0 0x7f5f8c19c050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:28:31.430 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:31.425+0000 7f5f83e00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5f8c105b00 0x7f5f8c19b420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:28:31.430 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:31.425+0000 7f5f8aa00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5f8c1049b0 0x7f5f8c19add0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:28:31.430 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:31.425+0000 7f5f83e00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5f8c105b00 0x7f5f8c19b420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.12:56298/0 (socket says 172.21.15.12:56298) 2024-09-11T00:28:31.430 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:31.425+0000 7f5f8b400640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5f8c1064c0 0x7f5f8c19c050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:50964/0 (socket says 172.21.15.12:50964) 2024-09-11T00:28:31.430 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:31.425+0000 7f5f83e00640 1 -- 172.21.15.12:0/1845328880 learned_addr learned my addr 172.21.15.12:0/1845328880 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:28:31.430 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:31.425+0000 7f5f83e00640 1 -- 172.21.15.12:0/1845328880 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5f8c1049b0 msgr2=0x7f5f8c19add0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:28:31.430 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:31.425+0000 7f5f83e00640 1 --2- 172.21.15.12:0/1845328880 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5f8c1049b0 0x7f5f8c19add0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:31.430 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:31.425+0000 7f5f83e00640 1 -- 172.21.15.12:0/1845328880 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5f8c1064c0 msgr2=0x7f5f8c19c050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:28:31.431 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:31.425+0000 7f5f83e00640 1 --2- 172.21.15.12:0/1845328880 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5f8c1064c0 0x7f5f8c19c050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:31.431 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:31.426+0000 7f5f83e00640 1 -- 172.21.15.12:0/1845328880 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f8c19c910 con 0x7f5f8c105b00 2024-09-11T00:28:31.431 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:31.426+0000 7f5f8aa00640 1 --2- 172.21.15.12:0/1845328880 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5f8c1049b0 0x7f5f8c19add0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:28:31.431 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:31.426+0000 7f5f83e00640 1 --2- 172.21.15.12:0/1845328880 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5f8c105b00 0x7f5f8c19b420 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f5f7400e9f0 tx=0x7f5f7400eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:28:31.431 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:31.426+0000 7f5f83400640 1 -- 172.21.15.12:0/1845328880 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f7400cd80 con 0x7f5f8c105b00 2024-09-11T00:28:31.431 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:31.426+0000 7f5f8be00640 1 -- 172.21.15.12:0/1845328880 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5f8c19cc00 con 0x7f5f8c105b00 2024-09-11T00:28:31.431 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:31.426+0000 7f5f83400640 1 -- 172.21.15.12:0/1845328880 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5f740108e0 con 0x7f5f8c105b00 2024-09-11T00:28:31.431 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:31.426+0000 7f5f83400640 1 -- 172.21.15.12:0/1845328880 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f74018980 con 0x7f5f8c105b00 2024-09-11T00:28:31.431 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:31.426+0000 7f5f8be00640 1 -- 172.21.15.12:0/1845328880 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5f8c1a32d0 con 0x7f5f8c105b00 2024-09-11T00:28:31.433 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:31.429+0000 7f5f94805740 1 Processor -- start 2024-09-11T00:28:31.434 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:31.429+0000 7f5f94805740 1 -- start start 2024-09-11T00:28:31.434 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:31.429+0000 7f5f94805740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563eda4d1970 0x563eda4d3d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:28:31.434 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:31.430+0000 7f5f94805740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563eda4d42a0 0x563eda4d6690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:28:31.434 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:31.430+0000 7f5f94805740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563eda4d6ce0 0x563eda4d91a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:28:31.434 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:31.430+0000 7f5f94805740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x563eda4446b0 con 0x563eda4d1970 2024-09-11T00:28:31.434 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:31.430+0000 7f5f94805740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x563eda433ae0 con 0x563eda4d42a0 2024-09-11T00:28:31.434 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:31.430+0000 7f5f94805740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x563eda4378f0 con 0x563eda4d6ce0 2024-09-11T00:28:31.434 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:31.430+0000 7f5f8b400640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563eda4d1970 0x563eda4d3d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:28:31.434 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:31.430+0000 7f5f8b400640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563eda4d1970 0x563eda4d3d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:50968/0 (socket says 172.21.15.12:50968) 2024-09-11T00:28:31.434 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:31.430+0000 7f5f8b400640 1 -- 172.21.15.12:0/2185926608 learned_addr learned my addr 172.21.15.12:0/2185926608 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:28:31.435 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:31.430+0000 7f5f8aa00640 1 --2- 172.21.15.12:0/2185926608 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563eda4d42a0 0x563eda4d6690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:28:31.435 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:31.430+0000 7f5f83400640 1 -- 172.21.15.12:0/1845328880 <== mon.1 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7f5f74010430 con 0x7f5f8c105b00 2024-09-11T00:28:31.435 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:31.430+0000 7f5f8b400640 1 -- 172.21.15.12:0/2185926608 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563eda4d6ce0 msgr2=0x563eda4d91a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-11T00:28:31.435 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:31.430+0000 7f5f8b400640 1 --2- 172.21.15.12:0/2185926608 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563eda4d6ce0 0x563eda4d91a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:31.435 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:31.430+0000 7f5f8b400640 1 -- 172.21.15.12:0/2185926608 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563eda4d42a0 msgr2=0x563eda4d6690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:28:31.435 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:31.430+0000 7f5f8b400640 1 --2- 172.21.15.12:0/2185926608 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563eda4d42a0 0x563eda4d6690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:31.435 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:31.430+0000 7f5f8b400640 1 -- 172.21.15.12:0/2185926608 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563eda400780 con 0x563eda4d1970 2024-09-11T00:28:31.435 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:31.430+0000 7f5f8aa00640 1 --2- 172.21.15.12:0/2185926608 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563eda4d42a0 0x563eda4d6690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:28:31.435 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:31.431+0000 7f5f8b400640 1 --2- 172.21.15.12:0/2185926608 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563eda4d1970 0x563eda4d3d60 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f5f84009f90 tx=0x7f5f840118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a273b4872ad5f7d9 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:28:31.435 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:31.431+0000 7f5f7be00640 1 -- 172.21.15.12:0/2185926608 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f84004d20 con 0x563eda4d1970 2024-09-11T00:28:31.435 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:31.431+0000 7f5f7be00640 1 -- 172.21.15.12:0/2185926608 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5f84004050 con 0x563eda4d1970 2024-09-11T00:28:31.436 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:31.431+0000 7f5f7be00640 1 -- 172.21.15.12:0/2185926608 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f84005280 con 0x563eda4d1970 2024-09-11T00:28:31.441 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:31.432+0000 7f5f94805740 1 -- 172.21.15.12:0/2185926608 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563eda4d1970 msgr2=0x563eda4d3d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:28:31.441 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:31.432+0000 7f5f94805740 1 --2- 172.21.15.12:0/2185926608 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563eda4d1970 0x563eda4d3d60 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f5f84009f90 tx=0x7f5f840118d0 comp rx=0 tx=0).stop 2024-09-11T00:28:31.441 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:31.432+0000 7f5f94805740 1 -- 172.21.15.12:0/2185926608 shutdown_connections 2024-09-11T00:28:31.441 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:31.432+0000 7f5f94805740 1 --2- 172.21.15.12:0/2185926608 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563eda4d6ce0 0x563eda4d91a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:31.441 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:31.432+0000 7f5f94805740 1 --2- 172.21.15.12:0/2185926608 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563eda4d42a0 0x563eda4d6690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:31.441 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:31.432+0000 7f5f94805740 1 --2- 172.21.15.12:0/2185926608 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563eda4d1970 0x563eda4d3d60 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:31.441 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:31.432+0000 7f5f94805740 1 -- 172.21.15.12:0/2185926608 >> 172.21.15.12:0/2185926608 conn(0x563eda4ccac0 msgr2=0x563eda4cd850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:28:31.441 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:31.432+0000 7f5f94805740 1 -- 172.21.15.12:0/2185926608 shutdown_connections 2024-09-11T00:28:31.441 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:31.432+0000 7f5f94805740 1 -- 172.21.15.12:0/2185926608 wait complete. 2024-09-11T00:28:31.441 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:31.432+0000 7f5f83400640 1 --2- 172.21.15.12:0/1845328880 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f5f700b51f0 0x7f5f700b76b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:28:31.441 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:31.432+0000 7f5f94805740 1 -- start start 2024-09-11T00:28:31.442 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:31.432+0000 7f5f83400640 1 -- 172.21.15.12:0/1845328880 <== mon.1 v2:172.21.15.178:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9208+0+0 (secure 0 0 0) 0x7f5f740e62f0 con 0x7f5f8c105b00 2024-09-11T00:28:31.442 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:31.432+0000 7f5f8aa00640 1 --2- 172.21.15.12:0/1845328880 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f5f700b51f0 0x7f5f700b76b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:28:31.442 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:31.433+0000 7f5f94805740 1 Processor -- start 2024-09-11T00:28:31.442 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:31.433+0000 7f5f94805740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563eda4d1970 0x563eda4db120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:28:31.442 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:31.433+0000 7f5f94805740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563eda4d42a0 0x563eda4db7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:28:31.442 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:31.433+0000 7f5f94805740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563eda4d6ce0 0x563eda4dc390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:28:31.442 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:31.433+0000 7f5f8aa00640 1 --2- 172.21.15.12:0/1845328880 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f5f700b51f0 0x7f5f700b76b0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f5f7c006010 tx=0x7f5f7c0314a0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:28:31.442 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:31.433+0000 7f5f94805740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x563eda36bab0 con 0x563eda4d6ce0 2024-09-11T00:28:31.442 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:31.433+0000 7f5f94805740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x563eda3da570 con 0x563eda4d42a0 2024-09-11T00:28:31.442 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:31.433+0000 7f5f94805740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x563eda4d9cc0 con 0x563eda4d1970 2024-09-11T00:28:31.442 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:31.433+0000 7f5f94805740 10 client.0 _ll_register_callbacks cb 0x563eda3e9100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:28:31.442 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:31.434+0000 7f5f8b400640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563eda4d6ce0 0x563eda4dc390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:28:31.443 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:31.434+0000 7f5f8b400640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563eda4d6ce0 0x563eda4dc390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:50970/0 (socket says 172.21.15.12:50970) 2024-09-11T00:28:31.443 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:31.434+0000 7f5f8b400640 1 -- 172.21.15.12:0/2285737129 learned_addr learned my addr 172.21.15.12:0/2285737129 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:28:31.443 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:31.434+0000 7f5f8aa00640 1 --2- 172.21.15.12:0/2285737129 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563eda4d1970 0x563eda4db120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:28:31.443 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:31.434+0000 7f5f83e00640 1 --2- 172.21.15.12:0/2285737129 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563eda4d42a0 0x563eda4db7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:28:31.443 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:31.434+0000 7f5f8b400640 1 -- 172.21.15.12:0/2285737129 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563eda4d1970 msgr2=0x563eda4db120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:28:31.443 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:31.434+0000 7f5f8b400640 1 --2- 172.21.15.12:0/2285737129 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563eda4d1970 0x563eda4db120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:31.443 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:31.434+0000 7f5f8b400640 1 -- 172.21.15.12:0/2285737129 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563eda4d42a0 msgr2=0x563eda4db7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:28:31.443 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:31.434+0000 7f5f8b400640 1 --2- 172.21.15.12:0/2285737129 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563eda4d42a0 0x563eda4db7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:31.443 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:31.434+0000 7f5f8b400640 1 -- 172.21.15.12:0/2285737129 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563eda195790 con 0x563eda4d6ce0 2024-09-11T00:28:31.443 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:31.434+0000 7f5f83e00640 1 --2- 172.21.15.12:0/2285737129 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563eda4d42a0 0x563eda4db7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:28:31.443 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:31.434+0000 7f5f8b400640 1 --2- 172.21.15.12:0/2285737129 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563eda4d6ce0 0x563eda4dc390 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f5f84011dd0 tx=0x7f5f840044b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f76d2d8c641e66 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:28:31.443 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:31.434+0000 7f5f78c00640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-09-11T00:28:31.443 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:31.435+0000 7f5f78c00640 1 -- 172.21.15.12:0/2285737129 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f84028070 con 0x563eda4d6ce0 2024-09-11T00:28:31.444 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:31.435+0000 7f5f78c00640 1 -- 172.21.15.12:0/2285737129 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 9208+0+0 (secure 0 0 0) 0x7f5f84005040 con 0x563eda4d6ce0 2024-09-11T00:28:31.444 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:31.435+0000 7f5f94805740 1 -- 172.21.15.12:0/2285737129 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5f84004050 con 0x563eda4d6ce0 2024-09-11T00:28:31.444 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:31.435+0000 7f5f78c00640 1 -- 172.21.15.12:0/2285737129 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=91) ==== 24+0+0 (secure 0 0 0) 0x7f5f84034050 con 0x563eda4d6ce0 2024-09-11T00:28:31.444 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:31.435+0000 7f5f94805740 10 client.17823 fetch_fsmap learned FSMap version 91 2024-09-11T00:28:31.444 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:31.435+0000 7f5f94805740 1 -- 172.21.15.12:0/2285737129 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=91,monmap=4+}) -- 0x563eda20e620 con 0x563eda4d6ce0 2024-09-11T00:28:31.444 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:31.435+0000 7f5f78c00640 1 -- 172.21.15.12:0/2285737129 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5f84004050 con 0x563eda4d6ce0 2024-09-11T00:28:31.444 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:31.436+0000 7f5f78c00640 1 -- 172.21.15.12:0/2285737129 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 91) ==== 8418+0+0 (secure 0 0 0) 0x7f5f8400f570 con 0x563eda4d6ce0 2024-09-11T00:28:31.444 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:31.436+0000 7f5f94805740 10 client.17823 fetch_fsmap finished waiting for FSMap version 91 2024-09-11T00:28:31.444 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:31.436+0000 7f5f94805740 10 client.17823 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T00:28:31.444 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:31.436+0000 7f5f94805740 20 client.17823 populate_metadata read hostname 'smithi012' 2024-09-11T00:28:31.444 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:31.436+0000 7f5f94805740 1 --2- 172.21.15.12:0/2285737129 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x563eda4e6860 0x563eda4e8c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:28:31.445 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:31.436+0000 7f5f94805740 4 client.17823 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T00:28:31.445 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:31.436+0000 7f5f94805740 1 -- 172.21.15.12:0/2285737129 --> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563eda446600 con 0x563eda4e6860 2024-09-11T00:28:31.445 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:31.436+0000 7f5f8aa00640 1 --2- 172.21.15.12:0/2285737129 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x563eda4e6860 0x563eda4e8c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:28:31.445 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:31.437+0000 7f5f8aa00640 1 --2- 172.21.15.12:0/2285737129 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x563eda4e6860 0x563eda4e8c80 crc :-1 s=READY pgs=297 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6cdee37406d3d373 server_cookie=24612cb7bdf97c28 in_seq=0 out_seq=0 2024-09-11T00:28:31.445 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:31.437+0000 7f5f78c00640 10 client.17823 ms_handle_connect on v2:172.21.15.165:6838/501312984 2024-09-11T00:28:31.445 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:31.438+0000 7f5f78c00640 1 -- 172.21.15.12:0/2285737129 <== mds.0 v2:172.21.15.165:6838/501312984 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x563eda446600 con 0x563eda4e6860 2024-09-11T00:28:31.445 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:31.438+0000 7f5f78c00640 10 client.17823 handle_command_reply: tid=0 2024-09-11T00:28:31.445 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:31.438+0000 7f5f94805740 1 client.17823 shutdown 2024-09-11T00:28:31.445 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:31.438+0000 7f5f94805740 1 -- 172.21.15.12:0/2285737129 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563eda4d6ce0 msgr2=0x563eda4dc390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:28:31.445 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:31.438+0000 7f5f94805740 1 --2- 172.21.15.12:0/2285737129 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563eda4d6ce0 0x563eda4dc390 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f5f84011dd0 tx=0x7f5f840044b0 comp rx=0 tx=0).stop 2024-09-11T00:28:31.446 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:31.438+0000 7f5f94805740 1 -- 172.21.15.12:0/2285737129 shutdown_connections 2024-09-11T00:28:31.446 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:31.438+0000 7f5f94805740 1 --2- 172.21.15.12:0/2285737129 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x563eda4e6860 0x563eda4e8c80 crc :-1 s=READY pgs=297 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:31.446 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:31.438+0000 7f5f94805740 1 --2- 172.21.15.12:0/2285737129 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563eda4d6ce0 0x563eda4dc390 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:31.446 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:31.438+0000 7f5f94805740 1 --2- 172.21.15.12:0/2285737129 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563eda4d42a0 0x563eda4db7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:31.446 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:31.438+0000 7f5f78c00640 0 client.17823 ms_handle_reset on v2:172.21.15.165:6838/501312984 2024-09-11T00:28:31.446 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:31.438+0000 7f5f94805740 1 --2- 172.21.15.12:0/2285737129 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563eda4d1970 0x563eda4db120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:31.446 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:31.438+0000 7f5f94805740 1 -- 172.21.15.12:0/2285737129 >> 172.21.15.12:0/2285737129 conn(0x563eda4ccac0 msgr2=0x563eda4cbbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:28:31.446 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:31.438+0000 7f5f94805740 1 -- 172.21.15.12:0/2285737129 shutdown_connections 2024-09-11T00:28:31.446 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:31.438+0000 7f5f94805740 1 -- 172.21.15.12:0/2285737129 wait complete. 2024-09-11T00:28:31.446 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:31.438+0000 7f5f94805740 20 client.17823 trim_cache size 0 max 16384 2024-09-11T00:28:31.458 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:31.454+0000 7f5f94805740 1 Processor -- start 2024-09-11T00:28:31.458 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:31.454+0000 7f5f94805740 1 -- start start 2024-09-11T00:28:31.458 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:31.455+0000 7f5f94805740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563eda4d1970 0x563eda4c7db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:28:31.458 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:31.455+0000 7f5f94805740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563eda4d42a0 0x563eda4cc3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:28:31.459 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:31.455+0000 7f5f94805740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563eda4d6ce0 0x563eda4d7130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:28:31.459 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:31.455+0000 7f5f94805740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f5f8c1ad810 con 0x563eda4d1970 2024-09-11T00:28:31.459 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:31.455+0000 7f5f94805740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f5f8c1af4a0 con 0x563eda4d42a0 2024-09-11T00:28:31.459 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:31.455+0000 7f5f94805740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f5f8c1adf10 con 0x563eda4d6ce0 2024-09-11T00:28:31.459 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:31.455+0000 7f5f8aa00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563eda4d1970 0x563eda4c7db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:28:31.459 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:31.455+0000 7f5f8aa00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563eda4d1970 0x563eda4c7db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:50978/0 (socket says 172.21.15.12:50978) 2024-09-11T00:28:31.459 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:31.455+0000 7f5f8aa00640 1 -- 172.21.15.12:0/1452224025 learned_addr learned my addr 172.21.15.12:0/1452224025 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:28:31.459 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:31.455+0000 7f5f8b400640 1 --2- 172.21.15.12:0/1452224025 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563eda4d6ce0 0x563eda4d7130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:28:31.459 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:31.455+0000 7f5f83e00640 1 --2- 172.21.15.12:0/1452224025 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563eda4d42a0 0x563eda4cc3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:28:31.459 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:31.455+0000 7f5f8aa00640 1 -- 172.21.15.12:0/1452224025 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563eda4d6ce0 msgr2=0x563eda4d7130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:28:31.459 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:31.455+0000 7f5f8aa00640 1 --2- 172.21.15.12:0/1452224025 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563eda4d6ce0 0x563eda4d7130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:31.459 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:31.455+0000 7f5f8aa00640 1 -- 172.21.15.12:0/1452224025 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563eda4d42a0 msgr2=0x563eda4cc3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:28:31.460 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:31.455+0000 7f5f8aa00640 1 --2- 172.21.15.12:0/1452224025 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563eda4d42a0 0x563eda4cc3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:31.460 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:31.455+0000 7f5f8aa00640 1 -- 172.21.15.12:0/1452224025 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563eda4d40d0 con 0x563eda4d1970 2024-09-11T00:28:31.460 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:31.455+0000 7f5f8b400640 1 --2- 172.21.15.12:0/1452224025 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563eda4d6ce0 0x563eda4d7130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T00:28:31.460 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:31.455+0000 7f5f83e00640 1 --2- 172.21.15.12:0/1452224025 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563eda4d42a0 0x563eda4cc3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T00:28:31.460 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:31.456+0000 7f5f8aa00640 1 --2- 172.21.15.12:0/1452224025 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563eda4d1970 0x563eda4c7db0 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f5f7c031a90 tx=0x7f5f7c040000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be7233d8fd8dd844 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:28:31.460 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:31.456+0000 7f5f7b400640 1 -- 172.21.15.12:0/1452224025 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f7c035040 con 0x563eda4d1970 2024-09-11T00:28:31.460 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:31.456+0000 7f5f7b400640 1 -- 172.21.15.12:0/1452224025 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5f7c004440 con 0x563eda4d1970 2024-09-11T00:28:31.460 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:31.456+0000 7f5f7b400640 1 -- 172.21.15.12:0/1452224025 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f7c03ddf0 con 0x563eda4d1970 2024-09-11T00:28:31.463 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:31.456+0000 7f5f94805740 1 -- 172.21.15.12:0/1452224025 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563eda4d1970 msgr2=0x563eda4c7db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:28:31.463 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:31.456+0000 7f5f94805740 1 --2- 172.21.15.12:0/1452224025 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563eda4d1970 0x563eda4c7db0 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f5f7c031a90 tx=0x7f5f7c040000 comp rx=0 tx=0).stop 2024-09-11T00:28:31.464 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:31.457+0000 7f5f94805740 1 -- 172.21.15.12:0/1452224025 shutdown_connections 2024-09-11T00:28:31.464 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:31.457+0000 7f5f94805740 1 --2- 172.21.15.12:0/1452224025 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563eda4d6ce0 0x563eda4d7130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:31.464 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:31.457+0000 7f5f94805740 1 --2- 172.21.15.12:0/1452224025 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563eda4d42a0 0x563eda4cc3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:31.464 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:31.457+0000 7f5f94805740 1 --2- 172.21.15.12:0/1452224025 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563eda4d1970 0x563eda4c7db0 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:31.464 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:31.457+0000 7f5f94805740 1 -- 172.21.15.12:0/1452224025 >> 172.21.15.12:0/1452224025 conn(0x563eda4ccac0 msgr2=0x563eda4cd4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:28:31.464 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:31.457+0000 7f5f94805740 1 -- 172.21.15.12:0/1452224025 shutdown_connections 2024-09-11T00:28:31.464 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:31.457+0000 7f5f94805740 1 -- 172.21.15.12:0/1452224025 wait complete. 2024-09-11T00:28:31.464 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:31.457+0000 7f5f94805740 1 -- start start 2024-09-11T00:28:31.464 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:31.457+0000 7f5f94805740 1 Processor -- start 2024-09-11T00:28:31.464 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:31.458+0000 7f5f94805740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563eda4d1970 0x563eda4e2d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:28:31.464 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:31.458+0000 7f5f94805740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563eda4d42a0 0x563eda4e1aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:28:31.464 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:31.458+0000 7f5f94805740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563eda4d6ce0 0x563eda4e2190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:28:31.464 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:31.458+0000 7f5f94805740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f5f8c1aedd0 con 0x563eda4d6ce0 2024-09-11T00:28:31.465 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:31.458+0000 7f5f94805740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f5f8c1ae6a0 con 0x563eda4d42a0 2024-09-11T00:28:31.465 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:31.458+0000 7f5f94805740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f5f8c1ab7f0 con 0x563eda4d1970 2024-09-11T00:28:31.465 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:31.458+0000 7f5f94805740 10 client.0 _ll_register_callbacks cb 0x7f5f840093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:28:31.465 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:31.458+0000 7f5f8b400640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563eda4d6ce0 0x563eda4e2190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:28:31.465 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:31.458+0000 7f5f8b400640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563eda4d6ce0 0x563eda4e2190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:50986/0 (socket says 172.21.15.12:50986) 2024-09-11T00:28:31.465 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:31.458+0000 7f5f8b400640 1 -- 172.21.15.12:0/1569850236 learned_addr learned my addr 172.21.15.12:0/1569850236 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:28:31.465 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:31.458+0000 7f5f83e00640 1 --2- 172.21.15.12:0/1569850236 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563eda4d42a0 0x563eda4e1aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:28:31.465 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:31.458+0000 7f5f8aa00640 1 --2- 172.21.15.12:0/1569850236 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563eda4d1970 0x563eda4e2d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:28:31.465 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:31.458+0000 7f5f8b400640 1 -- 172.21.15.12:0/1569850236 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563eda4d1970 msgr2=0x563eda4e2d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:28:31.465 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:31.458+0000 7f5f8b400640 1 --2- 172.21.15.12:0/1569850236 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563eda4d1970 0x563eda4e2d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:31.465 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:31.458+0000 7f5f8b400640 1 -- 172.21.15.12:0/1569850236 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563eda4d42a0 msgr2=0x563eda4e1aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:28:31.465 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:31.458+0000 7f5f8b400640 1 --2- 172.21.15.12:0/1569850236 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563eda4d42a0 0x563eda4e1aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:31.465 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:31.458+0000 7f5f8b400640 1 -- 172.21.15.12:0/1569850236 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563eda4e0580 con 0x563eda4d6ce0 2024-09-11T00:28:31.466 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:31.459+0000 7f5f8b400640 1 --2- 172.21.15.12:0/1569850236 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563eda4d6ce0 0x563eda4e2190 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x563eda4c7280 tx=0x7f5f840045a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c70dbe7192776a8 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:28:31.466 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:31.459+0000 7f5f78c00640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-09-11T00:28:31.466 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:31.459+0000 7f5f78c00640 1 -- 172.21.15.12:0/1569850236 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f8403d060 con 0x563eda4d6ce0 2024-09-11T00:28:31.466 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:31.459+0000 7f5f78c00640 1 -- 172.21.15.12:0/1569850236 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 9208+0+0 (secure 0 0 0) 0x7f5f84009a50 con 0x563eda4d6ce0 2024-09-11T00:28:31.466 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:31.459+0000 7f5f94805740 1 -- 172.21.15.12:0/1569850236 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5f7c004440 con 0x563eda4d6ce0 2024-09-11T00:28:31.466 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:31.459+0000 7f5f78c00640 1 -- 172.21.15.12:0/1569850236 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=91) ==== 24+0+0 (secure 0 0 0) 0x7f5f84044050 con 0x563eda4d6ce0 2024-09-11T00:28:31.466 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:31.459+0000 7f5f94805740 10 client.17829 fetch_fsmap learned FSMap version 91 2024-09-11T00:28:31.466 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:31.459+0000 7f5f94805740 1 -- 172.21.15.12:0/1569850236 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=91,monmap=4+}) -- 0x563eda5539a0 con 0x563eda4d6ce0 2024-09-11T00:28:31.466 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:31.459+0000 7f5f78c00640 1 -- 172.21.15.12:0/1569850236 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5f7c004440 con 0x563eda4d6ce0 2024-09-11T00:28:31.466 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:31.460+0000 7f5f78c00640 1 -- 172.21.15.12:0/1569850236 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 91) ==== 8418+0+0 (secure 0 0 0) 0x7f5f8400fbc0 con 0x563eda4d6ce0 2024-09-11T00:28:31.466 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:31.460+0000 7f5f94805740 10 client.17829 fetch_fsmap finished waiting for FSMap version 91 2024-09-11T00:28:31.466 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:31.460+0000 7f5f94805740 10 client.17829 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T00:28:31.467 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:31.460+0000 7f5f94805740 20 client.17829 populate_metadata read hostname 'smithi012' 2024-09-11T00:28:31.467 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:31.460+0000 7f5f94805740 1 --2- 172.21.15.12:0/1569850236 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x563eda5542f0 0x563eda556710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:28:31.467 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:31.460+0000 7f5f94805740 4 client.17829 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T00:28:31.467 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:31.460+0000 7f5f94805740 1 -- 172.21.15.12:0/1569850236 --> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] -- command(tid 0: {"prefix": "flush journal"}) -- 0x563eda4420b0 con 0x563eda5542f0 2024-09-11T00:28:31.467 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:31.460+0000 7f5f8aa00640 1 --2- 172.21.15.12:0/1569850236 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x563eda5542f0 0x563eda556710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:28:31.467 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:31.461+0000 7f5f8aa00640 1 --2- 172.21.15.12:0/1569850236 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x563eda5542f0 0x563eda556710 crc :-1 s=READY pgs=298 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ec8861a729687408 server_cookie=e3cb73f5f52a22e8 in_seq=0 out_seq=0 2024-09-11T00:28:31.467 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:31.461+0000 7f5f78c00640 10 client.17829 ms_handle_connect on v2:172.21.15.165:6838/501312984 2024-09-11T00:28:31.506 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:31.502+0000 7f5f78c00640 1 -- 172.21.15.12:0/1569850236 <== mds.0 v2:172.21.15.165:6838/501312984 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x563eda4420b0 con 0x563eda5542f0 2024-09-11T00:28:31.506 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:31.502+0000 7f5f78c00640 10 client.17829 handle_command_reply: tid=0 2024-09-11T00:28:31.506 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:31.502+0000 7f5f94805740 1 client.17829 shutdown 2024-09-11T00:28:31.506 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:31.503+0000 7f5f94805740 1 -- 172.21.15.12:0/1569850236 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563eda4d6ce0 msgr2=0x563eda4e2190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:28:31.506 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:31.503+0000 7f5f94805740 1 --2- 172.21.15.12:0/1569850236 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563eda4d6ce0 0x563eda4e2190 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x563eda4c7280 tx=0x7f5f840045a0 comp rx=0 tx=0).stop 2024-09-11T00:28:31.506 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:31.503+0000 7f5f94805740 1 -- 172.21.15.12:0/1569850236 shutdown_connections 2024-09-11T00:28:31.506 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:31.503+0000 7f5f94805740 1 --2- 172.21.15.12:0/1569850236 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x563eda5542f0 0x563eda556710 crc :-1 s=READY pgs=298 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:31.507 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:31.503+0000 7f5f94805740 1 --2- 172.21.15.12:0/1569850236 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563eda4d6ce0 0x563eda4e2190 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:31.507 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:31.503+0000 7f5f94805740 1 --2- 172.21.15.12:0/1569850236 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563eda4d42a0 0x563eda4e1aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:31.507 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:31.503+0000 7f5f94805740 1 --2- 172.21.15.12:0/1569850236 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563eda4d1970 0x563eda4e2d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:31.507 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:31.503+0000 7f5f78c00640 0 client.17829 ms_handle_reset on v2:172.21.15.165:6838/501312984 2024-09-11T00:28:31.507 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:31.503+0000 7f5f94805740 1 -- 172.21.15.12:0/1569850236 >> 172.21.15.12:0/1569850236 conn(0x563eda4ccac0 msgr2=0x563eda4d0ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:28:31.507 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:31.503+0000 7f5f94805740 1 -- 172.21.15.12:0/1569850236 shutdown_connections 2024-09-11T00:28:31.507 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:31.503+0000 7f5f94805740 1 -- 172.21.15.12:0/1569850236 wait complete. 2024-09-11T00:28:31.507 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:31.503+0000 7f5f94805740 20 client.17829 trim_cache size 0 max 16384 2024-09-11T00:28:31.507 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:{ 2024-09-11T00:28:31.508 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "message": "", 2024-09-11T00:28:31.508 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "return_code": 0 2024-09-11T00:28:31.508 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:} 2024-09-11T00:28:31.508 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:31.504+0000 7f5f8be00640 1 -- 172.21.15.12:0/1845328880 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f5f700b51f0 msgr2=0x7f5f700b76b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:28:31.508 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:31.504+0000 7f5f8be00640 1 --2- 172.21.15.12:0/1845328880 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f5f700b51f0 0x7f5f700b76b0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f5f7c006010 tx=0x7f5f7c0314a0 comp rx=0 tx=0).stop 2024-09-11T00:28:31.509 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:31.504+0000 7f5f8be00640 1 -- 172.21.15.12:0/1845328880 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5f8c105b00 msgr2=0x7f5f8c19b420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:28:31.509 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:31.504+0000 7f5f8be00640 1 --2- 172.21.15.12:0/1845328880 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5f8c105b00 0x7f5f8c19b420 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f5f7400e9f0 tx=0x7f5f7400eeb0 comp rx=0 tx=0).stop 2024-09-11T00:28:31.509 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:31.504+0000 7f5f8be00640 1 -- 172.21.15.12:0/1845328880 shutdown_connections 2024-09-11T00:28:31.509 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:31.504+0000 7f5f8be00640 1 --2- 172.21.15.12:0/1845328880 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f5f700b51f0 0x7f5f700b76b0 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:31.509 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:31.504+0000 7f5f8be00640 1 --2- 172.21.15.12:0/1845328880 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5f8c1064c0 0x7f5f8c19c050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:31.509 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:31.504+0000 7f5f8be00640 1 --2- 172.21.15.12:0/1845328880 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5f8c105b00 0x7f5f8c19b420 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:31.509 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:31.504+0000 7f5f8be00640 1 --2- 172.21.15.12:0/1845328880 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5f8c1049b0 0x7f5f8c19add0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:31.509 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:31.504+0000 7f5f8be00640 1 -- 172.21.15.12:0/1845328880 >> 172.21.15.12:0/1845328880 conn(0x7f5f8c100270 msgr2=0x7f5f8c108ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:28:31.509 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:31.504+0000 7f5f8be00640 1 -- 172.21.15.12:0/1845328880 shutdown_connections 2024-09-11T00:28:31.509 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:31.504+0000 7f5f8be00640 1 -- 172.21.15.12:0/1845328880 wait complete. 2024-09-11T00:28:32.033 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:28:31 smithi165 ceph-mon[35356]: pgmap v1043: 129 pgs: 129 active+clean; 685 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 29 KiB/s wr, 43 op/s 2024-09-11T00:28:32.063 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:28:31 smithi012 ceph-mon[30333]: pgmap v1043: 129 pgs: 129 active+clean; 685 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 29 KiB/s wr, 43 op/s 2024-09-11T00:28:32.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:28:31 smithi178 ceph-mon[35919]: pgmap v1043: 129 pgs: 129 active+clean; 685 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 29 KiB/s wr, 43 op/s 2024-09-11T00:28:32.252 INFO:tasks.workunit.client.0.smithi012.stdout: 10 135552 14.23 MB/sec execute 388 sec latency 20189.241 ms 2024-09-11T00:28:33.252 INFO:tasks.workunit.client.0.smithi012.stdout: 10 135552 14.19 MB/sec execute 389 sec latency 21189.349 ms 2024-09-11T00:28:34.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:28:33 smithi165 ceph-mon[35356]: pgmap v1044: 129 pgs: 129 active+clean; 685 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 36 KiB/s wr, 31 op/s 2024-09-11T00:28:34.033 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:28:33 smithi165 ceph-mon[35356]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T00:28:34.063 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:28:33 smithi012 ceph-mon[30333]: pgmap v1044: 129 pgs: 129 active+clean; 685 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 36 KiB/s wr, 31 op/s 2024-09-11T00:28:34.063 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:28:33 smithi012 ceph-mon[30333]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T00:28:34.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:28:33 smithi178 ceph-mon[35919]: pgmap v1044: 129 pgs: 129 active+clean; 685 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 36 KiB/s wr, 31 op/s 2024-09-11T00:28:34.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:28:33 smithi178 ceph-mon[35919]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T00:28:34.252 INFO:tasks.workunit.client.0.smithi012.stdout: 10 135552 14.15 MB/sec execute 390 sec latency 22189.469 ms 2024-09-11T00:28:35.196 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid ff3ad74a-6fcf-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T00:28:35.252 INFO:tasks.workunit.client.0.smithi012.stdout: 10 135552 14.12 MB/sec execute 391 sec latency 23189.561 ms 2024-09-11T00:28:35.751 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:28:36.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:28:35 smithi165 ceph-mon[35356]: pgmap v1045: 129 pgs: 129 active+clean; 685 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 18 KiB/s wr, 16 op/s 2024-09-11T00:28:36.063 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:28:35 smithi012 ceph-mon[30333]: pgmap v1045: 129 pgs: 129 active+clean; 685 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 18 KiB/s wr, 16 op/s 2024-09-11T00:28:36.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:28:35 smithi178 ceph-mon[35919]: pgmap v1045: 129 pgs: 129 active+clean; 685 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 18 KiB/s wr, 16 op/s 2024-09-11T00:28:36.252 INFO:tasks.workunit.client.0.smithi012.stdout: 10 135552 14.08 MB/sec execute 392 sec latency 24189.625 ms 2024-09-11T00:28:36.300 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:36.297+0000 7fd587200640 1 -- 172.21.15.12:0/364116614 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd58010b180 msgr2=0x7fd580111980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:28:36.300 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:36.297+0000 7fd587200640 1 --2- 172.21.15.12:0/364116614 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd58010b180 0x7fd580111980 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7fd57c009ca0 tx=0x7fd57c02a8c0 comp rx=0 tx=0).stop 2024-09-11T00:28:36.300 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:36.297+0000 7fd587200640 1 -- 172.21.15.12:0/364116614 shutdown_connections 2024-09-11T00:28:36.300 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:36.297+0000 7fd587200640 1 --2- 172.21.15.12:0/364116614 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd58010b180 0x7fd580111980 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:36.300 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:36.297+0000 7fd587200640 1 --2- 172.21.15.12:0/364116614 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd58010a800 0x7fd58010ac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:36.300 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:36.297+0000 7fd587200640 1 --2- 172.21.15.12:0/364116614 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd580104df0 0x7fd5801051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:36.301 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:36.297+0000 7fd587200640 1 -- 172.21.15.12:0/364116614 >> 172.21.15.12:0/364116614 conn(0x7fd580100840 msgr2=0x7fd580102c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:28:36.301 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:36.297+0000 7fd587200640 1 -- 172.21.15.12:0/364116614 shutdown_connections 2024-09-11T00:28:36.301 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:36.297+0000 7fd587200640 1 -- 172.21.15.12:0/364116614 wait complete. 2024-09-11T00:28:36.302 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:36.298+0000 7fd587200640 1 Processor -- start 2024-09-11T00:28:36.302 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:36.298+0000 7fd587200640 1 -- start start 2024-09-11T00:28:36.302 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:36.298+0000 7fd587200640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd580104df0 0x7fd58019b1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:28:36.302 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:36.299+0000 7fd587200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd58010a800 0x7fd58019b6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:28:36.302 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:36.299+0000 7fd587200640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd58010b180 0x7fd5801a0a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:28:36.302 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:36.299+0000 7fd587200640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fd580114e40 con 0x7fd580104df0 2024-09-11T00:28:36.303 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:36.299+0000 7fd587200640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fd580114cc0 con 0x7fd58010a800 2024-09-11T00:28:36.303 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:36.299+0000 7fd587200640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fd580114fc0 con 0x7fd58010b180 2024-09-11T00:28:36.303 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:36.299+0000 7fd585e00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd580104df0 0x7fd58019b1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:28:36.303 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:36.299+0000 7fd585e00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd580104df0 0x7fd58019b1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:51016/0 (socket says 172.21.15.12:51016) 2024-09-11T00:28:36.303 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:36.299+0000 7fd585e00640 1 -- 172.21.15.12:0/2724853860 learned_addr learned my addr 172.21.15.12:0/2724853860 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:28:36.303 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:36.299+0000 7fd586800640 1 --2- 172.21.15.12:0/2724853860 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd58010b180 0x7fd5801a0a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:28:36.303 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:36.299+0000 7fd585400640 1 --2- 172.21.15.12:0/2724853860 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd58010a800 0x7fd58019b6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:28:36.303 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:36.300+0000 7fd585e00640 1 -- 172.21.15.12:0/2724853860 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd58010b180 msgr2=0x7fd5801a0a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:28:36.303 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:36.300+0000 7fd585e00640 1 --2- 172.21.15.12:0/2724853860 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd58010b180 0x7fd5801a0a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:36.303 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:36.300+0000 7fd585e00640 1 -- 172.21.15.12:0/2724853860 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd58010a800 msgr2=0x7fd58019b6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:28:36.303 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:36.300+0000 7fd585e00640 1 --2- 172.21.15.12:0/2724853860 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd58010a800 0x7fd58019b6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:36.303 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:36.300+0000 7fd585e00640 1 -- 172.21.15.12:0/2724853860 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5801a1260 con 0x7fd580104df0 2024-09-11T00:28:36.304 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:36.300+0000 7fd586800640 1 --2- 172.21.15.12:0/2724853860 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd58010b180 0x7fd5801a0a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:28:36.304 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:36.300+0000 7fd585400640 1 --2- 172.21.15.12:0/2724853860 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd58010a800 0x7fd58019b6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T00:28:36.304 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:36.300+0000 7fd585e00640 1 --2- 172.21.15.12:0/2724853860 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd580104df0 0x7fd58019b1b0 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7fd57400e9f0 tx=0x7fd57400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:28:36.304 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:36.300+0000 7fd57aa00640 1 -- 172.21.15.12:0/2724853860 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd57400cd80 con 0x7fd580104df0 2024-09-11T00:28:36.305 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:36.300+0000 7fd57aa00640 1 -- 172.21.15.12:0/2724853860 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd5740054d0 con 0x7fd580104df0 2024-09-11T00:28:36.305 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:36.300+0000 7fd587200640 1 -- 172.21.15.12:0/2724853860 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd5801a1550 con 0x7fd580104df0 2024-09-11T00:28:36.305 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:36.300+0000 7fd57aa00640 1 -- 172.21.15.12:0/2724853860 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd574010690 con 0x7fd580104df0 2024-09-11T00:28:36.305 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:36.300+0000 7fd587200640 1 -- 172.21.15.12:0/2724853860 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5801a7c10 con 0x7fd580104df0 2024-09-11T00:28:36.305 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:36.302+0000 7fd587200640 1 -- 172.21.15.12:0/2724853860 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd5801051d0 con 0x7fd580104df0 2024-09-11T00:28:36.306 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:36.304+0000 7fd57aa00640 1 -- 172.21.15.12:0/2724853860 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7fd574004e90 con 0x7fd580104df0 2024-09-11T00:28:36.308 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:36.305+0000 7fd57aa00640 1 --2- 172.21.15.12:0/2724853860 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fd5680b5390 0x7fd5680b7850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:28:36.308 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:36.305+0000 7fd57aa00640 1 -- 172.21.15.12:0/2724853860 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9208+0+0 (secure 0 0 0) 0x7fd5740e6460 con 0x7fd580104df0 2024-09-11T00:28:36.308 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:36.305+0000 7fd585400640 1 --2- 172.21.15.12:0/2724853860 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fd5680b5390 0x7fd5680b7850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:28:36.308 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:36.306+0000 7fd585400640 1 --2- 172.21.15.12:0/2724853860 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fd5680b5390 0x7fd5680b7850 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fd580102ec0 tx=0x7fd570005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:28:36.311 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:36.309+0000 7fd57aa00640 1 -- 172.21.15.12:0/2724853860 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7fd5740acf50 con 0x7fd580104df0 2024-09-11T00:28:36.548 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:36.545+0000 7fd587200640 1 -- 172.21.15.12:0/2724853860 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd5801a7fc0 con 0x7fd580104df0 2024-09-11T00:28:36.548 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:36.546+0000 7fd57aa00640 1 -- 172.21.15.12:0/2724853860 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 92 v92) ==== 94+0+11985 (secure 0 0 0) 0x7fd5740ac850 con 0x7fd580104df0 2024-09-11T00:28:36.549 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-11T00:28:36.549 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":92,"btime":"2024-09-11T00:28:32:748656+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14577,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6835/447171591","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":447171591},{"type":"v1","addr":"172.21.15.12:6835","nonce":447171591}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14589,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/901486040","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":901486040},{"type":"v1","addr":"172.21.15.12:6837","nonce":901486040}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":92,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T00:04:04.739881+0000","modified":"2024-09-11T00:28:31.992740+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24674,"mds_1":24643,"mds_2":24638,"mds_3":24637,"mds_4":24614},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14631":{"gid":14631,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.12:6841/3069919437","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6840","nonce":3069919437},{"type":"v1","addr":"172.21.15.12:6841","nonce":3069919437}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24562":{"gid":24562,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6833/1141636112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":1141636112},{"type":"v1","addr":"172.21.15.178:6833","nonce":1141636112}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24581":{"gid":24581,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6833/3111980970","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6832","nonce":3111980970},{"type":"v1","addr":"172.21.15.165:6833","nonce":3111980970}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24607":{"gid":24607,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6835/4084868287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":4084868287},{"type":"v1","addr":"172.21.15.178:6835","nonce":4084868287}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24614":{"gid":24614,"name":"e","rank":4,"incarnation":26,"state":"up:active","state_seq":21,"addr":"172.21.15.165:6835/2425395274","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":2425395274},{"type":"v1","addr":"172.21.15.165:6835","nonce":2425395274}]},"join_fscid":-1,"export_targets":[0],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24637":{"gid":24637,"name":"i","rank":3,"incarnation":24,"state":"up:active","state_seq":18,"addr":"172.21.15.178:6839/70628097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6838","nonce":70628097},{"type":"v1","addr":"172.21.15.178:6839","nonce":70628097}]},"join_fscid":-1,"export_targets":[2],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24638":{"gid":24638,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":18,"addr":"172.21.15.165:6837/6215088","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6836","nonce":6215088},{"type":"v1","addr":"172.21.15.165:6837","nonce":6215088}]},"join_fscid":-1,"export_targets":[0,1,3,4],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24643":{"gid":24643,"name":"g","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.12:6839/3124093009","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6838","nonce":3124093009},{"type":"v1","addr":"172.21.15.12:6839","nonce":3124093009}]},"join_fscid":-1,"export_targets":[0,2,3,4],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24674":{"gid":24674,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.165:6839/501312984","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6838","nonce":501312984},{"type":"v1","addr":"172.21.15.165:6839","nonce":501312984}]},"join_fscid":-1,"export_targets":[1],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_27751":{"gid":27751,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6837/1436600611","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":1436600611},{"type":"v1","addr":"172.21.15.178:6837","nonce":1436600611}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24674,"qdb_cluster":[24674,24643,24638,24637,24614]},"id":1}]} 2024-09-11T00:28:36.549 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 92 2024-09-11T00:28:36.552 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:36.550+0000 7fd587200640 1 -- 172.21.15.12:0/2724853860 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fd5680b5390 msgr2=0x7fd5680b7850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:28:36.552 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:36.550+0000 7fd587200640 1 --2- 172.21.15.12:0/2724853860 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fd5680b5390 0x7fd5680b7850 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fd580102ec0 tx=0x7fd570005f90 comp rx=0 tx=0).stop 2024-09-11T00:28:36.553 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:36.550+0000 7fd587200640 1 -- 172.21.15.12:0/2724853860 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd580104df0 msgr2=0x7fd58019b1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:28:36.553 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:36.550+0000 7fd587200640 1 --2- 172.21.15.12:0/2724853860 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd580104df0 0x7fd58019b1b0 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7fd57400e9f0 tx=0x7fd57400edf0 comp rx=0 tx=0).stop 2024-09-11T00:28:36.553 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:36.550+0000 7fd587200640 1 -- 172.21.15.12:0/2724853860 shutdown_connections 2024-09-11T00:28:36.553 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:36.550+0000 7fd587200640 1 --2- 172.21.15.12:0/2724853860 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fd5680b5390 0x7fd5680b7850 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:36.553 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:36.550+0000 7fd587200640 1 --2- 172.21.15.12:0/2724853860 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd58010b180 0x7fd5801a0a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:36.553 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:36.550+0000 7fd587200640 1 --2- 172.21.15.12:0/2724853860 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd58010a800 0x7fd58019b6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:36.553 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:36.550+0000 7fd587200640 1 --2- 172.21.15.12:0/2724853860 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd580104df0 0x7fd58019b1b0 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:36.553 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:36.550+0000 7fd587200640 1 -- 172.21.15.12:0/2724853860 >> 172.21.15.12:0/2724853860 conn(0x7fd580100840 msgr2=0x7fd580103cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:28:36.553 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:36.550+0000 7fd587200640 1 -- 172.21.15.12:0/2724853860 shutdown_connections 2024-09-11T00:28:36.553 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:36.550+0000 7fd587200640 1 -- 172.21.15.12:0/2724853860 wait complete. 2024-09-11T00:28:36.813 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:28:36 smithi012 ceph-mon[30333]: from='client.? 172.21.15.12:0/2724853860' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T00:28:37.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:28:36 smithi165 ceph-mon[35356]: from='client.? 172.21.15.12:0/2724853860' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T00:28:37.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:28:36 smithi178 ceph-mon[35919]: from='client.? 172.21.15.12:0/2724853860' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T00:28:37.253 INFO:tasks.workunit.client.0.smithi012.stdout: 10 135552 14.05 MB/sec execute 393 sec latency 25189.687 ms 2024-09-11T00:28:37.280 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24674 quiesce db --set-id 248955d7 --release --await' 2024-09-11T00:28:37.280 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid ff3ad74a-6fcf-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24674 quiesce db --set-id 248955d7 --release --await 2024-09-11T00:28:37.668 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:28:38.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:28:37 smithi165 ceph-mon[35356]: pgmap v1046: 129 pgs: 129 active+clean; 686 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 36 KiB/s wr, 25 op/s 2024-09-11T00:28:38.065 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:28:37 smithi012 ceph-mon[30333]: pgmap v1046: 129 pgs: 129 active+clean; 686 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 36 KiB/s wr, 25 op/s 2024-09-11T00:28:38.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:28:37 smithi178 ceph-mon[35919]: pgmap v1046: 129 pgs: 129 active+clean; 686 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 36 KiB/s wr, 25 op/s 2024-09-11T00:28:38.239 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:38.236+0000 7efee5800640 1 -- 172.21.15.12:0/1115112695 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efee0075540 msgr2=0x7efee0075920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:28:38.239 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:38.236+0000 7efee5800640 1 --2- 172.21.15.12:0/1115112695 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efee0075540 0x7efee0075920 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7efed0009900 tx=0x7efed002a3f0 comp rx=0 tx=0).stop 2024-09-11T00:28:38.239 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:38.237+0000 7efee5800640 1 -- 172.21.15.12:0/1115112695 shutdown_connections 2024-09-11T00:28:38.240 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:38.237+0000 7efee5800640 1 --2- 172.21.15.12:0/1115112695 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efee0113b00 0x7efee0115ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:38.240 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:38.237+0000 7efee5800640 1 --2- 172.21.15.12:0/1115112695 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7efee0075e60 0x7efee01135c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:38.240 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:38.237+0000 7efee5800640 1 --2- 172.21.15.12:0/1115112695 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efee0075540 0x7efee0075920 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:38.240 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:38.237+0000 7efee5800640 1 -- 172.21.15.12:0/1115112695 >> 172.21.15.12:0/1115112695 conn(0x7efee00fe810 msgr2=0x7efee0100c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:28:38.240 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:38.237+0000 7efee5800640 1 -- 172.21.15.12:0/1115112695 shutdown_connections 2024-09-11T00:28:38.240 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:38.237+0000 7efee5800640 1 -- 172.21.15.12:0/1115112695 wait complete. 2024-09-11T00:28:38.241 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:38.238+0000 7efee5800640 1 Processor -- start 2024-09-11T00:28:38.241 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:38.238+0000 7efee5800640 1 -- start start 2024-09-11T00:28:38.241 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:38.238+0000 7efee5800640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efee0075540 0x7efee019f8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:28:38.241 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:38.238+0000 7efee5800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7efee0075e60 0x7efee019fe20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:28:38.241 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:38.238+0000 7efee5800640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efee0113b00 0x7efee01a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:28:38.241 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:38.238+0000 7efee5800640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7efee01193d0 con 0x7efee0075540 2024-09-11T00:28:38.241 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:38.238+0000 7efee5800640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7efee0119250 con 0x7efee0075e60 2024-09-11T00:28:38.242 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:38.238+0000 7efee5800640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7efee0119550 con 0x7efee0113b00 2024-09-11T00:28:38.242 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:38.238+0000 7efedea00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efee0075540 0x7efee019f8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:28:38.242 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:38.238+0000 7efedea00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efee0075540 0x7efee019f8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:51034/0 (socket says 172.21.15.12:51034) 2024-09-11T00:28:38.242 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:38.238+0000 7efedea00640 1 -- 172.21.15.12:0/3134569194 learned_addr learned my addr 172.21.15.12:0/3134569194 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:28:38.242 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:38.238+0000 7efed6000640 1 --2- 172.21.15.12:0/3134569194 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7efee0075e60 0x7efee019fe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:28:38.242 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:38.239+0000 7efedf400640 1 --2- 172.21.15.12:0/3134569194 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efee0113b00 0x7efee01a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:28:38.242 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:38.239+0000 7efedea00640 1 -- 172.21.15.12:0/3134569194 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efee0113b00 msgr2=0x7efee01a51c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:28:38.242 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:38.239+0000 7efedea00640 1 --2- 172.21.15.12:0/3134569194 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efee0113b00 0x7efee01a51c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:38.242 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:38.239+0000 7efedea00640 1 -- 172.21.15.12:0/3134569194 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7efee0075e60 msgr2=0x7efee019fe20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:28:38.242 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:38.239+0000 7efedea00640 1 --2- 172.21.15.12:0/3134569194 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7efee0075e60 0x7efee019fe20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:38.242 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:38.239+0000 7efedea00640 1 -- 172.21.15.12:0/3134569194 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efee01a5990 con 0x7efee0075540 2024-09-11T00:28:38.242 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:38.239+0000 7efed6000640 1 --2- 172.21.15.12:0/3134569194 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7efee0075e60 0x7efee019fe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:28:38.243 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:38.239+0000 7efedea00640 1 --2- 172.21.15.12:0/3134569194 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efee0075540 0x7efee019f8e0 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7efed0005ab0 tx=0x7efed0003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:28:38.243 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:38.239+0000 7efed7e00640 1 -- 172.21.15.12:0/3134569194 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efed003f070 con 0x7efee0075540 2024-09-11T00:28:38.243 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:38.240+0000 7efed7e00640 1 -- 172.21.15.12:0/3134569194 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efed0031b60 con 0x7efee0075540 2024-09-11T00:28:38.243 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:38.240+0000 7efee5800640 1 -- 172.21.15.12:0/3134569194 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efee01a5c20 con 0x7efee0075540 2024-09-11T00:28:38.243 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:38.240+0000 7efed7e00640 1 -- 172.21.15.12:0/3134569194 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efed0035030 con 0x7efee0075540 2024-09-11T00:28:38.243 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:38.240+0000 7efee5800640 1 -- 172.21.15.12:0/3134569194 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efee01ac340 con 0x7efee0075540 2024-09-11T00:28:38.245 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:38.243+0000 7efee8239740 1 Processor -- start 2024-09-11T00:28:38.245 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:38.243+0000 7efee8239740 1 -- start start 2024-09-11T00:28:38.246 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:38.243+0000 7efed7e00640 1 -- 172.21.15.12:0/3134569194 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7efed004c060 con 0x7efee0075540 2024-09-11T00:28:38.246 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:38.243+0000 7efee8239740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bb8eec9ff0 0x55bb8eeca3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:28:38.246 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:38.243+0000 7efee8239740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bb8eeca910 0x55bb8eed6360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:28:38.246 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:38.243+0000 7efee8239740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bb8eed68a0 0x55bb8eed8c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:28:38.246 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:38.243+0000 7efee8239740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55bb8ee41290 con 0x55bb8eed68a0 2024-09-11T00:28:38.246 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:38.243+0000 7efee8239740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55bb8ee306c0 con 0x55bb8eec9ff0 2024-09-11T00:28:38.246 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:38.243+0000 7efee8239740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55bb8ee344d0 con 0x55bb8eeca910 2024-09-11T00:28:38.247 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:38.244+0000 7efedf400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bb8eec9ff0 0x55bb8eeca3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:28:38.247 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:38.244+0000 7efedea00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bb8eeca910 0x55bb8eed6360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:28:38.247 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:38.244+0000 7efed6000640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bb8eed68a0 0x55bb8eed8c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:28:38.247 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:38.244+0000 7efedf400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bb8eec9ff0 0x55bb8eeca3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.12:56188/0 (socket says 172.21.15.12:56188) 2024-09-11T00:28:38.247 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:38.244+0000 7efedf400640 1 -- 172.21.15.12:0/2509723405 learned_addr learned my addr 172.21.15.12:0/2509723405 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:28:38.248 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:38.244+0000 7efedf400640 1 -- 172.21.15.12:0/2509723405 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bb8eeca910 msgr2=0x55bb8eed6360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:28:38.248 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:38.244+0000 7efedf400640 1 --2- 172.21.15.12:0/2509723405 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bb8eeca910 0x55bb8eed6360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:38.248 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:38.244+0000 7efedf400640 1 -- 172.21.15.12:0/2509723405 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bb8eed68a0 msgr2=0x55bb8eed8c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:28:38.248 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:38.244+0000 7efedf400640 1 --2- 172.21.15.12:0/2509723405 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bb8eed68a0 0x55bb8eed8c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:38.248 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:38.244+0000 7efedf400640 1 -- 172.21.15.12:0/2509723405 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bb8edbd460 con 0x55bb8eec9ff0 2024-09-11T00:28:38.248 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:38.244+0000 7efedea00640 1 --2- 172.21.15.12:0/2509723405 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bb8eeca910 0x55bb8eed6360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:28:38.248 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:38.244+0000 7efed6000640 1 --2- 172.21.15.12:0/2509723405 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bb8eed68a0 0x55bb8eed8c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:28:38.248 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:38.245+0000 7efedf400640 1 --2- 172.21.15.12:0/2509723405 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bb8eec9ff0 0x55bb8eeca3d0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7efed8009f90 tx=0x7efed80118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5fe56ba69e68fc3d server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:28:38.248 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:38.245+0000 7efecfe00640 1 -- 172.21.15.12:0/2509723405 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efed8004b20 con 0x55bb8eec9ff0 2024-09-11T00:28:38.248 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:38.245+0000 7efed7e00640 1 --2- 172.21.15.12:0/3134569194 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7efec40b5360 0x7efec40b7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:28:38.248 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:38.245+0000 7efecfe00640 1 -- 172.21.15.12:0/2509723405 <== mon.2 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efed8004cc0 con 0x55bb8eec9ff0 2024-09-11T00:28:38.249 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:38.245+0000 7efecfe00640 1 -- 172.21.15.12:0/2509723405 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efed8005020 con 0x55bb8eec9ff0 2024-09-11T00:28:38.250 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:38.245+0000 7efed7e00640 1 -- 172.21.15.12:0/3134569194 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9208+0+0 (secure 0 0 0) 0x7efed0106280 con 0x7efee0075540 2024-09-11T00:28:38.250 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:38.245+0000 7efedea00640 1 --2- 172.21.15.12:0/3134569194 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7efec40b5360 0x7efec40b7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:28:38.250 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:38.246+0000 7efee8239740 1 -- 172.21.15.12:0/2509723405 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bb8eec9ff0 msgr2=0x55bb8eeca3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:28:38.250 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:38.246+0000 7efee8239740 1 --2- 172.21.15.12:0/2509723405 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bb8eec9ff0 0x55bb8eeca3d0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7efed8009f90 tx=0x7efed80118d0 comp rx=0 tx=0).stop 2024-09-11T00:28:38.250 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:38.246+0000 7efee8239740 1 -- 172.21.15.12:0/2509723405 shutdown_connections 2024-09-11T00:28:38.250 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:38.246+0000 7efee8239740 1 --2- 172.21.15.12:0/2509723405 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bb8eed68a0 0x55bb8eed8c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:38.251 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:38.246+0000 7efee8239740 1 --2- 172.21.15.12:0/2509723405 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bb8eeca910 0x55bb8eed6360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:38.251 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:38.246+0000 7efee8239740 1 --2- 172.21.15.12:0/2509723405 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bb8eec9ff0 0x55bb8eeca3d0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:38.251 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:38.246+0000 7efee8239740 1 -- 172.21.15.12:0/2509723405 >> 172.21.15.12:0/2509723405 conn(0x55bb8eec7e20 msgr2=0x55bb8eec8fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:28:38.251 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:38.246+0000 7efedea00640 1 --2- 172.21.15.12:0/3134569194 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7efec40b5360 0x7efec40b7820 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7efed00c9930 tx=0x7efed0109cc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:28:38.251 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:38.246+0000 7efee8239740 1 -- 172.21.15.12:0/2509723405 shutdown_connections 2024-09-11T00:28:38.251 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:38.246+0000 7efee8239740 1 -- 172.21.15.12:0/2509723405 wait complete. 2024-09-11T00:28:38.251 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:38.247+0000 7efee8239740 1 -- start start 2024-09-11T00:28:38.251 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:38.247+0000 7efee8239740 1 Processor -- start 2024-09-11T00:28:38.251 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:38.247+0000 7efee8239740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bb8eec9ff0 0x55bb8eed6330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:28:38.251 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:38.247+0000 7efee8239740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bb8eeca910 0x55bb8eed37a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:28:38.251 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:38.247+0000 7efee8239740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bb8eed68a0 0x55bb8eed3f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:28:38.251 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:38.247+0000 7efee8239740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55bb8ec91bb0 con 0x55bb8eed68a0 2024-09-11T00:28:38.251 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:38.247+0000 7efee8239740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55bb8edcb6c0 con 0x55bb8eec9ff0 2024-09-11T00:28:38.252 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:38.247+0000 7efee8239740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55bb8eed9550 con 0x55bb8eeca910 2024-09-11T00:28:38.252 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:38.247+0000 7efee8239740 10 client.0 _ll_register_callbacks cb 0x55bb8ede6490 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:28:38.252 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:38.248+0000 7efedf400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bb8eec9ff0 0x55bb8eed6330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:28:38.252 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:38.248+0000 7efedf400640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bb8eed68a0 0x55bb8eed3f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:28:38.252 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:38.248+0000 7efed6000640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bb8eeca910 0x55bb8eed37a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:28:38.252 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:38.248+0000 7efedf400640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bb8eed68a0 0x55bb8eed3f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:51056/0 (socket says 172.21.15.12:51056) 2024-09-11T00:28:38.252 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:38.248+0000 7efedf400640 1 -- 172.21.15.12:0/1693559080 learned_addr learned my addr 172.21.15.12:0/1693559080 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:28:38.253 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:38.248+0000 7efed6000640 1 -- 172.21.15.12:0/1693559080 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bb8eec9ff0 msgr2=0x55bb8eed6330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:28:38.254 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:38.248+0000 7efed6000640 1 --2- 172.21.15.12:0/1693559080 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bb8eec9ff0 0x55bb8eed6330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:38.254 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:38.248+0000 7efed6000640 1 -- 172.21.15.12:0/1693559080 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bb8eed68a0 msgr2=0x55bb8eed3f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:28:38.254 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:38.248+0000 7efed6000640 1 --2- 172.21.15.12:0/1693559080 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bb8eed68a0 0x55bb8eed3f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:38.254 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:38.248+0000 7efed6000640 1 -- 172.21.15.12:0/1693559080 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bb8eb8eb00 con 0x55bb8eeca910 2024-09-11T00:28:38.254 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:38.248+0000 7efed6000640 1 --2- 172.21.15.12:0/1693559080 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bb8eeca910 0x55bb8eed37a0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7efee0116130 tx=0x7efec800f790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=65d8ca42e02c2f6f server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:28:38.254 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:38.248+0000 7efeccc00640 10 client.0 ms_handle_connect on v2:172.21.15.178:3300/0 2024-09-11T00:28:38.254 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:38.249+0000 7efeccc00640 1 -- 172.21.15.12:0/1693559080 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efec80120d0 con 0x55bb8eeca910 2024-09-11T00:28:38.254 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:38.249+0000 7efee8239740 1 -- 172.21.15.12:0/1693559080 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efed8004cc0 con 0x55bb8eeca910 2024-09-11T00:28:38.254 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:38.249+0000 7efeccc00640 1 -- 172.21.15.12:0/1693559080 <== mon.1 v2:172.21.15.178:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 9208+0+0 (secure 0 0 0) 0x7efec8004030 con 0x55bb8eeca910 2024-09-11T00:28:38.254 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:38.249+0000 7efeccc00640 1 -- 172.21.15.12:0/1693559080 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=93) ==== 24+0+0 (secure 0 0 0) 0x7efec8012270 con 0x55bb8eeca910 2024-09-11T00:28:38.254 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:38.249+0000 7efee8239740 10 client.27823 fetch_fsmap learned FSMap version 93 2024-09-11T00:28:38.254 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:38.249+0000 7efee8239740 1 -- 172.21.15.12:0/1693559080 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,fsmap=93,monmap=4+}) -- 0x55bb8ec05960 con 0x55bb8eeca910 2024-09-11T00:28:38.255 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:38.249+0000 7efeccc00640 1 -- 172.21.15.12:0/1693559080 <== mon.1 v2:172.21.15.178:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efed8004cc0 con 0x55bb8eeca910 2024-09-11T00:28:38.255 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:38.250+0000 7efeccc00640 1 -- 172.21.15.12:0/1693559080 <== mon.1 v2:172.21.15.178:3300/0 5 ==== fsmap(e 93) ==== 8410+0+0 (secure 0 0 0) 0x7efec801c460 con 0x55bb8eeca910 2024-09-11T00:28:38.255 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:38.250+0000 7efee8239740 10 client.27823 fetch_fsmap finished waiting for FSMap version 93 2024-09-11T00:28:38.255 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:38.250+0000 7efee8239740 10 client.27823 resolve_mds: validated gid 24674 aka daemon mds.b 2024-09-11T00:28:38.255 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:38.250+0000 7efee8239740 20 client.27823 populate_metadata read hostname 'smithi012' 2024-09-11T00:28:38.255 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:38.250+0000 7efee8239740 1 --2- 172.21.15.12:0/1693559080 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55bb8eee6360 0x55bb8eee8780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:28:38.255 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:38.250+0000 7efee8239740 4 client.27823 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T00:28:38.255 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:38.250+0000 7efee8239740 1 -- 172.21.15.12:0/1693559080 --> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bb8ee431e0 con 0x55bb8eee6360 2024-09-11T00:28:38.255 INFO:tasks.workunit.client.0.smithi012.stdout: 10 135552 14.01 MB/sec execute 394 sec latency 26189.730 ms 2024-09-11T00:28:38.256 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:38.250+0000 7efedf400640 1 --2- 172.21.15.12:0/1693559080 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55bb8eee6360 0x55bb8eee8780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:28:38.257 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:38.251+0000 7efedf400640 1 --2- 172.21.15.12:0/1693559080 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55bb8eee6360 0x55bb8eee8780 crc :-1 s=READY pgs=299 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=305e89825bb23c23 server_cookie=6e4a41a5611063b7 in_seq=0 out_seq=0 2024-09-11T00:28:38.257 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:38.251+0000 7efeccc00640 10 client.27823 ms_handle_connect on v2:172.21.15.165:6838/501312984 2024-09-11T00:28:38.257 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:38.252+0000 7efeccc00640 1 -- 172.21.15.12:0/1693559080 <== mds.0 v2:172.21.15.165:6838/501312984 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55bb8ee431e0 con 0x55bb8eee6360 2024-09-11T00:28:38.257 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:38.252+0000 7efeccc00640 10 client.27823 handle_command_reply: tid=0 2024-09-11T00:28:38.257 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:38.252+0000 7efee8239740 1 client.27823 shutdown 2024-09-11T00:28:38.257 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:38.252+0000 7efee8239740 1 -- 172.21.15.12:0/1693559080 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bb8eeca910 msgr2=0x55bb8eed37a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:28:38.257 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:38.252+0000 7efee8239740 1 --2- 172.21.15.12:0/1693559080 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bb8eeca910 0x55bb8eed37a0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7efee0116130 tx=0x7efec800f790 comp rx=0 tx=0).stop 2024-09-11T00:28:38.257 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:38.252+0000 7efee8239740 1 -- 172.21.15.12:0/1693559080 shutdown_connections 2024-09-11T00:28:38.257 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:38.252+0000 7efee8239740 1 --2- 172.21.15.12:0/1693559080 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55bb8eee6360 0x55bb8eee8780 crc :-1 s=READY pgs=299 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:38.257 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:38.252+0000 7efee8239740 1 --2- 172.21.15.12:0/1693559080 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bb8eed68a0 0x55bb8eed3f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:38.257 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:38.252+0000 7efee8239740 1 --2- 172.21.15.12:0/1693559080 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bb8eeca910 0x55bb8eed37a0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:38.257 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:38.252+0000 7efee8239740 1 --2- 172.21.15.12:0/1693559080 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bb8eec9ff0 0x55bb8eed6330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:38.258 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:38.252+0000 7efeccc00640 0 client.27823 ms_handle_reset on v2:172.21.15.165:6838/501312984 2024-09-11T00:28:38.258 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:38.252+0000 7efee8239740 1 -- 172.21.15.12:0/1693559080 >> 172.21.15.12:0/1693559080 conn(0x55bb8eec7e20 msgr2=0x55bb8eecd9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:28:38.258 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:38.252+0000 7efee8239740 1 -- 172.21.15.12:0/1693559080 shutdown_connections 2024-09-11T00:28:38.258 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:38.252+0000 7efee8239740 1 -- 172.21.15.12:0/1693559080 wait complete. 2024-09-11T00:28:38.258 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:38.252+0000 7efee8239740 20 client.27823 trim_cache size 0 max 16384 2024-09-11T00:28:38.273 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:38.271+0000 7efee8239740 1 Processor -- start 2024-09-11T00:28:38.273 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:38.271+0000 7efee8239740 1 -- start start 2024-09-11T00:28:38.274 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:38.271+0000 7efee8239740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bb8eec9ff0 0x55bb8eec9580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:28:38.274 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:38.271+0000 7efee8239740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bb8eeca910 0x55bb8eeee030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:28:38.274 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:38.271+0000 7efee8239740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bb8eed68a0 0x55bb8eeee570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:28:38.274 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:38.271+0000 7efee8239740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7efee01b6220 con 0x55bb8eeca910 2024-09-11T00:28:38.274 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:38.271+0000 7efee8239740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7efee01b7e50 con 0x55bb8eed68a0 2024-09-11T00:28:38.274 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:38.271+0000 7efee8239740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7efee01b6920 con 0x55bb8eec9ff0 2024-09-11T00:28:38.276 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:38.271+0000 7efed6000640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bb8eeca910 0x55bb8eeee030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:28:38.276 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:38.271+0000 7efed6000640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bb8eeca910 0x55bb8eeee030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:51064/0 (socket says 172.21.15.12:51064) 2024-09-11T00:28:38.276 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:38.271+0000 7efed6000640 1 -- 172.21.15.12:0/2170003879 learned_addr learned my addr 172.21.15.12:0/2170003879 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:28:38.276 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:38.272+0000 7efedf400640 1 --2- 172.21.15.12:0/2170003879 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bb8eec9ff0 0x55bb8eec9580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:28:38.276 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:38.272+0000 7efedf400640 1 --2- 172.21.15.12:0/2170003879 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bb8eed68a0 0x55bb8eeee570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:28:38.276 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:38.272+0000 7efed6000640 1 -- 172.21.15.12:0/2170003879 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bb8eec9ff0 msgr2=0x55bb8eec9580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:28:38.276 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:38.272+0000 7efed6000640 1 --2- 172.21.15.12:0/2170003879 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bb8eec9ff0 0x55bb8eec9580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:38.276 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:38.272+0000 7efed6000640 1 -- 172.21.15.12:0/2170003879 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bb8eed68a0 msgr2=0x55bb8eeee570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:28:38.276 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:38.272+0000 7efed6000640 1 --2- 172.21.15.12:0/2170003879 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bb8eed68a0 0x55bb8eeee570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:38.276 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:38.272+0000 7efed6000640 1 -- 172.21.15.12:0/2170003879 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bb8eec88a0 con 0x55bb8eeca910 2024-09-11T00:28:38.277 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:38.272+0000 7efedf400640 1 --2- 172.21.15.12:0/2170003879 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bb8eed68a0 0x55bb8eeee570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T00:28:38.277 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:38.272+0000 7efed6000640 1 --2- 172.21.15.12:0/2170003879 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bb8eeca910 0x55bb8eeee030 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7efec8012870 tx=0x7efec8009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd87a85e54626a21 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:28:38.277 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:38.272+0000 7efecf400640 1 -- 172.21.15.12:0/2170003879 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efec801c460 con 0x55bb8eeca910 2024-09-11T00:28:38.277 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:38.272+0000 7efecf400640 1 -- 172.21.15.12:0/2170003879 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efec801c600 con 0x55bb8eeca910 2024-09-11T00:28:38.277 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:38.272+0000 7efecf400640 1 -- 172.21.15.12:0/2170003879 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efec8003850 con 0x55bb8eeca910 2024-09-11T00:28:38.277 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:38.273+0000 7efee8239740 1 -- 172.21.15.12:0/2170003879 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bb8eeca910 msgr2=0x55bb8eeee030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:28:38.277 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:38.273+0000 7efee8239740 1 --2- 172.21.15.12:0/2170003879 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bb8eeca910 0x55bb8eeee030 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7efec8012870 tx=0x7efec8009c00 comp rx=0 tx=0).stop 2024-09-11T00:28:38.277 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:38.273+0000 7efee8239740 1 -- 172.21.15.12:0/2170003879 shutdown_connections 2024-09-11T00:28:38.277 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:38.273+0000 7efee8239740 1 --2- 172.21.15.12:0/2170003879 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bb8eed68a0 0x55bb8eeee570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:38.277 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:38.273+0000 7efee8239740 1 --2- 172.21.15.12:0/2170003879 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bb8eeca910 0x55bb8eeee030 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:38.277 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:38.273+0000 7efee8239740 1 --2- 172.21.15.12:0/2170003879 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bb8eec9ff0 0x55bb8eec9580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:38.277 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:38.273+0000 7efee8239740 1 -- 172.21.15.12:0/2170003879 >> 172.21.15.12:0/2170003879 conn(0x55bb8eec7e20 msgr2=0x55bb8eec8e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:28:38.277 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:38.273+0000 7efee8239740 1 -- 172.21.15.12:0/2170003879 shutdown_connections 2024-09-11T00:28:38.278 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:38.273+0000 7efee8239740 1 -- 172.21.15.12:0/2170003879 wait complete. 2024-09-11T00:28:38.278 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:38.274+0000 7efee8239740 1 -- start start 2024-09-11T00:28:38.280 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:38.274+0000 7efee8239740 1 Processor -- start 2024-09-11T00:28:38.280 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:38.274+0000 7efee8239740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bb8eec9ff0 0x55bb8eedeac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:28:38.280 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:38.274+0000 7efee8239740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bb8eeca910 0x55bb8eedf020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:28:38.280 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:38.274+0000 7efee8239740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bb8eed68a0 0x55bb8eed9800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:28:38.280 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:38.274+0000 7efee8239740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7efee01b7780 con 0x55bb8eeca910 2024-09-11T00:28:38.280 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:38.274+0000 7efee8239740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7efee01b70b0 con 0x55bb8eec9ff0 2024-09-11T00:28:38.280 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:38.274+0000 7efee8239740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7efee01b4200 con 0x55bb8eed68a0 2024-09-11T00:28:38.280 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:38.274+0000 7efee8239740 10 client.0 _ll_register_callbacks cb 0x7efec8009ce0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:28:38.281 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:38.275+0000 7efed6000640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bb8eeca910 0x55bb8eedf020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:28:38.281 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:38.275+0000 7efed6000640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bb8eeca910 0x55bb8eedf020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:51068/0 (socket says 172.21.15.12:51068) 2024-09-11T00:28:38.281 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:38.275+0000 7efed6000640 1 -- 172.21.15.12:0/3484582058 learned_addr learned my addr 172.21.15.12:0/3484582058 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:28:38.281 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:38.275+0000 7efedf400640 1 --2- 172.21.15.12:0/3484582058 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bb8eed68a0 0x55bb8eed9800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:28:38.281 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:38.275+0000 7efedf400640 1 --2- 172.21.15.12:0/3484582058 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bb8eec9ff0 0x55bb8eedeac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:28:38.281 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:38.275+0000 7efed6000640 1 -- 172.21.15.12:0/3484582058 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bb8eed68a0 msgr2=0x55bb8eed9800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:28:38.281 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:38.275+0000 7efed6000640 1 --2- 172.21.15.12:0/3484582058 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bb8eed68a0 0x55bb8eed9800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:38.281 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:38.275+0000 7efed6000640 1 -- 172.21.15.12:0/3484582058 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bb8eec9ff0 msgr2=0x55bb8eedeac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:28:38.281 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:38.275+0000 7efed6000640 1 --2- 172.21.15.12:0/3484582058 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bb8eec9ff0 0x55bb8eedeac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:38.281 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:38.275+0000 7efed6000640 1 -- 172.21.15.12:0/3484582058 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bb8eeda030 con 0x55bb8eeca910 2024-09-11T00:28:38.281 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:38.276+0000 7efed6000640 1 --2- 172.21.15.12:0/3484582058 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bb8eeca910 0x55bb8eedf020 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7efec801c9a0 tx=0x7efec8009fa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8b5f733421c3e2b server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:28:38.282 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:38.276+0000 7efeccc00640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-09-11T00:28:38.282 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:38.276+0000 7efeccc00640 1 -- 172.21.15.12:0/3484582058 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efec80052e0 con 0x55bb8eeca910 2024-09-11T00:28:38.282 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:38.276+0000 7efeccc00640 1 -- 172.21.15.12:0/3484582058 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 9208+0+0 (secure 0 0 0) 0x7efec80035f0 con 0x55bb8eeca910 2024-09-11T00:28:38.282 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:38.276+0000 7efee8239740 1 -- 172.21.15.12:0/3484582058 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efec801c600 con 0x55bb8eeca910 2024-09-11T00:28:38.282 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:38.276+0000 7efeccc00640 1 -- 172.21.15.12:0/3484582058 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=93) ==== 24+0+0 (secure 0 0 0) 0x7efec80037f0 con 0x55bb8eeca910 2024-09-11T00:28:38.282 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:38.276+0000 7efee8239740 10 client.17853 fetch_fsmap learned FSMap version 93 2024-09-11T00:28:38.282 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:38.276+0000 7efee8239740 1 -- 172.21.15.12:0/3484582058 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=93,monmap=4+}) -- 0x55bb8ef53d00 con 0x55bb8eeca910 2024-09-11T00:28:38.282 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:38.276+0000 7efeccc00640 1 -- 172.21.15.12:0/3484582058 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efec801c600 con 0x55bb8eeca910 2024-09-11T00:28:38.282 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:38.277+0000 7efeccc00640 1 -- 172.21.15.12:0/3484582058 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 93) ==== 8410+0+0 (secure 0 0 0) 0x7efec8028030 con 0x55bb8eeca910 2024-09-11T00:28:38.282 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:38.277+0000 7efee8239740 10 client.17853 fetch_fsmap finished waiting for FSMap version 93 2024-09-11T00:28:38.282 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:38.277+0000 7efee8239740 10 client.17853 resolve_mds: validated gid 24674 aka daemon mds.b 2024-09-11T00:28:38.282 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:38.277+0000 7efee8239740 20 client.17853 populate_metadata read hostname 'smithi012' 2024-09-11T00:28:38.282 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:38.277+0000 7efee8239740 1 --2- 172.21.15.12:0/3484582058 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55bb8ef54390 0x55bb8ef567b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:28:38.282 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:38.277+0000 7efee8239740 4 client.17853 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "248955d7", "release": true, "await": true}] 2024-09-11T00:28:38.283 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:38.277+0000 7efee8239740 1 -- 172.21.15.12:0/3484582058 --> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] -- command(tid 0: {"prefix": "quiesce db", "set_id": "248955d7", "release": true, "await": true}) -- 0x55bb8ee3ec90 con 0x55bb8ef54390 2024-09-11T00:28:38.283 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:38.277+0000 7efedf400640 1 --2- 172.21.15.12:0/3484582058 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55bb8ef54390 0x55bb8ef567b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:28:38.283 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:38.278+0000 7efedf400640 1 --2- 172.21.15.12:0/3484582058 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55bb8ef54390 0x55bb8ef567b0 crc :-1 s=READY pgs=300 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e921749a825a856e server_cookie=637c7bbbf54ea6a3 in_seq=0 out_seq=0 2024-09-11T00:28:38.283 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:38.278+0000 7efeccc00640 10 client.17853 ms_handle_connect on v2:172.21.15.165:6838/501312984 2024-09-11T00:28:38.312 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:38.309+0000 7efeccc00640 1 -- 172.21.15.12:0/3484582058 <== mds.0 v2:172.21.15.165:6838/501312984 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55bb8ee3ec90 con 0x55bb8ef54390 2024-09-11T00:28:38.312 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:38.309+0000 7efeccc00640 10 client.17853 handle_command_reply: tid=0 2024-09-11T00:28:38.312 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:38.309+0000 7efee8239740 1 client.17853 shutdown 2024-09-11T00:28:38.312 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:38.309+0000 7efee8239740 1 -- 172.21.15.12:0/3484582058 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bb8eeca910 msgr2=0x55bb8eedf020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:28:38.312 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:38.309+0000 7efee8239740 1 --2- 172.21.15.12:0/3484582058 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bb8eeca910 0x55bb8eedf020 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7efec801c9a0 tx=0x7efec8009fa0 comp rx=0 tx=0).stop 2024-09-11T00:28:38.313 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:38.309+0000 7efee8239740 1 -- 172.21.15.12:0/3484582058 shutdown_connections 2024-09-11T00:28:38.313 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:38.309+0000 7efee8239740 1 --2- 172.21.15.12:0/3484582058 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55bb8ef54390 0x55bb8ef567b0 crc :-1 s=READY pgs=300 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:38.313 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:38.309+0000 7efee8239740 1 --2- 172.21.15.12:0/3484582058 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55bb8eed68a0 0x55bb8eed9800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:38.313 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:38.309+0000 7efee8239740 1 --2- 172.21.15.12:0/3484582058 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bb8eeca910 0x55bb8eedf020 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:38.313 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:38.309+0000 7efee8239740 1 --2- 172.21.15.12:0/3484582058 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bb8eec9ff0 0x55bb8eedeac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:38.313 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:38.309+0000 7efeccc00640 0 client.17853 ms_handle_reset on v2:172.21.15.165:6838/501312984 2024-09-11T00:28:38.313 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:38.309+0000 7efee8239740 1 -- 172.21.15.12:0/3484582058 >> 172.21.15.12:0/3484582058 conn(0x55bb8eec7e20 msgr2=0x55bb8eec6590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:28:38.313 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:38.309+0000 7efee8239740 1 -- 172.21.15.12:0/3484582058 shutdown_connections 2024-09-11T00:28:38.313 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:38.309+0000 7efee8239740 1 -- 172.21.15.12:0/3484582058 wait complete. 2024-09-11T00:28:38.313 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:38.309+0000 7efee8239740 20 client.17853 trim_cache size 0 max 16384 2024-09-11T00:28:38.314 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-09-11T00:28:38.314 INFO:teuthology.orchestra.run.smithi012.stdout: "epoch": 93, 2024-09-11T00:28:38.314 INFO:teuthology.orchestra.run.smithi012.stdout: "leader": 24674, 2024-09-11T00:28:38.314 INFO:teuthology.orchestra.run.smithi012.stdout: "set_version": 68, 2024-09-11T00:28:38.314 INFO:teuthology.orchestra.run.smithi012.stdout: "sets": { 2024-09-11T00:28:38.314 INFO:teuthology.orchestra.run.smithi012.stdout: "248955d7": { 2024-09-11T00:28:38.314 INFO:teuthology.orchestra.run.smithi012.stdout: "version": 68, 2024-09-11T00:28:38.314 INFO:teuthology.orchestra.run.smithi012.stdout: "age_ref": 0.0, 2024-09-11T00:28:38.315 INFO:teuthology.orchestra.run.smithi012.stdout: "state": { 2024-09-11T00:28:38.315 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "RELEASED", 2024-09-11T00:28:38.315 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 0.0 2024-09-11T00:28:38.315 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:28:38.315 INFO:teuthology.orchestra.run.smithi012.stdout: "timeout": 90.0, 2024-09-11T00:28:38.315 INFO:teuthology.orchestra.run.smithi012.stdout: "expiration": 150.0, 2024-09-11T00:28:38.315 INFO:teuthology.orchestra.run.smithi012.stdout: "members": { 2024-09-11T00:28:38.315 INFO:teuthology.orchestra.run.smithi012.stdout: "file:/": { 2024-09-11T00:28:38.315 INFO:teuthology.orchestra.run.smithi012.stdout: "excluded": false, 2024-09-11T00:28:38.315 INFO:teuthology.orchestra.run.smithi012.stdout: "state": { 2024-09-11T00:28:38.315 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "RELEASED", 2024-09-11T00:28:38.315 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 0.0 2024-09-11T00:28:38.315 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:28:38.316 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:28:38.316 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:28:38.316 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:28:38.316 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:28:38.316 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-09-11T00:28:38.316 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:38.309+0000 7efee5800640 1 -- 172.21.15.12:0/3134569194 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7efec40b5360 msgr2=0x7efec40b7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:28:38.316 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:38.309+0000 7efee5800640 1 --2- 172.21.15.12:0/3134569194 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7efec40b5360 0x7efec40b7820 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7efed00c9930 tx=0x7efed0109cc0 comp rx=0 tx=0).stop 2024-09-11T00:28:38.316 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:38.309+0000 7efee5800640 1 -- 172.21.15.12:0/3134569194 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efee0075540 msgr2=0x7efee019f8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:28:38.316 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:38.309+0000 7efee5800640 1 --2- 172.21.15.12:0/3134569194 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efee0075540 0x7efee019f8e0 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7efed0005ab0 tx=0x7efed0003c10 comp rx=0 tx=0).stop 2024-09-11T00:28:38.317 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:38.309+0000 7efee5800640 1 -- 172.21.15.12:0/3134569194 shutdown_connections 2024-09-11T00:28:38.317 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:38.309+0000 7efee5800640 1 --2- 172.21.15.12:0/3134569194 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7efec40b5360 0x7efec40b7820 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:38.317 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:38.309+0000 7efee5800640 1 --2- 172.21.15.12:0/3134569194 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efee0113b00 0x7efee01a51c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:38.317 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:38.309+0000 7efee5800640 1 --2- 172.21.15.12:0/3134569194 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7efee0075e60 0x7efee019fe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:38.317 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:38.309+0000 7efee5800640 1 --2- 172.21.15.12:0/3134569194 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efee0075540 0x7efee019f8e0 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:38.317 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:38.309+0000 7efee5800640 1 -- 172.21.15.12:0/3134569194 >> 172.21.15.12:0/3134569194 conn(0x7efee00fe810 msgr2=0x7efee00ff410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:28:38.317 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:38.315+0000 7efee5800640 1 -- 172.21.15.12:0/3134569194 shutdown_connections 2024-09-11T00:28:38.318 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:28:38.315+0000 7efee5800640 1 -- 172.21.15.12:0/3134569194 wait complete. 2024-09-11T00:28:38.924 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:28:38 smithi012 ceph-mon[30333]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T00:28:39.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:28:38 smithi178 ceph-mon[35919]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T00:28:39.253 INFO:tasks.workunit.client.0.smithi012.stdout: 10 136046 14.04 MB/sec execute 395 sec latency 26529.357 ms 2024-09-11T00:28:39.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:28:38 smithi165 ceph-mon[35356]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T00:28:39.343 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '248955d7', total seconds elapsed: 34.2 2024-09-11T00:28:40.064 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:28:39 smithi012 ceph-mon[30333]: pgmap v1047: 129 pgs: 129 active+clean; 686 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 33 KiB/s wr, 19 op/s 2024-09-11T00:28:40.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:28:39 smithi178 ceph-mon[35919]: pgmap v1047: 129 pgs: 129 active+clean; 686 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 33 KiB/s wr, 19 op/s 2024-09-11T00:28:40.253 INFO:tasks.workunit.client.0.smithi012.stdout: 10 136781 14.13 MB/sec execute 396 sec latency 157.135 ms 2024-09-11T00:28:40.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:28:39 smithi165 ceph-mon[35356]: pgmap v1047: 129 pgs: 129 active+clean; 686 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 33 KiB/s wr, 19 op/s 2024-09-11T00:28:41.253 INFO:tasks.workunit.client.0.smithi012.stdout: 10 137284 14.16 MB/sec execute 397 sec latency 294.760 ms 2024-09-11T00:28:42.063 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:28:41 smithi012 ceph-mon[30333]: pgmap v1048: 129 pgs: 129 active+clean; 693 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 955 KiB/s wr, 86 op/s 2024-09-11T00:28:42.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:28:41 smithi178 ceph-mon[35919]: pgmap v1048: 129 pgs: 129 active+clean; 693 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 955 KiB/s wr, 86 op/s 2024-09-11T00:28:42.253 INFO:tasks.workunit.client.0.smithi012.stdout: 10 137952 14.21 MB/sec execute 398 sec latency 71.643 ms 2024-09-11T00:28:42.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:28:41 smithi165 ceph-mon[35356]: pgmap v1048: 129 pgs: 129 active+clean; 693 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 955 KiB/s wr, 86 op/s 2024-09-11T00:28:43.253 INFO:tasks.workunit.client.0.smithi012.stdout: 10 138504 14.23 MB/sec execute 399 sec latency 83.606 ms 2024-09-11T00:28:44.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:28:43 smithi178 ceph-mon[35919]: pgmap v1049: 129 pgs: 129 active+clean; 699 MiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 1.7 MiB/s wr, 156 op/s 2024-09-11T00:28:44.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:28:43 smithi178 ceph-mon[35919]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T00:28:44.253 INFO:tasks.workunit.client.0.smithi012.stdout: 10 139221 14.30 MB/sec execute 400 sec latency 176.686 ms 2024-09-11T00:28:44.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:28:43 smithi165 ceph-mon[35356]: pgmap v1049: 129 pgs: 129 active+clean; 699 MiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 1.7 MiB/s wr, 156 op/s 2024-09-11T00:28:44.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:28:43 smithi165 ceph-mon[35356]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T00:28:44.347 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:28:43 smithi012 ceph-mon[30333]: pgmap v1049: 129 pgs: 129 active+clean; 699 MiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 1.7 MiB/s wr, 156 op/s 2024-09-11T00:28:44.347 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:28:43 smithi012 ceph-mon[30333]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T00:28:45.253 INFO:tasks.workunit.client.0.smithi012.stdout: 10 139616 14.31 MB/sec execute 401 sec latency 467.175 ms 2024-09-11T00:28:46.055 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:46.052+0000 7fded8800640 1 -- 172.21.15.12:0/4037150831 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fded01066b0 msgr2=0x7fded010d030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:28:46.055 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:46.052+0000 7fded8800640 1 --2- 172.21.15.12:0/4037150831 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fded01066b0 0x7fded010d030 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fdecc009cd0 tx=0x7fdecc02a8f0 comp rx=0 tx=0).stop 2024-09-11T00:28:46.055 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:46.052+0000 7fded8800640 1 -- 172.21.15.12:0/4037150831 shutdown_connections 2024-09-11T00:28:46.055 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:46.052+0000 7fded8800640 1 --2- 172.21.15.12:0/4037150831 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fded01066b0 0x7fded010d030 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:46.055 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:46.052+0000 7fded8800640 1 --2- 172.21.15.12:0/4037150831 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fded0105cf0 0x7fded0106170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:46.056 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:46.052+0000 7fded8800640 1 --2- 172.21.15.12:0/4037150831 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fded0104ba0 0x7fded0104fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:46.056 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:46.052+0000 7fded8800640 1 -- 172.21.15.12:0/4037150831 >> 172.21.15.12:0/4037150831 conn(0x7fded0100440 msgr2=0x7fded0102880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:28:46.056 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:46.053+0000 7fded8800640 1 -- 172.21.15.12:0/4037150831 shutdown_connections 2024-09-11T00:28:46.056 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:46.053+0000 7fded8800640 1 -- 172.21.15.12:0/4037150831 wait complete. 2024-09-11T00:28:46.057 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:46.054+0000 7fded8800640 1 Processor -- start 2024-09-11T00:28:46.060 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:46.056+0000 7fded8800640 1 -- start start 2024-09-11T00:28:46.060 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:46.057+0000 7fded8800640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fded0104ba0 0x7fded019af20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:28:46.060 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:46.057+0000 7fded8800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fded0105cf0 0x7fded019b570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:28:46.060 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:46.057+0000 7fded8800640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fded01066b0 0x7fded019c1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:28:46.060 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:46.057+0000 7fded8800640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fded0110840 con 0x7fded0104ba0 2024-09-11T00:28:46.060 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:46.057+0000 7fded8800640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fded01106c0 con 0x7fded0105cf0 2024-09-11T00:28:46.060 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:46.057+0000 7fded8800640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fded01109c0 con 0x7fded01066b0 2024-09-11T00:28:46.060 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:46.057+0000 7fded6800640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fded01066b0 0x7fded019c1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:28:46.060 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:46.057+0000 7fded6800640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fded01066b0 0x7fded019c1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.12:34538/0 (socket says 172.21.15.12:34538) 2024-09-11T00:28:46.060 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:46.057+0000 7fded6800640 1 -- 172.21.15.12:0/1554429110 learned_addr learned my addr 172.21.15.12:0/1554429110 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:28:46.061 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:46.057+0000 7fded5400640 1 --2- 172.21.15.12:0/1554429110 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fded0105cf0 0x7fded019b570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:28:46.061 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:46.058+0000 7fded5e00640 1 --2- 172.21.15.12:0/1554429110 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fded0104ba0 0x7fded019af20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:28:46.061 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:46.058+0000 7fded6800640 1 -- 172.21.15.12:0/1554429110 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fded0105cf0 msgr2=0x7fded019b570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:28:46.061 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:46.058+0000 7fded6800640 1 --2- 172.21.15.12:0/1554429110 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fded0105cf0 0x7fded019b570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:46.061 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:46.058+0000 7fded6800640 1 -- 172.21.15.12:0/1554429110 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fded0104ba0 msgr2=0x7fded019af20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:28:46.061 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:46.058+0000 7fded6800640 1 --2- 172.21.15.12:0/1554429110 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fded0104ba0 0x7fded019af20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:46.061 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:46.058+0000 7fded6800640 1 -- 172.21.15.12:0/1554429110 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fded019ca60 con 0x7fded01066b0 2024-09-11T00:28:46.062 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:46.058+0000 7fded5400640 1 --2- 172.21.15.12:0/1554429110 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fded0105cf0 0x7fded019b570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T00:28:46.063 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:46.059+0000 7fded6800640 1 --2- 172.21.15.12:0/1554429110 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fded01066b0 0x7fded019c1a0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fdecc003ba0 tx=0x7fdecc003bd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:28:46.063 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:46.061+0000 7fdecaa00640 1 -- 172.21.15.12:0/1554429110 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdecc004240 con 0x7fded01066b0 2024-09-11T00:28:46.063 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:46.061+0000 7fdecaa00640 1 -- 172.21.15.12:0/1554429110 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdecc040070 con 0x7fded01066b0 2024-09-11T00:28:46.064 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:46.061+0000 7fdecaa00640 1 -- 172.21.15.12:0/1554429110 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdecc045bb0 con 0x7fded01066b0 2024-09-11T00:28:46.064 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:46.061+0000 7fded8800640 1 -- 172.21.15.12:0/1554429110 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fded019ccf0 con 0x7fded01066b0 2024-09-11T00:28:46.064 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:46.061+0000 7fded8800640 1 -- 172.21.15.12:0/1554429110 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fded006cd50 con 0x7fded01066b0 2024-09-11T00:28:46.070 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:46.067+0000 7fdecaa00640 1 -- 172.21.15.12:0/1554429110 <== mon.1 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7fdecc04d070 con 0x7fded01066b0 2024-09-11T00:28:46.074 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:46.070+0000 7fdecaa00640 1 --2- 172.21.15.12:0/1554429110 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fdeb80b54b0 0x7fdeb80b7970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:28:46.074 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:46.070+0000 7fdecaa00640 1 -- 172.21.15.12:0/1554429110 <== mon.1 v2:172.21.15.178:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9208+0+0 (secure 0 0 0) 0x7fdecc107440 con 0x7fded01066b0 2024-09-11T00:28:46.074 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:46.071+0000 7fded5e00640 1 --2- 172.21.15.12:0/1554429110 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fdeb80b54b0 0x7fdeb80b7970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:28:46.074 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:46.072+0000 7fded5e00640 1 --2- 172.21.15.12:0/1554429110 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fdeb80b54b0 0x7fdeb80b7970 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fdec0009aa0 tx=0x7fdec0005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:28:46.080 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:46.078+0000 7fdedb869740 1 Processor -- start 2024-09-11T00:28:46.080 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:46.078+0000 7fdedb869740 1 -- start start 2024-09-11T00:28:46.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:28:45 smithi178 ceph-mon[35919]: pgmap v1050: 129 pgs: 129 active+clean; 698 MiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 2.0 MiB/s wr, 163 op/s 2024-09-11T00:28:46.082 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:46.079+0000 7fdedb869740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d0e0342fa0 0x55d0e0345390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:28:46.082 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:46.080+0000 7fdedb869740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d0e03458d0 0x55d0e0347cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:28:46.082 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:46.080+0000 7fdedb869740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d0e0348310 0x55d0e034a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:28:46.082 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:46.080+0000 7fdedb869740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55d0e02ad6b0 con 0x55d0e03458d0 2024-09-11T00:28:46.082 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:46.080+0000 7fdedb869740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55d0e029cae0 con 0x55d0e0348310 2024-09-11T00:28:46.083 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:46.080+0000 7fdedb869740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55d0e02a08f0 con 0x55d0e0342fa0 2024-09-11T00:28:46.083 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:46.080+0000 7fded6800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d0e0348310 0x55d0e034a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:28:46.083 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:46.080+0000 7fded6800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d0e0348310 0x55d0e034a7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.12:42326/0 (socket says 172.21.15.12:42326) 2024-09-11T00:28:46.083 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:46.080+0000 7fded6800640 1 -- 172.21.15.12:0/3536896100 learned_addr learned my addr 172.21.15.12:0/3536896100 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:28:46.083 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:46.081+0000 7fded5e00640 1 --2- 172.21.15.12:0/3536896100 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d0e0342fa0 0x55d0e0345390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:28:46.084 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:46.081+0000 7fded6800640 1 -- 172.21.15.12:0/3536896100 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d0e0342fa0 msgr2=0x55d0e0345390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:28:46.084 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:46.081+0000 7fded6800640 1 --2- 172.21.15.12:0/3536896100 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d0e0342fa0 0x55d0e0345390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:46.084 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:46.081+0000 7fded6800640 1 -- 172.21.15.12:0/3536896100 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d0e03458d0 msgr2=0x55d0e0347cc0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-11T00:28:46.084 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:46.081+0000 7fded6800640 1 --2- 172.21.15.12:0/3536896100 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d0e03458d0 0x55d0e0347cc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:46.084 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:46.081+0000 7fded6800640 1 -- 172.21.15.12:0/3536896100 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d0e0269780 con 0x55d0e0348310 2024-09-11T00:28:46.084 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:46.082+0000 7fded6800640 1 --2- 172.21.15.12:0/3536896100 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d0e0348310 0x55d0e034a7d0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fdecc0c9da0 tx=0x7fdecc0095c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c8bbeb1b2fa42532 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:28:46.085 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:46.082+0000 7fdebf000640 1 -- 172.21.15.12:0/3536896100 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdecc0cd2e0 con 0x55d0e0348310 2024-09-11T00:28:46.085 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:46.083+0000 7fdebf000640 1 -- 172.21.15.12:0/3536896100 <== mon.2 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdecc036030 con 0x55d0e0348310 2024-09-11T00:28:46.085 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:46.083+0000 7fdebf000640 1 -- 172.21.15.12:0/3536896100 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdecc0c9a60 con 0x55d0e0348310 2024-09-11T00:28:46.087 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:46.084+0000 7fdedb869740 1 -- 172.21.15.12:0/3536896100 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d0e0348310 msgr2=0x55d0e034a7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:28:46.087 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:46.084+0000 7fdedb869740 1 --2- 172.21.15.12:0/3536896100 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d0e0348310 0x55d0e034a7d0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fdecc0c9da0 tx=0x7fdecc0095c0 comp rx=0 tx=0).stop 2024-09-11T00:28:46.087 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:46.085+0000 7fdedb869740 1 -- 172.21.15.12:0/3536896100 shutdown_connections 2024-09-11T00:28:46.087 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:46.085+0000 7fdedb869740 1 --2- 172.21.15.12:0/3536896100 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d0e0348310 0x55d0e034a7d0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:46.087 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:46.085+0000 7fdedb869740 1 --2- 172.21.15.12:0/3536896100 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d0e03458d0 0x55d0e0347cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:46.087 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:46.085+0000 7fdedb869740 1 --2- 172.21.15.12:0/3536896100 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d0e0342fa0 0x55d0e0345390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:46.087 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:46.085+0000 7fdedb869740 1 -- 172.21.15.12:0/3536896100 >> 172.21.15.12:0/3536896100 conn(0x55d0e033e0f0 msgr2=0x55d0e033ee80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:28:46.088 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:46.086+0000 7fdedb869740 1 -- 172.21.15.12:0/3536896100 shutdown_connections 2024-09-11T00:28:46.089 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:46.086+0000 7fdedb869740 1 -- 172.21.15.12:0/3536896100 wait complete. 2024-09-11T00:28:46.089 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:46.087+0000 7fdedb869740 1 -- start start 2024-09-11T00:28:46.090 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:46.088+0000 7fdedb869740 1 Processor -- start 2024-09-11T00:28:46.091 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:46.088+0000 7fdedb869740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d0e0342fa0 0x55d0e034c480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:28:46.091 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:46.089+0000 7fdedb869740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d0e03458d0 0x55d0e034cb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:28:46.091 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:46.089+0000 7fdedb869740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d0e0348310 0x55d0e034d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:28:46.091 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:46.089+0000 7fdedb869740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55d0e01d4ab0 con 0x55d0e0348310 2024-09-11T00:28:46.092 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:46.089+0000 7fdedb869740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55d0e0243570 con 0x55d0e0342fa0 2024-09-11T00:28:46.092 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:46.089+0000 7fdedb869740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55d0e034b0b0 con 0x55d0e03458d0 2024-09-11T00:28:46.092 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:46.089+0000 7fdedb869740 10 client.0 _ll_register_callbacks cb 0x55d0e0252100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:28:46.092 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:46.089+0000 7fded5e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d0e0342fa0 0x55d0e034c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:28:46.092 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:46.089+0000 7fded5400640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d0e03458d0 0x55d0e034cb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:28:46.092 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:46.089+0000 7fded5e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d0e0342fa0 0x55d0e034c480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.12:42338/0 (socket says 172.21.15.12:42338) 2024-09-11T00:28:46.092 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:46.089+0000 7fded5e00640 1 -- 172.21.15.12:0/4030347124 learned_addr learned my addr 172.21.15.12:0/4030347124 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:28:46.093 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:46.090+0000 7fded5400640 1 -- 172.21.15.12:0/4030347124 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d0e0342fa0 msgr2=0x55d0e034c480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:28:46.094 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:46.090+0000 7fded5400640 1 --2- 172.21.15.12:0/4030347124 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d0e0342fa0 0x55d0e034c480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:46.094 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:46.090+0000 7fded5400640 1 -- 172.21.15.12:0/4030347124 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d0e0348310 msgr2=0x55d0e034d6f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-11T00:28:46.094 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:46.090+0000 7fded6800640 1 --2- 172.21.15.12:0/4030347124 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d0e0348310 0x55d0e034d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:28:46.094 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:46.090+0000 7fded5400640 1 --2- 172.21.15.12:0/4030347124 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d0e0348310 0x55d0e034d6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:46.094 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:46.090+0000 7fded5400640 1 -- 172.21.15.12:0/4030347124 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d0dfffe790 con 0x55d0e03458d0 2024-09-11T00:28:46.094 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:46.090+0000 7fded6800640 1 --2- 172.21.15.12:0/4030347124 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d0e0348310 0x55d0e034d6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T00:28:46.094 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:46.090+0000 7fded5400640 1 --2- 172.21.15.12:0/4030347124 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d0e03458d0 0x55d0e034cb20 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fdec400dd30 tx=0x7fdec400f730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6fbbd660eb0155e5 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:28:46.094 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:46.091+0000 7fdeb3400640 10 client.0 ms_handle_connect on v2:172.21.15.178:3300/0 2024-09-11T00:28:46.094 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:46.091+0000 7fdeb3400640 1 -- 172.21.15.12:0/4030347124 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdec4004230 con 0x55d0e03458d0 2024-09-11T00:28:46.094 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:46.091+0000 7fdeb3400640 1 -- 172.21.15.12:0/4030347124 <== mon.1 v2:172.21.15.178:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 9208+0+0 (secure 0 0 0) 0x7fdec4004030 con 0x55d0e03458d0 2024-09-11T00:28:46.094 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:46.091+0000 7fdedb869740 1 -- 172.21.15.12:0/4030347124 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdecc036030 con 0x55d0e03458d0 2024-09-11T00:28:46.094 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:46.091+0000 7fdeb3400640 1 -- 172.21.15.12:0/4030347124 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=94) ==== 24+0+0 (secure 0 0 0) 0x7fdec4004230 con 0x55d0e03458d0 2024-09-11T00:28:46.095 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:46.091+0000 7fdedb869740 10 client.27841 fetch_fsmap learned FSMap version 94 2024-09-11T00:28:46.095 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:46.091+0000 7fdedb869740 1 -- 172.21.15.12:0/4030347124 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,fsmap=94,monmap=4+}) -- 0x55d0e0077620 con 0x55d0e03458d0 2024-09-11T00:28:46.095 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:46.092+0000 7fdeb3400640 1 -- 172.21.15.12:0/4030347124 <== mon.1 v2:172.21.15.178:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdec4003aa0 con 0x55d0e03458d0 2024-09-11T00:28:46.096 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:46.092+0000 7fdeb3400640 1 -- 172.21.15.12:0/4030347124 <== mon.1 v2:172.21.15.178:3300/0 5 ==== fsmap(e 94) ==== 8406+0+0 (secure 0 0 0) 0x7fdec4011880 con 0x55d0e03458d0 2024-09-11T00:28:46.096 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:46.092+0000 7fdedb869740 10 client.27841 fetch_fsmap finished waiting for FSMap version 94 2024-09-11T00:28:46.096 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:46.092+0000 7fdedb869740 10 client.27841 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T00:28:46.096 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:46.092+0000 7fdedb869740 20 client.27841 populate_metadata read hostname 'smithi012' 2024-09-11T00:28:46.096 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:46.092+0000 7fdedb869740 1 --2- 172.21.15.12:0/4030347124 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55d0e0338cb0 0x55d0e033b0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:28:46.096 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:46.092+0000 7fdedb869740 4 client.27841 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T00:28:46.096 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:46.092+0000 7fdedb869740 1 -- 172.21.15.12:0/4030347124 --> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fdecc036030 con 0x55d0e0338cb0 2024-09-11T00:28:46.096 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:46.093+0000 7fded5e00640 1 --2- 172.21.15.12:0/4030347124 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55d0e0338cb0 0x55d0e033b0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:28:46.097 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:46.094+0000 7fdeb3400640 10 client.27841 ms_handle_connect on v2:172.21.15.165:6838/501312984 2024-09-11T00:28:46.097 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:46.094+0000 7fded5e00640 1 --2- 172.21.15.12:0/4030347124 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55d0e0338cb0 0x55d0e033b0f0 crc :-1 s=READY pgs=301 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=baff55d036bb4cc6 server_cookie=28028199d90baf35 in_seq=0 out_seq=0 2024-09-11T00:28:46.097 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:46.095+0000 7fdeb3400640 1 -- 172.21.15.12:0/4030347124 <== mds.0 v2:172.21.15.165:6838/501312984 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x7fdecc036030 con 0x55d0e0338cb0 2024-09-11T00:28:46.097 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:46.095+0000 7fdeb3400640 10 client.27841 handle_command_reply: tid=0 2024-09-11T00:28:46.097 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:46.095+0000 7fdedb869740 1 client.27841 shutdown 2024-09-11T00:28:46.098 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:46.095+0000 7fdedb869740 1 -- 172.21.15.12:0/4030347124 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d0e03458d0 msgr2=0x55d0e034cb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:28:46.098 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:46.095+0000 7fdedb869740 1 --2- 172.21.15.12:0/4030347124 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d0e03458d0 0x55d0e034cb20 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fdec400dd30 tx=0x7fdec400f730 comp rx=0 tx=0).stop 2024-09-11T00:28:46.098 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:46.095+0000 7fdedb869740 1 -- 172.21.15.12:0/4030347124 shutdown_connections 2024-09-11T00:28:46.098 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:46.095+0000 7fdedb869740 1 --2- 172.21.15.12:0/4030347124 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55d0e0338cb0 0x55d0e033b0f0 crc :-1 s=READY pgs=301 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:46.098 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:46.095+0000 7fdeb3400640 0 client.27841 ms_handle_reset on v2:172.21.15.165:6838/501312984 2024-09-11T00:28:46.098 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:46.095+0000 7fdedb869740 1 --2- 172.21.15.12:0/4030347124 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d0e0348310 0x55d0e034d6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:46.099 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:46.095+0000 7fdedb869740 1 --2- 172.21.15.12:0/4030347124 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d0e03458d0 0x55d0e034cb20 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:46.099 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:46.095+0000 7fdedb869740 1 --2- 172.21.15.12:0/4030347124 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d0e0342fa0 0x55d0e034c480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:46.099 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:46.095+0000 7fdedb869740 1 -- 172.21.15.12:0/4030347124 >> 172.21.15.12:0/4030347124 conn(0x55d0e033e0f0 msgr2=0x55d0e0334a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:28:46.099 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:46.096+0000 7fdedb869740 1 -- 172.21.15.12:0/4030347124 shutdown_connections 2024-09-11T00:28:46.099 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:46.096+0000 7fdedb869740 1 -- 172.21.15.12:0/4030347124 wait complete. 2024-09-11T00:28:46.099 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:46.096+0000 7fdedb869740 20 client.27841 trim_cache size 0 max 16384 2024-09-11T00:28:46.127 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:46.124+0000 7fdedb869740 1 Processor -- start 2024-09-11T00:28:46.127 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:46.124+0000 7fdedb869740 1 -- start start 2024-09-11T00:28:46.129 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:46.125+0000 7fdedb869740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d0e0338cb0 0x55d0e0330960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:28:46.129 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:46.125+0000 7fdedb869740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d0e0342fa0 0x55d0e033e950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:28:46.129 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:46.125+0000 7fdedb869740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d0e03458d0 0x55d0e033ee90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:28:46.129 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:46.125+0000 7fdedb869740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fded0072f80 con 0x55d0e0338cb0 2024-09-11T00:28:46.130 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:46.125+0000 7fdedb869740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fded0074b80 con 0x55d0e03458d0 2024-09-11T00:28:46.130 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:46.125+0000 7fdedb869740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fded0073650 con 0x55d0e0342fa0 2024-09-11T00:28:46.130 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:46.125+0000 7fded5400640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d0e0342fa0 0x55d0e033e950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:28:46.130 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:46.125+0000 7fded5400640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d0e0342fa0 0x55d0e033e950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.12:34562/0 (socket says 172.21.15.12:34562) 2024-09-11T00:28:46.130 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:46.125+0000 7fded5400640 1 -- 172.21.15.12:0/1929746586 learned_addr learned my addr 172.21.15.12:0/1929746586 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:28:46.130 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:46.126+0000 7fded5e00640 1 --2- 172.21.15.12:0/1929746586 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d0e0338cb0 0x55d0e0330960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:28:46.130 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:46.126+0000 7fded5400640 1 -- 172.21.15.12:0/1929746586 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d0e03458d0 msgr2=0x55d0e033ee90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:28:46.130 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:46.126+0000 7fded5400640 1 --2- 172.21.15.12:0/1929746586 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d0e03458d0 0x55d0e033ee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:46.130 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:46.126+0000 7fded5400640 1 -- 172.21.15.12:0/1929746586 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d0e0338cb0 msgr2=0x55d0e0330960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:28:46.130 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:46.126+0000 7fded5400640 1 --2- 172.21.15.12:0/1929746586 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d0e0338cb0 0x55d0e0330960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:46.130 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:46.126+0000 7fded5400640 1 -- 172.21.15.12:0/1929746586 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d0e034da60 con 0x55d0e0342fa0 2024-09-11T00:28:46.130 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:46.126+0000 7fded5400640 1 --2- 172.21.15.12:0/1929746586 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d0e0342fa0 0x55d0e033e950 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fdec400fc30 tx=0x7fdec4005850 comp rx=0 tx=0).ready entity=mon.1 client_cookie=18d8ce362031da65 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:28:46.130 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:46.127+0000 7fdebe600640 1 -- 172.21.15.12:0/1929746586 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdec402a040 con 0x55d0e0342fa0 2024-09-11T00:28:46.130 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:46.127+0000 7fdebe600640 1 -- 172.21.15.12:0/1929746586 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdec400ba10 con 0x55d0e0342fa0 2024-09-11T00:28:46.131 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:46.127+0000 7fdebe600640 1 -- 172.21.15.12:0/1929746586 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdec4004230 con 0x55d0e0342fa0 2024-09-11T00:28:46.131 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:46.128+0000 7fdedb869740 1 -- 172.21.15.12:0/1929746586 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d0e0342fa0 msgr2=0x55d0e033e950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:28:46.132 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:46.128+0000 7fdedb869740 1 --2- 172.21.15.12:0/1929746586 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d0e0342fa0 0x55d0e033e950 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fdec400fc30 tx=0x7fdec4005850 comp rx=0 tx=0).stop 2024-09-11T00:28:46.132 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:46.128+0000 7fdedb869740 1 -- 172.21.15.12:0/1929746586 shutdown_connections 2024-09-11T00:28:46.132 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:46.128+0000 7fdedb869740 1 --2- 172.21.15.12:0/1929746586 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d0e03458d0 0x55d0e033ee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:46.132 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:46.128+0000 7fdedb869740 1 --2- 172.21.15.12:0/1929746586 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d0e0342fa0 0x55d0e033e950 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:46.132 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:46.128+0000 7fdedb869740 1 --2- 172.21.15.12:0/1929746586 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d0e0338cb0 0x55d0e0330960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:46.132 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:46.128+0000 7fdedb869740 1 -- 172.21.15.12:0/1929746586 >> 172.21.15.12:0/1929746586 conn(0x55d0e033e0f0 msgr2=0x55d0e0348780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:28:46.132 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:46.128+0000 7fdedb869740 1 -- 172.21.15.12:0/1929746586 shutdown_connections 2024-09-11T00:28:46.132 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:46.128+0000 7fdedb869740 1 -- 172.21.15.12:0/1929746586 wait complete. 2024-09-11T00:28:46.132 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:46.129+0000 7fdedb869740 1 -- start start 2024-09-11T00:28:46.132 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:46.129+0000 7fdedb869740 1 Processor -- start 2024-09-11T00:28:46.133 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:46.130+0000 7fdedb869740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d0e0338cb0 0x55d0e0354230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:28:46.133 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:46.130+0000 7fdedb869740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d0e0342fa0 0x55d0e0353090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:28:46.134 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:46.130+0000 7fdedb869740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d0e03458d0 0x55d0e0353780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:28:46.134 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:46.130+0000 7fdedb869740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fded00744b0 con 0x55d0e0342fa0 2024-09-11T00:28:46.134 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:46.130+0000 7fdedb869740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fded0073de0 con 0x55d0e0338cb0 2024-09-11T00:28:46.134 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:46.130+0000 7fdedb869740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fded0070f20 con 0x55d0e03458d0 2024-09-11T00:28:46.134 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:46.130+0000 7fdedb869740 10 client.0 _ll_register_callbacks cb 0x7fdec400d2e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:28:46.134 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:46.130+0000 7fded5e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d0e0338cb0 0x55d0e0354230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:28:46.134 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:46.130+0000 7fded6800640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d0e03458d0 0x55d0e0353780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:28:46.134 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:46.130+0000 7fded6800640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d0e03458d0 0x55d0e0353780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.12:34568/0 (socket says 172.21.15.12:34568) 2024-09-11T00:28:46.134 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:46.130+0000 7fded6800640 1 -- 172.21.15.12:0/768981159 learned_addr learned my addr 172.21.15.12:0/768981159 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:28:46.134 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:46.130+0000 7fded5400640 1 --2- 172.21.15.12:0/768981159 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d0e0342fa0 0x55d0e0353090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:28:46.135 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:46.131+0000 7fded6800640 1 -- 172.21.15.12:0/768981159 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d0e0338cb0 msgr2=0x55d0e0354230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:28:46.135 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:46.131+0000 7fded6800640 1 --2- 172.21.15.12:0/768981159 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d0e0338cb0 0x55d0e0354230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:46.135 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:46.131+0000 7fded6800640 1 -- 172.21.15.12:0/768981159 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d0e0342fa0 msgr2=0x55d0e0353090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:28:46.135 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:46.131+0000 7fded6800640 1 --2- 172.21.15.12:0/768981159 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d0e0342fa0 0x55d0e0353090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:46.135 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:46.131+0000 7fded6800640 1 -- 172.21.15.12:0/768981159 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d0e0345700 con 0x55d0e03458d0 2024-09-11T00:28:46.135 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:46.131+0000 7fded5400640 1 --2- 172.21.15.12:0/768981159 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d0e0342fa0 0x55d0e0353090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T00:28:46.135 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:46.131+0000 7fded6800640 1 --2- 172.21.15.12:0/768981159 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d0e03458d0 0x55d0e0353780 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fdecc0306c0 tx=0x7fdecc0cd220 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c5224bc988b6247 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:28:46.135 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:46.131+0000 7fdeb3400640 10 client.0 ms_handle_connect on v2:172.21.15.178:3300/0 2024-09-11T00:28:46.136 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:46.131+0000 7fdeb3400640 1 -- 172.21.15.12:0/768981159 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdecc0cd7d0 con 0x55d0e03458d0 2024-09-11T00:28:46.136 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:46.132+0000 7fdeb3400640 1 -- 172.21.15.12:0/768981159 <== mon.1 v2:172.21.15.178:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 9208+0+0 (secure 0 0 0) 0x7fdecc0cda30 con 0x55d0e03458d0 2024-09-11T00:28:46.136 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:46.132+0000 7fdedb869740 1 -- 172.21.15.12:0/768981159 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55d0e02af600 con 0x55d0e03458d0 2024-09-11T00:28:46.136 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:46.132+0000 7fdeb3400640 1 -- 172.21.15.12:0/768981159 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=94) ==== 24+0+0 (secure 0 0 0) 0x7fdecc0d6d60 con 0x55d0e03458d0 2024-09-11T00:28:46.136 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:46.132+0000 7fdedb869740 10 client.27847 fetch_fsmap learned FSMap version 94 2024-09-11T00:28:46.136 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:46.132+0000 7fdedb869740 1 -- 172.21.15.12:0/768981159 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,fsmap=94,monmap=4+}) -- 0x55d0e03cdbf0 con 0x55d0e03458d0 2024-09-11T00:28:46.137 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:46.132+0000 7fdeb3400640 1 -- 172.21.15.12:0/768981159 <== mon.1 v2:172.21.15.178:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55d0e02af600 con 0x55d0e03458d0 2024-09-11T00:28:46.137 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:46.133+0000 7fdeb3400640 1 -- 172.21.15.12:0/768981159 <== mon.1 v2:172.21.15.178:3300/0 5 ==== fsmap(e 94) ==== 8406+0+0 (secure 0 0 0) 0x7fdecc0d6960 con 0x55d0e03458d0 2024-09-11T00:28:46.137 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:46.133+0000 7fdedb869740 10 client.27847 fetch_fsmap finished waiting for FSMap version 94 2024-09-11T00:28:46.137 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:46.133+0000 7fdedb869740 10 client.27847 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T00:28:46.137 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:46.133+0000 7fdedb869740 20 client.27847 populate_metadata read hostname 'smithi012' 2024-09-11T00:28:46.137 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:46.133+0000 7fdedb869740 1 --2- 172.21.15.12:0/768981159 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55d0e0348310 0x55d0e03d02d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:28:46.137 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:46.133+0000 7fdedb869740 4 client.27847 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T00:28:46.137 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:46.133+0000 7fdedb869740 1 -- 172.21.15.12:0/768981159 --> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55d0e02ab0b0 con 0x55d0e0348310 2024-09-11T00:28:46.137 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:46.134+0000 7fded5e00640 1 --2- 172.21.15.12:0/768981159 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55d0e0348310 0x55d0e03d02d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:28:46.140 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:46.138+0000 7fded5e00640 1 --2- 172.21.15.12:0/768981159 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55d0e0348310 0x55d0e03d02d0 crc :-1 s=READY pgs=302 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ac644a6627127181 server_cookie=ce074c694a3a3bd in_seq=0 out_seq=0 2024-09-11T00:28:46.140 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:46.138+0000 7fdeb3400640 10 client.27847 ms_handle_connect on v2:172.21.15.165:6838/501312984 2024-09-11T00:28:46.253 INFO:tasks.workunit.client.0.smithi012.stdout: 10 140199 14.36 MB/sec execute 402 sec latency 201.131 ms 2024-09-11T00:28:46.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:28:45 smithi165 ceph-mon[35356]: pgmap v1050: 129 pgs: 129 active+clean; 698 MiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 2.0 MiB/s wr, 163 op/s 2024-09-11T00:28:46.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:28:45 smithi012 ceph-mon[30333]: pgmap v1050: 129 pgs: 129 active+clean; 698 MiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 2.0 MiB/s wr, 163 op/s 2024-09-11T00:28:46.338 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:46.335+0000 7fdeb3400640 1 -- 172.21.15.12:0/768981159 <== mds.0 v2:172.21.15.165:6838/501312984 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55d0e02ab0b0 con 0x55d0e0348310 2024-09-11T00:28:46.338 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:46.335+0000 7fdeb3400640 10 client.27847 handle_command_reply: tid=0 2024-09-11T00:28:46.338 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:46.335+0000 7fdedb869740 1 client.27847 shutdown 2024-09-11T00:28:46.338 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:46.336+0000 7fdedb869740 1 -- 172.21.15.12:0/768981159 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d0e03458d0 msgr2=0x55d0e0353780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:28:46.338 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:46.336+0000 7fdedb869740 1 --2- 172.21.15.12:0/768981159 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d0e03458d0 0x55d0e0353780 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fdecc0306c0 tx=0x7fdecc0cd220 comp rx=0 tx=0).stop 2024-09-11T00:28:46.339 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:46.336+0000 7fdedb869740 1 -- 172.21.15.12:0/768981159 shutdown_connections 2024-09-11T00:28:46.339 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:46.336+0000 7fdedb869740 1 --2- 172.21.15.12:0/768981159 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55d0e0348310 0x55d0e03d02d0 crc :-1 s=READY pgs=302 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:46.339 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:46.336+0000 7fdedb869740 1 --2- 172.21.15.12:0/768981159 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d0e03458d0 0x55d0e0353780 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:46.339 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:46.336+0000 7fdedb869740 1 --2- 172.21.15.12:0/768981159 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d0e0342fa0 0x55d0e0353090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:46.340 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:46.336+0000 7fdedb869740 1 --2- 172.21.15.12:0/768981159 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d0e0338cb0 0x55d0e0354230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:46.340 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:46.336+0000 7fdeb3400640 0 client.27847 ms_handle_reset on v2:172.21.15.165:6838/501312984 2024-09-11T00:28:46.340 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:46.336+0000 7fdedb869740 1 -- 172.21.15.12:0/768981159 >> 172.21.15.12:0/768981159 conn(0x55d0e033e0f0 msgr2=0x55d0e0348740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:28:46.340 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:46.336+0000 7fdedb869740 1 -- 172.21.15.12:0/768981159 shutdown_connections 2024-09-11T00:28:46.340 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:46.336+0000 7fdedb869740 1 -- 172.21.15.12:0/768981159 wait complete. 2024-09-11T00:28:46.340 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:46.336+0000 7fdedb869740 20 client.27847 trim_cache size 0 max 16384 2024-09-11T00:28:46.340 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:{ 2024-09-11T00:28:46.340 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "message": "", 2024-09-11T00:28:46.340 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "return_code": 0 2024-09-11T00:28:46.341 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:} 2024-09-11T00:28:46.341 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:46.337+0000 7fded8800640 1 -- 172.21.15.12:0/1554429110 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fdeb80b54b0 msgr2=0x7fdeb80b7970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:28:46.341 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:46.337+0000 7fded8800640 1 --2- 172.21.15.12:0/1554429110 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fdeb80b54b0 0x7fdeb80b7970 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fdec0009aa0 tx=0x7fdec0005f90 comp rx=0 tx=0).stop 2024-09-11T00:28:46.341 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:46.337+0000 7fded8800640 1 -- 172.21.15.12:0/1554429110 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fded01066b0 msgr2=0x7fded019c1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:28:46.341 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:46.337+0000 7fded8800640 1 --2- 172.21.15.12:0/1554429110 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fded01066b0 0x7fded019c1a0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fdecc003ba0 tx=0x7fdecc003bd0 comp rx=0 tx=0).stop 2024-09-11T00:28:46.342 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:46.337+0000 7fded8800640 1 -- 172.21.15.12:0/1554429110 shutdown_connections 2024-09-11T00:28:46.342 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:46.337+0000 7fded8800640 1 --2- 172.21.15.12:0/1554429110 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fdeb80b54b0 0x7fdeb80b7970 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:46.342 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:46.337+0000 7fded8800640 1 --2- 172.21.15.12:0/1554429110 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fded01066b0 0x7fded019c1a0 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:46.342 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:46.337+0000 7fded8800640 1 --2- 172.21.15.12:0/1554429110 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fded0105cf0 0x7fded019b570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:46.342 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:46.337+0000 7fded8800640 1 --2- 172.21.15.12:0/1554429110 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fded0104ba0 0x7fded019af20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:46.342 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:46.337+0000 7fded8800640 1 -- 172.21.15.12:0/1554429110 >> 172.21.15.12:0/1554429110 conn(0x7fded0100440 msgr2=0x7fded0109010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:28:46.342 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:46.338+0000 7fded8800640 1 -- 172.21.15.12:0/1554429110 shutdown_connections 2024-09-11T00:28:46.342 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:28:46.338+0000 7fded8800640 1 -- 172.21.15.12:0/1554429110 wait complete. 2024-09-11T00:28:47.253 INFO:tasks.workunit.client.0.smithi012.stdout: 10 140947 14.44 MB/sec execute 403 sec latency 171.574 ms 2024-09-11T00:28:48.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:28:47 smithi178 ceph-mon[35919]: pgmap v1051: 129 pgs: 129 active+clean; 709 MiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 5.8 MiB/s wr, 442 op/s 2024-09-11T00:28:48.253 INFO:tasks.workunit.client.0.smithi012.stdout: 10 141397 14.45 MB/sec execute 404 sec latency 289.808 ms 2024-09-11T00:28:48.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:28:47 smithi165 ceph-mon[35356]: pgmap v1051: 129 pgs: 129 active+clean; 709 MiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 5.8 MiB/s wr, 442 op/s 2024-09-11T00:28:48.314 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:28:47 smithi012 ceph-mon[30333]: pgmap v1051: 129 pgs: 129 active+clean; 709 MiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 5.8 MiB/s wr, 442 op/s 2024-09-11T00:28:49.253 INFO:tasks.workunit.client.0.smithi012.stdout: 10 141937 14.49 MB/sec execute 405 sec latency 133.088 ms 2024-09-11T00:28:50.064 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:28:49 smithi012 ceph-mon[30333]: pgmap v1052: 129 pgs: 129 active+clean; 710 MiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 6.0 MiB/s wr, 445 op/s 2024-09-11T00:28:50.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:28:49 smithi178 ceph-mon[35919]: pgmap v1052: 129 pgs: 129 active+clean; 710 MiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 6.0 MiB/s wr, 445 op/s 2024-09-11T00:28:50.254 INFO:tasks.workunit.client.0.smithi012.stdout: 10 142724 14.55 MB/sec execute 406 sec latency 178.233 ms 2024-09-11T00:28:50.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:28:49 smithi165 ceph-mon[35356]: pgmap v1052: 129 pgs: 129 active+clean; 710 MiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 6.0 MiB/s wr, 445 op/s 2024-09-11T00:28:51.184 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:28:50 smithi165 ceph-mon[35356]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T00:28:51.254 INFO:tasks.workunit.client.0.smithi012.stdout: 10 143654 14.64 MB/sec execute 407 sec latency 52.677 ms 2024-09-11T00:28:51.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:28:50 smithi012 ceph-mon[30333]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T00:28:51.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:28:50 smithi178 ceph-mon[35919]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T00:28:52.232 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:28:51 smithi178 ceph-mon[35919]: pgmap v1053: 129 pgs: 129 active+clean; 707 MiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 7.2 MiB/s wr, 648 op/s 2024-09-11T00:28:52.254 INFO:tasks.workunit.client.0.smithi012.stdout: 10 144238 14.68 MB/sec execute 408 sec latency 201.131 ms 2024-09-11T00:28:52.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:28:51 smithi165 ceph-mon[35356]: pgmap v1053: 129 pgs: 129 active+clean; 707 MiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 7.2 MiB/s wr, 648 op/s 2024-09-11T00:28:52.298 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:28:51 smithi012 ceph-mon[30333]: pgmap v1053: 129 pgs: 129 active+clean; 707 MiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 7.2 MiB/s wr, 648 op/s 2024-09-11T00:28:53.254 INFO:tasks.workunit.client.0.smithi012.stdout: 10 144745 14.72 MB/sec execute 409 sec latency 376.567 ms 2024-09-11T00:28:54.113 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:28:53 smithi012 ceph-mon[30333]: pgmap v1054: 129 pgs: 129 active+clean; 703 MiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 6.8 MiB/s wr, 672 op/s 2024-09-11T00:28:54.254 INFO:tasks.workunit.client.0.smithi012.stdout: 10 145343 14.75 MB/sec execute 410 sec latency 143.730 ms 2024-09-11T00:28:54.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:28:53 smithi165 ceph-mon[35356]: pgmap v1054: 129 pgs: 129 active+clean; 703 MiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 6.8 MiB/s wr, 672 op/s 2024-09-11T00:28:54.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:28:53 smithi178 ceph-mon[35919]: pgmap v1054: 129 pgs: 129 active+clean; 703 MiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 6.8 MiB/s wr, 672 op/s 2024-09-11T00:28:55.254 INFO:tasks.workunit.client.0.smithi012.stdout: 10 146260 14.84 MB/sec execute 411 sec latency 117.412 ms 2024-09-11T00:28:56.139 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:28:55 smithi012 ceph-mon[30333]: pgmap v1055: 129 pgs: 129 active+clean; 703 MiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 6.1 MiB/s wr, 604 op/s 2024-09-11T00:28:56.254 INFO:tasks.workunit.client.0.smithi012.stdout: 10 146905 14.89 MB/sec execute 412 sec latency 267.276 ms 2024-09-11T00:28:56.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:28:55 smithi165 ceph-mon[35356]: pgmap v1055: 129 pgs: 129 active+clean; 703 MiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 6.1 MiB/s wr, 604 op/s 2024-09-11T00:28:56.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:28:55 smithi178 ceph-mon[35919]: pgmap v1055: 129 pgs: 129 active+clean; 703 MiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 6.1 MiB/s wr, 604 op/s 2024-09-11T00:28:57.254 INFO:tasks.workunit.client.0.smithi012.stdout: 10 147606 14.95 MB/sec execute 413 sec latency 172.656 ms 2024-09-11T00:28:58.010 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:28:57 smithi012 ceph-mon[30333]: pgmap v1056: 129 pgs: 129 active+clean; 719 MiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 7.4 MiB/s wr, 835 op/s 2024-09-11T00:28:58.010 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:28:57 smithi012 ceph-mon[30333]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T00:28:58.254 INFO:tasks.workunit.client.0.smithi012.stdout: 10 148357 15.01 MB/sec execute 414 sec latency 362.053 ms 2024-09-11T00:28:58.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:28:57 smithi165 ceph-mon[35356]: pgmap v1056: 129 pgs: 129 active+clean; 719 MiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 7.4 MiB/s wr, 835 op/s 2024-09-11T00:28:58.283 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:28:57 smithi165 ceph-mon[35356]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T00:28:58.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:28:57 smithi178 ceph-mon[35919]: pgmap v1056: 129 pgs: 129 active+clean; 719 MiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 7.4 MiB/s wr, 835 op/s 2024-09-11T00:28:58.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:28:57 smithi178 ceph-mon[35919]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T00:28:59.064 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:28:58 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T00:28:59.255 INFO:tasks.workunit.client.0.smithi012.stdout: 10 148647 15.01 MB/sec execute 415 sec latency 543.236 ms 2024-09-11T00:28:59.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:28:58 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T00:28:59.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:28:58 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T00:28:59.940 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:28:59 smithi012 ceph-mon[30333]: pgmap v1057: 129 pgs: 129 active+clean; 719 MiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 3.6 MiB/s wr, 558 op/s 2024-09-11T00:29:00.255 INFO:tasks.workunit.client.0.smithi012.stdout: 10 148649 14.98 MB/sec execute 416 sec latency 1543.305 ms 2024-09-11T00:29:00.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:28:59 smithi165 ceph-mon[35356]: pgmap v1057: 129 pgs: 129 active+clean; 719 MiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 3.6 MiB/s wr, 558 op/s 2024-09-11T00:29:00.325 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:00.320+0000 7f1878a00640 1 -- 172.21.15.12:0/1371580158 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f186c0a5480 msgr2=0x7f186c0a5900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:29:00.326 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:00.320+0000 7f1878a00640 1 --2- 172.21.15.12:0/1371580158 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f186c0a5480 0x7f186c0a5900 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f1874068a60 tx=0x7f187408f340 comp rx=0 tx=0).stop 2024-09-11T00:29:00.326 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:00.320+0000 7f1878a00640 1 -- 172.21.15.12:0/1371580158 shutdown_connections 2024-09-11T00:29:00.326 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:00.320+0000 7f1878a00640 1 --2- 172.21.15.12:0/1371580158 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f186c0a5e40 0x7f186c0aa7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:29:00.326 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:00.320+0000 7f1878a00640 1 --2- 172.21.15.12:0/1371580158 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f186c0a5480 0x7f186c0a5900 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:29:00.326 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:00.320+0000 7f1878a00640 1 --2- 172.21.15.12:0/1371580158 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f186c0a4440 0x7f186c0a4840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:29:00.326 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:00.320+0000 7f1878a00640 1 -- 172.21.15.12:0/1371580158 >> 172.21.15.12:0/1371580158 conn(0x7f186c09fbd0 msgr2=0x7f186c0a2010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:29:00.326 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:00.320+0000 7f1878a00640 1 -- 172.21.15.12:0/1371580158 shutdown_connections 2024-09-11T00:29:00.326 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:00.320+0000 7f1878a00640 1 -- 172.21.15.12:0/1371580158 wait complete. 2024-09-11T00:29:00.326 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:00.322+0000 7f1878a00640 1 Processor -- start 2024-09-11T00:29:00.326 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:00.322+0000 7f1878a00640 1 -- start start 2024-09-11T00:29:00.327 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:00.322+0000 7f1878a00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f186c0a4440 0x7f186c00a860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:29:00.327 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:00.323+0000 7f1878a00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f186c0a5480 0x7f186c00aeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:29:00.327 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:00.323+0000 7f1878a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f186c0a5e40 0x7f186c00bae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:29:00.327 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:00.323+0000 7f1872800640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f186c0a4440 0x7f186c00a860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:29:00.329 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:00.323+0000 7f1872800640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f186c0a4440 0x7f186c00a860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.12:41334/0 (socket says 172.21.15.12:41334) 2024-09-11T00:29:00.329 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:00.323+0000 7f1878a00640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f186c0b8700 con 0x7f186c0a5480 2024-09-11T00:29:00.329 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:00.323+0000 7f1878a00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f186c0b8580 con 0x7f186c0a5e40 2024-09-11T00:29:00.329 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:00.323+0000 7f1878a00640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f186c0b8880 con 0x7f186c0a4440 2024-09-11T00:29:00.329 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:00.323+0000 7f1871e00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f186c0a5480 0x7f186c00aeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:29:00.329 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:00.323+0000 7f1872800640 1 -- 172.21.15.12:0/2594729874 learned_addr learned my addr 172.21.15.12:0/2594729874 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:29:00.330 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:00.323+0000 7f1873200640 1 --2- 172.21.15.12:0/2594729874 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f186c0a5e40 0x7f186c00bae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:29:00.330 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:00.324+0000 7f1872800640 1 -- 172.21.15.12:0/2594729874 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f186c0a5e40 msgr2=0x7f186c00bae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:29:00.330 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:00.324+0000 7f1872800640 1 --2- 172.21.15.12:0/2594729874 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f186c0a5e40 0x7f186c00bae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:29:00.330 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:00.324+0000 7f1872800640 1 -- 172.21.15.12:0/2594729874 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f186c0a5480 msgr2=0x7f186c00aeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:29:00.330 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:00.324+0000 7f1872800640 1 --2- 172.21.15.12:0/2594729874 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f186c0a5480 0x7f186c00aeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:29:00.330 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:00.324+0000 7f1872800640 1 -- 172.21.15.12:0/2594729874 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f186c00c3a0 con 0x7f186c0a4440 2024-09-11T00:29:00.330 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:00.324+0000 7f1872800640 1 --2- 172.21.15.12:0/2594729874 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f186c0a4440 0x7f186c00a860 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f186000c920 tx=0x7f186000cde0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:29:00.330 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:00.326+0000 7f186b400640 1 -- 172.21.15.12:0/2594729874 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1860007c70 con 0x7f186c0a4440 2024-09-11T00:29:00.330 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:00.326+0000 7f186b400640 1 -- 172.21.15.12:0/2594729874 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1860007e10 con 0x7f186c0a4440 2024-09-11T00:29:00.330 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:00.326+0000 7f186b400640 1 -- 172.21.15.12:0/2594729874 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1860020460 con 0x7f186c0a4440 2024-09-11T00:29:00.330 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:00.326+0000 7f1878a00640 1 -- 172.21.15.12:0/2594729874 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f186c00c690 con 0x7f186c0a4440 2024-09-11T00:29:00.330 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:00.326+0000 7f1878a00640 1 -- 172.21.15.12:0/2594729874 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f186c012d60 con 0x7f186c0a4440 2024-09-11T00:29:00.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:28:59 smithi178 ceph-mon[35919]: pgmap v1057: 129 pgs: 129 active+clean; 719 MiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 3.6 MiB/s wr, 558 op/s 2024-09-11T00:29:00.333 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:00.331+0000 7f186b400640 1 -- 172.21.15.12:0/2594729874 <== mon.1 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7f1860030030 con 0x7f186c0a4440 2024-09-11T00:29:00.337 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:00.333+0000 7f186b400640 1 --2- 172.21.15.12:0/2594729874 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f18580b5430 0x7f18580b78f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:29:00.337 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:00.334+0000 7f186b400640 1 -- 172.21.15.12:0/2594729874 <== mon.1 v2:172.21.15.178:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9208+0+0 (secure 0 0 0) 0x7f18600f6b00 con 0x7f186c0a4440 2024-09-11T00:29:00.338 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:00.335+0000 7f1871e00640 1 --2- 172.21.15.12:0/2594729874 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f18580b5430 0x7f18580b78f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:29:00.339 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:00.337+0000 7f1871e00640 1 --2- 172.21.15.12:0/2594729874 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f18580b5430 0x7f18580b78f0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f18740527e0 tx=0x7f187409a040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:29:00.340 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:00.338+0000 7f187c23f740 1 Processor -- start 2024-09-11T00:29:00.340 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:00.338+0000 7f187c23f740 1 -- start start 2024-09-11T00:29:00.340 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:00.338+0000 7f187c23f740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55dccd09a970 0x55dccd09cd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:29:00.341 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:00.338+0000 7f187c23f740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55dccd09d2a0 0x55dccd09f690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:29:00.341 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:00.338+0000 7f187c23f740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55dccd09fce0 0x55dccd0a21a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:29:00.341 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:00.338+0000 7f187c23f740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55dccd00d6b0 con 0x55dccd09a970 2024-09-11T00:29:00.341 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:00.338+0000 7f187c23f740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55dcccffcae0 con 0x55dccd09fce0 2024-09-11T00:29:00.341 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:00.338+0000 7f187c23f740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55dccd0008f0 con 0x55dccd09d2a0 2024-09-11T00:29:00.341 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:00.339+0000 7f1873200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55dccd09fce0 0x55dccd0a21a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:29:00.341 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:00.339+0000 7f1873200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55dccd09fce0 0x55dccd0a21a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.12:54700/0 (socket says 172.21.15.12:54700) 2024-09-11T00:29:00.342 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:00.339+0000 7f1873200640 1 -- 172.21.15.12:0/3124063792 learned_addr learned my addr 172.21.15.12:0/3124063792 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:29:00.342 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:00.339+0000 7f1871e00640 1 --2- 172.21.15.12:0/3124063792 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55dccd09d2a0 0x55dccd09f690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:29:00.342 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:00.339+0000 7f1873200640 1 -- 172.21.15.12:0/3124063792 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55dccd09d2a0 msgr2=0x55dccd09f690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:29:00.342 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:00.339+0000 7f1873200640 1 --2- 172.21.15.12:0/3124063792 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55dccd09d2a0 0x55dccd09f690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:29:00.342 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:00.339+0000 7f1873200640 1 -- 172.21.15.12:0/3124063792 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55dccd09a970 msgr2=0x55dccd09cd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:29:00.342 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:00.339+0000 7f1873200640 1 --2- 172.21.15.12:0/3124063792 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55dccd09a970 0x55dccd09cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:29:00.342 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:00.339+0000 7f1873200640 1 -- 172.21.15.12:0/3124063792 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dcccfc9780 con 0x55dccd09fce0 2024-09-11T00:29:00.343 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:00.340+0000 7f1871e00640 1 --2- 172.21.15.12:0/3124063792 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55dccd09d2a0 0x55dccd09f690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T00:29:00.343 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:00.341+0000 7f1873200640 1 --2- 172.21.15.12:0/3124063792 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55dccd09fce0 0x55dccd0a21a0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f186400b6b0 tx=0x7f186400bb70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2a64c6e7d2eaa433 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:29:00.344 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:00.342+0000 7f185f400640 1 -- 172.21.15.12:0/3124063792 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1864003bd0 con 0x55dccd09fce0 2024-09-11T00:29:00.344 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:00.342+0000 7f185f400640 1 -- 172.21.15.12:0/3124063792 <== mon.2 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1864015070 con 0x55dccd09fce0 2024-09-11T00:29:00.344 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:00.342+0000 7f185f400640 1 -- 172.21.15.12:0/3124063792 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18640106a0 con 0x55dccd09fce0 2024-09-11T00:29:00.345 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:00.343+0000 7f187c23f740 1 -- 172.21.15.12:0/3124063792 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55dccd09fce0 msgr2=0x55dccd0a21a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:29:00.345 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:00.343+0000 7f187c23f740 1 --2- 172.21.15.12:0/3124063792 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55dccd09fce0 0x55dccd0a21a0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f186400b6b0 tx=0x7f186400bb70 comp rx=0 tx=0).stop 2024-09-11T00:29:00.347 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:00.345+0000 7f187c23f740 1 -- 172.21.15.12:0/3124063792 shutdown_connections 2024-09-11T00:29:00.347 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:00.345+0000 7f187c23f740 1 --2- 172.21.15.12:0/3124063792 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55dccd09fce0 0x55dccd0a21a0 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:29:00.347 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:00.345+0000 7f187c23f740 1 --2- 172.21.15.12:0/3124063792 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55dccd09d2a0 0x55dccd09f690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:29:00.347 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:00.345+0000 7f187c23f740 1 --2- 172.21.15.12:0/3124063792 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55dccd09a970 0x55dccd09cd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:29:00.348 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:00.345+0000 7f187c23f740 1 -- 172.21.15.12:0/3124063792 >> 172.21.15.12:0/3124063792 conn(0x55dccd095ac0 msgr2=0x55dccd096850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:29:00.348 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:00.345+0000 7f187c23f740 1 -- 172.21.15.12:0/3124063792 shutdown_connections 2024-09-11T00:29:00.348 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:00.346+0000 7f187c23f740 1 -- 172.21.15.12:0/3124063792 wait complete. 2024-09-11T00:29:00.349 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:00.347+0000 7f187c23f740 1 -- start start 2024-09-11T00:29:00.349 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:00.347+0000 7f187c23f740 1 Processor -- start 2024-09-11T00:29:00.350 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:00.348+0000 7f187c23f740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55dccd09a970 0x55dccd0a4110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:29:00.351 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:00.348+0000 7f187c23f740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55dccd09d2a0 0x55dccd0a47b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:29:00.351 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:00.348+0000 7f187c23f740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55dccd09fce0 0x55dccd0a5380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:29:00.351 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:00.348+0000 7f187c23f740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55dcccf34ab0 con 0x55dccd09a970 2024-09-11T00:29:00.351 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:00.348+0000 7f187c23f740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55dcccfa3570 con 0x55dccd09d2a0 2024-09-11T00:29:00.351 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:00.348+0000 7f187c23f740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55dccd0a2cc0 con 0x55dccd09fce0 2024-09-11T00:29:00.351 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:00.348+0000 7f187c23f740 10 client.0 _ll_register_callbacks cb 0x55dcccfb2100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:29:00.352 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:00.349+0000 7f1871e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55dccd09d2a0 0x55dccd0a47b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:29:00.352 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:00.349+0000 7f1871e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55dccd09d2a0 0x55dccd0a47b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.12:54712/0 (socket says 172.21.15.12:54712) 2024-09-11T00:29:00.352 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:00.349+0000 7f1871e00640 1 -- 172.21.15.12:0/1766863681 learned_addr learned my addr 172.21.15.12:0/1766863681 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:29:00.352 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:00.349+0000 7f1873200640 1 --2- 172.21.15.12:0/1766863681 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55dccd09fce0 0x55dccd0a5380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:29:00.352 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:00.349+0000 7f1871e00640 1 -- 172.21.15.12:0/1766863681 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55dccd09fce0 msgr2=0x55dccd0a5380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:29:00.352 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:00.349+0000 7f1871e00640 1 --2- 172.21.15.12:0/1766863681 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55dccd09fce0 0x55dccd0a5380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:29:00.353 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:00.349+0000 7f1871e00640 1 -- 172.21.15.12:0/1766863681 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55dccd09a970 msgr2=0x55dccd0a4110 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-11T00:29:00.353 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:00.349+0000 7f1872800640 1 --2- 172.21.15.12:0/1766863681 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55dccd09a970 0x55dccd0a4110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:29:00.354 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:00.352+0000 7f1871e00640 1 --2- 172.21.15.12:0/1766863681 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55dccd09a970 0x55dccd0a4110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:29:00.354 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:00.352+0000 7f1871e00640 1 -- 172.21.15.12:0/1766863681 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dcccd5e790 con 0x55dccd09d2a0 2024-09-11T00:29:00.354 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:00.352+0000 7f1871e00640 1 --2- 172.21.15.12:0/1766863681 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55dccd09d2a0 0x55dccd0a47b0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f1874068980 tx=0x7f187406b860 comp rx=0 tx=0).ready entity=mon.2 client_cookie=923c45b71d1dffa server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:29:00.355 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:00.353+0000 7f1853e00640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-09-11T00:29:00.355 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:00.353+0000 7f1853e00640 1 -- 172.21.15.12:0/1766863681 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18740b3060 con 0x55dccd09d2a0 2024-09-11T00:29:00.355 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:00.353+0000 7f1853e00640 1 -- 172.21.15.12:0/1766863681 <== mon.2 v2:172.21.15.165:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 9208+0+0 (secure 0 0 0) 0x7f18740a3810 con 0x55dccd09d2a0 2024-09-11T00:29:00.355 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:00.353+0000 7f187c23f740 1 -- 172.21.15.12:0/1766863681 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1864015070 con 0x55dccd09d2a0 2024-09-11T00:29:00.355 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:00.353+0000 7f1853e00640 1 -- 172.21.15.12:0/1766863681 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=96) ==== 24+0+0 (secure 0 0 0) 0x7f187406b5a0 con 0x55dccd09d2a0 2024-09-11T00:29:00.356 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:00.354+0000 7f187c23f740 10 client.27884 fetch_fsmap learned FSMap version 96 2024-09-11T00:29:00.356 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:00.354+0000 7f187c23f740 1 -- 172.21.15.12:0/1766863681 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=96,monmap=4+}) -- 0x55dcccdd7620 con 0x55dccd09d2a0 2024-09-11T00:29:00.356 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:00.354+0000 7f1853e00640 1 -- 172.21.15.12:0/1766863681 <== mon.2 v2:172.21.15.165:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1864015070 con 0x55dccd09d2a0 2024-09-11T00:29:00.358 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:00.355+0000 7f1853e00640 1 -- 172.21.15.12:0/1766863681 <== mon.2 v2:172.21.15.165:3300/0 5 ==== fsmap(e 96) ==== 8394+0+0 (secure 0 0 0) 0x7f18740a2070 con 0x55dccd09d2a0 2024-09-11T00:29:00.358 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:00.355+0000 7f187c23f740 10 client.27884 fetch_fsmap finished waiting for FSMap version 96 2024-09-11T00:29:00.358 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:00.355+0000 7f187c23f740 10 client.27884 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T00:29:00.358 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:00.355+0000 7f187c23f740 20 client.27884 populate_metadata read hostname 'smithi012' 2024-09-11T00:29:00.358 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:00.355+0000 7f187c23f740 1 --2- 172.21.15.12:0/1766863681 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55dccd0af990 0x55dccd0b1df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:29:00.358 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:00.356+0000 7f187c23f740 4 client.27884 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T00:29:00.358 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:00.356+0000 7f187c23f740 1 -- 172.21.15.12:0/1766863681 --> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55dccd00f600 con 0x55dccd0af990 2024-09-11T00:29:00.358 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:00.356+0000 7f1872800640 1 --2- 172.21.15.12:0/1766863681 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55dccd0af990 0x55dccd0b1df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:29:00.359 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:00.356+0000 7f1853e00640 10 client.27884 ms_handle_connect on v2:172.21.15.165:6838/501312984 2024-09-11T00:29:00.359 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:00.357+0000 7f1872800640 1 --2- 172.21.15.12:0/1766863681 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55dccd0af990 0x55dccd0b1df0 crc :-1 s=READY pgs=303 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f8c955a2f108fb72 server_cookie=1cb4828daafcc625 in_seq=0 out_seq=0 2024-09-11T00:29:00.360 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:00.358+0000 7f1853e00640 1 -- 172.21.15.12:0/1766863681 <== mds.0 v2:172.21.15.165:6838/501312984 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55dccd00f600 con 0x55dccd0af990 2024-09-11T00:29:00.360 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:00.358+0000 7f1853e00640 10 client.27884 handle_command_reply: tid=0 2024-09-11T00:29:00.360 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:00.358+0000 7f187c23f740 1 client.27884 shutdown 2024-09-11T00:29:00.361 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:00.358+0000 7f187c23f740 1 -- 172.21.15.12:0/1766863681 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55dccd09d2a0 msgr2=0x55dccd0a47b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:29:00.362 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:00.358+0000 7f187c23f740 1 --2- 172.21.15.12:0/1766863681 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55dccd09d2a0 0x55dccd0a47b0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f1874068980 tx=0x7f187406b860 comp rx=0 tx=0).stop 2024-09-11T00:29:00.362 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:00.359+0000 7f187c23f740 1 -- 172.21.15.12:0/1766863681 shutdown_connections 2024-09-11T00:29:00.362 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:00.359+0000 7f187c23f740 1 --2- 172.21.15.12:0/1766863681 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55dccd0af990 0x55dccd0b1df0 crc :-1 s=READY pgs=303 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:29:00.362 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:00.359+0000 7f187c23f740 1 --2- 172.21.15.12:0/1766863681 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55dccd09fce0 0x55dccd0a5380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:29:00.362 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:00.359+0000 7f187c23f740 1 --2- 172.21.15.12:0/1766863681 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55dccd09d2a0 0x55dccd0a47b0 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:29:00.362 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:00.359+0000 7f187c23f740 1 --2- 172.21.15.12:0/1766863681 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55dccd09a970 0x55dccd0a4110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:29:00.362 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:00.359+0000 7f187c23f740 1 -- 172.21.15.12:0/1766863681 >> 172.21.15.12:0/1766863681 conn(0x55dccd095ac0 msgr2=0x55dccd094bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:29:00.362 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:00.359+0000 7f1853e00640 0 client.27884 ms_handle_reset on v2:172.21.15.165:6838/501312984 2024-09-11T00:29:00.362 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:00.359+0000 7f187c23f740 1 -- 172.21.15.12:0/1766863681 shutdown_connections 2024-09-11T00:29:00.362 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:00.359+0000 7f187c23f740 1 -- 172.21.15.12:0/1766863681 wait complete. 2024-09-11T00:29:00.362 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:00.359+0000 7f187c23f740 20 client.27884 trim_cache size 0 max 16384 2024-09-11T00:29:00.398 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:00.396+0000 7f187c23f740 1 Processor -- start 2024-09-11T00:29:00.399 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:00.396+0000 7f187c23f740 1 -- start start 2024-09-11T00:29:00.399 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:00.396+0000 7f187c23f740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55dccd09a970 0x55dccd0a00c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:29:00.399 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:00.396+0000 7f187c23f740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55dccd09d2a0 0x55dccd0a0600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:29:00.399 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:00.396+0000 7f187c23f740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55dccd09fce0 0x55dccd0978f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:29:00.400 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:00.396+0000 7f187c23f740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f186c0b2af0 con 0x55dccd09d2a0 2024-09-11T00:29:00.400 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:00.396+0000 7f187c23f740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f186c0b46f0 con 0x55dccd09fce0 2024-09-11T00:29:00.400 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:00.396+0000 7f187c23f740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f186c0b31c0 con 0x55dccd09a970 2024-09-11T00:29:00.400 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:00.397+0000 7f1872800640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55dccd09a970 0x55dccd0a00c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:29:00.400 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:00.397+0000 7f1871e00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55dccd09d2a0 0x55dccd0a0600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:29:00.400 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:00.397+0000 7f1872800640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55dccd09a970 0x55dccd0a00c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.12:41360/0 (socket says 172.21.15.12:41360) 2024-09-11T00:29:00.400 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:00.397+0000 7f1872800640 1 -- 172.21.15.12:0/3007881974 learned_addr learned my addr 172.21.15.12:0/3007881974 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:29:00.400 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:00.397+0000 7f1873200640 1 --2- 172.21.15.12:0/3007881974 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55dccd09fce0 0x55dccd0978f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:29:00.401 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:00.397+0000 7f1873200640 1 -- 172.21.15.12:0/3007881974 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55dccd09a970 msgr2=0x55dccd0a00c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:29:00.401 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:00.397+0000 7f1873200640 1 --2- 172.21.15.12:0/3007881974 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55dccd09a970 0x55dccd0a00c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:29:00.401 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:00.398+0000 7f1873200640 1 -- 172.21.15.12:0/3007881974 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55dccd09d2a0 msgr2=0x55dccd0a0600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:29:00.401 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:00.398+0000 7f1873200640 1 --2- 172.21.15.12:0/3007881974 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55dccd09d2a0 0x55dccd0a0600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:29:00.402 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:00.398+0000 7f1873200640 1 -- 172.21.15.12:0/3007881974 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dccd0a56f0 con 0x55dccd09fce0 2024-09-11T00:29:00.402 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:00.398+0000 7f1871e00640 1 --2- 172.21.15.12:0/3007881974 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55dccd09d2a0 0x55dccd0a0600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:29:00.402 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:00.398+0000 7f1873200640 1 --2- 172.21.15.12:0/3007881974 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55dccd09fce0 0x55dccd0978f0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f1864000c00 tx=0x7f186400b350 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6fc9e838a233c4be server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:29:00.402 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:00.399+0000 7f185ea00640 1 -- 172.21.15.12:0/3007881974 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1864027070 con 0x55dccd09fce0 2024-09-11T00:29:00.402 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:00.399+0000 7f185ea00640 1 -- 172.21.15.12:0/3007881974 <== mon.2 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1864015050 con 0x55dccd09fce0 2024-09-11T00:29:00.402 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:00.399+0000 7f185ea00640 1 -- 172.21.15.12:0/3007881974 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1864004c10 con 0x55dccd09fce0 2024-09-11T00:29:00.402 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:00.400+0000 7f187c23f740 1 -- 172.21.15.12:0/3007881974 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55dccd09fce0 msgr2=0x55dccd0978f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:29:00.402 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:00.400+0000 7f187c23f740 1 --2- 172.21.15.12:0/3007881974 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55dccd09fce0 0x55dccd0978f0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f1864000c00 tx=0x7f186400b350 comp rx=0 tx=0).stop 2024-09-11T00:29:00.405 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:00.402+0000 7f187c23f740 1 -- 172.21.15.12:0/3007881974 shutdown_connections 2024-09-11T00:29:00.405 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:00.402+0000 7f187c23f740 1 --2- 172.21.15.12:0/3007881974 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55dccd09fce0 0x55dccd0978f0 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:29:00.405 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:00.402+0000 7f187c23f740 1 --2- 172.21.15.12:0/3007881974 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55dccd09d2a0 0x55dccd0a0600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:29:00.405 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:00.402+0000 7f187c23f740 1 --2- 172.21.15.12:0/3007881974 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55dccd09a970 0x55dccd0a00c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:29:00.405 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:00.402+0000 7f187c23f740 1 -- 172.21.15.12:0/3007881974 >> 172.21.15.12:0/3007881974 conn(0x55dccd095ac0 msgr2=0x55dccd099e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:29:00.405 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:00.403+0000 7f187c23f740 1 -- 172.21.15.12:0/3007881974 shutdown_connections 2024-09-11T00:29:00.407 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:00.403+0000 7f187c23f740 1 -- 172.21.15.12:0/3007881974 wait complete. 2024-09-11T00:29:00.407 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:00.404+0000 7f187c23f740 1 -- start start 2024-09-11T00:29:00.407 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:00.404+0000 7f187c23f740 1 Processor -- start 2024-09-11T00:29:00.408 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:00.406+0000 7f187c23f740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55dccd09a970 0x55dccd0aba20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:29:00.411 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:00.408+0000 7f187c23f740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55dccd09d2a0 0x55dccd0aa720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:29:00.411 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:00.408+0000 7f187c23f740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55dccd09fce0 0x55dccd0aac60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:29:00.411 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:00.408+0000 7f1872800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55dccd09a970 0x55dccd0aba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:29:00.411 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:00.408+0000 7f1872800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55dccd09a970 0x55dccd0aba20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.12:54732/0 (socket says 172.21.15.12:54732) 2024-09-11T00:29:00.411 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:00.408+0000 7f187c23f740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f186c0b4020 con 0x55dccd09d2a0 2024-09-11T00:29:00.411 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:00.408+0000 7f187c23f740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f186c0b3950 con 0x55dccd09a970 2024-09-11T00:29:00.411 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:00.408+0000 7f187c23f740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f186c0b0ad0 con 0x55dccd09fce0 2024-09-11T00:29:00.411 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:00.408+0000 7f187c23f740 10 client.0 _ll_register_callbacks cb 0x7f18740696d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:29:00.411 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:00.408+0000 7f1872800640 1 -- 172.21.15.12:0/3768453109 learned_addr learned my addr 172.21.15.12:0/3768453109 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:29:00.411 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:00.408+0000 7f1873200640 1 --2- 172.21.15.12:0/3768453109 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55dccd09fce0 0x55dccd0aac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:29:00.413 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:00.409+0000 7f1871e00640 1 --2- 172.21.15.12:0/3768453109 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55dccd09d2a0 0x55dccd0aa720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:29:00.413 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:00.409+0000 7f1872800640 1 -- 172.21.15.12:0/3768453109 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55dccd09fce0 msgr2=0x55dccd0aac60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:29:00.413 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:00.409+0000 7f1872800640 1 --2- 172.21.15.12:0/3768453109 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55dccd09fce0 0x55dccd0aac60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:29:00.413 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:00.409+0000 7f1872800640 1 -- 172.21.15.12:0/3768453109 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55dccd09d2a0 msgr2=0x55dccd0aa720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:29:00.413 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:00.409+0000 7f1872800640 1 --2- 172.21.15.12:0/3768453109 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55dccd09d2a0 0x55dccd0aa720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:29:00.413 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:00.409+0000 7f1872800640 1 -- 172.21.15.12:0/3768453109 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dccd0ab2a0 con 0x55dccd09a970 2024-09-11T00:29:00.413 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:00.409+0000 7f1872800640 1 --2- 172.21.15.12:0/3768453109 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55dccd09a970 0x55dccd0aba20 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f18600bcc80 tx=0x7f1860005850 comp rx=0 tx=0).ready entity=mon.2 client_cookie=665cca39924f925d server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:29:00.413 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:00.409+0000 7f1853e00640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-09-11T00:29:00.413 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:00.410+0000 7f1853e00640 1 -- 172.21.15.12:0/3768453109 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18600d6020 con 0x55dccd09a970 2024-09-11T00:29:00.413 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:00.410+0000 7f1853e00640 1 -- 172.21.15.12:0/3768453109 <== mon.2 v2:172.21.15.165:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 9208+0+0 (secure 0 0 0) 0x7f18600a82d0 con 0x55dccd09a970 2024-09-11T00:29:00.413 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:00.410+0000 7f187c23f740 1 -- 172.21.15.12:0/3768453109 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1864015050 con 0x55dccd09a970 2024-09-11T00:29:00.414 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:00.410+0000 7f1853e00640 1 -- 172.21.15.12:0/3768453109 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=96) ==== 24+0+0 (secure 0 0 0) 0x7f18600bca90 con 0x55dccd09a970 2024-09-11T00:29:00.414 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:00.410+0000 7f187c23f740 10 client.27890 fetch_fsmap learned FSMap version 96 2024-09-11T00:29:00.414 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:00.410+0000 7f187c23f740 1 -- 172.21.15.12:0/3768453109 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=96,monmap=4+}) -- 0x55dccd11c8c0 con 0x55dccd09a970 2024-09-11T00:29:00.414 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:00.411+0000 7f1853e00640 1 -- 172.21.15.12:0/3768453109 <== mon.2 v2:172.21.15.165:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1864015050 con 0x55dccd09a970 2024-09-11T00:29:00.414 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:00.411+0000 7f1853e00640 1 -- 172.21.15.12:0/3768453109 <== mon.2 v2:172.21.15.165:3300/0 5 ==== fsmap(e 96) ==== 8394+0+0 (secure 0 0 0) 0x7f18600b88d0 con 0x55dccd09a970 2024-09-11T00:29:00.415 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:00.412+0000 7f187c23f740 10 client.27890 fetch_fsmap finished waiting for FSMap version 96 2024-09-11T00:29:00.415 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:00.412+0000 7f187c23f740 10 client.27890 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T00:29:00.415 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:00.412+0000 7f187c23f740 20 client.27890 populate_metadata read hostname 'smithi012' 2024-09-11T00:29:00.415 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:00.412+0000 7f187c23f740 1 --2- 172.21.15.12:0/3768453109 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55dccd11d160 0x55dccd11f580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:29:00.415 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:00.412+0000 7f187c23f740 4 client.27890 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T00:29:00.416 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:00.412+0000 7f187c23f740 1 -- 172.21.15.12:0/3768453109 --> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55dccd00b0b0 con 0x55dccd11d160 2024-09-11T00:29:00.416 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:00.413+0000 7f1871e00640 1 --2- 172.21.15.12:0/3768453109 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55dccd11d160 0x55dccd11f580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:29:00.416 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:00.414+0000 7f1871e00640 1 --2- 172.21.15.12:0/3768453109 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55dccd11d160 0x55dccd11f580 crc :-1 s=READY pgs=304 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=55010200165505a server_cookie=438fb5779167e6f1 in_seq=0 out_seq=0 2024-09-11T00:29:00.416 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:00.414+0000 7f1853e00640 10 client.27890 ms_handle_connect on v2:172.21.15.165:6838/501312984 2024-09-11T00:29:00.797 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:00.795+0000 7f1853e00640 1 -- 172.21.15.12:0/3768453109 <== mds.0 v2:172.21.15.165:6838/501312984 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55dccd00b0b0 con 0x55dccd11d160 2024-09-11T00:29:00.797 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:00.795+0000 7f1853e00640 10 client.27890 handle_command_reply: tid=0 2024-09-11T00:29:00.798 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:00.795+0000 7f187c23f740 1 client.27890 shutdown 2024-09-11T00:29:00.798 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:00.795+0000 7f187c23f740 1 -- 172.21.15.12:0/3768453109 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55dccd09a970 msgr2=0x55dccd0aba20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:29:00.798 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:00.795+0000 7f187c23f740 1 --2- 172.21.15.12:0/3768453109 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55dccd09a970 0x55dccd0aba20 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f18600bcc80 tx=0x7f1860005850 comp rx=0 tx=0).stop 2024-09-11T00:29:00.798 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:00.795+0000 7f187c23f740 1 -- 172.21.15.12:0/3768453109 shutdown_connections 2024-09-11T00:29:00.798 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:00.795+0000 7f187c23f740 1 --2- 172.21.15.12:0/3768453109 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55dccd11d160 0x55dccd11f580 crc :-1 s=READY pgs=304 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:29:00.798 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:00.795+0000 7f1853e00640 0 client.27890 ms_handle_reset on v2:172.21.15.165:6838/501312984 2024-09-11T00:29:00.799 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:00.795+0000 7f187c23f740 1 --2- 172.21.15.12:0/3768453109 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55dccd09fce0 0x55dccd0aac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:29:00.799 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:00.795+0000 7f187c23f740 1 --2- 172.21.15.12:0/3768453109 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55dccd09d2a0 0x55dccd0aa720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:29:00.799 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:00.795+0000 7f187c23f740 1 --2- 172.21.15.12:0/3768453109 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55dccd09a970 0x55dccd0aba20 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:29:00.799 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:00.795+0000 7f187c23f740 1 -- 172.21.15.12:0/3768453109 >> 172.21.15.12:0/3768453109 conn(0x55dccd095ac0 msgr2=0x55dccd099a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:29:00.799 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:00.796+0000 7f187c23f740 1 -- 172.21.15.12:0/3768453109 shutdown_connections 2024-09-11T00:29:00.799 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:00.796+0000 7f187c23f740 1 -- 172.21.15.12:0/3768453109 wait complete. 2024-09-11T00:29:00.799 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:00.797+0000 7f187c23f740 20 client.27890 trim_cache size 0 max 16384 2024-09-11T00:29:00.800 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:{ 2024-09-11T00:29:00.800 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "message": "", 2024-09-11T00:29:00.800 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "return_code": 0 2024-09-11T00:29:00.800 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:} 2024-09-11T00:29:00.801 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:00.798+0000 7f1878a00640 1 -- 172.21.15.12:0/2594729874 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f18580b5430 msgr2=0x7f18580b78f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:29:00.801 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:00.798+0000 7f1878a00640 1 --2- 172.21.15.12:0/2594729874 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f18580b5430 0x7f18580b78f0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f18740527e0 tx=0x7f187409a040 comp rx=0 tx=0).stop 2024-09-11T00:29:00.801 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:00.798+0000 7f1878a00640 1 -- 172.21.15.12:0/2594729874 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f186c0a4440 msgr2=0x7f186c00a860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:29:00.801 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:00.798+0000 7f1878a00640 1 --2- 172.21.15.12:0/2594729874 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f186c0a4440 0x7f186c00a860 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f186000c920 tx=0x7f186000cde0 comp rx=0 tx=0).stop 2024-09-11T00:29:00.801 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:00.799+0000 7f1878a00640 1 -- 172.21.15.12:0/2594729874 shutdown_connections 2024-09-11T00:29:00.801 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:00.799+0000 7f1878a00640 1 --2- 172.21.15.12:0/2594729874 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f18580b5430 0x7f18580b78f0 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:29:00.801 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:00.799+0000 7f1878a00640 1 --2- 172.21.15.12:0/2594729874 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f186c0a5e40 0x7f186c00bae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:29:00.801 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:00.799+0000 7f1878a00640 1 --2- 172.21.15.12:0/2594729874 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f186c0a5480 0x7f186c00aeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:29:00.801 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:00.799+0000 7f1878a00640 1 --2- 172.21.15.12:0/2594729874 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f186c0a4440 0x7f186c00a860 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:29:00.802 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:00.799+0000 7f1878a00640 1 -- 172.21.15.12:0/2594729874 >> 172.21.15.12:0/2594729874 conn(0x7f186c09fbd0 msgr2=0x7f186c0a6770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:29:00.802 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:00.799+0000 7f1878a00640 1 -- 172.21.15.12:0/2594729874 shutdown_connections 2024-09-11T00:29:00.802 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:00.799+0000 7f1878a00640 1 -- 172.21.15.12:0/2594729874 wait complete. 2024-09-11T00:29:01.255 INFO:tasks.workunit.client.0.smithi012.stdout: 10 148649 14.94 MB/sec execute 417 sec latency 2543.366 ms 2024-09-11T00:29:02.070 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:29:01 smithi012 ceph-mon[30333]: pgmap v1058: 129 pgs: 129 active+clean; 769 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 MiB/s rd, 8.4 MiB/s wr, 694 op/s 2024-09-11T00:29:02.255 INFO:tasks.workunit.client.0.smithi012.stdout: 10 148651 14.91 MB/sec execute 418 sec latency 3543.438 ms 2024-09-11T00:29:02.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:29:01 smithi165 ceph-mon[35356]: pgmap v1058: 129 pgs: 129 active+clean; 769 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 MiB/s rd, 8.4 MiB/s wr, 694 op/s 2024-09-11T00:29:02.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:29:01 smithi178 ceph-mon[35919]: pgmap v1058: 129 pgs: 129 active+clean; 769 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 MiB/s rd, 8.4 MiB/s wr, 694 op/s 2024-09-11T00:29:03.255 INFO:tasks.workunit.client.0.smithi012.stdout: 10 148652 14.87 MB/sec execute 419 sec latency 4543.488 ms 2024-09-11T00:29:04.015 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:29:03 smithi012 ceph-mon[30333]: pgmap v1059: 129 pgs: 129 active+clean; 794 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 11 MiB/s wr, 674 op/s 2024-09-11T00:29:04.255 INFO:tasks.workunit.client.0.smithi012.stdout: 10 148652 14.83 MB/sec execute 420 sec latency 5543.535 ms 2024-09-11T00:29:04.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:29:03 smithi165 ceph-mon[35356]: pgmap v1059: 129 pgs: 129 active+clean; 794 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 11 MiB/s wr, 674 op/s 2024-09-11T00:29:04.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:29:03 smithi178 ceph-mon[35919]: pgmap v1059: 129 pgs: 129 active+clean; 794 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 11 MiB/s wr, 674 op/s 2024-09-11T00:29:05.255 INFO:tasks.workunit.client.0.smithi012.stdout: 10 148652 14.80 MB/sec execute 421 sec latency 6543.588 ms 2024-09-11T00:29:06.029 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:29:05 smithi012 ceph-mon[30333]: pgmap v1060: 129 pgs: 129 active+clean; 791 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 10 MiB/s wr, 588 op/s 2024-09-11T00:29:06.030 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:29:05 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:29:06.030 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:29:05 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T00:29:06.030 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:29:05 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:29:06.255 INFO:tasks.workunit.client.0.smithi012.stdout: 10 148652 14.76 MB/sec execute 422 sec latency 7543.648 ms 2024-09-11T00:29:06.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:29:05 smithi165 ceph-mon[35356]: pgmap v1060: 129 pgs: 129 active+clean; 791 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 10 MiB/s wr, 588 op/s 2024-09-11T00:29:06.283 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:29:05 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:29:06.283 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:29:05 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T00:29:06.283 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:29:05 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:29:06.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:29:05 smithi178 ceph-mon[35919]: pgmap v1060: 129 pgs: 129 active+clean; 791 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 10 MiB/s wr, 588 op/s 2024-09-11T00:29:06.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:29:05 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:29:06.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:29:05 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T00:29:06.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:29:05 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:29:07.255 INFO:tasks.workunit.client.0.smithi012.stdout: 10 148663 14.74 MB/sec execute 423 sec latency 8543.702 ms 2024-09-11T00:29:08.219 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:29:07 smithi012 ceph-mon[30333]: pgmap v1061: 129 pgs: 129 active+clean; 754 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 10 MiB/s wr, 691 op/s 2024-09-11T00:29:08.255 INFO:tasks.workunit.client.0.smithi012.stdout: 10 148750 14.72 MB/sec execute 424 sec latency 9189.934 ms 2024-09-11T00:29:08.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:29:07 smithi165 ceph-mon[35356]: pgmap v1061: 129 pgs: 129 active+clean; 754 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 10 MiB/s wr, 691 op/s 2024-09-11T00:29:08.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:29:07 smithi178 ceph-mon[35919]: pgmap v1061: 129 pgs: 129 active+clean; 754 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 10 MiB/s wr, 691 op/s 2024-09-11T00:29:09.016 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:29:08 smithi012 ceph-mon[30333]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T00:29:09.255 INFO:tasks.workunit.client.0.smithi012.stdout: 10 148809 14.69 MB/sec execute 425 sec latency 9938.451 ms 2024-09-11T00:29:09.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:29:08 smithi165 ceph-mon[35356]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T00:29:09.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:29:08 smithi178 ceph-mon[35919]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T00:29:10.162 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:29:09 smithi012 ceph-mon[30333]: pgmap v1062: 129 pgs: 129 active+clean; 752 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 8.6 MiB/s wr, 450 op/s 2024-09-11T00:29:10.255 INFO:tasks.workunit.client.0.smithi012.stdout: 10 148813 14.65 MB/sec execute 426 sec latency 8249.712 ms 2024-09-11T00:29:10.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:29:09 smithi165 ceph-mon[35356]: pgmap v1062: 129 pgs: 129 active+clean; 752 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 8.6 MiB/s wr, 450 op/s 2024-09-11T00:29:10.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:29:09 smithi178 ceph-mon[35919]: pgmap v1062: 129 pgs: 129 active+clean; 752 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 8.6 MiB/s wr, 450 op/s 2024-09-11T00:29:11.256 INFO:tasks.workunit.client.0.smithi012.stdout: 10 148819 14.62 MB/sec execute 427 sec latency 9101.750 ms 2024-09-11T00:29:12.256 INFO:tasks.workunit.client.0.smithi012.stdout: 10 149489 14.66 MB/sec execute 428 sec latency 4904.696 ms 2024-09-11T00:29:12.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:29:11 smithi165 ceph-mon[35356]: pgmap v1063: 129 pgs: 129 active+clean; 737 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 8.6 MiB/s wr, 477 op/s 2024-09-11T00:29:12.314 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:29:11 smithi012 ceph-mon[30333]: pgmap v1063: 129 pgs: 129 active+clean; 737 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 8.6 MiB/s wr, 477 op/s 2024-09-11T00:29:12.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:29:11 smithi178 ceph-mon[35919]: pgmap v1063: 129 pgs: 129 active+clean; 737 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 8.6 MiB/s wr, 477 op/s 2024-09-11T00:29:13.256 INFO:tasks.workunit.client.0.smithi012.stdout: 10 150224 14.70 MB/sec execute 429 sec latency 194.508 ms 2024-09-11T00:29:14.121 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:14.119+0000 7fadaf000640 1 -- 172.21.15.12:0/2285804683 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fada8108670 msgr2=0x7fada810ab30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:29:14.121 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:14.119+0000 7fadaf000640 1 --2- 172.21.15.12:0/2285804683 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fada8108670 0x7fada810ab30 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fada0009ca0 tx=0x7fada002a8c0 comp rx=0 tx=0).stop 2024-09-11T00:29:14.122 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:14.119+0000 7fadaf000640 1 -- 172.21.15.12:0/2285804683 shutdown_connections 2024-09-11T00:29:14.122 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:14.119+0000 7fadaf000640 1 --2- 172.21.15.12:0/2285804683 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fada8108670 0x7fada810ab30 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:29:14.122 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:14.119+0000 7fadaf000640 1 --2- 172.21.15.12:0/2285804683 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fada8105ac0 0x7fada8107f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:29:14.122 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:14.119+0000 7fadaf000640 1 --2- 172.21.15.12:0/2285804683 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fada806b470 0x7fada8105530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:29:14.122 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:14.119+0000 7fadaf000640 1 -- 172.21.15.12:0/2285804683 >> 172.21.15.12:0/2285804683 conn(0x7fada80fd140 msgr2=0x7fada80ff560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:29:14.122 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:14.119+0000 7fadaf000640 1 -- 172.21.15.12:0/2285804683 shutdown_connections 2024-09-11T00:29:14.122 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:14.120+0000 7fadaf000640 1 -- 172.21.15.12:0/2285804683 wait complete. 2024-09-11T00:29:14.123 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:14.121+0000 7fadaf000640 1 Processor -- start 2024-09-11T00:29:14.123 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:14.121+0000 7fadaf000640 1 -- start start 2024-09-11T00:29:14.123 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:14.121+0000 7fadaf000640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fada806b470 0x7fada8196ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:29:14.123 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:14.121+0000 7fadaf000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fada8105ac0 0x7fada8197120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:29:14.123 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:14.121+0000 7fadaf000640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fada8108670 0x7fada8197d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:29:14.123 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:14.121+0000 7fadaf000640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fada810c440 con 0x7fada8108670 2024-09-11T00:29:14.123 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:14.121+0000 7fadaf000640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fada810c2c0 con 0x7fada8105ac0 2024-09-11T00:29:14.123 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:14.121+0000 7fadaf000640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fada810c5c0 con 0x7fada806b470 2024-09-11T00:29:14.124 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:14.122+0000 7fada7e00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fada806b470 0x7fada8196ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:29:14.124 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:14.122+0000 7fada7400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fada8105ac0 0x7fada8197120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:29:14.124 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:14.122+0000 7fada7e00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fada806b470 0x7fada8196ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.12:39464/0 (socket says 172.21.15.12:39464) 2024-09-11T00:29:14.124 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:14.122+0000 7fada7e00640 1 -- 172.21.15.12:0/1347347341 learned_addr learned my addr 172.21.15.12:0/1347347341 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:29:14.124 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:14.122+0000 7fadad000640 1 --2- 172.21.15.12:0/1347347341 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fada8108670 0x7fada8197d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:29:14.125 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:14.122+0000 7fada7e00640 1 -- 172.21.15.12:0/1347347341 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fada8105ac0 msgr2=0x7fada8197120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:29:14.125 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:14.122+0000 7fada7e00640 1 --2- 172.21.15.12:0/1347347341 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fada8105ac0 0x7fada8197120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:29:14.125 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:14.122+0000 7fada7e00640 1 -- 172.21.15.12:0/1347347341 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fada8108670 msgr2=0x7fada8197d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:29:14.125 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:14.122+0000 7fada7e00640 1 --2- 172.21.15.12:0/1347347341 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fada8108670 0x7fada8197d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:29:14.125 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:14.122+0000 7fada7e00640 1 -- 172.21.15.12:0/1347347341 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fada8198610 con 0x7fada806b470 2024-09-11T00:29:14.125 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:14.123+0000 7fada7e00640 1 --2- 172.21.15.12:0/1347347341 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fada806b470 0x7fada8196ad0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fad98012fc0 tx=0x7fad98010490 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:29:14.125 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:14.123+0000 7fada4c00640 1 -- 172.21.15.12:0/1347347341 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad9800de50 con 0x7fada806b470 2024-09-11T00:29:14.125 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:14.123+0000 7fada4c00640 1 -- 172.21.15.12:0/1347347341 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fad9801d070 con 0x7fada806b470 2024-09-11T00:29:14.126 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:14.123+0000 7fadaf000640 1 -- 172.21.15.12:0/1347347341 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fada81988a0 con 0x7fada806b470 2024-09-11T00:29:14.126 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:14.123+0000 7fada4c00640 1 -- 172.21.15.12:0/1347347341 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad980186e0 con 0x7fada806b470 2024-09-11T00:29:14.126 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:14.123+0000 7fadaf000640 1 -- 172.21.15.12:0/1347347341 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fada819efd0 con 0x7fada806b470 2024-09-11T00:29:14.131 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:14.128+0000 7fadb1e3c740 1 Processor -- start 2024-09-11T00:29:14.131 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:14.128+0000 7fadb1e3c740 1 -- start start 2024-09-11T00:29:14.131 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:14.129+0000 7fadb1e3c740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5615b385f7a0 0x5615b3861b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:29:14.131 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:14.129+0000 7fadb1e3c740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5615b38620d0 0x5615b38644c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:29:14.131 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:14.129+0000 7fadad000640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5615b385f7a0 0x5615b3861b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:29:14.131 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:14.129+0000 7fadb1e3c740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5615b3864b10 0x5615b3866fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:29:14.131 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:14.129+0000 7fadb1e3c740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x5615b37c16b0 con 0x5615b385f7a0 2024-09-11T00:29:14.131 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:14.129+0000 7fadb1e3c740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x5615b37b0ae0 con 0x5615b38620d0 2024-09-11T00:29:14.131 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:14.129+0000 7fadb1e3c740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5615b37b48f0 con 0x5615b3864b10 2024-09-11T00:29:14.132 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:14.129+0000 7fadad000640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5615b385f7a0 0x5615b3861b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:43560/0 (socket says 172.21.15.12:43560) 2024-09-11T00:29:14.132 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:14.129+0000 7fadad000640 1 -- 172.21.15.12:0/3194724088 learned_addr learned my addr 172.21.15.12:0/3194724088 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:29:14.132 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:14.130+0000 7fada4c00640 1 -- 172.21.15.12:0/1347347341 <== mon.1 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7fad9800b780 con 0x7fada806b470 2024-09-11T00:29:14.132 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:14.130+0000 7fada7400640 1 --2- 172.21.15.12:0/3194724088 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5615b3864b10 0x5615b3866fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:29:14.132 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:14.130+0000 7fadad000640 1 -- 172.21.15.12:0/3194724088 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5615b3864b10 msgr2=0x5615b3866fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:29:14.132 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:14.130+0000 7fadad000640 1 --2- 172.21.15.12:0/3194724088 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5615b3864b10 0x5615b3866fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:29:14.132 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:14.130+0000 7fadad000640 1 -- 172.21.15.12:0/3194724088 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5615b38620d0 msgr2=0x5615b38644c0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-11T00:29:14.132 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:14.130+0000 7fadad000640 1 --2- 172.21.15.12:0/3194724088 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5615b38620d0 0x5615b38644c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:29:14.133 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:14.130+0000 7fadad000640 1 -- 172.21.15.12:0/3194724088 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5615b377d780 con 0x5615b385f7a0 2024-09-11T00:29:14.133 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:14.132+0000 7fada4c00640 1 --2- 172.21.15.12:0/1347347341 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fad8c0b5480 0x7fad8c0b7940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:29:14.134 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:14.132+0000 7fadad000640 1 --2- 172.21.15.12:0/3194724088 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5615b385f7a0 0x5615b3861b90 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7fada0009ca0 tx=0x7fada00093c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=970e20e6e4ee1e43 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:29:14.134 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:14.132+0000 7fada4c00640 1 -- 172.21.15.12:0/1347347341 <== mon.1 v2:172.21.15.178:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9208+0+0 (secure 0 0 0) 0x7fad980ea110 con 0x7fada806b470 2024-09-11T00:29:14.134 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:14.132+0000 7fad9d600640 1 -- 172.21.15.12:0/3194724088 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fada0031650 con 0x5615b385f7a0 2024-09-11T00:29:14.134 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:14.132+0000 7fad9d600640 1 -- 172.21.15.12:0/3194724088 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fada0036030 con 0x5615b385f7a0 2024-09-11T00:29:14.135 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:14.132+0000 7fada7e00640 1 --2- 172.21.15.12:0/1347347341 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fad8c0b5480 0x7fad8c0b7940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:29:14.135 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:14.133+0000 7fad9d600640 1 -- 172.21.15.12:0/3194724088 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fada0031650 con 0x5615b385f7a0 2024-09-11T00:29:14.135 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:14.133+0000 7fadb1e3c740 1 -- 172.21.15.12:0/3194724088 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5615b385f7a0 msgr2=0x5615b3861b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:29:14.135 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:14.133+0000 7fadb1e3c740 1 --2- 172.21.15.12:0/3194724088 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5615b385f7a0 0x5615b3861b90 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7fada0009ca0 tx=0x7fada00093c0 comp rx=0 tx=0).stop 2024-09-11T00:29:14.135 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:14.134+0000 7fadb1e3c740 1 -- 172.21.15.12:0/3194724088 shutdown_connections 2024-09-11T00:29:14.136 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:14.134+0000 7fadb1e3c740 1 --2- 172.21.15.12:0/3194724088 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5615b3864b10 0x5615b3866fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:29:14.136 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:14.134+0000 7fadb1e3c740 1 --2- 172.21.15.12:0/3194724088 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5615b38620d0 0x5615b38644c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:29:14.136 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:14.134+0000 7fadb1e3c740 1 --2- 172.21.15.12:0/3194724088 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5615b385f7a0 0x5615b3861b90 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:29:14.136 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:14.134+0000 7fada7e00640 1 --2- 172.21.15.12:0/1347347341 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fad8c0b5480 0x7fad8c0b7940 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fad980e77e0 tx=0x7fad9800bb20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:29:14.136 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:14.134+0000 7fadb1e3c740 1 -- 172.21.15.12:0/3194724088 >> 172.21.15.12:0/3194724088 conn(0x5615b38498e0 msgr2=0x5615b384a670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:29:14.136 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:14.135+0000 7fadb1e3c740 1 -- 172.21.15.12:0/3194724088 shutdown_connections 2024-09-11T00:29:14.136 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:14.135+0000 7fadb1e3c740 1 -- 172.21.15.12:0/3194724088 wait complete. 2024-09-11T00:29:14.137 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:14.136+0000 7fadb1e3c740 1 -- start start 2024-09-11T00:29:14.137 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:14.136+0000 7fadb1e3c740 1 Processor -- start 2024-09-11T00:29:14.138 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:14.136+0000 7fadb1e3c740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5615b385f7a0 0x5615b3868d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:29:14.138 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:14.136+0000 7fadb1e3c740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5615b38620d0 0x5615b38693a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:29:14.139 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:14.137+0000 7fadb1e3c740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5615b3864b10 0x5615b3869f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:29:14.139 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:14.137+0000 7fadb1e3c740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x5615b36e8ab0 con 0x5615b3864b10 2024-09-11T00:29:14.139 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:14.137+0000 7fadb1e3c740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x5615b3757570 con 0x5615b385f7a0 2024-09-11T00:29:14.139 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:14.137+0000 7fadb1e3c740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5615b3867910 con 0x5615b38620d0 2024-09-11T00:29:14.139 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:14.137+0000 7fadb1e3c740 10 client.0 _ll_register_callbacks cb 0x5615b3766100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:29:14.139 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:14.137+0000 7fadad000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5615b385f7a0 0x5615b3868d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:29:14.140 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:14.137+0000 7fadad000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5615b385f7a0 0x5615b3868d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.12:42156/0 (socket says 172.21.15.12:42156) 2024-09-11T00:29:14.140 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:14.137+0000 7fadad000640 1 -- 172.21.15.12:0/652495203 learned_addr learned my addr 172.21.15.12:0/652495203 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:29:14.140 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:14.137+0000 7fadad000640 1 --2- 172.21.15.12:0/652495203 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5615b3864b10 0x5615b3869f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:29:14.140 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:14.138+0000 7fada7400640 1 --2- 172.21.15.12:0/652495203 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5615b38620d0 0x5615b38693a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:29:14.141 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:14.138+0000 7fadad000640 1 -- 172.21.15.12:0/652495203 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5615b38620d0 msgr2=0x5615b38693a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:29:14.141 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:14.138+0000 7fadad000640 1 --2- 172.21.15.12:0/652495203 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5615b38620d0 0x5615b38693a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:29:14.141 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:14.138+0000 7fadad000640 1 -- 172.21.15.12:0/652495203 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5615b385f7a0 msgr2=0x5615b3868d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:29:14.141 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:14.138+0000 7fadad000640 1 --2- 172.21.15.12:0/652495203 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5615b385f7a0 0x5615b3868d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:29:14.141 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:14.138+0000 7fadad000640 1 -- 172.21.15.12:0/652495203 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5615b3512790 con 0x5615b3864b10 2024-09-11T00:29:14.141 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:14.139+0000 7fadad000640 1 --2- 172.21.15.12:0/652495203 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5615b3864b10 0x5615b3869f00 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7fada0005e30 tx=0x7fada0031af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4cae424e564eab72 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:29:14.141 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:14.139+0000 7fad92000640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-09-11T00:29:14.142 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:14.140+0000 7fad92000640 1 -- 172.21.15.12:0/652495203 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fada0058060 con 0x5615b3864b10 2024-09-11T00:29:14.142 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:14.140+0000 7fad92000640 1 -- 172.21.15.12:0/652495203 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 9208+0+0 (secure 0 0 0) 0x7fada0003b70 con 0x5615b3864b10 2024-09-11T00:29:14.142 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:14.140+0000 7fadb1e3c740 1 -- 172.21.15.12:0/652495203 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fada0036030 con 0x5615b3864b10 2024-09-11T00:29:14.142 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:14.140+0000 7fad92000640 1 -- 172.21.15.12:0/652495203 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=98) ==== 24+0+0 (secure 0 0 0) 0x7fada004a040 con 0x5615b3864b10 2024-09-11T00:29:14.142 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:14.140+0000 7fadb1e3c740 10 client.17883 fetch_fsmap learned FSMap version 98 2024-09-11T00:29:14.142 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:14.140+0000 7fadb1e3c740 1 -- 172.21.15.12:0/652495203 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=98,monmap=4+}) -- 0x5615b358b620 con 0x5615b3864b10 2024-09-11T00:29:14.143 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:14.140+0000 7fad92000640 1 -- 172.21.15.12:0/652495203 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fada0036030 con 0x5615b3864b10 2024-09-11T00:29:14.143 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:14.141+0000 7fad92000640 1 -- 172.21.15.12:0/652495203 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 98) ==== 8402+0+0 (secure 0 0 0) 0x7fada0050050 con 0x5615b3864b10 2024-09-11T00:29:14.143 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:14.142+0000 7fadb1e3c740 10 client.17883 fetch_fsmap finished waiting for FSMap version 98 2024-09-11T00:29:14.143 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:14.142+0000 7fadb1e3c740 10 client.17883 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T00:29:14.143 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:14.142+0000 7fadb1e3c740 20 client.17883 populate_metadata read hostname 'smithi012' 2024-09-11T00:29:14.143 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:14.142+0000 7fadb1e3c740 1 --2- 172.21.15.12:0/652495203 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x5615b387c940 0x5615b387ed30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:29:14.144 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:14.142+0000 7fadb1e3c740 4 client.17883 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T00:29:14.144 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:14.142+0000 7fadb1e3c740 1 -- 172.21.15.12:0/652495203 --> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5615b37c3600 con 0x5615b387c940 2024-09-11T00:29:14.144 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:14.142+0000 7fada7400640 1 --2- 172.21.15.12:0/652495203 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x5615b387c940 0x5615b387ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:29:14.144 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:14.143+0000 7fada7400640 1 --2- 172.21.15.12:0/652495203 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x5615b387c940 0x5615b387ed30 crc :-1 s=READY pgs=305 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fb1ef937f07f85e9 server_cookie=2ff41879c51635c1 in_seq=0 out_seq=0 2024-09-11T00:29:14.144 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:14.143+0000 7fad92000640 10 client.17883 ms_handle_connect on v2:172.21.15.165:6838/501312984 2024-09-11T00:29:14.149 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:14.148+0000 7fad92000640 1 -- 172.21.15.12:0/652495203 <== mds.0 v2:172.21.15.165:6838/501312984 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5615b37c3600 con 0x5615b387c940 2024-09-11T00:29:14.149 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:14.148+0000 7fad92000640 10 client.17883 handle_command_reply: tid=0 2024-09-11T00:29:14.150 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:14.148+0000 7fadb1e3c740 1 client.17883 shutdown 2024-09-11T00:29:14.150 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:14.149+0000 7fadb1e3c740 1 -- 172.21.15.12:0/652495203 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5615b3864b10 msgr2=0x5615b3869f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:29:14.150 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:14.149+0000 7fadb1e3c740 1 --2- 172.21.15.12:0/652495203 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5615b3864b10 0x5615b3869f00 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7fada0005e30 tx=0x7fada0031af0 comp rx=0 tx=0).stop 2024-09-11T00:29:14.150 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:14.149+0000 7fadb1e3c740 1 -- 172.21.15.12:0/652495203 shutdown_connections 2024-09-11T00:29:14.150 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:14.149+0000 7fadb1e3c740 1 --2- 172.21.15.12:0/652495203 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x5615b387c940 0x5615b387ed30 crc :-1 s=READY pgs=305 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:29:14.150 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:14.149+0000 7fadb1e3c740 1 --2- 172.21.15.12:0/652495203 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5615b3864b10 0x5615b3869f00 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:29:14.150 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:14.149+0000 7fad92000640 0 client.17883 ms_handle_reset on v2:172.21.15.165:6838/501312984 2024-09-11T00:29:14.151 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:14.149+0000 7fadb1e3c740 1 --2- 172.21.15.12:0/652495203 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5615b38620d0 0x5615b38693a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:29:14.151 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:14.149+0000 7fadb1e3c740 1 --2- 172.21.15.12:0/652495203 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5615b385f7a0 0x5615b3868d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:29:14.151 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:14.149+0000 7fadb1e3c740 1 -- 172.21.15.12:0/652495203 >> 172.21.15.12:0/652495203 conn(0x5615b38498e0 msgr2=0x5615b3848a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:29:14.151 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:14.149+0000 7fadb1e3c740 1 -- 172.21.15.12:0/652495203 shutdown_connections 2024-09-11T00:29:14.151 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:14.149+0000 7fadb1e3c740 1 -- 172.21.15.12:0/652495203 wait complete. 2024-09-11T00:29:14.151 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:14.150+0000 7fadb1e3c740 20 client.17883 trim_cache size 0 max 16384 2024-09-11T00:29:14.176 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:14.174+0000 7fadb1e3c740 1 Processor -- start 2024-09-11T00:29:14.176 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:14.174+0000 7fadb1e3c740 1 -- start start 2024-09-11T00:29:14.176 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:14.175+0000 7fadb1e3c740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5615b385f7a0 0x5615b3865600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:29:14.176 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:14.175+0000 7fadb1e3c740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5615b38620d0 0x5615b3867ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:29:14.177 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:14.175+0000 7fadb1e3c740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5615b3864b10 0x5615b384b710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:29:14.177 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:14.175+0000 7fadb1e3c740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fada81a0d30 con 0x5615b385f7a0 2024-09-11T00:29:14.177 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:14.175+0000 7fadb1e3c740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fada81a2990 con 0x5615b3864b10 2024-09-11T00:29:14.177 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:14.175+0000 7fadb1e3c740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fada81a1460 con 0x5615b38620d0 2024-09-11T00:29:14.177 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:14.175+0000 7fada7400640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5615b38620d0 0x5615b3867ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:29:14.177 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:14.175+0000 7fada7400640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5615b38620d0 0x5615b3867ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.12:39504/0 (socket says 172.21.15.12:39504) 2024-09-11T00:29:14.177 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:14.175+0000 7fada7400640 1 -- 172.21.15.12:0/3005038678 learned_addr learned my addr 172.21.15.12:0/3005038678 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:29:14.178 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:14.176+0000 7fadad000640 1 --2- 172.21.15.12:0/3005038678 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5615b385f7a0 0x5615b3865600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:29:14.178 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:14.176+0000 7fada7400640 1 -- 172.21.15.12:0/3005038678 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5615b3864b10 msgr2=0x5615b384b710 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-11T00:29:14.178 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:14.176+0000 7fada7400640 1 --2- 172.21.15.12:0/3005038678 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5615b3864b10 0x5615b384b710 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:29:14.178 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:14.176+0000 7fada7400640 1 -- 172.21.15.12:0/3005038678 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5615b385f7a0 msgr2=0x5615b3865600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:29:14.179 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:14.176+0000 7fada7400640 1 --2- 172.21.15.12:0/3005038678 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5615b385f7a0 0x5615b3865600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:29:14.179 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:14.176+0000 7fada7400640 1 -- 172.21.15.12:0/3005038678 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5615b3864830 con 0x5615b38620d0 2024-09-11T00:29:14.179 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:14.176+0000 7fadad000640 1 --2- 172.21.15.12:0/3005038678 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5615b385f7a0 0x5615b3865600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T00:29:14.179 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:14.177+0000 7fada7400640 1 --2- 172.21.15.12:0/3005038678 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5615b38620d0 0x5615b3867ec0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fad94009bb0 tx=0x7fad94005f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c6fc1cf987b76938 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:29:14.179 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:14.177+0000 7fad9cc00640 1 -- 172.21.15.12:0/3005038678 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad94029070 con 0x5615b38620d0 2024-09-11T00:29:14.179 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:14.177+0000 7fad9cc00640 1 -- 172.21.15.12:0/3005038678 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fad940035f0 con 0x5615b38620d0 2024-09-11T00:29:14.180 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:14.177+0000 7fad9cc00640 1 -- 172.21.15.12:0/3005038678 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad940044e0 con 0x5615b38620d0 2024-09-11T00:29:14.180 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:14.178+0000 7fadb1e3c740 1 -- 172.21.15.12:0/3005038678 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5615b38620d0 msgr2=0x5615b3867ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:29:14.180 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:14.178+0000 7fadb1e3c740 1 --2- 172.21.15.12:0/3005038678 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5615b38620d0 0x5615b3867ec0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fad94009bb0 tx=0x7fad94005f90 comp rx=0 tx=0).stop 2024-09-11T00:29:14.180 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:14.178+0000 7fadb1e3c740 1 -- 172.21.15.12:0/3005038678 shutdown_connections 2024-09-11T00:29:14.180 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:14.178+0000 7fadb1e3c740 1 --2- 172.21.15.12:0/3005038678 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5615b3864b10 0x5615b384b710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:29:14.180 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:14.178+0000 7fadb1e3c740 1 --2- 172.21.15.12:0/3005038678 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5615b38620d0 0x5615b3867ec0 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:29:14.181 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:14.178+0000 7fadb1e3c740 1 --2- 172.21.15.12:0/3005038678 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5615b385f7a0 0x5615b3865600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:29:14.181 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:14.178+0000 7fadb1e3c740 1 -- 172.21.15.12:0/3005038678 >> 172.21.15.12:0/3005038678 conn(0x5615b38498e0 msgr2=0x5615b3864ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:29:14.181 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:14.179+0000 7fadb1e3c740 1 -- 172.21.15.12:0/3005038678 shutdown_connections 2024-09-11T00:29:14.181 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:14.179+0000 7fadb1e3c740 1 -- 172.21.15.12:0/3005038678 wait complete. 2024-09-11T00:29:14.182 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:14.179+0000 7fadb1e3c740 1 -- start start 2024-09-11T00:29:14.182 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:14.180+0000 7fadb1e3c740 1 Processor -- start 2024-09-11T00:29:14.183 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:14.180+0000 7fadb1e3c740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5615b385f7a0 0x5615b38702f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:29:14.183 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:14.180+0000 7fadb1e3c740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5615b38620d0 0x5615b3870990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:29:14.183 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:14.180+0000 7fadb1e3c740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5615b3864b10 0x5615b386ec60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:29:14.183 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:14.180+0000 7fadb1e3c740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fada81a22c0 con 0x5615b3864b10 2024-09-11T00:29:14.183 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:14.180+0000 7fadb1e3c740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fada81a1bf0 con 0x5615b38620d0 2024-09-11T00:29:14.183 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:14.180+0000 7fadb1e3c740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fada8074ee0 con 0x5615b385f7a0 2024-09-11T00:29:14.183 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:14.180+0000 7fadb1e3c740 10 client.0 _ll_register_callbacks cb 0x7fada0009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:29:14.183 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:14.181+0000 7fadad000640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5615b385f7a0 0x5615b38702f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:29:14.183 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:14.181+0000 7fadad000640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5615b385f7a0 0x5615b38702f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.12:39508/0 (socket says 172.21.15.12:39508) 2024-09-11T00:29:14.183 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:14.181+0000 7fadad000640 1 -- 172.21.15.12:0/1143834052 learned_addr learned my addr 172.21.15.12:0/1143834052 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:29:14.183 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:14.181+0000 7fada7400640 1 --2- 172.21.15.12:0/1143834052 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5615b38620d0 0x5615b3870990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:29:14.183 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:14.181+0000 7fadad000640 1 --2- 172.21.15.12:0/1143834052 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5615b3864b10 0x5615b386ec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:29:14.186 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:14.182+0000 7fadad000640 1 -- 172.21.15.12:0/1143834052 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5615b38620d0 msgr2=0x5615b3870990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:29:14.186 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:14.182+0000 7fadad000640 1 --2- 172.21.15.12:0/1143834052 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5615b38620d0 0x5615b3870990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:29:14.186 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:14.182+0000 7fadad000640 1 -- 172.21.15.12:0/1143834052 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5615b3864b10 msgr2=0x5615b386ec60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:29:14.186 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:14.182+0000 7fadad000640 1 --2- 172.21.15.12:0/1143834052 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5615b3864b10 0x5615b386ec60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:29:14.186 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:14.182+0000 7fadad000640 1 -- 172.21.15.12:0/1143834052 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5615b386f490 con 0x5615b385f7a0 2024-09-11T00:29:14.186 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:14.182+0000 7fadad000640 1 --2- 172.21.15.12:0/1143834052 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5615b385f7a0 0x5615b38702f0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fada0009620 tx=0x7fada002a8c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5bec7da696aa2c2a server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:29:14.186 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:14.182+0000 7fad92000640 10 client.0 ms_handle_connect on v2:172.21.15.178:3300/0 2024-09-11T00:29:14.186 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:14.182+0000 7fad92000640 1 -- 172.21.15.12:0/1143834052 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fada0058040 con 0x5615b385f7a0 2024-09-11T00:29:14.186 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:14.182+0000 7fad92000640 1 -- 172.21.15.12:0/1143834052 <== mon.1 v2:172.21.15.178:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 9208+0+0 (secure 0 0 0) 0x7fada00307c0 con 0x5615b385f7a0 2024-09-11T00:29:14.186 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:14.183+0000 7fadb1e3c740 1 -- 172.21.15.12:0/1143834052 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fad940035f0 con 0x5615b385f7a0 2024-09-11T00:29:14.186 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:14.183+0000 7fad92000640 1 -- 172.21.15.12:0/1143834052 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=98) ==== 24+0+0 (secure 0 0 0) 0x7fada0030bd0 con 0x5615b385f7a0 2024-09-11T00:29:14.187 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:14.183+0000 7fadb1e3c740 10 client.27877 fetch_fsmap learned FSMap version 98 2024-09-11T00:29:14.187 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:14.183+0000 7fadb1e3c740 1 -- 172.21.15.12:0/1143834052 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,fsmap=98,monmap=4+}) -- 0x5615b38ea4c0 con 0x5615b385f7a0 2024-09-11T00:29:14.187 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:14.183+0000 7fad92000640 1 -- 172.21.15.12:0/1143834052 <== mon.1 v2:172.21.15.178:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fad940035f0 con 0x5615b385f7a0 2024-09-11T00:29:14.187 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:14.183+0000 7fad92000640 1 -- 172.21.15.12:0/1143834052 <== mon.1 v2:172.21.15.178:3300/0 5 ==== fsmap(e 98) ==== 8402+0+0 (secure 0 0 0) 0x7fada0050070 con 0x5615b385f7a0 2024-09-11T00:29:14.187 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:14.184+0000 7fadb1e3c740 10 client.27877 fetch_fsmap finished waiting for FSMap version 98 2024-09-11T00:29:14.187 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:14.184+0000 7fadb1e3c740 10 client.27877 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T00:29:14.187 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:14.184+0000 7fadb1e3c740 20 client.27877 populate_metadata read hostname 'smithi012' 2024-09-11T00:29:14.187 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:14.184+0000 7fadb1e3c740 1 --2- 172.21.15.12:0/1143834052 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x5615b38eac40 0x5615b38ed030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:29:14.187 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:14.184+0000 7fadb1e3c740 4 client.27877 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T00:29:14.187 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:14.184+0000 7fadb1e3c740 1 -- 172.21.15.12:0/1143834052 --> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5615b37bf0b0 con 0x5615b38eac40 2024-09-11T00:29:14.187 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:14.185+0000 7fada7400640 1 --2- 172.21.15.12:0/1143834052 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x5615b38eac40 0x5615b38ed030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:29:14.188 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:14.185+0000 7fada7400640 1 --2- 172.21.15.12:0/1143834052 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x5615b38eac40 0x5615b38ed030 crc :-1 s=READY pgs=306 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=55302e16c78475a1 server_cookie=aa8446567a3d549a in_seq=0 out_seq=0 2024-09-11T00:29:14.188 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:14.185+0000 7fad92000640 10 client.27877 ms_handle_connect on v2:172.21.15.165:6838/501312984 2024-09-11T00:29:14.256 INFO:tasks.workunit.client.0.smithi012.stdout: 10 150907 14.76 MB/sec execute 430 sec latency 79.105 ms 2024-09-11T00:29:14.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:29:13 smithi165 ceph-mon[35356]: pgmap v1064: 129 pgs: 129 active+clean; 730 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.8 MiB/s wr, 353 op/s 2024-09-11T00:29:14.283 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:29:13 smithi165 ceph-mon[35356]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T00:29:14.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:29:13 smithi012 ceph-mon[30333]: pgmap v1064: 129 pgs: 129 active+clean; 730 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.8 MiB/s wr, 353 op/s 2024-09-11T00:29:14.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:29:13 smithi012 ceph-mon[30333]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T00:29:14.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:29:13 smithi178 ceph-mon[35919]: pgmap v1064: 129 pgs: 129 active+clean; 730 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.8 MiB/s wr, 353 op/s 2024-09-11T00:29:14.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:29:13 smithi178 ceph-mon[35919]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T00:29:14.421 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:14.420+0000 7fad92000640 1 -- 172.21.15.12:0/1143834052 <== mds.0 v2:172.21.15.165:6838/501312984 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5615b37bf0b0 con 0x5615b38eac40 2024-09-11T00:29:14.421 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:14.420+0000 7fad92000640 10 client.27877 handle_command_reply: tid=0 2024-09-11T00:29:14.422 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:14.420+0000 7fadb1e3c740 1 client.27877 shutdown 2024-09-11T00:29:14.422 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:14.420+0000 7fadb1e3c740 1 -- 172.21.15.12:0/1143834052 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5615b385f7a0 msgr2=0x5615b38702f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:29:14.422 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:14.420+0000 7fadb1e3c740 1 --2- 172.21.15.12:0/1143834052 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5615b385f7a0 0x5615b38702f0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fada0009620 tx=0x7fada002a8c0 comp rx=0 tx=0).stop 2024-09-11T00:29:14.423 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:14.420+0000 7fadb1e3c740 1 -- 172.21.15.12:0/1143834052 shutdown_connections 2024-09-11T00:29:14.423 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:14.420+0000 7fadb1e3c740 1 --2- 172.21.15.12:0/1143834052 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x5615b38eac40 0x5615b38ed030 crc :-1 s=READY pgs=306 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:29:14.423 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:14.420+0000 7fadb1e3c740 1 --2- 172.21.15.12:0/1143834052 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5615b3864b10 0x5615b386ec60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:29:14.423 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:14.420+0000 7fadb1e3c740 1 --2- 172.21.15.12:0/1143834052 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5615b38620d0 0x5615b3870990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:29:14.423 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:14.420+0000 7fadb1e3c740 1 --2- 172.21.15.12:0/1143834052 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5615b385f7a0 0x5615b38702f0 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:29:14.423 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:14.420+0000 7fadb1e3c740 1 -- 172.21.15.12:0/1143834052 >> 172.21.15.12:0/1143834052 conn(0x5615b38498e0 msgr2=0x5615b3848240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:29:14.423 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:14.421+0000 7fad92000640 0 client.27877 ms_handle_reset on v2:172.21.15.165:6838/501312984 2024-09-11T00:29:14.423 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:14.421+0000 7fadb1e3c740 1 -- 172.21.15.12:0/1143834052 shutdown_connections 2024-09-11T00:29:14.423 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:14.421+0000 7fadb1e3c740 1 -- 172.21.15.12:0/1143834052 wait complete. 2024-09-11T00:29:14.423 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:14.421+0000 7fadb1e3c740 20 client.27877 trim_cache size 0 max 16384 2024-09-11T00:29:14.424 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:{ 2024-09-11T00:29:14.424 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "message": "", 2024-09-11T00:29:14.424 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "return_code": 0 2024-09-11T00:29:14.424 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:} 2024-09-11T00:29:14.424 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:14.422+0000 7fadaf000640 1 -- 172.21.15.12:0/1347347341 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fad8c0b5480 msgr2=0x7fad8c0b7940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:29:14.425 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:14.422+0000 7fadaf000640 1 --2- 172.21.15.12:0/1347347341 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fad8c0b5480 0x7fad8c0b7940 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fad980e77e0 tx=0x7fad9800bb20 comp rx=0 tx=0).stop 2024-09-11T00:29:14.425 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:14.422+0000 7fadaf000640 1 -- 172.21.15.12:0/1347347341 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fada806b470 msgr2=0x7fada8196ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:29:14.425 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:14.422+0000 7fadaf000640 1 --2- 172.21.15.12:0/1347347341 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fada806b470 0x7fada8196ad0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fad98012fc0 tx=0x7fad98010490 comp rx=0 tx=0).stop 2024-09-11T00:29:14.425 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:14.422+0000 7fadaf000640 1 -- 172.21.15.12:0/1347347341 shutdown_connections 2024-09-11T00:29:14.425 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:14.422+0000 7fadaf000640 1 --2- 172.21.15.12:0/1347347341 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fad8c0b5480 0x7fad8c0b7940 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:29:14.425 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:14.422+0000 7fadaf000640 1 --2- 172.21.15.12:0/1347347341 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fada8108670 0x7fada8197d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:29:14.425 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:14.422+0000 7fadaf000640 1 --2- 172.21.15.12:0/1347347341 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fada8105ac0 0x7fada8197120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:29:14.425 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:14.422+0000 7fadaf000640 1 --2- 172.21.15.12:0/1347347341 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fada806b470 0x7fada8196ad0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:29:14.425 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:14.422+0000 7fadaf000640 1 -- 172.21.15.12:0/1347347341 >> 172.21.15.12:0/1347347341 conn(0x7fada80fd140 msgr2=0x7fada8103d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:29:14.425 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:14.422+0000 7fadaf000640 1 -- 172.21.15.12:0/1347347341 shutdown_connections 2024-09-11T00:29:14.425 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:14.422+0000 7fadaf000640 1 -- 172.21.15.12:0/1347347341 wait complete. 2024-09-11T00:29:15.256 INFO:tasks.workunit.client.0.smithi012.stdout: 10 151548 14.81 MB/sec execute 431 sec latency 198.847 ms 2024-09-11T00:29:16.195 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:29:15 smithi012 ceph-mon[30333]: pgmap v1065: 129 pgs: 129 active+clean; 728 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 954 KiB/s rd, 375 KiB/s wr, 186 op/s 2024-09-11T00:29:16.196 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:29:15 smithi012 ceph-mon[30333]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T00:29:16.256 INFO:tasks.workunit.client.0.smithi012.stdout: 10 152285 14.87 MB/sec execute 432 sec latency 61.126 ms 2024-09-11T00:29:16.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:29:15 smithi165 ceph-mon[35356]: pgmap v1065: 129 pgs: 129 active+clean; 728 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 954 KiB/s rd, 375 KiB/s wr, 186 op/s 2024-09-11T00:29:16.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:29:15 smithi165 ceph-mon[35356]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T00:29:16.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:29:15 smithi178 ceph-mon[35919]: pgmap v1065: 129 pgs: 129 active+clean; 728 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 954 KiB/s rd, 375 KiB/s wr, 186 op/s 2024-09-11T00:29:16.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:29:15 smithi178 ceph-mon[35919]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T00:29:17.256 INFO:tasks.workunit.client.0.smithi012.stdout: 10 153093 14.95 MB/sec execute 433 sec latency 125.275 ms 2024-09-11T00:29:18.256 INFO:tasks.workunit.client.0.smithi012.stdout: 10 153981 15.01 MB/sec execute 434 sec latency 51.438 ms 2024-09-11T00:29:18.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:29:18 smithi165 ceph-mon[35356]: pgmap v1066: 129 pgs: 129 active+clean; 725 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 1.6 MiB/s wr, 420 op/s 2024-09-11T00:29:18.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:29:17 smithi012 ceph-mon[30333]: pgmap v1066: 129 pgs: 129 active+clean; 725 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 1.6 MiB/s wr, 420 op/s 2024-09-11T00:29:18.332 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:29:17 smithi178 ceph-mon[35919]: pgmap v1066: 129 pgs: 129 active+clean; 725 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 1.6 MiB/s wr, 420 op/s 2024-09-11T00:29:19.256 INFO:tasks.workunit.client.0.smithi012.stdout: 10 154415 15.03 MB/sec execute 435 sec latency 193.630 ms 2024-09-11T00:29:20.256 INFO:tasks.workunit.client.0.smithi012.stdout: 10 155088 15.08 MB/sec execute 436 sec latency 113.804 ms 2024-09-11T00:29:20.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:29:20 smithi165 ceph-mon[35356]: pgmap v1067: 129 pgs: 129 active+clean; 727 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 1.7 MiB/s wr, 358 op/s 2024-09-11T00:29:20.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:29:20 smithi012 ceph-mon[30333]: pgmap v1067: 129 pgs: 129 active+clean; 727 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 1.7 MiB/s wr, 358 op/s 2024-09-11T00:29:20.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:29:20 smithi178 ceph-mon[35919]: pgmap v1067: 129 pgs: 129 active+clean; 727 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 1.7 MiB/s wr, 358 op/s 2024-09-11T00:29:21.256 INFO:tasks.workunit.client.0.smithi012.stdout: 10 155870 15.14 MB/sec execute 437 sec latency 128.414 ms 2024-09-11T00:29:22.256 INFO:tasks.workunit.client.0.smithi012.stdout: 10 156805 15.23 MB/sec execute 438 sec latency 46.899 ms 2024-09-11T00:29:22.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:29:22 smithi165 ceph-mon[35356]: pgmap v1068: 129 pgs: 129 active+clean; 732 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 2.4 MiB/s wr, 449 op/s 2024-09-11T00:29:22.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:29:22 smithi012 ceph-mon[30333]: pgmap v1068: 129 pgs: 129 active+clean; 732 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 2.4 MiB/s wr, 449 op/s 2024-09-11T00:29:22.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:29:22 smithi178 ceph-mon[35919]: pgmap v1068: 129 pgs: 129 active+clean; 732 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 2.4 MiB/s wr, 449 op/s 2024-09-11T00:29:23.256 INFO:tasks.workunit.client.0.smithi012.stdout: 10 157323 15.25 MB/sec execute 439 sec latency 304.366 ms 2024-09-11T00:29:24.188 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:29:24 smithi012 ceph-mon[30333]: pgmap v1069: 129 pgs: 129 active+clean; 742 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 3.4 MiB/s wr, 588 op/s 2024-09-11T00:29:24.256 INFO:tasks.workunit.client.0.smithi012.stdout: 10 157995 15.31 MB/sec execute 440 sec latency 76.574 ms 2024-09-11T00:29:24.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:29:24 smithi165 ceph-mon[35356]: pgmap v1069: 129 pgs: 129 active+clean; 742 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 3.4 MiB/s wr, 588 op/s 2024-09-11T00:29:24.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:29:24 smithi178 ceph-mon[35919]: pgmap v1069: 129 pgs: 129 active+clean; 742 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 3.4 MiB/s wr, 588 op/s 2024-09-11T00:29:25.257 INFO:tasks.workunit.client.0.smithi012.stdout: 10 158711 15.35 MB/sec execute 441 sec latency 75.633 ms 2024-09-11T00:29:26.256 INFO:tasks.workunit.client.0.smithi012.stdout: 10 159445 15.40 MB/sec execute 442 sec latency 206.198 ms 2024-09-11T00:29:26.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:29:26 smithi165 ceph-mon[35356]: pgmap v1070: 129 pgs: 129 active+clean; 743 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 MiB/s rd, 3.4 MiB/s wr, 575 op/s 2024-09-11T00:29:26.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:29:26 smithi012 ceph-mon[30333]: pgmap v1070: 129 pgs: 129 active+clean; 743 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 MiB/s rd, 3.4 MiB/s wr, 575 op/s 2024-09-11T00:29:26.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:29:26 smithi178 ceph-mon[35919]: pgmap v1070: 129 pgs: 129 active+clean; 743 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 MiB/s rd, 3.4 MiB/s wr, 575 op/s 2024-09-11T00:29:27.257 INFO:tasks.workunit.client.0.smithi012.stdout: 10 160221 15.47 MB/sec execute 443 sec latency 72.325 ms 2024-09-11T00:29:27.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:29:27 smithi012 ceph-mon[30333]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T00:29:27.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:29:27 smithi178 ceph-mon[35919]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T00:29:27.532 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:29:27 smithi165 ceph-mon[35356]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T00:29:27.643 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:27.642+0000 7f58fe600640 1 -- 172.21.15.12:0/2353021040 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f58f8078b70 msgr2=0x7f58f8078ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:29:27.643 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:27.642+0000 7f58fe600640 1 --2- 172.21.15.12:0/2353021040 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f58f8078b70 0x7f58f8078ff0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f58e4009860 tx=0x7f58e402a340 comp rx=0 tx=0).stop 2024-09-11T00:29:27.644 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:27.642+0000 7f58fe600640 1 -- 172.21.15.12:0/2353021040 shutdown_connections 2024-09-11T00:29:27.644 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:27.642+0000 7f58fe600640 1 --2- 172.21.15.12:0/2353021040 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f58f8079530 0x7f58f8079e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:29:27.644 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:27.642+0000 7f58fe600640 1 --2- 172.21.15.12:0/2353021040 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f58f8078b70 0x7f58f8078ff0 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:29:27.644 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:27.642+0000 7f58fe600640 1 --2- 172.21.15.12:0/2353021040 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f58f8077a20 0x7f58f8077e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:29:27.644 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:27.642+0000 7f58fe600640 1 -- 172.21.15.12:0/2353021040 >> 172.21.15.12:0/2353021040 conn(0x7f58f8100460 msgr2=0x7f58f8102880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:29:27.644 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:27.642+0000 7f58fe600640 1 -- 172.21.15.12:0/2353021040 shutdown_connections 2024-09-11T00:29:27.644 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:27.643+0000 7f58fe600640 1 -- 172.21.15.12:0/2353021040 wait complete. 2024-09-11T00:29:27.646 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:27.645+0000 7f58fe600640 1 Processor -- start 2024-09-11T00:29:27.646 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:27.645+0000 7f58fe600640 1 -- start start 2024-09-11T00:29:27.647 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:27.646+0000 7f58fe600640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f58f8077a20 0x7f58f819f330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:29:27.647 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:27.646+0000 7f58fe600640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f58f8078b70 0x7f58f819f980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:29:27.647 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:27.646+0000 7f58fe600640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f58f8079530 0x7f58f81a05b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:29:27.647 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:27.646+0000 7f58fe600640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f58f8114be0 con 0x7f58f8079530 2024-09-11T00:29:27.647 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:27.646+0000 7f58fe600640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f58f8114a60 con 0x7f58f8077a20 2024-09-11T00:29:27.647 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:27.646+0000 7f58fe600640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f58f8114d60 con 0x7f58f8078b70 2024-09-11T00:29:27.648 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:27.646+0000 7f58f6a00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f58f8078b70 0x7f58f819f980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:29:27.648 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:27.646+0000 7f58f6a00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f58f8078b70 0x7f58f819f980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.12:34104/0 (socket says 172.21.15.12:34104) 2024-09-11T00:29:27.648 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:27.646+0000 7f58f6a00640 1 -- 172.21.15.12:0/1277254572 learned_addr learned my addr 172.21.15.12:0/1277254572 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:29:27.648 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:27.647+0000 7f58f7400640 1 --2- 172.21.15.12:0/1277254572 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f58f8077a20 0x7f58f819f330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:29:27.648 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:27.647+0000 7f58f7e00640 1 --2- 172.21.15.12:0/1277254572 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f58f8079530 0x7f58f81a05b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:29:27.648 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:27.647+0000 7f58f6a00640 1 -- 172.21.15.12:0/1277254572 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f58f8077a20 msgr2=0x7f58f819f330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:29:27.649 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:27.647+0000 7f58f6a00640 1 --2- 172.21.15.12:0/1277254572 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f58f8077a20 0x7f58f819f330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:29:27.649 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:27.647+0000 7f58f6a00640 1 -- 172.21.15.12:0/1277254572 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f58f8079530 msgr2=0x7f58f81a05b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:29:27.649 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:27.647+0000 7f58f6a00640 1 --2- 172.21.15.12:0/1277254572 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f58f8079530 0x7f58f81a05b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:29:27.649 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:27.647+0000 7f58f6a00640 1 -- 172.21.15.12:0/1277254572 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58f81a0e70 con 0x7f58f8078b70 2024-09-11T00:29:27.649 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:27.647+0000 7f58f6a00640 1 --2- 172.21.15.12:0/1277254572 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f58f8078b70 0x7f58f819f980 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f58e40057c0 tx=0x7f58e4032eb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:29:27.650 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:27.648+0000 7f58efe00640 1 -- 172.21.15.12:0/1277254572 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58e4037040 con 0x7f58f8078b70 2024-09-11T00:29:27.650 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:27.648+0000 7f58efe00640 1 -- 172.21.15.12:0/1277254572 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f58e4030cd0 con 0x7f58f8078b70 2024-09-11T00:29:27.650 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:27.648+0000 7f58efe00640 1 -- 172.21.15.12:0/1277254572 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58e4005350 con 0x7f58f8078b70 2024-09-11T00:29:27.650 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:27.648+0000 7f58fe600640 1 -- 172.21.15.12:0/1277254572 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58f81a1100 con 0x7f58f8078b70 2024-09-11T00:29:27.650 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:27.648+0000 7f58fe600640 1 -- 172.21.15.12:0/1277254572 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58f81a7650 con 0x7f58f8078b70 2024-09-11T00:29:27.657 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:27.655+0000 7f58efe00640 1 -- 172.21.15.12:0/1277254572 <== mon.1 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7f58e4004050 con 0x7f58f8078b70 2024-09-11T00:29:27.657 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:27.656+0000 7f5901670740 1 Processor -- start 2024-09-11T00:29:27.658 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:27.656+0000 7f5901670740 1 -- start start 2024-09-11T00:29:27.658 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:27.657+0000 7f5901670740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561033326790 0x561033328b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:29:27.659 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:27.657+0000 7f5901670740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5610333290c0 0x56103332b4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:29:27.659 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:27.657+0000 7f5901670740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56103332bb00 0x56103332dfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:29:27.659 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:27.657+0000 7f5901670740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x5610332996b0 con 0x561033326790 2024-09-11T00:29:27.659 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:27.657+0000 7f5901670740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x561033288ae0 con 0x5610333290c0 2024-09-11T00:29:27.659 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:27.657+0000 7f5901670740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x56103328c8f0 con 0x56103332bb00 2024-09-11T00:29:27.659 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:27.657+0000 7f58f7e00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561033326790 0x561033328b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:29:27.659 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:27.657+0000 7f58f7e00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561033326790 0x561033328b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:52300/0 (socket says 172.21.15.12:52300) 2024-09-11T00:29:27.659 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:27.657+0000 7f58f7e00640 1 -- 172.21.15.12:0/1873596809 learned_addr learned my addr 172.21.15.12:0/1873596809 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:29:27.660 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:27.658+0000 7f58f7400640 1 --2- 172.21.15.12:0/1873596809 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5610333290c0 0x56103332b4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:29:27.660 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:27.658+0000 7f58f7e00640 1 -- 172.21.15.12:0/1873596809 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56103332bb00 msgr2=0x56103332dfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:29:27.660 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:27.659+0000 7f58f6a00640 1 --2- 172.21.15.12:0/1873596809 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56103332bb00 0x56103332dfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:29:27.662 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:27.659+0000 7f58f7e00640 1 --2- 172.21.15.12:0/1873596809 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56103332bb00 0x56103332dfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:29:27.662 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:27.659+0000 7f58f7e00640 1 -- 172.21.15.12:0/1873596809 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5610333290c0 msgr2=0x56103332b4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:29:27.662 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:27.659+0000 7f58f7e00640 1 --2- 172.21.15.12:0/1873596809 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5610333290c0 0x56103332b4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:29:27.662 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:27.659+0000 7f58f7e00640 1 -- 172.21.15.12:0/1873596809 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561033255780 con 0x561033326790 2024-09-11T00:29:27.662 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:27.659+0000 7f58f7400640 1 --2- 172.21.15.12:0/1873596809 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5610333290c0 0x56103332b4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:29:27.662 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:27.659+0000 7f58f6a00640 1 --2- 172.21.15.12:0/1873596809 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56103332bb00 0x56103332dfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T00:29:27.662 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:27.660+0000 7f58efe00640 1 --2- 172.21.15.12:0/1277254572 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f58dc0b53e0 0x7f58dc0b78a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:29:27.662 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:27.660+0000 7f58efe00640 1 -- 172.21.15.12:0/1277254572 <== mon.1 v2:172.21.15.178:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9208+0+0 (secure 0 0 0) 0x7f58e4105a40 con 0x7f58f8078b70 2024-09-11T00:29:27.663 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:27.661+0000 7f58f7400640 1 --2- 172.21.15.12:0/1277254572 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f58dc0b53e0 0x7f58dc0b78a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:29:27.664 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:27.663+0000 7f58f7400640 1 --2- 172.21.15.12:0/1277254572 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f58dc0b53e0 0x7f58dc0b78a0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f58f0005c40 tx=0x7f58f000a6d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:29:27.664 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:27.663+0000 7f58f7e00640 1 --2- 172.21.15.12:0/1873596809 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561033326790 0x561033328b80 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f58e8009c80 tx=0x7f58e80117b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94cfe111e445a964 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:29:27.667 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:27.665+0000 7f58ecc00640 1 -- 172.21.15.12:0/1873596809 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58e8012450 con 0x561033326790 2024-09-11T00:29:27.667 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:27.665+0000 7f58ecc00640 1 -- 172.21.15.12:0/1873596809 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f58e8012aa0 con 0x561033326790 2024-09-11T00:29:27.667 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:27.665+0000 7f58ecc00640 1 -- 172.21.15.12:0/1873596809 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58e800fb20 con 0x561033326790 2024-09-11T00:29:27.667 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:27.666+0000 7f5901670740 1 -- 172.21.15.12:0/1873596809 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561033326790 msgr2=0x561033328b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:29:27.667 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:27.666+0000 7f5901670740 1 --2- 172.21.15.12:0/1873596809 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561033326790 0x561033328b80 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f58e8009c80 tx=0x7f58e80117b0 comp rx=0 tx=0).stop 2024-09-11T00:29:27.668 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:27.666+0000 7f5901670740 1 -- 172.21.15.12:0/1873596809 shutdown_connections 2024-09-11T00:29:27.668 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:27.666+0000 7f5901670740 1 --2- 172.21.15.12:0/1873596809 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56103332bb00 0x56103332dfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:29:27.668 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:27.666+0000 7f5901670740 1 --2- 172.21.15.12:0/1873596809 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5610333290c0 0x56103332b4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:29:27.668 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:27.666+0000 7f5901670740 1 --2- 172.21.15.12:0/1873596809 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561033326790 0x561033328b80 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:29:27.668 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:27.666+0000 7f5901670740 1 -- 172.21.15.12:0/1873596809 >> 172.21.15.12:0/1873596809 conn(0x5610333218e0 msgr2=0x561033322670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:29:27.668 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:27.667+0000 7f5901670740 1 -- 172.21.15.12:0/1873596809 shutdown_connections 2024-09-11T00:29:27.669 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:27.667+0000 7f5901670740 1 -- 172.21.15.12:0/1873596809 wait complete. 2024-09-11T00:29:27.669 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:27.667+0000 7f5901670740 1 -- start start 2024-09-11T00:29:27.669 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:27.668+0000 7f5901670740 1 Processor -- start 2024-09-11T00:29:27.669 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:27.668+0000 7f5901670740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561033326790 0x56103332ff20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:29:27.670 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:27.668+0000 7f5901670740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5610333290c0 0x5610333305c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:29:27.670 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:27.669+0000 7f5901670740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56103332bb00 0x561033331190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:29:27.671 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:27.669+0000 7f58f7400640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561033326790 0x56103332ff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:29:27.671 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:27.669+0000 7f58f7400640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561033326790 0x56103332ff20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.12:34126/0 (socket says 172.21.15.12:34126) 2024-09-11T00:29:27.671 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:27.669+0000 7f58f7400640 1 -- 172.21.15.12:0/464656592 learned_addr learned my addr 172.21.15.12:0/464656592 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:29:27.672 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:27.669+0000 7f58f7e00640 1 --2- 172.21.15.12:0/464656592 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56103332bb00 0x561033331190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:29:27.672 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:27.669+0000 7f5901670740 1 -- 172.21.15.12:0/464656592 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x5610331c0ab0 con 0x5610333290c0 2024-09-11T00:29:27.672 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:27.669+0000 7f5901670740 1 -- 172.21.15.12:0/464656592 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x56103322f570 con 0x56103332bb00 2024-09-11T00:29:27.672 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:27.669+0000 7f5901670740 1 -- 172.21.15.12:0/464656592 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x56103332eae0 con 0x561033326790 2024-09-11T00:29:27.672 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:27.669+0000 7f5901670740 10 client.0 _ll_register_callbacks cb 0x56103323e100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:29:27.672 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:27.670+0000 7f58f6a00640 1 --2- 172.21.15.12:0/464656592 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5610333290c0 0x5610333305c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:29:27.672 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:27.670+0000 7f58f7e00640 1 -- 172.21.15.12:0/464656592 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561033326790 msgr2=0x56103332ff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:29:27.672 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:27.670+0000 7f58f7e00640 1 --2- 172.21.15.12:0/464656592 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561033326790 0x56103332ff20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:29:27.672 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:27.670+0000 7f58f7e00640 1 -- 172.21.15.12:0/464656592 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5610333290c0 msgr2=0x5610333305c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:29:27.672 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:27.670+0000 7f58f7e00640 1 --2- 172.21.15.12:0/464656592 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5610333290c0 0x5610333305c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:29:27.673 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:27.670+0000 7f58f7e00640 1 -- 172.21.15.12:0/464656592 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561032fea790 con 0x56103332bb00 2024-09-11T00:29:27.673 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:27.672+0000 7f58f7e00640 1 --2- 172.21.15.12:0/464656592 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56103332bb00 0x561033331190 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f58e801c040 tx=0x7f58e8009fa0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=92a992172acb33d server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:29:27.673 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:27.672+0000 7f58e1600640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-09-11T00:29:27.674 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:27.673+0000 7f58e1600640 1 -- 172.21.15.12:0/464656592 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58e8028070 con 0x56103332bb00 2024-09-11T00:29:27.674 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:27.673+0000 7f58e1600640 1 -- 172.21.15.12:0/464656592 <== mon.2 v2:172.21.15.165:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 9208+0+0 (secure 0 0 0) 0x7f58e8004450 con 0x56103332bb00 2024-09-11T00:29:27.674 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:27.673+0000 7f5901670740 1 -- 172.21.15.12:0/464656592 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f58e8012aa0 con 0x56103332bb00 2024-09-11T00:29:27.675 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:27.673+0000 7f58e1600640 1 -- 172.21.15.12:0/464656592 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=100) ==== 24+0+0 (secure 0 0 0) 0x7f58e8032050 con 0x56103332bb00 2024-09-11T00:29:27.675 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:27.673+0000 7f5901670740 10 client.27914 fetch_fsmap learned FSMap version 100 2024-09-11T00:29:27.675 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:27.673+0000 7f5901670740 1 -- 172.21.15.12:0/464656592 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=100,monmap=4+}) -- 0x561033063620 con 0x56103332bb00 2024-09-11T00:29:27.675 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:27.674+0000 7f58e1600640 1 -- 172.21.15.12:0/464656592 <== mon.2 v2:172.21.15.165:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f58e8012aa0 con 0x56103332bb00 2024-09-11T00:29:27.677 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:27.675+0000 7f58e1600640 1 -- 172.21.15.12:0/464656592 <== mon.2 v2:172.21.15.165:3300/0 5 ==== fsmap(e 100) ==== 8390+0+0 (secure 0 0 0) 0x7f58e80035f0 con 0x56103332bb00 2024-09-11T00:29:27.677 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:27.675+0000 7f5901670740 10 client.27914 fetch_fsmap finished waiting for FSMap version 100 2024-09-11T00:29:27.677 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:27.675+0000 7f5901670740 10 client.27914 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T00:29:27.677 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:27.675+0000 7f5901670740 20 client.27914 populate_metadata read hostname 'smithi012' 2024-09-11T00:29:27.677 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:27.676+0000 7f5901670740 1 --2- 172.21.15.12:0/464656592 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x56103333b4a0 0x56103333d890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:29:27.677 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:27.676+0000 7f5901670740 4 client.27914 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T00:29:27.677 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:27.676+0000 7f5901670740 1 -- 172.21.15.12:0/464656592 --> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56103329b600 con 0x56103333b4a0 2024-09-11T00:29:27.681 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:27.680+0000 7f58f7400640 1 --2- 172.21.15.12:0/464656592 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x56103333b4a0 0x56103333d890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:29:27.683 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:27.682+0000 7f58f7400640 1 --2- 172.21.15.12:0/464656592 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x56103333b4a0 0x56103333d890 crc :-1 s=READY pgs=307 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=64c62505863a33b4 server_cookie=24edd4d9e0ab7c8c in_seq=0 out_seq=0 2024-09-11T00:29:27.683 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:27.682+0000 7f58e1600640 10 client.27914 ms_handle_connect on v2:172.21.15.165:6838/501312984 2024-09-11T00:29:27.689 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:27.688+0000 7f58e1600640 1 -- 172.21.15.12:0/464656592 <== mds.0 v2:172.21.15.165:6838/501312984 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x56103329b600 con 0x56103333b4a0 2024-09-11T00:29:27.689 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:27.688+0000 7f58e1600640 10 client.27914 handle_command_reply: tid=0 2024-09-11T00:29:27.689 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:27.688+0000 7f5901670740 1 client.27914 shutdown 2024-09-11T00:29:27.690 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:27.689+0000 7f5901670740 1 -- 172.21.15.12:0/464656592 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56103332bb00 msgr2=0x561033331190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:29:27.690 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:27.689+0000 7f5901670740 1 --2- 172.21.15.12:0/464656592 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56103332bb00 0x561033331190 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f58e801c040 tx=0x7f58e8009fa0 comp rx=0 tx=0).stop 2024-09-11T00:29:27.690 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:27.689+0000 7f5901670740 1 -- 172.21.15.12:0/464656592 shutdown_connections 2024-09-11T00:29:27.690 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:27.689+0000 7f5901670740 1 --2- 172.21.15.12:0/464656592 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x56103333b4a0 0x56103333d890 crc :-1 s=READY pgs=307 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:29:27.691 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:27.689+0000 7f5901670740 1 --2- 172.21.15.12:0/464656592 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56103332bb00 0x561033331190 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:29:27.691 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:27.689+0000 7f5901670740 1 --2- 172.21.15.12:0/464656592 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5610333290c0 0x5610333305c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:29:27.691 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:27.689+0000 7f5901670740 1 --2- 172.21.15.12:0/464656592 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561033326790 0x56103332ff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:29:27.691 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:27.689+0000 7f5901670740 1 -- 172.21.15.12:0/464656592 >> 172.21.15.12:0/464656592 conn(0x5610333218e0 msgr2=0x561033320a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:29:27.691 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:27.689+0000 7f58e1600640 0 client.27914 ms_handle_reset on v2:172.21.15.165:6838/501312984 2024-09-11T00:29:27.691 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:27.690+0000 7f5901670740 1 -- 172.21.15.12:0/464656592 shutdown_connections 2024-09-11T00:29:27.692 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:27.690+0000 7f5901670740 1 -- 172.21.15.12:0/464656592 wait complete. 2024-09-11T00:29:27.692 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:27.690+0000 7f5901670740 20 client.27914 trim_cache size 0 max 16384 2024-09-11T00:29:27.726 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:27.724+0000 7f5901670740 1 Processor -- start 2024-09-11T00:29:27.726 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:27.724+0000 7f5901670740 1 -- start start 2024-09-11T00:29:27.726 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:27.724+0000 7f5901670740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561033326790 0x56103332eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:29:27.727 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:27.724+0000 7f5901670740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5610333290c0 0x56103332f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:29:27.727 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:27.724+0000 7f5901670740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56103332bb00 0x561033323710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:29:27.727 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:27.724+0000 7f5901670740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f58f81a9480 con 0x56103332bb00 2024-09-11T00:29:27.727 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:27.724+0000 7f5901670740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f58f81ab0b0 con 0x561033326790 2024-09-11T00:29:27.727 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:27.724+0000 7f5901670740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f58f81a9bb0 con 0x5610333290c0 2024-09-11T00:29:27.727 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:27.725+0000 7f58f7e00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56103332bb00 0x561033323710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:29:27.727 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:27.725+0000 7f58f7e00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56103332bb00 0x561033323710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:52308/0 (socket says 172.21.15.12:52308) 2024-09-11T00:29:27.727 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:27.725+0000 7f58f7e00640 1 -- 172.21.15.12:0/852541252 learned_addr learned my addr 172.21.15.12:0/852541252 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:29:27.727 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:27.725+0000 7f58f6a00640 1 --2- 172.21.15.12:0/852541252 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5610333290c0 0x56103332f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:29:27.728 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:27.726+0000 7f58f7e00640 1 -- 172.21.15.12:0/852541252 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5610333290c0 msgr2=0x56103332f3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:29:27.728 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:27.726+0000 7f58f7e00640 1 --2- 172.21.15.12:0/852541252 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5610333290c0 0x56103332f3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:29:27.728 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:27.726+0000 7f58f7e00640 1 -- 172.21.15.12:0/852541252 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561033326790 msgr2=0x56103332eeb0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-11T00:29:27.728 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:27.726+0000 7f58f7e00640 1 --2- 172.21.15.12:0/852541252 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561033326790 0x56103332eeb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:29:27.728 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:27.726+0000 7f58f7e00640 1 -- 172.21.15.12:0/852541252 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56103333dc00 con 0x56103332bb00 2024-09-11T00:29:27.728 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:27.726+0000 7f58f7e00640 1 --2- 172.21.15.12:0/852541252 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56103332bb00 0x561033323710 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f58e8003b30 tx=0x7f58e80097a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5492785766217f2 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:29:27.729 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:27.727+0000 7f58e3e00640 1 -- 172.21.15.12:0/852541252 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58e803b070 con 0x56103332bb00 2024-09-11T00:29:27.729 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:27.727+0000 7f58e3e00640 1 -- 172.21.15.12:0/852541252 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f58e80035f0 con 0x56103332bb00 2024-09-11T00:29:27.729 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:27.727+0000 7f58e3e00640 1 -- 172.21.15.12:0/852541252 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58e8028040 con 0x56103332bb00 2024-09-11T00:29:27.730 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:27.728+0000 7f5901670740 1 -- 172.21.15.12:0/852541252 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56103332bb00 msgr2=0x561033323710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:29:27.730 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:27.728+0000 7f5901670740 1 --2- 172.21.15.12:0/852541252 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56103332bb00 0x561033323710 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f58e8003b30 tx=0x7f58e80097a0 comp rx=0 tx=0).stop 2024-09-11T00:29:27.730 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:27.729+0000 7f5901670740 1 -- 172.21.15.12:0/852541252 shutdown_connections 2024-09-11T00:29:27.730 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:27.729+0000 7f5901670740 1 --2- 172.21.15.12:0/852541252 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56103332bb00 0x561033323710 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:29:27.730 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:27.729+0000 7f5901670740 1 --2- 172.21.15.12:0/852541252 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5610333290c0 0x56103332f3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:29:27.730 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:27.729+0000 7f5901670740 1 --2- 172.21.15.12:0/852541252 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561033326790 0x56103332eeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:29:27.730 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:27.729+0000 7f5901670740 1 -- 172.21.15.12:0/852541252 >> 172.21.15.12:0/852541252 conn(0x5610333218e0 msgr2=0x56103332bee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:29:27.731 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:27.730+0000 7f5901670740 1 -- 172.21.15.12:0/852541252 shutdown_connections 2024-09-11T00:29:27.731 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:27.730+0000 7f5901670740 1 -- 172.21.15.12:0/852541252 wait complete. 2024-09-11T00:29:27.732 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:27.730+0000 7f5901670740 1 -- start start 2024-09-11T00:29:27.732 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:27.730+0000 7f5901670740 1 Processor -- start 2024-09-11T00:29:27.732 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:27.731+0000 7f5901670740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561033326790 0x561033337260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:29:27.732 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:27.731+0000 7f5901670740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5610333290c0 0x5610333377a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:29:27.732 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:27.731+0000 7f5901670740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56103332bb00 0x561033335b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:29:27.732 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:27.731+0000 7f5901670740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f58f81aa9e0 con 0x5610333290c0 2024-09-11T00:29:27.733 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:27.731+0000 7f5901670740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f58f81aa310 con 0x561033326790 2024-09-11T00:29:27.733 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:27.731+0000 7f5901670740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f58f810caf0 con 0x56103332bb00 2024-09-11T00:29:27.733 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:27.731+0000 7f5901670740 10 client.0 _ll_register_callbacks cb 0x7f58f000a570 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:29:27.733 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:27.732+0000 7f58f7e00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56103332bb00 0x561033335b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:29:27.734 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:27.732+0000 7f58f7e00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56103332bb00 0x561033335b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.12:34138/0 (socket says 172.21.15.12:34138) 2024-09-11T00:29:27.734 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:27.732+0000 7f58f7e00640 1 -- 172.21.15.12:0/4012326413 learned_addr learned my addr 172.21.15.12:0/4012326413 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:29:27.734 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:27.732+0000 7f58f7400640 1 --2- 172.21.15.12:0/4012326413 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561033326790 0x561033337260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:29:27.734 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:27.732+0000 7f58f6a00640 1 --2- 172.21.15.12:0/4012326413 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5610333290c0 0x5610333377a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:29:27.734 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:27.732+0000 7f58f7400640 1 -- 172.21.15.12:0/4012326413 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56103332bb00 msgr2=0x561033335b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:29:27.734 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:27.732+0000 7f58f7400640 1 --2- 172.21.15.12:0/4012326413 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56103332bb00 0x561033335b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:29:27.734 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:27.732+0000 7f58f7400640 1 -- 172.21.15.12:0/4012326413 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5610333290c0 msgr2=0x5610333377a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:29:27.734 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:27.732+0000 7f58f7400640 1 --2- 172.21.15.12:0/4012326413 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5610333290c0 0x5610333377a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:29:27.734 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:27.732+0000 7f58f7400640 1 -- 172.21.15.12:0/4012326413 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561033328ef0 con 0x561033326790 2024-09-11T00:29:27.734 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:27.732+0000 7f58f6a00640 1 --2- 172.21.15.12:0/4012326413 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5610333290c0 0x5610333377a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T00:29:27.735 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:27.733+0000 7f58f7400640 1 --2- 172.21.15.12:0/4012326413 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561033326790 0x561033337260 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x56103333df90 tx=0x7f58f0004ab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=88777929958b9a05 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:29:27.735 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:27.733+0000 7f58e1600640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-09-11T00:29:27.735 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:27.734+0000 7f58e1600640 1 -- 172.21.15.12:0/4012326413 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58f0010570 con 0x561033326790 2024-09-11T00:29:27.735 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:27.734+0000 7f58e1600640 1 -- 172.21.15.12:0/4012326413 <== mon.2 v2:172.21.15.165:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 9208+0+0 (secure 0 0 0) 0x7f58f0004b20 con 0x561033326790 2024-09-11T00:29:27.736 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:27.735+0000 7f5901670740 1 -- 172.21.15.12:0/4012326413 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f58e80035f0 con 0x561033326790 2024-09-11T00:29:27.737 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:27.736+0000 7f58e1600640 1 -- 172.21.15.12:0/4012326413 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=100) ==== 24+0+0 (secure 0 0 0) 0x7f58f0004d20 con 0x561033326790 2024-09-11T00:29:27.737 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:27.736+0000 7f5901670740 10 client.27917 fetch_fsmap learned FSMap version 100 2024-09-11T00:29:27.737 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:27.736+0000 7f5901670740 1 -- 172.21.15.12:0/4012326413 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=100,monmap=4+}) -- 0x5610333a8ab0 con 0x561033326790 2024-09-11T00:29:27.738 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:27.736+0000 7f58e1600640 1 -- 172.21.15.12:0/4012326413 <== mon.2 v2:172.21.15.165:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f58e80035f0 con 0x561033326790 2024-09-11T00:29:27.739 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:27.737+0000 7f58e1600640 1 -- 172.21.15.12:0/4012326413 <== mon.2 v2:172.21.15.165:3300/0 5 ==== fsmap(e 100) ==== 8390+0+0 (secure 0 0 0) 0x7f58f0005400 con 0x561033326790 2024-09-11T00:29:27.741 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:27.738+0000 7f5901670740 10 client.27917 fetch_fsmap finished waiting for FSMap version 100 2024-09-11T00:29:27.741 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:27.738+0000 7f5901670740 10 client.27917 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T00:29:27.741 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:27.738+0000 7f5901670740 20 client.27917 populate_metadata read hostname 'smithi012' 2024-09-11T00:29:27.741 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:27.738+0000 7f5901670740 1 --2- 172.21.15.12:0/4012326413 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x5610333a9400 0x5610333ab820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:29:27.741 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:27.738+0000 7f5901670740 4 client.27917 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T00:29:27.741 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:27.738+0000 7f5901670740 1 -- 172.21.15.12:0/4012326413 --> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5610332970b0 con 0x5610333a9400 2024-09-11T00:29:27.742 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:27.740+0000 7f58f6a00640 1 --2- 172.21.15.12:0/4012326413 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x5610333a9400 0x5610333ab820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:29:27.743 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:27.742+0000 7f58f6a00640 1 --2- 172.21.15.12:0/4012326413 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x5610333a9400 0x5610333ab820 crc :-1 s=READY pgs=308 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5e7d5270f16b0f2f server_cookie=cb518be5802f141 in_seq=0 out_seq=0 2024-09-11T00:29:27.743 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:27.742+0000 7f58e1600640 10 client.27917 ms_handle_connect on v2:172.21.15.165:6838/501312984 2024-09-11T00:29:28.189 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:28.187+0000 7f58e1600640 1 -- 172.21.15.12:0/4012326413 <== mds.0 v2:172.21.15.165:6838/501312984 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5610332970b0 con 0x5610333a9400 2024-09-11T00:29:28.189 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:28.188+0000 7f58e1600640 10 client.27917 handle_command_reply: tid=0 2024-09-11T00:29:28.190 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:28.188+0000 7f5901670740 1 client.27917 shutdown 2024-09-11T00:29:28.190 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:28.188+0000 7f5901670740 1 -- 172.21.15.12:0/4012326413 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561033326790 msgr2=0x561033337260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:29:28.190 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:28.188+0000 7f5901670740 1 --2- 172.21.15.12:0/4012326413 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561033326790 0x561033337260 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x56103333df90 tx=0x7f58f0004ab0 comp rx=0 tx=0).stop 2024-09-11T00:29:28.190 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:28.188+0000 7f5901670740 1 -- 172.21.15.12:0/4012326413 shutdown_connections 2024-09-11T00:29:28.190 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:28.188+0000 7f5901670740 1 --2- 172.21.15.12:0/4012326413 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x5610333a9400 0x5610333ab820 crc :-1 s=READY pgs=308 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:29:28.190 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:28.188+0000 7f5901670740 1 --2- 172.21.15.12:0/4012326413 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56103332bb00 0x561033335b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:29:28.190 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:28.188+0000 7f5901670740 1 --2- 172.21.15.12:0/4012326413 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5610333290c0 0x5610333377a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:29:28.191 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:28.188+0000 7f5901670740 1 --2- 172.21.15.12:0/4012326413 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561033326790 0x561033337260 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:29:28.191 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:28.188+0000 7f5901670740 1 -- 172.21.15.12:0/4012326413 >> 172.21.15.12:0/4012326413 conn(0x5610333218e0 msgr2=0x561033322140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:29:28.191 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:28.189+0000 7f58e1600640 0 client.27917 ms_handle_reset on v2:172.21.15.165:6838/501312984 2024-09-11T00:29:28.191 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:28.189+0000 7f5901670740 1 -- 172.21.15.12:0/4012326413 shutdown_connections 2024-09-11T00:29:28.191 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:28.189+0000 7f5901670740 1 -- 172.21.15.12:0/4012326413 wait complete. 2024-09-11T00:29:28.191 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:28.190+0000 7f5901670740 20 client.27917 trim_cache size 0 max 16384 2024-09-11T00:29:28.193 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:{ 2024-09-11T00:29:28.193 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "message": "", 2024-09-11T00:29:28.193 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "return_code": 0 2024-09-11T00:29:28.193 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:} 2024-09-11T00:29:28.193 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:28.192+0000 7f58fe600640 1 -- 172.21.15.12:0/1277254572 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f58dc0b53e0 msgr2=0x7f58dc0b78a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:29:28.193 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:28.192+0000 7f58fe600640 1 --2- 172.21.15.12:0/1277254572 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f58dc0b53e0 0x7f58dc0b78a0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f58f0005c40 tx=0x7f58f000a6d0 comp rx=0 tx=0).stop 2024-09-11T00:29:28.194 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:28.192+0000 7f58fe600640 1 -- 172.21.15.12:0/1277254572 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f58f8078b70 msgr2=0x7f58f819f980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:29:28.194 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:28.192+0000 7f58fe600640 1 --2- 172.21.15.12:0/1277254572 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f58f8078b70 0x7f58f819f980 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f58e40057c0 tx=0x7f58e4032eb0 comp rx=0 tx=0).stop 2024-09-11T00:29:28.194 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:28.193+0000 7f58fe600640 1 -- 172.21.15.12:0/1277254572 shutdown_connections 2024-09-11T00:29:28.194 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:28.193+0000 7f58fe600640 1 --2- 172.21.15.12:0/1277254572 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f58dc0b53e0 0x7f58dc0b78a0 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:29:28.194 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:28.193+0000 7f58fe600640 1 --2- 172.21.15.12:0/1277254572 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f58f8079530 0x7f58f81a05b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:29:28.194 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:28.193+0000 7f58fe600640 1 --2- 172.21.15.12:0/1277254572 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f58f8078b70 0x7f58f819f980 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:29:28.194 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:28.193+0000 7f58fe600640 1 --2- 172.21.15.12:0/1277254572 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f58f8077a20 0x7f58f819f330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:29:28.194 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:28.193+0000 7f58fe600640 1 -- 172.21.15.12:0/1277254572 >> 172.21.15.12:0/1277254572 conn(0x7f58f8100460 msgr2=0x7f58f8101f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:29:28.196 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:28.195+0000 7f58fe600640 1 -- 172.21.15.12:0/1277254572 shutdown_connections 2024-09-11T00:29:28.196 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:28.195+0000 7f58fe600640 1 -- 172.21.15.12:0/1277254572 wait complete. 2024-09-11T00:29:28.259 INFO:tasks.workunit.client.0.smithi012.stdout: 10 160623 15.48 MB/sec execute 444 sec latency 335.333 ms 2024-09-11T00:29:28.314 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:29:28 smithi012 ceph-mon[30333]: pgmap v1071: 129 pgs: 129 active+clean; 757 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 4.9 MiB/s wr, 792 op/s 2024-09-11T00:29:28.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:29:28 smithi178 ceph-mon[35919]: pgmap v1071: 129 pgs: 129 active+clean; 757 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 4.9 MiB/s wr, 792 op/s 2024-09-11T00:29:28.532 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:29:28 smithi165 ceph-mon[35356]: pgmap v1071: 129 pgs: 129 active+clean; 757 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 4.9 MiB/s wr, 792 op/s 2024-09-11T00:29:29.259 INFO:tasks.workunit.client.0.smithi012.stdout: 10 161189 15.51 MB/sec execute 445 sec latency 114.781 ms 2024-09-11T00:29:30.259 INFO:tasks.workunit.client.0.smithi012.stdout: 10 161984 15.58 MB/sec execute 446 sec latency 160.536 ms 2024-09-11T00:29:30.314 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:29:30 smithi012 ceph-mon[30333]: pgmap v1072: 129 pgs: 129 active+clean; 758 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 MiB/s rd, 3.7 MiB/s wr, 556 op/s 2024-09-11T00:29:30.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:29:30 smithi178 ceph-mon[35919]: pgmap v1072: 129 pgs: 129 active+clean; 758 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 MiB/s rd, 3.7 MiB/s wr, 556 op/s 2024-09-11T00:29:30.532 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:29:30 smithi165 ceph-mon[35356]: pgmap v1072: 129 pgs: 129 active+clean; 758 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 MiB/s rd, 3.7 MiB/s wr, 556 op/s 2024-09-11T00:29:31.259 INFO:tasks.workunit.client.0.smithi012.stdout: 10 162650 15.61 MB/sec execute 447 sec latency 91.077 ms 2024-09-11T00:29:32.259 INFO:tasks.workunit.client.0.smithi012.stdout: 10 163397 15.67 MB/sec execute 448 sec latency 58.127 ms 2024-09-11T00:29:32.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:29:32 smithi012 ceph-mon[30333]: pgmap v1073: 129 pgs: 129 active+clean; 763 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 4.1 MiB/s wr, 641 op/s 2024-09-11T00:29:32.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:29:32 smithi178 ceph-mon[35919]: pgmap v1073: 129 pgs: 129 active+clean; 763 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 4.1 MiB/s wr, 641 op/s 2024-09-11T00:29:32.532 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:29:32 smithi165 ceph-mon[35356]: pgmap v1073: 129 pgs: 129 active+clean; 763 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 4.1 MiB/s wr, 641 op/s 2024-09-11T00:29:33.259 INFO:tasks.workunit.client.0.smithi012.stdout: 10 164113 15.73 MB/sec execute 449 sec latency 60.617 ms 2024-09-11T00:29:34.264 INFO:tasks.workunit.client.0.smithi012.stdout: 10 164399 15.73 MB/sec execute 450 sec latency 312.075 ms 2024-09-11T00:29:34.292 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:29:34 smithi012 ceph-mon[30333]: pgmap v1074: 129 pgs: 129 active+clean; 770 MiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 MiB/s rd, 4.5 MiB/s wr, 762 op/s 2024-09-11T00:29:34.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:29:34 smithi178 ceph-mon[35919]: pgmap v1074: 129 pgs: 129 active+clean; 770 MiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 MiB/s rd, 4.5 MiB/s wr, 762 op/s 2024-09-11T00:29:34.532 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:29:34 smithi165 ceph-mon[35356]: pgmap v1074: 129 pgs: 129 active+clean; 770 MiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 MiB/s rd, 4.5 MiB/s wr, 762 op/s 2024-09-11T00:29:35.274 INFO:tasks.workunit.client.0.smithi012.stdout: 10 164401 15.70 MB/sec execute 451 sec latency 1308.198 ms 2024-09-11T00:29:36.221 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:29:36 smithi012 ceph-mon[30333]: pgmap v1075: 129 pgs: 129 active+clean; 771 MiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 3.5 MiB/s wr, 596 op/s 2024-09-11T00:29:36.274 INFO:tasks.workunit.client.0.smithi012.stdout: 10 164404 15.66 MB/sec execute 452 sec latency 2308.264 ms 2024-09-11T00:29:36.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:29:36 smithi178 ceph-mon[35919]: pgmap v1075: 129 pgs: 129 active+clean; 771 MiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 3.5 MiB/s wr, 596 op/s 2024-09-11T00:29:36.532 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:29:36 smithi165 ceph-mon[35356]: pgmap v1075: 129 pgs: 129 active+clean; 771 MiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 3.5 MiB/s wr, 596 op/s 2024-09-11T00:29:37.274 INFO:tasks.workunit.client.0.smithi012.stdout: 10 164406 15.63 MB/sec execute 453 sec latency 3308.328 ms 2024-09-11T00:29:37.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:29:37 smithi012 ceph-mon[30333]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T00:29:37.532 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:29:37 smithi165 ceph-mon[35356]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T00:29:37.581 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:29:37 smithi178 ceph-mon[35919]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T00:29:38.275 INFO:tasks.workunit.client.0.smithi012.stdout: 10 164406 15.59 MB/sec execute 454 sec latency 4309.386 ms 2024-09-11T00:29:38.406 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:29:38 smithi012 ceph-mon[30333]: pgmap v1076: 129 pgs: 129 active+clean; 856 MiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 MiB/s rd, 12 MiB/s wr, 852 op/s 2024-09-11T00:29:38.532 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:29:38 smithi165 ceph-mon[35356]: pgmap v1076: 129 pgs: 129 active+clean; 856 MiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 MiB/s rd, 12 MiB/s wr, 852 op/s 2024-09-11T00:29:38.581 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:29:38 smithi178 ceph-mon[35919]: pgmap v1076: 129 pgs: 129 active+clean; 856 MiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 MiB/s rd, 12 MiB/s wr, 852 op/s 2024-09-11T00:29:39.275 INFO:tasks.workunit.client.0.smithi012.stdout: 10 164406 15.56 MB/sec execute 455 sec latency 5309.455 ms 2024-09-11T00:29:40.276 INFO:tasks.workunit.client.0.smithi012.stdout: 10 164408 15.52 MB/sec execute 456 sec latency 6309.513 ms 2024-09-11T00:29:40.532 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:29:40 smithi165 ceph-mon[35356]: pgmap v1077: 129 pgs: 129 active+clean; 868 MiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 11 MiB/s wr, 649 op/s 2024-09-11T00:29:40.563 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:29:40 smithi012 ceph-mon[30333]: pgmap v1077: 129 pgs: 129 active+clean; 868 MiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 11 MiB/s wr, 649 op/s 2024-09-11T00:29:40.581 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:29:40 smithi178 ceph-mon[35919]: pgmap v1077: 129 pgs: 129 active+clean; 868 MiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 11 MiB/s wr, 649 op/s 2024-09-11T00:29:41.276 INFO:tasks.workunit.client.0.smithi012.stdout: 10 164408 15.49 MB/sec execute 457 sec latency 7309.578 ms 2024-09-11T00:29:41.924 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:41.922+0000 7f5893200640 1 -- 172.21.15.12:0/2472715805 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f589406b470 msgr2=0x7f5894105360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:29:41.924 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:41.922+0000 7f5893200640 1 --2- 172.21.15.12:0/2472715805 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f589406b470 0x7f5894105360 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f5884009900 tx=0x7f588402a410 comp rx=0 tx=0).stop 2024-09-11T00:29:41.924 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:41.922+0000 7f5893200640 1 -- 172.21.15.12:0/2472715805 shutdown_connections 2024-09-11T00:29:41.924 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:41.922+0000 7f5893200640 1 --2- 172.21.15.12:0/2472715805 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f58941084a0 0x7f589410a960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:29:41.924 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:41.922+0000 7f5893200640 1 --2- 172.21.15.12:0/2472715805 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f58941058f0 0x7f5894107d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:29:41.924 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:41.922+0000 7f5893200640 1 --2- 172.21.15.12:0/2472715805 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f589406b470 0x7f5894105360 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:29:41.924 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:41.922+0000 7f5893200640 1 -- 172.21.15.12:0/2472715805 >> 172.21.15.12:0/2472715805 conn(0x7f58940fcf30 msgr2=0x7f58940ff370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:29:41.925 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:41.922+0000 7f5893200640 1 -- 172.21.15.12:0/2472715805 shutdown_connections 2024-09-11T00:29:41.925 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:41.922+0000 7f5893200640 1 -- 172.21.15.12:0/2472715805 wait complete. 2024-09-11T00:29:41.927 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:41.923+0000 7f5893200640 1 Processor -- start 2024-09-11T00:29:41.927 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:41.923+0000 7f5893200640 1 -- start start 2024-09-11T00:29:41.927 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:41.923+0000 7f5893200640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f589406b470 0x7f5894102aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:29:41.927 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:41.923+0000 7f5893200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f58941058f0 0x7f5894101300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:29:41.927 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:41.923+0000 7f5893200640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f58941084a0 0x7f5894101840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:29:41.927 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:41.923+0000 7f5893200640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f589410e3f0 con 0x7f58941084a0 2024-09-11T00:29:41.927 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:41.923+0000 7f5893200640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f589410e270 con 0x7f58941058f0 2024-09-11T00:29:41.927 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:41.923+0000 7f5893200640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f589410e570 con 0x7f589406b470 2024-09-11T00:29:41.927 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:41.923+0000 7f5892800640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f58941084a0 0x7f5894101840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:29:41.927 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:41.923+0000 7f5892800640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f58941084a0 0x7f5894101840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:51580/0 (socket says 172.21.15.12:51580) 2024-09-11T00:29:41.928 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:41.923+0000 7f5892800640 1 -- 172.21.15.12:0/2624477162 learned_addr learned my addr 172.21.15.12:0/2624477162 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:29:41.928 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:41.923+0000 7f5889400640 1 --2- 172.21.15.12:0/2624477162 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f58941058f0 0x7f5894101300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:29:41.928 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:41.923+0000 7f5891e00640 1 --2- 172.21.15.12:0/2624477162 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f589406b470 0x7f5894102aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:29:41.928 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:41.923+0000 7f5892800640 1 -- 172.21.15.12:0/2624477162 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f589406b470 msgr2=0x7f5894102aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:29:41.928 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:41.923+0000 7f5892800640 1 --2- 172.21.15.12:0/2624477162 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f589406b470 0x7f5894102aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:29:41.928 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:41.923+0000 7f5892800640 1 -- 172.21.15.12:0/2624477162 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f58941058f0 msgr2=0x7f5894101300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:29:41.928 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:41.923+0000 7f5892800640 1 --2- 172.21.15.12:0/2624477162 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f58941058f0 0x7f5894101300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:29:41.928 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:41.923+0000 7f5892800640 1 -- 172.21.15.12:0/2624477162 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58941020b0 con 0x7f58941084a0 2024-09-11T00:29:41.928 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:41.923+0000 7f5892800640 1 --2- 172.21.15.12:0/2624477162 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f58941084a0 0x7f5894101840 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f588c00eaa0 tx=0x7f588c00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:29:41.928 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:41.923+0000 7f588b400640 1 -- 172.21.15.12:0/2624477162 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f588c00cdb0 con 0x7f58941084a0 2024-09-11T00:29:41.928 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:41.923+0000 7f588b400640 1 -- 172.21.15.12:0/2624477162 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f588c005020 con 0x7f58941084a0 2024-09-11T00:29:41.928 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:41.923+0000 7f588b400640 1 -- 172.21.15.12:0/2624477162 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f588c0116d0 con 0x7f58941084a0 2024-09-11T00:29:41.928 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:41.923+0000 7f5893200640 1 -- 172.21.15.12:0/2624477162 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58941a2c50 con 0x7f58941084a0 2024-09-11T00:29:41.929 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:41.923+0000 7f5893200640 1 -- 172.21.15.12:0/2624477162 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58941a31c0 con 0x7f58941084a0 2024-09-11T00:29:41.929 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:41.923+0000 7f589c0b9740 1 Processor -- start 2024-09-11T00:29:41.931 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:41.923+0000 7f588b400640 1 -- 172.21.15.12:0/2624477162 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7f588c00f040 con 0x7f58941084a0 2024-09-11T00:29:41.932 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:41.929+0000 7f589c0b9740 1 -- start start 2024-09-11T00:29:41.932 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:41.929+0000 7f589c0b9740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56261bf39970 0x56261bf3bd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:29:41.932 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:41.929+0000 7f589c0b9740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56261bf3c2a0 0x56261bf3e690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:29:41.932 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:41.929+0000 7f589c0b9740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56261bf3ece0 0x56261bf411a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:29:41.932 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:41.929+0000 7f589c0b9740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x56261beac6b0 con 0x56261bf39970 2024-09-11T00:29:41.932 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:41.929+0000 7f589c0b9740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x56261be9bae0 con 0x56261bf3ece0 2024-09-11T00:29:41.932 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:41.929+0000 7f589c0b9740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x56261be9f8f0 con 0x56261bf3c2a0 2024-09-11T00:29:41.932 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:41.929+0000 7f5891e00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56261bf3c2a0 0x56261bf3e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:29:41.932 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:41.929+0000 7f5889400640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56261bf39970 0x56261bf3bd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:29:41.932 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:41.929+0000 7f5891e00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56261bf3c2a0 0x56261bf3e690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.12:56938/0 (socket says 172.21.15.12:56938) 2024-09-11T00:29:41.932 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:41.929+0000 7f5891e00640 1 -- 172.21.15.12:0/3365826723 learned_addr learned my addr 172.21.15.12:0/3365826723 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:29:41.932 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:41.929+0000 7f5889400640 1 --2- 172.21.15.12:0/3365826723 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56261bf3ece0 0x56261bf411a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:29:41.933 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:41.929+0000 7f5891e00640 1 -- 172.21.15.12:0/3365826723 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56261bf3ece0 msgr2=0x56261bf411a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:29:41.933 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:41.929+0000 7f5891e00640 1 --2- 172.21.15.12:0/3365826723 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56261bf3ece0 0x56261bf411a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:29:41.933 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:41.929+0000 7f5891e00640 1 -- 172.21.15.12:0/3365826723 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56261bf39970 msgr2=0x56261bf3bd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:29:41.933 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:41.929+0000 7f5891e00640 1 --2- 172.21.15.12:0/3365826723 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56261bf39970 0x56261bf3bd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:29:41.933 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:41.929+0000 7f5891e00640 1 -- 172.21.15.12:0/3365826723 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56261be68780 con 0x56261bf3c2a0 2024-09-11T00:29:41.933 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:41.929+0000 7f5889400640 1 --2- 172.21.15.12:0/3365826723 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56261bf39970 0x56261bf3bd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:29:41.933 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:41.929+0000 7f5891e00640 1 --2- 172.21.15.12:0/3365826723 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56261bf3c2a0 0x56261bf3e690 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f5884009900 tx=0x7f5884031b10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=aa256a4d5438dcf9 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:29:41.933 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:41.929+0000 7f5883400640 1 -- 172.21.15.12:0/3365826723 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5884035030 con 0x56261bf3c2a0 2024-09-11T00:29:41.933 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:41.929+0000 7f5883400640 1 -- 172.21.15.12:0/3365826723 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5884040070 con 0x56261bf3c2a0 2024-09-11T00:29:41.933 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:41.929+0000 7f5883400640 1 -- 172.21.15.12:0/3365826723 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5884003970 con 0x56261bf3c2a0 2024-09-11T00:29:41.933 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:41.929+0000 7f588b400640 1 --2- 172.21.15.12:0/2624477162 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f58780b5360 0x7f58780b7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:29:41.937 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:41.929+0000 7f588b400640 1 -- 172.21.15.12:0/2624477162 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9208+0+0 (secure 0 0 0) 0x7f588c0e5d00 con 0x7f58941084a0 2024-09-11T00:29:41.937 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:41.929+0000 7f5889400640 1 --2- 172.21.15.12:0/2624477162 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f58780b5360 0x7f58780b7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:29:41.937 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:41.929+0000 7f589c0b9740 1 -- 172.21.15.12:0/3365826723 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56261bf3c2a0 msgr2=0x56261bf3e690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:29:41.938 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:41.929+0000 7f589c0b9740 1 --2- 172.21.15.12:0/3365826723 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56261bf3c2a0 0x56261bf3e690 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f5884009900 tx=0x7f5884031b10 comp rx=0 tx=0).stop 2024-09-11T00:29:41.938 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:41.929+0000 7f589c0b9740 1 -- 172.21.15.12:0/3365826723 shutdown_connections 2024-09-11T00:29:41.938 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:41.929+0000 7f589c0b9740 1 --2- 172.21.15.12:0/3365826723 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56261bf3ece0 0x56261bf411a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:29:41.938 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:41.929+0000 7f589c0b9740 1 --2- 172.21.15.12:0/3365826723 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56261bf3c2a0 0x56261bf3e690 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:29:41.938 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:41.929+0000 7f589c0b9740 1 --2- 172.21.15.12:0/3365826723 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56261bf39970 0x56261bf3bd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:29:41.938 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:41.929+0000 7f589c0b9740 1 -- 172.21.15.12:0/3365826723 >> 172.21.15.12:0/3365826723 conn(0x56261bf34ac0 msgr2=0x56261bf35850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:29:41.938 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:41.929+0000 7f589c0b9740 1 -- 172.21.15.12:0/3365826723 shutdown_connections 2024-09-11T00:29:41.938 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:41.929+0000 7f589c0b9740 1 -- 172.21.15.12:0/3365826723 wait complete. 2024-09-11T00:29:41.938 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:41.929+0000 7f5889400640 1 --2- 172.21.15.12:0/2624477162 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f58780b5360 0x7f58780b7820 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f587c009fd0 tx=0x7f587c005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:29:41.938 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:41.929+0000 7f589c0b9740 1 -- start start 2024-09-11T00:29:41.938 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:41.929+0000 7f589c0b9740 1 Processor -- start 2024-09-11T00:29:41.938 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:41.929+0000 7f589c0b9740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56261bf39970 0x56261bf43010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:29:41.939 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:41.929+0000 7f589c0b9740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56261bf3c2a0 0x56261bf436b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:29:41.939 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:41.929+0000 7f589c0b9740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56261bf3ece0 0x56261bf44280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:29:41.939 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:41.929+0000 7f589c0b9740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x56261bdd3ab0 con 0x56261bf3ece0 2024-09-11T00:29:41.939 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:41.929+0000 7f589c0b9740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x56261be42570 con 0x56261bf39970 2024-09-11T00:29:41.939 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:41.929+0000 7f589c0b9740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x56261bf41c60 con 0x56261bf3c2a0 2024-09-11T00:29:41.939 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:41.929+0000 7f589c0b9740 10 client.0 _ll_register_callbacks cb 0x56261be51100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:29:41.939 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:41.929+0000 7f5892800640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56261bf3ece0 0x56261bf44280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:29:41.939 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:41.929+0000 7f5891e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56261bf39970 0x56261bf43010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:29:41.939 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:41.929+0000 7f5889400640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56261bf3c2a0 0x56261bf436b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:29:41.939 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:41.929+0000 7f5891e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56261bf39970 0x56261bf43010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.12:48218/0 (socket says 172.21.15.12:48218) 2024-09-11T00:29:41.939 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:41.929+0000 7f5891e00640 1 -- 172.21.15.12:0/2996561543 learned_addr learned my addr 172.21.15.12:0/2996561543 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:29:41.939 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:41.929+0000 7f5889400640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56261bf3c2a0 0x56261bf436b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.12:56952/0 (socket says 172.21.15.12:56952) 2024-09-11T00:29:41.939 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:41.929+0000 7f5891e00640 1 -- 172.21.15.12:0/2996561543 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56261bf3c2a0 msgr2=0x56261bf436b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:29:41.940 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:41.929+0000 7f5891e00640 1 --2- 172.21.15.12:0/2996561543 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56261bf3c2a0 0x56261bf436b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:29:41.940 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:41.929+0000 7f5891e00640 1 -- 172.21.15.12:0/2996561543 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56261bf3ece0 msgr2=0x56261bf44280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:29:41.940 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:41.929+0000 7f5891e00640 1 --2- 172.21.15.12:0/2996561543 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56261bf3ece0 0x56261bf44280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:29:41.940 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:41.929+0000 7f5891e00640 1 -- 172.21.15.12:0/2996561543 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56261bbfd790 con 0x56261bf39970 2024-09-11T00:29:41.940 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:41.929+0000 7f5892800640 1 --2- 172.21.15.12:0/2996561543 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56261bf3ece0 0x56261bf44280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:29:41.940 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:41.934+0000 7f5891e00640 1 --2- 172.21.15.12:0/2996561543 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56261bf39970 0x56261bf43010 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f5884000c00 tx=0x7f58840042f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=88eb5518009438ed server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:29:41.940 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:41.934+0000 7f5877e00640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-09-11T00:29:41.940 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:41.934+0000 7f5877e00640 1 -- 172.21.15.12:0/2996561543 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5884052060 con 0x56261bf39970 2024-09-11T00:29:41.940 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:41.934+0000 7f589c0b9740 1 -- 172.21.15.12:0/2996561543 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5884040070 con 0x56261bf39970 2024-09-11T00:29:41.940 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:41.934+0000 7f5877e00640 1 -- 172.21.15.12:0/2996561543 <== mon.2 v2:172.21.15.165:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 9208+0+0 (secure 0 0 0) 0x7f58840036f0 con 0x56261bf39970 2024-09-11T00:29:41.940 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:41.934+0000 7f5877e00640 1 -- 172.21.15.12:0/2996561543 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=101) ==== 24+0+0 (secure 0 0 0) 0x7f5884059050 con 0x56261bf39970 2024-09-11T00:29:41.940 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:41.934+0000 7f589c0b9740 10 client.27929 fetch_fsmap learned FSMap version 101 2024-09-11T00:29:41.941 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:41.934+0000 7f589c0b9740 1 -- 172.21.15.12:0/2996561543 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=101,monmap=4+}) -- 0x56261bc76620 con 0x56261bf39970 2024-09-11T00:29:41.941 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:41.934+0000 7f5877e00640 1 -- 172.21.15.12:0/2996561543 <== mon.2 v2:172.21.15.165:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5884040070 con 0x56261bf39970 2024-09-11T00:29:41.941 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:41.934+0000 7f5877e00640 1 -- 172.21.15.12:0/2996561543 <== mon.2 v2:172.21.15.165:3300/0 5 ==== fsmap(e 101) ==== 8382+0+0 (secure 0 0 0) 0x7f5884045880 con 0x56261bf39970 2024-09-11T00:29:41.941 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:41.934+0000 7f589c0b9740 10 client.27929 fetch_fsmap finished waiting for FSMap version 101 2024-09-11T00:29:41.941 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:41.934+0000 7f589c0b9740 10 client.27929 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T00:29:41.941 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:41.934+0000 7f589c0b9740 20 client.27929 populate_metadata read hostname 'smithi012' 2024-09-11T00:29:41.941 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:41.934+0000 7f589c0b9740 1 --2- 172.21.15.12:0/2996561543 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x56261bf4e4b0 0x56261bf508a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:29:41.941 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:41.934+0000 7f589c0b9740 4 client.27929 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T00:29:41.941 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:41.934+0000 7f589c0b9740 1 -- 172.21.15.12:0/2996561543 --> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56261beae600 con 0x56261bf4e4b0 2024-09-11T00:29:41.941 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:41.934+0000 7f5889400640 1 --2- 172.21.15.12:0/2996561543 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x56261bf4e4b0 0x56261bf508a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:29:41.941 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:41.934+0000 7f5889400640 1 --2- 172.21.15.12:0/2996561543 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x56261bf4e4b0 0x56261bf508a0 crc :-1 s=READY pgs=309 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=631c12050357725a server_cookie=8c455c840a66a595 in_seq=0 out_seq=0 2024-09-11T00:29:41.941 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:41.934+0000 7f5877e00640 10 client.27929 ms_handle_connect on v2:172.21.15.165:6838/501312984 2024-09-11T00:29:41.942 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:41.934+0000 7f5877e00640 1 -- 172.21.15.12:0/2996561543 <== mds.0 v2:172.21.15.165:6838/501312984 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x56261beae600 con 0x56261bf4e4b0 2024-09-11T00:29:41.942 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:41.934+0000 7f5877e00640 10 client.27929 handle_command_reply: tid=0 2024-09-11T00:29:41.943 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:41.934+0000 7f589c0b9740 1 client.27929 shutdown 2024-09-11T00:29:41.943 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:41.934+0000 7f589c0b9740 1 -- 172.21.15.12:0/2996561543 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56261bf39970 msgr2=0x56261bf43010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:29:41.943 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:41.934+0000 7f589c0b9740 1 --2- 172.21.15.12:0/2996561543 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56261bf39970 0x56261bf43010 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f5884000c00 tx=0x7f58840042f0 comp rx=0 tx=0).stop 2024-09-11T00:29:41.943 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:41.934+0000 7f589c0b9740 1 -- 172.21.15.12:0/2996561543 shutdown_connections 2024-09-11T00:29:41.943 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:41.934+0000 7f589c0b9740 1 --2- 172.21.15.12:0/2996561543 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x56261bf4e4b0 0x56261bf508a0 crc :-1 s=READY pgs=309 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:29:41.943 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:41.934+0000 7f589c0b9740 1 --2- 172.21.15.12:0/2996561543 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56261bf3ece0 0x56261bf44280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:29:41.943 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:41.934+0000 7f589c0b9740 1 --2- 172.21.15.12:0/2996561543 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56261bf3c2a0 0x56261bf436b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:29:41.943 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:41.934+0000 7f589c0b9740 1 --2- 172.21.15.12:0/2996561543 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56261bf39970 0x56261bf43010 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:29:41.944 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:41.934+0000 7f5877e00640 0 client.27929 ms_handle_reset on v2:172.21.15.165:6838/501312984 2024-09-11T00:29:41.944 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:41.934+0000 7f589c0b9740 1 -- 172.21.15.12:0/2996561543 >> 172.21.15.12:0/2996561543 conn(0x56261bf34ac0 msgr2=0x56261bf33bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:29:41.944 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:41.934+0000 7f589c0b9740 1 -- 172.21.15.12:0/2996561543 shutdown_connections 2024-09-11T00:29:41.944 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:41.934+0000 7f589c0b9740 1 -- 172.21.15.12:0/2996561543 wait complete. 2024-09-11T00:29:41.944 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:41.934+0000 7f589c0b9740 20 client.27929 trim_cache size 0 max 16384 2024-09-11T00:29:41.969 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:41.964+0000 7f589c0b9740 1 Processor -- start 2024-09-11T00:29:41.969 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:41.964+0000 7f589c0b9740 1 -- start start 2024-09-11T00:29:41.969 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:41.964+0000 7f589c0b9740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56261bf39970 0x56261bf2fb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:29:41.969 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:41.964+0000 7f589c0b9740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56261bf3c2a0 0x56261bf3f0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:29:41.969 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:41.964+0000 7f589c0b9740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56261bf3ece0 0x56261bf3f600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:29:41.969 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:41.964+0000 7f589c0b9740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f58941ad6e0 con 0x56261bf39970 2024-09-11T00:29:41.969 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:41.964+0000 7f589c0b9740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f58941af2b0 con 0x56261bf3ece0 2024-09-11T00:29:41.969 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:41.964+0000 7f589c0b9740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f58941ade10 con 0x56261bf3c2a0 2024-09-11T00:29:41.969 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:41.964+0000 7f5891e00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56261bf39970 0x56261bf2fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:29:41.969 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:41.964+0000 7f5891e00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56261bf39970 0x56261bf2fb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:51598/0 (socket says 172.21.15.12:51598) 2024-09-11T00:29:41.969 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:41.964+0000 7f5891e00640 1 -- 172.21.15.12:0/184846294 learned_addr learned my addr 172.21.15.12:0/184846294 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:29:41.970 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:41.964+0000 7f5892800640 1 --2- 172.21.15.12:0/184846294 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56261bf3ece0 0x56261bf3f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:29:41.970 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:41.964+0000 7f5889400640 1 --2- 172.21.15.12:0/184846294 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56261bf3c2a0 0x56261bf3f0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:29:41.970 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:41.964+0000 7f5891e00640 1 -- 172.21.15.12:0/184846294 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56261bf3c2a0 msgr2=0x56261bf3f0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:29:41.970 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:41.964+0000 7f5891e00640 1 --2- 172.21.15.12:0/184846294 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56261bf3c2a0 0x56261bf3f0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:29:41.970 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:41.964+0000 7f5891e00640 1 -- 172.21.15.12:0/184846294 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56261bf3ece0 msgr2=0x56261bf3f600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:29:41.970 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:41.964+0000 7f5891e00640 1 --2- 172.21.15.12:0/184846294 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56261bf3ece0 0x56261bf3f600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:29:41.970 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:41.964+0000 7f5891e00640 1 -- 172.21.15.12:0/184846294 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56261bf43460 con 0x56261bf39970 2024-09-11T00:29:41.970 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:41.964+0000 7f5889400640 1 --2- 172.21.15.12:0/184846294 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56261bf3c2a0 0x56261bf3f0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T00:29:41.970 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:41.964+0000 7f5891e00640 1 --2- 172.21.15.12:0/184846294 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56261bf39970 0x56261bf2fb80 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f5884045c80 tx=0x7f588402ab90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bcf3b82355b033c5 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:29:41.970 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:41.964+0000 7f5882a00640 1 -- 172.21.15.12:0/184846294 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5884052040 con 0x56261bf39970 2024-09-11T00:29:41.970 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:41.964+0000 7f5882a00640 1 -- 172.21.15.12:0/184846294 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5884040070 con 0x56261bf39970 2024-09-11T00:29:41.970 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:41.964+0000 7f5882a00640 1 -- 172.21.15.12:0/184846294 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5884035050 con 0x56261bf39970 2024-09-11T00:29:41.974 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:41.964+0000 7f589c0b9740 1 -- 172.21.15.12:0/184846294 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56261bf39970 msgr2=0x56261bf2fb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:29:41.974 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:41.964+0000 7f589c0b9740 1 --2- 172.21.15.12:0/184846294 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56261bf39970 0x56261bf2fb80 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f5884045c80 tx=0x7f588402ab90 comp rx=0 tx=0).stop 2024-09-11T00:29:41.974 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:41.964+0000 7f589c0b9740 1 -- 172.21.15.12:0/184846294 shutdown_connections 2024-09-11T00:29:41.974 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:41.964+0000 7f589c0b9740 1 --2- 172.21.15.12:0/184846294 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56261bf3ece0 0x56261bf3f600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:29:41.974 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:41.964+0000 7f589c0b9740 1 --2- 172.21.15.12:0/184846294 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56261bf3c2a0 0x56261bf3f0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:29:41.974 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:41.964+0000 7f589c0b9740 1 --2- 172.21.15.12:0/184846294 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56261bf39970 0x56261bf2fb80 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:29:41.974 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:41.964+0000 7f589c0b9740 1 -- 172.21.15.12:0/184846294 >> 172.21.15.12:0/184846294 conn(0x56261bf34ac0 msgr2=0x56261bf38dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:29:41.974 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:41.964+0000 7f589c0b9740 1 -- 172.21.15.12:0/184846294 shutdown_connections 2024-09-11T00:29:41.974 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:41.964+0000 7f589c0b9740 1 -- 172.21.15.12:0/184846294 wait complete. 2024-09-11T00:29:41.974 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:41.969+0000 7f589c0b9740 1 -- start start 2024-09-11T00:29:41.974 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:41.969+0000 7f589c0b9740 1 Processor -- start 2024-09-11T00:29:41.974 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:41.969+0000 7f589c0b9740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56261bf39970 0x56261bf4ac70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:29:41.975 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:41.969+0000 7f589c0b9740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56261bf3c2a0 0x56261bf49970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:29:41.975 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:41.969+0000 7f589c0b9740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56261bf3ece0 0x56261bf4a040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:29:41.975 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:41.969+0000 7f589c0b9740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f58941aebe0 con 0x56261bf39970 2024-09-11T00:29:41.975 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:41.969+0000 7f589c0b9740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f58941ae510 con 0x56261bf3ece0 2024-09-11T00:29:41.975 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:41.969+0000 7f589c0b9740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f58941ab6c0 con 0x56261bf3c2a0 2024-09-11T00:29:41.975 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:41.969+0000 7f589c0b9740 10 client.0 _ll_register_callbacks cb 0x7f587c0097a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:29:41.975 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:41.969+0000 7f5891e00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56261bf39970 0x56261bf4ac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:29:41.975 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:41.969+0000 7f5891e00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56261bf39970 0x56261bf4ac70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:51600/0 (socket says 172.21.15.12:51600) 2024-09-11T00:29:41.975 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:41.969+0000 7f5891e00640 1 -- 172.21.15.12:0/1576209928 learned_addr learned my addr 172.21.15.12:0/1576209928 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:29:41.975 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:41.969+0000 7f5892800640 1 --2- 172.21.15.12:0/1576209928 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56261bf3ece0 0x56261bf4a040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:29:41.975 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:41.969+0000 7f5889400640 1 --2- 172.21.15.12:0/1576209928 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56261bf3c2a0 0x56261bf49970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:29:41.975 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:41.970+0000 7f5891e00640 1 -- 172.21.15.12:0/1576209928 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56261bf3c2a0 msgr2=0x56261bf49970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:29:41.975 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:41.970+0000 7f5891e00640 1 --2- 172.21.15.12:0/1576209928 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56261bf3c2a0 0x56261bf49970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:29:41.976 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:41.970+0000 7f5891e00640 1 -- 172.21.15.12:0/1576209928 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56261bf3ece0 msgr2=0x56261bf4a040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:29:41.976 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:41.970+0000 7f5891e00640 1 --2- 172.21.15.12:0/1576209928 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56261bf3ece0 0x56261bf4a040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:29:41.976 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:41.970+0000 7f5891e00640 1 -- 172.21.15.12:0/1576209928 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56261bf35d20 con 0x56261bf39970 2024-09-11T00:29:41.976 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:41.970+0000 7f5889400640 1 --2- 172.21.15.12:0/1576209928 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56261bf3c2a0 0x56261bf49970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T00:29:41.976 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:41.970+0000 7f5891e00640 1 --2- 172.21.15.12:0/1576209928 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56261bf39970 0x56261bf4ac70 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f5884045d70 tx=0x7f58840319e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7585f08482cd1498 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:29:41.976 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:41.970+0000 7f5877e00640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-09-11T00:29:41.976 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:41.970+0000 7f5877e00640 1 -- 172.21.15.12:0/1576209928 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5884035070 con 0x56261bf39970 2024-09-11T00:29:41.976 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:41.970+0000 7f589c0b9740 1 -- 172.21.15.12:0/1576209928 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5884040070 con 0x56261bf39970 2024-09-11T00:29:41.976 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:41.970+0000 7f5877e00640 1 -- 172.21.15.12:0/1576209928 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 9208+0+0 (secure 0 0 0) 0x7f5884045670 con 0x56261bf39970 2024-09-11T00:29:41.976 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:41.970+0000 7f5877e00640 1 -- 172.21.15.12:0/1576209928 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=101) ==== 24+0+0 (secure 0 0 0) 0x7f588406f050 con 0x56261bf39970 2024-09-11T00:29:41.976 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:41.970+0000 7f589c0b9740 10 client.17919 fetch_fsmap learned FSMap version 101 2024-09-11T00:29:41.976 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:41.970+0000 7f589c0b9740 1 -- 172.21.15.12:0/1576209928 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=101,monmap=4+}) -- 0x56261bfbb3b0 con 0x56261bf39970 2024-09-11T00:29:41.976 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:41.970+0000 7f5877e00640 1 -- 172.21.15.12:0/1576209928 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5884040070 con 0x56261bf39970 2024-09-11T00:29:41.977 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:41.970+0000 7f5877e00640 1 -- 172.21.15.12:0/1576209928 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 101) ==== 8382+0+0 (secure 0 0 0) 0x7f5884065030 con 0x56261bf39970 2024-09-11T00:29:41.977 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:41.970+0000 7f589c0b9740 10 client.17919 fetch_fsmap finished waiting for FSMap version 101 2024-09-11T00:29:41.977 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:41.970+0000 7f589c0b9740 10 client.17919 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T00:29:41.977 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:41.970+0000 7f589c0b9740 20 client.17919 populate_metadata read hostname 'smithi012' 2024-09-11T00:29:41.977 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:41.970+0000 7f589c0b9740 1 --2- 172.21.15.12:0/1576209928 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x56261bfbbcd0 0x56261bfbe0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:29:41.977 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:41.970+0000 7f589c0b9740 4 client.17919 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T00:29:41.977 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:41.970+0000 7f589c0b9740 1 -- 172.21.15.12:0/1576209928 --> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56261bf30260 con 0x56261bfbbcd0 2024-09-11T00:29:41.977 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:41.970+0000 7f5889400640 1 --2- 172.21.15.12:0/1576209928 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x56261bfbbcd0 0x56261bfbe0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:29:41.977 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:41.970+0000 7f5889400640 1 --2- 172.21.15.12:0/1576209928 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x56261bfbbcd0 0x56261bfbe0f0 crc :-1 s=READY pgs=310 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1d2817d871cb0da4 server_cookie=dfd046d046749fd6 in_seq=0 out_seq=0 2024-09-11T00:29:41.977 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:41.970+0000 7f5877e00640 10 client.17919 ms_handle_connect on v2:172.21.15.165:6838/501312984 2024-09-11T00:29:42.206 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:42.204+0000 7f5877e00640 1 -- 172.21.15.12:0/1576209928 <== mds.0 v2:172.21.15.165:6838/501312984 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56261bf30260 con 0x56261bfbbcd0 2024-09-11T00:29:42.206 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:42.204+0000 7f5877e00640 10 client.17919 handle_command_reply: tid=0 2024-09-11T00:29:42.206 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:42.204+0000 7f589c0b9740 1 client.17919 shutdown 2024-09-11T00:29:42.207 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:42.204+0000 7f589c0b9740 1 -- 172.21.15.12:0/1576209928 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56261bf39970 msgr2=0x56261bf4ac70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:29:42.207 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:42.204+0000 7f589c0b9740 1 --2- 172.21.15.12:0/1576209928 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56261bf39970 0x56261bf4ac70 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f5884045d70 tx=0x7f58840319e0 comp rx=0 tx=0).stop 2024-09-11T00:29:42.207 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:42.204+0000 7f589c0b9740 1 -- 172.21.15.12:0/1576209928 shutdown_connections 2024-09-11T00:29:42.207 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:42.204+0000 7f589c0b9740 1 --2- 172.21.15.12:0/1576209928 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x56261bfbbcd0 0x56261bfbe0f0 crc :-1 s=READY pgs=310 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:29:42.207 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:42.204+0000 7f589c0b9740 1 --2- 172.21.15.12:0/1576209928 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56261bf3ece0 0x56261bf4a040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:29:42.207 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:42.204+0000 7f589c0b9740 1 --2- 172.21.15.12:0/1576209928 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56261bf3c2a0 0x56261bf49970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:29:42.207 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:42.204+0000 7f589c0b9740 1 --2- 172.21.15.12:0/1576209928 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56261bf39970 0x56261bf4ac70 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:29:42.207 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:42.204+0000 7f5877e00640 0 client.17919 ms_handle_reset on v2:172.21.15.165:6838/501312984 2024-09-11T00:29:42.207 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:42.204+0000 7f589c0b9740 1 -- 172.21.15.12:0/1576209928 >> 172.21.15.12:0/1576209928 conn(0x56261bf34ac0 msgr2=0x56261bf38d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:29:42.207 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:42.204+0000 7f589c0b9740 1 -- 172.21.15.12:0/1576209928 shutdown_connections 2024-09-11T00:29:42.207 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:42.204+0000 7f589c0b9740 1 -- 172.21.15.12:0/1576209928 wait complete. 2024-09-11T00:29:42.207 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:42.204+0000 7f589c0b9740 20 client.17919 trim_cache size 0 max 16384 2024-09-11T00:29:42.208 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:{ 2024-09-11T00:29:42.208 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "message": "", 2024-09-11T00:29:42.208 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "return_code": 0 2024-09-11T00:29:42.208 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:} 2024-09-11T00:29:42.209 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:42.204+0000 7f5893200640 1 -- 172.21.15.12:0/2624477162 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f58780b5360 msgr2=0x7f58780b7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:29:42.209 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:42.204+0000 7f5893200640 1 --2- 172.21.15.12:0/2624477162 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f58780b5360 0x7f58780b7820 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f587c009fd0 tx=0x7f587c005850 comp rx=0 tx=0).stop 2024-09-11T00:29:42.209 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:42.204+0000 7f5893200640 1 -- 172.21.15.12:0/2624477162 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f58941084a0 msgr2=0x7f5894101840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:29:42.209 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:42.204+0000 7f5893200640 1 --2- 172.21.15.12:0/2624477162 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f58941084a0 0x7f5894101840 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f588c00eaa0 tx=0x7f588c00ef60 comp rx=0 tx=0).stop 2024-09-11T00:29:42.209 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:42.204+0000 7f5893200640 1 -- 172.21.15.12:0/2624477162 shutdown_connections 2024-09-11T00:29:42.209 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:42.204+0000 7f5893200640 1 --2- 172.21.15.12:0/2624477162 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f58780b5360 0x7f58780b7820 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:29:42.209 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:42.204+0000 7f5893200640 1 --2- 172.21.15.12:0/2624477162 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f58941084a0 0x7f5894101840 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:29:42.209 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:42.204+0000 7f5893200640 1 --2- 172.21.15.12:0/2624477162 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f58941058f0 0x7f5894101300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:29:42.209 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:42.204+0000 7f5893200640 1 --2- 172.21.15.12:0/2624477162 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f589406b470 0x7f5894102aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:29:42.209 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:42.204+0000 7f5893200640 1 -- 172.21.15.12:0/2624477162 >> 172.21.15.12:0/2624477162 conn(0x7f58940fcf30 msgr2=0x7f5894103b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:29:42.210 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:42.204+0000 7f5893200640 1 -- 172.21.15.12:0/2624477162 shutdown_connections 2024-09-11T00:29:42.210 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:42.204+0000 7f5893200640 1 -- 172.21.15.12:0/2624477162 wait complete. 2024-09-11T00:29:42.271 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:29:42 smithi012 ceph-mon[30333]: pgmap v1078: 129 pgs: 129 active+clean; 848 MiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 13 MiB/s wr, 765 op/s 2024-09-11T00:29:42.275 INFO:tasks.workunit.client.0.smithi012.stdout: 10 164408 15.46 MB/sec execute 458 sec latency 8309.614 ms 2024-09-11T00:29:42.532 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:29:42 smithi165 ceph-mon[35356]: pgmap v1078: 129 pgs: 129 active+clean; 848 MiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 13 MiB/s wr, 765 op/s 2024-09-11T00:29:42.581 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:29:42 smithi178 ceph-mon[35919]: pgmap v1078: 129 pgs: 129 active+clean; 848 MiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 13 MiB/s wr, 765 op/s 2024-09-11T00:29:43.276 INFO:tasks.workunit.client.0.smithi012.stdout: 10 164408 15.42 MB/sec execute 459 sec latency 9309.677 ms 2024-09-11T00:29:44.242 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:29:44 smithi012 ceph-mon[30333]: pgmap v1079: 129 pgs: 129 active+clean; 819 MiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 13 MiB/s wr, 705 op/s 2024-09-11T00:29:44.276 INFO:tasks.workunit.client.0.smithi012.stdout: 10 164408 15.39 MB/sec execute 460 sec latency 10309.742 ms 2024-09-11T00:29:44.532 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:29:44 smithi165 ceph-mon[35356]: pgmap v1079: 129 pgs: 129 active+clean; 819 MiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 13 MiB/s wr, 705 op/s 2024-09-11T00:29:44.581 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:29:44 smithi178 ceph-mon[35919]: pgmap v1079: 129 pgs: 129 active+clean; 819 MiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 13 MiB/s wr, 705 op/s 2024-09-11T00:29:45.276 INFO:tasks.workunit.client.0.smithi012.stdout: 10 164409 15.36 MB/sec execute 461 sec latency 11236.343 ms 2024-09-11T00:29:46.226 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:29:46 smithi012 ceph-mon[30333]: pgmap v1080: 129 pgs: 129 active+clean; 818 MiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 12 MiB/s wr, 492 op/s 2024-09-11T00:29:46.276 INFO:tasks.workunit.client.0.smithi012.stdout: 10 164409 15.32 MB/sec execute 462 sec latency 12236.411 ms 2024-09-11T00:29:46.532 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:29:46 smithi165 ceph-mon[35356]: pgmap v1080: 129 pgs: 129 active+clean; 818 MiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 12 MiB/s wr, 492 op/s 2024-09-11T00:29:46.581 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:29:46 smithi178 ceph-mon[35919]: pgmap v1080: 129 pgs: 129 active+clean; 818 MiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 12 MiB/s wr, 492 op/s 2024-09-11T00:29:47.276 INFO:tasks.workunit.client.0.smithi012.stdout: 10 164409 15.29 MB/sec execute 463 sec latency 13236.456 ms 2024-09-11T00:29:47.324 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:29:47 smithi012 ceph-mon[30333]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T00:29:47.532 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:29:47 smithi165 ceph-mon[35356]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T00:29:47.581 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:29:47 smithi178 ceph-mon[35919]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T00:29:48.276 INFO:tasks.workunit.client.0.smithi012.stdout: 10 164409 15.26 MB/sec execute 464 sec latency 14236.501 ms 2024-09-11T00:29:48.420 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:29:48 smithi012 ceph-mon[30333]: pgmap v1081: 129 pgs: 129 active+clean; 810 MiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 12 MiB/s wr, 538 op/s 2024-09-11T00:29:48.532 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:29:48 smithi165 ceph-mon[35356]: pgmap v1081: 129 pgs: 129 active+clean; 810 MiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 12 MiB/s wr, 538 op/s 2024-09-11T00:29:48.581 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:29:48 smithi178 ceph-mon[35919]: pgmap v1081: 129 pgs: 129 active+clean; 810 MiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 12 MiB/s wr, 538 op/s 2024-09-11T00:29:49.276 INFO:tasks.workunit.client.0.smithi012.stdout: 10 164412 15.22 MB/sec execute 465 sec latency 15236.544 ms 2024-09-11T00:29:50.277 INFO:tasks.workunit.client.0.smithi012.stdout: 10 164412 15.19 MB/sec execute 466 sec latency 16236.605 ms 2024-09-11T00:29:50.408 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:29:50 smithi012 ceph-mon[30333]: pgmap v1082: 129 pgs: 129 active+clean; 810 MiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 544 KiB/s rd, 3.9 MiB/s wr, 279 op/s 2024-09-11T00:29:50.533 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:29:50 smithi165 ceph-mon[35356]: pgmap v1082: 129 pgs: 129 active+clean; 810 MiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 544 KiB/s rd, 3.9 MiB/s wr, 279 op/s 2024-09-11T00:29:50.581 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:29:50 smithi178 ceph-mon[35919]: pgmap v1082: 129 pgs: 129 active+clean; 810 MiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 544 KiB/s rd, 3.9 MiB/s wr, 279 op/s 2024-09-11T00:29:51.276 INFO:tasks.workunit.client.0.smithi012.stdout: 10 164412 15.16 MB/sec execute 467 sec latency 17236.648 ms 2024-09-11T00:29:52.276 INFO:tasks.workunit.client.0.smithi012.stdout: 10 164412 15.13 MB/sec execute 468 sec latency 18236.710 ms 2024-09-11T00:29:52.351 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:29:52 smithi012 ceph-mon[30333]: pgmap v1083: 129 pgs: 129 active+clean; 792 MiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 355 KiB/s rd, 2.8 MiB/s wr, 280 op/s 2024-09-11T00:29:52.532 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:29:52 smithi165 ceph-mon[35356]: pgmap v1083: 129 pgs: 129 active+clean; 792 MiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 355 KiB/s rd, 2.8 MiB/s wr, 280 op/s 2024-09-11T00:29:52.581 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:29:52 smithi178 ceph-mon[35919]: pgmap v1083: 129 pgs: 129 active+clean; 792 MiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 355 KiB/s rd, 2.8 MiB/s wr, 280 op/s 2024-09-11T00:29:53.276 INFO:tasks.workunit.client.0.smithi012.stdout: 10 164412 15.09 MB/sec execute 469 sec latency 19236.777 ms 2024-09-11T00:29:53.794 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:29:53 smithi012 ceph-mon[30333]: pgmap v1084: 129 pgs: 129 active+clean; 777 MiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 273 KiB/s rd, 602 KiB/s wr, 198 op/s 2024-09-11T00:29:54.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:29:53 smithi165 ceph-mon[35356]: pgmap v1084: 129 pgs: 129 active+clean; 777 MiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 273 KiB/s rd, 602 KiB/s wr, 198 op/s 2024-09-11T00:29:54.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:29:53 smithi178 ceph-mon[35919]: pgmap v1084: 129 pgs: 129 active+clean; 777 MiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 273 KiB/s rd, 602 KiB/s wr, 198 op/s 2024-09-11T00:29:54.276 INFO:tasks.workunit.client.0.smithi012.stdout: 10 164412 15.06 MB/sec execute 470 sec latency 20236.839 ms 2024-09-11T00:29:55.276 INFO:tasks.workunit.client.0.smithi012.stdout: 10 164412 15.03 MB/sec execute 471 sec latency 21236.901 ms 2024-09-11T00:29:55.328 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:55.326+0000 7f9ffd400640 1 -- 172.21.15.12:0/572932739 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9ff8104ba0 msgr2=0x7f9ff8104fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:29:55.328 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:55.326+0000 7f9ffd400640 1 --2- 172.21.15.12:0/572932739 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9ff8104ba0 0x7f9ff8104fa0 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f9fe8009930 tx=0x7f9fe802a440 comp rx=0 tx=0).stop 2024-09-11T00:29:55.329 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:55.326+0000 7f9ffd400640 1 -- 172.21.15.12:0/572932739 shutdown_connections 2024-09-11T00:29:55.329 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:55.326+0000 7f9ffd400640 1 --2- 172.21.15.12:0/572932739 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9ff81066b0 0x7f9ff810d030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:29:55.329 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:55.326+0000 7f9ffd400640 1 --2- 172.21.15.12:0/572932739 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9ff8105cf0 0x7f9ff8106170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:29:55.329 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:55.326+0000 7f9ffd400640 1 --2- 172.21.15.12:0/572932739 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9ff8104ba0 0x7f9ff8104fa0 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:29:55.329 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:55.326+0000 7f9ffd400640 1 -- 172.21.15.12:0/572932739 >> 172.21.15.12:0/572932739 conn(0x7f9ff8100460 msgr2=0x7f9ff8102880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:29:55.329 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:55.327+0000 7f9ffd400640 1 -- 172.21.15.12:0/572932739 shutdown_connections 2024-09-11T00:29:55.329 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:55.327+0000 7f9ffd400640 1 -- 172.21.15.12:0/572932739 wait complete. 2024-09-11T00:29:55.331 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:55.327+0000 7f9ffd400640 1 Processor -- start 2024-09-11T00:29:55.331 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:55.327+0000 7f9ffd400640 1 -- start start 2024-09-11T00:29:55.331 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:55.327+0000 7f9ffd400640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9ff8104ba0 0x7f9ff819af90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:29:55.331 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:55.327+0000 7f9ffd400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9ff8105cf0 0x7f9ff819b5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:29:55.331 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:55.327+0000 7f9ffd400640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9ff81066b0 0x7f9ff819c210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:29:55.331 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:55.327+0000 7f9ffd400640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f9ff8110810 con 0x7f9ff81066b0 2024-09-11T00:29:55.331 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:55.327+0000 7f9ffd400640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f9ff8110690 con 0x7f9ff8105cf0 2024-09-11T00:29:55.331 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:55.327+0000 7f9ffd400640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f9ff8110990 con 0x7f9ff8104ba0 2024-09-11T00:29:55.331 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:55.327+0000 7f9ff7200640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9ff81066b0 0x7f9ff819c210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:29:55.332 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:55.327+0000 7f9ff6800640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9ff8104ba0 0x7f9ff819af90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:29:55.332 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:55.327+0000 7f9ff6800640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9ff8104ba0 0x7f9ff819af90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.12:33796/0 (socket says 172.21.15.12:33796) 2024-09-11T00:29:55.332 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:55.327+0000 7f9ff6800640 1 -- 172.21.15.12:0/2774816578 learned_addr learned my addr 172.21.15.12:0/2774816578 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:29:55.332 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:55.327+0000 7f9ff7200640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9ff81066b0 0x7f9ff819c210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:47654/0 (socket says 172.21.15.12:47654) 2024-09-11T00:29:55.332 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:55.327+0000 7f9ff7200640 1 -- 172.21.15.12:0/2774816578 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9ff8104ba0 msgr2=0x7f9ff819af90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:29:55.332 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:55.327+0000 7f9ff7200640 1 --2- 172.21.15.12:0/2774816578 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9ff8104ba0 0x7f9ff819af90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:29:55.332 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:55.327+0000 7f9ff7200640 1 -- 172.21.15.12:0/2774816578 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9ff8105cf0 msgr2=0x7f9ff819b5e0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-11T00:29:55.332 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:55.327+0000 7f9ff7200640 1 --2- 172.21.15.12:0/2774816578 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9ff8105cf0 0x7f9ff819b5e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:29:55.332 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:55.327+0000 7f9ff7200640 1 -- 172.21.15.12:0/2774816578 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ff819cad0 con 0x7f9ff81066b0 2024-09-11T00:29:55.332 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:55.327+0000 7f9ff6800640 1 --2- 172.21.15.12:0/2774816578 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9ff8104ba0 0x7f9ff819af90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:29:55.332 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:55.327+0000 7f9ff7200640 1 --2- 172.21.15.12:0/2774816578 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9ff81066b0 0x7f9ff819c210 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f9ff000e940 tx=0x7f9ff000ee00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:29:55.332 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:55.327+0000 7f9fef400640 1 -- 172.21.15.12:0/2774816578 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ff0004d20 con 0x7f9ff81066b0 2024-09-11T00:29:55.332 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:55.327+0000 7f9fef400640 1 -- 172.21.15.12:0/2774816578 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9ff0005020 con 0x7f9ff81066b0 2024-09-11T00:29:55.333 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:55.327+0000 7f9fef400640 1 -- 172.21.15.12:0/2774816578 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ff00116d0 con 0x7f9ff81066b0 2024-09-11T00:29:55.333 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:55.327+0000 7f9ffd400640 1 -- 172.21.15.12:0/2774816578 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ff819cdc0 con 0x7f9ff81066b0 2024-09-11T00:29:55.333 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:55.327+0000 7f9ffd400640 1 -- 172.21.15.12:0/2774816578 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ff81a3490 con 0x7f9ff81066b0 2024-09-11T00:29:55.334 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:55.333+0000 7fa000418740 1 Processor -- start 2024-09-11T00:29:55.334 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:55.333+0000 7f9fef400640 1 -- 172.21.15.12:0/2774816578 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7f9ff000f040 con 0x7f9ff81066b0 2024-09-11T00:29:55.334 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:55.333+0000 7fa000418740 1 -- start start 2024-09-11T00:29:55.335 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:55.333+0000 7fa000418740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5637fed22790 0x5637fed24b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:29:55.336 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:55.333+0000 7fa000418740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5637fed250c0 0x5637fed274b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:29:55.336 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:55.333+0000 7fa000418740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5637fed27b00 0x5637fed29fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:29:55.336 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:55.333+0000 7fa000418740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x5637fec956b0 con 0x5637fed27b00 2024-09-11T00:29:55.336 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:55.333+0000 7fa000418740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x5637fec84ae0 con 0x5637fed22790 2024-09-11T00:29:55.336 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:55.333+0000 7fa000418740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5637fec888f0 con 0x5637fed250c0 2024-09-11T00:29:55.336 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:55.333+0000 7f9fefe00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5637fed27b00 0x5637fed29fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:29:55.336 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:55.333+0000 7f9fefe00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5637fed27b00 0x5637fed29fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:47660/0 (socket says 172.21.15.12:47660) 2024-09-11T00:29:55.336 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:55.333+0000 7f9ff6800640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5637fed250c0 0x5637fed274b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:29:55.336 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:55.333+0000 7f9fefe00640 1 -- 172.21.15.12:0/1845312317 learned_addr learned my addr 172.21.15.12:0/1845312317 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:29:55.336 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:55.333+0000 7f9fefe00640 1 -- 172.21.15.12:0/1845312317 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5637fed250c0 msgr2=0x5637fed274b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:29:55.336 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:55.333+0000 7f9fefe00640 1 --2- 172.21.15.12:0/1845312317 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5637fed250c0 0x5637fed274b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:29:55.336 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:55.333+0000 7f9fefe00640 1 -- 172.21.15.12:0/1845312317 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5637fed22790 msgr2=0x5637fed24b80 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-11T00:29:55.337 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:55.333+0000 7f9fefe00640 1 --2- 172.21.15.12:0/1845312317 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5637fed22790 0x5637fed24b80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:29:55.337 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:55.333+0000 7f9fefe00640 1 -- 172.21.15.12:0/1845312317 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5637fec51780 con 0x5637fed27b00 2024-09-11T00:29:55.337 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:55.333+0000 7f9ff6800640 1 --2- 172.21.15.12:0/1845312317 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5637fed250c0 0x5637fed274b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:29:55.339 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:55.333+0000 7f9fefe00640 1 --2- 172.21.15.12:0/1845312317 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5637fed27b00 0x5637fed29fc0 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f9fe000ba50 tx=0x7f9fe000bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16b6ffb035bd9379 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:29:55.339 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:55.333+0000 7f9fe7400640 1 -- 172.21.15.12:0/1845312317 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fe0016070 con 0x5637fed27b00 2024-09-11T00:29:55.339 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:55.333+0000 7f9fe7400640 1 -- 172.21.15.12:0/1845312317 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9fe00054d0 con 0x5637fed27b00 2024-09-11T00:29:55.339 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:55.333+0000 7f9fe7400640 1 -- 172.21.15.12:0/1845312317 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fe0004de0 con 0x5637fed27b00 2024-09-11T00:29:55.339 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:55.333+0000 7f9fef400640 1 --2- 172.21.15.12:0/2774816578 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f9fdc0b5360 0x7f9fdc0b7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:29:55.339 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:55.333+0000 7f9fef400640 1 -- 172.21.15.12:0/2774816578 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9208+0+0 (secure 0 0 0) 0x7f9ff00e6080 con 0x7f9ff81066b0 2024-09-11T00:29:55.340 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:55.333+0000 7f9ff6800640 1 --2- 172.21.15.12:0/2774816578 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f9fdc0b5360 0x7f9fdc0b7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:29:55.340 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:55.333+0000 7fa000418740 1 -- 172.21.15.12:0/1845312317 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5637fed27b00 msgr2=0x5637fed29fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:29:55.340 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:55.333+0000 7fa000418740 1 --2- 172.21.15.12:0/1845312317 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5637fed27b00 0x5637fed29fc0 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f9fe000ba50 tx=0x7f9fe000bf10 comp rx=0 tx=0).stop 2024-09-11T00:29:55.340 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:55.333+0000 7fa000418740 1 -- 172.21.15.12:0/1845312317 shutdown_connections 2024-09-11T00:29:55.340 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:55.333+0000 7fa000418740 1 --2- 172.21.15.12:0/1845312317 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5637fed27b00 0x5637fed29fc0 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:29:55.340 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:55.333+0000 7fa000418740 1 --2- 172.21.15.12:0/1845312317 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5637fed250c0 0x5637fed274b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:29:55.340 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:55.333+0000 7fa000418740 1 --2- 172.21.15.12:0/1845312317 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5637fed22790 0x5637fed24b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:29:55.340 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:55.333+0000 7fa000418740 1 -- 172.21.15.12:0/1845312317 >> 172.21.15.12:0/1845312317 conn(0x5637fed1d8e0 msgr2=0x5637fed1e670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:29:55.340 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:55.333+0000 7fa000418740 1 -- 172.21.15.12:0/1845312317 shutdown_connections 2024-09-11T00:29:55.340 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:55.333+0000 7fa000418740 1 -- 172.21.15.12:0/1845312317 wait complete. 2024-09-11T00:29:55.340 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:55.333+0000 7f9ff6800640 1 --2- 172.21.15.12:0/2774816578 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f9fdc0b5360 0x7f9fdc0b7820 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f9fe8005c40 tx=0x7f9fe80314c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:29:55.340 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:55.333+0000 7fa000418740 1 -- start start 2024-09-11T00:29:55.341 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:55.333+0000 7fa000418740 1 Processor -- start 2024-09-11T00:29:55.341 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:55.333+0000 7fa000418740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5637fed22790 0x5637fed34700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:29:55.341 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:55.333+0000 7fa000418740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5637fed250c0 0x5637fed34da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:29:55.341 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:55.333+0000 7fa000418740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5637fed27b00 0x5637fed35970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:29:55.341 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:55.333+0000 7fa000418740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x5637febbcab0 con 0x5637fed22790 2024-09-11T00:29:55.341 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:55.333+0000 7fa000418740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x5637fec2b570 con 0x5637fed250c0 2024-09-11T00:29:55.341 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:55.333+0000 7fa000418740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5637fed2aae0 con 0x5637fed27b00 2024-09-11T00:29:55.341 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:55.333+0000 7fa000418740 10 client.0 _ll_register_callbacks cb 0x5637fec3a100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:29:55.341 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:55.338+0000 7f9ff6800640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5637fed22790 0x5637fed34700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:29:55.341 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:55.338+0000 7f9ff6800640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5637fed22790 0x5637fed34700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:47666/0 (socket says 172.21.15.12:47666) 2024-09-11T00:29:55.341 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:55.338+0000 7f9ff6800640 1 -- 172.21.15.12:0/3934185214 learned_addr learned my addr 172.21.15.12:0/3934185214 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:29:55.342 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:55.338+0000 7f9fefe00640 1 --2- 172.21.15.12:0/3934185214 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5637fed250c0 0x5637fed34da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:29:55.344 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:55.338+0000 7f9ff6800640 1 -- 172.21.15.12:0/3934185214 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5637fed27b00 msgr2=0x5637fed35970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:29:55.344 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:55.338+0000 7f9ff6800640 1 --2- 172.21.15.12:0/3934185214 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5637fed27b00 0x5637fed35970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:29:55.344 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:55.338+0000 7f9ff6800640 1 -- 172.21.15.12:0/3934185214 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5637fed250c0 msgr2=0x5637fed34da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:29:55.345 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:55.338+0000 7f9ff6800640 1 --2- 172.21.15.12:0/3934185214 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5637fed250c0 0x5637fed34da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:29:55.345 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:55.338+0000 7f9ff6800640 1 -- 172.21.15.12:0/3934185214 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5637fe9e6790 con 0x5637fed22790 2024-09-11T00:29:55.348 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:55.338+0000 7f9ff6800640 1 --2- 172.21.15.12:0/3934185214 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5637fed22790 0x5637fed34700 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f9fe8009a50 tx=0x7f9fe8009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b485b02e98007b server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:29:55.348 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:55.338+0000 7f9fdbe00640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-09-11T00:29:55.348 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:55.338+0000 7f9fdbe00640 1 -- 172.21.15.12:0/3934185214 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fe8035040 con 0x5637fed22790 2024-09-11T00:29:55.348 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:55.338+0000 7f9fdbe00640 1 -- 172.21.15.12:0/3934185214 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 9208+0+0 (secure 0 0 0) 0x7f9fe8051470 con 0x5637fed22790 2024-09-11T00:29:55.348 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:55.338+0000 7fa000418740 1 -- 172.21.15.12:0/3934185214 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9fe00054d0 con 0x5637fed22790 2024-09-11T00:29:55.348 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:55.338+0000 7f9fdbe00640 1 -- 172.21.15.12:0/3934185214 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=102) ==== 24+0+0 (secure 0 0 0) 0x7f9fe8051670 con 0x5637fed22790 2024-09-11T00:29:55.349 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:55.338+0000 7fa000418740 10 client.17931 fetch_fsmap learned FSMap version 102 2024-09-11T00:29:55.349 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:55.338+0000 7fa000418740 1 -- 172.21.15.12:0/3934185214 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=102,monmap=4+}) -- 0x5637fea5f620 con 0x5637fed22790 2024-09-11T00:29:55.349 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:55.338+0000 7f9fdbe00640 1 -- 172.21.15.12:0/3934185214 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9fe00054d0 con 0x5637fed22790 2024-09-11T00:29:55.349 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:55.338+0000 7f9fdbe00640 1 -- 172.21.15.12:0/3934185214 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 102) ==== 8378+0+0 (secure 0 0 0) 0x7f9fe8044030 con 0x5637fed22790 2024-09-11T00:29:55.349 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:55.338+0000 7fa000418740 10 client.17931 fetch_fsmap finished waiting for FSMap version 102 2024-09-11T00:29:55.349 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:55.338+0000 7fa000418740 10 client.17931 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T00:29:55.349 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:55.338+0000 7fa000418740 20 client.17931 populate_metadata read hostname 'smithi012' 2024-09-11T00:29:55.349 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:55.338+0000 7fa000418740 1 --2- 172.21.15.12:0/3934185214 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x5637fed40050 0x5637fed424e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:29:55.349 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:55.338+0000 7fa000418740 4 client.17931 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T00:29:55.349 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:55.338+0000 7fa000418740 1 -- 172.21.15.12:0/3934185214 --> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5637fec97600 con 0x5637fed40050 2024-09-11T00:29:55.349 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:55.338+0000 7f9fefe00640 1 --2- 172.21.15.12:0/3934185214 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x5637fed40050 0x5637fed424e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:29:55.349 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:55.338+0000 7f9fdbe00640 10 client.17931 ms_handle_connect on v2:172.21.15.165:6838/501312984 2024-09-11T00:29:55.350 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:55.338+0000 7f9fefe00640 1 --2- 172.21.15.12:0/3934185214 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x5637fed40050 0x5637fed424e0 crc :-1 s=READY pgs=311 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b3d3bb985436093b server_cookie=42df5c1920b2b56b in_seq=0 out_seq=0 2024-09-11T00:29:55.350 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:55.338+0000 7f9fdbe00640 1 -- 172.21.15.12:0/3934185214 <== mds.0 v2:172.21.15.165:6838/501312984 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5637fec97600 con 0x5637fed40050 2024-09-11T00:29:55.350 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:55.338+0000 7f9fdbe00640 10 client.17931 handle_command_reply: tid=0 2024-09-11T00:29:55.350 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:55.338+0000 7fa000418740 1 client.17931 shutdown 2024-09-11T00:29:55.350 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:55.338+0000 7fa000418740 1 -- 172.21.15.12:0/3934185214 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5637fed22790 msgr2=0x5637fed34700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:29:55.350 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:55.338+0000 7fa000418740 1 --2- 172.21.15.12:0/3934185214 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5637fed22790 0x5637fed34700 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f9fe8009a50 tx=0x7f9fe8009260 comp rx=0 tx=0).stop 2024-09-11T00:29:55.350 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:55.338+0000 7fa000418740 1 -- 172.21.15.12:0/3934185214 shutdown_connections 2024-09-11T00:29:55.350 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:55.338+0000 7fa000418740 1 --2- 172.21.15.12:0/3934185214 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x5637fed40050 0x5637fed424e0 crc :-1 s=READY pgs=311 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:29:55.350 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:55.338+0000 7fa000418740 1 --2- 172.21.15.12:0/3934185214 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5637fed27b00 0x5637fed35970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:29:55.350 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:55.338+0000 7fa000418740 1 --2- 172.21.15.12:0/3934185214 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5637fed250c0 0x5637fed34da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:29:55.350 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:55.338+0000 7fa000418740 1 --2- 172.21.15.12:0/3934185214 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5637fed22790 0x5637fed34700 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:29:55.350 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:55.338+0000 7f9fdbe00640 0 client.17931 ms_handle_reset on v2:172.21.15.165:6838/501312984 2024-09-11T00:29:55.350 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:55.338+0000 7fa000418740 1 -- 172.21.15.12:0/3934185214 >> 172.21.15.12:0/3934185214 conn(0x5637fed1d8e0 msgr2=0x5637fed1ca60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:29:55.350 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:55.338+0000 7fa000418740 1 -- 172.21.15.12:0/3934185214 shutdown_connections 2024-09-11T00:29:55.351 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:55.338+0000 7fa000418740 1 -- 172.21.15.12:0/3934185214 wait complete. 2024-09-11T00:29:55.351 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:55.338+0000 7fa000418740 20 client.17931 trim_cache size 0 max 16384 2024-09-11T00:29:55.373 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:55.368+0000 7fa000418740 1 Processor -- start 2024-09-11T00:29:55.373 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:55.368+0000 7fa000418740 1 -- start start 2024-09-11T00:29:55.373 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:55.368+0000 7fa000418740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5637fed22790 0x5637fed27f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:29:55.373 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:55.368+0000 7fa000418740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5637fed250c0 0x5637fed28490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:29:55.373 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:55.368+0000 7fa000418740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5637fed27b00 0x5637fed1f710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:29:55.374 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:55.368+0000 7fa000418740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f9ff81a9580 con 0x5637fed250c0 2024-09-11T00:29:55.374 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:55.368+0000 7fa000418740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f9ff81ab120 con 0x5637fed27b00 2024-09-11T00:29:55.374 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:55.368+0000 7fa000418740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f9ff81a9cb0 con 0x5637fed22790 2024-09-11T00:29:55.374 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:55.368+0000 7f9fefe00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5637fed250c0 0x5637fed28490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:29:55.374 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:55.368+0000 7f9fefe00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5637fed250c0 0x5637fed28490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:47672/0 (socket says 172.21.15.12:47672) 2024-09-11T00:29:55.374 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:55.368+0000 7f9fefe00640 1 -- 172.21.15.12:0/872539729 learned_addr learned my addr 172.21.15.12:0/872539729 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:29:55.374 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:55.368+0000 7f9ff7200640 1 --2- 172.21.15.12:0/872539729 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5637fed27b00 0x5637fed1f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:29:55.374 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:55.368+0000 7f9ff6800640 1 --2- 172.21.15.12:0/872539729 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5637fed22790 0x5637fed27f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:29:55.374 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:55.368+0000 7f9fefe00640 1 -- 172.21.15.12:0/872539729 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5637fed22790 msgr2=0x5637fed27f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:29:55.374 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:55.368+0000 7f9fefe00640 1 --2- 172.21.15.12:0/872539729 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5637fed22790 0x5637fed27f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:29:55.374 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:55.368+0000 7f9fefe00640 1 -- 172.21.15.12:0/872539729 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5637fed27b00 msgr2=0x5637fed1f710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:29:55.374 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:55.368+0000 7f9fefe00640 1 --2- 172.21.15.12:0/872539729 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5637fed27b00 0x5637fed1f710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:29:55.374 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:55.368+0000 7f9fefe00640 1 -- 172.21.15.12:0/872539729 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5637fed34200 con 0x5637fed250c0 2024-09-11T00:29:55.375 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:55.368+0000 7f9fefe00640 1 --2- 172.21.15.12:0/872539729 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5637fed250c0 0x5637fed28490 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f9fe000ba20 tx=0x7f9fe0005680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3dbf2eaaae25c111 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:29:55.375 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:55.368+0000 7f9fe6a00640 1 -- 172.21.15.12:0/872539729 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fe0016040 con 0x5637fed250c0 2024-09-11T00:29:55.375 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:55.368+0000 7f9fe6a00640 1 -- 172.21.15.12:0/872539729 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9fe000b6a0 con 0x5637fed250c0 2024-09-11T00:29:55.375 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:55.368+0000 7f9fe6a00640 1 -- 172.21.15.12:0/872539729 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fe0003970 con 0x5637fed250c0 2024-09-11T00:29:55.378 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:55.373+0000 7fa000418740 1 -- 172.21.15.12:0/872539729 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5637fed250c0 msgr2=0x5637fed28490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:29:55.378 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:55.373+0000 7fa000418740 1 --2- 172.21.15.12:0/872539729 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5637fed250c0 0x5637fed28490 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f9fe000ba20 tx=0x7f9fe0005680 comp rx=0 tx=0).stop 2024-09-11T00:29:55.378 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:55.373+0000 7fa000418740 1 -- 172.21.15.12:0/872539729 shutdown_connections 2024-09-11T00:29:55.378 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:55.373+0000 7fa000418740 1 --2- 172.21.15.12:0/872539729 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5637fed27b00 0x5637fed1f710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:29:55.378 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:55.373+0000 7fa000418740 1 --2- 172.21.15.12:0/872539729 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5637fed250c0 0x5637fed28490 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:29:55.378 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:55.373+0000 7fa000418740 1 --2- 172.21.15.12:0/872539729 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5637fed22790 0x5637fed27f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:29:55.378 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:55.373+0000 7fa000418740 1 -- 172.21.15.12:0/872539729 >> 172.21.15.12:0/872539729 conn(0x5637fed1d8e0 msgr2=0x5637fed21c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:29:55.378 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:55.373+0000 7fa000418740 1 -- 172.21.15.12:0/872539729 shutdown_connections 2024-09-11T00:29:55.378 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:55.373+0000 7fa000418740 1 -- 172.21.15.12:0/872539729 wait complete. 2024-09-11T00:29:55.378 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:55.373+0000 7fa000418740 1 -- start start 2024-09-11T00:29:55.378 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:55.373+0000 7fa000418740 1 Processor -- start 2024-09-11T00:29:55.379 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:55.374+0000 7fa000418740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5637fed22790 0x5637fed3b4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:29:55.379 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:55.374+0000 7fa000418740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5637fed250c0 0x5637fed3b9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:29:55.379 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:55.374+0000 7fa000418740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5637fed27b00 0x5637fed3a010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:29:55.379 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:55.374+0000 7fa000418740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f9ff81aaa50 con 0x5637fed22790 2024-09-11T00:29:55.379 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:55.374+0000 7fa000418740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f9ff81aa380 con 0x5637fed250c0 2024-09-11T00:29:55.379 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:55.374+0000 7fa000418740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f9ff81a7560 con 0x5637fed27b00 2024-09-11T00:29:55.379 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:55.374+0000 7fa000418740 10 client.0 _ll_register_callbacks cb 0x7f9fe8032ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:29:55.379 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:55.374+0000 7f9ff6800640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5637fed22790 0x5637fed3b4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:29:55.379 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:55.374+0000 7f9ff6800640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5637fed22790 0x5637fed3b4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:47676/0 (socket says 172.21.15.12:47676) 2024-09-11T00:29:55.379 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:55.374+0000 7f9ff6800640 1 -- 172.21.15.12:0/1176394405 learned_addr learned my addr 172.21.15.12:0/1176394405 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:29:55.379 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:55.374+0000 7f9ff7200640 1 --2- 172.21.15.12:0/1176394405 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5637fed27b00 0x5637fed3a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:29:55.379 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:55.374+0000 7f9fefe00640 1 --2- 172.21.15.12:0/1176394405 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5637fed250c0 0x5637fed3b9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:29:55.379 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:55.374+0000 7f9ff6800640 1 -- 172.21.15.12:0/1176394405 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5637fed27b00 msgr2=0x5637fed3a010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:29:55.379 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:55.374+0000 7f9ff6800640 1 --2- 172.21.15.12:0/1176394405 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5637fed27b00 0x5637fed3a010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:29:55.380 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:55.374+0000 7f9ff6800640 1 -- 172.21.15.12:0/1176394405 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5637fed250c0 msgr2=0x5637fed3b9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:29:55.380 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:55.374+0000 7f9ff6800640 1 --2- 172.21.15.12:0/1176394405 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5637fed250c0 0x5637fed3b9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:29:55.380 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:55.374+0000 7f9ff6800640 1 -- 172.21.15.12:0/1176394405 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5637fed42b60 con 0x5637fed22790 2024-09-11T00:29:55.380 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:55.374+0000 7f9ff6800640 1 --2- 172.21.15.12:0/1176394405 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5637fed22790 0x5637fed3b4a0 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f9fe8000c00 tx=0x7f9fe803d6e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e4e92ca17d585c8 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:29:55.380 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:55.374+0000 7f9fdbe00640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-09-11T00:29:55.380 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:55.374+0000 7f9fdbe00640 1 -- 172.21.15.12:0/1176394405 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fe8035040 con 0x5637fed22790 2024-09-11T00:29:55.380 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:55.374+0000 7f9fdbe00640 1 -- 172.21.15.12:0/1176394405 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 9208+0+0 (secure 0 0 0) 0x7f9fe8057020 con 0x5637fed22790 2024-09-11T00:29:55.380 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:55.374+0000 7fa000418740 1 -- 172.21.15.12:0/1176394405 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9fe000b6a0 con 0x5637fed22790 2024-09-11T00:29:55.380 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:55.374+0000 7f9fdbe00640 1 -- 172.21.15.12:0/1176394405 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=102) ==== 24+0+0 (secure 0 0 0) 0x7f9fe806b050 con 0x5637fed22790 2024-09-11T00:29:55.380 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:55.374+0000 7fa000418740 10 client.17937 fetch_fsmap learned FSMap version 102 2024-09-11T00:29:55.380 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:55.374+0000 7fa000418740 1 -- 172.21.15.12:0/1176394405 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=102,monmap=4+}) -- 0x5637fedad500 con 0x5637fed22790 2024-09-11T00:29:55.380 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:55.374+0000 7f9fdbe00640 1 -- 172.21.15.12:0/1176394405 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9fe000b6a0 con 0x5637fed22790 2024-09-11T00:29:55.380 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:55.374+0000 7f9fdbe00640 1 -- 172.21.15.12:0/1176394405 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 102) ==== 8378+0+0 (secure 0 0 0) 0x7f9fe8044050 con 0x5637fed22790 2024-09-11T00:29:55.381 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:55.374+0000 7fa000418740 10 client.17937 fetch_fsmap finished waiting for FSMap version 102 2024-09-11T00:29:55.381 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:55.374+0000 7fa000418740 10 client.17937 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T00:29:55.381 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:55.374+0000 7fa000418740 20 client.17937 populate_metadata read hostname 'smithi012' 2024-09-11T00:29:55.381 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:55.374+0000 7fa000418740 1 --2- 172.21.15.12:0/1176394405 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x5637fedadf30 0x5637fedb0350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:29:55.381 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:55.374+0000 7fa000418740 4 client.17937 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T00:29:55.381 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:55.374+0000 7fa000418740 1 -- 172.21.15.12:0/1176394405 --> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5637fec930b0 con 0x5637fedadf30 2024-09-11T00:29:55.381 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:55.374+0000 7f9fefe00640 1 --2- 172.21.15.12:0/1176394405 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x5637fedadf30 0x5637fedb0350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:29:55.381 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:55.374+0000 7f9fefe00640 1 --2- 172.21.15.12:0/1176394405 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x5637fedadf30 0x5637fedb0350 crc :-1 s=READY pgs=312 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6e1b4adbcb33f0c8 server_cookie=2b0bb1996d439253 in_seq=0 out_seq=0 2024-09-11T00:29:55.381 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:55.374+0000 7f9fdbe00640 10 client.17937 ms_handle_connect on v2:172.21.15.165:6838/501312984 2024-09-11T00:29:55.445 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:55.443+0000 7f9fdbe00640 1 -- 172.21.15.12:0/1176394405 <== mds.0 v2:172.21.15.165:6838/501312984 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5637fec930b0 con 0x5637fedadf30 2024-09-11T00:29:55.445 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:55.443+0000 7f9fdbe00640 10 client.17937 handle_command_reply: tid=0 2024-09-11T00:29:55.445 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:55.443+0000 7fa000418740 1 client.17937 shutdown 2024-09-11T00:29:55.445 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:55.443+0000 7fa000418740 1 -- 172.21.15.12:0/1176394405 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5637fed22790 msgr2=0x5637fed3b4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:29:55.445 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:55.443+0000 7fa000418740 1 --2- 172.21.15.12:0/1176394405 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5637fed22790 0x5637fed3b4a0 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f9fe8000c00 tx=0x7f9fe803d6e0 comp rx=0 tx=0).stop 2024-09-11T00:29:55.445 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:55.443+0000 7fa000418740 1 -- 172.21.15.12:0/1176394405 shutdown_connections 2024-09-11T00:29:55.445 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:55.443+0000 7fa000418740 1 --2- 172.21.15.12:0/1176394405 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x5637fedadf30 0x5637fedb0350 crc :-1 s=READY pgs=312 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:29:55.445 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:55.443+0000 7fa000418740 1 --2- 172.21.15.12:0/1176394405 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5637fed27b00 0x5637fed3a010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:29:55.445 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:55.443+0000 7fa000418740 1 --2- 172.21.15.12:0/1176394405 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5637fed250c0 0x5637fed3b9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:29:55.446 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:55.443+0000 7fa000418740 1 --2- 172.21.15.12:0/1176394405 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5637fed22790 0x5637fed3b4a0 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:29:55.446 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:55.443+0000 7f9fdbe00640 0 client.17937 ms_handle_reset on v2:172.21.15.165:6838/501312984 2024-09-11T00:29:55.446 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:55.443+0000 7fa000418740 1 -- 172.21.15.12:0/1176394405 >> 172.21.15.12:0/1176394405 conn(0x5637fed1d8e0 msgr2=0x5637fed21810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:29:55.446 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:55.443+0000 7fa000418740 1 -- 172.21.15.12:0/1176394405 shutdown_connections 2024-09-11T00:29:55.446 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:55.443+0000 7fa000418740 1 -- 172.21.15.12:0/1176394405 wait complete. 2024-09-11T00:29:55.446 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:55.443+0000 7fa000418740 20 client.17937 trim_cache size 0 max 16384 2024-09-11T00:29:55.446 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:{ 2024-09-11T00:29:55.446 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "message": "", 2024-09-11T00:29:55.446 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "return_code": 0 2024-09-11T00:29:55.447 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:} 2024-09-11T00:29:55.447 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:55.444+0000 7f9ffd400640 1 -- 172.21.15.12:0/2774816578 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f9fdc0b5360 msgr2=0x7f9fdc0b7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:29:55.447 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:55.444+0000 7f9ffd400640 1 --2- 172.21.15.12:0/2774816578 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f9fdc0b5360 0x7f9fdc0b7820 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f9fe8005c40 tx=0x7f9fe80314c0 comp rx=0 tx=0).stop 2024-09-11T00:29:55.447 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:55.444+0000 7f9ffd400640 1 -- 172.21.15.12:0/2774816578 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9ff81066b0 msgr2=0x7f9ff819c210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:29:55.447 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:55.444+0000 7f9ffd400640 1 --2- 172.21.15.12:0/2774816578 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9ff81066b0 0x7f9ff819c210 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f9ff000e940 tx=0x7f9ff000ee00 comp rx=0 tx=0).stop 2024-09-11T00:29:55.447 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:55.444+0000 7f9ffd400640 1 -- 172.21.15.12:0/2774816578 shutdown_connections 2024-09-11T00:29:55.447 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:55.444+0000 7f9ffd400640 1 --2- 172.21.15.12:0/2774816578 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f9fdc0b5360 0x7f9fdc0b7820 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:29:55.448 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:55.444+0000 7f9ffd400640 1 --2- 172.21.15.12:0/2774816578 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9ff81066b0 0x7f9ff819c210 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:29:55.448 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:55.444+0000 7f9ffd400640 1 --2- 172.21.15.12:0/2774816578 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9ff8105cf0 0x7f9ff819b5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:29:55.448 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:55.444+0000 7f9ffd400640 1 --2- 172.21.15.12:0/2774816578 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9ff8104ba0 0x7f9ff819af90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:29:55.448 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:55.444+0000 7f9ffd400640 1 -- 172.21.15.12:0/2774816578 >> 172.21.15.12:0/2774816578 conn(0x7f9ff8100460 msgr2=0x7f9ff81090e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:29:55.448 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:55.444+0000 7f9ffd400640 1 -- 172.21.15.12:0/2774816578 shutdown_connections 2024-09-11T00:29:55.448 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:29:55.444+0000 7f9ffd400640 1 -- 172.21.15.12:0/2774816578 wait complete. 2024-09-11T00:29:55.768 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:29:55 smithi012 ceph-mon[30333]: pgmap v1085: 129 pgs: 129 active+clean; 775 MiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 142 KiB/s rd, 151 KiB/s wr, 133 op/s 2024-09-11T00:29:56.033 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:29:55 smithi165 ceph-mon[35356]: pgmap v1085: 129 pgs: 129 active+clean; 775 MiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 142 KiB/s rd, 151 KiB/s wr, 133 op/s 2024-09-11T00:29:56.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:29:55 smithi178 ceph-mon[35919]: pgmap v1085: 129 pgs: 129 active+clean; 775 MiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 142 KiB/s rd, 151 KiB/s wr, 133 op/s 2024-09-11T00:29:56.277 INFO:tasks.workunit.client.0.smithi012.stdout: 10 164413 15.00 MB/sec execute 472 sec latency 22040.243 ms 2024-09-11T00:29:57.277 INFO:tasks.workunit.client.0.smithi012.stdout: 10 164568 14.99 MB/sec execute 473 sec latency 23040.293 ms 2024-09-11T00:29:57.989 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:29:57 smithi012 ceph-mon[30333]: pgmap v1086: 129 pgs: 129 active+clean; 768 MiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 145 KiB/s rd, 155 KiB/s wr, 160 op/s 2024-09-11T00:29:58.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:29:57 smithi165 ceph-mon[35356]: pgmap v1086: 129 pgs: 129 active+clean; 768 MiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 145 KiB/s rd, 155 KiB/s wr, 160 op/s 2024-09-11T00:29:58.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:29:57 smithi178 ceph-mon[35919]: pgmap v1086: 129 pgs: 129 active+clean; 768 MiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 145 KiB/s rd, 155 KiB/s wr, 160 op/s 2024-09-11T00:29:58.277 INFO:tasks.workunit.client.0.smithi012.stdout: 10 164843 14.99 MB/sec execute 474 sec latency 24040.358 ms 2024-09-11T00:29:59.277 INFO:tasks.workunit.client.0.smithi012.stdout: 10 165347 15.00 MB/sec execute 475 sec latency 24313.437 ms 2024-09-11T00:30:00.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:29:59 smithi165 ceph-mon[35356]: pgmap v1087: 129 pgs: 129 active+clean; 767 MiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 149 KiB/s rd, 169 KiB/s wr, 120 op/s 2024-09-11T00:30:00.063 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:29:59 smithi012 ceph-mon[30333]: pgmap v1087: 129 pgs: 129 active+clean; 767 MiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 149 KiB/s rd, 169 KiB/s wr, 120 op/s 2024-09-11T00:30:00.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:29:59 smithi178 ceph-mon[35919]: pgmap v1087: 129 pgs: 129 active+clean; 767 MiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 149 KiB/s rd, 169 KiB/s wr, 120 op/s 2024-09-11T00:30:00.277 INFO:tasks.workunit.client.0.smithi012.stdout: 10 166199 15.07 MB/sec execute 476 sec latency 50.723 ms 2024-09-11T00:30:01.032 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:30:00 smithi012 ceph-mon[30333]: overall HEALTH_OK 2024-09-11T00:30:01.033 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:30:00 smithi165 ceph-mon[35356]: overall HEALTH_OK 2024-09-11T00:30:01.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:30:00 smithi178 ceph-mon[35919]: overall HEALTH_OK 2024-09-11T00:30:01.277 INFO:tasks.workunit.client.0.smithi012.stdout: 10 167007 15.14 MB/sec execute 477 sec latency 82.544 ms 2024-09-11T00:30:02.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:30:01 smithi165 ceph-mon[35356]: pgmap v1088: 129 pgs: 129 active+clean; 770 MiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 666 KiB/s wr, 183 op/s 2024-09-11T00:30:02.063 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:30:01 smithi012 ceph-mon[30333]: pgmap v1088: 129 pgs: 129 active+clean; 770 MiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 666 KiB/s wr, 183 op/s 2024-09-11T00:30:02.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:30:01 smithi178 ceph-mon[35919]: pgmap v1088: 129 pgs: 129 active+clean; 770 MiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 666 KiB/s wr, 183 op/s 2024-09-11T00:30:02.277 INFO:tasks.workunit.client.0.smithi012.stdout: 10 167570 15.17 MB/sec execute 478 sec latency 160.812 ms 2024-09-11T00:30:03.277 INFO:tasks.workunit.client.0.smithi012.stdout: 10 168210 15.21 MB/sec execute 479 sec latency 116.855 ms 2024-09-11T00:30:04.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:30:03 smithi165 ceph-mon[35356]: pgmap v1089: 129 pgs: 129 active+clean; 771 MiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 1.1 MiB/s wr, 231 op/s 2024-09-11T00:30:04.063 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:30:03 smithi012 ceph-mon[30333]: pgmap v1089: 129 pgs: 129 active+clean; 771 MiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 1.1 MiB/s wr, 231 op/s 2024-09-11T00:30:04.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:30:03 smithi178 ceph-mon[35919]: pgmap v1089: 129 pgs: 129 active+clean; 771 MiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 1.1 MiB/s wr, 231 op/s 2024-09-11T00:30:04.277 INFO:tasks.workunit.client.0.smithi012.stdout: 10 168807 15.22 MB/sec execute 480 sec latency 75.151 ms 2024-09-11T00:30:05.277 INFO:tasks.workunit.client.0.smithi012.stdout: 10 169564 15.29 MB/sec execute 481 sec latency 315.589 ms 2024-09-11T00:30:06.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:30:05 smithi165 ceph-mon[35356]: pgmap v1090: 129 pgs: 129 active+clean; 772 MiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 1.1 MiB/s wr, 201 op/s 2024-09-11T00:30:06.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:30:05 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T00:30:06.063 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:30:05 smithi012 ceph-mon[30333]: pgmap v1090: 129 pgs: 129 active+clean; 772 MiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 1.1 MiB/s wr, 201 op/s 2024-09-11T00:30:06.063 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:30:05 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T00:30:06.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:30:05 smithi178 ceph-mon[35919]: pgmap v1090: 129 pgs: 129 active+clean; 772 MiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 1.1 MiB/s wr, 201 op/s 2024-09-11T00:30:06.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:30:05 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T00:30:06.277 INFO:tasks.workunit.client.0.smithi012.stdout: 10 170355 15.36 MB/sec execute 482 sec latency 99.238 ms 2024-09-11T00:30:07.277 INFO:tasks.workunit.client.0.smithi012.stdout: 10 170786 15.37 MB/sec execute 483 sec latency 454.395 ms 2024-09-11T00:30:07.887 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:30:07 smithi012 ceph-mon[30333]: pgmap v1091: 129 pgs: 129 active+clean; 779 MiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 2.5 MiB/s wr, 426 op/s 2024-09-11T00:30:07.888 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:30:07 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:30:07.888 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:30:07 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T00:30:07.888 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:30:07 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:30:08.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:30:07 smithi165 ceph-mon[35356]: pgmap v1091: 129 pgs: 129 active+clean; 779 MiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 2.5 MiB/s wr, 426 op/s 2024-09-11T00:30:08.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:30:07 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:30:08.033 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:30:07 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T00:30:08.033 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:30:07 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:30:08.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:30:07 smithi178 ceph-mon[35919]: pgmap v1091: 129 pgs: 129 active+clean; 779 MiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 2.5 MiB/s wr, 426 op/s 2024-09-11T00:30:08.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:30:07 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:30:08.082 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:30:07 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T00:30:08.082 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:30:07 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:30:08.277 INFO:tasks.workunit.client.0.smithi012.stdout: 10 170820 15.34 MB/sec execute 484 sec latency 1128.435 ms 2024-09-11T00:30:08.748 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:08.745+0000 7fa0e1200640 1 -- 172.21.15.12:0/1507557255 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa0dc100860 msgr2=0x7fa0dc100c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:30:08.748 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:08.745+0000 7fa0e1200640 1 --2- 172.21.15.12:0/1507557255 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa0dc100860 0x7fa0dc100c60 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fa0c4009900 tx=0x7fa0c402a410 comp rx=0 tx=0).stop 2024-09-11T00:30:08.751 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:08.745+0000 7fa0e1200640 1 -- 172.21.15.12:0/1507557255 shutdown_connections 2024-09-11T00:30:08.751 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:08.745+0000 7fa0e1200640 1 --2- 172.21.15.12:0/1507557255 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa0dc102260 0x7fa0dc106bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:30:08.751 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:08.745+0000 7fa0e1200640 1 --2- 172.21.15.12:0/1507557255 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa0dc1018a0 0x7fa0dc101d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:30:08.751 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:08.745+0000 7fa0e1200640 1 --2- 172.21.15.12:0/1507557255 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa0dc100860 0x7fa0dc100c60 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:30:08.751 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:08.745+0000 7fa0e1200640 1 -- 172.21.15.12:0/1507557255 >> 172.21.15.12:0/1507557255 conn(0x7fa0dc0fc440 msgr2=0x7fa0dc0fe880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:30:08.752 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:08.745+0000 7fa0e1200640 1 -- 172.21.15.12:0/1507557255 shutdown_connections 2024-09-11T00:30:08.752 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:08.745+0000 7fa0e1200640 1 -- 172.21.15.12:0/1507557255 wait complete. 2024-09-11T00:30:08.753 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:08.751+0000 7fa0e1200640 1 Processor -- start 2024-09-11T00:30:08.754 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:08.751+0000 7fa0e1200640 1 -- start start 2024-09-11T00:30:08.756 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:08.751+0000 7fa0e1200640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa0dc100860 0x7fa0dc19f3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:30:08.756 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:08.751+0000 7fa0e1200640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa0dc1018a0 0x7fa0dc19fa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:30:08.756 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:08.751+0000 7fa0e1200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa0dc102260 0x7fa0dc1a0640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:30:08.756 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:08.751+0000 7fa0d9400640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa0dc1018a0 0x7fa0dc19fa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:30:08.756 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:08.751+0000 7fa0d9400640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa0dc1018a0 0x7fa0dc19fa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.12:57938/0 (socket says 172.21.15.12:57938) 2024-09-11T00:30:08.756 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:08.751+0000 7fa0e1200640 1 -- 172.21.15.12:0/123813247 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fa0dc114c90 con 0x7fa0dc100860 2024-09-11T00:30:08.756 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:08.751+0000 7fa0e1200640 1 -- 172.21.15.12:0/123813247 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fa0dc114b10 con 0x7fa0dc102260 2024-09-11T00:30:08.757 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:08.751+0000 7fa0e1200640 1 -- 172.21.15.12:0/123813247 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fa0dc114e10 con 0x7fa0dc1018a0 2024-09-11T00:30:08.757 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:08.751+0000 7fa0d9400640 1 -- 172.21.15.12:0/123813247 learned_addr learned my addr 172.21.15.12:0/123813247 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:30:08.757 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:08.751+0000 7fa0d9e00640 1 --2- 172.21.15.12:0/123813247 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa0dc100860 0x7fa0dc19f3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:30:08.757 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:08.751+0000 7fa0da800640 1 --2- 172.21.15.12:0/123813247 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa0dc102260 0x7fa0dc1a0640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:30:08.757 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:08.751+0000 7fa0d9400640 1 -- 172.21.15.12:0/123813247 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa0dc102260 msgr2=0x7fa0dc1a0640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:30:08.757 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:08.751+0000 7fa0d9400640 1 --2- 172.21.15.12:0/123813247 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa0dc102260 0x7fa0dc1a0640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:30:08.757 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:08.751+0000 7fa0d9400640 1 -- 172.21.15.12:0/123813247 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa0dc100860 msgr2=0x7fa0dc19f3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:30:08.757 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:08.751+0000 7fa0d9400640 1 --2- 172.21.15.12:0/123813247 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa0dc100860 0x7fa0dc19f3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:30:08.757 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:08.751+0000 7fa0d9400640 1 -- 172.21.15.12:0/123813247 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0dc1a0f00 con 0x7fa0dc1018a0 2024-09-11T00:30:08.757 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:08.751+0000 7fa0d9400640 1 --2- 172.21.15.12:0/123813247 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa0dc1018a0 0x7fa0dc19fa10 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fa0cc00e9f0 tx=0x7fa0cc00edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:30:08.758 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:08.751+0000 7fa0d2a00640 1 -- 172.21.15.12:0/123813247 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0cc00cd80 con 0x7fa0dc1018a0 2024-09-11T00:30:08.758 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:08.751+0000 7fa0e1200640 1 -- 172.21.15.12:0/123813247 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa0dc1a11f0 con 0x7fa0dc1018a0 2024-09-11T00:30:08.758 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:08.751+0000 7fa0d2a00640 1 -- 172.21.15.12:0/123813247 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa0cc0054d0 con 0x7fa0dc1018a0 2024-09-11T00:30:08.758 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:08.751+0000 7fa0d2a00640 1 -- 172.21.15.12:0/123813247 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0cc010430 con 0x7fa0dc1018a0 2024-09-11T00:30:08.758 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:08.751+0000 7fa0e1200640 1 -- 172.21.15.12:0/123813247 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0dc1a78c0 con 0x7fa0dc1018a0 2024-09-11T00:30:08.766 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:08.763+0000 7fa0e3ffd740 1 Processor -- start 2024-09-11T00:30:08.766 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:08.763+0000 7fa0d2a00640 1 -- 172.21.15.12:0/123813247 <== mon.1 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7fa0cc005020 con 0x7fa0dc1018a0 2024-09-11T00:30:08.766 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:08.763+0000 7fa0e3ffd740 1 -- start start 2024-09-11T00:30:08.767 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:08.763+0000 7fa0e3ffd740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5609701f7790 0x5609701f9b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:30:08.767 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:08.763+0000 7fa0e3ffd740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5609701fa0c0 0x5609701fc4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:30:08.767 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:08.763+0000 7fa0e3ffd740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5609701fcb00 0x5609701fefc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:30:08.768 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:08.763+0000 7fa0e3ffd740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x56097016a6b0 con 0x5609701f7790 2024-09-11T00:30:08.768 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:08.763+0000 7fa0e3ffd740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x560970159ae0 con 0x5609701fcb00 2024-09-11T00:30:08.768 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:08.763+0000 7fa0e3ffd740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x56097015d8f0 con 0x5609701fa0c0 2024-09-11T00:30:08.768 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:08.763+0000 7fa0d9400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5609701fcb00 0x5609701fefc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:30:08.768 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:08.763+0000 7fa0d9400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5609701fcb00 0x5609701fefc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.12:38448/0 (socket says 172.21.15.12:38448) 2024-09-11T00:30:08.768 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:08.763+0000 7fa0d9400640 1 -- 172.21.15.12:0/3994942435 learned_addr learned my addr 172.21.15.12:0/3994942435 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:30:08.768 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:08.763+0000 7fa0d9e00640 1 --2- 172.21.15.12:0/3994942435 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5609701fa0c0 0x5609701fc4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:30:08.768 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:08.763+0000 7fa0da800640 1 --2- 172.21.15.12:0/3994942435 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5609701f7790 0x5609701f9b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:30:08.768 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:08.763+0000 7fa0d9400640 1 -- 172.21.15.12:0/3994942435 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5609701fa0c0 msgr2=0x5609701fc4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:30:08.768 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:08.763+0000 7fa0d9400640 1 --2- 172.21.15.12:0/3994942435 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5609701fa0c0 0x5609701fc4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:30:08.768 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:08.763+0000 7fa0d9400640 1 -- 172.21.15.12:0/3994942435 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5609701f7790 msgr2=0x5609701f9b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:30:08.768 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:08.763+0000 7fa0d9400640 1 --2- 172.21.15.12:0/3994942435 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5609701f7790 0x5609701f9b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:30:08.769 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:08.763+0000 7fa0d9400640 1 -- 172.21.15.12:0/3994942435 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560970126780 con 0x5609701fcb00 2024-09-11T00:30:08.769 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:08.763+0000 7fa0da800640 1 --2- 172.21.15.12:0/3994942435 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5609701f7790 0x5609701f9b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T00:30:08.769 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:08.763+0000 7fa0d9400640 1 --2- 172.21.15.12:0/3994942435 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5609701fcb00 0x5609701fefc0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fa0cc0e3a30 tx=0x7fa0cc0ee4b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d39c5a6f2d7c5aa2 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:30:08.770 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:08.763+0000 7fa0cb400640 1 -- 172.21.15.12:0/3994942435 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0cc0fa070 con 0x5609701fcb00 2024-09-11T00:30:08.770 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:08.763+0000 7fa0cb400640 1 -- 172.21.15.12:0/3994942435 <== mon.2 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa0cc0e80d0 con 0x5609701fcb00 2024-09-11T00:30:08.770 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:08.763+0000 7fa0cb400640 1 -- 172.21.15.12:0/3994942435 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0cc0f2700 con 0x5609701fcb00 2024-09-11T00:30:08.770 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:08.763+0000 7fa0d2a00640 1 --2- 172.21.15.12:0/123813247 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fa0c00b52e0 0x7fa0c00b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:30:08.770 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:08.763+0000 7fa0d2a00640 1 -- 172.21.15.12:0/123813247 <== mon.1 v2:172.21.15.178:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9208+0+0 (secure 0 0 0) 0x7fa0cc0e63c0 con 0x7fa0dc1018a0 2024-09-11T00:30:08.770 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:08.763+0000 7fa0da800640 1 --2- 172.21.15.12:0/123813247 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fa0c00b52e0 0x7fa0c00b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:30:08.771 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:08.763+0000 7fa0e3ffd740 1 -- 172.21.15.12:0/3994942435 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5609701fcb00 msgr2=0x5609701fefc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:30:08.771 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:08.763+0000 7fa0e3ffd740 1 --2- 172.21.15.12:0/3994942435 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5609701fcb00 0x5609701fefc0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fa0cc0e3a30 tx=0x7fa0cc0ee4b0 comp rx=0 tx=0).stop 2024-09-11T00:30:08.771 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:08.768+0000 7fa0e3ffd740 1 -- 172.21.15.12:0/3994942435 shutdown_connections 2024-09-11T00:30:08.771 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:08.768+0000 7fa0e3ffd740 1 --2- 172.21.15.12:0/3994942435 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5609701fcb00 0x5609701fefc0 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:30:08.771 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:08.768+0000 7fa0e3ffd740 1 --2- 172.21.15.12:0/3994942435 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5609701fa0c0 0x5609701fc4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:30:08.771 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:08.768+0000 7fa0e3ffd740 1 --2- 172.21.15.12:0/3994942435 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5609701f7790 0x5609701f9b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:30:08.771 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:08.768+0000 7fa0e3ffd740 1 -- 172.21.15.12:0/3994942435 >> 172.21.15.12:0/3994942435 conn(0x5609701f28e0 msgr2=0x5609701f3670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:30:08.771 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:08.768+0000 7fa0e3ffd740 1 -- 172.21.15.12:0/3994942435 shutdown_connections 2024-09-11T00:30:08.771 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:08.768+0000 7fa0e3ffd740 1 -- 172.21.15.12:0/3994942435 wait complete. 2024-09-11T00:30:08.772 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:08.768+0000 7fa0e3ffd740 1 -- start start 2024-09-11T00:30:08.773 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:08.768+0000 7fa0e3ffd740 1 Processor -- start 2024-09-11T00:30:08.773 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:08.768+0000 7fa0e3ffd740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5609701f7790 0x560970200ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:30:08.773 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:08.768+0000 7fa0e3ffd740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5609701fa0c0 0x560970201560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:30:08.773 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:08.768+0000 7fa0e3ffd740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5609701fcb00 0x560970202130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:30:08.773 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:08.768+0000 7fa0e3ffd740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x560970091ab0 con 0x5609701fcb00 2024-09-11T00:30:08.773 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:08.768+0000 7fa0e3ffd740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x560970100570 con 0x5609701f7790 2024-09-11T00:30:08.773 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:08.768+0000 7fa0e3ffd740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5609701ffae0 con 0x5609701fa0c0 2024-09-11T00:30:08.773 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:08.768+0000 7fa0d9400640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5609701fa0c0 0x560970201560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:30:08.773 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:08.768+0000 7fa0e3ffd740 10 client.0 _ll_register_callbacks cb 0x56097010f100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:30:08.773 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:08.768+0000 7fa0d9400640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5609701fa0c0 0x560970201560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.12:57952/0 (socket says 172.21.15.12:57952) 2024-09-11T00:30:08.773 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:08.768+0000 7fa0d9400640 1 -- 172.21.15.12:0/3611040397 learned_addr learned my addr 172.21.15.12:0/3611040397 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:30:08.774 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:08.768+0000 7fa0da800640 1 --2- 172.21.15.12:0/123813247 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fa0c00b52e0 0x7fa0c00b77a0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7fa0dc1a0140 tx=0x7fa0d4009650 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:30:08.774 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:08.768+0000 7fa0d9400640 1 -- 172.21.15.12:0/3611040397 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5609701f7790 msgr2=0x560970200ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:30:08.775 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:08.768+0000 7fa0d9400640 1 --2- 172.21.15.12:0/3611040397 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5609701f7790 0x560970200ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:30:08.775 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:08.768+0000 7fa0d9400640 1 -- 172.21.15.12:0/3611040397 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5609701fcb00 msgr2=0x560970202130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:30:08.775 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:08.768+0000 7fa0d9400640 1 --2- 172.21.15.12:0/3611040397 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5609701fcb00 0x560970202130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:30:08.775 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:08.768+0000 7fa0d9400640 1 -- 172.21.15.12:0/3611040397 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56096febb790 con 0x5609701fa0c0 2024-09-11T00:30:08.775 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:08.768+0000 7fa0d9400640 1 --2- 172.21.15.12:0/3611040397 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5609701fa0c0 0x560970201560 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fa0cc0a8d80 tx=0x7fa0cc0e3a60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=343afb6d47b66f07 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:30:08.775 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:08.768+0000 7fa0bfe00640 10 client.0 ms_handle_connect on v2:172.21.15.178:3300/0 2024-09-11T00:30:08.775 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:08.768+0000 7fa0bfe00640 1 -- 172.21.15.12:0/3611040397 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0cc0a8e00 con 0x5609701fa0c0 2024-09-11T00:30:08.775 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:08.768+0000 7fa0bfe00640 1 -- 172.21.15.12:0/3611040397 <== mon.1 v2:172.21.15.178:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 9208+0+0 (secure 0 0 0) 0x7fa0cc0aca10 con 0x5609701fa0c0 2024-09-11T00:30:08.775 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:08.768+0000 7fa0e3ffd740 1 -- 172.21.15.12:0/3611040397 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa0cc0e80d0 con 0x5609701fa0c0 2024-09-11T00:30:08.775 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:08.768+0000 7fa0bfe00640 1 -- 172.21.15.12:0/3611040397 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=102) ==== 24+0+0 (secure 0 0 0) 0x7fa0cc0abf20 con 0x5609701fa0c0 2024-09-11T00:30:08.775 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:08.768+0000 7fa0e3ffd740 10 client.27937 fetch_fsmap learned FSMap version 102 2024-09-11T00:30:08.775 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:08.768+0000 7fa0e3ffd740 1 -- 172.21.15.12:0/3611040397 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,fsmap=102,monmap=4+}) -- 0x56096ff34620 con 0x5609701fa0c0 2024-09-11T00:30:08.776 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:08.768+0000 7fa0bfe00640 1 -- 172.21.15.12:0/3611040397 <== mon.1 v2:172.21.15.178:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa0cc0e80d0 con 0x5609701fa0c0 2024-09-11T00:30:08.776 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:08.768+0000 7fa0bfe00640 1 -- 172.21.15.12:0/3611040397 <== mon.1 v2:172.21.15.178:3300/0 5 ==== fsmap(e 102) ==== 8378+0+0 (secure 0 0 0) 0x7fa0cc0b58b0 con 0x5609701fa0c0 2024-09-11T00:30:08.776 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:08.773+0000 7fa0e3ffd740 10 client.27937 fetch_fsmap finished waiting for FSMap version 102 2024-09-11T00:30:08.776 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:08.773+0000 7fa0e3ffd740 10 client.27937 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T00:30:08.776 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:08.773+0000 7fa0e3ffd740 20 client.27937 populate_metadata read hostname 'smithi012' 2024-09-11T00:30:08.777 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:08.773+0000 7fa0e3ffd740 1 --2- 172.21.15.12:0/3611040397 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x56097020c2d0 0x56097020e6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:30:08.777 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:08.773+0000 7fa0e3ffd740 4 client.27937 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T00:30:08.777 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:08.773+0000 7fa0e3ffd740 1 -- 172.21.15.12:0/3611040397 --> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56097016c600 con 0x56097020c2d0 2024-09-11T00:30:08.777 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:08.773+0000 7fa0d9e00640 1 --2- 172.21.15.12:0/3611040397 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x56097020c2d0 0x56097020e6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:30:08.780 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:08.778+0000 7fa0d9e00640 1 --2- 172.21.15.12:0/3611040397 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x56097020c2d0 0x56097020e6c0 crc :-1 s=READY pgs=313 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=88370b138a968567 server_cookie=5e3564000d9c90e1 in_seq=0 out_seq=0 2024-09-11T00:30:08.780 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:08.778+0000 7fa0bfe00640 10 client.27937 ms_handle_connect on v2:172.21.15.165:6838/501312984 2024-09-11T00:30:08.781 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:08.778+0000 7fa0bfe00640 1 -- 172.21.15.12:0/3611040397 <== mds.0 v2:172.21.15.165:6838/501312984 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x56097016c600 con 0x56097020c2d0 2024-09-11T00:30:08.781 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:08.778+0000 7fa0bfe00640 10 client.27937 handle_command_reply: tid=0 2024-09-11T00:30:08.781 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:08.778+0000 7fa0e3ffd740 1 client.27937 shutdown 2024-09-11T00:30:08.783 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:08.781+0000 7fa0e3ffd740 1 -- 172.21.15.12:0/3611040397 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5609701fa0c0 msgr2=0x560970201560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:30:08.783 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:08.781+0000 7fa0e3ffd740 1 --2- 172.21.15.12:0/3611040397 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5609701fa0c0 0x560970201560 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fa0cc0a8d80 tx=0x7fa0cc0e3a60 comp rx=0 tx=0).stop 2024-09-11T00:30:08.783 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:08.781+0000 7fa0e3ffd740 1 -- 172.21.15.12:0/3611040397 shutdown_connections 2024-09-11T00:30:08.784 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:08.781+0000 7fa0e3ffd740 1 --2- 172.21.15.12:0/3611040397 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x56097020c2d0 0x56097020e6c0 crc :-1 s=READY pgs=313 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:30:08.784 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:08.781+0000 7fa0e3ffd740 1 --2- 172.21.15.12:0/3611040397 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5609701fcb00 0x560970202130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:30:08.784 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:08.781+0000 7fa0e3ffd740 1 --2- 172.21.15.12:0/3611040397 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5609701fa0c0 0x560970201560 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:30:08.784 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:08.781+0000 7fa0bfe00640 0 client.27937 ms_handle_reset on v2:172.21.15.165:6838/501312984 2024-09-11T00:30:08.784 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:08.781+0000 7fa0e3ffd740 1 --2- 172.21.15.12:0/3611040397 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5609701f7790 0x560970200ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:30:08.784 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:08.781+0000 7fa0e3ffd740 1 -- 172.21.15.12:0/3611040397 >> 172.21.15.12:0/3611040397 conn(0x5609701f28e0 msgr2=0x5609701f1a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:30:08.784 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:08.781+0000 7fa0e3ffd740 1 -- 172.21.15.12:0/3611040397 shutdown_connections 2024-09-11T00:30:08.785 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:08.781+0000 7fa0e3ffd740 1 -- 172.21.15.12:0/3611040397 wait complete. 2024-09-11T00:30:08.785 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:08.781+0000 7fa0e3ffd740 20 client.27937 trim_cache size 0 max 16384 2024-09-11T00:30:08.838 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:08.828+0000 7fa0e3ffd740 1 Processor -- start 2024-09-11T00:30:08.838 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:08.828+0000 7fa0e3ffd740 1 -- start start 2024-09-11T00:30:08.838 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:08.828+0000 7fa0e3ffd740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5609701f7790 0x5609701f3140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:30:08.838 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:08.828+0000 7fa0e3ffd740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5609701fa0c0 0x5609702091b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:30:08.838 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:08.828+0000 7fa0e3ffd740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5609701fcb00 0x5609701f3680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:30:08.839 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:08.828+0000 7fa0e3ffd740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fa0dc1b1e90 con 0x5609701f7790 2024-09-11T00:30:08.839 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:08.828+0000 7fa0e3ffd740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fa0dc1b3a30 con 0x5609701fcb00 2024-09-11T00:30:08.839 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:08.828+0000 7fa0e3ffd740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fa0dc1b25c0 con 0x5609701fa0c0 2024-09-11T00:30:08.839 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:08.828+0000 7fa0da800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5609701fcb00 0x5609701f3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:30:08.839 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:08.828+0000 7fa0d9e00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5609701f7790 0x5609701f3140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:30:08.839 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:08.828+0000 7fa0da800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5609701fcb00 0x5609701f3680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.12:38468/0 (socket says 172.21.15.12:38468) 2024-09-11T00:30:08.839 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:08.828+0000 7fa0da800640 1 -- 172.21.15.12:0/1922279369 learned_addr learned my addr 172.21.15.12:0/1922279369 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:30:08.839 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:08.828+0000 7fa0d9400640 1 --2- 172.21.15.12:0/1922279369 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5609701fa0c0 0x5609702091b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:30:08.839 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:08.828+0000 7fa0d9e00640 1 -- 172.21.15.12:0/1922279369 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5609701fa0c0 msgr2=0x5609702091b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:30:08.839 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:08.828+0000 7fa0d9e00640 1 --2- 172.21.15.12:0/1922279369 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5609701fa0c0 0x5609702091b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:30:08.839 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:08.828+0000 7fa0d9e00640 1 -- 172.21.15.12:0/1922279369 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5609701fcb00 msgr2=0x5609701f3680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:30:08.839 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:08.828+0000 7fa0d9e00640 1 --2- 172.21.15.12:0/1922279369 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5609701fcb00 0x5609701f3680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:30:08.839 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:08.828+0000 7fa0d9e00640 1 -- 172.21.15.12:0/1922279369 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56097020ea30 con 0x5609701f7790 2024-09-11T00:30:08.840 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:08.828+0000 7fa0d9400640 1 --2- 172.21.15.12:0/1922279369 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5609701fa0c0 0x5609702091b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T00:30:08.840 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:08.834+0000 7fa0d9e00640 1 --2- 172.21.15.12:0/1922279369 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5609701f7790 0x5609701f3140 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7fa0c4006010 tx=0x7fa0c4031430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9fc869503dd51236 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:30:08.840 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:08.834+0000 7fa0caa00640 1 -- 172.21.15.12:0/1922279369 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0c4035040 con 0x5609701f7790 2024-09-11T00:30:08.840 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:08.834+0000 7fa0caa00640 1 -- 172.21.15.12:0/1922279369 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa0c4047070 con 0x5609701f7790 2024-09-11T00:30:08.840 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:08.834+0000 7fa0caa00640 1 -- 172.21.15.12:0/1922279369 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0c4031900 con 0x5609701f7790 2024-09-11T00:30:08.840 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:08.834+0000 7fa0e3ffd740 1 -- 172.21.15.12:0/1922279369 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5609701f7790 msgr2=0x5609701f3140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:30:08.840 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:08.834+0000 7fa0e3ffd740 1 --2- 172.21.15.12:0/1922279369 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5609701f7790 0x5609701f3140 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7fa0c4006010 tx=0x7fa0c4031430 comp rx=0 tx=0).stop 2024-09-11T00:30:08.841 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:08.834+0000 7fa0e3ffd740 1 -- 172.21.15.12:0/1922279369 shutdown_connections 2024-09-11T00:30:08.842 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:08.834+0000 7fa0e3ffd740 1 --2- 172.21.15.12:0/1922279369 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5609701fcb00 0x5609701f3680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:30:08.842 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:08.834+0000 7fa0e3ffd740 1 --2- 172.21.15.12:0/1922279369 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5609701fa0c0 0x5609702091b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:30:08.842 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:08.834+0000 7fa0e3ffd740 1 --2- 172.21.15.12:0/1922279369 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5609701f7790 0x5609701f3140 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:30:08.842 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:08.834+0000 7fa0e3ffd740 1 -- 172.21.15.12:0/1922279369 >> 172.21.15.12:0/1922279369 conn(0x5609701f28e0 msgr2=0x5609701fcf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:30:08.842 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:08.834+0000 7fa0e3ffd740 1 -- 172.21.15.12:0/1922279369 shutdown_connections 2024-09-11T00:30:08.842 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:08.834+0000 7fa0e3ffd740 1 -- 172.21.15.12:0/1922279369 wait complete. 2024-09-11T00:30:08.842 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:08.834+0000 7fa0e3ffd740 1 -- start start 2024-09-11T00:30:08.842 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:08.834+0000 7fa0e3ffd740 1 Processor -- start 2024-09-11T00:30:08.842 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:08.839+0000 7fa0e3ffd740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5609701f7790 0x5609702080b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:30:08.843 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:08.839+0000 7fa0e3ffd740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5609701fa0c0 0x5609702085f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:30:08.843 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:08.840+0000 7fa0e3ffd740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5609701fcb00 0x560970208b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:30:08.843 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:08.840+0000 7fa0e3ffd740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fa0dc1b3360 con 0x5609701fcb00 2024-09-11T00:30:08.843 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:08.840+0000 7fa0d9400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5609701fa0c0 0x5609702085f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:30:08.843 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:08.840+0000 7fa0e3ffd740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fa0dc1b2c90 con 0x5609701fa0c0 2024-09-11T00:30:08.844 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:08.840+0000 7fa0d9400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5609701fa0c0 0x5609702085f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.12:38476/0 (socket says 172.21.15.12:38476) 2024-09-11T00:30:08.844 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:08.840+0000 7fa0e3ffd740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fa0dc1afe70 con 0x5609701f7790 2024-09-11T00:30:08.844 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:08.840+0000 7fa0d9400640 1 -- 172.21.15.12:0/1444983581 learned_addr learned my addr 172.21.15.12:0/1444983581 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:30:08.844 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:08.840+0000 7fa0e3ffd740 10 client.0 _ll_register_callbacks cb 0x7fa0cc0f8070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:30:08.845 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:08.840+0000 7fa0da800640 1 --2- 172.21.15.12:0/1444983581 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5609701fcb00 0x560970208b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:30:08.845 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:08.840+0000 7fa0da800640 1 -- 172.21.15.12:0/1444983581 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5609701f7790 msgr2=0x5609702080b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:30:08.846 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:08.840+0000 7fa0da800640 1 --2- 172.21.15.12:0/1444983581 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5609701f7790 0x5609702080b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:30:08.846 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:08.840+0000 7fa0da800640 1 -- 172.21.15.12:0/1444983581 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5609701fa0c0 msgr2=0x5609702085f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:30:08.846 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:08.840+0000 7fa0da800640 1 --2- 172.21.15.12:0/1444983581 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5609701fa0c0 0x5609702085f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:30:08.846 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:08.840+0000 7fa0da800640 1 -- 172.21.15.12:0/1444983581 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5609702024a0 con 0x5609701fcb00 2024-09-11T00:30:08.846 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:08.840+0000 7fa0da800640 1 --2- 172.21.15.12:0/1444983581 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5609701fcb00 0x560970208b30 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7fa0d4009a60 tx=0x7fa0d4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e46dfbd0cfaa3837 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:30:08.846 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:08.840+0000 7fa0bfe00640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-09-11T00:30:08.846 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:08.840+0000 7fa0bfe00640 1 -- 172.21.15.12:0/1444983581 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0d4018c30 con 0x5609701fcb00 2024-09-11T00:30:08.846 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:08.840+0000 7fa0bfe00640 1 -- 172.21.15.12:0/1444983581 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 9208+0+0 (secure 0 0 0) 0x7fa0d4008040 con 0x5609701fcb00 2024-09-11T00:30:08.846 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:08.840+0000 7fa0e3ffd740 1 -- 172.21.15.12:0/1444983581 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa0c4047070 con 0x5609701fcb00 2024-09-11T00:30:08.847 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:08.840+0000 7fa0bfe00640 1 -- 172.21.15.12:0/1444983581 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=102) ==== 24+0+0 (secure 0 0 0) 0x7fa0d4030050 con 0x5609701fcb00 2024-09-11T00:30:08.847 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:08.840+0000 7fa0e3ffd740 10 client.17943 fetch_fsmap learned FSMap version 102 2024-09-11T00:30:08.847 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:08.840+0000 7fa0e3ffd740 1 -- 172.21.15.12:0/1444983581 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=102,monmap=4+}) -- 0x560970279570 con 0x5609701fcb00 2024-09-11T00:30:08.849 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:08.840+0000 7fa0bfe00640 1 -- 172.21.15.12:0/1444983581 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa0c4047070 con 0x5609701fcb00 2024-09-11T00:30:08.850 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:08.840+0000 7fa0bfe00640 1 -- 172.21.15.12:0/1444983581 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 102) ==== 8378+0+0 (secure 0 0 0) 0x7fa0d402b030 con 0x5609701fcb00 2024-09-11T00:30:08.850 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:08.840+0000 7fa0e3ffd740 10 client.17943 fetch_fsmap finished waiting for FSMap version 102 2024-09-11T00:30:08.850 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:08.840+0000 7fa0e3ffd740 10 client.17943 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T00:30:08.850 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:08.840+0000 7fa0e3ffd740 20 client.17943 populate_metadata read hostname 'smithi012' 2024-09-11T00:30:08.850 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:08.840+0000 7fa0e3ffd740 1 --2- 172.21.15.12:0/1444983581 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x560970279fa0 0x56097027c3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:30:08.850 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:08.840+0000 7fa0e3ffd740 4 client.17943 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T00:30:08.850 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:08.840+0000 7fa0e3ffd740 1 -- 172.21.15.12:0/1444983581 --> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5609701680b0 con 0x560970279fa0 2024-09-11T00:30:08.850 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:08.846+0000 7fa0d9e00640 1 --2- 172.21.15.12:0/1444983581 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x560970279fa0 0x56097027c3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:30:08.866 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:08.864+0000 7fa0d9e00640 1 --2- 172.21.15.12:0/1444983581 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x560970279fa0 0x56097027c3c0 crc :-1 s=READY pgs=314 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b5451062bd5c2cbf server_cookie=6f6c2eaa112835ad in_seq=0 out_seq=0 2024-09-11T00:30:08.867 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:08.864+0000 7fa0bfe00640 10 client.17943 ms_handle_connect on v2:172.21.15.165:6838/501312984 2024-09-11T00:30:09.275 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:09.271+0000 7fa0bfe00640 1 -- 172.21.15.12:0/1444983581 <== mds.0 v2:172.21.15.165:6838/501312984 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5609701680b0 con 0x560970279fa0 2024-09-11T00:30:09.275 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:09.271+0000 7fa0bfe00640 10 client.17943 handle_command_reply: tid=0 2024-09-11T00:30:09.276 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:09.271+0000 7fa0e3ffd740 1 client.17943 shutdown 2024-09-11T00:30:09.277 INFO:tasks.workunit.client.0.smithi012.stdout: 10 170847 15.31 MB/sec execute 485 sec latency 1499.028 ms 2024-09-11T00:30:09.278 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:09.271+0000 7fa0e3ffd740 1 -- 172.21.15.12:0/1444983581 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5609701fcb00 msgr2=0x560970208b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:30:09.278 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:09.271+0000 7fa0e3ffd740 1 --2- 172.21.15.12:0/1444983581 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5609701fcb00 0x560970208b30 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7fa0d4009a60 tx=0x7fa0d4004a40 comp rx=0 tx=0).stop 2024-09-11T00:30:09.278 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:09.271+0000 7fa0e3ffd740 1 -- 172.21.15.12:0/1444983581 shutdown_connections 2024-09-11T00:30:09.279 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:09.271+0000 7fa0e3ffd740 1 --2- 172.21.15.12:0/1444983581 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x560970279fa0 0x56097027c3c0 crc :-1 s=READY pgs=314 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:30:09.279 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:09.271+0000 7fa0e3ffd740 1 --2- 172.21.15.12:0/1444983581 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5609701fcb00 0x560970208b30 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:30:09.279 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:09.271+0000 7fa0e3ffd740 1 --2- 172.21.15.12:0/1444983581 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5609701fa0c0 0x5609702085f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:30:09.279 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:09.271+0000 7fa0e3ffd740 1 --2- 172.21.15.12:0/1444983581 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5609701f7790 0x5609702080b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:30:09.279 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:09.271+0000 7fa0bfe00640 0 client.17943 ms_handle_reset on v2:172.21.15.165:6838/501312984 2024-09-11T00:30:09.279 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:09.271+0000 7fa0e3ffd740 1 -- 172.21.15.12:0/1444983581 >> 172.21.15.12:0/1444983581 conn(0x5609701f28e0 msgr2=0x5609701f6850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:30:09.279 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:09.271+0000 7fa0e3ffd740 1 -- 172.21.15.12:0/1444983581 shutdown_connections 2024-09-11T00:30:09.280 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:09.271+0000 7fa0e3ffd740 1 -- 172.21.15.12:0/1444983581 wait complete. 2024-09-11T00:30:09.280 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:09.277+0000 7fa0e3ffd740 20 client.17943 trim_cache size 0 max 16384 2024-09-11T00:30:09.280 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:{ 2024-09-11T00:30:09.280 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "message": "", 2024-09-11T00:30:09.280 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "return_code": 0 2024-09-11T00:30:09.280 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:} 2024-09-11T00:30:09.283 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:09.277+0000 7fa0e1200640 1 -- 172.21.15.12:0/123813247 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fa0c00b52e0 msgr2=0x7fa0c00b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:30:09.283 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:09.277+0000 7fa0e1200640 1 --2- 172.21.15.12:0/123813247 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fa0c00b52e0 0x7fa0c00b77a0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7fa0dc1a0140 tx=0x7fa0d4009650 comp rx=0 tx=0).stop 2024-09-11T00:30:09.283 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:09.277+0000 7fa0e1200640 1 -- 172.21.15.12:0/123813247 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa0dc1018a0 msgr2=0x7fa0dc19fa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:30:09.283 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:09.277+0000 7fa0e1200640 1 --2- 172.21.15.12:0/123813247 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa0dc1018a0 0x7fa0dc19fa10 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fa0cc00e9f0 tx=0x7fa0cc00edf0 comp rx=0 tx=0).stop 2024-09-11T00:30:09.283 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:09.277+0000 7fa0e1200640 1 -- 172.21.15.12:0/123813247 shutdown_connections 2024-09-11T00:30:09.284 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:09.277+0000 7fa0e1200640 1 --2- 172.21.15.12:0/123813247 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fa0c00b52e0 0x7fa0c00b77a0 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:30:09.284 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:09.277+0000 7fa0e1200640 1 --2- 172.21.15.12:0/123813247 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa0dc102260 0x7fa0dc1a0640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:30:09.284 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:09.277+0000 7fa0e1200640 1 --2- 172.21.15.12:0/123813247 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa0dc1018a0 0x7fa0dc19fa10 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:30:09.284 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:09.277+0000 7fa0e1200640 1 --2- 172.21.15.12:0/123813247 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa0dc100860 0x7fa0dc19f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:30:09.284 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:09.277+0000 7fa0e1200640 1 -- 172.21.15.12:0/123813247 >> 172.21.15.12:0/123813247 conn(0x7fa0dc0fc440 msgr2=0x7fa0dc102bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:30:09.284 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:09.277+0000 7fa0e1200640 1 -- 172.21.15.12:0/123813247 shutdown_connections 2024-09-11T00:30:09.284 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:09.277+0000 7fa0e1200640 1 -- 172.21.15.12:0/123813247 wait complete. 2024-09-11T00:30:10.032 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:30:09 smithi165 ceph-mon[35356]: pgmap v1092: 129 pgs: 129 active+clean; 781 MiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 2.8 MiB/s wr, 411 op/s 2024-09-11T00:30:10.065 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:30:09 smithi012 ceph-mon[30333]: pgmap v1092: 129 pgs: 129 active+clean; 781 MiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 2.8 MiB/s wr, 411 op/s 2024-09-11T00:30:10.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:30:09 smithi178 ceph-mon[35919]: pgmap v1092: 129 pgs: 129 active+clean; 781 MiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 2.8 MiB/s wr, 411 op/s 2024-09-11T00:30:10.277 INFO:tasks.workunit.client.0.smithi012.stdout: 10 170847 15.28 MB/sec execute 486 sec latency 2247.506 ms 2024-09-11T00:30:11.277 INFO:tasks.workunit.client.0.smithi012.stdout: 10 170847 15.25 MB/sec execute 487 sec latency 3247.591 ms 2024-09-11T00:30:12.063 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:30:11 smithi012 ceph-mon[30333]: pgmap v1093: 129 pgs: 129 active+clean; 787 MiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 6.1 MiB/s wr, 564 op/s 2024-09-11T00:30:12.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:30:11 smithi178 ceph-mon[35919]: pgmap v1093: 129 pgs: 129 active+clean; 787 MiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 6.1 MiB/s wr, 564 op/s 2024-09-11T00:30:12.277 INFO:tasks.workunit.client.0.smithi012.stdout: 10 170847 15.22 MB/sec execute 488 sec latency 4247.635 ms 2024-09-11T00:30:12.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:30:11 smithi165 ceph-mon[35356]: pgmap v1093: 129 pgs: 129 active+clean; 787 MiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 6.1 MiB/s wr, 564 op/s 2024-09-11T00:30:13.278 INFO:tasks.workunit.client.0.smithi012.stdout: 10 170998 15.20 MB/sec execute 489 sec latency 5007.830 ms 2024-09-11T00:30:14.063 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:30:13 smithi012 ceph-mon[30333]: pgmap v1094: 129 pgs: 129 active+clean; 788 MiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 7.8 MiB/s wr, 605 op/s 2024-09-11T00:30:14.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:30:13 smithi178 ceph-mon[35919]: pgmap v1094: 129 pgs: 129 active+clean; 788 MiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 7.8 MiB/s wr, 605 op/s 2024-09-11T00:30:14.278 INFO:tasks.workunit.client.0.smithi012.stdout: 10 171672 15.24 MB/sec execute 490 sec latency 234.010 ms 2024-09-11T00:30:14.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:30:13 smithi165 ceph-mon[35356]: pgmap v1094: 129 pgs: 129 active+clean; 788 MiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 7.8 MiB/s wr, 605 op/s 2024-09-11T00:30:15.278 INFO:tasks.workunit.client.0.smithi012.stdout: 10 172272 15.27 MB/sec execute 491 sec latency 214.708 ms 2024-09-11T00:30:16.063 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:30:15 smithi012 ceph-mon[30333]: pgmap v1095: 129 pgs: 129 active+clean; 787 MiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 7.4 MiB/s wr, 540 op/s 2024-09-11T00:30:16.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:30:15 smithi178 ceph-mon[35919]: pgmap v1095: 129 pgs: 129 active+clean; 787 MiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 7.4 MiB/s wr, 540 op/s 2024-09-11T00:30:16.277 INFO:tasks.workunit.client.0.smithi012.stdout: 10 173248 15.36 MB/sec execute 492 sec latency 51.650 ms 2024-09-11T00:30:16.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:30:15 smithi165 ceph-mon[35356]: pgmap v1095: 129 pgs: 129 active+clean; 787 MiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 7.4 MiB/s wr, 540 op/s 2024-09-11T00:30:17.278 INFO:tasks.workunit.client.0.smithi012.stdout: 10 173992 15.41 MB/sec execute 493 sec latency 86.212 ms 2024-09-11T00:30:18.063 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:30:17 smithi012 ceph-mon[30333]: pgmap v1096: 129 pgs: 129 active+clean; 785 MiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 8.2 MiB/s wr, 671 op/s 2024-09-11T00:30:18.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:30:17 smithi178 ceph-mon[35919]: pgmap v1096: 129 pgs: 129 active+clean; 785 MiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 8.2 MiB/s wr, 671 op/s 2024-09-11T00:30:18.278 INFO:tasks.workunit.client.0.smithi012.stdout: 10 174715 15.47 MB/sec execute 494 sec latency 59.338 ms 2024-09-11T00:30:18.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:30:17 smithi165 ceph-mon[35356]: pgmap v1096: 129 pgs: 129 active+clean; 785 MiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 8.2 MiB/s wr, 671 op/s 2024-09-11T00:30:19.278 INFO:tasks.workunit.client.0.smithi012.stdout: 10 175318 15.48 MB/sec execute 495 sec latency 145.789 ms 2024-09-11T00:30:20.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:30:19 smithi178 ceph-mon[35919]: pgmap v1097: 129 pgs: 129 active+clean; 785 MiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 6.7 MiB/s wr, 445 op/s 2024-09-11T00:30:20.278 INFO:tasks.workunit.client.0.smithi012.stdout: 10 176033 15.53 MB/sec execute 496 sec latency 76.922 ms 2024-09-11T00:30:20.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:30:19 smithi165 ceph-mon[35356]: pgmap v1097: 129 pgs: 129 active+clean; 785 MiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 6.7 MiB/s wr, 445 op/s 2024-09-11T00:30:20.317 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:30:19 smithi012 ceph-mon[30333]: pgmap v1097: 129 pgs: 129 active+clean; 785 MiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 6.7 MiB/s wr, 445 op/s 2024-09-11T00:30:21.289 INFO:tasks.workunit.client.0.smithi012.stdout: 10 176753 15.57 MB/sec execute 497 sec latency 105.853 ms 2024-09-11T00:30:22.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:30:21 smithi178 ceph-mon[35919]: pgmap v1098: 129 pgs: 129 active+clean; 790 MiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 7.2 MiB/s wr, 535 op/s 2024-09-11T00:30:22.084 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:30:21 smithi165 ceph-mon[35356]: pgmap v1098: 129 pgs: 129 active+clean; 790 MiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 7.2 MiB/s wr, 535 op/s 2024-09-11T00:30:22.289 INFO:tasks.workunit.client.0.smithi012.stdout: 10 177527 15.64 MB/sec execute 498 sec latency 281.190 ms 2024-09-11T00:30:22.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:30:21 smithi012 ceph-mon[30333]: pgmap v1098: 129 pgs: 129 active+clean; 790 MiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 7.2 MiB/s wr, 535 op/s 2024-09-11T00:30:22.476 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:22.474+0000 7f4001e00640 1 -- 172.21.15.12:0/2386028798 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3ffc1049d0 msgr2=0x7f3ffc104dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:30:22.477 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:22.474+0000 7f4001e00640 1 --2- 172.21.15.12:0/2386028798 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3ffc1049d0 0x7f3ffc104dd0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f3ff001a910 tx=0x7f3ff003b450 comp rx=0 tx=0).stop 2024-09-11T00:30:22.477 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:22.474+0000 7f4001e00640 1 -- 172.21.15.12:0/2386028798 shutdown_connections 2024-09-11T00:30:22.477 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:22.474+0000 7f4001e00640 1 --2- 172.21.15.12:0/2386028798 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3ffc1064e0 0x7f3ffc10ce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:30:22.477 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:22.474+0000 7f4001e00640 1 --2- 172.21.15.12:0/2386028798 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3ffc105b20 0x7f3ffc105fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:30:22.477 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:22.474+0000 7f4001e00640 1 --2- 172.21.15.12:0/2386028798 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3ffc1049d0 0x7f3ffc104dd0 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:30:22.477 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:22.474+0000 7f4001e00640 1 -- 172.21.15.12:0/2386028798 >> 172.21.15.12:0/2386028798 conn(0x7f3ffc100270 msgr2=0x7f3ffc1026b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:30:22.477 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:22.474+0000 7f4001e00640 1 -- 172.21.15.12:0/2386028798 shutdown_connections 2024-09-11T00:30:22.477 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:22.474+0000 7f4001e00640 1 -- 172.21.15.12:0/2386028798 wait complete. 2024-09-11T00:30:22.478 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:22.475+0000 7f4001e00640 1 Processor -- start 2024-09-11T00:30:22.479 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:22.477+0000 7f4001e00640 1 -- start start 2024-09-11T00:30:22.479 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:22.477+0000 7f4001e00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3ffc1049d0 0x7f3ffc192ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:30:22.480 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:22.477+0000 7f4001e00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3ffc105b20 0x7f3ffc1930b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:30:22.480 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:22.477+0000 7f4001e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3ffc1064e0 0x7f3ffc199910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:30:22.480 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:22.477+0000 7f4001e00640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f3ffc06f6a0 con 0x7f3ffc1049d0 2024-09-11T00:30:22.480 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:22.477+0000 7f4001e00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f3ffc06f520 con 0x7f3ffc1064e0 2024-09-11T00:30:22.480 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:22.477+0000 7f4001e00640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f3ffc06f820 con 0x7f3ffc105b20 2024-09-11T00:30:22.480 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:22.478+0000 7f4001400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3ffc1064e0 0x7f3ffc199910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:30:22.481 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:22.478+0000 7f4000a00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3ffc1049d0 0x7f3ffc192ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:30:22.481 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:22.478+0000 7f4001400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3ffc1064e0 0x7f3ffc199910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.12:58794/0 (socket says 172.21.15.12:58794) 2024-09-11T00:30:22.481 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:22.478+0000 7f4001400640 1 -- 172.21.15.12:0/2937240763 learned_addr learned my addr 172.21.15.12:0/2937240763 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:30:22.481 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:22.478+0000 7f4001400640 1 -- 172.21.15.12:0/2937240763 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3ffc105b20 msgr2=0x7f3ffc1930b0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-11T00:30:22.481 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:22.478+0000 7f4001400640 1 --2- 172.21.15.12:0/2937240763 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3ffc105b20 0x7f3ffc1930b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:30:22.482 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:22.478+0000 7f4001400640 1 -- 172.21.15.12:0/2937240763 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3ffc1049d0 msgr2=0x7f3ffc192ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:30:22.482 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:22.478+0000 7f4001400640 1 --2- 172.21.15.12:0/2937240763 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3ffc1049d0 0x7f3ffc192ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:30:22.482 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:22.478+0000 7f4001400640 1 -- 172.21.15.12:0/2937240763 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ffc19a130 con 0x7f3ffc1064e0 2024-09-11T00:30:22.482 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:22.479+0000 7f4001400640 1 --2- 172.21.15.12:0/2937240763 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3ffc1064e0 0x7f3ffc199910 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f3ff400eb10 tx=0x7f3ff400c2d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:30:22.482 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:22.479+0000 7f3ff9600640 1 -- 172.21.15.12:0/2937240763 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ff4004b20 con 0x7f3ffc1064e0 2024-09-11T00:30:22.482 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:22.479+0000 7f4001e00640 1 -- 172.21.15.12:0/2937240763 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ffc19a420 con 0x7f3ffc1064e0 2024-09-11T00:30:22.483 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:22.480+0000 7f4001e00640 1 -- 172.21.15.12:0/2937240763 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ffc19a970 con 0x7f3ffc1064e0 2024-09-11T00:30:22.483 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:22.480+0000 7f3ff9600640 1 -- 172.21.15.12:0/2937240763 <== mon.2 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3ff4004cc0 con 0x7f3ffc1064e0 2024-09-11T00:30:22.483 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:22.480+0000 7f3ff9600640 1 -- 172.21.15.12:0/2937240763 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ff4010670 con 0x7f3ffc1064e0 2024-09-11T00:30:22.489 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:22.487+0000 7f40061df740 1 Processor -- start 2024-09-11T00:30:22.490 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:22.488+0000 7f3ff9600640 1 -- 172.21.15.12:0/2937240763 <== mon.2 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7f3ff4005020 con 0x7f3ffc1064e0 2024-09-11T00:30:22.491 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:22.488+0000 7f40061df740 1 -- start start 2024-09-11T00:30:22.492 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:22.489+0000 7f40061df740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5614875b5970 0x5614875b7d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:30:22.492 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:22.489+0000 7f40061df740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5614875b82a0 0x5614875ba690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:30:22.492 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:22.489+0000 7f40061df740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5614875bace0 0x5614875bd1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:30:22.492 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:22.489+0000 7f40061df740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x5614875286b0 con 0x5614875b82a0 2024-09-11T00:30:22.492 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:22.489+0000 7f40061df740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x561487517ae0 con 0x5614875b5970 2024-09-11T00:30:22.492 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:22.489+0000 7f40061df740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x56148751b8f0 con 0x5614875bace0 2024-09-11T00:30:22.492 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:22.489+0000 7f3ffbe00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5614875b5970 0x5614875b7d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:30:22.492 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:22.489+0000 7f3ffbe00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5614875b5970 0x5614875b7d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.12:58806/0 (socket says 172.21.15.12:58806) 2024-09-11T00:30:22.492 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:22.489+0000 7f3ffbe00640 1 -- 172.21.15.12:0/3867359898 learned_addr learned my addr 172.21.15.12:0/3867359898 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:30:22.492 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:22.489+0000 7f3ffbe00640 1 --2- 172.21.15.12:0/3867359898 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5614875bace0 0x5614875bd1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:30:22.493 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:22.490+0000 7f4000a00640 1 --2- 172.21.15.12:0/3867359898 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5614875b82a0 0x5614875ba690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:30:22.493 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:22.490+0000 7f3ffbe00640 1 -- 172.21.15.12:0/3867359898 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5614875bace0 msgr2=0x5614875bd1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:30:22.493 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:22.490+0000 7f3ffbe00640 1 --2- 172.21.15.12:0/3867359898 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5614875bace0 0x5614875bd1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:30:22.493 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:22.490+0000 7f3ffbe00640 1 -- 172.21.15.12:0/3867359898 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5614875b82a0 msgr2=0x5614875ba690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:30:22.493 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:22.490+0000 7f3ffbe00640 1 --2- 172.21.15.12:0/3867359898 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5614875b82a0 0x5614875ba690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:30:22.494 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:22.490+0000 7f3ffbe00640 1 -- 172.21.15.12:0/3867359898 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5614874e4780 con 0x5614875b5970 2024-09-11T00:30:22.494 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:22.491+0000 7f3ffbe00640 1 --2- 172.21.15.12:0/3867359898 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5614875b5970 0x5614875b7d60 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f3fe8013820 tx=0x7f3fe8009660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8730864de8c312c3 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:30:22.494 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:22.491+0000 7f3fedc00640 1 -- 172.21.15.12:0/3867359898 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fe8013e60 con 0x5614875b5970 2024-09-11T00:30:22.494 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:22.492+0000 7f3fedc00640 1 -- 172.21.15.12:0/3867359898 <== mon.2 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3fe8013310 con 0x5614875b5970 2024-09-11T00:30:22.494 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:22.492+0000 7f3fedc00640 1 -- 172.21.15.12:0/3867359898 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fe80047a0 con 0x5614875b5970 2024-09-11T00:30:22.496 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:22.493+0000 7f40061df740 1 -- 172.21.15.12:0/3867359898 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5614875b5970 msgr2=0x5614875b7d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:30:22.496 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:22.493+0000 7f40061df740 1 --2- 172.21.15.12:0/3867359898 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5614875b5970 0x5614875b7d60 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f3fe8013820 tx=0x7f3fe8009660 comp rx=0 tx=0).stop 2024-09-11T00:30:22.496 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:22.494+0000 7f40061df740 1 -- 172.21.15.12:0/3867359898 shutdown_connections 2024-09-11T00:30:22.497 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:22.494+0000 7f40061df740 1 --2- 172.21.15.12:0/3867359898 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5614875bace0 0x5614875bd1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:30:22.497 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:22.494+0000 7f40061df740 1 --2- 172.21.15.12:0/3867359898 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5614875b82a0 0x5614875ba690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:30:22.497 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:22.494+0000 7f40061df740 1 --2- 172.21.15.12:0/3867359898 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5614875b5970 0x5614875b7d60 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:30:22.497 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:22.494+0000 7f40061df740 1 -- 172.21.15.12:0/3867359898 >> 172.21.15.12:0/3867359898 conn(0x5614875b0ac0 msgr2=0x5614875b1850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:30:22.498 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:22.495+0000 7f3ff9600640 1 --2- 172.21.15.12:0/2937240763 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f3fe40b5320 0x7f3fe40b77e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:30:22.498 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:22.496+0000 7f3ff9600640 1 -- 172.21.15.12:0/2937240763 <== mon.2 v2:172.21.15.165:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9208+0+0 (secure 0 0 0) 0x7f3ff40e6500 con 0x7f3ffc1064e0 2024-09-11T00:30:22.499 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:22.497+0000 7f40061df740 1 -- 172.21.15.12:0/3867359898 shutdown_connections 2024-09-11T00:30:22.499 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:22.497+0000 7f40061df740 1 -- 172.21.15.12:0/3867359898 wait complete. 2024-09-11T00:30:22.501 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:22.497+0000 7f40061df740 1 -- start start 2024-09-11T00:30:22.501 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:22.498+0000 7f40061df740 1 Processor -- start 2024-09-11T00:30:22.502 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:22.499+0000 7f40061df740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5614875b5970 0x5614875c7980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:30:22.502 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:22.499+0000 7f40061df740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5614875b82a0 0x5614875c8020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:30:22.502 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:22.499+0000 7f40061df740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5614875bace0 0x5614875c8bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:30:22.502 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:22.499+0000 7f40061df740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x56148744fab0 con 0x5614875b5970 2024-09-11T00:30:22.502 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:22.499+0000 7f40061df740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x5614874be570 con 0x5614875b82a0 2024-09-11T00:30:22.502 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:22.499+0000 7f40061df740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5614875bdcc0 con 0x5614875bace0 2024-09-11T00:30:22.502 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:22.500+0000 7f40061df740 10 client.0 _ll_register_callbacks cb 0x5614874cd100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:30:22.503 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:22.500+0000 7f4000a00640 1 --2- 172.21.15.12:0/2937240763 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f3fe40b5320 0x7f3fe40b77e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:30:22.503 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:22.500+0000 7f3ffbe00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5614875b82a0 0x5614875c8020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:30:22.503 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:22.500+0000 7f3ffbe00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5614875b82a0 0x5614875c8020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.12:58812/0 (socket says 172.21.15.12:58812) 2024-09-11T00:30:22.504 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:22.500+0000 7f3ffbe00640 1 -- 172.21.15.12:0/3185155441 learned_addr learned my addr 172.21.15.12:0/3185155441 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:30:22.504 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:22.501+0000 7f4001400640 1 --2- 172.21.15.12:0/3185155441 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5614875bace0 0x5614875c8bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:30:22.504 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:22.501+0000 7f4000a00640 1 --2- 172.21.15.12:0/3185155441 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5614875b5970 0x5614875c7980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:30:22.504 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:22.501+0000 7f3ffbe00640 1 -- 172.21.15.12:0/3185155441 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5614875bace0 msgr2=0x5614875c8bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:30:22.504 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:22.501+0000 7f3ffbe00640 1 --2- 172.21.15.12:0/3185155441 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5614875bace0 0x5614875c8bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:30:22.504 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:22.501+0000 7f3ffbe00640 1 -- 172.21.15.12:0/3185155441 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5614875b5970 msgr2=0x5614875c7980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:30:22.504 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:22.501+0000 7f3ffbe00640 1 --2- 172.21.15.12:0/3185155441 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5614875b5970 0x5614875c7980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:30:22.504 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:22.501+0000 7f3ffbe00640 1 -- 172.21.15.12:0/3185155441 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561487279790 con 0x5614875b82a0 2024-09-11T00:30:22.504 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:22.501+0000 7f4000a00640 1 --2- 172.21.15.12:0/3185155441 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5614875b5970 0x5614875c7980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T00:30:22.505 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:22.503+0000 7f3ffbe00640 1 --2- 172.21.15.12:0/3185155441 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5614875b82a0 0x5614875c8020 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f3fe8005720 tx=0x7f3fe8005750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3a8c23f8a2eb9055 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:30:22.505 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:22.503+0000 7f3fe2000640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-09-11T00:30:22.506 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:22.503+0000 7f4000a00640 1 --2- 172.21.15.12:0/2937240763 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f3fe40b5320 0x7f3fe40b77e0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f3ff0000c00 tx=0x7f3ff001a4e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:30:22.506 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:22.503+0000 7f3fe2000640 1 -- 172.21.15.12:0/3185155441 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fe8013e60 con 0x5614875b82a0 2024-09-11T00:30:22.506 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:22.503+0000 7f3fe2000640 1 -- 172.21.15.12:0/3185155441 <== mon.2 v2:172.21.15.165:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 9208+0+0 (secure 0 0 0) 0x7f3fe800e960 con 0x5614875b82a0 2024-09-11T00:30:22.506 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:22.504+0000 7f40061df740 1 -- 172.21.15.12:0/3185155441 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3fe8013310 con 0x5614875b82a0 2024-09-11T00:30:22.507 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:22.504+0000 7f3fe2000640 1 -- 172.21.15.12:0/3185155441 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=102) ==== 24+0+0 (secure 0 0 0) 0x7f3fe8030050 con 0x5614875b82a0 2024-09-11T00:30:22.507 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:22.504+0000 7f40061df740 10 client.27974 fetch_fsmap learned FSMap version 102 2024-09-11T00:30:22.507 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:22.504+0000 7f40061df740 1 -- 172.21.15.12:0/3185155441 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=102,monmap=4+}) -- 0x5614872f2620 con 0x5614875b82a0 2024-09-11T00:30:22.507 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:22.505+0000 7f3fe2000640 1 -- 172.21.15.12:0/3185155441 <== mon.2 v2:172.21.15.165:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3fe8013310 con 0x5614875b82a0 2024-09-11T00:30:22.509 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:22.506+0000 7f3fe2000640 1 -- 172.21.15.12:0/3185155441 <== mon.2 v2:172.21.15.165:3300/0 5 ==== fsmap(e 102) ==== 8378+0+0 (secure 0 0 0) 0x7f3fe80039a0 con 0x5614875b82a0 2024-09-11T00:30:22.509 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:22.507+0000 7f40061df740 10 client.27974 fetch_fsmap finished waiting for FSMap version 102 2024-09-11T00:30:22.509 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:22.507+0000 7f40061df740 10 client.27974 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T00:30:22.509 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:22.507+0000 7f40061df740 20 client.27974 populate_metadata read hostname 'smithi012' 2024-09-11T00:30:22.509 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:22.507+0000 7f40061df740 1 --2- 172.21.15.12:0/3185155441 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x5614875d2fa0 0x5614875d5390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:30:22.509 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:22.507+0000 7f40061df740 4 client.27974 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T00:30:22.510 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:22.507+0000 7f40061df740 1 -- 172.21.15.12:0/3185155441 --> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56148752a600 con 0x5614875d2fa0 2024-09-11T00:30:22.510 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:22.507+0000 7f4000a00640 1 --2- 172.21.15.12:0/3185155441 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x5614875d2fa0 0x5614875d5390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:30:22.510 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:22.508+0000 7f3fe2000640 10 client.27974 ms_handle_connect on v2:172.21.15.165:6838/501312984 2024-09-11T00:30:22.511 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:22.508+0000 7f4000a00640 1 --2- 172.21.15.12:0/3185155441 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x5614875d2fa0 0x5614875d5390 crc :-1 s=READY pgs=315 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1903a6b2195ba0c3 server_cookie=38ffc5b35c1c89c in_seq=0 out_seq=0 2024-09-11T00:30:22.513 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:22.511+0000 7f3fe2000640 1 -- 172.21.15.12:0/3185155441 <== mds.0 v2:172.21.15.165:6838/501312984 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x56148752a600 con 0x5614875d2fa0 2024-09-11T00:30:22.513 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:22.511+0000 7f3fe2000640 10 client.27974 handle_command_reply: tid=0 2024-09-11T00:30:22.513 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:22.511+0000 7f40061df740 1 client.27974 shutdown 2024-09-11T00:30:22.514 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:22.511+0000 7f40061df740 1 -- 172.21.15.12:0/3185155441 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5614875b82a0 msgr2=0x5614875c8020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:30:22.514 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:22.511+0000 7f40061df740 1 --2- 172.21.15.12:0/3185155441 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5614875b82a0 0x5614875c8020 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f3fe8005720 tx=0x7f3fe8005750 comp rx=0 tx=0).stop 2024-09-11T00:30:22.514 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:22.512+0000 7f40061df740 1 -- 172.21.15.12:0/3185155441 shutdown_connections 2024-09-11T00:30:22.514 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:22.512+0000 7f40061df740 1 --2- 172.21.15.12:0/3185155441 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x5614875d2fa0 0x5614875d5390 crc :-1 s=READY pgs=315 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:30:22.514 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:22.512+0000 7f40061df740 1 --2- 172.21.15.12:0/3185155441 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5614875bace0 0x5614875c8bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:30:22.514 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:22.512+0000 7f40061df740 1 --2- 172.21.15.12:0/3185155441 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5614875b82a0 0x5614875c8020 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:30:22.514 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:22.512+0000 7f40061df740 1 --2- 172.21.15.12:0/3185155441 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5614875b5970 0x5614875c7980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:30:22.514 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:22.512+0000 7f3fe2000640 0 client.27974 ms_handle_reset on v2:172.21.15.165:6838/501312984 2024-09-11T00:30:22.515 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:22.512+0000 7f40061df740 1 -- 172.21.15.12:0/3185155441 >> 172.21.15.12:0/3185155441 conn(0x5614875b0ac0 msgr2=0x5614875afbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:30:22.518 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:22.515+0000 7f40061df740 1 -- 172.21.15.12:0/3185155441 shutdown_connections 2024-09-11T00:30:22.519 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:22.516+0000 7f40061df740 1 -- 172.21.15.12:0/3185155441 wait complete. 2024-09-11T00:30:22.519 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:22.516+0000 7f40061df740 20 client.27974 trim_cache size 0 max 16384 2024-09-11T00:30:22.552 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:22.549+0000 7f40061df740 1 Processor -- start 2024-09-11T00:30:22.552 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:22.549+0000 7f40061df740 1 -- start start 2024-09-11T00:30:22.552 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:22.550+0000 7f40061df740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5614875b5970 0x5614875b4ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:30:22.552 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:22.550+0000 7f40061df740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5614875b82a0 0x5614875b5430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:30:22.552 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:22.550+0000 7f40061df740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5614875bace0 0x5614875c6b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:30:22.553 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:22.550+0000 7f40061df740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f3fdc00fb30 con 0x5614875bace0 2024-09-11T00:30:22.553 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:22.550+0000 7f40061df740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f3fdc0116d0 con 0x5614875b5970 2024-09-11T00:30:22.553 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:22.550+0000 7f40061df740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f3fdc010260 con 0x5614875b82a0 2024-09-11T00:30:22.554 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:22.550+0000 7f4000a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5614875b5970 0x5614875b4ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:30:22.554 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:22.550+0000 7f4000a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5614875b5970 0x5614875b4ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.12:58828/0 (socket says 172.21.15.12:58828) 2024-09-11T00:30:22.554 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:22.550+0000 7f4000a00640 1 -- 172.21.15.12:0/2085748184 learned_addr learned my addr 172.21.15.12:0/2085748184 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:30:22.554 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:22.551+0000 7f4001400640 1 --2- 172.21.15.12:0/2085748184 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5614875bace0 0x5614875c6b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:30:22.554 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:22.551+0000 7f4000a00640 1 -- 172.21.15.12:0/2085748184 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5614875b82a0 msgr2=0x5614875b5430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:30:22.554 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:22.551+0000 7f3ffbe00640 1 --2- 172.21.15.12:0/2085748184 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5614875b82a0 0x5614875b5430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:30:22.554 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:22.551+0000 7f4000a00640 1 --2- 172.21.15.12:0/2085748184 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5614875b82a0 0x5614875b5430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:30:22.554 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:22.551+0000 7f4000a00640 1 -- 172.21.15.12:0/2085748184 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5614875bace0 msgr2=0x5614875c6b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:30:22.554 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:22.551+0000 7f4000a00640 1 --2- 172.21.15.12:0/2085748184 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5614875bace0 0x5614875c6b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:30:22.554 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:22.551+0000 7f4000a00640 1 -- 172.21.15.12:0/2085748184 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5614875c7cf0 con 0x5614875b5970 2024-09-11T00:30:22.554 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:22.551+0000 7f4001400640 1 --2- 172.21.15.12:0/2085748184 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5614875bace0 0x5614875c6b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T00:30:22.554 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:22.551+0000 7f4000a00640 1 --2- 172.21.15.12:0/2085748184 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5614875b5970 0x5614875b4ef0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f3ff001a910 tx=0x7f3ff004c000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=da775fd13c8d57a4 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:30:22.555 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:22.552+0000 7f3fed200640 1 -- 172.21.15.12:0/2085748184 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ff006a070 con 0x5614875b5970 2024-09-11T00:30:22.555 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:22.552+0000 7f3fed200640 1 -- 172.21.15.12:0/2085748184 <== mon.2 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x56148752a600 con 0x5614875b5970 2024-09-11T00:30:22.555 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:22.552+0000 7f3fed200640 1 -- 172.21.15.12:0/2085748184 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ff004f660 con 0x5614875b5970 2024-09-11T00:30:22.556 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:22.553+0000 7f40061df740 1 -- 172.21.15.12:0/2085748184 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5614875b5970 msgr2=0x5614875b4ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:30:22.556 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:22.553+0000 7f40061df740 1 --2- 172.21.15.12:0/2085748184 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5614875b5970 0x5614875b4ef0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f3ff001a910 tx=0x7f3ff004c000 comp rx=0 tx=0).stop 2024-09-11T00:30:22.556 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:22.553+0000 7f40061df740 1 -- 172.21.15.12:0/2085748184 shutdown_connections 2024-09-11T00:30:22.556 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:22.553+0000 7f40061df740 1 --2- 172.21.15.12:0/2085748184 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5614875bace0 0x5614875c6b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:30:22.556 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:22.553+0000 7f40061df740 1 --2- 172.21.15.12:0/2085748184 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5614875b82a0 0x5614875b5430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:30:22.556 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:22.553+0000 7f40061df740 1 --2- 172.21.15.12:0/2085748184 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5614875b5970 0x5614875b4ef0 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:30:22.556 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:22.553+0000 7f40061df740 1 -- 172.21.15.12:0/2085748184 >> 172.21.15.12:0/2085748184 conn(0x5614875b0ac0 msgr2=0x5614875abac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:30:22.556 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:22.553+0000 7f40061df740 1 -- 172.21.15.12:0/2085748184 shutdown_connections 2024-09-11T00:30:22.556 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:22.554+0000 7f40061df740 1 -- 172.21.15.12:0/2085748184 wait complete. 2024-09-11T00:30:22.557 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:22.554+0000 7f40061df740 1 -- start start 2024-09-11T00:30:22.557 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:22.554+0000 7f40061df740 1 Processor -- start 2024-09-11T00:30:22.557 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:22.555+0000 7f40061df740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5614875b5970 0x5614875cf3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:30:22.557 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:22.555+0000 7f40061df740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5614875b82a0 0x5614875ce0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:30:22.557 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:22.555+0000 7f40061df740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5614875bace0 0x5614875ce5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:30:22.558 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:22.555+0000 7f4000a00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5614875b5970 0x5614875cf3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:30:22.558 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:22.555+0000 7f40061df740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f3fdc011000 con 0x5614875b82a0 2024-09-11T00:30:22.558 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:22.555+0000 7f40061df740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f3fdc010930 con 0x5614875bace0 2024-09-11T00:30:22.559 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:22.555+0000 7f40061df740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f3fdc00dbc0 con 0x5614875b5970 2024-09-11T00:30:22.559 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:22.555+0000 7f40061df740 10 client.0 _ll_register_callbacks cb 0x7f3fe8009da0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:30:22.559 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:22.555+0000 7f4000a00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5614875b5970 0x5614875cf3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.12:39342/0 (socket says 172.21.15.12:39342) 2024-09-11T00:30:22.559 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:22.555+0000 7f4000a00640 1 -- 172.21.15.12:0/70807325 learned_addr learned my addr 172.21.15.12:0/70807325 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:30:22.559 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:22.555+0000 7f3ffbe00640 1 --2- 172.21.15.12:0/70807325 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5614875b82a0 0x5614875ce0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:30:22.559 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:22.556+0000 7f4001400640 1 --2- 172.21.15.12:0/70807325 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5614875bace0 0x5614875ce5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:30:22.559 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:22.556+0000 7f4000a00640 1 -- 172.21.15.12:0/70807325 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5614875bace0 msgr2=0x5614875ce5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:30:22.559 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:22.556+0000 7f4000a00640 1 --2- 172.21.15.12:0/70807325 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5614875bace0 0x5614875ce5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:30:22.559 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:22.556+0000 7f4000a00640 1 -- 172.21.15.12:0/70807325 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5614875b82a0 msgr2=0x5614875ce0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:30:22.559 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:22.556+0000 7f4000a00640 1 --2- 172.21.15.12:0/70807325 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5614875b82a0 0x5614875ce0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:30:22.559 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:22.556+0000 7f4000a00640 1 -- 172.21.15.12:0/70807325 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5614875ced50 con 0x5614875b5970 2024-09-11T00:30:22.560 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:22.556+0000 7f4000a00640 1 --2- 172.21.15.12:0/70807325 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5614875b5970 0x5614875cf3a0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f3ff003b490 tx=0x7f3ff0040fa0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=39bc588b11c177b3 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:30:22.560 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:22.556+0000 7f3fe2000640 10 client.0 ms_handle_connect on v2:172.21.15.178:3300/0 2024-09-11T00:30:22.560 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:22.557+0000 7f3fe2000640 1 -- 172.21.15.12:0/70807325 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ff006a070 con 0x5614875b5970 2024-09-11T00:30:22.560 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:22.557+0000 7f3fe2000640 1 -- 172.21.15.12:0/70807325 <== mon.1 v2:172.21.15.178:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 9208+0+0 (secure 0 0 0) 0x7f3ff0040510 con 0x5614875b5970 2024-09-11T00:30:22.560 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:22.557+0000 7f40061df740 1 -- 172.21.15.12:0/70807325 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x56148752a600 con 0x5614875b5970 2024-09-11T00:30:22.561 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:22.558+0000 7f3fe2000640 1 -- 172.21.15.12:0/70807325 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=102) ==== 24+0+0 (secure 0 0 0) 0x7f3ff0078050 con 0x5614875b5970 2024-09-11T00:30:22.561 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:22.558+0000 7f40061df740 10 client.27952 fetch_fsmap learned FSMap version 102 2024-09-11T00:30:22.561 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:22.558+0000 7f40061df740 1 -- 172.21.15.12:0/70807325 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,fsmap=102,monmap=4+}) -- 0x56148763fe50 con 0x5614875b5970 2024-09-11T00:30:22.561 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:22.558+0000 7f3fe2000640 1 -- 172.21.15.12:0/70807325 <== mon.1 v2:172.21.15.178:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x56148752a600 con 0x5614875b5970 2024-09-11T00:30:22.561 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:22.558+0000 7f3fe2000640 1 -- 172.21.15.12:0/70807325 <== mon.1 v2:172.21.15.178:3300/0 5 ==== fsmap(e 102) ==== 8378+0+0 (secure 0 0 0) 0x7f3ff004b040 con 0x5614875b5970 2024-09-11T00:30:22.562 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:22.559+0000 7f40061df740 10 client.27952 fetch_fsmap finished waiting for FSMap version 102 2024-09-11T00:30:22.562 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:22.559+0000 7f40061df740 10 client.27952 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T00:30:22.562 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:22.559+0000 7f40061df740 20 client.27952 populate_metadata read hostname 'smithi012' 2024-09-11T00:30:22.562 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:22.559+0000 7f40061df740 1 --2- 172.21.15.12:0/70807325 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x5614876406f0 0x561487642b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:30:22.562 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:22.559+0000 7f40061df740 4 client.27952 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T00:30:22.562 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:22.559+0000 7f40061df740 1 -- 172.21.15.12:0/70807325 --> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5614875260b0 con 0x5614876406f0 2024-09-11T00:30:22.562 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:22.560+0000 7f3ffbe00640 1 --2- 172.21.15.12:0/70807325 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x5614876406f0 0x561487642b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:30:22.563 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:22.561+0000 7f3ffbe00640 1 --2- 172.21.15.12:0/70807325 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x5614876406f0 0x561487642b10 crc :-1 s=READY pgs=316 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6ce9954d82ed88dd server_cookie=fab3fd7712befcb1 in_seq=0 out_seq=0 2024-09-11T00:30:22.563 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:22.561+0000 7f3fe2000640 10 client.27952 ms_handle_connect on v2:172.21.15.165:6838/501312984 2024-09-11T00:30:22.921 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:22.919+0000 7f3fe2000640 1 -- 172.21.15.12:0/70807325 <== mds.0 v2:172.21.15.165:6838/501312984 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5614875260b0 con 0x5614876406f0 2024-09-11T00:30:22.922 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:22.919+0000 7f3fe2000640 10 client.27952 handle_command_reply: tid=0 2024-09-11T00:30:22.922 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:22.919+0000 7f40061df740 1 client.27952 shutdown 2024-09-11T00:30:22.922 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:22.919+0000 7f40061df740 1 -- 172.21.15.12:0/70807325 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5614875b5970 msgr2=0x5614875cf3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:30:22.922 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:22.919+0000 7f40061df740 1 --2- 172.21.15.12:0/70807325 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5614875b5970 0x5614875cf3a0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f3ff003b490 tx=0x7f3ff0040fa0 comp rx=0 tx=0).stop 2024-09-11T00:30:22.922 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:22.919+0000 7f40061df740 1 -- 172.21.15.12:0/70807325 shutdown_connections 2024-09-11T00:30:22.922 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:22.919+0000 7f40061df740 1 --2- 172.21.15.12:0/70807325 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x5614876406f0 0x561487642b10 crc :-1 s=READY pgs=316 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:30:22.922 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:22.919+0000 7f40061df740 1 --2- 172.21.15.12:0/70807325 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5614875bace0 0x5614875ce5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:30:22.922 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:22.919+0000 7f40061df740 1 --2- 172.21.15.12:0/70807325 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5614875b82a0 0x5614875ce0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:30:22.922 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:22.919+0000 7f3fe2000640 0 client.27952 ms_handle_reset on v2:172.21.15.165:6838/501312984 2024-09-11T00:30:22.922 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:22.919+0000 7f40061df740 1 --2- 172.21.15.12:0/70807325 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5614875b5970 0x5614875cf3a0 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:30:22.923 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:22.919+0000 7f40061df740 1 -- 172.21.15.12:0/70807325 >> 172.21.15.12:0/70807325 conn(0x5614875b0ac0 msgr2=0x5614875b4e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:30:22.923 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:22.920+0000 7f40061df740 1 -- 172.21.15.12:0/70807325 shutdown_connections 2024-09-11T00:30:22.923 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:22.920+0000 7f40061df740 1 -- 172.21.15.12:0/70807325 wait complete. 2024-09-11T00:30:22.923 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:22.920+0000 7f40061df740 20 client.27952 trim_cache size 0 max 16384 2024-09-11T00:30:22.924 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:{ 2024-09-11T00:30:22.924 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "message": "", 2024-09-11T00:30:22.924 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "return_code": 0 2024-09-11T00:30:22.924 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:} 2024-09-11T00:30:22.925 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:22.921+0000 7f4001e00640 1 -- 172.21.15.12:0/2937240763 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f3fe40b5320 msgr2=0x7f3fe40b77e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:30:22.925 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:22.921+0000 7f4001e00640 1 --2- 172.21.15.12:0/2937240763 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f3fe40b5320 0x7f3fe40b77e0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f3ff0000c00 tx=0x7f3ff001a4e0 comp rx=0 tx=0).stop 2024-09-11T00:30:22.925 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:22.921+0000 7f4001e00640 1 -- 172.21.15.12:0/2937240763 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3ffc1064e0 msgr2=0x7f3ffc199910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:30:22.925 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:22.921+0000 7f4001e00640 1 --2- 172.21.15.12:0/2937240763 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3ffc1064e0 0x7f3ffc199910 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f3ff400eb10 tx=0x7f3ff400c2d0 comp rx=0 tx=0).stop 2024-09-11T00:30:22.925 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:22.921+0000 7f4001e00640 1 -- 172.21.15.12:0/2937240763 shutdown_connections 2024-09-11T00:30:22.925 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:22.921+0000 7f4001e00640 1 --2- 172.21.15.12:0/2937240763 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f3fe40b5320 0x7f3fe40b77e0 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:30:22.925 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:22.921+0000 7f4001e00640 1 --2- 172.21.15.12:0/2937240763 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3ffc1064e0 0x7f3ffc199910 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:30:22.925 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:22.921+0000 7f4001e00640 1 --2- 172.21.15.12:0/2937240763 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3ffc105b20 0x7f3ffc1930b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:30:22.925 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:22.921+0000 7f4001e00640 1 --2- 172.21.15.12:0/2937240763 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3ffc1049d0 0x7f3ffc192ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:30:22.925 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:22.921+0000 7f4001e00640 1 -- 172.21.15.12:0/2937240763 >> 172.21.15.12:0/2937240763 conn(0x7f3ffc100270 msgr2=0x7f3ffc0720b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:30:22.925 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:22.921+0000 7f4001e00640 1 -- 172.21.15.12:0/2937240763 shutdown_connections 2024-09-11T00:30:22.926 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:22.921+0000 7f4001e00640 1 -- 172.21.15.12:0/2937240763 wait complete. 2024-09-11T00:30:23.289 INFO:tasks.workunit.client.0.smithi012.stdout: 10 178215 15.69 MB/sec execute 499 sec latency 269.939 ms 2024-09-11T00:30:24.081 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:30:23 smithi178 ceph-mon[35919]: pgmap v1099: 129 pgs: 129 active+clean; 800 MiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 4.9 MiB/s wr, 536 op/s 2024-09-11T00:30:24.178 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:30:23 smithi012 ceph-mon[30333]: pgmap v1099: 129 pgs: 129 active+clean; 800 MiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 4.9 MiB/s wr, 536 op/s 2024-09-11T00:30:24.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:30:23 smithi165 ceph-mon[35356]: pgmap v1099: 129 pgs: 129 active+clean; 800 MiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 4.9 MiB/s wr, 536 op/s 2024-09-11T00:30:24.289 INFO:tasks.workunit.client.0.smithi012.stdout: 10 178618 15.69 MB/sec execute 500 sec latency 344.221 ms 2024-09-11T00:30:25.289 INFO:tasks.workunit.client.0.smithi012.stdout: 10 179333 15.74 MB/sec execute 501 sec latency 108.771 ms 2024-09-11T00:30:26.220 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:30:25 smithi012 ceph-mon[30333]: pgmap v1100: 129 pgs: 129 active+clean; 804 MiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 3.1 MiB/s wr, 505 op/s 2024-09-11T00:30:26.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:30:25 smithi165 ceph-mon[35356]: pgmap v1100: 129 pgs: 129 active+clean; 804 MiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 3.1 MiB/s wr, 505 op/s 2024-09-11T00:30:26.289 INFO:tasks.workunit.client.0.smithi012.stdout: 10 179947 15.76 MB/sec execute 502 sec latency 91.031 ms 2024-09-11T00:30:26.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:30:25 smithi178 ceph-mon[35919]: pgmap v1100: 129 pgs: 129 active+clean; 804 MiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 3.1 MiB/s wr, 505 op/s 2024-09-11T00:30:27.290 INFO:tasks.workunit.client.0.smithi012.stdout: 10 181003 15.86 MB/sec execute 503 sec latency 42.921 ms 2024-09-11T00:30:28.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:30:27 smithi165 ceph-mon[35356]: pgmap v1101: 129 pgs: 129 active+clean; 810 MiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 4.2 MiB/s wr, 705 op/s 2024-09-11T00:30:28.289 INFO:tasks.workunit.client.0.smithi012.stdout: 10 181819 15.91 MB/sec execute 504 sec latency 71.008 ms 2024-09-11T00:30:28.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:30:27 smithi012 ceph-mon[30333]: pgmap v1101: 129 pgs: 129 active+clean; 810 MiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 4.2 MiB/s wr, 705 op/s 2024-09-11T00:30:28.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:30:27 smithi178 ceph-mon[35919]: pgmap v1101: 129 pgs: 129 active+clean; 810 MiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 4.2 MiB/s wr, 705 op/s 2024-09-11T00:30:29.290 INFO:tasks.workunit.client.0.smithi012.stdout: 10 182353 15.94 MB/sec execute 505 sec latency 110.054 ms 2024-09-11T00:30:30.283 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:30:29 smithi165 ceph-mon[35356]: pgmap v1102: 129 pgs: 129 active+clean; 811 MiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 3.5 MiB/s wr, 575 op/s 2024-09-11T00:30:30.289 INFO:tasks.workunit.client.0.smithi012.stdout: 10 183011 15.97 MB/sec execute 506 sec latency 111.096 ms 2024-09-11T00:30:30.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:30:29 smithi012 ceph-mon[30333]: pgmap v1102: 129 pgs: 129 active+clean; 811 MiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 3.5 MiB/s wr, 575 op/s 2024-09-11T00:30:30.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:30:29 smithi178 ceph-mon[35919]: pgmap v1102: 129 pgs: 129 active+clean; 811 MiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 3.5 MiB/s wr, 575 op/s 2024-09-11T00:30:30.772 INFO:tasks.quiescer.fs.[cephfs]:Total duration (37.7) is longer than `split_if_longer` (35.0), will split into two consecutive quiesces 2024-09-11T00:30:30.772 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 18.85 2024-09-11T00:30:30.773 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid ff3ad74a-6fcf-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T00:30:31.290 INFO:tasks.workunit.client.0.smithi012.stdout: 10 183803 16.01 MB/sec execute 507 sec latency 77.667 ms 2024-09-11T00:30:31.391 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:30:32.200 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:30:31 smithi012 ceph-mon[30333]: pgmap v1103: 129 pgs: 129 active+clean; 818 MiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 MiB/s rd, 4.2 MiB/s wr, 684 op/s 2024-09-11T00:30:32.234 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:32.230+0000 7fc599c00640 1 -- 172.21.15.12:0/7768770 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc59410ede0 msgr2=0x7fc59410f220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:30:32.234 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:32.231+0000 7fc599c00640 1 --2- 172.21.15.12:0/7768770 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc59410ede0 0x7fc59410f220 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fc580009900 tx=0x7fc58002a3f0 comp rx=0 tx=0).stop 2024-09-11T00:30:32.234 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:32.232+0000 7fc599c00640 1 -- 172.21.15.12:0/7768770 shutdown_connections 2024-09-11T00:30:32.234 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:32.232+0000 7fc599c00640 1 --2- 172.21.15.12:0/7768770 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc59410f760 0x7fc594115f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:30:32.234 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:32.232+0000 7fc599c00640 1 --2- 172.21.15.12:0/7768770 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc59410ede0 0x7fc59410f220 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:30:32.235 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:32.232+0000 7fc599c00640 1 --2- 172.21.15.12:0/7768770 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc5941093d0 0x7fc5941097b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:30:32.235 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:32.232+0000 7fc599c00640 1 -- 172.21.15.12:0/7768770 >> 172.21.15.12:0/7768770 conn(0x7fc594078a40 msgr2=0x7fc59407ae60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:30:32.235 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:32.232+0000 7fc599c00640 1 -- 172.21.15.12:0/7768770 shutdown_connections 2024-09-11T00:30:32.236 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:32.232+0000 7fc599c00640 1 -- 172.21.15.12:0/7768770 wait complete. 2024-09-11T00:30:32.236 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:32.234+0000 7fc599c00640 1 Processor -- start 2024-09-11T00:30:32.237 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:32.234+0000 7fc599c00640 1 -- start start 2024-09-11T00:30:32.238 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:32.235+0000 7fc599c00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc5941093d0 0x7fc594083520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:30:32.238 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:32.236+0000 7fc599c00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc59410ede0 0x7fc594083a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:30:32.238 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:32.236+0000 7fc599c00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc59410f760 0x7fc59407c7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:30:32.239 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:32.237+0000 7fc599c00640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fc5941193b0 con 0x7fc59410ede0 2024-09-11T00:30:32.240 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:32.237+0000 7fc593400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc59410f760 0x7fc59407c7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:30:32.240 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:32.237+0000 7fc593400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc59410f760 0x7fc59407c7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.12:37918/0 (socket says 172.21.15.12:37918) 2024-09-11T00:30:32.240 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:32.237+0000 7fc593400640 1 -- 172.21.15.12:0/388442219 learned_addr learned my addr 172.21.15.12:0/388442219 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:30:32.240 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:32.237+0000 7fc599c00640 1 -- 172.21.15.12:0/388442219 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fc594119230 con 0x7fc59410f760 2024-09-11T00:30:32.240 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:32.237+0000 7fc599c00640 1 -- 172.21.15.12:0/388442219 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fc594119530 con 0x7fc5941093d0 2024-09-11T00:30:32.241 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:32.238+0000 7fc592000640 1 --2- 172.21.15.12:0/388442219 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc59410ede0 0x7fc594083a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:30:32.241 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:32.238+0000 7fc592a00640 1 --2- 172.21.15.12:0/388442219 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc5941093d0 0x7fc594083520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:30:32.241 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:32.239+0000 7fc593400640 1 -- 172.21.15.12:0/388442219 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc5941093d0 msgr2=0x7fc594083520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:30:32.241 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:32.239+0000 7fc593400640 1 --2- 172.21.15.12:0/388442219 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc5941093d0 0x7fc594083520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:30:32.241 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:32.239+0000 7fc593400640 1 -- 172.21.15.12:0/388442219 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc59410ede0 msgr2=0x7fc594083a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:30:32.241 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:32.239+0000 7fc593400640 1 --2- 172.21.15.12:0/388442219 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc59410ede0 0x7fc594083a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:30:32.242 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:32.239+0000 7fc593400640 1 -- 172.21.15.12:0/388442219 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc59407d0b0 con 0x7fc59410f760 2024-09-11T00:30:32.244 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:32.240+0000 7fc593400640 1 --2- 172.21.15.12:0/388442219 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc59410f760 0x7fc59407c7f0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fc58c00eb10 tx=0x7fc58c00c2d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:30:32.244 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:32.241+0000 7fc58b400640 1 -- 172.21.15.12:0/388442219 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc58c004b20 con 0x7fc59410f760 2024-09-11T00:30:32.244 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:32.241+0000 7fc58b400640 1 -- 172.21.15.12:0/388442219 <== mon.2 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc58c004cc0 con 0x7fc59410f760 2024-09-11T00:30:32.244 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:32.241+0000 7fc58b400640 1 -- 172.21.15.12:0/388442219 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc58c004b20 con 0x7fc59410f760 2024-09-11T00:30:32.244 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:32.241+0000 7fc599c00640 1 -- 172.21.15.12:0/388442219 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc59407d3a0 con 0x7fc59410f760 2024-09-11T00:30:32.244 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:32.241+0000 7fc592000640 1 --2- 172.21.15.12:0/388442219 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc59410ede0 0x7fc594083a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T00:30:32.244 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:32.241+0000 7fc599c00640 1 -- 172.21.15.12:0/388442219 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc594082360 con 0x7fc59410f760 2024-09-11T00:30:32.247 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:32.245+0000 7fc599c00640 1 -- 172.21.15.12:0/388442219 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc59410a680 con 0x7fc59410f760 2024-09-11T00:30:32.248 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:32.246+0000 7fc592a00640 1 --2- 172.21.15.12:0/388442219 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc5941093d0 0x7fc594083520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T00:30:32.251 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:32.248+0000 7fc58b400640 1 -- 172.21.15.12:0/388442219 <== mon.2 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7fc58c010470 con 0x7fc59410f760 2024-09-11T00:30:32.254 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:32.252+0000 7fc58b400640 1 --2- 172.21.15.12:0/388442219 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fc5780b5620 0x7fc5780b7ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:30:32.255 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:32.253+0000 7fc58b400640 1 -- 172.21.15.12:0/388442219 <== mon.2 v2:172.21.15.165:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9208+0+0 (secure 0 0 0) 0x7fc58c0e63c0 con 0x7fc59410f760 2024-09-11T00:30:32.258 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:32.255+0000 7fc592a00640 1 --2- 172.21.15.12:0/388442219 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fc5780b5620 0x7fc5780b7ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:30:32.259 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:32.256+0000 7fc592a00640 1 --2- 172.21.15.12:0/388442219 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fc5780b5620 0x7fc5780b7ae0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fc58400baa0 tx=0x7fc584005d90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:30:32.266 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:32.263+0000 7fc58b400640 1 -- 172.21.15.12:0/388442219 <== mon.2 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7fc58c0ace50 con 0x7fc59410f760 2024-09-11T00:30:32.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:30:31 smithi165 ceph-mon[35356]: pgmap v1103: 129 pgs: 129 active+clean; 818 MiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 MiB/s rd, 4.2 MiB/s wr, 684 op/s 2024-09-11T00:30:32.290 INFO:tasks.workunit.client.0.smithi012.stdout: 10 184506 16.06 MB/sec execute 508 sec latency 196.282 ms 2024-09-11T00:30:32.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:30:31 smithi178 ceph-mon[35919]: pgmap v1103: 129 pgs: 129 active+clean; 818 MiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 MiB/s rd, 4.2 MiB/s wr, 684 op/s 2024-09-11T00:30:32.837 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:32.834+0000 7fc599c00640 1 -- 172.21.15.12:0/388442219 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc5941097b0 con 0x7fc59410f760 2024-09-11T00:30:32.838 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:32.836+0000 7fc58b400640 1 -- 172.21.15.12:0/388442219 <== mon.2 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 102 v102) ==== 95+0+11973 (secure 0 0 0) 0x7fc58c0ac750 con 0x7fc59410f760 2024-09-11T00:30:32.845 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-11T00:30:32.846 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 102 2024-09-11T00:30:32.846 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":102,"btime":"2024-09-11T00:29:46:174935+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14577,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6835/447171591","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":447171591},{"type":"v1","addr":"172.21.15.12:6835","nonce":447171591}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14589,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/901486040","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":901486040},{"type":"v1","addr":"172.21.15.12:6837","nonce":901486040}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":102,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T00:04:04.739881+0000","modified":"2024-09-11T00:29:45.675612+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24674,"mds_1":24643,"mds_2":24638,"mds_3":24637,"mds_4":24614},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14631":{"gid":14631,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.12:6841/3069919437","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6840","nonce":3069919437},{"type":"v1","addr":"172.21.15.12:6841","nonce":3069919437}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24562":{"gid":24562,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6833/1141636112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":1141636112},{"type":"v1","addr":"172.21.15.178:6833","nonce":1141636112}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24581":{"gid":24581,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6833/3111980970","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6832","nonce":3111980970},{"type":"v1","addr":"172.21.15.165:6833","nonce":3111980970}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24607":{"gid":24607,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6835/4084868287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":4084868287},{"type":"v1","addr":"172.21.15.178:6835","nonce":4084868287}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24614":{"gid":24614,"name":"e","rank":4,"incarnation":26,"state":"up:active","state_seq":21,"addr":"172.21.15.165:6835/2425395274","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":2425395274},{"type":"v1","addr":"172.21.15.165:6835","nonce":2425395274}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24637":{"gid":24637,"name":"i","rank":3,"incarnation":24,"state":"up:active","state_seq":18,"addr":"172.21.15.178:6839/70628097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6838","nonce":70628097},{"type":"v1","addr":"172.21.15.178:6839","nonce":70628097}]},"join_fscid":-1,"export_targets":[0,2],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24638":{"gid":24638,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":18,"addr":"172.21.15.165:6837/6215088","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6836","nonce":6215088},{"type":"v1","addr":"172.21.15.165:6837","nonce":6215088}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24643":{"gid":24643,"name":"g","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.12:6839/3124093009","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6838","nonce":3124093009},{"type":"v1","addr":"172.21.15.12:6839","nonce":3124093009}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24674":{"gid":24674,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.165:6839/501312984","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6838","nonce":501312984},{"type":"v1","addr":"172.21.15.165:6839","nonce":501312984}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_27751":{"gid":27751,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6837/1436600611","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":1436600611},{"type":"v1","addr":"172.21.15.178:6837","nonce":1436600611}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24674,"qdb_cluster":[24674,24643,24638,24637,24614]},"id":1}]} 2024-09-11T00:30:32.847 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:32.843+0000 7fc599c00640 1 -- 172.21.15.12:0/388442219 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fc5780b5620 msgr2=0x7fc5780b7ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:30:32.847 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:32.843+0000 7fc599c00640 1 --2- 172.21.15.12:0/388442219 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fc5780b5620 0x7fc5780b7ae0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fc58400baa0 tx=0x7fc584005d90 comp rx=0 tx=0).stop 2024-09-11T00:30:32.848 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:32.843+0000 7fc599c00640 1 -- 172.21.15.12:0/388442219 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc59410f760 msgr2=0x7fc59407c7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:30:32.848 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:32.843+0000 7fc599c00640 1 --2- 172.21.15.12:0/388442219 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc59410f760 0x7fc59407c7f0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fc58c00eb10 tx=0x7fc58c00c2d0 comp rx=0 tx=0).stop 2024-09-11T00:30:32.848 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:32.844+0000 7fc599c00640 1 -- 172.21.15.12:0/388442219 shutdown_connections 2024-09-11T00:30:32.848 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:32.844+0000 7fc599c00640 1 --2- 172.21.15.12:0/388442219 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fc5780b5620 0x7fc5780b7ae0 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:30:32.848 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:32.844+0000 7fc599c00640 1 --2- 172.21.15.12:0/388442219 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc59410f760 0x7fc59407c7f0 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:30:32.848 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:32.844+0000 7fc599c00640 1 --2- 172.21.15.12:0/388442219 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc59410ede0 0x7fc594083a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:30:32.848 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:32.844+0000 7fc599c00640 1 --2- 172.21.15.12:0/388442219 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc5941093d0 0x7fc594083520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:30:32.848 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:32.844+0000 7fc599c00640 1 -- 172.21.15.12:0/388442219 >> 172.21.15.12:0/388442219 conn(0x7fc594078a40 msgr2=0x7fc59407ae30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:30:32.848 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:32.845+0000 7fc599c00640 1 -- 172.21.15.12:0/388442219 shutdown_connections 2024-09-11T00:30:32.848 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:32.845+0000 7fc599c00640 1 -- 172.21.15.12:0/388442219 wait complete. 2024-09-11T00:30:33.017 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:30:32 smithi012 ceph-mon[30333]: from='client.? 172.21.15.12:0/388442219' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T00:30:33.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:30:32 smithi165 ceph-mon[35356]: from='client.? 172.21.15.12:0/388442219' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T00:30:33.290 INFO:tasks.workunit.client.0.smithi012.stdout: 10 184518 16.03 MB/sec execute 509 sec latency 1151.233 ms 2024-09-11T00:30:33.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:30:32 smithi178 ceph-mon[35919]: from='client.? 172.21.15.12:0/388442219' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T00:30:34.092 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:30:33 smithi012 ceph-mon[30333]: pgmap v1104: 129 pgs: 129 active+clean; 828 MiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 MiB/s rd, 4.6 MiB/s wr, 740 op/s 2024-09-11T00:30:34.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:30:33 smithi165 ceph-mon[35356]: pgmap v1104: 129 pgs: 129 active+clean; 828 MiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 MiB/s rd, 4.6 MiB/s wr, 740 op/s 2024-09-11T00:30:34.290 INFO:tasks.workunit.client.0.smithi012.stdout: 10 184519 16.00 MB/sec execute 510 sec latency 2151.307 ms 2024-09-11T00:30:34.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:30:33 smithi178 ceph-mon[35919]: pgmap v1104: 129 pgs: 129 active+clean; 828 MiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 MiB/s rd, 4.6 MiB/s wr, 740 op/s 2024-09-11T00:30:35.053 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24674 quiesce db / --timeout 90 --expiration 138.85 --await-for 45.0' 2024-09-11T00:30:35.053 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid ff3ad74a-6fcf-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24674 quiesce db / --timeout 90 --expiration 138.85 --await-for 45.0 2024-09-11T00:30:35.290 INFO:tasks.workunit.client.0.smithi012.stdout: 10 184522 15.97 MB/sec execute 511 sec latency 3151.453 ms 2024-09-11T00:30:35.636 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:30:36.015 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:30:35 smithi012 ceph-mon[30333]: pgmap v1105: 129 pgs: 129 active+clean; 846 MiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 5.1 MiB/s wr, 609 op/s 2024-09-11T00:30:36.079 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:36.076+0000 7fd3bbe00640 1 -- 172.21.15.12:0/4257464922 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd3bc1042a0 msgr2=0x7fd3bc10ac20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:30:36.079 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:36.076+0000 7fd3bbe00640 1 --2- 172.21.15.12:0/4257464922 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd3bc1042a0 0x7fd3bc10ac20 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fd3b4009ca0 tx=0x7fd3b402a8d0 comp rx=0 tx=0).stop 2024-09-11T00:30:36.079 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:36.076+0000 7fd3bbe00640 1 -- 172.21.15.12:0/4257464922 shutdown_connections 2024-09-11T00:30:36.079 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:36.076+0000 7fd3bbe00640 1 --2- 172.21.15.12:0/4257464922 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd3bc1042a0 0x7fd3bc10ac20 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:30:36.079 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:36.076+0000 7fd3bbe00640 1 --2- 172.21.15.12:0/4257464922 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd3bc1038e0 0x7fd3bc103d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:30:36.079 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:36.076+0000 7fd3bbe00640 1 --2- 172.21.15.12:0/4257464922 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd3bc102790 0x7fd3bc102b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:30:36.080 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:36.076+0000 7fd3bbe00640 1 -- 172.21.15.12:0/4257464922 >> 172.21.15.12:0/4257464922 conn(0x7fd3bc0fe030 msgr2=0x7fd3bc100470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:30:36.080 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:36.077+0000 7fd3bbe00640 1 -- 172.21.15.12:0/4257464922 shutdown_connections 2024-09-11T00:30:36.080 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:36.077+0000 7fd3bbe00640 1 -- 172.21.15.12:0/4257464922 wait complete. 2024-09-11T00:30:36.081 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:36.078+0000 7fd3bbe00640 1 Processor -- start 2024-09-11T00:30:36.081 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:36.079+0000 7fd3bbe00640 1 -- start start 2024-09-11T00:30:36.082 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:36.079+0000 7fd3bbe00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd3bc102790 0x7fd3bc105080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:30:36.082 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:36.079+0000 7fd3bbe00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd3bc1038e0 0x7fd3bc1056d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:30:36.082 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:36.079+0000 7fd3bbe00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd3bc1042a0 0x7fd3bc106300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:30:36.082 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:36.079+0000 7fd3bbe00640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fd3bc10e430 con 0x7fd3bc102790 2024-09-11T00:30:36.082 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:36.079+0000 7fd3bbe00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fd3bc10e2b0 con 0x7fd3bc1042a0 2024-09-11T00:30:36.082 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:36.079+0000 7fd3bbe00640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fd3bc10e5b0 con 0x7fd3bc1038e0 2024-09-11T00:30:36.083 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:36.080+0000 7fd3bb400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd3bc1042a0 0x7fd3bc106300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:30:36.083 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:36.080+0000 7fd3bb400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd3bc1042a0 0x7fd3bc106300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.12:37946/0 (socket says 172.21.15.12:37946) 2024-09-11T00:30:36.083 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:36.080+0000 7fd3bb400640 1 -- 172.21.15.12:0/2312881619 learned_addr learned my addr 172.21.15.12:0/2312881619 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:30:36.083 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:36.080+0000 7fd3baa00640 1 --2- 172.21.15.12:0/2312881619 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd3bc102790 0x7fd3bc105080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:30:36.083 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:36.080+0000 7fd3bb400640 1 -- 172.21.15.12:0/2312881619 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd3bc1038e0 msgr2=0x7fd3bc1056d0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-11T00:30:36.083 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:36.080+0000 7fd3bb400640 1 --2- 172.21.15.12:0/2312881619 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd3bc1038e0 0x7fd3bc1056d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:30:36.083 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:36.080+0000 7fd3bb400640 1 -- 172.21.15.12:0/2312881619 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd3bc102790 msgr2=0x7fd3bc105080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:30:36.084 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:36.080+0000 7fd3bb400640 1 --2- 172.21.15.12:0/2312881619 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd3bc102790 0x7fd3bc105080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:30:36.084 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:36.080+0000 7fd3bb400640 1 -- 172.21.15.12:0/2312881619 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3bc106bc0 con 0x7fd3bc1042a0 2024-09-11T00:30:36.084 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:36.080+0000 7fd3baa00640 1 --2- 172.21.15.12:0/2312881619 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd3bc102790 0x7fd3bc105080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:30:36.084 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:36.081+0000 7fd3bb400640 1 --2- 172.21.15.12:0/2312881619 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd3bc1042a0 0x7fd3bc106300 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fd3b402adf0 tx=0x7fd3b4009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:30:36.084 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:36.081+0000 7fd3b3400640 1 -- 172.21.15.12:0/2312881619 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3b4003c70 con 0x7fd3bc1042a0 2024-09-11T00:30:36.085 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:36.081+0000 7fd3b3400640 1 -- 172.21.15.12:0/2312881619 <== mon.2 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd3b4003e10 con 0x7fd3bc1042a0 2024-09-11T00:30:36.085 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:36.081+0000 7fd3b3400640 1 -- 172.21.15.12:0/2312881619 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3b4030c10 con 0x7fd3bc1042a0 2024-09-11T00:30:36.085 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:36.081+0000 7fd3bbe00640 1 -- 172.21.15.12:0/2312881619 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd3bc106e50 con 0x7fd3bc1042a0 2024-09-11T00:30:36.085 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:36.081+0000 7fd3bbe00640 1 -- 172.21.15.12:0/2312881619 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3bc107550 con 0x7fd3bc1042a0 2024-09-11T00:30:36.087 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:36.085+0000 7fd3c4807740 1 Processor -- start 2024-09-11T00:30:36.087 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:36.085+0000 7fd3b3400640 1 -- 172.21.15.12:0/2312881619 <== mon.2 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7fd3b4004040 con 0x7fd3bc1042a0 2024-09-11T00:30:36.088 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:36.086+0000 7fd3c4807740 1 -- start start 2024-09-11T00:30:36.089 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:36.086+0000 7fd3c4807740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556d0a004970 0x556d0a006d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:30:36.089 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:36.086+0000 7fd3c4807740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556d0a0072a0 0x556d0a009690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:30:36.089 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:36.087+0000 7fd3c4807740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556d0a009ce0 0x556d0a00c1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:30:36.089 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:36.087+0000 7fd3c4807740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x556d09f776b0 con 0x556d0a004970 2024-09-11T00:30:36.089 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:36.087+0000 7fd3c4807740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x556d09f66ae0 con 0x556d0a009ce0 2024-09-11T00:30:36.089 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:36.087+0000 7fd3c4807740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x556d09f6a8f0 con 0x556d0a0072a0 2024-09-11T00:30:36.089 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:36.087+0000 7fd3ba000640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556d0a004970 0x556d0a006d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:30:36.090 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:36.087+0000 7fd3ba000640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556d0a004970 0x556d0a006d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:53208/0 (socket says 172.21.15.12:53208) 2024-09-11T00:30:36.090 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:36.087+0000 7fd3ba000640 1 -- 172.21.15.12:0/2915009732 learned_addr learned my addr 172.21.15.12:0/2915009732 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:30:36.090 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:36.087+0000 7fd3ba000640 1 -- 172.21.15.12:0/2915009732 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556d0a0072a0 msgr2=0x556d0a009690 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-11T00:30:36.090 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:36.087+0000 7fd3ba000640 1 --2- 172.21.15.12:0/2915009732 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556d0a0072a0 0x556d0a009690 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:30:36.090 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:36.087+0000 7fd3ba000640 1 -- 172.21.15.12:0/2915009732 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556d0a009ce0 msgr2=0x556d0a00c1a0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-11T00:30:36.090 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:36.087+0000 7fd3ba000640 1 --2- 172.21.15.12:0/2915009732 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556d0a009ce0 0x556d0a00c1a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:30:36.090 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:36.087+0000 7fd3ba000640 1 -- 172.21.15.12:0/2915009732 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556d09f33780 con 0x556d0a004970 2024-09-11T00:30:36.091 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:36.088+0000 7fd3b3400640 1 --2- 172.21.15.12:0/2312881619 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fd3a00b5360 0x7fd3a00b7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:30:36.091 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:36.088+0000 7fd3ba000640 1 --2- 172.21.15.12:0/2915009732 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556d0a004970 0x556d0a006d60 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7fd3a8009b60 tx=0x7fd3a800e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea1fd8d3792ca9ed server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:30:36.091 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:36.088+0000 7fd3b3400640 1 -- 172.21.15.12:0/2312881619 <== mon.2 v2:172.21.15.165:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9208+0+0 (secure 0 0 0) 0x7fd3b4107230 con 0x7fd3bc1042a0 2024-09-11T00:30:36.091 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:36.088+0000 7fd3baa00640 1 --2- 172.21.15.12:0/2312881619 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fd3a00b5360 0x7fd3a00b7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:30:36.092 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:36.088+0000 7fd3a7e00640 1 -- 172.21.15.12:0/2915009732 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3a8009e50 con 0x556d0a004970 2024-09-11T00:30:36.092 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:36.088+0000 7fd3a7e00640 1 -- 172.21.15.12:0/2915009732 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd3a8005020 con 0x556d0a004970 2024-09-11T00:30:36.092 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:36.088+0000 7fd3a7e00640 1 -- 172.21.15.12:0/2915009732 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3a80049e0 con 0x556d0a004970 2024-09-11T00:30:36.093 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:36.089+0000 7fd3c4807740 1 -- 172.21.15.12:0/2915009732 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556d0a004970 msgr2=0x556d0a006d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:30:36.093 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:36.089+0000 7fd3baa00640 1 --2- 172.21.15.12:0/2312881619 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fd3a00b5360 0x7fd3a00b7820 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fd3ac009fc0 tx=0x7fd3ac005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:30:36.093 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:36.089+0000 7fd3c4807740 1 --2- 172.21.15.12:0/2915009732 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556d0a004970 0x556d0a006d60 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7fd3a8009b60 tx=0x7fd3a800e720 comp rx=0 tx=0).stop 2024-09-11T00:30:36.093 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:36.089+0000 7fd3c4807740 1 -- 172.21.15.12:0/2915009732 shutdown_connections 2024-09-11T00:30:36.093 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:36.089+0000 7fd3c4807740 1 --2- 172.21.15.12:0/2915009732 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556d0a009ce0 0x556d0a00c1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:30:36.093 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:36.089+0000 7fd3c4807740 1 --2- 172.21.15.12:0/2915009732 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556d0a0072a0 0x556d0a009690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:30:36.093 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:36.089+0000 7fd3c4807740 1 --2- 172.21.15.12:0/2915009732 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556d0a004970 0x556d0a006d60 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:30:36.093 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:36.089+0000 7fd3c4807740 1 -- 172.21.15.12:0/2915009732 >> 172.21.15.12:0/2915009732 conn(0x556d09fffac0 msgr2=0x556d0a000850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:30:36.093 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:36.090+0000 7fd3c4807740 1 -- 172.21.15.12:0/2915009732 shutdown_connections 2024-09-11T00:30:36.093 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:36.090+0000 7fd3c4807740 1 -- 172.21.15.12:0/2915009732 wait complete. 2024-09-11T00:30:36.093 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:36.090+0000 7fd3c4807740 1 -- start start 2024-09-11T00:30:36.094 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:36.090+0000 7fd3c4807740 1 Processor -- start 2024-09-11T00:30:36.094 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:36.091+0000 7fd3c4807740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556d0a004970 0x556d0a00e160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:30:36.095 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:36.091+0000 7fd3c4807740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556d0a0072a0 0x556d0a00e800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:30:36.096 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:36.091+0000 7fd3c4807740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556d0a009ce0 0x556d0a00f360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:30:36.096 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:36.091+0000 7fd3c4807740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x556d09e9eab0 con 0x556d0a009ce0 2024-09-11T00:30:36.096 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:36.091+0000 7fd3c4807740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x556d09f0d570 con 0x556d0a0072a0 2024-09-11T00:30:36.096 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:36.091+0000 7fd3c4807740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x556d0a00ccc0 con 0x556d0a004970 2024-09-11T00:30:36.096 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:36.091+0000 7fd3c4807740 10 client.0 _ll_register_callbacks cb 0x556d09f1c100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:30:36.096 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:36.091+0000 7fd3baa00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556d0a004970 0x556d0a00e160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:30:36.096 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:36.091+0000 7fd3bb400640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556d0a009ce0 0x556d0a00f360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:30:36.096 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:36.091+0000 7fd3baa00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556d0a004970 0x556d0a00e160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.12:52620/0 (socket says 172.21.15.12:52620) 2024-09-11T00:30:36.096 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:36.091+0000 7fd3baa00640 1 -- 172.21.15.12:0/940294866 learned_addr learned my addr 172.21.15.12:0/940294866 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:30:36.096 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:36.092+0000 7fd3baa00640 1 -- 172.21.15.12:0/940294866 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556d0a0072a0 msgr2=0x556d0a00e800 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-11T00:30:36.096 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:36.092+0000 7fd3baa00640 1 --2- 172.21.15.12:0/940294866 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556d0a0072a0 0x556d0a00e800 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:30:36.096 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:36.092+0000 7fd3baa00640 1 -- 172.21.15.12:0/940294866 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556d0a009ce0 msgr2=0x556d0a00f360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:30:36.097 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:36.092+0000 7fd3baa00640 1 --2- 172.21.15.12:0/940294866 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556d0a009ce0 0x556d0a00f360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:30:36.097 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:36.092+0000 7fd3baa00640 1 -- 172.21.15.12:0/940294866 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556d09cc8790 con 0x556d0a004970 2024-09-11T00:30:36.097 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:36.092+0000 7fd3bb400640 1 --2- 172.21.15.12:0/940294866 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556d0a009ce0 0x556d0a00f360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:30:36.097 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:36.092+0000 7fd3baa00640 1 --2- 172.21.15.12:0/940294866 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556d0a004970 0x556d0a00e160 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fd3ac01bb60 tx=0x7fd3ac004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7353e2b243429da4 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:30:36.097 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:36.092+0000 7fd3a4c00640 10 client.0 ms_handle_connect on v2:172.21.15.178:3300/0 2024-09-11T00:30:36.097 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:36.092+0000 7fd3a4c00640 1 -- 172.21.15.12:0/940294866 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3ac02c060 con 0x556d0a004970 2024-09-11T00:30:36.097 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:36.092+0000 7fd3a4c00640 1 -- 172.21.15.12:0/940294866 <== mon.1 v2:172.21.15.178:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 9208+0+0 (secure 0 0 0) 0x7fd3ac01d920 con 0x556d0a004970 2024-09-11T00:30:36.097 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:36.092+0000 7fd3c4807740 1 -- 172.21.15.12:0/940294866 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd3a8005020 con 0x556d0a004970 2024-09-11T00:30:36.099 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:36.093+0000 7fd3a4c00640 1 -- 172.21.15.12:0/940294866 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=102) ==== 24+0+0 (secure 0 0 0) 0x7fd3ac033050 con 0x556d0a004970 2024-09-11T00:30:36.099 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:36.093+0000 7fd3c4807740 10 client.27961 fetch_fsmap learned FSMap version 102 2024-09-11T00:30:36.099 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:36.093+0000 7fd3c4807740 1 -- 172.21.15.12:0/940294866 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,fsmap=102,monmap=4+}) -- 0x556d09d41620 con 0x556d0a004970 2024-09-11T00:30:36.099 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:36.093+0000 7fd3a4c00640 1 -- 172.21.15.12:0/940294866 <== mon.1 v2:172.21.15.178:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd3a8005020 con 0x556d0a004970 2024-09-11T00:30:36.099 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:36.093+0000 7fd3a4c00640 1 -- 172.21.15.12:0/940294866 <== mon.1 v2:172.21.15.178:3300/0 5 ==== fsmap(e 102) ==== 8378+0+0 (secure 0 0 0) 0x7fd3ac00fc40 con 0x556d0a004970 2024-09-11T00:30:36.099 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:36.093+0000 7fd3c4807740 10 client.27961 fetch_fsmap finished waiting for FSMap version 102 2024-09-11T00:30:36.099 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:36.093+0000 7fd3c4807740 10 client.27961 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T00:30:36.099 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:36.093+0000 7fd3c4807740 20 client.27961 populate_metadata read hostname 'smithi012' 2024-09-11T00:30:36.100 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:36.093+0000 7fd3c4807740 1 --2- 172.21.15.12:0/940294866 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x556d0a019a30 0x556d0a01bed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:30:36.100 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:36.093+0000 7fd3c4807740 4 client.27961 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T00:30:36.100 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:36.093+0000 7fd3c4807740 1 -- 172.21.15.12:0/940294866 --> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556d09f79600 con 0x556d0a019a30 2024-09-11T00:30:36.100 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:36.094+0000 7fd3ba000640 1 --2- 172.21.15.12:0/940294866 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x556d0a019a30 0x556d0a01bed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:30:36.100 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:36.094+0000 7fd3ba000640 1 --2- 172.21.15.12:0/940294866 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x556d0a019a30 0x556d0a01bed0 crc :-1 s=READY pgs=317 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6eb774416186d93f server_cookie=924fc2821b992133 in_seq=0 out_seq=0 2024-09-11T00:30:36.100 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:36.094+0000 7fd3a4c00640 10 client.27961 ms_handle_connect on v2:172.21.15.165:6838/501312984 2024-09-11T00:30:36.100 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:36.095+0000 7fd3a4c00640 1 -- 172.21.15.12:0/940294866 <== mds.0 v2:172.21.15.165:6838/501312984 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x556d09f79600 con 0x556d0a019a30 2024-09-11T00:30:36.100 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:36.095+0000 7fd3a4c00640 10 client.27961 handle_command_reply: tid=0 2024-09-11T00:30:36.100 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:36.095+0000 7fd3c4807740 1 client.27961 shutdown 2024-09-11T00:30:36.100 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:36.096+0000 7fd3c4807740 1 -- 172.21.15.12:0/940294866 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556d0a004970 msgr2=0x556d0a00e160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:30:36.100 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:36.096+0000 7fd3c4807740 1 --2- 172.21.15.12:0/940294866 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556d0a004970 0x556d0a00e160 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fd3ac01bb60 tx=0x7fd3ac004730 comp rx=0 tx=0).stop 2024-09-11T00:30:36.100 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:36.096+0000 7fd3c4807740 1 -- 172.21.15.12:0/940294866 shutdown_connections 2024-09-11T00:30:36.100 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:36.096+0000 7fd3c4807740 1 --2- 172.21.15.12:0/940294866 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x556d0a019a30 0x556d0a01bed0 crc :-1 s=READY pgs=317 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:30:36.100 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:36.096+0000 7fd3c4807740 1 --2- 172.21.15.12:0/940294866 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556d0a009ce0 0x556d0a00f360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:30:36.101 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:36.096+0000 7fd3c4807740 1 --2- 172.21.15.12:0/940294866 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556d0a0072a0 0x556d0a00e800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:30:36.101 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:36.096+0000 7fd3c4807740 1 --2- 172.21.15.12:0/940294866 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556d0a004970 0x556d0a00e160 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:30:36.101 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:36.097+0000 7fd3c4807740 1 -- 172.21.15.12:0/940294866 >> 172.21.15.12:0/940294866 conn(0x556d09fffac0 msgr2=0x556d09ffebc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:30:36.101 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:36.097+0000 7fd3a4c00640 0 client.27961 ms_handle_reset on v2:172.21.15.165:6838/501312984 2024-09-11T00:30:36.101 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:36.097+0000 7fd3c4807740 1 -- 172.21.15.12:0/940294866 shutdown_connections 2024-09-11T00:30:36.102 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:36.097+0000 7fd3c4807740 1 -- 172.21.15.12:0/940294866 wait complete. 2024-09-11T00:30:36.102 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:36.097+0000 7fd3c4807740 20 client.27961 trim_cache size 0 max 16384 2024-09-11T00:30:36.123 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:36.121+0000 7fd3c4807740 1 Processor -- start 2024-09-11T00:30:36.123 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:36.121+0000 7fd3c4807740 1 -- start start 2024-09-11T00:30:36.124 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:36.122+0000 7fd3c4807740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556d0a004970 0x556d0a00d090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:30:36.124 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:36.122+0000 7fd3c4807740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556d0a0072a0 0x556d0a00d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:30:36.125 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:36.122+0000 7fd3c4807740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556d0a009ce0 0x556d0a00a0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:30:36.125 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:36.123+0000 7fd3c4807740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fd3bc1ba330 con 0x556d0a004970 2024-09-11T00:30:36.125 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:36.123+0000 7fd3c4807740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fd3bc1bbed0 con 0x556d0a0072a0 2024-09-11T00:30:36.125 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:36.123+0000 7fd3c4807740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fd3bc1baa60 con 0x556d0a009ce0 2024-09-11T00:30:36.125 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:36.123+0000 7fd3baa00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556d0a004970 0x556d0a00d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:30:36.126 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:36.123+0000 7fd3baa00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556d0a004970 0x556d0a00d090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:53224/0 (socket says 172.21.15.12:53224) 2024-09-11T00:30:36.126 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:36.123+0000 7fd3baa00640 1 -- 172.21.15.12:0/1965081588 learned_addr learned my addr 172.21.15.12:0/1965081588 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:30:36.126 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:36.123+0000 7fd3ba000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556d0a0072a0 0x556d0a00d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:30:36.126 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:36.124+0000 7fd3ba000640 1 -- 172.21.15.12:0/1965081588 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556d0a009ce0 msgr2=0x556d0a00a0c0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-11T00:30:36.126 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:36.124+0000 7fd3ba000640 1 --2- 172.21.15.12:0/1965081588 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556d0a009ce0 0x556d0a00a0c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:30:36.126 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:36.124+0000 7fd3ba000640 1 -- 172.21.15.12:0/1965081588 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556d0a004970 msgr2=0x556d0a00d090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:30:36.126 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:36.124+0000 7fd3ba000640 1 --2- 172.21.15.12:0/1965081588 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556d0a004970 0x556d0a00d090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:30:36.127 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:36.124+0000 7fd3ba000640 1 -- 172.21.15.12:0/1965081588 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556d0a00dc60 con 0x556d0a0072a0 2024-09-11T00:30:36.133 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:36.131+0000 7fd3ba000640 1 --2- 172.21.15.12:0/1965081588 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556d0a0072a0 0x556d0a00d5d0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x556d0a01c6e0 tx=0x7fd3a8009850 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fa2acc5e36f74190 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:30:36.133 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:36.131+0000 7fd3a7400640 1 -- 172.21.15.12:0/1965081588 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3a8009e50 con 0x556d0a0072a0 2024-09-11T00:30:36.134 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:36.131+0000 7fd3a7400640 1 -- 172.21.15.12:0/1965081588 <== mon.2 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd3a8005500 con 0x556d0a0072a0 2024-09-11T00:30:36.134 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:36.131+0000 7fd3a7400640 1 -- 172.21.15.12:0/1965081588 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3a8004dc0 con 0x556d0a0072a0 2024-09-11T00:30:36.134 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:36.132+0000 7fd3c4807740 1 -- 172.21.15.12:0/1965081588 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556d0a0072a0 msgr2=0x556d0a00d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:30:36.134 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:36.132+0000 7fd3c4807740 1 --2- 172.21.15.12:0/1965081588 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556d0a0072a0 0x556d0a00d5d0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x556d0a01c6e0 tx=0x7fd3a8009850 comp rx=0 tx=0).stop 2024-09-11T00:30:36.135 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:36.132+0000 7fd3c4807740 1 -- 172.21.15.12:0/1965081588 shutdown_connections 2024-09-11T00:30:36.135 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:36.132+0000 7fd3c4807740 1 --2- 172.21.15.12:0/1965081588 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556d0a009ce0 0x556d0a00a0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:30:36.135 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:36.132+0000 7fd3c4807740 1 --2- 172.21.15.12:0/1965081588 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556d0a0072a0 0x556d0a00d5d0 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:30:36.135 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:36.132+0000 7fd3c4807740 1 --2- 172.21.15.12:0/1965081588 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556d0a004970 0x556d0a00d090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:30:36.135 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:36.132+0000 7fd3c4807740 1 -- 172.21.15.12:0/1965081588 >> 172.21.15.12:0/1965081588 conn(0x556d09fffac0 msgr2=0x556d0a003ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:30:36.135 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:36.132+0000 7fd3c4807740 1 -- 172.21.15.12:0/1965081588 shutdown_connections 2024-09-11T00:30:36.136 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:36.132+0000 7fd3c4807740 1 -- 172.21.15.12:0/1965081588 wait complete. 2024-09-11T00:30:36.138 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:36.132+0000 7fd3c4807740 1 -- start start 2024-09-11T00:30:36.139 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:36.133+0000 7fd3c4807740 1 Processor -- start 2024-09-11T00:30:36.139 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:36.133+0000 7fd3c4807740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556d0a004970 0x556d0a012ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:30:36.139 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:36.133+0000 7fd3c4807740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556d0a0072a0 0x556d0a013560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:30:36.139 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:36.133+0000 7fd3c4807740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556d0a009ce0 0x556d0a014130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:30:36.139 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:36.133+0000 7fd3c4807740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fd3bc1bb800 con 0x556d0a0072a0 2024-09-11T00:30:36.139 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:36.133+0000 7fd3c4807740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fd3bc1bb130 con 0x556d0a009ce0 2024-09-11T00:30:36.139 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:36.133+0000 7fd3c4807740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fd3bc1a12d0 con 0x556d0a004970 2024-09-11T00:30:36.139 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:36.133+0000 7fd3c4807740 10 client.0 _ll_register_callbacks cb 0x7fd3ac01b3a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:30:36.139 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:36.134+0000 7fd3ba000640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556d0a0072a0 0x556d0a013560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:30:36.139 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:36.134+0000 7fd3baa00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556d0a004970 0x556d0a012ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:30:36.139 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:36.134+0000 7fd3baa00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556d0a004970 0x556d0a012ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.12:52646/0 (socket says 172.21.15.12:52646) 2024-09-11T00:30:36.140 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:36.134+0000 7fd3baa00640 1 -- 172.21.15.12:0/4094468789 learned_addr learned my addr 172.21.15.12:0/4094468789 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:30:36.140 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:36.134+0000 7fd3ba000640 1 --2- 172.21.15.12:0/4094468789 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556d0a0072a0 0x556d0a013560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:53226/0 (socket says 172.21.15.12:53226) 2024-09-11T00:30:36.140 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:36.134+0000 7fd3ba000640 1 -- 172.21.15.12:0/4094468789 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556d0a004970 msgr2=0x556d0a012ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:30:36.140 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:36.134+0000 7fd3ba000640 1 --2- 172.21.15.12:0/4094468789 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556d0a004970 0x556d0a012ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:30:36.140 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:36.134+0000 7fd3ba000640 1 -- 172.21.15.12:0/4094468789 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556d0a009ce0 msgr2=0x556d0a014130 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-11T00:30:36.140 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:36.134+0000 7fd3ba000640 1 --2- 172.21.15.12:0/4094468789 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556d0a009ce0 0x556d0a014130 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:30:36.140 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:36.134+0000 7fd3ba000640 1 -- 172.21.15.12:0/4094468789 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556d0a014960 con 0x556d0a0072a0 2024-09-11T00:30:36.140 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:36.134+0000 7fd3baa00640 1 --2- 172.21.15.12:0/4094468789 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556d0a004970 0x556d0a012ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:30:36.140 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:36.134+0000 7fd3ba000640 1 --2- 172.21.15.12:0/4094468789 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556d0a0072a0 0x556d0a013560 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7fd3a8009b30 tx=0x7fd3a8005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=549b986d13fb33e4 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:30:36.140 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:36.134+0000 7fd3a4c00640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-09-11T00:30:36.140 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:36.135+0000 7fd3a4c00640 1 -- 172.21.15.12:0/4094468789 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3a8009c50 con 0x556d0a0072a0 2024-09-11T00:30:36.140 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:36.135+0000 7fd3a4c00640 1 -- 172.21.15.12:0/4094468789 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 9208+0+0 (secure 0 0 0) 0x7fd3a8009df0 con 0x556d0a0072a0 2024-09-11T00:30:36.140 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:36.135+0000 7fd3c4807740 1 -- 172.21.15.12:0/4094468789 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd3a8005500 con 0x556d0a0072a0 2024-09-11T00:30:36.141 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:36.135+0000 7fd3a4c00640 1 -- 172.21.15.12:0/4094468789 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=102) ==== 24+0+0 (secure 0 0 0) 0x7fd3a8004b40 con 0x556d0a0072a0 2024-09-11T00:30:36.141 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:36.135+0000 7fd3c4807740 10 client.17970 fetch_fsmap learned FSMap version 102 2024-09-11T00:30:36.141 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:36.135+0000 7fd3c4807740 1 -- 172.21.15.12:0/4094468789 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=102,monmap=4+}) -- 0x556d0a015cd0 con 0x556d0a0072a0 2024-09-11T00:30:36.141 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:36.135+0000 7fd3a4c00640 1 -- 172.21.15.12:0/4094468789 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd3a8005500 con 0x556d0a0072a0 2024-09-11T00:30:36.141 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:36.135+0000 7fd3a4c00640 1 -- 172.21.15.12:0/4094468789 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 102) ==== 8378+0+0 (secure 0 0 0) 0x7fd3a8021040 con 0x556d0a0072a0 2024-09-11T00:30:36.141 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:36.135+0000 7fd3c4807740 10 client.17970 fetch_fsmap finished waiting for FSMap version 102 2024-09-11T00:30:36.141 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:36.135+0000 7fd3c4807740 10 client.17970 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T00:30:36.141 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:36.135+0000 7fd3c4807740 20 client.17970 populate_metadata read hostname 'smithi012' 2024-09-11T00:30:36.141 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:36.135+0000 7fd3c4807740 1 --2- 172.21.15.12:0/4094468789 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x556d0a0869a0 0x556d0a088dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:30:36.141 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:36.135+0000 7fd3c4807740 4 client.17970 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T00:30:36.141 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:36.135+0000 7fd3c4807740 1 -- 172.21.15.12:0/4094468789 --> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] -- command(tid 0: {"prefix": "flush journal"}) -- 0x556d09f750b0 con 0x556d0a0869a0 2024-09-11T00:30:36.142 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:36.136+0000 7fd3baa00640 1 --2- 172.21.15.12:0/4094468789 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x556d0a0869a0 0x556d0a088dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:30:36.142 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:36.136+0000 7fd3baa00640 1 --2- 172.21.15.12:0/4094468789 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x556d0a0869a0 0x556d0a088dc0 crc :-1 s=READY pgs=318 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=15328c287e55949a server_cookie=6849daa6703e1ad9 in_seq=0 out_seq=0 2024-09-11T00:30:36.142 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:36.136+0000 7fd3a4c00640 10 client.17970 ms_handle_connect on v2:172.21.15.165:6838/501312984 2024-09-11T00:30:36.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:30:35 smithi165 ceph-mon[35356]: pgmap v1105: 129 pgs: 129 active+clean; 846 MiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 5.1 MiB/s wr, 609 op/s 2024-09-11T00:30:36.290 INFO:tasks.workunit.client.0.smithi012.stdout: 10 184522 15.93 MB/sec execute 512 sec latency 4151.516 ms 2024-09-11T00:30:36.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:30:35 smithi178 ceph-mon[35919]: pgmap v1105: 129 pgs: 129 active+clean; 846 MiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 5.1 MiB/s wr, 609 op/s 2024-09-11T00:30:36.801 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:36.798+0000 7f46d8e00640 1 -- 172.21.15.12:0/2140252380 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f46d4102dc0 msgr2=0x7f46d41031a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:30:36.801 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:36.798+0000 7f46d8e00640 1 --2- 172.21.15.12:0/2140252380 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f46d4102dc0 0x7f46d41031a0 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f46c4009900 tx=0x7f46c402a420 comp rx=0 tx=0).stop 2024-09-11T00:30:36.801 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:36.798+0000 7f46d8e00640 1 -- 172.21.15.12:0/2140252380 shutdown_connections 2024-09-11T00:30:36.802 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:36.798+0000 7f46d8e00640 1 --2- 172.21.15.12:0/2140252380 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f46d4109150 0x7f46d410f950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:30:36.802 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:36.798+0000 7f46d8e00640 1 --2- 172.21.15.12:0/2140252380 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f46d41087d0 0x7f46d4108c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:30:36.802 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:36.798+0000 7f46d8e00640 1 --2- 172.21.15.12:0/2140252380 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f46d4102dc0 0x7f46d41031a0 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:30:36.802 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:36.798+0000 7f46d8e00640 1 -- 172.21.15.12:0/2140252380 >> 172.21.15.12:0/2140252380 conn(0x7f46d40fe830 msgr2=0x7f46d4100c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:30:36.805 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:36.800+0000 7f46d8e00640 1 -- 172.21.15.12:0/2140252380 shutdown_connections 2024-09-11T00:30:36.805 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:36.800+0000 7f46d8e00640 1 -- 172.21.15.12:0/2140252380 wait complete. 2024-09-11T00:30:36.805 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:36.801+0000 7f46d8e00640 1 Processor -- start 2024-09-11T00:30:36.805 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:36.801+0000 7f46d8e00640 1 -- start start 2024-09-11T00:30:36.805 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:36.801+0000 7f46d8e00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f46d4102dc0 0x7f46d419b550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:30:36.805 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:36.801+0000 7f46d8e00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f46d41087d0 0x7f46d419ba90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:30:36.806 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:36.801+0000 7f46d8e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f46d4109150 0x7f46d41a0e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:30:36.806 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:36.801+0000 7f46d8e00640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f46d4112e10 con 0x7f46d4102dc0 2024-09-11T00:30:36.806 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:36.801+0000 7f46d8e00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f46d4112c90 con 0x7f46d4109150 2024-09-11T00:30:36.806 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:36.801+0000 7f46d8e00640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f46d4112f90 con 0x7f46d41087d0 2024-09-11T00:30:36.806 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:36.802+0000 7f46d1e00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f46d4102dc0 0x7f46d419b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:30:36.806 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:36.802+0000 7f46d1e00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f46d4102dc0 0x7f46d419b550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:53250/0 (socket says 172.21.15.12:53250) 2024-09-11T00:30:36.806 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:36.802+0000 7f46d1e00640 1 -- 172.21.15.12:0/3559187954 learned_addr learned my addr 172.21.15.12:0/3559187954 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:30:36.806 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:36.802+0000 7f46d2800640 1 --2- 172.21.15.12:0/3559187954 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f46d4109150 0x7f46d41a0e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:30:36.806 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:36.802+0000 7f46d1e00640 1 -- 172.21.15.12:0/3559187954 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f46d41087d0 msgr2=0x7f46d419ba90 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-11T00:30:36.806 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:36.802+0000 7f46d1e00640 1 --2- 172.21.15.12:0/3559187954 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f46d41087d0 0x7f46d419ba90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:30:36.807 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:36.802+0000 7f46d1e00640 1 -- 172.21.15.12:0/3559187954 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f46d4109150 msgr2=0x7f46d41a0e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:30:36.807 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:36.802+0000 7f46d1e00640 1 --2- 172.21.15.12:0/3559187954 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f46d4109150 0x7f46d41a0e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:30:36.807 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:36.802+0000 7f46d1e00640 1 -- 172.21.15.12:0/3559187954 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46d41a1600 con 0x7f46d4102dc0 2024-09-11T00:30:36.807 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:36.803+0000 7f46d1e00640 1 --2- 172.21.15.12:0/3559187954 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f46d4102dc0 0x7f46d419b550 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f46c4005940 tx=0x7f46c4004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:30:36.807 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:36.803+0000 7f46caa00640 1 -- 172.21.15.12:0/3559187954 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46c403f030 con 0x7f46d4102dc0 2024-09-11T00:30:36.807 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:36.803+0000 7f46caa00640 1 -- 172.21.15.12:0/3559187954 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f46c4003aa0 con 0x7f46d4102dc0 2024-09-11T00:30:36.808 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:36.803+0000 7f46d8e00640 1 -- 172.21.15.12:0/3559187954 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46d41a1890 con 0x7f46d4102dc0 2024-09-11T00:30:36.808 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:36.803+0000 7f46caa00640 1 -- 172.21.15.12:0/3559187954 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46c4035070 con 0x7f46d4102dc0 2024-09-11T00:30:36.808 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:36.804+0000 7f46d8e00640 1 -- 172.21.15.12:0/3559187954 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46d41a7fb0 con 0x7f46d4102dc0 2024-09-11T00:30:36.813 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:36.809+0000 7f46caa00640 1 -- 172.21.15.12:0/3559187954 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7f46c404c060 con 0x7f46d4102dc0 2024-09-11T00:30:36.816 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:36.810+0000 7f46db80c740 1 Processor -- start 2024-09-11T00:30:36.816 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:36.810+0000 7f46db80c740 1 -- start start 2024-09-11T00:30:36.816 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:36.810+0000 7f46db80c740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564ccb9633e0 0x564ccb9619b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:30:36.816 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:36.810+0000 7f46db80c740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564ccb961ef0 0x564ccb962330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:30:36.816 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:36.810+0000 7f46db80c740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564ccb95c920 0x564ccb978cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:30:36.816 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:36.810+0000 7f46db80c740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x564ccb8cea90 con 0x564ccb95c920 2024-09-11T00:30:36.816 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:36.810+0000 7f46db80c740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x564ccb8bdec0 con 0x564ccb961ef0 2024-09-11T00:30:36.816 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:36.810+0000 7f46db80c740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x564ccb8c1cd0 con 0x564ccb9633e0 2024-09-11T00:30:36.816 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:36.811+0000 7f46d2800640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564ccb9633e0 0x564ccb9619b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:30:36.816 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:36.811+0000 7f46d2800640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564ccb9633e0 0x564ccb9619b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.12:52672/0 (socket says 172.21.15.12:52672) 2024-09-11T00:30:36.817 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:36.811+0000 7f46d2800640 1 -- 172.21.15.12:0/1595927837 learned_addr learned my addr 172.21.15.12:0/1595927837 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:30:36.817 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:36.811+0000 7f46d1400640 1 --2- 172.21.15.12:0/1595927837 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564ccb95c920 0x564ccb978cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:30:36.817 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:36.811+0000 7f46caa00640 1 --2- 172.21.15.12:0/3559187954 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f46b80b5360 0x7f46b80b7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:30:36.817 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:36.811+0000 7f46d2800640 1 -- 172.21.15.12:0/1595927837 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564ccb961ef0 msgr2=0x564ccb962330 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-11T00:30:36.817 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:36.811+0000 7f46d2800640 1 --2- 172.21.15.12:0/1595927837 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564ccb961ef0 0x564ccb962330 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:30:36.817 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:36.811+0000 7f46caa00640 1 -- 172.21.15.12:0/3559187954 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9208+0+0 (secure 0 0 0) 0x7f46c4106100 con 0x7f46d4102dc0 2024-09-11T00:30:36.817 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:36.811+0000 7f46d2800640 1 -- 172.21.15.12:0/1595927837 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564ccb95c920 msgr2=0x564ccb978cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:30:36.817 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:36.812+0000 7f46d2800640 1 --2- 172.21.15.12:0/1595927837 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564ccb95c920 0x564ccb978cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:30:36.817 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:36.812+0000 7f46d2800640 1 -- 172.21.15.12:0/1595927837 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564ccb8639d0 con 0x564ccb9633e0 2024-09-11T00:30:36.817 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:36.812+0000 7f46d2800640 1 --2- 172.21.15.12:0/3559187954 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f46b80b5360 0x7f46b80b7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:30:36.817 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:36.812+0000 7f46d2800640 1 --2- 172.21.15.12:0/1595927837 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564ccb9633e0 0x564ccb9619b0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f46cc0104f0 tx=0x7f46cc0109b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f6e8a0a4e976ae19 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:30:36.817 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:36.812+0000 7f46bf400640 1 -- 172.21.15.12:0/1595927837 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46cc019070 con 0x564ccb9633e0 2024-09-11T00:30:36.818 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:36.813+0000 7f46bf400640 1 -- 172.21.15.12:0/1595927837 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f46cc00edb0 con 0x564ccb9633e0 2024-09-11T00:30:36.818 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:36.813+0000 7f46bf400640 1 -- 172.21.15.12:0/1595927837 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46cc01db10 con 0x564ccb9633e0 2024-09-11T00:30:36.819 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:36.813+0000 7f46db80c740 1 -- 172.21.15.12:0/1595927837 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564ccb9633e0 msgr2=0x564ccb9619b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:30:36.820 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:36.813+0000 7f46db80c740 1 --2- 172.21.15.12:0/1595927837 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564ccb9633e0 0x564ccb9619b0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f46cc0104f0 tx=0x7f46cc0109b0 comp rx=0 tx=0).stop 2024-09-11T00:30:36.820 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:36.813+0000 7f46d2800640 1 --2- 172.21.15.12:0/3559187954 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f46b80b5360 0x7f46b80b7820 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f46cc021000 tx=0x7f46cc01e000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:30:36.820 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:36.813+0000 7f46db80c740 1 -- 172.21.15.12:0/1595927837 shutdown_connections 2024-09-11T00:30:36.820 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:36.813+0000 7f46db80c740 1 --2- 172.21.15.12:0/1595927837 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564ccb95c920 0x564ccb978cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:30:36.820 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:36.813+0000 7f46db80c740 1 --2- 172.21.15.12:0/1595927837 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564ccb961ef0 0x564ccb962330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:30:36.820 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:36.813+0000 7f46db80c740 1 --2- 172.21.15.12:0/1595927837 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564ccb9633e0 0x564ccb9619b0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:30:36.820 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:36.813+0000 7f46db80c740 1 -- 172.21.15.12:0/1595927837 >> 172.21.15.12:0/1595927837 conn(0x564ccb9236f0 msgr2=0x564ccb95f7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:30:36.821 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:36.814+0000 7f46db80c740 1 -- 172.21.15.12:0/1595927837 shutdown_connections 2024-09-11T00:30:36.821 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:36.814+0000 7f46db80c740 1 -- 172.21.15.12:0/1595927837 wait complete. 2024-09-11T00:30:36.821 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:36.814+0000 7f46db80c740 1 -- start start 2024-09-11T00:30:36.821 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:36.814+0000 7f46db80c740 1 Processor -- start 2024-09-11T00:30:36.821 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:36.815+0000 7f46db80c740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564ccb95c920 0x564ccb959890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:30:36.821 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:36.815+0000 7f46db80c740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564ccb961ef0 0x564ccb959dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:30:36.821 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:36.815+0000 7f46db80c740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564ccb9633e0 0x564ccb95a310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:30:36.821 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:36.815+0000 7f46db80c740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x564ccb71f7b0 con 0x564ccb9633e0 2024-09-11T00:30:36.821 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:36.815+0000 7f46db80c740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x564ccb862650 con 0x564ccb961ef0 2024-09-11T00:30:36.821 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:36.815+0000 7f46db80c740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x564ccb979350 con 0x564ccb95c920 2024-09-11T00:30:36.821 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:36.815+0000 7f46db80c740 10 client.0 _ll_register_callbacks cb 0x564ccb5c90a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:30:36.821 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:36.815+0000 7f46d1e00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564ccb95c920 0x564ccb959890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:30:36.822 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:36.815+0000 7f46d1e00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564ccb95c920 0x564ccb959890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.12:52686/0 (socket says 172.21.15.12:52686) 2024-09-11T00:30:36.822 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:36.815+0000 7f46d1e00640 1 -- 172.21.15.12:0/4220228054 learned_addr learned my addr 172.21.15.12:0/4220228054 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:30:36.822 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:36.816+0000 7f46d1e00640 1 -- 172.21.15.12:0/4220228054 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564ccb961ef0 msgr2=0x564ccb959dd0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-11T00:30:36.822 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:36.816+0000 7f46d1e00640 1 --2- 172.21.15.12:0/4220228054 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564ccb961ef0 0x564ccb959dd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:30:36.822 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:36.816+0000 7f46d2800640 1 --2- 172.21.15.12:0/4220228054 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564ccb9633e0 0x564ccb95a310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:30:36.822 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:36.816+0000 7f46d1e00640 1 -- 172.21.15.12:0/4220228054 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564ccb9633e0 msgr2=0x564ccb95a310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:30:36.822 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:36.816+0000 7f46d1e00640 1 --2- 172.21.15.12:0/4220228054 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564ccb9633e0 0x564ccb95a310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:30:36.822 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:36.816+0000 7f46d1e00640 1 -- 172.21.15.12:0/4220228054 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564ccb62bba0 con 0x564ccb95c920 2024-09-11T00:30:36.822 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:36.816+0000 7f46d1e00640 1 --2- 172.21.15.12:0/4220228054 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564ccb95c920 0x564ccb959890 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f46c4106430 tx=0x7f46c40c8450 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fde53f7bb3c4a46a server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:30:36.822 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:36.816+0000 7f46b3e00640 10 client.0 ms_handle_connect on v2:172.21.15.178:3300/0 2024-09-11T00:30:36.822 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:36.817+0000 7f46b3e00640 1 -- 172.21.15.12:0/4220228054 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46c4049040 con 0x564ccb95c920 2024-09-11T00:30:36.822 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:36.817+0000 7f46b3e00640 1 -- 172.21.15.12:0/4220228054 <== mon.1 v2:172.21.15.178:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 9208+0+0 (secure 0 0 0) 0x7f46c40d55f0 con 0x564ccb95c920 2024-09-11T00:30:36.822 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:36.817+0000 7f46db80c740 1 -- 172.21.15.12:0/4220228054 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f46cc00edb0 con 0x564ccb95c920 2024-09-11T00:30:36.823 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:36.817+0000 7f46b3e00640 1 -- 172.21.15.12:0/4220228054 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=102) ==== 24+0+0 (secure 0 0 0) 0x7f46c40cc000 con 0x564ccb95c920 2024-09-11T00:30:36.823 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:36.817+0000 7f46db80c740 10 client.27970 fetch_fsmap learned FSMap version 102 2024-09-11T00:30:36.823 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:36.817+0000 7f46db80c740 1 -- 172.21.15.12:0/4220228054 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,fsmap=102,monmap=4+}) -- 0x564ccb984f10 con 0x564ccb95c920 2024-09-11T00:30:36.824 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:36.817+0000 7f46b3e00640 1 -- 172.21.15.12:0/4220228054 <== mon.1 v2:172.21.15.178:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f46cc00edb0 con 0x564ccb95c920 2024-09-11T00:30:36.824 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:36.817+0000 7f46b3e00640 1 -- 172.21.15.12:0/4220228054 <== mon.1 v2:172.21.15.178:3300/0 5 ==== fsmap(e 102) ==== 8378+0+0 (secure 0 0 0) 0x7f46c40cb750 con 0x564ccb95c920 2024-09-11T00:30:36.824 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:36.818+0000 7f46db80c740 10 client.27970 fetch_fsmap finished waiting for FSMap version 102 2024-09-11T00:30:36.824 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:36.818+0000 7f46db80c740 10 client.27970 resolve_mds: validated gid 24674 aka daemon mds.b 2024-09-11T00:30:36.824 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:36.818+0000 7f46db80c740 20 client.27970 populate_metadata read hostname 'smithi012' 2024-09-11T00:30:36.824 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:36.818+0000 7f46db80c740 1 --2- 172.21.15.12:0/4220228054 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x564ccb9862b0 0x564ccb988730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:30:36.824 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:36.818+0000 7f46db80c740 4 client.27970 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T00:30:36.824 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:36.818+0000 7f46db80c740 1 -- 172.21.15.12:0/4220228054 --> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564ccb8d09e0 con 0x564ccb9862b0 2024-09-11T00:30:36.852 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:36.849+0000 7f46d1400640 1 --2- 172.21.15.12:0/4220228054 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x564ccb9862b0 0x564ccb988730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:30:36.860 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:36.858+0000 7fd3a4c00640 1 -- 172.21.15.12:0/4094468789 <== mds.0 v2:172.21.15.165:6838/501312984 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x556d09f750b0 con 0x556d0a0869a0 2024-09-11T00:30:36.860 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:36.858+0000 7fd3a4c00640 10 client.17970 handle_command_reply: tid=0 2024-09-11T00:30:36.860 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:36.858+0000 7fd3c4807740 1 client.17970 shutdown 2024-09-11T00:30:36.861 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:36.858+0000 7fd3c4807740 1 -- 172.21.15.12:0/4094468789 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556d0a0072a0 msgr2=0x556d0a013560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:30:36.861 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:36.858+0000 7fd3c4807740 1 --2- 172.21.15.12:0/4094468789 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556d0a0072a0 0x556d0a013560 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7fd3a8009b30 tx=0x7fd3a8005740 comp rx=0 tx=0).stop 2024-09-11T00:30:36.861 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:36.858+0000 7fd3c4807740 1 -- 172.21.15.12:0/4094468789 shutdown_connections 2024-09-11T00:30:36.861 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:36.858+0000 7fd3c4807740 1 --2- 172.21.15.12:0/4094468789 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x556d0a0869a0 0x556d0a088dc0 crc :-1 s=READY pgs=318 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:30:36.862 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:36.858+0000 7fd3c4807740 1 --2- 172.21.15.12:0/4094468789 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556d0a009ce0 0x556d0a014130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:30:36.862 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:36.858+0000 7fd3c4807740 1 --2- 172.21.15.12:0/4094468789 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556d0a0072a0 0x556d0a013560 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:30:36.862 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:36.858+0000 7fd3c4807740 1 --2- 172.21.15.12:0/4094468789 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556d0a004970 0x556d0a012ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:30:36.862 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:36.858+0000 7fd3c4807740 1 -- 172.21.15.12:0/4094468789 >> 172.21.15.12:0/4094468789 conn(0x556d09fffac0 msgr2=0x556d0a00d090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:30:36.862 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:36.858+0000 7fd3a4c00640 0 client.17970 ms_handle_reset on v2:172.21.15.165:6838/501312984 2024-09-11T00:30:36.862 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:36.859+0000 7fd3c4807740 1 -- 172.21.15.12:0/4094468789 shutdown_connections 2024-09-11T00:30:36.862 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:36.859+0000 7fd3c4807740 1 -- 172.21.15.12:0/4094468789 wait complete. 2024-09-11T00:30:36.862 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:36.859+0000 7fd3c4807740 20 client.17970 trim_cache size 0 max 16384 2024-09-11T00:30:36.863 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:{ 2024-09-11T00:30:36.863 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "message": "", 2024-09-11T00:30:36.863 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "return_code": 0 2024-09-11T00:30:36.863 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:} 2024-09-11T00:30:36.864 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:36.860+0000 7fd3bbe00640 1 -- 172.21.15.12:0/2312881619 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fd3a00b5360 msgr2=0x7fd3a00b7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:30:36.864 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:36.860+0000 7fd3bbe00640 1 --2- 172.21.15.12:0/2312881619 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fd3a00b5360 0x7fd3a00b7820 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fd3ac009fc0 tx=0x7fd3ac005f90 comp rx=0 tx=0).stop 2024-09-11T00:30:36.864 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:36.860+0000 7fd3bbe00640 1 -- 172.21.15.12:0/2312881619 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd3bc1042a0 msgr2=0x7fd3bc106300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:30:36.864 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:36.860+0000 7fd3bbe00640 1 --2- 172.21.15.12:0/2312881619 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd3bc1042a0 0x7fd3bc106300 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fd3b402adf0 tx=0x7fd3b4009780 comp rx=0 tx=0).stop 2024-09-11T00:30:36.864 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:36.860+0000 7fd3bbe00640 1 -- 172.21.15.12:0/2312881619 shutdown_connections 2024-09-11T00:30:36.864 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:36.860+0000 7fd3bbe00640 1 --2- 172.21.15.12:0/2312881619 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fd3a00b5360 0x7fd3a00b7820 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:30:36.864 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:36.860+0000 7fd3bbe00640 1 --2- 172.21.15.12:0/2312881619 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd3bc1042a0 0x7fd3bc106300 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:30:36.864 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:36.860+0000 7fd3bbe00640 1 --2- 172.21.15.12:0/2312881619 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd3bc1038e0 0x7fd3bc1056d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:30:36.864 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:36.860+0000 7fd3bbe00640 1 --2- 172.21.15.12:0/2312881619 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd3bc102790 0x7fd3bc105080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:30:36.864 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:36.860+0000 7fd3bbe00640 1 -- 172.21.15.12:0/2312881619 >> 172.21.15.12:0/2312881619 conn(0x7fd3bc0fe030 msgr2=0x7fd3bc075380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:30:36.864 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:36.860+0000 7fd3bbe00640 1 -- 172.21.15.12:0/2312881619 shutdown_connections 2024-09-11T00:30:36.865 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:36.860+0000 7fd3bbe00640 1 -- 172.21.15.12:0/2312881619 wait complete. 2024-09-11T00:30:36.923 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:36.919+0000 7f46d1400640 1 --2- 172.21.15.12:0/4220228054 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x564ccb9862b0 0x564ccb988730 crc :-1 s=READY pgs=319 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6a6ac34c64ce4f04 server_cookie=63d728dc761c53e5 in_seq=0 out_seq=0 2024-09-11T00:30:36.923 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:36.919+0000 7f46b3e00640 10 client.27970 ms_handle_connect on v2:172.21.15.165:6838/501312984 2024-09-11T00:30:36.948 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:36.945+0000 7f46b3e00640 1 -- 172.21.15.12:0/4220228054 <== mds.0 v2:172.21.15.165:6838/501312984 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x564ccb8d09e0 con 0x564ccb9862b0 2024-09-11T00:30:36.948 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:36.945+0000 7f46b3e00640 10 client.27970 handle_command_reply: tid=0 2024-09-11T00:30:36.949 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:36.945+0000 7f46db80c740 1 client.27970 shutdown 2024-09-11T00:30:36.949 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:36.946+0000 7f46db80c740 1 -- 172.21.15.12:0/4220228054 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564ccb95c920 msgr2=0x564ccb959890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:30:36.949 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:36.946+0000 7f46db80c740 1 --2- 172.21.15.12:0/4220228054 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564ccb95c920 0x564ccb959890 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f46c4106430 tx=0x7f46c40c8450 comp rx=0 tx=0).stop 2024-09-11T00:30:36.950 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:36.946+0000 7f46db80c740 1 -- 172.21.15.12:0/4220228054 shutdown_connections 2024-09-11T00:30:36.950 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:36.946+0000 7f46db80c740 1 --2- 172.21.15.12:0/4220228054 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x564ccb9862b0 0x564ccb988730 crc :-1 s=READY pgs=319 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:30:36.951 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:36.946+0000 7f46db80c740 1 --2- 172.21.15.12:0/4220228054 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564ccb9633e0 0x564ccb95a310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:30:36.951 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:36.946+0000 7f46db80c740 1 --2- 172.21.15.12:0/4220228054 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564ccb961ef0 0x564ccb959dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:30:36.951 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:36.946+0000 7f46db80c740 1 --2- 172.21.15.12:0/4220228054 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564ccb95c920 0x564ccb959890 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:30:36.951 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:36.946+0000 7f46db80c740 1 -- 172.21.15.12:0/4220228054 >> 172.21.15.12:0/4220228054 conn(0x564ccb9236f0 msgr2=0x564ccb965380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:30:36.951 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:36.947+0000 7f46b3e00640 0 client.27970 ms_handle_reset on v2:172.21.15.165:6838/501312984 2024-09-11T00:30:36.951 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:36.947+0000 7f46db80c740 1 -- 172.21.15.12:0/4220228054 shutdown_connections 2024-09-11T00:30:36.951 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:36.947+0000 7f46db80c740 1 -- 172.21.15.12:0/4220228054 wait complete. 2024-09-11T00:30:36.951 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:36.947+0000 7f46db80c740 20 client.27970 trim_cache size 0 max 16384 2024-09-11T00:30:36.977 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:36.974+0000 7f46db80c740 1 Processor -- start 2024-09-11T00:30:36.978 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:36.975+0000 7f46db80c740 1 -- start start 2024-09-11T00:30:36.978 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:36.975+0000 7f46db80c740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564ccb95c920 0x564ccb964110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:30:36.978 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:36.975+0000 7f46db80c740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564ccb961ef0 0x564ccb958ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:30:36.978 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:36.975+0000 7f46db80c740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564ccb9633e0 0x564ccb956c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:30:36.978 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:36.975+0000 7f46db80c740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f46d41ad9c0 con 0x564ccb9633e0 2024-09-11T00:30:36.979 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:36.975+0000 7f46db80c740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f46d41af560 con 0x564ccb95c920 2024-09-11T00:30:36.979 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:36.975+0000 7f46db80c740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f46d41ae0f0 con 0x564ccb961ef0 2024-09-11T00:30:36.979 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:36.975+0000 7f46d1e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564ccb95c920 0x564ccb964110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:30:36.979 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:36.976+0000 7f46d1e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564ccb95c920 0x564ccb964110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.12:38002/0 (socket says 172.21.15.12:38002) 2024-09-11T00:30:36.979 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:36.976+0000 7f46d1e00640 1 -- 172.21.15.12:0/407916310 learned_addr learned my addr 172.21.15.12:0/407916310 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:30:36.979 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:36.976+0000 7f46d2800640 1 --2- 172.21.15.12:0/407916310 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564ccb9633e0 0x564ccb956c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:30:36.980 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:36.976+0000 7f46d2800640 1 -- 172.21.15.12:0/407916310 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564ccb961ef0 msgr2=0x564ccb958ae0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-11T00:30:36.980 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:36.976+0000 7f46d2800640 1 --2- 172.21.15.12:0/407916310 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564ccb961ef0 0x564ccb958ae0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:30:36.980 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:36.976+0000 7f46d2800640 1 -- 172.21.15.12:0/407916310 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564ccb95c920 msgr2=0x564ccb964110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:30:36.980 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:36.976+0000 7f46d2800640 1 --2- 172.21.15.12:0/407916310 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564ccb95c920 0x564ccb964110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:30:36.980 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:36.976+0000 7f46d2800640 1 -- 172.21.15.12:0/407916310 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564ccb95ebc0 con 0x564ccb9633e0 2024-09-11T00:30:36.980 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:36.977+0000 7f46d2800640 1 --2- 172.21.15.12:0/407916310 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564ccb9633e0 0x564ccb956c60 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f46cc0104f0 tx=0x7f46cc013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f980313cb84716e server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:30:36.980 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:36.977+0000 7f46bea00640 1 -- 172.21.15.12:0/407916310 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46cc034070 con 0x564ccb9633e0 2024-09-11T00:30:36.980 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:36.977+0000 7f46bea00640 1 -- 172.21.15.12:0/407916310 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f46cc0099e0 con 0x564ccb9633e0 2024-09-11T00:30:36.981 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:36.977+0000 7f46bea00640 1 -- 172.21.15.12:0/407916310 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46cc019040 con 0x564ccb9633e0 2024-09-11T00:30:36.981 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:36.978+0000 7f46db80c740 1 -- 172.21.15.12:0/407916310 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564ccb9633e0 msgr2=0x564ccb956c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:30:36.981 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:36.978+0000 7f46db80c740 1 --2- 172.21.15.12:0/407916310 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564ccb9633e0 0x564ccb956c60 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f46cc0104f0 tx=0x7f46cc013000 comp rx=0 tx=0).stop 2024-09-11T00:30:36.982 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:36.980+0000 7f46db80c740 1 -- 172.21.15.12:0/407916310 shutdown_connections 2024-09-11T00:30:36.982 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:36.980+0000 7f46db80c740 1 --2- 172.21.15.12:0/407916310 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564ccb9633e0 0x564ccb956c60 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:30:36.982 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:36.980+0000 7f46db80c740 1 --2- 172.21.15.12:0/407916310 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564ccb961ef0 0x564ccb958ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:30:36.982 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:36.980+0000 7f46db80c740 1 --2- 172.21.15.12:0/407916310 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564ccb95c920 0x564ccb964110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:30:36.982 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:36.980+0000 7f46db80c740 1 -- 172.21.15.12:0/407916310 >> 172.21.15.12:0/407916310 conn(0x564ccb9236f0 msgr2=0x564ccb963a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:30:36.983 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:36.981+0000 7f46db80c740 1 -- 172.21.15.12:0/407916310 shutdown_connections 2024-09-11T00:30:36.984 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:36.981+0000 7f46db80c740 1 -- 172.21.15.12:0/407916310 wait complete. 2024-09-11T00:30:36.984 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:36.982+0000 7f46db80c740 1 -- start start 2024-09-11T00:30:36.985 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:36.982+0000 7f46db80c740 1 Processor -- start 2024-09-11T00:30:36.985 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:36.982+0000 7f46db80c740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564ccb95c920 0x564ccb97ea90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:30:36.985 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:36.982+0000 7f46db80c740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564ccb961ef0 0x564ccb97f030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:30:36.985 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:36.982+0000 7f46db80c740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564ccb9633e0 0x564ccb979800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:30:36.985 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:36.982+0000 7f46db80c740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f46d41aee90 con 0x564ccb961ef0 2024-09-11T00:30:36.985 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:36.982+0000 7f46db80c740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f46d41ae7c0 con 0x564ccb95c920 2024-09-11T00:30:36.986 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:36.982+0000 7f46db80c740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f46d41ab9a0 con 0x564ccb9633e0 2024-09-11T00:30:36.986 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:36.982+0000 7f46db80c740 10 client.0 _ll_register_callbacks cb 0x7f46c4005b70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:30:36.986 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:36.983+0000 7f46d2800640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564ccb9633e0 0x564ccb979800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:30:36.986 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:36.983+0000 7f46d2800640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564ccb9633e0 0x564ccb979800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.12:52706/0 (socket says 172.21.15.12:52706) 2024-09-11T00:30:36.986 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:36.983+0000 7f46d2800640 1 -- 172.21.15.12:0/2770481180 learned_addr learned my addr 172.21.15.12:0/2770481180 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:30:36.986 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:36.983+0000 7f46d1e00640 1 --2- 172.21.15.12:0/2770481180 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564ccb95c920 0x564ccb97ea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:30:36.987 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:36.983+0000 7f46d2800640 1 -- 172.21.15.12:0/2770481180 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564ccb95c920 msgr2=0x564ccb97ea90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:30:36.987 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:36.983+0000 7f46d2800640 1 --2- 172.21.15.12:0/2770481180 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564ccb95c920 0x564ccb97ea90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:30:36.987 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:36.983+0000 7f46d2800640 1 -- 172.21.15.12:0/2770481180 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564ccb961ef0 msgr2=0x564ccb97f030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:30:36.987 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:36.983+0000 7f46d2800640 1 --2- 172.21.15.12:0/2770481180 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564ccb961ef0 0x564ccb97f030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:30:36.987 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:36.983+0000 7f46d2800640 1 -- 172.21.15.12:0/2770481180 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564ccb97a030 con 0x564ccb9633e0 2024-09-11T00:30:36.987 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:36.983+0000 7f46d1e00640 1 --2- 172.21.15.12:0/2770481180 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564ccb95c920 0x564ccb97ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T00:30:37.004 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:37.002+0000 7f46d2800640 1 --2- 172.21.15.12:0/2770481180 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564ccb9633e0 0x564ccb979800 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f46cc004f80 tx=0x7f46cc009f20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3fdb50142f60683a server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:30:37.005 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:37.002+0000 7f46b3e00640 10 client.0 ms_handle_connect on v2:172.21.15.178:3300/0 2024-09-11T00:30:37.005 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:37.002+0000 7f46b3e00640 1 -- 172.21.15.12:0/2770481180 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46cc03d060 con 0x564ccb9633e0 2024-09-11T00:30:37.005 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:37.002+0000 7f46db80c740 1 -- 172.21.15.12:0/2770481180 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f46cc0099e0 con 0x564ccb9633e0 2024-09-11T00:30:37.005 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:37.002+0000 7f46b3e00640 1 -- 172.21.15.12:0/2770481180 <== mon.1 v2:172.21.15.178:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 9208+0+0 (secure 0 0 0) 0x7f46cc013580 con 0x564ccb9633e0 2024-09-11T00:30:37.016 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:37.013+0000 7f46b3e00640 1 -- 172.21.15.12:0/2770481180 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=102) ==== 24+0+0 (secure 0 0 0) 0x7f46cc044050 con 0x564ccb9633e0 2024-09-11T00:30:37.016 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:37.013+0000 7f46db80c740 10 client.27973 fetch_fsmap learned FSMap version 102 2024-09-11T00:30:37.017 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:37.013+0000 7f46db80c740 1 -- 172.21.15.12:0/2770481180 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,fsmap=102,monmap=4+}) -- 0x564ccb9f3f10 con 0x564ccb9633e0 2024-09-11T00:30:37.040 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:37.037+0000 7f46b3e00640 1 -- 172.21.15.12:0/2770481180 <== mon.1 v2:172.21.15.178:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f46cc0099e0 con 0x564ccb9633e0 2024-09-11T00:30:37.041 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:37.037+0000 7f46b3e00640 1 -- 172.21.15.12:0/2770481180 <== mon.1 v2:172.21.15.178:3300/0 5 ==== fsmap(e 102) ==== 8378+0+0 (secure 0 0 0) 0x7f46cc037070 con 0x564ccb9633e0 2024-09-11T00:30:37.041 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:37.038+0000 7f46db80c740 10 client.27973 fetch_fsmap finished waiting for FSMap version 102 2024-09-11T00:30:37.041 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:37.038+0000 7f46db80c740 10 client.27973 resolve_mds: validated gid 24674 aka daemon mds.b 2024-09-11T00:30:37.041 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:37.038+0000 7f46db80c740 20 client.27973 populate_metadata read hostname 'smithi012' 2024-09-11T00:30:37.042 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:37.038+0000 7f46db80c740 1 --2- 172.21.15.12:0/2770481180 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x564ccb9f44c0 0x564ccb9f68e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:30:37.042 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:37.038+0000 7f46db80c740 4 client.27973 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 138.85, "await_for": 45.0}] 2024-09-11T00:30:37.042 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:37.038+0000 7f46db80c740 1 -- 172.21.15.12:0/2770481180 --> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 138.85, "await_for": 45.0}) -- 0x564ccb8cc490 con 0x564ccb9f44c0 2024-09-11T00:30:37.043 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:37.040+0000 7f46d1e00640 1 --2- 172.21.15.12:0/2770481180 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x564ccb9f44c0 0x564ccb9f68e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:30:37.044 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:37.041+0000 7f46d1e00640 1 --2- 172.21.15.12:0/2770481180 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x564ccb9f44c0 0x564ccb9f68e0 crc :-1 s=READY pgs=320 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=33689b01e38f8c9c server_cookie=431fefa8bedfcba8 in_seq=0 out_seq=0 2024-09-11T00:30:37.044 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:37.041+0000 7f46b3e00640 10 client.27973 ms_handle_connect on v2:172.21.15.165:6838/501312984 2024-09-11T00:30:37.292 INFO:tasks.workunit.client.0.smithi012.stdout: 10 184522 15.90 MB/sec execute 513 sec latency 5151.567 ms 2024-09-11T00:30:38.217 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:30:37 smithi012 ceph-mon[30333]: pgmap v1106: 129 pgs: 129 active+clean; 923 MiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 13 MiB/s wr, 783 op/s 2024-09-11T00:30:38.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:30:37 smithi165 ceph-mon[35356]: pgmap v1106: 129 pgs: 129 active+clean; 923 MiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 13 MiB/s wr, 783 op/s 2024-09-11T00:30:38.290 INFO:tasks.workunit.client.0.smithi012.stdout: 10 184522 15.87 MB/sec execute 514 sec latency 6151.629 ms 2024-09-11T00:30:38.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:30:37 smithi178 ceph-mon[35919]: pgmap v1106: 129 pgs: 129 active+clean; 923 MiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 13 MiB/s wr, 783 op/s 2024-09-11T00:30:39.290 INFO:tasks.workunit.client.0.smithi012.stdout: 10 184522 15.84 MB/sec execute 515 sec latency 7151.930 ms 2024-09-11T00:30:40.151 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:30:39 smithi012 ceph-mon[30333]: pgmap v1107: 129 pgs: 129 active+clean; 929 MiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 12 MiB/s wr, 588 op/s 2024-09-11T00:30:40.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:30:39 smithi165 ceph-mon[35356]: pgmap v1107: 129 pgs: 129 active+clean; 929 MiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 12 MiB/s wr, 588 op/s 2024-09-11T00:30:40.290 INFO:tasks.workunit.client.0.smithi012.stdout: 10 184522 15.81 MB/sec execute 516 sec latency 8151.985 ms 2024-09-11T00:30:40.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:30:39 smithi178 ceph-mon[35919]: pgmap v1107: 129 pgs: 129 active+clean; 929 MiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 12 MiB/s wr, 588 op/s 2024-09-11T00:30:41.291 INFO:tasks.workunit.client.0.smithi012.stdout: 10 184523 15.78 MB/sec execute 517 sec latency 8645.489 ms 2024-09-11T00:30:42.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:30:41 smithi165 ceph-mon[35356]: pgmap v1108: 129 pgs: 129 active+clean; 889 MiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 12 MiB/s wr, 668 op/s 2024-09-11T00:30:42.291 INFO:tasks.workunit.client.0.smithi012.stdout: 10 184523 15.75 MB/sec execute 518 sec latency 9572.076 ms 2024-09-11T00:30:42.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:30:41 smithi012 ceph-mon[30333]: pgmap v1108: 129 pgs: 129 active+clean; 889 MiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 12 MiB/s wr, 668 op/s 2024-09-11T00:30:42.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:30:41 smithi178 ceph-mon[35919]: pgmap v1108: 129 pgs: 129 active+clean; 889 MiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 12 MiB/s wr, 668 op/s 2024-09-11T00:30:43.291 INFO:tasks.workunit.client.0.smithi012.stdout: 10 184523 15.72 MB/sec execute 519 sec latency 10572.124 ms 2024-09-11T00:30:44.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:30:43 smithi165 ceph-mon[35356]: pgmap v1109: 129 pgs: 129 active+clean; 870 MiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 12 MiB/s wr, 619 op/s 2024-09-11T00:30:44.291 INFO:tasks.workunit.client.0.smithi012.stdout: 10 184523 15.69 MB/sec execute 520 sec latency 11572.166 ms 2024-09-11T00:30:44.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:30:43 smithi012 ceph-mon[30333]: pgmap v1109: 129 pgs: 129 active+clean; 870 MiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 12 MiB/s wr, 619 op/s 2024-09-11T00:30:44.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:30:43 smithi178 ceph-mon[35919]: pgmap v1109: 129 pgs: 129 active+clean; 870 MiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 12 MiB/s wr, 619 op/s 2024-09-11T00:30:45.291 INFO:tasks.workunit.client.0.smithi012.stdout: 10 184523 15.66 MB/sec execute 521 sec latency 12572.209 ms 2024-09-11T00:30:46.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:30:45 smithi165 ceph-mon[35356]: pgmap v1110: 129 pgs: 129 active+clean; 854 MiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 11 MiB/s wr, 484 op/s 2024-09-11T00:30:46.291 INFO:tasks.workunit.client.0.smithi012.stdout: 10 184523 15.63 MB/sec execute 522 sec latency 13572.251 ms 2024-09-11T00:30:46.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:30:45 smithi012 ceph-mon[30333]: pgmap v1110: 129 pgs: 129 active+clean; 854 MiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 11 MiB/s wr, 484 op/s 2024-09-11T00:30:46.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:30:45 smithi178 ceph-mon[35919]: pgmap v1110: 129 pgs: 129 active+clean; 854 MiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 11 MiB/s wr, 484 op/s 2024-09-11T00:30:47.291 INFO:tasks.workunit.client.0.smithi012.stdout: 10 184523 15.60 MB/sec execute 523 sec latency 14572.309 ms 2024-09-11T00:30:48.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:30:47 smithi165 ceph-mon[35356]: pgmap v1111: 129 pgs: 129 active+clean; 826 MiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 9.0 MiB/s wr, 510 op/s 2024-09-11T00:30:48.291 INFO:tasks.workunit.client.0.smithi012.stdout: 10 184523 15.57 MB/sec execute 524 sec latency 15572.367 ms 2024-09-11T00:30:48.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:30:47 smithi012 ceph-mon[30333]: pgmap v1111: 129 pgs: 129 active+clean; 826 MiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 9.0 MiB/s wr, 510 op/s 2024-09-11T00:30:48.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:30:47 smithi178 ceph-mon[35919]: pgmap v1111: 129 pgs: 129 active+clean; 826 MiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 9.0 MiB/s wr, 510 op/s 2024-09-11T00:30:49.291 INFO:tasks.workunit.client.0.smithi012.stdout: 10 184523 15.54 MB/sec execute 525 sec latency 16572.427 ms 2024-09-11T00:30:49.989 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:49.987+0000 7fdb96a00640 1 -- 172.21.15.12:0/3537813795 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdb90105cf0 msgr2=0x7fdb90106170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:30:49.989 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:49.987+0000 7fdb96a00640 1 --2- 172.21.15.12:0/3537813795 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdb90105cf0 0x7fdb90106170 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7fdb7c009930 tx=0x7fdb7c02a440 comp rx=0 tx=0).stop 2024-09-11T00:30:49.990 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:49.987+0000 7fdb96a00640 1 -- 172.21.15.12:0/3537813795 shutdown_connections 2024-09-11T00:30:49.990 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:49.987+0000 7fdb96a00640 1 --2- 172.21.15.12:0/3537813795 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fdb901066b0 0x7fdb9010d030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:30:49.990 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:49.987+0000 7fdb96a00640 1 --2- 172.21.15.12:0/3537813795 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdb90105cf0 0x7fdb90106170 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:30:49.990 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:49.987+0000 7fdb96a00640 1 --2- 172.21.15.12:0/3537813795 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdb90104ba0 0x7fdb90104fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:30:49.990 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:49.987+0000 7fdb96a00640 1 -- 172.21.15.12:0/3537813795 >> 172.21.15.12:0/3537813795 conn(0x7fdb90100460 msgr2=0x7fdb90102880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:30:49.990 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:49.987+0000 7fdb96a00640 1 -- 172.21.15.12:0/3537813795 shutdown_connections 2024-09-11T00:30:49.990 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:49.987+0000 7fdb96a00640 1 -- 172.21.15.12:0/3537813795 wait complete. 2024-09-11T00:30:49.991 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:49.988+0000 7fdb96a00640 1 Processor -- start 2024-09-11T00:30:49.992 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:49.988+0000 7fdb96a00640 1 -- start start 2024-09-11T00:30:49.992 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:49.988+0000 7fdb96a00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdb90104ba0 0x7fdb9019b010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:30:49.992 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:49.988+0000 7fdb96a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fdb90105cf0 0x7fdb9019b660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:30:49.992 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:49.988+0000 7fdb96a00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdb901066b0 0x7fdb9019c290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:30:49.992 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:49.988+0000 7fdb96a00640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fdb90110890 con 0x7fdb901066b0 2024-09-11T00:30:49.992 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:49.988+0000 7fdb96a00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fdb90110710 con 0x7fdb90105cf0 2024-09-11T00:30:49.992 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:49.988+0000 7fdb96a00640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fdb90110a10 con 0x7fdb90104ba0 2024-09-11T00:30:49.992 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:49.989+0000 7fdb94a00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdb901066b0 0x7fdb9019c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:30:49.992 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:49.989+0000 7fdb94a00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdb901066b0 0x7fdb9019c290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:55980/0 (socket says 172.21.15.12:55980) 2024-09-11T00:30:49.992 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:49.989+0000 7fdb94a00640 1 -- 172.21.15.12:0/2534831202 learned_addr learned my addr 172.21.15.12:0/2534831202 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:30:49.992 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:49.989+0000 7fdb8f400640 1 --2- 172.21.15.12:0/2534831202 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fdb90105cf0 0x7fdb9019b660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:30:49.992 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:49.989+0000 7fdb8fe00640 1 --2- 172.21.15.12:0/2534831202 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdb90104ba0 0x7fdb9019b010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:30:49.993 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:49.989+0000 7fdb94a00640 1 -- 172.21.15.12:0/2534831202 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdb90104ba0 msgr2=0x7fdb9019b010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:30:49.993 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:49.989+0000 7fdb94a00640 1 --2- 172.21.15.12:0/2534831202 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdb90104ba0 0x7fdb9019b010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:30:49.993 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:49.989+0000 7fdb94a00640 1 -- 172.21.15.12:0/2534831202 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fdb90105cf0 msgr2=0x7fdb9019b660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:30:49.993 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:49.989+0000 7fdb94a00640 1 --2- 172.21.15.12:0/2534831202 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fdb90105cf0 0x7fdb9019b660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:30:49.993 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:49.989+0000 7fdb94a00640 1 -- 172.21.15.12:0/2534831202 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb9019cb50 con 0x7fdb901066b0 2024-09-11T00:30:49.993 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:49.989+0000 7fdb94a00640 1 --2- 172.21.15.12:0/2534831202 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdb901066b0 0x7fdb9019c290 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7fdb8800eb10 tx=0x7fdb8800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:30:49.993 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:49.989+0000 7fdb8cc00640 1 -- 172.21.15.12:0/2534831202 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb88004b20 con 0x7fdb901066b0 2024-09-11T00:30:49.994 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:49.989+0000 7fdb96a00640 1 -- 172.21.15.12:0/2534831202 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdb9019ce40 con 0x7fdb901066b0 2024-09-11T00:30:49.994 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:49.989+0000 7fdb8cc00640 1 -- 172.21.15.12:0/2534831202 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdb88004cc0 con 0x7fdb901066b0 2024-09-11T00:30:49.994 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:49.989+0000 7fdb8cc00640 1 -- 172.21.15.12:0/2534831202 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb88018630 con 0x7fdb901066b0 2024-09-11T00:30:49.994 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:49.989+0000 7fdb96a00640 1 -- 172.21.15.12:0/2534831202 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdb901a3510 con 0x7fdb901066b0 2024-09-11T00:30:49.996 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:49.993+0000 7fdb997f4740 1 Processor -- start 2024-09-11T00:30:49.997 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:49.993+0000 7fdb8cc00640 1 -- 172.21.15.12:0/2534831202 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7fdb88005000 con 0x7fdb901066b0 2024-09-11T00:30:49.997 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:49.994+0000 7fdb997f4740 1 -- start start 2024-09-11T00:30:49.997 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:49.994+0000 7fdb997f4740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55610b745790 0x55610b747b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:30:49.997 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:49.994+0000 7fdb997f4740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55610b7480c0 0x55610b74a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:30:49.997 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:49.994+0000 7fdb997f4740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55610b74ab00 0x55610b74cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:30:49.998 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:49.994+0000 7fdb997f4740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55610b6b86b0 con 0x55610b7480c0 2024-09-11T00:30:49.999 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:49.994+0000 7fdb997f4740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55610b6a7ae0 con 0x55610b74ab00 2024-09-11T00:30:49.999 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:49.994+0000 7fdb997f4740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55610b6ab8f0 con 0x55610b745790 2024-09-11T00:30:49.999 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:49.994+0000 7fdb8fe00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55610b7480c0 0x55610b74a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:30:49.999 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:49.994+0000 7fdb8f400640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55610b745790 0x55610b747b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:30:49.999 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:49.994+0000 7fdb8fe00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55610b7480c0 0x55610b74a4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:55986/0 (socket says 172.21.15.12:55986) 2024-09-11T00:30:49.999 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:49.994+0000 7fdb8fe00640 1 -- 172.21.15.12:0/764175087 learned_addr learned my addr 172.21.15.12:0/764175087 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:30:49.999 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:49.994+0000 7fdb8f400640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55610b745790 0x55610b747b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.12:45846/0 (socket says 172.21.15.12:45846) 2024-09-11T00:30:49.999 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:49.995+0000 7fdb8f400640 1 --2- 172.21.15.12:0/764175087 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55610b74ab00 0x55610b74cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:30:49.999 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:49.995+0000 7fdb8fe00640 1 -- 172.21.15.12:0/764175087 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55610b745790 msgr2=0x55610b747b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:30:49.999 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:49.995+0000 7fdb8fe00640 1 --2- 172.21.15.12:0/764175087 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55610b745790 0x55610b747b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:30:49.999 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:49.995+0000 7fdb8fe00640 1 -- 172.21.15.12:0/764175087 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55610b74ab00 msgr2=0x55610b74cfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:30:50.000 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:49.995+0000 7fdb8fe00640 1 --2- 172.21.15.12:0/764175087 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55610b74ab00 0x55610b74cfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:30:50.000 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:49.995+0000 7fdb8fe00640 1 -- 172.21.15.12:0/764175087 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55610b674780 con 0x55610b7480c0 2024-09-11T00:30:50.000 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:49.995+0000 7fdb8cc00640 1 --2- 172.21.15.12:0/2534831202 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fdb740b52e0 0x7fdb740b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:30:50.000 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:49.995+0000 7fdb8cc00640 1 -- 172.21.15.12:0/2534831202 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9208+0+0 (secure 0 0 0) 0x7fdb880e66e0 con 0x7fdb901066b0 2024-09-11T00:30:50.000 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:49.995+0000 7fdb8fe00640 1 --2- 172.21.15.12:0/764175087 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55610b7480c0 0x55610b74a4b0 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7fdb80009a40 tx=0x7fdb80009f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=faaae404e8184bb4 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:30:50.000 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:49.995+0000 7fdb8f400640 1 --2- 172.21.15.12:0/2534831202 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fdb740b52e0 0x7fdb740b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:30:50.000 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:49.995+0000 7fdb85200640 1 -- 172.21.15.12:0/764175087 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb80017070 con 0x55610b7480c0 2024-09-11T00:30:50.000 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:49.995+0000 7fdb85200640 1 -- 172.21.15.12:0/764175087 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdb80013470 con 0x55610b7480c0 2024-09-11T00:30:50.000 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:49.995+0000 7fdb85200640 1 -- 172.21.15.12:0/764175087 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb80012630 con 0x55610b7480c0 2024-09-11T00:30:50.000 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:49.995+0000 7fdb8f400640 1 --2- 172.21.15.12:0/2534831202 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fdb740b52e0 0x7fdb740b77a0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7fdb7c006010 tx=0x7fdb7c0314a0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:30:50.001 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:49.997+0000 7fdb997f4740 1 -- 172.21.15.12:0/764175087 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55610b7480c0 msgr2=0x55610b74a4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:30:50.001 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:49.997+0000 7fdb997f4740 1 --2- 172.21.15.12:0/764175087 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55610b7480c0 0x55610b74a4b0 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7fdb80009a40 tx=0x7fdb80009f00 comp rx=0 tx=0).stop 2024-09-11T00:30:50.002 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:49.997+0000 7fdb997f4740 1 -- 172.21.15.12:0/764175087 shutdown_connections 2024-09-11T00:30:50.002 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:49.997+0000 7fdb997f4740 1 --2- 172.21.15.12:0/764175087 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55610b74ab00 0x55610b74cfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:30:50.003 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:49.997+0000 7fdb997f4740 1 --2- 172.21.15.12:0/764175087 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55610b7480c0 0x55610b74a4b0 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:30:50.003 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:49.997+0000 7fdb997f4740 1 --2- 172.21.15.12:0/764175087 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55610b745790 0x55610b747b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:30:50.003 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:49.997+0000 7fdb997f4740 1 -- 172.21.15.12:0/764175087 >> 172.21.15.12:0/764175087 conn(0x55610b7408e0 msgr2=0x55610b741670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:30:50.003 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:49.997+0000 7fdb997f4740 1 -- 172.21.15.12:0/764175087 shutdown_connections 2024-09-11T00:30:50.003 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:49.997+0000 7fdb997f4740 1 -- 172.21.15.12:0/764175087 wait complete. 2024-09-11T00:30:50.003 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:49.997+0000 7fdb997f4740 1 -- start start 2024-09-11T00:30:50.003 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:49.997+0000 7fdb997f4740 1 Processor -- start 2024-09-11T00:30:50.003 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:49.997+0000 7fdb997f4740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55610b745790 0x55610b74eec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:30:50.003 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:49.997+0000 7fdb997f4740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55610b7480c0 0x55610b74f560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:30:50.003 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:49.997+0000 7fdb997f4740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55610b74ab00 0x55610b750130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:30:50.003 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:49.997+0000 7fdb997f4740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55610b5dfab0 con 0x55610b745790 2024-09-11T00:30:50.003 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:49.997+0000 7fdb997f4740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55610b64e570 con 0x55610b7480c0 2024-09-11T00:30:50.004 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:49.997+0000 7fdb997f4740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55610b74dae0 con 0x55610b74ab00 2024-09-11T00:30:50.004 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:49.997+0000 7fdb997f4740 10 client.0 _ll_register_callbacks cb 0x55610b65d100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:30:50.004 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:49.997+0000 7fdb8fe00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55610b745790 0x55610b74eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:30:50.004 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:49.997+0000 7fdb8fe00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55610b745790 0x55610b74eec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:56000/0 (socket says 172.21.15.12:56000) 2024-09-11T00:30:50.004 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:49.997+0000 7fdb8f400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55610b7480c0 0x55610b74f560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:30:50.004 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:49.997+0000 7fdb8fe00640 1 -- 172.21.15.12:0/531669791 learned_addr learned my addr 172.21.15.12:0/531669791 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:30:50.004 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:49.997+0000 7fdb94a00640 1 --2- 172.21.15.12:0/531669791 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55610b74ab00 0x55610b750130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:30:50.004 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:50.000+0000 7fdb8fe00640 1 -- 172.21.15.12:0/531669791 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55610b74ab00 msgr2=0x55610b750130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:30:50.004 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:50.000+0000 7fdb8fe00640 1 --2- 172.21.15.12:0/531669791 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55610b74ab00 0x55610b750130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:30:50.004 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:50.000+0000 7fdb8fe00640 1 -- 172.21.15.12:0/531669791 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55610b7480c0 msgr2=0x55610b74f560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:30:50.004 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:50.000+0000 7fdb8fe00640 1 --2- 172.21.15.12:0/531669791 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55610b7480c0 0x55610b74f560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:30:50.004 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:50.000+0000 7fdb8fe00640 1 -- 172.21.15.12:0/531669791 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55610b409790 con 0x55610b745790 2024-09-11T00:30:50.004 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:50.000+0000 7fdb8fe00640 1 --2- 172.21.15.12:0/531669791 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55610b745790 0x55610b74eec0 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7fdb8000a010 tx=0x7fdb8000ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0d4e7855007a3ca server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:30:50.005 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:50.000+0000 7fdb79600640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-09-11T00:30:50.007 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:50.000+0000 7fdb79600640 1 -- 172.21.15.12:0/531669791 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb80017050 con 0x55610b745790 2024-09-11T00:30:50.007 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:50.000+0000 7fdb79600640 1 -- 172.21.15.12:0/531669791 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 9208+0+0 (secure 0 0 0) 0x7fdb8001c450 con 0x55610b745790 2024-09-11T00:30:50.007 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:50.000+0000 7fdb997f4740 1 -- 172.21.15.12:0/531669791 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdb80013470 con 0x55610b745790 2024-09-11T00:30:50.007 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:50.000+0000 7fdb79600640 1 -- 172.21.15.12:0/531669791 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=102) ==== 24+0+0 (secure 0 0 0) 0x7fdb80013a30 con 0x55610b745790 2024-09-11T00:30:50.007 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:50.000+0000 7fdb997f4740 10 client.17994 fetch_fsmap learned FSMap version 102 2024-09-11T00:30:50.007 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:50.000+0000 7fdb997f4740 1 -- 172.21.15.12:0/531669791 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=102,monmap=4+}) -- 0x55610b482620 con 0x55610b745790 2024-09-11T00:30:50.007 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:50.000+0000 7fdb79600640 1 -- 172.21.15.12:0/531669791 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdb80013470 con 0x55610b745790 2024-09-11T00:30:50.007 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:50.000+0000 7fdb79600640 1 -- 172.21.15.12:0/531669791 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 102) ==== 8378+0+0 (secure 0 0 0) 0x7fdb80026710 con 0x55610b745790 2024-09-11T00:30:50.007 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:50.000+0000 7fdb997f4740 10 client.17994 fetch_fsmap finished waiting for FSMap version 102 2024-09-11T00:30:50.008 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:50.000+0000 7fdb997f4740 10 client.17994 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T00:30:50.008 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:50.000+0000 7fdb997f4740 20 client.17994 populate_metadata read hostname 'smithi012' 2024-09-11T00:30:50.008 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:50.000+0000 7fdb997f4740 1 --2- 172.21.15.12:0/531669791 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55610b75a470 0x55610b75c860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:30:50.008 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:50.000+0000 7fdb997f4740 4 client.17994 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T00:30:50.008 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:50.000+0000 7fdb997f4740 1 -- 172.21.15.12:0/531669791 --> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55610b6ba600 con 0x55610b75a470 2024-09-11T00:30:50.008 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:50.000+0000 7fdb8f400640 1 --2- 172.21.15.12:0/531669791 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55610b75a470 0x55610b75c860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:30:50.008 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:50.000+0000 7fdb8f400640 1 --2- 172.21.15.12:0/531669791 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55610b75a470 0x55610b75c860 crc :-1 s=READY pgs=321 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=be9e883339888a1b server_cookie=93b11c18d2954d44 in_seq=0 out_seq=0 2024-09-11T00:30:50.008 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:50.000+0000 7fdb79600640 10 client.17994 ms_handle_connect on v2:172.21.15.165:6838/501312984 2024-09-11T00:30:50.008 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:50.000+0000 7fdb79600640 1 -- 172.21.15.12:0/531669791 <== mds.0 v2:172.21.15.165:6838/501312984 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55610b6ba600 con 0x55610b75a470 2024-09-11T00:30:50.008 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:50.000+0000 7fdb79600640 10 client.17994 handle_command_reply: tid=0 2024-09-11T00:30:50.008 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:50.000+0000 7fdb997f4740 1 client.17994 shutdown 2024-09-11T00:30:50.008 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:50.000+0000 7fdb997f4740 1 -- 172.21.15.12:0/531669791 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55610b745790 msgr2=0x55610b74eec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:30:50.009 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:50.000+0000 7fdb997f4740 1 --2- 172.21.15.12:0/531669791 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55610b745790 0x55610b74eec0 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7fdb8000a010 tx=0x7fdb8000ed70 comp rx=0 tx=0).stop 2024-09-11T00:30:50.009 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:50.000+0000 7fdb997f4740 1 -- 172.21.15.12:0/531669791 shutdown_connections 2024-09-11T00:30:50.009 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:50.000+0000 7fdb997f4740 1 --2- 172.21.15.12:0/531669791 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55610b75a470 0x55610b75c860 crc :-1 s=READY pgs=321 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:30:50.009 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:50.000+0000 7fdb997f4740 1 --2- 172.21.15.12:0/531669791 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55610b74ab00 0x55610b750130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:30:50.009 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:50.000+0000 7fdb997f4740 1 --2- 172.21.15.12:0/531669791 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55610b7480c0 0x55610b74f560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:30:50.009 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:50.000+0000 7fdb79600640 0 client.17994 ms_handle_reset on v2:172.21.15.165:6838/501312984 2024-09-11T00:30:50.009 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:50.000+0000 7fdb997f4740 1 --2- 172.21.15.12:0/531669791 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55610b745790 0x55610b74eec0 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:30:50.009 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:50.000+0000 7fdb997f4740 1 -- 172.21.15.12:0/531669791 >> 172.21.15.12:0/531669791 conn(0x55610b7408e0 msgr2=0x55610b73fa60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:30:50.009 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:50.000+0000 7fdb997f4740 1 -- 172.21.15.12:0/531669791 shutdown_connections 2024-09-11T00:30:50.009 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:50.000+0000 7fdb997f4740 1 -- 172.21.15.12:0/531669791 wait complete. 2024-09-11T00:30:50.009 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:50.000+0000 7fdb997f4740 20 client.17994 trim_cache size 0 max 16384 2024-09-11T00:30:50.022 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:50.017+0000 7fdb997f4740 1 Processor -- start 2024-09-11T00:30:50.023 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:50.017+0000 7fdb997f4740 1 -- start start 2024-09-11T00:30:50.023 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:50.017+0000 7fdb997f4740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55610b745790 0x55610b74af30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:30:50.023 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:50.017+0000 7fdb997f4740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55610b7480c0 0x55610b74b470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:30:50.023 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:50.017+0000 7fdb997f4740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55610b74ab00 0x55610b742710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:30:50.023 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:50.017+0000 7fdb997f4740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fdb901ada30 con 0x55610b7480c0 2024-09-11T00:30:50.023 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:50.017+0000 7fdb997f4740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fdb901af5d0 con 0x55610b74ab00 2024-09-11T00:30:50.023 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:50.017+0000 7fdb997f4740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fdb901ae160 con 0x55610b745790 2024-09-11T00:30:50.023 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:50.017+0000 7fdb8f400640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55610b7480c0 0x55610b74b470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:30:50.023 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:50.017+0000 7fdb8f400640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55610b7480c0 0x55610b74b470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:56002/0 (socket says 172.21.15.12:56002) 2024-09-11T00:30:50.023 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:50.017+0000 7fdb8f400640 1 -- 172.21.15.12:0/421527514 learned_addr learned my addr 172.21.15.12:0/421527514 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:30:50.023 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:50.017+0000 7fdb8fe00640 1 --2- 172.21.15.12:0/421527514 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55610b745790 0x55610b74af30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:30:50.023 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:50.017+0000 7fdb8f400640 1 -- 172.21.15.12:0/421527514 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55610b745790 msgr2=0x55610b74af30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:30:50.023 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:50.017+0000 7fdb8f400640 1 --2- 172.21.15.12:0/421527514 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55610b745790 0x55610b74af30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:30:50.024 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:50.017+0000 7fdb8f400640 1 -- 172.21.15.12:0/421527514 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55610b74ab00 msgr2=0x55610b742710 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-11T00:30:50.024 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:50.017+0000 7fdb8f400640 1 --2- 172.21.15.12:0/421527514 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55610b74ab00 0x55610b742710 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:30:50.024 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:50.017+0000 7fdb8f400640 1 -- 172.21.15.12:0/421527514 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55610b747ef0 con 0x55610b7480c0 2024-09-11T00:30:50.027 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:50.017+0000 7fdb8f400640 1 --2- 172.21.15.12:0/421527514 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55610b7480c0 0x55610b74b470 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7fdb7c031a90 tx=0x7fdb7c02a440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58e39163a52f94af server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:30:50.027 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:50.017+0000 7fdb7be00640 1 -- 172.21.15.12:0/421527514 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb7c035040 con 0x55610b7480c0 2024-09-11T00:30:50.027 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:50.017+0000 7fdb7be00640 1 -- 172.21.15.12:0/421527514 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdb7c004240 con 0x55610b7480c0 2024-09-11T00:30:50.027 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:50.017+0000 7fdb7be00640 1 -- 172.21.15.12:0/421527514 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb7c057760 con 0x55610b7480c0 2024-09-11T00:30:50.027 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:50.022+0000 7fdb997f4740 1 -- 172.21.15.12:0/421527514 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55610b7480c0 msgr2=0x55610b74b470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:30:50.027 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:50.022+0000 7fdb997f4740 1 --2- 172.21.15.12:0/421527514 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55610b7480c0 0x55610b74b470 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7fdb7c031a90 tx=0x7fdb7c02a440 comp rx=0 tx=0).stop 2024-09-11T00:30:50.027 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:50.022+0000 7fdb997f4740 1 -- 172.21.15.12:0/421527514 shutdown_connections 2024-09-11T00:30:50.027 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:50.022+0000 7fdb997f4740 1 --2- 172.21.15.12:0/421527514 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55610b74ab00 0x55610b742710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:30:50.027 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:50.022+0000 7fdb997f4740 1 --2- 172.21.15.12:0/421527514 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55610b7480c0 0x55610b74b470 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:30:50.027 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:50.022+0000 7fdb997f4740 1 --2- 172.21.15.12:0/421527514 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55610b745790 0x55610b74af30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:30:50.028 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:50.022+0000 7fdb997f4740 1 -- 172.21.15.12:0/421527514 >> 172.21.15.12:0/421527514 conn(0x55610b7408e0 msgr2=0x55610b74e060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:30:50.028 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:50.022+0000 7fdb997f4740 1 -- 172.21.15.12:0/421527514 shutdown_connections 2024-09-11T00:30:50.028 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:50.022+0000 7fdb997f4740 1 -- 172.21.15.12:0/421527514 wait complete. 2024-09-11T00:30:50.028 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:50.022+0000 7fdb997f4740 1 -- start start 2024-09-11T00:30:50.028 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:50.022+0000 7fdb997f4740 1 Processor -- start 2024-09-11T00:30:50.028 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:50.023+0000 7fdb997f4740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55610b745790 0x55610b756b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:30:50.028 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:50.023+0000 7fdb997f4740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55610b7480c0 0x55610b755850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:30:50.028 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:50.023+0000 7fdb997f4740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55610b74ab00 0x55610b755f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:30:50.028 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:50.023+0000 7fdb997f4740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fdb901aef00 con 0x55610b7480c0 2024-09-11T00:30:50.028 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:50.023+0000 7fdb997f4740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fdb901ae830 con 0x55610b745790 2024-09-11T00:30:50.028 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:50.023+0000 7fdb997f4740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fdb901aba10 con 0x55610b74ab00 2024-09-11T00:30:50.029 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:50.023+0000 7fdb997f4740 10 client.0 _ll_register_callbacks cb 0x7fdb7c032cf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:30:50.029 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:50.023+0000 7fdb8f400640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55610b7480c0 0x55610b755850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:30:50.029 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:50.023+0000 7fdb8fe00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55610b745790 0x55610b756b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:30:50.029 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:50.023+0000 7fdb8f400640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55610b7480c0 0x55610b755850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:56004/0 (socket says 172.21.15.12:56004) 2024-09-11T00:30:50.029 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:50.023+0000 7fdb8f400640 1 -- 172.21.15.12:0/3261664173 learned_addr learned my addr 172.21.15.12:0/3261664173 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:30:50.029 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:50.023+0000 7fdb94a00640 1 --2- 172.21.15.12:0/3261664173 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55610b74ab00 0x55610b755f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:30:50.029 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:50.023+0000 7fdb8f400640 1 -- 172.21.15.12:0/3261664173 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55610b74ab00 msgr2=0x55610b755f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:30:50.029 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:50.023+0000 7fdb8f400640 1 --2- 172.21.15.12:0/3261664173 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55610b74ab00 0x55610b755f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:30:50.029 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:50.023+0000 7fdb8f400640 1 -- 172.21.15.12:0/3261664173 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55610b745790 msgr2=0x55610b756b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:30:50.029 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:50.023+0000 7fdb8f400640 1 --2- 172.21.15.12:0/3261664173 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55610b745790 0x55610b756b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:30:50.029 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:50.023+0000 7fdb8f400640 1 -- 172.21.15.12:0/3261664173 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55610b754370 con 0x55610b7480c0 2024-09-11T00:30:50.030 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:50.023+0000 7fdb8fe00640 1 --2- 172.21.15.12:0/3261664173 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55610b745790 0x55610b756b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:30:50.030 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:50.023+0000 7fdb8f400640 1 --2- 172.21.15.12:0/3261664173 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55610b7480c0 0x55610b755850 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7fdb7c0092c0 tx=0x7fdb7c0043f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d25cb66e40438fa server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:30:50.030 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:50.023+0000 7fdb79600640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-09-11T00:30:50.030 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:50.023+0000 7fdb79600640 1 -- 172.21.15.12:0/3261664173 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb7c035040 con 0x55610b7480c0 2024-09-11T00:30:50.030 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:50.023+0000 7fdb79600640 1 -- 172.21.15.12:0/3261664173 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 9208+0+0 (secure 0 0 0) 0x7fdb7c0036b0 con 0x55610b7480c0 2024-09-11T00:30:50.030 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:50.023+0000 7fdb997f4740 1 -- 172.21.15.12:0/3261664173 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdb7c004240 con 0x55610b7480c0 2024-09-11T00:30:50.030 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:50.023+0000 7fdb79600640 1 -- 172.21.15.12:0/3261664173 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=102) ==== 24+0+0 (secure 0 0 0) 0x7fdb7c070050 con 0x55610b7480c0 2024-09-11T00:30:50.030 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:50.023+0000 7fdb997f4740 10 client.18000 fetch_fsmap learned FSMap version 102 2024-09-11T00:30:50.030 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:50.023+0000 7fdb997f4740 1 -- 172.21.15.12:0/3261664173 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=102,monmap=4+}) -- 0x55610b7c74c0 con 0x55610b7480c0 2024-09-11T00:30:50.031 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:50.023+0000 7fdb79600640 1 -- 172.21.15.12:0/3261664173 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdb7c004240 con 0x55610b7480c0 2024-09-11T00:30:50.031 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:50.023+0000 7fdb79600640 1 -- 172.21.15.12:0/3261664173 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 102) ==== 8378+0+0 (secure 0 0 0) 0x7fdb7c057c30 con 0x55610b7480c0 2024-09-11T00:30:50.031 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:50.023+0000 7fdb997f4740 10 client.18000 fetch_fsmap finished waiting for FSMap version 102 2024-09-11T00:30:50.031 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:50.023+0000 7fdb997f4740 10 client.18000 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T00:30:50.031 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:50.023+0000 7fdb997f4740 20 client.18000 populate_metadata read hostname 'smithi012' 2024-09-11T00:30:50.031 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:50.023+0000 7fdb997f4740 1 --2- 172.21.15.12:0/3261664173 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55610b7c7ec0 0x55610b7ca2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:30:50.032 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:50.023+0000 7fdb997f4740 4 client.18000 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T00:30:50.032 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:50.023+0000 7fdb997f4740 1 -- 172.21.15.12:0/3261664173 --> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55610b6b60b0 con 0x55610b7c7ec0 2024-09-11T00:30:50.032 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:50.023+0000 7fdb8fe00640 1 --2- 172.21.15.12:0/3261664173 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55610b7c7ec0 0x55610b7ca2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:30:50.032 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:50.023+0000 7fdb8fe00640 1 --2- 172.21.15.12:0/3261664173 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55610b7c7ec0 0x55610b7ca2e0 crc :-1 s=READY pgs=322 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=347996c4e34d1d12 server_cookie=27fc3de422963c38 in_seq=0 out_seq=0 2024-09-11T00:30:50.032 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:50.023+0000 7fdb79600640 10 client.18000 ms_handle_connect on v2:172.21.15.165:6838/501312984 2024-09-11T00:30:50.082 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:50.079+0000 7fdb79600640 1 -- 172.21.15.12:0/3261664173 <== mds.0 v2:172.21.15.165:6838/501312984 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55610b6b60b0 con 0x55610b7c7ec0 2024-09-11T00:30:50.082 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:50.079+0000 7fdb79600640 10 client.18000 handle_command_reply: tid=0 2024-09-11T00:30:50.082 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:50.079+0000 7fdb997f4740 1 client.18000 shutdown 2024-09-11T00:30:50.082 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:50.079+0000 7fdb997f4740 1 -- 172.21.15.12:0/3261664173 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55610b7480c0 msgr2=0x55610b755850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:30:50.082 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:50.079+0000 7fdb997f4740 1 --2- 172.21.15.12:0/3261664173 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55610b7480c0 0x55610b755850 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7fdb7c0092c0 tx=0x7fdb7c0043f0 comp rx=0 tx=0).stop 2024-09-11T00:30:50.082 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:50.079+0000 7fdb997f4740 1 -- 172.21.15.12:0/3261664173 shutdown_connections 2024-09-11T00:30:50.082 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:50.079+0000 7fdb997f4740 1 --2- 172.21.15.12:0/3261664173 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55610b7c7ec0 0x55610b7ca2e0 crc :-1 s=READY pgs=322 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:30:50.083 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:50.079+0000 7fdb997f4740 1 --2- 172.21.15.12:0/3261664173 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55610b74ab00 0x55610b755f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:30:50.083 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:50.079+0000 7fdb997f4740 1 --2- 172.21.15.12:0/3261664173 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55610b7480c0 0x55610b755850 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:30:50.083 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:50.079+0000 7fdb997f4740 1 --2- 172.21.15.12:0/3261664173 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55610b745790 0x55610b756b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:30:50.083 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:50.079+0000 7fdb997f4740 1 -- 172.21.15.12:0/3261664173 >> 172.21.15.12:0/3261664173 conn(0x55610b7408e0 msgr2=0x55610b744bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:30:50.083 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:50.079+0000 7fdb79600640 0 client.18000 ms_handle_reset on v2:172.21.15.165:6838/501312984 2024-09-11T00:30:50.083 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:50.079+0000 7fdb997f4740 1 -- 172.21.15.12:0/3261664173 shutdown_connections 2024-09-11T00:30:50.083 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:50.079+0000 7fdb997f4740 1 -- 172.21.15.12:0/3261664173 wait complete. 2024-09-11T00:30:50.083 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:50.079+0000 7fdb997f4740 20 client.18000 trim_cache size 0 max 16384 2024-09-11T00:30:50.083 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:{ 2024-09-11T00:30:50.084 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "message": "", 2024-09-11T00:30:50.084 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "return_code": 0 2024-09-11T00:30:50.084 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:} 2024-09-11T00:30:50.084 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:50.079+0000 7fdb96a00640 1 -- 172.21.15.12:0/2534831202 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fdb740b52e0 msgr2=0x7fdb740b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:30:50.084 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:50.079+0000 7fdb96a00640 1 --2- 172.21.15.12:0/2534831202 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fdb740b52e0 0x7fdb740b77a0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7fdb7c006010 tx=0x7fdb7c0314a0 comp rx=0 tx=0).stop 2024-09-11T00:30:50.084 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:50.079+0000 7fdb96a00640 1 -- 172.21.15.12:0/2534831202 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdb901066b0 msgr2=0x7fdb9019c290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:30:50.084 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:50.079+0000 7fdb96a00640 1 --2- 172.21.15.12:0/2534831202 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdb901066b0 0x7fdb9019c290 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7fdb8800eb10 tx=0x7fdb8800c2d0 comp rx=0 tx=0).stop 2024-09-11T00:30:50.085 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:50.079+0000 7fdb96a00640 1 -- 172.21.15.12:0/2534831202 shutdown_connections 2024-09-11T00:30:50.085 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:50.079+0000 7fdb96a00640 1 --2- 172.21.15.12:0/2534831202 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fdb740b52e0 0x7fdb740b77a0 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:30:50.085 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:50.079+0000 7fdb96a00640 1 --2- 172.21.15.12:0/2534831202 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdb901066b0 0x7fdb9019c290 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:30:50.085 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:50.079+0000 7fdb96a00640 1 --2- 172.21.15.12:0/2534831202 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fdb90105cf0 0x7fdb9019b660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:30:50.085 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:50.079+0000 7fdb96a00640 1 --2- 172.21.15.12:0/2534831202 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdb90104ba0 0x7fdb9019b010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:30:50.085 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:50.079+0000 7fdb96a00640 1 -- 172.21.15.12:0/2534831202 >> 172.21.15.12:0/2534831202 conn(0x7fdb90100460 msgr2=0x7fdb901090e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:30:50.085 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:50.079+0000 7fdb96a00640 1 -- 172.21.15.12:0/2534831202 shutdown_connections 2024-09-11T00:30:50.085 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:30:50.079+0000 7fdb96a00640 1 -- 172.21.15.12:0/2534831202 wait complete. 2024-09-11T00:30:50.120 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:30:49 smithi012 ceph-mon[30333]: pgmap v1112: 129 pgs: 129 active+clean; 826 MiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 310 KiB/s rd, 905 KiB/s wr, 256 op/s 2024-09-11T00:30:50.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:30:49 smithi165 ceph-mon[35356]: pgmap v1112: 129 pgs: 129 active+clean; 826 MiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 310 KiB/s rd, 905 KiB/s wr, 256 op/s 2024-09-11T00:30:50.291 INFO:tasks.workunit.client.0.smithi012.stdout: 10 184523 15.51 MB/sec execute 526 sec latency 17572.488 ms 2024-09-11T00:30:50.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:30:49 smithi178 ceph-mon[35919]: pgmap v1112: 129 pgs: 129 active+clean; 826 MiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 310 KiB/s rd, 905 KiB/s wr, 256 op/s 2024-09-11T00:30:51.291 INFO:tasks.workunit.client.0.smithi012.stdout: 10 184523 15.48 MB/sec execute 527 sec latency 18572.542 ms 2024-09-11T00:30:52.075 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:30:52 smithi012 ceph-mon[30333]: pgmap v1113: 129 pgs: 129 active+clean; 826 MiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 289 KiB/s rd, 384 KiB/s wr, 237 op/s 2024-09-11T00:30:52.283 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:30:52 smithi165 ceph-mon[35356]: pgmap v1113: 129 pgs: 129 active+clean; 826 MiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 289 KiB/s rd, 384 KiB/s wr, 237 op/s 2024-09-11T00:30:52.291 INFO:tasks.workunit.client.0.smithi012.stdout: 10 184524 15.45 MB/sec execute 528 sec latency 19199.170 ms 2024-09-11T00:30:52.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:30:52 smithi178 ceph-mon[35919]: pgmap v1113: 129 pgs: 129 active+clean; 826 MiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 289 KiB/s rd, 384 KiB/s wr, 237 op/s 2024-09-11T00:30:53.291 INFO:tasks.workunit.client.0.smithi012.stdout: 10 184527 15.42 MB/sec execute 529 sec latency 19120.105 ms 2024-09-11T00:30:54.030 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:30:54 smithi012 ceph-mon[30333]: pgmap v1114: 129 pgs: 129 active+clean; 822 MiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 296 KiB/s rd, 356 KiB/s wr, 163 op/s 2024-09-11T00:30:54.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:30:54 smithi165 ceph-mon[35356]: pgmap v1114: 129 pgs: 129 active+clean; 822 MiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 296 KiB/s rd, 356 KiB/s wr, 163 op/s 2024-09-11T00:30:54.291 INFO:tasks.workunit.client.0.smithi012.stdout: 10 184527 15.39 MB/sec execute 530 sec latency 20120.157 ms 2024-09-11T00:30:54.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:30:54 smithi178 ceph-mon[35919]: pgmap v1114: 129 pgs: 129 active+clean; 822 MiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 296 KiB/s rd, 356 KiB/s wr, 163 op/s 2024-09-11T00:30:54.465 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:54.461+0000 7f46b3e00640 1 -- 172.21.15.12:0/2770481180 <== mds.0 v2:172.21.15.165:6838/501312984 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x564ccb8cc490 con 0x564ccb9f44c0 2024-09-11T00:30:54.465 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:54.461+0000 7f46b3e00640 10 client.27973 handle_command_reply: tid=0 2024-09-11T00:30:54.465 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:54.461+0000 7f46db80c740 1 client.27973 shutdown 2024-09-11T00:30:54.465 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:54.461+0000 7f46db80c740 1 -- 172.21.15.12:0/2770481180 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564ccb9633e0 msgr2=0x564ccb979800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:30:54.466 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:54.461+0000 7f46db80c740 1 --2- 172.21.15.12:0/2770481180 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564ccb9633e0 0x564ccb979800 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f46cc004f80 tx=0x7f46cc009f20 comp rx=0 tx=0).stop 2024-09-11T00:30:54.466 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:54.461+0000 7f46db80c740 1 -- 172.21.15.12:0/2770481180 shutdown_connections 2024-09-11T00:30:54.466 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:54.461+0000 7f46db80c740 1 --2- 172.21.15.12:0/2770481180 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x564ccb9f44c0 0x564ccb9f68e0 crc :-1 s=READY pgs=320 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:30:54.466 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:54.461+0000 7f46db80c740 1 --2- 172.21.15.12:0/2770481180 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564ccb9633e0 0x564ccb979800 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:30:54.466 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:54.461+0000 7f46db80c740 1 --2- 172.21.15.12:0/2770481180 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564ccb961ef0 0x564ccb97f030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:30:54.466 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:54.461+0000 7f46db80c740 1 --2- 172.21.15.12:0/2770481180 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564ccb95c920 0x564ccb97ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:30:54.466 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:54.461+0000 7f46b3e00640 0 client.27973 ms_handle_reset on v2:172.21.15.165:6838/501312984 2024-09-11T00:30:54.466 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:54.461+0000 7f46db80c740 1 -- 172.21.15.12:0/2770481180 >> 172.21.15.12:0/2770481180 conn(0x564ccb9236f0 msgr2=0x564ccb95b6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:30:54.466 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:54.461+0000 7f46db80c740 1 -- 172.21.15.12:0/2770481180 shutdown_connections 2024-09-11T00:30:54.466 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:54.461+0000 7f46db80c740 1 -- 172.21.15.12:0/2770481180 wait complete. 2024-09-11T00:30:54.466 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:54.461+0000 7f46db80c740 20 client.27973 trim_cache size 0 max 16384 2024-09-11T00:30:54.467 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-09-11T00:30:54.467 INFO:teuthology.orchestra.run.smithi012.stdout: "epoch": 102, 2024-09-11T00:30:54.467 INFO:teuthology.orchestra.run.smithi012.stdout: "leader": 24674, 2024-09-11T00:30:54.467 INFO:teuthology.orchestra.run.smithi012.stdout: "set_version": 70, 2024-09-11T00:30:54.467 INFO:teuthology.orchestra.run.smithi012.stdout: "sets": { 2024-09-11T00:30:54.467 INFO:teuthology.orchestra.run.smithi012.stdout: "dfb8983": { 2024-09-11T00:30:54.467 INFO:teuthology.orchestra.run.smithi012.stdout: "version": 70, 2024-09-11T00:30:54.467 INFO:teuthology.orchestra.run.smithi012.stdout: "age_ref": 0.0, 2024-09-11T00:30:54.467 INFO:teuthology.orchestra.run.smithi012.stdout: "state": { 2024-09-11T00:30:54.467 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "QUIESCED", 2024-09-11T00:30:54.468 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 0.0 2024-09-11T00:30:54.468 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:30:54.468 INFO:teuthology.orchestra.run.smithi012.stdout: "timeout": 90.0, 2024-09-11T00:30:54.468 INFO:teuthology.orchestra.run.smithi012.stdout: "expiration": 138.8, 2024-09-11T00:30:54.468 INFO:teuthology.orchestra.run.smithi012.stdout: "members": { 2024-09-11T00:30:54.468 INFO:teuthology.orchestra.run.smithi012.stdout: "file:/": { 2024-09-11T00:30:54.468 INFO:teuthology.orchestra.run.smithi012.stdout: "excluded": false, 2024-09-11T00:30:54.468 INFO:teuthology.orchestra.run.smithi012.stdout: "state": { 2024-09-11T00:30:54.468 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "QUIESCED", 2024-09-11T00:30:54.468 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 0.0 2024-09-11T00:30:54.468 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:30:54.468 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:30:54.468 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:30:54.469 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:30:54.469 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:30:54.469 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-09-11T00:30:54.469 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:54.462+0000 7f46d8e00640 1 -- 172.21.15.12:0/3559187954 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f46b80b5360 msgr2=0x7f46b80b7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:30:54.469 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:54.462+0000 7f46d8e00640 1 --2- 172.21.15.12:0/3559187954 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f46b80b5360 0x7f46b80b7820 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f46cc021000 tx=0x7f46cc01e000 comp rx=0 tx=0).stop 2024-09-11T00:30:54.469 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:54.462+0000 7f46d8e00640 1 -- 172.21.15.12:0/3559187954 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f46d4102dc0 msgr2=0x7f46d419b550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:30:54.469 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:54.462+0000 7f46d8e00640 1 --2- 172.21.15.12:0/3559187954 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f46d4102dc0 0x7f46d419b550 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f46c4005940 tx=0x7f46c4004270 comp rx=0 tx=0).stop 2024-09-11T00:30:54.469 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:54.462+0000 7f46d8e00640 1 -- 172.21.15.12:0/3559187954 shutdown_connections 2024-09-11T00:30:54.470 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:54.462+0000 7f46d8e00640 1 --2- 172.21.15.12:0/3559187954 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f46b80b5360 0x7f46b80b7820 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:30:54.470 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:54.462+0000 7f46d8e00640 1 --2- 172.21.15.12:0/3559187954 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f46d4109150 0x7f46d41a0e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:30:54.470 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:54.462+0000 7f46d8e00640 1 --2- 172.21.15.12:0/3559187954 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f46d41087d0 0x7f46d419ba90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:30:54.470 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:54.462+0000 7f46d8e00640 1 --2- 172.21.15.12:0/3559187954 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f46d4102dc0 0x7f46d419b550 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:30:54.470 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:54.462+0000 7f46d8e00640 1 -- 172.21.15.12:0/3559187954 >> 172.21.15.12:0/3559187954 conn(0x7f46d40fe830 msgr2=0x7f46d4104d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:30:54.470 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:54.462+0000 7f46d8e00640 1 -- 172.21.15.12:0/3559187954 shutdown_connections 2024-09-11T00:30:54.470 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:54.462+0000 7f46d8e00640 1 -- 172.21.15.12:0/3559187954 wait complete. 2024-09-11T00:30:55.292 INFO:tasks.workunit.client.0.smithi012.stdout: 10 184528 15.36 MB/sec execute 531 sec latency 20263.846 ms 2024-09-11T00:30:55.378 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'dfb8983', quiesce took 24.6 seconds. Will release after: -5.75 2024-09-11T00:30:55.378 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid ff3ad74a-6fcf-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T00:30:55.761 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:30:56.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:30:56 smithi165 ceph-mon[35356]: pgmap v1115: 129 pgs: 129 active+clean; 822 MiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 78 KiB/s wr, 106 op/s 2024-09-11T00:30:56.292 INFO:tasks.workunit.client.0.smithi012.stdout: 10 184528 15.34 MB/sec execute 532 sec latency 4006.880 ms 2024-09-11T00:30:56.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:30:56 smithi012 ceph-mon[30333]: pgmap v1115: 129 pgs: 129 active+clean; 822 MiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 78 KiB/s wr, 106 op/s 2024-09-11T00:30:56.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:30:56 smithi178 ceph-mon[35919]: pgmap v1115: 129 pgs: 129 active+clean; 822 MiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 78 KiB/s wr, 106 op/s 2024-09-11T00:30:56.452 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:56.449+0000 7f01a3e00640 1 -- 172.21.15.12:0/3098347551 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f019c10a9b0 msgr2=0x7f019c10adf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:30:56.452 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:56.449+0000 7f01a3e00640 1 --2- 172.21.15.12:0/3098347551 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f019c10a9b0 0x7f019c10adf0 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f0188009900 tx=0x7f018802a410 comp rx=0 tx=0).stop 2024-09-11T00:30:56.452 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:56.449+0000 7f01a3e00640 1 -- 172.21.15.12:0/3098347551 shutdown_connections 2024-09-11T00:30:56.452 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:56.449+0000 7f01a3e00640 1 --2- 172.21.15.12:0/3098347551 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f019c10b330 0x7f019c111b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:30:56.452 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:56.449+0000 7f01a3e00640 1 --2- 172.21.15.12:0/3098347551 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f019c10a9b0 0x7f019c10adf0 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:30:56.452 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:56.449+0000 7f01a3e00640 1 --2- 172.21.15.12:0/3098347551 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f019c104fa0 0x7f019c105380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:30:56.452 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:56.449+0000 7f01a3e00640 1 -- 172.21.15.12:0/3098347551 >> 172.21.15.12:0/3098347551 conn(0x7f019c100a50 msgr2=0x7f019c102e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:30:56.453 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:56.449+0000 7f01a3e00640 1 -- 172.21.15.12:0/3098347551 shutdown_connections 2024-09-11T00:30:56.453 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:56.449+0000 7f01a3e00640 1 -- 172.21.15.12:0/3098347551 wait complete. 2024-09-11T00:30:56.454 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:56.450+0000 7f01a3e00640 1 Processor -- start 2024-09-11T00:30:56.454 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:56.451+0000 7f01a3e00640 1 -- start start 2024-09-11T00:30:56.454 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:56.451+0000 7f01a3e00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f019c104fa0 0x7f019c19b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:30:56.454 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:56.451+0000 7f01a3e00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f019c10a9b0 0x7f019c19ba00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:30:56.454 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:56.451+0000 7f01a3e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f019c10b330 0x7f019c1a0da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:30:56.454 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:56.451+0000 7f01a3e00640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f019c114fb0 con 0x7f019c104fa0 2024-09-11T00:30:56.454 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:56.451+0000 7f01a3e00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f019c114e30 con 0x7f019c10b330 2024-09-11T00:30:56.454 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:56.451+0000 7f01a3e00640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f019c115130 con 0x7f019c10a9b0 2024-09-11T00:30:56.454 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:56.451+0000 7f01a1400640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f019c104fa0 0x7f019c19b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:30:56.454 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:56.451+0000 7f01a1400640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f019c104fa0 0x7f019c19b4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:56024/0 (socket says 172.21.15.12:56024) 2024-09-11T00:30:56.454 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:56.451+0000 7f01a1400640 1 -- 172.21.15.12:0/1783031542 learned_addr learned my addr 172.21.15.12:0/1783031542 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:30:56.454 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:56.451+0000 7f01a1e00640 1 --2- 172.21.15.12:0/1783031542 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f019c10b330 0x7f019c1a0da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:30:56.454 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:56.451+0000 7f01a0a00640 1 --2- 172.21.15.12:0/1783031542 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f019c10a9b0 0x7f019c19ba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:30:56.455 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:56.452+0000 7f01a1400640 1 -- 172.21.15.12:0/1783031542 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f019c10a9b0 msgr2=0x7f019c19ba00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:30:56.455 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:56.452+0000 7f01a1400640 1 --2- 172.21.15.12:0/1783031542 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f019c10a9b0 0x7f019c19ba00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:30:56.455 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:56.452+0000 7f01a1400640 1 -- 172.21.15.12:0/1783031542 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f019c10b330 msgr2=0x7f019c1a0da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:30:56.455 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:56.452+0000 7f01a1400640 1 --2- 172.21.15.12:0/1783031542 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f019c10b330 0x7f019c1a0da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:30:56.456 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:56.452+0000 7f01a1400640 1 -- 172.21.15.12:0/1783031542 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f019c1a1570 con 0x7f019c104fa0 2024-09-11T00:30:56.456 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:56.452+0000 7f01a1400640 1 --2- 172.21.15.12:0/1783031542 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f019c104fa0 0x7f019c19b4c0 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f019000e9f0 tx=0x7f019000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:30:56.456 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:56.452+0000 7f019a000640 1 -- 172.21.15.12:0/1783031542 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f019000cd80 con 0x7f019c104fa0 2024-09-11T00:30:56.456 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:56.452+0000 7f019a000640 1 -- 172.21.15.12:0/1783031542 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f01900108e0 con 0x7f019c104fa0 2024-09-11T00:30:56.456 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:56.452+0000 7f019a000640 1 -- 172.21.15.12:0/1783031542 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0190018980 con 0x7f019c104fa0 2024-09-11T00:30:56.456 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:56.452+0000 7f01a3e00640 1 -- 172.21.15.12:0/1783031542 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f019c1a1860 con 0x7f019c104fa0 2024-09-11T00:30:56.456 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:56.453+0000 7f01a3e00640 1 -- 172.21.15.12:0/1783031542 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f019c1a7f20 con 0x7f019c104fa0 2024-09-11T00:30:56.456 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:56.454+0000 7f01a3e00640 1 -- 172.21.15.12:0/1783031542 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f019c105380 con 0x7f019c104fa0 2024-09-11T00:30:56.458 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:56.456+0000 7f019a000640 1 -- 172.21.15.12:0/1783031542 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7f0190010430 con 0x7f019c104fa0 2024-09-11T00:30:56.459 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:56.457+0000 7f019a000640 1 --2- 172.21.15.12:0/1783031542 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f01840b5360 0x7f01840b7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:30:56.460 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:56.457+0000 7f019a000640 1 -- 172.21.15.12:0/1783031542 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9208+0+0 (secure 0 0 0) 0x7f01900e62d0 con 0x7f019c104fa0 2024-09-11T00:30:56.460 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:56.458+0000 7f01a0a00640 1 --2- 172.21.15.12:0/1783031542 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f01840b5360 0x7f01840b7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:30:56.461 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:56.458+0000 7f01a0a00640 1 --2- 172.21.15.12:0/1783031542 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f01840b5360 0x7f01840b7820 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f0188005ab0 tx=0x7f0188005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:30:56.464 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:56.461+0000 7f019a000640 1 -- 172.21.15.12:0/1783031542 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f01900ac710 con 0x7f019c104fa0 2024-09-11T00:30:56.695 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:56.692+0000 7f01a3e00640 1 -- 172.21.15.12:0/1783031542 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f019c02e5c0 con 0x7f019c104fa0 2024-09-11T00:30:56.695 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:56.693+0000 7f019a000640 1 -- 172.21.15.12:0/1783031542 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 102 v102) ==== 95+0+11973 (secure 0 0 0) 0x7f01900b1140 con 0x7f019c104fa0 2024-09-11T00:30:56.696 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-11T00:30:56.696 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":102,"btime":"2024-09-11T00:29:46:174935+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14577,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6835/447171591","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":447171591},{"type":"v1","addr":"172.21.15.12:6835","nonce":447171591}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14589,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/901486040","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":901486040},{"type":"v1","addr":"172.21.15.12:6837","nonce":901486040}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":102,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T00:04:04.739881+0000","modified":"2024-09-11T00:29:45.675612+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24674,"mds_1":24643,"mds_2":24638,"mds_3":24637,"mds_4":24614},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14631":{"gid":14631,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.12:6841/3069919437","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6840","nonce":3069919437},{"type":"v1","addr":"172.21.15.12:6841","nonce":3069919437}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24562":{"gid":24562,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6833/1141636112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":1141636112},{"type":"v1","addr":"172.21.15.178:6833","nonce":1141636112}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24581":{"gid":24581,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6833/3111980970","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6832","nonce":3111980970},{"type":"v1","addr":"172.21.15.165:6833","nonce":3111980970}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24607":{"gid":24607,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6835/4084868287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":4084868287},{"type":"v1","addr":"172.21.15.178:6835","nonce":4084868287}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24614":{"gid":24614,"name":"e","rank":4,"incarnation":26,"state":"up:active","state_seq":21,"addr":"172.21.15.165:6835/2425395274","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":2425395274},{"type":"v1","addr":"172.21.15.165:6835","nonce":2425395274}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24637":{"gid":24637,"name":"i","rank":3,"incarnation":24,"state":"up:active","state_seq":18,"addr":"172.21.15.178:6839/70628097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6838","nonce":70628097},{"type":"v1","addr":"172.21.15.178:6839","nonce":70628097}]},"join_fscid":-1,"export_targets":[0,2],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24638":{"gid":24638,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":18,"addr":"172.21.15.165:6837/6215088","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6836","nonce":6215088},{"type":"v1","addr":"172.21.15.165:6837","nonce":6215088}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24643":{"gid":24643,"name":"g","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.12:6839/3124093009","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6838","nonce":3124093009},{"type":"v1","addr":"172.21.15.12:6839","nonce":3124093009}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24674":{"gid":24674,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.165:6839/501312984","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6838","nonce":501312984},{"type":"v1","addr":"172.21.15.165:6839","nonce":501312984}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_27751":{"gid":27751,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6837/1436600611","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":1436600611},{"type":"v1","addr":"172.21.15.178:6837","nonce":1436600611}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24674,"qdb_cluster":[24674,24643,24638,24637,24614]},"id":1}]} 2024-09-11T00:30:56.696 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 102 2024-09-11T00:30:56.698 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:56.696+0000 7f01a3e00640 1 -- 172.21.15.12:0/1783031542 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f01840b5360 msgr2=0x7f01840b7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:30:56.698 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:56.696+0000 7f01a3e00640 1 --2- 172.21.15.12:0/1783031542 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f01840b5360 0x7f01840b7820 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f0188005ab0 tx=0x7f0188005a20 comp rx=0 tx=0).stop 2024-09-11T00:30:56.698 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:56.696+0000 7f01a3e00640 1 -- 172.21.15.12:0/1783031542 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f019c104fa0 msgr2=0x7f019c19b4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:30:56.698 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:56.696+0000 7f01a3e00640 1 --2- 172.21.15.12:0/1783031542 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f019c104fa0 0x7f019c19b4c0 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f019000e9f0 tx=0x7f019000eeb0 comp rx=0 tx=0).stop 2024-09-11T00:30:56.699 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:56.696+0000 7f01a3e00640 1 -- 172.21.15.12:0/1783031542 shutdown_connections 2024-09-11T00:30:56.699 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:56.696+0000 7f01a3e00640 1 --2- 172.21.15.12:0/1783031542 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f01840b5360 0x7f01840b7820 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:30:56.699 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:56.696+0000 7f01a3e00640 1 --2- 172.21.15.12:0/1783031542 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f019c10b330 0x7f019c1a0da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:30:56.699 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:56.696+0000 7f01a3e00640 1 --2- 172.21.15.12:0/1783031542 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f019c10a9b0 0x7f019c19ba00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:30:56.699 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:56.696+0000 7f01a3e00640 1 --2- 172.21.15.12:0/1783031542 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f019c104fa0 0x7f019c19b4c0 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:30:56.699 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:56.696+0000 7f01a3e00640 1 -- 172.21.15.12:0/1783031542 >> 172.21.15.12:0/1783031542 conn(0x7f019c100a50 msgr2=0x7f019c102e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:30:56.699 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:56.696+0000 7f01a3e00640 1 -- 172.21.15.12:0/1783031542 shutdown_connections 2024-09-11T00:30:56.699 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:56.696+0000 7f01a3e00640 1 -- 172.21.15.12:0/1783031542 wait complete. 2024-09-11T00:30:57.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:30:57 smithi165 ceph-mon[35356]: from='client.? 172.21.15.12:0/1783031542' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T00:30:57.292 INFO:tasks.workunit.client.0.smithi012.stdout: 10 184528 15.31 MB/sec execute 533 sec latency 5007.004 ms 2024-09-11T00:30:57.300 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:30:57 smithi012 ceph-mon[30333]: from='client.? 172.21.15.12:0/1783031542' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T00:30:57.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:30:57 smithi178 ceph-mon[35919]: from='client.? 172.21.15.12:0/1783031542' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T00:30:57.687 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24674 quiesce db --set-id dfb8983 --release --await' 2024-09-11T00:30:57.688 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid ff3ad74a-6fcf-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24674 quiesce db --set-id dfb8983 --release --await 2024-09-11T00:30:58.070 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:30:58.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:30:58 smithi165 ceph-mon[35356]: pgmap v1116: 129 pgs: 129 active+clean; 817 MiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 191 KiB/s rd, 174 KiB/s wr, 131 op/s 2024-09-11T00:30:58.292 INFO:tasks.workunit.client.0.smithi012.stdout: 10 184528 15.28 MB/sec execute 534 sec latency 6007.091 ms 2024-09-11T00:30:58.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:30:58 smithi012 ceph-mon[30333]: pgmap v1116: 129 pgs: 129 active+clean; 817 MiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 191 KiB/s rd, 174 KiB/s wr, 131 op/s 2024-09-11T00:30:58.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:30:58 smithi178 ceph-mon[35919]: pgmap v1116: 129 pgs: 129 active+clean; 817 MiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 191 KiB/s rd, 174 KiB/s wr, 131 op/s 2024-09-11T00:30:58.641 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:58.638+0000 7fa156e00640 1 -- 172.21.15.12:0/3537135751 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa150104fa0 msgr2=0x7fa150105380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:30:58.641 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:58.638+0000 7fa156e00640 1 --2- 172.21.15.12:0/3537135751 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa150104fa0 0x7fa150105380 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7fa140009900 tx=0x7fa14002a3f0 comp rx=0 tx=0).stop 2024-09-11T00:30:58.641 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:58.638+0000 7fa156e00640 1 -- 172.21.15.12:0/3537135751 shutdown_connections 2024-09-11T00:30:58.641 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:58.638+0000 7fa156e00640 1 --2- 172.21.15.12:0/3537135751 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa15010b330 0x7fa150111b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:30:58.641 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:58.638+0000 7fa156e00640 1 --2- 172.21.15.12:0/3537135751 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa15010a9b0 0x7fa15010adf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:30:58.641 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:58.638+0000 7fa156e00640 1 --2- 172.21.15.12:0/3537135751 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa150104fa0 0x7fa150105380 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:30:58.641 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:58.638+0000 7fa156e00640 1 -- 172.21.15.12:0/3537135751 >> 172.21.15.12:0/3537135751 conn(0x7fa150100a50 msgr2=0x7fa150102e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:30:58.642 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:58.638+0000 7fa156e00640 1 -- 172.21.15.12:0/3537135751 shutdown_connections 2024-09-11T00:30:58.642 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:58.638+0000 7fa156e00640 1 -- 172.21.15.12:0/3537135751 wait complete. 2024-09-11T00:30:58.643 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:58.639+0000 7fa156e00640 1 Processor -- start 2024-09-11T00:30:58.643 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:58.639+0000 7fa156e00640 1 -- start start 2024-09-11T00:30:58.643 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:58.640+0000 7fa156e00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa150104fa0 0x7fa15019b450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:30:58.643 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:58.640+0000 7fa156e00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa15010a9b0 0x7fa15019b990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:30:58.643 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:58.640+0000 7fa156e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa15010b330 0x7fa1501a0d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:30:58.643 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:58.640+0000 7fa156e00640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fa150114f90 con 0x7fa15010a9b0 2024-09-11T00:30:58.643 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:58.640+0000 7fa156e00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fa150114e10 con 0x7fa15010b330 2024-09-11T00:30:58.643 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:58.640+0000 7fa156e00640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fa150115110 con 0x7fa150104fa0 2024-09-11T00:30:58.644 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:58.640+0000 7fa147e00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa15010a9b0 0x7fa15019b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:30:58.644 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:58.640+0000 7fa14fe00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa150104fa0 0x7fa15019b450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:30:58.644 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:58.640+0000 7fa147e00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa15010a9b0 0x7fa15019b990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:56040/0 (socket says 172.21.15.12:56040) 2024-09-11T00:30:58.644 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:58.640+0000 7fa147e00640 1 -- 172.21.15.12:0/2728840077 learned_addr learned my addr 172.21.15.12:0/2728840077 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:30:58.644 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:58.640+0000 7fa154e00640 1 --2- 172.21.15.12:0/2728840077 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa15010b330 0x7fa1501a0d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:30:58.644 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:58.641+0000 7fa14fe00640 1 -- 172.21.15.12:0/2728840077 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa15010b330 msgr2=0x7fa1501a0d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:30:58.644 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:58.641+0000 7fa14fe00640 1 --2- 172.21.15.12:0/2728840077 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa15010b330 0x7fa1501a0d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:30:58.644 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:58.641+0000 7fa14fe00640 1 -- 172.21.15.12:0/2728840077 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa15010a9b0 msgr2=0x7fa15019b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:30:58.644 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:58.641+0000 7fa14fe00640 1 --2- 172.21.15.12:0/2728840077 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa15010a9b0 0x7fa15019b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:30:58.644 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:58.641+0000 7fa14fe00640 1 -- 172.21.15.12:0/2728840077 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1501a1500 con 0x7fa150104fa0 2024-09-11T00:30:58.644 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:58.641+0000 7fa147e00640 1 --2- 172.21.15.12:0/2728840077 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa15010a9b0 0x7fa15019b990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:30:58.645 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:58.641+0000 7fa14fe00640 1 --2- 172.21.15.12:0/2728840077 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa150104fa0 0x7fa15019b450 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fa14002a9e0 tx=0x7fa140003b90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:30:58.645 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:58.641+0000 7fa14d600640 1 -- 172.21.15.12:0/2728840077 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa14003f070 con 0x7fa150104fa0 2024-09-11T00:30:58.645 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:58.641+0000 7fa156e00640 1 -- 172.21.15.12:0/2728840077 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1501a1790 con 0x7fa150104fa0 2024-09-11T00:30:58.645 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:58.641+0000 7fa14d600640 1 -- 172.21.15.12:0/2728840077 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa140031b50 con 0x7fa150104fa0 2024-09-11T00:30:58.645 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:58.641+0000 7fa14d600640 1 -- 172.21.15.12:0/2728840077 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa140035030 con 0x7fa150104fa0 2024-09-11T00:30:58.645 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:58.642+0000 7fa156e00640 1 -- 172.21.15.12:0/2728840077 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1501a7eb0 con 0x7fa150104fa0 2024-09-11T00:30:58.647 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:58.645+0000 7fa1598ae740 1 Processor -- start 2024-09-11T00:30:58.647 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:58.645+0000 7fa1598ae740 1 -- start start 2024-09-11T00:30:58.648 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:58.645+0000 7fa1598ae740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561746069ff0 0x56174606a3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:30:58.648 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:58.645+0000 7fa1598ae740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56174606a910 0x561746076360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:30:58.648 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:58.645+0000 7fa1598ae740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5617460768a0 0x561746078c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:30:58.648 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:58.645+0000 7fa1598ae740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x561745fe1290 con 0x5617460768a0 2024-09-11T00:30:58.648 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:58.645+0000 7fa1598ae740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x561745fd06c0 con 0x561746069ff0 2024-09-11T00:30:58.648 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:58.645+0000 7fa1598ae740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x561745fd44d0 con 0x56174606a910 2024-09-11T00:30:58.649 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:58.646+0000 7fa14d600640 1 -- 172.21.15.12:0/2728840077 <== mon.1 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7fa1400316a0 con 0x7fa150104fa0 2024-09-11T00:30:58.649 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:58.646+0000 7fa154e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561746069ff0 0x56174606a3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:30:58.650 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:58.646+0000 7fa154e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561746069ff0 0x56174606a3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.12:33290/0 (socket says 172.21.15.12:33290) 2024-09-11T00:30:58.650 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:58.646+0000 7fa154e00640 1 -- 172.21.15.12:0/1200261353 learned_addr learned my addr 172.21.15.12:0/1200261353 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:30:58.650 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:58.646+0000 7fa147e00640 1 --2- 172.21.15.12:0/1200261353 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5617460768a0 0x561746078c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:30:58.650 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:58.646+0000 7fa14fe00640 1 --2- 172.21.15.12:0/1200261353 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56174606a910 0x561746076360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:30:58.650 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:58.646+0000 7fa154e00640 1 -- 172.21.15.12:0/1200261353 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56174606a910 msgr2=0x561746076360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:30:58.650 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:58.646+0000 7fa154e00640 1 --2- 172.21.15.12:0/1200261353 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56174606a910 0x561746076360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:30:58.650 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:58.646+0000 7fa154e00640 1 -- 172.21.15.12:0/1200261353 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5617460768a0 msgr2=0x561746078c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:30:58.650 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:58.646+0000 7fa154e00640 1 --2- 172.21.15.12:0/1200261353 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5617460768a0 0x561746078c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:30:58.650 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:58.646+0000 7fa154e00640 1 -- 172.21.15.12:0/1200261353 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561745f5d460 con 0x561746069ff0 2024-09-11T00:30:58.650 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:58.646+0000 7fa154e00640 1 --2- 172.21.15.12:0/1200261353 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561746069ff0 0x56174606a3d0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fa148009fd0 tx=0x7fa1480118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=13c1e95d7078a372 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:30:58.650 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:58.647+0000 7fa145600640 1 -- 172.21.15.12:0/1200261353 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa148004d20 con 0x561746069ff0 2024-09-11T00:30:58.651 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:58.647+0000 7fa145600640 1 -- 172.21.15.12:0/1200261353 <== mon.2 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa148004050 con 0x561746069ff0 2024-09-11T00:30:58.651 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:58.647+0000 7fa145600640 1 -- 172.21.15.12:0/1200261353 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa148005260 con 0x561746069ff0 2024-09-11T00:30:58.653 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:58.648+0000 7fa14d600640 1 --2- 172.21.15.12:0/2728840077 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fa1340b52e0 0x7fa1340b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:30:58.653 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:58.648+0000 7fa1598ae740 1 -- 172.21.15.12:0/1200261353 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561746069ff0 msgr2=0x56174606a3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:30:58.653 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:58.648+0000 7fa1598ae740 1 --2- 172.21.15.12:0/1200261353 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561746069ff0 0x56174606a3d0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fa148009fd0 tx=0x7fa1480118d0 comp rx=0 tx=0).stop 2024-09-11T00:30:58.653 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:58.648+0000 7fa1598ae740 1 -- 172.21.15.12:0/1200261353 shutdown_connections 2024-09-11T00:30:58.654 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:58.648+0000 7fa1598ae740 1 --2- 172.21.15.12:0/1200261353 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5617460768a0 0x561746078c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:30:58.654 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:58.648+0000 7fa1598ae740 1 --2- 172.21.15.12:0/1200261353 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56174606a910 0x561746076360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:30:58.654 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:58.648+0000 7fa1598ae740 1 --2- 172.21.15.12:0/1200261353 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561746069ff0 0x56174606a3d0 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:30:58.654 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:58.648+0000 7fa1598ae740 1 -- 172.21.15.12:0/1200261353 >> 172.21.15.12:0/1200261353 conn(0x561746067e20 msgr2=0x561746068fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:30:58.654 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:58.648+0000 7fa14fe00640 1 --2- 172.21.15.12:0/2728840077 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fa1340b52e0 0x7fa1340b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:30:58.654 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:58.648+0000 7fa14d600640 1 -- 172.21.15.12:0/2728840077 <== mon.1 v2:172.21.15.178:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9208+0+0 (secure 0 0 0) 0x7fa1401068b0 con 0x7fa150104fa0 2024-09-11T00:30:58.654 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:58.648+0000 7fa1598ae740 1 -- 172.21.15.12:0/1200261353 shutdown_connections 2024-09-11T00:30:58.654 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:58.648+0000 7fa1598ae740 1 -- 172.21.15.12:0/1200261353 wait complete. 2024-09-11T00:30:58.654 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:58.649+0000 7fa14fe00640 1 --2- 172.21.15.12:0/2728840077 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fa1340b52e0 0x7fa1340b77a0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7fa1400c92c0 tx=0x7fa140031400 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:30:58.654 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:58.649+0000 7fa1598ae740 1 -- start start 2024-09-11T00:30:58.654 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:58.649+0000 7fa1598ae740 1 Processor -- start 2024-09-11T00:30:58.654 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:58.649+0000 7fa1598ae740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561746069ff0 0x561746073830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:30:58.655 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:58.649+0000 7fa1598ae740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56174606a910 0x561746073d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:30:58.655 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:58.649+0000 7fa1598ae740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5617460768a0 0x5617460742b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:30:58.655 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:58.649+0000 7fa1598ae740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x561745e31bb0 con 0x561746069ff0 2024-09-11T00:30:58.655 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:58.649+0000 7fa1598ae740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x561745f6b6c0 con 0x5617460768a0 2024-09-11T00:30:58.655 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:58.649+0000 7fa1598ae740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x561746079550 con 0x56174606a910 2024-09-11T00:30:58.655 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:58.650+0000 7fa1598ae740 10 client.0 _ll_register_callbacks cb 0x561745f86490 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:30:58.655 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:58.650+0000 7fa154e00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561746069ff0 0x561746073830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:30:58.655 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:58.650+0000 7fa154e00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561746069ff0 0x561746073830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:56060/0 (socket says 172.21.15.12:56060) 2024-09-11T00:30:58.655 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:58.650+0000 7fa154e00640 1 -- 172.21.15.12:0/1660728563 learned_addr learned my addr 172.21.15.12:0/1660728563 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:30:58.655 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:58.650+0000 7fa147e00640 1 --2- 172.21.15.12:0/1660728563 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56174606a910 0x561746073d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:30:58.655 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:58.650+0000 7fa154e00640 1 --2- 172.21.15.12:0/1660728563 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5617460768a0 0x5617460742b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:30:58.655 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:58.650+0000 7fa154e00640 1 -- 172.21.15.12:0/1660728563 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56174606a910 msgr2=0x561746073d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:30:58.656 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:58.650+0000 7fa154e00640 1 --2- 172.21.15.12:0/1660728563 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56174606a910 0x561746073d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:30:58.656 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:58.650+0000 7fa154e00640 1 -- 172.21.15.12:0/1660728563 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5617460768a0 msgr2=0x5617460742b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:30:58.656 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:58.650+0000 7fa154e00640 1 --2- 172.21.15.12:0/1660728563 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5617460768a0 0x5617460742b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:30:58.656 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:58.650+0000 7fa154e00640 1 -- 172.21.15.12:0/1660728563 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561745d2eb00 con 0x561746069ff0 2024-09-11T00:30:58.656 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:58.651+0000 7fa154e00640 1 --2- 172.21.15.12:0/1660728563 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561746069ff0 0x561746073830 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7fa148011dd0 tx=0x7fa148009be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fda84150ec132abd server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:30:58.656 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:58.651+0000 7fa13e000640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-09-11T00:30:58.656 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:58.651+0000 7fa13e000640 1 -- 172.21.15.12:0/1660728563 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa14802d060 con 0x561746069ff0 2024-09-11T00:30:58.656 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:58.651+0000 7fa13e000640 1 -- 172.21.15.12:0/1660728563 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 9208+0+0 (secure 0 0 0) 0x7fa148005050 con 0x561746069ff0 2024-09-11T00:30:58.656 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:58.651+0000 7fa1598ae740 1 -- 172.21.15.12:0/1660728563 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa148004050 con 0x561746069ff0 2024-09-11T00:30:58.658 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:58.651+0000 7fa13e000640 1 -- 172.21.15.12:0/1660728563 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=102) ==== 24+0+0 (secure 0 0 0) 0x7fa148034050 con 0x561746069ff0 2024-09-11T00:30:58.658 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:58.651+0000 7fa1598ae740 10 client.18018 fetch_fsmap learned FSMap version 102 2024-09-11T00:30:58.658 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:58.651+0000 7fa1598ae740 1 -- 172.21.15.12:0/1660728563 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=102,monmap=4+}) -- 0x561745da5960 con 0x561746069ff0 2024-09-11T00:30:58.658 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:58.652+0000 7fa13e000640 1 -- 172.21.15.12:0/1660728563 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa148004050 con 0x561746069ff0 2024-09-11T00:30:58.658 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:58.652+0000 7fa13e000640 1 -- 172.21.15.12:0/1660728563 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 102) ==== 8378+0+0 (secure 0 0 0) 0x7fa14800f600 con 0x561746069ff0 2024-09-11T00:30:58.659 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:58.652+0000 7fa1598ae740 10 client.18018 fetch_fsmap finished waiting for FSMap version 102 2024-09-11T00:30:58.659 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:58.652+0000 7fa1598ae740 10 client.18018 resolve_mds: validated gid 24674 aka daemon mds.b 2024-09-11T00:30:58.659 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:58.652+0000 7fa1598ae740 20 client.18018 populate_metadata read hostname 'smithi012' 2024-09-11T00:30:58.659 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:58.652+0000 7fa1598ae740 1 --2- 172.21.15.12:0/1660728563 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x561746086400 0x5617460887f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:30:58.659 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:58.652+0000 7fa1598ae740 4 client.18018 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T00:30:58.659 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:58.652+0000 7fa1598ae740 1 -- 172.21.15.12:0/1660728563 --> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561745fe31e0 con 0x561746086400 2024-09-11T00:30:58.659 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:58.652+0000 7fa147e00640 1 --2- 172.21.15.12:0/1660728563 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x561746086400 0x5617460887f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:30:58.659 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:58.653+0000 7fa147e00640 1 --2- 172.21.15.12:0/1660728563 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x561746086400 0x5617460887f0 crc :-1 s=READY pgs=323 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=556847a00cccf274 server_cookie=61cc838d6c70da51 in_seq=0 out_seq=0 2024-09-11T00:30:58.659 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:58.653+0000 7fa13e000640 10 client.18018 ms_handle_connect on v2:172.21.15.165:6838/501312984 2024-09-11T00:30:58.659 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:58.654+0000 7fa13e000640 1 -- 172.21.15.12:0/1660728563 <== mds.0 v2:172.21.15.165:6838/501312984 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x561745fe31e0 con 0x561746086400 2024-09-11T00:30:58.659 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:58.654+0000 7fa13e000640 10 client.18018 handle_command_reply: tid=0 2024-09-11T00:30:58.659 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:58.654+0000 7fa1598ae740 1 client.18018 shutdown 2024-09-11T00:30:58.659 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:58.654+0000 7fa1598ae740 1 -- 172.21.15.12:0/1660728563 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561746069ff0 msgr2=0x561746073830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:30:58.660 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:58.654+0000 7fa1598ae740 1 --2- 172.21.15.12:0/1660728563 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561746069ff0 0x561746073830 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7fa148011dd0 tx=0x7fa148009be0 comp rx=0 tx=0).stop 2024-09-11T00:30:58.660 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:58.654+0000 7fa1598ae740 1 -- 172.21.15.12:0/1660728563 shutdown_connections 2024-09-11T00:30:58.660 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:58.654+0000 7fa1598ae740 1 --2- 172.21.15.12:0/1660728563 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x561746086400 0x5617460887f0 crc :-1 s=READY pgs=323 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:30:58.660 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:58.654+0000 7fa1598ae740 1 --2- 172.21.15.12:0/1660728563 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5617460768a0 0x5617460742b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:30:58.660 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:58.654+0000 7fa1598ae740 1 --2- 172.21.15.12:0/1660728563 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56174606a910 0x561746073d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:30:58.660 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:58.654+0000 7fa1598ae740 1 --2- 172.21.15.12:0/1660728563 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561746069ff0 0x561746073830 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:30:58.660 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:58.654+0000 7fa1598ae740 1 -- 172.21.15.12:0/1660728563 >> 172.21.15.12:0/1660728563 conn(0x561746067e20 msgr2=0x56174606d9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:30:58.660 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:58.654+0000 7fa13e000640 0 client.18018 ms_handle_reset on v2:172.21.15.165:6838/501312984 2024-09-11T00:30:58.660 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:58.654+0000 7fa1598ae740 1 -- 172.21.15.12:0/1660728563 shutdown_connections 2024-09-11T00:30:58.660 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:58.654+0000 7fa1598ae740 1 -- 172.21.15.12:0/1660728563 wait complete. 2024-09-11T00:30:58.660 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:58.655+0000 7fa1598ae740 20 client.18018 trim_cache size 0 max 16384 2024-09-11T00:30:58.675 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:58.673+0000 7fa1598ae740 1 Processor -- start 2024-09-11T00:30:58.675 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:58.673+0000 7fa1598ae740 1 -- start start 2024-09-11T00:30:58.676 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:58.673+0000 7fa1598ae740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561746069ff0 0x56174608df60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:30:58.676 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:58.673+0000 7fa1598ae740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56174606a910 0x56174608e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:30:58.676 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:58.673+0000 7fa1598ae740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5617460768a0 0x561746066590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:30:58.676 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:58.673+0000 7fa1598ae740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fa1501ad960 con 0x561746069ff0 2024-09-11T00:30:58.676 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:58.673+0000 7fa1598ae740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fa1501af500 con 0x56174606a910 2024-09-11T00:30:58.677 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:58.673+0000 7fa1598ae740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fa1501ae090 con 0x5617460768a0 2024-09-11T00:30:58.677 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:58.673+0000 7fa154e00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561746069ff0 0x56174608df60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:30:58.677 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:58.673+0000 7fa154e00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561746069ff0 0x56174608df60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:56072/0 (socket says 172.21.15.12:56072) 2024-09-11T00:30:58.677 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:58.673+0000 7fa154e00640 1 -- 172.21.15.12:0/2509343699 learned_addr learned my addr 172.21.15.12:0/2509343699 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:30:58.677 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:58.673+0000 7fa147e00640 1 --2- 172.21.15.12:0/2509343699 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56174606a910 0x56174608e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:30:58.677 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:58.674+0000 7fa154e00640 1 -- 172.21.15.12:0/2509343699 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5617460768a0 msgr2=0x561746066590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:30:58.677 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:58.674+0000 7fa154e00640 1 --2- 172.21.15.12:0/2509343699 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5617460768a0 0x561746066590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:30:58.677 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:58.674+0000 7fa154e00640 1 -- 172.21.15.12:0/2509343699 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56174606a910 msgr2=0x56174608e4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:30:58.677 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:58.674+0000 7fa154e00640 1 --2- 172.21.15.12:0/2509343699 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56174606a910 0x56174608e4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:30:58.677 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:58.674+0000 7fa154e00640 1 -- 172.21.15.12:0/2509343699 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5617460688a0 con 0x561746069ff0 2024-09-11T00:30:58.677 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:58.674+0000 7fa154e00640 1 --2- 172.21.15.12:0/2509343699 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561746069ff0 0x56174608df60 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7fa148009bb0 tx=0x7fa148003a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68a8184e4508a424 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:30:58.679 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:58.674+0000 7fa144c00640 1 -- 172.21.15.12:0/2509343699 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa14802d040 con 0x561746069ff0 2024-09-11T00:30:58.679 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:58.674+0000 7fa144c00640 1 -- 172.21.15.12:0/2509343699 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa14800fd70 con 0x561746069ff0 2024-09-11T00:30:58.679 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:58.674+0000 7fa144c00640 1 -- 172.21.15.12:0/2509343699 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1480286d0 con 0x561746069ff0 2024-09-11T00:30:58.679 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:58.675+0000 7fa1598ae740 1 -- 172.21.15.12:0/2509343699 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561746069ff0 msgr2=0x56174608df60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:30:58.679 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:58.675+0000 7fa1598ae740 1 --2- 172.21.15.12:0/2509343699 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561746069ff0 0x56174608df60 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7fa148009bb0 tx=0x7fa148003a30 comp rx=0 tx=0).stop 2024-09-11T00:30:58.679 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:58.675+0000 7fa1598ae740 1 -- 172.21.15.12:0/2509343699 shutdown_connections 2024-09-11T00:30:58.679 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:58.675+0000 7fa1598ae740 1 --2- 172.21.15.12:0/2509343699 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5617460768a0 0x561746066590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:30:58.679 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:58.675+0000 7fa1598ae740 1 --2- 172.21.15.12:0/2509343699 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56174606a910 0x56174608e4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:30:58.679 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:58.675+0000 7fa1598ae740 1 --2- 172.21.15.12:0/2509343699 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561746069ff0 0x56174608df60 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:30:58.679 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:58.675+0000 7fa1598ae740 1 -- 172.21.15.12:0/2509343699 >> 172.21.15.12:0/2509343699 conn(0x561746067e20 msgr2=0x561746069420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:30:58.679 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:58.675+0000 7fa1598ae740 1 -- 172.21.15.12:0/2509343699 shutdown_connections 2024-09-11T00:30:58.679 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:58.675+0000 7fa1598ae740 1 -- 172.21.15.12:0/2509343699 wait complete. 2024-09-11T00:30:58.680 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:58.676+0000 7fa1598ae740 1 -- start start 2024-09-11T00:30:58.680 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:58.676+0000 7fa1598ae740 1 Processor -- start 2024-09-11T00:30:58.680 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:58.676+0000 7fa1598ae740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561746069ff0 0x56174607f0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:30:58.680 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:58.676+0000 7fa1598ae740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56174606a910 0x5617460797d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:30:58.680 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:58.676+0000 7fa1598ae740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5617460768a0 0x561746079e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:30:58.680 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:58.676+0000 7fa1598ae740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fa1501aee30 con 0x5617460768a0 2024-09-11T00:30:58.680 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:58.676+0000 7fa1598ae740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fa1501ae760 con 0x56174606a910 2024-09-11T00:30:58.680 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:58.676+0000 7fa1598ae740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fa1501ab940 con 0x561746069ff0 2024-09-11T00:30:58.680 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:58.676+0000 7fa1598ae740 10 client.0 _ll_register_callbacks cb 0x7fa13800bc70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:30:58.682 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:58.677+0000 7fa147e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56174606a910 0x5617460797d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:30:58.682 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:58.677+0000 7fa154e00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561746069ff0 0x56174607f0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:30:58.682 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:58.677+0000 7fa147e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56174606a910 0x5617460797d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.12:33308/0 (socket says 172.21.15.12:33308) 2024-09-11T00:30:58.682 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:58.677+0000 7fa147e00640 1 -- 172.21.15.12:0/3537634735 learned_addr learned my addr 172.21.15.12:0/3537634735 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:30:58.682 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:58.677+0000 7fa154e00640 1 --2- 172.21.15.12:0/3537634735 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5617460768a0 0x561746079e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:30:58.682 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:58.677+0000 7fa154e00640 1 -- 172.21.15.12:0/3537634735 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56174606a910 msgr2=0x5617460797d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:30:58.682 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:58.677+0000 7fa154e00640 1 --2- 172.21.15.12:0/3537634735 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56174606a910 0x5617460797d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:30:58.682 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:58.677+0000 7fa154e00640 1 -- 172.21.15.12:0/3537634735 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5617460768a0 msgr2=0x561746079e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:30:58.683 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:58.677+0000 7fa154e00640 1 --2- 172.21.15.12:0/3537634735 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5617460768a0 0x561746079e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:30:58.683 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:58.677+0000 7fa154e00640 1 -- 172.21.15.12:0/3537634735 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5617460f3160 con 0x561746069ff0 2024-09-11T00:30:58.683 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:58.677+0000 7fa147e00640 1 --2- 172.21.15.12:0/3537634735 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56174606a910 0x5617460797d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:30:58.683 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:58.678+0000 7fa154e00640 1 --2- 172.21.15.12:0/3537634735 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561746069ff0 0x56174607f0c0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fa148009da0 tx=0x7fa148003ff0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=66f66aa7dbd03cc6 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:30:58.683 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:58.678+0000 7fa13e000640 10 client.0 ms_handle_connect on v2:172.21.15.178:3300/0 2024-09-11T00:30:58.683 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:58.678+0000 7fa13e000640 1 -- 172.21.15.12:0/3537634735 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa14800f4e0 con 0x561746069ff0 2024-09-11T00:30:58.683 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:58.678+0000 7fa1598ae740 1 -- 172.21.15.12:0/3537634735 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa14800fd70 con 0x561746069ff0 2024-09-11T00:30:58.683 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:58.678+0000 7fa13e000640 1 -- 172.21.15.12:0/3537634735 <== mon.1 v2:172.21.15.178:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 9208+0+0 (secure 0 0 0) 0x7fa14800f680 con 0x561746069ff0 2024-09-11T00:30:58.683 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:58.678+0000 7fa13e000640 1 -- 172.21.15.12:0/3537634735 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=102) ==== 24+0+0 (secure 0 0 0) 0x7fa14800f880 con 0x561746069ff0 2024-09-11T00:30:58.683 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:58.678+0000 7fa1598ae740 10 client.28009 fetch_fsmap learned FSMap version 102 2024-09-11T00:30:58.683 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:58.678+0000 7fa1598ae740 1 -- 172.21.15.12:0/3537634735 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,fsmap=102,monmap=4+}) -- 0x5617460f4300 con 0x561746069ff0 2024-09-11T00:30:58.683 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:58.679+0000 7fa13e000640 1 -- 172.21.15.12:0/3537634735 <== mon.1 v2:172.21.15.178:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa14800fd70 con 0x561746069ff0 2024-09-11T00:30:58.683 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:58.679+0000 7fa13e000640 1 -- 172.21.15.12:0/3537634735 <== mon.1 v2:172.21.15.178:3300/0 5 ==== fsmap(e 102) ==== 8378+0+0 (secure 0 0 0) 0x7fa1480384c0 con 0x561746069ff0 2024-09-11T00:30:58.684 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:58.679+0000 7fa1598ae740 10 client.28009 fetch_fsmap finished waiting for FSMap version 102 2024-09-11T00:30:58.684 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:58.679+0000 7fa1598ae740 10 client.28009 resolve_mds: validated gid 24674 aka daemon mds.b 2024-09-11T00:30:58.684 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:58.679+0000 7fa1598ae740 20 client.28009 populate_metadata read hostname 'smithi012' 2024-09-11T00:30:58.684 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:58.679+0000 7fa1598ae740 1 --2- 172.21.15.12:0/3537634735 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x5617460f4910 0x5617460f6d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:30:58.684 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:58.679+0000 7fa1598ae740 4 client.28009 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "dfb8983", "release": true, "await": true}] 2024-09-11T00:30:58.684 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:58.679+0000 7fa1598ae740 1 -- 172.21.15.12:0/3537634735 --> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] -- command(tid 0: {"prefix": "quiesce db", "set_id": "dfb8983", "release": true, "await": true}) -- 0x561746088d20 con 0x5617460f4910 2024-09-11T00:30:58.684 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:58.679+0000 7fa147e00640 1 --2- 172.21.15.12:0/3537634735 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x5617460f4910 0x5617460f6d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:30:58.684 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:58.680+0000 7fa147e00640 1 --2- 172.21.15.12:0/3537634735 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x5617460f4910 0x5617460f6d30 crc :-1 s=READY pgs=324 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a43bd1031541a985 server_cookie=b43a27bedb290045 in_seq=0 out_seq=0 2024-09-11T00:30:58.684 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:58.680+0000 7fa13e000640 10 client.28009 ms_handle_connect on v2:172.21.15.165:6838/501312984 2024-09-11T00:30:58.714 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:58.710+0000 7fa13e000640 1 -- 172.21.15.12:0/3537634735 <== mds.0 v2:172.21.15.165:6838/501312984 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x561746088d20 con 0x5617460f4910 2024-09-11T00:30:58.714 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:58.710+0000 7fa13e000640 10 client.28009 handle_command_reply: tid=0 2024-09-11T00:30:58.714 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:58.710+0000 7fa1598ae740 1 client.28009 shutdown 2024-09-11T00:30:58.714 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:58.710+0000 7fa1598ae740 1 -- 172.21.15.12:0/3537634735 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561746069ff0 msgr2=0x56174607f0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:30:58.714 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:58.710+0000 7fa1598ae740 1 --2- 172.21.15.12:0/3537634735 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561746069ff0 0x56174607f0c0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fa148009da0 tx=0x7fa148003ff0 comp rx=0 tx=0).stop 2024-09-11T00:30:58.714 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:58.710+0000 7fa1598ae740 1 -- 172.21.15.12:0/3537634735 shutdown_connections 2024-09-11T00:30:58.714 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:58.710+0000 7fa1598ae740 1 --2- 172.21.15.12:0/3537634735 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x5617460f4910 0x5617460f6d30 crc :-1 s=READY pgs=324 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:30:58.714 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:58.710+0000 7fa1598ae740 1 --2- 172.21.15.12:0/3537634735 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5617460768a0 0x561746079e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:30:58.715 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:58.711+0000 7fa1598ae740 1 --2- 172.21.15.12:0/3537634735 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56174606a910 0x5617460797d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:30:58.715 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:58.711+0000 7fa1598ae740 1 --2- 172.21.15.12:0/3537634735 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561746069ff0 0x56174607f0c0 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:30:58.715 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:58.711+0000 7fa1598ae740 1 -- 172.21.15.12:0/3537634735 >> 172.21.15.12:0/3537634735 conn(0x561746067e20 msgr2=0x561746074e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:30:58.715 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:58.711+0000 7fa13e000640 0 client.28009 ms_handle_reset on v2:172.21.15.165:6838/501312984 2024-09-11T00:30:58.715 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:58.711+0000 7fa1598ae740 1 -- 172.21.15.12:0/3537634735 shutdown_connections 2024-09-11T00:30:58.715 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:58.711+0000 7fa1598ae740 1 -- 172.21.15.12:0/3537634735 wait complete. 2024-09-11T00:30:58.715 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:58.711+0000 7fa1598ae740 20 client.28009 trim_cache size 0 max 16384 2024-09-11T00:30:58.715 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-09-11T00:30:58.715 INFO:teuthology.orchestra.run.smithi012.stdout: "epoch": 102, 2024-09-11T00:30:58.716 INFO:teuthology.orchestra.run.smithi012.stdout: "leader": 24674, 2024-09-11T00:30:58.716 INFO:teuthology.orchestra.run.smithi012.stdout: "set_version": 72, 2024-09-11T00:30:58.716 INFO:teuthology.orchestra.run.smithi012.stdout: "sets": { 2024-09-11T00:30:58.716 INFO:teuthology.orchestra.run.smithi012.stdout: "dfb8983": { 2024-09-11T00:30:58.716 INFO:teuthology.orchestra.run.smithi012.stdout: "version": 72, 2024-09-11T00:30:58.716 INFO:teuthology.orchestra.run.smithi012.stdout: "age_ref": 0.0, 2024-09-11T00:30:58.716 INFO:teuthology.orchestra.run.smithi012.stdout: "state": { 2024-09-11T00:30:58.716 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "RELEASED", 2024-09-11T00:30:58.716 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 0.0 2024-09-11T00:30:58.716 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:30:58.716 INFO:teuthology.orchestra.run.smithi012.stdout: "timeout": 90.0, 2024-09-11T00:30:58.716 INFO:teuthology.orchestra.run.smithi012.stdout: "expiration": 138.8, 2024-09-11T00:30:58.717 INFO:teuthology.orchestra.run.smithi012.stdout: "members": { 2024-09-11T00:30:58.717 INFO:teuthology.orchestra.run.smithi012.stdout: "file:/": { 2024-09-11T00:30:58.717 INFO:teuthology.orchestra.run.smithi012.stdout: "excluded": false, 2024-09-11T00:30:58.717 INFO:teuthology.orchestra.run.smithi012.stdout: "state": { 2024-09-11T00:30:58.717 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "RELEASED", 2024-09-11T00:30:58.717 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 0.0 2024-09-11T00:30:58.717 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:30:58.717 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:30:58.717 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:30:58.717 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:30:58.717 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:30:58.717 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-09-11T00:30:58.718 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:58.716+0000 7fa156e00640 1 -- 172.21.15.12:0/2728840077 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fa1340b52e0 msgr2=0x7fa1340b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:30:58.719 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:58.716+0000 7fa156e00640 1 --2- 172.21.15.12:0/2728840077 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fa1340b52e0 0x7fa1340b77a0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7fa1400c92c0 tx=0x7fa140031400 comp rx=0 tx=0).stop 2024-09-11T00:30:58.719 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:58.716+0000 7fa156e00640 1 -- 172.21.15.12:0/2728840077 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa150104fa0 msgr2=0x7fa15019b450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:30:58.719 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:58.716+0000 7fa156e00640 1 --2- 172.21.15.12:0/2728840077 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa150104fa0 0x7fa15019b450 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fa14002a9e0 tx=0x7fa140003b90 comp rx=0 tx=0).stop 2024-09-11T00:30:58.719 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:58.717+0000 7fa156e00640 1 -- 172.21.15.12:0/2728840077 shutdown_connections 2024-09-11T00:30:58.719 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:58.717+0000 7fa156e00640 1 --2- 172.21.15.12:0/2728840077 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fa1340b52e0 0x7fa1340b77a0 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:30:58.719 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:58.717+0000 7fa156e00640 1 --2- 172.21.15.12:0/2728840077 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa15010b330 0x7fa1501a0d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:30:58.720 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:58.717+0000 7fa156e00640 1 --2- 172.21.15.12:0/2728840077 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa15010a9b0 0x7fa15019b990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:30:58.720 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:58.717+0000 7fa156e00640 1 --2- 172.21.15.12:0/2728840077 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa150104fa0 0x7fa15019b450 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:30:58.720 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:58.717+0000 7fa156e00640 1 -- 172.21.15.12:0/2728840077 >> 172.21.15.12:0/2728840077 conn(0x7fa150100a50 msgr2=0x7fa150101360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:30:58.720 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:58.718+0000 7fa156e00640 1 -- 172.21.15.12:0/2728840077 shutdown_connections 2024-09-11T00:30:58.720 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:30:58.718+0000 7fa156e00640 1 -- 172.21.15.12:0/2728840077 wait complete. 2024-09-11T00:30:59.292 INFO:tasks.workunit.client.0.smithi012.stdout: 10 184966 15.30 MB/sec execute 535 sec latency 6423.016 ms 2024-09-11T00:30:59.533 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'dfb8983', total seconds elapsed: 28.8 2024-09-11T00:30:59.533 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 18.85 2024-09-11T00:30:59.533 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid ff3ad74a-6fcf-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T00:31:00.214 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:31:00.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:31:00 smithi165 ceph-mon[35356]: pgmap v1117: 129 pgs: 129 active+clean; 817 MiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 141 KiB/s rd, 125 KiB/s wr, 75 op/s 2024-09-11T00:31:00.293 INFO:tasks.workunit.client.0.smithi012.stdout: 10 185718 15.35 MB/sec execute 536 sec latency 67.223 ms 2024-09-11T00:31:00.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:31:00 smithi012 ceph-mon[30333]: pgmap v1117: 129 pgs: 129 active+clean; 817 MiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 141 KiB/s rd, 125 KiB/s wr, 75 op/s 2024-09-11T00:31:00.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:31:00 smithi178 ceph-mon[35919]: pgmap v1117: 129 pgs: 129 active+clean; 817 MiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 141 KiB/s rd, 125 KiB/s wr, 75 op/s 2024-09-11T00:31:01.292 INFO:tasks.workunit.client.0.smithi012.stdout: 10 185994 15.35 MB/sec execute 537 sec latency 562.341 ms 2024-09-11T00:31:02.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:31:02 smithi165 ceph-mon[35356]: pgmap v1118: 129 pgs: 129 active+clean; 818 MiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 213 KiB/s wr, 84 op/s 2024-09-11T00:31:02.292 INFO:tasks.workunit.client.0.smithi012.stdout: 10 186484 15.38 MB/sec execute 538 sec latency 758.754 ms 2024-09-11T00:31:02.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:31:02 smithi012 ceph-mon[30333]: pgmap v1118: 129 pgs: 129 active+clean; 818 MiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 213 KiB/s wr, 84 op/s 2024-09-11T00:31:02.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:31:02 smithi178 ceph-mon[35919]: pgmap v1118: 129 pgs: 129 active+clean; 818 MiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 213 KiB/s wr, 84 op/s 2024-09-11T00:31:02.379 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:02.375+0000 7f9646a00640 1 -- 172.21.15.12:0/1399423214 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f96401087d0 msgr2=0x7f9640108c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:31:02.379 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:02.375+0000 7f9646a00640 1 --2- 172.21.15.12:0/1399423214 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f96401087d0 0x7f9640108c10 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f962c009860 tx=0x7f962c02a380 comp rx=0 tx=0).stop 2024-09-11T00:31:02.379 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:02.375+0000 7f9646a00640 1 -- 172.21.15.12:0/1399423214 shutdown_connections 2024-09-11T00:31:02.379 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:02.375+0000 7f9646a00640 1 --2- 172.21.15.12:0/1399423214 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9640109150 0x7f964010f950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:02.379 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:02.375+0000 7f9646a00640 1 --2- 172.21.15.12:0/1399423214 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f96401087d0 0x7f9640108c10 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:02.379 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:02.375+0000 7f9646a00640 1 --2- 172.21.15.12:0/1399423214 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9640102dc0 0x7f96401031a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:02.379 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:02.375+0000 7f9646a00640 1 -- 172.21.15.12:0/1399423214 >> 172.21.15.12:0/1399423214 conn(0x7f96400fe830 msgr2=0x7f9640100c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:31:02.380 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:02.377+0000 7f9646a00640 1 -- 172.21.15.12:0/1399423214 shutdown_connections 2024-09-11T00:31:02.380 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:02.378+0000 7f9646a00640 1 -- 172.21.15.12:0/1399423214 wait complete. 2024-09-11T00:31:02.383 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:02.380+0000 7f9646a00640 1 Processor -- start 2024-09-11T00:31:02.383 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:02.381+0000 7f9646a00640 1 -- start start 2024-09-11T00:31:02.383 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:02.381+0000 7f9646a00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9640102dc0 0x7f96401993a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:31:02.384 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:02.381+0000 7f9646a00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f96401087d0 0x7f96401998e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:31:02.384 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:02.381+0000 7f9646a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9640109150 0x7f964019ec80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:31:02.384 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:02.381+0000 7f9646a00640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f9640112e10 con 0x7f9640102dc0 2024-09-11T00:31:02.384 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:02.381+0000 7f9646a00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f9640112c90 con 0x7f9640109150 2024-09-11T00:31:02.384 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:02.381+0000 7f9646a00640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f9640112f90 con 0x7f96401087d0 2024-09-11T00:31:02.385 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:02.382+0000 7f963f400640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f96401087d0 0x7f96401998e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:31:02.385 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:02.382+0000 7f963f400640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f96401087d0 0x7f96401998e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.12:44364/0 (socket says 172.21.15.12:44364) 2024-09-11T00:31:02.385 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:02.382+0000 7f963f400640 1 -- 172.21.15.12:0/408065692 learned_addr learned my addr 172.21.15.12:0/408065692 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:31:02.385 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:02.382+0000 7f963fe00640 1 --2- 172.21.15.12:0/408065692 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9640102dc0 0x7f96401993a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:31:02.385 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:02.382+0000 7f9644a00640 1 --2- 172.21.15.12:0/408065692 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9640109150 0x7f964019ec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:31:02.385 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:02.382+0000 7f963f400640 1 -- 172.21.15.12:0/408065692 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9640109150 msgr2=0x7f964019ec80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:31:02.385 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:02.382+0000 7f963f400640 1 --2- 172.21.15.12:0/408065692 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9640109150 0x7f964019ec80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:02.385 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:02.382+0000 7f963f400640 1 -- 172.21.15.12:0/408065692 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9640102dc0 msgr2=0x7f96401993a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:31:02.386 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:02.383+0000 7f963f400640 1 --2- 172.21.15.12:0/408065692 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9640102dc0 0x7f96401993a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:02.386 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:02.383+0000 7f963f400640 1 -- 172.21.15.12:0/408065692 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f964019f450 con 0x7f96401087d0 2024-09-11T00:31:02.386 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:02.383+0000 7f9644a00640 1 --2- 172.21.15.12:0/408065692 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9640109150 0x7f964019ec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T00:31:02.386 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:02.383+0000 7f963fe00640 1 --2- 172.21.15.12:0/408065692 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9640102dc0 0x7f96401993a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:31:02.387 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:02.384+0000 7f963f400640 1 --2- 172.21.15.12:0/408065692 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f96401087d0 0x7f96401998e0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f962c009980 tx=0x7f962c032cd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:31:02.387 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:02.384+0000 7f963cc00640 1 -- 172.21.15.12:0/408065692 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f962c037050 con 0x7f96401087d0 2024-09-11T00:31:02.387 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:02.384+0000 7f963cc00640 1 -- 172.21.15.12:0/408065692 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f962c043070 con 0x7f96401087d0 2024-09-11T00:31:02.387 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:02.384+0000 7f963cc00640 1 -- 172.21.15.12:0/408065692 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f962c003cd0 con 0x7f96401087d0 2024-09-11T00:31:02.387 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:02.384+0000 7f9646a00640 1 -- 172.21.15.12:0/408065692 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f964019f6e0 con 0x7f96401087d0 2024-09-11T00:31:02.387 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:02.385+0000 7f9646a00640 1 -- 172.21.15.12:0/408065692 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9640075380 con 0x7f96401087d0 2024-09-11T00:31:02.389 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:02.387+0000 7f9646a00640 1 -- 172.21.15.12:0/408065692 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9640104070 con 0x7f96401087d0 2024-09-11T00:31:02.392 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:02.390+0000 7f963cc00640 1 -- 172.21.15.12:0/408065692 <== mon.1 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7f962c004040 con 0x7f96401087d0 2024-09-11T00:31:02.395 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:02.392+0000 7f963cc00640 1 --2- 172.21.15.12:0/408065692 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f96240b9770 0x7f96240bbc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:31:02.395 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:02.393+0000 7f963cc00640 1 -- 172.21.15.12:0/408065692 <== mon.1 v2:172.21.15.178:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9208+0+0 (secure 0 0 0) 0x7f962c106040 con 0x7f96401087d0 2024-09-11T00:31:02.395 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:02.393+0000 7f963fe00640 1 --2- 172.21.15.12:0/408065692 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f96240b9770 0x7f96240bbc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:31:02.396 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:02.394+0000 7f963fe00640 1 --2- 172.21.15.12:0/408065692 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f96240b9770 0x7f96240bbc30 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f9630006fd0 tx=0x7f96300090e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:31:02.400 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:02.398+0000 7f963cc00640 1 -- 172.21.15.12:0/408065692 <== mon.1 v2:172.21.15.178:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f962c04c060 con 0x7f96401087d0 2024-09-11T00:31:02.871 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:02.868+0000 7f9646a00640 1 -- 172.21.15.12:0/408065692 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f964010af30 con 0x7f96401087d0 2024-09-11T00:31:02.872 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:02.870+0000 7f963cc00640 1 -- 172.21.15.12:0/408065692 <== mon.1 v2:172.21.15.178:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 102 v102) ==== 95+0+11973 (secure 0 0 0) 0x7f962c0ccbc0 con 0x7f96401087d0 2024-09-11T00:31:02.873 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-11T00:31:02.874 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":102,"btime":"2024-09-11T00:29:46:174935+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14577,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6835/447171591","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":447171591},{"type":"v1","addr":"172.21.15.12:6835","nonce":447171591}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14589,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/901486040","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":901486040},{"type":"v1","addr":"172.21.15.12:6837","nonce":901486040}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":102,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T00:04:04.739881+0000","modified":"2024-09-11T00:29:45.675612+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24674,"mds_1":24643,"mds_2":24638,"mds_3":24637,"mds_4":24614},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14631":{"gid":14631,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.12:6841/3069919437","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6840","nonce":3069919437},{"type":"v1","addr":"172.21.15.12:6841","nonce":3069919437}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24562":{"gid":24562,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6833/1141636112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":1141636112},{"type":"v1","addr":"172.21.15.178:6833","nonce":1141636112}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24581":{"gid":24581,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6833/3111980970","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6832","nonce":3111980970},{"type":"v1","addr":"172.21.15.165:6833","nonce":3111980970}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24607":{"gid":24607,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6835/4084868287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":4084868287},{"type":"v1","addr":"172.21.15.178:6835","nonce":4084868287}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24614":{"gid":24614,"name":"e","rank":4,"incarnation":26,"state":"up:active","state_seq":21,"addr":"172.21.15.165:6835/2425395274","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":2425395274},{"type":"v1","addr":"172.21.15.165:6835","nonce":2425395274}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24637":{"gid":24637,"name":"i","rank":3,"incarnation":24,"state":"up:active","state_seq":18,"addr":"172.21.15.178:6839/70628097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6838","nonce":70628097},{"type":"v1","addr":"172.21.15.178:6839","nonce":70628097}]},"join_fscid":-1,"export_targets":[0,2],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24638":{"gid":24638,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":18,"addr":"172.21.15.165:6837/6215088","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6836","nonce":6215088},{"type":"v1","addr":"172.21.15.165:6837","nonce":6215088}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24643":{"gid":24643,"name":"g","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.12:6839/3124093009","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6838","nonce":3124093009},{"type":"v1","addr":"172.21.15.12:6839","nonce":3124093009}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24674":{"gid":24674,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.165:6839/501312984","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6838","nonce":501312984},{"type":"v1","addr":"172.21.15.165:6839","nonce":501312984}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_27751":{"gid":27751,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6837/1436600611","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":1436600611},{"type":"v1","addr":"172.21.15.178:6837","nonce":1436600611}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24674,"qdb_cluster":[24674,24643,24638,24637,24614]},"id":1}]} 2024-09-11T00:31:02.874 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 102 2024-09-11T00:31:02.877 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:02.874+0000 7f9646a00640 1 -- 172.21.15.12:0/408065692 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f96240b9770 msgr2=0x7f96240bbc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:31:02.877 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:02.874+0000 7f9646a00640 1 --2- 172.21.15.12:0/408065692 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f96240b9770 0x7f96240bbc30 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f9630006fd0 tx=0x7f96300090e0 comp rx=0 tx=0).stop 2024-09-11T00:31:02.877 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:02.874+0000 7f9646a00640 1 -- 172.21.15.12:0/408065692 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f96401087d0 msgr2=0x7f96401998e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:31:02.877 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:02.874+0000 7f9646a00640 1 --2- 172.21.15.12:0/408065692 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f96401087d0 0x7f96401998e0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f962c009980 tx=0x7f962c032cd0 comp rx=0 tx=0).stop 2024-09-11T00:31:02.878 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:02.874+0000 7f9646a00640 1 -- 172.21.15.12:0/408065692 shutdown_connections 2024-09-11T00:31:02.878 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:02.874+0000 7f9646a00640 1 --2- 172.21.15.12:0/408065692 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f96240b9770 0x7f96240bbc30 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:02.878 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:02.874+0000 7f9646a00640 1 --2- 172.21.15.12:0/408065692 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9640109150 0x7f964019ec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:02.878 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:02.874+0000 7f9646a00640 1 --2- 172.21.15.12:0/408065692 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f96401087d0 0x7f96401998e0 secure :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f962c009980 tx=0x7f962c032cd0 comp rx=0 tx=0).stop 2024-09-11T00:31:02.878 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:02.874+0000 7f9646a00640 1 --2- 172.21.15.12:0/408065692 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9640102dc0 0x7f96401993a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:02.879 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:02.874+0000 7f9646a00640 1 -- 172.21.15.12:0/408065692 >> 172.21.15.12:0/408065692 conn(0x7f96400fe830 msgr2=0x7f9640100c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:31:02.879 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:02.875+0000 7f9646a00640 1 -- 172.21.15.12:0/408065692 shutdown_connections 2024-09-11T00:31:02.879 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:02.876+0000 7f9646a00640 1 -- 172.21.15.12:0/408065692 wait complete. 2024-09-11T00:31:03.150 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:31:03 smithi012 ceph-mon[30333]: from='client.? 172.21.15.12:0/408065692' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T00:31:03.294 INFO:tasks.workunit.client.0.smithi012.stdout: 10 187063 15.40 MB/sec execute 539 sec latency 201.467 ms 2024-09-11T00:31:03.323 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:03.320+0000 7f9a0f200640 1 -- 172.21.15.12:0/1721398283 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9a101084a0 msgr2=0x7f9a1010a960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:31:03.323 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:03.320+0000 7f9a0f200640 1 --2- 172.21.15.12:0/1721398283 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9a101084a0 0x7f9a1010a960 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f9a00009860 tx=0x7f9a0002a380 comp rx=0 tx=0).stop 2024-09-11T00:31:03.324 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:03.322+0000 7f9a0f200640 1 -- 172.21.15.12:0/1721398283 shutdown_connections 2024-09-11T00:31:03.324 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:03.322+0000 7f9a0f200640 1 --2- 172.21.15.12:0/1721398283 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9a101084a0 0x7f9a1010a960 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:03.324 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:03.322+0000 7f9a0f200640 1 --2- 172.21.15.12:0/1721398283 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9a101058f0 0x7f9a10107d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:03.324 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:03.322+0000 7f9a0f200640 1 --2- 172.21.15.12:0/1721398283 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9a1006c030 0x7f9a10105360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:03.324 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:03.322+0000 7f9a0f200640 1 -- 172.21.15.12:0/1721398283 >> 172.21.15.12:0/1721398283 conn(0x7f9a100fcf70 msgr2=0x7f9a100ff390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:31:03.324 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:03.322+0000 7f9a0f200640 1 -- 172.21.15.12:0/1721398283 shutdown_connections 2024-09-11T00:31:03.324 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:03.322+0000 7f9a0f200640 1 -- 172.21.15.12:0/1721398283 wait complete. 2024-09-11T00:31:03.327 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:03.323+0000 7f9a0f200640 1 Processor -- start 2024-09-11T00:31:03.327 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:03.325+0000 7f9a0f200640 1 -- start start 2024-09-11T00:31:03.327 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:03.325+0000 7f9a0f200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9a1006c030 0x7f9a100719f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:31:03.327 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:03.325+0000 7f9a0f200640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9a101058f0 0x7f9a10072040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:31:03.328 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:03.325+0000 7f9a0f200640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9a101084a0 0x7f9a10072c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:31:03.328 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:03.326+0000 7f9a0de00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9a1006c030 0x7f9a100719f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:31:03.328 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:03.326+0000 7f9a0de00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9a1006c030 0x7f9a100719f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.12:35316/0 (socket says 172.21.15.12:35316) 2024-09-11T00:31:03.329 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:03.326+0000 7f9a0f200640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f9a1010e470 con 0x7f9a101084a0 2024-09-11T00:31:03.329 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:03.326+0000 7f9a0f200640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f9a1010e2f0 con 0x7f9a1006c030 2024-09-11T00:31:03.329 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:03.326+0000 7f9a0f200640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f9a1010e5f0 con 0x7f9a101058f0 2024-09-11T00:31:03.329 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:03.327+0000 7f9a0de00640 1 -- 172.21.15.12:0/3069447346 learned_addr learned my addr 172.21.15.12:0/3069447346 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:31:03.329 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:03.327+0000 7f9a15200640 1 --2- 172.21.15.12:0/3069447346 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9a101084a0 0x7f9a10072c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:31:03.329 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:03.327+0000 7f9a0de00640 1 -- 172.21.15.12:0/3069447346 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9a101058f0 msgr2=0x7f9a10072040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:31:03.330 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:03.327+0000 7f9a0de00640 1 --2- 172.21.15.12:0/3069447346 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9a101058f0 0x7f9a10072040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:03.330 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:03.327+0000 7f9a0de00640 1 -- 172.21.15.12:0/3069447346 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9a101084a0 msgr2=0x7f9a10072c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:31:03.330 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:03.327+0000 7f9a0de00640 1 --2- 172.21.15.12:0/3069447346 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9a101084a0 0x7f9a10072c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:03.330 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:03.327+0000 7f9a0de00640 1 -- 172.21.15.12:0/3069447346 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a10073530 con 0x7f9a1006c030 2024-09-11T00:31:03.331 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:03.328+0000 7f9a0de00640 1 --2- 172.21.15.12:0/3069447346 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9a1006c030 0x7f9a100719f0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f9a0800ed80 tx=0x7f9a0800c6e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:31:03.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:31:03 smithi178 ceph-mon[35919]: from='client.? 172.21.15.12:0/408065692' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T00:31:03.334 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:03.329+0000 7f9a06a00640 1 -- 172.21.15.12:0/3069447346 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a0801c070 con 0x7f9a1006c030 2024-09-11T00:31:03.334 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:03.329+0000 7f9a06a00640 1 -- 172.21.15.12:0/3069447346 <== mon.2 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9a08017af0 con 0x7f9a1006c030 2024-09-11T00:31:03.334 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:03.329+0000 7f9a06a00640 1 -- 172.21.15.12:0/3069447346 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a0800fb90 con 0x7f9a1006c030 2024-09-11T00:31:03.334 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:03.329+0000 7f9a0f200640 1 -- 172.21.15.12:0/3069447346 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9a10073820 con 0x7f9a1006c030 2024-09-11T00:31:03.334 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:03.330+0000 7f9a0f200640 1 -- 172.21.15.12:0/3069447346 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9a10073f40 con 0x7f9a1006c030 2024-09-11T00:31:03.343 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:03.338+0000 7f9a06a00640 1 -- 172.21.15.12:0/3069447346 <== mon.2 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7f9a0800b030 con 0x7f9a1006c030 2024-09-11T00:31:03.343 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:03.338+0000 7f9a1822b740 1 Processor -- start 2024-09-11T00:31:03.344 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:03.340+0000 7f9a06a00640 1 --2- 172.21.15.12:0/3069447346 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f99f40b53e0 0x7f99f40b78a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:31:03.344 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:03.340+0000 7f9a06a00640 1 -- 172.21.15.12:0/3069447346 <== mon.2 v2:172.21.15.165:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9208+0+0 (secure 0 0 0) 0x7f9a080e5d20 con 0x7f9a1006c030 2024-09-11T00:31:03.345 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:03.342+0000 7f9a15200640 1 --2- 172.21.15.12:0/3069447346 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f99f40b53e0 0x7f99f40b78a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:31:03.345 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:03.342+0000 7f9a1822b740 1 -- start start 2024-09-11T00:31:03.345 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:03.343+0000 7f9a1822b740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558f9ae42970 0x558f9ae44d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:31:03.346 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:03.343+0000 7f9a1822b740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558f9ae452a0 0x558f9ae47690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:31:03.346 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:03.343+0000 7f9a1822b740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558f9ae47ce0 0x558f9ae4a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:31:03.346 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:03.343+0000 7f9a1822b740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x558f9adb56b0 con 0x558f9ae42970 2024-09-11T00:31:03.346 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:03.343+0000 7f9a1822b740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x558f9ada4ae0 con 0x558f9ae452a0 2024-09-11T00:31:03.346 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:03.343+0000 7f9a1822b740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x558f9ada88f0 con 0x558f9ae47ce0 2024-09-11T00:31:03.347 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:03.343+0000 7f9a0de00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558f9ae42970 0x558f9ae44d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:31:03.347 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:03.344+0000 7f9a0de00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558f9ae42970 0x558f9ae44d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:48828/0 (socket says 172.21.15.12:48828) 2024-09-11T00:31:03.347 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:03.344+0000 7f9a0de00640 1 -- 172.21.15.12:0/2212955523 learned_addr learned my addr 172.21.15.12:0/2212955523 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:31:03.347 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:03.344+0000 7f9a0de00640 1 -- 172.21.15.12:0/2212955523 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558f9ae47ce0 msgr2=0x558f9ae4a1a0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-11T00:31:03.347 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:03.344+0000 7f9a0de00640 1 --2- 172.21.15.12:0/2212955523 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558f9ae47ce0 0x558f9ae4a1a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:03.347 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:03.344+0000 7f9a0de00640 1 -- 172.21.15.12:0/2212955523 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558f9ae452a0 msgr2=0x558f9ae47690 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-11T00:31:03.348 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:03.344+0000 7f9a0de00640 1 --2- 172.21.15.12:0/2212955523 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558f9ae452a0 0x558f9ae47690 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:03.348 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:03.344+0000 7f9a0de00640 1 -- 172.21.15.12:0/2212955523 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558f9ad71780 con 0x558f9ae42970 2024-09-11T00:31:03.348 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:03.345+0000 7f9a0de00640 1 --2- 172.21.15.12:0/2212955523 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558f9ae42970 0x558f9ae44d60 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f9a0800ed50 tx=0x7f9a0800ef90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7117f997f5a8aa30 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:31:03.348 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:03.345+0000 7f99fb400640 1 -- 172.21.15.12:0/2212955523 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a080cf110 con 0x558f9ae42970 2024-09-11T00:31:03.348 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:03.345+0000 7f99fb400640 1 -- 172.21.15.12:0/2212955523 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9a080ac0b0 con 0x558f9ae42970 2024-09-11T00:31:03.348 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:03.345+0000 7f99fb400640 1 -- 172.21.15.12:0/2212955523 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a080b51b0 con 0x558f9ae42970 2024-09-11T00:31:03.349 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:03.346+0000 7f9a1822b740 1 -- 172.21.15.12:0/2212955523 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558f9ae42970 msgr2=0x558f9ae44d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:31:03.349 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:03.346+0000 7f9a1822b740 1 --2- 172.21.15.12:0/2212955523 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558f9ae42970 0x558f9ae44d60 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f9a0800ed50 tx=0x7f9a0800ef90 comp rx=0 tx=0).stop 2024-09-11T00:31:03.349 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:03.347+0000 7f9a1822b740 1 -- 172.21.15.12:0/2212955523 shutdown_connections 2024-09-11T00:31:03.349 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:03.347+0000 7f9a1822b740 1 --2- 172.21.15.12:0/2212955523 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558f9ae47ce0 0x558f9ae4a1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:03.350 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:03.347+0000 7f9a1822b740 1 --2- 172.21.15.12:0/2212955523 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558f9ae452a0 0x558f9ae47690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:03.350 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:03.347+0000 7f9a1822b740 1 --2- 172.21.15.12:0/2212955523 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558f9ae42970 0x558f9ae44d60 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:03.350 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:03.347+0000 7f9a1822b740 1 -- 172.21.15.12:0/2212955523 >> 172.21.15.12:0/2212955523 conn(0x558f9ae3dac0 msgr2=0x558f9ae3e850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:31:03.351 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:03.347+0000 7f9a1822b740 1 -- 172.21.15.12:0/2212955523 shutdown_connections 2024-09-11T00:31:03.352 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:03.347+0000 7f9a1822b740 1 -- 172.21.15.12:0/2212955523 wait complete. 2024-09-11T00:31:03.352 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:03.348+0000 7f9a1822b740 1 -- start start 2024-09-11T00:31:03.352 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:03.348+0000 7f9a1822b740 1 Processor -- start 2024-09-11T00:31:03.352 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:03.350+0000 7f9a1822b740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558f9ae42970 0x558f9ae4c100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:31:03.353 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:03.350+0000 7f9a1822b740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558f9ae452a0 0x558f9ae4c7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:31:03.353 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:03.350+0000 7f9a1822b740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558f9ae47ce0 0x558f9ae4d370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:31:03.354 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:03.350+0000 7f9a0de00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558f9ae42970 0x558f9ae4c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:31:03.355 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:03.351+0000 7f9a0de00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558f9ae42970 0x558f9ae4c100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.12:35326/0 (socket says 172.21.15.12:35326) 2024-09-11T00:31:03.355 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:03.351+0000 7f9a0d400640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558f9ae452a0 0x558f9ae4c7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:31:03.355 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:03.351+0000 7f9a0d400640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558f9ae452a0 0x558f9ae4c7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.12:44392/0 (socket says 172.21.15.12:44392) 2024-09-11T00:31:03.355 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:03.351+0000 7f9a15200640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558f9ae47ce0 0x558f9ae4d370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:31:03.355 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:03.352+0000 7f9a15200640 1 --2- 172.21.15.12:0/3069447346 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f99f40b53e0 0x7f99f40b78a0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f9a0000b230 tx=0x7f9a0000b040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:31:03.355 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:03.352+0000 7f9a0d400640 1 -- 172.21.15.12:0/914959750 learned_addr learned my addr 172.21.15.12:0/914959750 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:31:03.355 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:03.352+0000 7f9a1822b740 1 -- 172.21.15.12:0/914959750 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x558f9acdcab0 con 0x558f9ae47ce0 2024-09-11T00:31:03.355 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:03.352+0000 7f9a1822b740 1 -- 172.21.15.12:0/914959750 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x558f9ad4b570 con 0x558f9ae42970 2024-09-11T00:31:03.355 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:03.352+0000 7f9a1822b740 1 -- 172.21.15.12:0/914959750 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x558f9ae4acc0 con 0x558f9ae452a0 2024-09-11T00:31:03.355 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:03.352+0000 7f9a1822b740 10 client.0 _ll_register_callbacks cb 0x558f9ad5a100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:31:03.356 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:03.352+0000 7f9a0de00640 1 -- 172.21.15.12:0/914959750 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558f9ae452a0 msgr2=0x558f9ae4c7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:31:03.357 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:03.352+0000 7f9a0de00640 1 --2- 172.21.15.12:0/914959750 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558f9ae452a0 0x558f9ae4c7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:03.357 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:03.352+0000 7f9a0de00640 1 -- 172.21.15.12:0/914959750 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558f9ae47ce0 msgr2=0x558f9ae4d370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:31:03.357 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:03.352+0000 7f9a0de00640 1 --2- 172.21.15.12:0/914959750 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558f9ae47ce0 0x558f9ae4d370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:03.357 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:03.352+0000 7f9a0de00640 1 -- 172.21.15.12:0/914959750 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558f9ab06790 con 0x558f9ae42970 2024-09-11T00:31:03.357 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:03.353+0000 7f99efe00640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-09-11T00:31:03.357 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:03.353+0000 7f9a0de00640 1 --2- 172.21.15.12:0/914959750 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558f9ae42970 0x558f9ae4c100 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f9a080abd60 tx=0x7f9a080ac8e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9daadeb6835e3186 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:31:03.357 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:03.353+0000 7f99efe00640 1 -- 172.21.15.12:0/914959750 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a080cf110 con 0x558f9ae42970 2024-09-11T00:31:03.357 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:03.354+0000 7f99efe00640 1 -- 172.21.15.12:0/914959750 <== mon.2 v2:172.21.15.165:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 9208+0+0 (secure 0 0 0) 0x7f9a08017840 con 0x558f9ae42970 2024-09-11T00:31:03.357 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:03.354+0000 7f9a1822b740 1 -- 172.21.15.12:0/914959750 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9a080ac0b0 con 0x558f9ae42970 2024-09-11T00:31:03.357 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:03.354+0000 7f99efe00640 1 -- 172.21.15.12:0/914959750 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=103) ==== 24+0+0 (secure 0 0 0) 0x7f9a080b5640 con 0x558f9ae42970 2024-09-11T00:31:03.358 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:03.355+0000 7f9a1822b740 10 client.28055 fetch_fsmap learned FSMap version 103 2024-09-11T00:31:03.358 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:03.355+0000 7f9a1822b740 1 -- 172.21.15.12:0/914959750 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=103,monmap=4+}) -- 0x558f9ab7f620 con 0x558f9ae42970 2024-09-11T00:31:03.358 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:03.355+0000 7f99efe00640 1 -- 172.21.15.12:0/914959750 <== mon.2 v2:172.21.15.165:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9a080ac0b0 con 0x558f9ae42970 2024-09-11T00:31:03.359 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:03.355+0000 7f99efe00640 1 -- 172.21.15.12:0/914959750 <== mon.2 v2:172.21.15.165:3300/0 5 ==== fsmap(e 103) ==== 8374+0+0 (secure 0 0 0) 0x7f9a080aad80 con 0x558f9ae42970 2024-09-11T00:31:03.359 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:03.355+0000 7f9a1822b740 10 client.28055 fetch_fsmap finished waiting for FSMap version 103 2024-09-11T00:31:03.359 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:03.355+0000 7f9a1822b740 10 client.28055 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T00:31:03.359 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:03.356+0000 7f9a1822b740 20 client.28055 populate_metadata read hostname 'smithi012' 2024-09-11T00:31:03.359 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:03.356+0000 7f9a1822b740 1 --2- 172.21.15.12:0/914959750 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x558f9ae57580 0x558f9ae59970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:31:03.359 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:03.356+0000 7f9a1822b740 4 client.28055 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T00:31:03.359 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:03.356+0000 7f9a1822b740 1 -- 172.21.15.12:0/914959750 --> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558f9adb7600 con 0x558f9ae57580 2024-09-11T00:31:03.359 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:03.356+0000 7f9a0d400640 1 --2- 172.21.15.12:0/914959750 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x558f9ae57580 0x558f9ae59970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:31:03.366 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:03.359+0000 7f9a0d400640 1 --2- 172.21.15.12:0/914959750 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x558f9ae57580 0x558f9ae59970 crc :-1 s=READY pgs=325 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=82d27d0586d445d3 server_cookie=ac95f49d9f880c3f in_seq=0 out_seq=0 2024-09-11T00:31:03.366 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:03.363+0000 7f99efe00640 10 client.28055 ms_handle_connect on v2:172.21.15.165:6838/501312984 2024-09-11T00:31:03.366 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:03.363+0000 7f99efe00640 1 -- 172.21.15.12:0/914959750 <== mds.0 v2:172.21.15.165:6838/501312984 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x558f9adb7600 con 0x558f9ae57580 2024-09-11T00:31:03.366 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:03.363+0000 7f99efe00640 10 client.28055 handle_command_reply: tid=0 2024-09-11T00:31:03.367 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:03.363+0000 7f9a1822b740 1 client.28055 shutdown 2024-09-11T00:31:03.367 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:03.363+0000 7f9a1822b740 1 -- 172.21.15.12:0/914959750 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558f9ae42970 msgr2=0x558f9ae4c100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:31:03.367 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:03.363+0000 7f9a1822b740 1 --2- 172.21.15.12:0/914959750 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558f9ae42970 0x558f9ae4c100 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f9a080abd60 tx=0x7f9a080ac8e0 comp rx=0 tx=0).stop 2024-09-11T00:31:03.367 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:03.363+0000 7f9a1822b740 1 -- 172.21.15.12:0/914959750 shutdown_connections 2024-09-11T00:31:03.367 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:03.363+0000 7f9a1822b740 1 --2- 172.21.15.12:0/914959750 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x558f9ae57580 0x558f9ae59970 crc :-1 s=READY pgs=325 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:03.367 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:03.363+0000 7f9a1822b740 1 --2- 172.21.15.12:0/914959750 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558f9ae47ce0 0x558f9ae4d370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:03.367 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:03.363+0000 7f9a1822b740 1 --2- 172.21.15.12:0/914959750 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558f9ae452a0 0x558f9ae4c7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:03.367 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:03.363+0000 7f9a1822b740 1 --2- 172.21.15.12:0/914959750 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558f9ae42970 0x558f9ae4c100 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:03.367 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:03.363+0000 7f9a1822b740 1 -- 172.21.15.12:0/914959750 >> 172.21.15.12:0/914959750 conn(0x558f9ae3dac0 msgr2=0x558f9ae3cbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:31:03.367 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:03.363+0000 7f99efe00640 0 client.28055 ms_handle_reset on v2:172.21.15.165:6838/501312984 2024-09-11T00:31:03.367 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:03.364+0000 7f9a1822b740 1 -- 172.21.15.12:0/914959750 shutdown_connections 2024-09-11T00:31:03.367 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:03.364+0000 7f9a1822b740 1 -- 172.21.15.12:0/914959750 wait complete. 2024-09-11T00:31:03.367 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:03.364+0000 7f9a1822b740 20 client.28055 trim_cache size 0 max 16384 2024-09-11T00:31:03.417 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:03.414+0000 7f9a1822b740 1 Processor -- start 2024-09-11T00:31:03.418 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:03.415+0000 7f9a1822b740 1 -- start start 2024-09-11T00:31:03.418 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:03.416+0000 7f9a1822b740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558f9ae42970 0x558f9ae38ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:31:03.418 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:03.416+0000 7f9a1822b740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558f9ae452a0 0x558f9ae48110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:31:03.418 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:03.416+0000 7f9a1822b740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558f9ae47ce0 0x558f9ae48650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:31:03.418 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:03.416+0000 7f9a1822b740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f9a101b1d10 con 0x558f9ae452a0 2024-09-11T00:31:03.419 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:03.416+0000 7f9a1822b740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f9a101b3880 con 0x558f9ae47ce0 2024-09-11T00:31:03.419 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:03.416+0000 7f9a1822b740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f9a101b2410 con 0x558f9ae42970 2024-09-11T00:31:03.419 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:03.417+0000 7f9a0d400640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558f9ae452a0 0x558f9ae48110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:31:03.419 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:03.417+0000 7f9a0d400640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558f9ae452a0 0x558f9ae48110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:48858/0 (socket says 172.21.15.12:48858) 2024-09-11T00:31:03.419 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:03.417+0000 7f9a0d400640 1 -- 172.21.15.12:0/2969064748 learned_addr learned my addr 172.21.15.12:0/2969064748 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:31:03.420 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:03.417+0000 7f9a0de00640 1 --2- 172.21.15.12:0/2969064748 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558f9ae42970 0x558f9ae38ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:31:03.420 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:03.418+0000 7f9a0de00640 1 -- 172.21.15.12:0/2969064748 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558f9ae47ce0 msgr2=0x558f9ae48650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:31:03.421 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:03.418+0000 7f9a0de00640 1 --2- 172.21.15.12:0/2969064748 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558f9ae47ce0 0x558f9ae48650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:03.421 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:03.418+0000 7f9a0de00640 1 -- 172.21.15.12:0/2969064748 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558f9ae452a0 msgr2=0x558f9ae48110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:31:03.421 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:03.418+0000 7f9a0de00640 1 --2- 172.21.15.12:0/2969064748 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558f9ae452a0 0x558f9ae48110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:03.421 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:03.418+0000 7f9a0de00640 1 -- 172.21.15.12:0/2969064748 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558f9ae450d0 con 0x558f9ae42970 2024-09-11T00:31:03.422 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:03.420+0000 7f9a0de00640 1 --2- 172.21.15.12:0/2969064748 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558f9ae42970 0x558f9ae38ac0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f9a0800ed50 tx=0x7f9a0800ef90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=620a3785915f09e6 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:31:03.425 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:03.423+0000 7f99faa00640 1 -- 172.21.15.12:0/2969064748 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a080a8700 con 0x558f9ae42970 2024-09-11T00:31:03.425 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:03.423+0000 7f99faa00640 1 -- 172.21.15.12:0/2969064748 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9a080cf110 con 0x558f9ae42970 2024-09-11T00:31:03.425 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:03.423+0000 7f99faa00640 1 -- 172.21.15.12:0/2969064748 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a080abbc0 con 0x558f9ae42970 2024-09-11T00:31:03.426 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:03.424+0000 7f9a1822b740 1 -- 172.21.15.12:0/2969064748 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558f9ae42970 msgr2=0x558f9ae38ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:31:03.427 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:03.424+0000 7f9a1822b740 1 --2- 172.21.15.12:0/2969064748 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558f9ae42970 0x558f9ae38ac0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f9a0800ed50 tx=0x7f9a0800ef90 comp rx=0 tx=0).stop 2024-09-11T00:31:03.427 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:03.425+0000 7f9a1822b740 1 -- 172.21.15.12:0/2969064748 shutdown_connections 2024-09-11T00:31:03.427 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:03.425+0000 7f9a1822b740 1 --2- 172.21.15.12:0/2969064748 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558f9ae47ce0 0x558f9ae48650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:03.427 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:03.425+0000 7f9a1822b740 1 --2- 172.21.15.12:0/2969064748 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558f9ae452a0 0x558f9ae48110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:03.427 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:03.425+0000 7f9a1822b740 1 --2- 172.21.15.12:0/2969064748 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558f9ae42970 0x558f9ae38ac0 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:03.428 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:03.425+0000 7f9a1822b740 1 -- 172.21.15.12:0/2969064748 >> 172.21.15.12:0/2969064748 conn(0x558f9ae3dac0 msgr2=0x558f9ae3e4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:31:03.428 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:03.425+0000 7f9a1822b740 1 -- 172.21.15.12:0/2969064748 shutdown_connections 2024-09-11T00:31:03.428 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:03.425+0000 7f9a1822b740 1 -- 172.21.15.12:0/2969064748 wait complete. 2024-09-11T00:31:03.428 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:03.426+0000 7f9a1822b740 1 -- start start 2024-09-11T00:31:03.428 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:03.426+0000 7f9a1822b740 1 Processor -- start 2024-09-11T00:31:03.429 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:03.427+0000 7f9a1822b740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558f9ae42970 0x558f9ae52d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:31:03.429 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:03.427+0000 7f9a1822b740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558f9ae452a0 0x558f9ae53250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:31:03.430 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:03.427+0000 7f9a1822b740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558f9ae47ce0 0x558f9ae53790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:31:03.430 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:03.427+0000 7f9a1822b740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f9a101b31b0 con 0x558f9ae452a0 2024-09-11T00:31:03.430 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:03.427+0000 7f9a1822b740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f9a101b2ae0 con 0x558f9ae42970 2024-09-11T00:31:03.430 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:03.427+0000 7f9a1822b740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f9a101afcf0 con 0x558f9ae47ce0 2024-09-11T00:31:03.430 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:03.427+0000 7f9a1822b740 10 client.0 _ll_register_callbacks cb 0x7f9a08091070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:31:03.430 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:03.427+0000 7f9a0d400640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558f9ae452a0 0x558f9ae53250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:31:03.430 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:03.428+0000 7f9a15200640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558f9ae47ce0 0x558f9ae53790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:31:03.430 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:03.428+0000 7f9a15200640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558f9ae47ce0 0x558f9ae53790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.12:44414/0 (socket says 172.21.15.12:44414) 2024-09-11T00:31:03.430 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:03.428+0000 7f9a15200640 1 -- 172.21.15.12:0/4239995829 learned_addr learned my addr 172.21.15.12:0/4239995829 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:31:03.431 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:03.428+0000 7f9a15200640 1 -- 172.21.15.12:0/4239995829 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558f9ae42970 msgr2=0x558f9ae52d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:31:03.431 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:03.428+0000 7f9a15200640 1 --2- 172.21.15.12:0/4239995829 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558f9ae42970 0x558f9ae52d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:03.431 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:03.428+0000 7f9a15200640 1 -- 172.21.15.12:0/4239995829 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558f9ae452a0 msgr2=0x558f9ae53250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:31:03.431 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:03.428+0000 7f9a15200640 1 --2- 172.21.15.12:0/4239995829 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558f9ae452a0 0x558f9ae53250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:03.431 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:03.428+0000 7f9a15200640 1 -- 172.21.15.12:0/4239995829 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558f9ae4d6e0 con 0x558f9ae47ce0 2024-09-11T00:31:03.432 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:03.429+0000 7f9a15200640 1 --2- 172.21.15.12:0/4239995829 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558f9ae47ce0 0x558f9ae53790 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f9a000058c0 tx=0x7f9a0002a380 comp rx=0 tx=0).ready entity=mon.1 client_cookie=20fa7700b34c4cc5 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:31:03.432 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:03.429+0000 7f99efe00640 10 client.0 ms_handle_connect on v2:172.21.15.178:3300/0 2024-09-11T00:31:03.432 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:03.429+0000 7f99efe00640 1 -- 172.21.15.12:0/4239995829 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a00005460 con 0x558f9ae47ce0 2024-09-11T00:31:03.432 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:03.429+0000 7f99efe00640 1 -- 172.21.15.12:0/4239995829 <== mon.1 v2:172.21.15.178:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 9208+0+0 (secure 0 0 0) 0x7f9a000046c0 con 0x558f9ae47ce0 2024-09-11T00:31:03.433 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:03.429+0000 7f9a1822b740 1 -- 172.21.15.12:0/4239995829 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9a080cf110 con 0x558f9ae47ce0 2024-09-11T00:31:03.433 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:03.430+0000 7f99efe00640 1 -- 172.21.15.12:0/4239995829 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=103) ==== 24+0+0 (secure 0 0 0) 0x7f9a00005600 con 0x558f9ae47ce0 2024-09-11T00:31:03.433 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:03.430+0000 7f9a1822b740 10 client.28027 fetch_fsmap learned FSMap version 103 2024-09-11T00:31:03.433 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:03.430+0000 7f9a1822b740 1 -- 172.21.15.12:0/4239995829 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,fsmap=103,monmap=4+}) -- 0x558f9aec45d0 con 0x558f9ae47ce0 2024-09-11T00:31:03.433 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:03.430+0000 7f99efe00640 1 -- 172.21.15.12:0/4239995829 <== mon.1 v2:172.21.15.178:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9a080cf110 con 0x558f9ae47ce0 2024-09-11T00:31:03.433 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:03.430+0000 7f99efe00640 1 -- 172.21.15.12:0/4239995829 <== mon.1 v2:172.21.15.178:3300/0 5 ==== fsmap(e 103) ==== 8374+0+0 (secure 0 0 0) 0x7f9a00003af0 con 0x558f9ae47ce0 2024-09-11T00:31:03.433 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:03.430+0000 7f9a1822b740 10 client.28027 fetch_fsmap finished waiting for FSMap version 103 2024-09-11T00:31:03.434 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:03.430+0000 7f9a1822b740 10 client.28027 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T00:31:03.434 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:03.430+0000 7f9a1822b740 20 client.28027 populate_metadata read hostname 'smithi012' 2024-09-11T00:31:03.434 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:03.431+0000 7f9a1822b740 1 --2- 172.21.15.12:0/4239995829 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x558f9aec5000 0x558f9aec7420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:31:03.434 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:03.431+0000 7f9a1822b740 4 client.28027 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T00:31:03.434 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:03.431+0000 7f9a1822b740 1 -- 172.21.15.12:0/4239995829 --> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] -- command(tid 0: {"prefix": "flush journal"}) -- 0x558f9adb30b0 con 0x558f9aec5000 2024-09-11T00:31:03.434 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:03.431+0000 7f9a0de00640 1 --2- 172.21.15.12:0/4239995829 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x558f9aec5000 0x558f9aec7420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:31:03.436 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:03.434+0000 7f9a0de00640 1 --2- 172.21.15.12:0/4239995829 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x558f9aec5000 0x558f9aec7420 crc :-1 s=READY pgs=326 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7444caea4fba66cb server_cookie=bd537dcbf384905c in_seq=0 out_seq=0 2024-09-11T00:31:03.438 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:03.434+0000 7f99efe00640 10 client.28027 ms_handle_connect on v2:172.21.15.165:6838/501312984 2024-09-11T00:31:03.533 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:31:03 smithi165 ceph-mon[35356]: from='client.? 172.21.15.12:0/408065692' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T00:31:03.694 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:03.690+0000 7f99efe00640 1 -- 172.21.15.12:0/4239995829 <== mds.0 v2:172.21.15.165:6838/501312984 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x558f9adb30b0 con 0x558f9aec5000 2024-09-11T00:31:03.694 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:03.690+0000 7f99efe00640 10 client.28027 handle_command_reply: tid=0 2024-09-11T00:31:03.695 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:03.690+0000 7f9a1822b740 1 client.28027 shutdown 2024-09-11T00:31:03.695 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:03.691+0000 7f9a1822b740 1 -- 172.21.15.12:0/4239995829 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558f9ae47ce0 msgr2=0x558f9ae53790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:31:03.695 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:03.691+0000 7f9a1822b740 1 --2- 172.21.15.12:0/4239995829 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558f9ae47ce0 0x558f9ae53790 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f9a000058c0 tx=0x7f9a0002a380 comp rx=0 tx=0).stop 2024-09-11T00:31:03.695 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:03.691+0000 7f9a1822b740 1 -- 172.21.15.12:0/4239995829 shutdown_connections 2024-09-11T00:31:03.695 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:03.691+0000 7f9a1822b740 1 --2- 172.21.15.12:0/4239995829 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x558f9aec5000 0x558f9aec7420 crc :-1 s=READY pgs=326 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:03.696 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:03.691+0000 7f9a1822b740 1 --2- 172.21.15.12:0/4239995829 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558f9ae47ce0 0x558f9ae53790 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:03.696 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:03.691+0000 7f9a1822b740 1 --2- 172.21.15.12:0/4239995829 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558f9ae452a0 0x558f9ae53250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:03.696 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:03.691+0000 7f9a1822b740 1 --2- 172.21.15.12:0/4239995829 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558f9ae42970 0x558f9ae52d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:03.696 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:03.691+0000 7f9a1822b740 1 -- 172.21.15.12:0/4239995829 >> 172.21.15.12:0/4239995829 conn(0x558f9ae3dac0 msgr2=0x558f9ae41a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:31:03.696 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:03.691+0000 7f99efe00640 0 client.28027 ms_handle_reset on v2:172.21.15.165:6838/501312984 2024-09-11T00:31:03.696 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:03.691+0000 7f9a1822b740 1 -- 172.21.15.12:0/4239995829 shutdown_connections 2024-09-11T00:31:03.697 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:03.691+0000 7f9a1822b740 1 -- 172.21.15.12:0/4239995829 wait complete. 2024-09-11T00:31:03.697 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:03.692+0000 7f9a1822b740 20 client.28027 trim_cache size 0 max 16384 2024-09-11T00:31:03.698 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:{ 2024-09-11T00:31:03.698 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "message": "", 2024-09-11T00:31:03.698 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "return_code": 0 2024-09-11T00:31:03.698 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:} 2024-09-11T00:31:03.699 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:03.694+0000 7f9a0f200640 1 -- 172.21.15.12:0/3069447346 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f99f40b53e0 msgr2=0x7f99f40b78a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:31:03.699 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:03.694+0000 7f9a0f200640 1 --2- 172.21.15.12:0/3069447346 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f99f40b53e0 0x7f99f40b78a0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f9a0000b230 tx=0x7f9a0000b040 comp rx=0 tx=0).stop 2024-09-11T00:31:03.700 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:03.694+0000 7f9a0f200640 1 -- 172.21.15.12:0/3069447346 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9a1006c030 msgr2=0x7f9a100719f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:31:03.700 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:03.694+0000 7f9a0f200640 1 --2- 172.21.15.12:0/3069447346 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9a1006c030 0x7f9a100719f0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f9a0800ed80 tx=0x7f9a0800c6e0 comp rx=0 tx=0).stop 2024-09-11T00:31:03.700 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:03.694+0000 7f9a0f200640 1 -- 172.21.15.12:0/3069447346 shutdown_connections 2024-09-11T00:31:03.700 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:03.694+0000 7f9a0f200640 1 --2- 172.21.15.12:0/3069447346 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f99f40b53e0 0x7f99f40b78a0 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:03.700 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:03.694+0000 7f9a0f200640 1 --2- 172.21.15.12:0/3069447346 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9a101084a0 0x7f9a10072c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:03.700 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:03.694+0000 7f9a0f200640 1 --2- 172.21.15.12:0/3069447346 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9a101058f0 0x7f9a10072040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:03.701 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:03.694+0000 7f9a0f200640 1 --2- 172.21.15.12:0/3069447346 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9a1006c030 0x7f9a100719f0 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:03.701 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:03.694+0000 7f9a0f200640 1 -- 172.21.15.12:0/3069447346 >> 172.21.15.12:0/3069447346 conn(0x7f9a100fcf70 msgr2=0x7f9a10106570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:31:03.701 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:03.694+0000 7f9a0f200640 1 -- 172.21.15.12:0/3069447346 shutdown_connections 2024-09-11T00:31:03.701 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:03.694+0000 7f9a0f200640 1 -- 172.21.15.12:0/3069447346 wait complete. 2024-09-11T00:31:04.260 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:31:04 smithi012 ceph-mon[30333]: pgmap v1119: 129 pgs: 129 active+clean; 828 MiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 1.3 MiB/s wr, 192 op/s 2024-09-11T00:31:04.261 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:31:04 smithi012 ceph-mon[30333]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T00:31:04.294 INFO:tasks.workunit.client.0.smithi012.stdout: 10 187432 15.41 MB/sec execute 540 sec latency 162.196 ms 2024-09-11T00:31:04.330 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24674 quiesce db / --timeout 90 --expiration 138.85 --await-for 45.0' 2024-09-11T00:31:04.331 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid ff3ad74a-6fcf-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24674 quiesce db / --timeout 90 --expiration 138.85 --await-for 45.0 2024-09-11T00:31:04.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:31:04 smithi178 ceph-mon[35919]: pgmap v1119: 129 pgs: 129 active+clean; 828 MiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 1.3 MiB/s wr, 192 op/s 2024-09-11T00:31:04.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:31:04 smithi178 ceph-mon[35919]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T00:31:04.532 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:31:04 smithi165 ceph-mon[35356]: pgmap v1119: 129 pgs: 129 active+clean; 828 MiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 1.3 MiB/s wr, 192 op/s 2024-09-11T00:31:04.533 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:31:04 smithi165 ceph-mon[35356]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T00:31:05.083 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:31:05.294 INFO:tasks.workunit.client.0.smithi012.stdout: 10 187830 15.43 MB/sec execute 541 sec latency 154.502 ms 2024-09-11T00:31:06.294 INFO:tasks.workunit.client.0.smithi012.stdout: 10 188214 15.42 MB/sec execute 542 sec latency 567.834 ms 2024-09-11T00:31:06.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:31:06 smithi012 ceph-mon[30333]: pgmap v1120: 129 pgs: 129 active+clean; 828 MiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 1.8 MiB/s wr, 204 op/s 2024-09-11T00:31:06.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:31:06 smithi178 ceph-mon[35919]: pgmap v1120: 129 pgs: 129 active+clean; 828 MiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 1.8 MiB/s wr, 204 op/s 2024-09-11T00:31:06.532 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:31:06 smithi165 ceph-mon[35356]: pgmap v1120: 129 pgs: 129 active+clean; 828 MiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 1.8 MiB/s wr, 204 op/s 2024-09-11T00:31:06.940 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:06.932+0000 7f712ce00640 1 -- 172.21.15.12:0/2074973025 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f712811c0a0 msgr2=0x7f712811e490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:31:06.941 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:06.932+0000 7f712ce00640 1 --2- 172.21.15.12:0/2074973025 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f712811c0a0 0x7f712811e490 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f7120009be0 tx=0x7f712002a750 comp rx=0 tx=0).stop 2024-09-11T00:31:06.941 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:06.932+0000 7f712ce00640 1 -- 172.21.15.12:0/2074973025 shutdown_connections 2024-09-11T00:31:06.941 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:06.932+0000 7f712ce00640 1 --2- 172.21.15.12:0/2074973025 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f712811c0a0 0x7f712811e490 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:06.941 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:06.932+0000 7f712ce00640 1 --2- 172.21.15.12:0/2074973025 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f712810a530 0x7f712810a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:06.941 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:06.932+0000 7f712ce00640 1 --2- 172.21.15.12:0/2074973025 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f712810a150 0x7f71281111b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:06.941 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:06.932+0000 7f712ce00640 1 -- 172.21.15.12:0/2074973025 >> 172.21.15.12:0/2074973025 conn(0x7f712806dfc0 msgr2=0x7f712806e3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:31:06.941 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:06.932+0000 7f712ce00640 1 -- 172.21.15.12:0/2074973025 shutdown_connections 2024-09-11T00:31:06.941 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:06.932+0000 7f712ce00640 1 -- 172.21.15.12:0/2074973025 wait complete. 2024-09-11T00:31:06.941 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:06.934+0000 7f712ce00640 1 Processor -- start 2024-09-11T00:31:06.941 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:06.934+0000 7f712ce00640 1 -- start start 2024-09-11T00:31:06.942 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:06.935+0000 7f712ce00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f712810a150 0x7f71281176e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:31:06.942 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:06.935+0000 7f712ce00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f712810a530 0x7f7128117c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:31:06.942 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:06.935+0000 7f712ce00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f712811c0a0 0x7f71281a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:31:06.942 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:06.936+0000 7f7126800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f712811c0a0 0x7f71281a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:31:06.942 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:06.936+0000 7f7126800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f712811c0a0 0x7f71281a5040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.12:35368/0 (socket says 172.21.15.12:35368) 2024-09-11T00:31:06.942 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:06.936+0000 7f7125e00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f712810a150 0x7f71281176e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:31:06.942 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:06.936+0000 7f7125e00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f712810a150 0x7f71281176e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.12:44442/0 (socket says 172.21.15.12:44442) 2024-09-11T00:31:06.942 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:06.936+0000 7f7126800640 1 -- 172.21.15.12:0/1218687957 learned_addr learned my addr 172.21.15.12:0/1218687957 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:31:06.942 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:06.936+0000 7f712ce00640 1 -- 172.21.15.12:0/1218687957 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f7128114100 con 0x7f712810a530 2024-09-11T00:31:06.942 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:06.936+0000 7f712ce00640 1 -- 172.21.15.12:0/1218687957 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f7128113f80 con 0x7f712811c0a0 2024-09-11T00:31:06.942 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:06.936+0000 7f712ce00640 1 -- 172.21.15.12:0/1218687957 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f7128114280 con 0x7f712810a150 2024-09-11T00:31:06.942 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:06.937+0000 7f7126800640 1 -- 172.21.15.12:0/1218687957 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f712810a150 msgr2=0x7f71281176e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:31:06.942 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:06.937+0000 7f7126800640 1 --2- 172.21.15.12:0/1218687957 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f712810a150 0x7f71281176e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:06.943 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:06.937+0000 7f7126800640 1 -- 172.21.15.12:0/1218687957 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f712810a530 msgr2=0x7f7128117c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:31:06.943 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:06.937+0000 7f7126800640 1 --2- 172.21.15.12:0/1218687957 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f712810a530 0x7f7128117c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:06.943 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:06.937+0000 7f7126800640 1 -- 172.21.15.12:0/1218687957 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71281a57c0 con 0x7f712811c0a0 2024-09-11T00:31:06.943 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:06.938+0000 7f7126800640 1 --2- 172.21.15.12:0/1218687957 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f712811c0a0 0x7f71281a5040 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f712002ad40 tx=0x7f7120003e60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:31:06.943 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:06.939+0000 7f711ea00640 1 -- 172.21.15.12:0/1218687957 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7120041020 con 0x7f712811c0a0 2024-09-11T00:31:06.943 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:06.939+0000 7f712ce00640 1 -- 172.21.15.12:0/1218687957 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71281a5a50 con 0x7f712811c0a0 2024-09-11T00:31:06.951 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:06.940+0000 7f712ce00640 1 -- 172.21.15.12:0/1218687957 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71281ac0f0 con 0x7f712811c0a0 2024-09-11T00:31:06.951 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:06.940+0000 7f711ea00640 1 -- 172.21.15.12:0/1218687957 <== mon.2 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7120030de0 con 0x7f712811c0a0 2024-09-11T00:31:06.951 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:06.940+0000 7f711ea00640 1 -- 172.21.15.12:0/1218687957 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7120036040 con 0x7f712811c0a0 2024-09-11T00:31:06.953 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:06.948+0000 7f712fa11740 1 Processor -- start 2024-09-11T00:31:06.953 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:06.946+0000 7f711ea00640 1 -- 172.21.15.12:0/1218687957 <== mon.2 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7f7120004600 con 0x7f712811c0a0 2024-09-11T00:31:06.958 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:06.953+0000 7f712fa11740 1 -- start start 2024-09-11T00:31:06.958 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:06.956+0000 7f712fa11740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560316f00c40 0x560316eff1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:31:06.958 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:06.956+0000 7f712fa11740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560316eff6e0 0x560316effb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:31:06.959 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:06.956+0000 7f712fa11740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560316f0b510 0x560316f0d900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:31:06.959 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:06.956+0000 7f712fa11740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x560316e74a90 con 0x560316f0b510 2024-09-11T00:31:06.959 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:06.956+0000 7f712fa11740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x560316e63ec0 con 0x560316eff6e0 2024-09-11T00:31:06.959 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:06.956+0000 7f712fa11740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x560316e67cd0 con 0x560316f00c40 2024-09-11T00:31:06.964 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:06.957+0000 7f711fe00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560316f0b510 0x560316f0d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:31:06.964 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:06.961+0000 7f711fe00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560316f0b510 0x560316f0d900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:48888/0 (socket says 172.21.15.12:48888) 2024-09-11T00:31:06.964 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:06.958+0000 7f7125e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560316eff6e0 0x560316effb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:31:06.964 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:06.961+0000 7f711ea00640 1 --2- 172.21.15.12:0/1218687957 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f710c0b54b0 0x7f710c0b7970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:31:06.964 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:06.961+0000 7f711fe00640 1 -- 172.21.15.12:0/3498759371 learned_addr learned my addr 172.21.15.12:0/3498759371 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:31:06.964 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:06.961+0000 7f711fe00640 1 --2- 172.21.15.12:0/3498759371 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560316f00c40 0x560316eff1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:31:06.965 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:06.962+0000 7f711fe00640 1 -- 172.21.15.12:0/3498759371 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560316f00c40 msgr2=0x560316eff1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:31:06.965 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:06.962+0000 7f711fe00640 1 --2- 172.21.15.12:0/3498759371 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560316f00c40 0x560316eff1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:06.965 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:06.962+0000 7f711fe00640 1 -- 172.21.15.12:0/3498759371 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560316eff6e0 msgr2=0x560316effb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:31:06.965 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:06.963+0000 7f711fe00640 1 --2- 172.21.15.12:0/3498759371 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560316eff6e0 0x560316effb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:06.966 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:06.963+0000 7f711fe00640 1 -- 172.21.15.12:0/3498759371 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560316e099d0 con 0x560316f0b510 2024-09-11T00:31:06.966 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:06.963+0000 7f7125e00640 1 --2- 172.21.15.12:0/3498759371 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560316eff6e0 0x560316effb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T00:31:06.966 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:06.963+0000 7f711fe00640 1 --2- 172.21.15.12:0/3498759371 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560316f0b510 0x560316f0d900 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f7110001cb0 tx=0x7f7110009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d433a7ddc5dc217 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:31:06.967 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:06.964+0000 7f7126800640 1 --2- 172.21.15.12:0/1218687957 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f710c0b54b0 0x7f710c0b7970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:31:06.967 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:06.964+0000 7f7117400640 1 -- 172.21.15.12:0/3498759371 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7110012040 con 0x560316f0b510 2024-09-11T00:31:06.967 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:06.964+0000 7f7117400640 1 -- 172.21.15.12:0/3498759371 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7110008e30 con 0x560316f0b510 2024-09-11T00:31:06.967 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:06.964+0000 7f7117400640 1 -- 172.21.15.12:0/3498759371 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7110004ac0 con 0x560316f0b510 2024-09-11T00:31:06.968 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:06.964+0000 7f711ea00640 1 -- 172.21.15.12:0/1218687957 <== mon.2 v2:172.21.15.165:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9208+0+0 (secure 0 0 0) 0x7f7120106310 con 0x7f712811c0a0 2024-09-11T00:31:06.968 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:06.966+0000 7f712fa11740 1 -- 172.21.15.12:0/3498759371 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560316f0b510 msgr2=0x560316f0d900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:31:06.968 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:06.966+0000 7f712fa11740 1 --2- 172.21.15.12:0/3498759371 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560316f0b510 0x560316f0d900 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f7110001cb0 tx=0x7f7110009e30 comp rx=0 tx=0).stop 2024-09-11T00:31:06.969 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:06.966+0000 7f712fa11740 1 -- 172.21.15.12:0/3498759371 shutdown_connections 2024-09-11T00:31:06.969 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:06.966+0000 7f712fa11740 1 --2- 172.21.15.12:0/3498759371 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560316f0b510 0x560316f0d900 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:06.970 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:06.966+0000 7f712fa11740 1 --2- 172.21.15.12:0/3498759371 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560316eff6e0 0x560316effb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:06.970 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:06.966+0000 7f712fa11740 1 --2- 172.21.15.12:0/3498759371 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560316f00c40 0x560316eff1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:06.970 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:06.966+0000 7f712fa11740 1 -- 172.21.15.12:0/3498759371 >> 172.21.15.12:0/3498759371 conn(0x560316ec96f0 msgr2=0x560316efcfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:31:06.970 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:06.966+0000 7f712fa11740 1 -- 172.21.15.12:0/3498759371 shutdown_connections 2024-09-11T00:31:06.970 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:06.966+0000 7f712fa11740 1 -- 172.21.15.12:0/3498759371 wait complete. 2024-09-11T00:31:06.971 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:06.967+0000 7f712fa11740 1 -- start start 2024-09-11T00:31:06.971 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:06.967+0000 7f712fa11740 1 Processor -- start 2024-09-11T00:31:06.971 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:06.968+0000 7f7126800640 1 --2- 172.21.15.12:0/1218687957 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f710c0b54b0 0x7f710c0b7970 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f7120009540 tx=0x7f71201099e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:31:06.972 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:06.970+0000 7f712fa11740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560316eff6e0 0x560316f08430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:31:06.973 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:06.970+0000 7f712fa11740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560316f00c40 0x560316f08970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:31:06.973 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:06.970+0000 7f712fa11740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560316f0b510 0x560316f08eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:31:06.975 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:06.971+0000 7f712fa11740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x560316cc57b0 con 0x560316f00c40 2024-09-11T00:31:06.975 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:06.971+0000 7f712fa11740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x560316e08650 con 0x560316f0b510 2024-09-11T00:31:06.975 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:06.971+0000 7f712fa11740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x560316f0e330 con 0x560316eff6e0 2024-09-11T00:31:06.975 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:06.971+0000 7f711fe00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560316eff6e0 0x560316f08430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:31:06.975 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:06.971+0000 7f711fe00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560316eff6e0 0x560316f08430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.12:44470/0 (socket says 172.21.15.12:44470) 2024-09-11T00:31:06.975 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:06.971+0000 7f711fe00640 1 -- 172.21.15.12:0/3211966072 learned_addr learned my addr 172.21.15.12:0/3211966072 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:31:06.975 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:06.971+0000 7f712fa11740 10 client.0 _ll_register_callbacks cb 0x560316b6f0a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:31:06.975 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:06.971+0000 7f711fe00640 1 -- 172.21.15.12:0/3211966072 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560316f0b510 msgr2=0x560316f08eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:31:06.975 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:06.971+0000 7f711fe00640 1 --2- 172.21.15.12:0/3211966072 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560316f0b510 0x560316f08eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:06.975 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:06.971+0000 7f711fe00640 1 -- 172.21.15.12:0/3211966072 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560316f00c40 msgr2=0x560316f08970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:31:06.975 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:06.971+0000 7f711fe00640 1 --2- 172.21.15.12:0/3211966072 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560316f00c40 0x560316f08970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:06.976 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:06.971+0000 7f711fe00640 1 -- 172.21.15.12:0/3211966072 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560316bd1ba0 con 0x560316eff6e0 2024-09-11T00:31:06.976 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:06.972+0000 7f711fe00640 1 --2- 172.21.15.12:0/3211966072 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560316eff6e0 0x560316f08430 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f7110009dd0 tx=0x7f7110001990 comp rx=0 tx=0).ready entity=mon.1 client_cookie=17b4b1c9d1361c5c server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:31:06.976 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:06.972+0000 7f710be00640 10 client.0 ms_handle_connect on v2:172.21.15.178:3300/0 2024-09-11T00:31:06.976 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:06.973+0000 7f710be00640 1 -- 172.21.15.12:0/3211966072 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7110004840 con 0x560316eff6e0 2024-09-11T00:31:06.976 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:06.973+0000 7f710be00640 1 -- 172.21.15.12:0/3211966072 <== mon.1 v2:172.21.15.178:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 9208+0+0 (secure 0 0 0) 0x7f71100049e0 con 0x560316eff6e0 2024-09-11T00:31:06.977 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:06.974+0000 7f712fa11740 1 -- 172.21.15.12:0/3211966072 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7110008e30 con 0x560316eff6e0 2024-09-11T00:31:06.978 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:06.976+0000 7f710be00640 1 -- 172.21.15.12:0/3211966072 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=103) ==== 24+0+0 (secure 0 0 0) 0x7f7110004be0 con 0x560316eff6e0 2024-09-11T00:31:06.978 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:06.976+0000 7f712fa11740 10 client.28039 fetch_fsmap learned FSMap version 103 2024-09-11T00:31:06.979 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:06.976+0000 7f712fa11740 1 -- 172.21.15.12:0/3211966072 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,fsmap=103,monmap=4+}) -- 0x560316f19e20 con 0x560316eff6e0 2024-09-11T00:31:06.980 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:06.977+0000 7f710be00640 1 -- 172.21.15.12:0/3211966072 <== mon.1 v2:172.21.15.178:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7110008e30 con 0x560316eff6e0 2024-09-11T00:31:06.980 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:06.978+0000 7f710be00640 1 -- 172.21.15.12:0/3211966072 <== mon.1 v2:172.21.15.178:3300/0 5 ==== fsmap(e 103) ==== 8374+0+0 (secure 0 0 0) 0x7f7110021020 con 0x560316eff6e0 2024-09-11T00:31:06.982 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:06.980+0000 7f712fa11740 10 client.28039 fetch_fsmap finished waiting for FSMap version 103 2024-09-11T00:31:06.983 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:06.980+0000 7f712fa11740 10 client.28039 resolve_mds: validated gid 24674 aka daemon mds.b 2024-09-11T00:31:06.983 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:06.980+0000 7f712fa11740 20 client.28039 populate_metadata read hostname 'smithi012' 2024-09-11T00:31:06.983 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:06.980+0000 7f712fa11740 1 --2- 172.21.15.12:0/3211966072 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x560316f1b170 0x560316f1d560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:31:06.983 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:06.981+0000 7f712fa11740 4 client.28039 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T00:31:06.984 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:06.981+0000 7f712fa11740 1 -- 172.21.15.12:0/3211966072 --> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560316e769e0 con 0x560316f1b170 2024-09-11T00:31:06.984 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:06.981+0000 7f7125e00640 1 --2- 172.21.15.12:0/3211966072 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x560316f1b170 0x560316f1d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:31:06.986 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:06.983+0000 7f7125e00640 1 --2- 172.21.15.12:0/3211966072 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x560316f1b170 0x560316f1d560 crc :-1 s=READY pgs=327 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=759680298118095b server_cookie=d30bb0ad7ccd439f in_seq=0 out_seq=0 2024-09-11T00:31:06.986 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:06.983+0000 7f710be00640 10 client.28039 ms_handle_connect on v2:172.21.15.165:6838/501312984 2024-09-11T00:31:06.991 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:06.988+0000 7f710be00640 1 -- 172.21.15.12:0/3211966072 <== mds.0 v2:172.21.15.165:6838/501312984 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x560316e769e0 con 0x560316f1b170 2024-09-11T00:31:06.991 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:06.988+0000 7f710be00640 10 client.28039 handle_command_reply: tid=0 2024-09-11T00:31:06.991 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:06.988+0000 7f712fa11740 1 client.28039 shutdown 2024-09-11T00:31:06.994 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:06.989+0000 7f712fa11740 1 -- 172.21.15.12:0/3211966072 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560316eff6e0 msgr2=0x560316f08430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:31:06.994 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:06.989+0000 7f712fa11740 1 --2- 172.21.15.12:0/3211966072 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560316eff6e0 0x560316f08430 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f7110009dd0 tx=0x7f7110001990 comp rx=0 tx=0).stop 2024-09-11T00:31:06.994 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:06.990+0000 7f712fa11740 1 -- 172.21.15.12:0/3211966072 shutdown_connections 2024-09-11T00:31:06.994 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:06.990+0000 7f712fa11740 1 --2- 172.21.15.12:0/3211966072 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x560316f1b170 0x560316f1d560 crc :-1 s=READY pgs=327 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:06.994 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:06.990+0000 7f712fa11740 1 --2- 172.21.15.12:0/3211966072 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560316f0b510 0x560316f08eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:06.994 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:06.990+0000 7f712fa11740 1 --2- 172.21.15.12:0/3211966072 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560316f00c40 0x560316f08970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:06.995 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:06.990+0000 7f712fa11740 1 --2- 172.21.15.12:0/3211966072 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560316eff6e0 0x560316f08430 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:06.995 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:06.990+0000 7f712fa11740 1 -- 172.21.15.12:0/3211966072 >> 172.21.15.12:0/3211966072 conn(0x560316ec96f0 msgr2=0x560316f02b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:31:06.995 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:06.990+0000 7f710be00640 0 client.28039 ms_handle_reset on v2:172.21.15.165:6838/501312984 2024-09-11T00:31:06.995 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:06.991+0000 7f712fa11740 1 -- 172.21.15.12:0/3211966072 shutdown_connections 2024-09-11T00:31:06.995 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:06.991+0000 7f712fa11740 1 -- 172.21.15.12:0/3211966072 wait complete. 2024-09-11T00:31:06.996 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:06.991+0000 7f712fa11740 20 client.28039 trim_cache size 0 max 16384 2024-09-11T00:31:07.027 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:07.025+0000 7f712fa11740 1 Processor -- start 2024-09-11T00:31:07.028 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:07.025+0000 7f712fa11740 1 -- start start 2024-09-11T00:31:07.029 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:07.025+0000 7f712fa11740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560316eff6e0 0x560316f0a170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:31:07.029 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:07.025+0000 7f712fa11740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560316f00c40 0x560316efc8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:31:07.029 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:07.026+0000 7f712fa11740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560316f0b510 0x560316efcde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:31:07.029 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:07.026+0000 7f712fa11740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f710400fdb0 con 0x560316eff6e0 2024-09-11T00:31:07.029 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:07.026+0000 7f712fa11740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f7104011920 con 0x560316f0b510 2024-09-11T00:31:07.029 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:07.026+0000 7f712fa11740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f71040104b0 con 0x560316f00c40 2024-09-11T00:31:07.029 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:07.026+0000 7f711fe00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560316eff6e0 0x560316f0a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:31:07.030 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:07.026+0000 7f711fe00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560316eff6e0 0x560316f0a170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:48898/0 (socket says 172.21.15.12:48898) 2024-09-11T00:31:07.030 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:07.026+0000 7f711fe00640 1 -- 172.21.15.12:0/1338720131 learned_addr learned my addr 172.21.15.12:0/1338720131 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:31:07.030 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:07.026+0000 7f7125e00640 1 --2- 172.21.15.12:0/1338720131 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560316f00c40 0x560316efc8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:31:07.030 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:07.026+0000 7f711fe00640 1 --2- 172.21.15.12:0/1338720131 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560316f0b510 0x560316efcde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:31:07.031 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:07.027+0000 7f7125e00640 1 -- 172.21.15.12:0/1338720131 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560316f0b510 msgr2=0x560316efcde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:31:07.031 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:07.027+0000 7f7125e00640 1 --2- 172.21.15.12:0/1338720131 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560316f0b510 0x560316efcde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:07.031 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:07.027+0000 7f7125e00640 1 -- 172.21.15.12:0/1338720131 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560316eff6e0 msgr2=0x560316f0a170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:31:07.031 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:07.027+0000 7f7125e00640 1 --2- 172.21.15.12:0/1338720131 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560316eff6e0 0x560316f0a170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:07.031 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:07.027+0000 7f7125e00640 1 -- 172.21.15.12:0/1338720131 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560316f06930 con 0x560316f00c40 2024-09-11T00:31:07.031 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:07.028+0000 7f7125e00640 1 --2- 172.21.15.12:0/1338720131 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560316f00c40 0x560316efc8a0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x560316f1dc60 tx=0x7f7118024840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=35043e33ff0f50b6 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:31:07.031 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:07.028+0000 7f7116a00640 1 -- 172.21.15.12:0/1338720131 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f711801ae10 con 0x560316f00c40 2024-09-11T00:31:07.032 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:07.028+0000 7f7116a00640 1 -- 172.21.15.12:0/1338720131 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7118024110 con 0x560316f00c40 2024-09-11T00:31:07.032 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:07.030+0000 7f712fa11740 1 -- 172.21.15.12:0/1338720131 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560316f00c40 msgr2=0x560316efc8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:31:07.032 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:07.030+0000 7f712fa11740 1 --2- 172.21.15.12:0/1338720131 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560316f00c40 0x560316efc8a0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x560316f1dc60 tx=0x7f7118024840 comp rx=0 tx=0).stop 2024-09-11T00:31:07.033 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:07.030+0000 7f712fa11740 1 -- 172.21.15.12:0/1338720131 shutdown_connections 2024-09-11T00:31:07.033 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:07.031+0000 7f712fa11740 1 --2- 172.21.15.12:0/1338720131 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560316f0b510 0x560316efcde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:07.033 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:07.031+0000 7f712fa11740 1 --2- 172.21.15.12:0/1338720131 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560316f00c40 0x560316efc8a0 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:07.034 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:07.031+0000 7f712fa11740 1 --2- 172.21.15.12:0/1338720131 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560316eff6e0 0x560316f0a170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:07.034 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:07.031+0000 7f712fa11740 1 -- 172.21.15.12:0/1338720131 >> 172.21.15.12:0/1338720131 conn(0x560316ec96f0 msgr2=0x560316f09a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:31:07.034 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:07.031+0000 7f712fa11740 1 -- 172.21.15.12:0/1338720131 shutdown_connections 2024-09-11T00:31:07.034 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:07.032+0000 7f712fa11740 1 -- 172.21.15.12:0/1338720131 wait complete. 2024-09-11T00:31:07.035 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:07.032+0000 7f712fa11740 1 -- start start 2024-09-11T00:31:07.035 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:07.033+0000 7f712fa11740 1 Processor -- start 2024-09-11T00:31:07.036 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:07.034+0000 7f712fa11740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560316eff6e0 0x560316f13660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:31:07.038 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:07.035+0000 7f712fa11740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560316f00c40 0x560316f13ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:31:07.038 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:07.035+0000 7f711fe00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560316eff6e0 0x560316f13660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:31:07.038 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:07.035+0000 7f712fa11740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560316f0b510 0x560316f0e610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:31:07.038 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:07.035+0000 7f712fa11740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f7104011250 con 0x560316eff6e0 2024-09-11T00:31:07.038 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:07.035+0000 7f712fa11740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f7104010b80 con 0x560316f0b510 2024-09-11T00:31:07.038 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:07.035+0000 7f712fa11740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f710400dd90 con 0x560316f00c40 2024-09-11T00:31:07.038 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:07.035+0000 7f712fa11740 10 client.0 _ll_register_callbacks cb 0x7f7110001b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:31:07.039 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:07.036+0000 7f7125e00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560316f00c40 0x560316f13ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:31:07.039 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:07.036+0000 7f7125e00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560316f00c40 0x560316f13ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.12:44490/0 (socket says 172.21.15.12:44490) 2024-09-11T00:31:07.039 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:07.036+0000 7f711fe00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560316f0b510 0x560316f0e610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:31:07.039 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:07.036+0000 7f711fe00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560316f0b510 0x560316f0e610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.12:35392/0 (socket says 172.21.15.12:35392) 2024-09-11T00:31:07.039 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:07.036+0000 7f711fe00640 1 -- 172.21.15.12:0/290351560 learned_addr learned my addr 172.21.15.12:0/290351560 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:31:07.040 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:07.037+0000 7f711fe00640 1 -- 172.21.15.12:0/290351560 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560316f00c40 msgr2=0x560316f13ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:31:07.040 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:07.037+0000 7f711fe00640 1 --2- 172.21.15.12:0/290351560 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560316f00c40 0x560316f13ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:07.041 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:07.037+0000 7f711fe00640 1 -- 172.21.15.12:0/290351560 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560316eff6e0 msgr2=0x560316f13660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:31:07.041 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:07.037+0000 7f711fe00640 1 --2- 172.21.15.12:0/290351560 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560316eff6e0 0x560316f13660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:07.041 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:07.037+0000 7f711fe00640 1 -- 172.21.15.12:0/290351560 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560316f0eb50 con 0x560316f0b510 2024-09-11T00:31:07.044 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:07.041+0000 7f711fe00640 1 --2- 172.21.15.12:0/290351560 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560316f0b510 0x560316f0e610 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f711000c040 tx=0x7f7110001990 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d810ac1afce6feec server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:31:07.044 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:07.041+0000 7f710be00640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-09-11T00:31:07.045 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:07.043+0000 7f710be00640 1 -- 172.21.15.12:0/290351560 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71100047d0 con 0x560316f0b510 2024-09-11T00:31:07.045 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:07.043+0000 7f710be00640 1 -- 172.21.15.12:0/290351560 <== mon.2 v2:172.21.15.165:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 9208+0+0 (secure 0 0 0) 0x7f7110004970 con 0x560316f0b510 2024-09-11T00:31:07.046 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:07.043+0000 7f712fa11740 1 -- 172.21.15.12:0/290351560 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7118024110 con 0x560316f0b510 2024-09-11T00:31:07.057 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:07.054+0000 7f710be00640 1 -- 172.21.15.12:0/290351560 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=103) ==== 24+0+0 (secure 0 0 0) 0x7f711003b050 con 0x560316f0b510 2024-09-11T00:31:07.057 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:07.055+0000 7f712fa11740 10 client.28067 fetch_fsmap learned FSMap version 103 2024-09-11T00:31:07.057 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:07.055+0000 7f712fa11740 1 -- 172.21.15.12:0/290351560 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=103,monmap=4+}) -- 0x560316f88660 con 0x560316f0b510 2024-09-11T00:31:07.058 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:07.056+0000 7f710be00640 1 -- 172.21.15.12:0/290351560 <== mon.2 v2:172.21.15.165:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7118024110 con 0x560316f0b510 2024-09-11T00:31:07.059 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:07.057+0000 7f710be00640 1 -- 172.21.15.12:0/290351560 <== mon.2 v2:172.21.15.165:3300/0 5 ==== fsmap(e 103) ==== 8374+0+0 (secure 0 0 0) 0x7f7110021040 con 0x560316f0b510 2024-09-11T00:31:07.060 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:07.057+0000 7f712fa11740 10 client.28067 fetch_fsmap finished waiting for FSMap version 103 2024-09-11T00:31:07.060 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:07.057+0000 7f712fa11740 10 client.28067 resolve_mds: validated gid 24674 aka daemon mds.b 2024-09-11T00:31:07.060 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:07.057+0000 7f712fa11740 20 client.28067 populate_metadata read hostname 'smithi012' 2024-09-11T00:31:07.060 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:07.058+0000 7f712fa11740 1 --2- 172.21.15.12:0/290351560 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x560316f88e00 0x560316f8b220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:31:07.060 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:07.058+0000 7f712fa11740 4 client.28067 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 138.85, "await_for": 45.0}] 2024-09-11T00:31:07.060 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:07.058+0000 7f712fa11740 1 -- 172.21.15.12:0/290351560 --> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 138.85, "await_for": 45.0}) -- 0x560316e72490 con 0x560316f88e00 2024-09-11T00:31:07.062 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:07.059+0000 7f7125e00640 1 --2- 172.21.15.12:0/290351560 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x560316f88e00 0x560316f8b220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:31:07.065 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:07.062+0000 7f7125e00640 1 --2- 172.21.15.12:0/290351560 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x560316f88e00 0x560316f8b220 crc :-1 s=READY pgs=328 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4d9d62f479230e63 server_cookie=5eb1d186dda00dc1 in_seq=0 out_seq=0 2024-09-11T00:31:07.065 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:07.062+0000 7f710be00640 10 client.28067 ms_handle_connect on v2:172.21.15.165:6838/501312984 2024-09-11T00:31:07.294 INFO:tasks.workunit.client.0.smithi012.stdout: 10 188614 15.42 MB/sec execute 543 sec latency 361.411 ms 2024-09-11T00:31:07.722 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:07.719+0000 7f710be00640 1 -- 172.21.15.12:0/290351560 <== mds.0 v2:172.21.15.165:6838/501312984 1 ==== command_reply(tid 0: 0 ) ==== 8+0+592 (crc 0 0 0) 0x560316e72490 con 0x560316f88e00 2024-09-11T00:31:07.722 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:07.719+0000 7f710be00640 10 client.28067 handle_command_reply: tid=0 2024-09-11T00:31:07.723 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:07.719+0000 7f712fa11740 1 client.28067 shutdown 2024-09-11T00:31:07.723 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:07.719+0000 7f712fa11740 1 -- 172.21.15.12:0/290351560 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560316f0b510 msgr2=0x560316f0e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:31:07.724 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:07.719+0000 7f712fa11740 1 --2- 172.21.15.12:0/290351560 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560316f0b510 0x560316f0e610 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f711000c040 tx=0x7f7110001990 comp rx=0 tx=0).stop 2024-09-11T00:31:07.724 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:07.720+0000 7f712fa11740 1 -- 172.21.15.12:0/290351560 shutdown_connections 2024-09-11T00:31:07.724 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:07.720+0000 7f712fa11740 1 --2- 172.21.15.12:0/290351560 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x560316f88e00 0x560316f8b220 crc :-1 s=READY pgs=328 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:07.724 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:07.720+0000 7f712fa11740 1 --2- 172.21.15.12:0/290351560 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560316f0b510 0x560316f0e610 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:07.724 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:07.720+0000 7f712fa11740 1 --2- 172.21.15.12:0/290351560 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560316f00c40 0x560316f13ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:07.725 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:07.720+0000 7f712fa11740 1 --2- 172.21.15.12:0/290351560 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560316eff6e0 0x560316f13660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:07.725 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:07.720+0000 7f710be00640 0 client.28067 ms_handle_reset on v2:172.21.15.165:6838/501312984 2024-09-11T00:31:07.725 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:07.720+0000 7f712fa11740 1 -- 172.21.15.12:0/290351560 >> 172.21.15.12:0/290351560 conn(0x560316ec96f0 msgr2=0x560316f02b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:31:07.725 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:07.720+0000 7f712fa11740 1 -- 172.21.15.12:0/290351560 shutdown_connections 2024-09-11T00:31:07.725 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:07.720+0000 7f712fa11740 1 -- 172.21.15.12:0/290351560 wait complete. 2024-09-11T00:31:07.726 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:07.721+0000 7f712fa11740 20 client.28067 trim_cache size 0 max 16384 2024-09-11T00:31:07.726 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-09-11T00:31:07.726 INFO:teuthology.orchestra.run.smithi012.stdout: "epoch": 103, 2024-09-11T00:31:07.727 INFO:teuthology.orchestra.run.smithi012.stdout: "leader": 24674, 2024-09-11T00:31:07.727 INFO:teuthology.orchestra.run.smithi012.stdout: "set_version": 74, 2024-09-11T00:31:07.727 INFO:teuthology.orchestra.run.smithi012.stdout: "sets": { 2024-09-11T00:31:07.728 INFO:teuthology.orchestra.run.smithi012.stdout: "7c006079": { 2024-09-11T00:31:07.728 INFO:teuthology.orchestra.run.smithi012.stdout: "version": 74, 2024-09-11T00:31:07.728 INFO:teuthology.orchestra.run.smithi012.stdout: "age_ref": 0.0, 2024-09-11T00:31:07.728 INFO:teuthology.orchestra.run.smithi012.stdout: "state": { 2024-09-11T00:31:07.728 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "QUIESCED", 2024-09-11T00:31:07.728 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 0.0 2024-09-11T00:31:07.729 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:31:07.729 INFO:teuthology.orchestra.run.smithi012.stdout: "timeout": 90.0, 2024-09-11T00:31:07.729 INFO:teuthology.orchestra.run.smithi012.stdout: "expiration": 138.8, 2024-09-11T00:31:07.729 INFO:teuthology.orchestra.run.smithi012.stdout: "members": { 2024-09-11T00:31:07.729 INFO:teuthology.orchestra.run.smithi012.stdout: "file:/": { 2024-09-11T00:31:07.730 INFO:teuthology.orchestra.run.smithi012.stdout: "excluded": false, 2024-09-11T00:31:07.730 INFO:teuthology.orchestra.run.smithi012.stdout: "state": { 2024-09-11T00:31:07.730 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "QUIESCED", 2024-09-11T00:31:07.730 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 0.0 2024-09-11T00:31:07.730 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:31:07.730 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:31:07.731 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:31:07.731 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:31:07.731 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:31:07.731 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-09-11T00:31:07.732 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:07.722+0000 7f712ce00640 1 -- 172.21.15.12:0/1218687957 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f710c0b54b0 msgr2=0x7f710c0b7970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:31:07.732 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:07.722+0000 7f712ce00640 1 --2- 172.21.15.12:0/1218687957 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f710c0b54b0 0x7f710c0b7970 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f7120009540 tx=0x7f71201099e0 comp rx=0 tx=0).stop 2024-09-11T00:31:07.732 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:07.722+0000 7f712ce00640 1 -- 172.21.15.12:0/1218687957 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f712811c0a0 msgr2=0x7f71281a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:31:07.733 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:07.722+0000 7f712ce00640 1 --2- 172.21.15.12:0/1218687957 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f712811c0a0 0x7f71281a5040 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f712002ad40 tx=0x7f7120003e60 comp rx=0 tx=0).stop 2024-09-11T00:31:07.733 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:07.723+0000 7f712ce00640 1 -- 172.21.15.12:0/1218687957 shutdown_connections 2024-09-11T00:31:07.733 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:07.723+0000 7f712ce00640 1 --2- 172.21.15.12:0/1218687957 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f710c0b54b0 0x7f710c0b7970 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:07.733 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:07.723+0000 7f712ce00640 1 --2- 172.21.15.12:0/1218687957 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f712811c0a0 0x7f71281a5040 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:07.733 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:07.723+0000 7f712ce00640 1 --2- 172.21.15.12:0/1218687957 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f712810a530 0x7f7128117c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:07.733 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:07.723+0000 7f712ce00640 1 --2- 172.21.15.12:0/1218687957 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f712810a150 0x7f71281176e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:07.734 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:07.723+0000 7f712ce00640 1 -- 172.21.15.12:0/1218687957 >> 172.21.15.12:0/1218687957 conn(0x7f712806dfc0 msgr2=0x7f71280716d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:31:07.734 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:07.723+0000 7f712ce00640 1 -- 172.21.15.12:0/1218687957 shutdown_connections 2024-09-11T00:31:07.734 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:07.723+0000 7f712ce00640 1 -- 172.21.15.12:0/1218687957 wait complete. 2024-09-11T00:31:08.294 INFO:tasks.workunit.client.0.smithi012.stdout: 10 188625 15.40 MB/sec execute 544 sec latency 1181.855 ms 2024-09-11T00:31:08.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:31:08 smithi012 ceph-mon[30333]: pgmap v1121: 129 pgs: 129 active+clean; 836 MiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 5.6 MiB/s wr, 452 op/s 2024-09-11T00:31:08.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:31:08 smithi178 ceph-mon[35919]: pgmap v1121: 129 pgs: 129 active+clean; 836 MiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 5.6 MiB/s wr, 452 op/s 2024-09-11T00:31:08.533 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:31:08 smithi165 ceph-mon[35356]: pgmap v1121: 129 pgs: 129 active+clean; 836 MiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 5.6 MiB/s wr, 452 op/s 2024-09-11T00:31:09.295 INFO:tasks.workunit.client.0.smithi012.stdout: 10 188625 15.37 MB/sec execute 545 sec latency 2181.916 ms 2024-09-11T00:31:09.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:31:09 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T00:31:09.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:31:09 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T00:31:09.357 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '7c006079', quiesce took 9.8 seconds. Will release after: 9.05 2024-09-11T00:31:09.533 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:31:09 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T00:31:10.221 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:31:10 smithi012 ceph-mon[30333]: pgmap v1122: 129 pgs: 129 active+clean; 835 MiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 5.7 MiB/s wr, 414 op/s 2024-09-11T00:31:10.221 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:31:10 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:31:10.221 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:31:10 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T00:31:10.221 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:31:10 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:31:10.295 INFO:tasks.workunit.client.0.smithi012.stdout: 10 188625 15.34 MB/sec execute 546 sec latency 3181.981 ms 2024-09-11T00:31:10.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:31:10 smithi178 ceph-mon[35919]: pgmap v1122: 129 pgs: 129 active+clean; 835 MiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 5.7 MiB/s wr, 414 op/s 2024-09-11T00:31:10.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:31:10 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:31:10.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:31:10 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T00:31:10.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:31:10 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:31:10.533 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:31:10 smithi165 ceph-mon[35356]: pgmap v1122: 129 pgs: 129 active+clean; 835 MiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 5.7 MiB/s wr, 414 op/s 2024-09-11T00:31:10.533 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:31:10 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:31:10.534 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:31:10 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T00:31:10.534 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:31:10 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:31:11.295 INFO:tasks.workunit.client.0.smithi012.stdout: 10 188625 15.31 MB/sec execute 547 sec latency 4182.107 ms 2024-09-11T00:31:12.297 INFO:tasks.workunit.client.0.smithi012.stdout: 10 188625 15.28 MB/sec execute 548 sec latency 5182.263 ms 2024-09-11T00:31:12.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:31:12 smithi012 ceph-mon[30333]: pgmap v1123: 129 pgs: 129 active+clean; 828 MiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 6.9 MiB/s wr, 467 op/s 2024-09-11T00:31:12.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:31:12 smithi178 ceph-mon[35919]: pgmap v1123: 129 pgs: 129 active+clean; 828 MiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 6.9 MiB/s wr, 467 op/s 2024-09-11T00:31:12.533 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:31:12 smithi165 ceph-mon[35356]: pgmap v1123: 129 pgs: 129 active+clean; 828 MiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 6.9 MiB/s wr, 467 op/s 2024-09-11T00:31:13.295 INFO:tasks.workunit.client.0.smithi012.stdout: 10 188625 15.26 MB/sec execute 549 sec latency 6182.398 ms 2024-09-11T00:31:14.295 INFO:tasks.workunit.client.0.smithi012.stdout: 10 188625 15.23 MB/sec execute 550 sec latency 7182.545 ms 2024-09-11T00:31:14.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:31:14 smithi178 ceph-mon[35919]: pgmap v1124: 129 pgs: 129 active+clean; 826 MiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 7.3 MiB/s wr, 516 op/s 2024-09-11T00:31:14.532 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:31:14 smithi165 ceph-mon[35356]: pgmap v1124: 129 pgs: 129 active+clean; 826 MiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 7.3 MiB/s wr, 516 op/s 2024-09-11T00:31:14.563 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:31:14 smithi012 ceph-mon[30333]: pgmap v1124: 129 pgs: 129 active+clean; 826 MiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 7.3 MiB/s wr, 516 op/s 2024-09-11T00:31:15.295 INFO:tasks.workunit.client.0.smithi012.stdout: 10 188625 15.20 MB/sec execute 551 sec latency 8182.598 ms 2024-09-11T00:31:16.296 INFO:tasks.workunit.client.0.smithi012.stdout: 10 188625 15.17 MB/sec execute 552 sec latency 9182.738 ms 2024-09-11T00:31:16.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:31:16 smithi178 ceph-mon[35919]: pgmap v1125: 129 pgs: 129 active+clean; 825 MiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 6.2 MiB/s wr, 409 op/s 2024-09-11T00:31:16.533 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:31:16 smithi165 ceph-mon[35356]: pgmap v1125: 129 pgs: 129 active+clean; 825 MiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 6.2 MiB/s wr, 409 op/s 2024-09-11T00:31:16.563 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:31:16 smithi012 ceph-mon[30333]: pgmap v1125: 129 pgs: 129 active+clean; 825 MiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 6.2 MiB/s wr, 409 op/s 2024-09-11T00:31:16.866 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:16.864+0000 7f9948800640 1 -- 172.21.15.12:0/2815740330 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9940103ac0 msgr2=0x7f9940103f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:31:16.866 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:16.864+0000 7f9948800640 1 --2- 172.21.15.12:0/2815740330 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9940103ac0 0x7f9940103f40 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f9930009890 tx=0x7f993002a360 comp rx=0 tx=0).stop 2024-09-11T00:31:16.867 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:16.865+0000 7f9948800640 1 -- 172.21.15.12:0/2815740330 shutdown_connections 2024-09-11T00:31:16.867 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:16.865+0000 7f9948800640 1 --2- 172.21.15.12:0/2815740330 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9940104480 0x7f994010ae00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:16.867 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:16.865+0000 7f9948800640 1 --2- 172.21.15.12:0/2815740330 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9940103ac0 0x7f9940103f40 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:16.867 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:16.865+0000 7f9948800640 1 --2- 172.21.15.12:0/2815740330 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9940102970 0x7f9940102d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:16.867 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:16.865+0000 7f9948800640 1 -- 172.21.15.12:0/2815740330 >> 172.21.15.12:0/2815740330 conn(0x7f99400fe250 msgr2=0x7f9940100670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:31:16.867 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:16.865+0000 7f9948800640 1 -- 172.21.15.12:0/2815740330 shutdown_connections 2024-09-11T00:31:16.867 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:16.865+0000 7f9948800640 1 -- 172.21.15.12:0/2815740330 wait complete. 2024-09-11T00:31:16.869 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:16.866+0000 7f9948800640 1 Processor -- start 2024-09-11T00:31:16.869 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:16.866+0000 7f9948800640 1 -- start start 2024-09-11T00:31:16.869 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:16.866+0000 7f9948800640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9940102970 0x7f994019b010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:31:16.870 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:16.866+0000 7f9948800640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9940103ac0 0x7f994019b660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:31:16.870 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:16.866+0000 7f9948800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9940104480 0x7f994019c290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:31:16.870 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:16.866+0000 7f9948800640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f994010e5c0 con 0x7f9940102970 2024-09-11T00:31:16.870 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:16.866+0000 7f9948800640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f994010e440 con 0x7f9940104480 2024-09-11T00:31:16.870 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:16.866+0000 7f9948800640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f994010e740 con 0x7f9940103ac0 2024-09-11T00:31:16.870 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:16.866+0000 7f9945e00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9940102970 0x7f994019b010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:31:16.870 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:16.866+0000 7f9945e00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9940102970 0x7f994019b010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:47990/0 (socket says 172.21.15.12:47990) 2024-09-11T00:31:16.870 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:16.866+0000 7f9945e00640 1 -- 172.21.15.12:0/3770956686 learned_addr learned my addr 172.21.15.12:0/3770956686 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:31:16.870 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:16.866+0000 7f9946800640 1 --2- 172.21.15.12:0/3770956686 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9940104480 0x7f994019c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:31:16.870 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:16.866+0000 7f9945400640 1 --2- 172.21.15.12:0/3770956686 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9940103ac0 0x7f994019b660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:31:16.870 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:16.867+0000 7f9945e00640 1 -- 172.21.15.12:0/3770956686 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9940103ac0 msgr2=0x7f994019b660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:31:16.870 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:16.867+0000 7f9945e00640 1 --2- 172.21.15.12:0/3770956686 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9940103ac0 0x7f994019b660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:16.870 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:16.867+0000 7f9945e00640 1 -- 172.21.15.12:0/3770956686 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9940104480 msgr2=0x7f994019c290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:31:16.871 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:16.867+0000 7f9945e00640 1 --2- 172.21.15.12:0/3770956686 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9940104480 0x7f994019c290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:16.871 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:16.867+0000 7f9945e00640 1 -- 172.21.15.12:0/3770956686 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f994019cb50 con 0x7f9940102970 2024-09-11T00:31:16.871 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:16.867+0000 7f9946800640 1 --2- 172.21.15.12:0/3770956686 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9940104480 0x7f994019c290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:31:16.871 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:16.867+0000 7f9945e00640 1 --2- 172.21.15.12:0/3770956686 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9940102970 0x7f994019b010 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f993400e9c0 tx=0x7f993400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:31:16.871 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:16.867+0000 7f993aa00640 1 -- 172.21.15.12:0/3770956686 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f993400cde0 con 0x7f9940102970 2024-09-11T00:31:16.871 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:16.867+0000 7f9948800640 1 -- 172.21.15.12:0/3770956686 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f994019ce40 con 0x7f9940102970 2024-09-11T00:31:16.871 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:16.867+0000 7f993aa00640 1 -- 172.21.15.12:0/3770956686 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f99340035f0 con 0x7f9940102970 2024-09-11T00:31:16.871 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:16.867+0000 7f993aa00640 1 -- 172.21.15.12:0/3770956686 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9934010690 con 0x7f9940102970 2024-09-11T00:31:16.871 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:16.868+0000 7f9948800640 1 -- 172.21.15.12:0/3770956686 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99401a3510 con 0x7f9940102970 2024-09-11T00:31:16.873 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:16.871+0000 7f994b615740 1 Processor -- start 2024-09-11T00:31:16.873 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:16.871+0000 7f994b615740 1 -- start start 2024-09-11T00:31:16.873 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:16.871+0000 7f993aa00640 1 -- 172.21.15.12:0/3770956686 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7f9934004000 con 0x7f9940102970 2024-09-11T00:31:16.874 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:16.871+0000 7f994b615740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b65123e790 0x55b651240b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:31:16.874 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:16.871+0000 7f994b615740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b6512410c0 0x55b6512434b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:31:16.874 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:16.871+0000 7f994b615740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b651243b00 0x55b651245fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:31:16.874 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:16.871+0000 7f994b615740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55b6511b16b0 con 0x55b6512410c0 2024-09-11T00:31:16.874 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:16.871+0000 7f994b615740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55b6511a0ae0 con 0x55b65123e790 2024-09-11T00:31:16.874 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:16.871+0000 7f994b615740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55b6511a48f0 con 0x55b651243b00 2024-09-11T00:31:16.874 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:16.872+0000 7f9945e00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b6512410c0 0x55b6512434b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:31:16.874 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:16.872+0000 7f9945400640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b651243b00 0x55b651245fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:31:16.874 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:16.872+0000 7f9945e00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b6512410c0 0x55b6512434b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:48002/0 (socket says 172.21.15.12:48002) 2024-09-11T00:31:16.874 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:16.872+0000 7f9945e00640 1 -- 172.21.15.12:0/1420245943 learned_addr learned my addr 172.21.15.12:0/1420245943 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:31:16.874 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:16.872+0000 7f9945400640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b651243b00 0x55b651245fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.12:33552/0 (socket says 172.21.15.12:33552) 2024-09-11T00:31:16.874 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:16.872+0000 7f9946800640 1 --2- 172.21.15.12:0/1420245943 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b65123e790 0x55b651240b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:31:16.875 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:16.872+0000 7f9945400640 1 -- 172.21.15.12:0/1420245943 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b65123e790 msgr2=0x55b651240b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:31:16.875 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:16.872+0000 7f9945400640 1 --2- 172.21.15.12:0/1420245943 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b65123e790 0x55b651240b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:16.875 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:16.872+0000 7f9945400640 1 -- 172.21.15.12:0/1420245943 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b6512410c0 msgr2=0x55b6512434b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:31:16.875 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:16.872+0000 7f9945400640 1 --2- 172.21.15.12:0/1420245943 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b6512410c0 0x55b6512434b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:16.875 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:16.872+0000 7f9945400640 1 -- 172.21.15.12:0/1420245943 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b65116d780 con 0x55b651243b00 2024-09-11T00:31:16.879 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:16.873+0000 7f9945400640 1 --2- 172.21.15.12:0/1420245943 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b651243b00 0x55b651245fc0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f9930030480 tx=0x7f9930005d30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=14b61aa1ac44ae14 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:31:16.879 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:16.873+0000 7f992f000640 1 -- 172.21.15.12:0/1420245943 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9930041020 con 0x55b651243b00 2024-09-11T00:31:16.879 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:16.873+0000 7f992f000640 1 -- 172.21.15.12:0/1420245943 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9930030520 con 0x55b651243b00 2024-09-11T00:31:16.879 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:16.873+0000 7f992f000640 1 -- 172.21.15.12:0/1420245943 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9930037040 con 0x55b651243b00 2024-09-11T00:31:16.879 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:16.873+0000 7f993aa00640 1 --2- 172.21.15.12:0/3770956686 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f99280b52e0 0x7f99280b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:31:16.879 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:16.873+0000 7f993aa00640 1 -- 172.21.15.12:0/3770956686 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9208+0+0 (secure 0 0 0) 0x7f99340e6310 con 0x7f9940102970 2024-09-11T00:31:16.879 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:16.873+0000 7f9946800640 1 --2- 172.21.15.12:0/3770956686 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f99280b52e0 0x7f99280b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:31:16.879 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:16.873+0000 7f994b615740 1 -- 172.21.15.12:0/1420245943 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b651243b00 msgr2=0x55b651245fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:31:16.879 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:16.873+0000 7f994b615740 1 --2- 172.21.15.12:0/1420245943 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b651243b00 0x55b651245fc0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f9930030480 tx=0x7f9930005d30 comp rx=0 tx=0).stop 2024-09-11T00:31:16.880 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:16.874+0000 7f994b615740 1 -- 172.21.15.12:0/1420245943 shutdown_connections 2024-09-11T00:31:16.880 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:16.874+0000 7f994b615740 1 --2- 172.21.15.12:0/1420245943 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b651243b00 0x55b651245fc0 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:16.880 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:16.874+0000 7f994b615740 1 --2- 172.21.15.12:0/1420245943 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b6512410c0 0x55b6512434b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:16.880 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:16.874+0000 7f994b615740 1 --2- 172.21.15.12:0/1420245943 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b65123e790 0x55b651240b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:16.880 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:16.874+0000 7f994b615740 1 -- 172.21.15.12:0/1420245943 >> 172.21.15.12:0/1420245943 conn(0x55b6512398e0 msgr2=0x55b65123a670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:31:16.880 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:16.874+0000 7f994b615740 1 -- 172.21.15.12:0/1420245943 shutdown_connections 2024-09-11T00:31:16.880 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:16.874+0000 7f994b615740 1 -- 172.21.15.12:0/1420245943 wait complete. 2024-09-11T00:31:16.880 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:16.874+0000 7f9946800640 1 --2- 172.21.15.12:0/3770956686 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f99280b52e0 0x7f99280b77a0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f993c009fd0 tx=0x7f993c009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:31:16.880 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:16.874+0000 7f994b615740 1 -- start start 2024-09-11T00:31:16.880 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:16.875+0000 7f994b615740 1 Processor -- start 2024-09-11T00:31:16.880 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:16.875+0000 7f994b615740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b65123e790 0x55b651247e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:31:16.880 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:16.875+0000 7f994b615740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b6512410c0 0x55b6512484d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:31:16.880 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:16.875+0000 7f994b615740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b651243b00 0x55b6512490a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:31:16.881 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:16.875+0000 7f994b615740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55b6510d8ab0 con 0x55b651243b00 2024-09-11T00:31:16.881 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:16.875+0000 7f994b615740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55b651147570 con 0x55b6512410c0 2024-09-11T00:31:16.881 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:16.875+0000 7f994b615740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55b651246a80 con 0x55b65123e790 2024-09-11T00:31:16.881 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:16.875+0000 7f994b615740 10 client.0 _ll_register_callbacks cb 0x55b651156100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:31:16.881 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:16.875+0000 7f9946800640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b651243b00 0x55b6512490a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:31:16.881 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:16.875+0000 7f9946800640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b651243b00 0x55b6512490a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:48004/0 (socket says 172.21.15.12:48004) 2024-09-11T00:31:16.881 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:16.875+0000 7f9945e00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b65123e790 0x55b651247e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:31:16.881 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:16.875+0000 7f9945400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b6512410c0 0x55b6512484d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:31:16.881 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:16.875+0000 7f9946800640 1 -- 172.21.15.12:0/4079466127 learned_addr learned my addr 172.21.15.12:0/4079466127 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:31:16.881 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:16.876+0000 7f9945400640 1 -- 172.21.15.12:0/4079466127 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b65123e790 msgr2=0x55b651247e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:31:16.881 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:16.876+0000 7f9945400640 1 --2- 172.21.15.12:0/4079466127 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b65123e790 0x55b651247e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:16.881 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:16.876+0000 7f9945400640 1 -- 172.21.15.12:0/4079466127 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b651243b00 msgr2=0x55b6512490a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:31:16.882 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:16.876+0000 7f9945400640 1 --2- 172.21.15.12:0/4079466127 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b651243b00 0x55b6512490a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:16.882 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:16.876+0000 7f9945400640 1 -- 172.21.15.12:0/4079466127 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b650f02790 con 0x55b6512410c0 2024-09-11T00:31:16.882 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:16.876+0000 7f9945400640 1 --2- 172.21.15.12:0/4079466127 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b6512410c0 0x55b6512484d0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f9930000c00 tx=0x7f99300058d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9d47b44e2800b711 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:31:16.882 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:16.876+0000 7f9923400640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-09-11T00:31:16.882 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:16.876+0000 7f9923400640 1 -- 172.21.15.12:0/4079466127 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9930041030 con 0x55b6512410c0 2024-09-11T00:31:16.882 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:16.876+0000 7f994b615740 1 -- 172.21.15.12:0/4079466127 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9930030520 con 0x55b6512410c0 2024-09-11T00:31:16.882 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:16.877+0000 7f9923400640 1 -- 172.21.15.12:0/4079466127 <== mon.2 v2:172.21.15.165:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 9208+0+0 (secure 0 0 0) 0x7f9930030b20 con 0x55b6512410c0 2024-09-11T00:31:16.882 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:16.877+0000 7f9923400640 1 -- 172.21.15.12:0/4079466127 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=103) ==== 24+0+0 (secure 0 0 0) 0x7f9930057050 con 0x55b6512410c0 2024-09-11T00:31:16.882 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:16.877+0000 7f994b615740 10 client.28079 fetch_fsmap learned FSMap version 103 2024-09-11T00:31:16.882 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:16.877+0000 7f994b615740 1 -- 172.21.15.12:0/4079466127 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=103,monmap=4+}) -- 0x55b650f7b620 con 0x55b6512410c0 2024-09-11T00:31:16.882 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:16.877+0000 7f9923400640 1 -- 172.21.15.12:0/4079466127 <== mon.2 v2:172.21.15.165:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9930030520 con 0x55b6512410c0 2024-09-11T00:31:16.882 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:16.877+0000 7f9923400640 1 -- 172.21.15.12:0/4079466127 <== mon.2 v2:172.21.15.165:3300/0 5 ==== fsmap(e 103) ==== 8374+0+0 (secure 0 0 0) 0x7f9930053500 con 0x55b6512410c0 2024-09-11T00:31:16.882 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:16.877+0000 7f994b615740 10 client.28079 fetch_fsmap finished waiting for FSMap version 103 2024-09-11T00:31:16.883 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:16.877+0000 7f994b615740 10 client.28079 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T00:31:16.883 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:16.877+0000 7f994b615740 20 client.28079 populate_metadata read hostname 'smithi012' 2024-09-11T00:31:16.883 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:16.877+0000 7f994b615740 1 --2- 172.21.15.12:0/4079466127 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55b6512532d0 0x55b6512556c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:31:16.883 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:16.877+0000 7f994b615740 4 client.28079 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T00:31:16.883 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:16.877+0000 7f994b615740 1 -- 172.21.15.12:0/4079466127 --> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b6511b3600 con 0x55b6512532d0 2024-09-11T00:31:16.884 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:16.878+0000 7f9945e00640 1 --2- 172.21.15.12:0/4079466127 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55b6512532d0 0x55b6512556c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:31:16.884 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:16.878+0000 7f9945e00640 1 --2- 172.21.15.12:0/4079466127 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55b6512532d0 0x55b6512556c0 crc :-1 s=READY pgs=329 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=74666cb48f3d8951 server_cookie=9a643c04ebb2525b in_seq=0 out_seq=0 2024-09-11T00:31:16.884 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:16.878+0000 7f9923400640 10 client.28079 ms_handle_connect on v2:172.21.15.165:6838/501312984 2024-09-11T00:31:16.884 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:16.879+0000 7f9923400640 1 -- 172.21.15.12:0/4079466127 <== mds.0 v2:172.21.15.165:6838/501312984 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55b6511b3600 con 0x55b6512532d0 2024-09-11T00:31:16.884 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:16.879+0000 7f9923400640 10 client.28079 handle_command_reply: tid=0 2024-09-11T00:31:16.885 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:16.879+0000 7f994b615740 1 client.28079 shutdown 2024-09-11T00:31:16.885 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:16.879+0000 7f994b615740 1 -- 172.21.15.12:0/4079466127 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b6512410c0 msgr2=0x55b6512484d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:31:16.885 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:16.879+0000 7f994b615740 1 --2- 172.21.15.12:0/4079466127 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b6512410c0 0x55b6512484d0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f9930000c00 tx=0x7f99300058d0 comp rx=0 tx=0).stop 2024-09-11T00:31:16.885 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:16.879+0000 7f994b615740 1 -- 172.21.15.12:0/4079466127 shutdown_connections 2024-09-11T00:31:16.885 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:16.879+0000 7f994b615740 1 --2- 172.21.15.12:0/4079466127 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55b6512532d0 0x55b6512556c0 crc :-1 s=READY pgs=329 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:16.885 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:16.879+0000 7f994b615740 1 --2- 172.21.15.12:0/4079466127 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b651243b00 0x55b6512490a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:16.885 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:16.879+0000 7f994b615740 1 --2- 172.21.15.12:0/4079466127 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b6512410c0 0x55b6512484d0 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:16.885 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:16.879+0000 7f9923400640 0 client.28079 ms_handle_reset on v2:172.21.15.165:6838/501312984 2024-09-11T00:31:16.885 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:16.879+0000 7f994b615740 1 --2- 172.21.15.12:0/4079466127 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b65123e790 0x55b651247e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:16.885 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:16.879+0000 7f994b615740 1 -- 172.21.15.12:0/4079466127 >> 172.21.15.12:0/4079466127 conn(0x55b6512398e0 msgr2=0x55b651238a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:31:16.885 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:16.880+0000 7f994b615740 1 -- 172.21.15.12:0/4079466127 shutdown_connections 2024-09-11T00:31:16.885 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:16.880+0000 7f994b615740 1 -- 172.21.15.12:0/4079466127 wait complete. 2024-09-11T00:31:16.885 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:16.880+0000 7f994b615740 20 client.28079 trim_cache size 0 max 16384 2024-09-11T00:31:16.898 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:16.896+0000 7f994b615740 1 Processor -- start 2024-09-11T00:31:16.898 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:16.896+0000 7f994b615740 1 -- start start 2024-09-11T00:31:16.898 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:16.896+0000 7f994b615740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b65123e790 0x55b65123a260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:31:16.898 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:16.896+0000 7f994b615740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b6512410c0 0x55b65123a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:31:16.898 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:16.896+0000 7f994b615740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b651243b00 0x55b651247e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:31:16.898 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:16.896+0000 7f994b615740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f99401ada40 con 0x55b6512410c0 2024-09-11T00:31:16.898 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:16.896+0000 7f994b615740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f99401af5b0 con 0x55b65123e790 2024-09-11T00:31:16.898 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:16.896+0000 7f994b615740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f99401ae140 con 0x55b651243b00 2024-09-11T00:31:16.898 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:16.896+0000 7f9945400640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b6512410c0 0x55b65123a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:31:16.899 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:16.896+0000 7f9945400640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b6512410c0 0x55b65123a7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:48008/0 (socket says 172.21.15.12:48008) 2024-09-11T00:31:16.899 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:16.896+0000 7f9945400640 1 -- 172.21.15.12:0/1303099335 learned_addr learned my addr 172.21.15.12:0/1303099335 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:31:16.899 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:16.896+0000 7f9945e00640 1 --2- 172.21.15.12:0/1303099335 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b65123e790 0x55b65123a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:31:16.899 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:16.896+0000 7f9946800640 1 --2- 172.21.15.12:0/1303099335 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b651243b00 0x55b651247e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:31:16.899 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:16.897+0000 7f9945400640 1 -- 172.21.15.12:0/1303099335 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b651243b00 msgr2=0x55b651247e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:31:16.899 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:16.897+0000 7f9945400640 1 --2- 172.21.15.12:0/1303099335 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b651243b00 0x55b651247e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:16.899 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:16.897+0000 7f9945400640 1 -- 172.21.15.12:0/1303099335 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b65123e790 msgr2=0x55b65123a260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:31:16.899 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:16.897+0000 7f9945400640 1 --2- 172.21.15.12:0/1303099335 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b65123e790 0x55b65123a260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:16.899 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:16.897+0000 7f9945400640 1 -- 172.21.15.12:0/1303099335 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b651243820 con 0x55b6512410c0 2024-09-11T00:31:16.903 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:16.897+0000 7f9945400640 1 --2- 172.21.15.12:0/1303099335 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b6512410c0 0x55b65123a7a0 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f9930053920 tx=0x7f99300303b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=947bb7c8bad15ef4 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:31:16.903 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:16.897+0000 7f992e600640 1 -- 172.21.15.12:0/1303099335 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9930041070 con 0x55b6512410c0 2024-09-11T00:31:16.903 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:16.897+0000 7f992e600640 1 -- 172.21.15.12:0/1303099335 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9930030b20 con 0x55b6512410c0 2024-09-11T00:31:16.903 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:16.897+0000 7f992e600640 1 -- 172.21.15.12:0/1303099335 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9930037040 con 0x55b6512410c0 2024-09-11T00:31:16.903 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:16.898+0000 7f994b615740 1 -- 172.21.15.12:0/1303099335 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b6512410c0 msgr2=0x55b65123a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:31:16.903 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:16.898+0000 7f994b615740 1 --2- 172.21.15.12:0/1303099335 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b6512410c0 0x55b65123a7a0 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f9930053920 tx=0x7f99300303b0 comp rx=0 tx=0).stop 2024-09-11T00:31:16.903 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:16.898+0000 7f994b615740 1 -- 172.21.15.12:0/1303099335 shutdown_connections 2024-09-11T00:31:16.903 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:16.898+0000 7f994b615740 1 --2- 172.21.15.12:0/1303099335 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b651243b00 0x55b651247e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:16.903 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:16.898+0000 7f994b615740 1 --2- 172.21.15.12:0/1303099335 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b6512410c0 0x55b65123a7a0 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:16.903 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:16.898+0000 7f994b615740 1 --2- 172.21.15.12:0/1303099335 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b65123e790 0x55b65123a260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:16.903 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:16.898+0000 7f994b615740 1 -- 172.21.15.12:0/1303099335 >> 172.21.15.12:0/1303099335 conn(0x55b6512398e0 msgr2=0x55b65123dc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:31:16.903 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:16.898+0000 7f994b615740 1 -- 172.21.15.12:0/1303099335 shutdown_connections 2024-09-11T00:31:16.904 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:16.898+0000 7f994b615740 1 -- 172.21.15.12:0/1303099335 wait complete. 2024-09-11T00:31:16.904 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:16.898+0000 7f994b615740 1 -- start start 2024-09-11T00:31:16.904 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:16.899+0000 7f994b615740 1 Processor -- start 2024-09-11T00:31:16.904 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:16.899+0000 7f994b615740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b65123e790 0x55b65124f7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:31:16.904 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:16.899+0000 7f994b615740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b6512410c0 0x55b65124e4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:31:16.904 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:16.899+0000 7f994b615740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b651243b00 0x55b65124ea30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:31:16.904 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:16.899+0000 7f994b615740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f99401aeee0 con 0x55b651243b00 2024-09-11T00:31:16.904 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:16.899+0000 7f994b615740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f99401ae810 con 0x55b65123e790 2024-09-11T00:31:16.904 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:16.899+0000 7f994b615740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f99401aba20 con 0x55b6512410c0 2024-09-11T00:31:16.904 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:16.899+0000 7f994b615740 10 client.0 _ll_register_callbacks cb 0x7f99340e9c20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:31:16.904 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:16.899+0000 7f9946800640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b651243b00 0x55b65124ea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:31:16.904 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:16.899+0000 7f9945400640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b6512410c0 0x55b65124e4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:31:16.905 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:16.899+0000 7f9945e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b65123e790 0x55b65124f7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:31:16.905 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:16.899+0000 7f9945400640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b6512410c0 0x55b65124e4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.12:33570/0 (socket says 172.21.15.12:33570) 2024-09-11T00:31:16.905 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:16.899+0000 7f9945400640 1 -- 172.21.15.12:0/207837620 learned_addr learned my addr 172.21.15.12:0/207837620 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:31:16.905 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:16.900+0000 7f9945400640 1 -- 172.21.15.12:0/207837620 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b65123e790 msgr2=0x55b65124f7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:31:16.905 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:16.900+0000 7f9945400640 1 --2- 172.21.15.12:0/207837620 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b65123e790 0x55b65124f7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:16.905 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:16.900+0000 7f9945400640 1 -- 172.21.15.12:0/207837620 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b651243b00 msgr2=0x55b65124ea30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:31:16.905 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:16.900+0000 7f9945400640 1 --2- 172.21.15.12:0/207837620 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b651243b00 0x55b65124ea30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:16.905 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:16.900+0000 7f9945400640 1 -- 172.21.15.12:0/207837620 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b65124f120 con 0x55b6512410c0 2024-09-11T00:31:16.905 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:16.900+0000 7f9945400640 1 --2- 172.21.15.12:0/207837620 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b6512410c0 0x55b65124e4f0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f993002a880 tx=0x7f99300058d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e879c08900bb8333 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:31:16.905 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:16.900+0000 7f9923400640 10 client.0 ms_handle_connect on v2:172.21.15.178:3300/0 2024-09-11T00:31:16.905 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:16.900+0000 7f9923400640 1 -- 172.21.15.12:0/207837620 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9930037070 con 0x55b6512410c0 2024-09-11T00:31:16.905 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:16.900+0000 7f994b615740 1 -- 172.21.15.12:0/207837620 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9930030b20 con 0x55b6512410c0 2024-09-11T00:31:16.906 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:16.900+0000 7f9923400640 1 -- 172.21.15.12:0/207837620 <== mon.1 v2:172.21.15.178:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 9208+0+0 (secure 0 0 0) 0x7f9930053650 con 0x55b6512410c0 2024-09-11T00:31:16.906 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:16.900+0000 7f9923400640 1 -- 172.21.15.12:0/207837620 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=103) ==== 24+0+0 (secure 0 0 0) 0x7f9930041040 con 0x55b6512410c0 2024-09-11T00:31:16.906 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:16.900+0000 7f994b615740 10 client.28063 fetch_fsmap learned FSMap version 103 2024-09-11T00:31:16.906 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:16.900+0000 7f994b615740 1 -- 172.21.15.12:0/207837620 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,fsmap=103,monmap=4+}) -- 0x55b6512c0190 con 0x55b6512410c0 2024-09-11T00:31:16.906 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:16.901+0000 7f9923400640 1 -- 172.21.15.12:0/207837620 <== mon.1 v2:172.21.15.178:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9930030b20 con 0x55b6512410c0 2024-09-11T00:31:16.906 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:16.901+0000 7f9923400640 1 -- 172.21.15.12:0/207837620 <== mon.1 v2:172.21.15.178:3300/0 5 ==== fsmap(e 103) ==== 8374+0+0 (secure 0 0 0) 0x7f9930062030 con 0x55b6512410c0 2024-09-11T00:31:16.906 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:16.901+0000 7f994b615740 10 client.28063 fetch_fsmap finished waiting for FSMap version 103 2024-09-11T00:31:16.906 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:16.901+0000 7f994b615740 10 client.28063 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T00:31:16.906 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:16.901+0000 7f994b615740 20 client.28063 populate_metadata read hostname 'smithi012' 2024-09-11T00:31:16.906 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:16.901+0000 7f994b615740 1 --2- 172.21.15.12:0/207837620 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55b6512c0900 0x55b6512c2d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:31:16.906 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:16.901+0000 7f994b615740 4 client.28063 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T00:31:16.907 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:16.901+0000 7f994b615740 1 -- 172.21.15.12:0/207837620 --> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55b6511af0b0 con 0x55b6512c0900 2024-09-11T00:31:16.907 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:16.901+0000 7f9945e00640 1 --2- 172.21.15.12:0/207837620 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55b6512c0900 0x55b6512c2d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:31:16.907 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:16.902+0000 7f9945e00640 1 --2- 172.21.15.12:0/207837620 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55b6512c0900 0x55b6512c2d20 crc :-1 s=READY pgs=330 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=26d62057cc72caed server_cookie=4608d203c205cdf2 in_seq=0 out_seq=0 2024-09-11T00:31:16.907 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:16.902+0000 7f9923400640 10 client.28063 ms_handle_connect on v2:172.21.15.165:6838/501312984 2024-09-11T00:31:16.978 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:16.976+0000 7f9923400640 1 -- 172.21.15.12:0/207837620 <== mds.0 v2:172.21.15.165:6838/501312984 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55b6511af0b0 con 0x55b6512c0900 2024-09-11T00:31:16.979 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:16.976+0000 7f9923400640 10 client.28063 handle_command_reply: tid=0 2024-09-11T00:31:16.979 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:16.976+0000 7f994b615740 1 client.28063 shutdown 2024-09-11T00:31:16.979 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:16.976+0000 7f994b615740 1 -- 172.21.15.12:0/207837620 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b6512410c0 msgr2=0x55b65124e4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:31:16.979 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:16.976+0000 7f994b615740 1 --2- 172.21.15.12:0/207837620 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b6512410c0 0x55b65124e4f0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f993002a880 tx=0x7f99300058d0 comp rx=0 tx=0).stop 2024-09-11T00:31:16.979 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:16.976+0000 7f994b615740 1 -- 172.21.15.12:0/207837620 shutdown_connections 2024-09-11T00:31:16.979 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:16.976+0000 7f994b615740 1 --2- 172.21.15.12:0/207837620 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55b6512c0900 0x55b6512c2d20 crc :-1 s=READY pgs=330 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:16.979 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:16.976+0000 7f994b615740 1 --2- 172.21.15.12:0/207837620 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b651243b00 0x55b65124ea30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:16.979 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:16.976+0000 7f994b615740 1 --2- 172.21.15.12:0/207837620 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b6512410c0 0x55b65124e4f0 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:16.979 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:16.976+0000 7f994b615740 1 --2- 172.21.15.12:0/207837620 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b65123e790 0x55b65124f7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:16.979 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:16.976+0000 7f9923400640 0 client.28063 ms_handle_reset on v2:172.21.15.165:6838/501312984 2024-09-11T00:31:16.979 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:16.976+0000 7f994b615740 1 -- 172.21.15.12:0/207837620 >> 172.21.15.12:0/207837620 conn(0x55b6512398e0 msgr2=0x55b651247e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:31:16.979 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:16.977+0000 7f994b615740 1 -- 172.21.15.12:0/207837620 shutdown_connections 2024-09-11T00:31:16.979 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:16.977+0000 7f994b615740 1 -- 172.21.15.12:0/207837620 wait complete. 2024-09-11T00:31:16.980 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:16.977+0000 7f994b615740 20 client.28063 trim_cache size 0 max 16384 2024-09-11T00:31:16.980 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:{ 2024-09-11T00:31:16.980 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "message": "", 2024-09-11T00:31:16.980 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "return_code": 0 2024-09-11T00:31:16.980 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:} 2024-09-11T00:31:16.981 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:16.977+0000 7f9948800640 1 -- 172.21.15.12:0/3770956686 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f99280b52e0 msgr2=0x7f99280b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:31:16.981 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:16.977+0000 7f9948800640 1 --2- 172.21.15.12:0/3770956686 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f99280b52e0 0x7f99280b77a0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f993c009fd0 tx=0x7f993c009500 comp rx=0 tx=0).stop 2024-09-11T00:31:16.981 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:16.977+0000 7f9948800640 1 -- 172.21.15.12:0/3770956686 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9940102970 msgr2=0x7f994019b010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:31:16.981 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:16.977+0000 7f9948800640 1 --2- 172.21.15.12:0/3770956686 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9940102970 0x7f994019b010 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f993400e9c0 tx=0x7f993400ee80 comp rx=0 tx=0).stop 2024-09-11T00:31:16.981 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:16.978+0000 7f9948800640 1 -- 172.21.15.12:0/3770956686 shutdown_connections 2024-09-11T00:31:16.981 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:16.978+0000 7f9948800640 1 --2- 172.21.15.12:0/3770956686 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f99280b52e0 0x7f99280b77a0 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:16.981 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:16.978+0000 7f9948800640 1 --2- 172.21.15.12:0/3770956686 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9940104480 0x7f994019c290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:16.981 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:16.978+0000 7f9948800640 1 --2- 172.21.15.12:0/3770956686 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9940103ac0 0x7f994019b660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:16.982 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:16.978+0000 7f9948800640 1 --2- 172.21.15.12:0/3770956686 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9940102970 0x7f994019b010 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:16.982 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:16.978+0000 7f9948800640 1 -- 172.21.15.12:0/3770956686 >> 172.21.15.12:0/3770956686 conn(0x7f99400fe250 msgr2=0x7f9940109250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:31:16.982 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:16.978+0000 7f9948800640 1 -- 172.21.15.12:0/3770956686 shutdown_connections 2024-09-11T00:31:16.982 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:16.978+0000 7f9948800640 1 -- 172.21.15.12:0/3770956686 wait complete. 2024-09-11T00:31:17.296 INFO:tasks.workunit.client.0.smithi012.stdout: 10 188625 15.14 MB/sec execute 553 sec latency 10182.887 ms 2024-09-11T00:31:18.296 INFO:tasks.workunit.client.0.smithi012.stdout: 10 188625 15.12 MB/sec execute 554 sec latency 11183.016 ms 2024-09-11T00:31:18.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:31:18 smithi178 ceph-mon[35919]: pgmap v1126: 129 pgs: 129 active+clean; 826 MiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 5.6 MiB/s wr, 392 op/s 2024-09-11T00:31:18.408 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid ff3ad74a-6fcf-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T00:31:18.465 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:31:18 smithi012 ceph-mon[30333]: pgmap v1126: 129 pgs: 129 active+clean; 826 MiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 5.6 MiB/s wr, 392 op/s 2024-09-11T00:31:18.532 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:31:18 smithi165 ceph-mon[35356]: pgmap v1126: 129 pgs: 129 active+clean; 826 MiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 5.6 MiB/s wr, 392 op/s 2024-09-11T00:31:18.824 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:31:19.222 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:31:19 smithi012 ceph-mon[30333]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T00:31:19.296 INFO:tasks.workunit.client.0.smithi012.stdout: 10 188625 15.09 MB/sec execute 555 sec latency 12183.154 ms 2024-09-11T00:31:19.436 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:19.434+0000 7f8994a00640 1 -- 172.21.15.12:0/2794872073 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8990102c00 msgr2=0x7f8990102fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:31:19.436 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:19.434+0000 7f8994a00640 1 --2- 172.21.15.12:0/2794872073 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8990102c00 0x7f8990102fe0 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f8980009900 tx=0x7f898002a410 comp rx=0 tx=0).stop 2024-09-11T00:31:19.436 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:19.434+0000 7f8994a00640 1 -- 172.21.15.12:0/2794872073 shutdown_connections 2024-09-11T00:31:19.436 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:19.434+0000 7f8994a00640 1 --2- 172.21.15.12:0/2794872073 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8990108f90 0x7f899010f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:19.436 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:19.434+0000 7f8994a00640 1 --2- 172.21.15.12:0/2794872073 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8990108610 0x7f8990108a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:19.436 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:19.434+0000 7f8994a00640 1 --2- 172.21.15.12:0/2794872073 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8990102c00 0x7f8990102fe0 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:19.436 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:19.434+0000 7f8994a00640 1 -- 172.21.15.12:0/2794872073 >> 172.21.15.12:0/2794872073 conn(0x7f89900fe6d0 msgr2=0x7f8990100af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:31:19.436 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:19.434+0000 7f8994a00640 1 -- 172.21.15.12:0/2794872073 shutdown_connections 2024-09-11T00:31:19.437 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:19.435+0000 7f8994a00640 1 -- 172.21.15.12:0/2794872073 wait complete. 2024-09-11T00:31:19.437 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:19.436+0000 7f8994a00640 1 Processor -- start 2024-09-11T00:31:19.437 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:19.436+0000 7f8994a00640 1 -- start start 2024-09-11T00:31:19.438 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:19.436+0000 7f8994a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8990102c00 0x7f89900754a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:31:19.438 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:19.437+0000 7f8994a00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8990108610 0x7f89900759e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:31:19.438 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:19.437+0000 7f8994a00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8990108f90 0x7f8990075f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:31:19.438 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:19.437+0000 7f8994a00640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f8990112c50 con 0x7f8990108f90 2024-09-11T00:31:19.438 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:19.437+0000 7f8994a00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f8990112ad0 con 0x7f8990102c00 2024-09-11T00:31:19.438 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:19.437+0000 7f8994a00640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f8990112dd0 con 0x7f8990108610 2024-09-11T00:31:19.438 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:19.437+0000 7f898f200640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8990108f90 0x7f8990075f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:31:19.439 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:19.437+0000 7f898f200640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8990108f90 0x7f8990075f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:58324/0 (socket says 172.21.15.12:58324) 2024-09-11T00:31:19.440 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:19.437+0000 7f898f200640 1 -- 172.21.15.12:0/1887607198 learned_addr learned my addr 172.21.15.12:0/1887607198 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:31:19.440 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:19.437+0000 7f8987e00640 1 --2- 172.21.15.12:0/1887607198 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8990108610 0x7f89900759e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:31:19.440 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:19.437+0000 7f898e800640 1 --2- 172.21.15.12:0/1887607198 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8990102c00 0x7f89900754a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:31:19.440 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:19.437+0000 7f898f200640 1 -- 172.21.15.12:0/1887607198 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8990108610 msgr2=0x7f89900759e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:31:19.440 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:19.437+0000 7f898f200640 1 --2- 172.21.15.12:0/1887607198 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8990108610 0x7f89900759e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:19.440 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:19.437+0000 7f898f200640 1 -- 172.21.15.12:0/1887607198 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8990102c00 msgr2=0x7f89900754a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:31:19.440 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:19.437+0000 7f898f200640 1 --2- 172.21.15.12:0/1887607198 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8990102c00 0x7f89900754a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:19.440 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:19.437+0000 7f898f200640 1 -- 172.21.15.12:0/1887607198 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89901a73e0 con 0x7f8990108f90 2024-09-11T00:31:19.440 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:19.438+0000 7f898f200640 1 --2- 172.21.15.12:0/1887607198 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8990108f90 0x7f8990075f20 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f898800eb10 tx=0x7f898800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:31:19.440 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:19.438+0000 7f8987400640 1 -- 172.21.15.12:0/1887607198 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8988004b20 con 0x7f8990108f90 2024-09-11T00:31:19.440 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:19.438+0000 7f8987400640 1 -- 172.21.15.12:0/1887607198 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8988004cc0 con 0x7f8990108f90 2024-09-11T00:31:19.440 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:19.438+0000 7f8994a00640 1 -- 172.21.15.12:0/1887607198 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f89901a76d0 con 0x7f8990108f90 2024-09-11T00:31:19.441 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:19.438+0000 7f8987400640 1 -- 172.21.15.12:0/1887607198 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8988010670 con 0x7f8990108f90 2024-09-11T00:31:19.441 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:19.438+0000 7f8994a00640 1 -- 172.21.15.12:0/1887607198 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89901a7c60 con 0x7f8990108f90 2024-09-11T00:31:19.441 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:19.439+0000 7f8994a00640 1 -- 172.21.15.12:0/1887607198 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8990102fe0 con 0x7f8990108f90 2024-09-11T00:31:19.443 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:19.442+0000 7f8987400640 1 -- 172.21.15.12:0/1887607198 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7f8988005000 con 0x7f8990108f90 2024-09-11T00:31:19.444 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:19.443+0000 7f8987400640 1 --2- 172.21.15.12:0/1887607198 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f89740b5410 0x7f89740b78d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:31:19.445 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:19.443+0000 7f8987400640 1 -- 172.21.15.12:0/1887607198 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9208+0+0 (secure 0 0 0) 0x7f89880e66c0 con 0x7f8990108f90 2024-09-11T00:31:19.445 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:19.443+0000 7f898e800640 1 --2- 172.21.15.12:0/1887607198 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f89740b5410 0x7f89740b78d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:31:19.448 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:19.444+0000 7f898e800640 1 --2- 172.21.15.12:0/1887607198 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f89740b5410 0x7f89740b78d0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f8980031440 tx=0x7f8980005b50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:31:19.448 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:19.445+0000 7f8987400640 1 -- 172.21.15.12:0/1887607198 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f8988056220 con 0x7f8990108f90 2024-09-11T00:31:19.533 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:31:19 smithi165 ceph-mon[35356]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T00:31:19.581 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:31:19 smithi178 ceph-mon[35919]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T00:31:19.682 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:19.681+0000 7f8994a00640 1 -- 172.21.15.12:0/1887607198 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f89900767a0 con 0x7f8990108f90 2024-09-11T00:31:19.683 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:19.681+0000 7f8987400640 1 -- 172.21.15.12:0/1887607198 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 104 v104) ==== 95+0+11970 (secure 0 0 0) 0x7f89900767a0 con 0x7f8990108f90 2024-09-11T00:31:19.683 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-11T00:31:19.684 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":104,"btime":"2024-09-11T00:31:18:070011+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14577,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6835/447171591","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":447171591},{"type":"v1","addr":"172.21.15.12:6835","nonce":447171591}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14589,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/901486040","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":901486040},{"type":"v1","addr":"172.21.15.12:6837","nonce":901486040}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":104,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T00:04:04.739881+0000","modified":"2024-09-11T00:31:17.256094+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24674,"mds_1":24643,"mds_2":24638,"mds_3":24637,"mds_4":24614},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14631":{"gid":14631,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.12:6841/3069919437","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6840","nonce":3069919437},{"type":"v1","addr":"172.21.15.12:6841","nonce":3069919437}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24562":{"gid":24562,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6833/1141636112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":1141636112},{"type":"v1","addr":"172.21.15.178:6833","nonce":1141636112}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24581":{"gid":24581,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6833/3111980970","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6832","nonce":3111980970},{"type":"v1","addr":"172.21.15.165:6833","nonce":3111980970}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24607":{"gid":24607,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6835/4084868287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":4084868287},{"type":"v1","addr":"172.21.15.178:6835","nonce":4084868287}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24614":{"gid":24614,"name":"e","rank":4,"incarnation":26,"state":"up:active","state_seq":21,"addr":"172.21.15.165:6835/2425395274","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":2425395274},{"type":"v1","addr":"172.21.15.165:6835","nonce":2425395274}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24637":{"gid":24637,"name":"i","rank":3,"incarnation":24,"state":"up:active","state_seq":18,"addr":"172.21.15.178:6839/70628097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6838","nonce":70628097},{"type":"v1","addr":"172.21.15.178:6839","nonce":70628097}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24638":{"gid":24638,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":18,"addr":"172.21.15.165:6837/6215088","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6836","nonce":6215088},{"type":"v1","addr":"172.21.15.165:6837","nonce":6215088}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24643":{"gid":24643,"name":"g","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.12:6839/3124093009","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6838","nonce":3124093009},{"type":"v1","addr":"172.21.15.12:6839","nonce":3124093009}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24674":{"gid":24674,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.165:6839/501312984","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6838","nonce":501312984},{"type":"v1","addr":"172.21.15.165:6839","nonce":501312984}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_27751":{"gid":27751,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6837/1436600611","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":1436600611},{"type":"v1","addr":"172.21.15.178:6837","nonce":1436600611}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24674,"qdb_cluster":[24674,24643,24638,24637,24614]},"id":1}]} 2024-09-11T00:31:19.684 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 104 2024-09-11T00:31:19.686 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:19.684+0000 7f8994a00640 1 -- 172.21.15.12:0/1887607198 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f89740b5410 msgr2=0x7f89740b78d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:31:19.686 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:19.684+0000 7f8994a00640 1 --2- 172.21.15.12:0/1887607198 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f89740b5410 0x7f89740b78d0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f8980031440 tx=0x7f8980005b50 comp rx=0 tx=0).stop 2024-09-11T00:31:19.686 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:19.684+0000 7f8994a00640 1 -- 172.21.15.12:0/1887607198 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8990108f90 msgr2=0x7f8990075f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:31:19.686 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:19.684+0000 7f8994a00640 1 --2- 172.21.15.12:0/1887607198 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8990108f90 0x7f8990075f20 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f898800eb10 tx=0x7f898800c2d0 comp rx=0 tx=0).stop 2024-09-11T00:31:19.686 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:19.684+0000 7f8994a00640 1 -- 172.21.15.12:0/1887607198 shutdown_connections 2024-09-11T00:31:19.686 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:19.684+0000 7f8994a00640 1 --2- 172.21.15.12:0/1887607198 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f89740b5410 0x7f89740b78d0 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:19.686 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:19.684+0000 7f8994a00640 1 --2- 172.21.15.12:0/1887607198 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8990108f90 0x7f8990075f20 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:19.686 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:19.684+0000 7f8994a00640 1 --2- 172.21.15.12:0/1887607198 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8990108610 0x7f89900759e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:19.687 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:19.684+0000 7f8994a00640 1 --2- 172.21.15.12:0/1887607198 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8990102c00 0x7f89900754a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:19.687 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:19.684+0000 7f8994a00640 1 -- 172.21.15.12:0/1887607198 >> 172.21.15.12:0/1887607198 conn(0x7f89900fe6d0 msgr2=0x7f89900ffdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:31:19.687 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:19.684+0000 7f8994a00640 1 -- 172.21.15.12:0/1887607198 shutdown_connections 2024-09-11T00:31:19.687 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:19.685+0000 7f8994a00640 1 -- 172.21.15.12:0/1887607198 wait complete. 2024-09-11T00:31:20.296 INFO:tasks.workunit.client.0.smithi012.stdout: 10 188625 15.06 MB/sec execute 556 sec latency 13183.338 ms 2024-09-11T00:31:20.533 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:31:20 smithi165 ceph-mon[35356]: pgmap v1127: 129 pgs: 129 active+clean; 826 MiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.8 MiB/s wr, 139 op/s 2024-09-11T00:31:20.533 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:31:20 smithi165 ceph-mon[35356]: from='client.? 172.21.15.12:0/1887607198' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T00:31:20.563 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:31:20 smithi012 ceph-mon[30333]: pgmap v1127: 129 pgs: 129 active+clean; 826 MiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.8 MiB/s wr, 139 op/s 2024-09-11T00:31:20.563 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:31:20 smithi012 ceph-mon[30333]: from='client.? 172.21.15.12:0/1887607198' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T00:31:20.581 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:31:20 smithi178 ceph-mon[35919]: pgmap v1127: 129 pgs: 129 active+clean; 826 MiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.8 MiB/s wr, 139 op/s 2024-09-11T00:31:20.581 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:31:20 smithi178 ceph-mon[35919]: from='client.? 172.21.15.12:0/1887607198' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T00:31:20.712 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24674 quiesce db --set-id 7c006079 --release --await' 2024-09-11T00:31:20.713 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid ff3ad74a-6fcf-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24674 quiesce db --set-id 7c006079 --release --await 2024-09-11T00:31:21.101 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:31:21.296 INFO:tasks.workunit.client.0.smithi012.stdout: 10 188625 15.04 MB/sec execute 557 sec latency 14183.456 ms 2024-09-11T00:31:21.681 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:21.671+0000 7fc949a00640 1 -- 172.21.15.12:0/1385786730 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc944104df0 msgr2=0x7fc9441051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:31:21.681 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:21.671+0000 7fc949a00640 1 --2- 172.21.15.12:0/1385786730 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc944104df0 0x7fc9441051d0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fc934009900 tx=0x7fc93402a410 comp rx=0 tx=0).stop 2024-09-11T00:31:21.681 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:21.671+0000 7fc949a00640 1 -- 172.21.15.12:0/1385786730 shutdown_connections 2024-09-11T00:31:21.681 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:21.671+0000 7fc949a00640 1 --2- 172.21.15.12:0/1385786730 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc94410b180 0x7fc944111980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:21.681 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:21.671+0000 7fc949a00640 1 --2- 172.21.15.12:0/1385786730 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc94410a800 0x7fc94410ac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:21.682 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:21.671+0000 7fc949a00640 1 --2- 172.21.15.12:0/1385786730 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc944104df0 0x7fc9441051d0 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:21.682 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:21.671+0000 7fc949a00640 1 -- 172.21.15.12:0/1385786730 >> 172.21.15.12:0/1385786730 conn(0x7fc944100840 msgr2=0x7fc944102c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:31:21.682 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:21.671+0000 7fc949a00640 1 -- 172.21.15.12:0/1385786730 shutdown_connections 2024-09-11T00:31:21.682 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:21.671+0000 7fc949a00640 1 -- 172.21.15.12:0/1385786730 wait complete. 2024-09-11T00:31:21.682 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:21.672+0000 7fc949a00640 1 Processor -- start 2024-09-11T00:31:21.682 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:21.672+0000 7fc949a00640 1 -- start start 2024-09-11T00:31:21.682 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:21.672+0000 7fc949a00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc944104df0 0x7fc94419b1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:31:21.682 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:21.672+0000 7fc949a00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc94410a800 0x7fc94419b710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:31:21.682 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:21.672+0000 7fc949a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc94410b180 0x7fc9441a0ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:31:21.682 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:21.672+0000 7fc949a00640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fc944114e00 con 0x7fc94410a800 2024-09-11T00:31:21.682 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:21.672+0000 7fc949a00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fc944114c80 con 0x7fc94410b180 2024-09-11T00:31:21.682 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:21.672+0000 7fc949a00640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fc944114f80 con 0x7fc944104df0 2024-09-11T00:31:21.682 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:21.672+0000 7fc943e00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc944104df0 0x7fc94419b1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:31:21.683 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:21.673+0000 7fc943e00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc944104df0 0x7fc94419b1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.12:37250/0 (socket says 172.21.15.12:37250) 2024-09-11T00:31:21.683 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:21.673+0000 7fc943e00640 1 -- 172.21.15.12:0/2895743308 learned_addr learned my addr 172.21.15.12:0/2895743308 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:31:21.683 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:21.673+0000 7fc93b400640 1 --2- 172.21.15.12:0/2895743308 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc94410a800 0x7fc94419b710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:31:21.683 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:21.673+0000 7fc949000640 1 --2- 172.21.15.12:0/2895743308 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc94410b180 0x7fc9441a0ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:31:21.683 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:21.673+0000 7fc943e00640 1 -- 172.21.15.12:0/2895743308 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc94410b180 msgr2=0x7fc9441a0ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:31:21.683 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:21.673+0000 7fc943e00640 1 --2- 172.21.15.12:0/2895743308 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc94410b180 0x7fc9441a0ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:21.683 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:21.673+0000 7fc943e00640 1 -- 172.21.15.12:0/2895743308 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc94410a800 msgr2=0x7fc94419b710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:31:21.683 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:21.673+0000 7fc943e00640 1 --2- 172.21.15.12:0/2895743308 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc94410a800 0x7fc94419b710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:21.683 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:21.673+0000 7fc943e00640 1 -- 172.21.15.12:0/2895743308 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9441a1280 con 0x7fc944104df0 2024-09-11T00:31:21.683 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:21.673+0000 7fc943e00640 1 --2- 172.21.15.12:0/2895743308 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc944104df0 0x7fc94419b1d0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fc934005940 tx=0x7fc9340040c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:31:21.683 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:21.674+0000 7fc941600640 1 -- 172.21.15.12:0/2895743308 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc93403f030 con 0x7fc944104df0 2024-09-11T00:31:21.683 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:21.674+0000 7fc941600640 1 -- 172.21.15.12:0/2895743308 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc9340044c0 con 0x7fc944104df0 2024-09-11T00:31:21.683 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:21.674+0000 7fc949a00640 1 -- 172.21.15.12:0/2895743308 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc9441a1510 con 0x7fc944104df0 2024-09-11T00:31:21.683 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:21.674+0000 7fc941600640 1 -- 172.21.15.12:0/2895743308 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc934035070 con 0x7fc944104df0 2024-09-11T00:31:21.684 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:21.674+0000 7fc949a00640 1 -- 172.21.15.12:0/2895743308 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9441a7c30 con 0x7fc944104df0 2024-09-11T00:31:21.684 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:21.677+0000 7fc94da7f740 1 Processor -- start 2024-09-11T00:31:21.684 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:21.678+0000 7fc94da7f740 1 -- start start 2024-09-11T00:31:21.684 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:21.678+0000 7fc94da7f740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55999b0c11d0 0x55999b0c15b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:31:21.684 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:21.678+0000 7fc94da7f740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55999b0c1af0 0x55999b0cd540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:31:21.684 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:21.678+0000 7fc94da7f740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55999b0cda80 0x55999b0cfe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:31:21.684 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:21.678+0000 7fc94da7f740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55999b038290 con 0x55999b0c11d0 2024-09-11T00:31:21.684 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:21.678+0000 7fc94da7f740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55999b0276c0 con 0x55999b0c1af0 2024-09-11T00:31:21.684 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:21.678+0000 7fc94da7f740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55999b02b4d0 con 0x55999b0cda80 2024-09-11T00:31:21.684 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:21.679+0000 7fc949000640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55999b0c11d0 0x55999b0c15b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:31:21.684 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:21.679+0000 7fc949000640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55999b0c11d0 0x55999b0c15b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:58358/0 (socket says 172.21.15.12:58358) 2024-09-11T00:31:21.684 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:21.679+0000 7fc949000640 1 -- 172.21.15.12:0/391142612 learned_addr learned my addr 172.21.15.12:0/391142612 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:31:21.685 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:21.679+0000 7fc93b400640 1 --2- 172.21.15.12:0/391142612 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55999b0cda80 0x55999b0cfe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:31:21.685 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:21.679+0000 7fc949000640 1 -- 172.21.15.12:0/391142612 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55999b0cda80 msgr2=0x55999b0cfe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:31:21.685 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:21.679+0000 7fc949000640 1 --2- 172.21.15.12:0/391142612 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55999b0cda80 0x55999b0cfe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:21.685 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:21.679+0000 7fc949000640 1 -- 172.21.15.12:0/391142612 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55999b0c1af0 msgr2=0x55999b0cd540 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-11T00:31:21.685 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:21.679+0000 7fc949000640 1 --2- 172.21.15.12:0/391142612 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55999b0c1af0 0x55999b0cd540 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:21.685 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:21.679+0000 7fc949000640 1 -- 172.21.15.12:0/391142612 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55999afb4460 con 0x55999b0c11d0 2024-09-11T00:31:21.685 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:21.679+0000 7fc941600640 1 -- 172.21.15.12:0/2895743308 <== mon.1 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7fc93403ba20 con 0x7fc944104df0 2024-09-11T00:31:21.685 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:21.679+0000 7fc949000640 1 --2- 172.21.15.12:0/391142612 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55999b0c11d0 0x55999b0c15b0 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7fc93c009cd0 tx=0x7fc93c011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=738dbefe6fb72fd7 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:31:21.685 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:21.680+0000 7fc939600640 1 -- 172.21.15.12:0/391142612 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc93c01e070 con 0x55999b0c11d0 2024-09-11T00:31:21.685 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:21.680+0000 7fc939600640 1 -- 172.21.15.12:0/391142612 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc93c019ae0 con 0x55999b0c11d0 2024-09-11T00:31:21.685 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:21.680+0000 7fc939600640 1 -- 172.21.15.12:0/391142612 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc93c00e9c0 con 0x55999b0c11d0 2024-09-11T00:31:21.689 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:21.680+0000 7fc94da7f740 1 -- 172.21.15.12:0/391142612 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55999b0c11d0 msgr2=0x55999b0c15b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:31:21.689 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:21.680+0000 7fc94da7f740 1 --2- 172.21.15.12:0/391142612 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55999b0c11d0 0x55999b0c15b0 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7fc93c009cd0 tx=0x7fc93c011810 comp rx=0 tx=0).stop 2024-09-11T00:31:21.689 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:21.681+0000 7fc94da7f740 1 -- 172.21.15.12:0/391142612 shutdown_connections 2024-09-11T00:31:21.689 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:21.681+0000 7fc94da7f740 1 --2- 172.21.15.12:0/391142612 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55999b0cda80 0x55999b0cfe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:21.689 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:21.681+0000 7fc94da7f740 1 --2- 172.21.15.12:0/391142612 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55999b0c1af0 0x55999b0cd540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:21.689 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:21.681+0000 7fc94da7f740 1 --2- 172.21.15.12:0/391142612 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55999b0c11d0 0x55999b0c15b0 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:21.690 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:21.681+0000 7fc94da7f740 1 -- 172.21.15.12:0/391142612 >> 172.21.15.12:0/391142612 conn(0x55999b0bf000 msgr2=0x55999b0c01a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:31:21.690 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:21.681+0000 7fc94da7f740 1 -- 172.21.15.12:0/391142612 shutdown_connections 2024-09-11T00:31:21.690 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:21.681+0000 7fc94da7f740 1 -- 172.21.15.12:0/391142612 wait complete. 2024-09-11T00:31:21.690 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:21.681+0000 7fc941600640 1 --2- 172.21.15.12:0/2895743308 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fc9280b52a0 0x7fc9280b7760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:31:21.690 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:21.681+0000 7fc941600640 1 -- 172.21.15.12:0/2895743308 <== mon.1 v2:172.21.15.178:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9208+0+0 (secure 0 0 0) 0x7fc9341065d0 con 0x7fc944104df0 2024-09-11T00:31:21.690 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:21.681+0000 7fc93b400640 1 --2- 172.21.15.12:0/2895743308 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fc9280b52a0 0x7fc9280b7760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:31:21.690 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:21.681+0000 7fc94da7f740 1 -- start start 2024-09-11T00:31:21.690 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:21.682+0000 7fc94da7f740 1 Processor -- start 2024-09-11T00:31:21.690 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:21.682+0000 7fc93b400640 1 --2- 172.21.15.12:0/2895743308 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fc9280b52a0 0x7fc9280b7760 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x55999b0c1180 tx=0x7fc92c009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:31:21.690 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:21.682+0000 7fc94da7f740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55999b0c11d0 0x55999b0ca9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:31:21.690 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:21.682+0000 7fc94da7f740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55999b0c1af0 0x55999b0caf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:31:21.690 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:21.682+0000 7fc94da7f740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55999b0cda80 0x55999b0cb460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:31:21.691 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:21.682+0000 7fc94da7f740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55999ae88bb0 con 0x55999b0c1af0 2024-09-11T00:31:21.691 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:21.682+0000 7fc94da7f740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55999afc26c0 con 0x55999b0c11d0 2024-09-11T00:31:21.691 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:21.682+0000 7fc94da7f740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55999b0d0730 con 0x55999b0cda80 2024-09-11T00:31:21.691 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:21.682+0000 7fc94da7f740 10 client.0 _ll_register_callbacks cb 0x55999afdd490 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:31:21.691 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:21.683+0000 7fc93b400640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55999b0c1af0 0x55999b0caf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:31:21.691 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:21.683+0000 7fc93b400640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55999b0c1af0 0x55999b0caf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:58372/0 (socket says 172.21.15.12:58372) 2024-09-11T00:31:21.691 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:21.683+0000 7fc93b400640 1 -- 172.21.15.12:0/3455003871 learned_addr learned my addr 172.21.15.12:0/3455003871 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:31:21.691 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:21.683+0000 7fc949000640 1 --2- 172.21.15.12:0/3455003871 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55999b0cda80 0x55999b0cb460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:31:21.691 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:21.683+0000 7fc943e00640 1 --2- 172.21.15.12:0/3455003871 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55999b0c11d0 0x55999b0ca9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:31:21.691 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:21.683+0000 7fc93b400640 1 -- 172.21.15.12:0/3455003871 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55999b0cda80 msgr2=0x55999b0cb460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:31:21.691 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:21.683+0000 7fc93b400640 1 --2- 172.21.15.12:0/3455003871 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55999b0cda80 0x55999b0cb460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:21.691 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:21.683+0000 7fc93b400640 1 -- 172.21.15.12:0/3455003871 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55999b0c11d0 msgr2=0x55999b0ca9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:31:21.691 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:21.683+0000 7fc93b400640 1 --2- 172.21.15.12:0/3455003871 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55999b0c11d0 0x55999b0ca9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:21.692 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:21.683+0000 7fc93b400640 1 -- 172.21.15.12:0/3455003871 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55999ad85b00 con 0x55999b0c1af0 2024-09-11T00:31:21.692 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:21.683+0000 7fc93b400640 1 --2- 172.21.15.12:0/3455003871 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55999b0c1af0 0x55999b0caf20 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7fc92c004070 tx=0x7fc92c004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=678decbd0f2cd819 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:31:21.692 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:21.683+0000 7fc932000640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-09-11T00:31:21.692 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:21.684+0000 7fc932000640 1 -- 172.21.15.12:0/3455003871 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc92c004b20 con 0x55999b0c1af0 2024-09-11T00:31:21.692 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:21.684+0000 7fc932000640 1 -- 172.21.15.12:0/3455003871 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 9208+0+0 (secure 0 0 0) 0x7fc92c004cc0 con 0x55999b0c1af0 2024-09-11T00:31:21.692 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:21.684+0000 7fc94da7f740 1 -- 172.21.15.12:0/3455003871 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc93c019ae0 con 0x55999b0c1af0 2024-09-11T00:31:21.692 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:21.684+0000 7fc932000640 1 -- 172.21.15.12:0/3455003871 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=104) ==== 24+0+0 (secure 0 0 0) 0x7fc92c005020 con 0x55999b0c1af0 2024-09-11T00:31:21.692 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:21.684+0000 7fc94da7f740 10 client.18084 fetch_fsmap learned FSMap version 104 2024-09-11T00:31:21.692 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:21.684+0000 7fc94da7f740 1 -- 172.21.15.12:0/3455003871 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=104,monmap=4+}) -- 0x55999adfc960 con 0x55999b0c1af0 2024-09-11T00:31:21.692 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:21.684+0000 7fc932000640 1 -- 172.21.15.12:0/3455003871 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc93c019ae0 con 0x55999b0c1af0 2024-09-11T00:31:21.692 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:21.684+0000 7fc932000640 1 -- 172.21.15.12:0/3455003871 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 104) ==== 8370+0+0 (secure 0 0 0) 0x7fc92c028780 con 0x55999b0c1af0 2024-09-11T00:31:21.692 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:21.685+0000 7fc94da7f740 10 client.18084 fetch_fsmap finished waiting for FSMap version 104 2024-09-11T00:31:21.692 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:21.685+0000 7fc94da7f740 10 client.18084 resolve_mds: validated gid 24674 aka daemon mds.b 2024-09-11T00:31:21.692 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:21.685+0000 7fc94da7f740 20 client.18084 populate_metadata read hostname 'smithi012' 2024-09-11T00:31:21.693 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:21.685+0000 7fc94da7f740 1 --2- 172.21.15.12:0/3455003871 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55999b0dd610 0x55999b0dfa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:31:21.693 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:21.685+0000 7fc94da7f740 4 client.18084 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T00:31:21.693 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:21.685+0000 7fc94da7f740 1 -- 172.21.15.12:0/3455003871 --> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55999b03a1e0 con 0x55999b0dd610 2024-09-11T00:31:21.693 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:21.685+0000 7fc943e00640 1 --2- 172.21.15.12:0/3455003871 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55999b0dd610 0x55999b0dfa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:31:21.693 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:21.686+0000 7fc943e00640 1 --2- 172.21.15.12:0/3455003871 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55999b0dd610 0x55999b0dfa00 crc :-1 s=READY pgs=331 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=be1994c40a885e7d server_cookie=b16f555742dd34fd in_seq=0 out_seq=0 2024-09-11T00:31:21.693 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:21.686+0000 7fc932000640 10 client.18084 ms_handle_connect on v2:172.21.15.165:6838/501312984 2024-09-11T00:31:21.693 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:21.687+0000 7fc932000640 1 -- 172.21.15.12:0/3455003871 <== mds.0 v2:172.21.15.165:6838/501312984 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55999b03a1e0 con 0x55999b0dd610 2024-09-11T00:31:21.693 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:21.687+0000 7fc932000640 10 client.18084 handle_command_reply: tid=0 2024-09-11T00:31:21.693 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:21.687+0000 7fc94da7f740 1 client.18084 shutdown 2024-09-11T00:31:21.693 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:21.687+0000 7fc94da7f740 1 -- 172.21.15.12:0/3455003871 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55999b0c1af0 msgr2=0x55999b0caf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:31:21.693 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:21.687+0000 7fc94da7f740 1 --2- 172.21.15.12:0/3455003871 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55999b0c1af0 0x55999b0caf20 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7fc92c004070 tx=0x7fc92c004ab0 comp rx=0 tx=0).stop 2024-09-11T00:31:21.693 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:21.687+0000 7fc94da7f740 1 -- 172.21.15.12:0/3455003871 shutdown_connections 2024-09-11T00:31:21.694 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:21.687+0000 7fc94da7f740 1 --2- 172.21.15.12:0/3455003871 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55999b0dd610 0x55999b0dfa00 crc :-1 s=READY pgs=331 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:21.694 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:21.687+0000 7fc94da7f740 1 --2- 172.21.15.12:0/3455003871 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55999b0cda80 0x55999b0cb460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:21.694 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:21.687+0000 7fc94da7f740 1 --2- 172.21.15.12:0/3455003871 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55999b0c1af0 0x55999b0caf20 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:21.694 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:21.687+0000 7fc94da7f740 1 --2- 172.21.15.12:0/3455003871 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55999b0c11d0 0x55999b0ca9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:21.694 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:21.687+0000 7fc94da7f740 1 -- 172.21.15.12:0/3455003871 >> 172.21.15.12:0/3455003871 conn(0x55999b0bf000 msgr2=0x55999b0c4ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:31:21.694 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:21.687+0000 7fc932000640 0 client.18084 ms_handle_reset on v2:172.21.15.165:6838/501312984 2024-09-11T00:31:21.694 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:21.687+0000 7fc94da7f740 1 -- 172.21.15.12:0/3455003871 shutdown_connections 2024-09-11T00:31:21.694 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:21.687+0000 7fc94da7f740 1 -- 172.21.15.12:0/3455003871 wait complete. 2024-09-11T00:31:21.694 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:21.688+0000 7fc94da7f740 20 client.18084 trim_cache size 0 max 16384 2024-09-11T00:31:21.707 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:21.705+0000 7fc94da7f740 1 Processor -- start 2024-09-11T00:31:21.707 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:21.705+0000 7fc94da7f740 1 -- start start 2024-09-11T00:31:21.708 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:21.706+0000 7fc94da7f740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55999b0c11d0 0x55999b0c0580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:31:21.708 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:21.706+0000 7fc94da7f740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55999b0c1af0 0x55999b0c9260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:31:21.708 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:21.706+0000 7fc94da7f740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55999b0cda80 0x55999b0c7bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:31:21.708 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:21.706+0000 7fc94da7f740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fc9441b1b60 con 0x55999b0cda80 2024-09-11T00:31:21.708 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:21.706+0000 7fc94da7f740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fc9441b36a0 con 0x55999b0c11d0 2024-09-11T00:31:21.708 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:21.706+0000 7fc94da7f740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fc9441b2230 con 0x55999b0c1af0 2024-09-11T00:31:21.708 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:21.706+0000 7fc949000640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55999b0cda80 0x55999b0c7bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:31:21.709 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:21.706+0000 7fc949000640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55999b0cda80 0x55999b0c7bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:58378/0 (socket says 172.21.15.12:58378) 2024-09-11T00:31:21.709 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:21.706+0000 7fc949000640 1 -- 172.21.15.12:0/3957393094 learned_addr learned my addr 172.21.15.12:0/3957393094 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:31:21.709 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:21.706+0000 7fc943e00640 1 --2- 172.21.15.12:0/3957393094 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55999b0c11d0 0x55999b0c0580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:31:21.709 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:21.706+0000 7fc93b400640 1 --2- 172.21.15.12:0/3957393094 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55999b0c1af0 0x55999b0c9260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:31:21.709 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:21.706+0000 7fc943e00640 1 -- 172.21.15.12:0/3957393094 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55999b0c1af0 msgr2=0x55999b0c9260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:31:21.709 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:21.706+0000 7fc943e00640 1 --2- 172.21.15.12:0/3957393094 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55999b0c1af0 0x55999b0c9260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:21.709 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:21.706+0000 7fc943e00640 1 -- 172.21.15.12:0/3957393094 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55999b0cda80 msgr2=0x55999b0c7bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:31:21.709 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:21.706+0000 7fc943e00640 1 --2- 172.21.15.12:0/3957393094 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55999b0cda80 0x55999b0c7bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:21.709 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:21.706+0000 7fc943e00640 1 -- 172.21.15.12:0/3957393094 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55999b0bfa80 con 0x55999b0c11d0 2024-09-11T00:31:21.709 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:21.707+0000 7fc949000640 1 --2- 172.21.15.12:0/3957393094 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55999b0cda80 0x55999b0c7bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:31:21.709 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:21.707+0000 7fc943e00640 1 --2- 172.21.15.12:0/3957393094 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55999b0c11d0 0x55999b0c0580 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x55999b0c4350 tx=0x7fc934005b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9a641cb5dba72e0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:31:21.711 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:21.707+0000 7fc938c00640 1 -- 172.21.15.12:0/3957393094 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc934106040 con 0x55999b0c11d0 2024-09-11T00:31:21.711 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:21.707+0000 7fc938c00640 1 -- 172.21.15.12:0/3957393094 <== mon.2 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc9340d1070 con 0x55999b0c11d0 2024-09-11T00:31:21.712 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:21.707+0000 7fc938c00640 1 -- 172.21.15.12:0/3957393094 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc934049040 con 0x55999b0c11d0 2024-09-11T00:31:21.712 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:21.708+0000 7fc94da7f740 1 -- 172.21.15.12:0/3957393094 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55999b0c11d0 msgr2=0x55999b0c0580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:31:21.712 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:21.708+0000 7fc94da7f740 1 --2- 172.21.15.12:0/3957393094 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55999b0c11d0 0x55999b0c0580 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x55999b0c4350 tx=0x7fc934005b50 comp rx=0 tx=0).stop 2024-09-11T00:31:21.712 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:21.708+0000 7fc94da7f740 1 -- 172.21.15.12:0/3957393094 shutdown_connections 2024-09-11T00:31:21.712 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:21.708+0000 7fc94da7f740 1 --2- 172.21.15.12:0/3957393094 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55999b0cda80 0x55999b0c7bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:21.712 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:21.708+0000 7fc94da7f740 1 --2- 172.21.15.12:0/3957393094 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55999b0c1af0 0x55999b0c9260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:21.712 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:21.708+0000 7fc94da7f740 1 --2- 172.21.15.12:0/3957393094 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55999b0c11d0 0x55999b0c0580 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:21.712 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:21.708+0000 7fc94da7f740 1 -- 172.21.15.12:0/3957393094 >> 172.21.15.12:0/3957393094 conn(0x55999b0bf000 msgr2=0x55999b0cc830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:31:21.712 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:21.708+0000 7fc94da7f740 1 -- 172.21.15.12:0/3957393094 shutdown_connections 2024-09-11T00:31:21.712 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:21.708+0000 7fc94da7f740 1 -- 172.21.15.12:0/3957393094 wait complete. 2024-09-11T00:31:21.712 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:21.708+0000 7fc94da7f740 1 -- start start 2024-09-11T00:31:21.713 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:21.709+0000 7fc94da7f740 1 Processor -- start 2024-09-11T00:31:21.713 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:21.709+0000 7fc94da7f740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55999b0c11d0 0x55999b0d5f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:31:21.713 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:21.709+0000 7fc94da7f740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55999b0c1af0 0x55999b0d2470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:31:21.713 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:21.709+0000 7fc94da7f740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55999b0cda80 0x55999b0d29b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:31:21.713 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:21.709+0000 7fc94da7f740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fc9441b2fd0 con 0x55999b0c11d0 2024-09-11T00:31:21.713 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:21.709+0000 7fc94da7f740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fc9441b2900 con 0x55999b0cda80 2024-09-11T00:31:21.713 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:21.709+0000 7fc94da7f740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fc9441afb40 con 0x55999b0c1af0 2024-09-11T00:31:21.713 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:21.709+0000 7fc94da7f740 10 client.0 _ll_register_callbacks cb 0x7fc92c009c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:31:21.713 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:21.709+0000 7fc949000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55999b0cda80 0x55999b0d29b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:31:21.713 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:21.709+0000 7fc943e00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55999b0c11d0 0x55999b0d5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:31:21.713 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:21.709+0000 7fc949000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55999b0cda80 0x55999b0d29b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.12:58272/0 (socket says 172.21.15.12:58272) 2024-09-11T00:31:21.713 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:21.709+0000 7fc949000640 1 -- 172.21.15.12:0/1764905416 learned_addr learned my addr 172.21.15.12:0/1764905416 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:31:21.714 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:21.710+0000 7fc93b400640 1 --2- 172.21.15.12:0/1764905416 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55999b0c1af0 0x55999b0d2470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:31:21.714 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:21.710+0000 7fc949000640 1 -- 172.21.15.12:0/1764905416 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55999b0c1af0 msgr2=0x55999b0d2470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:31:21.714 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:21.710+0000 7fc949000640 1 --2- 172.21.15.12:0/1764905416 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55999b0c1af0 0x55999b0d2470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:21.714 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:21.710+0000 7fc949000640 1 -- 172.21.15.12:0/1764905416 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55999b0c11d0 msgr2=0x55999b0d5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:31:21.714 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:21.710+0000 7fc949000640 1 --2- 172.21.15.12:0/1764905416 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55999b0c11d0 0x55999b0d5f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:21.714 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:21.710+0000 7fc949000640 1 -- 172.21.15.12:0/1764905416 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55999b0d2fa0 con 0x55999b0cda80 2024-09-11T00:31:21.714 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:21.710+0000 7fc943e00640 1 --2- 172.21.15.12:0/1764905416 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55999b0c11d0 0x55999b0d5f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:31:21.714 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:21.710+0000 7fc949000640 1 --2- 172.21.15.12:0/1764905416 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55999b0cda80 0x55999b0d29b0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fc93c005890 tx=0x7fc93c0055f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5955e59747524687 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:31:21.714 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:21.710+0000 7fc932000640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-09-11T00:31:21.715 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:21.710+0000 7fc932000640 1 -- 172.21.15.12:0/1764905416 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc93c01e040 con 0x55999b0cda80 2024-09-11T00:31:21.716 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:21.711+0000 7fc94da7f740 1 -- 172.21.15.12:0/1764905416 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc9340d1070 con 0x55999b0cda80 2024-09-11T00:31:21.716 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:21.711+0000 7fc932000640 1 -- 172.21.15.12:0/1764905416 <== mon.2 v2:172.21.15.165:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 9208+0+0 (secure 0 0 0) 0x7fc93c00e8d0 con 0x55999b0cda80 2024-09-11T00:31:21.716 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:21.711+0000 7fc932000640 1 -- 172.21.15.12:0/1764905416 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=104) ==== 24+0+0 (secure 0 0 0) 0x7fc93c034050 con 0x55999b0cda80 2024-09-11T00:31:21.716 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:21.711+0000 7fc94da7f740 10 client.28106 fetch_fsmap learned FSMap version 104 2024-09-11T00:31:21.716 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:21.711+0000 7fc94da7f740 1 -- 172.21.15.12:0/1764905416 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=104,monmap=4+}) -- 0x55999b14b320 con 0x55999b0cda80 2024-09-11T00:31:21.716 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:21.711+0000 7fc932000640 1 -- 172.21.15.12:0/1764905416 <== mon.2 v2:172.21.15.165:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc9340d1070 con 0x55999b0cda80 2024-09-11T00:31:21.716 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:21.711+0000 7fc932000640 1 -- 172.21.15.12:0/1764905416 <== mon.2 v2:172.21.15.165:3300/0 5 ==== fsmap(e 104) ==== 8370+0+0 (secure 0 0 0) 0x7fc93c003af0 con 0x55999b0cda80 2024-09-11T00:31:21.716 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:21.711+0000 7fc94da7f740 10 client.28106 fetch_fsmap finished waiting for FSMap version 104 2024-09-11T00:31:21.716 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:21.712+0000 7fc94da7f740 10 client.28106 resolve_mds: validated gid 24674 aka daemon mds.b 2024-09-11T00:31:21.716 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:21.712+0000 7fc94da7f740 20 client.28106 populate_metadata read hostname 'smithi012' 2024-09-11T00:31:21.716 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:21.712+0000 7fc94da7f740 1 --2- 172.21.15.12:0/1764905416 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55999b14b800 0x55999b14dbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:31:21.716 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:21.712+0000 7fc94da7f740 4 client.28106 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "7c006079", "release": true, "await": true}] 2024-09-11T00:31:21.717 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:21.712+0000 7fc94da7f740 1 -- 172.21.15.12:0/1764905416 --> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] -- command(tid 0: {"prefix": "quiesce db", "set_id": "7c006079", "release": true, "await": true}) -- 0x55999b035c90 con 0x55999b14b800 2024-09-11T00:31:21.717 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:21.712+0000 7fc943e00640 1 --2- 172.21.15.12:0/1764905416 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55999b14b800 0x55999b14dbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:31:21.717 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:21.713+0000 7fc943e00640 1 --2- 172.21.15.12:0/1764905416 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55999b14b800 0x55999b14dbf0 crc :-1 s=READY pgs=332 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3ad0020c712dbcbd server_cookie=cf33715537e0a698 in_seq=0 out_seq=0 2024-09-11T00:31:21.717 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:21.713+0000 7fc932000640 10 client.28106 ms_handle_connect on v2:172.21.15.165:6838/501312984 2024-09-11T00:31:21.749 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:21.747+0000 7fc932000640 1 -- 172.21.15.12:0/1764905416 <== mds.0 v2:172.21.15.165:6838/501312984 1 ==== command_reply(tid 0: 0 ) ==== 8+0+592 (crc 0 0 0) 0x55999b035c90 con 0x55999b14b800 2024-09-11T00:31:21.749 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:21.747+0000 7fc932000640 10 client.28106 handle_command_reply: tid=0 2024-09-11T00:31:21.750 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:21.747+0000 7fc94da7f740 1 client.28106 shutdown 2024-09-11T00:31:21.750 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:21.748+0000 7fc94da7f740 1 -- 172.21.15.12:0/1764905416 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55999b0cda80 msgr2=0x55999b0d29b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:31:21.750 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:21.748+0000 7fc94da7f740 1 --2- 172.21.15.12:0/1764905416 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55999b0cda80 0x55999b0d29b0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fc93c005890 tx=0x7fc93c0055f0 comp rx=0 tx=0).stop 2024-09-11T00:31:21.750 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:21.748+0000 7fc94da7f740 1 -- 172.21.15.12:0/1764905416 shutdown_connections 2024-09-11T00:31:21.750 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:21.748+0000 7fc94da7f740 1 --2- 172.21.15.12:0/1764905416 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55999b14b800 0x55999b14dbf0 crc :-1 s=READY pgs=332 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:21.750 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:21.748+0000 7fc94da7f740 1 --2- 172.21.15.12:0/1764905416 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55999b0cda80 0x55999b0d29b0 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:21.750 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:21.748+0000 7fc94da7f740 1 --2- 172.21.15.12:0/1764905416 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55999b0c1af0 0x55999b0d2470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:21.750 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:21.748+0000 7fc94da7f740 1 --2- 172.21.15.12:0/1764905416 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55999b0c11d0 0x55999b0d5f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:21.750 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:21.748+0000 7fc94da7f740 1 -- 172.21.15.12:0/1764905416 >> 172.21.15.12:0/1764905416 conn(0x55999b0bf000 msgr2=0x55999b0cc830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:31:21.750 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:21.748+0000 7fc932000640 0 client.28106 ms_handle_reset on v2:172.21.15.165:6838/501312984 2024-09-11T00:31:21.750 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:21.748+0000 7fc94da7f740 1 -- 172.21.15.12:0/1764905416 shutdown_connections 2024-09-11T00:31:21.750 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:21.748+0000 7fc94da7f740 1 -- 172.21.15.12:0/1764905416 wait complete. 2024-09-11T00:31:21.751 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:21.748+0000 7fc94da7f740 20 client.28106 trim_cache size 0 max 16384 2024-09-11T00:31:21.751 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-09-11T00:31:21.751 INFO:teuthology.orchestra.run.smithi012.stdout: "epoch": 104, 2024-09-11T00:31:21.751 INFO:teuthology.orchestra.run.smithi012.stdout: "leader": 24674, 2024-09-11T00:31:21.751 INFO:teuthology.orchestra.run.smithi012.stdout: "set_version": 76, 2024-09-11T00:31:21.752 INFO:teuthology.orchestra.run.smithi012.stdout: "sets": { 2024-09-11T00:31:21.752 INFO:teuthology.orchestra.run.smithi012.stdout: "7c006079": { 2024-09-11T00:31:21.752 INFO:teuthology.orchestra.run.smithi012.stdout: "version": 76, 2024-09-11T00:31:21.752 INFO:teuthology.orchestra.run.smithi012.stdout: "age_ref": 0.0, 2024-09-11T00:31:21.752 INFO:teuthology.orchestra.run.smithi012.stdout: "state": { 2024-09-11T00:31:21.752 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "RELEASED", 2024-09-11T00:31:21.752 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 0.0 2024-09-11T00:31:21.752 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:31:21.752 INFO:teuthology.orchestra.run.smithi012.stdout: "timeout": 90.0, 2024-09-11T00:31:21.752 INFO:teuthology.orchestra.run.smithi012.stdout: "expiration": 138.8, 2024-09-11T00:31:21.752 INFO:teuthology.orchestra.run.smithi012.stdout: "members": { 2024-09-11T00:31:21.752 INFO:teuthology.orchestra.run.smithi012.stdout: "file:/": { 2024-09-11T00:31:21.752 INFO:teuthology.orchestra.run.smithi012.stdout: "excluded": false, 2024-09-11T00:31:21.752 INFO:teuthology.orchestra.run.smithi012.stdout: "state": { 2024-09-11T00:31:21.753 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "RELEASED", 2024-09-11T00:31:21.753 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 0.0 2024-09-11T00:31:21.753 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:31:21.753 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:31:21.753 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:31:21.753 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:31:21.753 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:31:21.753 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-09-11T00:31:21.754 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:21.749+0000 7fc949a00640 1 -- 172.21.15.12:0/2895743308 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fc9280b52a0 msgr2=0x7fc9280b7760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:31:21.754 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:21.749+0000 7fc949a00640 1 --2- 172.21.15.12:0/2895743308 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fc9280b52a0 0x7fc9280b7760 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x55999b0c1180 tx=0x7fc92c009500 comp rx=0 tx=0).stop 2024-09-11T00:31:21.754 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:21.749+0000 7fc949a00640 1 -- 172.21.15.12:0/2895743308 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc944104df0 msgr2=0x7fc94419b1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:31:21.754 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:21.749+0000 7fc949a00640 1 --2- 172.21.15.12:0/2895743308 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc944104df0 0x7fc94419b1d0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fc934005940 tx=0x7fc9340040c0 comp rx=0 tx=0).stop 2024-09-11T00:31:21.754 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:21.749+0000 7fc949a00640 1 -- 172.21.15.12:0/2895743308 shutdown_connections 2024-09-11T00:31:21.754 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:21.749+0000 7fc949a00640 1 --2- 172.21.15.12:0/2895743308 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fc9280b52a0 0x7fc9280b7760 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:21.754 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:21.749+0000 7fc949a00640 1 --2- 172.21.15.12:0/2895743308 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc94410b180 0x7fc9441a0ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:21.754 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:21.749+0000 7fc949a00640 1 --2- 172.21.15.12:0/2895743308 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc94410a800 0x7fc94419b710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:21.754 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:21.749+0000 7fc949a00640 1 --2- 172.21.15.12:0/2895743308 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc944104df0 0x7fc94419b1d0 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:21.754 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:21.749+0000 7fc949a00640 1 -- 172.21.15.12:0/2895743308 >> 172.21.15.12:0/2895743308 conn(0x7fc944100840 msgr2=0x7fc944106db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:31:21.755 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:21.750+0000 7fc949a00640 1 -- 172.21.15.12:0/2895743308 shutdown_connections 2024-09-11T00:31:21.755 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:31:21.750+0000 7fc949a00640 1 -- 172.21.15.12:0/2895743308 wait complete. 2024-09-11T00:31:22.270 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:31:22 smithi012 ceph-mon[30333]: pgmap v1128: 129 pgs: 129 active+clean; 826 MiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.7 MiB/s wr, 129 op/s 2024-09-11T00:31:22.296 INFO:tasks.workunit.client.0.smithi012.stdout: 10 188995 15.04 MB/sec execute 558 sec latency 14659.411 ms 2024-09-11T00:31:22.532 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:31:22 smithi165 ceph-mon[35356]: pgmap v1128: 129 pgs: 129 active+clean; 826 MiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.7 MiB/s wr, 129 op/s 2024-09-11T00:31:22.581 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:31:22 smithi178 ceph-mon[35919]: pgmap v1128: 129 pgs: 129 active+clean; 826 MiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.7 MiB/s wr, 129 op/s 2024-09-11T00:31:22.735 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '7c006079', total seconds elapsed: 23.2 2024-09-11T00:31:23.297 INFO:tasks.workunit.client.0.smithi012.stdout: 10 190020 15.14 MB/sec execute 559 sec latency 101.918 ms 2024-09-11T00:31:24.296 INFO:tasks.workunit.client.0.smithi012.stdout: 10 190642 15.17 MB/sec execute 560 sec latency 195.711 ms 2024-09-11T00:31:24.314 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:31:24 smithi012 ceph-mon[30333]: pgmap v1129: 129 pgs: 129 active+clean; 826 MiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 341 KiB/s rd, 458 KiB/s wr, 89 op/s 2024-09-11T00:31:24.382 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:31:24 smithi178 ceph-mon[35919]: pgmap v1129: 129 pgs: 129 active+clean; 826 MiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 341 KiB/s rd, 458 KiB/s wr, 89 op/s 2024-09-11T00:31:24.532 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:31:24 smithi165 ceph-mon[35356]: pgmap v1129: 129 pgs: 129 active+clean; 826 MiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 341 KiB/s rd, 458 KiB/s wr, 89 op/s 2024-09-11T00:31:25.297 INFO:tasks.workunit.client.0.smithi012.stdout: 10 191055 15.18 MB/sec execute 561 sec latency 466.526 ms 2024-09-11T00:31:26.297 INFO:tasks.workunit.client.0.smithi012.stdout: 10 191532 15.20 MB/sec execute 562 sec latency 92.674 ms 2024-09-11T00:31:26.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:31:26 smithi012 ceph-mon[30333]: pgmap v1130: 129 pgs: 129 active+clean; 829 MiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 451 KiB/s rd, 403 KiB/s wr, 40 op/s 2024-09-11T00:31:26.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:31:26 smithi012 ceph-mon[30333]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T00:31:26.532 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:31:26 smithi165 ceph-mon[35356]: pgmap v1130: 129 pgs: 129 active+clean; 829 MiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 451 KiB/s rd, 403 KiB/s wr, 40 op/s 2024-09-11T00:31:26.533 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:31:26 smithi165 ceph-mon[35356]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T00:31:26.581 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:31:26 smithi178 ceph-mon[35919]: pgmap v1130: 129 pgs: 129 active+clean; 829 MiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 451 KiB/s rd, 403 KiB/s wr, 40 op/s 2024-09-11T00:31:26.581 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:31:26 smithi178 ceph-mon[35919]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T00:31:27.297 INFO:tasks.workunit.client.0.smithi012.stdout: 10 192114 15.22 MB/sec execute 563 sec latency 79.441 ms 2024-09-11T00:31:28.297 INFO:tasks.workunit.client.0.smithi012.stdout: 10 193134 15.30 MB/sec execute 564 sec latency 64.862 ms 2024-09-11T00:31:28.532 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:31:28 smithi165 ceph-mon[35356]: pgmap v1131: 129 pgs: 129 active+clean; 852 MiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 3.5 MiB/s wr, 264 op/s 2024-09-11T00:31:28.563 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:31:28 smithi012 ceph-mon[30333]: pgmap v1131: 129 pgs: 129 active+clean; 852 MiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 3.5 MiB/s wr, 264 op/s 2024-09-11T00:31:28.581 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:31:28 smithi178 ceph-mon[35919]: pgmap v1131: 129 pgs: 129 active+clean; 852 MiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 3.5 MiB/s wr, 264 op/s 2024-09-11T00:31:29.297 INFO:tasks.workunit.client.0.smithi012.stdout: 10 193889 15.34 MB/sec execute 565 sec latency 53.160 ms 2024-09-11T00:31:30.256 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:30.254+0000 7f88ec000640 1 -- 172.21.15.12:0/677875787 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f88e4075ac0 msgr2=0x7f88e4075f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:31:30.256 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:30.254+0000 7f88ec000640 1 --2- 172.21.15.12:0/677875787 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f88e4075ac0 0x7f88e4075f40 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f88d0009900 tx=0x7f88d002a3f0 comp rx=0 tx=0).stop 2024-09-11T00:31:30.257 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:30.255+0000 7f88ec000640 1 -- 172.21.15.12:0/677875787 shutdown_connections 2024-09-11T00:31:30.257 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:30.255+0000 7f88ec000640 1 --2- 172.21.15.12:0/677875787 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f88e410a910 0x7f88e410cd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:30.257 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:30.255+0000 7f88ec000640 1 --2- 172.21.15.12:0/677875787 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f88e4075ac0 0x7f88e4075f40 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:30.257 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:30.255+0000 7f88ec000640 1 --2- 172.21.15.12:0/677875787 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f88e4076f30 0x7f88e4075580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:30.257 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:30.255+0000 7f88ec000640 1 -- 172.21.15.12:0/677875787 >> 172.21.15.12:0/677875787 conn(0x7f88e40fe250 msgr2=0x7f88e4100670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:31:30.257 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:30.255+0000 7f88ec000640 1 -- 172.21.15.12:0/677875787 shutdown_connections 2024-09-11T00:31:30.257 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:30.255+0000 7f88ec000640 1 -- 172.21.15.12:0/677875787 wait complete. 2024-09-11T00:31:30.258 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:30.256+0000 7f88ec000640 1 Processor -- start 2024-09-11T00:31:30.258 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:30.256+0000 7f88ec000640 1 -- start start 2024-09-11T00:31:30.258 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:30.257+0000 7f88ec000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f88e4075ac0 0x7f88e419af80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:31:30.259 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:30.257+0000 7f88ec000640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f88e4076f30 0x7f88e419b5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:31:30.259 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:30.257+0000 7f88ec000640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f88e410a910 0x7f88e419c200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:31:30.259 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:30.257+0000 7f88ec000640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f88e4110830 con 0x7f88e410a910 2024-09-11T00:31:30.259 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:30.257+0000 7f88ec000640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f88e41106b0 con 0x7f88e4075ac0 2024-09-11T00:31:30.259 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:30.257+0000 7f88ec000640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f88e41109b0 con 0x7f88e4076f30 2024-09-11T00:31:30.259 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:30.258+0000 7f88e8c00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f88e4076f30 0x7f88e419b5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:31:30.260 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:30.258+0000 7f88e8c00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f88e4076f30 0x7f88e419b5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.12:46304/0 (socket says 172.21.15.12:46304) 2024-09-11T00:31:30.260 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:30.258+0000 7f88e8c00640 1 -- 172.21.15.12:0/3530245837 learned_addr learned my addr 172.21.15.12:0/3530245837 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:31:30.260 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:30.258+0000 7f88e9600640 1 --2- 172.21.15.12:0/3530245837 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f88e4075ac0 0x7f88e419af80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:31:30.260 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:30.258+0000 7f88ea000640 1 --2- 172.21.15.12:0/3530245837 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f88e410a910 0x7f88e419c200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:31:30.261 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:30.259+0000 7f88e9600640 1 -- 172.21.15.12:0/3530245837 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f88e4076f30 msgr2=0x7f88e419b5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:31:30.261 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:30.259+0000 7f88e9600640 1 --2- 172.21.15.12:0/3530245837 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f88e4076f30 0x7f88e419b5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:30.261 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:30.259+0000 7f88e9600640 1 -- 172.21.15.12:0/3530245837 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f88e410a910 msgr2=0x7f88e419c200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:31:30.261 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:30.259+0000 7f88e9600640 1 --2- 172.21.15.12:0/3530245837 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f88e410a910 0x7f88e419c200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:30.261 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:30.259+0000 7f88e9600640 1 -- 172.21.15.12:0/3530245837 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88e419cac0 con 0x7f88e4075ac0 2024-09-11T00:31:30.261 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:30.259+0000 7f88e9600640 1 --2- 172.21.15.12:0/3530245837 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f88e4075ac0 0x7f88e419af80 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f88d8005fd0 tx=0x7f88d800dde0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:31:30.261 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:30.260+0000 7f88e2000640 1 -- 172.21.15.12:0/3530245837 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88d8009c30 con 0x7f88e4075ac0 2024-09-11T00:31:30.261 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:30.260+0000 7f88e2000640 1 -- 172.21.15.12:0/3530245837 <== mon.2 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f88d8009dd0 con 0x7f88e4075ac0 2024-09-11T00:31:30.262 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:30.260+0000 7f88e2000640 1 -- 172.21.15.12:0/3530245837 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88d8003e20 con 0x7f88e4075ac0 2024-09-11T00:31:30.262 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:30.261+0000 7f88ec000640 1 -- 172.21.15.12:0/3530245837 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f88e419cdb0 con 0x7f88e4075ac0 2024-09-11T00:31:30.262 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:30.261+0000 7f88ec000640 1 -- 172.21.15.12:0/3530245837 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88e41a3480 con 0x7f88e4075ac0 2024-09-11T00:31:30.271 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:30.268+0000 7f88eee5e740 1 Processor -- start 2024-09-11T00:31:30.271 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:30.270+0000 7f88eee5e740 1 -- start start 2024-09-11T00:31:30.272 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:30.270+0000 7f88eee5e740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557b45e3cfa0 0x557b45e3f390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:31:30.272 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:30.271+0000 7f88eee5e740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557b45e3f8d0 0x557b45e41cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:31:30.272 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:30.271+0000 7f88e2000640 1 -- 172.21.15.12:0/3530245837 <== mon.2 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7f88d8003fc0 con 0x7f88e4075ac0 2024-09-11T00:31:30.273 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:30.271+0000 7f88eee5e740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557b45e42310 0x557b45e447d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:31:30.273 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:30.271+0000 7f88eee5e740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x557b45da76b0 con 0x557b45e3f8d0 2024-09-11T00:31:30.274 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:30.272+0000 7f88e8c00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557b45e42310 0x557b45e447d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:31:30.274 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:30.272+0000 7f88ea000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557b45e3cfa0 0x557b45e3f390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:31:30.274 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:30.272+0000 7f88eee5e740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x557b45d96ae0 con 0x557b45e3cfa0 2024-09-11T00:31:30.274 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:30.272+0000 7f88eee5e740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x557b45d9a8f0 con 0x557b45e42310 2024-09-11T00:31:30.274 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:30.272+0000 7f88ea000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557b45e3cfa0 0x557b45e3f390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.12:41704/0 (socket says 172.21.15.12:41704) 2024-09-11T00:31:30.274 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:30.272+0000 7f88ea000640 1 -- 172.21.15.12:0/3121258999 learned_addr learned my addr 172.21.15.12:0/3121258999 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:31:30.274 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:30.272+0000 7f88e9600640 1 --2- 172.21.15.12:0/3121258999 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557b45e3f8d0 0x557b45e41cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:31:30.275 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:30.273+0000 7f88ea000640 1 -- 172.21.15.12:0/3121258999 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557b45e42310 msgr2=0x557b45e447d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:31:30.275 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:30.273+0000 7f88ea000640 1 --2- 172.21.15.12:0/3121258999 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557b45e42310 0x557b45e447d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:30.275 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:30.273+0000 7f88ea000640 1 -- 172.21.15.12:0/3121258999 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557b45e3f8d0 msgr2=0x557b45e41cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:31:30.275 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:30.273+0000 7f88ea000640 1 --2- 172.21.15.12:0/3121258999 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557b45e3f8d0 0x557b45e41cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:30.276 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:30.273+0000 7f88ea000640 1 -- 172.21.15.12:0/3121258999 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557b45d63780 con 0x557b45e3cfa0 2024-09-11T00:31:30.276 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:30.274+0000 7f88ea000640 1 --2- 172.21.15.12:0/3121258999 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557b45e3cfa0 0x557b45e3f390 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f88dc009f40 tx=0x7f88dc010800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=78aaeb54d7d64997 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:31:30.276 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:30.275+0000 7f88d6600640 1 -- 172.21.15.12:0/3121258999 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88dc016070 con 0x557b45e3cfa0 2024-09-11T00:31:30.277 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:30.275+0000 7f88d6600640 1 -- 172.21.15.12:0/3121258999 <== mon.2 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f88dc004d20 con 0x557b45e3cfa0 2024-09-11T00:31:30.277 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:30.275+0000 7f88d6600640 1 -- 172.21.15.12:0/3121258999 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88dc0127b0 con 0x557b45e3cfa0 2024-09-11T00:31:30.277 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:30.276+0000 7f88eee5e740 1 -- 172.21.15.12:0/3121258999 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557b45e3cfa0 msgr2=0x557b45e3f390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:31:30.277 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:30.276+0000 7f88eee5e740 1 --2- 172.21.15.12:0/3121258999 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557b45e3cfa0 0x557b45e3f390 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f88dc009f40 tx=0x7f88dc010800 comp rx=0 tx=0).stop 2024-09-11T00:31:30.277 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:30.276+0000 7f88eee5e740 1 -- 172.21.15.12:0/3121258999 shutdown_connections 2024-09-11T00:31:30.278 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:30.276+0000 7f88eee5e740 1 --2- 172.21.15.12:0/3121258999 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557b45e42310 0x557b45e447d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:30.278 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:30.276+0000 7f88eee5e740 1 --2- 172.21.15.12:0/3121258999 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557b45e3f8d0 0x557b45e41cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:30.278 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:30.276+0000 7f88eee5e740 1 --2- 172.21.15.12:0/3121258999 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557b45e3cfa0 0x557b45e3f390 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:30.278 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:30.276+0000 7f88eee5e740 1 -- 172.21.15.12:0/3121258999 >> 172.21.15.12:0/3121258999 conn(0x557b45e380f0 msgr2=0x557b45e38e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:31:30.278 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:30.276+0000 7f88e2000640 1 --2- 172.21.15.12:0/3530245837 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f88cc0b53e0 0x7f88cc0b78a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:31:30.278 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:30.276+0000 7f88e2000640 1 -- 172.21.15.12:0/3530245837 <== mon.2 v2:172.21.15.165:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9208+0+0 (secure 0 0 0) 0x7f88d80e54d0 con 0x7f88e4075ac0 2024-09-11T00:31:30.280 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:30.278+0000 7f88eee5e740 1 -- 172.21.15.12:0/3121258999 shutdown_connections 2024-09-11T00:31:30.280 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:30.278+0000 7f88eee5e740 1 -- 172.21.15.12:0/3121258999 wait complete. 2024-09-11T00:31:30.281 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:30.279+0000 7f88e8c00640 1 --2- 172.21.15.12:0/3530245837 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f88cc0b53e0 0x7f88cc0b78a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:31:30.282 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:30.280+0000 7f88eee5e740 1 -- start start 2024-09-11T00:31:30.282 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:30.281+0000 7f88e8c00640 1 --2- 172.21.15.12:0/3530245837 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f88cc0b53e0 0x7f88cc0b78a0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x557b45e44d30 tx=0x7f88d00314a0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:31:30.283 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:30.281+0000 7f88eee5e740 1 Processor -- start 2024-09-11T00:31:30.285 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:30.282+0000 7f88eee5e740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557b45e3cfa0 0x557b45e46800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:31:30.285 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:30.282+0000 7f88eee5e740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557b45e3f8d0 0x557b45e46ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:31:30.285 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:30.282+0000 7f88eee5e740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557b45e47a50 0x557b45e47ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:31:30.286 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:30.282+0000 7f88eee5e740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x557b45cceab0 con 0x557b45e47a50 2024-09-11T00:31:30.286 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:30.282+0000 7f88eee5e740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x557b45d3d570 con 0x557b45e3f8d0 2024-09-11T00:31:30.286 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:30.282+0000 7f88eee5e740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x557b45e452f0 con 0x557b45e3cfa0 2024-09-11T00:31:30.286 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:30.282+0000 7f88eee5e740 10 client.0 _ll_register_callbacks cb 0x557b45d4c100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:31:30.286 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:30.283+0000 7f88e9600640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557b45e3cfa0 0x557b45e46800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:31:30.286 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:30.283+0000 7f88e9600640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557b45e3cfa0 0x557b45e46800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.12:46320/0 (socket says 172.21.15.12:46320) 2024-09-11T00:31:30.286 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:30.283+0000 7f88e9600640 1 -- 172.21.15.12:0/1178322822 learned_addr learned my addr 172.21.15.12:0/1178322822 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:31:30.286 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:30.283+0000 7f88e9600640 1 -- 172.21.15.12:0/1178322822 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557b45e3f8d0 msgr2=0x557b45e46ea0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-11T00:31:30.286 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:30.283+0000 7f88e9600640 1 --2- 172.21.15.12:0/1178322822 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557b45e3f8d0 0x557b45e46ea0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:30.286 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:30.283+0000 7f88e9600640 1 -- 172.21.15.12:0/1178322822 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557b45e47a50 msgr2=0x557b45e47ed0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-11T00:31:30.286 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:30.283+0000 7f88e9600640 1 --2- 172.21.15.12:0/1178322822 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557b45e47a50 0x557b45e47ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:30.286 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:30.283+0000 7f88e9600640 1 -- 172.21.15.12:0/1178322822 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557b45af8790 con 0x557b45e3cfa0 2024-09-11T00:31:30.286 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:30.284+0000 7f88e9600640 1 --2- 172.21.15.12:0/1178322822 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557b45e3cfa0 0x557b45e46800 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f88d80a7e30 tx=0x7f88d800df10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a563cd2360d4ebf4 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:31:30.286 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:30.284+0000 7f88caa00640 10 client.0 ms_handle_connect on v2:172.21.15.178:3300/0 2024-09-11T00:31:30.287 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:30.284+0000 7f88caa00640 1 -- 172.21.15.12:0/1178322822 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88d8020050 con 0x557b45e3cfa0 2024-09-11T00:31:30.287 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:30.284+0000 7f88caa00640 1 -- 172.21.15.12:0/1178322822 <== mon.1 v2:172.21.15.178:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 9208+0+0 (secure 0 0 0) 0x7f88d80a7ad0 con 0x557b45e3cfa0 2024-09-11T00:31:30.287 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:30.284+0000 7f88eee5e740 1 -- 172.21.15.12:0/1178322822 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f88dc004d20 con 0x557b45e3cfa0 2024-09-11T00:31:30.287 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:30.285+0000 7f88caa00640 1 -- 172.21.15.12:0/1178322822 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=106) ==== 24+0+0 (secure 0 0 0) 0x7f88d80ab1b0 con 0x557b45e3cfa0 2024-09-11T00:31:30.287 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:30.285+0000 7f88eee5e740 10 client.28096 fetch_fsmap learned FSMap version 106 2024-09-11T00:31:30.287 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:30.285+0000 7f88eee5e740 1 -- 172.21.15.12:0/1178322822 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,fsmap=106,monmap=4+}) -- 0x557b45b71620 con 0x557b45e3cfa0 2024-09-11T00:31:30.288 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:30.287+0000 7f88caa00640 1 -- 172.21.15.12:0/1178322822 <== mon.1 v2:172.21.15.178:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f88dc004d20 con 0x557b45e3cfa0 2024-09-11T00:31:30.288 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:30.287+0000 7f88caa00640 1 -- 172.21.15.12:0/1178322822 <== mon.1 v2:172.21.15.178:3300/0 5 ==== fsmap(e 106) ==== 8382+0+0 (secure 0 0 0) 0x7f88d80acb90 con 0x557b45e3cfa0 2024-09-11T00:31:30.288 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:30.287+0000 7f88eee5e740 10 client.28096 fetch_fsmap finished waiting for FSMap version 106 2024-09-11T00:31:30.288 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:30.287+0000 7f88eee5e740 10 client.28096 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T00:31:30.288 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:30.287+0000 7f88eee5e740 20 client.28096 populate_metadata read hostname 'smithi012' 2024-09-11T00:31:30.288 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:30.287+0000 7f88eee5e740 1 --2- 172.21.15.12:0/1178322822 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x557b45e32c70 0x557b45e35110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:31:30.289 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:30.287+0000 7f88eee5e740 4 client.28096 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T00:31:30.289 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:30.287+0000 7f88eee5e740 1 -- 172.21.15.12:0/1178322822 --> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557b45da9600 con 0x557b45e32c70 2024-09-11T00:31:30.289 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:30.287+0000 7f88e8c00640 1 --2- 172.21.15.12:0/1178322822 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x557b45e32c70 0x557b45e35110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:31:30.289 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:30.288+0000 7f88caa00640 10 client.28096 ms_handle_connect on v2:172.21.15.165:6838/501312984 2024-09-11T00:31:30.290 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:30.288+0000 7f88e8c00640 1 --2- 172.21.15.12:0/1178322822 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x557b45e32c70 0x557b45e35110 crc :-1 s=READY pgs=333 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=670697b6a78bf02d server_cookie=46c4b4a99d5044af in_seq=0 out_seq=0 2024-09-11T00:31:30.292 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:30.291+0000 7f88caa00640 1 -- 172.21.15.12:0/1178322822 <== mds.0 v2:172.21.15.165:6838/501312984 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x557b45da9600 con 0x557b45e32c70 2024-09-11T00:31:30.292 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:30.291+0000 7f88caa00640 10 client.28096 handle_command_reply: tid=0 2024-09-11T00:31:30.292 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:30.291+0000 7f88eee5e740 1 client.28096 shutdown 2024-09-11T00:31:30.293 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:30.292+0000 7f88eee5e740 1 -- 172.21.15.12:0/1178322822 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557b45e3cfa0 msgr2=0x557b45e46800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:31:30.293 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:30.292+0000 7f88eee5e740 1 --2- 172.21.15.12:0/1178322822 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557b45e3cfa0 0x557b45e46800 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f88d80a7e30 tx=0x7f88d800df10 comp rx=0 tx=0).stop 2024-09-11T00:31:30.293 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:30.292+0000 7f88eee5e740 1 -- 172.21.15.12:0/1178322822 shutdown_connections 2024-09-11T00:31:30.293 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:30.292+0000 7f88eee5e740 1 --2- 172.21.15.12:0/1178322822 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x557b45e32c70 0x557b45e35110 crc :-1 s=READY pgs=333 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:30.294 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:30.292+0000 7f88eee5e740 1 --2- 172.21.15.12:0/1178322822 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557b45e47a50 0x557b45e47ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:30.294 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:30.292+0000 7f88eee5e740 1 --2- 172.21.15.12:0/1178322822 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557b45e3f8d0 0x557b45e46ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:30.294 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:30.292+0000 7f88eee5e740 1 --2- 172.21.15.12:0/1178322822 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557b45e3cfa0 0x557b45e46800 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:30.294 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:30.292+0000 7f88caa00640 0 client.28096 ms_handle_reset on v2:172.21.15.165:6838/501312984 2024-09-11T00:31:30.294 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:30.292+0000 7f88eee5e740 1 -- 172.21.15.12:0/1178322822 >> 172.21.15.12:0/1178322822 conn(0x557b45e380f0 msgr2=0x557b45e2ea60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:31:30.294 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:30.293+0000 7f88eee5e740 1 -- 172.21.15.12:0/1178322822 shutdown_connections 2024-09-11T00:31:30.294 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:30.293+0000 7f88eee5e740 1 -- 172.21.15.12:0/1178322822 wait complete. 2024-09-11T00:31:30.294 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:30.293+0000 7f88eee5e740 20 client.28096 trim_cache size 0 max 16384 2024-09-11T00:31:30.297 INFO:tasks.workunit.client.0.smithi012.stdout: 10 194463 15.38 MB/sec execute 566 sec latency 81.062 ms 2024-09-11T00:31:30.327 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:30.326+0000 7f88eee5e740 1 Processor -- start 2024-09-11T00:31:30.327 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:30.326+0000 7f88eee5e740 1 -- start start 2024-09-11T00:31:30.328 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:30.326+0000 7f88eee5e740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557b45e3cfa0 0x557b45e426f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:31:30.328 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:30.326+0000 7f88eee5e740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557b45e3f8d0 0x557b45e42c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:31:30.328 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:30.326+0000 7f88eee5e740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557b45e47a50 0x557b45e38950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:31:30.328 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:30.326+0000 7f88eee5e740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f88e41a9650 con 0x557b45e3cfa0 2024-09-11T00:31:30.328 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:30.326+0000 7f88eee5e740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f88e41ab190 con 0x557b45e47a50 2024-09-11T00:31:30.328 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:30.327+0000 7f88eee5e740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f88e41a9d20 con 0x557b45e3f8d0 2024-09-11T00:31:30.329 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:30.327+0000 7f88e9600640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557b45e3cfa0 0x557b45e426f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:31:30.329 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:30.327+0000 7f88ea000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557b45e47a50 0x557b45e38950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:31:30.329 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:30.328+0000 7f88e9600640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557b45e3cfa0 0x557b45e426f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:52448/0 (socket says 172.21.15.12:52448) 2024-09-11T00:31:30.329 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:30.328+0000 7f88e9600640 1 -- 172.21.15.12:0/2122994338 learned_addr learned my addr 172.21.15.12:0/2122994338 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:31:30.330 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:30.328+0000 7f88e8c00640 1 --2- 172.21.15.12:0/2122994338 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557b45e3f8d0 0x557b45e42c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:31:30.330 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:30.329+0000 7f88e9600640 1 -- 172.21.15.12:0/2122994338 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557b45e3f8d0 msgr2=0x557b45e42c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:31:30.330 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:30.329+0000 7f88e9600640 1 --2- 172.21.15.12:0/2122994338 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557b45e3f8d0 0x557b45e42c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:30.330 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:30.329+0000 7f88e9600640 1 -- 172.21.15.12:0/2122994338 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557b45e47a50 msgr2=0x557b45e38950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:31:30.331 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:30.329+0000 7f88e9600640 1 --2- 172.21.15.12:0/2122994338 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557b45e47a50 0x557b45e38950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:30.331 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:30.329+0000 7f88e9600640 1 -- 172.21.15.12:0/2122994338 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557b45e42030 con 0x557b45e3cfa0 2024-09-11T00:31:30.331 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:30.329+0000 7f88ea000640 1 --2- 172.21.15.12:0/2122994338 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557b45e47a50 0x557b45e38950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:31:30.332 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:30.331+0000 7f88e9600640 1 --2- 172.21.15.12:0/2122994338 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557b45e3cfa0 0x557b45e426f0 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f88d8000c00 tx=0x7f88d80b0490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69cc2cb38fe67b27 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:31:30.333 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:30.332+0000 7f88d5c00640 1 -- 172.21.15.12:0/2122994338 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88d8020070 con 0x557b45e3cfa0 2024-09-11T00:31:30.333 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:30.332+0000 7f88d5c00640 1 -- 172.21.15.12:0/2122994338 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f88d80b0100 con 0x557b45e3cfa0 2024-09-11T00:31:30.333 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:30.332+0000 7f88d5c00640 1 -- 172.21.15.12:0/2122994338 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88d80aafd0 con 0x557b45e3cfa0 2024-09-11T00:31:30.335 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:30.333+0000 7f88eee5e740 1 -- 172.21.15.12:0/2122994338 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557b45e3cfa0 msgr2=0x557b45e426f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:31:30.335 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:30.333+0000 7f88eee5e740 1 --2- 172.21.15.12:0/2122994338 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557b45e3cfa0 0x557b45e426f0 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f88d8000c00 tx=0x7f88d80b0490 comp rx=0 tx=0).stop 2024-09-11T00:31:30.335 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:30.334+0000 7f88eee5e740 1 -- 172.21.15.12:0/2122994338 shutdown_connections 2024-09-11T00:31:30.335 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:30.334+0000 7f88eee5e740 1 --2- 172.21.15.12:0/2122994338 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557b45e47a50 0x557b45e38950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:30.336 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:30.334+0000 7f88eee5e740 1 --2- 172.21.15.12:0/2122994338 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557b45e3f8d0 0x557b45e42c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:30.338 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:30.336+0000 7f88eee5e740 1 --2- 172.21.15.12:0/2122994338 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557b45e3cfa0 0x557b45e426f0 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:30.338 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:30.337+0000 7f88eee5e740 1 -- 172.21.15.12:0/2122994338 >> 172.21.15.12:0/2122994338 conn(0x557b45e380f0 msgr2=0x557b45e3c580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:31:30.338 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:30.337+0000 7f88eee5e740 1 -- 172.21.15.12:0/2122994338 shutdown_connections 2024-09-11T00:31:30.338 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:30.337+0000 7f88eee5e740 1 -- 172.21.15.12:0/2122994338 wait complete. 2024-09-11T00:31:30.339 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:30.338+0000 7f88eee5e740 1 -- start start 2024-09-11T00:31:30.339 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:30.338+0000 7f88eee5e740 1 Processor -- start 2024-09-11T00:31:30.340 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:30.339+0000 7f88eee5e740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557b45e3cfa0 0x557b45e4dfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:31:30.341 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:30.340+0000 7f88eee5e740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557b45e3f8d0 0x557b45e49d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:31:30.341 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:30.340+0000 7f88e9600640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557b45e3cfa0 0x557b45e4dfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:31:30.343 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:30.340+0000 7f88eee5e740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557b45e47a50 0x557b45e4a2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:31:30.343 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:30.340+0000 7f88eee5e740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f88e41aaac0 con 0x557b45e3f8d0 2024-09-11T00:31:30.343 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:30.340+0000 7f88eee5e740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f88e41aa3f0 con 0x557b45e3cfa0 2024-09-11T00:31:30.343 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:30.340+0000 7f88eee5e740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f88e41a7670 con 0x557b45e47a50 2024-09-11T00:31:30.343 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:30.340+0000 7f88eee5e740 10 client.0 _ll_register_callbacks cb 0x7f88d8005d00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:31:30.343 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:30.340+0000 7f88e8c00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557b45e3f8d0 0x557b45e49d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:31:30.344 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:30.340+0000 7f88e8c00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557b45e3f8d0 0x557b45e49d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:52458/0 (socket says 172.21.15.12:52458) 2024-09-11T00:31:30.344 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:30.340+0000 7f88e8c00640 1 -- 172.21.15.12:0/583064636 learned_addr learned my addr 172.21.15.12:0/583064636 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:31:30.344 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:30.341+0000 7f88e8c00640 1 -- 172.21.15.12:0/583064636 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557b45e47a50 msgr2=0x557b45e4a2c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-11T00:31:30.344 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:30.341+0000 7f88e8c00640 1 --2- 172.21.15.12:0/583064636 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557b45e47a50 0x557b45e4a2c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:30.344 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:30.341+0000 7f88e8c00640 1 -- 172.21.15.12:0/583064636 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557b45e3cfa0 msgr2=0x557b45e4dfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:31:30.344 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:30.341+0000 7f88e8c00640 1 --2- 172.21.15.12:0/583064636 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557b45e3cfa0 0x557b45e4dfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:30.344 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:30.341+0000 7f88e8c00640 1 -- 172.21.15.12:0/583064636 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557b45e4a800 con 0x557b45e3f8d0 2024-09-11T00:31:30.344 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:30.341+0000 7f88e9600640 1 --2- 172.21.15.12:0/583064636 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557b45e3cfa0 0x557b45e4dfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T00:31:30.344 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:30.342+0000 7f88caa00640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-09-11T00:31:30.344 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:30.342+0000 7f88e8c00640 1 --2- 172.21.15.12:0/583064636 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557b45e3f8d0 0x557b45e49d80 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f88d002a9e0 tx=0x7f88d003d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d2bd6e75da5fb96 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:31:30.345 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:30.342+0000 7f88caa00640 1 -- 172.21.15.12:0/583064636 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88d0005310 con 0x557b45e3f8d0 2024-09-11T00:31:30.346 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:30.343+0000 7f88eee5e740 1 -- 172.21.15.12:0/583064636 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f88d80b0100 con 0x557b45e3f8d0 2024-09-11T00:31:30.346 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:30.343+0000 7f88caa00640 1 -- 172.21.15.12:0/583064636 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 9208+0+0 (secure 0 0 0) 0x7f88d00054b0 con 0x557b45e3f8d0 2024-09-11T00:31:30.346 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:30.343+0000 7f88caa00640 1 -- 172.21.15.12:0/583064636 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=106) ==== 24+0+0 (secure 0 0 0) 0x7f88d005e050 con 0x557b45e3f8d0 2024-09-11T00:31:30.346 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:30.343+0000 7f88eee5e740 10 client.18105 fetch_fsmap learned FSMap version 106 2024-09-11T00:31:30.346 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:30.343+0000 7f88eee5e740 1 -- 172.21.15.12:0/583064636 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=106,monmap=4+}) -- 0x557b45ed8dd0 con 0x557b45e3f8d0 2024-09-11T00:31:30.346 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:30.344+0000 7f88caa00640 1 -- 172.21.15.12:0/583064636 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f88d80b0100 con 0x557b45e3f8d0 2024-09-11T00:31:30.346 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:30.344+0000 7f88caa00640 1 -- 172.21.15.12:0/583064636 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 106) ==== 8382+0+0 (secure 0 0 0) 0x7f88d00039c0 con 0x557b45e3f8d0 2024-09-11T00:31:30.346 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:30.344+0000 7f88eee5e740 10 client.18105 fetch_fsmap finished waiting for FSMap version 106 2024-09-11T00:31:30.346 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:30.344+0000 7f88eee5e740 10 client.18105 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T00:31:30.346 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:30.344+0000 7f88eee5e740 20 client.18105 populate_metadata read hostname 'smithi012' 2024-09-11T00:31:30.346 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:30.344+0000 7f88eee5e740 1 --2- 172.21.15.12:0/583064636 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x557b45ed9790 0x557b45edbbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:31:30.347 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:30.344+0000 7f88eee5e740 4 client.18105 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T00:31:30.347 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:30.344+0000 7f88eee5e740 1 -- 172.21.15.12:0/583064636 --> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] -- command(tid 0: {"prefix": "flush journal"}) -- 0x557b45da50b0 con 0x557b45ed9790 2024-09-11T00:31:30.347 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:30.345+0000 7f88e9600640 1 --2- 172.21.15.12:0/583064636 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x557b45ed9790 0x557b45edbbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:31:30.347 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:30.346+0000 7f88caa00640 10 client.18105 ms_handle_connect on v2:172.21.15.165:6838/501312984 2024-09-11T00:31:30.347 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:30.346+0000 7f88e9600640 1 --2- 172.21.15.12:0/583064636 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x557b45ed9790 0x557b45edbbb0 crc :-1 s=READY pgs=334 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e227208de4fb8b83 server_cookie=5591d451dbceb413 in_seq=0 out_seq=0 2024-09-11T00:31:30.532 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:31:30 smithi165 ceph-mon[35356]: pgmap v1132: 129 pgs: 129 active+clean; 852 MiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 3.8 MiB/s wr, 264 op/s 2024-09-11T00:31:30.565 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:31:30 smithi012 ceph-mon[30333]: pgmap v1132: 129 pgs: 129 active+clean; 852 MiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 3.8 MiB/s wr, 264 op/s 2024-09-11T00:31:30.581 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:31:30 smithi178 ceph-mon[35919]: pgmap v1132: 129 pgs: 129 active+clean; 852 MiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 3.8 MiB/s wr, 264 op/s 2024-09-11T00:31:30.926 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:30.925+0000 7f88caa00640 1 -- 172.21.15.12:0/583064636 <== mds.0 v2:172.21.15.165:6838/501312984 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x557b45da50b0 con 0x557b45ed9790 2024-09-11T00:31:30.926 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:30.925+0000 7f88caa00640 10 client.18105 handle_command_reply: tid=0 2024-09-11T00:31:30.926 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:30.925+0000 7f88eee5e740 1 client.18105 shutdown 2024-09-11T00:31:30.926 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:30.925+0000 7f88eee5e740 1 -- 172.21.15.12:0/583064636 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557b45e3f8d0 msgr2=0x557b45e49d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:31:30.927 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:30.925+0000 7f88eee5e740 1 --2- 172.21.15.12:0/583064636 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557b45e3f8d0 0x557b45e49d80 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f88d002a9e0 tx=0x7f88d003d000 comp rx=0 tx=0).stop 2024-09-11T00:31:30.927 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:30.925+0000 7f88eee5e740 1 -- 172.21.15.12:0/583064636 shutdown_connections 2024-09-11T00:31:30.927 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:30.925+0000 7f88eee5e740 1 --2- 172.21.15.12:0/583064636 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x557b45ed9790 0x557b45edbbb0 crc :-1 s=READY pgs=334 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:30.927 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:30.925+0000 7f88eee5e740 1 --2- 172.21.15.12:0/583064636 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x557b45e47a50 0x557b45e4a2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:30.927 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:30.925+0000 7f88caa00640 0 client.18105 ms_handle_reset on v2:172.21.15.165:6838/501312984 2024-09-11T00:31:30.927 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:30.925+0000 7f88eee5e740 1 --2- 172.21.15.12:0/583064636 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557b45e3f8d0 0x557b45e49d80 secure :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f88d002a9e0 tx=0x7f88d003d000 comp rx=0 tx=0).stop 2024-09-11T00:31:30.928 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:30.925+0000 7f88eee5e740 1 --2- 172.21.15.12:0/583064636 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557b45e3cfa0 0x557b45e4dfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:30.928 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:30.925+0000 7f88eee5e740 1 -- 172.21.15.12:0/583064636 >> 172.21.15.12:0/583064636 conn(0x557b45e380f0 msgr2=0x557b45e38950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:31:30.928 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:30.926+0000 7f88eee5e740 1 -- 172.21.15.12:0/583064636 shutdown_connections 2024-09-11T00:31:30.928 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:30.926+0000 7f88eee5e740 1 -- 172.21.15.12:0/583064636 wait complete. 2024-09-11T00:31:30.928 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:30.926+0000 7f88eee5e740 20 client.18105 trim_cache size 0 max 16384 2024-09-11T00:31:30.928 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:{ 2024-09-11T00:31:30.928 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "message": "", 2024-09-11T00:31:30.929 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "return_code": 0 2024-09-11T00:31:30.929 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:} 2024-09-11T00:31:30.929 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:30.927+0000 7f88ec000640 1 -- 172.21.15.12:0/3530245837 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f88cc0b53e0 msgr2=0x7f88cc0b78a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:31:30.929 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:30.927+0000 7f88ec000640 1 --2- 172.21.15.12:0/3530245837 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f88cc0b53e0 0x7f88cc0b78a0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x557b45e44d30 tx=0x7f88d00314a0 comp rx=0 tx=0).stop 2024-09-11T00:31:30.929 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:30.927+0000 7f88ec000640 1 -- 172.21.15.12:0/3530245837 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f88e4075ac0 msgr2=0x7f88e419af80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:31:30.930 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:30.927+0000 7f88ec000640 1 --2- 172.21.15.12:0/3530245837 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f88e4075ac0 0x7f88e419af80 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f88d8005fd0 tx=0x7f88d800dde0 comp rx=0 tx=0).stop 2024-09-11T00:31:30.930 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:30.927+0000 7f88ec000640 1 -- 172.21.15.12:0/3530245837 shutdown_connections 2024-09-11T00:31:30.930 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:30.927+0000 7f88ec000640 1 --2- 172.21.15.12:0/3530245837 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f88cc0b53e0 0x7f88cc0b78a0 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:30.930 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:30.927+0000 7f88ec000640 1 --2- 172.21.15.12:0/3530245837 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f88e410a910 0x7f88e419c200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:30.930 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:30.927+0000 7f88ec000640 1 --2- 172.21.15.12:0/3530245837 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f88e4076f30 0x7f88e419b5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:30.930 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:30.927+0000 7f88ec000640 1 --2- 172.21.15.12:0/3530245837 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f88e4075ac0 0x7f88e419af80 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:30.930 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:30.927+0000 7f88ec000640 1 -- 172.21.15.12:0/3530245837 >> 172.21.15.12:0/3530245837 conn(0x7f88e40fe250 msgr2=0x7f88e40fe870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:31:30.930 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:30.928+0000 7f88ec000640 1 -- 172.21.15.12:0/3530245837 shutdown_connections 2024-09-11T00:31:30.930 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:30.928+0000 7f88ec000640 1 -- 172.21.15.12:0/3530245837 wait complete. 2024-09-11T00:31:31.297 INFO:tasks.workunit.client.0.smithi012.stdout: 10 194795 15.38 MB/sec execute 567 sec latency 394.918 ms 2024-09-11T00:31:31.532 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:31:31 smithi165 ceph-mon[35356]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T00:31:31.563 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:31:31 smithi012 ceph-mon[30333]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T00:31:31.581 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:31:31 smithi178 ceph-mon[35919]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T00:31:32.298 INFO:tasks.workunit.client.0.smithi012.stdout: 10 195249 15.39 MB/sec execute 568 sec latency 110.404 ms 2024-09-11T00:31:32.532 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:31:32 smithi165 ceph-mon[35356]: pgmap v1133: 129 pgs: 129 active+clean; 848 MiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 4.6 MiB/s wr, 391 op/s 2024-09-11T00:31:32.563 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:31:32 smithi012 ceph-mon[30333]: pgmap v1133: 129 pgs: 129 active+clean; 848 MiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 4.6 MiB/s wr, 391 op/s 2024-09-11T00:31:32.581 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:31:32 smithi178 ceph-mon[35919]: pgmap v1133: 129 pgs: 129 active+clean; 848 MiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 4.6 MiB/s wr, 391 op/s 2024-09-11T00:31:33.297 INFO:tasks.workunit.client.0.smithi012.stdout: 10 196030 15.43 MB/sec execute 569 sec latency 93.119 ms 2024-09-11T00:31:34.297 INFO:tasks.workunit.client.0.smithi012.stdout: 10 196728 15.46 MB/sec execute 570 sec latency 60.317 ms 2024-09-11T00:31:34.532 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:31:34 smithi165 ceph-mon[35356]: pgmap v1134: 129 pgs: 129 active+clean; 846 MiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 MiB/s rd, 5.3 MiB/s wr, 525 op/s 2024-09-11T00:31:34.563 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:31:34 smithi012 ceph-mon[30333]: pgmap v1134: 129 pgs: 129 active+clean; 846 MiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 MiB/s rd, 5.3 MiB/s wr, 525 op/s 2024-09-11T00:31:34.581 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:31:34 smithi178 ceph-mon[35919]: pgmap v1134: 129 pgs: 129 active+clean; 846 MiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 MiB/s rd, 5.3 MiB/s wr, 525 op/s 2024-09-11T00:31:35.297 INFO:tasks.workunit.client.0.smithi012.stdout: 10 197622 15.53 MB/sec execute 571 sec latency 55.368 ms 2024-09-11T00:31:36.297 INFO:tasks.workunit.client.0.smithi012.stdout: 10 198332 15.58 MB/sec execute 572 sec latency 64.380 ms 2024-09-11T00:31:36.454 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:31:36 smithi012 ceph-mon[30333]: pgmap v1135: 129 pgs: 129 active+clean; 848 MiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 MiB/s rd, 5.5 MiB/s wr, 572 op/s 2024-09-11T00:31:36.532 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:31:36 smithi165 ceph-mon[35356]: pgmap v1135: 129 pgs: 129 active+clean; 848 MiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 MiB/s rd, 5.5 MiB/s wr, 572 op/s 2024-09-11T00:31:36.581 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:31:36 smithi178 ceph-mon[35919]: pgmap v1135: 129 pgs: 129 active+clean; 848 MiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 MiB/s rd, 5.5 MiB/s wr, 572 op/s 2024-09-11T00:31:37.298 INFO:tasks.workunit.client.0.smithi012.stdout: 10 198738 15.59 MB/sec execute 573 sec latency 110.531 ms 2024-09-11T00:31:38.297 INFO:tasks.workunit.client.0.smithi012.stdout: 10 199286 15.61 MB/sec execute 574 sec latency 75.561 ms 2024-09-11T00:31:38.516 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:31:38 smithi012 ceph-mon[30333]: pgmap v1136: 129 pgs: 129 active+clean; 859 MiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 6.4 MiB/s wr, 799 op/s 2024-09-11T00:31:38.532 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:31:38 smithi165 ceph-mon[35356]: pgmap v1136: 129 pgs: 129 active+clean; 859 MiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 6.4 MiB/s wr, 799 op/s 2024-09-11T00:31:38.581 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:31:38 smithi178 ceph-mon[35919]: pgmap v1136: 129 pgs: 129 active+clean; 859 MiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 6.4 MiB/s wr, 799 op/s 2024-09-11T00:31:39.298 INFO:tasks.workunit.client.0.smithi012.stdout: 10 199829 15.62 MB/sec execute 575 sec latency 77.724 ms 2024-09-11T00:31:40.298 INFO:tasks.workunit.client.0.smithi012.stdout: 10 200357 15.64 MB/sec execute 576 sec latency 370.004 ms 2024-09-11T00:31:40.532 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:31:40 smithi165 ceph-mon[35356]: pgmap v1137: 129 pgs: 129 active+clean; 861 MiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 MiB/s rd, 3.5 MiB/s wr, 613 op/s 2024-09-11T00:31:40.563 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:31:40 smithi012 ceph-mon[30333]: pgmap v1137: 129 pgs: 129 active+clean; 861 MiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 MiB/s rd, 3.5 MiB/s wr, 613 op/s 2024-09-11T00:31:40.581 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:31:40 smithi178 ceph-mon[35919]: pgmap v1137: 129 pgs: 129 active+clean; 861 MiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 MiB/s rd, 3.5 MiB/s wr, 613 op/s 2024-09-11T00:31:41.298 INFO:tasks.workunit.client.0.smithi012.stdout: 10 201308 15.73 MB/sec execute 577 sec latency 127.039 ms 2024-09-11T00:31:42.298 INFO:tasks.workunit.client.0.smithi012.stdout: 10 201898 15.75 MB/sec execute 578 sec latency 73.940 ms 2024-09-11T00:31:42.532 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:31:42 smithi165 ceph-mon[35356]: pgmap v1138: 129 pgs: 129 active+clean; 870 MiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 4.2 MiB/s wr, 746 op/s 2024-09-11T00:31:42.563 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:31:42 smithi012 ceph-mon[30333]: pgmap v1138: 129 pgs: 129 active+clean; 870 MiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 4.2 MiB/s wr, 746 op/s 2024-09-11T00:31:42.581 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:31:42 smithi178 ceph-mon[35919]: pgmap v1138: 129 pgs: 129 active+clean; 870 MiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 4.2 MiB/s wr, 746 op/s 2024-09-11T00:31:43.298 INFO:tasks.workunit.client.0.smithi012.stdout: 10 202256 15.75 MB/sec execute 579 sec latency 119.385 ms 2024-09-11T00:31:44.148 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:44.146+0000 7f4a8cc00640 1 -- 172.21.15.12:0/2622958281 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4a8806b360 msgr2=0x7f4a88107760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:31:44.148 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:44.146+0000 7f4a8cc00640 1 --2- 172.21.15.12:0/2622958281 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4a8806b360 0x7f4a88107760 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f4a78009900 tx=0x7f4a7802a410 comp rx=0 tx=0).stop 2024-09-11T00:31:44.148 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:44.147+0000 7f4a8cc00640 1 -- 172.21.15.12:0/2622958281 shutdown_connections 2024-09-11T00:31:44.149 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:44.147+0000 7f4a8cc00640 1 --2- 172.21.15.12:0/2622958281 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4a8810a6e0 0x7f4a8810cb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:44.149 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:44.147+0000 7f4a8cc00640 1 --2- 172.21.15.12:0/2622958281 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4a88107ca0 0x7f4a8810a090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:44.149 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:44.147+0000 7f4a8cc00640 1 --2- 172.21.15.12:0/2622958281 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4a8806b360 0x7f4a88107760 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:44.149 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:44.147+0000 7f4a8cc00640 1 -- 172.21.15.12:0/2622958281 >> 172.21.15.12:0/2622958281 conn(0x7f4a880fcf30 msgr2=0x7f4a880ff370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:31:44.149 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:44.147+0000 7f4a8cc00640 1 -- 172.21.15.12:0/2622958281 shutdown_connections 2024-09-11T00:31:44.149 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:44.147+0000 7f4a8cc00640 1 -- 172.21.15.12:0/2622958281 wait complete. 2024-09-11T00:31:44.149 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:44.148+0000 7f4a8cc00640 1 Processor -- start 2024-09-11T00:31:44.150 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:44.149+0000 7f4a8cc00640 1 -- start start 2024-09-11T00:31:44.151 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:44.149+0000 7f4a8cc00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4a8806b360 0x7f4a88103770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:31:44.151 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:44.149+0000 7f4a8cc00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4a88107ca0 0x7f4a88103dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:31:44.151 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:44.149+0000 7f4a8cc00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4a8810a6e0 0x7f4a881049f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:31:44.151 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:44.149+0000 7f4a8cc00640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f4a88110700 con 0x7f4a8810a6e0 2024-09-11T00:31:44.151 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:44.149+0000 7f4a8cc00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f4a88110580 con 0x7f4a88107ca0 2024-09-11T00:31:44.151 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:44.149+0000 7f4a8cc00640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f4a88110880 con 0x7f4a8806b360 2024-09-11T00:31:44.151 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:44.149+0000 7f4a86a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4a88107ca0 0x7f4a88103dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:31:44.151 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:44.149+0000 7f4a86a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4a88107ca0 0x7f4a88103dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.12:40534/0 (socket says 172.21.15.12:40534) 2024-09-11T00:31:44.151 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:44.149+0000 7f4a86a00640 1 -- 172.21.15.12:0/946312275 learned_addr learned my addr 172.21.15.12:0/946312275 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:31:44.151 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:44.150+0000 7f4a87e00640 1 --2- 172.21.15.12:0/946312275 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4a8810a6e0 0x7f4a881049f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:31:44.152 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:44.150+0000 7f4a87400640 1 --2- 172.21.15.12:0/946312275 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4a8806b360 0x7f4a88103770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:31:44.152 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:44.150+0000 7f4a86a00640 1 -- 172.21.15.12:0/946312275 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4a8806b360 msgr2=0x7f4a88103770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:31:44.153 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:44.150+0000 7f4a86a00640 1 --2- 172.21.15.12:0/946312275 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4a8806b360 0x7f4a88103770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:44.153 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:44.150+0000 7f4a86a00640 1 -- 172.21.15.12:0/946312275 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4a8810a6e0 msgr2=0x7f4a881049f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:31:44.153 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:44.150+0000 7f4a86a00640 1 --2- 172.21.15.12:0/946312275 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4a8810a6e0 0x7f4a881049f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:44.153 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:44.150+0000 7f4a86a00640 1 -- 172.21.15.12:0/946312275 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a881052b0 con 0x7f4a88107ca0 2024-09-11T00:31:44.153 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:44.150+0000 7f4a87400640 1 --2- 172.21.15.12:0/946312275 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4a8806b360 0x7f4a88103770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T00:31:44.153 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:44.150+0000 7f4a86a00640 1 --2- 172.21.15.12:0/946312275 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4a88107ca0 0x7f4a88103dc0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f4a7400b450 tx=0x7f4a7400b910 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:31:44.153 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:44.151+0000 7f4a7fe00640 1 -- 172.21.15.12:0/946312275 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a74015020 con 0x7f4a88107ca0 2024-09-11T00:31:44.153 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:44.151+0000 7f4a8cc00640 1 -- 172.21.15.12:0/946312275 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4a881055a0 con 0x7f4a88107ca0 2024-09-11T00:31:44.153 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:44.151+0000 7f4a7fe00640 1 -- 172.21.15.12:0/946312275 <== mon.2 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4a74011d40 con 0x7f4a88107ca0 2024-09-11T00:31:44.153 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:44.151+0000 7f4a7fe00640 1 -- 172.21.15.12:0/946312275 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a7400f340 con 0x7f4a88107ca0 2024-09-11T00:31:44.154 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:44.152+0000 7f4a8cc00640 1 -- 172.21.15.12:0/946312275 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4a88105cc0 con 0x7f4a88107ca0 2024-09-11T00:31:44.156 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:44.155+0000 7f4a911cc740 1 Processor -- start 2024-09-11T00:31:44.156 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:44.155+0000 7f4a911cc740 1 -- start start 2024-09-11T00:31:44.157 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:44.156+0000 7f4a911cc740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562c7b340970 0x562c7b342d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:31:44.157 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:44.156+0000 7f4a911cc740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562c7b3432a0 0x562c7b345690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:31:44.157 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:44.156+0000 7f4a911cc740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562c7b345ce0 0x562c7b3481a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:31:44.158 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:44.156+0000 7f4a911cc740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x562c7b2b36b0 con 0x562c7b340970 2024-09-11T00:31:44.158 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:44.156+0000 7f4a911cc740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x562c7b2a2ae0 con 0x562c7b345ce0 2024-09-11T00:31:44.158 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:44.156+0000 7f4a911cc740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x562c7b2a68f0 con 0x562c7b3432a0 2024-09-11T00:31:44.158 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:44.156+0000 7f4a7fe00640 1 -- 172.21.15.12:0/946312275 <== mon.2 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7f4a74010500 con 0x7f4a88107ca0 2024-09-11T00:31:44.158 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:44.156+0000 7f4a87e00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562c7b340970 0x562c7b342d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:31:44.158 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:44.156+0000 7f4a87400640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562c7b3432a0 0x562c7b345690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:31:44.158 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:44.156+0000 7f4a87400640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562c7b3432a0 0x562c7b345690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.12:54036/0 (socket says 172.21.15.12:54036) 2024-09-11T00:31:44.158 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:44.156+0000 7f4a87400640 1 -- 172.21.15.12:0/731738559 learned_addr learned my addr 172.21.15.12:0/731738559 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:31:44.159 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:44.156+0000 7f4a86a00640 1 --2- 172.21.15.12:0/731738559 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562c7b345ce0 0x562c7b3481a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:31:44.159 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:44.157+0000 7f4a87400640 1 -- 172.21.15.12:0/731738559 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562c7b345ce0 msgr2=0x562c7b3481a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:31:44.159 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:44.157+0000 7f4a87400640 1 --2- 172.21.15.12:0/731738559 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562c7b345ce0 0x562c7b3481a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:44.159 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:44.157+0000 7f4a87400640 1 -- 172.21.15.12:0/731738559 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562c7b340970 msgr2=0x562c7b342d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:31:44.159 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:44.157+0000 7f4a87400640 1 --2- 172.21.15.12:0/731738559 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562c7b340970 0x562c7b342d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:44.159 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:44.157+0000 7f4a87400640 1 -- 172.21.15.12:0/731738559 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562c7b26f780 con 0x562c7b3432a0 2024-09-11T00:31:44.159 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:44.157+0000 7f4a87e00640 1 --2- 172.21.15.12:0/731738559 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562c7b340970 0x562c7b342d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:31:44.160 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:44.159+0000 7f4a87400640 1 --2- 172.21.15.12:0/731738559 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562c7b3432a0 0x562c7b345690 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f4a78006010 tx=0x7f4a78004150 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c00c94f373debcb9 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:31:44.161 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:44.160+0000 7f4a73e00640 1 -- 172.21.15.12:0/731738559 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a7803f070 con 0x562c7b3432a0 2024-09-11T00:31:44.161 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:44.160+0000 7f4a73e00640 1 -- 172.21.15.12:0/731738559 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4a780054f0 con 0x562c7b3432a0 2024-09-11T00:31:44.161 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:44.160+0000 7f4a73e00640 1 -- 172.21.15.12:0/731738559 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a78035030 con 0x562c7b3432a0 2024-09-11T00:31:44.163 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:44.160+0000 7f4a7fe00640 1 --2- 172.21.15.12:0/946312275 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f4a6c0b5360 0x7f4a6c0b7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:31:44.164 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:44.161+0000 7f4a7fe00640 1 -- 172.21.15.12:0/946312275 <== mon.2 v2:172.21.15.165:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9208+0+0 (secure 0 0 0) 0x7f4a740e5c00 con 0x7f4a88107ca0 2024-09-11T00:31:44.164 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:44.161+0000 7f4a911cc740 1 -- 172.21.15.12:0/731738559 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562c7b3432a0 msgr2=0x562c7b345690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:31:44.164 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:44.161+0000 7f4a911cc740 1 --2- 172.21.15.12:0/731738559 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562c7b3432a0 0x562c7b345690 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f4a78006010 tx=0x7f4a78004150 comp rx=0 tx=0).stop 2024-09-11T00:31:44.164 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:44.161+0000 7f4a87e00640 1 --2- 172.21.15.12:0/946312275 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f4a6c0b5360 0x7f4a6c0b7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:31:44.164 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:44.161+0000 7f4a911cc740 1 -- 172.21.15.12:0/731738559 shutdown_connections 2024-09-11T00:31:44.164 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:44.161+0000 7f4a911cc740 1 --2- 172.21.15.12:0/731738559 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562c7b345ce0 0x562c7b3481a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:44.164 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:44.161+0000 7f4a911cc740 1 --2- 172.21.15.12:0/731738559 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562c7b3432a0 0x562c7b345690 secure :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f4a78006010 tx=0x7f4a78004150 comp rx=0 tx=0).stop 2024-09-11T00:31:44.164 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:44.161+0000 7f4a911cc740 1 --2- 172.21.15.12:0/731738559 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562c7b340970 0x562c7b342d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:44.164 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:44.161+0000 7f4a911cc740 1 -- 172.21.15.12:0/731738559 >> 172.21.15.12:0/731738559 conn(0x562c7b33bac0 msgr2=0x562c7b33c850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:31:44.164 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:44.162+0000 7f4a87e00640 1 --2- 172.21.15.12:0/946312275 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f4a6c0b5360 0x7f4a6c0b7820 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f4a881044f0 tx=0x7f4a80009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:31:44.165 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:44.163+0000 7f4a911cc740 1 -- 172.21.15.12:0/731738559 shutdown_connections 2024-09-11T00:31:44.165 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:44.163+0000 7f4a911cc740 1 -- 172.21.15.12:0/731738559 wait complete. 2024-09-11T00:31:44.165 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:44.163+0000 7f4a911cc740 1 -- start start 2024-09-11T00:31:44.165 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:44.164+0000 7f4a911cc740 1 Processor -- start 2024-09-11T00:31:44.166 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:44.164+0000 7f4a911cc740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562c7b340970 0x562c7b34a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:31:44.166 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:44.164+0000 7f4a911cc740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562c7b345ce0 0x562c7b34a980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:31:44.166 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:44.164+0000 7f4a911cc740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562c7b34b530 0x562c7b34b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:31:44.166 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:44.164+0000 7f4a911cc740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x562c7b1daab0 con 0x562c7b345ce0 2024-09-11T00:31:44.166 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:44.164+0000 7f4a911cc740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x562c7b249570 con 0x562c7b340970 2024-09-11T00:31:44.167 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:44.164+0000 7f4a911cc740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x562c7b348c60 con 0x562c7b34b530 2024-09-11T00:31:44.167 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:44.165+0000 7f4a911cc740 10 client.0 _ll_register_callbacks cb 0x562c7b258100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:31:44.167 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:44.165+0000 7f4a86a00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562c7b345ce0 0x562c7b34a980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:31:44.167 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:44.165+0000 7f4a87e00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562c7b34b530 0x562c7b34b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:31:44.167 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:44.165+0000 7f4a87400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562c7b340970 0x562c7b34a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:31:44.167 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:44.165+0000 7f4a87e00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562c7b34b530 0x562c7b34b9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.12:54040/0 (socket says 172.21.15.12:54040) 2024-09-11T00:31:44.167 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:44.165+0000 7f4a87e00640 1 -- 172.21.15.12:0/171092417 learned_addr learned my addr 172.21.15.12:0/171092417 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:31:44.168 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:44.165+0000 7f4a87e00640 1 -- 172.21.15.12:0/171092417 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562c7b340970 msgr2=0x562c7b34a2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:31:44.168 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:44.165+0000 7f4a87e00640 1 --2- 172.21.15.12:0/171092417 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562c7b340970 0x562c7b34a2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:44.168 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:44.166+0000 7f4a87e00640 1 -- 172.21.15.12:0/171092417 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562c7b345ce0 msgr2=0x562c7b34a980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:31:44.168 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:44.166+0000 7f4a87e00640 1 --2- 172.21.15.12:0/171092417 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562c7b345ce0 0x562c7b34a980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:44.168 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:44.166+0000 7f4a87e00640 1 -- 172.21.15.12:0/171092417 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562c7b004790 con 0x562c7b34b530 2024-09-11T00:31:44.168 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:44.166+0000 7f4a87e00640 1 --2- 172.21.15.12:0/171092417 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562c7b34b530 0x562c7b34b9b0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f4a8000f5c0 tx=0x7f4a80004ab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f9d212fe36758b75 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:31:44.168 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:44.166+0000 7f4a70c00640 10 client.0 ms_handle_connect on v2:172.21.15.178:3300/0 2024-09-11T00:31:44.168 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:44.166+0000 7f4a70c00640 1 -- 172.21.15.12:0/171092417 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a80003b80 con 0x562c7b34b530 2024-09-11T00:31:44.168 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:44.166+0000 7f4a70c00640 1 -- 172.21.15.12:0/171092417 <== mon.1 v2:172.21.15.178:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 9208+0+0 (secure 0 0 0) 0x7f4a80003d20 con 0x562c7b34b530 2024-09-11T00:31:44.169 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:44.167+0000 7f4a911cc740 1 -- 172.21.15.12:0/171092417 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4a780054f0 con 0x562c7b34b530 2024-09-11T00:31:44.169 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:44.167+0000 7f4a70c00640 1 -- 172.21.15.12:0/171092417 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=106) ==== 24+0+0 (secure 0 0 0) 0x7f4a80004050 con 0x562c7b34b530 2024-09-11T00:31:44.169 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:44.167+0000 7f4a911cc740 10 client.28108 fetch_fsmap learned FSMap version 106 2024-09-11T00:31:44.169 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:44.167+0000 7f4a911cc740 1 -- 172.21.15.12:0/171092417 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,fsmap=106,monmap=4+}) -- 0x562c7b07d620 con 0x562c7b34b530 2024-09-11T00:31:44.169 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:44.167+0000 7f4a70c00640 1 -- 172.21.15.12:0/171092417 <== mon.1 v2:172.21.15.178:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4a780054f0 con 0x562c7b34b530 2024-09-11T00:31:44.169 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:44.167+0000 7f4a70c00640 1 -- 172.21.15.12:0/171092417 <== mon.1 v2:172.21.15.178:3300/0 5 ==== fsmap(e 106) ==== 8382+0+0 (secure 0 0 0) 0x7f4a80026070 con 0x562c7b34b530 2024-09-11T00:31:44.170 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:44.168+0000 7f4a911cc740 10 client.28108 fetch_fsmap finished waiting for FSMap version 106 2024-09-11T00:31:44.170 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:44.168+0000 7f4a911cc740 10 client.28108 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T00:31:44.170 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:44.168+0000 7f4a911cc740 20 client.28108 populate_metadata read hostname 'smithi012' 2024-09-11T00:31:44.170 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:44.168+0000 7f4a911cc740 1 --2- 172.21.15.12:0/171092417 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x562c7b3560a0 0x562c7b3584b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:31:44.170 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:44.168+0000 7f4a911cc740 4 client.28108 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T00:31:44.170 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:44.168+0000 7f4a911cc740 1 -- 172.21.15.12:0/171092417 --> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562c7b2b5600 con 0x562c7b3560a0 2024-09-11T00:31:44.171 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:44.168+0000 7f4a87400640 1 --2- 172.21.15.12:0/171092417 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x562c7b3560a0 0x562c7b3584b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:31:44.171 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:44.169+0000 7f4a70c00640 10 client.28108 ms_handle_connect on v2:172.21.15.165:6838/501312984 2024-09-11T00:31:44.171 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:44.169+0000 7f4a87400640 1 --2- 172.21.15.12:0/171092417 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x562c7b3560a0 0x562c7b3584b0 crc :-1 s=READY pgs=335 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=488c9b2c6c0df161 server_cookie=8bf99479ad2172be in_seq=0 out_seq=0 2024-09-11T00:31:44.173 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:44.172+0000 7f4a70c00640 1 -- 172.21.15.12:0/171092417 <== mds.0 v2:172.21.15.165:6838/501312984 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x562c7b2b5600 con 0x562c7b3560a0 2024-09-11T00:31:44.173 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:44.172+0000 7f4a70c00640 10 client.28108 handle_command_reply: tid=0 2024-09-11T00:31:44.173 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:44.172+0000 7f4a911cc740 1 client.28108 shutdown 2024-09-11T00:31:44.174 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:44.173+0000 7f4a911cc740 1 -- 172.21.15.12:0/171092417 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562c7b34b530 msgr2=0x562c7b34b9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:31:44.174 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:44.173+0000 7f4a911cc740 1 --2- 172.21.15.12:0/171092417 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562c7b34b530 0x562c7b34b9b0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f4a8000f5c0 tx=0x7f4a80004ab0 comp rx=0 tx=0).stop 2024-09-11T00:31:44.174 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:44.173+0000 7f4a911cc740 1 -- 172.21.15.12:0/171092417 shutdown_connections 2024-09-11T00:31:44.175 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:44.173+0000 7f4a911cc740 1 --2- 172.21.15.12:0/171092417 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x562c7b3560a0 0x562c7b3584b0 crc :-1 s=READY pgs=335 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:44.175 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:44.173+0000 7f4a911cc740 1 --2- 172.21.15.12:0/171092417 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562c7b34b530 0x562c7b34b9b0 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:44.175 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:44.173+0000 7f4a911cc740 1 --2- 172.21.15.12:0/171092417 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562c7b345ce0 0x562c7b34a980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:44.175 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:44.173+0000 7f4a911cc740 1 --2- 172.21.15.12:0/171092417 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562c7b340970 0x562c7b34a2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:44.175 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:44.173+0000 7f4a70c00640 0 client.28108 ms_handle_reset on v2:172.21.15.165:6838/501312984 2024-09-11T00:31:44.175 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:44.173+0000 7f4a911cc740 1 -- 172.21.15.12:0/171092417 >> 172.21.15.12:0/171092417 conn(0x562c7b33bac0 msgr2=0x562c7b33abc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:31:44.175 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:44.173+0000 7f4a911cc740 1 -- 172.21.15.12:0/171092417 shutdown_connections 2024-09-11T00:31:44.175 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:44.173+0000 7f4a911cc740 1 -- 172.21.15.12:0/171092417 wait complete. 2024-09-11T00:31:44.175 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:44.174+0000 7f4a911cc740 20 client.28108 trim_cache size 0 max 16384 2024-09-11T00:31:44.207 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:44.205+0000 7f4a911cc740 1 Processor -- start 2024-09-11T00:31:44.207 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:44.206+0000 7f4a911cc740 1 -- start start 2024-09-11T00:31:44.208 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:44.207+0000 7f4a911cc740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562c7b340970 0x562c7b33cb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:31:44.208 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:44.207+0000 7f4a911cc740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562c7b345ce0 0x562c7b346730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:31:44.208 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:44.207+0000 7f4a911cc740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562c7b34b530 0x562c7b33db80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:31:44.208 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:44.207+0000 7f4a911cc740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f4a881b1d80 con 0x562c7b345ce0 2024-09-11T00:31:44.209 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:44.207+0000 7f4a911cc740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f4a881b38c0 con 0x562c7b340970 2024-09-11T00:31:44.209 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:44.207+0000 7f4a911cc740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f4a881b2450 con 0x562c7b34b530 2024-09-11T00:31:44.209 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:44.207+0000 7f4a87400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562c7b340970 0x562c7b33cb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:31:44.209 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:44.207+0000 7f4a87400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562c7b340970 0x562c7b33cb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.12:40564/0 (socket says 172.21.15.12:40564) 2024-09-11T00:31:44.209 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:44.207+0000 7f4a87400640 1 -- 172.21.15.12:0/2778709566 learned_addr learned my addr 172.21.15.12:0/2778709566 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:31:44.210 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:44.207+0000 7f4a87e00640 1 --2- 172.21.15.12:0/2778709566 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562c7b34b530 0x562c7b33db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:31:44.210 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:44.208+0000 7f4a86a00640 1 --2- 172.21.15.12:0/2778709566 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562c7b345ce0 0x562c7b346730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:31:44.210 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:44.208+0000 7f4a87400640 1 -- 172.21.15.12:0/2778709566 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562c7b34b530 msgr2=0x562c7b33db80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:31:44.210 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:44.208+0000 7f4a87400640 1 --2- 172.21.15.12:0/2778709566 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562c7b34b530 0x562c7b33db80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:44.210 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:44.208+0000 7f4a87400640 1 -- 172.21.15.12:0/2778709566 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562c7b345ce0 msgr2=0x562c7b346730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:31:44.210 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:44.208+0000 7f4a87400640 1 --2- 172.21.15.12:0/2778709566 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562c7b345ce0 0x562c7b346730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:44.210 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:44.208+0000 7f4a87400640 1 -- 172.21.15.12:0/2778709566 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562c7b34acf0 con 0x562c7b340970 2024-09-11T00:31:44.211 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:44.208+0000 7f4a87400640 1 --2- 172.21.15.12:0/2778709566 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562c7b340970 0x562c7b33cb40 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f4a78005ab0 tx=0x7f4a780040d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=72f879dae230087b server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:31:44.211 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:44.210+0000 7f4a73400640 1 -- 172.21.15.12:0/2778709566 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a78053070 con 0x562c7b340970 2024-09-11T00:31:44.211 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:44.210+0000 7f4a73400640 1 -- 172.21.15.12:0/2778709566 <== mon.2 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4a78003ca0 con 0x562c7b340970 2024-09-11T00:31:44.211 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:44.210+0000 7f4a73400640 1 -- 172.21.15.12:0/2778709566 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a78000e60 con 0x562c7b340970 2024-09-11T00:31:44.212 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:44.211+0000 7f4a911cc740 1 -- 172.21.15.12:0/2778709566 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562c7b340970 msgr2=0x562c7b33cb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:31:44.212 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:44.211+0000 7f4a911cc740 1 --2- 172.21.15.12:0/2778709566 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562c7b340970 0x562c7b33cb40 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f4a78005ab0 tx=0x7f4a780040d0 comp rx=0 tx=0).stop 2024-09-11T00:31:44.213 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:44.212+0000 7f4a911cc740 1 -- 172.21.15.12:0/2778709566 shutdown_connections 2024-09-11T00:31:44.214 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:44.212+0000 7f4a911cc740 1 --2- 172.21.15.12:0/2778709566 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562c7b34b530 0x562c7b33db80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:44.214 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:44.212+0000 7f4a911cc740 1 --2- 172.21.15.12:0/2778709566 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562c7b345ce0 0x562c7b346730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:44.214 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:44.212+0000 7f4a911cc740 1 --2- 172.21.15.12:0/2778709566 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562c7b340970 0x562c7b33cb40 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:44.214 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:44.212+0000 7f4a911cc740 1 -- 172.21.15.12:0/2778709566 >> 172.21.15.12:0/2778709566 conn(0x562c7b33bac0 msgr2=0x562c7b33a3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:31:44.214 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:44.212+0000 7f4a911cc740 1 -- 172.21.15.12:0/2778709566 shutdown_connections 2024-09-11T00:31:44.214 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:44.212+0000 7f4a911cc740 1 -- 172.21.15.12:0/2778709566 wait complete. 2024-09-11T00:31:44.214 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:44.212+0000 7f4a911cc740 1 -- start start 2024-09-11T00:31:44.214 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:44.212+0000 7f4a911cc740 1 Processor -- start 2024-09-11T00:31:44.215 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:44.213+0000 7f4a911cc740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562c7b340970 0x562c7b352090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:31:44.215 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:44.213+0000 7f4a911cc740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562c7b345ce0 0x562c7b3508c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:31:44.215 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:44.213+0000 7f4a911cc740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562c7b34b530 0x562c7b350e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:31:44.215 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:44.213+0000 7f4a911cc740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f4a881b31f0 con 0x562c7b345ce0 2024-09-11T00:31:44.215 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:44.213+0000 7f4a911cc740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f4a881b2b20 con 0x562c7b340970 2024-09-11T00:31:44.215 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:44.213+0000 7f4a911cc740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f4a881afd60 con 0x562c7b34b530 2024-09-11T00:31:44.215 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:44.213+0000 7f4a911cc740 10 client.0 _ll_register_callbacks cb 0x7f4a80009d00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:31:44.215 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:44.214+0000 7f4a86a00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562c7b345ce0 0x562c7b3508c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:31:44.216 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:44.214+0000 7f4a86a00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562c7b345ce0 0x562c7b3508c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:43016/0 (socket says 172.21.15.12:43016) 2024-09-11T00:31:44.216 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:44.214+0000 7f4a86a00640 1 -- 172.21.15.12:0/2894373576 learned_addr learned my addr 172.21.15.12:0/2894373576 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:31:44.216 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:44.215+0000 7f4a87e00640 1 --2- 172.21.15.12:0/2894373576 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562c7b34b530 0x562c7b350e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:31:44.216 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:44.215+0000 7f4a86a00640 1 -- 172.21.15.12:0/2894373576 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562c7b34b530 msgr2=0x562c7b350e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:31:44.216 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:44.215+0000 7f4a86a00640 1 --2- 172.21.15.12:0/2894373576 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562c7b34b530 0x562c7b350e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:44.216 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:44.215+0000 7f4a86a00640 1 -- 172.21.15.12:0/2894373576 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562c7b340970 msgr2=0x562c7b352090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:31:44.217 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:44.215+0000 7f4a86a00640 1 --2- 172.21.15.12:0/2894373576 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562c7b340970 0x562c7b352090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:44.217 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:44.215+0000 7f4a86a00640 1 -- 172.21.15.12:0/2894373576 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562c7b351570 con 0x562c7b345ce0 2024-09-11T00:31:44.217 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:44.215+0000 7f4a86a00640 1 --2- 172.21.15.12:0/2894373576 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562c7b345ce0 0x562c7b3508c0 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f4a740abf20 tx=0x7f4a740ab8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b65608adbb525485 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:31:44.217 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:44.215+0000 7f4a70c00640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-09-11T00:31:44.217 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:44.216+0000 7f4a70c00640 1 -- 172.21.15.12:0/2894373576 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a740ea040 con 0x562c7b345ce0 2024-09-11T00:31:44.218 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:44.216+0000 7f4a911cc740 1 -- 172.21.15.12:0/2894373576 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4a78003ca0 con 0x562c7b345ce0 2024-09-11T00:31:44.218 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:44.216+0000 7f4a70c00640 1 -- 172.21.15.12:0/2894373576 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 9208+0+0 (secure 0 0 0) 0x7f4a74011a90 con 0x562c7b345ce0 2024-09-11T00:31:44.218 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:44.217+0000 7f4a70c00640 1 -- 172.21.15.12:0/2894373576 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=106) ==== 24+0+0 (secure 0 0 0) 0x7f4a740ac690 con 0x562c7b345ce0 2024-09-11T00:31:44.219 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:44.217+0000 7f4a911cc740 10 client.18120 fetch_fsmap learned FSMap version 106 2024-09-11T00:31:44.219 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:44.217+0000 7f4a911cc740 1 -- 172.21.15.12:0/2894373576 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=106,monmap=4+}) -- 0x562c7b3c4030 con 0x562c7b345ce0 2024-09-11T00:31:44.219 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:44.218+0000 7f4a70c00640 1 -- 172.21.15.12:0/2894373576 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4a78003ca0 con 0x562c7b345ce0 2024-09-11T00:31:44.220 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:44.218+0000 7f4a70c00640 1 -- 172.21.15.12:0/2894373576 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 106) ==== 8382+0+0 (secure 0 0 0) 0x7f4a740a78e0 con 0x562c7b345ce0 2024-09-11T00:31:44.220 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:44.218+0000 7f4a911cc740 10 client.18120 fetch_fsmap finished waiting for FSMap version 106 2024-09-11T00:31:44.220 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:44.218+0000 7f4a911cc740 10 client.18120 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T00:31:44.220 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:44.219+0000 7f4a911cc740 20 client.18120 populate_metadata read hostname 'smithi012' 2024-09-11T00:31:44.220 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:44.219+0000 7f4a911cc740 1 --2- 172.21.15.12:0/2894373576 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x562c7b3c49b0 0x562c7b3c6dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:31:44.220 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:44.219+0000 7f4a911cc740 4 client.18120 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T00:31:44.220 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:44.219+0000 7f4a911cc740 1 -- 172.21.15.12:0/2894373576 --> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] -- command(tid 0: {"prefix": "flush journal"}) -- 0x562c7b2b10b0 con 0x562c7b3c49b0 2024-09-11T00:31:44.221 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:44.219+0000 7f4a87400640 1 --2- 172.21.15.12:0/2894373576 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x562c7b3c49b0 0x562c7b3c6dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:31:44.221 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:44.220+0000 7f4a87400640 1 --2- 172.21.15.12:0/2894373576 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x562c7b3c49b0 0x562c7b3c6dd0 crc :-1 s=READY pgs=336 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6ab5f4a3f4ca591c server_cookie=4571e5285b310364 in_seq=0 out_seq=0 2024-09-11T00:31:44.221 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:44.220+0000 7f4a70c00640 10 client.18120 ms_handle_connect on v2:172.21.15.165:6838/501312984 2024-09-11T00:31:44.298 INFO:tasks.workunit.client.0.smithi012.stdout: 10 202851 15.77 MB/sec execute 580 sec latency 101.850 ms 2024-09-11T00:31:44.532 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:31:44 smithi165 ceph-mon[35356]: pgmap v1139: 129 pgs: 129 active+clean; 872 MiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 MiB/s rd, 3.7 MiB/s wr, 665 op/s 2024-09-11T00:31:44.564 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:31:44 smithi012 ceph-mon[30333]: pgmap v1139: 129 pgs: 129 active+clean; 872 MiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 MiB/s rd, 3.7 MiB/s wr, 665 op/s 2024-09-11T00:31:44.581 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:31:44 smithi178 ceph-mon[35919]: pgmap v1139: 129 pgs: 129 active+clean; 872 MiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 MiB/s rd, 3.7 MiB/s wr, 665 op/s 2024-09-11T00:31:44.774 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:44.773+0000 7f4a70c00640 1 -- 172.21.15.12:0/2894373576 <== mds.0 v2:172.21.15.165:6838/501312984 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x562c7b2b10b0 con 0x562c7b3c49b0 2024-09-11T00:31:44.774 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:44.773+0000 7f4a70c00640 10 client.18120 handle_command_reply: tid=0 2024-09-11T00:31:44.774 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:44.773+0000 7f4a911cc740 1 client.18120 shutdown 2024-09-11T00:31:44.775 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:44.773+0000 7f4a911cc740 1 -- 172.21.15.12:0/2894373576 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562c7b345ce0 msgr2=0x562c7b3508c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:31:44.775 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:44.773+0000 7f4a911cc740 1 --2- 172.21.15.12:0/2894373576 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562c7b345ce0 0x562c7b3508c0 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f4a740abf20 tx=0x7f4a740ab8e0 comp rx=0 tx=0).stop 2024-09-11T00:31:44.775 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:44.774+0000 7f4a911cc740 1 -- 172.21.15.12:0/2894373576 shutdown_connections 2024-09-11T00:31:44.775 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:44.774+0000 7f4a911cc740 1 --2- 172.21.15.12:0/2894373576 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x562c7b3c49b0 0x562c7b3c6dd0 crc :-1 s=READY pgs=336 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:44.775 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:44.774+0000 7f4a911cc740 1 --2- 172.21.15.12:0/2894373576 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x562c7b34b530 0x562c7b350e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:44.775 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:44.774+0000 7f4a911cc740 1 --2- 172.21.15.12:0/2894373576 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562c7b345ce0 0x562c7b3508c0 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:44.776 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:44.774+0000 7f4a911cc740 1 --2- 172.21.15.12:0/2894373576 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562c7b340970 0x562c7b352090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:44.776 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:44.774+0000 7f4a911cc740 1 -- 172.21.15.12:0/2894373576 >> 172.21.15.12:0/2894373576 conn(0x562c7b33bac0 msgr2=0x562c7b33abc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:31:44.776 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:44.774+0000 7f4a70c00640 0 client.18120 ms_handle_reset on v2:172.21.15.165:6838/501312984 2024-09-11T00:31:44.776 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:44.775+0000 7f4a911cc740 1 -- 172.21.15.12:0/2894373576 shutdown_connections 2024-09-11T00:31:44.776 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:44.775+0000 7f4a911cc740 1 -- 172.21.15.12:0/2894373576 wait complete. 2024-09-11T00:31:44.776 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:44.775+0000 7f4a911cc740 20 client.18120 trim_cache size 0 max 16384 2024-09-11T00:31:44.777 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:{ 2024-09-11T00:31:44.777 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "message": "", 2024-09-11T00:31:44.777 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "return_code": 0 2024-09-11T00:31:44.777 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:} 2024-09-11T00:31:44.778 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:44.776+0000 7f4a8cc00640 1 -- 172.21.15.12:0/946312275 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f4a6c0b5360 msgr2=0x7f4a6c0b7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:31:44.778 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:44.776+0000 7f4a8cc00640 1 --2- 172.21.15.12:0/946312275 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f4a6c0b5360 0x7f4a6c0b7820 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f4a881044f0 tx=0x7f4a80009500 comp rx=0 tx=0).stop 2024-09-11T00:31:44.778 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:44.776+0000 7f4a8cc00640 1 -- 172.21.15.12:0/946312275 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4a88107ca0 msgr2=0x7f4a88103dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:31:44.778 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:44.776+0000 7f4a8cc00640 1 --2- 172.21.15.12:0/946312275 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4a88107ca0 0x7f4a88103dc0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f4a7400b450 tx=0x7f4a7400b910 comp rx=0 tx=0).stop 2024-09-11T00:31:44.778 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:44.777+0000 7f4a8cc00640 1 -- 172.21.15.12:0/946312275 shutdown_connections 2024-09-11T00:31:44.778 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:44.777+0000 7f4a8cc00640 1 --2- 172.21.15.12:0/946312275 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f4a6c0b5360 0x7f4a6c0b7820 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:44.778 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:44.777+0000 7f4a8cc00640 1 --2- 172.21.15.12:0/946312275 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4a8810a6e0 0x7f4a881049f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:44.779 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:44.777+0000 7f4a8cc00640 1 --2- 172.21.15.12:0/946312275 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4a88107ca0 0x7f4a88103dc0 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:44.779 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:44.777+0000 7f4a8cc00640 1 --2- 172.21.15.12:0/946312275 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4a8806b360 0x7f4a88103770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:44.779 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:44.777+0000 7f4a8cc00640 1 -- 172.21.15.12:0/946312275 >> 172.21.15.12:0/946312275 conn(0x7f4a880fcf30 msgr2=0x7f4a88108920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:31:44.779 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:44.777+0000 7f4a8cc00640 1 -- 172.21.15.12:0/946312275 shutdown_connections 2024-09-11T00:31:44.780 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:44.777+0000 7f4a8cc00640 1 -- 172.21.15.12:0/946312275 wait complete. 2024-09-11T00:31:45.298 INFO:tasks.workunit.client.0.smithi012.stdout: 10 203222 15.78 MB/sec execute 581 sec latency 464.928 ms 2024-09-11T00:31:46.298 INFO:tasks.workunit.client.0.smithi012.stdout: 10 204191 15.85 MB/sec execute 582 sec latency 50.978 ms 2024-09-11T00:31:46.532 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:31:46 smithi165 ceph-mon[35356]: pgmap v1140: 129 pgs: 129 active+clean; 874 MiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 3.3 MiB/s wr, 568 op/s 2024-09-11T00:31:46.541 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:31:46 smithi012 ceph-mon[30333]: pgmap v1140: 129 pgs: 129 active+clean; 874 MiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 3.3 MiB/s wr, 568 op/s 2024-09-11T00:31:46.581 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:31:46 smithi178 ceph-mon[35919]: pgmap v1140: 129 pgs: 129 active+clean; 874 MiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 3.3 MiB/s wr, 568 op/s 2024-09-11T00:31:47.298 INFO:tasks.workunit.client.0.smithi012.stdout: 10 204788 15.88 MB/sec execute 583 sec latency 221.335 ms 2024-09-11T00:31:48.298 INFO:tasks.workunit.client.0.smithi012.stdout: 10 205142 15.89 MB/sec execute 584 sec latency 332.383 ms 2024-09-11T00:31:48.532 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:31:48 smithi165 ceph-mon[35356]: pgmap v1141: 129 pgs: 129 active+clean; 880 MiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 4.4 MiB/s wr, 799 op/s 2024-09-11T00:31:48.532 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:31:48 smithi165 ceph-mon[35356]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T00:31:48.548 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:31:48 smithi012 ceph-mon[30333]: pgmap v1141: 129 pgs: 129 active+clean; 880 MiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 4.4 MiB/s wr, 799 op/s 2024-09-11T00:31:48.548 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:31:48 smithi012 ceph-mon[30333]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T00:31:48.581 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:31:48 smithi178 ceph-mon[35919]: pgmap v1141: 129 pgs: 129 active+clean; 880 MiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 4.4 MiB/s wr, 799 op/s 2024-09-11T00:31:48.597 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:31:48 smithi178 ceph-mon[35919]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T00:31:49.312 INFO:tasks.workunit.client.0.smithi012.stdout: 10 205300 15.87 MB/sec execute 585 sec latency 761.404 ms 2024-09-11T00:31:50.316 INFO:tasks.workunit.client.0.smithi012.stdout: 10 205306 15.85 MB/sec execute 586 sec latency 1767.711 ms 2024-09-11T00:31:50.391 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:31:50 smithi012 ceph-mon[30333]: pgmap v1142: 129 pgs: 129 active+clean; 880 MiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 3.2 MiB/s wr, 573 op/s 2024-09-11T00:31:50.532 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:31:50 smithi165 ceph-mon[35356]: pgmap v1142: 129 pgs: 129 active+clean; 880 MiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 3.2 MiB/s wr, 573 op/s 2024-09-11T00:31:50.581 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:31:50 smithi178 ceph-mon[35919]: pgmap v1142: 129 pgs: 129 active+clean; 880 MiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 3.2 MiB/s wr, 573 op/s 2024-09-11T00:31:51.323 INFO:tasks.workunit.client.0.smithi012.stdout: 10 205308 15.82 MB/sec execute 587 sec latency 2770.447 ms 2024-09-11T00:31:52.319 INFO:tasks.workunit.client.0.smithi012.stdout: 10 205310 15.79 MB/sec execute 588 sec latency 3713.865 ms 2024-09-11T00:31:52.337 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:31:52 smithi012 ceph-mon[30333]: pgmap v1143: 129 pgs: 129 active+clean; 938 MiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 8.6 MiB/s wr, 704 op/s 2024-09-11T00:31:52.581 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:31:52 smithi178 ceph-mon[35919]: pgmap v1143: 129 pgs: 129 active+clean; 938 MiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 8.6 MiB/s wr, 704 op/s 2024-09-11T00:31:52.782 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:31:52 smithi165 ceph-mon[35356]: pgmap v1143: 129 pgs: 129 active+clean; 938 MiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 8.6 MiB/s wr, 704 op/s 2024-09-11T00:31:53.319 INFO:tasks.workunit.client.0.smithi012.stdout: 10 205316 15.77 MB/sec execute 589 sec latency 4501.611 ms 2024-09-11T00:31:54.319 INFO:tasks.workunit.client.0.smithi012.stdout: 10 205316 15.74 MB/sec execute 590 sec latency 5502.130 ms 2024-09-11T00:31:54.581 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:31:54 smithi178 ceph-mon[35919]: pgmap v1144: 129 pgs: 129 active+clean; 969 MiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 10 MiB/s wr, 670 op/s 2024-09-11T00:31:54.782 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:31:54 smithi165 ceph-mon[35356]: pgmap v1144: 129 pgs: 129 active+clean; 969 MiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 10 MiB/s wr, 670 op/s 2024-09-11T00:31:54.813 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:31:54 smithi012 ceph-mon[30333]: pgmap v1144: 129 pgs: 129 active+clean; 969 MiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 10 MiB/s wr, 670 op/s 2024-09-11T00:31:55.319 INFO:tasks.workunit.client.0.smithi012.stdout: 10 205316 15.71 MB/sec execute 591 sec latency 6502.184 ms 2024-09-11T00:31:56.319 INFO:tasks.workunit.client.0.smithi012.stdout: 10 205316 15.69 MB/sec execute 592 sec latency 7502.241 ms 2024-09-11T00:31:56.581 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:31:56 smithi178 ceph-mon[35919]: pgmap v1145: 129 pgs: 129 active+clean; 956 MiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 11 MiB/s wr, 669 op/s 2024-09-11T00:31:56.783 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:31:56 smithi165 ceph-mon[35356]: pgmap v1145: 129 pgs: 129 active+clean; 956 MiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 11 MiB/s wr, 669 op/s 2024-09-11T00:31:56.813 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:31:56 smithi012 ceph-mon[30333]: pgmap v1145: 129 pgs: 129 active+clean; 956 MiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 11 MiB/s wr, 669 op/s 2024-09-11T00:31:57.319 INFO:tasks.workunit.client.0.smithi012.stdout: 10 205316 15.66 MB/sec execute 593 sec latency 8502.301 ms 2024-09-11T00:31:57.904 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:57.902+0000 7f56f0e00640 1 -- 172.21.15.12:0/1398366540 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f56ec104ba0 msgr2=0x7f56ec104fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:31:57.904 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:57.902+0000 7f56f0e00640 1 --2- 172.21.15.12:0/1398366540 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f56ec104ba0 0x7f56ec104fa0 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f56dc009930 tx=0x7f56dc02a420 comp rx=0 tx=0).stop 2024-09-11T00:31:57.904 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:57.902+0000 7f56f0e00640 1 -- 172.21.15.12:0/1398366540 shutdown_connections 2024-09-11T00:31:57.904 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:57.902+0000 7f56f0e00640 1 --2- 172.21.15.12:0/1398366540 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f56ec1066b0 0x7f56ec10d030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:57.905 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:57.902+0000 7f56f0e00640 1 --2- 172.21.15.12:0/1398366540 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f56ec105cf0 0x7f56ec106170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:57.905 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:57.902+0000 7f56f0e00640 1 --2- 172.21.15.12:0/1398366540 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f56ec104ba0 0x7f56ec104fa0 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:57.905 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:57.902+0000 7f56f0e00640 1 -- 172.21.15.12:0/1398366540 >> 172.21.15.12:0/1398366540 conn(0x7f56ec100460 msgr2=0x7f56ec102880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:31:57.905 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:57.902+0000 7f56f0e00640 1 -- 172.21.15.12:0/1398366540 shutdown_connections 2024-09-11T00:31:57.905 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:57.902+0000 7f56f0e00640 1 -- 172.21.15.12:0/1398366540 wait complete. 2024-09-11T00:31:57.912 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:57.903+0000 7f56f0e00640 1 Processor -- start 2024-09-11T00:31:57.912 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:57.903+0000 7f56f0e00640 1 -- start start 2024-09-11T00:31:57.912 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:57.903+0000 7f56f0e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f56ec104ba0 0x7f56ec0779b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:31:57.912 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:57.904+0000 7f56f0e00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f56ec105cf0 0x7f56ec078000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:31:57.912 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:57.904+0000 7f56f0e00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f56ec1066b0 0x7f56ec078c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:31:57.912 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:57.904+0000 7f56f0e00640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f56ec1107f0 con 0x7f56ec1066b0 2024-09-11T00:31:57.912 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:57.904+0000 7f56f0e00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f56ec110670 con 0x7f56ec104ba0 2024-09-11T00:31:57.913 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:57.904+0000 7f56f0e00640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f56ec110970 con 0x7f56ec105cf0 2024-09-11T00:31:57.913 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:57.904+0000 7f56ea800640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f56ec1066b0 0x7f56ec078c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:31:57.913 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:57.904+0000 7f56ea800640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f56ec1066b0 0x7f56ec078c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:48482/0 (socket says 172.21.15.12:48482) 2024-09-11T00:31:57.913 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:57.904+0000 7f56ea800640 1 -- 172.21.15.12:0/435951902 learned_addr learned my addr 172.21.15.12:0/435951902 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:31:57.913 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:57.904+0000 7f56e9400640 1 --2- 172.21.15.12:0/435951902 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f56ec105cf0 0x7f56ec078000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:31:57.913 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:57.904+0000 7f56ea800640 1 -- 172.21.15.12:0/435951902 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f56ec105cf0 msgr2=0x7f56ec078000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:31:57.913 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:57.904+0000 7f56ea800640 1 --2- 172.21.15.12:0/435951902 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f56ec105cf0 0x7f56ec078000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:57.913 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:57.904+0000 7f56ea800640 1 -- 172.21.15.12:0/435951902 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f56ec104ba0 msgr2=0x7f56ec0779b0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-11T00:31:57.913 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:57.904+0000 7f56ea800640 1 --2- 172.21.15.12:0/435951902 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f56ec104ba0 0x7f56ec0779b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:57.913 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:57.904+0000 7f56ea800640 1 -- 172.21.15.12:0/435951902 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56ec0794f0 con 0x7f56ec1066b0 2024-09-11T00:31:57.913 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:57.905+0000 7f56ea800640 1 --2- 172.21.15.12:0/435951902 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f56ec1066b0 0x7f56ec078c30 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f56e400eb10 tx=0x7f56e400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:31:57.913 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:57.905+0000 7f56e2a00640 1 -- 172.21.15.12:0/435951902 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56e4004b20 con 0x7f56ec1066b0 2024-09-11T00:31:57.914 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:57.905+0000 7f56f0e00640 1 -- 172.21.15.12:0/435951902 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f56ec0797e0 con 0x7f56ec1066b0 2024-09-11T00:31:57.914 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:57.905+0000 7f56e2a00640 1 -- 172.21.15.12:0/435951902 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f56e4004cc0 con 0x7f56ec1066b0 2024-09-11T00:31:57.914 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:57.905+0000 7f56e2a00640 1 -- 172.21.15.12:0/435951902 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56e4010670 con 0x7f56ec1066b0 2024-09-11T00:31:57.914 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:57.905+0000 7f56f0e00640 1 -- 172.21.15.12:0/435951902 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56ec079e80 con 0x7f56ec1066b0 2024-09-11T00:31:57.914 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:57.908+0000 7f56f3bf7740 1 Processor -- start 2024-09-11T00:31:57.914 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:57.908+0000 7f56e2a00640 1 -- 172.21.15.12:0/435951902 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7f56e4005020 con 0x7f56ec1066b0 2024-09-11T00:31:57.915 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:57.909+0000 7f56f3bf7740 1 -- start start 2024-09-11T00:31:57.915 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:57.909+0000 7f56f3bf7740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5566a9c48790 0x5566a9c4ab80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:31:57.915 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:57.909+0000 7f56f3bf7740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5566a9c4b0c0 0x5566a9c4d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:31:57.915 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:57.909+0000 7f56f3bf7740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5566a9c4db00 0x5566a9c4ffc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:31:57.915 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:57.909+0000 7f56f3bf7740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x5566a9bbb6b0 con 0x5566a9c4b0c0 2024-09-11T00:31:57.915 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:57.909+0000 7f56f3bf7740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x5566a9baaae0 con 0x5566a9c48790 2024-09-11T00:31:57.915 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:57.909+0000 7f56f3bf7740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5566a9bae8f0 con 0x5566a9c4db00 2024-09-11T00:31:57.915 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:57.909+0000 7f56e9e00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5566a9c4b0c0 0x5566a9c4d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:31:57.915 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:57.909+0000 7f56e9e00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5566a9c4b0c0 0x5566a9c4d4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:48498/0 (socket says 172.21.15.12:48498) 2024-09-11T00:31:57.915 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:57.909+0000 7f56e9e00640 1 -- 172.21.15.12:0/3923216920 learned_addr learned my addr 172.21.15.12:0/3923216920 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:31:57.915 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:57.909+0000 7f56e9400640 1 --2- 172.21.15.12:0/3923216920 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5566a9c48790 0x5566a9c4ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:31:57.915 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:57.909+0000 7f56e9e00640 1 -- 172.21.15.12:0/3923216920 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5566a9c4db00 msgr2=0x5566a9c4ffc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:31:57.915 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:57.909+0000 7f56e9400640 1 --2- 172.21.15.12:0/3923216920 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5566a9c4db00 0x5566a9c4ffc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:31:57.916 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:57.910+0000 7f56e9e00640 1 --2- 172.21.15.12:0/3923216920 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5566a9c4db00 0x5566a9c4ffc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:57.916 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:57.910+0000 7f56e9e00640 1 -- 172.21.15.12:0/3923216920 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5566a9c48790 msgr2=0x5566a9c4ab80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:31:57.916 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:57.910+0000 7f56e9e00640 1 --2- 172.21.15.12:0/3923216920 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5566a9c48790 0x5566a9c4ab80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:57.916 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:57.910+0000 7f56e9e00640 1 -- 172.21.15.12:0/3923216920 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5566a9b77780 con 0x5566a9c4b0c0 2024-09-11T00:31:57.916 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:57.910+0000 7f56e9400640 1 --2- 172.21.15.12:0/3923216920 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5566a9c4db00 0x5566a9c4ffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T00:31:57.916 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:57.910+0000 7f56e9e00640 1 --2- 172.21.15.12:0/3923216920 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5566a9c4b0c0 0x5566a9c4d4b0 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f56dc006010 tx=0x7f56dc0095b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c116f0e88e0f152 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:31:57.919 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:57.910+0000 7f56d7400640 1 -- 172.21.15.12:0/3923216920 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56dc03e030 con 0x5566a9c4b0c0 2024-09-11T00:31:57.919 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:57.910+0000 7f56d7400640 1 -- 172.21.15.12:0/3923216920 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f56dc032ce0 con 0x5566a9c4b0c0 2024-09-11T00:31:57.920 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:57.910+0000 7f56d7400640 1 -- 172.21.15.12:0/3923216920 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56dc035040 con 0x5566a9c4b0c0 2024-09-11T00:31:57.920 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:57.910+0000 7f56e2a00640 1 --2- 172.21.15.12:0/435951902 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f56d00b52e0 0x7f56d00b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:31:57.920 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:57.911+0000 7f56e2a00640 1 -- 172.21.15.12:0/435951902 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9208+0+0 (secure 0 0 0) 0x7f56e40e66a0 con 0x7f56ec1066b0 2024-09-11T00:31:57.920 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:57.911+0000 7f56e9400640 1 --2- 172.21.15.12:0/435951902 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f56d00b52e0 0x7f56d00b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:31:57.920 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:57.911+0000 7f56f3bf7740 1 -- 172.21.15.12:0/3923216920 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5566a9c4b0c0 msgr2=0x5566a9c4d4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:31:57.920 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:57.911+0000 7f56f3bf7740 1 --2- 172.21.15.12:0/3923216920 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5566a9c4b0c0 0x5566a9c4d4b0 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f56dc006010 tx=0x7f56dc0095b0 comp rx=0 tx=0).stop 2024-09-11T00:31:57.920 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:57.911+0000 7f56f3bf7740 1 -- 172.21.15.12:0/3923216920 shutdown_connections 2024-09-11T00:31:57.920 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:57.911+0000 7f56f3bf7740 1 --2- 172.21.15.12:0/3923216920 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5566a9c4db00 0x5566a9c4ffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:57.920 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:57.911+0000 7f56f3bf7740 1 --2- 172.21.15.12:0/3923216920 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5566a9c4b0c0 0x5566a9c4d4b0 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:57.920 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:57.911+0000 7f56f3bf7740 1 --2- 172.21.15.12:0/3923216920 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5566a9c48790 0x5566a9c4ab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:57.920 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:57.911+0000 7f56f3bf7740 1 -- 172.21.15.12:0/3923216920 >> 172.21.15.12:0/3923216920 conn(0x5566a9c438e0 msgr2=0x5566a9c44670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:31:57.920 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:57.911+0000 7f56f3bf7740 1 -- 172.21.15.12:0/3923216920 shutdown_connections 2024-09-11T00:31:57.921 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:57.911+0000 7f56f3bf7740 1 -- 172.21.15.12:0/3923216920 wait complete. 2024-09-11T00:31:57.921 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:57.912+0000 7f56e9400640 1 --2- 172.21.15.12:0/435951902 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f56d00b52e0 0x7f56d00b77a0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f56d8005d00 tx=0x7f56d8014040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:31:57.921 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:57.912+0000 7f56f3bf7740 1 -- start start 2024-09-11T00:31:57.921 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:57.912+0000 7f56f3bf7740 1 Processor -- start 2024-09-11T00:31:57.921 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:57.912+0000 7f56f3bf7740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5566a9c48790 0x5566a9c51db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:31:57.921 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:57.912+0000 7f56f3bf7740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5566a9c4b0c0 0x5566a9c52450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:31:57.921 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:57.912+0000 7f56f3bf7740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5566a9c4db00 0x5566a9c53020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:31:57.921 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:57.912+0000 7f56f3bf7740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x5566a9ae2ab0 con 0x5566a9c4db00 2024-09-11T00:31:57.921 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:57.912+0000 7f56f3bf7740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x5566a9b51570 con 0x5566a9c4b0c0 2024-09-11T00:31:57.921 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:57.912+0000 7f56f3bf7740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x5566a9c50a80 con 0x5566a9c48790 2024-09-11T00:31:57.921 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:57.912+0000 7f56f3bf7740 10 client.0 _ll_register_callbacks cb 0x5566a9b60100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:31:57.921 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:57.913+0000 7f56ea800640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5566a9c4db00 0x5566a9c53020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:31:57.922 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:57.913+0000 7f56ea800640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5566a9c4db00 0x5566a9c53020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:48508/0 (socket says 172.21.15.12:48508) 2024-09-11T00:31:57.922 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:57.913+0000 7f56ea800640 1 -- 172.21.15.12:0/3314696984 learned_addr learned my addr 172.21.15.12:0/3314696984 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:31:57.922 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:57.913+0000 7f56e9e00640 1 --2- 172.21.15.12:0/3314696984 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5566a9c48790 0x5566a9c51db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:31:57.922 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:57.913+0000 7f56e9400640 1 --2- 172.21.15.12:0/3314696984 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5566a9c4b0c0 0x5566a9c52450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:31:57.922 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:57.913+0000 7f56ea800640 1 -- 172.21.15.12:0/3314696984 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5566a9c48790 msgr2=0x5566a9c51db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:31:57.922 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:57.913+0000 7f56ea800640 1 --2- 172.21.15.12:0/3314696984 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5566a9c48790 0x5566a9c51db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:57.922 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:57.913+0000 7f56ea800640 1 -- 172.21.15.12:0/3314696984 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5566a9c4b0c0 msgr2=0x5566a9c52450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:31:57.922 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:57.913+0000 7f56ea800640 1 --2- 172.21.15.12:0/3314696984 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5566a9c4b0c0 0x5566a9c52450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:57.922 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:57.913+0000 7f56ea800640 1 -- 172.21.15.12:0/3314696984 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5566a990c790 con 0x5566a9c4db00 2024-09-11T00:31:57.922 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:57.913+0000 7f56ea800640 1 --2- 172.21.15.12:0/3314696984 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5566a9c4db00 0x5566a9c53020 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f56e40a9ca0 tx=0x7f56e40ac800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cbcadf6238f719db server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:31:57.922 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:57.913+0000 7f56cbe00640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-09-11T00:31:57.922 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:57.914+0000 7f56cbe00640 1 -- 172.21.15.12:0/3314696984 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56e40a81a0 con 0x5566a9c4db00 2024-09-11T00:31:57.923 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:57.914+0000 7f56cbe00640 1 -- 172.21.15.12:0/3314696984 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 9208+0+0 (secure 0 0 0) 0x7f56e40b5b30 con 0x5566a9c4db00 2024-09-11T00:31:57.923 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:57.914+0000 7f56f3bf7740 1 -- 172.21.15.12:0/3314696984 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f56dc032ce0 con 0x5566a9c4db00 2024-09-11T00:31:57.923 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:57.914+0000 7f56cbe00640 1 -- 172.21.15.12:0/3314696984 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=107) ==== 24+0+0 (secure 0 0 0) 0x7f56e4005540 con 0x5566a9c4db00 2024-09-11T00:31:57.923 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:57.914+0000 7f56f3bf7740 10 client.18132 fetch_fsmap learned FSMap version 107 2024-09-11T00:31:57.923 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:57.914+0000 7f56f3bf7740 1 -- 172.21.15.12:0/3314696984 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=107,monmap=4+}) -- 0x5566a9985620 con 0x5566a9c4db00 2024-09-11T00:31:57.923 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:57.914+0000 7f56cbe00640 1 -- 172.21.15.12:0/3314696984 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f56dc032ce0 con 0x5566a9c4db00 2024-09-11T00:31:57.923 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:57.914+0000 7f56cbe00640 1 -- 172.21.15.12:0/3314696984 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 107) ==== 8390+0+0 (secure 0 0 0) 0x7f56e40b1070 con 0x5566a9c4db00 2024-09-11T00:31:57.923 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:57.914+0000 7f56f3bf7740 10 client.18132 fetch_fsmap finished waiting for FSMap version 107 2024-09-11T00:31:57.923 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:57.914+0000 7f56f3bf7740 10 client.18132 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T00:31:57.923 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:57.914+0000 7f56f3bf7740 20 client.18132 populate_metadata read hostname 'smithi012' 2024-09-11T00:31:57.923 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:57.914+0000 7f56f3bf7740 1 --2- 172.21.15.12:0/3314696984 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x5566a9c5d680 0x5566a9c5fa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:31:57.923 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:57.914+0000 7f56f3bf7740 4 client.18132 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T00:31:57.923 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:57.914+0000 7f56f3bf7740 1 -- 172.21.15.12:0/3314696984 --> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5566a9bbd600 con 0x5566a9c5d680 2024-09-11T00:31:57.924 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:57.915+0000 7f56e9e00640 1 --2- 172.21.15.12:0/3314696984 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x5566a9c5d680 0x5566a9c5fa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:31:57.924 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:57.916+0000 7f56e9e00640 1 --2- 172.21.15.12:0/3314696984 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x5566a9c5d680 0x5566a9c5fa90 crc :-1 s=READY pgs=337 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7acc45eb86d57ec7 server_cookie=e9028b0f27e8a07b in_seq=0 out_seq=0 2024-09-11T00:31:57.924 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:57.916+0000 7f56cbe00640 10 client.18132 ms_handle_connect on v2:172.21.15.165:6838/501312984 2024-09-11T00:31:57.925 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:57.917+0000 7f56cbe00640 1 -- 172.21.15.12:0/3314696984 <== mds.0 v2:172.21.15.165:6838/501312984 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5566a9bbd600 con 0x5566a9c5d680 2024-09-11T00:31:57.925 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:57.917+0000 7f56cbe00640 10 client.18132 handle_command_reply: tid=0 2024-09-11T00:31:57.925 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:57.917+0000 7f56f3bf7740 1 client.18132 shutdown 2024-09-11T00:31:57.925 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:57.917+0000 7f56f3bf7740 1 -- 172.21.15.12:0/3314696984 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5566a9c4db00 msgr2=0x5566a9c53020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:31:57.925 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:57.917+0000 7f56f3bf7740 1 --2- 172.21.15.12:0/3314696984 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5566a9c4db00 0x5566a9c53020 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f56e40a9ca0 tx=0x7f56e40ac800 comp rx=0 tx=0).stop 2024-09-11T00:31:57.925 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:57.917+0000 7f56f3bf7740 1 -- 172.21.15.12:0/3314696984 shutdown_connections 2024-09-11T00:31:57.925 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:57.917+0000 7f56f3bf7740 1 --2- 172.21.15.12:0/3314696984 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x5566a9c5d680 0x5566a9c5fa90 crc :-1 s=READY pgs=337 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:57.926 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:57.917+0000 7f56f3bf7740 1 --2- 172.21.15.12:0/3314696984 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5566a9c4db00 0x5566a9c53020 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:57.926 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:57.917+0000 7f56f3bf7740 1 --2- 172.21.15.12:0/3314696984 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5566a9c4b0c0 0x5566a9c52450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:57.926 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:57.917+0000 7f56f3bf7740 1 --2- 172.21.15.12:0/3314696984 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5566a9c48790 0x5566a9c51db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:57.926 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:57.917+0000 7f56cbe00640 0 client.18132 ms_handle_reset on v2:172.21.15.165:6838/501312984 2024-09-11T00:31:57.926 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:57.917+0000 7f56f3bf7740 1 -- 172.21.15.12:0/3314696984 >> 172.21.15.12:0/3314696984 conn(0x5566a9c438e0 msgr2=0x5566a9c42a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:31:57.926 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:57.917+0000 7f56f3bf7740 1 -- 172.21.15.12:0/3314696984 shutdown_connections 2024-09-11T00:31:57.926 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:57.917+0000 7f56f3bf7740 1 -- 172.21.15.12:0/3314696984 wait complete. 2024-09-11T00:31:57.926 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:57.917+0000 7f56f3bf7740 20 client.18132 trim_cache size 0 max 16384 2024-09-11T00:31:57.935 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:57.933+0000 7f56f3bf7740 1 Processor -- start 2024-09-11T00:31:57.935 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:57.933+0000 7f56f3bf7740 1 -- start start 2024-09-11T00:31:57.935 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:57.933+0000 7f56f3bf7740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5566a9c48790 0x5566a9c45710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:31:57.935 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:57.933+0000 7f56f3bf7740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5566a9c4b0c0 0x5566a9c5a0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:31:57.935 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:57.933+0000 7f56f3bf7740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5566a9c4db00 0x5566a9c45c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:31:57.935 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:57.933+0000 7f56f3bf7740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f56ec1b1d70 con 0x5566a9c4db00 2024-09-11T00:31:57.936 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:57.933+0000 7f56f3bf7740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f56ec1b38b0 con 0x5566a9c4b0c0 2024-09-11T00:31:57.936 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:57.933+0000 7f56f3bf7740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f56ec1b2440 con 0x5566a9c48790 2024-09-11T00:31:57.936 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:57.933+0000 7f56ea800640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5566a9c4db00 0x5566a9c45c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:31:57.936 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:57.933+0000 7f56ea800640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5566a9c4db00 0x5566a9c45c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:48520/0 (socket says 172.21.15.12:48520) 2024-09-11T00:31:57.936 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:57.933+0000 7f56ea800640 1 -- 172.21.15.12:0/2807639308 learned_addr learned my addr 172.21.15.12:0/2807639308 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:31:57.936 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:57.933+0000 7f56e9e00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5566a9c48790 0x5566a9c45710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:31:57.936 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:57.933+0000 7f56ea800640 1 -- 172.21.15.12:0/2807639308 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5566a9c48790 msgr2=0x5566a9c45710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:31:57.936 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:57.933+0000 7f56ea800640 1 --2- 172.21.15.12:0/2807639308 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5566a9c48790 0x5566a9c45710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:57.936 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:57.933+0000 7f56ea800640 1 -- 172.21.15.12:0/2807639308 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5566a9c4b0c0 msgr2=0x5566a9c5a0a0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-11T00:31:57.936 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:57.933+0000 7f56ea800640 1 --2- 172.21.15.12:0/2807639308 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5566a9c4b0c0 0x5566a9c5a0a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:57.936 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:57.934+0000 7f56ea800640 1 -- 172.21.15.12:0/2807639308 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5566a9c4aef0 con 0x5566a9c4db00 2024-09-11T00:31:57.936 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:57.934+0000 7f56ea800640 1 --2- 172.21.15.12:0/2807639308 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5566a9c4db00 0x5566a9c45c50 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f56e40a9000 tx=0x7f56e40a90a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65c5892a004bd4c7 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:31:57.937 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:57.934+0000 7f56d6a00640 1 -- 172.21.15.12:0/2807639308 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56e40a81a0 con 0x5566a9c4db00 2024-09-11T00:31:57.937 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:57.934+0000 7f56d6a00640 1 -- 172.21.15.12:0/2807639308 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f56e40ac540 con 0x5566a9c4db00 2024-09-11T00:31:57.937 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:57.934+0000 7f56d6a00640 1 -- 172.21.15.12:0/2807639308 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56e40b5b30 con 0x5566a9c4db00 2024-09-11T00:31:57.938 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:57.935+0000 7f56f3bf7740 1 -- 172.21.15.12:0/2807639308 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5566a9c4db00 msgr2=0x5566a9c45c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:31:57.938 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:57.935+0000 7f56f3bf7740 1 --2- 172.21.15.12:0/2807639308 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5566a9c4db00 0x5566a9c45c50 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f56e40a9000 tx=0x7f56e40a90a0 comp rx=0 tx=0).stop 2024-09-11T00:31:57.938 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:57.935+0000 7f56f3bf7740 1 -- 172.21.15.12:0/2807639308 shutdown_connections 2024-09-11T00:31:57.938 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:57.935+0000 7f56f3bf7740 1 --2- 172.21.15.12:0/2807639308 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5566a9c4db00 0x5566a9c45c50 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:57.938 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:57.935+0000 7f56f3bf7740 1 --2- 172.21.15.12:0/2807639308 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5566a9c4b0c0 0x5566a9c5a0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:57.938 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:57.935+0000 7f56f3bf7740 1 --2- 172.21.15.12:0/2807639308 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5566a9c48790 0x5566a9c45710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:57.938 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:57.935+0000 7f56f3bf7740 1 -- 172.21.15.12:0/2807639308 >> 172.21.15.12:0/2807639308 conn(0x5566a9c438e0 msgr2=0x5566a9c47b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:31:57.939 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:57.935+0000 7f56f3bf7740 1 -- 172.21.15.12:0/2807639308 shutdown_connections 2024-09-11T00:31:57.939 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:57.935+0000 7f56f3bf7740 1 -- 172.21.15.12:0/2807639308 wait complete. 2024-09-11T00:31:57.939 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:57.935+0000 7f56f3bf7740 1 -- start start 2024-09-11T00:31:57.939 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:57.936+0000 7f56f3bf7740 1 Processor -- start 2024-09-11T00:31:57.939 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:57.936+0000 7f56f3bf7740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5566a9c48790 0x5566a9c59760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:31:57.939 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:57.936+0000 7f56f3bf7740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5566a9c4b0c0 0x5566a9c59de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:31:57.939 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:57.936+0000 7f56f3bf7740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5566a9c4db00 0x5566a9c58030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:31:57.939 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:57.936+0000 7f56f3bf7740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f56ec1b31e0 con 0x5566a9c4db00 2024-09-11T00:31:57.939 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:57.936+0000 7f56f3bf7740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f56ec1b2b10 con 0x5566a9c4b0c0 2024-09-11T00:31:57.939 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:57.936+0000 7f56f3bf7740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f56ec1afd50 con 0x5566a9c48790 2024-09-11T00:31:57.939 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:57.936+0000 7f56f3bf7740 10 client.0 _ll_register_callbacks cb 0x7f56e40a9130 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:31:57.941 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:57.936+0000 7f56ea800640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5566a9c4db00 0x5566a9c58030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:31:57.941 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:57.936+0000 7f56e9400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5566a9c4b0c0 0x5566a9c59de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:31:57.941 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:57.936+0000 7f56e9e00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5566a9c48790 0x5566a9c59760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:31:57.941 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:57.936+0000 7f56e9400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5566a9c4b0c0 0x5566a9c59de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.12:37058/0 (socket says 172.21.15.12:37058) 2024-09-11T00:31:57.941 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:57.936+0000 7f56ea800640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5566a9c4db00 0x5566a9c58030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:48526/0 (socket says 172.21.15.12:48526) 2024-09-11T00:31:57.942 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:57.936+0000 7f56e9400640 1 -- 172.21.15.12:0/4045725091 learned_addr learned my addr 172.21.15.12:0/4045725091 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:31:57.942 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:57.936+0000 7f56e9e00640 1 --2- 172.21.15.12:0/4045725091 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5566a9c48790 0x5566a9c59760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.12:46220/0 (socket says 172.21.15.12:46220) 2024-09-11T00:31:57.942 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:57.937+0000 7f56ea800640 1 -- 172.21.15.12:0/4045725091 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5566a9c48790 msgr2=0x5566a9c59760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:31:57.942 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:57.937+0000 7f56ea800640 1 --2- 172.21.15.12:0/4045725091 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5566a9c48790 0x5566a9c59760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:57.942 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:57.937+0000 7f56ea800640 1 -- 172.21.15.12:0/4045725091 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5566a9c4b0c0 msgr2=0x5566a9c59de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:31:57.942 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:57.937+0000 7f56ea800640 1 --2- 172.21.15.12:0/4045725091 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5566a9c4b0c0 0x5566a9c59de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:57.942 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:57.937+0000 7f56ea800640 1 -- 172.21.15.12:0/4045725091 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5566a9c58860 con 0x5566a9c4db00 2024-09-11T00:31:57.942 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:57.937+0000 7f56ea800640 1 --2- 172.21.15.12:0/4045725091 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5566a9c4db00 0x5566a9c58030 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x5566a9c3eab0 tx=0x7f56e40e65d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e476b8f4cafa50a8 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:31:57.942 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:57.937+0000 7f56cbe00640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-09-11T00:31:57.942 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:57.937+0000 7f56cbe00640 1 -- 172.21.15.12:0/4045725091 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56e40a81a0 con 0x5566a9c4db00 2024-09-11T00:31:57.942 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:57.937+0000 7f56cbe00640 1 -- 172.21.15.12:0/4045725091 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 9208+0+0 (secure 0 0 0) 0x7f56e40abea0 con 0x5566a9c4db00 2024-09-11T00:31:57.942 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:57.937+0000 7f56f3bf7740 1 -- 172.21.15.12:0/4045725091 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f56e40ac540 con 0x5566a9c4db00 2024-09-11T00:31:57.942 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:57.938+0000 7f56cbe00640 1 -- 172.21.15.12:0/4045725091 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=107) ==== 24+0+0 (secure 0 0 0) 0x7f56e40ac380 con 0x5566a9c4db00 2024-09-11T00:31:57.942 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:57.938+0000 7f56f3bf7740 10 client.18138 fetch_fsmap learned FSMap version 107 2024-09-11T00:31:57.943 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:57.938+0000 7f56f3bf7740 1 -- 172.21.15.12:0/4045725091 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=107,monmap=4+}) -- 0x5566a9c58860 con 0x5566a9c4db00 2024-09-11T00:31:57.943 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:57.938+0000 7f56cbe00640 1 -- 172.21.15.12:0/4045725091 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f56e40ac540 con 0x5566a9c4db00 2024-09-11T00:31:57.943 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:57.938+0000 7f56cbe00640 1 -- 172.21.15.12:0/4045725091 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 107) ==== 8390+0+0 (secure 0 0 0) 0x7f56e4071410 con 0x5566a9c4db00 2024-09-11T00:31:57.943 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:57.938+0000 7f56f3bf7740 10 client.18138 fetch_fsmap finished waiting for FSMap version 107 2024-09-11T00:31:57.943 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:57.938+0000 7f56f3bf7740 10 client.18138 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T00:31:57.943 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:57.938+0000 7f56f3bf7740 20 client.18138 populate_metadata read hostname 'smithi012' 2024-09-11T00:31:57.943 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:57.938+0000 7f56f3bf7740 1 --2- 172.21.15.12:0/4045725091 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x5566a9ccb5e0 0x5566a9ccda00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:31:57.943 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:57.938+0000 7f56f3bf7740 4 client.18138 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T00:31:57.943 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:57.938+0000 7f56f3bf7740 1 -- 172.21.15.12:0/4045725091 --> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5566a9bb90b0 con 0x5566a9ccb5e0 2024-09-11T00:31:57.970 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:57.969+0000 7f56e9e00640 1 --2- 172.21.15.12:0/4045725091 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x5566a9ccb5e0 0x5566a9ccda00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:31:58.089 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:58.087+0000 7f56e9e00640 1 --2- 172.21.15.12:0/4045725091 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x5566a9ccb5e0 0x5566a9ccda00 crc :-1 s=READY pgs=338 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=325ab59479243160 server_cookie=44516ef86b54485c in_seq=0 out_seq=0 2024-09-11T00:31:58.089 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:58.087+0000 7f56cbe00640 10 client.18138 ms_handle_connect on v2:172.21.15.165:6838/501312984 2024-09-11T00:31:58.243 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:58.241+0000 7f56cbe00640 1 -- 172.21.15.12:0/4045725091 <== mds.0 v2:172.21.15.165:6838/501312984 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5566a9bb90b0 con 0x5566a9ccb5e0 2024-09-11T00:31:58.243 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:58.241+0000 7f56cbe00640 10 client.18138 handle_command_reply: tid=0 2024-09-11T00:31:58.243 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:58.241+0000 7f56f3bf7740 1 client.18138 shutdown 2024-09-11T00:31:58.243 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:58.242+0000 7f56f3bf7740 1 -- 172.21.15.12:0/4045725091 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5566a9c4db00 msgr2=0x5566a9c58030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:31:58.243 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:58.242+0000 7f56f3bf7740 1 --2- 172.21.15.12:0/4045725091 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5566a9c4db00 0x5566a9c58030 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x5566a9c3eab0 tx=0x7f56e40e65d0 comp rx=0 tx=0).stop 2024-09-11T00:31:58.244 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:58.242+0000 7f56f3bf7740 1 -- 172.21.15.12:0/4045725091 shutdown_connections 2024-09-11T00:31:58.244 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:58.242+0000 7f56f3bf7740 1 --2- 172.21.15.12:0/4045725091 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x5566a9ccb5e0 0x5566a9ccda00 crc :-1 s=READY pgs=338 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:58.244 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:58.242+0000 7f56f3bf7740 1 --2- 172.21.15.12:0/4045725091 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5566a9c4db00 0x5566a9c58030 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:58.244 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:58.242+0000 7f56f3bf7740 1 --2- 172.21.15.12:0/4045725091 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5566a9c4b0c0 0x5566a9c59de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:58.244 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:58.242+0000 7f56f3bf7740 1 --2- 172.21.15.12:0/4045725091 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5566a9c48790 0x5566a9c59760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:58.244 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:58.242+0000 7f56f3bf7740 1 -- 172.21.15.12:0/4045725091 >> 172.21.15.12:0/4045725091 conn(0x5566a9c438e0 msgr2=0x5566a9c47f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:31:58.244 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:58.242+0000 7f56cbe00640 0 client.18138 ms_handle_reset on v2:172.21.15.165:6838/501312984 2024-09-11T00:31:58.244 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:58.242+0000 7f56f3bf7740 1 -- 172.21.15.12:0/4045725091 shutdown_connections 2024-09-11T00:31:58.244 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:58.242+0000 7f56f3bf7740 1 -- 172.21.15.12:0/4045725091 wait complete. 2024-09-11T00:31:58.244 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:58.242+0000 7f56f3bf7740 20 client.18138 trim_cache size 0 max 16384 2024-09-11T00:31:58.245 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:{ 2024-09-11T00:31:58.245 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "message": "", 2024-09-11T00:31:58.245 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "return_code": 0 2024-09-11T00:31:58.245 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:} 2024-09-11T00:31:58.246 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:58.243+0000 7f56f0e00640 1 -- 172.21.15.12:0/435951902 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f56d00b52e0 msgr2=0x7f56d00b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:31:58.246 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:58.243+0000 7f56f0e00640 1 --2- 172.21.15.12:0/435951902 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f56d00b52e0 0x7f56d00b77a0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f56d8005d00 tx=0x7f56d8014040 comp rx=0 tx=0).stop 2024-09-11T00:31:58.246 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:58.243+0000 7f56f0e00640 1 -- 172.21.15.12:0/435951902 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f56ec1066b0 msgr2=0x7f56ec078c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:31:58.246 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:58.243+0000 7f56f0e00640 1 --2- 172.21.15.12:0/435951902 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f56ec1066b0 0x7f56ec078c30 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f56e400eb10 tx=0x7f56e400c2d0 comp rx=0 tx=0).stop 2024-09-11T00:31:58.246 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:58.243+0000 7f56f0e00640 1 -- 172.21.15.12:0/435951902 shutdown_connections 2024-09-11T00:31:58.246 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:58.243+0000 7f56f0e00640 1 --2- 172.21.15.12:0/435951902 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f56d00b52e0 0x7f56d00b77a0 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:58.246 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:58.243+0000 7f56f0e00640 1 --2- 172.21.15.12:0/435951902 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f56ec1066b0 0x7f56ec078c30 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:58.246 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:58.243+0000 7f56f0e00640 1 --2- 172.21.15.12:0/435951902 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f56ec105cf0 0x7f56ec078000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:58.246 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:58.243+0000 7f56f0e00640 1 --2- 172.21.15.12:0/435951902 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f56ec104ba0 0x7f56ec0779b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:58.246 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:58.243+0000 7f56f0e00640 1 -- 172.21.15.12:0/435951902 >> 172.21.15.12:0/435951902 conn(0x7f56ec100460 msgr2=0x7f56ec1090e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:31:58.246 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:58.243+0000 7f56f0e00640 1 -- 172.21.15.12:0/435951902 shutdown_connections 2024-09-11T00:31:58.246 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:31:58.243+0000 7f56f0e00640 1 -- 172.21.15.12:0/435951902 wait complete. 2024-09-11T00:31:58.319 INFO:tasks.workunit.client.0.smithi012.stdout: 10 205316 15.63 MB/sec execute 594 sec latency 9502.356 ms 2024-09-11T00:31:58.782 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:31:58 smithi165 ceph-mon[35356]: pgmap v1146: 129 pgs: 129 active+clean; 885 MiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 13 MiB/s wr, 823 op/s 2024-09-11T00:31:58.813 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:31:58 smithi012 ceph-mon[30333]: pgmap v1146: 129 pgs: 129 active+clean; 885 MiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 13 MiB/s wr, 823 op/s 2024-09-11T00:31:58.831 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:31:58 smithi178 ceph-mon[35919]: pgmap v1146: 129 pgs: 129 active+clean; 885 MiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 13 MiB/s wr, 823 op/s 2024-09-11T00:31:59.319 INFO:tasks.workunit.client.0.smithi012.stdout: 10 205316 15.61 MB/sec execute 595 sec latency 10502.414 ms 2024-09-11T00:32:00.319 INFO:tasks.workunit.client.0.smithi012.stdout: 10 205316 15.58 MB/sec execute 596 sec latency 11502.476 ms 2024-09-11T00:32:00.813 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:32:00 smithi012 ceph-mon[30333]: pgmap v1147: 129 pgs: 129 active+clean; 882 MiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 12 MiB/s wr, 541 op/s 2024-09-11T00:32:00.831 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:32:00 smithi178 ceph-mon[35919]: pgmap v1147: 129 pgs: 129 active+clean; 882 MiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 12 MiB/s wr, 541 op/s 2024-09-11T00:32:00.988 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:32:00 smithi165 ceph-mon[35356]: pgmap v1147: 129 pgs: 129 active+clean; 882 MiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 12 MiB/s wr, 541 op/s 2024-09-11T00:32:01.320 INFO:tasks.workunit.client.0.smithi012.stdout: 10 205316 15.55 MB/sec execute 597 sec latency 12502.538 ms 2024-09-11T00:32:02.320 INFO:tasks.workunit.client.0.smithi012.stdout: 10 205316 15.53 MB/sec execute 598 sec latency 13502.600 ms 2024-09-11T00:32:02.782 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:32:02 smithi165 ceph-mon[35356]: pgmap v1148: 129 pgs: 129 active+clean; 882 MiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 12 MiB/s wr, 550 op/s 2024-09-11T00:32:02.813 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:32:02 smithi012 ceph-mon[30333]: pgmap v1148: 129 pgs: 129 active+clean; 882 MiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 12 MiB/s wr, 550 op/s 2024-09-11T00:32:02.831 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:32:02 smithi178 ceph-mon[35919]: pgmap v1148: 129 pgs: 129 active+clean; 882 MiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 12 MiB/s wr, 550 op/s 2024-09-11T00:32:03.320 INFO:tasks.workunit.client.0.smithi012.stdout: 10 205316 15.50 MB/sec execute 599 sec latency 14502.661 ms 2024-09-11T00:32:04.320 INFO:tasks.workunit.client.0.smithi012.stdout: 10 cleanup 600 sec 2024-09-11T00:32:04.783 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:32:04 smithi165 ceph-mon[35356]: pgmap v1149: 129 pgs: 129 active+clean; 882 MiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 956 KiB/s rd, 6.0 MiB/s wr, 398 op/s 2024-09-11T00:32:04.813 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:32:04 smithi012 ceph-mon[30333]: pgmap v1149: 129 pgs: 129 active+clean; 882 MiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 956 KiB/s rd, 6.0 MiB/s wr, 398 op/s 2024-09-11T00:32:04.831 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:32:04 smithi178 ceph-mon[35919]: pgmap v1149: 129 pgs: 129 active+clean; 882 MiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 956 KiB/s rd, 6.0 MiB/s wr, 398 op/s 2024-09-11T00:32:05.320 INFO:tasks.workunit.client.0.smithi012.stdout: 10 cleanup 601 sec 2024-09-11T00:32:05.783 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:32:05 smithi165 ceph-mon[35356]: pgmap v1150: 129 pgs: 129 active+clean; 882 MiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 536 KiB/s rd, 3.1 MiB/s wr, 292 op/s 2024-09-11T00:32:05.813 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:32:05 smithi012 ceph-mon[30333]: pgmap v1150: 129 pgs: 129 active+clean; 882 MiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 536 KiB/s rd, 3.1 MiB/s wr, 292 op/s 2024-09-11T00:32:05.831 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:32:05 smithi178 ceph-mon[35919]: pgmap v1150: 129 pgs: 129 active+clean; 882 MiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 536 KiB/s rd, 3.1 MiB/s wr, 292 op/s 2024-09-11T00:32:06.320 INFO:tasks.workunit.client.0.smithi012.stdout: 10 cleanup 602 sec 2024-09-11T00:32:07.320 INFO:tasks.workunit.client.0.smithi012.stdout: 10 cleanup 603 sec 2024-09-11T00:32:08.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:32:07 smithi165 ceph-mon[35356]: pgmap v1151: 129 pgs: 129 active+clean; 882 MiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 491 KiB/s rd, 2.4 MiB/s wr, 251 op/s 2024-09-11T00:32:08.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:32:07 smithi012 ceph-mon[30333]: pgmap v1151: 129 pgs: 129 active+clean; 882 MiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 491 KiB/s rd, 2.4 MiB/s wr, 251 op/s 2024-09-11T00:32:08.320 INFO:tasks.workunit.client.0.smithi012.stdout: 10 cleanup 604 sec 2024-09-11T00:32:08.332 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:32:07 smithi178 ceph-mon[35919]: pgmap v1151: 129 pgs: 129 active+clean; 882 MiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 491 KiB/s rd, 2.4 MiB/s wr, 251 op/s 2024-09-11T00:32:09.320 INFO:tasks.workunit.client.0.smithi012.stdout: 10 cleanup 605 sec 2024-09-11T00:32:10.032 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:32:09 smithi012 ceph-mon[30333]: pgmap v1152: 129 pgs: 129 active+clean; 882 MiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 27 KiB/s wr, 56 op/s 2024-09-11T00:32:10.032 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:32:09 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T00:32:10.283 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:32:09 smithi165 ceph-mon[35356]: pgmap v1152: 129 pgs: 129 active+clean; 882 MiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 27 KiB/s wr, 56 op/s 2024-09-11T00:32:10.283 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:32:09 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T00:32:10.320 INFO:tasks.workunit.client.0.smithi012.stdout: 10 cleanup 606 sec 2024-09-11T00:32:10.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:32:09 smithi178 ceph-mon[35919]: pgmap v1152: 129 pgs: 129 active+clean; 882 MiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 27 KiB/s wr, 56 op/s 2024-09-11T00:32:10.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:32:09 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T00:32:11.167 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:32:10 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:32:11.167 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:32:10 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T00:32:11.167 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:32:10 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:32:11.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:32:10 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:32:11.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:32:10 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T00:32:11.283 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:32:10 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:32:11.320 INFO:tasks.workunit.client.0.smithi012.stdout: 10 cleanup 607 sec 2024-09-11T00:32:11.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:32:10 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:32:11.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:32:10 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T00:32:11.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:32:10 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' 2024-09-11T00:32:11.360 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:32:11.357+0000 7f3505a00640 1 -- 172.21.15.12:0/1248034279 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3500104ba0 msgr2=0x7f3500104fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:32:11.360 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:32:11.357+0000 7f3505a00640 1 --2- 172.21.15.12:0/1248034279 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3500104ba0 0x7f3500104fa0 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f34f0009930 tx=0x7f34f002a440 comp rx=0 tx=0).stop 2024-09-11T00:32:11.361 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:32:11.357+0000 7f3505a00640 1 -- 172.21.15.12:0/1248034279 shutdown_connections 2024-09-11T00:32:11.361 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:32:11.357+0000 7f3505a00640 1 --2- 172.21.15.12:0/1248034279 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f35001066b0 0x7f350010d030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:32:11.361 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:32:11.357+0000 7f3505a00640 1 --2- 172.21.15.12:0/1248034279 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3500105cf0 0x7f3500106170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:32:11.361 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:32:11.357+0000 7f3505a00640 1 --2- 172.21.15.12:0/1248034279 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3500104ba0 0x7f3500104fa0 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:32:11.361 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:32:11.357+0000 7f3505a00640 1 -- 172.21.15.12:0/1248034279 >> 172.21.15.12:0/1248034279 conn(0x7f3500100460 msgr2=0x7f3500102880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:32:11.361 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:32:11.358+0000 7f3505a00640 1 -- 172.21.15.12:0/1248034279 shutdown_connections 2024-09-11T00:32:11.361 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:32:11.358+0000 7f3505a00640 1 -- 172.21.15.12:0/1248034279 wait complete. 2024-09-11T00:32:11.363 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:32:11.358+0000 7f3505a00640 1 Processor -- start 2024-09-11T00:32:11.363 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:32:11.359+0000 7f3505a00640 1 -- start start 2024-09-11T00:32:11.363 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:32:11.359+0000 7f3505a00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3500104ba0 0x7f350019af50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:32:11.363 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:32:11.359+0000 7f3505a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3500105cf0 0x7f350019b5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:32:11.363 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:32:11.359+0000 7f3505a00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f35001066b0 0x7f350019c1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:32:11.364 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:32:11.359+0000 7f3505a00640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f35001108c0 con 0x7f35001066b0 2024-09-11T00:32:11.364 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:32:11.359+0000 7f3505a00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f3500110740 con 0x7f3500105cf0 2024-09-11T00:32:11.364 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:32:11.359+0000 7f3505a00640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f3500110a40 con 0x7f3500104ba0 2024-09-11T00:32:11.364 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:32:11.359+0000 7f34ff400640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f35001066b0 0x7f350019c1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:32:11.364 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:32:11.359+0000 7f34ff400640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f35001066b0 0x7f350019c1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:47414/0 (socket says 172.21.15.12:47414) 2024-09-11T00:32:11.364 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:32:11.359+0000 7f34ff400640 1 -- 172.21.15.12:0/2062892254 learned_addr learned my addr 172.21.15.12:0/2062892254 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:32:11.364 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:32:11.359+0000 7f34fe000640 1 --2- 172.21.15.12:0/2062892254 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3500105cf0 0x7f350019b5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:32:11.364 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:32:11.359+0000 7f34fea00640 1 --2- 172.21.15.12:0/2062892254 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3500104ba0 0x7f350019af50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:32:11.364 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:32:11.360+0000 7f34ff400640 1 -- 172.21.15.12:0/2062892254 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3500104ba0 msgr2=0x7f350019af50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:32:11.364 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:32:11.360+0000 7f34ff400640 1 --2- 172.21.15.12:0/2062892254 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3500104ba0 0x7f350019af50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:32:11.364 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:32:11.360+0000 7f34ff400640 1 -- 172.21.15.12:0/2062892254 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3500105cf0 msgr2=0x7f350019b5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:32:11.364 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:32:11.360+0000 7f34ff400640 1 --2- 172.21.15.12:0/2062892254 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3500105cf0 0x7f350019b5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:32:11.364 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:32:11.360+0000 7f34ff400640 1 -- 172.21.15.12:0/2062892254 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f350019ca90 con 0x7f35001066b0 2024-09-11T00:32:11.365 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:32:11.360+0000 7f34fe000640 1 --2- 172.21.15.12:0/2062892254 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3500105cf0 0x7f350019b5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:32:11.365 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:32:11.360+0000 7f34ff400640 1 --2- 172.21.15.12:0/2062892254 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f35001066b0 0x7f350019c1d0 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f34f800b970 tx=0x7f34f800bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:32:11.365 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:32:11.360+0000 7f34f7400640 1 -- 172.21.15.12:0/2062892254 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34f8019070 con 0x7f35001066b0 2024-09-11T00:32:11.365 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:32:11.360+0000 7f34f7400640 1 -- 172.21.15.12:0/2062892254 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f34f8014440 con 0x7f35001066b0 2024-09-11T00:32:11.365 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:32:11.360+0000 7f34f7400640 1 -- 172.21.15.12:0/2062892254 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34f80075d0 con 0x7f35001066b0 2024-09-11T00:32:11.365 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:32:11.360+0000 7f3505a00640 1 -- 172.21.15.12:0/2062892254 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f350019cd80 con 0x7f35001066b0 2024-09-11T00:32:11.365 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:32:11.360+0000 7f3505a00640 1 -- 172.21.15.12:0/2062892254 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35001a3450 con 0x7f35001066b0 2024-09-11T00:32:11.365 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:32:11.363+0000 7f35088a2740 1 Processor -- start 2024-09-11T00:32:11.366 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:32:11.364+0000 7f34f7400640 1 -- 172.21.15.12:0/2062892254 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7f34f80145f0 con 0x7f35001066b0 2024-09-11T00:32:11.366 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:32:11.364+0000 7f35088a2740 1 -- start start 2024-09-11T00:32:11.368 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:32:11.364+0000 7f35088a2740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f2ee119790 0x55f2ee11bb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:32:11.368 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:32:11.364+0000 7f35088a2740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f2ee11c0c0 0x55f2ee11e4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:32:11.368 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:32:11.364+0000 7f35088a2740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f2ee11eb00 0x55f2ee120fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:32:11.368 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:32:11.364+0000 7f35088a2740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55f2ee08c6b0 con 0x55f2ee11c0c0 2024-09-11T00:32:11.368 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:32:11.364+0000 7f35088a2740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55f2ee07bae0 con 0x55f2ee119790 2024-09-11T00:32:11.369 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:32:11.364+0000 7f35088a2740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55f2ee07f8f0 con 0x55f2ee11eb00 2024-09-11T00:32:11.369 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:32:11.365+0000 7f34fea00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f2ee11c0c0 0x55f2ee11e4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:32:11.369 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:32:11.365+0000 7f34fea00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f2ee11c0c0 0x55f2ee11e4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:47428/0 (socket says 172.21.15.12:47428) 2024-09-11T00:32:11.369 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:32:11.365+0000 7f34fea00640 1 -- 172.21.15.12:0/1698062453 learned_addr learned my addr 172.21.15.12:0/1698062453 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:32:11.369 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:32:11.365+0000 7f34fe000640 1 --2- 172.21.15.12:0/1698062453 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f2ee119790 0x55f2ee11bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:32:11.369 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:32:11.365+0000 7f34fe000640 1 --2- 172.21.15.12:0/1698062453 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f2ee11eb00 0x55f2ee120fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:32:11.369 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:32:11.365+0000 7f34fea00640 1 -- 172.21.15.12:0/1698062453 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f2ee11eb00 msgr2=0x55f2ee120fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:32:11.369 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:32:11.365+0000 7f34fea00640 1 --2- 172.21.15.12:0/1698062453 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f2ee11eb00 0x55f2ee120fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:32:11.369 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:32:11.365+0000 7f34fea00640 1 -- 172.21.15.12:0/1698062453 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f2ee119790 msgr2=0x55f2ee11bb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:32:11.369 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:32:11.365+0000 7f34fea00640 1 --2- 172.21.15.12:0/1698062453 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f2ee119790 0x55f2ee11bb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:32:11.370 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:32:11.365+0000 7f34fea00640 1 -- 172.21.15.12:0/1698062453 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f2ee048780 con 0x55f2ee11c0c0 2024-09-11T00:32:11.370 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:32:11.365+0000 7f34fea00640 1 --2- 172.21.15.12:0/1698062453 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f2ee11c0c0 0x55f2ee11e4b0 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f34f002aa30 tx=0x7f34f00043b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ab61f636ab660e8 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:32:11.370 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:32:11.366+0000 7f34f7400640 1 --2- 172.21.15.12:0/2062892254 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f34e40b52e0 0x7f34e40b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:32:11.370 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:32:11.366+0000 7f34ebe00640 1 -- 172.21.15.12:0/1698062453 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34f0035030 con 0x55f2ee11c0c0 2024-09-11T00:32:11.370 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:32:11.366+0000 7f34ebe00640 1 -- 172.21.15.12:0/1698062453 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f34f0040070 con 0x55f2ee11c0c0 2024-09-11T00:32:11.370 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:32:11.366+0000 7f34ebe00640 1 -- 172.21.15.12:0/1698062453 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34f0031c10 con 0x55f2ee11c0c0 2024-09-11T00:32:11.370 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:32:11.366+0000 7f34f7400640 1 -- 172.21.15.12:0/2062892254 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9208+0+0 (secure 0 0 0) 0x7f34f80e5740 con 0x7f35001066b0 2024-09-11T00:32:11.370 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:32:11.366+0000 7f34fe000640 1 --2- 172.21.15.12:0/2062892254 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f34e40b52e0 0x7f34e40b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:32:11.374 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:32:11.366+0000 7f35088a2740 1 -- 172.21.15.12:0/1698062453 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f2ee11c0c0 msgr2=0x55f2ee11e4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:32:11.375 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:32:11.366+0000 7f35088a2740 1 --2- 172.21.15.12:0/1698062453 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f2ee11c0c0 0x55f2ee11e4b0 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f34f002aa30 tx=0x7f34f00043b0 comp rx=0 tx=0).stop 2024-09-11T00:32:11.375 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:32:11.367+0000 7f34fe000640 1 --2- 172.21.15.12:0/2062892254 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f34e40b52e0 0x7f34e40b77a0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f34ec010670 tx=0x7f34ec00aa90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:32:11.375 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:32:11.367+0000 7f35088a2740 1 -- 172.21.15.12:0/1698062453 shutdown_connections 2024-09-11T00:32:11.375 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:32:11.367+0000 7f35088a2740 1 --2- 172.21.15.12:0/1698062453 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f2ee11eb00 0x55f2ee120fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:32:11.375 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:32:11.367+0000 7f35088a2740 1 --2- 172.21.15.12:0/1698062453 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f2ee11c0c0 0x55f2ee11e4b0 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:32:11.375 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:32:11.367+0000 7f35088a2740 1 --2- 172.21.15.12:0/1698062453 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f2ee119790 0x55f2ee11bb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:32:11.375 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:32:11.367+0000 7f35088a2740 1 -- 172.21.15.12:0/1698062453 >> 172.21.15.12:0/1698062453 conn(0x55f2ee1148e0 msgr2=0x55f2ee115670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:32:11.375 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:32:11.367+0000 7f35088a2740 1 -- 172.21.15.12:0/1698062453 shutdown_connections 2024-09-11T00:32:11.375 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:32:11.367+0000 7f35088a2740 1 -- 172.21.15.12:0/1698062453 wait complete. 2024-09-11T00:32:11.375 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:32:11.367+0000 7f35088a2740 1 -- start start 2024-09-11T00:32:11.376 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:32:11.367+0000 7f35088a2740 1 Processor -- start 2024-09-11T00:32:11.376 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:32:11.368+0000 7f35088a2740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f2ee119790 0x55f2ee122e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:32:11.376 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:32:11.368+0000 7f35088a2740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f2ee11c0c0 0x55f2ee123530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:32:11.376 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:32:11.368+0000 7f35088a2740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f2ee11eb00 0x55f2ee124100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:32:11.376 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:32:11.368+0000 7f35088a2740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55f2edfb3ab0 con 0x55f2ee119790 2024-09-11T00:32:11.376 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:32:11.368+0000 7f35088a2740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55f2ee022570 con 0x55f2ee11eb00 2024-09-11T00:32:11.376 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:32:11.368+0000 7f35088a2740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x55f2ee121a80 con 0x55f2ee11c0c0 2024-09-11T00:32:11.376 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:32:11.368+0000 7f35088a2740 10 client.0 _ll_register_callbacks cb 0x55f2ee031100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:32:11.376 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:32:11.368+0000 7f34fea00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f2ee119790 0x55f2ee122e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:32:11.376 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:32:11.368+0000 7f34fea00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f2ee119790 0x55f2ee122e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:47442/0 (socket says 172.21.15.12:47442) 2024-09-11T00:32:11.376 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:32:11.368+0000 7f34fea00640 1 -- 172.21.15.12:0/2423063481 learned_addr learned my addr 172.21.15.12:0/2423063481 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:32:11.376 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:32:11.368+0000 7f34fe000640 1 --2- 172.21.15.12:0/2423063481 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f2ee11c0c0 0x55f2ee123530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:32:11.377 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:32:11.368+0000 7f34fea00640 1 -- 172.21.15.12:0/2423063481 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f2ee11c0c0 msgr2=0x55f2ee123530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:32:11.377 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:32:11.368+0000 7f34fea00640 1 --2- 172.21.15.12:0/2423063481 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f2ee11c0c0 0x55f2ee123530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:32:11.377 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:32:11.368+0000 7f34fea00640 1 -- 172.21.15.12:0/2423063481 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f2ee11eb00 msgr2=0x55f2ee124100 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-11T00:32:11.377 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:32:11.368+0000 7f34fea00640 1 --2- 172.21.15.12:0/2423063481 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f2ee11eb00 0x55f2ee124100 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:32:11.377 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:32:11.368+0000 7f34fea00640 1 -- 172.21.15.12:0/2423063481 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f2edddd790 con 0x55f2ee119790 2024-09-11T00:32:11.377 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:32:11.369+0000 7f34fea00640 1 --2- 172.21.15.12:0/2423063481 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f2ee119790 0x55f2ee122e90 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f34f002a480 tx=0x7f34f0004180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df42a8b8a59ee5fa server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:32:11.377 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:32:11.369+0000 7f34e8c00640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-09-11T00:32:11.377 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:32:11.369+0000 7f34e8c00640 1 -- 172.21.15.12:0/2423063481 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34f0053060 con 0x55f2ee119790 2024-09-11T00:32:11.377 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:32:11.369+0000 7f34e8c00640 1 -- 172.21.15.12:0/2423063481 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 9208+0+0 (secure 0 0 0) 0x7f34f00037f0 con 0x55f2ee119790 2024-09-11T00:32:11.377 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:32:11.369+0000 7f35088a2740 1 -- 172.21.15.12:0/2423063481 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f34f0040070 con 0x55f2ee119790 2024-09-11T00:32:11.377 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:32:11.369+0000 7f34e8c00640 1 -- 172.21.15.12:0/2423063481 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=107) ==== 24+0+0 (secure 0 0 0) 0x7f34f005a050 con 0x55f2ee119790 2024-09-11T00:32:11.378 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:32:11.369+0000 7f35088a2740 10 client.18150 fetch_fsmap learned FSMap version 107 2024-09-11T00:32:11.378 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:32:11.369+0000 7f35088a2740 1 -- 172.21.15.12:0/2423063481 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=107,monmap=4+}) -- 0x55f2ede56620 con 0x55f2ee119790 2024-09-11T00:32:11.378 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:32:11.369+0000 7f34e8c00640 1 -- 172.21.15.12:0/2423063481 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f34f0040070 con 0x55f2ee119790 2024-09-11T00:32:11.378 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:32:11.370+0000 7f34e8c00640 1 -- 172.21.15.12:0/2423063481 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 107) ==== 8390+0+0 (secure 0 0 0) 0x7f34f004d030 con 0x55f2ee119790 2024-09-11T00:32:11.378 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:32:11.370+0000 7f35088a2740 10 client.18150 fetch_fsmap finished waiting for FSMap version 107 2024-09-11T00:32:11.378 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:32:11.370+0000 7f35088a2740 10 client.18150 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T00:32:11.378 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:32:11.370+0000 7f35088a2740 20 client.18150 populate_metadata read hostname 'smithi012' 2024-09-11T00:32:11.378 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:32:11.370+0000 7f35088a2740 1 --2- 172.21.15.12:0/2423063481 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55f2ee12e330 0x55f2ee130720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:32:11.378 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:32:11.370+0000 7f35088a2740 4 client.18150 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T00:32:11.378 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:32:11.370+0000 7f35088a2740 1 -- 172.21.15.12:0/2423063481 --> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f2ee08e600 con 0x55f2ee12e330 2024-09-11T00:32:11.378 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:32:11.370+0000 7f34fe000640 1 --2- 172.21.15.12:0/2423063481 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55f2ee12e330 0x55f2ee130720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:32:11.378 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:32:11.371+0000 7f34fe000640 1 --2- 172.21.15.12:0/2423063481 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55f2ee12e330 0x55f2ee130720 crc :-1 s=READY pgs=339 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c24474dcaf99f4ff server_cookie=e7ec25cc619e52a5 in_seq=0 out_seq=0 2024-09-11T00:32:11.378 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:32:11.371+0000 7f34e8c00640 10 client.18150 ms_handle_connect on v2:172.21.15.165:6838/501312984 2024-09-11T00:32:11.379 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:32:11.372+0000 7f34e8c00640 1 -- 172.21.15.12:0/2423063481 <== mds.0 v2:172.21.15.165:6838/501312984 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55f2ee08e600 con 0x55f2ee12e330 2024-09-11T00:32:11.379 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:32:11.372+0000 7f34e8c00640 10 client.18150 handle_command_reply: tid=0 2024-09-11T00:32:11.379 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:32:11.372+0000 7f35088a2740 1 client.18150 shutdown 2024-09-11T00:32:11.379 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:32:11.372+0000 7f35088a2740 1 -- 172.21.15.12:0/2423063481 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f2ee119790 msgr2=0x55f2ee122e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:32:11.379 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:32:11.372+0000 7f35088a2740 1 --2- 172.21.15.12:0/2423063481 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f2ee119790 0x55f2ee122e90 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f34f002a480 tx=0x7f34f0004180 comp rx=0 tx=0).stop 2024-09-11T00:32:11.379 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:32:11.372+0000 7f35088a2740 1 -- 172.21.15.12:0/2423063481 shutdown_connections 2024-09-11T00:32:11.379 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:32:11.372+0000 7f35088a2740 1 --2- 172.21.15.12:0/2423063481 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55f2ee12e330 0x55f2ee130720 crc :-1 s=READY pgs=339 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:32:11.379 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:32:11.372+0000 7f35088a2740 1 --2- 172.21.15.12:0/2423063481 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f2ee11eb00 0x55f2ee124100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:32:11.379 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:32:11.372+0000 7f35088a2740 1 --2- 172.21.15.12:0/2423063481 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f2ee11c0c0 0x55f2ee123530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:32:11.379 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:32:11.372+0000 7f35088a2740 1 --2- 172.21.15.12:0/2423063481 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f2ee119790 0x55f2ee122e90 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:32:11.379 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:32:11.372+0000 7f34e8c00640 0 client.18150 ms_handle_reset on v2:172.21.15.165:6838/501312984 2024-09-11T00:32:11.379 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:32:11.372+0000 7f35088a2740 1 -- 172.21.15.12:0/2423063481 >> 172.21.15.12:0/2423063481 conn(0x55f2ee1148e0 msgr2=0x55f2ee113a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:32:11.379 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:32:11.372+0000 7f35088a2740 1 -- 172.21.15.12:0/2423063481 shutdown_connections 2024-09-11T00:32:11.379 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:32:11.372+0000 7f35088a2740 1 -- 172.21.15.12:0/2423063481 wait complete. 2024-09-11T00:32:11.379 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:32:11.372+0000 7f35088a2740 20 client.18150 trim_cache size 0 max 16384 2024-09-11T00:32:11.391 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:32:11.388+0000 7f35088a2740 1 Processor -- start 2024-09-11T00:32:11.391 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:32:11.388+0000 7f35088a2740 1 -- start start 2024-09-11T00:32:11.391 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:32:11.388+0000 7f35088a2740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f2ee119790 0x55f2ee115330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:32:11.391 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:32:11.388+0000 7f35088a2740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f2ee11c0c0 0x55f2ee115870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:32:11.391 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:32:11.388+0000 7f35088a2740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f2ee11eb00 0x55f2ee11eee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:32:11.391 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:32:11.388+0000 7f35088a2740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f35001ad880 con 0x55f2ee11c0c0 2024-09-11T00:32:11.391 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:32:11.388+0000 7f35088a2740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f35001af3c0 con 0x55f2ee11eb00 2024-09-11T00:32:11.391 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:32:11.388+0000 7f35088a2740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f35001adf50 con 0x55f2ee119790 2024-09-11T00:32:11.392 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:32:11.389+0000 7f34fe000640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f2ee11c0c0 0x55f2ee115870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:32:11.392 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:32:11.389+0000 7f34fe000640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f2ee11c0c0 0x55f2ee115870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:47454/0 (socket says 172.21.15.12:47454) 2024-09-11T00:32:11.392 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:32:11.389+0000 7f34fe000640 1 -- 172.21.15.12:0/4184082469 learned_addr learned my addr 172.21.15.12:0/4184082469 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:32:11.399 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:32:11.389+0000 7f34fe000640 1 -- 172.21.15.12:0/4184082469 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f2ee119790 msgr2=0x55f2ee115330 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-11T00:32:11.399 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:32:11.389+0000 7f34fe000640 1 --2- 172.21.15.12:0/4184082469 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f2ee119790 0x55f2ee115330 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:32:11.399 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:32:11.389+0000 7f34ff400640 1 --2- 172.21.15.12:0/4184082469 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f2ee11eb00 0x55f2ee11eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:32:11.399 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:32:11.389+0000 7f34fe000640 1 -- 172.21.15.12:0/4184082469 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f2ee11eb00 msgr2=0x55f2ee11eee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:32:11.399 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:32:11.389+0000 7f34fe000640 1 --2- 172.21.15.12:0/4184082469 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f2ee11eb00 0x55f2ee11eee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:32:11.399 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:32:11.389+0000 7f34fe000640 1 -- 172.21.15.12:0/4184082469 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f2ee12b4f0 con 0x55f2ee11c0c0 2024-09-11T00:32:11.399 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:32:11.389+0000 7f34ff400640 1 --2- 172.21.15.12:0/4184082469 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f2ee11eb00 0x55f2ee11eee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T00:32:11.399 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:32:11.389+0000 7f34fe000640 1 --2- 172.21.15.12:0/4184082469 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f2ee11c0c0 0x55f2ee115870 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f34ec005e10 tx=0x7f34ec017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22d5b7fe6eeed073 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:32:11.399 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:32:11.390+0000 7f34eb400640 1 -- 172.21.15.12:0/4184082469 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34ec00fe20 con 0x55f2ee11c0c0 2024-09-11T00:32:11.399 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:32:11.390+0000 7f34eb400640 1 -- 172.21.15.12:0/4184082469 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f34ec01fd30 con 0x55f2ee11c0c0 2024-09-11T00:32:11.400 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:32:11.390+0000 7f34eb400640 1 -- 172.21.15.12:0/4184082469 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34ec004a00 con 0x55f2ee11c0c0 2024-09-11T00:32:11.400 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:32:11.390+0000 7f35088a2740 1 -- 172.21.15.12:0/4184082469 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f2ee11c0c0 msgr2=0x55f2ee115870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:32:11.400 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:32:11.390+0000 7f35088a2740 1 --2- 172.21.15.12:0/4184082469 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f2ee11c0c0 0x55f2ee115870 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f34ec005e10 tx=0x7f34ec017000 comp rx=0 tx=0).stop 2024-09-11T00:32:11.400 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:32:11.390+0000 7f35088a2740 1 -- 172.21.15.12:0/4184082469 shutdown_connections 2024-09-11T00:32:11.400 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:32:11.390+0000 7f35088a2740 1 --2- 172.21.15.12:0/4184082469 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f2ee11eb00 0x55f2ee11eee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:32:11.400 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:32:11.390+0000 7f35088a2740 1 --2- 172.21.15.12:0/4184082469 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f2ee11c0c0 0x55f2ee115870 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:32:11.400 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:32:11.390+0000 7f35088a2740 1 --2- 172.21.15.12:0/4184082469 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f2ee119790 0x55f2ee115330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:32:11.400 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:32:11.390+0000 7f35088a2740 1 -- 172.21.15.12:0/4184082469 >> 172.21.15.12:0/4184082469 conn(0x55f2ee1148e0 msgr2=0x55f2ee118c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:32:11.400 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:32:11.390+0000 7f35088a2740 1 -- 172.21.15.12:0/4184082469 shutdown_connections 2024-09-11T00:32:11.400 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:32:11.390+0000 7f35088a2740 1 -- 172.21.15.12:0/4184082469 wait complete. 2024-09-11T00:32:11.400 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:32:11.391+0000 7f35088a2740 1 -- start start 2024-09-11T00:32:11.400 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:32:11.391+0000 7f35088a2740 1 Processor -- start 2024-09-11T00:32:11.400 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:32:11.392+0000 7f35088a2740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f2ee119790 0x55f2ee12ac70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:32:11.401 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:32:11.392+0000 7f35088a2740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f2ee11c0c0 0x55f2ee129ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:32:11.401 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:32:11.392+0000 7f35088a2740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f2ee11eb00 0x55f2ee12a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:32:11.401 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:32:11.392+0000 7f35088a2740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f35001aecf0 con 0x55f2ee11c0c0 2024-09-11T00:32:11.401 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:32:11.392+0000 7f35088a2740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f35001ae620 con 0x55f2ee11eb00 2024-09-11T00:32:11.401 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:32:11.392+0000 7f35088a2740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f35001ab860 con 0x55f2ee119790 2024-09-11T00:32:11.401 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:32:11.392+0000 7f35088a2740 10 client.0 _ll_register_callbacks cb 0x7f34f0004280 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:32:11.401 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:32:11.392+0000 7f34fe000640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f2ee11c0c0 0x55f2ee129ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:32:11.401 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:32:11.392+0000 7f34fe000640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f2ee11c0c0 0x55f2ee129ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:47458/0 (socket says 172.21.15.12:47458) 2024-09-11T00:32:11.401 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:32:11.392+0000 7f34fe000640 1 -- 172.21.15.12:0/1107063162 learned_addr learned my addr 172.21.15.12:0/1107063162 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:32:11.401 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:32:11.392+0000 7f34ff400640 1 --2- 172.21.15.12:0/1107063162 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f2ee11eb00 0x55f2ee12a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:32:11.401 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:32:11.392+0000 7f34fe000640 1 -- 172.21.15.12:0/1107063162 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f2ee119790 msgr2=0x55f2ee12ac70 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-11T00:32:11.401 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:32:11.392+0000 7f34fe000640 1 --2- 172.21.15.12:0/1107063162 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f2ee119790 0x55f2ee12ac70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:32:11.402 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:32:11.392+0000 7f34fe000640 1 -- 172.21.15.12:0/1107063162 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f2ee11eb00 msgr2=0x55f2ee12a1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:32:11.402 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:32:11.392+0000 7f34fe000640 1 --2- 172.21.15.12:0/1107063162 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f2ee11eb00 0x55f2ee12a1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:32:11.402 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:32:11.392+0000 7f34fe000640 1 -- 172.21.15.12:0/1107063162 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f2ee128670 con 0x55f2ee11c0c0 2024-09-11T00:32:11.402 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:32:11.393+0000 7f34fe000640 1 --2- 172.21.15.12:0/1107063162 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f2ee11c0c0 0x55f2ee129ab0 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f34ec00ff80 tx=0x7f34ec01f880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c4572b355c5b319 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:32:11.402 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:32:11.393+0000 7f34e8c00640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-09-11T00:32:11.402 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:32:11.393+0000 7f34e8c00640 1 -- 172.21.15.12:0/1107063162 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34ec031430 con 0x55f2ee11c0c0 2024-09-11T00:32:11.402 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:32:11.393+0000 7f34e8c00640 1 -- 172.21.15.12:0/1107063162 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 9208+0+0 (secure 0 0 0) 0x7f34ec0315d0 con 0x55f2ee11c0c0 2024-09-11T00:32:11.402 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:32:11.393+0000 7f35088a2740 1 -- 172.21.15.12:0/1107063162 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f34ec01fd30 con 0x55f2ee11c0c0 2024-09-11T00:32:11.402 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:32:11.393+0000 7f34e8c00640 1 -- 172.21.15.12:0/1107063162 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=107) ==== 24+0+0 (secure 0 0 0) 0x7f34ec0317d0 con 0x55f2ee11c0c0 2024-09-11T00:32:11.402 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:32:11.393+0000 7f35088a2740 10 client.18156 fetch_fsmap learned FSMap version 107 2024-09-11T00:32:11.402 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:32:11.393+0000 7f35088a2740 1 -- 172.21.15.12:0/1107063162 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=107,monmap=4+}) -- 0x55f2ee1a3cc0 con 0x55f2ee11c0c0 2024-09-11T00:32:11.402 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:32:11.393+0000 7f34e8c00640 1 -- 172.21.15.12:0/1107063162 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f34ec01fd30 con 0x55f2ee11c0c0 2024-09-11T00:32:11.402 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:32:11.394+0000 7f34e8c00640 1 -- 172.21.15.12:0/1107063162 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 107) ==== 8390+0+0 (secure 0 0 0) 0x7f34ec029ac0 con 0x55f2ee11c0c0 2024-09-11T00:32:11.403 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:32:11.394+0000 7f35088a2740 10 client.18156 fetch_fsmap finished waiting for FSMap version 107 2024-09-11T00:32:11.403 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:32:11.394+0000 7f35088a2740 10 client.18156 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T00:32:11.403 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:32:11.394+0000 7f35088a2740 20 client.18156 populate_metadata read hostname 'smithi012' 2024-09-11T00:32:11.403 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:32:11.394+0000 7f35088a2740 1 --2- 172.21.15.12:0/1107063162 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55f2ee1a46a0 0x55f2ee1a6ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:32:11.403 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:32:11.394+0000 7f35088a2740 4 client.18156 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T00:32:11.403 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:32:11.394+0000 7f35088a2740 1 -- 172.21.15.12:0/1107063162 --> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55f2ee08a0b0 con 0x55f2ee1a46a0 2024-09-11T00:32:11.414 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:32:11.412+0000 7f34fea00640 1 --2- 172.21.15.12:0/1107063162 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55f2ee1a46a0 0x55f2ee1a6ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:32:11.461 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:32:11.459+0000 7f34fea00640 1 --2- 172.21.15.12:0/1107063162 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55f2ee1a46a0 0x55f2ee1a6ac0 crc :-1 s=READY pgs=340 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=51650e643d94f0a4 server_cookie=f8f11025d0f5ee79 in_seq=0 out_seq=0 2024-09-11T00:32:11.462 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:32:11.459+0000 7f34e8c00640 10 client.18156 ms_handle_connect on v2:172.21.15.165:6838/501312984 2024-09-11T00:32:11.498 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:32:11.495+0000 7f34e8c00640 1 -- 172.21.15.12:0/1107063162 <== mds.0 v2:172.21.15.165:6838/501312984 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55f2ee08a0b0 con 0x55f2ee1a46a0 2024-09-11T00:32:11.498 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:32:11.495+0000 7f34e8c00640 10 client.18156 handle_command_reply: tid=0 2024-09-11T00:32:11.498 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:32:11.495+0000 7f35088a2740 1 client.18156 shutdown 2024-09-11T00:32:11.499 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:32:11.495+0000 7f35088a2740 1 -- 172.21.15.12:0/1107063162 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f2ee11c0c0 msgr2=0x55f2ee129ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:32:11.499 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:32:11.495+0000 7f35088a2740 1 --2- 172.21.15.12:0/1107063162 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f2ee11c0c0 0x55f2ee129ab0 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f34ec00ff80 tx=0x7f34ec01f880 comp rx=0 tx=0).stop 2024-09-11T00:32:11.499 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:32:11.495+0000 7f35088a2740 1 -- 172.21.15.12:0/1107063162 shutdown_connections 2024-09-11T00:32:11.499 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:32:11.495+0000 7f35088a2740 1 --2- 172.21.15.12:0/1107063162 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x55f2ee1a46a0 0x55f2ee1a6ac0 crc :-1 s=READY pgs=340 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:32:11.499 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:32:11.496+0000 7f35088a2740 1 --2- 172.21.15.12:0/1107063162 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f2ee11eb00 0x55f2ee12a1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:32:11.499 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:32:11.496+0000 7f35088a2740 1 --2- 172.21.15.12:0/1107063162 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f2ee11c0c0 0x55f2ee129ab0 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:32:11.499 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:32:11.496+0000 7f35088a2740 1 --2- 172.21.15.12:0/1107063162 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f2ee119790 0x55f2ee12ac70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:32:11.499 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:32:11.496+0000 7f35088a2740 1 -- 172.21.15.12:0/1107063162 >> 172.21.15.12:0/1107063162 conn(0x55f2ee1148e0 msgr2=0x55f2ee11eee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:32:11.499 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:32:11.496+0000 7f34e8c00640 0 client.18156 ms_handle_reset on v2:172.21.15.165:6838/501312984 2024-09-11T00:32:11.499 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:32:11.496+0000 7f35088a2740 1 -- 172.21.15.12:0/1107063162 shutdown_connections 2024-09-11T00:32:11.499 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:32:11.496+0000 7f35088a2740 1 -- 172.21.15.12:0/1107063162 wait complete. 2024-09-11T00:32:11.499 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:32:11.496+0000 7f35088a2740 20 client.18156 trim_cache size 0 max 16384 2024-09-11T00:32:11.500 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:{ 2024-09-11T00:32:11.500 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "message": "", 2024-09-11T00:32:11.500 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "return_code": 0 2024-09-11T00:32:11.500 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:} 2024-09-11T00:32:11.501 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:32:11.496+0000 7f3505a00640 1 -- 172.21.15.12:0/2062892254 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f34e40b52e0 msgr2=0x7f34e40b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:32:11.501 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:32:11.496+0000 7f3505a00640 1 --2- 172.21.15.12:0/2062892254 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f34e40b52e0 0x7f34e40b77a0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f34ec010670 tx=0x7f34ec00aa90 comp rx=0 tx=0).stop 2024-09-11T00:32:11.501 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:32:11.496+0000 7f3505a00640 1 -- 172.21.15.12:0/2062892254 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f35001066b0 msgr2=0x7f350019c1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:32:11.501 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:32:11.496+0000 7f3505a00640 1 --2- 172.21.15.12:0/2062892254 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f35001066b0 0x7f350019c1d0 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f34f800b970 tx=0x7f34f800bd70 comp rx=0 tx=0).stop 2024-09-11T00:32:11.501 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:32:11.497+0000 7f3505a00640 1 -- 172.21.15.12:0/2062892254 shutdown_connections 2024-09-11T00:32:11.501 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:32:11.497+0000 7f3505a00640 1 --2- 172.21.15.12:0/2062892254 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f34e40b52e0 0x7f34e40b77a0 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:32:11.501 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:32:11.497+0000 7f3505a00640 1 --2- 172.21.15.12:0/2062892254 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f35001066b0 0x7f350019c1d0 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:32:11.501 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:32:11.497+0000 7f3505a00640 1 --2- 172.21.15.12:0/2062892254 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3500105cf0 0x7f350019b5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:32:11.501 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:32:11.497+0000 7f3505a00640 1 --2- 172.21.15.12:0/2062892254 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3500104ba0 0x7f350019af50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:32:11.502 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:32:11.497+0000 7f3505a00640 1 -- 172.21.15.12:0/2062892254 >> 172.21.15.12:0/2062892254 conn(0x7f3500100460 msgr2=0x7f3500109010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:32:11.502 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:32:11.497+0000 7f3505a00640 1 -- 172.21.15.12:0/2062892254 shutdown_connections 2024-09-11T00:32:11.502 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-09-11T00:32:11.497+0000 7f3505a00640 1 -- 172.21.15.12:0/2062892254 wait complete. 2024-09-11T00:32:12.017 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:32:11 smithi012 ceph-mon[30333]: pgmap v1153: 129 pgs: 129 active+clean; 880 MiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 27 KiB/s wr, 51 op/s 2024-09-11T00:32:12.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:32:11 smithi165 ceph-mon[35356]: pgmap v1153: 129 pgs: 129 active+clean; 880 MiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 27 KiB/s wr, 51 op/s 2024-09-11T00:32:12.320 INFO:tasks.workunit.client.0.smithi012.stdout: 10 cleanup 608 sec 2024-09-11T00:32:12.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:32:11 smithi178 ceph-mon[35919]: pgmap v1153: 129 pgs: 129 active+clean; 880 MiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 27 KiB/s wr, 51 op/s 2024-09-11T00:32:13.065 INFO:tasks.workunit.client.0.smithi012.stdout: 0 cleanup 609 sec 2024-09-11T00:32:13.065 INFO:tasks.workunit.client.0.smithi012.stdout: 2024-09-11T00:32:13.065 INFO:tasks.workunit.client.0.smithi012.stdout: Operation Count AvgLat MaxLat 2024-09-11T00:32:13.065 INFO:tasks.workunit.client.0.smithi012.stdout: ---------------------------------------- 2024-09-11T00:32:13.065 INFO:tasks.workunit.client.0.smithi012.stdout: NTCreateX 297799 9.106 29155.612 2024-09-11T00:32:13.065 INFO:tasks.workunit.client.0.smithi012.stdout: Close 218530 0.008 2.500 2024-09-11T00:32:13.065 INFO:tasks.workunit.client.0.smithi012.stdout: Rename 12624 43.392 29121.619 2024-09-11T00:32:13.066 INFO:tasks.workunit.client.0.smithi012.stdout: Unlink 60303 17.327 31893.824 2024-09-11T00:32:13.066 INFO:tasks.workunit.client.0.smithi012.stdout: Qpathinfo 270010 3.967 29850.556 2024-09-11T00:32:13.066 INFO:tasks.workunit.client.0.smithi012.stdout: Qfileinfo 47049 0.003 11.575 2024-09-11T00:32:13.066 INFO:tasks.workunit.client.0.smithi012.stdout: Qfsinfo 49565 0.650 610.565 2024-09-11T00:32:13.066 INFO:tasks.workunit.client.0.smithi012.stdout: Sfileinfo 24246 9.556 28873.588 2024-09-11T00:32:13.066 INFO:tasks.workunit.client.0.smithi012.stdout: Find 104360 1.213 22929.000 2024-09-11T00:32:13.066 INFO:tasks.workunit.client.0.smithi012.stdout: WriteX 147105 0.270 21473.041 2024-09-11T00:32:13.066 INFO:tasks.workunit.client.0.smithi012.stdout: ReadX 467418 0.147 2853.074 2024-09-11T00:32:13.066 INFO:tasks.workunit.client.0.smithi012.stdout: LockX 972 2.108 141.507 2024-09-11T00:32:13.066 INFO:tasks.workunit.client.0.smithi012.stdout: UnlockX 972 1.520 16.178 2024-09-11T00:32:13.066 INFO:tasks.workunit.client.0.smithi012.stdout: Flush 20869 18.012 10310.873 2024-09-11T00:32:13.067 INFO:tasks.workunit.client.0.smithi012.stdout: 2024-09-11T00:32:13.067 INFO:tasks.workunit.client.0.smithi012.stdout:Throughput 15.5023 MB/sec 10 clients 10 procs max_latency=31893.834 ms 2024-09-11T00:32:13.070 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-09-11T00:32:13.071 DEBUG:teuthology.orchestra.run.smithi012:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-09-11T00:32:14.063 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:32:13 smithi012 ceph-mon[30333]: pgmap v1154: 129 pgs: 129 active+clean; 880 MiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 32 KiB/s wr, 37 op/s 2024-09-11T00:32:14.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:32:13 smithi165 ceph-mon[35356]: pgmap v1154: 129 pgs: 129 active+clean; 880 MiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 32 KiB/s wr, 37 op/s 2024-09-11T00:32:14.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:32:13 smithi178 ceph-mon[35919]: pgmap v1154: 129 pgs: 129 active+clean; 880 MiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 32 KiB/s wr, 37 op/s 2024-09-11T00:32:15.567 INFO:tasks.workunit:Stopping ['suites/dbench.sh'] on client.0... 2024-09-11T00:32:15.567 DEBUG:teuthology.orchestra.run.smithi012:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-09-11T00:32:16.283 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:32:15 smithi165 ceph-mon[35356]: pgmap v1155: 129 pgs: 129 active+clean; 861 MiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 53 KiB/s wr, 36 op/s 2024-09-11T00:32:16.283 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:32:15 smithi165 ceph-mon[35356]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T00:32:16.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:32:15 smithi012 ceph-mon[30333]: pgmap v1155: 129 pgs: 129 active+clean; 861 MiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 53 KiB/s wr, 36 op/s 2024-09-11T00:32:16.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:32:15 smithi012 ceph-mon[30333]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T00:32:16.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:32:15 smithi178 ceph-mon[35919]: pgmap v1155: 129 pgs: 129 active+clean; 861 MiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 53 KiB/s wr, 36 op/s 2024-09-11T00:32:16.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:32:15 smithi178 ceph-mon[35919]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T00:32:16.833 DEBUG:teuthology.parallel:result is None 2024-09-11T00:32:16.834 DEBUG:teuthology.orchestra.run.smithi012:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2024-09-11T00:32:16.898 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2024-09-11T00:32:16.898 DEBUG:teuthology.run_tasks:Unwinding manager quiescer 2024-09-11T00:32:16.909 INFO:tasks.quiescer:joining Quiescers 2024-09-11T00:32:16.909 WARNING:tasks.quiescer:The quiescer is requested to stop, running cancel all 2024-09-11T00:32:16.910 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid ff3ad74a-6fcf-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T00:32:17.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:32:16 smithi165 ceph-mon[35356]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T00:32:17.291 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:32:17.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:32:16 smithi012 ceph-mon[30333]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T00:32:17.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:32:16 smithi178 ceph-mon[35919]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T00:32:17.986 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:17.980+0000 7f7684c00640 1 -- 172.21.15.12:0/1136006829 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7680075f40 msgr2=0x7f768010ec50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:32:17.986 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:17.980+0000 7f7684c00640 1 --2- 172.21.15.12:0/1136006829 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7680075f40 0x7f768010ec50 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f7670009860 tx=0x7f767002a340 comp rx=0 tx=0).stop 2024-09-11T00:32:17.987 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:17.980+0000 7f7684c00640 1 -- 172.21.15.12:0/1136006829 shutdown_connections 2024-09-11T00:32:17.987 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:17.980+0000 7f7684c00640 1 --2- 172.21.15.12:0/1136006829 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f768010f220 0x7f7680111610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:32:17.987 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:17.980+0000 7f7684c00640 1 --2- 172.21.15.12:0/1136006829 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7680075f40 0x7f768010ec50 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:32:17.987 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:17.980+0000 7f7684c00640 1 --2- 172.21.15.12:0/1136006829 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7680075540 0x7f7680075920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:32:17.987 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:17.980+0000 7f7684c00640 1 -- 172.21.15.12:0/1136006829 >> 172.21.15.12:0/1136006829 conn(0x7f76800fe6d0 msgr2=0x7f7680100af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:32:17.987 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:17.980+0000 7f7684c00640 1 -- 172.21.15.12:0/1136006829 shutdown_connections 2024-09-11T00:32:17.987 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:17.980+0000 7f7684c00640 1 -- 172.21.15.12:0/1136006829 wait complete. 2024-09-11T00:32:17.988 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:17.980+0000 7f7684c00640 1 Processor -- start 2024-09-11T00:32:17.988 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:17.980+0000 7f7684c00640 1 -- start start 2024-09-11T00:32:17.988 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:17.980+0000 7f7684c00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7680075540 0x7f768019b240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:32:17.988 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:17.980+0000 7f7684c00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7680075f40 0x7f768019b780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:32:17.988 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:17.980+0000 7f7684c00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f768010f220 0x7f76801a0b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:32:17.988 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:17.980+0000 7f7684c00640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f7680114dd0 con 0x7f768010f220 2024-09-11T00:32:17.989 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:17.980+0000 7f7684c00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f7680114c50 con 0x7f7680075f40 2024-09-11T00:32:17.989 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:17.980+0000 7f7684c00640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f7680114f50 con 0x7f7680075540 2024-09-11T00:32:17.990 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:17.986+0000 7f767ea00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7680075f40 0x7f768019b780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:32:17.990 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:17.986+0000 7f767ea00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7680075f40 0x7f768019b780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.12:40274/0 (socket says 172.21.15.12:40274) 2024-09-11T00:32:17.991 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:17.986+0000 7f767ea00640 1 -- 172.21.15.12:0/3812829407 learned_addr learned my addr 172.21.15.12:0/3812829407 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:32:17.991 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:17.986+0000 7f767fe00640 1 --2- 172.21.15.12:0/3812829407 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f768010f220 0x7f76801a0b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:32:17.991 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:17.986+0000 7f767fe00640 1 -- 172.21.15.12:0/3812829407 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7680075540 msgr2=0x7f768019b240 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-11T00:32:17.991 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:17.986+0000 7f767fe00640 1 --2- 172.21.15.12:0/3812829407 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7680075540 0x7f768019b240 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:32:17.991 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:17.986+0000 7f767fe00640 1 -- 172.21.15.12:0/3812829407 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7680075f40 msgr2=0x7f768019b780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:32:17.991 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:17.986+0000 7f767fe00640 1 --2- 172.21.15.12:0/3812829407 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7680075f40 0x7f768019b780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:32:17.991 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:17.986+0000 7f767fe00640 1 -- 172.21.15.12:0/3812829407 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76801a12f0 con 0x7f768010f220 2024-09-11T00:32:17.991 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:17.986+0000 7f767ea00640 1 --2- 172.21.15.12:0/3812829407 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7680075f40 0x7f768019b780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:32:17.991 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:17.986+0000 7f767fe00640 1 --2- 172.21.15.12:0/3812829407 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f768010f220 0x7f76801a0b20 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f766c00e940 tx=0x7f766c00ee00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:32:17.991 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:17.986+0000 7f7677e00640 1 -- 172.21.15.12:0/3812829407 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f766c00ccc0 con 0x7f768010f220 2024-09-11T00:32:17.991 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:17.986+0000 7f7677e00640 1 -- 172.21.15.12:0/3812829407 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f766c005020 con 0x7f768010f220 2024-09-11T00:32:17.991 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:17.986+0000 7f7677e00640 1 -- 172.21.15.12:0/3812829407 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f766c004a00 con 0x7f768010f220 2024-09-11T00:32:17.991 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:17.986+0000 7f7684c00640 1 -- 172.21.15.12:0/3812829407 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f76801a15e0 con 0x7f768010f220 2024-09-11T00:32:17.992 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:17.986+0000 7f7684c00640 1 -- 172.21.15.12:0/3812829407 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76801a7ca0 con 0x7f768010f220 2024-09-11T00:32:17.992 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:17.986+0000 7f7684c00640 1 -- 172.21.15.12:0/3812829407 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7680075920 con 0x7f768010f220 2024-09-11T00:32:17.994 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:17.992+0000 7f7677e00640 1 -- 172.21.15.12:0/3812829407 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7f766c004ba0 con 0x7f768010f220 2024-09-11T00:32:17.996 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:17.993+0000 7f7677e00640 1 --2- 172.21.15.12:0/3812829407 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f76640b5390 0x7f76640b7850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:32:17.996 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:17.994+0000 7f7677e00640 1 -- 172.21.15.12:0/3812829407 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9208+0+0 (secure 0 0 0) 0x7f766c0e60b0 con 0x7f768010f220 2024-09-11T00:32:17.996 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:17.994+0000 7f767f400640 1 --2- 172.21.15.12:0/3812829407 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f76640b5390 0x7f76640b7850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:32:17.997 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:17.994+0000 7f767f400640 1 --2- 172.21.15.12:0/3812829407 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f76640b5390 0x7f76640b7850 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f7678009a60 tx=0x7f76780095c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:32:18.003 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:17.995+0000 7f7677e00640 1 -- 172.21.15.12:0/3812829407 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f766c0acbd0 con 0x7f768010f220 2024-09-11T00:32:18.063 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:32:17 smithi012 ceph-mon[30333]: pgmap v1156: 129 pgs: 129 active+clean; 691 MiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 264 KiB/s rd, 313 KiB/s wr, 153 op/s 2024-09-11T00:32:18.063 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:32:17 smithi012 ceph-mon[30333]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T00:32:18.248 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:18.246+0000 7f7684c00640 1 -- 172.21.15.12:0/3812829407 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f768010a310 con 0x7f768010f220 2024-09-11T00:32:18.249 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:18.246+0000 7f7677e00640 1 -- 172.21.15.12:0/3812829407 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 111 v111) ==== 95+0+11999 (secure 0 0 0) 0x7f766c0cde20 con 0x7f768010f220 2024-09-11T00:32:18.249 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-11T00:32:18.249 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":111,"btime":"2024-09-11T00:32:17:958816+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14577,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6835/447171591","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":447171591},{"type":"v1","addr":"172.21.15.12:6835","nonce":447171591}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14589,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/901486040","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":901486040},{"type":"v1","addr":"172.21.15.12:6837","nonce":901486040}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":111,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T00:04:04.739881+0000","modified":"2024-09-11T00:32:17.258471+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24674,"mds_1":24643,"mds_2":24638,"mds_3":24637,"mds_4":24614},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14631":{"gid":14631,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.12:6841/3069919437","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6840","nonce":3069919437},{"type":"v1","addr":"172.21.15.12:6841","nonce":3069919437}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24562":{"gid":24562,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6833/1141636112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":1141636112},{"type":"v1","addr":"172.21.15.178:6833","nonce":1141636112}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24581":{"gid":24581,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6833/3111980970","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6832","nonce":3111980970},{"type":"v1","addr":"172.21.15.165:6833","nonce":3111980970}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24607":{"gid":24607,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6835/4084868287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":4084868287},{"type":"v1","addr":"172.21.15.178:6835","nonce":4084868287}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24614":{"gid":24614,"name":"e","rank":4,"incarnation":26,"state":"up:active","state_seq":21,"addr":"172.21.15.165:6835/2425395274","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":2425395274},{"type":"v1","addr":"172.21.15.165:6835","nonce":2425395274}]},"join_fscid":-1,"export_targets":[0,1,2,3],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24637":{"gid":24637,"name":"i","rank":3,"incarnation":24,"state":"up:active","state_seq":18,"addr":"172.21.15.178:6839/70628097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6838","nonce":70628097},{"type":"v1","addr":"172.21.15.178:6839","nonce":70628097}]},"join_fscid":-1,"export_targets":[0,1],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24638":{"gid":24638,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":18,"addr":"172.21.15.165:6837/6215088","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6836","nonce":6215088},{"type":"v1","addr":"172.21.15.165:6837","nonce":6215088}]},"join_fscid":-1,"export_targets":[0,1,3,4],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24643":{"gid":24643,"name":"g","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.12:6839/3124093009","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6838","nonce":3124093009},{"type":"v1","addr":"172.21.15.12:6839","nonce":3124093009}]},"join_fscid":-1,"export_targets":[0,2,3,4],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24674":{"gid":24674,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.165:6839/501312984","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6838","nonce":501312984},{"type":"v1","addr":"172.21.15.165:6839","nonce":501312984}]},"join_fscid":-1,"export_targets":[1,2,4],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_27751":{"gid":27751,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6837/1436600611","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":1436600611},{"type":"v1","addr":"172.21.15.178:6837","nonce":1436600611}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24674,"qdb_cluster":[24614,24637,24638,24643,24674]},"id":1}]} 2024-09-11T00:32:18.250 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 111 2024-09-11T00:32:18.252 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:18.249+0000 7f7684c00640 1 -- 172.21.15.12:0/3812829407 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f76640b5390 msgr2=0x7f76640b7850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:32:18.252 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:18.249+0000 7f7684c00640 1 --2- 172.21.15.12:0/3812829407 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f76640b5390 0x7f76640b7850 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f7678009a60 tx=0x7f76780095c0 comp rx=0 tx=0).stop 2024-09-11T00:32:18.252 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:18.249+0000 7f7684c00640 1 -- 172.21.15.12:0/3812829407 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f768010f220 msgr2=0x7f76801a0b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:32:18.252 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:18.249+0000 7f7684c00640 1 --2- 172.21.15.12:0/3812829407 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f768010f220 0x7f76801a0b20 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f766c00e940 tx=0x7f766c00ee00 comp rx=0 tx=0).stop 2024-09-11T00:32:18.252 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:18.249+0000 7f7684c00640 1 -- 172.21.15.12:0/3812829407 shutdown_connections 2024-09-11T00:32:18.252 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:18.249+0000 7f7684c00640 1 --2- 172.21.15.12:0/3812829407 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f76640b5390 0x7f76640b7850 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:32:18.252 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:18.249+0000 7f7684c00640 1 --2- 172.21.15.12:0/3812829407 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f768010f220 0x7f76801a0b20 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:32:18.252 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:18.249+0000 7f7684c00640 1 --2- 172.21.15.12:0/3812829407 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7680075f40 0x7f768019b780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:32:18.252 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:18.249+0000 7f7684c00640 1 --2- 172.21.15.12:0/3812829407 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7680075540 0x7f768019b240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:32:18.252 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:18.249+0000 7f7684c00640 1 -- 172.21.15.12:0/3812829407 >> 172.21.15.12:0/3812829407 conn(0x7f76800fe6d0 msgr2=0x7f76800ff2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:32:18.252 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:18.249+0000 7f7684c00640 1 -- 172.21.15.12:0/3812829407 shutdown_connections 2024-09-11T00:32:18.253 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:18.249+0000 7f7684c00640 1 -- 172.21.15.12:0/3812829407 wait complete. 2024-09-11T00:32:18.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:32:17 smithi165 ceph-mon[35356]: pgmap v1156: 129 pgs: 129 active+clean; 691 MiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 264 KiB/s rd, 313 KiB/s wr, 153 op/s 2024-09-11T00:32:18.283 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:32:17 smithi165 ceph-mon[35356]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T00:32:18.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:32:17 smithi178 ceph-mon[35919]: pgmap v1156: 129 pgs: 129 active+clean; 691 MiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 264 KiB/s rd, 313 KiB/s wr, 153 op/s 2024-09-11T00:32:18.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:32:17 smithi178 ceph-mon[35919]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T00:32:18.991 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24674 quiesce db --cancel --all' 2024-09-11T00:32:18.991 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid ff3ad74a-6fcf-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24674 quiesce db --cancel --all 2024-09-11T00:32:19.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:32:18 smithi165 ceph-mon[35356]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T00:32:19.291 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:32:18 smithi165 ceph-mon[35356]: from='client.? 172.21.15.12:0/3812829407' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T00:32:19.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:32:18 smithi012 ceph-mon[30333]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T00:32:19.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:32:18 smithi012 ceph-mon[30333]: from='client.? 172.21.15.12:0/3812829407' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T00:32:19.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:32:18 smithi178 ceph-mon[35919]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T00:32:19.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:32:18 smithi178 ceph-mon[35919]: from='client.? 172.21.15.12:0/3812829407' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T00:32:19.372 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:32:19.998 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:19.977+0000 7f5fd2400640 1 -- 172.21.15.12:0/1607564677 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5fcc102c10 msgr2=0x7f5fcc102ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:32:19.999 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:19.977+0000 7f5fd2400640 1 --2- 172.21.15.12:0/1607564677 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5fcc102c10 0x7f5fcc102ff0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f5fc0009900 tx=0x7f5fc002a410 comp rx=0 tx=0).stop 2024-09-11T00:32:19.999 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:19.978+0000 7f5fd2400640 1 -- 172.21.15.12:0/1607564677 shutdown_connections 2024-09-11T00:32:19.999 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:19.978+0000 7f5fd2400640 1 --2- 172.21.15.12:0/1607564677 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5fcc108fa0 0x7f5fcc10f7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:32:19.999 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:19.978+0000 7f5fd2400640 1 --2- 172.21.15.12:0/1607564677 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5fcc108620 0x7f5fcc108a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:32:19.999 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:19.978+0000 7f5fd2400640 1 --2- 172.21.15.12:0/1607564677 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5fcc102c10 0x7f5fcc102ff0 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:32:19.999 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:19.978+0000 7f5fd2400640 1 -- 172.21.15.12:0/1607564677 >> 172.21.15.12:0/1607564677 conn(0x7f5fcc0fe660 msgr2=0x7f5fcc100a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:32:19.999 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:19.979+0000 7f5fd2400640 1 -- 172.21.15.12:0/1607564677 shutdown_connections 2024-09-11T00:32:19.999 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:19.979+0000 7f5fd2400640 1 -- 172.21.15.12:0/1607564677 wait complete. 2024-09-11T00:32:19.999 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:19.979+0000 7f5fd2400640 1 Processor -- start 2024-09-11T00:32:19.999 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:19.980+0000 7f5fd2400640 1 -- start start 2024-09-11T00:32:19.999 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:19.980+0000 7f5fd2400640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5fcc102c10 0x7f5fcc199060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:32:20.000 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:19.980+0000 7f5fd2400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5fcc108620 0x7f5fcc1995a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:32:20.000 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:19.980+0000 7f5fd2400640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5fcc108fa0 0x7f5fcc19e940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:32:20.000 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:19.980+0000 7f5fd2400640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f5fcc112cb0 con 0x7f5fcc102c10 2024-09-11T00:32:20.000 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:19.980+0000 7f5fd2400640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f5fcc112b30 con 0x7f5fcc108620 2024-09-11T00:32:20.000 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:19.980+0000 7f5fd2400640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f5fcc112e30 con 0x7f5fcc108fa0 2024-09-11T00:32:20.000 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:19.980+0000 7f5fd1000640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5fcc102c10 0x7f5fcc199060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:32:20.000 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:19.980+0000 7f5fd1000640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5fcc102c10 0x7f5fcc199060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:34194/0 (socket says 172.21.15.12:34194) 2024-09-11T00:32:20.000 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:19.980+0000 7f5fd1000640 1 -- 172.21.15.12:0/435356962 learned_addr learned my addr 172.21.15.12:0/435356962 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:32:20.000 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:19.980+0000 7f5fcbe00640 1 --2- 172.21.15.12:0/435356962 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5fcc108620 0x7f5fcc1995a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:32:20.000 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:19.980+0000 7f5fd1a00640 1 --2- 172.21.15.12:0/435356962 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5fcc108fa0 0x7f5fcc19e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:32:20.000 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:19.981+0000 7f5fd1000640 1 -- 172.21.15.12:0/435356962 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5fcc108fa0 msgr2=0x7f5fcc19e940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:32:20.000 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:19.981+0000 7f5fd1000640 1 --2- 172.21.15.12:0/435356962 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5fcc108fa0 0x7f5fcc19e940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:32:20.001 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:19.981+0000 7f5fd1000640 1 -- 172.21.15.12:0/435356962 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5fcc108620 msgr2=0x7f5fcc1995a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:32:20.001 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:19.981+0000 7f5fd1000640 1 --2- 172.21.15.12:0/435356962 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5fcc108620 0x7f5fcc1995a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:32:20.001 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:19.981+0000 7f5fd1000640 1 -- 172.21.15.12:0/435356962 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5fcc19f110 con 0x7f5fcc102c10 2024-09-11T00:32:20.001 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:19.981+0000 7f5fd1000640 1 --2- 172.21.15.12:0/435356962 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5fcc102c10 0x7f5fcc199060 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f5fc0005ab0 tx=0x7f5fc0003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:32:20.001 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:19.981+0000 7f5fc9600640 1 -- 172.21.15.12:0/435356962 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fc003f070 con 0x7f5fcc102c10 2024-09-11T00:32:20.001 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:19.981+0000 7f5fd2400640 1 -- 172.21.15.12:0/435356962 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5fcc19f3a0 con 0x7f5fcc102c10 2024-09-11T00:32:20.001 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:19.981+0000 7f5fc9600640 1 -- 172.21.15.12:0/435356962 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5fc0031b60 con 0x7f5fcc102c10 2024-09-11T00:32:20.001 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:19.981+0000 7f5fd2400640 1 -- 172.21.15.12:0/435356962 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5fcc1a5ac0 con 0x7f5fcc102c10 2024-09-11T00:32:20.001 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:19.982+0000 7f5fc9600640 1 -- 172.21.15.12:0/435356962 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fc0049060 con 0x7f5fcc102c10 2024-09-11T00:32:20.001 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:19.984+0000 7f5fd6467740 1 Processor -- start 2024-09-11T00:32:20.001 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:19.985+0000 7f5fd6467740 1 -- start start 2024-09-11T00:32:20.001 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:19.985+0000 7f5fd6467740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561c8d6af660 0x561c8d6afa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:32:20.001 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:19.985+0000 7f5fd6467740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561c8d6aff80 0x561c8d6bb9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:32:20.001 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:19.985+0000 7f5fd6467740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561c8d6bbf10 0x561c8d6be300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:32:20.002 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:19.985+0000 7f5fc9600640 1 -- 172.21.15.12:0/435356962 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7f5fc003b460 con 0x7f5fcc102c10 2024-09-11T00:32:20.002 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:19.985+0000 7f5fd6467740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x561c8d622ff0 con 0x561c8d6aff80 2024-09-11T00:32:20.002 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:19.985+0000 7f5fd6467740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x561c8d612420 con 0x561c8d6af660 2024-09-11T00:32:20.002 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:19.985+0000 7f5fd6467740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x561c8d616230 con 0x561c8d6bbf10 2024-09-11T00:32:20.002 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:19.985+0000 7f5fd1000640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561c8d6aff80 0x561c8d6bb9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:32:20.002 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:19.985+0000 7f5fcbe00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561c8d6bbf10 0x561c8d6be300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:32:20.002 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:19.985+0000 7f5fcbe00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561c8d6bbf10 0x561c8d6be300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.12:53564/0 (socket says 172.21.15.12:53564) 2024-09-11T00:32:20.002 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:19.985+0000 7f5fcbe00640 1 -- 172.21.15.12:0/2631312264 learned_addr learned my addr 172.21.15.12:0/2631312264 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:32:20.002 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:19.985+0000 7f5fd1a00640 1 --2- 172.21.15.12:0/2631312264 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561c8d6af660 0x561c8d6afa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:32:20.002 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:19.986+0000 7f5fcbe00640 1 -- 172.21.15.12:0/2631312264 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561c8d6af660 msgr2=0x561c8d6afa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:32:20.002 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:19.986+0000 7f5fcbe00640 1 --2- 172.21.15.12:0/2631312264 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561c8d6af660 0x561c8d6afa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:32:20.002 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:19.986+0000 7f5fcbe00640 1 -- 172.21.15.12:0/2631312264 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561c8d6aff80 msgr2=0x561c8d6bb9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:32:20.002 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:19.986+0000 7f5fcbe00640 1 --2- 172.21.15.12:0/2631312264 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561c8d6aff80 0x561c8d6bb9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:32:20.002 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:19.986+0000 7f5fcbe00640 1 -- 172.21.15.12:0/2631312264 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561c8d3708b0 con 0x561c8d6bbf10 2024-09-11T00:32:20.002 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:19.986+0000 7f5fd1a00640 1 --2- 172.21.15.12:0/2631312264 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561c8d6af660 0x561c8d6afa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:32:20.003 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:19.986+0000 7f5fd1000640 1 --2- 172.21.15.12:0/2631312264 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561c8d6aff80 0x561c8d6bb9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:32:20.003 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:19.986+0000 7f5fcbe00640 1 --2- 172.21.15.12:0/2631312264 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561c8d6bbf10 0x561c8d6be300 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f5fb8009ca0 tx=0x7f5fb8011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ab877db56a80f41a server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:32:20.003 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:19.987+0000 7f5fbe000640 1 -- 172.21.15.12:0/2631312264 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fb801e070 con 0x561c8d6bbf10 2024-09-11T00:32:20.003 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:19.987+0000 7f5fbe000640 1 -- 172.21.15.12:0/2631312264 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5fb8019ae0 con 0x561c8d6bbf10 2024-09-11T00:32:20.003 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:19.987+0000 7f5fbe000640 1 -- 172.21.15.12:0/2631312264 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fb800e950 con 0x561c8d6bbf10 2024-09-11T00:32:20.003 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:19.987+0000 7f5fc9600640 1 --2- 172.21.15.12:0/435356962 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f5fb40b5390 0x7f5fb40b7850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:32:20.003 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:19.987+0000 7f5fc9600640 1 -- 172.21.15.12:0/435356962 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9208+0+0 (secure 0 0 0) 0x7f5fc0106d30 con 0x7f5fcc102c10 2024-09-11T00:32:20.003 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:19.987+0000 7f5fd1a00640 1 --2- 172.21.15.12:0/435356962 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f5fb40b5390 0x7f5fb40b7850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:32:20.003 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:19.987+0000 7f5fd6467740 1 -- 172.21.15.12:0/2631312264 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561c8d6bbf10 msgr2=0x561c8d6be300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:32:20.003 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:19.987+0000 7f5fd6467740 1 --2- 172.21.15.12:0/2631312264 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561c8d6bbf10 0x561c8d6be300 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f5fb8009ca0 tx=0x7f5fb8011810 comp rx=0 tx=0).stop 2024-09-11T00:32:20.003 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:19.988+0000 7f5fd6467740 1 -- 172.21.15.12:0/2631312264 shutdown_connections 2024-09-11T00:32:20.003 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:19.988+0000 7f5fd6467740 1 --2- 172.21.15.12:0/2631312264 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561c8d6bbf10 0x561c8d6be300 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:32:20.003 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:19.988+0000 7f5fd6467740 1 --2- 172.21.15.12:0/2631312264 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561c8d6aff80 0x561c8d6bb9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:32:20.004 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:19.988+0000 7f5fd6467740 1 --2- 172.21.15.12:0/2631312264 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561c8d6af660 0x561c8d6afa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:32:20.004 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:19.988+0000 7f5fd6467740 1 -- 172.21.15.12:0/2631312264 >> 172.21.15.12:0/2631312264 conn(0x561c8d6ad490 msgr2=0x561c8d6ae630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:32:20.004 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:19.988+0000 7f5fd6467740 1 -- 172.21.15.12:0/2631312264 shutdown_connections 2024-09-11T00:32:20.004 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:19.988+0000 7f5fd1a00640 1 --2- 172.21.15.12:0/435356962 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f5fb40b5390 0x7f5fb40b7850 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f5fc4009fd0 tx=0x7f5fc4009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:32:20.004 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:19.988+0000 7f5fd6467740 1 -- 172.21.15.12:0/2631312264 wait complete. 2024-09-11T00:32:20.004 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:19.988+0000 7f5fd6467740 1 -- start start 2024-09-11T00:32:20.004 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:19.988+0000 7f5fd6467740 1 Processor -- start 2024-09-11T00:32:20.004 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:19.989+0000 7f5fd6467740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561c8d6af660 0x561c8d6bb780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:32:20.004 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:19.989+0000 7f5fd6467740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561c8d6aff80 0x561c8d6b8bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:32:20.004 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:19.989+0000 7f5fd6467740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561c8d6bbf10 0x561c8d6b9150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:32:20.004 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:19.989+0000 7f5fd6467740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x561c8d473810 con 0x561c8d6af660 2024-09-11T00:32:20.004 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:19.989+0000 7f5fd6467740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x561c8d5ad550 con 0x561c8d6aff80 2024-09-11T00:32:20.004 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:19.989+0000 7f5fd6467740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x561c8d6beb90 con 0x561c8d6bbf10 2024-09-11T00:32:20.004 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:19.989+0000 7f5fd6467740 10 client.0 _ll_register_callbacks cb 0x561c8d5bd040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:32:20.005 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:19.989+0000 7f5fd1000640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561c8d6af660 0x561c8d6bb780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:32:20.005 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:19.989+0000 7f5fd1a00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561c8d6bbf10 0x561c8d6b9150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:32:20.005 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:19.989+0000 7f5fd1000640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561c8d6af660 0x561c8d6bb780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:34204/0 (socket says 172.21.15.12:34204) 2024-09-11T00:32:20.005 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:19.989+0000 7f5fd1000640 1 -- 172.21.15.12:0/3177006880 learned_addr learned my addr 172.21.15.12:0/3177006880 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:32:20.005 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:19.989+0000 7f5fcbe00640 1 --2- 172.21.15.12:0/3177006880 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561c8d6aff80 0x561c8d6b8bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:32:20.005 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:19.990+0000 7f5fd1000640 1 -- 172.21.15.12:0/3177006880 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561c8d6bbf10 msgr2=0x561c8d6b9150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:32:20.005 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:19.990+0000 7f5fd1000640 1 --2- 172.21.15.12:0/3177006880 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561c8d6bbf10 0x561c8d6b9150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:32:20.005 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:19.990+0000 7f5fd1000640 1 -- 172.21.15.12:0/3177006880 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561c8d6aff80 msgr2=0x561c8d6b8bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:32:20.005 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:19.990+0000 7f5fd1000640 1 --2- 172.21.15.12:0/3177006880 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561c8d6aff80 0x561c8d6b8bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:32:20.005 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:19.990+0000 7f5fd1000640 1 -- 172.21.15.12:0/3177006880 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561c8d59f200 con 0x561c8d6af660 2024-09-11T00:32:20.005 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:19.990+0000 7f5fd1a00640 1 --2- 172.21.15.12:0/3177006880 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561c8d6bbf10 0x561c8d6b9150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:32:20.005 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:19.990+0000 7f5fd1000640 1 --2- 172.21.15.12:0/3177006880 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561c8d6af660 0x561c8d6bb780 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f5fc00ca3e0 tx=0x7f5fc002a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b704417014c2381 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:32:20.005 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:19.990+0000 7f5fb2a00640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-09-11T00:32:20.006 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:19.990+0000 7f5fb2a00640 1 -- 172.21.15.12:0/3177006880 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fc00ccf40 con 0x561c8d6af660 2024-09-11T00:32:20.006 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:19.990+0000 7f5fb2a00640 1 -- 172.21.15.12:0/3177006880 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 9208+0+0 (secure 0 0 0) 0x7f5fc00cd3d0 con 0x561c8d6af660 2024-09-11T00:32:20.006 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:19.990+0000 7f5fd6467740 1 -- 172.21.15.12:0/3177006880 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5fb8019ae0 con 0x561c8d6af660 2024-09-11T00:32:20.006 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:19.990+0000 7f5fb2a00640 1 -- 172.21.15.12:0/3177006880 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=111) ==== 24+0+0 (secure 0 0 0) 0x7f5fc00cc8e0 con 0x561c8d6af660 2024-09-11T00:32:20.006 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:19.990+0000 7f5fd6467740 10 client.18174 fetch_fsmap learned FSMap version 111 2024-09-11T00:32:20.006 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:19.990+0000 7f5fd6467740 1 -- 172.21.15.12:0/3177006880 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=111,monmap=4+}) -- 0x561c8d6ca4a0 con 0x561c8d6af660 2024-09-11T00:32:20.006 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:19.991+0000 7f5fb2a00640 1 -- 172.21.15.12:0/3177006880 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5fb8019ae0 con 0x561c8d6af660 2024-09-11T00:32:20.006 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:19.991+0000 7f5fb2a00640 1 -- 172.21.15.12:0/3177006880 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 111) ==== 8438+0+0 (secure 0 0 0) 0x7f5fc00d61c0 con 0x561c8d6af660 2024-09-11T00:32:20.006 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:19.991+0000 7f5fd6467740 10 client.18174 fetch_fsmap finished waiting for FSMap version 111 2024-09-11T00:32:20.006 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:19.991+0000 7f5fd6467740 10 client.18174 resolve_mds: validated gid 24674 aka daemon mds.b 2024-09-11T00:32:20.006 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:19.991+0000 7f5fd6467740 20 client.18174 populate_metadata read hostname 'smithi012' 2024-09-11T00:32:20.006 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:19.991+0000 7f5fd6467740 1 --2- 172.21.15.12:0/3177006880 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x561c8d6cb9a0 0x561c8d6cde00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:32:20.006 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:19.991+0000 7f5fd6467740 4 client.18174 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T00:32:20.007 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:19.991+0000 7f5fd6467740 1 -- 172.21.15.12:0/3177006880 --> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561c8d624f40 con 0x561c8d6cb9a0 2024-09-11T00:32:20.007 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:19.992+0000 7f5fcbe00640 1 --2- 172.21.15.12:0/3177006880 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x561c8d6cb9a0 0x561c8d6cde00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:32:20.007 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:19.993+0000 7f5fcbe00640 1 --2- 172.21.15.12:0/3177006880 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x561c8d6cb9a0 0x561c8d6cde00 crc :-1 s=READY pgs=341 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9b617906cb1dfbf3 server_cookie=566673cde6e142c5 in_seq=0 out_seq=0 2024-09-11T00:32:20.007 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:19.993+0000 7f5fb2a00640 10 client.18174 ms_handle_connect on v2:172.21.15.165:6838/501312984 2024-09-11T00:32:20.007 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:19.995+0000 7f5fb2a00640 1 -- 172.21.15.12:0/3177006880 <== mds.0 v2:172.21.15.165:6838/501312984 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x561c8d624f40 con 0x561c8d6cb9a0 2024-09-11T00:32:20.007 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:19.995+0000 7f5fb2a00640 10 client.18174 handle_command_reply: tid=0 2024-09-11T00:32:20.007 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:19.995+0000 7f5fd6467740 1 client.18174 shutdown 2024-09-11T00:32:20.007 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:19.995+0000 7f5fd6467740 1 -- 172.21.15.12:0/3177006880 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561c8d6af660 msgr2=0x561c8d6bb780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:32:20.007 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:19.995+0000 7f5fd6467740 1 --2- 172.21.15.12:0/3177006880 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561c8d6af660 0x561c8d6bb780 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f5fc00ca3e0 tx=0x7f5fc002a410 comp rx=0 tx=0).stop 2024-09-11T00:32:20.007 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:19.995+0000 7f5fd6467740 1 -- 172.21.15.12:0/3177006880 shutdown_connections 2024-09-11T00:32:20.007 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:19.995+0000 7f5fd6467740 1 --2- 172.21.15.12:0/3177006880 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x561c8d6cb9a0 0x561c8d6cde00 crc :-1 s=READY pgs=341 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:32:20.007 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:19.995+0000 7f5fd6467740 1 --2- 172.21.15.12:0/3177006880 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561c8d6bbf10 0x561c8d6b9150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:32:20.007 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:19.995+0000 7f5fd6467740 1 --2- 172.21.15.12:0/3177006880 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561c8d6aff80 0x561c8d6b8bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:32:20.007 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:19.995+0000 7f5fd6467740 1 --2- 172.21.15.12:0/3177006880 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561c8d6af660 0x561c8d6bb780 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:32:20.008 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:19.995+0000 7f5fd6467740 1 -- 172.21.15.12:0/3177006880 >> 172.21.15.12:0/3177006880 conn(0x561c8d6ad490 msgr2=0x561c8d6b3030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:32:20.008 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:19.995+0000 7f5fb2a00640 0 client.18174 ms_handle_reset on v2:172.21.15.165:6838/501312984 2024-09-11T00:32:20.008 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:19.995+0000 7f5fd6467740 1 -- 172.21.15.12:0/3177006880 shutdown_connections 2024-09-11T00:32:20.008 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:19.996+0000 7f5fd6467740 1 -- 172.21.15.12:0/3177006880 wait complete. 2024-09-11T00:32:20.008 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:19.996+0000 7f5fd6467740 20 client.18174 trim_cache size 0 max 16384 2024-09-11T00:32:20.016 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:20.014+0000 7f5fd6467740 1 Processor -- start 2024-09-11T00:32:20.016 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:20.014+0000 7f5fd6467740 1 -- start start 2024-09-11T00:32:20.018 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:20.014+0000 7f5fd6467740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561c8d6af660 0x561c8d6a4390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:32:20.018 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:20.014+0000 7f5fd6467740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561c8d6aff80 0x561c8d6ba910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:32:20.018 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:20.014+0000 7f5fd6467740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561c8d6bbf10 0x561c8d6bd300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:32:20.018 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:20.014+0000 7f5fd6467740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f5fcc1ad6d0 con 0x561c8d6af660 2024-09-11T00:32:20.018 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:20.014+0000 7f5fd6467740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f5fcc1af3c0 con 0x561c8d6aff80 2024-09-11T00:32:20.018 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:20.014+0000 7f5fd6467740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f5fcc1ade00 con 0x561c8d6bbf10 2024-09-11T00:32:20.018 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:20.014+0000 7f5fd1000640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561c8d6af660 0x561c8d6a4390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:32:20.018 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:20.014+0000 7f5fd1000640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561c8d6af660 0x561c8d6a4390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:34210/0 (socket says 172.21.15.12:34210) 2024-09-11T00:32:20.018 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:20.014+0000 7f5fd1000640 1 -- 172.21.15.12:0/238313560 learned_addr learned my addr 172.21.15.12:0/238313560 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:32:20.018 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:20.015+0000 7f5fcbe00640 1 --2- 172.21.15.12:0/238313560 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561c8d6aff80 0x561c8d6ba910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:32:20.019 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:20.015+0000 7f5fd1a00640 1 --2- 172.21.15.12:0/238313560 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561c8d6bbf10 0x561c8d6bd300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:32:20.019 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:20.015+0000 7f5fd1000640 1 -- 172.21.15.12:0/238313560 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561c8d6bbf10 msgr2=0x561c8d6bd300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:32:20.019 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:20.015+0000 7f5fd1000640 1 --2- 172.21.15.12:0/238313560 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561c8d6bbf10 0x561c8d6bd300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:32:20.019 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:20.015+0000 7f5fd1000640 1 -- 172.21.15.12:0/238313560 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561c8d6aff80 msgr2=0x561c8d6ba910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:32:20.019 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:20.015+0000 7f5fd1000640 1 --2- 172.21.15.12:0/238313560 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561c8d6aff80 0x561c8d6ba910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:32:20.019 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:20.015+0000 7f5fd1000640 1 -- 172.21.15.12:0/238313560 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561c8d6bbaf0 con 0x561c8d6af660 2024-09-11T00:32:20.019 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:20.015+0000 7f5fcbe00640 1 --2- 172.21.15.12:0/238313560 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561c8d6aff80 0x561c8d6ba910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:32:20.019 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:20.015+0000 7f5fd1000640 1 --2- 172.21.15.12:0/238313560 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561c8d6af660 0x561c8d6a4390 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f5fc0036040 tx=0x7f5fc00045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4bc48524f6f1c003 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:32:20.021 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:20.016+0000 7f5fbd600640 1 -- 172.21.15.12:0/238313560 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fc00ccac0 con 0x561c8d6af660 2024-09-11T00:32:20.021 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:20.016+0000 7f5fbd600640 1 -- 172.21.15.12:0/238313560 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5fc0109070 con 0x561c8d6af660 2024-09-11T00:32:20.021 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:20.016+0000 7f5fbd600640 1 -- 172.21.15.12:0/238313560 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fc00cd170 con 0x561c8d6af660 2024-09-11T00:32:20.021 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:20.016+0000 7f5fd6467740 1 -- 172.21.15.12:0/238313560 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561c8d6af660 msgr2=0x561c8d6a4390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:32:20.021 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:20.016+0000 7f5fd6467740 1 --2- 172.21.15.12:0/238313560 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561c8d6af660 0x561c8d6a4390 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f5fc0036040 tx=0x7f5fc00045e0 comp rx=0 tx=0).stop 2024-09-11T00:32:20.021 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:20.016+0000 7f5fd6467740 1 -- 172.21.15.12:0/238313560 shutdown_connections 2024-09-11T00:32:20.021 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:20.016+0000 7f5fd6467740 1 --2- 172.21.15.12:0/238313560 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561c8d6bbf10 0x561c8d6bd300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:32:20.021 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:20.016+0000 7f5fd6467740 1 --2- 172.21.15.12:0/238313560 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561c8d6aff80 0x561c8d6ba910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:32:20.022 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:20.016+0000 7f5fd6467740 1 --2- 172.21.15.12:0/238313560 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561c8d6af660 0x561c8d6a4390 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:32:20.022 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:20.016+0000 7f5fd6467740 1 -- 172.21.15.12:0/238313560 >> 172.21.15.12:0/238313560 conn(0x561c8d6ad490 msgr2=0x561c8d6ba200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:32:20.022 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:20.017+0000 7f5fd6467740 1 -- 172.21.15.12:0/238313560 shutdown_connections 2024-09-11T00:32:20.022 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:20.017+0000 7f5fd6467740 1 -- 172.21.15.12:0/238313560 wait complete. 2024-09-11T00:32:20.022 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:20.017+0000 7f5fd6467740 1 -- start start 2024-09-11T00:32:20.022 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:20.017+0000 7f5fd6467740 1 Processor -- start 2024-09-11T00:32:20.022 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:20.018+0000 7f5fd6467740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561c8d6af660 0x561c8d6bf270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:32:20.022 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:20.018+0000 7f5fd6467740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561c8d6aff80 0x561c8d6bf7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:32:20.022 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:20.018+0000 7f5fd6467740 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561c8d6bbf10 0x561c8d7388d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:32:20.022 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:20.018+0000 7f5fd6467740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f5fcc1aecf0 con 0x561c8d6aff80 2024-09-11T00:32:20.022 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:20.018+0000 7f5fd6467740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f5fcc1ae590 con 0x561c8d6af660 2024-09-11T00:32:20.022 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:20.018+0000 7f5fd6467740 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f5fcc1ab6b0 con 0x561c8d6bbf10 2024-09-11T00:32:20.023 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:20.018+0000 7f5fd6467740 10 client.0 _ll_register_callbacks cb 0x7f5fc0009260 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:32:20.023 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:20.018+0000 7f5fcbe00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561c8d6aff80 0x561c8d6bf7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:32:20.023 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:20.018+0000 7f5fd1a00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561c8d6bbf10 0x561c8d7388d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:32:20.023 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:20.018+0000 7f5fd1a00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561c8d6bbf10 0x561c8d7388d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.12:53584/0 (socket says 172.21.15.12:53584) 2024-09-11T00:32:20.023 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:20.018+0000 7f5fd1000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561c8d6af660 0x561c8d6bf270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:32:20.023 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:20.018+0000 7f5fd1a00640 1 -- 172.21.15.12:0/3814005565 learned_addr learned my addr 172.21.15.12:0/3814005565 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:32:20.023 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:20.018+0000 7f5fd1a00640 1 -- 172.21.15.12:0/3814005565 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561c8d6af660 msgr2=0x561c8d6bf270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:32:20.023 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:20.018+0000 7f5fd1a00640 1 --2- 172.21.15.12:0/3814005565 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561c8d6af660 0x561c8d6bf270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:32:20.023 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:20.018+0000 7f5fd1a00640 1 -- 172.21.15.12:0/3814005565 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561c8d6aff80 msgr2=0x561c8d6bf7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:32:20.023 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:20.018+0000 7f5fd1a00640 1 --2- 172.21.15.12:0/3814005565 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561c8d6aff80 0x561c8d6bf7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:32:20.023 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:20.018+0000 7f5fd1a00640 1 -- 172.21.15.12:0/3814005565 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561c8d738e10 con 0x561c8d6bbf10 2024-09-11T00:32:20.023 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:20.018+0000 7f5fd1000640 1 --2- 172.21.15.12:0/3814005565 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561c8d6af660 0x561c8d6bf270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:32:20.023 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:20.018+0000 7f5fcbe00640 1 --2- 172.21.15.12:0/3814005565 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561c8d6aff80 0x561c8d6bf7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:32:20.024 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:20.019+0000 7f5fd1a00640 1 --2- 172.21.15.12:0/3814005565 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561c8d6bbf10 0x561c8d7388d0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f5fc400fef0 tx=0x7f5fc4004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=46a465728b7039a8 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:32:20.024 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:20.019+0000 7f5fb2a00640 10 client.0 ms_handle_connect on v2:172.21.15.178:3300/0 2024-09-11T00:32:20.025 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:20.019+0000 7f5fb2a00640 1 -- 172.21.15.12:0/3814005565 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fc402e060 con 0x561c8d6bbf10 2024-09-11T00:32:20.025 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:20.019+0000 7f5fd6467740 1 -- 172.21.15.12:0/3814005565 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5fc0109070 con 0x561c8d6bbf10 2024-09-11T00:32:20.026 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:20.019+0000 7f5fb2a00640 1 -- 172.21.15.12:0/3814005565 <== mon.1 v2:172.21.15.178:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 9208+0+0 (secure 0 0 0) 0x7f5fc40109b0 con 0x561c8d6bbf10 2024-09-11T00:32:20.026 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:20.019+0000 7f5fb2a00640 1 -- 172.21.15.12:0/3814005565 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=111) ==== 24+0+0 (secure 0 0 0) 0x7f5fc4035050 con 0x561c8d6bbf10 2024-09-11T00:32:20.026 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:20.019+0000 7f5fd6467740 10 client.28156 fetch_fsmap learned FSMap version 111 2024-09-11T00:32:20.026 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:20.019+0000 7f5fd6467740 1 -- 172.21.15.12:0/3814005565 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,fsmap=111,monmap=4+}) -- 0x561c8d739fe0 con 0x561c8d6bbf10 2024-09-11T00:32:20.026 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:20.020+0000 7f5fb2a00640 1 -- 172.21.15.12:0/3814005565 <== mon.1 v2:172.21.15.178:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5fc0109070 con 0x561c8d6bbf10 2024-09-11T00:32:20.026 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:20.020+0000 7f5fb2a00640 1 -- 172.21.15.12:0/3814005565 <== mon.1 v2:172.21.15.178:3300/0 5 ==== fsmap(e 111) ==== 8438+0+0 (secure 0 0 0) 0x7f5fc4004290 con 0x561c8d6bbf10 2024-09-11T00:32:20.026 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:20.020+0000 7f5fd6467740 10 client.28156 fetch_fsmap finished waiting for FSMap version 111 2024-09-11T00:32:20.026 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:20.020+0000 7f5fd6467740 10 client.28156 resolve_mds: validated gid 24674 aka daemon mds.b 2024-09-11T00:32:20.026 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:20.020+0000 7f5fd6467740 20 client.28156 populate_metadata read hostname 'smithi012' 2024-09-11T00:32:20.026 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:20.020+0000 7f5fd6467740 1 --2- 172.21.15.12:0/3814005565 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x561c8d73a820 0x561c8d73cc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:32:20.026 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:20.020+0000 7f5fd6467740 4 client.28156 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "quiesce db", "cancel": true, "all": true}] 2024-09-11T00:32:20.026 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:20.020+0000 7f5fd6467740 1 -- 172.21.15.12:0/3814005565 --> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] -- command(tid 0: {"prefix": "quiesce db", "cancel": true, "all": true}) -- 0x561c8d6209f0 con 0x561c8d73a820 2024-09-11T00:32:20.026 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:20.020+0000 7f5fd1000640 1 --2- 172.21.15.12:0/3814005565 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x561c8d73a820 0x561c8d73cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:32:20.027 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:20.021+0000 7f5fd1000640 1 --2- 172.21.15.12:0/3814005565 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x561c8d73a820 0x561c8d73cc40 crc :-1 s=READY pgs=342 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=90fe72d79e53d910 server_cookie=dda0bd4b5157ce26 in_seq=0 out_seq=0 2024-09-11T00:32:20.027 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:20.021+0000 7f5fb2a00640 10 client.28156 ms_handle_connect on v2:172.21.15.165:6838/501312984 2024-09-11T00:32:20.027 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:20.021+0000 7f5fb2a00640 1 -- 172.21.15.12:0/3814005565 <== mds.0 v2:172.21.15.165:6838/501312984 1 ==== command_reply(tid 0: 0 ) ==== 8+0+81 (crc 0 0 0) 0x561c8d6209f0 con 0x561c8d73a820 2024-09-11T00:32:20.027 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:20.021+0000 7f5fb2a00640 10 client.28156 handle_command_reply: tid=0 2024-09-11T00:32:20.027 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:20.021+0000 7f5fd6467740 1 client.28156 shutdown 2024-09-11T00:32:20.027 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:20.021+0000 7f5fd6467740 1 -- 172.21.15.12:0/3814005565 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561c8d6bbf10 msgr2=0x561c8d7388d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:32:20.027 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:20.021+0000 7f5fd6467740 1 --2- 172.21.15.12:0/3814005565 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561c8d6bbf10 0x561c8d7388d0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f5fc400fef0 tx=0x7f5fc4004a40 comp rx=0 tx=0).stop 2024-09-11T00:32:20.027 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:20.022+0000 7f5fd6467740 1 -- 172.21.15.12:0/3814005565 shutdown_connections 2024-09-11T00:32:20.027 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:20.022+0000 7f5fd6467740 1 --2- 172.21.15.12:0/3814005565 >> [v2:172.21.15.165:6838/501312984,v1:172.21.15.165:6839/501312984] conn(0x561c8d73a820 0x561c8d73cc40 crc :-1 s=READY pgs=342 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:32:20.027 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:20.022+0000 7f5fd6467740 1 --2- 172.21.15.12:0/3814005565 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561c8d6bbf10 0x561c8d7388d0 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:32:20.027 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:20.022+0000 7f5fd6467740 1 --2- 172.21.15.12:0/3814005565 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561c8d6aff80 0x561c8d6bf7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:32:20.027 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:20.022+0000 7f5fd6467740 1 --2- 172.21.15.12:0/3814005565 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561c8d6af660 0x561c8d6bf270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:32:20.027 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:20.022+0000 7f5fb2a00640 0 client.28156 ms_handle_reset on v2:172.21.15.165:6838/501312984 2024-09-11T00:32:20.028 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:20.022+0000 7f5fd6467740 1 -- 172.21.15.12:0/3814005565 >> 172.21.15.12:0/3814005565 conn(0x561c8d6ad490 msgr2=0x561c8d6ba200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:32:20.028 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:20.022+0000 7f5fd6467740 1 -- 172.21.15.12:0/3814005565 shutdown_connections 2024-09-11T00:32:20.028 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:20.022+0000 7f5fd6467740 1 -- 172.21.15.12:0/3814005565 wait complete. 2024-09-11T00:32:20.028 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:20.022+0000 7f5fd6467740 20 client.28156 trim_cache size 0 max 16384 2024-09-11T00:32:20.028 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-09-11T00:32:20.028 INFO:teuthology.orchestra.run.smithi012.stdout: "epoch": 111, 2024-09-11T00:32:20.028 INFO:teuthology.orchestra.run.smithi012.stdout: "leader": 24674, 2024-09-11T00:32:20.028 INFO:teuthology.orchestra.run.smithi012.stdout: "set_version": 76, 2024-09-11T00:32:20.028 INFO:teuthology.orchestra.run.smithi012.stdout: "sets": {} 2024-09-11T00:32:20.029 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-09-11T00:32:20.029 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:20.022+0000 7f5fd2400640 1 -- 172.21.15.12:0/435356962 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f5fb40b5390 msgr2=0x7f5fb40b7850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:32:20.029 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:20.022+0000 7f5fd2400640 1 --2- 172.21.15.12:0/435356962 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f5fb40b5390 0x7f5fb40b7850 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f5fc4009fd0 tx=0x7f5fc4009500 comp rx=0 tx=0).stop 2024-09-11T00:32:20.029 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:20.022+0000 7f5fd2400640 1 -- 172.21.15.12:0/435356962 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5fcc102c10 msgr2=0x7f5fcc199060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:32:20.029 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:20.022+0000 7f5fd2400640 1 --2- 172.21.15.12:0/435356962 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5fcc102c10 0x7f5fcc199060 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f5fc0005ab0 tx=0x7f5fc0003c10 comp rx=0 tx=0).stop 2024-09-11T00:32:20.029 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:20.023+0000 7f5fd2400640 1 -- 172.21.15.12:0/435356962 shutdown_connections 2024-09-11T00:32:20.030 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:20.023+0000 7f5fd2400640 1 --2- 172.21.15.12:0/435356962 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f5fb40b5390 0x7f5fb40b7850 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:32:20.030 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:20.023+0000 7f5fd2400640 1 --2- 172.21.15.12:0/435356962 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5fcc108fa0 0x7f5fcc19e940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:32:20.030 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:20.023+0000 7f5fd2400640 1 --2- 172.21.15.12:0/435356962 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5fcc108620 0x7f5fcc1995a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:32:20.030 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:20.023+0000 7f5fd2400640 1 --2- 172.21.15.12:0/435356962 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5fcc102c10 0x7f5fcc199060 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:32:20.030 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:20.023+0000 7f5fd2400640 1 -- 172.21.15.12:0/435356962 >> 172.21.15.12:0/435356962 conn(0x7f5fcc0fe660 msgr2=0x7f5fcc107d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:32:20.030 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:20.023+0000 7f5fd2400640 1 -- 172.21.15.12:0/435356962 shutdown_connections 2024-09-11T00:32:20.030 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:20.023+0000 7f5fd2400640 1 -- 172.21.15.12:0/435356962 wait complete. 2024-09-11T00:32:20.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:32:19 smithi165 ceph-mon[35356]: pgmap v1157: 129 pgs: 129 active+clean; 677 MiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 309 KiB/s rd, 331 KiB/s wr, 161 op/s 2024-09-11T00:32:20.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:32:19 smithi012 ceph-mon[30333]: pgmap v1157: 129 pgs: 129 active+clean; 677 MiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 309 KiB/s rd, 331 KiB/s wr, 161 op/s 2024-09-11T00:32:20.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:32:19 smithi178 ceph-mon[35919]: pgmap v1157: 129 pgs: 129 active+clean; 677 MiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 309 KiB/s rd, 331 KiB/s wr, 161 op/s 2024-09-11T00:32:20.735 INFO:tasks.quiescer:done joining Quiescers 2024-09-11T00:32:20.735 DEBUG:teuthology.run_tasks:Unwinding manager background_exec 2024-09-11T00:32:20.745 INFO:teuthology.task.background_exec:Stopping background command on ubuntu@smithi012.front.sepia.ceph.com 2024-09-11T00:32:20.756 DEBUG:teuthology.run_tasks:Unwinding manager check-counter 2024-09-11T00:32:20.766 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-11T00:32:20.959 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:20.956+0000 7f4a18800640 1 -- 172.21.15.12:0/1765990053 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4a10105cf0 msgr2=0x7f4a10106170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:32:20.959 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:20.956+0000 7f4a18800640 1 --2- 172.21.15.12:0/1765990053 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4a10105cf0 0x7f4a10106170 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f4a00009930 tx=0x7f4a0002a420 comp rx=0 tx=0).stop 2024-09-11T00:32:20.959 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:20.956+0000 7f4a18800640 1 -- 172.21.15.12:0/1765990053 shutdown_connections 2024-09-11T00:32:20.959 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:20.956+0000 7f4a18800640 1 --2- 172.21.15.12:0/1765990053 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4a101066b0 0x7f4a1010d030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:32:20.959 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:20.956+0000 7f4a18800640 1 --2- 172.21.15.12:0/1765990053 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4a10105cf0 0x7f4a10106170 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:32:20.960 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:20.956+0000 7f4a18800640 1 --2- 172.21.15.12:0/1765990053 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4a10104ba0 0x7f4a10104fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:32:20.960 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:20.956+0000 7f4a18800640 1 -- 172.21.15.12:0/1765990053 >> 172.21.15.12:0/1765990053 conn(0x7f4a10100440 msgr2=0x7f4a10102880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:32:20.960 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:20.956+0000 7f4a18800640 1 -- 172.21.15.12:0/1765990053 shutdown_connections 2024-09-11T00:32:20.960 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:20.956+0000 7f4a18800640 1 -- 172.21.15.12:0/1765990053 wait complete. 2024-09-11T00:32:20.962 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:20.957+0000 7f4a18800640 1 Processor -- start 2024-09-11T00:32:20.962 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:20.957+0000 7f4a18800640 1 -- start start 2024-09-11T00:32:20.962 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:20.957+0000 7f4a18800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4a10104ba0 0x7f4a1019af40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:32:20.962 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:20.958+0000 7f4a18800640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4a10105cf0 0x7f4a1019b590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:32:20.962 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:20.958+0000 7f4a18800640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4a101066b0 0x7f4a1019c1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:32:20.962 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:20.958+0000 7f4a18800640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f4a10110840 con 0x7f4a101066b0 2024-09-11T00:32:20.963 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:20.958+0000 7f4a18800640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f4a101106c0 con 0x7f4a10104ba0 2024-09-11T00:32:20.963 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:20.958+0000 7f4a18800640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f4a101109c0 con 0x7f4a10105cf0 2024-09-11T00:32:20.963 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:20.958+0000 7f4a16800640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4a101066b0 0x7f4a1019c1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:32:20.963 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:20.958+0000 7f4a16800640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4a101066b0 0x7f4a1019c1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:34250/0 (socket says 172.21.15.12:34250) 2024-09-11T00:32:20.963 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:20.958+0000 7f4a16800640 1 -- 172.21.15.12:0/1457347394 learned_addr learned my addr 172.21.15.12:0/1457347394 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:32:20.963 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:20.958+0000 7f4a15e00640 1 --2- 172.21.15.12:0/1457347394 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4a10104ba0 0x7f4a1019af40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:32:20.963 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:20.958+0000 7f4a15400640 1 --2- 172.21.15.12:0/1457347394 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4a10105cf0 0x7f4a1019b590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:32:20.963 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:20.958+0000 7f4a16800640 1 -- 172.21.15.12:0/1457347394 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4a10105cf0 msgr2=0x7f4a1019b590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:32:20.963 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:20.958+0000 7f4a16800640 1 --2- 172.21.15.12:0/1457347394 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4a10105cf0 0x7f4a1019b590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:32:20.963 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:20.958+0000 7f4a16800640 1 -- 172.21.15.12:0/1457347394 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4a10104ba0 msgr2=0x7f4a1019af40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:32:20.964 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:20.958+0000 7f4a16800640 1 --2- 172.21.15.12:0/1457347394 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4a10104ba0 0x7f4a1019af40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:32:20.964 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:20.958+0000 7f4a16800640 1 -- 172.21.15.12:0/1457347394 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a1019ca80 con 0x7f4a101066b0 2024-09-11T00:32:20.964 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:20.959+0000 7f4a16800640 1 --2- 172.21.15.12:0/1457347394 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4a101066b0 0x7f4a1019c1c0 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f4a0c00eb10 tx=0x7f4a0c00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:32:20.964 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:20.959+0000 7f4a0aa00640 1 -- 172.21.15.12:0/1457347394 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a0c004b20 con 0x7f4a101066b0 2024-09-11T00:32:20.964 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:20.959+0000 7f4a0aa00640 1 -- 172.21.15.12:0/1457347394 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4a0c004cc0 con 0x7f4a101066b0 2024-09-11T00:32:20.964 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:20.959+0000 7f4a0aa00640 1 -- 172.21.15.12:0/1457347394 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a0c010670 con 0x7f4a101066b0 2024-09-11T00:32:20.964 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:20.959+0000 7f4a18800640 1 -- 172.21.15.12:0/1457347394 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4a1019cd70 con 0x7f4a101066b0 2024-09-11T00:32:20.964 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:20.959+0000 7f4a18800640 1 -- 172.21.15.12:0/1457347394 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4a101a3440 con 0x7f4a101066b0 2024-09-11T00:32:20.965 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:20.960+0000 7f4a18800640 1 -- 172.21.15.12:0/1457347394 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4a1010be40 con 0x7f4a101066b0 2024-09-11T00:32:20.965 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:20.962+0000 7f4a0aa00640 1 -- 172.21.15.12:0/1457347394 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7f4a0c005000 con 0x7f4a101066b0 2024-09-11T00:32:20.966 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:20.964+0000 7f4a0aa00640 1 --2- 172.21.15.12:0/1457347394 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f49f80b5360 0x7f49f80b7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:32:20.966 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:20.964+0000 7f4a0aa00640 1 -- 172.21.15.12:0/1457347394 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9208+0+0 (secure 0 0 0) 0x7f4a0c0e66c0 con 0x7f4a101066b0 2024-09-11T00:32:20.966 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:20.964+0000 7f4a15e00640 1 --2- 172.21.15.12:0/1457347394 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f49f80b5360 0x7f49f80b7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:32:20.967 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:20.965+0000 7f4a15e00640 1 --2- 172.21.15.12:0/1457347394 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f49f80b5360 0x7f49f80b7820 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f4a04005d00 tx=0x7f4a0400b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:32:20.967 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:20.965+0000 7f4a0aa00640 1 -- 172.21.15.12:0/1457347394 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f4a0c056220 con 0x7f4a101066b0 2024-09-11T00:32:21.164 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:21.162+0000 7f4a18800640 1 -- 172.21.15.12:0/1457347394 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f4a1002e830 con 0x7f4a101066b0 2024-09-11T00:32:21.166 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:21.163+0000 7f4a0aa00640 1 -- 172.21.15.12:0/1457347394 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) ==== 74+0+24948 (secure 0 0 0) 0x7f4a0c0ad150 con 0x7f4a101066b0 2024-09-11T00:32:21.166 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-11T00:32:21.166 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":87,"fsid":"ff3ad74a-6fcf-11ef-bcea-c7b262605968","created":"2024-09-10T23:55:16.317643+0000","modified":"2024-09-11T00:28:05.342224+0000","last_up_change":"2024-09-11T00:03:08.959256+0000","last_in_change":"2024-09-11T00:02:46.564631+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T23:59:10.488728+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-11T00:04:00.065618+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"74","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-11T00:04:02.689888+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"74","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-11T00:04:09.862120+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"6382c200-a912-42a1-958b-79191b21249b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6802","nonce":1237101991},{"type":"v1","addr":"172.21.15.12:6803","nonce":1237101991}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6804","nonce":1237101991},{"type":"v1","addr":"172.21.15.12:6805","nonce":1237101991}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6808","nonce":1237101991},{"type":"v1","addr":"172.21.15.12:6809","nonce":1237101991}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6806","nonce":1237101991},{"type":"v1","addr":"172.21.15.12:6807","nonce":1237101991}]},"public_addr":"172.21.15.12:6803/1237101991","cluster_addr":"172.21.15.12:6805/1237101991","heartbeat_back_addr":"172.21.15.12:6809/1237101991","heartbeat_front_addr":"172.21.15.12:6807/1237101991","state":["exists","up"]},{"osd":1,"uuid":"fe0e1d0a-bdc5-4859-aa84-9038059a45f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6800","nonce":545848456},{"type":"v1","addr":"172.21.15.165:6801","nonce":545848456}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6802","nonce":545848456},{"type":"v1","addr":"172.21.15.165:6803","nonce":545848456}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6806","nonce":545848456},{"type":"v1","addr":"172.21.15.165:6807","nonce":545848456}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6804","nonce":545848456},{"type":"v1","addr":"172.21.15.165:6805","nonce":545848456}]},"public_addr":"172.21.15.165:6801/545848456","cluster_addr":"172.21.15.165:6803/545848456","heartbeat_back_addr":"172.21.15.165:6807/545848456","heartbeat_front_addr":"172.21.15.165:6805/545848456","state":["exists","up"]},{"osd":2,"uuid":"94f640dc-5f78-4e2c-87d9-03e96a6e7ba0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6800","nonce":4131162695},{"type":"v1","addr":"172.21.15.178:6801","nonce":4131162695}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":4131162695},{"type":"v1","addr":"172.21.15.178:6803","nonce":4131162695}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":4131162695},{"type":"v1","addr":"172.21.15.178:6807","nonce":4131162695}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":4131162695},{"type":"v1","addr":"172.21.15.178:6805","nonce":4131162695}]},"public_addr":"172.21.15.178:6801/4131162695","cluster_addr":"172.21.15.178:6803/4131162695","heartbeat_back_addr":"172.21.15.178:6807/4131162695","heartbeat_front_addr":"172.21.15.178:6805/4131162695","state":["exists","up"]},{"osd":3,"uuid":"af2f64d6-c9be-44a9-b041-8d599a7c080d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6810","nonce":452464245},{"type":"v1","addr":"172.21.15.12:6811","nonce":452464245}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6812","nonce":452464245},{"type":"v1","addr":"172.21.15.12:6813","nonce":452464245}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6816","nonce":452464245},{"type":"v1","addr":"172.21.15.12:6817","nonce":452464245}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6814","nonce":452464245},{"type":"v1","addr":"172.21.15.12:6815","nonce":452464245}]},"public_addr":"172.21.15.12:6811/452464245","cluster_addr":"172.21.15.12:6813/452464245","heartbeat_back_addr":"172.21.15.12:6817/452464245","heartbeat_front_addr":"172.21.15.12:6815/452464245","state":["exists","up"]},{"osd":4,"uuid":"ad76ac70-20c0-44ae-9866-5f9cfce2334c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6808","nonce":1899879280},{"type":"v1","addr":"172.21.15.165:6809","nonce":1899879280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6810","nonce":1899879280},{"type":"v1","addr":"172.21.15.165:6811","nonce":1899879280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6814","nonce":1899879280},{"type":"v1","addr":"172.21.15.165:6815","nonce":1899879280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6812","nonce":1899879280},{"type":"v1","addr":"172.21.15.165:6813","nonce":1899879280}]},"public_addr":"172.21.15.165:6809/1899879280","cluster_addr":"172.21.15.165:6811/1899879280","heartbeat_back_addr":"172.21.15.165:6815/1899879280","heartbeat_front_addr":"172.21.15.165:6813/1899879280","state":["exists","up"]},{"osd":5,"uuid":"aafda2cc-62fe-4fcb-9a96-8732e0b501d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":3042228555},{"type":"v1","addr":"172.21.15.178:6809","nonce":3042228555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":3042228555},{"type":"v1","addr":"172.21.15.178:6811","nonce":3042228555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6814","nonce":3042228555},{"type":"v1","addr":"172.21.15.178:6815","nonce":3042228555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":3042228555},{"type":"v1","addr":"172.21.15.178:6813","nonce":3042228555}]},"public_addr":"172.21.15.178:6809/3042228555","cluster_addr":"172.21.15.178:6811/3042228555","heartbeat_back_addr":"172.21.15.178:6815/3042228555","heartbeat_front_addr":"172.21.15.178:6813/3042228555","state":["exists","up"]},{"osd":6,"uuid":"375eca52-efe3-42a5-bbb1-d7d392d71f15","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6818","nonce":3868322467},{"type":"v1","addr":"172.21.15.12:6819","nonce":3868322467}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6820","nonce":3868322467},{"type":"v1","addr":"172.21.15.12:6821","nonce":3868322467}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6824","nonce":3868322467},{"type":"v1","addr":"172.21.15.12:6825","nonce":3868322467}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6822","nonce":3868322467},{"type":"v1","addr":"172.21.15.12:6823","nonce":3868322467}]},"public_addr":"172.21.15.12:6819/3868322467","cluster_addr":"172.21.15.12:6821/3868322467","heartbeat_back_addr":"172.21.15.12:6825/3868322467","heartbeat_front_addr":"172.21.15.12:6823/3868322467","state":["exists","up"]},{"osd":7,"uuid":"0fd619f5-44c3-4cf7-ae7a-388dd798de36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6816","nonce":1143773735},{"type":"v1","addr":"172.21.15.165:6817","nonce":1143773735}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6818","nonce":1143773735},{"type":"v1","addr":"172.21.15.165:6819","nonce":1143773735}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6822","nonce":1143773735},{"type":"v1","addr":"172.21.15.165:6823","nonce":1143773735}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6820","nonce":1143773735},{"type":"v1","addr":"172.21.15.165:6821","nonce":1143773735}]},"public_addr":"172.21.15.165:6817/1143773735","cluster_addr":"172.21.15.165:6819/1143773735","heartbeat_back_addr":"172.21.15.165:6823/1143773735","heartbeat_front_addr":"172.21.15.165:6821/1143773735","state":["exists","up"]},{"osd":8,"uuid":"b03d14ce-8427-462a-9bcc-27683c549f53","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":2891445058},{"type":"v1","addr":"172.21.15.178:6817","nonce":2891445058}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6818","nonce":2891445058},{"type":"v1","addr":"172.21.15.178:6819","nonce":2891445058}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6822","nonce":2891445058},{"type":"v1","addr":"172.21.15.178:6823","nonce":2891445058}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":2891445058},{"type":"v1","addr":"172.21.15.178:6821","nonce":2891445058}]},"public_addr":"172.21.15.178:6817/2891445058","cluster_addr":"172.21.15.178:6819/2891445058","heartbeat_back_addr":"172.21.15.178:6823/2891445058","heartbeat_front_addr":"172.21.15.178:6821/2891445058","state":["exists","up"]},{"osd":9,"uuid":"81a6cf69-2a24-4a6c-9adf-6b7df38cfacb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6826","nonce":3359634903},{"type":"v1","addr":"172.21.15.12:6827","nonce":3359634903}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6828","nonce":3359634903},{"type":"v1","addr":"172.21.15.12:6829","nonce":3359634903}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6832","nonce":3359634903},{"type":"v1","addr":"172.21.15.12:6833","nonce":3359634903}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6830","nonce":3359634903},{"type":"v1","addr":"172.21.15.12:6831","nonce":3359634903}]},"public_addr":"172.21.15.12:6827/3359634903","cluster_addr":"172.21.15.12:6829/3359634903","heartbeat_back_addr":"172.21.15.12:6833/3359634903","heartbeat_front_addr":"172.21.15.12:6831/3359634903","state":["exists","up"]},{"osd":10,"uuid":"c1ad1f0a-33f4-4c32-91f6-4d030a29f1dd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6824","nonce":3642325778},{"type":"v1","addr":"172.21.15.165:6825","nonce":3642325778}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6826","nonce":3642325778},{"type":"v1","addr":"172.21.15.165:6827","nonce":3642325778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6830","nonce":3642325778},{"type":"v1","addr":"172.21.15.165:6831","nonce":3642325778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6828","nonce":3642325778},{"type":"v1","addr":"172.21.15.165:6829","nonce":3642325778}]},"public_addr":"172.21.15.165:6825/3642325778","cluster_addr":"172.21.15.165:6827/3642325778","heartbeat_back_addr":"172.21.15.165:6831/3642325778","heartbeat_front_addr":"172.21.15.165:6829/3642325778","state":["exists","up"]},{"osd":11,"uuid":"8413c9b0-680b-4d02-aeab-b3b3ff16c3d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":2937059756},{"type":"v1","addr":"172.21.15.178:6825","nonce":2937059756}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6826","nonce":2937059756},{"type":"v1","addr":"172.21.15.178:6827","nonce":2937059756}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":2937059756},{"type":"v1","addr":"172.21.15.178:6831","nonce":2937059756}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":2937059756},{"type":"v1","addr":"172.21.15.178:6829","nonce":2937059756}]},"public_addr":"172.21.15.178:6825/2937059756","cluster_addr":"172.21.15.178:6827/2937059756","heartbeat_back_addr":"172.21.15.178:6831/2937059756","heartbeat_front_addr":"172.21.15.178:6829/2937059756","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:58:16.036348+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:58:41.483150+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:59:07.107283+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:59:35.706574+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T00:00:02.555720+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T00:00:28.092847+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T00:00:57.396495+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T00:01:24.216775+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T00:01:48.807850+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T00:02:16.838565+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T00:02:43.131736+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T00:03:07.353155+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":8,"to":4}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"3.e","mappings":[{"from":5,"to":9}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9}]},{"pgid":"3.23","mappings":[{"from":4,"to":7}]},{"pgid":"3.31","mappings":[{"from":4,"to":8}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"3.39","mappings":[{"from":5,"to":6}]},{"pgid":"4.5","mappings":[{"from":9,"to":10}]},{"pgid":"4.6","mappings":[{"from":5,"to":8}]},{"pgid":"4.b","mappings":[{"from":9,"to":8}]},{"pgid":"4.13","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.12:6801/4177251511":"2024-09-11T23:56:17.999820+0000","172.21.15.12:6800/4177251511":"2024-09-11T23:56:17.999820+0000","172.21.15.12:0/3273692817":"2024-09-11T23:56:17.999820+0000","172.21.15.12:0/4017778158":"2024-09-11T23:56:17.999820+0000","172.21.15.12:0/2499921051":"2024-09-11T23:55:50.302721+0000","172.21.15.12:6801/760140910":"2024-09-11T23:55:50.302721+0000","172.21.15.12:6800/760140910":"2024-09-11T23:55:50.302721+0000","172.21.15.12:0/2810703302":"2024-09-11T23:56:17.999820+0000","172.21.15.12:0/2325436065":"2024-09-11T23:55:50.302721+0000","172.21.15.12:0/3137053479":"2024-09-11T23:55:50.302721+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-11T00:32:21.168 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:21.166+0000 7f4a18800640 1 -- 172.21.15.12:0/1457347394 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f49f80b5360 msgr2=0x7f49f80b7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:32:21.168 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:21.166+0000 7f4a18800640 1 --2- 172.21.15.12:0/1457347394 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f49f80b5360 0x7f49f80b7820 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f4a04005d00 tx=0x7f4a0400b410 comp rx=0 tx=0).stop 2024-09-11T00:32:21.168 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:21.166+0000 7f4a18800640 1 -- 172.21.15.12:0/1457347394 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4a101066b0 msgr2=0x7f4a1019c1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:32:21.169 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:21.166+0000 7f4a18800640 1 --2- 172.21.15.12:0/1457347394 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4a101066b0 0x7f4a1019c1c0 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f4a0c00eb10 tx=0x7f4a0c00c2d0 comp rx=0 tx=0).stop 2024-09-11T00:32:21.169 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:21.166+0000 7f4a18800640 1 -- 172.21.15.12:0/1457347394 shutdown_connections 2024-09-11T00:32:21.169 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:21.166+0000 7f4a18800640 1 --2- 172.21.15.12:0/1457347394 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f49f80b5360 0x7f49f80b7820 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:32:21.169 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:21.166+0000 7f4a18800640 1 --2- 172.21.15.12:0/1457347394 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4a101066b0 0x7f4a1019c1c0 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:32:21.169 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:21.166+0000 7f4a18800640 1 --2- 172.21.15.12:0/1457347394 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4a10105cf0 0x7f4a1019b590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:32:21.169 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:21.166+0000 7f4a18800640 1 --2- 172.21.15.12:0/1457347394 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4a10104ba0 0x7f4a1019af40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:32:21.169 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:21.166+0000 7f4a18800640 1 -- 172.21.15.12:0/1457347394 >> 172.21.15.12:0/1457347394 conn(0x7f4a10100440 msgr2=0x7f4a10109010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:32:21.169 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:21.166+0000 7f4a18800640 1 -- 172.21.15.12:0/1457347394 shutdown_connections 2024-09-11T00:32:21.169 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:21.166+0000 7f4a18800640 1 -- 172.21.15.12:0/1457347394 wait complete. 2024-09-11T00:32:21.185 INFO:tasks.check_counter.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-10T23:59:10.488728+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-09-11T00:04:00.065618+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '74', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-09-11T00:04:02.689888+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '74', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-09-11T00:04:09.862120+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '81', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-09-11T00:32:21.185 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-11T00:32:21.347 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:21.344+0000 7f22ab200640 1 -- 172.21.15.12:0/3921528901 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f22ac105ae0 msgr2=0x7f22ac105f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:32:21.347 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:21.344+0000 7f22ab200640 1 --2- 172.21.15.12:0/3921528901 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f22ac105ae0 0x7f22ac105f60 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f2298009930 tx=0x7f229802a420 comp rx=0 tx=0).stop 2024-09-11T00:32:21.347 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:21.344+0000 7f22ab200640 1 -- 172.21.15.12:0/3921528901 shutdown_connections 2024-09-11T00:32:21.347 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:21.344+0000 7f22ab200640 1 --2- 172.21.15.12:0/3921528901 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f22ac1064a0 0x7f22ac10ce20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:32:21.347 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:21.344+0000 7f22ab200640 1 --2- 172.21.15.12:0/3921528901 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f22ac105ae0 0x7f22ac105f60 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:32:21.348 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:21.344+0000 7f22ab200640 1 --2- 172.21.15.12:0/3921528901 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f22ac104990 0x7f22ac104d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:32:21.348 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:21.344+0000 7f22ab200640 1 -- 172.21.15.12:0/3921528901 >> 172.21.15.12:0/3921528901 conn(0x7f22ac100270 msgr2=0x7f22ac102690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:32:21.348 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:21.344+0000 7f22ab200640 1 -- 172.21.15.12:0/3921528901 shutdown_connections 2024-09-11T00:32:21.348 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:21.344+0000 7f22ab200640 1 -- 172.21.15.12:0/3921528901 wait complete. 2024-09-11T00:32:21.350 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:21.345+0000 7f22ab200640 1 Processor -- start 2024-09-11T00:32:21.350 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:21.345+0000 7f22ab200640 1 -- start start 2024-09-11T00:32:21.351 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:21.345+0000 7f22ab200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f22ac104990 0x7f22ac19add0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:32:21.351 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:21.346+0000 7f22ab200640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f22ac105ae0 0x7f22ac19b420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:32:21.351 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:21.346+0000 7f22ab200640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f22ac1064a0 0x7f22ac19c050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:32:21.351 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:21.346+0000 7f22ab200640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f22ac1105e0 con 0x7f22ac1064a0 2024-09-11T00:32:21.351 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:21.346+0000 7f22ab200640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f22ac110460 con 0x7f22ac104990 2024-09-11T00:32:21.351 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:21.346+0000 7f22ab200640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f22ac110760 con 0x7f22ac105ae0 2024-09-11T00:32:21.351 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:21.346+0000 7f22aa800640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f22ac1064a0 0x7f22ac19c050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:32:21.351 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:21.346+0000 7f22aa800640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f22ac1064a0 0x7f22ac19c050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:34268/0 (socket says 172.21.15.12:34268) 2024-09-11T00:32:21.351 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:21.346+0000 7f22aa800640 1 -- 172.21.15.12:0/3061411925 learned_addr learned my addr 172.21.15.12:0/3061411925 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:32:21.351 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:21.346+0000 7f22a9e00640 1 --2- 172.21.15.12:0/3061411925 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f22ac104990 0x7f22ac19add0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:32:21.351 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:21.346+0000 7f22a9400640 1 --2- 172.21.15.12:0/3061411925 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f22ac105ae0 0x7f22ac19b420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:32:21.352 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:21.346+0000 7f22aa800640 1 -- 172.21.15.12:0/3061411925 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f22ac105ae0 msgr2=0x7f22ac19b420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:32:21.352 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:21.346+0000 7f22aa800640 1 --2- 172.21.15.12:0/3061411925 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f22ac105ae0 0x7f22ac19b420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:32:21.352 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:21.346+0000 7f22aa800640 1 -- 172.21.15.12:0/3061411925 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f22ac104990 msgr2=0x7f22ac19add0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:32:21.352 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:21.346+0000 7f22aa800640 1 --2- 172.21.15.12:0/3061411925 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f22ac104990 0x7f22ac19add0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:32:21.352 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:21.346+0000 7f22aa800640 1 -- 172.21.15.12:0/3061411925 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22ac19c910 con 0x7f22ac1064a0 2024-09-11T00:32:21.352 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:21.346+0000 7f22a9e00640 1 --2- 172.21.15.12:0/3061411925 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f22ac104990 0x7f22ac19add0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:32:21.352 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:21.347+0000 7f22aa800640 1 --2- 172.21.15.12:0/3061411925 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f22ac1064a0 0x7f22ac19c050 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f22a400d950 tx=0x7f22a400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:32:21.352 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:21.347+0000 7f22a2a00640 1 -- 172.21.15.12:0/3061411925 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22a4017030 con 0x7f22ac1064a0 2024-09-11T00:32:21.352 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:21.347+0000 7f22a2a00640 1 -- 172.21.15.12:0/3061411925 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f22a4004020 con 0x7f22ac1064a0 2024-09-11T00:32:21.352 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:21.347+0000 7f22ab200640 1 -- 172.21.15.12:0/3061411925 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22ac19cc00 con 0x7f22ac1064a0 2024-09-11T00:32:21.353 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:21.347+0000 7f22a2a00640 1 -- 172.21.15.12:0/3061411925 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22a4003b50 con 0x7f22ac1064a0 2024-09-11T00:32:21.353 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:21.347+0000 7f22ab200640 1 -- 172.21.15.12:0/3061411925 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22ac1a32d0 con 0x7f22ac1064a0 2024-09-11T00:32:21.353 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:21.348+0000 7f22ab200640 1 -- 172.21.15.12:0/3061411925 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f22ac104d90 con 0x7f22ac1064a0 2024-09-11T00:32:21.354 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:21.352+0000 7f22a2a00640 1 -- 172.21.15.12:0/3061411925 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7f22a4003cf0 con 0x7f22ac1064a0 2024-09-11T00:32:21.356 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:21.353+0000 7f22a2a00640 1 --2- 172.21.15.12:0/3061411925 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f22900b52e0 0x7f22900b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:32:21.356 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:21.353+0000 7f22a2a00640 1 -- 172.21.15.12:0/3061411925 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9208+0+0 (secure 0 0 0) 0x7f22a40e5d90 con 0x7f22ac1064a0 2024-09-11T00:32:21.356 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:21.354+0000 7f22a9e00640 1 --2- 172.21.15.12:0/3061411925 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f22900b52e0 0x7f22900b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:32:21.356 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:21.354+0000 7f22a9e00640 1 --2- 172.21.15.12:0/3061411925 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f22900b52e0 0x7f22900b77a0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f229c005fd0 tx=0x7f229c009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:32:21.357 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:21.355+0000 7f22a2a00640 1 -- 172.21.15.12:0/3061411925 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f22a4021070 con 0x7f22ac1064a0 2024-09-11T00:32:21.553 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:21.550+0000 7f22ab200640 1 -- 172.21.15.12:0/3061411925 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f22ac02e5c0 con 0x7f22ac1064a0 2024-09-11T00:32:21.553 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:21.551+0000 7f22a2a00640 1 -- 172.21.15.12:0/3061411925 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v87) ==== 93+0+10 (secure 0 0 0) 0x7f22a40ac820 con 0x7f22ac1064a0 2024-09-11T00:32:21.554 INFO:teuthology.orchestra.run.smithi012.stdout:pg_num: 1 2024-09-11T00:32:21.556 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:21.553+0000 7f22ab200640 1 -- 172.21.15.12:0/3061411925 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f22900b52e0 msgr2=0x7f22900b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:32:21.556 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:21.553+0000 7f22ab200640 1 --2- 172.21.15.12:0/3061411925 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f22900b52e0 0x7f22900b77a0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f229c005fd0 tx=0x7f229c009500 comp rx=0 tx=0).stop 2024-09-11T00:32:21.556 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:21.553+0000 7f22ab200640 1 -- 172.21.15.12:0/3061411925 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f22ac1064a0 msgr2=0x7f22ac19c050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:32:21.556 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:21.553+0000 7f22ab200640 1 --2- 172.21.15.12:0/3061411925 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f22ac1064a0 0x7f22ac19c050 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f22a400d950 tx=0x7f22a400de10 comp rx=0 tx=0).stop 2024-09-11T00:32:21.556 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:21.553+0000 7f22ab200640 1 -- 172.21.15.12:0/3061411925 shutdown_connections 2024-09-11T00:32:21.556 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:21.553+0000 7f22ab200640 1 --2- 172.21.15.12:0/3061411925 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f22900b52e0 0x7f22900b77a0 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:32:21.556 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:21.553+0000 7f22ab200640 1 --2- 172.21.15.12:0/3061411925 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f22ac1064a0 0x7f22ac19c050 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:32:21.556 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:21.553+0000 7f22ab200640 1 --2- 172.21.15.12:0/3061411925 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f22ac105ae0 0x7f22ac19b420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:32:21.556 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:21.553+0000 7f22ab200640 1 --2- 172.21.15.12:0/3061411925 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f22ac104990 0x7f22ac19add0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:32:21.556 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:21.553+0000 7f22ab200640 1 -- 172.21.15.12:0/3061411925 >> 172.21.15.12:0/3061411925 conn(0x7f22ac100270 msgr2=0x7f22ac108ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:32:21.557 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:21.553+0000 7f22ab200640 1 -- 172.21.15.12:0/3061411925 shutdown_connections 2024-09-11T00:32:21.557 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:21.553+0000 7f22ab200640 1 -- 172.21.15.12:0/3061411925 wait complete. 2024-09-11T00:32:21.573 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-09-11T00:32:21.735 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:21.732+0000 7f03f9400640 1 -- 172.21.15.12:0/1934010308 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f03f4104b80 msgr2=0x7f03f4104f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:32:21.735 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:21.732+0000 7f03f9400640 1 --2- 172.21.15.12:0/1934010308 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f03f4104b80 0x7f03f4104f80 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f03e4009930 tx=0x7f03e402a420 comp rx=0 tx=0).stop 2024-09-11T00:32:21.735 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:21.732+0000 7f03f9400640 1 -- 172.21.15.12:0/1934010308 shutdown_connections 2024-09-11T00:32:21.736 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:21.732+0000 7f03f9400640 1 --2- 172.21.15.12:0/1934010308 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f03f4106690 0x7f03f410d010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:32:21.736 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:21.732+0000 7f03f9400640 1 --2- 172.21.15.12:0/1934010308 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f03f4105cd0 0x7f03f4106150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:32:21.736 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:21.732+0000 7f03f9400640 1 --2- 172.21.15.12:0/1934010308 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f03f4104b80 0x7f03f4104f80 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:32:21.736 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:21.732+0000 7f03f9400640 1 -- 172.21.15.12:0/1934010308 >> 172.21.15.12:0/1934010308 conn(0x7f03f4100440 msgr2=0x7f03f4102860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:32:21.736 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:21.733+0000 7f03f9400640 1 -- 172.21.15.12:0/1934010308 shutdown_connections 2024-09-11T00:32:21.736 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:21.733+0000 7f03f9400640 1 -- 172.21.15.12:0/1934010308 wait complete. 2024-09-11T00:32:21.738 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:21.733+0000 7f03f9400640 1 Processor -- start 2024-09-11T00:32:21.738 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:21.734+0000 7f03f9400640 1 -- start start 2024-09-11T00:32:21.738 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:21.734+0000 7f03f9400640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f03f4104b80 0x7f03f419b000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:32:21.738 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:21.734+0000 7f03f9400640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f03f4105cd0 0x7f03f419b650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:32:21.738 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:21.734+0000 7f03f9400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f03f4106690 0x7f03f419c280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:32:21.738 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:21.734+0000 7f03f9400640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f03f4110840 con 0x7f03f4105cd0 2024-09-11T00:32:21.738 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:21.734+0000 7f03f9400640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f03f41106c0 con 0x7f03f4106690 2024-09-11T00:32:21.739 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:21.734+0000 7f03f9400640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f03f41109c0 con 0x7f03f4104b80 2024-09-11T00:32:21.739 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:21.734+0000 7f03e9e00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f03f4105cd0 0x7f03f419b650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:32:21.739 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:21.734+0000 7f03e9e00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f03f4105cd0 0x7f03f419b650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:34296/0 (socket says 172.21.15.12:34296) 2024-09-11T00:32:21.739 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:21.734+0000 7f03e9e00640 1 -- 172.21.15.12:0/4141014109 learned_addr learned my addr 172.21.15.12:0/4141014109 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:32:21.739 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:21.734+0000 7f03f2800640 1 --2- 172.21.15.12:0/4141014109 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f03f4104b80 0x7f03f419b000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:32:21.739 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:21.734+0000 7f03f3200640 1 --2- 172.21.15.12:0/4141014109 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f03f4106690 0x7f03f419c280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:32:21.739 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:21.735+0000 7f03e9e00640 1 -- 172.21.15.12:0/4141014109 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f03f4104b80 msgr2=0x7f03f419b000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:32:21.739 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:21.735+0000 7f03e9e00640 1 --2- 172.21.15.12:0/4141014109 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f03f4104b80 0x7f03f419b000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:32:21.739 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:21.735+0000 7f03e9e00640 1 -- 172.21.15.12:0/4141014109 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f03f4106690 msgr2=0x7f03f419c280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:32:21.739 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:21.735+0000 7f03e9e00640 1 --2- 172.21.15.12:0/4141014109 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f03f4106690 0x7f03f419c280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:32:21.739 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:21.735+0000 7f03e9e00640 1 -- 172.21.15.12:0/4141014109 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03f419cb40 con 0x7f03f4105cd0 2024-09-11T00:32:21.739 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:21.735+0000 7f03f2800640 1 --2- 172.21.15.12:0/4141014109 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f03f4104b80 0x7f03f419b000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T00:32:21.739 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:21.735+0000 7f03f3200640 1 --2- 172.21.15.12:0/4141014109 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f03f4106690 0x7f03f419c280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T00:32:21.739 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:21.735+0000 7f03e9e00640 1 --2- 172.21.15.12:0/4141014109 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f03f4105cd0 0x7f03f419b650 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f03dc00e9f0 tx=0x7f03dc00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:32:21.740 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:21.735+0000 7f03ebe00640 1 -- 172.21.15.12:0/4141014109 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03dc00cd80 con 0x7f03f4105cd0 2024-09-11T00:32:21.740 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:21.735+0000 7f03ebe00640 1 -- 172.21.15.12:0/4141014109 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f03dc0054d0 con 0x7f03f4105cd0 2024-09-11T00:32:21.740 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:21.735+0000 7f03f9400640 1 -- 172.21.15.12:0/4141014109 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03f419ce30 con 0x7f03f4105cd0 2024-09-11T00:32:21.740 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:21.735+0000 7f03ebe00640 1 -- 172.21.15.12:0/4141014109 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03dc018630 con 0x7f03f4105cd0 2024-09-11T00:32:21.740 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:21.735+0000 7f03f9400640 1 -- 172.21.15.12:0/4141014109 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03f41a3500 con 0x7f03f4105cd0 2024-09-11T00:32:21.740 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:21.737+0000 7f03f9400640 1 -- 172.21.15.12:0/4141014109 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f03f4104f80 con 0x7f03f4105cd0 2024-09-11T00:32:21.741 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:21.739+0000 7f03ebe00640 1 -- 172.21.15.12:0/4141014109 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7f03dc004e70 con 0x7f03f4105cd0 2024-09-11T00:32:21.743 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:21.740+0000 7f03ebe00640 1 --2- 172.21.15.12:0/4141014109 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f03d80b52e0 0x7f03d80b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:32:21.743 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:21.741+0000 7f03ebe00640 1 -- 172.21.15.12:0/4141014109 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9208+0+0 (secure 0 0 0) 0x7f03dc0e6c00 con 0x7f03f4105cd0 2024-09-11T00:32:21.743 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:21.741+0000 7f03f2800640 1 --2- 172.21.15.12:0/4141014109 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f03d80b52e0 0x7f03d80b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:32:21.743 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:21.741+0000 7f03f2800640 1 --2- 172.21.15.12:0/4141014109 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f03d80b52e0 0x7f03d80b77a0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f03e4006010 tx=0x7f03e4000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:32:21.744 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:21.742+0000 7f03ebe00640 1 -- 172.21.15.12:0/4141014109 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f03dc0ad6f0 con 0x7f03f4105cd0 2024-09-11T00:32:21.941 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:21.938+0000 7f03f9400640 1 -- 172.21.15.12:0/4141014109 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f03f402e5c0 con 0x7f03f4105cd0 2024-09-11T00:32:21.941 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:21.939+0000 7f03ebe00640 1 -- 172.21.15.12:0/4141014109 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v87) ==== 104+0+11 (secure 0 0 0) 0x7f03dc0acff0 con 0x7f03f4105cd0 2024-09-11T00:32:21.941 INFO:teuthology.orchestra.run.smithi012.stdout:pg_num: 32 2024-09-11T00:32:21.944 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:21.941+0000 7f03f9400640 1 -- 172.21.15.12:0/4141014109 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f03d80b52e0 msgr2=0x7f03d80b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:32:21.944 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:21.941+0000 7f03f9400640 1 --2- 172.21.15.12:0/4141014109 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f03d80b52e0 0x7f03d80b77a0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f03e4006010 tx=0x7f03e4000bc0 comp rx=0 tx=0).stop 2024-09-11T00:32:21.944 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:21.941+0000 7f03f9400640 1 -- 172.21.15.12:0/4141014109 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f03f4105cd0 msgr2=0x7f03f419b650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:32:21.944 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:21.941+0000 7f03f9400640 1 --2- 172.21.15.12:0/4141014109 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f03f4105cd0 0x7f03f419b650 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f03dc00e9f0 tx=0x7f03dc00edf0 comp rx=0 tx=0).stop 2024-09-11T00:32:21.945 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:21.942+0000 7f03f9400640 1 -- 172.21.15.12:0/4141014109 shutdown_connections 2024-09-11T00:32:21.945 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:21.942+0000 7f03f9400640 1 --2- 172.21.15.12:0/4141014109 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f03d80b52e0 0x7f03d80b77a0 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:32:21.945 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:21.942+0000 7f03f9400640 1 --2- 172.21.15.12:0/4141014109 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f03f4106690 0x7f03f419c280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:32:21.945 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:21.942+0000 7f03f9400640 1 --2- 172.21.15.12:0/4141014109 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f03f4105cd0 0x7f03f419b650 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:32:21.945 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:21.942+0000 7f03f9400640 1 --2- 172.21.15.12:0/4141014109 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f03f4104b80 0x7f03f419b000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:32:21.945 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:21.942+0000 7f03f9400640 1 -- 172.21.15.12:0/4141014109 >> 172.21.15.12:0/4141014109 conn(0x7f03f4100440 msgr2=0x7f03f4109060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:32:21.945 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:21.942+0000 7f03f9400640 1 -- 172.21.15.12:0/4141014109 shutdown_connections 2024-09-11T00:32:21.945 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:21.942+0000 7f03f9400640 1 -- 172.21.15.12:0/4141014109 wait complete. 2024-09-11T00:32:21.960 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-09-11T00:32:22.122 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:22.119+0000 7f0b75000640 1 -- 172.21.15.12:0/1676094538 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0b70104b80 msgr2=0x7f0b70104f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:32:22.122 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:22.119+0000 7f0b75000640 1 --2- 172.21.15.12:0/1676094538 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0b70104b80 0x7f0b70104f80 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f0b60009930 tx=0x7f0b6002a420 comp rx=0 tx=0).stop 2024-09-11T00:32:22.122 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:22.119+0000 7f0b75000640 1 -- 172.21.15.12:0/1676094538 shutdown_connections 2024-09-11T00:32:22.122 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:22.119+0000 7f0b75000640 1 --2- 172.21.15.12:0/1676094538 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0b70106690 0x7f0b7010d010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:32:22.122 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:22.119+0000 7f0b75000640 1 --2- 172.21.15.12:0/1676094538 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0b70105cd0 0x7f0b70106150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:32:22.122 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:22.119+0000 7f0b75000640 1 --2- 172.21.15.12:0/1676094538 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0b70104b80 0x7f0b70104f80 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:32:22.122 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:22.119+0000 7f0b75000640 1 -- 172.21.15.12:0/1676094538 >> 172.21.15.12:0/1676094538 conn(0x7f0b70100440 msgr2=0x7f0b70102860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:32:22.122 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:22.119+0000 7f0b75000640 1 -- 172.21.15.12:0/1676094538 shutdown_connections 2024-09-11T00:32:22.122 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:22.119+0000 7f0b75000640 1 -- 172.21.15.12:0/1676094538 wait complete. 2024-09-11T00:32:22.124 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:22.120+0000 7f0b75000640 1 Processor -- start 2024-09-11T00:32:22.124 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:22.120+0000 7f0b75000640 1 -- start start 2024-09-11T00:32:22.125 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:22.120+0000 7f0b75000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0b70104b80 0x7f0b7019afb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:32:22.125 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:22.120+0000 7f0b75000640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0b70105cd0 0x7f0b7019b600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:32:22.125 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:22.121+0000 7f0b75000640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0b70106690 0x7f0b7019c230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:32:22.125 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:22.121+0000 7f0b75000640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f0b701107f0 con 0x7f0b70106690 2024-09-11T00:32:22.125 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:22.121+0000 7f0b75000640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f0b70110670 con 0x7f0b70104b80 2024-09-11T00:32:22.125 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:22.121+0000 7f0b75000640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f0b70110970 con 0x7f0b70105cd0 2024-09-11T00:32:22.125 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:22.121+0000 7f0b6e800640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0b70106690 0x7f0b7019c230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:32:22.125 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:22.121+0000 7f0b6e800640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0b70106690 0x7f0b7019c230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:34322/0 (socket says 172.21.15.12:34322) 2024-09-11T00:32:22.125 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:22.121+0000 7f0b6e800640 1 -- 172.21.15.12:0/3943533436 learned_addr learned my addr 172.21.15.12:0/3943533436 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:32:22.125 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:22.121+0000 7f0b6d400640 1 --2- 172.21.15.12:0/3943533436 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0b70105cd0 0x7f0b7019b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:32:22.125 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:22.121+0000 7f0b6de00640 1 --2- 172.21.15.12:0/3943533436 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0b70104b80 0x7f0b7019afb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:32:22.125 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:22.121+0000 7f0b6e800640 1 -- 172.21.15.12:0/3943533436 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0b70105cd0 msgr2=0x7f0b7019b600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:32:22.126 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:22.121+0000 7f0b6e800640 1 --2- 172.21.15.12:0/3943533436 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0b70105cd0 0x7f0b7019b600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:32:22.126 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:22.121+0000 7f0b6e800640 1 -- 172.21.15.12:0/3943533436 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0b70104b80 msgr2=0x7f0b7019afb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:32:22.126 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:22.121+0000 7f0b6e800640 1 --2- 172.21.15.12:0/3943533436 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0b70104b80 0x7f0b7019afb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:32:22.126 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:22.121+0000 7f0b6e800640 1 -- 172.21.15.12:0/3943533436 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b7019caf0 con 0x7f0b70106690 2024-09-11T00:32:22.126 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:22.122+0000 7f0b6e800640 1 --2- 172.21.15.12:0/3943533436 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0b70106690 0x7f0b7019c230 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f0b6800eb10 tx=0x7f0b6800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:32:22.126 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:22.122+0000 7f0b66a00640 1 -- 172.21.15.12:0/3943533436 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b68004b20 con 0x7f0b70106690 2024-09-11T00:32:22.126 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:22.122+0000 7f0b66a00640 1 -- 172.21.15.12:0/3943533436 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0b68004cc0 con 0x7f0b70106690 2024-09-11T00:32:22.126 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:22.122+0000 7f0b66a00640 1 -- 172.21.15.12:0/3943533436 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b68010670 con 0x7f0b70106690 2024-09-11T00:32:22.126 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:22.122+0000 7f0b75000640 1 -- 172.21.15.12:0/3943533436 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0b7019cde0 con 0x7f0b70106690 2024-09-11T00:32:22.126 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:22.122+0000 7f0b75000640 1 -- 172.21.15.12:0/3943533436 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0b701a34b0 con 0x7f0b70106690 2024-09-11T00:32:22.127 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:22.123+0000 7f0b75000640 1 -- 172.21.15.12:0/3943533436 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0b70104f80 con 0x7f0b70106690 2024-09-11T00:32:22.127 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:22.125+0000 7f0b66a00640 1 -- 172.21.15.12:0/3943533436 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7f0b68005020 con 0x7f0b70106690 2024-09-11T00:32:22.129 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:22.126+0000 7f0b66a00640 1 --2- 172.21.15.12:0/3943533436 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f0b540b52e0 0x7f0b540b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:32:22.129 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:22.127+0000 7f0b66a00640 1 -- 172.21.15.12:0/3943533436 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9208+0+0 (secure 0 0 0) 0x7f0b680e6560 con 0x7f0b70106690 2024-09-11T00:32:22.129 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:22.127+0000 7f0b6de00640 1 --2- 172.21.15.12:0/3943533436 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f0b540b52e0 0x7f0b540b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:32:22.130 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:22.127+0000 7f0b6de00640 1 --2- 172.21.15.12:0/3943533436 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f0b540b52e0 0x7f0b540b77a0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f0b60009900 tx=0x7f0b60000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:32:22.132 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:22.130+0000 7f0b66a00640 1 -- 172.21.15.12:0/3943533436 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f0b680acff0 con 0x7f0b70106690 2024-09-11T00:32:22.283 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:32:21 smithi165 ceph-mon[35356]: pgmap v1158: 129 pgs: 129 active+clean; 671 MiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 324 KiB/s rd, 341 KiB/s wr, 177 op/s 2024-09-11T00:32:22.283 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:32:21 smithi165 ceph-mon[35356]: from='client.? 172.21.15.12:0/1457347394' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-11T00:32:22.283 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:32:21 smithi165 ceph-mon[35356]: from='client.? 172.21.15.12:0/3061411925' entity='client.admin' cmd={"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} : dispatch 2024-09-11T00:32:22.283 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:32:21 smithi165 ceph-mon[35356]: from='client.? 172.21.15.12:0/4141014109' entity='client.admin' cmd={"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} : dispatch 2024-09-11T00:32:22.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:32:21 smithi012 ceph-mon[30333]: pgmap v1158: 129 pgs: 129 active+clean; 671 MiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 324 KiB/s rd, 341 KiB/s wr, 177 op/s 2024-09-11T00:32:22.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:32:21 smithi012 ceph-mon[30333]: from='client.? 172.21.15.12:0/1457347394' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-11T00:32:22.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:32:21 smithi012 ceph-mon[30333]: from='client.? 172.21.15.12:0/3061411925' entity='client.admin' cmd={"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} : dispatch 2024-09-11T00:32:22.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:32:21 smithi012 ceph-mon[30333]: from='client.? 172.21.15.12:0/4141014109' entity='client.admin' cmd={"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} : dispatch 2024-09-11T00:32:22.330 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:22.328+0000 7f0b75000640 1 -- 172.21.15.12:0/3943533436 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f0b70063d50 con 0x7f0b70106690 2024-09-11T00:32:22.331 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:22.328+0000 7f0b66a00640 1 -- 172.21.15.12:0/3943533436 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v87) ==== 100+0+11 (secure 0 0 0) 0x7f0b680ac8f0 con 0x7f0b70106690 2024-09-11T00:32:22.331 INFO:teuthology.orchestra.run.smithi012.stdout:pg_num: 64 2024-09-11T00:32:22.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:32:21 smithi178 ceph-mon[35919]: pgmap v1158: 129 pgs: 129 active+clean; 671 MiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 324 KiB/s rd, 341 KiB/s wr, 177 op/s 2024-09-11T00:32:22.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:32:21 smithi178 ceph-mon[35919]: from='client.? 172.21.15.12:0/1457347394' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-11T00:32:22.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:32:21 smithi178 ceph-mon[35919]: from='client.? 172.21.15.12:0/3061411925' entity='client.admin' cmd={"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} : dispatch 2024-09-11T00:32:22.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:32:21 smithi178 ceph-mon[35919]: from='client.? 172.21.15.12:0/4141014109' entity='client.admin' cmd={"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} : dispatch 2024-09-11T00:32:22.334 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:22.331+0000 7f0b75000640 1 -- 172.21.15.12:0/3943533436 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f0b540b52e0 msgr2=0x7f0b540b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:32:22.334 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:22.331+0000 7f0b75000640 1 --2- 172.21.15.12:0/3943533436 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f0b540b52e0 0x7f0b540b77a0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f0b60009900 tx=0x7f0b60000bc0 comp rx=0 tx=0).stop 2024-09-11T00:32:22.334 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:22.331+0000 7f0b75000640 1 -- 172.21.15.12:0/3943533436 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0b70106690 msgr2=0x7f0b7019c230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:32:22.334 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:22.331+0000 7f0b75000640 1 --2- 172.21.15.12:0/3943533436 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0b70106690 0x7f0b7019c230 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f0b6800eb10 tx=0x7f0b6800c2d0 comp rx=0 tx=0).stop 2024-09-11T00:32:22.334 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:22.331+0000 7f0b75000640 1 -- 172.21.15.12:0/3943533436 shutdown_connections 2024-09-11T00:32:22.334 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:22.331+0000 7f0b75000640 1 --2- 172.21.15.12:0/3943533436 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f0b540b52e0 0x7f0b540b77a0 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:32:22.334 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:22.331+0000 7f0b75000640 1 --2- 172.21.15.12:0/3943533436 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0b70106690 0x7f0b7019c230 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:32:22.334 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:22.331+0000 7f0b75000640 1 --2- 172.21.15.12:0/3943533436 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0b70105cd0 0x7f0b7019b600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:32:22.334 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:22.331+0000 7f0b75000640 1 --2- 172.21.15.12:0/3943533436 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0b70104b80 0x7f0b7019afb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:32:22.334 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:22.331+0000 7f0b75000640 1 -- 172.21.15.12:0/3943533436 >> 172.21.15.12:0/3943533436 conn(0x7f0b70100440 msgr2=0x7f0b70109060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:32:22.335 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:22.331+0000 7f0b75000640 1 -- 172.21.15.12:0/3943533436 shutdown_connections 2024-09-11T00:32:22.335 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:22.331+0000 7f0b75000640 1 -- 172.21.15.12:0/3943533436 wait complete. 2024-09-11T00:32:22.352 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-09-11T00:32:22.516 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:22.513+0000 7fa00e000640 1 -- 172.21.15.12:0/2927365932 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa008103900 msgr2=0x7fa008103d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:32:22.516 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:22.513+0000 7fa00e000640 1 --2- 172.21.15.12:0/2927365932 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa008103900 0x7fa008103d80 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f9ff0009860 tx=0x7f9ff002a340 comp rx=0 tx=0).stop 2024-09-11T00:32:22.516 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:22.513+0000 7fa00e000640 1 -- 172.21.15.12:0/2927365932 shutdown_connections 2024-09-11T00:32:22.516 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:22.513+0000 7fa00e000640 1 --2- 172.21.15.12:0/2927365932 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa0081042c0 0x7fa00810ac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:32:22.516 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:22.513+0000 7fa00e000640 1 --2- 172.21.15.12:0/2927365932 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa008103900 0x7fa008103d80 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:32:22.516 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:22.513+0000 7fa00e000640 1 --2- 172.21.15.12:0/2927365932 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa0081027b0 0x7fa008102bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:32:22.517 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:22.513+0000 7fa00e000640 1 -- 172.21.15.12:0/2927365932 >> 172.21.15.12:0/2927365932 conn(0x7fa0080fe050 msgr2=0x7fa008100490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:32:22.517 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:22.513+0000 7fa00e000640 1 -- 172.21.15.12:0/2927365932 shutdown_connections 2024-09-11T00:32:22.517 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:22.513+0000 7fa00e000640 1 -- 172.21.15.12:0/2927365932 wait complete. 2024-09-11T00:32:22.518 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:22.514+0000 7fa00e000640 1 Processor -- start 2024-09-11T00:32:22.519 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:22.514+0000 7fa00e000640 1 -- start start 2024-09-11T00:32:22.519 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:22.514+0000 7fa00e000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa0081027b0 0x7fa00819ad50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:32:22.519 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:22.515+0000 7fa00e000640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa008103900 0x7fa00819b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:32:22.519 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:22.515+0000 7fa00e000640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa0081042c0 0x7fa00819bfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:32:22.519 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:22.515+0000 7fa00e000640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fa00810e400 con 0x7fa008103900 2024-09-11T00:32:22.519 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:22.515+0000 7fa00e000640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fa00810e280 con 0x7fa0081027b0 2024-09-11T00:32:22.519 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:22.515+0000 7fa00e000640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fa00810e580 con 0x7fa0081042c0 2024-09-11T00:32:22.519 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:22.515+0000 7f9fffe00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa008103900 0x7fa00819b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:32:22.519 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:22.515+0000 7f9fffe00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa008103900 0x7fa00819b3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:34354/0 (socket says 172.21.15.12:34354) 2024-09-11T00:32:22.519 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:22.515+0000 7f9fffe00640 1 -- 172.21.15.12:0/4219392550 learned_addr learned my addr 172.21.15.12:0/4219392550 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:32:22.519 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:22.515+0000 7fa00cc00640 1 --2- 172.21.15.12:0/4219392550 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa0081027b0 0x7fa00819ad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:32:22.519 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:22.515+0000 7fa00d600640 1 --2- 172.21.15.12:0/4219392550 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa0081042c0 0x7fa00819bfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:32:22.520 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:22.515+0000 7f9fffe00640 1 -- 172.21.15.12:0/4219392550 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa0081042c0 msgr2=0x7fa00819bfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:32:22.520 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:22.515+0000 7f9fffe00640 1 --2- 172.21.15.12:0/4219392550 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa0081042c0 0x7fa00819bfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:32:22.520 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:22.515+0000 7f9fffe00640 1 -- 172.21.15.12:0/4219392550 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa0081027b0 msgr2=0x7fa00819ad50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:32:22.520 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:22.515+0000 7f9fffe00640 1 --2- 172.21.15.12:0/4219392550 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa0081027b0 0x7fa00819ad50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:32:22.520 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:22.515+0000 7f9fffe00640 1 -- 172.21.15.12:0/4219392550 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa00819c890 con 0x7fa008103900 2024-09-11T00:32:22.520 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:22.516+0000 7f9fffe00640 1 --2- 172.21.15.12:0/4219392550 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa008103900 0x7fa00819b3a0 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f9ff0005940 tx=0x7f9ff0032dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:32:22.520 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:22.516+0000 7fa006000640 1 -- 172.21.15.12:0/4219392550 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ff0040020 con 0x7fa008103900 2024-09-11T00:32:22.520 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:22.516+0000 7fa006000640 1 -- 172.21.15.12:0/4219392550 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9ff0030e80 con 0x7fa008103900 2024-09-11T00:32:22.520 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:22.516+0000 7fa00e000640 1 -- 172.21.15.12:0/4219392550 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa00819cb20 con 0x7fa008103900 2024-09-11T00:32:22.520 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:22.516+0000 7fa006000640 1 -- 172.21.15.12:0/4219392550 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ff0037040 con 0x7fa008103900 2024-09-11T00:32:22.520 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:22.516+0000 7fa00e000640 1 -- 172.21.15.12:0/4219392550 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0081a3250 con 0x7fa008103900 2024-09-11T00:32:22.521 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:22.517+0000 7fa00e000640 1 -- 172.21.15.12:0/4219392550 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa008102bb0 con 0x7fa008103900 2024-09-11T00:32:22.521 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:22.519+0000 7fa006000640 1 -- 172.21.15.12:0/4219392550 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7f9ff004d060 con 0x7fa008103900 2024-09-11T00:32:22.524 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:22.521+0000 7fa006000640 1 --2- 172.21.15.12:0/4219392550 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f9fec0b52e0 0x7f9fec0b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:32:22.524 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:22.521+0000 7fa006000640 1 -- 172.21.15.12:0/4219392550 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9208+0+0 (secure 0 0 0) 0x7f9ff0105ca0 con 0x7fa008103900 2024-09-11T00:32:22.524 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:22.521+0000 7fa00cc00640 1 --2- 172.21.15.12:0/4219392550 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f9fec0b52e0 0x7f9fec0b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:32:22.524 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:22.522+0000 7fa00cc00640 1 --2- 172.21.15.12:0/4219392550 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f9fec0b52e0 0x7f9fec0b77a0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f9ff8009bd0 tx=0x7f9ff8005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:32:22.527 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:22.525+0000 7fa006000640 1 -- 172.21.15.12:0/4219392550 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f9ff004a020 con 0x7fa008103900 2024-09-11T00:32:22.725 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:22.723+0000 7fa00e000640 1 -- 172.21.15.12:0/4219392550 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7fa0081a3420 con 0x7fa008103900 2024-09-11T00:32:22.726 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:22.723+0000 7fa006000640 1 -- 172.21.15.12:0/4219392550 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v87) ==== 103+0+11 (secure 0 0 0) 0x7f9ff00328c0 con 0x7fa008103900 2024-09-11T00:32:22.726 INFO:teuthology.orchestra.run.smithi012.stdout:pg_num: 32 2024-09-11T00:32:22.728 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:22.726+0000 7fa00e000640 1 -- 172.21.15.12:0/4219392550 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f9fec0b52e0 msgr2=0x7f9fec0b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:32:22.728 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:22.726+0000 7fa00e000640 1 --2- 172.21.15.12:0/4219392550 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f9fec0b52e0 0x7f9fec0b77a0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f9ff8009bd0 tx=0x7f9ff8005f90 comp rx=0 tx=0).stop 2024-09-11T00:32:22.728 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:22.726+0000 7fa00e000640 1 -- 172.21.15.12:0/4219392550 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa008103900 msgr2=0x7fa00819b3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:32:22.729 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:22.726+0000 7fa00e000640 1 --2- 172.21.15.12:0/4219392550 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa008103900 0x7fa00819b3a0 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f9ff0005940 tx=0x7f9ff0032dc0 comp rx=0 tx=0).stop 2024-09-11T00:32:22.729 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:22.726+0000 7fa00e000640 1 -- 172.21.15.12:0/4219392550 shutdown_connections 2024-09-11T00:32:22.729 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:22.726+0000 7fa00e000640 1 --2- 172.21.15.12:0/4219392550 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f9fec0b52e0 0x7f9fec0b77a0 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:32:22.729 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:22.726+0000 7fa00e000640 1 --2- 172.21.15.12:0/4219392550 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa0081042c0 0x7fa00819bfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:32:22.729 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:22.726+0000 7fa00e000640 1 --2- 172.21.15.12:0/4219392550 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa008103900 0x7fa00819b3a0 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:32:22.729 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:22.726+0000 7fa00e000640 1 --2- 172.21.15.12:0/4219392550 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa0081027b0 0x7fa00819ad50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:32:22.729 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:22.726+0000 7fa00e000640 1 -- 172.21.15.12:0/4219392550 >> 172.21.15.12:0/4219392550 conn(0x7fa0080fe050 msgr2=0x7fa008075380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:32:22.729 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:22.726+0000 7fa00e000640 1 -- 172.21.15.12:0/4219392550 shutdown_connections 2024-09-11T00:32:22.729 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:22.726+0000 7fa00e000640 1 -- 172.21.15.12:0/4219392550 wait complete. 2024-09-11T00:32:22.744 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json-pretty 2024-09-11T00:32:22.904 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:22.901+0000 7efc52a00640 1 -- 172.21.15.12:0/2828806350 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efc4c104ba0 msgr2=0x7efc4c104fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:32:22.904 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:22.901+0000 7efc52a00640 1 --2- 172.21.15.12:0/2828806350 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efc4c104ba0 0x7efc4c104fa0 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7efc3c009cd0 tx=0x7efc3c02a8f0 comp rx=0 tx=0).stop 2024-09-11T00:32:22.905 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:22.901+0000 7efc52a00640 1 -- 172.21.15.12:0/2828806350 shutdown_connections 2024-09-11T00:32:22.905 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:22.901+0000 7efc52a00640 1 --2- 172.21.15.12:0/2828806350 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efc4c1066b0 0x7efc4c10d030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:32:22.905 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:22.901+0000 7efc52a00640 1 --2- 172.21.15.12:0/2828806350 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7efc4c105cf0 0x7efc4c106170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:32:22.905 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:22.901+0000 7efc52a00640 1 --2- 172.21.15.12:0/2828806350 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efc4c104ba0 0x7efc4c104fa0 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:32:22.905 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:22.901+0000 7efc52a00640 1 -- 172.21.15.12:0/2828806350 >> 172.21.15.12:0/2828806350 conn(0x7efc4c100440 msgr2=0x7efc4c102880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:32:22.905 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:22.902+0000 7efc52a00640 1 -- 172.21.15.12:0/2828806350 shutdown_connections 2024-09-11T00:32:22.905 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:22.902+0000 7efc52a00640 1 -- 172.21.15.12:0/2828806350 wait complete. 2024-09-11T00:32:22.907 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:22.902+0000 7efc52a00640 1 Processor -- start 2024-09-11T00:32:22.907 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:22.903+0000 7efc52a00640 1 -- start start 2024-09-11T00:32:22.907 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:22.903+0000 7efc52a00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efc4c104ba0 0x7efc4c19af70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:32:22.907 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:22.903+0000 7efc52a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7efc4c105cf0 0x7efc4c19b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:32:22.907 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:22.903+0000 7efc52a00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efc4c1066b0 0x7efc4c19c1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:32:22.907 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:22.903+0000 7efc52a00640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7efc4c1107f0 con 0x7efc4c104ba0 2024-09-11T00:32:22.907 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:22.903+0000 7efc52a00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7efc4c110670 con 0x7efc4c105cf0 2024-09-11T00:32:22.907 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:22.903+0000 7efc52a00640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7efc4c110970 con 0x7efc4c1066b0 2024-09-11T00:32:22.908 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:22.903+0000 7efc4be00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efc4c104ba0 0x7efc4c19af70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:32:22.908 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:22.903+0000 7efc50a00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efc4c1066b0 0x7efc4c19c1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:32:22.908 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:22.903+0000 7efc50a00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efc4c1066b0 0x7efc4c19c1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.12:53696/0 (socket says 172.21.15.12:53696) 2024-09-11T00:32:22.908 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:22.903+0000 7efc50a00640 1 -- 172.21.15.12:0/3734959530 learned_addr learned my addr 172.21.15.12:0/3734959530 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:32:22.908 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:22.903+0000 7efc43400640 1 --2- 172.21.15.12:0/3734959530 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7efc4c105cf0 0x7efc4c19b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:32:22.908 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:22.904+0000 7efc50a00640 1 -- 172.21.15.12:0/3734959530 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7efc4c105cf0 msgr2=0x7efc4c19b5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:32:22.908 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:22.904+0000 7efc50a00640 1 --2- 172.21.15.12:0/3734959530 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7efc4c105cf0 0x7efc4c19b5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:32:22.908 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:22.904+0000 7efc50a00640 1 -- 172.21.15.12:0/3734959530 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efc4c104ba0 msgr2=0x7efc4c19af70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:32:22.908 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:22.904+0000 7efc50a00640 1 --2- 172.21.15.12:0/3734959530 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efc4c104ba0 0x7efc4c19af70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:32:22.908 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:22.904+0000 7efc50a00640 1 -- 172.21.15.12:0/3734959530 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc4c19cab0 con 0x7efc4c1066b0 2024-09-11T00:32:22.908 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:22.904+0000 7efc50a00640 1 --2- 172.21.15.12:0/3734959530 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efc4c1066b0 0x7efc4c19c1f0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7efc4400e9b0 tx=0x7efc4400ee70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:32:22.908 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:22.904+0000 7efc49600640 1 -- 172.21.15.12:0/3734959530 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc4400cd80 con 0x7efc4c1066b0 2024-09-11T00:32:22.908 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:22.904+0000 7efc52a00640 1 -- 172.21.15.12:0/3734959530 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efc4c19cda0 con 0x7efc4c1066b0 2024-09-11T00:32:22.909 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:22.904+0000 7efc49600640 1 -- 172.21.15.12:0/3734959530 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efc440035f0 con 0x7efc4c1066b0 2024-09-11T00:32:22.909 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:22.904+0000 7efc49600640 1 -- 172.21.15.12:0/3734959530 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc44010690 con 0x7efc4c1066b0 2024-09-11T00:32:22.909 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:22.904+0000 7efc52a00640 1 -- 172.21.15.12:0/3734959530 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efc4c1a3470 con 0x7efc4c1066b0 2024-09-11T00:32:22.909 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:22.905+0000 7efc52a00640 1 -- 172.21.15.12:0/3734959530 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efc4c104fa0 con 0x7efc4c1066b0 2024-09-11T00:32:22.910 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:22.908+0000 7efc49600640 1 -- 172.21.15.12:0/3734959530 <== mon.1 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7efc44004000 con 0x7efc4c1066b0 2024-09-11T00:32:22.912 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:22.909+0000 7efc49600640 1 --2- 172.21.15.12:0/3734959530 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7efc300b52e0 0x7efc300b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:32:22.912 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:22.909+0000 7efc49600640 1 -- 172.21.15.12:0/3734959530 <== mon.1 v2:172.21.15.178:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9208+0+0 (secure 0 0 0) 0x7efc440e6330 con 0x7efc4c1066b0 2024-09-11T00:32:22.912 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:22.910+0000 7efc4be00640 1 --2- 172.21.15.12:0/3734959530 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7efc300b52e0 0x7efc300b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:32:22.912 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:22.910+0000 7efc4be00640 1 --2- 172.21.15.12:0/3734959530 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7efc300b52e0 0x7efc300b77a0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7efc3c005e30 tx=0x7efc3c005dc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:32:22.913 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:22.911+0000 7efc49600640 1 -- 172.21.15.12:0/3734959530 <== mon.1 v2:172.21.15.178:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7efc440ace50 con 0x7efc4c1066b0 2024-09-11T00:32:23.063 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:32:22 smithi012 ceph-mon[30333]: from='client.? 172.21.15.12:0/3943533436' entity='client.admin' cmd={"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} : dispatch 2024-09-11T00:32:23.063 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:32:22 smithi012 ceph-mon[30333]: from='client.? 172.21.15.12:0/4219392550' entity='client.admin' cmd={"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} : dispatch 2024-09-11T00:32:23.159 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:23.157+0000 7efc52a00640 1 -- 172.21.15.12:0/3734959530 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json-pretty"} v 0) -- 0x7efc4c02e660 con 0x7efc4c1066b0 2024-09-11T00:32:23.168 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:23.166+0000 7efc49600640 1 -- 172.21.15.12:0/3734959530 <== mon.1 v2:172.21.15.178:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json-pretty"}]=0 v18) ==== 81+0+827944 (secure 0 0 0) 0x7efc4c02e660 con 0x7efc4c1066b0 2024-09-11T00:32:23.181 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-11T00:32:23.181 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-09-11T00:32:23.181 INFO:teuthology.orchestra.run.smithi012.stdout: "epoch": 18, 2024-09-11T00:32:23.182 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:23.182 INFO:teuthology.orchestra.run.smithi012.stdout: "active_gid": 14150, 2024-09-11T00:32:23.182 INFO:teuthology.orchestra.run.smithi012.stdout: "active_name": "x", 2024-09-11T00:32:23.182 INFO:teuthology.orchestra.run.smithi012.stdout: "active_addrs": { 2024-09-11T00:32:23.182 INFO:teuthology.orchestra.run.smithi012.stdout: "addrvec": [ 2024-09-11T00:32:23.182 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-11T00:32:23.182 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v2", 2024-09-11T00:32:23.182 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.12:6800", 2024-09-11T00:32:23.182 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 3442939137 2024-09-11T00:32:23.182 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.182 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-11T00:32:23.182 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v1", 2024-09-11T00:32:23.182 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.12:6801", 2024-09-11T00:32:23.183 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 3442939137 2024-09-11T00:32:23.183 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:32:23.183 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-11T00:32:23.183 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.183 INFO:teuthology.orchestra.run.smithi012.stdout: "active_addr": "172.21.15.12:6801/3442939137", 2024-09-11T00:32:23.183 INFO:teuthology.orchestra.run.smithi012.stdout: "active_change": "2024-09-10T23:56:17.999938+0000", 2024-09-11T00:32:23.183 INFO:teuthology.orchestra.run.smithi012.stdout: "active_mgr_features": 4540719139924082687, 2024-09-11T00:32:23.183 INFO:teuthology.orchestra.run.smithi012.stdout: "available": true, 2024-09-11T00:32:23.183 INFO:teuthology.orchestra.run.smithi012.stdout: "standbys": [ 2024-09-11T00:32:23.183 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-11T00:32:23.183 INFO:teuthology.orchestra.run.smithi012.stdout: "gid": 24124, 2024-09-11T00:32:23.183 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "z", 2024-09-11T00:32:23.183 INFO:teuthology.orchestra.run.smithi012.stdout: "mgr_features": 4540719139924082687, 2024-09-11T00:32:23.184 INFO:teuthology.orchestra.run.smithi012.stdout: "available_modules": [ 2024-09-11T00:32:23.184 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-11T00:32:23.184 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "alerts", 2024-09-11T00:32:23.184 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-09-11T00:32:23.184 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-09-11T00:32:23.184 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-09-11T00:32:23.184 INFO:teuthology.orchestra.run.smithi012.stdout: "interval": { 2024-09-11T00:32:23.184 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "interval", 2024-09-11T00:32:23.184 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "secs", 2024-09-11T00:32:23.184 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.184 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:23.184 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "60", 2024-09-11T00:32:23.184 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.185 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.185 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.185 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "How frequently to reexamine health status", 2024-09-11T00:32:23.185 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.185 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.185 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.185 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.185 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-09-11T00:32:23.185 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-09-11T00:32:23.185 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:23.185 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.185 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:23.185 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-09-11T00:32:23.186 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.186 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.186 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-09-11T00:32:23.186 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-09-11T00:32:23.186 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-09-11T00:32:23.186 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-09-11T00:32:23.186 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-09-11T00:32:23.186 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-09-11T00:32:23.186 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-09-11T00:32:23.186 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-09-11T00:32:23.186 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:23.186 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.186 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.187 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.187 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.187 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-09-11T00:32:23.187 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-09-11T00:32:23.187 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-09-11T00:32:23.187 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.187 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:23.187 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-09-11T00:32:23.187 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.187 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.187 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.187 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:23.187 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.188 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.188 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.188 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.188 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-09-11T00:32:23.188 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-09-11T00:32:23.188 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:23.188 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.188 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:23.188 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-09-11T00:32:23.188 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.188 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.188 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-09-11T00:32:23.188 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-09-11T00:32:23.189 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-09-11T00:32:23.189 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-09-11T00:32:23.189 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-09-11T00:32:23.189 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-09-11T00:32:23.189 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-09-11T00:32:23.189 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-09-11T00:32:23.189 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:23.189 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.189 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.189 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.189 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.189 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-09-11T00:32:23.189 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-09-11T00:32:23.190 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-09-11T00:32:23.190 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.190 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:23.190 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-09-11T00:32:23.190 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.190 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.190 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.190 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:23.190 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.190 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.190 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.190 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.190 INFO:teuthology.orchestra.run.smithi012.stdout: "smtp_destination": { 2024-09-11T00:32:23.191 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "smtp_destination", 2024-09-11T00:32:23.191 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:23.191 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.191 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:23.191 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-09-11T00:32:23.191 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.191 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.191 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.191 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Email address to send alerts to, use commas to separate multiple", 2024-09-11T00:32:23.191 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.191 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.191 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.191 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.191 INFO:teuthology.orchestra.run.smithi012.stdout: "smtp_from_name": { 2024-09-11T00:32:23.192 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "smtp_from_name", 2024-09-11T00:32:23.192 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:23.192 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.192 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:23.192 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "Ceph", 2024-09-11T00:32:23.192 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.192 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.192 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.192 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Email From: name", 2024-09-11T00:32:23.192 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.192 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.192 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.192 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.193 INFO:teuthology.orchestra.run.smithi012.stdout: "smtp_host": { 2024-09-11T00:32:23.193 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "smtp_host", 2024-09-11T00:32:23.193 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:23.193 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.193 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:23.193 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-09-11T00:32:23.193 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.193 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.193 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.193 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "SMTP server", 2024-09-11T00:32:23.193 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.193 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.193 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.194 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.194 INFO:teuthology.orchestra.run.smithi012.stdout: "smtp_password": { 2024-09-11T00:32:23.194 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "smtp_password", 2024-09-11T00:32:23.194 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:23.194 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.194 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:23.194 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-09-11T00:32:23.194 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.194 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.194 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.194 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Password to authenticate with", 2024-09-11T00:32:23.194 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.194 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.195 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.195 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.195 INFO:teuthology.orchestra.run.smithi012.stdout: "smtp_port": { 2024-09-11T00:32:23.195 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "smtp_port", 2024-09-11T00:32:23.195 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-09-11T00:32:23.195 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.195 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:23.195 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "465", 2024-09-11T00:32:23.195 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.195 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.195 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.195 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "SMTP port", 2024-09-11T00:32:23.195 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.195 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.196 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.196 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.196 INFO:teuthology.orchestra.run.smithi012.stdout: "smtp_sender": { 2024-09-11T00:32:23.196 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "smtp_sender", 2024-09-11T00:32:23.196 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:23.196 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.196 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:23.196 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-09-11T00:32:23.196 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.196 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.196 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.196 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "SMTP envelope sender", 2024-09-11T00:32:23.196 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.197 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.197 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.197 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.197 INFO:teuthology.orchestra.run.smithi012.stdout: "smtp_ssl": { 2024-09-11T00:32:23.197 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "smtp_ssl", 2024-09-11T00:32:23.197 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-09-11T00:32:23.197 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.197 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:23.197 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-09-11T00:32:23.197 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.197 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.198 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.198 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Use SSL to connect to SMTP server", 2024-09-11T00:32:23.198 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.198 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.198 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.198 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.198 INFO:teuthology.orchestra.run.smithi012.stdout: "smtp_user": { 2024-09-11T00:32:23.198 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "smtp_user", 2024-09-11T00:32:23.198 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:23.198 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.198 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:23.198 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-09-11T00:32:23.198 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.199 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.199 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.199 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "User to authenticate as", 2024-09-11T00:32:23.199 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.199 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.199 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.199 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.199 INFO:teuthology.orchestra.run.smithi012.stdout: "sqlite3_killpoint": { 2024-09-11T00:32:23.199 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "sqlite3_killpoint", 2024-09-11T00:32:23.199 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-09-11T00:32:23.199 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "dev", 2024-09-11T00:32:23.199 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:23.199 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "0", 2024-09-11T00:32:23.200 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.200 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.200 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.200 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:23.200 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.200 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.200 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.200 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:32:23.200 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:32:23.200 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.200 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-11T00:32:23.200 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "balancer", 2024-09-11T00:32:23.200 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-09-11T00:32:23.201 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-09-11T00:32:23.201 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-09-11T00:32:23.201 INFO:teuthology.orchestra.run.smithi012.stdout: "active": { 2024-09-11T00:32:23.201 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "active", 2024-09-11T00:32:23.201 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-09-11T00:32:23.201 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.201 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:23.201 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-09-11T00:32:23.201 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.201 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.201 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.201 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "automatically balance PGs across cluster", 2024-09-11T00:32:23.201 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.202 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.202 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.202 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.202 INFO:teuthology.orchestra.run.smithi012.stdout: "begin_time": { 2024-09-11T00:32:23.202 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "begin_time", 2024-09-11T00:32:23.202 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:23.202 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.202 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:23.202 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "0000", 2024-09-11T00:32:23.202 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.202 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.202 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.202 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "beginning time of day to automatically balance", 2024-09-11T00:32:23.203 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-11T00:32:23.203 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.203 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.203 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.203 INFO:teuthology.orchestra.run.smithi012.stdout: "begin_weekday": { 2024-09-11T00:32:23.203 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "begin_weekday", 2024-09-11T00:32:23.203 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "uint", 2024-09-11T00:32:23.203 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.203 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:23.203 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "0", 2024-09-11T00:32:23.203 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "0", 2024-09-11T00:32:23.203 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "6", 2024-09-11T00:32:23.203 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.204 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-09-11T00:32:23.204 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-11T00:32:23.204 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.204 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.204 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.204 INFO:teuthology.orchestra.run.smithi012.stdout: "crush_compat_max_iterations": { 2024-09-11T00:32:23.204 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "crush_compat_max_iterations", 2024-09-11T00:32:23.204 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "uint", 2024-09-11T00:32:23.204 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.204 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:23.204 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "25", 2024-09-11T00:32:23.204 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "1", 2024-09-11T00:32:23.204 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "250", 2024-09-11T00:32:23.205 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.205 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-09-11T00:32:23.205 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.205 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.205 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.205 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.205 INFO:teuthology.orchestra.run.smithi012.stdout: "crush_compat_metrics": { 2024-09-11T00:32:23.205 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "crush_compat_metrics", 2024-09-11T00:32:23.205 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:23.205 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.205 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:23.205 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "pgs,objects,bytes", 2024-09-11T00:32:23.205 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.206 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.206 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.206 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-09-11T00:32:23.206 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-09-11T00:32:23.206 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.206 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.206 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.206 INFO:teuthology.orchestra.run.smithi012.stdout: "crush_compat_step": { 2024-09-11T00:32:23.206 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "crush_compat_step", 2024-09-11T00:32:23.206 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "float", 2024-09-11T00:32:23.206 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.206 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:23.206 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "0.5", 2024-09-11T00:32:23.207 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "0.001", 2024-09-11T00:32:23.207 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "0.999", 2024-09-11T00:32:23.207 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.207 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "aggressiveness of optimization", 2024-09-11T00:32:23.207 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-09-11T00:32:23.207 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.207 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.207 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.207 INFO:teuthology.orchestra.run.smithi012.stdout: "end_time": { 2024-09-11T00:32:23.207 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "end_time", 2024-09-11T00:32:23.207 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:23.207 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.207 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:23.207 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "2359", 2024-09-11T00:32:23.208 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.208 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.208 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.208 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "ending time of day to automatically balance", 2024-09-11T00:32:23.208 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-11T00:32:23.208 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.208 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.208 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.208 INFO:teuthology.orchestra.run.smithi012.stdout: "end_weekday": { 2024-09-11T00:32:23.208 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "end_weekday", 2024-09-11T00:32:23.208 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "uint", 2024-09-11T00:32:23.209 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.209 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:23.209 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "0", 2024-09-11T00:32:23.209 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "0", 2024-09-11T00:32:23.209 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "6", 2024-09-11T00:32:23.209 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.209 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-09-11T00:32:23.209 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-11T00:32:23.209 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.209 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.209 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.209 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-09-11T00:32:23.209 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-09-11T00:32:23.210 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:23.210 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.210 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:23.210 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-09-11T00:32:23.210 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.210 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.210 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-09-11T00:32:23.210 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-09-11T00:32:23.210 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-09-11T00:32:23.210 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-09-11T00:32:23.210 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-09-11T00:32:23.210 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-09-11T00:32:23.211 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-09-11T00:32:23.211 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-09-11T00:32:23.211 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:23.211 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.211 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.211 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.211 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.211 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-09-11T00:32:23.211 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-09-11T00:32:23.211 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-09-11T00:32:23.211 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.211 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:23.211 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-09-11T00:32:23.212 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.212 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.212 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.212 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:23.212 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.212 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.212 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.212 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.212 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-09-11T00:32:23.212 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-09-11T00:32:23.212 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:23.212 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.213 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:23.213 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-09-11T00:32:23.213 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.213 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.213 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-09-11T00:32:23.213 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-09-11T00:32:23.213 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-09-11T00:32:23.213 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-09-11T00:32:23.213 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-09-11T00:32:23.213 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-09-11T00:32:23.213 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-09-11T00:32:23.213 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-09-11T00:32:23.213 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:23.214 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.214 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.214 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.214 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.214 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-09-11T00:32:23.214 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-09-11T00:32:23.214 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-09-11T00:32:23.214 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.214 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:23.214 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-09-11T00:32:23.214 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.214 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.214 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.215 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:23.215 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.215 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.215 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.215 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.215 INFO:teuthology.orchestra.run.smithi012.stdout: "min_score": { 2024-09-11T00:32:23.215 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "min_score", 2024-09-11T00:32:23.215 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "float", 2024-09-11T00:32:23.215 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.215 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:23.215 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "0", 2024-09-11T00:32:23.215 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.215 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.216 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.216 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-09-11T00:32:23.216 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.216 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.216 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.216 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.216 INFO:teuthology.orchestra.run.smithi012.stdout: "mode": { 2024-09-11T00:32:23.216 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "mode", 2024-09-11T00:32:23.216 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:23.216 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.216 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:23.216 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "upmap", 2024-09-11T00:32:23.216 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.217 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.217 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-09-11T00:32:23.217 INFO:teuthology.orchestra.run.smithi012.stdout: "crush-compat", 2024-09-11T00:32:23.217 INFO:teuthology.orchestra.run.smithi012.stdout: "none", 2024-09-11T00:32:23.217 INFO:teuthology.orchestra.run.smithi012.stdout: "read", 2024-09-11T00:32:23.217 INFO:teuthology.orchestra.run.smithi012.stdout: "upmap", 2024-09-11T00:32:23.217 INFO:teuthology.orchestra.run.smithi012.stdout: "upmap-read" 2024-09-11T00:32:23.217 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-09-11T00:32:23.217 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Balancer mode", 2024-09-11T00:32:23.217 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.217 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.217 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.217 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.218 INFO:teuthology.orchestra.run.smithi012.stdout: "pool_ids": { 2024-09-11T00:32:23.218 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "pool_ids", 2024-09-11T00:32:23.218 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:23.218 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.218 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:23.218 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-09-11T00:32:23.218 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.218 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.218 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.218 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-09-11T00:32:23.218 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.218 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.218 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.219 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.219 INFO:teuthology.orchestra.run.smithi012.stdout: "sleep_interval": { 2024-09-11T00:32:23.219 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "sleep_interval", 2024-09-11T00:32:23.219 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "secs", 2024-09-11T00:32:23.219 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.219 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:23.219 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "60", 2024-09-11T00:32:23.219 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.219 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.219 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.219 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-09-11T00:32:23.219 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.219 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.220 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.220 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.220 INFO:teuthology.orchestra.run.smithi012.stdout: "sqlite3_killpoint": { 2024-09-11T00:32:23.220 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "sqlite3_killpoint", 2024-09-11T00:32:23.220 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-09-11T00:32:23.220 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "dev", 2024-09-11T00:32:23.220 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:23.220 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "0", 2024-09-11T00:32:23.220 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.220 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.220 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.220 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:23.220 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.221 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.221 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.221 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.221 INFO:teuthology.orchestra.run.smithi012.stdout: "upmap_max_deviation": { 2024-09-11T00:32:23.221 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "upmap_max_deviation", 2024-09-11T00:32:23.221 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-09-11T00:32:23.221 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.221 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:23.221 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "5", 2024-09-11T00:32:23.221 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "1", 2024-09-11T00:32:23.221 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.221 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.221 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "deviation below which no optimization is attempted", 2024-09-11T00:32:23.222 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-09-11T00:32:23.222 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.222 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.222 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.222 INFO:teuthology.orchestra.run.smithi012.stdout: "upmap_max_optimizations": { 2024-09-11T00:32:23.222 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "upmap_max_optimizations", 2024-09-11T00:32:23.222 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "uint", 2024-09-11T00:32:23.222 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.222 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:23.222 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "10", 2024-09-11T00:32:23.222 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.222 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.222 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.223 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-09-11T00:32:23.223 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.223 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.223 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.223 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:32:23.223 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:32:23.223 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.223 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-11T00:32:23.223 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "cephadm", 2024-09-11T00:32:23.223 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-09-11T00:32:23.223 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-09-11T00:32:23.223 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-09-11T00:32:23.223 INFO:teuthology.orchestra.run.smithi012.stdout: "agent_down_multiplier": { 2024-09-11T00:32:23.223 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "agent_down_multiplier", 2024-09-11T00:32:23.224 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "float", 2024-09-11T00:32:23.224 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.224 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:23.224 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "3.0", 2024-09-11T00:32:23.224 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.224 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.224 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.224 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-09-11T00:32:23.224 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.224 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.224 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.224 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.225 INFO:teuthology.orchestra.run.smithi012.stdout: "agent_refresh_rate": { 2024-09-11T00:32:23.225 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "agent_refresh_rate", 2024-09-11T00:32:23.225 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "secs", 2024-09-11T00:32:23.225 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.225 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:23.225 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "20", 2024-09-11T00:32:23.225 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.225 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.225 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.225 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-09-11T00:32:23.225 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.225 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.226 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.226 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.226 INFO:teuthology.orchestra.run.smithi012.stdout: "agent_starting_port": { 2024-09-11T00:32:23.226 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "agent_starting_port", 2024-09-11T00:32:23.226 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-09-11T00:32:23.226 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.226 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:23.226 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "4721", 2024-09-11T00:32:23.226 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.226 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.226 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.226 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-09-11T00:32:23.227 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.227 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.227 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.227 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.227 INFO:teuthology.orchestra.run.smithi012.stdout: "allow_ptrace": { 2024-09-11T00:32:23.227 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "allow_ptrace", 2024-09-11T00:32:23.227 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-09-11T00:32:23.227 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.227 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:23.227 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-09-11T00:32:23.227 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.227 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.227 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.228 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-09-11T00:32:23.228 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-09-11T00:32:23.228 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.228 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.228 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.228 INFO:teuthology.orchestra.run.smithi012.stdout: "autotune_interval": { 2024-09-11T00:32:23.228 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "autotune_interval", 2024-09-11T00:32:23.228 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "secs", 2024-09-11T00:32:23.228 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.228 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:23.228 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "600", 2024-09-11T00:32:23.228 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.229 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.229 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.229 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "how frequently to autotune daemon memory", 2024-09-11T00:32:23.229 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.229 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.229 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.229 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.229 INFO:teuthology.orchestra.run.smithi012.stdout: "autotune_memory_target_ratio": { 2024-09-11T00:32:23.229 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "autotune_memory_target_ratio", 2024-09-11T00:32:23.229 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "float", 2024-09-11T00:32:23.229 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.229 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:23.229 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "0.7", 2024-09-11T00:32:23.230 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.230 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.230 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.230 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-09-11T00:32:23.230 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.230 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.230 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.230 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.230 INFO:teuthology.orchestra.run.smithi012.stdout: "cephadm_log_destination": { 2024-09-11T00:32:23.230 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "cephadm_log_destination", 2024-09-11T00:32:23.230 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:23.230 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.230 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:23.231 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-09-11T00:32:23.231 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.231 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.231 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-09-11T00:32:23.231 INFO:teuthology.orchestra.run.smithi012.stdout: "file", 2024-09-11T00:32:23.231 INFO:teuthology.orchestra.run.smithi012.stdout: "file,syslog", 2024-09-11T00:32:23.231 INFO:teuthology.orchestra.run.smithi012.stdout: "syslog" 2024-09-11T00:32:23.231 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-09-11T00:32:23.231 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-09-11T00:32:23.231 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.231 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.231 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.231 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.232 INFO:teuthology.orchestra.run.smithi012.stdout: "cgroups_split": { 2024-09-11T00:32:23.232 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "cgroups_split", 2024-09-11T00:32:23.232 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-09-11T00:32:23.232 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.232 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:23.232 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-09-11T00:32:23.232 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.232 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.232 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.232 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-09-11T00:32:23.232 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.232 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.232 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.233 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.233 INFO:teuthology.orchestra.run.smithi012.stdout: "config_checks_enabled": { 2024-09-11T00:32:23.233 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "config_checks_enabled", 2024-09-11T00:32:23.233 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-09-11T00:32:23.233 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.233 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:23.233 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-09-11T00:32:23.233 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.233 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.233 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.233 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-09-11T00:32:23.233 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.233 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.234 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.234 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.234 INFO:teuthology.orchestra.run.smithi012.stdout: "config_dashboard": { 2024-09-11T00:32:23.234 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "config_dashboard", 2024-09-11T00:32:23.234 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-09-11T00:32:23.234 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.234 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:23.234 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-09-11T00:32:23.234 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.234 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.234 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.234 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-09-11T00:32:23.234 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.235 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.235 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.235 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.235 INFO:teuthology.orchestra.run.smithi012.stdout: "container_image_alertmanager": { 2024-09-11T00:32:23.235 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "container_image_alertmanager", 2024-09-11T00:32:23.235 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:23.235 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.235 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:23.235 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.27.0", 2024-09-11T00:32:23.235 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.235 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.235 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.235 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Prometheus container image", 2024-09-11T00:32:23.235 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.236 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.236 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.236 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.236 INFO:teuthology.orchestra.run.smithi012.stdout: "container_image_base": { 2024-09-11T00:32:23.236 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "container_image_base", 2024-09-11T00:32:23.236 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:23.236 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.236 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:23.236 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "quay.io/ceph/ceph", 2024-09-11T00:32:23.236 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.236 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.236 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.236 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Container image name, without the tag", 2024-09-11T00:32:23.237 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.237 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.237 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.237 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.237 INFO:teuthology.orchestra.run.smithi012.stdout: "container_image_elasticsearch": { 2024-09-11T00:32:23.237 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "container_image_elasticsearch", 2024-09-11T00:32:23.237 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:23.237 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.237 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:23.237 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-09-11T00:32:23.237 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.237 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.238 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.238 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "elasticsearch container image", 2024-09-11T00:32:23.238 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.238 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.238 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.238 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.238 INFO:teuthology.orchestra.run.smithi012.stdout: "container_image_grafana": { 2024-09-11T00:32:23.238 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "container_image_grafana", 2024-09-11T00:32:23.238 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:23.238 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.238 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:23.238 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "quay.io/ceph/grafana:10.4.0", 2024-09-11T00:32:23.238 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.239 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.239 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.239 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Prometheus container image", 2024-09-11T00:32:23.239 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.239 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.239 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.239 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.239 INFO:teuthology.orchestra.run.smithi012.stdout: "container_image_haproxy": { 2024-09-11T00:32:23.239 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "container_image_haproxy", 2024-09-11T00:32:23.239 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:23.239 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.239 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:23.239 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-09-11T00:32:23.240 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.240 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.240 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.240 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "HAproxy container image", 2024-09-11T00:32:23.240 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.240 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.240 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.240 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.240 INFO:teuthology.orchestra.run.smithi012.stdout: "container_image_jaeger_agent": { 2024-09-11T00:32:23.240 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "container_image_jaeger_agent", 2024-09-11T00:32:23.240 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:23.240 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.240 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:23.241 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-09-11T00:32:23.241 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.241 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.241 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.241 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Jaeger agent container image", 2024-09-11T00:32:23.241 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.241 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.241 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.241 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.241 INFO:teuthology.orchestra.run.smithi012.stdout: "container_image_jaeger_collector": { 2024-09-11T00:32:23.241 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "container_image_jaeger_collector", 2024-09-11T00:32:23.241 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:23.241 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.242 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:23.242 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-09-11T00:32:23.242 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.242 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.242 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.242 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Jaeger collector container image", 2024-09-11T00:32:23.242 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.242 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.242 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.242 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.242 INFO:teuthology.orchestra.run.smithi012.stdout: "container_image_jaeger_query": { 2024-09-11T00:32:23.242 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "container_image_jaeger_query", 2024-09-11T00:32:23.242 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:23.242 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.243 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:23.243 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-09-11T00:32:23.243 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.243 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.243 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.243 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Jaeger query container image", 2024-09-11T00:32:23.243 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.243 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.243 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.243 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.243 INFO:teuthology.orchestra.run.smithi012.stdout: "container_image_keepalived": { 2024-09-11T00:32:23.243 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "container_image_keepalived", 2024-09-11T00:32:23.244 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:23.244 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.244 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:23.244 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-09-11T00:32:23.244 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.244 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.244 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.244 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Keepalived container image", 2024-09-11T00:32:23.244 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.244 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.244 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.244 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.244 INFO:teuthology.orchestra.run.smithi012.stdout: "container_image_loki": { 2024-09-11T00:32:23.244 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "container_image_loki", 2024-09-11T00:32:23.245 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:23.245 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.245 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:23.245 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "docker.io/grafana/loki:3.0.0", 2024-09-11T00:32:23.245 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.245 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.245 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.245 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Loki container image", 2024-09-11T00:32:23.245 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.245 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.245 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.245 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.245 INFO:teuthology.orchestra.run.smithi012.stdout: "container_image_nginx": { 2024-09-11T00:32:23.246 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "container_image_nginx", 2024-09-11T00:32:23.246 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:23.246 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.246 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:23.246 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "quay.io/ceph/nginx:1.26.1", 2024-09-11T00:32:23.246 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.246 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.246 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.246 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Nginx container image", 2024-09-11T00:32:23.246 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.246 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.246 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.246 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.246 INFO:teuthology.orchestra.run.smithi012.stdout: "container_image_node_exporter": { 2024-09-11T00:32:23.247 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "container_image_node_exporter", 2024-09-11T00:32:23.247 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:23.247 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.247 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:23.247 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.7.0", 2024-09-11T00:32:23.247 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.247 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.247 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.247 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Prometheus container image", 2024-09-11T00:32:23.247 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.247 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.247 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.247 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.248 INFO:teuthology.orchestra.run.smithi012.stdout: "container_image_nvmeof": { 2024-09-11T00:32:23.248 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "container_image_nvmeof", 2024-09-11T00:32:23.248 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:23.248 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.248 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:23.248 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.17", 2024-09-11T00:32:23.248 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.248 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.248 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.248 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Nvme-of container image", 2024-09-11T00:32:23.248 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.248 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.248 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.249 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.249 INFO:teuthology.orchestra.run.smithi012.stdout: "container_image_oauth2_proxy": { 2024-09-11T00:32:23.249 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "container_image_oauth2_proxy", 2024-09-11T00:32:23.249 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:23.249 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.249 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:23.249 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "quay.io/oauth2-proxy/oauth2-proxy:v7.6.0", 2024-09-11T00:32:23.249 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.249 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.249 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.249 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "oauth2-proxy container image", 2024-09-11T00:32:23.249 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.249 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.249 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.250 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.250 INFO:teuthology.orchestra.run.smithi012.stdout: "container_image_prometheus": { 2024-09-11T00:32:23.250 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "container_image_prometheus", 2024-09-11T00:32:23.250 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:23.250 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.250 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:23.250 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "quay.io/prometheus/prometheus:v2.51.0", 2024-09-11T00:32:23.250 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.250 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.250 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.250 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Prometheus container image", 2024-09-11T00:32:23.250 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.251 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.251 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.251 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.251 INFO:teuthology.orchestra.run.smithi012.stdout: "container_image_promtail": { 2024-09-11T00:32:23.251 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "container_image_promtail", 2024-09-11T00:32:23.251 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:23.251 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.251 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:23.251 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "docker.io/grafana/promtail:3.0.0", 2024-09-11T00:32:23.251 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.251 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.251 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.252 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Promtail container image", 2024-09-11T00:32:23.252 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.252 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.252 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.252 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.252 INFO:teuthology.orchestra.run.smithi012.stdout: "container_image_samba": { 2024-09-11T00:32:23.252 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "container_image_samba", 2024-09-11T00:32:23.252 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:23.252 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.252 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:23.252 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-09-11T00:32:23.253 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.253 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.253 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.253 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Samba/SMB container image", 2024-09-11T00:32:23.253 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.253 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.253 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.253 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.253 INFO:teuthology.orchestra.run.smithi012.stdout: "container_image_samba_metrics": { 2024-09-11T00:32:23.253 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "container_image_samba_metrics", 2024-09-11T00:32:23.253 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:23.253 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.253 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:23.253 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "quay.io/samba.org/samba-metrics:latest", 2024-09-11T00:32:23.254 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.254 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.254 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.254 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Samba/SMB metrics exporter container image", 2024-09-11T00:32:23.254 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.254 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.254 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.254 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.254 INFO:teuthology.orchestra.run.smithi012.stdout: "container_image_snmp_gateway": { 2024-09-11T00:32:23.254 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "container_image_snmp_gateway", 2024-09-11T00:32:23.254 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:23.254 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.254 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:23.255 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-09-11T00:32:23.255 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.255 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.255 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.255 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "SNMP Gateway container image", 2024-09-11T00:32:23.255 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.255 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.255 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.255 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.255 INFO:teuthology.orchestra.run.smithi012.stdout: "container_init": { 2024-09-11T00:32:23.255 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "container_init", 2024-09-11T00:32:23.255 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-09-11T00:32:23.255 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.256 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:23.256 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-09-11T00:32:23.256 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.256 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.256 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.256 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Run podman/docker with `--init`", 2024-09-11T00:32:23.256 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.256 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.256 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.256 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.256 INFO:teuthology.orchestra.run.smithi012.stdout: "daemon_cache_timeout": { 2024-09-11T00:32:23.256 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "daemon_cache_timeout", 2024-09-11T00:32:23.256 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "secs", 2024-09-11T00:32:23.257 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.257 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:23.257 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "600", 2024-09-11T00:32:23.257 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.257 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.257 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.257 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-09-11T00:32:23.257 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.257 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.257 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.257 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.257 INFO:teuthology.orchestra.run.smithi012.stdout: "default_cephadm_command_timeout": { 2024-09-11T00:32:23.257 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "default_cephadm_command_timeout", 2024-09-11T00:32:23.258 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-09-11T00:32:23.258 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.258 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:23.258 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "900", 2024-09-11T00:32:23.258 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.258 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.258 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.258 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-09-11T00:32:23.258 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.258 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.258 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.258 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.258 INFO:teuthology.orchestra.run.smithi012.stdout: "default_registry": { 2024-09-11T00:32:23.258 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "default_registry", 2024-09-11T00:32:23.259 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:23.259 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.259 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:23.259 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "docker.io", 2024-09-11T00:32:23.259 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.259 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.259 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.259 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-09-11T00:32:23.259 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.259 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.259 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.259 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.259 INFO:teuthology.orchestra.run.smithi012.stdout: "device_cache_timeout": { 2024-09-11T00:32:23.260 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "device_cache_timeout", 2024-09-11T00:32:23.260 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "secs", 2024-09-11T00:32:23.260 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.260 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:23.260 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "1800", 2024-09-11T00:32:23.260 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.260 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.260 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.260 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "seconds to cache device inventory", 2024-09-11T00:32:23.260 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.260 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.260 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.260 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.261 INFO:teuthology.orchestra.run.smithi012.stdout: "device_enhanced_scan": { 2024-09-11T00:32:23.261 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "device_enhanced_scan", 2024-09-11T00:32:23.261 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-09-11T00:32:23.261 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.261 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:23.261 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-09-11T00:32:23.261 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.261 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.261 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.261 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Use libstoragemgmt during device scans", 2024-09-11T00:32:23.261 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.261 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.261 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.264 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.264 INFO:teuthology.orchestra.run.smithi012.stdout: "facts_cache_timeout": { 2024-09-11T00:32:23.264 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "facts_cache_timeout", 2024-09-11T00:32:23.264 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "secs", 2024-09-11T00:32:23.264 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.264 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:23.264 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "60", 2024-09-11T00:32:23.264 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.265 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.265 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.265 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "seconds to cache host facts data", 2024-09-11T00:32:23.265 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.265 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.265 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.265 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.265 INFO:teuthology.orchestra.run.smithi012.stdout: "grafana_dashboards_path": { 2024-09-11T00:32:23.265 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "grafana_dashboards_path", 2024-09-11T00:32:23.265 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:23.265 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.265 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:23.265 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-09-11T00:32:23.266 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.266 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.266 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.266 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-09-11T00:32:23.266 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.266 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.266 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.266 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.266 INFO:teuthology.orchestra.run.smithi012.stdout: "host_check_interval": { 2024-09-11T00:32:23.266 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "host_check_interval", 2024-09-11T00:32:23.266 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "secs", 2024-09-11T00:32:23.266 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.267 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:23.267 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "600", 2024-09-11T00:32:23.267 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.267 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.267 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.267 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "how frequently to perform a host check", 2024-09-11T00:32:23.267 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.267 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.267 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.267 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.267 INFO:teuthology.orchestra.run.smithi012.stdout: "hw_monitoring": { 2024-09-11T00:32:23.267 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "hw_monitoring", 2024-09-11T00:32:23.267 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-09-11T00:32:23.268 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.268 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:23.268 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-09-11T00:32:23.268 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.268 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.268 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.268 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-09-11T00:32:23.268 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.268 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.268 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.268 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.268 INFO:teuthology.orchestra.run.smithi012.stdout: "inventory_list_all": { 2024-09-11T00:32:23.268 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "inventory_list_all", 2024-09-11T00:32:23.269 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-09-11T00:32:23.269 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.269 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:23.269 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-09-11T00:32:23.269 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.269 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.269 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.269 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-09-11T00:32:23.269 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.269 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.269 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.269 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.270 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-09-11T00:32:23.270 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-09-11T00:32:23.270 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:23.270 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.270 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:23.270 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-09-11T00:32:23.270 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.270 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.270 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-09-11T00:32:23.270 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-09-11T00:32:23.270 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-09-11T00:32:23.270 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-09-11T00:32:23.270 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-09-11T00:32:23.271 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-09-11T00:32:23.271 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-09-11T00:32:23.271 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-09-11T00:32:23.271 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:23.271 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.271 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.271 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.271 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.271 INFO:teuthology.orchestra.run.smithi012.stdout: "log_refresh_metadata": { 2024-09-11T00:32:23.271 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_refresh_metadata", 2024-09-11T00:32:23.271 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-09-11T00:32:23.271 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.271 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:23.271 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-09-11T00:32:23.272 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.272 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.272 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.272 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-09-11T00:32:23.272 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.272 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.272 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.272 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.272 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-09-11T00:32:23.272 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-09-11T00:32:23.272 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-09-11T00:32:23.272 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.273 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:23.273 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-09-11T00:32:23.273 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.273 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.273 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.273 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-09-11T00:32:23.273 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.273 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.273 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.273 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.273 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-09-11T00:32:23.273 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-09-11T00:32:23.273 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:23.273 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.274 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:23.274 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-09-11T00:32:23.274 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.274 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.274 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-09-11T00:32:23.274 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-09-11T00:32:23.274 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-09-11T00:32:23.274 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-09-11T00:32:23.274 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-09-11T00:32:23.274 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-09-11T00:32:23.274 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-09-11T00:32:23.274 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-09-11T00:32:23.274 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:23.275 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.275 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.275 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.275 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.275 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-09-11T00:32:23.275 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-09-11T00:32:23.275 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-09-11T00:32:23.275 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.275 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:23.275 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-09-11T00:32:23.275 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.275 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.275 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.275 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:23.276 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.276 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.276 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.276 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.276 INFO:teuthology.orchestra.run.smithi012.stdout: "manage_etc_ceph_ceph_conf": { 2024-09-11T00:32:23.276 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-09-11T00:32:23.276 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-09-11T00:32:23.276 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.276 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:23.276 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-09-11T00:32:23.276 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.276 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.276 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.277 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-09-11T00:32:23.277 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.277 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.277 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.277 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.277 INFO:teuthology.orchestra.run.smithi012.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-09-11T00:32:23.277 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-09-11T00:32:23.277 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:23.277 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.277 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:23.277 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "*", 2024-09-11T00:32:23.277 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.277 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.278 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.278 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-09-11T00:32:23.278 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.278 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.278 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.278 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.278 INFO:teuthology.orchestra.run.smithi012.stdout: "max_count_per_host": { 2024-09-11T00:32:23.278 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "max_count_per_host", 2024-09-11T00:32:23.278 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-09-11T00:32:23.278 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.278 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:23.278 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "10", 2024-09-11T00:32:23.278 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.278 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.279 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.279 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "max number of daemons per service per host", 2024-09-11T00:32:23.279 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.279 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.279 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.279 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.279 INFO:teuthology.orchestra.run.smithi012.stdout: "max_osd_draining_count": { 2024-09-11T00:32:23.279 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "max_osd_draining_count", 2024-09-11T00:32:23.279 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-09-11T00:32:23.279 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.279 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:23.280 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "10", 2024-09-11T00:32:23.280 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.280 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.280 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.280 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-09-11T00:32:23.281 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.281 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.281 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.281 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.281 INFO:teuthology.orchestra.run.smithi012.stdout: "migration_current": { 2024-09-11T00:32:23.281 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "migration_current", 2024-09-11T00:32:23.281 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-09-11T00:32:23.281 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.281 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:23.281 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-09-11T00:32:23.281 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.281 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.281 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.282 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "internal - do not modify", 2024-09-11T00:32:23.282 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.282 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.282 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.282 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.282 INFO:teuthology.orchestra.run.smithi012.stdout: "mode": { 2024-09-11T00:32:23.282 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "mode", 2024-09-11T00:32:23.282 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:23.282 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.282 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:23.282 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "root", 2024-09-11T00:32:23.282 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.282 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.283 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-09-11T00:32:23.283 INFO:teuthology.orchestra.run.smithi012.stdout: "cephadm-package", 2024-09-11T00:32:23.283 INFO:teuthology.orchestra.run.smithi012.stdout: "root" 2024-09-11T00:32:23.283 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-09-11T00:32:23.283 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "mode for remote execution of cephadm", 2024-09-11T00:32:23.283 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.283 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.283 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.283 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.283 INFO:teuthology.orchestra.run.smithi012.stdout: "oob_default_addr": { 2024-09-11T00:32:23.283 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "oob_default_addr", 2024-09-11T00:32:23.283 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:23.283 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.284 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:23.284 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "169.254.1.1", 2024-09-11T00:32:23.284 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.284 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.284 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.284 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Default address for RedFish API (oob management).", 2024-09-11T00:32:23.284 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.284 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.284 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.284 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.284 INFO:teuthology.orchestra.run.smithi012.stdout: "prometheus_alerts_path": { 2024-09-11T00:32:23.284 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "prometheus_alerts_path", 2024-09-11T00:32:23.285 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:23.285 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.285 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:23.285 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-09-11T00:32:23.285 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.285 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.285 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.285 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-09-11T00:32:23.285 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.285 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.285 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.285 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.285 INFO:teuthology.orchestra.run.smithi012.stdout: "registry_insecure": { 2024-09-11T00:32:23.286 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "registry_insecure", 2024-09-11T00:32:23.286 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-09-11T00:32:23.286 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.286 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:23.286 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-09-11T00:32:23.286 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.286 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.286 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.286 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-09-11T00:32:23.286 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.286 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.286 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.286 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.287 INFO:teuthology.orchestra.run.smithi012.stdout: "registry_password": { 2024-09-11T00:32:23.287 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "registry_password", 2024-09-11T00:32:23.287 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:23.287 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.287 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:23.287 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-09-11T00:32:23.287 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.287 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.287 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.287 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-09-11T00:32:23.287 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.287 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.287 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.288 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.288 INFO:teuthology.orchestra.run.smithi012.stdout: "registry_url": { 2024-09-11T00:32:23.288 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "registry_url", 2024-09-11T00:32:23.288 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:23.288 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.288 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:23.288 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-09-11T00:32:23.288 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.288 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.288 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.288 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-09-11T00:32:23.288 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.288 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.288 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.289 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.289 INFO:teuthology.orchestra.run.smithi012.stdout: "registry_username": { 2024-09-11T00:32:23.289 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "registry_username", 2024-09-11T00:32:23.289 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:23.289 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.289 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:23.289 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-09-11T00:32:23.289 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.289 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.289 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.289 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-09-11T00:32:23.289 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.290 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.290 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.290 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.290 INFO:teuthology.orchestra.run.smithi012.stdout: "secure_monitoring_stack": { 2024-09-11T00:32:23.290 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "secure_monitoring_stack", 2024-09-11T00:32:23.290 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-09-11T00:32:23.290 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.290 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:23.290 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-09-11T00:32:23.290 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.290 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.290 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.290 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-09-11T00:32:23.290 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.291 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.291 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.291 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.291 INFO:teuthology.orchestra.run.smithi012.stdout: "service_discovery_port": { 2024-09-11T00:32:23.291 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "service_discovery_port", 2024-09-11T00:32:23.291 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-09-11T00:32:23.291 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.291 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:23.291 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "8765", 2024-09-11T00:32:23.291 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.291 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.291 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.291 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "cephadm service discovery port", 2024-09-11T00:32:23.291 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.292 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.292 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.292 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.292 INFO:teuthology.orchestra.run.smithi012.stdout: "sqlite3_killpoint": { 2024-09-11T00:32:23.292 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "sqlite3_killpoint", 2024-09-11T00:32:23.292 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-09-11T00:32:23.292 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "dev", 2024-09-11T00:32:23.292 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:23.292 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "0", 2024-09-11T00:32:23.292 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.292 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.292 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.292 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:23.293 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.293 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.293 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.293 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.293 INFO:teuthology.orchestra.run.smithi012.stdout: "ssh_config_file": { 2024-09-11T00:32:23.293 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "ssh_config_file", 2024-09-11T00:32:23.293 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:23.293 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.293 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:23.293 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-09-11T00:32:23.293 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.293 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.294 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.294 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-09-11T00:32:23.294 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.294 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.294 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.294 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.294 INFO:teuthology.orchestra.run.smithi012.stdout: "use_agent": { 2024-09-11T00:32:23.294 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "use_agent", 2024-09-11T00:32:23.294 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-09-11T00:32:23.294 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.294 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:23.294 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-09-11T00:32:23.294 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.295 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.295 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.295 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-09-11T00:32:23.295 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.295 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.295 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.295 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.295 INFO:teuthology.orchestra.run.smithi012.stdout: "use_repo_digest": { 2024-09-11T00:32:23.295 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "use_repo_digest", 2024-09-11T00:32:23.295 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-09-11T00:32:23.295 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.295 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:23.295 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-09-11T00:32:23.295 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.296 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.296 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.296 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-09-11T00:32:23.296 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.296 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.296 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.296 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.296 INFO:teuthology.orchestra.run.smithi012.stdout: "warn_on_failed_host_check": { 2024-09-11T00:32:23.296 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "warn_on_failed_host_check", 2024-09-11T00:32:23.296 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-09-11T00:32:23.296 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.296 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:23.297 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-09-11T00:32:23.297 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.297 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.297 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.297 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "raise a health warning if the host check fails", 2024-09-11T00:32:23.297 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.297 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.297 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.297 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.297 INFO:teuthology.orchestra.run.smithi012.stdout: "warn_on_stray_daemons": { 2024-09-11T00:32:23.297 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "warn_on_stray_daemons", 2024-09-11T00:32:23.297 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-09-11T00:32:23.297 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.298 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:23.298 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-09-11T00:32:23.298 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.298 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.298 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.298 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-09-11T00:32:23.298 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.298 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.298 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.298 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.298 INFO:teuthology.orchestra.run.smithi012.stdout: "warn_on_stray_hosts": { 2024-09-11T00:32:23.298 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "warn_on_stray_hosts", 2024-09-11T00:32:23.299 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-09-11T00:32:23.299 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.299 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:23.299 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-09-11T00:32:23.299 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.299 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.299 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.299 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-09-11T00:32:23.299 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.299 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.299 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.299 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:32:23.299 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:32:23.300 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.300 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-11T00:32:23.300 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "crash", 2024-09-11T00:32:23.300 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-09-11T00:32:23.300 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-09-11T00:32:23.300 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-09-11T00:32:23.300 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-09-11T00:32:23.300 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-09-11T00:32:23.300 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:23.300 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.300 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:23.300 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-09-11T00:32:23.300 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.301 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.301 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-09-11T00:32:23.301 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-09-11T00:32:23.301 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-09-11T00:32:23.301 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-09-11T00:32:23.301 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-09-11T00:32:23.301 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-09-11T00:32:23.301 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-09-11T00:32:23.301 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-09-11T00:32:23.301 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:23.301 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.301 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.302 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.302 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.302 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-09-11T00:32:23.302 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-09-11T00:32:23.302 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-09-11T00:32:23.302 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.302 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:23.302 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-09-11T00:32:23.302 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.302 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.302 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.302 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:23.303 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.303 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.303 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.303 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.303 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-09-11T00:32:23.303 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-09-11T00:32:23.303 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:23.303 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.303 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:23.303 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-09-11T00:32:23.303 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.303 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.303 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-09-11T00:32:23.303 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-09-11T00:32:23.304 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-09-11T00:32:23.304 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-09-11T00:32:23.304 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-09-11T00:32:23.304 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-09-11T00:32:23.304 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-09-11T00:32:23.304 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-09-11T00:32:23.304 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:23.304 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.304 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.304 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.304 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.305 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-09-11T00:32:23.305 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-09-11T00:32:23.305 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-09-11T00:32:23.305 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.305 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:23.305 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-09-11T00:32:23.305 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.305 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.305 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.305 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:23.305 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.305 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.306 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.306 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.306 INFO:teuthology.orchestra.run.smithi012.stdout: "retain_interval": { 2024-09-11T00:32:23.306 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "retain_interval", 2024-09-11T00:32:23.306 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "secs", 2024-09-11T00:32:23.306 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.306 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:23.306 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "31536000", 2024-09-11T00:32:23.306 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.306 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.306 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.306 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "how long to retain crashes before pruning them", 2024-09-11T00:32:23.306 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.306 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.307 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.307 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.307 INFO:teuthology.orchestra.run.smithi012.stdout: "sqlite3_killpoint": { 2024-09-11T00:32:23.307 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "sqlite3_killpoint", 2024-09-11T00:32:23.307 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-09-11T00:32:23.307 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "dev", 2024-09-11T00:32:23.307 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:23.307 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "0", 2024-09-11T00:32:23.307 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.307 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.307 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.307 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:23.307 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.307 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.308 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.308 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.308 INFO:teuthology.orchestra.run.smithi012.stdout: "warn_recent_interval": { 2024-09-11T00:32:23.308 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "warn_recent_interval", 2024-09-11T00:32:23.308 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "secs", 2024-09-11T00:32:23.308 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.308 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:23.308 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "1209600", 2024-09-11T00:32:23.308 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.308 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.308 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.309 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "time interval in which to warn about recent crashes", 2024-09-11T00:32:23.309 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.309 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.309 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.309 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:32:23.309 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:32:23.309 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.309 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-11T00:32:23.309 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "dashboard", 2024-09-11T00:32:23.309 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-09-11T00:32:23.309 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-09-11T00:32:23.309 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-09-11T00:32:23.309 INFO:teuthology.orchestra.run.smithi012.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-09-11T00:32:23.310 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-09-11T00:32:23.310 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-09-11T00:32:23.310 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.310 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:23.310 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "10", 2024-09-11T00:32:23.310 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.310 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.310 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.310 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:23.310 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.310 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.310 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.310 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.310 INFO:teuthology.orchestra.run.smithi012.stdout: "ALERTMANAGER_API_HOST": { 2024-09-11T00:32:23.311 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "ALERTMANAGER_API_HOST", 2024-09-11T00:32:23.311 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:23.311 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.311 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:23.311 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-09-11T00:32:23.311 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.311 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.311 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.311 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:23.311 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.311 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.311 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.311 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.312 INFO:teuthology.orchestra.run.smithi012.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-09-11T00:32:23.312 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-09-11T00:32:23.312 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-09-11T00:32:23.312 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.312 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:23.312 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-09-11T00:32:23.312 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.312 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.312 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.312 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:23.312 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.312 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.312 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.313 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.313 INFO:teuthology.orchestra.run.smithi012.stdout: "AUDIT_API_ENABLED": { 2024-09-11T00:32:23.313 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "AUDIT_API_ENABLED", 2024-09-11T00:32:23.313 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-09-11T00:32:23.313 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.313 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:23.313 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-09-11T00:32:23.313 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.313 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.313 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.313 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:23.313 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.314 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.314 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.314 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.314 INFO:teuthology.orchestra.run.smithi012.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-09-11T00:32:23.314 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-09-11T00:32:23.314 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-09-11T00:32:23.314 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.314 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:23.314 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-09-11T00:32:23.314 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.314 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.314 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.314 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:23.315 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.315 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.315 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.315 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.315 INFO:teuthology.orchestra.run.smithi012.stdout: "ENABLE_BROWSABLE_API": { 2024-09-11T00:32:23.315 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "ENABLE_BROWSABLE_API", 2024-09-11T00:32:23.315 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-09-11T00:32:23.315 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.315 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:23.315 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-09-11T00:32:23.315 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.315 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.315 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.315 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:23.315 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.316 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.316 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.316 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.316 INFO:teuthology.orchestra.run.smithi012.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-09-11T00:32:23.316 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-09-11T00:32:23.316 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-09-11T00:32:23.316 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.316 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:23.316 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-09-11T00:32:23.316 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.316 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.316 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.316 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:23.317 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.317 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.317 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.317 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.317 INFO:teuthology.orchestra.run.smithi012.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-09-11T00:32:23.317 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-09-11T00:32:23.317 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-09-11T00:32:23.317 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.317 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:23.317 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-09-11T00:32:23.317 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.317 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.317 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.318 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:23.318 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.318 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.318 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.318 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.318 INFO:teuthology.orchestra.run.smithi012.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-09-11T00:32:23.318 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-09-11T00:32:23.318 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-09-11T00:32:23.318 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.318 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:23.318 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-09-11T00:32:23.318 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.318 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.319 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.319 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:23.319 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.319 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.319 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.319 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.319 INFO:teuthology.orchestra.run.smithi012.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-09-11T00:32:23.319 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-09-11T00:32:23.319 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-09-11T00:32:23.319 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.319 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:23.319 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-09-11T00:32:23.319 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.320 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.320 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.320 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:23.320 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.320 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.320 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.320 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.320 INFO:teuthology.orchestra.run.smithi012.stdout: "FEATURE_TOGGLE_NFS": { 2024-09-11T00:32:23.320 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-09-11T00:32:23.320 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-09-11T00:32:23.320 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.320 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:23.320 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-09-11T00:32:23.321 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.321 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.321 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.321 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:23.321 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.321 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.321 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.321 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.321 INFO:teuthology.orchestra.run.smithi012.stdout: "FEATURE_TOGGLE_RBD": { 2024-09-11T00:32:23.321 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-09-11T00:32:23.321 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-09-11T00:32:23.321 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.321 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:23.322 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-09-11T00:32:23.322 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.322 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.322 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.322 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:23.322 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.322 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.322 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.322 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.322 INFO:teuthology.orchestra.run.smithi012.stdout: "FEATURE_TOGGLE_RGW": { 2024-09-11T00:32:23.322 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-09-11T00:32:23.322 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-09-11T00:32:23.323 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.323 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:23.323 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-09-11T00:32:23.323 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.323 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.323 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.323 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:23.323 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.323 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.323 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.323 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.323 INFO:teuthology.orchestra.run.smithi012.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-09-11T00:32:23.323 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-09-11T00:32:23.324 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:23.324 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.324 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:23.324 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-09-11T00:32:23.324 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.324 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.324 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.324 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:23.324 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.324 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.324 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.324 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.325 INFO:teuthology.orchestra.run.smithi012.stdout: "GRAFANA_API_PASSWORD": { 2024-09-11T00:32:23.325 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "GRAFANA_API_PASSWORD", 2024-09-11T00:32:23.325 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:23.325 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.325 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:23.325 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "admin", 2024-09-11T00:32:23.325 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.325 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.325 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.325 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:23.325 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.325 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.325 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.325 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.326 INFO:teuthology.orchestra.run.smithi012.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-09-11T00:32:23.326 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-09-11T00:32:23.326 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-09-11T00:32:23.326 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.326 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:23.326 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-09-11T00:32:23.326 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.326 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.326 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.326 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:23.326 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.326 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.326 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.327 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.327 INFO:teuthology.orchestra.run.smithi012.stdout: "GRAFANA_API_URL": { 2024-09-11T00:32:23.327 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "GRAFANA_API_URL", 2024-09-11T00:32:23.327 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:23.327 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.327 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:23.327 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-09-11T00:32:23.327 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.327 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.327 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.327 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:23.327 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.328 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.328 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.328 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.328 INFO:teuthology.orchestra.run.smithi012.stdout: "GRAFANA_API_USERNAME": { 2024-09-11T00:32:23.328 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "GRAFANA_API_USERNAME", 2024-09-11T00:32:23.328 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:23.328 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.328 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:23.328 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "admin", 2024-09-11T00:32:23.328 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.328 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.328 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.328 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:23.328 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.329 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.329 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.329 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.329 INFO:teuthology.orchestra.run.smithi012.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-09-11T00:32:23.329 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-09-11T00:32:23.329 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:23.329 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.329 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:23.329 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-09-11T00:32:23.329 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.329 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.329 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.329 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:23.330 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.330 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.330 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.330 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.330 INFO:teuthology.orchestra.run.smithi012.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-09-11T00:32:23.330 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-09-11T00:32:23.330 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-09-11T00:32:23.330 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.330 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:23.330 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-09-11T00:32:23.330 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.330 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.330 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.331 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:23.331 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.331 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.331 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.331 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.331 INFO:teuthology.orchestra.run.smithi012.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-09-11T00:32:23.331 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-09-11T00:32:23.331 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-09-11T00:32:23.331 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.331 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:23.331 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-09-11T00:32:23.331 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.331 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.332 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.332 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:23.332 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.332 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.332 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.332 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.332 INFO:teuthology.orchestra.run.smithi012.stdout: "ISSUE_TRACKER_API_KEY": { 2024-09-11T00:32:23.332 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-09-11T00:32:23.332 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:23.332 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.332 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:23.332 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-09-11T00:32:23.332 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.333 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.333 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.333 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:23.333 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.333 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.333 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.333 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.333 INFO:teuthology.orchestra.run.smithi012.stdout: "MANAGED_BY_CLUSTERS": { 2024-09-11T00:32:23.333 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-09-11T00:32:23.333 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:23.333 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.333 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:23.333 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "[]", 2024-09-11T00:32:23.333 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.334 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.334 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.334 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:23.334 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.334 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.334 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.334 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.334 INFO:teuthology.orchestra.run.smithi012.stdout: "MULTICLUSTER_CONFIG": { 2024-09-11T00:32:23.334 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "MULTICLUSTER_CONFIG", 2024-09-11T00:32:23.334 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:23.334 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.334 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:23.334 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "{}", 2024-09-11T00:32:23.335 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.335 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.335 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.335 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:23.335 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.335 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.335 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.335 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.335 INFO:teuthology.orchestra.run.smithi012.stdout: "PROMETHEUS_API_HOST": { 2024-09-11T00:32:23.335 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "PROMETHEUS_API_HOST", 2024-09-11T00:32:23.335 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:23.335 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.335 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:23.336 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-09-11T00:32:23.336 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.336 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.336 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.336 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:23.336 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.336 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.336 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.336 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.336 INFO:teuthology.orchestra.run.smithi012.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-09-11T00:32:23.336 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-09-11T00:32:23.337 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-09-11T00:32:23.337 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.337 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:23.337 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-09-11T00:32:23.337 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.337 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.337 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.337 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:23.337 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.337 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.337 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.337 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.337 INFO:teuthology.orchestra.run.smithi012.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-09-11T00:32:23.338 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-09-11T00:32:23.338 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-09-11T00:32:23.338 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.338 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:23.338 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-09-11T00:32:23.338 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.338 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.338 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.338 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:23.338 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.338 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.338 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.338 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.338 INFO:teuthology.orchestra.run.smithi012.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-09-11T00:32:23.339 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-09-11T00:32:23.339 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-09-11T00:32:23.339 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.339 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:23.339 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-09-11T00:32:23.339 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.339 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.339 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.339 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:23.339 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.339 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.339 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.339 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.340 INFO:teuthology.orchestra.run.smithi012.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-09-11T00:32:23.340 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-09-11T00:32:23.340 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-09-11T00:32:23.340 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.340 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:23.340 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-09-11T00:32:23.340 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.340 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.340 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.340 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:23.340 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.340 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.340 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.341 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.341 INFO:teuthology.orchestra.run.smithi012.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-09-11T00:32:23.341 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-09-11T00:32:23.341 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-09-11T00:32:23.341 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.341 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:23.341 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-09-11T00:32:23.341 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.341 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.341 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.341 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:23.341 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.342 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.342 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.342 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.342 INFO:teuthology.orchestra.run.smithi012.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-09-11T00:32:23.342 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-09-11T00:32:23.342 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-09-11T00:32:23.342 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.342 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:23.342 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-09-11T00:32:23.342 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.342 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.342 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.342 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:23.343 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.343 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.343 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.343 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.343 INFO:teuthology.orchestra.run.smithi012.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-09-11T00:32:23.343 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-09-11T00:32:23.343 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-09-11T00:32:23.343 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.343 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:23.343 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-09-11T00:32:23.343 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.343 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.343 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.343 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:23.344 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.344 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.344 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.344 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.344 INFO:teuthology.orchestra.run.smithi012.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-09-11T00:32:23.344 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-09-11T00:32:23.344 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-09-11T00:32:23.344 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.344 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:23.344 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-09-11T00:32:23.344 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.344 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.344 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.345 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:23.345 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.345 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.345 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.345 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.345 INFO:teuthology.orchestra.run.smithi012.stdout: "PWD_POLICY_ENABLED": { 2024-09-11T00:32:23.345 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "PWD_POLICY_ENABLED", 2024-09-11T00:32:23.345 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-09-11T00:32:23.345 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.345 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:23.345 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-09-11T00:32:23.345 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.345 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.345 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.346 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:23.346 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.346 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.346 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.346 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.346 INFO:teuthology.orchestra.run.smithi012.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-09-11T00:32:23.346 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-09-11T00:32:23.346 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:23.346 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.346 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:23.346 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-09-11T00:32:23.346 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.346 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.347 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.347 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:23.347 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.347 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.347 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.347 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.347 INFO:teuthology.orchestra.run.smithi012.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-09-11T00:32:23.347 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-09-11T00:32:23.347 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-09-11T00:32:23.347 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.347 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:23.347 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "10", 2024-09-11T00:32:23.347 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.347 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.348 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.348 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:23.348 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.348 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.348 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.348 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.348 INFO:teuthology.orchestra.run.smithi012.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-09-11T00:32:23.348 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-09-11T00:32:23.348 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-09-11T00:32:23.348 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.348 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:23.348 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "8", 2024-09-11T00:32:23.349 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.349 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.349 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.349 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:23.349 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.349 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.349 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.349 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.349 INFO:teuthology.orchestra.run.smithi012.stdout: "REST_REQUESTS_TIMEOUT": { 2024-09-11T00:32:23.349 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-09-11T00:32:23.349 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-09-11T00:32:23.349 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.349 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:23.349 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "45", 2024-09-11T00:32:23.350 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.350 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.350 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.350 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:23.350 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.350 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.350 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.350 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.350 INFO:teuthology.orchestra.run.smithi012.stdout: "RGW_API_ACCESS_KEY": { 2024-09-11T00:32:23.350 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "RGW_API_ACCESS_KEY", 2024-09-11T00:32:23.350 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:23.350 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.351 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:23.351 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-09-11T00:32:23.351 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.351 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.351 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.351 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:23.351 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.351 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.351 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.351 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.351 INFO:teuthology.orchestra.run.smithi012.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-09-11T00:32:23.351 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-09-11T00:32:23.351 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:23.352 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.352 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:23.352 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "admin", 2024-09-11T00:32:23.352 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.352 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.352 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.352 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:23.352 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.352 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.352 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.352 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.352 INFO:teuthology.orchestra.run.smithi012.stdout: "RGW_API_SECRET_KEY": { 2024-09-11T00:32:23.353 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "RGW_API_SECRET_KEY", 2024-09-11T00:32:23.353 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:23.353 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.353 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:23.353 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-09-11T00:32:23.353 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.353 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.353 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.353 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:23.353 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.353 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.353 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.353 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.354 INFO:teuthology.orchestra.run.smithi012.stdout: "RGW_API_SSL_VERIFY": { 2024-09-11T00:32:23.354 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "RGW_API_SSL_VERIFY", 2024-09-11T00:32:23.354 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-09-11T00:32:23.354 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.354 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:23.354 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-09-11T00:32:23.354 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.354 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.354 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.354 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:23.354 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.354 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.354 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.355 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.355 INFO:teuthology.orchestra.run.smithi012.stdout: "UNSAFE_TLS_v1_2": { 2024-09-11T00:32:23.355 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "UNSAFE_TLS_v1_2", 2024-09-11T00:32:23.355 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-09-11T00:32:23.355 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.355 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:23.355 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-09-11T00:32:23.355 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.355 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.355 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.355 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:23.355 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.355 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.355 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.356 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.356 INFO:teuthology.orchestra.run.smithi012.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-09-11T00:32:23.356 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-09-11T00:32:23.356 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-09-11T00:32:23.356 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.356 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:23.356 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "0", 2024-09-11T00:32:23.356 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.356 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.356 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.356 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:23.356 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.356 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.357 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.357 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.357 INFO:teuthology.orchestra.run.smithi012.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-09-11T00:32:23.357 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-09-11T00:32:23.357 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-09-11T00:32:23.357 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.357 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:23.357 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "10", 2024-09-11T00:32:23.357 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.357 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.357 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.357 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:23.357 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.357 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.358 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.358 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.358 INFO:teuthology.orchestra.run.smithi012.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-09-11T00:32:23.358 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-09-11T00:32:23.358 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-09-11T00:32:23.358 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.358 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:23.358 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "5", 2024-09-11T00:32:23.358 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.358 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.358 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.358 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:23.359 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.359 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.359 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.359 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.359 INFO:teuthology.orchestra.run.smithi012.stdout: "cross_origin_url": { 2024-09-11T00:32:23.359 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "cross_origin_url", 2024-09-11T00:32:23.359 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:23.359 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.359 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:23.359 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-09-11T00:32:23.359 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.359 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.359 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.359 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:23.360 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.360 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.360 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.360 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.360 INFO:teuthology.orchestra.run.smithi012.stdout: "crt_file": { 2024-09-11T00:32:23.360 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "crt_file", 2024-09-11T00:32:23.360 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:23.360 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.360 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:23.360 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-09-11T00:32:23.360 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.361 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.361 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.361 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:23.361 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.361 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.361 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.361 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.361 INFO:teuthology.orchestra.run.smithi012.stdout: "debug": { 2024-09-11T00:32:23.361 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "debug", 2024-09-11T00:32:23.361 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-09-11T00:32:23.361 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.361 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:23.361 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-09-11T00:32:23.361 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.362 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.362 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.362 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Enable/disable debug options", 2024-09-11T00:32:23.362 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.362 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.362 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.362 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.362 INFO:teuthology.orchestra.run.smithi012.stdout: "jwt_token_ttl": { 2024-09-11T00:32:23.362 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "jwt_token_ttl", 2024-09-11T00:32:23.362 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-09-11T00:32:23.362 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.362 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:23.362 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "28800", 2024-09-11T00:32:23.363 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.363 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.363 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.363 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:23.363 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.363 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.363 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.363 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.363 INFO:teuthology.orchestra.run.smithi012.stdout: "key_file": { 2024-09-11T00:32:23.363 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "key_file", 2024-09-11T00:32:23.363 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:23.363 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.364 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:23.364 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-09-11T00:32:23.364 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.364 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.364 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.364 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:23.364 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.364 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.364 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.364 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.364 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-09-11T00:32:23.364 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-09-11T00:32:23.364 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:23.365 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.365 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:23.365 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-09-11T00:32:23.365 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.365 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.365 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-09-11T00:32:23.365 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-09-11T00:32:23.365 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-09-11T00:32:23.365 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-09-11T00:32:23.365 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-09-11T00:32:23.365 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-09-11T00:32:23.365 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-09-11T00:32:23.365 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-09-11T00:32:23.366 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:23.366 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.366 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.366 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.366 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.366 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-09-11T00:32:23.366 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-09-11T00:32:23.366 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-09-11T00:32:23.366 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.366 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:23.366 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-09-11T00:32:23.366 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.366 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.367 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.367 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:23.367 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.367 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.367 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.367 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.367 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-09-11T00:32:23.367 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-09-11T00:32:23.367 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:23.367 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.367 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:23.367 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-09-11T00:32:23.367 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.368 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.368 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-09-11T00:32:23.368 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-09-11T00:32:23.368 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-09-11T00:32:23.368 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-09-11T00:32:23.368 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-09-11T00:32:23.368 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-09-11T00:32:23.368 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-09-11T00:32:23.368 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-09-11T00:32:23.368 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:23.368 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.368 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.368 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.369 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.369 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-09-11T00:32:23.369 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-09-11T00:32:23.369 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-09-11T00:32:23.369 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.369 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:23.369 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-09-11T00:32:23.369 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.369 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.369 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.369 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:23.369 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.369 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.369 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.370 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.370 INFO:teuthology.orchestra.run.smithi012.stdout: "motd": { 2024-09-11T00:32:23.370 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "motd", 2024-09-11T00:32:23.370 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:23.370 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.370 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:23.370 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-09-11T00:32:23.370 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.370 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.370 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.370 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "The message of the day", 2024-09-11T00:32:23.370 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.370 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.371 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.371 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.371 INFO:teuthology.orchestra.run.smithi012.stdout: "redirect_resolve_ip_addr": { 2024-09-11T00:32:23.371 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "redirect_resolve_ip_addr", 2024-09-11T00:32:23.371 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-09-11T00:32:23.371 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.371 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:23.371 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-09-11T00:32:23.371 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.371 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.371 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.371 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:23.371 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.372 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.372 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.372 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.372 INFO:teuthology.orchestra.run.smithi012.stdout: "server_addr": { 2024-09-11T00:32:23.372 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "server_addr", 2024-09-11T00:32:23.372 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:23.372 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.372 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:23.372 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "::", 2024-09-11T00:32:23.372 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.372 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.372 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.372 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:23.373 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.373 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.373 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.373 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.373 INFO:teuthology.orchestra.run.smithi012.stdout: "server_port": { 2024-09-11T00:32:23.373 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "server_port", 2024-09-11T00:32:23.373 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-09-11T00:32:23.373 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.373 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:23.373 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "8080", 2024-09-11T00:32:23.373 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.373 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.373 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.373 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:23.374 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.374 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.374 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.374 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.374 INFO:teuthology.orchestra.run.smithi012.stdout: "sqlite3_killpoint": { 2024-09-11T00:32:23.374 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "sqlite3_killpoint", 2024-09-11T00:32:23.374 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-09-11T00:32:23.374 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "dev", 2024-09-11T00:32:23.374 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:23.374 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "0", 2024-09-11T00:32:23.374 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.374 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.374 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.374 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:23.375 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.375 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.375 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.375 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.375 INFO:teuthology.orchestra.run.smithi012.stdout: "ssl": { 2024-09-11T00:32:23.375 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "ssl", 2024-09-11T00:32:23.375 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-09-11T00:32:23.375 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.375 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:23.375 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-09-11T00:32:23.375 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.375 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.375 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.376 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:23.376 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.376 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.376 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.376 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.376 INFO:teuthology.orchestra.run.smithi012.stdout: "ssl_server_port": { 2024-09-11T00:32:23.376 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "ssl_server_port", 2024-09-11T00:32:23.376 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-09-11T00:32:23.376 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.376 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:23.376 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "8443", 2024-09-11T00:32:23.376 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.376 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.377 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.377 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:23.377 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.377 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.377 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.377 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.377 INFO:teuthology.orchestra.run.smithi012.stdout: "standby_behaviour": { 2024-09-11T00:32:23.377 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "standby_behaviour", 2024-09-11T00:32:23.377 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:23.377 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.377 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:23.377 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "redirect", 2024-09-11T00:32:23.377 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.378 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.378 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-09-11T00:32:23.378 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-09-11T00:32:23.378 INFO:teuthology.orchestra.run.smithi012.stdout: "redirect" 2024-09-11T00:32:23.378 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-09-11T00:32:23.378 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:23.378 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.378 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.378 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.378 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.378 INFO:teuthology.orchestra.run.smithi012.stdout: "standby_error_status_code": { 2024-09-11T00:32:23.378 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "standby_error_status_code", 2024-09-11T00:32:23.378 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-09-11T00:32:23.378 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.379 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:23.379 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "500", 2024-09-11T00:32:23.379 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "400", 2024-09-11T00:32:23.379 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "599", 2024-09-11T00:32:23.379 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.379 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:23.379 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.379 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.379 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.379 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.379 INFO:teuthology.orchestra.run.smithi012.stdout: "url_prefix": { 2024-09-11T00:32:23.379 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "url_prefix", 2024-09-11T00:32:23.380 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:23.380 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.380 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:23.380 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-09-11T00:32:23.380 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.380 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.381 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.381 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:23.381 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.381 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.381 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.381 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:32:23.381 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:32:23.381 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.381 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-11T00:32:23.381 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "devicehealth", 2024-09-11T00:32:23.381 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-09-11T00:32:23.381 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-09-11T00:32:23.381 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-09-11T00:32:23.382 INFO:teuthology.orchestra.run.smithi012.stdout: "enable_monitoring": { 2024-09-11T00:32:23.382 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "enable_monitoring", 2024-09-11T00:32:23.382 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-09-11T00:32:23.382 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.382 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:23.382 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-09-11T00:32:23.382 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.382 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.382 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.382 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "monitor device health metrics", 2024-09-11T00:32:23.382 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.382 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.383 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.383 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.383 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-09-11T00:32:23.383 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-09-11T00:32:23.383 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:23.383 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.383 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:23.383 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-09-11T00:32:23.383 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.383 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.383 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-09-11T00:32:23.383 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-09-11T00:32:23.383 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-09-11T00:32:23.383 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-09-11T00:32:23.384 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-09-11T00:32:23.384 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-09-11T00:32:23.384 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-09-11T00:32:23.384 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-09-11T00:32:23.384 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:23.384 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.384 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.384 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.384 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.385 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-09-11T00:32:23.385 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-09-11T00:32:23.385 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-09-11T00:32:23.385 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.385 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:23.385 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-09-11T00:32:23.385 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.385 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.385 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.385 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:23.385 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.385 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.386 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.386 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.386 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-09-11T00:32:23.386 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-09-11T00:32:23.386 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:23.386 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.386 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:23.386 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-09-11T00:32:23.386 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.386 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.386 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-09-11T00:32:23.386 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-09-11T00:32:23.386 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-09-11T00:32:23.387 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-09-11T00:32:23.387 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-09-11T00:32:23.387 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-09-11T00:32:23.387 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-09-11T00:32:23.387 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-09-11T00:32:23.387 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:23.387 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.387 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.387 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.387 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.387 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-09-11T00:32:23.387 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-09-11T00:32:23.387 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-09-11T00:32:23.388 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.388 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:23.388 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-09-11T00:32:23.388 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.388 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.388 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.388 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:23.388 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.388 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.388 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.388 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.388 INFO:teuthology.orchestra.run.smithi012.stdout: "mark_out_threshold": { 2024-09-11T00:32:23.388 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "mark_out_threshold", 2024-09-11T00:32:23.389 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "secs", 2024-09-11T00:32:23.389 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.389 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:23.389 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "2419200", 2024-09-11T00:32:23.389 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.389 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.389 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.389 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-09-11T00:32:23.389 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.389 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.389 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.389 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.389 INFO:teuthology.orchestra.run.smithi012.stdout: "pool_name": { 2024-09-11T00:32:23.390 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "pool_name", 2024-09-11T00:32:23.390 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:23.390 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.390 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:23.390 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "device_health_metrics", 2024-09-11T00:32:23.390 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.390 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.390 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.390 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "name of pool in which to store device health metrics", 2024-09-11T00:32:23.390 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.390 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.390 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.390 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.391 INFO:teuthology.orchestra.run.smithi012.stdout: "retention_period": { 2024-09-11T00:32:23.391 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "retention_period", 2024-09-11T00:32:23.391 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "secs", 2024-09-11T00:32:23.391 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.391 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:23.391 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "15552000", 2024-09-11T00:32:23.391 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.391 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.391 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.391 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "how long to retain device health metrics", 2024-09-11T00:32:23.391 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.391 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.391 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.391 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.392 INFO:teuthology.orchestra.run.smithi012.stdout: "scrape_frequency": { 2024-09-11T00:32:23.392 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "scrape_frequency", 2024-09-11T00:32:23.392 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "secs", 2024-09-11T00:32:23.392 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.392 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:23.392 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "86400", 2024-09-11T00:32:23.392 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.392 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.392 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.392 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "how frequently to scrape device health metrics", 2024-09-11T00:32:23.392 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.392 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.392 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.393 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.393 INFO:teuthology.orchestra.run.smithi012.stdout: "self_heal": { 2024-09-11T00:32:23.393 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "self_heal", 2024-09-11T00:32:23.393 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-09-11T00:32:23.393 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.393 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:23.393 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-09-11T00:32:23.393 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.393 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.393 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.393 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-09-11T00:32:23.393 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.393 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.393 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.394 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.394 INFO:teuthology.orchestra.run.smithi012.stdout: "sleep_interval": { 2024-09-11T00:32:23.394 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "sleep_interval", 2024-09-11T00:32:23.394 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "secs", 2024-09-11T00:32:23.394 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.394 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:23.394 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "600", 2024-09-11T00:32:23.394 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.394 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.394 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.394 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "how frequently to wake up and check device health", 2024-09-11T00:32:23.395 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.395 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.395 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.395 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.395 INFO:teuthology.orchestra.run.smithi012.stdout: "sqlite3_killpoint": { 2024-09-11T00:32:23.395 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "sqlite3_killpoint", 2024-09-11T00:32:23.395 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-09-11T00:32:23.395 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "dev", 2024-09-11T00:32:23.395 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:23.395 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "0", 2024-09-11T00:32:23.395 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.395 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.395 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.395 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:23.396 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.396 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.396 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.396 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.396 INFO:teuthology.orchestra.run.smithi012.stdout: "warn_threshold": { 2024-09-11T00:32:23.396 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "warn_threshold", 2024-09-11T00:32:23.396 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "secs", 2024-09-11T00:32:23.396 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.396 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:23.396 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "7257600", 2024-09-11T00:32:23.396 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.396 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.397 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.397 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-09-11T00:32:23.397 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.397 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.397 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.397 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:32:23.397 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:32:23.397 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.397 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-11T00:32:23.397 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "diskprediction_local", 2024-09-11T00:32:23.397 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-09-11T00:32:23.397 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-09-11T00:32:23.397 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-09-11T00:32:23.397 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-09-11T00:32:23.398 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-09-11T00:32:23.398 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:23.398 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.398 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:23.398 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-09-11T00:32:23.398 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.398 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.398 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-09-11T00:32:23.398 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-09-11T00:32:23.398 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-09-11T00:32:23.398 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-09-11T00:32:23.398 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-09-11T00:32:23.398 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-09-11T00:32:23.399 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-09-11T00:32:23.399 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-09-11T00:32:23.399 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:23.399 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.399 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.399 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.399 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.399 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-09-11T00:32:23.399 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-09-11T00:32:23.399 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-09-11T00:32:23.399 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.399 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:23.399 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-09-11T00:32:23.400 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.400 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.400 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.400 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:23.400 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.400 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.400 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.400 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.400 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-09-11T00:32:23.400 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-09-11T00:32:23.400 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:23.400 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.401 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:23.401 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-09-11T00:32:23.401 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.401 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.401 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-09-11T00:32:23.401 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-09-11T00:32:23.401 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-09-11T00:32:23.401 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-09-11T00:32:23.401 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-09-11T00:32:23.401 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-09-11T00:32:23.401 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-09-11T00:32:23.401 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-09-11T00:32:23.401 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:23.402 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.402 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.402 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.402 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.402 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-09-11T00:32:23.402 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-09-11T00:32:23.402 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-09-11T00:32:23.402 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.402 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:23.402 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-09-11T00:32:23.402 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.402 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.402 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.403 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:23.403 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.403 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.403 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.403 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.403 INFO:teuthology.orchestra.run.smithi012.stdout: "predict_interval": { 2024-09-11T00:32:23.403 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "predict_interval", 2024-09-11T00:32:23.403 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:23.403 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.403 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:23.403 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "86400", 2024-09-11T00:32:23.403 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.403 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.403 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.404 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:23.404 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.404 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.404 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.404 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.404 INFO:teuthology.orchestra.run.smithi012.stdout: "predictor_model": { 2024-09-11T00:32:23.404 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "predictor_model", 2024-09-11T00:32:23.404 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:23.404 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.404 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:23.404 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "prophetstor", 2024-09-11T00:32:23.404 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.405 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.405 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.405 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:23.405 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.405 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.405 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.405 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.405 INFO:teuthology.orchestra.run.smithi012.stdout: "sleep_interval": { 2024-09-11T00:32:23.405 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "sleep_interval", 2024-09-11T00:32:23.405 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:23.405 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.405 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:23.405 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "600", 2024-09-11T00:32:23.406 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.406 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.406 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.406 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:23.406 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.406 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.406 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.406 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.406 INFO:teuthology.orchestra.run.smithi012.stdout: "sqlite3_killpoint": { 2024-09-11T00:32:23.406 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "sqlite3_killpoint", 2024-09-11T00:32:23.406 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-09-11T00:32:23.406 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "dev", 2024-09-11T00:32:23.406 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:23.406 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "0", 2024-09-11T00:32:23.407 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.407 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.407 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.407 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:23.407 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.407 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.407 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.407 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:32:23.407 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:32:23.407 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.407 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-11T00:32:23.407 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "influx", 2024-09-11T00:32:23.407 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": false, 2024-09-11T00:32:23.408 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "influxdb python module not found", 2024-09-11T00:32:23.408 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-09-11T00:32:23.408 INFO:teuthology.orchestra.run.smithi012.stdout: "batch_size": { 2024-09-11T00:32:23.408 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "batch_size", 2024-09-11T00:32:23.408 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-09-11T00:32:23.408 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.408 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:23.408 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "5000", 2024-09-11T00:32:23.408 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.408 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.408 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.408 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-09-11T00:32:23.408 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.409 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.409 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.409 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.409 INFO:teuthology.orchestra.run.smithi012.stdout: "database": { 2024-09-11T00:32:23.409 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "database", 2024-09-11T00:32:23.409 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:23.409 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.409 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:23.409 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "ceph", 2024-09-11T00:32:23.409 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.409 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.409 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.410 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-09-11T00:32:23.410 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.410 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.410 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.410 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.410 INFO:teuthology.orchestra.run.smithi012.stdout: "hostname": { 2024-09-11T00:32:23.410 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "hostname", 2024-09-11T00:32:23.410 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:23.410 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.410 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:23.410 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-09-11T00:32:23.410 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.410 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.411 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.411 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "InfluxDB server hostname", 2024-09-11T00:32:23.411 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.411 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.411 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.411 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.411 INFO:teuthology.orchestra.run.smithi012.stdout: "interval": { 2024-09-11T00:32:23.411 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "interval", 2024-09-11T00:32:23.411 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "secs", 2024-09-11T00:32:23.411 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.411 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:23.411 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "30", 2024-09-11T00:32:23.411 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "5", 2024-09-11T00:32:23.411 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.412 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.412 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-09-11T00:32:23.412 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.412 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.412 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.412 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.412 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-09-11T00:32:23.412 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-09-11T00:32:23.412 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:23.412 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.412 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:23.412 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-09-11T00:32:23.413 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.413 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.413 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-09-11T00:32:23.413 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-09-11T00:32:23.413 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-09-11T00:32:23.413 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-09-11T00:32:23.413 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-09-11T00:32:23.413 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-09-11T00:32:23.413 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-09-11T00:32:23.413 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-09-11T00:32:23.413 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:23.413 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.413 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.414 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.414 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.414 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-09-11T00:32:23.414 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-09-11T00:32:23.414 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-09-11T00:32:23.414 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.414 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:23.414 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-09-11T00:32:23.414 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.414 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.414 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.415 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:23.415 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.415 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.415 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.415 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.415 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-09-11T00:32:23.415 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-09-11T00:32:23.415 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:23.415 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.415 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:23.415 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-09-11T00:32:23.415 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.416 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.416 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-09-11T00:32:23.416 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-09-11T00:32:23.416 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-09-11T00:32:23.416 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-09-11T00:32:23.416 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-09-11T00:32:23.416 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-09-11T00:32:23.416 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-09-11T00:32:23.416 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-09-11T00:32:23.416 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:23.416 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.416 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.417 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.417 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.417 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-09-11T00:32:23.417 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-09-11T00:32:23.417 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-09-11T00:32:23.417 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.417 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:23.417 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-09-11T00:32:23.417 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.417 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.417 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.417 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:23.417 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.417 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.418 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.418 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.418 INFO:teuthology.orchestra.run.smithi012.stdout: "password": { 2024-09-11T00:32:23.418 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "password", 2024-09-11T00:32:23.418 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:23.418 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.418 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:23.418 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-09-11T00:32:23.418 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.418 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.418 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.418 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "password of InfluxDB server user", 2024-09-11T00:32:23.419 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.419 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.419 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.419 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.419 INFO:teuthology.orchestra.run.smithi012.stdout: "port": { 2024-09-11T00:32:23.419 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "port", 2024-09-11T00:32:23.419 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-09-11T00:32:23.419 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.419 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:23.419 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "8086", 2024-09-11T00:32:23.419 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.419 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.419 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.420 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "InfluxDB server port", 2024-09-11T00:32:23.420 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.420 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.420 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.420 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.420 INFO:teuthology.orchestra.run.smithi012.stdout: "sqlite3_killpoint": { 2024-09-11T00:32:23.420 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "sqlite3_killpoint", 2024-09-11T00:32:23.420 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-09-11T00:32:23.420 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "dev", 2024-09-11T00:32:23.420 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:23.420 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "0", 2024-09-11T00:32:23.420 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.421 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.421 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.421 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:23.421 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.421 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.421 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.421 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.421 INFO:teuthology.orchestra.run.smithi012.stdout: "ssl": { 2024-09-11T00:32:23.421 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "ssl", 2024-09-11T00:32:23.421 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:23.421 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.421 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:23.422 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "false", 2024-09-11T00:32:23.422 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.422 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.422 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.422 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-09-11T00:32:23.422 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.422 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.422 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.422 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.422 INFO:teuthology.orchestra.run.smithi012.stdout: "threads": { 2024-09-11T00:32:23.422 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "threads", 2024-09-11T00:32:23.422 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-09-11T00:32:23.422 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.422 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:23.423 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "5", 2024-09-11T00:32:23.423 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "1", 2024-09-11T00:32:23.423 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "32", 2024-09-11T00:32:23.423 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.423 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-09-11T00:32:23.423 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.423 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.423 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.423 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.423 INFO:teuthology.orchestra.run.smithi012.stdout: "username": { 2024-09-11T00:32:23.423 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "username", 2024-09-11T00:32:23.423 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:23.424 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.424 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:23.424 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-09-11T00:32:23.424 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.424 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.424 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.424 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "username of InfluxDB server user", 2024-09-11T00:32:23.424 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.424 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.424 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.424 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.424 INFO:teuthology.orchestra.run.smithi012.stdout: "verify_ssl": { 2024-09-11T00:32:23.424 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "verify_ssl", 2024-09-11T00:32:23.425 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:23.425 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.425 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:23.425 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "true", 2024-09-11T00:32:23.425 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.425 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.425 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.425 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-09-11T00:32:23.425 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.425 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.425 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.425 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:32:23.426 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:32:23.426 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.426 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-11T00:32:23.426 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "insights", 2024-09-11T00:32:23.426 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-09-11T00:32:23.426 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-09-11T00:32:23.426 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-09-11T00:32:23.426 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-09-11T00:32:23.426 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-09-11T00:32:23.426 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:23.426 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.426 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:23.426 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-09-11T00:32:23.426 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.427 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.427 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-09-11T00:32:23.427 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-09-11T00:32:23.427 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-09-11T00:32:23.427 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-09-11T00:32:23.427 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-09-11T00:32:23.427 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-09-11T00:32:23.427 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-09-11T00:32:23.427 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-09-11T00:32:23.427 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:23.427 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.427 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.428 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.428 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.428 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-09-11T00:32:23.428 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-09-11T00:32:23.428 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-09-11T00:32:23.428 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.428 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:23.428 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-09-11T00:32:23.428 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.428 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.428 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.428 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:23.428 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.429 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.429 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.429 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.429 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-09-11T00:32:23.429 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-09-11T00:32:23.429 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:23.429 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.429 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:23.429 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-09-11T00:32:23.429 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.429 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.429 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-09-11T00:32:23.429 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-09-11T00:32:23.430 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-09-11T00:32:23.430 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-09-11T00:32:23.430 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-09-11T00:32:23.430 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-09-11T00:32:23.430 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-09-11T00:32:23.430 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-09-11T00:32:23.430 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:23.430 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.430 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.430 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.430 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.430 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-09-11T00:32:23.431 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-09-11T00:32:23.431 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-09-11T00:32:23.431 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.431 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:23.431 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-09-11T00:32:23.431 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.431 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.431 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.431 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:23.431 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.431 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.431 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.431 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.431 INFO:teuthology.orchestra.run.smithi012.stdout: "sqlite3_killpoint": { 2024-09-11T00:32:23.432 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "sqlite3_killpoint", 2024-09-11T00:32:23.432 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-09-11T00:32:23.432 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "dev", 2024-09-11T00:32:23.432 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:23.432 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "0", 2024-09-11T00:32:23.432 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.432 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.432 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.432 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:23.432 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.432 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.432 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.433 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:32:23.433 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:32:23.433 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.433 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-11T00:32:23.433 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "iostat", 2024-09-11T00:32:23.433 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-09-11T00:32:23.433 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-09-11T00:32:23.433 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-09-11T00:32:23.433 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-09-11T00:32:23.433 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-09-11T00:32:23.433 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:23.433 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.433 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:23.434 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-09-11T00:32:23.434 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.434 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.434 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-09-11T00:32:23.434 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-09-11T00:32:23.434 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-09-11T00:32:23.434 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-09-11T00:32:23.434 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-09-11T00:32:23.434 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-09-11T00:32:23.434 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-09-11T00:32:23.434 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-09-11T00:32:23.434 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:23.434 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.435 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.435 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.435 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.435 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-09-11T00:32:23.435 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-09-11T00:32:23.435 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-09-11T00:32:23.435 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.435 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:23.435 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-09-11T00:32:23.435 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.435 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.435 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.435 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:23.436 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.436 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.436 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.436 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.436 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-09-11T00:32:23.436 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-09-11T00:32:23.436 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:23.436 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.436 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:23.436 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-09-11T00:32:23.436 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.436 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.437 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-09-11T00:32:23.437 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-09-11T00:32:23.437 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-09-11T00:32:23.437 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-09-11T00:32:23.437 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-09-11T00:32:23.437 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-09-11T00:32:23.437 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-09-11T00:32:23.437 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-09-11T00:32:23.437 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:23.437 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.437 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.437 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.437 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.437 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-09-11T00:32:23.438 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-09-11T00:32:23.438 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-09-11T00:32:23.438 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.438 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:23.438 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-09-11T00:32:23.438 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.438 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.438 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.438 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:23.438 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.438 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.438 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.438 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.439 INFO:teuthology.orchestra.run.smithi012.stdout: "sqlite3_killpoint": { 2024-09-11T00:32:23.439 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "sqlite3_killpoint", 2024-09-11T00:32:23.439 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-09-11T00:32:23.439 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "dev", 2024-09-11T00:32:23.439 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:23.439 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "0", 2024-09-11T00:32:23.439 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.439 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.439 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.439 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:23.439 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.439 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.440 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.440 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:32:23.440 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:32:23.440 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.440 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-11T00:32:23.440 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "k8sevents", 2024-09-11T00:32:23.440 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-09-11T00:32:23.440 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-09-11T00:32:23.440 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-09-11T00:32:23.440 INFO:teuthology.orchestra.run.smithi012.stdout: "ceph_event_retention_days": { 2024-09-11T00:32:23.440 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "ceph_event_retention_days", 2024-09-11T00:32:23.440 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-09-11T00:32:23.440 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.441 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:23.441 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "7", 2024-09-11T00:32:23.441 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.441 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.441 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.441 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Days to hold ceph event information within local cache", 2024-09-11T00:32:23.441 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.441 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.441 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.441 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.441 INFO:teuthology.orchestra.run.smithi012.stdout: "config_check_secs": { 2024-09-11T00:32:23.441 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "config_check_secs", 2024-09-11T00:32:23.442 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-09-11T00:32:23.442 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.442 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:23.442 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "10", 2024-09-11T00:32:23.442 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "10", 2024-09-11T00:32:23.442 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.442 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.442 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-09-11T00:32:23.442 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.442 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.442 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.442 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.442 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-09-11T00:32:23.443 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-09-11T00:32:23.443 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:23.443 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.443 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:23.443 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-09-11T00:32:23.443 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.443 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.443 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-09-11T00:32:23.443 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-09-11T00:32:23.443 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-09-11T00:32:23.443 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-09-11T00:32:23.443 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-09-11T00:32:23.443 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-09-11T00:32:23.443 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-09-11T00:32:23.444 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-09-11T00:32:23.444 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:23.444 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.444 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.444 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.444 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.444 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-09-11T00:32:23.444 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-09-11T00:32:23.444 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-09-11T00:32:23.444 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.444 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:23.444 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-09-11T00:32:23.444 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.445 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.445 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.445 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:23.445 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.445 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.445 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.445 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.445 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-09-11T00:32:23.445 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-09-11T00:32:23.445 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:23.445 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.445 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:23.445 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-09-11T00:32:23.446 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.446 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.446 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-09-11T00:32:23.446 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-09-11T00:32:23.446 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-09-11T00:32:23.446 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-09-11T00:32:23.446 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-09-11T00:32:23.446 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-09-11T00:32:23.446 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-09-11T00:32:23.446 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-09-11T00:32:23.446 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:23.446 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.446 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.446 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.447 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.447 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-09-11T00:32:23.447 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-09-11T00:32:23.447 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-09-11T00:32:23.447 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.447 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:23.447 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-09-11T00:32:23.447 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.447 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.447 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.447 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:23.447 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.447 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.447 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.448 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.448 INFO:teuthology.orchestra.run.smithi012.stdout: "sqlite3_killpoint": { 2024-09-11T00:32:23.448 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "sqlite3_killpoint", 2024-09-11T00:32:23.448 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-09-11T00:32:23.448 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "dev", 2024-09-11T00:32:23.448 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:23.448 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "0", 2024-09-11T00:32:23.448 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.448 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.448 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.448 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:23.448 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.448 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.449 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.449 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:32:23.449 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:32:23.449 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.449 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-11T00:32:23.449 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "localpool", 2024-09-11T00:32:23.449 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-09-11T00:32:23.449 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-09-11T00:32:23.449 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-09-11T00:32:23.449 INFO:teuthology.orchestra.run.smithi012.stdout: "failure_domain": { 2024-09-11T00:32:23.449 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "failure_domain", 2024-09-11T00:32:23.449 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:23.449 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.450 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:23.450 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "host", 2024-09-11T00:32:23.450 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.450 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.450 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.450 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "failure domain for any created local pool", 2024-09-11T00:32:23.450 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-09-11T00:32:23.450 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.450 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.450 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.450 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-09-11T00:32:23.450 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-09-11T00:32:23.450 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:23.451 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.451 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:23.451 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-09-11T00:32:23.451 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.451 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.451 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-09-11T00:32:23.451 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-09-11T00:32:23.451 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-09-11T00:32:23.451 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-09-11T00:32:23.451 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-09-11T00:32:23.451 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-09-11T00:32:23.451 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-09-11T00:32:23.451 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-09-11T00:32:23.452 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:23.452 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.452 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.452 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.452 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.452 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-09-11T00:32:23.452 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-09-11T00:32:23.452 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-09-11T00:32:23.452 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.452 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:23.452 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-09-11T00:32:23.452 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.452 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.453 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.453 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:23.453 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.453 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.453 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.453 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.453 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-09-11T00:32:23.453 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-09-11T00:32:23.453 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:23.453 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.453 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:23.453 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-09-11T00:32:23.453 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.454 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.454 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-09-11T00:32:23.454 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-09-11T00:32:23.454 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-09-11T00:32:23.454 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-09-11T00:32:23.454 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-09-11T00:32:23.454 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-09-11T00:32:23.454 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-09-11T00:32:23.454 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-09-11T00:32:23.454 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:23.454 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.454 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.454 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.455 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.455 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-09-11T00:32:23.455 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-09-11T00:32:23.455 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-09-11T00:32:23.455 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.455 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:23.455 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-09-11T00:32:23.455 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.455 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.455 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.455 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:23.455 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.455 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.456 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.456 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.456 INFO:teuthology.orchestra.run.smithi012.stdout: "min_size": { 2024-09-11T00:32:23.456 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "min_size", 2024-09-11T00:32:23.456 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-09-11T00:32:23.456 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.456 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:23.456 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-09-11T00:32:23.456 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.456 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.456 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.456 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "default min_size for any created local pool", 2024-09-11T00:32:23.457 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-09-11T00:32:23.457 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.457 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.457 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.457 INFO:teuthology.orchestra.run.smithi012.stdout: "num_rep": { 2024-09-11T00:32:23.457 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "num_rep", 2024-09-11T00:32:23.457 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-09-11T00:32:23.457 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.457 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:23.457 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "3", 2024-09-11T00:32:23.457 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.457 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.457 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.458 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "default replica count for any created local pool", 2024-09-11T00:32:23.458 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.458 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.458 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.458 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.458 INFO:teuthology.orchestra.run.smithi012.stdout: "pg_num": { 2024-09-11T00:32:23.458 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "pg_num", 2024-09-11T00:32:23.458 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-09-11T00:32:23.458 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.458 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:23.458 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "128", 2024-09-11T00:32:23.458 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.458 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.458 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.459 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "default pg_num for any created local pool", 2024-09-11T00:32:23.459 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.459 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.459 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.459 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.459 INFO:teuthology.orchestra.run.smithi012.stdout: "prefix": { 2024-09-11T00:32:23.459 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "prefix", 2024-09-11T00:32:23.459 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:23.459 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.459 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:23.459 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-09-11T00:32:23.459 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.460 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.460 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.460 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "name prefix for any created local pool", 2024-09-11T00:32:23.460 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.460 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.460 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.460 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.460 INFO:teuthology.orchestra.run.smithi012.stdout: "sqlite3_killpoint": { 2024-09-11T00:32:23.460 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "sqlite3_killpoint", 2024-09-11T00:32:23.460 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-09-11T00:32:23.460 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "dev", 2024-09-11T00:32:23.460 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:23.460 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "0", 2024-09-11T00:32:23.461 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.461 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.461 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.461 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:23.461 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.461 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.461 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.461 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.461 INFO:teuthology.orchestra.run.smithi012.stdout: "subtree": { 2024-09-11T00:32:23.461 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "subtree", 2024-09-11T00:32:23.461 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:23.461 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.462 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:23.462 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "rack", 2024-09-11T00:32:23.462 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.462 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.462 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.462 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "CRUSH level for which to create a local pool", 2024-09-11T00:32:23.462 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-09-11T00:32:23.462 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.462 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.462 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:32:23.462 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:32:23.462 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.462 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-11T00:32:23.463 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "mds_autoscaler", 2024-09-11T00:32:23.463 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-09-11T00:32:23.463 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-09-11T00:32:23.463 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-09-11T00:32:23.463 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-09-11T00:32:23.463 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-09-11T00:32:23.463 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:23.463 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.463 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:23.463 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-09-11T00:32:23.463 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.463 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.463 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-09-11T00:32:23.463 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-09-11T00:32:23.464 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-09-11T00:32:23.464 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-09-11T00:32:23.464 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-09-11T00:32:23.464 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-09-11T00:32:23.464 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-09-11T00:32:23.464 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-09-11T00:32:23.464 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:23.464 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.464 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.464 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.464 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.464 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-09-11T00:32:23.464 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-09-11T00:32:23.465 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-09-11T00:32:23.465 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.465 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:23.465 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-09-11T00:32:23.465 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.465 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.468 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.468 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:23.468 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.468 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.468 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.468 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.469 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-09-11T00:32:23.469 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-09-11T00:32:23.469 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:23.469 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.469 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:23.469 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-09-11T00:32:23.469 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.469 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.469 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-09-11T00:32:23.469 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-09-11T00:32:23.469 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-09-11T00:32:23.469 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-09-11T00:32:23.469 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-09-11T00:32:23.470 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-09-11T00:32:23.470 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-09-11T00:32:23.470 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-09-11T00:32:23.470 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:23.470 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.470 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.470 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.470 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.470 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-09-11T00:32:23.470 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-09-11T00:32:23.470 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-09-11T00:32:23.470 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.470 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:23.471 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-09-11T00:32:23.471 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.471 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.471 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.471 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:23.471 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.471 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.471 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.471 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.471 INFO:teuthology.orchestra.run.smithi012.stdout: "sqlite3_killpoint": { 2024-09-11T00:32:23.471 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "sqlite3_killpoint", 2024-09-11T00:32:23.471 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-09-11T00:32:23.472 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "dev", 2024-09-11T00:32:23.472 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:23.472 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "0", 2024-09-11T00:32:23.472 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.472 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.472 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.472 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:23.472 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.472 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.472 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.472 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:32:23.472 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:32:23.473 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.473 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-11T00:32:23.473 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "mirroring", 2024-09-11T00:32:23.473 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-09-11T00:32:23.473 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-09-11T00:32:23.473 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-09-11T00:32:23.473 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-09-11T00:32:23.473 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-09-11T00:32:23.473 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:23.473 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.473 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:23.473 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-09-11T00:32:23.473 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.474 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.474 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-09-11T00:32:23.474 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-09-11T00:32:23.474 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-09-11T00:32:23.474 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-09-11T00:32:23.474 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-09-11T00:32:23.474 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-09-11T00:32:23.474 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-09-11T00:32:23.474 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-09-11T00:32:23.474 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:23.474 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.474 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.474 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.475 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.475 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-09-11T00:32:23.475 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-09-11T00:32:23.475 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-09-11T00:32:23.475 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.475 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:23.475 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-09-11T00:32:23.475 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.475 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.475 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.475 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:23.475 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.475 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.475 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.476 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.476 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-09-11T00:32:23.476 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-09-11T00:32:23.476 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:23.476 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.476 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:23.476 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-09-11T00:32:23.476 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.476 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.476 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-09-11T00:32:23.476 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-09-11T00:32:23.476 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-09-11T00:32:23.476 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-09-11T00:32:23.477 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-09-11T00:32:23.477 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-09-11T00:32:23.477 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-09-11T00:32:23.477 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-09-11T00:32:23.477 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:23.477 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.477 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.477 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.477 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.477 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-09-11T00:32:23.477 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-09-11T00:32:23.477 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-09-11T00:32:23.478 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.478 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:23.478 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-09-11T00:32:23.478 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.478 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.478 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.478 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:23.478 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.478 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.478 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.478 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.478 INFO:teuthology.orchestra.run.smithi012.stdout: "sqlite3_killpoint": { 2024-09-11T00:32:23.478 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "sqlite3_killpoint", 2024-09-11T00:32:23.478 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-09-11T00:32:23.479 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "dev", 2024-09-11T00:32:23.479 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:23.479 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "0", 2024-09-11T00:32:23.479 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.479 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.479 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.479 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:23.479 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.479 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.479 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.479 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:32:23.479 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:32:23.479 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.480 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-11T00:32:23.480 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "nfs", 2024-09-11T00:32:23.480 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-09-11T00:32:23.480 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-09-11T00:32:23.480 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-09-11T00:32:23.480 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-09-11T00:32:23.480 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-09-11T00:32:23.480 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:23.481 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.481 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:23.481 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-09-11T00:32:23.481 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.481 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.481 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-09-11T00:32:23.481 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-09-11T00:32:23.481 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-09-11T00:32:23.481 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-09-11T00:32:23.481 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-09-11T00:32:23.482 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-09-11T00:32:23.482 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-09-11T00:32:23.482 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-09-11T00:32:23.482 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:23.482 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.482 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.482 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.482 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.482 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-09-11T00:32:23.482 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-09-11T00:32:23.482 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-09-11T00:32:23.482 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.482 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:23.483 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-09-11T00:32:23.483 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.483 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.483 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.483 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:23.483 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.483 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.483 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.483 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.483 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-09-11T00:32:23.483 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-09-11T00:32:23.483 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:23.483 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.484 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:23.484 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-09-11T00:32:23.484 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.484 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.484 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-09-11T00:32:23.484 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-09-11T00:32:23.484 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-09-11T00:32:23.484 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-09-11T00:32:23.485 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-09-11T00:32:23.485 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-09-11T00:32:23.485 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-09-11T00:32:23.485 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-09-11T00:32:23.485 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:23.485 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.485 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.485 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.485 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.485 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-09-11T00:32:23.485 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-09-11T00:32:23.485 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-09-11T00:32:23.485 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.486 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:23.486 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-09-11T00:32:23.486 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.486 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.486 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.486 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:23.486 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.486 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.486 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.486 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.486 INFO:teuthology.orchestra.run.smithi012.stdout: "sqlite3_killpoint": { 2024-09-11T00:32:23.486 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "sqlite3_killpoint", 2024-09-11T00:32:23.486 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-09-11T00:32:23.486 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "dev", 2024-09-11T00:32:23.487 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:23.487 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "0", 2024-09-11T00:32:23.487 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.487 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.487 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.487 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:23.487 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.487 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.487 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.487 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:32:23.487 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:32:23.487 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.488 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-11T00:32:23.488 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "orchestrator", 2024-09-11T00:32:23.488 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-09-11T00:32:23.488 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-09-11T00:32:23.488 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-09-11T00:32:23.488 INFO:teuthology.orchestra.run.smithi012.stdout: "fail_fs": { 2024-09-11T00:32:23.488 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "fail_fs", 2024-09-11T00:32:23.488 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-09-11T00:32:23.488 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.488 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:23.488 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-09-11T00:32:23.488 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.489 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.489 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.489 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-09-11T00:32:23.489 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.489 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.489 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.489 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.489 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-09-11T00:32:23.489 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-09-11T00:32:23.489 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:23.489 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.489 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:23.489 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-09-11T00:32:23.490 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.490 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.490 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-09-11T00:32:23.490 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-09-11T00:32:23.490 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-09-11T00:32:23.490 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-09-11T00:32:23.490 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-09-11T00:32:23.490 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-09-11T00:32:23.490 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-09-11T00:32:23.490 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-09-11T00:32:23.490 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:23.490 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.490 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.491 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.491 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.491 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-09-11T00:32:23.491 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-09-11T00:32:23.491 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-09-11T00:32:23.491 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.491 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:23.491 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-09-11T00:32:23.491 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.491 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.491 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.491 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:23.491 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.492 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.492 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.492 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.492 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-09-11T00:32:23.492 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-09-11T00:32:23.492 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:23.492 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.492 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:23.492 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-09-11T00:32:23.492 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.492 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.492 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-09-11T00:32:23.492 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-09-11T00:32:23.493 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-09-11T00:32:23.493 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-09-11T00:32:23.493 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-09-11T00:32:23.493 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-09-11T00:32:23.493 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-09-11T00:32:23.493 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-09-11T00:32:23.493 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:23.493 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.493 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.493 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.493 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.493 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-09-11T00:32:23.493 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-09-11T00:32:23.494 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-09-11T00:32:23.494 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.494 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:23.494 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-09-11T00:32:23.494 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.494 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.494 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.494 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:23.494 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.494 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.494 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.494 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.494 INFO:teuthology.orchestra.run.smithi012.stdout: "orchestrator": { 2024-09-11T00:32:23.495 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "orchestrator", 2024-09-11T00:32:23.495 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:23.495 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.495 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:23.495 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-09-11T00:32:23.495 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.495 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.495 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-09-11T00:32:23.495 INFO:teuthology.orchestra.run.smithi012.stdout: "cephadm", 2024-09-11T00:32:23.495 INFO:teuthology.orchestra.run.smithi012.stdout: "rook", 2024-09-11T00:32:23.495 INFO:teuthology.orchestra.run.smithi012.stdout: "test_orchestrator" 2024-09-11T00:32:23.495 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-09-11T00:32:23.495 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Orchestrator backend", 2024-09-11T00:32:23.496 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.496 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.496 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.496 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.496 INFO:teuthology.orchestra.run.smithi012.stdout: "sqlite3_killpoint": { 2024-09-11T00:32:23.496 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "sqlite3_killpoint", 2024-09-11T00:32:23.496 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-09-11T00:32:23.496 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "dev", 2024-09-11T00:32:23.496 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:23.496 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "0", 2024-09-11T00:32:23.496 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.496 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.496 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.497 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:23.497 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.497 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.497 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.497 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:32:23.497 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:32:23.497 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.497 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-11T00:32:23.497 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "osd_perf_query", 2024-09-11T00:32:23.497 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-09-11T00:32:23.497 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-09-11T00:32:23.497 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-09-11T00:32:23.497 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-09-11T00:32:23.498 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-09-11T00:32:23.498 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:23.498 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.498 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:23.498 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-09-11T00:32:23.498 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.498 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.498 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-09-11T00:32:23.498 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-09-11T00:32:23.498 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-09-11T00:32:23.498 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-09-11T00:32:23.498 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-09-11T00:32:23.498 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-09-11T00:32:23.499 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-09-11T00:32:23.499 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-09-11T00:32:23.499 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:23.499 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.499 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.499 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.499 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.499 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-09-11T00:32:23.499 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-09-11T00:32:23.499 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-09-11T00:32:23.499 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.499 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:23.500 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-09-11T00:32:23.500 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.500 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.500 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.500 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:23.500 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.500 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.500 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.500 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.500 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-09-11T00:32:23.500 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-09-11T00:32:23.500 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:23.500 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.501 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:23.501 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-09-11T00:32:23.501 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.501 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.501 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-09-11T00:32:23.501 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-09-11T00:32:23.501 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-09-11T00:32:23.501 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-09-11T00:32:23.501 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-09-11T00:32:23.501 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-09-11T00:32:23.501 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-09-11T00:32:23.501 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-09-11T00:32:23.501 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:23.502 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.502 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.502 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.502 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.502 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-09-11T00:32:23.502 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-09-11T00:32:23.502 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-09-11T00:32:23.502 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.502 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:23.502 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-09-11T00:32:23.502 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.502 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.503 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.503 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:23.503 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.503 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.503 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.503 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.503 INFO:teuthology.orchestra.run.smithi012.stdout: "sqlite3_killpoint": { 2024-09-11T00:32:23.503 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "sqlite3_killpoint", 2024-09-11T00:32:23.503 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-09-11T00:32:23.503 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "dev", 2024-09-11T00:32:23.503 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:23.503 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "0", 2024-09-11T00:32:23.503 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.504 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.504 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.504 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:23.504 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.504 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.504 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.504 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:32:23.504 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:32:23.504 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.504 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-11T00:32:23.504 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "osd_support", 2024-09-11T00:32:23.504 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-09-11T00:32:23.504 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-09-11T00:32:23.505 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-09-11T00:32:23.505 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-09-11T00:32:23.505 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-09-11T00:32:23.505 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:23.505 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.505 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:23.505 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-09-11T00:32:23.505 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.505 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.505 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-09-11T00:32:23.505 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-09-11T00:32:23.505 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-09-11T00:32:23.505 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-09-11T00:32:23.506 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-09-11T00:32:23.506 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-09-11T00:32:23.506 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-09-11T00:32:23.506 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-09-11T00:32:23.506 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:23.506 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.506 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.506 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.506 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.506 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-09-11T00:32:23.506 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-09-11T00:32:23.506 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-09-11T00:32:23.506 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.507 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:23.507 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-09-11T00:32:23.507 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.507 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.507 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.507 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:23.507 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.507 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.507 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.507 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.507 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-09-11T00:32:23.507 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-09-11T00:32:23.507 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:23.508 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.508 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:23.508 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-09-11T00:32:23.508 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.508 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.508 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-09-11T00:32:23.508 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-09-11T00:32:23.508 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-09-11T00:32:23.508 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-09-11T00:32:23.508 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-09-11T00:32:23.508 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-09-11T00:32:23.508 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-09-11T00:32:23.509 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-09-11T00:32:23.509 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:23.509 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.509 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.509 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.509 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.509 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-09-11T00:32:23.509 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-09-11T00:32:23.509 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-09-11T00:32:23.509 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.509 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:23.509 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-09-11T00:32:23.510 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.510 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.510 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.510 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:23.510 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.510 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.510 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.510 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.510 INFO:teuthology.orchestra.run.smithi012.stdout: "sqlite3_killpoint": { 2024-09-11T00:32:23.510 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "sqlite3_killpoint", 2024-09-11T00:32:23.510 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-09-11T00:32:23.510 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "dev", 2024-09-11T00:32:23.510 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:23.510 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "0", 2024-09-11T00:32:23.511 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.511 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.511 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.511 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:23.511 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.511 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.511 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.511 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:32:23.511 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:32:23.511 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.511 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-11T00:32:23.511 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "pg_autoscaler", 2024-09-11T00:32:23.511 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-09-11T00:32:23.511 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-09-11T00:32:23.512 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-09-11T00:32:23.512 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-09-11T00:32:23.512 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-09-11T00:32:23.512 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:23.512 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.512 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:23.512 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-09-11T00:32:23.512 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.512 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.512 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-09-11T00:32:23.512 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-09-11T00:32:23.512 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-09-11T00:32:23.512 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-09-11T00:32:23.513 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-09-11T00:32:23.513 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-09-11T00:32:23.513 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-09-11T00:32:23.513 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-09-11T00:32:23.513 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:23.513 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.513 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.513 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.513 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.513 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-09-11T00:32:23.513 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-09-11T00:32:23.513 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-09-11T00:32:23.514 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.514 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:23.514 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-09-11T00:32:23.514 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.514 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.514 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.514 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:23.514 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.514 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.514 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.514 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.514 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-09-11T00:32:23.514 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-09-11T00:32:23.514 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:23.515 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.515 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:23.515 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-09-11T00:32:23.515 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.515 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.515 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-09-11T00:32:23.515 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-09-11T00:32:23.515 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-09-11T00:32:23.515 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-09-11T00:32:23.515 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-09-11T00:32:23.515 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-09-11T00:32:23.515 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-09-11T00:32:23.515 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-09-11T00:32:23.515 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:23.516 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.516 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.516 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.516 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.516 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-09-11T00:32:23.516 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-09-11T00:32:23.516 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-09-11T00:32:23.516 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.516 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:23.516 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-09-11T00:32:23.516 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.516 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.516 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.517 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:23.517 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.517 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.517 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.517 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.517 INFO:teuthology.orchestra.run.smithi012.stdout: "sleep_interval": { 2024-09-11T00:32:23.517 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "sleep_interval", 2024-09-11T00:32:23.517 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "secs", 2024-09-11T00:32:23.517 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.517 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:23.517 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "60", 2024-09-11T00:32:23.517 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.518 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.518 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.518 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:23.518 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.518 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.518 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.518 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.518 INFO:teuthology.orchestra.run.smithi012.stdout: "sqlite3_killpoint": { 2024-09-11T00:32:23.518 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "sqlite3_killpoint", 2024-09-11T00:32:23.518 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-09-11T00:32:23.518 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "dev", 2024-09-11T00:32:23.518 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:23.518 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "0", 2024-09-11T00:32:23.519 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.519 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.519 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.519 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:23.519 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.519 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.519 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.519 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.519 INFO:teuthology.orchestra.run.smithi012.stdout: "threshold": { 2024-09-11T00:32:23.519 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "threshold", 2024-09-11T00:32:23.519 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "float", 2024-09-11T00:32:23.519 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.519 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:23.520 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "3.0", 2024-09-11T00:32:23.520 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "1.0", 2024-09-11T00:32:23.520 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.520 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.520 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "scaling threshold", 2024-09-11T00:32:23.520 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-09-11T00:32:23.520 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.520 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.520 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:32:23.520 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:32:23.520 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.520 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-11T00:32:23.521 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "progress", 2024-09-11T00:32:23.521 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-09-11T00:32:23.521 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-09-11T00:32:23.521 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-09-11T00:32:23.521 INFO:teuthology.orchestra.run.smithi012.stdout: "allow_pg_recovery_event": { 2024-09-11T00:32:23.521 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "allow_pg_recovery_event", 2024-09-11T00:32:23.521 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-09-11T00:32:23.521 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.521 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:23.521 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-09-11T00:32:23.521 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.521 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.521 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.522 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "allow the module to show pg recovery progress", 2024-09-11T00:32:23.522 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.522 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.522 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.522 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.522 INFO:teuthology.orchestra.run.smithi012.stdout: "enabled": { 2024-09-11T00:32:23.522 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "enabled", 2024-09-11T00:32:23.522 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-09-11T00:32:23.522 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.522 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:23.522 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-09-11T00:32:23.522 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.522 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.523 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.523 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:23.523 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.523 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.523 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.523 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.523 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-09-11T00:32:23.523 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-09-11T00:32:23.523 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:23.523 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.523 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:23.523 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-09-11T00:32:23.523 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.524 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.524 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-09-11T00:32:23.524 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-09-11T00:32:23.524 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-09-11T00:32:23.524 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-09-11T00:32:23.524 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-09-11T00:32:23.524 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-09-11T00:32:23.524 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-09-11T00:32:23.524 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-09-11T00:32:23.524 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:23.524 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.524 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.524 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.525 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.525 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-09-11T00:32:23.525 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-09-11T00:32:23.525 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-09-11T00:32:23.525 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.525 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:23.525 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-09-11T00:32:23.525 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.525 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.525 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.525 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:23.525 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.525 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.526 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.526 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.526 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-09-11T00:32:23.526 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-09-11T00:32:23.526 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:23.526 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.526 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:23.526 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-09-11T00:32:23.526 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.526 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.526 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-09-11T00:32:23.526 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-09-11T00:32:23.526 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-09-11T00:32:23.527 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-09-11T00:32:23.527 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-09-11T00:32:23.527 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-09-11T00:32:23.527 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-09-11T00:32:23.527 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-09-11T00:32:23.527 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:23.527 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.527 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.527 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.527 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.527 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-09-11T00:32:23.527 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-09-11T00:32:23.528 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-09-11T00:32:23.528 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.528 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:23.528 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-09-11T00:32:23.528 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.528 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.528 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.528 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:23.528 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.528 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.528 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.528 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.528 INFO:teuthology.orchestra.run.smithi012.stdout: "max_completed_events": { 2024-09-11T00:32:23.528 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "max_completed_events", 2024-09-11T00:32:23.529 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-09-11T00:32:23.529 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.529 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:23.529 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "50", 2024-09-11T00:32:23.529 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.529 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.529 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.529 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "number of past completed events to remember", 2024-09-11T00:32:23.529 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.529 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.529 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.529 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.530 INFO:teuthology.orchestra.run.smithi012.stdout: "sleep_interval": { 2024-09-11T00:32:23.530 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "sleep_interval", 2024-09-11T00:32:23.530 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "secs", 2024-09-11T00:32:23.530 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.530 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:23.530 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "5", 2024-09-11T00:32:23.530 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.530 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.530 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.530 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "how long the module is going to sleep", 2024-09-11T00:32:23.530 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.530 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.530 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.530 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.531 INFO:teuthology.orchestra.run.smithi012.stdout: "sqlite3_killpoint": { 2024-09-11T00:32:23.531 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "sqlite3_killpoint", 2024-09-11T00:32:23.531 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-09-11T00:32:23.531 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "dev", 2024-09-11T00:32:23.531 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:23.531 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "0", 2024-09-11T00:32:23.531 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.531 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.531 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.531 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:23.531 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.531 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.531 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.532 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:32:23.532 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:32:23.532 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.532 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-11T00:32:23.532 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "prometheus", 2024-09-11T00:32:23.532 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-09-11T00:32:23.532 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-09-11T00:32:23.532 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-09-11T00:32:23.532 INFO:teuthology.orchestra.run.smithi012.stdout: "cache": { 2024-09-11T00:32:23.533 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "cache", 2024-09-11T00:32:23.533 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-09-11T00:32:23.533 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.533 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:23.533 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-09-11T00:32:23.533 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.533 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.533 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.533 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:23.533 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.533 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.533 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.533 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.534 INFO:teuthology.orchestra.run.smithi012.stdout: "exclude_perf_counters": { 2024-09-11T00:32:23.534 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "exclude_perf_counters", 2024-09-11T00:32:23.534 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-09-11T00:32:23.534 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.534 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:23.534 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-09-11T00:32:23.534 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.534 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.534 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.534 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-09-11T00:32:23.534 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-09-11T00:32:23.534 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.534 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.534 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.535 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-09-11T00:32:23.535 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-09-11T00:32:23.535 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:23.535 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.535 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:23.535 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-09-11T00:32:23.535 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.535 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.535 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-09-11T00:32:23.535 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-09-11T00:32:23.535 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-09-11T00:32:23.535 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-09-11T00:32:23.535 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-09-11T00:32:23.536 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-09-11T00:32:23.536 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-09-11T00:32:23.536 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-09-11T00:32:23.536 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:23.536 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.536 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.536 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.536 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.536 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-09-11T00:32:23.536 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-09-11T00:32:23.536 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-09-11T00:32:23.536 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.536 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:23.537 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-09-11T00:32:23.537 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.537 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.537 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.537 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:23.537 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.537 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.537 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.537 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.537 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-09-11T00:32:23.537 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-09-11T00:32:23.537 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:23.537 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.538 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:23.538 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-09-11T00:32:23.538 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.538 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.538 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-09-11T00:32:23.538 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-09-11T00:32:23.538 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-09-11T00:32:23.538 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-09-11T00:32:23.538 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-09-11T00:32:23.538 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-09-11T00:32:23.538 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-09-11T00:32:23.538 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-09-11T00:32:23.538 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:23.539 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.539 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.539 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.539 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.539 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-09-11T00:32:23.539 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-09-11T00:32:23.539 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-09-11T00:32:23.539 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.539 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:23.539 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-09-11T00:32:23.539 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.539 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.539 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.539 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:23.540 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.540 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.540 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.540 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.540 INFO:teuthology.orchestra.run.smithi012.stdout: "rbd_stats_pools": { 2024-09-11T00:32:23.540 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "rbd_stats_pools", 2024-09-11T00:32:23.540 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:23.540 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.540 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:23.540 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-09-11T00:32:23.540 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.540 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.540 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.541 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:23.541 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.541 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.541 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.541 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.541 INFO:teuthology.orchestra.run.smithi012.stdout: "rbd_stats_pools_refresh_interval": { 2024-09-11T00:32:23.541 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-09-11T00:32:23.541 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-09-11T00:32:23.541 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.541 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:23.541 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "300", 2024-09-11T00:32:23.541 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.541 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.541 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.542 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:23.542 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.542 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.542 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.542 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.542 INFO:teuthology.orchestra.run.smithi012.stdout: "scrape_interval": { 2024-09-11T00:32:23.542 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "scrape_interval", 2024-09-11T00:32:23.542 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "float", 2024-09-11T00:32:23.542 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.542 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:23.542 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "15.0", 2024-09-11T00:32:23.542 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.542 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.543 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.543 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:23.543 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.543 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.543 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.543 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.543 INFO:teuthology.orchestra.run.smithi012.stdout: "server_addr": { 2024-09-11T00:32:23.543 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "server_addr", 2024-09-11T00:32:23.543 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:23.543 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.543 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:23.543 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "::", 2024-09-11T00:32:23.543 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.544 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.544 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.544 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-09-11T00:32:23.544 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.544 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.544 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.544 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.544 INFO:teuthology.orchestra.run.smithi012.stdout: "server_port": { 2024-09-11T00:32:23.544 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "server_port", 2024-09-11T00:32:23.544 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-09-11T00:32:23.544 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.544 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:23.544 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "9283", 2024-09-11T00:32:23.544 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.545 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.545 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.545 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-09-11T00:32:23.545 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.545 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.545 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.545 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.545 INFO:teuthology.orchestra.run.smithi012.stdout: "sqlite3_killpoint": { 2024-09-11T00:32:23.545 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "sqlite3_killpoint", 2024-09-11T00:32:23.545 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-09-11T00:32:23.545 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "dev", 2024-09-11T00:32:23.545 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:23.545 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "0", 2024-09-11T00:32:23.546 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.546 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.546 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.546 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:23.546 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.546 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.546 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.546 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.546 INFO:teuthology.orchestra.run.smithi012.stdout: "stale_cache_strategy": { 2024-09-11T00:32:23.546 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "stale_cache_strategy", 2024-09-11T00:32:23.546 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:23.546 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.546 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:23.547 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "log", 2024-09-11T00:32:23.547 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.547 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.547 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.547 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:23.547 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.547 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.547 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.547 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.547 INFO:teuthology.orchestra.run.smithi012.stdout: "standby_behaviour": { 2024-09-11T00:32:23.547 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "standby_behaviour", 2024-09-11T00:32:23.547 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:23.547 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.548 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:23.548 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "default", 2024-09-11T00:32:23.548 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.548 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.548 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-09-11T00:32:23.548 INFO:teuthology.orchestra.run.smithi012.stdout: "default", 2024-09-11T00:32:23.548 INFO:teuthology.orchestra.run.smithi012.stdout: "error" 2024-09-11T00:32:23.548 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-09-11T00:32:23.548 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:23.548 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.548 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.548 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.549 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.549 INFO:teuthology.orchestra.run.smithi012.stdout: "standby_error_status_code": { 2024-09-11T00:32:23.549 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "standby_error_status_code", 2024-09-11T00:32:23.549 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-09-11T00:32:23.549 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.549 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:23.549 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "500", 2024-09-11T00:32:23.549 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "400", 2024-09-11T00:32:23.549 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "599", 2024-09-11T00:32:23.549 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.549 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:23.549 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.549 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.550 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.550 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:32:23.550 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:32:23.550 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.550 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-11T00:32:23.550 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "rbd_support", 2024-09-11T00:32:23.550 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-09-11T00:32:23.550 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-09-11T00:32:23.550 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-09-11T00:32:23.550 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-09-11T00:32:23.550 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-09-11T00:32:23.550 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:23.550 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.551 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:23.551 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-09-11T00:32:23.551 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.551 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.551 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-09-11T00:32:23.551 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-09-11T00:32:23.551 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-09-11T00:32:23.551 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-09-11T00:32:23.551 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-09-11T00:32:23.551 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-09-11T00:32:23.551 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-09-11T00:32:23.551 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-09-11T00:32:23.551 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:23.552 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.552 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.552 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.552 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.552 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-09-11T00:32:23.552 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-09-11T00:32:23.552 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-09-11T00:32:23.552 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.552 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:23.552 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-09-11T00:32:23.552 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.552 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.552 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.553 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:23.553 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.553 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.553 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.553 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.553 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-09-11T00:32:23.553 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-09-11T00:32:23.553 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:23.553 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.553 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:23.553 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-09-11T00:32:23.553 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.553 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.554 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-09-11T00:32:23.554 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-09-11T00:32:23.554 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-09-11T00:32:23.554 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-09-11T00:32:23.554 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-09-11T00:32:23.554 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-09-11T00:32:23.554 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-09-11T00:32:23.554 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-09-11T00:32:23.554 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:23.554 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.554 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.554 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.554 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.555 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-09-11T00:32:23.555 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-09-11T00:32:23.555 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-09-11T00:32:23.555 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.555 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:23.555 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-09-11T00:32:23.555 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.555 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.555 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.555 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:23.555 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.555 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.555 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.555 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.556 INFO:teuthology.orchestra.run.smithi012.stdout: "max_concurrent_snap_create": { 2024-09-11T00:32:23.556 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "max_concurrent_snap_create", 2024-09-11T00:32:23.556 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-09-11T00:32:23.556 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.556 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:23.556 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "10", 2024-09-11T00:32:23.556 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.556 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.556 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.556 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:23.556 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.556 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.556 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.557 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.557 INFO:teuthology.orchestra.run.smithi012.stdout: "mirror_snapshot_schedule": { 2024-09-11T00:32:23.557 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "mirror_snapshot_schedule", 2024-09-11T00:32:23.557 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:23.557 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.557 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:23.557 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-09-11T00:32:23.557 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.557 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.557 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.557 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:23.557 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.557 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.558 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.558 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.558 INFO:teuthology.orchestra.run.smithi012.stdout: "sqlite3_killpoint": { 2024-09-11T00:32:23.558 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "sqlite3_killpoint", 2024-09-11T00:32:23.558 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-09-11T00:32:23.558 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "dev", 2024-09-11T00:32:23.558 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:23.558 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "0", 2024-09-11T00:32:23.558 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.558 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.558 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.558 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:23.558 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.558 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.559 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.559 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.559 INFO:teuthology.orchestra.run.smithi012.stdout: "trash_purge_schedule": { 2024-09-11T00:32:23.559 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "trash_purge_schedule", 2024-09-11T00:32:23.559 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:23.559 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.559 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:23.559 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-09-11T00:32:23.559 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.559 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.559 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.559 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:23.559 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.560 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.560 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.560 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:32:23.560 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:32:23.560 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.560 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-11T00:32:23.560 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "restful", 2024-09-11T00:32:23.560 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-09-11T00:32:23.560 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-09-11T00:32:23.560 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-09-11T00:32:23.560 INFO:teuthology.orchestra.run.smithi012.stdout: "enable_auth": { 2024-09-11T00:32:23.560 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "enable_auth", 2024-09-11T00:32:23.560 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-09-11T00:32:23.561 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.561 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:23.561 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-09-11T00:32:23.561 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.561 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.561 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.561 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:23.561 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.561 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.561 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.561 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.561 INFO:teuthology.orchestra.run.smithi012.stdout: "key_file": { 2024-09-11T00:32:23.562 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "key_file", 2024-09-11T00:32:23.562 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:23.562 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.562 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:23.562 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-09-11T00:32:23.562 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.562 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.562 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.562 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:23.562 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.562 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.562 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.563 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.563 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-09-11T00:32:23.563 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-09-11T00:32:23.563 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:23.563 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.563 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:23.563 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-09-11T00:32:23.563 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.563 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.563 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-09-11T00:32:23.563 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-09-11T00:32:23.563 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-09-11T00:32:23.563 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-09-11T00:32:23.564 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-09-11T00:32:23.564 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-09-11T00:32:23.564 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-09-11T00:32:23.564 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-09-11T00:32:23.564 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:23.564 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.564 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.564 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.564 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.564 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-09-11T00:32:23.564 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-09-11T00:32:23.564 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-09-11T00:32:23.564 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.565 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:23.565 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-09-11T00:32:23.565 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.565 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.565 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.565 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:23.565 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.565 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.565 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.565 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.565 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-09-11T00:32:23.565 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-09-11T00:32:23.565 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:23.566 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.566 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:23.566 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-09-11T00:32:23.566 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.566 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.566 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-09-11T00:32:23.566 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-09-11T00:32:23.566 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-09-11T00:32:23.566 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-09-11T00:32:23.566 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-09-11T00:32:23.566 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-09-11T00:32:23.566 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-09-11T00:32:23.566 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-09-11T00:32:23.566 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:23.567 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.567 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.567 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.567 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.567 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-09-11T00:32:23.567 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-09-11T00:32:23.567 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-09-11T00:32:23.567 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.567 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:23.567 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-09-11T00:32:23.567 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.567 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.567 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.568 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:23.568 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.568 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.568 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.568 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.568 INFO:teuthology.orchestra.run.smithi012.stdout: "max_requests": { 2024-09-11T00:32:23.568 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "max_requests", 2024-09-11T00:32:23.568 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-09-11T00:32:23.568 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.568 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:23.568 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "500", 2024-09-11T00:32:23.568 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.568 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.569 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.569 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Maximum number of requests to keep in memory. When new request comes in, the oldest request will be removed if the number of requests exceeds the max request number.if un-finished request is removed, error message will be logged in the ceph-mgr log.", 2024-09-11T00:32:23.569 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.569 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.569 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.569 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.569 INFO:teuthology.orchestra.run.smithi012.stdout: "server_addr": { 2024-09-11T00:32:23.569 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "server_addr", 2024-09-11T00:32:23.569 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:23.569 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.569 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:23.569 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-09-11T00:32:23.569 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.570 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.570 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.570 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:23.570 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.570 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.570 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.570 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.570 INFO:teuthology.orchestra.run.smithi012.stdout: "server_port": { 2024-09-11T00:32:23.570 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "server_port", 2024-09-11T00:32:23.570 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:23.570 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.570 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:23.570 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-09-11T00:32:23.571 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.571 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.571 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.571 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:23.571 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.571 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.571 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.571 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.571 INFO:teuthology.orchestra.run.smithi012.stdout: "sqlite3_killpoint": { 2024-09-11T00:32:23.571 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "sqlite3_killpoint", 2024-09-11T00:32:23.571 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-09-11T00:32:23.571 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "dev", 2024-09-11T00:32:23.571 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:23.572 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "0", 2024-09-11T00:32:23.572 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.572 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.572 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.572 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:23.572 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.572 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.572 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.572 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:32:23.572 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:32:23.572 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.572 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-11T00:32:23.572 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "rgw", 2024-09-11T00:32:23.573 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-09-11T00:32:23.573 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-09-11T00:32:23.573 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-09-11T00:32:23.573 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-09-11T00:32:23.573 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-09-11T00:32:23.573 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:23.573 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.573 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:23.573 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-09-11T00:32:23.573 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.573 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.573 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-09-11T00:32:23.573 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-09-11T00:32:23.574 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-09-11T00:32:23.574 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-09-11T00:32:23.574 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-09-11T00:32:23.574 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-09-11T00:32:23.574 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-09-11T00:32:23.574 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-09-11T00:32:23.574 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:23.574 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.574 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.574 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.574 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.574 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-09-11T00:32:23.574 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-09-11T00:32:23.574 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-09-11T00:32:23.575 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.575 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:23.575 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-09-11T00:32:23.575 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.575 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.575 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.575 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:23.575 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.575 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.575 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.575 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.575 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-09-11T00:32:23.576 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-09-11T00:32:23.576 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:23.576 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.576 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:23.576 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-09-11T00:32:23.576 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.576 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.576 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-09-11T00:32:23.576 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-09-11T00:32:23.576 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-09-11T00:32:23.576 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-09-11T00:32:23.576 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-09-11T00:32:23.576 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-09-11T00:32:23.577 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-09-11T00:32:23.577 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-09-11T00:32:23.577 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:23.577 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.577 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.577 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.577 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.577 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-09-11T00:32:23.577 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-09-11T00:32:23.577 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-09-11T00:32:23.577 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.577 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:23.577 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-09-11T00:32:23.578 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.578 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.578 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.578 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:23.578 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.578 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.578 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.578 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.578 INFO:teuthology.orchestra.run.smithi012.stdout: "secondary_zone_period_retry_limit": { 2024-09-11T00:32:23.578 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "secondary_zone_period_retry_limit", 2024-09-11T00:32:23.578 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-09-11T00:32:23.578 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.578 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:23.578 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "5", 2024-09-11T00:32:23.579 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.579 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.579 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.579 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "RGW module period update retry limit for secondary site", 2024-09-11T00:32:23.579 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.579 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.579 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.579 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.579 INFO:teuthology.orchestra.run.smithi012.stdout: "sqlite3_killpoint": { 2024-09-11T00:32:23.579 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "sqlite3_killpoint", 2024-09-11T00:32:23.579 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-09-11T00:32:23.579 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "dev", 2024-09-11T00:32:23.579 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:23.580 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "0", 2024-09-11T00:32:23.580 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.580 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.580 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.580 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:23.580 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.580 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.580 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.580 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:32:23.580 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:32:23.580 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.580 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-11T00:32:23.581 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "rook", 2024-09-11T00:32:23.581 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-09-11T00:32:23.581 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-09-11T00:32:23.581 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-09-11T00:32:23.581 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-09-11T00:32:23.581 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-09-11T00:32:23.581 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:23.581 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.581 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:23.582 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-09-11T00:32:23.582 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.582 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.582 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-09-11T00:32:23.582 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-09-11T00:32:23.582 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-09-11T00:32:23.582 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-09-11T00:32:23.582 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-09-11T00:32:23.582 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-09-11T00:32:23.582 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-09-11T00:32:23.582 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-09-11T00:32:23.582 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:23.582 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.583 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.583 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.583 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.583 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-09-11T00:32:23.583 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-09-11T00:32:23.583 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-09-11T00:32:23.583 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.583 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:23.583 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-09-11T00:32:23.583 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.583 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.583 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.583 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:23.583 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.584 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.584 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.584 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.584 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-09-11T00:32:23.584 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-09-11T00:32:23.584 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:23.584 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.584 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:23.584 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-09-11T00:32:23.584 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.584 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.584 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-09-11T00:32:23.585 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-09-11T00:32:23.585 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-09-11T00:32:23.585 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-09-11T00:32:23.585 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-09-11T00:32:23.585 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-09-11T00:32:23.585 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-09-11T00:32:23.585 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-09-11T00:32:23.585 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:23.585 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.585 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.585 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.585 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.585 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-09-11T00:32:23.586 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-09-11T00:32:23.586 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-09-11T00:32:23.586 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.586 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:23.586 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-09-11T00:32:23.586 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.586 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.586 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.586 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:23.586 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.586 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.586 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.586 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.587 INFO:teuthology.orchestra.run.smithi012.stdout: "sqlite3_killpoint": { 2024-09-11T00:32:23.587 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "sqlite3_killpoint", 2024-09-11T00:32:23.587 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-09-11T00:32:23.587 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "dev", 2024-09-11T00:32:23.587 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:23.587 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "0", 2024-09-11T00:32:23.587 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.587 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.587 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.587 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:23.587 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.587 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.587 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.588 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.588 INFO:teuthology.orchestra.run.smithi012.stdout: "storage_class": { 2024-09-11T00:32:23.588 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "storage_class", 2024-09-11T00:32:23.588 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:23.588 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.588 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:23.588 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "local", 2024-09-11T00:32:23.588 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.588 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.588 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.588 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-09-11T00:32:23.588 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.588 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.589 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.589 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:32:23.589 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:32:23.589 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.589 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-11T00:32:23.589 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "selftest", 2024-09-11T00:32:23.589 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-09-11T00:32:23.589 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-09-11T00:32:23.589 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-09-11T00:32:23.589 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-09-11T00:32:23.589 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-09-11T00:32:23.589 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:23.589 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.590 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:23.590 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-09-11T00:32:23.590 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.590 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.590 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-09-11T00:32:23.590 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-09-11T00:32:23.590 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-09-11T00:32:23.590 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-09-11T00:32:23.590 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-09-11T00:32:23.590 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-09-11T00:32:23.590 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-09-11T00:32:23.590 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-09-11T00:32:23.590 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:23.590 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.591 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.591 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.591 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.591 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-09-11T00:32:23.591 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-09-11T00:32:23.591 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-09-11T00:32:23.591 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.591 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:23.591 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-09-11T00:32:23.591 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.591 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.591 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.591 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:23.592 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.592 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.592 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.592 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.592 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-09-11T00:32:23.592 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-09-11T00:32:23.592 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:23.592 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.592 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:23.592 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-09-11T00:32:23.592 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.592 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.592 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-09-11T00:32:23.593 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-09-11T00:32:23.593 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-09-11T00:32:23.593 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-09-11T00:32:23.593 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-09-11T00:32:23.593 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-09-11T00:32:23.593 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-09-11T00:32:23.593 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-09-11T00:32:23.593 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:23.593 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.593 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.593 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.593 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.594 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-09-11T00:32:23.594 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-09-11T00:32:23.594 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-09-11T00:32:23.594 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.594 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:23.594 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-09-11T00:32:23.594 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.594 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.594 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.594 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:23.594 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.594 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.594 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.594 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.595 INFO:teuthology.orchestra.run.smithi012.stdout: "roption1": { 2024-09-11T00:32:23.595 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "roption1", 2024-09-11T00:32:23.595 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:23.595 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.595 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:23.595 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-09-11T00:32:23.595 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.595 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.595 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.595 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:23.595 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.595 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.595 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.595 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.596 INFO:teuthology.orchestra.run.smithi012.stdout: "roption2": { 2024-09-11T00:32:23.596 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "roption2", 2024-09-11T00:32:23.596 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:23.596 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.596 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:23.596 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "xyz", 2024-09-11T00:32:23.596 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.596 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.596 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.596 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:23.596 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.596 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.596 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.597 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.597 INFO:teuthology.orchestra.run.smithi012.stdout: "rwoption1": { 2024-09-11T00:32:23.597 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "rwoption1", 2024-09-11T00:32:23.597 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:23.597 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.597 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:23.597 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-09-11T00:32:23.597 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.597 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.597 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.597 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:23.597 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.597 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.598 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.598 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.598 INFO:teuthology.orchestra.run.smithi012.stdout: "rwoption2": { 2024-09-11T00:32:23.598 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "rwoption2", 2024-09-11T00:32:23.598 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-09-11T00:32:23.598 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.598 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:23.598 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-09-11T00:32:23.598 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.598 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.598 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.598 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:23.598 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.599 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.599 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.599 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.599 INFO:teuthology.orchestra.run.smithi012.stdout: "rwoption3": { 2024-09-11T00:32:23.599 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "rwoption3", 2024-09-11T00:32:23.599 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "float", 2024-09-11T00:32:23.599 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.599 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:23.599 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-09-11T00:32:23.599 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.599 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.599 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.600 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:23.600 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.600 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.600 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.600 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.600 INFO:teuthology.orchestra.run.smithi012.stdout: "rwoption4": { 2024-09-11T00:32:23.600 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "rwoption4", 2024-09-11T00:32:23.600 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:23.600 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.600 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:23.600 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-09-11T00:32:23.600 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.600 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.601 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.601 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:23.601 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.601 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.601 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.601 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.601 INFO:teuthology.orchestra.run.smithi012.stdout: "rwoption5": { 2024-09-11T00:32:23.601 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "rwoption5", 2024-09-11T00:32:23.601 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-09-11T00:32:23.601 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.601 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:23.601 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-09-11T00:32:23.601 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.602 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.602 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.602 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:23.602 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.602 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.602 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.602 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.602 INFO:teuthology.orchestra.run.smithi012.stdout: "rwoption6": { 2024-09-11T00:32:23.602 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "rwoption6", 2024-09-11T00:32:23.602 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-09-11T00:32:23.602 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.603 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:23.603 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-09-11T00:32:23.603 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.603 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.603 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.603 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:23.603 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.603 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.603 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.603 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.604 INFO:teuthology.orchestra.run.smithi012.stdout: "rwoption7": { 2024-09-11T00:32:23.604 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "rwoption7", 2024-09-11T00:32:23.604 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-09-11T00:32:23.604 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.604 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:23.604 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-09-11T00:32:23.604 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "1", 2024-09-11T00:32:23.604 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "42", 2024-09-11T00:32:23.604 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.604 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:23.604 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.604 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.604 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.605 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.605 INFO:teuthology.orchestra.run.smithi012.stdout: "sqlite3_killpoint": { 2024-09-11T00:32:23.605 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "sqlite3_killpoint", 2024-09-11T00:32:23.605 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-09-11T00:32:23.605 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "dev", 2024-09-11T00:32:23.605 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:23.605 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "0", 2024-09-11T00:32:23.605 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.605 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.605 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.605 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:23.605 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.606 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.606 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.606 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.606 INFO:teuthology.orchestra.run.smithi012.stdout: "testkey": { 2024-09-11T00:32:23.606 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "testkey", 2024-09-11T00:32:23.606 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:23.606 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.606 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:23.606 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-09-11T00:32:23.606 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.606 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.606 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.607 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:23.607 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.607 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.607 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.607 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.607 INFO:teuthology.orchestra.run.smithi012.stdout: "testlkey": { 2024-09-11T00:32:23.607 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "testlkey", 2024-09-11T00:32:23.607 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:23.607 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.607 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:23.607 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-09-11T00:32:23.607 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.608 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.608 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.608 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:23.608 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.608 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.608 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.608 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.608 INFO:teuthology.orchestra.run.smithi012.stdout: "testnewline": { 2024-09-11T00:32:23.608 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "testnewline", 2024-09-11T00:32:23.608 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:23.608 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.608 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:23.608 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-09-11T00:32:23.609 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.609 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.609 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.609 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:23.609 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.609 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.609 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.609 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:32:23.609 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:32:23.609 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.609 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-11T00:32:23.609 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "smb", 2024-09-11T00:32:23.609 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-09-11T00:32:23.610 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-09-11T00:32:23.610 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-09-11T00:32:23.610 INFO:teuthology.orchestra.run.smithi012.stdout: "internal_store_backend": { 2024-09-11T00:32:23.610 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "internal_store_backend", 2024-09-11T00:32:23.610 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:23.610 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "dev", 2024-09-11T00:32:23.610 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:23.610 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-09-11T00:32:23.610 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.610 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.610 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.610 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "set internal store backend. for develoment and testing only", 2024-09-11T00:32:23.610 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.611 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.611 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.611 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.611 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-09-11T00:32:23.611 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-09-11T00:32:23.611 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:23.611 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.611 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:23.611 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-09-11T00:32:23.611 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.611 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.611 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-09-11T00:32:23.611 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-09-11T00:32:23.611 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-09-11T00:32:23.612 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-09-11T00:32:23.612 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-09-11T00:32:23.612 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-09-11T00:32:23.612 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-09-11T00:32:23.612 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-09-11T00:32:23.612 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:23.612 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.612 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.612 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.612 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.612 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-09-11T00:32:23.612 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-09-11T00:32:23.612 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-09-11T00:32:23.613 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.613 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:23.613 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-09-11T00:32:23.613 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.613 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.613 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.613 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:23.613 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.613 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.613 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.613 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.613 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-09-11T00:32:23.614 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-09-11T00:32:23.614 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:23.614 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.614 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:23.614 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-09-11T00:32:23.614 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.614 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.614 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-09-11T00:32:23.614 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-09-11T00:32:23.614 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-09-11T00:32:23.614 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-09-11T00:32:23.614 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-09-11T00:32:23.614 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-09-11T00:32:23.614 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-09-11T00:32:23.615 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-09-11T00:32:23.615 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:23.615 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.615 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.615 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.615 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.615 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-09-11T00:32:23.615 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-09-11T00:32:23.615 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-09-11T00:32:23.615 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.615 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:23.615 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-09-11T00:32:23.615 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.615 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.616 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.616 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:23.616 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.616 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.616 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.616 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.616 INFO:teuthology.orchestra.run.smithi012.stdout: "sqlite3_killpoint": { 2024-09-11T00:32:23.616 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "sqlite3_killpoint", 2024-09-11T00:32:23.616 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-09-11T00:32:23.616 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "dev", 2024-09-11T00:32:23.616 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:23.616 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "0", 2024-09-11T00:32:23.616 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.617 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.617 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.617 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:23.617 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.617 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.617 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.617 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.617 INFO:teuthology.orchestra.run.smithi012.stdout: "update_orchestration": { 2024-09-11T00:32:23.617 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "update_orchestration", 2024-09-11T00:32:23.617 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-09-11T00:32:23.617 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.617 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:23.617 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-09-11T00:32:23.618 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.618 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.618 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.618 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "automatically update orchestration when smb resources are changed", 2024-09-11T00:32:23.618 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.618 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.618 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.618 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:32:23.618 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:32:23.618 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.618 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-11T00:32:23.618 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "snap_schedule", 2024-09-11T00:32:23.618 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-09-11T00:32:23.619 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-09-11T00:32:23.619 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-09-11T00:32:23.619 INFO:teuthology.orchestra.run.smithi012.stdout: "allow_m_granularity": { 2024-09-11T00:32:23.619 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "allow_m_granularity", 2024-09-11T00:32:23.619 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-09-11T00:32:23.619 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.619 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:23.619 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-09-11T00:32:23.619 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.619 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.619 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.619 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "allow minute scheduled snapshots", 2024-09-11T00:32:23.619 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.620 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.620 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.620 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.620 INFO:teuthology.orchestra.run.smithi012.stdout: "dump_on_update": { 2024-09-11T00:32:23.620 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "dump_on_update", 2024-09-11T00:32:23.620 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-09-11T00:32:23.620 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.620 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:23.620 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-09-11T00:32:23.620 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.620 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.620 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.621 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "dump database to debug log on update", 2024-09-11T00:32:23.621 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.621 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.621 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.621 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.621 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-09-11T00:32:23.621 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-09-11T00:32:23.621 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:23.621 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.621 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:23.621 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-09-11T00:32:23.621 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.622 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.622 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-09-11T00:32:23.622 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-09-11T00:32:23.622 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-09-11T00:32:23.622 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-09-11T00:32:23.622 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-09-11T00:32:23.622 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-09-11T00:32:23.622 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-09-11T00:32:23.622 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-09-11T00:32:23.622 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:23.622 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.622 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.623 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.623 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.623 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-09-11T00:32:23.623 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-09-11T00:32:23.623 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-09-11T00:32:23.623 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.623 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:23.623 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-09-11T00:32:23.623 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.623 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.623 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.623 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:23.624 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.624 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.624 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.624 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.624 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-09-11T00:32:23.624 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-09-11T00:32:23.624 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:23.624 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.624 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:23.624 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-09-11T00:32:23.624 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.624 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.624 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-09-11T00:32:23.625 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-09-11T00:32:23.625 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-09-11T00:32:23.625 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-09-11T00:32:23.625 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-09-11T00:32:23.625 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-09-11T00:32:23.625 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-09-11T00:32:23.625 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-09-11T00:32:23.625 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:23.625 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.625 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.625 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.625 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.626 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-09-11T00:32:23.626 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-09-11T00:32:23.626 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-09-11T00:32:23.626 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.626 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:23.626 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-09-11T00:32:23.626 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.626 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.626 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.626 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:23.626 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.626 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.627 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.627 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.627 INFO:teuthology.orchestra.run.smithi012.stdout: "sqlite3_killpoint": { 2024-09-11T00:32:23.627 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "sqlite3_killpoint", 2024-09-11T00:32:23.627 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-09-11T00:32:23.627 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "dev", 2024-09-11T00:32:23.627 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:23.627 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "0", 2024-09-11T00:32:23.627 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.627 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.627 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.627 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:23.627 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.628 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.628 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.628 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:32:23.628 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:32:23.628 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.628 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-11T00:32:23.628 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "stats", 2024-09-11T00:32:23.628 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-09-11T00:32:23.628 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-09-11T00:32:23.628 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-09-11T00:32:23.628 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-09-11T00:32:23.628 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-09-11T00:32:23.628 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:23.629 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.629 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:23.629 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-09-11T00:32:23.629 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.629 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.629 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-09-11T00:32:23.629 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-09-11T00:32:23.629 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-09-11T00:32:23.629 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-09-11T00:32:23.629 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-09-11T00:32:23.629 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-09-11T00:32:23.629 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-09-11T00:32:23.629 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-09-11T00:32:23.630 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:23.630 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.630 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.630 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.630 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.630 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-09-11T00:32:23.630 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-09-11T00:32:23.630 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-09-11T00:32:23.630 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.630 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:23.630 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-09-11T00:32:23.630 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.630 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.630 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.631 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:23.631 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.631 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.631 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.631 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.631 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-09-11T00:32:23.631 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-09-11T00:32:23.631 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:23.631 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.631 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:23.631 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-09-11T00:32:23.631 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.631 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.631 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-09-11T00:32:23.632 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-09-11T00:32:23.632 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-09-11T00:32:23.632 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-09-11T00:32:23.632 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-09-11T00:32:23.632 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-09-11T00:32:23.632 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-09-11T00:32:23.632 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-09-11T00:32:23.632 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:23.632 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.632 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.632 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.632 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.633 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-09-11T00:32:23.633 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-09-11T00:32:23.633 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-09-11T00:32:23.633 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.633 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:23.633 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-09-11T00:32:23.633 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.633 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.633 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.633 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:23.633 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.633 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.633 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.633 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.634 INFO:teuthology.orchestra.run.smithi012.stdout: "sqlite3_killpoint": { 2024-09-11T00:32:23.634 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "sqlite3_killpoint", 2024-09-11T00:32:23.634 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-09-11T00:32:23.634 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "dev", 2024-09-11T00:32:23.634 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:23.634 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "0", 2024-09-11T00:32:23.634 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.634 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.634 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.634 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:23.634 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.634 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.634 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.634 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:32:23.635 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:32:23.635 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.635 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-11T00:32:23.635 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "status", 2024-09-11T00:32:23.635 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-09-11T00:32:23.635 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-09-11T00:32:23.635 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-09-11T00:32:23.635 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-09-11T00:32:23.635 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-09-11T00:32:23.635 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:23.635 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.635 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:23.635 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-09-11T00:32:23.636 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.636 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.636 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-09-11T00:32:23.636 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-09-11T00:32:23.636 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-09-11T00:32:23.636 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-09-11T00:32:23.636 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-09-11T00:32:23.636 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-09-11T00:32:23.636 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-09-11T00:32:23.636 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-09-11T00:32:23.636 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:23.636 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.637 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.637 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.637 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.637 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-09-11T00:32:23.637 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-09-11T00:32:23.637 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-09-11T00:32:23.637 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.637 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:23.637 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-09-11T00:32:23.637 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.637 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.637 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.637 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:23.638 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.638 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.638 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.638 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.638 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-09-11T00:32:23.638 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-09-11T00:32:23.638 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:23.638 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.638 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:23.638 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-09-11T00:32:23.638 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.638 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.638 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-09-11T00:32:23.639 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-09-11T00:32:23.639 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-09-11T00:32:23.639 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-09-11T00:32:23.639 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-09-11T00:32:23.639 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-09-11T00:32:23.639 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-09-11T00:32:23.639 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-09-11T00:32:23.639 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:23.639 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.639 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.639 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.639 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.639 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-09-11T00:32:23.640 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-09-11T00:32:23.640 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-09-11T00:32:23.640 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.640 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:23.640 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-09-11T00:32:23.640 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.640 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.640 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.640 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:23.640 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.640 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.640 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.640 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.640 INFO:teuthology.orchestra.run.smithi012.stdout: "sqlite3_killpoint": { 2024-09-11T00:32:23.641 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "sqlite3_killpoint", 2024-09-11T00:32:23.641 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-09-11T00:32:23.641 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "dev", 2024-09-11T00:32:23.641 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:23.641 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "0", 2024-09-11T00:32:23.641 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.641 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.641 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.641 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:23.641 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.641 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.641 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.641 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:32:23.642 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:32:23.642 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.642 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-11T00:32:23.642 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "telegraf", 2024-09-11T00:32:23.642 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-09-11T00:32:23.642 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-09-11T00:32:23.642 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-09-11T00:32:23.642 INFO:teuthology.orchestra.run.smithi012.stdout: "address": { 2024-09-11T00:32:23.642 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "address", 2024-09-11T00:32:23.642 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:23.642 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.642 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:23.642 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-09-11T00:32:23.643 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.643 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.643 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.643 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:23.643 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.643 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.643 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.643 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.643 INFO:teuthology.orchestra.run.smithi012.stdout: "interval": { 2024-09-11T00:32:23.643 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "interval", 2024-09-11T00:32:23.643 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "secs", 2024-09-11T00:32:23.643 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.643 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:23.644 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "15", 2024-09-11T00:32:23.644 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.644 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.644 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.644 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:23.644 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.644 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.644 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.644 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.644 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-09-11T00:32:23.644 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-09-11T00:32:23.644 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:23.644 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.645 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:23.645 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-09-11T00:32:23.645 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.645 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.645 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-09-11T00:32:23.645 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-09-11T00:32:23.645 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-09-11T00:32:23.645 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-09-11T00:32:23.645 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-09-11T00:32:23.645 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-09-11T00:32:23.645 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-09-11T00:32:23.645 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-09-11T00:32:23.645 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:23.646 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.646 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.646 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.646 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.646 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-09-11T00:32:23.646 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-09-11T00:32:23.646 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-09-11T00:32:23.646 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.646 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:23.646 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-09-11T00:32:23.646 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.647 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.647 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.647 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:23.647 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.647 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.647 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.647 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.647 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-09-11T00:32:23.647 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-09-11T00:32:23.647 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:23.647 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.647 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:23.647 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-09-11T00:32:23.648 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.648 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.648 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-09-11T00:32:23.648 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-09-11T00:32:23.648 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-09-11T00:32:23.648 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-09-11T00:32:23.648 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-09-11T00:32:23.648 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-09-11T00:32:23.648 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-09-11T00:32:23.648 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-09-11T00:32:23.648 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:23.648 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.648 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.648 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.649 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.649 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-09-11T00:32:23.649 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-09-11T00:32:23.649 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-09-11T00:32:23.649 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.649 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:23.649 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-09-11T00:32:23.649 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.649 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.649 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.649 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:23.649 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.649 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.650 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.650 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.650 INFO:teuthology.orchestra.run.smithi012.stdout: "sqlite3_killpoint": { 2024-09-11T00:32:23.650 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "sqlite3_killpoint", 2024-09-11T00:32:23.650 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-09-11T00:32:23.650 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "dev", 2024-09-11T00:32:23.650 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:23.650 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "0", 2024-09-11T00:32:23.650 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.650 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.650 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.650 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:23.650 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.650 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.651 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.651 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:32:23.651 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:32:23.651 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.651 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-11T00:32:23.651 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "telemetry", 2024-09-11T00:32:23.651 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-09-11T00:32:23.651 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-09-11T00:32:23.651 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-09-11T00:32:23.651 INFO:teuthology.orchestra.run.smithi012.stdout: "channel_basic": { 2024-09-11T00:32:23.651 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "channel_basic", 2024-09-11T00:32:23.651 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-09-11T00:32:23.651 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.652 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:23.652 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-09-11T00:32:23.652 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.652 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.652 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.652 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Share basic cluster information (size, version)", 2024-09-11T00:32:23.652 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.652 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.652 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.652 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.652 INFO:teuthology.orchestra.run.smithi012.stdout: "channel_crash": { 2024-09-11T00:32:23.652 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "channel_crash", 2024-09-11T00:32:23.652 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-09-11T00:32:23.653 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.653 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:23.653 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-09-11T00:32:23.653 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.653 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.653 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.653 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-09-11T00:32:23.653 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.653 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.653 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.653 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.653 INFO:teuthology.orchestra.run.smithi012.stdout: "channel_device": { 2024-09-11T00:32:23.653 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "channel_device", 2024-09-11T00:32:23.654 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-09-11T00:32:23.654 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.654 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:23.654 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-09-11T00:32:23.654 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.654 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.654 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.654 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-09-11T00:32:23.654 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.654 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.654 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.654 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.654 INFO:teuthology.orchestra.run.smithi012.stdout: "channel_ident": { 2024-09-11T00:32:23.655 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "channel_ident", 2024-09-11T00:32:23.655 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-09-11T00:32:23.655 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.655 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:23.655 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-09-11T00:32:23.655 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.655 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.655 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.655 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-09-11T00:32:23.655 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.655 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.655 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.655 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.656 INFO:teuthology.orchestra.run.smithi012.stdout: "channel_perf": { 2024-09-11T00:32:23.656 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "channel_perf", 2024-09-11T00:32:23.656 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-09-11T00:32:23.656 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.656 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:23.656 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-09-11T00:32:23.656 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.656 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.656 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.656 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Share various performance metrics of a cluster", 2024-09-11T00:32:23.656 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.656 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.656 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.657 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.657 INFO:teuthology.orchestra.run.smithi012.stdout: "contact": { 2024-09-11T00:32:23.657 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "contact", 2024-09-11T00:32:23.657 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:23.657 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.657 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:23.657 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-09-11T00:32:23.657 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.657 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.657 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.657 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:23.657 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.657 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.658 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.658 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.658 INFO:teuthology.orchestra.run.smithi012.stdout: "description": { 2024-09-11T00:32:23.658 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "description", 2024-09-11T00:32:23.658 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:23.658 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.658 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:23.658 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-09-11T00:32:23.658 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.658 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.658 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.658 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:23.658 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.659 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.659 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.659 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.659 INFO:teuthology.orchestra.run.smithi012.stdout: "device_url": { 2024-09-11T00:32:23.659 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "device_url", 2024-09-11T00:32:23.659 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:23.659 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.659 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:23.659 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-09-11T00:32:23.659 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.659 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.659 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.659 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:23.659 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.660 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.660 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.660 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.660 INFO:teuthology.orchestra.run.smithi012.stdout: "enabled": { 2024-09-11T00:32:23.660 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "enabled", 2024-09-11T00:32:23.660 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-09-11T00:32:23.660 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.660 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:23.660 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-09-11T00:32:23.660 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.660 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.660 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.660 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:23.661 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.661 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.661 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.661 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.661 INFO:teuthology.orchestra.run.smithi012.stdout: "interval": { 2024-09-11T00:32:23.661 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "interval", 2024-09-11T00:32:23.661 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-09-11T00:32:23.661 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.661 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:23.661 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "24", 2024-09-11T00:32:23.661 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "8", 2024-09-11T00:32:23.661 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.661 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.662 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:23.662 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.662 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.662 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.662 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.662 INFO:teuthology.orchestra.run.smithi012.stdout: "last_opt_revision": { 2024-09-11T00:32:23.662 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "last_opt_revision", 2024-09-11T00:32:23.662 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-09-11T00:32:23.662 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.662 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:23.662 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "1", 2024-09-11T00:32:23.662 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.662 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.663 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.663 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:23.663 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.663 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.663 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.663 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.663 INFO:teuthology.orchestra.run.smithi012.stdout: "leaderboard": { 2024-09-11T00:32:23.663 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "leaderboard", 2024-09-11T00:32:23.663 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-09-11T00:32:23.663 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.663 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:23.663 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-09-11T00:32:23.663 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.663 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.664 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.664 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:23.664 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.664 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.664 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.664 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.664 INFO:teuthology.orchestra.run.smithi012.stdout: "leaderboard_description": { 2024-09-11T00:32:23.664 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "leaderboard_description", 2024-09-11T00:32:23.664 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:23.664 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.664 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:23.664 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-09-11T00:32:23.664 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.665 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.665 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.665 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:23.665 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.665 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.665 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.665 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.665 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-09-11T00:32:23.665 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-09-11T00:32:23.665 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:23.665 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.665 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:23.666 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-09-11T00:32:23.666 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.666 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.666 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-09-11T00:32:23.666 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-09-11T00:32:23.666 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-09-11T00:32:23.666 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-09-11T00:32:23.666 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-09-11T00:32:23.666 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-09-11T00:32:23.666 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-09-11T00:32:23.666 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-09-11T00:32:23.666 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:23.667 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.667 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.667 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.667 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.667 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-09-11T00:32:23.667 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-09-11T00:32:23.667 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-09-11T00:32:23.667 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.667 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:23.667 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-09-11T00:32:23.667 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.667 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.667 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.667 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:23.668 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.668 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.668 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.668 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.668 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-09-11T00:32:23.668 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-09-11T00:32:23.668 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:23.668 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.668 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:23.668 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-09-11T00:32:23.668 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.668 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.669 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-09-11T00:32:23.669 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-09-11T00:32:23.669 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-09-11T00:32:23.669 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-09-11T00:32:23.669 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-09-11T00:32:23.669 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-09-11T00:32:23.669 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-09-11T00:32:23.669 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-09-11T00:32:23.669 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:23.669 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.669 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.669 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.669 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.672 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-09-11T00:32:23.672 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-09-11T00:32:23.672 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-09-11T00:32:23.672 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.672 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:23.672 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-09-11T00:32:23.673 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.673 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.673 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.673 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:23.673 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.673 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.673 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.673 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.673 INFO:teuthology.orchestra.run.smithi012.stdout: "organization": { 2024-09-11T00:32:23.673 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "organization", 2024-09-11T00:32:23.673 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:23.673 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.674 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:23.674 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-09-11T00:32:23.674 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.674 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.674 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.674 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:23.674 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.674 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.674 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.674 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.674 INFO:teuthology.orchestra.run.smithi012.stdout: "proxy": { 2024-09-11T00:32:23.674 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "proxy", 2024-09-11T00:32:23.674 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:23.674 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.675 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:23.675 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-09-11T00:32:23.675 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.675 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.675 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.675 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:23.675 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.675 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.675 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.675 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.675 INFO:teuthology.orchestra.run.smithi012.stdout: "sqlite3_killpoint": { 2024-09-11T00:32:23.675 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "sqlite3_killpoint", 2024-09-11T00:32:23.675 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-09-11T00:32:23.676 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "dev", 2024-09-11T00:32:23.676 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:23.676 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "0", 2024-09-11T00:32:23.676 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.676 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.676 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.676 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:23.676 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.676 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.676 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.676 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.676 INFO:teuthology.orchestra.run.smithi012.stdout: "url": { 2024-09-11T00:32:23.676 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "url", 2024-09-11T00:32:23.677 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:23.677 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.677 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:23.677 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-09-11T00:32:23.677 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.677 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.677 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.677 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:23.677 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.677 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.677 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.677 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:32:23.678 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:32:23.678 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.678 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-11T00:32:23.678 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "test_orchestrator", 2024-09-11T00:32:23.678 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-09-11T00:32:23.678 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-09-11T00:32:23.678 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-09-11T00:32:23.678 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-09-11T00:32:23.678 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-09-11T00:32:23.678 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:23.678 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.678 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:23.678 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-09-11T00:32:23.679 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.679 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.679 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-09-11T00:32:23.679 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-09-11T00:32:23.679 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-09-11T00:32:23.679 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-09-11T00:32:23.679 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-09-11T00:32:23.679 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-09-11T00:32:23.679 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-09-11T00:32:23.679 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-09-11T00:32:23.679 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:23.679 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.679 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.680 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.680 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.680 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-09-11T00:32:23.680 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-09-11T00:32:23.680 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-09-11T00:32:23.680 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.680 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:23.680 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-09-11T00:32:23.680 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.680 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.680 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.680 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:23.681 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.681 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.681 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.681 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.681 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-09-11T00:32:23.681 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-09-11T00:32:23.681 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:23.681 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.682 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:23.682 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-09-11T00:32:23.682 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.682 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.682 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-09-11T00:32:23.682 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-09-11T00:32:23.682 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-09-11T00:32:23.682 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-09-11T00:32:23.682 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-09-11T00:32:23.682 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-09-11T00:32:23.682 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-09-11T00:32:23.682 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-09-11T00:32:23.683 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:23.683 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.683 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.683 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.683 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.683 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-09-11T00:32:23.683 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-09-11T00:32:23.683 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-09-11T00:32:23.683 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.683 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:23.683 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-09-11T00:32:23.683 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.684 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.684 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.684 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:23.684 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.684 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.684 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.684 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.684 INFO:teuthology.orchestra.run.smithi012.stdout: "sqlite3_killpoint": { 2024-09-11T00:32:23.684 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "sqlite3_killpoint", 2024-09-11T00:32:23.684 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-09-11T00:32:23.684 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "dev", 2024-09-11T00:32:23.684 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:23.684 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "0", 2024-09-11T00:32:23.685 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.685 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.685 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.685 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:23.685 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.685 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.685 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.685 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:32:23.685 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:32:23.685 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.685 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-11T00:32:23.685 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "volumes", 2024-09-11T00:32:23.685 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-09-11T00:32:23.686 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-09-11T00:32:23.686 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-09-11T00:32:23.686 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-09-11T00:32:23.686 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-09-11T00:32:23.686 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:23.686 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.686 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:23.686 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-09-11T00:32:23.686 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.686 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.686 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-09-11T00:32:23.686 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-09-11T00:32:23.686 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-09-11T00:32:23.686 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-09-11T00:32:23.687 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-09-11T00:32:23.687 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-09-11T00:32:23.687 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-09-11T00:32:23.687 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-09-11T00:32:23.687 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:23.687 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.687 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.687 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.687 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.687 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-09-11T00:32:23.687 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-09-11T00:32:23.687 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-09-11T00:32:23.687 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.688 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:23.688 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-09-11T00:32:23.688 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.688 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.688 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.688 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:23.688 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.688 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.688 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.688 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.688 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-09-11T00:32:23.688 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-09-11T00:32:23.688 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:23.689 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.689 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:23.689 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-09-11T00:32:23.689 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.689 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.689 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-09-11T00:32:23.689 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-09-11T00:32:23.689 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-09-11T00:32:23.689 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-09-11T00:32:23.689 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-09-11T00:32:23.689 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-09-11T00:32:23.689 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-09-11T00:32:23.689 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-09-11T00:32:23.690 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:23.690 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.690 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.690 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.690 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.690 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-09-11T00:32:23.690 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-09-11T00:32:23.690 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-09-11T00:32:23.690 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.690 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:23.690 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-09-11T00:32:23.690 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.690 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.690 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.691 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:23.691 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.691 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.691 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.691 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.691 INFO:teuthology.orchestra.run.smithi012.stdout: "max_concurrent_clones": { 2024-09-11T00:32:23.691 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "max_concurrent_clones", 2024-09-11T00:32:23.691 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-09-11T00:32:23.691 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.691 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:23.691 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "4", 2024-09-11T00:32:23.691 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.691 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.692 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.692 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Number of asynchronous cloner threads", 2024-09-11T00:32:23.692 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.692 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.692 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.692 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.692 INFO:teuthology.orchestra.run.smithi012.stdout: "periodic_async_work": { 2024-09-11T00:32:23.692 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "periodic_async_work", 2024-09-11T00:32:23.692 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-09-11T00:32:23.692 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.692 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:23.692 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-09-11T00:32:23.692 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.693 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.693 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.693 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Periodically check for async work", 2024-09-11T00:32:23.693 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.693 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.693 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.693 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.693 INFO:teuthology.orchestra.run.smithi012.stdout: "snapshot_clone_delay": { 2024-09-11T00:32:23.693 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "snapshot_clone_delay", 2024-09-11T00:32:23.693 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-09-11T00:32:23.693 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.693 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:23.693 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "0", 2024-09-11T00:32:23.694 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.694 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.694 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.694 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-09-11T00:32:23.694 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.694 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.694 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.694 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.694 INFO:teuthology.orchestra.run.smithi012.stdout: "snapshot_clone_no_wait": { 2024-09-11T00:32:23.694 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "snapshot_clone_no_wait", 2024-09-11T00:32:23.694 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-09-11T00:32:23.695 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.695 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:23.695 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-09-11T00:32:23.695 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.695 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.695 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.695 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-09-11T00:32:23.695 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.695 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.695 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.695 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.695 INFO:teuthology.orchestra.run.smithi012.stdout: "sqlite3_killpoint": { 2024-09-11T00:32:23.695 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "sqlite3_killpoint", 2024-09-11T00:32:23.695 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-09-11T00:32:23.696 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "dev", 2024-09-11T00:32:23.696 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:23.696 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "0", 2024-09-11T00:32:23.696 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.696 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.696 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.696 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:23.696 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.696 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.696 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.696 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:32:23.696 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:32:23.696 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.697 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-11T00:32:23.697 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "zabbix", 2024-09-11T00:32:23.697 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-09-11T00:32:23.697 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-09-11T00:32:23.697 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-09-11T00:32:23.697 INFO:teuthology.orchestra.run.smithi012.stdout: "discovery_interval": { 2024-09-11T00:32:23.697 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "discovery_interval", 2024-09-11T00:32:23.697 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "uint", 2024-09-11T00:32:23.697 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.697 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:23.697 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "100", 2024-09-11T00:32:23.697 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.698 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.698 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.698 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:23.698 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.698 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.698 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.698 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.698 INFO:teuthology.orchestra.run.smithi012.stdout: "identifier": { 2024-09-11T00:32:23.698 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "identifier", 2024-09-11T00:32:23.698 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:23.699 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.699 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:23.699 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-09-11T00:32:23.699 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.699 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.699 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.699 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:23.699 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.699 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.699 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.699 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.699 INFO:teuthology.orchestra.run.smithi012.stdout: "interval": { 2024-09-11T00:32:23.699 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "interval", 2024-09-11T00:32:23.700 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "secs", 2024-09-11T00:32:23.700 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.700 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:23.700 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "60", 2024-09-11T00:32:23.700 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.700 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.700 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.700 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:23.700 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.700 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.700 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.700 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.701 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-09-11T00:32:23.701 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-09-11T00:32:23.701 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:23.701 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.701 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:23.701 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-09-11T00:32:23.701 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.701 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.701 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-09-11T00:32:23.701 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-09-11T00:32:23.701 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-09-11T00:32:23.701 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-09-11T00:32:23.701 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-09-11T00:32:23.702 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-09-11T00:32:23.702 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-09-11T00:32:23.702 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-09-11T00:32:23.702 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:23.702 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.702 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.702 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.702 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.702 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-09-11T00:32:23.702 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-09-11T00:32:23.702 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-09-11T00:32:23.702 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.702 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:23.703 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-09-11T00:32:23.703 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.703 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.703 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.703 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:23.703 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.703 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.703 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.703 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.703 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-09-11T00:32:23.703 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-09-11T00:32:23.703 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:23.704 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.704 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:23.704 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-09-11T00:32:23.704 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.704 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.704 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-09-11T00:32:23.704 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-09-11T00:32:23.704 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-09-11T00:32:23.704 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-09-11T00:32:23.704 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-09-11T00:32:23.704 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-09-11T00:32:23.704 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-09-11T00:32:23.705 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-09-11T00:32:23.705 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:23.705 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.705 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.705 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.705 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.705 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-09-11T00:32:23.705 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-09-11T00:32:23.705 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-09-11T00:32:23.705 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.705 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:23.706 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-09-11T00:32:23.706 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.706 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.706 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.706 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:23.706 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.706 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.706 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.706 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.706 INFO:teuthology.orchestra.run.smithi012.stdout: "sqlite3_killpoint": { 2024-09-11T00:32:23.706 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "sqlite3_killpoint", 2024-09-11T00:32:23.706 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-09-11T00:32:23.706 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "dev", 2024-09-11T00:32:23.707 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:23.707 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "0", 2024-09-11T00:32:23.707 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.707 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.707 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.707 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:23.707 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.707 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.707 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.707 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.707 INFO:teuthology.orchestra.run.smithi012.stdout: "zabbix_host": { 2024-09-11T00:32:23.707 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "zabbix_host", 2024-09-11T00:32:23.708 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:23.708 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.708 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:23.708 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-09-11T00:32:23.708 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.708 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.708 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.708 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:23.708 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.708 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.708 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.708 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.708 INFO:teuthology.orchestra.run.smithi012.stdout: "zabbix_port": { 2024-09-11T00:32:23.708 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "zabbix_port", 2024-09-11T00:32:23.709 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-09-11T00:32:23.709 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.709 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:23.709 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "10051", 2024-09-11T00:32:23.709 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.709 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.709 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.709 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:23.709 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.709 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.709 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.709 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.710 INFO:teuthology.orchestra.run.smithi012.stdout: "zabbix_sender": { 2024-09-11T00:32:23.710 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "zabbix_sender", 2024-09-11T00:32:23.710 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:23.710 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.710 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:23.710 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-09-11T00:32:23.710 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.710 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.710 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.710 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:23.710 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.710 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.710 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.710 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:32:23.711 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:32:23.711 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:32:23.711 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-11T00:32:23.711 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.711 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-11T00:32:23.711 INFO:teuthology.orchestra.run.smithi012.stdout: "gid": 24131, 2024-09-11T00:32:23.711 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "y", 2024-09-11T00:32:23.711 INFO:teuthology.orchestra.run.smithi012.stdout: "mgr_features": 4540719139924082687, 2024-09-11T00:32:23.711 INFO:teuthology.orchestra.run.smithi012.stdout: "available_modules": [ 2024-09-11T00:32:23.711 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-11T00:32:23.711 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "alerts", 2024-09-11T00:32:23.711 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-09-11T00:32:23.712 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-09-11T00:32:23.712 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-09-11T00:32:23.712 INFO:teuthology.orchestra.run.smithi012.stdout: "interval": { 2024-09-11T00:32:23.712 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "interval", 2024-09-11T00:32:23.712 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "secs", 2024-09-11T00:32:23.712 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.712 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:23.712 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "60", 2024-09-11T00:32:23.712 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.712 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.712 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.712 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "How frequently to reexamine health status", 2024-09-11T00:32:23.712 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.713 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.713 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.713 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.713 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-09-11T00:32:23.713 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-09-11T00:32:23.713 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:23.713 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.713 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:23.713 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-09-11T00:32:23.713 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.713 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.713 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-09-11T00:32:23.714 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-09-11T00:32:23.714 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-09-11T00:32:23.714 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-09-11T00:32:23.714 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-09-11T00:32:23.714 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-09-11T00:32:23.714 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-09-11T00:32:23.714 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-09-11T00:32:23.714 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:23.714 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.714 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.715 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.715 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.715 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-09-11T00:32:23.715 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-09-11T00:32:23.715 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-09-11T00:32:23.715 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.715 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:23.715 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-09-11T00:32:23.715 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.715 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.715 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.715 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:23.715 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.715 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.716 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.716 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.716 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-09-11T00:32:23.716 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-09-11T00:32:23.716 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:23.716 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.716 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:23.716 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-09-11T00:32:23.716 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.716 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.716 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-09-11T00:32:23.716 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-09-11T00:32:23.717 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-09-11T00:32:23.717 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-09-11T00:32:23.717 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-09-11T00:32:23.717 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-09-11T00:32:23.717 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-09-11T00:32:23.717 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-09-11T00:32:23.717 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:23.717 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.717 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.717 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.717 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.717 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-09-11T00:32:23.717 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-09-11T00:32:23.718 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-09-11T00:32:23.718 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.718 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:23.718 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-09-11T00:32:23.718 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.718 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.718 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.718 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:23.718 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.718 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.718 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.718 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.718 INFO:teuthology.orchestra.run.smithi012.stdout: "smtp_destination": { 2024-09-11T00:32:23.719 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "smtp_destination", 2024-09-11T00:32:23.719 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:23.719 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.719 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:23.719 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-09-11T00:32:23.719 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.719 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.719 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.719 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Email address to send alerts to, use commas to separate multiple", 2024-09-11T00:32:23.719 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.719 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.719 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.719 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.720 INFO:teuthology.orchestra.run.smithi012.stdout: "smtp_from_name": { 2024-09-11T00:32:23.720 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "smtp_from_name", 2024-09-11T00:32:23.720 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:23.720 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.720 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:23.720 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "Ceph", 2024-09-11T00:32:23.720 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.720 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.720 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.720 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Email From: name", 2024-09-11T00:32:23.720 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.720 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.720 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.721 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.721 INFO:teuthology.orchestra.run.smithi012.stdout: "smtp_host": { 2024-09-11T00:32:23.721 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "smtp_host", 2024-09-11T00:32:23.721 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:23.721 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.721 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:23.721 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-09-11T00:32:23.721 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.721 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.721 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.721 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "SMTP server", 2024-09-11T00:32:23.721 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.721 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.722 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.722 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.722 INFO:teuthology.orchestra.run.smithi012.stdout: "smtp_password": { 2024-09-11T00:32:23.722 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "smtp_password", 2024-09-11T00:32:23.722 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:23.722 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.722 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:23.722 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-09-11T00:32:23.722 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.722 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.722 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.722 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Password to authenticate with", 2024-09-11T00:32:23.723 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.723 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.723 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.723 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.723 INFO:teuthology.orchestra.run.smithi012.stdout: "smtp_port": { 2024-09-11T00:32:23.723 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "smtp_port", 2024-09-11T00:32:23.723 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-09-11T00:32:23.723 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.723 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:23.723 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "465", 2024-09-11T00:32:23.723 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.723 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.723 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.723 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "SMTP port", 2024-09-11T00:32:23.724 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.724 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.724 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.724 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.724 INFO:teuthology.orchestra.run.smithi012.stdout: "smtp_sender": { 2024-09-11T00:32:23.724 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "smtp_sender", 2024-09-11T00:32:23.724 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:23.724 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.724 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:23.724 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-09-11T00:32:23.724 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.724 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.724 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.725 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "SMTP envelope sender", 2024-09-11T00:32:23.725 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.725 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.725 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.725 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.725 INFO:teuthology.orchestra.run.smithi012.stdout: "smtp_ssl": { 2024-09-11T00:32:23.725 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "smtp_ssl", 2024-09-11T00:32:23.725 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-09-11T00:32:23.725 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.725 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:23.725 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-09-11T00:32:23.725 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.726 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.726 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.726 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Use SSL to connect to SMTP server", 2024-09-11T00:32:23.726 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.726 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.726 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.726 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.726 INFO:teuthology.orchestra.run.smithi012.stdout: "smtp_user": { 2024-09-11T00:32:23.726 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "smtp_user", 2024-09-11T00:32:23.726 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:23.726 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.726 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:23.726 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-09-11T00:32:23.726 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.727 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.727 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.727 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "User to authenticate as", 2024-09-11T00:32:23.727 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.727 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.727 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.727 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.727 INFO:teuthology.orchestra.run.smithi012.stdout: "sqlite3_killpoint": { 2024-09-11T00:32:23.727 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "sqlite3_killpoint", 2024-09-11T00:32:23.727 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-09-11T00:32:23.727 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "dev", 2024-09-11T00:32:23.727 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:23.727 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "0", 2024-09-11T00:32:23.728 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.728 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.728 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.728 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:23.728 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.728 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.728 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.728 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:32:23.728 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:32:23.728 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.728 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-11T00:32:23.728 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "balancer", 2024-09-11T00:32:23.729 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-09-11T00:32:23.729 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-09-11T00:32:23.729 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-09-11T00:32:23.729 INFO:teuthology.orchestra.run.smithi012.stdout: "active": { 2024-09-11T00:32:23.729 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "active", 2024-09-11T00:32:23.729 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-09-11T00:32:23.729 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.729 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:23.729 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-09-11T00:32:23.729 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.729 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.729 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.730 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "automatically balance PGs across cluster", 2024-09-11T00:32:23.730 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.730 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.730 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.730 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.730 INFO:teuthology.orchestra.run.smithi012.stdout: "begin_time": { 2024-09-11T00:32:23.730 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "begin_time", 2024-09-11T00:32:23.730 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:23.730 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.730 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:23.730 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "0000", 2024-09-11T00:32:23.730 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.730 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.731 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.731 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "beginning time of day to automatically balance", 2024-09-11T00:32:23.731 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-11T00:32:23.731 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.731 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.731 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.731 INFO:teuthology.orchestra.run.smithi012.stdout: "begin_weekday": { 2024-09-11T00:32:23.731 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "begin_weekday", 2024-09-11T00:32:23.731 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "uint", 2024-09-11T00:32:23.731 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.731 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:23.731 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "0", 2024-09-11T00:32:23.731 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "0", 2024-09-11T00:32:23.731 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "6", 2024-09-11T00:32:23.732 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.732 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-09-11T00:32:23.732 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-11T00:32:23.732 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.732 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.732 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.732 INFO:teuthology.orchestra.run.smithi012.stdout: "crush_compat_max_iterations": { 2024-09-11T00:32:23.732 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "crush_compat_max_iterations", 2024-09-11T00:32:23.732 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "uint", 2024-09-11T00:32:23.732 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.732 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:23.732 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "25", 2024-09-11T00:32:23.733 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "1", 2024-09-11T00:32:23.733 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "250", 2024-09-11T00:32:23.733 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.733 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-09-11T00:32:23.733 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.733 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.733 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.733 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.733 INFO:teuthology.orchestra.run.smithi012.stdout: "crush_compat_metrics": { 2024-09-11T00:32:23.733 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "crush_compat_metrics", 2024-09-11T00:32:23.733 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:23.733 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.733 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:23.734 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "pgs,objects,bytes", 2024-09-11T00:32:23.734 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.734 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.734 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.734 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-09-11T00:32:23.734 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-09-11T00:32:23.734 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.734 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.734 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.734 INFO:teuthology.orchestra.run.smithi012.stdout: "crush_compat_step": { 2024-09-11T00:32:23.734 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "crush_compat_step", 2024-09-11T00:32:23.734 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "float", 2024-09-11T00:32:23.735 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.735 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:23.735 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "0.5", 2024-09-11T00:32:23.735 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "0.001", 2024-09-11T00:32:23.735 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "0.999", 2024-09-11T00:32:23.735 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.735 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "aggressiveness of optimization", 2024-09-11T00:32:23.735 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-09-11T00:32:23.735 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.735 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.735 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.735 INFO:teuthology.orchestra.run.smithi012.stdout: "end_time": { 2024-09-11T00:32:23.735 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "end_time", 2024-09-11T00:32:23.735 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:23.736 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.736 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:23.736 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "2359", 2024-09-11T00:32:23.736 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.736 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.736 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.736 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "ending time of day to automatically balance", 2024-09-11T00:32:23.736 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-11T00:32:23.736 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.736 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.736 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.736 INFO:teuthology.orchestra.run.smithi012.stdout: "end_weekday": { 2024-09-11T00:32:23.737 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "end_weekday", 2024-09-11T00:32:23.737 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "uint", 2024-09-11T00:32:23.737 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.737 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:23.737 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "0", 2024-09-11T00:32:23.737 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "0", 2024-09-11T00:32:23.737 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "6", 2024-09-11T00:32:23.737 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.737 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-09-11T00:32:23.737 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-11T00:32:23.737 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.738 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.738 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.738 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-09-11T00:32:23.738 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-09-11T00:32:23.738 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:23.738 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.738 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:23.738 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-09-11T00:32:23.738 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.738 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.738 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-09-11T00:32:23.738 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-09-11T00:32:23.738 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-09-11T00:32:23.739 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-09-11T00:32:23.739 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-09-11T00:32:23.739 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-09-11T00:32:23.739 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-09-11T00:32:23.739 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-09-11T00:32:23.739 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:23.739 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.739 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.739 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.739 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.739 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-09-11T00:32:23.739 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-09-11T00:32:23.739 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-09-11T00:32:23.740 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.740 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:23.740 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-09-11T00:32:23.740 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.740 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.740 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.740 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:23.740 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.740 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.740 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.740 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.740 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-09-11T00:32:23.740 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-09-11T00:32:23.741 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:23.741 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.741 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:23.741 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-09-11T00:32:23.741 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.741 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.741 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-09-11T00:32:23.741 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-09-11T00:32:23.741 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-09-11T00:32:23.741 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-09-11T00:32:23.741 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-09-11T00:32:23.741 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-09-11T00:32:23.741 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-09-11T00:32:23.742 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-09-11T00:32:23.742 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:23.742 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.742 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.742 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.742 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.742 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-09-11T00:32:23.742 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-09-11T00:32:23.742 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-09-11T00:32:23.742 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.742 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:23.742 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-09-11T00:32:23.742 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.743 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.743 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.743 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:23.743 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.743 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.743 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.743 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.743 INFO:teuthology.orchestra.run.smithi012.stdout: "min_score": { 2024-09-11T00:32:23.743 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "min_score", 2024-09-11T00:32:23.743 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "float", 2024-09-11T00:32:23.743 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.743 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:23.744 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "0", 2024-09-11T00:32:23.744 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.744 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.744 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.744 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-09-11T00:32:23.744 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.744 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.744 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.744 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.744 INFO:teuthology.orchestra.run.smithi012.stdout: "mode": { 2024-09-11T00:32:23.744 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "mode", 2024-09-11T00:32:23.744 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:23.745 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.745 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:23.745 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "upmap", 2024-09-11T00:32:23.745 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.745 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.745 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-09-11T00:32:23.745 INFO:teuthology.orchestra.run.smithi012.stdout: "crush-compat", 2024-09-11T00:32:23.745 INFO:teuthology.orchestra.run.smithi012.stdout: "none", 2024-09-11T00:32:23.745 INFO:teuthology.orchestra.run.smithi012.stdout: "read", 2024-09-11T00:32:23.745 INFO:teuthology.orchestra.run.smithi012.stdout: "upmap", 2024-09-11T00:32:23.745 INFO:teuthology.orchestra.run.smithi012.stdout: "upmap-read" 2024-09-11T00:32:23.745 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-09-11T00:32:23.745 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Balancer mode", 2024-09-11T00:32:23.746 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.746 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.746 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.746 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.746 INFO:teuthology.orchestra.run.smithi012.stdout: "pool_ids": { 2024-09-11T00:32:23.746 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "pool_ids", 2024-09-11T00:32:23.746 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:23.746 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.746 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:23.746 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-09-11T00:32:23.746 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.746 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.746 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.747 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-09-11T00:32:23.747 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.747 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.747 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.747 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.747 INFO:teuthology.orchestra.run.smithi012.stdout: "sleep_interval": { 2024-09-11T00:32:23.747 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "sleep_interval", 2024-09-11T00:32:23.747 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "secs", 2024-09-11T00:32:23.747 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.747 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:23.747 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "60", 2024-09-11T00:32:23.747 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.747 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.748 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.748 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-09-11T00:32:23.748 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.748 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.748 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.748 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.748 INFO:teuthology.orchestra.run.smithi012.stdout: "sqlite3_killpoint": { 2024-09-11T00:32:23.748 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "sqlite3_killpoint", 2024-09-11T00:32:23.748 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-09-11T00:32:23.748 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "dev", 2024-09-11T00:32:23.748 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:23.748 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "0", 2024-09-11T00:32:23.749 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.749 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.749 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.749 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:23.749 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.749 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.749 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.749 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.749 INFO:teuthology.orchestra.run.smithi012.stdout: "upmap_max_deviation": { 2024-09-11T00:32:23.749 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "upmap_max_deviation", 2024-09-11T00:32:23.749 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-09-11T00:32:23.749 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.749 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:23.750 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "5", 2024-09-11T00:32:23.750 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "1", 2024-09-11T00:32:23.750 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.750 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.750 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "deviation below which no optimization is attempted", 2024-09-11T00:32:23.750 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-09-11T00:32:23.750 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.750 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.750 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.750 INFO:teuthology.orchestra.run.smithi012.stdout: "upmap_max_optimizations": { 2024-09-11T00:32:23.750 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "upmap_max_optimizations", 2024-09-11T00:32:23.750 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "uint", 2024-09-11T00:32:23.751 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.751 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:23.751 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "10", 2024-09-11T00:32:23.751 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.751 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.751 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.751 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-09-11T00:32:23.751 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.751 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.751 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.751 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:32:23.751 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:32:23.751 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.751 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-11T00:32:23.752 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "cephadm", 2024-09-11T00:32:23.752 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-09-11T00:32:23.752 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-09-11T00:32:23.752 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-09-11T00:32:23.752 INFO:teuthology.orchestra.run.smithi012.stdout: "agent_down_multiplier": { 2024-09-11T00:32:23.752 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "agent_down_multiplier", 2024-09-11T00:32:23.752 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "float", 2024-09-11T00:32:23.752 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.752 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:23.752 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "3.0", 2024-09-11T00:32:23.752 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.752 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.752 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.753 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-09-11T00:32:23.753 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.753 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.753 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.753 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.753 INFO:teuthology.orchestra.run.smithi012.stdout: "agent_refresh_rate": { 2024-09-11T00:32:23.753 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "agent_refresh_rate", 2024-09-11T00:32:23.753 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "secs", 2024-09-11T00:32:23.753 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.753 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:23.753 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "20", 2024-09-11T00:32:23.753 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.754 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.754 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.754 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-09-11T00:32:23.754 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.754 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.754 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.754 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.754 INFO:teuthology.orchestra.run.smithi012.stdout: "agent_starting_port": { 2024-09-11T00:32:23.754 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "agent_starting_port", 2024-09-11T00:32:23.754 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-09-11T00:32:23.754 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.754 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:23.754 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "4721", 2024-09-11T00:32:23.754 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.755 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.755 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.755 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-09-11T00:32:23.755 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.755 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.755 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.755 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.755 INFO:teuthology.orchestra.run.smithi012.stdout: "allow_ptrace": { 2024-09-11T00:32:23.755 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "allow_ptrace", 2024-09-11T00:32:23.755 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-09-11T00:32:23.755 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.755 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:23.755 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-09-11T00:32:23.756 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.756 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.756 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.756 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-09-11T00:32:23.756 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-09-11T00:32:23.756 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.756 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.756 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.756 INFO:teuthology.orchestra.run.smithi012.stdout: "autotune_interval": { 2024-09-11T00:32:23.756 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "autotune_interval", 2024-09-11T00:32:23.756 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "secs", 2024-09-11T00:32:23.756 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.757 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:23.757 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "600", 2024-09-11T00:32:23.757 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.757 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.757 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.757 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "how frequently to autotune daemon memory", 2024-09-11T00:32:23.757 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.757 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.757 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.757 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.757 INFO:teuthology.orchestra.run.smithi012.stdout: "autotune_memory_target_ratio": { 2024-09-11T00:32:23.757 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "autotune_memory_target_ratio", 2024-09-11T00:32:23.757 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "float", 2024-09-11T00:32:23.758 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.758 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:23.758 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "0.7", 2024-09-11T00:32:23.758 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.758 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.758 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.758 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-09-11T00:32:23.758 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.758 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.758 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.758 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.758 INFO:teuthology.orchestra.run.smithi012.stdout: "cephadm_log_destination": { 2024-09-11T00:32:23.758 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "cephadm_log_destination", 2024-09-11T00:32:23.759 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:23.759 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.759 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:23.759 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-09-11T00:32:23.759 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.759 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.759 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-09-11T00:32:23.759 INFO:teuthology.orchestra.run.smithi012.stdout: "file", 2024-09-11T00:32:23.759 INFO:teuthology.orchestra.run.smithi012.stdout: "file,syslog", 2024-09-11T00:32:23.759 INFO:teuthology.orchestra.run.smithi012.stdout: "syslog" 2024-09-11T00:32:23.759 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-09-11T00:32:23.759 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-09-11T00:32:23.759 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.759 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.760 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.760 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.760 INFO:teuthology.orchestra.run.smithi012.stdout: "cgroups_split": { 2024-09-11T00:32:23.760 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "cgroups_split", 2024-09-11T00:32:23.760 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-09-11T00:32:23.760 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.760 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:23.760 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-09-11T00:32:23.760 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.760 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.760 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.760 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-09-11T00:32:23.761 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.761 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.761 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.761 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.761 INFO:teuthology.orchestra.run.smithi012.stdout: "config_checks_enabled": { 2024-09-11T00:32:23.761 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "config_checks_enabled", 2024-09-11T00:32:23.761 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-09-11T00:32:23.761 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.761 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:23.761 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-09-11T00:32:23.761 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.761 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.761 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.762 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-09-11T00:32:23.762 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.762 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.762 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.762 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.762 INFO:teuthology.orchestra.run.smithi012.stdout: "config_dashboard": { 2024-09-11T00:32:23.762 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "config_dashboard", 2024-09-11T00:32:23.762 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-09-11T00:32:23.762 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.762 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:23.762 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-09-11T00:32:23.762 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.762 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.762 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.763 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-09-11T00:32:23.763 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.763 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.763 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.763 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.763 INFO:teuthology.orchestra.run.smithi012.stdout: "container_image_alertmanager": { 2024-09-11T00:32:23.763 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "container_image_alertmanager", 2024-09-11T00:32:23.763 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:23.763 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.763 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:23.763 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.27.0", 2024-09-11T00:32:23.763 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.763 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.764 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.764 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Prometheus container image", 2024-09-11T00:32:23.764 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.764 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.764 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.764 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.764 INFO:teuthology.orchestra.run.smithi012.stdout: "container_image_base": { 2024-09-11T00:32:23.764 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "container_image_base", 2024-09-11T00:32:23.764 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:23.764 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.764 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:23.764 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "quay.io/ceph/ceph", 2024-09-11T00:32:23.764 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.764 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.765 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.765 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Container image name, without the tag", 2024-09-11T00:32:23.765 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.765 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.765 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.765 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.765 INFO:teuthology.orchestra.run.smithi012.stdout: "container_image_elasticsearch": { 2024-09-11T00:32:23.765 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "container_image_elasticsearch", 2024-09-11T00:32:23.765 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:23.765 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.765 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:23.765 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-09-11T00:32:23.765 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.766 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.766 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.766 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "elasticsearch container image", 2024-09-11T00:32:23.766 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.766 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.766 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.766 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.766 INFO:teuthology.orchestra.run.smithi012.stdout: "container_image_grafana": { 2024-09-11T00:32:23.766 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "container_image_grafana", 2024-09-11T00:32:23.766 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:23.766 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.766 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:23.767 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "quay.io/ceph/grafana:10.4.0", 2024-09-11T00:32:23.767 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.767 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.767 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.767 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Prometheus container image", 2024-09-11T00:32:23.767 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.767 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.767 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.767 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.767 INFO:teuthology.orchestra.run.smithi012.stdout: "container_image_haproxy": { 2024-09-11T00:32:23.767 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "container_image_haproxy", 2024-09-11T00:32:23.767 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:23.767 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.767 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:23.768 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-09-11T00:32:23.768 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.768 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.768 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.768 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "HAproxy container image", 2024-09-11T00:32:23.768 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.768 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.768 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.768 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.768 INFO:teuthology.orchestra.run.smithi012.stdout: "container_image_jaeger_agent": { 2024-09-11T00:32:23.768 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "container_image_jaeger_agent", 2024-09-11T00:32:23.768 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:23.768 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.768 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:23.769 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-09-11T00:32:23.769 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.769 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.769 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.769 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Jaeger agent container image", 2024-09-11T00:32:23.769 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.769 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.769 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.769 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.769 INFO:teuthology.orchestra.run.smithi012.stdout: "container_image_jaeger_collector": { 2024-09-11T00:32:23.769 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "container_image_jaeger_collector", 2024-09-11T00:32:23.769 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:23.770 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.770 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:23.770 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-09-11T00:32:23.770 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.770 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.770 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.770 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Jaeger collector container image", 2024-09-11T00:32:23.770 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.770 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.770 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.770 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.770 INFO:teuthology.orchestra.run.smithi012.stdout: "container_image_jaeger_query": { 2024-09-11T00:32:23.771 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "container_image_jaeger_query", 2024-09-11T00:32:23.771 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:23.771 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.771 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:23.771 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-09-11T00:32:23.771 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.771 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.771 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.771 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Jaeger query container image", 2024-09-11T00:32:23.771 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.771 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.771 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.771 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.772 INFO:teuthology.orchestra.run.smithi012.stdout: "container_image_keepalived": { 2024-09-11T00:32:23.772 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "container_image_keepalived", 2024-09-11T00:32:23.772 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:23.772 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.772 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:23.772 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-09-11T00:32:23.772 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.772 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.772 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.772 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Keepalived container image", 2024-09-11T00:32:23.772 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.772 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.772 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.773 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.773 INFO:teuthology.orchestra.run.smithi012.stdout: "container_image_loki": { 2024-09-11T00:32:23.773 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "container_image_loki", 2024-09-11T00:32:23.773 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:23.773 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.773 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:23.773 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "docker.io/grafana/loki:3.0.0", 2024-09-11T00:32:23.773 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.773 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.773 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.773 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Loki container image", 2024-09-11T00:32:23.773 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.773 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.774 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.774 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.774 INFO:teuthology.orchestra.run.smithi012.stdout: "container_image_nginx": { 2024-09-11T00:32:23.774 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "container_image_nginx", 2024-09-11T00:32:23.774 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:23.774 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.774 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:23.774 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "quay.io/ceph/nginx:1.26.1", 2024-09-11T00:32:23.774 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.774 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.774 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.774 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Nginx container image", 2024-09-11T00:32:23.775 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.775 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.775 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.775 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.775 INFO:teuthology.orchestra.run.smithi012.stdout: "container_image_node_exporter": { 2024-09-11T00:32:23.775 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "container_image_node_exporter", 2024-09-11T00:32:23.775 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:23.775 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.775 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:23.775 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.7.0", 2024-09-11T00:32:23.775 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.775 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.775 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.775 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Prometheus container image", 2024-09-11T00:32:23.776 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.776 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.776 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.776 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.776 INFO:teuthology.orchestra.run.smithi012.stdout: "container_image_nvmeof": { 2024-09-11T00:32:23.776 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "container_image_nvmeof", 2024-09-11T00:32:23.776 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:23.776 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.776 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:23.776 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.17", 2024-09-11T00:32:23.776 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.776 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.776 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.777 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Nvme-of container image", 2024-09-11T00:32:23.777 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.777 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.777 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.777 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.777 INFO:teuthology.orchestra.run.smithi012.stdout: "container_image_oauth2_proxy": { 2024-09-11T00:32:23.777 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "container_image_oauth2_proxy", 2024-09-11T00:32:23.777 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:23.777 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.777 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:23.777 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "quay.io/oauth2-proxy/oauth2-proxy:v7.6.0", 2024-09-11T00:32:23.777 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.777 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.777 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.778 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "oauth2-proxy container image", 2024-09-11T00:32:23.778 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.778 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.778 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.778 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.778 INFO:teuthology.orchestra.run.smithi012.stdout: "container_image_prometheus": { 2024-09-11T00:32:23.778 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "container_image_prometheus", 2024-09-11T00:32:23.778 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:23.778 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.778 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:23.778 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "quay.io/prometheus/prometheus:v2.51.0", 2024-09-11T00:32:23.778 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.778 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.779 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.779 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Prometheus container image", 2024-09-11T00:32:23.779 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.779 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.779 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.779 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.779 INFO:teuthology.orchestra.run.smithi012.stdout: "container_image_promtail": { 2024-09-11T00:32:23.779 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "container_image_promtail", 2024-09-11T00:32:23.779 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:23.779 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.779 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:23.779 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "docker.io/grafana/promtail:3.0.0", 2024-09-11T00:32:23.779 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.779 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.780 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.780 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Promtail container image", 2024-09-11T00:32:23.780 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.780 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.780 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.780 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.780 INFO:teuthology.orchestra.run.smithi012.stdout: "container_image_samba": { 2024-09-11T00:32:23.780 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "container_image_samba", 2024-09-11T00:32:23.780 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:23.780 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.780 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:23.780 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-09-11T00:32:23.780 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.780 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.781 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.781 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Samba/SMB container image", 2024-09-11T00:32:23.781 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.781 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.781 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.781 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.781 INFO:teuthology.orchestra.run.smithi012.stdout: "container_image_samba_metrics": { 2024-09-11T00:32:23.781 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "container_image_samba_metrics", 2024-09-11T00:32:23.781 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:23.781 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.781 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:23.781 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "quay.io/samba.org/samba-metrics:latest", 2024-09-11T00:32:23.782 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.782 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.782 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.782 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Samba/SMB metrics exporter container image", 2024-09-11T00:32:23.782 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.782 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.782 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.782 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.782 INFO:teuthology.orchestra.run.smithi012.stdout: "container_image_snmp_gateway": { 2024-09-11T00:32:23.783 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "container_image_snmp_gateway", 2024-09-11T00:32:23.783 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:23.783 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.783 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:23.783 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-09-11T00:32:23.783 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.783 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.783 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.783 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "SNMP Gateway container image", 2024-09-11T00:32:23.783 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.783 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.783 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.783 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.784 INFO:teuthology.orchestra.run.smithi012.stdout: "container_init": { 2024-09-11T00:32:23.784 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "container_init", 2024-09-11T00:32:23.784 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-09-11T00:32:23.784 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.784 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:23.784 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-09-11T00:32:23.784 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.784 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.784 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.784 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Run podman/docker with `--init`", 2024-09-11T00:32:23.784 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.784 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.784 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.785 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.785 INFO:teuthology.orchestra.run.smithi012.stdout: "daemon_cache_timeout": { 2024-09-11T00:32:23.785 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "daemon_cache_timeout", 2024-09-11T00:32:23.785 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "secs", 2024-09-11T00:32:23.785 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.785 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:23.785 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "600", 2024-09-11T00:32:23.785 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.785 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.785 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.785 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-09-11T00:32:23.785 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.785 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.786 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.786 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.786 INFO:teuthology.orchestra.run.smithi012.stdout: "default_cephadm_command_timeout": { 2024-09-11T00:32:23.786 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "default_cephadm_command_timeout", 2024-09-11T00:32:23.786 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-09-11T00:32:23.786 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.786 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:23.786 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "900", 2024-09-11T00:32:23.786 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.786 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.786 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.786 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-09-11T00:32:23.786 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.787 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.787 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.787 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.787 INFO:teuthology.orchestra.run.smithi012.stdout: "default_registry": { 2024-09-11T00:32:23.787 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "default_registry", 2024-09-11T00:32:23.787 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:23.787 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.787 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:23.787 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "docker.io", 2024-09-11T00:32:23.787 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.787 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.787 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.787 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-09-11T00:32:23.788 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.788 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.788 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.788 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.788 INFO:teuthology.orchestra.run.smithi012.stdout: "device_cache_timeout": { 2024-09-11T00:32:23.788 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "device_cache_timeout", 2024-09-11T00:32:23.788 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "secs", 2024-09-11T00:32:23.788 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.788 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:23.788 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "1800", 2024-09-11T00:32:23.788 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.788 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.788 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.788 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "seconds to cache device inventory", 2024-09-11T00:32:23.789 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.789 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.789 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.789 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.789 INFO:teuthology.orchestra.run.smithi012.stdout: "device_enhanced_scan": { 2024-09-11T00:32:23.789 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "device_enhanced_scan", 2024-09-11T00:32:23.789 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-09-11T00:32:23.789 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.789 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:23.789 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-09-11T00:32:23.789 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.789 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.789 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.790 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Use libstoragemgmt during device scans", 2024-09-11T00:32:23.790 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.790 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.790 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.790 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.790 INFO:teuthology.orchestra.run.smithi012.stdout: "facts_cache_timeout": { 2024-09-11T00:32:23.790 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "facts_cache_timeout", 2024-09-11T00:32:23.790 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "secs", 2024-09-11T00:32:23.790 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.790 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:23.790 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "60", 2024-09-11T00:32:23.790 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.790 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.790 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.791 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "seconds to cache host facts data", 2024-09-11T00:32:23.791 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.791 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.791 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.791 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.791 INFO:teuthology.orchestra.run.smithi012.stdout: "grafana_dashboards_path": { 2024-09-11T00:32:23.791 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "grafana_dashboards_path", 2024-09-11T00:32:23.791 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:23.791 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.791 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:23.791 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-09-11T00:32:23.791 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.791 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.791 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.792 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-09-11T00:32:23.792 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.792 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.792 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.792 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.792 INFO:teuthology.orchestra.run.smithi012.stdout: "host_check_interval": { 2024-09-11T00:32:23.792 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "host_check_interval", 2024-09-11T00:32:23.792 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "secs", 2024-09-11T00:32:23.792 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.792 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:23.792 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "600", 2024-09-11T00:32:23.792 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.792 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.793 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.793 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "how frequently to perform a host check", 2024-09-11T00:32:23.793 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.793 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.793 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.793 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.793 INFO:teuthology.orchestra.run.smithi012.stdout: "hw_monitoring": { 2024-09-11T00:32:23.793 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "hw_monitoring", 2024-09-11T00:32:23.793 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-09-11T00:32:23.793 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.793 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:23.793 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-09-11T00:32:23.793 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.794 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.794 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.794 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-09-11T00:32:23.794 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.794 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.794 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.794 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.794 INFO:teuthology.orchestra.run.smithi012.stdout: "inventory_list_all": { 2024-09-11T00:32:23.794 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "inventory_list_all", 2024-09-11T00:32:23.794 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-09-11T00:32:23.794 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.794 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:23.794 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-09-11T00:32:23.794 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.795 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.795 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.795 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-09-11T00:32:23.795 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.795 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.795 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.795 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.795 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-09-11T00:32:23.795 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-09-11T00:32:23.795 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:23.795 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.795 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:23.795 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-09-11T00:32:23.796 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.796 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.796 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-09-11T00:32:23.796 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-09-11T00:32:23.796 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-09-11T00:32:23.796 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-09-11T00:32:23.796 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-09-11T00:32:23.796 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-09-11T00:32:23.796 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-09-11T00:32:23.796 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-09-11T00:32:23.796 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:23.796 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.796 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.797 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.797 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.797 INFO:teuthology.orchestra.run.smithi012.stdout: "log_refresh_metadata": { 2024-09-11T00:32:23.797 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_refresh_metadata", 2024-09-11T00:32:23.797 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-09-11T00:32:23.797 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.797 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:23.797 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-09-11T00:32:23.797 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.797 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.797 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.797 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-09-11T00:32:23.797 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.798 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.798 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.798 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.798 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-09-11T00:32:23.798 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-09-11T00:32:23.798 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-09-11T00:32:23.798 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.798 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:23.798 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-09-11T00:32:23.798 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.798 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.798 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.798 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-09-11T00:32:23.798 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.799 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.799 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.799 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.799 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-09-11T00:32:23.799 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-09-11T00:32:23.799 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:23.799 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.799 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:23.799 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-09-11T00:32:23.799 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.799 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.799 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-09-11T00:32:23.799 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-09-11T00:32:23.800 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-09-11T00:32:23.800 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-09-11T00:32:23.800 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-09-11T00:32:23.800 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-09-11T00:32:23.800 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-09-11T00:32:23.800 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-09-11T00:32:23.800 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:23.800 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.800 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.800 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.800 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.800 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-09-11T00:32:23.801 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-09-11T00:32:23.801 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-09-11T00:32:23.801 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.801 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:23.801 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-09-11T00:32:23.801 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.801 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.801 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.801 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:23.801 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.801 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.801 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.801 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.801 INFO:teuthology.orchestra.run.smithi012.stdout: "manage_etc_ceph_ceph_conf": { 2024-09-11T00:32:23.802 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-09-11T00:32:23.802 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-09-11T00:32:23.802 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.802 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:23.802 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-09-11T00:32:23.802 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.802 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.802 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.802 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-09-11T00:32:23.802 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.802 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.802 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.802 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.803 INFO:teuthology.orchestra.run.smithi012.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-09-11T00:32:23.803 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-09-11T00:32:23.803 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:23.803 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.803 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:23.803 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "*", 2024-09-11T00:32:23.803 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.803 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.803 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.803 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-09-11T00:32:23.803 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.803 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.803 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.804 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.804 INFO:teuthology.orchestra.run.smithi012.stdout: "max_count_per_host": { 2024-09-11T00:32:23.804 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "max_count_per_host", 2024-09-11T00:32:23.804 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-09-11T00:32:23.804 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.804 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:23.804 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "10", 2024-09-11T00:32:23.804 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.804 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.804 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.804 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "max number of daemons per service per host", 2024-09-11T00:32:23.804 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.804 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.805 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.805 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.805 INFO:teuthology.orchestra.run.smithi012.stdout: "max_osd_draining_count": { 2024-09-11T00:32:23.805 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "max_osd_draining_count", 2024-09-11T00:32:23.805 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-09-11T00:32:23.805 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.805 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:23.805 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "10", 2024-09-11T00:32:23.805 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.805 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.805 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.805 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-09-11T00:32:23.805 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.806 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.806 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.806 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.806 INFO:teuthology.orchestra.run.smithi012.stdout: "migration_current": { 2024-09-11T00:32:23.806 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "migration_current", 2024-09-11T00:32:23.806 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-09-11T00:32:23.806 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.806 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:23.806 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-09-11T00:32:23.806 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.806 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.806 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.806 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "internal - do not modify", 2024-09-11T00:32:23.807 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.807 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.807 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.807 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.807 INFO:teuthology.orchestra.run.smithi012.stdout: "mode": { 2024-09-11T00:32:23.807 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "mode", 2024-09-11T00:32:23.807 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:23.807 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.807 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:23.807 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "root", 2024-09-11T00:32:23.807 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.807 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.807 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-09-11T00:32:23.807 INFO:teuthology.orchestra.run.smithi012.stdout: "cephadm-package", 2024-09-11T00:32:23.807 INFO:teuthology.orchestra.run.smithi012.stdout: "root" 2024-09-11T00:32:23.808 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-09-11T00:32:23.808 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "mode for remote execution of cephadm", 2024-09-11T00:32:23.808 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.808 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.808 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.808 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.808 INFO:teuthology.orchestra.run.smithi012.stdout: "oob_default_addr": { 2024-09-11T00:32:23.808 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "oob_default_addr", 2024-09-11T00:32:23.808 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:23.808 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.808 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:23.808 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "169.254.1.1", 2024-09-11T00:32:23.808 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.809 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.809 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.809 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Default address for RedFish API (oob management).", 2024-09-11T00:32:23.809 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.809 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.809 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.809 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.809 INFO:teuthology.orchestra.run.smithi012.stdout: "prometheus_alerts_path": { 2024-09-11T00:32:23.809 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "prometheus_alerts_path", 2024-09-11T00:32:23.809 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:23.809 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.809 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:23.809 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-09-11T00:32:23.810 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.810 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.810 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.810 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-09-11T00:32:23.810 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.810 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.810 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.810 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.810 INFO:teuthology.orchestra.run.smithi012.stdout: "registry_insecure": { 2024-09-11T00:32:23.810 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "registry_insecure", 2024-09-11T00:32:23.810 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-09-11T00:32:23.810 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.810 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:23.811 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-09-11T00:32:23.811 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.811 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.811 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.811 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-09-11T00:32:23.811 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.811 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.811 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.811 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.811 INFO:teuthology.orchestra.run.smithi012.stdout: "registry_password": { 2024-09-11T00:32:23.811 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "registry_password", 2024-09-11T00:32:23.811 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:23.811 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.811 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:23.812 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-09-11T00:32:23.812 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.812 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.812 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.812 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-09-11T00:32:23.812 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.812 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.812 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.812 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.812 INFO:teuthology.orchestra.run.smithi012.stdout: "registry_url": { 2024-09-11T00:32:23.812 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "registry_url", 2024-09-11T00:32:23.812 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:23.812 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.813 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:23.813 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-09-11T00:32:23.813 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.813 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.813 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.813 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-09-11T00:32:23.813 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.813 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.813 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.813 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.813 INFO:teuthology.orchestra.run.smithi012.stdout: "registry_username": { 2024-09-11T00:32:23.813 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "registry_username", 2024-09-11T00:32:23.813 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:23.813 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.814 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:23.814 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-09-11T00:32:23.814 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.814 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.814 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.814 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-09-11T00:32:23.814 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.814 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.814 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.814 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.814 INFO:teuthology.orchestra.run.smithi012.stdout: "secure_monitoring_stack": { 2024-09-11T00:32:23.814 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "secure_monitoring_stack", 2024-09-11T00:32:23.814 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-09-11T00:32:23.815 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.815 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:23.815 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-09-11T00:32:23.815 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.815 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.815 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.815 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-09-11T00:32:23.815 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.815 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.815 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.815 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.815 INFO:teuthology.orchestra.run.smithi012.stdout: "service_discovery_port": { 2024-09-11T00:32:23.815 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "service_discovery_port", 2024-09-11T00:32:23.815 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-09-11T00:32:23.816 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.816 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:23.816 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "8765", 2024-09-11T00:32:23.816 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.816 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.816 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.816 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "cephadm service discovery port", 2024-09-11T00:32:23.816 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.816 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.816 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.816 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.816 INFO:teuthology.orchestra.run.smithi012.stdout: "sqlite3_killpoint": { 2024-09-11T00:32:23.816 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "sqlite3_killpoint", 2024-09-11T00:32:23.817 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-09-11T00:32:23.817 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "dev", 2024-09-11T00:32:23.817 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:23.817 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "0", 2024-09-11T00:32:23.817 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.817 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.817 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.817 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:23.817 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.817 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.817 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.817 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.817 INFO:teuthology.orchestra.run.smithi012.stdout: "ssh_config_file": { 2024-09-11T00:32:23.817 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "ssh_config_file", 2024-09-11T00:32:23.818 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:23.818 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.818 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:23.818 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-09-11T00:32:23.818 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.818 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.818 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.818 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-09-11T00:32:23.818 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.818 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.818 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.818 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.818 INFO:teuthology.orchestra.run.smithi012.stdout: "use_agent": { 2024-09-11T00:32:23.818 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "use_agent", 2024-09-11T00:32:23.819 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-09-11T00:32:23.819 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.819 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:23.819 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-09-11T00:32:23.819 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.819 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.819 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.819 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-09-11T00:32:23.819 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.819 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.819 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.819 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.819 INFO:teuthology.orchestra.run.smithi012.stdout: "use_repo_digest": { 2024-09-11T00:32:23.820 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "use_repo_digest", 2024-09-11T00:32:23.820 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-09-11T00:32:23.820 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.820 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:23.820 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-09-11T00:32:23.820 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.820 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.820 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.820 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-09-11T00:32:23.820 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.820 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.820 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.820 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.821 INFO:teuthology.orchestra.run.smithi012.stdout: "warn_on_failed_host_check": { 2024-09-11T00:32:23.821 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "warn_on_failed_host_check", 2024-09-11T00:32:23.821 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-09-11T00:32:23.821 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.821 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:23.821 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-09-11T00:32:23.821 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.821 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.821 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.821 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "raise a health warning if the host check fails", 2024-09-11T00:32:23.821 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.821 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.821 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.822 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.822 INFO:teuthology.orchestra.run.smithi012.stdout: "warn_on_stray_daemons": { 2024-09-11T00:32:23.822 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "warn_on_stray_daemons", 2024-09-11T00:32:23.822 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-09-11T00:32:23.822 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.822 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:23.822 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-09-11T00:32:23.822 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.822 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.822 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.822 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-09-11T00:32:23.822 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.822 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.822 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.823 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.823 INFO:teuthology.orchestra.run.smithi012.stdout: "warn_on_stray_hosts": { 2024-09-11T00:32:23.823 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "warn_on_stray_hosts", 2024-09-11T00:32:23.823 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-09-11T00:32:23.823 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.823 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:23.823 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-09-11T00:32:23.823 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.823 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.823 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.823 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-09-11T00:32:23.823 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.823 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.824 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.824 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:32:23.824 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:32:23.824 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.824 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-11T00:32:23.824 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "crash", 2024-09-11T00:32:23.824 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-09-11T00:32:23.824 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-09-11T00:32:23.824 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-09-11T00:32:23.824 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-09-11T00:32:23.824 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-09-11T00:32:23.824 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:23.824 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.825 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:23.825 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-09-11T00:32:23.825 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.825 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.825 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-09-11T00:32:23.825 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-09-11T00:32:23.825 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-09-11T00:32:23.825 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-09-11T00:32:23.825 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-09-11T00:32:23.825 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-09-11T00:32:23.825 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-09-11T00:32:23.825 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-09-11T00:32:23.825 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:23.826 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.826 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.826 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.826 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.826 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-09-11T00:32:23.826 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-09-11T00:32:23.826 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-09-11T00:32:23.826 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.826 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:23.826 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-09-11T00:32:23.826 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.826 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.826 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.827 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:23.827 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.827 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.827 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.827 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.827 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-09-11T00:32:23.827 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-09-11T00:32:23.827 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:23.827 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.827 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:23.827 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-09-11T00:32:23.827 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.827 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.827 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-09-11T00:32:23.828 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-09-11T00:32:23.828 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-09-11T00:32:23.828 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-09-11T00:32:23.828 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-09-11T00:32:23.828 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-09-11T00:32:23.828 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-09-11T00:32:23.828 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-09-11T00:32:23.828 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:23.828 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.828 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.828 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.828 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.828 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-09-11T00:32:23.828 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-09-11T00:32:23.829 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-09-11T00:32:23.829 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.829 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:23.829 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-09-11T00:32:23.829 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.829 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.829 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.829 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:23.829 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.829 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.829 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.829 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.829 INFO:teuthology.orchestra.run.smithi012.stdout: "retain_interval": { 2024-09-11T00:32:23.830 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "retain_interval", 2024-09-11T00:32:23.830 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "secs", 2024-09-11T00:32:23.830 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.830 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:23.830 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "31536000", 2024-09-11T00:32:23.830 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.830 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.830 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.830 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "how long to retain crashes before pruning them", 2024-09-11T00:32:23.830 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.830 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.830 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.830 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.830 INFO:teuthology.orchestra.run.smithi012.stdout: "sqlite3_killpoint": { 2024-09-11T00:32:23.831 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "sqlite3_killpoint", 2024-09-11T00:32:23.831 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-09-11T00:32:23.831 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "dev", 2024-09-11T00:32:23.831 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:23.831 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "0", 2024-09-11T00:32:23.831 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.831 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.831 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.831 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:23.831 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.831 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.831 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.831 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.831 INFO:teuthology.orchestra.run.smithi012.stdout: "warn_recent_interval": { 2024-09-11T00:32:23.832 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "warn_recent_interval", 2024-09-11T00:32:23.832 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "secs", 2024-09-11T00:32:23.832 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.832 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:23.832 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "1209600", 2024-09-11T00:32:23.832 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.832 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.832 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.832 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "time interval in which to warn about recent crashes", 2024-09-11T00:32:23.832 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.832 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.832 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.832 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:32:23.833 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:32:23.833 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.833 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-11T00:32:23.833 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "dashboard", 2024-09-11T00:32:23.833 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-09-11T00:32:23.833 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-09-11T00:32:23.833 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-09-11T00:32:23.833 INFO:teuthology.orchestra.run.smithi012.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-09-11T00:32:23.833 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-09-11T00:32:23.833 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-09-11T00:32:23.833 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.833 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:23.833 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "10", 2024-09-11T00:32:23.834 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.834 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.834 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.834 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:23.834 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.834 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.834 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.834 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.834 INFO:teuthology.orchestra.run.smithi012.stdout: "ALERTMANAGER_API_HOST": { 2024-09-11T00:32:23.834 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "ALERTMANAGER_API_HOST", 2024-09-11T00:32:23.834 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:23.834 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.834 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:23.835 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-09-11T00:32:23.835 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.835 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.835 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.835 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:23.835 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.835 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.835 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.835 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.835 INFO:teuthology.orchestra.run.smithi012.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-09-11T00:32:23.835 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-09-11T00:32:23.835 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-09-11T00:32:23.835 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.836 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:23.836 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-09-11T00:32:23.836 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.836 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.836 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.836 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:23.836 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.836 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.836 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.836 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.836 INFO:teuthology.orchestra.run.smithi012.stdout: "AUDIT_API_ENABLED": { 2024-09-11T00:32:23.836 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "AUDIT_API_ENABLED", 2024-09-11T00:32:23.837 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-09-11T00:32:23.837 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.837 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:23.837 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-09-11T00:32:23.837 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.837 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.837 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.837 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:23.837 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.837 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.837 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.837 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.837 INFO:teuthology.orchestra.run.smithi012.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-09-11T00:32:23.838 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-09-11T00:32:23.838 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-09-11T00:32:23.838 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.838 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:23.838 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-09-11T00:32:23.838 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.838 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.838 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.838 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:23.838 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.838 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.838 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.838 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.839 INFO:teuthology.orchestra.run.smithi012.stdout: "ENABLE_BROWSABLE_API": { 2024-09-11T00:32:23.839 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "ENABLE_BROWSABLE_API", 2024-09-11T00:32:23.839 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-09-11T00:32:23.839 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.839 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:23.839 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-09-11T00:32:23.839 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.839 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.839 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.839 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:23.839 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.839 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.839 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.840 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.840 INFO:teuthology.orchestra.run.smithi012.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-09-11T00:32:23.840 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-09-11T00:32:23.840 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-09-11T00:32:23.840 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.840 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:23.840 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-09-11T00:32:23.840 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.840 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.840 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.840 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:23.840 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.840 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.840 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.841 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.841 INFO:teuthology.orchestra.run.smithi012.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-09-11T00:32:23.841 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-09-11T00:32:23.841 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-09-11T00:32:23.841 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.841 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:23.841 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-09-11T00:32:23.841 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.841 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.841 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.841 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:23.841 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.841 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.842 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.842 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.842 INFO:teuthology.orchestra.run.smithi012.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-09-11T00:32:23.842 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-09-11T00:32:23.842 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-09-11T00:32:23.842 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.842 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:23.842 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-09-11T00:32:23.842 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.842 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.842 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.842 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:23.843 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.843 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.843 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.843 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.843 INFO:teuthology.orchestra.run.smithi012.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-09-11T00:32:23.843 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-09-11T00:32:23.843 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-09-11T00:32:23.843 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.843 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:23.843 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-09-11T00:32:23.843 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.843 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.843 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.843 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:23.844 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.844 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.844 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.844 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.844 INFO:teuthology.orchestra.run.smithi012.stdout: "FEATURE_TOGGLE_NFS": { 2024-09-11T00:32:23.844 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-09-11T00:32:23.844 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-09-11T00:32:23.844 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.844 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:23.844 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-09-11T00:32:23.844 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.844 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.844 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.845 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:23.845 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.845 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.845 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.845 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.845 INFO:teuthology.orchestra.run.smithi012.stdout: "FEATURE_TOGGLE_RBD": { 2024-09-11T00:32:23.845 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-09-11T00:32:23.845 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-09-11T00:32:23.845 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.845 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:23.845 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-09-11T00:32:23.845 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.845 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.845 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.846 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:23.846 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.846 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.846 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.846 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.846 INFO:teuthology.orchestra.run.smithi012.stdout: "FEATURE_TOGGLE_RGW": { 2024-09-11T00:32:23.846 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-09-11T00:32:23.846 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-09-11T00:32:23.846 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.846 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:23.846 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-09-11T00:32:23.846 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.846 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.847 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.847 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:23.847 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.847 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.847 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.847 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.847 INFO:teuthology.orchestra.run.smithi012.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-09-11T00:32:23.847 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-09-11T00:32:23.847 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:23.847 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.847 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:23.847 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-09-11T00:32:23.847 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.848 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.848 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.848 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:23.848 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.848 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.848 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.848 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.848 INFO:teuthology.orchestra.run.smithi012.stdout: "GRAFANA_API_PASSWORD": { 2024-09-11T00:32:23.848 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "GRAFANA_API_PASSWORD", 2024-09-11T00:32:23.848 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:23.848 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.848 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:23.848 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "admin", 2024-09-11T00:32:23.848 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.849 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.849 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.849 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:23.849 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.849 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.849 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.849 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.849 INFO:teuthology.orchestra.run.smithi012.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-09-11T00:32:23.849 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-09-11T00:32:23.849 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-09-11T00:32:23.849 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.849 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:23.850 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-09-11T00:32:23.850 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.850 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.850 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.850 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:23.850 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.850 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.850 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.850 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.850 INFO:teuthology.orchestra.run.smithi012.stdout: "GRAFANA_API_URL": { 2024-09-11T00:32:23.850 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "GRAFANA_API_URL", 2024-09-11T00:32:23.850 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:23.850 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.851 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:23.851 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-09-11T00:32:23.851 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.851 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.851 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.851 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:23.851 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.851 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.851 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.851 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.851 INFO:teuthology.orchestra.run.smithi012.stdout: "GRAFANA_API_USERNAME": { 2024-09-11T00:32:23.851 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "GRAFANA_API_USERNAME", 2024-09-11T00:32:23.851 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:23.851 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.852 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:23.852 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "admin", 2024-09-11T00:32:23.852 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.852 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.852 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.852 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:23.852 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.852 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.852 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.852 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.852 INFO:teuthology.orchestra.run.smithi012.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-09-11T00:32:23.852 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-09-11T00:32:23.852 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:23.853 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.853 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:23.853 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-09-11T00:32:23.853 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.853 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.853 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.853 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:23.853 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.853 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.853 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.853 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.853 INFO:teuthology.orchestra.run.smithi012.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-09-11T00:32:23.853 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-09-11T00:32:23.854 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-09-11T00:32:23.854 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.854 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:23.854 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-09-11T00:32:23.854 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.854 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.854 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.854 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:23.854 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.854 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.854 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.854 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.854 INFO:teuthology.orchestra.run.smithi012.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-09-11T00:32:23.855 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-09-11T00:32:23.855 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-09-11T00:32:23.855 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.855 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:23.855 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-09-11T00:32:23.855 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.855 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.855 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.855 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:23.855 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.855 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.855 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.855 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.855 INFO:teuthology.orchestra.run.smithi012.stdout: "ISSUE_TRACKER_API_KEY": { 2024-09-11T00:32:23.856 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-09-11T00:32:23.856 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:23.856 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.856 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:23.856 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-09-11T00:32:23.856 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.856 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.856 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.856 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:23.856 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.856 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.856 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.856 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.857 INFO:teuthology.orchestra.run.smithi012.stdout: "MANAGED_BY_CLUSTERS": { 2024-09-11T00:32:23.857 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-09-11T00:32:23.857 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:23.857 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.857 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:23.857 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "[]", 2024-09-11T00:32:23.857 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.857 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.857 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.857 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:23.857 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.857 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.857 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.858 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.858 INFO:teuthology.orchestra.run.smithi012.stdout: "MULTICLUSTER_CONFIG": { 2024-09-11T00:32:23.858 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "MULTICLUSTER_CONFIG", 2024-09-11T00:32:23.858 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:23.858 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.858 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:23.858 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "{}", 2024-09-11T00:32:23.858 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.858 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.858 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.858 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:23.858 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.858 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.858 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.859 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.859 INFO:teuthology.orchestra.run.smithi012.stdout: "PROMETHEUS_API_HOST": { 2024-09-11T00:32:23.859 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "PROMETHEUS_API_HOST", 2024-09-11T00:32:23.859 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:23.859 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.859 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:23.859 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-09-11T00:32:23.859 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.859 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.859 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.859 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:23.859 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.859 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.859 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.860 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.860 INFO:teuthology.orchestra.run.smithi012.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-09-11T00:32:23.860 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-09-11T00:32:23.860 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-09-11T00:32:23.860 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.860 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:23.860 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-09-11T00:32:23.860 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.860 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.860 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.860 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:23.860 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.860 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.861 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.861 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.861 INFO:teuthology.orchestra.run.smithi012.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-09-11T00:32:23.861 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-09-11T00:32:23.861 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-09-11T00:32:23.861 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.861 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:23.861 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-09-11T00:32:23.861 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.861 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.861 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.861 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:23.862 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.862 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.862 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.862 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.862 INFO:teuthology.orchestra.run.smithi012.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-09-11T00:32:23.862 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-09-11T00:32:23.862 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-09-11T00:32:23.862 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.862 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:23.862 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-09-11T00:32:23.862 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.862 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.863 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.863 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:23.863 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.863 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.863 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.863 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.863 INFO:teuthology.orchestra.run.smithi012.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-09-11T00:32:23.863 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-09-11T00:32:23.863 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-09-11T00:32:23.863 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.863 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:23.863 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-09-11T00:32:23.864 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.864 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.864 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.864 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:23.864 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.864 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.864 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.864 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.864 INFO:teuthology.orchestra.run.smithi012.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-09-11T00:32:23.864 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-09-11T00:32:23.864 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-09-11T00:32:23.864 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.864 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:23.865 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-09-11T00:32:23.865 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.865 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.865 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.865 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:23.865 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.865 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.865 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.865 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.865 INFO:teuthology.orchestra.run.smithi012.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-09-11T00:32:23.865 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-09-11T00:32:23.865 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-09-11T00:32:23.866 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.866 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:23.866 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-09-11T00:32:23.866 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.866 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.866 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.866 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:23.866 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.866 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.866 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.866 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.866 INFO:teuthology.orchestra.run.smithi012.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-09-11T00:32:23.866 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-09-11T00:32:23.867 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-09-11T00:32:23.867 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.867 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:23.867 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-09-11T00:32:23.867 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.867 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.867 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.867 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:23.867 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.867 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.867 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.867 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.868 INFO:teuthology.orchestra.run.smithi012.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-09-11T00:32:23.868 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-09-11T00:32:23.868 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-09-11T00:32:23.868 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.868 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:23.868 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-09-11T00:32:23.868 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.868 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.868 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.868 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:23.868 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.868 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.868 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.868 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.869 INFO:teuthology.orchestra.run.smithi012.stdout: "PWD_POLICY_ENABLED": { 2024-09-11T00:32:23.869 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "PWD_POLICY_ENABLED", 2024-09-11T00:32:23.869 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-09-11T00:32:23.869 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.869 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:23.869 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-09-11T00:32:23.869 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.869 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.869 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.869 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:23.869 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.869 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.869 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.870 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.870 INFO:teuthology.orchestra.run.smithi012.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-09-11T00:32:23.870 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-09-11T00:32:23.870 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:23.870 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.870 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:23.870 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-09-11T00:32:23.870 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.870 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.870 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.870 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:23.870 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.870 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.870 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.871 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.871 INFO:teuthology.orchestra.run.smithi012.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-09-11T00:32:23.871 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-09-11T00:32:23.871 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-09-11T00:32:23.871 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.871 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:23.871 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "10", 2024-09-11T00:32:23.871 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.871 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.871 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.871 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:23.871 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.871 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.871 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.872 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.872 INFO:teuthology.orchestra.run.smithi012.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-09-11T00:32:23.872 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-09-11T00:32:23.872 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-09-11T00:32:23.872 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.872 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:23.872 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "8", 2024-09-11T00:32:23.872 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.872 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.872 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.872 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:23.872 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.872 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.873 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.873 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.873 INFO:teuthology.orchestra.run.smithi012.stdout: "REST_REQUESTS_TIMEOUT": { 2024-09-11T00:32:23.873 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-09-11T00:32:23.873 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-09-11T00:32:23.873 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.873 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:23.876 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "45", 2024-09-11T00:32:23.876 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.876 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.876 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.876 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:23.876 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.877 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.877 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.877 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.877 INFO:teuthology.orchestra.run.smithi012.stdout: "RGW_API_ACCESS_KEY": { 2024-09-11T00:32:23.877 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "RGW_API_ACCESS_KEY", 2024-09-11T00:32:23.877 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:23.877 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.877 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:23.877 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-09-11T00:32:23.877 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.877 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.877 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.877 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:23.877 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.878 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.878 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.878 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.878 INFO:teuthology.orchestra.run.smithi012.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-09-11T00:32:23.878 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-09-11T00:32:23.878 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:23.878 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.878 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:23.878 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "admin", 2024-09-11T00:32:23.878 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.878 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.878 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.878 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:23.879 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.879 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.879 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.879 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.879 INFO:teuthology.orchestra.run.smithi012.stdout: "RGW_API_SECRET_KEY": { 2024-09-11T00:32:23.879 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "RGW_API_SECRET_KEY", 2024-09-11T00:32:23.879 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:23.879 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.879 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:23.879 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-09-11T00:32:23.879 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.879 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.879 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.880 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:23.880 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.880 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.880 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.880 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.880 INFO:teuthology.orchestra.run.smithi012.stdout: "RGW_API_SSL_VERIFY": { 2024-09-11T00:32:23.880 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "RGW_API_SSL_VERIFY", 2024-09-11T00:32:23.880 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-09-11T00:32:23.880 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.880 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:23.880 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-09-11T00:32:23.880 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.881 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.881 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.881 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:23.881 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.881 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.881 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.881 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.881 INFO:teuthology.orchestra.run.smithi012.stdout: "UNSAFE_TLS_v1_2": { 2024-09-11T00:32:23.881 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "UNSAFE_TLS_v1_2", 2024-09-11T00:32:23.881 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-09-11T00:32:23.881 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.881 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:23.882 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-09-11T00:32:23.882 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.882 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.882 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.882 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:23.882 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.882 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.882 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.882 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.883 INFO:teuthology.orchestra.run.smithi012.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-09-11T00:32:23.883 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-09-11T00:32:23.883 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-09-11T00:32:23.883 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.883 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:23.883 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "0", 2024-09-11T00:32:23.883 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.883 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.883 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.883 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:23.883 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.883 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.883 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.884 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.884 INFO:teuthology.orchestra.run.smithi012.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-09-11T00:32:23.884 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-09-11T00:32:23.884 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-09-11T00:32:23.884 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.884 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:23.884 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "10", 2024-09-11T00:32:23.884 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.884 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.884 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.884 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:23.884 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.884 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.885 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.885 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.885 INFO:teuthology.orchestra.run.smithi012.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-09-11T00:32:23.885 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-09-11T00:32:23.885 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-09-11T00:32:23.885 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.885 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:23.885 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "5", 2024-09-11T00:32:23.885 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.885 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.885 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.885 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:23.886 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.886 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.886 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.886 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.886 INFO:teuthology.orchestra.run.smithi012.stdout: "cross_origin_url": { 2024-09-11T00:32:23.886 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "cross_origin_url", 2024-09-11T00:32:23.886 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:23.886 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.886 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:23.886 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-09-11T00:32:23.886 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.887 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.887 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.887 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:23.887 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.887 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.887 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.887 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.887 INFO:teuthology.orchestra.run.smithi012.stdout: "crt_file": { 2024-09-11T00:32:23.887 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "crt_file", 2024-09-11T00:32:23.887 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:23.887 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.887 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:23.887 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-09-11T00:32:23.888 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.888 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.888 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.888 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:23.888 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.888 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.888 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.888 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.888 INFO:teuthology.orchestra.run.smithi012.stdout: "debug": { 2024-09-11T00:32:23.888 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "debug", 2024-09-11T00:32:23.888 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-09-11T00:32:23.888 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.888 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:23.889 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-09-11T00:32:23.889 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.889 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.889 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.889 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Enable/disable debug options", 2024-09-11T00:32:23.889 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.889 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.889 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.889 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.889 INFO:teuthology.orchestra.run.smithi012.stdout: "jwt_token_ttl": { 2024-09-11T00:32:23.889 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "jwt_token_ttl", 2024-09-11T00:32:23.889 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-09-11T00:32:23.889 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.890 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:23.890 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "28800", 2024-09-11T00:32:23.890 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.890 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.890 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.890 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:23.890 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.890 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.890 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.890 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.890 INFO:teuthology.orchestra.run.smithi012.stdout: "key_file": { 2024-09-11T00:32:23.890 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "key_file", 2024-09-11T00:32:23.890 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:23.890 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.891 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:23.891 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-09-11T00:32:23.891 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.891 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.891 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.891 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:23.891 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.891 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.891 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.891 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.891 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-09-11T00:32:23.891 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-09-11T00:32:23.891 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:23.891 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.892 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:23.892 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-09-11T00:32:23.892 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.892 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.892 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-09-11T00:32:23.892 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-09-11T00:32:23.892 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-09-11T00:32:23.892 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-09-11T00:32:23.892 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-09-11T00:32:23.892 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-09-11T00:32:23.892 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-09-11T00:32:23.892 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-09-11T00:32:23.892 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:23.892 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.893 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.893 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.893 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.893 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-09-11T00:32:23.893 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-09-11T00:32:23.893 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-09-11T00:32:23.893 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.893 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:23.893 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-09-11T00:32:23.893 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.893 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.893 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.893 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:23.894 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.894 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.894 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.894 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.894 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-09-11T00:32:23.894 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-09-11T00:32:23.894 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:23.894 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.894 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:23.894 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-09-11T00:32:23.894 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.894 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.894 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-09-11T00:32:23.894 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-09-11T00:32:23.894 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-09-11T00:32:23.895 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-09-11T00:32:23.895 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-09-11T00:32:23.895 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-09-11T00:32:23.895 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-09-11T00:32:23.895 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-09-11T00:32:23.895 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:23.895 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.895 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.895 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.895 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.895 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-09-11T00:32:23.895 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-09-11T00:32:23.895 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-09-11T00:32:23.896 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.896 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:23.896 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-09-11T00:32:23.896 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.896 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.896 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.896 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:23.896 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.896 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.896 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.896 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.896 INFO:teuthology.orchestra.run.smithi012.stdout: "motd": { 2024-09-11T00:32:23.896 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "motd", 2024-09-11T00:32:23.896 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:23.897 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.897 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:23.897 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-09-11T00:32:23.897 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.897 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.897 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.897 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "The message of the day", 2024-09-11T00:32:23.897 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.897 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.897 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.897 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.897 INFO:teuthology.orchestra.run.smithi012.stdout: "redirect_resolve_ip_addr": { 2024-09-11T00:32:23.898 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "redirect_resolve_ip_addr", 2024-09-11T00:32:23.898 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-09-11T00:32:23.898 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.898 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:23.898 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-09-11T00:32:23.898 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.898 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.898 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.898 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:23.898 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.898 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.898 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.899 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.899 INFO:teuthology.orchestra.run.smithi012.stdout: "server_addr": { 2024-09-11T00:32:23.899 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "server_addr", 2024-09-11T00:32:23.899 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:23.899 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.899 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:23.899 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "::", 2024-09-11T00:32:23.899 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.899 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.899 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.899 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:23.899 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.900 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.900 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.900 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.900 INFO:teuthology.orchestra.run.smithi012.stdout: "server_port": { 2024-09-11T00:32:23.900 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "server_port", 2024-09-11T00:32:23.900 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-09-11T00:32:23.900 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.900 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:23.900 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "8080", 2024-09-11T00:32:23.900 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.900 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.900 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.900 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:23.901 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.901 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.901 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.901 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.901 INFO:teuthology.orchestra.run.smithi012.stdout: "sqlite3_killpoint": { 2024-09-11T00:32:23.901 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "sqlite3_killpoint", 2024-09-11T00:32:23.901 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-09-11T00:32:23.901 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "dev", 2024-09-11T00:32:23.901 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:23.901 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "0", 2024-09-11T00:32:23.901 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.901 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.901 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.902 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:23.902 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.902 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.902 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.902 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.902 INFO:teuthology.orchestra.run.smithi012.stdout: "ssl": { 2024-09-11T00:32:23.902 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "ssl", 2024-09-11T00:32:23.902 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-09-11T00:32:23.902 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.902 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:23.902 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-09-11T00:32:23.902 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.902 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.903 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.903 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:23.903 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.903 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.903 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.903 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.903 INFO:teuthology.orchestra.run.smithi012.stdout: "ssl_server_port": { 2024-09-11T00:32:23.903 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "ssl_server_port", 2024-09-11T00:32:23.903 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-09-11T00:32:23.903 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.903 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:23.903 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "8443", 2024-09-11T00:32:23.904 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.904 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.904 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.904 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:23.904 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.904 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.904 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.904 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.904 INFO:teuthology.orchestra.run.smithi012.stdout: "standby_behaviour": { 2024-09-11T00:32:23.904 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "standby_behaviour", 2024-09-11T00:32:23.904 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:23.904 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.904 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:23.904 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "redirect", 2024-09-11T00:32:23.905 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.905 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.905 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-09-11T00:32:23.905 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-09-11T00:32:23.905 INFO:teuthology.orchestra.run.smithi012.stdout: "redirect" 2024-09-11T00:32:23.905 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-09-11T00:32:23.905 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:23.905 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.905 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.905 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.905 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.905 INFO:teuthology.orchestra.run.smithi012.stdout: "standby_error_status_code": { 2024-09-11T00:32:23.905 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "standby_error_status_code", 2024-09-11T00:32:23.906 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-09-11T00:32:23.906 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.906 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:23.906 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "500", 2024-09-11T00:32:23.906 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "400", 2024-09-11T00:32:23.906 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "599", 2024-09-11T00:32:23.906 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.906 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:23.906 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.906 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.906 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.906 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.907 INFO:teuthology.orchestra.run.smithi012.stdout: "url_prefix": { 2024-09-11T00:32:23.907 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "url_prefix", 2024-09-11T00:32:23.907 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:23.907 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.907 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:23.907 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-09-11T00:32:23.907 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.907 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.907 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.907 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:23.907 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.907 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.907 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.907 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:32:23.908 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:32:23.908 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.908 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-11T00:32:23.908 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "devicehealth", 2024-09-11T00:32:23.908 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-09-11T00:32:23.908 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-09-11T00:32:23.908 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-09-11T00:32:23.908 INFO:teuthology.orchestra.run.smithi012.stdout: "enable_monitoring": { 2024-09-11T00:32:23.908 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "enable_monitoring", 2024-09-11T00:32:23.908 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-09-11T00:32:23.908 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.908 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:23.908 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-09-11T00:32:23.909 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.909 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.909 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.909 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "monitor device health metrics", 2024-09-11T00:32:23.909 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.909 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.909 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.909 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.909 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-09-11T00:32:23.909 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-09-11T00:32:23.909 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:23.909 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.909 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:23.910 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-09-11T00:32:23.910 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.910 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.910 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-09-11T00:32:23.910 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-09-11T00:32:23.910 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-09-11T00:32:23.910 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-09-11T00:32:23.910 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-09-11T00:32:23.910 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-09-11T00:32:23.910 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-09-11T00:32:23.910 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-09-11T00:32:23.911 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:23.911 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.911 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.911 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.911 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.911 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-09-11T00:32:23.911 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-09-11T00:32:23.911 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-09-11T00:32:23.911 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.911 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:23.911 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-09-11T00:32:23.911 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.912 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.912 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.912 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:23.912 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.912 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.912 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.912 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.912 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-09-11T00:32:23.912 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-09-11T00:32:23.912 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:23.912 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.912 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:23.912 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-09-11T00:32:23.912 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.913 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.913 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-09-11T00:32:23.913 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-09-11T00:32:23.913 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-09-11T00:32:23.913 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-09-11T00:32:23.913 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-09-11T00:32:23.913 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-09-11T00:32:23.913 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-09-11T00:32:23.913 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-09-11T00:32:23.913 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:23.913 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.913 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.913 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.914 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.914 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-09-11T00:32:23.914 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-09-11T00:32:23.914 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-09-11T00:32:23.914 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.914 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:23.914 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-09-11T00:32:23.914 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.914 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.914 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.914 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:23.915 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.915 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.915 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.915 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.915 INFO:teuthology.orchestra.run.smithi012.stdout: "mark_out_threshold": { 2024-09-11T00:32:23.915 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "mark_out_threshold", 2024-09-11T00:32:23.915 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "secs", 2024-09-11T00:32:23.915 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.915 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:23.915 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "2419200", 2024-09-11T00:32:23.915 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.915 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.915 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.916 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-09-11T00:32:23.916 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.916 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.916 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.916 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.916 INFO:teuthology.orchestra.run.smithi012.stdout: "pool_name": { 2024-09-11T00:32:23.916 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "pool_name", 2024-09-11T00:32:23.916 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:23.916 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.916 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:23.916 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "device_health_metrics", 2024-09-11T00:32:23.916 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.917 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.917 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.917 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "name of pool in which to store device health metrics", 2024-09-11T00:32:23.917 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.917 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.917 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.917 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.917 INFO:teuthology.orchestra.run.smithi012.stdout: "retention_period": { 2024-09-11T00:32:23.917 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "retention_period", 2024-09-11T00:32:23.917 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "secs", 2024-09-11T00:32:23.917 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.917 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:23.917 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "15552000", 2024-09-11T00:32:23.918 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.918 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.918 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.918 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "how long to retain device health metrics", 2024-09-11T00:32:23.918 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.918 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.918 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.918 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.918 INFO:teuthology.orchestra.run.smithi012.stdout: "scrape_frequency": { 2024-09-11T00:32:23.918 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "scrape_frequency", 2024-09-11T00:32:23.918 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "secs", 2024-09-11T00:32:23.918 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.918 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:23.919 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "86400", 2024-09-11T00:32:23.919 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.919 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.919 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.919 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "how frequently to scrape device health metrics", 2024-09-11T00:32:23.919 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.919 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.919 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.919 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.919 INFO:teuthology.orchestra.run.smithi012.stdout: "self_heal": { 2024-09-11T00:32:23.919 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "self_heal", 2024-09-11T00:32:23.919 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-09-11T00:32:23.919 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.920 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:23.920 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-09-11T00:32:23.920 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.920 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.920 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.920 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-09-11T00:32:23.920 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.920 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.920 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.920 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.920 INFO:teuthology.orchestra.run.smithi012.stdout: "sleep_interval": { 2024-09-11T00:32:23.920 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "sleep_interval", 2024-09-11T00:32:23.920 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "secs", 2024-09-11T00:32:23.921 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.921 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:23.921 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "600", 2024-09-11T00:32:23.921 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.921 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.921 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.921 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "how frequently to wake up and check device health", 2024-09-11T00:32:23.921 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.921 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.921 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.921 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.921 INFO:teuthology.orchestra.run.smithi012.stdout: "sqlite3_killpoint": { 2024-09-11T00:32:23.921 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "sqlite3_killpoint", 2024-09-11T00:32:23.922 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-09-11T00:32:23.922 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "dev", 2024-09-11T00:32:23.922 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:23.922 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "0", 2024-09-11T00:32:23.922 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.922 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.922 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.922 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:23.922 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.922 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.922 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.922 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.922 INFO:teuthology.orchestra.run.smithi012.stdout: "warn_threshold": { 2024-09-11T00:32:23.923 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "warn_threshold", 2024-09-11T00:32:23.923 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "secs", 2024-09-11T00:32:23.923 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.923 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:23.923 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "7257600", 2024-09-11T00:32:23.923 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.923 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.923 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.923 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-09-11T00:32:23.923 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.923 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.924 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.924 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:32:23.924 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:32:23.924 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.924 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-11T00:32:23.924 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "diskprediction_local", 2024-09-11T00:32:23.924 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-09-11T00:32:23.924 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-09-11T00:32:23.924 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-09-11T00:32:23.924 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-09-11T00:32:23.924 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-09-11T00:32:23.924 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:23.924 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.925 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:23.925 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-09-11T00:32:23.925 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.925 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.925 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-09-11T00:32:23.925 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-09-11T00:32:23.925 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-09-11T00:32:23.925 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-09-11T00:32:23.925 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-09-11T00:32:23.925 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-09-11T00:32:23.925 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-09-11T00:32:23.926 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-09-11T00:32:23.926 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:23.926 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.926 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.926 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.926 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.926 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-09-11T00:32:23.926 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-09-11T00:32:23.926 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-09-11T00:32:23.926 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.926 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:23.926 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-09-11T00:32:23.926 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.927 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.927 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.927 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:23.927 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.927 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.927 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.927 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.927 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-09-11T00:32:23.927 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-09-11T00:32:23.927 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:23.927 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.927 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:23.927 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-09-11T00:32:23.928 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.928 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.928 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-09-11T00:32:23.928 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-09-11T00:32:23.928 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-09-11T00:32:23.928 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-09-11T00:32:23.928 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-09-11T00:32:23.928 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-09-11T00:32:23.928 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-09-11T00:32:23.928 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-09-11T00:32:23.928 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:23.928 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.928 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.928 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.929 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.929 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-09-11T00:32:23.929 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-09-11T00:32:23.929 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-09-11T00:32:23.929 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.929 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:23.929 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-09-11T00:32:23.929 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.929 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.929 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.929 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:23.929 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.930 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.930 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.930 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.930 INFO:teuthology.orchestra.run.smithi012.stdout: "predict_interval": { 2024-09-11T00:32:23.930 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "predict_interval", 2024-09-11T00:32:23.930 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:23.930 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.930 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:23.930 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "86400", 2024-09-11T00:32:23.930 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.930 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.930 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.930 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:23.931 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.931 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.931 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.931 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.931 INFO:teuthology.orchestra.run.smithi012.stdout: "predictor_model": { 2024-09-11T00:32:23.931 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "predictor_model", 2024-09-11T00:32:23.931 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:23.931 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.931 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:23.931 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "prophetstor", 2024-09-11T00:32:23.931 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.931 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.931 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.932 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:23.932 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.932 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.932 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.932 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.932 INFO:teuthology.orchestra.run.smithi012.stdout: "sleep_interval": { 2024-09-11T00:32:23.932 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "sleep_interval", 2024-09-11T00:32:23.932 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:23.932 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.932 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:23.932 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "600", 2024-09-11T00:32:23.932 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.932 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.933 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.933 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:23.933 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.933 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.933 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.933 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.933 INFO:teuthology.orchestra.run.smithi012.stdout: "sqlite3_killpoint": { 2024-09-11T00:32:23.933 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "sqlite3_killpoint", 2024-09-11T00:32:23.933 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-09-11T00:32:23.933 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "dev", 2024-09-11T00:32:23.933 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:23.933 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "0", 2024-09-11T00:32:23.934 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.934 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.934 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.934 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:23.934 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.934 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.934 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.934 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:32:23.934 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:32:23.934 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.934 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-11T00:32:23.934 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "influx", 2024-09-11T00:32:23.934 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": false, 2024-09-11T00:32:23.935 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "influxdb python module not found", 2024-09-11T00:32:23.935 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-09-11T00:32:23.935 INFO:teuthology.orchestra.run.smithi012.stdout: "batch_size": { 2024-09-11T00:32:23.935 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "batch_size", 2024-09-11T00:32:23.935 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-09-11T00:32:23.935 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.935 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:23.935 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "5000", 2024-09-11T00:32:23.935 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.935 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.935 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.936 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-09-11T00:32:23.936 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.936 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.936 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.936 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.936 INFO:teuthology.orchestra.run.smithi012.stdout: "database": { 2024-09-11T00:32:23.936 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "database", 2024-09-11T00:32:23.936 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:23.936 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.936 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:23.936 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "ceph", 2024-09-11T00:32:23.936 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.937 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.937 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.937 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-09-11T00:32:23.937 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.937 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.937 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.937 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.937 INFO:teuthology.orchestra.run.smithi012.stdout: "hostname": { 2024-09-11T00:32:23.937 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "hostname", 2024-09-11T00:32:23.937 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:23.937 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.937 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:23.938 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-09-11T00:32:23.938 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.938 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.938 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.938 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "InfluxDB server hostname", 2024-09-11T00:32:23.938 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.938 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.938 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.938 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.938 INFO:teuthology.orchestra.run.smithi012.stdout: "interval": { 2024-09-11T00:32:23.938 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "interval", 2024-09-11T00:32:23.938 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "secs", 2024-09-11T00:32:23.938 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.939 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:23.939 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "30", 2024-09-11T00:32:23.939 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "5", 2024-09-11T00:32:23.939 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.939 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.939 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-09-11T00:32:23.939 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.939 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.939 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.939 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.939 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-09-11T00:32:23.939 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-09-11T00:32:23.939 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:23.940 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.940 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:23.940 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-09-11T00:32:23.940 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.940 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.940 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-09-11T00:32:23.940 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-09-11T00:32:23.940 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-09-11T00:32:23.940 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-09-11T00:32:23.940 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-09-11T00:32:23.940 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-09-11T00:32:23.940 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-09-11T00:32:23.941 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-09-11T00:32:23.941 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:23.941 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.941 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.941 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.941 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.941 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-09-11T00:32:23.941 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-09-11T00:32:23.941 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-09-11T00:32:23.941 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.941 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:23.941 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-09-11T00:32:23.941 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.941 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.942 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.942 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:23.942 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.942 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.942 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.942 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.942 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-09-11T00:32:23.942 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-09-11T00:32:23.942 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:23.942 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.942 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:23.942 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-09-11T00:32:23.942 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.942 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.943 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-09-11T00:32:23.943 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-09-11T00:32:23.943 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-09-11T00:32:23.943 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-09-11T00:32:23.943 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-09-11T00:32:23.943 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-09-11T00:32:23.943 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-09-11T00:32:23.943 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-09-11T00:32:23.943 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:23.943 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.943 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.943 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.943 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.944 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-09-11T00:32:23.944 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-09-11T00:32:23.944 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-09-11T00:32:23.944 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.944 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:23.944 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-09-11T00:32:23.944 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.944 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.944 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.944 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:23.944 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.944 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.944 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.945 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.945 INFO:teuthology.orchestra.run.smithi012.stdout: "password": { 2024-09-11T00:32:23.945 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "password", 2024-09-11T00:32:23.945 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:23.945 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.945 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:23.945 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-09-11T00:32:23.945 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.945 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.945 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.945 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "password of InfluxDB server user", 2024-09-11T00:32:23.945 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.945 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.946 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.946 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.946 INFO:teuthology.orchestra.run.smithi012.stdout: "port": { 2024-09-11T00:32:23.946 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "port", 2024-09-11T00:32:23.946 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-09-11T00:32:23.946 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.946 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:23.946 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "8086", 2024-09-11T00:32:23.946 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.946 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.946 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.946 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "InfluxDB server port", 2024-09-11T00:32:23.947 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.947 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.947 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.947 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.947 INFO:teuthology.orchestra.run.smithi012.stdout: "sqlite3_killpoint": { 2024-09-11T00:32:23.947 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "sqlite3_killpoint", 2024-09-11T00:32:23.947 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-09-11T00:32:23.947 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "dev", 2024-09-11T00:32:23.947 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:23.947 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "0", 2024-09-11T00:32:23.947 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.948 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.948 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.948 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:23.948 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.948 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.948 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.948 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.948 INFO:teuthology.orchestra.run.smithi012.stdout: "ssl": { 2024-09-11T00:32:23.948 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "ssl", 2024-09-11T00:32:23.948 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:23.948 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.948 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:23.948 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "false", 2024-09-11T00:32:23.949 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.949 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.949 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.949 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-09-11T00:32:23.949 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.949 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.949 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.949 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.949 INFO:teuthology.orchestra.run.smithi012.stdout: "threads": { 2024-09-11T00:32:23.949 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "threads", 2024-09-11T00:32:23.949 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-09-11T00:32:23.949 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.950 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:23.950 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "5", 2024-09-11T00:32:23.950 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "1", 2024-09-11T00:32:23.950 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "32", 2024-09-11T00:32:23.950 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.950 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-09-11T00:32:23.950 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.950 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.950 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.950 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.950 INFO:teuthology.orchestra.run.smithi012.stdout: "username": { 2024-09-11T00:32:23.950 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "username", 2024-09-11T00:32:23.951 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:23.951 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.951 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:23.951 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-09-11T00:32:23.951 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.951 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.951 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.951 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "username of InfluxDB server user", 2024-09-11T00:32:23.951 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.951 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.951 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.951 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.951 INFO:teuthology.orchestra.run.smithi012.stdout: "verify_ssl": { 2024-09-11T00:32:23.952 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "verify_ssl", 2024-09-11T00:32:23.952 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:23.952 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.952 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:23.952 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "true", 2024-09-11T00:32:23.952 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.952 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.952 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.952 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-09-11T00:32:23.952 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.952 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.952 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.952 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:32:23.952 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:32:23.953 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.953 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-11T00:32:23.953 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "insights", 2024-09-11T00:32:23.953 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-09-11T00:32:23.953 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-09-11T00:32:23.953 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-09-11T00:32:23.953 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-09-11T00:32:23.953 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-09-11T00:32:23.953 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:23.953 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.953 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:23.953 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-09-11T00:32:23.953 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.954 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.954 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-09-11T00:32:23.954 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-09-11T00:32:23.954 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-09-11T00:32:23.954 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-09-11T00:32:23.954 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-09-11T00:32:23.954 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-09-11T00:32:23.954 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-09-11T00:32:23.954 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-09-11T00:32:23.954 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:23.954 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.954 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.954 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.954 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.955 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-09-11T00:32:23.955 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-09-11T00:32:23.955 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-09-11T00:32:23.955 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.955 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:23.955 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-09-11T00:32:23.955 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.955 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.955 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.955 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:23.955 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.955 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.955 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.956 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.956 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-09-11T00:32:23.956 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-09-11T00:32:23.956 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:23.956 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.956 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:23.956 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-09-11T00:32:23.956 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.956 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.956 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-09-11T00:32:23.956 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-09-11T00:32:23.957 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-09-11T00:32:23.957 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-09-11T00:32:23.957 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-09-11T00:32:23.957 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-09-11T00:32:23.957 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-09-11T00:32:23.957 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-09-11T00:32:23.957 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:23.957 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.957 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.957 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.957 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.957 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-09-11T00:32:23.957 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-09-11T00:32:23.958 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-09-11T00:32:23.958 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.958 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:23.958 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-09-11T00:32:23.958 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.958 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.958 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.958 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:23.958 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.958 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.958 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.958 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.958 INFO:teuthology.orchestra.run.smithi012.stdout: "sqlite3_killpoint": { 2024-09-11T00:32:23.959 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "sqlite3_killpoint", 2024-09-11T00:32:23.959 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-09-11T00:32:23.959 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "dev", 2024-09-11T00:32:23.959 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:23.959 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "0", 2024-09-11T00:32:23.959 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.959 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.959 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.959 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:23.959 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.959 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.959 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.959 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:32:23.959 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:32:23.960 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.960 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-11T00:32:23.960 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "iostat", 2024-09-11T00:32:23.960 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-09-11T00:32:23.960 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-09-11T00:32:23.960 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-09-11T00:32:23.960 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-09-11T00:32:23.960 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-09-11T00:32:23.960 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:23.960 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.960 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:23.960 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-09-11T00:32:23.960 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.960 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.961 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-09-11T00:32:23.961 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-09-11T00:32:23.961 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-09-11T00:32:23.961 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-09-11T00:32:23.961 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-09-11T00:32:23.961 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-09-11T00:32:23.961 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-09-11T00:32:23.961 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-09-11T00:32:23.961 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:23.961 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.961 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.961 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.961 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.962 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-09-11T00:32:23.962 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-09-11T00:32:23.962 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-09-11T00:32:23.962 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.962 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:23.962 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-09-11T00:32:23.962 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.962 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.962 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.962 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:23.962 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.962 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.962 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.963 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.963 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-09-11T00:32:23.963 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-09-11T00:32:23.963 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:23.963 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.963 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:23.963 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-09-11T00:32:23.963 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.963 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.963 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-09-11T00:32:23.963 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-09-11T00:32:23.963 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-09-11T00:32:23.963 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-09-11T00:32:23.963 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-09-11T00:32:23.964 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-09-11T00:32:23.964 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-09-11T00:32:23.964 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-09-11T00:32:23.964 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:23.964 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.964 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.964 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.964 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.964 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-09-11T00:32:23.964 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-09-11T00:32:23.964 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-09-11T00:32:23.964 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.964 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:23.965 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-09-11T00:32:23.965 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.965 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.965 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.965 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:23.965 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.965 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.965 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.965 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.965 INFO:teuthology.orchestra.run.smithi012.stdout: "sqlite3_killpoint": { 2024-09-11T00:32:23.965 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "sqlite3_killpoint", 2024-09-11T00:32:23.965 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-09-11T00:32:23.965 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "dev", 2024-09-11T00:32:23.966 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:23.966 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "0", 2024-09-11T00:32:23.966 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.966 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.966 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.966 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:23.966 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.966 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.966 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.966 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:32:23.966 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:32:23.966 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.966 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-11T00:32:23.966 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "k8sevents", 2024-09-11T00:32:23.967 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-09-11T00:32:23.967 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-09-11T00:32:23.967 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-09-11T00:32:23.967 INFO:teuthology.orchestra.run.smithi012.stdout: "ceph_event_retention_days": { 2024-09-11T00:32:23.967 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "ceph_event_retention_days", 2024-09-11T00:32:23.967 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-09-11T00:32:23.967 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.967 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:23.967 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "7", 2024-09-11T00:32:23.967 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.967 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.967 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.967 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Days to hold ceph event information within local cache", 2024-09-11T00:32:23.968 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.968 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.968 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.968 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.968 INFO:teuthology.orchestra.run.smithi012.stdout: "config_check_secs": { 2024-09-11T00:32:23.968 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "config_check_secs", 2024-09-11T00:32:23.968 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-09-11T00:32:23.968 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.968 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:23.968 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "10", 2024-09-11T00:32:23.968 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "10", 2024-09-11T00:32:23.968 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.968 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.969 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-09-11T00:32:23.969 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.969 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.969 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.969 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.969 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-09-11T00:32:23.969 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-09-11T00:32:23.969 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:23.969 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.969 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:23.969 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-09-11T00:32:23.970 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.970 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.970 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-09-11T00:32:23.970 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-09-11T00:32:23.970 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-09-11T00:32:23.970 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-09-11T00:32:23.970 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-09-11T00:32:23.970 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-09-11T00:32:23.970 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-09-11T00:32:23.970 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-09-11T00:32:23.970 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:23.970 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.971 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.971 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.971 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.971 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-09-11T00:32:23.971 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-09-11T00:32:23.971 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-09-11T00:32:23.971 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.971 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:23.971 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-09-11T00:32:23.971 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.971 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.971 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.971 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:23.971 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.972 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.972 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.972 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.972 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-09-11T00:32:23.972 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-09-11T00:32:23.972 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:23.972 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.972 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:23.972 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-09-11T00:32:23.972 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.972 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.972 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-09-11T00:32:23.973 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-09-11T00:32:23.973 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-09-11T00:32:23.973 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-09-11T00:32:23.973 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-09-11T00:32:23.973 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-09-11T00:32:23.973 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-09-11T00:32:23.973 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-09-11T00:32:23.973 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:23.973 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.973 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.973 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.973 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.973 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-09-11T00:32:23.974 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-09-11T00:32:23.974 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-09-11T00:32:23.974 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.974 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:23.974 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-09-11T00:32:23.974 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.974 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.974 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.974 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:23.974 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.974 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.974 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.975 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.975 INFO:teuthology.orchestra.run.smithi012.stdout: "sqlite3_killpoint": { 2024-09-11T00:32:23.975 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "sqlite3_killpoint", 2024-09-11T00:32:23.975 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-09-11T00:32:23.975 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "dev", 2024-09-11T00:32:23.975 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:23.975 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "0", 2024-09-11T00:32:23.975 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.975 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.975 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.975 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:23.975 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.976 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.976 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.976 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:32:23.976 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:32:23.976 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.976 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-11T00:32:23.976 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "localpool", 2024-09-11T00:32:23.976 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-09-11T00:32:23.976 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-09-11T00:32:23.976 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-09-11T00:32:23.976 INFO:teuthology.orchestra.run.smithi012.stdout: "failure_domain": { 2024-09-11T00:32:23.976 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "failure_domain", 2024-09-11T00:32:23.976 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:23.977 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.977 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:23.977 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "host", 2024-09-11T00:32:23.977 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.977 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.977 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.977 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "failure domain for any created local pool", 2024-09-11T00:32:23.977 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-09-11T00:32:23.977 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.977 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.977 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.977 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-09-11T00:32:23.977 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-09-11T00:32:23.978 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:23.978 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.978 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:23.978 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-09-11T00:32:23.978 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.978 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.978 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-09-11T00:32:23.978 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-09-11T00:32:23.978 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-09-11T00:32:23.978 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-09-11T00:32:23.978 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-09-11T00:32:23.979 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-09-11T00:32:23.979 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-09-11T00:32:23.979 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-09-11T00:32:23.979 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:23.979 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.979 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.979 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.979 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.979 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-09-11T00:32:23.979 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-09-11T00:32:23.979 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-09-11T00:32:23.979 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.979 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:23.979 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-09-11T00:32:23.980 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.980 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.980 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.980 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:23.980 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.980 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.980 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.980 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.980 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-09-11T00:32:23.980 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-09-11T00:32:23.980 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:23.980 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.980 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:23.981 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-09-11T00:32:23.981 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.981 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.981 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-09-11T00:32:23.981 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-09-11T00:32:23.981 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-09-11T00:32:23.981 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-09-11T00:32:23.981 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-09-11T00:32:23.981 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-09-11T00:32:23.981 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-09-11T00:32:23.981 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-09-11T00:32:23.981 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:23.982 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.982 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.982 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.982 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.982 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-09-11T00:32:23.982 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-09-11T00:32:23.982 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-09-11T00:32:23.982 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.982 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:23.982 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-09-11T00:32:23.983 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.983 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.983 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.983 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:23.983 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.983 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.983 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.983 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.983 INFO:teuthology.orchestra.run.smithi012.stdout: "min_size": { 2024-09-11T00:32:23.983 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "min_size", 2024-09-11T00:32:23.983 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-09-11T00:32:23.983 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.984 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:23.984 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-09-11T00:32:23.984 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.984 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.984 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.984 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "default min_size for any created local pool", 2024-09-11T00:32:23.984 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-09-11T00:32:23.984 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.984 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.984 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.984 INFO:teuthology.orchestra.run.smithi012.stdout: "num_rep": { 2024-09-11T00:32:23.984 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "num_rep", 2024-09-11T00:32:23.984 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-09-11T00:32:23.985 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.985 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:23.985 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "3", 2024-09-11T00:32:23.985 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.985 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.985 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.985 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "default replica count for any created local pool", 2024-09-11T00:32:23.985 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.985 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.985 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.985 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.985 INFO:teuthology.orchestra.run.smithi012.stdout: "pg_num": { 2024-09-11T00:32:23.986 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "pg_num", 2024-09-11T00:32:23.986 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-09-11T00:32:23.986 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.986 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:23.986 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "128", 2024-09-11T00:32:23.986 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.986 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.986 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.986 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "default pg_num for any created local pool", 2024-09-11T00:32:23.986 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.986 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.986 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.986 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.987 INFO:teuthology.orchestra.run.smithi012.stdout: "prefix": { 2024-09-11T00:32:23.987 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "prefix", 2024-09-11T00:32:23.987 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:23.987 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.987 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:23.987 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-09-11T00:32:23.987 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.987 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.987 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.987 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "name prefix for any created local pool", 2024-09-11T00:32:23.987 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.987 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.987 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.988 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.988 INFO:teuthology.orchestra.run.smithi012.stdout: "sqlite3_killpoint": { 2024-09-11T00:32:23.988 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "sqlite3_killpoint", 2024-09-11T00:32:23.988 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-09-11T00:32:23.988 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "dev", 2024-09-11T00:32:23.988 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:23.988 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "0", 2024-09-11T00:32:23.988 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.988 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.988 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.988 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:23.988 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.988 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.989 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.989 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.989 INFO:teuthology.orchestra.run.smithi012.stdout: "subtree": { 2024-09-11T00:32:23.989 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "subtree", 2024-09-11T00:32:23.989 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:23.989 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.989 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:23.989 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "rack", 2024-09-11T00:32:23.989 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.989 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.989 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.989 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "CRUSH level for which to create a local pool", 2024-09-11T00:32:23.990 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-09-11T00:32:23.990 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.990 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.990 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:32:23.990 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:32:23.990 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.990 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-11T00:32:23.990 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "mds_autoscaler", 2024-09-11T00:32:23.990 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-09-11T00:32:23.990 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-09-11T00:32:23.990 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-09-11T00:32:23.990 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-09-11T00:32:23.991 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-09-11T00:32:23.991 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:23.991 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.991 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:23.991 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-09-11T00:32:23.991 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.991 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.991 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-09-11T00:32:23.991 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-09-11T00:32:23.991 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-09-11T00:32:23.991 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-09-11T00:32:23.991 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-09-11T00:32:23.992 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-09-11T00:32:23.992 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-09-11T00:32:23.992 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-09-11T00:32:23.992 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:23.992 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.992 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.992 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.992 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.992 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-09-11T00:32:23.992 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-09-11T00:32:23.992 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-09-11T00:32:23.992 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.992 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:23.993 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-09-11T00:32:23.993 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.993 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.993 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.993 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:23.993 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.993 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.993 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.993 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.993 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-09-11T00:32:23.993 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-09-11T00:32:23.993 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:23.994 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.994 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:23.994 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-09-11T00:32:23.994 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.994 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.994 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-09-11T00:32:23.994 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-09-11T00:32:23.994 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-09-11T00:32:23.994 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-09-11T00:32:23.994 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-09-11T00:32:23.994 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-09-11T00:32:23.994 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-09-11T00:32:23.994 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-09-11T00:32:23.995 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:23.995 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.995 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.995 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.995 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.995 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-09-11T00:32:23.995 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-09-11T00:32:23.995 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-09-11T00:32:23.995 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.995 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:23.995 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-09-11T00:32:23.995 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.995 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.996 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.996 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:23.996 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.996 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.996 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.996 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.996 INFO:teuthology.orchestra.run.smithi012.stdout: "sqlite3_killpoint": { 2024-09-11T00:32:23.996 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "sqlite3_killpoint", 2024-09-11T00:32:23.996 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-09-11T00:32:23.996 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "dev", 2024-09-11T00:32:23.996 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:23.996 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "0", 2024-09-11T00:32:23.997 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.997 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.997 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:23.997 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:23.997 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.997 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.997 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.997 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:32:23.997 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:32:23.997 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.997 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-11T00:32:23.997 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "mirroring", 2024-09-11T00:32:23.997 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-09-11T00:32:23.998 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-09-11T00:32:23.998 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-09-11T00:32:23.998 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-09-11T00:32:23.998 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-09-11T00:32:23.998 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:23.998 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:23.998 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:23.998 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-09-11T00:32:23.998 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:23.998 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:23.998 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-09-11T00:32:23.998 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-09-11T00:32:23.998 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-09-11T00:32:23.999 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-09-11T00:32:23.999 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-09-11T00:32:23.999 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-09-11T00:32:23.999 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-09-11T00:32:23.999 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-09-11T00:32:23.999 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:23.999 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:23.999 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:23.999 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:23.999 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:23.999 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-09-11T00:32:23.999 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-09-11T00:32:23.999 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-09-11T00:32:24.000 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.000 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:24.000 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-09-11T00:32:24.000 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.000 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.000 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.000 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.000 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.000 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.000 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.000 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.000 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-09-11T00:32:24.000 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-09-11T00:32:24.001 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:24.001 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.001 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:24.001 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-09-11T00:32:24.001 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.001 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.001 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-09-11T00:32:24.001 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-09-11T00:32:24.001 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-09-11T00:32:24.001 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-09-11T00:32:24.001 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-09-11T00:32:24.001 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-09-11T00:32:24.001 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-09-11T00:32:24.002 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-09-11T00:32:24.002 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.002 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.002 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.002 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.002 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.002 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-09-11T00:32:24.002 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-09-11T00:32:24.002 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-09-11T00:32:24.002 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.002 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:24.002 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-09-11T00:32:24.002 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.003 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.003 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.003 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.003 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.003 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.003 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.003 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.003 INFO:teuthology.orchestra.run.smithi012.stdout: "sqlite3_killpoint": { 2024-09-11T00:32:24.003 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "sqlite3_killpoint", 2024-09-11T00:32:24.003 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-09-11T00:32:24.003 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "dev", 2024-09-11T00:32:24.003 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:24.004 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "0", 2024-09-11T00:32:24.004 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.004 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.004 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.004 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.004 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.004 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.004 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.004 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:32:24.004 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:32:24.004 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.004 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-11T00:32:24.004 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "nfs", 2024-09-11T00:32:24.005 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-09-11T00:32:24.005 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-09-11T00:32:24.005 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-09-11T00:32:24.005 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-09-11T00:32:24.005 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-09-11T00:32:24.005 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:24.005 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.005 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:24.005 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-09-11T00:32:24.005 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.005 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.005 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-09-11T00:32:24.006 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-09-11T00:32:24.006 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-09-11T00:32:24.006 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-09-11T00:32:24.006 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-09-11T00:32:24.006 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-09-11T00:32:24.006 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-09-11T00:32:24.006 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-09-11T00:32:24.006 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.006 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.006 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.006 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.006 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.006 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-09-11T00:32:24.007 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-09-11T00:32:24.007 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-09-11T00:32:24.007 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.007 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:24.007 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-09-11T00:32:24.007 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.007 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.007 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.007 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.007 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.007 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.007 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.008 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.008 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-09-11T00:32:24.008 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-09-11T00:32:24.008 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:24.008 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.008 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:24.008 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-09-11T00:32:24.008 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.008 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.008 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-09-11T00:32:24.008 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-09-11T00:32:24.008 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-09-11T00:32:24.009 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-09-11T00:32:24.009 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-09-11T00:32:24.009 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-09-11T00:32:24.009 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-09-11T00:32:24.009 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-09-11T00:32:24.009 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.009 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.009 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.009 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.009 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.009 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-09-11T00:32:24.009 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-09-11T00:32:24.010 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-09-11T00:32:24.010 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.010 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:24.010 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-09-11T00:32:24.010 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.010 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.010 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.010 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.010 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.010 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.010 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.010 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.010 INFO:teuthology.orchestra.run.smithi012.stdout: "sqlite3_killpoint": { 2024-09-11T00:32:24.010 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "sqlite3_killpoint", 2024-09-11T00:32:24.010 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-09-11T00:32:24.011 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "dev", 2024-09-11T00:32:24.011 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:24.011 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "0", 2024-09-11T00:32:24.011 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.011 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.011 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.011 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.011 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.011 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.011 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.011 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:32:24.011 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:32:24.011 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.011 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-11T00:32:24.012 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "orchestrator", 2024-09-11T00:32:24.012 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-09-11T00:32:24.012 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-09-11T00:32:24.012 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-09-11T00:32:24.012 INFO:teuthology.orchestra.run.smithi012.stdout: "fail_fs": { 2024-09-11T00:32:24.012 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "fail_fs", 2024-09-11T00:32:24.012 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-09-11T00:32:24.012 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.012 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:24.012 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-09-11T00:32:24.012 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.012 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.012 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.012 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-09-11T00:32:24.013 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.013 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.013 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.013 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.013 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-09-11T00:32:24.013 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-09-11T00:32:24.013 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:24.013 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.013 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:24.013 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-09-11T00:32:24.013 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.013 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.013 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-09-11T00:32:24.014 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-09-11T00:32:24.014 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-09-11T00:32:24.014 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-09-11T00:32:24.014 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-09-11T00:32:24.014 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-09-11T00:32:24.014 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-09-11T00:32:24.014 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-09-11T00:32:24.014 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.014 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.014 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.014 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.014 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.014 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-09-11T00:32:24.014 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-09-11T00:32:24.015 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-09-11T00:32:24.015 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.015 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:24.015 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-09-11T00:32:24.015 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.015 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.015 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.015 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.015 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.015 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.015 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.015 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.015 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-09-11T00:32:24.015 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-09-11T00:32:24.016 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:24.016 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.016 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:24.016 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-09-11T00:32:24.016 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.016 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.016 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-09-11T00:32:24.016 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-09-11T00:32:24.016 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-09-11T00:32:24.016 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-09-11T00:32:24.016 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-09-11T00:32:24.016 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-09-11T00:32:24.017 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-09-11T00:32:24.017 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-09-11T00:32:24.017 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.017 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.017 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.017 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.017 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.017 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-09-11T00:32:24.017 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-09-11T00:32:24.017 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-09-11T00:32:24.017 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.018 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:24.018 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-09-11T00:32:24.018 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.018 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.018 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.018 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.018 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.018 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.018 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.018 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.018 INFO:teuthology.orchestra.run.smithi012.stdout: "orchestrator": { 2024-09-11T00:32:24.018 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "orchestrator", 2024-09-11T00:32:24.018 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:24.018 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.019 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:24.019 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-09-11T00:32:24.019 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.019 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.019 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-09-11T00:32:24.019 INFO:teuthology.orchestra.run.smithi012.stdout: "cephadm", 2024-09-11T00:32:24.019 INFO:teuthology.orchestra.run.smithi012.stdout: "rook", 2024-09-11T00:32:24.019 INFO:teuthology.orchestra.run.smithi012.stdout: "test_orchestrator" 2024-09-11T00:32:24.019 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-09-11T00:32:24.019 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Orchestrator backend", 2024-09-11T00:32:24.019 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.019 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.019 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.019 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.020 INFO:teuthology.orchestra.run.smithi012.stdout: "sqlite3_killpoint": { 2024-09-11T00:32:24.020 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "sqlite3_killpoint", 2024-09-11T00:32:24.020 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-09-11T00:32:24.020 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "dev", 2024-09-11T00:32:24.020 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:24.020 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "0", 2024-09-11T00:32:24.020 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.020 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.020 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.020 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.020 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.020 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.020 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.021 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:32:24.021 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:32:24.021 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.021 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-11T00:32:24.021 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "osd_perf_query", 2024-09-11T00:32:24.021 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-09-11T00:32:24.021 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-09-11T00:32:24.021 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-09-11T00:32:24.021 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-09-11T00:32:24.021 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-09-11T00:32:24.021 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:24.021 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.021 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:24.022 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-09-11T00:32:24.022 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.022 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.022 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-09-11T00:32:24.022 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-09-11T00:32:24.022 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-09-11T00:32:24.022 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-09-11T00:32:24.022 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-09-11T00:32:24.022 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-09-11T00:32:24.022 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-09-11T00:32:24.022 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-09-11T00:32:24.022 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.022 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.022 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.023 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.023 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.023 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-09-11T00:32:24.023 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-09-11T00:32:24.023 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-09-11T00:32:24.023 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.023 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:24.023 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-09-11T00:32:24.023 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.023 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.023 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.023 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.023 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.023 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.024 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.024 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.024 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-09-11T00:32:24.024 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-09-11T00:32:24.024 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:24.024 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.024 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:24.024 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-09-11T00:32:24.024 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.024 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.024 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-09-11T00:32:24.024 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-09-11T00:32:24.024 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-09-11T00:32:24.025 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-09-11T00:32:24.025 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-09-11T00:32:24.025 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-09-11T00:32:24.025 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-09-11T00:32:24.025 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-09-11T00:32:24.025 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.025 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.025 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.025 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.025 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.025 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-09-11T00:32:24.025 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-09-11T00:32:24.025 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-09-11T00:32:24.026 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.026 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:24.026 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-09-11T00:32:24.026 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.026 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.026 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.026 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.026 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.026 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.026 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.026 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.026 INFO:teuthology.orchestra.run.smithi012.stdout: "sqlite3_killpoint": { 2024-09-11T00:32:24.026 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "sqlite3_killpoint", 2024-09-11T00:32:24.027 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-09-11T00:32:24.027 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "dev", 2024-09-11T00:32:24.027 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:24.027 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "0", 2024-09-11T00:32:24.027 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.027 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.027 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.027 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.027 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.027 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.027 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.028 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:32:24.028 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:32:24.028 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.028 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-11T00:32:24.028 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "osd_support", 2024-09-11T00:32:24.028 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-09-11T00:32:24.028 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-09-11T00:32:24.028 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-09-11T00:32:24.028 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-09-11T00:32:24.028 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-09-11T00:32:24.028 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:24.028 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.029 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:24.029 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-09-11T00:32:24.029 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.029 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.029 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-09-11T00:32:24.029 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-09-11T00:32:24.029 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-09-11T00:32:24.029 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-09-11T00:32:24.029 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-09-11T00:32:24.029 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-09-11T00:32:24.029 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-09-11T00:32:24.029 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-09-11T00:32:24.029 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.030 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.030 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.030 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.030 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.030 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-09-11T00:32:24.030 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-09-11T00:32:24.030 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-09-11T00:32:24.030 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.030 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:24.030 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-09-11T00:32:24.030 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.030 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.031 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.031 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.031 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.031 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.031 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.031 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.031 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-09-11T00:32:24.031 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-09-11T00:32:24.032 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:24.032 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.032 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:24.032 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-09-11T00:32:24.032 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.032 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.032 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-09-11T00:32:24.032 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-09-11T00:32:24.032 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-09-11T00:32:24.033 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-09-11T00:32:24.033 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-09-11T00:32:24.033 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-09-11T00:32:24.033 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-09-11T00:32:24.033 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-09-11T00:32:24.033 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.033 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.033 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.033 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.033 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.033 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-09-11T00:32:24.033 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-09-11T00:32:24.034 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-09-11T00:32:24.034 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.034 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:24.034 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-09-11T00:32:24.034 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.034 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.034 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.034 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.034 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.034 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.034 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.034 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.034 INFO:teuthology.orchestra.run.smithi012.stdout: "sqlite3_killpoint": { 2024-09-11T00:32:24.034 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "sqlite3_killpoint", 2024-09-11T00:32:24.035 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-09-11T00:32:24.035 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "dev", 2024-09-11T00:32:24.035 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:24.035 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "0", 2024-09-11T00:32:24.035 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.035 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.035 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.035 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.035 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.035 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.035 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.035 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:32:24.036 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:32:24.036 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.036 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-11T00:32:24.036 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "pg_autoscaler", 2024-09-11T00:32:24.036 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-09-11T00:32:24.036 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-09-11T00:32:24.036 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-09-11T00:32:24.036 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-09-11T00:32:24.036 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-09-11T00:32:24.036 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:24.036 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.037 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:24.037 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-09-11T00:32:24.037 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.037 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.037 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-09-11T00:32:24.037 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-09-11T00:32:24.037 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-09-11T00:32:24.037 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-09-11T00:32:24.037 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-09-11T00:32:24.037 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-09-11T00:32:24.037 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-09-11T00:32:24.038 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-09-11T00:32:24.038 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.038 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.038 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.038 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.038 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.038 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-09-11T00:32:24.038 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-09-11T00:32:24.038 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-09-11T00:32:24.038 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.038 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:24.039 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-09-11T00:32:24.039 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.039 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.039 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.039 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.039 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.039 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.039 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.039 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.039 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-09-11T00:32:24.039 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-09-11T00:32:24.039 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:24.039 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.040 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:24.040 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-09-11T00:32:24.040 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.040 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.040 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-09-11T00:32:24.040 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-09-11T00:32:24.040 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-09-11T00:32:24.040 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-09-11T00:32:24.040 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-09-11T00:32:24.040 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-09-11T00:32:24.040 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-09-11T00:32:24.040 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-09-11T00:32:24.041 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.041 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.041 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.041 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.041 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.041 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-09-11T00:32:24.041 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-09-11T00:32:24.041 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-09-11T00:32:24.041 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.041 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:24.041 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-09-11T00:32:24.041 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.042 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.042 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.042 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.042 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.042 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.042 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.042 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.042 INFO:teuthology.orchestra.run.smithi012.stdout: "sleep_interval": { 2024-09-11T00:32:24.042 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "sleep_interval", 2024-09-11T00:32:24.042 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "secs", 2024-09-11T00:32:24.042 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.043 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:24.043 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "60", 2024-09-11T00:32:24.043 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.043 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.043 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.043 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.043 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.043 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.043 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.043 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.043 INFO:teuthology.orchestra.run.smithi012.stdout: "sqlite3_killpoint": { 2024-09-11T00:32:24.043 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "sqlite3_killpoint", 2024-09-11T00:32:24.043 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-09-11T00:32:24.044 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "dev", 2024-09-11T00:32:24.044 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:24.044 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "0", 2024-09-11T00:32:24.044 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.044 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.044 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.044 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.044 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.044 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.044 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.044 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.044 INFO:teuthology.orchestra.run.smithi012.stdout: "threshold": { 2024-09-11T00:32:24.045 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "threshold", 2024-09-11T00:32:24.045 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "float", 2024-09-11T00:32:24.045 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.045 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:24.045 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "3.0", 2024-09-11T00:32:24.045 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "1.0", 2024-09-11T00:32:24.045 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.045 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.045 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "scaling threshold", 2024-09-11T00:32:24.045 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-09-11T00:32:24.045 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.046 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.046 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:32:24.046 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:32:24.046 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.046 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-11T00:32:24.046 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "progress", 2024-09-11T00:32:24.046 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-09-11T00:32:24.046 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-09-11T00:32:24.046 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-09-11T00:32:24.046 INFO:teuthology.orchestra.run.smithi012.stdout: "allow_pg_recovery_event": { 2024-09-11T00:32:24.046 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "allow_pg_recovery_event", 2024-09-11T00:32:24.047 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-09-11T00:32:24.047 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.047 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:24.047 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-09-11T00:32:24.047 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.047 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.047 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.047 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "allow the module to show pg recovery progress", 2024-09-11T00:32:24.047 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.047 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.047 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.048 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.048 INFO:teuthology.orchestra.run.smithi012.stdout: "enabled": { 2024-09-11T00:32:24.048 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "enabled", 2024-09-11T00:32:24.048 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-09-11T00:32:24.048 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.048 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:24.048 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-09-11T00:32:24.048 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.048 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.048 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.048 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.049 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.049 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.049 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.049 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.049 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-09-11T00:32:24.049 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-09-11T00:32:24.049 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:24.049 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.049 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:24.049 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-09-11T00:32:24.049 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.050 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.050 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-09-11T00:32:24.050 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-09-11T00:32:24.050 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-09-11T00:32:24.050 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-09-11T00:32:24.050 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-09-11T00:32:24.050 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-09-11T00:32:24.050 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-09-11T00:32:24.050 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-09-11T00:32:24.050 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.050 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.050 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.051 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.051 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.051 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-09-11T00:32:24.051 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-09-11T00:32:24.051 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-09-11T00:32:24.051 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.051 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:24.051 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-09-11T00:32:24.051 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.051 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.051 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.051 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.051 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.052 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.052 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.052 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.052 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-09-11T00:32:24.052 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-09-11T00:32:24.052 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:24.052 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.052 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:24.052 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-09-11T00:32:24.052 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.052 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.052 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-09-11T00:32:24.053 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-09-11T00:32:24.053 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-09-11T00:32:24.053 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-09-11T00:32:24.053 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-09-11T00:32:24.053 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-09-11T00:32:24.053 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-09-11T00:32:24.053 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-09-11T00:32:24.053 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.053 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.053 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.053 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.053 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.053 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-09-11T00:32:24.054 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-09-11T00:32:24.054 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-09-11T00:32:24.054 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.054 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:24.054 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-09-11T00:32:24.054 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.054 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.054 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.054 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.054 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.054 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.054 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.054 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.055 INFO:teuthology.orchestra.run.smithi012.stdout: "max_completed_events": { 2024-09-11T00:32:24.055 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "max_completed_events", 2024-09-11T00:32:24.055 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-09-11T00:32:24.055 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.055 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:24.055 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "50", 2024-09-11T00:32:24.055 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.055 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.055 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.055 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "number of past completed events to remember", 2024-09-11T00:32:24.055 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.056 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.056 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.056 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.056 INFO:teuthology.orchestra.run.smithi012.stdout: "sleep_interval": { 2024-09-11T00:32:24.056 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "sleep_interval", 2024-09-11T00:32:24.056 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "secs", 2024-09-11T00:32:24.056 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.056 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:24.056 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "5", 2024-09-11T00:32:24.056 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.056 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.057 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.057 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "how long the module is going to sleep", 2024-09-11T00:32:24.057 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.057 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.057 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.057 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.057 INFO:teuthology.orchestra.run.smithi012.stdout: "sqlite3_killpoint": { 2024-09-11T00:32:24.057 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "sqlite3_killpoint", 2024-09-11T00:32:24.057 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-09-11T00:32:24.057 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "dev", 2024-09-11T00:32:24.057 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:24.057 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "0", 2024-09-11T00:32:24.057 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.058 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.058 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.058 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.058 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.058 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.058 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.058 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:32:24.058 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:32:24.058 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.058 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-11T00:32:24.058 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "prometheus", 2024-09-11T00:32:24.058 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-09-11T00:32:24.058 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-09-11T00:32:24.059 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-09-11T00:32:24.059 INFO:teuthology.orchestra.run.smithi012.stdout: "cache": { 2024-09-11T00:32:24.059 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "cache", 2024-09-11T00:32:24.059 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-09-11T00:32:24.059 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.059 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:24.059 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-09-11T00:32:24.059 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.059 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.059 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.059 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.059 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.059 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.060 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.060 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.060 INFO:teuthology.orchestra.run.smithi012.stdout: "exclude_perf_counters": { 2024-09-11T00:32:24.060 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "exclude_perf_counters", 2024-09-11T00:32:24.060 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-09-11T00:32:24.060 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.060 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:24.060 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-09-11T00:32:24.060 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.060 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.060 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.060 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-09-11T00:32:24.060 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-09-11T00:32:24.061 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.061 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.061 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.061 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-09-11T00:32:24.061 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-09-11T00:32:24.061 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:24.061 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.061 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:24.061 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-09-11T00:32:24.061 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.061 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.062 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-09-11T00:32:24.062 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-09-11T00:32:24.062 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-09-11T00:32:24.062 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-09-11T00:32:24.062 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-09-11T00:32:24.062 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-09-11T00:32:24.062 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-09-11T00:32:24.062 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-09-11T00:32:24.062 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.062 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.062 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.062 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.062 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.063 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-09-11T00:32:24.063 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-09-11T00:32:24.063 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-09-11T00:32:24.063 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.063 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:24.063 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-09-11T00:32:24.063 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.063 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.063 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.063 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.063 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.063 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.064 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.064 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.064 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-09-11T00:32:24.064 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-09-11T00:32:24.064 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:24.064 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.064 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:24.064 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-09-11T00:32:24.064 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.064 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.064 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-09-11T00:32:24.064 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-09-11T00:32:24.064 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-09-11T00:32:24.065 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-09-11T00:32:24.065 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-09-11T00:32:24.065 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-09-11T00:32:24.065 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-09-11T00:32:24.065 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-09-11T00:32:24.065 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.065 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.065 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.065 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.065 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.065 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-09-11T00:32:24.065 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-09-11T00:32:24.066 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-09-11T00:32:24.066 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.066 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:24.066 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-09-11T00:32:24.066 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.066 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.066 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.066 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.066 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.066 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.066 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.066 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.067 INFO:teuthology.orchestra.run.smithi012.stdout: "rbd_stats_pools": { 2024-09-11T00:32:24.067 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "rbd_stats_pools", 2024-09-11T00:32:24.067 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:24.067 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.067 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:24.067 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-09-11T00:32:24.067 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.067 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.067 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.067 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.067 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.067 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.067 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.068 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.068 INFO:teuthology.orchestra.run.smithi012.stdout: "rbd_stats_pools_refresh_interval": { 2024-09-11T00:32:24.068 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-09-11T00:32:24.068 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-09-11T00:32:24.068 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.068 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:24.068 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "300", 2024-09-11T00:32:24.068 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.068 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.068 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.068 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.068 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.068 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.069 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.069 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.069 INFO:teuthology.orchestra.run.smithi012.stdout: "scrape_interval": { 2024-09-11T00:32:24.069 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "scrape_interval", 2024-09-11T00:32:24.069 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "float", 2024-09-11T00:32:24.069 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.069 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:24.069 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "15.0", 2024-09-11T00:32:24.069 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.069 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.069 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.069 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.070 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.070 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.070 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.070 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.070 INFO:teuthology.orchestra.run.smithi012.stdout: "server_addr": { 2024-09-11T00:32:24.070 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "server_addr", 2024-09-11T00:32:24.070 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:24.070 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.070 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:24.070 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "::", 2024-09-11T00:32:24.070 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.070 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.070 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.071 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-09-11T00:32:24.071 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.071 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.071 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.071 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.071 INFO:teuthology.orchestra.run.smithi012.stdout: "server_port": { 2024-09-11T00:32:24.071 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "server_port", 2024-09-11T00:32:24.071 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-09-11T00:32:24.071 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.071 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:24.071 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "9283", 2024-09-11T00:32:24.071 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.072 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.072 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.072 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-09-11T00:32:24.072 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.072 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.072 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.072 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.072 INFO:teuthology.orchestra.run.smithi012.stdout: "sqlite3_killpoint": { 2024-09-11T00:32:24.072 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "sqlite3_killpoint", 2024-09-11T00:32:24.072 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-09-11T00:32:24.072 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "dev", 2024-09-11T00:32:24.072 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:24.073 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "0", 2024-09-11T00:32:24.073 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.073 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.073 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.073 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.073 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.073 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.073 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.073 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.073 INFO:teuthology.orchestra.run.smithi012.stdout: "stale_cache_strategy": { 2024-09-11T00:32:24.073 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "stale_cache_strategy", 2024-09-11T00:32:24.074 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:24.074 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.074 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:24.074 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "log", 2024-09-11T00:32:24.074 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.074 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.074 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.074 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.074 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.074 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.074 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.074 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.075 INFO:teuthology.orchestra.run.smithi012.stdout: "standby_behaviour": { 2024-09-11T00:32:24.075 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "standby_behaviour", 2024-09-11T00:32:24.075 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:24.075 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.075 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:24.075 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "default", 2024-09-11T00:32:24.075 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.075 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.075 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-09-11T00:32:24.075 INFO:teuthology.orchestra.run.smithi012.stdout: "default", 2024-09-11T00:32:24.075 INFO:teuthology.orchestra.run.smithi012.stdout: "error" 2024-09-11T00:32:24.075 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-09-11T00:32:24.076 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.076 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.076 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.076 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.076 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.076 INFO:teuthology.orchestra.run.smithi012.stdout: "standby_error_status_code": { 2024-09-11T00:32:24.076 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "standby_error_status_code", 2024-09-11T00:32:24.076 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-09-11T00:32:24.076 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.076 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:24.076 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "500", 2024-09-11T00:32:24.077 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "400", 2024-09-11T00:32:24.077 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "599", 2024-09-11T00:32:24.077 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.077 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.077 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.077 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.077 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.077 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:32:24.077 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:32:24.077 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.078 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-11T00:32:24.078 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "rbd_support", 2024-09-11T00:32:24.078 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-09-11T00:32:24.078 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-09-11T00:32:24.078 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-09-11T00:32:24.078 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-09-11T00:32:24.078 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-09-11T00:32:24.078 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:24.078 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.078 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:24.079 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-09-11T00:32:24.079 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.079 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.079 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-09-11T00:32:24.079 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-09-11T00:32:24.079 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-09-11T00:32:24.079 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-09-11T00:32:24.079 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-09-11T00:32:24.079 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-09-11T00:32:24.079 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-09-11T00:32:24.079 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-09-11T00:32:24.079 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.079 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.079 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.080 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.080 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.080 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-09-11T00:32:24.080 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-09-11T00:32:24.080 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-09-11T00:32:24.080 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.080 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:24.080 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-09-11T00:32:24.080 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.080 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.080 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.080 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.080 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.080 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.081 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.081 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.081 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-09-11T00:32:24.081 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-09-11T00:32:24.081 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:24.081 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.081 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:24.081 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-09-11T00:32:24.081 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.081 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.081 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-09-11T00:32:24.081 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-09-11T00:32:24.082 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-09-11T00:32:24.082 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-09-11T00:32:24.082 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-09-11T00:32:24.082 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-09-11T00:32:24.082 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-09-11T00:32:24.082 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-09-11T00:32:24.082 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.082 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.082 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.082 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.082 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.082 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-09-11T00:32:24.083 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-09-11T00:32:24.083 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-09-11T00:32:24.083 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.083 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:24.083 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-09-11T00:32:24.083 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.083 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.083 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.083 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.084 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.084 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.084 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.084 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.084 INFO:teuthology.orchestra.run.smithi012.stdout: "max_concurrent_snap_create": { 2024-09-11T00:32:24.084 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "max_concurrent_snap_create", 2024-09-11T00:32:24.084 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-09-11T00:32:24.084 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.084 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:24.084 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "10", 2024-09-11T00:32:24.084 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.084 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.085 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.085 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.085 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.085 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.085 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.085 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.085 INFO:teuthology.orchestra.run.smithi012.stdout: "mirror_snapshot_schedule": { 2024-09-11T00:32:24.085 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "mirror_snapshot_schedule", 2024-09-11T00:32:24.085 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:24.085 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.085 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:24.085 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-09-11T00:32:24.085 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.086 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.086 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.086 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.086 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.086 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.086 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.086 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.086 INFO:teuthology.orchestra.run.smithi012.stdout: "sqlite3_killpoint": { 2024-09-11T00:32:24.086 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "sqlite3_killpoint", 2024-09-11T00:32:24.086 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-09-11T00:32:24.086 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "dev", 2024-09-11T00:32:24.086 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:24.086 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "0", 2024-09-11T00:32:24.086 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.087 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.087 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.087 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.087 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.087 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.087 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.087 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.087 INFO:teuthology.orchestra.run.smithi012.stdout: "trash_purge_schedule": { 2024-09-11T00:32:24.087 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "trash_purge_schedule", 2024-09-11T00:32:24.087 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:24.087 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.087 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:24.088 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-09-11T00:32:24.088 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.088 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.088 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.088 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.088 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.088 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.088 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.088 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:32:24.088 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:32:24.088 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.088 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-11T00:32:24.088 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "restful", 2024-09-11T00:32:24.089 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-09-11T00:32:24.089 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-09-11T00:32:24.089 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-09-11T00:32:24.089 INFO:teuthology.orchestra.run.smithi012.stdout: "enable_auth": { 2024-09-11T00:32:24.089 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "enable_auth", 2024-09-11T00:32:24.089 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-09-11T00:32:24.089 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.089 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:24.089 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-09-11T00:32:24.089 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.089 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.089 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.090 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.090 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.090 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.090 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.090 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.090 INFO:teuthology.orchestra.run.smithi012.stdout: "key_file": { 2024-09-11T00:32:24.090 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "key_file", 2024-09-11T00:32:24.090 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:24.090 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.090 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:24.090 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-09-11T00:32:24.090 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.090 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.090 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.091 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.091 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.091 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.091 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.091 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.091 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-09-11T00:32:24.091 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-09-11T00:32:24.091 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:24.091 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.091 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:24.091 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-09-11T00:32:24.091 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.091 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.091 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-09-11T00:32:24.092 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-09-11T00:32:24.092 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-09-11T00:32:24.092 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-09-11T00:32:24.092 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-09-11T00:32:24.092 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-09-11T00:32:24.092 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-09-11T00:32:24.092 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-09-11T00:32:24.092 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.092 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.092 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.092 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.092 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.092 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-09-11T00:32:24.093 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-09-11T00:32:24.093 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-09-11T00:32:24.093 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.093 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:24.093 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-09-11T00:32:24.093 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.093 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.093 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.093 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.093 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.093 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.094 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.094 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.094 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-09-11T00:32:24.094 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-09-11T00:32:24.094 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:24.094 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.094 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:24.094 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-09-11T00:32:24.094 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.094 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.094 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-09-11T00:32:24.094 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-09-11T00:32:24.094 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-09-11T00:32:24.094 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-09-11T00:32:24.095 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-09-11T00:32:24.095 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-09-11T00:32:24.095 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-09-11T00:32:24.095 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-09-11T00:32:24.095 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.095 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.095 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.095 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.095 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.095 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-09-11T00:32:24.095 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-09-11T00:32:24.095 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-09-11T00:32:24.095 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.096 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:24.096 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-09-11T00:32:24.096 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.096 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.096 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.096 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.096 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.096 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.096 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.096 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.096 INFO:teuthology.orchestra.run.smithi012.stdout: "max_requests": { 2024-09-11T00:32:24.096 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "max_requests", 2024-09-11T00:32:24.096 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-09-11T00:32:24.097 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.097 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:24.097 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "500", 2024-09-11T00:32:24.097 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.097 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.097 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.097 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Maximum number of requests to keep in memory. When new request comes in, the oldest request will be removed if the number of requests exceeds the max request number.if un-finished request is removed, error message will be logged in the ceph-mgr log.", 2024-09-11T00:32:24.097 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.097 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.097 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.097 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.097 INFO:teuthology.orchestra.run.smithi012.stdout: "server_addr": { 2024-09-11T00:32:24.097 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "server_addr", 2024-09-11T00:32:24.098 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:24.098 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.098 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:24.098 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-09-11T00:32:24.098 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.098 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.098 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.098 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.098 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.098 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.098 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.098 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.098 INFO:teuthology.orchestra.run.smithi012.stdout: "server_port": { 2024-09-11T00:32:24.099 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "server_port", 2024-09-11T00:32:24.099 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:24.099 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.099 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:24.099 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-09-11T00:32:24.099 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.099 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.099 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.099 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.099 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.099 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.099 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.099 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.099 INFO:teuthology.orchestra.run.smithi012.stdout: "sqlite3_killpoint": { 2024-09-11T00:32:24.100 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "sqlite3_killpoint", 2024-09-11T00:32:24.100 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-09-11T00:32:24.100 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "dev", 2024-09-11T00:32:24.100 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:24.100 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "0", 2024-09-11T00:32:24.100 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.100 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.100 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.100 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.100 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.100 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.100 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.100 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:32:24.101 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:32:24.101 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.101 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-11T00:32:24.101 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "rgw", 2024-09-11T00:32:24.101 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-09-11T00:32:24.101 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-09-11T00:32:24.101 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-09-11T00:32:24.101 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-09-11T00:32:24.101 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-09-11T00:32:24.101 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:24.101 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.101 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:24.101 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-09-11T00:32:24.102 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.102 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.102 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-09-11T00:32:24.102 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-09-11T00:32:24.102 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-09-11T00:32:24.102 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-09-11T00:32:24.102 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-09-11T00:32:24.102 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-09-11T00:32:24.102 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-09-11T00:32:24.102 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-09-11T00:32:24.102 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.102 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.102 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.103 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.103 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.103 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-09-11T00:32:24.103 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-09-11T00:32:24.103 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-09-11T00:32:24.103 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.103 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:24.103 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-09-11T00:32:24.103 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.103 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.103 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.103 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.103 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.103 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.104 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.104 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.104 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-09-11T00:32:24.104 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-09-11T00:32:24.104 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:24.104 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.104 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:24.104 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-09-11T00:32:24.104 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.104 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.104 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-09-11T00:32:24.104 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-09-11T00:32:24.105 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-09-11T00:32:24.105 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-09-11T00:32:24.105 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-09-11T00:32:24.105 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-09-11T00:32:24.105 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-09-11T00:32:24.105 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-09-11T00:32:24.105 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.105 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.105 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.105 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.105 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.105 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-09-11T00:32:24.105 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-09-11T00:32:24.105 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-09-11T00:32:24.106 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.106 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:24.106 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-09-11T00:32:24.106 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.106 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.106 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.106 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.106 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.106 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.106 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.106 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.106 INFO:teuthology.orchestra.run.smithi012.stdout: "secondary_zone_period_retry_limit": { 2024-09-11T00:32:24.106 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "secondary_zone_period_retry_limit", 2024-09-11T00:32:24.107 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-09-11T00:32:24.107 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.107 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:24.107 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "5", 2024-09-11T00:32:24.107 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.107 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.107 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.107 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "RGW module period update retry limit for secondary site", 2024-09-11T00:32:24.107 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.107 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.107 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.107 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.107 INFO:teuthology.orchestra.run.smithi012.stdout: "sqlite3_killpoint": { 2024-09-11T00:32:24.107 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "sqlite3_killpoint", 2024-09-11T00:32:24.108 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-09-11T00:32:24.108 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "dev", 2024-09-11T00:32:24.108 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:24.108 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "0", 2024-09-11T00:32:24.108 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.108 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.108 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.108 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.108 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.108 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.108 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.108 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:32:24.108 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:32:24.109 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.109 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-11T00:32:24.109 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "rook", 2024-09-11T00:32:24.109 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-09-11T00:32:24.109 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-09-11T00:32:24.109 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-09-11T00:32:24.109 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-09-11T00:32:24.109 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-09-11T00:32:24.109 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:24.109 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.109 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:24.109 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-09-11T00:32:24.110 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.110 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.110 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-09-11T00:32:24.110 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-09-11T00:32:24.110 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-09-11T00:32:24.110 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-09-11T00:32:24.110 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-09-11T00:32:24.110 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-09-11T00:32:24.110 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-09-11T00:32:24.110 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-09-11T00:32:24.110 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.110 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.111 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.111 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.111 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.111 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-09-11T00:32:24.111 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-09-11T00:32:24.111 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-09-11T00:32:24.111 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.111 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:24.111 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-09-11T00:32:24.111 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.111 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.111 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.111 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.112 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.112 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.112 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.112 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.112 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-09-11T00:32:24.112 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-09-11T00:32:24.112 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:24.112 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.112 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:24.112 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-09-11T00:32:24.112 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.112 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.112 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-09-11T00:32:24.113 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-09-11T00:32:24.113 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-09-11T00:32:24.113 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-09-11T00:32:24.113 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-09-11T00:32:24.113 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-09-11T00:32:24.113 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-09-11T00:32:24.113 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-09-11T00:32:24.113 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.113 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.113 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.113 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.113 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.113 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-09-11T00:32:24.114 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-09-11T00:32:24.114 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-09-11T00:32:24.114 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.114 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:24.114 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-09-11T00:32:24.114 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.114 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.114 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.114 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.114 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.114 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.114 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.114 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.115 INFO:teuthology.orchestra.run.smithi012.stdout: "sqlite3_killpoint": { 2024-09-11T00:32:24.115 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "sqlite3_killpoint", 2024-09-11T00:32:24.115 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-09-11T00:32:24.115 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "dev", 2024-09-11T00:32:24.115 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:24.115 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "0", 2024-09-11T00:32:24.115 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.115 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.115 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.115 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.115 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.115 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.115 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.115 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.116 INFO:teuthology.orchestra.run.smithi012.stdout: "storage_class": { 2024-09-11T00:32:24.116 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "storage_class", 2024-09-11T00:32:24.116 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:24.116 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.116 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:24.116 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "local", 2024-09-11T00:32:24.116 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.116 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.116 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.116 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-09-11T00:32:24.116 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.116 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.116 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.116 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:32:24.117 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:32:24.117 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.117 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-11T00:32:24.117 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "selftest", 2024-09-11T00:32:24.117 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-09-11T00:32:24.117 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-09-11T00:32:24.117 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-09-11T00:32:24.117 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-09-11T00:32:24.117 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-09-11T00:32:24.117 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:24.117 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.117 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:24.118 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-09-11T00:32:24.118 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.118 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.118 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-09-11T00:32:24.118 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-09-11T00:32:24.118 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-09-11T00:32:24.118 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-09-11T00:32:24.118 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-09-11T00:32:24.118 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-09-11T00:32:24.118 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-09-11T00:32:24.118 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-09-11T00:32:24.118 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.118 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.119 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.119 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.119 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.119 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-09-11T00:32:24.119 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-09-11T00:32:24.119 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-09-11T00:32:24.119 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.119 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:24.119 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-09-11T00:32:24.119 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.119 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.119 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.119 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.120 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.120 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.120 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.120 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.120 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-09-11T00:32:24.120 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-09-11T00:32:24.120 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:24.120 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.120 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:24.120 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-09-11T00:32:24.120 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.120 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.120 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-09-11T00:32:24.121 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-09-11T00:32:24.121 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-09-11T00:32:24.121 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-09-11T00:32:24.121 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-09-11T00:32:24.121 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-09-11T00:32:24.121 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-09-11T00:32:24.121 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-09-11T00:32:24.121 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.121 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.121 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.121 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.121 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.121 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-09-11T00:32:24.122 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-09-11T00:32:24.122 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-09-11T00:32:24.122 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.122 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:24.122 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-09-11T00:32:24.122 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.122 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.122 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.122 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.122 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.122 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.122 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.122 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.122 INFO:teuthology.orchestra.run.smithi012.stdout: "roption1": { 2024-09-11T00:32:24.122 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "roption1", 2024-09-11T00:32:24.123 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:24.123 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.123 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:24.123 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-09-11T00:32:24.123 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.123 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.123 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.123 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.123 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.123 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.123 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.123 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.123 INFO:teuthology.orchestra.run.smithi012.stdout: "roption2": { 2024-09-11T00:32:24.123 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "roption2", 2024-09-11T00:32:24.124 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:24.124 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.124 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:24.124 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "xyz", 2024-09-11T00:32:24.124 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.124 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.124 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.124 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.124 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.124 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.124 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.124 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.125 INFO:teuthology.orchestra.run.smithi012.stdout: "rwoption1": { 2024-09-11T00:32:24.125 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "rwoption1", 2024-09-11T00:32:24.125 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:24.125 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.125 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:24.125 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-09-11T00:32:24.125 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.125 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.125 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.125 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.125 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.125 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.125 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.126 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.126 INFO:teuthology.orchestra.run.smithi012.stdout: "rwoption2": { 2024-09-11T00:32:24.126 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "rwoption2", 2024-09-11T00:32:24.126 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-09-11T00:32:24.126 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.126 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:24.126 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-09-11T00:32:24.126 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.126 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.126 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.126 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.126 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.126 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.126 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.127 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.127 INFO:teuthology.orchestra.run.smithi012.stdout: "rwoption3": { 2024-09-11T00:32:24.127 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "rwoption3", 2024-09-11T00:32:24.127 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "float", 2024-09-11T00:32:24.127 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.127 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:24.127 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-09-11T00:32:24.127 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.127 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.127 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.127 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.127 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.127 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.128 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.128 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.128 INFO:teuthology.orchestra.run.smithi012.stdout: "rwoption4": { 2024-09-11T00:32:24.128 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "rwoption4", 2024-09-11T00:32:24.128 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:24.128 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.128 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:24.128 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-09-11T00:32:24.128 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.128 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.128 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.129 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.129 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.129 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.129 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.129 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.129 INFO:teuthology.orchestra.run.smithi012.stdout: "rwoption5": { 2024-09-11T00:32:24.129 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "rwoption5", 2024-09-11T00:32:24.129 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-09-11T00:32:24.129 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.129 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:24.129 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-09-11T00:32:24.129 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.129 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.130 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.130 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.130 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.130 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.130 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.130 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.130 INFO:teuthology.orchestra.run.smithi012.stdout: "rwoption6": { 2024-09-11T00:32:24.130 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "rwoption6", 2024-09-11T00:32:24.130 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-09-11T00:32:24.130 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.130 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:24.130 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-09-11T00:32:24.130 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.131 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.131 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.131 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.131 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.131 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.131 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.131 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.131 INFO:teuthology.orchestra.run.smithi012.stdout: "rwoption7": { 2024-09-11T00:32:24.131 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "rwoption7", 2024-09-11T00:32:24.131 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-09-11T00:32:24.131 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.131 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:24.132 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-09-11T00:32:24.132 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "1", 2024-09-11T00:32:24.132 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "42", 2024-09-11T00:32:24.132 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.132 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.132 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.132 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.132 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.132 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.132 INFO:teuthology.orchestra.run.smithi012.stdout: "sqlite3_killpoint": { 2024-09-11T00:32:24.132 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "sqlite3_killpoint", 2024-09-11T00:32:24.132 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-09-11T00:32:24.133 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "dev", 2024-09-11T00:32:24.133 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:24.133 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "0", 2024-09-11T00:32:24.133 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.133 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.133 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.133 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.133 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.133 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.133 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.133 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.133 INFO:teuthology.orchestra.run.smithi012.stdout: "testkey": { 2024-09-11T00:32:24.133 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "testkey", 2024-09-11T00:32:24.134 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:24.134 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.134 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:24.134 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-09-11T00:32:24.134 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.134 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.134 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.134 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.134 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.134 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.134 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.134 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.134 INFO:teuthology.orchestra.run.smithi012.stdout: "testlkey": { 2024-09-11T00:32:24.135 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "testlkey", 2024-09-11T00:32:24.135 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:24.135 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.135 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:24.135 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-09-11T00:32:24.135 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.135 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.135 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.135 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.135 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.135 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.135 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.135 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.136 INFO:teuthology.orchestra.run.smithi012.stdout: "testnewline": { 2024-09-11T00:32:24.136 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "testnewline", 2024-09-11T00:32:24.136 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:24.136 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.136 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:24.136 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-09-11T00:32:24.136 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.136 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.136 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.136 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.136 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.136 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.136 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.137 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:32:24.137 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:32:24.137 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.137 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-11T00:32:24.137 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "smb", 2024-09-11T00:32:24.137 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-09-11T00:32:24.137 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-09-11T00:32:24.137 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-09-11T00:32:24.137 INFO:teuthology.orchestra.run.smithi012.stdout: "internal_store_backend": { 2024-09-11T00:32:24.137 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "internal_store_backend", 2024-09-11T00:32:24.137 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:24.137 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "dev", 2024-09-11T00:32:24.137 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:24.138 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-09-11T00:32:24.138 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.138 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.138 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.138 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "set internal store backend. for develoment and testing only", 2024-09-11T00:32:24.138 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.138 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.138 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.138 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.138 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-09-11T00:32:24.138 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-09-11T00:32:24.138 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:24.139 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.139 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:24.139 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-09-11T00:32:24.139 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.139 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.139 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-09-11T00:32:24.139 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-09-11T00:32:24.139 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-09-11T00:32:24.139 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-09-11T00:32:24.139 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-09-11T00:32:24.139 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-09-11T00:32:24.139 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-09-11T00:32:24.139 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-09-11T00:32:24.140 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.140 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.140 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.140 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.140 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.140 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-09-11T00:32:24.140 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-09-11T00:32:24.140 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-09-11T00:32:24.140 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.140 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:24.140 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-09-11T00:32:24.141 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.141 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.141 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.141 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.141 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.141 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.141 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.141 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.141 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-09-11T00:32:24.141 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-09-11T00:32:24.141 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:24.141 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.142 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:24.142 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-09-11T00:32:24.142 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.142 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.142 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-09-11T00:32:24.142 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-09-11T00:32:24.142 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-09-11T00:32:24.142 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-09-11T00:32:24.142 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-09-11T00:32:24.142 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-09-11T00:32:24.142 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-09-11T00:32:24.142 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-09-11T00:32:24.142 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.143 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.143 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.143 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.143 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.143 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-09-11T00:32:24.143 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-09-11T00:32:24.143 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-09-11T00:32:24.143 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.143 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:24.143 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-09-11T00:32:24.143 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.144 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.144 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.144 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.144 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.144 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.144 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.144 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.144 INFO:teuthology.orchestra.run.smithi012.stdout: "sqlite3_killpoint": { 2024-09-11T00:32:24.144 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "sqlite3_killpoint", 2024-09-11T00:32:24.144 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-09-11T00:32:24.144 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "dev", 2024-09-11T00:32:24.144 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:24.144 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "0", 2024-09-11T00:32:24.145 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.145 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.145 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.145 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.145 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.145 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.145 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.145 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.145 INFO:teuthology.orchestra.run.smithi012.stdout: "update_orchestration": { 2024-09-11T00:32:24.145 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "update_orchestration", 2024-09-11T00:32:24.145 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-09-11T00:32:24.145 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.145 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:24.146 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-09-11T00:32:24.146 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.146 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.146 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.146 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "automatically update orchestration when smb resources are changed", 2024-09-11T00:32:24.146 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.146 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.146 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.146 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:32:24.146 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:32:24.146 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.146 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-11T00:32:24.147 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "snap_schedule", 2024-09-11T00:32:24.147 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-09-11T00:32:24.147 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-09-11T00:32:24.147 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-09-11T00:32:24.147 INFO:teuthology.orchestra.run.smithi012.stdout: "allow_m_granularity": { 2024-09-11T00:32:24.147 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "allow_m_granularity", 2024-09-11T00:32:24.147 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-09-11T00:32:24.147 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.147 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:24.147 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-09-11T00:32:24.147 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.147 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.147 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.147 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "allow minute scheduled snapshots", 2024-09-11T00:32:24.148 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.148 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.148 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.148 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.148 INFO:teuthology.orchestra.run.smithi012.stdout: "dump_on_update": { 2024-09-11T00:32:24.148 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "dump_on_update", 2024-09-11T00:32:24.148 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-09-11T00:32:24.148 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.148 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:24.148 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-09-11T00:32:24.148 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.148 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.148 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.148 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "dump database to debug log on update", 2024-09-11T00:32:24.149 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.149 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.149 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.149 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.149 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-09-11T00:32:24.149 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-09-11T00:32:24.149 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:24.149 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.149 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:24.149 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-09-11T00:32:24.149 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.149 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.149 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-09-11T00:32:24.150 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-09-11T00:32:24.150 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-09-11T00:32:24.150 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-09-11T00:32:24.150 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-09-11T00:32:24.150 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-09-11T00:32:24.150 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-09-11T00:32:24.150 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-09-11T00:32:24.150 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.150 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.150 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.150 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.150 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.150 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-09-11T00:32:24.151 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-09-11T00:32:24.151 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-09-11T00:32:24.151 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.151 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:24.151 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-09-11T00:32:24.151 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.151 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.151 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.151 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.151 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.151 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.151 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.151 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.152 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-09-11T00:32:24.152 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-09-11T00:32:24.152 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:24.152 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.152 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:24.152 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-09-11T00:32:24.152 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.152 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.152 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-09-11T00:32:24.152 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-09-11T00:32:24.152 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-09-11T00:32:24.152 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-09-11T00:32:24.152 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-09-11T00:32:24.153 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-09-11T00:32:24.153 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-09-11T00:32:24.153 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-09-11T00:32:24.153 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.153 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.153 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.153 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.153 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.153 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-09-11T00:32:24.153 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-09-11T00:32:24.153 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-09-11T00:32:24.153 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.153 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:24.154 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-09-11T00:32:24.154 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.154 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.154 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.154 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.154 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.154 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.154 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.154 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.154 INFO:teuthology.orchestra.run.smithi012.stdout: "sqlite3_killpoint": { 2024-09-11T00:32:24.154 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "sqlite3_killpoint", 2024-09-11T00:32:24.154 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-09-11T00:32:24.154 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "dev", 2024-09-11T00:32:24.155 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:24.155 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "0", 2024-09-11T00:32:24.155 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.155 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.155 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.155 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.155 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.155 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.155 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.155 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:32:24.155 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:32:24.155 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.155 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-11T00:32:24.155 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "stats", 2024-09-11T00:32:24.156 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-09-11T00:32:24.156 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-09-11T00:32:24.156 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-09-11T00:32:24.156 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-09-11T00:32:24.156 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-09-11T00:32:24.156 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:24.156 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.156 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:24.156 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-09-11T00:32:24.156 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.156 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.156 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-09-11T00:32:24.157 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-09-11T00:32:24.157 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-09-11T00:32:24.157 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-09-11T00:32:24.157 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-09-11T00:32:24.157 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-09-11T00:32:24.157 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-09-11T00:32:24.157 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-09-11T00:32:24.157 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.157 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.157 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.157 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.157 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.157 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-09-11T00:32:24.158 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-09-11T00:32:24.158 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-09-11T00:32:24.158 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.158 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:24.158 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-09-11T00:32:24.158 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.158 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.158 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.158 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.158 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.158 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.158 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.158 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.159 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-09-11T00:32:24.159 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-09-11T00:32:24.159 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:24.159 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.159 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:24.159 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-09-11T00:32:24.159 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.159 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.159 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-09-11T00:32:24.159 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-09-11T00:32:24.159 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-09-11T00:32:24.159 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-09-11T00:32:24.160 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-09-11T00:32:24.160 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-09-11T00:32:24.160 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-09-11T00:32:24.160 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-09-11T00:32:24.160 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.160 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.160 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.160 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.160 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.160 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-09-11T00:32:24.160 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-09-11T00:32:24.160 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-09-11T00:32:24.160 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.161 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:24.161 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-09-11T00:32:24.161 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.161 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.161 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.161 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.161 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.161 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.161 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.161 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.161 INFO:teuthology.orchestra.run.smithi012.stdout: "sqlite3_killpoint": { 2024-09-11T00:32:24.161 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "sqlite3_killpoint", 2024-09-11T00:32:24.162 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-09-11T00:32:24.162 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "dev", 2024-09-11T00:32:24.162 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:24.162 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "0", 2024-09-11T00:32:24.162 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.162 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.162 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.162 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.162 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.162 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.162 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.162 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:32:24.162 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:32:24.163 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.163 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-11T00:32:24.163 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "status", 2024-09-11T00:32:24.163 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-09-11T00:32:24.163 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-09-11T00:32:24.163 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-09-11T00:32:24.163 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-09-11T00:32:24.163 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-09-11T00:32:24.163 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:24.163 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.163 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:24.163 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-09-11T00:32:24.163 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.163 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.164 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-09-11T00:32:24.164 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-09-11T00:32:24.164 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-09-11T00:32:24.164 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-09-11T00:32:24.164 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-09-11T00:32:24.164 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-09-11T00:32:24.164 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-09-11T00:32:24.164 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-09-11T00:32:24.164 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.164 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.164 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.164 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.164 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.165 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-09-11T00:32:24.165 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-09-11T00:32:24.165 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-09-11T00:32:24.165 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.165 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:24.165 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-09-11T00:32:24.165 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.165 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.165 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.165 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.165 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.165 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.166 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.166 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.166 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-09-11T00:32:24.166 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-09-11T00:32:24.166 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:24.166 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.166 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:24.166 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-09-11T00:32:24.166 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.166 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.166 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-09-11T00:32:24.166 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-09-11T00:32:24.167 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-09-11T00:32:24.167 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-09-11T00:32:24.167 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-09-11T00:32:24.167 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-09-11T00:32:24.167 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-09-11T00:32:24.167 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-09-11T00:32:24.167 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.167 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.167 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.167 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.167 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.167 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-09-11T00:32:24.167 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-09-11T00:32:24.167 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-09-11T00:32:24.168 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.168 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:24.168 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-09-11T00:32:24.168 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.168 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.168 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.168 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.168 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.168 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.168 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.168 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.168 INFO:teuthology.orchestra.run.smithi012.stdout: "sqlite3_killpoint": { 2024-09-11T00:32:24.168 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "sqlite3_killpoint", 2024-09-11T00:32:24.168 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-09-11T00:32:24.169 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "dev", 2024-09-11T00:32:24.169 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:24.169 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "0", 2024-09-11T00:32:24.169 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.169 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.169 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.169 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.169 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.169 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.169 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.169 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:32:24.169 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:32:24.169 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.169 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-11T00:32:24.170 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "telegraf", 2024-09-11T00:32:24.170 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-09-11T00:32:24.170 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-09-11T00:32:24.170 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-09-11T00:32:24.170 INFO:teuthology.orchestra.run.smithi012.stdout: "address": { 2024-09-11T00:32:24.170 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "address", 2024-09-11T00:32:24.170 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:24.170 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.170 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:24.170 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-09-11T00:32:24.170 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.170 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.170 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.170 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.171 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.171 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.171 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.171 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.171 INFO:teuthology.orchestra.run.smithi012.stdout: "interval": { 2024-09-11T00:32:24.171 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "interval", 2024-09-11T00:32:24.171 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "secs", 2024-09-11T00:32:24.171 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.171 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:24.171 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "15", 2024-09-11T00:32:24.171 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.171 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.171 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.171 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.171 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.172 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.172 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.172 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.172 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-09-11T00:32:24.172 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-09-11T00:32:24.172 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:24.172 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.172 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:24.172 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-09-11T00:32:24.172 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.172 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.172 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-09-11T00:32:24.172 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-09-11T00:32:24.172 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-09-11T00:32:24.173 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-09-11T00:32:24.173 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-09-11T00:32:24.173 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-09-11T00:32:24.173 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-09-11T00:32:24.173 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-09-11T00:32:24.173 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.173 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.173 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.173 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.173 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.173 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-09-11T00:32:24.173 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-09-11T00:32:24.173 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-09-11T00:32:24.173 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.174 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:24.174 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-09-11T00:32:24.174 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.174 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.174 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.174 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.174 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.174 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.174 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.174 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.174 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-09-11T00:32:24.174 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-09-11T00:32:24.174 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:24.175 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.175 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:24.175 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-09-11T00:32:24.175 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.175 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.175 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-09-11T00:32:24.175 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-09-11T00:32:24.175 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-09-11T00:32:24.175 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-09-11T00:32:24.175 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-09-11T00:32:24.175 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-09-11T00:32:24.175 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-09-11T00:32:24.175 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-09-11T00:32:24.175 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.175 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.176 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.176 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.176 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.176 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-09-11T00:32:24.176 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-09-11T00:32:24.176 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-09-11T00:32:24.176 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.176 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:24.176 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-09-11T00:32:24.176 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.176 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.176 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.177 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.177 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.177 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.177 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.177 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.177 INFO:teuthology.orchestra.run.smithi012.stdout: "sqlite3_killpoint": { 2024-09-11T00:32:24.177 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "sqlite3_killpoint", 2024-09-11T00:32:24.177 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-09-11T00:32:24.177 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "dev", 2024-09-11T00:32:24.177 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:24.177 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "0", 2024-09-11T00:32:24.177 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.177 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.178 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.178 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.178 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.178 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.178 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.178 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:32:24.178 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:32:24.178 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.178 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-11T00:32:24.178 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "telemetry", 2024-09-11T00:32:24.178 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-09-11T00:32:24.178 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-09-11T00:32:24.178 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-09-11T00:32:24.179 INFO:teuthology.orchestra.run.smithi012.stdout: "channel_basic": { 2024-09-11T00:32:24.179 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "channel_basic", 2024-09-11T00:32:24.179 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-09-11T00:32:24.179 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.179 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:24.179 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-09-11T00:32:24.179 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.179 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.179 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.179 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Share basic cluster information (size, version)", 2024-09-11T00:32:24.179 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.179 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.179 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.179 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.180 INFO:teuthology.orchestra.run.smithi012.stdout: "channel_crash": { 2024-09-11T00:32:24.180 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "channel_crash", 2024-09-11T00:32:24.180 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-09-11T00:32:24.180 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.180 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:24.180 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-09-11T00:32:24.180 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.180 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.180 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.180 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-09-11T00:32:24.180 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.180 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.180 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.180 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.181 INFO:teuthology.orchestra.run.smithi012.stdout: "channel_device": { 2024-09-11T00:32:24.181 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "channel_device", 2024-09-11T00:32:24.181 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-09-11T00:32:24.181 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.181 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:24.181 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-09-11T00:32:24.181 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.181 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.181 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.181 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-09-11T00:32:24.181 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.181 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.181 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.182 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.182 INFO:teuthology.orchestra.run.smithi012.stdout: "channel_ident": { 2024-09-11T00:32:24.182 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "channel_ident", 2024-09-11T00:32:24.182 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-09-11T00:32:24.182 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.182 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:24.182 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-09-11T00:32:24.182 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.182 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.182 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.182 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-09-11T00:32:24.182 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.183 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.183 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.183 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.183 INFO:teuthology.orchestra.run.smithi012.stdout: "channel_perf": { 2024-09-11T00:32:24.183 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "channel_perf", 2024-09-11T00:32:24.183 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-09-11T00:32:24.183 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.183 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:24.183 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-09-11T00:32:24.183 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.183 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.183 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.183 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Share various performance metrics of a cluster", 2024-09-11T00:32:24.184 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.184 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.184 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.184 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.184 INFO:teuthology.orchestra.run.smithi012.stdout: "contact": { 2024-09-11T00:32:24.184 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "contact", 2024-09-11T00:32:24.184 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:24.184 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.184 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:24.184 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-09-11T00:32:24.184 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.184 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.185 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.185 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.185 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.185 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.185 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.185 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.185 INFO:teuthology.orchestra.run.smithi012.stdout: "description": { 2024-09-11T00:32:24.185 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "description", 2024-09-11T00:32:24.185 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:24.185 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.185 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:24.185 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-09-11T00:32:24.186 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.186 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.186 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.186 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.186 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.186 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.186 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.186 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.186 INFO:teuthology.orchestra.run.smithi012.stdout: "device_url": { 2024-09-11T00:32:24.186 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "device_url", 2024-09-11T00:32:24.186 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:24.186 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.187 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:24.187 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-09-11T00:32:24.187 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.187 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.187 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.187 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.187 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.187 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.187 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.187 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.187 INFO:teuthology.orchestra.run.smithi012.stdout: "enabled": { 2024-09-11T00:32:24.187 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "enabled", 2024-09-11T00:32:24.187 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-09-11T00:32:24.188 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.188 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:24.188 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-09-11T00:32:24.188 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.188 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.188 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.188 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.188 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.188 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.188 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.188 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.188 INFO:teuthology.orchestra.run.smithi012.stdout: "interval": { 2024-09-11T00:32:24.188 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "interval", 2024-09-11T00:32:24.189 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-09-11T00:32:24.189 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.189 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:24.189 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "24", 2024-09-11T00:32:24.189 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "8", 2024-09-11T00:32:24.189 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.189 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.189 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.189 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.189 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.189 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.189 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.189 INFO:teuthology.orchestra.run.smithi012.stdout: "last_opt_revision": { 2024-09-11T00:32:24.192 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "last_opt_revision", 2024-09-11T00:32:24.192 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-09-11T00:32:24.192 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.192 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:24.192 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "1", 2024-09-11T00:32:24.192 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.192 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.192 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.193 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.193 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.193 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.193 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.193 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.193 INFO:teuthology.orchestra.run.smithi012.stdout: "leaderboard": { 2024-09-11T00:32:24.193 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "leaderboard", 2024-09-11T00:32:24.193 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-09-11T00:32:24.193 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.193 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:24.193 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-09-11T00:32:24.193 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.194 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.194 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.194 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.194 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.194 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.194 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.194 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.194 INFO:teuthology.orchestra.run.smithi012.stdout: "leaderboard_description": { 2024-09-11T00:32:24.194 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "leaderboard_description", 2024-09-11T00:32:24.194 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:24.194 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.194 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:24.194 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-09-11T00:32:24.194 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.195 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.195 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.195 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.195 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.195 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.195 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.195 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.195 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-09-11T00:32:24.195 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-09-11T00:32:24.195 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:24.195 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.195 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:24.195 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-09-11T00:32:24.196 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.196 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.196 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-09-11T00:32:24.196 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-09-11T00:32:24.196 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-09-11T00:32:24.196 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-09-11T00:32:24.196 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-09-11T00:32:24.196 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-09-11T00:32:24.196 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-09-11T00:32:24.196 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-09-11T00:32:24.196 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.196 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.196 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.197 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.197 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.197 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-09-11T00:32:24.197 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-09-11T00:32:24.197 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-09-11T00:32:24.197 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.197 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:24.197 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-09-11T00:32:24.197 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.197 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.197 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.197 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.197 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.198 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.198 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.198 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.198 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-09-11T00:32:24.198 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-09-11T00:32:24.198 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:24.198 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.198 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:24.198 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-09-11T00:32:24.198 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.198 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.198 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-09-11T00:32:24.198 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-09-11T00:32:24.198 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-09-11T00:32:24.199 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-09-11T00:32:24.199 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-09-11T00:32:24.199 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-09-11T00:32:24.199 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-09-11T00:32:24.199 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-09-11T00:32:24.199 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.199 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.199 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.199 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.199 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.199 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-09-11T00:32:24.199 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-09-11T00:32:24.199 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-09-11T00:32:24.200 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.200 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:24.200 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-09-11T00:32:24.200 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.200 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.200 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.200 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.200 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.200 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.200 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.200 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.200 INFO:teuthology.orchestra.run.smithi012.stdout: "organization": { 2024-09-11T00:32:24.201 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "organization", 2024-09-11T00:32:24.201 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:24.201 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.201 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:24.201 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-09-11T00:32:24.201 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.201 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.201 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.201 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.201 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.201 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.201 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.201 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.201 INFO:teuthology.orchestra.run.smithi012.stdout: "proxy": { 2024-09-11T00:32:24.202 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "proxy", 2024-09-11T00:32:24.202 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:24.202 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.202 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:24.202 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-09-11T00:32:24.202 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.202 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.202 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.202 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.202 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.202 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.202 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.202 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.203 INFO:teuthology.orchestra.run.smithi012.stdout: "sqlite3_killpoint": { 2024-09-11T00:32:24.203 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "sqlite3_killpoint", 2024-09-11T00:32:24.203 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-09-11T00:32:24.203 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "dev", 2024-09-11T00:32:24.203 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:24.203 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "0", 2024-09-11T00:32:24.203 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.203 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.203 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.203 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.203 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.204 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.204 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.204 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.204 INFO:teuthology.orchestra.run.smithi012.stdout: "url": { 2024-09-11T00:32:24.204 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "url", 2024-09-11T00:32:24.204 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:24.204 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.204 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:24.204 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-09-11T00:32:24.204 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.204 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.204 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.204 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.205 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.205 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.205 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.205 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:32:24.205 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:32:24.205 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.205 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-11T00:32:24.205 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "test_orchestrator", 2024-09-11T00:32:24.205 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-09-11T00:32:24.205 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-09-11T00:32:24.205 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-09-11T00:32:24.205 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-09-11T00:32:24.205 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-09-11T00:32:24.206 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:24.206 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.206 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:24.206 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-09-11T00:32:24.206 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.206 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.206 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-09-11T00:32:24.206 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-09-11T00:32:24.206 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-09-11T00:32:24.206 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-09-11T00:32:24.206 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-09-11T00:32:24.206 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-09-11T00:32:24.206 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-09-11T00:32:24.207 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-09-11T00:32:24.207 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.207 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.207 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.207 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.207 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.207 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-09-11T00:32:24.207 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-09-11T00:32:24.207 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-09-11T00:32:24.207 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.207 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:24.207 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-09-11T00:32:24.207 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.207 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.207 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.208 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.208 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.208 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.208 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.208 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.208 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-09-11T00:32:24.208 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-09-11T00:32:24.208 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:24.208 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.208 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:24.208 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-09-11T00:32:24.208 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.208 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.208 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-09-11T00:32:24.209 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-09-11T00:32:24.209 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-09-11T00:32:24.209 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-09-11T00:32:24.209 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-09-11T00:32:24.209 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-09-11T00:32:24.209 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-09-11T00:32:24.209 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-09-11T00:32:24.209 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.209 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.209 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.209 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.209 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.209 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-09-11T00:32:24.209 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-09-11T00:32:24.210 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-09-11T00:32:24.210 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.210 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:24.210 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-09-11T00:32:24.210 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.210 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.210 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.210 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.210 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.210 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.210 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.210 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.210 INFO:teuthology.orchestra.run.smithi012.stdout: "sqlite3_killpoint": { 2024-09-11T00:32:24.210 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "sqlite3_killpoint", 2024-09-11T00:32:24.211 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-09-11T00:32:24.211 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "dev", 2024-09-11T00:32:24.211 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:24.211 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "0", 2024-09-11T00:32:24.211 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.211 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.211 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.211 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.211 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.211 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.211 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.211 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:32:24.212 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:32:24.212 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.212 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-11T00:32:24.212 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "volumes", 2024-09-11T00:32:24.212 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-09-11T00:32:24.212 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-09-11T00:32:24.212 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-09-11T00:32:24.212 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-09-11T00:32:24.212 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-09-11T00:32:24.212 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:24.213 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.213 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:24.213 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-09-11T00:32:24.213 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.213 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.213 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-09-11T00:32:24.213 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-09-11T00:32:24.213 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-09-11T00:32:24.213 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-09-11T00:32:24.213 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-09-11T00:32:24.213 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-09-11T00:32:24.213 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-09-11T00:32:24.213 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-09-11T00:32:24.214 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.214 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.214 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.214 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.214 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.214 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-09-11T00:32:24.214 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-09-11T00:32:24.214 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-09-11T00:32:24.214 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.214 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:24.214 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-09-11T00:32:24.215 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.215 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.215 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.215 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.215 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.215 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.215 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.215 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.215 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-09-11T00:32:24.215 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-09-11T00:32:24.215 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:24.216 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.216 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:24.216 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-09-11T00:32:24.216 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.216 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.216 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-09-11T00:32:24.216 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-09-11T00:32:24.216 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-09-11T00:32:24.216 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-09-11T00:32:24.216 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-09-11T00:32:24.216 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-09-11T00:32:24.216 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-09-11T00:32:24.217 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-09-11T00:32:24.217 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.217 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.217 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.217 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.217 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.217 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-09-11T00:32:24.217 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-09-11T00:32:24.217 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-09-11T00:32:24.217 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.217 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:24.217 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-09-11T00:32:24.217 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.218 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.218 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.218 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.218 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.218 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.218 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.218 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.218 INFO:teuthology.orchestra.run.smithi012.stdout: "max_concurrent_clones": { 2024-09-11T00:32:24.218 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "max_concurrent_clones", 2024-09-11T00:32:24.218 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-09-11T00:32:24.218 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.218 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:24.219 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "4", 2024-09-11T00:32:24.219 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.219 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.219 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.219 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Number of asynchronous cloner threads", 2024-09-11T00:32:24.219 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.219 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.219 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.219 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.219 INFO:teuthology.orchestra.run.smithi012.stdout: "periodic_async_work": { 2024-09-11T00:32:24.219 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "periodic_async_work", 2024-09-11T00:32:24.219 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-09-11T00:32:24.219 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.220 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:24.220 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-09-11T00:32:24.220 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.220 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.220 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.220 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Periodically check for async work", 2024-09-11T00:32:24.220 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.220 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.220 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.220 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.220 INFO:teuthology.orchestra.run.smithi012.stdout: "snapshot_clone_delay": { 2024-09-11T00:32:24.220 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "snapshot_clone_delay", 2024-09-11T00:32:24.220 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-09-11T00:32:24.221 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.221 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:24.221 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "0", 2024-09-11T00:32:24.221 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.221 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.221 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.221 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-09-11T00:32:24.221 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.221 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.221 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.221 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.221 INFO:teuthology.orchestra.run.smithi012.stdout: "snapshot_clone_no_wait": { 2024-09-11T00:32:24.221 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "snapshot_clone_no_wait", 2024-09-11T00:32:24.222 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-09-11T00:32:24.222 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.222 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:24.222 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-09-11T00:32:24.222 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.222 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.222 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.222 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-09-11T00:32:24.222 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.222 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.222 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.222 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.222 INFO:teuthology.orchestra.run.smithi012.stdout: "sqlite3_killpoint": { 2024-09-11T00:32:24.222 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "sqlite3_killpoint", 2024-09-11T00:32:24.223 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-09-11T00:32:24.223 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "dev", 2024-09-11T00:32:24.223 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:24.223 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "0", 2024-09-11T00:32:24.223 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.223 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.223 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.223 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.223 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.223 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.223 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.223 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:32:24.223 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:32:24.224 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.224 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-11T00:32:24.224 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "zabbix", 2024-09-11T00:32:24.224 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-09-11T00:32:24.224 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-09-11T00:32:24.224 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-09-11T00:32:24.224 INFO:teuthology.orchestra.run.smithi012.stdout: "discovery_interval": { 2024-09-11T00:32:24.224 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "discovery_interval", 2024-09-11T00:32:24.224 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "uint", 2024-09-11T00:32:24.224 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.224 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:24.224 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "100", 2024-09-11T00:32:24.224 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.224 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.225 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.225 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.225 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.225 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.225 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.225 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.225 INFO:teuthology.orchestra.run.smithi012.stdout: "identifier": { 2024-09-11T00:32:24.225 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "identifier", 2024-09-11T00:32:24.225 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:24.225 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.225 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:24.225 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-09-11T00:32:24.226 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.226 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.226 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.226 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.226 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.226 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.226 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.226 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.226 INFO:teuthology.orchestra.run.smithi012.stdout: "interval": { 2024-09-11T00:32:24.226 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "interval", 2024-09-11T00:32:24.226 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "secs", 2024-09-11T00:32:24.226 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.226 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:24.226 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "60", 2024-09-11T00:32:24.227 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.227 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.227 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.227 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.227 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.227 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.227 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.227 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.227 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-09-11T00:32:24.227 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-09-11T00:32:24.227 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:24.227 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.227 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:24.227 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-09-11T00:32:24.228 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.228 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.228 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-09-11T00:32:24.228 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-09-11T00:32:24.228 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-09-11T00:32:24.228 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-09-11T00:32:24.228 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-09-11T00:32:24.228 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-09-11T00:32:24.228 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-09-11T00:32:24.228 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-09-11T00:32:24.228 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.228 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.229 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.229 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.229 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.229 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-09-11T00:32:24.229 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-09-11T00:32:24.229 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-09-11T00:32:24.229 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.229 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:24.229 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-09-11T00:32:24.229 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.229 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.229 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.229 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.230 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.230 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.230 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.230 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.230 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-09-11T00:32:24.230 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-09-11T00:32:24.230 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:24.230 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.230 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:24.230 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-09-11T00:32:24.230 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.230 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.230 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-09-11T00:32:24.230 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-09-11T00:32:24.231 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-09-11T00:32:24.231 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-09-11T00:32:24.231 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-09-11T00:32:24.231 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-09-11T00:32:24.231 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-09-11T00:32:24.231 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-09-11T00:32:24.231 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.231 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.231 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.231 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.231 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.231 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-09-11T00:32:24.231 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-09-11T00:32:24.232 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-09-11T00:32:24.232 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.232 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:24.232 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-09-11T00:32:24.232 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.232 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.232 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.232 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.232 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.232 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.232 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.232 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.232 INFO:teuthology.orchestra.run.smithi012.stdout: "sqlite3_killpoint": { 2024-09-11T00:32:24.233 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "sqlite3_killpoint", 2024-09-11T00:32:24.233 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-09-11T00:32:24.233 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "dev", 2024-09-11T00:32:24.233 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:24.233 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "0", 2024-09-11T00:32:24.233 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.233 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.233 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.233 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.233 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.233 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.233 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.233 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.233 INFO:teuthology.orchestra.run.smithi012.stdout: "zabbix_host": { 2024-09-11T00:32:24.234 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "zabbix_host", 2024-09-11T00:32:24.234 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:24.234 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.234 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:24.234 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-09-11T00:32:24.234 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.234 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.234 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.234 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.234 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.234 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.234 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.234 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.235 INFO:teuthology.orchestra.run.smithi012.stdout: "zabbix_port": { 2024-09-11T00:32:24.235 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "zabbix_port", 2024-09-11T00:32:24.235 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-09-11T00:32:24.235 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.235 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:24.235 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "10051", 2024-09-11T00:32:24.235 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.235 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.235 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.235 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.235 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.235 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.235 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.236 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.236 INFO:teuthology.orchestra.run.smithi012.stdout: "zabbix_sender": { 2024-09-11T00:32:24.236 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "zabbix_sender", 2024-09-11T00:32:24.236 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:24.236 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.236 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:24.236 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-09-11T00:32:24.236 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.236 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.236 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.236 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.236 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.236 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.236 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.237 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:32:24.237 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:32:24.237 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:32:24.237 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-11T00:32:24.237 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:32:24.237 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-09-11T00:32:24.237 INFO:teuthology.orchestra.run.smithi012.stdout: "modules": [ 2024-09-11T00:32:24.237 INFO:teuthology.orchestra.run.smithi012.stdout: "cephadm", 2024-09-11T00:32:24.237 INFO:teuthology.orchestra.run.smithi012.stdout: "dashboard", 2024-09-11T00:32:24.237 INFO:teuthology.orchestra.run.smithi012.stdout: "iostat", 2024-09-11T00:32:24.237 INFO:teuthology.orchestra.run.smithi012.stdout: "nfs", 2024-09-11T00:32:24.237 INFO:teuthology.orchestra.run.smithi012.stdout: "restful" 2024-09-11T00:32:24.237 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-09-11T00:32:24.238 INFO:teuthology.orchestra.run.smithi012.stdout: "available_modules": [ 2024-09-11T00:32:24.238 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-11T00:32:24.238 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "alerts", 2024-09-11T00:32:24.238 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-09-11T00:32:24.238 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-09-11T00:32:24.238 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-09-11T00:32:24.238 INFO:teuthology.orchestra.run.smithi012.stdout: "interval": { 2024-09-11T00:32:24.238 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "interval", 2024-09-11T00:32:24.238 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "secs", 2024-09-11T00:32:24.238 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.238 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:24.238 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "60", 2024-09-11T00:32:24.238 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.238 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.239 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.239 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "How frequently to reexamine health status", 2024-09-11T00:32:24.239 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.239 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.239 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.239 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.239 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-09-11T00:32:24.239 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-09-11T00:32:24.239 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:24.239 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.239 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:24.239 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-09-11T00:32:24.239 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.240 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.240 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-09-11T00:32:24.240 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-09-11T00:32:24.240 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-09-11T00:32:24.240 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-09-11T00:32:24.240 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-09-11T00:32:24.240 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-09-11T00:32:24.240 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-09-11T00:32:24.240 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-09-11T00:32:24.240 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.240 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.240 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.240 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.241 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.241 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-09-11T00:32:24.241 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-09-11T00:32:24.241 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-09-11T00:32:24.241 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.241 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:24.241 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-09-11T00:32:24.241 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.241 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.241 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.241 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.241 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.241 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.241 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.242 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.242 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-09-11T00:32:24.242 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-09-11T00:32:24.242 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:24.242 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.242 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:24.242 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-09-11T00:32:24.242 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.242 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.242 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-09-11T00:32:24.242 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-09-11T00:32:24.242 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-09-11T00:32:24.242 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-09-11T00:32:24.243 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-09-11T00:32:24.243 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-09-11T00:32:24.243 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-09-11T00:32:24.243 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-09-11T00:32:24.243 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.243 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.243 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.243 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.243 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.243 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-09-11T00:32:24.243 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-09-11T00:32:24.243 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-09-11T00:32:24.243 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.243 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:24.244 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-09-11T00:32:24.244 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.244 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.244 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.244 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.244 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.244 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.244 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.244 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.244 INFO:teuthology.orchestra.run.smithi012.stdout: "smtp_destination": { 2024-09-11T00:32:24.244 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "smtp_destination", 2024-09-11T00:32:24.244 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:24.244 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.245 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:24.245 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-09-11T00:32:24.245 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.245 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.245 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.245 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Email address to send alerts to, use commas to separate multiple", 2024-09-11T00:32:24.245 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.245 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.245 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.245 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.245 INFO:teuthology.orchestra.run.smithi012.stdout: "smtp_from_name": { 2024-09-11T00:32:24.246 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "smtp_from_name", 2024-09-11T00:32:24.246 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:24.246 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.246 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:24.246 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "Ceph", 2024-09-11T00:32:24.246 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.246 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.246 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.246 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Email From: name", 2024-09-11T00:32:24.246 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.247 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.247 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.247 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.247 INFO:teuthology.orchestra.run.smithi012.stdout: "smtp_host": { 2024-09-11T00:32:24.247 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "smtp_host", 2024-09-11T00:32:24.247 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:24.247 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.247 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:24.247 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-09-11T00:32:24.247 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.247 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.247 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.247 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "SMTP server", 2024-09-11T00:32:24.248 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.248 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.248 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.248 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.248 INFO:teuthology.orchestra.run.smithi012.stdout: "smtp_password": { 2024-09-11T00:32:24.248 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "smtp_password", 2024-09-11T00:32:24.248 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:24.248 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.248 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:24.248 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-09-11T00:32:24.248 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.249 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.249 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.249 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Password to authenticate with", 2024-09-11T00:32:24.249 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.249 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.249 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.249 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.249 INFO:teuthology.orchestra.run.smithi012.stdout: "smtp_port": { 2024-09-11T00:32:24.249 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "smtp_port", 2024-09-11T00:32:24.249 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-09-11T00:32:24.249 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.249 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:24.250 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "465", 2024-09-11T00:32:24.250 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.250 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.250 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.250 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "SMTP port", 2024-09-11T00:32:24.250 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.250 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.250 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.250 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.250 INFO:teuthology.orchestra.run.smithi012.stdout: "smtp_sender": { 2024-09-11T00:32:24.250 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "smtp_sender", 2024-09-11T00:32:24.250 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:24.250 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.251 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:24.251 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-09-11T00:32:24.251 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.251 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.251 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.251 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "SMTP envelope sender", 2024-09-11T00:32:24.251 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.251 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.251 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.251 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.251 INFO:teuthology.orchestra.run.smithi012.stdout: "smtp_ssl": { 2024-09-11T00:32:24.251 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "smtp_ssl", 2024-09-11T00:32:24.251 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-09-11T00:32:24.252 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.252 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:24.252 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-09-11T00:32:24.252 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.252 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.252 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.252 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Use SSL to connect to SMTP server", 2024-09-11T00:32:24.252 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.252 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.252 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.252 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.252 INFO:teuthology.orchestra.run.smithi012.stdout: "smtp_user": { 2024-09-11T00:32:24.253 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "smtp_user", 2024-09-11T00:32:24.253 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:24.253 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.253 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:24.253 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-09-11T00:32:24.253 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.253 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.253 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.253 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "User to authenticate as", 2024-09-11T00:32:24.253 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.253 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.253 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.253 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.254 INFO:teuthology.orchestra.run.smithi012.stdout: "sqlite3_killpoint": { 2024-09-11T00:32:24.254 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "sqlite3_killpoint", 2024-09-11T00:32:24.254 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-09-11T00:32:24.254 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "dev", 2024-09-11T00:32:24.254 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:24.254 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "0", 2024-09-11T00:32:24.254 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.254 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.254 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.254 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.254 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.254 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.255 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.255 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:32:24.255 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:32:24.255 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.255 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-11T00:32:24.255 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "balancer", 2024-09-11T00:32:24.255 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-09-11T00:32:24.255 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-09-11T00:32:24.255 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-09-11T00:32:24.255 INFO:teuthology.orchestra.run.smithi012.stdout: "active": { 2024-09-11T00:32:24.255 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "active", 2024-09-11T00:32:24.255 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-09-11T00:32:24.255 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.256 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:24.256 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-09-11T00:32:24.256 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.256 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.256 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.256 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "automatically balance PGs across cluster", 2024-09-11T00:32:24.256 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.256 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.256 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.256 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.256 INFO:teuthology.orchestra.run.smithi012.stdout: "begin_time": { 2024-09-11T00:32:24.257 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "begin_time", 2024-09-11T00:32:24.257 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:24.257 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.257 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:24.257 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "0000", 2024-09-11T00:32:24.257 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.257 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.257 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.257 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "beginning time of day to automatically balance", 2024-09-11T00:32:24.257 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-11T00:32:24.257 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.257 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.257 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.258 INFO:teuthology.orchestra.run.smithi012.stdout: "begin_weekday": { 2024-09-11T00:32:24.258 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "begin_weekday", 2024-09-11T00:32:24.258 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "uint", 2024-09-11T00:32:24.258 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.258 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:24.258 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "0", 2024-09-11T00:32:24.258 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "0", 2024-09-11T00:32:24.258 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "6", 2024-09-11T00:32:24.258 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.258 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-09-11T00:32:24.258 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-11T00:32:24.258 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.258 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.259 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.259 INFO:teuthology.orchestra.run.smithi012.stdout: "crush_compat_max_iterations": { 2024-09-11T00:32:24.259 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "crush_compat_max_iterations", 2024-09-11T00:32:24.259 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "uint", 2024-09-11T00:32:24.259 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.259 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:24.259 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "25", 2024-09-11T00:32:24.259 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "1", 2024-09-11T00:32:24.259 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "250", 2024-09-11T00:32:24.259 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.259 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-09-11T00:32:24.259 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.259 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.260 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.260 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.260 INFO:teuthology.orchestra.run.smithi012.stdout: "crush_compat_metrics": { 2024-09-11T00:32:24.260 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "crush_compat_metrics", 2024-09-11T00:32:24.260 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:24.260 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.260 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:24.260 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "pgs,objects,bytes", 2024-09-11T00:32:24.260 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.260 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.260 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.260 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-09-11T00:32:24.260 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-09-11T00:32:24.261 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.261 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.261 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.261 INFO:teuthology.orchestra.run.smithi012.stdout: "crush_compat_step": { 2024-09-11T00:32:24.261 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "crush_compat_step", 2024-09-11T00:32:24.261 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "float", 2024-09-11T00:32:24.261 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.261 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:24.261 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "0.5", 2024-09-11T00:32:24.261 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "0.001", 2024-09-11T00:32:24.261 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "0.999", 2024-09-11T00:32:24.261 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.261 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "aggressiveness of optimization", 2024-09-11T00:32:24.262 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-09-11T00:32:24.262 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.262 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.262 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.262 INFO:teuthology.orchestra.run.smithi012.stdout: "end_time": { 2024-09-11T00:32:24.262 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "end_time", 2024-09-11T00:32:24.262 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:24.262 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.262 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:24.262 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "2359", 2024-09-11T00:32:24.262 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.262 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.262 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.263 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "ending time of day to automatically balance", 2024-09-11T00:32:24.263 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-11T00:32:24.263 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.263 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.263 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.263 INFO:teuthology.orchestra.run.smithi012.stdout: "end_weekday": { 2024-09-11T00:32:24.263 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "end_weekday", 2024-09-11T00:32:24.263 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "uint", 2024-09-11T00:32:24.263 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.263 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:24.263 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "0", 2024-09-11T00:32:24.263 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "0", 2024-09-11T00:32:24.263 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "6", 2024-09-11T00:32:24.264 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.264 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-09-11T00:32:24.264 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-11T00:32:24.264 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.264 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.264 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.264 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-09-11T00:32:24.264 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-09-11T00:32:24.264 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:24.264 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.264 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:24.264 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-09-11T00:32:24.265 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.265 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.265 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-09-11T00:32:24.265 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-09-11T00:32:24.265 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-09-11T00:32:24.265 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-09-11T00:32:24.265 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-09-11T00:32:24.265 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-09-11T00:32:24.265 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-09-11T00:32:24.265 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-09-11T00:32:24.265 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.265 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.265 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.266 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.266 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.266 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-09-11T00:32:24.266 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-09-11T00:32:24.266 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-09-11T00:32:24.266 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.266 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:24.266 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-09-11T00:32:24.266 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.266 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.266 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.266 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.267 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.267 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.267 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.267 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.267 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-09-11T00:32:24.267 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-09-11T00:32:24.267 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:24.267 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.267 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:24.267 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-09-11T00:32:24.267 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.268 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.268 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-09-11T00:32:24.268 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-09-11T00:32:24.268 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-09-11T00:32:24.268 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-09-11T00:32:24.268 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-09-11T00:32:24.268 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-09-11T00:32:24.268 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-09-11T00:32:24.268 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-09-11T00:32:24.268 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.268 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.268 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.268 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.269 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.269 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-09-11T00:32:24.269 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-09-11T00:32:24.269 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-09-11T00:32:24.269 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.269 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:24.269 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-09-11T00:32:24.269 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.269 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.269 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.269 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.269 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.269 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.269 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.270 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.270 INFO:teuthology.orchestra.run.smithi012.stdout: "min_score": { 2024-09-11T00:32:24.270 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "min_score", 2024-09-11T00:32:24.270 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "float", 2024-09-11T00:32:24.270 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.270 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:24.270 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "0", 2024-09-11T00:32:24.270 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.270 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.270 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.270 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-09-11T00:32:24.270 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.270 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.271 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.271 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.271 INFO:teuthology.orchestra.run.smithi012.stdout: "mode": { 2024-09-11T00:32:24.271 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "mode", 2024-09-11T00:32:24.271 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:24.271 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.271 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:24.271 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "upmap", 2024-09-11T00:32:24.271 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.271 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.271 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-09-11T00:32:24.271 INFO:teuthology.orchestra.run.smithi012.stdout: "crush-compat", 2024-09-11T00:32:24.271 INFO:teuthology.orchestra.run.smithi012.stdout: "none", 2024-09-11T00:32:24.272 INFO:teuthology.orchestra.run.smithi012.stdout: "read", 2024-09-11T00:32:24.272 INFO:teuthology.orchestra.run.smithi012.stdout: "upmap", 2024-09-11T00:32:24.272 INFO:teuthology.orchestra.run.smithi012.stdout: "upmap-read" 2024-09-11T00:32:24.272 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-09-11T00:32:24.272 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Balancer mode", 2024-09-11T00:32:24.272 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.272 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.272 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.272 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.272 INFO:teuthology.orchestra.run.smithi012.stdout: "pool_ids": { 2024-09-11T00:32:24.272 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "pool_ids", 2024-09-11T00:32:24.272 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:24.272 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.273 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:24.273 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-09-11T00:32:24.273 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.273 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.273 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.273 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-09-11T00:32:24.273 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.273 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.273 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.273 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.273 INFO:teuthology.orchestra.run.smithi012.stdout: "sleep_interval": { 2024-09-11T00:32:24.273 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "sleep_interval", 2024-09-11T00:32:24.274 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "secs", 2024-09-11T00:32:24.274 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.274 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:24.274 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "60", 2024-09-11T00:32:24.274 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.274 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.274 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.274 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-09-11T00:32:24.274 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.275 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.275 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.275 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.275 INFO:teuthology.orchestra.run.smithi012.stdout: "sqlite3_killpoint": { 2024-09-11T00:32:24.275 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "sqlite3_killpoint", 2024-09-11T00:32:24.275 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-09-11T00:32:24.275 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "dev", 2024-09-11T00:32:24.275 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:24.276 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "0", 2024-09-11T00:32:24.276 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.276 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.276 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.276 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.276 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.276 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.276 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.276 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.276 INFO:teuthology.orchestra.run.smithi012.stdout: "upmap_max_deviation": { 2024-09-11T00:32:24.276 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "upmap_max_deviation", 2024-09-11T00:32:24.277 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-09-11T00:32:24.277 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.277 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:24.277 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "5", 2024-09-11T00:32:24.277 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "1", 2024-09-11T00:32:24.277 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.277 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.277 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "deviation below which no optimization is attempted", 2024-09-11T00:32:24.277 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-09-11T00:32:24.277 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.277 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.278 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.278 INFO:teuthology.orchestra.run.smithi012.stdout: "upmap_max_optimizations": { 2024-09-11T00:32:24.278 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "upmap_max_optimizations", 2024-09-11T00:32:24.278 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "uint", 2024-09-11T00:32:24.278 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.278 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:24.278 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "10", 2024-09-11T00:32:24.278 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.278 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.278 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.278 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-09-11T00:32:24.278 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.278 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.278 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.279 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:32:24.279 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:32:24.279 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.279 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-11T00:32:24.279 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "cephadm", 2024-09-11T00:32:24.279 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-09-11T00:32:24.279 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-09-11T00:32:24.279 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-09-11T00:32:24.279 INFO:teuthology.orchestra.run.smithi012.stdout: "agent_down_multiplier": { 2024-09-11T00:32:24.279 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "agent_down_multiplier", 2024-09-11T00:32:24.279 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "float", 2024-09-11T00:32:24.279 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.279 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:24.279 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "3.0", 2024-09-11T00:32:24.280 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.280 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.280 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.280 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-09-11T00:32:24.280 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.280 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.280 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.280 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.280 INFO:teuthology.orchestra.run.smithi012.stdout: "agent_refresh_rate": { 2024-09-11T00:32:24.280 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "agent_refresh_rate", 2024-09-11T00:32:24.280 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "secs", 2024-09-11T00:32:24.280 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.280 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:24.281 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "20", 2024-09-11T00:32:24.281 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.281 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.281 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.281 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-09-11T00:32:24.281 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.281 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.281 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.281 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.281 INFO:teuthology.orchestra.run.smithi012.stdout: "agent_starting_port": { 2024-09-11T00:32:24.281 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "agent_starting_port", 2024-09-11T00:32:24.281 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-09-11T00:32:24.282 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.282 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:24.282 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "4721", 2024-09-11T00:32:24.282 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.282 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.282 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.282 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-09-11T00:32:24.282 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.282 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.282 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.282 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.282 INFO:teuthology.orchestra.run.smithi012.stdout: "allow_ptrace": { 2024-09-11T00:32:24.282 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "allow_ptrace", 2024-09-11T00:32:24.283 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-09-11T00:32:24.283 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.283 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:24.283 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-09-11T00:32:24.283 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.283 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.283 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.283 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-09-11T00:32:24.283 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-09-11T00:32:24.283 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.283 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.283 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.283 INFO:teuthology.orchestra.run.smithi012.stdout: "autotune_interval": { 2024-09-11T00:32:24.283 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "autotune_interval", 2024-09-11T00:32:24.284 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "secs", 2024-09-11T00:32:24.284 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.284 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:24.284 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "600", 2024-09-11T00:32:24.284 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.284 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.284 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.284 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "how frequently to autotune daemon memory", 2024-09-11T00:32:24.284 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.284 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.284 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.284 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.284 INFO:teuthology.orchestra.run.smithi012.stdout: "autotune_memory_target_ratio": { 2024-09-11T00:32:24.285 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "autotune_memory_target_ratio", 2024-09-11T00:32:24.285 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "float", 2024-09-11T00:32:24.285 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.285 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:24.285 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "0.7", 2024-09-11T00:32:24.285 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.285 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.285 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.285 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-09-11T00:32:24.285 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.285 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.285 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.286 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.286 INFO:teuthology.orchestra.run.smithi012.stdout: "cephadm_log_destination": { 2024-09-11T00:32:24.286 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "cephadm_log_destination", 2024-09-11T00:32:24.286 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:24.286 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.286 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:24.286 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-09-11T00:32:24.286 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.286 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.286 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-09-11T00:32:24.286 INFO:teuthology.orchestra.run.smithi012.stdout: "file", 2024-09-11T00:32:24.286 INFO:teuthology.orchestra.run.smithi012.stdout: "file,syslog", 2024-09-11T00:32:24.286 INFO:teuthology.orchestra.run.smithi012.stdout: "syslog" 2024-09-11T00:32:24.287 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-09-11T00:32:24.287 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-09-11T00:32:24.287 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.287 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.287 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.287 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.287 INFO:teuthology.orchestra.run.smithi012.stdout: "cgroups_split": { 2024-09-11T00:32:24.287 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "cgroups_split", 2024-09-11T00:32:24.287 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-09-11T00:32:24.287 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.287 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:24.287 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-09-11T00:32:24.287 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.288 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.288 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.288 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-09-11T00:32:24.288 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.288 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.288 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.288 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.288 INFO:teuthology.orchestra.run.smithi012.stdout: "config_checks_enabled": { 2024-09-11T00:32:24.288 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "config_checks_enabled", 2024-09-11T00:32:24.288 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-09-11T00:32:24.288 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.288 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:24.289 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-09-11T00:32:24.289 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.289 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.289 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.289 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-09-11T00:32:24.289 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.289 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.289 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.289 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.289 INFO:teuthology.orchestra.run.smithi012.stdout: "config_dashboard": { 2024-09-11T00:32:24.289 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "config_dashboard", 2024-09-11T00:32:24.289 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-09-11T00:32:24.289 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.290 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:24.290 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-09-11T00:32:24.290 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.290 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.290 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.290 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-09-11T00:32:24.290 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.290 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.290 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.290 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.290 INFO:teuthology.orchestra.run.smithi012.stdout: "container_image_alertmanager": { 2024-09-11T00:32:24.290 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "container_image_alertmanager", 2024-09-11T00:32:24.290 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:24.290 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.291 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:24.291 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.27.0", 2024-09-11T00:32:24.291 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.291 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.291 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.291 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Prometheus container image", 2024-09-11T00:32:24.291 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.291 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.291 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.291 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.291 INFO:teuthology.orchestra.run.smithi012.stdout: "container_image_base": { 2024-09-11T00:32:24.291 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "container_image_base", 2024-09-11T00:32:24.291 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:24.291 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.292 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:24.292 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "quay.io/ceph/ceph", 2024-09-11T00:32:24.292 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.292 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.292 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.292 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Container image name, without the tag", 2024-09-11T00:32:24.292 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.292 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.292 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.292 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.292 INFO:teuthology.orchestra.run.smithi012.stdout: "container_image_elasticsearch": { 2024-09-11T00:32:24.292 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "container_image_elasticsearch", 2024-09-11T00:32:24.292 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:24.292 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.293 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:24.293 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-09-11T00:32:24.293 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.293 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.293 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.293 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "elasticsearch container image", 2024-09-11T00:32:24.293 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.293 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.293 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.293 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.293 INFO:teuthology.orchestra.run.smithi012.stdout: "container_image_grafana": { 2024-09-11T00:32:24.293 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "container_image_grafana", 2024-09-11T00:32:24.294 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:24.294 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.294 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:24.294 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "quay.io/ceph/grafana:10.4.0", 2024-09-11T00:32:24.294 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.294 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.294 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.294 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Prometheus container image", 2024-09-11T00:32:24.294 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.294 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.294 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.294 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.295 INFO:teuthology.orchestra.run.smithi012.stdout: "container_image_haproxy": { 2024-09-11T00:32:24.295 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "container_image_haproxy", 2024-09-11T00:32:24.295 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:24.295 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.295 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:24.295 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-09-11T00:32:24.295 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.295 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.295 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.295 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "HAproxy container image", 2024-09-11T00:32:24.295 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.295 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.295 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.296 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.296 INFO:teuthology.orchestra.run.smithi012.stdout: "container_image_jaeger_agent": { 2024-09-11T00:32:24.296 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "container_image_jaeger_agent", 2024-09-11T00:32:24.296 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:24.296 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.296 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:24.296 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-09-11T00:32:24.296 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.296 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.296 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.296 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Jaeger agent container image", 2024-09-11T00:32:24.296 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.296 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.297 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.297 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.297 INFO:teuthology.orchestra.run.smithi012.stdout: "container_image_jaeger_collector": { 2024-09-11T00:32:24.297 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "container_image_jaeger_collector", 2024-09-11T00:32:24.297 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:24.297 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.297 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:24.297 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-09-11T00:32:24.297 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.297 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.298 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.298 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Jaeger collector container image", 2024-09-11T00:32:24.298 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.298 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.298 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.298 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.298 INFO:teuthology.orchestra.run.smithi012.stdout: "container_image_jaeger_query": { 2024-09-11T00:32:24.298 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "container_image_jaeger_query", 2024-09-11T00:32:24.298 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:24.299 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.299 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:24.299 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-09-11T00:32:24.299 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.299 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.299 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.299 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Jaeger query container image", 2024-09-11T00:32:24.299 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.299 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.299 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.299 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.299 INFO:teuthology.orchestra.run.smithi012.stdout: "container_image_keepalived": { 2024-09-11T00:32:24.299 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "container_image_keepalived", 2024-09-11T00:32:24.300 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:24.300 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.300 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:24.300 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-09-11T00:32:24.300 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.300 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.300 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.300 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Keepalived container image", 2024-09-11T00:32:24.300 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.300 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.300 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.300 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.301 INFO:teuthology.orchestra.run.smithi012.stdout: "container_image_loki": { 2024-09-11T00:32:24.301 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "container_image_loki", 2024-09-11T00:32:24.301 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:24.301 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.301 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:24.301 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "docker.io/grafana/loki:3.0.0", 2024-09-11T00:32:24.301 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.301 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.301 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.301 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Loki container image", 2024-09-11T00:32:24.301 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.301 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.301 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.302 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.302 INFO:teuthology.orchestra.run.smithi012.stdout: "container_image_nginx": { 2024-09-11T00:32:24.302 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "container_image_nginx", 2024-09-11T00:32:24.302 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:24.302 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.302 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:24.302 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "quay.io/ceph/nginx:1.26.1", 2024-09-11T00:32:24.302 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.302 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.302 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.302 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Nginx container image", 2024-09-11T00:32:24.302 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.302 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.303 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.303 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.303 INFO:teuthology.orchestra.run.smithi012.stdout: "container_image_node_exporter": { 2024-09-11T00:32:24.303 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "container_image_node_exporter", 2024-09-11T00:32:24.303 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:24.303 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.303 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:24.303 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.7.0", 2024-09-11T00:32:24.303 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.303 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.303 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.303 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Prometheus container image", 2024-09-11T00:32:24.303 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.303 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.304 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.304 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.304 INFO:teuthology.orchestra.run.smithi012.stdout: "container_image_nvmeof": { 2024-09-11T00:32:24.304 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "container_image_nvmeof", 2024-09-11T00:32:24.304 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:24.304 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.304 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:24.304 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.17", 2024-09-11T00:32:24.304 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.304 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.304 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.305 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Nvme-of container image", 2024-09-11T00:32:24.305 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.305 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.305 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.305 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.305 INFO:teuthology.orchestra.run.smithi012.stdout: "container_image_oauth2_proxy": { 2024-09-11T00:32:24.305 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "container_image_oauth2_proxy", 2024-09-11T00:32:24.305 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:24.305 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.306 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:24.306 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "quay.io/oauth2-proxy/oauth2-proxy:v7.6.0", 2024-09-11T00:32:24.306 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.306 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.306 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.306 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "oauth2-proxy container image", 2024-09-11T00:32:24.306 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.306 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.306 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.307 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.307 INFO:teuthology.orchestra.run.smithi012.stdout: "container_image_prometheus": { 2024-09-11T00:32:24.307 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "container_image_prometheus", 2024-09-11T00:32:24.307 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:24.307 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.307 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:24.307 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "quay.io/prometheus/prometheus:v2.51.0", 2024-09-11T00:32:24.307 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.307 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.307 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.307 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Prometheus container image", 2024-09-11T00:32:24.307 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.307 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.308 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.308 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.308 INFO:teuthology.orchestra.run.smithi012.stdout: "container_image_promtail": { 2024-09-11T00:32:24.308 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "container_image_promtail", 2024-09-11T00:32:24.308 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:24.308 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.308 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:24.308 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "docker.io/grafana/promtail:3.0.0", 2024-09-11T00:32:24.308 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.308 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.308 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.308 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Promtail container image", 2024-09-11T00:32:24.308 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.309 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.309 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.309 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.309 INFO:teuthology.orchestra.run.smithi012.stdout: "container_image_samba": { 2024-09-11T00:32:24.309 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "container_image_samba", 2024-09-11T00:32:24.309 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:24.309 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.309 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:24.309 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-09-11T00:32:24.309 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.309 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.309 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.310 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Samba/SMB container image", 2024-09-11T00:32:24.310 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.310 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.310 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.310 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.310 INFO:teuthology.orchestra.run.smithi012.stdout: "container_image_samba_metrics": { 2024-09-11T00:32:24.310 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "container_image_samba_metrics", 2024-09-11T00:32:24.310 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:24.310 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.310 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:24.310 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "quay.io/samba.org/samba-metrics:latest", 2024-09-11T00:32:24.310 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.310 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.311 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.311 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Samba/SMB metrics exporter container image", 2024-09-11T00:32:24.311 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.311 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.311 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.311 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.311 INFO:teuthology.orchestra.run.smithi012.stdout: "container_image_snmp_gateway": { 2024-09-11T00:32:24.311 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "container_image_snmp_gateway", 2024-09-11T00:32:24.311 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:24.311 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.311 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:24.311 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-09-11T00:32:24.311 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.312 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.312 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.312 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "SNMP Gateway container image", 2024-09-11T00:32:24.312 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.312 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.312 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.312 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.312 INFO:teuthology.orchestra.run.smithi012.stdout: "container_init": { 2024-09-11T00:32:24.312 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "container_init", 2024-09-11T00:32:24.312 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-09-11T00:32:24.312 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.312 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:24.313 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-09-11T00:32:24.313 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.313 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.313 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.313 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Run podman/docker with `--init`", 2024-09-11T00:32:24.313 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.313 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.313 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.313 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.313 INFO:teuthology.orchestra.run.smithi012.stdout: "daemon_cache_timeout": { 2024-09-11T00:32:24.313 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "daemon_cache_timeout", 2024-09-11T00:32:24.313 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "secs", 2024-09-11T00:32:24.314 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.314 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:24.314 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "600", 2024-09-11T00:32:24.314 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.314 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.314 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.314 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-09-11T00:32:24.314 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.314 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.314 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.315 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.315 INFO:teuthology.orchestra.run.smithi012.stdout: "default_cephadm_command_timeout": { 2024-09-11T00:32:24.315 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "default_cephadm_command_timeout", 2024-09-11T00:32:24.315 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-09-11T00:32:24.315 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.315 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:24.315 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "900", 2024-09-11T00:32:24.315 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.315 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.315 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.315 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-09-11T00:32:24.315 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.315 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.316 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.316 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.316 INFO:teuthology.orchestra.run.smithi012.stdout: "default_registry": { 2024-09-11T00:32:24.316 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "default_registry", 2024-09-11T00:32:24.316 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:24.316 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.316 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:24.316 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "docker.io", 2024-09-11T00:32:24.316 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.316 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.316 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.316 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-09-11T00:32:24.317 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.317 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.317 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.317 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.317 INFO:teuthology.orchestra.run.smithi012.stdout: "device_cache_timeout": { 2024-09-11T00:32:24.317 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "device_cache_timeout", 2024-09-11T00:32:24.317 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "secs", 2024-09-11T00:32:24.317 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.317 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:24.317 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "1800", 2024-09-11T00:32:24.317 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.317 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.317 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.317 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "seconds to cache device inventory", 2024-09-11T00:32:24.318 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.318 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.318 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.318 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.318 INFO:teuthology.orchestra.run.smithi012.stdout: "device_enhanced_scan": { 2024-09-11T00:32:24.318 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "device_enhanced_scan", 2024-09-11T00:32:24.318 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-09-11T00:32:24.318 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.318 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:24.318 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-09-11T00:32:24.318 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.318 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.319 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.319 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Use libstoragemgmt during device scans", 2024-09-11T00:32:24.319 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.319 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.319 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.319 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.319 INFO:teuthology.orchestra.run.smithi012.stdout: "facts_cache_timeout": { 2024-09-11T00:32:24.319 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "facts_cache_timeout", 2024-09-11T00:32:24.319 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "secs", 2024-09-11T00:32:24.319 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.319 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:24.319 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "60", 2024-09-11T00:32:24.319 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.319 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.320 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.320 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "seconds to cache host facts data", 2024-09-11T00:32:24.320 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.320 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.320 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.320 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.320 INFO:teuthology.orchestra.run.smithi012.stdout: "grafana_dashboards_path": { 2024-09-11T00:32:24.320 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "grafana_dashboards_path", 2024-09-11T00:32:24.320 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:24.320 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.320 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:24.320 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-09-11T00:32:24.320 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.321 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.321 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.321 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-09-11T00:32:24.321 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.321 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.321 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.321 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.321 INFO:teuthology.orchestra.run.smithi012.stdout: "host_check_interval": { 2024-09-11T00:32:24.321 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "host_check_interval", 2024-09-11T00:32:24.321 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "secs", 2024-09-11T00:32:24.322 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.322 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:24.322 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "600", 2024-09-11T00:32:24.322 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.322 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.322 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.322 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "how frequently to perform a host check", 2024-09-11T00:32:24.322 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.322 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.322 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.322 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.322 INFO:teuthology.orchestra.run.smithi012.stdout: "hw_monitoring": { 2024-09-11T00:32:24.323 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "hw_monitoring", 2024-09-11T00:32:24.323 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-09-11T00:32:24.323 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.323 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:24.323 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-09-11T00:32:24.323 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.323 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.323 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.323 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-09-11T00:32:24.323 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.323 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.323 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.323 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.324 INFO:teuthology.orchestra.run.smithi012.stdout: "inventory_list_all": { 2024-09-11T00:32:24.324 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "inventory_list_all", 2024-09-11T00:32:24.324 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-09-11T00:32:24.324 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.324 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:24.324 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-09-11T00:32:24.324 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.324 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.324 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.324 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-09-11T00:32:24.324 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.324 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.325 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.325 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.325 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-09-11T00:32:24.325 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-09-11T00:32:24.325 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:24.325 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.325 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:24.325 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-09-11T00:32:24.325 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.325 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.325 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-09-11T00:32:24.325 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-09-11T00:32:24.325 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-09-11T00:32:24.326 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-09-11T00:32:24.326 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-09-11T00:32:24.326 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-09-11T00:32:24.326 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-09-11T00:32:24.326 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-09-11T00:32:24.326 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.326 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.326 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.326 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.326 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.326 INFO:teuthology.orchestra.run.smithi012.stdout: "log_refresh_metadata": { 2024-09-11T00:32:24.326 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_refresh_metadata", 2024-09-11T00:32:24.326 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-09-11T00:32:24.327 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.327 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:24.327 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-09-11T00:32:24.327 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.327 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.327 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.327 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-09-11T00:32:24.327 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.327 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.327 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.327 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.327 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-09-11T00:32:24.328 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-09-11T00:32:24.328 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-09-11T00:32:24.328 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.328 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:24.328 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-09-11T00:32:24.328 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.328 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.328 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.328 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-09-11T00:32:24.328 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.328 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.328 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.329 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.329 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-09-11T00:32:24.329 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-09-11T00:32:24.329 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:24.329 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.329 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:24.329 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-09-11T00:32:24.329 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.329 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.329 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-09-11T00:32:24.329 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-09-11T00:32:24.329 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-09-11T00:32:24.329 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-09-11T00:32:24.329 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-09-11T00:32:24.330 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-09-11T00:32:24.330 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-09-11T00:32:24.330 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-09-11T00:32:24.330 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.330 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.330 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.330 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.330 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.330 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-09-11T00:32:24.330 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-09-11T00:32:24.330 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-09-11T00:32:24.330 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.331 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:24.331 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-09-11T00:32:24.331 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.331 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.331 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.331 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.331 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.331 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.331 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.331 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.331 INFO:teuthology.orchestra.run.smithi012.stdout: "manage_etc_ceph_ceph_conf": { 2024-09-11T00:32:24.331 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-09-11T00:32:24.331 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-09-11T00:32:24.332 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.332 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:24.332 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-09-11T00:32:24.332 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.332 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.332 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.332 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-09-11T00:32:24.332 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.332 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.332 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.332 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.332 INFO:teuthology.orchestra.run.smithi012.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-09-11T00:32:24.333 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-09-11T00:32:24.333 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:24.333 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.333 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:24.333 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "*", 2024-09-11T00:32:24.333 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.333 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.333 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.333 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-09-11T00:32:24.333 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.333 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.333 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.333 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.334 INFO:teuthology.orchestra.run.smithi012.stdout: "max_count_per_host": { 2024-09-11T00:32:24.334 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "max_count_per_host", 2024-09-11T00:32:24.334 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-09-11T00:32:24.334 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.334 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:24.334 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "10", 2024-09-11T00:32:24.334 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.334 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.334 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.334 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "max number of daemons per service per host", 2024-09-11T00:32:24.334 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.334 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.334 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.334 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.335 INFO:teuthology.orchestra.run.smithi012.stdout: "max_osd_draining_count": { 2024-09-11T00:32:24.335 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "max_osd_draining_count", 2024-09-11T00:32:24.335 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-09-11T00:32:24.335 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.335 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:24.335 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "10", 2024-09-11T00:32:24.335 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.335 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.335 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.335 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-09-11T00:32:24.335 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.335 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.335 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.336 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.336 INFO:teuthology.orchestra.run.smithi012.stdout: "migration_current": { 2024-09-11T00:32:24.336 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "migration_current", 2024-09-11T00:32:24.336 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-09-11T00:32:24.336 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.336 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:24.336 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-09-11T00:32:24.336 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.336 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.336 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.336 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "internal - do not modify", 2024-09-11T00:32:24.336 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.336 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.337 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.337 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.337 INFO:teuthology.orchestra.run.smithi012.stdout: "mode": { 2024-09-11T00:32:24.337 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "mode", 2024-09-11T00:32:24.337 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:24.337 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.337 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:24.337 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "root", 2024-09-11T00:32:24.337 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.337 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.337 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-09-11T00:32:24.337 INFO:teuthology.orchestra.run.smithi012.stdout: "cephadm-package", 2024-09-11T00:32:24.337 INFO:teuthology.orchestra.run.smithi012.stdout: "root" 2024-09-11T00:32:24.338 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-09-11T00:32:24.338 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "mode for remote execution of cephadm", 2024-09-11T00:32:24.338 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.338 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.338 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.338 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.338 INFO:teuthology.orchestra.run.smithi012.stdout: "oob_default_addr": { 2024-09-11T00:32:24.338 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "oob_default_addr", 2024-09-11T00:32:24.338 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:24.338 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.338 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:24.338 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "169.254.1.1", 2024-09-11T00:32:24.338 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.339 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.339 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.339 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Default address for RedFish API (oob management).", 2024-09-11T00:32:24.339 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.339 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.339 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.339 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.339 INFO:teuthology.orchestra.run.smithi012.stdout: "prometheus_alerts_path": { 2024-09-11T00:32:24.339 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "prometheus_alerts_path", 2024-09-11T00:32:24.339 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:24.339 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.339 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:24.339 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-09-11T00:32:24.340 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.340 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.340 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.340 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-09-11T00:32:24.340 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.340 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.340 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.340 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.340 INFO:teuthology.orchestra.run.smithi012.stdout: "registry_insecure": { 2024-09-11T00:32:24.340 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "registry_insecure", 2024-09-11T00:32:24.340 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-09-11T00:32:24.340 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.340 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:24.340 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-09-11T00:32:24.341 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.341 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.341 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.341 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-09-11T00:32:24.341 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.341 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.341 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.341 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.341 INFO:teuthology.orchestra.run.smithi012.stdout: "registry_password": { 2024-09-11T00:32:24.341 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "registry_password", 2024-09-11T00:32:24.341 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:24.341 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.341 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:24.342 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-09-11T00:32:24.342 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.342 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.342 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.342 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-09-11T00:32:24.342 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.342 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.342 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.342 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.342 INFO:teuthology.orchestra.run.smithi012.stdout: "registry_url": { 2024-09-11T00:32:24.342 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "registry_url", 2024-09-11T00:32:24.342 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:24.342 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.342 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:24.343 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-09-11T00:32:24.343 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.343 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.343 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.343 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-09-11T00:32:24.343 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.343 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.343 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.343 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.343 INFO:teuthology.orchestra.run.smithi012.stdout: "registry_username": { 2024-09-11T00:32:24.343 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "registry_username", 2024-09-11T00:32:24.344 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:24.344 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.344 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:24.344 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-09-11T00:32:24.344 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.344 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.344 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.344 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-09-11T00:32:24.345 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.345 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.345 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.345 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.345 INFO:teuthology.orchestra.run.smithi012.stdout: "secure_monitoring_stack": { 2024-09-11T00:32:24.345 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "secure_monitoring_stack", 2024-09-11T00:32:24.345 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-09-11T00:32:24.345 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.345 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:24.345 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-09-11T00:32:24.345 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.345 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.345 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.346 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-09-11T00:32:24.346 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.346 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.346 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.346 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.346 INFO:teuthology.orchestra.run.smithi012.stdout: "service_discovery_port": { 2024-09-11T00:32:24.346 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "service_discovery_port", 2024-09-11T00:32:24.346 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-09-11T00:32:24.346 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.346 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:24.346 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "8765", 2024-09-11T00:32:24.346 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.346 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.347 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.347 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "cephadm service discovery port", 2024-09-11T00:32:24.347 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.347 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.347 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.347 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.347 INFO:teuthology.orchestra.run.smithi012.stdout: "sqlite3_killpoint": { 2024-09-11T00:32:24.347 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "sqlite3_killpoint", 2024-09-11T00:32:24.347 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-09-11T00:32:24.347 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "dev", 2024-09-11T00:32:24.347 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:24.347 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "0", 2024-09-11T00:32:24.347 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.348 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.348 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.348 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.348 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.348 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.348 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.348 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.348 INFO:teuthology.orchestra.run.smithi012.stdout: "ssh_config_file": { 2024-09-11T00:32:24.348 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "ssh_config_file", 2024-09-11T00:32:24.348 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:24.348 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.348 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:24.348 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-09-11T00:32:24.349 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.349 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.349 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.349 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-09-11T00:32:24.349 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.349 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.349 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.349 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.349 INFO:teuthology.orchestra.run.smithi012.stdout: "use_agent": { 2024-09-11T00:32:24.349 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "use_agent", 2024-09-11T00:32:24.349 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-09-11T00:32:24.349 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.349 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:24.350 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-09-11T00:32:24.350 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.350 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.350 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.350 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-09-11T00:32:24.350 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.350 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.350 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.350 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.350 INFO:teuthology.orchestra.run.smithi012.stdout: "use_repo_digest": { 2024-09-11T00:32:24.350 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "use_repo_digest", 2024-09-11T00:32:24.350 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-09-11T00:32:24.350 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.351 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:24.351 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-09-11T00:32:24.351 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.351 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.351 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.351 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-09-11T00:32:24.351 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.351 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.351 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.351 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.351 INFO:teuthology.orchestra.run.smithi012.stdout: "warn_on_failed_host_check": { 2024-09-11T00:32:24.351 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "warn_on_failed_host_check", 2024-09-11T00:32:24.351 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-09-11T00:32:24.352 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.352 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:24.352 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-09-11T00:32:24.352 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.352 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.352 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.352 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "raise a health warning if the host check fails", 2024-09-11T00:32:24.352 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.352 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.352 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.352 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.352 INFO:teuthology.orchestra.run.smithi012.stdout: "warn_on_stray_daemons": { 2024-09-11T00:32:24.352 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "warn_on_stray_daemons", 2024-09-11T00:32:24.353 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-09-11T00:32:24.353 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.353 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:24.353 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-09-11T00:32:24.353 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.353 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.353 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.353 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-09-11T00:32:24.353 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.353 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.353 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.353 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.353 INFO:teuthology.orchestra.run.smithi012.stdout: "warn_on_stray_hosts": { 2024-09-11T00:32:24.354 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "warn_on_stray_hosts", 2024-09-11T00:32:24.354 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-09-11T00:32:24.354 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.354 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:24.354 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-09-11T00:32:24.354 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.354 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.354 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.354 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-09-11T00:32:24.354 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.354 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.354 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.355 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:32:24.355 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:32:24.355 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.355 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-11T00:32:24.355 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "crash", 2024-09-11T00:32:24.355 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-09-11T00:32:24.355 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-09-11T00:32:24.355 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-09-11T00:32:24.355 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-09-11T00:32:24.355 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-09-11T00:32:24.355 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:24.355 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.356 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:24.356 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-09-11T00:32:24.356 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.356 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.356 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-09-11T00:32:24.356 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-09-11T00:32:24.356 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-09-11T00:32:24.356 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-09-11T00:32:24.356 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-09-11T00:32:24.356 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-09-11T00:32:24.356 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-09-11T00:32:24.356 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-09-11T00:32:24.357 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.357 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.357 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.357 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.357 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.357 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-09-11T00:32:24.357 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-09-11T00:32:24.357 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-09-11T00:32:24.357 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.357 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:24.357 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-09-11T00:32:24.357 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.357 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.357 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.358 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.358 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.358 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.358 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.358 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.358 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-09-11T00:32:24.358 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-09-11T00:32:24.358 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:24.358 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.358 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:24.358 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-09-11T00:32:24.358 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.358 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.358 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-09-11T00:32:24.359 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-09-11T00:32:24.359 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-09-11T00:32:24.359 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-09-11T00:32:24.359 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-09-11T00:32:24.359 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-09-11T00:32:24.359 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-09-11T00:32:24.359 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-09-11T00:32:24.359 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.359 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.359 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.359 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.359 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.359 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-09-11T00:32:24.360 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-09-11T00:32:24.360 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-09-11T00:32:24.360 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.360 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:24.360 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-09-11T00:32:24.360 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.360 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.360 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.360 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.360 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.360 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.360 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.360 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.361 INFO:teuthology.orchestra.run.smithi012.stdout: "retain_interval": { 2024-09-11T00:32:24.361 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "retain_interval", 2024-09-11T00:32:24.361 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "secs", 2024-09-11T00:32:24.361 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.361 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:24.361 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "31536000", 2024-09-11T00:32:24.361 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.361 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.361 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.361 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "how long to retain crashes before pruning them", 2024-09-11T00:32:24.361 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.361 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.361 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.361 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.362 INFO:teuthology.orchestra.run.smithi012.stdout: "sqlite3_killpoint": { 2024-09-11T00:32:24.362 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "sqlite3_killpoint", 2024-09-11T00:32:24.362 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-09-11T00:32:24.362 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "dev", 2024-09-11T00:32:24.362 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:24.362 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "0", 2024-09-11T00:32:24.362 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.362 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.362 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.362 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.362 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.362 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.362 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.362 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.363 INFO:teuthology.orchestra.run.smithi012.stdout: "warn_recent_interval": { 2024-09-11T00:32:24.363 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "warn_recent_interval", 2024-09-11T00:32:24.363 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "secs", 2024-09-11T00:32:24.363 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.363 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:24.363 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "1209600", 2024-09-11T00:32:24.363 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.363 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.363 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.363 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "time interval in which to warn about recent crashes", 2024-09-11T00:32:24.363 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.363 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.363 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.363 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:32:24.364 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:32:24.364 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.364 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-11T00:32:24.364 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "dashboard", 2024-09-11T00:32:24.364 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-09-11T00:32:24.364 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-09-11T00:32:24.364 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-09-11T00:32:24.364 INFO:teuthology.orchestra.run.smithi012.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-09-11T00:32:24.364 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-09-11T00:32:24.364 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-09-11T00:32:24.364 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.364 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:24.365 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "10", 2024-09-11T00:32:24.365 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.365 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.365 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.365 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.365 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.365 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.365 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.365 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.365 INFO:teuthology.orchestra.run.smithi012.stdout: "ALERTMANAGER_API_HOST": { 2024-09-11T00:32:24.365 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "ALERTMANAGER_API_HOST", 2024-09-11T00:32:24.365 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:24.366 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.366 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:24.366 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-09-11T00:32:24.366 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.366 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.366 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.366 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.366 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.366 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.366 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.366 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.366 INFO:teuthology.orchestra.run.smithi012.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-09-11T00:32:24.366 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-09-11T00:32:24.366 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-09-11T00:32:24.367 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.367 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:24.367 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-09-11T00:32:24.367 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.367 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.367 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.367 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.367 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.367 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.367 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.367 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.367 INFO:teuthology.orchestra.run.smithi012.stdout: "AUDIT_API_ENABLED": { 2024-09-11T00:32:24.367 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "AUDIT_API_ENABLED", 2024-09-11T00:32:24.368 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-09-11T00:32:24.368 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.368 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:24.368 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-09-11T00:32:24.368 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.368 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.368 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.368 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.368 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.368 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.368 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.368 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.368 INFO:teuthology.orchestra.run.smithi012.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-09-11T00:32:24.369 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-09-11T00:32:24.369 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-09-11T00:32:24.369 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.369 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:24.369 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-09-11T00:32:24.369 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.369 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.369 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.369 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.369 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.369 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.369 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.370 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.370 INFO:teuthology.orchestra.run.smithi012.stdout: "ENABLE_BROWSABLE_API": { 2024-09-11T00:32:24.370 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "ENABLE_BROWSABLE_API", 2024-09-11T00:32:24.370 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-09-11T00:32:24.370 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.370 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:24.370 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-09-11T00:32:24.370 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.370 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.370 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.370 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.370 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.370 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.370 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.371 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.371 INFO:teuthology.orchestra.run.smithi012.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-09-11T00:32:24.371 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-09-11T00:32:24.371 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-09-11T00:32:24.371 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.371 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:24.371 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-09-11T00:32:24.371 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.371 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.371 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.371 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.371 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.372 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.372 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.372 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.372 INFO:teuthology.orchestra.run.smithi012.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-09-11T00:32:24.372 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-09-11T00:32:24.372 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-09-11T00:32:24.372 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.372 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:24.372 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-09-11T00:32:24.372 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.372 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.372 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.372 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.373 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.373 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.373 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.373 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.373 INFO:teuthology.orchestra.run.smithi012.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-09-11T00:32:24.373 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-09-11T00:32:24.373 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-09-11T00:32:24.373 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.373 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:24.373 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-09-11T00:32:24.373 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.373 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.373 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.374 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.374 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.374 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.374 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.374 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.374 INFO:teuthology.orchestra.run.smithi012.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-09-11T00:32:24.374 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-09-11T00:32:24.374 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-09-11T00:32:24.374 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.374 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:24.374 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-09-11T00:32:24.374 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.374 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.374 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.375 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.375 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.375 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.375 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.375 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.375 INFO:teuthology.orchestra.run.smithi012.stdout: "FEATURE_TOGGLE_NFS": { 2024-09-11T00:32:24.375 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-09-11T00:32:24.375 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-09-11T00:32:24.375 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.375 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:24.375 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-09-11T00:32:24.375 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.375 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.376 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.376 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.376 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.376 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.376 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.376 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.376 INFO:teuthology.orchestra.run.smithi012.stdout: "FEATURE_TOGGLE_RBD": { 2024-09-11T00:32:24.376 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-09-11T00:32:24.376 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-09-11T00:32:24.376 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.376 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:24.376 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-09-11T00:32:24.376 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.376 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.377 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.377 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.377 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.377 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.377 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.377 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.377 INFO:teuthology.orchestra.run.smithi012.stdout: "FEATURE_TOGGLE_RGW": { 2024-09-11T00:32:24.377 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-09-11T00:32:24.377 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-09-11T00:32:24.377 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.377 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:24.378 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-09-11T00:32:24.378 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.378 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.378 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.378 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.378 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.378 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.378 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.378 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.379 INFO:teuthology.orchestra.run.smithi012.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-09-11T00:32:24.379 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-09-11T00:32:24.379 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:24.379 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.379 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:24.379 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-09-11T00:32:24.379 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.379 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.379 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.379 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.379 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.379 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.380 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.380 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.380 INFO:teuthology.orchestra.run.smithi012.stdout: "GRAFANA_API_PASSWORD": { 2024-09-11T00:32:24.380 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "GRAFANA_API_PASSWORD", 2024-09-11T00:32:24.380 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:24.380 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.380 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:24.380 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "admin", 2024-09-11T00:32:24.380 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.380 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.380 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.380 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.380 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.381 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.381 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.381 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.381 INFO:teuthology.orchestra.run.smithi012.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-09-11T00:32:24.381 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-09-11T00:32:24.381 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-09-11T00:32:24.381 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.381 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:24.381 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-09-11T00:32:24.381 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.381 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.381 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.382 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.382 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.382 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.382 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.382 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.382 INFO:teuthology.orchestra.run.smithi012.stdout: "GRAFANA_API_URL": { 2024-09-11T00:32:24.382 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "GRAFANA_API_URL", 2024-09-11T00:32:24.382 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:24.382 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.382 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:24.382 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-09-11T00:32:24.382 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.382 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.383 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.383 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.383 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.383 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.383 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.383 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.383 INFO:teuthology.orchestra.run.smithi012.stdout: "GRAFANA_API_USERNAME": { 2024-09-11T00:32:24.383 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "GRAFANA_API_USERNAME", 2024-09-11T00:32:24.383 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:24.383 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.383 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:24.383 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "admin", 2024-09-11T00:32:24.384 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.384 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.384 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.384 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.384 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.384 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.384 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.384 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.384 INFO:teuthology.orchestra.run.smithi012.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-09-11T00:32:24.384 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-09-11T00:32:24.384 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:24.384 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.384 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:24.385 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-09-11T00:32:24.385 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.385 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.385 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.385 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.385 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.385 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.385 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.385 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.385 INFO:teuthology.orchestra.run.smithi012.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-09-11T00:32:24.385 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-09-11T00:32:24.385 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-09-11T00:32:24.385 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.386 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:24.386 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-09-11T00:32:24.386 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.386 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.386 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.386 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.386 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.386 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.386 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.386 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.386 INFO:teuthology.orchestra.run.smithi012.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-09-11T00:32:24.387 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-09-11T00:32:24.387 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-09-11T00:32:24.387 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.387 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:24.387 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-09-11T00:32:24.387 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.387 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.387 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.387 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.387 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.387 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.387 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.387 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.388 INFO:teuthology.orchestra.run.smithi012.stdout: "ISSUE_TRACKER_API_KEY": { 2024-09-11T00:32:24.388 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-09-11T00:32:24.388 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:24.388 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.388 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:24.388 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-09-11T00:32:24.388 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.388 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.388 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.388 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.388 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.388 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.388 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.389 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.389 INFO:teuthology.orchestra.run.smithi012.stdout: "MANAGED_BY_CLUSTERS": { 2024-09-11T00:32:24.389 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-09-11T00:32:24.389 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:24.389 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.389 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:24.389 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "[]", 2024-09-11T00:32:24.389 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.389 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.389 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.389 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.389 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.389 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.390 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.390 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.390 INFO:teuthology.orchestra.run.smithi012.stdout: "MULTICLUSTER_CONFIG": { 2024-09-11T00:32:24.390 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "MULTICLUSTER_CONFIG", 2024-09-11T00:32:24.390 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:24.390 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.390 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:24.390 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "{}", 2024-09-11T00:32:24.390 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.390 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.390 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.390 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.390 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.391 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.391 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.391 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.391 INFO:teuthology.orchestra.run.smithi012.stdout: "PROMETHEUS_API_HOST": { 2024-09-11T00:32:24.391 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "PROMETHEUS_API_HOST", 2024-09-11T00:32:24.391 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:24.391 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.391 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:24.391 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-09-11T00:32:24.391 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.391 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.391 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.391 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.391 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.392 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.392 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.392 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.392 INFO:teuthology.orchestra.run.smithi012.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-09-11T00:32:24.392 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-09-11T00:32:24.392 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-09-11T00:32:24.392 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.392 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:24.392 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-09-11T00:32:24.392 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.392 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.392 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.393 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.393 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.393 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.393 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.393 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.393 INFO:teuthology.orchestra.run.smithi012.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-09-11T00:32:24.393 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-09-11T00:32:24.393 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-09-11T00:32:24.393 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.393 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:24.393 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-09-11T00:32:24.393 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.393 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.394 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.394 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.394 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.394 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.394 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.394 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.394 INFO:teuthology.orchestra.run.smithi012.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-09-11T00:32:24.394 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-09-11T00:32:24.394 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-09-11T00:32:24.394 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.394 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:24.394 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-09-11T00:32:24.394 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.395 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.395 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.395 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.395 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.395 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.395 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.395 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.395 INFO:teuthology.orchestra.run.smithi012.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-09-11T00:32:24.395 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-09-11T00:32:24.395 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-09-11T00:32:24.395 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.395 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:24.395 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-09-11T00:32:24.396 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.396 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.396 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.396 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.396 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.396 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.396 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.396 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.396 INFO:teuthology.orchestra.run.smithi012.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-09-11T00:32:24.396 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-09-11T00:32:24.396 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-09-11T00:32:24.397 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.397 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:24.397 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-09-11T00:32:24.397 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.397 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.397 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.397 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.397 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.397 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.397 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.397 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.397 INFO:teuthology.orchestra.run.smithi012.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-09-11T00:32:24.397 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-09-11T00:32:24.398 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-09-11T00:32:24.398 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.398 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:24.398 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-09-11T00:32:24.398 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.398 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.398 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.398 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.398 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.398 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.398 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.398 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.398 INFO:teuthology.orchestra.run.smithi012.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-09-11T00:32:24.399 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-09-11T00:32:24.399 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-09-11T00:32:24.399 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.399 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:24.399 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-09-11T00:32:24.399 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.399 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.399 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.399 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.399 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.399 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.399 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.399 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.400 INFO:teuthology.orchestra.run.smithi012.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-09-11T00:32:24.400 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-09-11T00:32:24.400 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-09-11T00:32:24.400 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.400 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:24.400 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-09-11T00:32:24.400 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.400 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.400 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.400 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.400 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.400 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.400 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.401 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.401 INFO:teuthology.orchestra.run.smithi012.stdout: "PWD_POLICY_ENABLED": { 2024-09-11T00:32:24.401 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "PWD_POLICY_ENABLED", 2024-09-11T00:32:24.401 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-09-11T00:32:24.401 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.401 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:24.401 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-09-11T00:32:24.401 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.401 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.401 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.401 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.401 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.401 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.402 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.402 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.402 INFO:teuthology.orchestra.run.smithi012.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-09-11T00:32:24.402 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-09-11T00:32:24.402 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:24.402 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.402 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:24.402 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-09-11T00:32:24.402 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.402 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.402 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.402 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.402 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.402 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.403 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.403 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.403 INFO:teuthology.orchestra.run.smithi012.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-09-11T00:32:24.403 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-09-11T00:32:24.403 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-09-11T00:32:24.403 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.403 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:24.403 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "10", 2024-09-11T00:32:24.403 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.403 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.403 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.403 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.403 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.404 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.404 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.404 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.404 INFO:teuthology.orchestra.run.smithi012.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-09-11T00:32:24.404 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-09-11T00:32:24.404 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-09-11T00:32:24.404 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.404 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:24.404 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "8", 2024-09-11T00:32:24.404 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.404 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.404 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.404 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.405 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.405 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.405 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.405 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.405 INFO:teuthology.orchestra.run.smithi012.stdout: "REST_REQUESTS_TIMEOUT": { 2024-09-11T00:32:24.405 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-09-11T00:32:24.405 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-09-11T00:32:24.405 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.405 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:24.405 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "45", 2024-09-11T00:32:24.405 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.405 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.405 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.406 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.406 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.406 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.406 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.406 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.406 INFO:teuthology.orchestra.run.smithi012.stdout: "RGW_API_ACCESS_KEY": { 2024-09-11T00:32:24.406 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "RGW_API_ACCESS_KEY", 2024-09-11T00:32:24.406 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:24.406 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.406 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:24.406 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-09-11T00:32:24.406 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.406 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.406 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.407 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.407 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.407 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.407 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.407 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.407 INFO:teuthology.orchestra.run.smithi012.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-09-11T00:32:24.407 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-09-11T00:32:24.407 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:24.407 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.407 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:24.407 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "admin", 2024-09-11T00:32:24.407 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.408 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.408 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.408 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.408 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.408 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.408 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.408 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.408 INFO:teuthology.orchestra.run.smithi012.stdout: "RGW_API_SECRET_KEY": { 2024-09-11T00:32:24.408 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "RGW_API_SECRET_KEY", 2024-09-11T00:32:24.408 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:24.409 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.409 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:24.409 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-09-11T00:32:24.409 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.409 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.409 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.409 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.409 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.409 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.409 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.410 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.410 INFO:teuthology.orchestra.run.smithi012.stdout: "RGW_API_SSL_VERIFY": { 2024-09-11T00:32:24.410 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "RGW_API_SSL_VERIFY", 2024-09-11T00:32:24.410 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-09-11T00:32:24.410 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.410 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:24.410 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-09-11T00:32:24.410 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.410 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.410 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.410 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.410 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.410 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.410 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.411 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.411 INFO:teuthology.orchestra.run.smithi012.stdout: "UNSAFE_TLS_v1_2": { 2024-09-11T00:32:24.411 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "UNSAFE_TLS_v1_2", 2024-09-11T00:32:24.411 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-09-11T00:32:24.411 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.411 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:24.411 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-09-11T00:32:24.411 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.411 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.411 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.411 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.411 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.411 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.412 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.412 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.412 INFO:teuthology.orchestra.run.smithi012.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-09-11T00:32:24.412 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-09-11T00:32:24.412 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-09-11T00:32:24.412 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.412 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:24.412 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "0", 2024-09-11T00:32:24.412 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.412 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.412 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.412 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.412 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.413 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.413 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.413 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.413 INFO:teuthology.orchestra.run.smithi012.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-09-11T00:32:24.413 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-09-11T00:32:24.413 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-09-11T00:32:24.413 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.413 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:24.413 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "10", 2024-09-11T00:32:24.413 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.413 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.413 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.413 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.414 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.414 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.414 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.414 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.414 INFO:teuthology.orchestra.run.smithi012.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-09-11T00:32:24.414 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-09-11T00:32:24.414 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-09-11T00:32:24.414 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.414 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:24.414 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "5", 2024-09-11T00:32:24.414 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.414 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.414 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.415 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.415 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.415 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.415 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.415 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.415 INFO:teuthology.orchestra.run.smithi012.stdout: "cross_origin_url": { 2024-09-11T00:32:24.415 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "cross_origin_url", 2024-09-11T00:32:24.415 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:24.415 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.415 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:24.415 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-09-11T00:32:24.415 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.415 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.415 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.416 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.416 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.416 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.416 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.416 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.416 INFO:teuthology.orchestra.run.smithi012.stdout: "crt_file": { 2024-09-11T00:32:24.416 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "crt_file", 2024-09-11T00:32:24.416 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:24.416 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.416 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:24.416 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-09-11T00:32:24.416 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.416 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.417 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.417 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.417 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.417 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.417 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.417 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.417 INFO:teuthology.orchestra.run.smithi012.stdout: "debug": { 2024-09-11T00:32:24.417 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "debug", 2024-09-11T00:32:24.417 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-09-11T00:32:24.417 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.417 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:24.417 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-09-11T00:32:24.417 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.418 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.418 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.418 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Enable/disable debug options", 2024-09-11T00:32:24.418 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.418 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.418 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.418 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.418 INFO:teuthology.orchestra.run.smithi012.stdout: "jwt_token_ttl": { 2024-09-11T00:32:24.418 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "jwt_token_ttl", 2024-09-11T00:32:24.418 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-09-11T00:32:24.418 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.418 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:24.418 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "28800", 2024-09-11T00:32:24.418 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.419 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.419 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.419 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.419 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.419 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.419 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.419 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.419 INFO:teuthology.orchestra.run.smithi012.stdout: "key_file": { 2024-09-11T00:32:24.419 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "key_file", 2024-09-11T00:32:24.419 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:24.419 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.419 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:24.419 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-09-11T00:32:24.420 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.420 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.420 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.420 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.420 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.420 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.420 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.420 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.420 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-09-11T00:32:24.420 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-09-11T00:32:24.420 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:24.420 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.420 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:24.420 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-09-11T00:32:24.421 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.421 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.421 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-09-11T00:32:24.421 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-09-11T00:32:24.421 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-09-11T00:32:24.421 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-09-11T00:32:24.421 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-09-11T00:32:24.421 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-09-11T00:32:24.421 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-09-11T00:32:24.421 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-09-11T00:32:24.421 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.421 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.421 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.422 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.422 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.422 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-09-11T00:32:24.422 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-09-11T00:32:24.422 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-09-11T00:32:24.422 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.422 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:24.422 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-09-11T00:32:24.422 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.422 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.422 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.422 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.422 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.423 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.423 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.423 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.423 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-09-11T00:32:24.423 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-09-11T00:32:24.423 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:24.423 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.423 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:24.423 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-09-11T00:32:24.423 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.423 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.423 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-09-11T00:32:24.423 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-09-11T00:32:24.423 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-09-11T00:32:24.424 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-09-11T00:32:24.424 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-09-11T00:32:24.424 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-09-11T00:32:24.424 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-09-11T00:32:24.424 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-09-11T00:32:24.424 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.424 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.424 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.424 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.424 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.424 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-09-11T00:32:24.424 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-09-11T00:32:24.424 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-09-11T00:32:24.425 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.425 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:24.425 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-09-11T00:32:24.425 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.425 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.425 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.425 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.425 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.425 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.425 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.425 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.425 INFO:teuthology.orchestra.run.smithi012.stdout: "motd": { 2024-09-11T00:32:24.425 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "motd", 2024-09-11T00:32:24.425 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:24.426 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.426 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:24.426 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-09-11T00:32:24.426 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.426 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.426 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.426 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "The message of the day", 2024-09-11T00:32:24.426 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.426 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.426 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.426 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.426 INFO:teuthology.orchestra.run.smithi012.stdout: "redirect_resolve_ip_addr": { 2024-09-11T00:32:24.426 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "redirect_resolve_ip_addr", 2024-09-11T00:32:24.427 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-09-11T00:32:24.427 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.427 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:24.427 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-09-11T00:32:24.427 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.427 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.427 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.427 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.427 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.427 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.427 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.427 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.428 INFO:teuthology.orchestra.run.smithi012.stdout: "server_addr": { 2024-09-11T00:32:24.428 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "server_addr", 2024-09-11T00:32:24.428 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:24.428 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.428 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:24.428 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "::", 2024-09-11T00:32:24.428 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.428 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.428 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.428 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.428 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.428 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.428 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.429 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.429 INFO:teuthology.orchestra.run.smithi012.stdout: "server_port": { 2024-09-11T00:32:24.429 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "server_port", 2024-09-11T00:32:24.429 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-09-11T00:32:24.429 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.429 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:24.429 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "8080", 2024-09-11T00:32:24.429 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.429 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.429 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.429 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.429 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.429 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.429 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.430 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.430 INFO:teuthology.orchestra.run.smithi012.stdout: "sqlite3_killpoint": { 2024-09-11T00:32:24.430 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "sqlite3_killpoint", 2024-09-11T00:32:24.430 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-09-11T00:32:24.430 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "dev", 2024-09-11T00:32:24.430 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:24.430 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "0", 2024-09-11T00:32:24.430 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.430 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.430 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.430 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.430 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.430 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.431 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.431 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.431 INFO:teuthology.orchestra.run.smithi012.stdout: "ssl": { 2024-09-11T00:32:24.431 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "ssl", 2024-09-11T00:32:24.431 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-09-11T00:32:24.431 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.431 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:24.431 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-09-11T00:32:24.431 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.431 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.431 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.431 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.431 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.431 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.432 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.432 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.432 INFO:teuthology.orchestra.run.smithi012.stdout: "ssl_server_port": { 2024-09-11T00:32:24.432 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "ssl_server_port", 2024-09-11T00:32:24.432 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-09-11T00:32:24.432 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.432 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:24.432 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "8443", 2024-09-11T00:32:24.432 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.432 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.432 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.432 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.432 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.432 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.433 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.433 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.433 INFO:teuthology.orchestra.run.smithi012.stdout: "standby_behaviour": { 2024-09-11T00:32:24.433 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "standby_behaviour", 2024-09-11T00:32:24.433 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:24.433 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.433 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:24.433 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "redirect", 2024-09-11T00:32:24.433 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.433 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.433 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-09-11T00:32:24.433 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-09-11T00:32:24.433 INFO:teuthology.orchestra.run.smithi012.stdout: "redirect" 2024-09-11T00:32:24.434 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-09-11T00:32:24.434 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.434 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.434 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.434 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.434 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.434 INFO:teuthology.orchestra.run.smithi012.stdout: "standby_error_status_code": { 2024-09-11T00:32:24.434 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "standby_error_status_code", 2024-09-11T00:32:24.434 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-09-11T00:32:24.434 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.434 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:24.434 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "500", 2024-09-11T00:32:24.434 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "400", 2024-09-11T00:32:24.435 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "599", 2024-09-11T00:32:24.435 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.435 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.435 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.435 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.435 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.435 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.435 INFO:teuthology.orchestra.run.smithi012.stdout: "url_prefix": { 2024-09-11T00:32:24.435 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "url_prefix", 2024-09-11T00:32:24.435 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:24.435 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.435 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:24.436 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-09-11T00:32:24.436 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.436 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.436 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.436 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.436 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.436 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.436 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.436 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:32:24.436 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:32:24.437 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.437 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-11T00:32:24.437 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "devicehealth", 2024-09-11T00:32:24.437 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-09-11T00:32:24.437 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-09-11T00:32:24.437 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-09-11T00:32:24.437 INFO:teuthology.orchestra.run.smithi012.stdout: "enable_monitoring": { 2024-09-11T00:32:24.437 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "enable_monitoring", 2024-09-11T00:32:24.438 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-09-11T00:32:24.438 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.438 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:24.438 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-09-11T00:32:24.438 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.438 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.438 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.438 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "monitor device health metrics", 2024-09-11T00:32:24.438 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.438 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.438 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.438 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.439 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-09-11T00:32:24.439 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-09-11T00:32:24.439 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:24.439 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.439 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:24.439 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-09-11T00:32:24.439 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.439 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.439 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-09-11T00:32:24.439 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-09-11T00:32:24.439 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-09-11T00:32:24.439 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-09-11T00:32:24.439 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-09-11T00:32:24.440 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-09-11T00:32:24.440 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-09-11T00:32:24.440 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-09-11T00:32:24.440 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.440 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.440 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.440 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.440 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.440 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-09-11T00:32:24.440 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-09-11T00:32:24.440 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-09-11T00:32:24.440 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.441 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:24.441 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-09-11T00:32:24.441 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.441 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.441 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.441 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.441 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.441 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.441 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.441 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.441 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-09-11T00:32:24.441 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-09-11T00:32:24.442 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:24.442 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.442 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:24.442 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-09-11T00:32:24.442 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.442 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.442 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-09-11T00:32:24.442 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-09-11T00:32:24.442 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-09-11T00:32:24.442 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-09-11T00:32:24.442 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-09-11T00:32:24.442 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-09-11T00:32:24.442 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-09-11T00:32:24.442 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-09-11T00:32:24.443 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.443 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.443 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.443 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.443 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.443 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-09-11T00:32:24.443 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-09-11T00:32:24.443 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-09-11T00:32:24.443 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.443 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:24.443 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-09-11T00:32:24.443 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.443 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.443 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.444 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.444 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.444 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.444 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.444 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.444 INFO:teuthology.orchestra.run.smithi012.stdout: "mark_out_threshold": { 2024-09-11T00:32:24.444 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "mark_out_threshold", 2024-09-11T00:32:24.444 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "secs", 2024-09-11T00:32:24.444 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.444 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:24.444 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "2419200", 2024-09-11T00:32:24.444 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.444 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.445 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.445 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-09-11T00:32:24.445 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.445 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.445 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.445 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.445 INFO:teuthology.orchestra.run.smithi012.stdout: "pool_name": { 2024-09-11T00:32:24.445 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "pool_name", 2024-09-11T00:32:24.445 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:24.445 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.445 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:24.445 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "device_health_metrics", 2024-09-11T00:32:24.445 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.446 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.446 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.446 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "name of pool in which to store device health metrics", 2024-09-11T00:32:24.446 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.446 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.446 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.446 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.446 INFO:teuthology.orchestra.run.smithi012.stdout: "retention_period": { 2024-09-11T00:32:24.446 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "retention_period", 2024-09-11T00:32:24.446 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "secs", 2024-09-11T00:32:24.446 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.446 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:24.446 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "15552000", 2024-09-11T00:32:24.446 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.447 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.447 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.447 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "how long to retain device health metrics", 2024-09-11T00:32:24.447 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.447 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.447 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.447 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.447 INFO:teuthology.orchestra.run.smithi012.stdout: "scrape_frequency": { 2024-09-11T00:32:24.447 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "scrape_frequency", 2024-09-11T00:32:24.447 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "secs", 2024-09-11T00:32:24.447 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.447 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:24.447 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "86400", 2024-09-11T00:32:24.448 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.448 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.448 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.448 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "how frequently to scrape device health metrics", 2024-09-11T00:32:24.448 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.448 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.448 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.448 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.448 INFO:teuthology.orchestra.run.smithi012.stdout: "self_heal": { 2024-09-11T00:32:24.448 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "self_heal", 2024-09-11T00:32:24.448 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-09-11T00:32:24.448 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.448 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:24.448 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-09-11T00:32:24.449 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.449 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.449 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.449 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-09-11T00:32:24.449 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.449 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.449 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.449 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.449 INFO:teuthology.orchestra.run.smithi012.stdout: "sleep_interval": { 2024-09-11T00:32:24.449 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "sleep_interval", 2024-09-11T00:32:24.449 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "secs", 2024-09-11T00:32:24.449 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.449 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:24.450 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "600", 2024-09-11T00:32:24.450 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.450 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.450 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.450 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "how frequently to wake up and check device health", 2024-09-11T00:32:24.450 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.450 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.450 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.450 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.450 INFO:teuthology.orchestra.run.smithi012.stdout: "sqlite3_killpoint": { 2024-09-11T00:32:24.450 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "sqlite3_killpoint", 2024-09-11T00:32:24.450 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-09-11T00:32:24.450 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "dev", 2024-09-11T00:32:24.450 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:24.451 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "0", 2024-09-11T00:32:24.451 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.451 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.451 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.451 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.451 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.451 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.451 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.451 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.451 INFO:teuthology.orchestra.run.smithi012.stdout: "warn_threshold": { 2024-09-11T00:32:24.451 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "warn_threshold", 2024-09-11T00:32:24.451 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "secs", 2024-09-11T00:32:24.451 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.451 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:24.452 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "7257600", 2024-09-11T00:32:24.452 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.452 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.452 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.452 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-09-11T00:32:24.452 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.452 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.452 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.452 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:32:24.452 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:32:24.452 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.452 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-11T00:32:24.453 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "diskprediction_local", 2024-09-11T00:32:24.453 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-09-11T00:32:24.453 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-09-11T00:32:24.453 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-09-11T00:32:24.453 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-09-11T00:32:24.453 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-09-11T00:32:24.453 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:24.453 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.453 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:24.453 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-09-11T00:32:24.453 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.453 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.453 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-09-11T00:32:24.453 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-09-11T00:32:24.454 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-09-11T00:32:24.454 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-09-11T00:32:24.454 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-09-11T00:32:24.454 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-09-11T00:32:24.454 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-09-11T00:32:24.454 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-09-11T00:32:24.454 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.454 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.454 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.454 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.454 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.454 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-09-11T00:32:24.454 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-09-11T00:32:24.455 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-09-11T00:32:24.455 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.455 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:24.455 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-09-11T00:32:24.455 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.455 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.455 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.455 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.455 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.455 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.455 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.455 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.455 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-09-11T00:32:24.455 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-09-11T00:32:24.456 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:24.456 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.456 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:24.456 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-09-11T00:32:24.456 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.456 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.456 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-09-11T00:32:24.456 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-09-11T00:32:24.456 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-09-11T00:32:24.456 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-09-11T00:32:24.456 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-09-11T00:32:24.456 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-09-11T00:32:24.456 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-09-11T00:32:24.457 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-09-11T00:32:24.457 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.457 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.457 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.457 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.457 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.457 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-09-11T00:32:24.457 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-09-11T00:32:24.457 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-09-11T00:32:24.457 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.457 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:24.457 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-09-11T00:32:24.457 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.457 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.458 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.458 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.458 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.458 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.458 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.458 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.458 INFO:teuthology.orchestra.run.smithi012.stdout: "predict_interval": { 2024-09-11T00:32:24.458 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "predict_interval", 2024-09-11T00:32:24.458 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:24.458 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.458 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:24.458 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "86400", 2024-09-11T00:32:24.458 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.459 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.459 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.459 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.459 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.459 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.459 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.459 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.459 INFO:teuthology.orchestra.run.smithi012.stdout: "predictor_model": { 2024-09-11T00:32:24.459 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "predictor_model", 2024-09-11T00:32:24.459 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:24.459 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.459 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:24.459 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "prophetstor", 2024-09-11T00:32:24.459 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.460 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.460 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.460 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.460 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.460 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.460 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.460 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.460 INFO:teuthology.orchestra.run.smithi012.stdout: "sleep_interval": { 2024-09-11T00:32:24.460 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "sleep_interval", 2024-09-11T00:32:24.460 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:24.460 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.460 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:24.461 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "600", 2024-09-11T00:32:24.461 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.461 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.461 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.461 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.461 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.461 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.461 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.461 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.461 INFO:teuthology.orchestra.run.smithi012.stdout: "sqlite3_killpoint": { 2024-09-11T00:32:24.461 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "sqlite3_killpoint", 2024-09-11T00:32:24.461 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-09-11T00:32:24.461 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "dev", 2024-09-11T00:32:24.461 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:24.462 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "0", 2024-09-11T00:32:24.462 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.462 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.462 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.462 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.462 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.462 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.462 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.462 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:32:24.462 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:32:24.462 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.462 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-11T00:32:24.462 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "influx", 2024-09-11T00:32:24.463 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": false, 2024-09-11T00:32:24.463 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "influxdb python module not found", 2024-09-11T00:32:24.463 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-09-11T00:32:24.463 INFO:teuthology.orchestra.run.smithi012.stdout: "batch_size": { 2024-09-11T00:32:24.463 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "batch_size", 2024-09-11T00:32:24.463 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-09-11T00:32:24.463 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.463 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:24.463 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "5000", 2024-09-11T00:32:24.463 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.463 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.463 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.463 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-09-11T00:32:24.463 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.464 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.464 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.464 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.464 INFO:teuthology.orchestra.run.smithi012.stdout: "database": { 2024-09-11T00:32:24.464 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "database", 2024-09-11T00:32:24.464 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:24.464 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.464 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:24.464 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "ceph", 2024-09-11T00:32:24.464 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.464 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.464 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.464 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-09-11T00:32:24.465 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.465 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.465 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.465 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.465 INFO:teuthology.orchestra.run.smithi012.stdout: "hostname": { 2024-09-11T00:32:24.465 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "hostname", 2024-09-11T00:32:24.465 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:24.465 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.465 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:24.465 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-09-11T00:32:24.465 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.465 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.465 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.465 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "InfluxDB server hostname", 2024-09-11T00:32:24.466 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.466 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.466 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.466 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.466 INFO:teuthology.orchestra.run.smithi012.stdout: "interval": { 2024-09-11T00:32:24.466 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "interval", 2024-09-11T00:32:24.466 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "secs", 2024-09-11T00:32:24.466 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.466 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:24.466 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "30", 2024-09-11T00:32:24.466 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "5", 2024-09-11T00:32:24.466 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.466 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.467 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-09-11T00:32:24.467 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.467 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.467 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.467 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.467 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-09-11T00:32:24.467 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-09-11T00:32:24.467 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:24.467 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.467 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:24.467 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-09-11T00:32:24.467 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.467 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.468 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-09-11T00:32:24.468 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-09-11T00:32:24.468 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-09-11T00:32:24.468 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-09-11T00:32:24.468 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-09-11T00:32:24.468 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-09-11T00:32:24.468 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-09-11T00:32:24.468 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-09-11T00:32:24.468 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.468 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.468 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.468 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.468 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.469 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-09-11T00:32:24.469 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-09-11T00:32:24.469 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-09-11T00:32:24.469 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.469 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:24.469 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-09-11T00:32:24.469 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.469 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.469 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.470 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.470 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.470 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.470 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.470 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.470 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-09-11T00:32:24.470 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-09-11T00:32:24.470 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:24.470 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.470 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:24.471 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-09-11T00:32:24.471 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.471 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.471 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-09-11T00:32:24.471 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-09-11T00:32:24.471 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-09-11T00:32:24.471 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-09-11T00:32:24.471 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-09-11T00:32:24.471 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-09-11T00:32:24.471 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-09-11T00:32:24.471 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-09-11T00:32:24.471 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.471 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.471 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.472 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.472 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.472 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-09-11T00:32:24.472 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-09-11T00:32:24.472 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-09-11T00:32:24.472 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.472 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:24.472 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-09-11T00:32:24.472 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.472 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.472 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.472 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.472 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.473 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.473 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.473 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.473 INFO:teuthology.orchestra.run.smithi012.stdout: "password": { 2024-09-11T00:32:24.473 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "password", 2024-09-11T00:32:24.473 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:24.473 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.473 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:24.473 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-09-11T00:32:24.473 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.473 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.473 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.474 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "password of InfluxDB server user", 2024-09-11T00:32:24.474 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.474 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.474 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.474 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.474 INFO:teuthology.orchestra.run.smithi012.stdout: "port": { 2024-09-11T00:32:24.474 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "port", 2024-09-11T00:32:24.474 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-09-11T00:32:24.474 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.474 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:24.474 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "8086", 2024-09-11T00:32:24.474 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.474 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.475 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.475 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "InfluxDB server port", 2024-09-11T00:32:24.475 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.475 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.475 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.475 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.475 INFO:teuthology.orchestra.run.smithi012.stdout: "sqlite3_killpoint": { 2024-09-11T00:32:24.475 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "sqlite3_killpoint", 2024-09-11T00:32:24.475 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-09-11T00:32:24.475 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "dev", 2024-09-11T00:32:24.475 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:24.475 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "0", 2024-09-11T00:32:24.475 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.476 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.476 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.476 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.476 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.476 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.476 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.476 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.476 INFO:teuthology.orchestra.run.smithi012.stdout: "ssl": { 2024-09-11T00:32:24.476 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "ssl", 2024-09-11T00:32:24.476 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:24.476 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.476 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:24.476 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "false", 2024-09-11T00:32:24.477 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.477 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.477 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.477 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-09-11T00:32:24.477 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.477 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.477 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.477 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.477 INFO:teuthology.orchestra.run.smithi012.stdout: "threads": { 2024-09-11T00:32:24.477 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "threads", 2024-09-11T00:32:24.477 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-09-11T00:32:24.477 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.477 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:24.478 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "5", 2024-09-11T00:32:24.478 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "1", 2024-09-11T00:32:24.478 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "32", 2024-09-11T00:32:24.478 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.478 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-09-11T00:32:24.478 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.478 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.478 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.478 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.478 INFO:teuthology.orchestra.run.smithi012.stdout: "username": { 2024-09-11T00:32:24.478 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "username", 2024-09-11T00:32:24.478 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:24.479 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.479 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:24.479 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-09-11T00:32:24.479 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.479 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.479 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.479 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "username of InfluxDB server user", 2024-09-11T00:32:24.479 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.479 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.479 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.479 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.479 INFO:teuthology.orchestra.run.smithi012.stdout: "verify_ssl": { 2024-09-11T00:32:24.479 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "verify_ssl", 2024-09-11T00:32:24.480 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:24.480 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.480 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:24.480 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "true", 2024-09-11T00:32:24.480 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.480 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.480 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.480 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-09-11T00:32:24.480 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.480 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.480 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.480 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:32:24.481 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:32:24.481 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.481 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-11T00:32:24.481 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "insights", 2024-09-11T00:32:24.481 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-09-11T00:32:24.481 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-09-11T00:32:24.481 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-09-11T00:32:24.481 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-09-11T00:32:24.481 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-09-11T00:32:24.481 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:24.481 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.481 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:24.481 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-09-11T00:32:24.482 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.482 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.482 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-09-11T00:32:24.482 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-09-11T00:32:24.482 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-09-11T00:32:24.482 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-09-11T00:32:24.482 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-09-11T00:32:24.482 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-09-11T00:32:24.482 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-09-11T00:32:24.482 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-09-11T00:32:24.482 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.482 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.482 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.482 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.483 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.483 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-09-11T00:32:24.483 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-09-11T00:32:24.483 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-09-11T00:32:24.483 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.483 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:24.483 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-09-11T00:32:24.483 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.483 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.483 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.483 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.483 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.483 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.483 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.484 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.484 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-09-11T00:32:24.484 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-09-11T00:32:24.484 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:24.484 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.484 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:24.484 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-09-11T00:32:24.484 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.484 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.484 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-09-11T00:32:24.484 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-09-11T00:32:24.484 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-09-11T00:32:24.484 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-09-11T00:32:24.485 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-09-11T00:32:24.485 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-09-11T00:32:24.485 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-09-11T00:32:24.485 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-09-11T00:32:24.485 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.485 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.485 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.485 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.485 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.485 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-09-11T00:32:24.485 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-09-11T00:32:24.485 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-09-11T00:32:24.485 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.486 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:24.486 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-09-11T00:32:24.486 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.486 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.486 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.486 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.486 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.486 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.486 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.486 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.486 INFO:teuthology.orchestra.run.smithi012.stdout: "sqlite3_killpoint": { 2024-09-11T00:32:24.486 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "sqlite3_killpoint", 2024-09-11T00:32:24.486 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-09-11T00:32:24.487 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "dev", 2024-09-11T00:32:24.487 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:24.487 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "0", 2024-09-11T00:32:24.487 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.487 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.487 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.487 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.487 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.487 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.487 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.487 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:32:24.487 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:32:24.487 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.488 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-11T00:32:24.488 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "iostat", 2024-09-11T00:32:24.488 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-09-11T00:32:24.488 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-09-11T00:32:24.488 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-09-11T00:32:24.488 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-09-11T00:32:24.488 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-09-11T00:32:24.488 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:24.488 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.488 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:24.488 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-09-11T00:32:24.488 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.488 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.489 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-09-11T00:32:24.489 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-09-11T00:32:24.489 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-09-11T00:32:24.489 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-09-11T00:32:24.489 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-09-11T00:32:24.489 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-09-11T00:32:24.489 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-09-11T00:32:24.489 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-09-11T00:32:24.489 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.489 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.489 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.489 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.490 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.490 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-09-11T00:32:24.490 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-09-11T00:32:24.490 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-09-11T00:32:24.490 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.490 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:24.490 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-09-11T00:32:24.490 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.492 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.492 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.492 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.492 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.492 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.492 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.492 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.492 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-09-11T00:32:24.492 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-09-11T00:32:24.493 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:24.493 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.493 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:24.493 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-09-11T00:32:24.493 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.493 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.493 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-09-11T00:32:24.493 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-09-11T00:32:24.493 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-09-11T00:32:24.493 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-09-11T00:32:24.493 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-09-11T00:32:24.494 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-09-11T00:32:24.494 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-09-11T00:32:24.494 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-09-11T00:32:24.494 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.494 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.494 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.494 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.494 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.494 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-09-11T00:32:24.494 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-09-11T00:32:24.494 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-09-11T00:32:24.494 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.495 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:24.495 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-09-11T00:32:24.495 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.495 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.495 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.495 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.495 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.495 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.495 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.495 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.495 INFO:teuthology.orchestra.run.smithi012.stdout: "sqlite3_killpoint": { 2024-09-11T00:32:24.495 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "sqlite3_killpoint", 2024-09-11T00:32:24.495 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-09-11T00:32:24.496 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "dev", 2024-09-11T00:32:24.496 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:24.496 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "0", 2024-09-11T00:32:24.496 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.496 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.496 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.496 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.496 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.496 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.496 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.496 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:32:24.496 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:32:24.496 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.497 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-11T00:32:24.497 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "k8sevents", 2024-09-11T00:32:24.497 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-09-11T00:32:24.497 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-09-11T00:32:24.497 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-09-11T00:32:24.497 INFO:teuthology.orchestra.run.smithi012.stdout: "ceph_event_retention_days": { 2024-09-11T00:32:24.497 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "ceph_event_retention_days", 2024-09-11T00:32:24.497 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-09-11T00:32:24.497 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.497 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:24.497 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "7", 2024-09-11T00:32:24.497 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.497 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.498 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.498 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Days to hold ceph event information within local cache", 2024-09-11T00:32:24.498 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.498 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.498 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.498 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.498 INFO:teuthology.orchestra.run.smithi012.stdout: "config_check_secs": { 2024-09-11T00:32:24.498 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "config_check_secs", 2024-09-11T00:32:24.498 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-09-11T00:32:24.498 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.498 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:24.499 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "10", 2024-09-11T00:32:24.499 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "10", 2024-09-11T00:32:24.499 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.499 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.499 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-09-11T00:32:24.499 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.499 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.499 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.499 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.499 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-09-11T00:32:24.499 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-09-11T00:32:24.499 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:24.499 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.499 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:24.500 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-09-11T00:32:24.500 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.500 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.500 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-09-11T00:32:24.500 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-09-11T00:32:24.500 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-09-11T00:32:24.500 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-09-11T00:32:24.500 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-09-11T00:32:24.500 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-09-11T00:32:24.500 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-09-11T00:32:24.500 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-09-11T00:32:24.500 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.501 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.501 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.501 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.501 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.501 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-09-11T00:32:24.501 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-09-11T00:32:24.501 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-09-11T00:32:24.501 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.501 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:24.501 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-09-11T00:32:24.501 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.501 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.501 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.502 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.502 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.502 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.502 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.502 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.502 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-09-11T00:32:24.502 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-09-11T00:32:24.502 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:24.502 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.502 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:24.502 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-09-11T00:32:24.502 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.503 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.503 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-09-11T00:32:24.503 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-09-11T00:32:24.503 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-09-11T00:32:24.503 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-09-11T00:32:24.503 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-09-11T00:32:24.503 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-09-11T00:32:24.503 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-09-11T00:32:24.504 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-09-11T00:32:24.504 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.504 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.504 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.504 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.504 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.504 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-09-11T00:32:24.504 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-09-11T00:32:24.504 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-09-11T00:32:24.504 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.504 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:24.504 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-09-11T00:32:24.505 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.505 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.505 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.505 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.505 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.505 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.505 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.505 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.505 INFO:teuthology.orchestra.run.smithi012.stdout: "sqlite3_killpoint": { 2024-09-11T00:32:24.505 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "sqlite3_killpoint", 2024-09-11T00:32:24.505 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-09-11T00:32:24.505 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "dev", 2024-09-11T00:32:24.505 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:24.506 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "0", 2024-09-11T00:32:24.506 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.506 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.506 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.506 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.506 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.506 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.506 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.506 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:32:24.506 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:32:24.506 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.506 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-11T00:32:24.506 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "localpool", 2024-09-11T00:32:24.506 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-09-11T00:32:24.507 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-09-11T00:32:24.507 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-09-11T00:32:24.507 INFO:teuthology.orchestra.run.smithi012.stdout: "failure_domain": { 2024-09-11T00:32:24.507 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "failure_domain", 2024-09-11T00:32:24.507 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:24.507 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.507 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:24.507 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "host", 2024-09-11T00:32:24.507 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.507 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.507 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.507 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "failure domain for any created local pool", 2024-09-11T00:32:24.507 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-09-11T00:32:24.508 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.508 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.508 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.508 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-09-11T00:32:24.508 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-09-11T00:32:24.508 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:24.508 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.508 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:24.508 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-09-11T00:32:24.508 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.508 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.508 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-09-11T00:32:24.508 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-09-11T00:32:24.509 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-09-11T00:32:24.509 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-09-11T00:32:24.509 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-09-11T00:32:24.509 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-09-11T00:32:24.509 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-09-11T00:32:24.509 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-09-11T00:32:24.509 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.509 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.509 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.509 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.509 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.509 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-09-11T00:32:24.509 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-09-11T00:32:24.510 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-09-11T00:32:24.510 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.510 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:24.510 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-09-11T00:32:24.510 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.510 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.510 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.510 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.510 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.510 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.510 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.510 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.510 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-09-11T00:32:24.511 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-09-11T00:32:24.511 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:24.511 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.511 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:24.511 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-09-11T00:32:24.511 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.511 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.511 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-09-11T00:32:24.511 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-09-11T00:32:24.511 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-09-11T00:32:24.511 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-09-11T00:32:24.511 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-09-11T00:32:24.511 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-09-11T00:32:24.511 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-09-11T00:32:24.512 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-09-11T00:32:24.512 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.512 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.512 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.512 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.512 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.512 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-09-11T00:32:24.512 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-09-11T00:32:24.512 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-09-11T00:32:24.512 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.512 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:24.512 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-09-11T00:32:24.512 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.513 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.513 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.513 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.513 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.513 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.513 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.513 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.513 INFO:teuthology.orchestra.run.smithi012.stdout: "min_size": { 2024-09-11T00:32:24.513 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "min_size", 2024-09-11T00:32:24.513 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-09-11T00:32:24.513 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.513 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:24.513 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-09-11T00:32:24.514 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.514 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.514 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.514 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "default min_size for any created local pool", 2024-09-11T00:32:24.514 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-09-11T00:32:24.514 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.514 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.514 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.514 INFO:teuthology.orchestra.run.smithi012.stdout: "num_rep": { 2024-09-11T00:32:24.514 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "num_rep", 2024-09-11T00:32:24.514 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-09-11T00:32:24.514 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.514 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:24.515 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "3", 2024-09-11T00:32:24.515 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.515 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.515 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.515 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "default replica count for any created local pool", 2024-09-11T00:32:24.515 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.515 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.515 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.515 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.515 INFO:teuthology.orchestra.run.smithi012.stdout: "pg_num": { 2024-09-11T00:32:24.515 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "pg_num", 2024-09-11T00:32:24.515 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-09-11T00:32:24.515 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.515 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:24.516 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "128", 2024-09-11T00:32:24.516 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.516 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.516 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.516 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "default pg_num for any created local pool", 2024-09-11T00:32:24.516 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.516 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.516 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.516 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.516 INFO:teuthology.orchestra.run.smithi012.stdout: "prefix": { 2024-09-11T00:32:24.517 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "prefix", 2024-09-11T00:32:24.517 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:24.517 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.517 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:24.517 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-09-11T00:32:24.517 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.517 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.517 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.517 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "name prefix for any created local pool", 2024-09-11T00:32:24.518 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.518 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.518 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.518 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.518 INFO:teuthology.orchestra.run.smithi012.stdout: "sqlite3_killpoint": { 2024-09-11T00:32:24.518 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "sqlite3_killpoint", 2024-09-11T00:32:24.518 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-09-11T00:32:24.518 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "dev", 2024-09-11T00:32:24.518 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:24.518 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "0", 2024-09-11T00:32:24.518 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.519 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.519 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.519 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.519 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.519 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.519 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.519 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.519 INFO:teuthology.orchestra.run.smithi012.stdout: "subtree": { 2024-09-11T00:32:24.519 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "subtree", 2024-09-11T00:32:24.519 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:24.519 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.519 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:24.519 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "rack", 2024-09-11T00:32:24.520 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.520 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.520 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.520 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "CRUSH level for which to create a local pool", 2024-09-11T00:32:24.520 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-09-11T00:32:24.520 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.520 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.520 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:32:24.520 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:32:24.520 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.520 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-11T00:32:24.520 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "mds_autoscaler", 2024-09-11T00:32:24.520 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-09-11T00:32:24.521 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-09-11T00:32:24.521 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-09-11T00:32:24.521 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-09-11T00:32:24.521 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-09-11T00:32:24.521 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:24.521 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.521 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:24.521 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-09-11T00:32:24.521 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.521 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.521 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-09-11T00:32:24.521 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-09-11T00:32:24.521 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-09-11T00:32:24.521 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-09-11T00:32:24.522 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-09-11T00:32:24.522 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-09-11T00:32:24.522 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-09-11T00:32:24.522 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-09-11T00:32:24.522 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.522 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.522 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.522 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.522 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.522 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-09-11T00:32:24.522 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-09-11T00:32:24.522 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-09-11T00:32:24.522 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.522 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:24.523 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-09-11T00:32:24.523 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.523 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.523 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.523 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.523 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.523 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.523 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.523 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.523 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-09-11T00:32:24.523 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-09-11T00:32:24.523 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:24.523 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.524 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:24.524 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-09-11T00:32:24.524 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.524 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.524 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-09-11T00:32:24.524 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-09-11T00:32:24.524 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-09-11T00:32:24.524 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-09-11T00:32:24.524 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-09-11T00:32:24.524 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-09-11T00:32:24.524 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-09-11T00:32:24.524 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-09-11T00:32:24.524 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.525 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.525 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.525 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.525 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.525 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-09-11T00:32:24.525 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-09-11T00:32:24.525 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-09-11T00:32:24.525 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.525 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:24.525 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-09-11T00:32:24.526 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.526 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.526 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.526 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.526 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.526 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.526 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.526 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.526 INFO:teuthology.orchestra.run.smithi012.stdout: "sqlite3_killpoint": { 2024-09-11T00:32:24.527 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "sqlite3_killpoint", 2024-09-11T00:32:24.527 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-09-11T00:32:24.527 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "dev", 2024-09-11T00:32:24.527 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:24.527 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "0", 2024-09-11T00:32:24.527 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.527 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.527 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.527 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.527 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.527 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.527 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.527 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:32:24.527 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:32:24.528 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.528 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-11T00:32:24.528 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "mirroring", 2024-09-11T00:32:24.528 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-09-11T00:32:24.528 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-09-11T00:32:24.528 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-09-11T00:32:24.528 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-09-11T00:32:24.528 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-09-11T00:32:24.528 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:24.528 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.528 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:24.528 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-09-11T00:32:24.529 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.529 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.529 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-09-11T00:32:24.529 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-09-11T00:32:24.529 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-09-11T00:32:24.529 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-09-11T00:32:24.529 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-09-11T00:32:24.529 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-09-11T00:32:24.529 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-09-11T00:32:24.529 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-09-11T00:32:24.529 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.529 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.529 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.530 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.530 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.530 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-09-11T00:32:24.530 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-09-11T00:32:24.530 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-09-11T00:32:24.530 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.530 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:24.530 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-09-11T00:32:24.530 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.530 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.530 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.530 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.530 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.531 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.531 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.531 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.531 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-09-11T00:32:24.531 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-09-11T00:32:24.531 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:24.531 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.531 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:24.531 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-09-11T00:32:24.531 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.531 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.531 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-09-11T00:32:24.531 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-09-11T00:32:24.532 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-09-11T00:32:24.532 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-09-11T00:32:24.532 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-09-11T00:32:24.532 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-09-11T00:32:24.532 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-09-11T00:32:24.532 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-09-11T00:32:24.532 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.532 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.532 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.532 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.532 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.532 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-09-11T00:32:24.532 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-09-11T00:32:24.533 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-09-11T00:32:24.533 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.533 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:24.533 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-09-11T00:32:24.533 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.533 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.533 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.533 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.533 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.533 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.533 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.533 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.534 INFO:teuthology.orchestra.run.smithi012.stdout: "sqlite3_killpoint": { 2024-09-11T00:32:24.534 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "sqlite3_killpoint", 2024-09-11T00:32:24.534 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-09-11T00:32:24.534 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "dev", 2024-09-11T00:32:24.534 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:24.534 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "0", 2024-09-11T00:32:24.534 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.534 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.534 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.534 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.534 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.534 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.534 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.534 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:32:24.535 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:32:24.535 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.535 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-11T00:32:24.535 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "nfs", 2024-09-11T00:32:24.535 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-09-11T00:32:24.535 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-09-11T00:32:24.535 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-09-11T00:32:24.535 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-09-11T00:32:24.535 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-09-11T00:32:24.535 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:24.535 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.535 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:24.535 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-09-11T00:32:24.536 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.536 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.536 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-09-11T00:32:24.536 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-09-11T00:32:24.536 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-09-11T00:32:24.536 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-09-11T00:32:24.536 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-09-11T00:32:24.536 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-09-11T00:32:24.536 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-09-11T00:32:24.536 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-09-11T00:32:24.536 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.536 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.537 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.537 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.537 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.537 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-09-11T00:32:24.537 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-09-11T00:32:24.537 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-09-11T00:32:24.537 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.537 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:24.537 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-09-11T00:32:24.537 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.537 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.537 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.537 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.538 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.538 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.538 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.538 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.538 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-09-11T00:32:24.538 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-09-11T00:32:24.538 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:24.538 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.538 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:24.538 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-09-11T00:32:24.538 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.538 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.538 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-09-11T00:32:24.539 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-09-11T00:32:24.539 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-09-11T00:32:24.539 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-09-11T00:32:24.539 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-09-11T00:32:24.539 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-09-11T00:32:24.539 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-09-11T00:32:24.539 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-09-11T00:32:24.539 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.539 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.539 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.539 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.539 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.539 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-09-11T00:32:24.540 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-09-11T00:32:24.540 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-09-11T00:32:24.540 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.540 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:24.540 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-09-11T00:32:24.540 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.540 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.540 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.541 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.541 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.541 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.541 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.541 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.541 INFO:teuthology.orchestra.run.smithi012.stdout: "sqlite3_killpoint": { 2024-09-11T00:32:24.541 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "sqlite3_killpoint", 2024-09-11T00:32:24.541 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-09-11T00:32:24.541 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "dev", 2024-09-11T00:32:24.541 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:24.541 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "0", 2024-09-11T00:32:24.542 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.542 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.542 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.542 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.542 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.542 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.542 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.542 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:32:24.542 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:32:24.542 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.542 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-11T00:32:24.542 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "orchestrator", 2024-09-11T00:32:24.542 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-09-11T00:32:24.543 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-09-11T00:32:24.543 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-09-11T00:32:24.543 INFO:teuthology.orchestra.run.smithi012.stdout: "fail_fs": { 2024-09-11T00:32:24.543 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "fail_fs", 2024-09-11T00:32:24.543 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-09-11T00:32:24.543 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.543 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:24.543 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-09-11T00:32:24.543 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.543 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.543 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.543 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-09-11T00:32:24.544 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.544 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.544 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.544 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.544 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-09-11T00:32:24.544 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-09-11T00:32:24.544 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:24.544 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.544 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:24.544 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-09-11T00:32:24.544 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.544 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.544 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-09-11T00:32:24.545 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-09-11T00:32:24.545 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-09-11T00:32:24.545 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-09-11T00:32:24.545 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-09-11T00:32:24.545 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-09-11T00:32:24.545 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-09-11T00:32:24.545 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-09-11T00:32:24.545 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.545 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.545 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.545 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.545 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.545 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-09-11T00:32:24.546 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-09-11T00:32:24.546 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-09-11T00:32:24.546 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.546 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:24.546 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-09-11T00:32:24.546 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.546 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.546 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.546 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.546 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.546 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.546 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.547 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.547 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-09-11T00:32:24.547 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-09-11T00:32:24.547 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:24.547 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.547 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:24.547 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-09-11T00:32:24.547 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.547 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.547 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-09-11T00:32:24.547 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-09-11T00:32:24.547 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-09-11T00:32:24.547 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-09-11T00:32:24.548 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-09-11T00:32:24.548 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-09-11T00:32:24.548 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-09-11T00:32:24.548 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-09-11T00:32:24.548 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.548 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.548 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.548 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.548 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.548 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-09-11T00:32:24.548 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-09-11T00:32:24.548 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-09-11T00:32:24.549 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.549 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:24.549 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-09-11T00:32:24.549 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.549 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.549 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.549 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.549 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.549 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.549 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.549 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.549 INFO:teuthology.orchestra.run.smithi012.stdout: "orchestrator": { 2024-09-11T00:32:24.549 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "orchestrator", 2024-09-11T00:32:24.550 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:24.550 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.550 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:24.550 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-09-11T00:32:24.550 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.550 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.550 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-09-11T00:32:24.550 INFO:teuthology.orchestra.run.smithi012.stdout: "cephadm", 2024-09-11T00:32:24.550 INFO:teuthology.orchestra.run.smithi012.stdout: "rook", 2024-09-11T00:32:24.550 INFO:teuthology.orchestra.run.smithi012.stdout: "test_orchestrator" 2024-09-11T00:32:24.550 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-09-11T00:32:24.550 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Orchestrator backend", 2024-09-11T00:32:24.550 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.551 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.551 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.551 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.551 INFO:teuthology.orchestra.run.smithi012.stdout: "sqlite3_killpoint": { 2024-09-11T00:32:24.551 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "sqlite3_killpoint", 2024-09-11T00:32:24.551 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-09-11T00:32:24.551 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "dev", 2024-09-11T00:32:24.551 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:24.551 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "0", 2024-09-11T00:32:24.551 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.551 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.551 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.551 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.552 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.552 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.552 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.552 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:32:24.552 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:32:24.552 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.552 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-11T00:32:24.552 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "osd_perf_query", 2024-09-11T00:32:24.552 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-09-11T00:32:24.552 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-09-11T00:32:24.552 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-09-11T00:32:24.552 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-09-11T00:32:24.553 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-09-11T00:32:24.553 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:24.553 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.553 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:24.553 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-09-11T00:32:24.553 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.553 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.553 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-09-11T00:32:24.553 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-09-11T00:32:24.553 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-09-11T00:32:24.554 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-09-11T00:32:24.554 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-09-11T00:32:24.554 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-09-11T00:32:24.554 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-09-11T00:32:24.554 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-09-11T00:32:24.554 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.554 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.554 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.554 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.554 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.554 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-09-11T00:32:24.554 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-09-11T00:32:24.554 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-09-11T00:32:24.554 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.555 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:24.555 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-09-11T00:32:24.555 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.555 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.555 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.555 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.555 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.555 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.555 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.555 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.555 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-09-11T00:32:24.555 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-09-11T00:32:24.556 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:24.556 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.556 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:24.556 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-09-11T00:32:24.556 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.556 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.556 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-09-11T00:32:24.556 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-09-11T00:32:24.556 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-09-11T00:32:24.556 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-09-11T00:32:24.556 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-09-11T00:32:24.556 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-09-11T00:32:24.556 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-09-11T00:32:24.556 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-09-11T00:32:24.557 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.557 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.557 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.557 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.557 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.557 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-09-11T00:32:24.557 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-09-11T00:32:24.557 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-09-11T00:32:24.557 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.558 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:24.558 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-09-11T00:32:24.558 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.558 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.558 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.558 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.558 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.558 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.558 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.558 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.558 INFO:teuthology.orchestra.run.smithi012.stdout: "sqlite3_killpoint": { 2024-09-11T00:32:24.559 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "sqlite3_killpoint", 2024-09-11T00:32:24.559 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-09-11T00:32:24.559 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "dev", 2024-09-11T00:32:24.559 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:24.559 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "0", 2024-09-11T00:32:24.559 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.559 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.559 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.559 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.559 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.559 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.559 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.559 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:32:24.560 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:32:24.560 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.560 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-11T00:32:24.560 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "osd_support", 2024-09-11T00:32:24.560 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-09-11T00:32:24.560 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-09-11T00:32:24.560 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-09-11T00:32:24.560 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-09-11T00:32:24.560 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-09-11T00:32:24.560 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:24.560 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.560 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:24.561 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-09-11T00:32:24.561 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.561 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.561 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-09-11T00:32:24.561 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-09-11T00:32:24.561 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-09-11T00:32:24.561 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-09-11T00:32:24.561 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-09-11T00:32:24.561 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-09-11T00:32:24.561 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-09-11T00:32:24.561 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-09-11T00:32:24.561 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.561 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.562 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.562 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.562 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.562 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-09-11T00:32:24.562 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-09-11T00:32:24.562 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-09-11T00:32:24.562 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.562 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:24.562 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-09-11T00:32:24.562 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.562 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.562 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.562 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.563 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.563 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.563 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.563 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.563 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-09-11T00:32:24.563 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-09-11T00:32:24.563 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:24.563 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.563 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:24.563 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-09-11T00:32:24.563 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.563 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.563 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-09-11T00:32:24.564 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-09-11T00:32:24.564 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-09-11T00:32:24.564 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-09-11T00:32:24.564 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-09-11T00:32:24.564 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-09-11T00:32:24.564 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-09-11T00:32:24.564 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-09-11T00:32:24.564 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.564 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.564 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.564 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.564 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.564 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-09-11T00:32:24.565 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-09-11T00:32:24.565 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-09-11T00:32:24.565 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.565 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:24.565 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-09-11T00:32:24.565 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.565 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.565 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.565 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.565 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.565 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.565 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.565 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.566 INFO:teuthology.orchestra.run.smithi012.stdout: "sqlite3_killpoint": { 2024-09-11T00:32:24.566 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "sqlite3_killpoint", 2024-09-11T00:32:24.566 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-09-11T00:32:24.566 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "dev", 2024-09-11T00:32:24.566 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:24.566 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "0", 2024-09-11T00:32:24.566 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.566 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.566 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.566 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.566 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.566 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.567 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.567 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:32:24.567 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:32:24.567 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.567 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-11T00:32:24.567 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "pg_autoscaler", 2024-09-11T00:32:24.567 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-09-11T00:32:24.567 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-09-11T00:32:24.567 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-09-11T00:32:24.567 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-09-11T00:32:24.567 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-09-11T00:32:24.567 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:24.567 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.568 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:24.568 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-09-11T00:32:24.568 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.568 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.568 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-09-11T00:32:24.568 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-09-11T00:32:24.568 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-09-11T00:32:24.568 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-09-11T00:32:24.568 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-09-11T00:32:24.568 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-09-11T00:32:24.568 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-09-11T00:32:24.568 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-09-11T00:32:24.568 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.569 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.569 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.569 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.569 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.569 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-09-11T00:32:24.569 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-09-11T00:32:24.569 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-09-11T00:32:24.569 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.569 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:24.569 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-09-11T00:32:24.569 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.569 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.570 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.570 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.570 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.570 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.570 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.570 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.570 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-09-11T00:32:24.570 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-09-11T00:32:24.570 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:24.570 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.570 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:24.570 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-09-11T00:32:24.571 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.571 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.571 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-09-11T00:32:24.571 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-09-11T00:32:24.571 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-09-11T00:32:24.571 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-09-11T00:32:24.571 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-09-11T00:32:24.571 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-09-11T00:32:24.571 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-09-11T00:32:24.571 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-09-11T00:32:24.571 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.571 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.571 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.572 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.572 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.572 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-09-11T00:32:24.572 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-09-11T00:32:24.572 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-09-11T00:32:24.572 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.572 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:24.572 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-09-11T00:32:24.572 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.572 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.572 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.572 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.573 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.573 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.573 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.573 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.573 INFO:teuthology.orchestra.run.smithi012.stdout: "sleep_interval": { 2024-09-11T00:32:24.573 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "sleep_interval", 2024-09-11T00:32:24.573 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "secs", 2024-09-11T00:32:24.573 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.573 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:24.573 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "60", 2024-09-11T00:32:24.573 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.573 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.574 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.574 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.574 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.574 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.574 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.574 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.574 INFO:teuthology.orchestra.run.smithi012.stdout: "sqlite3_killpoint": { 2024-09-11T00:32:24.574 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "sqlite3_killpoint", 2024-09-11T00:32:24.574 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-09-11T00:32:24.574 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "dev", 2024-09-11T00:32:24.574 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:24.574 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "0", 2024-09-11T00:32:24.574 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.575 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.575 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.575 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.575 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.575 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.575 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.575 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.575 INFO:teuthology.orchestra.run.smithi012.stdout: "threshold": { 2024-09-11T00:32:24.575 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "threshold", 2024-09-11T00:32:24.575 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "float", 2024-09-11T00:32:24.575 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.575 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:24.576 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "3.0", 2024-09-11T00:32:24.576 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "1.0", 2024-09-11T00:32:24.576 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.576 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.576 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "scaling threshold", 2024-09-11T00:32:24.576 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-09-11T00:32:24.576 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.576 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.576 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:32:24.576 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:32:24.576 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.576 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-11T00:32:24.577 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "progress", 2024-09-11T00:32:24.577 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-09-11T00:32:24.577 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-09-11T00:32:24.577 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-09-11T00:32:24.577 INFO:teuthology.orchestra.run.smithi012.stdout: "allow_pg_recovery_event": { 2024-09-11T00:32:24.577 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "allow_pg_recovery_event", 2024-09-11T00:32:24.577 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-09-11T00:32:24.577 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.577 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:24.577 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-09-11T00:32:24.577 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.577 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.577 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.578 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "allow the module to show pg recovery progress", 2024-09-11T00:32:24.578 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.578 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.578 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.578 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.578 INFO:teuthology.orchestra.run.smithi012.stdout: "enabled": { 2024-09-11T00:32:24.578 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "enabled", 2024-09-11T00:32:24.578 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-09-11T00:32:24.578 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.578 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:24.578 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-09-11T00:32:24.579 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.579 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.579 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.579 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.579 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.579 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.579 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.579 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.579 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-09-11T00:32:24.579 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-09-11T00:32:24.579 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:24.579 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.580 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:24.580 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-09-11T00:32:24.580 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.580 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.580 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-09-11T00:32:24.580 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-09-11T00:32:24.580 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-09-11T00:32:24.580 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-09-11T00:32:24.580 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-09-11T00:32:24.580 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-09-11T00:32:24.580 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-09-11T00:32:24.580 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-09-11T00:32:24.580 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.580 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.581 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.581 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.581 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.581 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-09-11T00:32:24.581 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-09-11T00:32:24.581 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-09-11T00:32:24.581 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.581 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:24.581 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-09-11T00:32:24.581 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.581 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.581 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.582 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.582 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.582 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.582 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.582 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.582 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-09-11T00:32:24.582 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-09-11T00:32:24.582 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:24.582 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.582 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:24.582 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-09-11T00:32:24.582 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.582 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.583 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-09-11T00:32:24.583 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-09-11T00:32:24.583 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-09-11T00:32:24.583 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-09-11T00:32:24.583 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-09-11T00:32:24.583 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-09-11T00:32:24.583 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-09-11T00:32:24.583 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-09-11T00:32:24.583 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.583 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.583 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.583 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.584 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.584 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-09-11T00:32:24.584 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-09-11T00:32:24.584 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-09-11T00:32:24.584 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.584 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:24.584 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-09-11T00:32:24.584 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.584 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.584 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.584 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.584 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.584 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.584 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.585 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.585 INFO:teuthology.orchestra.run.smithi012.stdout: "max_completed_events": { 2024-09-11T00:32:24.585 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "max_completed_events", 2024-09-11T00:32:24.585 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-09-11T00:32:24.585 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.585 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:24.585 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "50", 2024-09-11T00:32:24.585 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.585 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.585 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.585 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "number of past completed events to remember", 2024-09-11T00:32:24.585 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.585 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.586 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.586 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.586 INFO:teuthology.orchestra.run.smithi012.stdout: "sleep_interval": { 2024-09-11T00:32:24.586 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "sleep_interval", 2024-09-11T00:32:24.586 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "secs", 2024-09-11T00:32:24.586 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.586 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:24.586 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "5", 2024-09-11T00:32:24.586 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.586 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.586 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.586 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "how long the module is going to sleep", 2024-09-11T00:32:24.586 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.587 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.587 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.587 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.587 INFO:teuthology.orchestra.run.smithi012.stdout: "sqlite3_killpoint": { 2024-09-11T00:32:24.587 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "sqlite3_killpoint", 2024-09-11T00:32:24.587 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-09-11T00:32:24.587 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "dev", 2024-09-11T00:32:24.587 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:24.587 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "0", 2024-09-11T00:32:24.587 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.587 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.587 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.587 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.588 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.588 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.588 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.588 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:32:24.588 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:32:24.588 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.588 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-11T00:32:24.588 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "prometheus", 2024-09-11T00:32:24.588 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-09-11T00:32:24.588 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-09-11T00:32:24.588 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-09-11T00:32:24.588 INFO:teuthology.orchestra.run.smithi012.stdout: "cache": { 2024-09-11T00:32:24.589 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "cache", 2024-09-11T00:32:24.589 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-09-11T00:32:24.589 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.589 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:24.589 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-09-11T00:32:24.589 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.589 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.589 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.589 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.589 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.589 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.589 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.589 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.590 INFO:teuthology.orchestra.run.smithi012.stdout: "exclude_perf_counters": { 2024-09-11T00:32:24.590 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "exclude_perf_counters", 2024-09-11T00:32:24.590 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-09-11T00:32:24.590 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.590 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:24.590 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-09-11T00:32:24.590 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.590 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.590 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.590 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-09-11T00:32:24.591 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-09-11T00:32:24.591 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.591 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.591 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.591 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-09-11T00:32:24.591 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-09-11T00:32:24.591 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:24.591 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.591 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:24.591 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-09-11T00:32:24.592 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.592 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.592 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-09-11T00:32:24.592 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-09-11T00:32:24.592 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-09-11T00:32:24.592 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-09-11T00:32:24.592 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-09-11T00:32:24.592 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-09-11T00:32:24.592 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-09-11T00:32:24.592 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-09-11T00:32:24.592 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.592 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.592 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.593 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.593 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.593 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-09-11T00:32:24.593 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-09-11T00:32:24.593 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-09-11T00:32:24.593 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.593 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:24.593 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-09-11T00:32:24.593 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.593 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.593 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.593 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.593 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.594 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.594 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.594 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.594 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-09-11T00:32:24.594 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-09-11T00:32:24.594 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:24.594 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.594 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:24.594 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-09-11T00:32:24.594 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.594 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.594 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-09-11T00:32:24.594 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-09-11T00:32:24.595 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-09-11T00:32:24.595 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-09-11T00:32:24.595 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-09-11T00:32:24.595 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-09-11T00:32:24.595 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-09-11T00:32:24.595 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-09-11T00:32:24.595 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.595 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.595 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.595 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.595 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.595 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-09-11T00:32:24.595 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-09-11T00:32:24.595 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-09-11T00:32:24.595 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.596 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:24.596 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-09-11T00:32:24.596 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.596 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.596 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.596 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.596 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.596 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.596 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.596 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.596 INFO:teuthology.orchestra.run.smithi012.stdout: "rbd_stats_pools": { 2024-09-11T00:32:24.596 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "rbd_stats_pools", 2024-09-11T00:32:24.597 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:24.597 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.597 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:24.597 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-09-11T00:32:24.597 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.597 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.597 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.597 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.597 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.597 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.597 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.597 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.597 INFO:teuthology.orchestra.run.smithi012.stdout: "rbd_stats_pools_refresh_interval": { 2024-09-11T00:32:24.598 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-09-11T00:32:24.598 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-09-11T00:32:24.598 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.598 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:24.598 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "300", 2024-09-11T00:32:24.598 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.598 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.598 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.598 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.598 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.598 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.598 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.598 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.599 INFO:teuthology.orchestra.run.smithi012.stdout: "scrape_interval": { 2024-09-11T00:32:24.599 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "scrape_interval", 2024-09-11T00:32:24.599 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "float", 2024-09-11T00:32:24.599 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.599 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:24.599 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "15.0", 2024-09-11T00:32:24.599 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.599 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.599 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.599 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.599 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.599 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.599 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.599 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.600 INFO:teuthology.orchestra.run.smithi012.stdout: "server_addr": { 2024-09-11T00:32:24.600 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "server_addr", 2024-09-11T00:32:24.600 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:24.600 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.600 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:24.600 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "::", 2024-09-11T00:32:24.600 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.600 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.600 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.600 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-09-11T00:32:24.600 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.600 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.601 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.601 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.601 INFO:teuthology.orchestra.run.smithi012.stdout: "server_port": { 2024-09-11T00:32:24.601 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "server_port", 2024-09-11T00:32:24.601 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-09-11T00:32:24.601 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.601 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:24.601 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "9283", 2024-09-11T00:32:24.601 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.601 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.601 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.601 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-09-11T00:32:24.601 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.601 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.602 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.602 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.602 INFO:teuthology.orchestra.run.smithi012.stdout: "sqlite3_killpoint": { 2024-09-11T00:32:24.602 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "sqlite3_killpoint", 2024-09-11T00:32:24.602 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-09-11T00:32:24.602 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "dev", 2024-09-11T00:32:24.602 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:24.602 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "0", 2024-09-11T00:32:24.602 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.602 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.602 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.602 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.602 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.603 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.603 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.603 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.603 INFO:teuthology.orchestra.run.smithi012.stdout: "stale_cache_strategy": { 2024-09-11T00:32:24.603 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "stale_cache_strategy", 2024-09-11T00:32:24.603 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:24.603 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.603 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:24.603 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "log", 2024-09-11T00:32:24.603 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.603 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.603 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.603 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.604 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.604 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.604 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.604 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.604 INFO:teuthology.orchestra.run.smithi012.stdout: "standby_behaviour": { 2024-09-11T00:32:24.604 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "standby_behaviour", 2024-09-11T00:32:24.604 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:24.604 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.604 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:24.604 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "default", 2024-09-11T00:32:24.604 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.604 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.605 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-09-11T00:32:24.605 INFO:teuthology.orchestra.run.smithi012.stdout: "default", 2024-09-11T00:32:24.605 INFO:teuthology.orchestra.run.smithi012.stdout: "error" 2024-09-11T00:32:24.605 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-09-11T00:32:24.605 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.605 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.605 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.605 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.605 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.605 INFO:teuthology.orchestra.run.smithi012.stdout: "standby_error_status_code": { 2024-09-11T00:32:24.605 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "standby_error_status_code", 2024-09-11T00:32:24.605 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-09-11T00:32:24.605 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.606 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:24.606 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "500", 2024-09-11T00:32:24.606 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "400", 2024-09-11T00:32:24.606 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "599", 2024-09-11T00:32:24.606 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.606 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.606 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.606 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.606 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.606 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:32:24.606 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:32:24.606 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.607 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-11T00:32:24.607 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "rbd_support", 2024-09-11T00:32:24.607 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-09-11T00:32:24.607 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-09-11T00:32:24.607 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-09-11T00:32:24.607 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-09-11T00:32:24.607 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-09-11T00:32:24.607 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:24.607 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.607 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:24.607 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-09-11T00:32:24.607 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.607 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.608 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-09-11T00:32:24.608 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-09-11T00:32:24.608 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-09-11T00:32:24.608 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-09-11T00:32:24.608 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-09-11T00:32:24.608 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-09-11T00:32:24.608 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-09-11T00:32:24.608 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-09-11T00:32:24.608 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.608 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.608 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.608 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.608 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.609 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-09-11T00:32:24.609 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-09-11T00:32:24.609 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-09-11T00:32:24.609 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.609 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:24.609 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-09-11T00:32:24.609 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.609 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.609 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.609 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.609 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.609 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.609 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.610 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.610 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-09-11T00:32:24.610 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-09-11T00:32:24.610 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:24.610 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.610 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:24.610 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-09-11T00:32:24.610 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.610 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.610 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-09-11T00:32:24.610 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-09-11T00:32:24.611 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-09-11T00:32:24.611 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-09-11T00:32:24.611 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-09-11T00:32:24.611 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-09-11T00:32:24.611 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-09-11T00:32:24.611 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-09-11T00:32:24.611 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.611 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.611 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.612 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.612 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.612 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-09-11T00:32:24.612 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-09-11T00:32:24.612 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-09-11T00:32:24.612 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.612 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:24.612 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-09-11T00:32:24.612 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.612 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.612 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.612 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.612 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.613 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.613 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.613 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.613 INFO:teuthology.orchestra.run.smithi012.stdout: "max_concurrent_snap_create": { 2024-09-11T00:32:24.613 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "max_concurrent_snap_create", 2024-09-11T00:32:24.613 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-09-11T00:32:24.613 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.613 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:24.613 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "10", 2024-09-11T00:32:24.613 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.613 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.613 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.613 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.614 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.614 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.614 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.614 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.614 INFO:teuthology.orchestra.run.smithi012.stdout: "mirror_snapshot_schedule": { 2024-09-11T00:32:24.614 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "mirror_snapshot_schedule", 2024-09-11T00:32:24.614 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:24.614 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.614 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:24.614 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-09-11T00:32:24.614 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.614 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.614 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.615 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.615 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.615 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.615 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.615 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.615 INFO:teuthology.orchestra.run.smithi012.stdout: "sqlite3_killpoint": { 2024-09-11T00:32:24.615 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "sqlite3_killpoint", 2024-09-11T00:32:24.615 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-09-11T00:32:24.615 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "dev", 2024-09-11T00:32:24.615 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:24.615 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "0", 2024-09-11T00:32:24.615 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.615 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.616 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.616 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.616 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.616 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.616 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.616 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.616 INFO:teuthology.orchestra.run.smithi012.stdout: "trash_purge_schedule": { 2024-09-11T00:32:24.616 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "trash_purge_schedule", 2024-09-11T00:32:24.616 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:24.616 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.616 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:24.616 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-09-11T00:32:24.617 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.617 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.617 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.617 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.617 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.617 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.617 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.617 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:32:24.617 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:32:24.617 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.617 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-11T00:32:24.617 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "restful", 2024-09-11T00:32:24.617 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-09-11T00:32:24.618 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-09-11T00:32:24.618 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-09-11T00:32:24.618 INFO:teuthology.orchestra.run.smithi012.stdout: "enable_auth": { 2024-09-11T00:32:24.618 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "enable_auth", 2024-09-11T00:32:24.618 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-09-11T00:32:24.618 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.618 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:24.618 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-09-11T00:32:24.618 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.618 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.618 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.618 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.618 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.619 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.619 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.619 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.619 INFO:teuthology.orchestra.run.smithi012.stdout: "key_file": { 2024-09-11T00:32:24.619 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "key_file", 2024-09-11T00:32:24.619 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:24.619 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.619 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:24.619 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-09-11T00:32:24.619 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.619 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.619 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.619 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.620 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.620 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.620 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.620 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.620 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-09-11T00:32:24.620 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-09-11T00:32:24.620 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:24.620 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.620 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:24.620 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-09-11T00:32:24.620 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.620 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.620 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-09-11T00:32:24.621 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-09-11T00:32:24.621 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-09-11T00:32:24.621 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-09-11T00:32:24.621 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-09-11T00:32:24.621 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-09-11T00:32:24.621 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-09-11T00:32:24.621 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-09-11T00:32:24.621 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.621 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.621 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.621 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.621 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.622 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-09-11T00:32:24.622 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-09-11T00:32:24.622 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-09-11T00:32:24.622 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.622 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:24.622 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-09-11T00:32:24.622 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.622 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.622 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.622 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.622 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.622 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.622 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.623 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.623 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-09-11T00:32:24.623 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-09-11T00:32:24.623 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:24.623 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.623 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:24.623 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-09-11T00:32:24.623 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.623 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.623 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-09-11T00:32:24.623 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-09-11T00:32:24.623 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-09-11T00:32:24.623 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-09-11T00:32:24.624 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-09-11T00:32:24.624 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-09-11T00:32:24.624 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-09-11T00:32:24.624 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-09-11T00:32:24.624 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.624 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.624 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.624 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.624 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.624 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-09-11T00:32:24.624 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-09-11T00:32:24.624 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-09-11T00:32:24.624 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.625 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:24.625 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-09-11T00:32:24.625 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.625 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.625 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.625 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.625 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.625 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.625 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.625 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.625 INFO:teuthology.orchestra.run.smithi012.stdout: "max_requests": { 2024-09-11T00:32:24.625 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "max_requests", 2024-09-11T00:32:24.625 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-09-11T00:32:24.626 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.626 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:24.626 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "500", 2024-09-11T00:32:24.626 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.626 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.626 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.626 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Maximum number of requests to keep in memory. When new request comes in, the oldest request will be removed if the number of requests exceeds the max request number.if un-finished request is removed, error message will be logged in the ceph-mgr log.", 2024-09-11T00:32:24.626 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.626 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.626 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.626 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.626 INFO:teuthology.orchestra.run.smithi012.stdout: "server_addr": { 2024-09-11T00:32:24.627 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "server_addr", 2024-09-11T00:32:24.627 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:24.627 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.627 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:24.627 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-09-11T00:32:24.627 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.627 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.627 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.627 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.627 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.627 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.627 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.627 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.627 INFO:teuthology.orchestra.run.smithi012.stdout: "server_port": { 2024-09-11T00:32:24.628 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "server_port", 2024-09-11T00:32:24.628 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:24.628 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.628 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:24.628 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-09-11T00:32:24.628 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.628 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.628 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.628 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.628 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.628 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.628 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.629 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.629 INFO:teuthology.orchestra.run.smithi012.stdout: "sqlite3_killpoint": { 2024-09-11T00:32:24.629 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "sqlite3_killpoint", 2024-09-11T00:32:24.629 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-09-11T00:32:24.629 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "dev", 2024-09-11T00:32:24.629 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:24.629 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "0", 2024-09-11T00:32:24.629 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.629 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.629 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.629 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.629 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.629 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.630 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.630 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:32:24.630 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:32:24.630 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.630 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-11T00:32:24.630 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "rgw", 2024-09-11T00:32:24.630 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-09-11T00:32:24.630 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-09-11T00:32:24.630 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-09-11T00:32:24.630 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-09-11T00:32:24.630 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-09-11T00:32:24.630 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:24.630 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.631 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:24.631 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-09-11T00:32:24.631 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.631 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.631 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-09-11T00:32:24.631 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-09-11T00:32:24.631 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-09-11T00:32:24.631 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-09-11T00:32:24.631 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-09-11T00:32:24.631 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-09-11T00:32:24.631 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-09-11T00:32:24.631 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-09-11T00:32:24.631 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.632 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.632 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.632 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.632 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.632 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-09-11T00:32:24.632 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-09-11T00:32:24.632 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-09-11T00:32:24.632 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.632 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:24.632 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-09-11T00:32:24.632 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.632 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.632 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.633 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.633 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.633 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.633 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.633 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.633 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-09-11T00:32:24.633 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-09-11T00:32:24.633 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:24.633 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.633 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:24.633 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-09-11T00:32:24.633 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.633 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.633 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-09-11T00:32:24.634 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-09-11T00:32:24.634 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-09-11T00:32:24.634 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-09-11T00:32:24.634 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-09-11T00:32:24.634 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-09-11T00:32:24.634 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-09-11T00:32:24.634 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-09-11T00:32:24.634 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.634 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.634 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.634 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.634 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.634 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-09-11T00:32:24.635 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-09-11T00:32:24.635 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-09-11T00:32:24.635 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.635 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:24.635 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-09-11T00:32:24.635 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.635 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.635 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.635 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.635 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.636 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.636 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.636 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.636 INFO:teuthology.orchestra.run.smithi012.stdout: "secondary_zone_period_retry_limit": { 2024-09-11T00:32:24.636 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "secondary_zone_period_retry_limit", 2024-09-11T00:32:24.636 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-09-11T00:32:24.636 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.636 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:24.636 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "5", 2024-09-11T00:32:24.636 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.636 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.637 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.637 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "RGW module period update retry limit for secondary site", 2024-09-11T00:32:24.637 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.637 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.637 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.637 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.637 INFO:teuthology.orchestra.run.smithi012.stdout: "sqlite3_killpoint": { 2024-09-11T00:32:24.637 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "sqlite3_killpoint", 2024-09-11T00:32:24.637 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-09-11T00:32:24.637 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "dev", 2024-09-11T00:32:24.637 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:24.637 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "0", 2024-09-11T00:32:24.637 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.638 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.638 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.638 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.638 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.638 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.638 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.638 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:32:24.638 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:32:24.638 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.638 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-11T00:32:24.638 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "rook", 2024-09-11T00:32:24.638 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-09-11T00:32:24.638 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-09-11T00:32:24.638 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-09-11T00:32:24.639 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-09-11T00:32:24.639 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-09-11T00:32:24.639 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:24.639 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.639 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:24.639 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-09-11T00:32:24.639 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.639 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.639 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-09-11T00:32:24.639 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-09-11T00:32:24.639 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-09-11T00:32:24.639 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-09-11T00:32:24.639 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-09-11T00:32:24.640 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-09-11T00:32:24.640 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-09-11T00:32:24.640 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-09-11T00:32:24.640 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.640 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.640 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.640 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.640 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.640 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-09-11T00:32:24.640 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-09-11T00:32:24.640 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-09-11T00:32:24.640 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.640 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:24.641 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-09-11T00:32:24.641 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.641 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.641 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.641 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.641 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.641 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.641 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.641 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.641 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-09-11T00:32:24.641 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-09-11T00:32:24.641 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:24.641 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.641 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:24.642 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-09-11T00:32:24.642 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.642 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.642 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-09-11T00:32:24.642 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-09-11T00:32:24.642 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-09-11T00:32:24.642 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-09-11T00:32:24.642 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-09-11T00:32:24.642 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-09-11T00:32:24.642 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-09-11T00:32:24.642 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-09-11T00:32:24.642 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.643 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.643 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.643 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.643 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.643 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-09-11T00:32:24.643 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-09-11T00:32:24.643 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-09-11T00:32:24.643 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.643 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:24.643 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-09-11T00:32:24.643 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.643 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.643 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.644 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.644 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.644 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.644 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.644 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.644 INFO:teuthology.orchestra.run.smithi012.stdout: "sqlite3_killpoint": { 2024-09-11T00:32:24.644 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "sqlite3_killpoint", 2024-09-11T00:32:24.644 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-09-11T00:32:24.644 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "dev", 2024-09-11T00:32:24.644 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:24.644 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "0", 2024-09-11T00:32:24.644 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.644 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.645 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.645 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.645 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.645 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.645 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.645 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.645 INFO:teuthology.orchestra.run.smithi012.stdout: "storage_class": { 2024-09-11T00:32:24.645 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "storage_class", 2024-09-11T00:32:24.645 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:24.645 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.645 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:24.645 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "local", 2024-09-11T00:32:24.646 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.646 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.646 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.646 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-09-11T00:32:24.646 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.646 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.646 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.646 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:32:24.646 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:32:24.646 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.646 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-11T00:32:24.646 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "selftest", 2024-09-11T00:32:24.646 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-09-11T00:32:24.647 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-09-11T00:32:24.647 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-09-11T00:32:24.647 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-09-11T00:32:24.647 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-09-11T00:32:24.647 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:24.647 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.647 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:24.647 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-09-11T00:32:24.647 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.647 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.647 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-09-11T00:32:24.647 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-09-11T00:32:24.647 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-09-11T00:32:24.648 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-09-11T00:32:24.648 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-09-11T00:32:24.648 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-09-11T00:32:24.648 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-09-11T00:32:24.648 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-09-11T00:32:24.648 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.648 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.648 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.648 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.648 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.648 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-09-11T00:32:24.648 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-09-11T00:32:24.648 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-09-11T00:32:24.649 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.649 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:24.649 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-09-11T00:32:24.649 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.649 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.649 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.649 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.649 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.649 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.649 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.649 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.649 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-09-11T00:32:24.650 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-09-11T00:32:24.650 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:24.650 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.650 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:24.650 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-09-11T00:32:24.650 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.650 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.650 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-09-11T00:32:24.650 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-09-11T00:32:24.650 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-09-11T00:32:24.650 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-09-11T00:32:24.650 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-09-11T00:32:24.650 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-09-11T00:32:24.651 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-09-11T00:32:24.651 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-09-11T00:32:24.651 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.651 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.651 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.651 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.651 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.651 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-09-11T00:32:24.651 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-09-11T00:32:24.651 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-09-11T00:32:24.651 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.651 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:24.651 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-09-11T00:32:24.651 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.652 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.652 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.652 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.652 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.652 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.652 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.652 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.652 INFO:teuthology.orchestra.run.smithi012.stdout: "roption1": { 2024-09-11T00:32:24.652 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "roption1", 2024-09-11T00:32:24.652 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:24.652 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.652 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:24.653 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-09-11T00:32:24.653 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.653 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.653 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.653 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.653 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.653 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.653 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.653 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.653 INFO:teuthology.orchestra.run.smithi012.stdout: "roption2": { 2024-09-11T00:32:24.653 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "roption2", 2024-09-11T00:32:24.653 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:24.654 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.654 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:24.654 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "xyz", 2024-09-11T00:32:24.654 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.654 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.654 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.654 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.654 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.654 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.654 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.654 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.654 INFO:teuthology.orchestra.run.smithi012.stdout: "rwoption1": { 2024-09-11T00:32:24.654 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "rwoption1", 2024-09-11T00:32:24.655 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:24.655 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.655 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:24.655 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-09-11T00:32:24.655 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.655 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.655 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.655 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.655 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.655 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.655 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.655 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.655 INFO:teuthology.orchestra.run.smithi012.stdout: "rwoption2": { 2024-09-11T00:32:24.656 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "rwoption2", 2024-09-11T00:32:24.656 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-09-11T00:32:24.656 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.656 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:24.656 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-09-11T00:32:24.656 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.656 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.656 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.656 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.656 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.656 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.656 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.657 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.657 INFO:teuthology.orchestra.run.smithi012.stdout: "rwoption3": { 2024-09-11T00:32:24.657 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "rwoption3", 2024-09-11T00:32:24.657 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "float", 2024-09-11T00:32:24.657 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.657 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:24.657 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-09-11T00:32:24.657 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.657 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.657 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.657 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.657 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.657 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.658 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.658 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.658 INFO:teuthology.orchestra.run.smithi012.stdout: "rwoption4": { 2024-09-11T00:32:24.658 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "rwoption4", 2024-09-11T00:32:24.658 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:24.658 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.658 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:24.658 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-09-11T00:32:24.658 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.658 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.658 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.658 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.658 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.658 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.659 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.659 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.659 INFO:teuthology.orchestra.run.smithi012.stdout: "rwoption5": { 2024-09-11T00:32:24.659 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "rwoption5", 2024-09-11T00:32:24.659 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-09-11T00:32:24.659 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.659 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:24.659 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-09-11T00:32:24.659 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.659 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.659 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.659 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.660 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.660 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.660 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.660 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.660 INFO:teuthology.orchestra.run.smithi012.stdout: "rwoption6": { 2024-09-11T00:32:24.660 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "rwoption6", 2024-09-11T00:32:24.660 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-09-11T00:32:24.660 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.660 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:24.660 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-09-11T00:32:24.660 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.660 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.660 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.661 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.661 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.661 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.661 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.661 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.661 INFO:teuthology.orchestra.run.smithi012.stdout: "rwoption7": { 2024-09-11T00:32:24.661 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "rwoption7", 2024-09-11T00:32:24.661 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-09-11T00:32:24.661 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.661 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:24.661 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-09-11T00:32:24.661 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "1", 2024-09-11T00:32:24.661 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "42", 2024-09-11T00:32:24.662 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.662 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.662 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.662 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.662 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.662 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.662 INFO:teuthology.orchestra.run.smithi012.stdout: "sqlite3_killpoint": { 2024-09-11T00:32:24.662 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "sqlite3_killpoint", 2024-09-11T00:32:24.662 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-09-11T00:32:24.663 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "dev", 2024-09-11T00:32:24.663 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:24.663 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "0", 2024-09-11T00:32:24.663 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.663 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.663 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.663 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.663 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.663 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.663 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.664 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.664 INFO:teuthology.orchestra.run.smithi012.stdout: "testkey": { 2024-09-11T00:32:24.664 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "testkey", 2024-09-11T00:32:24.664 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:24.664 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.664 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:24.664 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-09-11T00:32:24.664 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.664 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.664 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.664 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.664 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.664 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.665 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.665 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.665 INFO:teuthology.orchestra.run.smithi012.stdout: "testlkey": { 2024-09-11T00:32:24.665 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "testlkey", 2024-09-11T00:32:24.665 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:24.665 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.665 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:24.665 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-09-11T00:32:24.665 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.665 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.665 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.665 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.666 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.666 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.666 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.666 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.666 INFO:teuthology.orchestra.run.smithi012.stdout: "testnewline": { 2024-09-11T00:32:24.666 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "testnewline", 2024-09-11T00:32:24.666 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:24.666 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.666 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:24.666 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-09-11T00:32:24.666 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.666 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.666 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.667 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.667 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.667 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.667 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.667 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:32:24.667 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:32:24.667 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.667 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-11T00:32:24.667 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "smb", 2024-09-11T00:32:24.667 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-09-11T00:32:24.667 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-09-11T00:32:24.667 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-09-11T00:32:24.667 INFO:teuthology.orchestra.run.smithi012.stdout: "internal_store_backend": { 2024-09-11T00:32:24.668 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "internal_store_backend", 2024-09-11T00:32:24.668 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:24.668 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "dev", 2024-09-11T00:32:24.668 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:24.668 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-09-11T00:32:24.668 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.668 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.668 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.668 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "set internal store backend. for develoment and testing only", 2024-09-11T00:32:24.668 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.668 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.668 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.668 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.669 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-09-11T00:32:24.669 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-09-11T00:32:24.669 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:24.669 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.669 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:24.669 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-09-11T00:32:24.669 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.669 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.669 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-09-11T00:32:24.669 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-09-11T00:32:24.669 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-09-11T00:32:24.669 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-09-11T00:32:24.669 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-09-11T00:32:24.670 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-09-11T00:32:24.670 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-09-11T00:32:24.670 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-09-11T00:32:24.670 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.670 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.670 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.670 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.670 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.670 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-09-11T00:32:24.670 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-09-11T00:32:24.670 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-09-11T00:32:24.670 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.670 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:24.671 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-09-11T00:32:24.671 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.671 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.671 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.671 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.671 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.671 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.671 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.672 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.672 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-09-11T00:32:24.672 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-09-11T00:32:24.672 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:24.672 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.672 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:24.672 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-09-11T00:32:24.672 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.672 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.672 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-09-11T00:32:24.672 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-09-11T00:32:24.672 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-09-11T00:32:24.672 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-09-11T00:32:24.673 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-09-11T00:32:24.673 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-09-11T00:32:24.673 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-09-11T00:32:24.673 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-09-11T00:32:24.673 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.673 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.673 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.673 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.673 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.673 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-09-11T00:32:24.673 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-09-11T00:32:24.673 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-09-11T00:32:24.674 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.674 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:24.674 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-09-11T00:32:24.674 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.674 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.674 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.674 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.674 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.674 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.674 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.674 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.674 INFO:teuthology.orchestra.run.smithi012.stdout: "sqlite3_killpoint": { 2024-09-11T00:32:24.674 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "sqlite3_killpoint", 2024-09-11T00:32:24.675 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-09-11T00:32:24.675 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "dev", 2024-09-11T00:32:24.675 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:24.675 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "0", 2024-09-11T00:32:24.675 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.675 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.675 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.675 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.675 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.675 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.675 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.675 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.675 INFO:teuthology.orchestra.run.smithi012.stdout: "update_orchestration": { 2024-09-11T00:32:24.675 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "update_orchestration", 2024-09-11T00:32:24.676 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-09-11T00:32:24.676 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.676 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:24.676 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-09-11T00:32:24.676 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.676 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.676 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.676 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "automatically update orchestration when smb resources are changed", 2024-09-11T00:32:24.676 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.676 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.676 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.676 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:32:24.676 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:32:24.677 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.677 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-11T00:32:24.677 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "snap_schedule", 2024-09-11T00:32:24.677 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-09-11T00:32:24.677 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-09-11T00:32:24.677 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-09-11T00:32:24.677 INFO:teuthology.orchestra.run.smithi012.stdout: "allow_m_granularity": { 2024-09-11T00:32:24.677 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "allow_m_granularity", 2024-09-11T00:32:24.677 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-09-11T00:32:24.677 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.677 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:24.677 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-09-11T00:32:24.677 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.677 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.678 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.678 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "allow minute scheduled snapshots", 2024-09-11T00:32:24.678 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.678 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.678 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.678 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.678 INFO:teuthology.orchestra.run.smithi012.stdout: "dump_on_update": { 2024-09-11T00:32:24.678 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "dump_on_update", 2024-09-11T00:32:24.678 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-09-11T00:32:24.678 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.678 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:24.678 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-09-11T00:32:24.678 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.679 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.679 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.679 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "dump database to debug log on update", 2024-09-11T00:32:24.679 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.679 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.679 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.679 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.679 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-09-11T00:32:24.679 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-09-11T00:32:24.679 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:24.679 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.679 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:24.679 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-09-11T00:32:24.679 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.680 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.680 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-09-11T00:32:24.680 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-09-11T00:32:24.680 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-09-11T00:32:24.680 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-09-11T00:32:24.680 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-09-11T00:32:24.680 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-09-11T00:32:24.680 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-09-11T00:32:24.680 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-09-11T00:32:24.680 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.680 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.680 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.681 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.681 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.681 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-09-11T00:32:24.681 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-09-11T00:32:24.681 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-09-11T00:32:24.681 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.681 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:24.681 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-09-11T00:32:24.681 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.681 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.681 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.681 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.682 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.682 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.682 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.682 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.682 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-09-11T00:32:24.682 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-09-11T00:32:24.682 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:24.682 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.682 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:24.682 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-09-11T00:32:24.682 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.682 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.682 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-09-11T00:32:24.683 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-09-11T00:32:24.683 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-09-11T00:32:24.683 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-09-11T00:32:24.683 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-09-11T00:32:24.683 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-09-11T00:32:24.683 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-09-11T00:32:24.683 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-09-11T00:32:24.683 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.683 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.683 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.683 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.683 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.683 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-09-11T00:32:24.684 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-09-11T00:32:24.684 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-09-11T00:32:24.684 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.684 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:24.684 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-09-11T00:32:24.684 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.684 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.684 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.684 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.684 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.684 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.684 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.684 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.685 INFO:teuthology.orchestra.run.smithi012.stdout: "sqlite3_killpoint": { 2024-09-11T00:32:24.685 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "sqlite3_killpoint", 2024-09-11T00:32:24.685 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-09-11T00:32:24.685 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "dev", 2024-09-11T00:32:24.685 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:24.685 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "0", 2024-09-11T00:32:24.685 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.685 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.685 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.685 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.685 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.685 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.685 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.686 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:32:24.686 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:32:24.686 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.686 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-11T00:32:24.686 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "stats", 2024-09-11T00:32:24.686 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-09-11T00:32:24.686 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-09-11T00:32:24.686 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-09-11T00:32:24.686 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-09-11T00:32:24.686 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-09-11T00:32:24.686 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:24.687 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.687 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:24.687 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-09-11T00:32:24.687 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.687 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.687 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-09-11T00:32:24.687 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-09-11T00:32:24.687 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-09-11T00:32:24.688 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-09-11T00:32:24.688 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-09-11T00:32:24.688 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-09-11T00:32:24.688 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-09-11T00:32:24.688 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-09-11T00:32:24.688 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.688 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.688 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.688 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.688 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.688 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-09-11T00:32:24.688 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-09-11T00:32:24.689 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-09-11T00:32:24.689 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.689 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:24.689 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-09-11T00:32:24.689 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.689 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.689 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.689 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.689 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.689 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.689 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.689 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.689 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-09-11T00:32:24.690 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-09-11T00:32:24.690 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:24.690 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.690 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:24.690 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-09-11T00:32:24.690 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.690 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.690 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-09-11T00:32:24.690 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-09-11T00:32:24.690 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-09-11T00:32:24.690 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-09-11T00:32:24.690 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-09-11T00:32:24.690 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-09-11T00:32:24.691 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-09-11T00:32:24.691 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-09-11T00:32:24.691 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.691 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.691 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.691 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.691 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.691 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-09-11T00:32:24.691 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-09-11T00:32:24.691 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-09-11T00:32:24.691 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.692 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:24.692 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-09-11T00:32:24.692 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.692 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.692 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.692 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.692 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.692 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.692 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.692 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.692 INFO:teuthology.orchestra.run.smithi012.stdout: "sqlite3_killpoint": { 2024-09-11T00:32:24.692 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "sqlite3_killpoint", 2024-09-11T00:32:24.692 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-09-11T00:32:24.693 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "dev", 2024-09-11T00:32:24.693 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:24.693 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "0", 2024-09-11T00:32:24.693 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.693 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.693 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.693 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.696 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.696 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.696 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.696 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:32:24.696 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:32:24.696 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.696 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-11T00:32:24.697 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "status", 2024-09-11T00:32:24.697 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-09-11T00:32:24.697 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-09-11T00:32:24.697 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-09-11T00:32:24.697 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-09-11T00:32:24.697 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-09-11T00:32:24.697 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:24.697 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.697 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:24.697 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-09-11T00:32:24.697 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.697 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.697 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-09-11T00:32:24.698 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-09-11T00:32:24.698 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-09-11T00:32:24.698 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-09-11T00:32:24.698 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-09-11T00:32:24.698 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-09-11T00:32:24.698 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-09-11T00:32:24.698 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-09-11T00:32:24.698 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.698 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.698 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.698 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.698 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.698 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-09-11T00:32:24.698 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-09-11T00:32:24.699 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-09-11T00:32:24.699 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.699 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:24.699 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-09-11T00:32:24.699 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.699 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.699 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.699 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.699 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.699 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.699 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.699 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.699 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-09-11T00:32:24.700 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-09-11T00:32:24.700 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:24.700 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.700 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:24.700 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-09-11T00:32:24.700 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.700 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.700 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-09-11T00:32:24.700 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-09-11T00:32:24.700 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-09-11T00:32:24.700 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-09-11T00:32:24.700 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-09-11T00:32:24.700 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-09-11T00:32:24.701 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-09-11T00:32:24.701 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-09-11T00:32:24.701 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.701 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.701 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.701 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.701 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.701 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-09-11T00:32:24.701 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-09-11T00:32:24.701 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-09-11T00:32:24.701 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.701 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:24.701 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-09-11T00:32:24.702 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.702 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.702 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.702 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.702 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.702 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.702 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.702 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.702 INFO:teuthology.orchestra.run.smithi012.stdout: "sqlite3_killpoint": { 2024-09-11T00:32:24.702 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "sqlite3_killpoint", 2024-09-11T00:32:24.702 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-09-11T00:32:24.702 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "dev", 2024-09-11T00:32:24.702 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:24.703 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "0", 2024-09-11T00:32:24.703 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.703 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.703 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.703 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.703 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.703 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.703 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.703 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:32:24.703 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:32:24.703 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.703 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-11T00:32:24.703 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "telegraf", 2024-09-11T00:32:24.704 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-09-11T00:32:24.704 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-09-11T00:32:24.704 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-09-11T00:32:24.704 INFO:teuthology.orchestra.run.smithi012.stdout: "address": { 2024-09-11T00:32:24.704 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "address", 2024-09-11T00:32:24.704 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:24.704 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.704 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:24.704 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-09-11T00:32:24.704 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.704 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.704 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.704 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.705 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.705 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.705 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.705 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.705 INFO:teuthology.orchestra.run.smithi012.stdout: "interval": { 2024-09-11T00:32:24.705 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "interval", 2024-09-11T00:32:24.705 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "secs", 2024-09-11T00:32:24.705 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.705 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:24.705 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "15", 2024-09-11T00:32:24.705 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.705 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.705 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.706 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.706 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.706 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.706 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.706 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.706 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-09-11T00:32:24.706 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-09-11T00:32:24.706 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:24.706 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.706 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:24.706 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-09-11T00:32:24.706 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.706 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.707 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-09-11T00:32:24.707 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-09-11T00:32:24.707 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-09-11T00:32:24.707 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-09-11T00:32:24.707 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-09-11T00:32:24.707 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-09-11T00:32:24.707 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-09-11T00:32:24.707 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-09-11T00:32:24.707 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.707 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.707 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.708 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.708 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.708 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-09-11T00:32:24.708 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-09-11T00:32:24.708 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-09-11T00:32:24.708 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.708 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:24.708 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-09-11T00:32:24.708 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.708 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.708 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.709 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.709 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.709 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.709 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.709 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.709 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-09-11T00:32:24.709 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-09-11T00:32:24.709 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:24.709 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.709 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:24.709 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-09-11T00:32:24.709 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.710 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.710 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-09-11T00:32:24.710 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-09-11T00:32:24.710 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-09-11T00:32:24.710 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-09-11T00:32:24.710 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-09-11T00:32:24.710 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-09-11T00:32:24.710 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-09-11T00:32:24.710 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-09-11T00:32:24.710 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.710 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.710 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.711 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.711 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.711 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-09-11T00:32:24.711 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-09-11T00:32:24.711 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-09-11T00:32:24.711 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.711 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:24.711 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-09-11T00:32:24.711 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.711 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.711 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.711 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.711 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.712 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.712 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.712 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.712 INFO:teuthology.orchestra.run.smithi012.stdout: "sqlite3_killpoint": { 2024-09-11T00:32:24.712 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "sqlite3_killpoint", 2024-09-11T00:32:24.712 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-09-11T00:32:24.712 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "dev", 2024-09-11T00:32:24.712 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:24.712 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "0", 2024-09-11T00:32:24.712 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.712 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.713 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.713 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.713 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.713 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.713 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.713 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:32:24.713 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:32:24.713 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.713 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-11T00:32:24.713 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "telemetry", 2024-09-11T00:32:24.713 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-09-11T00:32:24.713 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-09-11T00:32:24.713 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-09-11T00:32:24.714 INFO:teuthology.orchestra.run.smithi012.stdout: "channel_basic": { 2024-09-11T00:32:24.714 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "channel_basic", 2024-09-11T00:32:24.714 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-09-11T00:32:24.714 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.714 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:24.714 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-09-11T00:32:24.714 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.714 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.714 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.714 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Share basic cluster information (size, version)", 2024-09-11T00:32:24.714 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.714 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.714 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.715 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.715 INFO:teuthology.orchestra.run.smithi012.stdout: "channel_crash": { 2024-09-11T00:32:24.715 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "channel_crash", 2024-09-11T00:32:24.715 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-09-11T00:32:24.715 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.715 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:24.715 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-09-11T00:32:24.715 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.715 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.715 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.715 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-09-11T00:32:24.715 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.715 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.716 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.716 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.716 INFO:teuthology.orchestra.run.smithi012.stdout: "channel_device": { 2024-09-11T00:32:24.716 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "channel_device", 2024-09-11T00:32:24.716 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-09-11T00:32:24.716 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.716 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:24.716 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-09-11T00:32:24.716 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.716 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.716 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.716 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-09-11T00:32:24.717 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.717 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.717 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.717 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.717 INFO:teuthology.orchestra.run.smithi012.stdout: "channel_ident": { 2024-09-11T00:32:24.717 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "channel_ident", 2024-09-11T00:32:24.717 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-09-11T00:32:24.717 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.717 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:24.717 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-09-11T00:32:24.717 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.717 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.717 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.718 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-09-11T00:32:24.718 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.718 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.718 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.718 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.718 INFO:teuthology.orchestra.run.smithi012.stdout: "channel_perf": { 2024-09-11T00:32:24.718 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "channel_perf", 2024-09-11T00:32:24.718 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-09-11T00:32:24.718 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.718 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:24.718 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-09-11T00:32:24.718 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.719 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.719 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.719 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Share various performance metrics of a cluster", 2024-09-11T00:32:24.719 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.719 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.719 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.719 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.719 INFO:teuthology.orchestra.run.smithi012.stdout: "contact": { 2024-09-11T00:32:24.719 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "contact", 2024-09-11T00:32:24.719 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:24.719 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.720 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:24.720 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-09-11T00:32:24.720 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.720 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.720 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.720 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.720 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.720 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.720 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.720 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.720 INFO:teuthology.orchestra.run.smithi012.stdout: "description": { 2024-09-11T00:32:24.721 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "description", 2024-09-11T00:32:24.721 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:24.721 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.721 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:24.721 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-09-11T00:32:24.721 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.721 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.721 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.721 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.722 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.722 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.722 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.722 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.722 INFO:teuthology.orchestra.run.smithi012.stdout: "device_url": { 2024-09-11T00:32:24.722 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "device_url", 2024-09-11T00:32:24.722 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:24.722 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.722 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:24.722 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-09-11T00:32:24.722 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.723 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.723 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.723 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.723 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.723 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.723 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.723 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.723 INFO:teuthology.orchestra.run.smithi012.stdout: "enabled": { 2024-09-11T00:32:24.723 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "enabled", 2024-09-11T00:32:24.723 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-09-11T00:32:24.723 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.723 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:24.724 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-09-11T00:32:24.724 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.724 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.724 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.724 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.724 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.724 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.724 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.724 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.724 INFO:teuthology.orchestra.run.smithi012.stdout: "interval": { 2024-09-11T00:32:24.724 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "interval", 2024-09-11T00:32:24.724 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-09-11T00:32:24.724 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.725 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:24.725 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "24", 2024-09-11T00:32:24.725 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "8", 2024-09-11T00:32:24.725 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.725 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.725 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.725 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.725 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.725 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.725 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.725 INFO:teuthology.orchestra.run.smithi012.stdout: "last_opt_revision": { 2024-09-11T00:32:24.725 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "last_opt_revision", 2024-09-11T00:32:24.726 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-09-11T00:32:24.726 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.726 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:24.726 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "1", 2024-09-11T00:32:24.726 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.726 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.726 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.726 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.726 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.726 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.726 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.726 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.726 INFO:teuthology.orchestra.run.smithi012.stdout: "leaderboard": { 2024-09-11T00:32:24.726 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "leaderboard", 2024-09-11T00:32:24.727 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-09-11T00:32:24.727 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.727 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:24.727 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-09-11T00:32:24.727 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.727 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.727 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.727 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.727 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.727 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.727 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.727 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.727 INFO:teuthology.orchestra.run.smithi012.stdout: "leaderboard_description": { 2024-09-11T00:32:24.728 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "leaderboard_description", 2024-09-11T00:32:24.728 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:24.728 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.728 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:24.728 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-09-11T00:32:24.728 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.728 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.728 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.728 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.728 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.728 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.728 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.728 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.729 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-09-11T00:32:24.729 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-09-11T00:32:24.729 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:24.729 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.729 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:24.729 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-09-11T00:32:24.729 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.729 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.729 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-09-11T00:32:24.729 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-09-11T00:32:24.729 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-09-11T00:32:24.729 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-09-11T00:32:24.729 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-09-11T00:32:24.730 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-09-11T00:32:24.730 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-09-11T00:32:24.730 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-09-11T00:32:24.730 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.730 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.730 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.730 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.730 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.730 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-09-11T00:32:24.730 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-09-11T00:32:24.730 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-09-11T00:32:24.730 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.730 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:24.731 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-09-11T00:32:24.731 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.731 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.731 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.731 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.731 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.731 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.731 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.731 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.731 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-09-11T00:32:24.731 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-09-11T00:32:24.731 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:24.731 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.731 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:24.732 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-09-11T00:32:24.732 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.732 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.732 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-09-11T00:32:24.732 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-09-11T00:32:24.732 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-09-11T00:32:24.732 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-09-11T00:32:24.732 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-09-11T00:32:24.732 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-09-11T00:32:24.732 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-09-11T00:32:24.732 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-09-11T00:32:24.732 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.732 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.733 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.733 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.733 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.733 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-09-11T00:32:24.733 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-09-11T00:32:24.733 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-09-11T00:32:24.733 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.733 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:24.733 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-09-11T00:32:24.733 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.733 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.733 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.733 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.734 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.734 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.734 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.734 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.734 INFO:teuthology.orchestra.run.smithi012.stdout: "organization": { 2024-09-11T00:32:24.734 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "organization", 2024-09-11T00:32:24.734 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:24.734 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.734 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:24.734 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-09-11T00:32:24.734 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.734 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.734 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.734 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.735 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.735 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.735 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.735 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.735 INFO:teuthology.orchestra.run.smithi012.stdout: "proxy": { 2024-09-11T00:32:24.735 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "proxy", 2024-09-11T00:32:24.735 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:24.735 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.735 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:24.735 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-09-11T00:32:24.735 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.735 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.735 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.735 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.736 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.736 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.736 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.736 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.736 INFO:teuthology.orchestra.run.smithi012.stdout: "sqlite3_killpoint": { 2024-09-11T00:32:24.736 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "sqlite3_killpoint", 2024-09-11T00:32:24.736 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-09-11T00:32:24.736 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "dev", 2024-09-11T00:32:24.736 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:24.736 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "0", 2024-09-11T00:32:24.736 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.736 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.737 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.737 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.737 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.737 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.737 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.737 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.737 INFO:teuthology.orchestra.run.smithi012.stdout: "url": { 2024-09-11T00:32:24.737 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "url", 2024-09-11T00:32:24.737 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:24.737 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.737 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:24.737 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-09-11T00:32:24.737 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.737 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.738 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.738 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.738 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.738 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.738 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.738 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:32:24.738 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:32:24.738 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.738 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-11T00:32:24.738 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "test_orchestrator", 2024-09-11T00:32:24.738 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-09-11T00:32:24.738 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-09-11T00:32:24.738 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-09-11T00:32:24.739 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-09-11T00:32:24.739 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-09-11T00:32:24.739 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:24.739 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.739 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:24.739 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-09-11T00:32:24.739 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.739 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.739 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-09-11T00:32:24.739 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-09-11T00:32:24.739 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-09-11T00:32:24.739 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-09-11T00:32:24.739 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-09-11T00:32:24.740 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-09-11T00:32:24.740 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-09-11T00:32:24.740 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-09-11T00:32:24.740 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.740 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.740 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.740 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.740 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.740 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-09-11T00:32:24.740 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-09-11T00:32:24.740 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-09-11T00:32:24.740 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.741 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:24.741 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-09-11T00:32:24.741 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.741 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.741 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.741 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.741 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.741 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.741 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.741 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.741 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-09-11T00:32:24.741 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-09-11T00:32:24.741 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:24.742 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.742 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:24.742 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-09-11T00:32:24.742 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.742 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.742 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-09-11T00:32:24.742 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-09-11T00:32:24.742 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-09-11T00:32:24.742 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-09-11T00:32:24.742 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-09-11T00:32:24.742 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-09-11T00:32:24.742 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-09-11T00:32:24.742 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-09-11T00:32:24.742 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.743 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.743 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.743 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.743 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.743 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-09-11T00:32:24.743 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-09-11T00:32:24.743 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-09-11T00:32:24.743 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.743 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:24.743 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-09-11T00:32:24.743 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.743 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.744 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.744 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.744 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.744 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.744 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.744 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.744 INFO:teuthology.orchestra.run.smithi012.stdout: "sqlite3_killpoint": { 2024-09-11T00:32:24.744 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "sqlite3_killpoint", 2024-09-11T00:32:24.744 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-09-11T00:32:24.744 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "dev", 2024-09-11T00:32:24.744 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:24.744 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "0", 2024-09-11T00:32:24.745 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.745 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.745 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.745 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.745 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.745 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.745 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.745 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:32:24.745 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:32:24.745 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.745 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-11T00:32:24.745 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "volumes", 2024-09-11T00:32:24.746 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-09-11T00:32:24.746 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-09-11T00:32:24.746 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-09-11T00:32:24.746 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-09-11T00:32:24.746 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-09-11T00:32:24.746 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:24.746 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.746 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:24.746 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-09-11T00:32:24.747 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.747 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.747 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-09-11T00:32:24.747 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-09-11T00:32:24.747 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-09-11T00:32:24.747 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-09-11T00:32:24.747 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-09-11T00:32:24.747 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-09-11T00:32:24.747 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-09-11T00:32:24.747 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-09-11T00:32:24.747 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.747 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.747 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.748 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.748 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.748 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-09-11T00:32:24.748 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-09-11T00:32:24.748 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-09-11T00:32:24.748 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.748 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:24.748 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-09-11T00:32:24.748 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.748 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.748 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.748 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.748 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.749 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.749 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.749 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.749 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-09-11T00:32:24.749 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-09-11T00:32:24.749 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:24.749 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.749 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:24.749 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-09-11T00:32:24.749 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.749 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.749 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-09-11T00:32:24.749 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-09-11T00:32:24.750 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-09-11T00:32:24.750 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-09-11T00:32:24.750 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-09-11T00:32:24.750 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-09-11T00:32:24.750 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-09-11T00:32:24.750 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-09-11T00:32:24.750 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.750 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.750 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.750 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.750 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.750 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-09-11T00:32:24.750 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-09-11T00:32:24.751 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-09-11T00:32:24.751 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.751 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:24.751 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-09-11T00:32:24.751 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.751 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.751 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.751 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.751 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.751 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.751 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.751 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.751 INFO:teuthology.orchestra.run.smithi012.stdout: "max_concurrent_clones": { 2024-09-11T00:32:24.751 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "max_concurrent_clones", 2024-09-11T00:32:24.752 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-09-11T00:32:24.752 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.752 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:24.752 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "4", 2024-09-11T00:32:24.752 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.752 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.752 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.752 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Number of asynchronous cloner threads", 2024-09-11T00:32:24.752 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.752 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.752 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.752 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.752 INFO:teuthology.orchestra.run.smithi012.stdout: "periodic_async_work": { 2024-09-11T00:32:24.753 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "periodic_async_work", 2024-09-11T00:32:24.753 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-09-11T00:32:24.753 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.753 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:24.753 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-09-11T00:32:24.753 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.753 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.753 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.753 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Periodically check for async work", 2024-09-11T00:32:24.753 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.753 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.753 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.753 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.753 INFO:teuthology.orchestra.run.smithi012.stdout: "snapshot_clone_delay": { 2024-09-11T00:32:24.754 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "snapshot_clone_delay", 2024-09-11T00:32:24.754 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-09-11T00:32:24.754 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.754 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:24.754 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "0", 2024-09-11T00:32:24.754 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.754 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.754 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.754 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-09-11T00:32:24.754 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.754 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.754 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.754 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.755 INFO:teuthology.orchestra.run.smithi012.stdout: "snapshot_clone_no_wait": { 2024-09-11T00:32:24.755 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "snapshot_clone_no_wait", 2024-09-11T00:32:24.755 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-09-11T00:32:24.755 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.755 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:24.755 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-09-11T00:32:24.755 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.755 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.755 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.755 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-09-11T00:32:24.755 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.755 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.755 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.755 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.756 INFO:teuthology.orchestra.run.smithi012.stdout: "sqlite3_killpoint": { 2024-09-11T00:32:24.756 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "sqlite3_killpoint", 2024-09-11T00:32:24.756 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-09-11T00:32:24.756 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "dev", 2024-09-11T00:32:24.756 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:24.756 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "0", 2024-09-11T00:32:24.756 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.756 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.756 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.756 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.756 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.756 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.756 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.757 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:32:24.757 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:32:24.757 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.757 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-11T00:32:24.757 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "zabbix", 2024-09-11T00:32:24.757 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-09-11T00:32:24.757 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-09-11T00:32:24.757 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-09-11T00:32:24.757 INFO:teuthology.orchestra.run.smithi012.stdout: "discovery_interval": { 2024-09-11T00:32:24.757 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "discovery_interval", 2024-09-11T00:32:24.757 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "uint", 2024-09-11T00:32:24.757 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.757 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:24.758 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "100", 2024-09-11T00:32:24.758 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.758 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.758 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.758 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.758 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.758 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.758 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.758 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.758 INFO:teuthology.orchestra.run.smithi012.stdout: "identifier": { 2024-09-11T00:32:24.758 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "identifier", 2024-09-11T00:32:24.758 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:24.758 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.758 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:24.759 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-09-11T00:32:24.759 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.759 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.759 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.759 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.759 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.759 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.759 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.759 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.759 INFO:teuthology.orchestra.run.smithi012.stdout: "interval": { 2024-09-11T00:32:24.759 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "interval", 2024-09-11T00:32:24.759 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "secs", 2024-09-11T00:32:24.759 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.759 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:24.760 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "60", 2024-09-11T00:32:24.760 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.760 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.760 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.760 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.760 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.760 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.760 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.760 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.760 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-09-11T00:32:24.760 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-09-11T00:32:24.760 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:24.761 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.761 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:24.761 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-09-11T00:32:24.761 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.761 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.761 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-09-11T00:32:24.761 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-09-11T00:32:24.761 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-09-11T00:32:24.761 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-09-11T00:32:24.761 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-09-11T00:32:24.761 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-09-11T00:32:24.761 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-09-11T00:32:24.761 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-09-11T00:32:24.762 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.762 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.762 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.762 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.762 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.762 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-09-11T00:32:24.762 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-09-11T00:32:24.762 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-09-11T00:32:24.762 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.762 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:24.762 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-09-11T00:32:24.762 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.762 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.762 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.763 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.763 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.763 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.763 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.763 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.763 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-09-11T00:32:24.763 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-09-11T00:32:24.763 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:24.763 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.763 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:24.764 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-09-11T00:32:24.764 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.764 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.764 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-09-11T00:32:24.764 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-09-11T00:32:24.764 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-09-11T00:32:24.764 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-09-11T00:32:24.764 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-09-11T00:32:24.764 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-09-11T00:32:24.764 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-09-11T00:32:24.765 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-09-11T00:32:24.765 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.765 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.765 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.765 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.765 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.765 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-09-11T00:32:24.765 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-09-11T00:32:24.765 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-09-11T00:32:24.765 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.765 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:24.765 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-09-11T00:32:24.765 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.766 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.766 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.766 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.766 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.766 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.766 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.766 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.766 INFO:teuthology.orchestra.run.smithi012.stdout: "sqlite3_killpoint": { 2024-09-11T00:32:24.766 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "sqlite3_killpoint", 2024-09-11T00:32:24.767 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-09-11T00:32:24.767 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "dev", 2024-09-11T00:32:24.767 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-11T00:32:24.767 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "0", 2024-09-11T00:32:24.767 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.767 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.767 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.767 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.767 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.767 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.767 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.767 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.767 INFO:teuthology.orchestra.run.smithi012.stdout: "zabbix_host": { 2024-09-11T00:32:24.768 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "zabbix_host", 2024-09-11T00:32:24.768 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:24.768 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.768 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:24.768 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-09-11T00:32:24.768 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.768 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.768 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.768 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.768 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.768 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.768 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.768 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.769 INFO:teuthology.orchestra.run.smithi012.stdout: "zabbix_port": { 2024-09-11T00:32:24.769 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "zabbix_port", 2024-09-11T00:32:24.769 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-09-11T00:32:24.769 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.769 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:24.769 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "10051", 2024-09-11T00:32:24.769 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.769 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.769 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.769 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.769 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.770 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.770 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.770 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.770 INFO:teuthology.orchestra.run.smithi012.stdout: "zabbix_sender": { 2024-09-11T00:32:24.770 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "zabbix_sender", 2024-09-11T00:32:24.770 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-09-11T00:32:24.770 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-09-11T00:32:24.770 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-09-11T00:32:24.770 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-09-11T00:32:24.771 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-09-11T00:32:24.771 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-09-11T00:32:24.771 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-09-11T00:32:24.771 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-09-11T00:32:24.771 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-09-11T00:32:24.771 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-09-11T00:32:24.771 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-09-11T00:32:24.771 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:32:24.771 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:32:24.772 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:32:24.772 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-09-11T00:32:24.772 INFO:teuthology.orchestra.run.smithi012.stdout: "services": { 2024-09-11T00:32:24.772 INFO:teuthology.orchestra.run.smithi012.stdout: "dashboard": "https://172.21.15.12:8443/" 2024-09-11T00:32:24.772 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.772 INFO:teuthology.orchestra.run.smithi012.stdout: "always_on_modules": { 2024-09-11T00:32:24.772 INFO:teuthology.orchestra.run.smithi012.stdout: "octopus": [ 2024-09-11T00:32:24.772 INFO:teuthology.orchestra.run.smithi012.stdout: "balancer", 2024-09-11T00:32:24.772 INFO:teuthology.orchestra.run.smithi012.stdout: "crash", 2024-09-11T00:32:24.772 INFO:teuthology.orchestra.run.smithi012.stdout: "devicehealth", 2024-09-11T00:32:24.772 INFO:teuthology.orchestra.run.smithi012.stdout: "orchestrator", 2024-09-11T00:32:24.772 INFO:teuthology.orchestra.run.smithi012.stdout: "pg_autoscaler", 2024-09-11T00:32:24.773 INFO:teuthology.orchestra.run.smithi012.stdout: "progress", 2024-09-11T00:32:24.773 INFO:teuthology.orchestra.run.smithi012.stdout: "rbd_support", 2024-09-11T00:32:24.773 INFO:teuthology.orchestra.run.smithi012.stdout: "status", 2024-09-11T00:32:24.773 INFO:teuthology.orchestra.run.smithi012.stdout: "telemetry", 2024-09-11T00:32:24.773 INFO:teuthology.orchestra.run.smithi012.stdout: "volumes" 2024-09-11T00:32:24.773 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-09-11T00:32:24.773 INFO:teuthology.orchestra.run.smithi012.stdout: "pacific": [ 2024-09-11T00:32:24.773 INFO:teuthology.orchestra.run.smithi012.stdout: "balancer", 2024-09-11T00:32:24.773 INFO:teuthology.orchestra.run.smithi012.stdout: "crash", 2024-09-11T00:32:24.773 INFO:teuthology.orchestra.run.smithi012.stdout: "devicehealth", 2024-09-11T00:32:24.773 INFO:teuthology.orchestra.run.smithi012.stdout: "orchestrator", 2024-09-11T00:32:24.773 INFO:teuthology.orchestra.run.smithi012.stdout: "pg_autoscaler", 2024-09-11T00:32:24.774 INFO:teuthology.orchestra.run.smithi012.stdout: "progress", 2024-09-11T00:32:24.774 INFO:teuthology.orchestra.run.smithi012.stdout: "rbd_support", 2024-09-11T00:32:24.774 INFO:teuthology.orchestra.run.smithi012.stdout: "status", 2024-09-11T00:32:24.774 INFO:teuthology.orchestra.run.smithi012.stdout: "telemetry", 2024-09-11T00:32:24.774 INFO:teuthology.orchestra.run.smithi012.stdout: "volumes" 2024-09-11T00:32:24.774 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-09-11T00:32:24.774 INFO:teuthology.orchestra.run.smithi012.stdout: "quincy": [ 2024-09-11T00:32:24.774 INFO:teuthology.orchestra.run.smithi012.stdout: "balancer", 2024-09-11T00:32:24.774 INFO:teuthology.orchestra.run.smithi012.stdout: "crash", 2024-09-11T00:32:24.774 INFO:teuthology.orchestra.run.smithi012.stdout: "devicehealth", 2024-09-11T00:32:24.774 INFO:teuthology.orchestra.run.smithi012.stdout: "orchestrator", 2024-09-11T00:32:24.774 INFO:teuthology.orchestra.run.smithi012.stdout: "pg_autoscaler", 2024-09-11T00:32:24.774 INFO:teuthology.orchestra.run.smithi012.stdout: "progress", 2024-09-11T00:32:24.774 INFO:teuthology.orchestra.run.smithi012.stdout: "rbd_support", 2024-09-11T00:32:24.775 INFO:teuthology.orchestra.run.smithi012.stdout: "status", 2024-09-11T00:32:24.775 INFO:teuthology.orchestra.run.smithi012.stdout: "telemetry", 2024-09-11T00:32:24.775 INFO:teuthology.orchestra.run.smithi012.stdout: "volumes" 2024-09-11T00:32:24.775 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-09-11T00:32:24.775 INFO:teuthology.orchestra.run.smithi012.stdout: "reef": [ 2024-09-11T00:32:24.775 INFO:teuthology.orchestra.run.smithi012.stdout: "balancer", 2024-09-11T00:32:24.775 INFO:teuthology.orchestra.run.smithi012.stdout: "crash", 2024-09-11T00:32:24.775 INFO:teuthology.orchestra.run.smithi012.stdout: "devicehealth", 2024-09-11T00:32:24.775 INFO:teuthology.orchestra.run.smithi012.stdout: "orchestrator", 2024-09-11T00:32:24.775 INFO:teuthology.orchestra.run.smithi012.stdout: "pg_autoscaler", 2024-09-11T00:32:24.776 INFO:teuthology.orchestra.run.smithi012.stdout: "progress", 2024-09-11T00:32:24.776 INFO:teuthology.orchestra.run.smithi012.stdout: "rbd_support", 2024-09-11T00:32:24.776 INFO:teuthology.orchestra.run.smithi012.stdout: "status", 2024-09-11T00:32:24.776 INFO:teuthology.orchestra.run.smithi012.stdout: "telemetry", 2024-09-11T00:32:24.776 INFO:teuthology.orchestra.run.smithi012.stdout: "volumes" 2024-09-11T00:32:24.776 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-09-11T00:32:24.776 INFO:teuthology.orchestra.run.smithi012.stdout: "squid": [ 2024-09-11T00:32:24.776 INFO:teuthology.orchestra.run.smithi012.stdout: "balancer", 2024-09-11T00:32:24.776 INFO:teuthology.orchestra.run.smithi012.stdout: "crash", 2024-09-11T00:32:24.776 INFO:teuthology.orchestra.run.smithi012.stdout: "devicehealth", 2024-09-11T00:32:24.776 INFO:teuthology.orchestra.run.smithi012.stdout: "orchestrator", 2024-09-11T00:32:24.776 INFO:teuthology.orchestra.run.smithi012.stdout: "pg_autoscaler", 2024-09-11T00:32:24.776 INFO:teuthology.orchestra.run.smithi012.stdout: "progress", 2024-09-11T00:32:24.777 INFO:teuthology.orchestra.run.smithi012.stdout: "rbd_support", 2024-09-11T00:32:24.777 INFO:teuthology.orchestra.run.smithi012.stdout: "status", 2024-09-11T00:32:24.777 INFO:teuthology.orchestra.run.smithi012.stdout: "telemetry", 2024-09-11T00:32:24.777 INFO:teuthology.orchestra.run.smithi012.stdout: "volumes" 2024-09-11T00:32:24.777 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-11T00:32:24.777 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.777 INFO:teuthology.orchestra.run.smithi012.stdout: "force_disabled_modules": {}, 2024-09-11T00:32:24.777 INFO:teuthology.orchestra.run.smithi012.stdout: "last_failure_osd_epoch": 3, 2024-09-11T00:32:24.777 INFO:teuthology.orchestra.run.smithi012.stdout: "active_clients": [ 2024-09-11T00:32:24.777 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-11T00:32:24.777 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "devicehealth", 2024-09-11T00:32:24.777 INFO:teuthology.orchestra.run.smithi012.stdout: "addrvec": [ 2024-09-11T00:32:24.777 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-11T00:32:24.778 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v2", 2024-09-11T00:32:24.778 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.12:0", 2024-09-11T00:32:24.778 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 1700982273 2024-09-11T00:32:24.778 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:32:24.778 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-11T00:32:24.778 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.778 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-11T00:32:24.778 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "libcephsqlite", 2024-09-11T00:32:24.778 INFO:teuthology.orchestra.run.smithi012.stdout: "addrvec": [ 2024-09-11T00:32:24.778 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-11T00:32:24.778 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v2", 2024-09-11T00:32:24.778 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.12:0", 2024-09-11T00:32:24.778 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 3715726086 2024-09-11T00:32:24.778 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:32:24.779 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-11T00:32:24.779 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.779 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-11T00:32:24.779 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "rbd_support", 2024-09-11T00:32:24.779 INFO:teuthology.orchestra.run.smithi012.stdout: "addrvec": [ 2024-09-11T00:32:24.779 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-11T00:32:24.779 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v2", 2024-09-11T00:32:24.779 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.12:0", 2024-09-11T00:32:24.779 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 4005282650 2024-09-11T00:32:24.779 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:32:24.779 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-11T00:32:24.779 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:24.779 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-11T00:32:24.779 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "volumes", 2024-09-11T00:32:24.780 INFO:teuthology.orchestra.run.smithi012.stdout: "addrvec": [ 2024-09-11T00:32:24.780 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-11T00:32:24.780 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v2", 2024-09-11T00:32:24.780 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.12:0", 2024-09-11T00:32:24.780 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 1844819639 2024-09-11T00:32:24.780 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:32:24.780 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-11T00:32:24.780 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:32:24.780 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-11T00:32:24.780 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-09-11T00:32:24.782 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:23.174+0000 7efc52a00640 1 -- 172.21.15.12:0/3734959530 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7efc300b52e0 msgr2=0x7efc300b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:32:24.782 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:23.174+0000 7efc52a00640 1 --2- 172.21.15.12:0/3734959530 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7efc300b52e0 0x7efc300b77a0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7efc3c005e30 tx=0x7efc3c005dc0 comp rx=0 tx=0).stop 2024-09-11T00:32:24.782 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:23.174+0000 7efc52a00640 1 -- 172.21.15.12:0/3734959530 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efc4c1066b0 msgr2=0x7efc4c19c1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:32:24.782 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:23.174+0000 7efc52a00640 1 --2- 172.21.15.12:0/3734959530 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efc4c1066b0 0x7efc4c19c1f0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7efc4400e9b0 tx=0x7efc4400ee70 comp rx=0 tx=0).stop 2024-09-11T00:32:24.782 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:23.174+0000 7efc52a00640 1 -- 172.21.15.12:0/3734959530 shutdown_connections 2024-09-11T00:32:24.782 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:23.174+0000 7efc52a00640 1 --2- 172.21.15.12:0/3734959530 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7efc300b52e0 0x7efc300b77a0 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:32:24.782 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:23.174+0000 7efc52a00640 1 --2- 172.21.15.12:0/3734959530 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efc4c1066b0 0x7efc4c19c1f0 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:32:24.782 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:23.174+0000 7efc52a00640 1 --2- 172.21.15.12:0/3734959530 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7efc4c105cf0 0x7efc4c19b5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:32:24.782 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:23.174+0000 7efc52a00640 1 --2- 172.21.15.12:0/3734959530 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efc4c104ba0 0x7efc4c19af70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:32:24.782 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:23.174+0000 7efc52a00640 1 -- 172.21.15.12:0/3734959530 >> 172.21.15.12:0/3734959530 conn(0x7efc4c100440 msgr2=0x7efc4c1090e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:32:24.783 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:23.174+0000 7efc52a00640 1 -- 172.21.15.12:0/3734959530 shutdown_connections 2024-09-11T00:32:24.783 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:23.174+0000 7efc52a00640 1 -- 172.21.15.12:0/3734959530 wait complete. 2024-09-11T00:32:24.791 DEBUG:tasks.check_counter:Getting stats from a 2024-09-11T00:32:24.791 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid ff3ad74a-6fcf-11ef-bcea-c7b262605968 -- ceph daemon mds.a perf dump 2024-09-11T00:32:24.791 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:32:23 smithi012 ceph-mon[30333]: pgmap v1159: 129 pgs: 129 active+clean; 671 MiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 329 KiB/s rd, 346 KiB/s wr, 181 op/s 2024-09-11T00:32:24.791 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:32:23 smithi012 ceph-mon[30333]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T00:32:24.792 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:32:23 smithi012 ceph-mon[30333]: from='client.? 172.21.15.12:0/3734959530' entity='client.admin' cmd={"prefix": "mgr dump", "format": "json-pretty"} : dispatch 2024-09-11T00:32:24.793 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:32:22 smithi178 ceph-mon[35919]: from='client.? 172.21.15.12:0/3943533436' entity='client.admin' cmd={"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} : dispatch 2024-09-11T00:32:24.793 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:32:22 smithi178 ceph-mon[35919]: from='client.? 172.21.15.12:0/4219392550' entity='client.admin' cmd={"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} : dispatch 2024-09-11T00:32:24.793 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:32:23 smithi178 ceph-mon[35919]: pgmap v1159: 129 pgs: 129 active+clean; 671 MiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 329 KiB/s rd, 346 KiB/s wr, 181 op/s 2024-09-11T00:32:24.793 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:32:23 smithi178 ceph-mon[35919]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T00:32:24.793 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:32:23 smithi178 ceph-mon[35919]: from='client.? 172.21.15.12:0/3734959530' entity='client.admin' cmd={"prefix": "mgr dump", "format": "json-pretty"} : dispatch 2024-09-11T00:32:24.793 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:32:22 smithi165 ceph-mon[35356]: from='client.? 172.21.15.12:0/3943533436' entity='client.admin' cmd={"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} : dispatch 2024-09-11T00:32:24.793 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:32:22 smithi165 ceph-mon[35356]: from='client.? 172.21.15.12:0/4219392550' entity='client.admin' cmd={"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} : dispatch 2024-09-11T00:32:24.793 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:32:23 smithi165 ceph-mon[35356]: pgmap v1159: 129 pgs: 129 active+clean; 671 MiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 329 KiB/s rd, 346 KiB/s wr, 181 op/s 2024-09-11T00:32:24.793 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:32:23 smithi165 ceph-mon[35356]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T00:32:24.793 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:32:23 smithi165 ceph-mon[35356]: from='client.? 172.21.15.12:0/3734959530' entity='client.admin' cmd={"prefix": "mgr dump", "format": "json-pretty"} : dispatch 2024-09-11T00:32:25.186 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:32:26.261 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:32:25 smithi012 ceph-mon[30333]: pgmap v1160: 129 pgs: 129 active+clean; 671 MiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 320 KiB/s rd, 336 KiB/s wr, 176 op/s 2024-09-11T00:32:26.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:32:25 smithi165 ceph-mon[35356]: pgmap v1160: 129 pgs: 129 active+clean; 671 MiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 320 KiB/s rd, 336 KiB/s wr, 176 op/s 2024-09-11T00:32:26.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:32:25 smithi178 ceph-mon[35919]: pgmap v1160: 129 pgs: 129 active+clean; 671 MiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 320 KiB/s rd, 336 KiB/s wr, 176 op/s 2024-09-11T00:32:26.528 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-09-11T00:32:26.528 INFO:teuthology.orchestra.run.smithi012.stdout: "AsyncMessenger::Worker-0": { 2024-09-11T00:32:26.528 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_recv_messages": 2377, 2024-09-11T00:32:26.528 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_send_messages": 2377, 2024-09-11T00:32:26.528 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_recv_bytes": 55189503, 2024-09-11T00:32:26.528 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_send_bytes": 755063, 2024-09-11T00:32:26.528 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_created_connections": 8, 2024-09-11T00:32:26.528 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_active_connections": 4, 2024-09-11T00:32:26.528 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_running_total_time": 0.502588962, 2024-09-11T00:32:26.529 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_running_send_time": 0.183248490, 2024-09-11T00:32:26.532 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_running_recv_time": 0.208266475, 2024-09-11T00:32:26.532 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_running_fast_dispatch_time": 0.081976258, 2024-09-11T00:32:26.532 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_send_messages_queue_lat": { 2024-09-11T00:32:26.532 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 2376, 2024-09-11T00:32:26.532 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.207979819, 2024-09-11T00:32:26.532 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000087533 2024-09-11T00:32:26.532 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:26.533 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_handle_ack_lat": { 2024-09-11T00:32:26.533 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 0, 2024-09-11T00:32:26.533 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.000000000, 2024-09-11T00:32:26.533 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000000000 2024-09-11T00:32:26.533 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:26.533 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_recv_encrypted_bytes": 2944, 2024-09-11T00:32:26.533 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_send_encrypted_bytes": 448 2024-09-11T00:32:26.533 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:26.533 INFO:teuthology.orchestra.run.smithi012.stdout: "AsyncMessenger::Worker-1": { 2024-09-11T00:32:26.533 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_recv_messages": 2493, 2024-09-11T00:32:26.533 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_send_messages": 2839, 2024-09-11T00:32:26.533 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_recv_bytes": 50964156, 2024-09-11T00:32:26.533 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_send_bytes": 1488913, 2024-09-11T00:32:26.534 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_created_connections": 7, 2024-09-11T00:32:26.534 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_active_connections": 6, 2024-09-11T00:32:26.534 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_running_total_time": 0.403131273, 2024-09-11T00:32:26.534 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_running_send_time": 0.140991411, 2024-09-11T00:32:26.534 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_running_recv_time": 0.172412489, 2024-09-11T00:32:26.534 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_running_fast_dispatch_time": 0.067920036, 2024-09-11T00:32:26.534 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_send_messages_queue_lat": { 2024-09-11T00:32:26.534 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 2837, 2024-09-11T00:32:26.534 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.340815907, 2024-09-11T00:32:26.534 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000120132 2024-09-11T00:32:26.534 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:26.534 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_handle_ack_lat": { 2024-09-11T00:32:26.534 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 0, 2024-09-11T00:32:26.535 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.000000000, 2024-09-11T00:32:26.535 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000000000 2024-09-11T00:32:26.535 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:26.535 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_recv_encrypted_bytes": 160, 2024-09-11T00:32:26.535 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_send_encrypted_bytes": 696400 2024-09-11T00:32:26.535 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:26.535 INFO:teuthology.orchestra.run.smithi012.stdout: "AsyncMessenger::Worker-2": { 2024-09-11T00:32:26.535 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_recv_messages": 1176, 2024-09-11T00:32:26.535 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_send_messages": 1089, 2024-09-11T00:32:26.535 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_recv_bytes": 51670572, 2024-09-11T00:32:26.535 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_send_bytes": 481885, 2024-09-11T00:32:26.535 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_created_connections": 6, 2024-09-11T00:32:26.535 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_active_connections": 4, 2024-09-11T00:32:26.536 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_running_total_time": 0.296312917, 2024-09-11T00:32:26.536 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_running_send_time": 0.088428520, 2024-09-11T00:32:26.536 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_running_recv_time": 0.180883636, 2024-09-11T00:32:26.536 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_running_fast_dispatch_time": 0.028488482, 2024-09-11T00:32:26.536 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_send_messages_queue_lat": { 2024-09-11T00:32:26.536 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 1088, 2024-09-11T00:32:26.536 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.229861973, 2024-09-11T00:32:26.536 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000211270 2024-09-11T00:32:26.536 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:26.536 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_handle_ack_lat": { 2024-09-11T00:32:26.536 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 0, 2024-09-11T00:32:26.536 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.000000000, 2024-09-11T00:32:26.537 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000000000 2024-09-11T00:32:26.537 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:26.537 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_recv_encrypted_bytes": 1179696, 2024-09-11T00:32:26.537 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_send_encrypted_bytes": 283760 2024-09-11T00:32:26.537 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:26.537 INFO:teuthology.orchestra.run.smithi012.stdout: "cct": { 2024-09-11T00:32:26.537 INFO:teuthology.orchestra.run.smithi012.stdout: "total_workers": 1, 2024-09-11T00:32:26.537 INFO:teuthology.orchestra.run.smithi012.stdout: "unhealthy_workers": 0 2024-09-11T00:32:26.538 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:26.538 INFO:teuthology.orchestra.run.smithi012.stdout: "finisher-MDSRank": { 2024-09-11T00:32:26.538 INFO:teuthology.orchestra.run.smithi012.stdout: "queue_len": 0, 2024-09-11T00:32:26.538 INFO:teuthology.orchestra.run.smithi012.stdout: "complete_latency": { 2024-09-11T00:32:26.538 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 9238, 2024-09-11T00:32:26.538 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.259976197, 2024-09-11T00:32:26.538 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000028142 2024-09-11T00:32:26.538 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:32:26.538 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:26.538 INFO:teuthology.orchestra.run.smithi012.stdout: "finisher-PurgeQueue": { 2024-09-11T00:32:26.539 INFO:teuthology.orchestra.run.smithi012.stdout: "queue_len": 0, 2024-09-11T00:32:26.539 INFO:teuthology.orchestra.run.smithi012.stdout: "complete_latency": { 2024-09-11T00:32:26.539 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 0, 2024-09-11T00:32:26.539 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.000000000, 2024-09-11T00:32:26.540 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000000000 2024-09-11T00:32:26.540 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:32:26.540 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:26.540 INFO:teuthology.orchestra.run.smithi012.stdout: "mds": { 2024-09-11T00:32:26.540 INFO:teuthology.orchestra.run.smithi012.stdout: "request": 0, 2024-09-11T00:32:26.540 INFO:teuthology.orchestra.run.smithi012.stdout: "reply": 0, 2024-09-11T00:32:26.540 INFO:teuthology.orchestra.run.smithi012.stdout: "reply_latency": { 2024-09-11T00:32:26.540 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 0, 2024-09-11T00:32:26.540 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.000000000, 2024-09-11T00:32:26.540 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000000000 2024-09-11T00:32:26.541 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:26.541 INFO:teuthology.orchestra.run.smithi012.stdout: "slow_reply": 0, 2024-09-11T00:32:26.541 INFO:teuthology.orchestra.run.smithi012.stdout: "forward": 0, 2024-09-11T00:32:26.541 INFO:teuthology.orchestra.run.smithi012.stdout: "dir_fetch_complete": 0, 2024-09-11T00:32:26.541 INFO:teuthology.orchestra.run.smithi012.stdout: "dir_fetch_keys": 0, 2024-09-11T00:32:26.541 INFO:teuthology.orchestra.run.smithi012.stdout: "dir_commit": 0, 2024-09-11T00:32:26.541 INFO:teuthology.orchestra.run.smithi012.stdout: "dir_split": 0, 2024-09-11T00:32:26.541 INFO:teuthology.orchestra.run.smithi012.stdout: "dir_merge": 0, 2024-09-11T00:32:26.541 INFO:teuthology.orchestra.run.smithi012.stdout: "inodes": 15570, 2024-09-11T00:32:26.541 INFO:teuthology.orchestra.run.smithi012.stdout: "inodes_top": 3, 2024-09-11T00:32:26.541 INFO:teuthology.orchestra.run.smithi012.stdout: "inodes_bottom": 15567, 2024-09-11T00:32:26.541 INFO:teuthology.orchestra.run.smithi012.stdout: "inodes_pin_tail": 0, 2024-09-11T00:32:26.541 INFO:teuthology.orchestra.run.smithi012.stdout: "inodes_pinned": 15565, 2024-09-11T00:32:26.541 INFO:teuthology.orchestra.run.smithi012.stdout: "inodes_expired": 1, 2024-09-11T00:32:26.542 INFO:teuthology.orchestra.run.smithi012.stdout: "inodes_with_caps": 0, 2024-09-11T00:32:26.542 INFO:teuthology.orchestra.run.smithi012.stdout: "caps": 0, 2024-09-11T00:32:26.542 INFO:teuthology.orchestra.run.smithi012.stdout: "subtrees": 1, 2024-09-11T00:32:26.542 INFO:teuthology.orchestra.run.smithi012.stdout: "traverse": 0, 2024-09-11T00:32:26.542 INFO:teuthology.orchestra.run.smithi012.stdout: "traverse_hit": 0, 2024-09-11T00:32:26.542 INFO:teuthology.orchestra.run.smithi012.stdout: "traverse_forward": 0, 2024-09-11T00:32:26.542 INFO:teuthology.orchestra.run.smithi012.stdout: "traverse_discover": 0, 2024-09-11T00:32:26.542 INFO:teuthology.orchestra.run.smithi012.stdout: "traverse_dir_fetch": 0, 2024-09-11T00:32:26.542 INFO:teuthology.orchestra.run.smithi012.stdout: "traverse_remote_ino": 0, 2024-09-11T00:32:26.542 INFO:teuthology.orchestra.run.smithi012.stdout: "traverse_lock": 0, 2024-09-11T00:32:26.542 INFO:teuthology.orchestra.run.smithi012.stdout: "load_cent": 0, 2024-09-11T00:32:26.542 INFO:teuthology.orchestra.run.smithi012.stdout: "q": 0, 2024-09-11T00:32:26.542 INFO:teuthology.orchestra.run.smithi012.stdout: "exported": 0, 2024-09-11T00:32:26.543 INFO:teuthology.orchestra.run.smithi012.stdout: "exported_inodes": 0, 2024-09-11T00:32:26.543 INFO:teuthology.orchestra.run.smithi012.stdout: "imported": 0, 2024-09-11T00:32:26.543 INFO:teuthology.orchestra.run.smithi012.stdout: "imported_inodes": 0, 2024-09-11T00:32:26.543 INFO:teuthology.orchestra.run.smithi012.stdout: "openino_dir_fetch": 0, 2024-09-11T00:32:26.543 INFO:teuthology.orchestra.run.smithi012.stdout: "openino_backtrace_fetch": 0, 2024-09-11T00:32:26.543 INFO:teuthology.orchestra.run.smithi012.stdout: "openino_peer_discover": 0, 2024-09-11T00:32:26.543 INFO:teuthology.orchestra.run.smithi012.stdout: "root_rfiles": 0, 2024-09-11T00:32:26.543 INFO:teuthology.orchestra.run.smithi012.stdout: "root_rbytes": 0, 2024-09-11T00:32:26.543 INFO:teuthology.orchestra.run.smithi012.stdout: "root_rsnaps": 0, 2024-09-11T00:32:26.543 INFO:teuthology.orchestra.run.smithi012.stdout: "scrub_backtrace_fetch": 0, 2024-09-11T00:32:26.543 INFO:teuthology.orchestra.run.smithi012.stdout: "scrub_set_tag": 0, 2024-09-11T00:32:26.543 INFO:teuthology.orchestra.run.smithi012.stdout: "scrub_backtrace_repaired": 0, 2024-09-11T00:32:26.543 INFO:teuthology.orchestra.run.smithi012.stdout: "scrub_inotable_repaired": 0, 2024-09-11T00:32:26.544 INFO:teuthology.orchestra.run.smithi012.stdout: "scrub_dir_inodes": 0, 2024-09-11T00:32:26.544 INFO:teuthology.orchestra.run.smithi012.stdout: "scrub_dir_base_inodes": 0, 2024-09-11T00:32:26.544 INFO:teuthology.orchestra.run.smithi012.stdout: "scrub_dirfrag_rstats": 0, 2024-09-11T00:32:26.544 INFO:teuthology.orchestra.run.smithi012.stdout: "scrub_file_inodes": 0, 2024-09-11T00:32:26.544 INFO:teuthology.orchestra.run.smithi012.stdout: "handle_inode_file_caps": 0, 2024-09-11T00:32:26.544 INFO:teuthology.orchestra.run.smithi012.stdout: "ceph_cap_op_revoke": 0, 2024-09-11T00:32:26.544 INFO:teuthology.orchestra.run.smithi012.stdout: "ceph_cap_op_grant": 0, 2024-09-11T00:32:26.544 INFO:teuthology.orchestra.run.smithi012.stdout: "ceph_cap_op_trunc": 0, 2024-09-11T00:32:26.544 INFO:teuthology.orchestra.run.smithi012.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-11T00:32:26.544 INFO:teuthology.orchestra.run.smithi012.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-11T00:32:26.544 INFO:teuthology.orchestra.run.smithi012.stdout: "handle_client_caps": 0, 2024-09-11T00:32:26.544 INFO:teuthology.orchestra.run.smithi012.stdout: "handle_client_caps_dirty": 0, 2024-09-11T00:32:26.545 INFO:teuthology.orchestra.run.smithi012.stdout: "handle_client_cap_release": 0, 2024-09-11T00:32:26.545 INFO:teuthology.orchestra.run.smithi012.stdout: "process_request_cap_release": 0 2024-09-11T00:32:26.545 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:26.545 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_cache": { 2024-09-11T00:32:26.545 INFO:teuthology.orchestra.run.smithi012.stdout: "dir_update": 0, 2024-09-11T00:32:26.545 INFO:teuthology.orchestra.run.smithi012.stdout: "dir_update_receipt": 0, 2024-09-11T00:32:26.545 INFO:teuthology.orchestra.run.smithi012.stdout: "dir_try_discover": 0, 2024-09-11T00:32:26.545 INFO:teuthology.orchestra.run.smithi012.stdout: "dir_send_discover": 0, 2024-09-11T00:32:26.545 INFO:teuthology.orchestra.run.smithi012.stdout: "dir_handle_discover": 0, 2024-09-11T00:32:26.545 INFO:teuthology.orchestra.run.smithi012.stdout: "num_strays": 0, 2024-09-11T00:32:26.545 INFO:teuthology.orchestra.run.smithi012.stdout: "num_strays_delayed": 0, 2024-09-11T00:32:26.545 INFO:teuthology.orchestra.run.smithi012.stdout: "num_strays_enqueuing": 0, 2024-09-11T00:32:26.545 INFO:teuthology.orchestra.run.smithi012.stdout: "strays_created": 0, 2024-09-11T00:32:26.546 INFO:teuthology.orchestra.run.smithi012.stdout: "strays_enqueued": 0, 2024-09-11T00:32:26.546 INFO:teuthology.orchestra.run.smithi012.stdout: "strays_reintegrated": 0, 2024-09-11T00:32:26.546 INFO:teuthology.orchestra.run.smithi012.stdout: "strays_migrated": 0, 2024-09-11T00:32:26.546 INFO:teuthology.orchestra.run.smithi012.stdout: "num_recovering_processing": 0, 2024-09-11T00:32:26.546 INFO:teuthology.orchestra.run.smithi012.stdout: "num_recovering_enqueued": 0, 2024-09-11T00:32:26.546 INFO:teuthology.orchestra.run.smithi012.stdout: "num_recovering_prioritized": 0, 2024-09-11T00:32:26.546 INFO:teuthology.orchestra.run.smithi012.stdout: "recovery_started": 0, 2024-09-11T00:32:26.546 INFO:teuthology.orchestra.run.smithi012.stdout: "recovery_completed": 0, 2024-09-11T00:32:26.546 INFO:teuthology.orchestra.run.smithi012.stdout: "ireq_quiesce_path": 0, 2024-09-11T00:32:26.546 INFO:teuthology.orchestra.run.smithi012.stdout: "ireq_quiesce_inode": 0, 2024-09-11T00:32:26.546 INFO:teuthology.orchestra.run.smithi012.stdout: "ireq_enqueue_scrub": 0, 2024-09-11T00:32:26.546 INFO:teuthology.orchestra.run.smithi012.stdout: "ireq_exportdir": 0, 2024-09-11T00:32:26.546 INFO:teuthology.orchestra.run.smithi012.stdout: "ireq_flush": 0, 2024-09-11T00:32:26.547 INFO:teuthology.orchestra.run.smithi012.stdout: "ireq_fragmentdir": 0, 2024-09-11T00:32:26.547 INFO:teuthology.orchestra.run.smithi012.stdout: "ireq_fragstats": 0, 2024-09-11T00:32:26.547 INFO:teuthology.orchestra.run.smithi012.stdout: "ireq_inodestats": 0, 2024-09-11T00:32:26.547 INFO:teuthology.orchestra.run.smithi012.stdout: "uninline_started": 0, 2024-09-11T00:32:26.547 INFO:teuthology.orchestra.run.smithi012.stdout: "uninline_succeeded": 0, 2024-09-11T00:32:26.547 INFO:teuthology.orchestra.run.smithi012.stdout: "uninline_write_failed": 0 2024-09-11T00:32:26.547 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:26.547 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_log": { 2024-09-11T00:32:26.547 INFO:teuthology.orchestra.run.smithi012.stdout: "evlrg": 0, 2024-09-11T00:32:26.547 INFO:teuthology.orchestra.run.smithi012.stdout: "evadd": 0, 2024-09-11T00:32:26.547 INFO:teuthology.orchestra.run.smithi012.stdout: "evex": 0, 2024-09-11T00:32:26.547 INFO:teuthology.orchestra.run.smithi012.stdout: "evtrm": 0, 2024-09-11T00:32:26.547 INFO:teuthology.orchestra.run.smithi012.stdout: "ev": 78981, 2024-09-11T00:32:26.547 INFO:teuthology.orchestra.run.smithi012.stdout: "evexg": 0, 2024-09-11T00:32:26.548 INFO:teuthology.orchestra.run.smithi012.stdout: "evexd": 0, 2024-09-11T00:32:26.548 INFO:teuthology.orchestra.run.smithi012.stdout: "segadd": 0, 2024-09-11T00:32:26.548 INFO:teuthology.orchestra.run.smithi012.stdout: "segex": 0, 2024-09-11T00:32:26.548 INFO:teuthology.orchestra.run.smithi012.stdout: "segtrm": 0, 2024-09-11T00:32:26.548 INFO:teuthology.orchestra.run.smithi012.stdout: "seg": 100, 2024-09-11T00:32:26.548 INFO:teuthology.orchestra.run.smithi012.stdout: "segmjr": 8, 2024-09-11T00:32:26.548 INFO:teuthology.orchestra.run.smithi012.stdout: "segexg": 0, 2024-09-11T00:32:26.548 INFO:teuthology.orchestra.run.smithi012.stdout: "segexd": 0, 2024-09-11T00:32:26.548 INFO:teuthology.orchestra.run.smithi012.stdout: "expos": 4194304, 2024-09-11T00:32:26.548 INFO:teuthology.orchestra.run.smithi012.stdout: "wrpos": 159306380, 2024-09-11T00:32:26.548 INFO:teuthology.orchestra.run.smithi012.stdout: "rdpos": 159305242, 2024-09-11T00:32:26.548 INFO:teuthology.orchestra.run.smithi012.stdout: "jlat": { 2024-09-11T00:32:26.549 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 0, 2024-09-11T00:32:26.549 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.000000000, 2024-09-11T00:32:26.549 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000000000 2024-09-11T00:32:26.549 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:26.549 INFO:teuthology.orchestra.run.smithi012.stdout: "replayed": 78981 2024-09-11T00:32:26.549 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:26.549 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_mem": { 2024-09-11T00:32:26.549 INFO:teuthology.orchestra.run.smithi012.stdout: "ino": 15, 2024-09-11T00:32:26.549 INFO:teuthology.orchestra.run.smithi012.stdout: "ino+": 15630, 2024-09-11T00:32:26.549 INFO:teuthology.orchestra.run.smithi012.stdout: "ino-": 15615, 2024-09-11T00:32:26.549 INFO:teuthology.orchestra.run.smithi012.stdout: "dir": 13, 2024-09-11T00:32:26.549 INFO:teuthology.orchestra.run.smithi012.stdout: "dir+": 66, 2024-09-11T00:32:26.549 INFO:teuthology.orchestra.run.smithi012.stdout: "dir-": 53, 2024-09-11T00:32:26.550 INFO:teuthology.orchestra.run.smithi012.stdout: "dn": 15570, 2024-09-11T00:32:26.550 INFO:teuthology.orchestra.run.smithi012.stdout: "dn+": 16029, 2024-09-11T00:32:26.550 INFO:teuthology.orchestra.run.smithi012.stdout: "dn-": 459, 2024-09-11T00:32:26.550 INFO:teuthology.orchestra.run.smithi012.stdout: "cap": 0, 2024-09-11T00:32:26.550 INFO:teuthology.orchestra.run.smithi012.stdout: "cap+": 0, 2024-09-11T00:32:26.550 INFO:teuthology.orchestra.run.smithi012.stdout: "cap-": 0, 2024-09-11T00:32:26.550 INFO:teuthology.orchestra.run.smithi012.stdout: "rss": 63732, 2024-09-11T00:32:26.550 INFO:teuthology.orchestra.run.smithi012.stdout: "heap": 269377536 2024-09-11T00:32:26.550 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:26.550 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_server": { 2024-09-11T00:32:26.550 INFO:teuthology.orchestra.run.smithi012.stdout: "dispatch_client_request": 0, 2024-09-11T00:32:26.550 INFO:teuthology.orchestra.run.smithi012.stdout: "dispatch_server_request": 0, 2024-09-11T00:32:26.551 INFO:teuthology.orchestra.run.smithi012.stdout: "handle_client_request": 0, 2024-09-11T00:32:26.551 INFO:teuthology.orchestra.run.smithi012.stdout: "handle_client_session": 0, 2024-09-11T00:32:26.551 INFO:teuthology.orchestra.run.smithi012.stdout: "handle_peer_request": 0, 2024-09-11T00:32:26.551 INFO:teuthology.orchestra.run.smithi012.stdout: "req_create_latency": { 2024-09-11T00:32:26.551 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 0, 2024-09-11T00:32:26.551 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.000000000, 2024-09-11T00:32:26.551 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000000000 2024-09-11T00:32:26.551 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:26.551 INFO:teuthology.orchestra.run.smithi012.stdout: "req_getattr_latency": { 2024-09-11T00:32:26.551 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 0, 2024-09-11T00:32:26.551 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.000000000, 2024-09-11T00:32:26.551 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000000000 2024-09-11T00:32:26.551 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:26.552 INFO:teuthology.orchestra.run.smithi012.stdout: "req_getfilelock_latency": { 2024-09-11T00:32:26.552 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 0, 2024-09-11T00:32:26.552 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.000000000, 2024-09-11T00:32:26.552 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000000000 2024-09-11T00:32:26.552 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:26.552 INFO:teuthology.orchestra.run.smithi012.stdout: "req_link_latency": { 2024-09-11T00:32:26.552 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 0, 2024-09-11T00:32:26.552 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.000000000, 2024-09-11T00:32:26.552 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000000000 2024-09-11T00:32:26.552 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:26.552 INFO:teuthology.orchestra.run.smithi012.stdout: "req_lookup_latency": { 2024-09-11T00:32:26.552 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 0, 2024-09-11T00:32:26.552 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.000000000, 2024-09-11T00:32:26.553 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000000000 2024-09-11T00:32:26.553 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:26.553 INFO:teuthology.orchestra.run.smithi012.stdout: "req_lookuphash_latency": { 2024-09-11T00:32:26.553 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 0, 2024-09-11T00:32:26.553 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.000000000, 2024-09-11T00:32:26.553 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000000000 2024-09-11T00:32:26.553 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:26.553 INFO:teuthology.orchestra.run.smithi012.stdout: "req_lookupino_latency": { 2024-09-11T00:32:26.553 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 0, 2024-09-11T00:32:26.553 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.000000000, 2024-09-11T00:32:26.553 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000000000 2024-09-11T00:32:26.553 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:26.553 INFO:teuthology.orchestra.run.smithi012.stdout: "req_lookupname_latency": { 2024-09-11T00:32:26.554 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 0, 2024-09-11T00:32:26.554 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.000000000, 2024-09-11T00:32:26.554 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000000000 2024-09-11T00:32:26.554 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:26.554 INFO:teuthology.orchestra.run.smithi012.stdout: "req_lookupparent_latency": { 2024-09-11T00:32:26.554 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 0, 2024-09-11T00:32:26.554 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.000000000, 2024-09-11T00:32:26.554 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000000000 2024-09-11T00:32:26.554 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:26.554 INFO:teuthology.orchestra.run.smithi012.stdout: "req_lookupsnap_latency": { 2024-09-11T00:32:26.554 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 0, 2024-09-11T00:32:26.554 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.000000000, 2024-09-11T00:32:26.554 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000000000 2024-09-11T00:32:26.554 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:26.555 INFO:teuthology.orchestra.run.smithi012.stdout: "req_lssnap_latency": { 2024-09-11T00:32:26.555 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 0, 2024-09-11T00:32:26.555 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.000000000, 2024-09-11T00:32:26.555 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000000000 2024-09-11T00:32:26.555 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:26.555 INFO:teuthology.orchestra.run.smithi012.stdout: "req_mkdir_latency": { 2024-09-11T00:32:26.556 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 0, 2024-09-11T00:32:26.556 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.000000000, 2024-09-11T00:32:26.556 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000000000 2024-09-11T00:32:26.556 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:26.556 INFO:teuthology.orchestra.run.smithi012.stdout: "req_mknod_latency": { 2024-09-11T00:32:26.556 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 0, 2024-09-11T00:32:26.556 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.000000000, 2024-09-11T00:32:26.556 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000000000 2024-09-11T00:32:26.556 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:26.556 INFO:teuthology.orchestra.run.smithi012.stdout: "req_mksnap_latency": { 2024-09-11T00:32:26.557 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 0, 2024-09-11T00:32:26.557 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.000000000, 2024-09-11T00:32:26.557 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000000000 2024-09-11T00:32:26.557 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:26.557 INFO:teuthology.orchestra.run.smithi012.stdout: "req_open_latency": { 2024-09-11T00:32:26.557 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 0, 2024-09-11T00:32:26.557 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.000000000, 2024-09-11T00:32:26.557 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000000000 2024-09-11T00:32:26.557 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:26.557 INFO:teuthology.orchestra.run.smithi012.stdout: "req_readdir_latency": { 2024-09-11T00:32:26.557 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 0, 2024-09-11T00:32:26.557 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.000000000, 2024-09-11T00:32:26.557 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000000000 2024-09-11T00:32:26.558 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:26.558 INFO:teuthology.orchestra.run.smithi012.stdout: "req_rename_latency": { 2024-09-11T00:32:26.558 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 0, 2024-09-11T00:32:26.558 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.000000000, 2024-09-11T00:32:26.558 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000000000 2024-09-11T00:32:26.558 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:26.558 INFO:teuthology.orchestra.run.smithi012.stdout: "req_renamesnap_latency": { 2024-09-11T00:32:26.558 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 0, 2024-09-11T00:32:26.558 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.000000000, 2024-09-11T00:32:26.558 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000000000 2024-09-11T00:32:26.558 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:26.558 INFO:teuthology.orchestra.run.smithi012.stdout: "req_snapdiff_latency": { 2024-09-11T00:32:26.559 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 0, 2024-09-11T00:32:26.559 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.000000000, 2024-09-11T00:32:26.559 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000000000 2024-09-11T00:32:26.559 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:26.559 INFO:teuthology.orchestra.run.smithi012.stdout: "req_rmdir_latency": { 2024-09-11T00:32:26.559 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 0, 2024-09-11T00:32:26.559 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.000000000, 2024-09-11T00:32:26.559 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000000000 2024-09-11T00:32:26.559 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:26.559 INFO:teuthology.orchestra.run.smithi012.stdout: "req_rmsnap_latency": { 2024-09-11T00:32:26.559 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 0, 2024-09-11T00:32:26.559 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.000000000, 2024-09-11T00:32:26.559 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000000000 2024-09-11T00:32:26.560 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:26.560 INFO:teuthology.orchestra.run.smithi012.stdout: "req_rmxattr_latency": { 2024-09-11T00:32:26.560 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 0, 2024-09-11T00:32:26.560 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.000000000, 2024-09-11T00:32:26.560 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000000000 2024-09-11T00:32:26.560 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:26.560 INFO:teuthology.orchestra.run.smithi012.stdout: "req_setattr_latency": { 2024-09-11T00:32:26.560 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 0, 2024-09-11T00:32:26.560 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.000000000, 2024-09-11T00:32:26.560 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000000000 2024-09-11T00:32:26.561 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:26.561 INFO:teuthology.orchestra.run.smithi012.stdout: "req_setdirlayout_latency": { 2024-09-11T00:32:26.561 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 0, 2024-09-11T00:32:26.561 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.000000000, 2024-09-11T00:32:26.561 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000000000 2024-09-11T00:32:26.561 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:26.561 INFO:teuthology.orchestra.run.smithi012.stdout: "req_setfilelock_latency": { 2024-09-11T00:32:26.561 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 0, 2024-09-11T00:32:26.561 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.000000000, 2024-09-11T00:32:26.561 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000000000 2024-09-11T00:32:26.561 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:26.561 INFO:teuthology.orchestra.run.smithi012.stdout: "req_setlayout_latency": { 2024-09-11T00:32:26.561 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 0, 2024-09-11T00:32:26.561 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.000000000, 2024-09-11T00:32:26.562 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000000000 2024-09-11T00:32:26.562 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:26.562 INFO:teuthology.orchestra.run.smithi012.stdout: "req_setxattr_latency": { 2024-09-11T00:32:26.562 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 0, 2024-09-11T00:32:26.562 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.000000000, 2024-09-11T00:32:26.562 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000000000 2024-09-11T00:32:26.562 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:26.562 INFO:teuthology.orchestra.run.smithi012.stdout: "req_symlink_latency": { 2024-09-11T00:32:26.562 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 0, 2024-09-11T00:32:26.562 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.000000000, 2024-09-11T00:32:26.562 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000000000 2024-09-11T00:32:26.562 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:26.562 INFO:teuthology.orchestra.run.smithi012.stdout: "req_unlink_latency": { 2024-09-11T00:32:26.563 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 0, 2024-09-11T00:32:26.563 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.000000000, 2024-09-11T00:32:26.563 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000000000 2024-09-11T00:32:26.563 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:26.563 INFO:teuthology.orchestra.run.smithi012.stdout: "cap_revoke_eviction": 0, 2024-09-11T00:32:26.563 INFO:teuthology.orchestra.run.smithi012.stdout: "cap_acquisition_throttle": 0, 2024-09-11T00:32:26.563 INFO:teuthology.orchestra.run.smithi012.stdout: "req_getvxattr_latency": { 2024-09-11T00:32:26.563 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 0, 2024-09-11T00:32:26.563 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.000000000, 2024-09-11T00:32:26.563 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000000000 2024-09-11T00:32:26.563 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:32:26.563 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:26.564 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_sessions": { 2024-09-11T00:32:26.564 INFO:teuthology.orchestra.run.smithi012.stdout: "session_count": 1, 2024-09-11T00:32:26.564 INFO:teuthology.orchestra.run.smithi012.stdout: "session_add": 2, 2024-09-11T00:32:26.564 INFO:teuthology.orchestra.run.smithi012.stdout: "session_remove": 1, 2024-09-11T00:32:26.564 INFO:teuthology.orchestra.run.smithi012.stdout: "sessions_open": 1, 2024-09-11T00:32:26.564 INFO:teuthology.orchestra.run.smithi012.stdout: "sessions_stale": 0, 2024-09-11T00:32:26.564 INFO:teuthology.orchestra.run.smithi012.stdout: "total_load": 0, 2024-09-11T00:32:26.564 INFO:teuthology.orchestra.run.smithi012.stdout: "average_load": 0, 2024-09-11T00:32:26.564 INFO:teuthology.orchestra.run.smithi012.stdout: "avg_session_uptime": 2718, 2024-09-11T00:32:26.564 INFO:teuthology.orchestra.run.smithi012.stdout: "mdthresh_evicted": 0 2024-09-11T00:32:26.564 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:26.564 INFO:teuthology.orchestra.run.smithi012.stdout: "mempool": { 2024-09-11T00:32:26.564 INFO:teuthology.orchestra.run.smithi012.stdout: "bloom_filter_bytes": 0, 2024-09-11T00:32:26.565 INFO:teuthology.orchestra.run.smithi012.stdout: "bloom_filter_items": 0, 2024-09-11T00:32:26.565 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_alloc_bytes": 0, 2024-09-11T00:32:26.565 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_alloc_items": 0, 2024-09-11T00:32:26.565 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_cache_data_bytes": 0, 2024-09-11T00:32:26.565 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_cache_data_items": 0, 2024-09-11T00:32:26.565 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-11T00:32:26.565 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_cache_onode_items": 0, 2024-09-11T00:32:26.565 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-11T00:32:26.565 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_cache_meta_items": 0, 2024-09-11T00:32:26.566 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_cache_other_bytes": 0, 2024-09-11T00:32:26.566 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_cache_other_items": 0, 2024-09-11T00:32:26.566 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-11T00:32:26.566 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_cache_buffer_items": 0, 2024-09-11T00:32:26.566 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_extent_bytes": 0, 2024-09-11T00:32:26.566 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_extent_items": 0, 2024-09-11T00:32:26.566 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_blob_bytes": 0, 2024-09-11T00:32:26.566 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_blob_items": 0, 2024-09-11T00:32:26.566 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-11T00:32:26.566 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_shared_blob_items": 0, 2024-09-11T00:32:26.566 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-11T00:32:26.566 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_inline_bl_items": 0, 2024-09-11T00:32:26.566 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_fsck_bytes": 0, 2024-09-11T00:32:26.566 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_fsck_items": 0, 2024-09-11T00:32:26.567 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_txc_bytes": 0, 2024-09-11T00:32:26.567 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_txc_items": 0, 2024-09-11T00:32:26.567 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-11T00:32:26.567 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_writing_deferred_items": 0, 2024-09-11T00:32:26.567 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_writing_bytes": 0, 2024-09-11T00:32:26.567 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_writing_items": 0, 2024-09-11T00:32:26.567 INFO:teuthology.orchestra.run.smithi012.stdout: "bluefs_bytes": 0, 2024-09-11T00:32:26.567 INFO:teuthology.orchestra.run.smithi012.stdout: "bluefs_items": 0, 2024-09-11T00:32:26.567 INFO:teuthology.orchestra.run.smithi012.stdout: "bluefs_file_reader_bytes": 0, 2024-09-11T00:32:26.567 INFO:teuthology.orchestra.run.smithi012.stdout: "bluefs_file_reader_items": 0, 2024-09-11T00:32:26.567 INFO:teuthology.orchestra.run.smithi012.stdout: "bluefs_file_writer_bytes": 0, 2024-09-11T00:32:26.567 INFO:teuthology.orchestra.run.smithi012.stdout: "bluefs_file_writer_items": 0, 2024-09-11T00:32:26.567 INFO:teuthology.orchestra.run.smithi012.stdout: "buffer_anon_bytes": 241320, 2024-09-11T00:32:26.568 INFO:teuthology.orchestra.run.smithi012.stdout: "buffer_anon_items": 83, 2024-09-11T00:32:26.568 INFO:teuthology.orchestra.run.smithi012.stdout: "buffer_meta_bytes": 0, 2024-09-11T00:32:26.568 INFO:teuthology.orchestra.run.smithi012.stdout: "buffer_meta_items": 0, 2024-09-11T00:32:26.568 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_bytes": 0, 2024-09-11T00:32:26.568 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_items": 0, 2024-09-11T00:32:26.568 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_mapbl_bytes": 0, 2024-09-11T00:32:26.568 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_mapbl_items": 0, 2024-09-11T00:32:26.568 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_pglog_bytes": 0, 2024-09-11T00:32:26.568 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_pglog_items": 0, 2024-09-11T00:32:26.568 INFO:teuthology.orchestra.run.smithi012.stdout: "osdmap_bytes": 9048, 2024-09-11T00:32:26.568 INFO:teuthology.orchestra.run.smithi012.stdout: "osdmap_items": 170, 2024-09-11T00:32:26.568 INFO:teuthology.orchestra.run.smithi012.stdout: "osdmap_mapping_bytes": 0, 2024-09-11T00:32:26.568 INFO:teuthology.orchestra.run.smithi012.stdout: "osdmap_mapping_items": 0, 2024-09-11T00:32:26.568 INFO:teuthology.orchestra.run.smithi012.stdout: "pgmap_bytes": 0, 2024-09-11T00:32:26.569 INFO:teuthology.orchestra.run.smithi012.stdout: "pgmap_items": 0, 2024-09-11T00:32:26.569 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_co_bytes": 10639186, 2024-09-11T00:32:26.569 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_co_items": 62627, 2024-09-11T00:32:26.569 INFO:teuthology.orchestra.run.smithi012.stdout: "unittest_1_bytes": 0, 2024-09-11T00:32:26.569 INFO:teuthology.orchestra.run.smithi012.stdout: "unittest_1_items": 0, 2024-09-11T00:32:26.569 INFO:teuthology.orchestra.run.smithi012.stdout: "unittest_2_bytes": 0, 2024-09-11T00:32:26.569 INFO:teuthology.orchestra.run.smithi012.stdout: "unittest_2_items": 0 2024-09-11T00:32:26.569 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:26.569 INFO:teuthology.orchestra.run.smithi012.stdout: "objecter": { 2024-09-11T00:32:26.569 INFO:teuthology.orchestra.run.smithi012.stdout: "op_active": 0, 2024-09-11T00:32:26.569 INFO:teuthology.orchestra.run.smithi012.stdout: "op_laggy": 0, 2024-09-11T00:32:26.569 INFO:teuthology.orchestra.run.smithi012.stdout: "op_send": 5488, 2024-09-11T00:32:26.569 INFO:teuthology.orchestra.run.smithi012.stdout: "op_send_bytes": 21, 2024-09-11T00:32:26.570 INFO:teuthology.orchestra.run.smithi012.stdout: "op_resend": 3, 2024-09-11T00:32:26.570 INFO:teuthology.orchestra.run.smithi012.stdout: "op_reply": 5488, 2024-09-11T00:32:26.570 INFO:teuthology.orchestra.run.smithi012.stdout: "op_latency": { 2024-09-11T00:32:26.570 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 5488, 2024-09-11T00:32:26.570 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 26.294785000, 2024-09-11T00:32:26.570 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.004791323 2024-09-11T00:32:26.570 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:26.570 INFO:teuthology.orchestra.run.smithi012.stdout: "op_inflight": 0, 2024-09-11T00:32:26.570 INFO:teuthology.orchestra.run.smithi012.stdout: "oplen_avg": { 2024-09-11T00:32:26.570 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 5488, 2024-09-11T00:32:26.570 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 5489 2024-09-11T00:32:26.570 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:26.570 INFO:teuthology.orchestra.run.smithi012.stdout: "op": 5488, 2024-09-11T00:32:26.570 INFO:teuthology.orchestra.run.smithi012.stdout: "op_r": 5488, 2024-09-11T00:32:26.571 INFO:teuthology.orchestra.run.smithi012.stdout: "op_w": 0, 2024-09-11T00:32:26.571 INFO:teuthology.orchestra.run.smithi012.stdout: "op_rmw": 0, 2024-09-11T00:32:26.571 INFO:teuthology.orchestra.run.smithi012.stdout: "op_pg": 0, 2024-09-11T00:32:26.571 INFO:teuthology.orchestra.run.smithi012.stdout: "osdop_stat": 3171, 2024-09-11T00:32:26.571 INFO:teuthology.orchestra.run.smithi012.stdout: "osdop_create": 0, 2024-09-11T00:32:26.571 INFO:teuthology.orchestra.run.smithi012.stdout: "osdop_read": 2315, 2024-09-11T00:32:26.571 INFO:teuthology.orchestra.run.smithi012.stdout: "osdop_write": 0, 2024-09-11T00:32:26.571 INFO:teuthology.orchestra.run.smithi012.stdout: "osdop_writefull": 0, 2024-09-11T00:32:26.571 INFO:teuthology.orchestra.run.smithi012.stdout: "osdop_writesame": 0, 2024-09-11T00:32:26.571 INFO:teuthology.orchestra.run.smithi012.stdout: "osdop_append": 0, 2024-09-11T00:32:26.571 INFO:teuthology.orchestra.run.smithi012.stdout: "osdop_zero": 0, 2024-09-11T00:32:26.571 INFO:teuthology.orchestra.run.smithi012.stdout: "osdop_truncate": 0, 2024-09-11T00:32:26.571 INFO:teuthology.orchestra.run.smithi012.stdout: "osdop_delete": 0, 2024-09-11T00:32:26.572 INFO:teuthology.orchestra.run.smithi012.stdout: "osdop_mapext": 0, 2024-09-11T00:32:26.572 INFO:teuthology.orchestra.run.smithi012.stdout: "osdop_sparse_read": 0, 2024-09-11T00:32:26.572 INFO:teuthology.orchestra.run.smithi012.stdout: "osdop_clonerange": 0, 2024-09-11T00:32:26.572 INFO:teuthology.orchestra.run.smithi012.stdout: "osdop_getxattr": 1, 2024-09-11T00:32:26.572 INFO:teuthology.orchestra.run.smithi012.stdout: "osdop_setxattr": 0, 2024-09-11T00:32:26.572 INFO:teuthology.orchestra.run.smithi012.stdout: "osdop_cmpxattr": 0, 2024-09-11T00:32:26.572 INFO:teuthology.orchestra.run.smithi012.stdout: "osdop_rmxattr": 0, 2024-09-11T00:32:26.572 INFO:teuthology.orchestra.run.smithi012.stdout: "osdop_resetxattrs": 0, 2024-09-11T00:32:26.572 INFO:teuthology.orchestra.run.smithi012.stdout: "osdop_call": 0, 2024-09-11T00:32:26.572 INFO:teuthology.orchestra.run.smithi012.stdout: "osdop_watch": 0, 2024-09-11T00:32:26.572 INFO:teuthology.orchestra.run.smithi012.stdout: "osdop_notify": 0, 2024-09-11T00:32:26.572 INFO:teuthology.orchestra.run.smithi012.stdout: "osdop_src_cmpxattr": 0, 2024-09-11T00:32:26.572 INFO:teuthology.orchestra.run.smithi012.stdout: "osdop_pgls": 0, 2024-09-11T00:32:26.572 INFO:teuthology.orchestra.run.smithi012.stdout: "osdop_pgls_filter": 0, 2024-09-11T00:32:26.572 INFO:teuthology.orchestra.run.smithi012.stdout: "osdop_other": 0, 2024-09-11T00:32:26.573 INFO:teuthology.orchestra.run.smithi012.stdout: "linger_active": 0, 2024-09-11T00:32:26.573 INFO:teuthology.orchestra.run.smithi012.stdout: "linger_send": 0, 2024-09-11T00:32:26.573 INFO:teuthology.orchestra.run.smithi012.stdout: "linger_resend": 0, 2024-09-11T00:32:26.573 INFO:teuthology.orchestra.run.smithi012.stdout: "linger_ping": 0, 2024-09-11T00:32:26.573 INFO:teuthology.orchestra.run.smithi012.stdout: "poolop_active": 0, 2024-09-11T00:32:26.573 INFO:teuthology.orchestra.run.smithi012.stdout: "poolop_send": 0, 2024-09-11T00:32:26.573 INFO:teuthology.orchestra.run.smithi012.stdout: "poolop_resend": 0, 2024-09-11T00:32:26.573 INFO:teuthology.orchestra.run.smithi012.stdout: "poolstat_active": 0, 2024-09-11T00:32:26.573 INFO:teuthology.orchestra.run.smithi012.stdout: "poolstat_send": 0, 2024-09-11T00:32:26.573 INFO:teuthology.orchestra.run.smithi012.stdout: "poolstat_resend": 0, 2024-09-11T00:32:26.573 INFO:teuthology.orchestra.run.smithi012.stdout: "statfs_active": 0, 2024-09-11T00:32:26.573 INFO:teuthology.orchestra.run.smithi012.stdout: "statfs_send": 0, 2024-09-11T00:32:26.574 INFO:teuthology.orchestra.run.smithi012.stdout: "statfs_resend": 0, 2024-09-11T00:32:26.574 INFO:teuthology.orchestra.run.smithi012.stdout: "command_active": 0, 2024-09-11T00:32:26.574 INFO:teuthology.orchestra.run.smithi012.stdout: "command_send": 0, 2024-09-11T00:32:26.574 INFO:teuthology.orchestra.run.smithi012.stdout: "command_resend": 0, 2024-09-11T00:32:26.574 INFO:teuthology.orchestra.run.smithi012.stdout: "map_epoch": 87, 2024-09-11T00:32:26.574 INFO:teuthology.orchestra.run.smithi012.stdout: "map_full": 0, 2024-09-11T00:32:26.574 INFO:teuthology.orchestra.run.smithi012.stdout: "map_inc": 6, 2024-09-11T00:32:26.574 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_sessions": 11, 2024-09-11T00:32:26.574 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_session_open": 12, 2024-09-11T00:32:26.574 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_session_close": 1, 2024-09-11T00:32:26.574 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_laggy": 0, 2024-09-11T00:32:26.574 INFO:teuthology.orchestra.run.smithi012.stdout: "omap_wr": 0, 2024-09-11T00:32:26.574 INFO:teuthology.orchestra.run.smithi012.stdout: "omap_rd": 2, 2024-09-11T00:32:26.574 INFO:teuthology.orchestra.run.smithi012.stdout: "omap_del": 0 2024-09-11T00:32:26.575 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:26.575 INFO:teuthology.orchestra.run.smithi012.stdout: "oft": { 2024-09-11T00:32:26.575 INFO:teuthology.orchestra.run.smithi012.stdout: "omap_total_objs": 0, 2024-09-11T00:32:26.575 INFO:teuthology.orchestra.run.smithi012.stdout: "omap_total_kv_pairs": 0, 2024-09-11T00:32:26.575 INFO:teuthology.orchestra.run.smithi012.stdout: "omap_total_updates": 0, 2024-09-11T00:32:26.575 INFO:teuthology.orchestra.run.smithi012.stdout: "omap_total_removes": 0 2024-09-11T00:32:26.575 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:26.575 INFO:teuthology.orchestra.run.smithi012.stdout: "purge_queue": { 2024-09-11T00:32:26.575 INFO:teuthology.orchestra.run.smithi012.stdout: "pq_executing_ops": 0, 2024-09-11T00:32:26.575 INFO:teuthology.orchestra.run.smithi012.stdout: "pq_executing_ops_high_water": 0, 2024-09-11T00:32:26.575 INFO:teuthology.orchestra.run.smithi012.stdout: "pq_executing": 0, 2024-09-11T00:32:26.575 INFO:teuthology.orchestra.run.smithi012.stdout: "pq_executing_high_water": 0, 2024-09-11T00:32:26.575 INFO:teuthology.orchestra.run.smithi012.stdout: "pq_executed_ops": 0, 2024-09-11T00:32:26.575 INFO:teuthology.orchestra.run.smithi012.stdout: "pq_executed": 0, 2024-09-11T00:32:26.576 INFO:teuthology.orchestra.run.smithi012.stdout: "pq_item_in_journal": 0 2024-09-11T00:32:26.576 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:26.576 INFO:teuthology.orchestra.run.smithi012.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-11T00:32:26.576 INFO:teuthology.orchestra.run.smithi012.stdout: "val": 0, 2024-09-11T00:32:26.576 INFO:teuthology.orchestra.run.smithi012.stdout: "max": 104857600, 2024-09-11T00:32:26.576 INFO:teuthology.orchestra.run.smithi012.stdout: "get_started": 0, 2024-09-11T00:32:26.576 INFO:teuthology.orchestra.run.smithi012.stdout: "get": 6042, 2024-09-11T00:32:26.576 INFO:teuthology.orchestra.run.smithi012.stdout: "get_sum": 157249363, 2024-09-11T00:32:26.576 INFO:teuthology.orchestra.run.smithi012.stdout: "get_or_fail_fail": 0, 2024-09-11T00:32:26.576 INFO:teuthology.orchestra.run.smithi012.stdout: "get_or_fail_success": 6042, 2024-09-11T00:32:26.576 INFO:teuthology.orchestra.run.smithi012.stdout: "take": 0, 2024-09-11T00:32:26.576 INFO:teuthology.orchestra.run.smithi012.stdout: "take_sum": 0, 2024-09-11T00:32:26.577 INFO:teuthology.orchestra.run.smithi012.stdout: "put": 6042, 2024-09-11T00:32:26.577 INFO:teuthology.orchestra.run.smithi012.stdout: "put_sum": 157249363, 2024-09-11T00:32:26.577 INFO:teuthology.orchestra.run.smithi012.stdout: "wait": { 2024-09-11T00:32:26.577 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 0, 2024-09-11T00:32:26.577 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.000000000, 2024-09-11T00:32:26.577 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000000000 2024-09-11T00:32:26.577 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:32:26.577 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:26.577 INFO:teuthology.orchestra.run.smithi012.stdout: "throttle-objecter_bytes": { 2024-09-11T00:32:26.577 INFO:teuthology.orchestra.run.smithi012.stdout: "val": 0, 2024-09-11T00:32:26.577 INFO:teuthology.orchestra.run.smithi012.stdout: "max": 104857600, 2024-09-11T00:32:26.577 INFO:teuthology.orchestra.run.smithi012.stdout: "get_started": 0, 2024-09-11T00:32:26.578 INFO:teuthology.orchestra.run.smithi012.stdout: "get": 0, 2024-09-11T00:32:26.578 INFO:teuthology.orchestra.run.smithi012.stdout: "get_sum": 0, 2024-09-11T00:32:26.578 INFO:teuthology.orchestra.run.smithi012.stdout: "get_or_fail_fail": 0, 2024-09-11T00:32:26.578 INFO:teuthology.orchestra.run.smithi012.stdout: "get_or_fail_success": 0, 2024-09-11T00:32:26.578 INFO:teuthology.orchestra.run.smithi012.stdout: "take": 5488, 2024-09-11T00:32:26.578 INFO:teuthology.orchestra.run.smithi012.stdout: "take_sum": 155114943, 2024-09-11T00:32:26.578 INFO:teuthology.orchestra.run.smithi012.stdout: "put": 5488, 2024-09-11T00:32:26.578 INFO:teuthology.orchestra.run.smithi012.stdout: "put_sum": 155114943, 2024-09-11T00:32:26.578 INFO:teuthology.orchestra.run.smithi012.stdout: "wait": { 2024-09-11T00:32:26.578 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 0, 2024-09-11T00:32:26.578 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.000000000, 2024-09-11T00:32:26.578 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000000000 2024-09-11T00:32:26.578 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:32:26.578 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:26.579 INFO:teuthology.orchestra.run.smithi012.stdout: "throttle-objecter_ops": { 2024-09-11T00:32:26.579 INFO:teuthology.orchestra.run.smithi012.stdout: "val": 0, 2024-09-11T00:32:26.579 INFO:teuthology.orchestra.run.smithi012.stdout: "max": 1024, 2024-09-11T00:32:26.579 INFO:teuthology.orchestra.run.smithi012.stdout: "get_started": 0, 2024-09-11T00:32:26.579 INFO:teuthology.orchestra.run.smithi012.stdout: "get": 0, 2024-09-11T00:32:26.579 INFO:teuthology.orchestra.run.smithi012.stdout: "get_sum": 0, 2024-09-11T00:32:26.579 INFO:teuthology.orchestra.run.smithi012.stdout: "get_or_fail_fail": 0, 2024-09-11T00:32:26.579 INFO:teuthology.orchestra.run.smithi012.stdout: "get_or_fail_success": 0, 2024-09-11T00:32:26.579 INFO:teuthology.orchestra.run.smithi012.stdout: "take": 5488, 2024-09-11T00:32:26.579 INFO:teuthology.orchestra.run.smithi012.stdout: "take_sum": 5488, 2024-09-11T00:32:26.579 INFO:teuthology.orchestra.run.smithi012.stdout: "put": 5488, 2024-09-11T00:32:26.579 INFO:teuthology.orchestra.run.smithi012.stdout: "put_sum": 5488, 2024-09-11T00:32:26.579 INFO:teuthology.orchestra.run.smithi012.stdout: "wait": { 2024-09-11T00:32:26.580 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 0, 2024-09-11T00:32:26.580 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.000000000, 2024-09-11T00:32:26.580 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000000000 2024-09-11T00:32:26.580 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:32:26.580 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:26.580 INFO:teuthology.orchestra.run.smithi012.stdout: "throttle-write_buf_throttle": { 2024-09-11T00:32:26.580 INFO:teuthology.orchestra.run.smithi012.stdout: "val": 0, 2024-09-11T00:32:26.580 INFO:teuthology.orchestra.run.smithi012.stdout: "max": 3758096384, 2024-09-11T00:32:26.580 INFO:teuthology.orchestra.run.smithi012.stdout: "get_started": 0, 2024-09-11T00:32:26.580 INFO:teuthology.orchestra.run.smithi012.stdout: "get": 0, 2024-09-11T00:32:26.580 INFO:teuthology.orchestra.run.smithi012.stdout: "get_sum": 0, 2024-09-11T00:32:26.580 INFO:teuthology.orchestra.run.smithi012.stdout: "get_or_fail_fail": 0, 2024-09-11T00:32:26.580 INFO:teuthology.orchestra.run.smithi012.stdout: "get_or_fail_success": 0, 2024-09-11T00:32:26.580 INFO:teuthology.orchestra.run.smithi012.stdout: "take": 0, 2024-09-11T00:32:26.581 INFO:teuthology.orchestra.run.smithi012.stdout: "take_sum": 0, 2024-09-11T00:32:26.581 INFO:teuthology.orchestra.run.smithi012.stdout: "put": 0, 2024-09-11T00:32:26.581 INFO:teuthology.orchestra.run.smithi012.stdout: "put_sum": 0, 2024-09-11T00:32:26.581 INFO:teuthology.orchestra.run.smithi012.stdout: "wait": { 2024-09-11T00:32:26.581 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 0, 2024-09-11T00:32:26.581 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.000000000, 2024-09-11T00:32:26.581 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000000000 2024-09-11T00:32:26.581 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:32:26.581 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:26.581 INFO:teuthology.orchestra.run.smithi012.stdout: "throttle-write_buf_throttle-0x55865887f040": { 2024-09-11T00:32:26.581 INFO:teuthology.orchestra.run.smithi012.stdout: "val": 0, 2024-09-11T00:32:26.581 INFO:teuthology.orchestra.run.smithi012.stdout: "max": 3758096384, 2024-09-11T00:32:26.581 INFO:teuthology.orchestra.run.smithi012.stdout: "get_started": 0, 2024-09-11T00:32:26.581 INFO:teuthology.orchestra.run.smithi012.stdout: "get": 0, 2024-09-11T00:32:26.582 INFO:teuthology.orchestra.run.smithi012.stdout: "get_sum": 0, 2024-09-11T00:32:26.582 INFO:teuthology.orchestra.run.smithi012.stdout: "get_or_fail_fail": 0, 2024-09-11T00:32:26.582 INFO:teuthology.orchestra.run.smithi012.stdout: "get_or_fail_success": 0, 2024-09-11T00:32:26.582 INFO:teuthology.orchestra.run.smithi012.stdout: "take": 0, 2024-09-11T00:32:26.582 INFO:teuthology.orchestra.run.smithi012.stdout: "take_sum": 0, 2024-09-11T00:32:26.582 INFO:teuthology.orchestra.run.smithi012.stdout: "put": 0, 2024-09-11T00:32:26.582 INFO:teuthology.orchestra.run.smithi012.stdout: "put_sum": 0, 2024-09-11T00:32:26.582 INFO:teuthology.orchestra.run.smithi012.stdout: "wait": { 2024-09-11T00:32:26.582 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 0, 2024-09-11T00:32:26.582 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.000000000, 2024-09-11T00:32:26.582 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000000000 2024-09-11T00:32:26.582 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:32:26.582 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:26.582 INFO:teuthology.orchestra.run.smithi012.stdout: "trackedop": { 2024-09-11T00:32:26.583 INFO:teuthology.orchestra.run.smithi012.stdout: "slow_ops_count": 0 2024-09-11T00:32:26.583 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:32:26.583 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-09-11T00:32:27.374 INFO:tasks.check_counter:Daemon mds.a mds.exported=0 2024-09-11T00:32:27.374 INFO:tasks.check_counter:Daemon mds.a mds.imported=0 2024-09-11T00:32:27.374 DEBUG:tasks.check_counter:Getting stats from d 2024-09-11T00:32:27.374 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid ff3ad74a-6fcf-11ef-bcea-c7b262605968 -- ceph daemon mds.d perf dump 2024-09-11T00:32:27.756 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:32:28.231 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:32:27 smithi012 ceph-mon[30333]: pgmap v1161: 129 pgs: 129 active+clean; 671 MiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 322 KiB/s rd, 302 KiB/s wr, 163 op/s 2024-09-11T00:32:28.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:32:27 smithi165 ceph-mon[35356]: pgmap v1161: 129 pgs: 129 active+clean; 671 MiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 322 KiB/s rd, 302 KiB/s wr, 163 op/s 2024-09-11T00:32:28.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:32:27 smithi178 ceph-mon[35919]: pgmap v1161: 129 pgs: 129 active+clean; 671 MiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 322 KiB/s rd, 302 KiB/s wr, 163 op/s 2024-09-11T00:32:28.378 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-09-11T00:32:28.379 INFO:teuthology.orchestra.run.smithi012.stdout: "AsyncMessenger::Worker-0": { 2024-09-11T00:32:28.379 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_recv_messages": 0, 2024-09-11T00:32:28.379 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_send_messages": 2, 2024-09-11T00:32:28.379 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_recv_bytes": 0, 2024-09-11T00:32:28.379 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_send_bytes": 0, 2024-09-11T00:32:28.379 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_created_connections": 4, 2024-09-11T00:32:28.379 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_active_connections": 0, 2024-09-11T00:32:28.379 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_running_total_time": 0.003249361, 2024-09-11T00:32:28.379 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_running_send_time": 0.000000000, 2024-09-11T00:32:28.379 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_running_recv_time": 0.000587776, 2024-09-11T00:32:28.379 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-11T00:32:28.380 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_send_messages_queue_lat": { 2024-09-11T00:32:28.380 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 0, 2024-09-11T00:32:28.380 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.000000000, 2024-09-11T00:32:28.380 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000000000 2024-09-11T00:32:28.380 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:28.380 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_handle_ack_lat": { 2024-09-11T00:32:28.380 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 0, 2024-09-11T00:32:28.380 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.000000000, 2024-09-11T00:32:28.380 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000000000 2024-09-11T00:32:28.380 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:28.380 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-11T00:32:28.381 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_send_encrypted_bytes": 0 2024-09-11T00:32:28.381 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:28.381 INFO:teuthology.orchestra.run.smithi012.stdout: "AsyncMessenger::Worker-1": { 2024-09-11T00:32:28.381 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_recv_messages": 2, 2024-09-11T00:32:28.381 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_send_messages": 352, 2024-09-11T00:32:28.381 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_recv_bytes": 320, 2024-09-11T00:32:28.381 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_send_bytes": 222688, 2024-09-11T00:32:28.381 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_created_connections": 4, 2024-09-11T00:32:28.381 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_active_connections": 2, 2024-09-11T00:32:28.381 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_running_total_time": 0.044553378, 2024-09-11T00:32:28.381 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_running_send_time": 0.037669973, 2024-09-11T00:32:28.381 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_running_recv_time": 0.002232733, 2024-09-11T00:32:28.381 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-11T00:32:28.382 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_send_messages_queue_lat": { 2024-09-11T00:32:28.382 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 350, 2024-09-11T00:32:28.382 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.032028285, 2024-09-11T00:32:28.382 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000091509 2024-09-11T00:32:28.382 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:28.382 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_handle_ack_lat": { 2024-09-11T00:32:28.382 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 0, 2024-09-11T00:32:28.382 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.000000000, 2024-09-11T00:32:28.382 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000000000 2024-09-11T00:32:28.382 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:28.382 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_recv_encrypted_bytes": 320, 2024-09-11T00:32:28.382 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_send_encrypted_bytes": 222688 2024-09-11T00:32:28.382 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:28.383 INFO:teuthology.orchestra.run.smithi012.stdout: "AsyncMessenger::Worker-2": { 2024-09-11T00:32:28.383 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_recv_messages": 467, 2024-09-11T00:32:28.383 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_send_messages": 462, 2024-09-11T00:32:28.383 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_recv_bytes": 597280, 2024-09-11T00:32:28.383 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_send_bytes": 285520, 2024-09-11T00:32:28.383 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_created_connections": 2, 2024-09-11T00:32:28.383 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_active_connections": 1, 2024-09-11T00:32:28.383 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_running_total_time": 0.094755057, 2024-09-11T00:32:28.383 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_running_send_time": 0.057362272, 2024-09-11T00:32:28.383 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_running_recv_time": 0.041651185, 2024-09-11T00:32:28.383 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-11T00:32:28.383 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_send_messages_queue_lat": { 2024-09-11T00:32:28.383 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 462, 2024-09-11T00:32:28.383 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.046745138, 2024-09-11T00:32:28.384 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000101179 2024-09-11T00:32:28.384 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:28.384 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_handle_ack_lat": { 2024-09-11T00:32:28.384 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 0, 2024-09-11T00:32:28.384 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.000000000, 2024-09-11T00:32:28.384 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000000000 2024-09-11T00:32:28.384 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:28.384 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_recv_encrypted_bytes": 597280, 2024-09-11T00:32:28.384 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_send_encrypted_bytes": 285520 2024-09-11T00:32:28.384 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:28.384 INFO:teuthology.orchestra.run.smithi012.stdout: "cct": { 2024-09-11T00:32:28.384 INFO:teuthology.orchestra.run.smithi012.stdout: "total_workers": 0, 2024-09-11T00:32:28.384 INFO:teuthology.orchestra.run.smithi012.stdout: "unhealthy_workers": 0 2024-09-11T00:32:28.385 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:28.385 INFO:teuthology.orchestra.run.smithi012.stdout: "mempool": { 2024-09-11T00:32:28.385 INFO:teuthology.orchestra.run.smithi012.stdout: "bloom_filter_bytes": 0, 2024-09-11T00:32:28.385 INFO:teuthology.orchestra.run.smithi012.stdout: "bloom_filter_items": 0, 2024-09-11T00:32:28.385 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_alloc_bytes": 0, 2024-09-11T00:32:28.385 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_alloc_items": 0, 2024-09-11T00:32:28.385 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_cache_data_bytes": 0, 2024-09-11T00:32:28.385 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_cache_data_items": 0, 2024-09-11T00:32:28.385 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-11T00:32:28.385 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_cache_onode_items": 0, 2024-09-11T00:32:28.385 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-11T00:32:28.385 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_cache_meta_items": 0, 2024-09-11T00:32:28.385 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_cache_other_bytes": 0, 2024-09-11T00:32:28.386 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_cache_other_items": 0, 2024-09-11T00:32:28.386 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-11T00:32:28.386 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_cache_buffer_items": 0, 2024-09-11T00:32:28.386 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_extent_bytes": 0, 2024-09-11T00:32:28.386 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_extent_items": 0, 2024-09-11T00:32:28.386 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_blob_bytes": 0, 2024-09-11T00:32:28.386 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_blob_items": 0, 2024-09-11T00:32:28.386 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-11T00:32:28.386 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_shared_blob_items": 0, 2024-09-11T00:32:28.386 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-11T00:32:28.386 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_inline_bl_items": 0, 2024-09-11T00:32:28.386 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_fsck_bytes": 0, 2024-09-11T00:32:28.387 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_fsck_items": 0, 2024-09-11T00:32:28.387 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_txc_bytes": 0, 2024-09-11T00:32:28.387 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_txc_items": 0, 2024-09-11T00:32:28.387 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-11T00:32:28.387 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_writing_deferred_items": 0, 2024-09-11T00:32:28.387 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_writing_bytes": 0, 2024-09-11T00:32:28.387 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_writing_items": 0, 2024-09-11T00:32:28.387 INFO:teuthology.orchestra.run.smithi012.stdout: "bluefs_bytes": 0, 2024-09-11T00:32:28.387 INFO:teuthology.orchestra.run.smithi012.stdout: "bluefs_items": 0, 2024-09-11T00:32:28.387 INFO:teuthology.orchestra.run.smithi012.stdout: "bluefs_file_reader_bytes": 0, 2024-09-11T00:32:28.387 INFO:teuthology.orchestra.run.smithi012.stdout: "bluefs_file_reader_items": 0, 2024-09-11T00:32:28.387 INFO:teuthology.orchestra.run.smithi012.stdout: "bluefs_file_writer_bytes": 0, 2024-09-11T00:32:28.387 INFO:teuthology.orchestra.run.smithi012.stdout: "bluefs_file_writer_items": 0, 2024-09-11T00:32:28.388 INFO:teuthology.orchestra.run.smithi012.stdout: "buffer_anon_bytes": 149735, 2024-09-11T00:32:28.388 INFO:teuthology.orchestra.run.smithi012.stdout: "buffer_anon_items": 30, 2024-09-11T00:32:28.388 INFO:teuthology.orchestra.run.smithi012.stdout: "buffer_meta_bytes": 0, 2024-09-11T00:32:28.388 INFO:teuthology.orchestra.run.smithi012.stdout: "buffer_meta_items": 0, 2024-09-11T00:32:28.388 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_bytes": 0, 2024-09-11T00:32:28.388 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_items": 0, 2024-09-11T00:32:28.388 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_mapbl_bytes": 0, 2024-09-11T00:32:28.388 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_mapbl_items": 0, 2024-09-11T00:32:28.388 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_pglog_bytes": 0, 2024-09-11T00:32:28.388 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_pglog_items": 0, 2024-09-11T00:32:28.388 INFO:teuthology.orchestra.run.smithi012.stdout: "osdmap_bytes": 0, 2024-09-11T00:32:28.388 INFO:teuthology.orchestra.run.smithi012.stdout: "osdmap_items": 0, 2024-09-11T00:32:28.388 INFO:teuthology.orchestra.run.smithi012.stdout: "osdmap_mapping_bytes": 0, 2024-09-11T00:32:28.389 INFO:teuthology.orchestra.run.smithi012.stdout: "osdmap_mapping_items": 0, 2024-09-11T00:32:28.389 INFO:teuthology.orchestra.run.smithi012.stdout: "pgmap_bytes": 0, 2024-09-11T00:32:28.389 INFO:teuthology.orchestra.run.smithi012.stdout: "pgmap_items": 0, 2024-09-11T00:32:28.389 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_co_bytes": 968, 2024-09-11T00:32:28.389 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_co_items": 2, 2024-09-11T00:32:28.389 INFO:teuthology.orchestra.run.smithi012.stdout: "unittest_1_bytes": 0, 2024-09-11T00:32:28.389 INFO:teuthology.orchestra.run.smithi012.stdout: "unittest_1_items": 0, 2024-09-11T00:32:28.389 INFO:teuthology.orchestra.run.smithi012.stdout: "unittest_2_bytes": 0, 2024-09-11T00:32:28.389 INFO:teuthology.orchestra.run.smithi012.stdout: "unittest_2_items": 0 2024-09-11T00:32:28.389 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:28.389 INFO:teuthology.orchestra.run.smithi012.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-11T00:32:28.389 INFO:teuthology.orchestra.run.smithi012.stdout: "val": 0, 2024-09-11T00:32:28.390 INFO:teuthology.orchestra.run.smithi012.stdout: "max": 104857600, 2024-09-11T00:32:28.390 INFO:teuthology.orchestra.run.smithi012.stdout: "get_started": 0, 2024-09-11T00:32:28.390 INFO:teuthology.orchestra.run.smithi012.stdout: "get": 465, 2024-09-11T00:32:28.390 INFO:teuthology.orchestra.run.smithi012.stdout: "get_sum": 528830, 2024-09-11T00:32:28.390 INFO:teuthology.orchestra.run.smithi012.stdout: "get_or_fail_fail": 0, 2024-09-11T00:32:28.390 INFO:teuthology.orchestra.run.smithi012.stdout: "get_or_fail_success": 465, 2024-09-11T00:32:28.390 INFO:teuthology.orchestra.run.smithi012.stdout: "take": 0, 2024-09-11T00:32:28.390 INFO:teuthology.orchestra.run.smithi012.stdout: "take_sum": 0, 2024-09-11T00:32:28.390 INFO:teuthology.orchestra.run.smithi012.stdout: "put": 465, 2024-09-11T00:32:28.390 INFO:teuthology.orchestra.run.smithi012.stdout: "put_sum": 528830, 2024-09-11T00:32:28.390 INFO:teuthology.orchestra.run.smithi012.stdout: "wait": { 2024-09-11T00:32:28.390 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 0, 2024-09-11T00:32:28.390 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.000000000, 2024-09-11T00:32:28.390 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000000000 2024-09-11T00:32:28.391 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:32:28.391 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:32:28.391 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-09-11T00:32:29.081 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.d 2024-09-11T00:32:29.081 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.d 2024-09-11T00:32:29.081 DEBUG:tasks.check_counter:Getting stats from g 2024-09-11T00:32:29.082 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid ff3ad74a-6fcf-11ef-bcea-c7b262605968 -- ceph daemon mds.g perf dump 2024-09-11T00:32:29.461 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:32:30.239 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-09-11T00:32:30.239 INFO:teuthology.orchestra.run.smithi012.stdout: "AsyncMessenger::Worker-0": { 2024-09-11T00:32:30.239 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_recv_messages": 290148, 2024-09-11T00:32:30.239 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_send_messages": 381279, 2024-09-11T00:32:30.240 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_recv_bytes": 87338384, 2024-09-11T00:32:30.240 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_send_bytes": 300930432, 2024-09-11T00:32:30.240 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_created_connections": 13, 2024-09-11T00:32:30.240 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_active_connections": 7, 2024-09-11T00:32:30.240 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_running_total_time": 69.188015688, 2024-09-11T00:32:30.240 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_running_send_time": 53.779939785, 2024-09-11T00:32:30.240 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_running_recv_time": 22.740824163, 2024-09-11T00:32:30.240 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_running_fast_dispatch_time": 1.516385810, 2024-09-11T00:32:30.240 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_send_messages_queue_lat": { 2024-09-11T00:32:30.240 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 381275, 2024-09-11T00:32:30.240 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 69.076623940, 2024-09-11T00:32:30.240 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000181172 2024-09-11T00:32:30.241 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:30.241 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_handle_ack_lat": { 2024-09-11T00:32:30.241 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 388921, 2024-09-11T00:32:30.241 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.157673929, 2024-09-11T00:32:30.241 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000000405 2024-09-11T00:32:30.241 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:30.241 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-11T00:32:30.241 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_send_encrypted_bytes": 0 2024-09-11T00:32:30.241 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:30.241 INFO:teuthology.orchestra.run.smithi012.stdout: "AsyncMessenger::Worker-1": { 2024-09-11T00:32:30.241 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_recv_messages": 51199, 2024-09-11T00:32:30.241 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_send_messages": 51429, 2024-09-11T00:32:30.241 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_recv_bytes": 13637142, 2024-09-11T00:32:30.242 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_send_bytes": 60879873, 2024-09-11T00:32:30.242 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_created_connections": 15, 2024-09-11T00:32:30.242 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_active_connections": 6, 2024-09-11T00:32:30.242 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_running_total_time": 7.620510537, 2024-09-11T00:32:30.242 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_running_send_time": 3.630719902, 2024-09-11T00:32:30.242 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_running_recv_time": 3.121892292, 2024-09-11T00:32:30.242 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_running_fast_dispatch_time": 1.050072148, 2024-09-11T00:32:30.242 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_send_messages_queue_lat": { 2024-09-11T00:32:30.242 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 51429, 2024-09-11T00:32:30.242 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 10.564239435, 2024-09-11T00:32:30.242 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000205414 2024-09-11T00:32:30.242 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:30.242 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_handle_ack_lat": { 2024-09-11T00:32:30.243 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 62305, 2024-09-11T00:32:30.243 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.019274605, 2024-09-11T00:32:30.243 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000000309 2024-09-11T00:32:30.243 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:30.243 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_recv_encrypted_bytes": 320, 2024-09-11T00:32:30.243 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_send_encrypted_bytes": 788592 2024-09-11T00:32:30.243 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:30.243 INFO:teuthology.orchestra.run.smithi012.stdout: "AsyncMessenger::Worker-2": { 2024-09-11T00:32:30.243 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_recv_messages": 61777, 2024-09-11T00:32:30.243 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_send_messages": 60246, 2024-09-11T00:32:30.243 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_recv_bytes": 19373982, 2024-09-11T00:32:30.243 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_send_bytes": 79194199, 2024-09-11T00:32:30.243 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_created_connections": 8, 2024-09-11T00:32:30.244 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_active_connections": 6, 2024-09-11T00:32:30.244 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_running_total_time": 9.633282211, 2024-09-11T00:32:30.244 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_running_send_time": 4.305317148, 2024-09-11T00:32:30.244 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_running_recv_time": 3.567901888, 2024-09-11T00:32:30.244 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_running_fast_dispatch_time": 1.836055198, 2024-09-11T00:32:30.244 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_send_messages_queue_lat": { 2024-09-11T00:32:30.244 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 60246, 2024-09-11T00:32:30.244 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 11.610485907, 2024-09-11T00:32:30.244 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000192717 2024-09-11T00:32:30.244 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:30.244 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_handle_ack_lat": { 2024-09-11T00:32:30.244 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 62092, 2024-09-11T00:32:30.245 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.016338088, 2024-09-11T00:32:30.245 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000000263 2024-09-11T00:32:30.245 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:30.245 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_recv_encrypted_bytes": 1211120, 2024-09-11T00:32:30.245 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_send_encrypted_bytes": 293408 2024-09-11T00:32:30.245 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:30.245 INFO:teuthology.orchestra.run.smithi012.stdout: "cct": { 2024-09-11T00:32:30.245 INFO:teuthology.orchestra.run.smithi012.stdout: "total_workers": 1, 2024-09-11T00:32:30.245 INFO:teuthology.orchestra.run.smithi012.stdout: "unhealthy_workers": 0 2024-09-11T00:32:30.245 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:30.245 INFO:teuthology.orchestra.run.smithi012.stdout: "finisher-MDSRank": { 2024-09-11T00:32:30.245 INFO:teuthology.orchestra.run.smithi012.stdout: "queue_len": 0, 2024-09-11T00:32:30.245 INFO:teuthology.orchestra.run.smithi012.stdout: "complete_latency": { 2024-09-11T00:32:30.246 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 52663, 2024-09-11T00:32:30.246 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 79.914486176, 2024-09-11T00:32:30.246 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.001517469 2024-09-11T00:32:30.246 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:32:30.246 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:30.246 INFO:teuthology.orchestra.run.smithi012.stdout: "finisher-PurgeQueue": { 2024-09-11T00:32:30.246 INFO:teuthology.orchestra.run.smithi012.stdout: "queue_len": 0, 2024-09-11T00:32:30.246 INFO:teuthology.orchestra.run.smithi012.stdout: "complete_latency": { 2024-09-11T00:32:30.246 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 37592, 2024-09-11T00:32:30.246 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 7.098065886, 2024-09-11T00:32:30.246 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000188818 2024-09-11T00:32:30.247 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:32:30.247 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:30.247 INFO:teuthology.orchestra.run.smithi012.stdout: "mds": { 2024-09-11T00:32:30.247 INFO:teuthology.orchestra.run.smithi012.stdout: "request": 119550, 2024-09-11T00:32:30.247 INFO:teuthology.orchestra.run.smithi012.stdout: "reply": 111058, 2024-09-11T00:32:30.247 INFO:teuthology.orchestra.run.smithi012.stdout: "reply_latency": { 2024-09-11T00:32:30.247 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 111058, 2024-09-11T00:32:30.247 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 960.947315444, 2024-09-11T00:32:30.247 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.008652661 2024-09-11T00:32:30.247 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:30.247 INFO:teuthology.orchestra.run.smithi012.stdout: "slow_reply": 0, 2024-09-11T00:32:30.247 INFO:teuthology.orchestra.run.smithi012.stdout: "forward": 8492, 2024-09-11T00:32:30.248 INFO:teuthology.orchestra.run.smithi012.stdout: "dir_fetch_complete": 0, 2024-09-11T00:32:30.248 INFO:teuthology.orchestra.run.smithi012.stdout: "dir_fetch_keys": 0, 2024-09-11T00:32:30.248 INFO:teuthology.orchestra.run.smithi012.stdout: "dir_commit": 11, 2024-09-11T00:32:30.248 INFO:teuthology.orchestra.run.smithi012.stdout: "dir_split": 0, 2024-09-11T00:32:30.248 INFO:teuthology.orchestra.run.smithi012.stdout: "dir_merge": 0, 2024-09-11T00:32:30.248 INFO:teuthology.orchestra.run.smithi012.stdout: "inodes": 26, 2024-09-11T00:32:30.248 INFO:teuthology.orchestra.run.smithi012.stdout: "inodes_top": 7, 2024-09-11T00:32:30.248 INFO:teuthology.orchestra.run.smithi012.stdout: "inodes_bottom": 7, 2024-09-11T00:32:30.248 INFO:teuthology.orchestra.run.smithi012.stdout: "inodes_pin_tail": 12, 2024-09-11T00:32:30.248 INFO:teuthology.orchestra.run.smithi012.stdout: "inodes_pinned": 15, 2024-09-11T00:32:30.248 INFO:teuthology.orchestra.run.smithi012.stdout: "inodes_expired": 701, 2024-09-11T00:32:30.248 INFO:teuthology.orchestra.run.smithi012.stdout: "inodes_with_caps": 0, 2024-09-11T00:32:30.248 INFO:teuthology.orchestra.run.smithi012.stdout: "caps": 0, 2024-09-11T00:32:30.248 INFO:teuthology.orchestra.run.smithi012.stdout: "subtrees": 6, 2024-09-11T00:32:30.249 INFO:teuthology.orchestra.run.smithi012.stdout: "traverse": 193655, 2024-09-11T00:32:30.249 INFO:teuthology.orchestra.run.smithi012.stdout: "traverse_hit": 100565, 2024-09-11T00:32:30.249 INFO:teuthology.orchestra.run.smithi012.stdout: "traverse_forward": 8484, 2024-09-11T00:32:30.249 INFO:teuthology.orchestra.run.smithi012.stdout: "traverse_discover": 92, 2024-09-11T00:32:30.249 INFO:teuthology.orchestra.run.smithi012.stdout: "traverse_dir_fetch": 0, 2024-09-11T00:32:30.249 INFO:teuthology.orchestra.run.smithi012.stdout: "traverse_remote_ino": 0, 2024-09-11T00:32:30.249 INFO:teuthology.orchestra.run.smithi012.stdout: "traverse_lock": 1247, 2024-09-11T00:32:30.249 INFO:teuthology.orchestra.run.smithi012.stdout: "load_cent": 5053, 2024-09-11T00:32:30.249 INFO:teuthology.orchestra.run.smithi012.stdout: "q": 3, 2024-09-11T00:32:30.249 INFO:teuthology.orchestra.run.smithi012.stdout: "exported": 20, 2024-09-11T00:32:30.249 INFO:teuthology.orchestra.run.smithi012.stdout: "exported_inodes": 372, 2024-09-11T00:32:30.249 INFO:teuthology.orchestra.run.smithi012.stdout: "imported": 20, 2024-09-11T00:32:30.249 INFO:teuthology.orchestra.run.smithi012.stdout: "imported_inodes": 513, 2024-09-11T00:32:30.250 INFO:teuthology.orchestra.run.smithi012.stdout: "openino_dir_fetch": 0, 2024-09-11T00:32:30.250 INFO:teuthology.orchestra.run.smithi012.stdout: "openino_backtrace_fetch": 0, 2024-09-11T00:32:30.250 INFO:teuthology.orchestra.run.smithi012.stdout: "openino_peer_discover": 0, 2024-09-11T00:32:30.250 INFO:teuthology.orchestra.run.smithi012.stdout: "root_rfiles": 4, 2024-09-11T00:32:30.250 INFO:teuthology.orchestra.run.smithi012.stdout: "root_rbytes": 234, 2024-09-11T00:32:30.250 INFO:teuthology.orchestra.run.smithi012.stdout: "root_rsnaps": 0, 2024-09-11T00:32:30.250 INFO:teuthology.orchestra.run.smithi012.stdout: "scrub_backtrace_fetch": 0, 2024-09-11T00:32:30.250 INFO:teuthology.orchestra.run.smithi012.stdout: "scrub_set_tag": 0, 2024-09-11T00:32:30.250 INFO:teuthology.orchestra.run.smithi012.stdout: "scrub_backtrace_repaired": 0, 2024-09-11T00:32:30.250 INFO:teuthology.orchestra.run.smithi012.stdout: "scrub_inotable_repaired": 0, 2024-09-11T00:32:30.250 INFO:teuthology.orchestra.run.smithi012.stdout: "scrub_dir_inodes": 0, 2024-09-11T00:32:30.250 INFO:teuthology.orchestra.run.smithi012.stdout: "scrub_dir_base_inodes": 0, 2024-09-11T00:32:30.251 INFO:teuthology.orchestra.run.smithi012.stdout: "scrub_dirfrag_rstats": 0, 2024-09-11T00:32:30.251 INFO:teuthology.orchestra.run.smithi012.stdout: "scrub_file_inodes": 0, 2024-09-11T00:32:30.251 INFO:teuthology.orchestra.run.smithi012.stdout: "handle_inode_file_caps": 52, 2024-09-11T00:32:30.251 INFO:teuthology.orchestra.run.smithi012.stdout: "ceph_cap_op_revoke": 17431, 2024-09-11T00:32:30.251 INFO:teuthology.orchestra.run.smithi012.stdout: "ceph_cap_op_grant": 88109, 2024-09-11T00:32:30.251 INFO:teuthology.orchestra.run.smithi012.stdout: "ceph_cap_op_trunc": 0, 2024-09-11T00:32:30.251 INFO:teuthology.orchestra.run.smithi012.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-11T00:32:30.251 INFO:teuthology.orchestra.run.smithi012.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-11T00:32:30.251 INFO:teuthology.orchestra.run.smithi012.stdout: "handle_client_caps": 49968, 2024-09-11T00:32:30.251 INFO:teuthology.orchestra.run.smithi012.stdout: "handle_client_caps_dirty": 12037, 2024-09-11T00:32:30.251 INFO:teuthology.orchestra.run.smithi012.stdout: "handle_client_cap_release": 1862, 2024-09-11T00:32:30.251 INFO:teuthology.orchestra.run.smithi012.stdout: "process_request_cap_release": 82589 2024-09-11T00:32:30.251 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:30.251 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_cache": { 2024-09-11T00:32:30.251 INFO:teuthology.orchestra.run.smithi012.stdout: "dir_update": 143, 2024-09-11T00:32:30.252 INFO:teuthology.orchestra.run.smithi012.stdout: "dir_update_receipt": 108, 2024-09-11T00:32:30.252 INFO:teuthology.orchestra.run.smithi012.stdout: "dir_try_discover": 82, 2024-09-11T00:32:30.252 INFO:teuthology.orchestra.run.smithi012.stdout: "dir_send_discover": 135, 2024-09-11T00:32:30.252 INFO:teuthology.orchestra.run.smithi012.stdout: "dir_handle_discover": 240, 2024-09-11T00:32:30.252 INFO:teuthology.orchestra.run.smithi012.stdout: "num_strays": 2, 2024-09-11T00:32:30.252 INFO:teuthology.orchestra.run.smithi012.stdout: "num_strays_delayed": 0, 2024-09-11T00:32:30.252 INFO:teuthology.orchestra.run.smithi012.stdout: "num_strays_enqueuing": 0, 2024-09-11T00:32:30.252 INFO:teuthology.orchestra.run.smithi012.stdout: "strays_created": 20165, 2024-09-11T00:32:30.252 INFO:teuthology.orchestra.run.smithi012.stdout: "strays_enqueued": 20163, 2024-09-11T00:32:30.253 INFO:teuthology.orchestra.run.smithi012.stdout: "strays_reintegrated": 0, 2024-09-11T00:32:30.253 INFO:teuthology.orchestra.run.smithi012.stdout: "strays_migrated": 0, 2024-09-11T00:32:30.253 INFO:teuthology.orchestra.run.smithi012.stdout: "num_recovering_processing": 0, 2024-09-11T00:32:30.253 INFO:teuthology.orchestra.run.smithi012.stdout: "num_recovering_enqueued": 0, 2024-09-11T00:32:30.253 INFO:teuthology.orchestra.run.smithi012.stdout: "num_recovering_prioritized": 0, 2024-09-11T00:32:30.253 INFO:teuthology.orchestra.run.smithi012.stdout: "recovery_started": 0, 2024-09-11T00:32:30.253 INFO:teuthology.orchestra.run.smithi012.stdout: "recovery_completed": 0, 2024-09-11T00:32:30.253 INFO:teuthology.orchestra.run.smithi012.stdout: "ireq_quiesce_path": 19, 2024-09-11T00:32:30.253 INFO:teuthology.orchestra.run.smithi012.stdout: "ireq_quiesce_inode": 2038, 2024-09-11T00:32:30.254 INFO:teuthology.orchestra.run.smithi012.stdout: "ireq_enqueue_scrub": 0, 2024-09-11T00:32:30.254 INFO:teuthology.orchestra.run.smithi012.stdout: "ireq_exportdir": 21, 2024-09-11T00:32:30.254 INFO:teuthology.orchestra.run.smithi012.stdout: "ireq_flush": 0, 2024-09-11T00:32:30.254 INFO:teuthology.orchestra.run.smithi012.stdout: "ireq_fragmentdir": 0, 2024-09-11T00:32:30.254 INFO:teuthology.orchestra.run.smithi012.stdout: "ireq_fragstats": 0, 2024-09-11T00:32:30.254 INFO:teuthology.orchestra.run.smithi012.stdout: "ireq_inodestats": 0, 2024-09-11T00:32:30.254 INFO:teuthology.orchestra.run.smithi012.stdout: "uninline_started": 0, 2024-09-11T00:32:30.254 INFO:teuthology.orchestra.run.smithi012.stdout: "uninline_succeeded": 0, 2024-09-11T00:32:30.254 INFO:teuthology.orchestra.run.smithi012.stdout: "uninline_write_failed": 0 2024-09-11T00:32:30.254 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:30.254 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_log": { 2024-09-11T00:32:30.255 INFO:teuthology.orchestra.run.smithi012.stdout: "evlrg": 0, 2024-09-11T00:32:30.255 INFO:teuthology.orchestra.run.smithi012.stdout: "evadd": 98584, 2024-09-11T00:32:30.255 INFO:teuthology.orchestra.run.smithi012.stdout: "evex": 1025, 2024-09-11T00:32:30.255 INFO:teuthology.orchestra.run.smithi012.stdout: "evtrm": 1, 2024-09-11T00:32:30.255 INFO:teuthology.orchestra.run.smithi012.stdout: "ev": 98583, 2024-09-11T00:32:30.255 INFO:teuthology.orchestra.run.smithi012.stdout: "evexg": 0, 2024-09-11T00:32:30.255 INFO:teuthology.orchestra.run.smithi012.stdout: "evexd": 1024, 2024-09-11T00:32:30.255 INFO:teuthology.orchestra.run.smithi012.stdout: "segadd": 130, 2024-09-11T00:32:30.255 INFO:teuthology.orchestra.run.smithi012.stdout: "segex": 2, 2024-09-11T00:32:30.255 INFO:teuthology.orchestra.run.smithi012.stdout: "segtrm": 1, 2024-09-11T00:32:30.255 INFO:teuthology.orchestra.run.smithi012.stdout: "seg": 129, 2024-09-11T00:32:30.255 INFO:teuthology.orchestra.run.smithi012.stdout: "segmjr": 9, 2024-09-11T00:32:30.255 INFO:teuthology.orchestra.run.smithi012.stdout: "segexg": 0, 2024-09-11T00:32:30.255 INFO:teuthology.orchestra.run.smithi012.stdout: "segexd": 1, 2024-09-11T00:32:30.256 INFO:teuthology.orchestra.run.smithi012.stdout: "expos": 4194352, 2024-09-11T00:32:30.256 INFO:teuthology.orchestra.run.smithi012.stdout: "wrpos": 179539983, 2024-09-11T00:32:30.256 INFO:teuthology.orchestra.run.smithi012.stdout: "rdpos": 0, 2024-09-11T00:32:30.256 INFO:teuthology.orchestra.run.smithi012.stdout: "jlat": { 2024-09-11T00:32:30.256 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 29750, 2024-09-11T00:32:30.256 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 303.167243433, 2024-09-11T00:32:30.256 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.010190495 2024-09-11T00:32:30.256 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:30.256 INFO:teuthology.orchestra.run.smithi012.stdout: "replayed": 0 2024-09-11T00:32:30.256 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:30.256 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_mem": { 2024-09-11T00:32:30.256 INFO:teuthology.orchestra.run.smithi012.stdout: "ino": 29, 2024-09-11T00:32:30.256 INFO:teuthology.orchestra.run.smithi012.stdout: "ino+": 20545, 2024-09-11T00:32:30.257 INFO:teuthology.orchestra.run.smithi012.stdout: "ino-": 20516, 2024-09-11T00:32:30.257 INFO:teuthology.orchestra.run.smithi012.stdout: "dir": 28, 2024-09-11T00:32:30.257 INFO:teuthology.orchestra.run.smithi012.stdout: "dir+": 153, 2024-09-11T00:32:30.257 INFO:teuthology.orchestra.run.smithi012.stdout: "dir-": 125, 2024-09-11T00:32:30.257 INFO:teuthology.orchestra.run.smithi012.stdout: "dn": 26, 2024-09-11T00:32:30.257 INFO:teuthology.orchestra.run.smithi012.stdout: "dn+": 36437, 2024-09-11T00:32:30.257 INFO:teuthology.orchestra.run.smithi012.stdout: "dn-": 36411, 2024-09-11T00:32:30.257 INFO:teuthology.orchestra.run.smithi012.stdout: "cap": 0, 2024-09-11T00:32:30.257 INFO:teuthology.orchestra.run.smithi012.stdout: "cap+": 22378, 2024-09-11T00:32:30.257 INFO:teuthology.orchestra.run.smithi012.stdout: "cap-": 22378, 2024-09-11T00:32:30.257 INFO:teuthology.orchestra.run.smithi012.stdout: "rss": 70440, 2024-09-11T00:32:30.257 INFO:teuthology.orchestra.run.smithi012.stdout: "heap": 281923584 2024-09-11T00:32:30.257 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:30.257 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_server": { 2024-09-11T00:32:30.258 INFO:teuthology.orchestra.run.smithi012.stdout: "dispatch_client_request": 174620, 2024-09-11T00:32:30.258 INFO:teuthology.orchestra.run.smithi012.stdout: "dispatch_server_request": 2538, 2024-09-11T00:32:30.258 INFO:teuthology.orchestra.run.smithi012.stdout: "handle_client_request": 119550, 2024-09-11T00:32:30.258 INFO:teuthology.orchestra.run.smithi012.stdout: "handle_client_session": 1796, 2024-09-11T00:32:30.258 INFO:teuthology.orchestra.run.smithi012.stdout: "handle_peer_request": 14012, 2024-09-11T00:32:30.258 INFO:teuthology.orchestra.run.smithi012.stdout: "req_create_latency": { 2024-09-11T00:32:30.258 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 20014, 2024-09-11T00:32:30.258 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 365.196017222, 2024-09-11T00:32:30.258 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.018247027 2024-09-11T00:32:30.258 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:30.258 INFO:teuthology.orchestra.run.smithi012.stdout: "req_getattr_latency": { 2024-09-11T00:32:30.258 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 4859, 2024-09-11T00:32:30.258 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 88.229756699, 2024-09-11T00:32:30.258 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.018158007 2024-09-11T00:32:30.259 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:30.259 INFO:teuthology.orchestra.run.smithi012.stdout: "req_getfilelock_latency": { 2024-09-11T00:32:30.259 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 0, 2024-09-11T00:32:30.259 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.000000000, 2024-09-11T00:32:30.259 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000000000 2024-09-11T00:32:30.259 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:30.259 INFO:teuthology.orchestra.run.smithi012.stdout: "req_link_latency": { 2024-09-11T00:32:30.259 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 0, 2024-09-11T00:32:30.259 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.000000000, 2024-09-11T00:32:30.259 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000000000 2024-09-11T00:32:30.259 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:30.259 INFO:teuthology.orchestra.run.smithi012.stdout: "req_lookup_latency": { 2024-09-11T00:32:30.259 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 28281, 2024-09-11T00:32:30.259 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 66.194025668, 2024-09-11T00:32:30.259 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.002340582 2024-09-11T00:32:30.260 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:30.260 INFO:teuthology.orchestra.run.smithi012.stdout: "req_lookuphash_latency": { 2024-09-11T00:32:30.260 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 0, 2024-09-11T00:32:30.260 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.000000000, 2024-09-11T00:32:30.260 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000000000 2024-09-11T00:32:30.260 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:30.260 INFO:teuthology.orchestra.run.smithi012.stdout: "req_lookupino_latency": { 2024-09-11T00:32:30.260 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 0, 2024-09-11T00:32:30.260 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.000000000, 2024-09-11T00:32:30.260 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000000000 2024-09-11T00:32:30.260 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:30.260 INFO:teuthology.orchestra.run.smithi012.stdout: "req_lookupname_latency": { 2024-09-11T00:32:30.260 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 0, 2024-09-11T00:32:30.260 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.000000000, 2024-09-11T00:32:30.261 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000000000 2024-09-11T00:32:30.261 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:30.261 INFO:teuthology.orchestra.run.smithi012.stdout: "req_lookupparent_latency": { 2024-09-11T00:32:30.261 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 0, 2024-09-11T00:32:30.261 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.000000000, 2024-09-11T00:32:30.261 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000000000 2024-09-11T00:32:30.261 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:30.261 INFO:teuthology.orchestra.run.smithi012.stdout: "req_lookupsnap_latency": { 2024-09-11T00:32:30.261 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 0, 2024-09-11T00:32:30.261 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.000000000, 2024-09-11T00:32:30.261 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000000000 2024-09-11T00:32:30.261 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:30.261 INFO:teuthology.orchestra.run.smithi012.stdout: "req_lssnap_latency": { 2024-09-11T00:32:30.261 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 0, 2024-09-11T00:32:30.262 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.000000000, 2024-09-11T00:32:30.262 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000000000 2024-09-11T00:32:30.262 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:30.262 INFO:teuthology.orchestra.run.smithi012.stdout: "req_mkdir_latency": { 2024-09-11T00:32:30.262 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 0, 2024-09-11T00:32:30.262 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.000000000, 2024-09-11T00:32:30.262 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000000000 2024-09-11T00:32:30.262 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:30.262 INFO:teuthology.orchestra.run.smithi012.stdout: "req_mknod_latency": { 2024-09-11T00:32:30.262 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 0, 2024-09-11T00:32:30.262 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.000000000, 2024-09-11T00:32:30.262 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000000000 2024-09-11T00:32:30.262 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:30.263 INFO:teuthology.orchestra.run.smithi012.stdout: "req_mksnap_latency": { 2024-09-11T00:32:30.263 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 0, 2024-09-11T00:32:30.263 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.000000000, 2024-09-11T00:32:30.263 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000000000 2024-09-11T00:32:30.263 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:30.263 INFO:teuthology.orchestra.run.smithi012.stdout: "req_open_latency": { 2024-09-11T00:32:30.263 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 11, 2024-09-11T00:32:30.263 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.012407057, 2024-09-11T00:32:30.263 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.001127914 2024-09-11T00:32:30.263 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:30.263 INFO:teuthology.orchestra.run.smithi012.stdout: "req_readdir_latency": { 2024-09-11T00:32:30.263 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 34318, 2024-09-11T00:32:30.263 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 159.252913238, 2024-09-11T00:32:30.263 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.004640506 2024-09-11T00:32:30.263 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:30.264 INFO:teuthology.orchestra.run.smithi012.stdout: "req_rename_latency": { 2024-09-11T00:32:30.264 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 2922, 2024-09-11T00:32:30.264 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 50.651621929, 2024-09-11T00:32:30.264 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.017334572 2024-09-11T00:32:30.264 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:30.264 INFO:teuthology.orchestra.run.smithi012.stdout: "req_renamesnap_latency": { 2024-09-11T00:32:30.264 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 0, 2024-09-11T00:32:30.264 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.000000000, 2024-09-11T00:32:30.264 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000000000 2024-09-11T00:32:30.264 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:30.264 INFO:teuthology.orchestra.run.smithi012.stdout: "req_snapdiff_latency": { 2024-09-11T00:32:30.264 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 0, 2024-09-11T00:32:30.264 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.000000000, 2024-09-11T00:32:30.264 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000000000 2024-09-11T00:32:30.265 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:30.265 INFO:teuthology.orchestra.run.smithi012.stdout: "req_rmdir_latency": { 2024-09-11T00:32:30.265 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 20, 2024-09-11T00:32:30.265 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.262090935, 2024-09-11T00:32:30.265 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.013104546 2024-09-11T00:32:30.265 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:30.265 INFO:teuthology.orchestra.run.smithi012.stdout: "req_rmsnap_latency": { 2024-09-11T00:32:30.265 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 0, 2024-09-11T00:32:30.265 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.000000000, 2024-09-11T00:32:30.265 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000000000 2024-09-11T00:32:30.265 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:30.265 INFO:teuthology.orchestra.run.smithi012.stdout: "req_rmxattr_latency": { 2024-09-11T00:32:30.266 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 0, 2024-09-11T00:32:30.266 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.000000000, 2024-09-11T00:32:30.266 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000000000 2024-09-11T00:32:30.266 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:30.266 INFO:teuthology.orchestra.run.smithi012.stdout: "req_setattr_latency": { 2024-09-11T00:32:30.266 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 52, 2024-09-11T00:32:30.266 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 24.577636057, 2024-09-11T00:32:30.266 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.472646847 2024-09-11T00:32:30.266 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:30.266 INFO:teuthology.orchestra.run.smithi012.stdout: "req_setdirlayout_latency": { 2024-09-11T00:32:30.266 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 0, 2024-09-11T00:32:30.266 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.000000000, 2024-09-11T00:32:30.266 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000000000 2024-09-11T00:32:30.266 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:30.267 INFO:teuthology.orchestra.run.smithi012.stdout: "req_setfilelock_latency": { 2024-09-11T00:32:30.267 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 436, 2024-09-11T00:32:30.267 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.772826560, 2024-09-11T00:32:30.267 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.001772537 2024-09-11T00:32:30.267 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:30.267 INFO:teuthology.orchestra.run.smithi012.stdout: "req_setlayout_latency": { 2024-09-11T00:32:30.267 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 0, 2024-09-11T00:32:30.267 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.000000000, 2024-09-11T00:32:30.267 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000000000 2024-09-11T00:32:30.267 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:30.267 INFO:teuthology.orchestra.run.smithi012.stdout: "req_setxattr_latency": { 2024-09-11T00:32:30.267 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 0, 2024-09-11T00:32:30.267 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.000000000, 2024-09-11T00:32:30.267 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000000000 2024-09-11T00:32:30.268 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:30.268 INFO:teuthology.orchestra.run.smithi012.stdout: "req_symlink_latency": { 2024-09-11T00:32:30.268 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 0, 2024-09-11T00:32:30.268 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.000000000, 2024-09-11T00:32:30.268 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000000000 2024-09-11T00:32:30.268 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:30.268 INFO:teuthology.orchestra.run.smithi012.stdout: "req_unlink_latency": { 2024-09-11T00:32:30.268 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 20145, 2024-09-11T00:32:30.268 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 205.798020079, 2024-09-11T00:32:30.268 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.010215836 2024-09-11T00:32:30.268 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:30.268 INFO:teuthology.orchestra.run.smithi012.stdout: "cap_revoke_eviction": 0, 2024-09-11T00:32:30.269 INFO:teuthology.orchestra.run.smithi012.stdout: "cap_acquisition_throttle": 0, 2024-09-11T00:32:30.269 INFO:teuthology.orchestra.run.smithi012.stdout: "req_getvxattr_latency": { 2024-09-11T00:32:30.269 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 0, 2024-09-11T00:32:30.269 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.000000000, 2024-09-11T00:32:30.269 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000000000 2024-09-11T00:32:30.269 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:32:30.269 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:30.269 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_sessions": { 2024-09-11T00:32:30.269 INFO:teuthology.orchestra.run.smithi012.stdout: "session_count": 1, 2024-09-11T00:32:30.269 INFO:teuthology.orchestra.run.smithi012.stdout: "session_add": 1, 2024-09-11T00:32:30.269 INFO:teuthology.orchestra.run.smithi012.stdout: "session_remove": 0, 2024-09-11T00:32:30.269 INFO:teuthology.orchestra.run.smithi012.stdout: "sessions_open": 1, 2024-09-11T00:32:30.269 INFO:teuthology.orchestra.run.smithi012.stdout: "sessions_stale": 0, 2024-09-11T00:32:30.269 INFO:teuthology.orchestra.run.smithi012.stdout: "total_load": 8880, 2024-09-11T00:32:30.270 INFO:teuthology.orchestra.run.smithi012.stdout: "average_load": 8880, 2024-09-11T00:32:30.270 INFO:teuthology.orchestra.run.smithi012.stdout: "avg_session_uptime": 1458, 2024-09-11T00:32:30.270 INFO:teuthology.orchestra.run.smithi012.stdout: "mdthresh_evicted": 0 2024-09-11T00:32:30.270 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:30.270 INFO:teuthology.orchestra.run.smithi012.stdout: "mempool": { 2024-09-11T00:32:30.270 INFO:teuthology.orchestra.run.smithi012.stdout: "bloom_filter_bytes": 0, 2024-09-11T00:32:30.270 INFO:teuthology.orchestra.run.smithi012.stdout: "bloom_filter_items": 0, 2024-09-11T00:32:30.270 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_alloc_bytes": 0, 2024-09-11T00:32:30.270 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_alloc_items": 0, 2024-09-11T00:32:30.270 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_cache_data_bytes": 0, 2024-09-11T00:32:30.270 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_cache_data_items": 0, 2024-09-11T00:32:30.270 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-11T00:32:30.270 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_cache_onode_items": 0, 2024-09-11T00:32:30.271 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-11T00:32:30.271 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_cache_meta_items": 0, 2024-09-11T00:32:30.271 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_cache_other_bytes": 0, 2024-09-11T00:32:30.271 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_cache_other_items": 0, 2024-09-11T00:32:30.271 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-11T00:32:30.271 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_cache_buffer_items": 0, 2024-09-11T00:32:30.271 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_extent_bytes": 0, 2024-09-11T00:32:30.271 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_extent_items": 0, 2024-09-11T00:32:30.271 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_blob_bytes": 0, 2024-09-11T00:32:30.271 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_blob_items": 0, 2024-09-11T00:32:30.271 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-11T00:32:30.271 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_shared_blob_items": 0, 2024-09-11T00:32:30.271 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-11T00:32:30.271 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_inline_bl_items": 0, 2024-09-11T00:32:30.272 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_fsck_bytes": 0, 2024-09-11T00:32:30.272 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_fsck_items": 0, 2024-09-11T00:32:30.272 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_txc_bytes": 0, 2024-09-11T00:32:30.272 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_txc_items": 0, 2024-09-11T00:32:30.272 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-11T00:32:30.272 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_writing_deferred_items": 0, 2024-09-11T00:32:30.272 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_writing_bytes": 0, 2024-09-11T00:32:30.272 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_writing_items": 0, 2024-09-11T00:32:30.272 INFO:teuthology.orchestra.run.smithi012.stdout: "bluefs_bytes": 0, 2024-09-11T00:32:30.272 INFO:teuthology.orchestra.run.smithi012.stdout: "bluefs_items": 0, 2024-09-11T00:32:30.272 INFO:teuthology.orchestra.run.smithi012.stdout: "bluefs_file_reader_bytes": 0, 2024-09-11T00:32:30.272 INFO:teuthology.orchestra.run.smithi012.stdout: "bluefs_file_reader_items": 0, 2024-09-11T00:32:30.272 INFO:teuthology.orchestra.run.smithi012.stdout: "bluefs_file_writer_bytes": 0, 2024-09-11T00:32:30.273 INFO:teuthology.orchestra.run.smithi012.stdout: "bluefs_file_writer_items": 0, 2024-09-11T00:32:30.273 INFO:teuthology.orchestra.run.smithi012.stdout: "buffer_anon_bytes": 297880, 2024-09-11T00:32:30.273 INFO:teuthology.orchestra.run.smithi012.stdout: "buffer_anon_items": 116, 2024-09-11T00:32:30.273 INFO:teuthology.orchestra.run.smithi012.stdout: "buffer_meta_bytes": 352, 2024-09-11T00:32:30.273 INFO:teuthology.orchestra.run.smithi012.stdout: "buffer_meta_items": 4, 2024-09-11T00:32:30.273 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_bytes": 0, 2024-09-11T00:32:30.273 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_items": 0, 2024-09-11T00:32:30.273 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_mapbl_bytes": 0, 2024-09-11T00:32:30.273 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_mapbl_items": 0, 2024-09-11T00:32:30.273 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_pglog_bytes": 0, 2024-09-11T00:32:30.273 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_pglog_items": 0, 2024-09-11T00:32:30.273 INFO:teuthology.orchestra.run.smithi012.stdout: "osdmap_bytes": 9048, 2024-09-11T00:32:30.273 INFO:teuthology.orchestra.run.smithi012.stdout: "osdmap_items": 170, 2024-09-11T00:32:30.274 INFO:teuthology.orchestra.run.smithi012.stdout: "osdmap_mapping_bytes": 0, 2024-09-11T00:32:30.274 INFO:teuthology.orchestra.run.smithi012.stdout: "osdmap_mapping_items": 0, 2024-09-11T00:32:30.274 INFO:teuthology.orchestra.run.smithi012.stdout: "pgmap_bytes": 0, 2024-09-11T00:32:30.274 INFO:teuthology.orchestra.run.smithi012.stdout: "pgmap_items": 0, 2024-09-11T00:32:30.274 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_co_bytes": 122555, 2024-09-11T00:32:30.274 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_co_items": 850, 2024-09-11T00:32:30.274 INFO:teuthology.orchestra.run.smithi012.stdout: "unittest_1_bytes": 0, 2024-09-11T00:32:30.274 INFO:teuthology.orchestra.run.smithi012.stdout: "unittest_1_items": 0, 2024-09-11T00:32:30.274 INFO:teuthology.orchestra.run.smithi012.stdout: "unittest_2_bytes": 0, 2024-09-11T00:32:30.274 INFO:teuthology.orchestra.run.smithi012.stdout: "unittest_2_items": 0 2024-09-11T00:32:30.274 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:30.274 INFO:teuthology.orchestra.run.smithi012.stdout: "objecter": { 2024-09-11T00:32:30.274 INFO:teuthology.orchestra.run.smithi012.stdout: "op_active": 0, 2024-09-11T00:32:30.275 INFO:teuthology.orchestra.run.smithi012.stdout: "op_laggy": 0, 2024-09-11T00:32:30.275 INFO:teuthology.orchestra.run.smithi012.stdout: "op_send": 73180, 2024-09-11T00:32:30.275 INFO:teuthology.orchestra.run.smithi012.stdout: "op_send_bytes": 179608327, 2024-09-11T00:32:30.275 INFO:teuthology.orchestra.run.smithi012.stdout: "op_resend": 32, 2024-09-11T00:32:30.275 INFO:teuthology.orchestra.run.smithi012.stdout: "op_reply": 73180, 2024-09-11T00:32:30.275 INFO:teuthology.orchestra.run.smithi012.stdout: "op_latency": { 2024-09-11T00:32:30.275 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 73180, 2024-09-11T00:32:30.275 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 833.593194562, 2024-09-11T00:32:30.275 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.011390997 2024-09-11T00:32:30.275 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:30.275 INFO:teuthology.orchestra.run.smithi012.stdout: "op_inflight": 0, 2024-09-11T00:32:30.275 INFO:teuthology.orchestra.run.smithi012.stdout: "oplen_avg": { 2024-09-11T00:32:30.275 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 73180, 2024-09-11T00:32:30.276 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 74408 2024-09-11T00:32:30.276 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:30.276 INFO:teuthology.orchestra.run.smithi012.stdout: "op": 73180, 2024-09-11T00:32:30.276 INFO:teuthology.orchestra.run.smithi012.stdout: "op_r": 875, 2024-09-11T00:32:30.276 INFO:teuthology.orchestra.run.smithi012.stdout: "op_w": 72305, 2024-09-11T00:32:30.276 INFO:teuthology.orchestra.run.smithi012.stdout: "op_rmw": 0, 2024-09-11T00:32:30.276 INFO:teuthology.orchestra.run.smithi012.stdout: "op_pg": 0, 2024-09-11T00:32:30.276 INFO:teuthology.orchestra.run.smithi012.stdout: "osdop_stat": 0, 2024-09-11T00:32:30.276 INFO:teuthology.orchestra.run.smithi012.stdout: "osdop_create": 12, 2024-09-11T00:32:30.276 INFO:teuthology.orchestra.run.smithi012.stdout: "osdop_read": 875, 2024-09-11T00:32:30.276 INFO:teuthology.orchestra.run.smithi012.stdout: "osdop_write": 30625, 2024-09-11T00:32:30.276 INFO:teuthology.orchestra.run.smithi012.stdout: "osdop_writefull": 522, 2024-09-11T00:32:30.276 INFO:teuthology.orchestra.run.smithi012.stdout: "osdop_writesame": 0, 2024-09-11T00:32:30.277 INFO:teuthology.orchestra.run.smithi012.stdout: "osdop_append": 0, 2024-09-11T00:32:30.277 INFO:teuthology.orchestra.run.smithi012.stdout: "osdop_zero": 0, 2024-09-11T00:32:30.277 INFO:teuthology.orchestra.run.smithi012.stdout: "osdop_truncate": 0, 2024-09-11T00:32:30.277 INFO:teuthology.orchestra.run.smithi012.stdout: "osdop_delete": 40361, 2024-09-11T00:32:30.277 INFO:teuthology.orchestra.run.smithi012.stdout: "osdop_mapext": 0, 2024-09-11T00:32:30.277 INFO:teuthology.orchestra.run.smithi012.stdout: "osdop_sparse_read": 0, 2024-09-11T00:32:30.277 INFO:teuthology.orchestra.run.smithi012.stdout: "osdop_clonerange": 0, 2024-09-11T00:32:30.277 INFO:teuthology.orchestra.run.smithi012.stdout: "osdop_getxattr": 0, 2024-09-11T00:32:30.277 INFO:teuthology.orchestra.run.smithi012.stdout: "osdop_setxattr": 24, 2024-09-11T00:32:30.277 INFO:teuthology.orchestra.run.smithi012.stdout: "osdop_cmpxattr": 0, 2024-09-11T00:32:30.277 INFO:teuthology.orchestra.run.smithi012.stdout: "osdop_rmxattr": 0, 2024-09-11T00:32:30.277 INFO:teuthology.orchestra.run.smithi012.stdout: "osdop_resetxattrs": 0, 2024-09-11T00:32:30.277 INFO:teuthology.orchestra.run.smithi012.stdout: "osdop_call": 0, 2024-09-11T00:32:30.278 INFO:teuthology.orchestra.run.smithi012.stdout: "osdop_watch": 0, 2024-09-11T00:32:30.278 INFO:teuthology.orchestra.run.smithi012.stdout: "osdop_notify": 0, 2024-09-11T00:32:30.278 INFO:teuthology.orchestra.run.smithi012.stdout: "osdop_src_cmpxattr": 0, 2024-09-11T00:32:30.278 INFO:teuthology.orchestra.run.smithi012.stdout: "osdop_pgls": 0, 2024-09-11T00:32:30.278 INFO:teuthology.orchestra.run.smithi012.stdout: "osdop_pgls_filter": 0, 2024-09-11T00:32:30.278 INFO:teuthology.orchestra.run.smithi012.stdout: "osdop_other": 0, 2024-09-11T00:32:30.278 INFO:teuthology.orchestra.run.smithi012.stdout: "linger_active": 0, 2024-09-11T00:32:30.278 INFO:teuthology.orchestra.run.smithi012.stdout: "linger_send": 0, 2024-09-11T00:32:30.278 INFO:teuthology.orchestra.run.smithi012.stdout: "linger_resend": 0, 2024-09-11T00:32:30.278 INFO:teuthology.orchestra.run.smithi012.stdout: "linger_ping": 0, 2024-09-11T00:32:30.278 INFO:teuthology.orchestra.run.smithi012.stdout: "poolop_active": 0, 2024-09-11T00:32:30.278 INFO:teuthology.orchestra.run.smithi012.stdout: "poolop_send": 0, 2024-09-11T00:32:30.278 INFO:teuthology.orchestra.run.smithi012.stdout: "poolop_resend": 0, 2024-09-11T00:32:30.279 INFO:teuthology.orchestra.run.smithi012.stdout: "poolstat_active": 0, 2024-09-11T00:32:30.279 INFO:teuthology.orchestra.run.smithi012.stdout: "poolstat_send": 0, 2024-09-11T00:32:30.279 INFO:teuthology.orchestra.run.smithi012.stdout: "poolstat_resend": 0, 2024-09-11T00:32:30.279 INFO:teuthology.orchestra.run.smithi012.stdout: "statfs_active": 0, 2024-09-11T00:32:30.279 INFO:teuthology.orchestra.run.smithi012.stdout: "statfs_send": 0, 2024-09-11T00:32:30.279 INFO:teuthology.orchestra.run.smithi012.stdout: "statfs_resend": 0, 2024-09-11T00:32:30.279 INFO:teuthology.orchestra.run.smithi012.stdout: "command_active": 0, 2024-09-11T00:32:30.279 INFO:teuthology.orchestra.run.smithi012.stdout: "command_send": 0, 2024-09-11T00:32:30.279 INFO:teuthology.orchestra.run.smithi012.stdout: "command_resend": 0, 2024-09-11T00:32:30.279 INFO:teuthology.orchestra.run.smithi012.stdout: "map_epoch": 87, 2024-09-11T00:32:30.279 INFO:teuthology.orchestra.run.smithi012.stdout: "map_full": 0, 2024-09-11T00:32:30.279 INFO:teuthology.orchestra.run.smithi012.stdout: "map_inc": 6, 2024-09-11T00:32:30.279 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_sessions": 12, 2024-09-11T00:32:30.279 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_session_open": 12, 2024-09-11T00:32:30.280 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_session_close": 0, 2024-09-11T00:32:30.280 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_laggy": 0, 2024-09-11T00:32:30.280 INFO:teuthology.orchestra.run.smithi012.stdout: "omap_wr": 1433, 2024-09-11T00:32:30.280 INFO:teuthology.orchestra.run.smithi012.stdout: "omap_rd": 0, 2024-09-11T00:32:30.280 INFO:teuthology.orchestra.run.smithi012.stdout: "omap_del": 556 2024-09-11T00:32:30.280 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:30.280 INFO:teuthology.orchestra.run.smithi012.stdout: "oft": { 2024-09-11T00:32:30.280 INFO:teuthology.orchestra.run.smithi012.stdout: "omap_total_objs": 1, 2024-09-11T00:32:30.280 INFO:teuthology.orchestra.run.smithi012.stdout: "omap_total_kv_pairs": 0, 2024-09-11T00:32:30.280 INFO:teuthology.orchestra.run.smithi012.stdout: "omap_total_updates": 0, 2024-09-11T00:32:30.280 INFO:teuthology.orchestra.run.smithi012.stdout: "omap_total_removes": 0 2024-09-11T00:32:30.280 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:30.280 INFO:teuthology.orchestra.run.smithi012.stdout: "purge_queue": { 2024-09-11T00:32:30.281 INFO:teuthology.orchestra.run.smithi012.stdout: "pq_executing_ops": 0, 2024-09-11T00:32:30.281 INFO:teuthology.orchestra.run.smithi012.stdout: "pq_executing_ops_high_water": 10, 2024-09-11T00:32:30.281 INFO:teuthology.orchestra.run.smithi012.stdout: "pq_executing": 0, 2024-09-11T00:32:30.281 INFO:teuthology.orchestra.run.smithi012.stdout: "pq_executing_high_water": 9, 2024-09-11T00:32:30.281 INFO:teuthology.orchestra.run.smithi012.stdout: "pq_executed_ops": 40308, 2024-09-11T00:32:30.281 INFO:teuthology.orchestra.run.smithi012.stdout: "pq_executed": 20163, 2024-09-11T00:32:30.281 INFO:teuthology.orchestra.run.smithi012.stdout: "pq_item_in_journal": 0 2024-09-11T00:32:30.281 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:30.281 INFO:teuthology.orchestra.run.smithi012.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-11T00:32:30.281 INFO:teuthology.orchestra.run.smithi012.stdout: "val": 0, 2024-09-11T00:32:30.281 INFO:teuthology.orchestra.run.smithi012.stdout: "max": 104857600, 2024-09-11T00:32:30.281 INFO:teuthology.orchestra.run.smithi012.stdout: "get_started": 0, 2024-09-11T00:32:30.281 INFO:teuthology.orchestra.run.smithi012.stdout: "get": 403120, 2024-09-11T00:32:30.282 INFO:teuthology.orchestra.run.smithi012.stdout: "get_sum": 86917247, 2024-09-11T00:32:30.282 INFO:teuthology.orchestra.run.smithi012.stdout: "get_or_fail_fail": 0, 2024-09-11T00:32:30.282 INFO:teuthology.orchestra.run.smithi012.stdout: "get_or_fail_success": 403120, 2024-09-11T00:32:30.282 INFO:teuthology.orchestra.run.smithi012.stdout: "take": 0, 2024-09-11T00:32:30.282 INFO:teuthology.orchestra.run.smithi012.stdout: "take_sum": 0, 2024-09-11T00:32:30.282 INFO:teuthology.orchestra.run.smithi012.stdout: "put": 403120, 2024-09-11T00:32:30.282 INFO:teuthology.orchestra.run.smithi012.stdout: "put_sum": 86917247, 2024-09-11T00:32:30.282 INFO:teuthology.orchestra.run.smithi012.stdout: "wait": { 2024-09-11T00:32:30.282 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 0, 2024-09-11T00:32:30.282 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.000000000, 2024-09-11T00:32:30.282 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000000000 2024-09-11T00:32:30.282 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:32:30.282 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:30.282 INFO:teuthology.orchestra.run.smithi012.stdout: "throttle-objecter_bytes": { 2024-09-11T00:32:30.283 INFO:teuthology.orchestra.run.smithi012.stdout: "val": 0, 2024-09-11T00:32:30.283 INFO:teuthology.orchestra.run.smithi012.stdout: "max": 104857600, 2024-09-11T00:32:30.283 INFO:teuthology.orchestra.run.smithi012.stdout: "get_started": 0, 2024-09-11T00:32:30.283 INFO:teuthology.orchestra.run.smithi012.stdout: "get": 0, 2024-09-11T00:32:30.283 INFO:teuthology.orchestra.run.smithi012.stdout: "get_sum": 0, 2024-09-11T00:32:30.283 INFO:teuthology.orchestra.run.smithi012.stdout: "get_or_fail_fail": 0, 2024-09-11T00:32:30.283 INFO:teuthology.orchestra.run.smithi012.stdout: "get_or_fail_success": 0, 2024-09-11T00:32:30.283 INFO:teuthology.orchestra.run.smithi012.stdout: "take": 73180, 2024-09-11T00:32:30.283 INFO:teuthology.orchestra.run.smithi012.stdout: "take_sum": 181805950, 2024-09-11T00:32:30.283 INFO:teuthology.orchestra.run.smithi012.stdout: "put": 73180, 2024-09-11T00:32:30.283 INFO:teuthology.orchestra.run.smithi012.stdout: "put_sum": 181805950, 2024-09-11T00:32:30.283 INFO:teuthology.orchestra.run.smithi012.stdout: "wait": { 2024-09-11T00:32:30.283 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 0, 2024-09-11T00:32:30.284 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.000000000, 2024-09-11T00:32:30.284 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000000000 2024-09-11T00:32:30.284 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:32:30.284 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:30.284 INFO:teuthology.orchestra.run.smithi012.stdout: "throttle-objecter_ops": { 2024-09-11T00:32:30.284 INFO:teuthology.orchestra.run.smithi012.stdout: "val": 0, 2024-09-11T00:32:30.284 INFO:teuthology.orchestra.run.smithi012.stdout: "max": 1024, 2024-09-11T00:32:30.284 INFO:teuthology.orchestra.run.smithi012.stdout: "get_started": 0, 2024-09-11T00:32:30.284 INFO:teuthology.orchestra.run.smithi012.stdout: "get": 0, 2024-09-11T00:32:30.284 INFO:teuthology.orchestra.run.smithi012.stdout: "get_sum": 0, 2024-09-11T00:32:30.284 INFO:teuthology.orchestra.run.smithi012.stdout: "get_or_fail_fail": 0, 2024-09-11T00:32:30.284 INFO:teuthology.orchestra.run.smithi012.stdout: "get_or_fail_success": 0, 2024-09-11T00:32:30.284 INFO:teuthology.orchestra.run.smithi012.stdout: "take": 73180, 2024-09-11T00:32:30.284 INFO:teuthology.orchestra.run.smithi012.stdout: "take_sum": 73180, 2024-09-11T00:32:30.284 INFO:teuthology.orchestra.run.smithi012.stdout: "put": 73180, 2024-09-11T00:32:30.285 INFO:teuthology.orchestra.run.smithi012.stdout: "put_sum": 73180, 2024-09-11T00:32:30.285 INFO:teuthology.orchestra.run.smithi012.stdout: "wait": { 2024-09-11T00:32:30.285 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 0, 2024-09-11T00:32:30.285 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.000000000, 2024-09-11T00:32:30.285 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000000000 2024-09-11T00:32:30.285 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:32:30.285 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:30.285 INFO:teuthology.orchestra.run.smithi012.stdout: "throttle-write_buf_throttle": { 2024-09-11T00:32:30.285 INFO:teuthology.orchestra.run.smithi012.stdout: "val": 0, 2024-09-11T00:32:30.285 INFO:teuthology.orchestra.run.smithi012.stdout: "max": 3758096384, 2024-09-11T00:32:30.285 INFO:teuthology.orchestra.run.smithi012.stdout: "get_started": 0, 2024-09-11T00:32:30.285 INFO:teuthology.orchestra.run.smithi012.stdout: "get": 20163, 2024-09-11T00:32:30.285 INFO:teuthology.orchestra.run.smithi012.stdout: "get_sum": 2197623, 2024-09-11T00:32:30.285 INFO:teuthology.orchestra.run.smithi012.stdout: "get_or_fail_fail": 0, 2024-09-11T00:32:30.286 INFO:teuthology.orchestra.run.smithi012.stdout: "get_or_fail_success": 20163, 2024-09-11T00:32:30.286 INFO:teuthology.orchestra.run.smithi012.stdout: "take": 0, 2024-09-11T00:32:30.286 INFO:teuthology.orchestra.run.smithi012.stdout: "take_sum": 0, 2024-09-11T00:32:30.286 INFO:teuthology.orchestra.run.smithi012.stdout: "put": 875, 2024-09-11T00:32:30.286 INFO:teuthology.orchestra.run.smithi012.stdout: "put_sum": 2197623, 2024-09-11T00:32:30.286 INFO:teuthology.orchestra.run.smithi012.stdout: "wait": { 2024-09-11T00:32:30.286 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 0, 2024-09-11T00:32:30.286 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.000000000, 2024-09-11T00:32:30.286 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000000000 2024-09-11T00:32:30.286 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:32:30.286 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:30.286 INFO:teuthology.orchestra.run.smithi012.stdout: "throttle-write_buf_throttle-0x560d1057adc0": { 2024-09-11T00:32:30.286 INFO:teuthology.orchestra.run.smithi012.stdout: "val": 0, 2024-09-11T00:32:30.286 INFO:teuthology.orchestra.run.smithi012.stdout: "max": 3758096384, 2024-09-11T00:32:30.286 INFO:teuthology.orchestra.run.smithi012.stdout: "get_started": 0, 2024-09-11T00:32:30.287 INFO:teuthology.orchestra.run.smithi012.stdout: "get": 98584, 2024-09-11T00:32:30.287 INFO:teuthology.orchestra.run.smithi012.stdout: "get_sum": 175345679, 2024-09-11T00:32:30.287 INFO:teuthology.orchestra.run.smithi012.stdout: "get_or_fail_fail": 0, 2024-09-11T00:32:30.287 INFO:teuthology.orchestra.run.smithi012.stdout: "get_or_fail_success": 98584, 2024-09-11T00:32:30.287 INFO:teuthology.orchestra.run.smithi012.stdout: "take": 0, 2024-09-11T00:32:30.287 INFO:teuthology.orchestra.run.smithi012.stdout: "take_sum": 0, 2024-09-11T00:32:30.287 INFO:teuthology.orchestra.run.smithi012.stdout: "put": 29750, 2024-09-11T00:32:30.287 INFO:teuthology.orchestra.run.smithi012.stdout: "put_sum": 175345679, 2024-09-11T00:32:30.287 INFO:teuthology.orchestra.run.smithi012.stdout: "wait": { 2024-09-11T00:32:30.287 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 0, 2024-09-11T00:32:30.287 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.000000000, 2024-09-11T00:32:30.287 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000000000 2024-09-11T00:32:30.287 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:32:30.287 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:30.288 INFO:teuthology.orchestra.run.smithi012.stdout: "trackedop": { 2024-09-11T00:32:30.288 INFO:teuthology.orchestra.run.smithi012.stdout: "slow_ops_count": 1949 2024-09-11T00:32:30.288 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:32:30.288 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-09-11T00:32:30.288 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:32:29 smithi012 ceph-mon[30333]: pgmap v1162: 129 pgs: 129 active+clean; 671 MiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 40 KiB/s wr, 44 op/s 2024-09-11T00:32:30.288 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:32:29 smithi165 ceph-mon[35356]: pgmap v1162: 129 pgs: 129 active+clean; 671 MiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 40 KiB/s wr, 44 op/s 2024-09-11T00:32:30.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:32:29 smithi178 ceph-mon[35919]: pgmap v1162: 129 pgs: 129 active+clean; 671 MiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 40 KiB/s wr, 44 op/s 2024-09-11T00:32:31.215 INFO:tasks.check_counter:Daemon mds.g mds.exported=20 2024-09-11T00:32:31.215 INFO:tasks.check_counter:Daemon mds.g mds.imported=20 2024-09-11T00:32:31.216 DEBUG:tasks.check_counter:Getting stats from j 2024-09-11T00:32:31.216 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid ff3ad74a-6fcf-11ef-bcea-c7b262605968 -- ceph daemon mds.j perf dump 2024-09-11T00:32:31.595 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:32:32.268 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:32:32 smithi012 ceph-mon[30333]: pgmap v1163: 129 pgs: 129 active+clean; 671 MiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 18 KiB/s wr, 33 op/s 2024-09-11T00:32:32.270 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-09-11T00:32:32.270 INFO:teuthology.orchestra.run.smithi012.stdout: "AsyncMessenger::Worker-0": { 2024-09-11T00:32:32.270 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_recv_messages": 469, 2024-09-11T00:32:32.270 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_send_messages": 464, 2024-09-11T00:32:32.270 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_recv_bytes": 597824, 2024-09-11T00:32:32.270 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_send_bytes": 286800, 2024-09-11T00:32:32.270 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_created_connections": 4, 2024-09-11T00:32:32.270 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_active_connections": 0, 2024-09-11T00:32:32.270 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_running_total_time": 0.095105579, 2024-09-11T00:32:32.270 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_running_send_time": 0.053268151, 2024-09-11T00:32:32.270 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_running_recv_time": 0.045424751, 2024-09-11T00:32:32.270 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-11T00:32:32.271 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_send_messages_queue_lat": { 2024-09-11T00:32:32.271 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 464, 2024-09-11T00:32:32.271 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.046324269, 2024-09-11T00:32:32.271 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000099836 2024-09-11T00:32:32.271 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:32.271 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_handle_ack_lat": { 2024-09-11T00:32:32.271 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 0, 2024-09-11T00:32:32.271 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.000000000, 2024-09-11T00:32:32.271 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000000000 2024-09-11T00:32:32.271 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:32.271 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_recv_encrypted_bytes": 597824, 2024-09-11T00:32:32.271 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_send_encrypted_bytes": 286800 2024-09-11T00:32:32.272 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:32.272 INFO:teuthology.orchestra.run.smithi012.stdout: "AsyncMessenger::Worker-1": { 2024-09-11T00:32:32.272 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_recv_messages": 1, 2024-09-11T00:32:32.272 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_send_messages": 353, 2024-09-11T00:32:32.272 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_recv_bytes": 160, 2024-09-11T00:32:32.272 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_send_bytes": 142320, 2024-09-11T00:32:32.272 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_created_connections": 3, 2024-09-11T00:32:32.272 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_active_connections": 2, 2024-09-11T00:32:32.272 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_running_total_time": 0.044621583, 2024-09-11T00:32:32.272 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_running_send_time": 0.038485850, 2024-09-11T00:32:32.272 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_running_recv_time": 0.001649144, 2024-09-11T00:32:32.272 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-11T00:32:32.272 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_send_messages_queue_lat": { 2024-09-11T00:32:32.272 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 351, 2024-09-11T00:32:32.273 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.025888171, 2024-09-11T00:32:32.273 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000073755 2024-09-11T00:32:32.273 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:32.273 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_handle_ack_lat": { 2024-09-11T00:32:32.273 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 0, 2024-09-11T00:32:32.273 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.000000000, 2024-09-11T00:32:32.273 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000000000 2024-09-11T00:32:32.273 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:32.273 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_recv_encrypted_bytes": 160, 2024-09-11T00:32:32.273 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_send_encrypted_bytes": 142320 2024-09-11T00:32:32.273 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:32.273 INFO:teuthology.orchestra.run.smithi012.stdout: "AsyncMessenger::Worker-2": { 2024-09-11T00:32:32.273 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_recv_messages": 0, 2024-09-11T00:32:32.273 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_send_messages": 2, 2024-09-11T00:32:32.274 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_recv_bytes": 0, 2024-09-11T00:32:32.274 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_send_bytes": 0, 2024-09-11T00:32:32.274 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_created_connections": 2, 2024-09-11T00:32:32.274 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_active_connections": 1, 2024-09-11T00:32:32.274 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_running_total_time": 0.000680009, 2024-09-11T00:32:32.274 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_running_send_time": 0.000000000, 2024-09-11T00:32:32.274 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_running_recv_time": 0.000547076, 2024-09-11T00:32:32.274 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-11T00:32:32.274 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_send_messages_queue_lat": { 2024-09-11T00:32:32.274 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 0, 2024-09-11T00:32:32.274 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.000000000, 2024-09-11T00:32:32.274 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000000000 2024-09-11T00:32:32.274 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:32.274 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_handle_ack_lat": { 2024-09-11T00:32:32.275 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 0, 2024-09-11T00:32:32.275 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.000000000, 2024-09-11T00:32:32.275 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000000000 2024-09-11T00:32:32.275 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:32.275 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-11T00:32:32.275 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_send_encrypted_bytes": 0 2024-09-11T00:32:32.275 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:32.275 INFO:teuthology.orchestra.run.smithi012.stdout: "cct": { 2024-09-11T00:32:32.275 INFO:teuthology.orchestra.run.smithi012.stdout: "total_workers": 0, 2024-09-11T00:32:32.275 INFO:teuthology.orchestra.run.smithi012.stdout: "unhealthy_workers": 0 2024-09-11T00:32:32.275 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:32.275 INFO:teuthology.orchestra.run.smithi012.stdout: "mempool": { 2024-09-11T00:32:32.275 INFO:teuthology.orchestra.run.smithi012.stdout: "bloom_filter_bytes": 0, 2024-09-11T00:32:32.275 INFO:teuthology.orchestra.run.smithi012.stdout: "bloom_filter_items": 0, 2024-09-11T00:32:32.276 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_alloc_bytes": 0, 2024-09-11T00:32:32.276 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_alloc_items": 0, 2024-09-11T00:32:32.276 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_cache_data_bytes": 0, 2024-09-11T00:32:32.276 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_cache_data_items": 0, 2024-09-11T00:32:32.276 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-11T00:32:32.276 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_cache_onode_items": 0, 2024-09-11T00:32:32.276 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-11T00:32:32.276 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_cache_meta_items": 0, 2024-09-11T00:32:32.276 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_cache_other_bytes": 0, 2024-09-11T00:32:32.276 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_cache_other_items": 0, 2024-09-11T00:32:32.276 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-11T00:32:32.276 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_cache_buffer_items": 0, 2024-09-11T00:32:32.276 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_extent_bytes": 0, 2024-09-11T00:32:32.277 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_extent_items": 0, 2024-09-11T00:32:32.277 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_blob_bytes": 0, 2024-09-11T00:32:32.277 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_blob_items": 0, 2024-09-11T00:32:32.277 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-11T00:32:32.277 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_shared_blob_items": 0, 2024-09-11T00:32:32.277 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-11T00:32:32.277 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_inline_bl_items": 0, 2024-09-11T00:32:32.277 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_fsck_bytes": 0, 2024-09-11T00:32:32.277 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_fsck_items": 0, 2024-09-11T00:32:32.277 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_txc_bytes": 0, 2024-09-11T00:32:32.277 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_txc_items": 0, 2024-09-11T00:32:32.277 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-11T00:32:32.277 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_writing_deferred_items": 0, 2024-09-11T00:32:32.277 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_writing_bytes": 0, 2024-09-11T00:32:32.277 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_writing_items": 0, 2024-09-11T00:32:32.278 INFO:teuthology.orchestra.run.smithi012.stdout: "bluefs_bytes": 0, 2024-09-11T00:32:32.278 INFO:teuthology.orchestra.run.smithi012.stdout: "bluefs_items": 0, 2024-09-11T00:32:32.278 INFO:teuthology.orchestra.run.smithi012.stdout: "bluefs_file_reader_bytes": 0, 2024-09-11T00:32:32.278 INFO:teuthology.orchestra.run.smithi012.stdout: "bluefs_file_reader_items": 0, 2024-09-11T00:32:32.278 INFO:teuthology.orchestra.run.smithi012.stdout: "bluefs_file_writer_bytes": 0, 2024-09-11T00:32:32.278 INFO:teuthology.orchestra.run.smithi012.stdout: "bluefs_file_writer_items": 0, 2024-09-11T00:32:32.278 INFO:teuthology.orchestra.run.smithi012.stdout: "buffer_anon_bytes": 148967, 2024-09-11T00:32:32.278 INFO:teuthology.orchestra.run.smithi012.stdout: "buffer_anon_items": 29, 2024-09-11T00:32:32.278 INFO:teuthology.orchestra.run.smithi012.stdout: "buffer_meta_bytes": 0, 2024-09-11T00:32:32.278 INFO:teuthology.orchestra.run.smithi012.stdout: "buffer_meta_items": 0, 2024-09-11T00:32:32.278 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_bytes": 0, 2024-09-11T00:32:32.278 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_items": 0, 2024-09-11T00:32:32.278 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_mapbl_bytes": 0, 2024-09-11T00:32:32.278 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_mapbl_items": 0, 2024-09-11T00:32:32.279 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_pglog_bytes": 0, 2024-09-11T00:32:32.279 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_pglog_items": 0, 2024-09-11T00:32:32.279 INFO:teuthology.orchestra.run.smithi012.stdout: "osdmap_bytes": 0, 2024-09-11T00:32:32.279 INFO:teuthology.orchestra.run.smithi012.stdout: "osdmap_items": 0, 2024-09-11T00:32:32.279 INFO:teuthology.orchestra.run.smithi012.stdout: "osdmap_mapping_bytes": 0, 2024-09-11T00:32:32.279 INFO:teuthology.orchestra.run.smithi012.stdout: "osdmap_mapping_items": 0, 2024-09-11T00:32:32.279 INFO:teuthology.orchestra.run.smithi012.stdout: "pgmap_bytes": 0, 2024-09-11T00:32:32.279 INFO:teuthology.orchestra.run.smithi012.stdout: "pgmap_items": 0, 2024-09-11T00:32:32.279 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_co_bytes": 968, 2024-09-11T00:32:32.279 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_co_items": 2, 2024-09-11T00:32:32.279 INFO:teuthology.orchestra.run.smithi012.stdout: "unittest_1_bytes": 0, 2024-09-11T00:32:32.279 INFO:teuthology.orchestra.run.smithi012.stdout: "unittest_1_items": 0, 2024-09-11T00:32:32.279 INFO:teuthology.orchestra.run.smithi012.stdout: "unittest_2_bytes": 0, 2024-09-11T00:32:32.279 INFO:teuthology.orchestra.run.smithi012.stdout: "unittest_2_items": 0 2024-09-11T00:32:32.280 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-11T00:32:32.280 INFO:teuthology.orchestra.run.smithi012.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-11T00:32:32.280 INFO:teuthology.orchestra.run.smithi012.stdout: "val": 0, 2024-09-11T00:32:32.280 INFO:teuthology.orchestra.run.smithi012.stdout: "max": 104857600, 2024-09-11T00:32:32.280 INFO:teuthology.orchestra.run.smithi012.stdout: "get_started": 0, 2024-09-11T00:32:32.280 INFO:teuthology.orchestra.run.smithi012.stdout: "get": 466, 2024-09-11T00:32:32.280 INFO:teuthology.orchestra.run.smithi012.stdout: "get_sum": 529063, 2024-09-11T00:32:32.280 INFO:teuthology.orchestra.run.smithi012.stdout: "get_or_fail_fail": 0, 2024-09-11T00:32:32.280 INFO:teuthology.orchestra.run.smithi012.stdout: "get_or_fail_success": 466, 2024-09-11T00:32:32.280 INFO:teuthology.orchestra.run.smithi012.stdout: "take": 0, 2024-09-11T00:32:32.280 INFO:teuthology.orchestra.run.smithi012.stdout: "take_sum": 0, 2024-09-11T00:32:32.280 INFO:teuthology.orchestra.run.smithi012.stdout: "put": 466, 2024-09-11T00:32:32.280 INFO:teuthology.orchestra.run.smithi012.stdout: "put_sum": 529063, 2024-09-11T00:32:32.280 INFO:teuthology.orchestra.run.smithi012.stdout: "wait": { 2024-09-11T00:32:32.281 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 0, 2024-09-11T00:32:32.281 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.000000000, 2024-09-11T00:32:32.281 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000000000 2024-09-11T00:32:32.281 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:32:32.281 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-11T00:32:32.281 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-09-11T00:32:32.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:32:32 smithi165 ceph-mon[35356]: pgmap v1163: 129 pgs: 129 active+clean; 671 MiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 18 KiB/s wr, 33 op/s 2024-09-11T00:32:32.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:32:32 smithi178 ceph-mon[35919]: pgmap v1163: 129 pgs: 129 active+clean; 671 MiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 18 KiB/s wr, 33 op/s 2024-09-11T00:32:33.133 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.j 2024-09-11T00:32:33.133 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.j 2024-09-11T00:32:33.133 DEBUG:tasks.check_counter:Getting stats from b 2024-09-11T00:32:33.134 DEBUG:teuthology.orchestra.run.smithi165:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid ff3ad74a-6fcf-11ef-bcea-c7b262605968 -- ceph daemon mds.b perf dump 2024-09-11T00:32:33.465 INFO:teuthology.orchestra.run.smithi165.stderr:Inferring config /var/lib/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T00:32:34.179 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:32:34 smithi165 ceph-mon[35356]: pgmap v1164: 129 pgs: 129 active+clean; 671 MiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 8.8 KiB/s wr, 19 op/s 2024-09-11T00:32:34.180 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-09-11T00:32:34.180 INFO:teuthology.orchestra.run.smithi165.stdout: "AsyncMessenger::Worker-0": { 2024-09-11T00:32:34.180 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_recv_messages": 162908, 2024-09-11T00:32:34.180 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_messages": 196726, 2024-09-11T00:32:34.180 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_recv_bytes": 39996877, 2024-09-11T00:32:34.180 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_bytes": 151803935, 2024-09-11T00:32:34.180 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_created_connections": 19, 2024-09-11T00:32:34.180 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_active_connections": 3, 2024-09-11T00:32:34.180 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_total_time": 13.894845459, 2024-09-11T00:32:34.181 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_send_time": 7.137782124, 2024-09-11T00:32:34.181 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_recv_time": 6.162636346, 2024-09-11T00:32:34.181 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_fast_dispatch_time": 1.273937428, 2024-09-11T00:32:34.181 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_messages_queue_lat": { 2024-09-11T00:32:34.181 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 196730, 2024-09-11T00:32:34.181 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 20.906605960, 2024-09-11T00:32:34.181 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000106270 2024-09-11T00:32:34.181 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-11T00:32:34.181 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_handle_ack_lat": { 2024-09-11T00:32:34.181 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 274765, 2024-09-11T00:32:34.181 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.051334461, 2024-09-11T00:32:34.181 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000186 2024-09-11T00:32:34.182 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-11T00:32:34.182 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-11T00:32:34.182 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_encrypted_bytes": 0 2024-09-11T00:32:34.182 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-11T00:32:34.182 INFO:teuthology.orchestra.run.smithi165.stdout: "AsyncMessenger::Worker-1": { 2024-09-11T00:32:34.182 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_recv_messages": 123543, 2024-09-11T00:32:34.182 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_messages": 146378, 2024-09-11T00:32:34.182 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_recv_bytes": 32752226, 2024-09-11T00:32:34.182 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_bytes": 156422653, 2024-09-11T00:32:34.182 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_created_connections": 12, 2024-09-11T00:32:34.182 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_active_connections": 6, 2024-09-11T00:32:34.182 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_total_time": 11.393622780, 2024-09-11T00:32:34.182 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_send_time": 5.992679456, 2024-09-11T00:32:34.182 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_recv_time": 4.320687809, 2024-09-11T00:32:34.183 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_fast_dispatch_time": 1.359682196, 2024-09-11T00:32:34.183 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_messages_queue_lat": { 2024-09-11T00:32:34.183 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 146379, 2024-09-11T00:32:34.183 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 25.552948112, 2024-09-11T00:32:34.183 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000174567 2024-09-11T00:32:34.183 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-11T00:32:34.183 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_handle_ack_lat": { 2024-09-11T00:32:34.183 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 169691, 2024-09-11T00:32:34.183 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.029826798, 2024-09-11T00:32:34.183 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000175 2024-09-11T00:32:34.183 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-11T00:32:34.183 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_recv_encrypted_bytes": 320, 2024-09-11T00:32:34.183 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_encrypted_bytes": 810544 2024-09-11T00:32:34.184 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-11T00:32:34.184 INFO:teuthology.orchestra.run.smithi165.stdout: "AsyncMessenger::Worker-2": { 2024-09-11T00:32:34.184 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_recv_messages": 358252, 2024-09-11T00:32:34.184 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_messages": 468906, 2024-09-11T00:32:34.184 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_recv_bytes": 107258277, 2024-09-11T00:32:34.184 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_bytes": 359920469, 2024-09-11T00:32:34.184 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_created_connections": 316, 2024-09-11T00:32:34.184 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_active_connections": 11, 2024-09-11T00:32:34.184 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_total_time": 27.329162475, 2024-09-11T00:32:34.184 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_send_time": 15.559145060, 2024-09-11T00:32:34.184 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_recv_time": 12.933768269, 2024-09-11T00:32:34.184 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_fast_dispatch_time": 1.104073473, 2024-09-11T00:32:34.184 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_messages_queue_lat": { 2024-09-11T00:32:34.185 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 468897, 2024-09-11T00:32:34.185 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 40.947400827, 2024-09-11T00:32:34.185 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000087327 2024-09-11T00:32:34.185 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-11T00:32:34.185 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_handle_ack_lat": { 2024-09-11T00:32:34.185 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 446034, 2024-09-11T00:32:34.185 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.118970309, 2024-09-11T00:32:34.185 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000266 2024-09-11T00:32:34.185 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-11T00:32:34.185 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_recv_encrypted_bytes": 1279664, 2024-09-11T00:32:34.185 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_encrypted_bytes": 324928 2024-09-11T00:32:34.185 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-11T00:32:34.185 INFO:teuthology.orchestra.run.smithi165.stdout: "cct": { 2024-09-11T00:32:34.186 INFO:teuthology.orchestra.run.smithi165.stdout: "total_workers": 1, 2024-09-11T00:32:34.186 INFO:teuthology.orchestra.run.smithi165.stdout: "unhealthy_workers": 0 2024-09-11T00:32:34.186 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-11T00:32:34.186 INFO:teuthology.orchestra.run.smithi165.stdout: "finisher-MDSRank": { 2024-09-11T00:32:34.186 INFO:teuthology.orchestra.run.smithi165.stdout: "queue_len": 0, 2024-09-11T00:32:34.186 INFO:teuthology.orchestra.run.smithi165.stdout: "complete_latency": { 2024-09-11T00:32:34.186 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 100131, 2024-09-11T00:32:34.186 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 89.376092816, 2024-09-11T00:32:34.186 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000892591 2024-09-11T00:32:34.186 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-11T00:32:34.186 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-11T00:32:34.186 INFO:teuthology.orchestra.run.smithi165.stdout: "finisher-PurgeQueue": { 2024-09-11T00:32:34.186 INFO:teuthology.orchestra.run.smithi165.stdout: "queue_len": 0, 2024-09-11T00:32:34.187 INFO:teuthology.orchestra.run.smithi165.stdout: "complete_latency": { 2024-09-11T00:32:34.187 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 37875, 2024-09-11T00:32:34.187 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 3.863039543, 2024-09-11T00:32:34.188 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000101994 2024-09-11T00:32:34.188 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-11T00:32:34.188 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-11T00:32:34.188 INFO:teuthology.orchestra.run.smithi165.stdout: "mds": { 2024-09-11T00:32:34.188 INFO:teuthology.orchestra.run.smithi165.stdout: "request": 148167, 2024-09-11T00:32:34.188 INFO:teuthology.orchestra.run.smithi165.stdout: "reply": 133970, 2024-09-11T00:32:34.188 INFO:teuthology.orchestra.run.smithi165.stdout: "reply_latency": { 2024-09-11T00:32:34.188 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 133970, 2024-09-11T00:32:34.188 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 1172.048864247, 2024-09-11T00:32:34.188 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.008748591 2024-09-11T00:32:34.188 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-11T00:32:34.188 INFO:teuthology.orchestra.run.smithi165.stdout: "slow_reply": 0, 2024-09-11T00:32:34.188 INFO:teuthology.orchestra.run.smithi165.stdout: "forward": 14197, 2024-09-11T00:32:34.189 INFO:teuthology.orchestra.run.smithi165.stdout: "dir_fetch_complete": 2, 2024-09-11T00:32:34.189 INFO:teuthology.orchestra.run.smithi165.stdout: "dir_fetch_keys": 0, 2024-09-11T00:32:34.189 INFO:teuthology.orchestra.run.smithi165.stdout: "dir_commit": 2849, 2024-09-11T00:32:34.189 INFO:teuthology.orchestra.run.smithi165.stdout: "dir_split": 5, 2024-09-11T00:32:34.189 INFO:teuthology.orchestra.run.smithi165.stdout: "dir_merge": 5, 2024-09-11T00:32:34.189 INFO:teuthology.orchestra.run.smithi165.stdout: "inodes": 101, 2024-09-11T00:32:34.189 INFO:teuthology.orchestra.run.smithi165.stdout: "inodes_top": 18, 2024-09-11T00:32:34.189 INFO:teuthology.orchestra.run.smithi165.stdout: "inodes_bottom": 35, 2024-09-11T00:32:34.189 INFO:teuthology.orchestra.run.smithi165.stdout: "inodes_pin_tail": 48, 2024-09-11T00:32:34.189 INFO:teuthology.orchestra.run.smithi165.stdout: "inodes_pinned": 74, 2024-09-11T00:32:34.189 INFO:teuthology.orchestra.run.smithi165.stdout: "inodes_expired": 2155, 2024-09-11T00:32:34.189 INFO:teuthology.orchestra.run.smithi165.stdout: "inodes_with_caps": 2, 2024-09-11T00:32:34.189 INFO:teuthology.orchestra.run.smithi165.stdout: "caps": 2, 2024-09-11T00:32:34.190 INFO:teuthology.orchestra.run.smithi165.stdout: "subtrees": 5, 2024-09-11T00:32:34.190 INFO:teuthology.orchestra.run.smithi165.stdout: "traverse": 244054, 2024-09-11T00:32:34.190 INFO:teuthology.orchestra.run.smithi165.stdout: "traverse_hit": 150980, 2024-09-11T00:32:34.190 INFO:teuthology.orchestra.run.smithi165.stdout: "traverse_forward": 14179, 2024-09-11T00:32:34.190 INFO:teuthology.orchestra.run.smithi165.stdout: "traverse_discover": 78, 2024-09-11T00:32:34.190 INFO:teuthology.orchestra.run.smithi165.stdout: "traverse_dir_fetch": 0, 2024-09-11T00:32:34.190 INFO:teuthology.orchestra.run.smithi165.stdout: "traverse_remote_ino": 0, 2024-09-11T00:32:34.190 INFO:teuthology.orchestra.run.smithi165.stdout: "traverse_lock": 2287, 2024-09-11T00:32:34.190 INFO:teuthology.orchestra.run.smithi165.stdout: "load_cent": 169, 2024-09-11T00:32:34.190 INFO:teuthology.orchestra.run.smithi165.stdout: "q": 0, 2024-09-11T00:32:34.190 INFO:teuthology.orchestra.run.smithi165.stdout: "exported": 31, 2024-09-11T00:32:34.190 INFO:teuthology.orchestra.run.smithi165.stdout: "exported_inodes": 1432, 2024-09-11T00:32:34.191 INFO:teuthology.orchestra.run.smithi165.stdout: "imported": 31, 2024-09-11T00:32:34.191 INFO:teuthology.orchestra.run.smithi165.stdout: "imported_inodes": 525, 2024-09-11T00:32:34.191 INFO:teuthology.orchestra.run.smithi165.stdout: "openino_dir_fetch": 0, 2024-09-11T00:32:34.191 INFO:teuthology.orchestra.run.smithi165.stdout: "openino_backtrace_fetch": 0, 2024-09-11T00:32:34.191 INFO:teuthology.orchestra.run.smithi165.stdout: "openino_peer_discover": 0, 2024-09-11T00:32:34.191 INFO:teuthology.orchestra.run.smithi165.stdout: "root_rfiles": 4, 2024-09-11T00:32:34.191 INFO:teuthology.orchestra.run.smithi165.stdout: "root_rbytes": 234, 2024-09-11T00:32:34.191 INFO:teuthology.orchestra.run.smithi165.stdout: "root_rsnaps": 0, 2024-09-11T00:32:34.191 INFO:teuthology.orchestra.run.smithi165.stdout: "scrub_backtrace_fetch": 0, 2024-09-11T00:32:34.191 INFO:teuthology.orchestra.run.smithi165.stdout: "scrub_set_tag": 0, 2024-09-11T00:32:34.191 INFO:teuthology.orchestra.run.smithi165.stdout: "scrub_backtrace_repaired": 0, 2024-09-11T00:32:34.191 INFO:teuthology.orchestra.run.smithi165.stdout: "scrub_inotable_repaired": 0, 2024-09-11T00:32:34.192 INFO:teuthology.orchestra.run.smithi165.stdout: "scrub_dir_inodes": 0, 2024-09-11T00:32:34.192 INFO:teuthology.orchestra.run.smithi165.stdout: "scrub_dir_base_inodes": 0, 2024-09-11T00:32:34.192 INFO:teuthology.orchestra.run.smithi165.stdout: "scrub_dirfrag_rstats": 0, 2024-09-11T00:32:34.192 INFO:teuthology.orchestra.run.smithi165.stdout: "scrub_file_inodes": 0, 2024-09-11T00:32:34.192 INFO:teuthology.orchestra.run.smithi165.stdout: "handle_inode_file_caps": 99, 2024-09-11T00:32:34.192 INFO:teuthology.orchestra.run.smithi165.stdout: "ceph_cap_op_revoke": 35110, 2024-09-11T00:32:34.192 INFO:teuthology.orchestra.run.smithi165.stdout: "ceph_cap_op_grant": 86857, 2024-09-11T00:32:34.192 INFO:teuthology.orchestra.run.smithi165.stdout: "ceph_cap_op_trunc": 0, 2024-09-11T00:32:34.192 INFO:teuthology.orchestra.run.smithi165.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-11T00:32:34.192 INFO:teuthology.orchestra.run.smithi165.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-11T00:32:34.192 INFO:teuthology.orchestra.run.smithi165.stdout: "handle_client_caps": 87707, 2024-09-11T00:32:34.192 INFO:teuthology.orchestra.run.smithi165.stdout: "handle_client_caps_dirty": 16627, 2024-09-11T00:32:34.192 INFO:teuthology.orchestra.run.smithi165.stdout: "handle_client_cap_release": 4102, 2024-09-11T00:32:34.193 INFO:teuthology.orchestra.run.smithi165.stdout: "process_request_cap_release": 80740 2024-09-11T00:32:34.193 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-11T00:32:34.193 INFO:teuthology.orchestra.run.smithi165.stdout: "mds_cache": { 2024-09-11T00:32:34.193 INFO:teuthology.orchestra.run.smithi165.stdout: "dir_update": 33, 2024-09-11T00:32:34.193 INFO:teuthology.orchestra.run.smithi165.stdout: "dir_update_receipt": 69, 2024-09-11T00:32:34.193 INFO:teuthology.orchestra.run.smithi165.stdout: "dir_try_discover": 46, 2024-09-11T00:32:34.193 INFO:teuthology.orchestra.run.smithi165.stdout: "dir_send_discover": 101, 2024-09-11T00:32:34.193 INFO:teuthology.orchestra.run.smithi165.stdout: "dir_handle_discover": 365, 2024-09-11T00:32:34.193 INFO:teuthology.orchestra.run.smithi165.stdout: "num_strays": 7, 2024-09-11T00:32:34.193 INFO:teuthology.orchestra.run.smithi165.stdout: "num_strays_delayed": 0, 2024-09-11T00:32:34.193 INFO:teuthology.orchestra.run.smithi165.stdout: "num_strays_enqueuing": 0, 2024-09-11T00:32:34.193 INFO:teuthology.orchestra.run.smithi165.stdout: "strays_created": 19284, 2024-09-11T00:32:34.193 INFO:teuthology.orchestra.run.smithi165.stdout: "strays_enqueued": 19277, 2024-09-11T00:32:34.194 INFO:teuthology.orchestra.run.smithi165.stdout: "strays_reintegrated": 0, 2024-09-11T00:32:34.194 INFO:teuthology.orchestra.run.smithi165.stdout: "strays_migrated": 0, 2024-09-11T00:32:34.194 INFO:teuthology.orchestra.run.smithi165.stdout: "num_recovering_processing": 0, 2024-09-11T00:32:34.194 INFO:teuthology.orchestra.run.smithi165.stdout: "num_recovering_enqueued": 0, 2024-09-11T00:32:34.194 INFO:teuthology.orchestra.run.smithi165.stdout: "num_recovering_prioritized": 0, 2024-09-11T00:32:34.194 INFO:teuthology.orchestra.run.smithi165.stdout: "recovery_started": 0, 2024-09-11T00:32:34.194 INFO:teuthology.orchestra.run.smithi165.stdout: "recovery_completed": 0, 2024-09-11T00:32:34.194 INFO:teuthology.orchestra.run.smithi165.stdout: "ireq_quiesce_path": 19, 2024-09-11T00:32:34.194 INFO:teuthology.orchestra.run.smithi165.stdout: "ireq_quiesce_inode": 2318, 2024-09-11T00:32:34.194 INFO:teuthology.orchestra.run.smithi165.stdout: "ireq_enqueue_scrub": 0, 2024-09-11T00:32:34.194 INFO:teuthology.orchestra.run.smithi165.stdout: "ireq_exportdir": 31, 2024-09-11T00:32:34.194 INFO:teuthology.orchestra.run.smithi165.stdout: "ireq_flush": 0, 2024-09-11T00:32:34.194 INFO:teuthology.orchestra.run.smithi165.stdout: "ireq_fragmentdir": 10, 2024-09-11T00:32:34.194 INFO:teuthology.orchestra.run.smithi165.stdout: "ireq_fragstats": 0, 2024-09-11T00:32:34.195 INFO:teuthology.orchestra.run.smithi165.stdout: "ireq_inodestats": 0, 2024-09-11T00:32:34.195 INFO:teuthology.orchestra.run.smithi165.stdout: "uninline_started": 0, 2024-09-11T00:32:34.195 INFO:teuthology.orchestra.run.smithi165.stdout: "uninline_succeeded": 0, 2024-09-11T00:32:34.195 INFO:teuthology.orchestra.run.smithi165.stdout: "uninline_write_failed": 0 2024-09-11T00:32:34.195 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-11T00:32:34.195 INFO:teuthology.orchestra.run.smithi165.stdout: "mds_log": { 2024-09-11T00:32:34.195 INFO:teuthology.orchestra.run.smithi165.stdout: "evlrg": 0, 2024-09-11T00:32:34.195 INFO:teuthology.orchestra.run.smithi165.stdout: "evadd": 121921, 2024-09-11T00:32:34.195 INFO:teuthology.orchestra.run.smithi165.stdout: "evex": 121336, 2024-09-11T00:32:34.195 INFO:teuthology.orchestra.run.smithi165.stdout: "evtrm": 121336, 2024-09-11T00:32:34.195 INFO:teuthology.orchestra.run.smithi165.stdout: "ev": 585, 2024-09-11T00:32:34.195 INFO:teuthology.orchestra.run.smithi165.stdout: "evexg": 0, 2024-09-11T00:32:34.195 INFO:teuthology.orchestra.run.smithi165.stdout: "evexd": 27, 2024-09-11T00:32:34.196 INFO:teuthology.orchestra.run.smithi165.stdout: "segadd": 227, 2024-09-11T00:32:34.196 INFO:teuthology.orchestra.run.smithi165.stdout: "segex": 225, 2024-09-11T00:32:34.196 INFO:teuthology.orchestra.run.smithi165.stdout: "segtrm": 225, 2024-09-11T00:32:34.196 INFO:teuthology.orchestra.run.smithi165.stdout: "seg": 2, 2024-09-11T00:32:34.196 INFO:teuthology.orchestra.run.smithi165.stdout: "segmjr": 1, 2024-09-11T00:32:34.196 INFO:teuthology.orchestra.run.smithi165.stdout: "segexg": 0, 2024-09-11T00:32:34.196 INFO:teuthology.orchestra.run.smithi165.stdout: "segexd": 1, 2024-09-11T00:32:34.196 INFO:teuthology.orchestra.run.smithi165.stdout: "expos": 255441103, 2024-09-11T00:32:34.196 INFO:teuthology.orchestra.run.smithi165.stdout: "wrpos": 256501529, 2024-09-11T00:32:34.196 INFO:teuthology.orchestra.run.smithi165.stdout: "rdpos": 0, 2024-09-11T00:32:34.196 INFO:teuthology.orchestra.run.smithi165.stdout: "jlat": { 2024-09-11T00:32:34.196 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 52276, 2024-09-11T00:32:34.196 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 381.851799404, 2024-09-11T00:32:34.197 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.007304533 2024-09-11T00:32:34.197 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-11T00:32:34.197 INFO:teuthology.orchestra.run.smithi165.stdout: "replayed": 0 2024-09-11T00:32:34.197 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-11T00:32:34.197 INFO:teuthology.orchestra.run.smithi165.stdout: "mds_mem": { 2024-09-11T00:32:34.197 INFO:teuthology.orchestra.run.smithi165.stdout: "ino": 31, 2024-09-11T00:32:34.197 INFO:teuthology.orchestra.run.smithi165.stdout: "ino+": 20336, 2024-09-11T00:32:34.197 INFO:teuthology.orchestra.run.smithi165.stdout: "ino-": 20305, 2024-09-11T00:32:34.197 INFO:teuthology.orchestra.run.smithi165.stdout: "dir": 29, 2024-09-11T00:32:34.197 INFO:teuthology.orchestra.run.smithi165.stdout: "dir+": 255, 2024-09-11T00:32:34.197 INFO:teuthology.orchestra.run.smithi165.stdout: "dir-": 226, 2024-09-11T00:32:34.197 INFO:teuthology.orchestra.run.smithi165.stdout: "dn": 101, 2024-09-11T00:32:34.197 INFO:teuthology.orchestra.run.smithi165.stdout: "dn+": 42294, 2024-09-11T00:32:34.198 INFO:teuthology.orchestra.run.smithi165.stdout: "dn-": 42193, 2024-09-11T00:32:34.198 INFO:teuthology.orchestra.run.smithi165.stdout: "cap": 2, 2024-09-11T00:32:34.198 INFO:teuthology.orchestra.run.smithi165.stdout: "cap+": 23100, 2024-09-11T00:32:34.198 INFO:teuthology.orchestra.run.smithi165.stdout: "cap-": 23098, 2024-09-11T00:32:34.198 INFO:teuthology.orchestra.run.smithi165.stdout: "rss": 82812, 2024-09-11T00:32:34.198 INFO:teuthology.orchestra.run.smithi165.stdout: "heap": 315486208 2024-09-11T00:32:34.198 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-11T00:32:34.198 INFO:teuthology.orchestra.run.smithi165.stdout: "mds_server": { 2024-09-11T00:32:34.198 INFO:teuthology.orchestra.run.smithi165.stdout: "dispatch_client_request": 217818, 2024-09-11T00:32:34.198 INFO:teuthology.orchestra.run.smithi165.stdout: "dispatch_server_request": 19793, 2024-09-11T00:32:34.198 INFO:teuthology.orchestra.run.smithi165.stdout: "handle_client_request": 148167, 2024-09-11T00:32:34.198 INFO:teuthology.orchestra.run.smithi165.stdout: "handle_client_session": 2419, 2024-09-11T00:32:34.199 INFO:teuthology.orchestra.run.smithi165.stdout: "handle_peer_request": 42749, 2024-09-11T00:32:34.199 INFO:teuthology.orchestra.run.smithi165.stdout: "req_create_latency": { 2024-09-11T00:32:34.199 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 19780, 2024-09-11T00:32:34.199 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 388.870376496, 2024-09-11T00:32:34.199 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.019659776 2024-09-11T00:32:34.199 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-11T00:32:34.199 INFO:teuthology.orchestra.run.smithi165.stdout: "req_getattr_latency": { 2024-09-11T00:32:34.199 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 27594, 2024-09-11T00:32:34.199 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 214.946768358, 2024-09-11T00:32:34.199 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.007789619 2024-09-11T00:32:34.199 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-11T00:32:34.199 INFO:teuthology.orchestra.run.smithi165.stdout: "req_getfilelock_latency": { 2024-09-11T00:32:34.199 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-09-11T00:32:34.200 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-09-11T00:32:34.200 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-09-11T00:32:34.200 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-11T00:32:34.200 INFO:teuthology.orchestra.run.smithi165.stdout: "req_link_latency": { 2024-09-11T00:32:34.200 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-09-11T00:32:34.200 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-09-11T00:32:34.200 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-09-11T00:32:34.200 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-11T00:32:34.200 INFO:teuthology.orchestra.run.smithi165.stdout: "req_lookup_latency": { 2024-09-11T00:32:34.200 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 32775, 2024-09-11T00:32:34.200 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 151.555953665, 2024-09-11T00:32:34.200 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.004624132 2024-09-11T00:32:34.201 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-11T00:32:34.201 INFO:teuthology.orchestra.run.smithi165.stdout: "req_lookuphash_latency": { 2024-09-11T00:32:34.201 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-09-11T00:32:34.201 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-09-11T00:32:34.201 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-09-11T00:32:34.201 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-11T00:32:34.201 INFO:teuthology.orchestra.run.smithi165.stdout: "req_lookupino_latency": { 2024-09-11T00:32:34.201 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 1, 2024-09-11T00:32:34.201 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000339799, 2024-09-11T00:32:34.201 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000339799 2024-09-11T00:32:34.201 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-11T00:32:34.201 INFO:teuthology.orchestra.run.smithi165.stdout: "req_lookupname_latency": { 2024-09-11T00:32:34.201 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-09-11T00:32:34.202 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-09-11T00:32:34.202 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-09-11T00:32:34.202 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-11T00:32:34.202 INFO:teuthology.orchestra.run.smithi165.stdout: "req_lookupparent_latency": { 2024-09-11T00:32:34.202 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-09-11T00:32:34.202 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-09-11T00:32:34.203 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-09-11T00:32:34.203 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-11T00:32:34.203 INFO:teuthology.orchestra.run.smithi165.stdout: "req_lookupsnap_latency": { 2024-09-11T00:32:34.203 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-09-11T00:32:34.203 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-09-11T00:32:34.203 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-09-11T00:32:34.203 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-11T00:32:34.203 INFO:teuthology.orchestra.run.smithi165.stdout: "req_lssnap_latency": { 2024-09-11T00:32:34.203 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-09-11T00:32:34.203 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-09-11T00:32:34.203 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-09-11T00:32:34.203 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-11T00:32:34.204 INFO:teuthology.orchestra.run.smithi165.stdout: "req_mkdir_latency": { 2024-09-11T00:32:34.204 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 119, 2024-09-11T00:32:34.204 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.656585828, 2024-09-11T00:32:34.204 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.005517527 2024-09-11T00:32:34.204 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-11T00:32:34.204 INFO:teuthology.orchestra.run.smithi165.stdout: "req_mknod_latency": { 2024-09-11T00:32:34.204 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-09-11T00:32:34.204 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-09-11T00:32:34.204 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-09-11T00:32:34.204 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-11T00:32:34.204 INFO:teuthology.orchestra.run.smithi165.stdout: "req_mksnap_latency": { 2024-09-11T00:32:34.204 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-09-11T00:32:34.205 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-09-11T00:32:34.205 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-09-11T00:32:34.205 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-11T00:32:34.205 INFO:teuthology.orchestra.run.smithi165.stdout: "req_open_latency": { 2024-09-11T00:32:34.205 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 12, 2024-09-11T00:32:34.205 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.025651813, 2024-09-11T00:32:34.205 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.002137651 2024-09-11T00:32:34.205 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-11T00:32:34.205 INFO:teuthology.orchestra.run.smithi165.stdout: "req_readdir_latency": { 2024-09-11T00:32:34.205 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 29948, 2024-09-11T00:32:34.205 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 128.270319424, 2024-09-11T00:32:34.205 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.004283101 2024-09-11T00:32:34.205 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-11T00:32:34.206 INFO:teuthology.orchestra.run.smithi165.stdout: "req_rename_latency": { 2024-09-11T00:32:34.206 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 3895, 2024-09-11T00:32:34.206 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 57.772231302, 2024-09-11T00:32:34.206 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.014832408 2024-09-11T00:32:34.206 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-11T00:32:34.206 INFO:teuthology.orchestra.run.smithi165.stdout: "req_renamesnap_latency": { 2024-09-11T00:32:34.206 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-09-11T00:32:34.206 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-09-11T00:32:34.206 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-09-11T00:32:34.206 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-11T00:32:34.206 INFO:teuthology.orchestra.run.smithi165.stdout: "req_snapdiff_latency": { 2024-09-11T00:32:34.206 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-09-11T00:32:34.206 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-09-11T00:32:34.207 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-09-11T00:32:34.207 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-11T00:32:34.207 INFO:teuthology.orchestra.run.smithi165.stdout: "req_rmdir_latency": { 2024-09-11T00:32:34.207 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 44, 2024-09-11T00:32:34.207 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.665815689, 2024-09-11T00:32:34.207 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.015132174 2024-09-11T00:32:34.207 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-11T00:32:34.207 INFO:teuthology.orchestra.run.smithi165.stdout: "req_rmsnap_latency": { 2024-09-11T00:32:34.207 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-09-11T00:32:34.207 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-09-11T00:32:34.207 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-09-11T00:32:34.207 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-11T00:32:34.207 INFO:teuthology.orchestra.run.smithi165.stdout: "req_rmxattr_latency": { 2024-09-11T00:32:34.208 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-09-11T00:32:34.208 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-09-11T00:32:34.208 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-09-11T00:32:34.208 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-11T00:32:34.208 INFO:teuthology.orchestra.run.smithi165.stdout: "req_setattr_latency": { 2024-09-11T00:32:34.208 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 123, 2024-09-11T00:32:34.208 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 59.399416319, 2024-09-11T00:32:34.208 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.482922083 2024-09-11T00:32:34.208 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-11T00:32:34.208 INFO:teuthology.orchestra.run.smithi165.stdout: "req_setdirlayout_latency": { 2024-09-11T00:32:34.208 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-09-11T00:32:34.208 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-09-11T00:32:34.208 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-09-11T00:32:34.209 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-11T00:32:34.209 INFO:teuthology.orchestra.run.smithi165.stdout: "req_setfilelock_latency": { 2024-09-11T00:32:34.209 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 420, 2024-09-11T00:32:34.209 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.523977147, 2024-09-11T00:32:34.209 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.001247564 2024-09-11T00:32:34.209 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-11T00:32:34.209 INFO:teuthology.orchestra.run.smithi165.stdout: "req_setlayout_latency": { 2024-09-11T00:32:34.209 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-09-11T00:32:34.209 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-09-11T00:32:34.209 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-09-11T00:32:34.209 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-11T00:32:34.210 INFO:teuthology.orchestra.run.smithi165.stdout: "req_setxattr_latency": { 2024-09-11T00:32:34.210 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 8, 2024-09-11T00:32:34.210 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.022816907, 2024-09-11T00:32:34.210 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.002852113 2024-09-11T00:32:34.210 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-11T00:32:34.210 INFO:teuthology.orchestra.run.smithi165.stdout: "req_symlink_latency": { 2024-09-11T00:32:34.210 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-09-11T00:32:34.210 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-09-11T00:32:34.210 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-09-11T00:32:34.210 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-11T00:32:34.210 INFO:teuthology.orchestra.run.smithi165.stdout: "req_unlink_latency": { 2024-09-11T00:32:34.210 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 19251, 2024-09-11T00:32:34.210 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 169.338611500, 2024-09-11T00:32:34.211 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.008796354 2024-09-11T00:32:34.211 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-11T00:32:34.211 INFO:teuthology.orchestra.run.smithi165.stdout: "cap_revoke_eviction": 0, 2024-09-11T00:32:34.211 INFO:teuthology.orchestra.run.smithi165.stdout: "cap_acquisition_throttle": 0, 2024-09-11T00:32:34.211 INFO:teuthology.orchestra.run.smithi165.stdout: "req_getvxattr_latency": { 2024-09-11T00:32:34.211 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-09-11T00:32:34.211 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-09-11T00:32:34.211 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-09-11T00:32:34.211 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-11T00:32:34.211 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-11T00:32:34.211 INFO:teuthology.orchestra.run.smithi165.stdout: "mds_sessions": { 2024-09-11T00:32:34.211 INFO:teuthology.orchestra.run.smithi165.stdout: "session_count": 1, 2024-09-11T00:32:34.211 INFO:teuthology.orchestra.run.smithi165.stdout: "session_add": 3, 2024-09-11T00:32:34.212 INFO:teuthology.orchestra.run.smithi165.stdout: "session_remove": 2, 2024-09-11T00:32:34.212 INFO:teuthology.orchestra.run.smithi165.stdout: "sessions_open": 1, 2024-09-11T00:32:34.212 INFO:teuthology.orchestra.run.smithi165.stdout: "sessions_stale": 0, 2024-09-11T00:32:34.212 INFO:teuthology.orchestra.run.smithi165.stdout: "total_load": 10005, 2024-09-11T00:32:34.212 INFO:teuthology.orchestra.run.smithi165.stdout: "average_load": 10005, 2024-09-11T00:32:34.212 INFO:teuthology.orchestra.run.smithi165.stdout: "avg_session_uptime": 1548, 2024-09-11T00:32:34.212 INFO:teuthology.orchestra.run.smithi165.stdout: "mdthresh_evicted": 0 2024-09-11T00:32:34.212 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-11T00:32:34.212 INFO:teuthology.orchestra.run.smithi165.stdout: "mempool": { 2024-09-11T00:32:34.212 INFO:teuthology.orchestra.run.smithi165.stdout: "bloom_filter_bytes": 360, 2024-09-11T00:32:34.212 INFO:teuthology.orchestra.run.smithi165.stdout: "bloom_filter_items": 360, 2024-09-11T00:32:34.212 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_alloc_bytes": 0, 2024-09-11T00:32:34.213 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_alloc_items": 0, 2024-09-11T00:32:34.213 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_cache_data_bytes": 0, 2024-09-11T00:32:34.213 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_cache_data_items": 0, 2024-09-11T00:32:34.213 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-11T00:32:34.213 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_cache_onode_items": 0, 2024-09-11T00:32:34.213 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-11T00:32:34.213 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_cache_meta_items": 0, 2024-09-11T00:32:34.213 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_cache_other_bytes": 0, 2024-09-11T00:32:34.213 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_cache_other_items": 0, 2024-09-11T00:32:34.213 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-11T00:32:34.213 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_cache_buffer_items": 0, 2024-09-11T00:32:34.213 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_extent_bytes": 0, 2024-09-11T00:32:34.213 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_extent_items": 0, 2024-09-11T00:32:34.214 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_blob_bytes": 0, 2024-09-11T00:32:34.214 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_blob_items": 0, 2024-09-11T00:32:34.214 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-11T00:32:34.214 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_shared_blob_items": 0, 2024-09-11T00:32:34.214 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-11T00:32:34.214 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_inline_bl_items": 0, 2024-09-11T00:32:34.214 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_fsck_bytes": 0, 2024-09-11T00:32:34.214 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_fsck_items": 0, 2024-09-11T00:32:34.214 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_txc_bytes": 0, 2024-09-11T00:32:34.214 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_txc_items": 0, 2024-09-11T00:32:34.214 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-11T00:32:34.214 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_writing_deferred_items": 0, 2024-09-11T00:32:34.215 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_writing_bytes": 0, 2024-09-11T00:32:34.215 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_writing_items": 0, 2024-09-11T00:32:34.215 INFO:teuthology.orchestra.run.smithi165.stdout: "bluefs_bytes": 0, 2024-09-11T00:32:34.215 INFO:teuthology.orchestra.run.smithi165.stdout: "bluefs_items": 0, 2024-09-11T00:32:34.215 INFO:teuthology.orchestra.run.smithi165.stdout: "bluefs_file_reader_bytes": 0, 2024-09-11T00:32:34.215 INFO:teuthology.orchestra.run.smithi165.stdout: "bluefs_file_reader_items": 0, 2024-09-11T00:32:34.215 INFO:teuthology.orchestra.run.smithi165.stdout: "bluefs_file_writer_bytes": 0, 2024-09-11T00:32:34.215 INFO:teuthology.orchestra.run.smithi165.stdout: "bluefs_file_writer_items": 0, 2024-09-11T00:32:34.215 INFO:teuthology.orchestra.run.smithi165.stdout: "buffer_anon_bytes": 280517, 2024-09-11T00:32:34.216 INFO:teuthology.orchestra.run.smithi165.stdout: "buffer_anon_items": 108, 2024-09-11T00:32:34.216 INFO:teuthology.orchestra.run.smithi165.stdout: "buffer_meta_bytes": 88, 2024-09-11T00:32:34.216 INFO:teuthology.orchestra.run.smithi165.stdout: "buffer_meta_items": 1, 2024-09-11T00:32:34.216 INFO:teuthology.orchestra.run.smithi165.stdout: "osd_bytes": 0, 2024-09-11T00:32:34.216 INFO:teuthology.orchestra.run.smithi165.stdout: "osd_items": 0, 2024-09-11T00:32:34.216 INFO:teuthology.orchestra.run.smithi165.stdout: "osd_mapbl_bytes": 0, 2024-09-11T00:32:34.216 INFO:teuthology.orchestra.run.smithi165.stdout: "osd_mapbl_items": 0, 2024-09-11T00:32:34.216 INFO:teuthology.orchestra.run.smithi165.stdout: "osd_pglog_bytes": 0, 2024-09-11T00:32:34.216 INFO:teuthology.orchestra.run.smithi165.stdout: "osd_pglog_items": 0, 2024-09-11T00:32:34.216 INFO:teuthology.orchestra.run.smithi165.stdout: "osdmap_bytes": 9048, 2024-09-11T00:32:34.216 INFO:teuthology.orchestra.run.smithi165.stdout: "osdmap_items": 170, 2024-09-11T00:32:34.216 INFO:teuthology.orchestra.run.smithi165.stdout: "osdmap_mapping_bytes": 0, 2024-09-11T00:32:34.217 INFO:teuthology.orchestra.run.smithi165.stdout: "osdmap_mapping_items": 0, 2024-09-11T00:32:34.217 INFO:teuthology.orchestra.run.smithi165.stdout: "pgmap_bytes": 0, 2024-09-11T00:32:34.217 INFO:teuthology.orchestra.run.smithi165.stdout: "pgmap_items": 0, 2024-09-11T00:32:34.217 INFO:teuthology.orchestra.run.smithi165.stdout: "mds_co_bytes": 188580, 2024-09-11T00:32:34.217 INFO:teuthology.orchestra.run.smithi165.stdout: "mds_co_items": 2206, 2024-09-11T00:32:34.217 INFO:teuthology.orchestra.run.smithi165.stdout: "unittest_1_bytes": 0, 2024-09-11T00:32:34.217 INFO:teuthology.orchestra.run.smithi165.stdout: "unittest_1_items": 0, 2024-09-11T00:32:34.217 INFO:teuthology.orchestra.run.smithi165.stdout: "unittest_2_bytes": 0, 2024-09-11T00:32:34.217 INFO:teuthology.orchestra.run.smithi165.stdout: "unittest_2_items": 0 2024-09-11T00:32:34.217 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-11T00:32:34.217 INFO:teuthology.orchestra.run.smithi165.stdout: "objecter": { 2024-09-11T00:32:34.217 INFO:teuthology.orchestra.run.smithi165.stdout: "op_active": 0, 2024-09-11T00:32:34.217 INFO:teuthology.orchestra.run.smithi165.stdout: "op_laggy": 0, 2024-09-11T00:32:34.217 INFO:teuthology.orchestra.run.smithi165.stdout: "op_send": 119176, 2024-09-11T00:32:34.218 INFO:teuthology.orchestra.run.smithi165.stdout: "op_send_bytes": 274541988, 2024-09-11T00:32:34.218 INFO:teuthology.orchestra.run.smithi165.stdout: "op_resend": 32, 2024-09-11T00:32:34.218 INFO:teuthology.orchestra.run.smithi165.stdout: "op_reply": 119176, 2024-09-11T00:32:34.218 INFO:teuthology.orchestra.run.smithi165.stdout: "op_latency": { 2024-09-11T00:32:34.218 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 119176, 2024-09-11T00:32:34.218 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 3411.778237863, 2024-09-11T00:32:34.218 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.028628064 2024-09-11T00:32:34.218 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-11T00:32:34.218 INFO:teuthology.orchestra.run.smithi165.stdout: "op_inflight": 0, 2024-09-11T00:32:34.218 INFO:teuthology.orchestra.run.smithi165.stdout: "oplen_avg": { 2024-09-11T00:32:34.218 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 119176, 2024-09-11T00:32:34.218 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 159029 2024-09-11T00:32:34.218 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-11T00:32:34.219 INFO:teuthology.orchestra.run.smithi165.stdout: "op": 119176, 2024-09-11T00:32:34.219 INFO:teuthology.orchestra.run.smithi165.stdout: "op_r": 757, 2024-09-11T00:32:34.219 INFO:teuthology.orchestra.run.smithi165.stdout: "op_w": 118419, 2024-09-11T00:32:34.219 INFO:teuthology.orchestra.run.smithi165.stdout: "op_rmw": 0, 2024-09-11T00:32:34.219 INFO:teuthology.orchestra.run.smithi165.stdout: "op_pg": 0, 2024-09-11T00:32:34.219 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_stat": 2691, 2024-09-11T00:32:34.219 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_create": 21452, 2024-09-11T00:32:34.219 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_read": 755, 2024-09-11T00:32:34.219 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_write": 53031, 2024-09-11T00:32:34.219 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_writefull": 884, 2024-09-11T00:32:34.219 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_writesame": 0, 2024-09-11T00:32:34.219 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_append": 0, 2024-09-11T00:32:34.219 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_zero": 0, 2024-09-11T00:32:34.219 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_truncate": 5, 2024-09-11T00:32:34.220 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_delete": 39199, 2024-09-11T00:32:34.220 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_mapext": 0, 2024-09-11T00:32:34.220 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_sparse_read": 0, 2024-09-11T00:32:34.220 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_clonerange": 0, 2024-09-11T00:32:34.220 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_getxattr": 2, 2024-09-11T00:32:34.220 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_setxattr": 32432, 2024-09-11T00:32:34.220 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_cmpxattr": 0, 2024-09-11T00:32:34.220 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_rmxattr": 0, 2024-09-11T00:32:34.220 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_resetxattrs": 0, 2024-09-11T00:32:34.220 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_call": 0, 2024-09-11T00:32:34.220 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_watch": 0, 2024-09-11T00:32:34.221 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_notify": 0, 2024-09-11T00:32:34.221 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_src_cmpxattr": 0, 2024-09-11T00:32:34.221 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_pgls": 0, 2024-09-11T00:32:34.221 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_pgls_filter": 0, 2024-09-11T00:32:34.221 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_other": 0, 2024-09-11T00:32:34.221 INFO:teuthology.orchestra.run.smithi165.stdout: "linger_active": 0, 2024-09-11T00:32:34.221 INFO:teuthology.orchestra.run.smithi165.stdout: "linger_send": 0, 2024-09-11T00:32:34.221 INFO:teuthology.orchestra.run.smithi165.stdout: "linger_resend": 0, 2024-09-11T00:32:34.221 INFO:teuthology.orchestra.run.smithi165.stdout: "linger_ping": 0, 2024-09-11T00:32:34.221 INFO:teuthology.orchestra.run.smithi165.stdout: "poolop_active": 0, 2024-09-11T00:32:34.221 INFO:teuthology.orchestra.run.smithi165.stdout: "poolop_send": 0, 2024-09-11T00:32:34.221 INFO:teuthology.orchestra.run.smithi165.stdout: "poolop_resend": 0, 2024-09-11T00:32:34.222 INFO:teuthology.orchestra.run.smithi165.stdout: "poolstat_active": 0, 2024-09-11T00:32:34.222 INFO:teuthology.orchestra.run.smithi165.stdout: "poolstat_send": 0, 2024-09-11T00:32:34.222 INFO:teuthology.orchestra.run.smithi165.stdout: "poolstat_resend": 0, 2024-09-11T00:32:34.222 INFO:teuthology.orchestra.run.smithi165.stdout: "statfs_active": 0, 2024-09-11T00:32:34.222 INFO:teuthology.orchestra.run.smithi165.stdout: "statfs_send": 0, 2024-09-11T00:32:34.222 INFO:teuthology.orchestra.run.smithi165.stdout: "statfs_resend": 0, 2024-09-11T00:32:34.222 INFO:teuthology.orchestra.run.smithi165.stdout: "command_active": 0, 2024-09-11T00:32:34.222 INFO:teuthology.orchestra.run.smithi165.stdout: "command_send": 0, 2024-09-11T00:32:34.222 INFO:teuthology.orchestra.run.smithi165.stdout: "command_resend": 0, 2024-09-11T00:32:34.222 INFO:teuthology.orchestra.run.smithi165.stdout: "map_epoch": 87, 2024-09-11T00:32:34.222 INFO:teuthology.orchestra.run.smithi165.stdout: "map_full": 0, 2024-09-11T00:32:34.222 INFO:teuthology.orchestra.run.smithi165.stdout: "map_inc": 13, 2024-09-11T00:32:34.222 INFO:teuthology.orchestra.run.smithi165.stdout: "osd_sessions": 12, 2024-09-11T00:32:34.222 INFO:teuthology.orchestra.run.smithi165.stdout: "osd_session_open": 12, 2024-09-11T00:32:34.223 INFO:teuthology.orchestra.run.smithi165.stdout: "osd_session_close": 0, 2024-09-11T00:32:34.223 INFO:teuthology.orchestra.run.smithi165.stdout: "osd_laggy": 0, 2024-09-11T00:32:34.223 INFO:teuthology.orchestra.run.smithi165.stdout: "omap_wr": 7073, 2024-09-11T00:32:34.223 INFO:teuthology.orchestra.run.smithi165.stdout: "omap_rd": 4, 2024-09-11T00:32:34.223 INFO:teuthology.orchestra.run.smithi165.stdout: "omap_del": 1501 2024-09-11T00:32:34.223 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-11T00:32:34.223 INFO:teuthology.orchestra.run.smithi165.stdout: "oft": { 2024-09-11T00:32:34.223 INFO:teuthology.orchestra.run.smithi165.stdout: "omap_total_objs": 1, 2024-09-11T00:32:34.223 INFO:teuthology.orchestra.run.smithi165.stdout: "omap_total_kv_pairs": 0, 2024-09-11T00:32:34.223 INFO:teuthology.orchestra.run.smithi165.stdout: "omap_total_updates": 0, 2024-09-11T00:32:34.223 INFO:teuthology.orchestra.run.smithi165.stdout: "omap_total_removes": 0 2024-09-11T00:32:34.223 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-11T00:32:34.223 INFO:teuthology.orchestra.run.smithi165.stdout: "purge_queue": { 2024-09-11T00:32:34.224 INFO:teuthology.orchestra.run.smithi165.stdout: "pq_executing_ops": 0, 2024-09-11T00:32:34.224 INFO:teuthology.orchestra.run.smithi165.stdout: "pq_executing_ops_high_water": 10, 2024-09-11T00:32:34.224 INFO:teuthology.orchestra.run.smithi165.stdout: "pq_executing": 0, 2024-09-11T00:32:34.224 INFO:teuthology.orchestra.run.smithi165.stdout: "pq_executing_high_water": 6, 2024-09-11T00:32:34.224 INFO:teuthology.orchestra.run.smithi165.stdout: "pq_executed_ops": 38528, 2024-09-11T00:32:34.224 INFO:teuthology.orchestra.run.smithi165.stdout: "pq_executed": 19277, 2024-09-11T00:32:34.224 INFO:teuthology.orchestra.run.smithi165.stdout: "pq_item_in_journal": 0 2024-09-11T00:32:34.224 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-11T00:32:34.224 INFO:teuthology.orchestra.run.smithi165.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-11T00:32:34.224 INFO:teuthology.orchestra.run.smithi165.stdout: "val": 0, 2024-09-11T00:32:34.224 INFO:teuthology.orchestra.run.smithi165.stdout: "max": 104857600, 2024-09-11T00:32:34.224 INFO:teuthology.orchestra.run.smithi165.stdout: "get_started": 0, 2024-09-11T00:32:34.225 INFO:teuthology.orchestra.run.smithi165.stdout: "get": 644699, 2024-09-11T00:32:34.225 INFO:teuthology.orchestra.run.smithi165.stdout: "get_sum": 125945229, 2024-09-11T00:32:34.225 INFO:teuthology.orchestra.run.smithi165.stdout: "get_or_fail_fail": 0, 2024-09-11T00:32:34.225 INFO:teuthology.orchestra.run.smithi165.stdout: "get_or_fail_success": 644699, 2024-09-11T00:32:34.225 INFO:teuthology.orchestra.run.smithi165.stdout: "take": 0, 2024-09-11T00:32:34.225 INFO:teuthology.orchestra.run.smithi165.stdout: "take_sum": 0, 2024-09-11T00:32:34.225 INFO:teuthology.orchestra.run.smithi165.stdout: "put": 644699, 2024-09-11T00:32:34.225 INFO:teuthology.orchestra.run.smithi165.stdout: "put_sum": 125945229, 2024-09-11T00:32:34.225 INFO:teuthology.orchestra.run.smithi165.stdout: "wait": { 2024-09-11T00:32:34.225 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-09-11T00:32:34.225 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-09-11T00:32:34.225 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-09-11T00:32:34.225 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-11T00:32:34.226 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-11T00:32:34.226 INFO:teuthology.orchestra.run.smithi165.stdout: "throttle-objecter_bytes": { 2024-09-11T00:32:34.226 INFO:teuthology.orchestra.run.smithi165.stdout: "val": 0, 2024-09-11T00:32:34.226 INFO:teuthology.orchestra.run.smithi165.stdout: "max": 104857600, 2024-09-11T00:32:34.226 INFO:teuthology.orchestra.run.smithi165.stdout: "get_started": 0, 2024-09-11T00:32:34.226 INFO:teuthology.orchestra.run.smithi165.stdout: "get": 0, 2024-09-11T00:32:34.226 INFO:teuthology.orchestra.run.smithi165.stdout: "get_sum": 0, 2024-09-11T00:32:34.226 INFO:teuthology.orchestra.run.smithi165.stdout: "get_or_fail_fail": 0, 2024-09-11T00:32:34.226 INFO:teuthology.orchestra.run.smithi165.stdout: "get_or_fail_success": 0, 2024-09-11T00:32:34.226 INFO:teuthology.orchestra.run.smithi165.stdout: "take": 119176, 2024-09-11T00:32:34.226 INFO:teuthology.orchestra.run.smithi165.stdout: "take_sum": 276642941, 2024-09-11T00:32:34.226 INFO:teuthology.orchestra.run.smithi165.stdout: "put": 119176, 2024-09-11T00:32:34.226 INFO:teuthology.orchestra.run.smithi165.stdout: "put_sum": 276642941, 2024-09-11T00:32:34.226 INFO:teuthology.orchestra.run.smithi165.stdout: "wait": { 2024-09-11T00:32:34.227 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-09-11T00:32:34.227 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-09-11T00:32:34.227 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-09-11T00:32:34.227 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-11T00:32:34.227 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-11T00:32:34.227 INFO:teuthology.orchestra.run.smithi165.stdout: "throttle-objecter_ops": { 2024-09-11T00:32:34.227 INFO:teuthology.orchestra.run.smithi165.stdout: "val": 0, 2024-09-11T00:32:34.227 INFO:teuthology.orchestra.run.smithi165.stdout: "max": 1024, 2024-09-11T00:32:34.227 INFO:teuthology.orchestra.run.smithi165.stdout: "get_started": 0, 2024-09-11T00:32:34.227 INFO:teuthology.orchestra.run.smithi165.stdout: "get": 0, 2024-09-11T00:32:34.227 INFO:teuthology.orchestra.run.smithi165.stdout: "get_sum": 0, 2024-09-11T00:32:34.227 INFO:teuthology.orchestra.run.smithi165.stdout: "get_or_fail_fail": 0, 2024-09-11T00:32:34.227 INFO:teuthology.orchestra.run.smithi165.stdout: "get_or_fail_success": 0, 2024-09-11T00:32:34.228 INFO:teuthology.orchestra.run.smithi165.stdout: "take": 119176, 2024-09-11T00:32:34.228 INFO:teuthology.orchestra.run.smithi165.stdout: "take_sum": 119176, 2024-09-11T00:32:34.228 INFO:teuthology.orchestra.run.smithi165.stdout: "put": 119176, 2024-09-11T00:32:34.228 INFO:teuthology.orchestra.run.smithi165.stdout: "put_sum": 119176, 2024-09-11T00:32:34.228 INFO:teuthology.orchestra.run.smithi165.stdout: "wait": { 2024-09-11T00:32:34.228 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-09-11T00:32:34.228 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-09-11T00:32:34.228 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-09-11T00:32:34.228 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-11T00:32:34.228 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-11T00:32:34.228 INFO:teuthology.orchestra.run.smithi165.stdout: "throttle-write_buf_throttle": { 2024-09-11T00:32:34.228 INFO:teuthology.orchestra.run.smithi165.stdout: "val": 0, 2024-09-11T00:32:34.228 INFO:teuthology.orchestra.run.smithi165.stdout: "max": 3758096384, 2024-09-11T00:32:34.229 INFO:teuthology.orchestra.run.smithi165.stdout: "get_started": 0, 2024-09-11T00:32:34.229 INFO:teuthology.orchestra.run.smithi165.stdout: "get": 19277, 2024-09-11T00:32:34.229 INFO:teuthology.orchestra.run.smithi165.stdout: "get_sum": 2100985, 2024-09-11T00:32:34.229 INFO:teuthology.orchestra.run.smithi165.stdout: "get_or_fail_fail": 0, 2024-09-11T00:32:34.229 INFO:teuthology.orchestra.run.smithi165.stdout: "get_or_fail_success": 19277, 2024-09-11T00:32:34.229 INFO:teuthology.orchestra.run.smithi165.stdout: "take": 0, 2024-09-11T00:32:34.229 INFO:teuthology.orchestra.run.smithi165.stdout: "take_sum": 0, 2024-09-11T00:32:34.229 INFO:teuthology.orchestra.run.smithi165.stdout: "put": 755, 2024-09-11T00:32:34.229 INFO:teuthology.orchestra.run.smithi165.stdout: "put_sum": 2100985, 2024-09-11T00:32:34.229 INFO:teuthology.orchestra.run.smithi165.stdout: "wait": { 2024-09-11T00:32:34.229 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-09-11T00:32:34.229 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-09-11T00:32:34.229 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-09-11T00:32:34.230 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-11T00:32:34.230 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-11T00:32:34.230 INFO:teuthology.orchestra.run.smithi165.stdout: "throttle-write_buf_throttle-0x556e76e7cdc0": { 2024-09-11T00:32:34.230 INFO:teuthology.orchestra.run.smithi165.stdout: "val": 0, 2024-09-11T00:32:34.230 INFO:teuthology.orchestra.run.smithi165.stdout: "max": 3758096384, 2024-09-11T00:32:34.230 INFO:teuthology.orchestra.run.smithi165.stdout: "get_started": 0, 2024-09-11T00:32:34.230 INFO:teuthology.orchestra.run.smithi165.stdout: "get": 121921, 2024-09-11T00:32:34.230 INFO:teuthology.orchestra.run.smithi165.stdout: "get_sum": 252307225, 2024-09-11T00:32:34.230 INFO:teuthology.orchestra.run.smithi165.stdout: "get_or_fail_fail": 0, 2024-09-11T00:32:34.230 INFO:teuthology.orchestra.run.smithi165.stdout: "get_or_fail_success": 121921, 2024-09-11T00:32:34.230 INFO:teuthology.orchestra.run.smithi165.stdout: "take": 0, 2024-09-11T00:32:34.230 INFO:teuthology.orchestra.run.smithi165.stdout: "take_sum": 0, 2024-09-11T00:32:34.230 INFO:teuthology.orchestra.run.smithi165.stdout: "put": 52276, 2024-09-11T00:32:34.230 INFO:teuthology.orchestra.run.smithi165.stdout: "put_sum": 252307225, 2024-09-11T00:32:34.231 INFO:teuthology.orchestra.run.smithi165.stdout: "wait": { 2024-09-11T00:32:34.231 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-09-11T00:32:34.231 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-09-11T00:32:34.231 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-09-11T00:32:34.231 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-11T00:32:34.231 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-11T00:32:34.231 INFO:teuthology.orchestra.run.smithi165.stdout: "trackedop": { 2024-09-11T00:32:34.231 INFO:teuthology.orchestra.run.smithi165.stdout: "slow_ops_count": 2091 2024-09-11T00:32:34.231 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-11T00:32:34.231 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-09-11T00:32:34.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:32:34 smithi012 ceph-mon[30333]: pgmap v1164: 129 pgs: 129 active+clean; 671 MiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 8.8 KiB/s wr, 19 op/s 2024-09-11T00:32:34.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:32:34 smithi178 ceph-mon[35919]: pgmap v1164: 129 pgs: 129 active+clean; 671 MiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 8.8 KiB/s wr, 19 op/s 2024-09-11T00:32:35.335 INFO:tasks.check_counter:Daemon mds.b mds.exported=31 2024-09-11T00:32:35.335 INFO:tasks.check_counter:Daemon mds.b mds.imported=31 2024-09-11T00:32:35.335 DEBUG:tasks.check_counter:Getting stats from e 2024-09-11T00:32:35.335 DEBUG:teuthology.orchestra.run.smithi165:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid ff3ad74a-6fcf-11ef-bcea-c7b262605968 -- ceph daemon mds.e perf dump 2024-09-11T00:32:35.595 INFO:teuthology.orchestra.run.smithi165.stderr:Inferring config /var/lib/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T00:32:36.142 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:32:36 smithi165 ceph-mon[35356]: pgmap v1165: 129 pgs: 129 active+clean; 671 MiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 2.7 KiB/s wr, 12 op/s 2024-09-11T00:32:36.214 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-09-11T00:32:36.214 INFO:teuthology.orchestra.run.smithi165.stdout: "AsyncMessenger::Worker-0": { 2024-09-11T00:32:36.214 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_recv_messages": 106976, 2024-09-11T00:32:36.214 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_messages": 88784, 2024-09-11T00:32:36.214 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_recv_bytes": 29860599, 2024-09-11T00:32:36.215 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_bytes": 70792455, 2024-09-11T00:32:36.215 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_created_connections": 11, 2024-09-11T00:32:36.215 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_active_connections": 6, 2024-09-11T00:32:36.215 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_total_time": 8.502951994, 2024-09-11T00:32:36.215 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_send_time": 4.285137904, 2024-09-11T00:32:36.215 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_recv_time": 4.209460295, 2024-09-11T00:32:36.215 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_fast_dispatch_time": 0.735446686, 2024-09-11T00:32:36.215 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_messages_queue_lat": { 2024-09-11T00:32:36.215 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 88783, 2024-09-11T00:32:36.215 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 5.008442906, 2024-09-11T00:32:36.215 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000056412 2024-09-11T00:32:36.215 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-11T00:32:36.215 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_handle_ack_lat": { 2024-09-11T00:32:36.216 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 157320, 2024-09-11T00:32:36.216 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.027759877, 2024-09-11T00:32:36.216 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000176 2024-09-11T00:32:36.216 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-11T00:32:36.216 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_recv_encrypted_bytes": 2944, 2024-09-11T00:32:36.216 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_encrypted_bytes": 448 2024-09-11T00:32:36.216 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-11T00:32:36.216 INFO:teuthology.orchestra.run.smithi165.stdout: "AsyncMessenger::Worker-1": { 2024-09-11T00:32:36.216 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_recv_messages": 169626, 2024-09-11T00:32:36.216 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_messages": 246729, 2024-09-11T00:32:36.216 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_recv_bytes": 53552397, 2024-09-11T00:32:36.216 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_bytes": 219024780, 2024-09-11T00:32:36.216 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_created_connections": 13, 2024-09-11T00:32:36.217 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_active_connections": 7, 2024-09-11T00:32:36.217 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_total_time": 14.755315828, 2024-09-11T00:32:36.217 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_send_time": 8.178156248, 2024-09-11T00:32:36.217 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_recv_time": 6.282571223, 2024-09-11T00:32:36.217 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_fast_dispatch_time": 0.861515373, 2024-09-11T00:32:36.217 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_messages_queue_lat": { 2024-09-11T00:32:36.217 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 246728, 2024-09-11T00:32:36.217 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 18.506151907, 2024-09-11T00:32:36.217 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000075006 2024-09-11T00:32:36.217 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-11T00:32:36.217 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_handle_ack_lat": { 2024-09-11T00:32:36.217 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 192859, 2024-09-11T00:32:36.217 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.070964419, 2024-09-11T00:32:36.218 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000367 2024-09-11T00:32:36.218 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-11T00:32:36.218 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_recv_encrypted_bytes": 1194992, 2024-09-11T00:32:36.218 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_encrypted_bytes": 295504 2024-09-11T00:32:36.218 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-11T00:32:36.218 INFO:teuthology.orchestra.run.smithi165.stdout: "AsyncMessenger::Worker-2": { 2024-09-11T00:32:36.218 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_recv_messages": 38407, 2024-09-11T00:32:36.218 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_messages": 38356, 2024-09-11T00:32:36.218 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_recv_bytes": 9925765, 2024-09-11T00:32:36.218 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_bytes": 80840155, 2024-09-11T00:32:36.218 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_created_connections": 13, 2024-09-11T00:32:36.218 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_active_connections": 8, 2024-09-11T00:32:36.218 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_total_time": 3.960009300, 2024-09-11T00:32:36.219 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_send_time": 1.936839966, 2024-09-11T00:32:36.219 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_recv_time": 1.245403629, 2024-09-11T00:32:36.219 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_fast_dispatch_time": 0.775801505, 2024-09-11T00:32:36.219 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_messages_queue_lat": { 2024-09-11T00:32:36.219 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 38354, 2024-09-11T00:32:36.219 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 2.931950961, 2024-09-11T00:32:36.219 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000076444 2024-09-11T00:32:36.219 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-11T00:32:36.219 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_handle_ack_lat": { 2024-09-11T00:32:36.219 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 33099, 2024-09-11T00:32:36.219 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.008203082, 2024-09-11T00:32:36.219 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000247 2024-09-11T00:32:36.220 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-11T00:32:36.220 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_recv_encrypted_bytes": 160, 2024-09-11T00:32:36.220 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_encrypted_bytes": 700576 2024-09-11T00:32:36.220 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-11T00:32:36.220 INFO:teuthology.orchestra.run.smithi165.stdout: "cct": { 2024-09-11T00:32:36.220 INFO:teuthology.orchestra.run.smithi165.stdout: "total_workers": 1, 2024-09-11T00:32:36.220 INFO:teuthology.orchestra.run.smithi165.stdout: "unhealthy_workers": 0 2024-09-11T00:32:36.220 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-11T00:32:36.220 INFO:teuthology.orchestra.run.smithi165.stdout: "finisher-MDSRank": { 2024-09-11T00:32:36.220 INFO:teuthology.orchestra.run.smithi165.stdout: "queue_len": 0, 2024-09-11T00:32:36.220 INFO:teuthology.orchestra.run.smithi165.stdout: "complete_latency": { 2024-09-11T00:32:36.220 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 43843, 2024-09-11T00:32:36.220 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 57.152583418, 2024-09-11T00:32:36.221 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.001303573 2024-09-11T00:32:36.221 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-11T00:32:36.221 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-11T00:32:36.221 INFO:teuthology.orchestra.run.smithi165.stdout: "finisher-PurgeQueue": { 2024-09-11T00:32:36.221 INFO:teuthology.orchestra.run.smithi165.stdout: "queue_len": 0, 2024-09-11T00:32:36.221 INFO:teuthology.orchestra.run.smithi165.stdout: "complete_latency": { 2024-09-11T00:32:36.221 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 30772, 2024-09-11T00:32:36.221 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 2.752668960, 2024-09-11T00:32:36.221 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000089453 2024-09-11T00:32:36.221 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-11T00:32:36.221 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-11T00:32:36.221 INFO:teuthology.orchestra.run.smithi165.stdout: "mds": { 2024-09-11T00:32:36.221 INFO:teuthology.orchestra.run.smithi165.stdout: "request": 86787, 2024-09-11T00:32:36.222 INFO:teuthology.orchestra.run.smithi165.stdout: "reply": 83975, 2024-09-11T00:32:36.222 INFO:teuthology.orchestra.run.smithi165.stdout: "reply_latency": { 2024-09-11T00:32:36.222 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 83975, 2024-09-11T00:32:36.222 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 831.068518376, 2024-09-11T00:32:36.222 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.009896618 2024-09-11T00:32:36.222 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-11T00:32:36.222 INFO:teuthology.orchestra.run.smithi165.stdout: "slow_reply": 0, 2024-09-11T00:32:36.222 INFO:teuthology.orchestra.run.smithi165.stdout: "forward": 2812, 2024-09-11T00:32:36.222 INFO:teuthology.orchestra.run.smithi165.stdout: "dir_fetch_complete": 2, 2024-09-11T00:32:36.222 INFO:teuthology.orchestra.run.smithi165.stdout: "dir_fetch_keys": 0, 2024-09-11T00:32:36.222 INFO:teuthology.orchestra.run.smithi165.stdout: "dir_commit": 11, 2024-09-11T00:32:36.222 INFO:teuthology.orchestra.run.smithi165.stdout: "dir_split": 0, 2024-09-11T00:32:36.222 INFO:teuthology.orchestra.run.smithi165.stdout: "dir_merge": 0, 2024-09-11T00:32:36.222 INFO:teuthology.orchestra.run.smithi165.stdout: "inodes": 33, 2024-09-11T00:32:36.223 INFO:teuthology.orchestra.run.smithi165.stdout: "inodes_top": 6, 2024-09-11T00:32:36.223 INFO:teuthology.orchestra.run.smithi165.stdout: "inodes_bottom": 3, 2024-09-11T00:32:36.223 INFO:teuthology.orchestra.run.smithi165.stdout: "inodes_pin_tail": 24, 2024-09-11T00:32:36.223 INFO:teuthology.orchestra.run.smithi165.stdout: "inodes_pinned": 24, 2024-09-11T00:32:36.223 INFO:teuthology.orchestra.run.smithi165.stdout: "inodes_expired": 312, 2024-09-11T00:32:36.223 INFO:teuthology.orchestra.run.smithi165.stdout: "inodes_with_caps": 1, 2024-09-11T00:32:36.223 INFO:teuthology.orchestra.run.smithi165.stdout: "caps": 1, 2024-09-11T00:32:36.223 INFO:teuthology.orchestra.run.smithi165.stdout: "subtrees": 5, 2024-09-11T00:32:36.224 INFO:teuthology.orchestra.run.smithi165.stdout: "traverse": 136094, 2024-09-11T00:32:36.224 INFO:teuthology.orchestra.run.smithi165.stdout: "traverse_hit": 79885, 2024-09-11T00:32:36.224 INFO:teuthology.orchestra.run.smithi165.stdout: "traverse_forward": 2810, 2024-09-11T00:32:36.224 INFO:teuthology.orchestra.run.smithi165.stdout: "traverse_discover": 47, 2024-09-11T00:32:36.224 INFO:teuthology.orchestra.run.smithi165.stdout: "traverse_dir_fetch": 0, 2024-09-11T00:32:36.224 INFO:teuthology.orchestra.run.smithi165.stdout: "traverse_remote_ino": 0, 2024-09-11T00:32:36.224 INFO:teuthology.orchestra.run.smithi165.stdout: "traverse_lock": 1229, 2024-09-11T00:32:36.224 INFO:teuthology.orchestra.run.smithi165.stdout: "load_cent": 0, 2024-09-11T00:32:36.224 INFO:teuthology.orchestra.run.smithi165.stdout: "q": 0, 2024-09-11T00:32:36.225 INFO:teuthology.orchestra.run.smithi165.stdout: "exported": 12, 2024-09-11T00:32:36.225 INFO:teuthology.orchestra.run.smithi165.stdout: "exported_inodes": 203, 2024-09-11T00:32:36.225 INFO:teuthology.orchestra.run.smithi165.stdout: "imported": 12, 2024-09-11T00:32:36.225 INFO:teuthology.orchestra.run.smithi165.stdout: "imported_inodes": 453, 2024-09-11T00:32:36.225 INFO:teuthology.orchestra.run.smithi165.stdout: "openino_dir_fetch": 0, 2024-09-11T00:32:36.225 INFO:teuthology.orchestra.run.smithi165.stdout: "openino_backtrace_fetch": 0, 2024-09-11T00:32:36.225 INFO:teuthology.orchestra.run.smithi165.stdout: "openino_peer_discover": 0, 2024-09-11T00:32:36.225 INFO:teuthology.orchestra.run.smithi165.stdout: "root_rfiles": 4, 2024-09-11T00:32:36.225 INFO:teuthology.orchestra.run.smithi165.stdout: "root_rbytes": 234, 2024-09-11T00:32:36.225 INFO:teuthology.orchestra.run.smithi165.stdout: "root_rsnaps": 0, 2024-09-11T00:32:36.225 INFO:teuthology.orchestra.run.smithi165.stdout: "scrub_backtrace_fetch": 0, 2024-09-11T00:32:36.225 INFO:teuthology.orchestra.run.smithi165.stdout: "scrub_set_tag": 0, 2024-09-11T00:32:36.226 INFO:teuthology.orchestra.run.smithi165.stdout: "scrub_backtrace_repaired": 0, 2024-09-11T00:32:36.226 INFO:teuthology.orchestra.run.smithi165.stdout: "scrub_inotable_repaired": 0, 2024-09-11T00:32:36.226 INFO:teuthology.orchestra.run.smithi165.stdout: "scrub_dir_inodes": 0, 2024-09-11T00:32:36.226 INFO:teuthology.orchestra.run.smithi165.stdout: "scrub_dir_base_inodes": 0, 2024-09-11T00:32:36.226 INFO:teuthology.orchestra.run.smithi165.stdout: "scrub_dirfrag_rstats": 0, 2024-09-11T00:32:36.226 INFO:teuthology.orchestra.run.smithi165.stdout: "scrub_file_inodes": 0, 2024-09-11T00:32:36.226 INFO:teuthology.orchestra.run.smithi165.stdout: "handle_inode_file_caps": 76, 2024-09-11T00:32:36.226 INFO:teuthology.orchestra.run.smithi165.stdout: "ceph_cap_op_revoke": 8017, 2024-09-11T00:32:36.226 INFO:teuthology.orchestra.run.smithi165.stdout: "ceph_cap_op_grant": 66385, 2024-09-11T00:32:36.226 INFO:teuthology.orchestra.run.smithi165.stdout: "ceph_cap_op_trunc": 0, 2024-09-11T00:32:36.226 INFO:teuthology.orchestra.run.smithi165.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-11T00:32:36.226 INFO:teuthology.orchestra.run.smithi165.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-11T00:32:36.226 INFO:teuthology.orchestra.run.smithi165.stdout: "handle_client_caps": 40981, 2024-09-11T00:32:36.227 INFO:teuthology.orchestra.run.smithi165.stdout: "handle_client_caps_dirty": 11820, 2024-09-11T00:32:36.227 INFO:teuthology.orchestra.run.smithi165.stdout: "handle_client_cap_release": 2014, 2024-09-11T00:32:36.227 INFO:teuthology.orchestra.run.smithi165.stdout: "process_request_cap_release": 54944 2024-09-11T00:32:36.227 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-11T00:32:36.227 INFO:teuthology.orchestra.run.smithi165.stdout: "mds_cache": { 2024-09-11T00:32:36.227 INFO:teuthology.orchestra.run.smithi165.stdout: "dir_update": 37, 2024-09-11T00:32:36.227 INFO:teuthology.orchestra.run.smithi165.stdout: "dir_update_receipt": 154, 2024-09-11T00:32:36.227 INFO:teuthology.orchestra.run.smithi165.stdout: "dir_try_discover": 94, 2024-09-11T00:32:36.227 INFO:teuthology.orchestra.run.smithi165.stdout: "dir_send_discover": 161, 2024-09-11T00:32:36.227 INFO:teuthology.orchestra.run.smithi165.stdout: "dir_handle_discover": 105, 2024-09-11T00:32:36.227 INFO:teuthology.orchestra.run.smithi165.stdout: "num_strays": 2, 2024-09-11T00:32:36.227 INFO:teuthology.orchestra.run.smithi165.stdout: "num_strays_delayed": 0, 2024-09-11T00:32:36.227 INFO:teuthology.orchestra.run.smithi165.stdout: "num_strays_enqueuing": 0, 2024-09-11T00:32:36.228 INFO:teuthology.orchestra.run.smithi165.stdout: "strays_created": 15530, 2024-09-11T00:32:36.228 INFO:teuthology.orchestra.run.smithi165.stdout: "strays_enqueued": 15528, 2024-09-11T00:32:36.228 INFO:teuthology.orchestra.run.smithi165.stdout: "strays_reintegrated": 0, 2024-09-11T00:32:36.228 INFO:teuthology.orchestra.run.smithi165.stdout: "strays_migrated": 0, 2024-09-11T00:32:36.228 INFO:teuthology.orchestra.run.smithi165.stdout: "num_recovering_processing": 0, 2024-09-11T00:32:36.228 INFO:teuthology.orchestra.run.smithi165.stdout: "num_recovering_enqueued": 0, 2024-09-11T00:32:36.228 INFO:teuthology.orchestra.run.smithi165.stdout: "num_recovering_prioritized": 0, 2024-09-11T00:32:36.228 INFO:teuthology.orchestra.run.smithi165.stdout: "recovery_started": 0, 2024-09-11T00:32:36.228 INFO:teuthology.orchestra.run.smithi165.stdout: "recovery_completed": 0, 2024-09-11T00:32:36.228 INFO:teuthology.orchestra.run.smithi165.stdout: "ireq_quiesce_path": 19, 2024-09-11T00:32:36.228 INFO:teuthology.orchestra.run.smithi165.stdout: "ireq_quiesce_inode": 1945, 2024-09-11T00:32:36.228 INFO:teuthology.orchestra.run.smithi165.stdout: "ireq_enqueue_scrub": 0, 2024-09-11T00:32:36.228 INFO:teuthology.orchestra.run.smithi165.stdout: "ireq_exportdir": 14, 2024-09-11T00:32:36.229 INFO:teuthology.orchestra.run.smithi165.stdout: "ireq_flush": 0, 2024-09-11T00:32:36.229 INFO:teuthology.orchestra.run.smithi165.stdout: "ireq_fragmentdir": 0, 2024-09-11T00:32:36.229 INFO:teuthology.orchestra.run.smithi165.stdout: "ireq_fragstats": 0, 2024-09-11T00:32:36.229 INFO:teuthology.orchestra.run.smithi165.stdout: "ireq_inodestats": 0, 2024-09-11T00:32:36.229 INFO:teuthology.orchestra.run.smithi165.stdout: "uninline_started": 0, 2024-09-11T00:32:36.229 INFO:teuthology.orchestra.run.smithi165.stdout: "uninline_succeeded": 0, 2024-09-11T00:32:36.229 INFO:teuthology.orchestra.run.smithi165.stdout: "uninline_write_failed": 0 2024-09-11T00:32:36.229 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-11T00:32:36.229 INFO:teuthology.orchestra.run.smithi165.stdout: "mds_log": { 2024-09-11T00:32:36.229 INFO:teuthology.orchestra.run.smithi165.stdout: "evlrg": 0, 2024-09-11T00:32:36.229 INFO:teuthology.orchestra.run.smithi165.stdout: "evadd": 78981, 2024-09-11T00:32:36.229 INFO:teuthology.orchestra.run.smithi165.stdout: "evex": 0, 2024-09-11T00:32:36.229 INFO:teuthology.orchestra.run.smithi165.stdout: "evtrm": 0, 2024-09-11T00:32:36.229 INFO:teuthology.orchestra.run.smithi165.stdout: "ev": 78981, 2024-09-11T00:32:36.230 INFO:teuthology.orchestra.run.smithi165.stdout: "evexg": 0, 2024-09-11T00:32:36.230 INFO:teuthology.orchestra.run.smithi165.stdout: "evexd": 0, 2024-09-11T00:32:36.230 INFO:teuthology.orchestra.run.smithi165.stdout: "segadd": 100, 2024-09-11T00:32:36.230 INFO:teuthology.orchestra.run.smithi165.stdout: "segex": 0, 2024-09-11T00:32:36.230 INFO:teuthology.orchestra.run.smithi165.stdout: "segtrm": 0, 2024-09-11T00:32:36.230 INFO:teuthology.orchestra.run.smithi165.stdout: "seg": 100, 2024-09-11T00:32:36.230 INFO:teuthology.orchestra.run.smithi165.stdout: "segmjr": 8, 2024-09-11T00:32:36.230 INFO:teuthology.orchestra.run.smithi165.stdout: "segexg": 0, 2024-09-11T00:32:36.230 INFO:teuthology.orchestra.run.smithi165.stdout: "segexd": 0, 2024-09-11T00:32:36.230 INFO:teuthology.orchestra.run.smithi165.stdout: "expos": 4194304, 2024-09-11T00:32:36.230 INFO:teuthology.orchestra.run.smithi165.stdout: "wrpos": 159306380, 2024-09-11T00:32:36.230 INFO:teuthology.orchestra.run.smithi165.stdout: "rdpos": 0, 2024-09-11T00:32:36.230 INFO:teuthology.orchestra.run.smithi165.stdout: "jlat": { 2024-09-11T00:32:36.230 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 24672, 2024-09-11T00:32:36.231 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 236.811954135, 2024-09-11T00:32:36.231 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.009598409 2024-09-11T00:32:36.231 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-11T00:32:36.231 INFO:teuthology.orchestra.run.smithi165.stdout: "replayed": 0 2024-09-11T00:32:36.231 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-11T00:32:36.231 INFO:teuthology.orchestra.run.smithi165.stdout: "mds_mem": { 2024-09-11T00:32:36.231 INFO:teuthology.orchestra.run.smithi165.stdout: "ino": 27, 2024-09-11T00:32:36.231 INFO:teuthology.orchestra.run.smithi165.stdout: "ino+": 15684, 2024-09-11T00:32:36.231 INFO:teuthology.orchestra.run.smithi165.stdout: "ino-": 15657, 2024-09-11T00:32:36.231 INFO:teuthology.orchestra.run.smithi165.stdout: "dir": 25, 2024-09-11T00:32:36.231 INFO:teuthology.orchestra.run.smithi165.stdout: "dir+": 147, 2024-09-11T00:32:36.231 INFO:teuthology.orchestra.run.smithi165.stdout: "dir-": 122, 2024-09-11T00:32:36.231 INFO:teuthology.orchestra.run.smithi165.stdout: "dn": 58, 2024-09-11T00:32:36.231 INFO:teuthology.orchestra.run.smithi165.stdout: "dn+": 29837, 2024-09-11T00:32:36.231 INFO:teuthology.orchestra.run.smithi165.stdout: "dn-": 29779, 2024-09-11T00:32:36.232 INFO:teuthology.orchestra.run.smithi165.stdout: "cap": 1, 2024-09-11T00:32:36.232 INFO:teuthology.orchestra.run.smithi165.stdout: "cap+": 17782, 2024-09-11T00:32:36.232 INFO:teuthology.orchestra.run.smithi165.stdout: "cap-": 17781, 2024-09-11T00:32:36.232 INFO:teuthology.orchestra.run.smithi165.stdout: "rss": 75316, 2024-09-11T00:32:36.232 INFO:teuthology.orchestra.run.smithi165.stdout: "heap": 281927680 2024-09-11T00:32:36.232 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-11T00:32:36.232 INFO:teuthology.orchestra.run.smithi165.stdout: "mds_server": { 2024-09-11T00:32:36.232 INFO:teuthology.orchestra.run.smithi165.stdout: "dispatch_client_request": 122348, 2024-09-11T00:32:36.232 INFO:teuthology.orchestra.run.smithi165.stdout: "dispatch_server_request": 1870, 2024-09-11T00:32:36.232 INFO:teuthology.orchestra.run.smithi165.stdout: "handle_client_request": 86787, 2024-09-11T00:32:36.232 INFO:teuthology.orchestra.run.smithi165.stdout: "handle_client_session": 1594, 2024-09-11T00:32:36.232 INFO:teuthology.orchestra.run.smithi165.stdout: "handle_peer_request": 6907, 2024-09-11T00:32:36.233 INFO:teuthology.orchestra.run.smithi165.stdout: "req_create_latency": { 2024-09-11T00:32:36.233 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 15394, 2024-09-11T00:32:36.233 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 276.356933468, 2024-09-11T00:32:36.233 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.017952249 2024-09-11T00:32:36.233 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-11T00:32:36.233 INFO:teuthology.orchestra.run.smithi165.stdout: "req_getattr_latency": { 2024-09-11T00:32:36.233 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 5433, 2024-09-11T00:32:36.233 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 73.718741822, 2024-09-11T00:32:36.233 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.013568699 2024-09-11T00:32:36.233 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-11T00:32:36.233 INFO:teuthology.orchestra.run.smithi165.stdout: "req_getfilelock_latency": { 2024-09-11T00:32:36.233 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-09-11T00:32:36.233 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-09-11T00:32:36.233 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-09-11T00:32:36.234 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-11T00:32:36.234 INFO:teuthology.orchestra.run.smithi165.stdout: "req_link_latency": { 2024-09-11T00:32:36.234 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-09-11T00:32:36.234 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-09-11T00:32:36.234 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-09-11T00:32:36.234 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-11T00:32:36.234 INFO:teuthology.orchestra.run.smithi165.stdout: "req_lookup_latency": { 2024-09-11T00:32:36.234 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 21955, 2024-09-11T00:32:36.234 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 26.126231131, 2024-09-11T00:32:36.234 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.001189990 2024-09-11T00:32:36.234 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-11T00:32:36.234 INFO:teuthology.orchestra.run.smithi165.stdout: "req_lookuphash_latency": { 2024-09-11T00:32:36.234 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-09-11T00:32:36.234 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-09-11T00:32:36.235 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-09-11T00:32:36.235 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-11T00:32:36.235 INFO:teuthology.orchestra.run.smithi165.stdout: "req_lookupino_latency": { 2024-09-11T00:32:36.235 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 1, 2024-09-11T00:32:36.235 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000109869, 2024-09-11T00:32:36.235 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000109869 2024-09-11T00:32:36.235 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-11T00:32:36.235 INFO:teuthology.orchestra.run.smithi165.stdout: "req_lookupname_latency": { 2024-09-11T00:32:36.235 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-09-11T00:32:36.235 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-09-11T00:32:36.235 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-09-11T00:32:36.235 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-11T00:32:36.235 INFO:teuthology.orchestra.run.smithi165.stdout: "req_lookupparent_latency": { 2024-09-11T00:32:36.235 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-09-11T00:32:36.236 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-09-11T00:32:36.236 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-09-11T00:32:36.236 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-11T00:32:36.236 INFO:teuthology.orchestra.run.smithi165.stdout: "req_lookupsnap_latency": { 2024-09-11T00:32:36.236 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-09-11T00:32:36.236 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-09-11T00:32:36.236 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-09-11T00:32:36.236 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-11T00:32:36.236 INFO:teuthology.orchestra.run.smithi165.stdout: "req_lssnap_latency": { 2024-09-11T00:32:36.236 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-09-11T00:32:36.236 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-09-11T00:32:36.236 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-09-11T00:32:36.236 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-11T00:32:36.236 INFO:teuthology.orchestra.run.smithi165.stdout: "req_mkdir_latency": { 2024-09-11T00:32:36.237 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-09-11T00:32:36.237 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-09-11T00:32:36.237 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-09-11T00:32:36.237 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-11T00:32:36.237 INFO:teuthology.orchestra.run.smithi165.stdout: "req_mknod_latency": { 2024-09-11T00:32:36.237 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-09-11T00:32:36.237 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-09-11T00:32:36.237 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-09-11T00:32:36.237 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-11T00:32:36.237 INFO:teuthology.orchestra.run.smithi165.stdout: "req_mksnap_latency": { 2024-09-11T00:32:36.237 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-09-11T00:32:36.237 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-09-11T00:32:36.237 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-09-11T00:32:36.237 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-11T00:32:36.238 INFO:teuthology.orchestra.run.smithi165.stdout: "req_open_latency": { 2024-09-11T00:32:36.238 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 3, 2024-09-11T00:32:36.238 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.001642754, 2024-09-11T00:32:36.238 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000547584 2024-09-11T00:32:36.238 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-11T00:32:36.238 INFO:teuthology.orchestra.run.smithi165.stdout: "req_readdir_latency": { 2024-09-11T00:32:36.238 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 21864, 2024-09-11T00:32:36.238 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 147.610049050, 2024-09-11T00:32:36.238 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.006751282 2024-09-11T00:32:36.238 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-11T00:32:36.238 INFO:teuthology.orchestra.run.smithi165.stdout: "req_rename_latency": { 2024-09-11T00:32:36.238 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 3208, 2024-09-11T00:32:36.238 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 96.076946189, 2024-09-11T00:32:36.239 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.029949172 2024-09-11T00:32:36.239 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-11T00:32:36.239 INFO:teuthology.orchestra.run.smithi165.stdout: "req_renamesnap_latency": { 2024-09-11T00:32:36.239 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-09-11T00:32:36.239 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-09-11T00:32:36.239 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-09-11T00:32:36.239 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-11T00:32:36.239 INFO:teuthology.orchestra.run.smithi165.stdout: "req_snapdiff_latency": { 2024-09-11T00:32:36.239 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-09-11T00:32:36.239 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-09-11T00:32:36.239 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-09-11T00:32:36.239 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-11T00:32:36.239 INFO:teuthology.orchestra.run.smithi165.stdout: "req_rmdir_latency": { 2024-09-11T00:32:36.239 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 20, 2024-09-11T00:32:36.239 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.181902860, 2024-09-11T00:32:36.240 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.009095143 2024-09-11T00:32:36.240 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-11T00:32:36.240 INFO:teuthology.orchestra.run.smithi165.stdout: "req_rmsnap_latency": { 2024-09-11T00:32:36.240 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-09-11T00:32:36.240 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-09-11T00:32:36.240 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-09-11T00:32:36.240 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-11T00:32:36.240 INFO:teuthology.orchestra.run.smithi165.stdout: "req_rmxattr_latency": { 2024-09-11T00:32:36.240 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-09-11T00:32:36.240 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-09-11T00:32:36.240 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-09-11T00:32:36.240 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-11T00:32:36.240 INFO:teuthology.orchestra.run.smithi165.stdout: "req_setattr_latency": { 2024-09-11T00:32:36.241 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 119, 2024-09-11T00:32:36.241 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 53.620798366, 2024-09-11T00:32:36.241 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.450594944 2024-09-11T00:32:36.241 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-11T00:32:36.241 INFO:teuthology.orchestra.run.smithi165.stdout: "req_setdirlayout_latency": { 2024-09-11T00:32:36.241 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-09-11T00:32:36.241 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-09-11T00:32:36.241 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-09-11T00:32:36.241 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-11T00:32:36.241 INFO:teuthology.orchestra.run.smithi165.stdout: "req_setfilelock_latency": { 2024-09-11T00:32:36.241 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 468, 2024-09-11T00:32:36.241 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.463250543, 2024-09-11T00:32:36.241 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000989851 2024-09-11T00:32:36.241 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-11T00:32:36.242 INFO:teuthology.orchestra.run.smithi165.stdout: "req_setlayout_latency": { 2024-09-11T00:32:36.242 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-09-11T00:32:36.242 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-09-11T00:32:36.242 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-09-11T00:32:36.242 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-11T00:32:36.242 INFO:teuthology.orchestra.run.smithi165.stdout: "req_setxattr_latency": { 2024-09-11T00:32:36.242 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-09-11T00:32:36.242 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-09-11T00:32:36.242 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-09-11T00:32:36.242 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-11T00:32:36.242 INFO:teuthology.orchestra.run.smithi165.stdout: "req_symlink_latency": { 2024-09-11T00:32:36.242 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-09-11T00:32:36.242 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-09-11T00:32:36.242 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-09-11T00:32:36.243 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-11T00:32:36.243 INFO:teuthology.orchestra.run.smithi165.stdout: "req_unlink_latency": { 2024-09-11T00:32:36.243 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 15510, 2024-09-11T00:32:36.243 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 156.911912324, 2024-09-11T00:32:36.243 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.010116822 2024-09-11T00:32:36.243 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-11T00:32:36.243 INFO:teuthology.orchestra.run.smithi165.stdout: "cap_revoke_eviction": 0, 2024-09-11T00:32:36.243 INFO:teuthology.orchestra.run.smithi165.stdout: "cap_acquisition_throttle": 0, 2024-09-11T00:32:36.243 INFO:teuthology.orchestra.run.smithi165.stdout: "req_getvxattr_latency": { 2024-09-11T00:32:36.243 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-09-11T00:32:36.243 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-09-11T00:32:36.243 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-09-11T00:32:36.243 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-11T00:32:36.243 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-11T00:32:36.244 INFO:teuthology.orchestra.run.smithi165.stdout: "mds_sessions": { 2024-09-11T00:32:36.244 INFO:teuthology.orchestra.run.smithi165.stdout: "session_count": 1, 2024-09-11T00:32:36.244 INFO:teuthology.orchestra.run.smithi165.stdout: "session_add": 2, 2024-09-11T00:32:36.244 INFO:teuthology.orchestra.run.smithi165.stdout: "session_remove": 1, 2024-09-11T00:32:36.244 INFO:teuthology.orchestra.run.smithi165.stdout: "sessions_open": 1, 2024-09-11T00:32:36.244 INFO:teuthology.orchestra.run.smithi165.stdout: "sessions_stale": 0, 2024-09-11T00:32:36.244 INFO:teuthology.orchestra.run.smithi165.stdout: "total_load": 9546, 2024-09-11T00:32:36.244 INFO:teuthology.orchestra.run.smithi165.stdout: "average_load": 9546, 2024-09-11T00:32:36.244 INFO:teuthology.orchestra.run.smithi165.stdout: "avg_session_uptime": 1532, 2024-09-11T00:32:36.244 INFO:teuthology.orchestra.run.smithi165.stdout: "mdthresh_evicted": 0 2024-09-11T00:32:36.244 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-11T00:32:36.244 INFO:teuthology.orchestra.run.smithi165.stdout: "mempool": { 2024-09-11T00:32:36.244 INFO:teuthology.orchestra.run.smithi165.stdout: "bloom_filter_bytes": 0, 2024-09-11T00:32:36.245 INFO:teuthology.orchestra.run.smithi165.stdout: "bloom_filter_items": 0, 2024-09-11T00:32:36.245 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_alloc_bytes": 0, 2024-09-11T00:32:36.245 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_alloc_items": 0, 2024-09-11T00:32:36.245 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_cache_data_bytes": 0, 2024-09-11T00:32:36.245 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_cache_data_items": 0, 2024-09-11T00:32:36.245 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-11T00:32:36.245 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_cache_onode_items": 0, 2024-09-11T00:32:36.245 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-11T00:32:36.245 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_cache_meta_items": 0, 2024-09-11T00:32:36.245 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_cache_other_bytes": 0, 2024-09-11T00:32:36.245 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_cache_other_items": 0, 2024-09-11T00:32:36.245 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-11T00:32:36.245 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_cache_buffer_items": 0, 2024-09-11T00:32:36.246 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_extent_bytes": 0, 2024-09-11T00:32:36.246 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_extent_items": 0, 2024-09-11T00:32:36.246 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_blob_bytes": 0, 2024-09-11T00:32:36.246 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_blob_items": 0, 2024-09-11T00:32:36.246 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-11T00:32:36.246 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_shared_blob_items": 0, 2024-09-11T00:32:36.246 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-11T00:32:36.246 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_inline_bl_items": 0, 2024-09-11T00:32:36.246 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_fsck_bytes": 0, 2024-09-11T00:32:36.246 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_fsck_items": 0, 2024-09-11T00:32:36.246 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_txc_bytes": 0, 2024-09-11T00:32:36.246 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_txc_items": 0, 2024-09-11T00:32:36.246 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-11T00:32:36.246 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_writing_deferred_items": 0, 2024-09-11T00:32:36.247 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_writing_bytes": 0, 2024-09-11T00:32:36.247 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_writing_items": 0, 2024-09-11T00:32:36.247 INFO:teuthology.orchestra.run.smithi165.stdout: "bluefs_bytes": 0, 2024-09-11T00:32:36.247 INFO:teuthology.orchestra.run.smithi165.stdout: "bluefs_items": 0, 2024-09-11T00:32:36.247 INFO:teuthology.orchestra.run.smithi165.stdout: "bluefs_file_reader_bytes": 0, 2024-09-11T00:32:36.247 INFO:teuthology.orchestra.run.smithi165.stdout: "bluefs_file_reader_items": 0, 2024-09-11T00:32:36.247 INFO:teuthology.orchestra.run.smithi165.stdout: "bluefs_file_writer_bytes": 0, 2024-09-11T00:32:36.247 INFO:teuthology.orchestra.run.smithi165.stdout: "bluefs_file_writer_items": 0, 2024-09-11T00:32:36.247 INFO:teuthology.orchestra.run.smithi165.stdout: "buffer_anon_bytes": 305606, 2024-09-11T00:32:36.247 INFO:teuthology.orchestra.run.smithi165.stdout: "buffer_anon_items": 118, 2024-09-11T00:32:36.247 INFO:teuthology.orchestra.run.smithi165.stdout: "buffer_meta_bytes": 0, 2024-09-11T00:32:36.247 INFO:teuthology.orchestra.run.smithi165.stdout: "buffer_meta_items": 0, 2024-09-11T00:32:36.247 INFO:teuthology.orchestra.run.smithi165.stdout: "osd_bytes": 0, 2024-09-11T00:32:36.247 INFO:teuthology.orchestra.run.smithi165.stdout: "osd_items": 0, 2024-09-11T00:32:36.247 INFO:teuthology.orchestra.run.smithi165.stdout: "osd_mapbl_bytes": 0, 2024-09-11T00:32:36.248 INFO:teuthology.orchestra.run.smithi165.stdout: "osd_mapbl_items": 0, 2024-09-11T00:32:36.248 INFO:teuthology.orchestra.run.smithi165.stdout: "osd_pglog_bytes": 0, 2024-09-11T00:32:36.248 INFO:teuthology.orchestra.run.smithi165.stdout: "osd_pglog_items": 0, 2024-09-11T00:32:36.248 INFO:teuthology.orchestra.run.smithi165.stdout: "osdmap_bytes": 9048, 2024-09-11T00:32:36.248 INFO:teuthology.orchestra.run.smithi165.stdout: "osdmap_items": 170, 2024-09-11T00:32:36.248 INFO:teuthology.orchestra.run.smithi165.stdout: "osdmap_mapping_bytes": 0, 2024-09-11T00:32:36.248 INFO:teuthology.orchestra.run.smithi165.stdout: "osdmap_mapping_items": 0, 2024-09-11T00:32:36.248 INFO:teuthology.orchestra.run.smithi165.stdout: "pgmap_bytes": 0, 2024-09-11T00:32:36.248 INFO:teuthology.orchestra.run.smithi165.stdout: "pgmap_items": 0, 2024-09-11T00:32:36.248 INFO:teuthology.orchestra.run.smithi165.stdout: "mds_co_bytes": 141498, 2024-09-11T00:32:36.248 INFO:teuthology.orchestra.run.smithi165.stdout: "mds_co_items": 1297, 2024-09-11T00:32:36.248 INFO:teuthology.orchestra.run.smithi165.stdout: "unittest_1_bytes": 0, 2024-09-11T00:32:36.248 INFO:teuthology.orchestra.run.smithi165.stdout: "unittest_1_items": 0, 2024-09-11T00:32:36.248 INFO:teuthology.orchestra.run.smithi165.stdout: "unittest_2_bytes": 0, 2024-09-11T00:32:36.249 INFO:teuthology.orchestra.run.smithi165.stdout: "unittest_2_items": 0 2024-09-11T00:32:36.249 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-11T00:32:36.249 INFO:teuthology.orchestra.run.smithi165.stdout: "objecter": { 2024-09-11T00:32:36.249 INFO:teuthology.orchestra.run.smithi165.stdout: "op_active": 0, 2024-09-11T00:32:36.249 INFO:teuthology.orchestra.run.smithi165.stdout: "op_laggy": 0, 2024-09-11T00:32:36.249 INFO:teuthology.orchestra.run.smithi165.stdout: "op_send": 58179, 2024-09-11T00:32:36.249 INFO:teuthology.orchestra.run.smithi165.stdout: "op_send_bytes": 158529841, 2024-09-11T00:32:36.249 INFO:teuthology.orchestra.run.smithi165.stdout: "op_resend": 33, 2024-09-11T00:32:36.249 INFO:teuthology.orchestra.run.smithi165.stdout: "op_reply": 58179, 2024-09-11T00:32:36.249 INFO:teuthology.orchestra.run.smithi165.stdout: "op_latency": { 2024-09-11T00:32:36.249 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 58179, 2024-09-11T00:32:36.249 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 747.522946330, 2024-09-11T00:32:36.249 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.012848672 2024-09-11T00:32:36.250 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-11T00:32:36.250 INFO:teuthology.orchestra.run.smithi165.stdout: "op_inflight": 0, 2024-09-11T00:32:36.250 INFO:teuthology.orchestra.run.smithi165.stdout: "oplen_avg": { 2024-09-11T00:32:36.250 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 58179, 2024-09-11T00:32:36.250 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 59152 2024-09-11T00:32:36.250 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-11T00:32:36.250 INFO:teuthology.orchestra.run.smithi165.stdout: "op": 58179, 2024-09-11T00:32:36.250 INFO:teuthology.orchestra.run.smithi165.stdout: "op_r": 671, 2024-09-11T00:32:36.250 INFO:teuthology.orchestra.run.smithi165.stdout: "op_w": 57508, 2024-09-11T00:32:36.250 INFO:teuthology.orchestra.run.smithi165.stdout: "op_rmw": 0, 2024-09-11T00:32:36.250 INFO:teuthology.orchestra.run.smithi165.stdout: "op_pg": 0, 2024-09-11T00:32:36.250 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_stat": 0, 2024-09-11T00:32:36.250 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_create": 10, 2024-09-11T00:32:36.250 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_read": 669, 2024-09-11T00:32:36.251 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_write": 25341, 2024-09-11T00:32:36.251 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_writefull": 408, 2024-09-11T00:32:36.251 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_writesame": 0, 2024-09-11T00:32:36.251 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_append": 0, 2024-09-11T00:32:36.251 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_zero": 0, 2024-09-11T00:32:36.251 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_truncate": 0, 2024-09-11T00:32:36.251 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_delete": 31086, 2024-09-11T00:32:36.251 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_mapext": 0, 2024-09-11T00:32:36.251 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_sparse_read": 0, 2024-09-11T00:32:36.251 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_clonerange": 0, 2024-09-11T00:32:36.251 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_getxattr": 2, 2024-09-11T00:32:36.251 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_setxattr": 20, 2024-09-11T00:32:36.251 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_cmpxattr": 0, 2024-09-11T00:32:36.251 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_rmxattr": 0, 2024-09-11T00:32:36.251 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_resetxattrs": 0, 2024-09-11T00:32:36.252 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_call": 0, 2024-09-11T00:32:36.252 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_watch": 0, 2024-09-11T00:32:36.252 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_notify": 0, 2024-09-11T00:32:36.252 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_src_cmpxattr": 0, 2024-09-11T00:32:36.252 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_pgls": 0, 2024-09-11T00:32:36.252 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_pgls_filter": 0, 2024-09-11T00:32:36.252 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_other": 0, 2024-09-11T00:32:36.252 INFO:teuthology.orchestra.run.smithi165.stdout: "linger_active": 0, 2024-09-11T00:32:36.252 INFO:teuthology.orchestra.run.smithi165.stdout: "linger_send": 0, 2024-09-11T00:32:36.252 INFO:teuthology.orchestra.run.smithi165.stdout: "linger_resend": 0, 2024-09-11T00:32:36.252 INFO:teuthology.orchestra.run.smithi165.stdout: "linger_ping": 0, 2024-09-11T00:32:36.252 INFO:teuthology.orchestra.run.smithi165.stdout: "poolop_active": 0, 2024-09-11T00:32:36.252 INFO:teuthology.orchestra.run.smithi165.stdout: "poolop_send": 0, 2024-09-11T00:32:36.253 INFO:teuthology.orchestra.run.smithi165.stdout: "poolop_resend": 0, 2024-09-11T00:32:36.253 INFO:teuthology.orchestra.run.smithi165.stdout: "poolstat_active": 0, 2024-09-11T00:32:36.253 INFO:teuthology.orchestra.run.smithi165.stdout: "poolstat_send": 0, 2024-09-11T00:32:36.253 INFO:teuthology.orchestra.run.smithi165.stdout: "poolstat_resend": 0, 2024-09-11T00:32:36.253 INFO:teuthology.orchestra.run.smithi165.stdout: "statfs_active": 0, 2024-09-11T00:32:36.253 INFO:teuthology.orchestra.run.smithi165.stdout: "statfs_send": 0, 2024-09-11T00:32:36.253 INFO:teuthology.orchestra.run.smithi165.stdout: "statfs_resend": 0, 2024-09-11T00:32:36.253 INFO:teuthology.orchestra.run.smithi165.stdout: "command_active": 0, 2024-09-11T00:32:36.253 INFO:teuthology.orchestra.run.smithi165.stdout: "command_send": 0, 2024-09-11T00:32:36.253 INFO:teuthology.orchestra.run.smithi165.stdout: "command_resend": 0, 2024-09-11T00:32:36.253 INFO:teuthology.orchestra.run.smithi165.stdout: "map_epoch": 87, 2024-09-11T00:32:36.253 INFO:teuthology.orchestra.run.smithi165.stdout: "map_full": 0, 2024-09-11T00:32:36.253 INFO:teuthology.orchestra.run.smithi165.stdout: "map_inc": 6, 2024-09-11T00:32:36.253 INFO:teuthology.orchestra.run.smithi165.stdout: "osd_sessions": 12, 2024-09-11T00:32:36.254 INFO:teuthology.orchestra.run.smithi165.stdout: "osd_session_open": 12, 2024-09-11T00:32:36.254 INFO:teuthology.orchestra.run.smithi165.stdout: "osd_session_close": 0, 2024-09-11T00:32:36.254 INFO:teuthology.orchestra.run.smithi165.stdout: "osd_laggy": 0, 2024-09-11T00:32:36.254 INFO:teuthology.orchestra.run.smithi165.stdout: "omap_wr": 1220, 2024-09-11T00:32:36.254 INFO:teuthology.orchestra.run.smithi165.stdout: "omap_rd": 4, 2024-09-11T00:32:36.254 INFO:teuthology.orchestra.run.smithi165.stdout: "omap_del": 392 2024-09-11T00:32:36.254 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-11T00:32:36.254 INFO:teuthology.orchestra.run.smithi165.stdout: "oft": { 2024-09-11T00:32:36.254 INFO:teuthology.orchestra.run.smithi165.stdout: "omap_total_objs": 1, 2024-09-11T00:32:36.254 INFO:teuthology.orchestra.run.smithi165.stdout: "omap_total_kv_pairs": 0, 2024-09-11T00:32:36.254 INFO:teuthology.orchestra.run.smithi165.stdout: "omap_total_updates": 0, 2024-09-11T00:32:36.254 INFO:teuthology.orchestra.run.smithi165.stdout: "omap_total_removes": 0 2024-09-11T00:32:36.254 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-11T00:32:36.254 INFO:teuthology.orchestra.run.smithi165.stdout: "purge_queue": { 2024-09-11T00:32:36.255 INFO:teuthology.orchestra.run.smithi165.stdout: "pq_executing_ops": 0, 2024-09-11T00:32:36.255 INFO:teuthology.orchestra.run.smithi165.stdout: "pq_executing_ops_high_water": 10, 2024-09-11T00:32:36.255 INFO:teuthology.orchestra.run.smithi165.stdout: "pq_executing": 0, 2024-09-11T00:32:36.255 INFO:teuthology.orchestra.run.smithi165.stdout: "pq_executing_high_water": 9, 2024-09-11T00:32:36.255 INFO:teuthology.orchestra.run.smithi165.stdout: "pq_executed_ops": 31038, 2024-09-11T00:32:36.255 INFO:teuthology.orchestra.run.smithi165.stdout: "pq_executed": 15528, 2024-09-11T00:32:36.255 INFO:teuthology.orchestra.run.smithi165.stdout: "pq_item_in_journal": 0 2024-09-11T00:32:36.255 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-11T00:32:36.255 INFO:teuthology.orchestra.run.smithi165.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-11T00:32:36.255 INFO:teuthology.orchestra.run.smithi165.stdout: "val": 0, 2024-09-11T00:32:36.255 INFO:teuthology.orchestra.run.smithi165.stdout: "max": 104857600, 2024-09-11T00:32:36.255 INFO:teuthology.orchestra.run.smithi165.stdout: "get_started": 0, 2024-09-11T00:32:36.255 INFO:teuthology.orchestra.run.smithi165.stdout: "get": 315005, 2024-09-11T00:32:36.255 INFO:teuthology.orchestra.run.smithi165.stdout: "get_sum": 67104095, 2024-09-11T00:32:36.256 INFO:teuthology.orchestra.run.smithi165.stdout: "get_or_fail_fail": 0, 2024-09-11T00:32:36.256 INFO:teuthology.orchestra.run.smithi165.stdout: "get_or_fail_success": 315005, 2024-09-11T00:32:36.256 INFO:teuthology.orchestra.run.smithi165.stdout: "take": 0, 2024-09-11T00:32:36.256 INFO:teuthology.orchestra.run.smithi165.stdout: "take_sum": 0, 2024-09-11T00:32:36.256 INFO:teuthology.orchestra.run.smithi165.stdout: "put": 315005, 2024-09-11T00:32:36.256 INFO:teuthology.orchestra.run.smithi165.stdout: "put_sum": 67104095, 2024-09-11T00:32:36.256 INFO:teuthology.orchestra.run.smithi165.stdout: "wait": { 2024-09-11T00:32:36.256 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-09-11T00:32:36.256 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-09-11T00:32:36.256 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-09-11T00:32:36.256 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-11T00:32:36.256 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-11T00:32:36.256 INFO:teuthology.orchestra.run.smithi165.stdout: "throttle-objecter_bytes": { 2024-09-11T00:32:36.257 INFO:teuthology.orchestra.run.smithi165.stdout: "val": 0, 2024-09-11T00:32:36.257 INFO:teuthology.orchestra.run.smithi165.stdout: "max": 104857600, 2024-09-11T00:32:36.257 INFO:teuthology.orchestra.run.smithi165.stdout: "get_started": 0, 2024-09-11T00:32:36.257 INFO:teuthology.orchestra.run.smithi165.stdout: "get": 0, 2024-09-11T00:32:36.257 INFO:teuthology.orchestra.run.smithi165.stdout: "get_sum": 0, 2024-09-11T00:32:36.257 INFO:teuthology.orchestra.run.smithi165.stdout: "get_or_fail_fail": 0, 2024-09-11T00:32:36.257 INFO:teuthology.orchestra.run.smithi165.stdout: "get_or_fail_success": 0, 2024-09-11T00:32:36.257 INFO:teuthology.orchestra.run.smithi165.stdout: "take": 58179, 2024-09-11T00:32:36.257 INFO:teuthology.orchestra.run.smithi165.stdout: "take_sum": 160222217, 2024-09-11T00:32:36.257 INFO:teuthology.orchestra.run.smithi165.stdout: "put": 58179, 2024-09-11T00:32:36.257 INFO:teuthology.orchestra.run.smithi165.stdout: "put_sum": 160222217, 2024-09-11T00:32:36.257 INFO:teuthology.orchestra.run.smithi165.stdout: "wait": { 2024-09-11T00:32:36.257 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-09-11T00:32:36.257 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-09-11T00:32:36.258 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-09-11T00:32:36.258 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-11T00:32:36.258 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-11T00:32:36.258 INFO:teuthology.orchestra.run.smithi165.stdout: "throttle-objecter_ops": { 2024-09-11T00:32:36.258 INFO:teuthology.orchestra.run.smithi165.stdout: "val": 0, 2024-09-11T00:32:36.258 INFO:teuthology.orchestra.run.smithi165.stdout: "max": 1024, 2024-09-11T00:32:36.258 INFO:teuthology.orchestra.run.smithi165.stdout: "get_started": 0, 2024-09-11T00:32:36.258 INFO:teuthology.orchestra.run.smithi165.stdout: "get": 0, 2024-09-11T00:32:36.258 INFO:teuthology.orchestra.run.smithi165.stdout: "get_sum": 0, 2024-09-11T00:32:36.258 INFO:teuthology.orchestra.run.smithi165.stdout: "get_or_fail_fail": 0, 2024-09-11T00:32:36.258 INFO:teuthology.orchestra.run.smithi165.stdout: "get_or_fail_success": 0, 2024-09-11T00:32:36.258 INFO:teuthology.orchestra.run.smithi165.stdout: "take": 58179, 2024-09-11T00:32:36.258 INFO:teuthology.orchestra.run.smithi165.stdout: "take_sum": 58179, 2024-09-11T00:32:36.258 INFO:teuthology.orchestra.run.smithi165.stdout: "put": 58179, 2024-09-11T00:32:36.259 INFO:teuthology.orchestra.run.smithi165.stdout: "put_sum": 58179, 2024-09-11T00:32:36.259 INFO:teuthology.orchestra.run.smithi165.stdout: "wait": { 2024-09-11T00:32:36.259 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-09-11T00:32:36.259 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-09-11T00:32:36.259 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-09-11T00:32:36.259 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-11T00:32:36.259 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-11T00:32:36.259 INFO:teuthology.orchestra.run.smithi165.stdout: "throttle-write_buf_throttle": { 2024-09-11T00:32:36.259 INFO:teuthology.orchestra.run.smithi165.stdout: "val": 0, 2024-09-11T00:32:36.259 INFO:teuthology.orchestra.run.smithi165.stdout: "max": 3758096384, 2024-09-11T00:32:36.259 INFO:teuthology.orchestra.run.smithi165.stdout: "get_started": 0, 2024-09-11T00:32:36.259 INFO:teuthology.orchestra.run.smithi165.stdout: "get": 15528, 2024-09-11T00:32:36.259 INFO:teuthology.orchestra.run.smithi165.stdout: "get_sum": 1692408, 2024-09-11T00:32:36.259 INFO:teuthology.orchestra.run.smithi165.stdout: "get_or_fail_fail": 0, 2024-09-11T00:32:36.259 INFO:teuthology.orchestra.run.smithi165.stdout: "get_or_fail_success": 15528, 2024-09-11T00:32:36.260 INFO:teuthology.orchestra.run.smithi165.stdout: "take": 0, 2024-09-11T00:32:36.260 INFO:teuthology.orchestra.run.smithi165.stdout: "take_sum": 0, 2024-09-11T00:32:36.260 INFO:teuthology.orchestra.run.smithi165.stdout: "put": 669, 2024-09-11T00:32:36.260 INFO:teuthology.orchestra.run.smithi165.stdout: "put_sum": 1692408, 2024-09-11T00:32:36.260 INFO:teuthology.orchestra.run.smithi165.stdout: "wait": { 2024-09-11T00:32:36.260 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-09-11T00:32:36.260 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-09-11T00:32:36.260 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-09-11T00:32:36.260 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-11T00:32:36.260 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-11T00:32:36.260 INFO:teuthology.orchestra.run.smithi165.stdout: "throttle-write_buf_throttle-0x55da895e2dc0": { 2024-09-11T00:32:36.260 INFO:teuthology.orchestra.run.smithi165.stdout: "val": 0, 2024-09-11T00:32:36.260 INFO:teuthology.orchestra.run.smithi165.stdout: "max": 3758096384, 2024-09-11T00:32:36.260 INFO:teuthology.orchestra.run.smithi165.stdout: "get_started": 0, 2024-09-11T00:32:36.261 INFO:teuthology.orchestra.run.smithi165.stdout: "get": 78981, 2024-09-11T00:32:36.261 INFO:teuthology.orchestra.run.smithi165.stdout: "get_sum": 155112076, 2024-09-11T00:32:36.261 INFO:teuthology.orchestra.run.smithi165.stdout: "get_or_fail_fail": 0, 2024-09-11T00:32:36.261 INFO:teuthology.orchestra.run.smithi165.stdout: "get_or_fail_success": 78981, 2024-09-11T00:32:36.261 INFO:teuthology.orchestra.run.smithi165.stdout: "take": 0, 2024-09-11T00:32:36.261 INFO:teuthology.orchestra.run.smithi165.stdout: "take_sum": 0, 2024-09-11T00:32:36.261 INFO:teuthology.orchestra.run.smithi165.stdout: "put": 24672, 2024-09-11T00:32:36.261 INFO:teuthology.orchestra.run.smithi165.stdout: "put_sum": 155112076, 2024-09-11T00:32:36.261 INFO:teuthology.orchestra.run.smithi165.stdout: "wait": { 2024-09-11T00:32:36.261 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-09-11T00:32:36.261 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-09-11T00:32:36.261 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-09-11T00:32:36.261 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-11T00:32:36.261 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-11T00:32:36.262 INFO:teuthology.orchestra.run.smithi165.stdout: "trackedop": { 2024-09-11T00:32:36.262 INFO:teuthology.orchestra.run.smithi165.stdout: "slow_ops_count": 1852 2024-09-11T00:32:36.262 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-11T00:32:36.262 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-09-11T00:32:36.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:32:36 smithi012 ceph-mon[30333]: pgmap v1165: 129 pgs: 129 active+clean; 671 MiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 2.7 KiB/s wr, 12 op/s 2024-09-11T00:32:36.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:32:36 smithi178 ceph-mon[35919]: pgmap v1165: 129 pgs: 129 active+clean; 671 MiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 2.7 KiB/s wr, 12 op/s 2024-09-11T00:32:37.381 INFO:tasks.check_counter:Daemon mds.e mds.exported=12 2024-09-11T00:32:37.381 INFO:tasks.check_counter:Daemon mds.e mds.imported=12 2024-09-11T00:32:37.381 DEBUG:tasks.check_counter:Getting stats from h 2024-09-11T00:32:37.381 DEBUG:teuthology.orchestra.run.smithi165:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid ff3ad74a-6fcf-11ef-bcea-c7b262605968 -- ceph daemon mds.h perf dump 2024-09-11T00:32:37.642 INFO:teuthology.orchestra.run.smithi165.stderr:Inferring config /var/lib/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T00:32:38.192 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-09-11T00:32:38.192 INFO:teuthology.orchestra.run.smithi165.stdout: "AsyncMessenger::Worker-0": { 2024-09-11T00:32:38.192 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_recv_messages": 208198, 2024-09-11T00:32:38.193 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_messages": 302542, 2024-09-11T00:32:38.193 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_recv_bytes": 64439197, 2024-09-11T00:32:38.193 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_bytes": 262005017, 2024-09-11T00:32:38.193 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_created_connections": 16, 2024-09-11T00:32:38.193 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_active_connections": 5, 2024-09-11T00:32:38.193 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_total_time": 17.823147041, 2024-09-11T00:32:38.193 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_send_time": 9.909792408, 2024-09-11T00:32:38.193 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_recv_time": 7.612505525, 2024-09-11T00:32:38.193 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_fast_dispatch_time": 1.263981906, 2024-09-11T00:32:38.193 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_messages_queue_lat": { 2024-09-11T00:32:38.193 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 302541, 2024-09-11T00:32:38.193 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 20.410581298, 2024-09-11T00:32:38.193 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000067463 2024-09-11T00:32:38.194 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-11T00:32:38.194 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_handle_ack_lat": { 2024-09-11T00:32:38.194 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 238362, 2024-09-11T00:32:38.194 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.082148299, 2024-09-11T00:32:38.194 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000344 2024-09-11T00:32:38.194 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-11T00:32:38.194 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_recv_encrypted_bytes": 2944, 2024-09-11T00:32:38.194 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_encrypted_bytes": 448 2024-09-11T00:32:38.194 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-11T00:32:38.194 INFO:teuthology.orchestra.run.smithi165.stdout: "AsyncMessenger::Worker-1": { 2024-09-11T00:32:38.194 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_recv_messages": 47981, 2024-09-11T00:32:38.194 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_messages": 48733, 2024-09-11T00:32:38.194 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_recv_bytes": 12468020, 2024-09-11T00:32:38.194 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_bytes": 109122294, 2024-09-11T00:32:38.195 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_created_connections": 12, 2024-09-11T00:32:38.195 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_active_connections": 9, 2024-09-11T00:32:38.195 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_total_time": 5.030355038, 2024-09-11T00:32:38.195 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_send_time": 2.280967555, 2024-09-11T00:32:38.195 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_recv_time": 1.395735432, 2024-09-11T00:32:38.195 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_fast_dispatch_time": 1.271223631, 2024-09-11T00:32:38.195 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_messages_queue_lat": { 2024-09-11T00:32:38.195 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 48731, 2024-09-11T00:32:38.195 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 3.468571529, 2024-09-11T00:32:38.195 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000071177 2024-09-11T00:32:38.195 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-11T00:32:38.195 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_handle_ack_lat": { 2024-09-11T00:32:38.195 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 33034, 2024-09-11T00:32:38.196 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.006359450, 2024-09-11T00:32:38.196 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000192 2024-09-11T00:32:38.196 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-11T00:32:38.196 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_recv_encrypted_bytes": 160, 2024-09-11T00:32:38.196 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_encrypted_bytes": 702048 2024-09-11T00:32:38.196 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-11T00:32:38.196 INFO:teuthology.orchestra.run.smithi165.stdout: "AsyncMessenger::Worker-2": { 2024-09-11T00:32:38.196 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_recv_messages": 129566, 2024-09-11T00:32:38.196 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_messages": 106076, 2024-09-11T00:32:38.196 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_recv_bytes": 37420090, 2024-09-11T00:32:38.196 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_bytes": 62656747, 2024-09-11T00:32:38.196 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_created_connections": 8, 2024-09-11T00:32:38.196 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_active_connections": 6, 2024-09-11T00:32:38.197 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_total_time": 9.445774077, 2024-09-11T00:32:38.197 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_send_time": 4.834376667, 2024-09-11T00:32:38.197 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_recv_time": 5.040439102, 2024-09-11T00:32:38.197 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_fast_dispatch_time": 0.535174032, 2024-09-11T00:32:38.197 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_messages_queue_lat": { 2024-09-11T00:32:38.197 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 106075, 2024-09-11T00:32:38.197 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 5.259401330, 2024-09-11T00:32:38.197 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000049581 2024-09-11T00:32:38.197 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-11T00:32:38.197 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_handle_ack_lat": { 2024-09-11T00:32:38.197 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 200085, 2024-09-11T00:32:38.197 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.033973075, 2024-09-11T00:32:38.197 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000169 2024-09-11T00:32:38.198 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-11T00:32:38.198 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_recv_encrypted_bytes": 1204368, 2024-09-11T00:32:38.198 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_encrypted_bytes": 293360 2024-09-11T00:32:38.198 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-11T00:32:38.198 INFO:teuthology.orchestra.run.smithi165.stdout: "cct": { 2024-09-11T00:32:38.198 INFO:teuthology.orchestra.run.smithi165.stdout: "total_workers": 1, 2024-09-11T00:32:38.198 INFO:teuthology.orchestra.run.smithi165.stdout: "unhealthy_workers": 0 2024-09-11T00:32:38.198 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-11T00:32:38.198 INFO:teuthology.orchestra.run.smithi165.stdout: "finisher-MDSRank": { 2024-09-11T00:32:38.198 INFO:teuthology.orchestra.run.smithi165.stdout: "queue_len": 0, 2024-09-11T00:32:38.198 INFO:teuthology.orchestra.run.smithi165.stdout: "complete_latency": { 2024-09-11T00:32:38.199 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 58018, 2024-09-11T00:32:38.199 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 63.295879990, 2024-09-11T00:32:38.199 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.001090969 2024-09-11T00:32:38.199 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-11T00:32:38.199 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-11T00:32:38.199 INFO:teuthology.orchestra.run.smithi165.stdout: "finisher-PurgeQueue": { 2024-09-11T00:32:38.199 INFO:teuthology.orchestra.run.smithi165.stdout: "queue_len": 0, 2024-09-11T00:32:38.199 INFO:teuthology.orchestra.run.smithi165.stdout: "complete_latency": { 2024-09-11T00:32:38.199 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 36221, 2024-09-11T00:32:38.199 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 2.998581334, 2024-09-11T00:32:38.199 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000082785 2024-09-11T00:32:38.199 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-11T00:32:38.199 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-11T00:32:38.199 INFO:teuthology.orchestra.run.smithi165.stdout: "mds": { 2024-09-11T00:32:38.200 INFO:teuthology.orchestra.run.smithi165.stdout: "request": 108063, 2024-09-11T00:32:38.200 INFO:teuthology.orchestra.run.smithi165.stdout: "reply": 101126, 2024-09-11T00:32:38.200 INFO:teuthology.orchestra.run.smithi165.stdout: "reply_latency": { 2024-09-11T00:32:38.200 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 101126, 2024-09-11T00:32:38.200 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 870.323878120, 2024-09-11T00:32:38.200 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.008606331 2024-09-11T00:32:38.200 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-11T00:32:38.200 INFO:teuthology.orchestra.run.smithi165.stdout: "slow_reply": 0, 2024-09-11T00:32:38.200 INFO:teuthology.orchestra.run.smithi165.stdout: "forward": 6937, 2024-09-11T00:32:38.200 INFO:teuthology.orchestra.run.smithi165.stdout: "dir_fetch_complete": 0, 2024-09-11T00:32:38.200 INFO:teuthology.orchestra.run.smithi165.stdout: "dir_fetch_keys": 0, 2024-09-11T00:32:38.201 INFO:teuthology.orchestra.run.smithi165.stdout: "dir_commit": 11, 2024-09-11T00:32:38.201 INFO:teuthology.orchestra.run.smithi165.stdout: "dir_split": 0, 2024-09-11T00:32:38.201 INFO:teuthology.orchestra.run.smithi165.stdout: "dir_merge": 0, 2024-09-11T00:32:38.201 INFO:teuthology.orchestra.run.smithi165.stdout: "inodes": 22, 2024-09-11T00:32:38.201 INFO:teuthology.orchestra.run.smithi165.stdout: "inodes_top": 3, 2024-09-11T00:32:38.201 INFO:teuthology.orchestra.run.smithi165.stdout: "inodes_bottom": 3, 2024-09-11T00:32:38.201 INFO:teuthology.orchestra.run.smithi165.stdout: "inodes_pin_tail": 16, 2024-09-11T00:32:38.201 INFO:teuthology.orchestra.run.smithi165.stdout: "inodes_pinned": 17, 2024-09-11T00:32:38.201 INFO:teuthology.orchestra.run.smithi165.stdout: "inodes_expired": 650, 2024-09-11T00:32:38.201 INFO:teuthology.orchestra.run.smithi165.stdout: "inodes_with_caps": 1, 2024-09-11T00:32:38.201 INFO:teuthology.orchestra.run.smithi165.stdout: "caps": 1, 2024-09-11T00:32:38.201 INFO:teuthology.orchestra.run.smithi165.stdout: "subtrees": 6, 2024-09-11T00:32:38.201 INFO:teuthology.orchestra.run.smithi165.stdout: "traverse": 176446, 2024-09-11T00:32:38.202 INFO:teuthology.orchestra.run.smithi165.stdout: "traverse_hit": 101521, 2024-09-11T00:32:38.202 INFO:teuthology.orchestra.run.smithi165.stdout: "traverse_forward": 6927, 2024-09-11T00:32:38.202 INFO:teuthology.orchestra.run.smithi165.stdout: "traverse_discover": 115, 2024-09-11T00:32:38.202 INFO:teuthology.orchestra.run.smithi165.stdout: "traverse_dir_fetch": 0, 2024-09-11T00:32:38.202 INFO:teuthology.orchestra.run.smithi165.stdout: "traverse_remote_ino": 0, 2024-09-11T00:32:38.202 INFO:teuthology.orchestra.run.smithi165.stdout: "traverse_lock": 2480, 2024-09-11T00:32:38.202 INFO:teuthology.orchestra.run.smithi165.stdout: "load_cent": 0, 2024-09-11T00:32:38.202 INFO:teuthology.orchestra.run.smithi165.stdout: "q": 0, 2024-09-11T00:32:38.202 INFO:teuthology.orchestra.run.smithi165.stdout: "exported": 18, 2024-09-11T00:32:38.202 INFO:teuthology.orchestra.run.smithi165.stdout: "exported_inodes": 263, 2024-09-11T00:32:38.202 INFO:teuthology.orchestra.run.smithi165.stdout: "imported": 18, 2024-09-11T00:32:38.202 INFO:teuthology.orchestra.run.smithi165.stdout: "imported_inodes": 508, 2024-09-11T00:32:38.202 INFO:teuthology.orchestra.run.smithi165.stdout: "openino_dir_fetch": 0, 2024-09-11T00:32:38.202 INFO:teuthology.orchestra.run.smithi165.stdout: "openino_backtrace_fetch": 0, 2024-09-11T00:32:38.203 INFO:teuthology.orchestra.run.smithi165.stdout: "openino_peer_discover": 0, 2024-09-11T00:32:38.203 INFO:teuthology.orchestra.run.smithi165.stdout: "root_rfiles": 4, 2024-09-11T00:32:38.203 INFO:teuthology.orchestra.run.smithi165.stdout: "root_rbytes": 234, 2024-09-11T00:32:38.203 INFO:teuthology.orchestra.run.smithi165.stdout: "root_rsnaps": 0, 2024-09-11T00:32:38.203 INFO:teuthology.orchestra.run.smithi165.stdout: "scrub_backtrace_fetch": 0, 2024-09-11T00:32:38.203 INFO:teuthology.orchestra.run.smithi165.stdout: "scrub_set_tag": 0, 2024-09-11T00:32:38.203 INFO:teuthology.orchestra.run.smithi165.stdout: "scrub_backtrace_repaired": 0, 2024-09-11T00:32:38.203 INFO:teuthology.orchestra.run.smithi165.stdout: "scrub_inotable_repaired": 0, 2024-09-11T00:32:38.203 INFO:teuthology.orchestra.run.smithi165.stdout: "scrub_dir_inodes": 0, 2024-09-11T00:32:38.203 INFO:teuthology.orchestra.run.smithi165.stdout: "scrub_dir_base_inodes": 0, 2024-09-11T00:32:38.203 INFO:teuthology.orchestra.run.smithi165.stdout: "scrub_dirfrag_rstats": 0, 2024-09-11T00:32:38.203 INFO:teuthology.orchestra.run.smithi165.stdout: "scrub_file_inodes": 0, 2024-09-11T00:32:38.203 INFO:teuthology.orchestra.run.smithi165.stdout: "handle_inode_file_caps": 31, 2024-09-11T00:32:38.204 INFO:teuthology.orchestra.run.smithi165.stdout: "ceph_cap_op_revoke": 10671, 2024-09-11T00:32:38.204 INFO:teuthology.orchestra.run.smithi165.stdout: "ceph_cap_op_grant": 78603, 2024-09-11T00:32:38.204 INFO:teuthology.orchestra.run.smithi165.stdout: "ceph_cap_op_trunc": 0, 2024-09-11T00:32:38.204 INFO:teuthology.orchestra.run.smithi165.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-11T00:32:38.204 INFO:teuthology.orchestra.run.smithi165.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-11T00:32:38.204 INFO:teuthology.orchestra.run.smithi165.stdout: "handle_client_caps": 48726, 2024-09-11T00:32:38.204 INFO:teuthology.orchestra.run.smithi165.stdout: "handle_client_caps_dirty": 18183, 2024-09-11T00:32:38.204 INFO:teuthology.orchestra.run.smithi165.stdout: "handle_client_cap_release": 2473, 2024-09-11T00:32:38.204 INFO:teuthology.orchestra.run.smithi165.stdout: "process_request_cap_release": 71287 2024-09-11T00:32:38.204 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-11T00:32:38.204 INFO:teuthology.orchestra.run.smithi165.stdout: "mds_cache": { 2024-09-11T00:32:38.204 INFO:teuthology.orchestra.run.smithi165.stdout: "dir_update": 32, 2024-09-11T00:32:38.204 INFO:teuthology.orchestra.run.smithi165.stdout: "dir_update_receipt": 201, 2024-09-11T00:32:38.204 INFO:teuthology.orchestra.run.smithi165.stdout: "dir_try_discover": 126, 2024-09-11T00:32:38.205 INFO:teuthology.orchestra.run.smithi165.stdout: "dir_send_discover": 259, 2024-09-11T00:32:38.205 INFO:teuthology.orchestra.run.smithi165.stdout: "dir_handle_discover": 136, 2024-09-11T00:32:38.205 INFO:teuthology.orchestra.run.smithi165.stdout: "num_strays": 1, 2024-09-11T00:32:38.205 INFO:teuthology.orchestra.run.smithi165.stdout: "num_strays_delayed": 0, 2024-09-11T00:32:38.205 INFO:teuthology.orchestra.run.smithi165.stdout: "num_strays_enqueuing": 0, 2024-09-11T00:32:38.205 INFO:teuthology.orchestra.run.smithi165.stdout: "strays_created": 18239, 2024-09-11T00:32:38.205 INFO:teuthology.orchestra.run.smithi165.stdout: "strays_enqueued": 18238, 2024-09-11T00:32:38.205 INFO:teuthology.orchestra.run.smithi165.stdout: "strays_reintegrated": 0, 2024-09-11T00:32:38.206 INFO:teuthology.orchestra.run.smithi165.stdout: "strays_migrated": 0, 2024-09-11T00:32:38.206 INFO:teuthology.orchestra.run.smithi165.stdout: "num_recovering_processing": 0, 2024-09-11T00:32:38.206 INFO:teuthology.orchestra.run.smithi165.stdout: "num_recovering_enqueued": 0, 2024-09-11T00:32:38.206 INFO:teuthology.orchestra.run.smithi165.stdout: "num_recovering_prioritized": 0, 2024-09-11T00:32:38.206 INFO:teuthology.orchestra.run.smithi165.stdout: "recovery_started": 0, 2024-09-11T00:32:38.206 INFO:teuthology.orchestra.run.smithi165.stdout: "recovery_completed": 0, 2024-09-11T00:32:38.206 INFO:teuthology.orchestra.run.smithi165.stdout: "ireq_quiesce_path": 19, 2024-09-11T00:32:38.206 INFO:teuthology.orchestra.run.smithi165.stdout: "ireq_quiesce_inode": 1984, 2024-09-11T00:32:38.206 INFO:teuthology.orchestra.run.smithi165.stdout: "ireq_enqueue_scrub": 0, 2024-09-11T00:32:38.207 INFO:teuthology.orchestra.run.smithi165.stdout: "ireq_exportdir": 20, 2024-09-11T00:32:38.207 INFO:teuthology.orchestra.run.smithi165.stdout: "ireq_flush": 0, 2024-09-11T00:32:38.207 INFO:teuthology.orchestra.run.smithi165.stdout: "ireq_fragmentdir": 0, 2024-09-11T00:32:38.207 INFO:teuthology.orchestra.run.smithi165.stdout: "ireq_fragstats": 0, 2024-09-11T00:32:38.207 INFO:teuthology.orchestra.run.smithi165.stdout: "ireq_inodestats": 0, 2024-09-11T00:32:38.207 INFO:teuthology.orchestra.run.smithi165.stdout: "uninline_started": 0, 2024-09-11T00:32:38.207 INFO:teuthology.orchestra.run.smithi165.stdout: "uninline_succeeded": 0, 2024-09-11T00:32:38.207 INFO:teuthology.orchestra.run.smithi165.stdout: "uninline_write_failed": 0 2024-09-11T00:32:38.207 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-11T00:32:38.207 INFO:teuthology.orchestra.run.smithi165.stdout: "mds_log": { 2024-09-11T00:32:38.207 INFO:teuthology.orchestra.run.smithi165.stdout: "evlrg": 0, 2024-09-11T00:32:38.207 INFO:teuthology.orchestra.run.smithi165.stdout: "evadd": 97378, 2024-09-11T00:32:38.207 INFO:teuthology.orchestra.run.smithi165.stdout: "evex": 0, 2024-09-11T00:32:38.207 INFO:teuthology.orchestra.run.smithi165.stdout: "evtrm": 0, 2024-09-11T00:32:38.208 INFO:teuthology.orchestra.run.smithi165.stdout: "ev": 97378, 2024-09-11T00:32:38.208 INFO:teuthology.orchestra.run.smithi165.stdout: "evexg": 0, 2024-09-11T00:32:38.208 INFO:teuthology.orchestra.run.smithi165.stdout: "evexd": 0, 2024-09-11T00:32:38.208 INFO:teuthology.orchestra.run.smithi165.stdout: "segadd": 124, 2024-09-11T00:32:38.208 INFO:teuthology.orchestra.run.smithi165.stdout: "segex": 0, 2024-09-11T00:32:38.208 INFO:teuthology.orchestra.run.smithi165.stdout: "segtrm": 0, 2024-09-11T00:32:38.208 INFO:teuthology.orchestra.run.smithi165.stdout: "seg": 124, 2024-09-11T00:32:38.208 INFO:teuthology.orchestra.run.smithi165.stdout: "segmjr": 9, 2024-09-11T00:32:38.208 INFO:teuthology.orchestra.run.smithi165.stdout: "segexg": 0, 2024-09-11T00:32:38.208 INFO:teuthology.orchestra.run.smithi165.stdout: "segexd": 0, 2024-09-11T00:32:38.208 INFO:teuthology.orchestra.run.smithi165.stdout: "expos": 4194304, 2024-09-11T00:32:38.208 INFO:teuthology.orchestra.run.smithi165.stdout: "wrpos": 184170688, 2024-09-11T00:32:38.209 INFO:teuthology.orchestra.run.smithi165.stdout: "rdpos": 0, 2024-09-11T00:32:38.209 INFO:teuthology.orchestra.run.smithi165.stdout: "jlat": { 2024-09-11T00:32:38.209 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 32036, 2024-09-11T00:32:38.209 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 271.846835623, 2024-09-11T00:32:38.209 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.008485667 2024-09-11T00:32:38.209 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-11T00:32:38.209 INFO:teuthology.orchestra.run.smithi165.stdout: "replayed": 0 2024-09-11T00:32:38.209 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-11T00:32:38.209 INFO:teuthology.orchestra.run.smithi165.stdout: "mds_mem": { 2024-09-11T00:32:38.209 INFO:teuthology.orchestra.run.smithi165.stdout: "ino": 29, 2024-09-11T00:32:38.209 INFO:teuthology.orchestra.run.smithi165.stdout: "ino+": 18580, 2024-09-11T00:32:38.209 INFO:teuthology.orchestra.run.smithi165.stdout: "ino-": 18551, 2024-09-11T00:32:38.209 INFO:teuthology.orchestra.run.smithi165.stdout: "dir": 28, 2024-09-11T00:32:38.209 INFO:teuthology.orchestra.run.smithi165.stdout: "dir+": 204, 2024-09-11T00:32:38.210 INFO:teuthology.orchestra.run.smithi165.stdout: "dir-": 176, 2024-09-11T00:32:38.210 INFO:teuthology.orchestra.run.smithi165.stdout: "dn": 40, 2024-09-11T00:32:38.210 INFO:teuthology.orchestra.run.smithi165.stdout: "dn+": 33309, 2024-09-11T00:32:38.210 INFO:teuthology.orchestra.run.smithi165.stdout: "dn-": 33269, 2024-09-11T00:32:38.210 INFO:teuthology.orchestra.run.smithi165.stdout: "cap": 1, 2024-09-11T00:32:38.210 INFO:teuthology.orchestra.run.smithi165.stdout: "cap+": 20335, 2024-09-11T00:32:38.210 INFO:teuthology.orchestra.run.smithi165.stdout: "cap-": 20334, 2024-09-11T00:32:38.210 INFO:teuthology.orchestra.run.smithi165.stdout: "rss": 72888, 2024-09-11T00:32:38.210 INFO:teuthology.orchestra.run.smithi165.stdout: "heap": 281927680 2024-09-11T00:32:38.210 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-11T00:32:38.210 INFO:teuthology.orchestra.run.smithi165.stdout: "mds_server": { 2024-09-11T00:32:38.210 INFO:teuthology.orchestra.run.smithi165.stdout: "dispatch_client_request": 155437, 2024-09-11T00:32:38.210 INFO:teuthology.orchestra.run.smithi165.stdout: "dispatch_server_request": 1586, 2024-09-11T00:32:38.211 INFO:teuthology.orchestra.run.smithi165.stdout: "handle_client_request": 108063, 2024-09-11T00:32:38.211 INFO:teuthology.orchestra.run.smithi165.stdout: "handle_client_session": 3319, 2024-09-11T00:32:38.211 INFO:teuthology.orchestra.run.smithi165.stdout: "handle_peer_request": 9387, 2024-09-11T00:32:38.211 INFO:teuthology.orchestra.run.smithi165.stdout: "req_create_latency": { 2024-09-11T00:32:38.211 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 18098, 2024-09-11T00:32:38.211 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 243.431158482, 2024-09-11T00:32:38.211 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.013450721 2024-09-11T00:32:38.211 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-11T00:32:38.211 INFO:teuthology.orchestra.run.smithi165.stdout: "req_getattr_latency": { 2024-09-11T00:32:38.211 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 3701, 2024-09-11T00:32:38.211 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 63.306271943, 2024-09-11T00:32:38.211 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.017105180 2024-09-11T00:32:38.211 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-11T00:32:38.212 INFO:teuthology.orchestra.run.smithi165.stdout: "req_getfilelock_latency": { 2024-09-11T00:32:38.212 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-09-11T00:32:38.212 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-09-11T00:32:38.212 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-09-11T00:32:38.212 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-11T00:32:38.212 INFO:teuthology.orchestra.run.smithi165.stdout: "req_link_latency": { 2024-09-11T00:32:38.212 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-09-11T00:32:38.212 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-09-11T00:32:38.212 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-09-11T00:32:38.212 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-11T00:32:38.212 INFO:teuthology.orchestra.run.smithi165.stdout: "req_lookup_latency": { 2024-09-11T00:32:38.212 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 27903, 2024-09-11T00:32:38.213 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 39.717878386, 2024-09-11T00:32:38.213 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.001423426 2024-09-11T00:32:38.213 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-11T00:32:38.213 INFO:teuthology.orchestra.run.smithi165.stdout: "req_lookuphash_latency": { 2024-09-11T00:32:38.213 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-09-11T00:32:38.213 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-09-11T00:32:38.213 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-09-11T00:32:38.213 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-11T00:32:38.213 INFO:teuthology.orchestra.run.smithi165.stdout: "req_lookupino_latency": { 2024-09-11T00:32:38.213 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-09-11T00:32:38.214 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-09-11T00:32:38.214 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-09-11T00:32:38.214 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-11T00:32:38.214 INFO:teuthology.orchestra.run.smithi165.stdout: "req_lookupname_latency": { 2024-09-11T00:32:38.214 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-09-11T00:32:38.214 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-09-11T00:32:38.214 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-09-11T00:32:38.214 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-11T00:32:38.214 INFO:teuthology.orchestra.run.smithi165.stdout: "req_lookupparent_latency": { 2024-09-11T00:32:38.214 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-09-11T00:32:38.214 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-09-11T00:32:38.214 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-09-11T00:32:38.214 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-11T00:32:38.215 INFO:teuthology.orchestra.run.smithi165.stdout: "req_lookupsnap_latency": { 2024-09-11T00:32:38.215 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-09-11T00:32:38.215 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-09-11T00:32:38.215 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-09-11T00:32:38.215 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-11T00:32:38.215 INFO:teuthology.orchestra.run.smithi165.stdout: "req_lssnap_latency": { 2024-09-11T00:32:38.215 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-09-11T00:32:38.215 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-09-11T00:32:38.216 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-09-11T00:32:38.216 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-11T00:32:38.216 INFO:teuthology.orchestra.run.smithi165.stdout: "req_mkdir_latency": { 2024-09-11T00:32:38.216 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-09-11T00:32:38.216 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-09-11T00:32:38.216 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-09-11T00:32:38.216 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-11T00:32:38.216 INFO:teuthology.orchestra.run.smithi165.stdout: "req_mknod_latency": { 2024-09-11T00:32:38.216 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-09-11T00:32:38.216 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-09-11T00:32:38.216 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-09-11T00:32:38.216 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-11T00:32:38.217 INFO:teuthology.orchestra.run.smithi165.stdout: "req_mksnap_latency": { 2024-09-11T00:32:38.217 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-09-11T00:32:38.217 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-09-11T00:32:38.217 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-09-11T00:32:38.217 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-11T00:32:38.217 INFO:teuthology.orchestra.run.smithi165.stdout: "req_open_latency": { 2024-09-11T00:32:38.217 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 2, 2024-09-11T00:32:38.217 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.001227928, 2024-09-11T00:32:38.217 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000613964 2024-09-11T00:32:38.217 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-11T00:32:38.217 INFO:teuthology.orchestra.run.smithi165.stdout: "req_readdir_latency": { 2024-09-11T00:32:38.218 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 27911, 2024-09-11T00:32:38.218 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 164.027177340, 2024-09-11T00:32:38.218 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.005876793 2024-09-11T00:32:38.218 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-11T00:32:38.218 INFO:teuthology.orchestra.run.smithi165.stdout: "req_rename_latency": { 2024-09-11T00:32:38.218 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 4242, 2024-09-11T00:32:38.218 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 61.887817907, 2024-09-11T00:32:38.218 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.014589301 2024-09-11T00:32:38.218 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-11T00:32:38.218 INFO:teuthology.orchestra.run.smithi165.stdout: "req_renamesnap_latency": { 2024-09-11T00:32:38.218 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-09-11T00:32:38.218 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-09-11T00:32:38.218 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-09-11T00:32:38.218 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-11T00:32:38.219 INFO:teuthology.orchestra.run.smithi165.stdout: "req_snapdiff_latency": { 2024-09-11T00:32:38.219 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-09-11T00:32:38.219 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-09-11T00:32:38.219 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-09-11T00:32:38.219 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-11T00:32:38.219 INFO:teuthology.orchestra.run.smithi165.stdout: "req_rmdir_latency": { 2024-09-11T00:32:38.219 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 20, 2024-09-11T00:32:38.219 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.166961308, 2024-09-11T00:32:38.219 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.008348065 2024-09-11T00:32:38.219 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-11T00:32:38.219 INFO:teuthology.orchestra.run.smithi165.stdout: "req_rmsnap_latency": { 2024-09-11T00:32:38.219 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-09-11T00:32:38.220 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-09-11T00:32:38.220 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-09-11T00:32:38.220 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-11T00:32:38.220 INFO:teuthology.orchestra.run.smithi165.stdout: "req_rmxattr_latency": { 2024-09-11T00:32:38.220 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-09-11T00:32:38.220 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-09-11T00:32:38.220 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-09-11T00:32:38.220 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-11T00:32:38.220 INFO:teuthology.orchestra.run.smithi165.stdout: "req_setattr_latency": { 2024-09-11T00:32:38.220 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 66, 2024-09-11T00:32:38.220 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 118.209331306, 2024-09-11T00:32:38.221 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 1.791050474 2024-09-11T00:32:38.221 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-11T00:32:38.221 INFO:teuthology.orchestra.run.smithi165.stdout: "req_setdirlayout_latency": { 2024-09-11T00:32:38.221 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-09-11T00:32:38.221 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-09-11T00:32:38.221 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-09-11T00:32:38.221 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-11T00:32:38.221 INFO:teuthology.orchestra.run.smithi165.stdout: "req_setfilelock_latency": { 2024-09-11T00:32:38.221 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 964, 2024-09-11T00:32:38.221 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.584955898, 2024-09-11T00:32:38.222 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000606800 2024-09-11T00:32:38.222 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-11T00:32:38.222 INFO:teuthology.orchestra.run.smithi165.stdout: "req_setlayout_latency": { 2024-09-11T00:32:38.222 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-09-11T00:32:38.222 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-09-11T00:32:38.222 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-09-11T00:32:38.222 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-11T00:32:38.222 INFO:teuthology.orchestra.run.smithi165.stdout: "req_setxattr_latency": { 2024-09-11T00:32:38.222 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-09-11T00:32:38.222 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-09-11T00:32:38.222 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-09-11T00:32:38.222 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-11T00:32:38.222 INFO:teuthology.orchestra.run.smithi165.stdout: "req_symlink_latency": { 2024-09-11T00:32:38.223 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-09-11T00:32:38.223 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-09-11T00:32:38.223 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-09-11T00:32:38.223 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-11T00:32:38.223 INFO:teuthology.orchestra.run.smithi165.stdout: "req_unlink_latency": { 2024-09-11T00:32:38.223 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 18219, 2024-09-11T00:32:38.223 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 178.991097622, 2024-09-11T00:32:38.223 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.009824419 2024-09-11T00:32:38.223 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-11T00:32:38.223 INFO:teuthology.orchestra.run.smithi165.stdout: "cap_revoke_eviction": 0, 2024-09-11T00:32:38.223 INFO:teuthology.orchestra.run.smithi165.stdout: "cap_acquisition_throttle": 0, 2024-09-11T00:32:38.223 INFO:teuthology.orchestra.run.smithi165.stdout: "req_getvxattr_latency": { 2024-09-11T00:32:38.223 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-09-11T00:32:38.223 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-09-11T00:32:38.223 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-09-11T00:32:38.224 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-11T00:32:38.224 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-11T00:32:38.224 INFO:teuthology.orchestra.run.smithi165.stdout: "mds_sessions": { 2024-09-11T00:32:38.224 INFO:teuthology.orchestra.run.smithi165.stdout: "session_count": 1, 2024-09-11T00:32:38.224 INFO:teuthology.orchestra.run.smithi165.stdout: "session_add": 1, 2024-09-11T00:32:38.224 INFO:teuthology.orchestra.run.smithi165.stdout: "session_remove": 0, 2024-09-11T00:32:38.224 INFO:teuthology.orchestra.run.smithi165.stdout: "sessions_open": 1, 2024-09-11T00:32:38.224 INFO:teuthology.orchestra.run.smithi165.stdout: "sessions_stale": 0, 2024-09-11T00:32:38.224 INFO:teuthology.orchestra.run.smithi165.stdout: "total_load": 8775, 2024-09-11T00:32:38.224 INFO:teuthology.orchestra.run.smithi165.stdout: "average_load": 8775, 2024-09-11T00:32:38.224 INFO:teuthology.orchestra.run.smithi165.stdout: "avg_session_uptime": 1550, 2024-09-11T00:32:38.224 INFO:teuthology.orchestra.run.smithi165.stdout: "mdthresh_evicted": 0 2024-09-11T00:32:38.224 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-11T00:32:38.225 INFO:teuthology.orchestra.run.smithi165.stdout: "mempool": { 2024-09-11T00:32:38.225 INFO:teuthology.orchestra.run.smithi165.stdout: "bloom_filter_bytes": 0, 2024-09-11T00:32:38.225 INFO:teuthology.orchestra.run.smithi165.stdout: "bloom_filter_items": 0, 2024-09-11T00:32:38.225 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_alloc_bytes": 0, 2024-09-11T00:32:38.225 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_alloc_items": 0, 2024-09-11T00:32:38.225 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_cache_data_bytes": 0, 2024-09-11T00:32:38.225 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_cache_data_items": 0, 2024-09-11T00:32:38.225 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-11T00:32:38.225 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_cache_onode_items": 0, 2024-09-11T00:32:38.225 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-11T00:32:38.225 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_cache_meta_items": 0, 2024-09-11T00:32:38.225 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_cache_other_bytes": 0, 2024-09-11T00:32:38.225 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_cache_other_items": 0, 2024-09-11T00:32:38.226 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-11T00:32:38.226 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_cache_buffer_items": 0, 2024-09-11T00:32:38.226 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_extent_bytes": 0, 2024-09-11T00:32:38.226 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_extent_items": 0, 2024-09-11T00:32:38.226 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_blob_bytes": 0, 2024-09-11T00:32:38.226 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_blob_items": 0, 2024-09-11T00:32:38.226 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-11T00:32:38.226 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_shared_blob_items": 0, 2024-09-11T00:32:38.226 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-11T00:32:38.226 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_inline_bl_items": 0, 2024-09-11T00:32:38.226 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_fsck_bytes": 0, 2024-09-11T00:32:38.226 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_fsck_items": 0, 2024-09-11T00:32:38.226 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_txc_bytes": 0, 2024-09-11T00:32:38.226 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_txc_items": 0, 2024-09-11T00:32:38.227 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-11T00:32:38.227 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_writing_deferred_items": 0, 2024-09-11T00:32:38.227 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_writing_bytes": 0, 2024-09-11T00:32:38.227 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_writing_items": 0, 2024-09-11T00:32:38.227 INFO:teuthology.orchestra.run.smithi165.stdout: "bluefs_bytes": 0, 2024-09-11T00:32:38.227 INFO:teuthology.orchestra.run.smithi165.stdout: "bluefs_items": 0, 2024-09-11T00:32:38.227 INFO:teuthology.orchestra.run.smithi165.stdout: "bluefs_file_reader_bytes": 0, 2024-09-11T00:32:38.227 INFO:teuthology.orchestra.run.smithi165.stdout: "bluefs_file_reader_items": 0, 2024-09-11T00:32:38.227 INFO:teuthology.orchestra.run.smithi165.stdout: "bluefs_file_writer_bytes": 0, 2024-09-11T00:32:38.227 INFO:teuthology.orchestra.run.smithi165.stdout: "bluefs_file_writer_items": 0, 2024-09-11T00:32:38.227 INFO:teuthology.orchestra.run.smithi165.stdout: "buffer_anon_bytes": 288221, 2024-09-11T00:32:38.227 INFO:teuthology.orchestra.run.smithi165.stdout: "buffer_anon_items": 108, 2024-09-11T00:32:38.227 INFO:teuthology.orchestra.run.smithi165.stdout: "buffer_meta_bytes": 88, 2024-09-11T00:32:38.227 INFO:teuthology.orchestra.run.smithi165.stdout: "buffer_meta_items": 1, 2024-09-11T00:32:38.228 INFO:teuthology.orchestra.run.smithi165.stdout: "osd_bytes": 0, 2024-09-11T00:32:38.228 INFO:teuthology.orchestra.run.smithi165.stdout: "osd_items": 0, 2024-09-11T00:32:38.228 INFO:teuthology.orchestra.run.smithi165.stdout: "osd_mapbl_bytes": 0, 2024-09-11T00:32:38.228 INFO:teuthology.orchestra.run.smithi165.stdout: "osd_mapbl_items": 0, 2024-09-11T00:32:38.228 INFO:teuthology.orchestra.run.smithi165.stdout: "osd_pglog_bytes": 0, 2024-09-11T00:32:38.228 INFO:teuthology.orchestra.run.smithi165.stdout: "osd_pglog_items": 0, 2024-09-11T00:32:38.228 INFO:teuthology.orchestra.run.smithi165.stdout: "osdmap_bytes": 9048, 2024-09-11T00:32:38.228 INFO:teuthology.orchestra.run.smithi165.stdout: "osdmap_items": 170, 2024-09-11T00:32:38.228 INFO:teuthology.orchestra.run.smithi165.stdout: "osdmap_mapping_bytes": 0, 2024-09-11T00:32:38.228 INFO:teuthology.orchestra.run.smithi165.stdout: "osdmap_mapping_items": 0, 2024-09-11T00:32:38.228 INFO:teuthology.orchestra.run.smithi165.stdout: "pgmap_bytes": 0, 2024-09-11T00:32:38.229 INFO:teuthology.orchestra.run.smithi165.stdout: "pgmap_items": 0, 2024-09-11T00:32:38.229 INFO:teuthology.orchestra.run.smithi165.stdout: "mds_co_bytes": 136103, 2024-09-11T00:32:38.229 INFO:teuthology.orchestra.run.smithi165.stdout: "mds_co_items": 1293, 2024-09-11T00:32:38.229 INFO:teuthology.orchestra.run.smithi165.stdout: "unittest_1_bytes": 0, 2024-09-11T00:32:38.229 INFO:teuthology.orchestra.run.smithi165.stdout: "unittest_1_items": 0, 2024-09-11T00:32:38.229 INFO:teuthology.orchestra.run.smithi165.stdout: "unittest_2_bytes": 0, 2024-09-11T00:32:38.229 INFO:teuthology.orchestra.run.smithi165.stdout: "unittest_2_items": 0 2024-09-11T00:32:38.229 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-11T00:32:38.229 INFO:teuthology.orchestra.run.smithi165.stdout: "objecter": { 2024-09-11T00:32:38.229 INFO:teuthology.orchestra.run.smithi165.stdout: "op_active": 0, 2024-09-11T00:32:38.229 INFO:teuthology.orchestra.run.smithi165.stdout: "op_laggy": 0, 2024-09-11T00:32:38.230 INFO:teuthology.orchestra.run.smithi165.stdout: "op_send": 71543, 2024-09-11T00:32:38.230 INFO:teuthology.orchestra.run.smithi165.stdout: "op_send_bytes": 183905652, 2024-09-11T00:32:38.230 INFO:teuthology.orchestra.run.smithi165.stdout: "op_resend": 28, 2024-09-11T00:32:38.230 INFO:teuthology.orchestra.run.smithi165.stdout: "op_reply": 71543, 2024-09-11T00:32:38.230 INFO:teuthology.orchestra.run.smithi165.stdout: "op_latency": { 2024-09-11T00:32:38.230 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 71543, 2024-09-11T00:32:38.230 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 766.148653912, 2024-09-11T00:32:38.230 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.010708925 2024-09-11T00:32:38.230 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-11T00:32:38.231 INFO:teuthology.orchestra.run.smithi165.stdout: "op_inflight": 0, 2024-09-11T00:32:38.231 INFO:teuthology.orchestra.run.smithi165.stdout: "oplen_avg": { 2024-09-11T00:32:38.231 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 71543, 2024-09-11T00:32:38.231 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 72759 2024-09-11T00:32:38.231 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-11T00:32:38.231 INFO:teuthology.orchestra.run.smithi165.stdout: "op": 71543, 2024-09-11T00:32:38.231 INFO:teuthology.orchestra.run.smithi165.stdout: "op_r": 835, 2024-09-11T00:32:38.231 INFO:teuthology.orchestra.run.smithi165.stdout: "op_w": 70708, 2024-09-11T00:32:38.231 INFO:teuthology.orchestra.run.smithi165.stdout: "op_rmw": 0, 2024-09-11T00:32:38.231 INFO:teuthology.orchestra.run.smithi165.stdout: "op_pg": 0, 2024-09-11T00:32:38.231 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_stat": 0, 2024-09-11T00:32:38.231 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_create": 10, 2024-09-11T00:32:38.232 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_read": 835, 2024-09-11T00:32:38.232 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_write": 32872, 2024-09-11T00:32:38.232 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_writefull": 524, 2024-09-11T00:32:38.232 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_writesame": 0, 2024-09-11T00:32:38.232 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_append": 0, 2024-09-11T00:32:38.232 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_zero": 0, 2024-09-11T00:32:38.232 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_truncate": 0, 2024-09-11T00:32:38.232 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_delete": 36511, 2024-09-11T00:32:38.232 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_mapext": 0, 2024-09-11T00:32:38.233 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_sparse_read": 0, 2024-09-11T00:32:38.233 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_clonerange": 0, 2024-09-11T00:32:38.233 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_getxattr": 0, 2024-09-11T00:32:38.233 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_setxattr": 20, 2024-09-11T00:32:38.233 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_cmpxattr": 0, 2024-09-11T00:32:38.233 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_rmxattr": 0, 2024-09-11T00:32:38.233 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_resetxattrs": 0, 2024-09-11T00:32:38.233 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_call": 0, 2024-09-11T00:32:38.233 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_watch": 0, 2024-09-11T00:32:38.233 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_notify": 0, 2024-09-11T00:32:38.233 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_src_cmpxattr": 0, 2024-09-11T00:32:38.233 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_pgls": 0, 2024-09-11T00:32:38.234 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_pgls_filter": 0, 2024-09-11T00:32:38.234 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_other": 0, 2024-09-11T00:32:38.234 INFO:teuthology.orchestra.run.smithi165.stdout: "linger_active": 0, 2024-09-11T00:32:38.234 INFO:teuthology.orchestra.run.smithi165.stdout: "linger_send": 0, 2024-09-11T00:32:38.234 INFO:teuthology.orchestra.run.smithi165.stdout: "linger_resend": 0, 2024-09-11T00:32:38.234 INFO:teuthology.orchestra.run.smithi165.stdout: "linger_ping": 0, 2024-09-11T00:32:38.234 INFO:teuthology.orchestra.run.smithi165.stdout: "poolop_active": 0, 2024-09-11T00:32:38.234 INFO:teuthology.orchestra.run.smithi165.stdout: "poolop_send": 0, 2024-09-11T00:32:38.234 INFO:teuthology.orchestra.run.smithi165.stdout: "poolop_resend": 0, 2024-09-11T00:32:38.234 INFO:teuthology.orchestra.run.smithi165.stdout: "poolstat_active": 0, 2024-09-11T00:32:38.234 INFO:teuthology.orchestra.run.smithi165.stdout: "poolstat_send": 0, 2024-09-11T00:32:38.234 INFO:teuthology.orchestra.run.smithi165.stdout: "poolstat_resend": 0, 2024-09-11T00:32:38.235 INFO:teuthology.orchestra.run.smithi165.stdout: "statfs_active": 0, 2024-09-11T00:32:38.235 INFO:teuthology.orchestra.run.smithi165.stdout: "statfs_send": 0, 2024-09-11T00:32:38.235 INFO:teuthology.orchestra.run.smithi165.stdout: "statfs_resend": 0, 2024-09-11T00:32:38.235 INFO:teuthology.orchestra.run.smithi165.stdout: "command_active": 0, 2024-09-11T00:32:38.235 INFO:teuthology.orchestra.run.smithi165.stdout: "command_send": 0, 2024-09-11T00:32:38.235 INFO:teuthology.orchestra.run.smithi165.stdout: "command_resend": 0, 2024-09-11T00:32:38.235 INFO:teuthology.orchestra.run.smithi165.stdout: "map_epoch": 87, 2024-09-11T00:32:38.235 INFO:teuthology.orchestra.run.smithi165.stdout: "map_full": 0, 2024-09-11T00:32:38.235 INFO:teuthology.orchestra.run.smithi165.stdout: "map_inc": 6, 2024-09-11T00:32:38.235 INFO:teuthology.orchestra.run.smithi165.stdout: "osd_sessions": 12, 2024-09-11T00:32:38.235 INFO:teuthology.orchestra.run.smithi165.stdout: "osd_session_open": 12, 2024-09-11T00:32:38.235 INFO:teuthology.orchestra.run.smithi165.stdout: "osd_session_close": 0, 2024-09-11T00:32:38.235 INFO:teuthology.orchestra.run.smithi165.stdout: "osd_laggy": 0, 2024-09-11T00:32:38.235 INFO:teuthology.orchestra.run.smithi165.stdout: "omap_wr": 1441, 2024-09-11T00:32:38.236 INFO:teuthology.orchestra.run.smithi165.stdout: "omap_rd": 0, 2024-09-11T00:32:38.236 INFO:teuthology.orchestra.run.smithi165.stdout: "omap_del": 546 2024-09-11T00:32:38.236 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-11T00:32:38.236 INFO:teuthology.orchestra.run.smithi165.stdout: "oft": { 2024-09-11T00:32:38.236 INFO:teuthology.orchestra.run.smithi165.stdout: "omap_total_objs": 1, 2024-09-11T00:32:38.236 INFO:teuthology.orchestra.run.smithi165.stdout: "omap_total_kv_pairs": 0, 2024-09-11T00:32:38.236 INFO:teuthology.orchestra.run.smithi165.stdout: "omap_total_updates": 0, 2024-09-11T00:32:38.236 INFO:teuthology.orchestra.run.smithi165.stdout: "omap_total_removes": 0 2024-09-11T00:32:38.236 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-11T00:32:38.236 INFO:teuthology.orchestra.run.smithi165.stdout: "purge_queue": { 2024-09-11T00:32:38.236 INFO:teuthology.orchestra.run.smithi165.stdout: "pq_executing_ops": 0, 2024-09-11T00:32:38.236 INFO:teuthology.orchestra.run.smithi165.stdout: "pq_executing_ops_high_water": 10, 2024-09-11T00:32:38.236 INFO:teuthology.orchestra.run.smithi165.stdout: "pq_executing": 0, 2024-09-11T00:32:38.236 INFO:teuthology.orchestra.run.smithi165.stdout: "pq_executing_high_water": 8, 2024-09-11T00:32:38.237 INFO:teuthology.orchestra.run.smithi165.stdout: "pq_executed_ops": 36457, 2024-09-11T00:32:38.237 INFO:teuthology.orchestra.run.smithi165.stdout: "pq_executed": 18238, 2024-09-11T00:32:38.237 INFO:teuthology.orchestra.run.smithi165.stdout: "pq_item_in_journal": 0 2024-09-11T00:32:38.237 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-11T00:32:38.237 INFO:teuthology.orchestra.run.smithi165.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-11T00:32:38.237 INFO:teuthology.orchestra.run.smithi165.stdout: "val": 0, 2024-09-11T00:32:38.237 INFO:teuthology.orchestra.run.smithi165.stdout: "max": 104857600, 2024-09-11T00:32:38.237 INFO:teuthology.orchestra.run.smithi165.stdout: "get_started": 0, 2024-09-11T00:32:38.237 INFO:teuthology.orchestra.run.smithi165.stdout: "get": 385741, 2024-09-11T00:32:38.237 INFO:teuthology.orchestra.run.smithi165.stdout: "get_sum": 82224693, 2024-09-11T00:32:38.237 INFO:teuthology.orchestra.run.smithi165.stdout: "get_or_fail_fail": 0, 2024-09-11T00:32:38.237 INFO:teuthology.orchestra.run.smithi165.stdout: "get_or_fail_success": 385741, 2024-09-11T00:32:38.237 INFO:teuthology.orchestra.run.smithi165.stdout: "take": 0, 2024-09-11T00:32:38.237 INFO:teuthology.orchestra.run.smithi165.stdout: "take_sum": 0, 2024-09-11T00:32:38.238 INFO:teuthology.orchestra.run.smithi165.stdout: "put": 385741, 2024-09-11T00:32:38.238 INFO:teuthology.orchestra.run.smithi165.stdout: "put_sum": 82224693, 2024-09-11T00:32:38.238 INFO:teuthology.orchestra.run.smithi165.stdout: "wait": { 2024-09-11T00:32:38.238 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-09-11T00:32:38.238 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-09-11T00:32:38.238 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-09-11T00:32:38.238 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-11T00:32:38.238 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:32:38 smithi165 ceph-mon[35356]: pgmap v1166: 129 pgs: 129 active+clean; 671 MiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 2.2 KiB/s wr, 13 op/s 2024-09-11T00:32:38.238 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-11T00:32:38.239 INFO:teuthology.orchestra.run.smithi165.stdout: "throttle-objecter_bytes": { 2024-09-11T00:32:38.239 INFO:teuthology.orchestra.run.smithi165.stdout: "val": 0, 2024-09-11T00:32:38.239 INFO:teuthology.orchestra.run.smithi165.stdout: "max": 104857600, 2024-09-11T00:32:38.239 INFO:teuthology.orchestra.run.smithi165.stdout: "get_started": 0, 2024-09-11T00:32:38.239 INFO:teuthology.orchestra.run.smithi165.stdout: "get": 0, 2024-09-11T00:32:38.239 INFO:teuthology.orchestra.run.smithi165.stdout: "get_sum": 0, 2024-09-11T00:32:38.239 INFO:teuthology.orchestra.run.smithi165.stdout: "get_or_fail_fail": 0, 2024-09-11T00:32:38.239 INFO:teuthology.orchestra.run.smithi165.stdout: "get_or_fail_success": 0, 2024-09-11T00:32:38.239 INFO:teuthology.orchestra.run.smithi165.stdout: "take": 71543, 2024-09-11T00:32:38.239 INFO:teuthology.orchestra.run.smithi165.stdout: "take_sum": 185893442, 2024-09-11T00:32:38.239 INFO:teuthology.orchestra.run.smithi165.stdout: "put": 71543, 2024-09-11T00:32:38.239 INFO:teuthology.orchestra.run.smithi165.stdout: "put_sum": 185893442, 2024-09-11T00:32:38.239 INFO:teuthology.orchestra.run.smithi165.stdout: "wait": { 2024-09-11T00:32:38.240 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-09-11T00:32:38.240 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-09-11T00:32:38.240 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-09-11T00:32:38.240 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-11T00:32:38.240 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-11T00:32:38.240 INFO:teuthology.orchestra.run.smithi165.stdout: "throttle-objecter_ops": { 2024-09-11T00:32:38.240 INFO:teuthology.orchestra.run.smithi165.stdout: "val": 0, 2024-09-11T00:32:38.240 INFO:teuthology.orchestra.run.smithi165.stdout: "max": 1024, 2024-09-11T00:32:38.240 INFO:teuthology.orchestra.run.smithi165.stdout: "get_started": 0, 2024-09-11T00:32:38.240 INFO:teuthology.orchestra.run.smithi165.stdout: "get": 0, 2024-09-11T00:32:38.240 INFO:teuthology.orchestra.run.smithi165.stdout: "get_sum": 0, 2024-09-11T00:32:38.240 INFO:teuthology.orchestra.run.smithi165.stdout: "get_or_fail_fail": 0, 2024-09-11T00:32:38.240 INFO:teuthology.orchestra.run.smithi165.stdout: "get_or_fail_success": 0, 2024-09-11T00:32:38.241 INFO:teuthology.orchestra.run.smithi165.stdout: "take": 71543, 2024-09-11T00:32:38.241 INFO:teuthology.orchestra.run.smithi165.stdout: "take_sum": 71543, 2024-09-11T00:32:38.241 INFO:teuthology.orchestra.run.smithi165.stdout: "put": 71543, 2024-09-11T00:32:38.241 INFO:teuthology.orchestra.run.smithi165.stdout: "put_sum": 71543, 2024-09-11T00:32:38.241 INFO:teuthology.orchestra.run.smithi165.stdout: "wait": { 2024-09-11T00:32:38.241 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-09-11T00:32:38.241 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-09-11T00:32:38.241 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-09-11T00:32:38.241 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-11T00:32:38.241 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-11T00:32:38.241 INFO:teuthology.orchestra.run.smithi165.stdout: "throttle-write_buf_throttle": { 2024-09-11T00:32:38.241 INFO:teuthology.orchestra.run.smithi165.stdout: "val": 0, 2024-09-11T00:32:38.242 INFO:teuthology.orchestra.run.smithi165.stdout: "max": 3758096384, 2024-09-11T00:32:38.242 INFO:teuthology.orchestra.run.smithi165.stdout: "get_started": 0, 2024-09-11T00:32:38.242 INFO:teuthology.orchestra.run.smithi165.stdout: "get": 18238, 2024-09-11T00:32:38.242 INFO:teuthology.orchestra.run.smithi165.stdout: "get_sum": 1987790, 2024-09-11T00:32:38.242 INFO:teuthology.orchestra.run.smithi165.stdout: "get_or_fail_fail": 0, 2024-09-11T00:32:38.242 INFO:teuthology.orchestra.run.smithi165.stdout: "get_or_fail_success": 18238, 2024-09-11T00:32:38.242 INFO:teuthology.orchestra.run.smithi165.stdout: "take": 0, 2024-09-11T00:32:38.242 INFO:teuthology.orchestra.run.smithi165.stdout: "take_sum": 0, 2024-09-11T00:32:38.242 INFO:teuthology.orchestra.run.smithi165.stdout: "put": 836, 2024-09-11T00:32:38.242 INFO:teuthology.orchestra.run.smithi165.stdout: "put_sum": 1987790, 2024-09-11T00:32:38.242 INFO:teuthology.orchestra.run.smithi165.stdout: "wait": { 2024-09-11T00:32:38.243 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-09-11T00:32:38.243 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-09-11T00:32:38.243 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-09-11T00:32:38.243 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-11T00:32:38.243 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-11T00:32:38.243 INFO:teuthology.orchestra.run.smithi165.stdout: "throttle-write_buf_throttle-0x55bd31978dc0": { 2024-09-11T00:32:38.243 INFO:teuthology.orchestra.run.smithi165.stdout: "val": 0, 2024-09-11T00:32:38.243 INFO:teuthology.orchestra.run.smithi165.stdout: "max": 3758096384, 2024-09-11T00:32:38.243 INFO:teuthology.orchestra.run.smithi165.stdout: "get_started": 0, 2024-09-11T00:32:38.243 INFO:teuthology.orchestra.run.smithi165.stdout: "get": 97378, 2024-09-11T00:32:38.243 INFO:teuthology.orchestra.run.smithi165.stdout: "get_sum": 179976384, 2024-09-11T00:32:38.243 INFO:teuthology.orchestra.run.smithi165.stdout: "get_or_fail_fail": 0, 2024-09-11T00:32:38.243 INFO:teuthology.orchestra.run.smithi165.stdout: "get_or_fail_success": 97378, 2024-09-11T00:32:38.243 INFO:teuthology.orchestra.run.smithi165.stdout: "take": 0, 2024-09-11T00:32:38.243 INFO:teuthology.orchestra.run.smithi165.stdout: "take_sum": 0, 2024-09-11T00:32:38.243 INFO:teuthology.orchestra.run.smithi165.stdout: "put": 32036, 2024-09-11T00:32:38.244 INFO:teuthology.orchestra.run.smithi165.stdout: "put_sum": 179976384, 2024-09-11T00:32:38.244 INFO:teuthology.orchestra.run.smithi165.stdout: "wait": { 2024-09-11T00:32:38.244 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-09-11T00:32:38.244 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-09-11T00:32:38.244 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-09-11T00:32:38.244 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-11T00:32:38.244 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-11T00:32:38.244 INFO:teuthology.orchestra.run.smithi165.stdout: "trackedop": { 2024-09-11T00:32:38.244 INFO:teuthology.orchestra.run.smithi165.stdout: "slow_ops_count": 1892 2024-09-11T00:32:38.244 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-11T00:32:38.244 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-09-11T00:32:38.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:32:38 smithi012 ceph-mon[30333]: pgmap v1166: 129 pgs: 129 active+clean; 671 MiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 2.2 KiB/s wr, 13 op/s 2024-09-11T00:32:38.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:32:38 smithi178 ceph-mon[35919]: pgmap v1166: 129 pgs: 129 active+clean; 671 MiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 2.2 KiB/s wr, 13 op/s 2024-09-11T00:32:39.215 INFO:tasks.check_counter:Daemon mds.h mds.exported=18 2024-09-11T00:32:39.215 INFO:tasks.check_counter:Daemon mds.h mds.imported=18 2024-09-11T00:32:39.215 DEBUG:tasks.check_counter:Getting stats from k 2024-09-11T00:32:39.215 DEBUG:teuthology.orchestra.run.smithi165:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid ff3ad74a-6fcf-11ef-bcea-c7b262605968 -- ceph daemon mds.k perf dump 2024-09-11T00:32:39.468 INFO:teuthology.orchestra.run.smithi165.stderr:Inferring config /var/lib/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T00:32:40.023 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-09-11T00:32:40.024 INFO:teuthology.orchestra.run.smithi165.stdout: "AsyncMessenger::Worker-0": { 2024-09-11T00:32:40.024 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_recv_messages": 1289, 2024-09-11T00:32:40.024 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_messages": 1291, 2024-09-11T00:32:40.024 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_recv_bytes": 38080069, 2024-09-11T00:32:40.024 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_bytes": 409966, 2024-09-11T00:32:40.024 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_created_connections": 9, 2024-09-11T00:32:40.024 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_active_connections": 4, 2024-09-11T00:32:40.024 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_total_time": 0.144629160, 2024-09-11T00:32:40.024 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_send_time": 0.055953631, 2024-09-11T00:32:40.025 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_recv_time": 0.050821616, 2024-09-11T00:32:40.025 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_fast_dispatch_time": 0.028103101, 2024-09-11T00:32:40.025 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_messages_queue_lat": { 2024-09-11T00:32:40.025 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 1289, 2024-09-11T00:32:40.025 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.044270135, 2024-09-11T00:32:40.025 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000034344 2024-09-11T00:32:40.025 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-11T00:32:40.025 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_handle_ack_lat": { 2024-09-11T00:32:40.025 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-09-11T00:32:40.025 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-09-11T00:32:40.025 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-09-11T00:32:40.026 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-11T00:32:40.026 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-11T00:32:40.026 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_encrypted_bytes": 0 2024-09-11T00:32:40.026 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-11T00:32:40.026 INFO:teuthology.orchestra.run.smithi165.stdout: "AsyncMessenger::Worker-1": { 2024-09-11T00:32:40.026 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_recv_messages": 1628, 2024-09-11T00:32:40.026 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_messages": 1982, 2024-09-11T00:32:40.026 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_recv_bytes": 75539626, 2024-09-11T00:32:40.026 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_bytes": 1310177, 2024-09-11T00:32:40.026 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_created_connections": 7, 2024-09-11T00:32:40.026 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_active_connections": 5, 2024-09-11T00:32:40.027 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_total_time": 0.227013935, 2024-09-11T00:32:40.027 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_send_time": 0.078462587, 2024-09-11T00:32:40.027 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_recv_time": 0.091101958, 2024-09-11T00:32:40.027 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_fast_dispatch_time": 0.043143825, 2024-09-11T00:32:40.027 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_messages_queue_lat": { 2024-09-11T00:32:40.027 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 1980, 2024-09-11T00:32:40.027 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.090193330, 2024-09-11T00:32:40.027 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000045552 2024-09-11T00:32:40.027 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-11T00:32:40.027 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_handle_ack_lat": { 2024-09-11T00:32:40.027 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-09-11T00:32:40.027 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-09-11T00:32:40.027 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-09-11T00:32:40.028 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-11T00:32:40.028 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_recv_encrypted_bytes": 320, 2024-09-11T00:32:40.028 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_encrypted_bytes": 793104 2024-09-11T00:32:40.028 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-11T00:32:40.028 INFO:teuthology.orchestra.run.smithi165.stdout: "AsyncMessenger::Worker-2": { 2024-09-11T00:32:40.028 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_recv_messages": 3341, 2024-09-11T00:32:40.028 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_messages": 3252, 2024-09-11T00:32:40.028 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_recv_bytes": 69142683, 2024-09-11T00:32:40.028 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_bytes": 1172740, 2024-09-11T00:32:40.028 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_created_connections": 5, 2024-09-11T00:32:40.028 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_active_connections": 4, 2024-09-11T00:32:40.028 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_total_time": 0.414684795, 2024-09-11T00:32:40.028 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_send_time": 0.163134575, 2024-09-11T00:32:40.028 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_recv_time": 0.160196541, 2024-09-11T00:32:40.029 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_fast_dispatch_time": 0.081175748, 2024-09-11T00:32:40.029 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_messages_queue_lat": { 2024-09-11T00:32:40.029 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 3252, 2024-09-11T00:32:40.029 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.144652311, 2024-09-11T00:32:40.029 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000044481 2024-09-11T00:32:40.029 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-11T00:32:40.029 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_handle_ack_lat": { 2024-09-11T00:32:40.029 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-09-11T00:32:40.029 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-09-11T00:32:40.029 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-09-11T00:32:40.029 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-11T00:32:40.029 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_recv_encrypted_bytes": 1185360, 2024-09-11T00:32:40.030 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_encrypted_bytes": 290608 2024-09-11T00:32:40.030 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-11T00:32:40.030 INFO:teuthology.orchestra.run.smithi165.stdout: "cct": { 2024-09-11T00:32:40.030 INFO:teuthology.orchestra.run.smithi165.stdout: "total_workers": 1, 2024-09-11T00:32:40.030 INFO:teuthology.orchestra.run.smithi165.stdout: "unhealthy_workers": 0 2024-09-11T00:32:40.030 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-11T00:32:40.030 INFO:teuthology.orchestra.run.smithi165.stdout: "finisher-MDSRank": { 2024-09-11T00:32:40.030 INFO:teuthology.orchestra.run.smithi165.stdout: "queue_len": 0, 2024-09-11T00:32:40.030 INFO:teuthology.orchestra.run.smithi165.stdout: "complete_latency": { 2024-09-11T00:32:40.030 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 9666, 2024-09-11T00:32:40.030 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.234231449, 2024-09-11T00:32:40.030 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000024232 2024-09-11T00:32:40.031 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-11T00:32:40.031 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-11T00:32:40.031 INFO:teuthology.orchestra.run.smithi165.stdout: "finisher-PurgeQueue": { 2024-09-11T00:32:40.031 INFO:teuthology.orchestra.run.smithi165.stdout: "queue_len": 0, 2024-09-11T00:32:40.031 INFO:teuthology.orchestra.run.smithi165.stdout: "complete_latency": { 2024-09-11T00:32:40.031 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-09-11T00:32:40.031 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-09-11T00:32:40.031 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-09-11T00:32:40.031 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-11T00:32:40.031 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-11T00:32:40.031 INFO:teuthology.orchestra.run.smithi165.stdout: "mds": { 2024-09-11T00:32:40.031 INFO:teuthology.orchestra.run.smithi165.stdout: "request": 0, 2024-09-11T00:32:40.031 INFO:teuthology.orchestra.run.smithi165.stdout: "reply": 0, 2024-09-11T00:32:40.032 INFO:teuthology.orchestra.run.smithi165.stdout: "reply_latency": { 2024-09-11T00:32:40.032 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-09-11T00:32:40.032 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-09-11T00:32:40.032 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-09-11T00:32:40.032 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-11T00:32:40.032 INFO:teuthology.orchestra.run.smithi165.stdout: "slow_reply": 0, 2024-09-11T00:32:40.032 INFO:teuthology.orchestra.run.smithi165.stdout: "forward": 0, 2024-09-11T00:32:40.032 INFO:teuthology.orchestra.run.smithi165.stdout: "dir_fetch_complete": 0, 2024-09-11T00:32:40.032 INFO:teuthology.orchestra.run.smithi165.stdout: "dir_fetch_keys": 0, 2024-09-11T00:32:40.032 INFO:teuthology.orchestra.run.smithi165.stdout: "dir_commit": 0, 2024-09-11T00:32:40.032 INFO:teuthology.orchestra.run.smithi165.stdout: "dir_split": 0, 2024-09-11T00:32:40.032 INFO:teuthology.orchestra.run.smithi165.stdout: "dir_merge": 0, 2024-09-11T00:32:40.032 INFO:teuthology.orchestra.run.smithi165.stdout: "inodes": 18303, 2024-09-11T00:32:40.033 INFO:teuthology.orchestra.run.smithi165.stdout: "inodes_top": 23, 2024-09-11T00:32:40.033 INFO:teuthology.orchestra.run.smithi165.stdout: "inodes_bottom": 18280, 2024-09-11T00:32:40.033 INFO:teuthology.orchestra.run.smithi165.stdout: "inodes_pin_tail": 0, 2024-09-11T00:32:40.033 INFO:teuthology.orchestra.run.smithi165.stdout: "inodes_pinned": 18269, 2024-09-11T00:32:40.033 INFO:teuthology.orchestra.run.smithi165.stdout: "inodes_expired": 0, 2024-09-11T00:32:40.033 INFO:teuthology.orchestra.run.smithi165.stdout: "inodes_with_caps": 0, 2024-09-11T00:32:40.033 INFO:teuthology.orchestra.run.smithi165.stdout: "caps": 0, 2024-09-11T00:32:40.033 INFO:teuthology.orchestra.run.smithi165.stdout: "subtrees": 1, 2024-09-11T00:32:40.033 INFO:teuthology.orchestra.run.smithi165.stdout: "traverse": 0, 2024-09-11T00:32:40.033 INFO:teuthology.orchestra.run.smithi165.stdout: "traverse_hit": 0, 2024-09-11T00:32:40.033 INFO:teuthology.orchestra.run.smithi165.stdout: "traverse_forward": 0, 2024-09-11T00:32:40.033 INFO:teuthology.orchestra.run.smithi165.stdout: "traverse_discover": 0, 2024-09-11T00:32:40.034 INFO:teuthology.orchestra.run.smithi165.stdout: "traverse_dir_fetch": 0, 2024-09-11T00:32:40.034 INFO:teuthology.orchestra.run.smithi165.stdout: "traverse_remote_ino": 0, 2024-09-11T00:32:40.034 INFO:teuthology.orchestra.run.smithi165.stdout: "traverse_lock": 0, 2024-09-11T00:32:40.034 INFO:teuthology.orchestra.run.smithi165.stdout: "load_cent": 0, 2024-09-11T00:32:40.034 INFO:teuthology.orchestra.run.smithi165.stdout: "q": 0, 2024-09-11T00:32:40.034 INFO:teuthology.orchestra.run.smithi165.stdout: "exported": 0, 2024-09-11T00:32:40.034 INFO:teuthology.orchestra.run.smithi165.stdout: "exported_inodes": 0, 2024-09-11T00:32:40.034 INFO:teuthology.orchestra.run.smithi165.stdout: "imported": 0, 2024-09-11T00:32:40.034 INFO:teuthology.orchestra.run.smithi165.stdout: "imported_inodes": 0, 2024-09-11T00:32:40.034 INFO:teuthology.orchestra.run.smithi165.stdout: "openino_dir_fetch": 0, 2024-09-11T00:32:40.034 INFO:teuthology.orchestra.run.smithi165.stdout: "openino_backtrace_fetch": 0, 2024-09-11T00:32:40.034 INFO:teuthology.orchestra.run.smithi165.stdout: "openino_peer_discover": 0, 2024-09-11T00:32:40.034 INFO:teuthology.orchestra.run.smithi165.stdout: "root_rfiles": 0, 2024-09-11T00:32:40.035 INFO:teuthology.orchestra.run.smithi165.stdout: "root_rbytes": 0, 2024-09-11T00:32:40.035 INFO:teuthology.orchestra.run.smithi165.stdout: "root_rsnaps": 0, 2024-09-11T00:32:40.035 INFO:teuthology.orchestra.run.smithi165.stdout: "scrub_backtrace_fetch": 0, 2024-09-11T00:32:40.035 INFO:teuthology.orchestra.run.smithi165.stdout: "scrub_set_tag": 0, 2024-09-11T00:32:40.035 INFO:teuthology.orchestra.run.smithi165.stdout: "scrub_backtrace_repaired": 0, 2024-09-11T00:32:40.035 INFO:teuthology.orchestra.run.smithi165.stdout: "scrub_inotable_repaired": 0, 2024-09-11T00:32:40.035 INFO:teuthology.orchestra.run.smithi165.stdout: "scrub_dir_inodes": 0, 2024-09-11T00:32:40.035 INFO:teuthology.orchestra.run.smithi165.stdout: "scrub_dir_base_inodes": 0, 2024-09-11T00:32:40.035 INFO:teuthology.orchestra.run.smithi165.stdout: "scrub_dirfrag_rstats": 0, 2024-09-11T00:32:40.035 INFO:teuthology.orchestra.run.smithi165.stdout: "scrub_file_inodes": 0, 2024-09-11T00:32:40.035 INFO:teuthology.orchestra.run.smithi165.stdout: "handle_inode_file_caps": 0, 2024-09-11T00:32:40.035 INFO:teuthology.orchestra.run.smithi165.stdout: "ceph_cap_op_revoke": 0, 2024-09-11T00:32:40.035 INFO:teuthology.orchestra.run.smithi165.stdout: "ceph_cap_op_grant": 0, 2024-09-11T00:32:40.036 INFO:teuthology.orchestra.run.smithi165.stdout: "ceph_cap_op_trunc": 0, 2024-09-11T00:32:40.036 INFO:teuthology.orchestra.run.smithi165.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-11T00:32:40.036 INFO:teuthology.orchestra.run.smithi165.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-11T00:32:40.036 INFO:teuthology.orchestra.run.smithi165.stdout: "handle_client_caps": 0, 2024-09-11T00:32:40.036 INFO:teuthology.orchestra.run.smithi165.stdout: "handle_client_caps_dirty": 0, 2024-09-11T00:32:40.036 INFO:teuthology.orchestra.run.smithi165.stdout: "handle_client_cap_release": 0, 2024-09-11T00:32:40.036 INFO:teuthology.orchestra.run.smithi165.stdout: "process_request_cap_release": 0 2024-09-11T00:32:40.036 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-11T00:32:40.036 INFO:teuthology.orchestra.run.smithi165.stdout: "mds_cache": { 2024-09-11T00:32:40.036 INFO:teuthology.orchestra.run.smithi165.stdout: "dir_update": 0, 2024-09-11T00:32:40.036 INFO:teuthology.orchestra.run.smithi165.stdout: "dir_update_receipt": 0, 2024-09-11T00:32:40.036 INFO:teuthology.orchestra.run.smithi165.stdout: "dir_try_discover": 0, 2024-09-11T00:32:40.037 INFO:teuthology.orchestra.run.smithi165.stdout: "dir_send_discover": 0, 2024-09-11T00:32:40.037 INFO:teuthology.orchestra.run.smithi165.stdout: "dir_handle_discover": 0, 2024-09-11T00:32:40.037 INFO:teuthology.orchestra.run.smithi165.stdout: "num_strays": 0, 2024-09-11T00:32:40.037 INFO:teuthology.orchestra.run.smithi165.stdout: "num_strays_delayed": 0, 2024-09-11T00:32:40.037 INFO:teuthology.orchestra.run.smithi165.stdout: "num_strays_enqueuing": 0, 2024-09-11T00:32:40.037 INFO:teuthology.orchestra.run.smithi165.stdout: "strays_created": 0, 2024-09-11T00:32:40.037 INFO:teuthology.orchestra.run.smithi165.stdout: "strays_enqueued": 0, 2024-09-11T00:32:40.037 INFO:teuthology.orchestra.run.smithi165.stdout: "strays_reintegrated": 0, 2024-09-11T00:32:40.037 INFO:teuthology.orchestra.run.smithi165.stdout: "strays_migrated": 0, 2024-09-11T00:32:40.037 INFO:teuthology.orchestra.run.smithi165.stdout: "num_recovering_processing": 0, 2024-09-11T00:32:40.037 INFO:teuthology.orchestra.run.smithi165.stdout: "num_recovering_enqueued": 0, 2024-09-11T00:32:40.037 INFO:teuthology.orchestra.run.smithi165.stdout: "num_recovering_prioritized": 0, 2024-09-11T00:32:40.037 INFO:teuthology.orchestra.run.smithi165.stdout: "recovery_started": 0, 2024-09-11T00:32:40.038 INFO:teuthology.orchestra.run.smithi165.stdout: "recovery_completed": 0, 2024-09-11T00:32:40.038 INFO:teuthology.orchestra.run.smithi165.stdout: "ireq_quiesce_path": 0, 2024-09-11T00:32:40.038 INFO:teuthology.orchestra.run.smithi165.stdout: "ireq_quiesce_inode": 0, 2024-09-11T00:32:40.038 INFO:teuthology.orchestra.run.smithi165.stdout: "ireq_enqueue_scrub": 0, 2024-09-11T00:32:40.038 INFO:teuthology.orchestra.run.smithi165.stdout: "ireq_exportdir": 0, 2024-09-11T00:32:40.038 INFO:teuthology.orchestra.run.smithi165.stdout: "ireq_flush": 0, 2024-09-11T00:32:40.038 INFO:teuthology.orchestra.run.smithi165.stdout: "ireq_fragmentdir": 0, 2024-09-11T00:32:40.038 INFO:teuthology.orchestra.run.smithi165.stdout: "ireq_fragstats": 0, 2024-09-11T00:32:40.038 INFO:teuthology.orchestra.run.smithi165.stdout: "ireq_inodestats": 0, 2024-09-11T00:32:40.038 INFO:teuthology.orchestra.run.smithi165.stdout: "uninline_started": 0, 2024-09-11T00:32:40.038 INFO:teuthology.orchestra.run.smithi165.stdout: "uninline_succeeded": 0, 2024-09-11T00:32:40.038 INFO:teuthology.orchestra.run.smithi165.stdout: "uninline_write_failed": 0 2024-09-11T00:32:40.038 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-11T00:32:40.039 INFO:teuthology.orchestra.run.smithi165.stdout: "mds_log": { 2024-09-11T00:32:40.039 INFO:teuthology.orchestra.run.smithi165.stdout: "evlrg": 0, 2024-09-11T00:32:40.039 INFO:teuthology.orchestra.run.smithi165.stdout: "evadd": 0, 2024-09-11T00:32:40.039 INFO:teuthology.orchestra.run.smithi165.stdout: "evex": 0, 2024-09-11T00:32:40.039 INFO:teuthology.orchestra.run.smithi165.stdout: "evtrm": 0, 2024-09-11T00:32:40.039 INFO:teuthology.orchestra.run.smithi165.stdout: "ev": 97378, 2024-09-11T00:32:40.039 INFO:teuthology.orchestra.run.smithi165.stdout: "evexg": 0, 2024-09-11T00:32:40.039 INFO:teuthology.orchestra.run.smithi165.stdout: "evexd": 0, 2024-09-11T00:32:40.039 INFO:teuthology.orchestra.run.smithi165.stdout: "segadd": 0, 2024-09-11T00:32:40.039 INFO:teuthology.orchestra.run.smithi165.stdout: "segex": 0, 2024-09-11T00:32:40.039 INFO:teuthology.orchestra.run.smithi165.stdout: "segtrm": 0, 2024-09-11T00:32:40.039 INFO:teuthology.orchestra.run.smithi165.stdout: "seg": 124, 2024-09-11T00:32:40.039 INFO:teuthology.orchestra.run.smithi165.stdout: "segmjr": 9, 2024-09-11T00:32:40.040 INFO:teuthology.orchestra.run.smithi165.stdout: "segexg": 0, 2024-09-11T00:32:40.040 INFO:teuthology.orchestra.run.smithi165.stdout: "segexd": 0, 2024-09-11T00:32:40.040 INFO:teuthology.orchestra.run.smithi165.stdout: "expos": 4194304, 2024-09-11T00:32:40.040 INFO:teuthology.orchestra.run.smithi165.stdout: "wrpos": 184170688, 2024-09-11T00:32:40.040 INFO:teuthology.orchestra.run.smithi165.stdout: "rdpos": 184170089, 2024-09-11T00:32:40.040 INFO:teuthology.orchestra.run.smithi165.stdout: "jlat": { 2024-09-11T00:32:40.040 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-09-11T00:32:40.040 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-09-11T00:32:40.040 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-09-11T00:32:40.040 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-11T00:32:40.040 INFO:teuthology.orchestra.run.smithi165.stdout: "replayed": 97378 2024-09-11T00:32:40.040 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-11T00:32:40.040 INFO:teuthology.orchestra.run.smithi165.stdout: "mds_mem": { 2024-09-11T00:32:40.041 INFO:teuthology.orchestra.run.smithi165.stdout: "ino": 14, 2024-09-11T00:32:40.041 INFO:teuthology.orchestra.run.smithi165.stdout: "ino+": 18355, 2024-09-11T00:32:40.041 INFO:teuthology.orchestra.run.smithi165.stdout: "ino-": 18341, 2024-09-11T00:32:40.041 INFO:teuthology.orchestra.run.smithi165.stdout: "dir": 12, 2024-09-11T00:32:40.041 INFO:teuthology.orchestra.run.smithi165.stdout: "dir+": 69, 2024-09-11T00:32:40.041 INFO:teuthology.orchestra.run.smithi165.stdout: "dir-": 57, 2024-09-11T00:32:40.041 INFO:teuthology.orchestra.run.smithi165.stdout: "dn": 18303, 2024-09-11T00:32:40.041 INFO:teuthology.orchestra.run.smithi165.stdout: "dn+": 18801, 2024-09-11T00:32:40.041 INFO:teuthology.orchestra.run.smithi165.stdout: "dn-": 498, 2024-09-11T00:32:40.041 INFO:teuthology.orchestra.run.smithi165.stdout: "cap": 0, 2024-09-11T00:32:40.041 INFO:teuthology.orchestra.run.smithi165.stdout: "cap+": 0, 2024-09-11T00:32:40.041 INFO:teuthology.orchestra.run.smithi165.stdout: "cap-": 0, 2024-09-11T00:32:40.041 INFO:teuthology.orchestra.run.smithi165.stdout: "rss": 68824, 2024-09-11T00:32:40.042 INFO:teuthology.orchestra.run.smithi165.stdout: "heap": 270389248 2024-09-11T00:32:40.042 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-11T00:32:40.042 INFO:teuthology.orchestra.run.smithi165.stdout: "mds_server": { 2024-09-11T00:32:40.042 INFO:teuthology.orchestra.run.smithi165.stdout: "dispatch_client_request": 0, 2024-09-11T00:32:40.042 INFO:teuthology.orchestra.run.smithi165.stdout: "dispatch_server_request": 0, 2024-09-11T00:32:40.042 INFO:teuthology.orchestra.run.smithi165.stdout: "handle_client_request": 0, 2024-09-11T00:32:40.042 INFO:teuthology.orchestra.run.smithi165.stdout: "handle_client_session": 0, 2024-09-11T00:32:40.042 INFO:teuthology.orchestra.run.smithi165.stdout: "handle_peer_request": 0, 2024-09-11T00:32:40.042 INFO:teuthology.orchestra.run.smithi165.stdout: "req_create_latency": { 2024-09-11T00:32:40.042 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-09-11T00:32:40.042 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-09-11T00:32:40.042 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-09-11T00:32:40.042 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-11T00:32:40.043 INFO:teuthology.orchestra.run.smithi165.stdout: "req_getattr_latency": { 2024-09-11T00:32:40.043 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-09-11T00:32:40.043 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-09-11T00:32:40.043 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-09-11T00:32:40.043 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-11T00:32:40.043 INFO:teuthology.orchestra.run.smithi165.stdout: "req_getfilelock_latency": { 2024-09-11T00:32:40.043 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-09-11T00:32:40.043 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-09-11T00:32:40.043 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-09-11T00:32:40.043 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-11T00:32:40.043 INFO:teuthology.orchestra.run.smithi165.stdout: "req_link_latency": { 2024-09-11T00:32:40.043 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-09-11T00:32:40.043 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-09-11T00:32:40.046 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-09-11T00:32:40.046 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-11T00:32:40.046 INFO:teuthology.orchestra.run.smithi165.stdout: "req_lookup_latency": { 2024-09-11T00:32:40.046 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-09-11T00:32:40.046 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-09-11T00:32:40.046 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-09-11T00:32:40.046 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-11T00:32:40.047 INFO:teuthology.orchestra.run.smithi165.stdout: "req_lookuphash_latency": { 2024-09-11T00:32:40.047 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-09-11T00:32:40.047 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-09-11T00:32:40.047 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-09-11T00:32:40.047 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-11T00:32:40.047 INFO:teuthology.orchestra.run.smithi165.stdout: "req_lookupino_latency": { 2024-09-11T00:32:40.047 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-09-11T00:32:40.047 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-09-11T00:32:40.047 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-09-11T00:32:40.047 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-11T00:32:40.047 INFO:teuthology.orchestra.run.smithi165.stdout: "req_lookupname_latency": { 2024-09-11T00:32:40.047 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-09-11T00:32:40.047 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-09-11T00:32:40.048 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-09-11T00:32:40.048 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-11T00:32:40.048 INFO:teuthology.orchestra.run.smithi165.stdout: "req_lookupparent_latency": { 2024-09-11T00:32:40.048 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-09-11T00:32:40.048 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-09-11T00:32:40.048 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-09-11T00:32:40.048 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-11T00:32:40.048 INFO:teuthology.orchestra.run.smithi165.stdout: "req_lookupsnap_latency": { 2024-09-11T00:32:40.048 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-09-11T00:32:40.048 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-09-11T00:32:40.048 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-09-11T00:32:40.048 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-11T00:32:40.048 INFO:teuthology.orchestra.run.smithi165.stdout: "req_lssnap_latency": { 2024-09-11T00:32:40.049 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-09-11T00:32:40.049 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-09-11T00:32:40.049 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-09-11T00:32:40.049 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-11T00:32:40.049 INFO:teuthology.orchestra.run.smithi165.stdout: "req_mkdir_latency": { 2024-09-11T00:32:40.049 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-09-11T00:32:40.049 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-09-11T00:32:40.049 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-09-11T00:32:40.049 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-11T00:32:40.049 INFO:teuthology.orchestra.run.smithi165.stdout: "req_mknod_latency": { 2024-09-11T00:32:40.049 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-09-11T00:32:40.049 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-09-11T00:32:40.049 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-09-11T00:32:40.049 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-11T00:32:40.050 INFO:teuthology.orchestra.run.smithi165.stdout: "req_mksnap_latency": { 2024-09-11T00:32:40.050 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-09-11T00:32:40.050 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-09-11T00:32:40.050 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-09-11T00:32:40.050 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-11T00:32:40.050 INFO:teuthology.orchestra.run.smithi165.stdout: "req_open_latency": { 2024-09-11T00:32:40.050 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-09-11T00:32:40.050 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-09-11T00:32:40.050 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-09-11T00:32:40.050 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-11T00:32:40.050 INFO:teuthology.orchestra.run.smithi165.stdout: "req_readdir_latency": { 2024-09-11T00:32:40.050 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-09-11T00:32:40.050 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-09-11T00:32:40.051 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-09-11T00:32:40.051 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-11T00:32:40.051 INFO:teuthology.orchestra.run.smithi165.stdout: "req_rename_latency": { 2024-09-11T00:32:40.051 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-09-11T00:32:40.051 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-09-11T00:32:40.051 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-09-11T00:32:40.051 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-11T00:32:40.051 INFO:teuthology.orchestra.run.smithi165.stdout: "req_renamesnap_latency": { 2024-09-11T00:32:40.051 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-09-11T00:32:40.051 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-09-11T00:32:40.051 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-09-11T00:32:40.051 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-11T00:32:40.051 INFO:teuthology.orchestra.run.smithi165.stdout: "req_snapdiff_latency": { 2024-09-11T00:32:40.051 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-09-11T00:32:40.052 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-09-11T00:32:40.052 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-09-11T00:32:40.052 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-11T00:32:40.052 INFO:teuthology.orchestra.run.smithi165.stdout: "req_rmdir_latency": { 2024-09-11T00:32:40.052 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-09-11T00:32:40.052 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-09-11T00:32:40.052 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-09-11T00:32:40.052 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-11T00:32:40.052 INFO:teuthology.orchestra.run.smithi165.stdout: "req_rmsnap_latency": { 2024-09-11T00:32:40.052 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-09-11T00:32:40.052 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-09-11T00:32:40.052 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-09-11T00:32:40.052 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-11T00:32:40.052 INFO:teuthology.orchestra.run.smithi165.stdout: "req_rmxattr_latency": { 2024-09-11T00:32:40.053 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-09-11T00:32:40.053 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-09-11T00:32:40.053 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-09-11T00:32:40.053 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-11T00:32:40.053 INFO:teuthology.orchestra.run.smithi165.stdout: "req_setattr_latency": { 2024-09-11T00:32:40.053 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-09-11T00:32:40.053 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-09-11T00:32:40.053 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-09-11T00:32:40.053 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-11T00:32:40.053 INFO:teuthology.orchestra.run.smithi165.stdout: "req_setdirlayout_latency": { 2024-09-11T00:32:40.053 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-09-11T00:32:40.053 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-09-11T00:32:40.053 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-09-11T00:32:40.053 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-11T00:32:40.054 INFO:teuthology.orchestra.run.smithi165.stdout: "req_setfilelock_latency": { 2024-09-11T00:32:40.054 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-09-11T00:32:40.054 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-09-11T00:32:40.054 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-09-11T00:32:40.054 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-11T00:32:40.054 INFO:teuthology.orchestra.run.smithi165.stdout: "req_setlayout_latency": { 2024-09-11T00:32:40.054 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-09-11T00:32:40.054 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-09-11T00:32:40.054 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-09-11T00:32:40.054 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-11T00:32:40.054 INFO:teuthology.orchestra.run.smithi165.stdout: "req_setxattr_latency": { 2024-09-11T00:32:40.054 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-09-11T00:32:40.055 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-09-11T00:32:40.055 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-09-11T00:32:40.055 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-11T00:32:40.055 INFO:teuthology.orchestra.run.smithi165.stdout: "req_symlink_latency": { 2024-09-11T00:32:40.055 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-09-11T00:32:40.055 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-09-11T00:32:40.055 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-09-11T00:32:40.055 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-11T00:32:40.055 INFO:teuthology.orchestra.run.smithi165.stdout: "req_unlink_latency": { 2024-09-11T00:32:40.055 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-09-11T00:32:40.055 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-09-11T00:32:40.055 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-09-11T00:32:40.055 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-11T00:32:40.055 INFO:teuthology.orchestra.run.smithi165.stdout: "cap_revoke_eviction": 0, 2024-09-11T00:32:40.056 INFO:teuthology.orchestra.run.smithi165.stdout: "cap_acquisition_throttle": 0, 2024-09-11T00:32:40.056 INFO:teuthology.orchestra.run.smithi165.stdout: "req_getvxattr_latency": { 2024-09-11T00:32:40.056 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-09-11T00:32:40.056 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-09-11T00:32:40.056 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-09-11T00:32:40.056 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-11T00:32:40.056 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-11T00:32:40.056 INFO:teuthology.orchestra.run.smithi165.stdout: "mds_sessions": { 2024-09-11T00:32:40.056 INFO:teuthology.orchestra.run.smithi165.stdout: "session_count": 1, 2024-09-11T00:32:40.056 INFO:teuthology.orchestra.run.smithi165.stdout: "session_add": 1, 2024-09-11T00:32:40.056 INFO:teuthology.orchestra.run.smithi165.stdout: "session_remove": 0, 2024-09-11T00:32:40.056 INFO:teuthology.orchestra.run.smithi165.stdout: "sessions_open": 1, 2024-09-11T00:32:40.057 INFO:teuthology.orchestra.run.smithi165.stdout: "sessions_stale": 0, 2024-09-11T00:32:40.057 INFO:teuthology.orchestra.run.smithi165.stdout: "total_load": 0, 2024-09-11T00:32:40.057 INFO:teuthology.orchestra.run.smithi165.stdout: "average_load": 0, 2024-09-11T00:32:40.057 INFO:teuthology.orchestra.run.smithi165.stdout: "avg_session_uptime": 2954, 2024-09-11T00:32:40.057 INFO:teuthology.orchestra.run.smithi165.stdout: "mdthresh_evicted": 0 2024-09-11T00:32:40.057 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-11T00:32:40.057 INFO:teuthology.orchestra.run.smithi165.stdout: "mempool": { 2024-09-11T00:32:40.057 INFO:teuthology.orchestra.run.smithi165.stdout: "bloom_filter_bytes": 0, 2024-09-11T00:32:40.057 INFO:teuthology.orchestra.run.smithi165.stdout: "bloom_filter_items": 0, 2024-09-11T00:32:40.057 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_alloc_bytes": 0, 2024-09-11T00:32:40.057 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_alloc_items": 0, 2024-09-11T00:32:40.057 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_cache_data_bytes": 0, 2024-09-11T00:32:40.057 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_cache_data_items": 0, 2024-09-11T00:32:40.058 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-11T00:32:40.058 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_cache_onode_items": 0, 2024-09-11T00:32:40.058 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-11T00:32:40.058 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_cache_meta_items": 0, 2024-09-11T00:32:40.058 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_cache_other_bytes": 0, 2024-09-11T00:32:40.058 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_cache_other_items": 0, 2024-09-11T00:32:40.058 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-11T00:32:40.058 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_cache_buffer_items": 0, 2024-09-11T00:32:40.058 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_extent_bytes": 0, 2024-09-11T00:32:40.058 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_extent_items": 0, 2024-09-11T00:32:40.058 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_blob_bytes": 0, 2024-09-11T00:32:40.058 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_blob_items": 0, 2024-09-11T00:32:40.058 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-11T00:32:40.058 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_shared_blob_items": 0, 2024-09-11T00:32:40.059 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-11T00:32:40.059 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_inline_bl_items": 0, 2024-09-11T00:32:40.059 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_fsck_bytes": 0, 2024-09-11T00:32:40.059 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_fsck_items": 0, 2024-09-11T00:32:40.059 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_txc_bytes": 0, 2024-09-11T00:32:40.059 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_txc_items": 0, 2024-09-11T00:32:40.059 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-11T00:32:40.059 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_writing_deferred_items": 0, 2024-09-11T00:32:40.059 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_writing_bytes": 0, 2024-09-11T00:32:40.059 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_writing_items": 0, 2024-09-11T00:32:40.059 INFO:teuthology.orchestra.run.smithi165.stdout: "bluefs_bytes": 0, 2024-09-11T00:32:40.059 INFO:teuthology.orchestra.run.smithi165.stdout: "bluefs_items": 0, 2024-09-11T00:32:40.059 INFO:teuthology.orchestra.run.smithi165.stdout: "bluefs_file_reader_bytes": 0, 2024-09-11T00:32:40.059 INFO:teuthology.orchestra.run.smithi165.stdout: "bluefs_file_reader_items": 0, 2024-09-11T00:32:40.060 INFO:teuthology.orchestra.run.smithi165.stdout: "bluefs_file_writer_bytes": 0, 2024-09-11T00:32:40.060 INFO:teuthology.orchestra.run.smithi165.stdout: "bluefs_file_writer_items": 0, 2024-09-11T00:32:40.060 INFO:teuthology.orchestra.run.smithi165.stdout: "buffer_anon_bytes": 232212, 2024-09-11T00:32:40.060 INFO:teuthology.orchestra.run.smithi165.stdout: "buffer_anon_items": 78, 2024-09-11T00:32:40.060 INFO:teuthology.orchestra.run.smithi165.stdout: "buffer_meta_bytes": 0, 2024-09-11T00:32:40.060 INFO:teuthology.orchestra.run.smithi165.stdout: "buffer_meta_items": 0, 2024-09-11T00:32:40.060 INFO:teuthology.orchestra.run.smithi165.stdout: "osd_bytes": 0, 2024-09-11T00:32:40.060 INFO:teuthology.orchestra.run.smithi165.stdout: "osd_items": 0, 2024-09-11T00:32:40.060 INFO:teuthology.orchestra.run.smithi165.stdout: "osd_mapbl_bytes": 0, 2024-09-11T00:32:40.060 INFO:teuthology.orchestra.run.smithi165.stdout: "osd_mapbl_items": 0, 2024-09-11T00:32:40.060 INFO:teuthology.orchestra.run.smithi165.stdout: "osd_pglog_bytes": 0, 2024-09-11T00:32:40.060 INFO:teuthology.orchestra.run.smithi165.stdout: "osd_pglog_items": 0, 2024-09-11T00:32:40.060 INFO:teuthology.orchestra.run.smithi165.stdout: "osdmap_bytes": 9048, 2024-09-11T00:32:40.060 INFO:teuthology.orchestra.run.smithi165.stdout: "osdmap_items": 170, 2024-09-11T00:32:40.061 INFO:teuthology.orchestra.run.smithi165.stdout: "osdmap_mapping_bytes": 0, 2024-09-11T00:32:40.061 INFO:teuthology.orchestra.run.smithi165.stdout: "osdmap_mapping_items": 0, 2024-09-11T00:32:40.061 INFO:teuthology.orchestra.run.smithi165.stdout: "pgmap_bytes": 0, 2024-09-11T00:32:40.061 INFO:teuthology.orchestra.run.smithi165.stdout: "pgmap_items": 0, 2024-09-11T00:32:40.061 INFO:teuthology.orchestra.run.smithi165.stdout: "mds_co_bytes": 12493366, 2024-09-11T00:32:40.061 INFO:teuthology.orchestra.run.smithi165.stdout: "mds_co_items": 73379, 2024-09-11T00:32:40.061 INFO:teuthology.orchestra.run.smithi165.stdout: "unittest_1_bytes": 0, 2024-09-11T00:32:40.061 INFO:teuthology.orchestra.run.smithi165.stdout: "unittest_1_items": 0, 2024-09-11T00:32:40.061 INFO:teuthology.orchestra.run.smithi165.stdout: "unittest_2_bytes": 0, 2024-09-11T00:32:40.061 INFO:teuthology.orchestra.run.smithi165.stdout: "unittest_2_items": 0 2024-09-11T00:32:40.061 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-11T00:32:40.061 INFO:teuthology.orchestra.run.smithi165.stdout: "objecter": { 2024-09-11T00:32:40.061 INFO:teuthology.orchestra.run.smithi165.stdout: "op_active": 0, 2024-09-11T00:32:40.062 INFO:teuthology.orchestra.run.smithi165.stdout: "op_laggy": 0, 2024-09-11T00:32:40.062 INFO:teuthology.orchestra.run.smithi165.stdout: "op_send": 5689, 2024-09-11T00:32:40.062 INFO:teuthology.orchestra.run.smithi165.stdout: "op_send_bytes": 21, 2024-09-11T00:32:40.062 INFO:teuthology.orchestra.run.smithi165.stdout: "op_resend": 3, 2024-09-11T00:32:40.062 INFO:teuthology.orchestra.run.smithi165.stdout: "op_reply": 5689, 2024-09-11T00:32:40.062 INFO:teuthology.orchestra.run.smithi165.stdout: "op_latency": { 2024-09-11T00:32:40.062 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 5689, 2024-09-11T00:32:40.062 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 14.639782561, 2024-09-11T00:32:40.062 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.002573349 2024-09-11T00:32:40.062 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-11T00:32:40.062 INFO:teuthology.orchestra.run.smithi165.stdout: "op_inflight": 0, 2024-09-11T00:32:40.062 INFO:teuthology.orchestra.run.smithi165.stdout: "oplen_avg": { 2024-09-11T00:32:40.062 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 5689, 2024-09-11T00:32:40.063 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 5690 2024-09-11T00:32:40.063 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-11T00:32:40.063 INFO:teuthology.orchestra.run.smithi165.stdout: "op": 5689, 2024-09-11T00:32:40.063 INFO:teuthology.orchestra.run.smithi165.stdout: "op_r": 5689, 2024-09-11T00:32:40.063 INFO:teuthology.orchestra.run.smithi165.stdout: "op_w": 0, 2024-09-11T00:32:40.063 INFO:teuthology.orchestra.run.smithi165.stdout: "op_rmw": 0, 2024-09-11T00:32:40.063 INFO:teuthology.orchestra.run.smithi165.stdout: "op_pg": 0, 2024-09-11T00:32:40.063 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_stat": 3214, 2024-09-11T00:32:40.063 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_create": 0, 2024-09-11T00:32:40.063 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_read": 2473, 2024-09-11T00:32:40.063 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_write": 0, 2024-09-11T00:32:40.063 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_writefull": 0, 2024-09-11T00:32:40.063 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_writesame": 0, 2024-09-11T00:32:40.063 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_append": 0, 2024-09-11T00:32:40.064 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_zero": 0, 2024-09-11T00:32:40.064 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_truncate": 0, 2024-09-11T00:32:40.064 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_delete": 0, 2024-09-11T00:32:40.064 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_mapext": 0, 2024-09-11T00:32:40.064 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_sparse_read": 0, 2024-09-11T00:32:40.064 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_clonerange": 0, 2024-09-11T00:32:40.064 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_getxattr": 1, 2024-09-11T00:32:40.064 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_setxattr": 0, 2024-09-11T00:32:40.064 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_cmpxattr": 0, 2024-09-11T00:32:40.064 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_rmxattr": 0, 2024-09-11T00:32:40.064 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_resetxattrs": 0, 2024-09-11T00:32:40.064 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_call": 0, 2024-09-11T00:32:40.064 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_watch": 0, 2024-09-11T00:32:40.064 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_notify": 0, 2024-09-11T00:32:40.065 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_src_cmpxattr": 0, 2024-09-11T00:32:40.065 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_pgls": 0, 2024-09-11T00:32:40.065 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_pgls_filter": 0, 2024-09-11T00:32:40.065 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_other": 0, 2024-09-11T00:32:40.065 INFO:teuthology.orchestra.run.smithi165.stdout: "linger_active": 0, 2024-09-11T00:32:40.065 INFO:teuthology.orchestra.run.smithi165.stdout: "linger_send": 0, 2024-09-11T00:32:40.065 INFO:teuthology.orchestra.run.smithi165.stdout: "linger_resend": 0, 2024-09-11T00:32:40.065 INFO:teuthology.orchestra.run.smithi165.stdout: "linger_ping": 0, 2024-09-11T00:32:40.065 INFO:teuthology.orchestra.run.smithi165.stdout: "poolop_active": 0, 2024-09-11T00:32:40.065 INFO:teuthology.orchestra.run.smithi165.stdout: "poolop_send": 0, 2024-09-11T00:32:40.065 INFO:teuthology.orchestra.run.smithi165.stdout: "poolop_resend": 0, 2024-09-11T00:32:40.065 INFO:teuthology.orchestra.run.smithi165.stdout: "poolstat_active": 0, 2024-09-11T00:32:40.065 INFO:teuthology.orchestra.run.smithi165.stdout: "poolstat_send": 0, 2024-09-11T00:32:40.066 INFO:teuthology.orchestra.run.smithi165.stdout: "poolstat_resend": 0, 2024-09-11T00:32:40.066 INFO:teuthology.orchestra.run.smithi165.stdout: "statfs_active": 0, 2024-09-11T00:32:40.066 INFO:teuthology.orchestra.run.smithi165.stdout: "statfs_send": 0, 2024-09-11T00:32:40.066 INFO:teuthology.orchestra.run.smithi165.stdout: "statfs_resend": 0, 2024-09-11T00:32:40.066 INFO:teuthology.orchestra.run.smithi165.stdout: "command_active": 0, 2024-09-11T00:32:40.066 INFO:teuthology.orchestra.run.smithi165.stdout: "command_send": 0, 2024-09-11T00:32:40.066 INFO:teuthology.orchestra.run.smithi165.stdout: "command_resend": 0, 2024-09-11T00:32:40.066 INFO:teuthology.orchestra.run.smithi165.stdout: "map_epoch": 87, 2024-09-11T00:32:40.066 INFO:teuthology.orchestra.run.smithi165.stdout: "map_full": 0, 2024-09-11T00:32:40.066 INFO:teuthology.orchestra.run.smithi165.stdout: "map_inc": 6, 2024-09-11T00:32:40.066 INFO:teuthology.orchestra.run.smithi165.stdout: "osd_sessions": 10, 2024-09-11T00:32:40.066 INFO:teuthology.orchestra.run.smithi165.stdout: "osd_session_open": 11, 2024-09-11T00:32:40.066 INFO:teuthology.orchestra.run.smithi165.stdout: "osd_session_close": 1, 2024-09-11T00:32:40.066 INFO:teuthology.orchestra.run.smithi165.stdout: "osd_laggy": 0, 2024-09-11T00:32:40.067 INFO:teuthology.orchestra.run.smithi165.stdout: "omap_wr": 0, 2024-09-11T00:32:40.067 INFO:teuthology.orchestra.run.smithi165.stdout: "omap_rd": 2, 2024-09-11T00:32:40.067 INFO:teuthology.orchestra.run.smithi165.stdout: "omap_del": 0 2024-09-11T00:32:40.067 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-11T00:32:40.067 INFO:teuthology.orchestra.run.smithi165.stdout: "oft": { 2024-09-11T00:32:40.067 INFO:teuthology.orchestra.run.smithi165.stdout: "omap_total_objs": 0, 2024-09-11T00:32:40.067 INFO:teuthology.orchestra.run.smithi165.stdout: "omap_total_kv_pairs": 0, 2024-09-11T00:32:40.067 INFO:teuthology.orchestra.run.smithi165.stdout: "omap_total_updates": 0, 2024-09-11T00:32:40.067 INFO:teuthology.orchestra.run.smithi165.stdout: "omap_total_removes": 0 2024-09-11T00:32:40.067 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-11T00:32:40.067 INFO:teuthology.orchestra.run.smithi165.stdout: "purge_queue": { 2024-09-11T00:32:40.067 INFO:teuthology.orchestra.run.smithi165.stdout: "pq_executing_ops": 0, 2024-09-11T00:32:40.067 INFO:teuthology.orchestra.run.smithi165.stdout: "pq_executing_ops_high_water": 0, 2024-09-11T00:32:40.067 INFO:teuthology.orchestra.run.smithi165.stdout: "pq_executing": 0, 2024-09-11T00:32:40.068 INFO:teuthology.orchestra.run.smithi165.stdout: "pq_executing_high_water": 0, 2024-09-11T00:32:40.068 INFO:teuthology.orchestra.run.smithi165.stdout: "pq_executed_ops": 0, 2024-09-11T00:32:40.068 INFO:teuthology.orchestra.run.smithi165.stdout: "pq_executed": 0, 2024-09-11T00:32:40.068 INFO:teuthology.orchestra.run.smithi165.stdout: "pq_item_in_journal": 0 2024-09-11T00:32:40.068 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-11T00:32:40.068 INFO:teuthology.orchestra.run.smithi165.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-11T00:32:40.068 INFO:teuthology.orchestra.run.smithi165.stdout: "val": 0, 2024-09-11T00:32:40.068 INFO:teuthology.orchestra.run.smithi165.stdout: "max": 104857600, 2024-09-11T00:32:40.068 INFO:teuthology.orchestra.run.smithi165.stdout: "get_started": 0, 2024-09-11T00:32:40.068 INFO:teuthology.orchestra.run.smithi165.stdout: "get": 6254, 2024-09-11T00:32:40.068 INFO:teuthology.orchestra.run.smithi165.stdout: "get_sum": 182167867, 2024-09-11T00:32:40.068 INFO:teuthology.orchestra.run.smithi165.stdout: "get_or_fail_fail": 0, 2024-09-11T00:32:40.068 INFO:teuthology.orchestra.run.smithi165.stdout: "get_or_fail_success": 6254, 2024-09-11T00:32:40.069 INFO:teuthology.orchestra.run.smithi165.stdout: "take": 0, 2024-09-11T00:32:40.069 INFO:teuthology.orchestra.run.smithi165.stdout: "take_sum": 0, 2024-09-11T00:32:40.069 INFO:teuthology.orchestra.run.smithi165.stdout: "put": 6254, 2024-09-11T00:32:40.069 INFO:teuthology.orchestra.run.smithi165.stdout: "put_sum": 182167867, 2024-09-11T00:32:40.069 INFO:teuthology.orchestra.run.smithi165.stdout: "wait": { 2024-09-11T00:32:40.069 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-09-11T00:32:40.069 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-09-11T00:32:40.069 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-09-11T00:32:40.069 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-11T00:32:40.069 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-11T00:32:40.069 INFO:teuthology.orchestra.run.smithi165.stdout: "throttle-objecter_bytes": { 2024-09-11T00:32:40.070 INFO:teuthology.orchestra.run.smithi165.stdout: "val": 0, 2024-09-11T00:32:40.070 INFO:teuthology.orchestra.run.smithi165.stdout: "max": 104857600, 2024-09-11T00:32:40.070 INFO:teuthology.orchestra.run.smithi165.stdout: "get_started": 0, 2024-09-11T00:32:40.070 INFO:teuthology.orchestra.run.smithi165.stdout: "get": 0, 2024-09-11T00:32:40.070 INFO:teuthology.orchestra.run.smithi165.stdout: "get_sum": 0, 2024-09-11T00:32:40.070 INFO:teuthology.orchestra.run.smithi165.stdout: "get_or_fail_fail": 0, 2024-09-11T00:32:40.070 INFO:teuthology.orchestra.run.smithi165.stdout: "get_or_fail_success": 0, 2024-09-11T00:32:40.070 INFO:teuthology.orchestra.run.smithi165.stdout: "take": 5689, 2024-09-11T00:32:40.070 INFO:teuthology.orchestra.run.smithi165.stdout: "take_sum": 179997080, 2024-09-11T00:32:40.070 INFO:teuthology.orchestra.run.smithi165.stdout: "put": 5689, 2024-09-11T00:32:40.070 INFO:teuthology.orchestra.run.smithi165.stdout: "put_sum": 179997080, 2024-09-11T00:32:40.070 INFO:teuthology.orchestra.run.smithi165.stdout: "wait": { 2024-09-11T00:32:40.070 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-09-11T00:32:40.070 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-09-11T00:32:40.071 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-09-11T00:32:40.071 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-11T00:32:40.071 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-11T00:32:40.071 INFO:teuthology.orchestra.run.smithi165.stdout: "throttle-objecter_ops": { 2024-09-11T00:32:40.071 INFO:teuthology.orchestra.run.smithi165.stdout: "val": 0, 2024-09-11T00:32:40.071 INFO:teuthology.orchestra.run.smithi165.stdout: "max": 1024, 2024-09-11T00:32:40.071 INFO:teuthology.orchestra.run.smithi165.stdout: "get_started": 0, 2024-09-11T00:32:40.071 INFO:teuthology.orchestra.run.smithi165.stdout: "get": 0, 2024-09-11T00:32:40.071 INFO:teuthology.orchestra.run.smithi165.stdout: "get_sum": 0, 2024-09-11T00:32:40.071 INFO:teuthology.orchestra.run.smithi165.stdout: "get_or_fail_fail": 0, 2024-09-11T00:32:40.071 INFO:teuthology.orchestra.run.smithi165.stdout: "get_or_fail_success": 0, 2024-09-11T00:32:40.071 INFO:teuthology.orchestra.run.smithi165.stdout: "take": 5689, 2024-09-11T00:32:40.071 INFO:teuthology.orchestra.run.smithi165.stdout: "take_sum": 5689, 2024-09-11T00:32:40.071 INFO:teuthology.orchestra.run.smithi165.stdout: "put": 5689, 2024-09-11T00:32:40.072 INFO:teuthology.orchestra.run.smithi165.stdout: "put_sum": 5689, 2024-09-11T00:32:40.072 INFO:teuthology.orchestra.run.smithi165.stdout: "wait": { 2024-09-11T00:32:40.072 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-09-11T00:32:40.072 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-09-11T00:32:40.072 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-09-11T00:32:40.072 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-11T00:32:40.072 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-11T00:32:40.072 INFO:teuthology.orchestra.run.smithi165.stdout: "throttle-write_buf_throttle": { 2024-09-11T00:32:40.072 INFO:teuthology.orchestra.run.smithi165.stdout: "val": 0, 2024-09-11T00:32:40.072 INFO:teuthology.orchestra.run.smithi165.stdout: "max": 3758096384, 2024-09-11T00:32:40.072 INFO:teuthology.orchestra.run.smithi165.stdout: "get_started": 0, 2024-09-11T00:32:40.072 INFO:teuthology.orchestra.run.smithi165.stdout: "get": 0, 2024-09-11T00:32:40.072 INFO:teuthology.orchestra.run.smithi165.stdout: "get_sum": 0, 2024-09-11T00:32:40.072 INFO:teuthology.orchestra.run.smithi165.stdout: "get_or_fail_fail": 0, 2024-09-11T00:32:40.073 INFO:teuthology.orchestra.run.smithi165.stdout: "get_or_fail_success": 0, 2024-09-11T00:32:40.073 INFO:teuthology.orchestra.run.smithi165.stdout: "take": 0, 2024-09-11T00:32:40.073 INFO:teuthology.orchestra.run.smithi165.stdout: "take_sum": 0, 2024-09-11T00:32:40.073 INFO:teuthology.orchestra.run.smithi165.stdout: "put": 0, 2024-09-11T00:32:40.073 INFO:teuthology.orchestra.run.smithi165.stdout: "put_sum": 0, 2024-09-11T00:32:40.073 INFO:teuthology.orchestra.run.smithi165.stdout: "wait": { 2024-09-11T00:32:40.073 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-09-11T00:32:40.073 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-09-11T00:32:40.073 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-09-11T00:32:40.073 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-11T00:32:40.073 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-11T00:32:40.073 INFO:teuthology.orchestra.run.smithi165.stdout: "throttle-write_buf_throttle-0x5602b530d040": { 2024-09-11T00:32:40.073 INFO:teuthology.orchestra.run.smithi165.stdout: "val": 0, 2024-09-11T00:32:40.073 INFO:teuthology.orchestra.run.smithi165.stdout: "max": 3758096384, 2024-09-11T00:32:40.074 INFO:teuthology.orchestra.run.smithi165.stdout: "get_started": 0, 2024-09-11T00:32:40.074 INFO:teuthology.orchestra.run.smithi165.stdout: "get": 0, 2024-09-11T00:32:40.074 INFO:teuthology.orchestra.run.smithi165.stdout: "get_sum": 0, 2024-09-11T00:32:40.074 INFO:teuthology.orchestra.run.smithi165.stdout: "get_or_fail_fail": 0, 2024-09-11T00:32:40.074 INFO:teuthology.orchestra.run.smithi165.stdout: "get_or_fail_success": 0, 2024-09-11T00:32:40.074 INFO:teuthology.orchestra.run.smithi165.stdout: "take": 0, 2024-09-11T00:32:40.074 INFO:teuthology.orchestra.run.smithi165.stdout: "take_sum": 0, 2024-09-11T00:32:40.074 INFO:teuthology.orchestra.run.smithi165.stdout: "put": 0, 2024-09-11T00:32:40.074 INFO:teuthology.orchestra.run.smithi165.stdout: "put_sum": 0, 2024-09-11T00:32:40.074 INFO:teuthology.orchestra.run.smithi165.stdout: "wait": { 2024-09-11T00:32:40.074 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-09-11T00:32:40.074 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-09-11T00:32:40.074 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-09-11T00:32:40.074 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-11T00:32:40.075 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-11T00:32:40.075 INFO:teuthology.orchestra.run.smithi165.stdout: "trackedop": { 2024-09-11T00:32:40.075 INFO:teuthology.orchestra.run.smithi165.stdout: "slow_ops_count": 0 2024-09-11T00:32:40.075 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-11T00:32:40.075 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-09-11T00:32:40.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:32:40 smithi165 ceph-mon[35356]: pgmap v1167: 129 pgs: 129 active+clean; 671 MiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 767 B/s wr, 9 op/s 2024-09-11T00:32:40.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:32:40 smithi012 ceph-mon[30333]: pgmap v1167: 129 pgs: 129 active+clean; 671 MiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 767 B/s wr, 9 op/s 2024-09-11T00:32:40.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:32:40 smithi178 ceph-mon[35919]: pgmap v1167: 129 pgs: 129 active+clean; 671 MiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 767 B/s wr, 9 op/s 2024-09-11T00:32:40.881 INFO:tasks.check_counter:Daemon mds.k mds.exported=0 2024-09-11T00:32:40.881 INFO:tasks.check_counter:Daemon mds.k mds.imported=0 2024-09-11T00:32:40.881 DEBUG:tasks.check_counter:Getting stats from c 2024-09-11T00:32:40.882 DEBUG:teuthology.orchestra.run.smithi178:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid ff3ad74a-6fcf-11ef-bcea-c7b262605968 -- ceph daemon mds.c perf dump 2024-09-11T00:32:41.208 INFO:teuthology.orchestra.run.smithi178.stderr:Inferring config /var/lib/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/mon.c/config 2024-09-11T00:32:41.828 INFO:teuthology.orchestra.run.smithi178.stdout:{ 2024-09-11T00:32:41.829 INFO:teuthology.orchestra.run.smithi178.stdout: "AsyncMessenger::Worker-0": { 2024-09-11T00:32:41.829 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_recv_messages": 595, 2024-09-11T00:32:41.829 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_messages": 570, 2024-09-11T00:32:41.829 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_recv_bytes": 13086384, 2024-09-11T00:32:41.829 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_bytes": 202456, 2024-09-11T00:32:41.829 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_created_connections": 6, 2024-09-11T00:32:41.829 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_active_connections": 2, 2024-09-11T00:32:41.829 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_total_time": 0.073179481, 2024-09-11T00:32:41.829 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_send_time": 0.025634782, 2024-09-11T00:32:41.829 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_recv_time": 0.032332903, 2024-09-11T00:32:41.829 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_fast_dispatch_time": 0.014583923, 2024-09-11T00:32:41.829 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_messages_queue_lat": { 2024-09-11T00:32:41.830 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 569, 2024-09-11T00:32:41.830 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.027930007, 2024-09-11T00:32:41.830 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000049086 2024-09-11T00:32:41.830 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T00:32:41.830 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_handle_ack_lat": { 2024-09-11T00:32:41.830 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-11T00:32:41.830 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-11T00:32:41.830 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-11T00:32:41.830 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T00:32:41.830 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_recv_encrypted_bytes": 357312, 2024-09-11T00:32:41.830 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_encrypted_bytes": 47904 2024-09-11T00:32:41.830 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T00:32:41.830 INFO:teuthology.orchestra.run.smithi178.stdout: "AsyncMessenger::Worker-1": { 2024-09-11T00:32:41.831 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_recv_messages": 331, 2024-09-11T00:32:41.831 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_messages": 387, 2024-09-11T00:32:41.831 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_recv_bytes": 21559302, 2024-09-11T00:32:41.831 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_bytes": 296685, 2024-09-11T00:32:41.831 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_created_connections": 6, 2024-09-11T00:32:41.831 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_active_connections": 5, 2024-09-11T00:32:41.831 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_total_time": 0.048288318, 2024-09-11T00:32:41.831 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_send_time": 0.015964263, 2024-09-11T00:32:41.831 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_recv_time": 0.022259637, 2024-09-11T00:32:41.831 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_fast_dispatch_time": 0.007669828, 2024-09-11T00:32:41.831 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_messages_queue_lat": { 2024-09-11T00:32:41.831 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 386, 2024-09-11T00:32:41.831 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.025768734, 2024-09-11T00:32:41.831 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000066758 2024-09-11T00:32:41.832 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T00:32:41.832 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_handle_ack_lat": { 2024-09-11T00:32:41.832 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-11T00:32:41.832 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-11T00:32:41.832 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-11T00:32:41.832 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T00:32:41.832 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_recv_encrypted_bytes": 3104, 2024-09-11T00:32:41.832 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_encrypted_bytes": 192960 2024-09-11T00:32:41.832 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T00:32:41.832 INFO:teuthology.orchestra.run.smithi178.stdout: "AsyncMessenger::Worker-2": { 2024-09-11T00:32:41.832 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_recv_messages": 228, 2024-09-11T00:32:41.832 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_messages": 230, 2024-09-11T00:32:41.832 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_recv_bytes": 21030156, 2024-09-11T00:32:41.832 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_bytes": 72520, 2024-09-11T00:32:41.833 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_created_connections": 6, 2024-09-11T00:32:41.833 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_active_connections": 5, 2024-09-11T00:32:41.833 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_total_time": 0.033757429, 2024-09-11T00:32:41.833 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_send_time": 0.009935622, 2024-09-11T00:32:41.833 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_recv_time": 0.016383691, 2024-09-11T00:32:41.833 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_fast_dispatch_time": 0.005594540, 2024-09-11T00:32:41.833 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_messages_queue_lat": { 2024-09-11T00:32:41.833 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 228, 2024-09-11T00:32:41.833 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.029313439, 2024-09-11T00:32:41.833 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000128567 2024-09-11T00:32:41.833 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T00:32:41.833 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_handle_ack_lat": { 2024-09-11T00:32:41.833 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-11T00:32:41.834 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-11T00:32:41.834 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-11T00:32:41.834 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T00:32:41.834 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-11T00:32:41.834 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_encrypted_bytes": 0 2024-09-11T00:32:41.834 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T00:32:41.834 INFO:teuthology.orchestra.run.smithi178.stdout: "cct": { 2024-09-11T00:32:41.834 INFO:teuthology.orchestra.run.smithi178.stdout: "total_workers": 1, 2024-09-11T00:32:41.834 INFO:teuthology.orchestra.run.smithi178.stdout: "unhealthy_workers": 0 2024-09-11T00:32:41.834 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T00:32:41.834 INFO:teuthology.orchestra.run.smithi178.stdout: "finisher-MDSRank": { 2024-09-11T00:32:41.834 INFO:teuthology.orchestra.run.smithi178.stdout: "queue_len": 0, 2024-09-11T00:32:41.834 INFO:teuthology.orchestra.run.smithi178.stdout: "complete_latency": { 2024-09-11T00:32:41.834 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 1770, 2024-09-11T00:32:41.835 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.085135745, 2024-09-11T00:32:41.835 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000048099 2024-09-11T00:32:41.835 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-11T00:32:41.835 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T00:32:41.835 INFO:teuthology.orchestra.run.smithi178.stdout: "finisher-PurgeQueue": { 2024-09-11T00:32:41.835 INFO:teuthology.orchestra.run.smithi178.stdout: "queue_len": 0, 2024-09-11T00:32:41.835 INFO:teuthology.orchestra.run.smithi178.stdout: "complete_latency": { 2024-09-11T00:32:41.835 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-11T00:32:41.835 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-11T00:32:41.835 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-11T00:32:41.835 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-11T00:32:41.835 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T00:32:41.835 INFO:teuthology.orchestra.run.smithi178.stdout: "mds": { 2024-09-11T00:32:41.836 INFO:teuthology.orchestra.run.smithi178.stdout: "request": 0, 2024-09-11T00:32:41.836 INFO:teuthology.orchestra.run.smithi178.stdout: "reply": 0, 2024-09-11T00:32:41.836 INFO:teuthology.orchestra.run.smithi178.stdout: "reply_latency": { 2024-09-11T00:32:41.836 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-11T00:32:41.836 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-11T00:32:41.836 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-11T00:32:41.836 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T00:32:41.836 INFO:teuthology.orchestra.run.smithi178.stdout: "slow_reply": 0, 2024-09-11T00:32:41.836 INFO:teuthology.orchestra.run.smithi178.stdout: "forward": 0, 2024-09-11T00:32:41.836 INFO:teuthology.orchestra.run.smithi178.stdout: "dir_fetch_complete": 0, 2024-09-11T00:32:41.836 INFO:teuthology.orchestra.run.smithi178.stdout: "dir_fetch_keys": 0, 2024-09-11T00:32:41.836 INFO:teuthology.orchestra.run.smithi178.stdout: "dir_commit": 0, 2024-09-11T00:32:41.836 INFO:teuthology.orchestra.run.smithi178.stdout: "dir_split": 0, 2024-09-11T00:32:41.837 INFO:teuthology.orchestra.run.smithi178.stdout: "dir_merge": 0, 2024-09-11T00:32:41.837 INFO:teuthology.orchestra.run.smithi178.stdout: "inodes": 5667, 2024-09-11T00:32:41.837 INFO:teuthology.orchestra.run.smithi178.stdout: "inodes_top": 3745, 2024-09-11T00:32:41.837 INFO:teuthology.orchestra.run.smithi178.stdout: "inodes_bottom": 1922, 2024-09-11T00:32:41.837 INFO:teuthology.orchestra.run.smithi178.stdout: "inodes_pin_tail": 0, 2024-09-11T00:32:41.837 INFO:teuthology.orchestra.run.smithi178.stdout: "inodes_pinned": 316, 2024-09-11T00:32:41.837 INFO:teuthology.orchestra.run.smithi178.stdout: "inodes_expired": 0, 2024-09-11T00:32:41.837 INFO:teuthology.orchestra.run.smithi178.stdout: "inodes_with_caps": 0, 2024-09-11T00:32:41.837 INFO:teuthology.orchestra.run.smithi178.stdout: "caps": 0, 2024-09-11T00:32:41.837 INFO:teuthology.orchestra.run.smithi178.stdout: "subtrees": 2, 2024-09-11T00:32:41.837 INFO:teuthology.orchestra.run.smithi178.stdout: "traverse": 0, 2024-09-11T00:32:41.837 INFO:teuthology.orchestra.run.smithi178.stdout: "traverse_hit": 0, 2024-09-11T00:32:41.837 INFO:teuthology.orchestra.run.smithi178.stdout: "traverse_forward": 0, 2024-09-11T00:32:41.838 INFO:teuthology.orchestra.run.smithi178.stdout: "traverse_discover": 0, 2024-09-11T00:32:41.838 INFO:teuthology.orchestra.run.smithi178.stdout: "traverse_dir_fetch": 0, 2024-09-11T00:32:41.838 INFO:teuthology.orchestra.run.smithi178.stdout: "traverse_remote_ino": 0, 2024-09-11T00:32:41.838 INFO:teuthology.orchestra.run.smithi178.stdout: "traverse_lock": 0, 2024-09-11T00:32:41.838 INFO:teuthology.orchestra.run.smithi178.stdout: "load_cent": 0, 2024-09-11T00:32:41.838 INFO:teuthology.orchestra.run.smithi178.stdout: "q": 0, 2024-09-11T00:32:41.838 INFO:teuthology.orchestra.run.smithi178.stdout: "exported": 0, 2024-09-11T00:32:41.838 INFO:teuthology.orchestra.run.smithi178.stdout: "exported_inodes": 0, 2024-09-11T00:32:41.838 INFO:teuthology.orchestra.run.smithi178.stdout: "imported": 0, 2024-09-11T00:32:41.838 INFO:teuthology.orchestra.run.smithi178.stdout: "imported_inodes": 0, 2024-09-11T00:32:41.838 INFO:teuthology.orchestra.run.smithi178.stdout: "openino_dir_fetch": 0, 2024-09-11T00:32:41.839 INFO:teuthology.orchestra.run.smithi178.stdout: "openino_backtrace_fetch": 0, 2024-09-11T00:32:41.839 INFO:teuthology.orchestra.run.smithi178.stdout: "openino_peer_discover": 0, 2024-09-11T00:32:41.839 INFO:teuthology.orchestra.run.smithi178.stdout: "root_rfiles": 4, 2024-09-11T00:32:41.839 INFO:teuthology.orchestra.run.smithi178.stdout: "root_rbytes": 234, 2024-09-11T00:32:41.839 INFO:teuthology.orchestra.run.smithi178.stdout: "root_rsnaps": 0, 2024-09-11T00:32:41.839 INFO:teuthology.orchestra.run.smithi178.stdout: "scrub_backtrace_fetch": 0, 2024-09-11T00:32:41.839 INFO:teuthology.orchestra.run.smithi178.stdout: "scrub_set_tag": 0, 2024-09-11T00:32:41.839 INFO:teuthology.orchestra.run.smithi178.stdout: "scrub_backtrace_repaired": 0, 2024-09-11T00:32:41.839 INFO:teuthology.orchestra.run.smithi178.stdout: "scrub_inotable_repaired": 0, 2024-09-11T00:32:41.839 INFO:teuthology.orchestra.run.smithi178.stdout: "scrub_dir_inodes": 0, 2024-09-11T00:32:41.839 INFO:teuthology.orchestra.run.smithi178.stdout: "scrub_dir_base_inodes": 0, 2024-09-11T00:32:41.840 INFO:teuthology.orchestra.run.smithi178.stdout: "scrub_dirfrag_rstats": 0, 2024-09-11T00:32:41.840 INFO:teuthology.orchestra.run.smithi178.stdout: "scrub_file_inodes": 0, 2024-09-11T00:32:41.840 INFO:teuthology.orchestra.run.smithi178.stdout: "handle_inode_file_caps": 0, 2024-09-11T00:32:41.840 INFO:teuthology.orchestra.run.smithi178.stdout: "ceph_cap_op_revoke": 0, 2024-09-11T00:32:41.840 INFO:teuthology.orchestra.run.smithi178.stdout: "ceph_cap_op_grant": 0, 2024-09-11T00:32:41.840 INFO:teuthology.orchestra.run.smithi178.stdout: "ceph_cap_op_trunc": 0, 2024-09-11T00:32:41.840 INFO:teuthology.orchestra.run.smithi178.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-11T00:32:41.840 INFO:teuthology.orchestra.run.smithi178.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-11T00:32:41.840 INFO:teuthology.orchestra.run.smithi178.stdout: "handle_client_caps": 0, 2024-09-11T00:32:41.840 INFO:teuthology.orchestra.run.smithi178.stdout: "handle_client_caps_dirty": 0, 2024-09-11T00:32:41.840 INFO:teuthology.orchestra.run.smithi178.stdout: "handle_client_cap_release": 0, 2024-09-11T00:32:41.840 INFO:teuthology.orchestra.run.smithi178.stdout: "process_request_cap_release": 0 2024-09-11T00:32:41.841 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T00:32:41.841 INFO:teuthology.orchestra.run.smithi178.stdout: "mds_cache": { 2024-09-11T00:32:41.841 INFO:teuthology.orchestra.run.smithi178.stdout: "dir_update": 0, 2024-09-11T00:32:41.841 INFO:teuthology.orchestra.run.smithi178.stdout: "dir_update_receipt": 0, 2024-09-11T00:32:41.841 INFO:teuthology.orchestra.run.smithi178.stdout: "dir_try_discover": 0, 2024-09-11T00:32:41.841 INFO:teuthology.orchestra.run.smithi178.stdout: "dir_send_discover": 0, 2024-09-11T00:32:41.842 INFO:teuthology.orchestra.run.smithi178.stdout: "dir_handle_discover": 0, 2024-09-11T00:32:41.842 INFO:teuthology.orchestra.run.smithi178.stdout: "num_strays": 0, 2024-09-11T00:32:41.842 INFO:teuthology.orchestra.run.smithi178.stdout: "num_strays_delayed": 0, 2024-09-11T00:32:41.842 INFO:teuthology.orchestra.run.smithi178.stdout: "num_strays_enqueuing": 0, 2024-09-11T00:32:41.842 INFO:teuthology.orchestra.run.smithi178.stdout: "strays_created": 0, 2024-09-11T00:32:41.842 INFO:teuthology.orchestra.run.smithi178.stdout: "strays_enqueued": 0, 2024-09-11T00:32:41.843 INFO:teuthology.orchestra.run.smithi178.stdout: "strays_reintegrated": 0, 2024-09-11T00:32:41.843 INFO:teuthology.orchestra.run.smithi178.stdout: "strays_migrated": 0, 2024-09-11T00:32:41.843 INFO:teuthology.orchestra.run.smithi178.stdout: "num_recovering_processing": 0, 2024-09-11T00:32:41.843 INFO:teuthology.orchestra.run.smithi178.stdout: "num_recovering_enqueued": 0, 2024-09-11T00:32:41.843 INFO:teuthology.orchestra.run.smithi178.stdout: "num_recovering_prioritized": 0, 2024-09-11T00:32:41.843 INFO:teuthology.orchestra.run.smithi178.stdout: "recovery_started": 0, 2024-09-11T00:32:41.843 INFO:teuthology.orchestra.run.smithi178.stdout: "recovery_completed": 0, 2024-09-11T00:32:41.844 INFO:teuthology.orchestra.run.smithi178.stdout: "ireq_quiesce_path": 0, 2024-09-11T00:32:41.844 INFO:teuthology.orchestra.run.smithi178.stdout: "ireq_quiesce_inode": 0, 2024-09-11T00:32:41.844 INFO:teuthology.orchestra.run.smithi178.stdout: "ireq_enqueue_scrub": 0, 2024-09-11T00:32:41.844 INFO:teuthology.orchestra.run.smithi178.stdout: "ireq_exportdir": 0, 2024-09-11T00:32:41.844 INFO:teuthology.orchestra.run.smithi178.stdout: "ireq_flush": 0, 2024-09-11T00:32:41.844 INFO:teuthology.orchestra.run.smithi178.stdout: "ireq_fragmentdir": 0, 2024-09-11T00:32:41.845 INFO:teuthology.orchestra.run.smithi178.stdout: "ireq_fragstats": 0, 2024-09-11T00:32:41.845 INFO:teuthology.orchestra.run.smithi178.stdout: "ireq_inodestats": 0, 2024-09-11T00:32:41.845 INFO:teuthology.orchestra.run.smithi178.stdout: "uninline_started": 0, 2024-09-11T00:32:41.845 INFO:teuthology.orchestra.run.smithi178.stdout: "uninline_succeeded": 0, 2024-09-11T00:32:41.845 INFO:teuthology.orchestra.run.smithi178.stdout: "uninline_write_failed": 0 2024-09-11T00:32:41.845 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T00:32:41.846 INFO:teuthology.orchestra.run.smithi178.stdout: "mds_log": { 2024-09-11T00:32:41.846 INFO:teuthology.orchestra.run.smithi178.stdout: "evlrg": 0, 2024-09-11T00:32:41.846 INFO:teuthology.orchestra.run.smithi178.stdout: "evadd": 0, 2024-09-11T00:32:41.846 INFO:teuthology.orchestra.run.smithi178.stdout: "evex": 0, 2024-09-11T00:32:41.846 INFO:teuthology.orchestra.run.smithi178.stdout: "evtrm": 0, 2024-09-11T00:32:41.846 INFO:teuthology.orchestra.run.smithi178.stdout: "ev": 28999, 2024-09-11T00:32:41.847 INFO:teuthology.orchestra.run.smithi178.stdout: "evexg": 0, 2024-09-11T00:32:41.847 INFO:teuthology.orchestra.run.smithi178.stdout: "evexd": 0, 2024-09-11T00:32:41.847 INFO:teuthology.orchestra.run.smithi178.stdout: "segadd": 0, 2024-09-11T00:32:41.847 INFO:teuthology.orchestra.run.smithi178.stdout: "segex": 0, 2024-09-11T00:32:41.847 INFO:teuthology.orchestra.run.smithi178.stdout: "segtrm": 0, 2024-09-11T00:32:41.847 INFO:teuthology.orchestra.run.smithi178.stdout: "seg": 2, 2024-09-11T00:32:41.848 INFO:teuthology.orchestra.run.smithi178.stdout: "segmjr": 19, 2024-09-11T00:32:41.848 INFO:teuthology.orchestra.run.smithi178.stdout: "segexg": 0, 2024-09-11T00:32:41.848 INFO:teuthology.orchestra.run.smithi178.stdout: "segexd": 0, 2024-09-11T00:32:41.848 INFO:teuthology.orchestra.run.smithi178.stdout: "expos": 255441103, 2024-09-11T00:32:41.848 INFO:teuthology.orchestra.run.smithi178.stdout: "wrpos": 256507080, 2024-09-11T00:32:41.848 INFO:teuthology.orchestra.run.smithi178.stdout: "rdpos": 256505093, 2024-09-11T00:32:41.849 INFO:teuthology.orchestra.run.smithi178.stdout: "jlat": { 2024-09-11T00:32:41.849 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-11T00:32:41.849 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-11T00:32:41.849 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-11T00:32:41.849 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T00:32:41.849 INFO:teuthology.orchestra.run.smithi178.stdout: "replayed": 28999 2024-09-11T00:32:41.850 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T00:32:41.850 INFO:teuthology.orchestra.run.smithi178.stdout: "mds_mem": { 2024-09-11T00:32:41.850 INFO:teuthology.orchestra.run.smithi178.stdout: "ino": 24, 2024-09-11T00:32:41.850 INFO:teuthology.orchestra.run.smithi178.stdout: "ino+": 5555, 2024-09-11T00:32:41.850 INFO:teuthology.orchestra.run.smithi178.stdout: "ino-": 5531, 2024-09-11T00:32:41.850 INFO:teuthology.orchestra.run.smithi178.stdout: "dir": 23, 2024-09-11T00:32:41.851 INFO:teuthology.orchestra.run.smithi178.stdout: "dir+": 67, 2024-09-11T00:32:41.851 INFO:teuthology.orchestra.run.smithi178.stdout: "dir-": 44, 2024-09-11T00:32:41.851 INFO:teuthology.orchestra.run.smithi178.stdout: "dn": 5667, 2024-09-11T00:32:41.851 INFO:teuthology.orchestra.run.smithi178.stdout: "dn+": 5991, 2024-09-11T00:32:41.851 INFO:teuthology.orchestra.run.smithi178.stdout: "dn-": 324, 2024-09-11T00:32:41.851 INFO:teuthology.orchestra.run.smithi178.stdout: "cap": 0, 2024-09-11T00:32:41.852 INFO:teuthology.orchestra.run.smithi178.stdout: "cap+": 0, 2024-09-11T00:32:41.852 INFO:teuthology.orchestra.run.smithi178.stdout: "cap-": 0, 2024-09-11T00:32:41.852 INFO:teuthology.orchestra.run.smithi178.stdout: "rss": 54616, 2024-09-11T00:32:41.852 INFO:teuthology.orchestra.run.smithi178.stdout: "heap": 258863104 2024-09-11T00:32:41.852 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T00:32:41.852 INFO:teuthology.orchestra.run.smithi178.stdout: "mds_server": { 2024-09-11T00:32:41.853 INFO:teuthology.orchestra.run.smithi178.stdout: "dispatch_client_request": 0, 2024-09-11T00:32:41.853 INFO:teuthology.orchestra.run.smithi178.stdout: "dispatch_server_request": 0, 2024-09-11T00:32:41.853 INFO:teuthology.orchestra.run.smithi178.stdout: "handle_client_request": 0, 2024-09-11T00:32:41.853 INFO:teuthology.orchestra.run.smithi178.stdout: "handle_client_session": 0, 2024-09-11T00:32:41.853 INFO:teuthology.orchestra.run.smithi178.stdout: "handle_peer_request": 0, 2024-09-11T00:32:41.853 INFO:teuthology.orchestra.run.smithi178.stdout: "req_create_latency": { 2024-09-11T00:32:41.854 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-11T00:32:41.854 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-11T00:32:41.854 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-11T00:32:41.854 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T00:32:41.854 INFO:teuthology.orchestra.run.smithi178.stdout: "req_getattr_latency": { 2024-09-11T00:32:41.854 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-11T00:32:41.855 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-11T00:32:41.855 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-11T00:32:41.855 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T00:32:41.855 INFO:teuthology.orchestra.run.smithi178.stdout: "req_getfilelock_latency": { 2024-09-11T00:32:41.855 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-11T00:32:41.855 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-11T00:32:41.856 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-11T00:32:41.856 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T00:32:41.856 INFO:teuthology.orchestra.run.smithi178.stdout: "req_link_latency": { 2024-09-11T00:32:41.856 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-11T00:32:41.856 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-11T00:32:41.856 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-11T00:32:41.856 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T00:32:41.857 INFO:teuthology.orchestra.run.smithi178.stdout: "req_lookup_latency": { 2024-09-11T00:32:41.857 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-11T00:32:41.857 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-11T00:32:41.857 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-11T00:32:41.857 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T00:32:41.857 INFO:teuthology.orchestra.run.smithi178.stdout: "req_lookuphash_latency": { 2024-09-11T00:32:41.858 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-11T00:32:41.858 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-11T00:32:41.858 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-11T00:32:41.858 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T00:32:41.858 INFO:teuthology.orchestra.run.smithi178.stdout: "req_lookupino_latency": { 2024-09-11T00:32:41.858 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-11T00:32:41.858 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-11T00:32:41.858 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-11T00:32:41.858 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T00:32:41.858 INFO:teuthology.orchestra.run.smithi178.stdout: "req_lookupname_latency": { 2024-09-11T00:32:41.859 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-11T00:32:41.859 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-11T00:32:41.859 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-11T00:32:41.859 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T00:32:41.859 INFO:teuthology.orchestra.run.smithi178.stdout: "req_lookupparent_latency": { 2024-09-11T00:32:41.859 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-11T00:32:41.859 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-11T00:32:41.859 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-11T00:32:41.859 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T00:32:41.859 INFO:teuthology.orchestra.run.smithi178.stdout: "req_lookupsnap_latency": { 2024-09-11T00:32:41.859 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-11T00:32:41.859 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-11T00:32:41.859 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-11T00:32:41.859 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T00:32:41.860 INFO:teuthology.orchestra.run.smithi178.stdout: "req_lssnap_latency": { 2024-09-11T00:32:41.860 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-11T00:32:41.860 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-11T00:32:41.860 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-11T00:32:41.860 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T00:32:41.860 INFO:teuthology.orchestra.run.smithi178.stdout: "req_mkdir_latency": { 2024-09-11T00:32:41.861 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-11T00:32:41.861 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-11T00:32:41.861 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-11T00:32:41.861 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T00:32:41.861 INFO:teuthology.orchestra.run.smithi178.stdout: "req_mknod_latency": { 2024-09-11T00:32:41.861 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-11T00:32:41.861 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-11T00:32:41.861 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-11T00:32:41.861 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T00:32:41.861 INFO:teuthology.orchestra.run.smithi178.stdout: "req_mksnap_latency": { 2024-09-11T00:32:41.861 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-11T00:32:41.861 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-11T00:32:41.861 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-11T00:32:41.862 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T00:32:41.862 INFO:teuthology.orchestra.run.smithi178.stdout: "req_open_latency": { 2024-09-11T00:32:41.862 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-11T00:32:41.862 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-11T00:32:41.862 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-11T00:32:41.862 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T00:32:41.862 INFO:teuthology.orchestra.run.smithi178.stdout: "req_readdir_latency": { 2024-09-11T00:32:41.862 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-11T00:32:41.862 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-11T00:32:41.862 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-11T00:32:41.862 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T00:32:41.862 INFO:teuthology.orchestra.run.smithi178.stdout: "req_rename_latency": { 2024-09-11T00:32:41.862 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-11T00:32:41.862 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-11T00:32:41.863 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-11T00:32:41.863 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T00:32:41.863 INFO:teuthology.orchestra.run.smithi178.stdout: "req_renamesnap_latency": { 2024-09-11T00:32:41.863 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-11T00:32:41.863 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-11T00:32:41.863 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-11T00:32:41.863 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T00:32:41.863 INFO:teuthology.orchestra.run.smithi178.stdout: "req_snapdiff_latency": { 2024-09-11T00:32:41.863 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-11T00:32:41.863 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-11T00:32:41.863 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-11T00:32:41.863 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T00:32:41.863 INFO:teuthology.orchestra.run.smithi178.stdout: "req_rmdir_latency": { 2024-09-11T00:32:41.863 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-11T00:32:41.864 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-11T00:32:41.864 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-11T00:32:41.864 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T00:32:41.864 INFO:teuthology.orchestra.run.smithi178.stdout: "req_rmsnap_latency": { 2024-09-11T00:32:41.864 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-11T00:32:41.864 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-11T00:32:41.864 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-11T00:32:41.864 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T00:32:41.864 INFO:teuthology.orchestra.run.smithi178.stdout: "req_rmxattr_latency": { 2024-09-11T00:32:41.864 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-11T00:32:41.865 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-11T00:32:41.865 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-11T00:32:41.865 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T00:32:41.865 INFO:teuthology.orchestra.run.smithi178.stdout: "req_setattr_latency": { 2024-09-11T00:32:41.865 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-11T00:32:41.865 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-11T00:32:41.865 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-11T00:32:41.865 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T00:32:41.865 INFO:teuthology.orchestra.run.smithi178.stdout: "req_setdirlayout_latency": { 2024-09-11T00:32:41.865 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-11T00:32:41.865 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-11T00:32:41.866 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-11T00:32:41.866 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T00:32:41.866 INFO:teuthology.orchestra.run.smithi178.stdout: "req_setfilelock_latency": { 2024-09-11T00:32:41.866 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-11T00:32:41.866 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-11T00:32:41.866 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-11T00:32:41.866 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T00:32:41.866 INFO:teuthology.orchestra.run.smithi178.stdout: "req_setlayout_latency": { 2024-09-11T00:32:41.866 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-11T00:32:41.866 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-11T00:32:41.866 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-11T00:32:41.866 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T00:32:41.866 INFO:teuthology.orchestra.run.smithi178.stdout: "req_setxattr_latency": { 2024-09-11T00:32:41.867 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-11T00:32:41.867 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-11T00:32:41.867 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-11T00:32:41.867 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T00:32:41.867 INFO:teuthology.orchestra.run.smithi178.stdout: "req_symlink_latency": { 2024-09-11T00:32:41.867 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-11T00:32:41.867 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-11T00:32:41.867 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-11T00:32:41.867 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T00:32:41.867 INFO:teuthology.orchestra.run.smithi178.stdout: "req_unlink_latency": { 2024-09-11T00:32:41.867 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-11T00:32:41.867 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-11T00:32:41.867 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-11T00:32:41.868 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T00:32:41.868 INFO:teuthology.orchestra.run.smithi178.stdout: "cap_revoke_eviction": 0, 2024-09-11T00:32:41.868 INFO:teuthology.orchestra.run.smithi178.stdout: "cap_acquisition_throttle": 0, 2024-09-11T00:32:41.868 INFO:teuthology.orchestra.run.smithi178.stdout: "req_getvxattr_latency": { 2024-09-11T00:32:41.868 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-11T00:32:41.868 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-11T00:32:41.868 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-11T00:32:41.868 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-11T00:32:41.868 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T00:32:41.868 INFO:teuthology.orchestra.run.smithi178.stdout: "mds_sessions": { 2024-09-11T00:32:41.868 INFO:teuthology.orchestra.run.smithi178.stdout: "session_count": 1, 2024-09-11T00:32:41.869 INFO:teuthology.orchestra.run.smithi178.stdout: "session_add": 1, 2024-09-11T00:32:41.869 INFO:teuthology.orchestra.run.smithi178.stdout: "session_remove": 0, 2024-09-11T00:32:41.869 INFO:teuthology.orchestra.run.smithi178.stdout: "sessions_open": 0, 2024-09-11T00:32:41.869 INFO:teuthology.orchestra.run.smithi178.stdout: "sessions_stale": 0, 2024-09-11T00:32:41.869 INFO:teuthology.orchestra.run.smithi178.stdout: "total_load": 0, 2024-09-11T00:32:41.869 INFO:teuthology.orchestra.run.smithi178.stdout: "average_load": 0, 2024-09-11T00:32:41.869 INFO:teuthology.orchestra.run.smithi178.stdout: "avg_session_uptime": 2946, 2024-09-11T00:32:41.869 INFO:teuthology.orchestra.run.smithi178.stdout: "mdthresh_evicted": 0 2024-09-11T00:32:41.869 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T00:32:41.869 INFO:teuthology.orchestra.run.smithi178.stdout: "mempool": { 2024-09-11T00:32:41.869 INFO:teuthology.orchestra.run.smithi178.stdout: "bloom_filter_bytes": 0, 2024-09-11T00:32:41.869 INFO:teuthology.orchestra.run.smithi178.stdout: "bloom_filter_items": 0, 2024-09-11T00:32:41.869 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_alloc_bytes": 0, 2024-09-11T00:32:41.869 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_alloc_items": 0, 2024-09-11T00:32:41.870 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_cache_data_bytes": 0, 2024-09-11T00:32:41.870 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_cache_data_items": 0, 2024-09-11T00:32:41.870 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-11T00:32:41.870 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_cache_onode_items": 0, 2024-09-11T00:32:41.870 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-11T00:32:41.870 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_cache_meta_items": 0, 2024-09-11T00:32:41.870 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_cache_other_bytes": 0, 2024-09-11T00:32:41.870 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_cache_other_items": 0, 2024-09-11T00:32:41.870 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-11T00:32:41.870 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_cache_buffer_items": 0, 2024-09-11T00:32:41.870 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_extent_bytes": 0, 2024-09-11T00:32:41.870 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_extent_items": 0, 2024-09-11T00:32:41.870 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_blob_bytes": 0, 2024-09-11T00:32:41.871 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_blob_items": 0, 2024-09-11T00:32:41.871 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-11T00:32:41.871 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_shared_blob_items": 0, 2024-09-11T00:32:41.871 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-11T00:32:41.871 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_inline_bl_items": 0, 2024-09-11T00:32:41.871 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_fsck_bytes": 0, 2024-09-11T00:32:41.871 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_fsck_items": 0, 2024-09-11T00:32:41.871 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_txc_bytes": 0, 2024-09-11T00:32:41.871 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_txc_items": 0, 2024-09-11T00:32:41.871 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-11T00:32:41.871 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_writing_deferred_items": 0, 2024-09-11T00:32:41.871 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_writing_bytes": 0, 2024-09-11T00:32:41.871 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_writing_items": 0, 2024-09-11T00:32:41.871 INFO:teuthology.orchestra.run.smithi178.stdout: "bluefs_bytes": 0, 2024-09-11T00:32:41.871 INFO:teuthology.orchestra.run.smithi178.stdout: "bluefs_items": 0, 2024-09-11T00:32:41.872 INFO:teuthology.orchestra.run.smithi178.stdout: "bluefs_file_reader_bytes": 0, 2024-09-11T00:32:41.872 INFO:teuthology.orchestra.run.smithi178.stdout: "bluefs_file_reader_items": 0, 2024-09-11T00:32:41.872 INFO:teuthology.orchestra.run.smithi178.stdout: "bluefs_file_writer_bytes": 0, 2024-09-11T00:32:41.872 INFO:teuthology.orchestra.run.smithi178.stdout: "bluefs_file_writer_items": 0, 2024-09-11T00:32:41.872 INFO:teuthology.orchestra.run.smithi178.stdout: "buffer_anon_bytes": 222827, 2024-09-11T00:32:41.872 INFO:teuthology.orchestra.run.smithi178.stdout: "buffer_anon_items": 72, 2024-09-11T00:32:41.872 INFO:teuthology.orchestra.run.smithi178.stdout: "buffer_meta_bytes": 0, 2024-09-11T00:32:41.872 INFO:teuthology.orchestra.run.smithi178.stdout: "buffer_meta_items": 0, 2024-09-11T00:32:41.872 INFO:teuthology.orchestra.run.smithi178.stdout: "osd_bytes": 0, 2024-09-11T00:32:41.872 INFO:teuthology.orchestra.run.smithi178.stdout: "osd_items": 0, 2024-09-11T00:32:41.872 INFO:teuthology.orchestra.run.smithi178.stdout: "osd_mapbl_bytes": 0, 2024-09-11T00:32:41.872 INFO:teuthology.orchestra.run.smithi178.stdout: "osd_mapbl_items": 0, 2024-09-11T00:32:41.872 INFO:teuthology.orchestra.run.smithi178.stdout: "osd_pglog_bytes": 0, 2024-09-11T00:32:41.873 INFO:teuthology.orchestra.run.smithi178.stdout: "osd_pglog_items": 0, 2024-09-11T00:32:41.873 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap_bytes": 9048, 2024-09-11T00:32:41.873 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap_items": 170, 2024-09-11T00:32:41.873 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap_mapping_bytes": 0, 2024-09-11T00:32:41.873 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap_mapping_items": 0, 2024-09-11T00:32:41.873 INFO:teuthology.orchestra.run.smithi178.stdout: "pgmap_bytes": 0, 2024-09-11T00:32:41.873 INFO:teuthology.orchestra.run.smithi178.stdout: "pgmap_items": 0, 2024-09-11T00:32:41.873 INFO:teuthology.orchestra.run.smithi178.stdout: "mds_co_bytes": 3939300, 2024-09-11T00:32:41.873 INFO:teuthology.orchestra.run.smithi178.stdout: "mds_co_items": 23592, 2024-09-11T00:32:41.873 INFO:teuthology.orchestra.run.smithi178.stdout: "unittest_1_bytes": 0, 2024-09-11T00:32:41.873 INFO:teuthology.orchestra.run.smithi178.stdout: "unittest_1_items": 0, 2024-09-11T00:32:41.873 INFO:teuthology.orchestra.run.smithi178.stdout: "unittest_2_bytes": 0, 2024-09-11T00:32:41.873 INFO:teuthology.orchestra.run.smithi178.stdout: "unittest_2_items": 0 2024-09-11T00:32:41.874 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T00:32:41.874 INFO:teuthology.orchestra.run.smithi178.stdout: "objecter": { 2024-09-11T00:32:41.874 INFO:teuthology.orchestra.run.smithi178.stdout: "op_active": 0, 2024-09-11T00:32:41.874 INFO:teuthology.orchestra.run.smithi178.stdout: "op_laggy": 0, 2024-09-11T00:32:41.874 INFO:teuthology.orchestra.run.smithi178.stdout: "op_send": 1040, 2024-09-11T00:32:41.874 INFO:teuthology.orchestra.run.smithi178.stdout: "op_send_bytes": 26, 2024-09-11T00:32:41.874 INFO:teuthology.orchestra.run.smithi178.stdout: "op_resend": 4, 2024-09-11T00:32:41.874 INFO:teuthology.orchestra.run.smithi178.stdout: "op_reply": 1040, 2024-09-11T00:32:41.874 INFO:teuthology.orchestra.run.smithi178.stdout: "op_latency": { 2024-09-11T00:32:41.874 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 1040, 2024-09-11T00:32:41.874 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 4.104922095, 2024-09-11T00:32:41.874 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.003947040 2024-09-11T00:32:41.874 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T00:32:41.874 INFO:teuthology.orchestra.run.smithi178.stdout: "op_inflight": 0, 2024-09-11T00:32:41.875 INFO:teuthology.orchestra.run.smithi178.stdout: "oplen_avg": { 2024-09-11T00:32:41.875 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 1040, 2024-09-11T00:32:41.875 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 1041 2024-09-11T00:32:41.875 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T00:32:41.875 INFO:teuthology.orchestra.run.smithi178.stdout: "op": 1040, 2024-09-11T00:32:41.875 INFO:teuthology.orchestra.run.smithi178.stdout: "op_r": 1040, 2024-09-11T00:32:41.875 INFO:teuthology.orchestra.run.smithi178.stdout: "op_w": 0, 2024-09-11T00:32:41.875 INFO:teuthology.orchestra.run.smithi178.stdout: "op_rmw": 0, 2024-09-11T00:32:41.875 INFO:teuthology.orchestra.run.smithi178.stdout: "op_pg": 0, 2024-09-11T00:32:41.875 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_stat": 541, 2024-09-11T00:32:41.875 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_create": 0, 2024-09-11T00:32:41.875 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_read": 496, 2024-09-11T00:32:41.875 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_write": 0, 2024-09-11T00:32:41.875 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_writefull": 0, 2024-09-11T00:32:41.876 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_writesame": 0, 2024-09-11T00:32:41.876 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_append": 0, 2024-09-11T00:32:41.876 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_zero": 0, 2024-09-11T00:32:41.876 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_truncate": 0, 2024-09-11T00:32:41.876 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_delete": 0, 2024-09-11T00:32:41.876 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_mapext": 0, 2024-09-11T00:32:41.876 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_sparse_read": 0, 2024-09-11T00:32:41.876 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_clonerange": 0, 2024-09-11T00:32:41.876 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_getxattr": 2, 2024-09-11T00:32:41.876 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_setxattr": 0, 2024-09-11T00:32:41.876 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_cmpxattr": 0, 2024-09-11T00:32:41.876 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_rmxattr": 0, 2024-09-11T00:32:41.876 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_resetxattrs": 0, 2024-09-11T00:32:41.877 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_call": 0, 2024-09-11T00:32:41.877 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_watch": 0, 2024-09-11T00:32:41.877 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_notify": 0, 2024-09-11T00:32:41.877 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_src_cmpxattr": 0, 2024-09-11T00:32:41.877 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_pgls": 0, 2024-09-11T00:32:41.877 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_pgls_filter": 0, 2024-09-11T00:32:41.877 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_other": 0, 2024-09-11T00:32:41.877 INFO:teuthology.orchestra.run.smithi178.stdout: "linger_active": 0, 2024-09-11T00:32:41.877 INFO:teuthology.orchestra.run.smithi178.stdout: "linger_send": 0, 2024-09-11T00:32:41.877 INFO:teuthology.orchestra.run.smithi178.stdout: "linger_resend": 0, 2024-09-11T00:32:41.877 INFO:teuthology.orchestra.run.smithi178.stdout: "linger_ping": 0, 2024-09-11T00:32:41.877 INFO:teuthology.orchestra.run.smithi178.stdout: "poolop_active": 0, 2024-09-11T00:32:41.877 INFO:teuthology.orchestra.run.smithi178.stdout: "poolop_send": 0, 2024-09-11T00:32:41.877 INFO:teuthology.orchestra.run.smithi178.stdout: "poolop_resend": 0, 2024-09-11T00:32:41.878 INFO:teuthology.orchestra.run.smithi178.stdout: "poolstat_active": 0, 2024-09-11T00:32:41.878 INFO:teuthology.orchestra.run.smithi178.stdout: "poolstat_send": 0, 2024-09-11T00:32:41.878 INFO:teuthology.orchestra.run.smithi178.stdout: "poolstat_resend": 0, 2024-09-11T00:32:41.878 INFO:teuthology.orchestra.run.smithi178.stdout: "statfs_active": 0, 2024-09-11T00:32:41.878 INFO:teuthology.orchestra.run.smithi178.stdout: "statfs_send": 0, 2024-09-11T00:32:41.878 INFO:teuthology.orchestra.run.smithi178.stdout: "statfs_resend": 0, 2024-09-11T00:32:41.878 INFO:teuthology.orchestra.run.smithi178.stdout: "command_active": 0, 2024-09-11T00:32:41.878 INFO:teuthology.orchestra.run.smithi178.stdout: "command_send": 0, 2024-09-11T00:32:41.878 INFO:teuthology.orchestra.run.smithi178.stdout: "command_resend": 0, 2024-09-11T00:32:41.878 INFO:teuthology.orchestra.run.smithi178.stdout: "map_epoch": 0, 2024-09-11T00:32:41.878 INFO:teuthology.orchestra.run.smithi178.stdout: "map_full": 0, 2024-09-11T00:32:41.878 INFO:teuthology.orchestra.run.smithi178.stdout: "map_inc": 0, 2024-09-11T00:32:41.878 INFO:teuthology.orchestra.run.smithi178.stdout: "osd_sessions": 9, 2024-09-11T00:32:41.878 INFO:teuthology.orchestra.run.smithi178.stdout: "osd_session_open": 9, 2024-09-11T00:32:41.879 INFO:teuthology.orchestra.run.smithi178.stdout: "osd_session_close": 0, 2024-09-11T00:32:41.879 INFO:teuthology.orchestra.run.smithi178.stdout: "osd_laggy": 0, 2024-09-11T00:32:41.879 INFO:teuthology.orchestra.run.smithi178.stdout: "omap_wr": 0, 2024-09-11T00:32:41.879 INFO:teuthology.orchestra.run.smithi178.stdout: "omap_rd": 2, 2024-09-11T00:32:41.879 INFO:teuthology.orchestra.run.smithi178.stdout: "omap_del": 0 2024-09-11T00:32:41.879 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T00:32:41.879 INFO:teuthology.orchestra.run.smithi178.stdout: "oft": { 2024-09-11T00:32:41.879 INFO:teuthology.orchestra.run.smithi178.stdout: "omap_total_objs": 0, 2024-09-11T00:32:41.879 INFO:teuthology.orchestra.run.smithi178.stdout: "omap_total_kv_pairs": 0, 2024-09-11T00:32:41.879 INFO:teuthology.orchestra.run.smithi178.stdout: "omap_total_updates": 0, 2024-09-11T00:32:41.879 INFO:teuthology.orchestra.run.smithi178.stdout: "omap_total_removes": 0 2024-09-11T00:32:41.879 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T00:32:41.879 INFO:teuthology.orchestra.run.smithi178.stdout: "purge_queue": { 2024-09-11T00:32:41.880 INFO:teuthology.orchestra.run.smithi178.stdout: "pq_executing_ops": 0, 2024-09-11T00:32:41.880 INFO:teuthology.orchestra.run.smithi178.stdout: "pq_executing_ops_high_water": 0, 2024-09-11T00:32:41.880 INFO:teuthology.orchestra.run.smithi178.stdout: "pq_executing": 0, 2024-09-11T00:32:41.880 INFO:teuthology.orchestra.run.smithi178.stdout: "pq_executing_high_water": 0, 2024-09-11T00:32:41.880 INFO:teuthology.orchestra.run.smithi178.stdout: "pq_executed_ops": 0, 2024-09-11T00:32:41.880 INFO:teuthology.orchestra.run.smithi178.stdout: "pq_executed": 0, 2024-09-11T00:32:41.880 INFO:teuthology.orchestra.run.smithi178.stdout: "pq_item_in_journal": 0 2024-09-11T00:32:41.880 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T00:32:41.880 INFO:teuthology.orchestra.run.smithi178.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-11T00:32:41.880 INFO:teuthology.orchestra.run.smithi178.stdout: "val": 0, 2024-09-11T00:32:41.880 INFO:teuthology.orchestra.run.smithi178.stdout: "max": 104857600, 2024-09-11T00:32:41.881 INFO:teuthology.orchestra.run.smithi178.stdout: "get_started": 0, 2024-09-11T00:32:41.881 INFO:teuthology.orchestra.run.smithi178.stdout: "get": 1150, 2024-09-11T00:32:41.881 INFO:teuthology.orchestra.run.smithi178.stdout: "get_sum": 55563870, 2024-09-11T00:32:41.881 INFO:teuthology.orchestra.run.smithi178.stdout: "get_or_fail_fail": 0, 2024-09-11T00:32:41.881 INFO:teuthology.orchestra.run.smithi178.stdout: "get_or_fail_success": 1150, 2024-09-11T00:32:41.881 INFO:teuthology.orchestra.run.smithi178.stdout: "take": 0, 2024-09-11T00:32:41.881 INFO:teuthology.orchestra.run.smithi178.stdout: "take_sum": 0, 2024-09-11T00:32:41.881 INFO:teuthology.orchestra.run.smithi178.stdout: "put": 1150, 2024-09-11T00:32:41.881 INFO:teuthology.orchestra.run.smithi178.stdout: "put_sum": 55563870, 2024-09-11T00:32:41.881 INFO:teuthology.orchestra.run.smithi178.stdout: "wait": { 2024-09-11T00:32:41.881 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-11T00:32:41.882 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-11T00:32:41.882 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-11T00:32:41.882 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-11T00:32:41.882 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T00:32:41.882 INFO:teuthology.orchestra.run.smithi178.stdout: "throttle-objecter_bytes": { 2024-09-11T00:32:41.882 INFO:teuthology.orchestra.run.smithi178.stdout: "val": 0, 2024-09-11T00:32:41.882 INFO:teuthology.orchestra.run.smithi178.stdout: "max": 104857600, 2024-09-11T00:32:41.882 INFO:teuthology.orchestra.run.smithi178.stdout: "get_started": 0, 2024-09-11T00:32:41.882 INFO:teuthology.orchestra.run.smithi178.stdout: "get": 0, 2024-09-11T00:32:41.882 INFO:teuthology.orchestra.run.smithi178.stdout: "get_sum": 0, 2024-09-11T00:32:41.882 INFO:teuthology.orchestra.run.smithi178.stdout: "get_or_fail_fail": 0, 2024-09-11T00:32:41.882 INFO:teuthology.orchestra.run.smithi178.stdout: "get_or_fail_success": 0, 2024-09-11T00:32:41.882 INFO:teuthology.orchestra.run.smithi178.stdout: "take": 1040, 2024-09-11T00:32:41.882 INFO:teuthology.orchestra.run.smithi178.stdout: "take_sum": 55027421, 2024-09-11T00:32:41.883 INFO:teuthology.orchestra.run.smithi178.stdout: "put": 1040, 2024-09-11T00:32:41.883 INFO:teuthology.orchestra.run.smithi178.stdout: "put_sum": 55027421, 2024-09-11T00:32:41.883 INFO:teuthology.orchestra.run.smithi178.stdout: "wait": { 2024-09-11T00:32:41.883 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-11T00:32:41.883 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-11T00:32:41.883 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-11T00:32:41.883 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-11T00:32:41.883 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T00:32:41.883 INFO:teuthology.orchestra.run.smithi178.stdout: "throttle-objecter_ops": { 2024-09-11T00:32:41.883 INFO:teuthology.orchestra.run.smithi178.stdout: "val": 0, 2024-09-11T00:32:41.883 INFO:teuthology.orchestra.run.smithi178.stdout: "max": 1024, 2024-09-11T00:32:41.883 INFO:teuthology.orchestra.run.smithi178.stdout: "get_started": 0, 2024-09-11T00:32:41.883 INFO:teuthology.orchestra.run.smithi178.stdout: "get": 0, 2024-09-11T00:32:41.883 INFO:teuthology.orchestra.run.smithi178.stdout: "get_sum": 0, 2024-09-11T00:32:41.884 INFO:teuthology.orchestra.run.smithi178.stdout: "get_or_fail_fail": 0, 2024-09-11T00:32:41.884 INFO:teuthology.orchestra.run.smithi178.stdout: "get_or_fail_success": 0, 2024-09-11T00:32:41.884 INFO:teuthology.orchestra.run.smithi178.stdout: "take": 1040, 2024-09-11T00:32:41.884 INFO:teuthology.orchestra.run.smithi178.stdout: "take_sum": 1040, 2024-09-11T00:32:41.884 INFO:teuthology.orchestra.run.smithi178.stdout: "put": 1040, 2024-09-11T00:32:41.884 INFO:teuthology.orchestra.run.smithi178.stdout: "put_sum": 1040, 2024-09-11T00:32:41.884 INFO:teuthology.orchestra.run.smithi178.stdout: "wait": { 2024-09-11T00:32:41.884 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-11T00:32:41.884 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-11T00:32:41.884 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-11T00:32:41.884 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-11T00:32:41.884 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T00:32:41.884 INFO:teuthology.orchestra.run.smithi178.stdout: "throttle-write_buf_throttle": { 2024-09-11T00:32:41.884 INFO:teuthology.orchestra.run.smithi178.stdout: "val": 0, 2024-09-11T00:32:41.885 INFO:teuthology.orchestra.run.smithi178.stdout: "max": 3758096384, 2024-09-11T00:32:41.885 INFO:teuthology.orchestra.run.smithi178.stdout: "get_started": 0, 2024-09-11T00:32:41.885 INFO:teuthology.orchestra.run.smithi178.stdout: "get": 0, 2024-09-11T00:32:41.885 INFO:teuthology.orchestra.run.smithi178.stdout: "get_sum": 0, 2024-09-11T00:32:41.885 INFO:teuthology.orchestra.run.smithi178.stdout: "get_or_fail_fail": 0, 2024-09-11T00:32:41.885 INFO:teuthology.orchestra.run.smithi178.stdout: "get_or_fail_success": 0, 2024-09-11T00:32:41.885 INFO:teuthology.orchestra.run.smithi178.stdout: "take": 0, 2024-09-11T00:32:41.885 INFO:teuthology.orchestra.run.smithi178.stdout: "take_sum": 0, 2024-09-11T00:32:41.885 INFO:teuthology.orchestra.run.smithi178.stdout: "put": 0, 2024-09-11T00:32:41.885 INFO:teuthology.orchestra.run.smithi178.stdout: "put_sum": 0, 2024-09-11T00:32:41.885 INFO:teuthology.orchestra.run.smithi178.stdout: "wait": { 2024-09-11T00:32:41.885 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-11T00:32:41.885 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-11T00:32:41.886 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-11T00:32:41.886 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-11T00:32:41.886 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T00:32:41.886 INFO:teuthology.orchestra.run.smithi178.stdout: "throttle-write_buf_throttle-0x55e8381aac80": { 2024-09-11T00:32:41.886 INFO:teuthology.orchestra.run.smithi178.stdout: "val": 0, 2024-09-11T00:32:41.886 INFO:teuthology.orchestra.run.smithi178.stdout: "max": 3758096384, 2024-09-11T00:32:41.886 INFO:teuthology.orchestra.run.smithi178.stdout: "get_started": 0, 2024-09-11T00:32:41.886 INFO:teuthology.orchestra.run.smithi178.stdout: "get": 0, 2024-09-11T00:32:41.886 INFO:teuthology.orchestra.run.smithi178.stdout: "get_sum": 0, 2024-09-11T00:32:41.886 INFO:teuthology.orchestra.run.smithi178.stdout: "get_or_fail_fail": 0, 2024-09-11T00:32:41.886 INFO:teuthology.orchestra.run.smithi178.stdout: "get_or_fail_success": 0, 2024-09-11T00:32:41.886 INFO:teuthology.orchestra.run.smithi178.stdout: "take": 0, 2024-09-11T00:32:41.886 INFO:teuthology.orchestra.run.smithi178.stdout: "take_sum": 0, 2024-09-11T00:32:41.886 INFO:teuthology.orchestra.run.smithi178.stdout: "put": 0, 2024-09-11T00:32:41.887 INFO:teuthology.orchestra.run.smithi178.stdout: "put_sum": 0, 2024-09-11T00:32:41.887 INFO:teuthology.orchestra.run.smithi178.stdout: "wait": { 2024-09-11T00:32:41.887 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-11T00:32:41.887 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-11T00:32:41.887 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-11T00:32:41.887 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-11T00:32:41.887 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T00:32:41.887 INFO:teuthology.orchestra.run.smithi178.stdout: "trackedop": { 2024-09-11T00:32:41.887 INFO:teuthology.orchestra.run.smithi178.stdout: "slow_ops_count": 0 2024-09-11T00:32:41.887 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-11T00:32:41.887 INFO:teuthology.orchestra.run.smithi178.stdout:} 2024-09-11T00:32:42.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:32:42 smithi165 ceph-mon[35356]: pgmap v1168: 129 pgs: 129 active+clean; 671 MiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 767 B/s wr, 10 op/s 2024-09-11T00:32:42.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:32:42 smithi012 ceph-mon[30333]: pgmap v1168: 129 pgs: 129 active+clean; 671 MiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 767 B/s wr, 10 op/s 2024-09-11T00:32:42.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:32:42 smithi178 ceph-mon[35919]: pgmap v1168: 129 pgs: 129 active+clean; 671 MiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 767 B/s wr, 10 op/s 2024-09-11T00:32:42.891 INFO:tasks.check_counter:Daemon mds.c mds.exported=0 2024-09-11T00:32:42.891 INFO:tasks.check_counter:Daemon mds.c mds.imported=0 2024-09-11T00:32:42.891 DEBUG:tasks.check_counter:Getting stats from f 2024-09-11T00:32:42.891 DEBUG:teuthology.orchestra.run.smithi178:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid ff3ad74a-6fcf-11ef-bcea-c7b262605968 -- ceph daemon mds.f perf dump 2024-09-11T00:32:43.149 INFO:teuthology.orchestra.run.smithi178.stderr:Inferring config /var/lib/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/mon.c/config 2024-09-11T00:32:43.722 INFO:teuthology.orchestra.run.smithi178.stdout:{ 2024-09-11T00:32:43.723 INFO:teuthology.orchestra.run.smithi178.stdout: "AsyncMessenger::Worker-0": { 2024-09-11T00:32:43.723 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_recv_messages": 2660, 2024-09-11T00:32:43.723 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_messages": 2571, 2024-09-11T00:32:43.723 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_recv_bytes": 31212444, 2024-09-11T00:32:43.723 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_bytes": 955538, 2024-09-11T00:32:43.723 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_created_connections": 7, 2024-09-11T00:32:43.723 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_active_connections": 3, 2024-09-11T00:32:43.723 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_total_time": 0.327503333, 2024-09-11T00:32:43.723 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_send_time": 0.137576345, 2024-09-11T00:32:43.723 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_recv_time": 0.116052526, 2024-09-11T00:32:43.723 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_fast_dispatch_time": 0.064402160, 2024-09-11T00:32:43.723 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_messages_queue_lat": { 2024-09-11T00:32:43.724 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 2571, 2024-09-11T00:32:43.724 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.122495377, 2024-09-11T00:32:43.724 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000047645 2024-09-11T00:32:43.724 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T00:32:43.724 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_handle_ack_lat": { 2024-09-11T00:32:43.724 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-11T00:32:43.724 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-11T00:32:43.724 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-11T00:32:43.724 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T00:32:43.724 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_recv_encrypted_bytes": 1184816, 2024-09-11T00:32:43.724 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_encrypted_bytes": 289328 2024-09-11T00:32:43.724 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T00:32:43.724 INFO:teuthology.orchestra.run.smithi178.stdout: "AsyncMessenger::Worker-1": { 2024-09-11T00:32:43.725 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_recv_messages": 1106, 2024-09-11T00:32:43.725 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_messages": 1458, 2024-09-11T00:32:43.725 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_recv_bytes": 62381085, 2024-09-11T00:32:43.725 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_bytes": 1143902, 2024-09-11T00:32:43.725 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_created_connections": 8, 2024-09-11T00:32:43.725 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_active_connections": 6, 2024-09-11T00:32:43.725 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_total_time": 0.163812506, 2024-09-11T00:32:43.725 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_send_time": 0.060810992, 2024-09-11T00:32:43.725 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_recv_time": 0.063796978, 2024-09-11T00:32:43.725 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_fast_dispatch_time": 0.028367624, 2024-09-11T00:32:43.725 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_messages_queue_lat": { 2024-09-11T00:32:43.726 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 1456, 2024-09-11T00:32:43.726 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.058830745, 2024-09-11T00:32:43.726 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000040405 2024-09-11T00:32:43.726 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T00:32:43.726 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_handle_ack_lat": { 2024-09-11T00:32:43.726 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-11T00:32:43.726 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-11T00:32:43.726 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-11T00:32:43.726 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T00:32:43.726 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_recv_encrypted_bytes": 320, 2024-09-11T00:32:43.726 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_encrypted_bytes": 792768 2024-09-11T00:32:43.726 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T00:32:43.726 INFO:teuthology.orchestra.run.smithi178.stdout: "AsyncMessenger::Worker-2": { 2024-09-11T00:32:43.727 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_recv_messages": 2409, 2024-09-11T00:32:43.727 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_messages": 2411, 2024-09-11T00:32:43.727 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_recv_bytes": 84503045, 2024-09-11T00:32:43.727 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_bytes": 766069, 2024-09-11T00:32:43.727 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_created_connections": 7, 2024-09-11T00:32:43.727 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_active_connections": 5, 2024-09-11T00:32:43.727 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_total_time": 0.252884467, 2024-09-11T00:32:43.727 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_send_time": 0.090434938, 2024-09-11T00:32:43.727 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_recv_time": 0.095092477, 2024-09-11T00:32:43.727 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_fast_dispatch_time": 0.052864443, 2024-09-11T00:32:43.728 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_messages_queue_lat": { 2024-09-11T00:32:43.728 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 2409, 2024-09-11T00:32:43.728 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.089756131, 2024-09-11T00:32:43.728 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000037258 2024-09-11T00:32:43.728 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T00:32:43.728 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_handle_ack_lat": { 2024-09-11T00:32:43.728 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-11T00:32:43.728 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-11T00:32:43.728 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-11T00:32:43.729 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T00:32:43.729 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-11T00:32:43.729 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_encrypted_bytes": 0 2024-09-11T00:32:43.729 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T00:32:43.729 INFO:teuthology.orchestra.run.smithi178.stdout: "cct": { 2024-09-11T00:32:43.729 INFO:teuthology.orchestra.run.smithi178.stdout: "total_workers": 1, 2024-09-11T00:32:43.729 INFO:teuthology.orchestra.run.smithi178.stdout: "unhealthy_workers": 0 2024-09-11T00:32:43.729 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T00:32:43.729 INFO:teuthology.orchestra.run.smithi178.stdout: "finisher-MDSRank": { 2024-09-11T00:32:43.729 INFO:teuthology.orchestra.run.smithi178.stdout: "queue_len": 0, 2024-09-11T00:32:43.729 INFO:teuthology.orchestra.run.smithi178.stdout: "complete_latency": { 2024-09-11T00:32:43.729 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 9596, 2024-09-11T00:32:43.729 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.221185430, 2024-09-11T00:32:43.730 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000023049 2024-09-11T00:32:43.730 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-11T00:32:43.730 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T00:32:43.730 INFO:teuthology.orchestra.run.smithi178.stdout: "finisher-PurgeQueue": { 2024-09-11T00:32:43.730 INFO:teuthology.orchestra.run.smithi178.stdout: "queue_len": 0, 2024-09-11T00:32:43.730 INFO:teuthology.orchestra.run.smithi178.stdout: "complete_latency": { 2024-09-11T00:32:43.730 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-11T00:32:43.730 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-11T00:32:43.730 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-11T00:32:43.730 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-11T00:32:43.730 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T00:32:43.730 INFO:teuthology.orchestra.run.smithi178.stdout: "mds": { 2024-09-11T00:32:43.731 INFO:teuthology.orchestra.run.smithi178.stdout: "request": 0, 2024-09-11T00:32:43.731 INFO:teuthology.orchestra.run.smithi178.stdout: "reply": 0, 2024-09-11T00:32:43.731 INFO:teuthology.orchestra.run.smithi178.stdout: "reply_latency": { 2024-09-11T00:32:43.731 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-11T00:32:43.731 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-11T00:32:43.731 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-11T00:32:43.731 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T00:32:43.731 INFO:teuthology.orchestra.run.smithi178.stdout: "slow_reply": 0, 2024-09-11T00:32:43.731 INFO:teuthology.orchestra.run.smithi178.stdout: "forward": 0, 2024-09-11T00:32:43.731 INFO:teuthology.orchestra.run.smithi178.stdout: "dir_fetch_complete": 0, 2024-09-11T00:32:43.731 INFO:teuthology.orchestra.run.smithi178.stdout: "dir_fetch_keys": 0, 2024-09-11T00:32:43.731 INFO:teuthology.orchestra.run.smithi178.stdout: "dir_commit": 0, 2024-09-11T00:32:43.732 INFO:teuthology.orchestra.run.smithi178.stdout: "dir_split": 0, 2024-09-11T00:32:43.732 INFO:teuthology.orchestra.run.smithi178.stdout: "dir_merge": 0, 2024-09-11T00:32:43.732 INFO:teuthology.orchestra.run.smithi178.stdout: "inodes": 20193, 2024-09-11T00:32:43.732 INFO:teuthology.orchestra.run.smithi178.stdout: "inodes_top": 0, 2024-09-11T00:32:43.732 INFO:teuthology.orchestra.run.smithi178.stdout: "inodes_bottom": 20193, 2024-09-11T00:32:43.732 INFO:teuthology.orchestra.run.smithi178.stdout: "inodes_pin_tail": 0, 2024-09-11T00:32:43.732 INFO:teuthology.orchestra.run.smithi178.stdout: "inodes_pinned": 20193, 2024-09-11T00:32:43.732 INFO:teuthology.orchestra.run.smithi178.stdout: "inodes_expired": 0, 2024-09-11T00:32:43.732 INFO:teuthology.orchestra.run.smithi178.stdout: "inodes_with_caps": 0, 2024-09-11T00:32:43.732 INFO:teuthology.orchestra.run.smithi178.stdout: "caps": 0, 2024-09-11T00:32:43.732 INFO:teuthology.orchestra.run.smithi178.stdout: "subtrees": 1, 2024-09-11T00:32:43.732 INFO:teuthology.orchestra.run.smithi178.stdout: "traverse": 0, 2024-09-11T00:32:43.733 INFO:teuthology.orchestra.run.smithi178.stdout: "traverse_hit": 0, 2024-09-11T00:32:43.733 INFO:teuthology.orchestra.run.smithi178.stdout: "traverse_forward": 0, 2024-09-11T00:32:43.733 INFO:teuthology.orchestra.run.smithi178.stdout: "traverse_discover": 0, 2024-09-11T00:32:43.733 INFO:teuthology.orchestra.run.smithi178.stdout: "traverse_dir_fetch": 0, 2024-09-11T00:32:43.733 INFO:teuthology.orchestra.run.smithi178.stdout: "traverse_remote_ino": 0, 2024-09-11T00:32:43.733 INFO:teuthology.orchestra.run.smithi178.stdout: "traverse_lock": 0, 2024-09-11T00:32:43.733 INFO:teuthology.orchestra.run.smithi178.stdout: "load_cent": 0, 2024-09-11T00:32:43.733 INFO:teuthology.orchestra.run.smithi178.stdout: "q": 0, 2024-09-11T00:32:43.733 INFO:teuthology.orchestra.run.smithi178.stdout: "exported": 0, 2024-09-11T00:32:43.733 INFO:teuthology.orchestra.run.smithi178.stdout: "exported_inodes": 0, 2024-09-11T00:32:43.733 INFO:teuthology.orchestra.run.smithi178.stdout: "imported": 0, 2024-09-11T00:32:43.733 INFO:teuthology.orchestra.run.smithi178.stdout: "imported_inodes": 0, 2024-09-11T00:32:43.733 INFO:teuthology.orchestra.run.smithi178.stdout: "openino_dir_fetch": 0, 2024-09-11T00:32:43.734 INFO:teuthology.orchestra.run.smithi178.stdout: "openino_backtrace_fetch": 0, 2024-09-11T00:32:43.734 INFO:teuthology.orchestra.run.smithi178.stdout: "openino_peer_discover": 0, 2024-09-11T00:32:43.734 INFO:teuthology.orchestra.run.smithi178.stdout: "root_rfiles": 0, 2024-09-11T00:32:43.734 INFO:teuthology.orchestra.run.smithi178.stdout: "root_rbytes": 0, 2024-09-11T00:32:43.734 INFO:teuthology.orchestra.run.smithi178.stdout: "root_rsnaps": 0, 2024-09-11T00:32:43.734 INFO:teuthology.orchestra.run.smithi178.stdout: "scrub_backtrace_fetch": 0, 2024-09-11T00:32:43.734 INFO:teuthology.orchestra.run.smithi178.stdout: "scrub_set_tag": 0, 2024-09-11T00:32:43.734 INFO:teuthology.orchestra.run.smithi178.stdout: "scrub_backtrace_repaired": 0, 2024-09-11T00:32:43.734 INFO:teuthology.orchestra.run.smithi178.stdout: "scrub_inotable_repaired": 0, 2024-09-11T00:32:43.734 INFO:teuthology.orchestra.run.smithi178.stdout: "scrub_dir_inodes": 0, 2024-09-11T00:32:43.734 INFO:teuthology.orchestra.run.smithi178.stdout: "scrub_dir_base_inodes": 0, 2024-09-11T00:32:43.734 INFO:teuthology.orchestra.run.smithi178.stdout: "scrub_dirfrag_rstats": 0, 2024-09-11T00:32:43.734 INFO:teuthology.orchestra.run.smithi178.stdout: "scrub_file_inodes": 0, 2024-09-11T00:32:43.734 INFO:teuthology.orchestra.run.smithi178.stdout: "handle_inode_file_caps": 0, 2024-09-11T00:32:43.735 INFO:teuthology.orchestra.run.smithi178.stdout: "ceph_cap_op_revoke": 0, 2024-09-11T00:32:43.735 INFO:teuthology.orchestra.run.smithi178.stdout: "ceph_cap_op_grant": 0, 2024-09-11T00:32:43.735 INFO:teuthology.orchestra.run.smithi178.stdout: "ceph_cap_op_trunc": 0, 2024-09-11T00:32:43.735 INFO:teuthology.orchestra.run.smithi178.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-11T00:32:43.735 INFO:teuthology.orchestra.run.smithi178.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-11T00:32:43.735 INFO:teuthology.orchestra.run.smithi178.stdout: "handle_client_caps": 0, 2024-09-11T00:32:43.735 INFO:teuthology.orchestra.run.smithi178.stdout: "handle_client_caps_dirty": 0, 2024-09-11T00:32:43.735 INFO:teuthology.orchestra.run.smithi178.stdout: "handle_client_cap_release": 0, 2024-09-11T00:32:43.735 INFO:teuthology.orchestra.run.smithi178.stdout: "process_request_cap_release": 0 2024-09-11T00:32:43.735 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T00:32:43.735 INFO:teuthology.orchestra.run.smithi178.stdout: "mds_cache": { 2024-09-11T00:32:43.735 INFO:teuthology.orchestra.run.smithi178.stdout: "dir_update": 0, 2024-09-11T00:32:43.735 INFO:teuthology.orchestra.run.smithi178.stdout: "dir_update_receipt": 0, 2024-09-11T00:32:43.736 INFO:teuthology.orchestra.run.smithi178.stdout: "dir_try_discover": 0, 2024-09-11T00:32:43.736 INFO:teuthology.orchestra.run.smithi178.stdout: "dir_send_discover": 0, 2024-09-11T00:32:43.736 INFO:teuthology.orchestra.run.smithi178.stdout: "dir_handle_discover": 0, 2024-09-11T00:32:43.736 INFO:teuthology.orchestra.run.smithi178.stdout: "num_strays": 0, 2024-09-11T00:32:43.736 INFO:teuthology.orchestra.run.smithi178.stdout: "num_strays_delayed": 0, 2024-09-11T00:32:43.736 INFO:teuthology.orchestra.run.smithi178.stdout: "num_strays_enqueuing": 0, 2024-09-11T00:32:43.736 INFO:teuthology.orchestra.run.smithi178.stdout: "strays_created": 0, 2024-09-11T00:32:43.736 INFO:teuthology.orchestra.run.smithi178.stdout: "strays_enqueued": 0, 2024-09-11T00:32:43.736 INFO:teuthology.orchestra.run.smithi178.stdout: "strays_reintegrated": 0, 2024-09-11T00:32:43.736 INFO:teuthology.orchestra.run.smithi178.stdout: "strays_migrated": 0, 2024-09-11T00:32:43.736 INFO:teuthology.orchestra.run.smithi178.stdout: "num_recovering_processing": 0, 2024-09-11T00:32:43.736 INFO:teuthology.orchestra.run.smithi178.stdout: "num_recovering_enqueued": 0, 2024-09-11T00:32:43.736 INFO:teuthology.orchestra.run.smithi178.stdout: "num_recovering_prioritized": 0, 2024-09-11T00:32:43.736 INFO:teuthology.orchestra.run.smithi178.stdout: "recovery_started": 0, 2024-09-11T00:32:43.737 INFO:teuthology.orchestra.run.smithi178.stdout: "recovery_completed": 0, 2024-09-11T00:32:43.737 INFO:teuthology.orchestra.run.smithi178.stdout: "ireq_quiesce_path": 0, 2024-09-11T00:32:43.737 INFO:teuthology.orchestra.run.smithi178.stdout: "ireq_quiesce_inode": 0, 2024-09-11T00:32:43.737 INFO:teuthology.orchestra.run.smithi178.stdout: "ireq_enqueue_scrub": 0, 2024-09-11T00:32:43.737 INFO:teuthology.orchestra.run.smithi178.stdout: "ireq_exportdir": 0, 2024-09-11T00:32:43.737 INFO:teuthology.orchestra.run.smithi178.stdout: "ireq_flush": 0, 2024-09-11T00:32:43.737 INFO:teuthology.orchestra.run.smithi178.stdout: "ireq_fragmentdir": 0, 2024-09-11T00:32:43.737 INFO:teuthology.orchestra.run.smithi178.stdout: "ireq_fragstats": 0, 2024-09-11T00:32:43.737 INFO:teuthology.orchestra.run.smithi178.stdout: "ireq_inodestats": 0, 2024-09-11T00:32:43.737 INFO:teuthology.orchestra.run.smithi178.stdout: "uninline_started": 0, 2024-09-11T00:32:43.737 INFO:teuthology.orchestra.run.smithi178.stdout: "uninline_succeeded": 0, 2024-09-11T00:32:43.738 INFO:teuthology.orchestra.run.smithi178.stdout: "uninline_write_failed": 0 2024-09-11T00:32:43.738 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T00:32:43.738 INFO:teuthology.orchestra.run.smithi178.stdout: "mds_log": { 2024-09-11T00:32:43.738 INFO:teuthology.orchestra.run.smithi178.stdout: "evlrg": 0, 2024-09-11T00:32:43.738 INFO:teuthology.orchestra.run.smithi178.stdout: "evadd": 0, 2024-09-11T00:32:43.738 INFO:teuthology.orchestra.run.smithi178.stdout: "evex": 0, 2024-09-11T00:32:43.738 INFO:teuthology.orchestra.run.smithi178.stdout: "evtrm": 0, 2024-09-11T00:32:43.738 INFO:teuthology.orchestra.run.smithi178.stdout: "ev": 98584, 2024-09-11T00:32:43.738 INFO:teuthology.orchestra.run.smithi178.stdout: "evexg": 0, 2024-09-11T00:32:43.738 INFO:teuthology.orchestra.run.smithi178.stdout: "evexd": 0, 2024-09-11T00:32:43.738 INFO:teuthology.orchestra.run.smithi178.stdout: "segadd": 0, 2024-09-11T00:32:43.738 INFO:teuthology.orchestra.run.smithi178.stdout: "segex": 0, 2024-09-11T00:32:43.738 INFO:teuthology.orchestra.run.smithi178.stdout: "segtrm": 0, 2024-09-11T00:32:43.738 INFO:teuthology.orchestra.run.smithi178.stdout: "seg": 129, 2024-09-11T00:32:43.739 INFO:teuthology.orchestra.run.smithi178.stdout: "segmjr": 10, 2024-09-11T00:32:43.739 INFO:teuthology.orchestra.run.smithi178.stdout: "segexg": 0, 2024-09-11T00:32:43.739 INFO:teuthology.orchestra.run.smithi178.stdout: "segexd": 0, 2024-09-11T00:32:43.739 INFO:teuthology.orchestra.run.smithi178.stdout: "expos": 4194352, 2024-09-11T00:32:43.739 INFO:teuthology.orchestra.run.smithi178.stdout: "wrpos": 179539983, 2024-09-11T00:32:43.739 INFO:teuthology.orchestra.run.smithi178.stdout: "rdpos": 179538845, 2024-09-11T00:32:43.739 INFO:teuthology.orchestra.run.smithi178.stdout: "jlat": { 2024-09-11T00:32:43.739 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-11T00:32:43.739 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-11T00:32:43.739 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-11T00:32:43.739 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T00:32:43.739 INFO:teuthology.orchestra.run.smithi178.stdout: "replayed": 98584 2024-09-11T00:32:43.739 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T00:32:43.739 INFO:teuthology.orchestra.run.smithi178.stdout: "mds_mem": { 2024-09-11T00:32:43.740 INFO:teuthology.orchestra.run.smithi178.stdout: "ino": 15, 2024-09-11T00:32:43.740 INFO:teuthology.orchestra.run.smithi178.stdout: "ino+": 20392, 2024-09-11T00:32:43.740 INFO:teuthology.orchestra.run.smithi178.stdout: "ino-": 20377, 2024-09-11T00:32:43.740 INFO:teuthology.orchestra.run.smithi178.stdout: "dir": 13, 2024-09-11T00:32:43.740 INFO:teuthology.orchestra.run.smithi178.stdout: "dir+": 89, 2024-09-11T00:32:43.740 INFO:teuthology.orchestra.run.smithi178.stdout: "dir-": 76, 2024-09-11T00:32:43.740 INFO:teuthology.orchestra.run.smithi178.stdout: "dn": 20193, 2024-09-11T00:32:43.740 INFO:teuthology.orchestra.run.smithi178.stdout: "dn+": 20931, 2024-09-11T00:32:43.740 INFO:teuthology.orchestra.run.smithi178.stdout: "dn-": 738, 2024-09-11T00:32:43.740 INFO:teuthology.orchestra.run.smithi178.stdout: "cap": 0, 2024-09-11T00:32:43.740 INFO:teuthology.orchestra.run.smithi178.stdout: "cap+": 0, 2024-09-11T00:32:43.740 INFO:teuthology.orchestra.run.smithi178.stdout: "cap-": 0, 2024-09-11T00:32:43.740 INFO:teuthology.orchestra.run.smithi178.stdout: "rss": 69368, 2024-09-11T00:32:43.740 INFO:teuthology.orchestra.run.smithi178.stdout: "heap": 272879616 2024-09-11T00:32:43.740 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T00:32:43.741 INFO:teuthology.orchestra.run.smithi178.stdout: "mds_server": { 2024-09-11T00:32:43.741 INFO:teuthology.orchestra.run.smithi178.stdout: "dispatch_client_request": 0, 2024-09-11T00:32:43.741 INFO:teuthology.orchestra.run.smithi178.stdout: "dispatch_server_request": 0, 2024-09-11T00:32:43.741 INFO:teuthology.orchestra.run.smithi178.stdout: "handle_client_request": 0, 2024-09-11T00:32:43.741 INFO:teuthology.orchestra.run.smithi178.stdout: "handle_client_session": 0, 2024-09-11T00:32:43.741 INFO:teuthology.orchestra.run.smithi178.stdout: "handle_peer_request": 0, 2024-09-11T00:32:43.741 INFO:teuthology.orchestra.run.smithi178.stdout: "req_create_latency": { 2024-09-11T00:32:43.741 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-11T00:32:43.741 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-11T00:32:43.741 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-11T00:32:43.741 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T00:32:43.741 INFO:teuthology.orchestra.run.smithi178.stdout: "req_getattr_latency": { 2024-09-11T00:32:43.741 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-11T00:32:43.742 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-11T00:32:43.742 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-11T00:32:43.742 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T00:32:43.742 INFO:teuthology.orchestra.run.smithi178.stdout: "req_getfilelock_latency": { 2024-09-11T00:32:43.742 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-11T00:32:43.742 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-11T00:32:43.742 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-11T00:32:43.742 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T00:32:43.742 INFO:teuthology.orchestra.run.smithi178.stdout: "req_link_latency": { 2024-09-11T00:32:43.742 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-11T00:32:43.742 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-11T00:32:43.742 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-11T00:32:43.742 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T00:32:43.742 INFO:teuthology.orchestra.run.smithi178.stdout: "req_lookup_latency": { 2024-09-11T00:32:43.742 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-11T00:32:43.742 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-11T00:32:43.743 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-11T00:32:43.743 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T00:32:43.743 INFO:teuthology.orchestra.run.smithi178.stdout: "req_lookuphash_latency": { 2024-09-11T00:32:43.743 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-11T00:32:43.743 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-11T00:32:43.743 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-11T00:32:43.743 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T00:32:43.743 INFO:teuthology.orchestra.run.smithi178.stdout: "req_lookupino_latency": { 2024-09-11T00:32:43.743 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-11T00:32:43.743 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-11T00:32:43.743 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-11T00:32:43.743 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T00:32:43.743 INFO:teuthology.orchestra.run.smithi178.stdout: "req_lookupname_latency": { 2024-09-11T00:32:43.743 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-11T00:32:43.744 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-11T00:32:43.744 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-11T00:32:43.744 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T00:32:43.744 INFO:teuthology.orchestra.run.smithi178.stdout: "req_lookupparent_latency": { 2024-09-11T00:32:43.744 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-11T00:32:43.744 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-11T00:32:43.744 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-11T00:32:43.744 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T00:32:43.744 INFO:teuthology.orchestra.run.smithi178.stdout: "req_lookupsnap_latency": { 2024-09-11T00:32:43.744 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-11T00:32:43.744 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-11T00:32:43.744 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-11T00:32:43.744 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T00:32:43.744 INFO:teuthology.orchestra.run.smithi178.stdout: "req_lssnap_latency": { 2024-09-11T00:32:43.745 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-11T00:32:43.745 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-11T00:32:43.745 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-11T00:32:43.745 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T00:32:43.745 INFO:teuthology.orchestra.run.smithi178.stdout: "req_mkdir_latency": { 2024-09-11T00:32:43.746 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-11T00:32:43.746 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-11T00:32:43.746 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-11T00:32:43.746 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T00:32:43.746 INFO:teuthology.orchestra.run.smithi178.stdout: "req_mknod_latency": { 2024-09-11T00:32:43.746 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-11T00:32:43.746 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-11T00:32:43.746 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-11T00:32:43.746 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T00:32:43.746 INFO:teuthology.orchestra.run.smithi178.stdout: "req_mksnap_latency": { 2024-09-11T00:32:43.746 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-11T00:32:43.746 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-11T00:32:43.746 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-11T00:32:43.747 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T00:32:43.747 INFO:teuthology.orchestra.run.smithi178.stdout: "req_open_latency": { 2024-09-11T00:32:43.747 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-11T00:32:43.747 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-11T00:32:43.747 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-11T00:32:43.747 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T00:32:43.747 INFO:teuthology.orchestra.run.smithi178.stdout: "req_readdir_latency": { 2024-09-11T00:32:43.747 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-11T00:32:43.747 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-11T00:32:43.747 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-11T00:32:43.747 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T00:32:43.747 INFO:teuthology.orchestra.run.smithi178.stdout: "req_rename_latency": { 2024-09-11T00:32:43.747 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-11T00:32:43.748 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-11T00:32:43.748 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-11T00:32:43.748 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T00:32:43.748 INFO:teuthology.orchestra.run.smithi178.stdout: "req_renamesnap_latency": { 2024-09-11T00:32:43.748 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-11T00:32:43.748 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-11T00:32:43.748 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-11T00:32:43.748 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T00:32:43.748 INFO:teuthology.orchestra.run.smithi178.stdout: "req_snapdiff_latency": { 2024-09-11T00:32:43.748 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-11T00:32:43.748 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-11T00:32:43.748 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-11T00:32:43.748 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T00:32:43.748 INFO:teuthology.orchestra.run.smithi178.stdout: "req_rmdir_latency": { 2024-09-11T00:32:43.749 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-11T00:32:43.749 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-11T00:32:43.749 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-11T00:32:43.749 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T00:32:43.749 INFO:teuthology.orchestra.run.smithi178.stdout: "req_rmsnap_latency": { 2024-09-11T00:32:43.749 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-11T00:32:43.749 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-11T00:32:43.749 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-11T00:32:43.749 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T00:32:43.749 INFO:teuthology.orchestra.run.smithi178.stdout: "req_rmxattr_latency": { 2024-09-11T00:32:43.750 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-11T00:32:43.750 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-11T00:32:43.750 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-11T00:32:43.750 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T00:32:43.750 INFO:teuthology.orchestra.run.smithi178.stdout: "req_setattr_latency": { 2024-09-11T00:32:43.750 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-11T00:32:43.750 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-11T00:32:43.750 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-11T00:32:43.750 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T00:32:43.750 INFO:teuthology.orchestra.run.smithi178.stdout: "req_setdirlayout_latency": { 2024-09-11T00:32:43.751 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-11T00:32:43.751 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-11T00:32:43.751 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-11T00:32:43.751 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T00:32:43.751 INFO:teuthology.orchestra.run.smithi178.stdout: "req_setfilelock_latency": { 2024-09-11T00:32:43.751 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-11T00:32:43.751 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-11T00:32:43.751 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-11T00:32:43.751 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T00:32:43.751 INFO:teuthology.orchestra.run.smithi178.stdout: "req_setlayout_latency": { 2024-09-11T00:32:43.751 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-11T00:32:43.751 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-11T00:32:43.752 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-11T00:32:43.752 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T00:32:43.752 INFO:teuthology.orchestra.run.smithi178.stdout: "req_setxattr_latency": { 2024-09-11T00:32:43.752 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-11T00:32:43.752 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-11T00:32:43.752 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-11T00:32:43.752 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T00:32:43.752 INFO:teuthology.orchestra.run.smithi178.stdout: "req_symlink_latency": { 2024-09-11T00:32:43.752 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-11T00:32:43.752 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-11T00:32:43.752 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-11T00:32:43.752 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T00:32:43.752 INFO:teuthology.orchestra.run.smithi178.stdout: "req_unlink_latency": { 2024-09-11T00:32:43.752 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-11T00:32:43.753 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-11T00:32:43.753 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-11T00:32:43.753 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T00:32:43.753 INFO:teuthology.orchestra.run.smithi178.stdout: "cap_revoke_eviction": 0, 2024-09-11T00:32:43.753 INFO:teuthology.orchestra.run.smithi178.stdout: "cap_acquisition_throttle": 0, 2024-09-11T00:32:43.753 INFO:teuthology.orchestra.run.smithi178.stdout: "req_getvxattr_latency": { 2024-09-11T00:32:43.753 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-11T00:32:43.753 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-11T00:32:43.753 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-11T00:32:43.753 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-11T00:32:43.753 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T00:32:43.753 INFO:teuthology.orchestra.run.smithi178.stdout: "mds_sessions": { 2024-09-11T00:32:43.754 INFO:teuthology.orchestra.run.smithi178.stdout: "session_count": 1, 2024-09-11T00:32:43.754 INFO:teuthology.orchestra.run.smithi178.stdout: "session_add": 1, 2024-09-11T00:32:43.754 INFO:teuthology.orchestra.run.smithi178.stdout: "session_remove": 0, 2024-09-11T00:32:43.754 INFO:teuthology.orchestra.run.smithi178.stdout: "sessions_open": 1, 2024-09-11T00:32:43.754 INFO:teuthology.orchestra.run.smithi178.stdout: "sessions_stale": 0, 2024-09-11T00:32:43.754 INFO:teuthology.orchestra.run.smithi178.stdout: "total_load": 0, 2024-09-11T00:32:43.754 INFO:teuthology.orchestra.run.smithi178.stdout: "average_load": 0, 2024-09-11T00:32:43.754 INFO:teuthology.orchestra.run.smithi178.stdout: "avg_session_uptime": 2945, 2024-09-11T00:32:43.754 INFO:teuthology.orchestra.run.smithi178.stdout: "mdthresh_evicted": 0 2024-09-11T00:32:43.754 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T00:32:43.754 INFO:teuthology.orchestra.run.smithi178.stdout: "mempool": { 2024-09-11T00:32:43.754 INFO:teuthology.orchestra.run.smithi178.stdout: "bloom_filter_bytes": 0, 2024-09-11T00:32:43.754 INFO:teuthology.orchestra.run.smithi178.stdout: "bloom_filter_items": 0, 2024-09-11T00:32:43.755 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_alloc_bytes": 0, 2024-09-11T00:32:43.755 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_alloc_items": 0, 2024-09-11T00:32:43.755 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_cache_data_bytes": 0, 2024-09-11T00:32:43.755 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_cache_data_items": 0, 2024-09-11T00:32:43.755 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-11T00:32:43.755 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_cache_onode_items": 0, 2024-09-11T00:32:43.755 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-11T00:32:43.755 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_cache_meta_items": 0, 2024-09-11T00:32:43.755 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_cache_other_bytes": 0, 2024-09-11T00:32:43.755 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_cache_other_items": 0, 2024-09-11T00:32:43.755 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-11T00:32:43.755 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_cache_buffer_items": 0, 2024-09-11T00:32:43.755 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_extent_bytes": 0, 2024-09-11T00:32:43.756 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_extent_items": 0, 2024-09-11T00:32:43.756 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_blob_bytes": 0, 2024-09-11T00:32:43.756 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_blob_items": 0, 2024-09-11T00:32:43.756 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-11T00:32:43.756 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_shared_blob_items": 0, 2024-09-11T00:32:43.756 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-11T00:32:43.756 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_inline_bl_items": 0, 2024-09-11T00:32:43.756 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_fsck_bytes": 0, 2024-09-11T00:32:43.756 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_fsck_items": 0, 2024-09-11T00:32:43.756 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_txc_bytes": 0, 2024-09-11T00:32:43.756 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_txc_items": 0, 2024-09-11T00:32:43.756 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-11T00:32:43.756 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_writing_deferred_items": 0, 2024-09-11T00:32:43.757 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_writing_bytes": 0, 2024-09-11T00:32:43.757 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_writing_items": 0, 2024-09-11T00:32:43.757 INFO:teuthology.orchestra.run.smithi178.stdout: "bluefs_bytes": 0, 2024-09-11T00:32:43.757 INFO:teuthology.orchestra.run.smithi178.stdout: "bluefs_items": 0, 2024-09-11T00:32:43.757 INFO:teuthology.orchestra.run.smithi178.stdout: "bluefs_file_reader_bytes": 0, 2024-09-11T00:32:43.757 INFO:teuthology.orchestra.run.smithi178.stdout: "bluefs_file_reader_items": 0, 2024-09-11T00:32:43.757 INFO:teuthology.orchestra.run.smithi178.stdout: "bluefs_file_writer_bytes": 0, 2024-09-11T00:32:43.757 INFO:teuthology.orchestra.run.smithi178.stdout: "bluefs_file_writer_items": 0, 2024-09-11T00:32:43.757 INFO:teuthology.orchestra.run.smithi178.stdout: "buffer_anon_bytes": 241937, 2024-09-11T00:32:43.757 INFO:teuthology.orchestra.run.smithi178.stdout: "buffer_anon_items": 88, 2024-09-11T00:32:43.757 INFO:teuthology.orchestra.run.smithi178.stdout: "buffer_meta_bytes": 0, 2024-09-11T00:32:43.757 INFO:teuthology.orchestra.run.smithi178.stdout: "buffer_meta_items": 0, 2024-09-11T00:32:43.757 INFO:teuthology.orchestra.run.smithi178.stdout: "osd_bytes": 0, 2024-09-11T00:32:43.758 INFO:teuthology.orchestra.run.smithi178.stdout: "osd_items": 0, 2024-09-11T00:32:43.758 INFO:teuthology.orchestra.run.smithi178.stdout: "osd_mapbl_bytes": 0, 2024-09-11T00:32:43.758 INFO:teuthology.orchestra.run.smithi178.stdout: "osd_mapbl_items": 0, 2024-09-11T00:32:43.758 INFO:teuthology.orchestra.run.smithi178.stdout: "osd_pglog_bytes": 0, 2024-09-11T00:32:43.758 INFO:teuthology.orchestra.run.smithi178.stdout: "osd_pglog_items": 0, 2024-09-11T00:32:43.758 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap_bytes": 9048, 2024-09-11T00:32:43.758 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap_items": 170, 2024-09-11T00:32:43.758 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap_mapping_bytes": 0, 2024-09-11T00:32:43.758 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap_mapping_items": 0, 2024-09-11T00:32:43.758 INFO:teuthology.orchestra.run.smithi178.stdout: "pgmap_bytes": 0, 2024-09-11T00:32:43.758 INFO:teuthology.orchestra.run.smithi178.stdout: "pgmap_items": 0, 2024-09-11T00:32:43.758 INFO:teuthology.orchestra.run.smithi178.stdout: "mds_co_bytes": 13783078, 2024-09-11T00:32:43.759 INFO:teuthology.orchestra.run.smithi178.stdout: "mds_co_items": 81140, 2024-09-11T00:32:43.759 INFO:teuthology.orchestra.run.smithi178.stdout: "unittest_1_bytes": 0, 2024-09-11T00:32:43.759 INFO:teuthology.orchestra.run.smithi178.stdout: "unittest_1_items": 0, 2024-09-11T00:32:43.759 INFO:teuthology.orchestra.run.smithi178.stdout: "unittest_2_bytes": 0, 2024-09-11T00:32:43.759 INFO:teuthology.orchestra.run.smithi178.stdout: "unittest_2_items": 0 2024-09-11T00:32:43.759 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T00:32:43.759 INFO:teuthology.orchestra.run.smithi178.stdout: "objecter": { 2024-09-11T00:32:43.759 INFO:teuthology.orchestra.run.smithi178.stdout: "op_active": 0, 2024-09-11T00:32:43.759 INFO:teuthology.orchestra.run.smithi178.stdout: "op_laggy": 0, 2024-09-11T00:32:43.759 INFO:teuthology.orchestra.run.smithi178.stdout: "op_send": 5608, 2024-09-11T00:32:43.760 INFO:teuthology.orchestra.run.smithi178.stdout: "op_send_bytes": 21, 2024-09-11T00:32:43.760 INFO:teuthology.orchestra.run.smithi178.stdout: "op_resend": 3, 2024-09-11T00:32:43.760 INFO:teuthology.orchestra.run.smithi178.stdout: "op_reply": 5608, 2024-09-11T00:32:43.760 INFO:teuthology.orchestra.run.smithi178.stdout: "op_latency": { 2024-09-11T00:32:43.760 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 5608, 2024-09-11T00:32:43.760 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 12.761755337, 2024-09-11T00:32:43.760 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.002275633 2024-09-11T00:32:43.760 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T00:32:43.760 INFO:teuthology.orchestra.run.smithi178.stdout: "op_inflight": 0, 2024-09-11T00:32:43.760 INFO:teuthology.orchestra.run.smithi178.stdout: "oplen_avg": { 2024-09-11T00:32:43.760 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 5608, 2024-09-11T00:32:43.760 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 5609 2024-09-11T00:32:43.760 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T00:32:43.761 INFO:teuthology.orchestra.run.smithi178.stdout: "op": 5608, 2024-09-11T00:32:43.761 INFO:teuthology.orchestra.run.smithi178.stdout: "op_r": 5608, 2024-09-11T00:32:43.761 INFO:teuthology.orchestra.run.smithi178.stdout: "op_w": 0, 2024-09-11T00:32:43.761 INFO:teuthology.orchestra.run.smithi178.stdout: "op_rmw": 0, 2024-09-11T00:32:43.761 INFO:teuthology.orchestra.run.smithi178.stdout: "op_pg": 0, 2024-09-11T00:32:43.761 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_stat": 3137, 2024-09-11T00:32:43.761 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_create": 0, 2024-09-11T00:32:43.761 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_read": 2469, 2024-09-11T00:32:43.761 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_write": 0, 2024-09-11T00:32:43.761 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_writefull": 0, 2024-09-11T00:32:43.761 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_writesame": 0, 2024-09-11T00:32:43.761 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_append": 0, 2024-09-11T00:32:43.761 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_zero": 0, 2024-09-11T00:32:43.762 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_truncate": 0, 2024-09-11T00:32:43.762 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_delete": 0, 2024-09-11T00:32:43.762 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_mapext": 0, 2024-09-11T00:32:43.762 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_sparse_read": 0, 2024-09-11T00:32:43.762 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_clonerange": 0, 2024-09-11T00:32:43.762 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_getxattr": 1, 2024-09-11T00:32:43.762 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_setxattr": 0, 2024-09-11T00:32:43.762 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_cmpxattr": 0, 2024-09-11T00:32:43.762 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_rmxattr": 0, 2024-09-11T00:32:43.762 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_resetxattrs": 0, 2024-09-11T00:32:43.762 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_call": 0, 2024-09-11T00:32:43.762 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_watch": 0, 2024-09-11T00:32:43.762 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_notify": 0, 2024-09-11T00:32:43.762 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_src_cmpxattr": 0, 2024-09-11T00:32:43.763 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_pgls": 0, 2024-09-11T00:32:43.763 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_pgls_filter": 0, 2024-09-11T00:32:43.763 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_other": 0, 2024-09-11T00:32:43.763 INFO:teuthology.orchestra.run.smithi178.stdout: "linger_active": 0, 2024-09-11T00:32:43.763 INFO:teuthology.orchestra.run.smithi178.stdout: "linger_send": 0, 2024-09-11T00:32:43.763 INFO:teuthology.orchestra.run.smithi178.stdout: "linger_resend": 0, 2024-09-11T00:32:43.763 INFO:teuthology.orchestra.run.smithi178.stdout: "linger_ping": 0, 2024-09-11T00:32:43.763 INFO:teuthology.orchestra.run.smithi178.stdout: "poolop_active": 0, 2024-09-11T00:32:43.763 INFO:teuthology.orchestra.run.smithi178.stdout: "poolop_send": 0, 2024-09-11T00:32:43.763 INFO:teuthology.orchestra.run.smithi178.stdout: "poolop_resend": 0, 2024-09-11T00:32:43.763 INFO:teuthology.orchestra.run.smithi178.stdout: "poolstat_active": 0, 2024-09-11T00:32:43.763 INFO:teuthology.orchestra.run.smithi178.stdout: "poolstat_send": 0, 2024-09-11T00:32:43.763 INFO:teuthology.orchestra.run.smithi178.stdout: "poolstat_resend": 0, 2024-09-11T00:32:43.763 INFO:teuthology.orchestra.run.smithi178.stdout: "statfs_active": 0, 2024-09-11T00:32:43.764 INFO:teuthology.orchestra.run.smithi178.stdout: "statfs_send": 0, 2024-09-11T00:32:43.764 INFO:teuthology.orchestra.run.smithi178.stdout: "statfs_resend": 0, 2024-09-11T00:32:43.764 INFO:teuthology.orchestra.run.smithi178.stdout: "command_active": 0, 2024-09-11T00:32:43.764 INFO:teuthology.orchestra.run.smithi178.stdout: "command_send": 0, 2024-09-11T00:32:43.764 INFO:teuthology.orchestra.run.smithi178.stdout: "command_resend": 0, 2024-09-11T00:32:43.764 INFO:teuthology.orchestra.run.smithi178.stdout: "map_epoch": 87, 2024-09-11T00:32:43.764 INFO:teuthology.orchestra.run.smithi178.stdout: "map_full": 0, 2024-09-11T00:32:43.764 INFO:teuthology.orchestra.run.smithi178.stdout: "map_inc": 6, 2024-09-11T00:32:43.764 INFO:teuthology.orchestra.run.smithi178.stdout: "osd_sessions": 11, 2024-09-11T00:32:43.764 INFO:teuthology.orchestra.run.smithi178.stdout: "osd_session_open": 12, 2024-09-11T00:32:43.764 INFO:teuthology.orchestra.run.smithi178.stdout: "osd_session_close": 1, 2024-09-11T00:32:43.764 INFO:teuthology.orchestra.run.smithi178.stdout: "osd_laggy": 0, 2024-09-11T00:32:43.764 INFO:teuthology.orchestra.run.smithi178.stdout: "omap_wr": 0, 2024-09-11T00:32:43.764 INFO:teuthology.orchestra.run.smithi178.stdout: "omap_rd": 2, 2024-09-11T00:32:43.765 INFO:teuthology.orchestra.run.smithi178.stdout: "omap_del": 0 2024-09-11T00:32:43.765 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T00:32:43.765 INFO:teuthology.orchestra.run.smithi178.stdout: "oft": { 2024-09-11T00:32:43.765 INFO:teuthology.orchestra.run.smithi178.stdout: "omap_total_objs": 0, 2024-09-11T00:32:43.765 INFO:teuthology.orchestra.run.smithi178.stdout: "omap_total_kv_pairs": 0, 2024-09-11T00:32:43.765 INFO:teuthology.orchestra.run.smithi178.stdout: "omap_total_updates": 0, 2024-09-11T00:32:43.765 INFO:teuthology.orchestra.run.smithi178.stdout: "omap_total_removes": 0 2024-09-11T00:32:43.765 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T00:32:43.765 INFO:teuthology.orchestra.run.smithi178.stdout: "purge_queue": { 2024-09-11T00:32:43.765 INFO:teuthology.orchestra.run.smithi178.stdout: "pq_executing_ops": 0, 2024-09-11T00:32:43.765 INFO:teuthology.orchestra.run.smithi178.stdout: "pq_executing_ops_high_water": 0, 2024-09-11T00:32:43.765 INFO:teuthology.orchestra.run.smithi178.stdout: "pq_executing": 0, 2024-09-11T00:32:43.765 INFO:teuthology.orchestra.run.smithi178.stdout: "pq_executing_high_water": 0, 2024-09-11T00:32:43.765 INFO:teuthology.orchestra.run.smithi178.stdout: "pq_executed_ops": 0, 2024-09-11T00:32:43.766 INFO:teuthology.orchestra.run.smithi178.stdout: "pq_executed": 0, 2024-09-11T00:32:43.766 INFO:teuthology.orchestra.run.smithi178.stdout: "pq_item_in_journal": 0 2024-09-11T00:32:43.766 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T00:32:43.766 INFO:teuthology.orchestra.run.smithi178.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-11T00:32:43.766 INFO:teuthology.orchestra.run.smithi178.stdout: "val": 0, 2024-09-11T00:32:43.766 INFO:teuthology.orchestra.run.smithi178.stdout: "max": 104857600, 2024-09-11T00:32:43.766 INFO:teuthology.orchestra.run.smithi178.stdout: "get_started": 0, 2024-09-11T00:32:43.766 INFO:teuthology.orchestra.run.smithi178.stdout: "get": 6171, 2024-09-11T00:32:43.766 INFO:teuthology.orchestra.run.smithi178.stdout: "get_sum": 177509637, 2024-09-11T00:32:43.766 INFO:teuthology.orchestra.run.smithi178.stdout: "get_or_fail_fail": 0, 2024-09-11T00:32:43.766 INFO:teuthology.orchestra.run.smithi178.stdout: "get_or_fail_success": 6171, 2024-09-11T00:32:43.766 INFO:teuthology.orchestra.run.smithi178.stdout: "take": 0, 2024-09-11T00:32:43.766 INFO:teuthology.orchestra.run.smithi178.stdout: "take_sum": 0, 2024-09-11T00:32:43.766 INFO:teuthology.orchestra.run.smithi178.stdout: "put": 6171, 2024-09-11T00:32:43.767 INFO:teuthology.orchestra.run.smithi178.stdout: "put_sum": 177509637, 2024-09-11T00:32:43.767 INFO:teuthology.orchestra.run.smithi178.stdout: "wait": { 2024-09-11T00:32:43.767 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-11T00:32:43.767 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-11T00:32:43.767 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-11T00:32:43.767 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-11T00:32:43.767 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T00:32:43.767 INFO:teuthology.orchestra.run.smithi178.stdout: "throttle-objecter_bytes": { 2024-09-11T00:32:43.767 INFO:teuthology.orchestra.run.smithi178.stdout: "val": 0, 2024-09-11T00:32:43.767 INFO:teuthology.orchestra.run.smithi178.stdout: "max": 104857600, 2024-09-11T00:32:43.767 INFO:teuthology.orchestra.run.smithi178.stdout: "get_started": 0, 2024-09-11T00:32:43.767 INFO:teuthology.orchestra.run.smithi178.stdout: "get": 0, 2024-09-11T00:32:43.767 INFO:teuthology.orchestra.run.smithi178.stdout: "get_sum": 0, 2024-09-11T00:32:43.768 INFO:teuthology.orchestra.run.smithi178.stdout: "get_or_fail_fail": 0, 2024-09-11T00:32:43.768 INFO:teuthology.orchestra.run.smithi178.stdout: "get_or_fail_success": 0, 2024-09-11T00:32:43.768 INFO:teuthology.orchestra.run.smithi178.stdout: "take": 5608, 2024-09-11T00:32:43.768 INFO:teuthology.orchestra.run.smithi178.stdout: "take_sum": 175351084, 2024-09-11T00:32:43.768 INFO:teuthology.orchestra.run.smithi178.stdout: "put": 5608, 2024-09-11T00:32:43.768 INFO:teuthology.orchestra.run.smithi178.stdout: "put_sum": 175351084, 2024-09-11T00:32:43.768 INFO:teuthology.orchestra.run.smithi178.stdout: "wait": { 2024-09-11T00:32:43.768 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-11T00:32:43.768 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-11T00:32:43.768 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-11T00:32:43.768 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-11T00:32:43.769 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T00:32:43.769 INFO:teuthology.orchestra.run.smithi178.stdout: "throttle-objecter_ops": { 2024-09-11T00:32:43.769 INFO:teuthology.orchestra.run.smithi178.stdout: "val": 0, 2024-09-11T00:32:43.769 INFO:teuthology.orchestra.run.smithi178.stdout: "max": 1024, 2024-09-11T00:32:43.769 INFO:teuthology.orchestra.run.smithi178.stdout: "get_started": 0, 2024-09-11T00:32:43.769 INFO:teuthology.orchestra.run.smithi178.stdout: "get": 0, 2024-09-11T00:32:43.769 INFO:teuthology.orchestra.run.smithi178.stdout: "get_sum": 0, 2024-09-11T00:32:43.769 INFO:teuthology.orchestra.run.smithi178.stdout: "get_or_fail_fail": 0, 2024-09-11T00:32:43.769 INFO:teuthology.orchestra.run.smithi178.stdout: "get_or_fail_success": 0, 2024-09-11T00:32:43.769 INFO:teuthology.orchestra.run.smithi178.stdout: "take": 5608, 2024-09-11T00:32:43.769 INFO:teuthology.orchestra.run.smithi178.stdout: "take_sum": 5608, 2024-09-11T00:32:43.770 INFO:teuthology.orchestra.run.smithi178.stdout: "put": 5608, 2024-09-11T00:32:43.770 INFO:teuthology.orchestra.run.smithi178.stdout: "put_sum": 5608, 2024-09-11T00:32:43.770 INFO:teuthology.orchestra.run.smithi178.stdout: "wait": { 2024-09-11T00:32:43.770 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-11T00:32:43.770 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-11T00:32:43.770 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-11T00:32:43.770 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-11T00:32:43.770 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T00:32:43.770 INFO:teuthology.orchestra.run.smithi178.stdout: "throttle-write_buf_throttle": { 2024-09-11T00:32:43.770 INFO:teuthology.orchestra.run.smithi178.stdout: "val": 0, 2024-09-11T00:32:43.770 INFO:teuthology.orchestra.run.smithi178.stdout: "max": 3758096384, 2024-09-11T00:32:43.770 INFO:teuthology.orchestra.run.smithi178.stdout: "get_started": 0, 2024-09-11T00:32:43.770 INFO:teuthology.orchestra.run.smithi178.stdout: "get": 0, 2024-09-11T00:32:43.771 INFO:teuthology.orchestra.run.smithi178.stdout: "get_sum": 0, 2024-09-11T00:32:43.771 INFO:teuthology.orchestra.run.smithi178.stdout: "get_or_fail_fail": 0, 2024-09-11T00:32:43.771 INFO:teuthology.orchestra.run.smithi178.stdout: "get_or_fail_success": 0, 2024-09-11T00:32:43.771 INFO:teuthology.orchestra.run.smithi178.stdout: "take": 0, 2024-09-11T00:32:43.771 INFO:teuthology.orchestra.run.smithi178.stdout: "take_sum": 0, 2024-09-11T00:32:43.771 INFO:teuthology.orchestra.run.smithi178.stdout: "put": 0, 2024-09-11T00:32:43.771 INFO:teuthology.orchestra.run.smithi178.stdout: "put_sum": 0, 2024-09-11T00:32:43.771 INFO:teuthology.orchestra.run.smithi178.stdout: "wait": { 2024-09-11T00:32:43.771 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-11T00:32:43.771 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-11T00:32:43.771 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-11T00:32:43.771 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-11T00:32:43.771 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T00:32:43.771 INFO:teuthology.orchestra.run.smithi178.stdout: "throttle-write_buf_throttle-0x563590853040": { 2024-09-11T00:32:43.772 INFO:teuthology.orchestra.run.smithi178.stdout: "val": 0, 2024-09-11T00:32:43.772 INFO:teuthology.orchestra.run.smithi178.stdout: "max": 3758096384, 2024-09-11T00:32:43.772 INFO:teuthology.orchestra.run.smithi178.stdout: "get_started": 0, 2024-09-11T00:32:43.772 INFO:teuthology.orchestra.run.smithi178.stdout: "get": 0, 2024-09-11T00:32:43.772 INFO:teuthology.orchestra.run.smithi178.stdout: "get_sum": 0, 2024-09-11T00:32:43.772 INFO:teuthology.orchestra.run.smithi178.stdout: "get_or_fail_fail": 0, 2024-09-11T00:32:43.772 INFO:teuthology.orchestra.run.smithi178.stdout: "get_or_fail_success": 0, 2024-09-11T00:32:43.772 INFO:teuthology.orchestra.run.smithi178.stdout: "take": 0, 2024-09-11T00:32:43.772 INFO:teuthology.orchestra.run.smithi178.stdout: "take_sum": 0, 2024-09-11T00:32:43.772 INFO:teuthology.orchestra.run.smithi178.stdout: "put": 0, 2024-09-11T00:32:43.772 INFO:teuthology.orchestra.run.smithi178.stdout: "put_sum": 0, 2024-09-11T00:32:43.773 INFO:teuthology.orchestra.run.smithi178.stdout: "wait": { 2024-09-11T00:32:43.773 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-11T00:32:43.773 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-11T00:32:43.773 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-11T00:32:43.773 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-11T00:32:43.773 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T00:32:43.773 INFO:teuthology.orchestra.run.smithi178.stdout: "trackedop": { 2024-09-11T00:32:43.773 INFO:teuthology.orchestra.run.smithi178.stdout: "slow_ops_count": 0 2024-09-11T00:32:43.773 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-11T00:32:43.773 INFO:teuthology.orchestra.run.smithi178.stdout:} 2024-09-11T00:32:44.282 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:32:44 smithi165 ceph-mon[35356]: pgmap v1169: 129 pgs: 129 active+clean; 671 MiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 1.2 KiB/s wr, 11 op/s 2024-09-11T00:32:44.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:32:44 smithi012 ceph-mon[30333]: pgmap v1169: 129 pgs: 129 active+clean; 671 MiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 1.2 KiB/s wr, 11 op/s 2024-09-11T00:32:44.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:32:44 smithi178 ceph-mon[35919]: pgmap v1169: 129 pgs: 129 active+clean; 671 MiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 1.2 KiB/s wr, 11 op/s 2024-09-11T00:32:44.503 INFO:tasks.check_counter:Daemon mds.f mds.exported=0 2024-09-11T00:32:44.503 INFO:tasks.check_counter:Daemon mds.f mds.imported=0 2024-09-11T00:32:44.503 DEBUG:tasks.check_counter:Getting stats from i 2024-09-11T00:32:44.504 DEBUG:teuthology.orchestra.run.smithi178:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid ff3ad74a-6fcf-11ef-bcea-c7b262605968 -- ceph daemon mds.i perf dump 2024-09-11T00:32:44.753 INFO:teuthology.orchestra.run.smithi178.stderr:Inferring config /var/lib/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/mon.c/config 2024-09-11T00:32:45.296 INFO:teuthology.orchestra.run.smithi178.stdout:{ 2024-09-11T00:32:45.296 INFO:teuthology.orchestra.run.smithi178.stdout: "AsyncMessenger::Worker-0": { 2024-09-11T00:32:45.296 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_recv_messages": 220922, 2024-09-11T00:32:45.296 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_messages": 315485, 2024-09-11T00:32:45.297 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_recv_bytes": 69617678, 2024-09-11T00:32:45.297 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_bytes": 262258051, 2024-09-11T00:32:45.297 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_created_connections": 15, 2024-09-11T00:32:45.297 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_active_connections": 5, 2024-09-11T00:32:45.297 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_total_time": 15.702373597, 2024-09-11T00:32:45.297 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_send_time": 9.331561163, 2024-09-11T00:32:45.297 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_recv_time": 7.258379233, 2024-09-11T00:32:45.297 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_fast_dispatch_time": 0.718368535, 2024-09-11T00:32:45.297 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_messages_queue_lat": { 2024-09-11T00:32:45.297 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 315483, 2024-09-11T00:32:45.297 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 12.670863675, 2024-09-11T00:32:45.297 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000040163 2024-09-11T00:32:45.298 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T00:32:45.298 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_handle_ack_lat": { 2024-09-11T00:32:45.298 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 264048, 2024-09-11T00:32:45.298 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.068458481, 2024-09-11T00:32:45.298 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000259 2024-09-11T00:32:45.298 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T00:32:45.298 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_recv_encrypted_bytes": 1584, 2024-09-11T00:32:45.298 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_encrypted_bytes": 1440 2024-09-11T00:32:45.298 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T00:32:45.298 INFO:teuthology.orchestra.run.smithi178.stdout: "AsyncMessenger::Worker-1": { 2024-09-11T00:32:45.298 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_recv_messages": 47356, 2024-09-11T00:32:45.298 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_messages": 48545, 2024-09-11T00:32:45.298 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_recv_bytes": 12056276, 2024-09-11T00:32:45.299 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_bytes": 104846829, 2024-09-11T00:32:45.299 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_created_connections": 14, 2024-09-11T00:32:45.299 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_active_connections": 9, 2024-09-11T00:32:45.299 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_total_time": 4.196762829, 2024-09-11T00:32:45.299 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_send_time": 1.872128520, 2024-09-11T00:32:45.299 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_recv_time": 1.198925738, 2024-09-11T00:32:45.299 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_fast_dispatch_time": 1.063400271, 2024-09-11T00:32:45.299 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_messages_queue_lat": { 2024-09-11T00:32:45.299 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 48544, 2024-09-11T00:32:45.299 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 2.032240211, 2024-09-11T00:32:45.299 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000041863 2024-09-11T00:32:45.299 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T00:32:45.299 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_handle_ack_lat": { 2024-09-11T00:32:45.300 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 31536, 2024-09-11T00:32:45.300 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.005509641, 2024-09-11T00:32:45.300 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000174 2024-09-11T00:32:45.300 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T00:32:45.300 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_recv_encrypted_bytes": 3104, 2024-09-11T00:32:45.300 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_encrypted_bytes": 704320 2024-09-11T00:32:45.300 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T00:32:45.300 INFO:teuthology.orchestra.run.smithi178.stdout: "AsyncMessenger::Worker-2": { 2024-09-11T00:32:45.300 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_recv_messages": 134183, 2024-09-11T00:32:45.300 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_messages": 109914, 2024-09-11T00:32:45.300 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_recv_bytes": 36892795, 2024-09-11T00:32:45.300 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_bytes": 62899163, 2024-09-11T00:32:45.300 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_created_connections": 7, 2024-09-11T00:32:45.301 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_active_connections": 6, 2024-09-11T00:32:45.301 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_total_time": 8.011082349, 2024-09-11T00:32:45.301 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_send_time": 3.731466572, 2024-09-11T00:32:45.301 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_recv_time": 4.626298546, 2024-09-11T00:32:45.301 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_fast_dispatch_time": 0.570942346, 2024-09-11T00:32:45.301 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_messages_queue_lat": { 2024-09-11T00:32:45.301 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 109913, 2024-09-11T00:32:45.301 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 3.255515110, 2024-09-11T00:32:45.301 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000029619 2024-09-11T00:32:45.301 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T00:32:45.301 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_handle_ack_lat": { 2024-09-11T00:32:45.301 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 208841, 2024-09-11T00:32:45.301 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.030250777, 2024-09-11T00:32:45.302 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000144 2024-09-11T00:32:45.302 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T00:32:45.302 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_recv_encrypted_bytes": 1200496, 2024-09-11T00:32:45.302 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_encrypted_bytes": 296144 2024-09-11T00:32:45.302 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T00:32:45.302 INFO:teuthology.orchestra.run.smithi178.stdout: "cct": { 2024-09-11T00:32:45.302 INFO:teuthology.orchestra.run.smithi178.stdout: "total_workers": 1, 2024-09-11T00:32:45.302 INFO:teuthology.orchestra.run.smithi178.stdout: "unhealthy_workers": 0 2024-09-11T00:32:45.302 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T00:32:45.302 INFO:teuthology.orchestra.run.smithi178.stdout: "finisher-MDSRank": { 2024-09-11T00:32:45.302 INFO:teuthology.orchestra.run.smithi178.stdout: "queue_len": 0, 2024-09-11T00:32:45.302 INFO:teuthology.orchestra.run.smithi178.stdout: "complete_latency": { 2024-09-11T00:32:45.302 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 56085, 2024-09-11T00:32:45.303 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 50.457296319, 2024-09-11T00:32:45.303 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000899657 2024-09-11T00:32:45.303 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-11T00:32:45.303 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T00:32:45.303 INFO:teuthology.orchestra.run.smithi178.stdout: "finisher-PurgeQueue": { 2024-09-11T00:32:45.303 INFO:teuthology.orchestra.run.smithi178.stdout: "queue_len": 0, 2024-09-11T00:32:45.303 INFO:teuthology.orchestra.run.smithi178.stdout: "complete_latency": { 2024-09-11T00:32:45.303 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 34838, 2024-09-11T00:32:45.303 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 2.504777094, 2024-09-11T00:32:45.303 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000071897 2024-09-11T00:32:45.303 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-11T00:32:45.303 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T00:32:45.303 INFO:teuthology.orchestra.run.smithi178.stdout: "mds": { 2024-09-11T00:32:45.304 INFO:teuthology.orchestra.run.smithi178.stdout: "request": 107645, 2024-09-11T00:32:45.304 INFO:teuthology.orchestra.run.smithi178.stdout: "reply": 98803, 2024-09-11T00:32:45.304 INFO:teuthology.orchestra.run.smithi178.stdout: "reply_latency": { 2024-09-11T00:32:45.304 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 98803, 2024-09-11T00:32:45.304 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 772.384088782, 2024-09-11T00:32:45.304 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.007817415 2024-09-11T00:32:45.304 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T00:32:45.304 INFO:teuthology.orchestra.run.smithi178.stdout: "slow_reply": 0, 2024-09-11T00:32:45.304 INFO:teuthology.orchestra.run.smithi178.stdout: "forward": 8842, 2024-09-11T00:32:45.304 INFO:teuthology.orchestra.run.smithi178.stdout: "dir_fetch_complete": 0, 2024-09-11T00:32:45.304 INFO:teuthology.orchestra.run.smithi178.stdout: "dir_fetch_keys": 0, 2024-09-11T00:32:45.304 INFO:teuthology.orchestra.run.smithi178.stdout: "dir_commit": 11, 2024-09-11T00:32:45.305 INFO:teuthology.orchestra.run.smithi178.stdout: "dir_split": 0, 2024-09-11T00:32:45.305 INFO:teuthology.orchestra.run.smithi178.stdout: "dir_merge": 0, 2024-09-11T00:32:45.305 INFO:teuthology.orchestra.run.smithi178.stdout: "inodes": 22, 2024-09-11T00:32:45.305 INFO:teuthology.orchestra.run.smithi178.stdout: "inodes_top": 5, 2024-09-11T00:32:45.305 INFO:teuthology.orchestra.run.smithi178.stdout: "inodes_bottom": 6, 2024-09-11T00:32:45.305 INFO:teuthology.orchestra.run.smithi178.stdout: "inodes_pin_tail": 11, 2024-09-11T00:32:45.305 INFO:teuthology.orchestra.run.smithi178.stdout: "inodes_pinned": 14, 2024-09-11T00:32:45.305 INFO:teuthology.orchestra.run.smithi178.stdout: "inodes_expired": 521, 2024-09-11T00:32:45.305 INFO:teuthology.orchestra.run.smithi178.stdout: "inodes_with_caps": 0, 2024-09-11T00:32:45.305 INFO:teuthology.orchestra.run.smithi178.stdout: "caps": 0, 2024-09-11T00:32:45.305 INFO:teuthology.orchestra.run.smithi178.stdout: "subtrees": 6, 2024-09-11T00:32:45.305 INFO:teuthology.orchestra.run.smithi178.stdout: "traverse": 182206, 2024-09-11T00:32:45.305 INFO:teuthology.orchestra.run.smithi178.stdout: "traverse_hit": 103041, 2024-09-11T00:32:45.306 INFO:teuthology.orchestra.run.smithi178.stdout: "traverse_forward": 8833, 2024-09-11T00:32:45.306 INFO:teuthology.orchestra.run.smithi178.stdout: "traverse_discover": 112, 2024-09-11T00:32:45.306 INFO:teuthology.orchestra.run.smithi178.stdout: "traverse_dir_fetch": 0, 2024-09-11T00:32:45.306 INFO:teuthology.orchestra.run.smithi178.stdout: "traverse_remote_ino": 0, 2024-09-11T00:32:45.306 INFO:teuthology.orchestra.run.smithi178.stdout: "traverse_lock": 2540, 2024-09-11T00:32:45.306 INFO:teuthology.orchestra.run.smithi178.stdout: "load_cent": 0, 2024-09-11T00:32:45.306 INFO:teuthology.orchestra.run.smithi178.stdout: "q": 0, 2024-09-11T00:32:45.306 INFO:teuthology.orchestra.run.smithi178.stdout: "exported": 17, 2024-09-11T00:32:45.306 INFO:teuthology.orchestra.run.smithi178.stdout: "exported_inodes": 198, 2024-09-11T00:32:45.306 INFO:teuthology.orchestra.run.smithi178.stdout: "imported": 17, 2024-09-11T00:32:45.306 INFO:teuthology.orchestra.run.smithi178.stdout: "imported_inodes": 469, 2024-09-11T00:32:45.306 INFO:teuthology.orchestra.run.smithi178.stdout: "openino_dir_fetch": 0, 2024-09-11T00:32:45.306 INFO:teuthology.orchestra.run.smithi178.stdout: "openino_backtrace_fetch": 0, 2024-09-11T00:32:45.306 INFO:teuthology.orchestra.run.smithi178.stdout: "openino_peer_discover": 1, 2024-09-11T00:32:45.307 INFO:teuthology.orchestra.run.smithi178.stdout: "root_rfiles": 4, 2024-09-11T00:32:45.307 INFO:teuthology.orchestra.run.smithi178.stdout: "root_rbytes": 234, 2024-09-11T00:32:45.307 INFO:teuthology.orchestra.run.smithi178.stdout: "root_rsnaps": 0, 2024-09-11T00:32:45.307 INFO:teuthology.orchestra.run.smithi178.stdout: "scrub_backtrace_fetch": 0, 2024-09-11T00:32:45.307 INFO:teuthology.orchestra.run.smithi178.stdout: "scrub_set_tag": 0, 2024-09-11T00:32:45.307 INFO:teuthology.orchestra.run.smithi178.stdout: "scrub_backtrace_repaired": 0, 2024-09-11T00:32:45.307 INFO:teuthology.orchestra.run.smithi178.stdout: "scrub_inotable_repaired": 0, 2024-09-11T00:32:45.307 INFO:teuthology.orchestra.run.smithi178.stdout: "scrub_dir_inodes": 0, 2024-09-11T00:32:45.307 INFO:teuthology.orchestra.run.smithi178.stdout: "scrub_dir_base_inodes": 0, 2024-09-11T00:32:45.307 INFO:teuthology.orchestra.run.smithi178.stdout: "scrub_dirfrag_rstats": 0, 2024-09-11T00:32:45.307 INFO:teuthology.orchestra.run.smithi178.stdout: "scrub_file_inodes": 0, 2024-09-11T00:32:45.307 INFO:teuthology.orchestra.run.smithi178.stdout: "handle_inode_file_caps": 30, 2024-09-11T00:32:45.307 INFO:teuthology.orchestra.run.smithi178.stdout: "ceph_cap_op_revoke": 15555, 2024-09-11T00:32:45.308 INFO:teuthology.orchestra.run.smithi178.stdout: "ceph_cap_op_grant": 81040, 2024-09-11T00:32:45.308 INFO:teuthology.orchestra.run.smithi178.stdout: "ceph_cap_op_trunc": 0, 2024-09-11T00:32:45.308 INFO:teuthology.orchestra.run.smithi178.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-11T00:32:45.308 INFO:teuthology.orchestra.run.smithi178.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-11T00:32:45.308 INFO:teuthology.orchestra.run.smithi178.stdout: "handle_client_caps": 53035, 2024-09-11T00:32:45.308 INFO:teuthology.orchestra.run.smithi178.stdout: "handle_client_caps_dirty": 15019, 2024-09-11T00:32:45.308 INFO:teuthology.orchestra.run.smithi178.stdout: "handle_client_cap_release": 3063, 2024-09-11T00:32:45.308 INFO:teuthology.orchestra.run.smithi178.stdout: "process_request_cap_release": 70211 2024-09-11T00:32:45.308 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T00:32:45.308 INFO:teuthology.orchestra.run.smithi178.stdout: "mds_cache": { 2024-09-11T00:32:45.308 INFO:teuthology.orchestra.run.smithi178.stdout: "dir_update": 18, 2024-09-11T00:32:45.308 INFO:teuthology.orchestra.run.smithi178.stdout: "dir_update_receipt": 223, 2024-09-11T00:32:45.308 INFO:teuthology.orchestra.run.smithi178.stdout: "dir_try_discover": 144, 2024-09-11T00:32:45.309 INFO:teuthology.orchestra.run.smithi178.stdout: "dir_send_discover": 263, 2024-09-11T00:32:45.309 INFO:teuthology.orchestra.run.smithi178.stdout: "dir_handle_discover": 73, 2024-09-11T00:32:45.309 INFO:teuthology.orchestra.run.smithi178.stdout: "num_strays": 1, 2024-09-11T00:32:45.309 INFO:teuthology.orchestra.run.smithi178.stdout: "num_strays_delayed": 0, 2024-09-11T00:32:45.309 INFO:teuthology.orchestra.run.smithi178.stdout: "num_strays_enqueuing": 0, 2024-09-11T00:32:45.309 INFO:teuthology.orchestra.run.smithi178.stdout: "strays_created": 17474, 2024-09-11T00:32:45.309 INFO:teuthology.orchestra.run.smithi178.stdout: "strays_enqueued": 17473, 2024-09-11T00:32:45.309 INFO:teuthology.orchestra.run.smithi178.stdout: "strays_reintegrated": 0, 2024-09-11T00:32:45.309 INFO:teuthology.orchestra.run.smithi178.stdout: "strays_migrated": 0, 2024-09-11T00:32:45.309 INFO:teuthology.orchestra.run.smithi178.stdout: "num_recovering_processing": 0, 2024-09-11T00:32:45.309 INFO:teuthology.orchestra.run.smithi178.stdout: "num_recovering_enqueued": 0, 2024-09-11T00:32:45.309 INFO:teuthology.orchestra.run.smithi178.stdout: "num_recovering_prioritized": 0, 2024-09-11T00:32:45.309 INFO:teuthology.orchestra.run.smithi178.stdout: "recovery_started": 0, 2024-09-11T00:32:45.309 INFO:teuthology.orchestra.run.smithi178.stdout: "recovery_completed": 0, 2024-09-11T00:32:45.310 INFO:teuthology.orchestra.run.smithi178.stdout: "ireq_quiesce_path": 19, 2024-09-11T00:32:45.310 INFO:teuthology.orchestra.run.smithi178.stdout: "ireq_quiesce_inode": 2136, 2024-09-11T00:32:45.310 INFO:teuthology.orchestra.run.smithi178.stdout: "ireq_enqueue_scrub": 0, 2024-09-11T00:32:45.310 INFO:teuthology.orchestra.run.smithi178.stdout: "ireq_exportdir": 18, 2024-09-11T00:32:45.310 INFO:teuthology.orchestra.run.smithi178.stdout: "ireq_flush": 0, 2024-09-11T00:32:45.310 INFO:teuthology.orchestra.run.smithi178.stdout: "ireq_fragmentdir": 0, 2024-09-11T00:32:45.310 INFO:teuthology.orchestra.run.smithi178.stdout: "ireq_fragstats": 0, 2024-09-11T00:32:45.310 INFO:teuthology.orchestra.run.smithi178.stdout: "ireq_inodestats": 0, 2024-09-11T00:32:45.310 INFO:teuthology.orchestra.run.smithi178.stdout: "uninline_started": 0, 2024-09-11T00:32:45.310 INFO:teuthology.orchestra.run.smithi178.stdout: "uninline_succeeded": 0, 2024-09-11T00:32:45.310 INFO:teuthology.orchestra.run.smithi178.stdout: "uninline_write_failed": 0 2024-09-11T00:32:45.310 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T00:32:45.310 INFO:teuthology.orchestra.run.smithi178.stdout: "mds_log": { 2024-09-11T00:32:45.311 INFO:teuthology.orchestra.run.smithi178.stdout: "evlrg": 0, 2024-09-11T00:32:45.311 INFO:teuthology.orchestra.run.smithi178.stdout: "evadd": 97055, 2024-09-11T00:32:45.311 INFO:teuthology.orchestra.run.smithi178.stdout: "evex": 0, 2024-09-11T00:32:45.311 INFO:teuthology.orchestra.run.smithi178.stdout: "evtrm": 0, 2024-09-11T00:32:45.311 INFO:teuthology.orchestra.run.smithi178.stdout: "ev": 97055, 2024-09-11T00:32:45.311 INFO:teuthology.orchestra.run.smithi178.stdout: "evexg": 0, 2024-09-11T00:32:45.311 INFO:teuthology.orchestra.run.smithi178.stdout: "evexd": 0, 2024-09-11T00:32:45.311 INFO:teuthology.orchestra.run.smithi178.stdout: "segadd": 123, 2024-09-11T00:32:45.311 INFO:teuthology.orchestra.run.smithi178.stdout: "segex": 0, 2024-09-11T00:32:45.311 INFO:teuthology.orchestra.run.smithi178.stdout: "segtrm": 0, 2024-09-11T00:32:45.311 INFO:teuthology.orchestra.run.smithi178.stdout: "seg": 123, 2024-09-11T00:32:45.311 INFO:teuthology.orchestra.run.smithi178.stdout: "segmjr": 9, 2024-09-11T00:32:45.311 INFO:teuthology.orchestra.run.smithi178.stdout: "segexg": 0, 2024-09-11T00:32:45.311 INFO:teuthology.orchestra.run.smithi178.stdout: "segexd": 0, 2024-09-11T00:32:45.312 INFO:teuthology.orchestra.run.smithi178.stdout: "expos": 4194304, 2024-09-11T00:32:45.312 INFO:teuthology.orchestra.run.smithi178.stdout: "wrpos": 182061053, 2024-09-11T00:32:45.312 INFO:teuthology.orchestra.run.smithi178.stdout: "rdpos": 0, 2024-09-11T00:32:45.312 INFO:teuthology.orchestra.run.smithi178.stdout: "jlat": { 2024-09-11T00:32:45.312 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 30377, 2024-09-11T00:32:45.312 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 241.184017155, 2024-09-11T00:32:45.312 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.007939691 2024-09-11T00:32:45.312 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T00:32:45.312 INFO:teuthology.orchestra.run.smithi178.stdout: "replayed": 0 2024-09-11T00:32:45.312 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T00:32:45.312 INFO:teuthology.orchestra.run.smithi178.stdout: "mds_mem": { 2024-09-11T00:32:45.312 INFO:teuthology.orchestra.run.smithi178.stdout: "ino": 27, 2024-09-11T00:32:45.312 INFO:teuthology.orchestra.run.smithi178.stdout: "ino+": 17760, 2024-09-11T00:32:45.313 INFO:teuthology.orchestra.run.smithi178.stdout: "ino-": 17733, 2024-09-11T00:32:45.313 INFO:teuthology.orchestra.run.smithi178.stdout: "dir": 26, 2024-09-11T00:32:45.313 INFO:teuthology.orchestra.run.smithi178.stdout: "dir+": 214, 2024-09-11T00:32:45.313 INFO:teuthology.orchestra.run.smithi178.stdout: "dir-": 188, 2024-09-11T00:32:45.313 INFO:teuthology.orchestra.run.smithi178.stdout: "dn": 22, 2024-09-11T00:32:45.313 INFO:teuthology.orchestra.run.smithi178.stdout: "dn+": 32219, 2024-09-11T00:32:45.313 INFO:teuthology.orchestra.run.smithi178.stdout: "dn-": 32197, 2024-09-11T00:32:45.313 INFO:teuthology.orchestra.run.smithi178.stdout: "cap": 0, 2024-09-11T00:32:45.313 INFO:teuthology.orchestra.run.smithi178.stdout: "cap+": 19525, 2024-09-11T00:32:45.313 INFO:teuthology.orchestra.run.smithi178.stdout: "cap-": 19525, 2024-09-11T00:32:45.313 INFO:teuthology.orchestra.run.smithi178.stdout: "rss": 74184, 2024-09-11T00:32:45.313 INFO:teuthology.orchestra.run.smithi178.stdout: "heap": 282976256 2024-09-11T00:32:45.313 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T00:32:45.313 INFO:teuthology.orchestra.run.smithi178.stdout: "mds_server": { 2024-09-11T00:32:45.314 INFO:teuthology.orchestra.run.smithi178.stdout: "dispatch_client_request": 158733, 2024-09-11T00:32:45.314 INFO:teuthology.orchestra.run.smithi178.stdout: "dispatch_server_request": 3418, 2024-09-11T00:32:45.314 INFO:teuthology.orchestra.run.smithi178.stdout: "handle_client_request": 107645, 2024-09-11T00:32:45.314 INFO:teuthology.orchestra.run.smithi178.stdout: "handle_client_session": 2201, 2024-09-11T00:32:45.314 INFO:teuthology.orchestra.run.smithi178.stdout: "handle_peer_request": 14412, 2024-09-11T00:32:45.314 INFO:teuthology.orchestra.run.smithi178.stdout: "req_create_latency": { 2024-09-11T00:32:45.314 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 17297, 2024-09-11T00:32:45.314 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 313.783996135, 2024-09-11T00:32:45.314 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.018140949 2024-09-11T00:32:45.314 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T00:32:45.314 INFO:teuthology.orchestra.run.smithi178.stdout: "req_getattr_latency": { 2024-09-11T00:32:45.314 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 5192, 2024-09-11T00:32:45.314 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 94.781597508, 2024-09-11T00:32:45.314 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.018255315 2024-09-11T00:32:45.315 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T00:32:45.315 INFO:teuthology.orchestra.run.smithi178.stdout: "req_getfilelock_latency": { 2024-09-11T00:32:45.315 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-11T00:32:45.315 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-11T00:32:45.315 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-11T00:32:45.315 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T00:32:45.315 INFO:teuthology.orchestra.run.smithi178.stdout: "req_link_latency": { 2024-09-11T00:32:45.315 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-11T00:32:45.315 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-11T00:32:45.315 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-11T00:32:45.315 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T00:32:45.315 INFO:teuthology.orchestra.run.smithi178.stdout: "req_lookup_latency": { 2024-09-11T00:32:45.315 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 27222, 2024-09-11T00:32:45.316 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 27.585174477, 2024-09-11T00:32:45.316 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.001013341 2024-09-11T00:32:45.316 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T00:32:45.316 INFO:teuthology.orchestra.run.smithi178.stdout: "req_lookuphash_latency": { 2024-09-11T00:32:45.316 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-11T00:32:45.316 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-11T00:32:45.316 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-11T00:32:45.316 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T00:32:45.316 INFO:teuthology.orchestra.run.smithi178.stdout: "req_lookupino_latency": { 2024-09-11T00:32:45.316 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-11T00:32:45.316 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-11T00:32:45.316 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-11T00:32:45.316 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T00:32:45.317 INFO:teuthology.orchestra.run.smithi178.stdout: "req_lookupname_latency": { 2024-09-11T00:32:45.317 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-11T00:32:45.317 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-11T00:32:45.317 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-11T00:32:45.317 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T00:32:45.317 INFO:teuthology.orchestra.run.smithi178.stdout: "req_lookupparent_latency": { 2024-09-11T00:32:45.317 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-11T00:32:45.317 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-11T00:32:45.317 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-11T00:32:45.317 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T00:32:45.317 INFO:teuthology.orchestra.run.smithi178.stdout: "req_lookupsnap_latency": { 2024-09-11T00:32:45.317 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-11T00:32:45.317 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-11T00:32:45.318 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-11T00:32:45.318 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T00:32:45.318 INFO:teuthology.orchestra.run.smithi178.stdout: "req_lssnap_latency": { 2024-09-11T00:32:45.318 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-11T00:32:45.318 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-11T00:32:45.318 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-11T00:32:45.318 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T00:32:45.318 INFO:teuthology.orchestra.run.smithi178.stdout: "req_mkdir_latency": { 2024-09-11T00:32:45.318 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-11T00:32:45.318 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-11T00:32:45.318 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-11T00:32:45.318 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T00:32:45.318 INFO:teuthology.orchestra.run.smithi178.stdout: "req_mknod_latency": { 2024-09-11T00:32:45.318 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-11T00:32:45.319 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-11T00:32:45.319 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-11T00:32:45.319 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T00:32:45.319 INFO:teuthology.orchestra.run.smithi178.stdout: "req_mksnap_latency": { 2024-09-11T00:32:45.319 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-11T00:32:45.319 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-11T00:32:45.319 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-11T00:32:45.319 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T00:32:45.319 INFO:teuthology.orchestra.run.smithi178.stdout: "req_open_latency": { 2024-09-11T00:32:45.319 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 1, 2024-09-11T00:32:45.319 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000402222, 2024-09-11T00:32:45.319 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000402222 2024-09-11T00:32:45.320 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T00:32:45.320 INFO:teuthology.orchestra.run.smithi178.stdout: "req_readdir_latency": { 2024-09-11T00:32:45.320 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 26317, 2024-09-11T00:32:45.320 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 122.975449338, 2024-09-11T00:32:45.320 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.004672852 2024-09-11T00:32:45.320 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T00:32:45.320 INFO:teuthology.orchestra.run.smithi178.stdout: "req_rename_latency": { 2024-09-11T00:32:45.320 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 4627, 2024-09-11T00:32:45.320 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 62.204795437, 2024-09-11T00:32:45.320 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.013443871 2024-09-11T00:32:45.320 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T00:32:45.320 INFO:teuthology.orchestra.run.smithi178.stdout: "req_renamesnap_latency": { 2024-09-11T00:32:45.320 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-11T00:32:45.321 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-11T00:32:45.321 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-11T00:32:45.321 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T00:32:45.321 INFO:teuthology.orchestra.run.smithi178.stdout: "req_snapdiff_latency": { 2024-09-11T00:32:45.321 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-11T00:32:45.321 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-11T00:32:45.321 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-11T00:32:45.321 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T00:32:45.321 INFO:teuthology.orchestra.run.smithi178.stdout: "req_rmdir_latency": { 2024-09-11T00:32:45.321 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 20, 2024-09-11T00:32:45.321 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.277354173, 2024-09-11T00:32:45.321 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.013867708 2024-09-11T00:32:45.321 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T00:32:45.322 INFO:teuthology.orchestra.run.smithi178.stdout: "req_rmsnap_latency": { 2024-09-11T00:32:45.322 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-11T00:32:45.322 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-11T00:32:45.322 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-11T00:32:45.322 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T00:32:45.322 INFO:teuthology.orchestra.run.smithi178.stdout: "req_rmxattr_latency": { 2024-09-11T00:32:45.322 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-11T00:32:45.322 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-11T00:32:45.322 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-11T00:32:45.322 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T00:32:45.322 INFO:teuthology.orchestra.run.smithi178.stdout: "req_setattr_latency": { 2024-09-11T00:32:45.322 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 57, 2024-09-11T00:32:45.322 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.552091915, 2024-09-11T00:32:45.322 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.009685823 2024-09-11T00:32:45.323 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T00:32:45.323 INFO:teuthology.orchestra.run.smithi178.stdout: "req_setdirlayout_latency": { 2024-09-11T00:32:45.323 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-11T00:32:45.323 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-11T00:32:45.323 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-11T00:32:45.323 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T00:32:45.323 INFO:teuthology.orchestra.run.smithi178.stdout: "req_setfilelock_latency": { 2024-09-11T00:32:45.323 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 616, 2024-09-11T00:32:45.323 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.420860320, 2024-09-11T00:32:45.323 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000683214 2024-09-11T00:32:45.323 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T00:32:45.323 INFO:teuthology.orchestra.run.smithi178.stdout: "req_setlayout_latency": { 2024-09-11T00:32:45.323 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-11T00:32:45.324 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-11T00:32:45.324 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-11T00:32:45.324 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T00:32:45.324 INFO:teuthology.orchestra.run.smithi178.stdout: "req_setxattr_latency": { 2024-09-11T00:32:45.324 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-11T00:32:45.324 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-11T00:32:45.324 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-11T00:32:45.324 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T00:32:45.324 INFO:teuthology.orchestra.run.smithi178.stdout: "req_symlink_latency": { 2024-09-11T00:32:45.324 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-11T00:32:45.324 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-11T00:32:45.324 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-11T00:32:45.324 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T00:32:45.325 INFO:teuthology.orchestra.run.smithi178.stdout: "req_unlink_latency": { 2024-09-11T00:32:45.325 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 17454, 2024-09-11T00:32:45.325 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 149.802367257, 2024-09-11T00:32:45.325 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.008582695 2024-09-11T00:32:45.325 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T00:32:45.325 INFO:teuthology.orchestra.run.smithi178.stdout: "cap_revoke_eviction": 0, 2024-09-11T00:32:45.325 INFO:teuthology.orchestra.run.smithi178.stdout: "cap_acquisition_throttle": 0, 2024-09-11T00:32:45.325 INFO:teuthology.orchestra.run.smithi178.stdout: "req_getvxattr_latency": { 2024-09-11T00:32:45.325 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-11T00:32:45.325 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-11T00:32:45.325 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-11T00:32:45.325 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-11T00:32:45.325 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T00:32:45.326 INFO:teuthology.orchestra.run.smithi178.stdout: "mds_sessions": { 2024-09-11T00:32:45.326 INFO:teuthology.orchestra.run.smithi178.stdout: "session_count": 1, 2024-09-11T00:32:45.326 INFO:teuthology.orchestra.run.smithi178.stdout: "session_add": 2, 2024-09-11T00:32:45.326 INFO:teuthology.orchestra.run.smithi178.stdout: "session_remove": 1, 2024-09-11T00:32:45.326 INFO:teuthology.orchestra.run.smithi178.stdout: "sessions_open": 1, 2024-09-11T00:32:45.326 INFO:teuthology.orchestra.run.smithi178.stdout: "sessions_stale": 0, 2024-09-11T00:32:45.326 INFO:teuthology.orchestra.run.smithi178.stdout: "total_load": 8398, 2024-09-11T00:32:45.326 INFO:teuthology.orchestra.run.smithi178.stdout: "average_load": 8398, 2024-09-11T00:32:45.326 INFO:teuthology.orchestra.run.smithi178.stdout: "avg_session_uptime": 1542, 2024-09-11T00:32:45.326 INFO:teuthology.orchestra.run.smithi178.stdout: "mdthresh_evicted": 0 2024-09-11T00:32:45.326 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T00:32:45.326 INFO:teuthology.orchestra.run.smithi178.stdout: "mempool": { 2024-09-11T00:32:45.326 INFO:teuthology.orchestra.run.smithi178.stdout: "bloom_filter_bytes": 0, 2024-09-11T00:32:45.327 INFO:teuthology.orchestra.run.smithi178.stdout: "bloom_filter_items": 0, 2024-09-11T00:32:45.327 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_alloc_bytes": 0, 2024-09-11T00:32:45.327 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_alloc_items": 0, 2024-09-11T00:32:45.327 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_cache_data_bytes": 0, 2024-09-11T00:32:45.327 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_cache_data_items": 0, 2024-09-11T00:32:45.327 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-11T00:32:45.327 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_cache_onode_items": 0, 2024-09-11T00:32:45.327 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-11T00:32:45.327 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_cache_meta_items": 0, 2024-09-11T00:32:45.327 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_cache_other_bytes": 0, 2024-09-11T00:32:45.327 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_cache_other_items": 0, 2024-09-11T00:32:45.327 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-11T00:32:45.327 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_cache_buffer_items": 0, 2024-09-11T00:32:45.328 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_extent_bytes": 0, 2024-09-11T00:32:45.328 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_extent_items": 0, 2024-09-11T00:32:45.328 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_blob_bytes": 0, 2024-09-11T00:32:45.328 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_blob_items": 0, 2024-09-11T00:32:45.328 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-11T00:32:45.328 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_shared_blob_items": 0, 2024-09-11T00:32:45.328 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-11T00:32:45.328 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_inline_bl_items": 0, 2024-09-11T00:32:45.328 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_fsck_bytes": 0, 2024-09-11T00:32:45.328 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_fsck_items": 0, 2024-09-11T00:32:45.328 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_txc_bytes": 0, 2024-09-11T00:32:45.328 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_txc_items": 0, 2024-09-11T00:32:45.328 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-11T00:32:45.328 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_writing_deferred_items": 0, 2024-09-11T00:32:45.329 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_writing_bytes": 0, 2024-09-11T00:32:45.329 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_writing_items": 0, 2024-09-11T00:32:45.329 INFO:teuthology.orchestra.run.smithi178.stdout: "bluefs_bytes": 0, 2024-09-11T00:32:45.329 INFO:teuthology.orchestra.run.smithi178.stdout: "bluefs_items": 0, 2024-09-11T00:32:45.329 INFO:teuthology.orchestra.run.smithi178.stdout: "bluefs_file_reader_bytes": 0, 2024-09-11T00:32:45.329 INFO:teuthology.orchestra.run.smithi178.stdout: "bluefs_file_reader_items": 0, 2024-09-11T00:32:45.329 INFO:teuthology.orchestra.run.smithi178.stdout: "bluefs_file_writer_bytes": 0, 2024-09-11T00:32:45.329 INFO:teuthology.orchestra.run.smithi178.stdout: "bluefs_file_writer_items": 0, 2024-09-11T00:32:45.329 INFO:teuthology.orchestra.run.smithi178.stdout: "buffer_anon_bytes": 312357, 2024-09-11T00:32:45.329 INFO:teuthology.orchestra.run.smithi178.stdout: "buffer_anon_items": 116, 2024-09-11T00:32:45.329 INFO:teuthology.orchestra.run.smithi178.stdout: "buffer_meta_bytes": 176, 2024-09-11T00:32:45.329 INFO:teuthology.orchestra.run.smithi178.stdout: "buffer_meta_items": 2, 2024-09-11T00:32:45.329 INFO:teuthology.orchestra.run.smithi178.stdout: "osd_bytes": 0, 2024-09-11T00:32:45.330 INFO:teuthology.orchestra.run.smithi178.stdout: "osd_items": 0, 2024-09-11T00:32:45.330 INFO:teuthology.orchestra.run.smithi178.stdout: "osd_mapbl_bytes": 0, 2024-09-11T00:32:45.330 INFO:teuthology.orchestra.run.smithi178.stdout: "osd_mapbl_items": 0, 2024-09-11T00:32:45.330 INFO:teuthology.orchestra.run.smithi178.stdout: "osd_pglog_bytes": 0, 2024-09-11T00:32:45.330 INFO:teuthology.orchestra.run.smithi178.stdout: "osd_pglog_items": 0, 2024-09-11T00:32:45.330 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap_bytes": 9048, 2024-09-11T00:32:45.330 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap_items": 170, 2024-09-11T00:32:45.330 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap_mapping_bytes": 0, 2024-09-11T00:32:45.330 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap_mapping_items": 0, 2024-09-11T00:32:45.330 INFO:teuthology.orchestra.run.smithi178.stdout: "pgmap_bytes": 0, 2024-09-11T00:32:45.330 INFO:teuthology.orchestra.run.smithi178.stdout: "pgmap_items": 0, 2024-09-11T00:32:45.330 INFO:teuthology.orchestra.run.smithi178.stdout: "mds_co_bytes": 112448, 2024-09-11T00:32:45.330 INFO:teuthology.orchestra.run.smithi178.stdout: "mds_co_items": 663, 2024-09-11T00:32:45.330 INFO:teuthology.orchestra.run.smithi178.stdout: "unittest_1_bytes": 0, 2024-09-11T00:32:45.331 INFO:teuthology.orchestra.run.smithi178.stdout: "unittest_1_items": 0, 2024-09-11T00:32:45.331 INFO:teuthology.orchestra.run.smithi178.stdout: "unittest_2_bytes": 0, 2024-09-11T00:32:45.331 INFO:teuthology.orchestra.run.smithi178.stdout: "unittest_2_items": 0 2024-09-11T00:32:45.331 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T00:32:45.331 INFO:teuthology.orchestra.run.smithi178.stdout: "objecter": { 2024-09-11T00:32:45.331 INFO:teuthology.orchestra.run.smithi178.stdout: "op_active": 0, 2024-09-11T00:32:45.331 INFO:teuthology.orchestra.run.smithi178.stdout: "op_laggy": 0, 2024-09-11T00:32:45.331 INFO:teuthology.orchestra.run.smithi178.stdout: "op_send": 68191, 2024-09-11T00:32:45.331 INFO:teuthology.orchestra.run.smithi178.stdout: "op_send_bytes": 181714558, 2024-09-11T00:32:45.331 INFO:teuthology.orchestra.run.smithi178.stdout: "op_resend": 33, 2024-09-11T00:32:45.331 INFO:teuthology.orchestra.run.smithi178.stdout: "op_reply": 68191, 2024-09-11T00:32:45.331 INFO:teuthology.orchestra.run.smithi178.stdout: "op_latency": { 2024-09-11T00:32:45.332 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 68191, 2024-09-11T00:32:45.332 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 745.788710623, 2024-09-11T00:32:45.332 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.010936761 2024-09-11T00:32:45.332 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T00:32:45.332 INFO:teuthology.orchestra.run.smithi178.stdout: "op_inflight": 0, 2024-09-11T00:32:45.332 INFO:teuthology.orchestra.run.smithi178.stdout: "oplen_avg": { 2024-09-11T00:32:45.332 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 68191, 2024-09-11T00:32:45.332 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 69353 2024-09-11T00:32:45.332 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T00:32:45.332 INFO:teuthology.orchestra.run.smithi178.stdout: "op": 68191, 2024-09-11T00:32:45.332 INFO:teuthology.orchestra.run.smithi178.stdout: "op_r": 766, 2024-09-11T00:32:45.332 INFO:teuthology.orchestra.run.smithi178.stdout: "op_w": 67425, 2024-09-11T00:32:45.332 INFO:teuthology.orchestra.run.smithi178.stdout: "op_rmw": 0, 2024-09-11T00:32:45.332 INFO:teuthology.orchestra.run.smithi178.stdout: "op_pg": 0, 2024-09-11T00:32:45.333 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_stat": 0, 2024-09-11T00:32:45.333 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_create": 10, 2024-09-11T00:32:45.333 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_read": 766, 2024-09-11T00:32:45.333 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_write": 31143, 2024-09-11T00:32:45.333 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_writefull": 500, 2024-09-11T00:32:45.333 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_writesame": 0, 2024-09-11T00:32:45.333 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_append": 0, 2024-09-11T00:32:45.333 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_zero": 0, 2024-09-11T00:32:45.333 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_truncate": 0, 2024-09-11T00:32:45.333 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_delete": 34981, 2024-09-11T00:32:45.333 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_mapext": 0, 2024-09-11T00:32:45.333 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_sparse_read": 0, 2024-09-11T00:32:45.333 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_clonerange": 0, 2024-09-11T00:32:45.333 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_getxattr": 0, 2024-09-11T00:32:45.334 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_setxattr": 20, 2024-09-11T00:32:45.334 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_cmpxattr": 0, 2024-09-11T00:32:45.334 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_rmxattr": 0, 2024-09-11T00:32:45.334 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_resetxattrs": 0, 2024-09-11T00:32:45.334 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_call": 0, 2024-09-11T00:32:45.334 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_watch": 0, 2024-09-11T00:32:45.334 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_notify": 0, 2024-09-11T00:32:45.334 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_src_cmpxattr": 0, 2024-09-11T00:32:45.334 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_pgls": 0, 2024-09-11T00:32:45.334 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_pgls_filter": 0, 2024-09-11T00:32:45.334 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_other": 0, 2024-09-11T00:32:45.334 INFO:teuthology.orchestra.run.smithi178.stdout: "linger_active": 0, 2024-09-11T00:32:45.334 INFO:teuthology.orchestra.run.smithi178.stdout: "linger_send": 0, 2024-09-11T00:32:45.335 INFO:teuthology.orchestra.run.smithi178.stdout: "linger_resend": 0, 2024-09-11T00:32:45.335 INFO:teuthology.orchestra.run.smithi178.stdout: "linger_ping": 0, 2024-09-11T00:32:45.335 INFO:teuthology.orchestra.run.smithi178.stdout: "poolop_active": 0, 2024-09-11T00:32:45.335 INFO:teuthology.orchestra.run.smithi178.stdout: "poolop_send": 0, 2024-09-11T00:32:45.335 INFO:teuthology.orchestra.run.smithi178.stdout: "poolop_resend": 0, 2024-09-11T00:32:45.335 INFO:teuthology.orchestra.run.smithi178.stdout: "poolstat_active": 0, 2024-09-11T00:32:45.335 INFO:teuthology.orchestra.run.smithi178.stdout: "poolstat_send": 0, 2024-09-11T00:32:45.335 INFO:teuthology.orchestra.run.smithi178.stdout: "poolstat_resend": 0, 2024-09-11T00:32:45.335 INFO:teuthology.orchestra.run.smithi178.stdout: "statfs_active": 0, 2024-09-11T00:32:45.335 INFO:teuthology.orchestra.run.smithi178.stdout: "statfs_send": 0, 2024-09-11T00:32:45.335 INFO:teuthology.orchestra.run.smithi178.stdout: "statfs_resend": 0, 2024-09-11T00:32:45.335 INFO:teuthology.orchestra.run.smithi178.stdout: "command_active": 0, 2024-09-11T00:32:45.335 INFO:teuthology.orchestra.run.smithi178.stdout: "command_send": 0, 2024-09-11T00:32:45.335 INFO:teuthology.orchestra.run.smithi178.stdout: "command_resend": 0, 2024-09-11T00:32:45.336 INFO:teuthology.orchestra.run.smithi178.stdout: "map_epoch": 87, 2024-09-11T00:32:45.336 INFO:teuthology.orchestra.run.smithi178.stdout: "map_full": 0, 2024-09-11T00:32:45.336 INFO:teuthology.orchestra.run.smithi178.stdout: "map_inc": 6, 2024-09-11T00:32:45.336 INFO:teuthology.orchestra.run.smithi178.stdout: "osd_sessions": 12, 2024-09-11T00:32:45.336 INFO:teuthology.orchestra.run.smithi178.stdout: "osd_session_open": 12, 2024-09-11T00:32:45.336 INFO:teuthology.orchestra.run.smithi178.stdout: "osd_session_close": 0, 2024-09-11T00:32:45.336 INFO:teuthology.orchestra.run.smithi178.stdout: "osd_laggy": 0, 2024-09-11T00:32:45.336 INFO:teuthology.orchestra.run.smithi178.stdout: "omap_wr": 1449, 2024-09-11T00:32:45.336 INFO:teuthology.orchestra.run.smithi178.stdout: "omap_rd": 0, 2024-09-11T00:32:45.336 INFO:teuthology.orchestra.run.smithi178.stdout: "omap_del": 484 2024-09-11T00:32:45.336 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T00:32:45.336 INFO:teuthology.orchestra.run.smithi178.stdout: "oft": { 2024-09-11T00:32:45.336 INFO:teuthology.orchestra.run.smithi178.stdout: "omap_total_objs": 1, 2024-09-11T00:32:45.337 INFO:teuthology.orchestra.run.smithi178.stdout: "omap_total_kv_pairs": 0, 2024-09-11T00:32:45.337 INFO:teuthology.orchestra.run.smithi178.stdout: "omap_total_updates": 0, 2024-09-11T00:32:45.337 INFO:teuthology.orchestra.run.smithi178.stdout: "omap_total_removes": 0 2024-09-11T00:32:45.337 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T00:32:45.337 INFO:teuthology.orchestra.run.smithi178.stdout: "purge_queue": { 2024-09-11T00:32:45.337 INFO:teuthology.orchestra.run.smithi178.stdout: "pq_executing_ops": 0, 2024-09-11T00:32:45.337 INFO:teuthology.orchestra.run.smithi178.stdout: "pq_executing_ops_high_water": 10, 2024-09-11T00:32:45.337 INFO:teuthology.orchestra.run.smithi178.stdout: "pq_executing": 0, 2024-09-11T00:32:45.337 INFO:teuthology.orchestra.run.smithi178.stdout: "pq_executing_high_water": 7, 2024-09-11T00:32:45.337 INFO:teuthology.orchestra.run.smithi178.stdout: "pq_executed_ops": 34927, 2024-09-11T00:32:45.337 INFO:teuthology.orchestra.run.smithi178.stdout: "pq_executed": 17473, 2024-09-11T00:32:45.337 INFO:teuthology.orchestra.run.smithi178.stdout: "pq_item_in_journal": 0 2024-09-11T00:32:45.337 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T00:32:45.338 INFO:teuthology.orchestra.run.smithi178.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-11T00:32:45.338 INFO:teuthology.orchestra.run.smithi178.stdout: "val": 0, 2024-09-11T00:32:45.338 INFO:teuthology.orchestra.run.smithi178.stdout: "max": 104857600, 2024-09-11T00:32:45.338 INFO:teuthology.orchestra.run.smithi178.stdout: "get_started": 0, 2024-09-11T00:32:45.338 INFO:teuthology.orchestra.run.smithi178.stdout: "get": 402457, 2024-09-11T00:32:45.338 INFO:teuthology.orchestra.run.smithi178.stdout: "get_sum": 85016604, 2024-09-11T00:32:45.338 INFO:teuthology.orchestra.run.smithi178.stdout: "get_or_fail_fail": 0, 2024-09-11T00:32:45.338 INFO:teuthology.orchestra.run.smithi178.stdout: "get_or_fail_success": 402457, 2024-09-11T00:32:45.338 INFO:teuthology.orchestra.run.smithi178.stdout: "take": 0, 2024-09-11T00:32:45.338 INFO:teuthology.orchestra.run.smithi178.stdout: "take_sum": 0, 2024-09-11T00:32:45.338 INFO:teuthology.orchestra.run.smithi178.stdout: "put": 402457, 2024-09-11T00:32:45.338 INFO:teuthology.orchestra.run.smithi178.stdout: "put_sum": 85016604, 2024-09-11T00:32:45.338 INFO:teuthology.orchestra.run.smithi178.stdout: "wait": { 2024-09-11T00:32:45.338 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-11T00:32:45.339 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-11T00:32:45.339 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-11T00:32:45.339 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-11T00:32:45.339 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T00:32:45.339 INFO:teuthology.orchestra.run.smithi178.stdout: "throttle-objecter_bytes": { 2024-09-11T00:32:45.339 INFO:teuthology.orchestra.run.smithi178.stdout: "val": 0, 2024-09-11T00:32:45.339 INFO:teuthology.orchestra.run.smithi178.stdout: "max": 104857600, 2024-09-11T00:32:45.339 INFO:teuthology.orchestra.run.smithi178.stdout: "get_started": 0, 2024-09-11T00:32:45.339 INFO:teuthology.orchestra.run.smithi178.stdout: "get": 0, 2024-09-11T00:32:45.339 INFO:teuthology.orchestra.run.smithi178.stdout: "get_sum": 0, 2024-09-11T00:32:45.339 INFO:teuthology.orchestra.run.smithi178.stdout: "get_or_fail_fail": 0, 2024-09-11T00:32:45.339 INFO:teuthology.orchestra.run.smithi178.stdout: "get_or_fail_success": 0, 2024-09-11T00:32:45.339 INFO:teuthology.orchestra.run.smithi178.stdout: "take": 68191, 2024-09-11T00:32:45.340 INFO:teuthology.orchestra.run.smithi178.stdout: "take_sum": 183618963, 2024-09-11T00:32:45.340 INFO:teuthology.orchestra.run.smithi178.stdout: "put": 68191, 2024-09-11T00:32:45.340 INFO:teuthology.orchestra.run.smithi178.stdout: "put_sum": 183618963, 2024-09-11T00:32:45.340 INFO:teuthology.orchestra.run.smithi178.stdout: "wait": { 2024-09-11T00:32:45.340 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-11T00:32:45.340 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-11T00:32:45.340 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-11T00:32:45.340 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-11T00:32:45.340 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T00:32:45.340 INFO:teuthology.orchestra.run.smithi178.stdout: "throttle-objecter_ops": { 2024-09-11T00:32:45.340 INFO:teuthology.orchestra.run.smithi178.stdout: "val": 0, 2024-09-11T00:32:45.340 INFO:teuthology.orchestra.run.smithi178.stdout: "max": 1024, 2024-09-11T00:32:45.340 INFO:teuthology.orchestra.run.smithi178.stdout: "get_started": 0, 2024-09-11T00:32:45.341 INFO:teuthology.orchestra.run.smithi178.stdout: "get": 0, 2024-09-11T00:32:45.341 INFO:teuthology.orchestra.run.smithi178.stdout: "get_sum": 0, 2024-09-11T00:32:45.341 INFO:teuthology.orchestra.run.smithi178.stdout: "get_or_fail_fail": 0, 2024-09-11T00:32:45.341 INFO:teuthology.orchestra.run.smithi178.stdout: "get_or_fail_success": 0, 2024-09-11T00:32:45.341 INFO:teuthology.orchestra.run.smithi178.stdout: "take": 68191, 2024-09-11T00:32:45.341 INFO:teuthology.orchestra.run.smithi178.stdout: "take_sum": 68191, 2024-09-11T00:32:45.341 INFO:teuthology.orchestra.run.smithi178.stdout: "put": 68191, 2024-09-11T00:32:45.341 INFO:teuthology.orchestra.run.smithi178.stdout: "put_sum": 68191, 2024-09-11T00:32:45.341 INFO:teuthology.orchestra.run.smithi178.stdout: "wait": { 2024-09-11T00:32:45.341 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-11T00:32:45.341 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-11T00:32:45.341 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-11T00:32:45.341 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-11T00:32:45.341 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T00:32:45.342 INFO:teuthology.orchestra.run.smithi178.stdout: "throttle-write_buf_throttle": { 2024-09-11T00:32:45.342 INFO:teuthology.orchestra.run.smithi178.stdout: "val": 0, 2024-09-11T00:32:45.342 INFO:teuthology.orchestra.run.smithi178.stdout: "max": 3758096384, 2024-09-11T00:32:45.342 INFO:teuthology.orchestra.run.smithi178.stdout: "get_started": 0, 2024-09-11T00:32:45.342 INFO:teuthology.orchestra.run.smithi178.stdout: "get": 17473, 2024-09-11T00:32:45.342 INFO:teuthology.orchestra.run.smithi178.stdout: "get_sum": 1904405, 2024-09-11T00:32:45.343 INFO:teuthology.orchestra.run.smithi178.stdout: "get_or_fail_fail": 0, 2024-09-11T00:32:45.343 INFO:teuthology.orchestra.run.smithi178.stdout: "get_or_fail_success": 17473, 2024-09-11T00:32:45.343 INFO:teuthology.orchestra.run.smithi178.stdout: "take": 0, 2024-09-11T00:32:45.344 INFO:teuthology.orchestra.run.smithi178.stdout: "take_sum": 0, 2024-09-11T00:32:45.344 INFO:teuthology.orchestra.run.smithi178.stdout: "put": 766, 2024-09-11T00:32:45.344 INFO:teuthology.orchestra.run.smithi178.stdout: "put_sum": 1904405, 2024-09-11T00:32:45.344 INFO:teuthology.orchestra.run.smithi178.stdout: "wait": { 2024-09-11T00:32:45.344 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-11T00:32:45.344 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-11T00:32:45.344 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-11T00:32:45.344 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-11T00:32:45.344 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T00:32:45.344 INFO:teuthology.orchestra.run.smithi178.stdout: "throttle-write_buf_throttle-0x563675bbedc0": { 2024-09-11T00:32:45.344 INFO:teuthology.orchestra.run.smithi178.stdout: "val": 0, 2024-09-11T00:32:45.344 INFO:teuthology.orchestra.run.smithi178.stdout: "max": 3758096384, 2024-09-11T00:32:45.344 INFO:teuthology.orchestra.run.smithi178.stdout: "get_started": 0, 2024-09-11T00:32:45.345 INFO:teuthology.orchestra.run.smithi178.stdout: "get": 97055, 2024-09-11T00:32:45.345 INFO:teuthology.orchestra.run.smithi178.stdout: "get_sum": 177866749, 2024-09-11T00:32:45.345 INFO:teuthology.orchestra.run.smithi178.stdout: "get_or_fail_fail": 0, 2024-09-11T00:32:45.345 INFO:teuthology.orchestra.run.smithi178.stdout: "get_or_fail_success": 97055, 2024-09-11T00:32:45.345 INFO:teuthology.orchestra.run.smithi178.stdout: "take": 0, 2024-09-11T00:32:45.345 INFO:teuthology.orchestra.run.smithi178.stdout: "take_sum": 0, 2024-09-11T00:32:45.345 INFO:teuthology.orchestra.run.smithi178.stdout: "put": 30377, 2024-09-11T00:32:45.345 INFO:teuthology.orchestra.run.smithi178.stdout: "put_sum": 177866749, 2024-09-11T00:32:45.345 INFO:teuthology.orchestra.run.smithi178.stdout: "wait": { 2024-09-11T00:32:45.345 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-11T00:32:45.345 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-11T00:32:45.345 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-11T00:32:45.345 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-11T00:32:45.345 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T00:32:45.346 INFO:teuthology.orchestra.run.smithi178.stdout: "trackedop": { 2024-09-11T00:32:45.346 INFO:teuthology.orchestra.run.smithi178.stdout: "slow_ops_count": 2057 2024-09-11T00:32:45.346 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-11T00:32:45.346 INFO:teuthology.orchestra.run.smithi178.stdout:} 2024-09-11T00:32:46.177 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:32:46 smithi178 ceph-mon[35919]: pgmap v1170: 129 pgs: 129 active+clean; 671 MiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 511 B/s wr, 9 op/s 2024-09-11T00:32:46.205 INFO:tasks.check_counter:Daemon mds.i mds.exported=17 2024-09-11T00:32:46.205 INFO:tasks.check_counter:Daemon mds.i mds.imported=17 2024-09-11T00:32:46.205 DEBUG:tasks.check_counter:Getting stats from l 2024-09-11T00:32:46.206 DEBUG:teuthology.orchestra.run.smithi178:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid ff3ad74a-6fcf-11ef-bcea-c7b262605968 -- ceph daemon mds.l perf dump 2024-09-11T00:32:46.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:32:46 smithi012 ceph-mon[30333]: pgmap v1170: 129 pgs: 129 active+clean; 671 MiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 511 B/s wr, 9 op/s 2024-09-11T00:32:46.452 INFO:teuthology.orchestra.run.smithi178.stderr:Inferring config /var/lib/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/mon.c/config 2024-09-11T00:32:46.533 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:32:46 smithi165 ceph-mon[35356]: pgmap v1170: 129 pgs: 129 active+clean; 671 MiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 511 B/s wr, 9 op/s 2024-09-11T00:32:47.095 INFO:teuthology.orchestra.run.smithi178.stdout:{ 2024-09-11T00:32:47.095 INFO:teuthology.orchestra.run.smithi178.stdout: "AsyncMessenger::Worker-0": { 2024-09-11T00:32:47.095 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_recv_messages": 3097, 2024-09-11T00:32:47.095 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_messages": 3099, 2024-09-11T00:32:47.095 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_recv_bytes": 59646291, 2024-09-11T00:32:47.095 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_bytes": 984847, 2024-09-11T00:32:47.096 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_created_connections": 8, 2024-09-11T00:32:47.096 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_active_connections": 4, 2024-09-11T00:32:47.096 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_total_time": 0.362519297, 2024-09-11T00:32:47.096 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_send_time": 0.142058449, 2024-09-11T00:32:47.096 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_recv_time": 0.110583801, 2024-09-11T00:32:47.096 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_fast_dispatch_time": 0.084960700, 2024-09-11T00:32:47.096 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_messages_queue_lat": { 2024-09-11T00:32:47.096 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 3097, 2024-09-11T00:32:47.096 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.115534083, 2024-09-11T00:32:47.096 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000037305 2024-09-11T00:32:47.097 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T00:32:47.097 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_handle_ack_lat": { 2024-09-11T00:32:47.097 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-11T00:32:47.097 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-11T00:32:47.097 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-11T00:32:47.097 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T00:32:47.097 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-11T00:32:47.097 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_encrypted_bytes": 0 2024-09-11T00:32:47.097 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T00:32:47.097 INFO:teuthology.orchestra.run.smithi178.stdout: "AsyncMessenger::Worker-1": { 2024-09-11T00:32:47.098 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_recv_messages": 1254, 2024-09-11T00:32:47.098 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_messages": 1610, 2024-09-11T00:32:47.098 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_recv_bytes": 63232794, 2024-09-11T00:32:47.098 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_bytes": 1103274, 2024-09-11T00:32:47.098 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_created_connections": 7, 2024-09-11T00:32:47.098 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_active_connections": 6, 2024-09-11T00:32:47.098 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_total_time": 0.155039665, 2024-09-11T00:32:47.098 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_send_time": 0.065416138, 2024-09-11T00:32:47.098 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_recv_time": 0.052875092, 2024-09-11T00:32:47.098 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_fast_dispatch_time": 0.026616998, 2024-09-11T00:32:47.098 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_messages_queue_lat": { 2024-09-11T00:32:47.098 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 1608, 2024-09-11T00:32:47.098 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.070443398, 2024-09-11T00:32:47.098 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000043808 2024-09-11T00:32:47.099 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T00:32:47.099 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_handle_ack_lat": { 2024-09-11T00:32:47.099 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-11T00:32:47.099 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-11T00:32:47.099 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-11T00:32:47.099 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T00:32:47.099 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_recv_encrypted_bytes": 160, 2024-09-11T00:32:47.099 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_encrypted_bytes": 704752 2024-09-11T00:32:47.099 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T00:32:47.099 INFO:teuthology.orchestra.run.smithi178.stdout: "AsyncMessenger::Worker-2": { 2024-09-11T00:32:47.099 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_recv_messages": 1977, 2024-09-11T00:32:47.099 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_messages": 1888, 2024-09-11T00:32:47.099 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_recv_bytes": 57772509, 2024-09-11T00:32:47.100 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_bytes": 739954, 2024-09-11T00:32:47.100 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_created_connections": 5, 2024-09-11T00:32:47.100 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_active_connections": 4, 2024-09-11T00:32:47.100 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_total_time": 0.236631463, 2024-09-11T00:32:47.100 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_send_time": 0.096275857, 2024-09-11T00:32:47.100 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_recv_time": 0.104283686, 2024-09-11T00:32:47.100 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_fast_dispatch_time": 0.035604406, 2024-09-11T00:32:47.100 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_messages_queue_lat": { 2024-09-11T00:32:47.100 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 1888, 2024-09-11T00:32:47.100 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.077404474, 2024-09-11T00:32:47.100 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000040998 2024-09-11T00:32:47.101 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T00:32:47.101 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_handle_ack_lat": { 2024-09-11T00:32:47.101 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-11T00:32:47.101 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-11T00:32:47.101 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-11T00:32:47.101 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T00:32:47.101 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_recv_encrypted_bytes": 1186176, 2024-09-11T00:32:47.101 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_encrypted_bytes": 292528 2024-09-11T00:32:47.101 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T00:32:47.101 INFO:teuthology.orchestra.run.smithi178.stdout: "cct": { 2024-09-11T00:32:47.102 INFO:teuthology.orchestra.run.smithi178.stdout: "total_workers": 1, 2024-09-11T00:32:47.102 INFO:teuthology.orchestra.run.smithi178.stdout: "unhealthy_workers": 0 2024-09-11T00:32:47.102 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T00:32:47.102 INFO:teuthology.orchestra.run.smithi178.stdout: "finisher-MDSRank": { 2024-09-11T00:32:47.102 INFO:teuthology.orchestra.run.smithi178.stdout: "queue_len": 0, 2024-09-11T00:32:47.102 INFO:teuthology.orchestra.run.smithi178.stdout: "complete_latency": { 2024-09-11T00:32:47.102 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 9741, 2024-09-11T00:32:47.102 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.223371716, 2024-09-11T00:32:47.102 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000022931 2024-09-11T00:32:47.102 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-11T00:32:47.102 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T00:32:47.102 INFO:teuthology.orchestra.run.smithi178.stdout: "finisher-PurgeQueue": { 2024-09-11T00:32:47.102 INFO:teuthology.orchestra.run.smithi178.stdout: "queue_len": 0, 2024-09-11T00:32:47.103 INFO:teuthology.orchestra.run.smithi178.stdout: "complete_latency": { 2024-09-11T00:32:47.103 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-11T00:32:47.103 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-11T00:32:47.103 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-11T00:32:47.103 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-11T00:32:47.103 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T00:32:47.103 INFO:teuthology.orchestra.run.smithi178.stdout: "mds": { 2024-09-11T00:32:47.103 INFO:teuthology.orchestra.run.smithi178.stdout: "request": 0, 2024-09-11T00:32:47.103 INFO:teuthology.orchestra.run.smithi178.stdout: "reply": 0, 2024-09-11T00:32:47.103 INFO:teuthology.orchestra.run.smithi178.stdout: "reply_latency": { 2024-09-11T00:32:47.103 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-11T00:32:47.103 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-11T00:32:47.103 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-11T00:32:47.103 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T00:32:47.104 INFO:teuthology.orchestra.run.smithi178.stdout: "slow_reply": 0, 2024-09-11T00:32:47.104 INFO:teuthology.orchestra.run.smithi178.stdout: "forward": 0, 2024-09-11T00:32:47.104 INFO:teuthology.orchestra.run.smithi178.stdout: "dir_fetch_complete": 0, 2024-09-11T00:32:47.104 INFO:teuthology.orchestra.run.smithi178.stdout: "dir_fetch_keys": 0, 2024-09-11T00:32:47.104 INFO:teuthology.orchestra.run.smithi178.stdout: "dir_commit": 0, 2024-09-11T00:32:47.104 INFO:teuthology.orchestra.run.smithi178.stdout: "dir_split": 0, 2024-09-11T00:32:47.104 INFO:teuthology.orchestra.run.smithi178.stdout: "dir_merge": 0, 2024-09-11T00:32:47.104 INFO:teuthology.orchestra.run.smithi178.stdout: "inodes": 17493, 2024-09-11T00:32:47.104 INFO:teuthology.orchestra.run.smithi178.stdout: "inodes_top": 0, 2024-09-11T00:32:47.104 INFO:teuthology.orchestra.run.smithi178.stdout: "inodes_bottom": 17493, 2024-09-11T00:32:47.104 INFO:teuthology.orchestra.run.smithi178.stdout: "inodes_pin_tail": 0, 2024-09-11T00:32:47.104 INFO:teuthology.orchestra.run.smithi178.stdout: "inodes_pinned": 17493, 2024-09-11T00:32:47.104 INFO:teuthology.orchestra.run.smithi178.stdout: "inodes_expired": 0, 2024-09-11T00:32:47.105 INFO:teuthology.orchestra.run.smithi178.stdout: "inodes_with_caps": 0, 2024-09-11T00:32:47.105 INFO:teuthology.orchestra.run.smithi178.stdout: "caps": 0, 2024-09-11T00:32:47.105 INFO:teuthology.orchestra.run.smithi178.stdout: "subtrees": 1, 2024-09-11T00:32:47.105 INFO:teuthology.orchestra.run.smithi178.stdout: "traverse": 0, 2024-09-11T00:32:47.105 INFO:teuthology.orchestra.run.smithi178.stdout: "traverse_hit": 0, 2024-09-11T00:32:47.105 INFO:teuthology.orchestra.run.smithi178.stdout: "traverse_forward": 0, 2024-09-11T00:32:47.105 INFO:teuthology.orchestra.run.smithi178.stdout: "traverse_discover": 0, 2024-09-11T00:32:47.105 INFO:teuthology.orchestra.run.smithi178.stdout: "traverse_dir_fetch": 0, 2024-09-11T00:32:47.105 INFO:teuthology.orchestra.run.smithi178.stdout: "traverse_remote_ino": 0, 2024-09-11T00:32:47.105 INFO:teuthology.orchestra.run.smithi178.stdout: "traverse_lock": 0, 2024-09-11T00:32:47.105 INFO:teuthology.orchestra.run.smithi178.stdout: "load_cent": 0, 2024-09-11T00:32:47.105 INFO:teuthology.orchestra.run.smithi178.stdout: "q": 0, 2024-09-11T00:32:47.106 INFO:teuthology.orchestra.run.smithi178.stdout: "exported": 0, 2024-09-11T00:32:47.106 INFO:teuthology.orchestra.run.smithi178.stdout: "exported_inodes": 0, 2024-09-11T00:32:47.106 INFO:teuthology.orchestra.run.smithi178.stdout: "imported": 0, 2024-09-11T00:32:47.106 INFO:teuthology.orchestra.run.smithi178.stdout: "imported_inodes": 0, 2024-09-11T00:32:47.106 INFO:teuthology.orchestra.run.smithi178.stdout: "openino_dir_fetch": 0, 2024-09-11T00:32:47.106 INFO:teuthology.orchestra.run.smithi178.stdout: "openino_backtrace_fetch": 0, 2024-09-11T00:32:47.106 INFO:teuthology.orchestra.run.smithi178.stdout: "openino_peer_discover": 0, 2024-09-11T00:32:47.106 INFO:teuthology.orchestra.run.smithi178.stdout: "root_rfiles": 0, 2024-09-11T00:32:47.106 INFO:teuthology.orchestra.run.smithi178.stdout: "root_rbytes": 0, 2024-09-11T00:32:47.106 INFO:teuthology.orchestra.run.smithi178.stdout: "root_rsnaps": 0, 2024-09-11T00:32:47.106 INFO:teuthology.orchestra.run.smithi178.stdout: "scrub_backtrace_fetch": 0, 2024-09-11T00:32:47.106 INFO:teuthology.orchestra.run.smithi178.stdout: "scrub_set_tag": 0, 2024-09-11T00:32:47.106 INFO:teuthology.orchestra.run.smithi178.stdout: "scrub_backtrace_repaired": 0, 2024-09-11T00:32:47.106 INFO:teuthology.orchestra.run.smithi178.stdout: "scrub_inotable_repaired": 0, 2024-09-11T00:32:47.107 INFO:teuthology.orchestra.run.smithi178.stdout: "scrub_dir_inodes": 0, 2024-09-11T00:32:47.107 INFO:teuthology.orchestra.run.smithi178.stdout: "scrub_dir_base_inodes": 0, 2024-09-11T00:32:47.107 INFO:teuthology.orchestra.run.smithi178.stdout: "scrub_dirfrag_rstats": 0, 2024-09-11T00:32:47.107 INFO:teuthology.orchestra.run.smithi178.stdout: "scrub_file_inodes": 0, 2024-09-11T00:32:47.107 INFO:teuthology.orchestra.run.smithi178.stdout: "handle_inode_file_caps": 0, 2024-09-11T00:32:47.107 INFO:teuthology.orchestra.run.smithi178.stdout: "ceph_cap_op_revoke": 0, 2024-09-11T00:32:47.107 INFO:teuthology.orchestra.run.smithi178.stdout: "ceph_cap_op_grant": 0, 2024-09-11T00:32:47.107 INFO:teuthology.orchestra.run.smithi178.stdout: "ceph_cap_op_trunc": 0, 2024-09-11T00:32:47.107 INFO:teuthology.orchestra.run.smithi178.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-11T00:32:47.107 INFO:teuthology.orchestra.run.smithi178.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-11T00:32:47.107 INFO:teuthology.orchestra.run.smithi178.stdout: "handle_client_caps": 0, 2024-09-11T00:32:47.107 INFO:teuthology.orchestra.run.smithi178.stdout: "handle_client_caps_dirty": 0, 2024-09-11T00:32:47.107 INFO:teuthology.orchestra.run.smithi178.stdout: "handle_client_cap_release": 0, 2024-09-11T00:32:47.108 INFO:teuthology.orchestra.run.smithi178.stdout: "process_request_cap_release": 0 2024-09-11T00:32:47.108 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T00:32:47.108 INFO:teuthology.orchestra.run.smithi178.stdout: "mds_cache": { 2024-09-11T00:32:47.108 INFO:teuthology.orchestra.run.smithi178.stdout: "dir_update": 0, 2024-09-11T00:32:47.108 INFO:teuthology.orchestra.run.smithi178.stdout: "dir_update_receipt": 0, 2024-09-11T00:32:47.108 INFO:teuthology.orchestra.run.smithi178.stdout: "dir_try_discover": 0, 2024-09-11T00:32:47.108 INFO:teuthology.orchestra.run.smithi178.stdout: "dir_send_discover": 0, 2024-09-11T00:32:47.108 INFO:teuthology.orchestra.run.smithi178.stdout: "dir_handle_discover": 0, 2024-09-11T00:32:47.108 INFO:teuthology.orchestra.run.smithi178.stdout: "num_strays": 0, 2024-09-11T00:32:47.108 INFO:teuthology.orchestra.run.smithi178.stdout: "num_strays_delayed": 0, 2024-09-11T00:32:47.108 INFO:teuthology.orchestra.run.smithi178.stdout: "num_strays_enqueuing": 0, 2024-09-11T00:32:47.108 INFO:teuthology.orchestra.run.smithi178.stdout: "strays_created": 0, 2024-09-11T00:32:47.108 INFO:teuthology.orchestra.run.smithi178.stdout: "strays_enqueued": 0, 2024-09-11T00:32:47.108 INFO:teuthology.orchestra.run.smithi178.stdout: "strays_reintegrated": 0, 2024-09-11T00:32:47.109 INFO:teuthology.orchestra.run.smithi178.stdout: "strays_migrated": 0, 2024-09-11T00:32:47.109 INFO:teuthology.orchestra.run.smithi178.stdout: "num_recovering_processing": 0, 2024-09-11T00:32:47.109 INFO:teuthology.orchestra.run.smithi178.stdout: "num_recovering_enqueued": 0, 2024-09-11T00:32:47.109 INFO:teuthology.orchestra.run.smithi178.stdout: "num_recovering_prioritized": 0, 2024-09-11T00:32:47.109 INFO:teuthology.orchestra.run.smithi178.stdout: "recovery_started": 0, 2024-09-11T00:32:47.109 INFO:teuthology.orchestra.run.smithi178.stdout: "recovery_completed": 0, 2024-09-11T00:32:47.109 INFO:teuthology.orchestra.run.smithi178.stdout: "ireq_quiesce_path": 0, 2024-09-11T00:32:47.109 INFO:teuthology.orchestra.run.smithi178.stdout: "ireq_quiesce_inode": 0, 2024-09-11T00:32:47.109 INFO:teuthology.orchestra.run.smithi178.stdout: "ireq_enqueue_scrub": 0, 2024-09-11T00:32:47.109 INFO:teuthology.orchestra.run.smithi178.stdout: "ireq_exportdir": 0, 2024-09-11T00:32:47.109 INFO:teuthology.orchestra.run.smithi178.stdout: "ireq_flush": 0, 2024-09-11T00:32:47.110 INFO:teuthology.orchestra.run.smithi178.stdout: "ireq_fragmentdir": 0, 2024-09-11T00:32:47.110 INFO:teuthology.orchestra.run.smithi178.stdout: "ireq_fragstats": 0, 2024-09-11T00:32:47.110 INFO:teuthology.orchestra.run.smithi178.stdout: "ireq_inodestats": 0, 2024-09-11T00:32:47.110 INFO:teuthology.orchestra.run.smithi178.stdout: "uninline_started": 0, 2024-09-11T00:32:47.110 INFO:teuthology.orchestra.run.smithi178.stdout: "uninline_succeeded": 0, 2024-09-11T00:32:47.110 INFO:teuthology.orchestra.run.smithi178.stdout: "uninline_write_failed": 0 2024-09-11T00:32:47.110 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T00:32:47.110 INFO:teuthology.orchestra.run.smithi178.stdout: "mds_log": { 2024-09-11T00:32:47.110 INFO:teuthology.orchestra.run.smithi178.stdout: "evlrg": 0, 2024-09-11T00:32:47.110 INFO:teuthology.orchestra.run.smithi178.stdout: "evadd": 0, 2024-09-11T00:32:47.110 INFO:teuthology.orchestra.run.smithi178.stdout: "evex": 0, 2024-09-11T00:32:47.110 INFO:teuthology.orchestra.run.smithi178.stdout: "evtrm": 0, 2024-09-11T00:32:47.110 INFO:teuthology.orchestra.run.smithi178.stdout: "ev": 97055, 2024-09-11T00:32:47.110 INFO:teuthology.orchestra.run.smithi178.stdout: "evexg": 0, 2024-09-11T00:32:47.111 INFO:teuthology.orchestra.run.smithi178.stdout: "evexd": 0, 2024-09-11T00:32:47.111 INFO:teuthology.orchestra.run.smithi178.stdout: "segadd": 0, 2024-09-11T00:32:47.111 INFO:teuthology.orchestra.run.smithi178.stdout: "segex": 0, 2024-09-11T00:32:47.111 INFO:teuthology.orchestra.run.smithi178.stdout: "segtrm": 0, 2024-09-11T00:32:47.111 INFO:teuthology.orchestra.run.smithi178.stdout: "seg": 123, 2024-09-11T00:32:47.111 INFO:teuthology.orchestra.run.smithi178.stdout: "segmjr": 9, 2024-09-11T00:32:47.111 INFO:teuthology.orchestra.run.smithi178.stdout: "segexg": 0, 2024-09-11T00:32:47.111 INFO:teuthology.orchestra.run.smithi178.stdout: "segexd": 0, 2024-09-11T00:32:47.111 INFO:teuthology.orchestra.run.smithi178.stdout: "expos": 4194304, 2024-09-11T00:32:47.111 INFO:teuthology.orchestra.run.smithi178.stdout: "wrpos": 182061053, 2024-09-11T00:32:47.111 INFO:teuthology.orchestra.run.smithi178.stdout: "rdpos": 182059915, 2024-09-11T00:32:47.111 INFO:teuthology.orchestra.run.smithi178.stdout: "jlat": { 2024-09-11T00:32:47.111 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-11T00:32:47.111 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-11T00:32:47.112 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-11T00:32:47.112 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T00:32:47.112 INFO:teuthology.orchestra.run.smithi178.stdout: "replayed": 97055 2024-09-11T00:32:47.112 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T00:32:47.112 INFO:teuthology.orchestra.run.smithi178.stdout: "mds_mem": { 2024-09-11T00:32:47.112 INFO:teuthology.orchestra.run.smithi178.stdout: "ino": 14, 2024-09-11T00:32:47.112 INFO:teuthology.orchestra.run.smithi178.stdout: "ino+": 17585, 2024-09-11T00:32:47.112 INFO:teuthology.orchestra.run.smithi178.stdout: "ino-": 17571, 2024-09-11T00:32:47.112 INFO:teuthology.orchestra.run.smithi178.stdout: "dir": 12, 2024-09-11T00:32:47.112 INFO:teuthology.orchestra.run.smithi178.stdout: "dir+": 70, 2024-09-11T00:32:47.112 INFO:teuthology.orchestra.run.smithi178.stdout: "dir-": 58, 2024-09-11T00:32:47.112 INFO:teuthology.orchestra.run.smithi178.stdout: "dn": 17493, 2024-09-11T00:32:47.112 INFO:teuthology.orchestra.run.smithi178.stdout: "dn+": 17997, 2024-09-11T00:32:47.112 INFO:teuthology.orchestra.run.smithi178.stdout: "dn-": 504, 2024-09-11T00:32:47.113 INFO:teuthology.orchestra.run.smithi178.stdout: "cap": 0, 2024-09-11T00:32:47.113 INFO:teuthology.orchestra.run.smithi178.stdout: "cap+": 0, 2024-09-11T00:32:47.113 INFO:teuthology.orchestra.run.smithi178.stdout: "cap-": 0, 2024-09-11T00:32:47.113 INFO:teuthology.orchestra.run.smithi178.stdout: "rss": 67816, 2024-09-11T00:32:47.113 INFO:teuthology.orchestra.run.smithi178.stdout: "heap": 269348864 2024-09-11T00:32:47.113 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T00:32:47.113 INFO:teuthology.orchestra.run.smithi178.stdout: "mds_server": { 2024-09-11T00:32:47.113 INFO:teuthology.orchestra.run.smithi178.stdout: "dispatch_client_request": 0, 2024-09-11T00:32:47.113 INFO:teuthology.orchestra.run.smithi178.stdout: "dispatch_server_request": 0, 2024-09-11T00:32:47.113 INFO:teuthology.orchestra.run.smithi178.stdout: "handle_client_request": 0, 2024-09-11T00:32:47.113 INFO:teuthology.orchestra.run.smithi178.stdout: "handle_client_session": 0, 2024-09-11T00:32:47.113 INFO:teuthology.orchestra.run.smithi178.stdout: "handle_peer_request": 0, 2024-09-11T00:32:47.113 INFO:teuthology.orchestra.run.smithi178.stdout: "req_create_latency": { 2024-09-11T00:32:47.114 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-11T00:32:47.114 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-11T00:32:47.114 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-11T00:32:47.114 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T00:32:47.114 INFO:teuthology.orchestra.run.smithi178.stdout: "req_getattr_latency": { 2024-09-11T00:32:47.114 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-11T00:32:47.114 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-11T00:32:47.114 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-11T00:32:47.114 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T00:32:47.114 INFO:teuthology.orchestra.run.smithi178.stdout: "req_getfilelock_latency": { 2024-09-11T00:32:47.114 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-11T00:32:47.114 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-11T00:32:47.114 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-11T00:32:47.115 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T00:32:47.115 INFO:teuthology.orchestra.run.smithi178.stdout: "req_link_latency": { 2024-09-11T00:32:47.115 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-11T00:32:47.115 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-11T00:32:47.115 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-11T00:32:47.115 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T00:32:47.115 INFO:teuthology.orchestra.run.smithi178.stdout: "req_lookup_latency": { 2024-09-11T00:32:47.115 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-11T00:32:47.115 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-11T00:32:47.115 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-11T00:32:47.115 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T00:32:47.115 INFO:teuthology.orchestra.run.smithi178.stdout: "req_lookuphash_latency": { 2024-09-11T00:32:47.115 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-11T00:32:47.115 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-11T00:32:47.116 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-11T00:32:47.116 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T00:32:47.116 INFO:teuthology.orchestra.run.smithi178.stdout: "req_lookupino_latency": { 2024-09-11T00:32:47.116 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-11T00:32:47.116 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-11T00:32:47.116 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-11T00:32:47.116 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T00:32:47.116 INFO:teuthology.orchestra.run.smithi178.stdout: "req_lookupname_latency": { 2024-09-11T00:32:47.116 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-11T00:32:47.116 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-11T00:32:47.116 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-11T00:32:47.116 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T00:32:47.116 INFO:teuthology.orchestra.run.smithi178.stdout: "req_lookupparent_latency": { 2024-09-11T00:32:47.117 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-11T00:32:47.117 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-11T00:32:47.117 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-11T00:32:47.117 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T00:32:47.117 INFO:teuthology.orchestra.run.smithi178.stdout: "req_lookupsnap_latency": { 2024-09-11T00:32:47.117 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-11T00:32:47.117 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-11T00:32:47.117 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-11T00:32:47.117 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T00:32:47.117 INFO:teuthology.orchestra.run.smithi178.stdout: "req_lssnap_latency": { 2024-09-11T00:32:47.118 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-11T00:32:47.118 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-11T00:32:47.118 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-11T00:32:47.118 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T00:32:47.118 INFO:teuthology.orchestra.run.smithi178.stdout: "req_mkdir_latency": { 2024-09-11T00:32:47.118 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-11T00:32:47.119 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-11T00:32:47.119 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-11T00:32:47.119 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T00:32:47.119 INFO:teuthology.orchestra.run.smithi178.stdout: "req_mknod_latency": { 2024-09-11T00:32:47.119 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-11T00:32:47.119 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-11T00:32:47.119 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-11T00:32:47.119 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T00:32:47.119 INFO:teuthology.orchestra.run.smithi178.stdout: "req_mksnap_latency": { 2024-09-11T00:32:47.119 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-11T00:32:47.119 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-11T00:32:47.119 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-11T00:32:47.119 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T00:32:47.120 INFO:teuthology.orchestra.run.smithi178.stdout: "req_open_latency": { 2024-09-11T00:32:47.120 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-11T00:32:47.120 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-11T00:32:47.120 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-11T00:32:47.120 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T00:32:47.120 INFO:teuthology.orchestra.run.smithi178.stdout: "req_readdir_latency": { 2024-09-11T00:32:47.120 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-11T00:32:47.120 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-11T00:32:47.120 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-11T00:32:47.120 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T00:32:47.120 INFO:teuthology.orchestra.run.smithi178.stdout: "req_rename_latency": { 2024-09-11T00:32:47.120 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-11T00:32:47.120 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-11T00:32:47.121 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-11T00:32:47.121 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T00:32:47.121 INFO:teuthology.orchestra.run.smithi178.stdout: "req_renamesnap_latency": { 2024-09-11T00:32:47.121 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-11T00:32:47.121 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-11T00:32:47.121 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-11T00:32:47.121 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T00:32:47.121 INFO:teuthology.orchestra.run.smithi178.stdout: "req_snapdiff_latency": { 2024-09-11T00:32:47.121 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-11T00:32:47.121 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-11T00:32:47.121 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-11T00:32:47.121 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T00:32:47.121 INFO:teuthology.orchestra.run.smithi178.stdout: "req_rmdir_latency": { 2024-09-11T00:32:47.122 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-11T00:32:47.122 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-11T00:32:47.122 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-11T00:32:47.122 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T00:32:47.122 INFO:teuthology.orchestra.run.smithi178.stdout: "req_rmsnap_latency": { 2024-09-11T00:32:47.122 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-11T00:32:47.122 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-11T00:32:47.122 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-11T00:32:47.122 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T00:32:47.122 INFO:teuthology.orchestra.run.smithi178.stdout: "req_rmxattr_latency": { 2024-09-11T00:32:47.123 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-11T00:32:47.123 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-11T00:32:47.123 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-11T00:32:47.123 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T00:32:47.123 INFO:teuthology.orchestra.run.smithi178.stdout: "req_setattr_latency": { 2024-09-11T00:32:47.123 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-11T00:32:47.123 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-11T00:32:47.123 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-11T00:32:47.123 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T00:32:47.123 INFO:teuthology.orchestra.run.smithi178.stdout: "req_setdirlayout_latency": { 2024-09-11T00:32:47.123 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-11T00:32:47.123 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-11T00:32:47.124 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-11T00:32:47.124 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T00:32:47.124 INFO:teuthology.orchestra.run.smithi178.stdout: "req_setfilelock_latency": { 2024-09-11T00:32:47.124 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-11T00:32:47.124 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-11T00:32:47.124 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-11T00:32:47.124 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T00:32:47.124 INFO:teuthology.orchestra.run.smithi178.stdout: "req_setlayout_latency": { 2024-09-11T00:32:47.124 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-11T00:32:47.124 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-11T00:32:47.124 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-11T00:32:47.124 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T00:32:47.125 INFO:teuthology.orchestra.run.smithi178.stdout: "req_setxattr_latency": { 2024-09-11T00:32:47.125 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-11T00:32:47.125 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-11T00:32:47.125 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-11T00:32:47.125 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T00:32:47.125 INFO:teuthology.orchestra.run.smithi178.stdout: "req_symlink_latency": { 2024-09-11T00:32:47.125 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-11T00:32:47.125 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-11T00:32:47.125 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-11T00:32:47.125 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T00:32:47.125 INFO:teuthology.orchestra.run.smithi178.stdout: "req_unlink_latency": { 2024-09-11T00:32:47.125 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-11T00:32:47.125 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-11T00:32:47.126 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-11T00:32:47.126 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T00:32:47.126 INFO:teuthology.orchestra.run.smithi178.stdout: "cap_revoke_eviction": 0, 2024-09-11T00:32:47.126 INFO:teuthology.orchestra.run.smithi178.stdout: "cap_acquisition_throttle": 0, 2024-09-11T00:32:47.126 INFO:teuthology.orchestra.run.smithi178.stdout: "req_getvxattr_latency": { 2024-09-11T00:32:47.126 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-11T00:32:47.126 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-11T00:32:47.126 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-11T00:32:47.126 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-11T00:32:47.126 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T00:32:47.126 INFO:teuthology.orchestra.run.smithi178.stdout: "mds_sessions": { 2024-09-11T00:32:47.126 INFO:teuthology.orchestra.run.smithi178.stdout: "session_count": 1, 2024-09-11T00:32:47.127 INFO:teuthology.orchestra.run.smithi178.stdout: "session_add": 2, 2024-09-11T00:32:47.127 INFO:teuthology.orchestra.run.smithi178.stdout: "session_remove": 1, 2024-09-11T00:32:47.127 INFO:teuthology.orchestra.run.smithi178.stdout: "sessions_open": 1, 2024-09-11T00:32:47.127 INFO:teuthology.orchestra.run.smithi178.stdout: "sessions_stale": 0, 2024-09-11T00:32:47.127 INFO:teuthology.orchestra.run.smithi178.stdout: "total_load": 0, 2024-09-11T00:32:47.127 INFO:teuthology.orchestra.run.smithi178.stdout: "average_load": 0, 2024-09-11T00:32:47.127 INFO:teuthology.orchestra.run.smithi178.stdout: "avg_session_uptime": 2950, 2024-09-11T00:32:47.127 INFO:teuthology.orchestra.run.smithi178.stdout: "mdthresh_evicted": 0 2024-09-11T00:32:47.127 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T00:32:47.127 INFO:teuthology.orchestra.run.smithi178.stdout: "mempool": { 2024-09-11T00:32:47.127 INFO:teuthology.orchestra.run.smithi178.stdout: "bloom_filter_bytes": 0, 2024-09-11T00:32:47.127 INFO:teuthology.orchestra.run.smithi178.stdout: "bloom_filter_items": 0, 2024-09-11T00:32:47.128 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_alloc_bytes": 0, 2024-09-11T00:32:47.128 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_alloc_items": 0, 2024-09-11T00:32:47.128 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_cache_data_bytes": 0, 2024-09-11T00:32:47.128 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_cache_data_items": 0, 2024-09-11T00:32:47.128 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-11T00:32:47.128 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_cache_onode_items": 0, 2024-09-11T00:32:47.128 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-11T00:32:47.128 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_cache_meta_items": 0, 2024-09-11T00:32:47.128 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_cache_other_bytes": 0, 2024-09-11T00:32:47.128 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_cache_other_items": 0, 2024-09-11T00:32:47.129 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-11T00:32:47.129 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_cache_buffer_items": 0, 2024-09-11T00:32:47.129 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_extent_bytes": 0, 2024-09-11T00:32:47.129 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_extent_items": 0, 2024-09-11T00:32:47.129 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_blob_bytes": 0, 2024-09-11T00:32:47.129 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_blob_items": 0, 2024-09-11T00:32:47.129 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-11T00:32:47.129 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_shared_blob_items": 0, 2024-09-11T00:32:47.129 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-11T00:32:47.129 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_inline_bl_items": 0, 2024-09-11T00:32:47.130 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_fsck_bytes": 0, 2024-09-11T00:32:47.130 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_fsck_items": 0, 2024-09-11T00:32:47.130 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_txc_bytes": 0, 2024-09-11T00:32:47.130 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_txc_items": 0, 2024-09-11T00:32:47.130 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-11T00:32:47.130 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_writing_deferred_items": 0, 2024-09-11T00:32:47.130 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_writing_bytes": 0, 2024-09-11T00:32:47.130 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_writing_items": 0, 2024-09-11T00:32:47.131 INFO:teuthology.orchestra.run.smithi178.stdout: "bluefs_bytes": 0, 2024-09-11T00:32:47.131 INFO:teuthology.orchestra.run.smithi178.stdout: "bluefs_items": 0, 2024-09-11T00:32:47.131 INFO:teuthology.orchestra.run.smithi178.stdout: "bluefs_file_reader_bytes": 0, 2024-09-11T00:32:47.131 INFO:teuthology.orchestra.run.smithi178.stdout: "bluefs_file_reader_items": 0, 2024-09-11T00:32:47.131 INFO:teuthology.orchestra.run.smithi178.stdout: "bluefs_file_writer_bytes": 0, 2024-09-11T00:32:47.131 INFO:teuthology.orchestra.run.smithi178.stdout: "bluefs_file_writer_items": 0, 2024-09-11T00:32:47.131 INFO:teuthology.orchestra.run.smithi178.stdout: "buffer_anon_bytes": 242638, 2024-09-11T00:32:47.131 INFO:teuthology.orchestra.run.smithi178.stdout: "buffer_anon_items": 89, 2024-09-11T00:32:47.132 INFO:teuthology.orchestra.run.smithi178.stdout: "buffer_meta_bytes": 0, 2024-09-11T00:32:47.132 INFO:teuthology.orchestra.run.smithi178.stdout: "buffer_meta_items": 0, 2024-09-11T00:32:47.132 INFO:teuthology.orchestra.run.smithi178.stdout: "osd_bytes": 0, 2024-09-11T00:32:47.132 INFO:teuthology.orchestra.run.smithi178.stdout: "osd_items": 0, 2024-09-11T00:32:47.132 INFO:teuthology.orchestra.run.smithi178.stdout: "osd_mapbl_bytes": 0, 2024-09-11T00:32:47.132 INFO:teuthology.orchestra.run.smithi178.stdout: "osd_mapbl_items": 0, 2024-09-11T00:32:47.132 INFO:teuthology.orchestra.run.smithi178.stdout: "osd_pglog_bytes": 0, 2024-09-11T00:32:47.132 INFO:teuthology.orchestra.run.smithi178.stdout: "osd_pglog_items": 0, 2024-09-11T00:32:47.132 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap_bytes": 9048, 2024-09-11T00:32:47.133 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap_items": 170, 2024-09-11T00:32:47.133 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap_mapping_bytes": 0, 2024-09-11T00:32:47.133 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap_mapping_items": 0, 2024-09-11T00:32:47.133 INFO:teuthology.orchestra.run.smithi178.stdout: "pgmap_bytes": 0, 2024-09-11T00:32:47.133 INFO:teuthology.orchestra.run.smithi178.stdout: "pgmap_items": 0, 2024-09-11T00:32:47.133 INFO:teuthology.orchestra.run.smithi178.stdout: "mds_co_bytes": 11943235, 2024-09-11T00:32:47.133 INFO:teuthology.orchestra.run.smithi178.stdout: "mds_co_items": 70220, 2024-09-11T00:32:47.133 INFO:teuthology.orchestra.run.smithi178.stdout: "unittest_1_bytes": 0, 2024-09-11T00:32:47.133 INFO:teuthology.orchestra.run.smithi178.stdout: "unittest_1_items": 0, 2024-09-11T00:32:47.133 INFO:teuthology.orchestra.run.smithi178.stdout: "unittest_2_bytes": 0, 2024-09-11T00:32:47.133 INFO:teuthology.orchestra.run.smithi178.stdout: "unittest_2_items": 0 2024-09-11T00:32:47.133 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T00:32:47.133 INFO:teuthology.orchestra.run.smithi178.stdout: "objecter": { 2024-09-11T00:32:47.134 INFO:teuthology.orchestra.run.smithi178.stdout: "op_active": 0, 2024-09-11T00:32:47.134 INFO:teuthology.orchestra.run.smithi178.stdout: "op_laggy": 0, 2024-09-11T00:32:47.134 INFO:teuthology.orchestra.run.smithi178.stdout: "op_send": 5757, 2024-09-11T00:32:47.134 INFO:teuthology.orchestra.run.smithi178.stdout: "op_send_bytes": 21, 2024-09-11T00:32:47.134 INFO:teuthology.orchestra.run.smithi178.stdout: "op_resend": 3, 2024-09-11T00:32:47.134 INFO:teuthology.orchestra.run.smithi178.stdout: "op_reply": 5757, 2024-09-11T00:32:47.134 INFO:teuthology.orchestra.run.smithi178.stdout: "op_latency": { 2024-09-11T00:32:47.134 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 5757, 2024-09-11T00:32:47.134 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 15.531702857, 2024-09-11T00:32:47.134 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.002697881 2024-09-11T00:32:47.134 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T00:32:47.134 INFO:teuthology.orchestra.run.smithi178.stdout: "op_inflight": 0, 2024-09-11T00:32:47.135 INFO:teuthology.orchestra.run.smithi178.stdout: "oplen_avg": { 2024-09-11T00:32:47.135 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 5757, 2024-09-11T00:32:47.135 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 5758 2024-09-11T00:32:47.135 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T00:32:47.135 INFO:teuthology.orchestra.run.smithi178.stdout: "op": 5757, 2024-09-11T00:32:47.135 INFO:teuthology.orchestra.run.smithi178.stdout: "op_r": 5757, 2024-09-11T00:32:47.135 INFO:teuthology.orchestra.run.smithi178.stdout: "op_w": 0, 2024-09-11T00:32:47.135 INFO:teuthology.orchestra.run.smithi178.stdout: "op_rmw": 0, 2024-09-11T00:32:47.135 INFO:teuthology.orchestra.run.smithi178.stdout: "op_pg": 0, 2024-09-11T00:32:47.135 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_stat": 3241, 2024-09-11T00:32:47.136 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_create": 0, 2024-09-11T00:32:47.136 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_read": 2514, 2024-09-11T00:32:47.136 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_write": 0, 2024-09-11T00:32:47.136 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_writefull": 0, 2024-09-11T00:32:47.136 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_writesame": 0, 2024-09-11T00:32:47.136 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_append": 0, 2024-09-11T00:32:47.136 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_zero": 0, 2024-09-11T00:32:47.136 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_truncate": 0, 2024-09-11T00:32:47.136 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_delete": 0, 2024-09-11T00:32:47.136 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_mapext": 0, 2024-09-11T00:32:47.136 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_sparse_read": 0, 2024-09-11T00:32:47.136 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_clonerange": 0, 2024-09-11T00:32:47.136 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_getxattr": 1, 2024-09-11T00:32:47.137 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_setxattr": 0, 2024-09-11T00:32:47.137 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_cmpxattr": 0, 2024-09-11T00:32:47.137 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_rmxattr": 0, 2024-09-11T00:32:47.137 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_resetxattrs": 0, 2024-09-11T00:32:47.137 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_call": 0, 2024-09-11T00:32:47.137 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_watch": 0, 2024-09-11T00:32:47.137 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_notify": 0, 2024-09-11T00:32:47.137 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_src_cmpxattr": 0, 2024-09-11T00:32:47.137 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_pgls": 0, 2024-09-11T00:32:47.137 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_pgls_filter": 0, 2024-09-11T00:32:47.137 INFO:teuthology.orchestra.run.smithi178.stdout: "osdop_other": 0, 2024-09-11T00:32:47.137 INFO:teuthology.orchestra.run.smithi178.stdout: "linger_active": 0, 2024-09-11T00:32:47.138 INFO:teuthology.orchestra.run.smithi178.stdout: "linger_send": 0, 2024-09-11T00:32:47.138 INFO:teuthology.orchestra.run.smithi178.stdout: "linger_resend": 0, 2024-09-11T00:32:47.138 INFO:teuthology.orchestra.run.smithi178.stdout: "linger_ping": 0, 2024-09-11T00:32:47.138 INFO:teuthology.orchestra.run.smithi178.stdout: "poolop_active": 0, 2024-09-11T00:32:47.138 INFO:teuthology.orchestra.run.smithi178.stdout: "poolop_send": 0, 2024-09-11T00:32:47.138 INFO:teuthology.orchestra.run.smithi178.stdout: "poolop_resend": 0, 2024-09-11T00:32:47.138 INFO:teuthology.orchestra.run.smithi178.stdout: "poolstat_active": 0, 2024-09-11T00:32:47.138 INFO:teuthology.orchestra.run.smithi178.stdout: "poolstat_send": 0, 2024-09-11T00:32:47.138 INFO:teuthology.orchestra.run.smithi178.stdout: "poolstat_resend": 0, 2024-09-11T00:32:47.138 INFO:teuthology.orchestra.run.smithi178.stdout: "statfs_active": 0, 2024-09-11T00:32:47.138 INFO:teuthology.orchestra.run.smithi178.stdout: "statfs_send": 0, 2024-09-11T00:32:47.138 INFO:teuthology.orchestra.run.smithi178.stdout: "statfs_resend": 0, 2024-09-11T00:32:47.139 INFO:teuthology.orchestra.run.smithi178.stdout: "command_active": 0, 2024-09-11T00:32:47.139 INFO:teuthology.orchestra.run.smithi178.stdout: "command_send": 0, 2024-09-11T00:32:47.139 INFO:teuthology.orchestra.run.smithi178.stdout: "command_resend": 0, 2024-09-11T00:32:47.139 INFO:teuthology.orchestra.run.smithi178.stdout: "map_epoch": 87, 2024-09-11T00:32:47.139 INFO:teuthology.orchestra.run.smithi178.stdout: "map_full": 0, 2024-09-11T00:32:47.139 INFO:teuthology.orchestra.run.smithi178.stdout: "map_inc": 6, 2024-09-11T00:32:47.139 INFO:teuthology.orchestra.run.smithi178.stdout: "osd_sessions": 11, 2024-09-11T00:32:47.139 INFO:teuthology.orchestra.run.smithi178.stdout: "osd_session_open": 11, 2024-09-11T00:32:47.139 INFO:teuthology.orchestra.run.smithi178.stdout: "osd_session_close": 0, 2024-09-11T00:32:47.139 INFO:teuthology.orchestra.run.smithi178.stdout: "osd_laggy": 0, 2024-09-11T00:32:47.139 INFO:teuthology.orchestra.run.smithi178.stdout: "omap_wr": 0, 2024-09-11T00:32:47.139 INFO:teuthology.orchestra.run.smithi178.stdout: "omap_rd": 2, 2024-09-11T00:32:47.139 INFO:teuthology.orchestra.run.smithi178.stdout: "omap_del": 0 2024-09-11T00:32:47.140 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T00:32:47.140 INFO:teuthology.orchestra.run.smithi178.stdout: "oft": { 2024-09-11T00:32:47.140 INFO:teuthology.orchestra.run.smithi178.stdout: "omap_total_objs": 0, 2024-09-11T00:32:47.140 INFO:teuthology.orchestra.run.smithi178.stdout: "omap_total_kv_pairs": 0, 2024-09-11T00:32:47.140 INFO:teuthology.orchestra.run.smithi178.stdout: "omap_total_updates": 0, 2024-09-11T00:32:47.140 INFO:teuthology.orchestra.run.smithi178.stdout: "omap_total_removes": 0 2024-09-11T00:32:47.140 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T00:32:47.140 INFO:teuthology.orchestra.run.smithi178.stdout: "purge_queue": { 2024-09-11T00:32:47.141 INFO:teuthology.orchestra.run.smithi178.stdout: "pq_executing_ops": 0, 2024-09-11T00:32:47.141 INFO:teuthology.orchestra.run.smithi178.stdout: "pq_executing_ops_high_water": 0, 2024-09-11T00:32:47.141 INFO:teuthology.orchestra.run.smithi178.stdout: "pq_executing": 0, 2024-09-11T00:32:47.141 INFO:teuthology.orchestra.run.smithi178.stdout: "pq_executing_high_water": 0, 2024-09-11T00:32:47.141 INFO:teuthology.orchestra.run.smithi178.stdout: "pq_executed_ops": 0, 2024-09-11T00:32:47.141 INFO:teuthology.orchestra.run.smithi178.stdout: "pq_executed": 0, 2024-09-11T00:32:47.141 INFO:teuthology.orchestra.run.smithi178.stdout: "pq_item_in_journal": 0 2024-09-11T00:32:47.142 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T00:32:47.142 INFO:teuthology.orchestra.run.smithi178.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-11T00:32:47.142 INFO:teuthology.orchestra.run.smithi178.stdout: "val": 0, 2024-09-11T00:32:47.142 INFO:teuthology.orchestra.run.smithi178.stdout: "max": 104857600, 2024-09-11T00:32:47.142 INFO:teuthology.orchestra.run.smithi178.stdout: "get_started": 0, 2024-09-11T00:32:47.142 INFO:teuthology.orchestra.run.smithi178.stdout: "get": 6324, 2024-09-11T00:32:47.142 INFO:teuthology.orchestra.run.smithi178.stdout: "get_sum": 180050670, 2024-09-11T00:32:47.142 INFO:teuthology.orchestra.run.smithi178.stdout: "get_or_fail_fail": 0, 2024-09-11T00:32:47.142 INFO:teuthology.orchestra.run.smithi178.stdout: "get_or_fail_success": 6324, 2024-09-11T00:32:47.142 INFO:teuthology.orchestra.run.smithi178.stdout: "take": 0, 2024-09-11T00:32:47.142 INFO:teuthology.orchestra.run.smithi178.stdout: "take_sum": 0, 2024-09-11T00:32:47.142 INFO:teuthology.orchestra.run.smithi178.stdout: "put": 6324, 2024-09-11T00:32:47.143 INFO:teuthology.orchestra.run.smithi178.stdout: "put_sum": 180050670, 2024-09-11T00:32:47.143 INFO:teuthology.orchestra.run.smithi178.stdout: "wait": { 2024-09-11T00:32:47.143 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-11T00:32:47.143 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-11T00:32:47.143 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-11T00:32:47.143 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-11T00:32:47.143 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T00:32:47.143 INFO:teuthology.orchestra.run.smithi178.stdout: "throttle-objecter_bytes": { 2024-09-11T00:32:47.143 INFO:teuthology.orchestra.run.smithi178.stdout: "val": 0, 2024-09-11T00:32:47.143 INFO:teuthology.orchestra.run.smithi178.stdout: "max": 104857600, 2024-09-11T00:32:47.143 INFO:teuthology.orchestra.run.smithi178.stdout: "get_started": 0, 2024-09-11T00:32:47.143 INFO:teuthology.orchestra.run.smithi178.stdout: "get": 0, 2024-09-11T00:32:47.143 INFO:teuthology.orchestra.run.smithi178.stdout: "get_sum": 0, 2024-09-11T00:32:47.144 INFO:teuthology.orchestra.run.smithi178.stdout: "get_or_fail_fail": 0, 2024-09-11T00:32:47.144 INFO:teuthology.orchestra.run.smithi178.stdout: "get_or_fail_success": 0, 2024-09-11T00:32:47.144 INFO:teuthology.orchestra.run.smithi178.stdout: "take": 5757, 2024-09-11T00:32:47.144 INFO:teuthology.orchestra.run.smithi178.stdout: "take_sum": 177867584, 2024-09-11T00:32:47.144 INFO:teuthology.orchestra.run.smithi178.stdout: "put": 5757, 2024-09-11T00:32:47.144 INFO:teuthology.orchestra.run.smithi178.stdout: "put_sum": 177867584, 2024-09-11T00:32:47.144 INFO:teuthology.orchestra.run.smithi178.stdout: "wait": { 2024-09-11T00:32:47.144 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-11T00:32:47.144 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-11T00:32:47.144 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-11T00:32:47.144 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-11T00:32:47.144 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T00:32:47.144 INFO:teuthology.orchestra.run.smithi178.stdout: "throttle-objecter_ops": { 2024-09-11T00:32:47.145 INFO:teuthology.orchestra.run.smithi178.stdout: "val": 0, 2024-09-11T00:32:47.145 INFO:teuthology.orchestra.run.smithi178.stdout: "max": 1024, 2024-09-11T00:32:47.145 INFO:teuthology.orchestra.run.smithi178.stdout: "get_started": 0, 2024-09-11T00:32:47.145 INFO:teuthology.orchestra.run.smithi178.stdout: "get": 0, 2024-09-11T00:32:47.145 INFO:teuthology.orchestra.run.smithi178.stdout: "get_sum": 0, 2024-09-11T00:32:47.145 INFO:teuthology.orchestra.run.smithi178.stdout: "get_or_fail_fail": 0, 2024-09-11T00:32:47.145 INFO:teuthology.orchestra.run.smithi178.stdout: "get_or_fail_success": 0, 2024-09-11T00:32:47.145 INFO:teuthology.orchestra.run.smithi178.stdout: "take": 5757, 2024-09-11T00:32:47.145 INFO:teuthology.orchestra.run.smithi178.stdout: "take_sum": 5757, 2024-09-11T00:32:47.145 INFO:teuthology.orchestra.run.smithi178.stdout: "put": 5757, 2024-09-11T00:32:47.145 INFO:teuthology.orchestra.run.smithi178.stdout: "put_sum": 5757, 2024-09-11T00:32:47.145 INFO:teuthology.orchestra.run.smithi178.stdout: "wait": { 2024-09-11T00:32:47.145 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-11T00:32:47.146 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-11T00:32:47.146 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-11T00:32:47.146 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-11T00:32:47.146 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T00:32:47.146 INFO:teuthology.orchestra.run.smithi178.stdout: "throttle-write_buf_throttle": { 2024-09-11T00:32:47.146 INFO:teuthology.orchestra.run.smithi178.stdout: "val": 0, 2024-09-11T00:32:47.146 INFO:teuthology.orchestra.run.smithi178.stdout: "max": 3758096384, 2024-09-11T00:32:47.146 INFO:teuthology.orchestra.run.smithi178.stdout: "get_started": 0, 2024-09-11T00:32:47.146 INFO:teuthology.orchestra.run.smithi178.stdout: "get": 0, 2024-09-11T00:32:47.146 INFO:teuthology.orchestra.run.smithi178.stdout: "get_sum": 0, 2024-09-11T00:32:47.146 INFO:teuthology.orchestra.run.smithi178.stdout: "get_or_fail_fail": 0, 2024-09-11T00:32:47.146 INFO:teuthology.orchestra.run.smithi178.stdout: "get_or_fail_success": 0, 2024-09-11T00:32:47.146 INFO:teuthology.orchestra.run.smithi178.stdout: "take": 0, 2024-09-11T00:32:47.146 INFO:teuthology.orchestra.run.smithi178.stdout: "take_sum": 0, 2024-09-11T00:32:47.147 INFO:teuthology.orchestra.run.smithi178.stdout: "put": 0, 2024-09-11T00:32:47.147 INFO:teuthology.orchestra.run.smithi178.stdout: "put_sum": 0, 2024-09-11T00:32:47.147 INFO:teuthology.orchestra.run.smithi178.stdout: "wait": { 2024-09-11T00:32:47.147 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-11T00:32:47.147 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-11T00:32:47.147 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-11T00:32:47.147 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-11T00:32:47.147 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T00:32:47.147 INFO:teuthology.orchestra.run.smithi178.stdout: "throttle-write_buf_throttle-0x55b5c94fd040": { 2024-09-11T00:32:47.147 INFO:teuthology.orchestra.run.smithi178.stdout: "val": 0, 2024-09-11T00:32:47.147 INFO:teuthology.orchestra.run.smithi178.stdout: "max": 3758096384, 2024-09-11T00:32:47.147 INFO:teuthology.orchestra.run.smithi178.stdout: "get_started": 0, 2024-09-11T00:32:47.148 INFO:teuthology.orchestra.run.smithi178.stdout: "get": 0, 2024-09-11T00:32:47.148 INFO:teuthology.orchestra.run.smithi178.stdout: "get_sum": 0, 2024-09-11T00:32:47.148 INFO:teuthology.orchestra.run.smithi178.stdout: "get_or_fail_fail": 0, 2024-09-11T00:32:47.148 INFO:teuthology.orchestra.run.smithi178.stdout: "get_or_fail_success": 0, 2024-09-11T00:32:47.148 INFO:teuthology.orchestra.run.smithi178.stdout: "take": 0, 2024-09-11T00:32:47.148 INFO:teuthology.orchestra.run.smithi178.stdout: "take_sum": 0, 2024-09-11T00:32:47.148 INFO:teuthology.orchestra.run.smithi178.stdout: "put": 0, 2024-09-11T00:32:47.148 INFO:teuthology.orchestra.run.smithi178.stdout: "put_sum": 0, 2024-09-11T00:32:47.148 INFO:teuthology.orchestra.run.smithi178.stdout: "wait": { 2024-09-11T00:32:47.148 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2024-09-11T00:32:47.148 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2024-09-11T00:32:47.148 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2024-09-11T00:32:47.148 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-11T00:32:47.149 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-11T00:32:47.149 INFO:teuthology.orchestra.run.smithi178.stdout: "trackedop": { 2024-09-11T00:32:47.149 INFO:teuthology.orchestra.run.smithi178.stdout: "slow_ops_count": 0 2024-09-11T00:32:47.149 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-11T00:32:47.149 INFO:teuthology.orchestra.run.smithi178.stdout:} 2024-09-11T00:32:47.888 INFO:tasks.check_counter:Daemon mds.l mds.exported=0 2024-09-11T00:32:47.888 INFO:tasks.check_counter:Daemon mds.l mds.imported=0 2024-09-11T00:32:47.888 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2024-09-11T00:32:47.897 INFO:tasks.kclient:Unmounting kernel clients... 2024-09-11T00:32:47.898 DEBUG:teuthology.orchestra.run.smithi012:> set -ex 2024-09-11T00:32:47.898 DEBUG:teuthology.orchestra.run.smithi012:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-11T00:32:47.951 DEBUG:teuthology.orchestra.run.smithi012:> set -ex 2024-09-11T00:32:47.951 DEBUG:teuthology.orchestra.run.smithi012:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-11T00:32:48.032 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid ff3ad74a-6fcf-11ef-bcea-c7b262605968 -- ceph osd blocklist ls 2024-09-11T00:32:48.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:32:48 smithi012 ceph-mon[30333]: pgmap v1171: 129 pgs: 129 active+clean; 671 MiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 511 B/s wr, 11 op/s 2024-09-11T00:32:48.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:32:48 smithi178 ceph-mon[35919]: pgmap v1171: 129 pgs: 129 active+clean; 671 MiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 511 B/s wr, 11 op/s 2024-09-11T00:32:48.457 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:32:48.533 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:32:48 smithi165 ceph-mon[35356]: pgmap v1171: 129 pgs: 129 active+clean; 671 MiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 511 B/s wr, 11 op/s 2024-09-11T00:32:49.029 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:49.026+0000 7fa2dbe00640 1 -- 172.21.15.12:0/2177480174 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa2dc075f40 msgr2=0x7fa2dc10ec50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:32:49.029 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:49.026+0000 7fa2dbe00640 1 --2- 172.21.15.12:0/2177480174 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa2dc075f40 0x7fa2dc10ec50 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7fa2c4009900 tx=0x7fa2c402a400 comp rx=0 tx=0).stop 2024-09-11T00:32:49.029 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:49.026+0000 7fa2dbe00640 1 -- 172.21.15.12:0/2177480174 shutdown_connections 2024-09-11T00:32:49.029 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:49.026+0000 7fa2dbe00640 1 --2- 172.21.15.12:0/2177480174 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa2dc10f220 0x7fa2dc111610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:32:49.029 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:49.026+0000 7fa2dbe00640 1 --2- 172.21.15.12:0/2177480174 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa2dc075f40 0x7fa2dc10ec50 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:32:49.030 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:49.026+0000 7fa2dbe00640 1 --2- 172.21.15.12:0/2177480174 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa2dc075540 0x7fa2dc075920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:32:49.030 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:49.026+0000 7fa2dbe00640 1 -- 172.21.15.12:0/2177480174 >> 172.21.15.12:0/2177480174 conn(0x7fa2dc0fe6d0 msgr2=0x7fa2dc100af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:32:49.030 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:49.026+0000 7fa2dbe00640 1 -- 172.21.15.12:0/2177480174 shutdown_connections 2024-09-11T00:32:49.030 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:49.026+0000 7fa2dbe00640 1 -- 172.21.15.12:0/2177480174 wait complete. 2024-09-11T00:32:49.031 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:49.027+0000 7fa2dbe00640 1 Processor -- start 2024-09-11T00:32:49.031 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:49.027+0000 7fa2dbe00640 1 -- start start 2024-09-11T00:32:49.031 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:49.028+0000 7fa2dbe00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa2dc075540 0x7fa2dc19b280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:32:49.031 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:49.028+0000 7fa2dbe00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa2dc075f40 0x7fa2dc19b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:32:49.031 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:49.028+0000 7fa2dbe00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa2dc10f220 0x7fa2dc1a0b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:32:49.031 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:49.028+0000 7fa2dbe00640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fa2dc114dd0 con 0x7fa2dc10f220 2024-09-11T00:32:49.031 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:49.028+0000 7fa2dbe00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fa2dc114c50 con 0x7fa2dc075540 2024-09-11T00:32:49.032 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:49.028+0000 7fa2dbe00640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fa2dc114f50 con 0x7fa2dc075f40 2024-09-11T00:32:49.032 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:49.028+0000 7fa2d2000640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa2dc075f40 0x7fa2dc19b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:32:49.032 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:49.028+0000 7fa2db400640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa2dc10f220 0x7fa2dc1a0b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:32:49.032 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:49.028+0000 7fa2d2000640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa2dc075f40 0x7fa2dc19b7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.12:56840/0 (socket says 172.21.15.12:56840) 2024-09-11T00:32:49.032 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:49.028+0000 7fa2d2000640 1 -- 172.21.15.12:0/342158053 learned_addr learned my addr 172.21.15.12:0/342158053 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:32:49.032 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:49.028+0000 7fa2daa00640 1 --2- 172.21.15.12:0/342158053 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa2dc075540 0x7fa2dc19b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:32:49.032 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:49.029+0000 7fa2db400640 1 -- 172.21.15.12:0/342158053 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa2dc075f40 msgr2=0x7fa2dc19b7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:32:49.032 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:49.029+0000 7fa2db400640 1 --2- 172.21.15.12:0/342158053 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa2dc075f40 0x7fa2dc19b7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:32:49.032 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:49.029+0000 7fa2db400640 1 -- 172.21.15.12:0/342158053 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa2dc075540 msgr2=0x7fa2dc19b280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:32:49.032 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:49.029+0000 7fa2db400640 1 --2- 172.21.15.12:0/342158053 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa2dc075540 0x7fa2dc19b280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:32:49.032 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:49.029+0000 7fa2db400640 1 -- 172.21.15.12:0/342158053 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2dc1a1330 con 0x7fa2dc10f220 2024-09-11T00:32:49.032 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:49.029+0000 7fa2daa00640 1 --2- 172.21.15.12:0/342158053 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa2dc075540 0x7fa2dc19b280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:32:49.032 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:49.029+0000 7fa2db400640 1 --2- 172.21.15.12:0/342158053 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa2dc10f220 0x7fa2dc1a0b60 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7fa2d400d650 tx=0x7fa2d400db10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:32:49.033 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:49.029+0000 7fa2d3e00640 1 -- 172.21.15.12:0/342158053 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2d4017070 con 0x7fa2dc10f220 2024-09-11T00:32:49.033 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:49.029+0000 7fa2d3e00640 1 -- 172.21.15.12:0/342158053 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa2d4009820 con 0x7fa2dc10f220 2024-09-11T00:32:49.033 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:49.029+0000 7fa2dbe00640 1 -- 172.21.15.12:0/342158053 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa2dc1a1620 con 0x7fa2dc10f220 2024-09-11T00:32:49.034 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:49.029+0000 7fa2d3e00640 1 -- 172.21.15.12:0/342158053 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2d4004c60 con 0x7fa2dc10f220 2024-09-11T00:32:49.034 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:49.029+0000 7fa2dbe00640 1 -- 172.21.15.12:0/342158053 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2dc1a7ce0 con 0x7fa2dc10f220 2024-09-11T00:32:49.034 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:49.030+0000 7fa2dbe00640 1 -- 172.21.15.12:0/342158053 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa2dc075920 con 0x7fa2dc10f220 2024-09-11T00:32:49.035 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:49.033+0000 7fa2d3e00640 1 -- 172.21.15.12:0/342158053 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7fa2d4005220 con 0x7fa2dc10f220 2024-09-11T00:32:49.037 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:49.034+0000 7fa2d3e00640 1 --2- 172.21.15.12:0/342158053 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fa2c00b5410 0x7fa2c00b78d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:32:49.037 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:49.034+0000 7fa2d3e00640 1 -- 172.21.15.12:0/342158053 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9208+0+0 (secure 0 0 0) 0x7fa2d40e5800 con 0x7fa2dc10f220 2024-09-11T00:32:49.037 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:49.034+0000 7fa2daa00640 1 --2- 172.21.15.12:0/342158053 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fa2c00b5410 0x7fa2c00b78d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:32:49.037 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:49.035+0000 7fa2daa00640 1 --2- 172.21.15.12:0/342158053 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fa2c00b5410 0x7fa2c00b78d0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7fa2cc009fd0 tx=0x7fa2cc009680 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:32:49.040 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:49.038+0000 7fa2d3e00640 1 -- 172.21.15.12:0/342158053 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7fa2d40abc40 con 0x7fa2dc10f220 2024-09-11T00:32:49.250 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:49.247+0000 7fa2dbe00640 1 -- 172.21.15.12:0/342158053 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fa2dc1a7fa0 con 0x7fa2dc10f220 2024-09-11T00:32:49.251 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:49.248+0000 7fa2d3e00640 1 -- 172.21.15.12:0/342158053 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v87) ==== 81+0+590 (secure 0 0 0) 0x7fa2d40b0670 con 0x7fa2dc10f220 2024-09-11T00:32:49.252 INFO:teuthology.orchestra.run.smithi012.stdout:172.21.15.12:6801/4177251511 2024-09-11T23:56:17.999820+0000 2024-09-11T00:32:49.252 INFO:teuthology.orchestra.run.smithi012.stdout:172.21.15.12:6800/4177251511 2024-09-11T23:56:17.999820+0000 2024-09-11T00:32:49.252 INFO:teuthology.orchestra.run.smithi012.stdout:172.21.15.12:0/3273692817 2024-09-11T23:56:17.999820+0000 2024-09-11T00:32:49.252 INFO:teuthology.orchestra.run.smithi012.stdout:172.21.15.12:0/4017778158 2024-09-11T23:56:17.999820+0000 2024-09-11T00:32:49.252 INFO:teuthology.orchestra.run.smithi012.stdout:172.21.15.12:0/2499921051 2024-09-11T23:55:50.302721+0000 2024-09-11T00:32:49.252 INFO:teuthology.orchestra.run.smithi012.stdout:172.21.15.12:6801/760140910 2024-09-11T23:55:50.302721+0000 2024-09-11T00:32:49.252 INFO:teuthology.orchestra.run.smithi012.stdout:172.21.15.12:6800/760140910 2024-09-11T23:55:50.302721+0000 2024-09-11T00:32:49.253 INFO:teuthology.orchestra.run.smithi012.stdout:172.21.15.12:0/2810703302 2024-09-11T23:56:17.999820+0000 2024-09-11T00:32:49.253 INFO:teuthology.orchestra.run.smithi012.stdout:172.21.15.12:0/2325436065 2024-09-11T23:55:50.302721+0000 2024-09-11T00:32:49.253 INFO:teuthology.orchestra.run.smithi012.stdout:172.21.15.12:0/3137053479 2024-09-11T23:55:50.302721+0000 2024-09-11T00:32:49.253 INFO:teuthology.orchestra.run.smithi012.stderr:listed 10 entries 2024-09-11T00:32:49.254 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:49.250+0000 7fa2dbe00640 1 -- 172.21.15.12:0/342158053 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fa2c00b5410 msgr2=0x7fa2c00b78d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:32:49.254 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:49.250+0000 7fa2dbe00640 1 --2- 172.21.15.12:0/342158053 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fa2c00b5410 0x7fa2c00b78d0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7fa2cc009fd0 tx=0x7fa2cc009680 comp rx=0 tx=0).stop 2024-09-11T00:32:49.254 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:49.250+0000 7fa2dbe00640 1 -- 172.21.15.12:0/342158053 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa2dc10f220 msgr2=0x7fa2dc1a0b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:32:49.254 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:49.250+0000 7fa2dbe00640 1 --2- 172.21.15.12:0/342158053 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa2dc10f220 0x7fa2dc1a0b60 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7fa2d400d650 tx=0x7fa2d400db10 comp rx=0 tx=0).stop 2024-09-11T00:32:49.254 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:49.250+0000 7fa2dbe00640 1 -- 172.21.15.12:0/342158053 shutdown_connections 2024-09-11T00:32:49.254 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:49.250+0000 7fa2dbe00640 1 --2- 172.21.15.12:0/342158053 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fa2c00b5410 0x7fa2c00b78d0 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:32:49.254 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:49.250+0000 7fa2dbe00640 1 --2- 172.21.15.12:0/342158053 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa2dc10f220 0x7fa2dc1a0b60 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:32:49.254 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:49.250+0000 7fa2dbe00640 1 --2- 172.21.15.12:0/342158053 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa2dc075f40 0x7fa2dc19b7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:32:49.254 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:49.250+0000 7fa2dbe00640 1 --2- 172.21.15.12:0/342158053 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa2dc075540 0x7fa2dc19b280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:32:49.254 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:49.250+0000 7fa2dbe00640 1 -- 172.21.15.12:0/342158053 >> 172.21.15.12:0/342158053 conn(0x7fa2dc0fe6d0 msgr2=0x7fa2dc111410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:32:49.254 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:49.250+0000 7fa2dbe00640 1 -- 172.21.15.12:0/342158053 shutdown_connections 2024-09-11T00:32:49.254 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:49.250+0000 7fa2dbe00640 1 -- 172.21.15.12:0/342158053 wait complete. 2024-09-11T00:32:50.105 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-09-11T00:32:50.106 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T00:32:50.106 DEBUG:teuthology.orchestra.run.smithi012:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-09-11T00:32:50.254 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi012.front.sepia.ceph.com 2024-09-11T00:32:50.254 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T00:32:50.254 DEBUG:teuthology.orchestra.run.smithi012:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-11T00:32:50.314 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:32:50 smithi012 ceph-mon[30333]: pgmap v1172: 129 pgs: 129 active+clean; 671 MiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 511 B/s wr, 8 op/s 2024-09-11T00:32:50.314 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:32:50 smithi012 ceph-mon[30333]: from='client.? 172.21.15.12:0/342158053' entity='client.admin' cmd={"prefix": "osd blocklist ls"} : dispatch 2024-09-11T00:32:50.329 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T00:32:50.329 DEBUG:teuthology.orchestra.run.smithi012:> ip netns list 2024-09-11T00:32:50.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:32:50 smithi178 ceph-mon[35919]: pgmap v1172: 129 pgs: 129 active+clean; 671 MiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 511 B/s wr, 8 op/s 2024-09-11T00:32:50.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:32:50 smithi178 ceph-mon[35919]: from='client.? 172.21.15.12:0/342158053' entity='client.admin' cmd={"prefix": "osd blocklist ls"} : dispatch 2024-09-11T00:32:50.472 INFO:teuthology.orchestra.run.smithi012.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-11T00:32:50.472 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T00:32:50.473 DEBUG:teuthology.orchestra.run.smithi012:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-11T00:32:50.532 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:32:50 smithi165 ceph-mon[35356]: pgmap v1172: 129 pgs: 129 active+clean; 671 MiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 511 B/s wr, 8 op/s 2024-09-11T00:32:50.533 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:32:50 smithi165 ceph-mon[35356]: from='client.? 172.21.15.12:0/342158053' entity='client.admin' cmd={"prefix": "osd blocklist ls"} : dispatch 2024-09-11T00:32:50.597 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T00:32:50.597 DEBUG:teuthology.orchestra.run.smithi012:> sudo ip link delete ceph-brx 2024-09-11T00:32:50.907 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-09-11T00:32:50.923 INFO:tasks.cephadm:Teardown begin 2024-09-11T00:32:50.923 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-11T00:32:50.923 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid ff3ad74a-6fcf-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T00:32:51.308 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:32:52.089 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:32:52 smithi012 ceph-mon[30333]: pgmap v1173: 129 pgs: 129 active+clean; 671 MiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 511 B/s wr, 10 op/s 2024-09-11T00:32:52.091 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:52.085+0000 7f60bc800640 1 -- 172.21.15.12:0/4141332271 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f60b4075e60 msgr2=0x7f60b41135e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:32:52.091 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:52.085+0000 7f60bc800640 1 --2- 172.21.15.12:0/4141332271 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f60b4075e60 0x7f60b41135e0 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f60a4009900 tx=0x7f60a402a410 comp rx=0 tx=0).stop 2024-09-11T00:32:52.091 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:52.086+0000 7f60bc800640 1 -- 172.21.15.12:0/4141332271 shutdown_connections 2024-09-11T00:32:52.091 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:52.086+0000 7f60bc800640 1 --2- 172.21.15.12:0/4141332271 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f60b4113b20 0x7f60b4115f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:32:52.091 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:52.086+0000 7f60bc800640 1 --2- 172.21.15.12:0/4141332271 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f60b4075e60 0x7f60b41135e0 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:32:52.091 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:52.086+0000 7f60bc800640 1 --2- 172.21.15.12:0/4141332271 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f60b4075540 0x7f60b4075920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:32:52.091 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:52.086+0000 7f60bc800640 1 -- 172.21.15.12:0/4141332271 >> 172.21.15.12:0/4141332271 conn(0x7f60b40fe830 msgr2=0x7f60b4100c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:32:52.091 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:52.086+0000 7f60bc800640 1 -- 172.21.15.12:0/4141332271 shutdown_connections 2024-09-11T00:32:52.092 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:52.086+0000 7f60bc800640 1 -- 172.21.15.12:0/4141332271 wait complete. 2024-09-11T00:32:52.092 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:52.087+0000 7f60bc800640 1 Processor -- start 2024-09-11T00:32:52.092 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:52.087+0000 7f60bc800640 1 -- start start 2024-09-11T00:32:52.092 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:52.087+0000 7f60bc800640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f60b4075540 0x7f60b419f950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:32:52.092 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:52.087+0000 7f60bc800640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f60b4075e60 0x7f60b419fe90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:32:52.092 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:52.087+0000 7f60bc800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f60b4113b20 0x7f60b41a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:32:52.092 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:52.087+0000 7f60bc800640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f60b41193a0 con 0x7f60b4075e60 2024-09-11T00:32:52.092 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:52.087+0000 7f60bc800640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f60b4119220 con 0x7f60b4113b20 2024-09-11T00:32:52.092 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:52.087+0000 7f60bc800640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f60b4119520 con 0x7f60b4075540 2024-09-11T00:32:52.092 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:52.087+0000 7f60ba800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f60b4113b20 0x7f60b41a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:32:52.093 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:52.087+0000 7f60b9e00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f60b4075540 0x7f60b419f950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:32:52.093 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:52.088+0000 7f60ba800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f60b4113b20 0x7f60b41a5230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.12:42584/0 (socket says 172.21.15.12:42584) 2024-09-11T00:32:52.093 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:52.088+0000 7f60ba800640 1 -- 172.21.15.12:0/932937806 learned_addr learned my addr 172.21.15.12:0/932937806 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:32:52.093 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:52.088+0000 7f60b9400640 1 --2- 172.21.15.12:0/932937806 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f60b4075e60 0x7f60b419fe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:32:52.093 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:52.088+0000 7f60ba800640 1 -- 172.21.15.12:0/932937806 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f60b4075540 msgr2=0x7f60b419f950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:32:52.093 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:52.088+0000 7f60ba800640 1 --2- 172.21.15.12:0/932937806 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f60b4075540 0x7f60b419f950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:32:52.093 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:52.088+0000 7f60ba800640 1 -- 172.21.15.12:0/932937806 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f60b4075e60 msgr2=0x7f60b419fe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:32:52.093 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:52.088+0000 7f60ba800640 1 --2- 172.21.15.12:0/932937806 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f60b4075e60 0x7f60b419fe90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:32:52.093 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:52.088+0000 7f60ba800640 1 -- 172.21.15.12:0/932937806 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60b41a5a00 con 0x7f60b4113b20 2024-09-11T00:32:52.093 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:52.088+0000 7f60ba800640 1 --2- 172.21.15.12:0/932937806 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f60b4113b20 0x7f60b41a5230 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f60b000e9d0 tx=0x7f60b000ee90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:32:52.093 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:52.089+0000 7f60aea00640 1 -- 172.21.15.12:0/932937806 <== mon.2 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60b000cd80 con 0x7f60b4113b20 2024-09-11T00:32:52.094 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:52.089+0000 7f60bc800640 1 -- 172.21.15.12:0/932937806 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f60b41a5cf0 con 0x7f60b4113b20 2024-09-11T00:32:52.094 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:52.089+0000 7f60aea00640 1 -- 172.21.15.12:0/932937806 <== mon.2 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f60b0005020 con 0x7f60b4113b20 2024-09-11T00:32:52.094 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:52.089+0000 7f60aea00640 1 -- 172.21.15.12:0/932937806 <== mon.2 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60b00116d0 con 0x7f60b4113b20 2024-09-11T00:32:52.094 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:52.089+0000 7f60bc800640 1 -- 172.21.15.12:0/932937806 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60b41ac3b0 con 0x7f60b4113b20 2024-09-11T00:32:52.094 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:52.090+0000 7f60bc800640 1 -- 172.21.15.12:0/932937806 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f60b4075920 con 0x7f60b4113b20 2024-09-11T00:32:52.094 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:52.092+0000 7f60aea00640 1 -- 172.21.15.12:0/932937806 <== mon.2 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7f60b000f040 con 0x7f60b4113b20 2024-09-11T00:32:52.096 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:52.094+0000 7f60aea00640 1 --2- 172.21.15.12:0/932937806 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f609c0b52e0 0x7f609c0b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:32:52.096 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:52.094+0000 7f60aea00640 1 -- 172.21.15.12:0/932937806 <== mon.2 v2:172.21.15.165:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9208+0+0 (secure 0 0 0) 0x7f60b00e6290 con 0x7f60b4113b20 2024-09-11T00:32:52.096 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:52.094+0000 7f60b9e00640 1 --2- 172.21.15.12:0/932937806 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f609c0b52e0 0x7f609c0b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:32:52.097 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:52.094+0000 7f60b9e00640 1 --2- 172.21.15.12:0/932937806 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f609c0b52e0 0x7f609c0b77a0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f60a8009bd0 tx=0x7f60a8005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:32:52.097 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:52.095+0000 7f60aea00640 1 -- 172.21.15.12:0/932937806 <== mon.2 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f60b0016030 con 0x7f60b4113b20 2024-09-11T00:32:52.330 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:52.328+0000 7f60bc800640 1 -- 172.21.15.12:0/932937806 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f60b41a0a80 con 0x7f60b4113b20 2024-09-11T00:32:52.331 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:52.329+0000 7f60aea00640 1 -- 172.21.15.12:0/932937806 <== mon.2 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 112 v112) ==== 95+0+12001 (secure 0 0 0) 0x7f60b00acd20 con 0x7f60b4113b20 2024-09-11T00:32:52.332 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:32:52 smithi178 ceph-mon[35919]: pgmap v1173: 129 pgs: 129 active+clean; 671 MiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 511 B/s wr, 10 op/s 2024-09-11T00:32:52.332 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-11T00:32:52.332 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":112,"btime":"2024-09-11T00:32:22:975489+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14577,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6835/447171591","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":447171591},{"type":"v1","addr":"172.21.15.12:6835","nonce":447171591}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14589,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/901486040","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":901486040},{"type":"v1","addr":"172.21.15.12:6837","nonce":901486040}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":112,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T00:04:04.739881+0000","modified":"2024-09-11T00:32:22.258752+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24674,"mds_1":24643,"mds_2":24638,"mds_3":24637,"mds_4":24614},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14631":{"gid":14631,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.12:6841/3069919437","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6840","nonce":3069919437},{"type":"v1","addr":"172.21.15.12:6841","nonce":3069919437}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24562":{"gid":24562,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6833/1141636112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":1141636112},{"type":"v1","addr":"172.21.15.178:6833","nonce":1141636112}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24581":{"gid":24581,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6833/3111980970","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6832","nonce":3111980970},{"type":"v1","addr":"172.21.15.165:6833","nonce":3111980970}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24607":{"gid":24607,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6835/4084868287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":4084868287},{"type":"v1","addr":"172.21.15.178:6835","nonce":4084868287}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24614":{"gid":24614,"name":"e","rank":4,"incarnation":26,"state":"up:active","state_seq":21,"addr":"172.21.15.165:6835/2425395274","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":2425395274},{"type":"v1","addr":"172.21.15.165:6835","nonce":2425395274}]},"join_fscid":-1,"export_targets":[0,1,2,3],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24637":{"gid":24637,"name":"i","rank":3,"incarnation":24,"state":"up:active","state_seq":18,"addr":"172.21.15.178:6839/70628097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6838","nonce":70628097},{"type":"v1","addr":"172.21.15.178:6839","nonce":70628097}]},"join_fscid":-1,"export_targets":[0,1,2],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24638":{"gid":24638,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":18,"addr":"172.21.15.165:6837/6215088","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6836","nonce":6215088},{"type":"v1","addr":"172.21.15.165:6837","nonce":6215088}]},"join_fscid":-1,"export_targets":[0,1,3,4],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24643":{"gid":24643,"name":"g","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.12:6839/3124093009","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6838","nonce":3124093009},{"type":"v1","addr":"172.21.15.12:6839","nonce":3124093009}]},"join_fscid":-1,"export_targets":[0,2,3,4],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24674":{"gid":24674,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.165:6839/501312984","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6838","nonce":501312984},{"type":"v1","addr":"172.21.15.165:6839","nonce":501312984}]},"join_fscid":-1,"export_targets":[1,2,4],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_27751":{"gid":27751,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6837/1436600611","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":1436600611},{"type":"v1","addr":"172.21.15.178:6837","nonce":1436600611}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24674,"qdb_cluster":[24674,24643,24638,24637,24614]},"id":1}]} 2024-09-11T00:32:52.332 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 112 2024-09-11T00:32:52.334 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:52.331+0000 7f60bc800640 1 -- 172.21.15.12:0/932937806 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f609c0b52e0 msgr2=0x7f609c0b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:32:52.334 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:52.331+0000 7f60bc800640 1 --2- 172.21.15.12:0/932937806 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f609c0b52e0 0x7f609c0b77a0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f60a8009bd0 tx=0x7f60a8005f90 comp rx=0 tx=0).stop 2024-09-11T00:32:52.334 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:52.331+0000 7f60bc800640 1 -- 172.21.15.12:0/932937806 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f60b4113b20 msgr2=0x7f60b41a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:32:52.334 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:52.331+0000 7f60bc800640 1 --2- 172.21.15.12:0/932937806 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f60b4113b20 0x7f60b41a5230 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f60b000e9d0 tx=0x7f60b000ee90 comp rx=0 tx=0).stop 2024-09-11T00:32:52.334 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:52.331+0000 7f60bc800640 1 -- 172.21.15.12:0/932937806 shutdown_connections 2024-09-11T00:32:52.334 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:52.331+0000 7f60bc800640 1 --2- 172.21.15.12:0/932937806 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f609c0b52e0 0x7f609c0b77a0 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:32:52.334 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:52.331+0000 7f60bc800640 1 --2- 172.21.15.12:0/932937806 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f60b4113b20 0x7f60b41a5230 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:32:52.334 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:52.331+0000 7f60bc800640 1 --2- 172.21.15.12:0/932937806 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f60b4075e60 0x7f60b419fe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:32:52.334 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:52.331+0000 7f60bc800640 1 --2- 172.21.15.12:0/932937806 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f60b4075540 0x7f60b419f950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:32:52.335 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:52.331+0000 7f60bc800640 1 -- 172.21.15.12:0/932937806 >> 172.21.15.12:0/932937806 conn(0x7f60b40fe830 msgr2=0x7f60b40ff430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:32:52.335 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:52.331+0000 7f60bc800640 1 -- 172.21.15.12:0/932937806 shutdown_connections 2024-09-11T00:32:52.335 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:52.331+0000 7f60bc800640 1 -- 172.21.15.12:0/932937806 wait complete. 2024-09-11T00:32:52.335 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:32:52 smithi165 ceph-mon[35356]: pgmap v1173: 129 pgs: 129 active+clean; 671 MiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 511 B/s wr, 10 op/s 2024-09-11T00:32:53.060 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid ff3ad74a-6fcf-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T00:32:53.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:32:53 smithi012 ceph-mon[30333]: from='client.? 172.21.15.12:0/932937806' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T00:32:53.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:32:53 smithi178 ceph-mon[35919]: from='client.? 172.21.15.12:0/932937806' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T00:32:53.443 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:32:53.533 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:32:53 smithi165 ceph-mon[35356]: from='client.? 172.21.15.12:0/932937806' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T00:32:54.048 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:54.000+0000 7fb8c9e00640 1 -- 172.21.15.12:0/3908626354 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb8c4108620 msgr2=0x7fb8c4108a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:32:54.048 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:54.000+0000 7fb8c9e00640 1 --2- 172.21.15.12:0/3908626354 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb8c4108620 0x7fb8c4108a60 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fb8b0009900 tx=0x7fb8b002a3f0 comp rx=0 tx=0).stop 2024-09-11T00:32:54.048 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:54.000+0000 7fb8c9e00640 1 -- 172.21.15.12:0/3908626354 shutdown_connections 2024-09-11T00:32:54.048 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:54.000+0000 7fb8c9e00640 1 --2- 172.21.15.12:0/3908626354 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb8c4108fa0 0x7fb8c410f7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:32:54.048 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:54.000+0000 7fb8c9e00640 1 --2- 172.21.15.12:0/3908626354 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb8c4108620 0x7fb8c4108a60 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:32:54.048 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:54.000+0000 7fb8c9e00640 1 --2- 172.21.15.12:0/3908626354 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb8c4102c10 0x7fb8c4102ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:32:54.048 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:54.000+0000 7fb8c9e00640 1 -- 172.21.15.12:0/3908626354 >> 172.21.15.12:0/3908626354 conn(0x7fb8c40fe660 msgr2=0x7fb8c4100a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:32:54.048 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:54.001+0000 7fb8c9e00640 1 -- 172.21.15.12:0/3908626354 shutdown_connections 2024-09-11T00:32:54.048 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:54.001+0000 7fb8c9e00640 1 -- 172.21.15.12:0/3908626354 wait complete. 2024-09-11T00:32:54.049 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:54.002+0000 7fb8c9e00640 1 Processor -- start 2024-09-11T00:32:54.049 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:54.002+0000 7fb8c9e00640 1 -- start start 2024-09-11T00:32:54.049 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:54.002+0000 7fb8c9e00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb8c4102c10 0x7fb8c419b230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:32:54.049 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:54.002+0000 7fb8c9e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb8c4108620 0x7fb8c419b770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:32:54.049 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:54.002+0000 7fb8c9e00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb8c4108fa0 0x7fb8c41a0b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:32:54.049 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:54.002+0000 7fb8c9e00640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fb8c4112c30 con 0x7fb8c4102c10 2024-09-11T00:32:54.049 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:54.002+0000 7fb8c9e00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fb8c4112ab0 con 0x7fb8c4108620 2024-09-11T00:32:54.049 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:54.002+0000 7fb8c9e00640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7fb8c4112db0 con 0x7fb8c4108fa0 2024-09-11T00:32:54.049 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:54.002+0000 7fb8c8a00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb8c4102c10 0x7fb8c419b230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:32:54.049 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:54.002+0000 7fb8c8a00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb8c4102c10 0x7fb8c419b230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:60814/0 (socket says 172.21.15.12:60814) 2024-09-11T00:32:54.049 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:54.002+0000 7fb8c3e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb8c4108620 0x7fb8c419b770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:32:54.049 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:54.002+0000 7fb8c8a00640 1 -- 172.21.15.12:0/4225407549 learned_addr learned my addr 172.21.15.12:0/4225407549 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:32:54.049 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:54.002+0000 7fb8c9400640 1 --2- 172.21.15.12:0/4225407549 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb8c4108fa0 0x7fb8c41a0b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:32:54.049 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:54.003+0000 7fb8c8a00640 1 -- 172.21.15.12:0/4225407549 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb8c4108fa0 msgr2=0x7fb8c41a0b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:32:54.050 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:54.003+0000 7fb8c8a00640 1 --2- 172.21.15.12:0/4225407549 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb8c4108fa0 0x7fb8c41a0b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:32:54.050 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:54.003+0000 7fb8c8a00640 1 -- 172.21.15.12:0/4225407549 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb8c4108620 msgr2=0x7fb8c419b770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:32:54.050 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:54.003+0000 7fb8c8a00640 1 --2- 172.21.15.12:0/4225407549 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb8c4108620 0x7fb8c419b770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:32:54.050 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:54.003+0000 7fb8c8a00640 1 -- 172.21.15.12:0/4225407549 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8c41a12e0 con 0x7fb8c4102c10 2024-09-11T00:32:54.050 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:54.003+0000 7fb8c3e00640 1 --2- 172.21.15.12:0/4225407549 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb8c4108620 0x7fb8c419b770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:32:54.050 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:54.003+0000 7fb8c8a00640 1 --2- 172.21.15.12:0/4225407549 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb8c4102c10 0x7fb8c419b230 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7fb8b800ea60 tx=0x7fb8b800ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:32:54.050 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:54.003+0000 7fb8c1600640 1 -- 172.21.15.12:0/4225407549 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8b800cde0 con 0x7fb8c4102c10 2024-09-11T00:32:54.050 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:54.003+0000 7fb8c1600640 1 -- 172.21.15.12:0/4225407549 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb8b80035f0 con 0x7fb8c4102c10 2024-09-11T00:32:54.050 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:54.003+0000 7fb8c9e00640 1 -- 172.21.15.12:0/4225407549 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb8c41a15d0 con 0x7fb8c4102c10 2024-09-11T00:32:54.050 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:54.003+0000 7fb8c1600640 1 -- 172.21.15.12:0/4225407549 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8b8010690 con 0x7fb8c4102c10 2024-09-11T00:32:54.050 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:54.004+0000 7fb8c9e00640 1 -- 172.21.15.12:0/4225407549 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8c41a7c90 con 0x7fb8c4102c10 2024-09-11T00:32:54.050 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:54.005+0000 7fb8c9e00640 1 -- 172.21.15.12:0/4225407549 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb8c4102ff0 con 0x7fb8c4102c10 2024-09-11T00:32:54.050 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:54.007+0000 7fb8c1600640 1 -- 172.21.15.12:0/4225407549 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7fb8b8004000 con 0x7fb8c4102c10 2024-09-11T00:32:54.050 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:54.008+0000 7fb8c1600640 1 --2- 172.21.15.12:0/4225407549 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fb8ac0b5390 0x7fb8ac0b7850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:32:54.051 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:54.008+0000 7fb8c1600640 1 -- 172.21.15.12:0/4225407549 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9208+0+0 (secure 0 0 0) 0x7fb8b80e62c0 con 0x7fb8c4102c10 2024-09-11T00:32:54.051 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:54.008+0000 7fb8c3e00640 1 --2- 172.21.15.12:0/4225407549 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fb8ac0b5390 0x7fb8ac0b7850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:32:54.051 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:54.009+0000 7fb8c3e00640 1 --2- 172.21.15.12:0/4225407549 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fb8ac0b5390 0x7fb8ac0b7850 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7fb8b0031440 tx=0x7fb8b0005b50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:32:54.051 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:54.012+0000 7fb8c1600640 1 -- 172.21.15.12:0/4225407549 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7fb8b80acd50 con 0x7fb8c4102c10 2024-09-11T00:32:54.251 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:54.248+0000 7fb8c9e00640 1 -- 172.21.15.12:0/4225407549 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb8c402e5c0 con 0x7fb8c4102c10 2024-09-11T00:32:54.251 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:32:54 smithi012 ceph-mon[30333]: pgmap v1174: 129 pgs: 129 active+clean; 671 MiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 1.4 KiB/s wr, 11 op/s 2024-09-11T00:32:54.252 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:54.249+0000 7fb8c1600640 1 -- 172.21.15.12:0/4225407549 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 112 v112) ==== 95+0+12001 (secure 0 0 0) 0x7fb8b80ac650 con 0x7fb8c4102c10 2024-09-11T00:32:54.252 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-11T00:32:54.252 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":112,"btime":"2024-09-11T00:32:22:975489+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14577,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6835/447171591","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":447171591},{"type":"v1","addr":"172.21.15.12:6835","nonce":447171591}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14589,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/901486040","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":901486040},{"type":"v1","addr":"172.21.15.12:6837","nonce":901486040}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":112,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T00:04:04.739881+0000","modified":"2024-09-11T00:32:22.258752+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24674,"mds_1":24643,"mds_2":24638,"mds_3":24637,"mds_4":24614},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14631":{"gid":14631,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.12:6841/3069919437","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6840","nonce":3069919437},{"type":"v1","addr":"172.21.15.12:6841","nonce":3069919437}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24562":{"gid":24562,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6833/1141636112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":1141636112},{"type":"v1","addr":"172.21.15.178:6833","nonce":1141636112}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24581":{"gid":24581,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6833/3111980970","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6832","nonce":3111980970},{"type":"v1","addr":"172.21.15.165:6833","nonce":3111980970}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24607":{"gid":24607,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6835/4084868287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":4084868287},{"type":"v1","addr":"172.21.15.178:6835","nonce":4084868287}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24614":{"gid":24614,"name":"e","rank":4,"incarnation":26,"state":"up:active","state_seq":21,"addr":"172.21.15.165:6835/2425395274","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":2425395274},{"type":"v1","addr":"172.21.15.165:6835","nonce":2425395274}]},"join_fscid":-1,"export_targets":[0,1,2,3],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24637":{"gid":24637,"name":"i","rank":3,"incarnation":24,"state":"up:active","state_seq":18,"addr":"172.21.15.178:6839/70628097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6838","nonce":70628097},{"type":"v1","addr":"172.21.15.178:6839","nonce":70628097}]},"join_fscid":-1,"export_targets":[0,1,2],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24638":{"gid":24638,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":18,"addr":"172.21.15.165:6837/6215088","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6836","nonce":6215088},{"type":"v1","addr":"172.21.15.165:6837","nonce":6215088}]},"join_fscid":-1,"export_targets":[0,1,3,4],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24643":{"gid":24643,"name":"g","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.12:6839/3124093009","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6838","nonce":3124093009},{"type":"v1","addr":"172.21.15.12:6839","nonce":3124093009}]},"join_fscid":-1,"export_targets":[0,2,3,4],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24674":{"gid":24674,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.165:6839/501312984","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6838","nonce":501312984},{"type":"v1","addr":"172.21.15.165:6839","nonce":501312984}]},"join_fscid":-1,"export_targets":[1,2,4],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_27751":{"gid":27751,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6837/1436600611","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":1436600611},{"type":"v1","addr":"172.21.15.178:6837","nonce":1436600611}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24674,"qdb_cluster":[24674,24643,24638,24637,24614]},"id":1}]} 2024-09-11T00:32:54.252 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 112 2024-09-11T00:32:54.255 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:54.252+0000 7fb8c9e00640 1 -- 172.21.15.12:0/4225407549 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fb8ac0b5390 msgr2=0x7fb8ac0b7850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:32:54.255 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:54.252+0000 7fb8c9e00640 1 --2- 172.21.15.12:0/4225407549 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fb8ac0b5390 0x7fb8ac0b7850 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7fb8b0031440 tx=0x7fb8b0005b50 comp rx=0 tx=0).stop 2024-09-11T00:32:54.255 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:54.252+0000 7fb8c9e00640 1 -- 172.21.15.12:0/4225407549 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb8c4102c10 msgr2=0x7fb8c419b230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:32:54.255 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:54.252+0000 7fb8c9e00640 1 --2- 172.21.15.12:0/4225407549 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb8c4102c10 0x7fb8c419b230 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7fb8b800ea60 tx=0x7fb8b800ef20 comp rx=0 tx=0).stop 2024-09-11T00:32:54.255 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:54.252+0000 7fb8c9e00640 1 -- 172.21.15.12:0/4225407549 shutdown_connections 2024-09-11T00:32:54.255 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:54.252+0000 7fb8c9e00640 1 --2- 172.21.15.12:0/4225407549 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7fb8ac0b5390 0x7fb8ac0b7850 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:32:54.255 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:54.252+0000 7fb8c9e00640 1 --2- 172.21.15.12:0/4225407549 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb8c4108fa0 0x7fb8c41a0b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:32:54.255 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:54.252+0000 7fb8c9e00640 1 --2- 172.21.15.12:0/4225407549 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb8c4108620 0x7fb8c419b770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:32:54.255 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:54.252+0000 7fb8c9e00640 1 --2- 172.21.15.12:0/4225407549 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb8c4102c10 0x7fb8c419b230 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:32:54.255 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:54.252+0000 7fb8c9e00640 1 -- 172.21.15.12:0/4225407549 >> 172.21.15.12:0/4225407549 conn(0x7fb8c40fe660 msgr2=0x7fb8c4105c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:32:54.255 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:54.252+0000 7fb8c9e00640 1 -- 172.21.15.12:0/4225407549 shutdown_connections 2024-09-11T00:32:54.255 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:54.253+0000 7fb8c9e00640 1 -- 172.21.15.12:0/4225407549 wait complete. 2024-09-11T00:32:54.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:32:54 smithi178 ceph-mon[35919]: pgmap v1174: 129 pgs: 129 active+clean; 671 MiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 1.4 KiB/s wr, 11 op/s 2024-09-11T00:32:54.533 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:32:54 smithi165 ceph-mon[35356]: pgmap v1174: 129 pgs: 129 active+clean; 671 MiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 1.4 KiB/s wr, 11 op/s 2024-09-11T00:32:54.977 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid ff3ad74a-6fcf-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-11T00:32:55.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:32:55 smithi012 ceph-mon[30333]: from='client.? 172.21.15.12:0/4225407549' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T00:32:55.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:32:55 smithi178 ceph-mon[35919]: from='client.? 172.21.15.12:0/4225407549' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T00:32:55.357 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:32:55.532 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:32:55 smithi165 ceph-mon[35356]: from='client.? 172.21.15.12:0/4225407549' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T00:32:56.037 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:56.034+0000 7f00af000640 1 -- 172.21.15.12:0/1823169042 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f00a80776d0 msgr2=0x7f00a8077ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:32:56.037 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:56.034+0000 7f00af000640 1 --2- 172.21.15.12:0/1823169042 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f00a80776d0 0x7f00a8077ab0 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f0098009900 tx=0x7f009802a410 comp rx=0 tx=0).stop 2024-09-11T00:32:56.037 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:56.035+0000 7f00af000640 1 -- 172.21.15.12:0/1823169042 shutdown_connections 2024-09-11T00:32:56.038 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:56.035+0000 7f00af000640 1 --2- 172.21.15.12:0/1823169042 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f00a8113990 0x7f00a8115d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:32:56.038 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:56.035+0000 7f00af000640 1 --2- 172.21.15.12:0/1823169042 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f00a8077ff0 0x7f00a8113450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:32:56.038 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:56.035+0000 7f00af000640 1 --2- 172.21.15.12:0/1823169042 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f00a80776d0 0x7f00a8077ab0 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:32:56.038 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:56.035+0000 7f00af000640 1 -- 172.21.15.12:0/1823169042 >> 172.21.15.12:0/1823169042 conn(0x7f00a8100a50 msgr2=0x7f00a8102e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:32:56.038 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:56.035+0000 7f00af000640 1 -- 172.21.15.12:0/1823169042 shutdown_connections 2024-09-11T00:32:56.038 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:56.035+0000 7f00af000640 1 -- 172.21.15.12:0/1823169042 wait complete. 2024-09-11T00:32:56.039 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:56.035+0000 7f00af000640 1 Processor -- start 2024-09-11T00:32:56.039 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:56.036+0000 7f00af000640 1 -- start start 2024-09-11T00:32:56.039 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:56.036+0000 7f00af000640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f00a80776d0 0x7f00a819f930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:32:56.039 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:56.036+0000 7f00af000640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f00a8077ff0 0x7f00a819fe70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:32:56.039 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:56.036+0000 7f00af000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f00a8113990 0x7f00a81a5210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:32:56.039 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:56.036+0000 7f00af000640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f00a81193a0 con 0x7f00a8077ff0 2024-09-11T00:32:56.039 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:56.036+0000 7f00af000640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f00a8119220 con 0x7f00a8113990 2024-09-11T00:32:56.040 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:56.036+0000 7f00af000640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f00a8119520 con 0x7f00a80776d0 2024-09-11T00:32:56.040 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:56.036+0000 7f009fe00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f00a8077ff0 0x7f00a819fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:32:56.040 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:56.036+0000 7f009fe00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f00a8077ff0 0x7f00a819fe70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:60836/0 (socket says 172.21.15.12:60836) 2024-09-11T00:32:56.040 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:56.036+0000 7f009fe00640 1 -- 172.21.15.12:0/2281358280 learned_addr learned my addr 172.21.15.12:0/2281358280 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:32:56.040 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:56.037+0000 7f00ad000640 1 --2- 172.21.15.12:0/2281358280 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f00a8113990 0x7f00a81a5210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:32:56.040 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:56.037+0000 7f00a7e00640 1 --2- 172.21.15.12:0/2281358280 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f00a80776d0 0x7f00a819f930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:32:56.040 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:56.037+0000 7f009fe00640 1 -- 172.21.15.12:0/2281358280 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f00a80776d0 msgr2=0x7f00a819f930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:32:56.040 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:56.037+0000 7f009fe00640 1 --2- 172.21.15.12:0/2281358280 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f00a80776d0 0x7f00a819f930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:32:56.040 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:56.037+0000 7f009fe00640 1 -- 172.21.15.12:0/2281358280 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f00a8113990 msgr2=0x7f00a81a5210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:32:56.040 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:56.037+0000 7f009fe00640 1 --2- 172.21.15.12:0/2281358280 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f00a8113990 0x7f00a81a5210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:32:56.040 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:56.037+0000 7f009fe00640 1 -- 172.21.15.12:0/2281358280 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00a81a59e0 con 0x7f00a8077ff0 2024-09-11T00:32:56.040 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:56.037+0000 7f00ad000640 1 --2- 172.21.15.12:0/2281358280 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f00a8113990 0x7f00a81a5210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T00:32:56.040 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:56.037+0000 7f00a7e00640 1 --2- 172.21.15.12:0/2281358280 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f00a80776d0 0x7f00a819f930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T00:32:56.041 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:56.037+0000 7f009fe00640 1 --2- 172.21.15.12:0/2281358280 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f00a8077ff0 0x7f00a819fe70 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f009000e9f0 tx=0x7f009000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:32:56.041 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:56.037+0000 7f00a5600640 1 -- 172.21.15.12:0/2281358280 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f009000cd80 con 0x7f00a8077ff0 2024-09-11T00:32:56.041 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:56.037+0000 7f00a5600640 1 -- 172.21.15.12:0/2281358280 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f00900054d0 con 0x7f00a8077ff0 2024-09-11T00:32:56.041 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:56.037+0000 7f00a5600640 1 -- 172.21.15.12:0/2281358280 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0090010690 con 0x7f00a8077ff0 2024-09-11T00:32:56.041 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:56.037+0000 7f00af000640 1 -- 172.21.15.12:0/2281358280 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f00a81a5cd0 con 0x7f00a8077ff0 2024-09-11T00:32:56.042 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:56.038+0000 7f00af000640 1 -- 172.21.15.12:0/2281358280 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00a81ac390 con 0x7f00a8077ff0 2024-09-11T00:32:56.042 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:56.039+0000 7f00af000640 1 -- 172.21.15.12:0/2281358280 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f00a8077ab0 con 0x7f00a8077ff0 2024-09-11T00:32:56.043 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:56.041+0000 7f00a5600640 1 -- 172.21.15.12:0/2281358280 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7f0090004e90 con 0x7f00a8077ff0 2024-09-11T00:32:56.045 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:56.042+0000 7f00a5600640 1 --2- 172.21.15.12:0/2281358280 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f008c0b52e0 0x7f008c0b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:32:56.045 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:56.042+0000 7f00a5600640 1 -- 172.21.15.12:0/2281358280 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9208+0+0 (secure 0 0 0) 0x7f00900e6570 con 0x7f00a8077ff0 2024-09-11T00:32:56.045 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:56.043+0000 7f00a7e00640 1 --2- 172.21.15.12:0/2281358280 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f008c0b52e0 0x7f008c0b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:32:56.045 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:56.043+0000 7f00a7e00640 1 --2- 172.21.15.12:0/2281358280 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f008c0b52e0 0x7f008c0b77a0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f0098000c00 tx=0x7f0098031400 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:32:56.048 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:56.046+0000 7f00a5600640 1 -- 172.21.15.12:0/2281358280 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f00900ad000 con 0x7f00a8077ff0 2024-09-11T00:32:56.245 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:56.243+0000 7f00af000640 1 -- 172.21.15.12:0/2281358280 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f00a81ac7a0 con 0x7f00a8077ff0 2024-09-11T00:32:56.247 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:56.245+0000 7f00a5600640 1 -- 172.21.15.12:0/2281358280 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) ==== 74+0+24948 (secure 0 0 0) 0x7f00900ac900 con 0x7f00a8077ff0 2024-09-11T00:32:56.247 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-11T00:32:56.248 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":87,"fsid":"ff3ad74a-6fcf-11ef-bcea-c7b262605968","created":"2024-09-10T23:55:16.317643+0000","modified":"2024-09-11T00:28:05.342224+0000","last_up_change":"2024-09-11T00:03:08.959256+0000","last_in_change":"2024-09-11T00:02:46.564631+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T23:59:10.488728+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-11T00:04:00.065618+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"74","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-11T00:04:02.689888+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"74","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-11T00:04:09.862120+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"6382c200-a912-42a1-958b-79191b21249b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6802","nonce":1237101991},{"type":"v1","addr":"172.21.15.12:6803","nonce":1237101991}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6804","nonce":1237101991},{"type":"v1","addr":"172.21.15.12:6805","nonce":1237101991}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6808","nonce":1237101991},{"type":"v1","addr":"172.21.15.12:6809","nonce":1237101991}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6806","nonce":1237101991},{"type":"v1","addr":"172.21.15.12:6807","nonce":1237101991}]},"public_addr":"172.21.15.12:6803/1237101991","cluster_addr":"172.21.15.12:6805/1237101991","heartbeat_back_addr":"172.21.15.12:6809/1237101991","heartbeat_front_addr":"172.21.15.12:6807/1237101991","state":["exists","up"]},{"osd":1,"uuid":"fe0e1d0a-bdc5-4859-aa84-9038059a45f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6800","nonce":545848456},{"type":"v1","addr":"172.21.15.165:6801","nonce":545848456}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6802","nonce":545848456},{"type":"v1","addr":"172.21.15.165:6803","nonce":545848456}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6806","nonce":545848456},{"type":"v1","addr":"172.21.15.165:6807","nonce":545848456}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6804","nonce":545848456},{"type":"v1","addr":"172.21.15.165:6805","nonce":545848456}]},"public_addr":"172.21.15.165:6801/545848456","cluster_addr":"172.21.15.165:6803/545848456","heartbeat_back_addr":"172.21.15.165:6807/545848456","heartbeat_front_addr":"172.21.15.165:6805/545848456","state":["exists","up"]},{"osd":2,"uuid":"94f640dc-5f78-4e2c-87d9-03e96a6e7ba0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6800","nonce":4131162695},{"type":"v1","addr":"172.21.15.178:6801","nonce":4131162695}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":4131162695},{"type":"v1","addr":"172.21.15.178:6803","nonce":4131162695}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":4131162695},{"type":"v1","addr":"172.21.15.178:6807","nonce":4131162695}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":4131162695},{"type":"v1","addr":"172.21.15.178:6805","nonce":4131162695}]},"public_addr":"172.21.15.178:6801/4131162695","cluster_addr":"172.21.15.178:6803/4131162695","heartbeat_back_addr":"172.21.15.178:6807/4131162695","heartbeat_front_addr":"172.21.15.178:6805/4131162695","state":["exists","up"]},{"osd":3,"uuid":"af2f64d6-c9be-44a9-b041-8d599a7c080d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6810","nonce":452464245},{"type":"v1","addr":"172.21.15.12:6811","nonce":452464245}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6812","nonce":452464245},{"type":"v1","addr":"172.21.15.12:6813","nonce":452464245}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6816","nonce":452464245},{"type":"v1","addr":"172.21.15.12:6817","nonce":452464245}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6814","nonce":452464245},{"type":"v1","addr":"172.21.15.12:6815","nonce":452464245}]},"public_addr":"172.21.15.12:6811/452464245","cluster_addr":"172.21.15.12:6813/452464245","heartbeat_back_addr":"172.21.15.12:6817/452464245","heartbeat_front_addr":"172.21.15.12:6815/452464245","state":["exists","up"]},{"osd":4,"uuid":"ad76ac70-20c0-44ae-9866-5f9cfce2334c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6808","nonce":1899879280},{"type":"v1","addr":"172.21.15.165:6809","nonce":1899879280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6810","nonce":1899879280},{"type":"v1","addr":"172.21.15.165:6811","nonce":1899879280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6814","nonce":1899879280},{"type":"v1","addr":"172.21.15.165:6815","nonce":1899879280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6812","nonce":1899879280},{"type":"v1","addr":"172.21.15.165:6813","nonce":1899879280}]},"public_addr":"172.21.15.165:6809/1899879280","cluster_addr":"172.21.15.165:6811/1899879280","heartbeat_back_addr":"172.21.15.165:6815/1899879280","heartbeat_front_addr":"172.21.15.165:6813/1899879280","state":["exists","up"]},{"osd":5,"uuid":"aafda2cc-62fe-4fcb-9a96-8732e0b501d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":3042228555},{"type":"v1","addr":"172.21.15.178:6809","nonce":3042228555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":3042228555},{"type":"v1","addr":"172.21.15.178:6811","nonce":3042228555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6814","nonce":3042228555},{"type":"v1","addr":"172.21.15.178:6815","nonce":3042228555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":3042228555},{"type":"v1","addr":"172.21.15.178:6813","nonce":3042228555}]},"public_addr":"172.21.15.178:6809/3042228555","cluster_addr":"172.21.15.178:6811/3042228555","heartbeat_back_addr":"172.21.15.178:6815/3042228555","heartbeat_front_addr":"172.21.15.178:6813/3042228555","state":["exists","up"]},{"osd":6,"uuid":"375eca52-efe3-42a5-bbb1-d7d392d71f15","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6818","nonce":3868322467},{"type":"v1","addr":"172.21.15.12:6819","nonce":3868322467}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6820","nonce":3868322467},{"type":"v1","addr":"172.21.15.12:6821","nonce":3868322467}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6824","nonce":3868322467},{"type":"v1","addr":"172.21.15.12:6825","nonce":3868322467}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6822","nonce":3868322467},{"type":"v1","addr":"172.21.15.12:6823","nonce":3868322467}]},"public_addr":"172.21.15.12:6819/3868322467","cluster_addr":"172.21.15.12:6821/3868322467","heartbeat_back_addr":"172.21.15.12:6825/3868322467","heartbeat_front_addr":"172.21.15.12:6823/3868322467","state":["exists","up"]},{"osd":7,"uuid":"0fd619f5-44c3-4cf7-ae7a-388dd798de36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6816","nonce":1143773735},{"type":"v1","addr":"172.21.15.165:6817","nonce":1143773735}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6818","nonce":1143773735},{"type":"v1","addr":"172.21.15.165:6819","nonce":1143773735}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6822","nonce":1143773735},{"type":"v1","addr":"172.21.15.165:6823","nonce":1143773735}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6820","nonce":1143773735},{"type":"v1","addr":"172.21.15.165:6821","nonce":1143773735}]},"public_addr":"172.21.15.165:6817/1143773735","cluster_addr":"172.21.15.165:6819/1143773735","heartbeat_back_addr":"172.21.15.165:6823/1143773735","heartbeat_front_addr":"172.21.15.165:6821/1143773735","state":["exists","up"]},{"osd":8,"uuid":"b03d14ce-8427-462a-9bcc-27683c549f53","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":2891445058},{"type":"v1","addr":"172.21.15.178:6817","nonce":2891445058}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6818","nonce":2891445058},{"type":"v1","addr":"172.21.15.178:6819","nonce":2891445058}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6822","nonce":2891445058},{"type":"v1","addr":"172.21.15.178:6823","nonce":2891445058}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":2891445058},{"type":"v1","addr":"172.21.15.178:6821","nonce":2891445058}]},"public_addr":"172.21.15.178:6817/2891445058","cluster_addr":"172.21.15.178:6819/2891445058","heartbeat_back_addr":"172.21.15.178:6823/2891445058","heartbeat_front_addr":"172.21.15.178:6821/2891445058","state":["exists","up"]},{"osd":9,"uuid":"81a6cf69-2a24-4a6c-9adf-6b7df38cfacb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6826","nonce":3359634903},{"type":"v1","addr":"172.21.15.12:6827","nonce":3359634903}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6828","nonce":3359634903},{"type":"v1","addr":"172.21.15.12:6829","nonce":3359634903}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6832","nonce":3359634903},{"type":"v1","addr":"172.21.15.12:6833","nonce":3359634903}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6830","nonce":3359634903},{"type":"v1","addr":"172.21.15.12:6831","nonce":3359634903}]},"public_addr":"172.21.15.12:6827/3359634903","cluster_addr":"172.21.15.12:6829/3359634903","heartbeat_back_addr":"172.21.15.12:6833/3359634903","heartbeat_front_addr":"172.21.15.12:6831/3359634903","state":["exists","up"]},{"osd":10,"uuid":"c1ad1f0a-33f4-4c32-91f6-4d030a29f1dd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6824","nonce":3642325778},{"type":"v1","addr":"172.21.15.165:6825","nonce":3642325778}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6826","nonce":3642325778},{"type":"v1","addr":"172.21.15.165:6827","nonce":3642325778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6830","nonce":3642325778},{"type":"v1","addr":"172.21.15.165:6831","nonce":3642325778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6828","nonce":3642325778},{"type":"v1","addr":"172.21.15.165:6829","nonce":3642325778}]},"public_addr":"172.21.15.165:6825/3642325778","cluster_addr":"172.21.15.165:6827/3642325778","heartbeat_back_addr":"172.21.15.165:6831/3642325778","heartbeat_front_addr":"172.21.15.165:6829/3642325778","state":["exists","up"]},{"osd":11,"uuid":"8413c9b0-680b-4d02-aeab-b3b3ff16c3d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":2937059756},{"type":"v1","addr":"172.21.15.178:6825","nonce":2937059756}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6826","nonce":2937059756},{"type":"v1","addr":"172.21.15.178:6827","nonce":2937059756}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":2937059756},{"type":"v1","addr":"172.21.15.178:6831","nonce":2937059756}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":2937059756},{"type":"v1","addr":"172.21.15.178:6829","nonce":2937059756}]},"public_addr":"172.21.15.178:6825/2937059756","cluster_addr":"172.21.15.178:6827/2937059756","heartbeat_back_addr":"172.21.15.178:6831/2937059756","heartbeat_front_addr":"172.21.15.178:6829/2937059756","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:58:16.036348+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:58:41.483150+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:59:07.107283+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T23:59:35.706574+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T00:00:02.555720+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T00:00:28.092847+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T00:00:57.396495+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T00:01:24.216775+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T00:01:48.807850+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T00:02:16.838565+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T00:02:43.131736+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T00:03:07.353155+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":8,"to":4}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"3.e","mappings":[{"from":5,"to":9}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9}]},{"pgid":"3.23","mappings":[{"from":4,"to":7}]},{"pgid":"3.31","mappings":[{"from":4,"to":8}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"3.39","mappings":[{"from":5,"to":6}]},{"pgid":"4.5","mappings":[{"from":9,"to":10}]},{"pgid":"4.6","mappings":[{"from":5,"to":8}]},{"pgid":"4.b","mappings":[{"from":9,"to":8}]},{"pgid":"4.13","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.12:6801/4177251511":"2024-09-11T23:56:17.999820+0000","172.21.15.12:6800/4177251511":"2024-09-11T23:56:17.999820+0000","172.21.15.12:0/3273692817":"2024-09-11T23:56:17.999820+0000","172.21.15.12:0/4017778158":"2024-09-11T23:56:17.999820+0000","172.21.15.12:0/2499921051":"2024-09-11T23:55:50.302721+0000","172.21.15.12:6801/760140910":"2024-09-11T23:55:50.302721+0000","172.21.15.12:6800/760140910":"2024-09-11T23:55:50.302721+0000","172.21.15.12:0/2810703302":"2024-09-11T23:56:17.999820+0000","172.21.15.12:0/2325436065":"2024-09-11T23:55:50.302721+0000","172.21.15.12:0/3137053479":"2024-09-11T23:55:50.302721+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-11T00:32:56.250 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:56.247+0000 7f00af000640 1 -- 172.21.15.12:0/2281358280 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f008c0b52e0 msgr2=0x7f008c0b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:32:56.250 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:56.247+0000 7f00af000640 1 --2- 172.21.15.12:0/2281358280 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f008c0b52e0 0x7f008c0b77a0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f0098000c00 tx=0x7f0098031400 comp rx=0 tx=0).stop 2024-09-11T00:32:56.250 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:56.247+0000 7f00af000640 1 -- 172.21.15.12:0/2281358280 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f00a8077ff0 msgr2=0x7f00a819fe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:32:56.250 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:56.247+0000 7f00af000640 1 --2- 172.21.15.12:0/2281358280 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f00a8077ff0 0x7f00a819fe70 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f009000e9f0 tx=0x7f009000edf0 comp rx=0 tx=0).stop 2024-09-11T00:32:56.250 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:56.247+0000 7f00af000640 1 -- 172.21.15.12:0/2281358280 shutdown_connections 2024-09-11T00:32:56.250 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:56.247+0000 7f00af000640 1 --2- 172.21.15.12:0/2281358280 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f008c0b52e0 0x7f008c0b77a0 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:32:56.251 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:56.247+0000 7f00af000640 1 --2- 172.21.15.12:0/2281358280 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f00a8113990 0x7f00a81a5210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:32:56.251 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:56.247+0000 7f00af000640 1 --2- 172.21.15.12:0/2281358280 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f00a8077ff0 0x7f00a819fe70 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:32:56.251 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:56.247+0000 7f00af000640 1 --2- 172.21.15.12:0/2281358280 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f00a80776d0 0x7f00a819f930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:32:56.251 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:56.247+0000 7f00af000640 1 -- 172.21.15.12:0/2281358280 >> 172.21.15.12:0/2281358280 conn(0x7f00a8100a50 msgr2=0x7f00a8102e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:32:56.251 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:56.248+0000 7f00af000640 1 -- 172.21.15.12:0/2281358280 shutdown_connections 2024-09-11T00:32:56.251 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:56.248+0000 7f00af000640 1 -- 172.21.15.12:0/2281358280 wait complete. 2024-09-11T00:32:56.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:32:56 smithi012 ceph-mon[30333]: pgmap v1175: 129 pgs: 129 active+clean; 671 MiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.3 KiB/s wr, 11 op/s 2024-09-11T00:32:56.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:32:56 smithi178 ceph-mon[35919]: pgmap v1175: 129 pgs: 129 active+clean; 671 MiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.3 KiB/s wr, 11 op/s 2024-09-11T00:32:56.533 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:32:56 smithi165 ceph-mon[35356]: pgmap v1175: 129 pgs: 129 active+clean; 671 MiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.3 KiB/s wr, 11 op/s 2024-09-11T00:32:57.130 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid ff3ad74a-6fcf-11ef-bcea-c7b262605968 -- ceph fs fail cephfs 2024-09-11T00:32:57.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:32:57 smithi178 ceph-mon[35919]: from='client.? 172.21.15.12:0/2281358280' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-11T00:32:57.513 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:32:57.532 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:32:57 smithi165 ceph-mon[35356]: from='client.? 172.21.15.12:0/2281358280' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-11T00:32:57.563 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:32:57 smithi012 ceph-mon[30333]: from='client.? 172.21.15.12:0/2281358280' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-11T00:32:58.116 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:58.112+0000 7f6ce3200640 1 -- 172.21.15.12:0/1995058018 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6cdc101220 msgr2=0x7f6cdc101600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:32:58.116 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:58.112+0000 7f6ce3200640 1 --2- 172.21.15.12:0/1995058018 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6cdc101220 0x7f6cdc101600 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f6ccc009900 tx=0x7f6ccc02a410 comp rx=0 tx=0).stop 2024-09-11T00:32:58.116 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:58.112+0000 7f6ce3200640 1 -- 172.21.15.12:0/1995058018 shutdown_connections 2024-09-11T00:32:58.116 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:58.113+0000 7f6ce3200640 1 --2- 172.21.15.12:0/1995058018 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6cdc105ef0 0x7f6cdc10f930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:32:58.116 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:58.113+0000 7f6ce3200640 1 --2- 172.21.15.12:0/1995058018 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6cdc105570 0x7f6cdc1059b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:32:58.116 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:58.113+0000 7f6ce3200640 1 --2- 172.21.15.12:0/1995058018 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6cdc101220 0x7f6cdc101600 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:32:58.116 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:58.113+0000 7f6ce3200640 1 -- 172.21.15.12:0/1995058018 >> 172.21.15.12:0/1995058018 conn(0x7f6cdc078100 msgr2=0x7f6cdc0ff4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:32:58.116 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:58.113+0000 7f6ce3200640 1 -- 172.21.15.12:0/1995058018 shutdown_connections 2024-09-11T00:32:58.117 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:58.113+0000 7f6ce3200640 1 -- 172.21.15.12:0/1995058018 wait complete. 2024-09-11T00:32:58.118 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:58.114+0000 7f6ce3200640 1 Processor -- start 2024-09-11T00:32:58.118 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:58.114+0000 7f6ce3200640 1 -- start start 2024-09-11T00:32:58.118 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:58.114+0000 7f6ce3200640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6cdc101220 0x7f6cdc1021f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:32:58.118 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:58.115+0000 7f6ce3200640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6cdc105570 0x7f6cdc102730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:32:58.118 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:58.115+0000 7f6ce3200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6cdc105ef0 0x7f6cdc102c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:32:58.118 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:58.115+0000 7f6ce3200640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f6cdc112df0 con 0x7f6cdc105570 2024-09-11T00:32:58.118 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:58.115+0000 7f6ce3200640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f6cdc112c70 con 0x7f6cdc105ef0 2024-09-11T00:32:58.119 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:58.115+0000 7f6ce3200640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f6cdc112f70 con 0x7f6cdc101220 2024-09-11T00:32:58.119 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:58.115+0000 7f6cdb400640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6cdc105570 0x7f6cdc102730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:32:58.119 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:58.115+0000 7f6cdb400640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6cdc105570 0x7f6cdc102730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:60846/0 (socket says 172.21.15.12:60846) 2024-09-11T00:32:58.119 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:58.115+0000 7f6cdb400640 1 -- 172.21.15.12:0/428288020 learned_addr learned my addr 172.21.15.12:0/428288020 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:32:58.119 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:58.115+0000 7f6cdbe00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6cdc101220 0x7f6cdc1021f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:32:58.119 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:58.115+0000 7f6ce1200640 1 --2- 172.21.15.12:0/428288020 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6cdc105ef0 0x7f6cdc102c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:32:58.119 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:58.115+0000 7f6cdbe00640 1 -- 172.21.15.12:0/428288020 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6cdc105ef0 msgr2=0x7f6cdc102c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:32:58.119 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:58.115+0000 7f6cdbe00640 1 --2- 172.21.15.12:0/428288020 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6cdc105ef0 0x7f6cdc102c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:32:58.119 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:58.115+0000 7f6cdbe00640 1 -- 172.21.15.12:0/428288020 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6cdc105570 msgr2=0x7f6cdc102730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:32:58.119 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:58.115+0000 7f6cdbe00640 1 --2- 172.21.15.12:0/428288020 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6cdc105570 0x7f6cdc102730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:32:58.119 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:58.115+0000 7f6cdbe00640 1 -- 172.21.15.12:0/428288020 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6cdc1a76c0 con 0x7f6cdc101220 2024-09-11T00:32:58.119 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:58.116+0000 7f6cdbe00640 1 --2- 172.21.15.12:0/428288020 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6cdc101220 0x7f6cdc1021f0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f6ccc02a930 tx=0x7f6ccc031a20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:32:58.120 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:58.116+0000 7f6cd8c00640 1 -- 172.21.15.12:0/428288020 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ccc03f030 con 0x7f6cdc101220 2024-09-11T00:32:58.120 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:58.116+0000 7f6cd8c00640 1 -- 172.21.15.12:0/428288020 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6ccc004490 con 0x7f6cdc101220 2024-09-11T00:32:58.120 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:58.116+0000 7f6cd8c00640 1 -- 172.21.15.12:0/428288020 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ccc035070 con 0x7f6cdc101220 2024-09-11T00:32:58.120 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:58.116+0000 7f6ce3200640 1 -- 172.21.15.12:0/428288020 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6cdc1a7950 con 0x7f6cdc101220 2024-09-11T00:32:58.120 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:58.116+0000 7f6ce3200640 1 -- 172.21.15.12:0/428288020 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6cdc1a7de0 con 0x7f6cdc101220 2024-09-11T00:32:58.120 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:58.117+0000 7f6ce3200640 1 -- 172.21.15.12:0/428288020 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6cdc101600 con 0x7f6cdc101220 2024-09-11T00:32:58.122 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:58.120+0000 7f6cd8c00640 1 -- 172.21.15.12:0/428288020 <== mon.1 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7f6ccc03ba30 con 0x7f6cdc101220 2024-09-11T00:32:58.124 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:58.121+0000 7f6cd8c00640 1 --2- 172.21.15.12:0/428288020 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f6cc00b5360 0x7f6cc00b7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:32:58.124 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:58.122+0000 7f6cd8c00640 1 -- 172.21.15.12:0/428288020 <== mon.1 v2:172.21.15.178:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9208+0+0 (secure 0 0 0) 0x7f6ccc106730 con 0x7f6cdc101220 2024-09-11T00:32:58.124 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:58.122+0000 7f6cdb400640 1 --2- 172.21.15.12:0/428288020 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f6cc00b5360 0x7f6cc00b7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:32:58.125 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:58.122+0000 7f6cdb400640 1 --2- 172.21.15.12:0/428288020 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f6cc00b5360 0x7f6cc00b7820 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f6cdc103ce0 tx=0x7f6cc8005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:32:58.125 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:58.123+0000 7f6cd8c00640 1 -- 172.21.15.12:0/428288020 <== mon.1 v2:172.21.15.178:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f6ccc0ccb70 con 0x7f6cdc101220 2024-09-11T00:32:58.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:32:58 smithi012 ceph-mon[30333]: pgmap v1176: 129 pgs: 129 active+clean; 671 MiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 1.6 KiB/s wr, 13 op/s 2024-09-11T00:32:58.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:32:58 smithi178 ceph-mon[35919]: pgmap v1176: 129 pgs: 129 active+clean; 671 MiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 1.6 KiB/s wr, 13 op/s 2024-09-11T00:32:58.360 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:58.357+0000 7f6ce3200640 1 -- 172.21.15.12:0/428288020 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f6cdc02e5c0 con 0x7f6cdc101220 2024-09-11T00:32:58.371 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:58.368+0000 7f6cd8c00640 1 -- 172.21.15.12:0/428288020 <== mon.1 v2:172.21.15.178:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v113) ==== 161+0+0 (secure 0 0 0) 0x7f6ccc0d15a0 con 0x7f6cdc101220 2024-09-11T00:32:58.371 INFO:teuthology.orchestra.run.smithi012.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-11T00:32:58.374 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:58.372+0000 7f6ce3200640 1 -- 172.21.15.12:0/428288020 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f6cc00b5360 msgr2=0x7f6cc00b7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:32:58.374 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:58.372+0000 7f6ce3200640 1 --2- 172.21.15.12:0/428288020 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f6cc00b5360 0x7f6cc00b7820 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f6cdc103ce0 tx=0x7f6cc8005f90 comp rx=0 tx=0).stop 2024-09-11T00:32:58.374 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:58.372+0000 7f6ce3200640 1 -- 172.21.15.12:0/428288020 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6cdc101220 msgr2=0x7f6cdc1021f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:32:58.375 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:58.372+0000 7f6ce3200640 1 --2- 172.21.15.12:0/428288020 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6cdc101220 0x7f6cdc1021f0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f6ccc02a930 tx=0x7f6ccc031a20 comp rx=0 tx=0).stop 2024-09-11T00:32:58.375 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:58.372+0000 7f6ce3200640 1 -- 172.21.15.12:0/428288020 shutdown_connections 2024-09-11T00:32:58.375 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:58.372+0000 7f6ce3200640 1 --2- 172.21.15.12:0/428288020 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f6cc00b5360 0x7f6cc00b7820 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:32:58.375 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:58.372+0000 7f6ce3200640 1 --2- 172.21.15.12:0/428288020 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6cdc105ef0 0x7f6cdc102c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:32:58.375 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:58.372+0000 7f6ce3200640 1 --2- 172.21.15.12:0/428288020 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6cdc105570 0x7f6cdc102730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:32:58.375 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:58.372+0000 7f6ce3200640 1 --2- 172.21.15.12:0/428288020 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6cdc101220 0x7f6cdc1021f0 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:32:58.375 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:58.372+0000 7f6ce3200640 1 -- 172.21.15.12:0/428288020 >> 172.21.15.12:0/428288020 conn(0x7f6cdc078100 msgr2=0x7f6cdc0ff4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:32:58.375 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:58.372+0000 7f6ce3200640 1 -- 172.21.15.12:0/428288020 shutdown_connections 2024-09-11T00:32:58.375 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:32:58.372+0000 7f6ce3200640 1 -- 172.21.15.12:0/428288020 wait complete. 2024-09-11T00:32:58.436 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:32:58 smithi165 ceph-mon[35356]: pgmap v1176: 129 pgs: 129 active+clean; 671 MiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 1.6 KiB/s wr, 13 op/s 2024-09-11T00:32:58.783 INFO:journalctl@ceph.mds.k.smithi165.stdout:Sep 11 00:32:58 smithi165 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-mds-k[64659]: ignoring --setuser ceph since I am not root 2024-09-11T00:32:58.783 INFO:journalctl@ceph.mds.k.smithi165.stdout:Sep 11 00:32:58 smithi165 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-mds-k[64659]: ignoring --setgroup ceph since I am not root 2024-09-11T00:32:58.783 INFO:journalctl@ceph.mds.k.smithi165.stdout:Sep 11 00:32:58 smithi165 ceph-mds[64663]: -- 172.21.15.165:0/2095274478 <== mon.2 v2:172.21.15.165:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55a2bdc25860 con 0x55a2bdc02800 2024-09-11T00:32:58.783 INFO:journalctl@ceph.mds.k.smithi165.stdout:Sep 11 00:32:58 smithi165 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-mds-k[64659]: starting mds.k at 2024-09-11T00:32:58.783 INFO:journalctl@ceph.mds.b.smithi165.stdout:Sep 11 00:32:58 smithi165 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-mds-b[65581]: ignoring --setuser ceph since I am not root 2024-09-11T00:32:58.783 INFO:journalctl@ceph.mds.b.smithi165.stdout:Sep 11 00:32:58 smithi165 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-mds-b[65581]: ignoring --setgroup ceph since I am not root 2024-09-11T00:32:58.783 INFO:journalctl@ceph.mds.b.smithi165.stdout:Sep 11 00:32:58 smithi165 ceph-mds[65585]: -- 172.21.15.165:0/2587588606 <== mon.1 v2:172.21.15.178:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55f4ca19f860 con 0x55f4ca17d000 2024-09-11T00:32:58.783 INFO:journalctl@ceph.mds.b.smithi165.stdout:Sep 11 00:32:58 smithi165 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-mds-b[65581]: starting mds.b at 2024-09-11T00:32:58.784 INFO:journalctl@ceph.mds.e.smithi165.stdout:Sep 11 00:32:58 smithi165 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-mds-e[64969]: ignoring --setuser ceph since I am not root 2024-09-11T00:32:58.784 INFO:journalctl@ceph.mds.e.smithi165.stdout:Sep 11 00:32:58 smithi165 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-mds-e[64969]: ignoring --setgroup ceph since I am not root 2024-09-11T00:32:58.784 INFO:journalctl@ceph.mds.e.smithi165.stdout:Sep 11 00:32:58 smithi165 ceph-mds[64973]: -- 172.21.15.165:0/4025530918 <== mon.1 v2:172.21.15.178:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55fb58d9dc20 con 0x55fb58d7a800 2024-09-11T00:32:58.784 INFO:journalctl@ceph.mds.e.smithi165.stdout:Sep 11 00:32:58 smithi165 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-mds-e[64969]: starting mds.e at 2024-09-11T00:32:58.784 INFO:journalctl@ceph.mds.h.smithi165.stdout:Sep 11 00:32:58 smithi165 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-mds-h[65283]: ignoring --setuser ceph since I am not root 2024-09-11T00:32:58.784 INFO:journalctl@ceph.mds.h.smithi165.stdout:Sep 11 00:32:58 smithi165 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-mds-h[65283]: ignoring --setgroup ceph since I am not root 2024-09-11T00:32:58.784 INFO:journalctl@ceph.mds.h.smithi165.stdout:Sep 11 00:32:58 smithi165 ceph-mds[65287]: -- 172.21.15.165:0/2631970213 <== mon.2 v2:172.21.15.165:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x560c5407b860 con 0x560c54059400 2024-09-11T00:32:58.784 INFO:journalctl@ceph.mds.h.smithi165.stdout:Sep 11 00:32:58 smithi165 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-mds-h[65283]: starting mds.h at 2024-09-11T00:32:58.819 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:32:58 smithi012 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-mon-a[30329]: 2024-09-11T00:32:58.359+0000 7f0463c00640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-11T00:32:58.819 INFO:journalctl@ceph.mds.a.smithi012.stdout:Sep 11 00:32:58 smithi012 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-mds-a[73335]: ignoring --setuser ceph since I am not root 2024-09-11T00:32:58.820 INFO:journalctl@ceph.mds.a.smithi012.stdout:Sep 11 00:32:58 smithi012 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-mds-a[73335]: ignoring --setgroup ceph since I am not root 2024-09-11T00:32:58.820 INFO:journalctl@ceph.mds.a.smithi012.stdout:Sep 11 00:32:58 smithi012 ceph-mds[73339]: -- 172.21.15.12:0/2231797096 <== mon.1 v2:172.21.15.178:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55c167d11860 con 0x55c167cef400 2024-09-11T00:32:58.820 INFO:journalctl@ceph.mds.a.smithi012.stdout:Sep 11 00:32:58 smithi012 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-mds-a[73335]: starting mds.a at 2024-09-11T00:32:58.820 INFO:journalctl@ceph.mds.g.smithi012.stdout:Sep 11 00:32:58 smithi012 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-mds-g[72952]: ignoring --setuser ceph since I am not root 2024-09-11T00:32:58.820 INFO:journalctl@ceph.mds.g.smithi012.stdout:Sep 11 00:32:58 smithi012 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-mds-g[72952]: ignoring --setgroup ceph since I am not root 2024-09-11T00:32:58.820 INFO:journalctl@ceph.mds.g.smithi012.stdout:Sep 11 00:32:58 smithi012 ceph-mds[72966]: -- 172.21.15.12:0/1395140304 <== mon.2 v2:172.21.15.165:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55b05ce1f860 con 0x55b05cdfc800 2024-09-11T00:32:58.820 INFO:journalctl@ceph.mds.g.smithi012.stdout:Sep 11 00:32:58 smithi012 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-mds-g[72952]: starting mds.g at 2024-09-11T00:32:58.831 INFO:journalctl@ceph.mds.i.smithi178.stdout:Sep 11 00:32:58 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-mds-i[66387]: ignoring --setuser ceph since I am not root 2024-09-11T00:32:58.831 INFO:journalctl@ceph.mds.i.smithi178.stdout:Sep 11 00:32:58 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-mds-i[66387]: ignoring --setgroup ceph since I am not root 2024-09-11T00:32:58.832 INFO:journalctl@ceph.mds.i.smithi178.stdout:Sep 11 00:32:58 smithi178 ceph-mds[66391]: -- 172.21.15.178:0/51299025 <== mon.2 v2:172.21.15.165:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55f44ed09860 con 0x55f44ece6800 2024-09-11T00:32:58.832 INFO:journalctl@ceph.mds.i.smithi178.stdout:Sep 11 00:32:58 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-mds-i[66387]: starting mds.i at 2024-09-11T00:32:58.832 INFO:journalctl@ceph.mds.c.smithi178.stdout:Sep 11 00:32:58 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-mds-c[66074]: ignoring --setuser ceph since I am not root 2024-09-11T00:32:58.832 INFO:journalctl@ceph.mds.c.smithi178.stdout:Sep 11 00:32:58 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-mds-c[66074]: ignoring --setgroup ceph since I am not root 2024-09-11T00:32:58.832 INFO:journalctl@ceph.mds.c.smithi178.stdout:Sep 11 00:32:58 smithi178 ceph-mds[66078]: -- 172.21.15.178:0/1812627416 <== mon.1 v2:172.21.15.178:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5646de795c20 con 0x5646de772800 2024-09-11T00:32:58.832 INFO:journalctl@ceph.mds.c.smithi178.stdout:Sep 11 00:32:58 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-mds-c[66074]: starting mds.c at 2024-09-11T00:32:58.833 INFO:journalctl@ceph.mds.l.smithi178.stdout:Sep 11 00:32:58 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-mds-l[65476]: ignoring --setuser ceph since I am not root 2024-09-11T00:32:58.833 INFO:journalctl@ceph.mds.l.smithi178.stdout:Sep 11 00:32:58 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-mds-l[65476]: ignoring --setgroup ceph since I am not root 2024-09-11T00:32:58.833 INFO:journalctl@ceph.mds.l.smithi178.stdout:Sep 11 00:32:58 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-mds-l[65476]: starting mds.l at 2024-09-11T00:32:58.833 INFO:journalctl@ceph.mds.f.smithi178.stdout:Sep 11 00:32:58 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-mds-f[65776]: ignoring --setuser ceph since I am not root 2024-09-11T00:32:58.833 INFO:journalctl@ceph.mds.f.smithi178.stdout:Sep 11 00:32:58 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-mds-f[65776]: ignoring --setgroup ceph since I am not root 2024-09-11T00:32:58.833 INFO:journalctl@ceph.mds.f.smithi178.stdout:Sep 11 00:32:58 smithi178 ceph-mds[65780]: -- 172.21.15.178:0/1972080630 <== mon.1 v2:172.21.15.178:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55c471365860 con 0x55c471342800 2024-09-11T00:32:58.833 INFO:journalctl@ceph.mds.f.smithi178.stdout:Sep 11 00:32:58 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-mds-f[65776]: starting mds.f at 2024-09-11T00:32:59.208 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid ff3ad74a-6fcf-11ef-bcea-c7b262605968 -- ceph fs rm cephfs --yes-i-really-mean-it 2024-09-11T00:32:59.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:32:59 smithi178 ceph-mon[35919]: from='client.? 172.21.15.12:0/428288020' entity='client.admin' cmd={"prefix": "fs fail", "fs_name": "cephfs"} : dispatch 2024-09-11T00:32:59.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:32:59 smithi178 ceph-mon[35919]: from='client.? ' entity='client.admin' cmd={"prefix": "fs fail", "fs_name": "cephfs"} : dispatch 2024-09-11T00:32:59.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:32:59 smithi178 ceph-mon[35919]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-09-11T00:32:59.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:32:59 smithi178 ceph-mon[35919]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-11T00:32:59.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:32:59 smithi178 ceph-mon[35919]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-09-11T00:32:59.331 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:32:59 smithi178 ceph-mon[35919]: osdmap e88: 12 total, 12 up, 12 in 2024-09-11T00:32:59.332 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:32:59 smithi178 ceph-mon[35919]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-09-11T00:32:59.533 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:32:59 smithi165 ceph-mon[35356]: from='client.? 172.21.15.12:0/428288020' entity='client.admin' cmd={"prefix": "fs fail", "fs_name": "cephfs"} : dispatch 2024-09-11T00:32:59.533 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:32:59 smithi165 ceph-mon[35356]: from='client.? ' entity='client.admin' cmd={"prefix": "fs fail", "fs_name": "cephfs"} : dispatch 2024-09-11T00:32:59.533 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:32:59 smithi165 ceph-mon[35356]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-09-11T00:32:59.533 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:32:59 smithi165 ceph-mon[35356]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-11T00:32:59.533 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:32:59 smithi165 ceph-mon[35356]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-09-11T00:32:59.533 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:32:59 smithi165 ceph-mon[35356]: osdmap e88: 12 total, 12 up, 12 in 2024-09-11T00:32:59.533 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:32:59 smithi165 ceph-mon[35356]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-09-11T00:32:59.563 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:32:59 smithi012 ceph-mon[30333]: from='client.? 172.21.15.12:0/428288020' entity='client.admin' cmd={"prefix": "fs fail", "fs_name": "cephfs"} : dispatch 2024-09-11T00:32:59.563 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:32:59 smithi012 ceph-mon[30333]: from='client.? ' entity='client.admin' cmd={"prefix": "fs fail", "fs_name": "cephfs"} : dispatch 2024-09-11T00:32:59.563 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:32:59 smithi012 ceph-mon[30333]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-09-11T00:32:59.563 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:32:59 smithi012 ceph-mon[30333]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-11T00:32:59.563 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:32:59 smithi012 ceph-mon[30333]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-09-11T00:32:59.563 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:32:59 smithi012 ceph-mon[30333]: osdmap e88: 12 total, 12 up, 12 in 2024-09-11T00:32:59.564 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:32:59 smithi012 ceph-mon[30333]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-09-11T00:32:59.593 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:33:00.173 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:33:00.156+0000 7f03a8e00640 1 -- 172.21.15.12:0/2176685054 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f03a4077ff0 msgr2=0x7f03a41132c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:33:00.173 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:33:00.156+0000 7f03a8e00640 1 --2- 172.21.15.12:0/2176685054 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f03a4077ff0 0x7f03a41132c0 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f0390009900 tx=0x7f039002a410 comp rx=0 tx=0).stop 2024-09-11T00:33:00.173 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:33:00.156+0000 7f03a8e00640 1 -- 172.21.15.12:0/2176685054 shutdown_connections 2024-09-11T00:33:00.174 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:33:00.156+0000 7f03a8e00640 1 --2- 172.21.15.12:0/2176685054 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f03a4113800 0x7f03a4115bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:33:00.174 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:33:00.156+0000 7f03a8e00640 1 --2- 172.21.15.12:0/2176685054 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f03a4077ff0 0x7f03a41132c0 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:33:00.174 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:33:00.156+0000 7f03a8e00640 1 --2- 172.21.15.12:0/2176685054 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f03a40776d0 0x7f03a4077ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:33:00.174 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:33:00.156+0000 7f03a8e00640 1 -- 172.21.15.12:0/2176685054 >> 172.21.15.12:0/2176685054 conn(0x7f03a4100840 msgr2=0x7f03a4102c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:33:00.174 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:33:00.156+0000 7f03a8e00640 1 -- 172.21.15.12:0/2176685054 shutdown_connections 2024-09-11T00:33:00.174 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:33:00.156+0000 7f03a8e00640 1 -- 172.21.15.12:0/2176685054 wait complete. 2024-09-11T00:33:00.174 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:33:00.157+0000 7f03a8e00640 1 Processor -- start 2024-09-11T00:33:00.174 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:33:00.157+0000 7f03a8e00640 1 -- start start 2024-09-11T00:33:00.174 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:33:00.158+0000 7f03a8e00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f03a40776d0 0x7f03a419f690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:33:00.174 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:33:00.158+0000 7f03a8e00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f03a4077ff0 0x7f03a419fbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:33:00.174 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:33:00.158+0000 7f03a8e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f03a4113800 0x7f03a41a4f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:33:00.174 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:33:00.158+0000 7f03a8e00640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f03a4119240 con 0x7f03a4077ff0 2024-09-11T00:33:00.175 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:33:00.158+0000 7f03a8e00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f03a41190c0 con 0x7f03a4113800 2024-09-11T00:33:00.175 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:33:00.158+0000 7f03a8e00640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f03a41193c0 con 0x7f03a40776d0 2024-09-11T00:33:00.175 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:33:00.158+0000 7f03a2a00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f03a4077ff0 0x7f03a419fbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:33:00.175 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:33:00.158+0000 7f03a3400640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f03a40776d0 0x7f03a419f690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:33:00.175 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:33:00.158+0000 7f03a3400640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f03a40776d0 0x7f03a419f690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.12:33152/0 (socket says 172.21.15.12:33152) 2024-09-11T00:33:00.175 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:33:00.158+0000 7f03a2a00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f03a4077ff0 0x7f03a419fbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:44746/0 (socket says 172.21.15.12:44746) 2024-09-11T00:33:00.175 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:33:00.158+0000 7f03a3400640 1 -- 172.21.15.12:0/500631599 learned_addr learned my addr 172.21.15.12:0/500631599 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:33:00.175 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:33:00.158+0000 7f03a3e00640 1 --2- 172.21.15.12:0/500631599 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f03a4113800 0x7f03a41a4f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:33:00.175 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:33:00.159+0000 7f03a2a00640 1 -- 172.21.15.12:0/500631599 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f03a40776d0 msgr2=0x7f03a419f690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:33:00.175 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:33:00.159+0000 7f03a2a00640 1 --2- 172.21.15.12:0/500631599 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f03a40776d0 0x7f03a419f690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:33:00.175 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:33:00.159+0000 7f03a2a00640 1 -- 172.21.15.12:0/500631599 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f03a4113800 msgr2=0x7f03a41a4f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:33:00.176 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:33:00.159+0000 7f03a2a00640 1 --2- 172.21.15.12:0/500631599 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f03a4113800 0x7f03a41a4f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:33:00.176 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:33:00.159+0000 7f03a2a00640 1 -- 172.21.15.12:0/500631599 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03a41a5740 con 0x7f03a4077ff0 2024-09-11T00:33:00.176 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:33:00.159+0000 7f03a3400640 1 --2- 172.21.15.12:0/500631599 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f03a40776d0 0x7f03a419f690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:33:00.176 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:33:00.159+0000 7f03a2a00640 1 --2- 172.21.15.12:0/500631599 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f03a4077ff0 0x7f03a419fbd0 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f039002aa00 tx=0x7f0390003cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:33:00.176 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:33:00.159+0000 7f039be00640 1 -- 172.21.15.12:0/500631599 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f039003f070 con 0x7f03a4077ff0 2024-09-11T00:33:00.176 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:33:00.159+0000 7f039be00640 1 -- 172.21.15.12:0/500631599 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0390031d60 con 0x7f03a4077ff0 2024-09-11T00:33:00.176 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:33:00.159+0000 7f039be00640 1 -- 172.21.15.12:0/500631599 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0390035030 con 0x7f03a4077ff0 2024-09-11T00:33:00.176 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:33:00.159+0000 7f03a8e00640 1 -- 172.21.15.12:0/500631599 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03a41a59d0 con 0x7f03a4077ff0 2024-09-11T00:33:00.176 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:33:00.159+0000 7f03a8e00640 1 -- 172.21.15.12:0/500631599 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03a41ac0f0 con 0x7f03a4077ff0 2024-09-11T00:33:00.176 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:33:00.160+0000 7f03a8e00640 1 -- 172.21.15.12:0/500631599 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f03a4077ab0 con 0x7f03a4077ff0 2024-09-11T00:33:00.176 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:33:00.163+0000 7f039be00640 1 -- 172.21.15.12:0/500631599 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7f039003b460 con 0x7f03a4077ff0 2024-09-11T00:33:00.176 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:33:00.164+0000 7f039be00640 1 --2- 172.21.15.12:0/500631599 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f03880b5410 0x7f03880b78d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:33:00.177 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:33:00.164+0000 7f039be00640 1 -- 172.21.15.12:0/500631599 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9638+0+0 (secure 0 0 0) 0x7f0390106b60 con 0x7f03a4077ff0 2024-09-11T00:33:00.177 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:33:00.164+0000 7f03a3400640 1 --2- 172.21.15.12:0/500631599 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f03880b5410 0x7f03880b78d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:33:00.177 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:33:00.165+0000 7f03a3400640 1 --2- 172.21.15.12:0/500631599 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f03880b5410 0x7f03880b78d0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f0394005fd0 tx=0x7f039400b4a0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:33:00.177 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:33:00.168+0000 7f039be00640 1 -- 172.21.15.12:0/500631599 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f03900ccdf0 con 0x7f03a4077ff0 2024-09-11T00:33:00.406 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:33:00.403+0000 7f03a8e00640 1 -- 172.21.15.12:0/500631599 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f03a41ac420 con 0x7f03a4077ff0 2024-09-11T00:33:00.783 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:33:00 smithi165 ceph-mon[35356]: pgmap v1178: 129 pgs: 129 active+clean; 671 MiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 1.9 KiB/s wr, 12 op/s 2024-09-11T00:33:00.783 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:33:00 smithi165 ceph-mon[35356]: mds.? [v2:172.21.15.178:6832/375852337,v1:172.21.15.178:6833/375852337] up:boot 2024-09-11T00:33:00.783 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:33:00 smithi165 ceph-mon[35356]: mds.? [v2:172.21.15.178:6834/3249074395,v1:172.21.15.178:6835/3249074395] up:boot 2024-09-11T00:33:00.783 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:33:00 smithi165 ceph-mon[35356]: mds.? [v2:172.21.15.178:6836/412194990,v1:172.21.15.178:6837/412194990] up:boot 2024-09-11T00:33:00.783 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:33:00 smithi165 ceph-mon[35356]: mds.? [v2:172.21.15.178:6838/3203303169,v1:172.21.15.178:6839/3203303169] up:boot 2024-09-11T00:33:00.783 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:33:00 smithi165 ceph-mon[35356]: mds.? [v2:172.21.15.165:6832/26243610,v1:172.21.15.165:6834/26243610] up:boot 2024-09-11T00:33:00.783 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:33:00 smithi165 ceph-mon[35356]: mds.? [v2:172.21.15.165:6833/3889511551,v1:172.21.15.165:6835/3889511551] up:boot 2024-09-11T00:33:00.783 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:33:00 smithi165 ceph-mon[35356]: mds.? [v2:172.21.15.165:6836/3703467667,v1:172.21.15.165:6838/3703467667] up:boot 2024-09-11T00:33:00.783 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:33:00 smithi165 ceph-mon[35356]: mds.? [v2:172.21.15.165:6837/1148429730,v1:172.21.15.165:6839/1148429730] up:boot 2024-09-11T00:33:00.784 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:33:00 smithi165 ceph-mon[35356]: mds.? [v2:172.21.15.12:6838/3237937724,v1:172.21.15.12:6839/3237937724] up:boot 2024-09-11T00:33:00.784 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:33:00 smithi165 ceph-mon[35356]: mds.? [v2:172.21.15.12:6840/1793327693,v1:172.21.15.12:6841/1793327693] up:boot 2024-09-11T00:33:00.784 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:33:00 smithi165 ceph-mon[35356]: fsmap cephfs:0/5 12 up:standby, 5 failed 2024-09-11T00:33:00.784 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:33:00 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "g"} : dispatch 2024-09-11T00:33:00.784 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:33:00 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "c"} : dispatch 2024-09-11T00:33:00.784 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:33:00 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "i"} : dispatch 2024-09-11T00:33:00.784 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:33:00 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "l"} : dispatch 2024-09-11T00:33:00.784 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:33:00 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "f"} : dispatch 2024-09-11T00:33:00.784 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:33:00 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "e"} : dispatch 2024-09-11T00:33:00.784 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:33:00 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "a"} : dispatch 2024-09-11T00:33:00.784 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:33:00 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "b"} : dispatch 2024-09-11T00:33:00.784 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:33:00 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "k"} : dispatch 2024-09-11T00:33:00.785 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:33:00 smithi165 ceph-mon[35356]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "h"} : dispatch 2024-09-11T00:33:00.813 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:33:00 smithi012 ceph-mon[30333]: pgmap v1178: 129 pgs: 129 active+clean; 671 MiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 1.9 KiB/s wr, 12 op/s 2024-09-11T00:33:00.813 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:33:00 smithi012 ceph-mon[30333]: mds.? [v2:172.21.15.178:6832/375852337,v1:172.21.15.178:6833/375852337] up:boot 2024-09-11T00:33:00.813 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:33:00 smithi012 ceph-mon[30333]: mds.? [v2:172.21.15.178:6834/3249074395,v1:172.21.15.178:6835/3249074395] up:boot 2024-09-11T00:33:00.814 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:33:00 smithi012 ceph-mon[30333]: mds.? [v2:172.21.15.178:6836/412194990,v1:172.21.15.178:6837/412194990] up:boot 2024-09-11T00:33:00.814 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:33:00 smithi012 ceph-mon[30333]: mds.? [v2:172.21.15.178:6838/3203303169,v1:172.21.15.178:6839/3203303169] up:boot 2024-09-11T00:33:00.814 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:33:00 smithi012 ceph-mon[30333]: mds.? [v2:172.21.15.165:6832/26243610,v1:172.21.15.165:6834/26243610] up:boot 2024-09-11T00:33:00.814 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:33:00 smithi012 ceph-mon[30333]: mds.? [v2:172.21.15.165:6833/3889511551,v1:172.21.15.165:6835/3889511551] up:boot 2024-09-11T00:33:00.814 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:33:00 smithi012 ceph-mon[30333]: mds.? [v2:172.21.15.165:6836/3703467667,v1:172.21.15.165:6838/3703467667] up:boot 2024-09-11T00:33:00.814 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:33:00 smithi012 ceph-mon[30333]: mds.? [v2:172.21.15.165:6837/1148429730,v1:172.21.15.165:6839/1148429730] up:boot 2024-09-11T00:33:00.814 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:33:00 smithi012 ceph-mon[30333]: mds.? [v2:172.21.15.12:6838/3237937724,v1:172.21.15.12:6839/3237937724] up:boot 2024-09-11T00:33:00.814 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:33:00 smithi012 ceph-mon[30333]: mds.? [v2:172.21.15.12:6840/1793327693,v1:172.21.15.12:6841/1793327693] up:boot 2024-09-11T00:33:00.814 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:33:00 smithi012 ceph-mon[30333]: fsmap cephfs:0/5 12 up:standby, 5 failed 2024-09-11T00:33:00.814 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:33:00 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "g"} : dispatch 2024-09-11T00:33:00.814 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:33:00 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "c"} : dispatch 2024-09-11T00:33:00.814 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:33:00 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "i"} : dispatch 2024-09-11T00:33:00.814 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:33:00 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "l"} : dispatch 2024-09-11T00:33:00.815 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:33:00 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "f"} : dispatch 2024-09-11T00:33:00.815 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:33:00 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "e"} : dispatch 2024-09-11T00:33:00.815 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:33:00 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "a"} : dispatch 2024-09-11T00:33:00.815 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:33:00 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "b"} : dispatch 2024-09-11T00:33:00.815 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:33:00 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "k"} : dispatch 2024-09-11T00:33:00.815 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:33:00 smithi012 ceph-mon[30333]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "h"} : dispatch 2024-09-11T00:33:00.831 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:33:00 smithi178 ceph-mon[35919]: pgmap v1178: 129 pgs: 129 active+clean; 671 MiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 1.9 KiB/s wr, 12 op/s 2024-09-11T00:33:00.831 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:33:00 smithi178 ceph-mon[35919]: mds.? [v2:172.21.15.178:6832/375852337,v1:172.21.15.178:6833/375852337] up:boot 2024-09-11T00:33:00.831 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:33:00 smithi178 ceph-mon[35919]: mds.? [v2:172.21.15.178:6834/3249074395,v1:172.21.15.178:6835/3249074395] up:boot 2024-09-11T00:33:00.831 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:33:00 smithi178 ceph-mon[35919]: mds.? [v2:172.21.15.178:6836/412194990,v1:172.21.15.178:6837/412194990] up:boot 2024-09-11T00:33:00.832 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:33:00 smithi178 ceph-mon[35919]: mds.? [v2:172.21.15.178:6838/3203303169,v1:172.21.15.178:6839/3203303169] up:boot 2024-09-11T00:33:00.832 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:33:00 smithi178 ceph-mon[35919]: mds.? [v2:172.21.15.165:6832/26243610,v1:172.21.15.165:6834/26243610] up:boot 2024-09-11T00:33:00.832 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:33:00 smithi178 ceph-mon[35919]: mds.? [v2:172.21.15.165:6833/3889511551,v1:172.21.15.165:6835/3889511551] up:boot 2024-09-11T00:33:00.832 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:33:00 smithi178 ceph-mon[35919]: mds.? [v2:172.21.15.165:6836/3703467667,v1:172.21.15.165:6838/3703467667] up:boot 2024-09-11T00:33:00.832 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:33:00 smithi178 ceph-mon[35919]: mds.? [v2:172.21.15.165:6837/1148429730,v1:172.21.15.165:6839/1148429730] up:boot 2024-09-11T00:33:00.832 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:33:00 smithi178 ceph-mon[35919]: mds.? [v2:172.21.15.12:6838/3237937724,v1:172.21.15.12:6839/3237937724] up:boot 2024-09-11T00:33:00.832 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:33:00 smithi178 ceph-mon[35919]: mds.? [v2:172.21.15.12:6840/1793327693,v1:172.21.15.12:6841/1793327693] up:boot 2024-09-11T00:33:00.832 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:33:00 smithi178 ceph-mon[35919]: fsmap cephfs:0/5 12 up:standby, 5 failed 2024-09-11T00:33:00.832 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:33:00 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "g"} : dispatch 2024-09-11T00:33:00.832 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:33:00 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "c"} : dispatch 2024-09-11T00:33:00.832 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:33:00 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "i"} : dispatch 2024-09-11T00:33:00.832 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:33:00 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "l"} : dispatch 2024-09-11T00:33:00.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:33:00 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "f"} : dispatch 2024-09-11T00:33:00.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:33:00 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "e"} : dispatch 2024-09-11T00:33:00.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:33:00 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "a"} : dispatch 2024-09-11T00:33:00.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:33:00 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "b"} : dispatch 2024-09-11T00:33:00.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:33:00 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "k"} : dispatch 2024-09-11T00:33:00.833 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:33:00 smithi178 ceph-mon[35919]: from='mgr.14150 172.21.15.12:0/3735803940' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "h"} : dispatch 2024-09-11T00:33:01.378 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:33:01.375+0000 7f039be00640 1 -- 172.21.15.12:0/500631599 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v115) ==== 104+0+0 (secure 0 0 0) 0x7f03900d1820 con 0x7f03a4077ff0 2024-09-11T00:33:01.380 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:33:01.378+0000 7f03a8e00640 1 -- 172.21.15.12:0/500631599 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f03880b5410 msgr2=0x7f03880b78d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:33:01.380 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:33:01.378+0000 7f03a8e00640 1 --2- 172.21.15.12:0/500631599 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f03880b5410 0x7f03880b78d0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f0394005fd0 tx=0x7f039400b4a0 comp rx=0 tx=0).stop 2024-09-11T00:33:01.380 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:33:01.378+0000 7f03a8e00640 1 -- 172.21.15.12:0/500631599 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f03a4077ff0 msgr2=0x7f03a419fbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:33:01.381 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:33:01.378+0000 7f03a8e00640 1 --2- 172.21.15.12:0/500631599 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f03a4077ff0 0x7f03a419fbd0 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f039002aa00 tx=0x7f0390003cd0 comp rx=0 tx=0).stop 2024-09-11T00:33:01.381 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:33:01.378+0000 7f03a8e00640 1 -- 172.21.15.12:0/500631599 shutdown_connections 2024-09-11T00:33:01.381 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:33:01.378+0000 7f03a8e00640 1 --2- 172.21.15.12:0/500631599 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f03880b5410 0x7f03880b78d0 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:33:01.381 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:33:01.378+0000 7f03a8e00640 1 --2- 172.21.15.12:0/500631599 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f03a4113800 0x7f03a41a4f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:33:01.381 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:33:01.378+0000 7f03a8e00640 1 --2- 172.21.15.12:0/500631599 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f03a4077ff0 0x7f03a419fbd0 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:33:01.381 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:33:01.378+0000 7f03a8e00640 1 --2- 172.21.15.12:0/500631599 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f03a40776d0 0x7f03a419f690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:33:01.381 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:33:01.378+0000 7f03a8e00640 1 -- 172.21.15.12:0/500631599 >> 172.21.15.12:0/500631599 conn(0x7f03a4100840 msgr2=0x7f03a4102c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:33:01.381 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:33:01.378+0000 7f03a8e00640 1 -- 172.21.15.12:0/500631599 shutdown_connections 2024-09-11T00:33:01.381 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:33:01.378+0000 7f03a8e00640 1 -- 172.21.15.12:0/500631599 wait complete. 2024-09-11T00:33:01.563 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:33:01 smithi012 ceph-mon[30333]: from='client.? 172.21.15.12:0/500631599' entity='client.admin' cmd={"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} : dispatch 2024-09-11T00:33:01.782 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:33:01 smithi165 ceph-mon[35356]: from='client.? 172.21.15.12:0/500631599' entity='client.admin' cmd={"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} : dispatch 2024-09-11T00:33:01.831 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:33:01 smithi178 ceph-mon[35919]: from='client.? 172.21.15.12:0/500631599' entity='client.admin' cmd={"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} : dispatch 2024-09-11T00:33:02.143 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid ff3ad74a-6fcf-11ef-bcea-c7b262605968 -- ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-11T00:33:02.525 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:33:02.665 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:33:02 smithi012 ceph-mon[30333]: pgmap v1179: 129 pgs: 129 active+clean; 671 MiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 1.9 KiB/s wr, 12 op/s 2024-09-11T00:33:02.665 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:33:02 smithi012 ceph-mon[30333]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-09-11T00:33:02.665 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:33:02 smithi012 ceph-mon[30333]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-09-11T00:33:02.665 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:33:02 smithi012 ceph-mon[30333]: Cluster is now healthy 2024-09-11T00:33:02.665 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:33:02 smithi012 ceph-mon[30333]: from='client.? 172.21.15.12:0/500631599' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-09-11T00:33:02.665 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:33:02 smithi012 ceph-mon[30333]: fsmap 12 up:standby 2024-09-11T00:33:02.783 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:33:02 smithi165 ceph-mon[35356]: pgmap v1179: 129 pgs: 129 active+clean; 671 MiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 1.9 KiB/s wr, 12 op/s 2024-09-11T00:33:02.783 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:33:02 smithi165 ceph-mon[35356]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-09-11T00:33:02.783 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:33:02 smithi165 ceph-mon[35356]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-09-11T00:33:02.783 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:33:02 smithi165 ceph-mon[35356]: Cluster is now healthy 2024-09-11T00:33:02.783 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:33:02 smithi165 ceph-mon[35356]: from='client.? 172.21.15.12:0/500631599' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-09-11T00:33:02.783 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:33:02 smithi165 ceph-mon[35356]: fsmap 12 up:standby 2024-09-11T00:33:02.831 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:33:02 smithi178 ceph-mon[35919]: pgmap v1179: 129 pgs: 129 active+clean; 671 MiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 1.9 KiB/s wr, 12 op/s 2024-09-11T00:33:02.831 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:33:02 smithi178 ceph-mon[35919]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-09-11T00:33:02.832 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:33:02 smithi178 ceph-mon[35919]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-09-11T00:33:02.832 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:33:02 smithi178 ceph-mon[35919]: Cluster is now healthy 2024-09-11T00:33:02.832 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:33:02 smithi178 ceph-mon[35919]: from='client.? 172.21.15.12:0/500631599' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-09-11T00:33:02.832 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:33:02 smithi178 ceph-mon[35919]: fsmap 12 up:standby 2024-09-11T00:33:03.073 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:33:03.054+0000 7f8a7e000640 1 -- 172.21.15.12:0/4211766642 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8a78109130 msgr2=0x7f8a7810f930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:33:03.073 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:33:03.054+0000 7f8a7e000640 1 --2- 172.21.15.12:0/4211766642 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8a78109130 0x7f8a7810f930 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f8a70009ca0 tx=0x7f8a7002a8c0 comp rx=0 tx=0).stop 2024-09-11T00:33:03.073 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:33:03.054+0000 7f8a7e000640 1 -- 172.21.15.12:0/4211766642 shutdown_connections 2024-09-11T00:33:03.073 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:33:03.054+0000 7f8a7e000640 1 --2- 172.21.15.12:0/4211766642 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8a78109130 0x7f8a7810f930 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:33:03.073 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:33:03.054+0000 7f8a7e000640 1 --2- 172.21.15.12:0/4211766642 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8a781087b0 0x7f8a78108bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:33:03.073 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:33:03.054+0000 7f8a7e000640 1 --2- 172.21.15.12:0/4211766642 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8a78102da0 0x7f8a78103180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:33:03.073 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:33:03.054+0000 7f8a7e000640 1 -- 172.21.15.12:0/4211766642 >> 172.21.15.12:0/4211766642 conn(0x7f8a780fe810 msgr2=0x7f8a78100c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:33:03.073 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:33:03.054+0000 7f8a7e000640 1 -- 172.21.15.12:0/4211766642 shutdown_connections 2024-09-11T00:33:03.073 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:33:03.054+0000 7f8a7e000640 1 -- 172.21.15.12:0/4211766642 wait complete. 2024-09-11T00:33:03.073 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:33:03.055+0000 7f8a7e000640 1 Processor -- start 2024-09-11T00:33:03.074 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:33:03.055+0000 7f8a7e000640 1 -- start start 2024-09-11T00:33:03.074 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:33:03.055+0000 7f8a7e000640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8a78102da0 0x7f8a780754a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:33:03.074 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:33:03.055+0000 7f8a7e000640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8a781087b0 0x7f8a780759e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:33:03.074 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:33:03.055+0000 7f8a7e000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8a78109130 0x7f8a78075f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:33:03.074 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:33:03.055+0000 7f8a7e000640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f8a78112df0 con 0x7f8a78102da0 2024-09-11T00:33:03.074 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:33:03.055+0000 7f8a7e000640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f8a78112c70 con 0x7f8a78109130 2024-09-11T00:33:03.074 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:33:03.055+0000 7f8a7e000640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f8a78112f70 con 0x7f8a781087b0 2024-09-11T00:33:03.074 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:33:03.056+0000 7f8a77400640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8a78102da0 0x7f8a780754a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:33:03.074 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:33:03.056+0000 7f8a77400640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8a78102da0 0x7f8a780754a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:44766/0 (socket says 172.21.15.12:44766) 2024-09-11T00:33:03.074 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:33:03.056+0000 7f8a77400640 1 -- 172.21.15.12:0/1012408185 learned_addr learned my addr 172.21.15.12:0/1012408185 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:33:03.074 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:33:03.056+0000 7f8a6fe00640 1 --2- 172.21.15.12:0/1012408185 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8a781087b0 0x7f8a780759e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:33:03.074 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:33:03.056+0000 7f8a77e00640 1 --2- 172.21.15.12:0/1012408185 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8a78109130 0x7f8a78075f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:33:03.074 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:33:03.056+0000 7f8a77400640 1 -- 172.21.15.12:0/1012408185 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8a781087b0 msgr2=0x7f8a780759e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:33:03.075 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:33:03.056+0000 7f8a77400640 1 --2- 172.21.15.12:0/1012408185 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8a781087b0 0x7f8a780759e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:33:03.075 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:33:03.056+0000 7f8a77400640 1 -- 172.21.15.12:0/1012408185 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8a78109130 msgr2=0x7f8a78075f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:33:03.075 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:33:03.056+0000 7f8a77400640 1 --2- 172.21.15.12:0/1012408185 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8a78109130 0x7f8a78075f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:33:03.075 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:33:03.056+0000 7f8a77400640 1 -- 172.21.15.12:0/1012408185 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a781a76e0 con 0x7f8a78102da0 2024-09-11T00:33:03.075 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:33:03.056+0000 7f8a77e00640 1 --2- 172.21.15.12:0/1012408185 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8a78109130 0x7f8a78075f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T00:33:03.075 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:33:03.057+0000 7f8a77400640 1 --2- 172.21.15.12:0/1012408185 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8a78102da0 0x7f8a780754a0 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f8a6800e9f0 tx=0x7f8a6800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:33:03.075 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:33:03.057+0000 7f8a74c00640 1 -- 172.21.15.12:0/1012408185 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a6800cd80 con 0x7f8a78102da0 2024-09-11T00:33:03.075 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:33:03.057+0000 7f8a74c00640 1 -- 172.21.15.12:0/1012408185 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8a680108e0 con 0x7f8a78102da0 2024-09-11T00:33:03.075 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:33:03.057+0000 7f8a74c00640 1 -- 172.21.15.12:0/1012408185 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a68018980 con 0x7f8a78102da0 2024-09-11T00:33:03.075 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:33:03.057+0000 7f8a7e000640 1 -- 172.21.15.12:0/1012408185 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a781a79d0 con 0x7f8a78102da0 2024-09-11T00:33:03.075 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:33:03.057+0000 7f8a7e000640 1 -- 172.21.15.12:0/1012408185 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a781a7f10 con 0x7f8a78102da0 2024-09-11T00:33:03.075 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:33:03.058+0000 7f8a7e000640 1 -- 172.21.15.12:0/1012408185 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8a78103180 con 0x7f8a78102da0 2024-09-11T00:33:03.076 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:33:03.060+0000 7f8a74c00640 1 -- 172.21.15.12:0/1012408185 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7f8a68010430 con 0x7f8a78102da0 2024-09-11T00:33:03.076 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:33:03.061+0000 7f8a74c00640 1 --2- 172.21.15.12:0/1012408185 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f8a5c0b52e0 0x7f8a5c0b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:33:03.076 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:33:03.062+0000 7f8a74c00640 1 -- 172.21.15.12:0/1012408185 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9638+0+0 (secure 0 0 0) 0x7f8a680e6550 con 0x7f8a78102da0 2024-09-11T00:33:03.076 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:33:03.062+0000 7f8a6fe00640 1 --2- 172.21.15.12:0/1012408185 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f8a5c0b52e0 0x7f8a5c0b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:33:03.076 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:33:03.062+0000 7f8a6fe00640 1 --2- 172.21.15.12:0/1012408185 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f8a5c0b52e0 0x7f8a5c0b77a0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f8a78076f90 tx=0x7f8a6000b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:33:03.076 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:33:03.065+0000 7f8a74c00640 1 -- 172.21.15.12:0/1012408185 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f8a680ace30 con 0x7f8a78102da0 2024-09-11T00:33:03.263 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:33:03.260+0000 7f8a7e000640 1 -- 172.21.15.12:0/1012408185 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f8a7802e5c0 con 0x7f8a78102da0 2024-09-11T00:33:03.381 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:33:03.378+0000 7f8a74c00640 1 -- 172.21.15.12:0/1012408185 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v89) ==== 181+0+0 (secure 0 0 0) 0x7f8a680ac730 con 0x7f8a78102da0 2024-09-11T00:33:03.381 INFO:teuthology.orchestra.run.smithi012.stderr:pool 'cephfs_metadata' removed 2024-09-11T00:33:03.384 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:33:03.382+0000 7f8a7e000640 1 -- 172.21.15.12:0/1012408185 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f8a5c0b52e0 msgr2=0x7f8a5c0b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:33:03.384 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:33:03.382+0000 7f8a7e000640 1 --2- 172.21.15.12:0/1012408185 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f8a5c0b52e0 0x7f8a5c0b77a0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f8a78076f90 tx=0x7f8a6000b410 comp rx=0 tx=0).stop 2024-09-11T00:33:03.384 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:33:03.382+0000 7f8a7e000640 1 -- 172.21.15.12:0/1012408185 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8a78102da0 msgr2=0x7f8a780754a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:33:03.385 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:33:03.382+0000 7f8a7e000640 1 --2- 172.21.15.12:0/1012408185 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8a78102da0 0x7f8a780754a0 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f8a6800e9f0 tx=0x7f8a6800eeb0 comp rx=0 tx=0).stop 2024-09-11T00:33:03.385 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:33:03.382+0000 7f8a7e000640 1 -- 172.21.15.12:0/1012408185 shutdown_connections 2024-09-11T00:33:03.385 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:33:03.382+0000 7f8a7e000640 1 --2- 172.21.15.12:0/1012408185 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f8a5c0b52e0 0x7f8a5c0b77a0 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:33:03.385 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:33:03.382+0000 7f8a7e000640 1 --2- 172.21.15.12:0/1012408185 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8a78109130 0x7f8a78075f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:33:03.385 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:33:03.382+0000 7f8a7e000640 1 --2- 172.21.15.12:0/1012408185 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8a781087b0 0x7f8a780759e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:33:03.385 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:33:03.382+0000 7f8a7e000640 1 --2- 172.21.15.12:0/1012408185 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8a78102da0 0x7f8a780754a0 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:33:03.385 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:33:03.382+0000 7f8a7e000640 1 -- 172.21.15.12:0/1012408185 >> 172.21.15.12:0/1012408185 conn(0x7f8a780fe810 msgr2=0x7f8a78100c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:33:03.386 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:33:03.382+0000 7f8a7e000640 1 -- 172.21.15.12:0/1012408185 shutdown_connections 2024-09-11T00:33:03.386 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:33:03.382+0000 7f8a7e000640 1 -- 172.21.15.12:0/1012408185 wait complete. 2024-09-11T00:33:03.532 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:33:03 smithi165 ceph-mon[35356]: from='client.? 172.21.15.12:0/1012408185' entity='client.admin' cmd={"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} : dispatch 2024-09-11T00:33:03.563 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:33:03 smithi012 ceph-mon[30333]: from='client.? 172.21.15.12:0/1012408185' entity='client.admin' cmd={"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} : dispatch 2024-09-11T00:33:03.831 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:33:03 smithi178 ceph-mon[35919]: from='client.? 172.21.15.12:0/1012408185' entity='client.admin' cmd={"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} : dispatch 2024-09-11T00:33:04.304 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid ff3ad74a-6fcf-11ef-bcea-c7b262605968 -- ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-11T00:33:04.563 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:33:04 smithi012 ceph-mon[30333]: pgmap v1180: 129 pgs: 129 active+clean; 671 MiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 1023 B/s wr, 9 op/s 2024-09-11T00:33:04.563 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:33:04 smithi012 ceph-mon[30333]: from='client.? 172.21.15.12:0/1012408185' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-09-11T00:33:04.563 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:33:04 smithi012 ceph-mon[30333]: osdmap e89: 12 total, 12 up, 12 in 2024-09-11T00:33:04.684 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:33:04.783 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:33:04 smithi165 ceph-mon[35356]: pgmap v1180: 129 pgs: 129 active+clean; 671 MiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 1023 B/s wr, 9 op/s 2024-09-11T00:33:04.783 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:33:04 smithi165 ceph-mon[35356]: from='client.? 172.21.15.12:0/1012408185' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-09-11T00:33:04.783 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:33:04 smithi165 ceph-mon[35356]: osdmap e89: 12 total, 12 up, 12 in 2024-09-11T00:33:04.832 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:33:04 smithi178 ceph-mon[35919]: pgmap v1180: 129 pgs: 129 active+clean; 671 MiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 1023 B/s wr, 9 op/s 2024-09-11T00:33:04.832 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:33:04 smithi178 ceph-mon[35919]: from='client.? 172.21.15.12:0/1012408185' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-09-11T00:33:04.832 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:33:04 smithi178 ceph-mon[35919]: osdmap e89: 12 total, 12 up, 12 in 2024-09-11T00:33:05.256 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:33:05.252+0000 7f37bd800640 1 -- 172.21.15.12:0/2749595787 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f37b8102da0 msgr2=0x7f37b8103180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:33:05.256 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:33:05.252+0000 7f37bd800640 1 --2- 172.21.15.12:0/2749595787 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f37b8102da0 0x7f37b8103180 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f37a8009900 tx=0x7f37a802a3f0 comp rx=0 tx=0).stop 2024-09-11T00:33:05.256 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:33:05.252+0000 7f37bd800640 1 -- 172.21.15.12:0/2749595787 shutdown_connections 2024-09-11T00:33:05.256 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:33:05.252+0000 7f37bd800640 1 --2- 172.21.15.12:0/2749595787 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f37b8109130 0x7f37b810f930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:33:05.256 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:33:05.252+0000 7f37bd800640 1 --2- 172.21.15.12:0/2749595787 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f37b81087b0 0x7f37b8108bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:33:05.256 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:33:05.252+0000 7f37bd800640 1 --2- 172.21.15.12:0/2749595787 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f37b8102da0 0x7f37b8103180 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:33:05.256 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:33:05.252+0000 7f37bd800640 1 -- 172.21.15.12:0/2749595787 >> 172.21.15.12:0/2749595787 conn(0x7f37b80fe810 msgr2=0x7f37b8100c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:33:05.256 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:33:05.253+0000 7f37bd800640 1 -- 172.21.15.12:0/2749595787 shutdown_connections 2024-09-11T00:33:05.257 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:33:05.253+0000 7f37bd800640 1 -- 172.21.15.12:0/2749595787 wait complete. 2024-09-11T00:33:05.257 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:33:05.253+0000 7f37bd800640 1 Processor -- start 2024-09-11T00:33:05.257 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:33:05.254+0000 7f37bd800640 1 -- start start 2024-09-11T00:33:05.258 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:33:05.254+0000 7f37bd800640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f37b8102da0 0x7f37b8199260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:33:05.258 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:33:05.254+0000 7f37bd800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f37b81087b0 0x7f37b81997a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:33:05.258 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:33:05.254+0000 7f37bd800640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f37b8109130 0x7f37b819eb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:33:05.258 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:33:05.254+0000 7f37bd800640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f37b8112e00 con 0x7f37b8109130 2024-09-11T00:33:05.258 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:33:05.254+0000 7f37bd800640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f37b8112c80 con 0x7f37b81087b0 2024-09-11T00:33:05.258 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:33:05.254+0000 7f37bd800640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f37b8112f80 con 0x7f37b8102da0 2024-09-11T00:33:05.258 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:33:05.254+0000 7f37b6a00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f37b8102da0 0x7f37b8199260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:33:05.258 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:33:05.254+0000 7f37b6a00640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f37b8102da0 0x7f37b8199260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.12:33184/0 (socket says 172.21.15.12:33184) 2024-09-11T00:33:05.259 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:33:05.254+0000 7f37b6a00640 1 -- 172.21.15.12:0/543321312 learned_addr learned my addr 172.21.15.12:0/543321312 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:33:05.259 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:33:05.254+0000 7f37b7400640 1 --2- 172.21.15.12:0/543321312 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f37b8109130 0x7f37b819eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:33:05.259 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:33:05.255+0000 7f37b6000640 1 --2- 172.21.15.12:0/543321312 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f37b81087b0 0x7f37b81997a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:33:05.259 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:33:05.255+0000 7f37b6a00640 1 -- 172.21.15.12:0/543321312 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f37b81087b0 msgr2=0x7f37b81997a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:33:05.259 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:33:05.255+0000 7f37b6a00640 1 --2- 172.21.15.12:0/543321312 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f37b81087b0 0x7f37b81997a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:33:05.259 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:33:05.255+0000 7f37b6a00640 1 -- 172.21.15.12:0/543321312 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f37b8109130 msgr2=0x7f37b819eb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:33:05.259 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:33:05.255+0000 7f37b6a00640 1 --2- 172.21.15.12:0/543321312 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f37b8109130 0x7f37b819eb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:33:05.259 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:33:05.255+0000 7f37b6a00640 1 -- 172.21.15.12:0/543321312 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37b819f310 con 0x7f37b8102da0 2024-09-11T00:33:05.259 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:33:05.255+0000 7f37b6a00640 1 --2- 172.21.15.12:0/543321312 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f37b8102da0 0x7f37b8199260 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f37a8005ab0 tx=0x7f37a8003c10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:33:05.259 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:33:05.255+0000 7f37af400640 1 -- 172.21.15.12:0/543321312 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37a803f070 con 0x7f37b8102da0 2024-09-11T00:33:05.259 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:33:05.255+0000 7f37af400640 1 -- 172.21.15.12:0/543321312 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f37a8031b60 con 0x7f37b8102da0 2024-09-11T00:33:05.259 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:33:05.255+0000 7f37bd800640 1 -- 172.21.15.12:0/543321312 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f37b819f5a0 con 0x7f37b8102da0 2024-09-11T00:33:05.259 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:33:05.255+0000 7f37af400640 1 -- 172.21.15.12:0/543321312 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37a8035030 con 0x7f37b8102da0 2024-09-11T00:33:05.260 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:33:05.256+0000 7f37bd800640 1 -- 172.21.15.12:0/543321312 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37b81a5cc0 con 0x7f37b8102da0 2024-09-11T00:33:05.260 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:33:05.257+0000 7f37bd800640 1 -- 172.21.15.12:0/543321312 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f37b8103180 con 0x7f37b8102da0 2024-09-11T00:33:05.261 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:33:05.259+0000 7f37af400640 1 -- 172.21.15.12:0/543321312 <== mon.1 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7f37a803b460 con 0x7f37b8102da0 2024-09-11T00:33:05.263 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:33:05.261+0000 7f37af400640 1 --2- 172.21.15.12:0/543321312 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f379c0b52e0 0x7f379c0b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:33:05.263 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:33:05.261+0000 7f37af400640 1 -- 172.21.15.12:0/543321312 <== mon.1 v2:172.21.15.178:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9084+0+0 (secure 0 0 0) 0x7f37a8106ff0 con 0x7f37b8102da0 2024-09-11T00:33:05.263 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:33:05.261+0000 7f37b6000640 1 --2- 172.21.15.12:0/543321312 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f379c0b52e0 0x7f379c0b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:33:05.264 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:33:05.262+0000 7f37b6000640 1 --2- 172.21.15.12:0/543321312 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f379c0b52e0 0x7f379c0b77a0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f37b8102050 tx=0x7f37a4005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:33:05.266 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:33:05.264+0000 7f37af400640 1 -- 172.21.15.12:0/543321312 <== mon.1 v2:172.21.15.178:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f37a80cdb20 con 0x7f37b8102da0 2024-09-11T00:33:05.461 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:33:05.459+0000 7f37bd800640 1 -- 172.21.15.12:0/543321312 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f37b802e5c0 con 0x7f37b8102da0 2024-09-11T00:33:06.391 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:33:06.388+0000 7f37af400640 1 -- 172.21.15.12:0/543321312 <== mon.1 v2:172.21.15.178:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v90) ==== 169+0+0 (secure 0 0 0) 0x7f37a80d62d0 con 0x7f37b8102da0 2024-09-11T00:33:06.392 INFO:teuthology.orchestra.run.smithi012.stderr:pool 'cephfs_data' removed 2024-09-11T00:33:06.396 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:33:06.393+0000 7f37bd800640 1 -- 172.21.15.12:0/543321312 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f379c0b52e0 msgr2=0x7f379c0b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:33:06.396 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:33:06.393+0000 7f37bd800640 1 --2- 172.21.15.12:0/543321312 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f379c0b52e0 0x7f379c0b77a0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f37b8102050 tx=0x7f37a4005f90 comp rx=0 tx=0).stop 2024-09-11T00:33:06.396 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:33:06.393+0000 7f37bd800640 1 -- 172.21.15.12:0/543321312 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f37b8102da0 msgr2=0x7f37b8199260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:33:06.396 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:33:06.393+0000 7f37bd800640 1 --2- 172.21.15.12:0/543321312 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f37b8102da0 0x7f37b8199260 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f37a8005ab0 tx=0x7f37a8003c10 comp rx=0 tx=0).stop 2024-09-11T00:33:06.397 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:33:06.394+0000 7f37bd800640 1 -- 172.21.15.12:0/543321312 shutdown_connections 2024-09-11T00:33:06.397 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:33:06.394+0000 7f37bd800640 1 --2- 172.21.15.12:0/543321312 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f379c0b52e0 0x7f379c0b77a0 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:33:06.397 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:33:06.394+0000 7f37bd800640 1 --2- 172.21.15.12:0/543321312 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f37b8109130 0x7f37b819eb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:33:06.397 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:33:06.394+0000 7f37bd800640 1 --2- 172.21.15.12:0/543321312 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f37b81087b0 0x7f37b81997a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:33:06.397 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:33:06.394+0000 7f37bd800640 1 --2- 172.21.15.12:0/543321312 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f37b8102da0 0x7f37b8199260 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:33:06.397 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:33:06.394+0000 7f37bd800640 1 -- 172.21.15.12:0/543321312 >> 172.21.15.12:0/543321312 conn(0x7f37b80fe810 msgr2=0x7f37b8100c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:33:06.398 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:33:06.395+0000 7f37bd800640 1 -- 172.21.15.12:0/543321312 shutdown_connections 2024-09-11T00:33:06.398 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:33:06.395+0000 7f37bd800640 1 -- 172.21.15.12:0/543321312 wait complete. 2024-09-11T00:33:06.783 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:33:06 smithi165 ceph-mon[35356]: pgmap v1182: 97 pgs: 97 active+clean; 577 KiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T00:33:06.783 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:33:06 smithi165 ceph-mon[35356]: from='client.? 172.21.15.12:0/543321312' entity='client.admin' cmd={"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} : dispatch 2024-09-11T00:33:06.783 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:33:06 smithi165 ceph-mon[35356]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} : dispatch 2024-09-11T00:33:06.793 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:33:06 smithi012 ceph-mon[30333]: pgmap v1182: 97 pgs: 97 active+clean; 577 KiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T00:33:06.793 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:33:06 smithi012 ceph-mon[30333]: from='client.? 172.21.15.12:0/543321312' entity='client.admin' cmd={"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} : dispatch 2024-09-11T00:33:06.793 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:33:06 smithi012 ceph-mon[30333]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} : dispatch 2024-09-11T00:33:06.831 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:33:06 smithi178 ceph-mon[35919]: pgmap v1182: 97 pgs: 97 active+clean; 577 KiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T00:33:06.831 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:33:06 smithi178 ceph-mon[35919]: from='client.? 172.21.15.12:0/543321312' entity='client.admin' cmd={"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} : dispatch 2024-09-11T00:33:06.831 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:33:06 smithi178 ceph-mon[35919]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} : dispatch 2024-09-11T00:33:07.294 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid ff3ad74a-6fcf-11ef-bcea-c7b262605968 -- ceph osd pool rm cephfs_data_ec cephfs_data_ec --yes-i-really-really-mean-it 2024-09-11T00:33:07.563 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:33:07 smithi012 ceph-mon[30333]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-09-11T00:33:07.563 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:33:07 smithi012 ceph-mon[30333]: osdmap e90: 12 total, 12 up, 12 in 2024-09-11T00:33:07.672 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:33:07.783 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:33:07 smithi165 ceph-mon[35356]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-09-11T00:33:07.783 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:33:07 smithi165 ceph-mon[35356]: osdmap e90: 12 total, 12 up, 12 in 2024-09-11T00:33:07.831 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:33:07 smithi178 ceph-mon[35919]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-09-11T00:33:07.831 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:33:07 smithi178 ceph-mon[35919]: osdmap e90: 12 total, 12 up, 12 in 2024-09-11T00:33:08.330 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:33:08.327+0000 7f4a9a000640 1 -- 172.21.15.12:0/457236020 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4a9410b180 msgr2=0x7f4a94111980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:33:08.330 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:33:08.327+0000 7f4a9a000640 1 --2- 172.21.15.12:0/457236020 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4a9410b180 0x7f4a94111980 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f4a8c009ca0 tx=0x7f4a8c02a8c0 comp rx=0 tx=0).stop 2024-09-11T00:33:08.331 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:33:08.327+0000 7f4a9a000640 1 -- 172.21.15.12:0/457236020 shutdown_connections 2024-09-11T00:33:08.331 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:33:08.327+0000 7f4a9a000640 1 --2- 172.21.15.12:0/457236020 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4a9410b180 0x7f4a94111980 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:33:08.331 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:33:08.327+0000 7f4a9a000640 1 --2- 172.21.15.12:0/457236020 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4a9410a800 0x7f4a9410ac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:33:08.331 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:33:08.327+0000 7f4a9a000640 1 --2- 172.21.15.12:0/457236020 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4a94104df0 0x7f4a941051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:33:08.331 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:33:08.327+0000 7f4a9a000640 1 -- 172.21.15.12:0/457236020 >> 172.21.15.12:0/457236020 conn(0x7f4a94100840 msgr2=0x7f4a94102c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:33:08.331 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:33:08.327+0000 7f4a9a000640 1 -- 172.21.15.12:0/457236020 shutdown_connections 2024-09-11T00:33:08.331 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:33:08.327+0000 7f4a9a000640 1 -- 172.21.15.12:0/457236020 wait complete. 2024-09-11T00:33:08.332 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:33:08.328+0000 7f4a9a000640 1 Processor -- start 2024-09-11T00:33:08.333 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:33:08.329+0000 7f4a9a000640 1 -- start start 2024-09-11T00:33:08.333 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:33:08.329+0000 7f4a9a000640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4a94104df0 0x7f4a940776b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:33:08.333 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:33:08.329+0000 7f4a9a000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4a9410a800 0x7f4a94077bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:33:08.333 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:33:08.329+0000 7f4a9a000640 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4a9410b180 0x7f4a94078130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:33:08.333 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:33:08.329+0000 7f4a9a000640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f4a94114e50 con 0x7f4a94104df0 2024-09-11T00:33:08.333 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:33:08.329+0000 7f4a9a000640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f4a94114cd0 con 0x7f4a9410a800 2024-09-11T00:33:08.333 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:33:08.329+0000 7f4a9a000640 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 -- 0x7f4a94114fd0 con 0x7f4a9410b180 2024-09-11T00:33:08.333 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:33:08.329+0000 7f4a98c00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4a94104df0 0x7f4a940776b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:33:08.333 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:33:08.329+0000 7f4a93e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4a9410a800 0x7f4a94077bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:33:08.333 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:33:08.329+0000 7f4a98c00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4a94104df0 0x7f4a940776b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:44794/0 (socket says 172.21.15.12:44794) 2024-09-11T00:33:08.333 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:33:08.329+0000 7f4a98c00640 1 -- 172.21.15.12:0/1407077323 learned_addr learned my addr 172.21.15.12:0/1407077323 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-11T00:33:08.333 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:33:08.329+0000 7f4a99600640 1 --2- 172.21.15.12:0/1407077323 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4a9410b180 0x7f4a94078130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:33:08.334 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:33:08.330+0000 7f4a98c00640 1 -- 172.21.15.12:0/1407077323 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4a9410b180 msgr2=0x7f4a94078130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:33:08.334 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:33:08.330+0000 7f4a98c00640 1 --2- 172.21.15.12:0/1407077323 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4a9410b180 0x7f4a94078130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:33:08.334 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:33:08.330+0000 7f4a98c00640 1 -- 172.21.15.12:0/1407077323 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4a9410a800 msgr2=0x7f4a94077bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:33:08.334 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:33:08.330+0000 7f4a98c00640 1 --2- 172.21.15.12:0/1407077323 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4a9410a800 0x7f4a94077bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:33:08.334 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:33:08.330+0000 7f4a98c00640 1 -- 172.21.15.12:0/1407077323 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a941ab7f0 con 0x7f4a94104df0 2024-09-11T00:33:08.334 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:33:08.330+0000 7f4a98c00640 1 --2- 172.21.15.12:0/1407077323 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4a94104df0 0x7f4a940776b0 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f4a8800e9f0 tx=0x7f4a8800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:33:08.335 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:33:08.330+0000 7f4a91600640 1 -- 172.21.15.12:0/1407077323 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a8800cd80 con 0x7f4a94104df0 2024-09-11T00:33:08.335 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:33:08.331+0000 7f4a9a000640 1 -- 172.21.15.12:0/1407077323 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4a941abae0 con 0x7f4a94104df0 2024-09-11T00:33:08.335 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:33:08.331+0000 7f4a91600640 1 -- 172.21.15.12:0/1407077323 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4a880108e0 con 0x7f4a94104df0 2024-09-11T00:33:08.335 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:33:08.331+0000 7f4a91600640 1 -- 172.21.15.12:0/1407077323 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a88018980 con 0x7f4a94104df0 2024-09-11T00:33:08.335 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:33:08.331+0000 7f4a9a000640 1 -- 172.21.15.12:0/1407077323 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4a941ac070 con 0x7f4a94104df0 2024-09-11T00:33:08.335 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:33:08.332+0000 7f4a9a000640 1 -- 172.21.15.12:0/1407077323 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4a740052c0 con 0x7f4a94104df0 2024-09-11T00:33:08.337 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:33:08.335+0000 7f4a91600640 1 -- 172.21.15.12:0/1407077323 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7f4a88010430 con 0x7f4a94104df0 2024-09-11T00:33:08.339 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:33:08.336+0000 7f4a91600640 1 --2- 172.21.15.12:0/1407077323 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f4a7c0b5410 0x7f4a7c0b78d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:33:08.339 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:33:08.337+0000 7f4a91600640 1 -- 172.21.15.12:0/1407077323 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8496+0+0 (secure 0 0 0) 0x7f4a880e5f80 con 0x7f4a94104df0 2024-09-11T00:33:08.339 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:33:08.337+0000 7f4a93e00640 1 --2- 172.21.15.12:0/1407077323 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f4a7c0b5410 0x7f4a7c0b78d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:33:08.340 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:33:08.338+0000 7f4a93e00640 1 --2- 172.21.15.12:0/1407077323 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f4a7c0b5410 0x7f4a7c0b78d0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f4a9407b1c0 tx=0x7f4a8000b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:33:08.340 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:33:08.338+0000 7f4a91600640 1 -- 172.21.15.12:0/1407077323 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f4a880ac6b0 con 0x7f4a94104df0 2024-09-11T00:33:08.537 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:33:08.534+0000 7f4a9a000640 1 -- 172.21.15.12:0/1407077323 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true} v 0) -- 0x7f4a740055b0 con 0x7f4a94104df0 2024-09-11T00:33:08.783 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:33:08 smithi165 ceph-mon[35356]: pgmap v1184: 33 pgs: 33 active+clean; 577 KiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T00:33:08.813 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:33:08 smithi012 ceph-mon[30333]: pgmap v1184: 33 pgs: 33 active+clean; 577 KiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T00:33:08.831 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:33:08 smithi178 ceph-mon[35919]: pgmap v1184: 33 pgs: 33 active+clean; 577 KiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T00:33:09.411 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:33:09.408+0000 7f4a91600640 1 -- 172.21.15.12:0/1407077323 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data_ec' removed v91) ==== 178+0+0 (secure 0 0 0) 0x7f4a880b10e0 con 0x7f4a94104df0 2024-09-11T00:33:09.411 INFO:teuthology.orchestra.run.smithi012.stderr:pool 'cephfs_data_ec' removed 2024-09-11T00:33:09.416 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:33:09.413+0000 7f4a9a000640 1 -- 172.21.15.12:0/1407077323 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f4a7c0b5410 msgr2=0x7f4a7c0b78d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:33:09.416 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:33:09.413+0000 7f4a9a000640 1 --2- 172.21.15.12:0/1407077323 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f4a7c0b5410 0x7f4a7c0b78d0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f4a9407b1c0 tx=0x7f4a8000b390 comp rx=0 tx=0).stop 2024-09-11T00:33:09.416 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:33:09.413+0000 7f4a9a000640 1 -- 172.21.15.12:0/1407077323 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4a94104df0 msgr2=0x7f4a940776b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:33:09.416 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:33:09.413+0000 7f4a9a000640 1 --2- 172.21.15.12:0/1407077323 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4a94104df0 0x7f4a940776b0 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f4a8800e9f0 tx=0x7f4a8800eeb0 comp rx=0 tx=0).stop 2024-09-11T00:33:09.416 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:33:09.413+0000 7f4a9a000640 1 -- 172.21.15.12:0/1407077323 shutdown_connections 2024-09-11T00:33:09.416 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:33:09.413+0000 7f4a9a000640 1 --2- 172.21.15.12:0/1407077323 >> [v2:172.21.15.12:6800/3442939137,v1:172.21.15.12:6801/3442939137] conn(0x7f4a7c0b5410 0x7f4a7c0b78d0 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:33:09.416 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:33:09.413+0000 7f4a9a000640 1 --2- 172.21.15.12:0/1407077323 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4a9410b180 0x7f4a94078130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:33:09.417 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:33:09.413+0000 7f4a9a000640 1 --2- 172.21.15.12:0/1407077323 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4a9410a800 0x7f4a94077bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:33:09.417 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:33:09.413+0000 7f4a9a000640 1 --2- 172.21.15.12:0/1407077323 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4a94104df0 0x7f4a940776b0 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:33:09.417 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:33:09.413+0000 7f4a9a000640 1 -- 172.21.15.12:0/1407077323 >> 172.21.15.12:0/1407077323 conn(0x7f4a94100840 msgr2=0x7f4a94100f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:33:09.417 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:33:09.413+0000 7f4a9a000640 1 -- 172.21.15.12:0/1407077323 shutdown_connections 2024-09-11T00:33:09.417 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-11T00:33:09.414+0000 7f4a9a000640 1 -- 172.21.15.12:0/1407077323 wait complete. 2024-09-11T00:33:09.782 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:33:09 smithi165 ceph-mon[35356]: from='client.? 172.21.15.12:0/1407077323' entity='client.admin' cmd={"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true} : dispatch 2024-09-11T00:33:09.783 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:33:09 smithi165 ceph-mon[35356]: pgmap v1185: 33 pgs: 33 active+clean; 577 KiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T00:33:09.802 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:33:09 smithi012 ceph-mon[30333]: from='client.? 172.21.15.12:0/1407077323' entity='client.admin' cmd={"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true} : dispatch 2024-09-11T00:33:09.802 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:33:09 smithi012 ceph-mon[30333]: pgmap v1185: 33 pgs: 33 active+clean; 577 KiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T00:33:09.831 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:33:09 smithi178 ceph-mon[35919]: from='client.? 172.21.15.12:0/1407077323' entity='client.admin' cmd={"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true} : dispatch 2024-09-11T00:33:09.831 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:33:09 smithi178 ceph-mon[35919]: pgmap v1185: 33 pgs: 33 active+clean; 577 KiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T00:33:10.335 DEBUG:teuthology.orchestra.run.smithi012:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-11T00:33:10.422 DEBUG:teuthology.orchestra.run.smithi165:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-11T00:33:10.476 DEBUG:teuthology.orchestra.run.smithi178:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-11T00:33:10.547 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-09-11T00:33:10.547 DEBUG:teuthology.orchestra.run.smithi012:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-09-11T00:33:10.563 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:33:10 smithi012 ceph-mon[30333]: from='client.? 172.21.15.12:0/1407077323' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]': finished 2024-09-11T00:33:10.563 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:33:10 smithi012 ceph-mon[30333]: osdmap e91: 12 total, 12 up, 12 in 2024-09-11T00:33:10.583 DEBUG:teuthology.orchestra.run.smithi165:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-09-11T00:33:10.598 DEBUG:teuthology.orchestra.run.smithi178:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-09-11T00:33:10.613 INFO:tasks.cephadm:Stopping all daemons... 2024-09-11T00:33:10.614 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2024-09-11T00:33:10.614 DEBUG:teuthology.orchestra.run.smithi012:> sudo systemctl stop ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968@mon.a 2024-09-11T00:33:10.782 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:33:10 smithi165 ceph-mon[35356]: from='client.? 172.21.15.12:0/1407077323' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]': finished 2024-09-11T00:33:10.783 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:33:10 smithi165 ceph-mon[35356]: osdmap e91: 12 total, 12 up, 12 in 2024-09-11T00:33:10.831 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:33:10 smithi178 ceph-mon[35919]: from='client.? 172.21.15.12:0/1407077323' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]': finished 2024-09-11T00:33:10.831 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:33:10 smithi178 ceph-mon[35919]: osdmap e91: 12 total, 12 up, 12 in 2024-09-11T00:33:11.051 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:33:10 smithi012 systemd[1]: Stopping Ceph mon.a for ff3ad74a-6fcf-11ef-bcea-c7b262605968... 2024-09-11T00:33:11.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:33:11 smithi012 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-mon-a[30329]: 2024-09-11T00:33:11.047+0000 7f046aa00640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T00:33:11.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:33:11 smithi012 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-mon-a[30329]: 2024-09-11T00:33:11.047+0000 7f046aa00640 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2024-09-11T00:33:11.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:33:11 smithi012 podman[110974]: 2024-09-11 00:33:11.089901811 +0000 UTC m=+0.167195100 container died 08abb0cfbc0b538576931bc46d941dcfd693e99b9d1541a83e889de3fdae4390 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30, name=ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-mon-a, GIT_CLEAN=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-09-11T00:33:12.063 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:33:11 smithi012 podman[110974]: 2024-09-11 00:33:11.712160616 +0000 UTC m=+0.789453904 container cleanup 08abb0cfbc0b538576931bc46d941dcfd693e99b9d1541a83e889de3fdae4390 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30, name=ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-mon-a, GIT_CLEAN=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-11T00:33:12.063 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:33:11 smithi012 bash[110974]: ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-mon-a 2024-09-11T00:33:12.064 INFO:journalctl@ceph.mon.a.smithi012.stdout:Sep 11 00:33:11 smithi012 podman[111051]: 2024-09-11 00:33:11.938989867 +0000 UTC m=+0.842287026 container remove 08abb0cfbc0b538576931bc46d941dcfd693e99b9d1541a83e889de3fdae4390 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30, name=ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-mon-a, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, GIT_CLEAN=True, org.label-schema.schema-version=1.0) 2024-09-11T00:33:12.250 DEBUG:teuthology.orchestra.run.smithi012:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968@mon.a.service' 2024-09-11T00:33:12.412 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T00:33:12.413 INFO:tasks.cephadm.mon.a:Stopped mon.a 2024-09-11T00:33:12.413 INFO:tasks.cephadm.mon.c:Stopping mon.b... 2024-09-11T00:33:12.413 DEBUG:teuthology.orchestra.run.smithi165:> sudo systemctl stop ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968@mon.b 2024-09-11T00:33:12.757 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:33:12 smithi165 systemd[1]: Stopping Ceph mon.b for ff3ad74a-6fcf-11ef-bcea-c7b262605968... 2024-09-11T00:33:13.955 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:33:13 smithi165 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-mon-b[35352]: 2024-09-11T00:33:13.535+0000 7f4f41da2640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T00:33:13.956 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:33:13 smithi165 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-mon-b[35352]: 2024-09-11T00:33:13.535+0000 7f4f41da2640 -1 mon.b@2(peon) e3 *** Got Signal Terminated *** 2024-09-11T00:33:13.956 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:33:13 smithi165 podman[73281]: 2024-09-11 00:33:13.566284739 +0000 UTC m=+0.189263550 container died 77449392964c91580fa9aa94f7de799550d86a3308e285bee7b6cc19999096b7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-mon-b, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, ceph=True, CEPH_POINT_RELEASE=, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-11T00:33:14.210 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:33:14 smithi165 podman[73281]: 2024-09-11 00:33:14.053625971 +0000 UTC m=+0.676604781 container cleanup 77449392964c91580fa9aa94f7de799550d86a3308e285bee7b6cc19999096b7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-mon-b, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4) 2024-09-11T00:33:14.210 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:33:14 smithi165 bash[73281]: ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-mon-b 2024-09-11T00:33:14.782 INFO:journalctl@ceph.mon.b.smithi165.stdout:Sep 11 00:33:14 smithi165 podman[73296]: 2024-09-11 00:33:14.591206713 +0000 UTC m=+1.017551887 container remove 77449392964c91580fa9aa94f7de799550d86a3308e285bee7b6cc19999096b7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-mon-b, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS) 2024-09-11T00:33:15.156 DEBUG:teuthology.orchestra.run.smithi165:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968@mon.b.service' 2024-09-11T00:33:15.217 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T00:33:15.217 INFO:tasks.cephadm.mon.c:Stopped mon.b 2024-09-11T00:33:15.217 INFO:tasks.cephadm.mon.c:Stopping mon.c... 2024-09-11T00:33:15.218 DEBUG:teuthology.orchestra.run.smithi178:> sudo systemctl stop ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968@mon.c 2024-09-11T00:33:15.581 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:33:15 smithi178 systemd[1]: Stopping Ceph mon.c for ff3ad74a-6fcf-11ef-bcea-c7b262605968... 2024-09-11T00:33:15.883 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:33:15 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-mon-c[35915]: 2024-09-11T00:33:15.606+0000 7fe68def3640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T00:33:15.883 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:33:15 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-mon-c[35915]: 2024-09-11T00:33:15.606+0000 7fe68def3640 -1 mon.c@1(peon) e3 *** Got Signal Terminated *** 2024-09-11T00:33:15.883 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:33:15 smithi178 podman[78364]: 2024-09-11 00:33:15.718367991 +0000 UTC m=+0.259287404 container died 23ccd19ad543b365a550274fa9c35d98232a7fed1e1f94e1bf2a1838729ae31e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-mon-c, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.schema-version=1.0, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, io.buildah.version=1.37.2, GIT_BRANCH=HEAD) 2024-09-11T00:33:16.535 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:33:16 smithi178 podman[78364]: 2024-09-11 00:33:16.14065717 +0000 UTC m=+0.681576585 container cleanup 23ccd19ad543b365a550274fa9c35d98232a7fed1e1f94e1bf2a1838729ae31e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-mon-c, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, ceph=True, GIT_CLEAN=True) 2024-09-11T00:33:16.535 INFO:journalctl@ceph.mon.c.smithi178.stdout:Sep 11 00:33:16 smithi178 bash[78364]: ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-mon-c 2024-09-11T00:33:16.672 DEBUG:teuthology.orchestra.run.smithi178:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968@mon.c.service' 2024-09-11T00:33:16.734 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T00:33:16.734 INFO:tasks.cephadm.mon.c:Stopped mon.c 2024-09-11T00:33:16.734 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2024-09-11T00:33:16.734 DEBUG:teuthology.orchestra.run.smithi012:> sudo systemctl stop ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968@mgr.x 2024-09-11T00:33:17.063 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Sep 11 00:33:16 smithi012 systemd[1]: Stopping Ceph mgr.x for ff3ad74a-6fcf-11ef-bcea-c7b262605968... 2024-09-11T00:33:17.563 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Sep 11 00:33:17 smithi012 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-mgr-x[30560]: 2024-09-11T00:33:17.185+0000 7fa06bc00640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mgr -n mgr.x -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T00:33:17.563 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Sep 11 00:33:17 smithi012 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-mgr-x[30560]: 2024-09-11T00:33:17.185+0000 7fa06bc00640 -1 mgr handle_mgr_signal *** Got signal Terminated *** 2024-09-11T00:33:17.563 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Sep 11 00:33:17 smithi012 podman[114056]: 2024-09-11 00:33:17.278777812 +0000 UTC m=+0.257846806 container died a392e948a60bd82b4a684df9916521a914799a568af52c75b7f200fd373df4f5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30, name=ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-mgr-x, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-09-11T00:33:17.960 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Sep 11 00:33:17 smithi012 podman[114056]: 2024-09-11 00:33:17.765817871 +0000 UTC m=+0.744886856 container cleanup a392e948a60bd82b4a684df9916521a914799a568af52c75b7f200fd373df4f5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30, name=ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-mgr-x, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, ceph=True, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-11T00:33:17.960 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Sep 11 00:33:17 smithi012 bash[114056]: ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-mgr-x 2024-09-11T00:33:18.219 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Sep 11 00:33:17 smithi012 podman[114070]: 2024-09-11 00:33:17.959098785 +0000 UTC m=+0.675474547 container remove a392e948a60bd82b4a684df9916521a914799a568af52c75b7f200fd373df4f5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30, name=ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-mgr-x, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-11T00:33:18.228 DEBUG:teuthology.orchestra.run.smithi012:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968@mgr.x.service' 2024-09-11T00:33:18.307 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T00:33:18.307 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2024-09-11T00:33:18.307 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2024-09-11T00:33:18.307 DEBUG:teuthology.orchestra.run.smithi165:> sudo systemctl stop ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968@mgr.y 2024-09-11T00:33:18.662 INFO:journalctl@ceph.mgr.y.smithi165.stdout:Sep 11 00:33:18 smithi165 systemd[1]: Stopping Ceph mgr.y for ff3ad74a-6fcf-11ef-bcea-c7b262605968... 2024-09-11T00:33:18.975 INFO:journalctl@ceph.mgr.y.smithi165.stdout:Sep 11 00:33:18 smithi165 podman[75796]: 2024-09-11 00:33:18.660697556 +0000 UTC m=+0.154120068 container stop 7a55726343a6b90f231188e07f209ca366831f3becc0a11087ea828d41b1dbf9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-mgr-y, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, io.buildah.version=1.37.2, ceph=True, org.label-schema.build-date=20240731, GIT_CLEAN=True) 2024-09-11T00:33:18.975 INFO:journalctl@ceph.mgr.y.smithi165.stdout:Sep 11 00:33:18 smithi165 podman[75796]: 2024-09-11 00:33:18.689970551 +0000 UTC m=+0.183393066 container died 7a55726343a6b90f231188e07f209ca366831f3becc0a11087ea828d41b1dbf9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-mgr-y, ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-09-11T00:33:19.283 INFO:journalctl@ceph.mgr.y.smithi165.stdout:Sep 11 00:33:19 smithi165 podman[75796]: 2024-09-11 00:33:19.074273781 +0000 UTC m=+0.567696293 container cleanup 7a55726343a6b90f231188e07f209ca366831f3becc0a11087ea828d41b1dbf9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-mgr-y, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-11T00:33:19.283 INFO:journalctl@ceph.mgr.y.smithi165.stdout:Sep 11 00:33:19 smithi165 bash[75796]: ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-mgr-y 2024-09-11T00:33:19.549 DEBUG:teuthology.orchestra.run.smithi165:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968@mgr.y.service' 2024-09-11T00:33:19.570 INFO:journalctl@ceph.mgr.y.smithi165.stdout:Sep 11 00:33:19 smithi165 podman[75809]: 2024-09-11 00:33:19.318264698 +0000 UTC m=+0.651825972 container remove 7a55726343a6b90f231188e07f209ca366831f3becc0a11087ea828d41b1dbf9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-mgr-y, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, ceph=True, GIT_CLEAN=True, GIT_BRANCH=HEAD) 2024-09-11T00:33:19.571 INFO:journalctl@ceph.mgr.y.smithi165.stdout:Sep 11 00:33:19 smithi165 systemd[1]: ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968@mgr.y.service: Main process exited, code=exited, status=143/n/a 2024-09-11T00:33:19.571 INFO:journalctl@ceph.mgr.y.smithi165.stdout:Sep 11 00:33:19 smithi165 systemd[1]: ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968@mgr.y.service: Failed with result 'exit-code'. 2024-09-11T00:33:19.571 INFO:journalctl@ceph.mgr.y.smithi165.stdout:Sep 11 00:33:19 smithi165 systemd[1]: Stopped Ceph mgr.y for ff3ad74a-6fcf-11ef-bcea-c7b262605968. 2024-09-11T00:33:19.571 INFO:journalctl@ceph.mgr.y.smithi165.stdout:Sep 11 00:33:19 smithi165 systemd[1]: ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968@mgr.y.service: Consumed 7.771s CPU time. 2024-09-11T00:33:19.585 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T00:33:19.585 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2024-09-11T00:33:19.585 INFO:tasks.cephadm.mgr.z:Stopping mgr.z... 2024-09-11T00:33:19.585 DEBUG:teuthology.orchestra.run.smithi178:> sudo systemctl stop ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968@mgr.z 2024-09-11T00:33:19.936 INFO:journalctl@ceph.mgr.z.smithi178.stdout:Sep 11 00:33:19 smithi178 systemd[1]: Stopping Ceph mgr.z for ff3ad74a-6fcf-11ef-bcea-c7b262605968... 2024-09-11T00:33:20.240 INFO:journalctl@ceph.mgr.z.smithi178.stdout:Sep 11 00:33:19 smithi178 podman[78487]: 2024-09-11 00:33:19.935269569 +0000 UTC m=+0.157593623 container died 14fc6152c232862818efb28ca50714ad767f41bc99ea943105835cf378a96bae (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-mgr-z, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, GIT_CLEAN=True) 2024-09-11T00:33:20.581 INFO:journalctl@ceph.mgr.z.smithi178.stdout:Sep 11 00:33:20 smithi178 podman[78487]: 2024-09-11 00:33:20.346203544 +0000 UTC m=+0.568527602 container cleanup 14fc6152c232862818efb28ca50714ad767f41bc99ea943105835cf378a96bae (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-mgr-z, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_BRANCH=HEAD, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_CLEAN=True) 2024-09-11T00:33:20.581 INFO:journalctl@ceph.mgr.z.smithi178.stdout:Sep 11 00:33:20 smithi178 bash[78487]: ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-mgr-z 2024-09-11T00:33:20.757 DEBUG:teuthology.orchestra.run.smithi178:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968@mgr.z.service' 2024-09-11T00:33:20.792 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T00:33:20.792 INFO:tasks.cephadm.mgr.z:Stopped mgr.z 2024-09-11T00:33:20.792 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-09-11T00:33:20.792 DEBUG:teuthology.orchestra.run.smithi012:> sudo systemctl stop ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968@osd.0 2024-09-11T00:33:21.214 INFO:journalctl@ceph.osd.0.smithi012.stdout:Sep 11 00:33:20 smithi012 systemd[1]: Stopping Ceph osd.0 for ff3ad74a-6fcf-11ef-bcea-c7b262605968... 2024-09-11T00:33:21.563 INFO:journalctl@ceph.osd.0.smithi012.stdout:Sep 11 00:33:21 smithi012 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-0[46630]: 2024-09-11T00:33:21.213+0000 7f71a2c00640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T00:33:21.563 INFO:journalctl@ceph.osd.0.smithi012.stdout:Sep 11 00:33:21 smithi012 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-0[46630]: 2024-09-11T00:33:21.213+0000 7f71a2c00640 -1 osd.0 91 *** Got signal Terminated *** 2024-09-11T00:33:21.563 INFO:journalctl@ceph.osd.0.smithi012.stdout:Sep 11 00:33:21 smithi012 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-0[46630]: 2024-09-11T00:33:21.213+0000 7f71a2c00640 -1 osd.0 91 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-11T00:33:27.232 INFO:journalctl@ceph.osd.0.smithi012.stdout:Sep 11 00:33:26 smithi012 podman[114175]: 2024-09-11 00:33:26.908262262 +0000 UTC m=+5.819095002 container died 912c97baed4f2a5facaca6047b523fe0e1f8b14684be538921e32de2a1300fe1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-0, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, ceph=True, org.label-schema.build-date=20240731, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4) 2024-09-11T00:33:27.563 INFO:journalctl@ceph.osd.0.smithi012.stdout:Sep 11 00:33:27 smithi012 podman[114175]: 2024-09-11 00:33:27.386008533 +0000 UTC m=+6.296841282 container cleanup 912c97baed4f2a5facaca6047b523fe0e1f8b14684be538921e32de2a1300fe1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-0, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, GIT_CLEAN=True, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240731) 2024-09-11T00:33:27.564 INFO:journalctl@ceph.osd.0.smithi012.stdout:Sep 11 00:33:27 smithi012 bash[114175]: ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-0 2024-09-11T00:33:28.063 INFO:journalctl@ceph.osd.0.smithi012.stdout:Sep 11 00:33:27 smithi012 podman[114191]: 2024-09-11 00:33:27.623653811 +0000 UTC m=+0.694240397 container remove 912c97baed4f2a5facaca6047b523fe0e1f8b14684be538921e32de2a1300fe1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-0, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.build-date=20240731, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2) 2024-09-11T00:33:28.459 INFO:journalctl@ceph.osd.0.smithi012.stdout:Sep 11 00:33:28 smithi012 podman[114275]: 2024-09-11 00:33:28.100033592 +0000 UTC m=+0.028268839 image pull 589dd18248e494ba411dc838aa4ae0e5e569213f08fcd7a2cf22cd3ef7e3c480 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e 2024-09-11T00:33:28.459 INFO:journalctl@ceph.osd.0.smithi012.stdout:Sep 11 00:33:28 smithi012 podman[114275]: 2024-09-11 00:33:28.238764288 +0000 UTC m=+0.166999521 container create 03e5fff18a82acf53704bc45338e74261644c4bd41b398bef3fb683828abd3eb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-0-deactivate, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-11T00:33:28.725 INFO:journalctl@ceph.osd.0.smithi012.stdout:Sep 11 00:33:28 smithi012 podman[114275]: 2024-09-11 00:33:28.458094798 +0000 UTC m=+0.386330055 container init 03e5fff18a82acf53704bc45338e74261644c4bd41b398bef3fb683828abd3eb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-0-deactivate, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, ceph=True, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-11T00:33:28.725 INFO:journalctl@ceph.osd.0.smithi012.stdout:Sep 11 00:33:28 smithi012 podman[114275]: 2024-09-11 00:33:28.46518258 +0000 UTC m=+0.393417816 container start 03e5fff18a82acf53704bc45338e74261644c4bd41b398bef3fb683828abd3eb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-0-deactivate, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-11T00:33:28.726 INFO:journalctl@ceph.osd.0.smithi012.stdout:Sep 11 00:33:28 smithi012 podman[114275]: 2024-09-11 00:33:28.566025282 +0000 UTC m=+0.494260518 container attach 03e5fff18a82acf53704bc45338e74261644c4bd41b398bef3fb683828abd3eb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-0-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS) 2024-09-11T00:33:29.063 INFO:journalctl@ceph.osd.0.smithi012.stdout:Sep 11 00:33:28 smithi012 podman[114275]: 2024-09-11 00:33:28.7240753 +0000 UTC m=+0.652310537 container died 03e5fff18a82acf53704bc45338e74261644c4bd41b398bef3fb683828abd3eb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-0-deactivate, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, GIT_CLEAN=True, org.label-schema.vendor=CentOS) 2024-09-11T00:33:29.466 DEBUG:teuthology.orchestra.run.smithi012:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968@osd.0.service' 2024-09-11T00:33:29.547 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T00:33:29.547 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-09-11T00:33:29.547 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-09-11T00:33:29.547 DEBUG:teuthology.orchestra.run.smithi165:> sudo systemctl stop ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968@osd.1 2024-09-11T00:33:29.864 INFO:journalctl@ceph.osd.1.smithi165.stdout:Sep 11 00:33:29 smithi165 systemd[1]: Stopping Ceph osd.1 for ff3ad74a-6fcf-11ef-bcea-c7b262605968... 2024-09-11T00:33:30.283 INFO:journalctl@ceph.osd.1.smithi165.stdout:Sep 11 00:33:29 smithi165 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-1[39824]: 2024-09-11T00:33:29.861+0000 7f747dd26640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T00:33:30.283 INFO:journalctl@ceph.osd.1.smithi165.stdout:Sep 11 00:33:29 smithi165 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-1[39824]: 2024-09-11T00:33:29.861+0000 7f747dd26640 -1 osd.1 91 *** Got signal Terminated *** 2024-09-11T00:33:30.283 INFO:journalctl@ceph.osd.1.smithi165.stdout:Sep 11 00:33:29 smithi165 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-1[39824]: 2024-09-11T00:33:29.861+0000 7f747dd26640 -1 osd.1 91 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-11T00:33:35.533 INFO:journalctl@ceph.osd.1.smithi165.stdout:Sep 11 00:33:35 smithi165 podman[75928]: 2024-09-11 00:33:35.273305794 +0000 UTC m=+5.528776901 container died 2c5be6dbcc763ab52b936f4e6874e4ee0d4264de3f4768d498ec753cddc0a51e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-1, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, ceph=True, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, io.buildah.version=1.37.2) 2024-09-11T00:33:35.925 INFO:journalctl@ceph.osd.1.smithi165.stdout:Sep 11 00:33:35 smithi165 podman[75928]: 2024-09-11 00:33:35.68809995 +0000 UTC m=+5.943571053 container cleanup 2c5be6dbcc763ab52b936f4e6874e4ee0d4264de3f4768d498ec753cddc0a51e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-1, GIT_CLEAN=True, org.label-schema.build-date=20240731, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-11T00:33:35.925 INFO:journalctl@ceph.osd.1.smithi165.stdout:Sep 11 00:33:35 smithi165 bash[75928]: ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-1 2024-09-11T00:33:36.282 INFO:journalctl@ceph.osd.1.smithi165.stdout:Sep 11 00:33:35 smithi165 podman[75944]: 2024-09-11 00:33:35.923118161 +0000 UTC m=+0.652260789 container remove 2c5be6dbcc763ab52b936f4e6874e4ee0d4264de3f4768d498ec753cddc0a51e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-1, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , ceph=True, GIT_CLEAN=True) 2024-09-11T00:33:36.728 INFO:journalctl@ceph.osd.1.smithi165.stdout:Sep 11 00:33:36 smithi165 podman[76032]: 2024-09-11 00:33:36.374093102 +0000 UTC m=+0.111912214 container create 31299b743cd065704bed8b75202239bcd17bad74a515ea69937caf4a11bce828 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-1-deactivate, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS) 2024-09-11T00:33:36.728 INFO:journalctl@ceph.osd.1.smithi165.stdout:Sep 11 00:33:36 smithi165 podman[76032]: 2024-09-11 00:33:36.27770042 +0000 UTC m=+0.015519551 image pull 589dd18248e494ba411dc838aa4ae0e5e569213f08fcd7a2cf22cd3ef7e3c480 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e 2024-09-11T00:33:36.728 INFO:journalctl@ceph.osd.1.smithi165.stdout:Sep 11 00:33:36 smithi165 podman[76032]: 2024-09-11 00:33:36.610263854 +0000 UTC m=+0.348082966 container init 31299b743cd065704bed8b75202239bcd17bad74a515ea69937caf4a11bce828 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-1-deactivate, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-09-11T00:33:36.728 INFO:journalctl@ceph.osd.1.smithi165.stdout:Sep 11 00:33:36 smithi165 podman[76032]: 2024-09-11 00:33:36.613565521 +0000 UTC m=+0.351384641 container start 31299b743cd065704bed8b75202239bcd17bad74a515ea69937caf4a11bce828 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-1-deactivate, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-11T00:33:36.728 INFO:journalctl@ceph.osd.1.smithi165.stdout:Sep 11 00:33:36 smithi165 podman[76032]: 2024-09-11 00:33:36.72703652 +0000 UTC m=+0.464855656 container attach 31299b743cd065704bed8b75202239bcd17bad74a515ea69937caf4a11bce828 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-1-deactivate, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD) 2024-09-11T00:33:37.033 INFO:journalctl@ceph.osd.1.smithi165.stdout:Sep 11 00:33:36 smithi165 podman[76032]: 2024-09-11 00:33:36.800883682 +0000 UTC m=+0.538702813 container died 31299b743cd065704bed8b75202239bcd17bad74a515ea69937caf4a11bce828 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-1-deactivate, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, GIT_BRANCH=HEAD, io.buildah.version=1.37.2) 2024-09-11T00:33:37.484 DEBUG:teuthology.orchestra.run.smithi165:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968@osd.1.service' 2024-09-11T00:33:37.524 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T00:33:37.524 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-09-11T00:33:37.525 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-09-11T00:33:37.525 DEBUG:teuthology.orchestra.run.smithi178:> sudo systemctl stop ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968@osd.2 2024-09-11T00:33:37.831 INFO:journalctl@ceph.osd.2.smithi178.stdout:Sep 11 00:33:37 smithi178 systemd[1]: Stopping Ceph osd.2 for ff3ad74a-6fcf-11ef-bcea-c7b262605968... 2024-09-11T00:33:38.331 INFO:journalctl@ceph.osd.2.smithi178.stdout:Sep 11 00:33:37 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-2[40889]: 2024-09-11T00:33:37.849+0000 7f31170a3640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T00:33:38.331 INFO:journalctl@ceph.osd.2.smithi178.stdout:Sep 11 00:33:37 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-2[40889]: 2024-09-11T00:33:37.849+0000 7f31170a3640 -1 osd.2 91 *** Got signal Terminated *** 2024-09-11T00:33:38.331 INFO:journalctl@ceph.osd.2.smithi178.stdout:Sep 11 00:33:37 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-2[40889]: 2024-09-11T00:33:37.849+0000 7f31170a3640 -1 osd.2 91 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-11T00:33:43.581 INFO:journalctl@ceph.osd.2.smithi178.stdout:Sep 11 00:33:43 smithi178 podman[78616]: 2024-09-11 00:33:43.290254631 +0000 UTC m=+5.564365340 container died f44b163d0fef0cfc95fe40e5a600a818e674cb1f04d03ce1898e9684136d1a7d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.37.2) 2024-09-11T00:33:43.982 INFO:journalctl@ceph.osd.2.smithi178.stdout:Sep 11 00:33:43 smithi178 podman[78616]: 2024-09-11 00:33:43.683790309 +0000 UTC m=+5.957901017 container cleanup f44b163d0fef0cfc95fe40e5a600a818e674cb1f04d03ce1898e9684136d1a7d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-2, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, io.buildah.version=1.37.2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-09-11T00:33:43.982 INFO:journalctl@ceph.osd.2.smithi178.stdout:Sep 11 00:33:43 smithi178 bash[78616]: ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-2 2024-09-11T00:33:44.296 INFO:journalctl@ceph.osd.2.smithi178.stdout:Sep 11 00:33:43 smithi178 podman[78632]: 2024-09-11 00:33:43.981153809 +0000 UTC m=+0.688238978 container remove f44b163d0fef0cfc95fe40e5a600a818e674cb1f04d03ce1898e9684136d1a7d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-2, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_BRANCH=HEAD, ceph=True, org.label-schema.schema-version=1.0) 2024-09-11T00:33:44.557 INFO:journalctl@ceph.osd.2.smithi178.stdout:Sep 11 00:33:44 smithi178 podman[78721]: 2024-09-11 00:33:44.197125583 +0000 UTC m=+0.015427937 image pull 589dd18248e494ba411dc838aa4ae0e5e569213f08fcd7a2cf22cd3ef7e3c480 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e 2024-09-11T00:33:44.557 INFO:journalctl@ceph.osd.2.smithi178.stdout:Sep 11 00:33:44 smithi178 podman[78721]: 2024-09-11 00:33:44.311435795 +0000 UTC m=+0.129738195 container create 544b73dde0e4287144a6cbb3c7c62d592dec288a0527ca846e096aebfce414a4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-2-deactivate, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD) 2024-09-11T00:33:44.831 INFO:journalctl@ceph.osd.2.smithi178.stdout:Sep 11 00:33:44 smithi178 podman[78721]: 2024-09-11 00:33:44.555802125 +0000 UTC m=+0.374104479 container init 544b73dde0e4287144a6cbb3c7c62d592dec288a0527ca846e096aebfce414a4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-2-deactivate, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux ) 2024-09-11T00:33:44.831 INFO:journalctl@ceph.osd.2.smithi178.stdout:Sep 11 00:33:44 smithi178 podman[78721]: 2024-09-11 00:33:44.559154687 +0000 UTC m=+0.377457042 container start 544b73dde0e4287144a6cbb3c7c62d592dec288a0527ca846e096aebfce414a4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-2-deactivate, GIT_CLEAN=True, maintainer=Guillaume Abrioux , RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-09-11T00:33:44.831 INFO:journalctl@ceph.osd.2.smithi178.stdout:Sep 11 00:33:44 smithi178 podman[78721]: 2024-09-11 00:33:44.639016866 +0000 UTC m=+0.457319221 container attach 544b73dde0e4287144a6cbb3c7c62d592dec288a0527ca846e096aebfce414a4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-2-deactivate, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, ceph=True, org.label-schema.build-date=20240731, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.vendor=CentOS) 2024-09-11T00:33:44.832 INFO:journalctl@ceph.osd.2.smithi178.stdout:Sep 11 00:33:44 smithi178 podman[78721]: 2024-09-11 00:33:44.743132103 +0000 UTC m=+0.561434464 container died 544b73dde0e4287144a6cbb3c7c62d592dec288a0527ca846e096aebfce414a4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-2-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, ceph=True, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS) 2024-09-11T00:33:45.331 DEBUG:teuthology.orchestra.run.smithi178:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968@osd.2.service' 2024-09-11T00:33:45.372 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T00:33:45.372 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-09-11T00:33:45.372 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-09-11T00:33:45.372 DEBUG:teuthology.orchestra.run.smithi012:> sudo systemctl stop ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968@osd.3 2024-09-11T00:33:45.813 INFO:journalctl@ceph.osd.3.smithi012.stdout:Sep 11 00:33:45 smithi012 systemd[1]: Stopping Ceph osd.3 for ff3ad74a-6fcf-11ef-bcea-c7b262605968... 2024-09-11T00:33:46.150 INFO:journalctl@ceph.osd.3.smithi012.stdout:Sep 11 00:33:45 smithi012 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-3[53245]: 2024-09-11T00:33:45.959+0000 7f974a800640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T00:33:46.150 INFO:journalctl@ceph.osd.3.smithi012.stdout:Sep 11 00:33:45 smithi012 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-3[53245]: 2024-09-11T00:33:45.959+0000 7f974a800640 -1 osd.3 91 *** Got signal Terminated *** 2024-09-11T00:33:46.150 INFO:journalctl@ceph.osd.3.smithi012.stdout:Sep 11 00:33:45 smithi012 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-3[53245]: 2024-09-11T00:33:45.959+0000 7f974a800640 -1 osd.3 91 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-11T00:33:48.313 INFO:journalctl@ceph.osd.9.smithi012.stdout:Sep 11 00:33:47 smithi012 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-9[66733]: 2024-09-11T00:33:47.984+0000 7f5f4e800640 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-09-11T00:33:25.406156+0000 front 2024-09-11T00:33:25.406095+0000 (oldest deadline 2024-09-11T00:33:47.706239+0000) 2024-09-11T00:33:49.313 INFO:journalctl@ceph.osd.9.smithi012.stdout:Sep 11 00:33:48 smithi012 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-9[66733]: 2024-09-11T00:33:48.990+0000 7f5f4e800640 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-09-11T00:33:25.406156+0000 front 2024-09-11T00:33:25.406095+0000 (oldest deadline 2024-09-11T00:33:47.706239+0000) 2024-09-11T00:33:50.313 INFO:journalctl@ceph.osd.9.smithi012.stdout:Sep 11 00:33:49 smithi012 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-9[66733]: 2024-09-11T00:33:49.985+0000 7f5f4e800640 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-09-11T00:33:25.406156+0000 front 2024-09-11T00:33:25.406095+0000 (oldest deadline 2024-09-11T00:33:47.706239+0000) 2024-09-11T00:33:50.313 INFO:journalctl@ceph.osd.3.smithi012.stdout:Sep 11 00:33:49 smithi012 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-3[53245]: 2024-09-11T00:33:49.963+0000 7f9741200640 -1 osd.3 91 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-09-11T00:33:25.493158+0000 front 2024-09-11T00:33:25.493213+0000 (oldest deadline 2024-09-11T00:33:49.593389+0000) 2024-09-11T00:33:50.581 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 00:33:50 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-11[60852]: 2024-09-11T00:33:50.443+0000 7fd754b0b640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-09-11T00:33:24.081060+0000 front 2024-09-11T00:33:24.081080+0000 (oldest deadline 2024-09-11T00:33:49.980779+0000) 2024-09-11T00:33:50.582 INFO:journalctl@ceph.osd.5.smithi178.stdout:Sep 11 00:33:50 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-5[47367]: 2024-09-11T00:33:50.235+0000 7fc6e39ea640 -1 osd.5 91 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-09-11T00:33:25.841798+0000 front 2024-09-11T00:33:25.841738+0000 (oldest deadline 2024-09-11T00:33:49.941528+0000) 2024-09-11T00:33:51.033 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 11 00:33:50 smithi165 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-10[59927]: 2024-09-11T00:33:50.555+0000 7f2cff41d640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-09-11T00:33:25.146772+0000 front 2024-09-11T00:33:25.146620+0000 (oldest deadline 2024-09-11T00:33:49.846529+0000) 2024-09-11T00:33:51.313 INFO:journalctl@ceph.osd.3.smithi012.stdout:Sep 11 00:33:50 smithi012 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-3[53245]: 2024-09-11T00:33:50.933+0000 7f9741200640 -1 osd.3 91 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-09-11T00:33:25.493158+0000 front 2024-09-11T00:33:25.493213+0000 (oldest deadline 2024-09-11T00:33:49.593389+0000) 2024-09-11T00:33:51.314 INFO:journalctl@ceph.osd.9.smithi012.stdout:Sep 11 00:33:50 smithi012 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-9[66733]: 2024-09-11T00:33:50.955+0000 7f5f4e800640 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-09-11T00:33:25.406156+0000 front 2024-09-11T00:33:25.406095+0000 (oldest deadline 2024-09-11T00:33:47.706239+0000) 2024-09-11T00:33:51.581 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 00:33:51 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-11[60852]: 2024-09-11T00:33:51.464+0000 7fd754b0b640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-09-11T00:33:24.081060+0000 front 2024-09-11T00:33:24.081080+0000 (oldest deadline 2024-09-11T00:33:49.980779+0000) 2024-09-11T00:33:51.581 INFO:journalctl@ceph.osd.5.smithi178.stdout:Sep 11 00:33:51 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-5[47367]: 2024-09-11T00:33:51.277+0000 7fc6e39ea640 -1 osd.5 91 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-09-11T00:33:25.841798+0000 front 2024-09-11T00:33:25.841738+0000 (oldest deadline 2024-09-11T00:33:49.941528+0000) 2024-09-11T00:33:51.880 INFO:journalctl@ceph.osd.3.smithi012.stdout:Sep 11 00:33:51 smithi012 podman[114390]: 2024-09-11 00:33:51.575723811 +0000 UTC m=+5.860108450 container died 61cfe8f7eaa5cf06f36411516ee7dc4921f069197bc73852d221e61825f5c62f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-3, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-09-11T00:33:52.032 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 11 00:33:51 smithi165 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-10[59927]: 2024-09-11T00:33:51.572+0000 7f2cff41d640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-09-11T00:33:25.146772+0000 front 2024-09-11T00:33:25.146620+0000 (oldest deadline 2024-09-11T00:33:49.846529+0000) 2024-09-11T00:33:52.205 INFO:journalctl@ceph.osd.9.smithi012.stdout:Sep 11 00:33:51 smithi012 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-9[66733]: 2024-09-11T00:33:51.951+0000 7f5f4e800640 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-09-11T00:33:25.406156+0000 front 2024-09-11T00:33:25.406095+0000 (oldest deadline 2024-09-11T00:33:47.706239+0000) 2024-09-11T00:33:52.507 INFO:journalctl@ceph.osd.3.smithi012.stdout:Sep 11 00:33:52 smithi012 podman[114390]: 2024-09-11 00:33:52.203778408 +0000 UTC m=+6.488163046 container cleanup 61cfe8f7eaa5cf06f36411516ee7dc4921f069197bc73852d221e61825f5c62f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-3, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, CEPH_POINT_RELEASE=) 2024-09-11T00:33:52.507 INFO:journalctl@ceph.osd.3.smithi012.stdout:Sep 11 00:33:52 smithi012 bash[114390]: ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-3 2024-09-11T00:33:52.581 INFO:journalctl@ceph.osd.5.smithi178.stdout:Sep 11 00:33:52 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-5[47367]: 2024-09-11T00:33:52.291+0000 7fc6e39ea640 -1 osd.5 91 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-09-11T00:33:25.841798+0000 front 2024-09-11T00:33:25.841738+0000 (oldest deadline 2024-09-11T00:33:49.941528+0000) 2024-09-11T00:33:52.581 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 00:33:52 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-11[60852]: 2024-09-11T00:33:52.469+0000 7fd754b0b640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-09-11T00:33:24.081060+0000 front 2024-09-11T00:33:24.081080+0000 (oldest deadline 2024-09-11T00:33:49.980779+0000) 2024-09-11T00:33:52.813 INFO:journalctl@ceph.osd.3.smithi012.stdout:Sep 11 00:33:52 smithi012 podman[114418]: 2024-09-11 00:33:52.50597609 +0000 UTC m=+0.922522689 container remove 61cfe8f7eaa5cf06f36411516ee7dc4921f069197bc73852d221e61825f5c62f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-3, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.vendor=CentOS, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True) 2024-09-11T00:33:53.033 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 11 00:33:52 smithi165 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-10[59927]: 2024-09-11T00:33:52.580+0000 7f2cff41d640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-09-11T00:33:25.146772+0000 front 2024-09-11T00:33:25.146620+0000 (oldest deadline 2024-09-11T00:33:49.846529+0000) 2024-09-11T00:33:53.300 INFO:journalctl@ceph.osd.3.smithi012.stdout:Sep 11 00:33:53 smithi012 podman[114505]: 2024-09-11 00:33:53.03221881 +0000 UTC m=+0.027333266 image pull 589dd18248e494ba411dc838aa4ae0e5e569213f08fcd7a2cf22cd3ef7e3c480 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e 2024-09-11T00:33:53.300 INFO:journalctl@ceph.osd.9.smithi012.stdout:Sep 11 00:33:52 smithi012 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-9[66733]: 2024-09-11T00:33:52.987+0000 7f5f4e800640 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-09-11T00:33:25.406156+0000 front 2024-09-11T00:33:25.406095+0000 (oldest deadline 2024-09-11T00:33:47.706239+0000) 2024-09-11T00:33:53.552 INFO:journalctl@ceph.osd.3.smithi012.stdout:Sep 11 00:33:53 smithi012 podman[114505]: 2024-09-11 00:33:53.298723079 +0000 UTC m=+0.293837520 container create adedf64e3c662d77e4f61d9b84d0dfd2e9b4ce7536cf42f1a4d1145cfb97bcd2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-3-deactivate, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, io.buildah.version=1.37.2) 2024-09-11T00:33:53.552 INFO:journalctl@ceph.osd.3.smithi012.stdout:Sep 11 00:33:53 smithi012 podman[114505]: 2024-09-11 00:33:53.543699183 +0000 UTC m=+0.538813624 container init adedf64e3c662d77e4f61d9b84d0dfd2e9b4ce7536cf42f1a4d1145cfb97bcd2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-3-deactivate, ceph=True, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4) 2024-09-11T00:33:53.552 INFO:journalctl@ceph.osd.3.smithi012.stdout:Sep 11 00:33:53 smithi012 podman[114505]: 2024-09-11 00:33:53.550611702 +0000 UTC m=+0.545726146 container start adedf64e3c662d77e4f61d9b84d0dfd2e9b4ce7536cf42f1a4d1145cfb97bcd2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-3-deactivate, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , ceph=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.license=GPLv2) 2024-09-11T00:33:53.581 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 00:33:53 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-11[60852]: 2024-09-11T00:33:53.480+0000 7fd754b0b640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-09-11T00:33:24.081060+0000 front 2024-09-11T00:33:24.081080+0000 (oldest deadline 2024-09-11T00:33:49.980779+0000) 2024-09-11T00:33:53.581 INFO:journalctl@ceph.osd.5.smithi178.stdout:Sep 11 00:33:53 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-5[47367]: 2024-09-11T00:33:53.256+0000 7fc6e39ea640 -1 osd.5 91 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-09-11T00:33:25.841798+0000 front 2024-09-11T00:33:25.841738+0000 (oldest deadline 2024-09-11T00:33:49.941528+0000) 2024-09-11T00:33:53.813 INFO:journalctl@ceph.osd.3.smithi012.stdout:Sep 11 00:33:53 smithi012 podman[114505]: 2024-09-11 00:33:53.659289812 +0000 UTC m=+0.654404266 container attach adedf64e3c662d77e4f61d9b84d0dfd2e9b4ce7536cf42f1a4d1145cfb97bcd2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-3-deactivate, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.build-date=20240731) 2024-09-11T00:33:54.032 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 11 00:33:53 smithi165 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-10[59927]: 2024-09-11T00:33:53.618+0000 7f2cff41d640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-09-11T00:33:25.146772+0000 front 2024-09-11T00:33:25.146620+0000 (oldest deadline 2024-09-11T00:33:49.846529+0000) 2024-09-11T00:33:54.154 INFO:journalctl@ceph.osd.3.smithi012.stdout:Sep 11 00:33:53 smithi012 podman[114505]: 2024-09-11 00:33:53.81217667 +0000 UTC m=+0.807291106 container died adedf64e3c662d77e4f61d9b84d0dfd2e9b4ce7536cf42f1a4d1145cfb97bcd2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-3-deactivate, org.label-schema.build-date=20240731, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, ceph=True, org.label-schema.schema-version=1.0) 2024-09-11T00:33:54.155 INFO:journalctl@ceph.osd.9.smithi012.stdout:Sep 11 00:33:54 smithi012 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-9[66733]: 2024-09-11T00:33:54.029+0000 7f5f4e800640 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-09-11T00:33:25.406156+0000 front 2024-09-11T00:33:25.406095+0000 (oldest deadline 2024-09-11T00:33:47.706239+0000) 2024-09-11T00:33:54.431 INFO:journalctl@ceph.osd.3.smithi012.stdout:Sep 11 00:33:54 smithi012 podman[114528]: 2024-09-11 00:33:54.404267828 +0000 UTC m=+0.578348896 container remove adedf64e3c662d77e4f61d9b84d0dfd2e9b4ce7536cf42f1a4d1145cfb97bcd2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-3-deactivate, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.schema-version=1.0) 2024-09-11T00:33:54.440 DEBUG:teuthology.orchestra.run.smithi012:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968@osd.3.service' 2024-09-11T00:33:54.517 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T00:33:54.517 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-09-11T00:33:54.518 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-09-11T00:33:54.518 DEBUG:teuthology.orchestra.run.smithi165:> sudo systemctl stop ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968@osd.4 2024-09-11T00:33:54.522 INFO:journalctl@ceph.osd.5.smithi178.stdout:Sep 11 00:33:54 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-5[47367]: 2024-09-11T00:33:54.251+0000 7fc6e39ea640 -1 osd.5 91 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-09-11T00:33:25.841798+0000 front 2024-09-11T00:33:25.841738+0000 (oldest deadline 2024-09-11T00:33:49.941528+0000) 2024-09-11T00:33:54.831 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 00:33:54 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-11[60852]: 2024-09-11T00:33:54.521+0000 7fd754b0b640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-09-11T00:33:24.081060+0000 front 2024-09-11T00:33:24.081080+0000 (oldest deadline 2024-09-11T00:33:49.980779+0000) 2024-09-11T00:33:54.985 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 11 00:33:54 smithi165 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-10[59927]: 2024-09-11T00:33:54.644+0000 7f2cff41d640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-09-11T00:33:25.146772+0000 front 2024-09-11T00:33:25.146620+0000 (oldest deadline 2024-09-11T00:33:49.846529+0000) 2024-09-11T00:33:54.985 INFO:journalctl@ceph.osd.4.smithi165.stdout:Sep 11 00:33:54 smithi165 systemd[1]: Stopping Ceph osd.4 for ff3ad74a-6fcf-11ef-bcea-c7b262605968... 2024-09-11T00:33:55.283 INFO:journalctl@ceph.osd.4.smithi165.stdout:Sep 11 00:33:54 smithi165 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-4[46433]: 2024-09-11T00:33:54.983+0000 7f8663e11640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T00:33:55.283 INFO:journalctl@ceph.osd.4.smithi165.stdout:Sep 11 00:33:54 smithi165 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-4[46433]: 2024-09-11T00:33:54.983+0000 7f8663e11640 -1 osd.4 91 *** Got signal Terminated *** 2024-09-11T00:33:55.283 INFO:journalctl@ceph.osd.4.smithi165.stdout:Sep 11 00:33:54 smithi165 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-4[46433]: 2024-09-11T00:33:54.983+0000 7f8663e11640 -1 osd.4 91 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-11T00:33:55.313 INFO:journalctl@ceph.osd.9.smithi012.stdout:Sep 11 00:33:55 smithi012 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-9[66733]: 2024-09-11T00:33:55.054+0000 7f5f4e800640 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-09-11T00:33:25.406156+0000 front 2024-09-11T00:33:25.406095+0000 (oldest deadline 2024-09-11T00:33:47.706239+0000) 2024-09-11T00:33:55.581 INFO:journalctl@ceph.osd.5.smithi178.stdout:Sep 11 00:33:55 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-5[47367]: 2024-09-11T00:33:55.241+0000 7fc6e39ea640 -1 osd.5 91 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-09-11T00:33:25.841798+0000 front 2024-09-11T00:33:25.841738+0000 (oldest deadline 2024-09-11T00:33:49.941528+0000) 2024-09-11T00:33:55.581 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 00:33:55 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-11[60852]: 2024-09-11T00:33:55.486+0000 7fd754b0b640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-09-11T00:33:24.081060+0000 front 2024-09-11T00:33:24.081080+0000 (oldest deadline 2024-09-11T00:33:49.980779+0000) 2024-09-11T00:33:56.033 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 11 00:33:55 smithi165 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-10[59927]: 2024-09-11T00:33:55.597+0000 7f2cff41d640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-09-11T00:33:25.146772+0000 front 2024-09-11T00:33:25.146620+0000 (oldest deadline 2024-09-11T00:33:49.846529+0000) 2024-09-11T00:33:56.481 INFO:journalctl@ceph.osd.9.smithi012.stdout:Sep 11 00:33:56 smithi012 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-9[66733]: 2024-09-11T00:33:56.068+0000 7f5f4e800640 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-09-11T00:33:25.406156+0000 front 2024-09-11T00:33:25.406095+0000 (oldest deadline 2024-09-11T00:33:47.706239+0000) 2024-09-11T00:33:56.481 INFO:journalctl@ceph.osd.9.smithi012.stdout:Sep 11 00:33:56 smithi012 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-9[66733]: 2024-09-11T00:33:56.068+0000 7f5f4e800640 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-11T00:33:30.607371+0000 front 2024-09-11T00:33:30.607499+0000 (oldest deadline 2024-09-11T00:33:55.307466+0000) 2024-09-11T00:33:56.581 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 00:33:56 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-11[60852]: 2024-09-11T00:33:56.445+0000 7fd754b0b640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-09-11T00:33:24.081060+0000 front 2024-09-11T00:33:24.081080+0000 (oldest deadline 2024-09-11T00:33:49.980779+0000) 2024-09-11T00:33:56.582 INFO:journalctl@ceph.osd.5.smithi178.stdout:Sep 11 00:33:56 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-5[47367]: 2024-09-11T00:33:56.277+0000 7fc6e39ea640 -1 osd.5 91 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-09-11T00:33:25.841798+0000 front 2024-09-11T00:33:25.841738+0000 (oldest deadline 2024-09-11T00:33:49.941528+0000) 2024-09-11T00:33:56.640 INFO:journalctl@ceph.osd.4.smithi165.stdout:Sep 11 00:33:56 smithi165 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-4[46433]: 2024-09-11T00:33:56.344+0000 7f8660426640 -1 osd.4 91 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-11T00:33:32.677220+0000 front 2024-09-11T00:33:32.677181+0000 (oldest deadline 2024-09-11T00:33:55.577404+0000) 2024-09-11T00:33:57.019 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 11 00:33:56 smithi165 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-10[59927]: 2024-09-11T00:33:56.638+0000 7f2cff41d640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-09-11T00:33:25.146772+0000 front 2024-09-11T00:33:25.146620+0000 (oldest deadline 2024-09-11T00:33:49.846529+0000) 2024-09-11T00:33:57.283 INFO:journalctl@ceph.osd.7.smithi165.stdout:Sep 11 00:33:57 smithi165 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-7[52941]: 2024-09-11T00:33:57.018+0000 7ffbe0ef8640 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-11T00:33:32.707987+0000 front 2024-09-11T00:33:32.708022+0000 (oldest deadline 2024-09-11T00:33:56.808116+0000) 2024-09-11T00:33:57.313 INFO:journalctl@ceph.osd.9.smithi012.stdout:Sep 11 00:33:57 smithi012 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-9[66733]: 2024-09-11T00:33:57.020+0000 7f5f4e800640 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-09-11T00:33:25.406156+0000 front 2024-09-11T00:33:25.406095+0000 (oldest deadline 2024-09-11T00:33:47.706239+0000) 2024-09-11T00:33:57.313 INFO:journalctl@ceph.osd.9.smithi012.stdout:Sep 11 00:33:57 smithi012 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-9[66733]: 2024-09-11T00:33:57.020+0000 7f5f4e800640 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-11T00:33:30.607371+0000 front 2024-09-11T00:33:30.607499+0000 (oldest deadline 2024-09-11T00:33:55.307466+0000) 2024-09-11T00:33:57.466 INFO:journalctl@ceph.osd.5.smithi178.stdout:Sep 11 00:33:57 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-5[47367]: 2024-09-11T00:33:57.325+0000 7fc6e39ea640 -1 osd.5 91 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-09-11T00:33:25.841798+0000 front 2024-09-11T00:33:25.841738+0000 (oldest deadline 2024-09-11T00:33:49.941528+0000) 2024-09-11T00:33:57.467 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 11 00:33:57 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-8[54042]: 2024-09-11T00:33:57.156+0000 7f23083d6640 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-11T00:33:33.673118+0000 front 2024-09-11T00:33:33.673042+0000 (oldest deadline 2024-09-11T00:33:56.573057+0000) 2024-09-11T00:33:57.664 INFO:journalctl@ceph.osd.4.smithi165.stdout:Sep 11 00:33:57 smithi165 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-4[46433]: 2024-09-11T00:33:57.362+0000 7f8660426640 -1 osd.4 91 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-11T00:33:32.677220+0000 front 2024-09-11T00:33:32.677181+0000 (oldest deadline 2024-09-11T00:33:55.577404+0000) 2024-09-11T00:33:57.831 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 00:33:57 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-11[60852]: 2024-09-11T00:33:57.465+0000 7fd754b0b640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-09-11T00:33:24.081060+0000 front 2024-09-11T00:33:24.081080+0000 (oldest deadline 2024-09-11T00:33:49.980779+0000) 2024-09-11T00:33:58.032 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 11 00:33:57 smithi165 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-10[59927]: 2024-09-11T00:33:57.662+0000 7f2cff41d640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-09-11T00:33:25.146772+0000 front 2024-09-11T00:33:25.146620+0000 (oldest deadline 2024-09-11T00:33:49.846529+0000) 2024-09-11T00:33:58.313 INFO:journalctl@ceph.osd.9.smithi012.stdout:Sep 11 00:33:58 smithi012 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-9[66733]: 2024-09-11T00:33:58.040+0000 7f5f4e800640 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-09-11T00:33:25.406156+0000 front 2024-09-11T00:33:25.406095+0000 (oldest deadline 2024-09-11T00:33:47.706239+0000) 2024-09-11T00:33:58.313 INFO:journalctl@ceph.osd.9.smithi012.stdout:Sep 11 00:33:58 smithi012 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-9[66733]: 2024-09-11T00:33:58.040+0000 7f5f4e800640 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-11T00:33:30.607371+0000 front 2024-09-11T00:33:30.607499+0000 (oldest deadline 2024-09-11T00:33:55.307466+0000) 2024-09-11T00:33:58.314 INFO:journalctl@ceph.osd.6.smithi012.stdout:Sep 11 00:33:57 smithi012 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-6[59757]: 2024-09-11T00:33:57.827+0000 7f2e71000640 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-11T00:33:34.716146+0000 front 2024-09-11T00:33:34.715917+0000 (oldest deadline 2024-09-11T00:33:57.015981+0000) 2024-09-11T00:33:58.361 INFO:journalctl@ceph.osd.7.smithi165.stdout:Sep 11 00:33:58 smithi165 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-7[52941]: 2024-09-11T00:33:58.035+0000 7ffbe0ef8640 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-11T00:33:32.707987+0000 front 2024-09-11T00:33:32.708022+0000 (oldest deadline 2024-09-11T00:33:56.808116+0000) 2024-09-11T00:33:58.489 INFO:journalctl@ceph.osd.5.smithi178.stdout:Sep 11 00:33:58 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-5[47367]: 2024-09-11T00:33:58.322+0000 7fc6e39ea640 -1 osd.5 91 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-09-11T00:33:25.841798+0000 front 2024-09-11T00:33:25.841738+0000 (oldest deadline 2024-09-11T00:33:49.941528+0000) 2024-09-11T00:33:58.489 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 11 00:33:58 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-8[54042]: 2024-09-11T00:33:58.153+0000 7f23083d6640 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-11T00:33:33.673118+0000 front 2024-09-11T00:33:33.673042+0000 (oldest deadline 2024-09-11T00:33:56.573057+0000) 2024-09-11T00:33:58.710 INFO:journalctl@ceph.osd.4.smithi165.stdout:Sep 11 00:33:58 smithi165 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-4[46433]: 2024-09-11T00:33:58.360+0000 7f8660426640 -1 osd.4 91 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-11T00:33:32.677220+0000 front 2024-09-11T00:33:32.677181+0000 (oldest deadline 2024-09-11T00:33:55.577404+0000) 2024-09-11T00:33:58.831 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 00:33:58 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-11[60852]: 2024-09-11T00:33:58.487+0000 7fd754b0b640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-09-11T00:33:24.081060+0000 front 2024-09-11T00:33:24.081080+0000 (oldest deadline 2024-09-11T00:33:49.980779+0000) 2024-09-11T00:33:59.033 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 11 00:33:58 smithi165 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-10[59927]: 2024-09-11T00:33:58.709+0000 7f2cff41d640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-09-11T00:33:25.146772+0000 front 2024-09-11T00:33:25.146620+0000 (oldest deadline 2024-09-11T00:33:49.846529+0000) 2024-09-11T00:33:59.033 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 11 00:33:58 smithi165 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-10[59927]: 2024-09-11T00:33:58.709+0000 7f2cff41d640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-11T00:33:32.747514+0000 front 2024-09-11T00:33:32.747491+0000 (oldest deadline 2024-09-11T00:33:58.047686+0000) 2024-09-11T00:33:59.063 INFO:journalctl@ceph.osd.6.smithi012.stdout:Sep 11 00:33:58 smithi012 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-6[59757]: 2024-09-11T00:33:58.798+0000 7f2e71000640 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-11T00:33:34.716146+0000 front 2024-09-11T00:33:34.715917+0000 (oldest deadline 2024-09-11T00:33:57.015981+0000) 2024-09-11T00:33:59.364 INFO:journalctl@ceph.osd.7.smithi165.stdout:Sep 11 00:33:59 smithi165 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-7[52941]: 2024-09-11T00:33:59.080+0000 7ffbe0ef8640 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-11T00:33:32.707987+0000 front 2024-09-11T00:33:32.708022+0000 (oldest deadline 2024-09-11T00:33:56.808116+0000) 2024-09-11T00:33:59.539 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 11 00:33:59 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-8[54042]: 2024-09-11T00:33:59.181+0000 7f23083d6640 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-11T00:33:33.673118+0000 front 2024-09-11T00:33:33.673042+0000 (oldest deadline 2024-09-11T00:33:56.573057+0000) 2024-09-11T00:33:59.539 INFO:journalctl@ceph.osd.5.smithi178.stdout:Sep 11 00:33:59 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-5[47367]: 2024-09-11T00:33:59.346+0000 7fc6e39ea640 -1 osd.5 91 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-09-11T00:33:25.841798+0000 front 2024-09-11T00:33:25.841738+0000 (oldest deadline 2024-09-11T00:33:49.941528+0000) 2024-09-11T00:33:59.539 INFO:journalctl@ceph.osd.5.smithi178.stdout:Sep 11 00:33:59 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-5[47367]: 2024-09-11T00:33:59.346+0000 7fc6e39ea640 -1 osd.5 91 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-11T00:33:34.042676+0000 front 2024-09-11T00:33:34.042628+0000 (oldest deadline 2024-09-11T00:33:59.342829+0000) 2024-09-11T00:33:59.563 INFO:journalctl@ceph.osd.9.smithi012.stdout:Sep 11 00:33:59 smithi012 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-9[66733]: 2024-09-11T00:33:59.069+0000 7f5f4e800640 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-09-11T00:33:25.406156+0000 front 2024-09-11T00:33:25.406095+0000 (oldest deadline 2024-09-11T00:33:47.706239+0000) 2024-09-11T00:33:59.563 INFO:journalctl@ceph.osd.9.smithi012.stdout:Sep 11 00:33:59 smithi012 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-9[66733]: 2024-09-11T00:33:59.069+0000 7f5f4e800640 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-11T00:33:30.607371+0000 front 2024-09-11T00:33:30.607499+0000 (oldest deadline 2024-09-11T00:33:55.307466+0000) 2024-09-11T00:33:59.756 INFO:journalctl@ceph.osd.4.smithi165.stdout:Sep 11 00:33:59 smithi165 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-4[46433]: 2024-09-11T00:33:59.363+0000 7f8660426640 -1 osd.4 91 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-11T00:33:32.677220+0000 front 2024-09-11T00:33:32.677181+0000 (oldest deadline 2024-09-11T00:33:55.577404+0000) 2024-09-11T00:33:59.831 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 00:33:59 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-11[60852]: 2024-09-11T00:33:59.537+0000 7fd754b0b640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-09-11T00:33:24.081060+0000 front 2024-09-11T00:33:24.081080+0000 (oldest deadline 2024-09-11T00:33:49.980779+0000) 2024-09-11T00:34:00.033 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 11 00:33:59 smithi165 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-10[59927]: 2024-09-11T00:33:59.754+0000 7f2cff41d640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-09-11T00:33:25.146772+0000 front 2024-09-11T00:33:25.146620+0000 (oldest deadline 2024-09-11T00:33:49.846529+0000) 2024-09-11T00:34:00.033 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 11 00:33:59 smithi165 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-10[59927]: 2024-09-11T00:33:59.754+0000 7f2cff41d640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-11T00:33:32.747514+0000 front 2024-09-11T00:33:32.747491+0000 (oldest deadline 2024-09-11T00:33:58.047686+0000) 2024-09-11T00:34:00.054 INFO:journalctl@ceph.osd.6.smithi012.stdout:Sep 11 00:33:59 smithi012 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-6[59757]: 2024-09-11T00:33:59.767+0000 7f2e71000640 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-11T00:33:34.716146+0000 front 2024-09-11T00:33:34.715917+0000 (oldest deadline 2024-09-11T00:33:57.015981+0000) 2024-09-11T00:34:00.313 INFO:journalctl@ceph.osd.9.smithi012.stdout:Sep 11 00:34:00 smithi012 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-9[66733]: 2024-09-11T00:34:00.053+0000 7f5f4e800640 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-09-11T00:33:25.406156+0000 front 2024-09-11T00:33:25.406095+0000 (oldest deadline 2024-09-11T00:33:47.706239+0000) 2024-09-11T00:34:00.313 INFO:journalctl@ceph.osd.9.smithi012.stdout:Sep 11 00:34:00 smithi012 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-9[66733]: 2024-09-11T00:34:00.053+0000 7f5f4e800640 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-11T00:33:30.607371+0000 front 2024-09-11T00:33:30.607499+0000 (oldest deadline 2024-09-11T00:33:55.307466+0000) 2024-09-11T00:34:00.439 INFO:journalctl@ceph.osd.7.smithi165.stdout:Sep 11 00:34:00 smithi165 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-7[52941]: 2024-09-11T00:34:00.093+0000 7ffbe0ef8640 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-11T00:33:32.707987+0000 front 2024-09-11T00:33:32.708022+0000 (oldest deadline 2024-09-11T00:33:56.808116+0000) 2024-09-11T00:34:00.519 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 11 00:34:00 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-8[54042]: 2024-09-11T00:34:00.182+0000 7f23083d6640 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-11T00:33:33.673118+0000 front 2024-09-11T00:33:33.673042+0000 (oldest deadline 2024-09-11T00:33:56.573057+0000) 2024-09-11T00:34:00.519 INFO:journalctl@ceph.osd.5.smithi178.stdout:Sep 11 00:34:00 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-5[47367]: 2024-09-11T00:34:00.321+0000 7fc6e39ea640 -1 osd.5 91 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-09-11T00:33:25.841798+0000 front 2024-09-11T00:33:25.841738+0000 (oldest deadline 2024-09-11T00:33:49.941528+0000) 2024-09-11T00:34:00.519 INFO:journalctl@ceph.osd.5.smithi178.stdout:Sep 11 00:34:00 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-5[47367]: 2024-09-11T00:34:00.321+0000 7fc6e39ea640 -1 osd.5 91 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-11T00:33:34.042676+0000 front 2024-09-11T00:33:34.042628+0000 (oldest deadline 2024-09-11T00:33:59.342829+0000) 2024-09-11T00:34:00.695 INFO:journalctl@ceph.osd.4.smithi165.stdout:Sep 11 00:34:00 smithi165 podman[76134]: 2024-09-11 00:34:00.438542352 +0000 UTC m=+5.719845736 container died 63c9846d2d83a18e22e3e520910c0cb797f18838059326f976756d022facf826 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-4, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-09-11T00:34:00.831 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 00:34:00 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-11[60852]: 2024-09-11T00:34:00.518+0000 7fd754b0b640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-09-11T00:33:24.081060+0000 front 2024-09-11T00:33:24.081080+0000 (oldest deadline 2024-09-11T00:33:49.980779+0000) 2024-09-11T00:34:01.033 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 11 00:34:00 smithi165 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-10[59927]: 2024-09-11T00:34:00.768+0000 7f2cff41d640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-09-11T00:33:25.146772+0000 front 2024-09-11T00:33:25.146620+0000 (oldest deadline 2024-09-11T00:33:49.846529+0000) 2024-09-11T00:34:01.033 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 11 00:34:00 smithi165 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-10[59927]: 2024-09-11T00:34:00.768+0000 7f2cff41d640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-11T00:33:32.747514+0000 front 2024-09-11T00:33:32.747491+0000 (oldest deadline 2024-09-11T00:33:58.047686+0000) 2024-09-11T00:34:01.033 INFO:journalctl@ceph.osd.4.smithi165.stdout:Sep 11 00:34:00 smithi165 podman[76134]: 2024-09-11 00:34:00.792512045 +0000 UTC m=+6.073815428 container cleanup 63c9846d2d83a18e22e3e520910c0cb797f18838059326f976756d022facf826 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-4, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.37.2, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-11T00:34:01.033 INFO:journalctl@ceph.osd.4.smithi165.stdout:Sep 11 00:34:00 smithi165 bash[76134]: ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-4 2024-09-11T00:34:01.063 INFO:journalctl@ceph.osd.6.smithi012.stdout:Sep 11 00:34:00 smithi012 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-6[59757]: 2024-09-11T00:34:00.749+0000 7f2e71000640 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-11T00:33:34.716146+0000 front 2024-09-11T00:33:34.715917+0000 (oldest deadline 2024-09-11T00:33:57.015981+0000) 2024-09-11T00:34:01.473 INFO:journalctl@ceph.osd.7.smithi165.stdout:Sep 11 00:34:01 smithi165 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-7[52941]: 2024-09-11T00:34:01.138+0000 7ffbe0ef8640 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-11T00:33:32.707987+0000 front 2024-09-11T00:33:32.708022+0000 (oldest deadline 2024-09-11T00:33:56.808116+0000) 2024-09-11T00:34:01.473 INFO:journalctl@ceph.osd.4.smithi165.stdout:Sep 11 00:34:01 smithi165 podman[76161]: 2024-09-11 00:34:01.119399443 +0000 UTC m=+0.681960492 container remove 63c9846d2d83a18e22e3e520910c0cb797f18838059326f976756d022facf826 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-4, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-11T00:34:01.518 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 11 00:34:01 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-8[54042]: 2024-09-11T00:34:01.200+0000 7f23083d6640 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-11T00:33:33.673118+0000 front 2024-09-11T00:33:33.673042+0000 (oldest deadline 2024-09-11T00:33:56.573057+0000) 2024-09-11T00:34:01.518 INFO:journalctl@ceph.osd.5.smithi178.stdout:Sep 11 00:34:01 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-5[47367]: 2024-09-11T00:34:01.289+0000 7fc6e39ea640 -1 osd.5 91 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-09-11T00:33:25.841798+0000 front 2024-09-11T00:33:25.841738+0000 (oldest deadline 2024-09-11T00:33:49.941528+0000) 2024-09-11T00:34:01.519 INFO:journalctl@ceph.osd.5.smithi178.stdout:Sep 11 00:34:01 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-5[47367]: 2024-09-11T00:34:01.289+0000 7fc6e39ea640 -1 osd.5 91 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-11T00:33:34.042676+0000 front 2024-09-11T00:33:34.042628+0000 (oldest deadline 2024-09-11T00:33:59.342829+0000) 2024-09-11T00:34:01.563 INFO:journalctl@ceph.osd.9.smithi012.stdout:Sep 11 00:34:01 smithi012 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-9[66733]: 2024-09-11T00:34:01.074+0000 7f5f4e800640 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-09-11T00:33:25.406156+0000 front 2024-09-11T00:33:25.406095+0000 (oldest deadline 2024-09-11T00:33:47.706239+0000) 2024-09-11T00:34:01.563 INFO:journalctl@ceph.osd.9.smithi012.stdout:Sep 11 00:34:01 smithi012 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-9[66733]: 2024-09-11T00:34:01.074+0000 7f5f4e800640 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-11T00:33:30.607371+0000 front 2024-09-11T00:33:30.607499+0000 (oldest deadline 2024-09-11T00:33:55.307466+0000) 2024-09-11T00:34:01.749 INFO:journalctl@ceph.osd.4.smithi165.stdout:Sep 11 00:34:01 smithi165 podman[76250]: 2024-09-11 00:34:01.374005365 +0000 UTC m=+0.015837918 image pull 589dd18248e494ba411dc838aa4ae0e5e569213f08fcd7a2cf22cd3ef7e3c480 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e 2024-09-11T00:34:01.749 INFO:journalctl@ceph.osd.4.smithi165.stdout:Sep 11 00:34:01 smithi165 podman[76250]: 2024-09-11 00:34:01.486912232 +0000 UTC m=+0.128744770 container create bc4b15fad210bceb49550774c2a0e5ae5b23f68e7afd69e2bcb044bb27c5c484 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-4-deactivate, org.label-schema.build-date=20240731, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-09-11T00:34:01.831 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 00:34:01 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-11[60852]: 2024-09-11T00:34:01.517+0000 7fd754b0b640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-09-11T00:33:24.081060+0000 front 2024-09-11T00:33:24.081080+0000 (oldest deadline 2024-09-11T00:33:49.980779+0000) 2024-09-11T00:34:02.033 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 11 00:34:01 smithi165 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-10[59927]: 2024-09-11T00:34:01.801+0000 7f2cff41d640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-09-11T00:33:25.146772+0000 front 2024-09-11T00:33:25.146620+0000 (oldest deadline 2024-09-11T00:33:49.846529+0000) 2024-09-11T00:34:02.033 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 11 00:34:01 smithi165 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-10[59927]: 2024-09-11T00:34:01.801+0000 7f2cff41d640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-11T00:33:32.747514+0000 front 2024-09-11T00:33:32.747491+0000 (oldest deadline 2024-09-11T00:33:58.047686+0000) 2024-09-11T00:34:02.033 INFO:journalctl@ceph.osd.4.smithi165.stdout:Sep 11 00:34:01 smithi165 podman[76250]: 2024-09-11 00:34:01.748314453 +0000 UTC m=+0.390146992 container init bc4b15fad210bceb49550774c2a0e5ae5b23f68e7afd69e2bcb044bb27c5c484 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-4-deactivate, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0) 2024-09-11T00:34:02.033 INFO:journalctl@ceph.osd.4.smithi165.stdout:Sep 11 00:34:01 smithi165 podman[76250]: 2024-09-11 00:34:01.751571768 +0000 UTC m=+0.393404307 container start bc4b15fad210bceb49550774c2a0e5ae5b23f68e7afd69e2bcb044bb27c5c484 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-4-deactivate, GIT_CLEAN=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-09-11T00:34:02.033 INFO:journalctl@ceph.osd.4.smithi165.stdout:Sep 11 00:34:01 smithi165 podman[76250]: 2024-09-11 00:34:01.873536264 +0000 UTC m=+0.515368811 container attach bc4b15fad210bceb49550774c2a0e5ae5b23f68e7afd69e2bcb044bb27c5c484 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-4-deactivate, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True) 2024-09-11T00:34:02.033 INFO:journalctl@ceph.osd.4.smithi165.stdout:Sep 11 00:34:01 smithi165 podman[76250]: 2024-09-11 00:34:01.941208527 +0000 UTC m=+0.583041073 container died bc4b15fad210bceb49550774c2a0e5ae5b23f68e7afd69e2bcb044bb27c5c484 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-4-deactivate, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-09-11T00:34:02.064 INFO:journalctl@ceph.osd.6.smithi012.stdout:Sep 11 00:34:01 smithi012 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-6[59757]: 2024-09-11T00:34:01.731+0000 7f2e71000640 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-11T00:33:34.716146+0000 front 2024-09-11T00:33:34.715917+0000 (oldest deadline 2024-09-11T00:33:57.015981+0000) 2024-09-11T00:34:02.533 INFO:journalctl@ceph.osd.7.smithi165.stdout:Sep 11 00:34:02 smithi165 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-7[52941]: 2024-09-11T00:34:02.123+0000 7ffbe0ef8640 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-11T00:33:32.707987+0000 front 2024-09-11T00:33:32.708022+0000 (oldest deadline 2024-09-11T00:33:56.808116+0000) 2024-09-11T00:34:02.539 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 11 00:34:02 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-8[54042]: 2024-09-11T00:34:02.238+0000 7f23083d6640 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-11T00:33:33.673118+0000 front 2024-09-11T00:33:33.673042+0000 (oldest deadline 2024-09-11T00:33:56.573057+0000) 2024-09-11T00:34:02.539 INFO:journalctl@ceph.osd.5.smithi178.stdout:Sep 11 00:34:02 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-5[47367]: 2024-09-11T00:34:02.257+0000 7fc6e39ea640 -1 osd.5 91 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-09-11T00:33:25.841798+0000 front 2024-09-11T00:33:25.841738+0000 (oldest deadline 2024-09-11T00:33:49.941528+0000) 2024-09-11T00:34:02.539 INFO:journalctl@ceph.osd.5.smithi178.stdout:Sep 11 00:34:02 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-5[47367]: 2024-09-11T00:34:02.257+0000 7fc6e39ea640 -1 osd.5 91 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-11T00:33:34.042676+0000 front 2024-09-11T00:33:34.042628+0000 (oldest deadline 2024-09-11T00:33:59.342829+0000) 2024-09-11T00:34:02.554 DEBUG:teuthology.orchestra.run.smithi165:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968@osd.4.service' 2024-09-11T00:34:02.563 INFO:journalctl@ceph.osd.9.smithi012.stdout:Sep 11 00:34:02 smithi012 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-9[66733]: 2024-09-11T00:34:02.116+0000 7f5f4e800640 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-09-11T00:33:25.406156+0000 front 2024-09-11T00:33:25.406095+0000 (oldest deadline 2024-09-11T00:33:47.706239+0000) 2024-09-11T00:34:02.563 INFO:journalctl@ceph.osd.9.smithi012.stdout:Sep 11 00:34:02 smithi012 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-9[66733]: 2024-09-11T00:34:02.116+0000 7f5f4e800640 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-11T00:33:30.607371+0000 front 2024-09-11T00:33:30.607499+0000 (oldest deadline 2024-09-11T00:33:55.307466+0000) 2024-09-11T00:34:02.594 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T00:34:02.594 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-09-11T00:34:02.594 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-09-11T00:34:02.594 DEBUG:teuthology.orchestra.run.smithi178:> sudo systemctl stop ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968@osd.5 2024-09-11T00:34:02.831 INFO:journalctl@ceph.osd.5.smithi178.stdout:Sep 11 00:34:02 smithi178 systemd[1]: Stopping Ceph osd.5 for ff3ad74a-6fcf-11ef-bcea-c7b262605968... 2024-09-11T00:34:02.831 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 00:34:02 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-11[60852]: 2024-09-11T00:34:02.537+0000 7fd754b0b640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-09-11T00:33:24.081060+0000 front 2024-09-11T00:33:24.081080+0000 (oldest deadline 2024-09-11T00:33:49.980779+0000) 2024-09-11T00:34:03.063 INFO:journalctl@ceph.osd.6.smithi012.stdout:Sep 11 00:34:02 smithi012 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-6[59757]: 2024-09-11T00:34:02.699+0000 7f2e71000640 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-11T00:33:34.716146+0000 front 2024-09-11T00:33:34.715917+0000 (oldest deadline 2024-09-11T00:33:57.015981+0000) 2024-09-11T00:34:03.105 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 11 00:34:02 smithi165 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-10[59927]: 2024-09-11T00:34:02.843+0000 7f2cff41d640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-09-11T00:33:25.146772+0000 front 2024-09-11T00:33:25.146620+0000 (oldest deadline 2024-09-11T00:33:49.846529+0000) 2024-09-11T00:34:03.105 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 11 00:34:02 smithi165 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-10[59927]: 2024-09-11T00:34:02.843+0000 7f2cff41d640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-11T00:33:32.747514+0000 front 2024-09-11T00:33:32.747491+0000 (oldest deadline 2024-09-11T00:33:58.047686+0000) 2024-09-11T00:34:03.290 INFO:journalctl@ceph.osd.5.smithi178.stdout:Sep 11 00:34:02 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-5[47367]: 2024-09-11T00:34:02.936+0000 7fc6e73d5640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T00:34:03.290 INFO:journalctl@ceph.osd.5.smithi178.stdout:Sep 11 00:34:02 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-5[47367]: 2024-09-11T00:34:02.936+0000 7fc6e73d5640 -1 osd.5 91 *** Got signal Terminated *** 2024-09-11T00:34:03.290 INFO:journalctl@ceph.osd.5.smithi178.stdout:Sep 11 00:34:02 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-5[47367]: 2024-09-11T00:34:02.936+0000 7fc6e73d5640 -1 osd.5 91 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-11T00:34:03.533 INFO:journalctl@ceph.osd.7.smithi165.stdout:Sep 11 00:34:03 smithi165 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-7[52941]: 2024-09-11T00:34:03.104+0000 7ffbe0ef8640 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-11T00:33:32.707987+0000 front 2024-09-11T00:33:32.708022+0000 (oldest deadline 2024-09-11T00:33:56.808116+0000) 2024-09-11T00:34:03.563 INFO:journalctl@ceph.osd.9.smithi012.stdout:Sep 11 00:34:03 smithi012 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-9[66733]: 2024-09-11T00:34:03.090+0000 7f5f4e800640 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-09-11T00:33:25.406156+0000 front 2024-09-11T00:33:25.406095+0000 (oldest deadline 2024-09-11T00:33:47.706239+0000) 2024-09-11T00:34:03.563 INFO:journalctl@ceph.osd.9.smithi012.stdout:Sep 11 00:34:03 smithi012 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-9[66733]: 2024-09-11T00:34:03.090+0000 7f5f4e800640 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-11T00:33:30.607371+0000 front 2024-09-11T00:33:30.607499+0000 (oldest deadline 2024-09-11T00:33:55.307466+0000) 2024-09-11T00:34:03.581 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 11 00:34:03 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-8[54042]: 2024-09-11T00:34:03.288+0000 7f23083d6640 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-11T00:33:33.673118+0000 front 2024-09-11T00:33:33.673042+0000 (oldest deadline 2024-09-11T00:33:56.573057+0000) 2024-09-11T00:34:03.581 INFO:journalctl@ceph.osd.5.smithi178.stdout:Sep 11 00:34:03 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-5[47367]: 2024-09-11T00:34:03.300+0000 7fc6e39ea640 -1 osd.5 91 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-09-11T00:33:25.841798+0000 front 2024-09-11T00:33:25.841738+0000 (oldest deadline 2024-09-11T00:33:49.941528+0000) 2024-09-11T00:34:03.581 INFO:journalctl@ceph.osd.5.smithi178.stdout:Sep 11 00:34:03 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-5[47367]: 2024-09-11T00:34:03.300+0000 7fc6e39ea640 -1 osd.5 91 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-11T00:33:34.042676+0000 front 2024-09-11T00:33:34.042628+0000 (oldest deadline 2024-09-11T00:33:59.342829+0000) 2024-09-11T00:34:03.581 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 00:34:03 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-11[60852]: 2024-09-11T00:34:03.499+0000 7fd754b0b640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-09-11T00:33:24.081060+0000 front 2024-09-11T00:33:24.081080+0000 (oldest deadline 2024-09-11T00:33:49.980779+0000) 2024-09-11T00:34:04.063 INFO:journalctl@ceph.osd.6.smithi012.stdout:Sep 11 00:34:03 smithi012 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-6[59757]: 2024-09-11T00:34:03.674+0000 7f2e71000640 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-11T00:33:34.716146+0000 front 2024-09-11T00:33:34.715917+0000 (oldest deadline 2024-09-11T00:33:57.015981+0000) 2024-09-11T00:34:04.063 INFO:journalctl@ceph.osd.6.smithi012.stdout:Sep 11 00:34:03 smithi012 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-6[59757]: 2024-09-11T00:34:03.674+0000 7f2e71000640 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-11T00:33:38.717829+0000 front 2024-09-11T00:33:38.717632+0000 (oldest deadline 2024-09-11T00:34:03.417594+0000) 2024-09-11T00:34:04.283 INFO:journalctl@ceph.osd.7.smithi165.stdout:Sep 11 00:34:04 smithi165 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-7[52941]: 2024-09-11T00:34:04.093+0000 7ffbe0ef8640 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-11T00:33:32.707987+0000 front 2024-09-11T00:33:32.708022+0000 (oldest deadline 2024-09-11T00:33:56.808116+0000) 2024-09-11T00:34:04.283 INFO:journalctl@ceph.osd.7.smithi165.stdout:Sep 11 00:34:04 smithi165 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-7[52941]: 2024-09-11T00:34:04.093+0000 7ffbe0ef8640 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-11T00:33:41.509403+0000 front 2024-09-11T00:33:41.509317+0000 (oldest deadline 2024-09-11T00:34:03.809439+0000) 2024-09-11T00:34:04.283 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 11 00:34:03 smithi165 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-10[59927]: 2024-09-11T00:34:03.880+0000 7f2cff41d640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-09-11T00:33:25.146772+0000 front 2024-09-11T00:33:25.146620+0000 (oldest deadline 2024-09-11T00:33:49.846529+0000) 2024-09-11T00:34:04.283 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 11 00:34:03 smithi165 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-10[59927]: 2024-09-11T00:34:03.880+0000 7f2cff41d640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-11T00:33:32.747514+0000 front 2024-09-11T00:33:32.747491+0000 (oldest deadline 2024-09-11T00:33:58.047686+0000) 2024-09-11T00:34:04.563 INFO:journalctl@ceph.osd.9.smithi012.stdout:Sep 11 00:34:04 smithi012 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-9[66733]: 2024-09-11T00:34:04.081+0000 7f5f4e800640 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-09-11T00:33:25.406156+0000 front 2024-09-11T00:33:25.406095+0000 (oldest deadline 2024-09-11T00:33:47.706239+0000) 2024-09-11T00:34:04.564 INFO:journalctl@ceph.osd.9.smithi012.stdout:Sep 11 00:34:04 smithi012 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-9[66733]: 2024-09-11T00:34:04.081+0000 7f5f4e800640 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-11T00:33:30.607371+0000 front 2024-09-11T00:33:30.607499+0000 (oldest deadline 2024-09-11T00:33:55.307466+0000) 2024-09-11T00:34:04.581 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 11 00:34:04 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-8[54042]: 2024-09-11T00:34:04.255+0000 7f23083d6640 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-11T00:33:33.673118+0000 front 2024-09-11T00:33:33.673042+0000 (oldest deadline 2024-09-11T00:33:56.573057+0000) 2024-09-11T00:34:04.581 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 00:34:04 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-11[60852]: 2024-09-11T00:34:04.468+0000 7fd754b0b640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-09-11T00:33:24.081060+0000 front 2024-09-11T00:33:24.081080+0000 (oldest deadline 2024-09-11T00:33:49.980779+0000) 2024-09-11T00:34:04.582 INFO:journalctl@ceph.osd.5.smithi178.stdout:Sep 11 00:34:04 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-5[47367]: 2024-09-11T00:34:04.264+0000 7fc6e39ea640 -1 osd.5 91 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-09-11T00:33:25.841798+0000 front 2024-09-11T00:33:25.841738+0000 (oldest deadline 2024-09-11T00:33:49.941528+0000) 2024-09-11T00:34:04.582 INFO:journalctl@ceph.osd.5.smithi178.stdout:Sep 11 00:34:04 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-5[47367]: 2024-09-11T00:34:04.264+0000 7fc6e39ea640 -1 osd.5 91 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-11T00:33:34.042676+0000 front 2024-09-11T00:33:34.042628+0000 (oldest deadline 2024-09-11T00:33:59.342829+0000) 2024-09-11T00:34:05.043 INFO:journalctl@ceph.osd.6.smithi012.stdout:Sep 11 00:34:04 smithi012 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-6[59757]: 2024-09-11T00:34:04.633+0000 7f2e71000640 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-11T00:33:34.716146+0000 front 2024-09-11T00:33:34.715917+0000 (oldest deadline 2024-09-11T00:33:57.015981+0000) 2024-09-11T00:34:05.043 INFO:journalctl@ceph.osd.6.smithi012.stdout:Sep 11 00:34:04 smithi012 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-6[59757]: 2024-09-11T00:34:04.633+0000 7f2e71000640 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-11T00:33:38.717829+0000 front 2024-09-11T00:33:38.717632+0000 (oldest deadline 2024-09-11T00:34:03.417594+0000) 2024-09-11T00:34:05.282 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 11 00:34:04 smithi165 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-10[59927]: 2024-09-11T00:34:04.849+0000 7f2cff41d640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-09-11T00:33:25.146772+0000 front 2024-09-11T00:33:25.146620+0000 (oldest deadline 2024-09-11T00:33:49.846529+0000) 2024-09-11T00:34:05.283 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 11 00:34:04 smithi165 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-10[59927]: 2024-09-11T00:34:04.849+0000 7f2cff41d640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-11T00:33:32.747514+0000 front 2024-09-11T00:33:32.747491+0000 (oldest deadline 2024-09-11T00:33:58.047686+0000) 2024-09-11T00:34:05.283 INFO:journalctl@ceph.osd.7.smithi165.stdout:Sep 11 00:34:05 smithi165 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-7[52941]: 2024-09-11T00:34:05.047+0000 7ffbe0ef8640 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-11T00:33:32.707987+0000 front 2024-09-11T00:33:32.708022+0000 (oldest deadline 2024-09-11T00:33:56.808116+0000) 2024-09-11T00:34:05.283 INFO:journalctl@ceph.osd.7.smithi165.stdout:Sep 11 00:34:05 smithi165 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-7[52941]: 2024-09-11T00:34:05.047+0000 7ffbe0ef8640 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-11T00:33:41.509403+0000 front 2024-09-11T00:33:41.509317+0000 (oldest deadline 2024-09-11T00:34:03.809439+0000) 2024-09-11T00:34:05.313 INFO:journalctl@ceph.osd.9.smithi012.stdout:Sep 11 00:34:05 smithi012 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-9[66733]: 2024-09-11T00:34:05.041+0000 7f5f4e800640 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-09-11T00:33:25.406156+0000 front 2024-09-11T00:33:25.406095+0000 (oldest deadline 2024-09-11T00:33:47.706239+0000) 2024-09-11T00:34:05.313 INFO:journalctl@ceph.osd.9.smithi012.stdout:Sep 11 00:34:05 smithi012 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-9[66733]: 2024-09-11T00:34:05.041+0000 7f5f4e800640 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-11T00:33:30.607371+0000 front 2024-09-11T00:33:30.607499+0000 (oldest deadline 2024-09-11T00:33:55.307466+0000) 2024-09-11T00:34:05.484 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 11 00:34:05 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-8[54042]: 2024-09-11T00:34:05.216+0000 7f23083d6640 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-11T00:33:33.673118+0000 front 2024-09-11T00:33:33.673042+0000 (oldest deadline 2024-09-11T00:33:56.573057+0000) 2024-09-11T00:34:05.484 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 11 00:34:05 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-8[54042]: 2024-09-11T00:34:05.216+0000 7f23083d6640 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-11T00:33:38.774830+0000 front 2024-09-11T00:33:38.774774+0000 (oldest deadline 2024-09-11T00:34:04.674919+0000) 2024-09-11T00:34:05.484 INFO:journalctl@ceph.osd.5.smithi178.stdout:Sep 11 00:34:05 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-5[47367]: 2024-09-11T00:34:05.263+0000 7fc6e39ea640 -1 osd.5 91 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-09-11T00:33:25.841798+0000 front 2024-09-11T00:33:25.841738+0000 (oldest deadline 2024-09-11T00:33:49.941528+0000) 2024-09-11T00:34:05.484 INFO:journalctl@ceph.osd.5.smithi178.stdout:Sep 11 00:34:05 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-5[47367]: 2024-09-11T00:34:05.263+0000 7fc6e39ea640 -1 osd.5 91 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-11T00:33:34.042676+0000 front 2024-09-11T00:33:34.042628+0000 (oldest deadline 2024-09-11T00:33:59.342829+0000) 2024-09-11T00:34:05.484 INFO:journalctl@ceph.osd.5.smithi178.stdout:Sep 11 00:34:05 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-5[47367]: 2024-09-11T00:34:05.263+0000 7fc6e39ea640 -1 osd.5 91 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-11T00:33:39.343270+0000 front 2024-09-11T00:33:39.343291+0000 (oldest deadline 2024-09-11T00:34:05.243408+0000) 2024-09-11T00:34:05.831 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 00:34:05 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-11[60852]: 2024-09-11T00:34:05.482+0000 7fd754b0b640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-09-11T00:33:24.081060+0000 front 2024-09-11T00:33:24.081080+0000 (oldest deadline 2024-09-11T00:33:49.980779+0000) 2024-09-11T00:34:05.831 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 00:34:05 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-11[60852]: 2024-09-11T00:34:05.482+0000 7fd754b0b640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-11T00:33:41.682818+0000 front 2024-09-11T00:33:41.682848+0000 (oldest deadline 2024-09-11T00:34:05.182914+0000) 2024-09-11T00:34:06.063 INFO:journalctl@ceph.osd.6.smithi012.stdout:Sep 11 00:34:05 smithi012 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-6[59757]: 2024-09-11T00:34:05.643+0000 7f2e71000640 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-11T00:33:34.716146+0000 front 2024-09-11T00:33:34.715917+0000 (oldest deadline 2024-09-11T00:33:57.015981+0000) 2024-09-11T00:34:06.063 INFO:journalctl@ceph.osd.6.smithi012.stdout:Sep 11 00:34:05 smithi012 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-6[59757]: 2024-09-11T00:34:05.643+0000 7f2e71000640 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-11T00:33:38.717829+0000 front 2024-09-11T00:33:38.717632+0000 (oldest deadline 2024-09-11T00:34:03.417594+0000) 2024-09-11T00:34:06.283 INFO:journalctl@ceph.osd.7.smithi165.stdout:Sep 11 00:34:06 smithi165 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-7[52941]: 2024-09-11T00:34:06.019+0000 7ffbe0ef8640 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-11T00:33:32.707987+0000 front 2024-09-11T00:33:32.708022+0000 (oldest deadline 2024-09-11T00:33:56.808116+0000) 2024-09-11T00:34:06.283 INFO:journalctl@ceph.osd.7.smithi165.stdout:Sep 11 00:34:06 smithi165 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-7[52941]: 2024-09-11T00:34:06.019+0000 7ffbe0ef8640 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-11T00:33:41.509403+0000 front 2024-09-11T00:33:41.509317+0000 (oldest deadline 2024-09-11T00:34:03.809439+0000) 2024-09-11T00:34:06.283 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 11 00:34:05 smithi165 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-10[59927]: 2024-09-11T00:34:05.846+0000 7f2cff41d640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-09-11T00:33:25.146772+0000 front 2024-09-11T00:33:25.146620+0000 (oldest deadline 2024-09-11T00:33:49.846529+0000) 2024-09-11T00:34:06.283 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 11 00:34:05 smithi165 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-10[59927]: 2024-09-11T00:34:05.846+0000 7f2cff41d640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-11T00:33:32.747514+0000 front 2024-09-11T00:33:32.747491+0000 (oldest deadline 2024-09-11T00:33:58.047686+0000) 2024-09-11T00:34:06.283 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 11 00:34:05 smithi165 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-10[59927]: 2024-09-11T00:34:05.846+0000 7f2cff41d640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-11T00:33:41.548749+0000 front 2024-09-11T00:33:41.548701+0000 (oldest deadline 2024-09-11T00:34:05.648724+0000) 2024-09-11T00:34:06.563 INFO:journalctl@ceph.osd.9.smithi012.stdout:Sep 11 00:34:06 smithi012 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-9[66733]: 2024-09-11T00:34:06.063+0000 7f5f4e800640 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-09-11T00:33:25.406156+0000 front 2024-09-11T00:33:25.406095+0000 (oldest deadline 2024-09-11T00:33:47.706239+0000) 2024-09-11T00:34:06.563 INFO:journalctl@ceph.osd.9.smithi012.stdout:Sep 11 00:34:06 smithi012 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-9[66733]: 2024-09-11T00:34:06.063+0000 7f5f4e800640 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-11T00:33:30.607371+0000 front 2024-09-11T00:33:30.607499+0000 (oldest deadline 2024-09-11T00:33:55.307466+0000) 2024-09-11T00:34:06.581 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 11 00:34:06 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-8[54042]: 2024-09-11T00:34:06.190+0000 7f23083d6640 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-11T00:33:33.673118+0000 front 2024-09-11T00:33:33.673042+0000 (oldest deadline 2024-09-11T00:33:56.573057+0000) 2024-09-11T00:34:06.581 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 11 00:34:06 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-8[54042]: 2024-09-11T00:34:06.190+0000 7f23083d6640 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-11T00:33:38.774830+0000 front 2024-09-11T00:33:38.774774+0000 (oldest deadline 2024-09-11T00:34:04.674919+0000) 2024-09-11T00:34:06.581 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 00:34:06 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-11[60852]: 2024-09-11T00:34:06.440+0000 7fd754b0b640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-09-11T00:33:24.081060+0000 front 2024-09-11T00:33:24.081080+0000 (oldest deadline 2024-09-11T00:33:49.980779+0000) 2024-09-11T00:34:06.582 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 00:34:06 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-11[60852]: 2024-09-11T00:34:06.440+0000 7fd754b0b640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-11T00:33:41.682818+0000 front 2024-09-11T00:33:41.682848+0000 (oldest deadline 2024-09-11T00:34:05.182914+0000) 2024-09-11T00:34:06.582 INFO:journalctl@ceph.osd.5.smithi178.stdout:Sep 11 00:34:06 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-5[47367]: 2024-09-11T00:34:06.220+0000 7fc6e39ea640 -1 osd.5 91 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-09-11T00:33:25.841798+0000 front 2024-09-11T00:33:25.841738+0000 (oldest deadline 2024-09-11T00:33:49.941528+0000) 2024-09-11T00:34:06.582 INFO:journalctl@ceph.osd.5.smithi178.stdout:Sep 11 00:34:06 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-5[47367]: 2024-09-11T00:34:06.220+0000 7fc6e39ea640 -1 osd.5 91 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-11T00:33:34.042676+0000 front 2024-09-11T00:33:34.042628+0000 (oldest deadline 2024-09-11T00:33:59.342829+0000) 2024-09-11T00:34:06.582 INFO:journalctl@ceph.osd.5.smithi178.stdout:Sep 11 00:34:06 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-5[47367]: 2024-09-11T00:34:06.220+0000 7fc6e39ea640 -1 osd.5 91 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-11T00:33:39.343270+0000 front 2024-09-11T00:33:39.343291+0000 (oldest deadline 2024-09-11T00:34:05.243408+0000) 2024-09-11T00:34:07.031 INFO:journalctl@ceph.osd.6.smithi012.stdout:Sep 11 00:34:06 smithi012 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-6[59757]: 2024-09-11T00:34:06.685+0000 7f2e71000640 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-11T00:33:34.716146+0000 front 2024-09-11T00:33:34.715917+0000 (oldest deadline 2024-09-11T00:33:57.015981+0000) 2024-09-11T00:34:07.031 INFO:journalctl@ceph.osd.6.smithi012.stdout:Sep 11 00:34:06 smithi012 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-6[59757]: 2024-09-11T00:34:06.685+0000 7f2e71000640 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-11T00:33:38.717829+0000 front 2024-09-11T00:33:38.717632+0000 (oldest deadline 2024-09-11T00:34:03.417594+0000) 2024-09-11T00:34:07.283 INFO:journalctl@ceph.osd.7.smithi165.stdout:Sep 11 00:34:06 smithi165 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-7[52941]: 2024-09-11T00:34:06.983+0000 7ffbe0ef8640 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-11T00:33:32.707987+0000 front 2024-09-11T00:33:32.708022+0000 (oldest deadline 2024-09-11T00:33:56.808116+0000) 2024-09-11T00:34:07.283 INFO:journalctl@ceph.osd.7.smithi165.stdout:Sep 11 00:34:06 smithi165 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-7[52941]: 2024-09-11T00:34:06.983+0000 7ffbe0ef8640 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-11T00:33:41.509403+0000 front 2024-09-11T00:33:41.509317+0000 (oldest deadline 2024-09-11T00:34:03.809439+0000) 2024-09-11T00:34:07.283 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 11 00:34:06 smithi165 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-10[59927]: 2024-09-11T00:34:06.885+0000 7f2cff41d640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-09-11T00:33:25.146772+0000 front 2024-09-11T00:33:25.146620+0000 (oldest deadline 2024-09-11T00:33:49.846529+0000) 2024-09-11T00:34:07.283 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 11 00:34:06 smithi165 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-10[59927]: 2024-09-11T00:34:06.885+0000 7f2cff41d640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-11T00:33:32.747514+0000 front 2024-09-11T00:33:32.747491+0000 (oldest deadline 2024-09-11T00:33:58.047686+0000) 2024-09-11T00:34:07.283 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 11 00:34:06 smithi165 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-10[59927]: 2024-09-11T00:34:06.885+0000 7f2cff41d640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-11T00:33:41.548749+0000 front 2024-09-11T00:33:41.548701+0000 (oldest deadline 2024-09-11T00:34:05.648724+0000) 2024-09-11T00:34:07.313 INFO:journalctl@ceph.osd.9.smithi012.stdout:Sep 11 00:34:07 smithi012 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-9[66733]: 2024-09-11T00:34:07.029+0000 7f5f4e800640 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-09-11T00:33:25.406156+0000 front 2024-09-11T00:33:25.406095+0000 (oldest deadline 2024-09-11T00:33:47.706239+0000) 2024-09-11T00:34:07.313 INFO:journalctl@ceph.osd.9.smithi012.stdout:Sep 11 00:34:07 smithi012 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-9[66733]: 2024-09-11T00:34:07.029+0000 7f5f4e800640 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-11T00:33:30.607371+0000 front 2024-09-11T00:33:30.607499+0000 (oldest deadline 2024-09-11T00:33:55.307466+0000) 2024-09-11T00:34:07.313 INFO:journalctl@ceph.osd.9.smithi012.stdout:Sep 11 00:34:07 smithi012 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-9[66733]: 2024-09-11T00:34:07.029+0000 7f5f4e800640 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-11T00:33:41.710055+0000 front 2024-09-11T00:33:41.709869+0000 (oldest deadline 2024-09-11T00:34:07.010035+0000) 2024-09-11T00:34:07.581 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 11 00:34:07 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-8[54042]: 2024-09-11T00:34:07.177+0000 7f23083d6640 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-11T00:33:33.673118+0000 front 2024-09-11T00:33:33.673042+0000 (oldest deadline 2024-09-11T00:33:56.573057+0000) 2024-09-11T00:34:07.581 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 11 00:34:07 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-8[54042]: 2024-09-11T00:34:07.177+0000 7f23083d6640 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-11T00:33:38.774830+0000 front 2024-09-11T00:33:38.774774+0000 (oldest deadline 2024-09-11T00:34:04.674919+0000) 2024-09-11T00:34:07.582 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 00:34:07 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-11[60852]: 2024-09-11T00:34:07.414+0000 7fd754b0b640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-09-11T00:33:24.081060+0000 front 2024-09-11T00:33:24.081080+0000 (oldest deadline 2024-09-11T00:33:49.980779+0000) 2024-09-11T00:34:07.582 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 00:34:07 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-11[60852]: 2024-09-11T00:34:07.414+0000 7fd754b0b640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-11T00:33:41.682818+0000 front 2024-09-11T00:33:41.682848+0000 (oldest deadline 2024-09-11T00:34:05.182914+0000) 2024-09-11T00:34:07.582 INFO:journalctl@ceph.osd.5.smithi178.stdout:Sep 11 00:34:07 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-5[47367]: 2024-09-11T00:34:07.233+0000 7fc6e39ea640 -1 osd.5 91 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-09-11T00:33:25.841798+0000 front 2024-09-11T00:33:25.841738+0000 (oldest deadline 2024-09-11T00:33:49.941528+0000) 2024-09-11T00:34:07.582 INFO:journalctl@ceph.osd.5.smithi178.stdout:Sep 11 00:34:07 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-5[47367]: 2024-09-11T00:34:07.233+0000 7fc6e39ea640 -1 osd.5 91 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-11T00:33:34.042676+0000 front 2024-09-11T00:33:34.042628+0000 (oldest deadline 2024-09-11T00:33:59.342829+0000) 2024-09-11T00:34:07.582 INFO:journalctl@ceph.osd.5.smithi178.stdout:Sep 11 00:34:07 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-5[47367]: 2024-09-11T00:34:07.233+0000 7fc6e39ea640 -1 osd.5 91 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-11T00:33:39.343270+0000 front 2024-09-11T00:33:39.343291+0000 (oldest deadline 2024-09-11T00:34:05.243408+0000) 2024-09-11T00:34:08.047 INFO:journalctl@ceph.osd.6.smithi012.stdout:Sep 11 00:34:07 smithi012 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-6[59757]: 2024-09-11T00:34:07.654+0000 7f2e71000640 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-11T00:33:34.716146+0000 front 2024-09-11T00:33:34.715917+0000 (oldest deadline 2024-09-11T00:33:57.015981+0000) 2024-09-11T00:34:08.048 INFO:journalctl@ceph.osd.6.smithi012.stdout:Sep 11 00:34:07 smithi012 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-6[59757]: 2024-09-11T00:34:07.654+0000 7f2e71000640 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-11T00:33:38.717829+0000 front 2024-09-11T00:33:38.717632+0000 (oldest deadline 2024-09-11T00:34:03.417594+0000) 2024-09-11T00:34:08.283 INFO:journalctl@ceph.osd.7.smithi165.stdout:Sep 11 00:34:07 smithi165 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-7[52941]: 2024-09-11T00:34:07.955+0000 7ffbe0ef8640 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-11T00:33:32.707987+0000 front 2024-09-11T00:33:32.708022+0000 (oldest deadline 2024-09-11T00:33:56.808116+0000) 2024-09-11T00:34:08.283 INFO:journalctl@ceph.osd.7.smithi165.stdout:Sep 11 00:34:07 smithi165 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-7[52941]: 2024-09-11T00:34:07.955+0000 7ffbe0ef8640 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-11T00:33:41.509403+0000 front 2024-09-11T00:33:41.509317+0000 (oldest deadline 2024-09-11T00:34:03.809439+0000) 2024-09-11T00:34:08.283 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 11 00:34:07 smithi165 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-10[59927]: 2024-09-11T00:34:07.896+0000 7f2cff41d640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-09-11T00:33:25.146772+0000 front 2024-09-11T00:33:25.146620+0000 (oldest deadline 2024-09-11T00:33:49.846529+0000) 2024-09-11T00:34:08.283 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 11 00:34:07 smithi165 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-10[59927]: 2024-09-11T00:34:07.896+0000 7f2cff41d640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-11T00:33:32.747514+0000 front 2024-09-11T00:33:32.747491+0000 (oldest deadline 2024-09-11T00:33:58.047686+0000) 2024-09-11T00:34:08.283 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 11 00:34:07 smithi165 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-10[59927]: 2024-09-11T00:34:07.896+0000 7f2cff41d640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-11T00:33:41.548749+0000 front 2024-09-11T00:33:41.548701+0000 (oldest deadline 2024-09-11T00:34:05.648724+0000) 2024-09-11T00:34:08.313 INFO:journalctl@ceph.osd.9.smithi012.stdout:Sep 11 00:34:08 smithi012 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-9[66733]: 2024-09-11T00:34:08.046+0000 7f5f4e800640 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-09-11T00:33:25.406156+0000 front 2024-09-11T00:33:25.406095+0000 (oldest deadline 2024-09-11T00:33:47.706239+0000) 2024-09-11T00:34:08.313 INFO:journalctl@ceph.osd.9.smithi012.stdout:Sep 11 00:34:08 smithi012 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-9[66733]: 2024-09-11T00:34:08.046+0000 7f5f4e800640 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-11T00:33:30.607371+0000 front 2024-09-11T00:33:30.607499+0000 (oldest deadline 2024-09-11T00:33:55.307466+0000) 2024-09-11T00:34:08.313 INFO:journalctl@ceph.osd.9.smithi012.stdout:Sep 11 00:34:08 smithi012 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-9[66733]: 2024-09-11T00:34:08.046+0000 7f5f4e800640 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-11T00:33:41.710055+0000 front 2024-09-11T00:33:41.709869+0000 (oldest deadline 2024-09-11T00:34:07.010035+0000) 2024-09-11T00:34:08.581 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 11 00:34:08 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-8[54042]: 2024-09-11T00:34:08.227+0000 7f23083d6640 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-11T00:33:33.673118+0000 front 2024-09-11T00:33:33.673042+0000 (oldest deadline 2024-09-11T00:33:56.573057+0000) 2024-09-11T00:34:08.581 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 11 00:34:08 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-8[54042]: 2024-09-11T00:34:08.227+0000 7f23083d6640 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-11T00:33:38.774830+0000 front 2024-09-11T00:33:38.774774+0000 (oldest deadline 2024-09-11T00:34:04.674919+0000) 2024-09-11T00:34:08.582 INFO:journalctl@ceph.osd.5.smithi178.stdout:Sep 11 00:34:08 smithi178 podman[78836]: 2024-09-11 00:34:08.411360139 +0000 UTC m=+5.616384477 container died 78e183446c6e436ad1676ab3ad2cc7c3610774095af6f3e24f343797cab596e9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-5, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, ceph=True, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0) 2024-09-11T00:34:08.582 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 00:34:08 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-11[60852]: 2024-09-11T00:34:08.408+0000 7fd754b0b640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-09-11T00:33:24.081060+0000 front 2024-09-11T00:33:24.081080+0000 (oldest deadline 2024-09-11T00:33:49.980779+0000) 2024-09-11T00:34:08.582 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 00:34:08 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-11[60852]: 2024-09-11T00:34:08.408+0000 7fd754b0b640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-11T00:33:41.682818+0000 front 2024-09-11T00:33:41.682848+0000 (oldest deadline 2024-09-11T00:34:05.182914+0000) 2024-09-11T00:34:09.056 INFO:journalctl@ceph.osd.5.smithi178.stdout:Sep 11 00:34:08 smithi178 podman[78836]: 2024-09-11 00:34:08.828125142 +0000 UTC m=+6.033149482 container cleanup 78e183446c6e436ad1676ab3ad2cc7c3610774095af6f3e24f343797cab596e9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-5, org.label-schema.schema-version=1.0, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-11T00:34:09.056 INFO:journalctl@ceph.osd.5.smithi178.stdout:Sep 11 00:34:08 smithi178 bash[78836]: ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-5 2024-09-11T00:34:09.063 INFO:journalctl@ceph.osd.6.smithi012.stdout:Sep 11 00:34:08 smithi012 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-6[59757]: 2024-09-11T00:34:08.661+0000 7f2e71000640 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-11T00:33:34.716146+0000 front 2024-09-11T00:33:34.715917+0000 (oldest deadline 2024-09-11T00:33:57.015981+0000) 2024-09-11T00:34:09.063 INFO:journalctl@ceph.osd.6.smithi012.stdout:Sep 11 00:34:08 smithi012 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-6[59757]: 2024-09-11T00:34:08.661+0000 7f2e71000640 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-11T00:33:38.717829+0000 front 2024-09-11T00:33:38.717632+0000 (oldest deadline 2024-09-11T00:34:03.417594+0000) 2024-09-11T00:34:09.282 INFO:journalctl@ceph.osd.7.smithi165.stdout:Sep 11 00:34:08 smithi165 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-7[52941]: 2024-09-11T00:34:08.947+0000 7ffbe0ef8640 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-11T00:33:32.707987+0000 front 2024-09-11T00:33:32.708022+0000 (oldest deadline 2024-09-11T00:33:56.808116+0000) 2024-09-11T00:34:09.283 INFO:journalctl@ceph.osd.7.smithi165.stdout:Sep 11 00:34:08 smithi165 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-7[52941]: 2024-09-11T00:34:08.947+0000 7ffbe0ef8640 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-11T00:33:41.509403+0000 front 2024-09-11T00:33:41.509317+0000 (oldest deadline 2024-09-11T00:34:03.809439+0000) 2024-09-11T00:34:09.283 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 11 00:34:08 smithi165 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-10[59927]: 2024-09-11T00:34:08.857+0000 7f2cff41d640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-09-11T00:33:25.146772+0000 front 2024-09-11T00:33:25.146620+0000 (oldest deadline 2024-09-11T00:33:49.846529+0000) 2024-09-11T00:34:09.283 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 11 00:34:08 smithi165 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-10[59927]: 2024-09-11T00:34:08.857+0000 7f2cff41d640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-11T00:33:32.747514+0000 front 2024-09-11T00:33:32.747491+0000 (oldest deadline 2024-09-11T00:33:58.047686+0000) 2024-09-11T00:34:09.283 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 11 00:34:08 smithi165 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-10[59927]: 2024-09-11T00:34:08.857+0000 7f2cff41d640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-11T00:33:41.548749+0000 front 2024-09-11T00:33:41.548701+0000 (oldest deadline 2024-09-11T00:34:05.648724+0000) 2024-09-11T00:34:09.331 INFO:journalctl@ceph.osd.5.smithi178.stdout:Sep 11 00:34:09 smithi178 podman[78853]: 2024-09-11 00:34:09.054631962 +0000 UTC m=+0.640709840 container remove 78e183446c6e436ad1676ab3ad2cc7c3610774095af6f3e24f343797cab596e9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-5, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True) 2024-09-11T00:34:09.331 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 11 00:34:09 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-8[54042]: 2024-09-11T00:34:09.210+0000 7f23083d6640 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-11T00:33:33.673118+0000 front 2024-09-11T00:33:33.673042+0000 (oldest deadline 2024-09-11T00:33:56.573057+0000) 2024-09-11T00:34:09.331 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 11 00:34:09 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-8[54042]: 2024-09-11T00:34:09.210+0000 7f23083d6640 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-11T00:33:38.774830+0000 front 2024-09-11T00:33:38.774774+0000 (oldest deadline 2024-09-11T00:34:04.674919+0000) 2024-09-11T00:34:09.563 INFO:journalctl@ceph.osd.9.smithi012.stdout:Sep 11 00:34:09 smithi012 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-9[66733]: 2024-09-11T00:34:09.074+0000 7f5f4e800640 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-09-11T00:33:25.406156+0000 front 2024-09-11T00:33:25.406095+0000 (oldest deadline 2024-09-11T00:33:47.706239+0000) 2024-09-11T00:34:09.563 INFO:journalctl@ceph.osd.9.smithi012.stdout:Sep 11 00:34:09 smithi012 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-9[66733]: 2024-09-11T00:34:09.074+0000 7f5f4e800640 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-11T00:33:30.607371+0000 front 2024-09-11T00:33:30.607499+0000 (oldest deadline 2024-09-11T00:33:55.307466+0000) 2024-09-11T00:34:09.563 INFO:journalctl@ceph.osd.9.smithi012.stdout:Sep 11 00:34:09 smithi012 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-9[66733]: 2024-09-11T00:34:09.074+0000 7f5f4e800640 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-11T00:33:41.710055+0000 front 2024-09-11T00:33:41.709869+0000 (oldest deadline 2024-09-11T00:34:07.010035+0000) 2024-09-11T00:34:09.734 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 00:34:09 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-11[60852]: 2024-09-11T00:34:09.450+0000 7fd754b0b640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-09-11T00:33:24.081060+0000 front 2024-09-11T00:33:24.081080+0000 (oldest deadline 2024-09-11T00:33:49.980779+0000) 2024-09-11T00:34:09.734 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 00:34:09 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-11[60852]: 2024-09-11T00:34:09.450+0000 7fd754b0b640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-11T00:33:41.682818+0000 front 2024-09-11T00:33:41.682848+0000 (oldest deadline 2024-09-11T00:34:05.182914+0000) 2024-09-11T00:34:09.734 INFO:journalctl@ceph.osd.5.smithi178.stdout:Sep 11 00:34:09 smithi178 podman[78940]: 2024-09-11 00:34:09.384066675 +0000 UTC m=+0.015946223 image pull 589dd18248e494ba411dc838aa4ae0e5e569213f08fcd7a2cf22cd3ef7e3c480 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e 2024-09-11T00:34:09.734 INFO:journalctl@ceph.osd.5.smithi178.stdout:Sep 11 00:34:09 smithi178 podman[78940]: 2024-09-11 00:34:09.488868977 +0000 UTC m=+0.120748525 container create 6ae2c0351e0b2e3f942e4215646e0894fbe9d4a62898a4ac6143d8106baf9c3c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-5-deactivate, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, ceph=True, GIT_CLEAN=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.build-date=20240731) 2024-09-11T00:34:09.735 INFO:journalctl@ceph.osd.5.smithi178.stdout:Sep 11 00:34:09 smithi178 podman[78940]: 2024-09-11 00:34:09.733130008 +0000 UTC m=+0.365009557 container init 6ae2c0351e0b2e3f942e4215646e0894fbe9d4a62898a4ac6143d8106baf9c3c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-5-deactivate, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, io.buildah.version=1.37.2, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-11T00:34:10.063 INFO:journalctl@ceph.osd.6.smithi012.stdout:Sep 11 00:34:09 smithi012 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-6[59757]: 2024-09-11T00:34:09.659+0000 7f2e71000640 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-11T00:33:34.716146+0000 front 2024-09-11T00:33:34.715917+0000 (oldest deadline 2024-09-11T00:33:57.015981+0000) 2024-09-11T00:34:10.063 INFO:journalctl@ceph.osd.6.smithi012.stdout:Sep 11 00:34:09 smithi012 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-6[59757]: 2024-09-11T00:34:09.659+0000 7f2e71000640 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-11T00:33:38.717829+0000 front 2024-09-11T00:33:38.717632+0000 (oldest deadline 2024-09-11T00:34:03.417594+0000) 2024-09-11T00:34:10.081 INFO:journalctl@ceph.osd.5.smithi178.stdout:Sep 11 00:34:09 smithi178 podman[78940]: 2024-09-11 00:34:09.73649959 +0000 UTC m=+0.368379145 container start 6ae2c0351e0b2e3f942e4215646e0894fbe9d4a62898a4ac6143d8106baf9c3c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-5-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-11T00:34:10.081 INFO:journalctl@ceph.osd.5.smithi178.stdout:Sep 11 00:34:09 smithi178 podman[78940]: 2024-09-11 00:34:09.824848479 +0000 UTC m=+0.456728035 container attach 6ae2c0351e0b2e3f942e4215646e0894fbe9d4a62898a4ac6143d8106baf9c3c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-5-deactivate, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.37.2, ceph=True, org.label-schema.license=GPLv2, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS) 2024-09-11T00:34:10.081 INFO:journalctl@ceph.osd.5.smithi178.stdout:Sep 11 00:34:09 smithi178 podman[78940]: 2024-09-11 00:34:09.920341271 +0000 UTC m=+0.552220821 container died 6ae2c0351e0b2e3f942e4215646e0894fbe9d4a62898a4ac6143d8106baf9c3c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-5-deactivate, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4) 2024-09-11T00:34:10.283 INFO:journalctl@ceph.osd.7.smithi165.stdout:Sep 11 00:34:09 smithi165 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-7[52941]: 2024-09-11T00:34:09.993+0000 7ffbe0ef8640 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-11T00:33:32.707987+0000 front 2024-09-11T00:33:32.708022+0000 (oldest deadline 2024-09-11T00:33:56.808116+0000) 2024-09-11T00:34:10.283 INFO:journalctl@ceph.osd.7.smithi165.stdout:Sep 11 00:34:09 smithi165 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-7[52941]: 2024-09-11T00:34:09.993+0000 7ffbe0ef8640 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-11T00:33:41.509403+0000 front 2024-09-11T00:33:41.509317+0000 (oldest deadline 2024-09-11T00:34:03.809439+0000) 2024-09-11T00:34:10.283 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 11 00:34:09 smithi165 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-10[59927]: 2024-09-11T00:34:09.843+0000 7f2cff41d640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-09-11T00:33:25.146772+0000 front 2024-09-11T00:33:25.146620+0000 (oldest deadline 2024-09-11T00:33:49.846529+0000) 2024-09-11T00:34:10.283 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 11 00:34:09 smithi165 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-10[59927]: 2024-09-11T00:34:09.843+0000 7f2cff41d640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-11T00:33:32.747514+0000 front 2024-09-11T00:33:32.747491+0000 (oldest deadline 2024-09-11T00:33:58.047686+0000) 2024-09-11T00:34:10.283 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 11 00:34:09 smithi165 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-10[59927]: 2024-09-11T00:34:09.843+0000 7f2cff41d640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-11T00:33:41.548749+0000 front 2024-09-11T00:33:41.548701+0000 (oldest deadline 2024-09-11T00:34:05.648724+0000) 2024-09-11T00:34:10.495 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 11 00:34:10 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-8[54042]: 2024-09-11T00:34:10.242+0000 7f23083d6640 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-11T00:33:33.673118+0000 front 2024-09-11T00:33:33.673042+0000 (oldest deadline 2024-09-11T00:33:56.573057+0000) 2024-09-11T00:34:10.495 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 11 00:34:10 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-8[54042]: 2024-09-11T00:34:10.242+0000 7f23083d6640 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-11T00:33:38.774830+0000 front 2024-09-11T00:33:38.774774+0000 (oldest deadline 2024-09-11T00:34:04.674919+0000) 2024-09-11T00:34:10.563 INFO:journalctl@ceph.osd.9.smithi012.stdout:Sep 11 00:34:10 smithi012 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-9[66733]: 2024-09-11T00:34:10.094+0000 7f5f4e800640 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-09-11T00:33:25.406156+0000 front 2024-09-11T00:33:25.406095+0000 (oldest deadline 2024-09-11T00:33:47.706239+0000) 2024-09-11T00:34:10.563 INFO:journalctl@ceph.osd.9.smithi012.stdout:Sep 11 00:34:10 smithi012 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-9[66733]: 2024-09-11T00:34:10.094+0000 7f5f4e800640 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-11T00:33:30.607371+0000 front 2024-09-11T00:33:30.607499+0000 (oldest deadline 2024-09-11T00:33:55.307466+0000) 2024-09-11T00:34:10.563 INFO:journalctl@ceph.osd.9.smithi012.stdout:Sep 11 00:34:10 smithi012 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-9[66733]: 2024-09-11T00:34:10.094+0000 7f5f4e800640 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-11T00:33:41.710055+0000 front 2024-09-11T00:33:41.709869+0000 (oldest deadline 2024-09-11T00:34:07.010035+0000) 2024-09-11T00:34:10.620 DEBUG:teuthology.orchestra.run.smithi178:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968@osd.5.service' 2024-09-11T00:34:10.661 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T00:34:10.661 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-09-11T00:34:10.661 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2024-09-11T00:34:10.662 DEBUG:teuthology.orchestra.run.smithi012:> sudo systemctl stop ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968@osd.6 2024-09-11T00:34:10.831 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 00:34:10 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-11[60852]: 2024-09-11T00:34:10.493+0000 7fd754b0b640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-09-11T00:33:24.081060+0000 front 2024-09-11T00:33:24.081080+0000 (oldest deadline 2024-09-11T00:33:49.980779+0000) 2024-09-11T00:34:10.831 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 00:34:10 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-11[60852]: 2024-09-11T00:34:10.493+0000 7fd754b0b640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-11T00:33:41.682818+0000 front 2024-09-11T00:33:41.682848+0000 (oldest deadline 2024-09-11T00:34:05.182914+0000) 2024-09-11T00:34:11.052 INFO:journalctl@ceph.osd.6.smithi012.stdout:Sep 11 00:34:10 smithi012 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-6[59757]: 2024-09-11T00:34:10.705+0000 7f2e71000640 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-11T00:33:34.716146+0000 front 2024-09-11T00:33:34.715917+0000 (oldest deadline 2024-09-11T00:33:57.015981+0000) 2024-09-11T00:34:11.052 INFO:journalctl@ceph.osd.6.smithi012.stdout:Sep 11 00:34:10 smithi012 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-6[59757]: 2024-09-11T00:34:10.705+0000 7f2e71000640 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-11T00:33:38.717829+0000 front 2024-09-11T00:33:38.717632+0000 (oldest deadline 2024-09-11T00:34:03.417594+0000) 2024-09-11T00:34:11.052 INFO:journalctl@ceph.osd.6.smithi012.stdout:Sep 11 00:34:10 smithi012 systemd[1]: Stopping Ceph osd.6 for ff3ad74a-6fcf-11ef-bcea-c7b262605968... 2024-09-11T00:34:11.052 INFO:journalctl@ceph.osd.6.smithi012.stdout:Sep 11 00:34:11 smithi012 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-6[59757]: 2024-09-11T00:34:11.049+0000 7f2e7a600640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T00:34:11.052 INFO:journalctl@ceph.osd.6.smithi012.stdout:Sep 11 00:34:11 smithi012 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-6[59757]: 2024-09-11T00:34:11.050+0000 7f2e7a600640 -1 osd.6 91 *** Got signal Terminated *** 2024-09-11T00:34:11.052 INFO:journalctl@ceph.osd.6.smithi012.stdout:Sep 11 00:34:11 smithi012 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-6[59757]: 2024-09-11T00:34:11.050+0000 7f2e7a600640 -1 osd.6 91 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-11T00:34:11.282 INFO:journalctl@ceph.osd.7.smithi165.stdout:Sep 11 00:34:11 smithi165 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-7[52941]: 2024-09-11T00:34:11.008+0000 7ffbe0ef8640 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-11T00:33:32.707987+0000 front 2024-09-11T00:33:32.708022+0000 (oldest deadline 2024-09-11T00:33:56.808116+0000) 2024-09-11T00:34:11.282 INFO:journalctl@ceph.osd.7.smithi165.stdout:Sep 11 00:34:11 smithi165 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-7[52941]: 2024-09-11T00:34:11.008+0000 7ffbe0ef8640 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-11T00:33:41.509403+0000 front 2024-09-11T00:33:41.509317+0000 (oldest deadline 2024-09-11T00:34:03.809439+0000) 2024-09-11T00:34:11.283 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 11 00:34:10 smithi165 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-10[59927]: 2024-09-11T00:34:10.835+0000 7f2cff41d640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-09-11T00:33:25.146772+0000 front 2024-09-11T00:33:25.146620+0000 (oldest deadline 2024-09-11T00:33:49.846529+0000) 2024-09-11T00:34:11.283 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 11 00:34:10 smithi165 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-10[59927]: 2024-09-11T00:34:10.835+0000 7f2cff41d640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-11T00:33:32.747514+0000 front 2024-09-11T00:33:32.747491+0000 (oldest deadline 2024-09-11T00:33:58.047686+0000) 2024-09-11T00:34:11.283 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 11 00:34:10 smithi165 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-10[59927]: 2024-09-11T00:34:10.835+0000 7f2cff41d640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-11T00:33:41.548749+0000 front 2024-09-11T00:33:41.548701+0000 (oldest deadline 2024-09-11T00:34:05.648724+0000) 2024-09-11T00:34:11.313 INFO:journalctl@ceph.osd.9.smithi012.stdout:Sep 11 00:34:11 smithi012 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-9[66733]: 2024-09-11T00:34:11.075+0000 7f5f4e800640 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-09-11T00:33:25.406156+0000 front 2024-09-11T00:33:25.406095+0000 (oldest deadline 2024-09-11T00:33:47.706239+0000) 2024-09-11T00:34:11.313 INFO:journalctl@ceph.osd.9.smithi012.stdout:Sep 11 00:34:11 smithi012 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-9[66733]: 2024-09-11T00:34:11.075+0000 7f5f4e800640 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-11T00:33:30.607371+0000 front 2024-09-11T00:33:30.607499+0000 (oldest deadline 2024-09-11T00:33:55.307466+0000) 2024-09-11T00:34:11.313 INFO:journalctl@ceph.osd.9.smithi012.stdout:Sep 11 00:34:11 smithi012 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-9[66733]: 2024-09-11T00:34:11.075+0000 7f5f4e800640 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-11T00:33:41.710055+0000 front 2024-09-11T00:33:41.709869+0000 (oldest deadline 2024-09-11T00:34:07.010035+0000) 2024-09-11T00:34:11.581 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 00:34:11 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-11[60852]: 2024-09-11T00:34:11.480+0000 7fd754b0b640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-09-11T00:33:24.081060+0000 front 2024-09-11T00:33:24.081080+0000 (oldest deadline 2024-09-11T00:33:49.980779+0000) 2024-09-11T00:34:11.581 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 00:34:11 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-11[60852]: 2024-09-11T00:34:11.480+0000 7fd754b0b640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-11T00:33:41.682818+0000 front 2024-09-11T00:33:41.682848+0000 (oldest deadline 2024-09-11T00:34:05.182914+0000) 2024-09-11T00:34:11.581 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 11 00:34:11 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-8[54042]: 2024-09-11T00:34:11.273+0000 7f23083d6640 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-11T00:33:33.673118+0000 front 2024-09-11T00:33:33.673042+0000 (oldest deadline 2024-09-11T00:33:56.573057+0000) 2024-09-11T00:34:11.582 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 11 00:34:11 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-8[54042]: 2024-09-11T00:34:11.273+0000 7f23083d6640 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-11T00:33:38.774830+0000 front 2024-09-11T00:33:38.774774+0000 (oldest deadline 2024-09-11T00:34:04.674919+0000) 2024-09-11T00:34:12.063 INFO:journalctl@ceph.osd.6.smithi012.stdout:Sep 11 00:34:11 smithi012 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-6[59757]: 2024-09-11T00:34:11.714+0000 7f2e71000640 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-11T00:33:34.716146+0000 front 2024-09-11T00:33:34.715917+0000 (oldest deadline 2024-09-11T00:33:57.015981+0000) 2024-09-11T00:34:12.063 INFO:journalctl@ceph.osd.6.smithi012.stdout:Sep 11 00:34:11 smithi012 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-6[59757]: 2024-09-11T00:34:11.714+0000 7f2e71000640 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-11T00:33:38.717829+0000 front 2024-09-11T00:33:38.717632+0000 (oldest deadline 2024-09-11T00:34:03.417594+0000) 2024-09-11T00:34:12.282 INFO:journalctl@ceph.osd.7.smithi165.stdout:Sep 11 00:34:12 smithi165 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-7[52941]: 2024-09-11T00:34:12.010+0000 7ffbe0ef8640 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-11T00:33:32.707987+0000 front 2024-09-11T00:33:32.708022+0000 (oldest deadline 2024-09-11T00:33:56.808116+0000) 2024-09-11T00:34:12.283 INFO:journalctl@ceph.osd.7.smithi165.stdout:Sep 11 00:34:12 smithi165 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-7[52941]: 2024-09-11T00:34:12.010+0000 7ffbe0ef8640 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-11T00:33:41.509403+0000 front 2024-09-11T00:33:41.509317+0000 (oldest deadline 2024-09-11T00:34:03.809439+0000) 2024-09-11T00:34:12.283 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 11 00:34:11 smithi165 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-10[59927]: 2024-09-11T00:34:11.865+0000 7f2cff41d640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-09-11T00:33:25.146772+0000 front 2024-09-11T00:33:25.146620+0000 (oldest deadline 2024-09-11T00:33:49.846529+0000) 2024-09-11T00:34:12.283 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 11 00:34:11 smithi165 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-10[59927]: 2024-09-11T00:34:11.865+0000 7f2cff41d640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-11T00:33:32.747514+0000 front 2024-09-11T00:33:32.747491+0000 (oldest deadline 2024-09-11T00:33:58.047686+0000) 2024-09-11T00:34:12.283 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 11 00:34:11 smithi165 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-10[59927]: 2024-09-11T00:34:11.865+0000 7f2cff41d640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-11T00:33:41.548749+0000 front 2024-09-11T00:33:41.548701+0000 (oldest deadline 2024-09-11T00:34:05.648724+0000) 2024-09-11T00:34:12.563 INFO:journalctl@ceph.osd.9.smithi012.stdout:Sep 11 00:34:12 smithi012 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-9[66733]: 2024-09-11T00:34:12.098+0000 7f5f4e800640 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-09-11T00:33:25.406156+0000 front 2024-09-11T00:33:25.406095+0000 (oldest deadline 2024-09-11T00:33:47.706239+0000) 2024-09-11T00:34:12.563 INFO:journalctl@ceph.osd.9.smithi012.stdout:Sep 11 00:34:12 smithi012 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-9[66733]: 2024-09-11T00:34:12.098+0000 7f5f4e800640 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-11T00:33:30.607371+0000 front 2024-09-11T00:33:30.607499+0000 (oldest deadline 2024-09-11T00:33:55.307466+0000) 2024-09-11T00:34:12.563 INFO:journalctl@ceph.osd.9.smithi012.stdout:Sep 11 00:34:12 smithi012 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-9[66733]: 2024-09-11T00:34:12.098+0000 7f5f4e800640 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-11T00:33:41.710055+0000 front 2024-09-11T00:33:41.709869+0000 (oldest deadline 2024-09-11T00:34:07.010035+0000) 2024-09-11T00:34:12.581 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 00:34:12 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-11[60852]: 2024-09-11T00:34:12.478+0000 7fd754b0b640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-09-11T00:33:24.081060+0000 front 2024-09-11T00:33:24.081080+0000 (oldest deadline 2024-09-11T00:33:49.980779+0000) 2024-09-11T00:34:12.581 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 00:34:12 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-11[60852]: 2024-09-11T00:34:12.478+0000 7fd754b0b640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-11T00:33:41.682818+0000 front 2024-09-11T00:33:41.682848+0000 (oldest deadline 2024-09-11T00:34:05.182914+0000) 2024-09-11T00:34:12.581 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 11 00:34:12 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-8[54042]: 2024-09-11T00:34:12.315+0000 7f23083d6640 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-11T00:33:33.673118+0000 front 2024-09-11T00:33:33.673042+0000 (oldest deadline 2024-09-11T00:33:56.573057+0000) 2024-09-11T00:34:12.581 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 11 00:34:12 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-8[54042]: 2024-09-11T00:34:12.315+0000 7f23083d6640 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-11T00:33:38.774830+0000 front 2024-09-11T00:33:38.774774+0000 (oldest deadline 2024-09-11T00:34:04.674919+0000) 2024-09-11T00:34:13.063 INFO:journalctl@ceph.osd.6.smithi012.stdout:Sep 11 00:34:12 smithi012 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-6[59757]: 2024-09-11T00:34:12.696+0000 7f2e71000640 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-11T00:33:34.716146+0000 front 2024-09-11T00:33:34.715917+0000 (oldest deadline 2024-09-11T00:33:57.015981+0000) 2024-09-11T00:34:13.063 INFO:journalctl@ceph.osd.6.smithi012.stdout:Sep 11 00:34:12 smithi012 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-6[59757]: 2024-09-11T00:34:12.696+0000 7f2e71000640 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-11T00:33:38.717829+0000 front 2024-09-11T00:33:38.717632+0000 (oldest deadline 2024-09-11T00:34:03.417594+0000) 2024-09-11T00:34:13.064 INFO:journalctl@ceph.osd.6.smithi012.stdout:Sep 11 00:34:12 smithi012 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-6[59757]: 2024-09-11T00:34:12.696+0000 7f2e71000640 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-09-11T00:33:49.318682+0000 front 2024-09-11T00:33:49.318739+0000 (oldest deadline 2024-09-11T00:34:12.218590+0000) 2024-09-11T00:34:13.283 INFO:journalctl@ceph.osd.7.smithi165.stdout:Sep 11 00:34:12 smithi165 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-7[52941]: 2024-09-11T00:34:12.979+0000 7ffbe0ef8640 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-11T00:33:32.707987+0000 front 2024-09-11T00:33:32.708022+0000 (oldest deadline 2024-09-11T00:33:56.808116+0000) 2024-09-11T00:34:13.283 INFO:journalctl@ceph.osd.7.smithi165.stdout:Sep 11 00:34:12 smithi165 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-7[52941]: 2024-09-11T00:34:12.979+0000 7ffbe0ef8640 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-11T00:33:41.509403+0000 front 2024-09-11T00:33:41.509317+0000 (oldest deadline 2024-09-11T00:34:03.809439+0000) 2024-09-11T00:34:13.283 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 11 00:34:12 smithi165 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-10[59927]: 2024-09-11T00:34:12.898+0000 7f2cff41d640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-09-11T00:33:25.146772+0000 front 2024-09-11T00:33:25.146620+0000 (oldest deadline 2024-09-11T00:33:49.846529+0000) 2024-09-11T00:34:13.284 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 11 00:34:12 smithi165 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-10[59927]: 2024-09-11T00:34:12.898+0000 7f2cff41d640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-11T00:33:32.747514+0000 front 2024-09-11T00:33:32.747491+0000 (oldest deadline 2024-09-11T00:33:58.047686+0000) 2024-09-11T00:34:13.284 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 11 00:34:12 smithi165 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-10[59927]: 2024-09-11T00:34:12.898+0000 7f2cff41d640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-11T00:33:41.548749+0000 front 2024-09-11T00:33:41.548701+0000 (oldest deadline 2024-09-11T00:34:05.648724+0000) 2024-09-11T00:34:13.284 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 11 00:34:12 smithi165 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-10[59927]: 2024-09-11T00:34:12.898+0000 7f2cff41d640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-09-11T00:33:50.850789+0000 front 2024-09-11T00:33:50.850826+0000 (oldest deadline 2024-09-11T00:34:11.950647+0000) 2024-09-11T00:34:13.563 INFO:journalctl@ceph.osd.9.smithi012.stdout:Sep 11 00:34:13 smithi012 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-9[66733]: 2024-09-11T00:34:13.126+0000 7f5f4e800640 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-09-11T00:33:25.406156+0000 front 2024-09-11T00:33:25.406095+0000 (oldest deadline 2024-09-11T00:33:47.706239+0000) 2024-09-11T00:34:13.563 INFO:journalctl@ceph.osd.9.smithi012.stdout:Sep 11 00:34:13 smithi012 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-9[66733]: 2024-09-11T00:34:13.126+0000 7f5f4e800640 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-11T00:33:30.607371+0000 front 2024-09-11T00:33:30.607499+0000 (oldest deadline 2024-09-11T00:33:55.307466+0000) 2024-09-11T00:34:13.564 INFO:journalctl@ceph.osd.9.smithi012.stdout:Sep 11 00:34:13 smithi012 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-9[66733]: 2024-09-11T00:34:13.126+0000 7f5f4e800640 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-11T00:33:41.710055+0000 front 2024-09-11T00:33:41.709869+0000 (oldest deadline 2024-09-11T00:34:07.010035+0000) 2024-09-11T00:34:13.564 INFO:journalctl@ceph.osd.9.smithi012.stdout:Sep 11 00:34:13 smithi012 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-9[66733]: 2024-09-11T00:34:13.126+0000 7f5f4e800640 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-09-11T00:33:47.010623+0000 front 2024-09-11T00:33:47.010590+0000 (oldest deadline 2024-09-11T00:34:12.910692+0000) 2024-09-11T00:34:13.581 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 11 00:34:13 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-8[54042]: 2024-09-11T00:34:13.321+0000 7f23083d6640 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-11T00:33:33.673118+0000 front 2024-09-11T00:33:33.673042+0000 (oldest deadline 2024-09-11T00:33:56.573057+0000) 2024-09-11T00:34:13.581 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 11 00:34:13 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-8[54042]: 2024-09-11T00:34:13.321+0000 7f23083d6640 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-11T00:33:38.774830+0000 front 2024-09-11T00:33:38.774774+0000 (oldest deadline 2024-09-11T00:34:04.674919+0000) 2024-09-11T00:34:13.581 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 11 00:34:13 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-8[54042]: 2024-09-11T00:34:13.321+0000 7f23083d6640 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-09-11T00:33:47.977769+0000 front 2024-09-11T00:33:47.977619+0000 (oldest deadline 2024-09-11T00:34:13.277556+0000) 2024-09-11T00:34:13.581 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 00:34:13 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-11[60852]: 2024-09-11T00:34:13.484+0000 7fd754b0b640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-09-11T00:33:24.081060+0000 front 2024-09-11T00:33:24.081080+0000 (oldest deadline 2024-09-11T00:33:49.980779+0000) 2024-09-11T00:34:13.581 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 00:34:13 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-11[60852]: 2024-09-11T00:34:13.484+0000 7fd754b0b640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-11T00:33:41.682818+0000 front 2024-09-11T00:33:41.682848+0000 (oldest deadline 2024-09-11T00:34:05.182914+0000) 2024-09-11T00:34:13.582 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 00:34:13 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-11[60852]: 2024-09-11T00:34:13.484+0000 7fd754b0b640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-09-11T00:33:47.484088+0000 front 2024-09-11T00:33:47.484103+0000 (oldest deadline 2024-09-11T00:34:12.784060+0000) 2024-09-11T00:34:14.063 INFO:journalctl@ceph.osd.6.smithi012.stdout:Sep 11 00:34:13 smithi012 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-6[59757]: 2024-09-11T00:34:13.717+0000 7f2e71000640 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-11T00:33:34.716146+0000 front 2024-09-11T00:33:34.715917+0000 (oldest deadline 2024-09-11T00:33:57.015981+0000) 2024-09-11T00:34:14.063 INFO:journalctl@ceph.osd.6.smithi012.stdout:Sep 11 00:34:13 smithi012 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-6[59757]: 2024-09-11T00:34:13.717+0000 7f2e71000640 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-11T00:33:38.717829+0000 front 2024-09-11T00:33:38.717632+0000 (oldest deadline 2024-09-11T00:34:03.417594+0000) 2024-09-11T00:34:14.063 INFO:journalctl@ceph.osd.6.smithi012.stdout:Sep 11 00:34:13 smithi012 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-6[59757]: 2024-09-11T00:34:13.717+0000 7f2e71000640 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-09-11T00:33:49.318682+0000 front 2024-09-11T00:33:49.318739+0000 (oldest deadline 2024-09-11T00:34:12.218590+0000) 2024-09-11T00:34:14.282 INFO:journalctl@ceph.osd.7.smithi165.stdout:Sep 11 00:34:13 smithi165 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-7[52941]: 2024-09-11T00:34:13.973+0000 7ffbe0ef8640 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-11T00:33:32.707987+0000 front 2024-09-11T00:33:32.708022+0000 (oldest deadline 2024-09-11T00:33:56.808116+0000) 2024-09-11T00:34:14.283 INFO:journalctl@ceph.osd.7.smithi165.stdout:Sep 11 00:34:13 smithi165 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-7[52941]: 2024-09-11T00:34:13.973+0000 7ffbe0ef8640 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-11T00:33:41.509403+0000 front 2024-09-11T00:33:41.509317+0000 (oldest deadline 2024-09-11T00:34:03.809439+0000) 2024-09-11T00:34:14.283 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 11 00:34:13 smithi165 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-10[59927]: 2024-09-11T00:34:13.898+0000 7f2cff41d640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-09-11T00:33:25.146772+0000 front 2024-09-11T00:33:25.146620+0000 (oldest deadline 2024-09-11T00:33:49.846529+0000) 2024-09-11T00:34:14.283 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 11 00:34:13 smithi165 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-10[59927]: 2024-09-11T00:34:13.898+0000 7f2cff41d640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-11T00:33:32.747514+0000 front 2024-09-11T00:33:32.747491+0000 (oldest deadline 2024-09-11T00:33:58.047686+0000) 2024-09-11T00:34:14.283 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 11 00:34:13 smithi165 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-10[59927]: 2024-09-11T00:34:13.898+0000 7f2cff41d640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-11T00:33:41.548749+0000 front 2024-09-11T00:33:41.548701+0000 (oldest deadline 2024-09-11T00:34:05.648724+0000) 2024-09-11T00:34:14.283 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 11 00:34:13 smithi165 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-10[59927]: 2024-09-11T00:34:13.898+0000 7f2cff41d640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-09-11T00:33:50.850789+0000 front 2024-09-11T00:33:50.850826+0000 (oldest deadline 2024-09-11T00:34:11.950647+0000) 2024-09-11T00:34:14.563 INFO:journalctl@ceph.osd.9.smithi012.stdout:Sep 11 00:34:14 smithi012 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-9[66733]: 2024-09-11T00:34:14.175+0000 7f5f4e800640 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-09-11T00:33:25.406156+0000 front 2024-09-11T00:33:25.406095+0000 (oldest deadline 2024-09-11T00:33:47.706239+0000) 2024-09-11T00:34:14.563 INFO:journalctl@ceph.osd.9.smithi012.stdout:Sep 11 00:34:14 smithi012 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-9[66733]: 2024-09-11T00:34:14.175+0000 7f5f4e800640 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-11T00:33:30.607371+0000 front 2024-09-11T00:33:30.607499+0000 (oldest deadline 2024-09-11T00:33:55.307466+0000) 2024-09-11T00:34:14.564 INFO:journalctl@ceph.osd.9.smithi012.stdout:Sep 11 00:34:14 smithi012 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-9[66733]: 2024-09-11T00:34:14.175+0000 7f5f4e800640 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-11T00:33:41.710055+0000 front 2024-09-11T00:33:41.709869+0000 (oldest deadline 2024-09-11T00:34:07.010035+0000) 2024-09-11T00:34:14.564 INFO:journalctl@ceph.osd.9.smithi012.stdout:Sep 11 00:34:14 smithi012 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-9[66733]: 2024-09-11T00:34:14.175+0000 7f5f4e800640 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-09-11T00:33:47.010623+0000 front 2024-09-11T00:33:47.010590+0000 (oldest deadline 2024-09-11T00:34:12.910692+0000) 2024-09-11T00:34:14.581 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 00:34:14 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-11[60852]: 2024-09-11T00:34:14.453+0000 7fd754b0b640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-09-11T00:33:24.081060+0000 front 2024-09-11T00:33:24.081080+0000 (oldest deadline 2024-09-11T00:33:49.980779+0000) 2024-09-11T00:34:14.581 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 00:34:14 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-11[60852]: 2024-09-11T00:34:14.453+0000 7fd754b0b640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-11T00:33:41.682818+0000 front 2024-09-11T00:33:41.682848+0000 (oldest deadline 2024-09-11T00:34:05.182914+0000) 2024-09-11T00:34:14.581 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 00:34:14 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-11[60852]: 2024-09-11T00:34:14.453+0000 7fd754b0b640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-09-11T00:33:47.484088+0000 front 2024-09-11T00:33:47.484103+0000 (oldest deadline 2024-09-11T00:34:12.784060+0000) 2024-09-11T00:34:14.581 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 11 00:34:14 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-8[54042]: 2024-09-11T00:34:14.277+0000 7f23083d6640 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-11T00:33:33.673118+0000 front 2024-09-11T00:33:33.673042+0000 (oldest deadline 2024-09-11T00:33:56.573057+0000) 2024-09-11T00:34:14.582 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 11 00:34:14 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-8[54042]: 2024-09-11T00:34:14.277+0000 7f23083d6640 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-11T00:33:38.774830+0000 front 2024-09-11T00:33:38.774774+0000 (oldest deadline 2024-09-11T00:34:04.674919+0000) 2024-09-11T00:34:14.582 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 11 00:34:14 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-8[54042]: 2024-09-11T00:34:14.277+0000 7f23083d6640 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-09-11T00:33:47.977769+0000 front 2024-09-11T00:33:47.977619+0000 (oldest deadline 2024-09-11T00:34:13.277556+0000) 2024-09-11T00:34:15.063 INFO:journalctl@ceph.osd.6.smithi012.stdout:Sep 11 00:34:14 smithi012 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-6[59757]: 2024-09-11T00:34:14.717+0000 7f2e71000640 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-11T00:33:34.716146+0000 front 2024-09-11T00:33:34.715917+0000 (oldest deadline 2024-09-11T00:33:57.015981+0000) 2024-09-11T00:34:15.063 INFO:journalctl@ceph.osd.6.smithi012.stdout:Sep 11 00:34:14 smithi012 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-6[59757]: 2024-09-11T00:34:14.717+0000 7f2e71000640 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-11T00:33:38.717829+0000 front 2024-09-11T00:33:38.717632+0000 (oldest deadline 2024-09-11T00:34:03.417594+0000) 2024-09-11T00:34:15.064 INFO:journalctl@ceph.osd.6.smithi012.stdout:Sep 11 00:34:14 smithi012 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-6[59757]: 2024-09-11T00:34:14.717+0000 7f2e71000640 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-09-11T00:33:49.318682+0000 front 2024-09-11T00:33:49.318739+0000 (oldest deadline 2024-09-11T00:34:12.218590+0000) 2024-09-11T00:34:15.283 INFO:journalctl@ceph.osd.7.smithi165.stdout:Sep 11 00:34:15 smithi165 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-7[52941]: 2024-09-11T00:34:15.005+0000 7ffbe0ef8640 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-11T00:33:32.707987+0000 front 2024-09-11T00:33:32.708022+0000 (oldest deadline 2024-09-11T00:33:56.808116+0000) 2024-09-11T00:34:15.283 INFO:journalctl@ceph.osd.7.smithi165.stdout:Sep 11 00:34:15 smithi165 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-7[52941]: 2024-09-11T00:34:15.005+0000 7ffbe0ef8640 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-11T00:33:41.509403+0000 front 2024-09-11T00:33:41.509317+0000 (oldest deadline 2024-09-11T00:34:03.809439+0000) 2024-09-11T00:34:15.283 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 11 00:34:14 smithi165 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-10[59927]: 2024-09-11T00:34:14.931+0000 7f2cff41d640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-09-11T00:33:25.146772+0000 front 2024-09-11T00:33:25.146620+0000 (oldest deadline 2024-09-11T00:33:49.846529+0000) 2024-09-11T00:34:15.283 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 11 00:34:14 smithi165 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-10[59927]: 2024-09-11T00:34:14.931+0000 7f2cff41d640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-11T00:33:32.747514+0000 front 2024-09-11T00:33:32.747491+0000 (oldest deadline 2024-09-11T00:33:58.047686+0000) 2024-09-11T00:34:15.283 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 11 00:34:14 smithi165 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-10[59927]: 2024-09-11T00:34:14.931+0000 7f2cff41d640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-11T00:33:41.548749+0000 front 2024-09-11T00:33:41.548701+0000 (oldest deadline 2024-09-11T00:34:05.648724+0000) 2024-09-11T00:34:15.283 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 11 00:34:14 smithi165 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-10[59927]: 2024-09-11T00:34:14.931+0000 7f2cff41d640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-09-11T00:33:50.850789+0000 front 2024-09-11T00:33:50.850826+0000 (oldest deadline 2024-09-11T00:34:11.950647+0000) 2024-09-11T00:34:15.563 INFO:journalctl@ceph.osd.9.smithi012.stdout:Sep 11 00:34:15 smithi012 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-9[66733]: 2024-09-11T00:34:15.202+0000 7f5f4e800640 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-09-11T00:33:25.406156+0000 front 2024-09-11T00:33:25.406095+0000 (oldest deadline 2024-09-11T00:33:47.706239+0000) 2024-09-11T00:34:15.563 INFO:journalctl@ceph.osd.9.smithi012.stdout:Sep 11 00:34:15 smithi012 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-9[66733]: 2024-09-11T00:34:15.203+0000 7f5f4e800640 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-11T00:33:30.607371+0000 front 2024-09-11T00:33:30.607499+0000 (oldest deadline 2024-09-11T00:33:55.307466+0000) 2024-09-11T00:34:15.564 INFO:journalctl@ceph.osd.9.smithi012.stdout:Sep 11 00:34:15 smithi012 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-9[66733]: 2024-09-11T00:34:15.203+0000 7f5f4e800640 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-11T00:33:41.710055+0000 front 2024-09-11T00:33:41.709869+0000 (oldest deadline 2024-09-11T00:34:07.010035+0000) 2024-09-11T00:34:15.564 INFO:journalctl@ceph.osd.9.smithi012.stdout:Sep 11 00:34:15 smithi012 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-9[66733]: 2024-09-11T00:34:15.203+0000 7f5f4e800640 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-09-11T00:33:47.010623+0000 front 2024-09-11T00:33:47.010590+0000 (oldest deadline 2024-09-11T00:34:12.910692+0000) 2024-09-11T00:34:15.581 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 00:34:15 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-11[60852]: 2024-09-11T00:34:15.414+0000 7fd754b0b640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-09-11T00:33:24.081060+0000 front 2024-09-11T00:33:24.081080+0000 (oldest deadline 2024-09-11T00:33:49.980779+0000) 2024-09-11T00:34:15.581 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 00:34:15 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-11[60852]: 2024-09-11T00:34:15.414+0000 7fd754b0b640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-11T00:33:41.682818+0000 front 2024-09-11T00:33:41.682848+0000 (oldest deadline 2024-09-11T00:34:05.182914+0000) 2024-09-11T00:34:15.581 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 00:34:15 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-11[60852]: 2024-09-11T00:34:15.414+0000 7fd754b0b640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-09-11T00:33:47.484088+0000 front 2024-09-11T00:33:47.484103+0000 (oldest deadline 2024-09-11T00:34:12.784060+0000) 2024-09-11T00:34:15.581 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 11 00:34:15 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-8[54042]: 2024-09-11T00:34:15.314+0000 7f23083d6640 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-11T00:33:33.673118+0000 front 2024-09-11T00:33:33.673042+0000 (oldest deadline 2024-09-11T00:33:56.573057+0000) 2024-09-11T00:34:15.582 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 11 00:34:15 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-8[54042]: 2024-09-11T00:34:15.314+0000 7f23083d6640 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-11T00:33:38.774830+0000 front 2024-09-11T00:33:38.774774+0000 (oldest deadline 2024-09-11T00:34:04.674919+0000) 2024-09-11T00:34:15.582 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 11 00:34:15 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-8[54042]: 2024-09-11T00:34:15.314+0000 7f23083d6640 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-09-11T00:33:47.977769+0000 front 2024-09-11T00:33:47.977619+0000 (oldest deadline 2024-09-11T00:34:13.277556+0000) 2024-09-11T00:34:16.063 INFO:journalctl@ceph.osd.6.smithi012.stdout:Sep 11 00:34:15 smithi012 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-6[59757]: 2024-09-11T00:34:15.702+0000 7f2e71000640 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-11T00:33:34.716146+0000 front 2024-09-11T00:33:34.715917+0000 (oldest deadline 2024-09-11T00:33:57.015981+0000) 2024-09-11T00:34:16.063 INFO:journalctl@ceph.osd.6.smithi012.stdout:Sep 11 00:34:15 smithi012 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-6[59757]: 2024-09-11T00:34:15.702+0000 7f2e71000640 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-11T00:33:38.717829+0000 front 2024-09-11T00:33:38.717632+0000 (oldest deadline 2024-09-11T00:34:03.417594+0000) 2024-09-11T00:34:16.064 INFO:journalctl@ceph.osd.6.smithi012.stdout:Sep 11 00:34:15 smithi012 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-6[59757]: 2024-09-11T00:34:15.702+0000 7f2e71000640 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-09-11T00:33:49.318682+0000 front 2024-09-11T00:33:49.318739+0000 (oldest deadline 2024-09-11T00:34:12.218590+0000) 2024-09-11T00:34:16.282 INFO:journalctl@ceph.osd.7.smithi165.stdout:Sep 11 00:34:15 smithi165 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-7[52941]: 2024-09-11T00:34:15.964+0000 7ffbe0ef8640 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-11T00:33:32.707987+0000 front 2024-09-11T00:33:32.708022+0000 (oldest deadline 2024-09-11T00:33:56.808116+0000) 2024-09-11T00:34:16.283 INFO:journalctl@ceph.osd.7.smithi165.stdout:Sep 11 00:34:15 smithi165 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-7[52941]: 2024-09-11T00:34:15.964+0000 7ffbe0ef8640 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-11T00:33:41.509403+0000 front 2024-09-11T00:33:41.509317+0000 (oldest deadline 2024-09-11T00:34:03.809439+0000) 2024-09-11T00:34:16.283 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 11 00:34:15 smithi165 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-10[59927]: 2024-09-11T00:34:15.887+0000 7f2cff41d640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-09-11T00:33:25.146772+0000 front 2024-09-11T00:33:25.146620+0000 (oldest deadline 2024-09-11T00:33:49.846529+0000) 2024-09-11T00:34:16.283 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 11 00:34:15 smithi165 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-10[59927]: 2024-09-11T00:34:15.887+0000 7f2cff41d640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-11T00:33:32.747514+0000 front 2024-09-11T00:33:32.747491+0000 (oldest deadline 2024-09-11T00:33:58.047686+0000) 2024-09-11T00:34:16.283 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 11 00:34:15 smithi165 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-10[59927]: 2024-09-11T00:34:15.887+0000 7f2cff41d640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-11T00:33:41.548749+0000 front 2024-09-11T00:33:41.548701+0000 (oldest deadline 2024-09-11T00:34:05.648724+0000) 2024-09-11T00:34:16.283 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 11 00:34:15 smithi165 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-10[59927]: 2024-09-11T00:34:15.887+0000 7f2cff41d640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-09-11T00:33:50.850789+0000 front 2024-09-11T00:33:50.850826+0000 (oldest deadline 2024-09-11T00:34:11.950647+0000) 2024-09-11T00:34:16.408 INFO:journalctl@ceph.osd.9.smithi012.stdout:Sep 11 00:34:16 smithi012 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-9[66733]: 2024-09-11T00:34:16.170+0000 7f5f4e800640 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-09-11T00:33:25.406156+0000 front 2024-09-11T00:33:25.406095+0000 (oldest deadline 2024-09-11T00:33:47.706239+0000) 2024-09-11T00:34:16.408 INFO:journalctl@ceph.osd.9.smithi012.stdout:Sep 11 00:34:16 smithi012 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-9[66733]: 2024-09-11T00:34:16.170+0000 7f5f4e800640 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-11T00:33:30.607371+0000 front 2024-09-11T00:33:30.607499+0000 (oldest deadline 2024-09-11T00:33:55.307466+0000) 2024-09-11T00:34:16.408 INFO:journalctl@ceph.osd.9.smithi012.stdout:Sep 11 00:34:16 smithi012 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-9[66733]: 2024-09-11T00:34:16.170+0000 7f5f4e800640 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-11T00:33:41.710055+0000 front 2024-09-11T00:33:41.709869+0000 (oldest deadline 2024-09-11T00:34:07.010035+0000) 2024-09-11T00:34:16.408 INFO:journalctl@ceph.osd.9.smithi012.stdout:Sep 11 00:34:16 smithi012 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-9[66733]: 2024-09-11T00:34:16.170+0000 7f5f4e800640 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-09-11T00:33:47.010623+0000 front 2024-09-11T00:33:47.010590+0000 (oldest deadline 2024-09-11T00:34:12.910692+0000) 2024-09-11T00:34:16.581 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 00:34:16 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-11[60852]: 2024-09-11T00:34:16.398+0000 7fd754b0b640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-09-11T00:33:24.081060+0000 front 2024-09-11T00:33:24.081080+0000 (oldest deadline 2024-09-11T00:33:49.980779+0000) 2024-09-11T00:34:16.581 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 00:34:16 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-11[60852]: 2024-09-11T00:34:16.398+0000 7fd754b0b640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-11T00:33:41.682818+0000 front 2024-09-11T00:33:41.682848+0000 (oldest deadline 2024-09-11T00:34:05.182914+0000) 2024-09-11T00:34:16.582 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 00:34:16 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-11[60852]: 2024-09-11T00:34:16.398+0000 7fd754b0b640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-09-11T00:33:47.484088+0000 front 2024-09-11T00:33:47.484103+0000 (oldest deadline 2024-09-11T00:34:12.784060+0000) 2024-09-11T00:34:16.582 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 11 00:34:16 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-8[54042]: 2024-09-11T00:34:16.291+0000 7f23083d6640 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-11T00:33:33.673118+0000 front 2024-09-11T00:33:33.673042+0000 (oldest deadline 2024-09-11T00:33:56.573057+0000) 2024-09-11T00:34:16.582 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 11 00:34:16 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-8[54042]: 2024-09-11T00:34:16.291+0000 7f23083d6640 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-11T00:33:38.774830+0000 front 2024-09-11T00:33:38.774774+0000 (oldest deadline 2024-09-11T00:34:04.674919+0000) 2024-09-11T00:34:16.582 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 11 00:34:16 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-8[54042]: 2024-09-11T00:34:16.291+0000 7f23083d6640 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-09-11T00:33:47.977769+0000 front 2024-09-11T00:33:47.977619+0000 (oldest deadline 2024-09-11T00:34:13.277556+0000) 2024-09-11T00:34:16.732 INFO:journalctl@ceph.osd.6.smithi012.stdout:Sep 11 00:34:16 smithi012 podman[114606]: 2024-09-11 00:34:16.636466009 +0000 UTC m=+5.689926767 container died f0cf9000529edb85619e4d1ee9f8b67c4d4ab6b75105fa182dc6bdc6843f398b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-6, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2) 2024-09-11T00:34:17.283 INFO:journalctl@ceph.osd.7.smithi165.stdout:Sep 11 00:34:16 smithi165 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-7[52941]: 2024-09-11T00:34:16.981+0000 7ffbe0ef8640 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-11T00:33:32.707987+0000 front 2024-09-11T00:33:32.708022+0000 (oldest deadline 2024-09-11T00:33:56.808116+0000) 2024-09-11T00:34:17.283 INFO:journalctl@ceph.osd.7.smithi165.stdout:Sep 11 00:34:16 smithi165 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-7[52941]: 2024-09-11T00:34:16.981+0000 7ffbe0ef8640 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-11T00:33:41.509403+0000 front 2024-09-11T00:33:41.509317+0000 (oldest deadline 2024-09-11T00:34:03.809439+0000) 2024-09-11T00:34:17.283 INFO:journalctl@ceph.osd.7.smithi165.stdout:Sep 11 00:34:16 smithi165 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-7[52941]: 2024-09-11T00:34:16.981+0000 7ffbe0ef8640 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-09-11T00:33:50.712042+0000 front 2024-09-11T00:33:50.711963+0000 (oldest deadline 2024-09-11T00:34:16.011841+0000) 2024-09-11T00:34:17.284 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 11 00:34:16 smithi165 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-10[59927]: 2024-09-11T00:34:16.884+0000 7f2cff41d640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-09-11T00:33:25.146772+0000 front 2024-09-11T00:33:25.146620+0000 (oldest deadline 2024-09-11T00:33:49.846529+0000) 2024-09-11T00:34:17.284 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 11 00:34:16 smithi165 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-10[59927]: 2024-09-11T00:34:16.884+0000 7f2cff41d640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-11T00:33:32.747514+0000 front 2024-09-11T00:33:32.747491+0000 (oldest deadline 2024-09-11T00:33:58.047686+0000) 2024-09-11T00:34:17.284 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 11 00:34:16 smithi165 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-10[59927]: 2024-09-11T00:34:16.884+0000 7f2cff41d640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-11T00:33:41.548749+0000 front 2024-09-11T00:33:41.548701+0000 (oldest deadline 2024-09-11T00:34:05.648724+0000) 2024-09-11T00:34:17.284 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 11 00:34:16 smithi165 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-10[59927]: 2024-09-11T00:34:16.884+0000 7f2cff41d640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-09-11T00:33:50.850789+0000 front 2024-09-11T00:33:50.850826+0000 (oldest deadline 2024-09-11T00:34:11.950647+0000) 2024-09-11T00:34:17.563 INFO:journalctl@ceph.osd.9.smithi012.stdout:Sep 11 00:34:17 smithi012 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-9[66733]: 2024-09-11T00:34:17.196+0000 7f5f4e800640 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-09-11T00:33:25.406156+0000 front 2024-09-11T00:33:25.406095+0000 (oldest deadline 2024-09-11T00:33:47.706239+0000) 2024-09-11T00:34:17.563 INFO:journalctl@ceph.osd.9.smithi012.stdout:Sep 11 00:34:17 smithi012 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-9[66733]: 2024-09-11T00:34:17.196+0000 7f5f4e800640 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-11T00:33:30.607371+0000 front 2024-09-11T00:33:30.607499+0000 (oldest deadline 2024-09-11T00:33:55.307466+0000) 2024-09-11T00:34:17.563 INFO:journalctl@ceph.osd.9.smithi012.stdout:Sep 11 00:34:17 smithi012 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-9[66733]: 2024-09-11T00:34:17.196+0000 7f5f4e800640 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-11T00:33:41.710055+0000 front 2024-09-11T00:33:41.709869+0000 (oldest deadline 2024-09-11T00:34:07.010035+0000) 2024-09-11T00:34:17.563 INFO:journalctl@ceph.osd.9.smithi012.stdout:Sep 11 00:34:17 smithi012 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-9[66733]: 2024-09-11T00:34:17.196+0000 7f5f4e800640 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-09-11T00:33:47.010623+0000 front 2024-09-11T00:33:47.010590+0000 (oldest deadline 2024-09-11T00:34:12.910692+0000) 2024-09-11T00:34:17.564 INFO:journalctl@ceph.osd.6.smithi012.stdout:Sep 11 00:34:17 smithi012 podman[114606]: 2024-09-11 00:34:17.0879409 +0000 UTC m=+6.141401657 container cleanup f0cf9000529edb85619e4d1ee9f8b67c4d4ab6b75105fa182dc6bdc6843f398b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-6, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731) 2024-09-11T00:34:17.564 INFO:journalctl@ceph.osd.6.smithi012.stdout:Sep 11 00:34:17 smithi012 bash[114606]: ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-6 2024-09-11T00:34:17.564 INFO:journalctl@ceph.osd.6.smithi012.stdout:Sep 11 00:34:17 smithi012 podman[114628]: 2024-09-11 00:34:17.280997584 +0000 UTC m=+0.632100903 container remove f0cf9000529edb85619e4d1ee9f8b67c4d4ab6b75105fa182dc6bdc6843f398b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-6, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-09-11T00:34:17.581 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 00:34:17 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-11[60852]: 2024-09-11T00:34:17.359+0000 7fd754b0b640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-09-11T00:33:24.081060+0000 front 2024-09-11T00:33:24.081080+0000 (oldest deadline 2024-09-11T00:33:49.980779+0000) 2024-09-11T00:34:17.581 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 00:34:17 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-11[60852]: 2024-09-11T00:34:17.359+0000 7fd754b0b640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-11T00:33:41.682818+0000 front 2024-09-11T00:33:41.682848+0000 (oldest deadline 2024-09-11T00:34:05.182914+0000) 2024-09-11T00:34:17.581 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 00:34:17 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-11[60852]: 2024-09-11T00:34:17.359+0000 7fd754b0b640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-09-11T00:33:47.484088+0000 front 2024-09-11T00:33:47.484103+0000 (oldest deadline 2024-09-11T00:34:12.784060+0000) 2024-09-11T00:34:17.582 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 11 00:34:17 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-8[54042]: 2024-09-11T00:34:17.307+0000 7f23083d6640 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-11T00:33:33.673118+0000 front 2024-09-11T00:33:33.673042+0000 (oldest deadline 2024-09-11T00:33:56.573057+0000) 2024-09-11T00:34:17.582 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 11 00:34:17 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-8[54042]: 2024-09-11T00:34:17.307+0000 7f23083d6640 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-11T00:33:38.774830+0000 front 2024-09-11T00:33:38.774774+0000 (oldest deadline 2024-09-11T00:34:04.674919+0000) 2024-09-11T00:34:17.582 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 11 00:34:17 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-8[54042]: 2024-09-11T00:34:17.307+0000 7f23083d6640 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-09-11T00:33:47.977769+0000 front 2024-09-11T00:33:47.977619+0000 (oldest deadline 2024-09-11T00:34:13.277556+0000) 2024-09-11T00:34:18.063 INFO:journalctl@ceph.osd.6.smithi012.stdout:Sep 11 00:34:17 smithi012 podman[114722]: 2024-09-11 00:34:17.742397152 +0000 UTC m=+0.027094684 image pull 589dd18248e494ba411dc838aa4ae0e5e569213f08fcd7a2cf22cd3ef7e3c480 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e 2024-09-11T00:34:18.063 INFO:journalctl@ceph.osd.6.smithi012.stdout:Sep 11 00:34:17 smithi012 podman[114722]: 2024-09-11 00:34:17.865814103 +0000 UTC m=+0.150511624 container create 8ac038eb3ddd5373f53664d5de815448a72f357ad6928af0b3fc40ff84b1ca51 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-6-deactivate, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.license=GPLv2) 2024-09-11T00:34:18.283 INFO:journalctl@ceph.osd.7.smithi165.stdout:Sep 11 00:34:18 smithi165 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-7[52941]: 2024-09-11T00:34:18.021+0000 7ffbe0ef8640 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-11T00:33:32.707987+0000 front 2024-09-11T00:33:32.708022+0000 (oldest deadline 2024-09-11T00:33:56.808116+0000) 2024-09-11T00:34:18.283 INFO:journalctl@ceph.osd.7.smithi165.stdout:Sep 11 00:34:18 smithi165 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-7[52941]: 2024-09-11T00:34:18.021+0000 7ffbe0ef8640 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-11T00:33:41.509403+0000 front 2024-09-11T00:33:41.509317+0000 (oldest deadline 2024-09-11T00:34:03.809439+0000) 2024-09-11T00:34:18.283 INFO:journalctl@ceph.osd.7.smithi165.stdout:Sep 11 00:34:18 smithi165 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-7[52941]: 2024-09-11T00:34:18.021+0000 7ffbe0ef8640 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-09-11T00:33:50.712042+0000 front 2024-09-11T00:33:50.711963+0000 (oldest deadline 2024-09-11T00:34:16.011841+0000) 2024-09-11T00:34:18.283 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 11 00:34:17 smithi165 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-10[59927]: 2024-09-11T00:34:17.870+0000 7f2cff41d640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-09-11T00:33:25.146772+0000 front 2024-09-11T00:33:25.146620+0000 (oldest deadline 2024-09-11T00:33:49.846529+0000) 2024-09-11T00:34:18.283 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 11 00:34:17 smithi165 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-10[59927]: 2024-09-11T00:34:17.870+0000 7f2cff41d640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-11T00:33:32.747514+0000 front 2024-09-11T00:33:32.747491+0000 (oldest deadline 2024-09-11T00:33:58.047686+0000) 2024-09-11T00:34:18.283 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 11 00:34:17 smithi165 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-10[59927]: 2024-09-11T00:34:17.870+0000 7f2cff41d640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-11T00:33:41.548749+0000 front 2024-09-11T00:33:41.548701+0000 (oldest deadline 2024-09-11T00:34:05.648724+0000) 2024-09-11T00:34:18.284 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 11 00:34:17 smithi165 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-10[59927]: 2024-09-11T00:34:17.870+0000 7f2cff41d640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-09-11T00:33:50.850789+0000 front 2024-09-11T00:33:50.850826+0000 (oldest deadline 2024-09-11T00:34:11.950647+0000) 2024-09-11T00:34:18.377 INFO:journalctl@ceph.osd.6.smithi012.stdout:Sep 11 00:34:18 smithi012 podman[114722]: 2024-09-11 00:34:18.243199247 +0000 UTC m=+0.527896769 container init 8ac038eb3ddd5373f53664d5de815448a72f357ad6928af0b3fc40ff84b1ca51 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-6-deactivate, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.37.2, ceph=True) 2024-09-11T00:34:18.377 INFO:journalctl@ceph.osd.6.smithi012.stdout:Sep 11 00:34:18 smithi012 podman[114722]: 2024-09-11 00:34:18.249999701 +0000 UTC m=+0.534697224 container start 8ac038eb3ddd5373f53664d5de815448a72f357ad6928af0b3fc40ff84b1ca51 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-6-deactivate, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4) 2024-09-11T00:34:18.378 INFO:journalctl@ceph.osd.9.smithi012.stdout:Sep 11 00:34:18 smithi012 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-9[66733]: 2024-09-11T00:34:18.223+0000 7f5f4e800640 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-09-11T00:33:25.406156+0000 front 2024-09-11T00:33:25.406095+0000 (oldest deadline 2024-09-11T00:33:47.706239+0000) 2024-09-11T00:34:18.378 INFO:journalctl@ceph.osd.9.smithi012.stdout:Sep 11 00:34:18 smithi012 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-9[66733]: 2024-09-11T00:34:18.223+0000 7f5f4e800640 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-11T00:33:30.607371+0000 front 2024-09-11T00:33:30.607499+0000 (oldest deadline 2024-09-11T00:33:55.307466+0000) 2024-09-11T00:34:18.378 INFO:journalctl@ceph.osd.9.smithi012.stdout:Sep 11 00:34:18 smithi012 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-9[66733]: 2024-09-11T00:34:18.223+0000 7f5f4e800640 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-11T00:33:41.710055+0000 front 2024-09-11T00:33:41.709869+0000 (oldest deadline 2024-09-11T00:34:07.010035+0000) 2024-09-11T00:34:18.378 INFO:journalctl@ceph.osd.9.smithi012.stdout:Sep 11 00:34:18 smithi012 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-9[66733]: 2024-09-11T00:34:18.223+0000 7f5f4e800640 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-09-11T00:33:47.010623+0000 front 2024-09-11T00:33:47.010590+0000 (oldest deadline 2024-09-11T00:34:12.910692+0000) 2024-09-11T00:34:18.581 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 00:34:18 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-11[60852]: 2024-09-11T00:34:18.397+0000 7fd754b0b640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-09-11T00:33:24.081060+0000 front 2024-09-11T00:33:24.081080+0000 (oldest deadline 2024-09-11T00:33:49.980779+0000) 2024-09-11T00:34:18.582 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 00:34:18 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-11[60852]: 2024-09-11T00:34:18.397+0000 7fd754b0b640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-11T00:33:41.682818+0000 front 2024-09-11T00:33:41.682848+0000 (oldest deadline 2024-09-11T00:34:05.182914+0000) 2024-09-11T00:34:18.582 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 00:34:18 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-11[60852]: 2024-09-11T00:34:18.397+0000 7fd754b0b640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-09-11T00:33:47.484088+0000 front 2024-09-11T00:33:47.484103+0000 (oldest deadline 2024-09-11T00:34:12.784060+0000) 2024-09-11T00:34:18.582 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 11 00:34:18 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-8[54042]: 2024-09-11T00:34:18.265+0000 7f23083d6640 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-11T00:33:33.673118+0000 front 2024-09-11T00:33:33.673042+0000 (oldest deadline 2024-09-11T00:33:56.573057+0000) 2024-09-11T00:34:18.582 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 11 00:34:18 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-8[54042]: 2024-09-11T00:34:18.265+0000 7f23083d6640 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-11T00:33:38.774830+0000 front 2024-09-11T00:33:38.774774+0000 (oldest deadline 2024-09-11T00:34:04.674919+0000) 2024-09-11T00:34:18.582 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 11 00:34:18 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-8[54042]: 2024-09-11T00:34:18.265+0000 7f23083d6640 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-09-11T00:33:47.977769+0000 front 2024-09-11T00:33:47.977619+0000 (oldest deadline 2024-09-11T00:34:13.277556+0000) 2024-09-11T00:34:18.813 INFO:journalctl@ceph.osd.6.smithi012.stdout:Sep 11 00:34:18 smithi012 podman[114722]: 2024-09-11 00:34:18.376412449 +0000 UTC m=+0.661109971 container attach 8ac038eb3ddd5373f53664d5de815448a72f357ad6928af0b3fc40ff84b1ca51 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-6-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.vendor=CentOS, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731) 2024-09-11T00:34:18.814 INFO:journalctl@ceph.osd.6.smithi012.stdout:Sep 11 00:34:18 smithi012 podman[114722]: 2024-09-11 00:34:18.517589151 +0000 UTC m=+0.802286686 container died 8ac038eb3ddd5373f53664d5de815448a72f357ad6928af0b3fc40ff84b1ca51 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-6-deactivate, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.vendor=CentOS) 2024-09-11T00:34:19.148 DEBUG:teuthology.orchestra.run.smithi012:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968@osd.6.service' 2024-09-11T00:34:19.227 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T00:34:19.227 INFO:tasks.cephadm.osd.6:Stopped osd.6 2024-09-11T00:34:19.227 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2024-09-11T00:34:19.227 DEBUG:teuthology.orchestra.run.smithi165:> sudo systemctl stop ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968@osd.7 2024-09-11T00:34:19.255 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 11 00:34:18 smithi165 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-10[59927]: 2024-09-11T00:34:18.831+0000 7f2cff41d640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-09-11T00:33:25.146772+0000 front 2024-09-11T00:33:25.146620+0000 (oldest deadline 2024-09-11T00:33:49.846529+0000) 2024-09-11T00:34:19.255 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 11 00:34:18 smithi165 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-10[59927]: 2024-09-11T00:34:18.831+0000 7f2cff41d640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-11T00:33:32.747514+0000 front 2024-09-11T00:33:32.747491+0000 (oldest deadline 2024-09-11T00:33:58.047686+0000) 2024-09-11T00:34:19.255 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 11 00:34:18 smithi165 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-10[59927]: 2024-09-11T00:34:18.831+0000 7f2cff41d640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-11T00:33:41.548749+0000 front 2024-09-11T00:33:41.548701+0000 (oldest deadline 2024-09-11T00:34:05.648724+0000) 2024-09-11T00:34:19.255 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 11 00:34:18 smithi165 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-10[59927]: 2024-09-11T00:34:18.831+0000 7f2cff41d640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-09-11T00:33:50.850789+0000 front 2024-09-11T00:33:50.850826+0000 (oldest deadline 2024-09-11T00:34:11.950647+0000) 2024-09-11T00:34:19.256 INFO:journalctl@ceph.osd.7.smithi165.stdout:Sep 11 00:34:19 smithi165 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-7[52941]: 2024-09-11T00:34:19.010+0000 7ffbe0ef8640 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-11T00:33:32.707987+0000 front 2024-09-11T00:33:32.708022+0000 (oldest deadline 2024-09-11T00:33:56.808116+0000) 2024-09-11T00:34:19.256 INFO:journalctl@ceph.osd.7.smithi165.stdout:Sep 11 00:34:19 smithi165 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-7[52941]: 2024-09-11T00:34:19.010+0000 7ffbe0ef8640 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-11T00:33:41.509403+0000 front 2024-09-11T00:33:41.509317+0000 (oldest deadline 2024-09-11T00:34:03.809439+0000) 2024-09-11T00:34:19.256 INFO:journalctl@ceph.osd.7.smithi165.stdout:Sep 11 00:34:19 smithi165 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-7[52941]: 2024-09-11T00:34:19.010+0000 7ffbe0ef8640 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-09-11T00:33:50.712042+0000 front 2024-09-11T00:33:50.711963+0000 (oldest deadline 2024-09-11T00:34:16.011841+0000) 2024-09-11T00:34:19.531 INFO:journalctl@ceph.osd.7.smithi165.stdout:Sep 11 00:34:19 smithi165 systemd[1]: Stopping Ceph osd.7 for ff3ad74a-6fcf-11ef-bcea-c7b262605968... 2024-09-11T00:34:19.563 INFO:journalctl@ceph.osd.9.smithi012.stdout:Sep 11 00:34:19 smithi012 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-9[66733]: 2024-09-11T00:34:19.201+0000 7f5f4e800640 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-09-11T00:33:25.406156+0000 front 2024-09-11T00:33:25.406095+0000 (oldest deadline 2024-09-11T00:33:47.706239+0000) 2024-09-11T00:34:19.563 INFO:journalctl@ceph.osd.9.smithi012.stdout:Sep 11 00:34:19 smithi012 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-9[66733]: 2024-09-11T00:34:19.201+0000 7f5f4e800640 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-11T00:33:30.607371+0000 front 2024-09-11T00:33:30.607499+0000 (oldest deadline 2024-09-11T00:33:55.307466+0000) 2024-09-11T00:34:19.563 INFO:journalctl@ceph.osd.9.smithi012.stdout:Sep 11 00:34:19 smithi012 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-9[66733]: 2024-09-11T00:34:19.201+0000 7f5f4e800640 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-11T00:33:41.710055+0000 front 2024-09-11T00:33:41.709869+0000 (oldest deadline 2024-09-11T00:34:07.010035+0000) 2024-09-11T00:34:19.564 INFO:journalctl@ceph.osd.9.smithi012.stdout:Sep 11 00:34:19 smithi012 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-9[66733]: 2024-09-11T00:34:19.201+0000 7f5f4e800640 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-09-11T00:33:47.010623+0000 front 2024-09-11T00:33:47.010590+0000 (oldest deadline 2024-09-11T00:34:12.910692+0000) 2024-09-11T00:34:19.581 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 00:34:19 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-11[60852]: 2024-09-11T00:34:19.353+0000 7fd754b0b640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-09-11T00:33:24.081060+0000 front 2024-09-11T00:33:24.081080+0000 (oldest deadline 2024-09-11T00:33:49.980779+0000) 2024-09-11T00:34:19.581 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 00:34:19 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-11[60852]: 2024-09-11T00:34:19.353+0000 7fd754b0b640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-11T00:33:41.682818+0000 front 2024-09-11T00:33:41.682848+0000 (oldest deadline 2024-09-11T00:34:05.182914+0000) 2024-09-11T00:34:19.581 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 00:34:19 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-11[60852]: 2024-09-11T00:34:19.353+0000 7fd754b0b640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-09-11T00:33:47.484088+0000 front 2024-09-11T00:33:47.484103+0000 (oldest deadline 2024-09-11T00:34:12.784060+0000) 2024-09-11T00:34:19.581 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 11 00:34:19 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-8[54042]: 2024-09-11T00:34:19.285+0000 7f23083d6640 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-11T00:33:33.673118+0000 front 2024-09-11T00:33:33.673042+0000 (oldest deadline 2024-09-11T00:33:56.573057+0000) 2024-09-11T00:34:19.582 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 11 00:34:19 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-8[54042]: 2024-09-11T00:34:19.285+0000 7f23083d6640 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-11T00:33:38.774830+0000 front 2024-09-11T00:33:38.774774+0000 (oldest deadline 2024-09-11T00:34:04.674919+0000) 2024-09-11T00:34:19.582 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 11 00:34:19 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-8[54042]: 2024-09-11T00:34:19.285+0000 7f23083d6640 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-09-11T00:33:47.977769+0000 front 2024-09-11T00:33:47.977619+0000 (oldest deadline 2024-09-11T00:34:13.277556+0000) 2024-09-11T00:34:19.783 INFO:journalctl@ceph.osd.7.smithi165.stdout:Sep 11 00:34:19 smithi165 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-7[52941]: 2024-09-11T00:34:19.529+0000 7ffbe48e3640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T00:34:19.783 INFO:journalctl@ceph.osd.7.smithi165.stdout:Sep 11 00:34:19 smithi165 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-7[52941]: 2024-09-11T00:34:19.529+0000 7ffbe48e3640 -1 osd.7 91 *** Got signal Terminated *** 2024-09-11T00:34:19.783 INFO:journalctl@ceph.osd.7.smithi165.stdout:Sep 11 00:34:19 smithi165 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-7[52941]: 2024-09-11T00:34:19.529+0000 7ffbe48e3640 -1 osd.7 91 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-11T00:34:20.282 INFO:journalctl@ceph.osd.7.smithi165.stdout:Sep 11 00:34:20 smithi165 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-7[52941]: 2024-09-11T00:34:20.029+0000 7ffbe0ef8640 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-11T00:33:32.707987+0000 front 2024-09-11T00:33:32.708022+0000 (oldest deadline 2024-09-11T00:33:56.808116+0000) 2024-09-11T00:34:20.283 INFO:journalctl@ceph.osd.7.smithi165.stdout:Sep 11 00:34:20 smithi165 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-7[52941]: 2024-09-11T00:34:20.029+0000 7ffbe0ef8640 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-11T00:33:41.509403+0000 front 2024-09-11T00:33:41.509317+0000 (oldest deadline 2024-09-11T00:34:03.809439+0000) 2024-09-11T00:34:20.283 INFO:journalctl@ceph.osd.7.smithi165.stdout:Sep 11 00:34:20 smithi165 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-7[52941]: 2024-09-11T00:34:20.029+0000 7ffbe0ef8640 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-09-11T00:33:50.712042+0000 front 2024-09-11T00:33:50.711963+0000 (oldest deadline 2024-09-11T00:34:16.011841+0000) 2024-09-11T00:34:20.283 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 11 00:34:19 smithi165 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-10[59927]: 2024-09-11T00:34:19.862+0000 7f2cff41d640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-09-11T00:33:25.146772+0000 front 2024-09-11T00:33:25.146620+0000 (oldest deadline 2024-09-11T00:33:49.846529+0000) 2024-09-11T00:34:20.283 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 11 00:34:19 smithi165 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-10[59927]: 2024-09-11T00:34:19.862+0000 7f2cff41d640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-11T00:33:32.747514+0000 front 2024-09-11T00:33:32.747491+0000 (oldest deadline 2024-09-11T00:33:58.047686+0000) 2024-09-11T00:34:20.283 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 11 00:34:19 smithi165 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-10[59927]: 2024-09-11T00:34:19.862+0000 7f2cff41d640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-11T00:33:41.548749+0000 front 2024-09-11T00:33:41.548701+0000 (oldest deadline 2024-09-11T00:34:05.648724+0000) 2024-09-11T00:34:20.283 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 11 00:34:19 smithi165 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-10[59927]: 2024-09-11T00:34:19.862+0000 7f2cff41d640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-09-11T00:33:50.850789+0000 front 2024-09-11T00:33:50.850826+0000 (oldest deadline 2024-09-11T00:34:11.950647+0000) 2024-09-11T00:34:20.563 INFO:journalctl@ceph.osd.9.smithi012.stdout:Sep 11 00:34:20 smithi012 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-9[66733]: 2024-09-11T00:34:20.159+0000 7f5f4e800640 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-09-11T00:33:25.406156+0000 front 2024-09-11T00:33:25.406095+0000 (oldest deadline 2024-09-11T00:33:47.706239+0000) 2024-09-11T00:34:20.563 INFO:journalctl@ceph.osd.9.smithi012.stdout:Sep 11 00:34:20 smithi012 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-9[66733]: 2024-09-11T00:34:20.159+0000 7f5f4e800640 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-11T00:33:30.607371+0000 front 2024-09-11T00:33:30.607499+0000 (oldest deadline 2024-09-11T00:33:55.307466+0000) 2024-09-11T00:34:20.563 INFO:journalctl@ceph.osd.9.smithi012.stdout:Sep 11 00:34:20 smithi012 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-9[66733]: 2024-09-11T00:34:20.159+0000 7f5f4e800640 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-11T00:33:41.710055+0000 front 2024-09-11T00:33:41.709869+0000 (oldest deadline 2024-09-11T00:34:07.010035+0000) 2024-09-11T00:34:20.563 INFO:journalctl@ceph.osd.9.smithi012.stdout:Sep 11 00:34:20 smithi012 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-9[66733]: 2024-09-11T00:34:20.159+0000 7f5f4e800640 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-09-11T00:33:47.010623+0000 front 2024-09-11T00:33:47.010590+0000 (oldest deadline 2024-09-11T00:34:12.910692+0000) 2024-09-11T00:34:20.581 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 00:34:20 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-11[60852]: 2024-09-11T00:34:20.367+0000 7fd754b0b640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-09-11T00:33:24.081060+0000 front 2024-09-11T00:33:24.081080+0000 (oldest deadline 2024-09-11T00:33:49.980779+0000) 2024-09-11T00:34:20.581 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 00:34:20 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-11[60852]: 2024-09-11T00:34:20.367+0000 7fd754b0b640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-11T00:33:41.682818+0000 front 2024-09-11T00:33:41.682848+0000 (oldest deadline 2024-09-11T00:34:05.182914+0000) 2024-09-11T00:34:20.582 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 00:34:20 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-11[60852]: 2024-09-11T00:34:20.367+0000 7fd754b0b640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-09-11T00:33:47.484088+0000 front 2024-09-11T00:33:47.484103+0000 (oldest deadline 2024-09-11T00:34:12.784060+0000) 2024-09-11T00:34:20.582 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 11 00:34:20 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-8[54042]: 2024-09-11T00:34:20.280+0000 7f23083d6640 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-11T00:33:33.673118+0000 front 2024-09-11T00:33:33.673042+0000 (oldest deadline 2024-09-11T00:33:56.573057+0000) 2024-09-11T00:34:20.582 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 11 00:34:20 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-8[54042]: 2024-09-11T00:34:20.280+0000 7f23083d6640 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-11T00:33:38.774830+0000 front 2024-09-11T00:33:38.774774+0000 (oldest deadline 2024-09-11T00:34:04.674919+0000) 2024-09-11T00:34:20.582 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 11 00:34:20 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-8[54042]: 2024-09-11T00:34:20.280+0000 7f23083d6640 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-09-11T00:33:47.977769+0000 front 2024-09-11T00:33:47.977619+0000 (oldest deadline 2024-09-11T00:34:13.277556+0000) 2024-09-11T00:34:21.282 INFO:journalctl@ceph.osd.7.smithi165.stdout:Sep 11 00:34:21 smithi165 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-7[52941]: 2024-09-11T00:34:21.018+0000 7ffbe0ef8640 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-11T00:33:32.707987+0000 front 2024-09-11T00:33:32.708022+0000 (oldest deadline 2024-09-11T00:33:56.808116+0000) 2024-09-11T00:34:21.283 INFO:journalctl@ceph.osd.7.smithi165.stdout:Sep 11 00:34:21 smithi165 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-7[52941]: 2024-09-11T00:34:21.018+0000 7ffbe0ef8640 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-11T00:33:41.509403+0000 front 2024-09-11T00:33:41.509317+0000 (oldest deadline 2024-09-11T00:34:03.809439+0000) 2024-09-11T00:34:21.283 INFO:journalctl@ceph.osd.7.smithi165.stdout:Sep 11 00:34:21 smithi165 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-7[52941]: 2024-09-11T00:34:21.018+0000 7ffbe0ef8640 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-09-11T00:33:50.712042+0000 front 2024-09-11T00:33:50.711963+0000 (oldest deadline 2024-09-11T00:34:16.011841+0000) 2024-09-11T00:34:21.283 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 11 00:34:20 smithi165 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-10[59927]: 2024-09-11T00:34:20.875+0000 7f2cff41d640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-09-11T00:33:25.146772+0000 front 2024-09-11T00:33:25.146620+0000 (oldest deadline 2024-09-11T00:33:49.846529+0000) 2024-09-11T00:34:21.283 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 11 00:34:20 smithi165 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-10[59927]: 2024-09-11T00:34:20.875+0000 7f2cff41d640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-11T00:33:32.747514+0000 front 2024-09-11T00:33:32.747491+0000 (oldest deadline 2024-09-11T00:33:58.047686+0000) 2024-09-11T00:34:21.283 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 11 00:34:20 smithi165 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-10[59927]: 2024-09-11T00:34:20.875+0000 7f2cff41d640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-11T00:33:41.548749+0000 front 2024-09-11T00:33:41.548701+0000 (oldest deadline 2024-09-11T00:34:05.648724+0000) 2024-09-11T00:34:21.283 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 11 00:34:20 smithi165 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-10[59927]: 2024-09-11T00:34:20.875+0000 7f2cff41d640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-09-11T00:33:50.850789+0000 front 2024-09-11T00:33:50.850826+0000 (oldest deadline 2024-09-11T00:34:11.950647+0000) 2024-09-11T00:34:21.563 INFO:journalctl@ceph.osd.9.smithi012.stdout:Sep 11 00:34:21 smithi012 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-9[66733]: 2024-09-11T00:34:21.158+0000 7f5f4e800640 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-09-11T00:33:25.406156+0000 front 2024-09-11T00:33:25.406095+0000 (oldest deadline 2024-09-11T00:33:47.706239+0000) 2024-09-11T00:34:21.563 INFO:journalctl@ceph.osd.9.smithi012.stdout:Sep 11 00:34:21 smithi012 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-9[66733]: 2024-09-11T00:34:21.158+0000 7f5f4e800640 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-11T00:33:30.607371+0000 front 2024-09-11T00:33:30.607499+0000 (oldest deadline 2024-09-11T00:33:55.307466+0000) 2024-09-11T00:34:21.563 INFO:journalctl@ceph.osd.9.smithi012.stdout:Sep 11 00:34:21 smithi012 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-9[66733]: 2024-09-11T00:34:21.158+0000 7f5f4e800640 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-11T00:33:41.710055+0000 front 2024-09-11T00:33:41.709869+0000 (oldest deadline 2024-09-11T00:34:07.010035+0000) 2024-09-11T00:34:21.563 INFO:journalctl@ceph.osd.9.smithi012.stdout:Sep 11 00:34:21 smithi012 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-9[66733]: 2024-09-11T00:34:21.158+0000 7f5f4e800640 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-09-11T00:33:47.010623+0000 front 2024-09-11T00:33:47.010590+0000 (oldest deadline 2024-09-11T00:34:12.910692+0000) 2024-09-11T00:34:21.581 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 00:34:21 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-11[60852]: 2024-09-11T00:34:21.402+0000 7fd754b0b640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-09-11T00:33:24.081060+0000 front 2024-09-11T00:33:24.081080+0000 (oldest deadline 2024-09-11T00:33:49.980779+0000) 2024-09-11T00:34:21.581 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 00:34:21 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-11[60852]: 2024-09-11T00:34:21.402+0000 7fd754b0b640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-11T00:33:41.682818+0000 front 2024-09-11T00:33:41.682848+0000 (oldest deadline 2024-09-11T00:34:05.182914+0000) 2024-09-11T00:34:21.581 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 00:34:21 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-11[60852]: 2024-09-11T00:34:21.402+0000 7fd754b0b640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-09-11T00:33:47.484088+0000 front 2024-09-11T00:33:47.484103+0000 (oldest deadline 2024-09-11T00:34:12.784060+0000) 2024-09-11T00:34:21.581 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 11 00:34:21 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-8[54042]: 2024-09-11T00:34:21.275+0000 7f23083d6640 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-11T00:33:33.673118+0000 front 2024-09-11T00:33:33.673042+0000 (oldest deadline 2024-09-11T00:33:56.573057+0000) 2024-09-11T00:34:21.581 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 11 00:34:21 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-8[54042]: 2024-09-11T00:34:21.275+0000 7f23083d6640 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-11T00:33:38.774830+0000 front 2024-09-11T00:33:38.774774+0000 (oldest deadline 2024-09-11T00:34:04.674919+0000) 2024-09-11T00:34:21.582 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 11 00:34:21 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-8[54042]: 2024-09-11T00:34:21.275+0000 7f23083d6640 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-09-11T00:33:47.977769+0000 front 2024-09-11T00:33:47.977619+0000 (oldest deadline 2024-09-11T00:34:13.277556+0000) 2024-09-11T00:34:22.283 INFO:journalctl@ceph.osd.7.smithi165.stdout:Sep 11 00:34:21 smithi165 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-7[52941]: 2024-09-11T00:34:21.983+0000 7ffbe0ef8640 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-11T00:33:32.707987+0000 front 2024-09-11T00:33:32.708022+0000 (oldest deadline 2024-09-11T00:33:56.808116+0000) 2024-09-11T00:34:22.283 INFO:journalctl@ceph.osd.7.smithi165.stdout:Sep 11 00:34:21 smithi165 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-7[52941]: 2024-09-11T00:34:21.983+0000 7ffbe0ef8640 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-11T00:33:41.509403+0000 front 2024-09-11T00:33:41.509317+0000 (oldest deadline 2024-09-11T00:34:03.809439+0000) 2024-09-11T00:34:22.283 INFO:journalctl@ceph.osd.7.smithi165.stdout:Sep 11 00:34:21 smithi165 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-7[52941]: 2024-09-11T00:34:21.983+0000 7ffbe0ef8640 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-09-11T00:33:50.712042+0000 front 2024-09-11T00:33:50.711963+0000 (oldest deadline 2024-09-11T00:34:16.011841+0000) 2024-09-11T00:34:22.283 INFO:journalctl@ceph.osd.7.smithi165.stdout:Sep 11 00:34:21 smithi165 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-7[52941]: 2024-09-11T00:34:21.983+0000 7ffbe0ef8640 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.165:6812 osd.4 since back 2024-09-11T00:33:59.512755+0000 front 2024-09-11T00:33:59.512787+0000 (oldest deadline 2024-09-11T00:34:21.812906+0000) 2024-09-11T00:34:22.283 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 11 00:34:21 smithi165 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-10[59927]: 2024-09-11T00:34:21.900+0000 7f2cff41d640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-09-11T00:33:25.146772+0000 front 2024-09-11T00:33:25.146620+0000 (oldest deadline 2024-09-11T00:33:49.846529+0000) 2024-09-11T00:34:22.283 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 11 00:34:21 smithi165 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-10[59927]: 2024-09-11T00:34:21.900+0000 7f2cff41d640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-11T00:33:32.747514+0000 front 2024-09-11T00:33:32.747491+0000 (oldest deadline 2024-09-11T00:33:58.047686+0000) 2024-09-11T00:34:22.284 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 11 00:34:21 smithi165 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-10[59927]: 2024-09-11T00:34:21.900+0000 7f2cff41d640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-11T00:33:41.548749+0000 front 2024-09-11T00:33:41.548701+0000 (oldest deadline 2024-09-11T00:34:05.648724+0000) 2024-09-11T00:34:22.284 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 11 00:34:21 smithi165 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-10[59927]: 2024-09-11T00:34:21.900+0000 7f2cff41d640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-09-11T00:33:50.850789+0000 front 2024-09-11T00:33:50.850826+0000 (oldest deadline 2024-09-11T00:34:11.950647+0000) 2024-09-11T00:34:22.284 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 11 00:34:21 smithi165 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-10[59927]: 2024-09-11T00:34:21.900+0000 7f2cff41d640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.165:6812 osd.4 since back 2024-09-11T00:33:55.951809+0000 front 2024-09-11T00:33:55.951787+0000 (oldest deadline 2024-09-11T00:34:21.851850+0000) 2024-09-11T00:34:22.563 INFO:journalctl@ceph.osd.9.smithi012.stdout:Sep 11 00:34:22 smithi012 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-9[66733]: 2024-09-11T00:34:22.168+0000 7f5f4e800640 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-09-11T00:33:25.406156+0000 front 2024-09-11T00:33:25.406095+0000 (oldest deadline 2024-09-11T00:33:47.706239+0000) 2024-09-11T00:34:22.563 INFO:journalctl@ceph.osd.9.smithi012.stdout:Sep 11 00:34:22 smithi012 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-9[66733]: 2024-09-11T00:34:22.168+0000 7f5f4e800640 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-11T00:33:30.607371+0000 front 2024-09-11T00:33:30.607499+0000 (oldest deadline 2024-09-11T00:33:55.307466+0000) 2024-09-11T00:34:22.563 INFO:journalctl@ceph.osd.9.smithi012.stdout:Sep 11 00:34:22 smithi012 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-9[66733]: 2024-09-11T00:34:22.168+0000 7f5f4e800640 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-11T00:33:41.710055+0000 front 2024-09-11T00:33:41.709869+0000 (oldest deadline 2024-09-11T00:34:07.010035+0000) 2024-09-11T00:34:22.563 INFO:journalctl@ceph.osd.9.smithi012.stdout:Sep 11 00:34:22 smithi012 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-9[66733]: 2024-09-11T00:34:22.168+0000 7f5f4e800640 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-09-11T00:33:47.010623+0000 front 2024-09-11T00:33:47.010590+0000 (oldest deadline 2024-09-11T00:34:12.910692+0000) 2024-09-11T00:34:22.581 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 00:34:22 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-11[60852]: 2024-09-11T00:34:22.415+0000 7fd754b0b640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-09-11T00:33:24.081060+0000 front 2024-09-11T00:33:24.081080+0000 (oldest deadline 2024-09-11T00:33:49.980779+0000) 2024-09-11T00:34:22.581 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 00:34:22 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-11[60852]: 2024-09-11T00:34:22.415+0000 7fd754b0b640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-11T00:33:41.682818+0000 front 2024-09-11T00:33:41.682848+0000 (oldest deadline 2024-09-11T00:34:05.182914+0000) 2024-09-11T00:34:22.581 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 00:34:22 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-11[60852]: 2024-09-11T00:34:22.415+0000 7fd754b0b640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-09-11T00:33:47.484088+0000 front 2024-09-11T00:33:47.484103+0000 (oldest deadline 2024-09-11T00:34:12.784060+0000) 2024-09-11T00:34:22.581 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 11 00:34:22 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-8[54042]: 2024-09-11T00:34:22.231+0000 7f23083d6640 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-11T00:33:33.673118+0000 front 2024-09-11T00:33:33.673042+0000 (oldest deadline 2024-09-11T00:33:56.573057+0000) 2024-09-11T00:34:22.581 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 11 00:34:22 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-8[54042]: 2024-09-11T00:34:22.231+0000 7f23083d6640 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-11T00:33:38.774830+0000 front 2024-09-11T00:33:38.774774+0000 (oldest deadline 2024-09-11T00:34:04.674919+0000) 2024-09-11T00:34:22.581 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 11 00:34:22 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-8[54042]: 2024-09-11T00:34:22.231+0000 7f23083d6640 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-09-11T00:33:47.977769+0000 front 2024-09-11T00:33:47.977619+0000 (oldest deadline 2024-09-11T00:34:13.277556+0000) 2024-09-11T00:34:23.283 INFO:journalctl@ceph.osd.7.smithi165.stdout:Sep 11 00:34:22 smithi165 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-7[52941]: 2024-09-11T00:34:22.996+0000 7ffbe0ef8640 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-11T00:33:32.707987+0000 front 2024-09-11T00:33:32.708022+0000 (oldest deadline 2024-09-11T00:33:56.808116+0000) 2024-09-11T00:34:23.283 INFO:journalctl@ceph.osd.7.smithi165.stdout:Sep 11 00:34:22 smithi165 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-7[52941]: 2024-09-11T00:34:22.996+0000 7ffbe0ef8640 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-11T00:33:41.509403+0000 front 2024-09-11T00:33:41.509317+0000 (oldest deadline 2024-09-11T00:34:03.809439+0000) 2024-09-11T00:34:23.283 INFO:journalctl@ceph.osd.7.smithi165.stdout:Sep 11 00:34:22 smithi165 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-7[52941]: 2024-09-11T00:34:22.996+0000 7ffbe0ef8640 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-09-11T00:33:50.712042+0000 front 2024-09-11T00:33:50.711963+0000 (oldest deadline 2024-09-11T00:34:16.011841+0000) 2024-09-11T00:34:23.283 INFO:journalctl@ceph.osd.7.smithi165.stdout:Sep 11 00:34:22 smithi165 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-7[52941]: 2024-09-11T00:34:22.996+0000 7ffbe0ef8640 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.165:6812 osd.4 since back 2024-09-11T00:33:59.512755+0000 front 2024-09-11T00:33:59.512787+0000 (oldest deadline 2024-09-11T00:34:21.812906+0000) 2024-09-11T00:34:23.283 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 11 00:34:22 smithi165 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-10[59927]: 2024-09-11T00:34:22.941+0000 7f2cff41d640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-09-11T00:33:25.146772+0000 front 2024-09-11T00:33:25.146620+0000 (oldest deadline 2024-09-11T00:33:49.846529+0000) 2024-09-11T00:34:23.283 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 11 00:34:22 smithi165 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-10[59927]: 2024-09-11T00:34:22.941+0000 7f2cff41d640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-11T00:33:32.747514+0000 front 2024-09-11T00:33:32.747491+0000 (oldest deadline 2024-09-11T00:33:58.047686+0000) 2024-09-11T00:34:23.284 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 11 00:34:22 smithi165 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-10[59927]: 2024-09-11T00:34:22.941+0000 7f2cff41d640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-11T00:33:41.548749+0000 front 2024-09-11T00:33:41.548701+0000 (oldest deadline 2024-09-11T00:34:05.648724+0000) 2024-09-11T00:34:23.284 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 11 00:34:22 smithi165 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-10[59927]: 2024-09-11T00:34:22.941+0000 7f2cff41d640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-09-11T00:33:50.850789+0000 front 2024-09-11T00:33:50.850826+0000 (oldest deadline 2024-09-11T00:34:11.950647+0000) 2024-09-11T00:34:23.284 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 11 00:34:22 smithi165 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-10[59927]: 2024-09-11T00:34:22.941+0000 7f2cff41d640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.165:6812 osd.4 since back 2024-09-11T00:33:55.951809+0000 front 2024-09-11T00:33:55.951787+0000 (oldest deadline 2024-09-11T00:34:21.851850+0000) 2024-09-11T00:34:23.563 INFO:journalctl@ceph.osd.9.smithi012.stdout:Sep 11 00:34:23 smithi012 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-9[66733]: 2024-09-11T00:34:23.185+0000 7f5f4e800640 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-09-11T00:33:25.406156+0000 front 2024-09-11T00:33:25.406095+0000 (oldest deadline 2024-09-11T00:33:47.706239+0000) 2024-09-11T00:34:23.563 INFO:journalctl@ceph.osd.9.smithi012.stdout:Sep 11 00:34:23 smithi012 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-9[66733]: 2024-09-11T00:34:23.185+0000 7f5f4e800640 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-11T00:33:30.607371+0000 front 2024-09-11T00:33:30.607499+0000 (oldest deadline 2024-09-11T00:33:55.307466+0000) 2024-09-11T00:34:23.564 INFO:journalctl@ceph.osd.9.smithi012.stdout:Sep 11 00:34:23 smithi012 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-9[66733]: 2024-09-11T00:34:23.185+0000 7f5f4e800640 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-11T00:33:41.710055+0000 front 2024-09-11T00:33:41.709869+0000 (oldest deadline 2024-09-11T00:34:07.010035+0000) 2024-09-11T00:34:23.564 INFO:journalctl@ceph.osd.9.smithi012.stdout:Sep 11 00:34:23 smithi012 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-9[66733]: 2024-09-11T00:34:23.185+0000 7f5f4e800640 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-09-11T00:33:47.010623+0000 front 2024-09-11T00:33:47.010590+0000 (oldest deadline 2024-09-11T00:34:12.910692+0000) 2024-09-11T00:34:23.581 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 11 00:34:23 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-8[54042]: 2024-09-11T00:34:23.272+0000 7f23083d6640 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-11T00:33:33.673118+0000 front 2024-09-11T00:33:33.673042+0000 (oldest deadline 2024-09-11T00:33:56.573057+0000) 2024-09-11T00:34:23.581 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 11 00:34:23 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-8[54042]: 2024-09-11T00:34:23.272+0000 7f23083d6640 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-11T00:33:38.774830+0000 front 2024-09-11T00:33:38.774774+0000 (oldest deadline 2024-09-11T00:34:04.674919+0000) 2024-09-11T00:34:23.581 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 11 00:34:23 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-8[54042]: 2024-09-11T00:34:23.272+0000 7f23083d6640 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-09-11T00:33:47.977769+0000 front 2024-09-11T00:33:47.977619+0000 (oldest deadline 2024-09-11T00:34:13.277556+0000) 2024-09-11T00:34:23.581 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 00:34:23 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-11[60852]: 2024-09-11T00:34:23.392+0000 7fd754b0b640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-09-11T00:33:24.081060+0000 front 2024-09-11T00:33:24.081080+0000 (oldest deadline 2024-09-11T00:33:49.980779+0000) 2024-09-11T00:34:23.581 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 00:34:23 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-11[60852]: 2024-09-11T00:34:23.392+0000 7fd754b0b640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-11T00:33:41.682818+0000 front 2024-09-11T00:33:41.682848+0000 (oldest deadline 2024-09-11T00:34:05.182914+0000) 2024-09-11T00:34:23.582 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 00:34:23 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-11[60852]: 2024-09-11T00:34:23.392+0000 7fd754b0b640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-09-11T00:33:47.484088+0000 front 2024-09-11T00:33:47.484103+0000 (oldest deadline 2024-09-11T00:34:12.784060+0000) 2024-09-11T00:34:23.582 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 00:34:23 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-11[60852]: 2024-09-11T00:34:23.392+0000 7fd754b0b640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.165:6812 osd.4 since back 2024-09-11T00:33:58.685349+0000 front 2024-09-11T00:33:58.685336+0000 (oldest deadline 2024-09-11T00:34:23.385413+0000) 2024-09-11T00:34:24.283 INFO:journalctl@ceph.osd.7.smithi165.stdout:Sep 11 00:34:24 smithi165 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-7[52941]: 2024-09-11T00:34:24.036+0000 7ffbe0ef8640 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-11T00:33:32.707987+0000 front 2024-09-11T00:33:32.708022+0000 (oldest deadline 2024-09-11T00:33:56.808116+0000) 2024-09-11T00:34:24.283 INFO:journalctl@ceph.osd.7.smithi165.stdout:Sep 11 00:34:24 smithi165 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-7[52941]: 2024-09-11T00:34:24.036+0000 7ffbe0ef8640 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-11T00:33:41.509403+0000 front 2024-09-11T00:33:41.509317+0000 (oldest deadline 2024-09-11T00:34:03.809439+0000) 2024-09-11T00:34:24.283 INFO:journalctl@ceph.osd.7.smithi165.stdout:Sep 11 00:34:24 smithi165 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-7[52941]: 2024-09-11T00:34:24.036+0000 7ffbe0ef8640 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-09-11T00:33:50.712042+0000 front 2024-09-11T00:33:50.711963+0000 (oldest deadline 2024-09-11T00:34:16.011841+0000) 2024-09-11T00:34:24.283 INFO:journalctl@ceph.osd.7.smithi165.stdout:Sep 11 00:34:24 smithi165 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-7[52941]: 2024-09-11T00:34:24.036+0000 7ffbe0ef8640 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.165:6812 osd.4 since back 2024-09-11T00:33:59.512755+0000 front 2024-09-11T00:33:59.512787+0000 (oldest deadline 2024-09-11T00:34:21.812906+0000) 2024-09-11T00:34:24.284 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 11 00:34:23 smithi165 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-10[59927]: 2024-09-11T00:34:23.955+0000 7f2cff41d640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-09-11T00:33:25.146772+0000 front 2024-09-11T00:33:25.146620+0000 (oldest deadline 2024-09-11T00:33:49.846529+0000) 2024-09-11T00:34:24.284 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 11 00:34:23 smithi165 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-10[59927]: 2024-09-11T00:34:23.955+0000 7f2cff41d640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-11T00:33:32.747514+0000 front 2024-09-11T00:33:32.747491+0000 (oldest deadline 2024-09-11T00:33:58.047686+0000) 2024-09-11T00:34:24.284 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 11 00:34:23 smithi165 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-10[59927]: 2024-09-11T00:34:23.955+0000 7f2cff41d640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-11T00:33:41.548749+0000 front 2024-09-11T00:33:41.548701+0000 (oldest deadline 2024-09-11T00:34:05.648724+0000) 2024-09-11T00:34:24.284 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 11 00:34:23 smithi165 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-10[59927]: 2024-09-11T00:34:23.955+0000 7f2cff41d640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-09-11T00:33:50.850789+0000 front 2024-09-11T00:33:50.850826+0000 (oldest deadline 2024-09-11T00:34:11.950647+0000) 2024-09-11T00:34:24.284 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 11 00:34:23 smithi165 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-10[59927]: 2024-09-11T00:34:23.955+0000 7f2cff41d640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.165:6812 osd.4 since back 2024-09-11T00:33:55.951809+0000 front 2024-09-11T00:33:55.951787+0000 (oldest deadline 2024-09-11T00:34:21.851850+0000) 2024-09-11T00:34:24.563 INFO:journalctl@ceph.osd.9.smithi012.stdout:Sep 11 00:34:24 smithi012 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-9[66733]: 2024-09-11T00:34:24.191+0000 7f5f4e800640 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-09-11T00:33:25.406156+0000 front 2024-09-11T00:33:25.406095+0000 (oldest deadline 2024-09-11T00:33:47.706239+0000) 2024-09-11T00:34:24.563 INFO:journalctl@ceph.osd.9.smithi012.stdout:Sep 11 00:34:24 smithi012 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-9[66733]: 2024-09-11T00:34:24.191+0000 7f5f4e800640 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-11T00:33:30.607371+0000 front 2024-09-11T00:33:30.607499+0000 (oldest deadline 2024-09-11T00:33:55.307466+0000) 2024-09-11T00:34:24.563 INFO:journalctl@ceph.osd.9.smithi012.stdout:Sep 11 00:34:24 smithi012 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-9[66733]: 2024-09-11T00:34:24.191+0000 7f5f4e800640 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-11T00:33:41.710055+0000 front 2024-09-11T00:33:41.709869+0000 (oldest deadline 2024-09-11T00:34:07.010035+0000) 2024-09-11T00:34:24.563 INFO:journalctl@ceph.osd.9.smithi012.stdout:Sep 11 00:34:24 smithi012 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-9[66733]: 2024-09-11T00:34:24.191+0000 7f5f4e800640 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-09-11T00:33:47.010623+0000 front 2024-09-11T00:33:47.010590+0000 (oldest deadline 2024-09-11T00:34:12.910692+0000) 2024-09-11T00:34:24.581 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 11 00:34:24 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-8[54042]: 2024-09-11T00:34:24.226+0000 7f23083d6640 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-11T00:33:33.673118+0000 front 2024-09-11T00:33:33.673042+0000 (oldest deadline 2024-09-11T00:33:56.573057+0000) 2024-09-11T00:34:24.581 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 11 00:34:24 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-8[54042]: 2024-09-11T00:34:24.226+0000 7f23083d6640 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-11T00:33:38.774830+0000 front 2024-09-11T00:33:38.774774+0000 (oldest deadline 2024-09-11T00:34:04.674919+0000) 2024-09-11T00:34:24.581 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 11 00:34:24 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-8[54042]: 2024-09-11T00:34:24.226+0000 7f23083d6640 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-09-11T00:33:47.977769+0000 front 2024-09-11T00:33:47.977619+0000 (oldest deadline 2024-09-11T00:34:13.277556+0000) 2024-09-11T00:34:24.581 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 00:34:24 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-11[60852]: 2024-09-11T00:34:24.427+0000 7fd754b0b640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-09-11T00:33:24.081060+0000 front 2024-09-11T00:33:24.081080+0000 (oldest deadline 2024-09-11T00:33:49.980779+0000) 2024-09-11T00:34:24.581 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 00:34:24 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-11[60852]: 2024-09-11T00:34:24.427+0000 7fd754b0b640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-11T00:33:41.682818+0000 front 2024-09-11T00:33:41.682848+0000 (oldest deadline 2024-09-11T00:34:05.182914+0000) 2024-09-11T00:34:24.582 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 00:34:24 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-11[60852]: 2024-09-11T00:34:24.427+0000 7fd754b0b640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-09-11T00:33:47.484088+0000 front 2024-09-11T00:33:47.484103+0000 (oldest deadline 2024-09-11T00:34:12.784060+0000) 2024-09-11T00:34:24.582 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 00:34:24 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-11[60852]: 2024-09-11T00:34:24.427+0000 7fd754b0b640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.165:6812 osd.4 since back 2024-09-11T00:33:58.685349+0000 front 2024-09-11T00:33:58.685336+0000 (oldest deadline 2024-09-11T00:34:23.385413+0000) 2024-09-11T00:34:25.179 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 11 00:34:24 smithi165 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-10[59927]: 2024-09-11T00:34:24.921+0000 7f2cff41d640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-09-11T00:33:25.146772+0000 front 2024-09-11T00:33:25.146620+0000 (oldest deadline 2024-09-11T00:33:49.846529+0000) 2024-09-11T00:34:25.179 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 11 00:34:24 smithi165 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-10[59927]: 2024-09-11T00:34:24.921+0000 7f2cff41d640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-11T00:33:32.747514+0000 front 2024-09-11T00:33:32.747491+0000 (oldest deadline 2024-09-11T00:33:58.047686+0000) 2024-09-11T00:34:25.180 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 11 00:34:24 smithi165 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-10[59927]: 2024-09-11T00:34:24.921+0000 7f2cff41d640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-11T00:33:41.548749+0000 front 2024-09-11T00:33:41.548701+0000 (oldest deadline 2024-09-11T00:34:05.648724+0000) 2024-09-11T00:34:25.180 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 11 00:34:24 smithi165 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-10[59927]: 2024-09-11T00:34:24.921+0000 7f2cff41d640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-09-11T00:33:50.850789+0000 front 2024-09-11T00:33:50.850826+0000 (oldest deadline 2024-09-11T00:34:11.950647+0000) 2024-09-11T00:34:25.180 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 11 00:34:24 smithi165 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-10[59927]: 2024-09-11T00:34:24.921+0000 7f2cff41d640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.165:6812 osd.4 since back 2024-09-11T00:33:55.951809+0000 front 2024-09-11T00:33:55.951787+0000 (oldest deadline 2024-09-11T00:34:21.851850+0000) 2024-09-11T00:34:25.441 INFO:journalctl@ceph.osd.7.smithi165.stdout:Sep 11 00:34:25 smithi165 podman[76352]: 2024-09-11 00:34:25.178328819 +0000 UTC m=+5.741535377 container died 8e149d1cf9ebaf976a8c7b6c3cd8259fc5fba1a03f0624b5df2c7f09ca1996ef (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-7, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, GIT_CLEAN=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , ceph=True) 2024-09-11T00:34:25.497 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 11 00:34:25 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-8[54042]: 2024-09-11T00:34:25.234+0000 7f23083d6640 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-11T00:33:33.673118+0000 front 2024-09-11T00:33:33.673042+0000 (oldest deadline 2024-09-11T00:33:56.573057+0000) 2024-09-11T00:34:25.497 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 11 00:34:25 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-8[54042]: 2024-09-11T00:34:25.234+0000 7f23083d6640 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-11T00:33:38.774830+0000 front 2024-09-11T00:33:38.774774+0000 (oldest deadline 2024-09-11T00:34:04.674919+0000) 2024-09-11T00:34:25.497 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 11 00:34:25 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-8[54042]: 2024-09-11T00:34:25.234+0000 7f23083d6640 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-09-11T00:33:47.977769+0000 front 2024-09-11T00:33:47.977619+0000 (oldest deadline 2024-09-11T00:34:13.277556+0000) 2024-09-11T00:34:25.497 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 11 00:34:25 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-8[54042]: 2024-09-11T00:34:25.234+0000 7f23083d6640 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.165:6812 osd.4 since back 2024-09-11T00:33:58.578956+0000 front 2024-09-11T00:33:58.579029+0000 (oldest deadline 2024-09-11T00:34:24.979373+0000) 2024-09-11T00:34:25.497 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 00:34:25 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-11[60852]: 2024-09-11T00:34:25.464+0000 7fd754b0b640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-09-11T00:33:24.081060+0000 front 2024-09-11T00:33:24.081080+0000 (oldest deadline 2024-09-11T00:33:49.980779+0000) 2024-09-11T00:34:25.497 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 00:34:25 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-11[60852]: 2024-09-11T00:34:25.464+0000 7fd754b0b640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-11T00:33:41.682818+0000 front 2024-09-11T00:33:41.682848+0000 (oldest deadline 2024-09-11T00:34:05.182914+0000) 2024-09-11T00:34:25.498 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 00:34:25 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-11[60852]: 2024-09-11T00:34:25.464+0000 7fd754b0b640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-09-11T00:33:47.484088+0000 front 2024-09-11T00:33:47.484103+0000 (oldest deadline 2024-09-11T00:34:12.784060+0000) 2024-09-11T00:34:25.498 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 00:34:25 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-11[60852]: 2024-09-11T00:34:25.464+0000 7fd754b0b640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.165:6812 osd.4 since back 2024-09-11T00:33:58.685349+0000 front 2024-09-11T00:33:58.685336+0000 (oldest deadline 2024-09-11T00:34:23.385413+0000) 2024-09-11T00:34:25.563 INFO:journalctl@ceph.osd.9.smithi012.stdout:Sep 11 00:34:25 smithi012 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-9[66733]: 2024-09-11T00:34:25.206+0000 7f5f4e800640 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-09-11T00:33:25.406156+0000 front 2024-09-11T00:33:25.406095+0000 (oldest deadline 2024-09-11T00:33:47.706239+0000) 2024-09-11T00:34:25.563 INFO:journalctl@ceph.osd.9.smithi012.stdout:Sep 11 00:34:25 smithi012 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-9[66733]: 2024-09-11T00:34:25.206+0000 7f5f4e800640 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-11T00:33:30.607371+0000 front 2024-09-11T00:33:30.607499+0000 (oldest deadline 2024-09-11T00:33:55.307466+0000) 2024-09-11T00:34:25.563 INFO:journalctl@ceph.osd.9.smithi012.stdout:Sep 11 00:34:25 smithi012 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-9[66733]: 2024-09-11T00:34:25.206+0000 7f5f4e800640 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-11T00:33:41.710055+0000 front 2024-09-11T00:33:41.709869+0000 (oldest deadline 2024-09-11T00:34:07.010035+0000) 2024-09-11T00:34:25.564 INFO:journalctl@ceph.osd.9.smithi012.stdout:Sep 11 00:34:25 smithi012 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-9[66733]: 2024-09-11T00:34:25.206+0000 7f5f4e800640 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-09-11T00:33:47.010623+0000 front 2024-09-11T00:33:47.010590+0000 (oldest deadline 2024-09-11T00:34:12.910692+0000) 2024-09-11T00:34:25.564 INFO:journalctl@ceph.osd.9.smithi012.stdout:Sep 11 00:34:25 smithi012 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-9[66733]: 2024-09-11T00:34:25.206+0000 7f5f4e800640 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.165:6812 osd.4 since back 2024-09-11T00:33:59.213236+0000 front 2024-09-11T00:33:59.213312+0000 (oldest deadline 2024-09-11T00:34:24.513408+0000) 2024-09-11T00:34:25.783 INFO:journalctl@ceph.osd.7.smithi165.stdout:Sep 11 00:34:25 smithi165 podman[76352]: 2024-09-11 00:34:25.571836233 +0000 UTC m=+6.135042795 container cleanup 8e149d1cf9ebaf976a8c7b6c3cd8259fc5fba1a03f0624b5df2c7f09ca1996ef (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-7, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, GIT_BRANCH=HEAD, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4) 2024-09-11T00:34:25.783 INFO:journalctl@ceph.osd.7.smithi165.stdout:Sep 11 00:34:25 smithi165 bash[76352]: ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-7 2024-09-11T00:34:26.208 INFO:journalctl@ceph.osd.7.smithi165.stdout:Sep 11 00:34:25 smithi165 podman[76368]: 2024-09-11 00:34:25.823508486 +0000 UTC m=+0.645027906 container remove 8e149d1cf9ebaf976a8c7b6c3cd8259fc5fba1a03f0624b5df2c7f09ca1996ef (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-7, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.37.2, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0) 2024-09-11T00:34:26.208 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 11 00:34:25 smithi165 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-10[59927]: 2024-09-11T00:34:25.937+0000 7f2cff41d640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-09-11T00:33:25.146772+0000 front 2024-09-11T00:33:25.146620+0000 (oldest deadline 2024-09-11T00:33:49.846529+0000) 2024-09-11T00:34:26.209 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 11 00:34:25 smithi165 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-10[59927]: 2024-09-11T00:34:25.937+0000 7f2cff41d640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-11T00:33:32.747514+0000 front 2024-09-11T00:33:32.747491+0000 (oldest deadline 2024-09-11T00:33:58.047686+0000) 2024-09-11T00:34:26.209 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 11 00:34:25 smithi165 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-10[59927]: 2024-09-11T00:34:25.937+0000 7f2cff41d640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-11T00:33:41.548749+0000 front 2024-09-11T00:33:41.548701+0000 (oldest deadline 2024-09-11T00:34:05.648724+0000) 2024-09-11T00:34:26.209 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 11 00:34:25 smithi165 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-10[59927]: 2024-09-11T00:34:25.937+0000 7f2cff41d640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-09-11T00:33:50.850789+0000 front 2024-09-11T00:33:50.850826+0000 (oldest deadline 2024-09-11T00:34:11.950647+0000) 2024-09-11T00:34:26.209 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 11 00:34:25 smithi165 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-10[59927]: 2024-09-11T00:34:25.937+0000 7f2cff41d640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.165:6812 osd.4 since back 2024-09-11T00:33:55.951809+0000 front 2024-09-11T00:33:55.951787+0000 (oldest deadline 2024-09-11T00:34:21.851850+0000) 2024-09-11T00:34:26.494 INFO:journalctl@ceph.osd.7.smithi165.stdout:Sep 11 00:34:26 smithi165 podman[76466]: 2024-09-11 00:34:26.207335928 +0000 UTC m=+0.114781131 container create 9a5b259dead9a33ee381a155c2665d75cf5121039ac49d4433847d65a9e414d9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-7-deactivate, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-11T00:34:26.494 INFO:journalctl@ceph.osd.7.smithi165.stdout:Sep 11 00:34:26 smithi165 podman[76466]: 2024-09-11 00:34:26.114871907 +0000 UTC m=+0.022317114 image pull 589dd18248e494ba411dc838aa4ae0e5e569213f08fcd7a2cf22cd3ef7e3c480 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e 2024-09-11T00:34:26.563 INFO:journalctl@ceph.osd.9.smithi012.stdout:Sep 11 00:34:26 smithi012 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-9[66733]: 2024-09-11T00:34:26.203+0000 7f5f4e800640 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-09-11T00:33:25.406156+0000 front 2024-09-11T00:33:25.406095+0000 (oldest deadline 2024-09-11T00:33:47.706239+0000) 2024-09-11T00:34:26.563 INFO:journalctl@ceph.osd.9.smithi012.stdout:Sep 11 00:34:26 smithi012 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-9[66733]: 2024-09-11T00:34:26.203+0000 7f5f4e800640 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-11T00:33:30.607371+0000 front 2024-09-11T00:33:30.607499+0000 (oldest deadline 2024-09-11T00:33:55.307466+0000) 2024-09-11T00:34:26.563 INFO:journalctl@ceph.osd.9.smithi012.stdout:Sep 11 00:34:26 smithi012 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-9[66733]: 2024-09-11T00:34:26.203+0000 7f5f4e800640 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-11T00:33:41.710055+0000 front 2024-09-11T00:33:41.709869+0000 (oldest deadline 2024-09-11T00:34:07.010035+0000) 2024-09-11T00:34:26.563 INFO:journalctl@ceph.osd.9.smithi012.stdout:Sep 11 00:34:26 smithi012 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-9[66733]: 2024-09-11T00:34:26.203+0000 7f5f4e800640 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-09-11T00:33:47.010623+0000 front 2024-09-11T00:33:47.010590+0000 (oldest deadline 2024-09-11T00:34:12.910692+0000) 2024-09-11T00:34:26.563 INFO:journalctl@ceph.osd.9.smithi012.stdout:Sep 11 00:34:26 smithi012 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-9[66733]: 2024-09-11T00:34:26.203+0000 7f5f4e800640 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.165:6812 osd.4 since back 2024-09-11T00:33:59.213236+0000 front 2024-09-11T00:33:59.213312+0000 (oldest deadline 2024-09-11T00:34:24.513408+0000) 2024-09-11T00:34:26.581 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 11 00:34:26 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-8[54042]: 2024-09-11T00:34:26.239+0000 7f23083d6640 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-11T00:33:33.673118+0000 front 2024-09-11T00:33:33.673042+0000 (oldest deadline 2024-09-11T00:33:56.573057+0000) 2024-09-11T00:34:26.581 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 11 00:34:26 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-8[54042]: 2024-09-11T00:34:26.239+0000 7f23083d6640 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-11T00:33:38.774830+0000 front 2024-09-11T00:33:38.774774+0000 (oldest deadline 2024-09-11T00:34:04.674919+0000) 2024-09-11T00:34:26.581 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 11 00:34:26 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-8[54042]: 2024-09-11T00:34:26.239+0000 7f23083d6640 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-09-11T00:33:47.977769+0000 front 2024-09-11T00:33:47.977619+0000 (oldest deadline 2024-09-11T00:34:13.277556+0000) 2024-09-11T00:34:26.581 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 11 00:34:26 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-8[54042]: 2024-09-11T00:34:26.239+0000 7f23083d6640 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.165:6812 osd.4 since back 2024-09-11T00:33:58.578956+0000 front 2024-09-11T00:33:58.579029+0000 (oldest deadline 2024-09-11T00:34:24.979373+0000) 2024-09-11T00:34:26.581 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 00:34:26 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-11[60852]: 2024-09-11T00:34:26.478+0000 7fd754b0b640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-09-11T00:33:24.081060+0000 front 2024-09-11T00:33:24.081080+0000 (oldest deadline 2024-09-11T00:33:49.980779+0000) 2024-09-11T00:34:26.582 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 00:34:26 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-11[60852]: 2024-09-11T00:34:26.478+0000 7fd754b0b640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-11T00:33:41.682818+0000 front 2024-09-11T00:33:41.682848+0000 (oldest deadline 2024-09-11T00:34:05.182914+0000) 2024-09-11T00:34:26.582 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 00:34:26 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-11[60852]: 2024-09-11T00:34:26.478+0000 7fd754b0b640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-09-11T00:33:47.484088+0000 front 2024-09-11T00:33:47.484103+0000 (oldest deadline 2024-09-11T00:34:12.784060+0000) 2024-09-11T00:34:26.582 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 00:34:26 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-11[60852]: 2024-09-11T00:34:26.478+0000 7fd754b0b640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.165:6812 osd.4 since back 2024-09-11T00:33:58.685349+0000 front 2024-09-11T00:33:58.685336+0000 (oldest deadline 2024-09-11T00:34:23.385413+0000) 2024-09-11T00:34:26.783 INFO:journalctl@ceph.osd.7.smithi165.stdout:Sep 11 00:34:26 smithi165 podman[76466]: 2024-09-11 00:34:26.493362654 +0000 UTC m=+0.400807859 container init 9a5b259dead9a33ee381a155c2665d75cf5121039ac49d4433847d65a9e414d9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-7-deactivate, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-11T00:34:26.783 INFO:journalctl@ceph.osd.7.smithi165.stdout:Sep 11 00:34:26 smithi165 podman[76466]: 2024-09-11 00:34:26.496745545 +0000 UTC m=+0.404190745 container start 9a5b259dead9a33ee381a155c2665d75cf5121039ac49d4433847d65a9e414d9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-7-deactivate, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-11T00:34:26.783 INFO:journalctl@ceph.osd.7.smithi165.stdout:Sep 11 00:34:26 smithi165 podman[76466]: 2024-09-11 00:34:26.593568672 +0000 UTC m=+0.501013879 container attach 9a5b259dead9a33ee381a155c2665d75cf5121039ac49d4433847d65a9e414d9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-7-deactivate, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-09-11T00:34:26.783 INFO:journalctl@ceph.osd.7.smithi165.stdout:Sep 11 00:34:26 smithi165 podman[76466]: 2024-09-11 00:34:26.695843753 +0000 UTC m=+0.603288950 container died 9a5b259dead9a33ee381a155c2665d75cf5121039ac49d4433847d65a9e414d9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-7-deactivate, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=) 2024-09-11T00:34:27.283 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 11 00:34:26 smithi165 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-10[59927]: 2024-09-11T00:34:26.938+0000 7f2cff41d640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-09-11T00:33:25.146772+0000 front 2024-09-11T00:33:25.146620+0000 (oldest deadline 2024-09-11T00:33:49.846529+0000) 2024-09-11T00:34:27.283 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 11 00:34:26 smithi165 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-10[59927]: 2024-09-11T00:34:26.938+0000 7f2cff41d640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-11T00:33:32.747514+0000 front 2024-09-11T00:33:32.747491+0000 (oldest deadline 2024-09-11T00:33:58.047686+0000) 2024-09-11T00:34:27.283 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 11 00:34:26 smithi165 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-10[59927]: 2024-09-11T00:34:26.938+0000 7f2cff41d640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-11T00:33:41.548749+0000 front 2024-09-11T00:33:41.548701+0000 (oldest deadline 2024-09-11T00:34:05.648724+0000) 2024-09-11T00:34:27.283 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 11 00:34:26 smithi165 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-10[59927]: 2024-09-11T00:34:26.938+0000 7f2cff41d640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-09-11T00:33:50.850789+0000 front 2024-09-11T00:33:50.850826+0000 (oldest deadline 2024-09-11T00:34:11.950647+0000) 2024-09-11T00:34:27.283 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 11 00:34:26 smithi165 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-10[59927]: 2024-09-11T00:34:26.938+0000 7f2cff41d640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.165:6812 osd.4 since back 2024-09-11T00:33:55.951809+0000 front 2024-09-11T00:33:55.951787+0000 (oldest deadline 2024-09-11T00:34:21.851850+0000) 2024-09-11T00:34:27.380 DEBUG:teuthology.orchestra.run.smithi165:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968@osd.7.service' 2024-09-11T00:34:27.463 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T00:34:27.463 INFO:tasks.cephadm.osd.7:Stopped osd.7 2024-09-11T00:34:27.463 INFO:tasks.cephadm.osd.8:Stopping osd.8... 2024-09-11T00:34:27.463 DEBUG:teuthology.orchestra.run.smithi178:> sudo systemctl stop ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968@osd.8 2024-09-11T00:34:27.489 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 11 00:34:27 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-8[54042]: 2024-09-11T00:34:27.209+0000 7f23083d6640 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-11T00:33:33.673118+0000 front 2024-09-11T00:33:33.673042+0000 (oldest deadline 2024-09-11T00:33:56.573057+0000) 2024-09-11T00:34:27.489 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 11 00:34:27 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-8[54042]: 2024-09-11T00:34:27.209+0000 7f23083d6640 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-11T00:33:38.774830+0000 front 2024-09-11T00:33:38.774774+0000 (oldest deadline 2024-09-11T00:34:04.674919+0000) 2024-09-11T00:34:27.489 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 11 00:34:27 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-8[54042]: 2024-09-11T00:34:27.209+0000 7f23083d6640 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-09-11T00:33:47.977769+0000 front 2024-09-11T00:33:47.977619+0000 (oldest deadline 2024-09-11T00:34:13.277556+0000) 2024-09-11T00:34:27.489 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 11 00:34:27 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-8[54042]: 2024-09-11T00:34:27.209+0000 7f23083d6640 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.165:6812 osd.4 since back 2024-09-11T00:33:58.578956+0000 front 2024-09-11T00:33:58.579029+0000 (oldest deadline 2024-09-11T00:34:24.979373+0000) 2024-09-11T00:34:27.490 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 00:34:27 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-11[60852]: 2024-09-11T00:34:27.435+0000 7fd754b0b640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-09-11T00:33:24.081060+0000 front 2024-09-11T00:33:24.081080+0000 (oldest deadline 2024-09-11T00:33:49.980779+0000) 2024-09-11T00:34:27.490 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 00:34:27 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-11[60852]: 2024-09-11T00:34:27.435+0000 7fd754b0b640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-11T00:33:41.682818+0000 front 2024-09-11T00:33:41.682848+0000 (oldest deadline 2024-09-11T00:34:05.182914+0000) 2024-09-11T00:34:27.490 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 00:34:27 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-11[60852]: 2024-09-11T00:34:27.435+0000 7fd754b0b640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-09-11T00:33:47.484088+0000 front 2024-09-11T00:33:47.484103+0000 (oldest deadline 2024-09-11T00:34:12.784060+0000) 2024-09-11T00:34:27.490 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 00:34:27 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-11[60852]: 2024-09-11T00:34:27.435+0000 7fd754b0b640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.165:6812 osd.4 since back 2024-09-11T00:33:58.685349+0000 front 2024-09-11T00:33:58.685336+0000 (oldest deadline 2024-09-11T00:34:23.385413+0000) 2024-09-11T00:34:27.563 INFO:journalctl@ceph.osd.9.smithi012.stdout:Sep 11 00:34:27 smithi012 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-9[66733]: 2024-09-11T00:34:27.183+0000 7f5f4e800640 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-09-11T00:33:25.406156+0000 front 2024-09-11T00:33:25.406095+0000 (oldest deadline 2024-09-11T00:33:47.706239+0000) 2024-09-11T00:34:27.563 INFO:journalctl@ceph.osd.9.smithi012.stdout:Sep 11 00:34:27 smithi012 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-9[66733]: 2024-09-11T00:34:27.183+0000 7f5f4e800640 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-11T00:33:30.607371+0000 front 2024-09-11T00:33:30.607499+0000 (oldest deadline 2024-09-11T00:33:55.307466+0000) 2024-09-11T00:34:27.563 INFO:journalctl@ceph.osd.9.smithi012.stdout:Sep 11 00:34:27 smithi012 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-9[66733]: 2024-09-11T00:34:27.183+0000 7f5f4e800640 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-11T00:33:41.710055+0000 front 2024-09-11T00:33:41.709869+0000 (oldest deadline 2024-09-11T00:34:07.010035+0000) 2024-09-11T00:34:27.563 INFO:journalctl@ceph.osd.9.smithi012.stdout:Sep 11 00:34:27 smithi012 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-9[66733]: 2024-09-11T00:34:27.183+0000 7f5f4e800640 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-09-11T00:33:47.010623+0000 front 2024-09-11T00:33:47.010590+0000 (oldest deadline 2024-09-11T00:34:12.910692+0000) 2024-09-11T00:34:27.563 INFO:journalctl@ceph.osd.9.smithi012.stdout:Sep 11 00:34:27 smithi012 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-9[66733]: 2024-09-11T00:34:27.183+0000 7f5f4e800640 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.165:6812 osd.4 since back 2024-09-11T00:33:59.213236+0000 front 2024-09-11T00:33:59.213312+0000 (oldest deadline 2024-09-11T00:34:24.513408+0000) 2024-09-11T00:34:27.791 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 11 00:34:27 smithi178 systemd[1]: Stopping Ceph osd.8 for ff3ad74a-6fcf-11ef-bcea-c7b262605968... 2024-09-11T00:34:28.081 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 11 00:34:27 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-8[54042]: 2024-09-11T00:34:27.788+0000 7f230fdc9640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.8 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T00:34:28.081 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 11 00:34:27 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-8[54042]: 2024-09-11T00:34:27.788+0000 7f230fdc9640 -1 osd.8 91 *** Got signal Terminated *** 2024-09-11T00:34:28.081 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 11 00:34:27 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-8[54042]: 2024-09-11T00:34:27.788+0000 7f230fdc9640 -1 osd.8 91 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-11T00:34:28.283 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 11 00:34:27 smithi165 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-10[59927]: 2024-09-11T00:34:27.985+0000 7f2cff41d640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-09-11T00:33:25.146772+0000 front 2024-09-11T00:33:25.146620+0000 (oldest deadline 2024-09-11T00:33:49.846529+0000) 2024-09-11T00:34:28.283 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 11 00:34:27 smithi165 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-10[59927]: 2024-09-11T00:34:27.985+0000 7f2cff41d640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-11T00:33:32.747514+0000 front 2024-09-11T00:33:32.747491+0000 (oldest deadline 2024-09-11T00:33:58.047686+0000) 2024-09-11T00:34:28.283 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 11 00:34:27 smithi165 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-10[59927]: 2024-09-11T00:34:27.985+0000 7f2cff41d640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-11T00:33:41.548749+0000 front 2024-09-11T00:33:41.548701+0000 (oldest deadline 2024-09-11T00:34:05.648724+0000) 2024-09-11T00:34:28.283 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 11 00:34:27 smithi165 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-10[59927]: 2024-09-11T00:34:27.985+0000 7f2cff41d640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-09-11T00:33:50.850789+0000 front 2024-09-11T00:33:50.850826+0000 (oldest deadline 2024-09-11T00:34:11.950647+0000) 2024-09-11T00:34:28.283 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 11 00:34:27 smithi165 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-10[59927]: 2024-09-11T00:34:27.985+0000 7f2cff41d640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.165:6812 osd.4 since back 2024-09-11T00:33:55.951809+0000 front 2024-09-11T00:33:55.951787+0000 (oldest deadline 2024-09-11T00:34:21.851850+0000) 2024-09-11T00:34:28.563 INFO:journalctl@ceph.osd.9.smithi012.stdout:Sep 11 00:34:28 smithi012 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-9[66733]: 2024-09-11T00:34:28.219+0000 7f5f4e800640 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-09-11T00:33:25.406156+0000 front 2024-09-11T00:33:25.406095+0000 (oldest deadline 2024-09-11T00:33:47.706239+0000) 2024-09-11T00:34:28.563 INFO:journalctl@ceph.osd.9.smithi012.stdout:Sep 11 00:34:28 smithi012 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-9[66733]: 2024-09-11T00:34:28.219+0000 7f5f4e800640 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-11T00:33:30.607371+0000 front 2024-09-11T00:33:30.607499+0000 (oldest deadline 2024-09-11T00:33:55.307466+0000) 2024-09-11T00:34:28.564 INFO:journalctl@ceph.osd.9.smithi012.stdout:Sep 11 00:34:28 smithi012 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-9[66733]: 2024-09-11T00:34:28.219+0000 7f5f4e800640 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-11T00:33:41.710055+0000 front 2024-09-11T00:33:41.709869+0000 (oldest deadline 2024-09-11T00:34:07.010035+0000) 2024-09-11T00:34:28.564 INFO:journalctl@ceph.osd.9.smithi012.stdout:Sep 11 00:34:28 smithi012 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-9[66733]: 2024-09-11T00:34:28.219+0000 7f5f4e800640 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-09-11T00:33:47.010623+0000 front 2024-09-11T00:33:47.010590+0000 (oldest deadline 2024-09-11T00:34:12.910692+0000) 2024-09-11T00:34:28.564 INFO:journalctl@ceph.osd.9.smithi012.stdout:Sep 11 00:34:28 smithi012 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-9[66733]: 2024-09-11T00:34:28.219+0000 7f5f4e800640 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.165:6812 osd.4 since back 2024-09-11T00:33:59.213236+0000 front 2024-09-11T00:33:59.213312+0000 (oldest deadline 2024-09-11T00:34:24.513408+0000) 2024-09-11T00:34:28.581 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 00:34:28 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-11[60852]: 2024-09-11T00:34:28.401+0000 7fd754b0b640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-09-11T00:33:24.081060+0000 front 2024-09-11T00:33:24.081080+0000 (oldest deadline 2024-09-11T00:33:49.980779+0000) 2024-09-11T00:34:28.581 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 00:34:28 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-11[60852]: 2024-09-11T00:34:28.401+0000 7fd754b0b640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-11T00:33:41.682818+0000 front 2024-09-11T00:33:41.682848+0000 (oldest deadline 2024-09-11T00:34:05.182914+0000) 2024-09-11T00:34:28.582 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 00:34:28 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-11[60852]: 2024-09-11T00:34:28.401+0000 7fd754b0b640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-09-11T00:33:47.484088+0000 front 2024-09-11T00:33:47.484103+0000 (oldest deadline 2024-09-11T00:34:12.784060+0000) 2024-09-11T00:34:28.582 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 00:34:28 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-11[60852]: 2024-09-11T00:34:28.401+0000 7fd754b0b640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.165:6812 osd.4 since back 2024-09-11T00:33:58.685349+0000 front 2024-09-11T00:33:58.685336+0000 (oldest deadline 2024-09-11T00:34:23.385413+0000) 2024-09-11T00:34:28.582 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 11 00:34:28 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-8[54042]: 2024-09-11T00:34:28.174+0000 7f23083d6640 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-11T00:33:33.673118+0000 front 2024-09-11T00:33:33.673042+0000 (oldest deadline 2024-09-11T00:33:56.573057+0000) 2024-09-11T00:34:28.582 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 11 00:34:28 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-8[54042]: 2024-09-11T00:34:28.174+0000 7f23083d6640 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-11T00:33:38.774830+0000 front 2024-09-11T00:33:38.774774+0000 (oldest deadline 2024-09-11T00:34:04.674919+0000) 2024-09-11T00:34:28.582 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 11 00:34:28 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-8[54042]: 2024-09-11T00:34:28.174+0000 7f23083d6640 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-09-11T00:33:47.977769+0000 front 2024-09-11T00:33:47.977619+0000 (oldest deadline 2024-09-11T00:34:13.277556+0000) 2024-09-11T00:34:28.582 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 11 00:34:28 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-8[54042]: 2024-09-11T00:34:28.174+0000 7f23083d6640 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.165:6812 osd.4 since back 2024-09-11T00:33:58.578956+0000 front 2024-09-11T00:33:58.579029+0000 (oldest deadline 2024-09-11T00:34:24.979373+0000) 2024-09-11T00:34:29.282 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 11 00:34:29 smithi165 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-10[59927]: 2024-09-11T00:34:29.017+0000 7f2cff41d640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-09-11T00:33:25.146772+0000 front 2024-09-11T00:33:25.146620+0000 (oldest deadline 2024-09-11T00:33:49.846529+0000) 2024-09-11T00:34:29.283 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 11 00:34:29 smithi165 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-10[59927]: 2024-09-11T00:34:29.017+0000 7f2cff41d640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-11T00:33:32.747514+0000 front 2024-09-11T00:33:32.747491+0000 (oldest deadline 2024-09-11T00:33:58.047686+0000) 2024-09-11T00:34:29.283 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 11 00:34:29 smithi165 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-10[59927]: 2024-09-11T00:34:29.017+0000 7f2cff41d640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-11T00:33:41.548749+0000 front 2024-09-11T00:33:41.548701+0000 (oldest deadline 2024-09-11T00:34:05.648724+0000) 2024-09-11T00:34:29.283 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 11 00:34:29 smithi165 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-10[59927]: 2024-09-11T00:34:29.017+0000 7f2cff41d640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-09-11T00:33:50.850789+0000 front 2024-09-11T00:33:50.850826+0000 (oldest deadline 2024-09-11T00:34:11.950647+0000) 2024-09-11T00:34:29.283 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 11 00:34:29 smithi165 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-10[59927]: 2024-09-11T00:34:29.017+0000 7f2cff41d640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.165:6812 osd.4 since back 2024-09-11T00:33:55.951809+0000 front 2024-09-11T00:33:55.951787+0000 (oldest deadline 2024-09-11T00:34:21.851850+0000) 2024-09-11T00:34:29.283 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 11 00:34:29 smithi165 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-10[59927]: 2024-09-11T00:34:29.017+0000 7f2cff41d640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.178:6812 osd.5 since back 2024-09-11T00:34:07.752773+0000 front 2024-09-11T00:34:07.752884+0000 (oldest deadline 2024-09-11T00:34:28.852602+0000) 2024-09-11T00:34:29.419 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 11 00:34:29 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-8[54042]: 2024-09-11T00:34:29.159+0000 7f23083d6640 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-11T00:33:33.673118+0000 front 2024-09-11T00:33:33.673042+0000 (oldest deadline 2024-09-11T00:33:56.573057+0000) 2024-09-11T00:34:29.419 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 11 00:34:29 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-8[54042]: 2024-09-11T00:34:29.159+0000 7f23083d6640 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-11T00:33:38.774830+0000 front 2024-09-11T00:33:38.774774+0000 (oldest deadline 2024-09-11T00:34:04.674919+0000) 2024-09-11T00:34:29.419 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 11 00:34:29 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-8[54042]: 2024-09-11T00:34:29.159+0000 7f23083d6640 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-09-11T00:33:47.977769+0000 front 2024-09-11T00:33:47.977619+0000 (oldest deadline 2024-09-11T00:34:13.277556+0000) 2024-09-11T00:34:29.419 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 11 00:34:29 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-8[54042]: 2024-09-11T00:34:29.159+0000 7f23083d6640 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.165:6812 osd.4 since back 2024-09-11T00:33:58.578956+0000 front 2024-09-11T00:33:58.579029+0000 (oldest deadline 2024-09-11T00:34:24.979373+0000) 2024-09-11T00:34:29.563 INFO:journalctl@ceph.osd.9.smithi012.stdout:Sep 11 00:34:29 smithi012 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-9[66733]: 2024-09-11T00:34:29.263+0000 7f5f4e800640 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-09-11T00:33:25.406156+0000 front 2024-09-11T00:33:25.406095+0000 (oldest deadline 2024-09-11T00:33:47.706239+0000) 2024-09-11T00:34:29.563 INFO:journalctl@ceph.osd.9.smithi012.stdout:Sep 11 00:34:29 smithi012 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-9[66733]: 2024-09-11T00:34:29.263+0000 7f5f4e800640 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-11T00:33:30.607371+0000 front 2024-09-11T00:33:30.607499+0000 (oldest deadline 2024-09-11T00:33:55.307466+0000) 2024-09-11T00:34:29.563 INFO:journalctl@ceph.osd.9.smithi012.stdout:Sep 11 00:34:29 smithi012 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-9[66733]: 2024-09-11T00:34:29.263+0000 7f5f4e800640 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-11T00:33:41.710055+0000 front 2024-09-11T00:33:41.709869+0000 (oldest deadline 2024-09-11T00:34:07.010035+0000) 2024-09-11T00:34:29.563 INFO:journalctl@ceph.osd.9.smithi012.stdout:Sep 11 00:34:29 smithi012 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-9[66733]: 2024-09-11T00:34:29.263+0000 7f5f4e800640 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-09-11T00:33:47.010623+0000 front 2024-09-11T00:33:47.010590+0000 (oldest deadline 2024-09-11T00:34:12.910692+0000) 2024-09-11T00:34:29.563 INFO:journalctl@ceph.osd.9.smithi012.stdout:Sep 11 00:34:29 smithi012 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-9[66733]: 2024-09-11T00:34:29.263+0000 7f5f4e800640 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.165:6812 osd.4 since back 2024-09-11T00:33:59.213236+0000 front 2024-09-11T00:33:59.213312+0000 (oldest deadline 2024-09-11T00:34:24.513408+0000) 2024-09-11T00:34:29.831 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 00:34:29 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-11[60852]: 2024-09-11T00:34:29.416+0000 7fd754b0b640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-09-11T00:33:24.081060+0000 front 2024-09-11T00:33:24.081080+0000 (oldest deadline 2024-09-11T00:33:49.980779+0000) 2024-09-11T00:34:29.831 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 00:34:29 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-11[60852]: 2024-09-11T00:34:29.416+0000 7fd754b0b640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-11T00:33:41.682818+0000 front 2024-09-11T00:33:41.682848+0000 (oldest deadline 2024-09-11T00:34:05.182914+0000) 2024-09-11T00:34:29.832 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 00:34:29 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-11[60852]: 2024-09-11T00:34:29.416+0000 7fd754b0b640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-09-11T00:33:47.484088+0000 front 2024-09-11T00:33:47.484103+0000 (oldest deadline 2024-09-11T00:34:12.784060+0000) 2024-09-11T00:34:29.832 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 00:34:29 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-11[60852]: 2024-09-11T00:34:29.416+0000 7fd754b0b640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.165:6812 osd.4 since back 2024-09-11T00:33:58.685349+0000 front 2024-09-11T00:33:58.685336+0000 (oldest deadline 2024-09-11T00:34:23.385413+0000) 2024-09-11T00:34:29.832 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 00:34:29 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-11[60852]: 2024-09-11T00:34:29.416+0000 7fd754b0b640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.178:6812 osd.5 since back 2024-09-11T00:34:07.887744+0000 front 2024-09-11T00:34:07.887694+0000 (oldest deadline 2024-09-11T00:34:28.987855+0000) 2024-09-11T00:34:30.282 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 11 00:34:29 smithi165 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-10[59927]: 2024-09-11T00:34:29.993+0000 7f2cff41d640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-09-11T00:33:25.146772+0000 front 2024-09-11T00:33:25.146620+0000 (oldest deadline 2024-09-11T00:33:49.846529+0000) 2024-09-11T00:34:30.283 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 11 00:34:29 smithi165 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-10[59927]: 2024-09-11T00:34:29.993+0000 7f2cff41d640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-11T00:33:32.747514+0000 front 2024-09-11T00:33:32.747491+0000 (oldest deadline 2024-09-11T00:33:58.047686+0000) 2024-09-11T00:34:30.283 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 11 00:34:29 smithi165 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-10[59927]: 2024-09-11T00:34:29.993+0000 7f2cff41d640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-11T00:33:41.548749+0000 front 2024-09-11T00:33:41.548701+0000 (oldest deadline 2024-09-11T00:34:05.648724+0000) 2024-09-11T00:34:30.283 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 11 00:34:29 smithi165 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-10[59927]: 2024-09-11T00:34:29.993+0000 7f2cff41d640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-09-11T00:33:50.850789+0000 front 2024-09-11T00:33:50.850826+0000 (oldest deadline 2024-09-11T00:34:11.950647+0000) 2024-09-11T00:34:30.283 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 11 00:34:29 smithi165 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-10[59927]: 2024-09-11T00:34:29.993+0000 7f2cff41d640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.165:6812 osd.4 since back 2024-09-11T00:33:55.951809+0000 front 2024-09-11T00:33:55.951787+0000 (oldest deadline 2024-09-11T00:34:21.851850+0000) 2024-09-11T00:34:30.283 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 11 00:34:29 smithi165 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-10[59927]: 2024-09-11T00:34:29.993+0000 7f2cff41d640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.178:6812 osd.5 since back 2024-09-11T00:34:07.752773+0000 front 2024-09-11T00:34:07.752884+0000 (oldest deadline 2024-09-11T00:34:28.852602+0000) 2024-09-11T00:34:30.443 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 11 00:34:30 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-8[54042]: 2024-09-11T00:34:30.144+0000 7f23083d6640 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-11T00:33:33.673118+0000 front 2024-09-11T00:33:33.673042+0000 (oldest deadline 2024-09-11T00:33:56.573057+0000) 2024-09-11T00:34:30.443 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 11 00:34:30 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-8[54042]: 2024-09-11T00:34:30.144+0000 7f23083d6640 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-11T00:33:38.774830+0000 front 2024-09-11T00:33:38.774774+0000 (oldest deadline 2024-09-11T00:34:04.674919+0000) 2024-09-11T00:34:30.443 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 11 00:34:30 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-8[54042]: 2024-09-11T00:34:30.144+0000 7f23083d6640 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-09-11T00:33:47.977769+0000 front 2024-09-11T00:33:47.977619+0000 (oldest deadline 2024-09-11T00:34:13.277556+0000) 2024-09-11T00:34:30.443 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 11 00:34:30 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-8[54042]: 2024-09-11T00:34:30.144+0000 7f23083d6640 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.165:6812 osd.4 since back 2024-09-11T00:33:58.578956+0000 front 2024-09-11T00:33:58.579029+0000 (oldest deadline 2024-09-11T00:34:24.979373+0000) 2024-09-11T00:34:30.563 INFO:journalctl@ceph.osd.9.smithi012.stdout:Sep 11 00:34:30 smithi012 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-9[66733]: 2024-09-11T00:34:30.234+0000 7f5f4e800640 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-09-11T00:33:25.406156+0000 front 2024-09-11T00:33:25.406095+0000 (oldest deadline 2024-09-11T00:33:47.706239+0000) 2024-09-11T00:34:30.563 INFO:journalctl@ceph.osd.9.smithi012.stdout:Sep 11 00:34:30 smithi012 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-9[66733]: 2024-09-11T00:34:30.234+0000 7f5f4e800640 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-11T00:33:30.607371+0000 front 2024-09-11T00:33:30.607499+0000 (oldest deadline 2024-09-11T00:33:55.307466+0000) 2024-09-11T00:34:30.563 INFO:journalctl@ceph.osd.9.smithi012.stdout:Sep 11 00:34:30 smithi012 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-9[66733]: 2024-09-11T00:34:30.234+0000 7f5f4e800640 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-11T00:33:41.710055+0000 front 2024-09-11T00:33:41.709869+0000 (oldest deadline 2024-09-11T00:34:07.010035+0000) 2024-09-11T00:34:30.563 INFO:journalctl@ceph.osd.9.smithi012.stdout:Sep 11 00:34:30 smithi012 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-9[66733]: 2024-09-11T00:34:30.234+0000 7f5f4e800640 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-09-11T00:33:47.010623+0000 front 2024-09-11T00:33:47.010590+0000 (oldest deadline 2024-09-11T00:34:12.910692+0000) 2024-09-11T00:34:30.563 INFO:journalctl@ceph.osd.9.smithi012.stdout:Sep 11 00:34:30 smithi012 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-9[66733]: 2024-09-11T00:34:30.234+0000 7f5f4e800640 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.165:6812 osd.4 since back 2024-09-11T00:33:59.213236+0000 front 2024-09-11T00:33:59.213312+0000 (oldest deadline 2024-09-11T00:34:24.513408+0000) 2024-09-11T00:34:30.564 INFO:journalctl@ceph.osd.9.smithi012.stdout:Sep 11 00:34:30 smithi012 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-9[66733]: 2024-09-11T00:34:30.234+0000 7f5f4e800640 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.178:6812 osd.5 since back 2024-09-11T00:34:04.514055+0000 front 2024-09-11T00:34:04.514150+0000 (oldest deadline 2024-09-11T00:34:29.813900+0000) 2024-09-11T00:34:30.831 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 00:34:30 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-11[60852]: 2024-09-11T00:34:30.441+0000 7fd754b0b640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-09-11T00:33:24.081060+0000 front 2024-09-11T00:33:24.081080+0000 (oldest deadline 2024-09-11T00:33:49.980779+0000) 2024-09-11T00:34:30.831 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 00:34:30 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-11[60852]: 2024-09-11T00:34:30.441+0000 7fd754b0b640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-11T00:33:41.682818+0000 front 2024-09-11T00:33:41.682848+0000 (oldest deadline 2024-09-11T00:34:05.182914+0000) 2024-09-11T00:34:30.831 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 00:34:30 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-11[60852]: 2024-09-11T00:34:30.441+0000 7fd754b0b640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-09-11T00:33:47.484088+0000 front 2024-09-11T00:33:47.484103+0000 (oldest deadline 2024-09-11T00:34:12.784060+0000) 2024-09-11T00:34:30.831 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 00:34:30 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-11[60852]: 2024-09-11T00:34:30.441+0000 7fd754b0b640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.165:6812 osd.4 since back 2024-09-11T00:33:58.685349+0000 front 2024-09-11T00:33:58.685336+0000 (oldest deadline 2024-09-11T00:34:23.385413+0000) 2024-09-11T00:34:30.832 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 00:34:30 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-11[60852]: 2024-09-11T00:34:30.441+0000 7fd754b0b640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.178:6812 osd.5 since back 2024-09-11T00:34:07.887744+0000 front 2024-09-11T00:34:07.887694+0000 (oldest deadline 2024-09-11T00:34:28.987855+0000) 2024-09-11T00:34:31.283 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 11 00:34:31 smithi165 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-10[59927]: 2024-09-11T00:34:31.024+0000 7f2cff41d640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-09-11T00:33:25.146772+0000 front 2024-09-11T00:33:25.146620+0000 (oldest deadline 2024-09-11T00:33:49.846529+0000) 2024-09-11T00:34:31.283 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 11 00:34:31 smithi165 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-10[59927]: 2024-09-11T00:34:31.024+0000 7f2cff41d640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-11T00:33:32.747514+0000 front 2024-09-11T00:33:32.747491+0000 (oldest deadline 2024-09-11T00:33:58.047686+0000) 2024-09-11T00:34:31.283 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 11 00:34:31 smithi165 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-10[59927]: 2024-09-11T00:34:31.024+0000 7f2cff41d640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-11T00:33:41.548749+0000 front 2024-09-11T00:33:41.548701+0000 (oldest deadline 2024-09-11T00:34:05.648724+0000) 2024-09-11T00:34:31.283 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 11 00:34:31 smithi165 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-10[59927]: 2024-09-11T00:34:31.024+0000 7f2cff41d640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-09-11T00:33:50.850789+0000 front 2024-09-11T00:33:50.850826+0000 (oldest deadline 2024-09-11T00:34:11.950647+0000) 2024-09-11T00:34:31.283 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 11 00:34:31 smithi165 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-10[59927]: 2024-09-11T00:34:31.024+0000 7f2cff41d640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.165:6812 osd.4 since back 2024-09-11T00:33:55.951809+0000 front 2024-09-11T00:33:55.951787+0000 (oldest deadline 2024-09-11T00:34:21.851850+0000) 2024-09-11T00:34:31.283 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 11 00:34:31 smithi165 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-10[59927]: 2024-09-11T00:34:31.024+0000 7f2cff41d640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.178:6812 osd.5 since back 2024-09-11T00:34:07.752773+0000 front 2024-09-11T00:34:07.752884+0000 (oldest deadline 2024-09-11T00:34:28.852602+0000) 2024-09-11T00:34:31.468 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 11 00:34:31 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-8[54042]: 2024-09-11T00:34:31.144+0000 7f23083d6640 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-11T00:33:33.673118+0000 front 2024-09-11T00:33:33.673042+0000 (oldest deadline 2024-09-11T00:33:56.573057+0000) 2024-09-11T00:34:31.468 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 11 00:34:31 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-8[54042]: 2024-09-11T00:34:31.144+0000 7f23083d6640 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-11T00:33:38.774830+0000 front 2024-09-11T00:33:38.774774+0000 (oldest deadline 2024-09-11T00:34:04.674919+0000) 2024-09-11T00:34:31.468 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 11 00:34:31 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-8[54042]: 2024-09-11T00:34:31.144+0000 7f23083d6640 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-09-11T00:33:47.977769+0000 front 2024-09-11T00:33:47.977619+0000 (oldest deadline 2024-09-11T00:34:13.277556+0000) 2024-09-11T00:34:31.469 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 11 00:34:31 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-8[54042]: 2024-09-11T00:34:31.144+0000 7f23083d6640 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.165:6812 osd.4 since back 2024-09-11T00:33:58.578956+0000 front 2024-09-11T00:33:58.579029+0000 (oldest deadline 2024-09-11T00:34:24.979373+0000) 2024-09-11T00:34:31.469 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 11 00:34:31 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-8[54042]: 2024-09-11T00:34:31.144+0000 7f23083d6640 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.178:6812 osd.5 since back 2024-09-11T00:34:06.680265+0000 front 2024-09-11T00:34:06.680290+0000 (oldest deadline 2024-09-11T00:34:30.180337+0000) 2024-09-11T00:34:31.563 INFO:journalctl@ceph.osd.9.smithi012.stdout:Sep 11 00:34:31 smithi012 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-9[66733]: 2024-09-11T00:34:31.237+0000 7f5f4e800640 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-09-11T00:33:25.406156+0000 front 2024-09-11T00:33:25.406095+0000 (oldest deadline 2024-09-11T00:33:47.706239+0000) 2024-09-11T00:34:31.563 INFO:journalctl@ceph.osd.9.smithi012.stdout:Sep 11 00:34:31 smithi012 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-9[66733]: 2024-09-11T00:34:31.237+0000 7f5f4e800640 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-11T00:33:30.607371+0000 front 2024-09-11T00:33:30.607499+0000 (oldest deadline 2024-09-11T00:33:55.307466+0000) 2024-09-11T00:34:31.563 INFO:journalctl@ceph.osd.9.smithi012.stdout:Sep 11 00:34:31 smithi012 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-9[66733]: 2024-09-11T00:34:31.237+0000 7f5f4e800640 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-11T00:33:41.710055+0000 front 2024-09-11T00:33:41.709869+0000 (oldest deadline 2024-09-11T00:34:07.010035+0000) 2024-09-11T00:34:31.563 INFO:journalctl@ceph.osd.9.smithi012.stdout:Sep 11 00:34:31 smithi012 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-9[66733]: 2024-09-11T00:34:31.237+0000 7f5f4e800640 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-09-11T00:33:47.010623+0000 front 2024-09-11T00:33:47.010590+0000 (oldest deadline 2024-09-11T00:34:12.910692+0000) 2024-09-11T00:34:31.563 INFO:journalctl@ceph.osd.9.smithi012.stdout:Sep 11 00:34:31 smithi012 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-9[66733]: 2024-09-11T00:34:31.237+0000 7f5f4e800640 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.165:6812 osd.4 since back 2024-09-11T00:33:59.213236+0000 front 2024-09-11T00:33:59.213312+0000 (oldest deadline 2024-09-11T00:34:24.513408+0000) 2024-09-11T00:34:31.564 INFO:journalctl@ceph.osd.9.smithi012.stdout:Sep 11 00:34:31 smithi012 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-9[66733]: 2024-09-11T00:34:31.237+0000 7f5f4e800640 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.178:6812 osd.5 since back 2024-09-11T00:34:04.514055+0000 front 2024-09-11T00:34:04.514150+0000 (oldest deadline 2024-09-11T00:34:29.813900+0000) 2024-09-11T00:34:31.831 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 00:34:31 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-11[60852]: 2024-09-11T00:34:31.465+0000 7fd754b0b640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-09-11T00:33:24.081060+0000 front 2024-09-11T00:33:24.081080+0000 (oldest deadline 2024-09-11T00:33:49.980779+0000) 2024-09-11T00:34:31.831 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 00:34:31 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-11[60852]: 2024-09-11T00:34:31.465+0000 7fd754b0b640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-11T00:33:41.682818+0000 front 2024-09-11T00:33:41.682848+0000 (oldest deadline 2024-09-11T00:34:05.182914+0000) 2024-09-11T00:34:31.831 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 00:34:31 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-11[60852]: 2024-09-11T00:34:31.465+0000 7fd754b0b640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-09-11T00:33:47.484088+0000 front 2024-09-11T00:33:47.484103+0000 (oldest deadline 2024-09-11T00:34:12.784060+0000) 2024-09-11T00:34:31.832 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 00:34:31 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-11[60852]: 2024-09-11T00:34:31.465+0000 7fd754b0b640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.165:6812 osd.4 since back 2024-09-11T00:33:58.685349+0000 front 2024-09-11T00:33:58.685336+0000 (oldest deadline 2024-09-11T00:34:23.385413+0000) 2024-09-11T00:34:31.832 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 00:34:31 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-11[60852]: 2024-09-11T00:34:31.465+0000 7fd754b0b640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.178:6812 osd.5 since back 2024-09-11T00:34:07.887744+0000 front 2024-09-11T00:34:07.887694+0000 (oldest deadline 2024-09-11T00:34:28.987855+0000) 2024-09-11T00:34:32.421 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 11 00:34:32 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-8[54042]: 2024-09-11T00:34:32.156+0000 7f23083d6640 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-11T00:33:33.673118+0000 front 2024-09-11T00:33:33.673042+0000 (oldest deadline 2024-09-11T00:33:56.573057+0000) 2024-09-11T00:34:32.421 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 11 00:34:32 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-8[54042]: 2024-09-11T00:34:32.156+0000 7f23083d6640 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-11T00:33:38.774830+0000 front 2024-09-11T00:33:38.774774+0000 (oldest deadline 2024-09-11T00:34:04.674919+0000) 2024-09-11T00:34:32.421 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 11 00:34:32 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-8[54042]: 2024-09-11T00:34:32.156+0000 7f23083d6640 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-09-11T00:33:47.977769+0000 front 2024-09-11T00:33:47.977619+0000 (oldest deadline 2024-09-11T00:34:13.277556+0000) 2024-09-11T00:34:32.421 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 11 00:34:32 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-8[54042]: 2024-09-11T00:34:32.156+0000 7f23083d6640 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.165:6812 osd.4 since back 2024-09-11T00:33:58.578956+0000 front 2024-09-11T00:33:58.579029+0000 (oldest deadline 2024-09-11T00:34:24.979373+0000) 2024-09-11T00:34:32.421 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 11 00:34:32 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-8[54042]: 2024-09-11T00:34:32.156+0000 7f23083d6640 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.178:6812 osd.5 since back 2024-09-11T00:34:06.680265+0000 front 2024-09-11T00:34:06.680290+0000 (oldest deadline 2024-09-11T00:34:30.180337+0000) 2024-09-11T00:34:32.533 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 11 00:34:32 smithi165 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-10[59927]: 2024-09-11T00:34:32.059+0000 7f2cff41d640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-09-11T00:33:25.146772+0000 front 2024-09-11T00:33:25.146620+0000 (oldest deadline 2024-09-11T00:33:49.846529+0000) 2024-09-11T00:34:32.533 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 11 00:34:32 smithi165 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-10[59927]: 2024-09-11T00:34:32.059+0000 7f2cff41d640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-11T00:33:32.747514+0000 front 2024-09-11T00:33:32.747491+0000 (oldest deadline 2024-09-11T00:33:58.047686+0000) 2024-09-11T00:34:32.533 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 11 00:34:32 smithi165 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-10[59927]: 2024-09-11T00:34:32.059+0000 7f2cff41d640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-11T00:33:41.548749+0000 front 2024-09-11T00:33:41.548701+0000 (oldest deadline 2024-09-11T00:34:05.648724+0000) 2024-09-11T00:34:32.533 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 11 00:34:32 smithi165 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-10[59927]: 2024-09-11T00:34:32.059+0000 7f2cff41d640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-09-11T00:33:50.850789+0000 front 2024-09-11T00:33:50.850826+0000 (oldest deadline 2024-09-11T00:34:11.950647+0000) 2024-09-11T00:34:32.533 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 11 00:34:32 smithi165 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-10[59927]: 2024-09-11T00:34:32.059+0000 7f2cff41d640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.165:6812 osd.4 since back 2024-09-11T00:33:55.951809+0000 front 2024-09-11T00:33:55.951787+0000 (oldest deadline 2024-09-11T00:34:21.851850+0000) 2024-09-11T00:34:32.533 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 11 00:34:32 smithi165 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-10[59927]: 2024-09-11T00:34:32.059+0000 7f2cff41d640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.178:6812 osd.5 since back 2024-09-11T00:34:07.752773+0000 front 2024-09-11T00:34:07.752884+0000 (oldest deadline 2024-09-11T00:34:28.852602+0000) 2024-09-11T00:34:32.563 INFO:journalctl@ceph.osd.9.smithi012.stdout:Sep 11 00:34:32 smithi012 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-9[66733]: 2024-09-11T00:34:32.215+0000 7f5f4e800640 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-09-11T00:33:25.406156+0000 front 2024-09-11T00:33:25.406095+0000 (oldest deadline 2024-09-11T00:33:47.706239+0000) 2024-09-11T00:34:32.563 INFO:journalctl@ceph.osd.9.smithi012.stdout:Sep 11 00:34:32 smithi012 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-9[66733]: 2024-09-11T00:34:32.215+0000 7f5f4e800640 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-11T00:33:30.607371+0000 front 2024-09-11T00:33:30.607499+0000 (oldest deadline 2024-09-11T00:33:55.307466+0000) 2024-09-11T00:34:32.563 INFO:journalctl@ceph.osd.9.smithi012.stdout:Sep 11 00:34:32 smithi012 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-9[66733]: 2024-09-11T00:34:32.215+0000 7f5f4e800640 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-11T00:33:41.710055+0000 front 2024-09-11T00:33:41.709869+0000 (oldest deadline 2024-09-11T00:34:07.010035+0000) 2024-09-11T00:34:32.563 INFO:journalctl@ceph.osd.9.smithi012.stdout:Sep 11 00:34:32 smithi012 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-9[66733]: 2024-09-11T00:34:32.215+0000 7f5f4e800640 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-09-11T00:33:47.010623+0000 front 2024-09-11T00:33:47.010590+0000 (oldest deadline 2024-09-11T00:34:12.910692+0000) 2024-09-11T00:34:32.564 INFO:journalctl@ceph.osd.9.smithi012.stdout:Sep 11 00:34:32 smithi012 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-9[66733]: 2024-09-11T00:34:32.215+0000 7f5f4e800640 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.165:6812 osd.4 since back 2024-09-11T00:33:59.213236+0000 front 2024-09-11T00:33:59.213312+0000 (oldest deadline 2024-09-11T00:34:24.513408+0000) 2024-09-11T00:34:32.564 INFO:journalctl@ceph.osd.9.smithi012.stdout:Sep 11 00:34:32 smithi012 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-9[66733]: 2024-09-11T00:34:32.215+0000 7f5f4e800640 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.178:6812 osd.5 since back 2024-09-11T00:34:04.514055+0000 front 2024-09-11T00:34:04.514150+0000 (oldest deadline 2024-09-11T00:34:29.813900+0000) 2024-09-11T00:34:32.831 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 00:34:32 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-11[60852]: 2024-09-11T00:34:32.419+0000 7fd754b0b640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-09-11T00:33:24.081060+0000 front 2024-09-11T00:33:24.081080+0000 (oldest deadline 2024-09-11T00:33:49.980779+0000) 2024-09-11T00:34:32.831 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 00:34:32 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-11[60852]: 2024-09-11T00:34:32.419+0000 7fd754b0b640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-11T00:33:41.682818+0000 front 2024-09-11T00:33:41.682848+0000 (oldest deadline 2024-09-11T00:34:05.182914+0000) 2024-09-11T00:34:32.831 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 00:34:32 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-11[60852]: 2024-09-11T00:34:32.419+0000 7fd754b0b640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-09-11T00:33:47.484088+0000 front 2024-09-11T00:33:47.484103+0000 (oldest deadline 2024-09-11T00:34:12.784060+0000) 2024-09-11T00:34:32.832 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 00:34:32 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-11[60852]: 2024-09-11T00:34:32.419+0000 7fd754b0b640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.165:6812 osd.4 since back 2024-09-11T00:33:58.685349+0000 front 2024-09-11T00:33:58.685336+0000 (oldest deadline 2024-09-11T00:34:23.385413+0000) 2024-09-11T00:34:32.832 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 00:34:32 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-11[60852]: 2024-09-11T00:34:32.419+0000 7fd754b0b640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.178:6812 osd.5 since back 2024-09-11T00:34:07.887744+0000 front 2024-09-11T00:34:07.887694+0000 (oldest deadline 2024-09-11T00:34:28.987855+0000) 2024-09-11T00:34:33.283 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 11 00:34:33 smithi165 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-10[59927]: 2024-09-11T00:34:33.023+0000 7f2cff41d640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-09-11T00:33:25.146772+0000 front 2024-09-11T00:33:25.146620+0000 (oldest deadline 2024-09-11T00:33:49.846529+0000) 2024-09-11T00:34:33.283 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 11 00:34:33 smithi165 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-10[59927]: 2024-09-11T00:34:33.023+0000 7f2cff41d640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-11T00:33:32.747514+0000 front 2024-09-11T00:33:32.747491+0000 (oldest deadline 2024-09-11T00:33:58.047686+0000) 2024-09-11T00:34:33.283 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 11 00:34:33 smithi165 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-10[59927]: 2024-09-11T00:34:33.023+0000 7f2cff41d640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-11T00:33:41.548749+0000 front 2024-09-11T00:33:41.548701+0000 (oldest deadline 2024-09-11T00:34:05.648724+0000) 2024-09-11T00:34:33.283 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 11 00:34:33 smithi165 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-10[59927]: 2024-09-11T00:34:33.023+0000 7f2cff41d640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-09-11T00:33:50.850789+0000 front 2024-09-11T00:33:50.850826+0000 (oldest deadline 2024-09-11T00:34:11.950647+0000) 2024-09-11T00:34:33.283 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 11 00:34:33 smithi165 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-10[59927]: 2024-09-11T00:34:33.023+0000 7f2cff41d640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.165:6812 osd.4 since back 2024-09-11T00:33:55.951809+0000 front 2024-09-11T00:33:55.951787+0000 (oldest deadline 2024-09-11T00:34:21.851850+0000) 2024-09-11T00:34:33.283 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 11 00:34:33 smithi165 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-10[59927]: 2024-09-11T00:34:33.023+0000 7f2cff41d640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.178:6812 osd.5 since back 2024-09-11T00:34:07.752773+0000 front 2024-09-11T00:34:07.752884+0000 (oldest deadline 2024-09-11T00:34:28.852602+0000) 2024-09-11T00:34:33.563 INFO:journalctl@ceph.osd.9.smithi012.stdout:Sep 11 00:34:33 smithi012 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-9[66733]: 2024-09-11T00:34:33.237+0000 7f5f4e800640 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-09-11T00:33:25.406156+0000 front 2024-09-11T00:33:25.406095+0000 (oldest deadline 2024-09-11T00:33:47.706239+0000) 2024-09-11T00:34:33.563 INFO:journalctl@ceph.osd.9.smithi012.stdout:Sep 11 00:34:33 smithi012 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-9[66733]: 2024-09-11T00:34:33.237+0000 7f5f4e800640 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-11T00:33:30.607371+0000 front 2024-09-11T00:33:30.607499+0000 (oldest deadline 2024-09-11T00:33:55.307466+0000) 2024-09-11T00:34:33.563 INFO:journalctl@ceph.osd.9.smithi012.stdout:Sep 11 00:34:33 smithi012 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-9[66733]: 2024-09-11T00:34:33.237+0000 7f5f4e800640 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-11T00:33:41.710055+0000 front 2024-09-11T00:33:41.709869+0000 (oldest deadline 2024-09-11T00:34:07.010035+0000) 2024-09-11T00:34:33.563 INFO:journalctl@ceph.osd.9.smithi012.stdout:Sep 11 00:34:33 smithi012 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-9[66733]: 2024-09-11T00:34:33.237+0000 7f5f4e800640 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-09-11T00:33:47.010623+0000 front 2024-09-11T00:33:47.010590+0000 (oldest deadline 2024-09-11T00:34:12.910692+0000) 2024-09-11T00:34:33.563 INFO:journalctl@ceph.osd.9.smithi012.stdout:Sep 11 00:34:33 smithi012 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-9[66733]: 2024-09-11T00:34:33.237+0000 7f5f4e800640 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.165:6812 osd.4 since back 2024-09-11T00:33:59.213236+0000 front 2024-09-11T00:33:59.213312+0000 (oldest deadline 2024-09-11T00:34:24.513408+0000) 2024-09-11T00:34:33.564 INFO:journalctl@ceph.osd.9.smithi012.stdout:Sep 11 00:34:33 smithi012 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-9[66733]: 2024-09-11T00:34:33.237+0000 7f5f4e800640 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.178:6812 osd.5 since back 2024-09-11T00:34:04.514055+0000 front 2024-09-11T00:34:04.514150+0000 (oldest deadline 2024-09-11T00:34:29.813900+0000) 2024-09-11T00:34:33.675 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 11 00:34:33 smithi178 podman[79054]: 2024-09-11 00:34:33.37042614 +0000 UTC m=+5.715503896 container died cea9a534d126cc975408afc03574605178abf950c1a2cb5724e0fd3fc6d60f50 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-8, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-09-11T00:34:33.675 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 00:34:33 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-11[60852]: 2024-09-11T00:34:33.377+0000 7fd754b0b640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-09-11T00:33:24.081060+0000 front 2024-09-11T00:33:24.081080+0000 (oldest deadline 2024-09-11T00:33:49.980779+0000) 2024-09-11T00:34:33.675 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 00:34:33 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-11[60852]: 2024-09-11T00:34:33.377+0000 7fd754b0b640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-11T00:33:41.682818+0000 front 2024-09-11T00:33:41.682848+0000 (oldest deadline 2024-09-11T00:34:05.182914+0000) 2024-09-11T00:34:33.675 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 00:34:33 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-11[60852]: 2024-09-11T00:34:33.377+0000 7fd754b0b640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-09-11T00:33:47.484088+0000 front 2024-09-11T00:33:47.484103+0000 (oldest deadline 2024-09-11T00:34:12.784060+0000) 2024-09-11T00:34:33.676 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 00:34:33 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-11[60852]: 2024-09-11T00:34:33.377+0000 7fd754b0b640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.165:6812 osd.4 since back 2024-09-11T00:33:58.685349+0000 front 2024-09-11T00:33:58.685336+0000 (oldest deadline 2024-09-11T00:34:23.385413+0000) 2024-09-11T00:34:33.676 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 00:34:33 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-11[60852]: 2024-09-11T00:34:33.377+0000 7fd754b0b640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.178:6812 osd.5 since back 2024-09-11T00:34:07.887744+0000 front 2024-09-11T00:34:07.887694+0000 (oldest deadline 2024-09-11T00:34:28.987855+0000) 2024-09-11T00:34:34.007 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 11 00:34:33 smithi178 podman[79054]: 2024-09-11 00:34:33.77213964 +0000 UTC m=+6.117217468 container cleanup cea9a534d126cc975408afc03574605178abf950c1a2cb5724e0fd3fc6d60f50 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-8, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, ceph=True, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-09-11T00:34:34.007 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 11 00:34:33 smithi178 bash[79054]: ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-8 2024-09-11T00:34:34.291 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 11 00:34:34 smithi178 podman[79071]: 2024-09-11 00:34:34.006502246 +0000 UTC m=+0.629463550 container remove cea9a534d126cc975408afc03574605178abf950c1a2cb5724e0fd3fc6d60f50 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-8, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, ceph=True, GIT_BRANCH=HEAD) 2024-09-11T00:34:34.532 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 11 00:34:34 smithi165 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-10[59927]: 2024-09-11T00:34:34.054+0000 7f2cff41d640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-09-11T00:33:25.146772+0000 front 2024-09-11T00:33:25.146620+0000 (oldest deadline 2024-09-11T00:33:49.846529+0000) 2024-09-11T00:34:34.533 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 11 00:34:34 smithi165 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-10[59927]: 2024-09-11T00:34:34.054+0000 7f2cff41d640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-11T00:33:32.747514+0000 front 2024-09-11T00:33:32.747491+0000 (oldest deadline 2024-09-11T00:33:58.047686+0000) 2024-09-11T00:34:34.533 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 11 00:34:34 smithi165 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-10[59927]: 2024-09-11T00:34:34.054+0000 7f2cff41d640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-11T00:33:41.548749+0000 front 2024-09-11T00:33:41.548701+0000 (oldest deadline 2024-09-11T00:34:05.648724+0000) 2024-09-11T00:34:34.533 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 11 00:34:34 smithi165 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-10[59927]: 2024-09-11T00:34:34.054+0000 7f2cff41d640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-09-11T00:33:50.850789+0000 front 2024-09-11T00:33:50.850826+0000 (oldest deadline 2024-09-11T00:34:11.950647+0000) 2024-09-11T00:34:34.533 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 11 00:34:34 smithi165 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-10[59927]: 2024-09-11T00:34:34.054+0000 7f2cff41d640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.165:6812 osd.4 since back 2024-09-11T00:33:55.951809+0000 front 2024-09-11T00:33:55.951787+0000 (oldest deadline 2024-09-11T00:34:21.851850+0000) 2024-09-11T00:34:34.533 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 11 00:34:34 smithi165 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-10[59927]: 2024-09-11T00:34:34.054+0000 7f2cff41d640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.178:6812 osd.5 since back 2024-09-11T00:34:07.752773+0000 front 2024-09-11T00:34:07.752884+0000 (oldest deadline 2024-09-11T00:34:28.852602+0000) 2024-09-11T00:34:34.553 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 11 00:34:34 smithi178 podman[79157]: 2024-09-11 00:34:34.290638164 +0000 UTC m=+0.092991387 container create af07fc3bb250822495d55ed38ce13a0ac33d75542fedb07e3ab5d7a0c90b9a7c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-8-deactivate, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-11T00:34:34.553 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 11 00:34:34 smithi178 podman[79157]: 2024-09-11 00:34:34.213335126 +0000 UTC m=+0.015688357 image pull 589dd18248e494ba411dc838aa4ae0e5e569213f08fcd7a2cf22cd3ef7e3c480 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e 2024-09-11T00:34:34.553 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 00:34:34 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-11[60852]: 2024-09-11T00:34:34.386+0000 7fd754b0b640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-09-11T00:33:24.081060+0000 front 2024-09-11T00:33:24.081080+0000 (oldest deadline 2024-09-11T00:33:49.980779+0000) 2024-09-11T00:34:34.553 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 00:34:34 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-11[60852]: 2024-09-11T00:34:34.387+0000 7fd754b0b640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-11T00:33:41.682818+0000 front 2024-09-11T00:33:41.682848+0000 (oldest deadline 2024-09-11T00:34:05.182914+0000) 2024-09-11T00:34:34.553 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 00:34:34 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-11[60852]: 2024-09-11T00:34:34.387+0000 7fd754b0b640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-09-11T00:33:47.484088+0000 front 2024-09-11T00:33:47.484103+0000 (oldest deadline 2024-09-11T00:34:12.784060+0000) 2024-09-11T00:34:34.553 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 00:34:34 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-11[60852]: 2024-09-11T00:34:34.387+0000 7fd754b0b640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.165:6812 osd.4 since back 2024-09-11T00:33:58.685349+0000 front 2024-09-11T00:33:58.685336+0000 (oldest deadline 2024-09-11T00:34:23.385413+0000) 2024-09-11T00:34:34.553 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 00:34:34 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-11[60852]: 2024-09-11T00:34:34.387+0000 7fd754b0b640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.178:6812 osd.5 since back 2024-09-11T00:34:07.887744+0000 front 2024-09-11T00:34:07.887694+0000 (oldest deadline 2024-09-11T00:34:28.987855+0000) 2024-09-11T00:34:34.563 INFO:journalctl@ceph.osd.9.smithi012.stdout:Sep 11 00:34:34 smithi012 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-9[66733]: 2024-09-11T00:34:34.201+0000 7f5f4e800640 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-09-11T00:33:25.406156+0000 front 2024-09-11T00:33:25.406095+0000 (oldest deadline 2024-09-11T00:33:47.706239+0000) 2024-09-11T00:34:34.563 INFO:journalctl@ceph.osd.9.smithi012.stdout:Sep 11 00:34:34 smithi012 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-9[66733]: 2024-09-11T00:34:34.201+0000 7f5f4e800640 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-11T00:33:30.607371+0000 front 2024-09-11T00:33:30.607499+0000 (oldest deadline 2024-09-11T00:33:55.307466+0000) 2024-09-11T00:34:34.563 INFO:journalctl@ceph.osd.9.smithi012.stdout:Sep 11 00:34:34 smithi012 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-9[66733]: 2024-09-11T00:34:34.201+0000 7f5f4e800640 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-11T00:33:41.710055+0000 front 2024-09-11T00:33:41.709869+0000 (oldest deadline 2024-09-11T00:34:07.010035+0000) 2024-09-11T00:34:34.563 INFO:journalctl@ceph.osd.9.smithi012.stdout:Sep 11 00:34:34 smithi012 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-9[66733]: 2024-09-11T00:34:34.201+0000 7f5f4e800640 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-09-11T00:33:47.010623+0000 front 2024-09-11T00:33:47.010590+0000 (oldest deadline 2024-09-11T00:34:12.910692+0000) 2024-09-11T00:34:34.564 INFO:journalctl@ceph.osd.9.smithi012.stdout:Sep 11 00:34:34 smithi012 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-9[66733]: 2024-09-11T00:34:34.201+0000 7f5f4e800640 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.165:6812 osd.4 since back 2024-09-11T00:33:59.213236+0000 front 2024-09-11T00:33:59.213312+0000 (oldest deadline 2024-09-11T00:34:24.513408+0000) 2024-09-11T00:34:34.564 INFO:journalctl@ceph.osd.9.smithi012.stdout:Sep 11 00:34:34 smithi012 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-9[66733]: 2024-09-11T00:34:34.201+0000 7f5f4e800640 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.178:6812 osd.5 since back 2024-09-11T00:34:04.514055+0000 front 2024-09-11T00:34:04.514150+0000 (oldest deadline 2024-09-11T00:34:29.813900+0000) 2024-09-11T00:34:34.831 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 11 00:34:34 smithi178 podman[79157]: 2024-09-11 00:34:34.5516605 +0000 UTC m=+0.354013735 container init af07fc3bb250822495d55ed38ce13a0ac33d75542fedb07e3ab5d7a0c90b9a7c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-8-deactivate, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731) 2024-09-11T00:34:34.832 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 11 00:34:34 smithi178 podman[79157]: 2024-09-11 00:34:34.555231166 +0000 UTC m=+0.357584393 container start af07fc3bb250822495d55ed38ce13a0ac33d75542fedb07e3ab5d7a0c90b9a7c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-8-deactivate, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, ceph=True, io.buildah.version=1.37.2) 2024-09-11T00:34:34.832 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 11 00:34:34 smithi178 podman[79157]: 2024-09-11 00:34:34.685247747 +0000 UTC m=+0.487600994 container attach af07fc3bb250822495d55ed38ce13a0ac33d75542fedb07e3ab5d7a0c90b9a7c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-8-deactivate, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, GIT_CLEAN=True, GIT_BRANCH=HEAD) 2024-09-11T00:34:34.832 INFO:journalctl@ceph.osd.8.smithi178.stdout:Sep 11 00:34:34 smithi178 podman[79157]: 2024-09-11 00:34:34.739274601 +0000 UTC m=+0.541627826 container died af07fc3bb250822495d55ed38ce13a0ac33d75542fedb07e3ab5d7a0c90b9a7c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-8-deactivate, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, ceph=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-11T00:34:35.282 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 11 00:34:35 smithi165 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-10[59927]: 2024-09-11T00:34:35.023+0000 7f2cff41d640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-09-11T00:33:25.146772+0000 front 2024-09-11T00:33:25.146620+0000 (oldest deadline 2024-09-11T00:33:49.846529+0000) 2024-09-11T00:34:35.283 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 11 00:34:35 smithi165 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-10[59927]: 2024-09-11T00:34:35.023+0000 7f2cff41d640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-11T00:33:32.747514+0000 front 2024-09-11T00:33:32.747491+0000 (oldest deadline 2024-09-11T00:33:58.047686+0000) 2024-09-11T00:34:35.283 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 11 00:34:35 smithi165 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-10[59927]: 2024-09-11T00:34:35.023+0000 7f2cff41d640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-11T00:33:41.548749+0000 front 2024-09-11T00:33:41.548701+0000 (oldest deadline 2024-09-11T00:34:05.648724+0000) 2024-09-11T00:34:35.283 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 11 00:34:35 smithi165 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-10[59927]: 2024-09-11T00:34:35.023+0000 7f2cff41d640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-09-11T00:33:50.850789+0000 front 2024-09-11T00:33:50.850826+0000 (oldest deadline 2024-09-11T00:34:11.950647+0000) 2024-09-11T00:34:35.283 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 11 00:34:35 smithi165 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-10[59927]: 2024-09-11T00:34:35.023+0000 7f2cff41d640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.165:6812 osd.4 since back 2024-09-11T00:33:55.951809+0000 front 2024-09-11T00:33:55.951787+0000 (oldest deadline 2024-09-11T00:34:21.851850+0000) 2024-09-11T00:34:35.283 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 11 00:34:35 smithi165 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-10[59927]: 2024-09-11T00:34:35.023+0000 7f2cff41d640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.178:6812 osd.5 since back 2024-09-11T00:34:07.752773+0000 front 2024-09-11T00:34:07.752884+0000 (oldest deadline 2024-09-11T00:34:28.852602+0000) 2024-09-11T00:34:35.388 DEBUG:teuthology.orchestra.run.smithi178:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968@osd.8.service' 2024-09-11T00:34:35.469 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T00:34:35.469 INFO:tasks.cephadm.osd.8:Stopped osd.8 2024-09-11T00:34:35.469 INFO:tasks.cephadm.osd.9:Stopping osd.9... 2024-09-11T00:34:35.469 DEBUG:teuthology.orchestra.run.smithi012:> sudo systemctl stop ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968@osd.9 2024-09-11T00:34:35.524 INFO:journalctl@ceph.osd.9.smithi012.stdout:Sep 11 00:34:35 smithi012 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-9[66733]: 2024-09-11T00:34:35.223+0000 7f5f4e800640 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-09-11T00:33:25.406156+0000 front 2024-09-11T00:33:25.406095+0000 (oldest deadline 2024-09-11T00:33:47.706239+0000) 2024-09-11T00:34:35.524 INFO:journalctl@ceph.osd.9.smithi012.stdout:Sep 11 00:34:35 smithi012 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-9[66733]: 2024-09-11T00:34:35.223+0000 7f5f4e800640 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-11T00:33:30.607371+0000 front 2024-09-11T00:33:30.607499+0000 (oldest deadline 2024-09-11T00:33:55.307466+0000) 2024-09-11T00:34:35.524 INFO:journalctl@ceph.osd.9.smithi012.stdout:Sep 11 00:34:35 smithi012 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-9[66733]: 2024-09-11T00:34:35.223+0000 7f5f4e800640 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-11T00:33:41.710055+0000 front 2024-09-11T00:33:41.709869+0000 (oldest deadline 2024-09-11T00:34:07.010035+0000) 2024-09-11T00:34:35.524 INFO:journalctl@ceph.osd.9.smithi012.stdout:Sep 11 00:34:35 smithi012 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-9[66733]: 2024-09-11T00:34:35.223+0000 7f5f4e800640 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-09-11T00:33:47.010623+0000 front 2024-09-11T00:33:47.010590+0000 (oldest deadline 2024-09-11T00:34:12.910692+0000) 2024-09-11T00:34:35.524 INFO:journalctl@ceph.osd.9.smithi012.stdout:Sep 11 00:34:35 smithi012 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-9[66733]: 2024-09-11T00:34:35.223+0000 7f5f4e800640 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.165:6812 osd.4 since back 2024-09-11T00:33:59.213236+0000 front 2024-09-11T00:33:59.213312+0000 (oldest deadline 2024-09-11T00:34:24.513408+0000) 2024-09-11T00:34:35.524 INFO:journalctl@ceph.osd.9.smithi012.stdout:Sep 11 00:34:35 smithi012 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-9[66733]: 2024-09-11T00:34:35.223+0000 7f5f4e800640 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.178:6812 osd.5 since back 2024-09-11T00:34:04.514055+0000 front 2024-09-11T00:34:04.514150+0000 (oldest deadline 2024-09-11T00:34:29.813900+0000) 2024-09-11T00:34:35.813 INFO:journalctl@ceph.osd.9.smithi012.stdout:Sep 11 00:34:35 smithi012 systemd[1]: Stopping Ceph osd.9 for ff3ad74a-6fcf-11ef-bcea-c7b262605968... 2024-09-11T00:34:35.831 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 00:34:35 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-11[60852]: 2024-09-11T00:34:35.388+0000 7fd754b0b640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-09-11T00:33:24.081060+0000 front 2024-09-11T00:33:24.081080+0000 (oldest deadline 2024-09-11T00:33:49.980779+0000) 2024-09-11T00:34:35.831 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 00:34:35 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-11[60852]: 2024-09-11T00:34:35.388+0000 7fd754b0b640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-11T00:33:41.682818+0000 front 2024-09-11T00:33:41.682848+0000 (oldest deadline 2024-09-11T00:34:05.182914+0000) 2024-09-11T00:34:35.831 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 00:34:35 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-11[60852]: 2024-09-11T00:34:35.388+0000 7fd754b0b640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-09-11T00:33:47.484088+0000 front 2024-09-11T00:33:47.484103+0000 (oldest deadline 2024-09-11T00:34:12.784060+0000) 2024-09-11T00:34:35.831 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 00:34:35 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-11[60852]: 2024-09-11T00:34:35.388+0000 7fd754b0b640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.165:6812 osd.4 since back 2024-09-11T00:33:58.685349+0000 front 2024-09-11T00:33:58.685336+0000 (oldest deadline 2024-09-11T00:34:23.385413+0000) 2024-09-11T00:34:35.832 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 00:34:35 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-11[60852]: 2024-09-11T00:34:35.388+0000 7fd754b0b640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.178:6812 osd.5 since back 2024-09-11T00:34:07.887744+0000 front 2024-09-11T00:34:07.887694+0000 (oldest deadline 2024-09-11T00:34:28.987855+0000) 2024-09-11T00:34:36.237 INFO:journalctl@ceph.osd.9.smithi012.stdout:Sep 11 00:34:35 smithi012 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-9[66733]: 2024-09-11T00:34:35.900+0000 7f5f57e00640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.9 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T00:34:36.237 INFO:journalctl@ceph.osd.9.smithi012.stdout:Sep 11 00:34:35 smithi012 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-9[66733]: 2024-09-11T00:34:35.900+0000 7f5f57e00640 -1 osd.9 91 *** Got signal Terminated *** 2024-09-11T00:34:36.238 INFO:journalctl@ceph.osd.9.smithi012.stdout:Sep 11 00:34:35 smithi012 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-9[66733]: 2024-09-11T00:34:35.900+0000 7f5f57e00640 -1 osd.9 91 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-11T00:34:36.283 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 11 00:34:36 smithi165 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-10[59927]: 2024-09-11T00:34:36.003+0000 7f2cff41d640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-09-11T00:33:25.146772+0000 front 2024-09-11T00:33:25.146620+0000 (oldest deadline 2024-09-11T00:33:49.846529+0000) 2024-09-11T00:34:36.283 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 11 00:34:36 smithi165 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-10[59927]: 2024-09-11T00:34:36.003+0000 7f2cff41d640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-11T00:33:32.747514+0000 front 2024-09-11T00:33:32.747491+0000 (oldest deadline 2024-09-11T00:33:58.047686+0000) 2024-09-11T00:34:36.283 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 11 00:34:36 smithi165 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-10[59927]: 2024-09-11T00:34:36.003+0000 7f2cff41d640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-11T00:33:41.548749+0000 front 2024-09-11T00:33:41.548701+0000 (oldest deadline 2024-09-11T00:34:05.648724+0000) 2024-09-11T00:34:36.283 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 11 00:34:36 smithi165 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-10[59927]: 2024-09-11T00:34:36.003+0000 7f2cff41d640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-09-11T00:33:50.850789+0000 front 2024-09-11T00:33:50.850826+0000 (oldest deadline 2024-09-11T00:34:11.950647+0000) 2024-09-11T00:34:36.283 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 11 00:34:36 smithi165 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-10[59927]: 2024-09-11T00:34:36.003+0000 7f2cff41d640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.165:6812 osd.4 since back 2024-09-11T00:33:55.951809+0000 front 2024-09-11T00:33:55.951787+0000 (oldest deadline 2024-09-11T00:34:21.851850+0000) 2024-09-11T00:34:36.284 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 11 00:34:36 smithi165 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-10[59927]: 2024-09-11T00:34:36.003+0000 7f2cff41d640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.178:6812 osd.5 since back 2024-09-11T00:34:07.752773+0000 front 2024-09-11T00:34:07.752884+0000 (oldest deadline 2024-09-11T00:34:28.852602+0000) 2024-09-11T00:34:36.563 INFO:journalctl@ceph.osd.9.smithi012.stdout:Sep 11 00:34:36 smithi012 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-9[66733]: 2024-09-11T00:34:36.235+0000 7f5f4e800640 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-09-11T00:33:25.406156+0000 front 2024-09-11T00:33:25.406095+0000 (oldest deadline 2024-09-11T00:33:47.706239+0000) 2024-09-11T00:34:36.563 INFO:journalctl@ceph.osd.9.smithi012.stdout:Sep 11 00:34:36 smithi012 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-9[66733]: 2024-09-11T00:34:36.235+0000 7f5f4e800640 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-11T00:33:30.607371+0000 front 2024-09-11T00:33:30.607499+0000 (oldest deadline 2024-09-11T00:33:55.307466+0000) 2024-09-11T00:34:36.563 INFO:journalctl@ceph.osd.9.smithi012.stdout:Sep 11 00:34:36 smithi012 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-9[66733]: 2024-09-11T00:34:36.235+0000 7f5f4e800640 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-11T00:33:41.710055+0000 front 2024-09-11T00:33:41.709869+0000 (oldest deadline 2024-09-11T00:34:07.010035+0000) 2024-09-11T00:34:36.563 INFO:journalctl@ceph.osd.9.smithi012.stdout:Sep 11 00:34:36 smithi012 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-9[66733]: 2024-09-11T00:34:36.235+0000 7f5f4e800640 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-09-11T00:33:47.010623+0000 front 2024-09-11T00:33:47.010590+0000 (oldest deadline 2024-09-11T00:34:12.910692+0000) 2024-09-11T00:34:36.564 INFO:journalctl@ceph.osd.9.smithi012.stdout:Sep 11 00:34:36 smithi012 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-9[66733]: 2024-09-11T00:34:36.235+0000 7f5f4e800640 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.165:6812 osd.4 since back 2024-09-11T00:33:59.213236+0000 front 2024-09-11T00:33:59.213312+0000 (oldest deadline 2024-09-11T00:34:24.513408+0000) 2024-09-11T00:34:36.564 INFO:journalctl@ceph.osd.9.smithi012.stdout:Sep 11 00:34:36 smithi012 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-9[66733]: 2024-09-11T00:34:36.235+0000 7f5f4e800640 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.178:6812 osd.5 since back 2024-09-11T00:34:04.514055+0000 front 2024-09-11T00:34:04.514150+0000 (oldest deadline 2024-09-11T00:34:29.813900+0000) 2024-09-11T00:34:36.831 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 00:34:36 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-11[60852]: 2024-09-11T00:34:36.385+0000 7fd754b0b640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-09-11T00:33:24.081060+0000 front 2024-09-11T00:33:24.081080+0000 (oldest deadline 2024-09-11T00:33:49.980779+0000) 2024-09-11T00:34:36.832 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 00:34:36 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-11[60852]: 2024-09-11T00:34:36.385+0000 7fd754b0b640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-11T00:33:41.682818+0000 front 2024-09-11T00:33:41.682848+0000 (oldest deadline 2024-09-11T00:34:05.182914+0000) 2024-09-11T00:34:36.832 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 00:34:36 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-11[60852]: 2024-09-11T00:34:36.385+0000 7fd754b0b640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-09-11T00:33:47.484088+0000 front 2024-09-11T00:33:47.484103+0000 (oldest deadline 2024-09-11T00:34:12.784060+0000) 2024-09-11T00:34:36.832 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 00:34:36 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-11[60852]: 2024-09-11T00:34:36.385+0000 7fd754b0b640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.165:6812 osd.4 since back 2024-09-11T00:33:58.685349+0000 front 2024-09-11T00:33:58.685336+0000 (oldest deadline 2024-09-11T00:34:23.385413+0000) 2024-09-11T00:34:36.832 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 00:34:36 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-11[60852]: 2024-09-11T00:34:36.385+0000 7fd754b0b640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.178:6812 osd.5 since back 2024-09-11T00:34:07.887744+0000 front 2024-09-11T00:34:07.887694+0000 (oldest deadline 2024-09-11T00:34:28.987855+0000) 2024-09-11T00:34:37.283 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 11 00:34:37 smithi165 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-10[59927]: 2024-09-11T00:34:37.005+0000 7f2cff41d640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-09-11T00:33:25.146772+0000 front 2024-09-11T00:33:25.146620+0000 (oldest deadline 2024-09-11T00:33:49.846529+0000) 2024-09-11T00:34:37.283 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 11 00:34:37 smithi165 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-10[59927]: 2024-09-11T00:34:37.005+0000 7f2cff41d640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-11T00:33:32.747514+0000 front 2024-09-11T00:33:32.747491+0000 (oldest deadline 2024-09-11T00:33:58.047686+0000) 2024-09-11T00:34:37.283 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 11 00:34:37 smithi165 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-10[59927]: 2024-09-11T00:34:37.005+0000 7f2cff41d640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-11T00:33:41.548749+0000 front 2024-09-11T00:33:41.548701+0000 (oldest deadline 2024-09-11T00:34:05.648724+0000) 2024-09-11T00:34:37.283 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 11 00:34:37 smithi165 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-10[59927]: 2024-09-11T00:34:37.005+0000 7f2cff41d640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-09-11T00:33:50.850789+0000 front 2024-09-11T00:33:50.850826+0000 (oldest deadline 2024-09-11T00:34:11.950647+0000) 2024-09-11T00:34:37.283 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 11 00:34:37 smithi165 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-10[59927]: 2024-09-11T00:34:37.005+0000 7f2cff41d640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.165:6812 osd.4 since back 2024-09-11T00:33:55.951809+0000 front 2024-09-11T00:33:55.951787+0000 (oldest deadline 2024-09-11T00:34:21.851850+0000) 2024-09-11T00:34:37.283 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 11 00:34:37 smithi165 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-10[59927]: 2024-09-11T00:34:37.005+0000 7f2cff41d640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.178:6812 osd.5 since back 2024-09-11T00:34:07.752773+0000 front 2024-09-11T00:34:07.752884+0000 (oldest deadline 2024-09-11T00:34:28.852602+0000) 2024-09-11T00:34:37.563 INFO:journalctl@ceph.osd.9.smithi012.stdout:Sep 11 00:34:37 smithi012 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-9[66733]: 2024-09-11T00:34:37.266+0000 7f5f4e800640 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-09-11T00:33:25.406156+0000 front 2024-09-11T00:33:25.406095+0000 (oldest deadline 2024-09-11T00:33:47.706239+0000) 2024-09-11T00:34:37.563 INFO:journalctl@ceph.osd.9.smithi012.stdout:Sep 11 00:34:37 smithi012 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-9[66733]: 2024-09-11T00:34:37.266+0000 7f5f4e800640 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-11T00:33:30.607371+0000 front 2024-09-11T00:33:30.607499+0000 (oldest deadline 2024-09-11T00:33:55.307466+0000) 2024-09-11T00:34:37.563 INFO:journalctl@ceph.osd.9.smithi012.stdout:Sep 11 00:34:37 smithi012 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-9[66733]: 2024-09-11T00:34:37.266+0000 7f5f4e800640 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-11T00:33:41.710055+0000 front 2024-09-11T00:33:41.709869+0000 (oldest deadline 2024-09-11T00:34:07.010035+0000) 2024-09-11T00:34:37.564 INFO:journalctl@ceph.osd.9.smithi012.stdout:Sep 11 00:34:37 smithi012 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-9[66733]: 2024-09-11T00:34:37.266+0000 7f5f4e800640 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-09-11T00:33:47.010623+0000 front 2024-09-11T00:33:47.010590+0000 (oldest deadline 2024-09-11T00:34:12.910692+0000) 2024-09-11T00:34:37.564 INFO:journalctl@ceph.osd.9.smithi012.stdout:Sep 11 00:34:37 smithi012 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-9[66733]: 2024-09-11T00:34:37.266+0000 7f5f4e800640 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.165:6812 osd.4 since back 2024-09-11T00:33:59.213236+0000 front 2024-09-11T00:33:59.213312+0000 (oldest deadline 2024-09-11T00:34:24.513408+0000) 2024-09-11T00:34:37.564 INFO:journalctl@ceph.osd.9.smithi012.stdout:Sep 11 00:34:37 smithi012 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-9[66733]: 2024-09-11T00:34:37.266+0000 7f5f4e800640 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.178:6812 osd.5 since back 2024-09-11T00:34:04.514055+0000 front 2024-09-11T00:34:04.514150+0000 (oldest deadline 2024-09-11T00:34:29.813900+0000) 2024-09-11T00:34:37.831 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 00:34:37 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-11[60852]: 2024-09-11T00:34:37.365+0000 7fd754b0b640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-09-11T00:33:24.081060+0000 front 2024-09-11T00:33:24.081080+0000 (oldest deadline 2024-09-11T00:33:49.980779+0000) 2024-09-11T00:34:37.831 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 00:34:37 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-11[60852]: 2024-09-11T00:34:37.365+0000 7fd754b0b640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-11T00:33:41.682818+0000 front 2024-09-11T00:33:41.682848+0000 (oldest deadline 2024-09-11T00:34:05.182914+0000) 2024-09-11T00:34:37.832 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 00:34:37 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-11[60852]: 2024-09-11T00:34:37.365+0000 7fd754b0b640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-09-11T00:33:47.484088+0000 front 2024-09-11T00:33:47.484103+0000 (oldest deadline 2024-09-11T00:34:12.784060+0000) 2024-09-11T00:34:37.832 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 00:34:37 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-11[60852]: 2024-09-11T00:34:37.365+0000 7fd754b0b640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.165:6812 osd.4 since back 2024-09-11T00:33:58.685349+0000 front 2024-09-11T00:33:58.685336+0000 (oldest deadline 2024-09-11T00:34:23.385413+0000) 2024-09-11T00:34:37.832 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 00:34:37 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-11[60852]: 2024-09-11T00:34:37.365+0000 7fd754b0b640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.178:6812 osd.5 since back 2024-09-11T00:34:07.887744+0000 front 2024-09-11T00:34:07.887694+0000 (oldest deadline 2024-09-11T00:34:28.987855+0000) 2024-09-11T00:34:37.832 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 00:34:37 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-11[60852]: 2024-09-11T00:34:37.365+0000 7fd754b0b640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.12:6822 osd.6 since back 2024-09-11T00:34:11.189386+0000 front 2024-09-11T00:34:11.189460+0000 (oldest deadline 2024-09-11T00:34:37.089169+0000) 2024-09-11T00:34:38.283 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 11 00:34:37 smithi165 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-10[59927]: 2024-09-11T00:34:37.967+0000 7f2cff41d640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-09-11T00:33:25.146772+0000 front 2024-09-11T00:33:25.146620+0000 (oldest deadline 2024-09-11T00:33:49.846529+0000) 2024-09-11T00:34:38.283 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 11 00:34:37 smithi165 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-10[59927]: 2024-09-11T00:34:37.967+0000 7f2cff41d640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-11T00:33:32.747514+0000 front 2024-09-11T00:33:32.747491+0000 (oldest deadline 2024-09-11T00:33:58.047686+0000) 2024-09-11T00:34:38.283 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 11 00:34:37 smithi165 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-10[59927]: 2024-09-11T00:34:37.967+0000 7f2cff41d640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-11T00:33:41.548749+0000 front 2024-09-11T00:33:41.548701+0000 (oldest deadline 2024-09-11T00:34:05.648724+0000) 2024-09-11T00:34:38.283 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 11 00:34:37 smithi165 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-10[59927]: 2024-09-11T00:34:37.967+0000 7f2cff41d640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-09-11T00:33:50.850789+0000 front 2024-09-11T00:33:50.850826+0000 (oldest deadline 2024-09-11T00:34:11.950647+0000) 2024-09-11T00:34:38.283 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 11 00:34:37 smithi165 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-10[59927]: 2024-09-11T00:34:37.967+0000 7f2cff41d640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.165:6812 osd.4 since back 2024-09-11T00:33:55.951809+0000 front 2024-09-11T00:33:55.951787+0000 (oldest deadline 2024-09-11T00:34:21.851850+0000) 2024-09-11T00:34:38.283 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 11 00:34:37 smithi165 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-10[59927]: 2024-09-11T00:34:37.967+0000 7f2cff41d640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.178:6812 osd.5 since back 2024-09-11T00:34:07.752773+0000 front 2024-09-11T00:34:07.752884+0000 (oldest deadline 2024-09-11T00:34:28.852602+0000) 2024-09-11T00:34:38.563 INFO:journalctl@ceph.osd.9.smithi012.stdout:Sep 11 00:34:38 smithi012 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-9[66733]: 2024-09-11T00:34:38.263+0000 7f5f4e800640 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-09-11T00:33:25.406156+0000 front 2024-09-11T00:33:25.406095+0000 (oldest deadline 2024-09-11T00:33:47.706239+0000) 2024-09-11T00:34:38.563 INFO:journalctl@ceph.osd.9.smithi012.stdout:Sep 11 00:34:38 smithi012 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-9[66733]: 2024-09-11T00:34:38.263+0000 7f5f4e800640 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-11T00:33:30.607371+0000 front 2024-09-11T00:33:30.607499+0000 (oldest deadline 2024-09-11T00:33:55.307466+0000) 2024-09-11T00:34:38.563 INFO:journalctl@ceph.osd.9.smithi012.stdout:Sep 11 00:34:38 smithi012 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-9[66733]: 2024-09-11T00:34:38.263+0000 7f5f4e800640 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-11T00:33:41.710055+0000 front 2024-09-11T00:33:41.709869+0000 (oldest deadline 2024-09-11T00:34:07.010035+0000) 2024-09-11T00:34:38.564 INFO:journalctl@ceph.osd.9.smithi012.stdout:Sep 11 00:34:38 smithi012 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-9[66733]: 2024-09-11T00:34:38.263+0000 7f5f4e800640 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-09-11T00:33:47.010623+0000 front 2024-09-11T00:33:47.010590+0000 (oldest deadline 2024-09-11T00:34:12.910692+0000) 2024-09-11T00:34:38.564 INFO:journalctl@ceph.osd.9.smithi012.stdout:Sep 11 00:34:38 smithi012 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-9[66733]: 2024-09-11T00:34:38.263+0000 7f5f4e800640 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.165:6812 osd.4 since back 2024-09-11T00:33:59.213236+0000 front 2024-09-11T00:33:59.213312+0000 (oldest deadline 2024-09-11T00:34:24.513408+0000) 2024-09-11T00:34:38.564 INFO:journalctl@ceph.osd.9.smithi012.stdout:Sep 11 00:34:38 smithi012 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-9[66733]: 2024-09-11T00:34:38.263+0000 7f5f4e800640 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.178:6812 osd.5 since back 2024-09-11T00:34:04.514055+0000 front 2024-09-11T00:34:04.514150+0000 (oldest deadline 2024-09-11T00:34:29.813900+0000) 2024-09-11T00:34:38.581 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 00:34:38 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-11[60852]: 2024-09-11T00:34:38.321+0000 7fd754b0b640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-09-11T00:33:24.081060+0000 front 2024-09-11T00:33:24.081080+0000 (oldest deadline 2024-09-11T00:33:49.980779+0000) 2024-09-11T00:34:38.581 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 00:34:38 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-11[60852]: 2024-09-11T00:34:38.321+0000 7fd754b0b640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-11T00:33:41.682818+0000 front 2024-09-11T00:33:41.682848+0000 (oldest deadline 2024-09-11T00:34:05.182914+0000) 2024-09-11T00:34:38.582 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 00:34:38 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-11[60852]: 2024-09-11T00:34:38.321+0000 7fd754b0b640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-09-11T00:33:47.484088+0000 front 2024-09-11T00:33:47.484103+0000 (oldest deadline 2024-09-11T00:34:12.784060+0000) 2024-09-11T00:34:38.582 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 00:34:38 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-11[60852]: 2024-09-11T00:34:38.321+0000 7fd754b0b640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.165:6812 osd.4 since back 2024-09-11T00:33:58.685349+0000 front 2024-09-11T00:33:58.685336+0000 (oldest deadline 2024-09-11T00:34:23.385413+0000) 2024-09-11T00:34:38.582 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 00:34:38 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-11[60852]: 2024-09-11T00:34:38.321+0000 7fd754b0b640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.178:6812 osd.5 since back 2024-09-11T00:34:07.887744+0000 front 2024-09-11T00:34:07.887694+0000 (oldest deadline 2024-09-11T00:34:28.987855+0000) 2024-09-11T00:34:38.582 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 00:34:38 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-11[60852]: 2024-09-11T00:34:38.321+0000 7fd754b0b640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.12:6822 osd.6 since back 2024-09-11T00:34:11.189386+0000 front 2024-09-11T00:34:11.189460+0000 (oldest deadline 2024-09-11T00:34:37.089169+0000) 2024-09-11T00:34:39.283 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 11 00:34:38 smithi165 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-10[59927]: 2024-09-11T00:34:38.984+0000 7f2cff41d640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-09-11T00:33:25.146772+0000 front 2024-09-11T00:33:25.146620+0000 (oldest deadline 2024-09-11T00:33:49.846529+0000) 2024-09-11T00:34:39.283 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 11 00:34:38 smithi165 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-10[59927]: 2024-09-11T00:34:38.984+0000 7f2cff41d640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-11T00:33:32.747514+0000 front 2024-09-11T00:33:32.747491+0000 (oldest deadline 2024-09-11T00:33:58.047686+0000) 2024-09-11T00:34:39.283 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 11 00:34:38 smithi165 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-10[59927]: 2024-09-11T00:34:38.984+0000 7f2cff41d640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-11T00:33:41.548749+0000 front 2024-09-11T00:33:41.548701+0000 (oldest deadline 2024-09-11T00:34:05.648724+0000) 2024-09-11T00:34:39.283 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 11 00:34:38 smithi165 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-10[59927]: 2024-09-11T00:34:38.984+0000 7f2cff41d640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-09-11T00:33:50.850789+0000 front 2024-09-11T00:33:50.850826+0000 (oldest deadline 2024-09-11T00:34:11.950647+0000) 2024-09-11T00:34:39.283 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 11 00:34:38 smithi165 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-10[59927]: 2024-09-11T00:34:38.984+0000 7f2cff41d640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.165:6812 osd.4 since back 2024-09-11T00:33:55.951809+0000 front 2024-09-11T00:33:55.951787+0000 (oldest deadline 2024-09-11T00:34:21.851850+0000) 2024-09-11T00:34:39.283 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 11 00:34:38 smithi165 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-10[59927]: 2024-09-11T00:34:38.984+0000 7f2cff41d640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.178:6812 osd.5 since back 2024-09-11T00:34:07.752773+0000 front 2024-09-11T00:34:07.752884+0000 (oldest deadline 2024-09-11T00:34:28.852602+0000) 2024-09-11T00:34:39.283 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 11 00:34:38 smithi165 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-10[59927]: 2024-09-11T00:34:38.984+0000 7f2cff41d640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.12:6822 osd.6 since back 2024-09-11T00:34:14.753738+0000 front 2024-09-11T00:34:14.753757+0000 (oldest deadline 2024-09-11T00:34:38.853514+0000) 2024-09-11T00:34:39.563 INFO:journalctl@ceph.osd.9.smithi012.stdout:Sep 11 00:34:39 smithi012 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-9[66733]: 2024-09-11T00:34:39.250+0000 7f5f4e800640 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-09-11T00:33:25.406156+0000 front 2024-09-11T00:33:25.406095+0000 (oldest deadline 2024-09-11T00:33:47.706239+0000) 2024-09-11T00:34:39.563 INFO:journalctl@ceph.osd.9.smithi012.stdout:Sep 11 00:34:39 smithi012 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-9[66733]: 2024-09-11T00:34:39.250+0000 7f5f4e800640 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-11T00:33:30.607371+0000 front 2024-09-11T00:33:30.607499+0000 (oldest deadline 2024-09-11T00:33:55.307466+0000) 2024-09-11T00:34:39.563 INFO:journalctl@ceph.osd.9.smithi012.stdout:Sep 11 00:34:39 smithi012 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-9[66733]: 2024-09-11T00:34:39.250+0000 7f5f4e800640 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-11T00:33:41.710055+0000 front 2024-09-11T00:33:41.709869+0000 (oldest deadline 2024-09-11T00:34:07.010035+0000) 2024-09-11T00:34:39.563 INFO:journalctl@ceph.osd.9.smithi012.stdout:Sep 11 00:34:39 smithi012 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-9[66733]: 2024-09-11T00:34:39.250+0000 7f5f4e800640 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-09-11T00:33:47.010623+0000 front 2024-09-11T00:33:47.010590+0000 (oldest deadline 2024-09-11T00:34:12.910692+0000) 2024-09-11T00:34:39.563 INFO:journalctl@ceph.osd.9.smithi012.stdout:Sep 11 00:34:39 smithi012 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-9[66733]: 2024-09-11T00:34:39.250+0000 7f5f4e800640 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.165:6812 osd.4 since back 2024-09-11T00:33:59.213236+0000 front 2024-09-11T00:33:59.213312+0000 (oldest deadline 2024-09-11T00:34:24.513408+0000) 2024-09-11T00:34:39.563 INFO:journalctl@ceph.osd.9.smithi012.stdout:Sep 11 00:34:39 smithi012 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-9[66733]: 2024-09-11T00:34:39.250+0000 7f5f4e800640 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.178:6812 osd.5 since back 2024-09-11T00:34:04.514055+0000 front 2024-09-11T00:34:04.514150+0000 (oldest deadline 2024-09-11T00:34:29.813900+0000) 2024-09-11T00:34:39.581 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 00:34:39 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-11[60852]: 2024-09-11T00:34:39.304+0000 7fd754b0b640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-09-11T00:33:24.081060+0000 front 2024-09-11T00:33:24.081080+0000 (oldest deadline 2024-09-11T00:33:49.980779+0000) 2024-09-11T00:34:39.581 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 00:34:39 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-11[60852]: 2024-09-11T00:34:39.304+0000 7fd754b0b640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-11T00:33:41.682818+0000 front 2024-09-11T00:33:41.682848+0000 (oldest deadline 2024-09-11T00:34:05.182914+0000) 2024-09-11T00:34:39.581 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 00:34:39 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-11[60852]: 2024-09-11T00:34:39.304+0000 7fd754b0b640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-09-11T00:33:47.484088+0000 front 2024-09-11T00:33:47.484103+0000 (oldest deadline 2024-09-11T00:34:12.784060+0000) 2024-09-11T00:34:39.581 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 00:34:39 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-11[60852]: 2024-09-11T00:34:39.304+0000 7fd754b0b640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.165:6812 osd.4 since back 2024-09-11T00:33:58.685349+0000 front 2024-09-11T00:33:58.685336+0000 (oldest deadline 2024-09-11T00:34:23.385413+0000) 2024-09-11T00:34:39.581 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 00:34:39 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-11[60852]: 2024-09-11T00:34:39.304+0000 7fd754b0b640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.178:6812 osd.5 since back 2024-09-11T00:34:07.887744+0000 front 2024-09-11T00:34:07.887694+0000 (oldest deadline 2024-09-11T00:34:28.987855+0000) 2024-09-11T00:34:39.581 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 00:34:39 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-11[60852]: 2024-09-11T00:34:39.304+0000 7fd754b0b640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.12:6822 osd.6 since back 2024-09-11T00:34:11.189386+0000 front 2024-09-11T00:34:11.189460+0000 (oldest deadline 2024-09-11T00:34:37.089169+0000) 2024-09-11T00:34:40.283 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 11 00:34:40 smithi165 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-10[59927]: 2024-09-11T00:34:40.021+0000 7f2cff41d640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-09-11T00:33:25.146772+0000 front 2024-09-11T00:33:25.146620+0000 (oldest deadline 2024-09-11T00:33:49.846529+0000) 2024-09-11T00:34:40.283 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 11 00:34:40 smithi165 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-10[59927]: 2024-09-11T00:34:40.021+0000 7f2cff41d640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-11T00:33:32.747514+0000 front 2024-09-11T00:33:32.747491+0000 (oldest deadline 2024-09-11T00:33:58.047686+0000) 2024-09-11T00:34:40.283 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 11 00:34:40 smithi165 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-10[59927]: 2024-09-11T00:34:40.021+0000 7f2cff41d640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-11T00:33:41.548749+0000 front 2024-09-11T00:33:41.548701+0000 (oldest deadline 2024-09-11T00:34:05.648724+0000) 2024-09-11T00:34:40.283 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 11 00:34:40 smithi165 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-10[59927]: 2024-09-11T00:34:40.021+0000 7f2cff41d640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-09-11T00:33:50.850789+0000 front 2024-09-11T00:33:50.850826+0000 (oldest deadline 2024-09-11T00:34:11.950647+0000) 2024-09-11T00:34:40.283 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 11 00:34:40 smithi165 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-10[59927]: 2024-09-11T00:34:40.021+0000 7f2cff41d640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.165:6812 osd.4 since back 2024-09-11T00:33:55.951809+0000 front 2024-09-11T00:33:55.951787+0000 (oldest deadline 2024-09-11T00:34:21.851850+0000) 2024-09-11T00:34:40.283 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 11 00:34:40 smithi165 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-10[59927]: 2024-09-11T00:34:40.021+0000 7f2cff41d640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.178:6812 osd.5 since back 2024-09-11T00:34:07.752773+0000 front 2024-09-11T00:34:07.752884+0000 (oldest deadline 2024-09-11T00:34:28.852602+0000) 2024-09-11T00:34:40.283 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 11 00:34:40 smithi165 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-10[59927]: 2024-09-11T00:34:40.021+0000 7f2cff41d640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.12:6822 osd.6 since back 2024-09-11T00:34:14.753738+0000 front 2024-09-11T00:34:14.753757+0000 (oldest deadline 2024-09-11T00:34:38.853514+0000) 2024-09-11T00:34:40.563 INFO:journalctl@ceph.osd.9.smithi012.stdout:Sep 11 00:34:40 smithi012 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-9[66733]: 2024-09-11T00:34:40.284+0000 7f5f4e800640 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-09-11T00:33:25.406156+0000 front 2024-09-11T00:33:25.406095+0000 (oldest deadline 2024-09-11T00:33:47.706239+0000) 2024-09-11T00:34:40.564 INFO:journalctl@ceph.osd.9.smithi012.stdout:Sep 11 00:34:40 smithi012 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-9[66733]: 2024-09-11T00:34:40.284+0000 7f5f4e800640 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-11T00:33:30.607371+0000 front 2024-09-11T00:33:30.607499+0000 (oldest deadline 2024-09-11T00:33:55.307466+0000) 2024-09-11T00:34:40.564 INFO:journalctl@ceph.osd.9.smithi012.stdout:Sep 11 00:34:40 smithi012 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-9[66733]: 2024-09-11T00:34:40.284+0000 7f5f4e800640 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-11T00:33:41.710055+0000 front 2024-09-11T00:33:41.709869+0000 (oldest deadline 2024-09-11T00:34:07.010035+0000) 2024-09-11T00:34:40.564 INFO:journalctl@ceph.osd.9.smithi012.stdout:Sep 11 00:34:40 smithi012 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-9[66733]: 2024-09-11T00:34:40.284+0000 7f5f4e800640 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-09-11T00:33:47.010623+0000 front 2024-09-11T00:33:47.010590+0000 (oldest deadline 2024-09-11T00:34:12.910692+0000) 2024-09-11T00:34:40.564 INFO:journalctl@ceph.osd.9.smithi012.stdout:Sep 11 00:34:40 smithi012 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-9[66733]: 2024-09-11T00:34:40.284+0000 7f5f4e800640 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.165:6812 osd.4 since back 2024-09-11T00:33:59.213236+0000 front 2024-09-11T00:33:59.213312+0000 (oldest deadline 2024-09-11T00:34:24.513408+0000) 2024-09-11T00:34:40.564 INFO:journalctl@ceph.osd.9.smithi012.stdout:Sep 11 00:34:40 smithi012 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-9[66733]: 2024-09-11T00:34:40.284+0000 7f5f4e800640 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.178:6812 osd.5 since back 2024-09-11T00:34:04.514055+0000 front 2024-09-11T00:34:04.514150+0000 (oldest deadline 2024-09-11T00:34:29.813900+0000) 2024-09-11T00:34:40.831 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 00:34:40 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-11[60852]: 2024-09-11T00:34:40.339+0000 7fd754b0b640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-09-11T00:33:24.081060+0000 front 2024-09-11T00:33:24.081080+0000 (oldest deadline 2024-09-11T00:33:49.980779+0000) 2024-09-11T00:34:40.831 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 00:34:40 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-11[60852]: 2024-09-11T00:34:40.339+0000 7fd754b0b640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-11T00:33:41.682818+0000 front 2024-09-11T00:33:41.682848+0000 (oldest deadline 2024-09-11T00:34:05.182914+0000) 2024-09-11T00:34:40.831 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 00:34:40 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-11[60852]: 2024-09-11T00:34:40.339+0000 7fd754b0b640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-09-11T00:33:47.484088+0000 front 2024-09-11T00:33:47.484103+0000 (oldest deadline 2024-09-11T00:34:12.784060+0000) 2024-09-11T00:34:40.831 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 00:34:40 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-11[60852]: 2024-09-11T00:34:40.339+0000 7fd754b0b640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.165:6812 osd.4 since back 2024-09-11T00:33:58.685349+0000 front 2024-09-11T00:33:58.685336+0000 (oldest deadline 2024-09-11T00:34:23.385413+0000) 2024-09-11T00:34:40.832 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 00:34:40 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-11[60852]: 2024-09-11T00:34:40.339+0000 7fd754b0b640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.178:6812 osd.5 since back 2024-09-11T00:34:07.887744+0000 front 2024-09-11T00:34:07.887694+0000 (oldest deadline 2024-09-11T00:34:28.987855+0000) 2024-09-11T00:34:40.832 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 00:34:40 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-11[60852]: 2024-09-11T00:34:40.339+0000 7fd754b0b640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.12:6822 osd.6 since back 2024-09-11T00:34:11.189386+0000 front 2024-09-11T00:34:11.189460+0000 (oldest deadline 2024-09-11T00:34:37.089169+0000) 2024-09-11T00:34:41.533 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 11 00:34:41 smithi165 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-10[59927]: 2024-09-11T00:34:41.055+0000 7f2cff41d640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-09-11T00:33:25.146772+0000 front 2024-09-11T00:33:25.146620+0000 (oldest deadline 2024-09-11T00:33:49.846529+0000) 2024-09-11T00:34:41.533 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 11 00:34:41 smithi165 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-10[59927]: 2024-09-11T00:34:41.055+0000 7f2cff41d640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-11T00:33:32.747514+0000 front 2024-09-11T00:33:32.747491+0000 (oldest deadline 2024-09-11T00:33:58.047686+0000) 2024-09-11T00:34:41.533 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 11 00:34:41 smithi165 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-10[59927]: 2024-09-11T00:34:41.055+0000 7f2cff41d640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-11T00:33:41.548749+0000 front 2024-09-11T00:33:41.548701+0000 (oldest deadline 2024-09-11T00:34:05.648724+0000) 2024-09-11T00:34:41.533 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 11 00:34:41 smithi165 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-10[59927]: 2024-09-11T00:34:41.055+0000 7f2cff41d640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-09-11T00:33:50.850789+0000 front 2024-09-11T00:33:50.850826+0000 (oldest deadline 2024-09-11T00:34:11.950647+0000) 2024-09-11T00:34:41.533 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 11 00:34:41 smithi165 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-10[59927]: 2024-09-11T00:34:41.055+0000 7f2cff41d640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.165:6812 osd.4 since back 2024-09-11T00:33:55.951809+0000 front 2024-09-11T00:33:55.951787+0000 (oldest deadline 2024-09-11T00:34:21.851850+0000) 2024-09-11T00:34:41.533 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 11 00:34:41 smithi165 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-10[59927]: 2024-09-11T00:34:41.055+0000 7f2cff41d640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.178:6812 osd.5 since back 2024-09-11T00:34:07.752773+0000 front 2024-09-11T00:34:07.752884+0000 (oldest deadline 2024-09-11T00:34:28.852602+0000) 2024-09-11T00:34:41.533 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 11 00:34:41 smithi165 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-10[59927]: 2024-09-11T00:34:41.055+0000 7f2cff41d640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.12:6822 osd.6 since back 2024-09-11T00:34:14.753738+0000 front 2024-09-11T00:34:14.753757+0000 (oldest deadline 2024-09-11T00:34:38.853514+0000) 2024-09-11T00:34:41.831 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 00:34:41 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-11[60852]: 2024-09-11T00:34:41.383+0000 7fd754b0b640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-09-11T00:33:24.081060+0000 front 2024-09-11T00:33:24.081080+0000 (oldest deadline 2024-09-11T00:33:49.980779+0000) 2024-09-11T00:34:41.832 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 00:34:41 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-11[60852]: 2024-09-11T00:34:41.383+0000 7fd754b0b640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-11T00:33:41.682818+0000 front 2024-09-11T00:33:41.682848+0000 (oldest deadline 2024-09-11T00:34:05.182914+0000) 2024-09-11T00:34:41.832 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 00:34:41 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-11[60852]: 2024-09-11T00:34:41.383+0000 7fd754b0b640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-09-11T00:33:47.484088+0000 front 2024-09-11T00:33:47.484103+0000 (oldest deadline 2024-09-11T00:34:12.784060+0000) 2024-09-11T00:34:41.832 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 00:34:41 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-11[60852]: 2024-09-11T00:34:41.383+0000 7fd754b0b640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.165:6812 osd.4 since back 2024-09-11T00:33:58.685349+0000 front 2024-09-11T00:33:58.685336+0000 (oldest deadline 2024-09-11T00:34:23.385413+0000) 2024-09-11T00:34:41.832 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 00:34:41 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-11[60852]: 2024-09-11T00:34:41.383+0000 7fd754b0b640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.178:6812 osd.5 since back 2024-09-11T00:34:07.887744+0000 front 2024-09-11T00:34:07.887694+0000 (oldest deadline 2024-09-11T00:34:28.987855+0000) 2024-09-11T00:34:41.832 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 00:34:41 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-11[60852]: 2024-09-11T00:34:41.383+0000 7fd754b0b640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.12:6822 osd.6 since back 2024-09-11T00:34:11.189386+0000 front 2024-09-11T00:34:11.189460+0000 (oldest deadline 2024-09-11T00:34:37.089169+0000) 2024-09-11T00:34:41.874 INFO:journalctl@ceph.osd.9.smithi012.stdout:Sep 11 00:34:41 smithi012 podman[114824]: 2024-09-11 00:34:41.590890099 +0000 UTC m=+5.822704062 container died 5d28c3dc6dac0106451789b15aa3cd1b4874be59b38f22b6b845e8289e3c0f77 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-9, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, ceph=True, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD) 2024-09-11T00:34:42.299 INFO:journalctl@ceph.osd.9.smithi012.stdout:Sep 11 00:34:42 smithi012 podman[114824]: 2024-09-11 00:34:42.005236679 +0000 UTC m=+6.237050646 container cleanup 5d28c3dc6dac0106451789b15aa3cd1b4874be59b38f22b6b845e8289e3c0f77 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-9, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.37.2) 2024-09-11T00:34:42.299 INFO:journalctl@ceph.osd.9.smithi012.stdout:Sep 11 00:34:42 smithi012 bash[114824]: ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-9 2024-09-11T00:34:42.533 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 11 00:34:42 smithi165 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-10[59927]: 2024-09-11T00:34:42.078+0000 7f2cff41d640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-09-11T00:33:25.146772+0000 front 2024-09-11T00:33:25.146620+0000 (oldest deadline 2024-09-11T00:33:49.846529+0000) 2024-09-11T00:34:42.533 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 11 00:34:42 smithi165 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-10[59927]: 2024-09-11T00:34:42.078+0000 7f2cff41d640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-11T00:33:32.747514+0000 front 2024-09-11T00:33:32.747491+0000 (oldest deadline 2024-09-11T00:33:58.047686+0000) 2024-09-11T00:34:42.533 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 11 00:34:42 smithi165 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-10[59927]: 2024-09-11T00:34:42.078+0000 7f2cff41d640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-11T00:33:41.548749+0000 front 2024-09-11T00:33:41.548701+0000 (oldest deadline 2024-09-11T00:34:05.648724+0000) 2024-09-11T00:34:42.533 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 11 00:34:42 smithi165 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-10[59927]: 2024-09-11T00:34:42.078+0000 7f2cff41d640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-09-11T00:33:50.850789+0000 front 2024-09-11T00:33:50.850826+0000 (oldest deadline 2024-09-11T00:34:11.950647+0000) 2024-09-11T00:34:42.533 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 11 00:34:42 smithi165 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-10[59927]: 2024-09-11T00:34:42.078+0000 7f2cff41d640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.165:6812 osd.4 since back 2024-09-11T00:33:55.951809+0000 front 2024-09-11T00:33:55.951787+0000 (oldest deadline 2024-09-11T00:34:21.851850+0000) 2024-09-11T00:34:42.533 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 11 00:34:42 smithi165 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-10[59927]: 2024-09-11T00:34:42.078+0000 7f2cff41d640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.178:6812 osd.5 since back 2024-09-11T00:34:07.752773+0000 front 2024-09-11T00:34:07.752884+0000 (oldest deadline 2024-09-11T00:34:28.852602+0000) 2024-09-11T00:34:42.533 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 11 00:34:42 smithi165 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-10[59927]: 2024-09-11T00:34:42.078+0000 7f2cff41d640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.12:6822 osd.6 since back 2024-09-11T00:34:14.753738+0000 front 2024-09-11T00:34:14.753757+0000 (oldest deadline 2024-09-11T00:34:38.853514+0000) 2024-09-11T00:34:42.563 INFO:journalctl@ceph.osd.9.smithi012.stdout:Sep 11 00:34:42 smithi012 podman[114840]: 2024-09-11 00:34:42.298207466 +0000 UTC m=+0.692606412 container remove 5d28c3dc6dac0106451789b15aa3cd1b4874be59b38f22b6b845e8289e3c0f77 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-9, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, io.buildah.version=1.37.2, ceph=True, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-09-11T00:34:42.831 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 00:34:42 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-11[60852]: 2024-09-11T00:34:42.412+0000 7fd754b0b640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-09-11T00:33:24.081060+0000 front 2024-09-11T00:33:24.081080+0000 (oldest deadline 2024-09-11T00:33:49.980779+0000) 2024-09-11T00:34:42.831 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 00:34:42 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-11[60852]: 2024-09-11T00:34:42.412+0000 7fd754b0b640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-11T00:33:41.682818+0000 front 2024-09-11T00:33:41.682848+0000 (oldest deadline 2024-09-11T00:34:05.182914+0000) 2024-09-11T00:34:42.831 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 00:34:42 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-11[60852]: 2024-09-11T00:34:42.412+0000 7fd754b0b640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-09-11T00:33:47.484088+0000 front 2024-09-11T00:33:47.484103+0000 (oldest deadline 2024-09-11T00:34:12.784060+0000) 2024-09-11T00:34:42.831 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 00:34:42 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-11[60852]: 2024-09-11T00:34:42.412+0000 7fd754b0b640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.165:6812 osd.4 since back 2024-09-11T00:33:58.685349+0000 front 2024-09-11T00:33:58.685336+0000 (oldest deadline 2024-09-11T00:34:23.385413+0000) 2024-09-11T00:34:42.831 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 00:34:42 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-11[60852]: 2024-09-11T00:34:42.412+0000 7fd754b0b640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.178:6812 osd.5 since back 2024-09-11T00:34:07.887744+0000 front 2024-09-11T00:34:07.887694+0000 (oldest deadline 2024-09-11T00:34:28.987855+0000) 2024-09-11T00:34:42.832 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 00:34:42 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-11[60852]: 2024-09-11T00:34:42.412+0000 7fd754b0b640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.12:6822 osd.6 since back 2024-09-11T00:34:11.189386+0000 front 2024-09-11T00:34:11.189460+0000 (oldest deadline 2024-09-11T00:34:37.089169+0000) 2024-09-11T00:34:43.044 INFO:journalctl@ceph.osd.9.smithi012.stdout:Sep 11 00:34:42 smithi012 podman[114929]: 2024-09-11 00:34:42.641084497 +0000 UTC m=+0.027352490 image pull 589dd18248e494ba411dc838aa4ae0e5e569213f08fcd7a2cf22cd3ef7e3c480 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e 2024-09-11T00:34:43.044 INFO:journalctl@ceph.osd.9.smithi012.stdout:Sep 11 00:34:42 smithi012 podman[114929]: 2024-09-11 00:34:42.740850444 +0000 UTC m=+0.127118421 container create 966479ef5da17544c666f66794c7fe53a4466f3eb29b646710dc5733bb85c13e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-9-deactivate, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2) 2024-09-11T00:34:43.044 INFO:journalctl@ceph.osd.9.smithi012.stdout:Sep 11 00:34:42 smithi012 podman[114929]: 2024-09-11 00:34:42.959991798 +0000 UTC m=+0.346259782 container init 966479ef5da17544c666f66794c7fe53a4466f3eb29b646710dc5733bb85c13e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-9-deactivate, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=) 2024-09-11T00:34:43.044 INFO:journalctl@ceph.osd.9.smithi012.stdout:Sep 11 00:34:42 smithi012 podman[114929]: 2024-09-11 00:34:42.966726901 +0000 UTC m=+0.352994879 container start 966479ef5da17544c666f66794c7fe53a4466f3eb29b646710dc5733bb85c13e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-9-deactivate, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, maintainer=Guillaume Abrioux ) 2024-09-11T00:34:43.044 INFO:journalctl@ceph.osd.9.smithi012.stdout:Sep 11 00:34:43 smithi012 podman[114929]: 2024-09-11 00:34:43.042825876 +0000 UTC m=+0.429093868 container attach 966479ef5da17544c666f66794c7fe53a4466f3eb29b646710dc5733bb85c13e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-9-deactivate, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, ceph=True, GIT_BRANCH=HEAD) 2024-09-11T00:34:43.313 INFO:journalctl@ceph.osd.9.smithi012.stdout:Sep 11 00:34:43 smithi012 podman[114952]: 2024-09-11 00:34:43.261385131 +0000 UTC m=+0.026173557 container died 966479ef5da17544c666f66794c7fe53a4466f3eb29b646710dc5733bb85c13e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-9-deactivate, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.build-date=20240731, GIT_CLEAN=True, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-09-11T00:34:43.533 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 11 00:34:43 smithi165 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-10[59927]: 2024-09-11T00:34:43.074+0000 7f2cff41d640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-09-11T00:33:25.146772+0000 front 2024-09-11T00:33:25.146620+0000 (oldest deadline 2024-09-11T00:33:49.846529+0000) 2024-09-11T00:34:43.533 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 11 00:34:43 smithi165 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-10[59927]: 2024-09-11T00:34:43.074+0000 7f2cff41d640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-11T00:33:32.747514+0000 front 2024-09-11T00:33:32.747491+0000 (oldest deadline 2024-09-11T00:33:58.047686+0000) 2024-09-11T00:34:43.533 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 11 00:34:43 smithi165 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-10[59927]: 2024-09-11T00:34:43.074+0000 7f2cff41d640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-11T00:33:41.548749+0000 front 2024-09-11T00:33:41.548701+0000 (oldest deadline 2024-09-11T00:34:05.648724+0000) 2024-09-11T00:34:43.533 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 11 00:34:43 smithi165 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-10[59927]: 2024-09-11T00:34:43.074+0000 7f2cff41d640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-09-11T00:33:50.850789+0000 front 2024-09-11T00:33:50.850826+0000 (oldest deadline 2024-09-11T00:34:11.950647+0000) 2024-09-11T00:34:43.533 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 11 00:34:43 smithi165 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-10[59927]: 2024-09-11T00:34:43.074+0000 7f2cff41d640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.165:6812 osd.4 since back 2024-09-11T00:33:55.951809+0000 front 2024-09-11T00:33:55.951787+0000 (oldest deadline 2024-09-11T00:34:21.851850+0000) 2024-09-11T00:34:43.533 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 11 00:34:43 smithi165 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-10[59927]: 2024-09-11T00:34:43.074+0000 7f2cff41d640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.178:6812 osd.5 since back 2024-09-11T00:34:07.752773+0000 front 2024-09-11T00:34:07.752884+0000 (oldest deadline 2024-09-11T00:34:28.852602+0000) 2024-09-11T00:34:43.533 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 11 00:34:43 smithi165 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-10[59927]: 2024-09-11T00:34:43.074+0000 7f2cff41d640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.12:6822 osd.6 since back 2024-09-11T00:34:14.753738+0000 front 2024-09-11T00:34:14.753757+0000 (oldest deadline 2024-09-11T00:34:38.853514+0000) 2024-09-11T00:34:43.831 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 00:34:43 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-11[60852]: 2024-09-11T00:34:43.441+0000 7fd754b0b640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-09-11T00:33:24.081060+0000 front 2024-09-11T00:33:24.081080+0000 (oldest deadline 2024-09-11T00:33:49.980779+0000) 2024-09-11T00:34:43.831 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 00:34:43 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-11[60852]: 2024-09-11T00:34:43.441+0000 7fd754b0b640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-11T00:33:41.682818+0000 front 2024-09-11T00:33:41.682848+0000 (oldest deadline 2024-09-11T00:34:05.182914+0000) 2024-09-11T00:34:43.831 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 00:34:43 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-11[60852]: 2024-09-11T00:34:43.441+0000 7fd754b0b640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-09-11T00:33:47.484088+0000 front 2024-09-11T00:33:47.484103+0000 (oldest deadline 2024-09-11T00:34:12.784060+0000) 2024-09-11T00:34:43.831 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 00:34:43 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-11[60852]: 2024-09-11T00:34:43.441+0000 7fd754b0b640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.165:6812 osd.4 since back 2024-09-11T00:33:58.685349+0000 front 2024-09-11T00:33:58.685336+0000 (oldest deadline 2024-09-11T00:34:23.385413+0000) 2024-09-11T00:34:43.831 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 00:34:43 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-11[60852]: 2024-09-11T00:34:43.441+0000 7fd754b0b640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.178:6812 osd.5 since back 2024-09-11T00:34:07.887744+0000 front 2024-09-11T00:34:07.887694+0000 (oldest deadline 2024-09-11T00:34:28.987855+0000) 2024-09-11T00:34:43.832 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 00:34:43 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-11[60852]: 2024-09-11T00:34:43.441+0000 7fd754b0b640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.12:6822 osd.6 since back 2024-09-11T00:34:11.189386+0000 front 2024-09-11T00:34:11.189460+0000 (oldest deadline 2024-09-11T00:34:37.089169+0000) 2024-09-11T00:34:43.875 DEBUG:teuthology.orchestra.run.smithi012:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968@osd.9.service' 2024-09-11T00:34:43.951 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T00:34:43.951 INFO:tasks.cephadm.osd.9:Stopped osd.9 2024-09-11T00:34:43.951 INFO:tasks.cephadm.osd.10:Stopping osd.10... 2024-09-11T00:34:43.951 DEBUG:teuthology.orchestra.run.smithi165:> sudo systemctl stop ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968@osd.10 2024-09-11T00:34:44.268 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 11 00:34:44 smithi165 systemd[1]: Stopping Ceph osd.10 for ff3ad74a-6fcf-11ef-bcea-c7b262605968... 2024-09-11T00:34:44.268 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 11 00:34:44 smithi165 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-10[59927]: 2024-09-11T00:34:44.096+0000 7f2cff41d640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-09-11T00:33:25.146772+0000 front 2024-09-11T00:33:25.146620+0000 (oldest deadline 2024-09-11T00:33:49.846529+0000) 2024-09-11T00:34:44.268 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 11 00:34:44 smithi165 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-10[59927]: 2024-09-11T00:34:44.096+0000 7f2cff41d640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-11T00:33:32.747514+0000 front 2024-09-11T00:33:32.747491+0000 (oldest deadline 2024-09-11T00:33:58.047686+0000) 2024-09-11T00:34:44.268 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 11 00:34:44 smithi165 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-10[59927]: 2024-09-11T00:34:44.096+0000 7f2cff41d640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-11T00:33:41.548749+0000 front 2024-09-11T00:33:41.548701+0000 (oldest deadline 2024-09-11T00:34:05.648724+0000) 2024-09-11T00:34:44.268 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 11 00:34:44 smithi165 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-10[59927]: 2024-09-11T00:34:44.096+0000 7f2cff41d640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-09-11T00:33:50.850789+0000 front 2024-09-11T00:33:50.850826+0000 (oldest deadline 2024-09-11T00:34:11.950647+0000) 2024-09-11T00:34:44.268 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 11 00:34:44 smithi165 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-10[59927]: 2024-09-11T00:34:44.096+0000 7f2cff41d640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.165:6812 osd.4 since back 2024-09-11T00:33:55.951809+0000 front 2024-09-11T00:33:55.951787+0000 (oldest deadline 2024-09-11T00:34:21.851850+0000) 2024-09-11T00:34:44.269 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 11 00:34:44 smithi165 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-10[59927]: 2024-09-11T00:34:44.096+0000 7f2cff41d640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.178:6812 osd.5 since back 2024-09-11T00:34:07.752773+0000 front 2024-09-11T00:34:07.752884+0000 (oldest deadline 2024-09-11T00:34:28.852602+0000) 2024-09-11T00:34:44.269 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 11 00:34:44 smithi165 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-10[59927]: 2024-09-11T00:34:44.096+0000 7f2cff41d640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.12:6822 osd.6 since back 2024-09-11T00:34:14.753738+0000 front 2024-09-11T00:34:14.753757+0000 (oldest deadline 2024-09-11T00:34:38.853514+0000) 2024-09-11T00:34:44.269 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 11 00:34:44 smithi165 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-10[59927]: 2024-09-11T00:34:44.265+0000 7f2d02e08640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.10 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T00:34:44.269 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 11 00:34:44 smithi165 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-10[59927]: 2024-09-11T00:34:44.265+0000 7f2d02e08640 -1 osd.10 91 *** Got signal Terminated *** 2024-09-11T00:34:44.269 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 11 00:34:44 smithi165 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-10[59927]: 2024-09-11T00:34:44.265+0000 7f2d02e08640 -1 osd.10 91 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-11T00:34:44.831 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 00:34:44 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-11[60852]: 2024-09-11T00:34:44.423+0000 7fd754b0b640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-09-11T00:33:24.081060+0000 front 2024-09-11T00:33:24.081080+0000 (oldest deadline 2024-09-11T00:33:49.980779+0000) 2024-09-11T00:34:44.831 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 00:34:44 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-11[60852]: 2024-09-11T00:34:44.423+0000 7fd754b0b640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-11T00:33:41.682818+0000 front 2024-09-11T00:33:41.682848+0000 (oldest deadline 2024-09-11T00:34:05.182914+0000) 2024-09-11T00:34:44.831 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 00:34:44 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-11[60852]: 2024-09-11T00:34:44.423+0000 7fd754b0b640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-09-11T00:33:47.484088+0000 front 2024-09-11T00:33:47.484103+0000 (oldest deadline 2024-09-11T00:34:12.784060+0000) 2024-09-11T00:34:44.831 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 00:34:44 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-11[60852]: 2024-09-11T00:34:44.423+0000 7fd754b0b640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.165:6812 osd.4 since back 2024-09-11T00:33:58.685349+0000 front 2024-09-11T00:33:58.685336+0000 (oldest deadline 2024-09-11T00:34:23.385413+0000) 2024-09-11T00:34:44.832 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 00:34:44 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-11[60852]: 2024-09-11T00:34:44.423+0000 7fd754b0b640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.178:6812 osd.5 since back 2024-09-11T00:34:07.887744+0000 front 2024-09-11T00:34:07.887694+0000 (oldest deadline 2024-09-11T00:34:28.987855+0000) 2024-09-11T00:34:44.832 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 00:34:44 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-11[60852]: 2024-09-11T00:34:44.423+0000 7fd754b0b640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.12:6822 osd.6 since back 2024-09-11T00:34:11.189386+0000 front 2024-09-11T00:34:11.189460+0000 (oldest deadline 2024-09-11T00:34:37.089169+0000) 2024-09-11T00:34:45.533 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 11 00:34:45 smithi165 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-10[59927]: 2024-09-11T00:34:45.063+0000 7f2cff41d640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-09-11T00:33:25.146772+0000 front 2024-09-11T00:33:25.146620+0000 (oldest deadline 2024-09-11T00:33:49.846529+0000) 2024-09-11T00:34:45.533 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 11 00:34:45 smithi165 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-10[59927]: 2024-09-11T00:34:45.063+0000 7f2cff41d640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-11T00:33:32.747514+0000 front 2024-09-11T00:33:32.747491+0000 (oldest deadline 2024-09-11T00:33:58.047686+0000) 2024-09-11T00:34:45.533 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 11 00:34:45 smithi165 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-10[59927]: 2024-09-11T00:34:45.063+0000 7f2cff41d640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-11T00:33:41.548749+0000 front 2024-09-11T00:33:41.548701+0000 (oldest deadline 2024-09-11T00:34:05.648724+0000) 2024-09-11T00:34:45.533 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 11 00:34:45 smithi165 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-10[59927]: 2024-09-11T00:34:45.063+0000 7f2cff41d640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-09-11T00:33:50.850789+0000 front 2024-09-11T00:33:50.850826+0000 (oldest deadline 2024-09-11T00:34:11.950647+0000) 2024-09-11T00:34:45.533 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 11 00:34:45 smithi165 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-10[59927]: 2024-09-11T00:34:45.063+0000 7f2cff41d640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.165:6812 osd.4 since back 2024-09-11T00:33:55.951809+0000 front 2024-09-11T00:33:55.951787+0000 (oldest deadline 2024-09-11T00:34:21.851850+0000) 2024-09-11T00:34:45.533 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 11 00:34:45 smithi165 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-10[59927]: 2024-09-11T00:34:45.063+0000 7f2cff41d640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.178:6812 osd.5 since back 2024-09-11T00:34:07.752773+0000 front 2024-09-11T00:34:07.752884+0000 (oldest deadline 2024-09-11T00:34:28.852602+0000) 2024-09-11T00:34:45.533 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 11 00:34:45 smithi165 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-10[59927]: 2024-09-11T00:34:45.063+0000 7f2cff41d640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.12:6822 osd.6 since back 2024-09-11T00:34:14.753738+0000 front 2024-09-11T00:34:14.753757+0000 (oldest deadline 2024-09-11T00:34:38.853514+0000) 2024-09-11T00:34:45.831 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 00:34:45 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-11[60852]: 2024-09-11T00:34:45.391+0000 7fd754b0b640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-09-11T00:33:24.081060+0000 front 2024-09-11T00:33:24.081080+0000 (oldest deadline 2024-09-11T00:33:49.980779+0000) 2024-09-11T00:34:45.832 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 00:34:45 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-11[60852]: 2024-09-11T00:34:45.391+0000 7fd754b0b640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-11T00:33:41.682818+0000 front 2024-09-11T00:33:41.682848+0000 (oldest deadline 2024-09-11T00:34:05.182914+0000) 2024-09-11T00:34:45.832 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 00:34:45 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-11[60852]: 2024-09-11T00:34:45.391+0000 7fd754b0b640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-09-11T00:33:47.484088+0000 front 2024-09-11T00:33:47.484103+0000 (oldest deadline 2024-09-11T00:34:12.784060+0000) 2024-09-11T00:34:45.832 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 00:34:45 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-11[60852]: 2024-09-11T00:34:45.391+0000 7fd754b0b640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.165:6812 osd.4 since back 2024-09-11T00:33:58.685349+0000 front 2024-09-11T00:33:58.685336+0000 (oldest deadline 2024-09-11T00:34:23.385413+0000) 2024-09-11T00:34:45.832 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 00:34:45 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-11[60852]: 2024-09-11T00:34:45.391+0000 7fd754b0b640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.178:6812 osd.5 since back 2024-09-11T00:34:07.887744+0000 front 2024-09-11T00:34:07.887694+0000 (oldest deadline 2024-09-11T00:34:28.987855+0000) 2024-09-11T00:34:45.832 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 00:34:45 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-11[60852]: 2024-09-11T00:34:45.391+0000 7fd754b0b640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.12:6822 osd.6 since back 2024-09-11T00:34:11.189386+0000 front 2024-09-11T00:34:11.189460+0000 (oldest deadline 2024-09-11T00:34:37.089169+0000) 2024-09-11T00:34:46.533 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 11 00:34:46 smithi165 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-10[59927]: 2024-09-11T00:34:46.094+0000 7f2cff41d640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-09-11T00:33:25.146772+0000 front 2024-09-11T00:33:25.146620+0000 (oldest deadline 2024-09-11T00:33:49.846529+0000) 2024-09-11T00:34:46.533 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 11 00:34:46 smithi165 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-10[59927]: 2024-09-11T00:34:46.094+0000 7f2cff41d640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-11T00:33:32.747514+0000 front 2024-09-11T00:33:32.747491+0000 (oldest deadline 2024-09-11T00:33:58.047686+0000) 2024-09-11T00:34:46.533 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 11 00:34:46 smithi165 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-10[59927]: 2024-09-11T00:34:46.094+0000 7f2cff41d640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-11T00:33:41.548749+0000 front 2024-09-11T00:33:41.548701+0000 (oldest deadline 2024-09-11T00:34:05.648724+0000) 2024-09-11T00:34:46.533 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 11 00:34:46 smithi165 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-10[59927]: 2024-09-11T00:34:46.094+0000 7f2cff41d640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-09-11T00:33:50.850789+0000 front 2024-09-11T00:33:50.850826+0000 (oldest deadline 2024-09-11T00:34:11.950647+0000) 2024-09-11T00:34:46.533 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 11 00:34:46 smithi165 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-10[59927]: 2024-09-11T00:34:46.094+0000 7f2cff41d640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.165:6812 osd.4 since back 2024-09-11T00:33:55.951809+0000 front 2024-09-11T00:33:55.951787+0000 (oldest deadline 2024-09-11T00:34:21.851850+0000) 2024-09-11T00:34:46.534 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 11 00:34:46 smithi165 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-10[59927]: 2024-09-11T00:34:46.094+0000 7f2cff41d640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.178:6812 osd.5 since back 2024-09-11T00:34:07.752773+0000 front 2024-09-11T00:34:07.752884+0000 (oldest deadline 2024-09-11T00:34:28.852602+0000) 2024-09-11T00:34:46.534 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 11 00:34:46 smithi165 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-10[59927]: 2024-09-11T00:34:46.094+0000 7f2cff41d640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.12:6822 osd.6 since back 2024-09-11T00:34:14.753738+0000 front 2024-09-11T00:34:14.753757+0000 (oldest deadline 2024-09-11T00:34:38.853514+0000) 2024-09-11T00:34:46.831 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 00:34:46 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-11[60852]: 2024-09-11T00:34:46.392+0000 7fd754b0b640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-09-11T00:33:24.081060+0000 front 2024-09-11T00:33:24.081080+0000 (oldest deadline 2024-09-11T00:33:49.980779+0000) 2024-09-11T00:34:46.831 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 00:34:46 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-11[60852]: 2024-09-11T00:34:46.392+0000 7fd754b0b640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-11T00:33:41.682818+0000 front 2024-09-11T00:33:41.682848+0000 (oldest deadline 2024-09-11T00:34:05.182914+0000) 2024-09-11T00:34:46.831 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 00:34:46 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-11[60852]: 2024-09-11T00:34:46.392+0000 7fd754b0b640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-09-11T00:33:47.484088+0000 front 2024-09-11T00:33:47.484103+0000 (oldest deadline 2024-09-11T00:34:12.784060+0000) 2024-09-11T00:34:46.831 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 00:34:46 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-11[60852]: 2024-09-11T00:34:46.392+0000 7fd754b0b640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.165:6812 osd.4 since back 2024-09-11T00:33:58.685349+0000 front 2024-09-11T00:33:58.685336+0000 (oldest deadline 2024-09-11T00:34:23.385413+0000) 2024-09-11T00:34:46.832 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 00:34:46 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-11[60852]: 2024-09-11T00:34:46.392+0000 7fd754b0b640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.178:6812 osd.5 since back 2024-09-11T00:34:07.887744+0000 front 2024-09-11T00:34:07.887694+0000 (oldest deadline 2024-09-11T00:34:28.987855+0000) 2024-09-11T00:34:46.832 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 00:34:46 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-11[60852]: 2024-09-11T00:34:46.392+0000 7fd754b0b640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.12:6822 osd.6 since back 2024-09-11T00:34:11.189386+0000 front 2024-09-11T00:34:11.189460+0000 (oldest deadline 2024-09-11T00:34:37.089169+0000) 2024-09-11T00:34:46.832 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 00:34:46 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-11[60852]: 2024-09-11T00:34:46.392+0000 7fd754b0b640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.165:6820 osd.7 since back 2024-09-11T00:34:21.790130+0000 front 2024-09-11T00:34:21.790150+0000 (oldest deadline 2024-09-11T00:34:45.889979+0000) 2024-09-11T00:34:47.533 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 11 00:34:47 smithi165 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-10[59927]: 2024-09-11T00:34:47.073+0000 7f2cff41d640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-09-11T00:33:25.146772+0000 front 2024-09-11T00:33:25.146620+0000 (oldest deadline 2024-09-11T00:33:49.846529+0000) 2024-09-11T00:34:47.533 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 11 00:34:47 smithi165 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-10[59927]: 2024-09-11T00:34:47.073+0000 7f2cff41d640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-11T00:33:32.747514+0000 front 2024-09-11T00:33:32.747491+0000 (oldest deadline 2024-09-11T00:33:58.047686+0000) 2024-09-11T00:34:47.533 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 11 00:34:47 smithi165 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-10[59927]: 2024-09-11T00:34:47.073+0000 7f2cff41d640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-11T00:33:41.548749+0000 front 2024-09-11T00:33:41.548701+0000 (oldest deadline 2024-09-11T00:34:05.648724+0000) 2024-09-11T00:34:47.533 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 11 00:34:47 smithi165 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-10[59927]: 2024-09-11T00:34:47.073+0000 7f2cff41d640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-09-11T00:33:50.850789+0000 front 2024-09-11T00:33:50.850826+0000 (oldest deadline 2024-09-11T00:34:11.950647+0000) 2024-09-11T00:34:47.533 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 11 00:34:47 smithi165 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-10[59927]: 2024-09-11T00:34:47.073+0000 7f2cff41d640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.165:6812 osd.4 since back 2024-09-11T00:33:55.951809+0000 front 2024-09-11T00:33:55.951787+0000 (oldest deadline 2024-09-11T00:34:21.851850+0000) 2024-09-11T00:34:47.533 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 11 00:34:47 smithi165 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-10[59927]: 2024-09-11T00:34:47.073+0000 7f2cff41d640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.178:6812 osd.5 since back 2024-09-11T00:34:07.752773+0000 front 2024-09-11T00:34:07.752884+0000 (oldest deadline 2024-09-11T00:34:28.852602+0000) 2024-09-11T00:34:47.533 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 11 00:34:47 smithi165 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-10[59927]: 2024-09-11T00:34:47.073+0000 7f2cff41d640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.12:6822 osd.6 since back 2024-09-11T00:34:14.753738+0000 front 2024-09-11T00:34:14.753757+0000 (oldest deadline 2024-09-11T00:34:38.853514+0000) 2024-09-11T00:34:47.831 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 00:34:47 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-11[60852]: 2024-09-11T00:34:47.418+0000 7fd754b0b640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-09-11T00:33:24.081060+0000 front 2024-09-11T00:33:24.081080+0000 (oldest deadline 2024-09-11T00:33:49.980779+0000) 2024-09-11T00:34:47.832 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 00:34:47 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-11[60852]: 2024-09-11T00:34:47.418+0000 7fd754b0b640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-11T00:33:41.682818+0000 front 2024-09-11T00:33:41.682848+0000 (oldest deadline 2024-09-11T00:34:05.182914+0000) 2024-09-11T00:34:47.832 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 00:34:47 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-11[60852]: 2024-09-11T00:34:47.418+0000 7fd754b0b640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-09-11T00:33:47.484088+0000 front 2024-09-11T00:33:47.484103+0000 (oldest deadline 2024-09-11T00:34:12.784060+0000) 2024-09-11T00:34:47.832 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 00:34:47 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-11[60852]: 2024-09-11T00:34:47.418+0000 7fd754b0b640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.165:6812 osd.4 since back 2024-09-11T00:33:58.685349+0000 front 2024-09-11T00:33:58.685336+0000 (oldest deadline 2024-09-11T00:34:23.385413+0000) 2024-09-11T00:34:47.832 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 00:34:47 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-11[60852]: 2024-09-11T00:34:47.418+0000 7fd754b0b640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.178:6812 osd.5 since back 2024-09-11T00:34:07.887744+0000 front 2024-09-11T00:34:07.887694+0000 (oldest deadline 2024-09-11T00:34:28.987855+0000) 2024-09-11T00:34:47.832 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 00:34:47 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-11[60852]: 2024-09-11T00:34:47.418+0000 7fd754b0b640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.12:6822 osd.6 since back 2024-09-11T00:34:11.189386+0000 front 2024-09-11T00:34:11.189460+0000 (oldest deadline 2024-09-11T00:34:37.089169+0000) 2024-09-11T00:34:47.832 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 00:34:47 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-11[60852]: 2024-09-11T00:34:47.418+0000 7fd754b0b640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.165:6820 osd.7 since back 2024-09-11T00:34:21.790130+0000 front 2024-09-11T00:34:21.790150+0000 (oldest deadline 2024-09-11T00:34:45.889979+0000) 2024-09-11T00:34:48.533 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 11 00:34:48 smithi165 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-10[59927]: 2024-09-11T00:34:48.067+0000 7f2cff41d640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-09-11T00:33:25.146772+0000 front 2024-09-11T00:33:25.146620+0000 (oldest deadline 2024-09-11T00:33:49.846529+0000) 2024-09-11T00:34:48.533 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 11 00:34:48 smithi165 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-10[59927]: 2024-09-11T00:34:48.067+0000 7f2cff41d640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-11T00:33:32.747514+0000 front 2024-09-11T00:33:32.747491+0000 (oldest deadline 2024-09-11T00:33:58.047686+0000) 2024-09-11T00:34:48.533 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 11 00:34:48 smithi165 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-10[59927]: 2024-09-11T00:34:48.067+0000 7f2cff41d640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-11T00:33:41.548749+0000 front 2024-09-11T00:33:41.548701+0000 (oldest deadline 2024-09-11T00:34:05.648724+0000) 2024-09-11T00:34:48.533 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 11 00:34:48 smithi165 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-10[59927]: 2024-09-11T00:34:48.067+0000 7f2cff41d640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-09-11T00:33:50.850789+0000 front 2024-09-11T00:33:50.850826+0000 (oldest deadline 2024-09-11T00:34:11.950647+0000) 2024-09-11T00:34:48.533 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 11 00:34:48 smithi165 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-10[59927]: 2024-09-11T00:34:48.067+0000 7f2cff41d640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.165:6812 osd.4 since back 2024-09-11T00:33:55.951809+0000 front 2024-09-11T00:33:55.951787+0000 (oldest deadline 2024-09-11T00:34:21.851850+0000) 2024-09-11T00:34:48.533 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 11 00:34:48 smithi165 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-10[59927]: 2024-09-11T00:34:48.067+0000 7f2cff41d640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.178:6812 osd.5 since back 2024-09-11T00:34:07.752773+0000 front 2024-09-11T00:34:07.752884+0000 (oldest deadline 2024-09-11T00:34:28.852602+0000) 2024-09-11T00:34:48.533 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 11 00:34:48 smithi165 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-10[59927]: 2024-09-11T00:34:48.067+0000 7f2cff41d640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.12:6822 osd.6 since back 2024-09-11T00:34:14.753738+0000 front 2024-09-11T00:34:14.753757+0000 (oldest deadline 2024-09-11T00:34:38.853514+0000) 2024-09-11T00:34:48.831 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 00:34:48 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-11[60852]: 2024-09-11T00:34:48.464+0000 7fd754b0b640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-09-11T00:33:24.081060+0000 front 2024-09-11T00:33:24.081080+0000 (oldest deadline 2024-09-11T00:33:49.980779+0000) 2024-09-11T00:34:48.831 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 00:34:48 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-11[60852]: 2024-09-11T00:34:48.464+0000 7fd754b0b640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-11T00:33:41.682818+0000 front 2024-09-11T00:33:41.682848+0000 (oldest deadline 2024-09-11T00:34:05.182914+0000) 2024-09-11T00:34:48.831 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 00:34:48 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-11[60852]: 2024-09-11T00:34:48.464+0000 7fd754b0b640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-09-11T00:33:47.484088+0000 front 2024-09-11T00:33:47.484103+0000 (oldest deadline 2024-09-11T00:34:12.784060+0000) 2024-09-11T00:34:48.831 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 00:34:48 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-11[60852]: 2024-09-11T00:34:48.464+0000 7fd754b0b640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.165:6812 osd.4 since back 2024-09-11T00:33:58.685349+0000 front 2024-09-11T00:33:58.685336+0000 (oldest deadline 2024-09-11T00:34:23.385413+0000) 2024-09-11T00:34:48.832 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 00:34:48 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-11[60852]: 2024-09-11T00:34:48.464+0000 7fd754b0b640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.178:6812 osd.5 since back 2024-09-11T00:34:07.887744+0000 front 2024-09-11T00:34:07.887694+0000 (oldest deadline 2024-09-11T00:34:28.987855+0000) 2024-09-11T00:34:48.832 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 00:34:48 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-11[60852]: 2024-09-11T00:34:48.464+0000 7fd754b0b640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.12:6822 osd.6 since back 2024-09-11T00:34:11.189386+0000 front 2024-09-11T00:34:11.189460+0000 (oldest deadline 2024-09-11T00:34:37.089169+0000) 2024-09-11T00:34:48.832 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 00:34:48 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-11[60852]: 2024-09-11T00:34:48.464+0000 7fd754b0b640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.165:6820 osd.7 since back 2024-09-11T00:34:21.790130+0000 front 2024-09-11T00:34:21.790150+0000 (oldest deadline 2024-09-11T00:34:45.889979+0000) 2024-09-11T00:34:49.533 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 11 00:34:49 smithi165 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-10[59927]: 2024-09-11T00:34:49.070+0000 7f2cff41d640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-09-11T00:33:25.146772+0000 front 2024-09-11T00:33:25.146620+0000 (oldest deadline 2024-09-11T00:33:49.846529+0000) 2024-09-11T00:34:49.533 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 11 00:34:49 smithi165 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-10[59927]: 2024-09-11T00:34:49.070+0000 7f2cff41d640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.165:6804 osd.1 since back 2024-09-11T00:33:32.747514+0000 front 2024-09-11T00:33:32.747491+0000 (oldest deadline 2024-09-11T00:33:58.047686+0000) 2024-09-11T00:34:49.533 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 11 00:34:49 smithi165 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-10[59927]: 2024-09-11T00:34:49.070+0000 7f2cff41d640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-11T00:33:41.548749+0000 front 2024-09-11T00:33:41.548701+0000 (oldest deadline 2024-09-11T00:34:05.648724+0000) 2024-09-11T00:34:49.533 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 11 00:34:49 smithi165 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-10[59927]: 2024-09-11T00:34:49.070+0000 7f2cff41d640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-09-11T00:33:50.850789+0000 front 2024-09-11T00:33:50.850826+0000 (oldest deadline 2024-09-11T00:34:11.950647+0000) 2024-09-11T00:34:49.533 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 11 00:34:49 smithi165 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-10[59927]: 2024-09-11T00:34:49.070+0000 7f2cff41d640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.165:6812 osd.4 since back 2024-09-11T00:33:55.951809+0000 front 2024-09-11T00:33:55.951787+0000 (oldest deadline 2024-09-11T00:34:21.851850+0000) 2024-09-11T00:34:49.533 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 11 00:34:49 smithi165 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-10[59927]: 2024-09-11T00:34:49.070+0000 7f2cff41d640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.178:6812 osd.5 since back 2024-09-11T00:34:07.752773+0000 front 2024-09-11T00:34:07.752884+0000 (oldest deadline 2024-09-11T00:34:28.852602+0000) 2024-09-11T00:34:49.533 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 11 00:34:49 smithi165 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-10[59927]: 2024-09-11T00:34:49.070+0000 7f2cff41d640 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.12:6822 osd.6 since back 2024-09-11T00:34:14.753738+0000 front 2024-09-11T00:34:14.753757+0000 (oldest deadline 2024-09-11T00:34:38.853514+0000) 2024-09-11T00:34:49.831 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 00:34:49 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-11[60852]: 2024-09-11T00:34:49.494+0000 7fd754b0b640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-09-11T00:33:24.081060+0000 front 2024-09-11T00:33:24.081080+0000 (oldest deadline 2024-09-11T00:33:49.980779+0000) 2024-09-11T00:34:49.831 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 00:34:49 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-11[60852]: 2024-09-11T00:34:49.494+0000 7fd754b0b640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-11T00:33:41.682818+0000 front 2024-09-11T00:33:41.682848+0000 (oldest deadline 2024-09-11T00:34:05.182914+0000) 2024-09-11T00:34:49.831 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 00:34:49 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-11[60852]: 2024-09-11T00:34:49.494+0000 7fd754b0b640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-09-11T00:33:47.484088+0000 front 2024-09-11T00:33:47.484103+0000 (oldest deadline 2024-09-11T00:34:12.784060+0000) 2024-09-11T00:34:49.831 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 00:34:49 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-11[60852]: 2024-09-11T00:34:49.494+0000 7fd754b0b640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.165:6812 osd.4 since back 2024-09-11T00:33:58.685349+0000 front 2024-09-11T00:33:58.685336+0000 (oldest deadline 2024-09-11T00:34:23.385413+0000) 2024-09-11T00:34:49.832 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 00:34:49 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-11[60852]: 2024-09-11T00:34:49.494+0000 7fd754b0b640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.178:6812 osd.5 since back 2024-09-11T00:34:07.887744+0000 front 2024-09-11T00:34:07.887694+0000 (oldest deadline 2024-09-11T00:34:28.987855+0000) 2024-09-11T00:34:49.832 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 00:34:49 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-11[60852]: 2024-09-11T00:34:49.494+0000 7fd754b0b640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.12:6822 osd.6 since back 2024-09-11T00:34:11.189386+0000 front 2024-09-11T00:34:11.189460+0000 (oldest deadline 2024-09-11T00:34:37.089169+0000) 2024-09-11T00:34:49.832 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 00:34:49 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-11[60852]: 2024-09-11T00:34:49.494+0000 7fd754b0b640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.165:6820 osd.7 since back 2024-09-11T00:34:21.790130+0000 front 2024-09-11T00:34:21.790150+0000 (oldest deadline 2024-09-11T00:34:45.889979+0000) 2024-09-11T00:34:49.991 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 11 00:34:49 smithi165 podman[76571]: 2024-09-11 00:34:49.678267811 +0000 UTC m=+5.529394870 container died b557cfc179e9b80362032f18745bd5de568fe341a30d4bfd01b83d6959d4f945 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-10, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-11T00:34:50.283 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 11 00:34:49 smithi165 podman[76571]: 2024-09-11 00:34:49.990526616 +0000 UTC m=+5.841653683 container cleanup b557cfc179e9b80362032f18745bd5de568fe341a30d4bfd01b83d6959d4f945 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-10, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_BRANCH=HEAD) 2024-09-11T00:34:50.283 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 11 00:34:49 smithi165 bash[76571]: ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-10 2024-09-11T00:34:50.283 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 11 00:34:50 smithi165 podman[76586]: 2024-09-11 00:34:50.194947112 +0000 UTC m=+0.512648423 container remove b557cfc179e9b80362032f18745bd5de568fe341a30d4bfd01b83d6959d4f945 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-10, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True) 2024-09-11T00:34:50.832 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 00:34:50 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-11[60852]: 2024-09-11T00:34:50.475+0000 7fd754b0b640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-09-11T00:33:24.081060+0000 front 2024-09-11T00:33:24.081080+0000 (oldest deadline 2024-09-11T00:33:49.980779+0000) 2024-09-11T00:34:50.832 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 00:34:50 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-11[60852]: 2024-09-11T00:34:50.475+0000 7fd754b0b640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-11T00:33:41.682818+0000 front 2024-09-11T00:33:41.682848+0000 (oldest deadline 2024-09-11T00:34:05.182914+0000) 2024-09-11T00:34:50.832 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 00:34:50 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-11[60852]: 2024-09-11T00:34:50.475+0000 7fd754b0b640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-09-11T00:33:47.484088+0000 front 2024-09-11T00:33:47.484103+0000 (oldest deadline 2024-09-11T00:34:12.784060+0000) 2024-09-11T00:34:50.832 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 00:34:50 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-11[60852]: 2024-09-11T00:34:50.475+0000 7fd754b0b640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.165:6812 osd.4 since back 2024-09-11T00:33:58.685349+0000 front 2024-09-11T00:33:58.685336+0000 (oldest deadline 2024-09-11T00:34:23.385413+0000) 2024-09-11T00:34:50.832 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 00:34:50 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-11[60852]: 2024-09-11T00:34:50.475+0000 7fd754b0b640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.178:6812 osd.5 since back 2024-09-11T00:34:07.887744+0000 front 2024-09-11T00:34:07.887694+0000 (oldest deadline 2024-09-11T00:34:28.987855+0000) 2024-09-11T00:34:50.832 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 00:34:50 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-11[60852]: 2024-09-11T00:34:50.475+0000 7fd754b0b640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.12:6822 osd.6 since back 2024-09-11T00:34:11.189386+0000 front 2024-09-11T00:34:11.189460+0000 (oldest deadline 2024-09-11T00:34:37.089169+0000) 2024-09-11T00:34:50.832 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 00:34:50 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-11[60852]: 2024-09-11T00:34:50.475+0000 7fd754b0b640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.165:6820 osd.7 since back 2024-09-11T00:34:21.790130+0000 front 2024-09-11T00:34:21.790150+0000 (oldest deadline 2024-09-11T00:34:45.889979+0000) 2024-09-11T00:34:50.888 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 11 00:34:50 smithi165 podman[76674]: 2024-09-11 00:34:50.510348202 +0000 UTC m=+0.015739130 image pull 589dd18248e494ba411dc838aa4ae0e5e569213f08fcd7a2cf22cd3ef7e3c480 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e 2024-09-11T00:34:50.888 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 11 00:34:50 smithi165 podman[76674]: 2024-09-11 00:34:50.609541621 +0000 UTC m=+0.114932539 container create c57670770e30c339203c85a2831f47b72e063b5f56c7e9869fa175383918b184 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-10-deactivate, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0) 2024-09-11T00:34:50.888 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 11 00:34:50 smithi165 podman[76674]: 2024-09-11 00:34:50.803857092 +0000 UTC m=+0.309248023 container init c57670770e30c339203c85a2831f47b72e063b5f56c7e9869fa175383918b184 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-10-deactivate, GIT_CLEAN=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True) 2024-09-11T00:34:50.888 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 11 00:34:50 smithi165 podman[76674]: 2024-09-11 00:34:50.807435669 +0000 UTC m=+0.312826591 container start c57670770e30c339203c85a2831f47b72e063b5f56c7e9869fa175383918b184 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-10-deactivate, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.build-date=20240731) 2024-09-11T00:34:50.888 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 11 00:34:50 smithi165 podman[76674]: 2024-09-11 00:34:50.886875485 +0000 UTC m=+0.392266408 container attach c57670770e30c339203c85a2831f47b72e063b5f56c7e9869fa175383918b184 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-10-deactivate, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-11T00:34:51.265 INFO:journalctl@ceph.osd.10.smithi165.stdout:Sep 11 00:34:50 smithi165 podman[76674]: 2024-09-11 00:34:50.992882467 +0000 UTC m=+0.498273397 container died c57670770e30c339203c85a2831f47b72e063b5f56c7e9869fa175383918b184 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-10-deactivate, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-09-11T00:34:51.552 DEBUG:teuthology.orchestra.run.smithi165:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968@osd.10.service' 2024-09-11T00:34:51.590 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T00:34:51.590 INFO:tasks.cephadm.osd.10:Stopped osd.10 2024-09-11T00:34:51.590 INFO:tasks.cephadm.osd.11:Stopping osd.11... 2024-09-11T00:34:51.590 DEBUG:teuthology.orchestra.run.smithi178:> sudo systemctl stop ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968@osd.11 2024-09-11T00:34:51.831 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 00:34:51 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-11[60852]: 2024-09-11T00:34:51.492+0000 7fd754b0b640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-09-11T00:33:24.081060+0000 front 2024-09-11T00:33:24.081080+0000 (oldest deadline 2024-09-11T00:33:49.980779+0000) 2024-09-11T00:34:51.831 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 00:34:51 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-11[60852]: 2024-09-11T00:34:51.492+0000 7fd754b0b640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-11T00:33:41.682818+0000 front 2024-09-11T00:33:41.682848+0000 (oldest deadline 2024-09-11T00:34:05.182914+0000) 2024-09-11T00:34:51.832 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 00:34:51 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-11[60852]: 2024-09-11T00:34:51.492+0000 7fd754b0b640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-09-11T00:33:47.484088+0000 front 2024-09-11T00:33:47.484103+0000 (oldest deadline 2024-09-11T00:34:12.784060+0000) 2024-09-11T00:34:51.832 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 00:34:51 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-11[60852]: 2024-09-11T00:34:51.492+0000 7fd754b0b640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.165:6812 osd.4 since back 2024-09-11T00:33:58.685349+0000 front 2024-09-11T00:33:58.685336+0000 (oldest deadline 2024-09-11T00:34:23.385413+0000) 2024-09-11T00:34:51.832 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 00:34:51 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-11[60852]: 2024-09-11T00:34:51.492+0000 7fd754b0b640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.178:6812 osd.5 since back 2024-09-11T00:34:07.887744+0000 front 2024-09-11T00:34:07.887694+0000 (oldest deadline 2024-09-11T00:34:28.987855+0000) 2024-09-11T00:34:51.832 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 00:34:51 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-11[60852]: 2024-09-11T00:34:51.492+0000 7fd754b0b640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.12:6822 osd.6 since back 2024-09-11T00:34:11.189386+0000 front 2024-09-11T00:34:11.189460+0000 (oldest deadline 2024-09-11T00:34:37.089169+0000) 2024-09-11T00:34:51.832 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 00:34:51 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-11[60852]: 2024-09-11T00:34:51.492+0000 7fd754b0b640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.165:6820 osd.7 since back 2024-09-11T00:34:21.790130+0000 front 2024-09-11T00:34:21.790150+0000 (oldest deadline 2024-09-11T00:34:45.889979+0000) 2024-09-11T00:34:51.832 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 00:34:51 smithi178 systemd[1]: Stopping Ceph osd.11 for ff3ad74a-6fcf-11ef-bcea-c7b262605968... 2024-09-11T00:34:52.331 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 00:34:51 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-11[60852]: 2024-09-11T00:34:51.881+0000 7fd75c4fe640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.11 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T00:34:52.331 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 00:34:51 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-11[60852]: 2024-09-11T00:34:51.881+0000 7fd75c4fe640 -1 osd.11 91 *** Got signal Terminated *** 2024-09-11T00:34:52.331 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 00:34:51 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-11[60852]: 2024-09-11T00:34:51.881+0000 7fd75c4fe640 -1 osd.11 91 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-11T00:34:52.832 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 00:34:52 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-11[60852]: 2024-09-11T00:34:52.493+0000 7fd754b0b640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-09-11T00:33:24.081060+0000 front 2024-09-11T00:33:24.081080+0000 (oldest deadline 2024-09-11T00:33:49.980779+0000) 2024-09-11T00:34:52.832 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 00:34:52 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-11[60852]: 2024-09-11T00:34:52.493+0000 7fd754b0b640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-11T00:33:41.682818+0000 front 2024-09-11T00:33:41.682848+0000 (oldest deadline 2024-09-11T00:34:05.182914+0000) 2024-09-11T00:34:52.832 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 00:34:52 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-11[60852]: 2024-09-11T00:34:52.493+0000 7fd754b0b640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-09-11T00:33:47.484088+0000 front 2024-09-11T00:33:47.484103+0000 (oldest deadline 2024-09-11T00:34:12.784060+0000) 2024-09-11T00:34:52.832 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 00:34:52 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-11[60852]: 2024-09-11T00:34:52.493+0000 7fd754b0b640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.165:6812 osd.4 since back 2024-09-11T00:33:58.685349+0000 front 2024-09-11T00:33:58.685336+0000 (oldest deadline 2024-09-11T00:34:23.385413+0000) 2024-09-11T00:34:52.832 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 00:34:52 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-11[60852]: 2024-09-11T00:34:52.493+0000 7fd754b0b640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.178:6812 osd.5 since back 2024-09-11T00:34:07.887744+0000 front 2024-09-11T00:34:07.887694+0000 (oldest deadline 2024-09-11T00:34:28.987855+0000) 2024-09-11T00:34:52.832 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 00:34:52 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-11[60852]: 2024-09-11T00:34:52.494+0000 7fd754b0b640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.12:6822 osd.6 since back 2024-09-11T00:34:11.189386+0000 front 2024-09-11T00:34:11.189460+0000 (oldest deadline 2024-09-11T00:34:37.089169+0000) 2024-09-11T00:34:52.832 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 00:34:52 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-11[60852]: 2024-09-11T00:34:52.494+0000 7fd754b0b640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.165:6820 osd.7 since back 2024-09-11T00:34:21.790130+0000 front 2024-09-11T00:34:21.790150+0000 (oldest deadline 2024-09-11T00:34:45.889979+0000) 2024-09-11T00:34:53.831 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 00:34:53 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-11[60852]: 2024-09-11T00:34:53.445+0000 7fd754b0b640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-09-11T00:33:24.081060+0000 front 2024-09-11T00:33:24.081080+0000 (oldest deadline 2024-09-11T00:33:49.980779+0000) 2024-09-11T00:34:53.832 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 00:34:53 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-11[60852]: 2024-09-11T00:34:53.445+0000 7fd754b0b640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-11T00:33:41.682818+0000 front 2024-09-11T00:33:41.682848+0000 (oldest deadline 2024-09-11T00:34:05.182914+0000) 2024-09-11T00:34:53.832 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 00:34:53 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-11[60852]: 2024-09-11T00:34:53.445+0000 7fd754b0b640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-09-11T00:33:47.484088+0000 front 2024-09-11T00:33:47.484103+0000 (oldest deadline 2024-09-11T00:34:12.784060+0000) 2024-09-11T00:34:53.832 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 00:34:53 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-11[60852]: 2024-09-11T00:34:53.445+0000 7fd754b0b640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.165:6812 osd.4 since back 2024-09-11T00:33:58.685349+0000 front 2024-09-11T00:33:58.685336+0000 (oldest deadline 2024-09-11T00:34:23.385413+0000) 2024-09-11T00:34:53.832 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 00:34:53 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-11[60852]: 2024-09-11T00:34:53.445+0000 7fd754b0b640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.178:6812 osd.5 since back 2024-09-11T00:34:07.887744+0000 front 2024-09-11T00:34:07.887694+0000 (oldest deadline 2024-09-11T00:34:28.987855+0000) 2024-09-11T00:34:53.832 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 00:34:53 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-11[60852]: 2024-09-11T00:34:53.445+0000 7fd754b0b640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.12:6822 osd.6 since back 2024-09-11T00:34:11.189386+0000 front 2024-09-11T00:34:11.189460+0000 (oldest deadline 2024-09-11T00:34:37.089169+0000) 2024-09-11T00:34:53.832 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 00:34:53 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-11[60852]: 2024-09-11T00:34:53.445+0000 7fd754b0b640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.165:6820 osd.7 since back 2024-09-11T00:34:21.790130+0000 front 2024-09-11T00:34:21.790150+0000 (oldest deadline 2024-09-11T00:34:45.889979+0000) 2024-09-11T00:34:54.831 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 00:34:54 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-11[60852]: 2024-09-11T00:34:54.414+0000 7fd754b0b640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-09-11T00:33:24.081060+0000 front 2024-09-11T00:33:24.081080+0000 (oldest deadline 2024-09-11T00:33:49.980779+0000) 2024-09-11T00:34:54.831 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 00:34:54 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-11[60852]: 2024-09-11T00:34:54.414+0000 7fd754b0b640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-11T00:33:41.682818+0000 front 2024-09-11T00:33:41.682848+0000 (oldest deadline 2024-09-11T00:34:05.182914+0000) 2024-09-11T00:34:54.831 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 00:34:54 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-11[60852]: 2024-09-11T00:34:54.414+0000 7fd754b0b640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-09-11T00:33:47.484088+0000 front 2024-09-11T00:33:47.484103+0000 (oldest deadline 2024-09-11T00:34:12.784060+0000) 2024-09-11T00:34:54.831 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 00:34:54 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-11[60852]: 2024-09-11T00:34:54.414+0000 7fd754b0b640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.165:6812 osd.4 since back 2024-09-11T00:33:58.685349+0000 front 2024-09-11T00:33:58.685336+0000 (oldest deadline 2024-09-11T00:34:23.385413+0000) 2024-09-11T00:34:54.831 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 00:34:54 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-11[60852]: 2024-09-11T00:34:54.414+0000 7fd754b0b640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.178:6812 osd.5 since back 2024-09-11T00:34:07.887744+0000 front 2024-09-11T00:34:07.887694+0000 (oldest deadline 2024-09-11T00:34:28.987855+0000) 2024-09-11T00:34:54.832 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 00:34:54 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-11[60852]: 2024-09-11T00:34:54.414+0000 7fd754b0b640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.12:6822 osd.6 since back 2024-09-11T00:34:11.189386+0000 front 2024-09-11T00:34:11.189460+0000 (oldest deadline 2024-09-11T00:34:37.089169+0000) 2024-09-11T00:34:54.832 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 00:34:54 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-11[60852]: 2024-09-11T00:34:54.414+0000 7fd754b0b640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.165:6820 osd.7 since back 2024-09-11T00:34:21.790130+0000 front 2024-09-11T00:34:21.790150+0000 (oldest deadline 2024-09-11T00:34:45.889979+0000) 2024-09-11T00:34:55.733 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 00:34:55 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-11[60852]: 2024-09-11T00:34:55.429+0000 7fd754b0b640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-09-11T00:33:24.081060+0000 front 2024-09-11T00:33:24.081080+0000 (oldest deadline 2024-09-11T00:33:49.980779+0000) 2024-09-11T00:34:55.733 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 00:34:55 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-11[60852]: 2024-09-11T00:34:55.429+0000 7fd754b0b640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-11T00:33:41.682818+0000 front 2024-09-11T00:33:41.682848+0000 (oldest deadline 2024-09-11T00:34:05.182914+0000) 2024-09-11T00:34:55.733 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 00:34:55 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-11[60852]: 2024-09-11T00:34:55.429+0000 7fd754b0b640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-09-11T00:33:47.484088+0000 front 2024-09-11T00:33:47.484103+0000 (oldest deadline 2024-09-11T00:34:12.784060+0000) 2024-09-11T00:34:55.733 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 00:34:55 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-11[60852]: 2024-09-11T00:34:55.429+0000 7fd754b0b640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.165:6812 osd.4 since back 2024-09-11T00:33:58.685349+0000 front 2024-09-11T00:33:58.685336+0000 (oldest deadline 2024-09-11T00:34:23.385413+0000) 2024-09-11T00:34:55.733 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 00:34:55 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-11[60852]: 2024-09-11T00:34:55.429+0000 7fd754b0b640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.178:6812 osd.5 since back 2024-09-11T00:34:07.887744+0000 front 2024-09-11T00:34:07.887694+0000 (oldest deadline 2024-09-11T00:34:28.987855+0000) 2024-09-11T00:34:55.733 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 00:34:55 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-11[60852]: 2024-09-11T00:34:55.429+0000 7fd754b0b640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.12:6822 osd.6 since back 2024-09-11T00:34:11.189386+0000 front 2024-09-11T00:34:11.189460+0000 (oldest deadline 2024-09-11T00:34:37.089169+0000) 2024-09-11T00:34:55.733 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 00:34:55 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-11[60852]: 2024-09-11T00:34:55.429+0000 7fd754b0b640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.165:6820 osd.7 since back 2024-09-11T00:34:21.790130+0000 front 2024-09-11T00:34:21.790150+0000 (oldest deadline 2024-09-11T00:34:45.889979+0000) 2024-09-11T00:34:55.734 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 00:34:55 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-11[60852]: 2024-09-11T00:34:55.429+0000 7fd754b0b640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.178:6820 osd.8 since back 2024-09-11T00:34:32.291524+0000 front 2024-09-11T00:34:32.291561+0000 (oldest deadline 2024-09-11T00:34:55.191575+0000) 2024-09-11T00:34:56.832 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 00:34:56 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-11[60852]: 2024-09-11T00:34:56.394+0000 7fd754b0b640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-09-11T00:33:24.081060+0000 front 2024-09-11T00:33:24.081080+0000 (oldest deadline 2024-09-11T00:33:49.980779+0000) 2024-09-11T00:34:56.832 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 00:34:56 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-11[60852]: 2024-09-11T00:34:56.394+0000 7fd754b0b640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.178:6804 osd.2 since back 2024-09-11T00:33:41.682818+0000 front 2024-09-11T00:33:41.682848+0000 (oldest deadline 2024-09-11T00:34:05.182914+0000) 2024-09-11T00:34:56.832 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 00:34:56 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-11[60852]: 2024-09-11T00:34:56.394+0000 7fd754b0b640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-09-11T00:33:47.484088+0000 front 2024-09-11T00:33:47.484103+0000 (oldest deadline 2024-09-11T00:34:12.784060+0000) 2024-09-11T00:34:56.832 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 00:34:56 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-11[60852]: 2024-09-11T00:34:56.394+0000 7fd754b0b640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.165:6812 osd.4 since back 2024-09-11T00:33:58.685349+0000 front 2024-09-11T00:33:58.685336+0000 (oldest deadline 2024-09-11T00:34:23.385413+0000) 2024-09-11T00:34:56.832 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 00:34:56 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-11[60852]: 2024-09-11T00:34:56.394+0000 7fd754b0b640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.178:6812 osd.5 since back 2024-09-11T00:34:07.887744+0000 front 2024-09-11T00:34:07.887694+0000 (oldest deadline 2024-09-11T00:34:28.987855+0000) 2024-09-11T00:34:56.832 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 00:34:56 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-11[60852]: 2024-09-11T00:34:56.394+0000 7fd754b0b640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.12:6822 osd.6 since back 2024-09-11T00:34:11.189386+0000 front 2024-09-11T00:34:11.189460+0000 (oldest deadline 2024-09-11T00:34:37.089169+0000) 2024-09-11T00:34:56.832 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 00:34:56 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-11[60852]: 2024-09-11T00:34:56.394+0000 7fd754b0b640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.165:6820 osd.7 since back 2024-09-11T00:34:21.790130+0000 front 2024-09-11T00:34:21.790150+0000 (oldest deadline 2024-09-11T00:34:45.889979+0000) 2024-09-11T00:34:56.832 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 00:34:56 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-11[60852]: 2024-09-11T00:34:56.394+0000 7fd754b0b640 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.178:6820 osd.8 since back 2024-09-11T00:34:32.291524+0000 front 2024-09-11T00:34:32.291561+0000 (oldest deadline 2024-09-11T00:34:55.191575+0000) 2024-09-11T00:34:57.625 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 00:34:57 smithi178 podman[79258]: 2024-09-11 00:34:57.350202978 +0000 UTC m=+5.566479445 container died f73e807cea74270db34256f75bb7ee229f9ee0572aebd831ab0fb212ce7257f2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-11, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-11T00:34:57.975 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 00:34:57 smithi178 podman[79258]: 2024-09-11 00:34:57.722993044 +0000 UTC m=+5.939269512 container cleanup f73e807cea74270db34256f75bb7ee229f9ee0572aebd831ab0fb212ce7257f2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-11, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240731) 2024-09-11T00:34:57.975 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 00:34:57 smithi178 bash[79258]: ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-11 2024-09-11T00:34:58.331 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 00:34:57 smithi178 podman[79286]: 2024-09-11 00:34:57.974062637 +0000 UTC m=+0.622083650 container remove f73e807cea74270db34256f75bb7ee229f9ee0572aebd831ab0fb212ce7257f2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-11, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.schema-version=1.0) 2024-09-11T00:34:58.645 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 00:34:58 smithi178 podman[79373]: 2024-09-11 00:34:58.23391473 +0000 UTC m=+0.015299805 image pull 589dd18248e494ba411dc838aa4ae0e5e569213f08fcd7a2cf22cd3ef7e3c480 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e 2024-09-11T00:34:58.645 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 00:34:58 smithi178 podman[79373]: 2024-09-11 00:34:58.341378306 +0000 UTC m=+0.122763368 container create 741463f08cc2986891119c251afcf3e2c79d2b919861c1f5d1bcc7add978b886 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-11-deactivate, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_CLEAN=True, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-11T00:34:58.645 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 00:34:58 smithi178 podman[79373]: 2024-09-11 00:34:58.568556313 +0000 UTC m=+0.349941382 container init 741463f08cc2986891119c251afcf3e2c79d2b919861c1f5d1bcc7add978b886 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-11-deactivate, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, ceph=True, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, io.buildah.version=1.37.2) 2024-09-11T00:34:58.645 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 00:34:58 smithi178 podman[79373]: 2024-09-11 00:34:58.571719541 +0000 UTC m=+0.353104607 container start 741463f08cc2986891119c251afcf3e2c79d2b919861c1f5d1bcc7add978b886 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-11-deactivate, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, ceph=True, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux ) 2024-09-11T00:34:58.645 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 00:34:58 smithi178 podman[79373]: 2024-09-11 00:34:58.643515535 +0000 UTC m=+0.424900598 container attach 741463f08cc2986891119c251afcf3e2c79d2b919861c1f5d1bcc7add978b886 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-11-deactivate, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_BRANCH=HEAD, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4) 2024-09-11T00:34:59.056 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 00:34:58 smithi178 podman[79373]: 2024-09-11 00:34:58.762451647 +0000 UTC m=+0.543836726 container died 741463f08cc2986891119c251afcf3e2c79d2b919861c1f5d1bcc7add978b886 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-11-deactivate, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD) 2024-09-11T00:34:59.315 INFO:journalctl@ceph.osd.11.smithi178.stdout:Sep 11 00:34:59 smithi178 podman[79396]: 2024-09-11 00:34:59.296733119 +0000 UTC m=+0.528660070 container remove 741463f08cc2986891119c251afcf3e2c79d2b919861c1f5d1bcc7add978b886 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-osd-11-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , ceph=True, GIT_CLEAN=True, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-09-11T00:34:59.320 DEBUG:teuthology.orchestra.run.smithi178:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968@osd.11.service' 2024-09-11T00:34:59.361 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T00:34:59.361 INFO:tasks.cephadm.osd.11:Stopped osd.11 2024-09-11T00:34:59.361 INFO:tasks.cephadm.mds.a:Stopping mds.a... 2024-09-11T00:34:59.361 DEBUG:teuthology.orchestra.run.smithi012:> sudo systemctl stop ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968@mds.a 2024-09-11T00:34:59.759 INFO:journalctl@ceph.mds.a.smithi012.stdout:Sep 11 00:34:59 smithi012 systemd[1]: Stopping Ceph mds.a for ff3ad74a-6fcf-11ef-bcea-c7b262605968... 2024-09-11T00:35:00.063 INFO:journalctl@ceph.mds.a.smithi012.stdout:Sep 11 00:34:59 smithi012 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-mds-a[73335]: 2024-09-11T00:34:59.756+0000 7fdb6fe00640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T00:35:00.063 INFO:journalctl@ceph.mds.a.smithi012.stdout:Sep 11 00:34:59 smithi012 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-mds-a[73335]: 2024-09-11T00:34:59.756+0000 7fdb6fe00640 -1 mds.a *** got signal Terminated *** 2024-09-11T00:35:02.725 INFO:journalctl@ceph.mds.a.smithi012.stdout:Sep 11 00:35:02 smithi012 podman[115031]: 2024-09-11 00:35:02.473793249 +0000 UTC m=+2.824122205 container died 8e26bf6033aa9b348aab11dc982055ef7790ff8d53bfd6ae3ab93e12da1f6f63 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-mds-a, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_CLEAN=True, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731) 2024-09-11T00:35:03.063 INFO:journalctl@ceph.mds.a.smithi012.stdout:Sep 11 00:35:02 smithi012 podman[115031]: 2024-09-11 00:35:02.803371828 +0000 UTC m=+3.153700782 container cleanup 8e26bf6033aa9b348aab11dc982055ef7790ff8d53bfd6ae3ab93e12da1f6f63 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-mds-a, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-11T00:35:03.063 INFO:journalctl@ceph.mds.a.smithi012.stdout:Sep 11 00:35:02 smithi012 bash[115031]: ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-mds-a 2024-09-11T00:35:03.063 INFO:journalctl@ceph.mds.a.smithi012.stdout:Sep 11 00:35:02 smithi012 podman[115043]: 2024-09-11 00:35:02.954718399 +0000 UTC m=+0.452378447 container remove 8e26bf6033aa9b348aab11dc982055ef7790ff8d53bfd6ae3ab93e12da1f6f63 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-mds-a, maintainer=Guillaume Abrioux , RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-09-11T00:35:03.228 DEBUG:teuthology.orchestra.run.smithi012:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968@mds.a.service' 2024-09-11T00:35:03.300 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T00:35:03.301 INFO:tasks.cephadm.mds.a:Stopped mds.a 2024-09-11T00:35:03.301 INFO:tasks.cephadm.mds.d:Stopping mds.d... 2024-09-11T00:35:03.301 DEBUG:teuthology.orchestra.run.smithi012:> sudo systemctl stop ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968@mds.d 2024-09-11T00:35:03.563 INFO:journalctl@ceph.mds.d.smithi012.stdout:Sep 11 00:35:03 smithi012 systemd[1]: Stopping Ceph mds.d for ff3ad74a-6fcf-11ef-bcea-c7b262605968... 2024-09-11T00:35:04.063 INFO:journalctl@ceph.mds.d.smithi012.stdout:Sep 11 00:35:03 smithi012 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-mds-d[72159]: 2024-09-11T00:35:03.684+0000 7f2af0200640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.d -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T00:35:04.063 INFO:journalctl@ceph.mds.d.smithi012.stdout:Sep 11 00:35:03 smithi012 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-mds-d[72159]: 2024-09-11T00:35:03.684+0000 7f2af0200640 -1 mds.d *** got signal Terminated *** 2024-09-11T00:35:05.497 INFO:journalctl@ceph.mds.d.smithi012.stdout:Sep 11 00:35:05 smithi012 podman[115153]: 2024-09-11 00:35:05.090875053 +0000 UTC m=+1.510538100 container died 2726b503ecafd1e80d4a0f2918b3ca8dd51ed74472effa1c1f7f5d85c4e86948 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-mds-d, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux ) 2024-09-11T00:35:05.813 INFO:journalctl@ceph.mds.d.smithi012.stdout:Sep 11 00:35:05 smithi012 podman[115153]: 2024-09-11 00:35:05.607095709 +0000 UTC m=+2.026758759 container cleanup 2726b503ecafd1e80d4a0f2918b3ca8dd51ed74472effa1c1f7f5d85c4e86948 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-mds-d, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731) 2024-09-11T00:35:05.813 INFO:journalctl@ceph.mds.d.smithi012.stdout:Sep 11 00:35:05 smithi012 bash[115153]: ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-mds-d 2024-09-11T00:35:06.140 INFO:journalctl@ceph.mds.d.smithi012.stdout:Sep 11 00:35:05 smithi012 podman[115167]: 2024-09-11 00:35:05.875323652 +0000 UTC m=+0.777196500 container remove 2726b503ecafd1e80d4a0f2918b3ca8dd51ed74472effa1c1f7f5d85c4e86948 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-mds-d, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, ceph=True, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_CLEAN=True) 2024-09-11T00:35:06.162 DEBUG:teuthology.orchestra.run.smithi012:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968@mds.d.service' 2024-09-11T00:35:06.232 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T00:35:06.232 INFO:tasks.cephadm.mds.d:Stopped mds.d 2024-09-11T00:35:06.233 INFO:tasks.cephadm.mds.g:Stopping mds.g... 2024-09-11T00:35:06.233 DEBUG:teuthology.orchestra.run.smithi012:> sudo systemctl stop ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968@mds.g 2024-09-11T00:35:06.563 INFO:journalctl@ceph.mds.g.smithi012.stdout:Sep 11 00:35:06 smithi012 systemd[1]: Stopping Ceph mds.g for ff3ad74a-6fcf-11ef-bcea-c7b262605968... 2024-09-11T00:35:07.063 INFO:journalctl@ceph.mds.g.smithi012.stdout:Sep 11 00:35:06 smithi012 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-mds-g[72952]: 2024-09-11T00:35:06.646+0000 7fe7e7000640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.g -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T00:35:07.063 INFO:journalctl@ceph.mds.g.smithi012.stdout:Sep 11 00:35:06 smithi012 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-mds-g[72952]: 2024-09-11T00:35:06.646+0000 7fe7e7000640 -1 mds.g *** got signal Terminated *** 2024-09-11T00:35:10.774 INFO:journalctl@ceph.mds.g.smithi012.stdout:Sep 11 00:35:10 smithi012 podman[115275]: 2024-09-11 00:35:10.474941894 +0000 UTC m=+3.961367705 container died 04ca535a4928c5b8c2fa25f67ceabd5d3b96f951e1826eefefc064e42fbeb938 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-mds-g, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-11T00:35:11.063 INFO:journalctl@ceph.mds.g.smithi012.stdout:Sep 11 00:35:10 smithi012 podman[115275]: 2024-09-11 00:35:10.88593628 +0000 UTC m=+4.372362096 container cleanup 04ca535a4928c5b8c2fa25f67ceabd5d3b96f951e1826eefefc064e42fbeb938 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-mds-g, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.build-date=20240731) 2024-09-11T00:35:11.063 INFO:journalctl@ceph.mds.g.smithi012.stdout:Sep 11 00:35:10 smithi012 bash[115275]: ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-mds-g 2024-09-11T00:35:11.335 INFO:journalctl@ceph.mds.g.smithi012.stdout:Sep 11 00:35:11 smithi012 podman[115287]: 2024-09-11 00:35:11.070305747 +0000 UTC m=+0.581283764 container remove 04ca535a4928c5b8c2fa25f67ceabd5d3b96f951e1826eefefc064e42fbeb938 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-mds-g, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, GIT_BRANCH=HEAD) 2024-09-11T00:35:11.357 DEBUG:teuthology.orchestra.run.smithi012:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968@mds.g.service' 2024-09-11T00:35:11.426 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T00:35:11.427 INFO:tasks.cephadm.mds.g:Stopped mds.g 2024-09-11T00:35:11.427 INFO:tasks.cephadm.mds.j:Stopping mds.j... 2024-09-11T00:35:11.427 DEBUG:teuthology.orchestra.run.smithi012:> sudo systemctl stop ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968@mds.j 2024-09-11T00:35:11.813 INFO:journalctl@ceph.mds.j.smithi012.stdout:Sep 11 00:35:11 smithi012 systemd[1]: Stopping Ceph mds.j for ff3ad74a-6fcf-11ef-bcea-c7b262605968... 2024-09-11T00:35:12.313 INFO:journalctl@ceph.mds.j.smithi012.stdout:Sep 11 00:35:11 smithi012 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-mds-j[71795]: 2024-09-11T00:35:11.816+0000 7f0035c00640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.j -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T00:35:12.313 INFO:journalctl@ceph.mds.j.smithi012.stdout:Sep 11 00:35:11 smithi012 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-mds-j[71795]: 2024-09-11T00:35:11.816+0000 7f0035c00640 -1 mds.j *** got signal Terminated *** 2024-09-11T00:35:13.415 INFO:journalctl@ceph.mds.j.smithi012.stdout:Sep 11 00:35:13 smithi012 podman[115398]: 2024-09-11 00:35:13.151785871 +0000 UTC m=+1.456807342 container died 03bb22653329948ee894ab0727d5f9e6238c8f1d82179e4c84115d5893132343 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-mds-j, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731) 2024-09-11T00:35:13.675 INFO:journalctl@ceph.mds.j.smithi012.stdout:Sep 11 00:35:13 smithi012 podman[115398]: 2024-09-11 00:35:13.497624976 +0000 UTC m=+1.802646445 container cleanup 03bb22653329948ee894ab0727d5f9e6238c8f1d82179e4c84115d5893132343 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-mds-j, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True) 2024-09-11T00:35:13.675 INFO:journalctl@ceph.mds.j.smithi012.stdout:Sep 11 00:35:13 smithi012 bash[115398]: ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-mds-j 2024-09-11T00:35:13.926 INFO:journalctl@ceph.mds.j.smithi012.stdout:Sep 11 00:35:13 smithi012 podman[115410]: 2024-09-11 00:35:13.673766061 +0000 UTC m=+0.517075765 container remove 03bb22653329948ee894ab0727d5f9e6238c8f1d82179e4c84115d5893132343 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-mds-j, ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2) 2024-09-11T00:35:13.949 DEBUG:teuthology.orchestra.run.smithi012:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968@mds.j.service' 2024-09-11T00:35:14.018 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T00:35:14.018 INFO:tasks.cephadm.mds.j:Stopped mds.j 2024-09-11T00:35:14.018 INFO:tasks.cephadm.mds.b:Stopping mds.b... 2024-09-11T00:35:14.018 DEBUG:teuthology.orchestra.run.smithi165:> sudo systemctl stop ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968@mds.b 2024-09-11T00:35:14.335 INFO:journalctl@ceph.mds.b.smithi165.stdout:Sep 11 00:35:14 smithi165 systemd[1]: Stopping Ceph mds.b for ff3ad74a-6fcf-11ef-bcea-c7b262605968... 2024-09-11T00:35:14.782 INFO:journalctl@ceph.mds.b.smithi165.stdout:Sep 11 00:35:14 smithi165 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-mds-b[65581]: 2024-09-11T00:35:14.333+0000 7f2a681c3640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T00:35:14.783 INFO:journalctl@ceph.mds.b.smithi165.stdout:Sep 11 00:35:14 smithi165 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-mds-b[65581]: 2024-09-11T00:35:14.333+0000 7f2a681c3640 -1 mds.b *** got signal Terminated *** 2024-09-11T00:35:18.592 INFO:journalctl@ceph.mds.b.smithi165.stdout:Sep 11 00:35:18 smithi165 podman[76793]: 2024-09-11 00:35:18.33921942 +0000 UTC m=+4.125618723 container died 08c991ef5d62eda76a3f64774a0d3aefdff41c08f646f5aecb088ab64b187ebc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-mds-b, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-11T00:35:18.985 INFO:journalctl@ceph.mds.b.smithi165.stdout:Sep 11 00:35:18 smithi165 podman[76793]: 2024-09-11 00:35:18.691078283 +0000 UTC m=+4.477477585 container cleanup 08c991ef5d62eda76a3f64774a0d3aefdff41c08f646f5aecb088ab64b187ebc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-mds-b, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2) 2024-09-11T00:35:18.985 INFO:journalctl@ceph.mds.b.smithi165.stdout:Sep 11 00:35:18 smithi165 bash[76793]: ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-mds-b 2024-09-11T00:35:19.125 DEBUG:teuthology.orchestra.run.smithi165:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968@mds.b.service' 2024-09-11T00:35:19.161 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T00:35:19.162 INFO:tasks.cephadm.mds.b:Stopped mds.b 2024-09-11T00:35:19.162 INFO:tasks.cephadm.mds.e:Stopping mds.e... 2024-09-11T00:35:19.162 DEBUG:teuthology.orchestra.run.smithi165:> sudo systemctl stop ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968@mds.e 2024-09-11T00:35:19.533 INFO:journalctl@ceph.mds.e.smithi165.stdout:Sep 11 00:35:19 smithi165 systemd[1]: Stopping Ceph mds.e for ff3ad74a-6fcf-11ef-bcea-c7b262605968... 2024-09-11T00:35:19.533 INFO:journalctl@ceph.mds.e.smithi165.stdout:Sep 11 00:35:19 smithi165 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-mds-e[64969]: 2024-09-11T00:35:19.469+0000 7f4709962640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.e -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T00:35:19.533 INFO:journalctl@ceph.mds.e.smithi165.stdout:Sep 11 00:35:19 smithi165 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-mds-e[64969]: 2024-09-11T00:35:19.469+0000 7f4709962640 -1 mds.e *** got signal Terminated *** 2024-09-11T00:35:22.782 INFO:journalctl@ceph.mds.e.smithi165.stdout:Sep 11 00:35:22 smithi165 podman[76910]: 2024-09-11 00:35:22.457853096 +0000 UTC m=+3.080697591 container died ab3021c1e997422c7427fda2d2de6609cab63f982c382ea1744265f51e86fcae (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-mds-e, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-11T00:35:23.097 INFO:journalctl@ceph.mds.e.smithi165.stdout:Sep 11 00:35:22 smithi165 podman[76910]: 2024-09-11 00:35:22.903437488 +0000 UTC m=+3.526281980 container cleanup ab3021c1e997422c7427fda2d2de6609cab63f982c382ea1744265f51e86fcae (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-mds-e, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-11T00:35:23.097 INFO:journalctl@ceph.mds.e.smithi165.stdout:Sep 11 00:35:22 smithi165 bash[76910]: ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-mds-e 2024-09-11T00:35:23.310 DEBUG:teuthology.orchestra.run.smithi165:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968@mds.e.service' 2024-09-11T00:35:23.343 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T00:35:23.343 INFO:tasks.cephadm.mds.e:Stopped mds.e 2024-09-11T00:35:23.343 INFO:tasks.cephadm.mds.h:Stopping mds.h... 2024-09-11T00:35:23.343 DEBUG:teuthology.orchestra.run.smithi165:> sudo systemctl stop ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968@mds.h 2024-09-11T00:35:23.725 INFO:journalctl@ceph.mds.h.smithi165.stdout:Sep 11 00:35:23 smithi165 systemd[1]: Stopping Ceph mds.h for ff3ad74a-6fcf-11ef-bcea-c7b262605968... 2024-09-11T00:35:24.032 INFO:journalctl@ceph.mds.h.smithi165.stdout:Sep 11 00:35:23 smithi165 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-mds-h[65283]: 2024-09-11T00:35:23.723+0000 7f9e8d9b8640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.h -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T00:35:24.032 INFO:journalctl@ceph.mds.h.smithi165.stdout:Sep 11 00:35:23 smithi165 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-mds-h[65283]: 2024-09-11T00:35:23.723+0000 7f9e8d9b8640 -1 mds.h *** got signal Terminated *** 2024-09-11T00:35:26.734 INFO:journalctl@ceph.mds.h.smithi165.stdout:Sep 11 00:35:26 smithi165 podman[77031]: 2024-09-11 00:35:26.470054164 +0000 UTC m=+2.910138784 container died e73b97112e11ff1d50109bd64bd016310f71e41a86e4dd4dc1c4dcf117726462 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-mds-h, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.license=GPLv2) 2024-09-11T00:35:27.033 INFO:journalctl@ceph.mds.h.smithi165.stdout:Sep 11 00:35:26 smithi165 podman[77031]: 2024-09-11 00:35:26.823800878 +0000 UTC m=+3.263885510 container cleanup e73b97112e11ff1d50109bd64bd016310f71e41a86e4dd4dc1c4dcf117726462 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-mds-h, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4) 2024-09-11T00:35:27.033 INFO:journalctl@ceph.mds.h.smithi165.stdout:Sep 11 00:35:26 smithi165 bash[77031]: ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-mds-h 2024-09-11T00:35:27.307 DEBUG:teuthology.orchestra.run.smithi165:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968@mds.h.service' 2024-09-11T00:35:27.339 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T00:35:27.339 INFO:tasks.cephadm.mds.h:Stopped mds.h 2024-09-11T00:35:27.339 INFO:tasks.cephadm.mds.k:Stopping mds.k... 2024-09-11T00:35:27.339 DEBUG:teuthology.orchestra.run.smithi165:> sudo systemctl stop ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968@mds.k 2024-09-11T00:35:27.783 INFO:journalctl@ceph.mds.k.smithi165.stdout:Sep 11 00:35:27 smithi165 systemd[1]: Stopping Ceph mds.k for ff3ad74a-6fcf-11ef-bcea-c7b262605968... 2024-09-11T00:35:27.783 INFO:journalctl@ceph.mds.k.smithi165.stdout:Sep 11 00:35:27 smithi165 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-mds-k[64659]: 2024-09-11T00:35:27.644+0000 7f624a34c640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.k -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T00:35:27.783 INFO:journalctl@ceph.mds.k.smithi165.stdout:Sep 11 00:35:27 smithi165 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-mds-k[64659]: 2024-09-11T00:35:27.644+0000 7f624a34c640 -1 mds.k *** got signal Terminated *** 2024-09-11T00:35:30.746 INFO:journalctl@ceph.mds.k.smithi165.stdout:Sep 11 00:35:30 smithi165 podman[77150]: 2024-09-11 00:35:30.470571954 +0000 UTC m=+2.918409212 container died b4e8ce1b141f220d23af15f9e791320400e7c4586cb337961d305b338652fd8c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-mds-k, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-11T00:35:31.033 INFO:journalctl@ceph.mds.k.smithi165.stdout:Sep 11 00:35:30 smithi165 podman[77150]: 2024-09-11 00:35:30.819406583 +0000 UTC m=+3.267243836 container cleanup b4e8ce1b141f220d23af15f9e791320400e7c4586cb337961d305b338652fd8c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-mds-k, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-11T00:35:31.033 INFO:journalctl@ceph.mds.k.smithi165.stdout:Sep 11 00:35:30 smithi165 bash[77150]: ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-mds-k 2024-09-11T00:35:31.295 DEBUG:teuthology.orchestra.run.smithi165:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968@mds.k.service' 2024-09-11T00:35:31.327 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T00:35:31.327 INFO:tasks.cephadm.mds.k:Stopped mds.k 2024-09-11T00:35:31.327 INFO:tasks.cephadm.mds.c:Stopping mds.c... 2024-09-11T00:35:31.327 DEBUG:teuthology.orchestra.run.smithi178:> sudo systemctl stop ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968@mds.c 2024-09-11T00:35:31.657 INFO:journalctl@ceph.mds.c.smithi178.stdout:Sep 11 00:35:31 smithi178 systemd[1]: Stopping Ceph mds.c for ff3ad74a-6fcf-11ef-bcea-c7b262605968... 2024-09-11T00:35:31.913 INFO:journalctl@ceph.mds.c.smithi178.stdout:Sep 11 00:35:31 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-mds-c[66074]: 2024-09-11T00:35:31.720+0000 7f8ee8b0b640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T00:35:31.913 INFO:journalctl@ceph.mds.c.smithi178.stdout:Sep 11 00:35:31 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-mds-c[66074]: 2024-09-11T00:35:31.720+0000 7f8ee8b0b640 -1 mds.c *** got signal Terminated *** 2024-09-11T00:35:34.781 INFO:journalctl@ceph.mds.c.smithi178.stdout:Sep 11 00:35:34 smithi178 podman[79475]: 2024-09-11 00:35:34.456586515 +0000 UTC m=+2.933698718 container died 93a61ce72254fe725a8e5520275388e03a48417cad7a691c76a277ab16ff0dc6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-mds-c, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0) 2024-09-11T00:35:35.047 INFO:journalctl@ceph.mds.c.smithi178.stdout:Sep 11 00:35:34 smithi178 podman[79475]: 2024-09-11 00:35:34.861825148 +0000 UTC m=+3.338937356 container cleanup 93a61ce72254fe725a8e5520275388e03a48417cad7a691c76a277ab16ff0dc6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-mds-c, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-09-11T00:35:35.048 INFO:journalctl@ceph.mds.c.smithi178.stdout:Sep 11 00:35:34 smithi178 bash[79475]: ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-mds-c 2024-09-11T00:35:35.280 DEBUG:teuthology.orchestra.run.smithi178:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968@mds.c.service' 2024-09-11T00:35:35.302 INFO:journalctl@ceph.mds.c.smithi178.stdout:Sep 11 00:35:35 smithi178 podman[79501]: 2024-09-11 00:35:35.046212348 +0000 UTC m=+0.587875252 container remove 93a61ce72254fe725a8e5520275388e03a48417cad7a691c76a277ab16ff0dc6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-mds-c, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , ceph=True, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-11T00:35:35.302 INFO:journalctl@ceph.mds.c.smithi178.stdout:Sep 11 00:35:35 smithi178 systemd[1]: ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968@mds.c.service: Deactivated successfully. 2024-09-11T00:35:35.303 INFO:journalctl@ceph.mds.c.smithi178.stdout:Sep 11 00:35:35 smithi178 systemd[1]: Stopped Ceph mds.c for ff3ad74a-6fcf-11ef-bcea-c7b262605968. 2024-09-11T00:35:35.303 INFO:journalctl@ceph.mds.c.smithi178.stdout:Sep 11 00:35:35 smithi178 systemd[1]: ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968@mds.c.service: Consumed 22.406s CPU time. 2024-09-11T00:35:35.316 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T00:35:35.317 INFO:tasks.cephadm.mds.c:Stopped mds.c 2024-09-11T00:35:35.317 INFO:tasks.cephadm.mds.f:Stopping mds.f... 2024-09-11T00:35:35.317 DEBUG:teuthology.orchestra.run.smithi178:> sudo systemctl stop ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968@mds.f 2024-09-11T00:35:35.581 INFO:journalctl@ceph.mds.f.smithi178.stdout:Sep 11 00:35:35 smithi178 systemd[1]: Stopping Ceph mds.f for ff3ad74a-6fcf-11ef-bcea-c7b262605968... 2024-09-11T00:35:36.081 INFO:journalctl@ceph.mds.f.smithi178.stdout:Sep 11 00:35:35 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-mds-f[65776]: 2024-09-11T00:35:35.632+0000 7f795529a640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.f -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T00:35:36.081 INFO:journalctl@ceph.mds.f.smithi178.stdout:Sep 11 00:35:35 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-mds-f[65776]: 2024-09-11T00:35:35.632+0000 7f795529a640 -1 mds.f *** got signal Terminated *** 2024-09-11T00:35:38.717 INFO:journalctl@ceph.mds.f.smithi178.stdout:Sep 11 00:35:38 smithi178 podman[79606]: 2024-09-11 00:35:38.451486071 +0000 UTC m=+2.916130866 container died d1d64697e6990eaeca3e9a3fcdbc1b04051498931e300f9cd173b967494d28ab (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-mds-f, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240731, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-11T00:35:39.081 INFO:journalctl@ceph.mds.f.smithi178.stdout:Sep 11 00:35:38 smithi178 podman[79606]: 2024-09-11 00:35:38.81534536 +0000 UTC m=+3.279990224 container cleanup d1d64697e6990eaeca3e9a3fcdbc1b04051498931e300f9cd173b967494d28ab (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-mds-f, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.37.2) 2024-09-11T00:35:39.081 INFO:journalctl@ceph.mds.f.smithi178.stdout:Sep 11 00:35:38 smithi178 bash[79606]: ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-mds-f 2024-09-11T00:35:39.243 DEBUG:teuthology.orchestra.run.smithi178:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968@mds.f.service' 2024-09-11T00:35:39.277 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T00:35:39.277 INFO:tasks.cephadm.mds.f:Stopped mds.f 2024-09-11T00:35:39.278 INFO:tasks.cephadm.mds.i:Stopping mds.i... 2024-09-11T00:35:39.278 DEBUG:teuthology.orchestra.run.smithi178:> sudo systemctl stop ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968@mds.i 2024-09-11T00:35:39.831 INFO:journalctl@ceph.mds.i.smithi178.stdout:Sep 11 00:35:39 smithi178 systemd[1]: Stopping Ceph mds.i for ff3ad74a-6fcf-11ef-bcea-c7b262605968... 2024-09-11T00:35:39.831 INFO:journalctl@ceph.mds.i.smithi178.stdout:Sep 11 00:35:39 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-mds-i[66387]: 2024-09-11T00:35:39.602+0000 7fc510e69640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.i -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T00:35:39.831 INFO:journalctl@ceph.mds.i.smithi178.stdout:Sep 11 00:35:39 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-mds-i[66387]: 2024-09-11T00:35:39.602+0000 7fc510e69640 -1 mds.i *** got signal Terminated *** 2024-09-11T00:35:42.720 INFO:journalctl@ceph.mds.i.smithi178.stdout:Sep 11 00:35:42 smithi178 podman[79726]: 2024-09-11 00:35:42.43898492 +0000 UTC m=+2.939655300 container died 99c4f702fdec80f2d032dd2d254d6ce9eaf6c09e0099319e32a4b82697446762 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-mds-i, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , ceph=True, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-11T00:35:43.029 INFO:journalctl@ceph.mds.i.smithi178.stdout:Sep 11 00:35:42 smithi178 podman[79726]: 2024-09-11 00:35:42.802009262 +0000 UTC m=+3.302679640 container cleanup 99c4f702fdec80f2d032dd2d254d6ce9eaf6c09e0099319e32a4b82697446762 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-mds-i, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, ceph=True, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.schema-version=1.0, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-11T00:35:43.029 INFO:journalctl@ceph.mds.i.smithi178.stdout:Sep 11 00:35:42 smithi178 bash[79726]: ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-mds-i 2024-09-11T00:35:43.158 DEBUG:teuthology.orchestra.run.smithi178:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968@mds.i.service' 2024-09-11T00:35:43.191 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T00:35:43.191 INFO:tasks.cephadm.mds.i:Stopped mds.i 2024-09-11T00:35:43.191 INFO:tasks.cephadm.mds.l:Stopping mds.l... 2024-09-11T00:35:43.191 DEBUG:teuthology.orchestra.run.smithi178:> sudo systemctl stop ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968@mds.l 2024-09-11T00:35:43.331 INFO:journalctl@ceph.mds.l.smithi178.stdout:Sep 11 00:35:43 smithi178 systemd[1]: Stopping Ceph mds.l for ff3ad74a-6fcf-11ef-bcea-c7b262605968... 2024-09-11T00:35:43.831 INFO:journalctl@ceph.mds.l.smithi178.stdout:Sep 11 00:35:43 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-mds-l[65476]: 2024-09-11T00:35:43.505+0000 7f62f40a4640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.l -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T00:35:43.831 INFO:journalctl@ceph.mds.l.smithi178.stdout:Sep 11 00:35:43 smithi178 ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-mds-l[65476]: 2024-09-11T00:35:43.505+0000 7f62f40a4640 -1 mds.l *** got signal Terminated *** 2024-09-11T00:35:46.809 INFO:journalctl@ceph.mds.l.smithi178.stdout:Sep 11 00:35:46 smithi178 podman[79844]: 2024-09-11 00:35:46.5323165 +0000 UTC m=+3.126352377 container died f02db3543a9a1bb4f605b09d1bce6c8b7983e10355c399b367353636cae4b226 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-mds-l, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2) 2024-09-11T00:35:47.081 INFO:journalctl@ceph.mds.l.smithi178.stdout:Sep 11 00:35:46 smithi178 podman[79844]: 2024-09-11 00:35:46.897249884 +0000 UTC m=+3.491285761 container cleanup f02db3543a9a1bb4f605b09d1bce6c8b7983e10355c399b367353636cae4b226 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-mds-l, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4) 2024-09-11T00:35:47.081 INFO:journalctl@ceph.mds.l.smithi178.stdout:Sep 11 00:35:46 smithi178 bash[79844]: ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968-mds-l 2024-09-11T00:35:47.361 DEBUG:teuthology.orchestra.run.smithi178:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ff3ad74a-6fcf-11ef-bcea-c7b262605968@mds.l.service' 2024-09-11T00:35:47.392 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T00:35:47.393 INFO:tasks.cephadm.mds.l:Stopped mds.l 2024-09-11T00:35:47.393 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid ff3ad74a-6fcf-11ef-bcea-c7b262605968 --force --keep-logs 2024-09-11T00:35:47.678 INFO:teuthology.orchestra.run.smithi012.stdout:Deleting cluster with fsid: ff3ad74a-6fcf-11ef-bcea-c7b262605968 2024-09-11T00:35:51.874 DEBUG:teuthology.orchestra.run.smithi165:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid ff3ad74a-6fcf-11ef-bcea-c7b262605968 --force --keep-logs 2024-09-11T00:35:52.076 INFO:teuthology.orchestra.run.smithi165.stdout:Deleting cluster with fsid: ff3ad74a-6fcf-11ef-bcea-c7b262605968 2024-09-11T00:35:54.621 DEBUG:teuthology.orchestra.run.smithi178:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid ff3ad74a-6fcf-11ef-bcea-c7b262605968 --force --keep-logs 2024-09-11T00:35:54.823 INFO:teuthology.orchestra.run.smithi178.stdout:Deleting cluster with fsid: ff3ad74a-6fcf-11ef-bcea-c7b262605968 2024-09-11T00:35:57.372 DEBUG:teuthology.orchestra.run.smithi012:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-11T00:35:57.436 DEBUG:teuthology.orchestra.run.smithi165:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-11T00:35:57.468 DEBUG:teuthology.orchestra.run.smithi178:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-11T00:35:57.495 INFO:tasks.cephadm:Archiving crash dumps... 2024-09-11T00:35:57.498 DEBUG:teuthology.misc:Transferring archived files from smithi012:/var/lib/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/crash to /home/teuthworker/archive/hyelloji-2024-09-10_16:59:47-fs-wip-hemanth-testing-2024-09-10-0723-testing-default-smithi/7899811/remote/smithi012/crash 2024-09-11T00:35:57.498 DEBUG:teuthology.orchestra.run.smithi012:> sudo tar c -f - -C /var/lib/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/crash -- . 2024-09-11T00:35:57.569 INFO:teuthology.orchestra.run.smithi012.stderr:tar: /var/lib/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/crash: Cannot open: No such file or directory 2024-09-11T00:35:57.570 INFO:teuthology.orchestra.run.smithi012.stderr:tar: Error is not recoverable: exiting now 2024-09-11T00:35:57.573 DEBUG:teuthology.misc:Transferring archived files from smithi165:/var/lib/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/crash to /home/teuthworker/archive/hyelloji-2024-09-10_16:59:47-fs-wip-hemanth-testing-2024-09-10-0723-testing-default-smithi/7899811/remote/smithi165/crash 2024-09-11T00:35:57.574 DEBUG:teuthology.orchestra.run.smithi165:> sudo tar c -f - -C /var/lib/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/crash -- . 2024-09-11T00:35:57.618 INFO:teuthology.orchestra.run.smithi165.stderr:tar: /var/lib/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/crash: Cannot open: No such file or directory 2024-09-11T00:35:57.618 INFO:teuthology.orchestra.run.smithi165.stderr:tar: Error is not recoverable: exiting now 2024-09-11T00:35:57.620 DEBUG:teuthology.misc:Transferring archived files from smithi178:/var/lib/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/crash to /home/teuthworker/archive/hyelloji-2024-09-10_16:59:47-fs-wip-hemanth-testing-2024-09-10-0723-testing-default-smithi/7899811/remote/smithi178/crash 2024-09-11T00:35:57.621 DEBUG:teuthology.orchestra.run.smithi178:> sudo tar c -f - -C /var/lib/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/crash -- . 2024-09-11T00:35:57.666 INFO:teuthology.orchestra.run.smithi178.stderr:tar: /var/lib/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/crash: Cannot open: No such file or directory 2024-09-11T00:35:57.666 INFO:teuthology.orchestra.run.smithi178.stderr:tar: Error is not recoverable: exiting now 2024-09-11T00:35:57.667 INFO:tasks.cephadm:Checking cluster log for badness... 2024-09-11T00:35:57.667 DEBUG:teuthology.orchestra.run.smithi012:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v 'fs.*is degraded' | egrep -v 'filesystem is degraded' | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v 'filesystem is offline' | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v 'insufficient standby MDS daemons available' | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'online, but wants' | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'do not have an application enabled' | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'Degraded data redundancy' | head -n 1 2024-09-11T00:35:57.789 INFO:tasks.cephadm:Compressing logs... 2024-09-11T00:35:57.789 DEBUG:teuthology.orchestra.run.smithi012:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-11T00:35:57.793 DEBUG:teuthology.orchestra.run.smithi165:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-11T00:35:57.796 DEBUG:teuthology.orchestra.run.smithi178:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-11T00:35:57.826 INFO:teuthology.orchestra.run.smithi165.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-09-11T00:35:57.827 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/ceph-osd.10.log 2024-09-11T00:35:57.828 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/ceph-osd.1.log 2024-09-11T00:35:57.828 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/ceph-osd.10.log: gzip -5 --verbose -- /var/log/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/ceph-osd.7.log 2024-09-11T00:35:57.828 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/ceph-volume.log 2024-09-11T00:35:57.829 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/ceph-mds.b.log 2024-09-11T00:35:57.830 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/ceph-osd.7.log: /var/log/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/ceph-mds.k.log 2024-09-11T00:35:57.830 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/ceph-mds.b.log: gzip -5 --verbose -- /var/log/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/ceph-osd.4.log 2024-09-11T00:35:57.830 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/ceph-mds.k.log: gzip -5 --verbose -- /var/log/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/ceph.audit.log 2024-09-11T00:35:57.830 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/ceph.cephadm.log 2024-09-11T00:35:57.831 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/ceph-mgr.y.log 2024-09-11T00:35:57.831 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/ceph.log 2024-09-11T00:35:57.831 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/ceph-mds.h.log 2024-09-11T00:35:57.831 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/ceph-mon.b.log 2024-09-11T00:35:57.831 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/ceph-mds.e.log 2024-09-11T00:35:57.831 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-09-11T00:35:57.834 INFO:teuthology.orchestra.run.smithi178.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-09-11T00:35:57.835 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/ceph-volume.log 2024-09-11T00:35:57.836 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/ceph.audit.log 2024-09-11T00:35:57.836 INFO:teuthology.orchestra.run.smithi178.stderr:gzip/var/log/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/ceph-volume.log: -5 --verbose -- /var/log/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/ceph-mds.i.log 2024-09-11T00:35:57.839 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/ceph-mon.c.log 2024-09-11T00:35:57.839 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/ceph-mds.i.log: gzip -5 --verbose -- /var/log/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/ceph.cephadm.log 2024-09-11T00:35:57.840 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/ceph-osd.11.log 2024-09-11T00:35:57.840 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/ceph-osd.2.log 2024-09-11T00:35:57.840 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/ceph-osd.11.log: gzip -5 --verbose -- /var/log/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/ceph-osd.5.log 2024-09-11T00:35:57.840 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/ceph.log 2024-09-11T00:35:57.840 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/ceph-mds.l.log 2024-09-11T00:35:57.840 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/ceph-osd.8.log 2024-09-11T00:35:57.840 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/ceph-mds.l.log: gzip -5 --verbose -- /var/log/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/ceph-mgr.z.log 2024-09-11T00:35:57.840 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/ceph-osd.8.log: gzip -5 --verbose -- /var/log/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/ceph-mds.c.log 2024-09-11T00:35:57.840 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/ceph-mgr.z.log: gzip -5 --verbose -- /var/log/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/ceph-mds.f.log 2024-09-11T00:35:57.840 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-09-11T00:35:57.864 INFO:teuthology.orchestra.run.smithi012.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-09-11T00:35:57.865 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.log 2024-09-11T00:35:57.865 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/ceph-osd.0.log 2024-09-11T00:35:57.866 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.log: gzip -5 --verbose -- /var/log/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/ceph-osd.9.log 2024-09-11T00:35:57.867 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/ceph-volume.log 2024-09-11T00:35:57.867 INFO:teuthology.orchestra.run.smithi012.stderr: 92.4% -- replaced with /var/log/ceph/ceph-client.admin.log.gz 2024-09-11T00:35:57.868 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/ceph-osd.9.log: gzip -5 --verbose -- /var/log/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/ceph.audit.log 2024-09-11T00:35:57.868 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/ceph-mds.a.log 2024-09-11T00:35:57.869 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/ceph.cephadm.log 2024-09-11T00:35:57.870 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/ceph-mds.a.log: gzip -5 --verbose -- /var/log/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/ceph-mon.a.log 2024-09-11T00:35:57.871 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/ceph-mds.g.log 2024-09-11T00:35:57.871 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/ceph-osd.6.log 2024-09-11T00:35:57.872 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/ceph-mds.g.log: gzip -5 --verbose -- /var/log/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/ceph.log 2024-09-11T00:35:57.873 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/ceph-mds.d.log 2024-09-11T00:35:57.874 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/ceph-mgr.x.log 2024-09-11T00:35:57.874 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/ceph-mds.d.log: gzip -5 --verbose -- /var/log/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/ceph-mds.j.log 2024-09-11T00:35:57.875 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/ceph-osd.3.log 2024-09-11T00:35:57.876 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/ceph-mgr.x.log: /var/log/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/ceph-mds.j.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-09-11T00:35:57.878 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/ceph-osd.3.log: 86.6% -- replaced with /var/log/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/ceph.log.gz 2024-09-11T00:35:57.883 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/cephadm.log: 95.0% -- replaced with /var/log/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/ceph-mds.d.log.gz 2024-09-11T00:35:57.885 INFO:teuthology.orchestra.run.smithi012.stderr: 95.1% -- replaced with /var/log/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/ceph-mds.j.log.gz 2024-09-11T00:35:57.901 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/cephadm.log: /var/log/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/ceph-osd.4.log: /var/log/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/ceph.audit.log: /var/log/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/ceph.cephadm.log: /var/log/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/ceph-mgr.y.log: /var/log/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/ceph.log: /var/log/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/ceph-mds.h.log: /var/log/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/ceph-mon.b.log: /var/log/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/ceph-mds.e.log: 91.8% -- replaced with /var/log/ceph/cephadm.log.gz 2024-09-11T00:35:57.937 INFO:teuthology.orchestra.run.smithi012.stderr: 82.6% -- replaced with /var/log/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/ceph.cephadm.log.gz 2024-09-11T00:35:57.943 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/ceph-mds.f.log: /var/log/ceph/cephadm.log: 86.6% -- replaced with /var/log/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/ceph.log.gz 2024-09-11T00:35:57.966 INFO:teuthology.orchestra.run.smithi178.stderr: 82.6% -- replaced with /var/log/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/ceph.cephadm.log.gz 2024-09-11T00:35:57.992 INFO:teuthology.orchestra.run.smithi178.stderr: 90.5% -- replaced with /var/log/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/ceph.audit.log.gz 2024-09-11T00:35:58.014 INFO:teuthology.orchestra.run.smithi012.stderr: 93.7% -- replaced with /var/log/ceph/cephadm.log.gz 2024-09-11T00:35:58.043 INFO:teuthology.orchestra.run.smithi178.stderr: 93.4% -- replaced with /var/log/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/ceph-mgr.z.log.gz 2024-09-11T00:35:58.047 INFO:teuthology.orchestra.run.smithi165.stderr: 93.3% -- replaced with /var/log/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/ceph-mgr.y.log.gz 2024-09-11T00:35:58.055 INFO:teuthology.orchestra.run.smithi012.stderr: 90.3% -- replaced with /var/log/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/ceph.audit.log.gz 2024-09-11T00:35:58.087 INFO:teuthology.orchestra.run.smithi165.stderr: 91.0% -- replaced with /var/log/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/ceph-volume.log.gz 2024-09-11T00:35:58.136 INFO:teuthology.orchestra.run.smithi012.stderr: 90.9% -- replaced with /var/log/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/ceph-volume.log.gz 2024-09-11T00:35:58.152 INFO:teuthology.orchestra.run.smithi165.stderr: 82.7% -- replaced with /var/log/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/ceph.cephadm.log.gz 2024-09-11T00:35:58.296 INFO:teuthology.orchestra.run.smithi178.stderr: 92.2% -- replaced with /var/log/ceph/cephadm.log.gz 2024-09-11T00:35:58.507 INFO:teuthology.orchestra.run.smithi165.stderr: 90.4% -- replaced with /var/log/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/ceph.audit.log.gz 2024-09-11T00:35:58.637 INFO:teuthology.orchestra.run.smithi178.stderr: 90.9% -- replaced with /var/log/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/ceph-volume.log.gz 2024-09-11T00:35:58.657 INFO:teuthology.orchestra.run.smithi165.stderr: 86.6% -- replaced with /var/log/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/ceph.log.gz 2024-09-11T00:35:59.278 INFO:teuthology.orchestra.run.smithi178.stderr: 91.6% -- replaced with /var/log/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/ceph-mon.c.log.gz 2024-09-11T00:36:00.253 INFO:teuthology.orchestra.run.smithi012.stderr: 90.6% -- replaced with /var/log/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/ceph-mgr.x.log.gz 2024-09-11T00:36:06.246 INFO:teuthology.orchestra.run.smithi012.stderr: 90.2% -- replaced with /var/log/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/ceph-mon.a.log.gz 2024-09-11T00:36:41.070 INFO:teuthology.orchestra.run.smithi178.stderr: 95.2% -- replaced with /var/log/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/ceph-mds.l.log.gz 2024-09-11T00:36:47.341 INFO:teuthology.orchestra.run.smithi178.stderr: 95.2% -- replaced with /var/log/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/ceph-mds.c.log.gz 2024-09-11T00:36:49.747 INFO:teuthology.orchestra.run.smithi012.stderr: 95.2% -- replaced with /var/log/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/ceph-mds.a.log.gz 2024-09-11T00:36:55.627 INFO:teuthology.orchestra.run.smithi165.stderr: 93.0% -- replaced with /var/log/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/ceph-mon.b.log.gz 2024-09-11T00:36:58.552 INFO:teuthology.orchestra.run.smithi178.stderr: 95.3% -- replaced with /var/log/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/ceph-mds.f.log.gz 2024-09-11T00:37:52.115 INFO:teuthology.orchestra.run.smithi178.stderr: 91.3% -- replaced with /var/log/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/ceph-osd.2.log.gz 2024-09-11T00:38:05.824 INFO:teuthology.orchestra.run.smithi178.stderr: 91.0% -- replaced with /var/log/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/ceph-osd.8.log.gz 2024-09-11T00:38:33.749 INFO:teuthology.orchestra.run.smithi012.stderr: 91.5% -- replaced with /var/log/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/ceph-osd.0.log.gz 2024-09-11T00:38:40.618 INFO:teuthology.orchestra.run.smithi178.stderr: 91.3% -- replaced with /var/log/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/ceph-osd.11.log.gz 2024-09-11T00:38:44.263 INFO:teuthology.orchestra.run.smithi012.stderr: 91.4% -- replaced with /var/log/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/ceph-osd.3.log.gz 2024-09-11T00:39:03.541 INFO:teuthology.orchestra.run.smithi178.stderr: 91.1% -- replaced with /var/log/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/ceph-osd.5.log.gz 2024-09-11T00:39:08.306 INFO:teuthology.orchestra.run.smithi012.stderr: 91.4% -- replaced with /var/log/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/ceph-osd.6.log.gz 2024-09-11T00:39:17.542 INFO:teuthology.orchestra.run.smithi012.stderr: 91.1% -- replaced with /var/log/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/ceph-osd.9.log.gz 2024-09-11T00:39:36.559 INFO:teuthology.orchestra.run.smithi178.stderr: 93.2% -- replaced with /var/log/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/ceph-mds.i.log.gz 2024-09-11T00:39:36.560 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-11T00:39:36.560 INFO:teuthology.orchestra.run.smithi178.stderr:real 3m38.750s 2024-09-11T00:39:36.560 INFO:teuthology.orchestra.run.smithi178.stderr:user 4m41.937s 2024-09-11T00:39:36.561 INFO:teuthology.orchestra.run.smithi178.stderr:sys 0m10.357s 2024-09-11T00:40:09.254 INFO:teuthology.orchestra.run.smithi012.stderr: 93.2% -- replaced with /var/log/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/ceph-mds.g.log.gz 2024-09-11T00:40:09.257 INFO:teuthology.orchestra.run.smithi012.stderr: 2024-09-11T00:40:09.258 INFO:teuthology.orchestra.run.smithi012.stderr:real 4m11.435s 2024-09-11T00:40:09.258 INFO:teuthology.orchestra.run.smithi012.stderr:user 4m47.896s 2024-09-11T00:40:09.258 INFO:teuthology.orchestra.run.smithi012.stderr:sys 0m28.075s 2024-09-11T00:40:25.030 INFO:teuthology.orchestra.run.smithi165.stderr: 95.3% -- replaced with /var/log/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/ceph-mds.k.log.gz 2024-09-11T00:41:25.236 INFO:teuthology.orchestra.run.smithi165.stderr: 91.6% -- replaced with /var/log/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/ceph-osd.1.log.gz 2024-09-11T00:42:08.026 INFO:teuthology.orchestra.run.smithi165.stderr: 91.2% -- replaced with /var/log/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/ceph-osd.7.log.gz 2024-09-11T00:42:15.850 INFO:teuthology.orchestra.run.smithi165.stderr: 91.3% -- replaced with /var/log/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/ceph-osd.10.log.gz 2024-09-11T00:47:11.555 INFO:teuthology.orchestra.run.smithi165.stderr: 91.4% -- replaced with /var/log/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/ceph-osd.4.log.gz 2024-09-11T00:50:14.374 INFO:teuthology.orchestra.run.smithi165.stderr: 93.1% -- replaced with /var/log/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/ceph-mds.h.log.gz 2024-09-11T00:51:49.736 INFO:teuthology.orchestra.run.smithi165.stderr: 93.1% -- replaced with /var/log/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/ceph-mds.e.log.gz 2024-09-11T00:52:57.693 INFO:teuthology.orchestra.run.smithi165.stderr: 93.3% -- replaced with /var/log/ceph/ff3ad74a-6fcf-11ef-bcea-c7b262605968/ceph-mds.b.log.gz 2024-09-11T00:52:57.695 INFO:teuthology.orchestra.run.smithi165.stderr: 2024-09-11T00:52:57.695 INFO:teuthology.orchestra.run.smithi165.stderr:real 16m59.883s 2024-09-11T00:52:57.695 INFO:teuthology.orchestra.run.smithi165.stderr:user 7m40.577s 2024-09-11T00:52:57.695 INFO:teuthology.orchestra.run.smithi165.stderr:sys 0m21.789s 2024-09-11T00:52:57.696 INFO:tasks.cephadm:Archiving logs... 2024-09-11T00:52:57.697 DEBUG:teuthology.misc:Transferring archived files from smithi012:/var/log/ceph to /home/teuthworker/archive/hyelloji-2024-09-10_16:59:47-fs-wip-hemanth-testing-2024-09-10-0723-testing-default-smithi/7899811/remote/smithi012/log 2024-09-11T00:52:57.698 DEBUG:teuthology.orchestra.run.smithi012:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-11T00:53:40.920 DEBUG:teuthology.misc:Transferring archived files from smithi165:/var/log/ceph to /home/teuthworker/archive/hyelloji-2024-09-10_16:59:47-fs-wip-hemanth-testing-2024-09-10-0723-testing-default-smithi/7899811/remote/smithi165/log 2024-09-11T00:53:40.921 DEBUG:teuthology.orchestra.run.smithi165:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-11T00:54:51.594 DEBUG:teuthology.misc:Transferring archived files from smithi178:/var/log/ceph to /home/teuthworker/archive/hyelloji-2024-09-10_16:59:47-fs-wip-hemanth-testing-2024-09-10-0723-testing-default-smithi/7899811/remote/smithi178/log 2024-09-11T00:54:51.596 DEBUG:teuthology.orchestra.run.smithi178:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-11T00:55:40.928 INFO:tasks.cephadm:Removing cluster... 2024-09-11T00:55:40.929 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid ff3ad74a-6fcf-11ef-bcea-c7b262605968 --force 2024-09-11T00:55:41.305 INFO:teuthology.orchestra.run.smithi012.stdout:Deleting cluster with fsid: ff3ad74a-6fcf-11ef-bcea-c7b262605968 2024-09-11T00:55:42.878 DEBUG:teuthology.orchestra.run.smithi165:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid ff3ad74a-6fcf-11ef-bcea-c7b262605968 --force 2024-09-11T00:55:43.119 INFO:teuthology.orchestra.run.smithi165.stdout:Deleting cluster with fsid: ff3ad74a-6fcf-11ef-bcea-c7b262605968 2024-09-11T00:55:43.934 DEBUG:teuthology.orchestra.run.smithi178:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid ff3ad74a-6fcf-11ef-bcea-c7b262605968 --force 2024-09-11T00:55:44.136 INFO:teuthology.orchestra.run.smithi178.stdout:Deleting cluster with fsid: ff3ad74a-6fcf-11ef-bcea-c7b262605968 2024-09-11T00:55:44.826 INFO:tasks.cephadm:Removing cephadm ... 2024-09-11T00:55:44.826 DEBUG:teuthology.orchestra.run.smithi012:> rm -rf /home/ubuntu/cephtest/cephadm 2024-09-11T00:55:44.868 DEBUG:teuthology.orchestra.run.smithi165:> rm -rf /home/ubuntu/cephtest/cephadm 2024-09-11T00:55:44.887 DEBUG:teuthology.orchestra.run.smithi178:> rm -rf /home/ubuntu/cephtest/cephadm 2024-09-11T00:55:44.903 INFO:tasks.cephadm:Teardown complete 2024-09-11T00:55:44.903 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-09-11T00:55:44.914 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-09-11T00:55:44.914 DEBUG:teuthology.orchestra.run.smithi012:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-11T00:55:44.919 DEBUG:teuthology.orchestra.run.smithi165:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-11T00:55:44.930 DEBUG:teuthology.orchestra.run.smithi178:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-11T00:55:45.084 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-09-11T00:55:45.094 INFO:teuthology.task.clock:Checking final clock skew... 2024-09-11T00:55:45.094 DEBUG:teuthology.orchestra.run.smithi012:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-11T00:55:45.099 DEBUG:teuthology.orchestra.run.smithi165:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-11T00:55:45.102 DEBUG:teuthology.orchestra.run.smithi178:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-11T00:55:45.115 INFO:teuthology.orchestra.run.smithi165.stderr:bash: line 1: ntpq: command not found 2024-09-11T00:55:45.116 INFO:teuthology.orchestra.run.smithi178.stderr:bash: line 1: ntpq: command not found 2024-09-11T00:55:45.130 INFO:teuthology.orchestra.run.smithi012.stderr:bash: line 1: ntpq: command not found 2024-09-11T00:55:45.134 INFO:teuthology.orchestra.run.smithi178.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-11T00:55:45.134 INFO:teuthology.orchestra.run.smithi178.stdout:=============================================================================== 2024-09-11T00:55:45.134 INFO:teuthology.orchestra.run.smithi178.stdout:^* hv01.front.sepia.ceph.com 3 6 377 51 -881us[ -691us] +/- 50ms 2024-09-11T00:55:45.134 INFO:teuthology.orchestra.run.smithi178.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 49 +1121us[+1121us] +/- 41ms 2024-09-11T00:55:45.134 INFO:teuthology.orchestra.run.smithi178.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 52 -505us[ -314us] +/- 63ms 2024-09-11T00:55:45.134 INFO:teuthology.orchestra.run.smithi178.stdout:^? hv04.front.sepia.ceph.com 0 10 0 - +0ns[ +0ns] +/- 0ns 2024-09-11T00:55:45.135 INFO:teuthology.orchestra.run.smithi165.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-11T00:55:45.135 INFO:teuthology.orchestra.run.smithi165.stdout:=============================================================================== 2024-09-11T00:55:45.135 INFO:teuthology.orchestra.run.smithi165.stdout:^* hv01.front.sepia.ceph.com 3 6 377 53 -885us[ -722us] +/- 49ms 2024-09-11T00:55:45.135 INFO:teuthology.orchestra.run.smithi165.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 51 +1109us[+1109us] +/- 41ms 2024-09-11T00:55:45.135 INFO:teuthology.orchestra.run.smithi165.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 52 -531us[ -531us] +/- 63ms 2024-09-11T00:55:45.135 INFO:teuthology.orchestra.run.smithi165.stdout:^? hv04.front.sepia.ceph.com 0 10 0 - +0ns[ +0ns] +/- 0ns 2024-09-11T00:55:45.229 INFO:teuthology.orchestra.run.smithi012.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-11T00:55:45.230 INFO:teuthology.orchestra.run.smithi012.stdout:=============================================================================== 2024-09-11T00:55:45.230 INFO:teuthology.orchestra.run.smithi012.stdout:^* hv01.front.sepia.ceph.com 3 6 377 51 -874us[ -875us] +/- 50ms 2024-09-11T00:55:45.231 INFO:teuthology.orchestra.run.smithi012.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 53 +1128us[+1127us] +/- 40ms 2024-09-11T00:55:45.231 INFO:teuthology.orchestra.run.smithi012.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 48 -477us[ -477us] +/- 63ms 2024-09-11T00:55:45.231 INFO:teuthology.orchestra.run.smithi012.stdout:^? hv04.front.sepia.ceph.com 0 10 0 - +0ns[ +0ns] +/- 0ns 2024-09-11T00:55:45.232 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-09-11T00:55:45.242 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-09-11T00:55:45.242 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-09-11T00:55:45.281 DEBUG:teuthology.orchestra.run.smithi012:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-11T00:55:45.509 DEBUG:teuthology.orchestra.run.smithi165:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-11T00:55:45.623 DEBUG:teuthology.orchestra.run.smithi178:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-11T00:55:45.723 DEBUG:teuthology.orchestra.run.smithi012:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 2024-09-11T00:55:45.785 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T00:55:45.786 DEBUG:teuthology.orchestra.run.smithi165:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 2024-09-11T00:55:45.818 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T00:55:45.818 DEBUG:teuthology.orchestra.run.smithi178:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 2024-09-11T00:55:45.846 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T00:55:45.846 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-09-11T00:55:45.860 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-09-11T00:55:45.951 INFO:teuthology.task.internal:Duration was 4089.574944 seconds 2024-09-11T00:55:45.952 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-09-11T00:55:46.164 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-09-11T00:55:46.165 DEBUG:teuthology.orchestra.run.smithi012:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-11T00:55:46.170 DEBUG:teuthology.orchestra.run.smithi165:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-11T00:55:46.173 DEBUG:teuthology.orchestra.run.smithi178:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-11T00:55:46.243 INFO:teuthology.orchestra.run.smithi165.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-11T00:55:46.257 INFO:teuthology.orchestra.run.smithi178.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-11T00:55:46.279 INFO:teuthology.orchestra.run.smithi012.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-11T00:55:46.665 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-09-11T00:55:46.666 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi012.front.sepia.ceph.com 2024-09-11T00:55:46.666 DEBUG:teuthology.orchestra.run.smithi012:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-11T00:55:46.716 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi165.front.sepia.ceph.com 2024-09-11T00:55:46.717 DEBUG:teuthology.orchestra.run.smithi165:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-11T00:55:46.754 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi178.front.sepia.ceph.com 2024-09-11T00:55:46.754 DEBUG:teuthology.orchestra.run.smithi178:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-11T00:55:46.782 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-09-11T00:55:46.782 DEBUG:teuthology.orchestra.run.smithi012:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-11T00:55:46.787 DEBUG:teuthology.orchestra.run.smithi165:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-11T00:55:46.797 DEBUG:teuthology.orchestra.run.smithi178:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-11T00:55:46.850 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-09-11T00:55:46.851 DEBUG:teuthology.orchestra.run.smithi012:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-11T00:55:46.855 DEBUG:teuthology.orchestra.run.smithi165:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-11T00:55:46.869 DEBUG:teuthology.orchestra.run.smithi178:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-11T00:55:48.071 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-09-11T00:55:48.082 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-09-11T00:55:48.082 DEBUG:teuthology.orchestra.run.smithi012:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-11T00:55:48.157 DEBUG:teuthology.orchestra.run.smithi165:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-11T00:55:48.453 DEBUG:teuthology.orchestra.run.smithi178:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-11T00:55:48.495 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-09-11T00:55:48.506 DEBUG:teuthology.orchestra.run.smithi012:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-11T00:55:48.511 DEBUG:teuthology.orchestra.run.smithi165:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-11T00:55:48.513 DEBUG:teuthology.orchestra.run.smithi178:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-11T00:55:48.545 INFO:teuthology.orchestra.run.smithi165.stdout:kernel.core_pattern = core 2024-09-11T00:55:48.576 INFO:teuthology.orchestra.run.smithi178.stdout:kernel.core_pattern = core 2024-09-11T00:55:48.595 INFO:teuthology.orchestra.run.smithi012.stdout:kernel.core_pattern = core 2024-09-11T00:55:48.633 DEBUG:teuthology.orchestra.run.smithi012:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-11T00:55:48.666 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T00:55:48.667 DEBUG:teuthology.orchestra.run.smithi165:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-11T00:55:48.684 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T00:55:48.684 DEBUG:teuthology.orchestra.run.smithi178:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-11T00:55:48.701 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T00:55:48.702 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-09-11T00:55:48.782 INFO:teuthology.task.internal:Transferring archived files... 2024-09-11T00:55:48.784 DEBUG:teuthology.misc:Transferring archived files from smithi012:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/hyelloji-2024-09-10_16:59:47-fs-wip-hemanth-testing-2024-09-10-0723-testing-default-smithi/7899811/remote/smithi012 2024-09-11T00:55:48.784 DEBUG:teuthology.orchestra.run.smithi012:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-11T00:55:48.880 DEBUG:teuthology.misc:Transferring archived files from smithi165:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/hyelloji-2024-09-10_16:59:47-fs-wip-hemanth-testing-2024-09-10-0723-testing-default-smithi/7899811/remote/smithi165 2024-09-11T00:55:48.881 DEBUG:teuthology.orchestra.run.smithi165:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-11T00:55:48.922 DEBUG:teuthology.misc:Transferring archived files from smithi178:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/hyelloji-2024-09-10_16:59:47-fs-wip-hemanth-testing-2024-09-10-0723-testing-default-smithi/7899811/remote/smithi178 2024-09-11T00:55:48.923 DEBUG:teuthology.orchestra.run.smithi178:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-11T00:55:48.965 INFO:teuthology.task.internal:Removing archive directory... 2024-09-11T00:55:48.965 DEBUG:teuthology.orchestra.run.smithi012:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-11T00:55:48.969 DEBUG:teuthology.orchestra.run.smithi165:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-11T00:55:48.972 DEBUG:teuthology.orchestra.run.smithi178:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-11T00:55:49.018 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-09-11T00:55:49.051 INFO:teuthology.task.internal:Not uploading archives. 2024-09-11T00:55:49.051 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-09-11T00:55:49.121 INFO:teuthology.task.internal:Tidying up after the test... 2024-09-11T00:55:49.121 DEBUG:teuthology.orchestra.run.smithi012:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-11T00:55:49.125 DEBUG:teuthology.orchestra.run.smithi165:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-11T00:55:49.128 DEBUG:teuthology.orchestra.run.smithi178:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-11T00:55:49.143 INFO:teuthology.orchestra.run.smithi165.stdout: 262165 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 11 00:55 /home/ubuntu/cephtest 2024-09-11T00:55:49.146 INFO:teuthology.orchestra.run.smithi178.stdout: 262165 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 11 00:55 /home/ubuntu/cephtest 2024-09-11T00:55:49.156 INFO:teuthology.orchestra.run.smithi012.stdout: 393537 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 11 00:55 /home/ubuntu/cephtest 2024-09-11T00:55:49.159 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-09-11T00:55:49.170 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-09-11T00:55:49.375 INFO:teuthology.run:Summary data: client.0-kernel-sha1: 2ca44a2f35ca8f36983aaaa46d49312239ef519a description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/legacy wsync/no} objectstore-ec/bluestore-ec-root omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/automatic export-check n/5 replication/default} standby-replay tasks/{0-subvolume/{with-namespace-isolated-and-quota} 1-check-counter 2-scrub/no 3-snaps/no 4-flush/yes 5-quiesce/with-quiesce 6-workunit/suites/dbench}} duration: 4089.5749435424805 flavor: default owner: scheduled_hyelloji@teuthology success: true 2024-09-11T00:55:49.375 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-11T00:55:49.469 INFO:teuthology.run:pass